2023-12-19T17:56:19.871 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-19T17:56:19.871 DEBUG:teuthology.run:Teuthology command: teuthology --name vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} --verbose --archive /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229 --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229/orig.config.yaml 2023-12-19T17:56:19.917 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T17:56:20.006 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229 branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} email: vshankar@redhat.com first_in_suite: false job_id: '7497229' kernel: flavor: default kdb: true sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd - disabled conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 client.0: debug client: 20 debug objectcacher: 20 debug objecter: 20 client.1: debug client: 20 debug objectcacher: 20 debug objecter: 20 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 mon osd backfillfull ratio: 0.6 mon osd full ratio: 0.7 mon osd nearfull ratio: 0.6 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 memstore device bytes: 200000000 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 1.0 osd mon report interval: 5 osd objectstore: memstore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL - PG_RECOVERY_FULL - PG_DEGRADED sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e owner: scheduled_vshankar@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 430 sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e sleep_before_teardown: 0 subset: 301/1024 suite: fs suite_branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_568df92b7e50b2108bce4c77c393ff294d6a5e5e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi027.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKiwWr0Trl+pDhPf5nNOLJ6oFgL3wEGSWpZ+bLW0N63dQX6maGQvLOlhSSCJ/rdZIk1z6+T2Pv8zhBcahDNw244= smithi192.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLG3+8ZD65WGw+gpZxw4zFCbHyN3UXqVOhaEfVSHFoGNnoGBQCbRrunv60cTWg5kiZC1Dx8YHbcYlR5C6vpK1TI= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_full teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73 timestamp: 2023-12-19_13:22:41 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1485078 2023-12-19T17:56:20.007 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_568df92b7e50b2108bce4c77c393ff294d6a5e5e/qa; will attempt to use it 2023-12-19T17:56:20.007 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_568df92b7e50b2108bce4c77c393ff294d6a5e5e/qa/tasks 2023-12-19T17:56:20.008 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-19T17:56:20.009 INFO:teuthology.task.internal:Checking packages... 2023-12-19T17:56:20.033 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '568df92b7e50b2108bce4c77c393ff294d6a5e5e' 2023-12-19T17:56:20.033 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-19T17:56:20.033 INFO:teuthology.packaging:ref: None 2023-12-19T17:56:20.033 INFO:teuthology.packaging:tag: None 2023-12-19T17:56:20.033 INFO:teuthology.packaging:branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 2023-12-19T17:56:20.033 INFO:teuthology.packaging:sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-19T17:56:20.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-vshankar-testing1-2023-12-18-1207-reef-2 2023-12-19T17:56:20.264 INFO:teuthology.task.internal:Found packages for ceph version 18.2.1-132.g568df92b 2023-12-19T17:56:20.265 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-19T17:56:20.272 INFO:teuthology.task.internal:no buildpackages task found 2023-12-19T17:56:20.272 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-19T17:56:20.277 INFO:teuthology.task.internal:Saving configuration 2023-12-19T17:56:20.296 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-19T17:56:20.301 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-19T17:56:20.330 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 17:49:49.206968', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKiwWr0Trl+pDhPf5nNOLJ6oFgL3wEGSWpZ+bLW0N63dQX6maGQvLOlhSSCJ/rdZIk1z6+T2Pv8zhBcahDNw244='} 2023-12-19T17:56:20.359 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 17:49:49.209057', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLG3+8ZD65WGw+gpZxw4zFCbHyN3UXqVOhaEfVSHFoGNnoGBQCbRrunv60cTWg5kiZC1Dx8YHbcYlR5C6vpK1TI='} 2023-12-19T17:56:20.360 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-19T17:56:20.366 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-12-19T17:56:20.366 INFO:teuthology.task.internal:roles: ubuntu@smithi192.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-12-19T17:56:20.366 INFO:teuthology.run_tasks:Running task console_log... 2023-12-19T17:56:20.451 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f6e4f6e7dc0>, signals=[15]) 2023-12-19T17:56:20.452 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-19T17:56:20.458 INFO:teuthology.task.internal:Opening connections... 2023-12-19T17:56:20.459 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T17:56:20.460 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T17:56:20.565 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T17:56:20.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T17:56:20.658 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-19T17:56:20.666 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2023-12-19T17:56:20.684 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2023-12-19T17:56:20.684 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2023-12-19T17:56:20.741 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Stream" 2023-12-19T17:56:20.741 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="9" 2023-12-19T17:56:20.741 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2023-12-19T17:56:20.741 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2023-12-19T17:56:20.741 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="9" 2023-12-19T17:56:20.741 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el9" 2023-12-19T17:56:20.741 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-19T17:56:20.741 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2023-12-19T17:56:20.741 INFO:teuthology.orchestra.run.smithi027.stdout:LOGO="fedora-logo-icon" 2023-12-19T17:56:20.742 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-19T17:56:20.742 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2023-12-19T17:56:20.742 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T17:56:20.742 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-19T17:56:20.742 INFO:teuthology.orchestra.run.smithi027.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-19T17:56:20.742 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2023-12-19T17:56:20.772 DEBUG:teuthology.orchestra.run.smithi192:> uname -m 2023-12-19T17:56:20.789 INFO:teuthology.orchestra.run.smithi192.stdout:x86_64 2023-12-19T17:56:20.790 DEBUG:teuthology.orchestra.run.smithi192:> cat /etc/os-release 2023-12-19T17:56:20.845 INFO:teuthology.orchestra.run.smithi192.stdout:NAME="CentOS Stream" 2023-12-19T17:56:20.846 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION="9" 2023-12-19T17:56:20.846 INFO:teuthology.orchestra.run.smithi192.stdout:ID="centos" 2023-12-19T17:56:20.846 INFO:teuthology.orchestra.run.smithi192.stdout:ID_LIKE="rhel fedora" 2023-12-19T17:56:20.846 INFO:teuthology.orchestra.run.smithi192.stdout:VERSION_ID="9" 2023-12-19T17:56:20.846 INFO:teuthology.orchestra.run.smithi192.stdout:PLATFORM_ID="platform:el9" 2023-12-19T17:56:20.846 INFO:teuthology.orchestra.run.smithi192.stdout:PRETTY_NAME="CentOS Stream 9" 2023-12-19T17:56:20.846 INFO:teuthology.orchestra.run.smithi192.stdout:ANSI_COLOR="0;31" 2023-12-19T17:56:20.846 INFO:teuthology.orchestra.run.smithi192.stdout:LOGO="fedora-logo-icon" 2023-12-19T17:56:20.846 INFO:teuthology.orchestra.run.smithi192.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2023-12-19T17:56:20.846 INFO:teuthology.orchestra.run.smithi192.stdout:HOME_URL="https://centos.org/" 2023-12-19T17:56:20.846 INFO:teuthology.orchestra.run.smithi192.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-19T17:56:20.846 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2023-12-19T17:56:20.846 INFO:teuthology.orchestra.run.smithi192.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-12-19T17:56:20.847 INFO:teuthology.lock.ops:Updating smithi192.front.sepia.ceph.com on lock server 2023-12-19T17:56:20.880 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-19T17:56:20.889 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-19T17:56:20.895 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-19T17:56:20.895 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2023-12-19T17:56:20.898 DEBUG:teuthology.orchestra.run.smithi192:> test '!' -e /home/ubuntu/cephtest 2023-12-19T17:56:20.913 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-19T17:56:20.921 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-19T17:56:20.921 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2023-12-19T17:56:20.955 DEBUG:teuthology.orchestra.run.smithi192:> test -z $(ls -A /var/lib/ceph) 2023-12-19T17:56:21.059 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-19T17:56:21.134 INFO:teuthology.run_tasks:Running task kernel... 2023-12-19T17:56:21.148 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'} 2023-12-19T17:56:21.149 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}}, timeout 300 2023-12-19T17:56:21.149 DEBUG:teuthology.orchestra.run.smithi027:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-19T17:56:21.149 DEBUG:teuthology.orchestra.run.smithi192:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-19T17:56:21.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T17:56:21.165 DEBUG:teuthology.task.kernel:sha1 for mon.b is cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T17:56:21.165 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "cd68252d7b4d7802f3cae7f5945b38e9b20dad4c"... 2023-12-19T17:56:21.165 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2023-12-19T17:56:21.166 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T17:56:21.166 DEBUG:teuthology.task.kernel:sha1 for mon.a is cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T17:56:21.166 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "cd68252d7b4d7802f3cae7f5945b38e9b20dad4c"... 2023-12-19T17:56:21.167 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2023-12-19T17:56:21.221 INFO:teuthology.orchestra.run.smithi192.stdout:5.14.0-361.el9.x86_64 2023-12-19T17:56:21.222 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T17:56:21.222 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-19T17:56:21.222 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T17:56:21.235 INFO:teuthology.orchestra.run.smithi027.stdout:5.14.0-361.el9.x86_64 2023-12-19T17:56:21.235 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-361.el9.x86_64 vs cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T17:56:21.236 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-19T17:56:21.236 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T17:56:21.382 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum upgrade -y linux-firmware 2023-12-19T17:56:21.458 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum upgrade -y linux-firmware 2023-12-19T17:56:23.626 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - AppStream 10 MB/s | 6.4 MB 00:00 2023-12-19T17:56:23.639 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - AppStream 14 MB/s | 6.4 MB 00:00 2023-12-19T17:56:25.026 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - BaseOS 6.4 MB/s | 2.0 MB 00:00 2023-12-19T17:56:25.215 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - BaseOS 3.7 MB/s | 2.0 MB 00:00 2023-12-19T17:56:28.842 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 2.3 MB/s | 7.9 MB 00:03 2023-12-19T17:56:29.439 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 2.0 MB/s | 7.9 MB 00:03 2023-12-19T17:56:30.592 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 27 MB/s | 18 MB 00:00 2023-12-19T17:56:31.273 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 24 MB/s | 18 MB 00:00 2023-12-19T17:56:36.940 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 5.8 MB 00:02 2023-12-19T17:56:37.460 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 1.7 MB/s | 5.8 MB 00:03 2023-12-19T17:56:38.499 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 15 kB 00:00 2023-12-19T17:56:38.852 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 55 kB/s | 15 kB 00:00 2023-12-19T17:56:39.545 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 21 MB/s | 20 MB 00:00 2023-12-19T17:56:40.148 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2023-12-19T17:56:44.217 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 27 kB/s | 1.7 kB 00:00 2023-12-19T17:56:45.012 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2023-12-19T17:56:46.493 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-12-19T17:56:46.505 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-12-19T17:56:46.505 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2023-12-19T17:56:46.505 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-12-19T17:56:46.505 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2023-12-19T17:56:46.505 INFO:teuthology.orchestra.run.smithi192.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-12-19T17:56:46.505 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T17:56:46.505 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2023-12-19T17:56:46.505 INFO:teuthology.orchestra.run.smithi192.stdout:================================================================================ 2023-12-19T17:56:46.505 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 1 Package 2023-12-19T17:56:46.506 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T17:56:46.506 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 328 M 2023-12-19T17:56:46.506 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2023-12-19T17:56:47.352 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2023-12-19T17:56:47.365 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-12-19T17:56:47.365 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2023-12-19T17:56:47.365 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-12-19T17:56:47.366 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2023-12-19T17:56:47.366 INFO:teuthology.orchestra.run.smithi027.stdout: linux-firmware noarch 20231030-141.el9 CentOS-BaseOS 328 M 2023-12-19T17:56:47.366 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T17:56:47.366 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2023-12-19T17:56:47.366 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2023-12-19T17:56:47.366 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 1 Package 2023-12-19T17:56:47.366 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T17:56:47.366 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 328 M 2023-12-19T17:56:47.366 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2023-12-19T17:56:54.780 INFO:teuthology.orchestra.run.smithi027.stdout:linux-firmware-20231030-141.el9.noarch.rpm 44 MB/s | 328 MB 00:07 2023-12-19T17:56:54.781 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2023-12-19T17:56:54.781 INFO:teuthology.orchestra.run.smithi027.stdout:Total 44 MB/s | 328 MB 00:07 2023-12-19T17:56:54.781 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2023-12-19T17:56:54.886 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2023-12-19T17:56:54.886 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2023-12-19T17:56:56.659 INFO:teuthology.orchestra.run.smithi192.stdout:linux-firmware-20231030-141.el9.noarch.rpm 32 MB/s | 328 MB 00:10 2023-12-19T17:56:56.660 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2023-12-19T17:56:56.661 INFO:teuthology.orchestra.run.smithi192.stdout:Total 32 MB/s | 328 MB 00:10 2023-12-19T17:56:56.661 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2023-12-19T17:56:56.753 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2023-12-19T17:56:56.753 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2023-12-19T17:56:57.122 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2023-12-19T17:56:57.122 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2023-12-19T17:56:58.886 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2023-12-19T17:56:58.886 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2023-12-19T17:57:03.013 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2023-12-19T17:57:03.282 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-12-19T17:57:03.363 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-12-19T17:57:04.283 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-12-19T17:57:04.284 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-12-19T17:57:04.569 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2023-12-19T17:57:04.716 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-12-19T17:57:04.716 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T17:57:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2023-12-19T17:57:04.716 INFO:teuthology.orchestra.run.smithi027.stdout: linux-firmware-20231030-141.el9.noarch 2023-12-19T17:57:04.716 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T17:57:04.716 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2023-12-19T17:57:04.927 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : linux-firmware-20231030-141.el9.noarch 1/2 2023-12-19T17:57:04.990 INFO:teuthology.task.kernel:Downloading kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.a... 2023-12-19T17:57:04.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T17:57:05.009 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : linux-firmware-20230814-139.el9.noarch 2/2 2023-12-19T17:57:05.147 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/9/flavors/default/x86_64/ 2023-12-19T17:57:05.148 DEBUG:teuthology.orchestra.run.smithi027:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/9/flavors/default/x86_64/ --input-file=- 2023-12-19T17:57:05.150 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi027.front.sepia.ceph.com to complete... 2023-12-19T17:57:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: linux-firmware-20230814-139.el9.noarch 2/2 2023-12-19T17:57:05.933 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : linux-firmware-20231030-141.el9.noarch 1/2 2023-12-19T17:57:06.310 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : linux-firmware-20230814-139.el9.noarch 2/2 2023-12-19T17:57:06.310 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T17:57:06.310 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2023-12-19T17:57:06.310 INFO:teuthology.orchestra.run.smithi192.stdout: linux-firmware-20231030-141.el9.noarch 2023-12-19T17:57:06.310 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T17:57:06.310 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-12-19T17:57:06.603 INFO:teuthology.task.kernel:Downloading kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.b... 2023-12-19T17:57:06.603 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=cd68252d7b4d7802f3cae7f5945b38e9b20dad4c 2023-12-19T17:57:06.762 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/9/flavors/default/x86_64/ 2023-12-19T17:57:06.762 DEBUG:teuthology.orchestra.run.smithi192:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/9/flavors/default/x86_64/ --input-file=- 2023-12-19T17:57:06.764 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi192.front.sepia.ceph.com to complete... 2023-12-19T17:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19 17:57:18 URL:https://4.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/9/flavors/default/x86_64/kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm [321162729/321162729] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-19T17:57:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:FINISHED --2023-12-19 17:57:18-- 2023-12-19T17:57:18.187 INFO:teuthology.orchestra.run.smithi027.stderr:Total wall clock time: 13s 2023-12-19T17:57:18.187 INFO:teuthology.orchestra.run.smithi027.stderr:Downloaded: 1 files, 306M in 13s (24.1 MB/s) 2023-12-19T17:57:18.189 INFO:teuthology.task.kernel:Installing kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.a... 2023-12-19T17:57:18.189 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-19T17:57:20.136 INFO:teuthology.orchestra.run.smithi027.stdout:Verifying... ######################################## 2023-12-19T17:57:20.192 INFO:teuthology.orchestra.run.smithi027.stdout:Preparing... ######################################## 2023-12-19T17:57:20.277 INFO:teuthology.orchestra.run.smithi027.stdout:Updating / installing... 2023-12-19T17:57:27.123 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19 17:57:27 URL:https://4.chacra.ceph.com/r/kernel/testing/cd68252d7b4d7802f3cae7f5945b38e9b20dad4c/centos/9/flavors/default/x86_64/kernel-6.6.0_gcd68252d7b4d-1.x86_64.rpm [321162729/321162729] -> "/tmp/kernel.x86_64.rpm" [1] 2023-12-19T17:57:27.124 INFO:teuthology.orchestra.run.smithi192.stderr:FINISHED --2023-12-19 17:57:27-- 2023-12-19T17:57:27.124 INFO:teuthology.orchestra.run.smithi192.stderr:Total wall clock time: 20s 2023-12-19T17:57:27.124 INFO:teuthology.orchestra.run.smithi192.stderr:Downloaded: 1 files, 306M in 20s (15.4 MB/s) 2023-12-19T17:57:27.127 INFO:teuthology.task.kernel:Installing kernel cd68252d7b4d7802f3cae7f5945b38e9b20dad4c on mon.b... 2023-12-19T17:57:27.127 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-12-19T17:57:27.631 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-6.6.0_gcd68252d7b4d-1 ######################################## 2023-12-19T17:57:28.980 INFO:teuthology.orchestra.run.smithi192.stdout:Verifying... ######################################## 2023-12-19T17:57:29.099 INFO:teuthology.orchestra.run.smithi192.stdout:Preparing... ######################################## 2023-12-19T17:57:29.179 INFO:teuthology.orchestra.run.smithi192.stdout:Updating / installing... 2023-12-19T17:57:36.276 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-6.6.0_gcd68252d7b4d-1 ######################################## 2023-12-19T17:57:42.813 INFO:teuthology.orchestra.run.smithi027.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-19T17:57:51.705 INFO:teuthology.orchestra.run.smithi192.stderr:dracut: Disabling early microcode, because kernel does not support it. CONFIG_MICROCODE_[AMD|INTEL]!=y 2023-12-19T17:58:04.665 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-19T17:58:04.665 DEBUG:teuthology.orchestra.run.smithi027:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-19T17:58:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/System.map-6.6.0-gcd68252d7b4d 2023-12-19T17:58:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/config-6.6.0-gcd68252d7b4d 2023-12-19T17:58:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-19T17:58:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d 2023-12-19T17:58:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel 2023-12-19T17:58:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch 2023-12-19T17:58:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86 2023-12-19T17:58:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto 2023-12-19T17:58:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-19T17:58:04.730 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-19T17:58:04.731 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm.ko 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto 2023-12-19T17:58:04.732 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ansi_cprng.ko 2023-12-19T17:58:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/anubis.ko 2023-12-19T17:58:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/arc4.ko 2023-12-19T17:58:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx 2023-12-19T17:58:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_memcpy.ko 2023-12-19T17:58:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_pq.ko 2023-12-19T17:58:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-19T17:58:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_tx.ko 2023-12-19T17:58:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_xor.ko 2023-12-19T17:58:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/raid6test.ko 2023-12-19T17:58:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authenc.ko 2023-12-19T17:58:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authencesn.ko 2023-12-19T17:58:04.733 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blake2b_generic.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_common.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_generic.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/camellia_generic.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast5_generic.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast6_generic.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast_common.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ccm.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cmac.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crc32_generic.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_engine.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_user.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cts.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/des_generic.ko 2023-12-19T17:58:04.734 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/dh_generic.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/echainiv.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/essiv.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/fcrypt.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/khazad.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4hc.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/md4.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/michael_mic.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcbc.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcrypt.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/rmd160.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/seed.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/serpent_generic.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tcrypt.ko 2023-12-19T17:58:04.735 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tea.ko 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_common.ko 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_generic.ko 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/vmac.ko 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/wp512.ko 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xcbc.ko 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xor.ko 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xxhash_generic.ko 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_pad.ko 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/custom_method.ko 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ec_sys.ko 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/platform_profile.ko 2023-12-19T17:58:04.736 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbs.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbshc.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/video.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/acard-ahci.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci_platform.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ata_generic.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci_platform.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_acpi.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ali.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_amd.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_artop.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atiixp.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atp867x.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd640.ko 2023-12-19T17:58:04.737 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd64x.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cypress.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_efar.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt366.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt37x.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it8213.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it821x.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_jmicron.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_marvell.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_mpiix.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_netcell.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ninja32.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87410.ko 2023-12-19T17:58:04.738 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87415.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_oldpiix.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_opti.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_optidma.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pcmcia.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_piccolo.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rdc.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sch.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_serverworks.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sil680.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sis.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sl82c105.ko 2023-12-19T17:58:04.739 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_triflex.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_via.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pdc_adma.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_inic162x.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_mv.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_nv.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_promise.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_qstor.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil24.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sis.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_svw.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sx4.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_uli.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_via.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_vsc.ko 2023-12-19T17:58:04.740 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ks0108.ko 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/bcma.ko 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/aoe.ko 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/brd.ko 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/drbd.ko 2023-12-19T17:58:04.741 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/floppy.ko 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/loop.ko 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/nbd.ko 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/null_blk.ko 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/pktcdvd.ko 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/rbd.ko 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/virtio_blk.ko 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkfront.ko 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/zram.ko 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char 2023-12-19T17:58:04.742 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hangcheck-timer.ko 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/via-rng.ko 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-19T17:58:04.743 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/lp.ko 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/mwave.ko 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ppdev.ko 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tlclk.ko 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm.ko 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/uv_mmtimer.ko 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/virtio_console.ko 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq 2023-12-19T17:58:04.744 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp.ko 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-19T17:58:04.745 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-aes.ko 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-sha.ko 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/dca.ko 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac 2023-12-19T17:58:04.746 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/amd64_edac.ko 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/e752x_edac.ko 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/edac_mce_amd.ko 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3000_edac.ko 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3200_edac.ko 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5100_edac.ko 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5400_edac.ko 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7300_edac.ko 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7core_edac.ko 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i82975x_edac.ko 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ie31200_edac.ko 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/sb_edac.ko 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/x38_edac.ko 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware 2023-12-19T17:58:04.747 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/edd.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ich.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-appleir.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-aureal.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-axff.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-dr.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elecom.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elo.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-emsff.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gaff.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gt683r.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gyration.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-19T17:58:04.748 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtekff.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-hyperv.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-icade.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-keytouch.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kye.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lcpower.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-led.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lenovo.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-multitouch.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ortek.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-penmount.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-petalynx.ko 2023-12-19T17:58:04.749 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-picolcd.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-pl.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-plantronics.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-primax.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-rmi.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-common.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-19T17:58:04.750 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-saitek.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-samsung.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sjoy.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sony.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-speedlink.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-steelseries.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sunplus.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tivo.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tmff.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-topseed.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-twinhan.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-uclogic.ko 2023-12-19T17:58:04.751 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-waltop.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-wiimote.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-xinmo.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zpff.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zydacron.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/uhid.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/wacom.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_balloon.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_utils.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_vmbus.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru3.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7414.ko 2023-12-19T17:58:04.752 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7418.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adc128d818.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1025.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1026.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1029.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1031.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm9240.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ads7828.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7410.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7411.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7462.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7470.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7475.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7x10.ko 2023-12-19T17:58:04.753 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/amc6821.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/applesmc.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asb100.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asc7621.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/atxp1.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/coretemp.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dme1737.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds1621.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds620.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc1403.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc6w201.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71805f.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71882fg.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f75375s.ko 2023-12-19T17:58:04.754 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fam15h_power.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fschmd.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g760a.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g762.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl518sm.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl520sm.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5500_temp.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5k_amb.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmaem.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmpex.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina209.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina2xx.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/it87.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k10temp.ko 2023-12-19T17:58:04.755 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k8temp.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lineage-pem.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm63.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm73.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm75.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm77.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm78.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm80.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm83.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm85.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm87.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm90.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm92.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm93.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95234.ko 2023-12-19T17:58:04.756 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95241.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95245.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc2945.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4151.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4215.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4222.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4245.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4260.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4261.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max16065.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1619.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1668.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max197.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6639.ko 2023-12-19T17:58:04.757 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6650.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6697.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mcp3021.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6683.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775-core.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct7802.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87360.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87427.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pcf8591.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-19T17:58:04.758 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/powr1220.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5627.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5636.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht15.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht21.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/shtc1.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sis5595.ko 2023-12-19T17:58:04.759 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47b397.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m1.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m192.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/thmc50.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp102.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp103.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp401.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp421.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via-cputemp.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via686a.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt1211.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt8231.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627ehf.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627hf.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83781d.ko 2023-12-19T17:58:04.760 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83791d.ko 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83792d.ko 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83793.ko 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83795.ko 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l785ts.ko 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l786ng.ko 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-19T17:58:04.761 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-19T17:58:04.762 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-dev.ko 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-mux.ko 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-smbus.ko 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-stub.ko 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_core.ko 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-19T17:58:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib 2023-12-19T17:58:04.764 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/gameport.ko 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/joydev.ko 2023-12-19T17:58:04.765 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/appletouch.ko 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/bcm5974.ko 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/cyapatp.ko 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/sermouse.ko 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/altera_ps2.ko 2023-12-19T17:58:04.766 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/arc_ps2.ko 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serio_raw.ko 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/sparse-keymap.ko 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-blinkm.ko 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3530.ko 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp3944.ko 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-ss4200.ko 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-19T17:58:04.767 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/bcache.ko 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bio-prison.ko 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache-smq.ko 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache.ko 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-crypt.ko 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-delay.ko 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-flakey.ko 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log-userspace.ko 2023-12-19T17:58:04.768 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-multipath.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-queue-length.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-raid.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-round-robin.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-service-time.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-switch.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-thin-pool.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-verity.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/faulty.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/linear.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/multipath.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid0.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid1.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid10.ko 2023-12-19T17:58:04.769 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid456.ko 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptbase.ko 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptctl.ko 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptfc.ko 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptlan.ko 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptsas.ko 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptscsih.ko 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptspi.ko 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_ich.ko 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_sch.ko 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mfd-core.ko 2023-12-19T17:58:04.770 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/sm501.ko 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/viperboard.ko 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/vx855.ko 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wl1273-core.ko 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds9802als.ko 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds990x.ko 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/bh1770glc.ko 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/cb710.ko 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/at24.ko 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-19T17:58:04.771 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-19T17:58:04.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/max6875.ko 2023-12-19T17:58:04.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/enclosure.ko 2023-12-19T17:58:04.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hpilo.ko 2023-12-19T17:58:04.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29003.ko 2023-12-19T17:58:04.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29020.ko 2023-12-19T17:58:04.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d 2023-12-19T17:58:04.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-19T17:58:04.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-19T17:58:04.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei 2023-12-19T17:58:04.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-me.ko 2023-12-19T17:58:04.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-txe.ko 2023-12-19T17:58:04.772 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei.ko 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_7xx1.ko 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_core.ko 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tsl2550.ko 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_balloon.ko 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net 2023-12-19T17:58:04.773 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/bonding.ko 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dummy.ko 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/eql.ko 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-19T17:58:04.774 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-19T17:58:04.775 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-19T17:58:04.776 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-19T17:58:04.777 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-19T17:58:04.778 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dnet.ko 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ethoc.ko 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fealnx.ko 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-19T17:58:04.779 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-19T17:58:04.780 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/jme.ko 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-19T17:58:04.781 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-19T17:58:04.782 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-19T17:58:04.783 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun 2023-12-19T17:58:04.784 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-19T17:58:04.785 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/geneve.ko 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ifb.ko 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvlan.ko 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvtap.ko 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio.ko 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mii.ko 2023-12-19T17:58:04.786 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/net_failover.ko 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/netconsole.ko 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/nlmon.ko 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/amd.ko 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm87xx.ko 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/broadcom.ko 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/cicada.ko 2023-12-19T17:58:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/davicom.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/dp83640.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/et1011c.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/icplus.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/lxt.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/marvell.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/micrel.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/national.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/phylink.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/qsemi.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/realtek.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/smsc.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ste10Xp.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/vitesse.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-19T17:58:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_async.ko 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppoe.ko 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppox.ko 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pptp.ko 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slhc.ko 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slip.ko 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sungem_phy.ko 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tap.ko 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team.ko 2023-12-19T17:58:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_random.ko 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tun.ko 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/veth.ko 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/virtio_net.ko 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/vxlan.ko 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netfront.ko 2023-12-19T17:58:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ntb.ko 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme-core.ko 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme.ko 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport.ko 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_cs.ko 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_pc.ko 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_serial.ko 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie 2023-12-19T17:58:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/xen-pcifront.ko 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/i82092.ko 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pd6729.ko 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acerhdf.ko 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-19T17:58:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/hdaps.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel_ips.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-19T17:58:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi.ko 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps_parport.ko 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/pps_core.ko 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp.ko 2023-12-19T17:58:04.794 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp_kvm.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-em3027.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-19T17:58:04.795 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max6900.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-19T17:58:04.796 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-x1205.ko 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-9xxx.ko 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-sas.ko 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-xxxx.ko 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/BusLogic.ko 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/a100u2w.ko 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/advansys.ko 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/am53c974.ko 2023-12-19T17:58:04.797 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/atp870u.ko 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/bfa.ko 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ch.ko 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi 2023-12-19T17:58:04.798 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dc395x.ko 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dmx3191d.ko 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esp_scsi.ko 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe 2023-12-19T17:58:04.799 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/fnic.ko 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hpsa.ko 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hptiop.ko 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hv_storvsc.ko 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/initio.ko 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ipr.ko 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ips.ko 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/isci.ko 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc 2023-12-19T17:58:04.800 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/libfc.ko 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi.ko 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/libsas.ko 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid.ko 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-19T17:58:04.801 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvumi.ko 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pmcraid.ko 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla1280.ko 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/raid_class.ko 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_debug.ko 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-19T17:58:04.802 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ses.ko 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/st.ko 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/stex.ko 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/virtio_scsi.ko 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/wd719x.ko 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ssb.ko 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-19T17:58:04.803 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_file.ko 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_iblock.ko 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_mod.ko 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_pscsi.ko 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_user.ko 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-19T17:58:04.804 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_gsm.ko 2023-12-19T17:58:04.805 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_hdlc.ko 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/nozomi.ko 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/arc_uart.ko 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio.ko 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_aec.ko 2023-12-19T17:58:04.806 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_cif.ko 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pci_generic.ko 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_sercos3.ko 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-acm.ko 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usblp.ko 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usbtmc.ko 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/mdc800.ko 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/microtek.ko 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/adutux.ko 2023-12-19T17:58:04.807 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/appledisplay.ko 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi26.ko 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi62.ko 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ezusb.ko 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/idmouse.ko 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/iowarrior.ko 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ldusb.ko 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/legousbtower.ko 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usb3503.ko 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usblcd.ko 2023-12-19T17:58:04.808 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/uss720.ko 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/yurex.ko 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-generic.ko 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/aircable.ko 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ark3116.ko 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ch341.ko 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cp210x.ko 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cyberjack.ko 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/empeg.ko 2023-12-19T17:58:04.809 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_ti.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipaq.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipw.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ir-usb.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mct_u232.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7720.ko 2023-12-19T17:58:04.810 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7840.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/navman.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/omninet.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/opticon.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/option.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/oti6858.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/pl2303.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcaux.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcserial.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/quatech2.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/safe_serial.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/sierra.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ssu100.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/symbolserial.ko 2023-12-19T17:58:04.811 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_debug.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/visor.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/whiteheat.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/uas.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-19T17:58:04.812 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-karma.ko 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/usb-storage.ko 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-19T17:58:04.813 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio.ko 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost.ko 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_net.ko 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_scsi.ko 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/apple_bl.ko 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lcd.ko 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio.ko 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_balloon.ko 2023-12-19T17:58:04.814 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_mmio.ko 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci.ko 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_ring.ko 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2482.ko 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2490.ko 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-19T17:58:04.815 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/wire.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/hpwdt.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/i6300esb.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ib700wdt.ko 2023-12-19T17:58:04.816 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ibmasr.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it87_wdt.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/machzwd.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/nv_tco.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/softdog.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/via_wdt.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-19T17:58:04.817 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wdt_pci.ko 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/xen_wdt.ko 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-evtchn.ko 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntalloc.ko 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntdev.ko 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-privcmd.ko 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p 2023-12-19T17:58:04.818 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/9p.ko 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/affs.ko 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/befs.ko 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/binfmt_misc.ko 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/btrfs.ko 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/cachefiles.ko 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ceph.ko 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/coda.ko 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs 2023-12-19T17:58:04.819 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/cramfs.ko 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/dlm.ko 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/f2fs.ko 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/fat.ko 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/msdos.ko 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/vfat.ko 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/fscache.ko 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/cuse.ko 2023-12-19T17:58:04.820 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/fuse.ko 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/gfs2.ko 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/hfs.ko 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/hfsplus.ko 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/isofs.ko 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/jfs.ko 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/lockd.ko 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix 2023-12-19T17:58:04.821 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/minix.ko 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/netfs.ko 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfs.ko 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv3.ko 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv4.ko 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/grace.ko 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/nfs_acl.ko 2023-12-19T17:58:04.822 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/nfsd.ko 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/nilfs2.ko 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-celtic.ko 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-centeuro.ko 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-croatian.ko 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-cyrillic.ko 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-gaelic.ko 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-greek.ko 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-iceland.ko 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-inuit.ko 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-roman.ko 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-romanian.ko 2023-12-19T17:58:04.823 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-turkish.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1250.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1251.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1255.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp737.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp775.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp850.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp852.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp855.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp857.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp860.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp861.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp862.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp863.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp864.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp865.ko 2023-12-19T17:58:04.824 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp866.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp869.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp874.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp932.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp936.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp949.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp950.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_euc-jp.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-1.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-13.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-14.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-15.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-2.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-3.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-4.ko 2023-12-19T17:58:04.825 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-5.ko 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-6.ko 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-7.ko 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-9.ko 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-r.ko 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-ru.ko 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-u.ko 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_utf8.ko 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs 2023-12-19T17:58:04.826 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2.ko 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/overlay.ko 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ramoops.ko 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/reiserfs.ko 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/romfs.ko 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client 2023-12-19T17:58:04.827 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/cifs.ko 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_arc4.ko 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_md4.ko 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/squashfs.ko 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/sysv.ko 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/udf.ko 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ufs.ko 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/xfs.ko 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel 2023-12-19T17:58:04.828 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu/rcutorture.ko 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/torture.ko 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/asn1_encoder.ko 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc-itu-t.ko 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc8.ko 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libarc4.ko 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libdes.ko 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/libcrc32c.ko 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lru_cache.ko 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4 2023-12-19T17:58:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4_compress.ko 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4hc_compress.ko 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/cordic.ko 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/raid6_pq.ko 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_bm.ko 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_fsm.ko 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_kmp.ko 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd/zstd_compress.ko 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/hwpoison-inject.ko 2023-12-19T17:58:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/6lowpan.ko 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_dest.ko 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_fragment.ko 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_hop.ko 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_mobility.ko 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_routing.ko 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_udp.ko 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/garp.ko 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/mrp.ko 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/p8022.ko 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/psnap.ko 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/stp.ko 2023-12-19T17:58:04.831 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/8021q.ko 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet.ko 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_fd.ko 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_rdma.ko 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_virtio.ko 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/batman-adv.ko 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/br_netfilter.ko 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/bridge.ko 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-19T17:58:04.832 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-19T17:58:04.833 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtables.ko 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/libceph.ko 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/failover.ko 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/pktgen.ko 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp.ko 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_diag.ko 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv4.ko 2023-12-19T17:58:04.834 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv6.ko 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver/dns_resolver.ko 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/dsa_core.ko 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_brcm.ko 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_dsa.ko 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_none.ko 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_trailer.ko 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154.ko 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4 2023-12-19T17:58:04.835 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ah4.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/esp4.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/fou.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/gre.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/inet_diag.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_gre.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_tunnel.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_vti.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipcomp.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipip.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-19T17:58:04.836 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-19T17:58:04.837 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_bic.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_dctcp.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_diag.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_highspeed.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_htcp.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_hybla.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_illinois.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_lp.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_scalable.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_vegas.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_veno.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_westwood.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_yeah.ko 2023-12-19T17:58:04.838 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tunnel4.ko 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_diag.ko 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_tunnel.ko 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ah6.ko 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/esp6.ko 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/fou6.ko 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_tunnel.ko 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_vti.ko 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ipcomp6.ko 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-19T17:58:04.839 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-19T17:58:04.840 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/sit.ko 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/tunnel6.ko 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/af_key.ko 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_core.ko 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_eth.ko 2023-12-19T17:58:04.841 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip.ko 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip6.ko 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_netlink.ko 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ppp.ko 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc.ko 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/mac802154.ko 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/mpls_gso.ko 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set.ko 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-19T17:58:04.842 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs 2023-12-19T17:58:04.843 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conncount.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack.ko 2023-12-19T17:58:04.844 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-19T17:58:04.845 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_log_syslog.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_irc.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_sip.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_tables.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_log.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_chain_nat.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_compat.ko 2023-12-19T17:58:04.846 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_ct.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fib.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_hash.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_limit.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_log.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_masq.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_nat.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_numgen.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_queue.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_quota.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_redir.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject_inet.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_socket.ko 2023-12-19T17:58:04.847 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_tunnel.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_AUDIT.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CT.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_DSCP.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HL.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HMARK.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LED.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LOG.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NETMAP.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFLOG.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-19T17:58:04.848 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_RATEEST.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_SECMARK.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TEE.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TPROXY.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TRACE.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_addrtype.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_bpf.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cgroup.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cluster.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_comment.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connbytes.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlabel.ko 2023-12-19T17:58:04.849 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlimit.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connmark.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_conntrack.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cpu.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dccp.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_devgroup.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dscp.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ecn.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_esp.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hashlimit.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_helper.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hl.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipcomp.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_iprange.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipvs.ko 2023-12-19T17:58:04.850 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_l2tp.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_length.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_limit.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mac.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mark.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_multiport.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nat.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nfacct.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_osf.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_owner.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_physdev.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_pkttype.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_policy.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_quota.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_rateest.ko 2023-12-19T17:58:04.851 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_realm.ko 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_recent.ko 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_sctp.ko 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_set.ko 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_socket.ko 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_state.ko 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_statistic.ko 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_string.ko 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpmss.ko 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_time.ko 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_u32.ko 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/netlink_diag.ko 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/nsh.ko 2023-12-19T17:58:04.852 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/openvswitch.ko 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-geneve.ko 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-gre.ko 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-vxlan.ko 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/af_packet_diag.ko 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds.ko 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_rdma.ko 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_tcp.ko 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_csum.ko 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_gact.ko 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_ipt.ko 2023-12-19T17:58:04.853 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_mirred.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_nat.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_pedit.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_police.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_simple.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_skbedit.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_vlan.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_basic.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_bpf.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_flow.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_fw.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_route.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_u32.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_cmp.ko 2023-12-19T17:58:04.854 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_ipset.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_meta.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_nbyte.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_text.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_u32.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_choke.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_codel.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_drr.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_gred.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hfsc.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hhf.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_htb.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_ingress.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio_lib.ko 2023-12-19T17:58:04.855 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_multiq.ko 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_netem.ko 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_pie.ko 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_plug.ko 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_prio.ko 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_qfq.ko 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_red.ko 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfb.ko 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfq.ko 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_tbf.ko 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_teql.ko 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp.ko 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp_diag.ko 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss 2023-12-19T17:58:04.856 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/sunrpc.ko 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/diag.ko 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/tipc.ko 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/unix_diag.ko 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock.ko 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-19T17:58:04.857 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys/trusted.ko 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/irqbypass.ko 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias.bin 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin 2023-12-19T17:58:04.858 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.alias.bin 2023-12-19T17:58:04.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.bin 2023-12-19T17:58:04.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.modinfo 2023-12-19T17:58:04.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep 2023-12-19T17:58:04.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep.bin 2023-12-19T17:58:04.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.devname 2023-12-19T17:58:04.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.order 2023-12-19T17:58:04.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.softdep 2023-12-19T17:58:04.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols 2023-12-19T17:58:04.859 INFO:teuthology.orchestra.run.smithi027.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols.bin 2023-12-19T17:58:04.860 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-gcd68252d7b4d 2023-12-19T17:58:04.860 DEBUG:teuthology.orchestra.run.smithi027:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-19T17:58:04.883 INFO:teuthology.orchestra.run.smithi027.stdout:postinstall scriptlet (using /bin/sh): 2023-12-19T17:58:04.883 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-gcd68252d7b4d -a -r /boot/System.map-6.6.0-gcd68252d7b4d ]; then 2023-12-19T17:58:04.883 INFO:teuthology.orchestra.run.smithi027.stdout:cp /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm 2023-12-19T17:58:04.883 INFO:teuthology.orchestra.run.smithi027.stdout:cp /boot/System.map-6.6.0-gcd68252d7b4d /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-19T17:58:04.883 INFO:teuthology.orchestra.run.smithi027.stdout:rm -f /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/System.map-6.6.0-gcd68252d7b4d 2023-12-19T17:58:04.883 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/installkernel 6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-19T17:58:04.883 INFO:teuthology.orchestra.run.smithi027.stdout:rm -f /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-19T17:58:04.884 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2023-12-19T17:58:04.884 INFO:teuthology.orchestra.run.smithi027.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-19T17:58:04.884 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-19T17:58:04.884 INFO:teuthology.orchestra.run.smithi027.stdout:new-kernel-pkg --remove 6.6.0-gcd68252d7b4d --rminitrd --initrdfile=/boot/initramfs-6.6.0-gcd68252d7b4d.img 2023-12-19T17:58:04.884 INFO:teuthology.orchestra.run.smithi027.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-19T17:58:04.884 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-install remove 6.6.0-gcd68252d7b4d 2023-12-19T17:58:04.884 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2023-12-19T17:58:04.884 INFO:teuthology.orchestra.run.smithi027.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-19T17:58:04.884 INFO:teuthology.orchestra.run.smithi027.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-19T17:58:04.884 INFO:teuthology.orchestra.run.smithi027.stdout:/sbin/update-bootloader --remove 6.6.0-gcd68252d7b4d 2023-12-19T17:58:04.884 INFO:teuthology.orchestra.run.smithi027.stdout:fi 2023-12-19T17:58:04.885 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2023-12-19T17:58:04.965 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2023-12-19T17:58:04.965 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2023-12-19T17:58:04.965 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.06 2023-12-19T17:58:04.965 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 68.el9 2023-12-19T17:58:04.965 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2023-12-19T17:58:04.965 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-19T17:58:04.965 INFO:teuthology.orchestra.run.smithi027.stdout:Group : Unspecified 2023-12-19T17:58:04.965 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 8264254 2023-12-19T17:58:04.965 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2023-12-19T17:58:04.965 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-19T17:58:04.965 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : builder@centos.org 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T17:58:04.966 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2023-12-19T17:58:04.967 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-19T17:58:04.967 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 6.6.0-gcd68252d7b4d 2023-12-19T17:58:04.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-19T17:58:05.596 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2023-12-19T17:58:07.043 INFO:teuthology.orchestra.run.smithi027.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-19T17:58:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:done 2023-12-19T17:58:07.077 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2023-12-19T17:58:07.092 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.998ZFgrGLt 2023-12-19T17:58:07.092 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.998ZFgrGLt 2023-12-19T17:58:07.178 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.998ZFgrGLt 2023-12-19T17:58:07.289 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.998ZFgrGLt is 6KB 2023-12-19T17:58:07.309 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.998ZFgrGLt 2023-12-19T17:58:07.323 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2023-12-19T17:58:07.399 INFO:teuthology.orchestra.run.smithi027.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-19T17:58:07.399 INFO:teuthology.orchestra.run.smithi027.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-19T17:58:07.399 INFO:teuthology.orchestra.run.smithi027.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-19T17:58:07.399 INFO:teuthology.orchestra.run.smithi027.stdout:15949e3c88704300882482eddfd7baae-6.6.0-gcd68252d7b4d.conf 2023-12-19T17:58:07.400 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-gcd68252d7b4d 2023-12-19T17:58:07.554 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2023-12-19T17:58:12.777 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi192.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-12-19T17:58:12.777 DEBUG:teuthology.orchestra.run.smithi192:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-12-19T17:58:12.848 INFO:teuthology.orchestra.run.smithi192.stdout:/boot/System.map-6.6.0-gcd68252d7b4d 2023-12-19T17:58:12.848 INFO:teuthology.orchestra.run.smithi192.stdout:/boot/config-6.6.0-gcd68252d7b4d 2023-12-19T17:58:12.848 INFO:teuthology.orchestra.run.smithi192.stdout:/boot/vmlinuz-6.6.0-gcd68252d7b4d 2023-12-19T17:58:12.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d 2023-12-19T17:58:12.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel 2023-12-19T17:58:12.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch 2023-12-19T17:58:12.848 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crc32c-intel.ko 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-19T17:58:12.849 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/arch/x86/kvm/kvm.ko 2023-12-19T17:58:12.850 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ansi_cprng.ko 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/anubis.ko 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/arc4.ko 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_memcpy.ko 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_pq.ko 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_tx.ko 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/async_xor.ko 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/async_tx/raid6test.ko 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authenc.ko 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/authencesn.ko 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blake2b_generic.ko 2023-12-19T17:58:12.851 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_common.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/blowfish_generic.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/camellia_generic.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast5_generic.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast6_generic.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cast_common.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/ccm.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cmac.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crc32_generic.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_engine.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/crypto_user.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/cts.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/des_generic.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/dh_generic.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/echainiv.ko 2023-12-19T17:58:12.852 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/essiv.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/fcrypt.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/khazad.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/lz4hc.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/md4.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/michael_mic.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcbc.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/pcrypt.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/rmd160.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/seed.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/serpent_generic.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tcrypt.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/tea.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_common.ko 2023-12-19T17:58:12.853 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/twofish_generic.ko 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/vmac.ko 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/wp512.ko 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xcbc.ko 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xor.ko 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/crypto/xxhash_generic.ko 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/acpi_pad.ko 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/custom_method.ko 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/ec_sys.ko 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/platform_profile.ko 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbs.ko 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/sbshc.ko 2023-12-19T17:58:12.854 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/acpi/video.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/acard-ahci.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ahci_platform.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/ata_generic.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/libahci_platform.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_acpi.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ali.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_amd.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_artop.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atiixp.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_atp867x.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd640.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cmd64x.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_cypress.ko 2023-12-19T17:58:12.855 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_efar.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt366.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt37x.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it8213.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_it821x.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_jmicron.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_marvell.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_mpiix.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_netcell.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ninja32.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87410.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_ns87415.ko 2023-12-19T17:58:12.856 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_oldpiix.ko 2023-12-19T17:58:12.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_opti.ko 2023-12-19T17:58:12.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_optidma.ko 2023-12-19T17:58:12.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pcmcia.ko 2023-12-19T17:58:12.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-19T17:58:12.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-19T17:58:12.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_piccolo.ko 2023-12-19T17:58:12.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_rdc.ko 2023-12-19T17:58:12.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sch.ko 2023-12-19T17:58:12.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_serverworks.ko 2023-12-19T17:58:12.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sil680.ko 2023-12-19T17:58:12.857 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sis.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_sl82c105.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_triflex.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pata_via.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/pdc_adma.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_inic162x.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_mv.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_nv.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_promise.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_qstor.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sil24.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sis.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_svw.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_sx4.ko 2023-12-19T17:58:12.858 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_uli.ko 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_via.ko 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ata/sata_vsc.ko 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/auxdisplay/ks0108.ko 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-i2c.ko 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/base/regmap/regmap-mmio.ko 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/bcma/bcma.ko 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/aoe/aoe.ko 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/brd.ko 2023-12-19T17:58:12.859 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/drbd/drbd.ko 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/floppy.ko 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/loop.ko 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/nbd.ko 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/null_blk/null_blk.ko 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/pktcdvd.ko 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/rbd.ko 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/virtio_blk.ko 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/xen-blkfront.ko 2023-12-19T17:58:12.860 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/block/zram/zram.ko 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hangcheck-timer.ko 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/via-rng.ko 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-19T17:58:12.861 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/lp.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/mwave/mwave.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/ppdev.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tlclk.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/uv_mmtimer.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/char/virtio_console.ko 2023-12-19T17:58:12.862 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/powernow-k8.ko 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/ccp/ccp.ko 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-19T17:58:12.863 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-aes.ko 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/padlock-sha.ko 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dca/dca.ko 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac 2023-12-19T17:58:12.864 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/amd64_edac.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/e752x_edac.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/edac_mce_amd.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3000_edac.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i3200_edac.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5100_edac.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i5400_edac.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7300_edac.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i7core_edac.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/i82975x_edac.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/ie31200_edac.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/sb_edac.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/edac/x38_edac.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/edd.ko 2023-12-19T17:58:12.865 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-ich.ko 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-appleir.ko 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-aureal.ko 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-axff.ko 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-dr.ko 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elecom.ko 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-elo.ko 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-emsff.ko 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gaff.ko 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gt683r.ko 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-gyration.ko 2023-12-19T17:58:12.866 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-holtekff.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-hyperv.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-icade.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-keytouch.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-kye.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lcpower.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-led.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-lenovo.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-multitouch.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-ortek.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-penmount.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-petalynx.ko 2023-12-19T17:58:12.867 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-picolcd.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-pl.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-plantronics.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-primax.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-rmi.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-common.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-19T17:58:12.868 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-roccat.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-saitek.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-samsung.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sjoy.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sony.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-speedlink.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-steelseries.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-sunplus.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tivo.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-tmff.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-topseed.ko 2023-12-19T17:58:12.869 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-twinhan.ko 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-uclogic.ko 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-waltop.ko 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-wiimote.ko 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-xinmo.ko 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zpff.ko 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/hid-zydacron.ko 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/uhid.ko 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hid/wacom.ko 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_balloon.ko 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_utils.ko 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hv/hv_vmbus.ko 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru.ko 2023-12-19T17:58:12.870 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/abituguru3.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7414.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ad7418.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adc128d818.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1025.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1026.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1029.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm1031.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adm9240.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ads7828.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7410.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7411.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7462.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7470.ko 2023-12-19T17:58:12.871 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7475.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/adt7x10.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/amc6821.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/applesmc.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asb100.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asc7621.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/atxp1.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/coretemp.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/dme1737.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds1621.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ds620.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc1403.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/emc6w201.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71805f.ko 2023-12-19T17:58:12.872 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f71882fg.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/f75375s.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fam15h_power.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/fschmd.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g760a.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/g762.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl518sm.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/gl520sm.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5500_temp.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/i5k_amb.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmaem.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ibmpex.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina209.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ina2xx.ko 2023-12-19T17:58:12.873 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/it87.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k10temp.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/k8temp.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lineage-pem.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm63.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm73.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm75.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm77.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm78.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm80.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm83.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm85.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm87.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm90.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm92.ko 2023-12-19T17:58:12.874 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm93.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95234.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95241.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/lm95245.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc2945.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4151.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4215.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4222.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4245.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4260.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/ltc4261.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max16065.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1619.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max1668.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max197.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6639.ko 2023-12-19T17:58:12.875 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6650.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/max6697.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/mcp3021.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6683.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775-core.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct6775.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/nct7802.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87360.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pc87427.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pcf8591.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-19T17:58:12.876 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/powr1220.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5627.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch5636.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht15.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sht21.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/shtc1.ko 2023-12-19T17:58:12.877 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/sis5595.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47b397.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m1.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/smsc47m192.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/thmc50.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp102.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp103.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp401.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/tmp421.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via-cputemp.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/via686a.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt1211.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/vt8231.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627ehf.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83627hf.ko 2023-12-19T17:58:12.878 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83781d.ko 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83791d.ko 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83792d.ko 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83793.ko 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83795.ko 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l785ts.ko 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/hwmon/w83l786ng.ko 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-19T17:58:12.879 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-19T17:58:12.880 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-19T17:58:12.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-19T17:58:12.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-19T17:58:12.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-19T17:58:12.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-19T17:58:12.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-19T17:58:12.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-dev.ko 2023-12-19T17:58:12.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-mux.ko 2023-12-19T17:58:12.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-12-19T17:58:12.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-smbus.ko 2023-12-19T17:58:12.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/i2c/i2c-stub.ko 2023-12-19T17:58:12.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband 2023-12-19T17:58:12.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core 2023-12-19T17:58:12.881 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_core.ko 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca 2023-12-19T17:58:12.882 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-19T17:58:12.883 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/gameport/gameport.ko 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/joydev.ko 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/appletouch.ko 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/bcm5974.ko 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/cyapatp.ko 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/sermouse.ko 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-19T17:58:12.884 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/altera_ps2.ko 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/arc_ps2.ko 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/serio/serio_raw.ko 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/input/sparse-keymap.ko 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-blinkm.ko 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lm3530.ko 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-lp3944.ko 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/leds-ss4200.ko 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger 2023-12-19T17:58:12.885 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/bcache/bcache.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-bio-prison.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache-smq.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-cache.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-crypt.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-delay.ko 2023-12-19T17:58:12.886 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-flakey.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-log-userspace.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-multipath.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-queue-length.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-raid.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-round-robin.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-service-time.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-switch.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-thin-pool.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/dm-verity.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/faulty.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/linear.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/multipath.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-19T17:58:12.887 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid0.ko 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid1.ko 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid10.ko 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/md/raid456.ko 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptbase.ko 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptctl.ko 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptfc.ko 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptlan.ko 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptsas.ko 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptscsih.ko 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/message/fusion/mptspi.ko 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_ich.ko 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/lpc_sch.ko 2023-12-19T17:58:12.888 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/mfd-core.ko 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/sm501.ko 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/viperboard.ko 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/vx855.ko 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/mfd/wl1273-core.ko 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds9802als.ko 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/apds990x.ko 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/bh1770glc.ko 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/cb710/cb710.ko 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom 2023-12-19T17:58:12.889 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/at24.ko 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/eeprom/max6875.ko 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/enclosure.ko 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/hpilo.ko 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29003.ko 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/isl29020.ko 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-me.ko 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei-txe.ko 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/mei/mei.ko 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru 2023-12-19T17:58:12.890 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-gru/gru.ko 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xp.ko 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpc.ko 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_7xx1.ko 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tifm_core.ko 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/tsl2550.ko 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_balloon.ko 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/bonding/bonding.ko 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53 2023-12-19T17:58:12.891 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/dummy.ko 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/eql.ko 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-19T17:58:12.892 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-19T17:58:12.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-19T17:58:12.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-19T17:58:12.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec 2023-12-19T17:58:12.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-19T17:58:12.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere 2023-12-19T17:58:12.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-19T17:58:12.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon 2023-12-19T17:58:12.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-19T17:58:12.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera 2023-12-19T17:58:12.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-19T17:58:12.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd 2023-12-19T17:58:12.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-19T17:58:12.893 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-19T17:58:12.894 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-19T17:58:12.895 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-19T17:58:12.896 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/dnet.ko 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ethoc.ko 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/fealnx.ko 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e 2023-12-19T17:58:12.897 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/jme.ko 2023-12-19T17:58:12.898 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge 2023-12-19T17:58:12.899 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-19T17:58:12.900 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-19T17:58:12.901 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-19T17:58:12.902 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/geneve.ko 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv 2023-12-19T17:58:12.903 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ifb.ko 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvlan.ko 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/macvtap.ko 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio.ko 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/mii.ko 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/net_failover.ko 2023-12-19T17:58:12.904 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/netconsole.ko 2023-12-19T17:58:12.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/nlmon.ko 2023-12-19T17:58:12.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs 2023-12-19T17:58:12.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-19T17:58:12.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-19T17:58:12.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy 2023-12-19T17:58:12.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/amd.ko 2023-12-19T17:58:12.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-19T17:58:12.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-12-19T17:58:12.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-19T17:58:12.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/bcm87xx.ko 2023-12-19T17:58:12.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/broadcom.ko 2023-12-19T17:58:12.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/cicada.ko 2023-12-19T17:58:12.905 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/davicom.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/dp83640.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/et1011c.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/icplus.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/lxt.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/marvell.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/micrel.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/national.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/phylink.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/qsemi.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/realtek.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/smsc.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/ste10Xp.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/phy/vitesse.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-19T17:58:12.906 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_async.ko 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_generic.ko 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppoe.ko 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pppox.ko 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/ppp/pptp.ko 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slhc.ko 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/slip/slip.ko 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/sungem_phy.ko 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tap.ko 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team 2023-12-19T17:58:12.907 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team.ko 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_random.ko 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/tun.ko 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/veth.ko 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/virtio_net.ko 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/vxlan/vxlan.ko 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/net/xen-netfront.ko 2023-12-19T17:58:12.908 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ntb/ntb.ko 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme-core.ko 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/nvme/host/nvme.ko 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport.ko 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_cs.ko 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_pc.ko 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/parport/parport_serial.ko 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-19T17:58:12.909 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/pcie/aer_inject.ko 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pci/xen-pcifront.ko 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/i82092.ko 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/pd6729.ko 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/acerhdf.ko 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-19T17:58:12.910 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/hdaps.ko 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/intel_ips.ko 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-19T17:58:12.911 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/platform/x86/wmi.ko 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-19T17:58:12.912 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/clients/pps_parport.ko 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/pps/pps_core.ko 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp.ko 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ptp/ptp_kvm.ko 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-19T17:58:12.913 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-em3027.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-max6900.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-19T17:58:12.914 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/rtc/rtc-x1205.ko 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-9xxx.ko 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-sas.ko 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/3w-xxxx.ko 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/BusLogic.ko 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/a100u2w.ko 2023-12-19T17:58:12.915 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/advansys.ko 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/am53c974.ko 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/atp870u.ko 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bfa/bfa.ko 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-19T17:58:12.916 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i 2023-12-19T17:58:12.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-19T17:58:12.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ch.ko 2023-12-19T17:58:12.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor 2023-12-19T17:58:12.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-19T17:58:12.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi 2023-12-19T17:58:12.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i 2023-12-19T17:58:12.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-19T17:58:12.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i 2023-12-19T17:58:12.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-19T17:58:12.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-19T17:58:12.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dc395x.ko 2023-12-19T17:58:12.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler 2023-12-19T17:58:12.917 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/dmx3191d.ko 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/esp_scsi.ko 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/fnic/fnic.ko 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hpsa.ko 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hptiop.ko 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/hv_storvsc.ko 2023-12-19T17:58:12.918 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/initio.ko 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ipr.ko 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ips.ko 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/isci/isci.ko 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libfc/libfc.ko 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi.ko 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/libsas/libsas.ko 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-19T17:58:12.919 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid.ko 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/mvumi.ko 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/pmcraid.ko 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla1280.ko 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-19T17:58:12.920 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/raid_class.ko 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_debug.ko 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/ses.ko 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/st.ko 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/stex.ko 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-19T17:58:12.921 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/virtio_scsi.ko 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/scsi/wd719x.ko 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ssb/ssb.ko 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_file.ko 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_iblock.ko 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_mod.ko 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_pscsi.ko 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/target_core_user.ko 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc 2023-12-19T17:58:12.922 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-19T17:58:12.923 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_gsm.ko 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/n_hdlc.ko 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/nozomi.ko 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/arc_uart.ko 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm 2023-12-19T17:58:12.924 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio.ko 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_aec.ko 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_cif.ko 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_pci_generic.ko 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/uio/uio_sercos3.ko 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-acm.ko 2023-12-19T17:58:12.925 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usblp.ko 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/class/usbtmc.ko 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/mdc800.ko 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/image/microtek.ko 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/adutux.ko 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/appledisplay.ko 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi26.ko 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/emi62.ko 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ezusb.ko 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/idmouse.ko 2023-12-19T17:58:12.926 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/iowarrior.ko 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/ldusb.ko 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/legousbtower.ko 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usb3503.ko 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usblcd.ko 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/uss720.ko 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/misc/yurex.ko 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/phy/phy-generic.ko 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/aircable.ko 2023-12-19T17:58:12.927 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ark3116.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ch341.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cp210x.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cyberjack.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/empeg.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/io_ti.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipaq.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ipw.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ir-usb.ko 2023-12-19T17:58:12.928 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mct_u232.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7720.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/mos7840.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/navman.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/omninet.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/opticon.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/option.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/oti6858.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/pl2303.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcaux.ko 2023-12-19T17:58:12.929 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/qcserial.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/quatech2.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/safe_serial.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/sierra.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ssu100.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/symbolserial.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_debug.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/visor.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/whiteheat.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-19T17:58:12.930 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/uas.ko 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-karma.ko 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-19T17:58:12.931 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/storage/usb-storage.ko 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio.ko 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost.ko 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-19T17:58:12.932 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_net.ko 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/vhost/vhost_scsi.ko 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/apple_bl.ko 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/lcd.ko 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio.ko 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_balloon.ko 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_mmio.ko 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci.ko 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/virtio/virtio_ring.ko 2023-12-19T17:58:12.933 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2482.ko 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/masters/ds2490.ko 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-19T17:58:12.934 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/w1/wire.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/hpwdt.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/i6300esb.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ib700wdt.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ibmasr.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/it87_wdt.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/machzwd.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/nv_tco.ko 2023-12-19T17:58:12.935 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/softdog.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/via_wdt.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/wdt_pci.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/watchdog/xen_wdt.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-acpi-processor.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-evtchn.ko 2023-12-19T17:58:12.936 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntalloc.ko 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-gntdev.ko 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xen-privcmd.ko 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/9p/9p.ko 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/affs/affs.ko 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/befs/befs.ko 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/binfmt_misc.ko 2023-12-19T17:58:12.937 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/btrfs/btrfs.ko 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cachefiles/cachefiles.ko 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ceph/ceph.ko 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/coda/coda.ko 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/cramfs/cramfs.ko 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/dlm/dlm.ko 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ecryptfs/ecryptfs.ko 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs 2023-12-19T17:58:12.938 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/f2fs/f2fs.ko 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/fat.ko 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/msdos.ko 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fat/vfat.ko 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fscache/fscache.ko 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/cuse.ko 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/fuse/fuse.ko 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/gfs2/gfs2.ko 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfs/hfs.ko 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus 2023-12-19T17:58:12.939 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/hfsplus/hfsplus.ko 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/isofs/isofs.ko 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/jfs/jfs.ko 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/lockd/lockd.ko 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/minix/minix.ko 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/netfs/netfs.ko 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout 2023-12-19T17:58:12.940 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfs.ko 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv3.ko 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs/nfsv4.ko 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/grace.ko 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfs_common/nfs_acl.ko 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nfsd/nfsd.ko 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nilfs2/nilfs2.ko 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-celtic.ko 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-centeuro.ko 2023-12-19T17:58:12.941 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-croatian.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-cyrillic.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-gaelic.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-greek.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-iceland.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-inuit.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-roman.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-romanian.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/mac-turkish.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1250.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1251.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp1255.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp737.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp775.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp850.ko 2023-12-19T17:58:12.942 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp852.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp855.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp857.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp860.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp861.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp862.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp863.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp864.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp865.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp866.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp869.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp874.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp932.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp936.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp949.ko 2023-12-19T17:58:12.943 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_cp950.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_euc-jp.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-1.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-13.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-14.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-15.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-2.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-3.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-4.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-5.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-6.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-7.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_iso8859-9.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-r.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-ru.ko 2023-12-19T17:58:12.944 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_koi8-u.ko 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/nls/nls_utf8.ko 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2.ko 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-19T17:58:12.945 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/overlayfs/overlay.ko 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/pstore/ramoops.ko 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/reiserfs/reiserfs.ko 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/romfs/romfs.ko 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/client/cifs.ko 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_arc4.ko 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/smb/common/cifs_md4.ko 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs 2023-12-19T17:58:12.946 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/squashfs/squashfs.ko 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/sysv/sysv.ko 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/udf/udf.ko 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/ufs/ufs.ko 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/fs/xfs/xfs.ko 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/rcu/rcutorture.ko 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/torture.ko 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-12-19T17:58:12.947 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/asn1_encoder.ko 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc-itu-t.ko 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crc8.ko 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libarc4.ko 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/crypto/libdes.ko 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/libcrc32c.ko 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lru_cache.ko 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4_compress.ko 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/lz4/lz4hc_compress.ko 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/math/cordic.ko 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6 2023-12-19T17:58:12.948 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/raid6/raid6_pq.ko 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_bm.ko 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_fsm.ko 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/ts_kmp.ko 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/lib/zstd/zstd_compress.ko 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/mm/hwpoison-inject.ko 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/6lowpan.ko 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_dest.ko 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_fragment.ko 2023-12-19T17:58:12.949 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_hop.ko 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_mobility.ko 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_routing.ko 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/6lowpan/nhc_udp.ko 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/garp.ko 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/mrp.ko 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/p8022.ko 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/psnap.ko 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/802/stp.ko 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/8021q/8021q.ko 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet.ko 2023-12-19T17:58:12.950 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_fd.ko 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_rdma.ko 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/9p/9pnet_virtio.ko 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/batman-adv/batman-adv.ko 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/br_netfilter.ko 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/bridge.ko 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-19T17:58:12.951 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/ebtables.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-12-19T17:58:12.952 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ceph/libceph.ko 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/failover.ko 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/core/pktgen.ko 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp.ko 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_diag.ko 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv4.ko 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dccp/dccp_ipv6.ko 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dns_resolver/dns_resolver.ko 2023-12-19T17:58:12.953 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/dsa_core.ko 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_brcm.ko 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_dsa.ko 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_none.ko 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/dsa/tag_trailer.ko 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154.ko 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ah4.ko 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/esp4.ko 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/fou.ko 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/gre.ko 2023-12-19T17:58:12.954 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/inet_diag.ko 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_gre.ko 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_tunnel.ko 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ip_vti.ko 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipcomp.ko 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/ipip.ko 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-19T17:58:12.955 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_bic.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_dctcp.ko 2023-12-19T17:58:12.956 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_diag.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_highspeed.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_htcp.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_hybla.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_illinois.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_lp.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_scalable.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_vegas.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_veno.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_westwood.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tcp_yeah.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/tunnel4.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_diag.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/udp_tunnel.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6 2023-12-19T17:58:12.957 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ah6.ko 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/esp6.ko 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/fou6.ko 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_tunnel.ko 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ip6_vti.ko 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/ipcomp6.ko 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-19T17:58:12.958 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-19T17:58:12.959 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/sit.ko 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/tunnel6.ko 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/key/af_key.ko 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_core.ko 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_eth.ko 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip.ko 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ip6.ko 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_netlink.ko 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/l2tp/l2tp_ppp.ko 2023-12-19T17:58:12.960 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/llc/llc.ko 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mac802154/mac802154.ko 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/mpls/mpls_gso.ko 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set.ko 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-19T17:58:12.961 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-19T17:58:12.962 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conncount.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-19T17:58:12.963 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_log_syslog.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_irc.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_sip.ko 2023-12-19T17:58:12.964 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nf_tables.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_log.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_chain_nat.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_compat.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_ct.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fib.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_hash.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_limit.ko 2023-12-19T17:58:12.965 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_log.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_masq.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_nat.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_numgen.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_queue.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_quota.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_redir.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_reject_inet.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_socket.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/nft_tunnel.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_AUDIT.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-19T17:58:12.966 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_CT.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_DSCP.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HL.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_HMARK.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LED.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_LOG.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NETMAP.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFLOG.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_RATEEST.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_SECMARK.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-19T17:58:12.967 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TEE.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TPROXY.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_TRACE.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_addrtype.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_bpf.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cgroup.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cluster.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_comment.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connbytes.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlabel.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connlimit.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_connmark.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_conntrack.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_cpu.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dccp.ko 2023-12-19T17:58:12.968 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_devgroup.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_dscp.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ecn.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_esp.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hashlimit.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_helper.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_hl.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipcomp.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_iprange.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_ipvs.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_l2tp.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_length.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_limit.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mac.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_mark.ko 2023-12-19T17:58:12.969 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_multiport.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nat.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_nfacct.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_osf.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_owner.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_physdev.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_pkttype.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_policy.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_quota.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_rateest.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_realm.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_recent.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_sctp.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_set.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_socket.ko 2023-12-19T17:58:12.970 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_state.ko 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_statistic.ko 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_string.ko 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_tcpmss.ko 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_time.ko 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netfilter/xt_u32.ko 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/netlink/netlink_diag.ko 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/nsh/nsh.ko 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/openvswitch.ko 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-geneve.ko 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-gre.ko 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/openvswitch/vport-vxlan.ko 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet 2023-12-19T17:58:12.971 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/packet/af_packet_diag.ko 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds.ko 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_rdma.ko 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/rds/rds_tcp.ko 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_csum.ko 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_gact.ko 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_ipt.ko 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_mirred.ko 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_nat.ko 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_pedit.ko 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_police.ko 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_simple.ko 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_skbedit.ko 2023-12-19T17:58:12.972 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/act_vlan.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_basic.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_bpf.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_flow.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_fw.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_route.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/cls_u32.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_cmp.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_ipset.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_meta.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_nbyte.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_text.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/em_u32.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_choke.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_codel.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_drr.ko 2023-12-19T17:58:12.973 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_fq.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_gred.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hfsc.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_hhf.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_htb.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_ingress.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_mqprio_lib.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_multiq.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_netem.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_pie.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_plug.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_prio.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_qfq.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_red.ko 2023-12-19T17:58:12.974 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfb.ko 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_sfq.ko 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_tbf.ko 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sched/sch_teql.ko 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp.ko 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sctp/sctp_diag.ko 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/sunrpc.ko 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/diag.ko 2023-12-19T17:58:12.975 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/tipc/tipc.ko 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/unix/unix_diag.ko 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock.ko 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys 2023-12-19T17:58:12.976 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/security/keys/trusted-keys/trusted.ko 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/kernel/virt/lib/irqbypass.ko 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.alias.bin 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.alias.bin 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.bin 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.builtin.modinfo 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.dep.bin 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.devname 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.order 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.softdep 2023-12-19T17:58:12.977 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols 2023-12-19T17:58:12.978 INFO:teuthology.orchestra.run.smithi192.stdout:/lib/modules/6.6.0-gcd68252d7b4d/modules.symbols.bin 2023-12-19T17:58:12.978 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-gcd68252d7b4d 2023-12-19T17:58:12.978 DEBUG:teuthology.orchestra.run.smithi192:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-12-19T17:58:13.000 INFO:teuthology.orchestra.run.smithi192.stdout:postinstall scriptlet (using /bin/sh): 2023-12-19T17:58:13.000 INFO:teuthology.orchestra.run.smithi192.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.6.0-gcd68252d7b4d -a -r /boot/System.map-6.6.0-gcd68252d7b4d ]; then 2023-12-19T17:58:13.000 INFO:teuthology.orchestra.run.smithi192.stdout:cp /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm 2023-12-19T17:58:13.000 INFO:teuthology.orchestra.run.smithi192.stdout:cp /boot/System.map-6.6.0-gcd68252d7b4d /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-19T17:58:13.000 INFO:teuthology.orchestra.run.smithi192.stdout:rm -f /boot/vmlinuz-6.6.0-gcd68252d7b4d /boot/System.map-6.6.0-gcd68252d7b4d 2023-12-19T17:58:13.001 INFO:teuthology.orchestra.run.smithi192.stdout:/sbin/installkernel 6.6.0-gcd68252d7b4d /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-19T17:58:13.001 INFO:teuthology.orchestra.run.smithi192.stdout:rm -f /boot/.vmlinuz-6.6.0-gcd68252d7b4d-rpm /boot/.System.map-6.6.0-gcd68252d7b4d-rpm 2023-12-19T17:58:13.001 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2023-12-19T17:58:13.001 INFO:teuthology.orchestra.run.smithi192.stdout:preuninstall scriptlet (using /bin/sh): 2023-12-19T17:58:13.001 INFO:teuthology.orchestra.run.smithi192.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-12-19T17:58:13.001 INFO:teuthology.orchestra.run.smithi192.stdout:new-kernel-pkg --remove 6.6.0-gcd68252d7b4d --rminitrd --initrdfile=/boot/initramfs-6.6.0-gcd68252d7b4d.img 2023-12-19T17:58:13.001 INFO:teuthology.orchestra.run.smithi192.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-12-19T17:58:13.001 INFO:teuthology.orchestra.run.smithi192.stdout:kernel-install remove 6.6.0-gcd68252d7b4d 2023-12-19T17:58:13.001 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2023-12-19T17:58:13.001 INFO:teuthology.orchestra.run.smithi192.stdout:postuninstall scriptlet (using /bin/sh): 2023-12-19T17:58:13.001 INFO:teuthology.orchestra.run.smithi192.stdout:if [ -x /sbin/update-bootloader ]; then 2023-12-19T17:58:13.001 INFO:teuthology.orchestra.run.smithi192.stdout:/sbin/update-bootloader --remove 6.6.0-gcd68252d7b4d 2023-12-19T17:58:13.001 INFO:teuthology.orchestra.run.smithi192.stdout:fi 2023-12-19T17:58:13.002 DEBUG:teuthology.orchestra.run.smithi192:> sudo rpm -qi grub2-tools 2023-12-19T17:58:13.077 INFO:teuthology.orchestra.run.smithi192.stdout:Name : grub2-tools 2023-12-19T17:58:13.077 INFO:teuthology.orchestra.run.smithi192.stdout:Epoch : 1 2023-12-19T17:58:13.077 INFO:teuthology.orchestra.run.smithi192.stdout:Version : 2.06 2023-12-19T17:58:13.077 INFO:teuthology.orchestra.run.smithi192.stdout:Release : 68.el9 2023-12-19T17:58:13.077 INFO:teuthology.orchestra.run.smithi192.stdout:Architecture: x86_64 2023-12-19T17:58:13.077 INFO:teuthology.orchestra.run.smithi192.stdout:Install Date: Fri 01 Sep 2023 09:14:30 PM UTC 2023-12-19T17:58:13.077 INFO:teuthology.orchestra.run.smithi192.stdout:Group : Unspecified 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:Size : 8264254 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:License : GPLv3+ 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:Packager : builder@centos.org 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:Vendor : CentOS 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:URL : http://www.gnu.org/software/grub/ 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:Summary : Support tools for GRUB. 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:Description : 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:customizable bootloader with modular architecture. It supports a rich 2023-12-19T17:58:13.078 INFO:teuthology.orchestra.run.smithi192.stdout:variety of kernel formats, file systems, computer architectures and 2023-12-19T17:58:13.079 INFO:teuthology.orchestra.run.smithi192.stdout:hardware devices. 2023-12-19T17:58:13.079 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T17:58:13.079 INFO:teuthology.orchestra.run.smithi192.stdout:This subpackage provides tools for support of all platforms. 2023-12-19T17:58:13.079 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-12-19T17:58:13.080 INFO:teuthology.task.kernel:Updating grub on smithi192 to boot 6.6.0-gcd68252d7b4d 2023-12-19T17:58:13.080 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-12-19T17:58:13.690 INFO:teuthology.orchestra.run.smithi192.stderr:Generating grub configuration file ... 2023-12-19T17:58:15.086 INFO:teuthology.orchestra.run.smithi192.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2023-12-19T17:58:15.130 INFO:teuthology.orchestra.run.smithi192.stderr:done 2023-12-19T17:58:15.132 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2023-12-19T17:58:15.147 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.LJpDUsAevX 2023-12-19T17:58:15.147 DEBUG:teuthology.orchestra.run.smithi192:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LJpDUsAevX 2023-12-19T17:58:15.216 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.LJpDUsAevX 2023-12-19T17:58:15.358 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.LJpDUsAevX is 6KB 2023-12-19T17:58:15.408 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.LJpDUsAevX 2023-12-19T17:58:15.423 DEBUG:teuthology.orchestra.run.smithi192:> sudo /bin/ls /boot/loader/entries || true 2023-12-19T17:58:15.493 INFO:teuthology.orchestra.run.smithi192.stdout:15949e3c88704300882482eddfd7baae-0-rescue.conf 2023-12-19T17:58:15.494 INFO:teuthology.orchestra.run.smithi192.stdout:15949e3c88704300882482eddfd7baae-5.14.0-239.el9.x86_64.conf 2023-12-19T17:58:15.494 INFO:teuthology.orchestra.run.smithi192.stdout:15949e3c88704300882482eddfd7baae-5.14.0-361.el9.x86_64.conf 2023-12-19T17:58:15.494 INFO:teuthology.orchestra.run.smithi192.stdout:15949e3c88704300882482eddfd7baae-6.6.0-gcd68252d7b4d.conf 2023-12-19T17:58:15.495 DEBUG:teuthology.orchestra.run.smithi192:> sudo grub2-set-default 15949e3c88704300882482eddfd7baae-6.6.0-gcd68252d7b4d 2023-12-19T17:58:15.656 DEBUG:teuthology.orchestra.run.smithi192:> sudo shutdown -r now 2023-12-19T17:58:37.577 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-12-19T17:58:37.578 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-19T17:58:37.578 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T17:58:45.665 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-12-19T17:58:45.666 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-19T17:58:45.666 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T17:58:56.021 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-12-19T17:59:05.029 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-19T17:59:05.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T17:59:08.117 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-12-19T17:59:20.129 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-19T17:59:20.130 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T17:59:23.217 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-12-19T17:59:38.229 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-19T17:59:38.230 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T17:59:41.302 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.27 2023-12-19T17:59:45.677 DEBUG:teuthology.orchestra.remote:timed out 2023-12-19T17:59:54.685 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-19T17:59:54.686 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T17:59:59.305 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-19T17:59:59.306 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:00:10.369 DEBUG:teuthology.orchestra.run.smithi192:> true 2023-12-19T18:00:10.999 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-19T18:00:10.999 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.6.0-gcd68252d7b4d"... 2023-12-19T18:00:10.999 DEBUG:teuthology.orchestra.run.smithi192:> uname -r 2023-12-19T18:00:11.040 INFO:teuthology.orchestra.run.smithi192.stdout:6.6.0-gcd68252d7b4d 2023-12-19T18:00:11.040 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-gcd68252d7b4d vs 6.6.0-gcd68252d7b4d 2023-12-19T18:00:11.040 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-19T18:00:11.040 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-19T18:00:12.041 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-12-19T18:00:12.041 DEBUG:teuthology.orchestra.run.smithi192:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-19T18:00:12.159 INFO:teuthology.orchestra.run.smithi192.stdout:ttyS1 2023-12-19T18:00:12.203 DEBUG:teuthology.parallel:result is None 2023-12-19T18:00:59.317 DEBUG:teuthology.orchestra.remote:timed out 2023-12-19T18:01:20.318 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-19T18:01:20.319 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:01:20.735 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-12-19T18:01:21.399 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-19T18:01:21.400 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.6.0-gcd68252d7b4d"... 2023-12-19T18:01:21.400 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2023-12-19T18:01:21.459 INFO:teuthology.orchestra.run.smithi027.stdout:6.6.0-gcd68252d7b4d 2023-12-19T18:01:21.460 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-gcd68252d7b4d vs 6.6.0-gcd68252d7b4d 2023-12-19T18:01:21.460 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-19T18:01:21.460 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-12-19T18:01:22.461 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-12-19T18:01:22.461 DEBUG:teuthology.orchestra.run.smithi027:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-12-19T18:01:22.618 INFO:teuthology.orchestra.run.smithi027.stdout:ttyS1 2023-12-19T18:01:22.662 DEBUG:teuthology.parallel:result is None 2023-12-19T18:01:22.662 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-19T18:01:22.673 INFO:teuthology.task.internal:Creating test directory... 2023-12-19T18:01:22.674 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-19T18:01:22.683 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-19T18:01:22.743 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-19T18:01:22.755 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-19T18:01:22.766 INFO:teuthology.task.internal:Creating archive directory... 2023-12-19T18:01:22.767 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-19T18:01:22.776 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-19T18:01:22.847 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-19T18:01:22.860 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-19T18:01:22.861 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-19T18:01:22.870 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-19T18:01:23.002 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-19T18:01:23.005 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-19T18:01:23.047 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-19T18:01:23.059 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-19T18:01:23.064 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-19T18:01:23.073 INFO:teuthology.task.internal:Configuring sudo... 2023-12-19T18:01:23.074 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-19T18:01:23.082 DEBUG:teuthology.orchestra.run.smithi192:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-19T18:01:23.187 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-19T18:01:23.198 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-19T18:01:23.198 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-19T18:01:23.208 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-19T18:01:23.265 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-19T18:01:23.367 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-19T18:01:23.493 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-19T18:01:23.598 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-19T18:01:23.707 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:01:23.707 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-19T18:01:23.819 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-19T18:01:23.925 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-19T18:01:24.045 DEBUG:teuthology.orchestra.run.smithi192:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-19T18:01:24.143 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-19T18:01:24.245 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:01:24.245 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-19T18:01:24.342 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2023-12-19T18:01:24.351 DEBUG:teuthology.orchestra.run.smithi192:> sudo service rsyslog restart 2023-12-19T18:01:24.489 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-19T18:01:24.507 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-19T18:01:24.977 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-19T18:01:24.989 INFO:teuthology.task.internal:Starting timer... 2023-12-19T18:01:24.989 INFO:teuthology.run_tasks:Running task pcp... 2023-12-19T18:01:25.005 INFO:teuthology.run_tasks:Running task selinux... 2023-12-19T18:01:25.096 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2023-12-19T18:01:25.243 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: 2023-12-19T18:01:25.248 DEBUG:teuthology.orchestra.run.smithi192:> sudo service auditd rotate 2023-12-19T18:01:25.402 INFO:teuthology.orchestra.run.smithi192.stdout:Rotating logs: 2023-12-19T18:01:25.407 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-19T18:01:25.407 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2023-12-19T18:01:25.485 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2023-12-19T18:01:25.485 DEBUG:teuthology.orchestra.run.smithi192:> /usr/sbin/getenforce 2023-12-19T18:01:25.551 INFO:teuthology.orchestra.run.smithi192.stdout:Permissive 2023-12-19T18:01:25.551 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi192.front.sepia.ceph.com': 'permissive'} 2023-12-19T18:01:25.551 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-19T18:01:25.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:01:25.662 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-19T18:01:25.758 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:01:25.759 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-19T18:01:25.759 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2023-12-19T18:01:25.883 DEBUG:teuthology.orchestra.run.smithi192:> sudo /usr/sbin/setenforce permissive 2023-12-19T18:01:25.989 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-19T18:01:26.039 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-12-19T18:01:26.044 INFO:teuthology.repo_utils:Fetching main from origin 2023-12-19T18:01:26.209 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-19T18:01:26.225 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-19T18:01:26.227 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi192.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-19T18:12:08.695 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi192.front.sepia.ceph.com')] 2023-12-19T18:12:08.697 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-19T18:12:08.697 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:12:08.878 DEBUG:teuthology.orchestra.run.smithi027:> true 2023-12-19T18:12:09.039 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi027.front.sepia.ceph.com' 2023-12-19T18:12:09.040 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-19T18:12:09.040 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi192.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-19T18:12:09.198 DEBUG:teuthology.orchestra.run.smithi192:> true 2023-12-19T18:12:09.348 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi192.front.sepia.ceph.com' 2023-12-19T18:12:09.348 INFO:teuthology.run_tasks:Running task clock... 2023-12-19T18:12:09.360 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-19T18:12:09.360 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-19T18:12:09.361 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-19T18:12:09.370 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-19T18:12:09.370 DEBUG:teuthology.orchestra.run.smithi192:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-19T18:12:09.470 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-19T18:12:09.483 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-19T18:12:09.521 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-19T18:12:09.544 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-19T18:12:09.616 INFO:teuthology.orchestra.run.smithi192.stderr:sudo: ntpd: command not found 2023-12-19T18:12:09.657 INFO:teuthology.orchestra.run.smithi192.stdout:506 Cannot talk to daemon 2023-12-19T18:12:09.659 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2023-12-19T18:12:09.707 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-19T18:12:09.711 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2023-12-19T18:12:09.757 INFO:teuthology.orchestra.run.smithi192.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-19T18:12:09.769 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-19T18:12:09.833 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-19T18:12:09.890 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2023-12-19T18:12:09.898 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-19T18:12:09.899 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2023-12-19T18:12:09.899 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-19T18:12:09.899 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-19T18:12:09.899 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-19T18:12:09.899 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-19T18:12:09.972 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2023-12-19T18:12:09.982 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-19T18:12:09.982 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2023-12-19T18:12:09.982 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-19T18:12:09.982 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-19T18:12:09.982 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-19T18:12:09.982 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-19T18:12:09.983 INFO:teuthology.run_tasks:Running task install... 2023-12-19T18:12:09.994 DEBUG:teuthology.task.install:project ceph 2023-12-19T18:12:09.994 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}} 2023-12-19T18:12:09.994 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'} 2023-12-19T18:12:09.994 INFO:teuthology.task.install:Using flavor: default 2023-12-19T18:12:10.001 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-19T18:12:10.002 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2023-12-19T18:12:10.002 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e', 'tag': None, 'wait_for_package': False} 2023-12-19T18:12:10.002 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-19T18:12:10.004 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e', 'tag': None, 'wait_for_package': False} 2023-12-19T18:12:10.005 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-19T18:12:10.153 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/centos/9/flavors/default/ 2023-12-19T18:12:10.153 INFO:teuthology.task.install.rpm:Package version is 18.2.1-132.g568df92b 2023-12-19T18:12:10.166 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/centos/9/flavors/default/ 2023-12-19T18:12:10.167 INFO:teuthology.task.install.rpm:Package version is 18.2.1-132.g568df92b 2023-12-19T18:12:10.334 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-19T18:12:10.334 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:12:10.334 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-19T18:12:10.344 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-19T18:12:10.344 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:12:10.344 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-19T18:12:10.447 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-19T18:12:10.447 DEBUG:teuthology.orchestra.run.smithi192:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/568df92b7e50b2108bce4c77c393ff294d6a5e5e/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-19T18:12:10.452 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-19T18:12:10.452 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/568df92b7e50b2108bce4c77c393ff294d6a5e5e/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-19T18:12:10.588 DEBUG:teuthology.orchestra.run.smithi192:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-19T18:12:10.607 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-19T18:12:10.725 DEBUG:teuthology.orchestra.run.smithi192:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-19T18:12:10.766 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-19T18:12:10.818 INFO:teuthology.orchestra.run.smithi192.stdout:check_obsoletes = 1 2023-12-19T18:12:10.822 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum clean all 2023-12-19T18:12:10.866 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2023-12-19T18:12:10.871 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2023-12-19T18:12:11.181 INFO:teuthology.orchestra.run.smithi192.stdout:73 files removed 2023-12-19T18:12:11.219 DEBUG:teuthology.orchestra.run.smithi192:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-19T18:12:11.256 INFO:teuthology.orchestra.run.smithi027.stdout:73 files removed 2023-12-19T18:12:11.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-19T18:12:12.307 INFO:teuthology.orchestra.run.smithi192.stdout:ceph packages for x86_64 134 kB/s | 77 kB 00:00 2023-12-19T18:12:12.394 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 139 kB/s | 77 kB 00:00 2023-12-19T18:12:12.774 INFO:teuthology.orchestra.run.smithi192.stdout:ceph noarch packages 25 kB/s | 9.5 kB 00:00 2023-12-19T18:12:12.894 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 24 kB/s | 9.5 kB 00:00 2023-12-19T18:12:13.353 INFO:teuthology.orchestra.run.smithi192.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2023-12-19T18:12:13.479 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 3.7 kB/s | 1.9 kB 00:00 2023-12-19T18:12:13.982 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - AppStream 12 MB/s | 6.4 MB 00:00 2023-12-19T18:12:14.174 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - AppStream 10 MB/s | 6.4 MB 00:00 2023-12-19T18:12:15.624 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS-9 - BaseOS 5.0 MB/s | 2.0 MB 00:00 2023-12-19T18:12:15.758 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-9 - BaseOS 5.7 MB/s | 2.0 MB 00:00 2023-12-19T18:12:16.738 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 7.9 MB 00:00 2023-12-19T18:12:16.861 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 7.9 MB 00:00 2023-12-19T18:12:21.630 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 18 MB 00:03 2023-12-19T18:12:26.330 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - AppStream 2.2 MB/s | 18 MB 00:08 2023-12-19T18:12:27.119 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - CRB 3.2 MB/s | 5.8 MB 00:01 2023-12-19T18:12:28.914 INFO:teuthology.orchestra.run.smithi192.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 15 kB 00:00 2023-12-19T18:12:30.733 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - CRB 11 MB/s | 5.8 MB 00:00 2023-12-19T18:12:30.741 INFO:teuthology.orchestra.run.smithi192.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2023-12-19T18:12:33.068 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 15 kB 00:01 2023-12-19T18:12:36.299 INFO:teuthology.orchestra.run.smithi192.stdout:lab-extras 40 kB/s | 1.7 kB 00:00 2023-12-19T18:12:37.431 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 4.7 MB/s | 20 MB 00:04 2023-12-19T18:12:38.956 INFO:teuthology.orchestra.run.smithi192.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-19T18:12:38.957 INFO:teuthology.orchestra.run.smithi192.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-19T18:12:38.977 INFO:teuthology.orchestra.run.smithi192.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-12-19T18:12:38.978 INFO:teuthology.orchestra.run.smithi192.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-12-19T18:12:38.979 INFO:teuthology.orchestra.run.smithi192.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-12-19T18:12:38.981 INFO:teuthology.orchestra.run.smithi192.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-12-19T18:12:38.983 INFO:teuthology.orchestra.run.smithi192.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-12-19T18:12:38.986 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-12-19T18:12:38.988 INFO:teuthology.orchestra.run.smithi192.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-12-19T18:12:39.068 INFO:teuthology.orchestra.run.smithi192.stdout:Dependencies resolved. 2023-12-19T18:12:39.090 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================== 2023-12-19T18:12:39.091 INFO:teuthology.orchestra.run.smithi192.stdout: Package Arch Version Repository Size 2023-12-19T18:12:39.091 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================== 2023-12-19T18:12:39.091 INFO:teuthology.orchestra.run.smithi192.stdout:Installing: 2023-12-19T18:12:39.091 INFO:teuthology.orchestra.run.smithi192.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-12-19T18:12:39.091 INFO:teuthology.orchestra.run.smithi192.stdout: ceph x86_64 2:18.2.1-132.g568df92b.el9 ceph 6.5 k 2023-12-19T18:12:39.091 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base x86_64 2:18.2.1-132.g568df92b.el9 ceph 5.2 M 2023-12-19T18:12:39.091 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse x86_64 2:18.2.1-132.g568df92b.el9 ceph 840 k 2023-12-19T18:12:39.091 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-132.g568df92b.el9 ceph 142 k 2023-12-19T18:12:39.091 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr x86_64 2:18.2.1-132.g568df92b.el9 ceph 1.4 M 2023-12-19T18:12:39.091 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 132 k 2023-12-19T18:12:39.092 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 1.8 M 2023-12-19T18:12:39.092 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 7.4 M 2023-12-19T18:12:39.092 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 50 k 2023-12-19T18:12:39.092 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw x86_64 2:18.2.1-132.g568df92b.el9 ceph 7.7 M 2023-12-19T18:12:39.092 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test x86_64 2:18.2.1-132.g568df92b.el9 ceph 40 M 2023-12-19T18:12:39.092 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 262 k 2023-12-19T18:12:39.092 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 221 k 2023-12-19T18:12:39.092 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror x86_64 2:18.2.1-132.g568df92b.el9 ceph 219 k 2023-12-19T18:12:39.092 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell x86_64 2:18.2.1-132.g568df92b.el9 ceph 27 k 2023-12-19T18:12:39.092 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 18 k 2023-12-19T18:12:39.092 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-12-19T18:12:39.092 INFO:teuthology.orchestra.run.smithi192.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-12-19T18:12:39.093 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-12-19T18:12:39.093 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel x86_64 2:18.2.1-132.g568df92b.el9 ceph 31 k 2023-12-19T18:12:39.093 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2 x86_64 2:18.2.1-132.g568df92b.el9 ceph 658 k 2023-12-19T18:12:39.093 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel x86_64 2:18.2.1-132.g568df92b.el9 ceph 126 k 2023-12-19T18:12:39.093 INFO:teuthology.orchestra.run.smithi192.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-12-19T18:12:39.093 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-12-19T18:12:39.093 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-12-19T18:12:39.093 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-12-19T18:12:39.093 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-12-19T18:12:39.093 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-12-19T18:12:39.093 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs x86_64 2:18.2.1-132.g568df92b.el9 ceph 161 k 2023-12-19T18:12:39.093 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados x86_64 2:18.2.1-132.g568df92b.el9 ceph 321 k 2023-12-19T18:12:39.093 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd x86_64 2:18.2.1-132.g568df92b.el9 ceph 298 k 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw x86_64 2:18.2.1-132.g568df92b.el9 ceph 99 k 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse x86_64 2:18.2.1-132.g568df92b.el9 ceph 86 k 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror x86_64 2:18.2.1-132.g568df92b.el9 ceph 3.0 M 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd x86_64 2:18.2.1-132.g568df92b.el9 ceph 171 k 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrading: 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager x86_64 1:1.45.7-1.el9 baseos 2.3 M 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm x86_64 1:1.45.7-1.el9 baseos 1.8 M 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team x86_64 1:1.45.7-1.el9 baseos 43 k 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui x86_64 1:1.45.7-1.el9 baseos 246 k 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-12-19T18:12:39.094 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: librados2 x86_64 2:18.2.1-132.g568df92b.el9 ceph 3.3 M 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1 x86_64 2:18.2.1-132.g568df92b.el9 ceph 3.0 M 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout:Installing dependencies: 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common x86_64 2:18.2.1-132.g568df92b.el9 ceph 18 M 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 23 k 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds x86_64 2:18.2.1-132.g568df92b.el9 ceph 2.1 M 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 242 k 2023-12-19T18:12:39.095 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon x86_64 2:18.2.1-132.g568df92b.el9 ceph 4.4 M 2023-12-19T18:12:39.096 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd x86_64 2:18.2.1-132.g568df92b.el9 ceph 18 M 2023-12-19T18:12:39.096 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 15 k 2023-12-19T18:12:39.096 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux x86_64 2:18.2.1-132.g568df92b.el9 ceph 24 k 2023-12-19T18:12:39.096 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-19T18:12:39.096 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-19T18:12:39.096 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-19T18:12:39.096 INFO:teuthology.orchestra.run.smithi192.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-12-19T18:12:39.096 INFO:teuthology.orchestra.run.smithi192.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-19T18:12:39.096 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-19T18:12:39.096 INFO:teuthology.orchestra.run.smithi192.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-19T18:12:39.096 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2023-12-19T18:12:39.096 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2023-12-19T18:12:39.096 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite x86_64 2:18.2.1-132.g568df92b.el9 ceph 166 k 2023-12-19T18:12:39.097 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-19T18:12:39.097 INFO:teuthology.orchestra.run.smithi192.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-19T18:12:39.097 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-19T18:12:39.097 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-19T18:12:39.097 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-19T18:12:39.097 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1 x86_64 2:18.2.1-132.g568df92b.el9 ceph 475 k 2023-12-19T18:12:39.097 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-19T18:12:39.097 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2 x86_64 2:18.2.1-132.g568df92b.el9 ceph 4.5 M 2023-12-19T18:12:39.097 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-19T18:12:39.097 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-12-19T18:12:39.097 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-19T18:12:39.097 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-19T18:12:39.097 INFO:teuthology.orchestra.run.smithi192.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse x86_64 2:18.2.1-132.g568df92b.el9 ceph 45 k 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common x86_64 2:18.2.1-132.g568df92b.el9 ceph 124 k 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-19T18:12:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-19T18:12:39.099 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-19T18:12:39.100 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-19T18:12:39.100 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-19T18:12:39.100 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-19T18:12:39.100 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-19T18:12:39.100 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-19T18:12:39.100 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-19T18:12:39.100 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-19T18:12:39.100 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-19T18:12:39.100 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-19T18:12:39.100 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-19T18:12:39.100 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-19T18:12:39.101 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-19T18:12:39.101 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-19T18:12:39.101 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-19T18:12:39.101 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-19T18:12:39.101 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-19T18:12:39.101 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-19T18:12:39.101 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2023-12-19T18:12:39.101 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-19T18:12:39.101 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-19T18:12:39.101 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-19T18:12:39.101 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-19T18:12:39.101 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-19T18:12:39.101 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-19T18:12:39.102 INFO:teuthology.orchestra.run.smithi192.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-12-19T18:12:39.103 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-19T18:12:39.103 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-12-19T18:12:39.103 INFO:teuthology.orchestra.run.smithi192.stdout:Installing weak dependencies: 2023-12-19T18:12:39.103 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-19T18:12:39.103 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:12:39.103 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction Summary 2023-12-19T18:12:39.103 INFO:teuthology.orchestra.run.smithi192.stdout:=========================================================================================== 2023-12-19T18:12:39.103 INFO:teuthology.orchestra.run.smithi192.stdout:Install 139 Packages 2023-12-19T18:12:39.103 INFO:teuthology.orchestra.run.smithi192.stdout:Upgrade 15 Packages 2023-12-19T18:12:39.103 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:12:39.104 INFO:teuthology.orchestra.run.smithi192.stdout:Total download size: 200 M 2023-12-19T18:12:39.104 INFO:teuthology.orchestra.run.smithi192.stdout:Downloading Packages: 2023-12-19T18:12:40.208 INFO:teuthology.orchestra.run.smithi192.stdout:(1/154): ceph-18.2.1-132.g568df92b.el9.x86_64.r 26 kB/s | 6.5 kB 00:00 2023-12-19T18:12:40.442 INFO:teuthology.orchestra.run.smithi192.stdout:(2/154): ceph-fuse-18.2.1-132.g568df92b.el9.x86 3.5 MB/s | 840 kB 00:00 2023-12-19T18:12:40.492 INFO:teuthology.orchestra.run.smithi192.stdout:(3/154): ceph-immutable-object-cache-18.2.1-132 2.8 MB/s | 142 kB 00:00 2023-12-19T18:12:40.789 INFO:teuthology.orchestra.run.smithi192.stdout:(4/154): ceph-mds-18.2.1-132.g568df92b.el9.x86_ 7.1 MB/s | 2.1 MB 00:00 2023-12-19T18:12:40.969 INFO:teuthology.orchestra.run.smithi192.stdout:(5/154): ceph-mgr-18.2.1-132.g568df92b.el9.x86_ 8.1 MB/s | 1.4 MB 00:00 2023-12-19T18:12:41.318 INFO:teuthology.orchestra.run.smithi192.stdout:(6/154): ceph-base-18.2.1-132.g568df92b.el9.x86 3.8 MB/s | 5.2 MB 00:01 2023-12-19T18:12:41.435 INFO:teuthology.orchestra.run.smithi192.stdout:(7/154): ceph-mon-18.2.1-132.g568df92b.el9.x86_ 9.5 MB/s | 4.4 MB 00:00 2023-12-19T18:12:42.152 INFO:teuthology.orchestra.run.smithi192.stdout:(8/154): ceph-radosgw-18.2.1-132.g568df92b.el9. 11 MB/s | 7.7 MB 00:00 2023-12-19T18:12:42.227 INFO:teuthology.orchestra.run.smithi192.stdout:(9/154): ceph-selinux-18.2.1-132.g568df92b.el9. 325 kB/s | 24 kB 00:00 2023-12-19T18:12:43.028 INFO:teuthology.orchestra.run.smithi192.stdout:(10/154): ceph-common-18.2.1-132.g568df92b.el9. 6.0 MB/s | 18 MB 00:03 2023-12-19T18:12:43.111 INFO:teuthology.orchestra.run.smithi192.stdout:(11/154): cephfs-mirror-18.2.1-132.g568df92b.el 2.6 MB/s | 219 kB 00:00 2023-12-19T18:12:43.195 INFO:teuthology.orchestra.run.smithi192.stdout:(12/154): cephfs-shell-18.2.1-132.g568df92b.el9 327 kB/s | 27 kB 00:00 2023-12-19T18:12:43.258 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2023-12-19T18:12:43.278 INFO:teuthology.orchestra.run.smithi192.stdout:(13/154): libcephfs-devel-18.2.1-132.g568df92b. 377 kB/s | 31 kB 00:00 2023-12-19T18:12:43.454 INFO:teuthology.orchestra.run.smithi192.stdout:(14/154): libcephfs2-18.2.1-132.g568df92b.el9.x 3.7 MB/s | 658 kB 00:00 2023-12-19T18:12:43.671 INFO:teuthology.orchestra.run.smithi192.stdout:(15/154): ceph-osd-18.2.1-132.g568df92b.el9.x86 7.5 MB/s | 18 MB 00:02 2023-12-19T18:12:43.704 INFO:teuthology.orchestra.run.smithi192.stdout:(16/154): libcephsqlite-18.2.1-132.g568df92b.el 666 kB/s | 166 kB 00:00 2023-12-19T18:12:43.738 INFO:teuthology.orchestra.run.smithi192.stdout:(17/154): librados-devel-18.2.1-132.g568df92b.e 1.9 MB/s | 126 kB 00:00 2023-12-19T18:12:43.788 INFO:teuthology.orchestra.run.smithi192.stdout:(18/154): libradosstriper1-18.2.1-132.g568df92b 5.6 MB/s | 475 kB 00:00 2023-12-19T18:12:43.880 INFO:teuthology.orchestra.run.smithi192.stdout:(19/154): python3-ceph-argparse-18.2.1-132.g568 496 kB/s | 45 kB 00:00 2023-12-19T18:12:43.980 INFO:teuthology.orchestra.run.smithi192.stdout:(20/154): python3-ceph-common-18.2.1-132.g568df 1.2 MB/s | 124 kB 00:00 2023-12-19T18:12:44.072 INFO:teuthology.orchestra.run.smithi192.stdout:(21/154): python3-cephfs-18.2.1-132.g568df92b.e 1.7 MB/s | 161 kB 00:00 2023-12-19T18:12:44.181 INFO:teuthology.orchestra.run.smithi192.stdout:(22/154): python3-rados-18.2.1-132.g568df92b.el 2.9 MB/s | 321 kB 00:00 2023-12-19T18:12:44.281 INFO:teuthology.orchestra.run.smithi192.stdout:(23/154): librgw2-18.2.1-132.g568df92b.el9.x86_ 8.2 MB/s | 4.5 MB 00:00 2023-12-19T18:12:44.315 INFO:teuthology.orchestra.run.smithi192.stdout:(24/154): python3-rbd-18.2.1-132.g568df92b.el9. 2.2 MB/s | 298 kB 00:00 2023-12-19T18:12:44.365 INFO:teuthology.orchestra.run.smithi192.stdout:(25/154): python3-rgw-18.2.1-132.g568df92b.el9. 1.2 MB/s | 99 kB 00:00 2023-12-19T18:12:44.390 INFO:teuthology.orchestra.run.smithi192.stdout:(26/154): rbd-fuse-18.2.1-132.g568df92b.el9.x86 1.1 MB/s | 86 kB 00:00 2023-12-19T18:12:44.491 INFO:teuthology.orchestra.run.smithi192.stdout:(27/154): rbd-nbd-18.2.1-132.g568df92b.el9.x86_ 1.7 MB/s | 171 kB 00:00 2023-12-19T18:12:44.599 INFO:teuthology.orchestra.run.smithi192.stdout:(28/154): ceph-grafana-dashboards-18.2.1-132.g5 216 kB/s | 23 kB 00:00 2023-12-19T18:12:44.683 INFO:teuthology.orchestra.run.smithi192.stdout:(29/154): ceph-mgr-cephadm-18.2.1-132.g568df92b 1.6 MB/s | 132 kB 00:00 2023-12-19T18:12:44.766 INFO:teuthology.orchestra.run.smithi192.stdout:(30/154): rbd-mirror-18.2.1-132.g568df92b.el9.x 7.5 MB/s | 3.0 MB 00:00 2023-12-19T18:12:45.208 INFO:teuthology.orchestra.run.smithi192.stdout:(31/154): ceph-test-18.2.1-132.g568df92b.el9.x8 13 MB/s | 40 MB 00:02 2023-12-19T18:12:45.292 INFO:teuthology.orchestra.run.smithi192.stdout:(32/154): ceph-mgr-dashboard-18.2.1-132.g568df9 2.9 MB/s | 1.8 MB 00:00 2023-12-19T18:12:45.326 INFO:teuthology.orchestra.run.smithi192.stdout:(33/154): ceph-mgr-modules-core-18.2.1-132.g568 2.0 MB/s | 242 kB 00:00 2023-12-19T18:12:45.359 INFO:teuthology.orchestra.run.smithi192.stdout:(34/154): ceph-mgr-rook-18.2.1-132.g568df92b.el 760 kB/s | 50 kB 00:00 2023-12-19T18:12:45.384 INFO:teuthology.orchestra.run.smithi192.stdout:(35/154): ceph-prometheus-alerts-18.2.1-132.g56 253 kB/s | 15 kB 00:00 2023-12-19T18:12:45.418 INFO:teuthology.orchestra.run.smithi192.stdout:(36/154): ceph-volume-18.2.1-132.g568df92b.el9. 4.4 MB/s | 262 kB 00:00 2023-12-19T18:12:45.452 INFO:teuthology.orchestra.run.smithi192.stdout:(37/154): cephadm-18.2.1-132.g568df92b.el9.noar 3.3 MB/s | 221 kB 00:00 2023-12-19T18:12:45.485 INFO:teuthology.orchestra.run.smithi192.stdout:(38/154): cephfs-top-18.2.1-132.g568df92b.el9.n 272 kB/s | 18 kB 00:00 2023-12-19T18:12:45.761 INFO:teuthology.orchestra.run.smithi192.stdout:(39/154): ceph-mgr-diskprediction-local-18.2.1- 7.5 MB/s | 7.4 MB 00:00 2023-12-19T18:12:45.803 INFO:teuthology.orchestra.run.smithi192.stdout:(40/154): automake-1.16.2-8.el9.noarch.rpm 1.8 MB/s | 662 kB 00:00 2023-12-19T18:12:45.845 INFO:teuthology.orchestra.run.smithi192.stdout:(41/154): bison-3.7.4-5.el9.x86_64.rpm 2.5 MB/s | 920 kB 00:00 2023-12-19T18:12:45.870 INFO:teuthology.orchestra.run.smithi192.stdout:(42/154): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 106 kB 00:00 2023-12-19T18:12:45.928 INFO:teuthology.orchestra.run.smithi192.stdout:(43/154): elfutils-libelf-devel-0.190-1.el9.x86 264 kB/s | 21 kB 00:00 2023-12-19T18:12:45.954 INFO:teuthology.orchestra.run.smithi192.stdout:(44/154): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 307 kB 00:00 2023-12-19T18:12:45.979 INFO:teuthology.orchestra.run.smithi192.stdout:(45/154): flexiblas-3.0.4-8.el9.x86_64.rpm 293 kB/s | 32 kB 00:00 2023-12-19T18:12:46.004 INFO:teuthology.orchestra.run.smithi192.stdout:(46/154): flexiblas-openblas-openmp-3.0.4-8.el9 352 kB/s | 17 kB 00:00 2023-12-19T18:12:46.011 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-19T18:12:46.013 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2023-12-19T18:12:46.030 INFO:teuthology.orchestra.run.smithi027.stdout:Package NetworkManager-1:1.44.0-2.el9.x86_64 is already installed. 2023-12-19T18:12:46.032 INFO:teuthology.orchestra.run.smithi027.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2023-12-19T18:12:46.033 INFO:teuthology.orchestra.run.smithi027.stdout:Package util-linux-2.37.4-15.el9.x86_64 is already installed. 2023-12-19T18:12:46.034 INFO:teuthology.orchestra.run.smithi027.stdout:Package libacl-devel-2.3.1-3.el9.x86_64 is already installed. 2023-12-19T18:12:46.037 INFO:teuthology.orchestra.run.smithi027.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2023-12-19T18:12:46.039 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsdump-3.1.12-3.el9.x86_64 is already installed. 2023-12-19T18:12:46.040 INFO:teuthology.orchestra.run.smithi027.stdout:Package xfsprogs-5.19.0-4.el9.x86_64 is already installed. 2023-12-19T18:12:46.104 INFO:teuthology.orchestra.run.smithi192.stdout:(47/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 20 MB/s | 3.0 MB 00:00 2023-12-19T18:12:46.126 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2023-12-19T18:12:46.139 INFO:teuthology.orchestra.run.smithi192.stdout:(48/154): jq-1.6-16.el9.x86_64.rpm 1.1 MB/s | 186 kB 00:00 2023-12-19T18:12:46.147 INFO:teuthology.orchestra.run.smithi027.stdout:=========================================================================================== 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout:=========================================================================================== 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: bison x86_64 3.7.4-5.el9 CentOS-AppStream 920 k 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:18.2.1-132.g568df92b.el9 ceph 6.5 k 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:18.2.1-132.g568df92b.el9 ceph 5.2 M 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:18.2.1-132.g568df92b.el9 ceph 840 k 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:18.2.1-132.g568df92b.el9 ceph 142 k 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:18.2.1-132.g568df92b.el9 ceph 1.4 M 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 132 k 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 1.8 M 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 7.4 M 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 50 k 2023-12-19T18:12:46.148 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:18.2.1-132.g568df92b.el9 ceph 7.7 M 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:18.2.1-132.g568df92b.el9 ceph 40 M 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 262 k 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 221 k 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror x86_64 2:18.2.1-132.g568df92b.el9 ceph 219 k 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell x86_64 2:18.2.1-132.g568df92b.el9 ceph 27 k 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 18 k 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel x86_64 0.190-1.el9 CentOS-AppStream 21 k 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: flex x86_64 2.6.4-9.el9 CentOS-AppStream 307 k 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel x86_64 0.3.111-13.el9 CentOS-AppStream 10 k 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:18.2.1-132.g568df92b.el9 ceph 31 k 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:18.2.1-132.g568df92b.el9 ceph 658 k 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:18.2.1-132.g568df92b.el9 ceph 126 k 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: libtool x86_64 2.4.6-45.el9 CentOS-AppStream 578 k 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel x86_64 2.37.4-15.el9 CentOS-AppStream 24 k 2023-12-19T18:12:46.149 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel x86_64 1:3.0.7-25.el9 appstream 4.1 M 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql x86_64 13.11-1.el9 CentOS-AppStream 1.5 M 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib x86_64 13.11-1.el9 CentOS-AppStream 811 k 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server x86_64 13.11-1.el9 CentOS-AppStream 5.7 M 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:18.2.1-132.g568df92b.el9 ceph 161 k 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:18.2.1-132.g568df92b.el9 ceph 321 k 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:18.2.1-132.g568df92b.el9 ceph 298 k 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:18.2.1-132.g568df92b.el9 ceph 99 k 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:18.2.1-132.g568df92b.el9 ceph 86 k 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:18.2.1-132.g568df92b.el9 ceph 3.0 M 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:18.2.1-132.g568df92b.el9 ceph 171 k 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel x86_64 5.19.0-4.el9 CentOS-AppStream 85 k 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager x86_64 1:1.45.7-1.el9 baseos 2.3 M 2023-12-19T18:12:46.150 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm x86_64 1:1.45.7-1.el9 baseos 1.8 M 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team x86_64 1:1.45.7-1.el9 baseos 43 k 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui x86_64 1:1.45.7-1.el9 baseos 246 k 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: acl x86_64 2.3.1-4.el9 CentOS-BaseOS 69 k 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-debuginfod-client x86_64 0.190-1.el9 CentOS-BaseOS 36 k 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf x86_64 0.190-1.el9 CentOS-BaseOS 192 k 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs x86_64 0.190-1.el9 CentOS-BaseOS 255 k 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: libacl x86_64 2.3.1-4.el9 CentOS-BaseOS 22 k 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: libacl-devel x86_64 2.3.1-4.el9 CentOS-AppStream 68 k 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:18.2.1-132.g568df92b.el9 ceph 3.3 M 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:18.2.1-132.g568df92b.el9 ceph 3.0 M 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: openssl x86_64 1:3.0.7-25.el9 baseos 1.2 M 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs x86_64 1:3.0.7-25.el9 baseos 2.2 M 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: xfsdump x86_64 3.1.12-4.el9 CentOS-BaseOS 321 k 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2023-12-19T18:12:46.151 INFO:teuthology.orchestra.run.smithi027.stdout: automake noarch 1.16.2-8.el9 CentOS-AppStream 662 k 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options x86_64 1.75.0-8.el9 CentOS-AppStream 106 k 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:18.2.1-132.g568df92b.el9 ceph 18 M 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 23 k 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:18.2.1-132.g568df92b.el9 ceph 2.1 M 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 242 k 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:18.2.1-132.g568df92b.el9 ceph 4.4 M 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:18.2.1-132.g568df92b.el9 ceph 18 M 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:18.2.1-132.g568df92b.el9 ceph-noarch 15 k 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:18.2.1-132.g568df92b.el9 ceph 24 k 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas x86_64 3.0.4-8.el9 CentOS-AppStream 32 k 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 CentOS-AppStream 3.0 M 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 CentOS-AppStream 17 k 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.9-15.el9 CentOS-BaseOS 79 k 2023-12-19T18:12:46.152 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2023-12-19T18:12:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: jq x86_64 1.6-16.el9 CentOS-AppStream 186 k 2023-12-19T18:12:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2023-12-19T18:12:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2023-12-19T18:12:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:18.2.1-132.g568df92b.el9 ceph 166 k 2023-12-19T18:12:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 11.4.1-2.3.el9 CentOS-BaseOS 803 k 2023-12-19T18:12:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2023-12-19T18:12:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.12.1-1.el9 CentOS-AppStream 159 k 2023-12-19T18:12:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 11.4.1-2.3.el9 CentOS-BaseOS 195 k 2023-12-19T18:12:46.153 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.11.0-7.el9 CentOS-AppStream 44 k 2023-12-19T18:12:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:18.2.1-132.g568df92b.el9 ceph 475 k 2023-12-19T18:12:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 1.6.1-102.el9 CentOS-AppStream 661 k 2023-12-19T18:12:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:18.2.1-132.g568df92b.el9 ceph 4.5 M 2023-12-19T18:12:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2023-12-19T18:12:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel x86_64 1.5.1-2.el9 CentOS-AppStream 46 k 2023-12-19T18:12:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.12.0-6.el9 CentOS-AppStream 282 k 2023-12-19T18:12:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma x86_64 6.9.6-1.el9.5 CentOS-AppStream 217 k 2023-12-19T18:12:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.21-2.el9 CentOS-AppStream 34 k 2023-12-19T18:12:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp x86_64 0.3.21-2.el9 CentOS-AppStream 4.7 M 2023-12-19T18:12:46.154 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs x86_64 13.11-1.el9 CentOS-AppStream 134 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh noarch 2.13.2-3.el9 epel 548 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs noarch 20.3.0-7.el9 CentOS-AppStream 78 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.9.1-2.el9 CentOS-AppStream 5.8 M 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:18.2.1-132.g568df92b.el9 ceph 45 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:18.2.1-132.g568df92b.el9 ceph 124 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.14.5-5.el9 CentOS-BaseOS 241 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2023-12-19T18:12:46.155 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 36.0.1-4.el9 CentOS-BaseOS 1.1 M 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel x86_64 3.9.17-2.el9 appstream 245 k 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:2.23.4-1.el9 epel 197 k 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.11.3-4.el9 CentOS-AppStream 228 k 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-16.el9 CentOS-AppStream 24 k 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 2.0-4.el9 CentOS-AppStream 18 k 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2023-12-19T18:12:46.156 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.1.4-6.el9 CentOS-AppStream 144 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 1.1.1-12.el9 CentOS-AppStream 32 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.20.1-5.el9 CentOS-AppStream 4.8 M 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 CentOS-AppStream 217 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 3.1.1-5.el9 CentOS-AppStream 174 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply noarch 3.11-14.el9 CentOS-BaseOS 103 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-27.el9 CentOS-AppStream 41 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.4.8-6.el9 CentOS-AppStream 132 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 CentOS-AppStream 211 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.20-6.el9 CentOS-BaseOS 124 k 2023-12-19T18:12:46.157 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2021.1-5.el9 CentOS-AppStream 47 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.25.1-7.el9 CentOS-BaseOS 113 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 CentOS-AppStream 43 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.6.2-8.el9 CentOS-AppStream 16 M 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml noarch 0.10.2-6.el9 CentOS-AppStream 39 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.26.5-4.el9 CentOS-BaseOS 187 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth noarch 0.2.5-8.el9 CentOS-AppStream 41 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2023-12-19T18:12:46.158 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.4.1-5.el9 CentOS-AppStream 300 k 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: uuid x86_64 1.6.2-55.el9 CentOS-AppStream 56 k 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-20.el9 CentOS-AppStream 63 k 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel x86_64 1.2.11-40.el9 CentOS-AppStream 44 k 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout:=========================================================================================== 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout:Install 139 Packages 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 15 Packages 2023-12-19T18:12:46.159 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:12:46.160 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 200 M 2023-12-19T18:12:46.160 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2023-12-19T18:12:46.163 INFO:teuthology.orchestra.run.smithi192.stdout:(49/154): libaio-devel-0.3.111-13.el9.x86_64.rp 70 kB/s | 10 kB 00:00 2023-12-19T18:12:46.197 INFO:teuthology.orchestra.run.smithi192.stdout:(50/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.7 MB/s | 159 kB 00:00 2023-12-19T18:12:46.239 INFO:teuthology.orchestra.run.smithi192.stdout:(51/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 606 kB/s | 44 kB 00:00 2023-12-19T18:12:46.281 INFO:teuthology.orchestra.run.smithi192.stdout:(52/154): librdkafka-1.6.1-102.el9.x86_64.rpm 5.6 MB/s | 661 kB 00:00 2023-12-19T18:12:46.323 INFO:teuthology.orchestra.run.smithi192.stdout:(53/154): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 578 kB 00:00 2023-12-19T18:12:46.348 INFO:teuthology.orchestra.run.smithi192.stdout:(54/154): libuuid-devel-2.37.4-15.el9.x86_64.rp 400 kB/s | 24 kB 00:00 2023-12-19T18:12:46.373 INFO:teuthology.orchestra.run.smithi192.stdout:(55/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 539 kB/s | 46 kB 00:00 2023-12-19T18:12:46.415 INFO:teuthology.orchestra.run.smithi192.stdout:(56/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.0 MB/s | 282 kB 00:00 2023-12-19T18:12:46.440 INFO:teuthology.orchestra.run.smithi192.stdout:(57/154): openblas-0.3.21-2.el9.x86_64.rpm 537 kB/s | 34 kB 00:00 2023-12-19T18:12:46.482 INFO:teuthology.orchestra.run.smithi192.stdout:(58/154): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.0 MB/s | 217 kB 00:00 2023-12-19T18:12:46.753 INFO:teuthology.orchestra.run.smithi192.stdout:(59/154): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2023-12-19T18:12:46.808 INFO:teuthology.orchestra.run.smithi192.stdout:(60/154): postgresql-13.11-1.el9.x86_64.rpm 4.3 MB/s | 1.5 MB 00:00 2023-12-19T18:12:46.841 INFO:teuthology.orchestra.run.smithi192.stdout:(61/154): postgresql-contrib-13.11-1.el9.x86_64 2.3 MB/s | 811 kB 00:00 2023-12-19T18:12:46.858 INFO:teuthology.orchestra.run.smithi192.stdout:(62/154): postgresql-private-libs-13.11-1.el9.x 1.3 MB/s | 134 kB 00:00 2023-12-19T18:12:46.875 INFO:teuthology.orchestra.run.smithi192.stdout:(63/154): python3-attrs-20.3.0-7.el9.noarch.rpm 2.4 MB/s | 78 kB 00:00 2023-12-19T18:12:47.034 INFO:teuthology.orchestra.run.smithi192.stdout:(64/154): postgresql-server-13.11-1.el9.x86_64. 30 MB/s | 5.7 MB 00:00 2023-12-19T18:12:47.168 INFO:teuthology.orchestra.run.smithi192.stdout:(65/154): python3-babel-2.9.1-2.el9.noarch.rpm 19 MB/s | 5.8 MB 00:00 2023-12-19T18:12:47.193 INFO:teuthology.orchestra.run.smithi192.stdout:(66/154): python3-jinja2-2.11.3-4.el9.noarch.rp 760 kB/s | 228 kB 00:00 2023-12-19T18:12:47.210 INFO:teuthology.orchestra.run.smithi192.stdout:(67/154): python3-jsonpatch-1.21-16.el9.noarch. 151 kB/s | 24 kB 00:00 2023-12-19T18:12:47.243 INFO:teuthology.orchestra.run.smithi027.stdout:(1/154): ceph-18.2.1-132.g568df92b.el9.x86_64.r 19 kB/s | 6.5 kB 00:00 2023-12-19T18:12:47.251 INFO:teuthology.orchestra.run.smithi192.stdout:(68/154): python3-mako-1.1.4-6.el9.noarch.rpm 3.5 MB/s | 144 kB 00:00 2023-12-19T18:12:47.285 INFO:teuthology.orchestra.run.smithi192.stdout:(69/154): python3-jsonpointer-2.0-4.el9.noarch. 244 kB/s | 18 kB 00:00 2023-12-19T18:12:47.311 INFO:teuthology.orchestra.run.smithi192.stdout:(70/154): python3-markupsafe-1.1.1-12.el9.x86_6 319 kB/s | 32 kB 00:00 2023-12-19T18:12:47.436 INFO:teuthology.orchestra.run.smithi192.stdout:(71/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 26 MB/s | 4.8 MB 00:00 2023-12-19T18:12:47.470 INFO:teuthology.orchestra.run.smithi192.stdout:(72/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 1.5 MB/s | 217 kB 00:00 2023-12-19T18:12:47.495 INFO:teuthology.orchestra.run.smithi192.stdout:(73/154): python3-oauthlib-3.1.1-5.el9.noarch.r 1.0 MB/s | 174 kB 00:00 2023-12-19T18:12:47.520 INFO:teuthology.orchestra.run.smithi192.stdout:(74/154): python3-prettytable-0.7.2-27.el9.noar 518 kB/s | 41 kB 00:00 2023-12-19T18:12:47.554 INFO:teuthology.orchestra.run.smithi192.stdout:(75/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 132 kB 00:00 2023-12-19T18:12:47.579 INFO:teuthology.orchestra.run.smithi192.stdout:(76/154): python3-pyasn1-modules-0.4.8-6.el9.no 2.5 MB/s | 211 kB 00:00 2023-12-19T18:12:47.604 INFO:teuthology.orchestra.run.smithi192.stdout:(77/154): python3-pytz-2021.1-5.el9.noarch.rpm 570 kB/s | 47 kB 00:00 2023-12-19T18:12:47.646 INFO:teuthology.orchestra.run.smithi192.stdout:(78/154): python3-requests-oauthlib-1.3.0-12.el 653 kB/s | 43 kB 00:00 2023-12-19T18:12:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:(2/154): ceph-fuse-18.2.1-132.g568df92b.el9.x86 2.0 MB/s | 840 kB 00:00 2023-12-19T18:12:47.735 INFO:teuthology.orchestra.run.smithi027.stdout:(3/154): ceph-immutable-object-cache-18.2.1-132 1.7 MB/s | 142 kB 00:00 2023-12-19T18:12:47.796 INFO:teuthology.orchestra.run.smithi192.stdout:(79/154): python3-toml-0.10.2-6.el9.noarch.rpm 202 kB/s | 39 kB 00:00 2023-12-19T18:12:47.862 INFO:teuthology.orchestra.run.smithi027.stdout:(4/154): ceph-base-18.2.1-132.g568df92b.el9.x86 5.4 MB/s | 5.2 MB 00:00 2023-12-19T18:12:48.014 INFO:teuthology.orchestra.run.smithi192.stdout:(80/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2023-12-19T18:12:48.063 INFO:teuthology.orchestra.run.smithi192.stdout:(81/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 102 kB/s | 41 kB 00:00 2023-12-19T18:12:48.079 INFO:teuthology.orchestra.run.smithi027.stdout:(5/154): ceph-mgr-18.2.1-132.g568df92b.el9.x86_ 6.7 MB/s | 1.4 MB 00:00 2023-12-19T18:12:48.097 INFO:teuthology.orchestra.run.smithi192.stdout:(82/154): socat-1.7.4.1-5.el9.x86_64.rpm 1.0 MB/s | 300 kB 00:00 2023-12-19T18:12:48.147 INFO:teuthology.orchestra.run.smithi192.stdout:(83/154): uuid-1.6.2-55.el9.x86_64.rpm 706 kB/s | 56 kB 00:00 2023-12-19T18:12:48.172 INFO:teuthology.orchestra.run.smithi192.stdout:(84/154): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 813 kB/s | 85 kB 00:00 2023-12-19T18:12:48.223 INFO:teuthology.orchestra.run.smithi192.stdout:(85/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 503 kB/s | 63 kB 00:00 2023-12-19T18:12:48.274 INFO:teuthology.orchestra.run.smithi192.stdout:(86/154): zlib-devel-1.2.11-40.el9.x86_64.rpm 446 kB/s | 44 kB 00:00 2023-12-19T18:12:48.290 INFO:teuthology.orchestra.run.smithi192.stdout:(87/154): fuse-2.9.9-15.el9.x86_64.rpm 681 kB/s | 79 kB 00:00 2023-12-19T18:12:48.332 INFO:teuthology.orchestra.run.smithi192.stdout:(88/154): libgfortran-11.4.1-2.3.el9.x86_64.rpm 7.3 MB/s | 803 kB 00:00 2023-12-19T18:12:48.365 INFO:teuthology.orchestra.run.smithi192.stdout:(89/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.5 MB/s | 241 kB 00:00 2023-12-19T18:12:48.388 INFO:teuthology.orchestra.run.smithi027.stdout:(6/154): ceph-mds-18.2.1-132.g568df92b.el9.x86_ 3.2 MB/s | 2.1 MB 00:00 2023-12-19T18:12:48.391 INFO:teuthology.orchestra.run.smithi192.stdout:(90/154): libquadmath-11.4.1-2.3.el9.x86_64.rpm 2.1 MB/s | 195 kB 00:00 2023-12-19T18:12:48.433 INFO:teuthology.orchestra.run.smithi192.stdout:(91/154): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.1 MB 00:00 2023-12-19T18:12:48.458 INFO:teuthology.orchestra.run.smithi192.stdout:(92/154): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 103 kB 00:00 2023-12-19T18:12:48.483 INFO:teuthology.orchestra.run.smithi192.stdout:(93/154): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 124 kB 00:00 2023-12-19T18:12:48.508 INFO:teuthology.orchestra.run.smithi192.stdout:(94/154): python3-requests-2.25.1-7.el9.noarch. 1.5 MB/s | 113 kB 00:00 2023-12-19T18:12:48.542 INFO:teuthology.orchestra.run.smithi192.stdout:(95/154): python3-urllib3-1.26.5-4.el9.noarch.r 3.2 MB/s | 187 kB 00:00 2023-12-19T18:12:48.605 INFO:teuthology.orchestra.run.smithi027.stdout:(7/154): ceph-mon-18.2.1-132.g568df92b.el9.x86_ 8.4 MB/s | 4.4 MB 00:00 2023-12-19T18:12:48.742 INFO:teuthology.orchestra.run.smithi192.stdout:(96/154): python3-devel-3.9.17-2.el9.x86_64.rpm 1.0 MB/s | 245 kB 00:00 2023-12-19T18:12:48.793 INFO:teuthology.orchestra.run.smithi192.stdout:(97/154): fmt-8.1.1-5.el9.x86_64.rpm 444 kB/s | 111 kB 00:00 2023-12-19T18:12:48.864 INFO:teuthology.orchestra.run.smithi027.stdout:(8/154): ceph-common-18.2.1-132.g568df92b.el9.x 9.4 MB/s | 18 MB 00:01 2023-12-19T18:12:48.876 INFO:teuthology.orchestra.run.smithi192.stdout:(98/154): gperftools-libs-2.9.1-2.el9.x86_64.rp 2.3 MB/s | 309 kB 00:00 2023-12-19T18:12:48.930 INFO:teuthology.orchestra.run.smithi027.stdout:(9/154): ceph-selinux-18.2.1-132.g568df92b.el9. 368 kB/s | 24 kB 00:00 2023-12-19T18:12:48.985 INFO:teuthology.orchestra.run.smithi192.stdout:(99/154): openssl-devel-3.0.7-25.el9.x86_64.rpm 8.3 MB/s | 4.1 MB 00:00 2023-12-19T18:12:49.222 INFO:teuthology.orchestra.run.smithi027.stdout:(10/154): ceph-radosgw-18.2.1-132.g568df92b.el9 13 MB/s | 7.7 MB 00:00 2023-12-19T18:12:49.306 INFO:teuthology.orchestra.run.smithi027.stdout:(11/154): cephfs-mirror-18.2.1-132.g568df92b.el 2.6 MB/s | 219 kB 00:00 2023-12-19T18:12:49.373 INFO:teuthology.orchestra.run.smithi027.stdout:(12/154): cephfs-shell-18.2.1-132.g568df92b.el9 410 kB/s | 27 kB 00:00 2023-12-19T18:12:49.440 INFO:teuthology.orchestra.run.smithi027.stdout:(13/154): libcephfs-devel-18.2.1-132.g568df92b. 473 kB/s | 31 kB 00:00 2023-12-19T18:12:49.444 INFO:teuthology.orchestra.run.smithi192.stdout:(100/154): libarrow-9.0.0-10.el9.x86_64.rpm 6.8 MB/s | 4.4 MB 00:00 2023-12-19T18:12:49.486 INFO:teuthology.orchestra.run.smithi192.stdout:(101/154): libarrow-doc-9.0.0-10.el9.noarch.rpm 43 kB/s | 26 kB 00:00 2023-12-19T18:12:49.527 INFO:teuthology.orchestra.run.smithi192.stdout:(102/154): liboath-2.6.7-2.el9.x86_64.rpm 90 kB/s | 49 kB 00:00 2023-12-19T18:12:49.553 INFO:teuthology.orchestra.run.smithi192.stdout:(103/154): libunwind-1.6.2-1.el9.x86_64.rpm 629 kB/s | 67 kB 00:00 2023-12-19T18:12:49.557 INFO:teuthology.orchestra.run.smithi027.stdout:(14/154): libcephfs2-18.2.1-132.g568df92b.el9.x 5.6 MB/s | 658 kB 00:00 2023-12-19T18:12:49.649 INFO:teuthology.orchestra.run.smithi027.stdout:(15/154): libcephsqlite-18.2.1-132.g568df92b.el 1.8 MB/s | 166 kB 00:00 2023-12-19T18:12:49.695 INFO:teuthology.orchestra.run.smithi192.stdout:(104/154): parquet-libs-9.0.0-10.el9.x86_64.rpm 5.0 MB/s | 838 kB 00:00 2023-12-19T18:12:49.741 INFO:teuthology.orchestra.run.smithi027.stdout:(16/154): librados-devel-18.2.1-132.g568df92b.e 1.4 MB/s | 126 kB 00:00 2023-12-19T18:12:49.787 INFO:teuthology.orchestra.run.smithi192.stdout:(105/154): python3-asyncssh-2.13.2-3.el9.noarch 2.1 MB/s | 548 kB 00:00 2023-12-19T18:12:49.829 INFO:teuthology.orchestra.run.smithi192.stdout:(106/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 158 kB/s | 43 kB 00:00 2023-12-19T18:12:49.858 INFO:teuthology.orchestra.run.smithi027.stdout:(17/154): libradosstriper1-18.2.1-132.g568df92b 4.0 MB/s | 475 kB 00:00 2023-12-19T18:12:49.870 INFO:teuthology.orchestra.run.smithi192.stdout:(107/154): python3-cachetools-4.2.4-1.el9.noarc 187 kB/s | 32 kB 00:00 2023-12-19T18:12:49.912 INFO:teuthology.orchestra.run.smithi192.stdout:(108/154): python3-certifi-2021.10.8-2.el9.noar 182 kB/s | 15 kB 00:00 2023-12-19T18:12:49.938 INFO:teuthology.orchestra.run.smithi192.stdout:(109/154): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2023-12-19T18:12:49.971 INFO:teuthology.orchestra.run.smithi192.stdout:(110/154): python3-cherrypy-18.6.1-2.el9.noarch 3.6 MB/s | 358 kB 00:00 2023-12-19T18:12:50.029 INFO:teuthology.orchestra.run.smithi192.stdout:(111/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.4 MB/s | 303 kB 00:00 2023-12-19T18:12:50.072 INFO:teuthology.orchestra.run.smithi192.stdout:(112/154): python3-colorama-0.4.4-12.el9.noarch 272 kB/s | 35 kB 00:00 2023-12-19T18:12:50.105 INFO:teuthology.orchestra.run.smithi192.stdout:(113/154): python3-google-auth-2.23.4-1.el9.noa 1.5 MB/s | 197 kB 00:00 2023-12-19T18:12:50.147 INFO:teuthology.orchestra.run.smithi192.stdout:(114/154): python3-jaraco-8.2.1-3.el9.noarch.rp 147 kB/s | 11 kB 00:00 2023-12-19T18:12:50.189 INFO:teuthology.orchestra.run.smithi192.stdout:(115/154): python3-jaraco-classes-3.2.1-5.el9.n 155 kB/s | 18 kB 00:00 2023-12-19T18:12:50.231 INFO:teuthology.orchestra.run.smithi192.stdout:(116/154): python3-jaraco-collections-3.0.0-8.e 186 kB/s | 23 kB 00:00 2023-12-19T18:12:50.273 INFO:teuthology.orchestra.run.smithi192.stdout:(117/154): python3-jaraco-functools-3.5.0-2.el9 236 kB/s | 19 kB 00:00 2023-12-19T18:12:50.315 INFO:teuthology.orchestra.run.smithi192.stdout:(118/154): python3-jaraco-text-3.2.0-6.el9.noar 159 kB/s | 20 kB 00:00 2023-12-19T18:12:50.357 INFO:teuthology.orchestra.run.smithi192.stdout:(119/154): python3-jwt+crypto-2.4.0-1.el9.noarc 73 kB/s | 9.0 kB 00:00 2023-12-19T18:12:50.407 INFO:teuthology.orchestra.run.smithi192.stdout:(120/154): python3-jwt-2.4.0-1.el9.noarch.rpm 451 kB/s | 41 kB 00:00 2023-12-19T18:12:50.474 INFO:teuthology.orchestra.run.smithi192.stdout:(121/154): python3-kubernetes-26.1.0-2.el9.noar 6.5 MB/s | 1.0 MB 00:00 2023-12-19T18:12:50.515 INFO:teuthology.orchestra.run.smithi192.stdout:(122/154): python3-logutils-0.3.5-21.el9.noarch 302 kB/s | 46 kB 00:00 2023-12-19T18:12:50.533 INFO:teuthology.orchestra.run.smithi027.stdout:(18/154): librgw2-18.2.1-132.g568df92b.el9.x86_ 6.6 MB/s | 4.5 MB 00:00 2023-12-19T18:12:50.541 INFO:teuthology.orchestra.run.smithi192.stdout:(123/154): python3-more-itertools-8.12.0-2.el9. 612 kB/s | 79 kB 00:00 2023-12-19T18:12:50.583 INFO:teuthology.orchestra.run.smithi192.stdout:(124/154): python3-natsort-7.1.1-5.el9.noarch.r 884 kB/s | 58 kB 00:00 2023-12-19T18:12:50.608 INFO:teuthology.orchestra.run.smithi192.stdout:(125/154): python3-pecan-1.4.2-3.el9.noarch.rpm 2.9 MB/s | 272 kB 00:00 2023-12-19T18:12:50.625 INFO:teuthology.orchestra.run.smithi027.stdout:(19/154): python3-ceph-argparse-18.2.1-132.g568 497 kB/s | 45 kB 00:00 2023-12-19T18:12:50.650 INFO:teuthology.orchestra.run.smithi192.stdout:(126/154): python3-portend-3.1.0-2.el9.noarch.r 156 kB/s | 16 kB 00:00 2023-12-19T18:12:50.667 INFO:teuthology.orchestra.run.smithi192.stdout:(127/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2023-12-19T18:12:50.709 INFO:teuthology.orchestra.run.smithi027.stdout:(20/154): python3-ceph-common-18.2.1-132.g568df 1.5 MB/s | 124 kB 00:00 2023-12-19T18:12:50.709 INFO:teuthology.orchestra.run.smithi192.stdout:(128/154): python3-pyperclip-1.8.2-2.el9.noarch 285 kB/s | 28 kB 00:00 2023-12-19T18:12:50.751 INFO:teuthology.orchestra.run.smithi192.stdout:(129/154): python3-repoze-lru-0.7-16.el9.noarch 312 kB/s | 31 kB 00:00 2023-12-19T18:12:50.784 INFO:teuthology.orchestra.run.smithi192.stdout:(130/154): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2023-12-19T18:12:50.800 INFO:teuthology.orchestra.run.smithi027.stdout:(21/154): python3-cephfs-18.2.1-132.g568df92b.e 1.7 MB/s | 161 kB 00:00 2023-12-19T18:12:50.834 INFO:teuthology.orchestra.run.smithi192.stdout:(131/154): python3-rsa-4.9-2.el9.noarch.rpm 477 kB/s | 59 kB 00:00 2023-12-19T18:12:50.877 INFO:teuthology.orchestra.run.smithi192.stdout:(132/154): python3-tempora-5.0.0-2.el9.noarch.r 292 kB/s | 36 kB 00:00 2023-12-19T18:12:50.916 INFO:teuthology.orchestra.run.smithi027.stdout:(22/154): python3-rados-18.2.1-132.g568df92b.el 2.7 MB/s | 321 kB 00:00 2023-12-19T18:12:50.919 INFO:teuthology.orchestra.run.smithi192.stdout:(133/154): python3-typing-extensions-4.4.0-2.el 619 kB/s | 51 kB 00:00 2023-12-19T18:12:50.944 INFO:teuthology.orchestra.run.smithi192.stdout:(134/154): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2023-12-19T18:12:50.969 INFO:teuthology.orchestra.run.smithi192.stdout:(135/154): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2023-12-19T18:12:51.018 INFO:teuthology.orchestra.run.smithi027.stdout:(23/154): python3-rbd-18.2.1-132.g568df92b.el9. 2.9 MB/s | 298 kB 00:00 2023-12-19T18:12:51.024 INFO:teuthology.orchestra.run.smithi192.stdout:(136/154): python3-werkzeug-2.0.3-3.el9.1.noarc 5.3 MB/s | 427 kB 00:00 2023-12-19T18:12:51.070 INFO:teuthology.orchestra.run.smithi192.stdout:(137/154): python3-zc-lockfile-2.0-10.el9.noarc 161 kB/s | 20 kB 00:00 2023-12-19T18:12:51.101 INFO:teuthology.orchestra.run.smithi027.stdout:(24/154): python3-rgw-18.2.1-132.g568df92b.el9. 1.2 MB/s | 99 kB 00:00 2023-12-19T18:12:51.111 INFO:teuthology.orchestra.run.smithi192.stdout:(138/154): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2023-12-19T18:12:51.229 INFO:teuthology.orchestra.run.smithi192.stdout:(139/154): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2023-12-19T18:12:51.271 INFO:teuthology.orchestra.run.smithi192.stdout:(140/154): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2023-12-19T18:12:51.304 INFO:teuthology.orchestra.run.smithi192.stdout:(141/154): acl-2.3.1-4.el9.x86_64.rpm 2.1 MB/s | 69 kB 00:00 2023-12-19T18:12:51.327 INFO:teuthology.orchestra.run.smithi027.stdout:(25/154): ceph-osd-18.2.1-132.g568df92b.el9.x86 6.0 MB/s | 18 MB 00:02 2023-12-19T18:12:51.360 INFO:teuthology.orchestra.run.smithi027.stdout:(26/154): rbd-fuse-18.2.1-132.g568df92b.el9.x86 334 kB/s | 86 kB 00:00 2023-12-19T18:12:51.363 INFO:teuthology.orchestra.run.smithi192.stdout:(142/154): elfutils-debuginfod-client-0.190-1.e 634 kB/s | 36 kB 00:00 2023-12-19T18:12:51.396 INFO:teuthology.orchestra.run.smithi192.stdout:(143/154): elfutils-libelf-0.190-1.el9.x86_64.r 5.8 MB/s | 192 kB 00:00 2023-12-19T18:12:51.438 INFO:teuthology.orchestra.run.smithi192.stdout:(144/154): elfutils-libs-0.190-1.el9.x86_64.rpm 6.1 MB/s | 255 kB 00:00 2023-12-19T18:12:51.497 INFO:teuthology.orchestra.run.smithi192.stdout:(145/154): libacl-2.3.1-4.el9.x86_64.rpm 378 kB/s | 22 kB 00:00 2023-12-19T18:12:51.502 INFO:teuthology.orchestra.run.smithi027.stdout:(27/154): rbd-nbd-18.2.1-132.g568df92b.el9.x86_ 1.2 MB/s | 171 kB 00:00 2023-12-19T18:12:51.561 INFO:teuthology.orchestra.run.smithi027.stdout:(28/154): ceph-grafana-dashboards-18.2.1-132.g5 404 kB/s | 23 kB 00:00 2023-12-19T18:12:51.572 INFO:teuthology.orchestra.run.smithi192.stdout:(146/154): librados2-18.2.1-132.g568df92b.el9.x 6.6 MB/s | 3.3 MB 00:00 2023-12-19T18:12:51.606 INFO:teuthology.orchestra.run.smithi192.stdout:(147/154): xfsdump-3.1.12-4.el9.x86_64.rpm 2.9 MB/s | 321 kB 00:00 2023-12-19T18:12:51.781 INFO:teuthology.orchestra.run.smithi192.stdout:(148/154): librbd1-18.2.1-132.g568df92b.el9.x86 4.5 MB/s | 3.0 MB 00:00 2023-12-19T18:12:51.923 INFO:teuthology.orchestra.run.smithi192.stdout:(149/154): NetworkManager-team-1.45.7-1.el9.x86 306 kB/s | 43 kB 00:00 2023-12-19T18:12:52.007 INFO:teuthology.orchestra.run.smithi192.stdout:(150/154): NetworkManager-libnm-1.45.7-1.el9.x8 4.4 MB/s | 1.8 MB 00:00 2023-12-19T18:12:52.011 INFO:teuthology.orchestra.run.smithi027.stdout:(29/154): ceph-test-18.2.1-132.g568df92b.el9.x8 13 MB/s | 40 MB 00:03 2023-12-19T18:12:52.045 INFO:teuthology.orchestra.run.smithi027.stdout:(30/154): ceph-mgr-cephadm-18.2.1-132.g568df92b 274 kB/s | 132 kB 00:00 2023-12-19T18:12:52.090 INFO:teuthology.orchestra.run.smithi192.stdout:(151/154): NetworkManager-1.45.7-1.el9.x86_64.r 4.4 MB/s | 2.3 MB 00:00 2023-12-19T18:12:52.124 INFO:teuthology.orchestra.run.smithi192.stdout:(152/154): NetworkManager-tui-1.45.7-1.el9.x86_ 1.2 MB/s | 246 kB 00:00 2023-12-19T18:12:52.146 INFO:teuthology.orchestra.run.smithi027.stdout:(31/154): rbd-mirror-18.2.1-132.g568df92b.el9.x 3.7 MB/s | 3.0 MB 00:00 2023-12-19T18:12:52.199 INFO:teuthology.orchestra.run.smithi192.stdout:(153/154): openssl-3.0.7-25.el9.x86_64.rpm 6.3 MB/s | 1.2 MB 00:00 2023-12-19T18:12:52.229 INFO:teuthology.orchestra.run.smithi027.stdout:(32/154): ceph-mgr-dashboard-18.2.1-132.g568df9 8.1 MB/s | 1.8 MB 00:00 2023-12-19T18:12:52.263 INFO:teuthology.orchestra.run.smithi027.stdout:(33/154): ceph-mgr-modules-core-18.2.1-132.g568 2.0 MB/s | 242 kB 00:00 2023-12-19T18:12:52.274 INFO:teuthology.orchestra.run.smithi192.stdout:(154/154): openssl-libs-3.0.7-25.el9.x86_64.rpm 12 MB/s | 2.2 MB 00:00 2023-12-19T18:12:52.279 INFO:teuthology.orchestra.run.smithi027.stdout:(34/154): ceph-mgr-rook-18.2.1-132.g568df92b.el 1.0 MB/s | 50 kB 00:00 2023-12-19T18:12:52.285 INFO:teuthology.orchestra.run.smithi192.stdout:-------------------------------------------------------------------------------- 2023-12-19T18:12:52.285 INFO:teuthology.orchestra.run.smithi192.stdout:Total 15 MB/s | 200 MB 00:13 2023-12-19T18:12:52.347 INFO:teuthology.orchestra.run.smithi027.stdout:(35/154): ceph-volume-18.2.1-132.g568df92b.el9. 3.9 MB/s | 262 kB 00:00 2023-12-19T18:12:52.363 INFO:teuthology.orchestra.run.smithi027.stdout:(36/154): ceph-prometheus-alerts-18.2.1-132.g56 147 kB/s | 15 kB 00:00 2023-12-19T18:12:52.457 INFO:teuthology.orchestra.run.smithi027.stdout:(37/154): cephfs-top-18.2.1-132.g568df92b.el9.n 196 kB/s | 18 kB 00:00 2023-12-19T18:12:52.497 INFO:teuthology.orchestra.run.smithi027.stdout:(38/154): cephadm-18.2.1-132.g568df92b.el9.noar 1.4 MB/s | 221 kB 00:00 2023-12-19T18:12:52.703 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction check 2023-12-19T18:12:52.748 INFO:teuthology.orchestra.run.smithi027.stdout:(39/154): ceph-mgr-diskprediction-local-18.2.1- 11 MB/s | 7.4 MB 00:00 2023-12-19T18:12:52.806 INFO:teuthology.orchestra.run.smithi027.stdout:(40/154): automake-1.16.2-8.el9.noarch.rpm 1.9 MB/s | 662 kB 00:00 2023-12-19T18:12:52.840 INFO:teuthology.orchestra.run.smithi027.stdout:(41/154): boost-program-options-1.75.0-8.el9.x8 1.2 MB/s | 106 kB 00:00 2023-12-19T18:12:52.865 INFO:teuthology.orchestra.run.smithi027.stdout:(42/154): elfutils-libelf-devel-0.190-1.el9.x86 389 kB/s | 21 kB 00:00 2023-12-19T18:12:52.932 INFO:teuthology.orchestra.run.smithi027.stdout:(43/154): flex-2.6.4-9.el9.x86_64.rpm 3.4 MB/s | 307 kB 00:00 2023-12-19T18:12:52.982 INFO:teuthology.orchestra.run.smithi027.stdout:(44/154): bison-3.7.4-5.el9.x86_64.rpm 2.2 MB/s | 920 kB 00:00 2023-12-19T18:12:52.999 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction check succeeded. 2023-12-19T18:12:52.999 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction test 2023-12-19T18:12:53.007 INFO:teuthology.orchestra.run.smithi027.stdout:(45/154): flexiblas-3.0.4-8.el9.x86_64.rpm 232 kB/s | 32 kB 00:00 2023-12-19T18:12:53.158 INFO:teuthology.orchestra.run.smithi027.stdout:(46/154): flexiblas-netlib-3.0.4-8.el9.x86_64.r 13 MB/s | 3.0 MB 00:00 2023-12-19T18:12:53.183 INFO:teuthology.orchestra.run.smithi027.stdout:(47/154): flexiblas-openblas-openmp-3.0.4-8.el9 107 kB/s | 17 kB 00:00 2023-12-19T18:12:53.208 INFO:teuthology.orchestra.run.smithi027.stdout:(48/154): jq-1.6-16.el9.x86_64.rpm 1.0 MB/s | 186 kB 00:00 2023-12-19T18:12:53.234 INFO:teuthology.orchestra.run.smithi027.stdout:(49/154): libaio-devel-0.3.111-13.el9.x86_64.rp 135 kB/s | 10 kB 00:00 2023-12-19T18:12:53.259 INFO:teuthology.orchestra.run.smithi027.stdout:(50/154): librabbitmq-0.11.0-7.el9.x86_64.rpm 908 kB/s | 44 kB 00:00 2023-12-19T18:12:53.292 INFO:teuthology.orchestra.run.smithi027.stdout:(51/154): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 159 kB 00:00 2023-12-19T18:12:53.326 INFO:teuthology.orchestra.run.smithi027.stdout:(52/154): librdkafka-1.6.1-102.el9.x86_64.rpm 7.1 MB/s | 661 kB 00:00 2023-12-19T18:12:53.376 INFO:teuthology.orchestra.run.smithi027.stdout:(53/154): libtool-2.4.6-45.el9.x86_64.rpm 5.2 MB/s | 578 kB 00:00 2023-12-19T18:12:53.427 INFO:teuthology.orchestra.run.smithi027.stdout:(54/154): libuuid-devel-2.37.4-15.el9.x86_64.rp 260 kB/s | 24 kB 00:00 2023-12-19T18:12:53.452 INFO:teuthology.orchestra.run.smithi027.stdout:(55/154): libzstd-devel-1.5.1-2.el9.x86_64.rpm 390 kB/s | 46 kB 00:00 2023-12-19T18:12:53.485 INFO:teuthology.orchestra.run.smithi027.stdout:(56/154): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 282 kB 00:00 2023-12-19T18:12:53.511 INFO:teuthology.orchestra.run.smithi027.stdout:(57/154): openblas-0.3.21-2.el9.x86_64.rpm 625 kB/s | 34 kB 00:00 2023-12-19T18:12:53.553 INFO:teuthology.orchestra.run.smithi027.stdout:(58/154): oniguruma-6.9.6-1.el9.5.x86_64.rpm 2.2 MB/s | 217 kB 00:00 2023-12-19T18:12:53.711 INFO:teuthology.orchestra.run.smithi027.stdout:(59/154): openblas-openmp-0.3.21-2.el9.x86_64.r 21 MB/s | 4.7 MB 00:00 2023-12-19T18:12:53.787 INFO:teuthology.orchestra.run.smithi027.stdout:(60/154): postgresql-13.11-1.el9.x86_64.rpm 5.8 MB/s | 1.5 MB 00:00 2023-12-19T18:12:53.820 INFO:teuthology.orchestra.run.smithi027.stdout:(61/154): postgresql-contrib-13.11-1.el9.x86_64 3.2 MB/s | 811 kB 00:00 2023-12-19T18:12:53.881 INFO:teuthology.orchestra.run.smithi027.stdout:(62/154): postgresql-private-libs-13.11-1.el9.x 907 kB/s | 134 kB 00:00 2023-12-19T18:12:53.913 INFO:teuthology.orchestra.run.smithi027.stdout:(63/154): python3-attrs-20.3.0-7.el9.noarch.rpm 867 kB/s | 78 kB 00:00 2023-12-19T18:12:54.071 INFO:teuthology.orchestra.run.smithi027.stdout:(64/154): postgresql-server-13.11-1.el9.x86_64. 23 MB/s | 5.7 MB 00:00 2023-12-19T18:12:54.205 INFO:teuthology.orchestra.run.smithi027.stdout:(65/154): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 5.8 MB 00:00 2023-12-19T18:12:54.238 INFO:teuthology.orchestra.run.smithi027.stdout:(66/154): python3-jinja2-2.11.3-4.el9.noarch.rp 736 kB/s | 228 kB 00:00 2023-12-19T18:12:54.263 INFO:teuthology.orchestra.run.smithi027.stdout:(67/154): python3-jsonpatch-1.21-16.el9.noarch. 138 kB/s | 24 kB 00:00 2023-12-19T18:12:54.305 INFO:teuthology.orchestra.run.smithi027.stdout:(68/154): python3-jsonpointer-2.0-4.el9.noarch. 450 kB/s | 18 kB 00:00 2023-12-19T18:12:54.331 INFO:teuthology.orchestra.run.smithi027.stdout:(69/154): python3-markupsafe-1.1.1-12.el9.x86_6 487 kB/s | 32 kB 00:00 2023-12-19T18:12:54.398 INFO:teuthology.orchestra.run.smithi027.stdout:(70/154): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 144 kB 00:00 2023-12-19T18:12:54.431 INFO:teuthology.orchestra.run.smithi027.stdout:(71/154): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.2 MB/s | 217 kB 00:00 2023-12-19T18:12:54.557 INFO:teuthology.orchestra.run.smithi027.stdout:(72/154): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 4.8 MB 00:00 2023-12-19T18:12:54.590 INFO:teuthology.orchestra.run.smithi027.stdout:(73/154): python3-oauthlib-3.1.1-5.el9.noarch.r 915 kB/s | 174 kB 00:00 2023-12-19T18:12:54.607 INFO:teuthology.orchestra.run.smithi027.stdout:(74/154): python3-prettytable-0.7.2-27.el9.noar 258 kB/s | 41 kB 00:00 2023-12-19T18:12:54.649 INFO:teuthology.orchestra.run.smithi027.stdout:(75/154): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 211 kB 00:00 2023-12-19T18:12:54.666 INFO:teuthology.orchestra.run.smithi027.stdout:(76/154): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 132 kB 00:00 2023-12-19T18:12:54.691 INFO:teuthology.orchestra.run.smithi027.stdout:(77/154): python3-pytz-2021.1-5.el9.noarch.rpm 569 kB/s | 47 kB 00:00 2023-12-19T18:12:54.716 INFO:teuthology.orchestra.run.smithi027.stdout:(78/154): python3-requests-oauthlib-1.3.0-12.el 658 kB/s | 43 kB 00:00 2023-12-19T18:12:54.767 INFO:teuthology.orchestra.run.smithi027.stdout:(79/154): python3-toml-0.10.2-6.el9.noarch.rpm 526 kB/s | 39 kB 00:00 2023-12-19T18:12:55.081 INFO:teuthology.orchestra.run.smithi192.stdout:Transaction test succeeded. 2023-12-19T18:12:55.082 INFO:teuthology.orchestra.run.smithi192.stdout:Running transaction 2023-12-19T18:12:55.209 INFO:teuthology.orchestra.run.smithi027.stdout:(80/154): python3-wcwidth-0.2.5-8.el9.noarch.rp 83 kB/s | 41 kB 00:00 2023-12-19T18:12:55.593 INFO:teuthology.orchestra.run.smithi027.stdout:(81/154): python3-scipy-1.6.2-8.el9.x86_64.rpm 17 MB/s | 16 MB 00:00 2023-12-19T18:12:55.626 INFO:teuthology.orchestra.run.smithi027.stdout:(82/154): socat-1.7.4.1-5.el9.x86_64.rpm 352 kB/s | 300 kB 00:00 2023-12-19T18:12:55.651 INFO:teuthology.orchestra.run.smithi027.stdout:(83/154): uuid-1.6.2-55.el9.x86_64.rpm 132 kB/s | 56 kB 00:00 2023-12-19T18:12:55.735 INFO:teuthology.orchestra.run.smithi027.stdout:(84/154): xfsprogs-devel-5.19.0-4.el9.x86_64.rp 1.0 MB/s | 85 kB 00:00 2023-12-19T18:12:55.760 INFO:teuthology.orchestra.run.smithi027.stdout:(85/154): xmlstarlet-1.6.1-20.el9.x86_64.rpm 584 kB/s | 63 kB 00:00 2023-12-19T18:12:55.818 INFO:teuthology.orchestra.run.smithi027.stdout:(86/154): zlib-devel-1.2.11-40.el9.x86_64.rpm 267 kB/s | 44 kB 00:00 2023-12-19T18:12:55.852 INFO:teuthology.orchestra.run.smithi027.stdout:(87/154): fuse-2.9.9-15.el9.x86_64.rpm 676 kB/s | 79 kB 00:00 2023-12-19T18:12:55.919 INFO:teuthology.orchestra.run.smithi027.stdout:(88/154): libquadmath-11.4.1-2.3.el9.x86_64.rpm 1.9 MB/s | 195 kB 00:00 2023-12-19T18:12:55.969 INFO:teuthology.orchestra.run.smithi027.stdout:(89/154): libgfortran-11.4.1-2.3.el9.x86_64.rpm 5.3 MB/s | 803 kB 00:00 2023-12-19T18:12:55.994 INFO:teuthology.orchestra.run.smithi027.stdout:(90/154): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.7 MB/s | 241 kB 00:00 2023-12-19T18:12:56.086 INFO:teuthology.orchestra.run.smithi027.stdout:(91/154): python3-cryptography-36.0.1-4.el9.x86 7.0 MB/s | 1.1 MB 00:00 2023-12-19T18:12:56.120 INFO:teuthology.orchestra.run.smithi027.stdout:(92/154): python3-ply-3.11-14.el9.noarch.rpm 909 kB/s | 103 kB 00:00 2023-12-19T18:12:56.145 INFO:teuthology.orchestra.run.smithi027.stdout:(93/154): python3-pycparser-2.20-6.el9.noarch.r 899 kB/s | 124 kB 00:00 2023-12-19T18:12:56.171 INFO:teuthology.orchestra.run.smithi027.stdout:(94/154): python3-requests-2.25.1-7.el9.noarch. 1.3 MB/s | 113 kB 00:00 2023-12-19T18:12:56.229 INFO:teuthology.orchestra.run.smithi027.stdout:(95/154): python3-urllib3-1.26.5-4.el9.noarch.r 2.2 MB/s | 187 kB 00:00 2023-12-19T18:12:56.538 INFO:teuthology.orchestra.run.smithi027.stdout:(96/154): python3-devel-3.9.17-2.el9.x86_64.rpm 670 kB/s | 245 kB 00:00 2023-12-19T18:12:56.838 INFO:teuthology.orchestra.run.smithi027.stdout:(97/154): fmt-8.1.1-5.el9.x86_64.rpm 182 kB/s | 111 kB 00:00 2023-12-19T18:12:57.269 INFO:teuthology.orchestra.run.smithi027.stdout:(98/154): gperftools-libs-2.9.1-2.el9.x86_64.rp 423 kB/s | 309 kB 00:00 2023-12-19T18:12:57.397 INFO:teuthology.orchestra.run.smithi027.stdout:(99/154): libarrow-doc-9.0.0-10.el9.noarch.rpm 205 kB/s | 26 kB 00:00 2023-12-19T18:12:57.539 INFO:teuthology.orchestra.run.smithi027.stdout:(100/154): libarrow-9.0.0-10.el9.x86_64.rpm 6.3 MB/s | 4.4 MB 00:00 2023-12-19T18:12:57.565 INFO:teuthology.orchestra.run.smithi027.stdout:(101/154): liboath-2.6.7-2.el9.x86_64.rpm 292 kB/s | 49 kB 00:00 2023-12-19T18:12:57.641 INFO:teuthology.orchestra.run.smithi027.stdout:(102/154): libunwind-1.6.2-1.el9.x86_64.rpm 679 kB/s | 67 kB 00:00 2023-12-19T18:12:57.781 INFO:teuthology.orchestra.run.smithi027.stdout:(103/154): python3-asyncssh-2.13.2-3.el9.noarch 3.8 MB/s | 548 kB 00:00 2023-12-19T18:12:57.850 INFO:teuthology.orchestra.run.smithi027.stdout:(104/154): parquet-libs-9.0.0-10.el9.x86_64.rpm 2.9 MB/s | 838 kB 00:00 2023-12-19T18:12:57.907 INFO:teuthology.orchestra.run.smithi027.stdout:(105/154): python3-bcrypt-3.2.2-1.el9.x86_64.rp 349 kB/s | 43 kB 00:00 2023-12-19T18:12:57.966 INFO:teuthology.orchestra.run.smithi027.stdout:(106/154): python3-cachetools-4.2.4-1.el9.noarc 277 kB/s | 32 kB 00:00 2023-12-19T18:12:58.033 INFO:teuthology.orchestra.run.smithi027.stdout:(107/154): python3-certifi-2021.10.8-2.el9.noar 120 kB/s | 15 kB 00:00 2023-12-19T18:12:58.075 INFO:teuthology.orchestra.run.smithi027.stdout:(108/154): python3-cheroot-8.6.0-4.el9.noarch.r 1.6 MB/s | 172 kB 00:00 2023-12-19T18:12:58.158 INFO:teuthology.orchestra.run.smithi027.stdout:(109/154): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2023-12-19T18:12:58.242 INFO:teuthology.orchestra.run.smithi027.stdout:(110/154): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.8 MB/s | 303 kB 00:00 2023-12-19T18:12:58.267 INFO:teuthology.orchestra.run.smithi027.stdout:(111/154): python3-colorama-0.4.4-12.el9.noarch 327 kB/s | 35 kB 00:00 2023-12-19T18:12:58.359 INFO:teuthology.orchestra.run.smithi027.stdout:(112/154): python3-google-auth-2.23.4-1.el9.noa 1.7 MB/s | 197 kB 00:00 2023-12-19T18:12:58.376 INFO:teuthology.orchestra.run.smithi027.stdout:(113/154): python3-jaraco-8.2.1-3.el9.noarch.rp 99 kB/s | 11 kB 00:00 2023-12-19T18:12:58.443 INFO:teuthology.orchestra.run.smithi027.stdout:(114/154): python3-jaraco-classes-3.2.1-5.el9.n 213 kB/s | 18 kB 00:00 2023-12-19T18:12:58.471 INFO:teuthology.orchestra.run.smithi027.stdout:(115/154): python3-jaraco-collections-3.0.0-8.e 254 kB/s | 23 kB 00:00 2023-12-19T18:12:58.535 INFO:teuthology.orchestra.run.smithi027.stdout:(116/154): python3-jaraco-functools-3.5.0-2.el9 213 kB/s | 19 kB 00:00 2023-12-19T18:12:58.569 INFO:teuthology.orchestra.run.smithi027.stdout:(117/154): python3-jaraco-text-3.2.0-6.el9.noar 197 kB/s | 20 kB 00:00 2023-12-19T18:12:58.636 INFO:teuthology.orchestra.run.smithi027.stdout:(118/154): python3-jwt+crypto-2.4.0-1.el9.noarc 91 kB/s | 9.0 kB 00:00 2023-12-19T18:12:58.670 INFO:teuthology.orchestra.run.smithi027.stdout:(119/154): python3-jwt-2.4.0-1.el9.noarch.rpm 411 kB/s | 41 kB 00:00 2023-12-19T18:12:58.787 INFO:teuthology.orchestra.run.smithi027.stdout:(120/154): python3-kubernetes-26.1.0-2.el9.noar 6.8 MB/s | 1.0 MB 00:00 2023-12-19T18:12:58.812 INFO:teuthology.orchestra.run.smithi027.stdout:(121/154): python3-logutils-0.3.5-21.el9.noarch 327 kB/s | 46 kB 00:00 2023-12-19T18:12:58.904 INFO:teuthology.orchestra.run.smithi027.stdout:(122/154): python3-more-itertools-8.12.0-2.el9. 678 kB/s | 79 kB 00:00 2023-12-19T18:12:58.929 INFO:teuthology.orchestra.run.smithi027.stdout:(123/154): python3-natsort-7.1.1-5.el9.noarch.r 496 kB/s | 58 kB 00:00 2023-12-19T18:12:59.030 INFO:teuthology.orchestra.run.smithi027.stdout:(124/154): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2023-12-19T18:12:59.055 INFO:teuthology.orchestra.run.smithi027.stdout:(125/154): python3-portend-3.1.0-2.el9.noarch.r 132 kB/s | 16 kB 00:00 2023-12-19T18:12:59.147 INFO:teuthology.orchestra.run.smithi027.stdout:(126/154): python3-pyOpenSSL-21.0.0-1.el9.noarc 775 kB/s | 90 kB 00:00 2023-12-19T18:12:59.197 INFO:teuthology.orchestra.run.smithi027.stdout:(127/154): python3-pyperclip-1.8.2-2.el9.noarch 200 kB/s | 28 kB 00:00 2023-12-19T18:12:59.289 INFO:teuthology.orchestra.run.smithi027.stdout:(128/154): python3-repoze-lru-0.7-16.el9.noarch 218 kB/s | 31 kB 00:00 2023-12-19T18:12:59.381 INFO:teuthology.orchestra.run.smithi027.stdout:(129/154): python3-rsa-4.9-2.el9.noarch.rpm 651 kB/s | 59 kB 00:00 2023-12-19T18:12:59.415 INFO:teuthology.orchestra.run.smithi027.stdout:(130/154): python3-routes-2.5.1-5.el9.noarch.rp 870 kB/s | 188 kB 00:00 2023-12-19T18:12:59.507 INFO:teuthology.orchestra.run.smithi027.stdout:(131/154): python3-tempora-5.0.0-2.el9.noarch.r 287 kB/s | 36 kB 00:00 2023-12-19T18:12:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:(132/154): python3-typing-extensions-4.4.0-2.el 473 kB/s | 51 kB 00:00 2023-12-19T18:12:59.607 INFO:teuthology.orchestra.run.smithi027.stdout:(133/154): python3-webob-1.8.7-6.el9.noarch.rpm 2.3 MB/s | 230 kB 00:00 2023-12-19T18:12:59.633 INFO:teuthology.orchestra.run.smithi027.stdout:(134/154): python3-websocket-client-1.2.3-2.el9 830 kB/s | 90 kB 00:00 2023-12-19T18:12:59.733 INFO:teuthology.orchestra.run.smithi027.stdout:(135/154): python3-werkzeug-2.0.3-3.el9.1.noarc 3.4 MB/s | 427 kB 00:00 2023-12-19T18:12:59.758 INFO:teuthology.orchestra.run.smithi027.stdout:(136/154): python3-zc-lockfile-2.0-10.el9.noarc 160 kB/s | 20 kB 00:00 2023-12-19T18:12:59.867 INFO:teuthology.orchestra.run.smithi027.stdout:(137/154): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2023-12-19T18:13:00.042 INFO:teuthology.orchestra.run.smithi027.stdout:(138/154): thrift-0.15.0-2.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2023-12-19T18:13:00.359 INFO:teuthology.orchestra.run.smithi027.stdout:(139/154): librbd1-18.2.1-132.g568df92b.el9.x86 9.5 MB/s | 3.0 MB 00:00 2023-12-19T18:13:00.401 INFO:teuthology.orchestra.run.smithi027.stdout:(140/154): libacl-devel-2.3.1-4.el9.x86_64.rpm 1.6 MB/s | 68 kB 00:00 2023-12-19T18:13:00.443 INFO:teuthology.orchestra.run.smithi027.stdout:(141/154): acl-2.3.1-4.el9.x86_64.rpm 1.7 MB/s | 69 kB 00:00 2023-12-19T18:13:00.477 INFO:teuthology.orchestra.run.smithi027.stdout:(142/154): elfutils-debuginfod-client-0.190-1.e 1.1 MB/s | 36 kB 00:00 2023-12-19T18:13:00.552 INFO:teuthology.orchestra.run.smithi027.stdout:(143/154): librados2-18.2.1-132.g568df92b.el9.x 4.8 MB/s | 3.3 MB 00:00 2023-12-19T18:13:00.586 INFO:teuthology.orchestra.run.smithi027.stdout:(144/154): elfutils-libelf-0.190-1.el9.x86_64.r 1.7 MB/s | 192 kB 00:00 2023-12-19T18:13:00.650 INFO:teuthology.orchestra.run.smithi027.stdout:(145/154): elfutils-libs-0.190-1.el9.x86_64.rpm 2.6 MB/s | 255 kB 00:00 2023-12-19T18:13:00.678 INFO:teuthology.orchestra.run.smithi027.stdout:(146/154): libacl-2.3.1-4.el9.x86_64.rpm 247 kB/s | 22 kB 00:00 2023-12-19T18:13:00.712 INFO:teuthology.orchestra.run.smithi027.stdout:(147/154): xfsdump-3.1.12-4.el9.x86_64.rpm 5.2 MB/s | 321 kB 00:00 2023-12-19T18:13:01.079 INFO:teuthology.orchestra.run.smithi027.stdout:(148/154): NetworkManager-1.45.7-1.el9.x86_64.r 5.7 MB/s | 2.3 MB 00:00 2023-12-19T18:13:01.154 INFO:teuthology.orchestra.run.smithi027.stdout:(149/154): NetworkManager-libnm-1.45.7-1.el9.x8 4.0 MB/s | 1.8 MB 00:00 2023-12-19T18:13:01.179 INFO:teuthology.orchestra.run.smithi027.stdout:(150/154): NetworkManager-team-1.45.7-1.el9.x86 436 kB/s | 43 kB 00:00 2023-12-19T18:13:01.230 INFO:teuthology.orchestra.run.smithi027.stdout:(151/154): NetworkManager-tui-1.45.7-1.el9.x86_ 3.2 MB/s | 246 kB 00:00 2023-12-19T18:13:01.297 INFO:teuthology.orchestra.run.smithi027.stdout:(152/154): openssl-3.0.7-25.el9.x86_64.rpm 10 MB/s | 1.2 MB 00:00 2023-12-19T18:13:01.388 INFO:teuthology.orchestra.run.smithi027.stdout:(153/154): openssl-libs-3.0.7-25.el9.x86_64.rpm 14 MB/s | 2.2 MB 00:00 2023-12-19T18:13:01.584 INFO:teuthology.orchestra.run.smithi192.stdout: Preparing : 1/1 2023-12-19T18:13:01.778 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/169 2023-12-19T18:13:01.806 INFO:teuthology.orchestra.run.smithi027.stdout:(154/154): openssl-devel-3.0.7-25.el9.x86_64.rp 746 kB/s | 4.1 MB 00:05 2023-12-19T18:13:01.816 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/169 2023-12-19T18:13:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2023-12-19T18:13:01.818 INFO:teuthology.orchestra.run.smithi027.stdout:Total 13 MB/s | 200 MB 00:15 2023-12-19T18:13:01.897 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/169 2023-12-19T18:13:01.954 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/169 2023-12-19T18:13:01.991 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/169 2023-12-19T18:13:02.096 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/169 2023-12-19T18:13:02.256 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2023-12-19T18:13:02.508 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/169 2023-12-19T18:13:02.555 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librados2-2:18.2.1-132.g568df92b.el9.x86_64 8/169 2023-12-19T18:13:02.560 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2023-12-19T18:13:02.560 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2023-12-19T18:13:02.689 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:18.2.1-132.g568df92b.el9.x86_64 8/169 2023-12-19T18:13:02.715 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs2-2:18.2.1-132.g568df92b.el9.x86_64 9/169 2023-12-19T18:13:02.835 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephfs2-2:18.2.1-132.g568df92b.el9.x86_64 9/169 2023-12-19T18:13:02.878 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rados-2:18.2.1-132.g568df92b.el9.x86_64 10/169 2023-12-19T18:13:02.953 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/169 2023-12-19T18:13:03.269 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/169 2023-12-19T18:13:03.354 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/169 2023-12-19T18:13:03.386 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephsqlite-2:18.2.1-132.g568df92b.el9.x86_64 14/169 2023-12-19T18:13:03.585 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libcephsqlite-2:18.2.1-132.g568df92b.el9.x86_64 14/169 2023-12-19T18:13:03.613 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libradosstriper1-2:18.2.1-132.g568df92b.el9.x86_ 15/169 2023-12-19T18:13:03.983 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: libradosstriper1-2:18.2.1-132.g568df92b.el9.x86_ 15/169 2023-12-19T18:13:04.175 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/169 2023-12-19T18:13:04.264 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/169 2023-12-19T18:13:04.454 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/169 2023-12-19T18:13:04.504 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-1:1.45.7-1.el9.x86_64 17/169 2023-12-19T18:13:04.828 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/169 2023-12-19T18:13:04.829 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2023-12-19T18:13:04.830 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2023-12-19T18:13:04.951 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/169 2023-12-19T18:13:04.992 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/169 2023-12-19T18:13:05.087 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/169 2023-12-19T18:13:05.165 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/169 2023-12-19T18:13:05.196 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/169 2023-12-19T18:13:05.248 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/169 2023-12-19T18:13:05.295 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/169 2023-12-19T18:13:05.395 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/169 2023-12-19T18:13:05.449 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/169 2023-12-19T18:13:05.482 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/169 2023-12-19T18:13:05.589 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/169 2023-12-19T18:13:05.630 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-common-2:18.2.1-132.g568df92b.el9.x 29/169 2023-12-19T18:13:05.680 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ceph-argparse-2:18.2.1-132.g568df92b.el9 30/169 2023-12-19T18:13:05.779 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cephfs-2:18.2.1-132.g568df92b.el9.x86_64 31/169 2023-12-19T18:13:05.960 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/169 2023-12-19T18:13:06.021 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/169 2023-12-19T18:13:06.694 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/169 2023-12-19T18:13:06.786 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/169 2023-12-19T18:13:06.835 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/169 2023-12-19T18:13:06.897 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/169 2023-12-19T18:13:06.943 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librados-devel-2:18.2.1-132.g568df92b.el9.x86_64 38/169 2023-12-19T18:13:07.007 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/169 2023-12-19T18:13:07.249 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/169 2023-12-19T18:13:07.433 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/169 2023-12-19T18:13:07.609 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/169 2023-12-19T18:13:07.642 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 43/169 2023-12-19T18:13:07.706 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/169 2023-12-19T18:13:07.793 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/169 2023-12-19T18:13:07.843 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/169 2023-12-19T18:13:07.890 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/169 2023-12-19T18:13:07.959 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/169 2023-12-19T18:13:07.999 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/169 2023-12-19T18:13:08.062 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/169 2023-12-19T18:13:08.129 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/169 2023-12-19T18:13:08.320 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/169 2023-12-19T18:13:08.381 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/169 2023-12-19T18:13:08.436 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/169 2023-12-19T18:13:08.508 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/169 2023-12-19T18:13:08.636 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/169 2023-12-19T18:13:08.676 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 57/169 2023-12-19T18:13:08.724 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/169 2023-12-19T18:13:08.757 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/169 2023-12-19T18:13:08.796 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/169 2023-12-19T18:13:08.957 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fmt-8.1.1-5.el9.x86_64 61/169 2023-12-19T18:13:09.021 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 62/169 2023-12-19T18:13:09.090 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-ply-3.11-14.el9.noarch 63/169 2023-12-19T18:13:09.171 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 64/169 2023-12-19T18:13:09.435 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 65/169 2023-12-19T18:13:09.500 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 66/169 2023-12-19T18:13:09.664 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 67/169 2023-12-19T18:13:09.727 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 68/169 2023-12-19T18:13:09.752 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 69/169 2023-12-19T18:13:09.792 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 70/169 2023-12-19T18:13:09.835 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 71/169 2023-12-19T18:13:09.871 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : fuse-2.9.9-15.el9.x86_64 72/169 2023-12-19T18:13:09.913 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/169 2023-12-19T18:13:09.972 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/169 2023-12-19T18:13:10.044 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/169 2023-12-19T18:13:10.101 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/169 2023-12-19T18:13:10.147 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/169 2023-12-19T18:13:10.295 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/169 2023-12-19T18:13:10.342 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 79/169 2023-12-19T18:13:10.377 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 80/169 2023-12-19T18:13:10.430 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 81/169 2023-12-19T18:13:10.501 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 82/169 2023-12-19T18:13:10.644 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 83/169 2023-12-19T18:13:10.687 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 84/169 2023-12-19T18:13:10.838 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/169 2023-12-19T18:13:10.901 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 86/169 2023-12-19T18:13:11.669 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2023-12-19T18:13:11.817 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-libs-1:3.0.7-25.el9.x86_64 1/169 2023-12-19T18:13:11.856 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : thrift-0.15.0-2.el9.x86_64 2/169 2023-12-19T18:13:11.873 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 87/169 2023-12-19T18:13:11.925 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 88/169 2023-12-19T18:13:11.943 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/169 2023-12-19T18:13:11.959 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 89/169 2023-12-19T18:13:12.011 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 90/169 2023-12-19T18:13:12.011 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 4/169 2023-12-19T18:13:12.048 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/169 2023-12-19T18:13:12.115 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/169 2023-12-19T18:13:12.142 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/169 2023-12-19T18:13:12.151 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/169 2023-12-19T18:13:12.429 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/169 2023-12-19T18:13:12.572 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/169 2023-12-19T18:13:12.588 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/169 2023-12-19T18:13:12.621 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:18.2.1-132.g568df92b.el9.x86_64 8/169 2023-12-19T18:13:12.733 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/169 2023-12-19T18:13:12.758 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:18.2.1-132.g568df92b.el9.x86_64 8/169 2023-12-19T18:13:12.788 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:18.2.1-132.g568df92b.el9.x86_64 9/169 2023-12-19T18:13:12.923 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:18.2.1-132.g568df92b.el9.x86_64 9/169 2023-12-19T18:13:12.964 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:18.2.1-132.g568df92b.el9.x86_64 10/169 2023-12-19T18:13:13.030 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-private-libs-13.11-1.el9.x86_64 11/169 2023-12-19T18:13:13.315 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libelf-0.190-1.el9.x86_64 12/169 2023-12-19T18:13:13.377 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-13.11-1.el9.x86_64 13/169 2023-12-19T18:13:13.401 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:18.2.1-132.g568df92b.el9.x86_64 14/169 2023-12-19T18:13:13.538 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/169 2023-12-19T18:13:13.544 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:18.2.1-132.g568df92b.el9.x86_64 14/169 2023-12-19T18:13:13.578 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:18.2.1-132.g568df92b.el9.x86_ 15/169 2023-12-19T18:13:13.669 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/169 2023-12-19T18:13:13.965 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:18.2.1-132.g568df92b.el9.x86_ 15/169 2023-12-19T18:13:14.124 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/169 2023-12-19T18:13:14.239 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 16/169 2023-12-19T18:13:14.447 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/169 2023-12-19T18:13:14.510 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-1:1.45.7-1.el9.x86_64 17/169 2023-12-19T18:13:14.869 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.45.7-1.el9.x86_64 17/169 2023-12-19T18:13:15.046 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libacl-2.3.1-4.el9.x86_64 18/169 2023-12-19T18:13:15.091 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 19/169 2023-12-19T18:13:15.237 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/169 2023-12-19T18:13:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.7-2.el9.x86_64 20/169 2023-12-19T18:13:15.315 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.26.5-4.el9.noarch 21/169 2023-12-19T18:13:15.335 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/169 2023-12-19T18:13:15.354 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.25.1-7.el9.noarch 22/169 2023-12-19T18:13:15.373 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 100/169 2023-12-19T18:13:15.396 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : jq-1.6-16.el9.x86_64 101/169 2023-12-19T18:13:15.425 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-11.4.1-2.3.el9.x86_64 23/169 2023-12-19T18:13:15.430 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/169 2023-12-19T18:13:15.471 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/169 2023-12-19T18:13:15.478 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-11.4.1-2.3.el9.x86_64 24/169 2023-12-19T18:13:15.585 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 25/169 2023-12-19T18:13:15.658 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/169 2023-12-19T18:13:15.692 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 27/169 2023-12-19T18:13:15.812 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 28/169 2023-12-19T18:13:15.838 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/169 2023-12-19T18:13:15.863 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:18.2.1-132.g568df92b.el9.x 29/169 2023-12-19T18:13:15.868 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : librbd1-2:18.2.1-132.g568df92b.el9.x86_64 105/169 2023-12-19T18:13:15.917 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:18.2.1-132.g568df92b.el9 30/169 2023-12-19T18:13:16.030 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:18.2.1-132.g568df92b.el9.x86_64 105/169 2023-12-19T18:13:16.037 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:18.2.1-132.g568df92b.el9.x86_64 31/169 2023-12-19T18:13:16.066 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rbd-2:18.2.1-132.g568df92b.el9.x86_64 106/169 2023-12-19T18:13:16.120 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/169 2023-12-19T18:13:16.223 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 32/169 2023-12-19T18:13:16.289 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 33/169 2023-12-19T18:13:16.608 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/169 2023-12-19T18:13:16.638 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : librgw2-2:18.2.1-132.g568df92b.el9.x86_64 109/169 2023-12-19T18:13:16.755 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librgw2-2:18.2.1-132.g568df92b.el9.x86_64 109/169 2023-12-19T18:13:16.782 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : python3-rgw-2:18.2.1-132.g568df92b.el9.x86_64 110/169 2023-12-19T18:13:17.016 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.9-2.el9.noarch 34/169 2023-12-19T18:13:17.405 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 35/169 2023-12-19T18:13:17.456 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.11.3-4.el9.noarch 36/169 2023-12-19T18:13:17.525 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-libs-0.190-1.el9.x86_64 37/169 2023-12-19T18:13:17.601 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:18.2.1-132.g568df92b.el9.x86_64 38/169 2023-12-19T18:13:17.678 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 39/169 2023-12-19T18:13:17.893 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 40/169 2023-12-19T18:13:17.947 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : openssl-1:3.0.7-25.el9.x86_64 41/169 2023-12-19T18:13:18.099 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : re2-1:20211101-3.el9.x86_64 42/169 2023-12-19T18:13:18.142 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 43/169 2023-12-19T18:13:18.223 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/169 2023-12-19T18:13:18.316 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/169 2023-12-19T18:13:18.351 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/169 2023-12-19T18:13:18.390 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/169 2023-12-19T18:13:18.441 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/169 2023-12-19T18:13:18.481 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/169 2023-12-19T18:13:18.574 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/169 2023-12-19T18:13:18.637 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/169 2023-12-19T18:13:18.832 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/169 2023-12-19T18:13:18.888 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/169 2023-12-19T18:13:18.917 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/169 2023-12-19T18:13:18.957 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/169 2023-12-19T18:13:19.116 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/169 2023-12-19T18:13:19.159 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:2.23.4-1.el9.noarch 57/169 2023-12-19T18:13:19.210 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/169 2023-12-19T18:13:19.241 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/169 2023-12-19T18:13:19.281 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/169 2023-12-19T18:13:19.334 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:18.2.1-132.g568df92b.el9.x86_64 111/169 2023-12-19T18:13:19.378 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-common-2:18.2.1-132.g568df92b.el9.x86_64 111/169 2023-12-19T18:13:19.453 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fmt-8.1.1-5.el9.x86_64 61/169 2023-12-19T18:13:19.521 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-devel-3.9.17-2.el9.x86_64 62/169 2023-12-19T18:13:19.579 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ply-3.11-14.el9.noarch 63/169 2023-12-19T18:13:19.654 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 64/169 2023-12-19T18:13:19.905 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 65/169 2023-12-19T18:13:19.990 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 66/169 2023-12-19T18:13:20.010 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-common-2:18.2.1-132.g568df92b.el9.x86_64 111/169 2023-12-19T18:13:20.058 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-base-2:18.2.1-132.g568df92b.el9.x86_64 112/169 2023-12-19T18:13:20.179 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 67/169 2023-12-19T18:13:20.222 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-base-2:18.2.1-132.g568df92b.el9.x86_64 112/169 2023-12-19T18:13:20.223 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-19T18:13:20.223 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-19T18:13:20.223 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:20.237 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-asyncssh-2.13.2-3.el9.noarch 68/169 2023-12-19T18:13:20.268 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 69/169 2023-12-19T18:13:20.275 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-selinux-2:18.2.1-132.g568df92b.el9.x86_64 113/169 2023-12-19T18:13:20.309 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 70/169 2023-12-19T18:13:20.352 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 71/169 2023-12-19T18:13:20.388 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.9-15.el9.x86_64 72/169 2023-12-19T18:13:20.431 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : zlib-devel-1.2.11-40.el9.x86_64 73/169 2023-12-19T18:13:20.515 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 74/169 2023-12-19T18:13:20.593 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : uuid-1.6.2-55.el9.x86_64 75/169 2023-12-19T18:13:20.654 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 76/169 2023-12-19T18:13:20.699 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 77/169 2023-12-19T18:13:20.833 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 78/169 2023-12-19T18:13:20.876 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 79/169 2023-12-19T18:13:20.911 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 80/169 2023-12-19T18:13:20.991 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 81/169 2023-12-19T18:13:21.032 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 82/169 2023-12-19T18:13:21.183 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 83/169 2023-12-19T18:13:21.233 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 84/169 2023-12-19T18:13:21.382 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 85/169 2023-12-19T18:13:21.469 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 86/169 2023-12-19T18:13:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 87/169 2023-12-19T18:13:22.526 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 88/169 2023-12-19T18:13:22.559 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 89/169 2023-12-19T18:13:22.618 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 90/169 2023-12-19T18:13:22.770 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 91/169 2023-12-19T18:13:22.810 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 92/169 2023-12-19T18:13:23.093 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.21-2.el9.x86_64 93/169 2023-12-19T18:13:23.123 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 94/169 2023-12-19T18:13:23.228 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 95/169 2023-12-19T18:13:24.076 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 96/169 2023-12-19T18:13:24.216 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 97/169 2023-12-19T18:13:25.929 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 98/169 2023-12-19T18:13:26.037 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 99/169 2023-12-19T18:13:26.067 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : oniguruma-6.9.6-1.el9.5.x86_64 100/169 2023-12-19T18:13:26.097 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : jq-1.6-16.el9.x86_64 101/169 2023-12-19T18:13:26.140 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 102/169 2023-12-19T18:13:26.182 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libuuid-devel-2.37.4-15.el9.x86_64 103/169 2023-12-19T18:13:26.576 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 104/169 2023-12-19T18:13:26.612 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:18.2.1-132.g568df92b.el9.x86_64 105/169 2023-12-19T18:13:26.732 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:18.2.1-132.g568df92b.el9.x86_64 105/169 2023-12-19T18:13:26.775 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:18.2.1-132.g568df92b.el9.x86_64 106/169 2023-12-19T18:13:26.831 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 107/169 2023-12-19T18:13:27.306 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 108/169 2023-12-19T18:13:27.705 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:18.2.1-132.g568df92b.el9.x86_64 109/169 2023-12-19T18:13:27.831 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:18.2.1-132.g568df92b.el9.x86_64 109/169 2023-12-19T18:13:27.858 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:18.2.1-132.g568df92b.el9.x86_64 110/169 2023-12-19T18:13:30.446 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:18.2.1-132.g568df92b.el9.x86_64 111/169 2023-12-19T18:13:30.496 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:18.2.1-132.g568df92b.el9.x86_64 111/169 2023-12-19T18:13:31.167 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:18.2.1-132.g568df92b.el9.x86_64 111/169 2023-12-19T18:13:31.216 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:18.2.1-132.g568df92b.el9.x86_64 112/169 2023-12-19T18:13:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:18.2.1-132.g568df92b.el9.x86_64 112/169 2023-12-19T18:13:31.373 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2023-12-19T18:13:31.373 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2023-12-19T18:13:31.373 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:13:31.425 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:18.2.1-132.g568df92b.el9.x86_64 113/169 2023-12-19T18:13:33.054 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-selinux-2:18.2.1-132.g568df92b.el9.x86_64 113/169 2023-12-19T18:13:33.054 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /sys 2023-12-19T18:13:33.054 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /proc 2023-12-19T18:13:33.054 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /mnt 2023-12-19T18:13:33.054 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /var/tmp 2023-12-19T18:13:33.055 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /home 2023-12-19T18:13:33.055 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /root 2023-12-19T18:13:33.055 INFO:teuthology.orchestra.run.smithi192.stdout:skipping the directory /tmp 2023-12-19T18:13:33.055 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:33.303 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-volume-2:18.2.1-132.g568df92b.el9.noarch 114/169 2023-12-19T18:13:33.349 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-volume-2:18.2.1-132.g568df92b.el9.noarch 114/169 2023-12-19T18:13:33.349 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:33.350 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-19T18:13:33.350 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:35.091 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-osd-2:18.2.1-132.g568df92b.el9.x86_64 115/169 2023-12-19T18:13:35.204 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-osd-2:18.2.1-132.g568df92b.el9.x86_64 115/169 2023-12-19T18:13:35.204 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:35.204 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-19T18:13:35.204 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-19T18:13:35.204 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-19T18:13:35.204 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:35.466 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mds-2:18.2.1-132.g568df92b.el9.x86_64 116/169 2023-12-19T18:13:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mds-2:18.2.1-132.g568df92b.el9.x86_64 116/169 2023-12-19T18:13:35.565 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:35.565 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-19T18:13:35.565 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-19T18:13:35.565 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-19T18:13:35.565 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:36.066 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mon-2:18.2.1-132.g568df92b.el9.x86_64 117/169 2023-12-19T18:13:36.167 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mon-2:18.2.1-132.g568df92b.el9.x86_64 117/169 2023-12-19T18:13:36.168 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:36.168 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-19T18:13:36.168 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-19T18:13:36.168 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-19T18:13:36.168 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:36.285 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : automake-1.16.2-8.el9.noarch 118/169 2023-12-19T18:13:36.919 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephadm-2:18.2.1-132.g568df92b.el9.noarch 119/169 2023-12-19T18:13:36.955 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephadm-2:18.2.1-132.g568df92b.el9.noarch 119/169 2023-12-19T18:13:37.002 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-132.g568df92b.el 120/169 2023-12-19T18:13:37.122 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-132.g568df92b.e 121/169 2023-12-19T18:13:37.148 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-132.g568df92b.el9.noar 122/169 2023-12-19T18:13:37.473 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-132.g568df92b.el9.noar 122/169 2023-12-19T18:13:37.508 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-132.g568df92b.el9.no 123/169 2023-12-19T18:13:38.639 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-132.g568df92b.el9.no 123/169 2023-12-19T18:13:38.666 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-132.g568d 124/169 2023-12-19T18:13:38.811 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-132.g568d 124/169 2023-12-19T18:13:38.998 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-132.g568df92b.el9 125/169 2023-12-19T18:13:39.027 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-2:18.2.1-132.g568df92b.el9.x86_64 126/169 2023-12-19T18:13:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-2:18.2.1-132.g568df92b.el9.x86_64 126/169 2023-12-19T18:13:39.098 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:39.098 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-19T18:13:39.098 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-19T18:13:39.098 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-19T18:13:39.098 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:39.152 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-mgr-rook-2:18.2.1-132.g568df92b.el9.noarch 127/169 2023-12-19T18:13:39.190 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-132.g568df92b.el9.noarch 127/169 2023-12-19T18:13:39.249 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-2:18.2.1-132.g568df92b.el9.x86_64 128/169 2023-12-19T18:13:40.063 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/169 2023-12-19T18:13:40.096 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-radosgw-2:18.2.1-132.g568df92b.el9.x86_64 130/169 2023-12-19T18:13:40.169 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-132.g568df92b.el9.x86_64 130/169 2023-12-19T18:13:40.169 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:40.169 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-19T18:13:40.170 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-19T18:13:40.170 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-19T18:13:40.170 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:40.222 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-132.g568df9 131/169 2023-12-19T18:13:40.294 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-132.g568df9 131/169 2023-12-19T18:13:40.294 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:40.294 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-19T18:13:40.294 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:40.356 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-mirror-2:18.2.1-132.g568df92b.el9.x86_64 132/169 2023-12-19T18:13:40.432 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: cephfs-mirror-2:18.2.1-132.g568df92b.el9.x86_64 132/169 2023-12-19T18:13:40.432 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:40.432 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-12-19T18:13:40.432 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-19T18:13:40.432 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-19T18:13:40.432 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:40.798 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-mirror-2:18.2.1-132.g568df92b.el9.x86_64 133/169 2023-12-19T18:13:40.873 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: rbd-mirror-2:18.2.1-132.g568df92b.el9.x86_64 133/169 2023-12-19T18:13:40.873 INFO:teuthology.orchestra.run.smithi192.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:40.873 INFO:teuthology.orchestra.run.smithi192.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-19T18:13:40.874 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-19T18:13:40.874 INFO:teuthology.orchestra.run.smithi192.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-19T18:13:40.874 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:44.766 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:18.2.1-132.g568df92b.el9.x86_64 113/169 2023-12-19T18:13:44.767 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2023-12-19T18:13:44.767 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2023-12-19T18:13:44.767 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2023-12-19T18:13:44.767 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2023-12-19T18:13:44.767 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2023-12-19T18:13:44.767 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /root 2023-12-19T18:13:44.767 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2023-12-19T18:13:44.767 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:13:44.845 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-test-2:18.2.1-132.g568df92b.el9.x86_64 134/169 2023-12-19T18:13:44.898 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-fuse-2:18.2.1-132.g568df92b.el9.x86_64 135/169 2023-12-19T18:13:44.953 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : rbd-nbd-2:18.2.1-132.g568df92b.el9.x86_64 136/169 2023-12-19T18:13:44.987 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-volume-2:18.2.1-132.g568df92b.el9.noarch 114/169 2023-12-19T18:13:44.994 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 137/169 2023-12-19T18:13:45.032 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 138/169 2023-12-19T18:13:45.040 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-volume-2:18.2.1-132.g568df92b.el9.noarch 114/169 2023-12-19T18:13:45.041 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:45.041 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2023-12-19T18:13:45.041 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:13:45.298 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-shell-2:18.2.1-132.g568df92b.el9.x86_64 139/169 2023-12-19T18:13:45.418 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 140/169 2023-12-19T18:13:45.448 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : ceph-fuse-2:18.2.1-132.g568df92b.el9.x86_64 141/169 2023-12-19T18:13:45.482 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libcephfs-devel-2:18.2.1-132.g568df92b.el9.x86_6 142/169 2023-12-19T18:13:45.556 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 143/169 2023-12-19T18:13:45.619 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 144/169 2023-12-19T18:13:45.659 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 145/169 2023-12-19T18:13:45.727 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-team-1:1.45.7-1.el9.x86_64 146/169 2023-12-19T18:13:45.784 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : NetworkManager-tui-1:1.45.7-1.el9.x86_64 147/169 2023-12-19T18:13:46.645 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 148/169 2023-12-19T18:13:46.709 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 148/169 2023-12-19T18:13:46.770 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 148/169 2023-12-19T18:13:46.845 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:18.2.1-132.g568df92b.el9.x86_64 115/169 2023-12-19T18:13:46.955 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:18.2.1-132.g568df92b.el9.x86_64 115/169 2023-12-19T18:13:46.955 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:46.955 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2023-12-19T18:13:46.956 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-19T18:13:46.956 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2023-12-19T18:13:46.956 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:13:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:18.2.1-132.g568df92b.el9.x86_64 116/169 2023-12-19T18:13:47.305 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:18.2.1-132.g568df92b.el9.x86_64 116/169 2023-12-19T18:13:47.305 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:47.305 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2023-12-19T18:13:47.305 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-19T18:13:47.306 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2023-12-19T18:13:47.306 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:13:47.842 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:18.2.1-132.g568df92b.el9.x86_64 117/169 2023-12-19T18:13:47.931 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:18.2.1-132.g568df92b.el9.x86_64 117/169 2023-12-19T18:13:47.932 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:47.932 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2023-12-19T18:13:47.932 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-19T18:13:47.932 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2023-12-19T18:13:47.932 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:13:48.052 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : automake-1.16.2-8.el9.noarch 118/169 2023-12-19T18:13:48.103 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : cephfs-top-2:18.2.1-132.g568df92b.el9.noarch 149/169 2023-12-19T18:13:48.232 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 150/169 2023-12-19T18:13:48.286 INFO:teuthology.orchestra.run.smithi192.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 151/169 2023-12-19T18:13:48.357 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/169 2023-12-19T18:13:48.495 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/169 2023-12-19T18:13:48.532 INFO:teuthology.orchestra.run.smithi192.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/169 2023-12-19T18:13:48.556 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 155/169 2023-12-19T18:13:48.590 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 156/169 2023-12-19T18:13:48.631 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 157/169 2023-12-19T18:13:48.631 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/169 2023-12-19T18:13:48.678 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:18.2.1-132.g568df92b.el9.noarch 119/169 2023-12-19T18:13:48.714 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:18.2.1-132.g568df92b.el9.noarch 119/169 2023-12-19T18:13:48.771 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/169 2023-12-19T18:13:48.785 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:18.2.1-132.g568df92b.el 120/169 2023-12-19T18:13:48.824 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 159/169 2023-12-19T18:13:48.870 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 160/169 2023-12-19T18:13:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 161/169 2023-12-19T18:13:48.903 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/169 2023-12-19T18:13:48.904 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:18.2.1-132.g568df92b.e 121/169 2023-12-19T18:13:48.934 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:18.2.1-132.g568df92b.el9.noar 122/169 2023-12-19T18:13:49.032 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/169 2023-12-19T18:13:49.063 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 163/169 2023-12-19T18:13:49.102 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 164/169 2023-12-19T18:13:49.102 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 164/169 2023-12-19T18:13:49.274 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.1-132.g568df92b.el9.noar 122/169 2023-12-19T18:13:49.309 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:18.2.1-132.g568df92b.el9.no 123/169 2023-12-19T18:13:49.428 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 164/169 2023-12-19T18:13:49.429 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 165/169 2023-12-19T18:13:49.553 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 165/169 2023-12-19T18:13:49.590 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 166/169 2023-12-19T18:13:49.624 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 167/169 2023-12-19T18:13:49.658 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 168/169 2023-12-19T18:13:49.785 INFO:teuthology.orchestra.run.smithi192.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 169/169 2023-12-19T18:13:50.065 INFO:teuthology.orchestra.run.smithi192.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 169/169 2023-12-19T18:13:50.066 INFO:teuthology.orchestra.run.smithi192.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-19T18:13:50.066 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.1-132.g568df92b.el9.no 123/169 2023-12-19T18:13:50.527 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.1-132.g568d 124/169 2023-12-19T18:13:50.681 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.1-132.g568d 124/169 2023-12-19T18:13:50.858 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:18.2.1-132.g568df92b.el9 125/169 2023-12-19T18:13:50.895 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:18.2.1-132.g568df92b.el9.x86_64 126/169 2023-12-19T18:13:50.981 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:18.2.1-132.g568df92b.el9.x86_64 126/169 2023-12-19T18:13:50.981 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:50.981 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2023-12-19T18:13:50.982 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-19T18:13:50.982 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2023-12-19T18:13:50.982 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:13:51.029 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:18.2.1-132.g568df92b.el9.noarch 127/169 2023-12-19T18:13:51.070 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.1-132.g568df92b.el9.noarch 127/169 2023-12-19T18:13:51.162 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:18.2.1-132.g568df92b.el9.x86_64 128/169 2023-12-19T18:13:52.047 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libtool-2.4.6-45.el9.x86_64 129/169 2023-12-19T18:13:52.081 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:18.2.1-132.g568df92b.el9.x86_64 130/169 2023-12-19T18:13:52.177 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:18.2.1-132.g568df92b.el9.x86_64 130/169 2023-12-19T18:13:52.178 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:52.178 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2023-12-19T18:13:52.178 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-19T18:13:52.178 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2023-12-19T18:13:52.178 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:13:52.240 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:18.2.1-132.g568df9 131/169 2023-12-19T18:13:52.347 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.1-132.g568df9 131/169 2023-12-19T18:13:52.347 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:52.347 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2023-12-19T18:13:52.347 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:13:52.425 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-mirror-2:18.2.1-132.g568df92b.el9.x86_64 132/169 2023-12-19T18:13:52.536 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephfs-mirror-2:18.2.1-132.g568df92b.el9.x86_64 132/169 2023-12-19T18:13:52.545 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:52.545 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2023-12-19T18:13:52.545 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-19T18:13:52.545 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2023-12-19T18:13:52.546 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:13:52.917 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:18.2.1-132.g568df92b.el9.x86_64 133/169 2023-12-19T18:13:53.008 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:18.2.1-132.g568df92b.el9.x86_64 133/169 2023-12-19T18:13:53.008 INFO:teuthology.orchestra.run.smithi027.stdout:Glob pattern passed to enable, but globs are not supported for this. 2023-12-19T18:13:53.008 INFO:teuthology.orchestra.run.smithi027.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2023-12-19T18:13:53.008 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-19T18:13:53.008 INFO:teuthology.orchestra.run.smithi027.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2023-12-19T18:13:53.009 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:13:54.634 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-2:18.2.1-132.g568df92b.el9.x86_64 1/169 2023-12-19T18:13:54.634 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-base-2:18.2.1-132.g568df92b.el9.x86_64 2/169 2023-12-19T18:13:54.634 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-common-2:18.2.1-132.g568df92b.el9.x86_64 3/169 2023-12-19T18:13:54.634 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-fuse-2:18.2.1-132.g568df92b.el9.x86_64 4/169 2023-12-19T18:13:54.634 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-132.g568df9 5/169 2023-12-19T18:13:54.634 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mds-2:18.2.1-132.g568df92b.el9.x86_64 6/169 2023-12-19T18:13:54.634 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-2:18.2.1-132.g568df92b.el9.x86_64 7/169 2023-12-19T18:13:54.634 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mon-2:18.2.1-132.g568df92b.el9.x86_64 8/169 2023-12-19T18:13:54.634 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-osd-2:18.2.1-132.g568df92b.el9.x86_64 9/169 2023-12-19T18:13:54.634 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-radosgw-2:18.2.1-132.g568df92b.el9.x86_64 10/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-selinux-2:18.2.1-132.g568df92b.el9.x86_64 11/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-test-2:18.2.1-132.g568df92b.el9.x86_64 12/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-mirror-2:18.2.1-132.g568df92b.el9.x86_64 13/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-shell-2:18.2.1-132.g568df92b.el9.x86_64 14/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs-devel-2:18.2.1-132.g568df92b.el9.x86_6 15/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephfs2-2:18.2.1-132.g568df92b.el9.x86_64 16/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libcephsqlite-2:18.2.1-132.g568df92b.el9.x86_64 17/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados-devel-2:18.2.1-132.g568df92b.el9.x86_64 18/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libradosstriper1-2:18.2.1-132.g568df92b.el9.x86_ 19/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librgw2-2:18.2.1-132.g568df92b.el9.x86_64 20/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-argparse-2:18.2.1-132.g568df92b.el9 21/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ceph-common-2:18.2.1-132.g568df92b.el9.x 22/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cephfs-2:18.2.1-132.g568df92b.el9.x86_64 23/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rados-2:18.2.1-132.g568df92b.el9.x86_64 24/169 2023-12-19T18:13:54.635 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rbd-2:18.2.1-132.g568df92b.el9.x86_64 25/169 2023-12-19T18:13:54.648 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rgw-2:18.2.1-132.g568df92b.el9.x86_64 26/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-fuse-2:18.2.1-132.g568df92b.el9.x86_64 27/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-mirror-2:18.2.1-132.g568df92b.el9.x86_64 28/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : rbd-nbd-2:18.2.1-132.g568df92b.el9.x86_64 29/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-132.g568df92b.e 30/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-132.g568df92b.el9.noar 31/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-132.g568df92b.el9.no 32/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-132.g568d 33/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-132.g568df92b.el9 34/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-mgr-rook-2:18.2.1-132.g568df92b.el9.noarch 35/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-132.g568df92b.el 36/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : ceph-volume-2:18.2.1-132.g568df92b.el9.noarch 37/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephadm-2:18.2.1-132.g568df92b.el9.noarch 38/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : cephfs-top-2:18.2.1-132.g568df92b.el9.noarch 39/169 2023-12-19T18:13:54.649 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/169 2023-12-19T18:13:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/169 2023-12-19T18:13:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/169 2023-12-19T18:13:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 43/169 2023-12-19T18:13:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/169 2023-12-19T18:13:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/169 2023-12-19T18:13:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/169 2023-12-19T18:13:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/169 2023-12-19T18:13:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : jq-1.6-16.el9.x86_64 48/169 2023-12-19T18:13:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 49/169 2023-12-19T18:13:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 50/169 2023-12-19T18:13:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 51/169 2023-12-19T18:13:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 52/169 2023-12-19T18:13:54.650 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 53/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 54/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 55/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 56/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 57/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 58/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 59/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 64/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 65/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 66/169 2023-12-19T18:13:54.651 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 67/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 68/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 69/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 70/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 71/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 72/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 73/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 74/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/169 2023-12-19T18:13:54.652 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/169 2023-12-19T18:13:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/169 2023-12-19T18:13:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 84/169 2023-12-19T18:13:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 85/169 2023-12-19T18:13:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 86/169 2023-12-19T18:13:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 87/169 2023-12-19T18:13:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 88/169 2023-12-19T18:13:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 89/169 2023-12-19T18:13:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/169 2023-12-19T18:13:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/169 2023-12-19T18:13:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/169 2023-12-19T18:13:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/169 2023-12-19T18:13:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 94/169 2023-12-19T18:13:54.653 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 95/169 2023-12-19T18:13:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 96/169 2023-12-19T18:13:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 97/169 2023-12-19T18:13:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 98/169 2023-12-19T18:13:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 99/169 2023-12-19T18:13:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 100/169 2023-12-19T18:13:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 101/169 2023-12-19T18:13:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 102/169 2023-12-19T18:13:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 103/169 2023-12-19T18:13:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 104/169 2023-12-19T18:13:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 105/169 2023-12-19T18:13:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 106/169 2023-12-19T18:13:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 107/169 2023-12-19T18:13:54.654 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 108/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 109/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 110/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 111/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 112/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 113/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 114/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 115/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 116/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 117/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 118/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 119/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 120/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 121/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 122/169 2023-12-19T18:13:54.655 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 123/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 124/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 125/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 126/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 127/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 128/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 129/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 130/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 131/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 132/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 133/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 134/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 135/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 136/169 2023-12-19T18:13:54.656 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 137/169 2023-12-19T18:13:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : re2-1:20211101-3.el9.x86_64 138/169 2023-12-19T18:13:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 139/169 2023-12-19T18:13:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:18.2.1-132.g568df92b.el9.x86_64 140/169 2023-12-19T18:13:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 141/169 2023-12-19T18:13:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:18.2.1-132.g568df92b.el9.x86_64 142/169 2023-12-19T18:13:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 143/169 2023-12-19T18:13:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 144/169 2023-12-19T18:13:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 145/169 2023-12-19T18:13:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : acl-2.3.1-4.el9.x86_64 146/169 2023-12-19T18:13:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : acl-2.3.1-3.el9.x86_64 147/169 2023-12-19T18:13:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 148/169 2023-12-19T18:13:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/169 2023-12-19T18:13:54.657 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 150/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 151/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 152/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 153/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 154/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 155/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 156/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 157/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.45.7-1.el9.x86_64 158/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 159/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 160/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 161/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.45.7-1.el9.x86_64 162/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 163/169 2023-12-19T18:13:54.658 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.45.7-1.el9.x86_64 164/169 2023-12-19T18:13:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 165/169 2023-12-19T18:13:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 166/169 2023-12-19T18:13:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 167/169 2023-12-19T18:13:54.659 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 168/169 2023-12-19T18:13:56.790 INFO:teuthology.orchestra.run.smithi192.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 169/169 2023-12-19T18:13:56.790 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:56.790 INFO:teuthology.orchestra.run.smithi192.stdout:Upgraded: 2023-12-19T18:13:56.790 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-1:1.45.7-1.el9.x86_64 2023-12-19T18:13:56.790 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 2023-12-19T18:13:56.790 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-team-1:1.45.7-1.el9.x86_64 2023-12-19T18:13:56.790 INFO:teuthology.orchestra.run.smithi192.stdout: NetworkManager-tui-1:1.45.7-1.el9.x86_64 2023-12-19T18:13:56.790 INFO:teuthology.orchestra.run.smithi192.stdout: acl-2.3.1-4.el9.x86_64 2023-12-19T18:13:56.790 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-12-19T18:13:56.790 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-12-19T18:13:56.790 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: libacl-2.3.1-4.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: librados2-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: librbd1-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout:Installed: 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: automake-1.16.2-8.el9.noarch 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: bison-3.7.4-5.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-base-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-common-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-fuse-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.791 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-grafana-dashboards-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-immutable-object-cache-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mds-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-cephadm-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-dashboard-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-diskprediction-local-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-modules-core-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mgr-rook-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-mon-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-osd-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-prometheus-alerts-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-radosgw-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-selinux-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.792 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-test-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: ceph-volume-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: cephadm-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-mirror-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-shell-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: cephfs-top-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: flex-2.6.4-9.el9.x86_64 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: fmt-8.1.1-5.el9.x86_64 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: fuse-2.9.9-15.el9.x86_64 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-19T18:13:56.793 INFO:teuthology.orchestra.run.smithi192.stdout: jq-1.6-16.el9.x86_64 2023-12-19T18:13:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-12-19T18:13:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-9.0.0-10.el9.x86_64 2023-12-19T18:13:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: libarrow-doc-9.0.0-10.el9.noarch 2023-12-19T18:13:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs-devel-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: libcephfs2-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: libcephsqlite-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-19T18:13:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-19T18:13:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-19T18:13:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-19T18:13:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-19T18:13:56.794 INFO:teuthology.orchestra.run.smithi192.stdout: librados-devel-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: libradosstriper1-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-19T18:13:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: librgw2-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: libtool-2.4.6-45.el9.x86_64 2023-12-19T18:13:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-19T18:13:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-12-19T18:13:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-12-19T18:13:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-19T18:13:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-19T18:13:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-19T18:13:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-19T18:13:56.795 INFO:teuthology.orchestra.run.smithi192.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: parquet-libs-9.0.0-10.el9.x86_64 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-13.11-1.el9.x86_64 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: postgresql-server-13.11-1.el9.x86_64 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: python3-attrs-20.3.0-7.el9.noarch 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-argparse-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ceph-common-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cephfs-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.796 INFO:teuthology.orchestra.run.smithi192.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-19T18:13:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-19T18:13:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-19T18:13:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-19T18:13:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cmd2-2.4.2-2.el9.noarch 2023-12-19T18:13:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: python3-colorama-0.4.4-12.el9.noarch 2023-12-19T18:13:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-19T18:13:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-19T18:13:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-19T18:13:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-19T18:13:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-19T18:13:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-19T18:13:56.797 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-19T18:13:56.798 INFO:teuthology.orchestra.run.smithi192.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-ply-3.11-14.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rados-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rbd-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-19T18:13:56.799 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rgw-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-fuse-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-mirror-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: rbd-nbd-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:13:56.800 INFO:teuthology.orchestra.run.smithi192.stdout: re2-1:20211101-3.el9.x86_64 2023-12-19T18:13:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-19T18:13:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-19T18:13:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: uuid-1.6.2-55.el9.x86_64 2023-12-19T18:13:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-12-19T18:13:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-19T18:13:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-12-19T18:13:56.801 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:13:56.801 INFO:teuthology.orchestra.run.smithi192.stdout:Complete! 2023-12-19T18:13:57.265 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:18.2.1-132.g568df92b.el9.x86_64 134/169 2023-12-19T18:13:57.312 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:18.2.1-132.g568df92b.el9.x86_64 135/169 2023-12-19T18:13:57.352 DEBUG:teuthology.parallel:result is None 2023-12-19T18:13:57.367 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:18.2.1-132.g568df92b.el9.x86_64 136/169 2023-12-19T18:13:57.427 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xfsprogs-devel-5.19.0-4.el9.x86_64 137/169 2023-12-19T18:13:57.486 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : elfutils-libelf-devel-0.190-1.el9.x86_64 138/169 2023-12-19T18:13:57.810 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-shell-2:18.2.1-132.g568df92b.el9.x86_64 139/169 2023-12-19T18:13:57.945 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-contrib-13.11-1.el9.x86_64 140/169 2023-12-19T18:13:57.984 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:18.2.1-132.g568df92b.el9.x86_64 141/169 2023-12-19T18:13:58.019 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:18.2.1-132.g568df92b.el9.x86_6 142/169 2023-12-19T18:13:58.102 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : elfutils-debuginfod-client-0.190-1.el9.x86_64 143/169 2023-12-19T18:13:58.164 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libacl-devel-2.3.1-4.el9.x86_64 144/169 2023-12-19T18:13:58.204 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : acl-2.3.1-4.el9.x86_64 145/169 2023-12-19T18:13:58.255 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-team-1:1.45.7-1.el9.x86_64 146/169 2023-12-19T18:13:58.303 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : NetworkManager-tui-1:1.45.7-1.el9.x86_64 147/169 2023-12-19T18:13:59.329 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 148/169 2023-12-19T18:13:59.384 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : postgresql-server-13.11-1.el9.x86_64 148/169 2023-12-19T18:13:59.460 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: postgresql-server-13.11-1.el9.x86_64 148/169 2023-12-19T18:14:00.926 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephfs-top-2:18.2.1-132.g568df92b.el9.noarch 149/169 2023-12-19T18:14:01.055 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openssl-devel-1:3.0.7-25.el9.x86_64 150/169 2023-12-19T18:14:01.123 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : xfsdump-3.1.12-4.el9.x86_64 151/169 2023-12-19T18:14:01.199 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 152/169 2023-12-19T18:14:01.369 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : flex-2.6.4-9.el9.x86_64 153/169 2023-12-19T18:14:01.411 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : bison-3.7.4-5.el9.x86_64 154/169 2023-12-19T18:14:01.443 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-tui-1:1.44.0-2.el9.x86_64 155/169 2023-12-19T18:14:01.478 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-debuginfod-client-0.189-3.el9.x86_64 156/169 2023-12-19T18:14:01.520 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libs-0.189-3.el9.x86_64 157/169 2023-12-19T18:14:01.520 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/169 2023-12-19T18:14:01.661 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/169 2023-12-19T18:14:01.705 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libacl-devel-2.3.1-3.el9.x86_64 159/169 2023-12-19T18:14:01.774 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-1:3.0.7-24.el9.x86_64 160/169 2023-12-19T18:14:01.806 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : acl-2.3.1-3.el9.x86_64 161/169 2023-12-19T18:14:01.807 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 162/169 2023-12-19T18:14:01.944 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 162/169 2023-12-19T18:14:01.983 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-team-1:1.44.0-2.el9.x86_64 163/169 2023-12-19T18:14:02.025 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 164/169 2023-12-19T18:14:02.025 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-1:1.44.0-2.el9.x86_64 164/169 2023-12-19T18:14:02.375 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-1:1.44.0-2.el9.x86_64 164/169 2023-12-19T18:14:02.375 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 165/169 2023-12-19T18:14:02.524 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: NetworkManager-libnm-1:1.44.0-2.el9.x86_64 165/169 2023-12-19T18:14:02.561 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : openssl-libs-1:3.0.7-24.el9.x86_64 166/169 2023-12-19T18:14:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libacl-2.3.1-3.el9.x86_64 167/169 2023-12-19T18:14:02.645 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : elfutils-libelf-0.189-3.el9.x86_64 168/169 2023-12-19T18:14:02.776 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : xfsdump-3.1.12-3.el9.x86_64 169/169 2023-12-19T18:14:03.063 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: xfsdump-3.1.12-3.el9.x86_64 169/169 2023-12-19T18:14:03.063 INFO:teuthology.orchestra.run.smithi027.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-12-19T18:14:03.063 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:18.2.1-132.g568df92b.el9.x86_64 1/169 2023-12-19T18:14:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:18.2.1-132.g568df92b.el9.x86_64 2/169 2023-12-19T18:14:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:18.2.1-132.g568df92b.el9.x86_64 3/169 2023-12-19T18:14:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:18.2.1-132.g568df92b.el9.x86_64 4/169 2023-12-19T18:14:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:18.2.1-132.g568df9 5/169 2023-12-19T18:14:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:18.2.1-132.g568df92b.el9.x86_64 6/169 2023-12-19T18:14:07.740 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:18.2.1-132.g568df92b.el9.x86_64 7/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:18.2.1-132.g568df92b.el9.x86_64 8/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:18.2.1-132.g568df92b.el9.x86_64 9/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:18.2.1-132.g568df92b.el9.x86_64 10/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:18.2.1-132.g568df92b.el9.x86_64 11/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:18.2.1-132.g568df92b.el9.x86_64 12/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-mirror-2:18.2.1-132.g568df92b.el9.x86_64 13/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-shell-2:18.2.1-132.g568df92b.el9.x86_64 14/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:18.2.1-132.g568df92b.el9.x86_6 15/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:18.2.1-132.g568df92b.el9.x86_64 16/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:18.2.1-132.g568df92b.el9.x86_64 17/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:18.2.1-132.g568df92b.el9.x86_64 18/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:18.2.1-132.g568df92b.el9.x86_ 19/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:18.2.1-132.g568df92b.el9.x86_64 20/169 2023-12-19T18:14:07.741 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:18.2.1-132.g568df92b.el9 21/169 2023-12-19T18:14:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:18.2.1-132.g568df92b.el9.x 22/169 2023-12-19T18:14:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:18.2.1-132.g568df92b.el9.x86_64 23/169 2023-12-19T18:14:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:18.2.1-132.g568df92b.el9.x86_64 24/169 2023-12-19T18:14:07.742 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:18.2.1-132.g568df92b.el9.x86_64 25/169 2023-12-19T18:14:07.756 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:18.2.1-132.g568df92b.el9.x86_64 26/169 2023-12-19T18:14:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:18.2.1-132.g568df92b.el9.x86_64 27/169 2023-12-19T18:14:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:18.2.1-132.g568df92b.el9.x86_64 28/169 2023-12-19T18:14:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:18.2.1-132.g568df92b.el9.x86_64 29/169 2023-12-19T18:14:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:18.2.1-132.g568df92b.e 30/169 2023-12-19T18:14:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:18.2.1-132.g568df92b.el9.noar 31/169 2023-12-19T18:14:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:18.2.1-132.g568df92b.el9.no 32/169 2023-12-19T18:14:07.757 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.1-132.g568d 33/169 2023-12-19T18:14:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:18.2.1-132.g568df92b.el9 34/169 2023-12-19T18:14:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:18.2.1-132.g568df92b.el9.noarch 35/169 2023-12-19T18:14:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:18.2.1-132.g568df92b.el 36/169 2023-12-19T18:14:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-volume-2:18.2.1-132.g568df92b.el9.noarch 37/169 2023-12-19T18:14:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:18.2.1-132.g568df92b.el9.noarch 38/169 2023-12-19T18:14:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephfs-top-2:18.2.1-132.g568df92b.el9.noarch 39/169 2023-12-19T18:14:07.758 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : automake-1.16.2-8.el9.noarch 40/169 2023-12-19T18:14:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : bison-3.7.4-5.el9.x86_64 41/169 2023-12-19T18:14:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 42/169 2023-12-19T18:14:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-devel-0.190-1.el9.x86_64 43/169 2023-12-19T18:14:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flex-2.6.4-9.el9.x86_64 44/169 2023-12-19T18:14:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 45/169 2023-12-19T18:14:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 46/169 2023-12-19T18:14:07.759 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 47/169 2023-12-19T18:14:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : jq-1.6-16.el9.x86_64 48/169 2023-12-19T18:14:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 49/169 2023-12-19T18:14:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 50/169 2023-12-19T18:14:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 51/169 2023-12-19T18:14:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 52/169 2023-12-19T18:14:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 53/169 2023-12-19T18:14:07.760 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libuuid-devel-2.37.4-15.el9.x86_64 54/169 2023-12-19T18:14:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 55/169 2023-12-19T18:14:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 56/169 2023-12-19T18:14:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : oniguruma-6.9.6-1.el9.5.x86_64 57/169 2023-12-19T18:14:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 58/169 2023-12-19T18:14:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 59/169 2023-12-19T18:14:07.761 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-13.11-1.el9.x86_64 60/169 2023-12-19T18:14:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-contrib-13.11-1.el9.x86_64 61/169 2023-12-19T18:14:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-private-libs-13.11-1.el9.x86_64 62/169 2023-12-19T18:14:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : postgresql-server-13.11-1.el9.x86_64 63/169 2023-12-19T18:14:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 64/169 2023-12-19T18:14:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 65/169 2023-12-19T18:14:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.11.3-4.el9.noarch 66/169 2023-12-19T18:14:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 67/169 2023-12-19T18:14:07.762 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 68/169 2023-12-19T18:14:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 69/169 2023-12-19T18:14:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 70/169 2023-12-19T18:14:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 71/169 2023-12-19T18:14:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 72/169 2023-12-19T18:14:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 73/169 2023-12-19T18:14:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 74/169 2023-12-19T18:14:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 75/169 2023-12-19T18:14:07.763 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 76/169 2023-12-19T18:14:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 77/169 2023-12-19T18:14:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 78/169 2023-12-19T18:14:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 79/169 2023-12-19T18:14:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 80/169 2023-12-19T18:14:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 81/169 2023-12-19T18:14:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 82/169 2023-12-19T18:14:07.764 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 83/169 2023-12-19T18:14:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsprogs-devel-5.19.0-4.el9.x86_64 84/169 2023-12-19T18:14:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 85/169 2023-12-19T18:14:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : zlib-devel-1.2.11-40.el9.x86_64 86/169 2023-12-19T18:14:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 87/169 2023-12-19T18:14:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-11.4.1-2.3.el9.x86_64 88/169 2023-12-19T18:14:07.765 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-11.4.1-2.3.el9.x86_64 89/169 2023-12-19T18:14:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 90/169 2023-12-19T18:14:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 91/169 2023-12-19T18:14:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ply-3.11-14.el9.noarch 92/169 2023-12-19T18:14:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 93/169 2023-12-19T18:14:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.25.1-7.el9.noarch 94/169 2023-12-19T18:14:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.26.5-4.el9.noarch 95/169 2023-12-19T18:14:07.766 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-devel-1:3.0.7-25.el9.x86_64 96/169 2023-12-19T18:14:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-devel-3.9.17-2.el9.x86_64 97/169 2023-12-19T18:14:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 98/169 2023-12-19T18:14:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 99/169 2023-12-19T18:14:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 100/169 2023-12-19T18:14:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 101/169 2023-12-19T18:14:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 102/169 2023-12-19T18:14:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 103/169 2023-12-19T18:14:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 104/169 2023-12-19T18:14:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-asyncssh-2.13.2-3.el9.noarch 105/169 2023-12-19T18:14:07.767 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 106/169 2023-12-19T18:14:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 107/169 2023-12-19T18:14:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 108/169 2023-12-19T18:14:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 109/169 2023-12-19T18:14:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 110/169 2023-12-19T18:14:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 111/169 2023-12-19T18:14:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 112/169 2023-12-19T18:14:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:2.23.4-1.el9.noarch 113/169 2023-12-19T18:14:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 114/169 2023-12-19T18:14:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 115/169 2023-12-19T18:14:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 116/169 2023-12-19T18:14:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 117/169 2023-12-19T18:14:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 118/169 2023-12-19T18:14:07.768 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 119/169 2023-12-19T18:14:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 120/169 2023-12-19T18:14:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 121/169 2023-12-19T18:14:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 122/169 2023-12-19T18:14:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 123/169 2023-12-19T18:14:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 124/169 2023-12-19T18:14:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 125/169 2023-12-19T18:14:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 126/169 2023-12-19T18:14:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 127/169 2023-12-19T18:14:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 128/169 2023-12-19T18:14:07.769 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 129/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 130/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 131/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 132/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 133/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 134/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 135/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 136/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 137/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : re2-1:20211101-3.el9.x86_64 138/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 139/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:18.2.1-132.g568df92b.el9.x86_64 140/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 141/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:18.2.1-132.g568df92b.el9.x86_64 142/169 2023-12-19T18:14:07.770 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 143/169 2023-12-19T18:14:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libacl-devel-2.3.1-4.el9.x86_64 144/169 2023-12-19T18:14:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libacl-devel-2.3.1-3.el9.x86_64 145/169 2023-12-19T18:14:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : acl-2.3.1-4.el9.x86_64 146/169 2023-12-19T18:14:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : acl-2.3.1-3.el9.x86_64 147/169 2023-12-19T18:14:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-debuginfod-client-0.190-1.el9.x86_64 148/169 2023-12-19T18:14:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el9.x86_64 149/169 2023-12-19T18:14:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.190-1.el9.x86_64 150/169 2023-12-19T18:14:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libelf-0.189-3.el9.x86_64 151/169 2023-12-19T18:14:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.190-1.el9.x86_64 152/169 2023-12-19T18:14:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : elfutils-libs-0.189-3.el9.x86_64 153/169 2023-12-19T18:14:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libacl-2.3.1-4.el9.x86_64 154/169 2023-12-19T18:14:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libacl-2.3.1-3.el9.x86_64 155/169 2023-12-19T18:14:07.771 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsdump-3.1.12-4.el9.x86_64 156/169 2023-12-19T18:14:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xfsdump-3.1.12-3.el9.x86_64 157/169 2023-12-19T18:14:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.45.7-1.el9.x86_64 158/169 2023-12-19T18:14:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-1:1.44.0-2.el9.x86_64 159/169 2023-12-19T18:14:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.45.7-1.el9.x86_64 160/169 2023-12-19T18:14:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-libnm-1:1.44.0-2.el9.x86_64 161/169 2023-12-19T18:14:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.45.7-1.el9.x86_64 162/169 2023-12-19T18:14:07.772 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-team-1:1.44.0-2.el9.x86_64 163/169 2023-12-19T18:14:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.45.7-1.el9.x86_64 164/169 2023-12-19T18:14:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : NetworkManager-tui-1:1.44.0-2.el9.x86_64 165/169 2023-12-19T18:14:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.0.7-25.el9.x86_64 166/169 2023-12-19T18:14:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-1:3.0.7-24.el9.x86_64 167/169 2023-12-19T18:14:07.773 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.0.7-25.el9.x86_64 168/169 2023-12-19T18:14:10.283 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openssl-libs-1:3.0.7-24.el9.x86_64 169/169 2023-12-19T18:14:10.284 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:10.284 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2023-12-19T18:14:10.284 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-1:1.45.7-1.el9.x86_64 2023-12-19T18:14:10.284 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-libnm-1:1.45.7-1.el9.x86_64 2023-12-19T18:14:10.284 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-team-1:1.45.7-1.el9.x86_64 2023-12-19T18:14:10.284 INFO:teuthology.orchestra.run.smithi027.stdout: NetworkManager-tui-1:1.45.7-1.el9.x86_64 2023-12-19T18:14:10.284 INFO:teuthology.orchestra.run.smithi027.stdout: acl-2.3.1-4.el9.x86_64 2023-12-19T18:14:10.284 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-debuginfod-client-0.190-1.el9.x86_64 2023-12-19T18:14:10.284 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-0.190-1.el9.x86_64 2023-12-19T18:14:10.285 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libs-0.190-1.el9.x86_64 2023-12-19T18:14:10.285 INFO:teuthology.orchestra.run.smithi027.stdout: libacl-2.3.1-4.el9.x86_64 2023-12-19T18:14:10.285 INFO:teuthology.orchestra.run.smithi027.stdout: libacl-devel-2.3.1-4.el9.x86_64 2023-12-19T18:14:10.285 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.285 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.285 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-1:3.0.7-25.el9.x86_64 2023-12-19T18:14:10.285 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-libs-1:3.0.7-25.el9.x86_64 2023-12-19T18:14:10.285 INFO:teuthology.orchestra.run.smithi027.stdout: xfsdump-3.1.12-4.el9.x86_64 2023-12-19T18:14:10.285 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: automake-1.16.2-8.el9.noarch 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: bison-3.7.4-5.el9.x86_64 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: boost-program-options-1.75.0-8.el9.x86_64 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:14:10.286 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-volume-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-mirror-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-shell-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: cephfs-top-2:18.2.1-132.g568df92b.el9.noarch 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: elfutils-libelf-devel-0.190-1.el9.x86_64 2023-12-19T18:14:10.287 INFO:teuthology.orchestra.run.smithi027.stdout: flex-2.6.4-9.el9.x86_64 2023-12-19T18:14:10.288 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-3.0.4-8.el9.x86_64 2023-12-19T18:14:10.288 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2023-12-19T18:14:10.288 INFO:teuthology.orchestra.run.smithi027.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2023-12-19T18:14:10.288 INFO:teuthology.orchestra.run.smithi027.stdout: fmt-8.1.1-5.el9.x86_64 2023-12-19T18:14:10.288 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-2.9.9-15.el9.x86_64 2023-12-19T18:14:10.288 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2023-12-19T18:14:10.288 INFO:teuthology.orchestra.run.smithi027.stdout: jq-1.6-16.el9.x86_64 2023-12-19T18:14:10.288 INFO:teuthology.orchestra.run.smithi027.stdout: libaio-devel-0.3.111-13.el9.x86_64 2023-12-19T18:14:10.288 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-9.0.0-10.el9.x86_64 2023-12-19T18:14:10.288 INFO:teuthology.orchestra.run.smithi027.stdout: libarrow-doc-9.0.0-10.el9.noarch 2023-12-19T18:14:10.288 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.289 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.289 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.289 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-11.4.1-2.3.el9.x86_64 2023-12-19T18:14:10.289 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.7-2.el9.x86_64 2023-12-19T18:14:10.289 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.12.1-1.el9.x86_64 2023-12-19T18:14:10.289 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-11.4.1-2.3.el9.x86_64 2023-12-19T18:14:10.289 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.11.0-7.el9.x86_64 2023-12-19T18:14:10.289 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.289 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-1.6.1-102.el9.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: libtool-2.4.6-45.el9.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.6.2-1.el9.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: libuuid-devel-2.37.4-15.el9.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.12.0-6.el9.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma-6.9.6-1.el9.5.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.21-2.el9.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: openssl-devel-1:3.0.7-25.el9.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: parquet-libs-9.0.0-10.el9.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-13.11-1.el9.x86_64 2023-12-19T18:14:10.290 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-contrib-13.11-1.el9.x86_64 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-private-libs-13.11-1.el9.x86_64 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: postgresql-server-13.11-1.el9.x86_64 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: python3-asyncssh-2.13.2-3.el9.noarch 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: python3-attrs-20.3.0-7.el9.noarch 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.9.1-2.el9.noarch 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-4.2.4-1.el9.noarch 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2021.10.8-2.el9.noarch 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.14.5-5.el9.x86_64 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.6.0-4.el9.noarch 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2023-12-19T18:14:10.291 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cmd2-2.4.2-2.el9.noarch 2023-12-19T18:14:10.292 INFO:teuthology.orchestra.run.smithi027.stdout: python3-colorama-0.4.4-12.el9.noarch 2023-12-19T18:14:10.292 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2023-12-19T18:14:10.292 INFO:teuthology.orchestra.run.smithi027.stdout: python3-devel-3.9.17-2.el9.x86_64 2023-12-19T18:14:10.292 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:2.23.4-1.el9.noarch 2023-12-19T18:14:10.292 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-8.2.1-3.el9.noarch 2023-12-19T18:14:10.292 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2023-12-19T18:14:10.292 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2023-12-19T18:14:10.292 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2023-12-19T18:14:10.292 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2023-12-19T18:14:10.292 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.11.3-4.el9.noarch 2023-12-19T18:14:10.292 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-16.el9.noarch 2023-12-19T18:14:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-2.0-4.el9.noarch 2023-12-19T18:14:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt-2.4.0-1.el9.noarch 2023-12-19T18:14:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2023-12-19T18:14:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2023-12-19T18:14:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-21.el9.noarch 2023-12-19T18:14:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.1.4-6.el9.noarch 2023-12-19T18:14:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2023-12-19T18:14:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2023-12-19T18:14:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: python3-natsort-7.1.1-5.el9.noarch 2023-12-19T18:14:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2023-12-19T18:14:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2023-12-19T18:14:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2023-12-19T18:14:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.4.2-3.el9.noarch 2023-12-19T18:14:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ply-3.11-14.el9.noarch 2023-12-19T18:14:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-3.1.0-2.el9.noarch 2023-12-19T18:14:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-27.el9.noarch 2023-12-19T18:14:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2023-12-19T18:14:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2023-12-19T18:14:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2023-12-19T18:14:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.20-6.el9.noarch 2023-12-19T18:14:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2023-12-19T18:14:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2021.1-5.el9.noarch 2023-12-19T18:14:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-16.el9.noarch 2023-12-19T18:14:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.25.1-7.el9.noarch 2023-12-19T18:14:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2023-12-19T18:14:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.5.1-5.el9.noarch 2023-12-19T18:14:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.9-2.el9.noarch 2023-12-19T18:14:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.6.2-8.el9.x86_64 2023-12-19T18:14:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-5.0.0-2.el9.noarch 2023-12-19T18:14:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-toml-0.10.2-6.el9.noarch 2023-12-19T18:14:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2023-12-19T18:14:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.26.5-4.el9.noarch 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.7-6.el9.noarch 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:18.2.1-132.g568df92b.el9.x86_64 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: re2-1:20211101-3.el9.x86_64 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: socat-1.7.4.1-5.el9.x86_64 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: thrift-0.15.0-2.el9.x86_64 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: uuid-1.6.2-55.el9.x86_64 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: xfsprogs-devel-5.19.0-4.el9.x86_64 2023-12-19T18:14:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2023-12-19T18:14:10.297 INFO:teuthology.orchestra.run.smithi027.stdout: zlib-devel-1.2.11-40.el9.x86_64 2023-12-19T18:14:10.297 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:10.297 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2023-12-19T18:14:10.895 DEBUG:teuthology.parallel:result is None 2023-12-19T18:14:10.896 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-19T18:14:11.065 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-19T18:14:11.130 INFO:teuthology.orchestra.run.smithi027.stdout:18.2.1-132.g568df92b.el9 2023-12-19T18:14:11.131 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-132.g568df92b.el9 2023-12-19T18:14:11.131 INFO:teuthology.task.install:The correct ceph version 18.2.1-132.g568df92b is installed. 2023-12-19T18:14:11.133 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-19T18:14:11.359 DEBUG:teuthology.orchestra.run.smithi192:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-19T18:14:11.434 INFO:teuthology.orchestra.run.smithi192.stdout:18.2.1-132.g568df92b.el9 2023-12-19T18:14:11.435 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-132.g568df92b.el9 2023-12-19T18:14:11.435 INFO:teuthology.task.install:The correct ceph version 18.2.1-132.g568df92b is installed. 2023-12-19T18:14:11.436 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-19T18:14:11.437 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:11.437 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-19T18:14:11.522 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:11.523 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-19T18:14:11.624 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-19T18:14:11.624 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:11.624 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2023-12-19T18:14:11.708 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-19T18:14:11.859 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:11.859 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/daemon-helper 2023-12-19T18:14:11.957 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-19T18:14:12.097 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-19T18:14:12.098 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:12.098 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-19T18:14:12.186 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-19T18:14:12.307 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:12.307 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-19T18:14:12.405 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-19T18:14:12.549 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-19T18:14:12.549 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:12.549 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/stdin-killer 2023-12-19T18:14:12.635 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-19T18:14:12.756 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:12.756 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/usr/bin/stdin-killer 2023-12-19T18:14:12.857 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-19T18:14:12.948 INFO:teuthology.run_tasks:Running task ceph... 2023-12-19T18:14:13.016 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-12-19T18:14:13.017 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /var/log/ceph 2023-12-19T18:14:13.023 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 777 /var/log/ceph 2023-12-19T18:14:13.118 INFO:tasks.ceph:Disabling ceph logrotate... 2023-12-19T18:14:13.118 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-19T18:14:13.122 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-19T18:14:13.218 INFO:tasks.ceph:Creating extra log directories... 2023-12-19T18:14:13.218 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-19T18:14:13.223 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-19T18:14:13.325 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-12-19T18:14:13.325 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-12-19T18:14:13.326 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-12-19T18:14:13.326 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:13.326 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-19T18:14:13.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-19T18:14:13.647 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-19T18:14:13.729 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:13.729 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-19T18:14:13.787 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-19T18:14:14.069 DEBUG:teuthology.orchestra.run.smithi192:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-12-19T18:14:14.179 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-12-19T18:14:14.179 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-12-19T18:14:14.179 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229', 'branch': 'wip-vshankar-testing1-2023-12-18-1207-reef-2', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7497229', 'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing1-2023-12-18-1207-reef-2'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing1-2023-12-18-1207-reef-2', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 430, 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e', 'sleep_before_teardown': 0, 'subset': '301/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing1-2023-12-18-1207-reef-2', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_568df92b7e50b2108bce4c77c393ff294d6a5e5e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi027.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKiwWr0Trl+pDhPf5nNOLJ6oFgL3wEGSWpZ+bLW0N63dQX6maGQvLOlhSSCJ/rdZIk1z6+T2Pv8zhBcahDNw244=', 'smithi192.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLG3+8ZD65WGw+gpZxw4zFCbHyN3UXqVOhaEfVSHFoGNnoGBQCbRrunv60cTWg5kiZC1Dx8YHbcYlR5C6vpK1TI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'flavor': 'default', 'kdb': True, 'sha1': 'cd68252d7b4d7802f3cae7f5945b38e9b20dad4c'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}}, {'ceph': {'create_rbd_pool': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL', 'PG_RECOVERY_FULL', 'PG_DEGRADED'], 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_full']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73', 'timestamp': '2023-12-19_13:22:41', 'tube': 'smithi', 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1485078'} 2023-12-19T18:14:14.180 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-19T18:14:14.186 DEBUG:teuthology.orchestra.run.smithi192:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-19T18:14:14.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m0777 -- /var/run/ceph 2023-12-19T18:14:14.247 DEBUG:teuthology.orchestra.run.smithi192:> sudo install -d -m0777 -- /var/run/ceph 2023-12-19T18:14:14.343 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:14.343 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2023-12-19T18:14:14.389 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-19T18:14:14.389 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 2023-12-19T18:14:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-19T18:14:14.473 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T18:14:14.473 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 631 Links: 1 2023-12-19T18:14:14.473 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T18:14:14.474 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-12-19T18:14:14.474 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-12-19 18:14:03.571886829 +0000 2023-12-19T18:14:14.474 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-12-19 18:11:07.318997052 +0000 2023-12-19T18:14:14.474 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-12-19 18:11:07.318997052 +0000 2023-12-19T18:14:14.474 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-12-19 18:11:07.318997052 +0000 2023-12-19T18:14:14.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-19T18:14:14.599 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-12-19T18:14:14.599 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-12-19T18:14:14.599 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000369688 s, 1.4 MB/s 2023-12-19T18:14:14.603 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-19T18:14:14.648 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 2023-12-19T18:14:14.730 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-19T18:14:14.731 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T18:14:14.731 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 653 Links: 1 2023-12-19T18:14:14.731 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T18:14:14.731 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-12-19T18:14:14.731 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-12-19 18:14:03.572886803 +0000 2023-12-19T18:14:14.731 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-12-19 18:11:08.380969244 +0000 2023-12-19T18:14:14.731 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-12-19 18:11:08.380969244 +0000 2023-12-19T18:14:14.731 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-12-19 18:11:08.380969244 +0000 2023-12-19T18:14:14.731 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-19T18:14:14.847 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-12-19T18:14:14.847 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-12-19T18:14:14.847 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.00029731 s, 1.7 MB/s 2023-12-19T18:14:14.850 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-19T18:14:14.897 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 2023-12-19T18:14:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-19T18:14:14.982 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T18:14:14.982 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 672 Links: 1 2023-12-19T18:14:14.982 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T18:14:14.982 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-12-19T18:14:14.982 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-12-19 18:14:03.573886777 +0000 2023-12-19T18:14:14.983 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-12-19 18:11:09.442941437 +0000 2023-12-19T18:14:14.983 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-12-19 18:11:09.442941437 +0000 2023-12-19T18:14:14.983 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-12-19 18:11:09.442941437 +0000 2023-12-19T18:14:14.983 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-19T18:14:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-12-19T18:14:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-12-19T18:14:15.099 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000297558 s, 1.7 MB/s 2023-12-19T18:14:15.101 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-19T18:14:15.149 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 2023-12-19T18:14:15.231 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-19T18:14:15.231 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T18:14:15.231 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 5h/5d Inode: 691 Links: 1 2023-12-19T18:14:15.231 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T18:14:15.231 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2023-12-19T18:14:15.231 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2023-12-19 18:14:03.574886751 +0000 2023-12-19T18:14:15.231 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2023-12-19 18:11:10.531912923 +0000 2023-12-19T18:14:15.231 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2023-12-19 18:11:10.531912923 +0000 2023-12-19T18:14:15.231 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: 2023-12-19 18:11:10.531912923 +0000 2023-12-19T18:14:15.231 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-19T18:14:15.346 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-12-19T18:14:15.346 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2023-12-19T18:14:15.346 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.00035827 s, 1.4 MB/s 2023-12-19T18:14:15.349 DEBUG:teuthology.orchestra.run.smithi027:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-19T18:14:15.394 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-19T18:14:15.394 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:15.394 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/scratch_devs of=/dev/stdout 2023-12-19T18:14:15.455 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-19T18:14:15.455 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_1 2023-12-19T18:14:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-19T18:14:15.508 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T18:14:15.508 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 633 Links: 1 2023-12-19T18:14:15.509 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T18:14:15.509 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-12-19T18:14:15.509 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-12-19 18:13:50.568217011 +0000 2023-12-19T18:14:15.509 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-12-19 18:10:32.565147406 +0000 2023-12-19T18:14:15.509 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-12-19 18:10:32.565147406 +0000 2023-12-19T18:14:15.509 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2023-12-19 18:10:32.565147406 +0000 2023-12-19T18:14:15.509 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-19T18:14:15.601 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-12-19T18:14:15.601 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-12-19T18:14:15.601 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.00035312 s, 1.4 MB/s 2023-12-19T18:14:15.607 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-19T18:14:15.662 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_2 2023-12-19T18:14:15.712 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-19T18:14:15.712 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T18:14:15.712 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 653 Links: 1 2023-12-19T18:14:15.712 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T18:14:15.712 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-12-19T18:14:15.712 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-12-19 18:13:50.568217011 +0000 2023-12-19T18:14:15.712 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-12-19 18:10:33.627126381 +0000 2023-12-19T18:14:15.712 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-12-19 18:10:33.627126381 +0000 2023-12-19T18:14:15.712 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2023-12-19 18:10:33.627126381 +0000 2023-12-19T18:14:15.713 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-19T18:14:15.794 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-12-19T18:14:15.794 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-12-19T18:14:15.794 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000341434 s, 1.5 MB/s 2023-12-19T18:14:15.797 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-19T18:14:15.851 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_3 2023-12-19T18:14:15.903 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-19T18:14:15.904 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T18:14:15.904 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 669 Links: 1 2023-12-19T18:14:15.904 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T18:14:15.904 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-12-19T18:14:15.904 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-12-19 18:13:50.569216991 +0000 2023-12-19T18:14:15.904 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-12-19 18:10:34.631106504 +0000 2023-12-19T18:14:15.904 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-12-19 18:10:34.631106504 +0000 2023-12-19T18:14:15.904 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2023-12-19 18:10:34.631106504 +0000 2023-12-19T18:14:15.904 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-19T18:14:15.994 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-12-19T18:14:15.994 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-12-19T18:14:15.994 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.00031522 s, 1.6 MB/s 2023-12-19T18:14:15.998 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-19T18:14:16.052 DEBUG:teuthology.orchestra.run.smithi192:> stat /dev/vg_nvme/lv_4 2023-12-19T18:14:16.101 INFO:teuthology.orchestra.run.smithi192.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-19T18:14:16.101 INFO:teuthology.orchestra.run.smithi192.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-19T18:14:16.101 INFO:teuthology.orchestra.run.smithi192.stdout:Device: 5h/5d Inode: 688 Links: 1 2023-12-19T18:14:16.101 INFO:teuthology.orchestra.run.smithi192.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-19T18:14:16.101 INFO:teuthology.orchestra.run.smithi192.stdout:Context: system_u:object_r:device_t:s0 2023-12-19T18:14:16.101 INFO:teuthology.orchestra.run.smithi192.stdout:Access: 2023-12-19 18:13:50.570216972 +0000 2023-12-19T18:14:16.101 INFO:teuthology.orchestra.run.smithi192.stdout:Modify: 2023-12-19 18:10:35.647086390 +0000 2023-12-19T18:14:16.101 INFO:teuthology.orchestra.run.smithi192.stdout:Change: 2023-12-19 18:10:35.647086390 +0000 2023-12-19T18:14:16.101 INFO:teuthology.orchestra.run.smithi192.stdout: Birth: 2023-12-19 18:10:35.647086390 +0000 2023-12-19T18:14:16.102 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-19T18:14:16.191 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records in 2023-12-19T18:14:16.191 INFO:teuthology.orchestra.run.smithi192.stderr:1+0 records out 2023-12-19T18:14:16.191 INFO:teuthology.orchestra.run.smithi192.stderr:512 bytes copied, 0.000320761 s, 1.6 MB/s 2023-12-19T18:14:16.194 DEBUG:teuthology.orchestra.run.smithi192:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-19T18:14:16.250 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-19T18:14:16.250 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-19T18:14:16.250 INFO:tasks.ceph:Generating config... 2023-12-19T18:14:16.251 INFO:tasks.ceph:[client] client mount timeout = 600 2023-12-19T18:14:16.251 INFO:tasks.ceph:[client] debug client = 20 2023-12-19T18:14:16.252 INFO:tasks.ceph:[client] debug ms = 1 2023-12-19T18:14:16.252 INFO:tasks.ceph:[client] pid file = 2023-12-19T18:14:16.252 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-12-19T18:14:16.252 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-12-19T18:14:16.252 INFO:tasks.ceph:[client.0] debug client = 20 2023-12-19T18:14:16.252 INFO:tasks.ceph:[client.0] debug objectcacher = 20 2023-12-19T18:14:16.252 INFO:tasks.ceph:[client.0] debug objecter = 20 2023-12-19T18:14:16.252 INFO:tasks.ceph:[client.1] debug client = 20 2023-12-19T18:14:16.252 INFO:tasks.ceph:[client.1] debug objectcacher = 20 2023-12-19T18:14:16.252 INFO:tasks.ceph:[client.1] debug objecter = 20 2023-12-19T18:14:16.252 INFO:tasks.ceph:[mds] debug mds = 20 2023-12-19T18:14:16.252 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-12-19T18:14:16.252 INFO:tasks.ceph:[mds] debug ms = 1 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mds] mds debug frag = True 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mgr] debug ms = 1 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mon] debug mon = 20 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mon] debug ms = 1 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mon] debug paxos = 20 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mon] mon osd backfillfull ratio = 0.6 2023-12-19T18:14:16.253 INFO:tasks.ceph:[mon] mon osd full ratio = 0.7 2023-12-19T18:14:16.254 INFO:tasks.ceph:[mon] mon osd nearfull ratio = 0.6 2023-12-19T18:14:16.254 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-12-19T18:14:16.254 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-12-19T18:14:16.254 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-12-19T18:14:16.254 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-12-19T18:14:16.254 INFO:tasks.ceph:[osd] debug ms = 1 2023-12-19T18:14:16.254 INFO:tasks.ceph:[osd] debug osd = 20 2023-12-19T18:14:16.254 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-12-19T18:14:16.254 INFO:tasks.ceph:[osd] memstore device bytes = 200000000 2023-12-19T18:14:16.254 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-12-19T18:14:16.254 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-12-19T18:14:16.254 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-12-19T18:14:16.254 INFO:tasks.ceph:[osd] osd failsafe full ratio = 1.0 2023-12-19T18:14:16.254 INFO:tasks.ceph:[osd] osd mon report interval = 5 2023-12-19T18:14:16.255 INFO:tasks.ceph:[osd] osd objectstore = memstore 2023-12-19T18:14:16.255 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-12-19T18:14:16.255 INFO:tasks.ceph:Setting up mon.a... 2023-12-19T18:14:16.255 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-12-19T18:14:16.366 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.keyring 2023-12-19T18:14:16.373 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-12-19T18:14:16.481 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-19T18:14:16.553 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.27'), ('mon.b', '172.21.15.192'), ('mon.c', '[v2:172.21.15.192:3301,v1:172.21.15.192:6790]')] 2023-12-19T18:14:16.554 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.27,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'memstore device bytes': 200000000, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 1.0, 'osd mon report interval': 5, 'osd objectstore': 'memstore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120, 'mon osd backfillfull ratio': 0.6, 'mon osd full ratio': 0.7, 'mon osd nearfull ratio': 0.6}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'client.0': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'client.1': {'debug client': 20, 'debug objectcacher': 20, 'debug objecter': 20}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-12-19T18:14:16.555 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:16.555 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-12-19T18:14:16.594 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.27 --add b 172.21.15.192 --addv c '[v2:172.21.15.192:3301,v1:172.21.15.192:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:16.740 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:16.740 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: generated fsid dc17deb5-5eca-4f5c-983d-094ffb50339f 2023-12-19T18:14:16.740 INFO:teuthology.orchestra.run.smithi027.stdout:setting min_mon_release = pacific 2023-12-19T18:14:16.740 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 0 2023-12-19T18:14:16.740 INFO:teuthology.orchestra.run.smithi027.stdout:fsid dc17deb5-5eca-4f5c-983d-094ffb50339f 2023-12-19T18:14:16.740 INFO:teuthology.orchestra.run.smithi027.stdout:last_changed 2023-12-19T18:14:16.741246+0000 2023-12-19T18:14:16.741 INFO:teuthology.orchestra.run.smithi027.stdout:created 2023-12-19T18:14:16.741246+0000 2023-12-19T18:14:16.741 INFO:teuthology.orchestra.run.smithi027.stdout:min_mon_release 16 (pacific) 2023-12-19T18:14:16.741 INFO:teuthology.orchestra.run.smithi027.stdout:election_strategy: 1 2023-12-19T18:14:16.741 INFO:teuthology.orchestra.run.smithi027.stdout:0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.a 2023-12-19T18:14:16.741 INFO:teuthology.orchestra.run.smithi027.stdout:1: [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] mon.b 2023-12-19T18:14:16.741 INFO:teuthology.orchestra.run.smithi027.stdout:2: [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] mon.c 2023-12-19T18:14:16.741 INFO:teuthology.orchestra.run.smithi027.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-12-19T18:14:16.743 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-12-19T18:14:16.796 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID dc17deb5-5eca-4f5c-983d-094ffb50339f... 2023-12-19T18:14:16.797 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-19T18:14:16.802 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-19T18:14:16.985 INFO:teuthology.orchestra.run.smithi192.stdout:[global] 2023-12-19T18:14:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: chdir = "" 2023-12-19T18:14:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-19T18:14:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: auth supported = cephx 2023-12-19T18:14:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: filestore xattr use omap = true 2023-12-19T18:14:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: mon clock drift allowed = 1.000 2023-12-19T18:14:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.985 INFO:teuthology.orchestra.run.smithi192.stdout: osd crush chooseleaf type = 0 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: auth debug = true 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on old message = true 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: ms die on bug = true 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: mon pg warn max object skew = 0 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: # disable pg_autoscaler by default for new pools 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default size = 2 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow primary affinity = true 2023-12-19T18:14:16.986 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd allow pg remap = true 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on legacy crush tunables = false 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on crush straw calc version zero = false 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on no sortbitwise = false 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on osd down out interval zero = false 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: mon warn on too few osds = false 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: mon_allow_pool_size_one = true 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: osd default data pool replay window = 5 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: mon allow pool delete = true 2023-12-19T18:14:16.987 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: mon cluster log file level = debug 2023-12-19T18:14:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: debug asserts on shutdown = true 2023-12-19T18:14:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: mon health detail to clog = false 2023-12-19T18:14:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: mon host = "172.21.15.27,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]" 2023-12-19T18:14:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: fsid = dc17deb5-5eca-4f5c-983d-094ffb50339f 2023-12-19T18:14:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.988 INFO:teuthology.orchestra.run.smithi192.stdout:[osd] 2023-12-19T18:14:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: osd journal size = 100 2023-12-19T18:14:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub load threshold = 5.0 2023-12-19T18:14:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: osd scrub max interval = 600 2023-12-19T18:14:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: osd mclock profile = high_recovery_ops 2023-12-19T18:14:16.988 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: osd recover clone overlap = true 2023-12-19T18:14:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: osd recovery max chunk = 1048576 2023-12-19T18:14:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug shutdown = true 2023-12-19T18:14:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug op order = true 2023-12-19T18:14:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug verify stray on activate = true 2023-12-19T18:14:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: osd open classes on start = true 2023-12-19T18:14:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug pg log writeout = true 2023-12-19T18:14:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.989 INFO:teuthology.orchestra.run.smithi192.stdout: osd deep scrub update digest min age = 30 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: osd map max advance = 10 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: journal zero on create = true 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: filestore ondisk finisher threads = 3 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: filestore apply finisher threads = 3 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: bdev debug aio = true 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: osd debug misdirected ops = true 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore block size = 96636764160 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: bluestore fsck on mount = True 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluefs = 20 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: debug bluestore = 20 2023-12-19T18:14:16.990 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: debug osd = 20 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: debug rocksdb = 10 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: memstore device bytes = 200000000 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd backfillfull_ratio = 0.85 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd full ratio = 0.9 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd nearfull ratio = 0.8 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: osd failsafe full ratio = 1.0 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: osd mon report interval = 5 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: osd objectstore = memstore 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr] 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: debug mgr = 20 2023-12-19T18:14:16.991 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: mgr/telemetry/nag = false 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout:[mon] 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: debug mon = 20 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: debug paxos = 20 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: debug auth = 20 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: mon data avail warn = 5 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: mon mgr mkfs grace = 240 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min pgs per osd = 4 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd reporter subtree level = osd 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd prime pg temp = true 2023-12-19T18:14:16.992 INFO:teuthology.orchestra.run.smithi192.stdout: mon reweight min bytes per osd = 10 2023-12-19T18:14:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-19T18:14:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: auth mon ticket ttl = 660 # 11m 2023-12-19T18:14:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: auth service ticket ttl = 240 # 4m 2023-12-19T18:14:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: # don't complain about insecure global_id in the test suite 2023-12-19T18:14:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-19T18:14:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-19T18:14:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.993 INFO:teuthology.orchestra.run.smithi192.stdout: # 1m isn't quite enough 2023-12-19T18:14:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: mon_down_mkfs_grace = 2m 2023-12-19T18:14:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: mon_warn_on_filestore_osds = false 2023-12-19T18:14:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: mon op complaint time = 120 2023-12-19T18:14:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd backfillfull ratio = 0.6 2023-12-19T18:14:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd full ratio = 0.7 2023-12-19T18:14:16.994 INFO:teuthology.orchestra.run.smithi192.stdout: mon osd nearfull ratio = 0.6 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout:[client] 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: rgw cache enabled = true 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable ops log = true 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: rgw enable usage log = true 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: client mount timeout = 600 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: pid file = "" 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 900 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 900 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.a] 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.b] 2023-12-19T18:14:16.995 INFO:teuthology.orchestra.run.smithi192.stdout:[mon.c] 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.c] 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.d] 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout:[client.0] 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: debug objectcacher = 20 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: debug objecter = 20 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout:[client.1] 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: debug client = 20 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: debug objectcacher = 20 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: debug objecter = 20 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout:[mds] 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds = 20 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: debug mds balancer = 20 2023-12-19T18:14:16.996 INFO:teuthology.orchestra.run.smithi192.stdout: debug ms = 1 2023-12-19T18:14:16.997 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug frag = True 2023-12-19T18:14:16.997 INFO:teuthology.orchestra.run.smithi192.stdout: mds debug scatterstat = True 2023-12-19T18:14:16.997 INFO:teuthology.orchestra.run.smithi192.stdout: mds op complaint time = 180 2023-12-19T18:14:16.997 INFO:teuthology.orchestra.run.smithi192.stdout: mds verify scatter = True 2023-12-19T18:14:16.997 INFO:teuthology.orchestra.run.smithi192.stdout: osd op complaint time = 180 2023-12-19T18:14:16.997 INFO:teuthology.orchestra.run.smithi192.stdout: rados mon op timeout = 900 2023-12-19T18:14:16.997 INFO:teuthology.orchestra.run.smithi192.stdout: rados osd op timeout = 900 2023-12-19T18:14:16.997 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: chdir = "" 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: auth supported = cephx 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: filestore xattr use omap = true 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: mon clock drift allowed = 1.000 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: osd crush chooseleaf type = 0 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: auth debug = true 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on old message = true 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: ms die on bug = true 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:16.998 INFO:teuthology.orchestra.run.smithi027.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: mon pg warn max object skew = 0 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: # disable pg_autoscaler by default for new pools 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default size = 2 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow primary affinity = true 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd allow pg remap = true 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on legacy crush tunables = false 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on crush straw calc version zero = false 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on no sortbitwise = false 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on osd down out interval zero = false 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: mon warn on too few osds = false 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-19T18:14:16.999 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: mon_allow_pool_size_one = true 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: osd default data pool replay window = 5 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: mon allow pool delete = true 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: mon cluster log file level = debug 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: debug asserts on shutdown = true 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: mon health detail to clog = false 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: mon host = "172.21.15.27,172.21.15.192,[v2:172.21.15.192:3301,v1:172.21.15.192:6790]" 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = dc17deb5-5eca-4f5c-983d-094ffb50339f 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout:[osd] 2023-12-19T18:14:17.000 INFO:teuthology.orchestra.run.smithi027.stdout: osd journal size = 100 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub load threshold = 5.0 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: osd scrub max interval = 600 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: osd mclock profile = high_recovery_ops 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: osd recover clone overlap = true 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: osd recovery max chunk = 1048576 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug shutdown = true 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug op order = true 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug verify stray on activate = true 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: osd open classes on start = true 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug pg log writeout = true 2023-12-19T18:14:17.001 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.002 INFO:teuthology.orchestra.run.smithi027.stdout: osd deep scrub update digest min age = 30 2023-12-19T18:14:17.002 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.002 INFO:teuthology.orchestra.run.smithi027.stdout: osd map max advance = 10 2023-12-19T18:14:17.002 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.002 INFO:teuthology.orchestra.run.smithi027.stdout: journal zero on create = true 2023-12-19T18:14:17.002 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.002 INFO:teuthology.orchestra.run.smithi027.stdout: filestore ondisk finisher threads = 3 2023-12-19T18:14:17.002 INFO:teuthology.orchestra.run.smithi027.stdout: filestore apply finisher threads = 3 2023-12-19T18:14:17.002 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.002 INFO:teuthology.orchestra.run.smithi027.stdout: bdev debug aio = true 2023-12-19T18:14:17.002 INFO:teuthology.orchestra.run.smithi027.stdout: osd debug misdirected ops = true 2023-12-19T18:14:17.002 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore block size = 96636764160 2023-12-19T18:14:17.002 INFO:teuthology.orchestra.run.smithi027.stdout: bluestore fsck on mount = True 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluefs = 20 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: debug bluestore = 20 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: debug osd = 20 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: debug rocksdb = 10 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: memstore device bytes = 200000000 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd backfillfull_ratio = 0.85 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd full ratio = 0.9 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd nearfull ratio = 0.8 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: osd failsafe full ratio = 1.0 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: osd mon report interval = 5 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: osd objectstore = memstore 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: osd op complaint time = 180 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.003 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr] 2023-12-19T18:14:17.004 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-12-19T18:14:17.004 INFO:teuthology.orchestra.run.smithi027.stdout: debug mgr = 20 2023-12-19T18:14:17.004 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2023-12-19T18:14:17.004 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2023-12-19T18:14:17.004 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2023-12-19T18:14:17.004 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2023-12-19T18:14:17.004 INFO:teuthology.orchestra.run.smithi027.stdout: mgr/telemetry/nag = false 2023-12-19T18:14:17.004 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.004 INFO:teuthology.orchestra.run.smithi027.stdout:[mon] 2023-12-19T18:14:17.004 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: debug mon = 20 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: debug paxos = 20 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: debug auth = 20 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: mon data avail warn = 5 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: mon mgr mkfs grace = 240 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min pgs per osd = 4 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd reporter subtree level = osd 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd prime pg temp = true 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: mon reweight min bytes per osd = 10 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: auth mon ticket ttl = 660 # 11m 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: auth service ticket ttl = 240 # 4m 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.005 INFO:teuthology.orchestra.run.smithi027.stdout: # don't complain about insecure global_id in the test suite 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: # 1m isn't quite enough 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: mon_down_mkfs_grace = 2m 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: mon_warn_on_filestore_osds = false 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: mon op complaint time = 120 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd backfillfull ratio = 0.6 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd full ratio = 0.7 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: mon osd nearfull ratio = 0.6 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout:[client] 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: rgw cache enabled = true 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable ops log = true 2023-12-19T18:14:17.006 INFO:teuthology.orchestra.run.smithi027.stdout: rgw enable usage log = true 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout: client mount timeout = 600 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout: debug client = 20 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout: pid file = "" 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout: rados mon op timeout = 900 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout: rados osd op timeout = 900 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.a] 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.b] 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout:[mon.c] 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a] 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c] 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b] 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d] 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2023-12-19T18:14:17.007 INFO:teuthology.orchestra.run.smithi027.stdout: debug client = 20 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: debug objectcacher = 20 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: debug objecter = 20 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout:[client.1] 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: debug client = 20 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: debug objectcacher = 20 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: debug objecter = 20 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout:[mds] 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: debug mds = 20 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: debug mds balancer = 20 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: debug ms = 1 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: mds debug frag = True 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: mds debug scatterstat = True 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: mds op complaint time = 180 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: mds verify scatter = True 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: osd op complaint time = 180 2023-12-19T18:14:17.008 INFO:teuthology.orchestra.run.smithi027.stdout: rados mon op timeout = 900 2023-12-19T18:14:17.009 INFO:teuthology.orchestra.run.smithi027.stdout: rados osd op timeout = 900 2023-12-19T18:14:17.049 INFO:tasks.ceph:Creating admin key on mon.a... 2023-12-19T18:14:17.049 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-12-19T18:14:17.207 INFO:tasks.ceph:Copying monmap to all nodes... 2023-12-19T18:14:17.207 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:17.207 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-12-19T18:14:17.264 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:17.265 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-12-19T18:14:17.321 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:14:17.321 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:17.321 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-19T18:14:17.321 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-19T18:14:17.496 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:17.496 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:17.597 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:14:17.597 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:17.597 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-19T18:14:17.597 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-19T18:14:17.752 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:17.753 DEBUG:teuthology.orchestra.run.smithi192:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:17.851 INFO:tasks.ceph:Setting up mon nodes... 2023-12-19T18:14:17.851 INFO:tasks.ceph:Setting up mgr nodes... 2023-12-19T18:14:17.852 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-12-19T18:14:18.068 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-12-19T18:14:18.078 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-12-19T18:14:18.258 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-12-19T18:14:18.268 INFO:tasks.ceph:Setting up mds nodes... 2023-12-19T18:14:18.268 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-12-19T18:14:18.484 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-12-19T18:14:18.494 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-12-19T18:14:18.608 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-12-19T18:14:18.809 INFO:teuthology.orchestra.run.smithi027.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-12-19T18:14:18.818 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-12-19T18:14:18.921 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-12-19T18:14:19.109 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-12-19T18:14:19.116 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-12-19T18:14:19.210 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-12-19T18:14:19.373 INFO:teuthology.orchestra.run.smithi192.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-12-19T18:14:19.382 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-12-19T18:14:19.477 INFO:tasks.ceph_client:Setting up client nodes... 2023-12-19T18:14:19.477 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-19T18:14:19.624 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-12-19T18:14:19.704 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-12-19T18:14:19.843 INFO:teuthology.orchestra.run.smithi027.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-12-19T18:14:19.908 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-12-19T18:14:20.040 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-12-19T18:14:20.088 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-12-19T18:14:20.218 INFO:teuthology.orchestra.run.smithi192.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-12-19T18:14:20.278 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-12-19T18:14:20.279 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi027.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi192.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-19T18:14:20.279 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-12-19T18:14:20.389 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-19T18:14:20.389 INFO:tasks.ceph:role: osd.0 2023-12-19T18:14:20.389 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:14:20.389 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-19T18:14:20.499 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T18:14:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T18:14:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T18:14:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-19T18:14:20.499 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T18:14:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2023-12-19T18:14:20.499 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T18:14:20.499 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-19T18:14:20.499 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T18:14:20.499 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T18:14:21.015 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2023-12-19T18:14:21.021 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2023-12-19T18:14:21.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-12-19T18:14:21.213 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-12-19T18:14:21.342 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-12-19T18:14:21.439 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-19T18:14:21.439 INFO:tasks.ceph:role: osd.1 2023-12-19T18:14:21.439 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:14:21.439 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-19T18:14:21.548 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T18:14:21.548 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T18:14:21.548 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T18:14:21.548 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-19T18:14:21.548 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T18:14:21.548 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2023-12-19T18:14:21.548 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T18:14:21.548 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-19T18:14:21.548 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T18:14:21.548 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T18:14:22.039 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2023-12-19T18:14:22.046 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2023-12-19T18:14:22.046 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-12-19T18:14:22.217 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-12-19T18:14:22.333 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-12-19T18:14:22.441 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-19T18:14:22.442 INFO:tasks.ceph:role: osd.2 2023-12-19T18:14:22.442 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:14:22.442 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-19T18:14:22.556 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T18:14:22.556 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T18:14:22.557 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T18:14:22.557 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-19T18:14:22.557 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T18:14:22.557 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2023-12-19T18:14:22.557 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T18:14:22.557 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-19T18:14:22.557 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T18:14:22.557 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T18:14:23.056 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2023-12-19T18:14:23.062 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2023-12-19T18:14:23.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-12-19T18:14:23.253 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-12-19T18:14:23.374 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-12-19T18:14:23.483 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-19T18:14:23.483 INFO:tasks.ceph:role: osd.3 2023-12-19T18:14:23.483 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:14:23.483 DEBUG:teuthology.orchestra.run.smithi027:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-19T18:14:23.599 INFO:teuthology.orchestra.run.smithi027.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T18:14:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T18:14:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T18:14:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-19T18:14:23.599 INFO:teuthology.orchestra.run.smithi027.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T18:14:23.599 INFO:teuthology.orchestra.run.smithi027.stdout: = sunit=0 swidth=0 blks 2023-12-19T18:14:23.600 INFO:teuthology.orchestra.run.smithi027.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T18:14:23.600 INFO:teuthology.orchestra.run.smithi027.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-19T18:14:23.600 INFO:teuthology.orchestra.run.smithi027.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T18:14:23.600 INFO:teuthology.orchestra.run.smithi027.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T18:14:24.094 INFO:teuthology.orchestra.run.smithi027.stdout:Discarding blocks...Done. 2023-12-19T18:14:24.100 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi027.front.sepia.ceph.com -o noatime 2023-12-19T18:14:24.100 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-12-19T18:14:24.253 DEBUG:teuthology.orchestra.run.smithi027:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-12-19T18:14:24.368 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:24.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:24.544+0000 7f9110020500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-12-19T18:14:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:24.547+0000 7f9110020500 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-12-19T18:14:24.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:24.557+0000 7f9110020500 -1 memstore(/var/lib/ceph/osd/ceph-0) /var/lib/ceph/osd/ceph-0 2023-12-19T18:14:24.598 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-12-19T18:14:24.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:24.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:24.866+0000 7f0aa3d10500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-12-19T18:14:24.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:24.870+0000 7f0aa3d10500 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-12-19T18:14:24.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:24.881+0000 7f0aa3d10500 -1 memstore(/var/lib/ceph/osd/ceph-1) /var/lib/ceph/osd/ceph-1 2023-12-19T18:14:24.910 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-12-19T18:14:25.013 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:25.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:25.157+0000 7f375d80e500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-12-19T18:14:25.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:25.158+0000 7f375d80e500 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-12-19T18:14:25.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:25.163+0000 7f375d80e500 -1 memstore(/var/lib/ceph/osd/ceph-2) /var/lib/ceph/osd/ceph-2 2023-12-19T18:14:25.181 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-12-19T18:14:25.272 DEBUG:teuthology.orchestra.run.smithi027:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:25.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:25.432+0000 7f763321b500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-12-19T18:14:25.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:25.433+0000 7f763321b500 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-12-19T18:14:25.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:25.435+0000 7f763321b500 -1 memstore(/var/lib/ceph/osd/ceph-3) /var/lib/ceph/osd/ceph-3 2023-12-19T18:14:25.457 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-12-19T18:14:25.566 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-12-19T18:14:25.668 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-19T18:14:25.668 INFO:tasks.ceph:role: osd.4 2023-12-19T18:14:25.668 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:14:25.668 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-19T18:14:25.767 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T18:14:25.767 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T18:14:25.767 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T18:14:25.767 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-19T18:14:25.767 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T18:14:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2023-12-19T18:14:25.768 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T18:14:25.768 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-19T18:14:25.768 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T18:14:25.768 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T18:14:26.278 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2023-12-19T18:14:26.285 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2023-12-19T18:14:26.285 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-12-19T18:14:26.480 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-12-19T18:14:26.587 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-12-19T18:14:26.672 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-19T18:14:26.673 INFO:tasks.ceph:role: osd.5 2023-12-19T18:14:26.673 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:14:26.673 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-19T18:14:26.768 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T18:14:26.768 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T18:14:26.768 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T18:14:26.768 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-19T18:14:26.768 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T18:14:26.768 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2023-12-19T18:14:26.768 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T18:14:26.768 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-19T18:14:26.768 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T18:14:26.769 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T18:14:27.245 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2023-12-19T18:14:27.251 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2023-12-19T18:14:27.251 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-12-19T18:14:27.376 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-12-19T18:14:27.457 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-12-19T18:14:27.539 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-19T18:14:27.540 INFO:tasks.ceph:role: osd.6 2023-12-19T18:14:27.540 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:14:27.540 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-19T18:14:27.625 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T18:14:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T18:14:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T18:14:27.625 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-19T18:14:27.625 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T18:14:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2023-12-19T18:14:27.626 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T18:14:27.626 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-19T18:14:27.626 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T18:14:27.626 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T18:14:28.133 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2023-12-19T18:14:28.139 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2023-12-19T18:14:28.139 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-12-19T18:14:28.323 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-12-19T18:14:28.430 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-12-19T18:14:28.526 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-19T18:14:28.526 INFO:tasks.ceph:role: osd.7 2023-12-19T18:14:28.526 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:14:28.526 DEBUG:teuthology.orchestra.run.smithi192:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-19T18:14:28.631 INFO:teuthology.orchestra.run.smithi192.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-19T18:14:28.631 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-19T18:14:28.631 INFO:teuthology.orchestra.run.smithi192.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-19T18:14:28.631 INFO:teuthology.orchestra.run.smithi192.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2023-12-19T18:14:28.631 INFO:teuthology.orchestra.run.smithi192.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-19T18:14:28.632 INFO:teuthology.orchestra.run.smithi192.stdout: = sunit=0 swidth=0 blks 2023-12-19T18:14:28.632 INFO:teuthology.orchestra.run.smithi192.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-19T18:14:28.632 INFO:teuthology.orchestra.run.smithi192.stdout:log =internal log bsize=4096 blocks=16384, version=2 2023-12-19T18:14:28.632 INFO:teuthology.orchestra.run.smithi192.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-19T18:14:28.632 INFO:teuthology.orchestra.run.smithi192.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-19T18:14:29.085 INFO:teuthology.orchestra.run.smithi192.stdout:Discarding blocks...Done. 2023-12-19T18:14:29.090 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi192.front.sepia.ceph.com -o noatime 2023-12-19T18:14:29.091 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-12-19T18:14:29.230 DEBUG:teuthology.orchestra.run.smithi192:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-12-19T18:14:29.335 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:29.489 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:29.489+0000 7f9d06c25500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-12-19T18:14:29.494 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:29.493+0000 7f9d06c25500 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-12-19T18:14:29.504 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:29.503+0000 7f9d06c25500 -1 memstore(/var/lib/ceph/osd/ceph-4) /var/lib/ceph/osd/ceph-4 2023-12-19T18:14:29.537 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-12-19T18:14:29.628 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:29.766 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:29.766+0000 7f149341d500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-12-19T18:14:29.771 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:29.771+0000 7f149341d500 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-12-19T18:14:29.778 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:29.778+0000 7f149341d500 -1 memstore(/var/lib/ceph/osd/ceph-5) /var/lib/ceph/osd/ceph-5 2023-12-19T18:14:29.807 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-12-19T18:14:29.897 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:30.029 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:30.028+0000 7fc8b5412500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-12-19T18:14:30.034 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:30.034+0000 7fc8b5412500 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-12-19T18:14:30.045 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:30.045+0000 7fc8b5412500 -1 memstore(/var/lib/ceph/osd/ceph-6) /var/lib/ceph/osd/ceph-6 2023-12-19T18:14:30.076 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-12-19T18:14:30.171 DEBUG:teuthology.orchestra.run.smithi192:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:30.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:30.313+0000 7f85abcfa500 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-12-19T18:14:30.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:30.314+0000 7f85abcfa500 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-12-19T18:14:30.316 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:30.316+0000 7f85abcfa500 -1 memstore(/var/lib/ceph/osd/ceph-7) /var/lib/ceph/osd/ceph-7 2023-12-19T18:14:30.333 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-12-19T18:14:30.419 INFO:tasks.ceph:Reading keys from all nodes... 2023-12-19T18:14:30.420 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:30.420 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-12-19T18:14:30.539 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:30.539 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-12-19T18:14:30.651 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:30.651 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-12-19T18:14:30.762 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:30.762 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-12-19T18:14:30.873 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:30.873 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-12-19T18:14:30.983 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:30.983 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-12-19T18:14:31.093 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:31.093 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-12-19T18:14:31.204 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:31.204 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-12-19T18:14:31.311 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:31.311 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-12-19T18:14:31.412 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:31.412 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-12-19T18:14:31.509 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:31.510 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-12-19T18:14:31.606 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:31.606 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-12-19T18:14:31.697 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:31.697 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-12-19T18:14:31.827 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:31.827 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-12-19T18:14:31.911 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:31.911 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-12-19T18:14:31.975 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:31.976 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-12-19T18:14:32.033 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:32.033 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-12-19T18:14:32.091 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:32.091 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-12-19T18:14:32.145 INFO:tasks.ceph:Adding keys to all mons... 2023-12-19T18:14:32.145 DEBUG:teuthology.orchestra.run.smithi027:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-19T18:14:32.154 DEBUG:teuthology.orchestra.run.smithi192:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-19T18:14:32.244 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.y] 2023-12-19T18:14:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQB63YFlUnAhBBAAi++W9XK3o3Y/185hB3LuIg== 2023-12-19T18:14:32.244 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.a] 2023-12-19T18:14:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQB63YFl+4ruHBAAXwbpmT43SmlvNbuZj5e+bQ== 2023-12-19T18:14:32.244 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.c] 2023-12-19T18:14:32.244 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQB63YFlChNOMBAAX4GV3iIGkSQdAsYu9oCPDw== 2023-12-19T18:14:32.245 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.0] 2023-12-19T18:14:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCA3YFluleFIBAAs1goFVMOX4CbjwFhg0KoBg== 2023-12-19T18:14:32.245 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.1] 2023-12-19T18:14:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCA3YFlInW3MxAAI4hY1GBxXYXDswKKrEyDTg== 2023-12-19T18:14:32.245 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.2] 2023-12-19T18:14:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCB3YFlAWZ5CRAA82/2r9/sT0L9VlGHbXksQw== 2023-12-19T18:14:32.245 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.3] 2023-12-19T18:14:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCB3YFl1IvaGRAAqVacDDPLx+NqBOGp6MeX2w== 2023-12-19T18:14:32.245 INFO:teuthology.orchestra.run.smithi192.stdout:[mgr.x] 2023-12-19T18:14:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQB63YFltBhxDxAAiAS9CYPY86IMmJ7NGgA2lw== 2023-12-19T18:14:32.245 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.b] 2023-12-19T18:14:32.245 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQB73YFlmZyUBhAAG3kZkXvJIo7N9f1j+DPCSQ== 2023-12-19T18:14:32.245 INFO:teuthology.orchestra.run.smithi192.stdout:[mds.d] 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQB73YFlTmFXFhAABA0NMUxKBV1ExAhYc5RFbQ== 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.4] 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCF3YFlr0BBHRAA+7gfs/RG6DtABnYPg059Tw== 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.5] 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCF3YFlXZbELRAAKD0qOkhk7skfc4xcAC8Lhg== 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.6] 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCG3YFlF07MARAA2TJJ+5C1HIU+SVhqmKaVHA== 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout:[osd.7] 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQCG3YFlISPGEhAAdFIRRkXlLyOx/oMZ4MgL9Q== 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout:[client.0] 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQB73YFlmNM9JRAARgS20O8zpm6H70WDv+lgGw== 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout:[client.2] 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQB73YFllWFVMhAAD9ebDOppHTODVQjg6ppP3g== 2023-12-19T18:14:32.246 INFO:teuthology.orchestra.run.smithi192.stdout:[client.1] 2023-12-19T18:14:32.247 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQB83YFlWoZ8AhAADWvKVzeDhDGHtYNUFIOWsA== 2023-12-19T18:14:32.247 INFO:teuthology.orchestra.run.smithi192.stdout:[client.3] 2023-12-19T18:14:32.247 INFO:teuthology.orchestra.run.smithi192.stdout: key = AQB83YFlwbIQDRAAt/h55zBNCEId0vkYSUQ3TQ== 2023-12-19T18:14:32.257 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.y] 2023-12-19T18:14:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQB63YFlUnAhBBAAi++W9XK3o3Y/185hB3LuIg== 2023-12-19T18:14:32.257 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.a] 2023-12-19T18:14:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQB63YFl+4ruHBAAXwbpmT43SmlvNbuZj5e+bQ== 2023-12-19T18:14:32.257 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.c] 2023-12-19T18:14:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQB63YFlChNOMBAAX4GV3iIGkSQdAsYu9oCPDw== 2023-12-19T18:14:32.257 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.0] 2023-12-19T18:14:32.257 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCA3YFluleFIBAAs1goFVMOX4CbjwFhg0KoBg== 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.1] 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCA3YFlInW3MxAAI4hY1GBxXYXDswKKrEyDTg== 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.2] 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCB3YFlAWZ5CRAA82/2r9/sT0L9VlGHbXksQw== 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.3] 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCB3YFl1IvaGRAAqVacDDPLx+NqBOGp6MeX2w== 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout:[mgr.x] 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQB63YFltBhxDxAAiAS9CYPY86IMmJ7NGgA2lw== 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.b] 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQB73YFlmZyUBhAAG3kZkXvJIo7N9f1j+DPCSQ== 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.d] 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQB73YFlTmFXFhAABA0NMUxKBV1ExAhYc5RFbQ== 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.4] 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCF3YFlr0BBHRAA+7gfs/RG6DtABnYPg059Tw== 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.5] 2023-12-19T18:14:32.258 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCF3YFlXZbELRAAKD0qOkhk7skfc4xcAC8Lhg== 2023-12-19T18:14:32.259 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.6] 2023-12-19T18:14:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCG3YFlF07MARAA2TJJ+5C1HIU+SVhqmKaVHA== 2023-12-19T18:14:32.259 INFO:teuthology.orchestra.run.smithi027.stdout:[osd.7] 2023-12-19T18:14:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCG3YFlISPGEhAAdFIRRkXlLyOx/oMZ4MgL9Q== 2023-12-19T18:14:32.259 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2023-12-19T18:14:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQB73YFlmNM9JRAARgS20O8zpm6H70WDv+lgGw== 2023-12-19T18:14:32.259 INFO:teuthology.orchestra.run.smithi027.stdout:[client.2] 2023-12-19T18:14:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQB73YFllWFVMhAAD9ebDOppHTODVQjg6ppP3g== 2023-12-19T18:14:32.259 INFO:teuthology.orchestra.run.smithi027.stdout:[client.1] 2023-12-19T18:14:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQB83YFlWoZ8AhAADWvKVzeDhDGHtYNUFIOWsA== 2023-12-19T18:14:32.259 INFO:teuthology.orchestra.run.smithi027.stdout:[client.3] 2023-12-19T18:14:32.259 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQB83YFlwbIQDRAAt/h55zBNCEId0vkYSUQ3TQ== 2023-12-19T18:14:32.262 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-19T18:14:32.271 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-19T18:14:32.424 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T18:14:32.430 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T18:14:32.584 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T18:14:32.589 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T18:14:32.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:32.748 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:32.899 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:32.907 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:33.063 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:33.072 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:33.223 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:33.228 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:33.375 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-19T18:14:33.382 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-19T18:14:33.538 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T18:14:33.547 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T18:14:33.699 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T18:14:33.708 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-19T18:14:33.867 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:33.876 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:34.030 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:34.036 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:34.185 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:34.193 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:34.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:34.351 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-19T18:14:34.502 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T18:14:34.507 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T18:14:34.661 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T18:14:34.668 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T18:14:34.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T18:14:34.828 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T18:14:34.988 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T18:14:34.997 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-19T18:14:35.150 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-12-19T18:14:35.150 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-12-19T18:14:35.258 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-19T18:14:35.437 DEBUG:teuthology.orchestra.run.smithi027:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-12-19T18:14:35.541 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-12-19T18:14:35.644 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-19T18:14:35.797 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-12-19T18:14:35.887 DEBUG:teuthology.orchestra.run.smithi192:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-12-19T18:14:35.981 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-19T18:14:36.136 DEBUG:teuthology.orchestra.run.smithi192:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-12-19T18:14:36.229 DEBUG:teuthology.orchestra.run.smithi027:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:36.239 DEBUG:teuthology.orchestra.run.smithi192:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-19T18:14:36.294 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-12-19T18:14:36.295 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-19T18:14:36.295 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-12-19T18:14:36.301 INFO:tasks.ceph.mon.a:Started 2023-12-19T18:14:36.301 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-19T18:14:36.301 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-12-19T18:14:36.306 INFO:tasks.ceph.mon.b:Started 2023-12-19T18:14:36.307 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-19T18:14:36.307 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-12-19T18:14:36.315 INFO:tasks.ceph.mon.c:Started 2023-12-19T18:14:36.315 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-12-19T18:14:36.315 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-19T18:14:36.316 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-12-19T18:14:36.324 INFO:tasks.ceph.mgr.y:Started 2023-12-19T18:14:36.324 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-12-19T18:14:36.324 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-12-19T18:14:36.329 INFO:tasks.ceph.mgr.x:Started 2023-12-19T18:14:36.329 INFO:tasks.ceph:Setting crush tunables to default 2023-12-19T18:14:36.329 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph osd crush tunables default 2023-12-19T18:14:36.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.519+0000 7ff82aa28640 1 Processor -- start 2023-12-19T18:14:36.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.519+0000 7ff82aa28640 1 -- start start 2023-12-19T18:14:36.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.521+0000 7ff82aa28640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff8240b4c80 0x7ff824160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:36.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.521+0000 7ff82aa28640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff8240b8920 0x7ff82415cac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:36.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.521+0000 7ff82aa28640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8240d0d20 con 0x7ff8240b5060 2023-12-19T18:14:36.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.521+0000 7ff82aa28640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8240b6130 con 0x7ff8240b8920 2023-12-19T18:14:36.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.521+0000 7ff82aa28640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff824161bf0 con 0x7ff8240b4c80 2023-12-19T18:14:36.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.521+0000 7ff8237fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff8240b8920 0x7ff82415cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:36.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.521+0000 7ff828fc5640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff8240b5060 0x7ff82406a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34362/0 (socket says 172.21.15.27:34362) 2023-12-19T18:14:36.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.521+0000 7ff828fc5640 1 -- 172.21.15.27:0/4250214190 learned_addr learned my addr 172.21.15.27:0/4250214190 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:36.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.521+0000 7ff8237fe640 1 -- 172.21.15.27:0/4250214190 >> v2:172.21.15.27:3300/0 conn(0x7ff8240b8920 msgr2=0x7ff82415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-19T18:14:36.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.521+0000 7ff8237fe640 1 -- 172.21.15.27:0/4250214190 >> v2:172.21.15.27:3300/0 conn(0x7ff8240b8920 msgr2=0x7ff82415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-19T18:14:36.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.521+0000 7ff8237fe640 1 --2- 172.21.15.27:0/4250214190 >> v2:172.21.15.27:3300/0 conn(0x7ff8240b8920 0x7ff82415cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-19T18:14:36.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.521+0000 7ff8237fe640 1 --2- 172.21.15.27:0/4250214190 >> v2:172.21.15.27:3300/0 conn(0x7ff8240b8920 0x7ff82415cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:36.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.522+0000 7ff823fff640 1 --2- 172.21.15.27:0/4250214190 >> v2:172.21.15.192:3300/0 conn(0x7ff8240b4c80 0x7ff824160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:36.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.522+0000 7ff823fff640 1 -- 172.21.15.27:0/4250214190 >> v2:172.21.15.192:3300/0 conn(0x7ff8240b4c80 msgr2=0x7ff824160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-19T18:14:36.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.522+0000 7ff823fff640 1 -- 172.21.15.27:0/4250214190 >> v2:172.21.15.192:3300/0 conn(0x7ff8240b4c80 msgr2=0x7ff824160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-19T18:14:36.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.522+0000 7ff823fff640 1 --2- 172.21.15.27:0/4250214190 >> v2:172.21.15.192:3300/0 conn(0x7ff8240b4c80 0x7ff824160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-19T18:14:36.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.522+0000 7ff823fff640 1 --2- 172.21.15.27:0/4250214190 >> v2:172.21.15.192:3300/0 conn(0x7ff8240b4c80 0x7ff824160c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-19T18:14:36.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3383471020 0 0) 0x7ff8240d0d20 con 0x7ff8240b5060 2023-12-19T18:14:36.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff80c003520 con 0x7ff8240b5060 2023-12-19T18:14:36.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3857966236 0 0) 0x7ff80c003520 con 0x7ff8240b5060 2023-12-19T18:14:36.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 >> v2:172.21.15.192:3300/0 conn(0x7ff8240b4c80 msgr2=0x7ff824160c70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-19T18:14:36.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 --2- 172.21.15.27:0/4250214190 >> v2:172.21.15.192:3300/0 conn(0x7ff8240b4c80 0x7ff824160c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:36.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 >> v2:172.21.15.27:3300/0 conn(0x7ff8240b8920 msgr2=0x7ff82415cac0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-19T18:14:36.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 --2- 172.21.15.27:0/4250214190 >> v2:172.21.15.27:3300/0 conn(0x7ff8240b8920 0x7ff82415cac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:36.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff824161350 con 0x7ff8240b5060 2023-12-19T18:14:36.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7ff81c0019f0 con 0x7ff8240b5060 2023-12-19T18:14:36.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 >> v1:172.21.15.27:6789/0 conn(0x7ff8240b5060 legacy=0x7ff82406a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:36.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 --2- 172.21.15.27:0/4250214190 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff80c0039c0 0x7ff80c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:36.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 --2- 172.21.15.27:0/4250214190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff80c0047c0 0x7ff80c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:36.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 --2- 172.21.15.27:0/4250214190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff80c0051d0 0x7ff80c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:36.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240b6130 con 0x7ff80c0051d0 2023-12-19T18:14:36.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff824161bf0 con 0x7ff80c0047c0 2023-12-19T18:14:36.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.548+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff80c00c180 con 0x7ff80c0039c0 2023-12-19T18:14:36.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.549+0000 7ff8237fe640 1 --2- 172.21.15.27:0/4250214190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff80c0047c0 0x7ff80c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:36.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.549+0000 7ff823fff640 1 --2- 172.21.15.27:0/4250214190 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff80c0039c0 0x7ff80c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:36.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.549+0000 7ff828fc5640 1 --2- 172.21.15.27:0/4250214190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff80c0051d0 0x7ff80c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:36.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.549+0000 7ff828fc5640 1 -- 172.21.15.27:0/4250214190 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff80c0039c0 msgr2=0x7ff80c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:36.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.549+0000 7ff828fc5640 1 --2- 172.21.15.27:0/4250214190 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff80c0039c0 0x7ff80c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:36.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.549+0000 7ff828fc5640 1 -- 172.21.15.27:0/4250214190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff80c0047c0 msgr2=0x7ff80c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:36.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.549+0000 7ff828fc5640 1 --2- 172.21.15.27:0/4250214190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff80c0047c0 0x7ff80c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:36.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.549+0000 7ff828fc5640 1 -- 172.21.15.27:0/4250214190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff80c003520 con 0x7ff80c0051d0 2023-12-19T18:14:36.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.549+0000 7ff8237fe640 1 --2- 172.21.15.27:0/4250214190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff80c0047c0 0x7ff80c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:36.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.550+0000 7ff828fc5640 1 -- 172.21.15.27:0/4250214190 reap_dead start 2023-12-19T18:14:36.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.552+0000 7ff828fc5640 1 --2- 172.21.15.27:0/4250214190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff80c0051d0 0x7ff80c00b980 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff8240b5510 tx=0x7ff80c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591869bf3f2cf1d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:36.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.552+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c034630 con 0x7ff80c0051d0 2023-12-19T18:14:36.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.552+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81c02f070 con 0x7ff80c0051d0 2023-12-19T18:14:36.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.552+0000 7ff822ffd640 1 -- 172.21.15.27:0/4250214190 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81c034a10 con 0x7ff80c0051d0 2023-12-19T18:14:36.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.552+0000 7ff82aa28640 1 -- 172.21.15.27:0/4250214190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff80c0051d0 msgr2=0x7ff80c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:36.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.552+0000 7ff82aa28640 1 --2- 172.21.15.27:0/4250214190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff80c0051d0 0x7ff80c00b980 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7ff8240b5510 tx=0x7ff80c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:14:36.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.552+0000 7ff82aa28640 1 -- 172.21.15.27:0/4250214190 shutdown_connections 2023-12-19T18:14:36.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.552+0000 7ff82aa28640 1 --2- 172.21.15.27:0/4250214190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff80c0051d0 0x7ff80c00b980 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:36.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.552+0000 7ff82aa28640 1 -- 172.21.15.27:0/4250214190 >> 172.21.15.27:0/4250214190 conn(0x7ff8240d2f90 msgr2=0x7ff8240d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:36.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.552+0000 7ff82aa28640 1 -- 172.21.15.27:0/4250214190 shutdown_connections 2023-12-19T18:14:36.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.552+0000 7ff82aa28640 1 -- 172.21.15.27:0/4250214190 wait complete. 2023-12-19T18:14:36.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.555+0000 7ff82aa28640 1 Processor -- start 2023-12-19T18:14:36.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.555+0000 7ff82aa28640 1 -- start start 2023-12-19T18:14:36.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff82aa28640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff80c0051d0 0x7ff82415b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:36.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff82aa28640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8240caf50 0x7ff8240cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff82aa28640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8240cb3d0 0x7ff8240cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff82aa28640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff82414aa00 con 0x7ff80c0051d0 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff82aa28640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240ca190 con 0x7ff8240cb3d0 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff82aa28640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240ca470 con 0x7ff8240caf50 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff823fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff80c0051d0 0x7ff82415b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff828fc5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8240cb3d0 0x7ff8240cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff823fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff80c0051d0 0x7ff82415b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50408/0 (socket says 172.21.15.27:50408) 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff823fff640 1 -- 172.21.15.27:0/212111983 learned_addr learned my addr 172.21.15.27:0/212111983 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff8237fe640 1 --2- 172.21.15.27:0/212111983 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8240caf50 0x7ff8240cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff823fff640 1 -- 172.21.15.27:0/212111983 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8240caf50 msgr2=0x7ff8240cbdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff823fff640 1 --2- 172.21.15.27:0/212111983 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8240caf50 0x7ff8240cbdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff823fff640 1 -- 172.21.15.27:0/212111983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8240cb3d0 msgr2=0x7ff8240cc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff823fff640 1 --2- 172.21.15.27:0/212111983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8240cb3d0 0x7ff8240cc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:36.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff823fff640 1 -- 172.21.15.27:0/212111983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8240ca750 con 0x7ff80c0051d0 2023-12-19T18:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.557+0000 7ff8237fe640 1 --2- 172.21.15.27:0/212111983 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8240caf50 0x7ff8240cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.559+0000 7ff823fff640 1 --2- 172.21.15.27:0/212111983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff80c0051d0 0x7ff82415b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff8140108c0 tx=0x7ff814010d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.559+0000 7ff820ff9640 1 -- 172.21.15.27:0/212111983 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400e9c0 con 0x7ff80c0051d0 2023-12-19T18:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.559+0000 7ff820ff9640 1 -- 172.21.15.27:0/212111983 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff81400eb20 con 0x7ff80c0051d0 2023-12-19T18:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.559+0000 7ff820ff9640 1 -- 172.21.15.27:0/212111983 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff81400eed0 con 0x7ff80c0051d0 2023-12-19T18:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.559+0000 7ff82aa28640 1 -- 172.21.15.27:0/212111983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8240caa30 con 0x7ff80c0051d0 2023-12-19T18:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.559+0000 7ff82aa28640 1 -- 172.21.15.27:0/212111983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82415bb60 con 0x7ff80c0051d0 2023-12-19T18:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.559+0000 7ff820ff9640 1 -- 172.21.15.27:0/212111983 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff814016780 con 0x7ff80c0051d0 2023-12-19T18:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.559+0000 7ff820ff9640 1 -- 172.21.15.27:0/212111983 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff8140148b0 con 0x7ff80c0051d0 2023-12-19T18:14:36.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.559+0000 7ff82aa28640 1 -- 172.21.15.27:0/212111983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8240ca2d0 con 0x7ff80c0051d0 2023-12-19T18:14:36.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.562+0000 7ff820ff9640 1 -- 172.21.15.27:0/212111983 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7ff81403eb40 con 0x7ff80c0051d0 2023-12-19T18:14:36.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.636+0000 7ff82aa28640 1 -- 172.21.15.27:0/212111983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff8240cabc0 con 0x7ff80c0051d0 2023-12-19T18:14:36.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.641+0000 7ff820ff9640 1 -- 172.21.15.27:0/212111983 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff81402d020 con 0x7ff80c0051d0 2023-12-19T18:14:36.641 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2023-12-19T18:14:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.642+0000 7ff82aa28640 1 -- 172.21.15.27:0/212111983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff80c0051d0 msgr2=0x7ff82415b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.642+0000 7ff82aa28640 1 --2- 172.21.15.27:0/212111983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff80c0051d0 0x7ff82415b620 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7ff8140108c0 tx=0x7ff814010d80 comp rx=0 tx=0).stop 2023-12-19T18:14:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.642+0000 7ff82aa28640 1 -- 172.21.15.27:0/212111983 shutdown_connections 2023-12-19T18:14:36.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.642+0000 7ff82aa28640 1 --2- 172.21.15.27:0/212111983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8240cb3d0 0x7ff8240cc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:36.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.642+0000 7ff82aa28640 1 --2- 172.21.15.27:0/212111983 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8240caf50 0x7ff8240cbdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:36.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.642+0000 7ff82aa28640 1 --2- 172.21.15.27:0/212111983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff80c0051d0 0x7ff82415b620 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:36.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.642+0000 7ff82aa28640 1 -- 172.21.15.27:0/212111983 >> 172.21.15.27:0/212111983 conn(0x7ff8240d2f90 msgr2=0x7ff8240c6bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:36.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.643+0000 7ff82aa28640 1 -- 172.21.15.27:0/212111983 shutdown_connections 2023-12-19T18:14:36.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:36.643+0000 7ff82aa28640 1 -- 172.21.15.27:0/212111983 wait complete. 2023-12-19T18:14:36.659 INFO:tasks.ceph:check_enable_crimson: False 2023-12-19T18:14:36.659 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-12-19T18:14:36.660 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:36.660 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-12-19T18:14:36.739 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:36.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-12-19T18:14:36.879 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:36.879 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-12-19T18:14:36.970 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:14:36.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-12-19T18:14:37.064 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:37.064 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-12-19T18:14:37.136 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:37.137+0000 7fe47583b040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:14:37.155 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:37.155 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-12-19T18:14:37.199 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:37.199+0000 7fe47583b040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:14:37.226 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:37.226+0000 7fbc5e62e040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-19T18:14:37.234 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:37.234 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-12-19T18:14:37.287 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:37.287+0000 7fbc5e62e040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-19T18:14:37.363 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:14:37.364 DEBUG:teuthology.orchestra.run.smithi192:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-12-19T18:14:37.490 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 0 2023-12-19T18:14:37.724 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.724+0000 7fa0c3fff640 1 Processor -- start 2023-12-19T18:14:37.724 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.724+0000 7fa0c3fff640 1 -- start start 2023-12-19T18:14:37.726 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.725+0000 7fa0c3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa0c40b3980 0x7fa0c40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:37.726 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.725+0000 7fa0c3fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c40d3850 con 0x7fa0c415d120 2023-12-19T18:14:37.726 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.725+0000 7fa0c3fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c40d0b10 con 0x7fa0c40d1a20 2023-12-19T18:14:37.726 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.725+0000 7fa0c3fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c415d500 con 0x7fa0c40b3980 2023-12-19T18:14:37.726 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.726+0000 7fa0c37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa0c40b3980 0x7fa0c40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:37.726 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.726+0000 7fa0c37fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa0c40b3980 0x7fa0c40cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:52202/0 (socket says 172.21.15.192:52202) 2023-12-19T18:14:37.726 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.726+0000 7fa0c37fe640 1 -- 172.21.15.192:0/378131111 learned_addr learned my addr 172.21.15.192:0/378131111 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:37.726 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.726+0000 7fa0bbfff640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fa0c40d1a20 0x7fa0c40cd940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:43708/0 (socket says 172.21.15.192:43708) 2023-12-19T18:14:37.727 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.726+0000 7fa0c37fe640 1 -- 172.21.15.192:0/378131111 >> v1:172.21.15.192:6789/0 conn(0x7fa0c40d1a20 legacy=0x7fa0c40cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:37.727 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.726+0000 7fa0c37fe640 1 -- 172.21.15.192:0/378131111 >> v1:172.21.15.27:6789/0 conn(0x7fa0c415d120 legacy=0x7fa0c40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:37.727 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.726+0000 7fa0c37fe640 1 -- 172.21.15.192:0/378131111 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40d1040 con 0x7fa0c40b3980 2023-12-19T18:14:37.728 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.727+0000 7fa0c37fe640 1 --2- 172.21.15.192:0/378131111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40b3980 0x7fa0c40cfd70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009e50 tx=0x7fa0bc02bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ebc9587c307ecbd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:37.728 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.727+0000 7fa0c27fc640 1 -- 172.21.15.192:0/378131111 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc02cb00 con 0x7fa0c40b3980 2023-12-19T18:14:37.728 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.728+0000 7fa0c27fc640 1 -- 172.21.15.192:0/378131111 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc02cc60 con 0x7fa0c40b3980 2023-12-19T18:14:37.728 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.728+0000 7fa0c27fc640 1 -- 172.21.15.192:0/378131111 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc0343f0 con 0x7fa0c40b3980 2023-12-19T18:14:37.729 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.728+0000 7fa0c3fff640 1 -- 172.21.15.192:0/378131111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40b3980 msgr2=0x7fa0c40cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:37.729 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.728+0000 7fa0c3fff640 1 --2- 172.21.15.192:0/378131111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40b3980 0x7fa0c40cfd70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc009e50 tx=0x7fa0bc02bf60 comp rx=0 tx=0).stop 2023-12-19T18:14:37.729 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.728+0000 7fa0c3fff640 1 -- 172.21.15.192:0/378131111 shutdown_connections 2023-12-19T18:14:37.729 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.728+0000 7fa0c3fff640 1 --2- 172.21.15.192:0/378131111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40b3980 0x7fa0c40cfd70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.729 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.728+0000 7fa0c3fff640 1 -- 172.21.15.192:0/378131111 >> 172.21.15.192:0/378131111 conn(0x7fa0c40d2d30 msgr2=0x7fa0c40d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:37.729 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.729+0000 7fa0c3fff640 1 -- 172.21.15.192:0/378131111 shutdown_connections 2023-12-19T18:14:37.729 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.729+0000 7fa0c3fff640 1 -- 172.21.15.192:0/378131111 wait complete. 2023-12-19T18:14:37.730 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.730+0000 7fa0c3fff640 1 Processor -- start 2023-12-19T18:14:37.730 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.730+0000 7fa0c3fff640 1 -- start start 2023-12-19T18:14:37.732 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.731+0000 7fa0c3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c40b3980 0x7fa0c415f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:37.732 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.731+0000 7fa0c3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40d1a20 0x7fa0c40b6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.731+0000 7fa0c3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0c415d120 0x7fa0c40afb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.731+0000 7fa0c3fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b6ad0 con 0x7fa0c40b3980 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.731+0000 7fa0c3fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c415fde0 con 0x7fa0c40d1a20 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.731+0000 7fa0c3fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b5e50 con 0x7fa0c415d120 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.732+0000 7fa0bbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40d1a20 0x7fa0c40b6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.732+0000 7fa0bbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40d1a20 0x7fa0c40b6ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:52216/0 (socket says 172.21.15.192:52216) 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.732+0000 7fa0c37fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0c415d120 0x7fa0c40afb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.732+0000 7fa0bbfff640 1 -- 172.21.15.192:0/3537898421 learned_addr learned my addr 172.21.15.192:0/3537898421 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.732+0000 7fa0c2ffd640 1 --2- 172.21.15.192:0/3537898421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c40b3980 0x7fa0c415f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.732+0000 7fa0c37fe640 1 -- 172.21.15.192:0/3537898421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40d1a20 msgr2=0x7fa0c40b6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.732+0000 7fa0c37fe640 1 --2- 172.21.15.192:0/3537898421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40d1a20 0x7fa0c40b6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.732+0000 7fa0c37fe640 1 -- 172.21.15.192:0/3537898421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c40b3980 msgr2=0x7fa0c415f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.732+0000 7fa0c37fe640 1 --2- 172.21.15.192:0/3537898421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c40b3980 0x7fa0c415f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.733 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.732+0000 7fa0c37fe640 1 -- 172.21.15.192:0/3537898421 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc009b30 con 0x7fa0c415d120 2023-12-19T18:14:37.734 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.732+0000 7fa0bbfff640 1 --2- 172.21.15.192:0/3537898421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40d1a20 0x7fa0c40b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:37.734 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.733+0000 7fa0c37fe640 1 --2- 172.21.15.192:0/3537898421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0c415d120 0x7fa0c40afb60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc032000 tx=0x7fa0bc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:37.735 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.733+0000 7fa0bb7fe640 1 -- 172.21.15.192:0/3537898421 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc03e450 con 0x7fa0c415d120 2023-12-19T18:14:37.735 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.733+0000 7fa0bb7fe640 1 -- 172.21.15.192:0/3537898421 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0bc03e5b0 con 0x7fa0c415d120 2023-12-19T18:14:37.735 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.733+0000 7fa0bb7fe640 1 -- 172.21.15.192:0/3537898421 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0bc03e960 con 0x7fa0c415d120 2023-12-19T18:14:37.735 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.733+0000 7fa0c3fff640 1 -- 172.21.15.192:0/3537898421 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c40b6130 con 0x7fa0c415d120 2023-12-19T18:14:37.735 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.733+0000 7fa0c3fff640 1 -- 172.21.15.192:0/3537898421 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c40b66f0 con 0x7fa0c415d120 2023-12-19T18:14:37.735 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.733+0000 7fa0bb7fe640 1 -- 172.21.15.192:0/3537898421 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa0bc048410 con 0x7fa0c415d120 2023-12-19T18:14:37.735 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.733+0000 7fa0bb7fe640 1 -- 172.21.15.192:0/3537898421 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa0bc03edc0 con 0x7fa0c415d120 2023-12-19T18:14:37.735 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.734+0000 7fa0c3fff640 1 -- 172.21.15.192:0/3537898421 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c40b6c10 con 0x7fa0c415d120 2023-12-19T18:14:37.737 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.737+0000 7fa0bb7fe640 1 -- 172.21.15.192:0/3537898421 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7fa0c40b6c10 con 0x7fa0c415d120 2023-12-19T18:14:37.815 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.815+0000 7fa0c3fff640 1 -- 172.21.15.192:0/3537898421 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0} v 0) v1 -- 0x7fa0c404e840 con 0x7fa0c415d120 2023-12-19T18:14:37.825 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.824+0000 7fa0bb7fe640 1 -- 172.21.15.192:0/3537898421 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa0bc043020 con 0x7fa0c415d120 2023-12-19T18:14:37.825 INFO:teuthology.orchestra.run.smithi192.stdout:0 2023-12-19T18:14:37.826 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.826+0000 7fa0c3fff640 1 -- 172.21.15.192:0/3537898421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0c415d120 msgr2=0x7fa0c40afb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:37.826 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.826+0000 7fa0c3fff640 1 --2- 172.21.15.192:0/3537898421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0c415d120 0x7fa0c40afb60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc032000 tx=0x7fa0bc004a40 comp rx=0 tx=0).stop 2023-12-19T18:14:37.826 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.826+0000 7fa0c3fff640 1 -- 172.21.15.192:0/3537898421 shutdown_connections 2023-12-19T18:14:37.826 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.826+0000 7fa0c3fff640 1 --2- 172.21.15.192:0/3537898421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0c415d120 0x7fa0c40afb60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.826 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.826+0000 7fa0c3fff640 1 --2- 172.21.15.192:0/3537898421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c40d1a20 0x7fa0c40b6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.826 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.826+0000 7fa0c3fff640 1 --2- 172.21.15.192:0/3537898421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c40b3980 0x7fa0c415f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:37.827 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.826+0000 7fa0c3fff640 1 -- 172.21.15.192:0/3537898421 >> 172.21.15.192:0/3537898421 conn(0x7fa0c40d2d30 msgr2=0x7fa0c40d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:37.827 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.826+0000 7fa0c3fff640 1 -- 172.21.15.192:0/3537898421 shutdown_connections 2023-12-19T18:14:37.827 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:37.826+0000 7fa0c3fff640 1 -- 172.21.15.192:0/3537898421 wait complete. 2023-12-19T18:14:37.843 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 1 2023-12-19T18:14:37.859 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:37.855+0000 7fe47583b040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:14:37.913 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:37.912+0000 7fbc5e62e040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-19T18:14:37.920 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:37.920+0000 7fe47583b040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:14:37.975 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:37.974+0000 7fe47583b040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:14:37.977 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:37.977+0000 7fbc5e62e040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-19T18:14:38.031 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:38.032+0000 7fbc5e62e040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-19T18:14:38.033 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.033+0000 7fe54b751640 1 Processor -- start 2023-12-19T18:14:38.034 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.033+0000 7fe54b751640 1 -- start start 2023-12-19T18:14:38.035 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.036+0000 7fe54b751640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe5440b7c60 0x7fe5440d1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.035 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.036+0000 7fe54b751640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5440b67c0 con 0x7fe5440b8b10 2023-12-19T18:14:38.035 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.036+0000 7fe54b751640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5440b8ef0 con 0x7fe5440cfe80 2023-12-19T18:14:38.035 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.036+0000 7fe54b751640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5440d2070 con 0x7fe5440b7c60 2023-12-19T18:14:38.036 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.036+0000 7fe548cec640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fe5440cfe80 0x7fe5440d1340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:43720/0 (socket says 172.21.15.192:43720) 2023-12-19T18:14:38.036 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.036+0000 7fe548cec640 1 -- 172.21.15.192:0/1219596387 learned_addr learned my addr 172.21.15.192:0/1219596387 (peer_addr_for_me v1:172.21.15.192:0/0) 2023-12-19T18:14:38.037 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.036+0000 7fe549cee640 1 --2- 172.21.15.192:0/1219596387 >> v2:172.21.15.27:3300/0 conn(0x7fe5440b7c60 0x7fe5440d1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290377389 0 0) 0x7fe5440b8ef0 con 0x7fe5440cfe80 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe530003520 con 0x7fe5440cfe80 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3356094259 0 0) 0x7fe530003520 con 0x7fe5440cfe80 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 >> v2:172.21.15.27:3300/0 conn(0x7fe5440b7c60 msgr2=0x7fe5440d1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 --2- 172.21.15.192:0/1219596387 >> v2:172.21.15.27:3300/0 conn(0x7fe5440b7c60 0x7fe5440d1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 >> v1:172.21.15.27:6789/0 conn(0x7fe5440b8b10 legacy=0x7fe5440cf770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 --> v1:172.21.15.192:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5440d2350 con 0x7fe5440cfe80 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 <== mon.1 v1:172.21.15.192:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe534001670 con 0x7fe5440cfe80 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 >> v1:172.21.15.192:6789/0 conn(0x7fe5440cfe80 legacy=0x7fe5440d1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 --2- 172.21.15.192:0/1219596387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5300039c0 0x7fe530003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 --2- 172.21.15.192:0/1219596387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5300047c0 0x7fe530004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 --2- 172.21.15.192:0/1219596387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe530005180 0x7fe53000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5440d2070 con 0x7fe530005180 2023-12-19T18:14:38.038 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53000c080 con 0x7fe5300047c0 2023-12-19T18:14:38.039 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe53000c240 con 0x7fe5300039c0 2023-12-19T18:14:38.039 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe5494ed640 1 --2- 172.21.15.192:0/1219596387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5300039c0 0x7fe530003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.039 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.037+0000 7fe548cec640 1 --2- 172.21.15.192:0/1219596387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5300047c0 0x7fe530004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.039 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.038+0000 7fe549cee640 1 --2- 172.21.15.192:0/1219596387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe530005180 0x7fe53000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.039 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.038+0000 7fe5494ed640 1 -- 172.21.15.192:0/1219596387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5300047c0 msgr2=0x7fe530004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:38.039 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.038+0000 7fe5494ed640 1 --2- 172.21.15.192:0/1219596387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5300047c0 0x7fe530004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.039 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.038+0000 7fe5494ed640 1 -- 172.21.15.192:0/1219596387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe530005180 msgr2=0x7fe53000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:38.039 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.038+0000 7fe5494ed640 1 --2- 172.21.15.192:0/1219596387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe530005180 0x7fe53000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.039 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.038+0000 7fe5494ed640 1 -- 172.21.15.192:0/1219596387 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe530003520 con 0x7fe5300039c0 2023-12-19T18:14:38.039 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.038+0000 7fe549cee640 1 --2- 172.21.15.192:0/1219596387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe530005180 0x7fe53000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:38.039 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.038+0000 7fe549cee640 1 -- 172.21.15.192:0/1219596387 reap_dead start 2023-12-19T18:14:38.040 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.039+0000 7fe5494ed640 1 --2- 172.21.15.192:0/1219596387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5300039c0 0x7fe530003de0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe540001750 tx=0x7fe5300057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67d00254f353de4d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:38.040 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.039+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5400052d0 con 0x7fe5300039c0 2023-12-19T18:14:38.041 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.039+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54002e050 con 0x7fe5300039c0 2023-12-19T18:14:38.041 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.039+0000 7fe53bfff640 1 -- 172.21.15.192:0/1219596387 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe540003830 con 0x7fe5300039c0 2023-12-19T18:14:38.041 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.040+0000 7fe54b751640 1 -- 172.21.15.192:0/1219596387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5300039c0 msgr2=0x7fe530003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.041 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.040+0000 7fe54b751640 1 --2- 172.21.15.192:0/1219596387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5300039c0 0x7fe530003de0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe540001750 tx=0x7fe5300057f0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.041 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.040+0000 7fe54b751640 1 -- 172.21.15.192:0/1219596387 shutdown_connections 2023-12-19T18:14:38.041 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.040+0000 7fe54b751640 1 --2- 172.21.15.192:0/1219596387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5300039c0 0x7fe530003de0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.041 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.040+0000 7fe54b751640 1 -- 172.21.15.192:0/1219596387 >> 172.21.15.192:0/1219596387 conn(0x7fe5440d2fa0 msgr2=0x7fe5440d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:38.041 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.040+0000 7fe54b751640 1 -- 172.21.15.192:0/1219596387 shutdown_connections 2023-12-19T18:14:38.041 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.040+0000 7fe54b751640 1 -- 172.21.15.192:0/1219596387 wait complete. 2023-12-19T18:14:38.042 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:38.041+0000 7fe47583b040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:14:38.043 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.042+0000 7fe54b751640 1 Processor -- start 2023-12-19T18:14:38.043 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.042+0000 7fe54b751640 1 -- start start 2023-12-19T18:14:38.043 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe54b751640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5300039c0 0x7fe5441f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.043 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe54b751640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544156600 0x7fe544159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe54b751640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe544157e30 0x7fe5441582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe54b751640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe544156a80 con 0x7fe544156600 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe54b751640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe544156d90 con 0x7fe544157e30 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe54b751640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe544157070 con 0x7fe5300039c0 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe5494ed640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5300039c0 0x7fe5441f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe5494ed640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5300039c0 0x7fe5441f6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54270/0 (socket says 172.21.15.192:54270) 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe549cee640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe544157e30 0x7fe5441582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe5494ed640 1 -- 172.21.15.192:0/3463791487 learned_addr learned my addr 172.21.15.192:0/3463791487 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe548cec640 1 --2- 172.21.15.192:0/3463791487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544156600 0x7fe544159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe5494ed640 1 -- 172.21.15.192:0/3463791487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe544157e30 msgr2=0x7fe5441582b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe5494ed640 1 --2- 172.21.15.192:0/3463791487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe544157e30 0x7fe5441582b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe5494ed640 1 -- 172.21.15.192:0/3463791487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544156600 msgr2=0x7fe544159250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe5494ed640 1 --2- 172.21.15.192:0/3463791487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544156600 0x7fe544159250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe5494ed640 1 -- 172.21.15.192:0/3463791487 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe544157350 con 0x7fe5300039c0 2023-12-19T18:14:38.044 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.043+0000 7fe549cee640 1 --2- 172.21.15.192:0/3463791487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe544157e30 0x7fe5441582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:38.045 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.044+0000 7fe5494ed640 1 --2- 172.21.15.192:0/3463791487 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5300039c0 0x7fe5441f6e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe540034040 tx=0x7fe540001120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:38.045 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.044+0000 7fe539ffb640 1 -- 172.21.15.192:0/3463791487 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5400076c0 con 0x7fe5300039c0 2023-12-19T18:14:38.045 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.044+0000 7fe539ffb640 1 -- 172.21.15.192:0/3463791487 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5400069e0 con 0x7fe5300039c0 2023-12-19T18:14:38.045 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.044+0000 7fe539ffb640 1 -- 172.21.15.192:0/3463791487 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe540006d90 con 0x7fe5300039c0 2023-12-19T18:14:38.045 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.044+0000 7fe54b751640 1 -- 172.21.15.192:0/3463791487 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe544157630 con 0x7fe5300039c0 2023-12-19T18:14:38.045 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.044+0000 7fe54b751640 1 -- 172.21.15.192:0/3463791487 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe544159790 con 0x7fe5300039c0 2023-12-19T18:14:38.045 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.045+0000 7fe539ffb640 1 -- 172.21.15.192:0/3463791487 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe5400038f0 con 0x7fe5300039c0 2023-12-19T18:14:38.045 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.045+0000 7fe539ffb640 1 -- 172.21.15.192:0/3463791487 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7fe540049b70 con 0x7fe5300039c0 2023-12-19T18:14:38.046 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.045+0000 7fe54b751640 1 -- 172.21.15.192:0/3463791487 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe544157070 con 0x7fe5300039c0 2023-12-19T18:14:38.047 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.048+0000 7fe539ffb640 1 -- 172.21.15.192:0/3463791487 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7fe540001370 con 0x7fe5300039c0 2023-12-19T18:14:38.094 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:38.094+0000 7fe47583b040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:14:38.098 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:38.099+0000 7fbc5e62e040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-19T18:14:38.125 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.125+0000 7fe54b751640 1 -- 172.21.15.192:0/3463791487 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1} v 0) v1 -- 0x7fe544067110 con 0x7fe5300039c0 2023-12-19T18:14:38.137 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.135+0000 7fe539ffb640 1 -- 172.21.15.192:0/3463791487 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe540049d30 con 0x7fe5300039c0 2023-12-19T18:14:38.137 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-19T18:14:38.138 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.136+0000 7fe54b751640 1 -- 172.21.15.192:0/3463791487 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5300039c0 msgr2=0x7fe5441f6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.138 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.136+0000 7fe54b751640 1 --2- 172.21.15.192:0/3463791487 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5300039c0 0x7fe5441f6e20 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe540034040 tx=0x7fe540001120 comp rx=0 tx=0).stop 2023-12-19T18:14:38.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.136+0000 7fe54b751640 1 -- 172.21.15.192:0/3463791487 shutdown_connections 2023-12-19T18:14:38.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.136+0000 7fe54b751640 1 --2- 172.21.15.192:0/3463791487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe544157e30 0x7fe5441582b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.139 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.136+0000 7fe54b751640 1 --2- 172.21.15.192:0/3463791487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe544156600 0x7fe544159250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.140 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.136+0000 7fe54b751640 1 --2- 172.21.15.192:0/3463791487 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5300039c0 0x7fe5441f6e20 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.140 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.136+0000 7fe54b751640 1 -- 172.21.15.192:0/3463791487 >> 172.21.15.192:0/3463791487 conn(0x7fe5440d2fa0 msgr2=0x7fe544160350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:38.140 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.136+0000 7fe54b751640 1 -- 172.21.15.192:0/3463791487 shutdown_connections 2023-12-19T18:14:38.140 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.137+0000 7fe54b751640 1 -- 172.21.15.192:0/3463791487 wait complete. 2023-12-19T18:14:38.152 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:38.153+0000 7fbc5e62e040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-19T18:14:38.153 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 2 2023-12-19T18:14:38.214 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:38.214+0000 7fe47583b040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-19T18:14:38.275 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:38.276+0000 7fbc5e62e040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-19T18:14:38.299 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:38.299+0000 7fe47583b040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:14:38.345 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.346+0000 7ff77dc65640 1 Processor -- start 2023-12-19T18:14:38.345 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.346+0000 7ff77dc65640 1 -- start start 2023-12-19T18:14:38.347 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.348+0000 7ff77dc65640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff7780cbda0 0x7ff7780704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.348 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.348+0000 7ff77dc65640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff778070a00 0x7ff7780cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.348 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.348+0000 7ff77dc65640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77806b1e0 con 0x7ff77815abf0 2023-12-19T18:14:38.348 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.348+0000 7ff77dc65640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff778070e70 con 0x7ff7780cbda0 2023-12-19T18:14:38.348 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.348+0000 7ff77dc65640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77806b450 con 0x7ff778070a00 2023-12-19T18:14:38.348 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.348+0000 7ff77d464640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff778070a00 0x7ff7780cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.348 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.348+0000 7ff77d464640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff778070a00 0x7ff7780cd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54284/0 (socket says 172.21.15.192:54284) 2023-12-19T18:14:38.348 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.348+0000 7ff77d464640 1 -- 172.21.15.192:0/768485014 learned_addr learned my addr 172.21.15.192:0/768485014 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:38.348 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.348+0000 7ff777fff640 1 --2- 172.21.15.192:0/768485014 >> v2:172.21.15.27:3300/0 conn(0x7ff7780cbda0 0x7ff7780704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.348 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.348+0000 7ff77d464640 1 -- 172.21.15.192:0/768485014 >> v2:172.21.15.27:3300/0 conn(0x7ff7780cbda0 msgr2=0x7ff7780704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:38.348 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.348+0000 7ff77d464640 1 --2- 172.21.15.192:0/768485014 >> v2:172.21.15.27:3300/0 conn(0x7ff7780cbda0 0x7ff7780704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.348 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.348+0000 7ff77d464640 1 -- 172.21.15.192:0/768485014 >> v1:172.21.15.27:6789/0 conn(0x7ff77815abf0 legacy=0x7ff7780cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:38.348 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.348+0000 7ff77d464640 1 -- 172.21.15.192:0/768485014 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff77806b730 con 0x7ff778070a00 2023-12-19T18:14:38.349 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.349+0000 7ff77d464640 1 --2- 172.21.15.192:0/768485014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff778070a00 0x7ff7780cd630 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff7780b5bd0 tx=0x7ff77002bf50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a43d18b53d5078fa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:38.349 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.350+0000 7ff7777fe640 1 -- 172.21.15.192:0/768485014 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff77002caf0 con 0x7ff778070a00 2023-12-19T18:14:38.349 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.350+0000 7ff7777fe640 1 -- 172.21.15.192:0/768485014 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff77002cc50 con 0x7ff778070a00 2023-12-19T18:14:38.349 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.350+0000 7ff7777fe640 1 -- 172.21.15.192:0/768485014 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7700343f0 con 0x7ff778070a00 2023-12-19T18:14:38.350 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.350+0000 7ff77dc65640 1 -- 172.21.15.192:0/768485014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff778070a00 msgr2=0x7ff7780cd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.350 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.350+0000 7ff77dc65640 1 --2- 172.21.15.192:0/768485014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff778070a00 0x7ff7780cd630 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7ff7780b5bd0 tx=0x7ff77002bf50 comp rx=0 tx=0).stop 2023-12-19T18:14:38.350 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.351+0000 7ff77dc65640 1 -- 172.21.15.192:0/768485014 shutdown_connections 2023-12-19T18:14:38.350 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.351+0000 7ff77dc65640 1 --2- 172.21.15.192:0/768485014 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff778070a00 0x7ff7780cd630 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.350 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.351+0000 7ff77dc65640 1 --2- 172.21.15.192:0/768485014 >> v2:172.21.15.27:3300/0 conn(0x7ff7780cbda0 0x7ff7780704c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.351 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.351+0000 7ff77dc65640 1 -- 172.21.15.192:0/768485014 >> 172.21.15.192:0/768485014 conn(0x7ff7780cfa60 msgr2=0x7ff7780cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:38.351 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.351+0000 7ff77dc65640 1 -- 172.21.15.192:0/768485014 shutdown_connections 2023-12-19T18:14:38.351 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.351+0000 7ff77dc65640 1 -- 172.21.15.192:0/768485014 wait complete. 2023-12-19T18:14:38.351 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.352+0000 7ff77dc65640 1 Processor -- start 2023-12-19T18:14:38.351 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.352+0000 7ff77dc65640 1 -- start start 2023-12-19T18:14:38.354 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.353+0000 7ff77dc65640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff778070a00 0x7ff778159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.354 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.353+0000 7ff77dc65640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77815abf0 0x7ff77806cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.354 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.353+0000 7ff77dc65640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff778069c70 0x7ff77806a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.354 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.353+0000 7ff77dc65640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77806a660 con 0x7ff77815abf0 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.353+0000 7ff77dc65640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff77806c240 con 0x7ff778069c70 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.353+0000 7ff77dc65640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7780b42f0 con 0x7ff778070a00 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.354+0000 7ff77d464640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff778069c70 0x7ff77806a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.354+0000 7ff77cc63640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff778070a00 0x7ff778159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.354+0000 7ff77d464640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff778069c70 0x7ff77806a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:52246/0 (socket says 172.21.15.192:52246) 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.354+0000 7ff77d464640 1 -- 172.21.15.192:0/254788875 learned_addr learned my addr 172.21.15.192:0/254788875 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.354+0000 7ff777fff640 1 --2- 172.21.15.192:0/254788875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77815abf0 0x7ff77806cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.354+0000 7ff77cc63640 1 -- 172.21.15.192:0/254788875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff778069c70 msgr2=0x7ff77806a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.354+0000 7ff77cc63640 1 --2- 172.21.15.192:0/254788875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff778069c70 0x7ff77806a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.354+0000 7ff77cc63640 1 -- 172.21.15.192:0/254788875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77815abf0 msgr2=0x7ff77806cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.354+0000 7ff77cc63640 1 --2- 172.21.15.192:0/254788875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77815abf0 0x7ff77806cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.354+0000 7ff77cc63640 1 -- 172.21.15.192:0/254788875 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff770009b30 con 0x7ff778070a00 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.354+0000 7ff77d464640 1 --2- 172.21.15.192:0/254788875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff778069c70 0x7ff77806a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:38.355 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.354+0000 7ff77cc63640 1 --2- 172.21.15.192:0/254788875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff778070a00 0x7ff778159ed0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00b3e0 tx=0x7ff76c00b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:38.356 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.355+0000 7ff7757fa640 1 -- 172.21.15.192:0/254788875 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c011830 con 0x7ff778070a00 2023-12-19T18:14:38.356 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.355+0000 7ff7757fa640 1 -- 172.21.15.192:0/254788875 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76c011990 con 0x7ff778070a00 2023-12-19T18:14:38.356 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.355+0000 7ff77dc65640 1 -- 172.21.15.192:0/254788875 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7780b4540 con 0x7ff778070a00 2023-12-19T18:14:38.356 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.355+0000 7ff7757fa640 1 -- 172.21.15.192:0/254788875 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76c011d40 con 0x7ff778070a00 2023-12-19T18:14:38.356 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.355+0000 7ff77dc65640 1 -- 172.21.15.192:0/254788875 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77806c450 con 0x7ff778070a00 2023-12-19T18:14:38.356 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.355+0000 7ff7757fa640 1 -- 172.21.15.192:0/254788875 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff76c01a700 con 0x7ff778070a00 2023-12-19T18:14:38.356 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.355+0000 7ff7757fa640 1 -- 172.21.15.192:0/254788875 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7ff76c01add0 con 0x7ff778070a00 2023-12-19T18:14:38.356 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:38.356+0000 7fe47583b040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:14:38.357 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.356+0000 7ff77dc65640 1 -- 172.21.15.192:0/254788875 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77804e840 con 0x7ff778070a00 2023-12-19T18:14:38.358 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.359+0000 7ff7757fa640 1 -- 172.21.15.192:0/254788875 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7ff76c039b40 con 0x7ff778070a00 2023-12-19T18:14:38.360 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:38.361+0000 7fbc5e62e040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-19T18:14:38.416 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:38.416+0000 7fbc5e62e040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-19T18:14:38.420 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:38.420+0000 7fe47583b040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:14:38.436 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.437+0000 7ff77dc65640 1 -- 172.21.15.192:0/254788875 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2} v 0) v1 -- 0x7ff7780b36c0 con 0x7ff778070a00 2023-12-19T18:14:38.443 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.444+0000 7ff7757fa640 1 -- 172.21.15.192:0/254788875 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff76c028070 con 0x7ff778070a00 2023-12-19T18:14:38.443 INFO:teuthology.orchestra.run.smithi192.stdout:2 2023-12-19T18:14:38.445 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.445+0000 7ff77dc65640 1 -- 172.21.15.192:0/254788875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff778070a00 msgr2=0x7ff778159ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.445 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.445+0000 7ff77dc65640 1 --2- 172.21.15.192:0/254788875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff778070a00 0x7ff778159ed0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff76c00b3e0 tx=0x7ff76c00b8a0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.446 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.446+0000 7ff77dc65640 1 -- 172.21.15.192:0/254788875 shutdown_connections 2023-12-19T18:14:38.446 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.446+0000 7ff77dc65640 1 --2- 172.21.15.192:0/254788875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff778069c70 0x7ff77806a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.446 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.446+0000 7ff77dc65640 1 --2- 172.21.15.192:0/254788875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff77815abf0 0x7ff77806cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.446 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.446+0000 7ff77dc65640 1 --2- 172.21.15.192:0/254788875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff778070a00 0x7ff778159ed0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.446 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.446+0000 7ff77dc65640 1 -- 172.21.15.192:0/254788875 >> 172.21.15.192:0/254788875 conn(0x7ff7780cfa60 msgr2=0x7ff7780cec40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:38.446 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.446+0000 7ff77dc65640 1 -- 172.21.15.192:0/254788875 shutdown_connections 2023-12-19T18:14:38.446 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.446+0000 7ff77dc65640 1 -- 172.21.15.192:0/254788875 wait complete. 2023-12-19T18:14:38.468 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 3 2023-12-19T18:14:38.478 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:38.478+0000 7fe47583b040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:14:38.487 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:38.488+0000 7fbc5e62e040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-19T18:14:38.545 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:38.546+0000 7fbc5e62e040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-19T18:14:38.658 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.658+0000 7f01f08ed640 1 Processor -- start 2023-12-19T18:14:38.658 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.658+0000 7f01f08ed640 1 -- start start 2023-12-19T18:14:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.660+0000 7f01f08ed640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f01ec15d120 0x7f01ec0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.660+0000 7f01f08ed640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f01ec0d1310 0x7f01ec0d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.660+0000 7f01f08ed640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01ec0b2970 0x7f01ec0b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.660+0000 7f01f08ed640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ec0d3850 con 0x7f01ec0d1310 2023-12-19T18:14:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.660+0000 7f01f08ed640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01ec0d0b10 con 0x7f01ec15d120 2023-12-19T18:14:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.660+0000 7f01f08ed640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec0d1c90 con 0x7f01ec0b2970 2023-12-19T18:14:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.660+0000 7f01ead9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f01ec15d120 0x7f01ec0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.660+0000 7f01ead9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f01ec15d120 0x7f01ec0cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:52262/0 (socket says 172.21.15.192:52262) 2023-12-19T18:14:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.660+0000 7f01ead9d640 1 -- 172.21.15.192:0/382757676 learned_addr learned my addr 172.21.15.192:0/382757676 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:38.660 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.660+0000 7f01eb59e640 1 --2- 172.21.15.192:0/382757676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01ec0b2970 0x7f01ec0b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.661 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.661+0000 7f01ea59c640 1 --2- 172.21.15.192:0/382757676 >> v2:172.21.15.27:3300/0 conn(0x7f01ec0d1310 0x7f01ec0d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.661 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.661+0000 7f01ead9d640 1 -- 172.21.15.192:0/382757676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01ec0b2970 msgr2=0x7f01ec0b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:38.661 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.661+0000 7f01ead9d640 1 --2- 172.21.15.192:0/382757676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01ec0b2970 0x7f01ec0b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.661 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.661+0000 7f01ead9d640 1 -- 172.21.15.192:0/382757676 >> v2:172.21.15.27:3300/0 conn(0x7f01ec0d1310 msgr2=0x7f01ec0d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:38.661 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.661+0000 7f01ead9d640 1 --2- 172.21.15.192:0/382757676 >> v2:172.21.15.27:3300/0 conn(0x7f01ec0d1310 0x7f01ec0d1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.661 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.661+0000 7f01ead9d640 1 -- 172.21.15.192:0/382757676 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec15d500 con 0x7f01ec15d120 2023-12-19T18:14:38.661 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.661+0000 7f01eb59e640 1 --2- 172.21.15.192:0/382757676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01ec0b2970 0x7f01ec0b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:38.662 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.662+0000 7f01ead9d640 1 --2- 172.21.15.192:0/382757676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01ec15d120 0x7f01ec0cfd70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f01ec071e70 tx=0x7f01dc02b900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf499716939be40 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:38.662 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.662+0000 7f01e9d9b640 1 -- 172.21.15.192:0/382757676 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc031860 con 0x7f01ec15d120 2023-12-19T18:14:38.662 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.662+0000 7f01e9d9b640 1 -- 172.21.15.192:0/382757676 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc0319c0 con 0x7f01ec15d120 2023-12-19T18:14:38.662 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.662+0000 7f01e9d9b640 1 -- 172.21.15.192:0/382757676 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc031d50 con 0x7f01ec15d120 2023-12-19T18:14:38.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.663+0000 7f01f08ed640 1 -- 172.21.15.192:0/382757676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01ec15d120 msgr2=0x7f01ec0cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.663+0000 7f01f08ed640 1 --2- 172.21.15.192:0/382757676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01ec15d120 0x7f01ec0cfd70 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f01ec071e70 tx=0x7f01dc02b900 comp rx=0 tx=0).stop 2023-12-19T18:14:38.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.663+0000 7f01f08ed640 1 -- 172.21.15.192:0/382757676 shutdown_connections 2023-12-19T18:14:38.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.663+0000 7f01f08ed640 1 --2- 172.21.15.192:0/382757676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01ec0b2970 0x7f01ec0b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.663+0000 7f01f08ed640 1 --2- 172.21.15.192:0/382757676 >> v2:172.21.15.27:3300/0 conn(0x7f01ec0d1310 0x7f01ec0d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.663+0000 7f01f08ed640 1 --2- 172.21.15.192:0/382757676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01ec15d120 0x7f01ec0cfd70 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.663+0000 7f01f08ed640 1 -- 172.21.15.192:0/382757676 >> 172.21.15.192:0/382757676 conn(0x7f01ec0d2d30 msgr2=0x7f01ec0d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:38.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.663+0000 7f01f08ed640 1 -- 172.21.15.192:0/382757676 shutdown_connections 2023-12-19T18:14:38.663 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.663+0000 7f01f08ed640 1 -- 172.21.15.192:0/382757676 wait complete. 2023-12-19T18:14:38.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.665+0000 7f01f08ed640 1 Processor -- start 2023-12-19T18:14:38.664 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.665+0000 7f01f08ed640 1 -- start start 2023-12-19T18:14:38.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01f08ed640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01ec0b2970 0x7f01ec14e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01f08ed640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01ec0d1310 0x7f01ec14ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01f08ed640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01ec15d120 0x7f01ec0b5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:38.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01f08ed640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec14e7c0 con 0x7f01ec0b2970 2023-12-19T18:14:38.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01f08ed640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec15f660 con 0x7f01ec15d120 2023-12-19T18:14:38.667 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01f08ed640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec14f1a0 con 0x7f01ec0d1310 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01ea59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01ec0d1310 0x7f01ec14ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01eb59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01ec15d120 0x7f01ec0b5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01ea59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01ec0d1310 0x7f01ec14ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54308/0 (socket says 172.21.15.192:54308) 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01ea59c640 1 -- 172.21.15.192:0/1160819966 learned_addr learned my addr 172.21.15.192:0/1160819966 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01ead9d640 1 --2- 172.21.15.192:0/1160819966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01ec0b2970 0x7f01ec14e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01ea59c640 1 -- 172.21.15.192:0/1160819966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01ec15d120 msgr2=0x7f01ec0b5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01ea59c640 1 --2- 172.21.15.192:0/1160819966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01ec15d120 0x7f01ec0b5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01ea59c640 1 -- 172.21.15.192:0/1160819966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01ec0b2970 msgr2=0x7f01ec14e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01ea59c640 1 --2- 172.21.15.192:0/1160819966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01ec0b2970 0x7f01ec14e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.666+0000 7f01ea59c640 1 -- 172.21.15.192:0/1160819966 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e4009a70 con 0x7f01ec0d1310 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.667+0000 7f01ea59c640 1 --2- 172.21.15.192:0/1160819966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01ec0d1310 0x7f01ec14ec60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01e000d5f0 tx=0x7f01e000d9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.667+0000 7f01d37fe640 1 -- 172.21.15.192:0/1160819966 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e00125f0 con 0x7f01ec0d1310 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.667+0000 7f01d37fe640 1 -- 172.21.15.192:0/1160819966 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e0012750 con 0x7f01ec0d1310 2023-12-19T18:14:38.668 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.667+0000 7f01d37fe640 1 -- 172.21.15.192:0/1160819966 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0012b00 con 0x7f01ec0d1310 2023-12-19T18:14:38.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.667+0000 7f01f08ed640 1 -- 172.21.15.192:0/1160819966 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01dc0094d0 con 0x7f01ec0d1310 2023-12-19T18:14:38.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.667+0000 7f01f08ed640 1 -- 172.21.15.192:0/1160819966 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ec15ed30 con 0x7f01ec0d1310 2023-12-19T18:14:38.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.667+0000 7f01d37fe640 1 -- 172.21.15.192:0/1160819966 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f01e001e460 con 0x7f01ec0d1310 2023-12-19T18:14:38.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.668+0000 7f01d37fe640 1 -- 172.21.15.192:0/1160819966 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f01e0014960 con 0x7f01ec0d1310 2023-12-19T18:14:38.669 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.668+0000 7f01f08ed640 1 -- 172.21.15.192:0/1160819966 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01ec04e840 con 0x7f01ec0d1310 2023-12-19T18:14:38.670 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.671+0000 7f01d37fe640 1 -- 172.21.15.192:0/1160819966 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f01e0014cb0 con 0x7f01ec0d1310 2023-12-19T18:14:38.748 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.748+0000 7f01f08ed640 1 -- 172.21.15.192:0/1160819966 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3} v 0) v1 -- 0x7f01ec0b1910 con 0x7f01ec0d1310 2023-12-19T18:14:38.755 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.755+0000 7f01d37fe640 1 -- 172.21.15.192:0/1160819966 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f01e00144d0 con 0x7f01ec0d1310 2023-12-19T18:14:38.755 INFO:teuthology.orchestra.run.smithi192.stdout:3 2023-12-19T18:14:38.756 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.757+0000 7f01f08ed640 1 -- 172.21.15.192:0/1160819966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01ec0d1310 msgr2=0x7f01ec14ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:38.756 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.757+0000 7f01f08ed640 1 --2- 172.21.15.192:0/1160819966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01ec0d1310 0x7f01ec14ec60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f01e000d5f0 tx=0x7f01e000d9f0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.756 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.757+0000 7f01f08ed640 1 -- 172.21.15.192:0/1160819966 shutdown_connections 2023-12-19T18:14:38.756 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.757+0000 7f01f08ed640 1 --2- 172.21.15.192:0/1160819966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01ec15d120 0x7f01ec0b5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.756 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.757+0000 7f01f08ed640 1 --2- 172.21.15.192:0/1160819966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01ec0d1310 0x7f01ec14ec60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.756 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.757+0000 7f01f08ed640 1 --2- 172.21.15.192:0/1160819966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01ec0b2970 0x7f01ec14e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:38.756 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.757+0000 7f01f08ed640 1 -- 172.21.15.192:0/1160819966 >> 172.21.15.192:0/1160819966 conn(0x7f01ec0d2d30 msgr2=0x7f01ec06a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:38.756 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.757+0000 7f01f08ed640 1 -- 172.21.15.192:0/1160819966 shutdown_connections 2023-12-19T18:14:38.757 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:38.757+0000 7f01f08ed640 1 -- 172.21.15.192:0/1160819966 wait complete. 2023-12-19T18:14:38.773 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 4 2023-12-19T18:14:38.961 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:38.961+0000 7fe47583b040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-19T18:14:39.008 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.008+0000 7f100890b640 1 Processor -- start 2023-12-19T18:14:39.008 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.009+0000 7f100890b640 1 -- start start 2023-12-19T18:14:39.010 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.010+0000 7f100890b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f100415d120 0x7f10040cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.010 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.010+0000 7f100890b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f10040ce950 0x7f10040d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.010 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.010+0000 7f100890b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10040d3850 con 0x7f10040d1310 2023-12-19T18:14:39.010 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.010+0000 7f100890b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10040d0b10 con 0x7f10040ce950 2023-12-19T18:14:39.010 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.010+0000 7f100890b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10040d1c90 con 0x7f100415d120 2023-12-19T18:14:39.010 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.011+0000 7f10037fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f100415d120 0x7f10040cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.011 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.011+0000 7f10037fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f100415d120 0x7f10040cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:52282/0 (socket says 172.21.15.192:52282) 2023-12-19T18:14:39.011 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.011+0000 7f10037fe640 1 -- 172.21.15.192:0/3602540035 learned_addr learned my addr 172.21.15.192:0/3602540035 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:39.011 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.011+0000 7f1003fff640 1 --2- 172.21.15.192:0/3602540035 >> v2:172.21.15.27:3300/0 conn(0x7f10040ce950 0x7f10040d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.011 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.011+0000 7f10037fe640 1 -- 172.21.15.192:0/3602540035 >> v2:172.21.15.27:3300/0 conn(0x7f10040ce950 msgr2=0x7f10040d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:39.011 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.011+0000 7f10037fe640 1 --2- 172.21.15.192:0/3602540035 >> v2:172.21.15.27:3300/0 conn(0x7f10040ce950 0x7f10040d1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.011 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.011+0000 7f10037fe640 1 -- 172.21.15.192:0/3602540035 >> v1:172.21.15.27:6789/0 conn(0x7f10040d1310 legacy=0x7f10040b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:39.012 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.011+0000 7f10037fe640 1 -- 172.21.15.192:0/3602540035 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100415d500 con 0x7f100415d120 2023-12-19T18:14:39.013 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.012+0000 7f10037fe640 1 --2- 172.21.15.192:0/3602540035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f100415d120 0x7f10040cfd70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1004071e70 tx=0x7f0ff402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f2b66e9783706af server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.013 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.012+0000 7f10027fc640 1 -- 172.21.15.192:0/3602540035 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4031870 con 0x7f100415d120 2023-12-19T18:14:39.014 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.012+0000 7f10027fc640 1 -- 172.21.15.192:0/3602540035 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff40319d0 con 0x7f100415d120 2023-12-19T18:14:39.014 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.012+0000 7f10027fc640 1 -- 172.21.15.192:0/3602540035 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff4031d60 con 0x7f100415d120 2023-12-19T18:14:39.014 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.013+0000 7f100890b640 1 -- 172.21.15.192:0/3602540035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f100415d120 msgr2=0x7f10040cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.014 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.013+0000 7f100890b640 1 --2- 172.21.15.192:0/3602540035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f100415d120 0x7f10040cfd70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1004071e70 tx=0x7f0ff402b910 comp rx=0 tx=0).stop 2023-12-19T18:14:39.014 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.013+0000 7f100890b640 1 -- 172.21.15.192:0/3602540035 shutdown_connections 2023-12-19T18:14:39.014 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.013+0000 7f100890b640 1 --2- 172.21.15.192:0/3602540035 >> v2:172.21.15.27:3300/0 conn(0x7f10040ce950 0x7f10040d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.015 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.013+0000 7f100890b640 1 --2- 172.21.15.192:0/3602540035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f100415d120 0x7f10040cfd70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.015 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.013+0000 7f100890b640 1 -- 172.21.15.192:0/3602540035 >> 172.21.15.192:0/3602540035 conn(0x7f10040d2d30 msgr2=0x7f10040d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:39.015 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.013+0000 7f100890b640 1 -- 172.21.15.192:0/3602540035 shutdown_connections 2023-12-19T18:14:39.015 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.013+0000 7f100890b640 1 -- 172.21.15.192:0/3602540035 wait complete. 2023-12-19T18:14:39.020 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.015+0000 7f100890b640 1 Processor -- start 2023-12-19T18:14:39.020 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.015+0000 7f100890b640 1 -- start start 2023-12-19T18:14:39.021 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.016+0000 7f100890b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10040ce950 0x7f1004156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.021 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.016+0000 7f100890b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10040d1310 0x7f100415fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.021 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.016+0000 7f100890b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f100415d120 0x7f10040aff70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.021 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.016+0000 7f100890b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1004156e60 con 0x7f10040ce950 2023-12-19T18:14:39.021 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.016+0000 7f100890b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10040b2ee0 con 0x7f10040d1310 2023-12-19T18:14:39.021 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.016+0000 7f100890b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10040b31c0 con 0x7f100415d120 2023-12-19T18:14:39.022 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.016+0000 7f1002ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10040d1310 0x7f100415fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.022 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.016+0000 7f1003fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f100415d120 0x7f10040aff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.022 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.016+0000 7f1002ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10040d1310 0x7f100415fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:52288/0 (socket says 172.21.15.192:52288) 2023-12-19T18:14:39.022 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.016+0000 7f1002ffd640 1 -- 172.21.15.192:0/2766402976 learned_addr learned my addr 172.21.15.192:0/2766402976 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:39.022 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.016+0000 7f10037fe640 1 --2- 172.21.15.192:0/2766402976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10040ce950 0x7f1004156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.022 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.017+0000 7f1003fff640 1 -- 172.21.15.192:0/2766402976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10040d1310 msgr2=0x7f100415fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.023 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.017+0000 7f1003fff640 1 --2- 172.21.15.192:0/2766402976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10040d1310 0x7f100415fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.023 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.017+0000 7f1003fff640 1 -- 172.21.15.192:0/2766402976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10040ce950 msgr2=0x7f1004156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.023 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.017+0000 7f1003fff640 1 --2- 172.21.15.192:0/2766402976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10040ce950 0x7f1004156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.023 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.017+0000 7f1003fff640 1 -- 172.21.15.192:0/2766402976 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff40094d0 con 0x7f100415d120 2023-12-19T18:14:39.023 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.017+0000 7f1002ffd640 1 --2- 172.21.15.192:0/2766402976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10040d1310 0x7f100415fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:39.023 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.017+0000 7f1003fff640 1 --2- 172.21.15.192:0/2766402976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f100415d120 0x7f10040aff70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc007cf0 tx=0x7f0ffc00e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.024 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.018+0000 7f0febfff640 1 -- 172.21.15.192:0/2766402976 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc018070 con 0x7f100415d120 2023-12-19T18:14:39.024 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.018+0000 7f0febfff640 1 -- 172.21.15.192:0/2766402976 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc00edd0 con 0x7f100415d120 2023-12-19T18:14:39.024 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.018+0000 7f0febfff640 1 -- 172.21.15.192:0/2766402976 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc003bb0 con 0x7f100415d120 2023-12-19T18:14:39.024 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.018+0000 7f100890b640 1 -- 172.21.15.192:0/2766402976 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10040b34a0 con 0x7f100415d120 2023-12-19T18:14:39.024 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.018+0000 7f100890b640 1 -- 172.21.15.192:0/2766402976 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1004160390 con 0x7f100415d120 2023-12-19T18:14:39.024 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.018+0000 7f0febfff640 1 -- 172.21.15.192:0/2766402976 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0ffc004320 con 0x7f100415d120 2023-12-19T18:14:39.024 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.018+0000 7f0febfff640 1 -- 172.21.15.192:0/2766402976 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f0ffc003d10 con 0x7f100415d120 2023-12-19T18:14:39.025 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.019+0000 7f100890b640 1 -- 172.21.15.192:0/2766402976 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10040b39c0 con 0x7f100415d120 2023-12-19T18:14:39.025 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:39.021+0000 7fbc5e62e040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-19T18:14:39.026 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.021+0000 7f0febfff640 1 -- 172.21.15.192:0/2766402976 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f10040b39c0 con 0x7f100415d120 2023-12-19T18:14:39.099 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.099+0000 7f100890b640 1 -- 172.21.15.192:0/2766402976 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4} v 0) v1 -- 0x7f1004160550 con 0x7f100415d120 2023-12-19T18:14:39.106 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.106+0000 7f0febfff640 1 -- 172.21.15.192:0/2766402976 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0ffc0045d0 con 0x7f100415d120 2023-12-19T18:14:39.106 INFO:teuthology.orchestra.run.smithi192.stdout:4 2023-12-19T18:14:39.107 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.107+0000 7f100890b640 1 -- 172.21.15.192:0/2766402976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f100415d120 msgr2=0x7f10040aff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.107 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.107+0000 7f100890b640 1 --2- 172.21.15.192:0/2766402976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f100415d120 0x7f10040aff70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc007cf0 tx=0x7f0ffc00e4b0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.107 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.108+0000 7f100890b640 1 -- 172.21.15.192:0/2766402976 shutdown_connections 2023-12-19T18:14:39.108 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.108+0000 7f100890b640 1 --2- 172.21.15.192:0/2766402976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f100415d120 0x7f10040aff70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.108 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.108+0000 7f100890b640 1 --2- 172.21.15.192:0/2766402976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10040d1310 0x7f100415fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.108 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.108+0000 7f100890b640 1 --2- 172.21.15.192:0/2766402976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10040ce950 0x7f1004156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.108 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.108+0000 7f100890b640 1 -- 172.21.15.192:0/2766402976 >> 172.21.15.192:0/2766402976 conn(0x7f10040d2d30 msgr2=0x7f10040b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:39.108 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.108+0000 7f100890b640 1 -- 172.21.15.192:0/2766402976 shutdown_connections 2023-12-19T18:14:39.108 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.108+0000 7f100890b640 1 -- 172.21.15.192:0/2766402976 wait complete. 2023-12-19T18:14:39.123 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 5 2023-12-19T18:14:39.132 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:39.132+0000 7fe47583b040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:14:39.188 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:39.188+0000 7fbc5e62e040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-19T18:14:39.277 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:39.277+0000 7fe47583b040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:14:39.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.312+0000 7f1a90b77640 1 Processor -- start 2023-12-19T18:14:39.311 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.312+0000 7f1a90b77640 1 -- start start 2023-12-19T18:14:39.313 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.313+0000 7f1a90b77640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1a8c0b4c80 0x7f1a8c160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.313+0000 7f1a90b77640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1a8c0b5060 0x7f1a8c0d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.313+0000 7f1a90b77640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a8c0d0d20 con 0x7f1a8c0b8920 2023-12-19T18:14:39.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.313+0000 7f1a90b77640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a8c0b6130 con 0x7f1a8c0b4c80 2023-12-19T18:14:39.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.313+0000 7f1a90b77640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a8c161bf0 con 0x7f1a8c0b5060 2023-12-19T18:14:39.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.314+0000 7f1a8ad9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1a8c0b5060 0x7f1a8c0d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.314+0000 7f1a8ad9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1a8c0b5060 0x7f1a8c0d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:52292/0 (socket says 172.21.15.192:52292) 2023-12-19T18:14:39.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.314+0000 7f1a8ad9d640 1 -- 172.21.15.192:0/534373188 learned_addr learned my addr 172.21.15.192:0/534373188 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:39.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.314+0000 7f1a8a59c640 1 --2- 172.21.15.192:0/534373188 >> v2:172.21.15.27:3300/0 conn(0x7f1a8c0b4c80 0x7f1a8c160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.314+0000 7f1a8ad9d640 1 -- 172.21.15.192:0/534373188 >> v2:172.21.15.27:3300/0 conn(0x7f1a8c0b4c80 msgr2=0x7f1a8c160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:39.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.314+0000 7f1a8ad9d640 1 --2- 172.21.15.192:0/534373188 >> v2:172.21.15.27:3300/0 conn(0x7f1a8c0b4c80 0x7f1a8c160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.314+0000 7f1a8ad9d640 1 -- 172.21.15.192:0/534373188 >> v1:172.21.15.27:6789/0 conn(0x7f1a8c0b8920 legacy=0x7f1a8c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.314 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.314+0000 7f1a8ad9d640 1 -- 172.21.15.192:0/534373188 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a8c161350 con 0x7f1a8c0b5060 2023-12-19T18:14:39.315 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.316+0000 7f1a8ad9d640 1 --2- 172.21.15.192:0/534373188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a8c0b5060 0x7f1a8c0d1a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0cf170 tx=0x7f1a8402bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=73a0e328c0c163b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.315 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.316+0000 7f1a89d9b640 1 -- 172.21.15.192:0/534373188 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8402ca80 con 0x7f1a8c0b5060 2023-12-19T18:14:39.315 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.316+0000 7f1a89d9b640 1 -- 172.21.15.192:0/534373188 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8402cbe0 con 0x7f1a8c0b5060 2023-12-19T18:14:39.315 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.316+0000 7f1a89d9b640 1 -- 172.21.15.192:0/534373188 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a840343f0 con 0x7f1a8c0b5060 2023-12-19T18:14:39.317 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.317+0000 7f1a90b77640 1 -- 172.21.15.192:0/534373188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a8c0b5060 msgr2=0x7f1a8c0d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.317 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.317+0000 7f1a90b77640 1 --2- 172.21.15.192:0/534373188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a8c0b5060 0x7f1a8c0d1a20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f1a8c0cf170 tx=0x7f1a8402bee0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.317 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.317+0000 7f1a90b77640 1 -- 172.21.15.192:0/534373188 shutdown_connections 2023-12-19T18:14:39.317 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.317+0000 7f1a90b77640 1 --2- 172.21.15.192:0/534373188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a8c0b5060 0x7f1a8c0d1a20 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.317 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.317+0000 7f1a90b77640 1 --2- 172.21.15.192:0/534373188 >> v2:172.21.15.27:3300/0 conn(0x7f1a8c0b4c80 0x7f1a8c160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.318 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.317+0000 7f1a90b77640 1 -- 172.21.15.192:0/534373188 >> 172.21.15.192:0/534373188 conn(0x7f1a8c0d2f90 msgr2=0x7f1a8c0d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:39.318 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.317+0000 7f1a90b77640 1 -- 172.21.15.192:0/534373188 shutdown_connections 2023-12-19T18:14:39.318 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.317+0000 7f1a90b77640 1 -- 172.21.15.192:0/534373188 wait complete. 2023-12-19T18:14:39.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.320+0000 7f1a90b77640 1 Processor -- start 2023-12-19T18:14:39.319 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.320+0000 7f1a90b77640 1 -- start start 2023-12-19T18:14:39.322 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a90b77640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a8c0b4c80 0x7f1a8c150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.322 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a90b77640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a8c0b5060 0x7f1a8c0c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.322 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a90b77640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a8c0b8920 0x7f1a8c155280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a90b77640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c0c8390 con 0x7f1a8c0b4c80 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a90b77640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c0c7ad0 con 0x7f1a8c0b8920 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a90b77640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a8c1f2b90 con 0x7f1a8c0b5060 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a8ad9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a8c0b8920 0x7f1a8c155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a83fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a8c0b5060 0x7f1a8c0c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a8ad9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a8c0b8920 0x7f1a8c155280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:52308/0 (socket says 172.21.15.192:52308) 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a8ad9d640 1 -- 172.21.15.192:0/194732022 learned_addr learned my addr 172.21.15.192:0/194732022 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a8a59c640 1 --2- 172.21.15.192:0/194732022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a8c0b4c80 0x7f1a8c150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a8ad9d640 1 -- 172.21.15.192:0/194732022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a8c0b5060 msgr2=0x7f1a8c0c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a8ad9d640 1 --2- 172.21.15.192:0/194732022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a8c0b5060 0x7f1a8c0c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a8ad9d640 1 -- 172.21.15.192:0/194732022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a8c0b4c80 msgr2=0x7f1a8c150350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a8ad9d640 1 --2- 172.21.15.192:0/194732022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a8c0b4c80 0x7f1a8c150350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a8ad9d640 1 -- 172.21.15.192:0/194732022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a84009b30 con 0x7f1a8c0b8920 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.322+0000 7f1a83fff640 1 --2- 172.21.15.192:0/194732022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a8c0b5060 0x7f1a8c0c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:39.323 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.323+0000 7f1a8ad9d640 1 --2- 172.21.15.192:0/194732022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a8c0b8920 0x7f1a8c155280 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1a84036040 tx=0x7f1a84004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.324 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.323+0000 7f1a82ffd640 1 -- 172.21.15.192:0/194732022 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a84009510 con 0x7f1a8c0b8920 2023-12-19T18:14:39.324 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.323+0000 7f1a82ffd640 1 -- 172.21.15.192:0/194732022 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a8403e450 con 0x7f1a8c0b8920 2023-12-19T18:14:39.324 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.323+0000 7f1a90b77640 1 -- 172.21.15.192:0/194732022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8c0c7510 con 0x7f1a8c0b8920 2023-12-19T18:14:39.324 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.323+0000 7f1a82ffd640 1 -- 172.21.15.192:0/194732022 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a8403e800 con 0x7f1a8c0b8920 2023-12-19T18:14:39.324 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.323+0000 7f1a90b77640 1 -- 172.21.15.192:0/194732022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8c0c8020 con 0x7f1a8c0b8920 2023-12-19T18:14:39.324 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.323+0000 7f1a82ffd640 1 -- 172.21.15.192:0/194732022 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1a8402cd50 con 0x7f1a8c0b8920 2023-12-19T18:14:39.324 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.324+0000 7f1a82ffd640 1 -- 172.21.15.192:0/194732022 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f1a84048400 con 0x7f1a8c0b8920 2023-12-19T18:14:39.325 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.324+0000 7f1a90b77640 1 -- 172.21.15.192:0/194732022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a8c0c8540 con 0x7f1a8c0b8920 2023-12-19T18:14:39.327 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.327+0000 7f1a82ffd640 1 -- 172.21.15.192:0/194732022 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f1a84067b40 con 0x7f1a8c0b8920 2023-12-19T18:14:39.334 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:39.334+0000 7fe47583b040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:14:39.348 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:39.349+0000 7fbc5e62e040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-19T18:14:39.403 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:39.404+0000 7fbc5e62e040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-19T18:14:39.404 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.405+0000 7f1a90b77640 1 -- 172.21.15.192:0/194732022 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5} v 0) v1 -- 0x7f1a8c068e90 con 0x7f1a8c0b8920 2023-12-19T18:14:39.410 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.411+0000 7f1a82ffd640 1 -- 172.21.15.192:0/194732022 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1a84056070 con 0x7f1a8c0b8920 2023-12-19T18:14:39.410 INFO:teuthology.orchestra.run.smithi192.stdout:5 2023-12-19T18:14:39.412 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.412+0000 7f1a90b77640 1 -- 172.21.15.192:0/194732022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a8c0b8920 msgr2=0x7f1a8c155280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.412 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.412+0000 7f1a90b77640 1 --2- 172.21.15.192:0/194732022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a8c0b8920 0x7f1a8c155280 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f1a84036040 tx=0x7f1a84004a40 comp rx=0 tx=0).stop 2023-12-19T18:14:39.412 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.412+0000 7f1a90b77640 1 -- 172.21.15.192:0/194732022 shutdown_connections 2023-12-19T18:14:39.412 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.412+0000 7f1a90b77640 1 --2- 172.21.15.192:0/194732022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a8c0b8920 0x7f1a8c155280 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.412 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.412+0000 7f1a90b77640 1 --2- 172.21.15.192:0/194732022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a8c0b5060 0x7f1a8c0c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.412 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.412+0000 7f1a90b77640 1 --2- 172.21.15.192:0/194732022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a8c0b4c80 0x7f1a8c150350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.412 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.412+0000 7f1a90b77640 1 -- 172.21.15.192:0/194732022 >> 172.21.15.192:0/194732022 conn(0x7f1a8c0d2f90 msgr2=0x7f1a8c0b6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:39.412 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.412+0000 7f1a90b77640 1 -- 172.21.15.192:0/194732022 shutdown_connections 2023-12-19T18:14:39.412 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.412+0000 7f1a90b77640 1 -- 172.21.15.192:0/194732022 wait complete. 2023-12-19T18:14:39.429 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 6 2023-12-19T18:14:39.450 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:39.450+0000 7fe47583b040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:14:39.520 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:39.521+0000 7fbc5e62e040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:14:39.569 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:39.569+0000 7fe47583b040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:14:39.613 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.614+0000 7ff59a547640 1 Processor -- start 2023-12-19T18:14:39.614 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.614+0000 7ff59a547640 1 -- start start 2023-12-19T18:14:39.615 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.615+0000 7ff59a547640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940b4c80 0x7ff594160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.615 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.615+0000 7ff59a547640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff5940b5060 0x7ff5940d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.615 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.615+0000 7ff59a547640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5940d0d20 con 0x7ff5940b8920 2023-12-19T18:14:39.615 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.615+0000 7ff59a547640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5940b6130 con 0x7ff5940b5060 2023-12-19T18:14:39.615 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.615+0000 7ff59a547640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff594161bf0 con 0x7ff5940b4c80 2023-12-19T18:14:39.616 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.616+0000 7ff598ae4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff5940b5060 0x7ff5940d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.616 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.616+0000 7ff593fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940b4c80 0x7ff594160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.616 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.616+0000 7ff58bfff640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7ff5940b8920 0x7ff59415cac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.192:43724/0 (socket says 172.21.15.192:43724) 2023-12-19T18:14:39.616 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.616+0000 7ff593fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940b4c80 0x7ff594160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54328/0 (socket says 172.21.15.192:54328) 2023-12-19T18:14:39.616 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.616+0000 7ff58bfff640 1 -- 172.21.15.192:0/3242809020 learned_addr learned my addr 172.21.15.192:0/3242809020 (peer_addr_for_me v1:172.21.15.192:0/0) 2023-12-19T18:14:39.616 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.616+0000 7ff598ae4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff5940b5060 0x7ff5940d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:52314/0 (socket says 172.21.15.192:52314) 2023-12-19T18:14:39.616 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.616+0000 7ff593fff640 1 -- 172.21.15.192:0/3242809020 >> v2:172.21.15.192:3300/0 conn(0x7ff5940b5060 msgr2=0x7ff5940d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:39.616 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.616+0000 7ff593fff640 1 --2- 172.21.15.192:0/3242809020 >> v2:172.21.15.192:3300/0 conn(0x7ff5940b5060 0x7ff5940d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.616 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.616+0000 7ff593fff640 1 -- 172.21.15.192:0/3242809020 >> v1:172.21.15.192:6789/0 conn(0x7ff5940b8920 legacy=0x7ff59415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.616 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.616+0000 7ff5937fe640 1 -- 172.21.15.192:0/3242809020 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 782922623 0 0) 0x7ff5940d0d20 con 0x7ff5940b8920 2023-12-19T18:14:39.616 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.616+0000 7ff593fff640 1 -- 172.21.15.192:0/3242809020 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff594161350 con 0x7ff5940b4c80 2023-12-19T18:14:39.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.617+0000 7ff593fff640 1 --2- 172.21.15.192:0/3242809020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940b4c80 0x7ff594160c70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff5840097a0 tx=0x7ff58402b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61886731fe3e16be server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.618+0000 7ff5937fe640 1 -- 172.21.15.192:0/3242809020 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584031780 con 0x7ff5940b4c80 2023-12-19T18:14:39.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.618+0000 7ff5937fe640 1 -- 172.21.15.192:0/3242809020 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5840318e0 con 0x7ff5940b4c80 2023-12-19T18:14:39.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.618+0000 7ff5937fe640 1 -- 172.21.15.192:0/3242809020 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff584031c70 con 0x7ff5940b4c80 2023-12-19T18:14:39.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.618+0000 7ff59a547640 1 -- 172.21.15.192:0/3242809020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940b4c80 msgr2=0x7ff594160c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.618+0000 7ff59a547640 1 --2- 172.21.15.192:0/3242809020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940b4c80 0x7ff594160c70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff5840097a0 tx=0x7ff58402b820 comp rx=0 tx=0).stop 2023-12-19T18:14:39.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.618+0000 7ff59a547640 1 -- 172.21.15.192:0/3242809020 shutdown_connections 2023-12-19T18:14:39.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.618+0000 7ff59a547640 1 --2- 172.21.15.192:0/3242809020 >> v2:172.21.15.192:3300/0 conn(0x7ff5940b5060 0x7ff5940d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.618+0000 7ff59a547640 1 --2- 172.21.15.192:0/3242809020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940b4c80 0x7ff594160c70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.618+0000 7ff59a547640 1 -- 172.21.15.192:0/3242809020 >> 172.21.15.192:0/3242809020 conn(0x7ff5940d2f90 msgr2=0x7ff5940d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:39.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.618+0000 7ff59a547640 1 -- 172.21.15.192:0/3242809020 shutdown_connections 2023-12-19T18:14:39.618 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.618+0000 7ff59a547640 1 -- 172.21.15.192:0/3242809020 wait complete. 2023-12-19T18:14:39.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.620+0000 7ff59a547640 1 Processor -- start 2023-12-19T18:14:39.619 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.620+0000 7ff59a547640 1 -- start start 2023-12-19T18:14:39.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.621+0000 7ff59a547640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5940b4c80 0x7ff5940cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.621+0000 7ff59a547640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940b5060 0x7ff5941588e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.621+0000 7ff59a547640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5940b8920 0x7ff59415a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.621+0000 7ff59a547640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff594158e20 con 0x7ff5940b4c80 2023-12-19T18:14:39.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.621+0000 7ff59a547640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff594154870 con 0x7ff5940b8920 2023-12-19T18:14:39.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.621+0000 7ff59a547640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff594159030 con 0x7ff5940b5060 2023-12-19T18:14:39.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.621+0000 7ff598ae4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5940b8920 0x7ff59415a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.621+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940b5060 0x7ff5941588e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.622+0000 7ff598ae4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5940b8920 0x7ff59415a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.192:52316/0 (socket says 172.21.15.192:52316) 2023-12-19T18:14:39.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.622+0000 7ff598ae4640 1 -- 172.21.15.192:0/2377788359 learned_addr learned my addr 172.21.15.192:0/2377788359 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:39.622 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.622+0000 7ff593fff640 1 --2- 172.21.15.192:0/2377788359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5940b4c80 0x7ff5940cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.622+0000 7ff598ae4640 1 -- 172.21.15.192:0/2377788359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940b5060 msgr2=0x7ff5941588e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.622+0000 7ff598ae4640 1 --2- 172.21.15.192:0/2377788359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940b5060 0x7ff5941588e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.622+0000 7ff598ae4640 1 -- 172.21.15.192:0/2377788359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5940b4c80 msgr2=0x7ff5940cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.622+0000 7ff598ae4640 1 --2- 172.21.15.192:0/2377788359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5940b4c80 0x7ff5940cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.622+0000 7ff598ae4640 1 -- 172.21.15.192:0/2377788359 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff584009400 con 0x7ff5940b8920 2023-12-19T18:14:39.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.622+0000 7ff598ae4640 1 --2- 172.21.15.192:0/2377788359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5940b8920 0x7ff59415a0d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00efc0 tx=0x7ff58c00c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.622+0000 7ff5917fa640 1 -- 172.21.15.192:0/2377788359 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c014480 con 0x7ff5940b8920 2023-12-19T18:14:39.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.623+0000 7ff5917fa640 1 -- 172.21.15.192:0/2377788359 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58c00f040 con 0x7ff5940b8920 2023-12-19T18:14:39.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.623+0000 7ff5917fa640 1 -- 172.21.15.192:0/2377788359 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58c014820 con 0x7ff5940b8920 2023-12-19T18:14:39.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.623+0000 7ff59a547640 1 -- 172.21.15.192:0/2377788359 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff594159310 con 0x7ff5940b8920 2023-12-19T18:14:39.623 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.623+0000 7ff59a547640 1 -- 172.21.15.192:0/2377788359 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5941598d0 con 0x7ff5940b8920 2023-12-19T18:14:39.624 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.624+0000 7ff5917fa640 1 -- 172.21.15.192:0/2377788359 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ff58c014c80 con 0x7ff5940b8920 2023-12-19T18:14:39.624 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.624+0000 7ff5917fa640 1 -- 172.21.15.192:0/2377788359 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7ff58c01dd10 con 0x7ff5940b8920 2023-12-19T18:14:39.624 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.624+0000 7ff58a7fc640 1 -- 172.21.15.192:0/2377788359 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff594068e90 con 0x7ff5940b8920 2023-12-19T18:14:39.625 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.626+0000 7ff5917fa640 1 -- 172.21.15.192:0/2377788359 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7ff58c03bb40 con 0x7ff5940b8920 2023-12-19T18:14:39.649 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:39.649+0000 7fbc5e62e040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-19T18:14:39.725 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.726+0000 7ff58a7fc640 1 -- 172.21.15.192:0/2377788359 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6} v 0) v1 -- 0x7ff5940b84f0 con 0x7ff5940b8920 2023-12-19T18:14:39.732 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.733+0000 7ff5917fa640 1 -- 172.21.15.192:0/2377788359 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ff58c02a030 con 0x7ff5940b8920 2023-12-19T18:14:39.733 INFO:teuthology.orchestra.run.smithi192.stdout:6 2023-12-19T18:14:39.734 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.734+0000 7ff58a7fc640 1 -- 172.21.15.192:0/2377788359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5940b8920 msgr2=0x7ff59415a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.734 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.734+0000 7ff58a7fc640 1 --2- 172.21.15.192:0/2377788359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5940b8920 0x7ff59415a0d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7ff58c00efc0 tx=0x7ff58c00c490 comp rx=0 tx=0).stop 2023-12-19T18:14:39.734 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.734+0000 7ff58a7fc640 1 -- 172.21.15.192:0/2377788359 shutdown_connections 2023-12-19T18:14:39.734 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.734+0000 7ff58a7fc640 1 --2- 172.21.15.192:0/2377788359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5940b8920 0x7ff59415a0d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.734 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.734+0000 7ff58a7fc640 1 --2- 172.21.15.192:0/2377788359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5940b5060 0x7ff5941588e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.734 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.734+0000 7ff58a7fc640 1 --2- 172.21.15.192:0/2377788359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5940b4c80 0x7ff5940cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.734 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.734+0000 7ff58a7fc640 1 -- 172.21.15.192:0/2377788359 >> 172.21.15.192:0/2377788359 conn(0x7ff5940d2f90 msgr2=0x7ff5940cd7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:39.734 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.735+0000 7ff58a7fc640 1 -- 172.21.15.192:0/2377788359 shutdown_connections 2023-12-19T18:14:39.734 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.735+0000 7ff58a7fc640 1 -- 172.21.15.192:0/2377788359 wait complete. 2023-12-19T18:14:39.738 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:39.738+0000 7fe47583b040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:14:39.752 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --cluster ceph osd new 00000000-0000-0000-0000-000000000000 7 2023-12-19T18:14:39.791 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:39.791+0000 7fe47583b040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:14:39.823 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:39.824+0000 7fbc5e62e040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-19T18:14:39.875 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:39.876+0000 7fbc5e62e040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-19T18:14:39.930 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.931+0000 7f98bd5d4640 1 Processor -- start 2023-12-19T18:14:39.931 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.931+0000 7f98bd5d4640 1 -- start start 2023-12-19T18:14:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.932+0000 7f98bd5d4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f98b815aed0 0x7f98b80cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98bd5d4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80d1450 0x7f98b80cd390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98bd5d4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f98b80cd8d0 0x7f98b80cb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98bd5d4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b80b3fd0 con 0x7f98b80cd8d0 2023-12-19T18:14:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98bd5d4640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b80cdd00 con 0x7f98b815aed0 2023-12-19T18:14:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98bd5d4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80cf6f0 con 0x7f98b80d1450 2023-12-19T18:14:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98b77fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80d1450 0x7f98b80cd390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98b7fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f98b815aed0 0x7f98b80cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98b77fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80d1450 0x7f98b80cd390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54360/0 (socket says 172.21.15.192:54360) 2023-12-19T18:14:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98b77fe640 1 -- 172.21.15.192:0/3474473772 learned_addr learned my addr 172.21.15.192:0/3474473772 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98bcdd3640 1 --2- 172.21.15.192:0/3474473772 >> v2:172.21.15.27:3300/0 conn(0x7f98b80cd8d0 0x7f98b80cb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98b77fe640 1 -- 172.21.15.192:0/3474473772 >> v2:172.21.15.192:3300/0 conn(0x7f98b815aed0 msgr2=0x7f98b80cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:39.933 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98b77fe640 1 --2- 172.21.15.192:0/3474473772 >> v2:172.21.15.192:3300/0 conn(0x7f98b815aed0 0x7f98b80cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.934 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98b77fe640 1 -- 172.21.15.192:0/3474473772 >> v2:172.21.15.27:3300/0 conn(0x7f98b80cd8d0 msgr2=0x7f98b80cb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:39.934 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98b77fe640 1 --2- 172.21.15.192:0/3474473772 >> v2:172.21.15.27:3300/0 conn(0x7f98b80cd8d0 0x7f98b80cb6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.934 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98b77fe640 1 -- 172.21.15.192:0/3474473772 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b80cf9d0 con 0x7f98b80d1450 2023-12-19T18:14:39.934 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.933+0000 7f98b7fff640 1 --2- 172.21.15.192:0/3474473772 >> v2:172.21.15.192:3300/0 conn(0x7f98b815aed0 0x7f98b80cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:39.935 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.934+0000 7f98b77fe640 1 --2- 172.21.15.192:0/3474473772 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80d1450 0x7f98b80cd390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f98b80ce6b0 tx=0x7f98ac02b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f37bd0dcbdaf42c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.935 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.935+0000 7f98b6ffd640 1 -- 172.21.15.192:0/3474473772 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0317d0 con 0x7f98b80d1450 2023-12-19T18:14:39.935 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.935+0000 7f98b6ffd640 1 -- 172.21.15.192:0/3474473772 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac031930 con 0x7f98b80d1450 2023-12-19T18:14:39.935 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.935+0000 7f98b6ffd640 1 -- 172.21.15.192:0/3474473772 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac031cc0 con 0x7f98b80d1450 2023-12-19T18:14:39.935 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.935+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3474473772 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80d1450 msgr2=0x7f98b80cd390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.935 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.935+0000 7f98bd5d4640 1 --2- 172.21.15.192:0/3474473772 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80d1450 0x7f98b80cd390 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f98b80ce6b0 tx=0x7f98ac02b870 comp rx=0 tx=0).stop 2023-12-19T18:14:39.935 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.935+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3474473772 shutdown_connections 2023-12-19T18:14:39.935 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.935+0000 7f98bd5d4640 1 --2- 172.21.15.192:0/3474473772 >> v2:172.21.15.27:3300/0 conn(0x7f98b80cd8d0 0x7f98b80cb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.936 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.935+0000 7f98bd5d4640 1 --2- 172.21.15.192:0/3474473772 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80d1450 0x7f98b80cd390 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.936 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.935+0000 7f98bd5d4640 1 --2- 172.21.15.192:0/3474473772 >> v2:172.21.15.192:3300/0 conn(0x7f98b815aed0 0x7f98b80cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.936 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.935+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3474473772 >> 172.21.15.192:0/3474473772 conn(0x7f98b8099ee0 msgr2=0x7f98b80d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:39.936 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.936+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3474473772 shutdown_connections 2023-12-19T18:14:39.936 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.936+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3474473772 wait complete. 2023-12-19T18:14:39.936 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.937+0000 7f98bd5d4640 1 Processor -- start 2023-12-19T18:14:39.936 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.937+0000 7f98bd5d4640 1 -- start start 2023-12-19T18:14:39.939 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.938+0000 7f98bd5d4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80cd8d0 0x7f98b8158f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.939 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.938+0000 7f98bd5d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98b80d1450 0x7f98b80c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.939 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.938+0000 7f98bd5d4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98b80c8ac0 0x7f98b815d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:39.939 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.938+0000 7f98bd5d4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b815b5f0 con 0x7f98b80c8ac0 2023-12-19T18:14:39.939 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.938+0000 7f98bd5d4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80c8f70 con 0x7f98b80d1450 2023-12-19T18:14:39.939 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.938+0000 7f98bd5d4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80c91d0 con 0x7f98b80cd8d0 2023-12-19T18:14:39.939 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.938+0000 7f98b7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80cd8d0 0x7f98b8158f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.939 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.938+0000 7f98b77fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98b80d1450 0x7f98b80c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.940 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.938+0000 7f98b7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80cd8d0 0x7f98b8158f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.192:54368/0 (socket says 172.21.15.192:54368) 2023-12-19T18:14:39.940 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.938+0000 7f98b7fff640 1 -- 172.21.15.192:0/3615152263 learned_addr learned my addr 172.21.15.192:0/3615152263 (peer_addr_for_me v2:172.21.15.192:0/0) 2023-12-19T18:14:39.940 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.939+0000 7f98bcdd3640 1 --2- 172.21.15.192:0/3615152263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98b80c8ac0 0x7f98b815d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:39.940 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.939+0000 7f98b7fff640 1 -- 172.21.15.192:0/3615152263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98b80d1450 msgr2=0x7f98b80c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.940 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.939+0000 7f98b7fff640 1 --2- 172.21.15.192:0/3615152263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98b80d1450 0x7f98b80c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.940 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.939+0000 7f98b7fff640 1 -- 172.21.15.192:0/3615152263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98b80c8ac0 msgr2=0x7f98b815d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:39.940 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.939+0000 7f98b7fff640 1 --2- 172.21.15.192:0/3615152263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98b80c8ac0 0x7f98b815d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:39.940 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.939+0000 7f98b7fff640 1 -- 172.21.15.192:0/3615152263 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a8009400 con 0x7f98b80cd8d0 2023-12-19T18:14:39.940 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.939+0000 7f98bcdd3640 1 --2- 172.21.15.192:0/3615152263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98b80c8ac0 0x7f98b815d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:39.940 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.939+0000 7f98b7fff640 1 --2- 172.21.15.192:0/3615152263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80cd8d0 0x7f98b8158f30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98a800e990 tx=0x7f98a800ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:39.940 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.939+0000 7f98b4ff9640 1 -- 172.21.15.192:0/3615152263 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a800c9e0 con 0x7f98b80cd8d0 2023-12-19T18:14:39.941 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.939+0000 7f98b4ff9640 1 -- 172.21.15.192:0/3615152263 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a800cb40 con 0x7f98b80cd8d0 2023-12-19T18:14:39.941 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.939+0000 7f98b4ff9640 1 -- 172.21.15.192:0/3615152263 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a800ced0 con 0x7f98b80cd8d0 2023-12-19T18:14:39.941 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.939+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3615152263 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac0094d0 con 0x7f98b80cd8d0 2023-12-19T18:14:39.941 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.940+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3615152263 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b80c9790 con 0x7f98b80cd8d0 2023-12-19T18:14:39.941 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.940+0000 7f98b4ff9640 1 -- 172.21.15.192:0/3615152263 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f98a8005c90 con 0x7f98b80cd8d0 2023-12-19T18:14:39.941 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.940+0000 7f98b4ff9640 1 -- 172.21.15.192:0/3615152263 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f98a8013470 con 0x7f98b80cd8d0 2023-12-19T18:14:39.941 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.941+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3615152263 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b804e840 con 0x7f98b80cd8d0 2023-12-19T18:14:39.942 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:39.943+0000 7f98b4ff9640 1 -- 172.21.15.192:0/3615152263 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f98a8021060 con 0x7f98b80cd8d0 2023-12-19T18:14:39.958 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:39.958+0000 7fe47583b040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:14:40.019 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:40.020+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3615152263 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7} v 0) v1 -- 0x7f98b80b3950 con 0x7f98b80cd8d0 2023-12-19T18:14:40.027 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:40.027+0000 7f98b4ff9640 1 -- 172.21.15.192:0/3615152263 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "00000000-0000-0000-0000-000000000000", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f98a8029020 con 0x7f98b80cd8d0 2023-12-19T18:14:40.027 INFO:teuthology.orchestra.run.smithi192.stdout:7 2023-12-19T18:14:40.028 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:40.029+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3615152263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80cd8d0 msgr2=0x7f98b8158f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:40.029 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:40.029+0000 7f98bd5d4640 1 --2- 172.21.15.192:0/3615152263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80cd8d0 0x7f98b8158f30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f98a800e990 tx=0x7f98a800ee50 comp rx=0 tx=0).stop 2023-12-19T18:14:40.029 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:40.029+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3615152263 shutdown_connections 2023-12-19T18:14:40.029 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:40.029+0000 7f98bd5d4640 1 --2- 172.21.15.192:0/3615152263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98b80c8ac0 0x7f98b815d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.029 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:40.029+0000 7f98bd5d4640 1 --2- 172.21.15.192:0/3615152263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98b80d1450 0x7f98b80c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.029 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:40.029+0000 7f98bd5d4640 1 --2- 172.21.15.192:0/3615152263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98b80cd8d0 0x7f98b8158f30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.029 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:40.029+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3615152263 >> 172.21.15.192:0/3615152263 conn(0x7f98b8099ee0 msgr2=0x7f98b806e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:40.029 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:40.029+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3615152263 shutdown_connections 2023-12-19T18:14:40.029 INFO:teuthology.orchestra.run.smithi192.stderr:2023-12-19T18:14:40.029+0000 7f98bd5d4640 1 -- 172.21.15.192:0/3615152263 wait complete. 2023-12-19T18:14:40.042 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:40.042+0000 7fbc5e62e040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-19T18:14:40.044 INFO:tasks.ceph.osd.0:Restarting daemon 2023-12-19T18:14:40.044 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-12-19T18:14:40.053 INFO:tasks.ceph.osd.0:Started 2023-12-19T18:14:40.053 INFO:tasks.ceph.osd.1:Restarting daemon 2023-12-19T18:14:40.053 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-12-19T18:14:40.061 INFO:tasks.ceph.osd.1:Started 2023-12-19T18:14:40.061 INFO:tasks.ceph.osd.2:Restarting daemon 2023-12-19T18:14:40.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-12-19T18:14:40.067 INFO:tasks.ceph.osd.2:Started 2023-12-19T18:14:40.067 INFO:tasks.ceph.osd.3:Restarting daemon 2023-12-19T18:14:40.068 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-12-19T18:14:40.073 INFO:tasks.ceph.osd.3:Started 2023-12-19T18:14:40.073 INFO:tasks.ceph.osd.4:Restarting daemon 2023-12-19T18:14:40.073 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-12-19T18:14:40.078 INFO:tasks.ceph.osd.4:Started 2023-12-19T18:14:40.078 INFO:tasks.ceph.osd.5:Restarting daemon 2023-12-19T18:14:40.079 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-12-19T18:14:40.087 INFO:tasks.ceph.osd.5:Started 2023-12-19T18:14:40.087 INFO:tasks.ceph.osd.6:Restarting daemon 2023-12-19T18:14:40.087 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-12-19T18:14:40.094 INFO:tasks.ceph.osd.6:Started 2023-12-19T18:14:40.094 INFO:tasks.ceph.osd.7:Restarting daemon 2023-12-19T18:14:40.094 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-12-19T18:14:40.100 INFO:tasks.ceph.osd.7:Started 2023-12-19T18:14:40.101 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:40.241 INFO:tasks.ceph.osd.4.smithi192.stderr:2023-12-19T18:14:40.242+0000 7f0d5a507500 -1 Falling back to public interface 2023-12-19T18:14:40.253 INFO:tasks.ceph.osd.1.smithi027.stderr:2023-12-19T18:14:40.253+0000 7fe4125ad500 -1 Falling back to public interface 2023-12-19T18:14:40.254 INFO:tasks.ceph.osd.5.smithi192.stderr:2023-12-19T18:14:40.254+0000 7fb635c14500 -1 Falling back to public interface 2023-12-19T18:14:40.259 INFO:tasks.ceph.osd.7.smithi192.stderr:2023-12-19T18:14:40.260+0000 7f7f5760d500 -1 Falling back to public interface 2023-12-19T18:14:40.261 INFO:tasks.ceph.osd.4.smithi192.stderr:2023-12-19T18:14:40.262+0000 7f0d5a507500 -1 osd.4 0 log_to_monitors true 2023-12-19T18:14:40.267 INFO:tasks.ceph.osd.0.smithi027.stderr:2023-12-19T18:14:40.267+0000 7f6ef5110500 -1 Falling back to public interface 2023-12-19T18:14:40.276 INFO:tasks.ceph.osd.5.smithi192.stderr:2023-12-19T18:14:40.276+0000 7fb635c14500 -1 osd.5 0 log_to_monitors true 2023-12-19T18:14:40.281 INFO:tasks.ceph.osd.1.smithi027.stderr:2023-12-19T18:14:40.280+0000 7fe4125ad500 -1 osd.1 0 log_to_monitors true 2023-12-19T18:14:40.290 INFO:tasks.ceph.osd.3.smithi027.stderr:2023-12-19T18:14:40.290+0000 7f5ec309e500 -1 Falling back to public interface 2023-12-19T18:14:40.292 INFO:tasks.ceph.osd.7.smithi192.stderr:2023-12-19T18:14:40.293+0000 7f7f5760d500 -1 osd.7 0 log_to_monitors true 2023-12-19T18:14:40.295 INFO:tasks.ceph.osd.2.smithi027.stderr:2023-12-19T18:14:40.295+0000 7f009f520500 -1 Falling back to public interface 2023-12-19T18:14:40.298 INFO:tasks.ceph.osd.0.smithi027.stderr:2023-12-19T18:14:40.298+0000 7f6ef5110500 -1 osd.0 0 log_to_monitors true 2023-12-19T18:14:40.310 INFO:tasks.ceph.osd.6.smithi192.stderr:2023-12-19T18:14:40.311+0000 7fa48a80f500 -1 Falling back to public interface 2023-12-19T18:14:40.317 INFO:tasks.ceph.osd.3.smithi027.stderr:2023-12-19T18:14:40.317+0000 7f5ec309e500 -1 osd.3 0 log_to_monitors true 2023-12-19T18:14:40.326 INFO:tasks.ceph.osd.2.smithi027.stderr:2023-12-19T18:14:40.326+0000 7f009f520500 -1 osd.2 0 log_to_monitors true 2023-12-19T18:14:40.341 INFO:tasks.ceph.osd.6.smithi192.stderr:2023-12-19T18:14:40.341+0000 7fa48a80f500 -1 osd.6 0 log_to_monitors true 2023-12-19T18:14:40.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.354+0000 7fb4e442c640 1 Processor -- start 2023-12-19T18:14:40.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.354+0000 7fb4e442c640 1 -- start start 2023-12-19T18:14:40.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.355+0000 7fb4e442c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb4dc15d240 0x7fb4dc1618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:40.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.355+0000 7fb4e442c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4dc0d1970 con 0x7fb4dc0ce420 2023-12-19T18:14:40.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.355+0000 7fb4e442c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4dc161e10 con 0x7fb4dc0cdf90 2023-12-19T18:14:40.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.355+0000 7fb4e442c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4dc0b5a80 con 0x7fb4dc15d240 2023-12-19T18:14:40.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.356+0000 7fb4e21c8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb4dc15d240 0x7fb4dc1618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:40.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.356+0000 7fb4e29c9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb4dc0ce420 0x7fb4dc0b5370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34412/0 (socket says 172.21.15.27:34412) 2023-12-19T18:14:40.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.356+0000 7fb4e21c8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb4dc15d240 0x7fb4dc1618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50496/0 (socket says 172.21.15.27:50496) 2023-12-19T18:14:40.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.356+0000 7fb4e29c9640 1 -- 172.21.15.27:0/3261574602 learned_addr learned my addr 172.21.15.27:0/3261574602 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:40.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.356+0000 7fb4e21c8640 1 -- 172.21.15.27:0/3261574602 >> v1:172.21.15.192:6789/0 conn(0x7fb4dc0cdf90 legacy=0x7fb4dc0b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:40.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.356+0000 7fb4e21c8640 1 -- 172.21.15.27:0/3261574602 >> v1:172.21.15.27:6789/0 conn(0x7fb4dc0ce420 legacy=0x7fb4dc0b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:40.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.356+0000 7fb4e21c8640 1 -- 172.21.15.27:0/3261574602 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc06a5e0 con 0x7fb4dc15d240 2023-12-19T18:14:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.357+0000 7fb4e21c8640 1 --2- 172.21.15.27:0/3261574602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4dc15d240 0x7fb4dc1618d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009720 tx=0x7fb4d802b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ac05584275be2e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:40.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.357+0000 7fb4e11c6640 1 -- 172.21.15.27:0/3261574602 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8031520 con 0x7fb4dc15d240 2023-12-19T18:14:40.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.357+0000 7fb4e11c6640 1 -- 172.21.15.27:0/3261574602 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d8031680 con 0x7fb4dc15d240 2023-12-19T18:14:40.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.357+0000 7fb4e11c6640 1 -- 172.21.15.27:0/3261574602 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d8031a10 con 0x7fb4dc15d240 2023-12-19T18:14:40.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.358+0000 7fb4e442c640 1 -- 172.21.15.27:0/3261574602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4dc15d240 msgr2=0x7fb4dc1618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:40.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.358+0000 7fb4e442c640 1 --2- 172.21.15.27:0/3261574602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4dc15d240 0x7fb4dc1618d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8009720 tx=0x7fb4d802b7c0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.358+0000 7fb4e442c640 1 -- 172.21.15.27:0/3261574602 shutdown_connections 2023-12-19T18:14:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.358+0000 7fb4e442c640 1 --2- 172.21.15.27:0/3261574602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4dc15d240 0x7fb4dc1618d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.358+0000 7fb4e442c640 1 -- 172.21.15.27:0/3261574602 >> 172.21.15.27:0/3261574602 conn(0x7fb4dc0d31e0 msgr2=0x7fb4dc0d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.358+0000 7fb4e442c640 1 -- 172.21.15.27:0/3261574602 shutdown_connections 2023-12-19T18:14:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.358+0000 7fb4e442c640 1 -- 172.21.15.27:0/3261574602 wait complete. 2023-12-19T18:14:40.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.360+0000 7fb4e442c640 1 Processor -- start 2023-12-19T18:14:40.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.360+0000 7fb4e442c640 1 -- start start 2023-12-19T18:14:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.361+0000 7fb4e442c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4dc0cdf90 0x7fb4dc0cc910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.361+0000 7fb4e442c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4dc0ce420 0x7fb4dc1f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.361+0000 7fb4e442c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4dc15d240 0x7fb4dc1f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.361+0000 7fb4e442c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc152120 con 0x7fb4dc0ce420 2023-12-19T18:14:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.361+0000 7fb4e442c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc152400 con 0x7fb4dc0cdf90 2023-12-19T18:14:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.361+0000 7fb4e442c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc1526e0 con 0x7fb4dc15d240 2023-12-19T18:14:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.361+0000 7fb4e19c7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4dc0ce420 0x7fb4dc1f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.361+0000 7fb4e19c7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4dc0ce420 0x7fb4dc1f5fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50502/0 (socket says 172.21.15.27:50502) 2023-12-19T18:14:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.361+0000 7fb4e19c7640 1 -- 172.21.15.27:0/3212928936 learned_addr learned my addr 172.21.15.27:0/3212928936 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.361+0000 7fb4e21c8640 1 --2- 172.21.15.27:0/3212928936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4dc0cdf90 0x7fb4dc0cc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.361+0000 7fb4e29c9640 1 --2- 172.21.15.27:0/3212928936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4dc15d240 0x7fb4dc1f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.362+0000 7fb4e19c7640 1 -- 172.21.15.27:0/3212928936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4dc15d240 msgr2=0x7fb4dc1f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.362+0000 7fb4e19c7640 1 --2- 172.21.15.27:0/3212928936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4dc15d240 0x7fb4dc1f6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.362+0000 7fb4e19c7640 1 -- 172.21.15.27:0/3212928936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4dc0cdf90 msgr2=0x7fb4dc0cc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.362+0000 7fb4e19c7640 1 --2- 172.21.15.27:0/3212928936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4dc0cdf90 0x7fb4dc0cc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.362+0000 7fb4e19c7640 1 -- 172.21.15.27:0/3212928936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d8009400 con 0x7fb4dc0ce420 2023-12-19T18:14:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.362+0000 7fb4e19c7640 1 --2- 172.21.15.27:0/3212928936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4dc0ce420 0x7fb4dc1f5fd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00c780 tx=0x7fb4cc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.362+0000 7fb4d2ffd640 1 -- 172.21.15.27:0/3212928936 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc005270 con 0x7fb4dc0ce420 2023-12-19T18:14:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.362+0000 7fb4d2ffd640 1 -- 172.21.15.27:0/3212928936 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4cc0053f0 con 0x7fb4dc0ce420 2023-12-19T18:14:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.362+0000 7fb4e442c640 1 -- 172.21.15.27:0/3212928936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4dc1529c0 con 0x7fb4dc0ce420 2023-12-19T18:14:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.362+0000 7fb4e442c640 1 -- 172.21.15.27:0/3212928936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc0cd030 con 0x7fb4dc0ce420 2023-12-19T18:14:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.362+0000 7fb4d2ffd640 1 -- 172.21.15.27:0/3212928936 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc00a880 con 0x7fb4dc0ce420 2023-12-19T18:14:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.363+0000 7fb4d2ffd640 1 -- 172.21.15.27:0/3212928936 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb4cc00ace0 con 0x7fb4dc0ce420 2023-12-19T18:14:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.363+0000 7fb4d2ffd640 1 -- 172.21.15.27:0/3212928936 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fb4cc012400 con 0x7fb4dc0ce420 2023-12-19T18:14:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.364+0000 7fb4e442c640 1 -- 172.21.15.27:0/3212928936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4dc152bb0 con 0x7fb4dc0ce420 2023-12-19T18:14:40.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.368+0000 7fb4d2ffd640 1 -- 172.21.15.27:0/3212928936 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7fb4dc0cd030 con 0x7fb4dc0ce420 2023-12-19T18:14:40.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.433+0000 7fb4e442c640 1 -- 172.21.15.27:0/3212928936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4dc068e90 con 0x7fb4dc0ce420 2023-12-19T18:14:40.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.434+0000 7fb4d2ffd640 1 -- 172.21.15.27:0/3212928936 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb4cc017070 con 0x7fb4dc0ce420 2023-12-19T18:14:40.435 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:40.436 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":10,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:40.022208+0000","last_up_change":"0.000000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:40.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.435+0000 7fb4e442c640 1 -- 172.21.15.27:0/3212928936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4dc0ce420 msgr2=0x7fb4dc1f5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.435+0000 7fb4e442c640 1 --2- 172.21.15.27:0/3212928936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4dc0ce420 0x7fb4dc1f5fd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc00c780 tx=0x7fb4cc00cc40 comp rx=0 tx=0).stop 2023-12-19T18:14:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.436+0000 7fb4e442c640 1 -- 172.21.15.27:0/3212928936 shutdown_connections 2023-12-19T18:14:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.436+0000 7fb4e442c640 1 --2- 172.21.15.27:0/3212928936 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb4dc15d240 0x7fb4dc1f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.436+0000 7fb4e442c640 1 --2- 172.21.15.27:0/3212928936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4dc0ce420 0x7fb4dc1f5fd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.436+0000 7fb4e442c640 1 --2- 172.21.15.27:0/3212928936 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb4dc0cdf90 0x7fb4dc0cc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.436+0000 7fb4e442c640 1 -- 172.21.15.27:0/3212928936 >> 172.21.15.27:0/3212928936 conn(0x7fb4dc0d31e0 msgr2=0x7fb4dc0d1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.436+0000 7fb4e442c640 1 -- 172.21.15.27:0/3212928936 shutdown_connections 2023-12-19T18:14:40.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:40.436+0000 7fb4e442c640 1 -- 172.21.15.27:0/3212928936 wait complete. 2023-12-19T18:14:40.452 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-12-19T18:14:40.452 INFO:tasks.ceph:Waiting for OSDs to come up 2023-12-19T18:14:41.183 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:41.183+0000 7fe47583b040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:14:41.189 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:41.190+0000 7fbc5e62e040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-19T18:14:41.258 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:41.446 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.445+0000 7fce6ca77640 1 Processor -- start 2023-12-19T18:14:41.446 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.446+0000 7fce6ca77640 1 -- start start 2023-12-19T18:14:41.448 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.447+0000 7fce6ca77640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce680cc790 0x7fce6806fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:41.448 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.447+0000 7fce6ca77640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fce680b49f0 0x7fce680b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:41.448 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.447+0000 7fce6ca77640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce68072020 con 0x7fce6815f2a0 2023-12-19T18:14:41.448 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.447+0000 7fce6ca77640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce6806c690 con 0x7fce680b49f0 2023-12-19T18:14:41.448 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.447+0000 7fce6ca77640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce680712c0 con 0x7fce680cc790 2023-12-19T18:14:41.448 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.447+0000 7fce677fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fce6815f2a0 0x7fce680cc080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34424/0 (socket says 172.21.15.27:34424) 2023-12-19T18:14:41.448 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.448+0000 7fce677fe640 1 -- 172.21.15.27:0/2278758063 learned_addr learned my addr 172.21.15.27:0/2278758063 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:41.449 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.448+0000 7fce67fff640 1 --2- 172.21.15.27:0/2278758063 >> v2:172.21.15.27:3300/0 conn(0x7fce680b49f0 0x7fce680b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:41.449 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.448+0000 7fce5ffff640 1 --2- 172.21.15.27:0/2278758063 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce680cc790 0x7fce6806fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:41.449 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.448+0000 7fce67fff640 1 -- 172.21.15.27:0/2278758063 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce680cc790 msgr2=0x7fce6806fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:41.449 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.448+0000 7fce67fff640 1 --2- 172.21.15.27:0/2278758063 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce680cc790 0x7fce6806fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.449 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.448+0000 7fce67fff640 1 -- 172.21.15.27:0/2278758063 >> v1:172.21.15.27:6789/0 conn(0x7fce6815f2a0 legacy=0x7fce680cc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:41.449 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.448+0000 7fce67fff640 1 -- 172.21.15.27:0/2278758063 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6815f680 con 0x7fce680b49f0 2023-12-19T18:14:41.450 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.449+0000 7fce67fff640 1 --2- 172.21.15.27:0/2278758063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680b49f0 0x7fce680b0b80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce60009d90 tx=0x7fce6002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d367830f14082d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:41.450 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.449+0000 7fce66ffd640 1 -- 172.21.15.27:0/2278758063 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce6002cc60 con 0x7fce680b49f0 2023-12-19T18:14:41.450 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.449+0000 7fce66ffd640 1 -- 172.21.15.27:0/2278758063 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce6002cdc0 con 0x7fce680b49f0 2023-12-19T18:14:41.450 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.449+0000 7fce66ffd640 1 -- 172.21.15.27:0/2278758063 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce60034660 con 0x7fce680b49f0 2023-12-19T18:14:41.451 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.450+0000 7fce6ca77640 1 -- 172.21.15.27:0/2278758063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680b49f0 msgr2=0x7fce680b0b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:41.451 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.450+0000 7fce6ca77640 1 --2- 172.21.15.27:0/2278758063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680b49f0 0x7fce680b0b80 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fce60009d90 tx=0x7fce6002beb0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.451 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.450+0000 7fce6ca77640 1 -- 172.21.15.27:0/2278758063 shutdown_connections 2023-12-19T18:14:41.451 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.450+0000 7fce6ca77640 1 --2- 172.21.15.27:0/2278758063 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680b49f0 0x7fce680b0b80 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.451 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.450+0000 7fce6ca77640 1 --2- 172.21.15.27:0/2278758063 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce680cc790 0x7fce6806fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.451 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.450+0000 7fce6ca77640 1 -- 172.21.15.27:0/2278758063 >> 172.21.15.27:0/2278758063 conn(0x7fce6806a010 msgr2=0x7fce6806a420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:41.451 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.450+0000 7fce6ca77640 1 -- 172.21.15.27:0/2278758063 shutdown_connections 2023-12-19T18:14:41.451 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.450+0000 7fce6ca77640 1 -- 172.21.15.27:0/2278758063 wait complete. 2023-12-19T18:14:41.452 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.452+0000 7fce6ca77640 1 Processor -- start 2023-12-19T18:14:41.452 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.452+0000 7fce6ca77640 1 -- start start 2023-12-19T18:14:41.454 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.453+0000 7fce6ca77640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680b49f0 0x7fce681516c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:41.454 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.453+0000 7fce6ca77640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce680cc790 0x7fce680b2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:41.455 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.453+0000 7fce6ca77640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6815f2a0 0x7fce68153240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:41.455 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.453+0000 7fce6ca77640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce68151c00 con 0x7fce680b49f0 2023-12-19T18:14:41.455 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.453+0000 7fce6ca77640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce680c56c0 con 0x7fce6815f2a0 2023-12-19T18:14:41.455 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.453+0000 7fce6ca77640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce680c4830 con 0x7fce680cc790 2023-12-19T18:14:41.455 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.453+0000 7fce677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680b49f0 0x7fce681516c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:41.455 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.453+0000 7fce677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680b49f0 0x7fce681516c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50514/0 (socket says 172.21.15.27:50514) 2023-12-19T18:14:41.455 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.453+0000 7fce677fe640 1 -- 172.21.15.27:0/4051086562 learned_addr learned my addr 172.21.15.27:0/4051086562 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:41.455 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.454+0000 7fce5ffff640 1 --2- 172.21.15.27:0/4051086562 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce680cc790 0x7fce680b2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:41.455 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.454+0000 7fce677fe640 1 -- 172.21.15.27:0/4051086562 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce680cc790 msgr2=0x7fce680b2f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:41.455 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.454+0000 7fce67fff640 1 --2- 172.21.15.27:0/4051086562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6815f2a0 0x7fce68153240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:41.455 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.454+0000 7fce677fe640 1 --2- 172.21.15.27:0/4051086562 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce680cc790 0x7fce680b2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.455 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.454+0000 7fce677fe640 1 -- 172.21.15.27:0/4051086562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6815f2a0 msgr2=0x7fce68153240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:41.455 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.454+0000 7fce677fe640 1 --2- 172.21.15.27:0/4051086562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6815f2a0 0x7fce68153240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.456 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.454+0000 7fce677fe640 1 -- 172.21.15.27:0/4051086562 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce60009a70 con 0x7fce680b49f0 2023-12-19T18:14:41.456 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.454+0000 7fce677fe640 1 --2- 172.21.15.27:0/4051086562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680b49f0 0x7fce681516c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c4b0 tx=0x7fce5800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:41.456 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.454+0000 7fce64ff9640 1 -- 172.21.15.27:0/4051086562 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58005ce0 con 0x7fce680b49f0 2023-12-19T18:14:41.456 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.454+0000 7fce64ff9640 1 -- 172.21.15.27:0/4051086562 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce58005e40 con 0x7fce680b49f0 2023-12-19T18:14:41.456 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.454+0000 7fce64ff9640 1 -- 172.21.15.27:0/4051086562 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce58007270 con 0x7fce680b49f0 2023-12-19T18:14:41.456 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.454+0000 7fce6ca77640 1 -- 172.21.15.27:0/4051086562 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce680c4ab0 con 0x7fce680b49f0 2023-12-19T18:14:41.456 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.455+0000 7fce6ca77640 1 -- 172.21.15.27:0/4051086562 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce680c5070 con 0x7fce680b49f0 2023-12-19T18:14:41.456 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.455+0000 7fce64ff9640 1 -- 172.21.15.27:0/4051086562 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fce5801c440 con 0x7fce680b49f0 2023-12-19T18:14:41.457 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.455+0000 7fce64ff9640 1 -- 172.21.15.27:0/4051086562 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(11..11 src has 1..11) v4 ==== 1883+0+0 (secure 0 0 0) 0x7fce58012400 con 0x7fce680b49f0 2023-12-19T18:14:41.457 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.456+0000 7fce6ca77640 1 -- 172.21.15.27:0/4051086562 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce6804e840 con 0x7fce680b49f0 2023-12-19T18:14:41.458 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.458+0000 7fce64ff9640 1 -- 172.21.15.27:0/4051086562 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7fce58021060 con 0x7fce680b49f0 2023-12-19T18:14:41.495 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:41.496+0000 7fbc5e62e040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:14:41.519 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:41.519+0000 7fe47583b040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-19T18:14:41.527 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.527+0000 7fce6ca77640 1 -- 172.21.15.27:0/4051086562 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce6806bf60 con 0x7fce680b49f0 2023-12-19T18:14:41.528 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.528+0000 7fce64ff9640 1 -- 172.21.15.27:0/4051086562 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fce58017070 con 0x7fce680b49f0 2023-12-19T18:14:41.529 INFO:teuthology.misc.health.smithi027.stdout: 2023-12-19T18:14:41.529 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":11,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:41.024732+0000","last_up_change":"0.000000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:41.530 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.530+0000 7fce6ca77640 1 -- 172.21.15.27:0/4051086562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680b49f0 msgr2=0x7fce681516c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:41.530 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.530+0000 7fce6ca77640 1 --2- 172.21.15.27:0/4051086562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680b49f0 0x7fce681516c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fce5800c4b0 tx=0x7fce5800c970 comp rx=0 tx=0).stop 2023-12-19T18:14:41.530 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.530+0000 7fce6ca77640 1 -- 172.21.15.27:0/4051086562 shutdown_connections 2023-12-19T18:14:41.530 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.530+0000 7fce6ca77640 1 --2- 172.21.15.27:0/4051086562 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce6815f2a0 0x7fce68153240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.530 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.530+0000 7fce6ca77640 1 --2- 172.21.15.27:0/4051086562 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce680cc790 0x7fce680b2f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.530 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.530+0000 7fce6ca77640 1 --2- 172.21.15.27:0/4051086562 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680b49f0 0x7fce681516c0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:41.530 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.530+0000 7fce6ca77640 1 -- 172.21.15.27:0/4051086562 >> 172.21.15.27:0/4051086562 conn(0x7fce6806a010 msgr2=0x7fce6815a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:41.531 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.530+0000 7fce6ca77640 1 -- 172.21.15.27:0/4051086562 shutdown_connections 2023-12-19T18:14:41.531 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:41.530+0000 7fce6ca77640 1 -- 172.21.15.27:0/4051086562 wait complete. 2023-12-19T18:14:41.544 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-19T18:14:41.555 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:41.556+0000 7fbc5e62e040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:14:41.589 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:41.588+0000 7fe47583b040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-19T18:14:41.666 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:41.667+0000 7fbc5e62e040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:14:41.713 INFO:tasks.ceph.mgr.y.smithi027.stderr:2023-12-19T18:14:41.713+0000 7fe47583b040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-19T18:14:42.068 INFO:tasks.ceph.osd.5.smithi192.stderr:2023-12-19T18:14:42.069+0000 7fb631dce640 -1 osd.5 0 waiting for initial osdmap 2023-12-19T18:14:42.069 INFO:tasks.ceph.osd.0.smithi027.stderr:2023-12-19T18:14:42.069+0000 7f6ef109b640 -1 osd.0 0 waiting for initial osdmap 2023-12-19T18:14:42.071 INFO:tasks.ceph.osd.7.smithi192.stderr:2023-12-19T18:14:42.072+0000 7f7f5306a640 -1 osd.7 0 waiting for initial osdmap 2023-12-19T18:14:42.071 INFO:tasks.ceph.osd.5.smithi192.stderr:2023-12-19T18:14:42.072+0000 7fb62abeb640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T18:14:42.072 INFO:tasks.ceph.osd.0.smithi027.stderr:2023-12-19T18:14:42.071+0000 7f6eea6cf640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T18:14:42.074 INFO:tasks.ceph.osd.3.smithi027.stderr:2023-12-19T18:14:42.074+0000 7f5ebf029640 -1 osd.3 0 waiting for initial osdmap 2023-12-19T18:14:42.075 INFO:tasks.ceph.osd.7.smithi192.stderr:2023-12-19T18:14:42.075+0000 7f7f4c69e640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T18:14:42.078 INFO:tasks.ceph.osd.3.smithi027.stderr:2023-12-19T18:14:42.078+0000 7f5eb865d640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T18:14:42.084 INFO:tasks.ceph.osd.6.smithi192.stderr:2023-12-19T18:14:42.085+0000 7fa4868d4640 -1 osd.6 0 waiting for initial osdmap 2023-12-19T18:14:42.085 INFO:tasks.ceph.osd.4.smithi192.stderr:2023-12-19T18:14:42.086+0000 7f0d56494640 -1 osd.4 0 waiting for initial osdmap 2023-12-19T18:14:42.086 INFO:tasks.ceph.osd.1.smithi027.stderr:2023-12-19T18:14:42.086+0000 7fe40e538640 -1 osd.1 0 waiting for initial osdmap 2023-12-19T18:14:42.087 INFO:tasks.ceph.osd.6.smithi192.stderr:2023-12-19T18:14:42.088+0000 7fa47f6f1640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T18:14:42.087 INFO:tasks.ceph.osd.2.smithi027.stderr:2023-12-19T18:14:42.087+0000 7f009bcc2640 -1 osd.2 0 waiting for initial osdmap 2023-12-19T18:14:42.088 INFO:tasks.ceph.osd.4.smithi192.stderr:2023-12-19T18:14:42.089+0000 7f0d4fac8640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T18:14:42.091 INFO:tasks.ceph.osd.2.smithi027.stderr:2023-12-19T18:14:42.091+0000 7f0094adf640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T18:14:42.091 INFO:tasks.ceph.osd.1.smithi027.stderr:2023-12-19T18:14:42.091+0000 7fe407b6c640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-19T18:14:42.716 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:42.717+0000 7fbbff34c640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-12-19T18:14:42.724 INFO:tasks.ceph.mgr.x.smithi192.stderr:2023-12-19T18:14:42.724+0000 7fbbff34c640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-12-19T18:14:44.181 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:14:44.191 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:14:48.349 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:48.553 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.552+0000 7fcff445b640 1 Processor -- start 2023-12-19T18:14:48.553 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.552+0000 7fcff445b640 1 -- start start 2023-12-19T18:14:48.554 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.554+0000 7fcff445b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcfec0cbcb0 0x7fcfec0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:48.555 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.554+0000 7fcff445b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcfec15baa0 0x7fcfec0cc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:48.555 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.554+0000 7fcff445b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfec0ce310 con 0x7fcfec144120 2023-12-19T18:14:48.555 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.554+0000 7fcff445b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfec070f80 con 0x7fcfec0cbcb0 2023-12-19T18:14:48.555 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.554+0000 7fcff445b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfec144500 con 0x7fcfec15baa0 2023-12-19T18:14:48.555 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.555+0000 7fcff19f6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcfec0cbcb0 0x7fcfec0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:48.555 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.555+0000 7fcff19f6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcfec0cbcb0 0x7fcfec0712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38602/0 (socket says 172.21.15.27:38602) 2023-12-19T18:14:48.555 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.555+0000 7fcff19f6640 1 -- 172.21.15.27:0/507901093 learned_addr learned my addr 172.21.15.27:0/507901093 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:48.556 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.555+0000 7fcff29f8640 1 --2- 172.21.15.27:0/507901093 >> v2:172.21.15.192:3300/0 conn(0x7fcfec15baa0 0x7fcfec0cc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:48.556 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.555+0000 7fcff19f6640 1 -- 172.21.15.27:0/507901093 >> v2:172.21.15.192:3300/0 conn(0x7fcfec15baa0 msgr2=0x7fcfec0cc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:48.556 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.555+0000 7fcff19f6640 1 --2- 172.21.15.27:0/507901093 >> v2:172.21.15.192:3300/0 conn(0x7fcfec15baa0 0x7fcfec0cc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:48.556 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.555+0000 7fcff19f6640 1 -- 172.21.15.27:0/507901093 >> v1:172.21.15.27:6789/0 conn(0x7fcfec144120 legacy=0x7fcfec15b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:48.556 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.555+0000 7fcff19f6640 1 -- 172.21.15.27:0/507901093 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0cb820 con 0x7fcfec0cbcb0 2023-12-19T18:14:48.556 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.555+0000 7fcff11f5640 1 -- 172.21.15.27:0/507901093 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2375047327 0 0) 0x7fcfec0ce310 con 0x7fcfec144120 2023-12-19T18:14:48.557 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.556+0000 7fcff19f6640 1 --2- 172.21.15.27:0/507901093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec0cbcb0 0x7fcfec0712e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0ce220 tx=0x7fcfdc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af37c828b18f5620 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:48.558 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.557+0000 7fcff11f5640 1 -- 172.21.15.27:0/507901093 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc031870 con 0x7fcfec0cbcb0 2023-12-19T18:14:48.558 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.557+0000 7fcff11f5640 1 -- 172.21.15.27:0/507901093 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfdc0319d0 con 0x7fcfec0cbcb0 2023-12-19T18:14:48.558 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.557+0000 7fcff11f5640 1 -- 172.21.15.27:0/507901093 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfdc031d60 con 0x7fcfec0cbcb0 2023-12-19T18:14:48.558 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.557+0000 7fcff445b640 1 -- 172.21.15.27:0/507901093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec0cbcb0 msgr2=0x7fcfec0712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:48.558 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.557+0000 7fcff445b640 1 --2- 172.21.15.27:0/507901093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec0cbcb0 0x7fcfec0712e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fcfec0ce220 tx=0x7fcfdc02b910 comp rx=0 tx=0).stop 2023-12-19T18:14:48.558 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.557+0000 7fcff445b640 1 -- 172.21.15.27:0/507901093 shutdown_connections 2023-12-19T18:14:48.558 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.557+0000 7fcff445b640 1 --2- 172.21.15.27:0/507901093 >> v2:172.21.15.192:3300/0 conn(0x7fcfec15baa0 0x7fcfec0cc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:48.558 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.557+0000 7fcff445b640 1 --2- 172.21.15.27:0/507901093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec0cbcb0 0x7fcfec0712e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:48.558 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.557+0000 7fcff445b640 1 -- 172.21.15.27:0/507901093 >> 172.21.15.27:0/507901093 conn(0x7fcfec06a2e0 msgr2=0x7fcfec0b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:48.558 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.557+0000 7fcff445b640 1 -- 172.21.15.27:0/507901093 shutdown_connections 2023-12-19T18:14:48.558 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.557+0000 7fcff445b640 1 -- 172.21.15.27:0/507901093 wait complete. 2023-12-19T18:14:48.559 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.559+0000 7fcff445b640 1 Processor -- start 2023-12-19T18:14:48.559 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.560+0000 7fcff445b640 1 -- start start 2023-12-19T18:14:48.562 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.560+0000 7fcff445b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfec144120 0x7fcfec15a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:48.562 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff445b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec15baa0 0x7fcfec0c5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:48.562 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff445b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec0c6100 0x7fcfec15d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:48.563 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff445b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec0c65b0 con 0x7fcfec0c6100 2023-12-19T18:14:48.563 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff445b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec15e3b0 con 0x7fcfec144120 2023-12-19T18:14:48.563 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff445b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec15e0d0 con 0x7fcfec15baa0 2023-12-19T18:14:48.563 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff29f8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec0c6100 0x7fcfec15d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:48.563 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff29f8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec0c6100 0x7fcfec15d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38614/0 (socket says 172.21.15.27:38614) 2023-12-19T18:14:48.563 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff29f8640 1 -- 172.21.15.27:0/1227719581 learned_addr learned my addr 172.21.15.27:0/1227719581 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:48.563 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff21f7640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfec144120 0x7fcfec15a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:48.563 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff19f6640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec15baa0 0x7fcfec0c5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:48.563 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff29f8640 1 -- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec15baa0 msgr2=0x7fcfec0c5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:48.563 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff29f8640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec15baa0 0x7fcfec0c5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:48.563 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff29f8640 1 -- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfec144120 msgr2=0x7fcfec15a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:48.563 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff29f8640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfec144120 0x7fcfec15a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:48.563 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff29f8640 1 -- 172.21.15.27:0/1227719581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfdc0094d0 con 0x7fcfec0c6100 2023-12-19T18:14:48.564 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.561+0000 7fcff19f6640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec15baa0 0x7fcfec0c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:48.564 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.562+0000 7fcff29f8640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec0c6100 0x7fcfec15d310 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400d600 tx=0x7fcfe400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:48.564 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.562+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/1227719581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe40125f0 con 0x7fcfec0c6100 2023-12-19T18:14:48.564 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.562+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/1227719581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe4012770 con 0x7fcfec0c6100 2023-12-19T18:14:48.564 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.562+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/1227719581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4012b20 con 0x7fcfec0c6100 2023-12-19T18:14:48.564 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.562+0000 7fcff445b640 1 -- 172.21.15.27:0/1227719581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfec1f1560 con 0x7fcfec0c6100 2023-12-19T18:14:48.564 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.562+0000 7fcff445b640 1 -- 172.21.15.27:0/1227719581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfec1f1b20 con 0x7fcfec0c6100 2023-12-19T18:14:48.565 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.564+0000 7fcff445b640 1 -- 172.21.15.27:0/1227719581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfec068f00 con 0x7fcfec0c6100 2023-12-19T18:14:48.565 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.564+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/1227719581 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcfe4012c80 con 0x7fcfec0c6100 2023-12-19T18:14:48.565 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.565+0000 7fcfe2ffd640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcfd006a430 0x7fcfd006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:48.565 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.566+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/1227719581 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcfe4088a60 con 0x7fcfec0c6100 2023-12-19T18:14:48.566 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.566+0000 7fcff21f7640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcfd006a430 0x7fcfd006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:48.567 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.567+0000 7fcff21f7640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcfd006a430 0x7fcfd006c8f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8007fd0 tx=0x7fcfe8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:48.571 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.571+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/1227719581 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcfe40543b0 con 0x7fcfec0c6100 2023-12-19T18:14:48.765 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.764+0000 7fcff445b640 1 -- 172.21.15.27:0/1227719581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfec15f520 con 0x7fcfec0c6100 2023-12-19T18:14:48.766 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.766+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/1227719581 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12382 (secure 0 0 0) 0x7fcfe40579d0 con 0x7fcfec0c6100 2023-12-19T18:14:48.766 INFO:teuthology.misc.health.smithi027.stdout: 2023-12-19T18:14:48.766 INFO:teuthology.misc.health.smithi027.stdout:{"epoch":16,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:46.096679+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:48.769 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.768+0000 7fcff445b640 1 -- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcfd006a430 msgr2=0x7fcfd006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:48.769 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.768+0000 7fcff445b640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcfd006a430 0x7fcfd006c8f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8007fd0 tx=0x7fcfe8009040 comp rx=0 tx=0).stop 2023-12-19T18:14:48.769 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.768+0000 7fcff445b640 1 -- 172.21.15.27:0/1227719581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec0c6100 msgr2=0x7fcfec15d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:48.769 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.768+0000 7fcff445b640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec0c6100 0x7fcfec15d310 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcfe400d600 tx=0x7fcfe400dac0 comp rx=0 tx=0).stop 2023-12-19T18:14:48.769 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.769+0000 7fcff445b640 1 -- 172.21.15.27:0/1227719581 shutdown_connections 2023-12-19T18:14:48.769 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.769+0000 7fcff445b640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcfd006a430 0x7fcfd006c8f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:48.769 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.769+0000 7fcff445b640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec0c6100 0x7fcfec15d310 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:48.769 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.769+0000 7fcff445b640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec15baa0 0x7fcfec0c5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:48.769 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.769+0000 7fcff445b640 1 --2- 172.21.15.27:0/1227719581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfec144120 0x7fcfec15a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:48.770 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.769+0000 7fcff445b640 1 -- 172.21.15.27:0/1227719581 >> 172.21.15.27:0/1227719581 conn(0x7fcfec06a2e0 msgr2=0x7fcfec06ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:48.770 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.769+0000 7fcff445b640 1 -- 172.21.15.27:0/1227719581 shutdown_connections 2023-12-19T18:14:48.770 INFO:teuthology.misc.health.smithi027.stderr:2023-12-19T18:14:48.769+0000 7fcff445b640 1 -- 172.21.15.27:0/1227719581 wait complete. 2023-12-19T18:14:48.784 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-12-19T18:14:48.784 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-12-19T18:14:48.784 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:14:48.784 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:14:48.794 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:14:48.794 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:14:48.794 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:14:48.802 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:14:48.803 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:14:48.803 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:14:48.812 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:14:48.812 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:14:48.812 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:14:48.821 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:14:48.822 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-12-19T18:14:48.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:48.952 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:14:48.956 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:14:48.974 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-19T18:14:48.977 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-19T18:14:49.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.010+0000 7f9aa75a9640 1 Processor -- start 2023-12-19T18:14:49.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.010+0000 7f9aa75a9640 1 -- start start 2023-12-19T18:14:49.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.012+0000 7f9aa75a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa00cf9b0 0x7f9aa00cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.012+0000 7f9aa75a9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9aa00cda20 0x7f9aa00cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.012+0000 7f9aa75a9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9aa00d0350 con 0x7f9aa00b8c30 2023-12-19T18:14:49.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.012+0000 7f9aa75a9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aa00cea80 con 0x7f9aa00cda20 2023-12-19T18:14:49.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.012+0000 7f9aa75a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa00b5f10 con 0x7f9aa00cf9b0 2023-12-19T18:14:49.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.012+0000 7f9aa5345640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f9aa00b8c30 0x7f9aa00d1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:43052/0 (socket says 172.21.15.27:43052) 2023-12-19T18:14:49.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.012+0000 7f9aa5345640 1 -- 172.21.15.27:0/109257415 learned_addr learned my addr 172.21.15.27:0/109257415 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:49.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.012+0000 7f9aa4b44640 1 --2- 172.21.15.27:0/109257415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa00cf9b0 0x7f9aa00cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.012+0000 7f9aa5b46640 1 --2- 172.21.15.27:0/109257415 >> v2:172.21.15.192:3300/0 conn(0x7f9aa00cda20 0x7f9aa00cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.013+0000 7f9a97fff640 1 -- 172.21.15.27:0/109257415 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 846365150 0 0) 0x7f9aa00d0350 con 0x7f9aa00b8c30 2023-12-19T18:14:49.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.013+0000 7f9a97fff640 1 -- 172.21.15.27:0/109257415 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a8c003520 con 0x7f9aa00b8c30 2023-12-19T18:14:49.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.013+0000 7f9aa5b46640 1 -- 172.21.15.27:0/109257415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa00cf9b0 msgr2=0x7f9aa00cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:49.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.013+0000 7f9aa5b46640 1 --2- 172.21.15.27:0/109257415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa00cf9b0 0x7f9aa00cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.013+0000 7f9aa5b46640 1 -- 172.21.15.27:0/109257415 >> v1:172.21.15.192:6789/0 conn(0x7f9aa00b8c30 legacy=0x7f9aa00d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.013+0000 7f9aa5b46640 1 -- 172.21.15.27:0/109257415 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa00d26d0 con 0x7f9aa00cda20 2023-12-19T18:14:49.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.014+0000 7f9aa5b46640 1 --2- 172.21.15.27:0/109257415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9aa00cda20 0x7f9aa00cde70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009d90 tx=0x7f9a9802be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e6ad6273679e7d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:49.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.014+0000 7f9a97fff640 1 -- 172.21.15.27:0/109257415 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9802cbc0 con 0x7f9aa00cda20 2023-12-19T18:14:49.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.014+0000 7f9a97fff640 1 -- 172.21.15.27:0/109257415 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9802cd20 con 0x7f9aa00cda20 2023-12-19T18:14:49.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.014+0000 7f9a97fff640 1 -- 172.21.15.27:0/109257415 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98034660 con 0x7f9aa00cda20 2023-12-19T18:14:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.015+0000 7f9aa75a9640 1 -- 172.21.15.27:0/109257415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9aa00cda20 msgr2=0x7f9aa00cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.015+0000 7f9aa75a9640 1 --2- 172.21.15.27:0/109257415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9aa00cda20 0x7f9aa00cde70 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9a98009d90 tx=0x7f9a9802be10 comp rx=0 tx=0).stop 2023-12-19T18:14:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.015+0000 7f9aa75a9640 1 -- 172.21.15.27:0/109257415 shutdown_connections 2023-12-19T18:14:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.015+0000 7f9aa75a9640 1 --2- 172.21.15.27:0/109257415 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9aa00cda20 0x7f9aa00cde70 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.015+0000 7f9aa75a9640 1 --2- 172.21.15.27:0/109257415 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa00cf9b0 0x7f9aa00cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.015+0000 7f9aa75a9640 1 -- 172.21.15.27:0/109257415 >> 172.21.15.27:0/109257415 conn(0x7f9aa00d2fa0 msgr2=0x7f9aa00d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.015+0000 7f9aa75a9640 1 -- 172.21.15.27:0/109257415 shutdown_connections 2023-12-19T18:14:49.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.015+0000 7f9aa75a9640 1 -- 172.21.15.27:0/109257415 wait complete. 2023-12-19T18:14:49.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.017+0000 7f9aa75a9640 1 Processor -- start 2023-12-19T18:14:49.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.017+0000 7f9aa75a9640 1 -- start start 2023-12-19T18:14:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa75a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9aa00b8c30 0x7f9aa014de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa75a9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9aa00cda20 0x7f9aa01508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa75a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa00cf9b0 0x7f9aa014d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa75a9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa014f100 con 0x7f9aa00cda20 2023-12-19T18:14:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa75a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa0150e30 con 0x7f9aa00b8c30 2023-12-19T18:14:49.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa75a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aa014e4f0 con 0x7f9aa00cf9b0 2023-12-19T18:14:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa4b44640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9aa00cda20 0x7f9aa01508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa4b44640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9aa00cda20 0x7f9aa01508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38646/0 (socket says 172.21.15.27:38646) 2023-12-19T18:14:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa4b44640 1 -- 172.21.15.27:0/3666223251 learned_addr learned my addr 172.21.15.27:0/3666223251 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa5345640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9aa00b8c30 0x7f9aa014de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa5b46640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa00cf9b0 0x7f9aa014d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa4b44640 1 -- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa00cf9b0 msgr2=0x7f9aa014d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa4b44640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa00cf9b0 0x7f9aa014d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa4b44640 1 -- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9aa00b8c30 msgr2=0x7f9aa014de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa4b44640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9aa00b8c30 0x7f9aa014de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa4b44640 1 -- 172.21.15.27:0/3666223251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a98009a70 con 0x7f9aa00cda20 2023-12-19T18:14:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.018+0000 7f9aa5b46640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa00cf9b0 0x7f9aa014d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.019+0000 7f9aa4b44640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9aa00cda20 0x7f9aa01508f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000e870 tx=0x7f9a9000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:49.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.019+0000 7f9a95ffb640 1 -- 172.21.15.27:0/3666223251 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a900094f0 con 0x7f9aa00cda20 2023-12-19T18:14:49.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.019+0000 7f9a95ffb640 1 -- 172.21.15.27:0/3666223251 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9000c940 con 0x7f9aa00cda20 2023-12-19T18:14:49.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.019+0000 7f9aa75a9640 1 -- 172.21.15.27:0/3666223251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aa014e7d0 con 0x7f9aa00cda20 2023-12-19T18:14:49.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.019+0000 7f9a95ffb640 1 -- 172.21.15.27:0/3666223251 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9000ccf0 con 0x7f9aa00cda20 2023-12-19T18:14:49.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.019+0000 7f9aa75a9640 1 -- 172.21.15.27:0/3666223251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aa014ed90 con 0x7f9aa00cda20 2023-12-19T18:14:49.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.020+0000 7f9aa75a9640 1 -- 172.21.15.27:0/3666223251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aa00d14c0 con 0x7f9aa00cda20 2023-12-19T18:14:49.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.021+0000 7f9a95ffb640 1 -- 172.21.15.27:0/3666223251 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9a90013470 con 0x7f9aa00cda20 2023-12-19T18:14:49.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.022+0000 7f9a95ffb640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a8406a460 0x7f9a8406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.022+0000 7f9a95ffb640 1 -- 172.21.15.27:0/3666223251 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9a90088680 con 0x7f9aa00cda20 2023-12-19T18:14:49.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.022+0000 7f9aa5345640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a8406a460 0x7f9a8406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.023+0000 7f9aa5345640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a8406a460 0x7f9a8406c920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0011e0 tx=0x7f9a9c007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:49.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.027+0000 7f9a95ffb640 1 -- 172.21.15.27:0/3666223251 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9aa00d14c0 con 0x7f9aa00cda20 2023-12-19T18:14:49.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.219+0000 7f9aa75a9640 1 -- 172.21.15.27:0/3666223251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9aa00d14c0 con 0x7f9aa00cda20 2023-12-19T18:14:49.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.221+0000 7f9a95ffb640 1 -- 172.21.15.27:0/3666223251 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12382 (secure 0 0 0) 0x7f9aa00d14c0 con 0x7f9aa00cda20 2023-12-19T18:14:49.222 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:49.222 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:46.096679+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:49.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.224+0000 7f9aa75a9640 1 -- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a8406a460 msgr2=0x7f9a8406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.224+0000 7f9aa75a9640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a8406a460 0x7f9a8406c920 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c0011e0 tx=0x7f9a9c007d30 comp rx=0 tx=0).stop 2023-12-19T18:14:49.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.224+0000 7f9aa75a9640 1 -- 172.21.15.27:0/3666223251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9aa00cda20 msgr2=0x7f9aa01508f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.224+0000 7f9aa75a9640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9aa00cda20 0x7f9aa01508f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9a9000e870 tx=0x7f9a9000ed30 comp rx=0 tx=0).stop 2023-12-19T18:14:49.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.224+0000 7f9aa75a9640 1 -- 172.21.15.27:0/3666223251 shutdown_connections 2023-12-19T18:14:49.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.224+0000 7f9aa75a9640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a8406a460 0x7f9a8406c920 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.224+0000 7f9aa75a9640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9aa00cf9b0 0x7f9aa014d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.224+0000 7f9aa75a9640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9aa00cda20 0x7f9aa01508f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.224+0000 7f9aa75a9640 1 --2- 172.21.15.27:0/3666223251 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9aa00b8c30 0x7f9aa014de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.224+0000 7f9aa75a9640 1 -- 172.21.15.27:0/3666223251 >> 172.21.15.27:0/3666223251 conn(0x7f9aa00d2fa0 msgr2=0x7f9aa00ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:49.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.225+0000 7f9aa75a9640 1 -- 172.21.15.27:0/3666223251 shutdown_connections 2023-12-19T18:14:49.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.225+0000 7f9aa75a9640 1 -- 172.21.15.27:0/3666223251 wait complete. 2023-12-19T18:14:49.241 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:14:49.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:14:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.464+0000 7f8d01716640 1 Processor -- start 2023-12-19T18:14:49.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.464+0000 7f8d01716640 1 -- start start 2023-12-19T18:14:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.465+0000 7f8d01716640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8cfc0b8c30 0x7f8cfc0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.466+0000 7f8d01716640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8cfc0cf9b0 0x7f8cfc0cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.466+0000 7f8d01716640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cfc0ce540 con 0x7f8cfc0cda20 2023-12-19T18:14:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.466+0000 7f8d01716640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc0cea80 con 0x7f8cfc0cf9b0 2023-12-19T18:14:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.466+0000 7f8d01716640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cfc0b5f10 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.466+0000 7f8cfb7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8cfc0cda20 0x7f8cfc0cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42176/0 (socket says 172.21.15.27:42176) 2023-12-19T18:14:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.466+0000 7f8cfb7fe640 1 -- 172.21.15.27:0/2751853283 learned_addr learned my addr 172.21.15.27:0/2751853283 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:49.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.466+0000 7f8cfa7fc640 1 --2- 172.21.15.27:0/2751853283 >> v2:172.21.15.27:3300/0 conn(0x7f8cfc0cf9b0 0x7f8cfc0cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.466+0000 7f8cfaffd640 1 --2- 172.21.15.27:0/2751853283 >> v2:172.21.15.192:3300/0 conn(0x7f8cfc0b8c30 0x7f8cfc0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.466+0000 7f8cf9ffb640 1 -- 172.21.15.27:0/2751853283 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2667992489 0 0) 0x7f8cfc0ce540 con 0x7f8cfc0cda20 2023-12-19T18:14:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.466+0000 7f8cf9ffb640 1 -- 172.21.15.27:0/2751853283 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ce4003520 con 0x7f8cfc0cda20 2023-12-19T18:14:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.466+0000 7f8cfa7fc640 1 -- 172.21.15.27:0/2751853283 >> v2:172.21.15.192:3300/0 conn(0x7f8cfc0b8c30 msgr2=0x7f8cfc0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.466+0000 7f8cfa7fc640 1 --2- 172.21.15.27:0/2751853283 >> v2:172.21.15.192:3300/0 conn(0x7f8cfc0b8c30 0x7f8cfc0d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.467+0000 7f8cfa7fc640 1 -- 172.21.15.27:0/2751853283 >> v1:172.21.15.27:6789/0 conn(0x7f8cfc0cda20 legacy=0x7f8cfc0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.467+0000 7f8cfa7fc640 1 -- 172.21.15.27:0/2751853283 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cfc0ce7a0 con 0x7f8cfc0cf9b0 2023-12-19T18:14:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.468+0000 7f8cfa7fc640 1 --2- 172.21.15.27:0/2751853283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cfc0cf9b0 0x7f8cfc0cfe40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc073c20 tx=0x7f8cf002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17bc855d54bdbf87 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.468+0000 7f8cf9ffb640 1 -- 172.21.15.27:0/2751853283 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf002e070 con 0x7f8cfc0cf9b0 2023-12-19T18:14:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.468+0000 7f8cf9ffb640 1 -- 172.21.15.27:0/2751853283 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cf00315a0 con 0x7f8cfc0cf9b0 2023-12-19T18:14:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.468+0000 7f8cf9ffb640 1 -- 172.21.15.27:0/2751853283 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cf0031930 con 0x7f8cfc0cf9b0 2023-12-19T18:14:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.468+0000 7f8d01716640 1 -- 172.21.15.27:0/2751853283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cfc0cf9b0 msgr2=0x7f8cfc0cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.468+0000 7f8d01716640 1 --2- 172.21.15.27:0/2751853283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cfc0cf9b0 0x7f8cfc0cfe40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc073c20 tx=0x7f8cf002b840 comp rx=0 tx=0).stop 2023-12-19T18:14:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.469+0000 7f8d01716640 1 -- 172.21.15.27:0/2751853283 shutdown_connections 2023-12-19T18:14:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.469+0000 7f8d01716640 1 --2- 172.21.15.27:0/2751853283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cfc0cf9b0 0x7f8cfc0cfe40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.469+0000 7f8d01716640 1 --2- 172.21.15.27:0/2751853283 >> v2:172.21.15.192:3300/0 conn(0x7f8cfc0b8c30 0x7f8cfc0d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.469+0000 7f8d01716640 1 -- 172.21.15.27:0/2751853283 >> 172.21.15.27:0/2751853283 conn(0x7f8cfc0d2fa0 msgr2=0x7f8cfc0d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.469+0000 7f8d01716640 1 -- 172.21.15.27:0/2751853283 shutdown_connections 2023-12-19T18:14:49.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.469+0000 7f8d01716640 1 -- 172.21.15.27:0/2751853283 wait complete. 2023-12-19T18:14:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.470+0000 7f8d01716640 1 Processor -- start 2023-12-19T18:14:49.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.471+0000 7f8d01716640 1 -- start start 2023-12-19T18:14:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.471+0000 7f8d01716640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cfc0b8c30 0x7f8cfc154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8d01716640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cfc0cda20 0x7f8cfc0cbf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8d01716640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cfc0cf9b0 0x7f8cfc15a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8d01716640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc158cd0 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8d01716640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc159b30 con 0x7f8cfc0cda20 2023-12-19T18:14:49.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8d01716640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cfc158f20 con 0x7f8cfc0cf9b0 2023-12-19T18:14:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8cfaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cfc0b8c30 0x7f8cfc154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8cfaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cfc0b8c30 0x7f8cfc154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38668/0 (socket says 172.21.15.27:38668) 2023-12-19T18:14:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8cfaffd640 1 -- 172.21.15.27:0/1327461940 learned_addr learned my addr 172.21.15.27:0/1327461940 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8cfb7fe640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cfc0cf9b0 0x7f8cfc15a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8cfa7fc640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cfc0cda20 0x7f8cfc0cbf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8cfaffd640 1 -- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cfc0cf9b0 msgr2=0x7f8cfc15a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8cfaffd640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cfc0cf9b0 0x7f8cfc15a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8cfaffd640 1 -- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cfc0cda20 msgr2=0x7f8cfc0cbf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8cfaffd640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cfc0cda20 0x7f8cfc0cbf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.472+0000 7f8cfaffd640 1 -- 172.21.15.27:0/1327461940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf0009400 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.473+0000 7f8cfaffd640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cfc0b8c30 0x7f8cfc154b70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00d800 tx=0x7f8cec00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.473+0000 7f8ce37fe640 1 -- 172.21.15.27:0/1327461940 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec00b6c0 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.473+0000 7f8ce37fe640 1 -- 172.21.15.27:0/1327461940 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cec00b820 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.473+0000 7f8ce37fe640 1 -- 172.21.15.27:0/1327461940 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cec00bbb0 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.473+0000 7f8d01716640 1 -- 172.21.15.27:0/1327461940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cfc159200 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.473+0000 7f8d01716640 1 -- 172.21.15.27:0/1327461940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cfc1597c0 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.474+0000 7f8d01716640 1 -- 172.21.15.27:0/1327461940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cfc068e90 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.475+0000 7f8ce37fe640 1 -- 172.21.15.27:0/1327461940 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8cec00bd10 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.476+0000 7f8ce37fe640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ce806a480 0x7f8ce806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.476+0000 7f8ce37fe640 1 -- 172.21.15.27:0/1327461940 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8cec088b30 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.477+0000 7f8cfa7fc640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ce806a480 0x7f8ce806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.478+0000 7f8cfa7fc640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ce806a480 0x7f8ce806c940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0036000 tx=0x7f8cf003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.481+0000 7f8ce37fe640 1 -- 172.21.15.27:0/1327461940 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8cec054450 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.669+0000 7f8d01716640 1 -- 172.21.15.27:0/1327461940 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8cfc0cd360 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.670+0000 7f8ce37fe640 1 -- 172.21.15.27:0/1327461940 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8cec057a70 con 0x7f8cfc0b8c30 2023-12-19T18:14:49.670 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:14:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.673+0000 7f8d01716640 1 -- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ce806a480 msgr2=0x7f8ce806c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.673+0000 7f8d01716640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ce806a480 0x7f8ce806c940 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0036000 tx=0x7f8cf003d040 comp rx=0 tx=0).stop 2023-12-19T18:14:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.673+0000 7f8d01716640 1 -- 172.21.15.27:0/1327461940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cfc0b8c30 msgr2=0x7f8cfc154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.673+0000 7f8d01716640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cfc0b8c30 0x7f8cfc154b70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8cec00d800 tx=0x7f8cec00dcc0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.673+0000 7f8d01716640 1 -- 172.21.15.27:0/1327461940 shutdown_connections 2023-12-19T18:14:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.673+0000 7f8d01716640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ce806a480 0x7f8ce806c940 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.673+0000 7f8d01716640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cfc0cf9b0 0x7f8cfc15a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.673+0000 7f8d01716640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cfc0cda20 0x7f8cfc0cbf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.673+0000 7f8d01716640 1 --2- 172.21.15.27:0/1327461940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cfc0b8c30 0x7f8cfc154b70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.673+0000 7f8d01716640 1 -- 172.21.15.27:0/1327461940 >> 172.21.15.27:0/1327461940 conn(0x7f8cfc0d2fa0 msgr2=0x7f8cfc0d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.673+0000 7f8d01716640 1 -- 172.21.15.27:0/1327461940 shutdown_connections 2023-12-19T18:14:49.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.673+0000 7f8d01716640 1 -- 172.21.15.27:0/1327461940 wait complete. 2023-12-19T18:14:49.689 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:14:49.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.912+0000 7f94ea5e0640 1 Processor -- start 2023-12-19T18:14:49.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.913+0000 7f94ea5e0640 1 -- start start 2023-12-19T18:14:49.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.914+0000 7f94ea5e0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f94e40ae800 0x7f94e4070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.914+0000 7f94ea5e0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e4071a30 con 0x7f94e4071520 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.914+0000 7f94ea5e0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94e4071d10 con 0x7f94e406ccb0 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.914+0000 7f94ea5e0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94e40cceb0 con 0x7f94e40ae800 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.914+0000 7f94e8ddd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f94e4071520 0x7f94e406c5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42182/0 (socket says 172.21.15.27:42182) 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.914+0000 7f94e8ddd640 1 -- 172.21.15.27:0/2512003217 learned_addr learned my addr 172.21.15.27:0/2512003217 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94e95de640 1 --2- 172.21.15.27:0/2512003217 >> v2:172.21.15.192:3300/0 conn(0x7f94e40ae800 0x7f94e4070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864943029 0 0) 0x7f94e4071a30 con 0x7f94e4071520 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94d0003520 con 0x7f94e4071520 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2842135011 0 0) 0x7f94d0003520 con 0x7f94e4071520 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 >> v2:172.21.15.192:3300/0 conn(0x7f94e40ae800 msgr2=0x7f94e4070fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 --2- 172.21.15.27:0/2512003217 >> v2:172.21.15.192:3300/0 conn(0x7f94e40ae800 0x7f94e4070fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 >> v1:172.21.15.192:6789/0 conn(0x7f94e406ccb0 legacy=0x7f94e40cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e40cd190 con 0x7f94e4071520 2023-12-19T18:14:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f94d4001670 con 0x7f94e4071520 2023-12-19T18:14:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 >> v1:172.21.15.27:6789/0 conn(0x7f94e4071520 legacy=0x7f94e406c5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 --2- 172.21.15.27:0/2512003217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94d0003970 0x7f94d0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 --2- 172.21.15.27:0/2512003217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d00047c0 0x7f94d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 --2- 172.21.15.27:0/2512003217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94d0005180 0x7f94d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40cceb0 con 0x7f94d00047c0 2023-12-19T18:14:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d000c030 con 0x7f94d0003970 2023-12-19T18:14:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.915+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94d000c1f0 con 0x7f94d0005180 2023-12-19T18:14:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.916+0000 7f94e8ddd640 1 --2- 172.21.15.27:0/2512003217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d00047c0 0x7f94d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.916+0000 7f94e95de640 1 --2- 172.21.15.27:0/2512003217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94d0003970 0x7f94d0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.916+0000 7f94e9ddf640 1 --2- 172.21.15.27:0/2512003217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94d0005180 0x7f94d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.916+0000 7f94e8ddd640 1 -- 172.21.15.27:0/2512003217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94d0005180 msgr2=0x7f94d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:49.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.916+0000 7f94e8ddd640 1 --2- 172.21.15.27:0/2512003217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94d0005180 0x7f94d000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.916+0000 7f94e8ddd640 1 -- 172.21.15.27:0/2512003217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94d0003970 msgr2=0x7f94d0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:49.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.916+0000 7f94e8ddd640 1 --2- 172.21.15.27:0/2512003217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94d0003970 0x7f94d0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.916+0000 7f94e9ddf640 1 -- 172.21.15.27:0/2512003217 reap_dead start 2023-12-19T18:14:49.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.916+0000 7f94e8ddd640 1 -- 172.21.15.27:0/2512003217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94d0003520 con 0x7f94d00047c0 2023-12-19T18:14:49.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.916+0000 7f94e95de640 1 --2- 172.21.15.27:0/2512003217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94d0003970 0x7f94d0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:49.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.917+0000 7f94e8ddd640 1 --2- 172.21.15.27:0/2512003217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d00047c0 0x7f94d0004c40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94d4007ca0 tx=0x7f94d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f598de723808b2a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:49.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.918+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d402ece0 con 0x7f94d00047c0 2023-12-19T18:14:49.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.918+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94d402ee40 con 0x7f94d00047c0 2023-12-19T18:14:49.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.918+0000 7f94dbfff640 1 -- 172.21.15.27:0/2512003217 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94d4005af0 con 0x7f94d00047c0 2023-12-19T18:14:49.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.918+0000 7f94ea5e0640 1 -- 172.21.15.27:0/2512003217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d00047c0 msgr2=0x7f94d0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.918+0000 7f94ea5e0640 1 --2- 172.21.15.27:0/2512003217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d00047c0 0x7f94d0004c40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f94d4007ca0 tx=0x7f94d00057f0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.918+0000 7f94ea5e0640 1 -- 172.21.15.27:0/2512003217 shutdown_connections 2023-12-19T18:14:49.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.918+0000 7f94ea5e0640 1 --2- 172.21.15.27:0/2512003217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94d00047c0 0x7f94d0004c40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.918+0000 7f94ea5e0640 1 -- 172.21.15.27:0/2512003217 >> 172.21.15.27:0/2512003217 conn(0x7f94e40cf240 msgr2=0x7f94e40cf650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:49.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.918+0000 7f94ea5e0640 1 -- 172.21.15.27:0/2512003217 shutdown_connections 2023-12-19T18:14:49.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.918+0000 7f94ea5e0640 1 -- 172.21.15.27:0/2512003217 wait complete. 2023-12-19T18:14:49.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.920+0000 7f94ea5e0640 1 Processor -- start 2023-12-19T18:14:49.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.920+0000 7f94ea5e0640 1 -- start start 2023-12-19T18:14:49.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.921+0000 7f94ea5e0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94d00047c0 0x7f94e415c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.921+0000 7f94ea5e0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94e415c9c0 0x7f94e41eded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.921+0000 7f94ea5e0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94e41ee410 0x7f94e41ee890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.921+0000 7f94ea5e0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e41ef0d0 con 0x7f94e41ee410 2023-12-19T18:14:49.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.921+0000 7f94ea5e0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e40b2eb0 con 0x7f94e415c9c0 2023-12-19T18:14:49.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.921+0000 7f94ea5e0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e41eedd0 con 0x7f94d00047c0 2023-12-19T18:14:49.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.921+0000 7f94e9ddf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94e41ee410 0x7f94e41ee890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.922+0000 7f94e9ddf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94e41ee410 0x7f94e41ee890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38690/0 (socket says 172.21.15.27:38690) 2023-12-19T18:14:49.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.922+0000 7f94e9ddf640 1 -- 172.21.15.27:0/3393435941 learned_addr learned my addr 172.21.15.27:0/3393435941 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:49.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.922+0000 7f94e95de640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94d00047c0 0x7f94e415c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.922+0000 7f94e8ddd640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94e415c9c0 0x7f94e41eded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.922+0000 7f94e9ddf640 1 -- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94d00047c0 msgr2=0x7f94e415c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.922+0000 7f94e9ddf640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94d00047c0 0x7f94e415c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.922+0000 7f94e9ddf640 1 -- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94e415c9c0 msgr2=0x7f94e41eded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:49.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.922+0000 7f94e9ddf640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94e415c9c0 0x7f94e41eded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:49.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.922+0000 7f94e9ddf640 1 -- 172.21.15.27:0/3393435941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e415cf60 con 0x7f94e41ee410 2023-12-19T18:14:49.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.922+0000 7f94e9ddf640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94e41ee410 0x7f94e41ee890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0011e0 tx=0x7f94dc00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:49.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.923+0000 7f94d9ffb640 1 -- 172.21.15.27:0/3393435941 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0039c0 con 0x7f94e41ee410 2023-12-19T18:14:49.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.923+0000 7f94d9ffb640 1 -- 172.21.15.27:0/3393435941 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94dc003b20 con 0x7f94e41ee410 2023-12-19T18:14:49.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.923+0000 7f94ea5e0640 1 -- 172.21.15.27:0/3393435941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94e415d240 con 0x7f94e41ee410 2023-12-19T18:14:49.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.923+0000 7f94d9ffb640 1 -- 172.21.15.27:0/3393435941 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0050d0 con 0x7f94e41ee410 2023-12-19T18:14:49.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.923+0000 7f94ea5e0640 1 -- 172.21.15.27:0/3393435941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94e40b33e0 con 0x7f94e41ee410 2023-12-19T18:14:49.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.924+0000 7f94ea5e0640 1 -- 172.21.15.27:0/3393435941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94e404e840 con 0x7f94e41ee410 2023-12-19T18:14:49.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.925+0000 7f94d9ffb640 1 -- 172.21.15.27:0/3393435941 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f94dc005230 con 0x7f94e41ee410 2023-12-19T18:14:49.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.926+0000 7f94d9ffb640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f94c806a430 0x7f94c806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:49.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.926+0000 7f94d9ffb640 1 -- 172.21.15.27:0/3393435941 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f94dc088340 con 0x7f94e41ee410 2023-12-19T18:14:49.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.926+0000 7f94e95de640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f94c806a430 0x7f94c806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:49.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.927+0000 7f94e95de640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f94c806a430 0x7f94c806c8f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f94e404f8f0 tx=0x7f94e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:49.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:49.930+0000 7f94d9ffb640 1 -- 172.21.15.27:0/3393435941 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f94dc053d10 con 0x7f94e41ee410 2023-12-19T18:14:50.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.169+0000 7f94ea5e0640 1 -- 172.21.15.27:0/3393435941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f94e40ad9b0 con 0x7f94e41ee410 2023-12-19T18:14:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.170+0000 7f94d9ffb640 1 -- 172.21.15.27:0/3393435941 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3527 (secure 0 0 0) 0x7f94dc057330 con 0x7f94e41ee410 2023-12-19T18:14:50.170 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:50.171 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3679393226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3679393226},{"type":"v1","addr":"172.21.15.27:6835","nonce":3679393226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/809712436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":809712436},{"type":"v1","addr":"172.21.15.192:6835","nonce":809712436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3400689212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3400689212},{"type":"v1","addr":"172.21.15.27:6833","nonce":3400689212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4267,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1878426383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1878426383},{"type":"v1","addr":"172.21.15.192:6837","nonce":1878426383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-12-19T18:14:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 3 2023-12-19T18:14:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.173+0000 7f94ea5e0640 1 -- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f94c806a430 msgr2=0x7f94c806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.173+0000 7f94ea5e0640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f94c806a430 0x7f94c806c8f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f94e404f8f0 tx=0x7f94e0008040 comp rx=0 tx=0).stop 2023-12-19T18:14:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.173+0000 7f94ea5e0640 1 -- 172.21.15.27:0/3393435941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94e41ee410 msgr2=0x7f94e41ee890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.173+0000 7f94ea5e0640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94e41ee410 0x7f94e41ee890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f94dc0011e0 tx=0x7f94dc00ec10 comp rx=0 tx=0).stop 2023-12-19T18:14:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.173+0000 7f94ea5e0640 1 -- 172.21.15.27:0/3393435941 shutdown_connections 2023-12-19T18:14:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.173+0000 7f94ea5e0640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f94c806a430 0x7f94c806c8f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.173+0000 7f94ea5e0640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94e41ee410 0x7f94e41ee890 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.173+0000 7f94ea5e0640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94e415c9c0 0x7f94e41eded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.173+0000 7f94ea5e0640 1 --2- 172.21.15.27:0/3393435941 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94d00047c0 0x7f94e415c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.173+0000 7f94ea5e0640 1 -- 172.21.15.27:0/3393435941 >> 172.21.15.27:0/3393435941 conn(0x7f94e40cf240 msgr2=0x7f94e406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.174+0000 7f94ea5e0640 1 -- 172.21.15.27:0/3393435941 shutdown_connections 2023-12-19T18:14:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.174+0000 7f94ea5e0640 1 -- 172.21.15.27:0/3393435941 wait complete. 2023-12-19T18:14:50.195 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:50.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.425+0000 7fe2c2767640 1 Processor -- start 2023-12-19T18:14:50.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.425+0000 7fe2c2767640 1 -- start start 2023-12-19T18:14:50.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.427+0000 7fe2c2767640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe2bc15c7f0 0x7fe2bc0c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.427+0000 7fe2c2767640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2bc0cc510 0x7fe2bc0cc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.427+0000 7fe2c2767640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2bc0ca310 con 0x7fe2bc0cab50 2023-12-19T18:14:50.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.427+0000 7fe2c2767640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2bc0ccee0 con 0x7fe2bc15c7f0 2023-12-19T18:14:50.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.427+0000 7fe2c2767640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc15cbd0 con 0x7fe2bc0cc510 2023-12-19T18:14:50.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.427+0000 7fe2c0f64640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe2bc0cab50 0x7fe2bc0c9a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42186/0 (socket says 172.21.15.27:42186) 2023-12-19T18:14:50.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.427+0000 7fe2c0f64640 1 -- 172.21.15.27:0/3239263980 learned_addr learned my addr 172.21.15.27:0/3239263980 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2c1765640 1 --2- 172.21.15.27:0/3239263980 >> v2:172.21.15.192:3300/0 conn(0x7fe2bc15c7f0 0x7fe2bc0c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2c1f66640 1 --2- 172.21.15.27:0/3239263980 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2bc0cc510 0x7fe2bc0cc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4201581597 0 0) 0x7fe2bc0ca310 con 0x7fe2bc0cab50 2023-12-19T18:14:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2a8003520 con 0x7fe2bc0cab50 2023-12-19T18:14:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4044730779 0 0) 0x7fe2a8003520 con 0x7fe2bc0cab50 2023-12-19T18:14:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2bc0cc510 msgr2=0x7fe2bc0cc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 --2- 172.21.15.27:0/3239263980 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2bc0cc510 0x7fe2bc0cc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 >> v2:172.21.15.192:3300/0 conn(0x7fe2bc15c7f0 msgr2=0x7fe2bc0c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 --2- 172.21.15.27:0/3239263980 >> v2:172.21.15.192:3300/0 conn(0x7fe2bc15c7f0 0x7fe2bc0c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc0ae490 con 0x7fe2bc0cab50 2023-12-19T18:14:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe2ac001670 con 0x7fe2bc0cab50 2023-12-19T18:14:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 >> v1:172.21.15.27:6789/0 conn(0x7fe2bc0cab50 legacy=0x7fe2bc0c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 --2- 172.21.15.27:0/3239263980 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2bc0cc510 0x7fe2a8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 --2- 172.21.15.27:0/3239263980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2a8003ef0 0x7fe2a80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 --2- 172.21.15.27:0/3239263980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a8004d00 0x7fe2a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0ccee0 con 0x7fe2a8004d00 2023-12-19T18:14:50.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc15cbd0 con 0x7fe2a8003ef0 2023-12-19T18:14:50.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.428+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2a800ba60 con 0x7fe2bc0cc510 2023-12-19T18:14:50.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.429+0000 7fe2c1f66640 1 --2- 172.21.15.27:0/3239263980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a8004d00 0x7fe2a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:50.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.429+0000 7fe2c1f66640 1 -- 172.21.15.27:0/3239263980 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2bc0cc510 msgr2=0x7fe2a8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:50.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.429+0000 7fe2c1f66640 1 --2- 172.21.15.27:0/3239263980 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2bc0cc510 0x7fe2a8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.429+0000 7fe2c1f66640 1 -- 172.21.15.27:0/3239263980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2a8003ef0 msgr2=0x7fe2a80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:50.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.429+0000 7fe2c1f66640 1 --2- 172.21.15.27:0/3239263980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2a8003ef0 0x7fe2a80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.429+0000 7fe2c1f66640 1 -- 172.21.15.27:0/3239263980 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a8003520 con 0x7fe2a8004d00 2023-12-19T18:14:50.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.430+0000 7fe2c1f66640 1 --2- 172.21.15.27:0/3239263980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a8004d00 0x7fe2a8005180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009fc0 tx=0x7fe2a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ae70cf20273f18 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:50.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.430+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b402cc30 con 0x7fe2a8004d00 2023-12-19T18:14:50.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.431+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b400f040 con 0x7fe2a8004d00 2023-12-19T18:14:50.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.431+0000 7fe2b3fff640 1 -- 172.21.15.27:0/3239263980 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4034440 con 0x7fe2a8004d00 2023-12-19T18:14:50.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.431+0000 7fe2c2767640 1 -- 172.21.15.27:0/3239263980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a8004d00 msgr2=0x7fe2a8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:50.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.431+0000 7fe2c2767640 1 --2- 172.21.15.27:0/3239263980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2a8004d00 0x7fe2a8005180 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe2b4009fc0 tx=0x7fe2a80057f0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.431+0000 7fe2c1f66640 1 -- 172.21.15.27:0/3239263980 reap_dead start 2023-12-19T18:14:50.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.431+0000 7fe2c2767640 1 -- 172.21.15.27:0/3239263980 shutdown_connections 2023-12-19T18:14:50.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.431+0000 7fe2c2767640 1 -- 172.21.15.27:0/3239263980 >> 172.21.15.27:0/3239263980 conn(0x7fe2bc069c70 msgr2=0x7fe2bc06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:50.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.431+0000 7fe2c2767640 1 -- 172.21.15.27:0/3239263980 shutdown_connections 2023-12-19T18:14:50.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.431+0000 7fe2c2767640 1 -- 172.21.15.27:0/3239263980 wait complete. 2023-12-19T18:14:50.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.433+0000 7fe2c2767640 1 Processor -- start 2023-12-19T18:14:50.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.433+0000 7fe2c2767640 1 -- start start 2023-12-19T18:14:50.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.434+0000 7fe2c2767640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2a8004d00 0x7fe2bc0b3cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.434+0000 7fe2c2767640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2bc0b2e70 0x7fe2bc1f39e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.434+0000 7fe2c2767640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2bc1f3f20 0x7fe2bc150f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.434+0000 7fe2c2767640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc150a90 con 0x7fe2bc0b2e70 2023-12-19T18:14:50.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.434+0000 7fe2c2767640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0b4bc0 con 0x7fe2bc1f3f20 2023-12-19T18:14:50.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.434+0000 7fe2c2767640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc0b4600 con 0x7fe2a8004d00 2023-12-19T18:14:50.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.434+0000 7fe2c0f64640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2bc0b2e70 0x7fe2bc1f39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:50.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.435+0000 7fe2c0f64640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2bc0b2e70 0x7fe2bc1f39e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38702/0 (socket says 172.21.15.27:38702) 2023-12-19T18:14:50.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.435+0000 7fe2c0f64640 1 -- 172.21.15.27:0/34036987 learned_addr learned my addr 172.21.15.27:0/34036987 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:50.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.435+0000 7fe2c1f66640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2bc1f3f20 0x7fe2bc150f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:50.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.435+0000 7fe2c1765640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2a8004d00 0x7fe2bc0b3cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:50.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.435+0000 7fe2c0f64640 1 -- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2a8004d00 msgr2=0x7fe2bc0b3cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:50.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.435+0000 7fe2c0f64640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2a8004d00 0x7fe2bc0b3cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.435+0000 7fe2c0f64640 1 -- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2bc1f3f20 msgr2=0x7fe2bc150f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:50.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.435+0000 7fe2c0f64640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2bc1f3f20 0x7fe2bc150f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.435+0000 7fe2c0f64640 1 -- 172.21.15.27:0/34036987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2bc0b48e0 con 0x7fe2bc0b2e70 2023-12-19T18:14:50.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.435+0000 7fe2c1f66640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2bc1f3f20 0x7fe2bc150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:50.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.435+0000 7fe2c0f64640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2bc0b2e70 0x7fe2bc1f39e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac000f60 tx=0x7fe2ac00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:50.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.436+0000 7fe2b1ffb640 1 -- 172.21.15.27:0/34036987 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac0074e0 con 0x7fe2bc0b2e70 2023-12-19T18:14:50.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.436+0000 7fe2b1ffb640 1 -- 172.21.15.27:0/34036987 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ac007640 con 0x7fe2bc0b2e70 2023-12-19T18:14:50.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.436+0000 7fe2b1ffb640 1 -- 172.21.15.27:0/34036987 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac004a20 con 0x7fe2bc0b2e70 2023-12-19T18:14:50.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.436+0000 7fe2c2767640 1 -- 172.21.15.27:0/34036987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc150720 con 0x7fe2bc0b2e70 2023-12-19T18:14:50.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.436+0000 7fe2c2767640 1 -- 172.21.15.27:0/34036987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc1f43a0 con 0x7fe2bc0b2e70 2023-12-19T18:14:50.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.437+0000 7fe2c2767640 1 -- 172.21.15.27:0/34036987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc150c40 con 0x7fe2bc0b2e70 2023-12-19T18:14:50.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.438+0000 7fe2b1ffb640 1 -- 172.21.15.27:0/34036987 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe2ac004b80 con 0x7fe2bc0b2e70 2023-12-19T18:14:50.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.439+0000 7fe2b1ffb640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2a006a430 0x7fe2a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.439+0000 7fe2b1ffb640 1 -- 172.21.15.27:0/34036987 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe2ac087b40 con 0x7fe2bc0b2e70 2023-12-19T18:14:50.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.439+0000 7fe2c1765640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2a006a430 0x7fe2a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:50.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.440+0000 7fe2c1765640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2a006a430 0x7fe2a006c8f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8004770 tx=0x7fe2b8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:50.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.443+0000 7fe2b1ffb640 1 -- 172.21.15.27:0/34036987 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe2bc150c40 con 0x7fe2bc0b2e70 2023-12-19T18:14:50.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.644+0000 7fe2c2767640 1 -- 172.21.15.27:0/34036987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe2bc150c40 con 0x7fe2bc0b2e70 2023-12-19T18:14:50.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.646+0000 7fe2b1ffb640 1 -- 172.21.15.27:0/34036987 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12382 (secure 0 0 0) 0x7fe2bc150c40 con 0x7fe2bc0b2e70 2023-12-19T18:14:50.646 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:50.646 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":16,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:46.096679+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:50.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.648+0000 7fe2c2767640 1 -- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2a006a430 msgr2=0x7fe2a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:50.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.648+0000 7fe2c2767640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2a006a430 0x7fe2a006c8f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8004770 tx=0x7fe2b8008040 comp rx=0 tx=0).stop 2023-12-19T18:14:50.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.648+0000 7fe2c2767640 1 -- 172.21.15.27:0/34036987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2bc0b2e70 msgr2=0x7fe2bc1f39e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:50.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.648+0000 7fe2c2767640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2bc0b2e70 0x7fe2bc1f39e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac000f60 tx=0x7fe2ac00cc20 comp rx=0 tx=0).stop 2023-12-19T18:14:50.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.649+0000 7fe2c2767640 1 -- 172.21.15.27:0/34036987 shutdown_connections 2023-12-19T18:14:50.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.649+0000 7fe2c2767640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2a006a430 0x7fe2a006c8f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.649+0000 7fe2c2767640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2bc1f3f20 0x7fe2bc150f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.649+0000 7fe2c2767640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2bc0b2e70 0x7fe2bc1f39e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.649+0000 7fe2c2767640 1 --2- 172.21.15.27:0/34036987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2a8004d00 0x7fe2bc0b3cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.649+0000 7fe2c2767640 1 -- 172.21.15.27:0/34036987 >> 172.21.15.27:0/34036987 conn(0x7fe2bc069c70 msgr2=0x7fe2bc0ce920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:50.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.649+0000 7fe2c2767640 1 -- 172.21.15.27:0/34036987 shutdown_connections 2023-12-19T18:14:50.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.649+0000 7fe2c2767640 1 -- 172.21.15.27:0/34036987 wait complete. 2023-12-19T18:14:50.669 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:14:50.669 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:14:50.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.900+0000 7f8e33fff640 1 Processor -- start 2023-12-19T18:14:50.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.901+0000 7f8e33fff640 1 -- start start 2023-12-19T18:14:50.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.902+0000 7f8e33fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8e3415e9d0 0x7f8e340cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.902+0000 7f8e33fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3415b3e0 con 0x7f8e34151c90 2023-12-19T18:14:50.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.902+0000 7f8e33fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e3415eed0 con 0x7f8e34152070 2023-12-19T18:14:50.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.902+0000 7f8e33fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e3415f1b0 con 0x7f8e3415e9d0 2023-12-19T18:14:50.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.903+0000 7f8e2bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8e3415e9d0 0x7f8e340cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:50.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.903+0000 7f8e32ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8e34151c90 0x7f8e3406b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42190/0 (socket says 172.21.15.27:42190) 2023-12-19T18:14:50.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.903+0000 7f8e32ffd640 1 -- 172.21.15.27:0/548922945 learned_addr learned my addr 172.21.15.27:0/548922945 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:50.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.903+0000 7f8e2bfff640 1 -- 172.21.15.27:0/548922945 >> v1:172.21.15.192:6789/0 conn(0x7f8e34152070 legacy=0x7f8e3415b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:50.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.903+0000 7f8e2bfff640 1 -- 172.21.15.27:0/548922945 >> v1:172.21.15.27:6789/0 conn(0x7f8e34151c90 legacy=0x7f8e3406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:50.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.903+0000 7f8e2bfff640 1 -- 172.21.15.27:0/548922945 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e3415f490 con 0x7f8e3415e9d0 2023-12-19T18:14:50.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.904+0000 7f8e2bfff640 1 --2- 172.21.15.27:0/548922945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e3415e9d0 0x7f8e340cee40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0097d0 tx=0x7f8e1c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a147c51e58b384a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.905+0000 7f8e327fc640 1 -- 172.21.15.27:0/548922945 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c031780 con 0x7f8e3415e9d0 2023-12-19T18:14:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.905+0000 7f8e327fc640 1 -- 172.21.15.27:0/548922945 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1c0318e0 con 0x7f8e3415e9d0 2023-12-19T18:14:50.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.905+0000 7f8e327fc640 1 -- 172.21.15.27:0/548922945 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c031c70 con 0x7f8e3415e9d0 2023-12-19T18:14:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.905+0000 7f8e33fff640 1 -- 172.21.15.27:0/548922945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e3415e9d0 msgr2=0x7f8e340cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.905+0000 7f8e33fff640 1 --2- 172.21.15.27:0/548922945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e3415e9d0 0x7f8e340cee40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c0097d0 tx=0x7f8e1c02b820 comp rx=0 tx=0).stop 2023-12-19T18:14:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.905+0000 7f8e33fff640 1 -- 172.21.15.27:0/548922945 shutdown_connections 2023-12-19T18:14:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.905+0000 7f8e33fff640 1 --2- 172.21.15.27:0/548922945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e3415e9d0 0x7f8e340cee40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.905+0000 7f8e33fff640 1 -- 172.21.15.27:0/548922945 >> 172.21.15.27:0/548922945 conn(0x7f8e3406a060 msgr2=0x7f8e3406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.905+0000 7f8e33fff640 1 -- 172.21.15.27:0/548922945 shutdown_connections 2023-12-19T18:14:50.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.906+0000 7f8e33fff640 1 -- 172.21.15.27:0/548922945 wait complete. 2023-12-19T18:14:50.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.907+0000 7f8e33fff640 1 Processor -- start 2023-12-19T18:14:50.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.908+0000 7f8e33fff640 1 -- start start 2023-12-19T18:14:50.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e33fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e34151c90 0x7f8e3414f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e33fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e34152070 0x7f8e34156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e33fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e3415e9d0 0x7f8e3415d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e33fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e34157820 con 0x7f8e3415e9d0 2023-12-19T18:14:50.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e33fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e34157b00 con 0x7f8e34152070 2023-12-19T18:14:50.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e33fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e34157de0 con 0x7f8e34151c90 2023-12-19T18:14:50.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e3415e9d0 0x7f8e3415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:50.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e3415e9d0 0x7f8e3415d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38708/0 (socket says 172.21.15.27:38708) 2023-12-19T18:14:50.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e337fe640 1 -- 172.21.15.27:0/2493053031 learned_addr learned my addr 172.21.15.27:0/2493053031 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:50.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e32ffd640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e34151c90 0x7f8e3414f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:50.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e2bfff640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e34152070 0x7f8e34156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:50.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e337fe640 1 -- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e34151c90 msgr2=0x7f8e3414f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:50.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e337fe640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e34151c90 0x7f8e3414f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.909+0000 7f8e337fe640 1 -- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e34152070 msgr2=0x7f8e34156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:50.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.910+0000 7f8e337fe640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e34152070 0x7f8e34156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:50.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.910+0000 7f8e337fe640 1 -- 172.21.15.27:0/2493053031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c009400 con 0x7f8e3415e9d0 2023-12-19T18:14:50.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.910+0000 7f8e337fe640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e3415e9d0 0x7f8e3415d020 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00c780 tx=0x7f8e2c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:50.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.910+0000 7f8e2b7fe640 1 -- 172.21.15.27:0/2493053031 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c004b10 con 0x7f8e3415e9d0 2023-12-19T18:14:50.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.910+0000 7f8e2b7fe640 1 -- 172.21.15.27:0/2493053031 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c004c70 con 0x7f8e3415e9d0 2023-12-19T18:14:50.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.910+0000 7f8e2b7fe640 1 -- 172.21.15.27:0/2493053031 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c00aa90 con 0x7f8e3415e9d0 2023-12-19T18:14:50.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.910+0000 7f8e33fff640 1 -- 172.21.15.27:0/2493053031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e341580c0 con 0x7f8e3415e9d0 2023-12-19T18:14:50.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.910+0000 7f8e33fff640 1 -- 172.21.15.27:0/2493053031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e3414f790 con 0x7f8e3415e9d0 2023-12-19T18:14:50.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.911+0000 7f8e33fff640 1 -- 172.21.15.27:0/2493053031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e341585e0 con 0x7f8e3415e9d0 2023-12-19T18:14:50.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.913+0000 7f8e2b7fe640 1 -- 172.21.15.27:0/2493053031 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8e2c00abf0 con 0x7f8e3415e9d0 2023-12-19T18:14:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.914+0000 7f8e2b7fe640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8e1406a430 0x7f8e1406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.914+0000 7f8e2b7fe640 1 -- 172.21.15.27:0/2493053031 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8e2c088620 con 0x7f8e3415e9d0 2023-12-19T18:14:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.915+0000 7f8e32ffd640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8e1406a430 0x7f8e1406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.916+0000 7f8e32ffd640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8e1406a430 0x7f8e1406c8f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8e24005880 tx=0x7f8e24009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:50.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:50.919+0000 7f8e2b7fe640 1 -- 172.21.15.27:0/2493053031 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8e341585e0 con 0x7f8e3415e9d0 2023-12-19T18:14:51.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.110+0000 7f8e33fff640 1 -- 172.21.15.27:0/2493053031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8e341585e0 con 0x7f8e3415e9d0 2023-12-19T18:14:51.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.111+0000 7f8e2b7fe640 1 -- 172.21.15.27:0/2493053031 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8e341585e0 con 0x7f8e3415e9d0 2023-12-19T18:14:51.111 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:14:51.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.113+0000 7f8e33fff640 1 -- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8e1406a430 msgr2=0x7f8e1406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.113+0000 7f8e33fff640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8e1406a430 0x7f8e1406c8f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8e24005880 tx=0x7f8e24009040 comp rx=0 tx=0).stop 2023-12-19T18:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.113+0000 7f8e33fff640 1 -- 172.21.15.27:0/2493053031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e3415e9d0 msgr2=0x7f8e3415d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.113+0000 7f8e33fff640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e3415e9d0 0x7f8e3415d020 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c00c780 tx=0x7f8e2c00cc40 comp rx=0 tx=0).stop 2023-12-19T18:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.113+0000 7f8e33fff640 1 -- 172.21.15.27:0/2493053031 shutdown_connections 2023-12-19T18:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.113+0000 7f8e33fff640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8e1406a430 0x7f8e1406c8f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.113+0000 7f8e33fff640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e3415e9d0 0x7f8e3415d020 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.113+0000 7f8e33fff640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e34152070 0x7f8e34156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.113+0000 7f8e33fff640 1 --2- 172.21.15.27:0/2493053031 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e34151c90 0x7f8e3414f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.113+0000 7f8e33fff640 1 -- 172.21.15.27:0/2493053031 >> 172.21.15.27:0/2493053031 conn(0x7f8e3406a060 msgr2=0x7f8e3406a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.114+0000 7f8e33fff640 1 -- 172.21.15.27:0/2493053031 shutdown_connections 2023-12-19T18:14:51.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.114+0000 7f8e33fff640 1 -- 172.21.15.27:0/2493053031 wait complete. 2023-12-19T18:14:51.134 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:14:51.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.362+0000 7f2265099640 1 Processor -- start 2023-12-19T18:14:51.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.362+0000 7f2265099640 1 -- start start 2023-12-19T18:14:51.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.364+0000 7f2265099640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f22601593d0 0x7f226006bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.364+0000 7f2265099640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2260070750 0x7f22600c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.364+0000 7f2265099640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2260070da0 con 0x7f226006c0e0 2023-12-19T18:14:51.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.364+0000 7f2265099640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22600c9940 con 0x7f22601593d0 2023-12-19T18:14:51.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.364+0000 7f2265099640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226006c570 con 0x7f2260070750 2023-12-19T18:14:51.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.364+0000 7f2257fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f226006c0e0 0x7f2260070040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42202/0 (socket says 172.21.15.27:42202) 2023-12-19T18:14:51.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.364+0000 7f2257fff640 1 -- 172.21.15.27:0/1445669093 learned_addr learned my addr 172.21.15.27:0/1445669093 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:51.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.364+0000 7f2264898640 1 --2- 172.21.15.27:0/1445669093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2260070750 0x7f22600c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225ffff640 1 --2- 172.21.15.27:0/1445669093 >> v2:172.21.15.192:3300/0 conn(0x7f22601593d0 0x7f226006bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 452807135 0 0) 0x7f2260070da0 con 0x7f226006c0e0 2023-12-19T18:14:51.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f224c003520 con 0x7f226006c0e0 2023-12-19T18:14:51.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1643945169 0 0) 0x7f224c003520 con 0x7f226006c0e0 2023-12-19T18:14:51.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2260070750 msgr2=0x7f22600c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:51.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 --2- 172.21.15.27:0/1445669093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2260070750 0x7f22600c9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 >> v2:172.21.15.192:3300/0 conn(0x7f22601593d0 msgr2=0x7f226006bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:51.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 --2- 172.21.15.27:0/1445669093 >> v2:172.21.15.192:3300/0 conn(0x7f22601593d0 0x7f226006bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22600c8fa0 con 0x7f226006c0e0 2023-12-19T18:14:51.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f2248001670 con 0x7f226006c0e0 2023-12-19T18:14:51.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 >> v1:172.21.15.27:6789/0 conn(0x7f226006c0e0 legacy=0x7f2260070040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 --2- 172.21.15.27:0/1445669093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f224c003970 0x7f224c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 --2- 172.21.15.27:0/1445669093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2260070750 0x7f224c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 --2- 172.21.15.27:0/1445669093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f224c004d00 0x7f224c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22600c9940 con 0x7f224c003970 2023-12-19T18:14:51.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226006c570 con 0x7f224c004d00 2023-12-19T18:14:51.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.365+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f224c00b980 con 0x7f2260070750 2023-12-19T18:14:51.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.366+0000 7f225ffff640 1 --2- 172.21.15.27:0/1445669093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f224c003970 0x7f224c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.366+0000 7f2264898640 1 --2- 172.21.15.27:0/1445669093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f224c004d00 0x7f224c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.366+0000 7f2257fff640 1 --2- 172.21.15.27:0/1445669093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2260070750 0x7f224c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.366+0000 7f225ffff640 1 -- 172.21.15.27:0/1445669093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2260070750 msgr2=0x7f224c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:51.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.366+0000 7f225ffff640 1 --2- 172.21.15.27:0/1445669093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2260070750 0x7f224c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.366+0000 7f225ffff640 1 -- 172.21.15.27:0/1445669093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f224c004d00 msgr2=0x7f224c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:51.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.366+0000 7f225ffff640 1 --2- 172.21.15.27:0/1445669093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f224c004d00 0x7f224c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.366+0000 7f225ffff640 1 -- 172.21.15.27:0/1445669093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c003520 con 0x7f224c003970 2023-12-19T18:14:51.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.366+0000 7f2257fff640 1 --2- 172.21.15.27:0/1445669093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2260070750 0x7f224c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:51.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.367+0000 7f225ffff640 1 --2- 172.21.15.27:0/1445669093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f224c003970 0x7f224c003d90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2250009ac0 tx=0x7f224c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f51eed3a2e594ffb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:51.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.367+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225002c710 con 0x7f224c003970 2023-12-19T18:14:51.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.367+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f225002c870 con 0x7f224c003970 2023-12-19T18:14:51.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.367+0000 7f225f7fe640 1 -- 172.21.15.27:0/1445669093 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225002cc00 con 0x7f224c003970 2023-12-19T18:14:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.368+0000 7f2265099640 1 -- 172.21.15.27:0/1445669093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f224c003970 msgr2=0x7f224c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.368+0000 7f2265099640 1 --2- 172.21.15.27:0/1445669093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f224c003970 0x7f224c003d90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2250009ac0 tx=0x7f224c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.368+0000 7f2264898640 1 -- 172.21.15.27:0/1445669093 reap_dead start 2023-12-19T18:14:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.368+0000 7f2265099640 1 -- 172.21.15.27:0/1445669093 shutdown_connections 2023-12-19T18:14:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.368+0000 7f2265099640 1 -- 172.21.15.27:0/1445669093 >> 172.21.15.27:0/1445669093 conn(0x7f226006a7d0 msgr2=0x7f22600ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.368+0000 7f2265099640 1 -- 172.21.15.27:0/1445669093 shutdown_connections 2023-12-19T18:14:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.368+0000 7f2265099640 1 -- 172.21.15.27:0/1445669093 wait complete. 2023-12-19T18:14:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.370+0000 7f2265099640 1 Processor -- start 2023-12-19T18:14:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.370+0000 7f2265099640 1 -- start start 2023-12-19T18:14:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.371+0000 7f2265099640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f224c003970 0x7f226015c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.371+0000 7f2265099640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f226014e790 0x7f226014d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.371+0000 7f2265099640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f226015d250 0x7f226015d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.371+0000 7f2265099640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226015dc10 con 0x7f226014e790 2023-12-19T18:14:51.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.371+0000 7f2265099640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226015bc90 con 0x7f226015d250 2023-12-19T18:14:51.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.371+0000 7f2265099640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226014d7d0 con 0x7f224c003970 2023-12-19T18:14:51.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.371+0000 7f2257fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f226014e790 0x7f226014d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.372+0000 7f2257fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f226014e790 0x7f226014d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38730/0 (socket says 172.21.15.27:38730) 2023-12-19T18:14:51.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.372+0000 7f2257fff640 1 -- 172.21.15.27:0/3300361302 learned_addr learned my addr 172.21.15.27:0/3300361302 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:51.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.372+0000 7f225ffff640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f224c003970 0x7f226015c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.372+0000 7f2264898640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f226015d250 0x7f226015d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.372+0000 7f2257fff640 1 -- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f224c003970 msgr2=0x7f226015c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.372+0000 7f2257fff640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f224c003970 0x7f226015c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.372+0000 7f2257fff640 1 -- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f226015d250 msgr2=0x7f226015d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.372+0000 7f2257fff640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f226015d250 0x7f226015d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.372+0000 7f2257fff640 1 -- 172.21.15.27:0/3300361302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226014ec10 con 0x7f226014e790 2023-12-19T18:14:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.372+0000 7f2264898640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f226015d250 0x7f226015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.373+0000 7f2257fff640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f226014e790 0x7f226014d290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f22480018e0 tx=0x7f224800dc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.373+0000 7f225d7fa640 1 -- 172.21.15.27:0/3300361302 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224800dea0 con 0x7f226014e790 2023-12-19T18:14:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.373+0000 7f225d7fa640 1 -- 172.21.15.27:0/3300361302 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22480074e0 con 0x7f226014e790 2023-12-19T18:14:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.373+0000 7f225d7fa640 1 -- 172.21.15.27:0/3300361302 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2248004a00 con 0x7f226014e790 2023-12-19T18:14:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.373+0000 7f2265099640 1 -- 172.21.15.27:0/3300361302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f226015b080 con 0x7f226014e790 2023-12-19T18:14:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.373+0000 7f2265099640 1 -- 172.21.15.27:0/3300361302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226015b640 con 0x7f226014e790 2023-12-19T18:14:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.374+0000 7f2265099640 1 -- 172.21.15.27:0/3300361302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226004e840 con 0x7f226014e790 2023-12-19T18:14:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.375+0000 7f225d7fa640 1 -- 172.21.15.27:0/3300361302 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2248004b60 con 0x7f226014e790 2023-12-19T18:14:51.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.376+0000 7f225d7fa640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f224006a430 0x7f224006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.377+0000 7f225d7fa640 1 -- 172.21.15.27:0/3300361302 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2248087ea0 con 0x7f226014e790 2023-12-19T18:14:51.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.377+0000 7f225ffff640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f224006a430 0x7f224006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.378+0000 7f225ffff640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f224006a430 0x7f224006c8f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2250036000 tx=0x7f2250030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:51.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.380+0000 7f225d7fa640 1 -- 172.21.15.27:0/3300361302 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f22480537f0 con 0x7f226014e790 2023-12-19T18:14:51.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.575+0000 7f2265099640 1 -- 172.21.15.27:0/3300361302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f226006ce20 con 0x7f226014e790 2023-12-19T18:14:51.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.576+0000 7f225d7fa640 1 -- 172.21.15.27:0/3300361302 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7f2248019090 con 0x7f226014e790 2023-12-19T18:14:51.576 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:51.577 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-12-19T18:14:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:14:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-12-19T18:14:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-12-19T18:14:51.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:14:51.577 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-12-19T18:14:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.579+0000 7f2265099640 1 -- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f224006a430 msgr2=0x7f224006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.579+0000 7f2265099640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f224006a430 0x7f224006c8f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2250036000 tx=0x7f2250030000 comp rx=0 tx=0).stop 2023-12-19T18:14:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.579+0000 7f2265099640 1 -- 172.21.15.27:0/3300361302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f226014e790 msgr2=0x7f226014d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.579+0000 7f2265099640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f226014e790 0x7f226014d290 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f22480018e0 tx=0x7f224800dc50 comp rx=0 tx=0).stop 2023-12-19T18:14:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.579+0000 7f2265099640 1 -- 172.21.15.27:0/3300361302 shutdown_connections 2023-12-19T18:14:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.579+0000 7f2265099640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f224006a430 0x7f224006c8f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.579+0000 7f2265099640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f226015d250 0x7f226015d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.579+0000 7f2265099640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f226014e790 0x7f226014d290 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.579+0000 7f2265099640 1 --2- 172.21.15.27:0/3300361302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f224c003970 0x7f226015c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.579+0000 7f2265099640 1 -- 172.21.15.27:0/3300361302 >> 172.21.15.27:0/3300361302 conn(0x7f226006a7d0 msgr2=0x7f22600cc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.579+0000 7f2265099640 1 -- 172.21.15.27:0/3300361302 shutdown_connections 2023-12-19T18:14:51.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.579+0000 7f2265099640 1 -- 172.21.15.27:0/3300361302 wait complete. 2023-12-19T18:14:51.599 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:14:51.600 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:14:51.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.842+0000 7feef80fb640 1 Processor -- start 2023-12-19T18:14:51.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.843+0000 7feef80fb640 1 -- start start 2023-12-19T18:14:51.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.844+0000 7feef80fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feef015d4d0 0x7feef00cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.844+0000 7feef80fb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feef00ac8d0 0x7feef00cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.844+0000 7feef80fb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7feef00cd050 0x7feef0160fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.844+0000 7feef80fb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef006bea0 con 0x7feef00ac8d0 2023-12-19T18:14:51.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.844+0000 7feef80fb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef015d8b0 con 0x7feef00cd050 2023-12-19T18:14:51.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.844+0000 7feef80fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef006ccb0 con 0x7feef015d4d0 2023-12-19T18:14:51.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.845+0000 7feef5696640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feef00ac8d0 0x7feef00cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.845+0000 7feef5696640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feef00ac8d0 0x7feef00cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38738/0 (socket says 172.21.15.27:38738) 2023-12-19T18:14:51.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.845+0000 7feef5696640 1 -- 172.21.15.27:0/3720254315 learned_addr learned my addr 172.21.15.27:0/3720254315 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:51.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.845+0000 7feef5e97640 1 --2- 172.21.15.27:0/3720254315 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feef015d4d0 0x7feef00cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.845+0000 7feef6698640 1 --2- 172.21.15.27:0/3720254315 >> v2:172.21.15.192:3300/0 conn(0x7feef00cd050 0x7feef0160fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.845+0000 7feef5696640 1 -- 172.21.15.27:0/3720254315 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feef015d4d0 msgr2=0x7feef00cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:51.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.845+0000 7feef5696640 1 --2- 172.21.15.27:0/3720254315 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feef015d4d0 0x7feef00cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.845+0000 7feef5696640 1 -- 172.21.15.27:0/3720254315 >> v2:172.21.15.192:3300/0 conn(0x7feef00cd050 msgr2=0x7feef0160fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:51.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.845+0000 7feef5696640 1 --2- 172.21.15.27:0/3720254315 >> v2:172.21.15.192:3300/0 conn(0x7feef00cd050 0x7feef0160fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.845+0000 7feef5696640 1 -- 172.21.15.27:0/3720254315 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef006c120 con 0x7feef00ac8d0 2023-12-19T18:14:51.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.847+0000 7feef5696640 1 --2- 172.21.15.27:0/3720254315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feef00ac8d0 0x7feef00cb5f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feef006b1a0 tx=0x7feee002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=864ac37513a3b41d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:51.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.847+0000 7feef4e95640 1 -- 172.21.15.27:0/3720254315 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0030850 con 0x7feef00ac8d0 2023-12-19T18:14:51.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.847+0000 7feef4e95640 1 -- 172.21.15.27:0/3720254315 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee00309b0 con 0x7feef00ac8d0 2023-12-19T18:14:51.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.847+0000 7feef4e95640 1 -- 172.21.15.27:0/3720254315 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0030d40 con 0x7feef00ac8d0 2023-12-19T18:14:51.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.848+0000 7feef80fb640 1 -- 172.21.15.27:0/3720254315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feef00ac8d0 msgr2=0x7feef00cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.848+0000 7feef80fb640 1 --2- 172.21.15.27:0/3720254315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feef00ac8d0 0x7feef00cb5f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7feef006b1a0 tx=0x7feee002b990 comp rx=0 tx=0).stop 2023-12-19T18:14:51.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.848+0000 7feef80fb640 1 -- 172.21.15.27:0/3720254315 shutdown_connections 2023-12-19T18:14:51.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.848+0000 7feef80fb640 1 --2- 172.21.15.27:0/3720254315 >> v2:172.21.15.192:3300/0 conn(0x7feef00cd050 0x7feef0160fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.848+0000 7feef80fb640 1 --2- 172.21.15.27:0/3720254315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feef00ac8d0 0x7feef00cb5f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.848+0000 7feef80fb640 1 --2- 172.21.15.27:0/3720254315 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feef015d4d0 0x7feef00cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.848+0000 7feef80fb640 1 -- 172.21.15.27:0/3720254315 >> 172.21.15.27:0/3720254315 conn(0x7feef00b0620 msgr2=0x7feef00b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:51.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.848+0000 7feef80fb640 1 -- 172.21.15.27:0/3720254315 shutdown_connections 2023-12-19T18:14:51.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.848+0000 7feef80fb640 1 -- 172.21.15.27:0/3720254315 wait complete. 2023-12-19T18:14:51.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.850+0000 7feef80fb640 1 Processor -- start 2023-12-19T18:14:51.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.851+0000 7feef80fb640 1 -- start start 2023-12-19T18:14:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.852+0000 7feef80fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feef00ac8d0 0x7feef00c8290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.852+0000 7feef80fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feef00cd050 0x7feef00c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.852+0000 7feef80fb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feef015d4d0 0x7feef0146840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.852+0000 7feef80fb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef00c87d0 con 0x7feef00cd050 2023-12-19T18:14:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.852+0000 7feef80fb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef0147ca0 con 0x7feef015d4d0 2023-12-19T18:14:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.852+0000 7feef80fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef00c1df0 con 0x7feef00ac8d0 2023-12-19T18:14:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.852+0000 7feef5696640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feef00cd050 0x7feef00c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.852+0000 7feef5696640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feef00cd050 0x7feef00c4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38740/0 (socket says 172.21.15.27:38740) 2023-12-19T18:14:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.852+0000 7feef5696640 1 -- 172.21.15.27:0/2241544253 learned_addr learned my addr 172.21.15.27:0/2241544253 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.852+0000 7feef5e97640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feef00ac8d0 0x7feef00c8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.853+0000 7feef6698640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feef015d4d0 0x7feef0146840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.853+0000 7feef5696640 1 -- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feef00ac8d0 msgr2=0x7feef00c8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.853+0000 7feef5696640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feef00ac8d0 0x7feef00c8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.853+0000 7feef5696640 1 -- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feef015d4d0 msgr2=0x7feef0146840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.853+0000 7feef5696640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feef015d4d0 0x7feef0146840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.853+0000 7feef5696640 1 -- 172.21.15.27:0/2241544253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee00094d0 con 0x7feef00cd050 2023-12-19T18:14:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.853+0000 7feef5e97640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feef00ac8d0 0x7feef00c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.853+0000 7feef5696640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feef00cd050 0x7feef00c4da0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feee0000c00 tx=0x7feee0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.853+0000 7feee67fc640 1 -- 172.21.15.27:0/2241544253 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0039b60 con 0x7feef00cd050 2023-12-19T18:14:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.854+0000 7feee67fc640 1 -- 172.21.15.27:0/2241544253 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee0039ce0 con 0x7feef00cd050 2023-12-19T18:14:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.854+0000 7feee67fc640 1 -- 172.21.15.27:0/2241544253 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee0043650 con 0x7feef00cd050 2023-12-19T18:14:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.854+0000 7feef80fb640 1 -- 172.21.15.27:0/2241544253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef0147ef0 con 0x7feef00cd050 2023-12-19T18:14:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.854+0000 7feef80fb640 1 -- 172.21.15.27:0/2241544253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef01484b0 con 0x7feef00cd050 2023-12-19T18:14:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.855+0000 7feef80fb640 1 -- 172.21.15.27:0/2241544253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef00ccec0 con 0x7feef00cd050 2023-12-19T18:14:51.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.856+0000 7feee67fc640 1 -- 172.21.15.27:0/2241544253 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7feee002e070 con 0x7feef00cd050 2023-12-19T18:14:51.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.857+0000 7feee67fc640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feed406a430 0x7feed406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:51.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.858+0000 7feee67fc640 1 -- 172.21.15.27:0/2241544253 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7feee00aa880 con 0x7feef00cd050 2023-12-19T18:14:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.858+0000 7feef5e97640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feed406a430 0x7feed406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:51.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.859+0000 7feef5e97640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feed406a430 0x7feed406c8f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feeec009b20 tx=0x7feeec0039d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:51.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:51.863+0000 7feee67fc640 1 -- 172.21.15.27:0/2241544253 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7feee0076370 con 0x7feef00cd050 2023-12-19T18:14:52.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.060+0000 7feef80fb640 1 -- 172.21.15.27:0/2241544253 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7feef0068e90 con 0x7feef00cd050 2023-12-19T18:14:52.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.519+0000 7feee67fc640 1 -- 172.21.15.27:0/2241544253 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7feee00340b0 con 0x7feef00cd050 2023-12-19T18:14:52.520 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-12-19T18:14:52.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.524+0000 7feef80fb640 1 -- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feed406a430 msgr2=0x7feed406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.524+0000 7feef80fb640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feed406a430 0x7feed406c8f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feeec009b20 tx=0x7feeec0039d0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.524+0000 7feef80fb640 1 -- 172.21.15.27:0/2241544253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feef00cd050 msgr2=0x7feef00c4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.524+0000 7feef80fb640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feef00cd050 0x7feef00c4da0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7feee0000c00 tx=0x7feee0004730 comp rx=0 tx=0).stop 2023-12-19T18:14:52.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.525+0000 7feef80fb640 1 -- 172.21.15.27:0/2241544253 shutdown_connections 2023-12-19T18:14:52.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.525+0000 7feef80fb640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feed406a430 0x7feed406c8f0 secure :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7feeec009b20 tx=0x7feeec0039d0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.525+0000 7feef80fb640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feef015d4d0 0x7feef0146840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.525+0000 7feef80fb640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feef00cd050 0x7feef00c4da0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.525+0000 7feef80fb640 1 --2- 172.21.15.27:0/2241544253 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feef00ac8d0 0x7feef00c8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.525+0000 7feef80fb640 1 -- 172.21.15.27:0/2241544253 >> 172.21.15.27:0/2241544253 conn(0x7feef00b0620 msgr2=0x7feef00af790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:52.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.528+0000 7feef80fb640 1 -- 172.21.15.27:0/2241544253 shutdown_connections 2023-12-19T18:14:52.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.528+0000 7feef80fb640 1 -- 172.21.15.27:0/2241544253 wait complete. 2023-12-19T18:14:52.551 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:14:52.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.784+0000 7f88bd84d640 1 Processor -- start 2023-12-19T18:14:52.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.785+0000 7f88bd84d640 1 -- start start 2023-12-19T18:14:52.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.786+0000 7f88bd84d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f88b80cc870 0x7f88b806afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:52.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.786+0000 7f88bd84d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b80b03f0 con 0x7f88b80ad9d0 2023-12-19T18:14:52.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.786+0000 7f88bd84d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88b80b0c90 con 0x7f88b815ae60 2023-12-19T18:14:52.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.786+0000 7f88bd84d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88b80b4740 con 0x7f88b80cc870 2023-12-19T18:14:52.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.787+0000 7f88bd04c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f88b80ad9d0 0x7f88b806b4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35510/0 (socket says 172.21.15.27:35510) 2023-12-19T18:14:52.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.787+0000 7f88bd04c640 1 -- 172.21.15.27:0/983747395 learned_addr learned my addr 172.21.15.27:0/983747395 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:52.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.787+0000 7f88b7fff640 1 --2- 172.21.15.27:0/983747395 >> v2:172.21.15.192:3300/0 conn(0x7f88b80cc870 0x7f88b806afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:52.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.787+0000 7f88b77fe640 1 -- 172.21.15.27:0/983747395 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3657493042 0 0) 0x7f88b80b03f0 con 0x7f88b80ad9d0 2023-12-19T18:14:52.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.787+0000 7f88b77fe640 1 -- 172.21.15.27:0/983747395 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88a8003520 con 0x7f88b80ad9d0 2023-12-19T18:14:52.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.788+0000 7f88b77fe640 1 -- 172.21.15.27:0/983747395 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 414767365 0 0) 0x7f88a8003520 con 0x7f88b80ad9d0 2023-12-19T18:14:52.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.788+0000 7f88b7fff640 1 -- 172.21.15.27:0/983747395 >> v1:172.21.15.192:6789/0 conn(0x7f88b815ae60 legacy=0x7f88b8070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.788+0000 7f88b7fff640 1 -- 172.21.15.27:0/983747395 >> v1:172.21.15.27:6789/0 conn(0x7f88b80ad9d0 legacy=0x7f88b806b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.788+0000 7f88b7fff640 1 -- 172.21.15.27:0/983747395 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88b80b0640 con 0x7f88b80cc870 2023-12-19T18:14:52.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.789+0000 7f88b7fff640 1 --2- 172.21.15.27:0/983747395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80cc870 0x7f88b806afb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f88a40097a0 tx=0x7f88a402b790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dad0fc9431b1d40 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:52.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.790+0000 7f88b77fe640 1 -- 172.21.15.27:0/983747395 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88b00019f0 con 0x7f88b80cc870 2023-12-19T18:14:52.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.790+0000 7f88b77fe640 1 -- 172.21.15.27:0/983747395 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a402e050 con 0x7f88b80cc870 2023-12-19T18:14:52.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.790+0000 7f88b77fe640 1 -- 172.21.15.27:0/983747395 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a4030730 con 0x7f88b80cc870 2023-12-19T18:14:52.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.790+0000 7f88bd84d640 1 -- 172.21.15.27:0/983747395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80cc870 msgr2=0x7f88b806afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.790+0000 7f88bd84d640 1 --2- 172.21.15.27:0/983747395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80cc870 0x7f88b806afb0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f88a40097a0 tx=0x7f88a402b790 comp rx=0 tx=0).stop 2023-12-19T18:14:52.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.790+0000 7f88bd84d640 1 -- 172.21.15.27:0/983747395 shutdown_connections 2023-12-19T18:14:52.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.790+0000 7f88bd84d640 1 --2- 172.21.15.27:0/983747395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80cc870 0x7f88b806afb0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.790+0000 7f88bd84d640 1 -- 172.21.15.27:0/983747395 >> 172.21.15.27:0/983747395 conn(0x7f88b80cb3f0 msgr2=0x7f88b80cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:52.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.790+0000 7f88bd84d640 1 -- 172.21.15.27:0/983747395 shutdown_connections 2023-12-19T18:14:52.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.791+0000 7f88bd84d640 1 -- 172.21.15.27:0/983747395 wait complete. 2023-12-19T18:14:52.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.792+0000 7f88bd84d640 1 Processor -- start 2023-12-19T18:14:52.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.793+0000 7f88bd84d640 1 -- start start 2023-12-19T18:14:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88bd84d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80ad9d0 0x7f88b81f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88bd84d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b80cc870 0x7f88b80b2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88bd84d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b815ae60 0x7f88b80b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88bd84d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b80c27e0 con 0x7f88b80cc870 2023-12-19T18:14:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88bd84d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b81f39b0 con 0x7f88b80ad9d0 2023-12-19T18:14:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88bd84d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88b81f3c90 con 0x7f88b815ae60 2023-12-19T18:14:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88b7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b80cc870 0x7f88b80b2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88b7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b80cc870 0x7f88b80b2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54472/0 (socket says 172.21.15.27:54472) 2023-12-19T18:14:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88b7fff640 1 -- 172.21.15.27:0/1672943694 learned_addr learned my addr 172.21.15.27:0/1672943694 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:52.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88bc84b640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80ad9d0 0x7f88b81f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88bd04c640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b815ae60 0x7f88b80b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88b7fff640 1 -- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b815ae60 msgr2=0x7f88b80b30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88b7fff640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b815ae60 0x7f88b80b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88b7fff640 1 -- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80ad9d0 msgr2=0x7f88b81f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88b7fff640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80ad9d0 0x7f88b81f2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88b7fff640 1 -- 172.21.15.27:0/1672943694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a4009400 con 0x7f88b80cc870 2023-12-19T18:14:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.794+0000 7f88bc84b640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80ad9d0 0x7f88b81f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.795+0000 7f88b7fff640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b80cc870 0x7f88b80b2b60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f88a40098c0 tx=0x7f88a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.795+0000 7f88b57fa640 1 -- 172.21.15.27:0/1672943694 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a403c880 con 0x7f88b80cc870 2023-12-19T18:14:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.795+0000 7f88b57fa640 1 -- 172.21.15.27:0/1672943694 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a403c9e0 con 0x7f88b80cc870 2023-12-19T18:14:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.795+0000 7f88b57fa640 1 -- 172.21.15.27:0/1672943694 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a403cd90 con 0x7f88b80cc870 2023-12-19T18:14:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.795+0000 7f88bd84d640 1 -- 172.21.15.27:0/1672943694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b81f3f70 con 0x7f88b80cc870 2023-12-19T18:14:52.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.795+0000 7f88bd84d640 1 -- 172.21.15.27:0/1672943694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b81f3520 con 0x7f88b80cc870 2023-12-19T18:14:52.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.796+0000 7f88bd84d640 1 -- 172.21.15.27:0/1672943694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b81f3710 con 0x7f88b80cc870 2023-12-19T18:14:52.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.798+0000 7f88b57fa640 1 -- 172.21.15.27:0/1672943694 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f88a4039040 con 0x7f88b80cc870 2023-12-19T18:14:52.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.799+0000 7f88b57fa640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f889c06a430 0x7f889c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:52.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.799+0000 7f88b57fa640 1 -- 172.21.15.27:0/1672943694 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f88a40aa3a0 con 0x7f88b80cc870 2023-12-19T18:14:52.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.799+0000 7f88bc84b640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f889c06a430 0x7f889c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:52.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.800+0000 7f88bc84b640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f889c06a430 0x7f889c06c8f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f88b80c7b50 tx=0x7f88ac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:52.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.804+0000 7f88b57fa640 1 -- 172.21.15.27:0/1672943694 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f88a4075c90 con 0x7f88b80cc870 2023-12-19T18:14:52.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:52.999+0000 7f88bd84d640 1 -- 172.21.15.27:0/1672943694 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f88b804e840 con 0x7f88b80cc870 2023-12-19T18:14:53.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.537+0000 7f88b57fa640 1 -- 172.21.15.27:0/1672943694 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f88a40792b0 con 0x7f88b80cc870 2023-12-19T18:14:53.542 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-12-19T18:14:53.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.547+0000 7f88bd84d640 1 -- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f889c06a430 msgr2=0x7f889c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:53.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.547+0000 7f88bd84d640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f889c06a430 0x7f889c06c8f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f88b80c7b50 tx=0x7f88ac009040 comp rx=0 tx=0).stop 2023-12-19T18:14:53.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.547+0000 7f88bd84d640 1 -- 172.21.15.27:0/1672943694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b80cc870 msgr2=0x7f88b80b2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:53.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.547+0000 7f88bd84d640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b80cc870 0x7f88b80b2b60 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f88a40098c0 tx=0x7f88a40046c0 comp rx=0 tx=0).stop 2023-12-19T18:14:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.548+0000 7f88bd84d640 1 -- 172.21.15.27:0/1672943694 shutdown_connections 2023-12-19T18:14:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.548+0000 7f88bd84d640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f889c06a430 0x7f889c06c8f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.548+0000 7f88bd84d640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88b815ae60 0x7f88b80b30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.548+0000 7f88bd84d640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b80cc870 0x7f88b80b2b60 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.548+0000 7f88bd84d640 1 --2- 172.21.15.27:0/1672943694 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f88b80ad9d0 0x7f88b81f2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.548+0000 7f88bd84d640 1 -- 172.21.15.27:0/1672943694 >> 172.21.15.27:0/1672943694 conn(0x7f88b80cb3f0 msgr2=0x7f88b8070a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:53.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.549+0000 7f88bd84d640 1 -- 172.21.15.27:0/1672943694 shutdown_connections 2023-12-19T18:14:53.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.550+0000 7f88bd84d640 1 -- 172.21.15.27:0/1672943694 wait complete. 2023-12-19T18:14:53.577 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:14:53.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.783+0000 7f8a17fff640 1 Processor -- start 2023-12-19T18:14:53.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.783+0000 7f8a17fff640 1 -- start start 2023-12-19T18:14:53.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.784+0000 7f8a17fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8a1815d120 0x7f8a180cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:53.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.784+0000 7f8a17fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8a180d1310 0x7f8a180d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:53.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.784+0000 7f8a17fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a180b2970 0x7f8a180b2e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:53.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a17fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a180d3850 con 0x7f8a180d1310 2023-12-19T18:14:53.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a17fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a180d0b10 con 0x7f8a1815d120 2023-12-19T18:14:53.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a17fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a180d1c90 con 0x7f8a180b2970 2023-12-19T18:14:53.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a167fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8a180d1310 0x7f8a180d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:53.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a167fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8a180d1310 0x7f8a180d1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54484/0 (socket says 172.21.15.27:54484) 2023-12-19T18:14:53.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a167fc640 1 -- 172.21.15.27:0/350823131 learned_addr learned my addr 172.21.15.27:0/350823131 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:53.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a16ffd640 1 --2- 172.21.15.27:0/350823131 >> v2:172.21.15.192:3300/0 conn(0x7f8a1815d120 0x7f8a180cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:53.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a177fe640 1 --2- 172.21.15.27:0/350823131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a180b2970 0x7f8a180b2e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:53.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a167fc640 1 -- 172.21.15.27:0/350823131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a180b2970 msgr2=0x7f8a180b2e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:53.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a167fc640 1 --2- 172.21.15.27:0/350823131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a180b2970 0x7f8a180b2e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:53.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a167fc640 1 -- 172.21.15.27:0/350823131 >> v2:172.21.15.192:3300/0 conn(0x7f8a1815d120 msgr2=0x7f8a180cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:53.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a167fc640 1 --2- 172.21.15.27:0/350823131 >> v2:172.21.15.192:3300/0 conn(0x7f8a1815d120 0x7f8a180cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:53.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a167fc640 1 -- 172.21.15.27:0/350823131 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a1815d500 con 0x7f8a180d1310 2023-12-19T18:14:53.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.785+0000 7f8a177fe640 1 --2- 172.21.15.27:0/350823131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a180b2970 0x7f8a180b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:53.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.787+0000 7f8a167fc640 1 --2- 172.21.15.27:0/350823131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a180d1310 0x7f8a180d1750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8a18071e70 tx=0x7f8a0802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3958182aabe6f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:53.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.787+0000 7f8a15ffb640 1 -- 172.21.15.27:0/350823131 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0802ca70 con 0x7f8a180d1310 2023-12-19T18:14:53.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.787+0000 7f8a15ffb640 1 -- 172.21.15.27:0/350823131 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0802cbd0 con 0x7f8a180d1310 2023-12-19T18:14:53.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.787+0000 7f8a15ffb640 1 -- 172.21.15.27:0/350823131 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a080343f0 con 0x7f8a180d1310 2023-12-19T18:14:53.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.787+0000 7f8a17fff640 1 -- 172.21.15.27:0/350823131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a180d1310 msgr2=0x7f8a180d1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:53.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.787+0000 7f8a17fff640 1 --2- 172.21.15.27:0/350823131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a180d1310 0x7f8a180d1750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8a18071e70 tx=0x7f8a0802bed0 comp rx=0 tx=0).stop 2023-12-19T18:14:53.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.788+0000 7f8a17fff640 1 -- 172.21.15.27:0/350823131 shutdown_connections 2023-12-19T18:14:53.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.788+0000 7f8a17fff640 1 --2- 172.21.15.27:0/350823131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a180b2970 0x7f8a180b2e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:53.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.788+0000 7f8a17fff640 1 --2- 172.21.15.27:0/350823131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a180d1310 0x7f8a180d1750 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:53.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.788+0000 7f8a17fff640 1 --2- 172.21.15.27:0/350823131 >> v2:172.21.15.192:3300/0 conn(0x7f8a1815d120 0x7f8a180cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:53.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.788+0000 7f8a17fff640 1 -- 172.21.15.27:0/350823131 >> 172.21.15.27:0/350823131 conn(0x7f8a180d2d30 msgr2=0x7f8a180d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:53.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.788+0000 7f8a17fff640 1 -- 172.21.15.27:0/350823131 shutdown_connections 2023-12-19T18:14:53.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.788+0000 7f8a17fff640 1 -- 172.21.15.27:0/350823131 wait complete. 2023-12-19T18:14:53.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.790+0000 7f8a17fff640 1 Processor -- start 2023-12-19T18:14:53.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.790+0000 7f8a17fff640 1 -- start start 2023-12-19T18:14:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.791+0000 7f8a17fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a180b2970 0x7f8a1815e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.791+0000 7f8a17fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a180d1310 0x7f8a1814f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.791+0000 7f8a17fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a1815d120 0x7f8a18160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.791+0000 7f8a17fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a180b6410 con 0x7f8a1815d120 2023-12-19T18:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.791+0000 7f8a17fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1814faf0 con 0x7f8a180b2970 2023-12-19T18:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.791+0000 7f8a17fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a1815ecf0 con 0x7f8a180d1310 2023-12-19T18:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.791+0000 7f8a177fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a1815d120 0x7f8a18160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.791+0000 7f8a177fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a1815d120 0x7f8a18160000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54498/0 (socket says 172.21.15.27:54498) 2023-12-19T18:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.791+0000 7f8a177fe640 1 -- 172.21.15.27:0/760747889 learned_addr learned my addr 172.21.15.27:0/760747889 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.791+0000 7f8a16ffd640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a180b2970 0x7f8a1815e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.791+0000 7f8a167fc640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a180d1310 0x7f8a1814f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.792+0000 7f8a177fe640 1 -- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a180d1310 msgr2=0x7f8a1814f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.792+0000 7f8a177fe640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a180d1310 0x7f8a1814f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.792+0000 7f8a177fe640 1 -- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a180b2970 msgr2=0x7f8a1815e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.792+0000 7f8a177fe640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a180b2970 0x7f8a1815e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.792+0000 7f8a177fe640 1 -- 172.21.15.27:0/760747889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a08009b30 con 0x7f8a1815d120 2023-12-19T18:14:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.792+0000 7f8a16ffd640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a180b2970 0x7f8a1815e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:14:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.792+0000 7f8a177fe640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a1815d120 0x7f8a18160000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000d5d0 tx=0x7f8a1000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.792+0000 7f8a0f7fe640 1 -- 172.21.15.27:0/760747889 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10004030 con 0x7f8a1815d120 2023-12-19T18:14:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.792+0000 7f8a0f7fe640 1 -- 172.21.15.27:0/760747889 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a10004190 con 0x7f8a1815d120 2023-12-19T18:14:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.792+0000 7f8a0f7fe640 1 -- 172.21.15.27:0/760747889 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a10014620 con 0x7f8a1815d120 2023-12-19T18:14:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.792+0000 7f8a17fff640 1 -- 172.21.15.27:0/760747889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a1815efd0 con 0x7f8a1815d120 2023-12-19T18:14:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.792+0000 7f8a17fff640 1 -- 172.21.15.27:0/760747889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a1815f590 con 0x7f8a1815d120 2023-12-19T18:14:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.793+0000 7f8a17fff640 1 -- 172.21.15.27:0/760747889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a180b6550 con 0x7f8a1815d120 2023-12-19T18:14:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.795+0000 7f8a0f7fe640 1 -- 172.21.15.27:0/760747889 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8a10014780 con 0x7f8a1815d120 2023-12-19T18:14:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.796+0000 7f8a0f7fe640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f89f806a430 0x7f89f806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.796+0000 7f8a0f7fe640 1 -- 172.21.15.27:0/760747889 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f8a10088ed0 con 0x7f8a1815d120 2023-12-19T18:14:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.796+0000 7f8a16ffd640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f89f806a430 0x7f89f806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:53.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.797+0000 7f8a16ffd640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f89f806a430 0x7f89f806c8f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009af0 tx=0x7f8a00008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:53.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:53.801+0000 7f8a0f7fe640 1 -- 172.21.15.27:0/760747889 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8a180b6550 con 0x7f8a1815d120 2023-12-19T18:14:54.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.033+0000 7f8a17fff640 1 -- 172.21.15.27:0/760747889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8a180b6550 con 0x7f8a1815d120 2023-12-19T18:14:54.049 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 2 and data pool 3 2023-12-19T18:14:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.047+0000 7f8a0f7fe640 1 -- 172.21.15.27:0/760747889 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8a180b6550 con 0x7f8a1815d120 2023-12-19T18:14:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.054+0000 7f8a17fff640 1 -- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f89f806a430 msgr2=0x7f89f806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.054+0000 7f8a17fff640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f89f806a430 0x7f89f806c8f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009af0 tx=0x7f8a00008040 comp rx=0 tx=0).stop 2023-12-19T18:14:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.054+0000 7f8a17fff640 1 -- 172.21.15.27:0/760747889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a1815d120 msgr2=0x7f8a18160000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.054+0000 7f8a17fff640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a1815d120 0x7f8a18160000 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f8a1000d5d0 tx=0x7f8a1000da90 comp rx=0 tx=0).stop 2023-12-19T18:14:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.054+0000 7f8a17fff640 1 -- 172.21.15.27:0/760747889 shutdown_connections 2023-12-19T18:14:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.054+0000 7f8a17fff640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f89f806a430 0x7f89f806c8f0 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8a00009af0 tx=0x7f8a00008040 comp rx=0 tx=0).stop 2023-12-19T18:14:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.054+0000 7f8a17fff640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a1815d120 0x7f8a18160000 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.054+0000 7f8a17fff640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8a180d1310 0x7f8a1814f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.054+0000 7f8a17fff640 1 --2- 172.21.15.27:0/760747889 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8a180b2970 0x7f8a1815e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.054+0000 7f8a17fff640 1 -- 172.21.15.27:0/760747889 >> 172.21.15.27:0/760747889 conn(0x7f8a180d2d30 msgr2=0x7f8a180d3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.056+0000 7f8a17fff640 1 -- 172.21.15.27:0/760747889 shutdown_connections 2023-12-19T18:14:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.056+0000 7f8a17fff640 1 -- 172.21.15.27:0/760747889 wait complete. 2023-12-19T18:14:54.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:54.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.279+0000 7f22eea0e640 1 Processor -- start 2023-12-19T18:14:54.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.279+0000 7f22eea0e640 1 -- start start 2023-12-19T18:14:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.281+0000 7f22eea0e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e81553a0 0x7f22e80c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.281+0000 7f22eea0e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f22e80afeb0 0x7f22e815e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.281+0000 7f22eea0e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22e815ec60 0x7f22e815b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.281+0000 7f22eea0e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e80cbcf0 con 0x7f22e815ec60 2023-12-19T18:14:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.281+0000 7f22eea0e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e815f0d0 con 0x7f22e80afeb0 2023-12-19T18:14:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.281+0000 7f22eea0e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e80b0330 con 0x7f22e81553a0 2023-12-19T18:14:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.281+0000 7f22ee20d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22e815ec60 0x7f22e815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.281+0000 7f22ee20d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22e815ec60 0x7f22e815b130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54502/0 (socket says 172.21.15.27:54502) 2023-12-19T18:14:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.281+0000 7f22ee20d640 1 -- 172.21.15.27:0/1778432502 learned_addr learned my addr 172.21.15.27:0/1778432502 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:54.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.282+0000 7f22eda0c640 1 --2- 172.21.15.27:0/1778432502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e81553a0 0x7f22e80c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.282+0000 7f22ed20b640 1 --2- 172.21.15.27:0/1778432502 >> v2:172.21.15.192:3300/0 conn(0x7f22e80afeb0 0x7f22e815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.282+0000 7f22ee20d640 1 -- 172.21.15.27:0/1778432502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e81553a0 msgr2=0x7f22e80c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:54.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.282+0000 7f22ee20d640 1 --2- 172.21.15.27:0/1778432502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e81553a0 0x7f22e80c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.282+0000 7f22ee20d640 1 -- 172.21.15.27:0/1778432502 >> v2:172.21.15.192:3300/0 conn(0x7f22e80afeb0 msgr2=0x7f22e815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:54.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.282+0000 7f22ee20d640 1 --2- 172.21.15.27:0/1778432502 >> v2:172.21.15.192:3300/0 conn(0x7f22e80afeb0 0x7f22e815e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.282+0000 7f22ee20d640 1 -- 172.21.15.27:0/1778432502 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e80cb350 con 0x7f22e815ec60 2023-12-19T18:14:54.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.283+0000 7f22ee20d640 1 --2- 172.21.15.27:0/1778432502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e815ec60 0x7f22e815b130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f22e80ce830 tx=0x7f22e002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8745522e9319e33d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:54.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.283+0000 7f22eca0a640 1 -- 172.21.15.27:0/1778432502 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e002c9e0 con 0x7f22e815ec60 2023-12-19T18:14:54.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.283+0000 7f22eca0a640 1 -- 172.21.15.27:0/1778432502 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e002cb40 con 0x7f22e815ec60 2023-12-19T18:14:54.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.283+0000 7f22eca0a640 1 -- 172.21.15.27:0/1778432502 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e00343f0 con 0x7f22e815ec60 2023-12-19T18:14:54.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.284+0000 7f22eea0e640 1 -- 172.21.15.27:0/1778432502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e815ec60 msgr2=0x7f22e815b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:54.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.284+0000 7f22eea0e640 1 --2- 172.21.15.27:0/1778432502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e815ec60 0x7f22e815b130 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f22e80ce830 tx=0x7f22e002be40 comp rx=0 tx=0).stop 2023-12-19T18:14:54.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.284+0000 7f22eea0e640 1 -- 172.21.15.27:0/1778432502 shutdown_connections 2023-12-19T18:14:54.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.284+0000 7f22eea0e640 1 --2- 172.21.15.27:0/1778432502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e815ec60 0x7f22e815b130 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.284+0000 7f22eea0e640 1 --2- 172.21.15.27:0/1778432502 >> v2:172.21.15.192:3300/0 conn(0x7f22e80afeb0 0x7f22e815e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.284+0000 7f22eea0e640 1 --2- 172.21.15.27:0/1778432502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e81553a0 0x7f22e80c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.284+0000 7f22eea0e640 1 -- 172.21.15.27:0/1778432502 >> 172.21.15.27:0/1778432502 conn(0x7f22e8069c70 msgr2=0x7f22e806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:54.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.284+0000 7f22eea0e640 1 -- 172.21.15.27:0/1778432502 shutdown_connections 2023-12-19T18:14:54.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.284+0000 7f22eea0e640 1 -- 172.21.15.27:0/1778432502 wait complete. 2023-12-19T18:14:54.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.286+0000 7f22eea0e640 1 Processor -- start 2023-12-19T18:14:54.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.286+0000 7f22eea0e640 1 -- start start 2023-12-19T18:14:54.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.287+0000 7f22eea0e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e80afeb0 0x7f22e80cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.287+0000 7f22eea0e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e81553a0 0x7f22e815e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.287+0000 7f22eea0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e815ec60 0x7f22e81f66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.287+0000 7f22eea0e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e81f7730 con 0x7f22e815ec60 2023-12-19T18:14:54.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.287+0000 7f22eea0e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e81f6c30 con 0x7f22e81553a0 2023-12-19T18:14:54.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.287+0000 7f22eea0e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e815c520 con 0x7f22e80afeb0 2023-12-19T18:14:54.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.288+0000 7f22ee20d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e815ec60 0x7f22e81f66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.288+0000 7f22ee20d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e815ec60 0x7f22e81f66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54504/0 (socket says 172.21.15.27:54504) 2023-12-19T18:14:54.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.288+0000 7f22ee20d640 1 -- 172.21.15.27:0/2404700346 learned_addr learned my addr 172.21.15.27:0/2404700346 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:54.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.288+0000 7f22eda0c640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e80afeb0 0x7f22e80cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.288+0000 7f22ed20b640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e81553a0 0x7f22e815e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.288+0000 7f22ee20d640 1 -- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e80afeb0 msgr2=0x7f22e80cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:54.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.288+0000 7f22ee20d640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e80afeb0 0x7f22e80cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.288+0000 7f22ee20d640 1 -- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e81553a0 msgr2=0x7f22e815e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:54.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.288+0000 7f22ee20d640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e81553a0 0x7f22e815e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.288+0000 7f22ee20d640 1 -- 172.21.15.27:0/2404700346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e0009b30 con 0x7f22e815ec60 2023-12-19T18:14:54.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.288+0000 7f22ee20d640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e815ec60 0x7f22e81f66f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009e50 tx=0x7f22e0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:54.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.289+0000 7f22de7fc640 1 -- 172.21.15.27:0/2404700346 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e0009510 con 0x7f22e815ec60 2023-12-19T18:14:54.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.289+0000 7f22de7fc640 1 -- 172.21.15.27:0/2404700346 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e003e450 con 0x7f22e815ec60 2023-12-19T18:14:54.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.289+0000 7f22de7fc640 1 -- 172.21.15.27:0/2404700346 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e003e800 con 0x7f22e815ec60 2023-12-19T18:14:54.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.289+0000 7f22eea0e640 1 -- 172.21.15.27:0/2404700346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e81f6e90 con 0x7f22e815ec60 2023-12-19T18:14:54.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.289+0000 7f22eea0e640 1 -- 172.21.15.27:0/2404700346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e81f7450 con 0x7f22e815ec60 2023-12-19T18:14:54.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.290+0000 7f22eea0e640 1 -- 172.21.15.27:0/2404700346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e804e840 con 0x7f22e815ec60 2023-12-19T18:14:54.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.291+0000 7f22de7fc640 1 -- 172.21.15.27:0/2404700346 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f22e003e960 con 0x7f22e815ec60 2023-12-19T18:14:54.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.292+0000 7f22de7fc640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22cc06a430 0x7f22cc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.292+0000 7f22de7fc640 1 -- 172.21.15.27:0/2404700346 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22e00ab760 con 0x7f22e815ec60 2023-12-19T18:14:54.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.293+0000 7f22eda0c640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22cc06a430 0x7f22cc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.294+0000 7f22eda0c640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22cc06a430 0x7f22cc06c8f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f22e4006fd0 tx=0x7f22e4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:54.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.297+0000 7f22de7fc640 1 -- 172.21.15.27:0/2404700346 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f22e0076cb0 con 0x7f22e815ec60 2023-12-19T18:14:54.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.490+0000 7f22eea0e640 1 -- 172.21.15.27:0/2404700346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22e80af6a0 con 0x7f22e815ec60 2023-12-19T18:14:54.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.492+0000 7f22de7fc640 1 -- 172.21.15.27:0/2404700346 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16054 (secure 0 0 0) 0x7f22e007a2d0 con 0x7f22e815ec60 2023-12-19T18:14:54.493 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:54.493 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":19,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:54.035054+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:54.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.495+0000 7f22eea0e640 1 -- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22cc06a430 msgr2=0x7f22cc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:54.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.495+0000 7f22eea0e640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22cc06a430 0x7f22cc06c8f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f22e4006fd0 tx=0x7f22e4008040 comp rx=0 tx=0).stop 2023-12-19T18:14:54.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.495+0000 7f22eea0e640 1 -- 172.21.15.27:0/2404700346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e815ec60 msgr2=0x7f22e81f66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:54.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.495+0000 7f22eea0e640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e815ec60 0x7f22e81f66f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f22e0009e50 tx=0x7f22e0004a40 comp rx=0 tx=0).stop 2023-12-19T18:14:54.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.495+0000 7f22eea0e640 1 -- 172.21.15.27:0/2404700346 shutdown_connections 2023-12-19T18:14:54.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.495+0000 7f22eea0e640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22cc06a430 0x7f22cc06c8f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.495+0000 7f22eea0e640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e815ec60 0x7f22e81f66f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.495+0000 7f22eea0e640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e81553a0 0x7f22e815e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.495+0000 7f22eea0e640 1 --2- 172.21.15.27:0/2404700346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e80afeb0 0x7f22e80cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.495+0000 7f22eea0e640 1 -- 172.21.15.27:0/2404700346 >> 172.21.15.27:0/2404700346 conn(0x7f22e8069c70 msgr2=0x7f22e815afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:54.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.495+0000 7f22eea0e640 1 -- 172.21.15.27:0/2404700346 shutdown_connections 2023-12-19T18:14:54.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.495+0000 7f22eea0e640 1 -- 172.21.15.27:0/2404700346 wait complete. 2023-12-19T18:14:54.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.730+0000 7fbb4d79b640 1 Processor -- start 2023-12-19T18:14:54.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.730+0000 7fbb4d79b640 1 -- start start 2023-12-19T18:14:54.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.731+0000 7fbb4d79b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb4815a9a0 0x7fbb4806cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.731+0000 7fbb4d79b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbb4815ad80 0x7fbb480b0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.731+0000 7fbb4d79b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbb480add60 0x7fbb480b0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb4d79b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb480716d0 con 0x7fbb480add60 2023-12-19T18:14:54.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb4d79b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb480709e0 con 0x7fbb4815ad80 2023-12-19T18:14:54.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb4d79b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4806f5e0 con 0x7fbb4815a9a0 2023-12-19T18:14:54.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb4cf9a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbb480add60 0x7fbb480b0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb4cf9a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbb480add60 0x7fbb480b0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54512/0 (socket says 172.21.15.27:54512) 2023-12-19T18:14:54.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb4cf9a640 1 -- 172.21.15.27:0/3433723857 learned_addr learned my addr 172.21.15.27:0/3433723857 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:54.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb47fff640 1 --2- 172.21.15.27:0/3433723857 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb4815a9a0 0x7fbb4806cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb477fe640 1 --2- 172.21.15.27:0/3433723857 >> v2:172.21.15.192:3300/0 conn(0x7fbb4815ad80 0x7fbb480b0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb4cf9a640 1 -- 172.21.15.27:0/3433723857 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb4815a9a0 msgr2=0x7fbb4806cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:54.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb4cf9a640 1 --2- 172.21.15.27:0/3433723857 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb4815a9a0 0x7fbb4806cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb4cf9a640 1 -- 172.21.15.27:0/3433723857 >> v2:172.21.15.192:3300/0 conn(0x7fbb4815ad80 msgr2=0x7fbb480b0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:54.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb4cf9a640 1 --2- 172.21.15.27:0/3433723857 >> v2:172.21.15.192:3300/0 conn(0x7fbb4815ad80 0x7fbb480b0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb4cf9a640 1 -- 172.21.15.27:0/3433723857 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb48071240 con 0x7fbb480add60 2023-12-19T18:14:54.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb47fff640 1 --2- 172.21.15.27:0/3433723857 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb4815a9a0 0x7fbb4806cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:54.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.732+0000 7fbb477fe640 1 --2- 172.21.15.27:0/3433723857 >> v2:172.21.15.192:3300/0 conn(0x7fbb4815ad80 0x7fbb480b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:54.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.733+0000 7fbb4cf9a640 1 --2- 172.21.15.27:0/3433723857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb480add60 0x7fbb480b0a80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbb480ce710 tx=0x7fbb4002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da8566e4ee3a5dd5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:54.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.734+0000 7fbb46ffd640 1 -- 172.21.15.27:0/3433723857 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4002ca80 con 0x7fbb480add60 2023-12-19T18:14:54.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.734+0000 7fbb46ffd640 1 -- 172.21.15.27:0/3433723857 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4002cbe0 con 0x7fbb480add60 2023-12-19T18:14:54.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.734+0000 7fbb46ffd640 1 -- 172.21.15.27:0/3433723857 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb400343f0 con 0x7fbb480add60 2023-12-19T18:14:54.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.734+0000 7fbb4d79b640 1 -- 172.21.15.27:0/3433723857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb480add60 msgr2=0x7fbb480b0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:54.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.734+0000 7fbb4d79b640 1 --2- 172.21.15.27:0/3433723857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb480add60 0x7fbb480b0a80 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbb480ce710 tx=0x7fbb4002bee0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.734+0000 7fbb4d79b640 1 -- 172.21.15.27:0/3433723857 shutdown_connections 2023-12-19T18:14:54.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.734+0000 7fbb4d79b640 1 --2- 172.21.15.27:0/3433723857 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb480add60 0x7fbb480b0a80 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.734+0000 7fbb4d79b640 1 --2- 172.21.15.27:0/3433723857 >> v2:172.21.15.192:3300/0 conn(0x7fbb4815ad80 0x7fbb480b0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.734+0000 7fbb4d79b640 1 --2- 172.21.15.27:0/3433723857 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb4815a9a0 0x7fbb4806cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.734+0000 7fbb4d79b640 1 -- 172.21.15.27:0/3433723857 >> 172.21.15.27:0/3433723857 conn(0x7fbb480cb380 msgr2=0x7fbb480cb760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:54.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.734+0000 7fbb4d79b640 1 -- 172.21.15.27:0/3433723857 shutdown_connections 2023-12-19T18:14:54.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.734+0000 7fbb4d79b640 1 -- 172.21.15.27:0/3433723857 wait complete. 2023-12-19T18:14:54.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.736+0000 7fbb4d79b640 1 Processor -- start 2023-12-19T18:14:54.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.736+0000 7fbb4d79b640 1 -- start start 2023-12-19T18:14:54.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.737+0000 7fbb4d79b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb480add60 0x7fbb4814fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.737+0000 7fbb4d79b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb4815a9a0 0x7fbb480cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.737+0000 7fbb4d79b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb4815ad80 0x7fbb480ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.737+0000 7fbb4d79b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb480cd430 con 0x7fbb4815a9a0 2023-12-19T18:14:54.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.737+0000 7fbb4d79b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb480cd850 con 0x7fbb4815ad80 2023-12-19T18:14:54.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.737+0000 7fbb4d79b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb480ce4c0 con 0x7fbb480add60 2023-12-19T18:14:54.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.738+0000 7fbb477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb4815a9a0 0x7fbb480cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.738+0000 7fbb477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb4815a9a0 0x7fbb480cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54528/0 (socket says 172.21.15.27:54528) 2023-12-19T18:14:54.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.738+0000 7fbb477fe640 1 -- 172.21.15.27:0/2152800864 learned_addr learned my addr 172.21.15.27:0/2152800864 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:54.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.738+0000 7fbb4cf9a640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb4815ad80 0x7fbb480ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.738+0000 7fbb47fff640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb480add60 0x7fbb4814fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.738+0000 7fbb477fe640 1 -- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb480add60 msgr2=0x7fbb4814fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:54.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.738+0000 7fbb477fe640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb480add60 0x7fbb4814fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.738+0000 7fbb477fe640 1 -- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb4815ad80 msgr2=0x7fbb480ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:54.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.738+0000 7fbb477fe640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb4815ad80 0x7fbb480ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.738+0000 7fbb477fe640 1 -- 172.21.15.27:0/2152800864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb40009b30 con 0x7fbb4815a9a0 2023-12-19T18:14:54.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.738+0000 7fbb47fff640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb480add60 0x7fbb4814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:54.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.739+0000 7fbb477fe640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb4815a9a0 0x7fbb480cdf80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c7c0 tx=0x7fbb3c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:54.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.739+0000 7fbb44ff9640 1 -- 172.21.15.27:0/2152800864 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c007820 con 0x7fbb4815a9a0 2023-12-19T18:14:54.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.739+0000 7fbb44ff9640 1 -- 172.21.15.27:0/2152800864 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3c007980 con 0x7fbb4815a9a0 2023-12-19T18:14:54.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.739+0000 7fbb4d79b640 1 -- 172.21.15.27:0/2152800864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb480cc7b0 con 0x7fbb4815a9a0 2023-12-19T18:14:54.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.739+0000 7fbb44ff9640 1 -- 172.21.15.27:0/2152800864 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3c007d10 con 0x7fbb4815a9a0 2023-12-19T18:14:54.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.739+0000 7fbb4d79b640 1 -- 172.21.15.27:0/2152800864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb480ccd70 con 0x7fbb4815a9a0 2023-12-19T18:14:54.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.740+0000 7fbb4d79b640 1 -- 172.21.15.27:0/2152800864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb4804e840 con 0x7fbb4815a9a0 2023-12-19T18:14:54.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.741+0000 7fbb44ff9640 1 -- 172.21.15.27:0/2152800864 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbb3c0113f0 con 0x7fbb4815a9a0 2023-12-19T18:14:54.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.742+0000 7fbb44ff9640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb3406a430 0x7fbb3406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:54.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.742+0000 7fbb44ff9640 1 -- 172.21.15.27:0/2152800864 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb3c0888f0 con 0x7fbb4815a9a0 2023-12-19T18:14:54.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.743+0000 7fbb47fff640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb3406a430 0x7fbb3406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:54.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.744+0000 7fbb47fff640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb3406a430 0x7fbb3406c8f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbb38006fd0 tx=0x7fbb38008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:54.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.747+0000 7fbb44ff9640 1 -- 172.21.15.27:0/2152800864 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbb3c053f70 con 0x7fbb4815a9a0 2023-12-19T18:14:54.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.945+0000 7fbb4d79b640 1 -- 172.21.15.27:0/2152800864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb48154980 con 0x7fbb4815a9a0 2023-12-19T18:14:54.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.948+0000 7fbb44ff9640 1 -- 172.21.15.27:0/2152800864 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16054 (secure 0 0 0) 0x7fbb3c01f090 con 0x7fbb4815a9a0 2023-12-19T18:14:54.948 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:54.948 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":19,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:54.035054+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:54.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.950+0000 7fbb4d79b640 1 -- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb3406a430 msgr2=0x7fbb3406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:54.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.950+0000 7fbb4d79b640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb3406a430 0x7fbb3406c8f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbb38006fd0 tx=0x7fbb38008040 comp rx=0 tx=0).stop 2023-12-19T18:14:54.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.950+0000 7fbb4d79b640 1 -- 172.21.15.27:0/2152800864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb4815a9a0 msgr2=0x7fbb480cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:54.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.950+0000 7fbb4d79b640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb4815a9a0 0x7fbb480cdf80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fbb3c00c7c0 tx=0x7fbb3c00cc80 comp rx=0 tx=0).stop 2023-12-19T18:14:54.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.950+0000 7fbb4d79b640 1 -- 172.21.15.27:0/2152800864 shutdown_connections 2023-12-19T18:14:54.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.950+0000 7fbb4d79b640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb3406a430 0x7fbb3406c8f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.950+0000 7fbb4d79b640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb4815ad80 0x7fbb480ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.950+0000 7fbb4d79b640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb4815a9a0 0x7fbb480cdf80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.950+0000 7fbb4d79b640 1 --2- 172.21.15.27:0/2152800864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb480add60 0x7fbb4814fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:54.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.950+0000 7fbb4d79b640 1 -- 172.21.15.27:0/2152800864 >> 172.21.15.27:0/2152800864 conn(0x7fbb480cb380 msgr2=0x7fbb480b3750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:54.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.951+0000 7fbb4d79b640 1 -- 172.21.15.27:0/2152800864 shutdown_connections 2023-12-19T18:14:54.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:54.951+0000 7fbb4d79b640 1 -- 172.21.15.27:0/2152800864 wait complete. 2023-12-19T18:14:54.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:14:55.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.197+0000 7f545359e640 1 Processor -- start 2023-12-19T18:14:55.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.197+0000 7f545359e640 1 -- start start 2023-12-19T18:14:55.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.199+0000 7f545359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54540afdc0 0x7f54540cdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:55.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.199+0000 7f545359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54540b0be0 con 0x7f54540d0aa0 2023-12-19T18:14:55.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.199+0000 7f545359e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54540b0ec0 con 0x7f54540ce6e0 2023-12-19T18:14:55.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.199+0000 7f545359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54540b11a0 con 0x7f54540afdc0 2023-12-19T18:14:55.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.199+0000 7f545259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f54540d0aa0 0x7f54540d0e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35514/0 (socket says 172.21.15.27:35514) 2023-12-19T18:14:55.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.199+0000 7f545259c640 1 -- 172.21.15.27:0/2247710812 learned_addr learned my addr 172.21.15.27:0/2247710812 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:55.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.199+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3754715561 0 0) 0x7f54540b0be0 con 0x7f54540d0aa0 2023-12-19T18:14:55.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.199+0000 7f5451d9b640 1 --2- 172.21.15.27:0/2247710812 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54540afdc0 0x7f54540cdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:55.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.199+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5440003520 con 0x7f54540d0aa0 2023-12-19T18:14:55.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3963044991 0 0) 0x7f5440003520 con 0x7f54540d0aa0 2023-12-19T18:14:55.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54540afdc0 msgr2=0x7f54540cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:55.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545159a640 1 --2- 172.21.15.27:0/2247710812 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54540afdc0 0x7f54540cdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:55.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 >> v1:172.21.15.192:6789/0 conn(0x7f54540ce6e0 legacy=0x7f54540ceb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:55.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54540b1480 con 0x7f54540d0aa0 2023-12-19T18:14:55.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f543c001670 con 0x7f54540d0aa0 2023-12-19T18:14:55.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 >> v1:172.21.15.27:6789/0 conn(0x7f54540d0aa0 legacy=0x7f54540d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:55.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545159a640 1 --2- 172.21.15.27:0/2247710812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54400039a0 0x7f5440003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545159a640 1 --2- 172.21.15.27:0/2247710812 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54540afdc0 0x7f54400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545159a640 1 --2- 172.21.15.27:0/2247710812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5440004d00 0x7f5440005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54540b11a0 con 0x7f54400039a0 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544000bac0 con 0x7f5440004d00 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544000bc80 con 0x7f54540afdc0 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.200+0000 7f545259c640 1 --2- 172.21.15.27:0/2247710812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54400039a0 0x7f5440003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.201+0000 7f5451d9b640 1 --2- 172.21.15.27:0/2247710812 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54540afdc0 0x7f54400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.201+0000 7f5452d9d640 1 --2- 172.21.15.27:0/2247710812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5440004d00 0x7f5440005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.201+0000 7f545259c640 1 -- 172.21.15.27:0/2247710812 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54540afdc0 msgr2=0x7f54400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.201+0000 7f545259c640 1 --2- 172.21.15.27:0/2247710812 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54540afdc0 0x7f54400047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.201+0000 7f545259c640 1 -- 172.21.15.27:0/2247710812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5440004d00 msgr2=0x7f5440005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.201+0000 7f545259c640 1 --2- 172.21.15.27:0/2247710812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5440004d00 0x7f5440005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.201+0000 7f545259c640 1 -- 172.21.15.27:0/2247710812 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5440003520 con 0x7f54400039a0 2023-12-19T18:14:55.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.201+0000 7f5452d9d640 1 --2- 172.21.15.27:0/2247710812 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5440004d00 0x7f5440005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:55.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.202+0000 7f545259c640 1 --2- 172.21.15.27:0/2247710812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54400039a0 0x7f5440003dc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f543c00c0e0 tx=0x7f54400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f07c5e8aa857fc59 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:55.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.202+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c005ad0 con 0x7f54400039a0 2023-12-19T18:14:55.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.202+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543c005c30 con 0x7f54400039a0 2023-12-19T18:14:55.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.202+0000 7f545159a640 1 -- 172.21.15.27:0/2247710812 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c007070 con 0x7f54400039a0 2023-12-19T18:14:55.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.202+0000 7f545359e640 1 -- 172.21.15.27:0/2247710812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54400039a0 msgr2=0x7f5440003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:55.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.202+0000 7f545359e640 1 --2- 172.21.15.27:0/2247710812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54400039a0 0x7f5440003dc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f543c00c0e0 tx=0x7f54400057f0 comp rx=0 tx=0).stop 2023-12-19T18:14:55.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.202+0000 7f5452d9d640 1 -- 172.21.15.27:0/2247710812 reap_dead start 2023-12-19T18:14:55.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.203+0000 7f545359e640 1 -- 172.21.15.27:0/2247710812 shutdown_connections 2023-12-19T18:14:55.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.203+0000 7f545359e640 1 -- 172.21.15.27:0/2247710812 >> 172.21.15.27:0/2247710812 conn(0x7f5454099a50 msgr2=0x7f5454099e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:55.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.203+0000 7f545359e640 1 -- 172.21.15.27:0/2247710812 shutdown_connections 2023-12-19T18:14:55.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.203+0000 7f545359e640 1 -- 172.21.15.27:0/2247710812 wait complete. 2023-12-19T18:14:55.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.204+0000 7f545359e640 1 Processor -- start 2023-12-19T18:14:55.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.204+0000 7f545359e640 1 -- start start 2023-12-19T18:14:55.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.205+0000 7f545359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54400039a0 0x7f545415b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:55.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.205+0000 7f545359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54540c4640 0x7f54541570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:55.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.205+0000 7f545359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f545415bc60 0x7f545415c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:55.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.205+0000 7f545359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545415c620 con 0x7f54400039a0 2023-12-19T18:14:55.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.205+0000 7f545359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545415ba10 con 0x7f545415bc60 2023-12-19T18:14:55.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.205+0000 7f545359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54540c4c00 con 0x7f54540c4640 2023-12-19T18:14:55.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.206+0000 7f545259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54400039a0 0x7f545415b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:55.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.206+0000 7f545259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54400039a0 0x7f545415b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54544/0 (socket says 172.21.15.27:54544) 2023-12-19T18:14:55.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.206+0000 7f545259c640 1 -- 172.21.15.27:0/795765385 learned_addr learned my addr 172.21.15.27:0/795765385 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:55.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.206+0000 7f545259c640 1 -- 172.21.15.27:0/795765385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54540c4640 msgr2=0x7f54541570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:55.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.206+0000 7f545259c640 1 --2- 172.21.15.27:0/795765385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54540c4640 0x7f54541570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:55.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.206+0000 7f545259c640 1 -- 172.21.15.27:0/795765385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f545415bc60 msgr2=0x7f545415c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:55.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.206+0000 7f545259c640 1 --2- 172.21.15.27:0/795765385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f545415bc60 0x7f545415c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:55.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.206+0000 7f545259c640 1 -- 172.21.15.27:0/795765385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54540c4ee0 con 0x7f54400039a0 2023-12-19T18:14:55.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.206+0000 7f545259c640 1 --2- 172.21.15.27:0/795765385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54400039a0 0x7f545415b4d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f543c00c0e0 tx=0x7f543c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:55.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.206+0000 7f544affd640 1 -- 172.21.15.27:0/795765385 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c03c040 con 0x7f54400039a0 2023-12-19T18:14:55.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.206+0000 7f544affd640 1 -- 172.21.15.27:0/795765385 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f543c007500 con 0x7f54400039a0 2023-12-19T18:14:55.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.206+0000 7f544affd640 1 -- 172.21.15.27:0/795765385 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f543c02c740 con 0x7f54400039a0 2023-12-19T18:14:55.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.206+0000 7f545359e640 1 -- 172.21.15.27:0/795765385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54540c51c0 con 0x7f54400039a0 2023-12-19T18:14:55.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.207+0000 7f545359e640 1 -- 172.21.15.27:0/795765385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54540c5780 con 0x7f54400039a0 2023-12-19T18:14:55.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.208+0000 7f545359e640 1 -- 172.21.15.27:0/795765385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545404e840 con 0x7f54400039a0 2023-12-19T18:14:55.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.209+0000 7f544affd640 1 -- 172.21.15.27:0/795765385 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f543c032070 con 0x7f54400039a0 2023-12-19T18:14:55.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.210+0000 7f544affd640 1 --2- 172.21.15.27:0/795765385 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f543406a430 0x7f543406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:55.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.210+0000 7f544affd640 1 -- 172.21.15.27:0/795765385 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f543c0a9da0 con 0x7f54400039a0 2023-12-19T18:14:55.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.210+0000 7f5451d9b640 1 --2- 172.21.15.27:0/795765385 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f543406a430 0x7f543406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:55.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.211+0000 7f5451d9b640 1 --2- 172.21.15.27:0/795765385 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f543406a430 0x7f543406c8f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f545415d900 tx=0x7f5444009750 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:55.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.214+0000 7f544affd640 1 -- 172.21.15.27:0/795765385 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f543c075440 con 0x7f54400039a0 2023-12-19T18:14:55.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:55.446+0000 7f545359e640 1 -- 172.21.15.27:0/795765385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f5454157b70 con 0x7f54400039a0 2023-12-19T18:14:56.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.067+0000 7f544affd640 1 -- 172.21.15.27:0/795765385 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f543c0460b0 con 0x7f54400039a0 2023-12-19T18:14:56.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.070+0000 7f545359e640 1 -- 172.21.15.27:0/795765385 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f543406a430 msgr2=0x7f543406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:56.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.070+0000 7f545359e640 1 --2- 172.21.15.27:0/795765385 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f543406a430 0x7f543406c8f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f545415d900 tx=0x7f5444009750 comp rx=0 tx=0).stop 2023-12-19T18:14:56.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.070+0000 7f545359e640 1 -- 172.21.15.27:0/795765385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54400039a0 msgr2=0x7f545415b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:56.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.070+0000 7f545359e640 1 --2- 172.21.15.27:0/795765385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54400039a0 0x7f545415b4d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f543c00c0e0 tx=0x7f543c001930 comp rx=0 tx=0).stop 2023-12-19T18:14:56.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.070+0000 7f545359e640 1 -- 172.21.15.27:0/795765385 shutdown_connections 2023-12-19T18:14:56.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.071+0000 7f545359e640 1 --2- 172.21.15.27:0/795765385 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f543406a430 0x7f543406c8f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:56.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.071+0000 7f545359e640 1 --2- 172.21.15.27:0/795765385 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f545415bc60 0x7f545415c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:56.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.071+0000 7f545359e640 1 --2- 172.21.15.27:0/795765385 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54540c4640 0x7f54541570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:56.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.071+0000 7f545359e640 1 --2- 172.21.15.27:0/795765385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54400039a0 0x7f545415b4d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:56.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.071+0000 7f545359e640 1 -- 172.21.15.27:0/795765385 >> 172.21.15.27:0/795765385 conn(0x7f5454099a50 msgr2=0x7f545406e030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:56.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.071+0000 7f545359e640 1 -- 172.21.15.27:0/795765385 shutdown_connections 2023-12-19T18:14:56.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.071+0000 7f545359e640 1 -- 172.21.15.27:0/795765385 wait complete. 2023-12-19T18:14:56.090 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd', 'disabled']} 2023-12-19T18:14:56.090 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:14:56.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.321+0000 7fafb7fff640 1 Processor -- start 2023-12-19T18:14:56.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.321+0000 7fafb7fff640 1 -- start start 2023-12-19T18:14:56.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.323+0000 7fafb7fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fafb80d0aa0 0x7fafb80d0e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:56.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.323+0000 7fafb7fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb80d13c0 con 0x7fafb80afdc0 2023-12-19T18:14:56.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.323+0000 7fafb7fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafb80b0be0 con 0x7fafb80ce6e0 2023-12-19T18:14:56.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.323+0000 7fafb7fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafb80ce410 con 0x7fafb80d0aa0 2023-12-19T18:14:56.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.323+0000 7fafb67fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fafb80afdc0 0x7fafb80cdd00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35530/0 (socket says 172.21.15.27:35530) 2023-12-19T18:14:56.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.323+0000 7fafb67fc640 1 -- 172.21.15.27:0/1310262101 learned_addr learned my addr 172.21.15.27:0/1310262101 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:56.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.323+0000 7fafb6ffd640 1 --2- 172.21.15.27:0/1310262101 >> v2:172.21.15.192:3300/0 conn(0x7fafb80d0aa0 0x7fafb80d0e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:56.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806195088 0 0) 0x7fafb80d13c0 con 0x7fafb80afdc0 2023-12-19T18:14:56.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafa4003520 con 0x7fafb80afdc0 2023-12-19T18:14:56.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1306658631 0 0) 0x7fafa4003520 con 0x7fafb80afdc0 2023-12-19T18:14:56.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 >> v2:172.21.15.192:3300/0 conn(0x7fafb80d0aa0 msgr2=0x7fafb80d0e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:56.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 --2- 172.21.15.27:0/1310262101 >> v2:172.21.15.192:3300/0 conn(0x7fafb80d0aa0 0x7fafb80d0e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:56.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 >> v1:172.21.15.192:6789/0 conn(0x7fafb80ce6e0 legacy=0x7fafb80ceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:56.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb80b0ec0 con 0x7fafb80afdc0 2023-12-19T18:14:56.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fafa0001670 con 0x7fafb80afdc0 2023-12-19T18:14:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 >> v1:172.21.15.27:6789/0 conn(0x7fafb80afdc0 legacy=0x7fafb80cdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 --2- 172.21.15.27:0/1310262101 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa40039c0 0x7fafa4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 --2- 172.21.15.27:0/1310262101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa40047c0 0x7fafa4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 --2- 172.21.15.27:0/1310262101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa4005180 0x7fafa400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80ce410 con 0x7fafa40047c0 2023-12-19T18:14:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa400c080 con 0x7fafa4005180 2023-12-19T18:14:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafa400c240 con 0x7fafa40039c0 2023-12-19T18:14:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.324+0000 7fafb67fc640 1 --2- 172.21.15.27:0/1310262101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa40047c0 0x7fafa4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.325+0000 7fafb6ffd640 1 --2- 172.21.15.27:0/1310262101 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa40039c0 0x7fafa4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.325+0000 7fafb77fe640 1 --2- 172.21.15.27:0/1310262101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa4005180 0x7fafa400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.325+0000 7fafb67fc640 1 -- 172.21.15.27:0/1310262101 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa40039c0 msgr2=0x7fafa4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.325+0000 7fafb67fc640 1 --2- 172.21.15.27:0/1310262101 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa40039c0 0x7fafa4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:56.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.325+0000 7fafb67fc640 1 -- 172.21.15.27:0/1310262101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa4005180 msgr2=0x7fafa400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:56.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.325+0000 7fafb67fc640 1 --2- 172.21.15.27:0/1310262101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa4005180 0x7fafa400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:56.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.325+0000 7fafb67fc640 1 -- 172.21.15.27:0/1310262101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa4003520 con 0x7fafa40047c0 2023-12-19T18:14:56.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.325+0000 7fafb77fe640 1 --2- 172.21.15.27:0/1310262101 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafa4005180 0x7fafa400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:56.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.325+0000 7fafb77fe640 1 -- 172.21.15.27:0/1310262101 reap_dead start 2023-12-19T18:14:56.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.326+0000 7fafb67fc640 1 --2- 172.21.15.27:0/1310262101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa40047c0 0x7fafa4004c40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fafa000c0e0 tx=0x7fafa40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85ce9d5e6842ebf7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:56.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.326+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa0005ad0 con 0x7fafa40047c0 2023-12-19T18:14:56.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.326+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa0005c30 con 0x7fafa40047c0 2023-12-19T18:14:56.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.326+0000 7fafb5ffb640 1 -- 172.21.15.27:0/1310262101 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa002ca50 con 0x7fafa40047c0 2023-12-19T18:14:56.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.327+0000 7fafb7fff640 1 -- 172.21.15.27:0/1310262101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa40047c0 msgr2=0x7fafa4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:56.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.327+0000 7fafb7fff640 1 --2- 172.21.15.27:0/1310262101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa40047c0 0x7fafa4004c40 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fafa000c0e0 tx=0x7fafa40057f0 comp rx=0 tx=0).stop 2023-12-19T18:14:56.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.327+0000 7fafb7fff640 1 -- 172.21.15.27:0/1310262101 shutdown_connections 2023-12-19T18:14:56.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.327+0000 7fafb7fff640 1 --2- 172.21.15.27:0/1310262101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa40047c0 0x7fafa4004c40 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:56.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.327+0000 7fafb7fff640 1 -- 172.21.15.27:0/1310262101 >> 172.21.15.27:0/1310262101 conn(0x7fafb8099a50 msgr2=0x7fafb8099e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:56.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.327+0000 7fafb7fff640 1 -- 172.21.15.27:0/1310262101 shutdown_connections 2023-12-19T18:14:56.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.327+0000 7fafb7fff640 1 -- 172.21.15.27:0/1310262101 wait complete. 2023-12-19T18:14:56.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.329+0000 7fafb7fff640 1 Processor -- start 2023-12-19T18:14:56.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.329+0000 7fafb7fff640 1 -- start start 2023-12-19T18:14:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.330+0000 7fafb7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa40047c0 0x7fafb814ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.330+0000 7fafb7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb80c56e0 0x7fafb80c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.330+0000 7fafb7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb80c5b60 0x7fafb80c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.330+0000 7fafb7fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb815af20 con 0x7fafb80c56e0 2023-12-19T18:14:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.330+0000 7fafb7fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80c6080 con 0x7fafb80c5b60 2023-12-19T18:14:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.330+0000 7fafb7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafb80c6ee0 con 0x7fafa40047c0 2023-12-19T18:14:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.330+0000 7fafb67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb80c56e0 0x7fafb80c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:56.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.331+0000 7fafb67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb80c56e0 0x7fafb80c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54562/0 (socket says 172.21.15.27:54562) 2023-12-19T18:14:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.331+0000 7fafb67fc640 1 -- 172.21.15.27:0/1653777529 learned_addr learned my addr 172.21.15.27:0/1653777529 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.331+0000 7fafb6ffd640 1 --2- 172.21.15.27:0/1653777529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa40047c0 0x7fafb814ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.331+0000 7fafb67fc640 1 -- 172.21.15.27:0/1653777529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa40047c0 msgr2=0x7fafb814ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.331+0000 7fafb67fc640 1 --2- 172.21.15.27:0/1653777529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa40047c0 0x7fafb814ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.331+0000 7fafb67fc640 1 -- 172.21.15.27:0/1653777529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb80c5b60 msgr2=0x7fafb80c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.331+0000 7fafb67fc640 1 --2- 172.21.15.27:0/1653777529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb80c5b60 0x7fafb80c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.331+0000 7fafb67fc640 1 -- 172.21.15.27:0/1653777529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb80c5490 con 0x7fafb80c56e0 2023-12-19T18:14:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.331+0000 7fafb67fc640 1 --2- 172.21.15.27:0/1653777529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb80c56e0 0x7fafb80c4640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fafa0000c00 tx=0x7fafa002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:56.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.331+0000 7fafaffff640 1 -- 172.21.15.27:0/1653777529 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa003c040 con 0x7fafb80c56e0 2023-12-19T18:14:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.332+0000 7fafaffff640 1 -- 172.21.15.27:0/1653777529 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa002cd00 con 0x7fafb80c56e0 2023-12-19T18:14:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.332+0000 7fafb7fff640 1 -- 172.21.15.27:0/1653777529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb80c62d0 con 0x7fafb80c56e0 2023-12-19T18:14:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.332+0000 7fafaffff640 1 -- 172.21.15.27:0/1653777529 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa00072b0 con 0x7fafb80c56e0 2023-12-19T18:14:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.332+0000 7fafb7fff640 1 -- 172.21.15.27:0/1653777529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb80c6890 con 0x7fafb80c56e0 2023-12-19T18:14:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.333+0000 7fafb7fff640 1 -- 172.21.15.27:0/1653777529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb804e840 con 0x7fafb80c56e0 2023-12-19T18:14:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.334+0000 7fafaffff640 1 -- 172.21.15.27:0/1653777529 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fafa0032070 con 0x7fafb80c56e0 2023-12-19T18:14:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.335+0000 7fafaffff640 1 --2- 172.21.15.27:0/1653777529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf9806a430 0x7faf9806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.335+0000 7fafaffff640 1 -- 172.21.15.27:0/1653777529 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fafa00a9f70 con 0x7fafb80c56e0 2023-12-19T18:14:56.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.335+0000 7fafb6ffd640 1 --2- 172.21.15.27:0/1653777529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf9806a430 0x7faf9806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:56.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.337+0000 7fafb6ffd640 1 --2- 172.21.15.27:0/1653777529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf9806a430 0x7faf9806c8f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fafa8007380 tx=0x7fafa800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:56.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.339+0000 7fafaffff640 1 -- 172.21.15.27:0/1653777529 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fafa00754c0 con 0x7fafb80c56e0 2023-12-19T18:14:56.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:56.574+0000 7fafb7fff640 1 -- 172.21.15.27:0/1653777529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fafb80cafd0 con 0x7fafb80c56e0 2023-12-19T18:14:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.553+0000 7fafaffff640 1 -- 172.21.15.27:0/1653777529 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7fafa0078ae0 con 0x7fafb80c56e0 2023-12-19T18:14:57.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.557+0000 7fafb7fff640 1 -- 172.21.15.27:0/1653777529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf9806a430 msgr2=0x7faf9806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:57.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.557+0000 7fafb7fff640 1 --2- 172.21.15.27:0/1653777529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf9806a430 0x7faf9806c8f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fafa8007380 tx=0x7fafa800b040 comp rx=0 tx=0).stop 2023-12-19T18:14:57.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.557+0000 7fafb7fff640 1 -- 172.21.15.27:0/1653777529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb80c56e0 msgr2=0x7fafb80c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:57.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.557+0000 7fafb7fff640 1 --2- 172.21.15.27:0/1653777529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb80c56e0 0x7fafb80c4640 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fafa0000c00 tx=0x7fafa002e070 comp rx=0 tx=0).stop 2023-12-19T18:14:57.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.557+0000 7fafb7fff640 1 -- 172.21.15.27:0/1653777529 shutdown_connections 2023-12-19T18:14:57.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.557+0000 7fafb7fff640 1 --2- 172.21.15.27:0/1653777529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf9806a430 0x7faf9806c8f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:57.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.557+0000 7fafb7fff640 1 --2- 172.21.15.27:0/1653777529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fafb80c5b60 0x7fafb80c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:57.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.557+0000 7fafb7fff640 1 --2- 172.21.15.27:0/1653777529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb80c56e0 0x7fafb80c4640 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:57.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.557+0000 7fafb7fff640 1 --2- 172.21.15.27:0/1653777529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fafa40047c0 0x7fafb814ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:57.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.557+0000 7fafb7fff640 1 -- 172.21.15.27:0/1653777529 >> 172.21.15.27:0/1653777529 conn(0x7fafb8099a50 msgr2=0x7fafb80cc7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:57.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.557+0000 7fafb7fff640 1 -- 172.21.15.27:0/1653777529 shutdown_connections 2023-12-19T18:14:57.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.558+0000 7fafb7fff640 1 -- 172.21.15.27:0/1653777529 wait complete. 2023-12-19T18:14:57.579 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:14:57.580 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:14:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.813+0000 7fb140316640 1 Processor -- start 2023-12-19T18:14:57.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.813+0000 7fb140316640 1 -- start start 2023-12-19T18:14:57.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.814+0000 7fb140316640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb138159c40 0x7fb13806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:57.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.814+0000 7fb140316640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb13806c880 0x7fb13806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:57.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.814+0000 7fb140316640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb13815b180 con 0x7fb13815a020 2023-12-19T18:14:57.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.814+0000 7fb140316640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1380ca850 con 0x7fb13806c880 2023-12-19T18:14:57.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.814+0000 7fb140316640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13815a570 con 0x7fb138159c40 2023-12-19T18:14:57.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.815+0000 7fb13d8b1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb13806c880 0x7fb13806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:57.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.815+0000 7fb13e0b2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb138159c40 0x7fb13806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:57.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.815+0000 7fb13e0b2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb138159c40 0x7fb13806b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:39540/0 (socket says 172.21.15.27:39540) 2023-12-19T18:14:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.815+0000 7fb13d8b1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb13806c880 0x7fb13806ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:44744/0 (socket says 172.21.15.27:44744) 2023-12-19T18:14:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.815+0000 7fb13e0b2640 1 -- 172.21.15.27:0/3815753941 learned_addr learned my addr 172.21.15.27:0/3815753941 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.816+0000 7fb13d8b1640 1 -- 172.21.15.27:0/3815753941 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb138159c40 msgr2=0x7fb13806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.816+0000 7fb13d8b1640 1 --2- 172.21.15.27:0/3815753941 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb138159c40 0x7fb13806b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.816+0000 7fb13d8b1640 1 -- 172.21.15.27:0/3815753941 >> v1:172.21.15.192:6789/0 conn(0x7fb13815a020 legacy=0x7fb1380cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:57.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.816+0000 7fb13d8b1640 1 -- 172.21.15.27:0/3815753941 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb13815a850 con 0x7fb13806c880 2023-12-19T18:14:57.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.817+0000 7fb13d8b1640 1 --2- 172.21.15.27:0/3815753941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb13806c880 0x7fb13806ba20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb12400ba40 tx=0x7fb12402e080 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5a1b1219e6515dd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:57.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.818+0000 7fb13d0b0640 1 -- 172.21.15.27:0/3815753941 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12402ee30 con 0x7fb13806c880 2023-12-19T18:14:57.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.818+0000 7fb13d0b0640 1 -- 172.21.15.27:0/3815753941 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb124036440 con 0x7fb13806c880 2023-12-19T18:14:57.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.818+0000 7fb13d0b0640 1 -- 172.21.15.27:0/3815753941 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1240367d0 con 0x7fb13806c880 2023-12-19T18:14:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.819+0000 7fb140316640 1 -- 172.21.15.27:0/3815753941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb13806c880 msgr2=0x7fb13806ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:57.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.819+0000 7fb140316640 1 --2- 172.21.15.27:0/3815753941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb13806c880 0x7fb13806ba20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb12400ba40 tx=0x7fb12402e080 comp rx=0 tx=0).stop 2023-12-19T18:14:57.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.819+0000 7fb140316640 1 -- 172.21.15.27:0/3815753941 shutdown_connections 2023-12-19T18:14:57.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.819+0000 7fb140316640 1 --2- 172.21.15.27:0/3815753941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb13806c880 0x7fb13806ba20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:57.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.819+0000 7fb140316640 1 --2- 172.21.15.27:0/3815753941 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb138159c40 0x7fb13806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:57.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.819+0000 7fb140316640 1 -- 172.21.15.27:0/3815753941 >> 172.21.15.27:0/3815753941 conn(0x7fb13806a330 msgr2=0x7fb1380b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:57.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.819+0000 7fb140316640 1 -- 172.21.15.27:0/3815753941 shutdown_connections 2023-12-19T18:14:57.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.819+0000 7fb140316640 1 -- 172.21.15.27:0/3815753941 wait complete. 2023-12-19T18:14:57.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.821+0000 7fb140316640 1 Processor -- start 2023-12-19T18:14:57.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.821+0000 7fb140316640 1 -- start start 2023-12-19T18:14:57.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.822+0000 7fb140316640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb13806c880 0x7fb138153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:57.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.822+0000 7fb140316640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb138159c40 0x7fb1380b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:57.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.822+0000 7fb140316640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb13815a020 0x7fb1380ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:57.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.822+0000 7fb140316640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb138153cd0 con 0x7fb13806c880 2023-12-19T18:14:57.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.822+0000 7fb140316640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1380b2b40 con 0x7fb138159c40 2023-12-19T18:14:57.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.822+0000 7fb140316640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1380b1f30 con 0x7fb13815a020 2023-12-19T18:14:57.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.822+0000 7fb13e0b2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb13806c880 0x7fb138153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:57.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.822+0000 7fb13e0b2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb13806c880 0x7fb138153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54564/0 (socket says 172.21.15.27:54564) 2023-12-19T18:14:57.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.822+0000 7fb13e0b2640 1 -- 172.21.15.27:0/3735455123 learned_addr learned my addr 172.21.15.27:0/3735455123 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:57.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.822+0000 7fb13d8b1640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb138159c40 0x7fb1380b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:57.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.822+0000 7fb13e8b3640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb13815a020 0x7fb1380ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:57.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.823+0000 7fb13e0b2640 1 -- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb13815a020 msgr2=0x7fb1380ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:57.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.823+0000 7fb13e0b2640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb13815a020 0x7fb1380ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:57.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.823+0000 7fb13e0b2640 1 -- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb138159c40 msgr2=0x7fb1380b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:57.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.823+0000 7fb13e0b2640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb138159c40 0x7fb1380b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:57.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.823+0000 7fb13e0b2640 1 -- 172.21.15.27:0/3735455123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12400b6a0 con 0x7fb13806c880 2023-12-19T18:14:57.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.823+0000 7fb13e0b2640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb13806c880 0x7fb138153790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00efc0 tx=0x7fb12c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:57.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.823+0000 7fb136ffd640 1 -- 172.21.15.27:0/3735455123 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c00cda0 con 0x7fb13806c880 2023-12-19T18:14:57.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.823+0000 7fb136ffd640 1 -- 172.21.15.27:0/3735455123 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c00f040 con 0x7fb13806c880 2023-12-19T18:14:57.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.823+0000 7fb136ffd640 1 -- 172.21.15.27:0/3735455123 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c014680 con 0x7fb13806c880 2023-12-19T18:14:57.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.823+0000 7fb140316640 1 -- 172.21.15.27:0/3735455123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1380b2210 con 0x7fb13806c880 2023-12-19T18:14:57.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.824+0000 7fb140316640 1 -- 172.21.15.27:0/3735455123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1380b27d0 con 0x7fb13806c880 2023-12-19T18:14:57.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.825+0000 7fb140316640 1 -- 172.21.15.27:0/3735455123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1380b2960 con 0x7fb13806c880 2023-12-19T18:14:57.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.826+0000 7fb136ffd640 1 -- 172.21.15.27:0/3735455123 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb12c0147e0 con 0x7fb13806c880 2023-12-19T18:14:57.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.827+0000 7fb136ffd640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb11c06a460 0x7fb11c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:57.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.827+0000 7fb136ffd640 1 -- 172.21.15.27:0/3735455123 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb12c0890c0 con 0x7fb13806c880 2023-12-19T18:14:57.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.827+0000 7fb13d8b1640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb11c06a460 0x7fb11c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:57.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.828+0000 7fb13d8b1640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb11c06a460 0x7fb11c06c920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb124038000 tx=0x7fb124008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:57.831+0000 7fb136ffd640 1 -- 172.21.15.27:0/3735455123 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb12c0546a0 con 0x7fb13806c880 2023-12-19T18:14:58.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.061+0000 7fb140316640 1 -- 172.21.15.27:0/3735455123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb138068e90 con 0x7fb13806c880 2023-12-19T18:14:58.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.062+0000 7fb136ffd640 1 -- 172.21.15.27:0/3735455123 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fb12c057cc0 con 0x7fb13806c880 2023-12-19T18:14:58.063 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:58.063 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3679393226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3679393226},{"type":"v1","addr":"172.21.15.27:6835","nonce":3679393226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/809712436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":809712436},{"type":"v1","addr":"172.21.15.192:6835","nonce":809712436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3400689212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3400689212},{"type":"v1","addr":"172.21.15.27:6833","nonce":3400689212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:54.035577+0000","modified":"2023-12-19T18:14:57.547613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/1878426383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1878426383},{"type":"v1","addr":"172.21.15.192:6837","nonce":1878426383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T18:14:58.063 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2023-12-19T18:14:58.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.065+0000 7fb140316640 1 -- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb11c06a460 msgr2=0x7fb11c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.065+0000 7fb140316640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb11c06a460 0x7fb11c06c920 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb124038000 tx=0x7fb124008040 comp rx=0 tx=0).stop 2023-12-19T18:14:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.065+0000 7fb140316640 1 -- 172.21.15.27:0/3735455123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb13806c880 msgr2=0x7fb138153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.065+0000 7fb140316640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb13806c880 0x7fb138153790 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00efc0 tx=0x7fb12c00c490 comp rx=0 tx=0).stop 2023-12-19T18:14:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.065+0000 7fb140316640 1 -- 172.21.15.27:0/3735455123 shutdown_connections 2023-12-19T18:14:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.065+0000 7fb140316640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb11c06a460 0x7fb11c06c920 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.065+0000 7fb140316640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb13815a020 0x7fb1380ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.065+0000 7fb140316640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb138159c40 0x7fb1380b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.065+0000 7fb140316640 1 --2- 172.21.15.27:0/3735455123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb13806c880 0x7fb138153790 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.065+0000 7fb140316640 1 -- 172.21.15.27:0/3735455123 >> 172.21.15.27:0/3735455123 conn(0x7fb13806a330 msgr2=0x7fb13815d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.065+0000 7fb140316640 1 -- 172.21.15.27:0/3735455123 shutdown_connections 2023-12-19T18:14:58.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.065+0000 7fb140316640 1 -- 172.21.15.27:0/3735455123 wait complete. 2023-12-19T18:14:58.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:14:58.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.318+0000 7f96854b4640 1 Processor -- start 2023-12-19T18:14:58.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.318+0000 7f96854b4640 1 -- start start 2023-12-19T18:14:58.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.320+0000 7f96854b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f968015ee30 0x7f96800c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.320+0000 7f96854b4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f96800afeb0 0x7f96800c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.320+0000 7f96854b4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96800b0320 con 0x7f96801553a0 2023-12-19T18:14:58.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.320+0000 7f96854b4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96800cbcf0 con 0x7f96800afeb0 2023-12-19T18:14:58.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.321+0000 7f96854b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96800cb350 con 0x7f968015ee30 2023-12-19T18:14:58.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.321+0000 7f9684cb3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f96800afeb0 0x7f96800c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.321+0000 7f967ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f96801553a0 0x7f968015e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35536/0 (socket says 172.21.15.27:35536) 2023-12-19T18:14:58.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.321+0000 7f967ffff640 1 -- 172.21.15.27:0/1521824565 learned_addr learned my addr 172.21.15.27:0/1521824565 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:58.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.321+0000 7f96777fe640 1 --2- 172.21.15.27:0/1521824565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f968015ee30 0x7f96800c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.321+0000 7f9684cb3640 1 -- 172.21.15.27:0/1521824565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f968015ee30 msgr2=0x7f96800c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:58.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.321+0000 7f9684cb3640 1 --2- 172.21.15.27:0/1521824565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f968015ee30 0x7f96800c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.321+0000 7f9684cb3640 1 -- 172.21.15.27:0/1521824565 >> v1:172.21.15.27:6789/0 conn(0x7f96801553a0 legacy=0x7f968015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:58.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.322+0000 7f9684cb3640 1 -- 172.21.15.27:0/1521824565 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96800cb630 con 0x7f96800afeb0 2023-12-19T18:14:58.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.321+0000 7f967f7fe640 1 -- 172.21.15.27:0/1521824565 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391698850 0 0) 0x7f96800b0320 con 0x7f96801553a0 2023-12-19T18:14:58.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.323+0000 7f9684cb3640 1 --2- 172.21.15.27:0/1521824565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96800afeb0 0x7f96800c39d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96780097a0 tx=0x7f967802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=264e914a470f87fb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:58.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.323+0000 7f967f7fe640 1 -- 172.21.15.27:0/1521824565 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967802e070 con 0x7f96800afeb0 2023-12-19T18:14:58.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.323+0000 7f967f7fe640 1 -- 172.21.15.27:0/1521824565 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96780315a0 con 0x7f96800afeb0 2023-12-19T18:14:58.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.323+0000 7f967f7fe640 1 -- 172.21.15.27:0/1521824565 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9678031930 con 0x7f96800afeb0 2023-12-19T18:14:58.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.323+0000 7f96854b4640 1 -- 172.21.15.27:0/1521824565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96800afeb0 msgr2=0x7f96800c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:58.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.323+0000 7f96854b4640 1 --2- 172.21.15.27:0/1521824565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96800afeb0 0x7f96800c39d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f96780097a0 tx=0x7f967802b840 comp rx=0 tx=0).stop 2023-12-19T18:14:58.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.324+0000 7f96854b4640 1 -- 172.21.15.27:0/1521824565 shutdown_connections 2023-12-19T18:14:58.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.324+0000 7f96854b4640 1 --2- 172.21.15.27:0/1521824565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96800afeb0 0x7f96800c39d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.324+0000 7f96854b4640 1 --2- 172.21.15.27:0/1521824565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f968015ee30 0x7f96800c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.324+0000 7f96854b4640 1 -- 172.21.15.27:0/1521824565 >> 172.21.15.27:0/1521824565 conn(0x7f9680069c70 msgr2=0x7f968006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:58.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.324+0000 7f96854b4640 1 -- 172.21.15.27:0/1521824565 shutdown_connections 2023-12-19T18:14:58.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.324+0000 7f96854b4640 1 -- 172.21.15.27:0/1521824565 wait complete. 2023-12-19T18:14:58.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.326+0000 7f96854b4640 1 Processor -- start 2023-12-19T18:14:58.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.326+0000 7f96854b4640 1 -- start start 2023-12-19T18:14:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f96854b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96800afeb0 0x7f96800b2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f96854b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96801553a0 0x7f96800b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f96854b4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f968015ee30 0x7f96800adb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f96854b4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96800b47e0 con 0x7f96800afeb0 2023-12-19T18:14:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f96854b4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f968015cd30 con 0x7f968015ee30 2023-12-19T18:14:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f96854b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968015c400 con 0x7f96801553a0 2023-12-19T18:14:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f967ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96800afeb0 0x7f96800b2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f967ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96800afeb0 0x7f96800b2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54590/0 (socket says 172.21.15.27:54590) 2023-12-19T18:14:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f967ffff640 1 -- 172.21.15.27:0/2684205615 learned_addr learned my addr 172.21.15.27:0/2684205615 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f96777fe640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96801553a0 0x7f96800b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f9684cb3640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f968015ee30 0x7f96800adb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f967ffff640 1 -- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96801553a0 msgr2=0x7f96800b42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:58.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f967ffff640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96801553a0 0x7f96800b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f967ffff640 1 -- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f968015ee30 msgr2=0x7f96800adb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:58.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.327+0000 7f967ffff640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f968015ee30 0x7f96800adb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.328+0000 7f967ffff640 1 -- 172.21.15.27:0/2684205615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9678009400 con 0x7f96800afeb0 2023-12-19T18:14:58.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.328+0000 7f967ffff640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96800afeb0 0x7f96800b2490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f967000c890 tx=0x7f967000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:58.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.328+0000 7f967d7fa640 1 -- 172.21.15.27:0/2684205615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967000a640 con 0x7f96800afeb0 2023-12-19T18:14:58.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.328+0000 7f967d7fa640 1 -- 172.21.15.27:0/2684205615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f967000a7c0 con 0x7f96800afeb0 2023-12-19T18:14:58.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.328+0000 7f96854b4640 1 -- 172.21.15.27:0/2684205615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f968015c6e0 con 0x7f96800afeb0 2023-12-19T18:14:58.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.328+0000 7f967d7fa640 1 -- 172.21.15.27:0/2684205615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f967000ab70 con 0x7f96800afeb0 2023-12-19T18:14:58.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.328+0000 7f96854b4640 1 -- 172.21.15.27:0/2684205615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96800b2b70 con 0x7f96800afeb0 2023-12-19T18:14:58.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.330+0000 7f96854b4640 1 -- 172.21.15.27:0/2684205615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f968004e840 con 0x7f96800afeb0 2023-12-19T18:14:58.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.331+0000 7f967d7fa640 1 -- 172.21.15.27:0/2684205615 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f967000acd0 con 0x7f96800afeb0 2023-12-19T18:14:58.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.331+0000 7f967d7fa640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f966006a430 0x7f966006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.332+0000 7f967d7fa640 1 -- 172.21.15.27:0/2684205615 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96700881f0 con 0x7f96800afeb0 2023-12-19T18:14:58.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.332+0000 7f96777fe640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f966006a430 0x7f966006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.333+0000 7f96777fe640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f966006a430 0x7f966006c8f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9668006fd0 tx=0x7f9668008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:58.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.336+0000 7f967d7fa640 1 -- 172.21.15.27:0/2684205615 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f96700537d0 con 0x7f96800afeb0 2023-12-19T18:14:58.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.531+0000 7f96854b4640 1 -- 172.21.15.27:0/2684205615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96800af6a0 con 0x7f96800afeb0 2023-12-19T18:14:58.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.534+0000 7f967d7fa640 1 -- 172.21.15.27:0/2684205615 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f9670056df0 con 0x7f96800afeb0 2023-12-19T18:14:58.535 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:58.535 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:55.039690+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:14:58.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.537+0000 7f96854b4640 1 -- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f966006a430 msgr2=0x7f966006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.537+0000 7f96854b4640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f966006a430 0x7f966006c8f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f9668006fd0 tx=0x7f9668008040 comp rx=0 tx=0).stop 2023-12-19T18:14:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.537+0000 7f96854b4640 1 -- 172.21.15.27:0/2684205615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96800afeb0 msgr2=0x7f96800b2490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.537+0000 7f96854b4640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96800afeb0 0x7f96800b2490 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f967000c890 tx=0x7f967000cd50 comp rx=0 tx=0).stop 2023-12-19T18:14:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.537+0000 7f96854b4640 1 -- 172.21.15.27:0/2684205615 shutdown_connections 2023-12-19T18:14:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.537+0000 7f96854b4640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f966006a430 0x7f966006c8f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.537+0000 7f96854b4640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f968015ee30 0x7f96800adb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.537+0000 7f96854b4640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96801553a0 0x7f96800b42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.537+0000 7f96854b4640 1 --2- 172.21.15.27:0/2684205615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96800afeb0 0x7f96800b2490 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.537+0000 7f96854b4640 1 -- 172.21.15.27:0/2684205615 >> 172.21.15.27:0/2684205615 conn(0x7f9680069c70 msgr2=0x7f96800cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.538+0000 7f96854b4640 1 -- 172.21.15.27:0/2684205615 shutdown_connections 2023-12-19T18:14:58.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.538+0000 7f96854b4640 1 -- 172.21.15.27:0/2684205615 wait complete. 2023-12-19T18:14:58.556 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:14:58.556 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:14:58.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.774+0000 7f1f81aaf640 1 Processor -- start 2023-12-19T18:14:58.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.774+0000 7f1f81aaf640 1 -- start start 2023-12-19T18:14:58.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.776+0000 7f1f81aaf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1f7c0af630 0x7f1f7c0afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.776+0000 7f1f81aaf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c0cc190 con 0x7f1f7c154030 2023-12-19T18:14:58.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.776+0000 7f1f81aaf640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c0cb2a0 con 0x7f1f7c0c4550 2023-12-19T18:14:58.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.776+0000 7f1f81aaf640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7c0b0000 con 0x7f1f7c0af630 2023-12-19T18:14:58.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.776+0000 7f1f80aad640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1f7c154030 0x7f1f7c0cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:35550/0 (socket says 172.21.15.27:35550) 2023-12-19T18:14:58.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.776+0000 7f1f80aad640 1 -- 172.21.15.27:0/1914100362 learned_addr learned my addr 172.21.15.27:0/1914100362 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:14:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.776+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059233501 0 0) 0x7f1f7c0cc190 con 0x7f1f7c154030 2023-12-19T18:14:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7bfff640 1 --2- 172.21.15.27:0/1914100362 >> v2:172.21.15.192:3300/0 conn(0x7f1f7c0af630 0x7f1f7c0afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f6c005730 con 0x7f1f7c154030 2023-12-19T18:14:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1899558383 0 0) 0x7f1f6c005730 con 0x7f1f7c154030 2023-12-19T18:14:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 >> v2:172.21.15.192:3300/0 conn(0x7f1f7c0af630 msgr2=0x7f1f7c0afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 --2- 172.21.15.27:0/1914100362 >> v2:172.21.15.192:3300/0 conn(0x7f1f7c0af630 0x7f1f7c0afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 >> v1:172.21.15.192:6789/0 conn(0x7f1f7c0c4550 legacy=0x7f1f7c0c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c0cb580 con 0x7f1f7c154030 2023-12-19T18:14:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f1f70001670 con 0x7f1f7c154030 2023-12-19T18:14:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 >> v1:172.21.15.27:6789/0 conn(0x7f1f7c154030 legacy=0x7f1f7c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 --2- 172.21.15.27:0/1914100362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f6c005bd0 0x7f1f6c005fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 --2- 172.21.15.27:0/1914100362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f6c0069d0 0x7f1f6c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 --2- 172.21.15.27:0/1914100362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f6c007390 0x7f1f6c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0b0000 con 0x7f1f6c005bd0 2023-12-19T18:14:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c00e260 con 0x7f1f6c0069d0 2023-12-19T18:14:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6c00e420 con 0x7f1f6c007390 2023-12-19T18:14:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.777+0000 7f1f80aad640 1 --2- 172.21.15.27:0/1914100362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f6c005bd0 0x7f1f6c005fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.778+0000 7f1f7bfff640 1 --2- 172.21.15.27:0/1914100362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f6c0069d0 0x7f1f6c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.778+0000 7f1f80aad640 1 -- 172.21.15.27:0/1914100362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f6c007390 msgr2=0x7f1f6c00db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.778+0000 7f1f812ae640 1 --2- 172.21.15.27:0/1914100362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f6c007390 0x7f1f6c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.778+0000 7f1f80aad640 1 --2- 172.21.15.27:0/1914100362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f6c007390 0x7f1f6c00db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.778+0000 7f1f80aad640 1 -- 172.21.15.27:0/1914100362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f6c0069d0 msgr2=0x7f1f6c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.778+0000 7f1f80aad640 1 --2- 172.21.15.27:0/1914100362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f6c0069d0 0x7f1f6c006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.778+0000 7f1f80aad640 1 -- 172.21.15.27:0/1914100362 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c005730 con 0x7f1f6c005bd0 2023-12-19T18:14:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.778+0000 7f1f7bfff640 1 --2- 172.21.15.27:0/1914100362 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f6c0069d0 0x7f1f6c006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.778+0000 7f1f812ae640 1 --2- 172.21.15.27:0/1914100362 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f6c007390 0x7f1f6c00db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:14:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.778+0000 7f1f812ae640 1 -- 172.21.15.27:0/1914100362 reap_dead start 2023-12-19T18:14:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.779+0000 7f1f80aad640 1 --2- 172.21.15.27:0/1914100362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f6c005bd0 0x7f1f6c005fd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000c190 tx=0x7f1f6c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca72bdeebc5f5297 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.779+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70005ad0 con 0x7f1f6c005bd0 2023-12-19T18:14:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.779+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f70005c30 con 0x7f1f6c005bd0 2023-12-19T18:14:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.779+0000 7f1f7b7fe640 1 -- 172.21.15.27:0/1914100362 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7002c830 con 0x7f1f6c005bd0 2023-12-19T18:14:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.780+0000 7f1f81aaf640 1 -- 172.21.15.27:0/1914100362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f6c005bd0 msgr2=0x7f1f6c005fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.780+0000 7f1f81aaf640 1 --2- 172.21.15.27:0/1914100362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f6c005bd0 0x7f1f6c005fd0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1f7000c190 tx=0x7f1f6c007a00 comp rx=0 tx=0).stop 2023-12-19T18:14:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.780+0000 7f1f81aaf640 1 -- 172.21.15.27:0/1914100362 shutdown_connections 2023-12-19T18:14:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.780+0000 7f1f81aaf640 1 --2- 172.21.15.27:0/1914100362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f6c005bd0 0x7f1f6c005fd0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.780+0000 7f1f81aaf640 1 -- 172.21.15.27:0/1914100362 >> 172.21.15.27:0/1914100362 conn(0x7f1f7c069c70 msgr2=0x7f1f7c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.780+0000 7f1f81aaf640 1 -- 172.21.15.27:0/1914100362 shutdown_connections 2023-12-19T18:14:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.780+0000 7f1f81aaf640 1 -- 172.21.15.27:0/1914100362 wait complete. 2023-12-19T18:14:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.782+0000 7f1f81aaf640 1 Processor -- start 2023-12-19T18:14:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.782+0000 7f1f81aaf640 1 -- start start 2023-12-19T18:14:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.783+0000 7f1f81aaf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f6c005bd0 0x7f1f7c1478d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.783+0000 7f1f81aaf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f7c148180 0x7f1f7c0c74c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.783+0000 7f1f81aaf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f7c14e4d0 0x7f1f7c14e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.783+0000 7f1f81aaf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c14ee90 con 0x7f1f7c14e4d0 2023-12-19T18:14:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.783+0000 7f1f81aaf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0c7da0 con 0x7f1f6c005bd0 2023-12-19T18:14:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.783+0000 7f1f81aaf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c070a90 con 0x7f1f7c148180 2023-12-19T18:14:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.783+0000 7f1f812ae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f7c14e4d0 0x7f1f7c14e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.783+0000 7f1f812ae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f7c14e4d0 0x7f1f7c14e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54612/0 (socket says 172.21.15.27:54612) 2023-12-19T18:14:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.783+0000 7f1f812ae640 1 -- 172.21.15.27:0/496990197 learned_addr learned my addr 172.21.15.27:0/496990197 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.784+0000 7f1f7bfff640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f7c148180 0x7f1f7c0c74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.784+0000 7f1f80aad640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f6c005bd0 0x7f1f7c1478d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.784+0000 7f1f812ae640 1 -- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f7c148180 msgr2=0x7f1f7c0c74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.784+0000 7f1f812ae640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f7c148180 0x7f1f7c0c74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.784+0000 7f1f812ae640 1 -- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f6c005bd0 msgr2=0x7f1f7c1478d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.784+0000 7f1f812ae640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f6c005bd0 0x7f1f7c1478d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.784+0000 7f1f812ae640 1 -- 172.21.15.27:0/496990197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c070d70 con 0x7f1f7c14e4d0 2023-12-19T18:14:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.784+0000 7f1f812ae640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f7c14e4d0 0x7f1f7c14e950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0ce5a0 tx=0x7f1f7400d9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.785+0000 7f1f797fa640 1 -- 172.21.15.27:0/496990197 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f74006d50 con 0x7f1f7c14e4d0 2023-12-19T18:14:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.785+0000 7f1f797fa640 1 -- 172.21.15.27:0/496990197 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f74012600 con 0x7f1f7c14e4d0 2023-12-19T18:14:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.785+0000 7f1f797fa640 1 -- 172.21.15.27:0/496990197 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f740129b0 con 0x7f1f7c14e4d0 2023-12-19T18:14:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.785+0000 7f1f81aaf640 1 -- 172.21.15.27:0/496990197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c071050 con 0x7f1f7c14e4d0 2023-12-19T18:14:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.785+0000 7f1f81aaf640 1 -- 172.21.15.27:0/496990197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c071570 con 0x7f1f7c14e4d0 2023-12-19T18:14:58.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.786+0000 7f1f81aaf640 1 -- 172.21.15.27:0/496990197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c071240 con 0x7f1f7c14e4d0 2023-12-19T18:14:58.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.787+0000 7f1f797fa640 1 -- 172.21.15.27:0/496990197 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1f74012b10 con 0x7f1f7c14e4d0 2023-12-19T18:14:58.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.788+0000 7f1f797fa640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6006a460 0x7f1f6006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:58.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.789+0000 7f1f797fa640 1 -- 172.21.15.27:0/496990197 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f740884e0 con 0x7f1f7c14e4d0 2023-12-19T18:14:58.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.789+0000 7f1f80aad640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6006a460 0x7f1f6006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:58.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.790+0000 7f1f80aad640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6006a460 0x7f1f6006c920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1f70034000 tx=0x7f1f70009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:58.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.793+0000 7f1f797fa640 1 -- 172.21.15.27:0/496990197 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1f74053ae0 con 0x7f1f7c14e4d0 2023-12-19T18:14:58.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:58.985+0000 7f1f81aaf640 1 -- 172.21.15.27:0/496990197 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1f7c0cbb40 con 0x7f1f6006a460 2023-12-19T18:14:59.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.009+0000 7f1f797fa640 1 -- 172.21.15.27:0/496990197 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+257118 (secure 0 0 0) 0x7f1f7003d040 con 0x7f1f6006a460 2023-12-19T18:14:59.014 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:59.015 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:14:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.016+0000 7f1f81aaf640 1 -- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6006a460 msgr2=0x7f1f6006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.016+0000 7f1f81aaf640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6006a460 0x7f1f6006c920 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f1f70034000 tx=0x7f1f70009030 comp rx=0 tx=0).stop 2023-12-19T18:14:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.016+0000 7f1f81aaf640 1 -- 172.21.15.27:0/496990197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f7c14e4d0 msgr2=0x7f1f7c14e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.016+0000 7f1f81aaf640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f7c14e4d0 0x7f1f7c14e950 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c0ce5a0 tx=0x7f1f7400d9f0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.016+0000 7f1f81aaf640 1 -- 172.21.15.27:0/496990197 shutdown_connections 2023-12-19T18:14:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.016+0000 7f1f81aaf640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6006a460 0x7f1f6006c920 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.016+0000 7f1f81aaf640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f7c14e4d0 0x7f1f7c14e950 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.016+0000 7f1f81aaf640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f7c148180 0x7f1f7c0c74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.016+0000 7f1f81aaf640 1 --2- 172.21.15.27:0/496990197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f6c005bd0 0x7f1f7c1478d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.016+0000 7f1f81aaf640 1 -- 172.21.15.27:0/496990197 >> 172.21.15.27:0/496990197 conn(0x7f1f7c069c70 msgr2=0x7f1f7c14cbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:59.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.017+0000 7f1f81aaf640 1 -- 172.21.15.27:0/496990197 shutdown_connections 2023-12-19T18:14:59.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.017+0000 7f1f81aaf640 1 -- 172.21.15.27:0/496990197 wait complete. 2023-12-19T18:14:59.037 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-12-19T18:14:57.702586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1554,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560945,"statfs":{"total":1600000000,"available":1598408160,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.184128"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544997+0000","last_change":"2023-12-19T18:14:53.544997+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544997+0000","last_undegraded":"2023-12-19T18:14:53.544997+0000","last_fullsized":"2023-12-19T18:14:53.544997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.553614+0000","last_change":"2023-12-19T18:14:53.553614+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.553614+0000","last_undegraded":"2023-12-19T18:14:53.553614+0000","last_fullsized":"2023-12-19T18:14:53.553614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.552891+0000","last_change":"2023-12-19T18:14:53.552891+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.552891+0000","last_undegraded":"2023-12-19T18:14:53.552891+0000","last_fullsized":"2023-12-19T18:14:53.552891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.541737+0000","last_change":"2023-12-19T18:14:53.541737+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.541737+0000","last_undegraded":"2023-12-19T18:14:53.541737+0000","last_fullsized":"2023-12-19T18:14:53.541737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.545140+0000","last_change":"2023-12-19T18:14:53.545140+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.545140+0000","last_undegraded":"2023-12-19T18:14:53.545140+0000","last_fullsized":"2023-12-19T18:14:53.545140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542802+0000","last_change":"2023-12-19T18:14:53.542802+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542802+0000","last_undegraded":"2023-12-19T18:14:53.542802+0000","last_fullsized":"2023-12-19T18:14:53.542802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546687+0000","last_change":"2023-12-19T18:14:53.546687+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546687+0000","last_undegraded":"2023-12-19T18:14:53.546687+0000","last_fullsized":"2023-12-19T18:14:53.546687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.550985+0000","last_change":"2023-12-19T18:14:53.550985+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.550985+0000","last_undegraded":"2023-12-19T18:14:53.550985+0000","last_fullsized":"2023-12-19T18:14:53.550985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.541930+0000","last_change":"2023-12-19T18:14:53.541930+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.541930+0000","last_undegraded":"2023-12-19T18:14:53.541930+0000","last_fullsized":"2023-12-19T18:14:53.541930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544744+0000","last_change":"2023-12-19T18:14:53.544744+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544744+0000","last_undegraded":"2023-12-19T18:14:53.544744+0000","last_fullsized":"2023-12-19T18:14:53.544744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.543057+0000","last_change":"2023-12-19T18:14:53.543057+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.543057+0000","last_undegraded":"2023-12-19T18:14:53.543057+0000","last_fullsized":"2023-12-19T18:14:53.543057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546396+0000","last_change":"2023-12-19T18:14:53.546396+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546396+0000","last_undegraded":"2023-12-19T18:14:53.546396+0000","last_fullsized":"2023-12-19T18:14:53.546396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.555595+0000","last_change":"2023-12-19T18:14:53.555595+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.555595+0000","last_undegraded":"2023-12-19T18:14:53.555595+0000","last_fullsized":"2023-12-19T18:14:53.555595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:35:28.203585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.558753+0000","last_change":"2023-12-19T18:14:53.558753+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.558753+0000","last_undegraded":"2023-12-19T18:14:53.558753+0000","last_fullsized":"2023-12-19T18:14:53.558753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.551962+0000","last_change":"2023-12-19T18:14:53.551962+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.551962+0000","last_undegraded":"2023-12-19T18:14:53.551962+0000","last_fullsized":"2023-12-19T18:14:53.551962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.553912+0000","last_change":"2023-12-19T18:14:53.553912+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.553912+0000","last_undegraded":"2023-12-19T18:14:53.553912+0000","last_fullsized":"2023-12-19T18:14:53.553912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.554444+0000","last_change":"2023-12-19T18:14:53.554444+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.554444+0000","last_undegraded":"2023-12-19T18:14:53.554444+0000","last_fullsized":"2023-12-19T18:14:53.554444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:15.120564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.555038+0000","last_change":"2023-12-19T18:14:53.555038+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.555038+0000","last_undegraded":"2023-12-19T18:14:53.555038+0000","last_fullsized":"2023-12-19T18:14:53.555038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.552631+0000","last_change":"2023-12-19T18:14:53.552631+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.552631+0000","last_undegraded":"2023-12-19T18:14:53.552631+0000","last_fullsized":"2023-12-19T18:14:53.552631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.543933+0000","last_change":"2023-12-19T18:14:53.543933+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.543933+0000","last_undegraded":"2023-12-19T18:14:53.543933+0000","last_fullsized":"2023-12-19T18:14:53.543933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546584+0000","last_change":"2023-12-19T18:14:53.546584+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546584+0000","last_undegraded":"2023-12-19T18:14:53.546584+0000","last_fullsized":"2023-12-19T18:14:53.546584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.541921+0000","last_change":"2023-12-19T18:14:53.541921+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.541921+0000","last_undegraded":"2023-12-19T18:14:53.541921+0000","last_fullsized":"2023-12-19T18:14:53.541921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.547450+0000","last_change":"2023-12-19T18:14:53.547450+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.547450+0000","last_undegraded":"2023-12-19T18:14:53.547450+0000","last_fullsized":"2023-12-19T18:14:53.547450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.554626+0000","last_change":"2023-12-19T18:14:53.554626+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.554626+0000","last_undegraded":"2023-12-19T18:14:53.554626+0000","last_fullsized":"2023-12-19T18:14:53.554626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.548457+0000","last_change":"2023-12-19T18:14:53.548457+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.548457+0000","last_undegraded":"2023-12-19T18:14:53.548457+0000","last_fullsized":"2023-12-19T18:14:53.548457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.540860+0000","last_change":"2023-12-19T18:14:53.540860+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.540860+0000","last_undegraded":"2023-12-19T18:14:53.540860+0000","last_fullsized":"2023-12-19T18:14:53.540860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.559193+0000","last_change":"2023-12-19T18:14:53.559193+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.559193+0000","last_undegraded":"2023-12-19T18:14:53.559193+0000","last_fullsized":"2023-12-19T18:14:53.559193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.555018+0000","last_change":"2023-12-19T18:14:53.555018+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.555018+0000","last_undegraded":"2023-12-19T18:14:53.555018+0000","last_fullsized":"2023-12-19T18:14:53.555018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546797+0000","last_change":"2023-12-19T18:14:53.546797+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546797+0000","last_undegraded":"2023-12-19T18:14:53.546797+0000","last_fullsized":"2023-12-19T18:14:53.546797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.553554+0000","last_change":"2023-12-19T18:14:53.553554+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.553554+0000","last_undegraded":"2023-12-19T18:14:53.553554+0000","last_fullsized":"2023-12-19T18:14:53.553554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:47.213095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.550237+0000","last_change":"2023-12-19T18:14:53.550237+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.550237+0000","last_undegraded":"2023-12-19T18:14:53.550237+0000","last_fullsized":"2023-12-19T18:14:53.550237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542286+0000","last_change":"2023-12-19T18:14:53.542286+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542286+0000","last_undegraded":"2023-12-19T18:14:53.542286+0000","last_fullsized":"2023-12-19T18:14:53.542286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542615+0000","last_change":"2023-12-19T18:14:53.542615+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542615+0000","last_undegraded":"2023-12-19T18:14:53.542615+0000","last_fullsized":"2023-12-19T18:14:53.542615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.563172+0000","last_change":"2023-12-19T18:14:53.563064+0000","last_active":"2023-12-19T18:14:53.563172+0000","last_peered":"2023-12-19T18:14:53.563172+0000","last_clean":"2023-12-19T18:14:53.563172+0000","last_became_active":"2023-12-19T18:14:53.561534+0000","last_became_peered":"2023-12-19T18:14:53.561534+0000","last_unstale":"2023-12-19T18:14:53.563172+0000","last_undegraded":"2023-12-19T18:14:53.563172+0000","last_fullsized":"2023-12-19T18:14:53.563172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.547530+0000","last_change":"2023-12-19T18:14:53.547530+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.547530+0000","last_undegraded":"2023-12-19T18:14:53.547530+0000","last_fullsized":"2023-12-19T18:14:53.547530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.560946+0000","last_change":"2023-12-19T18:14:53.560763+0000","last_active":"2023-12-19T18:14:53.560946+0000","last_peered":"2023-12-19T18:14:53.560946+0000","last_clean":"2023-12-19T18:14:53.560946+0000","last_became_active":"2023-12-19T18:14:53.558648+0000","last_became_peered":"2023-12-19T18:14:53.558648+0000","last_unstale":"2023-12-19T18:14:53.560946+0000","last_undegraded":"2023-12-19T18:14:53.560946+0000","last_fullsized":"2023-12-19T18:14:53.560946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:51.179899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.561679+0000","last_change":"2023-12-19T18:14:53.560483+0000","last_active":"2023-12-19T18:14:53.561679+0000","last_peered":"2023-12-19T18:14:53.561679+0000","last_clean":"2023-12-19T18:14:53.561679+0000","last_became_active":"2023-12-19T18:14:53.560244+0000","last_became_peered":"2023-12-19T18:14:53.560244+0000","last_unstale":"2023-12-19T18:14:53.561679+0000","last_undegraded":"2023-12-19T18:14:53.561679+0000","last_fullsized":"2023-12-19T18:14:53.561679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544386+0000","last_change":"2023-12-19T18:14:53.544386+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544386+0000","last_undegraded":"2023-12-19T18:14:53.544386+0000","last_fullsized":"2023-12-19T18:14:53.544386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.559512+0000","last_change":"2023-12-19T18:14:53.558346+0000","last_active":"2023-12-19T18:14:53.559512+0000","last_peered":"2023-12-19T18:14:53.559512+0000","last_clean":"2023-12-19T18:14:53.559512+0000","last_became_active":"2023-12-19T18:14:53.558124+0000","last_became_peered":"2023-12-19T18:14:53.558124+0000","last_unstale":"2023-12-19T18:14:53.559512+0000","last_undegraded":"2023-12-19T18:14:53.559512+0000","last_fullsized":"2023-12-19T18:14:53.559512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.543259+0000","last_change":"2023-12-19T18:14:53.543259+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.543259+0000","last_undegraded":"2023-12-19T18:14:53.543259+0000","last_fullsized":"2023-12-19T18:14:53.543259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.557076+0000","last_change":"2023-12-19T18:14:53.554048+0000","last_active":"2023-12-19T18:14:53.557076+0000","last_peered":"2023-12-19T18:14:53.557076+0000","last_clean":"2023-12-19T18:14:53.557076+0000","last_became_active":"2023-12-19T18:14:53.553811+0000","last_became_peered":"2023-12-19T18:14:53.553811+0000","last_unstale":"2023-12-19T18:14:53.557076+0000","last_undegraded":"2023-12-19T18:14:53.557076+0000","last_fullsized":"2023-12-19T18:14:53.557076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.539109+0000","last_change":"2023-12-19T18:14:53.539109+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.539109+0000","last_undegraded":"2023-12-19T18:14:53.539109+0000","last_fullsized":"2023-12-19T18:14:53.539109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.558852+0000","last_change":"2023-12-19T18:14:53.557947+0000","last_active":"2023-12-19T18:14:53.558852+0000","last_peered":"2023-12-19T18:14:53.558852+0000","last_clean":"2023-12-19T18:14:53.558852+0000","last_became_active":"2023-12-19T18:14:53.557713+0000","last_became_peered":"2023-12-19T18:14:53.557713+0000","last_unstale":"2023-12-19T18:14:53.558852+0000","last_undegraded":"2023-12-19T18:14:53.558852+0000","last_fullsized":"2023-12-19T18:14:53.558852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546496+0000","last_change":"2023-12-19T18:14:53.546496+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546496+0000","last_undegraded":"2023-12-19T18:14:53.546496+0000","last_fullsized":"2023-12-19T18:14:53.546496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:30.847801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.568083+0000","last_change":"2023-12-19T18:14:53.568016+0000","last_active":"2023-12-19T18:14:53.568083+0000","last_peered":"2023-12-19T18:14:53.568083+0000","last_clean":"2023-12-19T18:14:53.568083+0000","last_became_active":"2023-12-19T18:14:53.567634+0000","last_became_peered":"2023-12-19T18:14:53.567634+0000","last_unstale":"2023-12-19T18:14:53.568083+0000","last_undegraded":"2023-12-19T18:14:53.568083+0000","last_fullsized":"2023-12-19T18:14:53.568083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.553864+0000","last_change":"2023-12-19T18:14:53.553864+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.553864+0000","last_undegraded":"2023-12-19T18:14:53.553864+0000","last_fullsized":"2023-12-19T18:14:53.553864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.548063+0000","last_change":"2023-12-19T18:14:53.547427+0000","last_active":"2023-12-19T18:14:53.548063+0000","last_peered":"2023-12-19T18:14:53.548063+0000","last_clean":"2023-12-19T18:14:53.548063+0000","last_became_active":"2023-12-19T18:14:53.546733+0000","last_became_peered":"2023-12-19T18:14:53.546733+0000","last_unstale":"2023-12-19T18:14:53.548063+0000","last_undegraded":"2023-12-19T18:14:53.548063+0000","last_fullsized":"2023-12-19T18:14:53.548063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:13:12.090625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542619+0000","last_change":"2023-12-19T18:14:53.542619+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542619+0000","last_undegraded":"2023-12-19T18:14:53.542619+0000","last_fullsized":"2023-12-19T18:14:53.542619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.566549+0000","last_change":"2023-12-19T18:14:53.566349+0000","last_active":"2023-12-19T18:14:53.566549+0000","last_peered":"2023-12-19T18:14:53.566549+0000","last_clean":"2023-12-19T18:14:53.566549+0000","last_became_active":"2023-12-19T18:14:53.565909+0000","last_became_peered":"2023-12-19T18:14:53.565909+0000","last_unstale":"2023-12-19T18:14:53.566549+0000","last_undegraded":"2023-12-19T18:14:53.566549+0000","last_fullsized":"2023-12-19T18:14:53.566549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.547788+0000","last_change":"2023-12-19T18:14:53.547788+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.547788+0000","last_undegraded":"2023-12-19T18:14:53.547788+0000","last_fullsized":"2023-12-19T18:14:53.547788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.561673+0000","last_change":"2023-12-19T18:14:53.561604+0000","last_active":"2023-12-19T18:14:53.561673+0000","last_peered":"2023-12-19T18:14:53.561673+0000","last_clean":"2023-12-19T18:14:53.561673+0000","last_became_active":"2023-12-19T18:14:53.561372+0000","last_became_peered":"2023-12-19T18:14:53.561372+0000","last_unstale":"2023-12-19T18:14:53.561673+0000","last_undegraded":"2023-12-19T18:14:53.561673+0000","last_fullsized":"2023-12-19T18:14:53.561673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:00.255541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.548390+0000","last_change":"2023-12-19T18:14:53.548390+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.548390+0000","last_undegraded":"2023-12-19T18:14:53.548390+0000","last_fullsized":"2023-12-19T18:14:53.548390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:12.180767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.562843+0000","last_change":"2023-12-19T18:14:53.562764+0000","last_active":"2023-12-19T18:14:53.562843+0000","last_peered":"2023-12-19T18:14:53.562843+0000","last_clean":"2023-12-19T18:14:53.562843+0000","last_became_active":"2023-12-19T18:14:53.562532+0000","last_became_peered":"2023-12-19T18:14:53.562532+0000","last_unstale":"2023-12-19T18:14:53.562843+0000","last_undegraded":"2023-12-19T18:14:53.562843+0000","last_fullsized":"2023-12-19T18:14:53.562843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542160+0000","last_change":"2023-12-19T18:14:53.542160+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542160+0000","last_undegraded":"2023-12-19T18:14:53.542160+0000","last_fullsized":"2023-12-19T18:14:53.542160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.565174+0000","last_change":"2023-12-19T18:14:53.565084+0000","last_active":"2023-12-19T18:14:53.565174+0000","last_peered":"2023-12-19T18:14:53.565174+0000","last_clean":"2023-12-19T18:14:53.565174+0000","last_became_active":"2023-12-19T18:14:53.564836+0000","last_became_peered":"2023-12-19T18:14:53.564836+0000","last_unstale":"2023-12-19T18:14:53.565174+0000","last_undegraded":"2023-12-19T18:14:53.565174+0000","last_fullsized":"2023-12-19T18:14:53.565174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.550552+0000","last_change":"2023-12-19T18:14:53.550552+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.550552+0000","last_undegraded":"2023-12-19T18:14:53.550552+0000","last_fullsized":"2023-12-19T18:14:53.550552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.550605+0000","last_change":"2023-12-19T18:14:53.550365+0000","last_active":"2023-12-19T18:14:53.550605+0000","last_peered":"2023-12-19T18:14:53.550605+0000","last_clean":"2023-12-19T18:14:53.550605+0000","last_became_active":"2023-12-19T18:14:53.550059+0000","last_became_peered":"2023-12-19T18:14:53.550059+0000","last_unstale":"2023-12-19T18:14:53.550605+0000","last_undegraded":"2023-12-19T18:14:53.550605+0000","last_fullsized":"2023-12-19T18:14:53.550605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.540960+0000","last_change":"2023-12-19T18:14:53.540960+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.540960+0000","last_undegraded":"2023-12-19T18:14:53.540960+0000","last_fullsized":"2023-12-19T18:14:53.540960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.567739+0000","last_change":"2023-12-19T18:14:53.567656+0000","last_active":"2023-12-19T18:14:53.567739+0000","last_peered":"2023-12-19T18:14:53.567739+0000","last_clean":"2023-12-19T18:14:53.567739+0000","last_became_active":"2023-12-19T18:14:53.567278+0000","last_became_peered":"2023-12-19T18:14:53.567278+0000","last_unstale":"2023-12-19T18:14:53.567739+0000","last_undegraded":"2023-12-19T18:14:53.567739+0000","last_fullsized":"2023-12-19T18:14:53.567739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544283+0000","last_change":"2023-12-19T18:14:53.544283+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544283+0000","last_undegraded":"2023-12-19T18:14:53.544283+0000","last_fullsized":"2023-12-19T18:14:53.544283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.559621+0000","last_change":"2023-12-19T18:14:53.559407+0000","last_active":"2023-12-19T18:14:53.559621+0000","last_peered":"2023-12-19T18:14:53.559621+0000","last_clean":"2023-12-19T18:14:53.559621+0000","last_became_active":"2023-12-19T18:14:53.559062+0000","last_became_peered":"2023-12-19T18:14:53.559062+0000","last_unstale":"2023-12-19T18:14:53.559621+0000","last_undegraded":"2023-12-19T18:14:53.559621+0000","last_fullsized":"2023-12-19T18:14:53.559621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546049+0000","last_change":"2023-12-19T18:14:53.546049+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546049+0000","last_undegraded":"2023-12-19T18:14:53.546049+0000","last_fullsized":"2023-12-19T18:14:53.546049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.563719+0000","last_change":"2023-12-19T18:14:53.563513+0000","last_active":"2023-12-19T18:14:53.563719+0000","last_peered":"2023-12-19T18:14:53.563719+0000","last_clean":"2023-12-19T18:14:53.563719+0000","last_became_active":"2023-12-19T18:14:53.562860+0000","last_became_peered":"2023-12-19T18:14:53.562860+0000","last_unstale":"2023-12-19T18:14:53.563719+0000","last_undegraded":"2023-12-19T18:14:53.563719+0000","last_fullsized":"2023-12-19T18:14:53.563719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:59:49.969450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.551179+0000","last_change":"2023-12-19T18:14:53.551179+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.551179+0000","last_undegraded":"2023-12-19T18:14:53.551179+0000","last_fullsized":"2023-12-19T18:14:53.551179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:27:34.736465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-12-19T18:14:52.517998+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:14:52.517998+0000","last_peered":"2023-12-19T18:14:52.517998+0000","last_clean":"2023-12-19T18:14:52.517998+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:14:52.517998+0000","last_undegraded":"2023-12-19T18:14:52.517998+0000","last_fullsized":"2023-12-19T18:14:52.517998+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.557318+0000","last_change":"2023-12-19T18:14:53.557129+0000","last_active":"2023-12-19T18:14:53.557318+0000","last_peered":"2023-12-19T18:14:53.557318+0000","last_clean":"2023-12-19T18:14:53.557318+0000","last_became_active":"2023-12-19T18:14:53.556812+0000","last_became_peered":"2023-12-19T18:14:53.556812+0000","last_unstale":"2023-12-19T18:14:53.557318+0000","last_undegraded":"2023-12-19T18:14:53.557318+0000","last_fullsized":"2023-12-19T18:14:53.557318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:53.113355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544270+0000","last_change":"2023-12-19T18:14:53.544270+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544270+0000","last_undegraded":"2023-12-19T18:14:53.544270+0000","last_fullsized":"2023-12-19T18:14:53.544270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.559529+0000","last_change":"2023-12-19T18:14:53.558583+0000","last_active":"2023-12-19T18:14:53.559529+0000","last_peered":"2023-12-19T18:14:53.559529+0000","last_clean":"2023-12-19T18:14:53.559529+0000","last_became_active":"2023-12-19T18:14:53.558000+0000","last_became_peered":"2023-12-19T18:14:53.558000+0000","last_unstale":"2023-12-19T18:14:53.559529+0000","last_undegraded":"2023-12-19T18:14:53.559529+0000","last_fullsized":"2023-12-19T18:14:53.559529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:53.818952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546374+0000","last_change":"2023-12-19T18:14:53.546374+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546374+0000","last_undegraded":"2023-12-19T18:14:53.546374+0000","last_fullsized":"2023-12-19T18:14:53.546374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.565697+0000","last_change":"2023-12-19T18:14:53.563881+0000","last_active":"2023-12-19T18:14:53.565697+0000","last_peered":"2023-12-19T18:14:53.565697+0000","last_clean":"2023-12-19T18:14:53.565697+0000","last_became_active":"2023-12-19T18:14:53.563564+0000","last_became_peered":"2023-12-19T18:14:53.563564+0000","last_unstale":"2023-12-19T18:14:53.565697+0000","last_undegraded":"2023-12-19T18:14:53.565697+0000","last_fullsized":"2023-12-19T18:14:53.565697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544768+0000","last_change":"2023-12-19T18:14:53.544768+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544768+0000","last_undegraded":"2023-12-19T18:14:53.544768+0000","last_fullsized":"2023-12-19T18:14:53.544768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.564141+0000","last_change":"2023-12-19T18:14:53.564065+0000","last_active":"2023-12-19T18:14:53.564141+0000","last_peered":"2023-12-19T18:14:53.564141+0000","last_clean":"2023-12-19T18:14:53.564141+0000","last_became_active":"2023-12-19T18:14:53.563841+0000","last_became_peered":"2023-12-19T18:14:53.563841+0000","last_unstale":"2023-12-19T18:14:53.564141+0000","last_undegraded":"2023-12-19T18:14:53.564141+0000","last_fullsized":"2023-12-19T18:14:53.564141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:46.161691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542633+0000","last_change":"2023-12-19T18:14:53.542633+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542633+0000","last_undegraded":"2023-12-19T18:14:53.542633+0000","last_fullsized":"2023-12-19T18:14:53.542633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.565247+0000","last_change":"2023-12-19T18:14:53.565164+0000","last_active":"2023-12-19T18:14:53.565247+0000","last_peered":"2023-12-19T18:14:53.565247+0000","last_clean":"2023-12-19T18:14:53.565247+0000","last_became_active":"2023-12-19T18:14:53.564538+0000","last_became_peered":"2023-12-19T18:14:53.564538+0000","last_unstale":"2023-12-19T18:14:53.565247+0000","last_undegraded":"2023-12-19T18:14:53.565247+0000","last_fullsized":"2023-12-19T18:14:53.565247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.538306+0000","last_change":"2023-12-19T18:14:53.538306+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.538306+0000","last_undegraded":"2023-12-19T18:14:53.538306+0000","last_fullsized":"2023-12-19T18:14:53.538306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.562513+0000","last_change":"2023-12-19T18:14:53.562234+0000","last_active":"2023-12-19T18:14:53.562513+0000","last_peered":"2023-12-19T18:14:53.562513+0000","last_clean":"2023-12-19T18:14:53.562513+0000","last_became_active":"2023-12-19T18:14:53.561326+0000","last_became_peered":"2023-12-19T18:14:53.561326+0000","last_unstale":"2023-12-19T18:14:53.562513+0000","last_undegraded":"2023-12-19T18:14:53.562513+0000","last_fullsized":"2023-12-19T18:14:53.562513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.551767+0000","last_change":"2023-12-19T18:14:53.551767+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.551767+0000","last_undegraded":"2023-12-19T18:14:53.551767+0000","last_fullsized":"2023-12-19T18:14:53.551767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.552202+0000","last_change":"2023-12-19T18:14:53.551251+0000","last_active":"2023-12-19T18:14:53.552202+0000","last_peered":"2023-12-19T18:14:53.552202+0000","last_clean":"2023-12-19T18:14:53.552202+0000","last_became_active":"2023-12-19T18:14:53.550691+0000","last_became_peered":"2023-12-19T18:14:53.550691+0000","last_unstale":"2023-12-19T18:14:53.552202+0000","last_undegraded":"2023-12-19T18:14:53.552202+0000","last_fullsized":"2023-12-19T18:14:53.552202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546662+0000","last_change":"2023-12-19T18:14:53.546662+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546662+0000","last_undegraded":"2023-12-19T18:14:53.546662+0000","last_fullsized":"2023-12-19T18:14:53.546662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.563737+0000","last_change":"2023-12-19T18:14:53.563561+0000","last_active":"2023-12-19T18:14:53.563737+0000","last_peered":"2023-12-19T18:14:53.563737+0000","last_clean":"2023-12-19T18:14:53.563737+0000","last_became_active":"2023-12-19T18:14:53.563126+0000","last_became_peered":"2023-12-19T18:14:53.563126+0000","last_unstale":"2023-12-19T18:14:53.563737+0000","last_undegraded":"2023-12-19T18:14:53.563737+0000","last_fullsized":"2023-12-19T18:14:53.563737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542025+0000","last_change":"2023-12-19T18:14:53.542025+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542025+0000","last_undegraded":"2023-12-19T18:14:53.542025+0000","last_fullsized":"2023-12-19T18:14:53.542025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.552212+0000","last_change":"2023-12-19T18:14:53.551856+0000","last_active":"2023-12-19T18:14:53.552212+0000","last_peered":"2023-12-19T18:14:53.552212+0000","last_clean":"2023-12-19T18:14:53.552212+0000","last_became_active":"2023-12-19T18:14:53.551105+0000","last_became_peered":"2023-12-19T18:14:53.551105+0000","last_unstale":"2023-12-19T18:14:53.552212+0000","last_undegraded":"2023-12-19T18:14:53.552212+0000","last_fullsized":"2023-12-19T18:14:53.552212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.547523+0000","last_change":"2023-12-19T18:14:53.547523+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.547523+0000","last_undegraded":"2023-12-19T18:14:53.547523+0000","last_fullsized":"2023-12-19T18:14:53.547523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.554359+0000","last_change":"2023-12-19T18:14:53.554185+0000","last_active":"2023-12-19T18:14:53.554359+0000","last_peered":"2023-12-19T18:14:53.554359+0000","last_clean":"2023-12-19T18:14:53.554359+0000","last_became_active":"2023-12-19T18:14:53.553750+0000","last_became_peered":"2023-12-19T18:14:53.553750+0000","last_unstale":"2023-12-19T18:14:53.554359+0000","last_undegraded":"2023-12-19T18:14:53.554359+0000","last_fullsized":"2023-12-19T18:14:53.554359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542880+0000","last_change":"2023-12-19T18:14:53.542880+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542880+0000","last_undegraded":"2023-12-19T18:14:53.542880+0000","last_fullsized":"2023-12-19T18:14:53.542880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.566807+0000","last_change":"2023-12-19T18:14:53.566640+0000","last_active":"2023-12-19T18:14:53.566807+0000","last_peered":"2023-12-19T18:14:53.566807+0000","last_clean":"2023-12-19T18:14:53.566807+0000","last_became_active":"2023-12-19T18:14:53.566413+0000","last_became_peered":"2023-12-19T18:14:53.566413+0000","last_unstale":"2023-12-19T18:14:53.566807+0000","last_undegraded":"2023-12-19T18:14:53.566807+0000","last_fullsized":"2023-12-19T18:14:53.566807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.554184+0000","last_change":"2023-12-19T18:14:53.554184+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.554184+0000","last_undegraded":"2023-12-19T18:14:53.554184+0000","last_fullsized":"2023-12-19T18:14:53.554184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.555549+0000","last_change":"2023-12-19T18:14:53.549245+0000","last_active":"2023-12-19T18:14:53.555549+0000","last_peered":"2023-12-19T18:14:53.555549+0000","last_clean":"2023-12-19T18:14:53.555549+0000","last_became_active":"2023-12-19T18:14:53.548992+0000","last_became_peered":"2023-12-19T18:14:53.548992+0000","last_unstale":"2023-12-19T18:14:53.555549+0000","last_undegraded":"2023-12-19T18:14:53.555549+0000","last_fullsized":"2023-12-19T18:14:53.555549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.553350+0000","last_change":"2023-12-19T18:14:53.553350+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.553350+0000","last_undegraded":"2023-12-19T18:14:53.553350+0000","last_fullsized":"2023-12-19T18:14:53.553350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:22.428929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.552630+0000","last_change":"2023-12-19T18:14:53.552630+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.552630+0000","last_undegraded":"2023-12-19T18:14:53.552630+0000","last_fullsized":"2023-12-19T18:14:53.552630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:47.825262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.555473+0000","last_change":"2023-12-19T18:14:53.555372+0000","last_active":"2023-12-19T18:14:53.555473+0000","last_peered":"2023-12-19T18:14:53.555473+0000","last_clean":"2023-12-19T18:14:53.555473+0000","last_became_active":"2023-12-19T18:14:53.555089+0000","last_became_peered":"2023-12-19T18:14:53.555089+0000","last_unstale":"2023-12-19T18:14:53.555473+0000","last_undegraded":"2023-12-19T18:14:53.555473+0000","last_fullsized":"2023-12-19T18:14:53.555473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544494+0000","last_change":"2023-12-19T18:14:53.544494+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544494+0000","last_undegraded":"2023-12-19T18:14:53.544494+0000","last_fullsized":"2023-12-19T18:14:53.544494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.557104+0000","last_change":"2023-12-19T18:14:53.556488+0000","last_active":"2023-12-19T18:14:53.557104+0000","last_peered":"2023-12-19T18:14:53.557104+0000","last_clean":"2023-12-19T18:14:53.557104+0000","last_became_active":"2023-12-19T18:14:53.555952+0000","last_became_peered":"2023-12-19T18:14:53.555952+0000","last_unstale":"2023-12-19T18:14:53.557104+0000","last_undegraded":"2023-12-19T18:14:53.557104+0000","last_fullsized":"2023-12-19T18:14:53.557104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546044+0000","last_change":"2023-12-19T18:14:53.546044+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546044+0000","last_undegraded":"2023-12-19T18:14:53.546044+0000","last_fullsized":"2023-12-19T18:14:53.546044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.566263+0000","last_change":"2023-12-19T18:14:53.566090+0000","last_active":"2023-12-19T18:14:53.566263+0000","last_peered":"2023-12-19T18:14:53.566263+0000","last_clean":"2023-12-19T18:14:53.566263+0000","last_became_active":"2023-12-19T18:14:53.565782+0000","last_became_peered":"2023-12-19T18:14:53.565782+0000","last_unstale":"2023-12-19T18:14:53.566263+0000","last_undegraded":"2023-12-19T18:14:53.566263+0000","last_fullsized":"2023-12-19T18:14:53.566263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.545957+0000","last_change":"2023-12-19T18:14:53.545957+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.545957+0000","last_undegraded":"2023-12-19T18:14:53.545957+0000","last_fullsized":"2023-12-19T18:14:53.545957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.559685+0000","last_change":"2023-12-19T18:14:53.558205+0000","last_active":"2023-12-19T18:14:53.559685+0000","last_peered":"2023-12-19T18:14:53.559685+0000","last_clean":"2023-12-19T18:14:53.559685+0000","last_became_active":"2023-12-19T18:14:53.557733+0000","last_became_peered":"2023-12-19T18:14:53.557733+0000","last_unstale":"2023-12-19T18:14:53.559685+0000","last_undegraded":"2023-12-19T18:14:53.559685+0000","last_fullsized":"2023-12-19T18:14:53.559685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4550000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3520000000000001}]}]}],"pool_statfs":[]}} 2023-12-19T18:14:59.042 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:14:59.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.316+0000 7f518d9ca640 1 Processor -- start 2023-12-19T18:14:59.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.317+0000 7f518d9ca640 1 -- start start 2023-12-19T18:14:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.318+0000 7f518d9ca640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f518815ceb0 0x7f51880b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.318+0000 7f518d9ca640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51880b4d10 0x7f51880d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.318+0000 7f518d9ca640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51880b5170 con 0x7f51880d1f60 2023-12-19T18:14:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.318+0000 7f518d9ca640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51880d23f0 con 0x7f518815ceb0 2023-12-19T18:14:59.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.318+0000 7f518d9ca640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51880b6540 con 0x7f51880b4d10 2023-12-19T18:14:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.319+0000 7f518c9c8640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f518815ceb0 0x7f51880b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.319+0000 7f5187fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51880b4d10 0x7f51880d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.319+0000 7f518c9c8640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f518815ceb0 0x7f51880b47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:44816/0 (socket says 172.21.15.27:44816) 2023-12-19T18:14:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.319+0000 7f518c9c8640 1 -- 172.21.15.27:0/1826452845 learned_addr learned my addr 172.21.15.27:0/1826452845 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.320+0000 7f518c9c8640 1 -- 172.21.15.27:0/1826452845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51880b4d10 msgr2=0x7f51880d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.320+0000 7f518c9c8640 1 --2- 172.21.15.27:0/1826452845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51880b4d10 0x7f51880d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.320+0000 7f518c9c8640 1 -- 172.21.15.27:0/1826452845 >> v1:172.21.15.192:6789/0 conn(0x7f51880d1f60 legacy=0x7f518806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.320+0000 7f518c9c8640 1 -- 172.21.15.27:0/1826452845 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51880cdfb0 con 0x7f518815ceb0 2023-12-19T18:14:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.320+0000 7f5187fff640 1 --2- 172.21.15.27:0/1826452845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51880b4d10 0x7f51880d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:14:59.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.321+0000 7f518c9c8640 1 --2- 172.21.15.27:0/1826452845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f518815ceb0 0x7f51880b47d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f517c009870 tx=0x7f517c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32916d77090e4091 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:59.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.322+0000 7f51877fe640 1 -- 172.21.15.27:0/1826452845 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c030840 con 0x7f518815ceb0 2023-12-19T18:14:59.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.322+0000 7f51877fe640 1 -- 172.21.15.27:0/1826452845 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c0309a0 con 0x7f518815ceb0 2023-12-19T18:14:59.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.322+0000 7f51877fe640 1 -- 172.21.15.27:0/1826452845 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c030d30 con 0x7f518815ceb0 2023-12-19T18:14:59.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.322+0000 7f518d9ca640 1 -- 172.21.15.27:0/1826452845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f518815ceb0 msgr2=0x7f51880b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:59.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.322+0000 7f518d9ca640 1 --2- 172.21.15.27:0/1826452845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f518815ceb0 0x7f51880b47d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f517c009870 tx=0x7f517c02b980 comp rx=0 tx=0).stop 2023-12-19T18:14:59.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.323+0000 7f518d9ca640 1 -- 172.21.15.27:0/1826452845 shutdown_connections 2023-12-19T18:14:59.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.323+0000 7f518d9ca640 1 --2- 172.21.15.27:0/1826452845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51880b4d10 0x7f51880d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.323+0000 7f518d9ca640 1 --2- 172.21.15.27:0/1826452845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f518815ceb0 0x7f51880b47d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.323+0000 7f518d9ca640 1 -- 172.21.15.27:0/1826452845 >> 172.21.15.27:0/1826452845 conn(0x7f51880cf7d0 msgr2=0x7f51880cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:59.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.323+0000 7f518d9ca640 1 -- 172.21.15.27:0/1826452845 shutdown_connections 2023-12-19T18:14:59.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.323+0000 7f518d9ca640 1 -- 172.21.15.27:0/1826452845 wait complete. 2023-12-19T18:14:59.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.325+0000 7f518d9ca640 1 Processor -- start 2023-12-19T18:14:59.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.325+0000 7f518d9ca640 1 -- start start 2023-12-19T18:14:59.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.326+0000 7f518d9ca640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51880b4d10 0x7f51881f5280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:59.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.326+0000 7f518d9ca640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51880d1f60 0x7f5188142370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:59.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.326+0000 7f518d9ca640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f518815ceb0 0x7f518813f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:59.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.326+0000 7f518d9ca640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51880c8c10 con 0x7f51880b4d10 2023-12-19T18:14:59.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.326+0000 7f518d9ca640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51881428b0 con 0x7f518815ceb0 2023-12-19T18:14:59.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.326+0000 7f518d9ca640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51880c7f90 con 0x7f51880d1f60 2023-12-19T18:14:59.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.326+0000 7f518c9c8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51880b4d10 0x7f51881f5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:59.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.326+0000 7f518c9c8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51880b4d10 0x7f51881f5280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54620/0 (socket says 172.21.15.27:54620) 2023-12-19T18:14:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.326+0000 7f518c9c8640 1 -- 172.21.15.27:0/1427335236 learned_addr learned my addr 172.21.15.27:0/1427335236 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.326+0000 7f5187fff640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51880d1f60 0x7f5188142370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.326+0000 7f518d1c9640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f518815ceb0 0x7f518813f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.327+0000 7f518c9c8640 1 -- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51880d1f60 msgr2=0x7f5188142370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.327+0000 7f518c9c8640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51880d1f60 0x7f5188142370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.327+0000 7f518c9c8640 1 -- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f518815ceb0 msgr2=0x7f518813f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.327+0000 7f518c9c8640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f518815ceb0 0x7f518813f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.327+0000 7f518c9c8640 1 -- 172.21.15.27:0/1427335236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5174009400 con 0x7f51880b4d10 2023-12-19T18:14:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.327+0000 7f518c9c8640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51880b4d10 0x7f51881f5280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f517c000c00 tx=0x7f517c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:59.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.327+0000 7f51857fa640 1 -- 172.21.15.27:0/1427335236 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c039c90 con 0x7f51880b4d10 2023-12-19T18:14:59.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.327+0000 7f51857fa640 1 -- 172.21.15.27:0/1427335236 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f517c039df0 con 0x7f51880b4d10 2023-12-19T18:14:59.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.327+0000 7f51857fa640 1 -- 172.21.15.27:0/1427335236 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517c043640 con 0x7f51880b4d10 2023-12-19T18:14:59.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.327+0000 7f518d9ca640 1 -- 172.21.15.27:0/1427335236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517c0094d0 con 0x7f51880b4d10 2023-12-19T18:14:59.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.328+0000 7f518d9ca640 1 -- 172.21.15.27:0/1427335236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51880c8550 con 0x7f51880b4d10 2023-12-19T18:14:59.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.329+0000 7f518d9ca640 1 -- 172.21.15.27:0/1427335236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f518804e840 con 0x7f51880b4d10 2023-12-19T18:14:59.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.330+0000 7f51857fa640 1 -- 172.21.15.27:0/1427335236 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f517c0437a0 con 0x7f51880b4d10 2023-12-19T18:14:59.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.331+0000 7f51857fa640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f516c06a460 0x7f516c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:59.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.331+0000 7f51857fa640 1 -- 172.21.15.27:0/1427335236 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f517c03e070 con 0x7f51880b4d10 2023-12-19T18:14:59.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.331+0000 7f5187fff640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f516c06a460 0x7f516c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:59.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.332+0000 7f5187fff640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f516c06a460 0x7f516c06c920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5188159760 tx=0x7f5174008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:59.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.335+0000 7f51857fa640 1 -- 172.21.15.27:0/1427335236 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f517c076ef0 con 0x7f51880b4d10 2023-12-19T18:14:59.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.529+0000 7f518d9ca640 1 -- 172.21.15.27:0/1427335236 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f51880d28e0 con 0x7f516c06a460 2023-12-19T18:14:59.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.550+0000 7f51857fa640 1 -- 172.21.15.27:0/1427335236 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+257118 (secure 0 0 0) 0x7f5174011a30 con 0x7f516c06a460 2023-12-19T18:14:59.552 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:14:59.555 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:14:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.557+0000 7f518d9ca640 1 -- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f516c06a460 msgr2=0x7f516c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.557+0000 7f518d9ca640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f516c06a460 0x7f516c06c920 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5188159760 tx=0x7f5174008040 comp rx=0 tx=0).stop 2023-12-19T18:14:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.557+0000 7f518d9ca640 1 -- 172.21.15.27:0/1427335236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51880b4d10 msgr2=0x7f51881f5280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.557+0000 7f518d9ca640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51880b4d10 0x7f51881f5280 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f517c000c00 tx=0x7f517c004730 comp rx=0 tx=0).stop 2023-12-19T18:14:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.557+0000 7f518d9ca640 1 -- 172.21.15.27:0/1427335236 shutdown_connections 2023-12-19T18:14:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.557+0000 7f518d9ca640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f516c06a460 0x7f516c06c920 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.557+0000 7f518d9ca640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f518815ceb0 0x7f518813f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.557+0000 7f518d9ca640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51880d1f60 0x7f5188142370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.557+0000 7f518d9ca640 1 --2- 172.21.15.27:0/1427335236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51880b4d10 0x7f51881f5280 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.557+0000 7f518d9ca640 1 -- 172.21.15.27:0/1427335236 >> 172.21.15.27:0/1427335236 conn(0x7f51880cf7d0 msgr2=0x7f518815d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.557+0000 7f518d9ca640 1 -- 172.21.15.27:0/1427335236 shutdown_connections 2023-12-19T18:14:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.557+0000 7f518d9ca640 1 -- 172.21.15.27:0/1427335236 wait complete. 2023-12-19T18:14:59.579 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2023-12-19T18:14:57.702586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1554,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560945,"statfs":{"total":1600000000,"available":1598408160,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.184128"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544997+0000","last_change":"2023-12-19T18:14:53.544997+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544997+0000","last_undegraded":"2023-12-19T18:14:53.544997+0000","last_fullsized":"2023-12-19T18:14:53.544997+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.553614+0000","last_change":"2023-12-19T18:14:53.553614+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.553614+0000","last_undegraded":"2023-12-19T18:14:53.553614+0000","last_fullsized":"2023-12-19T18:14:53.553614+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.552891+0000","last_change":"2023-12-19T18:14:53.552891+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.552891+0000","last_undegraded":"2023-12-19T18:14:53.552891+0000","last_fullsized":"2023-12-19T18:14:53.552891+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.541737+0000","last_change":"2023-12-19T18:14:53.541737+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.541737+0000","last_undegraded":"2023-12-19T18:14:53.541737+0000","last_fullsized":"2023-12-19T18:14:53.541737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.545140+0000","last_change":"2023-12-19T18:14:53.545140+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.545140+0000","last_undegraded":"2023-12-19T18:14:53.545140+0000","last_fullsized":"2023-12-19T18:14:53.545140+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542802+0000","last_change":"2023-12-19T18:14:53.542802+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542802+0000","last_undegraded":"2023-12-19T18:14:53.542802+0000","last_fullsized":"2023-12-19T18:14:53.542802+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546687+0000","last_change":"2023-12-19T18:14:53.546687+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546687+0000","last_undegraded":"2023-12-19T18:14:53.546687+0000","last_fullsized":"2023-12-19T18:14:53.546687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.550985+0000","last_change":"2023-12-19T18:14:53.550985+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.550985+0000","last_undegraded":"2023-12-19T18:14:53.550985+0000","last_fullsized":"2023-12-19T18:14:53.550985+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.541930+0000","last_change":"2023-12-19T18:14:53.541930+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.541930+0000","last_undegraded":"2023-12-19T18:14:53.541930+0000","last_fullsized":"2023-12-19T18:14:53.541930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544744+0000","last_change":"2023-12-19T18:14:53.544744+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544744+0000","last_undegraded":"2023-12-19T18:14:53.544744+0000","last_fullsized":"2023-12-19T18:14:53.544744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.543057+0000","last_change":"2023-12-19T18:14:53.543057+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.543057+0000","last_undegraded":"2023-12-19T18:14:53.543057+0000","last_fullsized":"2023-12-19T18:14:53.543057+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546396+0000","last_change":"2023-12-19T18:14:53.546396+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546396+0000","last_undegraded":"2023-12-19T18:14:53.546396+0000","last_fullsized":"2023-12-19T18:14:53.546396+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.555595+0000","last_change":"2023-12-19T18:14:53.555595+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.555595+0000","last_undegraded":"2023-12-19T18:14:53.555595+0000","last_fullsized":"2023-12-19T18:14:53.555595+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:35:28.203585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.558753+0000","last_change":"2023-12-19T18:14:53.558753+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.558753+0000","last_undegraded":"2023-12-19T18:14:53.558753+0000","last_fullsized":"2023-12-19T18:14:53.558753+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.551962+0000","last_change":"2023-12-19T18:14:53.551962+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.551962+0000","last_undegraded":"2023-12-19T18:14:53.551962+0000","last_fullsized":"2023-12-19T18:14:53.551962+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.553912+0000","last_change":"2023-12-19T18:14:53.553912+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.553912+0000","last_undegraded":"2023-12-19T18:14:53.553912+0000","last_fullsized":"2023-12-19T18:14:53.553912+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.554444+0000","last_change":"2023-12-19T18:14:53.554444+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.554444+0000","last_undegraded":"2023-12-19T18:14:53.554444+0000","last_fullsized":"2023-12-19T18:14:53.554444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:15.120564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.555038+0000","last_change":"2023-12-19T18:14:53.555038+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.555038+0000","last_undegraded":"2023-12-19T18:14:53.555038+0000","last_fullsized":"2023-12-19T18:14:53.555038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.552631+0000","last_change":"2023-12-19T18:14:53.552631+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.552631+0000","last_undegraded":"2023-12-19T18:14:53.552631+0000","last_fullsized":"2023-12-19T18:14:53.552631+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.543933+0000","last_change":"2023-12-19T18:14:53.543933+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.543933+0000","last_undegraded":"2023-12-19T18:14:53.543933+0000","last_fullsized":"2023-12-19T18:14:53.543933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546584+0000","last_change":"2023-12-19T18:14:53.546584+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546584+0000","last_undegraded":"2023-12-19T18:14:53.546584+0000","last_fullsized":"2023-12-19T18:14:53.546584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.541921+0000","last_change":"2023-12-19T18:14:53.541921+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.541921+0000","last_undegraded":"2023-12-19T18:14:53.541921+0000","last_fullsized":"2023-12-19T18:14:53.541921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.547450+0000","last_change":"2023-12-19T18:14:53.547450+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.547450+0000","last_undegraded":"2023-12-19T18:14:53.547450+0000","last_fullsized":"2023-12-19T18:14:53.547450+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.554626+0000","last_change":"2023-12-19T18:14:53.554626+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.554626+0000","last_undegraded":"2023-12-19T18:14:53.554626+0000","last_fullsized":"2023-12-19T18:14:53.554626+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.548457+0000","last_change":"2023-12-19T18:14:53.548457+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.548457+0000","last_undegraded":"2023-12-19T18:14:53.548457+0000","last_fullsized":"2023-12-19T18:14:53.548457+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.540860+0000","last_change":"2023-12-19T18:14:53.540860+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.540860+0000","last_undegraded":"2023-12-19T18:14:53.540860+0000","last_fullsized":"2023-12-19T18:14:53.540860+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.559193+0000","last_change":"2023-12-19T18:14:53.559193+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.559193+0000","last_undegraded":"2023-12-19T18:14:53.559193+0000","last_fullsized":"2023-12-19T18:14:53.559193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.555018+0000","last_change":"2023-12-19T18:14:53.555018+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.555018+0000","last_undegraded":"2023-12-19T18:14:53.555018+0000","last_fullsized":"2023-12-19T18:14:53.555018+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546797+0000","last_change":"2023-12-19T18:14:53.546797+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546797+0000","last_undegraded":"2023-12-19T18:14:53.546797+0000","last_fullsized":"2023-12-19T18:14:53.546797+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.553554+0000","last_change":"2023-12-19T18:14:53.553554+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.553554+0000","last_undegraded":"2023-12-19T18:14:53.553554+0000","last_fullsized":"2023-12-19T18:14:53.553554+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:47.213095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.550237+0000","last_change":"2023-12-19T18:14:53.550237+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.550237+0000","last_undegraded":"2023-12-19T18:14:53.550237+0000","last_fullsized":"2023-12-19T18:14:53.550237+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542286+0000","last_change":"2023-12-19T18:14:53.542286+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542286+0000","last_undegraded":"2023-12-19T18:14:53.542286+0000","last_fullsized":"2023-12-19T18:14:53.542286+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542615+0000","last_change":"2023-12-19T18:14:53.542615+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542615+0000","last_undegraded":"2023-12-19T18:14:53.542615+0000","last_fullsized":"2023-12-19T18:14:53.542615+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.563172+0000","last_change":"2023-12-19T18:14:53.563064+0000","last_active":"2023-12-19T18:14:53.563172+0000","last_peered":"2023-12-19T18:14:53.563172+0000","last_clean":"2023-12-19T18:14:53.563172+0000","last_became_active":"2023-12-19T18:14:53.561534+0000","last_became_peered":"2023-12-19T18:14:53.561534+0000","last_unstale":"2023-12-19T18:14:53.563172+0000","last_undegraded":"2023-12-19T18:14:53.563172+0000","last_fullsized":"2023-12-19T18:14:53.563172+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.547530+0000","last_change":"2023-12-19T18:14:53.547530+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.547530+0000","last_undegraded":"2023-12-19T18:14:53.547530+0000","last_fullsized":"2023-12-19T18:14:53.547530+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.560946+0000","last_change":"2023-12-19T18:14:53.560763+0000","last_active":"2023-12-19T18:14:53.560946+0000","last_peered":"2023-12-19T18:14:53.560946+0000","last_clean":"2023-12-19T18:14:53.560946+0000","last_became_active":"2023-12-19T18:14:53.558648+0000","last_became_peered":"2023-12-19T18:14:53.558648+0000","last_unstale":"2023-12-19T18:14:53.560946+0000","last_undegraded":"2023-12-19T18:14:53.560946+0000","last_fullsized":"2023-12-19T18:14:53.560946+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:51.179899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.561679+0000","last_change":"2023-12-19T18:14:53.560483+0000","last_active":"2023-12-19T18:14:53.561679+0000","last_peered":"2023-12-19T18:14:53.561679+0000","last_clean":"2023-12-19T18:14:53.561679+0000","last_became_active":"2023-12-19T18:14:53.560244+0000","last_became_peered":"2023-12-19T18:14:53.560244+0000","last_unstale":"2023-12-19T18:14:53.561679+0000","last_undegraded":"2023-12-19T18:14:53.561679+0000","last_fullsized":"2023-12-19T18:14:53.561679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544386+0000","last_change":"2023-12-19T18:14:53.544386+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544386+0000","last_undegraded":"2023-12-19T18:14:53.544386+0000","last_fullsized":"2023-12-19T18:14:53.544386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.559512+0000","last_change":"2023-12-19T18:14:53.558346+0000","last_active":"2023-12-19T18:14:53.559512+0000","last_peered":"2023-12-19T18:14:53.559512+0000","last_clean":"2023-12-19T18:14:53.559512+0000","last_became_active":"2023-12-19T18:14:53.558124+0000","last_became_peered":"2023-12-19T18:14:53.558124+0000","last_unstale":"2023-12-19T18:14:53.559512+0000","last_undegraded":"2023-12-19T18:14:53.559512+0000","last_fullsized":"2023-12-19T18:14:53.559512+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.543259+0000","last_change":"2023-12-19T18:14:53.543259+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.543259+0000","last_undegraded":"2023-12-19T18:14:53.543259+0000","last_fullsized":"2023-12-19T18:14:53.543259+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.557076+0000","last_change":"2023-12-19T18:14:53.554048+0000","last_active":"2023-12-19T18:14:53.557076+0000","last_peered":"2023-12-19T18:14:53.557076+0000","last_clean":"2023-12-19T18:14:53.557076+0000","last_became_active":"2023-12-19T18:14:53.553811+0000","last_became_peered":"2023-12-19T18:14:53.553811+0000","last_unstale":"2023-12-19T18:14:53.557076+0000","last_undegraded":"2023-12-19T18:14:53.557076+0000","last_fullsized":"2023-12-19T18:14:53.557076+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.539109+0000","last_change":"2023-12-19T18:14:53.539109+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.539109+0000","last_undegraded":"2023-12-19T18:14:53.539109+0000","last_fullsized":"2023-12-19T18:14:53.539109+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.558852+0000","last_change":"2023-12-19T18:14:53.557947+0000","last_active":"2023-12-19T18:14:53.558852+0000","last_peered":"2023-12-19T18:14:53.558852+0000","last_clean":"2023-12-19T18:14:53.558852+0000","last_became_active":"2023-12-19T18:14:53.557713+0000","last_became_peered":"2023-12-19T18:14:53.557713+0000","last_unstale":"2023-12-19T18:14:53.558852+0000","last_undegraded":"2023-12-19T18:14:53.558852+0000","last_fullsized":"2023-12-19T18:14:53.558852+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546496+0000","last_change":"2023-12-19T18:14:53.546496+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546496+0000","last_undegraded":"2023-12-19T18:14:53.546496+0000","last_fullsized":"2023-12-19T18:14:53.546496+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:30.847801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.568083+0000","last_change":"2023-12-19T18:14:53.568016+0000","last_active":"2023-12-19T18:14:53.568083+0000","last_peered":"2023-12-19T18:14:53.568083+0000","last_clean":"2023-12-19T18:14:53.568083+0000","last_became_active":"2023-12-19T18:14:53.567634+0000","last_became_peered":"2023-12-19T18:14:53.567634+0000","last_unstale":"2023-12-19T18:14:53.568083+0000","last_undegraded":"2023-12-19T18:14:53.568083+0000","last_fullsized":"2023-12-19T18:14:53.568083+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.553864+0000","last_change":"2023-12-19T18:14:53.553864+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.553864+0000","last_undegraded":"2023-12-19T18:14:53.553864+0000","last_fullsized":"2023-12-19T18:14:53.553864+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.548063+0000","last_change":"2023-12-19T18:14:53.547427+0000","last_active":"2023-12-19T18:14:53.548063+0000","last_peered":"2023-12-19T18:14:53.548063+0000","last_clean":"2023-12-19T18:14:53.548063+0000","last_became_active":"2023-12-19T18:14:53.546733+0000","last_became_peered":"2023-12-19T18:14:53.546733+0000","last_unstale":"2023-12-19T18:14:53.548063+0000","last_undegraded":"2023-12-19T18:14:53.548063+0000","last_fullsized":"2023-12-19T18:14:53.548063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:13:12.090625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542619+0000","last_change":"2023-12-19T18:14:53.542619+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542619+0000","last_undegraded":"2023-12-19T18:14:53.542619+0000","last_fullsized":"2023-12-19T18:14:53.542619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.566549+0000","last_change":"2023-12-19T18:14:53.566349+0000","last_active":"2023-12-19T18:14:53.566549+0000","last_peered":"2023-12-19T18:14:53.566549+0000","last_clean":"2023-12-19T18:14:53.566549+0000","last_became_active":"2023-12-19T18:14:53.565909+0000","last_became_peered":"2023-12-19T18:14:53.565909+0000","last_unstale":"2023-12-19T18:14:53.566549+0000","last_undegraded":"2023-12-19T18:14:53.566549+0000","last_fullsized":"2023-12-19T18:14:53.566549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.547788+0000","last_change":"2023-12-19T18:14:53.547788+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.547788+0000","last_undegraded":"2023-12-19T18:14:53.547788+0000","last_fullsized":"2023-12-19T18:14:53.547788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.561673+0000","last_change":"2023-12-19T18:14:53.561604+0000","last_active":"2023-12-19T18:14:53.561673+0000","last_peered":"2023-12-19T18:14:53.561673+0000","last_clean":"2023-12-19T18:14:53.561673+0000","last_became_active":"2023-12-19T18:14:53.561372+0000","last_became_peered":"2023-12-19T18:14:53.561372+0000","last_unstale":"2023-12-19T18:14:53.561673+0000","last_undegraded":"2023-12-19T18:14:53.561673+0000","last_fullsized":"2023-12-19T18:14:53.561673+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:00.255541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.548390+0000","last_change":"2023-12-19T18:14:53.548390+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.548390+0000","last_undegraded":"2023-12-19T18:14:53.548390+0000","last_fullsized":"2023-12-19T18:14:53.548390+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:12.180767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.562843+0000","last_change":"2023-12-19T18:14:53.562764+0000","last_active":"2023-12-19T18:14:53.562843+0000","last_peered":"2023-12-19T18:14:53.562843+0000","last_clean":"2023-12-19T18:14:53.562843+0000","last_became_active":"2023-12-19T18:14:53.562532+0000","last_became_peered":"2023-12-19T18:14:53.562532+0000","last_unstale":"2023-12-19T18:14:53.562843+0000","last_undegraded":"2023-12-19T18:14:53.562843+0000","last_fullsized":"2023-12-19T18:14:53.562843+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542160+0000","last_change":"2023-12-19T18:14:53.542160+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542160+0000","last_undegraded":"2023-12-19T18:14:53.542160+0000","last_fullsized":"2023-12-19T18:14:53.542160+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.565174+0000","last_change":"2023-12-19T18:14:53.565084+0000","last_active":"2023-12-19T18:14:53.565174+0000","last_peered":"2023-12-19T18:14:53.565174+0000","last_clean":"2023-12-19T18:14:53.565174+0000","last_became_active":"2023-12-19T18:14:53.564836+0000","last_became_peered":"2023-12-19T18:14:53.564836+0000","last_unstale":"2023-12-19T18:14:53.565174+0000","last_undegraded":"2023-12-19T18:14:53.565174+0000","last_fullsized":"2023-12-19T18:14:53.565174+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.550552+0000","last_change":"2023-12-19T18:14:53.550552+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.550552+0000","last_undegraded":"2023-12-19T18:14:53.550552+0000","last_fullsized":"2023-12-19T18:14:53.550552+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.550605+0000","last_change":"2023-12-19T18:14:53.550365+0000","last_active":"2023-12-19T18:14:53.550605+0000","last_peered":"2023-12-19T18:14:53.550605+0000","last_clean":"2023-12-19T18:14:53.550605+0000","last_became_active":"2023-12-19T18:14:53.550059+0000","last_became_peered":"2023-12-19T18:14:53.550059+0000","last_unstale":"2023-12-19T18:14:53.550605+0000","last_undegraded":"2023-12-19T18:14:53.550605+0000","last_fullsized":"2023-12-19T18:14:53.550605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.540960+0000","last_change":"2023-12-19T18:14:53.540960+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.540960+0000","last_undegraded":"2023-12-19T18:14:53.540960+0000","last_fullsized":"2023-12-19T18:14:53.540960+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.567739+0000","last_change":"2023-12-19T18:14:53.567656+0000","last_active":"2023-12-19T18:14:53.567739+0000","last_peered":"2023-12-19T18:14:53.567739+0000","last_clean":"2023-12-19T18:14:53.567739+0000","last_became_active":"2023-12-19T18:14:53.567278+0000","last_became_peered":"2023-12-19T18:14:53.567278+0000","last_unstale":"2023-12-19T18:14:53.567739+0000","last_undegraded":"2023-12-19T18:14:53.567739+0000","last_fullsized":"2023-12-19T18:14:53.567739+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544283+0000","last_change":"2023-12-19T18:14:53.544283+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544283+0000","last_undegraded":"2023-12-19T18:14:53.544283+0000","last_fullsized":"2023-12-19T18:14:53.544283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.559621+0000","last_change":"2023-12-19T18:14:53.559407+0000","last_active":"2023-12-19T18:14:53.559621+0000","last_peered":"2023-12-19T18:14:53.559621+0000","last_clean":"2023-12-19T18:14:53.559621+0000","last_became_active":"2023-12-19T18:14:53.559062+0000","last_became_peered":"2023-12-19T18:14:53.559062+0000","last_unstale":"2023-12-19T18:14:53.559621+0000","last_undegraded":"2023-12-19T18:14:53.559621+0000","last_fullsized":"2023-12-19T18:14:53.559621+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546049+0000","last_change":"2023-12-19T18:14:53.546049+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546049+0000","last_undegraded":"2023-12-19T18:14:53.546049+0000","last_fullsized":"2023-12-19T18:14:53.546049+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.563719+0000","last_change":"2023-12-19T18:14:53.563513+0000","last_active":"2023-12-19T18:14:53.563719+0000","last_peered":"2023-12-19T18:14:53.563719+0000","last_clean":"2023-12-19T18:14:53.563719+0000","last_became_active":"2023-12-19T18:14:53.562860+0000","last_became_peered":"2023-12-19T18:14:53.562860+0000","last_unstale":"2023-12-19T18:14:53.563719+0000","last_undegraded":"2023-12-19T18:14:53.563719+0000","last_fullsized":"2023-12-19T18:14:53.563719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:59:49.969450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.551179+0000","last_change":"2023-12-19T18:14:53.551179+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.551179+0000","last_undegraded":"2023-12-19T18:14:53.551179+0000","last_fullsized":"2023-12-19T18:14:53.551179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:27:34.736465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":162,"reported_epoch":17,"state":"active+clean","last_fresh":"2023-12-19T18:14:52.517998+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:14:52.517998+0000","last_peered":"2023-12-19T18:14:52.517998+0000","last_clean":"2023-12-19T18:14:52.517998+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:14:52.517998+0000","last_undegraded":"2023-12-19T18:14:52.517998+0000","last_fullsized":"2023-12-19T18:14:52.517998+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.557318+0000","last_change":"2023-12-19T18:14:53.557129+0000","last_active":"2023-12-19T18:14:53.557318+0000","last_peered":"2023-12-19T18:14:53.557318+0000","last_clean":"2023-12-19T18:14:53.557318+0000","last_became_active":"2023-12-19T18:14:53.556812+0000","last_became_peered":"2023-12-19T18:14:53.556812+0000","last_unstale":"2023-12-19T18:14:53.557318+0000","last_undegraded":"2023-12-19T18:14:53.557318+0000","last_fullsized":"2023-12-19T18:14:53.557318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:53.113355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544270+0000","last_change":"2023-12-19T18:14:53.544270+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544270+0000","last_undegraded":"2023-12-19T18:14:53.544270+0000","last_fullsized":"2023-12-19T18:14:53.544270+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.559529+0000","last_change":"2023-12-19T18:14:53.558583+0000","last_active":"2023-12-19T18:14:53.559529+0000","last_peered":"2023-12-19T18:14:53.559529+0000","last_clean":"2023-12-19T18:14:53.559529+0000","last_became_active":"2023-12-19T18:14:53.558000+0000","last_became_peered":"2023-12-19T18:14:53.558000+0000","last_unstale":"2023-12-19T18:14:53.559529+0000","last_undegraded":"2023-12-19T18:14:53.559529+0000","last_fullsized":"2023-12-19T18:14:53.559529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:53.818952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546374+0000","last_change":"2023-12-19T18:14:53.546374+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546374+0000","last_undegraded":"2023-12-19T18:14:53.546374+0000","last_fullsized":"2023-12-19T18:14:53.546374+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.565697+0000","last_change":"2023-12-19T18:14:53.563881+0000","last_active":"2023-12-19T18:14:53.565697+0000","last_peered":"2023-12-19T18:14:53.565697+0000","last_clean":"2023-12-19T18:14:53.565697+0000","last_became_active":"2023-12-19T18:14:53.563564+0000","last_became_peered":"2023-12-19T18:14:53.563564+0000","last_unstale":"2023-12-19T18:14:53.565697+0000","last_undegraded":"2023-12-19T18:14:53.565697+0000","last_fullsized":"2023-12-19T18:14:53.565697+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544768+0000","last_change":"2023-12-19T18:14:53.544768+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544768+0000","last_undegraded":"2023-12-19T18:14:53.544768+0000","last_fullsized":"2023-12-19T18:14:53.544768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.564141+0000","last_change":"2023-12-19T18:14:53.564065+0000","last_active":"2023-12-19T18:14:53.564141+0000","last_peered":"2023-12-19T18:14:53.564141+0000","last_clean":"2023-12-19T18:14:53.564141+0000","last_became_active":"2023-12-19T18:14:53.563841+0000","last_became_peered":"2023-12-19T18:14:53.563841+0000","last_unstale":"2023-12-19T18:14:53.564141+0000","last_undegraded":"2023-12-19T18:14:53.564141+0000","last_fullsized":"2023-12-19T18:14:53.564141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:46.161691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542633+0000","last_change":"2023-12-19T18:14:53.542633+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542633+0000","last_undegraded":"2023-12-19T18:14:53.542633+0000","last_fullsized":"2023-12-19T18:14:53.542633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.565247+0000","last_change":"2023-12-19T18:14:53.565164+0000","last_active":"2023-12-19T18:14:53.565247+0000","last_peered":"2023-12-19T18:14:53.565247+0000","last_clean":"2023-12-19T18:14:53.565247+0000","last_became_active":"2023-12-19T18:14:53.564538+0000","last_became_peered":"2023-12-19T18:14:53.564538+0000","last_unstale":"2023-12-19T18:14:53.565247+0000","last_undegraded":"2023-12-19T18:14:53.565247+0000","last_fullsized":"2023-12-19T18:14:53.565247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.538306+0000","last_change":"2023-12-19T18:14:53.538306+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.538306+0000","last_undegraded":"2023-12-19T18:14:53.538306+0000","last_fullsized":"2023-12-19T18:14:53.538306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.562513+0000","last_change":"2023-12-19T18:14:53.562234+0000","last_active":"2023-12-19T18:14:53.562513+0000","last_peered":"2023-12-19T18:14:53.562513+0000","last_clean":"2023-12-19T18:14:53.562513+0000","last_became_active":"2023-12-19T18:14:53.561326+0000","last_became_peered":"2023-12-19T18:14:53.561326+0000","last_unstale":"2023-12-19T18:14:53.562513+0000","last_undegraded":"2023-12-19T18:14:53.562513+0000","last_fullsized":"2023-12-19T18:14:53.562513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.551767+0000","last_change":"2023-12-19T18:14:53.551767+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.551767+0000","last_undegraded":"2023-12-19T18:14:53.551767+0000","last_fullsized":"2023-12-19T18:14:53.551767+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.552202+0000","last_change":"2023-12-19T18:14:53.551251+0000","last_active":"2023-12-19T18:14:53.552202+0000","last_peered":"2023-12-19T18:14:53.552202+0000","last_clean":"2023-12-19T18:14:53.552202+0000","last_became_active":"2023-12-19T18:14:53.550691+0000","last_became_peered":"2023-12-19T18:14:53.550691+0000","last_unstale":"2023-12-19T18:14:53.552202+0000","last_undegraded":"2023-12-19T18:14:53.552202+0000","last_fullsized":"2023-12-19T18:14:53.552202+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546662+0000","last_change":"2023-12-19T18:14:53.546662+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546662+0000","last_undegraded":"2023-12-19T18:14:53.546662+0000","last_fullsized":"2023-12-19T18:14:53.546662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.563737+0000","last_change":"2023-12-19T18:14:53.563561+0000","last_active":"2023-12-19T18:14:53.563737+0000","last_peered":"2023-12-19T18:14:53.563737+0000","last_clean":"2023-12-19T18:14:53.563737+0000","last_became_active":"2023-12-19T18:14:53.563126+0000","last_became_peered":"2023-12-19T18:14:53.563126+0000","last_unstale":"2023-12-19T18:14:53.563737+0000","last_undegraded":"2023-12-19T18:14:53.563737+0000","last_fullsized":"2023-12-19T18:14:53.563737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542025+0000","last_change":"2023-12-19T18:14:53.542025+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542025+0000","last_undegraded":"2023-12-19T18:14:53.542025+0000","last_fullsized":"2023-12-19T18:14:53.542025+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.552212+0000","last_change":"2023-12-19T18:14:53.551856+0000","last_active":"2023-12-19T18:14:53.552212+0000","last_peered":"2023-12-19T18:14:53.552212+0000","last_clean":"2023-12-19T18:14:53.552212+0000","last_became_active":"2023-12-19T18:14:53.551105+0000","last_became_peered":"2023-12-19T18:14:53.551105+0000","last_unstale":"2023-12-19T18:14:53.552212+0000","last_undegraded":"2023-12-19T18:14:53.552212+0000","last_fullsized":"2023-12-19T18:14:53.552212+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.547523+0000","last_change":"2023-12-19T18:14:53.547523+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.547523+0000","last_undegraded":"2023-12-19T18:14:53.547523+0000","last_fullsized":"2023-12-19T18:14:53.547523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.554359+0000","last_change":"2023-12-19T18:14:53.554185+0000","last_active":"2023-12-19T18:14:53.554359+0000","last_peered":"2023-12-19T18:14:53.554359+0000","last_clean":"2023-12-19T18:14:53.554359+0000","last_became_active":"2023-12-19T18:14:53.553750+0000","last_became_peered":"2023-12-19T18:14:53.553750+0000","last_unstale":"2023-12-19T18:14:53.554359+0000","last_undegraded":"2023-12-19T18:14:53.554359+0000","last_fullsized":"2023-12-19T18:14:53.554359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.542880+0000","last_change":"2023-12-19T18:14:53.542880+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.542880+0000","last_undegraded":"2023-12-19T18:14:53.542880+0000","last_fullsized":"2023-12-19T18:14:53.542880+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.566807+0000","last_change":"2023-12-19T18:14:53.566640+0000","last_active":"2023-12-19T18:14:53.566807+0000","last_peered":"2023-12-19T18:14:53.566807+0000","last_clean":"2023-12-19T18:14:53.566807+0000","last_became_active":"2023-12-19T18:14:53.566413+0000","last_became_peered":"2023-12-19T18:14:53.566413+0000","last_unstale":"2023-12-19T18:14:53.566807+0000","last_undegraded":"2023-12-19T18:14:53.566807+0000","last_fullsized":"2023-12-19T18:14:53.566807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.554184+0000","last_change":"2023-12-19T18:14:53.554184+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.554184+0000","last_undegraded":"2023-12-19T18:14:53.554184+0000","last_fullsized":"2023-12-19T18:14:53.554184+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.555549+0000","last_change":"2023-12-19T18:14:53.549245+0000","last_active":"2023-12-19T18:14:53.555549+0000","last_peered":"2023-12-19T18:14:53.555549+0000","last_clean":"2023-12-19T18:14:53.555549+0000","last_became_active":"2023-12-19T18:14:53.548992+0000","last_became_peered":"2023-12-19T18:14:53.548992+0000","last_unstale":"2023-12-19T18:14:53.555549+0000","last_undegraded":"2023-12-19T18:14:53.555549+0000","last_fullsized":"2023-12-19T18:14:53.555549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.553350+0000","last_change":"2023-12-19T18:14:53.553350+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.553350+0000","last_undegraded":"2023-12-19T18:14:53.553350+0000","last_fullsized":"2023-12-19T18:14:53.553350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:22.428929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.552630+0000","last_change":"2023-12-19T18:14:53.552630+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.552630+0000","last_undegraded":"2023-12-19T18:14:53.552630+0000","last_fullsized":"2023-12-19T18:14:53.552630+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:47.825262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.555473+0000","last_change":"2023-12-19T18:14:53.555372+0000","last_active":"2023-12-19T18:14:53.555473+0000","last_peered":"2023-12-19T18:14:53.555473+0000","last_clean":"2023-12-19T18:14:53.555473+0000","last_became_active":"2023-12-19T18:14:53.555089+0000","last_became_peered":"2023-12-19T18:14:53.555089+0000","last_unstale":"2023-12-19T18:14:53.555473+0000","last_undegraded":"2023-12-19T18:14:53.555473+0000","last_fullsized":"2023-12-19T18:14:53.555473+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.544494+0000","last_change":"2023-12-19T18:14:53.544494+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.544494+0000","last_undegraded":"2023-12-19T18:14:53.544494+0000","last_fullsized":"2023-12-19T18:14:53.544494+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.557104+0000","last_change":"2023-12-19T18:14:53.556488+0000","last_active":"2023-12-19T18:14:53.557104+0000","last_peered":"2023-12-19T18:14:53.557104+0000","last_clean":"2023-12-19T18:14:53.557104+0000","last_became_active":"2023-12-19T18:14:53.555952+0000","last_became_peered":"2023-12-19T18:14:53.555952+0000","last_unstale":"2023-12-19T18:14:53.557104+0000","last_undegraded":"2023-12-19T18:14:53.557104+0000","last_fullsized":"2023-12-19T18:14:53.557104+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.546044+0000","last_change":"2023-12-19T18:14:53.546044+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.546044+0000","last_undegraded":"2023-12-19T18:14:53.546044+0000","last_fullsized":"2023-12-19T18:14:53.546044+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.566263+0000","last_change":"2023-12-19T18:14:53.566090+0000","last_active":"2023-12-19T18:14:53.566263+0000","last_peered":"2023-12-19T18:14:53.566263+0000","last_clean":"2023-12-19T18:14:53.566263+0000","last_became_active":"2023-12-19T18:14:53.565782+0000","last_became_peered":"2023-12-19T18:14:53.565782+0000","last_unstale":"2023-12-19T18:14:53.566263+0000","last_undegraded":"2023-12-19T18:14:53.566263+0000","last_fullsized":"2023-12-19T18:14:53.566263+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":1,"reported_epoch":18,"state":"creating+peering","last_fresh":"2023-12-19T18:14:53.545957+0000","last_change":"2023-12-19T18:14:53.545957+0000","last_active":"2023-12-19T18:14:53.516959+0000","last_peered":"2023-12-19T18:14:53.516959+0000","last_clean":"2023-12-19T18:14:53.516959+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:14:53.545957+0000","last_undegraded":"2023-12-19T18:14:53.545957+0000","last_fullsized":"2023-12-19T18:14:53.545957+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-12-19T18:14:53.559685+0000","last_change":"2023-12-19T18:14:53.558205+0000","last_active":"2023-12-19T18:14:53.559685+0000","last_peered":"2023-12-19T18:14:53.559685+0000","last_clean":"2023-12-19T18:14:53.559685+0000","last_became_active":"2023-12-19T18:14:53.557733+0000","last_became_peered":"2023-12-19T18:14:53.557733+0000","last_unstale":"2023-12-19T18:14:53.559685+0000","last_undegraded":"2023-12-19T18:14:53.559685+0000","last_fullsized":"2023-12-19T18:14:53.559685+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.073}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0429999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0069999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3640000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574851,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4550000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1870000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2050000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0940000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0469999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90200000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1839999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1319999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2549999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.387}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.446}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.109}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":9,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2689999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.427}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.391}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1020000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199358244,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71099999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.169}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.141}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.411}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.458}]}]},{"osd":0,"up_from":13,"seq":55834574851,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":50,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195262,"statfs":{"total":200000000,"available":199948612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.878}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3520000000000001}]}]}],"pool_statfs":[]}} 2023-12-19T18:14:59.583 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:14:59.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.858+0000 7ff575915640 1 Processor -- start 2023-12-19T18:14:59.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.859+0000 7ff575915640 1 -- start start 2023-12-19T18:14:59.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.860+0000 7ff575915640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff5701604d0 0x7ff5700b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:59.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.860+0000 7ff575915640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5700b7310 0x7ff5700b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:59.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.860+0000 7ff575915640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5700b5e90 con 0x7ff5700d2350 2023-12-19T18:14:59.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.860+0000 7ff575915640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5700d2b30 con 0x7ff5701604d0 2023-12-19T18:14:59.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.860+0000 7ff575915640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5700d2850 con 0x7ff5700b7310 2023-12-19T18:14:59.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.860+0000 7ff574913640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff5701604d0 0x7ff5700b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:59.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.860+0000 7ff575114640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5700b7310 0x7ff5700b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:59.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.861+0000 7ff575114640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5700b7310 0x7ff5700b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:39620/0 (socket says 172.21.15.27:39620) 2023-12-19T18:14:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.861+0000 7ff574913640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff5701604d0 0x7ff5700b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:44838/0 (socket says 172.21.15.27:44838) 2023-12-19T18:14:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.861+0000 7ff575114640 1 -- 172.21.15.27:0/3038341981 learned_addr learned my addr 172.21.15.27:0/3038341981 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.861+0000 7ff575114640 1 -- 172.21.15.27:0/3038341981 >> v2:172.21.15.192:3300/0 conn(0x7ff5701604d0 msgr2=0x7ff5700b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:14:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.861+0000 7ff575114640 1 --2- 172.21.15.27:0/3038341981 >> v2:172.21.15.192:3300/0 conn(0x7ff5701604d0 0x7ff5700b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.861+0000 7ff575114640 1 -- 172.21.15.27:0/3038341981 >> v1:172.21.15.192:6789/0 conn(0x7ff5700d2350 legacy=0x7ff5700d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:59.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.861+0000 7ff575114640 1 -- 172.21.15.27:0/3038341981 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5700cded0 con 0x7ff5700b7310 2023-12-19T18:14:59.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.863+0000 7ff575114640 1 --2- 172.21.15.27:0/3038341981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5700b7310 0x7ff5700b4d20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff568009870 tx=0x7ff56802b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4e42768ea209408 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:59.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.863+0000 7ff56f7fe640 1 -- 172.21.15.27:0/3038341981 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568030840 con 0x7ff5700b7310 2023-12-19T18:14:59.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.863+0000 7ff56f7fe640 1 -- 172.21.15.27:0/3038341981 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5680309a0 con 0x7ff5700b7310 2023-12-19T18:14:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.863+0000 7ff56f7fe640 1 -- 172.21.15.27:0/3038341981 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568030d30 con 0x7ff5700b7310 2023-12-19T18:14:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.864+0000 7ff575915640 1 -- 172.21.15.27:0/3038341981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5700b7310 msgr2=0x7ff5700b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.864+0000 7ff575915640 1 --2- 172.21.15.27:0/3038341981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5700b7310 0x7ff5700b4d20 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7ff568009870 tx=0x7ff56802b980 comp rx=0 tx=0).stop 2023-12-19T18:14:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.864+0000 7ff575915640 1 -- 172.21.15.27:0/3038341981 shutdown_connections 2023-12-19T18:14:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.864+0000 7ff575915640 1 --2- 172.21.15.27:0/3038341981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5700b7310 0x7ff5700b4d20 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.864+0000 7ff575915640 1 --2- 172.21.15.27:0/3038341981 >> v2:172.21.15.192:3300/0 conn(0x7ff5701604d0 0x7ff5700b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.864+0000 7ff575915640 1 -- 172.21.15.27:0/3038341981 >> 172.21.15.27:0/3038341981 conn(0x7ff5700cf7e0 msgr2=0x7ff5700cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:14:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.864+0000 7ff575915640 1 -- 172.21.15.27:0/3038341981 shutdown_connections 2023-12-19T18:14:59.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.864+0000 7ff575915640 1 -- 172.21.15.27:0/3038341981 wait complete. 2023-12-19T18:14:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.866+0000 7ff575915640 1 Processor -- start 2023-12-19T18:14:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.866+0000 7ff575915640 1 -- start start 2023-12-19T18:14:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.867+0000 7ff575915640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5700b7310 0x7ff570154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.867+0000 7ff575915640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5700d2350 0x7ff5700c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.867+0000 7ff575915640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5701604d0 0x7ff57015cb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.867+0000 7ff575915640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff570155330 con 0x7ff5701604d0 2023-12-19T18:14:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.867+0000 7ff575915640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5700c8770 con 0x7ff5700b7310 2023-12-19T18:14:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.867+0000 7ff575915640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5700c7bf0 con 0x7ff5700d2350 2023-12-19T18:14:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.868+0000 7ff575114640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5701604d0 0x7ff57015cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.868+0000 7ff575114640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5701604d0 0x7ff57015cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54626/0 (socket says 172.21.15.27:54626) 2023-12-19T18:14:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.868+0000 7ff575114640 1 -- 172.21.15.27:0/4254143375 learned_addr learned my addr 172.21.15.27:0/4254143375 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:14:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.868+0000 7ff56ffff640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5700d2350 0x7ff5700c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.868+0000 7ff574913640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5700b7310 0x7ff570154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.868+0000 7ff575114640 1 -- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5700d2350 msgr2=0x7ff5700c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.868+0000 7ff575114640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5700d2350 0x7ff5700c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.868+0000 7ff575114640 1 -- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5700b7310 msgr2=0x7ff570154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:14:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.868+0000 7ff575114640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5700b7310 0x7ff570154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:14:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.868+0000 7ff575114640 1 -- 172.21.15.27:0/4254143375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5680094d0 con 0x7ff5701604d0 2023-12-19T18:14:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.868+0000 7ff575114640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5701604d0 0x7ff57015cb60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff568009870 tx=0x7ff568004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.869+0000 7ff56d7fa640 1 -- 172.21.15.27:0/4254143375 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568039a90 con 0x7ff5701604d0 2023-12-19T18:14:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.869+0000 7ff56d7fa640 1 -- 172.21.15.27:0/4254143375 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff568039c10 con 0x7ff5701604d0 2023-12-19T18:14:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.869+0000 7ff56d7fa640 1 -- 172.21.15.27:0/4254143375 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff568043410 con 0x7ff5701604d0 2023-12-19T18:14:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.869+0000 7ff575915640 1 -- 172.21.15.27:0/4254143375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5700c7ed0 con 0x7ff5701604d0 2023-12-19T18:14:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.869+0000 7ff575915640 1 -- 172.21.15.27:0/4254143375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5700c8490 con 0x7ff5701604d0 2023-12-19T18:14:59.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.870+0000 7ff575915640 1 -- 172.21.15.27:0/4254143375 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff570066f80 con 0x7ff5701604d0 2023-12-19T18:14:59.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.871+0000 7ff56d7fa640 1 -- 172.21.15.27:0/4254143375 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff56802f390 con 0x7ff5701604d0 2023-12-19T18:14:59.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.873+0000 7ff56d7fa640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff55c06a430 0x7ff55c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:14:59.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.873+0000 7ff56d7fa640 1 -- 172.21.15.27:0/4254143375 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5680aaac0 con 0x7ff5701604d0 2023-12-19T18:14:59.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.873+0000 7ff574913640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff55c06a430 0x7ff55c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:14:59.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.874+0000 7ff574913640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff55c06a430 0x7ff55c06c8f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff5600074a0 tx=0x7ff56000b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:14:59.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:14:59.877+0000 7ff56d7fa640 1 -- 172.21.15.27:0/4254143375 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff568076130 con 0x7ff5701604d0 2023-12-19T18:14:59.980 INFO:tasks.ceph.mds.d.smithi192.stderr:2023-12-19T18:14:59.979+0000 7fa91ec33640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:15:00.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.070+0000 7ff575915640 1 -- 172.21.15.27:0/4254143375 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff5700ce490 con 0x7ff55c06a430 2023-12-19T18:15:00.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.090+0000 7ff56d7fa640 1 -- 172.21.15.27:0/4254143375 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260098 (secure 0 0 0) 0x7ff56000f050 con 0x7ff55c06a430 2023-12-19T18:15:00.096 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:00.097 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:15:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.095+0000 7ff575915640 1 -- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff55c06a430 msgr2=0x7ff55c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.095+0000 7ff575915640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff55c06a430 0x7ff55c06c8f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7ff5600074a0 tx=0x7ff56000b040 comp rx=0 tx=0).stop 2023-12-19T18:15:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.095+0000 7ff575915640 1 -- 172.21.15.27:0/4254143375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5701604d0 msgr2=0x7ff57015cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.095+0000 7ff575915640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5701604d0 0x7ff57015cb60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff568009870 tx=0x7ff568004730 comp rx=0 tx=0).stop 2023-12-19T18:15:00.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.096+0000 7ff575915640 1 -- 172.21.15.27:0/4254143375 shutdown_connections 2023-12-19T18:15:00.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.096+0000 7ff575915640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff55c06a430 0x7ff55c06c8f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.096+0000 7ff575915640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5701604d0 0x7ff57015cb60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.096+0000 7ff575915640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5700d2350 0x7ff5700c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.096+0000 7ff575915640 1 --2- 172.21.15.27:0/4254143375 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5700b7310 0x7ff570154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.096+0000 7ff575915640 1 -- 172.21.15.27:0/4254143375 >> 172.21.15.27:0/4254143375 conn(0x7ff5700cf7e0 msgr2=0x7ff57015ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:00.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.096+0000 7ff575915640 1 -- 172.21.15.27:0/4254143375 shutdown_connections 2023-12-19T18:15:00.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.096+0000 7ff575915640 1 -- 172.21.15.27:0/4254143375 wait complete. 2023-12-19T18:15:00.115 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-12-19T18:14:59.703239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248636,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.167376"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056823+0000","last_change":"2023-12-19T18:14:54.056711+0000","last_active":"2023-12-19T18:14:54.056823+0000","last_peered":"2023-12-19T18:14:54.056823+0000","last_clean":"2023-12-19T18:14:54.056823+0000","last_became_active":"2023-12-19T18:14:54.056335+0000","last_became_peered":"2023-12-19T18:14:54.056335+0000","last_unstale":"2023-12-19T18:14:54.056823+0000","last_undegraded":"2023-12-19T18:14:54.056823+0000","last_fullsized":"2023-12-19T18:14:54.056823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069974+0000","last_change":"2023-12-19T18:14:54.069882+0000","last_active":"2023-12-19T18:14:54.069974+0000","last_peered":"2023-12-19T18:14:54.069974+0000","last_clean":"2023-12-19T18:14:54.069974+0000","last_became_active":"2023-12-19T18:14:54.069626+0000","last_became_peered":"2023-12-19T18:14:54.069626+0000","last_unstale":"2023-12-19T18:14:54.069974+0000","last_undegraded":"2023-12-19T18:14:54.069974+0000","last_fullsized":"2023-12-19T18:14:54.069974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053578+0000","last_change":"2023-12-19T18:14:54.053111+0000","last_active":"2023-12-19T18:14:54.053578+0000","last_peered":"2023-12-19T18:14:54.053578+0000","last_clean":"2023-12-19T18:14:54.053578+0000","last_became_active":"2023-12-19T18:14:54.052858+0000","last_became_peered":"2023-12-19T18:14:54.052858+0000","last_unstale":"2023-12-19T18:14:54.053578+0000","last_undegraded":"2023-12-19T18:14:54.053578+0000","last_fullsized":"2023-12-19T18:14:54.053578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050310+0000","last_change":"2023-12-19T18:14:54.049862+0000","last_active":"2023-12-19T18:14:54.050310+0000","last_peered":"2023-12-19T18:14:54.050310+0000","last_clean":"2023-12-19T18:14:54.050310+0000","last_became_active":"2023-12-19T18:14:54.049396+0000","last_became_peered":"2023-12-19T18:14:54.049396+0000","last_unstale":"2023-12-19T18:14:54.050310+0000","last_undegraded":"2023-12-19T18:14:54.050310+0000","last_fullsized":"2023-12-19T18:14:54.050310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053036+0000","last_change":"2023-12-19T18:14:54.052387+0000","last_active":"2023-12-19T18:14:54.053036+0000","last_peered":"2023-12-19T18:14:54.053036+0000","last_clean":"2023-12-19T18:14:54.053036+0000","last_became_active":"2023-12-19T18:14:54.052008+0000","last_became_peered":"2023-12-19T18:14:54.052008+0000","last_unstale":"2023-12-19T18:14:54.053036+0000","last_undegraded":"2023-12-19T18:14:54.053036+0000","last_fullsized":"2023-12-19T18:14:54.053036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061254+0000","last_change":"2023-12-19T18:14:54.061175+0000","last_active":"2023-12-19T18:14:54.061254+0000","last_peered":"2023-12-19T18:14:54.061254+0000","last_clean":"2023-12-19T18:14:54.061254+0000","last_became_active":"2023-12-19T18:14:54.060192+0000","last_became_peered":"2023-12-19T18:14:54.060192+0000","last_unstale":"2023-12-19T18:14:54.061254+0000","last_undegraded":"2023-12-19T18:14:54.061254+0000","last_fullsized":"2023-12-19T18:14:54.061254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.054262+0000","last_change":"2023-12-19T18:14:54.053283+0000","last_active":"2023-12-19T18:14:54.054262+0000","last_peered":"2023-12-19T18:14:54.054262+0000","last_clean":"2023-12-19T18:14:54.054262+0000","last_became_active":"2023-12-19T18:14:54.052854+0000","last_became_peered":"2023-12-19T18:14:54.052854+0000","last_unstale":"2023-12-19T18:14:54.054262+0000","last_undegraded":"2023-12-19T18:14:54.054262+0000","last_fullsized":"2023-12-19T18:14:54.054262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056579+0000","last_change":"2023-12-19T18:14:54.056483+0000","last_active":"2023-12-19T18:14:54.056579+0000","last_peered":"2023-12-19T18:14:54.056579+0000","last_clean":"2023-12-19T18:14:54.056579+0000","last_became_active":"2023-12-19T18:14:54.056196+0000","last_became_peered":"2023-12-19T18:14:54.056196+0000","last_unstale":"2023-12-19T18:14:54.056579+0000","last_undegraded":"2023-12-19T18:14:54.056579+0000","last_fullsized":"2023-12-19T18:14:54.056579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075945+0000","last_change":"2023-12-19T18:14:54.074709+0000","last_active":"2023-12-19T18:14:54.075945+0000","last_peered":"2023-12-19T18:14:54.075945+0000","last_clean":"2023-12-19T18:14:54.075945+0000","last_became_active":"2023-12-19T18:14:54.074482+0000","last_became_peered":"2023-12-19T18:14:54.074482+0000","last_unstale":"2023-12-19T18:14:54.075945+0000","last_undegraded":"2023-12-19T18:14:54.075945+0000","last_fullsized":"2023-12-19T18:14:54.075945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051054+0000","last_change":"2023-12-19T18:14:54.050203+0000","last_active":"2023-12-19T18:14:54.051054+0000","last_peered":"2023-12-19T18:14:54.051054+0000","last_clean":"2023-12-19T18:14:54.051054+0000","last_became_active":"2023-12-19T18:14:54.049582+0000","last_became_peered":"2023-12-19T18:14:54.049582+0000","last_unstale":"2023-12-19T18:14:54.051054+0000","last_undegraded":"2023-12-19T18:14:54.051054+0000","last_fullsized":"2023-12-19T18:14:54.051054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056806+0000","last_change":"2023-12-19T18:14:54.056564+0000","last_active":"2023-12-19T18:14:54.056806+0000","last_peered":"2023-12-19T18:14:54.056806+0000","last_clean":"2023-12-19T18:14:54.056806+0000","last_became_active":"2023-12-19T18:14:54.056052+0000","last_became_peered":"2023-12-19T18:14:54.056052+0000","last_unstale":"2023-12-19T18:14:54.056806+0000","last_undegraded":"2023-12-19T18:14:54.056806+0000","last_fullsized":"2023-12-19T18:14:54.056806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066528+0000","last_change":"2023-12-19T18:14:54.066257+0000","last_active":"2023-12-19T18:14:54.066528+0000","last_peered":"2023-12-19T18:14:54.066528+0000","last_clean":"2023-12-19T18:14:54.066528+0000","last_became_active":"2023-12-19T18:14:54.065876+0000","last_became_peered":"2023-12-19T18:14:54.065876+0000","last_unstale":"2023-12-19T18:14:54.066528+0000","last_undegraded":"2023-12-19T18:14:54.066528+0000","last_fullsized":"2023-12-19T18:14:54.066528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070727+0000","last_change":"2023-12-19T18:14:54.070658+0000","last_active":"2023-12-19T18:14:54.070727+0000","last_peered":"2023-12-19T18:14:54.070727+0000","last_clean":"2023-12-19T18:14:54.070727+0000","last_became_active":"2023-12-19T18:14:54.065952+0000","last_became_peered":"2023-12-19T18:14:54.065952+0000","last_unstale":"2023-12-19T18:14:54.070727+0000","last_undegraded":"2023-12-19T18:14:54.070727+0000","last_fullsized":"2023-12-19T18:14:54.070727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:35:28.203585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072334+0000","last_change":"2023-12-19T18:14:54.072211+0000","last_active":"2023-12-19T18:14:54.072334+0000","last_peered":"2023-12-19T18:14:54.072334+0000","last_clean":"2023-12-19T18:14:54.072334+0000","last_became_active":"2023-12-19T18:14:54.071912+0000","last_became_peered":"2023-12-19T18:14:54.071912+0000","last_unstale":"2023-12-19T18:14:54.072334+0000","last_undegraded":"2023-12-19T18:14:54.072334+0000","last_fullsized":"2023-12-19T18:14:54.072334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056480+0000","last_change":"2023-12-19T18:14:54.055761+0000","last_active":"2023-12-19T18:14:54.056480+0000","last_peered":"2023-12-19T18:14:54.056480+0000","last_clean":"2023-12-19T18:14:54.056480+0000","last_became_active":"2023-12-19T18:14:54.055529+0000","last_became_peered":"2023-12-19T18:14:54.055529+0000","last_unstale":"2023-12-19T18:14:54.056480+0000","last_undegraded":"2023-12-19T18:14:54.056480+0000","last_fullsized":"2023-12-19T18:14:54.056480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073307+0000","last_change":"2023-12-19T18:14:54.072540+0000","last_active":"2023-12-19T18:14:54.073307+0000","last_peered":"2023-12-19T18:14:54.073307+0000","last_clean":"2023-12-19T18:14:54.073307+0000","last_became_active":"2023-12-19T18:14:54.072162+0000","last_became_peered":"2023-12-19T18:14:54.072162+0000","last_unstale":"2023-12-19T18:14:54.073307+0000","last_undegraded":"2023-12-19T18:14:54.073307+0000","last_fullsized":"2023-12-19T18:14:54.073307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075801+0000","last_change":"2023-12-19T18:14:54.075571+0000","last_active":"2023-12-19T18:14:54.075801+0000","last_peered":"2023-12-19T18:14:54.075801+0000","last_clean":"2023-12-19T18:14:54.075801+0000","last_became_active":"2023-12-19T18:14:54.074613+0000","last_became_peered":"2023-12-19T18:14:54.074613+0000","last_unstale":"2023-12-19T18:14:54.075801+0000","last_undegraded":"2023-12-19T18:14:54.075801+0000","last_fullsized":"2023-12-19T18:14:54.075801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:15.120564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.062708+0000","last_change":"2023-12-19T18:14:54.062640+0000","last_active":"2023-12-19T18:14:54.062708+0000","last_peered":"2023-12-19T18:14:54.062708+0000","last_clean":"2023-12-19T18:14:54.062708+0000","last_became_active":"2023-12-19T18:14:54.061994+0000","last_became_peered":"2023-12-19T18:14:54.061994+0000","last_unstale":"2023-12-19T18:14:54.062708+0000","last_undegraded":"2023-12-19T18:14:54.062708+0000","last_fullsized":"2023-12-19T18:14:54.062708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073330+0000","last_change":"2023-12-19T18:14:54.073175+0000","last_active":"2023-12-19T18:14:54.073330+0000","last_peered":"2023-12-19T18:14:54.073330+0000","last_clean":"2023-12-19T18:14:54.073330+0000","last_became_active":"2023-12-19T18:14:54.072223+0000","last_became_peered":"2023-12-19T18:14:54.072223+0000","last_unstale":"2023-12-19T18:14:54.073330+0000","last_undegraded":"2023-12-19T18:14:54.073330+0000","last_fullsized":"2023-12-19T18:14:54.073330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064684+0000","last_change":"2023-12-19T18:14:54.064206+0000","last_active":"2023-12-19T18:14:54.064684+0000","last_peered":"2023-12-19T18:14:54.064684+0000","last_clean":"2023-12-19T18:14:54.064684+0000","last_became_active":"2023-12-19T18:14:54.063818+0000","last_became_peered":"2023-12-19T18:14:54.063818+0000","last_unstale":"2023-12-19T18:14:54.064684+0000","last_undegraded":"2023-12-19T18:14:54.064684+0000","last_fullsized":"2023-12-19T18:14:54.064684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063666+0000","last_change":"2023-12-19T18:14:54.063234+0000","last_active":"2023-12-19T18:14:54.063666+0000","last_peered":"2023-12-19T18:14:54.063666+0000","last_clean":"2023-12-19T18:14:54.063666+0000","last_became_active":"2023-12-19T18:14:54.062885+0000","last_became_peered":"2023-12-19T18:14:54.062885+0000","last_unstale":"2023-12-19T18:14:54.063666+0000","last_undegraded":"2023-12-19T18:14:54.063666+0000","last_fullsized":"2023-12-19T18:14:54.063666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063662+0000","last_change":"2023-12-19T18:14:54.063598+0000","last_active":"2023-12-19T18:14:54.063662+0000","last_peered":"2023-12-19T18:14:54.063662+0000","last_clean":"2023-12-19T18:14:54.063662+0000","last_became_active":"2023-12-19T18:14:54.063314+0000","last_became_peered":"2023-12-19T18:14:54.063314+0000","last_unstale":"2023-12-19T18:14:54.063662+0000","last_undegraded":"2023-12-19T18:14:54.063662+0000","last_fullsized":"2023-12-19T18:14:54.063662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060228+0000","last_change":"2023-12-19T18:14:54.059917+0000","last_active":"2023-12-19T18:14:54.060228+0000","last_peered":"2023-12-19T18:14:54.060228+0000","last_clean":"2023-12-19T18:14:54.060228+0000","last_became_active":"2023-12-19T18:14:54.059674+0000","last_became_peered":"2023-12-19T18:14:54.059674+0000","last_unstale":"2023-12-19T18:14:54.060228+0000","last_undegraded":"2023-12-19T18:14:54.060228+0000","last_fullsized":"2023-12-19T18:14:54.060228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074581+0000","last_change":"2023-12-19T18:14:54.073659+0000","last_active":"2023-12-19T18:14:54.074581+0000","last_peered":"2023-12-19T18:14:54.074581+0000","last_clean":"2023-12-19T18:14:54.074581+0000","last_became_active":"2023-12-19T18:14:54.072947+0000","last_became_peered":"2023-12-19T18:14:54.072947+0000","last_unstale":"2023-12-19T18:14:54.074581+0000","last_undegraded":"2023-12-19T18:14:54.074581+0000","last_fullsized":"2023-12-19T18:14:54.074581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067036+0000","last_change":"2023-12-19T18:14:54.066189+0000","last_active":"2023-12-19T18:14:54.067036+0000","last_peered":"2023-12-19T18:14:54.067036+0000","last_clean":"2023-12-19T18:14:54.067036+0000","last_became_active":"2023-12-19T18:14:54.065921+0000","last_became_peered":"2023-12-19T18:14:54.065921+0000","last_unstale":"2023-12-19T18:14:54.067036+0000","last_undegraded":"2023-12-19T18:14:54.067036+0000","last_fullsized":"2023-12-19T18:14:54.067036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061440+0000","last_change":"2023-12-19T18:14:54.061372+0000","last_active":"2023-12-19T18:14:54.061440+0000","last_peered":"2023-12-19T18:14:54.061440+0000","last_clean":"2023-12-19T18:14:54.061440+0000","last_became_active":"2023-12-19T18:14:54.060949+0000","last_became_peered":"2023-12-19T18:14:54.060949+0000","last_unstale":"2023-12-19T18:14:54.061440+0000","last_undegraded":"2023-12-19T18:14:54.061440+0000","last_fullsized":"2023-12-19T18:14:54.061440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073948+0000","last_change":"2023-12-19T18:14:54.073038+0000","last_active":"2023-12-19T18:14:54.073948+0000","last_peered":"2023-12-19T18:14:54.073948+0000","last_clean":"2023-12-19T18:14:54.073948+0000","last_became_active":"2023-12-19T18:14:54.071844+0000","last_became_peered":"2023-12-19T18:14:54.071844+0000","last_unstale":"2023-12-19T18:14:54.073948+0000","last_undegraded":"2023-12-19T18:14:54.073948+0000","last_fullsized":"2023-12-19T18:14:54.073948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070883+0000","last_change":"2023-12-19T18:14:54.070817+0000","last_active":"2023-12-19T18:14:54.070883+0000","last_peered":"2023-12-19T18:14:54.070883+0000","last_clean":"2023-12-19T18:14:54.070883+0000","last_became_active":"2023-12-19T18:14:54.068216+0000","last_became_peered":"2023-12-19T18:14:54.068216+0000","last_unstale":"2023-12-19T18:14:54.070883+0000","last_undegraded":"2023-12-19T18:14:54.070883+0000","last_fullsized":"2023-12-19T18:14:54.070883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063273+0000","last_change":"2023-12-19T18:14:54.060522+0000","last_active":"2023-12-19T18:14:54.063273+0000","last_peered":"2023-12-19T18:14:54.063273+0000","last_clean":"2023-12-19T18:14:54.063273+0000","last_became_active":"2023-12-19T18:14:54.059641+0000","last_became_peered":"2023-12-19T18:14:54.059641+0000","last_unstale":"2023-12-19T18:14:54.063273+0000","last_undegraded":"2023-12-19T18:14:54.063273+0000","last_fullsized":"2023-12-19T18:14:54.063273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065928+0000","last_change":"2023-12-19T18:14:54.065766+0000","last_active":"2023-12-19T18:14:54.065928+0000","last_peered":"2023-12-19T18:14:54.065928+0000","last_clean":"2023-12-19T18:14:54.065928+0000","last_became_active":"2023-12-19T18:14:54.058571+0000","last_became_peered":"2023-12-19T18:14:54.058571+0000","last_unstale":"2023-12-19T18:14:54.065928+0000","last_undegraded":"2023-12-19T18:14:54.065928+0000","last_fullsized":"2023-12-19T18:14:54.065928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:47.213095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070429+0000","last_change":"2023-12-19T18:14:54.067798+0000","last_active":"2023-12-19T18:14:54.070429+0000","last_peered":"2023-12-19T18:14:54.070429+0000","last_clean":"2023-12-19T18:14:54.070429+0000","last_became_active":"2023-12-19T18:14:54.067576+0000","last_became_peered":"2023-12-19T18:14:54.067576+0000","last_unstale":"2023-12-19T18:14:54.070429+0000","last_undegraded":"2023-12-19T18:14:54.070429+0000","last_fullsized":"2023-12-19T18:14:54.070429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069889+0000","last_change":"2023-12-19T18:14:54.069741+0000","last_active":"2023-12-19T18:14:54.069889+0000","last_peered":"2023-12-19T18:14:54.069889+0000","last_clean":"2023-12-19T18:14:54.069889+0000","last_became_active":"2023-12-19T18:14:54.069513+0000","last_became_peered":"2023-12-19T18:14:54.069513+0000","last_unstale":"2023-12-19T18:14:54.069889+0000","last_undegraded":"2023-12-19T18:14:54.069889+0000","last_fullsized":"2023-12-19T18:14:54.069889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070495+0000","last_change":"2023-12-19T18:14:54.070403+0000","last_active":"2023-12-19T18:14:54.070495+0000","last_peered":"2023-12-19T18:14:54.070495+0000","last_clean":"2023-12-19T18:14:54.070495+0000","last_became_active":"2023-12-19T18:14:54.069801+0000","last_became_peered":"2023-12-19T18:14:54.069801+0000","last_unstale":"2023-12-19T18:14:54.070495+0000","last_undegraded":"2023-12-19T18:14:54.070495+0000","last_fullsized":"2023-12-19T18:14:54.070495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068551+0000","last_change":"2023-12-19T18:14:53.563064+0000","last_active":"2023-12-19T18:14:54.068551+0000","last_peered":"2023-12-19T18:14:54.068551+0000","last_clean":"2023-12-19T18:14:54.068551+0000","last_became_active":"2023-12-19T18:14:53.561534+0000","last_became_peered":"2023-12-19T18:14:53.561534+0000","last_unstale":"2023-12-19T18:14:54.068551+0000","last_undegraded":"2023-12-19T18:14:54.068551+0000","last_fullsized":"2023-12-19T18:14:54.068551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051639+0000","last_change":"2023-12-19T18:14:54.051570+0000","last_active":"2023-12-19T18:14:54.051639+0000","last_peered":"2023-12-19T18:14:54.051639+0000","last_clean":"2023-12-19T18:14:54.051639+0000","last_became_active":"2023-12-19T18:14:54.051173+0000","last_became_peered":"2023-12-19T18:14:54.051173+0000","last_unstale":"2023-12-19T18:14:54.051639+0000","last_undegraded":"2023-12-19T18:14:54.051639+0000","last_fullsized":"2023-12-19T18:14:54.051639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073383+0000","last_change":"2023-12-19T18:14:53.560763+0000","last_active":"2023-12-19T18:14:54.073383+0000","last_peered":"2023-12-19T18:14:54.073383+0000","last_clean":"2023-12-19T18:14:54.073383+0000","last_became_active":"2023-12-19T18:14:53.558648+0000","last_became_peered":"2023-12-19T18:14:53.558648+0000","last_unstale":"2023-12-19T18:14:54.073383+0000","last_undegraded":"2023-12-19T18:14:54.073383+0000","last_fullsized":"2023-12-19T18:14:54.073383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:51.179899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066389+0000","last_change":"2023-12-19T18:14:53.560483+0000","last_active":"2023-12-19T18:14:54.066389+0000","last_peered":"2023-12-19T18:14:54.066389+0000","last_clean":"2023-12-19T18:14:54.066389+0000","last_became_active":"2023-12-19T18:14:53.560244+0000","last_became_peered":"2023-12-19T18:14:53.560244+0000","last_unstale":"2023-12-19T18:14:54.066389+0000","last_undegraded":"2023-12-19T18:14:54.066389+0000","last_fullsized":"2023-12-19T18:14:54.066389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072273+0000","last_change":"2023-12-19T18:14:54.071383+0000","last_active":"2023-12-19T18:14:54.072273+0000","last_peered":"2023-12-19T18:14:54.072273+0000","last_clean":"2023-12-19T18:14:54.072273+0000","last_became_active":"2023-12-19T18:14:54.070954+0000","last_became_peered":"2023-12-19T18:14:54.070954+0000","last_unstale":"2023-12-19T18:14:54.072273+0000","last_undegraded":"2023-12-19T18:14:54.072273+0000","last_fullsized":"2023-12-19T18:14:54.072273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053551+0000","last_change":"2023-12-19T18:14:53.558346+0000","last_active":"2023-12-19T18:14:54.053551+0000","last_peered":"2023-12-19T18:14:54.053551+0000","last_clean":"2023-12-19T18:14:54.053551+0000","last_became_active":"2023-12-19T18:14:53.558124+0000","last_became_peered":"2023-12-19T18:14:53.558124+0000","last_unstale":"2023-12-19T18:14:54.053551+0000","last_undegraded":"2023-12-19T18:14:54.053551+0000","last_fullsized":"2023-12-19T18:14:54.053551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066236+0000","last_change":"2023-12-19T18:14:54.064228+0000","last_active":"2023-12-19T18:14:54.066236+0000","last_peered":"2023-12-19T18:14:54.066236+0000","last_clean":"2023-12-19T18:14:54.066236+0000","last_became_active":"2023-12-19T18:14:54.063991+0000","last_became_peered":"2023-12-19T18:14:54.063991+0000","last_unstale":"2023-12-19T18:14:54.066236+0000","last_undegraded":"2023-12-19T18:14:54.066236+0000","last_fullsized":"2023-12-19T18:14:54.066236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064163+0000","last_change":"2023-12-19T18:14:53.554048+0000","last_active":"2023-12-19T18:14:54.064163+0000","last_peered":"2023-12-19T18:14:54.064163+0000","last_clean":"2023-12-19T18:14:54.064163+0000","last_became_active":"2023-12-19T18:14:53.553811+0000","last_became_peered":"2023-12-19T18:14:53.553811+0000","last_unstale":"2023-12-19T18:14:54.064163+0000","last_undegraded":"2023-12-19T18:14:54.064163+0000","last_fullsized":"2023-12-19T18:14:54.064163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073051+0000","last_change":"2023-12-19T18:14:54.072974+0000","last_active":"2023-12-19T18:14:54.073051+0000","last_peered":"2023-12-19T18:14:54.073051+0000","last_clean":"2023-12-19T18:14:54.073051+0000","last_became_active":"2023-12-19T18:14:54.072176+0000","last_became_peered":"2023-12-19T18:14:54.072176+0000","last_unstale":"2023-12-19T18:14:54.073051+0000","last_undegraded":"2023-12-19T18:14:54.073051+0000","last_fullsized":"2023-12-19T18:14:54.073051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.043565+0000","last_change":"2023-12-19T18:14:53.557947+0000","last_active":"2023-12-19T18:14:54.043565+0000","last_peered":"2023-12-19T18:14:54.043565+0000","last_clean":"2023-12-19T18:14:54.043565+0000","last_became_active":"2023-12-19T18:14:53.557713+0000","last_became_peered":"2023-12-19T18:14:53.557713+0000","last_unstale":"2023-12-19T18:14:54.043565+0000","last_undegraded":"2023-12-19T18:14:54.043565+0000","last_fullsized":"2023-12-19T18:14:54.043565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056501+0000","last_change":"2023-12-19T18:14:54.056275+0000","last_active":"2023-12-19T18:14:54.056501+0000","last_peered":"2023-12-19T18:14:54.056501+0000","last_clean":"2023-12-19T18:14:54.056501+0000","last_became_active":"2023-12-19T18:14:54.056002+0000","last_became_peered":"2023-12-19T18:14:54.056002+0000","last_unstale":"2023-12-19T18:14:54.056501+0000","last_undegraded":"2023-12-19T18:14:54.056501+0000","last_fullsized":"2023-12-19T18:14:54.056501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:30.847801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073216+0000","last_change":"2023-12-19T18:14:53.568016+0000","last_active":"2023-12-19T18:14:54.073216+0000","last_peered":"2023-12-19T18:14:54.073216+0000","last_clean":"2023-12-19T18:14:54.073216+0000","last_became_active":"2023-12-19T18:14:53.567634+0000","last_became_peered":"2023-12-19T18:14:53.567634+0000","last_unstale":"2023-12-19T18:14:54.073216+0000","last_undegraded":"2023-12-19T18:14:54.073216+0000","last_fullsized":"2023-12-19T18:14:54.073216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075897+0000","last_change":"2023-12-19T18:14:54.075827+0000","last_active":"2023-12-19T18:14:54.075897+0000","last_peered":"2023-12-19T18:14:54.075897+0000","last_clean":"2023-12-19T18:14:54.075897+0000","last_became_active":"2023-12-19T18:14:54.074747+0000","last_became_peered":"2023-12-19T18:14:54.074747+0000","last_unstale":"2023-12-19T18:14:54.075897+0000","last_undegraded":"2023-12-19T18:14:54.075897+0000","last_fullsized":"2023-12-19T18:14:54.075897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073455+0000","last_change":"2023-12-19T18:14:53.547427+0000","last_active":"2023-12-19T18:14:54.073455+0000","last_peered":"2023-12-19T18:14:54.073455+0000","last_clean":"2023-12-19T18:14:54.073455+0000","last_became_active":"2023-12-19T18:14:53.546733+0000","last_became_peered":"2023-12-19T18:14:53.546733+0000","last_unstale":"2023-12-19T18:14:54.073455+0000","last_undegraded":"2023-12-19T18:14:54.073455+0000","last_fullsized":"2023-12-19T18:14:54.073455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:13:12.090625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069234+0000","last_change":"2023-12-19T18:14:54.069070+0000","last_active":"2023-12-19T18:14:54.069234+0000","last_peered":"2023-12-19T18:14:54.069234+0000","last_clean":"2023-12-19T18:14:54.069234+0000","last_became_active":"2023-12-19T18:14:54.066495+0000","last_became_peered":"2023-12-19T18:14:54.066495+0000","last_unstale":"2023-12-19T18:14:54.069234+0000","last_undegraded":"2023-12-19T18:14:54.069234+0000","last_fullsized":"2023-12-19T18:14:54.069234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068132+0000","last_change":"2023-12-19T18:14:53.566349+0000","last_active":"2023-12-19T18:14:54.068132+0000","last_peered":"2023-12-19T18:14:54.068132+0000","last_clean":"2023-12-19T18:14:54.068132+0000","last_became_active":"2023-12-19T18:14:53.565909+0000","last_became_peered":"2023-12-19T18:14:53.565909+0000","last_unstale":"2023-12-19T18:14:54.068132+0000","last_undegraded":"2023-12-19T18:14:54.068132+0000","last_fullsized":"2023-12-19T18:14:54.068132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073929+0000","last_change":"2023-12-19T18:14:54.072903+0000","last_active":"2023-12-19T18:14:54.073929+0000","last_peered":"2023-12-19T18:14:54.073929+0000","last_clean":"2023-12-19T18:14:54.073929+0000","last_became_active":"2023-12-19T18:14:54.072678+0000","last_became_peered":"2023-12-19T18:14:54.072678+0000","last_unstale":"2023-12-19T18:14:54.073929+0000","last_undegraded":"2023-12-19T18:14:54.073929+0000","last_fullsized":"2023-12-19T18:14:54.073929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.052955+0000","last_change":"2023-12-19T18:14:53.561604+0000","last_active":"2023-12-19T18:14:54.052955+0000","last_peered":"2023-12-19T18:14:54.052955+0000","last_clean":"2023-12-19T18:14:54.052955+0000","last_became_active":"2023-12-19T18:14:53.561372+0000","last_became_peered":"2023-12-19T18:14:53.561372+0000","last_unstale":"2023-12-19T18:14:54.052955+0000","last_undegraded":"2023-12-19T18:14:54.052955+0000","last_fullsized":"2023-12-19T18:14:54.052955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:00.255541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.057807+0000","last_change":"2023-12-19T18:14:54.057328+0000","last_active":"2023-12-19T18:14:54.057807+0000","last_peered":"2023-12-19T18:14:54.057807+0000","last_clean":"2023-12-19T18:14:54.057807+0000","last_became_active":"2023-12-19T18:14:54.055611+0000","last_became_peered":"2023-12-19T18:14:54.055611+0000","last_unstale":"2023-12-19T18:14:54.057807+0000","last_undegraded":"2023-12-19T18:14:54.057807+0000","last_fullsized":"2023-12-19T18:14:54.057807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:12.180767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.043960+0000","last_change":"2023-12-19T18:14:53.562764+0000","last_active":"2023-12-19T18:14:54.043960+0000","last_peered":"2023-12-19T18:14:54.043960+0000","last_clean":"2023-12-19T18:14:54.043960+0000","last_became_active":"2023-12-19T18:14:53.562532+0000","last_became_peered":"2023-12-19T18:14:53.562532+0000","last_unstale":"2023-12-19T18:14:54.043960+0000","last_undegraded":"2023-12-19T18:14:54.043960+0000","last_fullsized":"2023-12-19T18:14:54.043960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069588+0000","last_change":"2023-12-19T18:14:54.068780+0000","last_active":"2023-12-19T18:14:54.069588+0000","last_peered":"2023-12-19T18:14:54.069588+0000","last_clean":"2023-12-19T18:14:54.069588+0000","last_became_active":"2023-12-19T18:14:54.068556+0000","last_became_peered":"2023-12-19T18:14:54.068556+0000","last_unstale":"2023-12-19T18:14:54.069588+0000","last_undegraded":"2023-12-19T18:14:54.069588+0000","last_fullsized":"2023-12-19T18:14:54.069588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074393+0000","last_change":"2023-12-19T18:14:53.565084+0000","last_active":"2023-12-19T18:14:54.074393+0000","last_peered":"2023-12-19T18:14:54.074393+0000","last_clean":"2023-12-19T18:14:54.074393+0000","last_became_active":"2023-12-19T18:14:53.564836+0000","last_became_peered":"2023-12-19T18:14:53.564836+0000","last_unstale":"2023-12-19T18:14:54.074393+0000","last_undegraded":"2023-12-19T18:14:54.074393+0000","last_fullsized":"2023-12-19T18:14:54.074393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.076309+0000","last_change":"2023-12-19T18:14:54.075967+0000","last_active":"2023-12-19T18:14:54.076309+0000","last_peered":"2023-12-19T18:14:54.076309+0000","last_clean":"2023-12-19T18:14:54.076309+0000","last_became_active":"2023-12-19T18:14:54.074991+0000","last_became_peered":"2023-12-19T18:14:54.074991+0000","last_unstale":"2023-12-19T18:14:54.076309+0000","last_undegraded":"2023-12-19T18:14:54.076309+0000","last_fullsized":"2023-12-19T18:14:54.076309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061847+0000","last_change":"2023-12-19T18:14:53.550365+0000","last_active":"2023-12-19T18:14:54.061847+0000","last_peered":"2023-12-19T18:14:54.061847+0000","last_clean":"2023-12-19T18:14:54.061847+0000","last_became_active":"2023-12-19T18:14:53.550059+0000","last_became_peered":"2023-12-19T18:14:53.550059+0000","last_unstale":"2023-12-19T18:14:54.061847+0000","last_undegraded":"2023-12-19T18:14:54.061847+0000","last_fullsized":"2023-12-19T18:14:54.061847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050074+0000","last_change":"2023-12-19T18:14:54.048927+0000","last_active":"2023-12-19T18:14:54.050074+0000","last_peered":"2023-12-19T18:14:54.050074+0000","last_clean":"2023-12-19T18:14:54.050074+0000","last_became_active":"2023-12-19T18:14:54.048515+0000","last_became_peered":"2023-12-19T18:14:54.048515+0000","last_unstale":"2023-12-19T18:14:54.050074+0000","last_undegraded":"2023-12-19T18:14:54.050074+0000","last_fullsized":"2023-12-19T18:14:54.050074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064254+0000","last_change":"2023-12-19T18:14:53.567656+0000","last_active":"2023-12-19T18:14:54.064254+0000","last_peered":"2023-12-19T18:14:54.064254+0000","last_clean":"2023-12-19T18:14:54.064254+0000","last_became_active":"2023-12-19T18:14:53.567278+0000","last_became_peered":"2023-12-19T18:14:53.567278+0000","last_unstale":"2023-12-19T18:14:54.064254+0000","last_undegraded":"2023-12-19T18:14:54.064254+0000","last_fullsized":"2023-12-19T18:14:54.064254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065685+0000","last_change":"2023-12-19T18:14:54.065628+0000","last_active":"2023-12-19T18:14:54.065685+0000","last_peered":"2023-12-19T18:14:54.065685+0000","last_clean":"2023-12-19T18:14:54.065685+0000","last_became_active":"2023-12-19T18:14:54.065414+0000","last_became_peered":"2023-12-19T18:14:54.065414+0000","last_unstale":"2023-12-19T18:14:54.065685+0000","last_undegraded":"2023-12-19T18:14:54.065685+0000","last_fullsized":"2023-12-19T18:14:54.065685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065227+0000","last_change":"2023-12-19T18:14:53.559407+0000","last_active":"2023-12-19T18:14:54.065227+0000","last_peered":"2023-12-19T18:14:54.065227+0000","last_clean":"2023-12-19T18:14:54.065227+0000","last_became_active":"2023-12-19T18:14:53.559062+0000","last_became_peered":"2023-12-19T18:14:53.559062+0000","last_unstale":"2023-12-19T18:14:54.065227+0000","last_undegraded":"2023-12-19T18:14:54.065227+0000","last_fullsized":"2023-12-19T18:14:54.065227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061207+0000","last_change":"2023-12-19T18:14:54.061139+0000","last_active":"2023-12-19T18:14:54.061207+0000","last_peered":"2023-12-19T18:14:54.061207+0000","last_clean":"2023-12-19T18:14:54.061207+0000","last_became_active":"2023-12-19T18:14:54.060925+0000","last_became_peered":"2023-12-19T18:14:54.060925+0000","last_unstale":"2023-12-19T18:14:54.061207+0000","last_undegraded":"2023-12-19T18:14:54.061207+0000","last_fullsized":"2023-12-19T18:14:54.061207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067789+0000","last_change":"2023-12-19T18:14:53.563513+0000","last_active":"2023-12-19T18:14:54.067789+0000","last_peered":"2023-12-19T18:14:54.067789+0000","last_clean":"2023-12-19T18:14:54.067789+0000","last_became_active":"2023-12-19T18:14:53.562860+0000","last_became_peered":"2023-12-19T18:14:53.562860+0000","last_unstale":"2023-12-19T18:14:54.067789+0000","last_undegraded":"2023-12-19T18:14:54.067789+0000","last_fullsized":"2023-12-19T18:14:54.067789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:59:49.969450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072181+0000","last_change":"2023-12-19T18:14:54.071273+0000","last_active":"2023-12-19T18:14:54.072181+0000","last_peered":"2023-12-19T18:14:54.072181+0000","last_clean":"2023-12-19T18:14:54.072181+0000","last_became_active":"2023-12-19T18:14:54.070908+0000","last_became_peered":"2023-12-19T18:14:54.070908+0000","last_unstale":"2023-12-19T18:14:54.072181+0000","last_undegraded":"2023-12-19T18:14:54.072181+0000","last_fullsized":"2023-12-19T18:14:54.072181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:27:34.736465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.055663+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:14:54.055663+0000","last_peered":"2023-12-19T18:14:54.055663+0000","last_clean":"2023-12-19T18:14:54.055663+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:14:54.055663+0000","last_undegraded":"2023-12-19T18:14:54.055663+0000","last_fullsized":"2023-12-19T18:14:54.055663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053073+0000","last_change":"2023-12-19T18:14:53.557129+0000","last_active":"2023-12-19T18:14:54.053073+0000","last_peered":"2023-12-19T18:14:54.053073+0000","last_clean":"2023-12-19T18:14:54.053073+0000","last_became_active":"2023-12-19T18:14:53.556812+0000","last_became_peered":"2023-12-19T18:14:53.556812+0000","last_unstale":"2023-12-19T18:14:54.053073+0000","last_undegraded":"2023-12-19T18:14:54.053073+0000","last_fullsized":"2023-12-19T18:14:54.053073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:53.113355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051508+0000","last_change":"2023-12-19T18:14:54.051423+0000","last_active":"2023-12-19T18:14:54.051508+0000","last_peered":"2023-12-19T18:14:54.051508+0000","last_clean":"2023-12-19T18:14:54.051508+0000","last_became_active":"2023-12-19T18:14:54.050816+0000","last_became_peered":"2023-12-19T18:14:54.050816+0000","last_unstale":"2023-12-19T18:14:54.051508+0000","last_undegraded":"2023-12-19T18:14:54.051508+0000","last_fullsized":"2023-12-19T18:14:54.051508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068099+0000","last_change":"2023-12-19T18:14:53.558583+0000","last_active":"2023-12-19T18:14:54.068099+0000","last_peered":"2023-12-19T18:14:54.068099+0000","last_clean":"2023-12-19T18:14:54.068099+0000","last_became_active":"2023-12-19T18:14:53.558000+0000","last_became_peered":"2023-12-19T18:14:53.558000+0000","last_unstale":"2023-12-19T18:14:54.068099+0000","last_undegraded":"2023-12-19T18:14:54.068099+0000","last_fullsized":"2023-12-19T18:14:54.068099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:53.818952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068077+0000","last_change":"2023-12-19T18:14:54.067973+0000","last_active":"2023-12-19T18:14:54.068077+0000","last_peered":"2023-12-19T18:14:54.068077+0000","last_clean":"2023-12-19T18:14:54.068077+0000","last_became_active":"2023-12-19T18:14:54.065516+0000","last_became_peered":"2023-12-19T18:14:54.065516+0000","last_unstale":"2023-12-19T18:14:54.068077+0000","last_undegraded":"2023-12-19T18:14:54.068077+0000","last_fullsized":"2023-12-19T18:14:54.068077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.058632+0000","last_change":"2023-12-19T18:14:53.563881+0000","last_active":"2023-12-19T18:14:54.058632+0000","last_peered":"2023-12-19T18:14:54.058632+0000","last_clean":"2023-12-19T18:14:54.058632+0000","last_became_active":"2023-12-19T18:14:53.563564+0000","last_became_peered":"2023-12-19T18:14:53.563564+0000","last_unstale":"2023-12-19T18:14:54.058632+0000","last_undegraded":"2023-12-19T18:14:54.058632+0000","last_fullsized":"2023-12-19T18:14:54.058632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060261+0000","last_change":"2023-12-19T18:14:54.060208+0000","last_active":"2023-12-19T18:14:54.060261+0000","last_peered":"2023-12-19T18:14:54.060261+0000","last_clean":"2023-12-19T18:14:54.060261+0000","last_became_active":"2023-12-19T18:14:54.059235+0000","last_became_peered":"2023-12-19T18:14:54.059235+0000","last_unstale":"2023-12-19T18:14:54.060261+0000","last_undegraded":"2023-12-19T18:14:54.060261+0000","last_fullsized":"2023-12-19T18:14:54.060261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065436+0000","last_change":"2023-12-19T18:14:53.564065+0000","last_active":"2023-12-19T18:14:54.065436+0000","last_peered":"2023-12-19T18:14:54.065436+0000","last_clean":"2023-12-19T18:14:54.065436+0000","last_became_active":"2023-12-19T18:14:53.563841+0000","last_became_peered":"2023-12-19T18:14:53.563841+0000","last_unstale":"2023-12-19T18:14:54.065436+0000","last_undegraded":"2023-12-19T18:14:54.065436+0000","last_fullsized":"2023-12-19T18:14:54.065436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:46.161691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060843+0000","last_change":"2023-12-19T18:14:54.060734+0000","last_active":"2023-12-19T18:14:54.060843+0000","last_peered":"2023-12-19T18:14:54.060843+0000","last_clean":"2023-12-19T18:14:54.060843+0000","last_became_active":"2023-12-19T18:14:54.060328+0000","last_became_peered":"2023-12-19T18:14:54.060328+0000","last_unstale":"2023-12-19T18:14:54.060843+0000","last_undegraded":"2023-12-19T18:14:54.060843+0000","last_fullsized":"2023-12-19T18:14:54.060843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050368+0000","last_change":"2023-12-19T18:14:53.565164+0000","last_active":"2023-12-19T18:14:54.050368+0000","last_peered":"2023-12-19T18:14:54.050368+0000","last_clean":"2023-12-19T18:14:54.050368+0000","last_became_active":"2023-12-19T18:14:53.564538+0000","last_became_peered":"2023-12-19T18:14:53.564538+0000","last_unstale":"2023-12-19T18:14:54.050368+0000","last_undegraded":"2023-12-19T18:14:54.050368+0000","last_fullsized":"2023-12-19T18:14:54.050368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068274+0000","last_change":"2023-12-19T18:14:54.068080+0000","last_active":"2023-12-19T18:14:54.068274+0000","last_peered":"2023-12-19T18:14:54.068274+0000","last_clean":"2023-12-19T18:14:54.068274+0000","last_became_active":"2023-12-19T18:14:54.067528+0000","last_became_peered":"2023-12-19T18:14:54.067528+0000","last_unstale":"2023-12-19T18:14:54.068274+0000","last_undegraded":"2023-12-19T18:14:54.068274+0000","last_fullsized":"2023-12-19T18:14:54.068274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067999+0000","last_change":"2023-12-19T18:14:53.562234+0000","last_active":"2023-12-19T18:14:54.067999+0000","last_peered":"2023-12-19T18:14:54.067999+0000","last_clean":"2023-12-19T18:14:54.067999+0000","last_became_active":"2023-12-19T18:14:53.561326+0000","last_became_peered":"2023-12-19T18:14:53.561326+0000","last_unstale":"2023-12-19T18:14:54.067999+0000","last_undegraded":"2023-12-19T18:14:54.067999+0000","last_fullsized":"2023-12-19T18:14:54.067999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074010+0000","last_change":"2023-12-19T18:14:54.073543+0000","last_active":"2023-12-19T18:14:54.074010+0000","last_peered":"2023-12-19T18:14:54.074010+0000","last_clean":"2023-12-19T18:14:54.074010+0000","last_became_active":"2023-12-19T18:14:54.073247+0000","last_became_peered":"2023-12-19T18:14:54.073247+0000","last_unstale":"2023-12-19T18:14:54.074010+0000","last_undegraded":"2023-12-19T18:14:54.074010+0000","last_fullsized":"2023-12-19T18:14:54.074010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.045518+0000","last_change":"2023-12-19T18:14:53.551251+0000","last_active":"2023-12-19T18:14:54.045518+0000","last_peered":"2023-12-19T18:14:54.045518+0000","last_clean":"2023-12-19T18:14:54.045518+0000","last_became_active":"2023-12-19T18:14:53.550691+0000","last_became_peered":"2023-12-19T18:14:53.550691+0000","last_unstale":"2023-12-19T18:14:54.045518+0000","last_undegraded":"2023-12-19T18:14:54.045518+0000","last_fullsized":"2023-12-19T18:14:54.045518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.071102+0000","last_change":"2023-12-19T18:14:54.070766+0000","last_active":"2023-12-19T18:14:54.071102+0000","last_peered":"2023-12-19T18:14:54.071102+0000","last_clean":"2023-12-19T18:14:54.071102+0000","last_became_active":"2023-12-19T18:14:54.069495+0000","last_became_peered":"2023-12-19T18:14:54.069495+0000","last_unstale":"2023-12-19T18:14:54.071102+0000","last_undegraded":"2023-12-19T18:14:54.071102+0000","last_fullsized":"2023-12-19T18:14:54.071102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.076793+0000","last_change":"2023-12-19T18:14:53.563561+0000","last_active":"2023-12-19T18:14:54.076793+0000","last_peered":"2023-12-19T18:14:54.076793+0000","last_clean":"2023-12-19T18:14:54.076793+0000","last_became_active":"2023-12-19T18:14:53.563126+0000","last_became_peered":"2023-12-19T18:14:53.563126+0000","last_unstale":"2023-12-19T18:14:54.076793+0000","last_undegraded":"2023-12-19T18:14:54.076793+0000","last_fullsized":"2023-12-19T18:14:54.076793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.059619+0000","last_change":"2023-12-19T18:14:54.059362+0000","last_active":"2023-12-19T18:14:54.059619+0000","last_peered":"2023-12-19T18:14:54.059619+0000","last_clean":"2023-12-19T18:14:54.059619+0000","last_became_active":"2023-12-19T18:14:54.059089+0000","last_became_peered":"2023-12-19T18:14:54.059089+0000","last_unstale":"2023-12-19T18:14:54.059619+0000","last_undegraded":"2023-12-19T18:14:54.059619+0000","last_fullsized":"2023-12-19T18:14:54.059619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.045320+0000","last_change":"2023-12-19T18:14:53.551856+0000","last_active":"2023-12-19T18:14:54.045320+0000","last_peered":"2023-12-19T18:14:54.045320+0000","last_clean":"2023-12-19T18:14:54.045320+0000","last_became_active":"2023-12-19T18:14:53.551105+0000","last_became_peered":"2023-12-19T18:14:53.551105+0000","last_unstale":"2023-12-19T18:14:54.045320+0000","last_undegraded":"2023-12-19T18:14:54.045320+0000","last_fullsized":"2023-12-19T18:14:54.045320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065908+0000","last_change":"2023-12-19T18:14:54.065539+0000","last_active":"2023-12-19T18:14:54.065908+0000","last_peered":"2023-12-19T18:14:54.065908+0000","last_clean":"2023-12-19T18:14:54.065908+0000","last_became_active":"2023-12-19T18:14:54.056448+0000","last_became_peered":"2023-12-19T18:14:54.056448+0000","last_unstale":"2023-12-19T18:14:54.065908+0000","last_undegraded":"2023-12-19T18:14:54.065908+0000","last_fullsized":"2023-12-19T18:14:54.065908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.045692+0000","last_change":"2023-12-19T18:14:53.554185+0000","last_active":"2023-12-19T18:14:54.045692+0000","last_peered":"2023-12-19T18:14:54.045692+0000","last_clean":"2023-12-19T18:14:54.045692+0000","last_became_active":"2023-12-19T18:14:53.553750+0000","last_became_peered":"2023-12-19T18:14:53.553750+0000","last_unstale":"2023-12-19T18:14:54.045692+0000","last_undegraded":"2023-12-19T18:14:54.045692+0000","last_fullsized":"2023-12-19T18:14:54.045692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.059756+0000","last_change":"2023-12-19T18:14:54.059501+0000","last_active":"2023-12-19T18:14:54.059756+0000","last_peered":"2023-12-19T18:14:54.059756+0000","last_clean":"2023-12-19T18:14:54.059756+0000","last_became_active":"2023-12-19T18:14:54.059055+0000","last_became_peered":"2023-12-19T18:14:54.059055+0000","last_unstale":"2023-12-19T18:14:54.059756+0000","last_undegraded":"2023-12-19T18:14:54.059756+0000","last_fullsized":"2023-12-19T18:14:54.059756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066496+0000","last_change":"2023-12-19T18:14:53.566640+0000","last_active":"2023-12-19T18:14:54.066496+0000","last_peered":"2023-12-19T18:14:54.066496+0000","last_clean":"2023-12-19T18:14:54.066496+0000","last_became_active":"2023-12-19T18:14:53.566413+0000","last_became_peered":"2023-12-19T18:14:53.566413+0000","last_unstale":"2023-12-19T18:14:54.066496+0000","last_undegraded":"2023-12-19T18:14:54.066496+0000","last_fullsized":"2023-12-19T18:14:54.066496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074538+0000","last_change":"2023-12-19T18:14:54.073374+0000","last_active":"2023-12-19T18:14:54.074538+0000","last_peered":"2023-12-19T18:14:54.074538+0000","last_clean":"2023-12-19T18:14:54.074538+0000","last_became_active":"2023-12-19T18:14:54.072676+0000","last_became_peered":"2023-12-19T18:14:54.072676+0000","last_unstale":"2023-12-19T18:14:54.074538+0000","last_undegraded":"2023-12-19T18:14:54.074538+0000","last_fullsized":"2023-12-19T18:14:54.074538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073825+0000","last_change":"2023-12-19T18:14:53.549245+0000","last_active":"2023-12-19T18:14:54.073825+0000","last_peered":"2023-12-19T18:14:54.073825+0000","last_clean":"2023-12-19T18:14:54.073825+0000","last_became_active":"2023-12-19T18:14:53.548992+0000","last_became_peered":"2023-12-19T18:14:53.548992+0000","last_unstale":"2023-12-19T18:14:54.073825+0000","last_undegraded":"2023-12-19T18:14:54.073825+0000","last_fullsized":"2023-12-19T18:14:54.073825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074558+0000","last_change":"2023-12-19T18:14:54.073488+0000","last_active":"2023-12-19T18:14:54.074558+0000","last_peered":"2023-12-19T18:14:54.074558+0000","last_clean":"2023-12-19T18:14:54.074558+0000","last_became_active":"2023-12-19T18:14:54.072567+0000","last_became_peered":"2023-12-19T18:14:54.072567+0000","last_unstale":"2023-12-19T18:14:54.074558+0000","last_undegraded":"2023-12-19T18:14:54.074558+0000","last_fullsized":"2023-12-19T18:14:54.074558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:22.428929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.057975+0000","last_change":"2023-12-19T18:14:54.057549+0000","last_active":"2023-12-19T18:14:54.057975+0000","last_peered":"2023-12-19T18:14:54.057975+0000","last_clean":"2023-12-19T18:14:54.057975+0000","last_became_active":"2023-12-19T18:14:54.056075+0000","last_became_peered":"2023-12-19T18:14:54.056075+0000","last_unstale":"2023-12-19T18:14:54.057975+0000","last_undegraded":"2023-12-19T18:14:54.057975+0000","last_fullsized":"2023-12-19T18:14:54.057975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:47.825262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064249+0000","last_change":"2023-12-19T18:14:53.555372+0000","last_active":"2023-12-19T18:14:54.064249+0000","last_peered":"2023-12-19T18:14:54.064249+0000","last_clean":"2023-12-19T18:14:54.064249+0000","last_became_active":"2023-12-19T18:14:53.555089+0000","last_became_peered":"2023-12-19T18:14:53.555089+0000","last_unstale":"2023-12-19T18:14:54.064249+0000","last_undegraded":"2023-12-19T18:14:54.064249+0000","last_fullsized":"2023-12-19T18:14:54.064249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065405+0000","last_change":"2023-12-19T18:14:54.065119+0000","last_active":"2023-12-19T18:14:54.065405+0000","last_peered":"2023-12-19T18:14:54.065405+0000","last_clean":"2023-12-19T18:14:54.065405+0000","last_became_active":"2023-12-19T18:14:54.064697+0000","last_became_peered":"2023-12-19T18:14:54.064697+0000","last_unstale":"2023-12-19T18:14:54.065405+0000","last_undegraded":"2023-12-19T18:14:54.065405+0000","last_fullsized":"2023-12-19T18:14:54.065405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063594+0000","last_change":"2023-12-19T18:14:53.556488+0000","last_active":"2023-12-19T18:14:54.063594+0000","last_peered":"2023-12-19T18:14:54.063594+0000","last_clean":"2023-12-19T18:14:54.063594+0000","last_became_active":"2023-12-19T18:14:53.555952+0000","last_became_peered":"2023-12-19T18:14:53.555952+0000","last_unstale":"2023-12-19T18:14:54.063594+0000","last_undegraded":"2023-12-19T18:14:54.063594+0000","last_fullsized":"2023-12-19T18:14:54.063594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070434+0000","last_change":"2023-12-19T18:14:54.070014+0000","last_active":"2023-12-19T18:14:54.070434+0000","last_peered":"2023-12-19T18:14:54.070434+0000","last_clean":"2023-12-19T18:14:54.070434+0000","last_became_active":"2023-12-19T18:14:54.067648+0000","last_became_peered":"2023-12-19T18:14:54.067648+0000","last_unstale":"2023-12-19T18:14:54.070434+0000","last_undegraded":"2023-12-19T18:14:54.070434+0000","last_fullsized":"2023-12-19T18:14:54.070434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067116+0000","last_change":"2023-12-19T18:14:53.566090+0000","last_active":"2023-12-19T18:14:54.067116+0000","last_peered":"2023-12-19T18:14:54.067116+0000","last_clean":"2023-12-19T18:14:54.067116+0000","last_became_active":"2023-12-19T18:14:53.565782+0000","last_became_peered":"2023-12-19T18:14:53.565782+0000","last_unstale":"2023-12-19T18:14:54.067116+0000","last_undegraded":"2023-12-19T18:14:54.067116+0000","last_fullsized":"2023-12-19T18:14:54.067116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063312+0000","last_change":"2023-12-19T18:14:54.060686+0000","last_active":"2023-12-19T18:14:54.063312+0000","last_peered":"2023-12-19T18:14:54.063312+0000","last_clean":"2023-12-19T18:14:54.063312+0000","last_became_active":"2023-12-19T18:14:54.059977+0000","last_became_peered":"2023-12-19T18:14:54.059977+0000","last_unstale":"2023-12-19T18:14:54.063312+0000","last_undegraded":"2023-12-19T18:14:54.063312+0000","last_fullsized":"2023-12-19T18:14:54.063312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073092+0000","last_change":"2023-12-19T18:14:53.558205+0000","last_active":"2023-12-19T18:14:54.073092+0000","last_peered":"2023-12-19T18:14:54.073092+0000","last_clean":"2023-12-19T18:14:54.073092+0000","last_became_active":"2023-12-19T18:14:53.557733+0000","last_became_peered":"2023-12-19T18:14:53.557733+0000","last_unstale":"2023-12-19T18:14:54.073092+0000","last_undegraded":"2023-12-19T18:14:54.073092+0000","last_fullsized":"2023-12-19T18:14:54.073092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929138,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":71,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927026,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199928680,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927698,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.54}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.613}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338824,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929206,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]}],"pool_statfs":[]}} 2023-12-19T18:15:00.120 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-19T18:15:00.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.395+0000 7f3d0aa2c640 1 Processor -- start 2023-12-19T18:15:00.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.395+0000 7f3d0aa2c640 1 -- start start 2023-12-19T18:15:00.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.397+0000 7f3d0aa2c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d0415adf0 0x7f3d0406d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:00.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.397+0000 7f3d0aa2c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3d04070c00 0x7f3d040712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:00.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.397+0000 7f3d0aa2c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d04071820 con 0x7f3d0406f500 2023-12-19T18:15:00.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.397+0000 7f3d0aa2c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d040703c0 con 0x7f3d04070c00 2023-12-19T18:15:00.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.397+0000 7f3d0aa2c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d0406d8c0 con 0x7f3d0415adf0 2023-12-19T18:15:00.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.398+0000 7f3d03fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d0415adf0 0x7f3d0406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:00.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.398+0000 7f3d037fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3d04070c00 0x7f3d040712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:00.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.398+0000 7f3d03fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d0415adf0 0x7f3d0406d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:39644/0 (socket says 172.21.15.27:39644) 2023-12-19T18:15:00.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.398+0000 7f3d03fff640 1 -- 172.21.15.27:0/2234128359 learned_addr learned my addr 172.21.15.27:0/2234128359 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:00.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.398+0000 7f3d03fff640 1 -- 172.21.15.27:0/2234128359 >> v2:172.21.15.192:3300/0 conn(0x7f3d04070c00 msgr2=0x7f3d040712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:00.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.398+0000 7f3d03fff640 1 --2- 172.21.15.27:0/2234128359 >> v2:172.21.15.192:3300/0 conn(0x7f3d04070c00 0x7f3d040712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.398+0000 7f3d03fff640 1 -- 172.21.15.27:0/2234128359 >> v1:172.21.15.192:6789/0 conn(0x7f3d0406f500 legacy=0x7f3d040c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:00.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.398+0000 7f3d03fff640 1 -- 172.21.15.27:0/2234128359 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0406b520 con 0x7f3d0415adf0 2023-12-19T18:15:00.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.400+0000 7f3d03fff640 1 --2- 172.21.15.27:0/2234128359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d0415adf0 0x7f3d0406d380 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009e50 tx=0x7f3cf402be50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6bb0cf5e9b65953 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:00.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.400+0000 7f3d02ffd640 1 -- 172.21.15.27:0/2234128359 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf402c9f0 con 0x7f3d0415adf0 2023-12-19T18:15:00.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.400+0000 7f3d02ffd640 1 -- 172.21.15.27:0/2234128359 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf402cb50 con 0x7f3d0415adf0 2023-12-19T18:15:00.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.400+0000 7f3d02ffd640 1 -- 172.21.15.27:0/2234128359 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf40343f0 con 0x7f3d0415adf0 2023-12-19T18:15:00.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.401+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/2234128359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d0415adf0 msgr2=0x7f3d0406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:00.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.401+0000 7f3d0aa2c640 1 --2- 172.21.15.27:0/2234128359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d0415adf0 0x7f3d0406d380 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009e50 tx=0x7f3cf402be50 comp rx=0 tx=0).stop 2023-12-19T18:15:00.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.401+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/2234128359 shutdown_connections 2023-12-19T18:15:00.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.401+0000 7f3d0aa2c640 1 --2- 172.21.15.27:0/2234128359 >> v2:172.21.15.192:3300/0 conn(0x7f3d04070c00 0x7f3d040712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.401+0000 7f3d0aa2c640 1 --2- 172.21.15.27:0/2234128359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d0415adf0 0x7f3d0406d380 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.401+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/2234128359 >> 172.21.15.27:0/2234128359 conn(0x7f3d040ceb50 msgr2=0x7f3d040cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:00.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.401+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/2234128359 shutdown_connections 2023-12-19T18:15:00.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.401+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/2234128359 wait complete. 2023-12-19T18:15:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.403+0000 7f3d0aa2c640 1 Processor -- start 2023-12-19T18:15:00.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.404+0000 7f3d0aa2c640 1 -- start start 2023-12-19T18:15:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.405+0000 7f3d0aa2c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d0406f500 0x7f3d040c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.405+0000 7f3d0aa2c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d04070c00 0x7f3d04154e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.405+0000 7f3d0aa2c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d0415adf0 0x7f3d0415d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.405+0000 7f3d0aa2c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d041f20c0 con 0x7f3d04070c00 2023-12-19T18:15:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.405+0000 7f3d0aa2c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040cc720 con 0x7f3d0415adf0 2023-12-19T18:15:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.405+0000 7f3d0aa2c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040b3350 con 0x7f3d0406f500 2023-12-19T18:15:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.405+0000 7f3d037fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d04070c00 0x7f3d04154e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.405+0000 7f3d037fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d04070c00 0x7f3d04154e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54640/0 (socket says 172.21.15.27:54640) 2023-12-19T18:15:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.405+0000 7f3d037fe640 1 -- 172.21.15.27:0/414842365 learned_addr learned my addr 172.21.15.27:0/414842365 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.405+0000 7f3d03fff640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d0406f500 0x7f3d040c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.406+0000 7f3d08fc9640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d0415adf0 0x7f3d0415d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:00.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.406+0000 7f3d037fe640 1 -- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d0406f500 msgr2=0x7f3d040c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:00.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.406+0000 7f3d037fe640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d0406f500 0x7f3d040c2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.406+0000 7f3d037fe640 1 -- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d0415adf0 msgr2=0x7f3d0415d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:00.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.406+0000 7f3d037fe640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d0415adf0 0x7f3d0415d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.406+0000 7f3d037fe640 1 -- 172.21.15.27:0/414842365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf4009b30 con 0x7f3d04070c00 2023-12-19T18:15:00.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.406+0000 7f3d08fc9640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d0415adf0 0x7f3d0415d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:00.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.406+0000 7f3d037fe640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d04070c00 0x7f3d04154e00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800e9e0 tx=0x7f3cf800eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:00.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.406+0000 7f3d00ff9640 1 -- 172.21.15.27:0/414842365 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf8009670 con 0x7f3d04070c00 2023-12-19T18:15:00.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.406+0000 7f3d00ff9640 1 -- 172.21.15.27:0/414842365 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf800c9d0 con 0x7f3d04070c00 2023-12-19T18:15:00.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.406+0000 7f3d00ff9640 1 -- 172.21.15.27:0/414842365 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf800cd80 con 0x7f3d04070c00 2023-12-19T18:15:00.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.406+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/414842365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d041f3060 con 0x7f3d04070c00 2023-12-19T18:15:00.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.407+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/414842365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d041f2350 con 0x7f3d04070c00 2023-12-19T18:15:00.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.408+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/414842365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d04068e90 con 0x7f3d04070c00 2023-12-19T18:15:00.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.409+0000 7f3d00ff9640 1 -- 172.21.15.27:0/414842365 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3cf8013470 con 0x7f3d04070c00 2023-12-19T18:15:00.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.410+0000 7f3d00ff9640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3cf006a430 0x7f3cf006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:00.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.410+0000 7f3d00ff9640 1 -- 172.21.15.27:0/414842365 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3cf8088ae0 con 0x7f3d04070c00 2023-12-19T18:15:00.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.411+0000 7f3d03fff640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3cf006a430 0x7f3cf006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:00.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.412+0000 7f3d03fff640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3cf006a430 0x7f3cf006c8f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009e50 tx=0x7f3cf4030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:00.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.415+0000 7f3d00ff9640 1 -- 172.21.15.27:0/414842365 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3cf8054160 con 0x7f3d04070c00 2023-12-19T18:15:00.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.681+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/414842365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f3d04071ca0 con 0x7f3d04070c00 2023-12-19T18:15:00.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.682+0000 7f3d00ff9640 1 -- 172.21.15.27:0/414842365 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1004 (secure 0 0 0) 0x7f3cf8017400 con 0x7f3d04070c00 2023-12-19T18:15:00.682 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:00.682 INFO:teuthology.orchestra.run.smithi027.stdout:{"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":24,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":20,"num_osds":8,"num_up_osds":8,"osd_up_since":1703009683,"num_in_osds":8,"osd_in_since":1703009680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":1751364,"bytes_avail":1598248636,"bytes_total":1600000000,"write_bytes_sec":2324,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":8,"id":1,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":1,"rank":0,"name":"d","status":"up:active","gid":4267}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":1,"modified":"2023-12-19T18:14:36.526866+0000","services":{}},"progress_events":{}} 2023-12-19T18:15:00.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.684+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3cf006a430 msgr2=0x7f3cf006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:00.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.684+0000 7f3d0aa2c640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3cf006a430 0x7f3cf006c8f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009e50 tx=0x7f3cf4030000 comp rx=0 tx=0).stop 2023-12-19T18:15:00.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.684+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/414842365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d04070c00 msgr2=0x7f3d04154e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:00.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.684+0000 7f3d0aa2c640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d04070c00 0x7f3d04154e00 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3cf800e9e0 tx=0x7f3cf800eea0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.684+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/414842365 shutdown_connections 2023-12-19T18:15:00.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.684+0000 7f3d0aa2c640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3cf006a430 0x7f3cf006c8f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.684+0000 7f3d0aa2c640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d0415adf0 0x7f3d0415d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.684+0000 7f3d0aa2c640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d04070c00 0x7f3d04154e00 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.684+0000 7f3d0aa2c640 1 --2- 172.21.15.27:0/414842365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d0406f500 0x7f3d040c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.684+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/414842365 >> 172.21.15.27:0/414842365 conn(0x7f3d040ceb50 msgr2=0x7f3d040cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:00.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.684+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/414842365 shutdown_connections 2023-12-19T18:15:00.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.684+0000 7f3d0aa2c640 1 -- 172.21.15.27:0/414842365 wait complete. 2023-12-19T18:15:00.706 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-19T18:15:00.706 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:15:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.939+0000 7f442742d640 1 Processor -- start 2023-12-19T18:15:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.940+0000 7f442742d640 1 -- start start 2023-12-19T18:15:00.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.941+0000 7f442742d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f44201553a0 0x7f44200c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:00.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.941+0000 7f442742d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f442015ee30 0x7f44200c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:00.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.941+0000 7f442742d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44200b0360 con 0x7f44200afeb0 2023-12-19T18:15:00.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.941+0000 7f442742d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44200cbcf0 con 0x7f44201553a0 2023-12-19T18:15:00.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.941+0000 7f442742d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44200cb350 con 0x7f442015ee30 2023-12-19T18:15:00.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.942+0000 7f4426c2c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f442015ee30 0x7f44200c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:00.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.942+0000 7f442642b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f44201553a0 0x7f44200c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:00.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.942+0000 7f4426c2c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f442015ee30 0x7f44200c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:39664/0 (socket says 172.21.15.27:39664) 2023-12-19T18:15:00.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.942+0000 7f4426c2c640 1 -- 172.21.15.27:0/618523842 learned_addr learned my addr 172.21.15.27:0/618523842 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:00.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.943+0000 7f442642b640 1 -- 172.21.15.27:0/618523842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f442015ee30 msgr2=0x7f44200c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:00.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.943+0000 7f442642b640 1 --2- 172.21.15.27:0/618523842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f442015ee30 0x7f44200c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.943+0000 7f442642b640 1 -- 172.21.15.27:0/618523842 >> v1:172.21.15.192:6789/0 conn(0x7f44200afeb0 legacy=0x7f442015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:00.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.943+0000 7f442642b640 1 -- 172.21.15.27:0/618523842 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44200cb630 con 0x7f44201553a0 2023-12-19T18:15:00.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.944+0000 7f442642b640 1 --2- 172.21.15.27:0/618523842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44201553a0 0x7f44200c50a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f441c0097a0 tx=0x7f441c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ac8c994d5319b716 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:00.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.945+0000 7f4425429640 1 -- 172.21.15.27:0/618523842 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c031780 con 0x7f44201553a0 2023-12-19T18:15:00.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.945+0000 7f4425429640 1 -- 172.21.15.27:0/618523842 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f441c0318e0 con 0x7f44201553a0 2023-12-19T18:15:00.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.945+0000 7f4425429640 1 -- 172.21.15.27:0/618523842 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f441c031c70 con 0x7f44201553a0 2023-12-19T18:15:00.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.945+0000 7f442742d640 1 -- 172.21.15.27:0/618523842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44201553a0 msgr2=0x7f44200c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.945+0000 7f442742d640 1 --2- 172.21.15.27:0/618523842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44201553a0 0x7f44200c50a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f441c0097a0 tx=0x7f441c02b820 comp rx=0 tx=0).stop 2023-12-19T18:15:00.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.946+0000 7f442742d640 1 -- 172.21.15.27:0/618523842 shutdown_connections 2023-12-19T18:15:00.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.946+0000 7f442742d640 1 --2- 172.21.15.27:0/618523842 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f442015ee30 0x7f44200c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.946+0000 7f442742d640 1 --2- 172.21.15.27:0/618523842 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44201553a0 0x7f44200c50a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.946+0000 7f442742d640 1 -- 172.21.15.27:0/618523842 >> 172.21.15.27:0/618523842 conn(0x7f4420069c70 msgr2=0x7f442006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:00.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.946+0000 7f442742d640 1 -- 172.21.15.27:0/618523842 shutdown_connections 2023-12-19T18:15:00.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.946+0000 7f442742d640 1 -- 172.21.15.27:0/618523842 wait complete. 2023-12-19T18:15:00.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.948+0000 7f442742d640 1 Processor -- start 2023-12-19T18:15:00.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.948+0000 7f442742d640 1 -- start start 2023-12-19T18:15:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.949+0000 7f442742d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44200afeb0 0x7f44201f65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.949+0000 7f442742d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44201553a0 0x7f442015a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.949+0000 7f442742d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f442015ee30 0x7f44200b4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.949+0000 7f442742d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200ae8e0 con 0x7f44201553a0 2023-12-19T18:15:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.949+0000 7f442742d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44200c6600 con 0x7f44200afeb0 2023-12-19T18:15:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.949+0000 7f442742d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44200c68e0 con 0x7f442015ee30 2023-12-19T18:15:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.949+0000 7f4425c2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44201553a0 0x7f442015a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.949+0000 7f4425c2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44201553a0 0x7f442015a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54648/0 (socket says 172.21.15.27:54648) 2023-12-19T18:15:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.949+0000 7f4425c2a640 1 -- 172.21.15.27:0/3691683633 learned_addr learned my addr 172.21.15.27:0/3691683633 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.949+0000 7f442642b640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44200afeb0 0x7f44201f65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.949+0000 7f4426c2c640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f442015ee30 0x7f44200b4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.950+0000 7f4425c2a640 1 -- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f442015ee30 msgr2=0x7f44200b4110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.950+0000 7f4425c2a640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f442015ee30 0x7f44200b4110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.950+0000 7f4425c2a640 1 -- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44200afeb0 msgr2=0x7f44201f65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.950+0000 7f4425c2a640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44200afeb0 0x7f44201f65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.950+0000 7f4425c2a640 1 -- 172.21.15.27:0/3691683633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f441c009400 con 0x7f44201553a0 2023-12-19T18:15:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.950+0000 7f442642b640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44200afeb0 0x7f44201f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.950+0000 7f4425c2a640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44201553a0 0x7f442015a020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f441400c450 tx=0x7f441400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.950+0000 7f4412ffd640 1 -- 172.21.15.27:0/3691683633 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44140052d0 con 0x7f44201553a0 2023-12-19T18:15:00.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.950+0000 7f4412ffd640 1 -- 172.21.15.27:0/3691683633 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4414005430 con 0x7f44201553a0 2023-12-19T18:15:00.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.950+0000 7f4412ffd640 1 -- 172.21.15.27:0/3691683633 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4414012870 con 0x7f44201553a0 2023-12-19T18:15:00.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.950+0000 7f442742d640 1 -- 172.21.15.27:0/3691683633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44201f5ad0 con 0x7f44201553a0 2023-12-19T18:15:00.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.951+0000 7f442742d640 1 -- 172.21.15.27:0/3691683633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44201f6090 con 0x7f44201553a0 2023-12-19T18:15:00.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.952+0000 7f442742d640 1 -- 172.21.15.27:0/3691683633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f442004e840 con 0x7f44201553a0 2023-12-19T18:15:00.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.953+0000 7f4412ffd640 1 -- 172.21.15.27:0/3691683633 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f44140129d0 con 0x7f44201553a0 2023-12-19T18:15:00.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.954+0000 7f4412ffd640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f440406a430 0x7f440406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:00.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.954+0000 7f4412ffd640 1 -- 172.21.15.27:0/3691683633 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4414088460 con 0x7f44201553a0 2023-12-19T18:15:00.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.954+0000 7f442642b640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f440406a430 0x7f440406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:00.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.956+0000 7f442642b640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f440406a430 0x7f440406c8f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f441c036000 tx=0x7f441c03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:00.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:00.959+0000 7f4412ffd640 1 -- 172.21.15.27:0/3691683633 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f44140539b0 con 0x7f44201553a0 2023-12-19T18:15:01.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.150+0000 7f442742d640 1 -- 172.21.15.27:0/3691683633 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f44200c5730 con 0x7f440406a430 2023-12-19T18:15:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.171+0000 7f4412ffd640 1 -- 172.21.15.27:0/3691683633 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260098 (secure 0 0 0) 0x7f441c031ad0 con 0x7f440406a430 2023-12-19T18:15:01.174 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:01.176 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:15:01.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.176+0000 7f442742d640 1 -- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f440406a430 msgr2=0x7f440406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:01.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.176+0000 7f442742d640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f440406a430 0x7f440406c8f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f441c036000 tx=0x7f441c03d040 comp rx=0 tx=0).stop 2023-12-19T18:15:01.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.176+0000 7f442742d640 1 -- 172.21.15.27:0/3691683633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44201553a0 msgr2=0x7f442015a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:01.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.176+0000 7f442742d640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44201553a0 0x7f442015a020 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f441400c450 tx=0x7f441400c910 comp rx=0 tx=0).stop 2023-12-19T18:15:01.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.176+0000 7f442742d640 1 -- 172.21.15.27:0/3691683633 shutdown_connections 2023-12-19T18:15:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.176+0000 7f442742d640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f440406a430 0x7f440406c8f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.176+0000 7f442742d640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f442015ee30 0x7f44200b4110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.176+0000 7f442742d640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44201553a0 0x7f442015a020 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.176+0000 7f442742d640 1 --2- 172.21.15.27:0/3691683633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44200afeb0 0x7f44201f65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.176+0000 7f442742d640 1 -- 172.21.15.27:0/3691683633 >> 172.21.15.27:0/3691683633 conn(0x7f4420069c70 msgr2=0x7f44200c43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.176+0000 7f442742d640 1 -- 172.21.15.27:0/3691683633 shutdown_connections 2023-12-19T18:15:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:01.176+0000 7f442742d640 1 -- 172.21.15.27:0/3691683633 wait complete. 2023-12-19T18:15:01.196 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-12-19T18:14:59.703239+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248636,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.167376"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056823+0000","last_change":"2023-12-19T18:14:54.056711+0000","last_active":"2023-12-19T18:14:54.056823+0000","last_peered":"2023-12-19T18:14:54.056823+0000","last_clean":"2023-12-19T18:14:54.056823+0000","last_became_active":"2023-12-19T18:14:54.056335+0000","last_became_peered":"2023-12-19T18:14:54.056335+0000","last_unstale":"2023-12-19T18:14:54.056823+0000","last_undegraded":"2023-12-19T18:14:54.056823+0000","last_fullsized":"2023-12-19T18:14:54.056823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069974+0000","last_change":"2023-12-19T18:14:54.069882+0000","last_active":"2023-12-19T18:14:54.069974+0000","last_peered":"2023-12-19T18:14:54.069974+0000","last_clean":"2023-12-19T18:14:54.069974+0000","last_became_active":"2023-12-19T18:14:54.069626+0000","last_became_peered":"2023-12-19T18:14:54.069626+0000","last_unstale":"2023-12-19T18:14:54.069974+0000","last_undegraded":"2023-12-19T18:14:54.069974+0000","last_fullsized":"2023-12-19T18:14:54.069974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053578+0000","last_change":"2023-12-19T18:14:54.053111+0000","last_active":"2023-12-19T18:14:54.053578+0000","last_peered":"2023-12-19T18:14:54.053578+0000","last_clean":"2023-12-19T18:14:54.053578+0000","last_became_active":"2023-12-19T18:14:54.052858+0000","last_became_peered":"2023-12-19T18:14:54.052858+0000","last_unstale":"2023-12-19T18:14:54.053578+0000","last_undegraded":"2023-12-19T18:14:54.053578+0000","last_fullsized":"2023-12-19T18:14:54.053578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050310+0000","last_change":"2023-12-19T18:14:54.049862+0000","last_active":"2023-12-19T18:14:54.050310+0000","last_peered":"2023-12-19T18:14:54.050310+0000","last_clean":"2023-12-19T18:14:54.050310+0000","last_became_active":"2023-12-19T18:14:54.049396+0000","last_became_peered":"2023-12-19T18:14:54.049396+0000","last_unstale":"2023-12-19T18:14:54.050310+0000","last_undegraded":"2023-12-19T18:14:54.050310+0000","last_fullsized":"2023-12-19T18:14:54.050310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053036+0000","last_change":"2023-12-19T18:14:54.052387+0000","last_active":"2023-12-19T18:14:54.053036+0000","last_peered":"2023-12-19T18:14:54.053036+0000","last_clean":"2023-12-19T18:14:54.053036+0000","last_became_active":"2023-12-19T18:14:54.052008+0000","last_became_peered":"2023-12-19T18:14:54.052008+0000","last_unstale":"2023-12-19T18:14:54.053036+0000","last_undegraded":"2023-12-19T18:14:54.053036+0000","last_fullsized":"2023-12-19T18:14:54.053036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061254+0000","last_change":"2023-12-19T18:14:54.061175+0000","last_active":"2023-12-19T18:14:54.061254+0000","last_peered":"2023-12-19T18:14:54.061254+0000","last_clean":"2023-12-19T18:14:54.061254+0000","last_became_active":"2023-12-19T18:14:54.060192+0000","last_became_peered":"2023-12-19T18:14:54.060192+0000","last_unstale":"2023-12-19T18:14:54.061254+0000","last_undegraded":"2023-12-19T18:14:54.061254+0000","last_fullsized":"2023-12-19T18:14:54.061254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.054262+0000","last_change":"2023-12-19T18:14:54.053283+0000","last_active":"2023-12-19T18:14:54.054262+0000","last_peered":"2023-12-19T18:14:54.054262+0000","last_clean":"2023-12-19T18:14:54.054262+0000","last_became_active":"2023-12-19T18:14:54.052854+0000","last_became_peered":"2023-12-19T18:14:54.052854+0000","last_unstale":"2023-12-19T18:14:54.054262+0000","last_undegraded":"2023-12-19T18:14:54.054262+0000","last_fullsized":"2023-12-19T18:14:54.054262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056579+0000","last_change":"2023-12-19T18:14:54.056483+0000","last_active":"2023-12-19T18:14:54.056579+0000","last_peered":"2023-12-19T18:14:54.056579+0000","last_clean":"2023-12-19T18:14:54.056579+0000","last_became_active":"2023-12-19T18:14:54.056196+0000","last_became_peered":"2023-12-19T18:14:54.056196+0000","last_unstale":"2023-12-19T18:14:54.056579+0000","last_undegraded":"2023-12-19T18:14:54.056579+0000","last_fullsized":"2023-12-19T18:14:54.056579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075945+0000","last_change":"2023-12-19T18:14:54.074709+0000","last_active":"2023-12-19T18:14:54.075945+0000","last_peered":"2023-12-19T18:14:54.075945+0000","last_clean":"2023-12-19T18:14:54.075945+0000","last_became_active":"2023-12-19T18:14:54.074482+0000","last_became_peered":"2023-12-19T18:14:54.074482+0000","last_unstale":"2023-12-19T18:14:54.075945+0000","last_undegraded":"2023-12-19T18:14:54.075945+0000","last_fullsized":"2023-12-19T18:14:54.075945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051054+0000","last_change":"2023-12-19T18:14:54.050203+0000","last_active":"2023-12-19T18:14:54.051054+0000","last_peered":"2023-12-19T18:14:54.051054+0000","last_clean":"2023-12-19T18:14:54.051054+0000","last_became_active":"2023-12-19T18:14:54.049582+0000","last_became_peered":"2023-12-19T18:14:54.049582+0000","last_unstale":"2023-12-19T18:14:54.051054+0000","last_undegraded":"2023-12-19T18:14:54.051054+0000","last_fullsized":"2023-12-19T18:14:54.051054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056806+0000","last_change":"2023-12-19T18:14:54.056564+0000","last_active":"2023-12-19T18:14:54.056806+0000","last_peered":"2023-12-19T18:14:54.056806+0000","last_clean":"2023-12-19T18:14:54.056806+0000","last_became_active":"2023-12-19T18:14:54.056052+0000","last_became_peered":"2023-12-19T18:14:54.056052+0000","last_unstale":"2023-12-19T18:14:54.056806+0000","last_undegraded":"2023-12-19T18:14:54.056806+0000","last_fullsized":"2023-12-19T18:14:54.056806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066528+0000","last_change":"2023-12-19T18:14:54.066257+0000","last_active":"2023-12-19T18:14:54.066528+0000","last_peered":"2023-12-19T18:14:54.066528+0000","last_clean":"2023-12-19T18:14:54.066528+0000","last_became_active":"2023-12-19T18:14:54.065876+0000","last_became_peered":"2023-12-19T18:14:54.065876+0000","last_unstale":"2023-12-19T18:14:54.066528+0000","last_undegraded":"2023-12-19T18:14:54.066528+0000","last_fullsized":"2023-12-19T18:14:54.066528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070727+0000","last_change":"2023-12-19T18:14:54.070658+0000","last_active":"2023-12-19T18:14:54.070727+0000","last_peered":"2023-12-19T18:14:54.070727+0000","last_clean":"2023-12-19T18:14:54.070727+0000","last_became_active":"2023-12-19T18:14:54.065952+0000","last_became_peered":"2023-12-19T18:14:54.065952+0000","last_unstale":"2023-12-19T18:14:54.070727+0000","last_undegraded":"2023-12-19T18:14:54.070727+0000","last_fullsized":"2023-12-19T18:14:54.070727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:35:28.203585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072334+0000","last_change":"2023-12-19T18:14:54.072211+0000","last_active":"2023-12-19T18:14:54.072334+0000","last_peered":"2023-12-19T18:14:54.072334+0000","last_clean":"2023-12-19T18:14:54.072334+0000","last_became_active":"2023-12-19T18:14:54.071912+0000","last_became_peered":"2023-12-19T18:14:54.071912+0000","last_unstale":"2023-12-19T18:14:54.072334+0000","last_undegraded":"2023-12-19T18:14:54.072334+0000","last_fullsized":"2023-12-19T18:14:54.072334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056480+0000","last_change":"2023-12-19T18:14:54.055761+0000","last_active":"2023-12-19T18:14:54.056480+0000","last_peered":"2023-12-19T18:14:54.056480+0000","last_clean":"2023-12-19T18:14:54.056480+0000","last_became_active":"2023-12-19T18:14:54.055529+0000","last_became_peered":"2023-12-19T18:14:54.055529+0000","last_unstale":"2023-12-19T18:14:54.056480+0000","last_undegraded":"2023-12-19T18:14:54.056480+0000","last_fullsized":"2023-12-19T18:14:54.056480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073307+0000","last_change":"2023-12-19T18:14:54.072540+0000","last_active":"2023-12-19T18:14:54.073307+0000","last_peered":"2023-12-19T18:14:54.073307+0000","last_clean":"2023-12-19T18:14:54.073307+0000","last_became_active":"2023-12-19T18:14:54.072162+0000","last_became_peered":"2023-12-19T18:14:54.072162+0000","last_unstale":"2023-12-19T18:14:54.073307+0000","last_undegraded":"2023-12-19T18:14:54.073307+0000","last_fullsized":"2023-12-19T18:14:54.073307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075801+0000","last_change":"2023-12-19T18:14:54.075571+0000","last_active":"2023-12-19T18:14:54.075801+0000","last_peered":"2023-12-19T18:14:54.075801+0000","last_clean":"2023-12-19T18:14:54.075801+0000","last_became_active":"2023-12-19T18:14:54.074613+0000","last_became_peered":"2023-12-19T18:14:54.074613+0000","last_unstale":"2023-12-19T18:14:54.075801+0000","last_undegraded":"2023-12-19T18:14:54.075801+0000","last_fullsized":"2023-12-19T18:14:54.075801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:15.120564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.062708+0000","last_change":"2023-12-19T18:14:54.062640+0000","last_active":"2023-12-19T18:14:54.062708+0000","last_peered":"2023-12-19T18:14:54.062708+0000","last_clean":"2023-12-19T18:14:54.062708+0000","last_became_active":"2023-12-19T18:14:54.061994+0000","last_became_peered":"2023-12-19T18:14:54.061994+0000","last_unstale":"2023-12-19T18:14:54.062708+0000","last_undegraded":"2023-12-19T18:14:54.062708+0000","last_fullsized":"2023-12-19T18:14:54.062708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073330+0000","last_change":"2023-12-19T18:14:54.073175+0000","last_active":"2023-12-19T18:14:54.073330+0000","last_peered":"2023-12-19T18:14:54.073330+0000","last_clean":"2023-12-19T18:14:54.073330+0000","last_became_active":"2023-12-19T18:14:54.072223+0000","last_became_peered":"2023-12-19T18:14:54.072223+0000","last_unstale":"2023-12-19T18:14:54.073330+0000","last_undegraded":"2023-12-19T18:14:54.073330+0000","last_fullsized":"2023-12-19T18:14:54.073330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064684+0000","last_change":"2023-12-19T18:14:54.064206+0000","last_active":"2023-12-19T18:14:54.064684+0000","last_peered":"2023-12-19T18:14:54.064684+0000","last_clean":"2023-12-19T18:14:54.064684+0000","last_became_active":"2023-12-19T18:14:54.063818+0000","last_became_peered":"2023-12-19T18:14:54.063818+0000","last_unstale":"2023-12-19T18:14:54.064684+0000","last_undegraded":"2023-12-19T18:14:54.064684+0000","last_fullsized":"2023-12-19T18:14:54.064684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063666+0000","last_change":"2023-12-19T18:14:54.063234+0000","last_active":"2023-12-19T18:14:54.063666+0000","last_peered":"2023-12-19T18:14:54.063666+0000","last_clean":"2023-12-19T18:14:54.063666+0000","last_became_active":"2023-12-19T18:14:54.062885+0000","last_became_peered":"2023-12-19T18:14:54.062885+0000","last_unstale":"2023-12-19T18:14:54.063666+0000","last_undegraded":"2023-12-19T18:14:54.063666+0000","last_fullsized":"2023-12-19T18:14:54.063666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063662+0000","last_change":"2023-12-19T18:14:54.063598+0000","last_active":"2023-12-19T18:14:54.063662+0000","last_peered":"2023-12-19T18:14:54.063662+0000","last_clean":"2023-12-19T18:14:54.063662+0000","last_became_active":"2023-12-19T18:14:54.063314+0000","last_became_peered":"2023-12-19T18:14:54.063314+0000","last_unstale":"2023-12-19T18:14:54.063662+0000","last_undegraded":"2023-12-19T18:14:54.063662+0000","last_fullsized":"2023-12-19T18:14:54.063662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060228+0000","last_change":"2023-12-19T18:14:54.059917+0000","last_active":"2023-12-19T18:14:54.060228+0000","last_peered":"2023-12-19T18:14:54.060228+0000","last_clean":"2023-12-19T18:14:54.060228+0000","last_became_active":"2023-12-19T18:14:54.059674+0000","last_became_peered":"2023-12-19T18:14:54.059674+0000","last_unstale":"2023-12-19T18:14:54.060228+0000","last_undegraded":"2023-12-19T18:14:54.060228+0000","last_fullsized":"2023-12-19T18:14:54.060228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074581+0000","last_change":"2023-12-19T18:14:54.073659+0000","last_active":"2023-12-19T18:14:54.074581+0000","last_peered":"2023-12-19T18:14:54.074581+0000","last_clean":"2023-12-19T18:14:54.074581+0000","last_became_active":"2023-12-19T18:14:54.072947+0000","last_became_peered":"2023-12-19T18:14:54.072947+0000","last_unstale":"2023-12-19T18:14:54.074581+0000","last_undegraded":"2023-12-19T18:14:54.074581+0000","last_fullsized":"2023-12-19T18:14:54.074581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067036+0000","last_change":"2023-12-19T18:14:54.066189+0000","last_active":"2023-12-19T18:14:54.067036+0000","last_peered":"2023-12-19T18:14:54.067036+0000","last_clean":"2023-12-19T18:14:54.067036+0000","last_became_active":"2023-12-19T18:14:54.065921+0000","last_became_peered":"2023-12-19T18:14:54.065921+0000","last_unstale":"2023-12-19T18:14:54.067036+0000","last_undegraded":"2023-12-19T18:14:54.067036+0000","last_fullsized":"2023-12-19T18:14:54.067036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061440+0000","last_change":"2023-12-19T18:14:54.061372+0000","last_active":"2023-12-19T18:14:54.061440+0000","last_peered":"2023-12-19T18:14:54.061440+0000","last_clean":"2023-12-19T18:14:54.061440+0000","last_became_active":"2023-12-19T18:14:54.060949+0000","last_became_peered":"2023-12-19T18:14:54.060949+0000","last_unstale":"2023-12-19T18:14:54.061440+0000","last_undegraded":"2023-12-19T18:14:54.061440+0000","last_fullsized":"2023-12-19T18:14:54.061440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073948+0000","last_change":"2023-12-19T18:14:54.073038+0000","last_active":"2023-12-19T18:14:54.073948+0000","last_peered":"2023-12-19T18:14:54.073948+0000","last_clean":"2023-12-19T18:14:54.073948+0000","last_became_active":"2023-12-19T18:14:54.071844+0000","last_became_peered":"2023-12-19T18:14:54.071844+0000","last_unstale":"2023-12-19T18:14:54.073948+0000","last_undegraded":"2023-12-19T18:14:54.073948+0000","last_fullsized":"2023-12-19T18:14:54.073948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070883+0000","last_change":"2023-12-19T18:14:54.070817+0000","last_active":"2023-12-19T18:14:54.070883+0000","last_peered":"2023-12-19T18:14:54.070883+0000","last_clean":"2023-12-19T18:14:54.070883+0000","last_became_active":"2023-12-19T18:14:54.068216+0000","last_became_peered":"2023-12-19T18:14:54.068216+0000","last_unstale":"2023-12-19T18:14:54.070883+0000","last_undegraded":"2023-12-19T18:14:54.070883+0000","last_fullsized":"2023-12-19T18:14:54.070883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063273+0000","last_change":"2023-12-19T18:14:54.060522+0000","last_active":"2023-12-19T18:14:54.063273+0000","last_peered":"2023-12-19T18:14:54.063273+0000","last_clean":"2023-12-19T18:14:54.063273+0000","last_became_active":"2023-12-19T18:14:54.059641+0000","last_became_peered":"2023-12-19T18:14:54.059641+0000","last_unstale":"2023-12-19T18:14:54.063273+0000","last_undegraded":"2023-12-19T18:14:54.063273+0000","last_fullsized":"2023-12-19T18:14:54.063273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065928+0000","last_change":"2023-12-19T18:14:54.065766+0000","last_active":"2023-12-19T18:14:54.065928+0000","last_peered":"2023-12-19T18:14:54.065928+0000","last_clean":"2023-12-19T18:14:54.065928+0000","last_became_active":"2023-12-19T18:14:54.058571+0000","last_became_peered":"2023-12-19T18:14:54.058571+0000","last_unstale":"2023-12-19T18:14:54.065928+0000","last_undegraded":"2023-12-19T18:14:54.065928+0000","last_fullsized":"2023-12-19T18:14:54.065928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:47.213095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070429+0000","last_change":"2023-12-19T18:14:54.067798+0000","last_active":"2023-12-19T18:14:54.070429+0000","last_peered":"2023-12-19T18:14:54.070429+0000","last_clean":"2023-12-19T18:14:54.070429+0000","last_became_active":"2023-12-19T18:14:54.067576+0000","last_became_peered":"2023-12-19T18:14:54.067576+0000","last_unstale":"2023-12-19T18:14:54.070429+0000","last_undegraded":"2023-12-19T18:14:54.070429+0000","last_fullsized":"2023-12-19T18:14:54.070429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069889+0000","last_change":"2023-12-19T18:14:54.069741+0000","last_active":"2023-12-19T18:14:54.069889+0000","last_peered":"2023-12-19T18:14:54.069889+0000","last_clean":"2023-12-19T18:14:54.069889+0000","last_became_active":"2023-12-19T18:14:54.069513+0000","last_became_peered":"2023-12-19T18:14:54.069513+0000","last_unstale":"2023-12-19T18:14:54.069889+0000","last_undegraded":"2023-12-19T18:14:54.069889+0000","last_fullsized":"2023-12-19T18:14:54.069889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070495+0000","last_change":"2023-12-19T18:14:54.070403+0000","last_active":"2023-12-19T18:14:54.070495+0000","last_peered":"2023-12-19T18:14:54.070495+0000","last_clean":"2023-12-19T18:14:54.070495+0000","last_became_active":"2023-12-19T18:14:54.069801+0000","last_became_peered":"2023-12-19T18:14:54.069801+0000","last_unstale":"2023-12-19T18:14:54.070495+0000","last_undegraded":"2023-12-19T18:14:54.070495+0000","last_fullsized":"2023-12-19T18:14:54.070495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068551+0000","last_change":"2023-12-19T18:14:53.563064+0000","last_active":"2023-12-19T18:14:54.068551+0000","last_peered":"2023-12-19T18:14:54.068551+0000","last_clean":"2023-12-19T18:14:54.068551+0000","last_became_active":"2023-12-19T18:14:53.561534+0000","last_became_peered":"2023-12-19T18:14:53.561534+0000","last_unstale":"2023-12-19T18:14:54.068551+0000","last_undegraded":"2023-12-19T18:14:54.068551+0000","last_fullsized":"2023-12-19T18:14:54.068551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051639+0000","last_change":"2023-12-19T18:14:54.051570+0000","last_active":"2023-12-19T18:14:54.051639+0000","last_peered":"2023-12-19T18:14:54.051639+0000","last_clean":"2023-12-19T18:14:54.051639+0000","last_became_active":"2023-12-19T18:14:54.051173+0000","last_became_peered":"2023-12-19T18:14:54.051173+0000","last_unstale":"2023-12-19T18:14:54.051639+0000","last_undegraded":"2023-12-19T18:14:54.051639+0000","last_fullsized":"2023-12-19T18:14:54.051639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073383+0000","last_change":"2023-12-19T18:14:53.560763+0000","last_active":"2023-12-19T18:14:54.073383+0000","last_peered":"2023-12-19T18:14:54.073383+0000","last_clean":"2023-12-19T18:14:54.073383+0000","last_became_active":"2023-12-19T18:14:53.558648+0000","last_became_peered":"2023-12-19T18:14:53.558648+0000","last_unstale":"2023-12-19T18:14:54.073383+0000","last_undegraded":"2023-12-19T18:14:54.073383+0000","last_fullsized":"2023-12-19T18:14:54.073383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:51.179899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066389+0000","last_change":"2023-12-19T18:14:53.560483+0000","last_active":"2023-12-19T18:14:54.066389+0000","last_peered":"2023-12-19T18:14:54.066389+0000","last_clean":"2023-12-19T18:14:54.066389+0000","last_became_active":"2023-12-19T18:14:53.560244+0000","last_became_peered":"2023-12-19T18:14:53.560244+0000","last_unstale":"2023-12-19T18:14:54.066389+0000","last_undegraded":"2023-12-19T18:14:54.066389+0000","last_fullsized":"2023-12-19T18:14:54.066389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072273+0000","last_change":"2023-12-19T18:14:54.071383+0000","last_active":"2023-12-19T18:14:54.072273+0000","last_peered":"2023-12-19T18:14:54.072273+0000","last_clean":"2023-12-19T18:14:54.072273+0000","last_became_active":"2023-12-19T18:14:54.070954+0000","last_became_peered":"2023-12-19T18:14:54.070954+0000","last_unstale":"2023-12-19T18:14:54.072273+0000","last_undegraded":"2023-12-19T18:14:54.072273+0000","last_fullsized":"2023-12-19T18:14:54.072273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053551+0000","last_change":"2023-12-19T18:14:53.558346+0000","last_active":"2023-12-19T18:14:54.053551+0000","last_peered":"2023-12-19T18:14:54.053551+0000","last_clean":"2023-12-19T18:14:54.053551+0000","last_became_active":"2023-12-19T18:14:53.558124+0000","last_became_peered":"2023-12-19T18:14:53.558124+0000","last_unstale":"2023-12-19T18:14:54.053551+0000","last_undegraded":"2023-12-19T18:14:54.053551+0000","last_fullsized":"2023-12-19T18:14:54.053551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066236+0000","last_change":"2023-12-19T18:14:54.064228+0000","last_active":"2023-12-19T18:14:54.066236+0000","last_peered":"2023-12-19T18:14:54.066236+0000","last_clean":"2023-12-19T18:14:54.066236+0000","last_became_active":"2023-12-19T18:14:54.063991+0000","last_became_peered":"2023-12-19T18:14:54.063991+0000","last_unstale":"2023-12-19T18:14:54.066236+0000","last_undegraded":"2023-12-19T18:14:54.066236+0000","last_fullsized":"2023-12-19T18:14:54.066236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064163+0000","last_change":"2023-12-19T18:14:53.554048+0000","last_active":"2023-12-19T18:14:54.064163+0000","last_peered":"2023-12-19T18:14:54.064163+0000","last_clean":"2023-12-19T18:14:54.064163+0000","last_became_active":"2023-12-19T18:14:53.553811+0000","last_became_peered":"2023-12-19T18:14:53.553811+0000","last_unstale":"2023-12-19T18:14:54.064163+0000","last_undegraded":"2023-12-19T18:14:54.064163+0000","last_fullsized":"2023-12-19T18:14:54.064163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073051+0000","last_change":"2023-12-19T18:14:54.072974+0000","last_active":"2023-12-19T18:14:54.073051+0000","last_peered":"2023-12-19T18:14:54.073051+0000","last_clean":"2023-12-19T18:14:54.073051+0000","last_became_active":"2023-12-19T18:14:54.072176+0000","last_became_peered":"2023-12-19T18:14:54.072176+0000","last_unstale":"2023-12-19T18:14:54.073051+0000","last_undegraded":"2023-12-19T18:14:54.073051+0000","last_fullsized":"2023-12-19T18:14:54.073051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.043565+0000","last_change":"2023-12-19T18:14:53.557947+0000","last_active":"2023-12-19T18:14:54.043565+0000","last_peered":"2023-12-19T18:14:54.043565+0000","last_clean":"2023-12-19T18:14:54.043565+0000","last_became_active":"2023-12-19T18:14:53.557713+0000","last_became_peered":"2023-12-19T18:14:53.557713+0000","last_unstale":"2023-12-19T18:14:54.043565+0000","last_undegraded":"2023-12-19T18:14:54.043565+0000","last_fullsized":"2023-12-19T18:14:54.043565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056501+0000","last_change":"2023-12-19T18:14:54.056275+0000","last_active":"2023-12-19T18:14:54.056501+0000","last_peered":"2023-12-19T18:14:54.056501+0000","last_clean":"2023-12-19T18:14:54.056501+0000","last_became_active":"2023-12-19T18:14:54.056002+0000","last_became_peered":"2023-12-19T18:14:54.056002+0000","last_unstale":"2023-12-19T18:14:54.056501+0000","last_undegraded":"2023-12-19T18:14:54.056501+0000","last_fullsized":"2023-12-19T18:14:54.056501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:30.847801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073216+0000","last_change":"2023-12-19T18:14:53.568016+0000","last_active":"2023-12-19T18:14:54.073216+0000","last_peered":"2023-12-19T18:14:54.073216+0000","last_clean":"2023-12-19T18:14:54.073216+0000","last_became_active":"2023-12-19T18:14:53.567634+0000","last_became_peered":"2023-12-19T18:14:53.567634+0000","last_unstale":"2023-12-19T18:14:54.073216+0000","last_undegraded":"2023-12-19T18:14:54.073216+0000","last_fullsized":"2023-12-19T18:14:54.073216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075897+0000","last_change":"2023-12-19T18:14:54.075827+0000","last_active":"2023-12-19T18:14:54.075897+0000","last_peered":"2023-12-19T18:14:54.075897+0000","last_clean":"2023-12-19T18:14:54.075897+0000","last_became_active":"2023-12-19T18:14:54.074747+0000","last_became_peered":"2023-12-19T18:14:54.074747+0000","last_unstale":"2023-12-19T18:14:54.075897+0000","last_undegraded":"2023-12-19T18:14:54.075897+0000","last_fullsized":"2023-12-19T18:14:54.075897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073455+0000","last_change":"2023-12-19T18:14:53.547427+0000","last_active":"2023-12-19T18:14:54.073455+0000","last_peered":"2023-12-19T18:14:54.073455+0000","last_clean":"2023-12-19T18:14:54.073455+0000","last_became_active":"2023-12-19T18:14:53.546733+0000","last_became_peered":"2023-12-19T18:14:53.546733+0000","last_unstale":"2023-12-19T18:14:54.073455+0000","last_undegraded":"2023-12-19T18:14:54.073455+0000","last_fullsized":"2023-12-19T18:14:54.073455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:13:12.090625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069234+0000","last_change":"2023-12-19T18:14:54.069070+0000","last_active":"2023-12-19T18:14:54.069234+0000","last_peered":"2023-12-19T18:14:54.069234+0000","last_clean":"2023-12-19T18:14:54.069234+0000","last_became_active":"2023-12-19T18:14:54.066495+0000","last_became_peered":"2023-12-19T18:14:54.066495+0000","last_unstale":"2023-12-19T18:14:54.069234+0000","last_undegraded":"2023-12-19T18:14:54.069234+0000","last_fullsized":"2023-12-19T18:14:54.069234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068132+0000","last_change":"2023-12-19T18:14:53.566349+0000","last_active":"2023-12-19T18:14:54.068132+0000","last_peered":"2023-12-19T18:14:54.068132+0000","last_clean":"2023-12-19T18:14:54.068132+0000","last_became_active":"2023-12-19T18:14:53.565909+0000","last_became_peered":"2023-12-19T18:14:53.565909+0000","last_unstale":"2023-12-19T18:14:54.068132+0000","last_undegraded":"2023-12-19T18:14:54.068132+0000","last_fullsized":"2023-12-19T18:14:54.068132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073929+0000","last_change":"2023-12-19T18:14:54.072903+0000","last_active":"2023-12-19T18:14:54.073929+0000","last_peered":"2023-12-19T18:14:54.073929+0000","last_clean":"2023-12-19T18:14:54.073929+0000","last_became_active":"2023-12-19T18:14:54.072678+0000","last_became_peered":"2023-12-19T18:14:54.072678+0000","last_unstale":"2023-12-19T18:14:54.073929+0000","last_undegraded":"2023-12-19T18:14:54.073929+0000","last_fullsized":"2023-12-19T18:14:54.073929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.052955+0000","last_change":"2023-12-19T18:14:53.561604+0000","last_active":"2023-12-19T18:14:54.052955+0000","last_peered":"2023-12-19T18:14:54.052955+0000","last_clean":"2023-12-19T18:14:54.052955+0000","last_became_active":"2023-12-19T18:14:53.561372+0000","last_became_peered":"2023-12-19T18:14:53.561372+0000","last_unstale":"2023-12-19T18:14:54.052955+0000","last_undegraded":"2023-12-19T18:14:54.052955+0000","last_fullsized":"2023-12-19T18:14:54.052955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:00.255541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.057807+0000","last_change":"2023-12-19T18:14:54.057328+0000","last_active":"2023-12-19T18:14:54.057807+0000","last_peered":"2023-12-19T18:14:54.057807+0000","last_clean":"2023-12-19T18:14:54.057807+0000","last_became_active":"2023-12-19T18:14:54.055611+0000","last_became_peered":"2023-12-19T18:14:54.055611+0000","last_unstale":"2023-12-19T18:14:54.057807+0000","last_undegraded":"2023-12-19T18:14:54.057807+0000","last_fullsized":"2023-12-19T18:14:54.057807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:12.180767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.043960+0000","last_change":"2023-12-19T18:14:53.562764+0000","last_active":"2023-12-19T18:14:54.043960+0000","last_peered":"2023-12-19T18:14:54.043960+0000","last_clean":"2023-12-19T18:14:54.043960+0000","last_became_active":"2023-12-19T18:14:53.562532+0000","last_became_peered":"2023-12-19T18:14:53.562532+0000","last_unstale":"2023-12-19T18:14:54.043960+0000","last_undegraded":"2023-12-19T18:14:54.043960+0000","last_fullsized":"2023-12-19T18:14:54.043960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069588+0000","last_change":"2023-12-19T18:14:54.068780+0000","last_active":"2023-12-19T18:14:54.069588+0000","last_peered":"2023-12-19T18:14:54.069588+0000","last_clean":"2023-12-19T18:14:54.069588+0000","last_became_active":"2023-12-19T18:14:54.068556+0000","last_became_peered":"2023-12-19T18:14:54.068556+0000","last_unstale":"2023-12-19T18:14:54.069588+0000","last_undegraded":"2023-12-19T18:14:54.069588+0000","last_fullsized":"2023-12-19T18:14:54.069588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074393+0000","last_change":"2023-12-19T18:14:53.565084+0000","last_active":"2023-12-19T18:14:54.074393+0000","last_peered":"2023-12-19T18:14:54.074393+0000","last_clean":"2023-12-19T18:14:54.074393+0000","last_became_active":"2023-12-19T18:14:53.564836+0000","last_became_peered":"2023-12-19T18:14:53.564836+0000","last_unstale":"2023-12-19T18:14:54.074393+0000","last_undegraded":"2023-12-19T18:14:54.074393+0000","last_fullsized":"2023-12-19T18:14:54.074393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.076309+0000","last_change":"2023-12-19T18:14:54.075967+0000","last_active":"2023-12-19T18:14:54.076309+0000","last_peered":"2023-12-19T18:14:54.076309+0000","last_clean":"2023-12-19T18:14:54.076309+0000","last_became_active":"2023-12-19T18:14:54.074991+0000","last_became_peered":"2023-12-19T18:14:54.074991+0000","last_unstale":"2023-12-19T18:14:54.076309+0000","last_undegraded":"2023-12-19T18:14:54.076309+0000","last_fullsized":"2023-12-19T18:14:54.076309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061847+0000","last_change":"2023-12-19T18:14:53.550365+0000","last_active":"2023-12-19T18:14:54.061847+0000","last_peered":"2023-12-19T18:14:54.061847+0000","last_clean":"2023-12-19T18:14:54.061847+0000","last_became_active":"2023-12-19T18:14:53.550059+0000","last_became_peered":"2023-12-19T18:14:53.550059+0000","last_unstale":"2023-12-19T18:14:54.061847+0000","last_undegraded":"2023-12-19T18:14:54.061847+0000","last_fullsized":"2023-12-19T18:14:54.061847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050074+0000","last_change":"2023-12-19T18:14:54.048927+0000","last_active":"2023-12-19T18:14:54.050074+0000","last_peered":"2023-12-19T18:14:54.050074+0000","last_clean":"2023-12-19T18:14:54.050074+0000","last_became_active":"2023-12-19T18:14:54.048515+0000","last_became_peered":"2023-12-19T18:14:54.048515+0000","last_unstale":"2023-12-19T18:14:54.050074+0000","last_undegraded":"2023-12-19T18:14:54.050074+0000","last_fullsized":"2023-12-19T18:14:54.050074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064254+0000","last_change":"2023-12-19T18:14:53.567656+0000","last_active":"2023-12-19T18:14:54.064254+0000","last_peered":"2023-12-19T18:14:54.064254+0000","last_clean":"2023-12-19T18:14:54.064254+0000","last_became_active":"2023-12-19T18:14:53.567278+0000","last_became_peered":"2023-12-19T18:14:53.567278+0000","last_unstale":"2023-12-19T18:14:54.064254+0000","last_undegraded":"2023-12-19T18:14:54.064254+0000","last_fullsized":"2023-12-19T18:14:54.064254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065685+0000","last_change":"2023-12-19T18:14:54.065628+0000","last_active":"2023-12-19T18:14:54.065685+0000","last_peered":"2023-12-19T18:14:54.065685+0000","last_clean":"2023-12-19T18:14:54.065685+0000","last_became_active":"2023-12-19T18:14:54.065414+0000","last_became_peered":"2023-12-19T18:14:54.065414+0000","last_unstale":"2023-12-19T18:14:54.065685+0000","last_undegraded":"2023-12-19T18:14:54.065685+0000","last_fullsized":"2023-12-19T18:14:54.065685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065227+0000","last_change":"2023-12-19T18:14:53.559407+0000","last_active":"2023-12-19T18:14:54.065227+0000","last_peered":"2023-12-19T18:14:54.065227+0000","last_clean":"2023-12-19T18:14:54.065227+0000","last_became_active":"2023-12-19T18:14:53.559062+0000","last_became_peered":"2023-12-19T18:14:53.559062+0000","last_unstale":"2023-12-19T18:14:54.065227+0000","last_undegraded":"2023-12-19T18:14:54.065227+0000","last_fullsized":"2023-12-19T18:14:54.065227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061207+0000","last_change":"2023-12-19T18:14:54.061139+0000","last_active":"2023-12-19T18:14:54.061207+0000","last_peered":"2023-12-19T18:14:54.061207+0000","last_clean":"2023-12-19T18:14:54.061207+0000","last_became_active":"2023-12-19T18:14:54.060925+0000","last_became_peered":"2023-12-19T18:14:54.060925+0000","last_unstale":"2023-12-19T18:14:54.061207+0000","last_undegraded":"2023-12-19T18:14:54.061207+0000","last_fullsized":"2023-12-19T18:14:54.061207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067789+0000","last_change":"2023-12-19T18:14:53.563513+0000","last_active":"2023-12-19T18:14:54.067789+0000","last_peered":"2023-12-19T18:14:54.067789+0000","last_clean":"2023-12-19T18:14:54.067789+0000","last_became_active":"2023-12-19T18:14:53.562860+0000","last_became_peered":"2023-12-19T18:14:53.562860+0000","last_unstale":"2023-12-19T18:14:54.067789+0000","last_undegraded":"2023-12-19T18:14:54.067789+0000","last_fullsized":"2023-12-19T18:14:54.067789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:59:49.969450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072181+0000","last_change":"2023-12-19T18:14:54.071273+0000","last_active":"2023-12-19T18:14:54.072181+0000","last_peered":"2023-12-19T18:14:54.072181+0000","last_clean":"2023-12-19T18:14:54.072181+0000","last_became_active":"2023-12-19T18:14:54.070908+0000","last_became_peered":"2023-12-19T18:14:54.070908+0000","last_unstale":"2023-12-19T18:14:54.072181+0000","last_undegraded":"2023-12-19T18:14:54.072181+0000","last_fullsized":"2023-12-19T18:14:54.072181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:27:34.736465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.055663+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:14:54.055663+0000","last_peered":"2023-12-19T18:14:54.055663+0000","last_clean":"2023-12-19T18:14:54.055663+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:14:54.055663+0000","last_undegraded":"2023-12-19T18:14:54.055663+0000","last_fullsized":"2023-12-19T18:14:54.055663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053073+0000","last_change":"2023-12-19T18:14:53.557129+0000","last_active":"2023-12-19T18:14:54.053073+0000","last_peered":"2023-12-19T18:14:54.053073+0000","last_clean":"2023-12-19T18:14:54.053073+0000","last_became_active":"2023-12-19T18:14:53.556812+0000","last_became_peered":"2023-12-19T18:14:53.556812+0000","last_unstale":"2023-12-19T18:14:54.053073+0000","last_undegraded":"2023-12-19T18:14:54.053073+0000","last_fullsized":"2023-12-19T18:14:54.053073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:53.113355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051508+0000","last_change":"2023-12-19T18:14:54.051423+0000","last_active":"2023-12-19T18:14:54.051508+0000","last_peered":"2023-12-19T18:14:54.051508+0000","last_clean":"2023-12-19T18:14:54.051508+0000","last_became_active":"2023-12-19T18:14:54.050816+0000","last_became_peered":"2023-12-19T18:14:54.050816+0000","last_unstale":"2023-12-19T18:14:54.051508+0000","last_undegraded":"2023-12-19T18:14:54.051508+0000","last_fullsized":"2023-12-19T18:14:54.051508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068099+0000","last_change":"2023-12-19T18:14:53.558583+0000","last_active":"2023-12-19T18:14:54.068099+0000","last_peered":"2023-12-19T18:14:54.068099+0000","last_clean":"2023-12-19T18:14:54.068099+0000","last_became_active":"2023-12-19T18:14:53.558000+0000","last_became_peered":"2023-12-19T18:14:53.558000+0000","last_unstale":"2023-12-19T18:14:54.068099+0000","last_undegraded":"2023-12-19T18:14:54.068099+0000","last_fullsized":"2023-12-19T18:14:54.068099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:53.818952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068077+0000","last_change":"2023-12-19T18:14:54.067973+0000","last_active":"2023-12-19T18:14:54.068077+0000","last_peered":"2023-12-19T18:14:54.068077+0000","last_clean":"2023-12-19T18:14:54.068077+0000","last_became_active":"2023-12-19T18:14:54.065516+0000","last_became_peered":"2023-12-19T18:14:54.065516+0000","last_unstale":"2023-12-19T18:14:54.068077+0000","last_undegraded":"2023-12-19T18:14:54.068077+0000","last_fullsized":"2023-12-19T18:14:54.068077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.058632+0000","last_change":"2023-12-19T18:14:53.563881+0000","last_active":"2023-12-19T18:14:54.058632+0000","last_peered":"2023-12-19T18:14:54.058632+0000","last_clean":"2023-12-19T18:14:54.058632+0000","last_became_active":"2023-12-19T18:14:53.563564+0000","last_became_peered":"2023-12-19T18:14:53.563564+0000","last_unstale":"2023-12-19T18:14:54.058632+0000","last_undegraded":"2023-12-19T18:14:54.058632+0000","last_fullsized":"2023-12-19T18:14:54.058632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060261+0000","last_change":"2023-12-19T18:14:54.060208+0000","last_active":"2023-12-19T18:14:54.060261+0000","last_peered":"2023-12-19T18:14:54.060261+0000","last_clean":"2023-12-19T18:14:54.060261+0000","last_became_active":"2023-12-19T18:14:54.059235+0000","last_became_peered":"2023-12-19T18:14:54.059235+0000","last_unstale":"2023-12-19T18:14:54.060261+0000","last_undegraded":"2023-12-19T18:14:54.060261+0000","last_fullsized":"2023-12-19T18:14:54.060261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065436+0000","last_change":"2023-12-19T18:14:53.564065+0000","last_active":"2023-12-19T18:14:54.065436+0000","last_peered":"2023-12-19T18:14:54.065436+0000","last_clean":"2023-12-19T18:14:54.065436+0000","last_became_active":"2023-12-19T18:14:53.563841+0000","last_became_peered":"2023-12-19T18:14:53.563841+0000","last_unstale":"2023-12-19T18:14:54.065436+0000","last_undegraded":"2023-12-19T18:14:54.065436+0000","last_fullsized":"2023-12-19T18:14:54.065436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:46.161691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060843+0000","last_change":"2023-12-19T18:14:54.060734+0000","last_active":"2023-12-19T18:14:54.060843+0000","last_peered":"2023-12-19T18:14:54.060843+0000","last_clean":"2023-12-19T18:14:54.060843+0000","last_became_active":"2023-12-19T18:14:54.060328+0000","last_became_peered":"2023-12-19T18:14:54.060328+0000","last_unstale":"2023-12-19T18:14:54.060843+0000","last_undegraded":"2023-12-19T18:14:54.060843+0000","last_fullsized":"2023-12-19T18:14:54.060843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050368+0000","last_change":"2023-12-19T18:14:53.565164+0000","last_active":"2023-12-19T18:14:54.050368+0000","last_peered":"2023-12-19T18:14:54.050368+0000","last_clean":"2023-12-19T18:14:54.050368+0000","last_became_active":"2023-12-19T18:14:53.564538+0000","last_became_peered":"2023-12-19T18:14:53.564538+0000","last_unstale":"2023-12-19T18:14:54.050368+0000","last_undegraded":"2023-12-19T18:14:54.050368+0000","last_fullsized":"2023-12-19T18:14:54.050368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068274+0000","last_change":"2023-12-19T18:14:54.068080+0000","last_active":"2023-12-19T18:14:54.068274+0000","last_peered":"2023-12-19T18:14:54.068274+0000","last_clean":"2023-12-19T18:14:54.068274+0000","last_became_active":"2023-12-19T18:14:54.067528+0000","last_became_peered":"2023-12-19T18:14:54.067528+0000","last_unstale":"2023-12-19T18:14:54.068274+0000","last_undegraded":"2023-12-19T18:14:54.068274+0000","last_fullsized":"2023-12-19T18:14:54.068274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067999+0000","last_change":"2023-12-19T18:14:53.562234+0000","last_active":"2023-12-19T18:14:54.067999+0000","last_peered":"2023-12-19T18:14:54.067999+0000","last_clean":"2023-12-19T18:14:54.067999+0000","last_became_active":"2023-12-19T18:14:53.561326+0000","last_became_peered":"2023-12-19T18:14:53.561326+0000","last_unstale":"2023-12-19T18:14:54.067999+0000","last_undegraded":"2023-12-19T18:14:54.067999+0000","last_fullsized":"2023-12-19T18:14:54.067999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074010+0000","last_change":"2023-12-19T18:14:54.073543+0000","last_active":"2023-12-19T18:14:54.074010+0000","last_peered":"2023-12-19T18:14:54.074010+0000","last_clean":"2023-12-19T18:14:54.074010+0000","last_became_active":"2023-12-19T18:14:54.073247+0000","last_became_peered":"2023-12-19T18:14:54.073247+0000","last_unstale":"2023-12-19T18:14:54.074010+0000","last_undegraded":"2023-12-19T18:14:54.074010+0000","last_fullsized":"2023-12-19T18:14:54.074010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.045518+0000","last_change":"2023-12-19T18:14:53.551251+0000","last_active":"2023-12-19T18:14:54.045518+0000","last_peered":"2023-12-19T18:14:54.045518+0000","last_clean":"2023-12-19T18:14:54.045518+0000","last_became_active":"2023-12-19T18:14:53.550691+0000","last_became_peered":"2023-12-19T18:14:53.550691+0000","last_unstale":"2023-12-19T18:14:54.045518+0000","last_undegraded":"2023-12-19T18:14:54.045518+0000","last_fullsized":"2023-12-19T18:14:54.045518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.071102+0000","last_change":"2023-12-19T18:14:54.070766+0000","last_active":"2023-12-19T18:14:54.071102+0000","last_peered":"2023-12-19T18:14:54.071102+0000","last_clean":"2023-12-19T18:14:54.071102+0000","last_became_active":"2023-12-19T18:14:54.069495+0000","last_became_peered":"2023-12-19T18:14:54.069495+0000","last_unstale":"2023-12-19T18:14:54.071102+0000","last_undegraded":"2023-12-19T18:14:54.071102+0000","last_fullsized":"2023-12-19T18:14:54.071102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.076793+0000","last_change":"2023-12-19T18:14:53.563561+0000","last_active":"2023-12-19T18:14:54.076793+0000","last_peered":"2023-12-19T18:14:54.076793+0000","last_clean":"2023-12-19T18:14:54.076793+0000","last_became_active":"2023-12-19T18:14:53.563126+0000","last_became_peered":"2023-12-19T18:14:53.563126+0000","last_unstale":"2023-12-19T18:14:54.076793+0000","last_undegraded":"2023-12-19T18:14:54.076793+0000","last_fullsized":"2023-12-19T18:14:54.076793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.059619+0000","last_change":"2023-12-19T18:14:54.059362+0000","last_active":"2023-12-19T18:14:54.059619+0000","last_peered":"2023-12-19T18:14:54.059619+0000","last_clean":"2023-12-19T18:14:54.059619+0000","last_became_active":"2023-12-19T18:14:54.059089+0000","last_became_peered":"2023-12-19T18:14:54.059089+0000","last_unstale":"2023-12-19T18:14:54.059619+0000","last_undegraded":"2023-12-19T18:14:54.059619+0000","last_fullsized":"2023-12-19T18:14:54.059619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.045320+0000","last_change":"2023-12-19T18:14:53.551856+0000","last_active":"2023-12-19T18:14:54.045320+0000","last_peered":"2023-12-19T18:14:54.045320+0000","last_clean":"2023-12-19T18:14:54.045320+0000","last_became_active":"2023-12-19T18:14:53.551105+0000","last_became_peered":"2023-12-19T18:14:53.551105+0000","last_unstale":"2023-12-19T18:14:54.045320+0000","last_undegraded":"2023-12-19T18:14:54.045320+0000","last_fullsized":"2023-12-19T18:14:54.045320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065908+0000","last_change":"2023-12-19T18:14:54.065539+0000","last_active":"2023-12-19T18:14:54.065908+0000","last_peered":"2023-12-19T18:14:54.065908+0000","last_clean":"2023-12-19T18:14:54.065908+0000","last_became_active":"2023-12-19T18:14:54.056448+0000","last_became_peered":"2023-12-19T18:14:54.056448+0000","last_unstale":"2023-12-19T18:14:54.065908+0000","last_undegraded":"2023-12-19T18:14:54.065908+0000","last_fullsized":"2023-12-19T18:14:54.065908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.045692+0000","last_change":"2023-12-19T18:14:53.554185+0000","last_active":"2023-12-19T18:14:54.045692+0000","last_peered":"2023-12-19T18:14:54.045692+0000","last_clean":"2023-12-19T18:14:54.045692+0000","last_became_active":"2023-12-19T18:14:53.553750+0000","last_became_peered":"2023-12-19T18:14:53.553750+0000","last_unstale":"2023-12-19T18:14:54.045692+0000","last_undegraded":"2023-12-19T18:14:54.045692+0000","last_fullsized":"2023-12-19T18:14:54.045692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.059756+0000","last_change":"2023-12-19T18:14:54.059501+0000","last_active":"2023-12-19T18:14:54.059756+0000","last_peered":"2023-12-19T18:14:54.059756+0000","last_clean":"2023-12-19T18:14:54.059756+0000","last_became_active":"2023-12-19T18:14:54.059055+0000","last_became_peered":"2023-12-19T18:14:54.059055+0000","last_unstale":"2023-12-19T18:14:54.059756+0000","last_undegraded":"2023-12-19T18:14:54.059756+0000","last_fullsized":"2023-12-19T18:14:54.059756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066496+0000","last_change":"2023-12-19T18:14:53.566640+0000","last_active":"2023-12-19T18:14:54.066496+0000","last_peered":"2023-12-19T18:14:54.066496+0000","last_clean":"2023-12-19T18:14:54.066496+0000","last_became_active":"2023-12-19T18:14:53.566413+0000","last_became_peered":"2023-12-19T18:14:53.566413+0000","last_unstale":"2023-12-19T18:14:54.066496+0000","last_undegraded":"2023-12-19T18:14:54.066496+0000","last_fullsized":"2023-12-19T18:14:54.066496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074538+0000","last_change":"2023-12-19T18:14:54.073374+0000","last_active":"2023-12-19T18:14:54.074538+0000","last_peered":"2023-12-19T18:14:54.074538+0000","last_clean":"2023-12-19T18:14:54.074538+0000","last_became_active":"2023-12-19T18:14:54.072676+0000","last_became_peered":"2023-12-19T18:14:54.072676+0000","last_unstale":"2023-12-19T18:14:54.074538+0000","last_undegraded":"2023-12-19T18:14:54.074538+0000","last_fullsized":"2023-12-19T18:14:54.074538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073825+0000","last_change":"2023-12-19T18:14:53.549245+0000","last_active":"2023-12-19T18:14:54.073825+0000","last_peered":"2023-12-19T18:14:54.073825+0000","last_clean":"2023-12-19T18:14:54.073825+0000","last_became_active":"2023-12-19T18:14:53.548992+0000","last_became_peered":"2023-12-19T18:14:53.548992+0000","last_unstale":"2023-12-19T18:14:54.073825+0000","last_undegraded":"2023-12-19T18:14:54.073825+0000","last_fullsized":"2023-12-19T18:14:54.073825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074558+0000","last_change":"2023-12-19T18:14:54.073488+0000","last_active":"2023-12-19T18:14:54.074558+0000","last_peered":"2023-12-19T18:14:54.074558+0000","last_clean":"2023-12-19T18:14:54.074558+0000","last_became_active":"2023-12-19T18:14:54.072567+0000","last_became_peered":"2023-12-19T18:14:54.072567+0000","last_unstale":"2023-12-19T18:14:54.074558+0000","last_undegraded":"2023-12-19T18:14:54.074558+0000","last_fullsized":"2023-12-19T18:14:54.074558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:22.428929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.057975+0000","last_change":"2023-12-19T18:14:54.057549+0000","last_active":"2023-12-19T18:14:54.057975+0000","last_peered":"2023-12-19T18:14:54.057975+0000","last_clean":"2023-12-19T18:14:54.057975+0000","last_became_active":"2023-12-19T18:14:54.056075+0000","last_became_peered":"2023-12-19T18:14:54.056075+0000","last_unstale":"2023-12-19T18:14:54.057975+0000","last_undegraded":"2023-12-19T18:14:54.057975+0000","last_fullsized":"2023-12-19T18:14:54.057975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:47.825262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064249+0000","last_change":"2023-12-19T18:14:53.555372+0000","last_active":"2023-12-19T18:14:54.064249+0000","last_peered":"2023-12-19T18:14:54.064249+0000","last_clean":"2023-12-19T18:14:54.064249+0000","last_became_active":"2023-12-19T18:14:53.555089+0000","last_became_peered":"2023-12-19T18:14:53.555089+0000","last_unstale":"2023-12-19T18:14:54.064249+0000","last_undegraded":"2023-12-19T18:14:54.064249+0000","last_fullsized":"2023-12-19T18:14:54.064249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065405+0000","last_change":"2023-12-19T18:14:54.065119+0000","last_active":"2023-12-19T18:14:54.065405+0000","last_peered":"2023-12-19T18:14:54.065405+0000","last_clean":"2023-12-19T18:14:54.065405+0000","last_became_active":"2023-12-19T18:14:54.064697+0000","last_became_peered":"2023-12-19T18:14:54.064697+0000","last_unstale":"2023-12-19T18:14:54.065405+0000","last_undegraded":"2023-12-19T18:14:54.065405+0000","last_fullsized":"2023-12-19T18:14:54.065405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063594+0000","last_change":"2023-12-19T18:14:53.556488+0000","last_active":"2023-12-19T18:14:54.063594+0000","last_peered":"2023-12-19T18:14:54.063594+0000","last_clean":"2023-12-19T18:14:54.063594+0000","last_became_active":"2023-12-19T18:14:53.555952+0000","last_became_peered":"2023-12-19T18:14:53.555952+0000","last_unstale":"2023-12-19T18:14:54.063594+0000","last_undegraded":"2023-12-19T18:14:54.063594+0000","last_fullsized":"2023-12-19T18:14:54.063594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070434+0000","last_change":"2023-12-19T18:14:54.070014+0000","last_active":"2023-12-19T18:14:54.070434+0000","last_peered":"2023-12-19T18:14:54.070434+0000","last_clean":"2023-12-19T18:14:54.070434+0000","last_became_active":"2023-12-19T18:14:54.067648+0000","last_became_peered":"2023-12-19T18:14:54.067648+0000","last_unstale":"2023-12-19T18:14:54.070434+0000","last_undegraded":"2023-12-19T18:14:54.070434+0000","last_fullsized":"2023-12-19T18:14:54.070434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067116+0000","last_change":"2023-12-19T18:14:53.566090+0000","last_active":"2023-12-19T18:14:54.067116+0000","last_peered":"2023-12-19T18:14:54.067116+0000","last_clean":"2023-12-19T18:14:54.067116+0000","last_became_active":"2023-12-19T18:14:53.565782+0000","last_became_peered":"2023-12-19T18:14:53.565782+0000","last_unstale":"2023-12-19T18:14:54.067116+0000","last_undegraded":"2023-12-19T18:14:54.067116+0000","last_fullsized":"2023-12-19T18:14:54.067116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063312+0000","last_change":"2023-12-19T18:14:54.060686+0000","last_active":"2023-12-19T18:14:54.063312+0000","last_peered":"2023-12-19T18:14:54.063312+0000","last_clean":"2023-12-19T18:14:54.063312+0000","last_became_active":"2023-12-19T18:14:54.059977+0000","last_became_peered":"2023-12-19T18:14:54.059977+0000","last_unstale":"2023-12-19T18:14:54.063312+0000","last_undegraded":"2023-12-19T18:14:54.063312+0000","last_fullsized":"2023-12-19T18:14:54.063312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073092+0000","last_change":"2023-12-19T18:14:53.558205+0000","last_active":"2023-12-19T18:14:54.073092+0000","last_peered":"2023-12-19T18:14:54.073092+0000","last_clean":"2023-12-19T18:14:54.073092+0000","last_became_active":"2023-12-19T18:14:53.557733+0000","last_became_peered":"2023-12-19T18:14:53.557733+0000","last_unstale":"2023-12-19T18:14:54.073092+0000","last_undegraded":"2023-12-19T18:14:54.073092+0000","last_fullsized":"2023-12-19T18:14:54.073092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929138,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":71,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927026,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199928680,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927698,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.54}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.613}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338824,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929206,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]}],"pool_statfs":[]}} 2023-12-19T18:15:04.201 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:15:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.450+0000 7f42e2129640 1 Processor -- start 2023-12-19T18:15:04.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.450+0000 7f42e2129640 1 -- start start 2023-12-19T18:15:04.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e2129640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f42dc1604d0 0x7f42dc0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e2129640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42dc0d2350 0x7f42dc0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e2129640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc0d0300 0x7f42dc0d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e2129640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42dc0d2b30 con 0x7f42dc0d2350 2023-12-19T18:15:04.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e2129640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42dc0b5e90 con 0x7f42dc1604d0 2023-12-19T18:15:04.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e2129640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc0d2850 con 0x7f42dc0d0300 2023-12-19T18:15:04.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e0926640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42dc0d2350 0x7f42dc0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e0926640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42dc0d2350 0x7f42dc0b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49426/0 (socket says 172.21.15.27:49426) 2023-12-19T18:15:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e0926640 1 -- 172.21.15.27:0/966258311 learned_addr learned my addr 172.21.15.27:0/966258311 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e1127640 1 --2- 172.21.15.27:0/966258311 >> v2:172.21.15.192:3300/0 conn(0x7f42dc1604d0 0x7f42dc0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e1928640 1 --2- 172.21.15.27:0/966258311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc0d0300 0x7f42dc0d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e0926640 1 -- 172.21.15.27:0/966258311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc0d0300 msgr2=0x7f42dc0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:04.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e0926640 1 --2- 172.21.15.27:0/966258311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc0d0300 0x7f42dc0d0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e0926640 1 -- 172.21.15.27:0/966258311 >> v2:172.21.15.192:3300/0 conn(0x7f42dc1604d0 msgr2=0x7f42dc0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e0926640 1 --2- 172.21.15.27:0/966258311 >> v2:172.21.15.192:3300/0 conn(0x7f42dc1604d0 0x7f42dc0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e0926640 1 -- 172.21.15.27:0/966258311 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42dc0cded0 con 0x7f42dc0d2350 2023-12-19T18:15:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.452+0000 7f42e1127640 1 --2- 172.21.15.27:0/966258311 >> v2:172.21.15.192:3300/0 conn(0x7f42dc1604d0 0x7f42dc0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:04.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.454+0000 7f42e0926640 1 --2- 172.21.15.27:0/966258311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc0d2350 0x7f42dc0b4d20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42dc06a040 tx=0x7f42c802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7ebffffce3a77ca server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.454+0000 7f42dbfff640 1 -- 172.21.15.27:0/966258311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8031870 con 0x7f42dc0d2350 2023-12-19T18:15:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.454+0000 7f42dbfff640 1 -- 172.21.15.27:0/966258311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c80319d0 con 0x7f42dc0d2350 2023-12-19T18:15:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.454+0000 7f42dbfff640 1 -- 172.21.15.27:0/966258311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c8031d60 con 0x7f42dc0d2350 2023-12-19T18:15:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.455+0000 7f42e2129640 1 -- 172.21.15.27:0/966258311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc0d2350 msgr2=0x7f42dc0b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.455+0000 7f42e2129640 1 --2- 172.21.15.27:0/966258311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc0d2350 0x7f42dc0b4d20 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f42dc06a040 tx=0x7f42c802b910 comp rx=0 tx=0).stop 2023-12-19T18:15:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.455+0000 7f42e2129640 1 -- 172.21.15.27:0/966258311 shutdown_connections 2023-12-19T18:15:04.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.455+0000 7f42e2129640 1 --2- 172.21.15.27:0/966258311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc0d0300 0x7f42dc0d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.455+0000 7f42e2129640 1 --2- 172.21.15.27:0/966258311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc0d2350 0x7f42dc0b4d20 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.455+0000 7f42e2129640 1 --2- 172.21.15.27:0/966258311 >> v2:172.21.15.192:3300/0 conn(0x7f42dc1604d0 0x7f42dc0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.455+0000 7f42e2129640 1 -- 172.21.15.27:0/966258311 >> 172.21.15.27:0/966258311 conn(0x7f42dc0cf7e0 msgr2=0x7f42dc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.455+0000 7f42e2129640 1 -- 172.21.15.27:0/966258311 shutdown_connections 2023-12-19T18:15:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.455+0000 7f42e2129640 1 -- 172.21.15.27:0/966258311 wait complete. 2023-12-19T18:15:04.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.457+0000 7f42e2129640 1 Processor -- start 2023-12-19T18:15:04.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.457+0000 7f42e2129640 1 -- start start 2023-12-19T18:15:04.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e2129640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc0d0300 0x7f42dc0d19f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e2129640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42dc0d2350 0x7f42dc148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e2129640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc1604d0 0x7f42dc1491c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e2129640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc149700 con 0x7f42dc1604d0 2023-12-19T18:15:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e2129640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc1e87d0 con 0x7f42dc0d2350 2023-12-19T18:15:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e2129640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42dc1e8ab0 con 0x7f42dc0d0300 2023-12-19T18:15:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e1928640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc1604d0 0x7f42dc1491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e1928640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc1604d0 0x7f42dc1491c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49442/0 (socket says 172.21.15.27:49442) 2023-12-19T18:15:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e1928640 1 -- 172.21.15.27:0/204279552 learned_addr learned my addr 172.21.15.27:0/204279552 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e1127640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc0d0300 0x7f42dc0d19f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e0926640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42dc0d2350 0x7f42dc148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e1928640 1 -- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc0d0300 msgr2=0x7f42dc0d19f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e1928640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc0d0300 0x7f42dc0d19f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e1928640 1 -- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42dc0d2350 msgr2=0x7f42dc148c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.458+0000 7f42e1928640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42dc0d2350 0x7f42dc148c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.459+0000 7f42e1928640 1 -- 172.21.15.27:0/204279552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c80094d0 con 0x7f42dc1604d0 2023-12-19T18:15:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.459+0000 7f42e0926640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42dc0d2350 0x7f42dc148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.459+0000 7f42e1928640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc1604d0 0x7f42dc1491c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f42d400c4c0 tx=0x7f42d400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:04.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.459+0000 7f42d9ffb640 1 -- 172.21.15.27:0/204279552 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4004220 con 0x7f42dc1604d0 2023-12-19T18:15:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.459+0000 7f42d9ffb640 1 -- 172.21.15.27:0/204279552 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d4004380 con 0x7f42dc1604d0 2023-12-19T18:15:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.459+0000 7f42d9ffb640 1 -- 172.21.15.27:0/204279552 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4003810 con 0x7f42dc1604d0 2023-12-19T18:15:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.459+0000 7f42e2129640 1 -- 172.21.15.27:0/204279552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42dc1e8d90 con 0x7f42dc1604d0 2023-12-19T18:15:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.459+0000 7f42e2129640 1 -- 172.21.15.27:0/204279552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42dc1e9570 con 0x7f42dc1604d0 2023-12-19T18:15:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.460+0000 7f42e2129640 1 -- 172.21.15.27:0/204279552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42dc066f80 con 0x7f42dc1604d0 2023-12-19T18:15:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.461+0000 7f42d9ffb640 1 -- 172.21.15.27:0/204279552 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f42d4003970 con 0x7f42dc1604d0 2023-12-19T18:15:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.462+0000 7f42d9ffb640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f42c006a430 0x7f42c006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.462+0000 7f42d9ffb640 1 -- 172.21.15.27:0/204279552 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42d4088620 con 0x7f42dc1604d0 2023-12-19T18:15:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.463+0000 7f42e1127640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f42c006a430 0x7f42c006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.464+0000 7f42e1127640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f42c006a430 0x7f42c006c8f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f42d00075b0 tx=0x7f42d0007540 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:04.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.467+0000 7f42d9ffb640 1 -- 172.21.15.27:0/204279552 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f42d4053b70 con 0x7f42dc1604d0 2023-12-19T18:15:04.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.658+0000 7f42e2129640 1 -- 172.21.15.27:0/204279552 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f42dc0ce490 con 0x7f42c006a430 2023-12-19T18:15:04.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.679+0000 7f42d9ffb640 1 -- 172.21.15.27:0/204279552 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260098 (secure 0 0 0) 0x7f42d0007cf0 con 0x7f42c006a430 2023-12-19T18:15:04.683 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:04.685 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:15:04.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.684+0000 7f42e2129640 1 -- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f42c006a430 msgr2=0x7f42c006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:04.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.684+0000 7f42e2129640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f42c006a430 0x7f42c006c8f0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f42d00075b0 tx=0x7f42d0007540 comp rx=0 tx=0).stop 2023-12-19T18:15:04.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.684+0000 7f42e2129640 1 -- 172.21.15.27:0/204279552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc1604d0 msgr2=0x7f42dc1491c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:04.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.684+0000 7f42e2129640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc1604d0 0x7f42dc1491c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f42d400c4c0 tx=0x7f42d400c980 comp rx=0 tx=0).stop 2023-12-19T18:15:04.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.684+0000 7f42e2129640 1 -- 172.21.15.27:0/204279552 shutdown_connections 2023-12-19T18:15:04.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.684+0000 7f42e2129640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f42c006a430 0x7f42c006c8f0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.684+0000 7f42e2129640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42dc1604d0 0x7f42dc1491c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.684+0000 7f42e2129640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42dc0d2350 0x7f42dc148c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.684+0000 7f42e2129640 1 --2- 172.21.15.27:0/204279552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42dc0d0300 0x7f42dc0d19f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.684+0000 7f42e2129640 1 -- 172.21.15.27:0/204279552 >> 172.21.15.27:0/204279552 conn(0x7f42dc0cf7e0 msgr2=0x7f42dc0b3050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:04.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.684+0000 7f42e2129640 1 -- 172.21.15.27:0/204279552 shutdown_connections 2023-12-19T18:15:04.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.685+0000 7f42e2129640 1 -- 172.21.15.27:0/204279552 wait complete. 2023-12-19T18:15:04.704 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-12-19T18:15:03.704846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248636,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.661897"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056823+0000","last_change":"2023-12-19T18:14:54.056711+0000","last_active":"2023-12-19T18:14:54.056823+0000","last_peered":"2023-12-19T18:14:54.056823+0000","last_clean":"2023-12-19T18:14:54.056823+0000","last_became_active":"2023-12-19T18:14:54.056335+0000","last_became_peered":"2023-12-19T18:14:54.056335+0000","last_unstale":"2023-12-19T18:14:54.056823+0000","last_undegraded":"2023-12-19T18:14:54.056823+0000","last_fullsized":"2023-12-19T18:14:54.056823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069974+0000","last_change":"2023-12-19T18:14:54.069882+0000","last_active":"2023-12-19T18:14:54.069974+0000","last_peered":"2023-12-19T18:14:54.069974+0000","last_clean":"2023-12-19T18:14:54.069974+0000","last_became_active":"2023-12-19T18:14:54.069626+0000","last_became_peered":"2023-12-19T18:14:54.069626+0000","last_unstale":"2023-12-19T18:14:54.069974+0000","last_undegraded":"2023-12-19T18:14:54.069974+0000","last_fullsized":"2023-12-19T18:14:54.069974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053578+0000","last_change":"2023-12-19T18:14:54.053111+0000","last_active":"2023-12-19T18:14:54.053578+0000","last_peered":"2023-12-19T18:14:54.053578+0000","last_clean":"2023-12-19T18:14:54.053578+0000","last_became_active":"2023-12-19T18:14:54.052858+0000","last_became_peered":"2023-12-19T18:14:54.052858+0000","last_unstale":"2023-12-19T18:14:54.053578+0000","last_undegraded":"2023-12-19T18:14:54.053578+0000","last_fullsized":"2023-12-19T18:14:54.053578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050310+0000","last_change":"2023-12-19T18:14:54.049862+0000","last_active":"2023-12-19T18:14:54.050310+0000","last_peered":"2023-12-19T18:14:54.050310+0000","last_clean":"2023-12-19T18:14:54.050310+0000","last_became_active":"2023-12-19T18:14:54.049396+0000","last_became_peered":"2023-12-19T18:14:54.049396+0000","last_unstale":"2023-12-19T18:14:54.050310+0000","last_undegraded":"2023-12-19T18:14:54.050310+0000","last_fullsized":"2023-12-19T18:14:54.050310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053036+0000","last_change":"2023-12-19T18:14:54.052387+0000","last_active":"2023-12-19T18:14:54.053036+0000","last_peered":"2023-12-19T18:14:54.053036+0000","last_clean":"2023-12-19T18:14:54.053036+0000","last_became_active":"2023-12-19T18:14:54.052008+0000","last_became_peered":"2023-12-19T18:14:54.052008+0000","last_unstale":"2023-12-19T18:14:54.053036+0000","last_undegraded":"2023-12-19T18:14:54.053036+0000","last_fullsized":"2023-12-19T18:14:54.053036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061254+0000","last_change":"2023-12-19T18:14:54.061175+0000","last_active":"2023-12-19T18:14:54.061254+0000","last_peered":"2023-12-19T18:14:54.061254+0000","last_clean":"2023-12-19T18:14:54.061254+0000","last_became_active":"2023-12-19T18:14:54.060192+0000","last_became_peered":"2023-12-19T18:14:54.060192+0000","last_unstale":"2023-12-19T18:14:54.061254+0000","last_undegraded":"2023-12-19T18:14:54.061254+0000","last_fullsized":"2023-12-19T18:14:54.061254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.054262+0000","last_change":"2023-12-19T18:14:54.053283+0000","last_active":"2023-12-19T18:14:54.054262+0000","last_peered":"2023-12-19T18:14:54.054262+0000","last_clean":"2023-12-19T18:14:54.054262+0000","last_became_active":"2023-12-19T18:14:54.052854+0000","last_became_peered":"2023-12-19T18:14:54.052854+0000","last_unstale":"2023-12-19T18:14:54.054262+0000","last_undegraded":"2023-12-19T18:14:54.054262+0000","last_fullsized":"2023-12-19T18:14:54.054262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056579+0000","last_change":"2023-12-19T18:14:54.056483+0000","last_active":"2023-12-19T18:14:54.056579+0000","last_peered":"2023-12-19T18:14:54.056579+0000","last_clean":"2023-12-19T18:14:54.056579+0000","last_became_active":"2023-12-19T18:14:54.056196+0000","last_became_peered":"2023-12-19T18:14:54.056196+0000","last_unstale":"2023-12-19T18:14:54.056579+0000","last_undegraded":"2023-12-19T18:14:54.056579+0000","last_fullsized":"2023-12-19T18:14:54.056579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075945+0000","last_change":"2023-12-19T18:14:54.074709+0000","last_active":"2023-12-19T18:14:54.075945+0000","last_peered":"2023-12-19T18:14:54.075945+0000","last_clean":"2023-12-19T18:14:54.075945+0000","last_became_active":"2023-12-19T18:14:54.074482+0000","last_became_peered":"2023-12-19T18:14:54.074482+0000","last_unstale":"2023-12-19T18:14:54.075945+0000","last_undegraded":"2023-12-19T18:14:54.075945+0000","last_fullsized":"2023-12-19T18:14:54.075945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051054+0000","last_change":"2023-12-19T18:14:54.050203+0000","last_active":"2023-12-19T18:14:54.051054+0000","last_peered":"2023-12-19T18:14:54.051054+0000","last_clean":"2023-12-19T18:14:54.051054+0000","last_became_active":"2023-12-19T18:14:54.049582+0000","last_became_peered":"2023-12-19T18:14:54.049582+0000","last_unstale":"2023-12-19T18:14:54.051054+0000","last_undegraded":"2023-12-19T18:14:54.051054+0000","last_fullsized":"2023-12-19T18:14:54.051054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056806+0000","last_change":"2023-12-19T18:14:54.056564+0000","last_active":"2023-12-19T18:14:54.056806+0000","last_peered":"2023-12-19T18:14:54.056806+0000","last_clean":"2023-12-19T18:14:54.056806+0000","last_became_active":"2023-12-19T18:14:54.056052+0000","last_became_peered":"2023-12-19T18:14:54.056052+0000","last_unstale":"2023-12-19T18:14:54.056806+0000","last_undegraded":"2023-12-19T18:14:54.056806+0000","last_fullsized":"2023-12-19T18:14:54.056806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066528+0000","last_change":"2023-12-19T18:14:54.066257+0000","last_active":"2023-12-19T18:14:54.066528+0000","last_peered":"2023-12-19T18:14:54.066528+0000","last_clean":"2023-12-19T18:14:54.066528+0000","last_became_active":"2023-12-19T18:14:54.065876+0000","last_became_peered":"2023-12-19T18:14:54.065876+0000","last_unstale":"2023-12-19T18:14:54.066528+0000","last_undegraded":"2023-12-19T18:14:54.066528+0000","last_fullsized":"2023-12-19T18:14:54.066528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070727+0000","last_change":"2023-12-19T18:14:54.070658+0000","last_active":"2023-12-19T18:14:54.070727+0000","last_peered":"2023-12-19T18:14:54.070727+0000","last_clean":"2023-12-19T18:14:54.070727+0000","last_became_active":"2023-12-19T18:14:54.065952+0000","last_became_peered":"2023-12-19T18:14:54.065952+0000","last_unstale":"2023-12-19T18:14:54.070727+0000","last_undegraded":"2023-12-19T18:14:54.070727+0000","last_fullsized":"2023-12-19T18:14:54.070727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:35:28.203585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072334+0000","last_change":"2023-12-19T18:14:54.072211+0000","last_active":"2023-12-19T18:14:54.072334+0000","last_peered":"2023-12-19T18:14:54.072334+0000","last_clean":"2023-12-19T18:14:54.072334+0000","last_became_active":"2023-12-19T18:14:54.071912+0000","last_became_peered":"2023-12-19T18:14:54.071912+0000","last_unstale":"2023-12-19T18:14:54.072334+0000","last_undegraded":"2023-12-19T18:14:54.072334+0000","last_fullsized":"2023-12-19T18:14:54.072334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056480+0000","last_change":"2023-12-19T18:14:54.055761+0000","last_active":"2023-12-19T18:14:54.056480+0000","last_peered":"2023-12-19T18:14:54.056480+0000","last_clean":"2023-12-19T18:14:54.056480+0000","last_became_active":"2023-12-19T18:14:54.055529+0000","last_became_peered":"2023-12-19T18:14:54.055529+0000","last_unstale":"2023-12-19T18:14:54.056480+0000","last_undegraded":"2023-12-19T18:14:54.056480+0000","last_fullsized":"2023-12-19T18:14:54.056480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073307+0000","last_change":"2023-12-19T18:14:54.072540+0000","last_active":"2023-12-19T18:14:54.073307+0000","last_peered":"2023-12-19T18:14:54.073307+0000","last_clean":"2023-12-19T18:14:54.073307+0000","last_became_active":"2023-12-19T18:14:54.072162+0000","last_became_peered":"2023-12-19T18:14:54.072162+0000","last_unstale":"2023-12-19T18:14:54.073307+0000","last_undegraded":"2023-12-19T18:14:54.073307+0000","last_fullsized":"2023-12-19T18:14:54.073307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075801+0000","last_change":"2023-12-19T18:14:54.075571+0000","last_active":"2023-12-19T18:14:54.075801+0000","last_peered":"2023-12-19T18:14:54.075801+0000","last_clean":"2023-12-19T18:14:54.075801+0000","last_became_active":"2023-12-19T18:14:54.074613+0000","last_became_peered":"2023-12-19T18:14:54.074613+0000","last_unstale":"2023-12-19T18:14:54.075801+0000","last_undegraded":"2023-12-19T18:14:54.075801+0000","last_fullsized":"2023-12-19T18:14:54.075801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:15.120564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.062708+0000","last_change":"2023-12-19T18:14:54.062640+0000","last_active":"2023-12-19T18:14:54.062708+0000","last_peered":"2023-12-19T18:14:54.062708+0000","last_clean":"2023-12-19T18:14:54.062708+0000","last_became_active":"2023-12-19T18:14:54.061994+0000","last_became_peered":"2023-12-19T18:14:54.061994+0000","last_unstale":"2023-12-19T18:14:54.062708+0000","last_undegraded":"2023-12-19T18:14:54.062708+0000","last_fullsized":"2023-12-19T18:14:54.062708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073330+0000","last_change":"2023-12-19T18:14:54.073175+0000","last_active":"2023-12-19T18:14:54.073330+0000","last_peered":"2023-12-19T18:14:54.073330+0000","last_clean":"2023-12-19T18:14:54.073330+0000","last_became_active":"2023-12-19T18:14:54.072223+0000","last_became_peered":"2023-12-19T18:14:54.072223+0000","last_unstale":"2023-12-19T18:14:54.073330+0000","last_undegraded":"2023-12-19T18:14:54.073330+0000","last_fullsized":"2023-12-19T18:14:54.073330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064684+0000","last_change":"2023-12-19T18:14:54.064206+0000","last_active":"2023-12-19T18:14:54.064684+0000","last_peered":"2023-12-19T18:14:54.064684+0000","last_clean":"2023-12-19T18:14:54.064684+0000","last_became_active":"2023-12-19T18:14:54.063818+0000","last_became_peered":"2023-12-19T18:14:54.063818+0000","last_unstale":"2023-12-19T18:14:54.064684+0000","last_undegraded":"2023-12-19T18:14:54.064684+0000","last_fullsized":"2023-12-19T18:14:54.064684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063666+0000","last_change":"2023-12-19T18:14:54.063234+0000","last_active":"2023-12-19T18:14:54.063666+0000","last_peered":"2023-12-19T18:14:54.063666+0000","last_clean":"2023-12-19T18:14:54.063666+0000","last_became_active":"2023-12-19T18:14:54.062885+0000","last_became_peered":"2023-12-19T18:14:54.062885+0000","last_unstale":"2023-12-19T18:14:54.063666+0000","last_undegraded":"2023-12-19T18:14:54.063666+0000","last_fullsized":"2023-12-19T18:14:54.063666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063662+0000","last_change":"2023-12-19T18:14:54.063598+0000","last_active":"2023-12-19T18:14:54.063662+0000","last_peered":"2023-12-19T18:14:54.063662+0000","last_clean":"2023-12-19T18:14:54.063662+0000","last_became_active":"2023-12-19T18:14:54.063314+0000","last_became_peered":"2023-12-19T18:14:54.063314+0000","last_unstale":"2023-12-19T18:14:54.063662+0000","last_undegraded":"2023-12-19T18:14:54.063662+0000","last_fullsized":"2023-12-19T18:14:54.063662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060228+0000","last_change":"2023-12-19T18:14:54.059917+0000","last_active":"2023-12-19T18:14:54.060228+0000","last_peered":"2023-12-19T18:14:54.060228+0000","last_clean":"2023-12-19T18:14:54.060228+0000","last_became_active":"2023-12-19T18:14:54.059674+0000","last_became_peered":"2023-12-19T18:14:54.059674+0000","last_unstale":"2023-12-19T18:14:54.060228+0000","last_undegraded":"2023-12-19T18:14:54.060228+0000","last_fullsized":"2023-12-19T18:14:54.060228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074581+0000","last_change":"2023-12-19T18:14:54.073659+0000","last_active":"2023-12-19T18:14:54.074581+0000","last_peered":"2023-12-19T18:14:54.074581+0000","last_clean":"2023-12-19T18:14:54.074581+0000","last_became_active":"2023-12-19T18:14:54.072947+0000","last_became_peered":"2023-12-19T18:14:54.072947+0000","last_unstale":"2023-12-19T18:14:54.074581+0000","last_undegraded":"2023-12-19T18:14:54.074581+0000","last_fullsized":"2023-12-19T18:14:54.074581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067036+0000","last_change":"2023-12-19T18:14:54.066189+0000","last_active":"2023-12-19T18:14:54.067036+0000","last_peered":"2023-12-19T18:14:54.067036+0000","last_clean":"2023-12-19T18:14:54.067036+0000","last_became_active":"2023-12-19T18:14:54.065921+0000","last_became_peered":"2023-12-19T18:14:54.065921+0000","last_unstale":"2023-12-19T18:14:54.067036+0000","last_undegraded":"2023-12-19T18:14:54.067036+0000","last_fullsized":"2023-12-19T18:14:54.067036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061440+0000","last_change":"2023-12-19T18:14:54.061372+0000","last_active":"2023-12-19T18:14:54.061440+0000","last_peered":"2023-12-19T18:14:54.061440+0000","last_clean":"2023-12-19T18:14:54.061440+0000","last_became_active":"2023-12-19T18:14:54.060949+0000","last_became_peered":"2023-12-19T18:14:54.060949+0000","last_unstale":"2023-12-19T18:14:54.061440+0000","last_undegraded":"2023-12-19T18:14:54.061440+0000","last_fullsized":"2023-12-19T18:14:54.061440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073948+0000","last_change":"2023-12-19T18:14:54.073038+0000","last_active":"2023-12-19T18:14:54.073948+0000","last_peered":"2023-12-19T18:14:54.073948+0000","last_clean":"2023-12-19T18:14:54.073948+0000","last_became_active":"2023-12-19T18:14:54.071844+0000","last_became_peered":"2023-12-19T18:14:54.071844+0000","last_unstale":"2023-12-19T18:14:54.073948+0000","last_undegraded":"2023-12-19T18:14:54.073948+0000","last_fullsized":"2023-12-19T18:14:54.073948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070883+0000","last_change":"2023-12-19T18:14:54.070817+0000","last_active":"2023-12-19T18:14:54.070883+0000","last_peered":"2023-12-19T18:14:54.070883+0000","last_clean":"2023-12-19T18:14:54.070883+0000","last_became_active":"2023-12-19T18:14:54.068216+0000","last_became_peered":"2023-12-19T18:14:54.068216+0000","last_unstale":"2023-12-19T18:14:54.070883+0000","last_undegraded":"2023-12-19T18:14:54.070883+0000","last_fullsized":"2023-12-19T18:14:54.070883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063273+0000","last_change":"2023-12-19T18:14:54.060522+0000","last_active":"2023-12-19T18:14:54.063273+0000","last_peered":"2023-12-19T18:14:54.063273+0000","last_clean":"2023-12-19T18:14:54.063273+0000","last_became_active":"2023-12-19T18:14:54.059641+0000","last_became_peered":"2023-12-19T18:14:54.059641+0000","last_unstale":"2023-12-19T18:14:54.063273+0000","last_undegraded":"2023-12-19T18:14:54.063273+0000","last_fullsized":"2023-12-19T18:14:54.063273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065928+0000","last_change":"2023-12-19T18:14:54.065766+0000","last_active":"2023-12-19T18:14:54.065928+0000","last_peered":"2023-12-19T18:14:54.065928+0000","last_clean":"2023-12-19T18:14:54.065928+0000","last_became_active":"2023-12-19T18:14:54.058571+0000","last_became_peered":"2023-12-19T18:14:54.058571+0000","last_unstale":"2023-12-19T18:14:54.065928+0000","last_undegraded":"2023-12-19T18:14:54.065928+0000","last_fullsized":"2023-12-19T18:14:54.065928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:47.213095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070429+0000","last_change":"2023-12-19T18:14:54.067798+0000","last_active":"2023-12-19T18:14:54.070429+0000","last_peered":"2023-12-19T18:14:54.070429+0000","last_clean":"2023-12-19T18:14:54.070429+0000","last_became_active":"2023-12-19T18:14:54.067576+0000","last_became_peered":"2023-12-19T18:14:54.067576+0000","last_unstale":"2023-12-19T18:14:54.070429+0000","last_undegraded":"2023-12-19T18:14:54.070429+0000","last_fullsized":"2023-12-19T18:14:54.070429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069889+0000","last_change":"2023-12-19T18:14:54.069741+0000","last_active":"2023-12-19T18:14:54.069889+0000","last_peered":"2023-12-19T18:14:54.069889+0000","last_clean":"2023-12-19T18:14:54.069889+0000","last_became_active":"2023-12-19T18:14:54.069513+0000","last_became_peered":"2023-12-19T18:14:54.069513+0000","last_unstale":"2023-12-19T18:14:54.069889+0000","last_undegraded":"2023-12-19T18:14:54.069889+0000","last_fullsized":"2023-12-19T18:14:54.069889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070495+0000","last_change":"2023-12-19T18:14:54.070403+0000","last_active":"2023-12-19T18:14:54.070495+0000","last_peered":"2023-12-19T18:14:54.070495+0000","last_clean":"2023-12-19T18:14:54.070495+0000","last_became_active":"2023-12-19T18:14:54.069801+0000","last_became_peered":"2023-12-19T18:14:54.069801+0000","last_unstale":"2023-12-19T18:14:54.070495+0000","last_undegraded":"2023-12-19T18:14:54.070495+0000","last_fullsized":"2023-12-19T18:14:54.070495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068551+0000","last_change":"2023-12-19T18:14:53.563064+0000","last_active":"2023-12-19T18:14:54.068551+0000","last_peered":"2023-12-19T18:14:54.068551+0000","last_clean":"2023-12-19T18:14:54.068551+0000","last_became_active":"2023-12-19T18:14:53.561534+0000","last_became_peered":"2023-12-19T18:14:53.561534+0000","last_unstale":"2023-12-19T18:14:54.068551+0000","last_undegraded":"2023-12-19T18:14:54.068551+0000","last_fullsized":"2023-12-19T18:14:54.068551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051639+0000","last_change":"2023-12-19T18:14:54.051570+0000","last_active":"2023-12-19T18:14:54.051639+0000","last_peered":"2023-12-19T18:14:54.051639+0000","last_clean":"2023-12-19T18:14:54.051639+0000","last_became_active":"2023-12-19T18:14:54.051173+0000","last_became_peered":"2023-12-19T18:14:54.051173+0000","last_unstale":"2023-12-19T18:14:54.051639+0000","last_undegraded":"2023-12-19T18:14:54.051639+0000","last_fullsized":"2023-12-19T18:14:54.051639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073383+0000","last_change":"2023-12-19T18:14:53.560763+0000","last_active":"2023-12-19T18:14:54.073383+0000","last_peered":"2023-12-19T18:14:54.073383+0000","last_clean":"2023-12-19T18:14:54.073383+0000","last_became_active":"2023-12-19T18:14:53.558648+0000","last_became_peered":"2023-12-19T18:14:53.558648+0000","last_unstale":"2023-12-19T18:14:54.073383+0000","last_undegraded":"2023-12-19T18:14:54.073383+0000","last_fullsized":"2023-12-19T18:14:54.073383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:51.179899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066389+0000","last_change":"2023-12-19T18:14:53.560483+0000","last_active":"2023-12-19T18:14:54.066389+0000","last_peered":"2023-12-19T18:14:54.066389+0000","last_clean":"2023-12-19T18:14:54.066389+0000","last_became_active":"2023-12-19T18:14:53.560244+0000","last_became_peered":"2023-12-19T18:14:53.560244+0000","last_unstale":"2023-12-19T18:14:54.066389+0000","last_undegraded":"2023-12-19T18:14:54.066389+0000","last_fullsized":"2023-12-19T18:14:54.066389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072273+0000","last_change":"2023-12-19T18:14:54.071383+0000","last_active":"2023-12-19T18:14:54.072273+0000","last_peered":"2023-12-19T18:14:54.072273+0000","last_clean":"2023-12-19T18:14:54.072273+0000","last_became_active":"2023-12-19T18:14:54.070954+0000","last_became_peered":"2023-12-19T18:14:54.070954+0000","last_unstale":"2023-12-19T18:14:54.072273+0000","last_undegraded":"2023-12-19T18:14:54.072273+0000","last_fullsized":"2023-12-19T18:14:54.072273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053551+0000","last_change":"2023-12-19T18:14:53.558346+0000","last_active":"2023-12-19T18:14:54.053551+0000","last_peered":"2023-12-19T18:14:54.053551+0000","last_clean":"2023-12-19T18:14:54.053551+0000","last_became_active":"2023-12-19T18:14:53.558124+0000","last_became_peered":"2023-12-19T18:14:53.558124+0000","last_unstale":"2023-12-19T18:14:54.053551+0000","last_undegraded":"2023-12-19T18:14:54.053551+0000","last_fullsized":"2023-12-19T18:14:54.053551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066236+0000","last_change":"2023-12-19T18:14:54.064228+0000","last_active":"2023-12-19T18:14:54.066236+0000","last_peered":"2023-12-19T18:14:54.066236+0000","last_clean":"2023-12-19T18:14:54.066236+0000","last_became_active":"2023-12-19T18:14:54.063991+0000","last_became_peered":"2023-12-19T18:14:54.063991+0000","last_unstale":"2023-12-19T18:14:54.066236+0000","last_undegraded":"2023-12-19T18:14:54.066236+0000","last_fullsized":"2023-12-19T18:14:54.066236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064163+0000","last_change":"2023-12-19T18:14:53.554048+0000","last_active":"2023-12-19T18:14:54.064163+0000","last_peered":"2023-12-19T18:14:54.064163+0000","last_clean":"2023-12-19T18:14:54.064163+0000","last_became_active":"2023-12-19T18:14:53.553811+0000","last_became_peered":"2023-12-19T18:14:53.553811+0000","last_unstale":"2023-12-19T18:14:54.064163+0000","last_undegraded":"2023-12-19T18:14:54.064163+0000","last_fullsized":"2023-12-19T18:14:54.064163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073051+0000","last_change":"2023-12-19T18:14:54.072974+0000","last_active":"2023-12-19T18:14:54.073051+0000","last_peered":"2023-12-19T18:14:54.073051+0000","last_clean":"2023-12-19T18:14:54.073051+0000","last_became_active":"2023-12-19T18:14:54.072176+0000","last_became_peered":"2023-12-19T18:14:54.072176+0000","last_unstale":"2023-12-19T18:14:54.073051+0000","last_undegraded":"2023-12-19T18:14:54.073051+0000","last_fullsized":"2023-12-19T18:14:54.073051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.043565+0000","last_change":"2023-12-19T18:14:53.557947+0000","last_active":"2023-12-19T18:14:54.043565+0000","last_peered":"2023-12-19T18:14:54.043565+0000","last_clean":"2023-12-19T18:14:54.043565+0000","last_became_active":"2023-12-19T18:14:53.557713+0000","last_became_peered":"2023-12-19T18:14:53.557713+0000","last_unstale":"2023-12-19T18:14:54.043565+0000","last_undegraded":"2023-12-19T18:14:54.043565+0000","last_fullsized":"2023-12-19T18:14:54.043565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056501+0000","last_change":"2023-12-19T18:14:54.056275+0000","last_active":"2023-12-19T18:14:54.056501+0000","last_peered":"2023-12-19T18:14:54.056501+0000","last_clean":"2023-12-19T18:14:54.056501+0000","last_became_active":"2023-12-19T18:14:54.056002+0000","last_became_peered":"2023-12-19T18:14:54.056002+0000","last_unstale":"2023-12-19T18:14:54.056501+0000","last_undegraded":"2023-12-19T18:14:54.056501+0000","last_fullsized":"2023-12-19T18:14:54.056501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:30.847801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073216+0000","last_change":"2023-12-19T18:14:53.568016+0000","last_active":"2023-12-19T18:14:54.073216+0000","last_peered":"2023-12-19T18:14:54.073216+0000","last_clean":"2023-12-19T18:14:54.073216+0000","last_became_active":"2023-12-19T18:14:53.567634+0000","last_became_peered":"2023-12-19T18:14:53.567634+0000","last_unstale":"2023-12-19T18:14:54.073216+0000","last_undegraded":"2023-12-19T18:14:54.073216+0000","last_fullsized":"2023-12-19T18:14:54.073216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075897+0000","last_change":"2023-12-19T18:14:54.075827+0000","last_active":"2023-12-19T18:14:54.075897+0000","last_peered":"2023-12-19T18:14:54.075897+0000","last_clean":"2023-12-19T18:14:54.075897+0000","last_became_active":"2023-12-19T18:14:54.074747+0000","last_became_peered":"2023-12-19T18:14:54.074747+0000","last_unstale":"2023-12-19T18:14:54.075897+0000","last_undegraded":"2023-12-19T18:14:54.075897+0000","last_fullsized":"2023-12-19T18:14:54.075897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073455+0000","last_change":"2023-12-19T18:14:53.547427+0000","last_active":"2023-12-19T18:14:54.073455+0000","last_peered":"2023-12-19T18:14:54.073455+0000","last_clean":"2023-12-19T18:14:54.073455+0000","last_became_active":"2023-12-19T18:14:53.546733+0000","last_became_peered":"2023-12-19T18:14:53.546733+0000","last_unstale":"2023-12-19T18:14:54.073455+0000","last_undegraded":"2023-12-19T18:14:54.073455+0000","last_fullsized":"2023-12-19T18:14:54.073455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:13:12.090625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069234+0000","last_change":"2023-12-19T18:14:54.069070+0000","last_active":"2023-12-19T18:14:54.069234+0000","last_peered":"2023-12-19T18:14:54.069234+0000","last_clean":"2023-12-19T18:14:54.069234+0000","last_became_active":"2023-12-19T18:14:54.066495+0000","last_became_peered":"2023-12-19T18:14:54.066495+0000","last_unstale":"2023-12-19T18:14:54.069234+0000","last_undegraded":"2023-12-19T18:14:54.069234+0000","last_fullsized":"2023-12-19T18:14:54.069234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068132+0000","last_change":"2023-12-19T18:14:53.566349+0000","last_active":"2023-12-19T18:14:54.068132+0000","last_peered":"2023-12-19T18:14:54.068132+0000","last_clean":"2023-12-19T18:14:54.068132+0000","last_became_active":"2023-12-19T18:14:53.565909+0000","last_became_peered":"2023-12-19T18:14:53.565909+0000","last_unstale":"2023-12-19T18:14:54.068132+0000","last_undegraded":"2023-12-19T18:14:54.068132+0000","last_fullsized":"2023-12-19T18:14:54.068132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073929+0000","last_change":"2023-12-19T18:14:54.072903+0000","last_active":"2023-12-19T18:14:54.073929+0000","last_peered":"2023-12-19T18:14:54.073929+0000","last_clean":"2023-12-19T18:14:54.073929+0000","last_became_active":"2023-12-19T18:14:54.072678+0000","last_became_peered":"2023-12-19T18:14:54.072678+0000","last_unstale":"2023-12-19T18:14:54.073929+0000","last_undegraded":"2023-12-19T18:14:54.073929+0000","last_fullsized":"2023-12-19T18:14:54.073929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.052955+0000","last_change":"2023-12-19T18:14:53.561604+0000","last_active":"2023-12-19T18:14:54.052955+0000","last_peered":"2023-12-19T18:14:54.052955+0000","last_clean":"2023-12-19T18:14:54.052955+0000","last_became_active":"2023-12-19T18:14:53.561372+0000","last_became_peered":"2023-12-19T18:14:53.561372+0000","last_unstale":"2023-12-19T18:14:54.052955+0000","last_undegraded":"2023-12-19T18:14:54.052955+0000","last_fullsized":"2023-12-19T18:14:54.052955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:00.255541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.057807+0000","last_change":"2023-12-19T18:14:54.057328+0000","last_active":"2023-12-19T18:14:54.057807+0000","last_peered":"2023-12-19T18:14:54.057807+0000","last_clean":"2023-12-19T18:14:54.057807+0000","last_became_active":"2023-12-19T18:14:54.055611+0000","last_became_peered":"2023-12-19T18:14:54.055611+0000","last_unstale":"2023-12-19T18:14:54.057807+0000","last_undegraded":"2023-12-19T18:14:54.057807+0000","last_fullsized":"2023-12-19T18:14:54.057807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:12.180767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.043960+0000","last_change":"2023-12-19T18:14:53.562764+0000","last_active":"2023-12-19T18:14:54.043960+0000","last_peered":"2023-12-19T18:14:54.043960+0000","last_clean":"2023-12-19T18:14:54.043960+0000","last_became_active":"2023-12-19T18:14:53.562532+0000","last_became_peered":"2023-12-19T18:14:53.562532+0000","last_unstale":"2023-12-19T18:14:54.043960+0000","last_undegraded":"2023-12-19T18:14:54.043960+0000","last_fullsized":"2023-12-19T18:14:54.043960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069588+0000","last_change":"2023-12-19T18:14:54.068780+0000","last_active":"2023-12-19T18:14:54.069588+0000","last_peered":"2023-12-19T18:14:54.069588+0000","last_clean":"2023-12-19T18:14:54.069588+0000","last_became_active":"2023-12-19T18:14:54.068556+0000","last_became_peered":"2023-12-19T18:14:54.068556+0000","last_unstale":"2023-12-19T18:14:54.069588+0000","last_undegraded":"2023-12-19T18:14:54.069588+0000","last_fullsized":"2023-12-19T18:14:54.069588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074393+0000","last_change":"2023-12-19T18:14:53.565084+0000","last_active":"2023-12-19T18:14:54.074393+0000","last_peered":"2023-12-19T18:14:54.074393+0000","last_clean":"2023-12-19T18:14:54.074393+0000","last_became_active":"2023-12-19T18:14:53.564836+0000","last_became_peered":"2023-12-19T18:14:53.564836+0000","last_unstale":"2023-12-19T18:14:54.074393+0000","last_undegraded":"2023-12-19T18:14:54.074393+0000","last_fullsized":"2023-12-19T18:14:54.074393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.076309+0000","last_change":"2023-12-19T18:14:54.075967+0000","last_active":"2023-12-19T18:14:54.076309+0000","last_peered":"2023-12-19T18:14:54.076309+0000","last_clean":"2023-12-19T18:14:54.076309+0000","last_became_active":"2023-12-19T18:14:54.074991+0000","last_became_peered":"2023-12-19T18:14:54.074991+0000","last_unstale":"2023-12-19T18:14:54.076309+0000","last_undegraded":"2023-12-19T18:14:54.076309+0000","last_fullsized":"2023-12-19T18:14:54.076309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061847+0000","last_change":"2023-12-19T18:14:53.550365+0000","last_active":"2023-12-19T18:14:54.061847+0000","last_peered":"2023-12-19T18:14:54.061847+0000","last_clean":"2023-12-19T18:14:54.061847+0000","last_became_active":"2023-12-19T18:14:53.550059+0000","last_became_peered":"2023-12-19T18:14:53.550059+0000","last_unstale":"2023-12-19T18:14:54.061847+0000","last_undegraded":"2023-12-19T18:14:54.061847+0000","last_fullsized":"2023-12-19T18:14:54.061847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050074+0000","last_change":"2023-12-19T18:14:54.048927+0000","last_active":"2023-12-19T18:14:54.050074+0000","last_peered":"2023-12-19T18:14:54.050074+0000","last_clean":"2023-12-19T18:14:54.050074+0000","last_became_active":"2023-12-19T18:14:54.048515+0000","last_became_peered":"2023-12-19T18:14:54.048515+0000","last_unstale":"2023-12-19T18:14:54.050074+0000","last_undegraded":"2023-12-19T18:14:54.050074+0000","last_fullsized":"2023-12-19T18:14:54.050074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064254+0000","last_change":"2023-12-19T18:14:53.567656+0000","last_active":"2023-12-19T18:14:54.064254+0000","last_peered":"2023-12-19T18:14:54.064254+0000","last_clean":"2023-12-19T18:14:54.064254+0000","last_became_active":"2023-12-19T18:14:53.567278+0000","last_became_peered":"2023-12-19T18:14:53.567278+0000","last_unstale":"2023-12-19T18:14:54.064254+0000","last_undegraded":"2023-12-19T18:14:54.064254+0000","last_fullsized":"2023-12-19T18:14:54.064254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065685+0000","last_change":"2023-12-19T18:14:54.065628+0000","last_active":"2023-12-19T18:14:54.065685+0000","last_peered":"2023-12-19T18:14:54.065685+0000","last_clean":"2023-12-19T18:14:54.065685+0000","last_became_active":"2023-12-19T18:14:54.065414+0000","last_became_peered":"2023-12-19T18:14:54.065414+0000","last_unstale":"2023-12-19T18:14:54.065685+0000","last_undegraded":"2023-12-19T18:14:54.065685+0000","last_fullsized":"2023-12-19T18:14:54.065685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065227+0000","last_change":"2023-12-19T18:14:53.559407+0000","last_active":"2023-12-19T18:14:54.065227+0000","last_peered":"2023-12-19T18:14:54.065227+0000","last_clean":"2023-12-19T18:14:54.065227+0000","last_became_active":"2023-12-19T18:14:53.559062+0000","last_became_peered":"2023-12-19T18:14:53.559062+0000","last_unstale":"2023-12-19T18:14:54.065227+0000","last_undegraded":"2023-12-19T18:14:54.065227+0000","last_fullsized":"2023-12-19T18:14:54.065227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061207+0000","last_change":"2023-12-19T18:14:54.061139+0000","last_active":"2023-12-19T18:14:54.061207+0000","last_peered":"2023-12-19T18:14:54.061207+0000","last_clean":"2023-12-19T18:14:54.061207+0000","last_became_active":"2023-12-19T18:14:54.060925+0000","last_became_peered":"2023-12-19T18:14:54.060925+0000","last_unstale":"2023-12-19T18:14:54.061207+0000","last_undegraded":"2023-12-19T18:14:54.061207+0000","last_fullsized":"2023-12-19T18:14:54.061207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067789+0000","last_change":"2023-12-19T18:14:53.563513+0000","last_active":"2023-12-19T18:14:54.067789+0000","last_peered":"2023-12-19T18:14:54.067789+0000","last_clean":"2023-12-19T18:14:54.067789+0000","last_became_active":"2023-12-19T18:14:53.562860+0000","last_became_peered":"2023-12-19T18:14:53.562860+0000","last_unstale":"2023-12-19T18:14:54.067789+0000","last_undegraded":"2023-12-19T18:14:54.067789+0000","last_fullsized":"2023-12-19T18:14:54.067789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:59:49.969450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072181+0000","last_change":"2023-12-19T18:14:54.071273+0000","last_active":"2023-12-19T18:14:54.072181+0000","last_peered":"2023-12-19T18:14:54.072181+0000","last_clean":"2023-12-19T18:14:54.072181+0000","last_became_active":"2023-12-19T18:14:54.070908+0000","last_became_peered":"2023-12-19T18:14:54.070908+0000","last_unstale":"2023-12-19T18:14:54.072181+0000","last_undegraded":"2023-12-19T18:14:54.072181+0000","last_fullsized":"2023-12-19T18:14:54.072181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:27:34.736465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.055663+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:14:54.055663+0000","last_peered":"2023-12-19T18:14:54.055663+0000","last_clean":"2023-12-19T18:14:54.055663+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:14:54.055663+0000","last_undegraded":"2023-12-19T18:14:54.055663+0000","last_fullsized":"2023-12-19T18:14:54.055663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053073+0000","last_change":"2023-12-19T18:14:53.557129+0000","last_active":"2023-12-19T18:14:54.053073+0000","last_peered":"2023-12-19T18:14:54.053073+0000","last_clean":"2023-12-19T18:14:54.053073+0000","last_became_active":"2023-12-19T18:14:53.556812+0000","last_became_peered":"2023-12-19T18:14:53.556812+0000","last_unstale":"2023-12-19T18:14:54.053073+0000","last_undegraded":"2023-12-19T18:14:54.053073+0000","last_fullsized":"2023-12-19T18:14:54.053073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:53.113355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051508+0000","last_change":"2023-12-19T18:14:54.051423+0000","last_active":"2023-12-19T18:14:54.051508+0000","last_peered":"2023-12-19T18:14:54.051508+0000","last_clean":"2023-12-19T18:14:54.051508+0000","last_became_active":"2023-12-19T18:14:54.050816+0000","last_became_peered":"2023-12-19T18:14:54.050816+0000","last_unstale":"2023-12-19T18:14:54.051508+0000","last_undegraded":"2023-12-19T18:14:54.051508+0000","last_fullsized":"2023-12-19T18:14:54.051508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068099+0000","last_change":"2023-12-19T18:14:53.558583+0000","last_active":"2023-12-19T18:14:54.068099+0000","last_peered":"2023-12-19T18:14:54.068099+0000","last_clean":"2023-12-19T18:14:54.068099+0000","last_became_active":"2023-12-19T18:14:53.558000+0000","last_became_peered":"2023-12-19T18:14:53.558000+0000","last_unstale":"2023-12-19T18:14:54.068099+0000","last_undegraded":"2023-12-19T18:14:54.068099+0000","last_fullsized":"2023-12-19T18:14:54.068099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:53.818952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068077+0000","last_change":"2023-12-19T18:14:54.067973+0000","last_active":"2023-12-19T18:14:54.068077+0000","last_peered":"2023-12-19T18:14:54.068077+0000","last_clean":"2023-12-19T18:14:54.068077+0000","last_became_active":"2023-12-19T18:14:54.065516+0000","last_became_peered":"2023-12-19T18:14:54.065516+0000","last_unstale":"2023-12-19T18:14:54.068077+0000","last_undegraded":"2023-12-19T18:14:54.068077+0000","last_fullsized":"2023-12-19T18:14:54.068077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.058632+0000","last_change":"2023-12-19T18:14:53.563881+0000","last_active":"2023-12-19T18:14:54.058632+0000","last_peered":"2023-12-19T18:14:54.058632+0000","last_clean":"2023-12-19T18:14:54.058632+0000","last_became_active":"2023-12-19T18:14:53.563564+0000","last_became_peered":"2023-12-19T18:14:53.563564+0000","last_unstale":"2023-12-19T18:14:54.058632+0000","last_undegraded":"2023-12-19T18:14:54.058632+0000","last_fullsized":"2023-12-19T18:14:54.058632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060261+0000","last_change":"2023-12-19T18:14:54.060208+0000","last_active":"2023-12-19T18:14:54.060261+0000","last_peered":"2023-12-19T18:14:54.060261+0000","last_clean":"2023-12-19T18:14:54.060261+0000","last_became_active":"2023-12-19T18:14:54.059235+0000","last_became_peered":"2023-12-19T18:14:54.059235+0000","last_unstale":"2023-12-19T18:14:54.060261+0000","last_undegraded":"2023-12-19T18:14:54.060261+0000","last_fullsized":"2023-12-19T18:14:54.060261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065436+0000","last_change":"2023-12-19T18:14:53.564065+0000","last_active":"2023-12-19T18:14:54.065436+0000","last_peered":"2023-12-19T18:14:54.065436+0000","last_clean":"2023-12-19T18:14:54.065436+0000","last_became_active":"2023-12-19T18:14:53.563841+0000","last_became_peered":"2023-12-19T18:14:53.563841+0000","last_unstale":"2023-12-19T18:14:54.065436+0000","last_undegraded":"2023-12-19T18:14:54.065436+0000","last_fullsized":"2023-12-19T18:14:54.065436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:46.161691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060843+0000","last_change":"2023-12-19T18:14:54.060734+0000","last_active":"2023-12-19T18:14:54.060843+0000","last_peered":"2023-12-19T18:14:54.060843+0000","last_clean":"2023-12-19T18:14:54.060843+0000","last_became_active":"2023-12-19T18:14:54.060328+0000","last_became_peered":"2023-12-19T18:14:54.060328+0000","last_unstale":"2023-12-19T18:14:54.060843+0000","last_undegraded":"2023-12-19T18:14:54.060843+0000","last_fullsized":"2023-12-19T18:14:54.060843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050368+0000","last_change":"2023-12-19T18:14:53.565164+0000","last_active":"2023-12-19T18:14:54.050368+0000","last_peered":"2023-12-19T18:14:54.050368+0000","last_clean":"2023-12-19T18:14:54.050368+0000","last_became_active":"2023-12-19T18:14:53.564538+0000","last_became_peered":"2023-12-19T18:14:53.564538+0000","last_unstale":"2023-12-19T18:14:54.050368+0000","last_undegraded":"2023-12-19T18:14:54.050368+0000","last_fullsized":"2023-12-19T18:14:54.050368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068274+0000","last_change":"2023-12-19T18:14:54.068080+0000","last_active":"2023-12-19T18:14:54.068274+0000","last_peered":"2023-12-19T18:14:54.068274+0000","last_clean":"2023-12-19T18:14:54.068274+0000","last_became_active":"2023-12-19T18:14:54.067528+0000","last_became_peered":"2023-12-19T18:14:54.067528+0000","last_unstale":"2023-12-19T18:14:54.068274+0000","last_undegraded":"2023-12-19T18:14:54.068274+0000","last_fullsized":"2023-12-19T18:14:54.068274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067999+0000","last_change":"2023-12-19T18:14:53.562234+0000","last_active":"2023-12-19T18:14:54.067999+0000","last_peered":"2023-12-19T18:14:54.067999+0000","last_clean":"2023-12-19T18:14:54.067999+0000","last_became_active":"2023-12-19T18:14:53.561326+0000","last_became_peered":"2023-12-19T18:14:53.561326+0000","last_unstale":"2023-12-19T18:14:54.067999+0000","last_undegraded":"2023-12-19T18:14:54.067999+0000","last_fullsized":"2023-12-19T18:14:54.067999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074010+0000","last_change":"2023-12-19T18:14:54.073543+0000","last_active":"2023-12-19T18:14:54.074010+0000","last_peered":"2023-12-19T18:14:54.074010+0000","last_clean":"2023-12-19T18:14:54.074010+0000","last_became_active":"2023-12-19T18:14:54.073247+0000","last_became_peered":"2023-12-19T18:14:54.073247+0000","last_unstale":"2023-12-19T18:14:54.074010+0000","last_undegraded":"2023-12-19T18:14:54.074010+0000","last_fullsized":"2023-12-19T18:14:54.074010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.045518+0000","last_change":"2023-12-19T18:14:53.551251+0000","last_active":"2023-12-19T18:14:54.045518+0000","last_peered":"2023-12-19T18:14:54.045518+0000","last_clean":"2023-12-19T18:14:54.045518+0000","last_became_active":"2023-12-19T18:14:53.550691+0000","last_became_peered":"2023-12-19T18:14:53.550691+0000","last_unstale":"2023-12-19T18:14:54.045518+0000","last_undegraded":"2023-12-19T18:14:54.045518+0000","last_fullsized":"2023-12-19T18:14:54.045518+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.071102+0000","last_change":"2023-12-19T18:14:54.070766+0000","last_active":"2023-12-19T18:14:54.071102+0000","last_peered":"2023-12-19T18:14:54.071102+0000","last_clean":"2023-12-19T18:14:54.071102+0000","last_became_active":"2023-12-19T18:14:54.069495+0000","last_became_peered":"2023-12-19T18:14:54.069495+0000","last_unstale":"2023-12-19T18:14:54.071102+0000","last_undegraded":"2023-12-19T18:14:54.071102+0000","last_fullsized":"2023-12-19T18:14:54.071102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.076793+0000","last_change":"2023-12-19T18:14:53.563561+0000","last_active":"2023-12-19T18:14:54.076793+0000","last_peered":"2023-12-19T18:14:54.076793+0000","last_clean":"2023-12-19T18:14:54.076793+0000","last_became_active":"2023-12-19T18:14:53.563126+0000","last_became_peered":"2023-12-19T18:14:53.563126+0000","last_unstale":"2023-12-19T18:14:54.076793+0000","last_undegraded":"2023-12-19T18:14:54.076793+0000","last_fullsized":"2023-12-19T18:14:54.076793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.059619+0000","last_change":"2023-12-19T18:14:54.059362+0000","last_active":"2023-12-19T18:14:54.059619+0000","last_peered":"2023-12-19T18:14:54.059619+0000","last_clean":"2023-12-19T18:14:54.059619+0000","last_became_active":"2023-12-19T18:14:54.059089+0000","last_became_peered":"2023-12-19T18:14:54.059089+0000","last_unstale":"2023-12-19T18:14:54.059619+0000","last_undegraded":"2023-12-19T18:14:54.059619+0000","last_fullsized":"2023-12-19T18:14:54.059619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.045320+0000","last_change":"2023-12-19T18:14:53.551856+0000","last_active":"2023-12-19T18:14:54.045320+0000","last_peered":"2023-12-19T18:14:54.045320+0000","last_clean":"2023-12-19T18:14:54.045320+0000","last_became_active":"2023-12-19T18:14:53.551105+0000","last_became_peered":"2023-12-19T18:14:53.551105+0000","last_unstale":"2023-12-19T18:14:54.045320+0000","last_undegraded":"2023-12-19T18:14:54.045320+0000","last_fullsized":"2023-12-19T18:14:54.045320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065908+0000","last_change":"2023-12-19T18:14:54.065539+0000","last_active":"2023-12-19T18:14:54.065908+0000","last_peered":"2023-12-19T18:14:54.065908+0000","last_clean":"2023-12-19T18:14:54.065908+0000","last_became_active":"2023-12-19T18:14:54.056448+0000","last_became_peered":"2023-12-19T18:14:54.056448+0000","last_unstale":"2023-12-19T18:14:54.065908+0000","last_undegraded":"2023-12-19T18:14:54.065908+0000","last_fullsized":"2023-12-19T18:14:54.065908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.045692+0000","last_change":"2023-12-19T18:14:53.554185+0000","last_active":"2023-12-19T18:14:54.045692+0000","last_peered":"2023-12-19T18:14:54.045692+0000","last_clean":"2023-12-19T18:14:54.045692+0000","last_became_active":"2023-12-19T18:14:53.553750+0000","last_became_peered":"2023-12-19T18:14:53.553750+0000","last_unstale":"2023-12-19T18:14:54.045692+0000","last_undegraded":"2023-12-19T18:14:54.045692+0000","last_fullsized":"2023-12-19T18:14:54.045692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.059756+0000","last_change":"2023-12-19T18:14:54.059501+0000","last_active":"2023-12-19T18:14:54.059756+0000","last_peered":"2023-12-19T18:14:54.059756+0000","last_clean":"2023-12-19T18:14:54.059756+0000","last_became_active":"2023-12-19T18:14:54.059055+0000","last_became_peered":"2023-12-19T18:14:54.059055+0000","last_unstale":"2023-12-19T18:14:54.059756+0000","last_undegraded":"2023-12-19T18:14:54.059756+0000","last_fullsized":"2023-12-19T18:14:54.059756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066496+0000","last_change":"2023-12-19T18:14:53.566640+0000","last_active":"2023-12-19T18:14:54.066496+0000","last_peered":"2023-12-19T18:14:54.066496+0000","last_clean":"2023-12-19T18:14:54.066496+0000","last_became_active":"2023-12-19T18:14:53.566413+0000","last_became_peered":"2023-12-19T18:14:53.566413+0000","last_unstale":"2023-12-19T18:14:54.066496+0000","last_undegraded":"2023-12-19T18:14:54.066496+0000","last_fullsized":"2023-12-19T18:14:54.066496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074538+0000","last_change":"2023-12-19T18:14:54.073374+0000","last_active":"2023-12-19T18:14:54.074538+0000","last_peered":"2023-12-19T18:14:54.074538+0000","last_clean":"2023-12-19T18:14:54.074538+0000","last_became_active":"2023-12-19T18:14:54.072676+0000","last_became_peered":"2023-12-19T18:14:54.072676+0000","last_unstale":"2023-12-19T18:14:54.074538+0000","last_undegraded":"2023-12-19T18:14:54.074538+0000","last_fullsized":"2023-12-19T18:14:54.074538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073825+0000","last_change":"2023-12-19T18:14:53.549245+0000","last_active":"2023-12-19T18:14:54.073825+0000","last_peered":"2023-12-19T18:14:54.073825+0000","last_clean":"2023-12-19T18:14:54.073825+0000","last_became_active":"2023-12-19T18:14:53.548992+0000","last_became_peered":"2023-12-19T18:14:53.548992+0000","last_unstale":"2023-12-19T18:14:54.073825+0000","last_undegraded":"2023-12-19T18:14:54.073825+0000","last_fullsized":"2023-12-19T18:14:54.073825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074558+0000","last_change":"2023-12-19T18:14:54.073488+0000","last_active":"2023-12-19T18:14:54.074558+0000","last_peered":"2023-12-19T18:14:54.074558+0000","last_clean":"2023-12-19T18:14:54.074558+0000","last_became_active":"2023-12-19T18:14:54.072567+0000","last_became_peered":"2023-12-19T18:14:54.072567+0000","last_unstale":"2023-12-19T18:14:54.074558+0000","last_undegraded":"2023-12-19T18:14:54.074558+0000","last_fullsized":"2023-12-19T18:14:54.074558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:22.428929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.057975+0000","last_change":"2023-12-19T18:14:54.057549+0000","last_active":"2023-12-19T18:14:54.057975+0000","last_peered":"2023-12-19T18:14:54.057975+0000","last_clean":"2023-12-19T18:14:54.057975+0000","last_became_active":"2023-12-19T18:14:54.056075+0000","last_became_peered":"2023-12-19T18:14:54.056075+0000","last_unstale":"2023-12-19T18:14:54.057975+0000","last_undegraded":"2023-12-19T18:14:54.057975+0000","last_fullsized":"2023-12-19T18:14:54.057975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:47.825262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064249+0000","last_change":"2023-12-19T18:14:53.555372+0000","last_active":"2023-12-19T18:14:54.064249+0000","last_peered":"2023-12-19T18:14:54.064249+0000","last_clean":"2023-12-19T18:14:54.064249+0000","last_became_active":"2023-12-19T18:14:53.555089+0000","last_became_peered":"2023-12-19T18:14:53.555089+0000","last_unstale":"2023-12-19T18:14:54.064249+0000","last_undegraded":"2023-12-19T18:14:54.064249+0000","last_fullsized":"2023-12-19T18:14:54.064249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065405+0000","last_change":"2023-12-19T18:14:54.065119+0000","last_active":"2023-12-19T18:14:54.065405+0000","last_peered":"2023-12-19T18:14:54.065405+0000","last_clean":"2023-12-19T18:14:54.065405+0000","last_became_active":"2023-12-19T18:14:54.064697+0000","last_became_peered":"2023-12-19T18:14:54.064697+0000","last_unstale":"2023-12-19T18:14:54.065405+0000","last_undegraded":"2023-12-19T18:14:54.065405+0000","last_fullsized":"2023-12-19T18:14:54.065405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063594+0000","last_change":"2023-12-19T18:14:53.556488+0000","last_active":"2023-12-19T18:14:54.063594+0000","last_peered":"2023-12-19T18:14:54.063594+0000","last_clean":"2023-12-19T18:14:54.063594+0000","last_became_active":"2023-12-19T18:14:53.555952+0000","last_became_peered":"2023-12-19T18:14:53.555952+0000","last_unstale":"2023-12-19T18:14:54.063594+0000","last_undegraded":"2023-12-19T18:14:54.063594+0000","last_fullsized":"2023-12-19T18:14:54.063594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070434+0000","last_change":"2023-12-19T18:14:54.070014+0000","last_active":"2023-12-19T18:14:54.070434+0000","last_peered":"2023-12-19T18:14:54.070434+0000","last_clean":"2023-12-19T18:14:54.070434+0000","last_became_active":"2023-12-19T18:14:54.067648+0000","last_became_peered":"2023-12-19T18:14:54.067648+0000","last_unstale":"2023-12-19T18:14:54.070434+0000","last_undegraded":"2023-12-19T18:14:54.070434+0000","last_fullsized":"2023-12-19T18:14:54.070434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067116+0000","last_change":"2023-12-19T18:14:53.566090+0000","last_active":"2023-12-19T18:14:54.067116+0000","last_peered":"2023-12-19T18:14:54.067116+0000","last_clean":"2023-12-19T18:14:54.067116+0000","last_became_active":"2023-12-19T18:14:53.565782+0000","last_became_peered":"2023-12-19T18:14:53.565782+0000","last_unstale":"2023-12-19T18:14:54.067116+0000","last_undegraded":"2023-12-19T18:14:54.067116+0000","last_fullsized":"2023-12-19T18:14:54.067116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063312+0000","last_change":"2023-12-19T18:14:54.060686+0000","last_active":"2023-12-19T18:14:54.063312+0000","last_peered":"2023-12-19T18:14:54.063312+0000","last_clean":"2023-12-19T18:14:54.063312+0000","last_became_active":"2023-12-19T18:14:54.059977+0000","last_became_peered":"2023-12-19T18:14:54.059977+0000","last_unstale":"2023-12-19T18:14:54.063312+0000","last_undegraded":"2023-12-19T18:14:54.063312+0000","last_fullsized":"2023-12-19T18:14:54.063312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073092+0000","last_change":"2023-12-19T18:14:53.558205+0000","last_active":"2023-12-19T18:14:54.073092+0000","last_peered":"2023-12-19T18:14:54.073092+0000","last_clean":"2023-12-19T18:14:54.073092+0000","last_became_active":"2023-12-19T18:14:53.557733+0000","last_became_peered":"2023-12-19T18:14:53.557733+0000","last_unstale":"2023-12-19T18:14:54.073092+0000","last_undegraded":"2023-12-19T18:14:54.073092+0000","last_fullsized":"2023-12-19T18:14:54.073092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929138,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3220000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0589999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94099999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2110000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3280000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.105}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.341}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0740000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2849999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0129999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":71,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927026,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0669999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93899999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2130000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4590000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.488}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199928680,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.621}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4379999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927698,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1859999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2490000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3340000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.54}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4930000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.613}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338824,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.123}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0409999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3819999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1719999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929206,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64300000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1579999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]}],"pool_statfs":[]}} 2023-12-19T18:15:04.708 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:15:04.709 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-19T18:15:04.709 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-12-19T18:15:04.709 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-19T18:15:04.709 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-12-19T18:15:04.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.983+0000 7f13f49f9640 1 Processor -- start 2023-12-19T18:15:04.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.983+0000 7f13f49f9640 1 -- start start 2023-12-19T18:15:04.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.985+0000 7f13f49f9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f13f00d2350 0x7f13f00b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.985+0000 7f13f49f9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f00b5e90 con 0x7f13f00b4ef0 2023-12-19T18:15:04.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.985+0000 7f13f49f9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f00d2850 con 0x7f13f01604d0 2023-12-19T18:15:04.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.985+0000 7f13f49f9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f00d2b30 con 0x7f13f00d2350 2023-12-19T18:15:04.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.985+0000 7f13eeffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f13f00b4ef0 0x7f13f00b6300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53938/0 (socket says 172.21.15.27:53938) 2023-12-19T18:15:04.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.985+0000 7f13eeffd640 1 -- 172.21.15.27:0/704237257 learned_addr learned my addr 172.21.15.27:0/704237257 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:04.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.985+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085621627 0 0) 0x7f13f00b5e90 con 0x7f13f00b4ef0 2023-12-19T18:15:04.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f13d8003520 con 0x7f13f00b4ef0 2023-12-19T18:15:04.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13effff640 1 --2- 172.21.15.27:0/704237257 >> v2:172.21.15.192:3300/0 conn(0x7f13f00d2350 0x7f13f00b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1887231843 0 0) 0x7f13d8003520 con 0x7f13f00b4ef0 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 >> v2:172.21.15.192:3300/0 conn(0x7f13f00d2350 msgr2=0x7f13f00b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 --2- 172.21.15.27:0/704237257 >> v2:172.21.15.192:3300/0 conn(0x7f13f00d2350 0x7f13f00b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 >> v1:172.21.15.192:6789/0 conn(0x7f13f01604d0 legacy=0x7f13f00b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f00cded0 con 0x7f13f00b4ef0 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f13e4001670 con 0x7f13f00b4ef0 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 >> v1:172.21.15.27:6789/0 conn(0x7f13f00b4ef0 legacy=0x7f13f00b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 --2- 172.21.15.27:0/704237257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d80039c0 0x7f13d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 --2- 172.21.15.27:0/704237257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13d80047c0 0x7f13d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 --2- 172.21.15.27:0/704237257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13d80051d0 0x7f13d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f00d2b30 con 0x7f13d80051d0 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13d800c120 con 0x7f13d80039c0 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13d800c2e0 con 0x7f13d80047c0 2023-12-19T18:15:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.986+0000 7f13effff640 1 --2- 172.21.15.27:0/704237257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13d80051d0 0x7f13d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.987+0000 7f13ef7fe640 1 --2- 172.21.15.27:0/704237257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d80039c0 0x7f13d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.987+0000 7f13eeffd640 1 --2- 172.21.15.27:0/704237257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13d80047c0 0x7f13d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.987+0000 7f13effff640 1 -- 172.21.15.27:0/704237257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13d80047c0 msgr2=0x7f13d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:04.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.987+0000 7f13effff640 1 --2- 172.21.15.27:0/704237257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13d80047c0 0x7f13d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.987+0000 7f13effff640 1 -- 172.21.15.27:0/704237257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d80039c0 msgr2=0x7f13d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:04.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.987+0000 7f13effff640 1 --2- 172.21.15.27:0/704237257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13d80039c0 0x7f13d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.987+0000 7f13effff640 1 -- 172.21.15.27:0/704237257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d8003520 con 0x7f13d80051d0 2023-12-19T18:15:04.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.987+0000 7f13eeffd640 1 --2- 172.21.15.27:0/704237257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13d80047c0 0x7f13d8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:04.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.988+0000 7f13effff640 1 -- 172.21.15.27:0/704237257 reap_dead start 2023-12-19T18:15:04.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.988+0000 7f13effff640 1 --2- 172.21.15.27:0/704237257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13d80051d0 0x7f13d800b980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f13e800ba30 tx=0x7f13d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eaa566c07767f7f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:04.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.988+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8004020 con 0x7f13d80051d0 2023-12-19T18:15:04.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.988+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e80041c0 con 0x7f13d80051d0 2023-12-19T18:15:04.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.988+0000 7f13ee7fc640 1 -- 172.21.15.27:0/704237257 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8003610 con 0x7f13d80051d0 2023-12-19T18:15:04.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.989+0000 7f13f49f9640 1 -- 172.21.15.27:0/704237257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13d80051d0 msgr2=0x7f13d800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:04.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.989+0000 7f13f49f9640 1 --2- 172.21.15.27:0/704237257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13d80051d0 0x7f13d800b980 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f13e800ba30 tx=0x7f13d80057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.989+0000 7f13f49f9640 1 -- 172.21.15.27:0/704237257 shutdown_connections 2023-12-19T18:15:04.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.989+0000 7f13f49f9640 1 --2- 172.21.15.27:0/704237257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13d80051d0 0x7f13d800b980 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.989+0000 7f13f49f9640 1 -- 172.21.15.27:0/704237257 >> 172.21.15.27:0/704237257 conn(0x7f13f00cf7e0 msgr2=0x7f13f00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:04.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.989+0000 7f13f49f9640 1 -- 172.21.15.27:0/704237257 shutdown_connections 2023-12-19T18:15:04.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.989+0000 7f13f49f9640 1 -- 172.21.15.27:0/704237257 wait complete. 2023-12-19T18:15:04.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.991+0000 7f13f49f9640 1 Processor -- start 2023-12-19T18:15:04.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.991+0000 7f13f49f9640 1 -- start start 2023-12-19T18:15:04.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.992+0000 7f13f49f9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13d80051d0 0x7f13f01482e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.992+0000 7f13f49f9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13f0157b10 0x7f13f00b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.992+0000 7f13f49f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13f01f4f00 0x7f13f01f5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.992+0000 7f13f49f9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f0158ce0 con 0x7f13f01f4f00 2023-12-19T18:15:04.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.992+0000 7f13f49f9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f0148820 con 0x7f13f0157b10 2023-12-19T18:15:04.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.992+0000 7f13f49f9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f0158060 con 0x7f13d80051d0 2023-12-19T18:15:04.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.992+0000 7f13effff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13f01f4f00 0x7f13f01f5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.992+0000 7f13effff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13f01f4f00 0x7f13f01f5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49458/0 (socket says 172.21.15.27:49458) 2023-12-19T18:15:04.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.992+0000 7f13effff640 1 -- 172.21.15.27:0/3697163896 learned_addr learned my addr 172.21.15.27:0/3697163896 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:04.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.993+0000 7f13eeffd640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13f0157b10 0x7f13f00b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.993+0000 7f13ef7fe640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13d80051d0 0x7f13f01482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.993+0000 7f13effff640 1 -- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13d80051d0 msgr2=0x7f13f01482e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:04.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.993+0000 7f13effff640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13d80051d0 0x7f13f01482e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.993+0000 7f13effff640 1 -- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13f0157b10 msgr2=0x7f13f00b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:04.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.993+0000 7f13effff640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13f0157b10 0x7f13f00b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:04.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.993+0000 7f13effff640 1 -- 172.21.15.27:0/3697163896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f0158340 con 0x7f13f01f4f00 2023-12-19T18:15:04.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.993+0000 7f13ef7fe640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13d80051d0 0x7f13f01482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:04.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.993+0000 7f13effff640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13f01f4f00 0x7f13f01f5340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f13e802bfd0 tx=0x7f13e8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:04.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.993+0000 7f13d7fff640 1 -- 172.21.15.27:0/3697163896 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e8004330 con 0x7f13f01f4f00 2023-12-19T18:15:04.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.993+0000 7f13d7fff640 1 -- 172.21.15.27:0/3697163896 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e8040070 con 0x7f13f01f4f00 2023-12-19T18:15:04.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.993+0000 7f13f49f9640 1 -- 172.21.15.27:0/3697163896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f0158620 con 0x7f13f01f4f00 2023-12-19T18:15:04.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.993+0000 7f13d7fff640 1 -- 172.21.15.27:0/3697163896 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e80326c0 con 0x7f13f01f4f00 2023-12-19T18:15:04.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.994+0000 7f13f49f9640 1 -- 172.21.15.27:0/3697163896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f01562b0 con 0x7f13f01f4f00 2023-12-19T18:15:04.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.995+0000 7f13f49f9640 1 -- 172.21.15.27:0/3697163896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f0158e20 con 0x7f13f01f4f00 2023-12-19T18:15:04.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.996+0000 7f13d7fff640 1 -- 172.21.15.27:0/3697163896 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f13e8042030 con 0x7f13f01f4f00 2023-12-19T18:15:04.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.997+0000 7f13d7fff640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f13dc06a430 0x7f13dc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:04.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.997+0000 7f13d7fff640 1 -- 172.21.15.27:0/3697163896 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f13e80aae30 con 0x7f13f01f4f00 2023-12-19T18:15:04.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.998+0000 7f13ef7fe640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f13dc06a430 0x7f13dc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:04.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:04.999+0000 7f13ef7fe640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f13dc06a430 0x7f13dc06c8f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f13e0001cc0 tx=0x7f13e0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:05.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.002+0000 7f13d7fff640 1 -- 172.21.15.27:0/3697163896 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f13f0158e20 con 0x7f13f01f4f00 2023-12-19T18:15:05.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.245+0000 7f13f49f9640 1 -- 172.21.15.27:0/3697163896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f13f0158e20 con 0x7f13f01f4f00 2023-12-19T18:15:05.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.254+0000 7f13d7fff640 1 -- 172.21.15.27:0/3697163896 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+169968 (secure 0 0 0) 0x7f13f0158e20 con 0x7f13f01f4f00 2023-12-19T18:15:05.256 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:05.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.258+0000 7f13f49f9640 1 -- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f13dc06a430 msgr2=0x7f13dc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:05.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.258+0000 7f13f49f9640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f13dc06a430 0x7f13dc06c8f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f13e0001cc0 tx=0x7f13e0009040 comp rx=0 tx=0).stop 2023-12-19T18:15:05.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.258+0000 7f13f49f9640 1 -- 172.21.15.27:0/3697163896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13f01f4f00 msgr2=0x7f13f01f5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:05.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.258+0000 7f13f49f9640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13f01f4f00 0x7f13f01f5340 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f13e802bfd0 tx=0x7f13e8004730 comp rx=0 tx=0).stop 2023-12-19T18:15:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.259+0000 7f13f49f9640 1 -- 172.21.15.27:0/3697163896 shutdown_connections 2023-12-19T18:15:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.259+0000 7f13f49f9640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f13dc06a430 0x7f13dc06c8f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.259+0000 7f13f49f9640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13f01f4f00 0x7f13f01f5340 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.259+0000 7f13f49f9640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f13f0157b10 0x7f13f00b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.259+0000 7f13f49f9640 1 --2- 172.21.15.27:0/3697163896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f13d80051d0 0x7f13f01482e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.259+0000 7f13f49f9640 1 -- 172.21.15.27:0/3697163896 >> 172.21.15.27:0/3697163896 conn(0x7f13f00cf7e0 msgr2=0x7f13f00b42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.259+0000 7f13f49f9640 1 -- 172.21.15.27:0/3697163896 shutdown_connections 2023-12-19T18:15:05.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.259+0000 7f13f49f9640 1 -- 172.21.15.27:0/3697163896 wait complete. 2023-12-19T18:15:05.278 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":5,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6832","nonce":83634},{"type":"v1","addr":"172.21.15.192:6833","nonce":83634}]},"active_addr":"172.21.15.192:6833/83634","active_change":"2023-12-19T18:14:41.672944+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4103,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":1296427852}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":206582195}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":2442717630}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.192:0","nonce":142960183}]}]} 2023-12-19T18:15:05.282 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-12-19T18:15:05.283 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-12-19T18:15:05.283 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:05.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.516+0000 7f8b651d4640 1 Processor -- start 2023-12-19T18:15:05.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.516+0000 7f8b651d4640 1 -- start start 2023-12-19T18:15:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.517+0000 7f8b651d4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8b60154030 0x7f8b600cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.517+0000 7f8b651d4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b600c5560 0x7f8b600afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.517+0000 7f8b651d4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b600cc190 con 0x7f8b600af630 2023-12-19T18:15:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.517+0000 7f8b651d4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b600b0000 con 0x7f8b600c5560 2023-12-19T18:15:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.517+0000 7f8b651d4640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b600cb2a0 con 0x7f8b60154030 2023-12-19T18:15:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.517+0000 7f8b649d3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b600c5560 0x7f8b600afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.518+0000 7f8b649d3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b600c5560 0x7f8b600afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49462/0 (socket says 172.21.15.27:49462) 2023-12-19T18:15:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.518+0000 7f8b649d3640 1 -- 172.21.15.27:0/1595845946 learned_addr learned my addr 172.21.15.27:0/1595845946 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.518+0000 7f8b5ffff640 1 --2- 172.21.15.27:0/1595845946 >> v2:172.21.15.192:3300/0 conn(0x7f8b60154030 0x7f8b600cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:05.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.518+0000 7f8b649d3640 1 -- 172.21.15.27:0/1595845946 >> v2:172.21.15.192:3300/0 conn(0x7f8b60154030 msgr2=0x7f8b600cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:05.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.518+0000 7f8b649d3640 1 --2- 172.21.15.27:0/1595845946 >> v2:172.21.15.192:3300/0 conn(0x7f8b60154030 0x7f8b600cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.518+0000 7f8b649d3640 1 -- 172.21.15.27:0/1595845946 >> v1:172.21.15.192:6789/0 conn(0x7f8b600af630 legacy=0x7f8b6015d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:05.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.518+0000 7f8b649d3640 1 -- 172.21.15.27:0/1595845946 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b600cb580 con 0x7f8b600c5560 2023-12-19T18:15:05.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.519+0000 7f8b649d3640 1 --2- 172.21.15.27:0/1595845946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b600c5560 0x7f8b600afac0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009870 tx=0x7f8b5802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fdfd7f8c24cfa5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:05.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.519+0000 7f8b5effd640 1 -- 172.21.15.27:0/1595845946 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b58031870 con 0x7f8b600c5560 2023-12-19T18:15:05.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.520+0000 7f8b5effd640 1 -- 172.21.15.27:0/1595845946 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b580319d0 con 0x7f8b600c5560 2023-12-19T18:15:05.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.520+0000 7f8b5effd640 1 -- 172.21.15.27:0/1595845946 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b58031d60 con 0x7f8b600c5560 2023-12-19T18:15:05.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.520+0000 7f8b651d4640 1 -- 172.21.15.27:0/1595845946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b600c5560 msgr2=0x7f8b600afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.520+0000 7f8b651d4640 1 --2- 172.21.15.27:0/1595845946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b600c5560 0x7f8b600afac0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009870 tx=0x7f8b5802b910 comp rx=0 tx=0).stop 2023-12-19T18:15:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.520+0000 7f8b651d4640 1 -- 172.21.15.27:0/1595845946 shutdown_connections 2023-12-19T18:15:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.520+0000 7f8b651d4640 1 --2- 172.21.15.27:0/1595845946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b600c5560 0x7f8b600afac0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.520+0000 7f8b651d4640 1 --2- 172.21.15.27:0/1595845946 >> v2:172.21.15.192:3300/0 conn(0x7f8b60154030 0x7f8b600cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.520+0000 7f8b651d4640 1 -- 172.21.15.27:0/1595845946 >> 172.21.15.27:0/1595845946 conn(0x7f8b60069c70 msgr2=0x7f8b6006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.520+0000 7f8b651d4640 1 -- 172.21.15.27:0/1595845946 shutdown_connections 2023-12-19T18:15:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.520+0000 7f8b651d4640 1 -- 172.21.15.27:0/1595845946 wait complete. 2023-12-19T18:15:05.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.522+0000 7f8b651d4640 1 Processor -- start 2023-12-19T18:15:05.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.522+0000 7f8b651d4640 1 -- start start 2023-12-19T18:15:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.523+0000 7f8b651d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b600af630 0x7f8b600b2d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.523+0000 7f8b651d4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b600c5560 0x7f8b600c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.523+0000 7f8b651d4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60154030 0x7f8b600c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.523+0000 7f8b651d4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600c6870 con 0x7f8b60154030 2023-12-19T18:15:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.523+0000 7f8b651d4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600c7770 con 0x7f8b600af630 2023-12-19T18:15:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.523+0000 7f8b651d4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6006b3f0 con 0x7f8b600c5560 2023-12-19T18:15:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.523+0000 7f8b649d3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60154030 0x7f8b600c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.524+0000 7f8b649d3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60154030 0x7f8b600c5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49464/0 (socket says 172.21.15.27:49464) 2023-12-19T18:15:05.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.524+0000 7f8b649d3640 1 -- 172.21.15.27:0/832210201 learned_addr learned my addr 172.21.15.27:0/832210201 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:05.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.524+0000 7f8b5ffff640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b600af630 0x7f8b600b2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:05.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.524+0000 7f8b5f7fe640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b600c5560 0x7f8b600c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:05.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.524+0000 7f8b649d3640 1 -- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b600c5560 msgr2=0x7f8b600c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:05.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.524+0000 7f8b649d3640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b600c5560 0x7f8b600c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.524+0000 7f8b649d3640 1 -- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b600af630 msgr2=0x7f8b600b2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:05.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.524+0000 7f8b649d3640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b600af630 0x7f8b600b2d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.524+0000 7f8b649d3640 1 -- 172.21.15.27:0/832210201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b580094d0 con 0x7f8b60154030 2023-12-19T18:15:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.524+0000 7f8b649d3640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60154030 0x7f8b600c5f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b5802b950 tx=0x7f8b580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.525+0000 7f8b5cff9640 1 -- 172.21.15.27:0/832210201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5803ac60 con 0x7f8b60154030 2023-12-19T18:15:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.525+0000 7f8b5cff9640 1 -- 172.21.15.27:0/832210201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5803adc0 con 0x7f8b60154030 2023-12-19T18:15:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.525+0000 7f8b5cff9640 1 -- 172.21.15.27:0/832210201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b58044630 con 0x7f8b60154030 2023-12-19T18:15:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.525+0000 7f8b651d4640 1 -- 172.21.15.27:0/832210201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b600b35d0 con 0x7f8b60154030 2023-12-19T18:15:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.525+0000 7f8b651d4640 1 -- 172.21.15.27:0/832210201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6006b640 con 0x7f8b60154030 2023-12-19T18:15:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.526+0000 7f8b651d4640 1 -- 172.21.15.27:0/832210201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6004e840 con 0x7f8b60154030 2023-12-19T18:15:05.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.527+0000 7f8b5cff9640 1 -- 172.21.15.27:0/832210201 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8b5804d450 con 0x7f8b60154030 2023-12-19T18:15:05.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.528+0000 7f8b5cff9640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b4006a430 0x7f8b4006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:05.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.528+0000 7f8b5cff9640 1 -- 172.21.15.27:0/832210201 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b5802d040 con 0x7f8b60154030 2023-12-19T18:15:05.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.528+0000 7f8b5ffff640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b4006a430 0x7f8b4006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:05.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.529+0000 7f8b5ffff640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b4006a430 0x7f8b4006c8f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8b48006fd0 tx=0x7f8b48008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:05.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.533+0000 7f8b5cff9640 1 -- 172.21.15.27:0/832210201 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8b58076560 con 0x7f8b60154030 2023-12-19T18:15:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.731+0000 7f8b651d4640 1 -- 172.21.15.27:0/832210201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b600c3c50 con 0x7f8b60154030 2023-12-19T18:15:05.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.734+0000 7f8b5cff9640 1 -- 172.21.15.27:0/832210201 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f8b58035020 con 0x7f8b60154030 2023-12-19T18:15:05.734 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:05.734 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:55.039690+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:05.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.736+0000 7f8b651d4640 1 -- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b4006a430 msgr2=0x7f8b4006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:05.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.736+0000 7f8b651d4640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b4006a430 0x7f8b4006c8f0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8b48006fd0 tx=0x7f8b48008040 comp rx=0 tx=0).stop 2023-12-19T18:15:05.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.736+0000 7f8b651d4640 1 -- 172.21.15.27:0/832210201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60154030 msgr2=0x7f8b600c5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:05.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.736+0000 7f8b651d4640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60154030 0x7f8b600c5f50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8b5802b950 tx=0x7f8b580046c0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.737+0000 7f8b651d4640 1 -- 172.21.15.27:0/832210201 shutdown_connections 2023-12-19T18:15:05.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.737+0000 7f8b651d4640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b4006a430 0x7f8b4006c8f0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.737+0000 7f8b651d4640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60154030 0x7f8b600c5f50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.737+0000 7f8b651d4640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b600c5560 0x7f8b600c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.737+0000 7f8b651d4640 1 --2- 172.21.15.27:0/832210201 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b600af630 0x7f8b600b2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:05.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.737+0000 7f8b651d4640 1 -- 172.21.15.27:0/832210201 >> 172.21.15.27:0/832210201 conn(0x7f8b60069c70 msgr2=0x7f8b6015a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:05.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.737+0000 7f8b651d4640 1 -- 172.21.15.27:0/832210201 shutdown_connections 2023-12-19T18:15:05.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:05.737+0000 7f8b651d4640 1 -- 172.21.15.27:0/832210201 wait complete. 2023-12-19T18:15:05.758 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-12-19T18:15:05.758 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:06.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.000+0000 7f5d0ffff640 1 Processor -- start 2023-12-19T18:15:06.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.000+0000 7f5d0ffff640 1 -- start start 2023-12-19T18:15:06.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.002+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d101553a0 0x7f5d100c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.002+0000 7f5d0ffff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d100b0330 con 0x7f5d100afeb0 2023-12-19T18:15:06.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.002+0000 7f5d0ffff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d100cb350 con 0x7f5d1015ee30 2023-12-19T18:15:06.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.002+0000 7f5d0ffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d100cbcf0 con 0x7f5d101553a0 2023-12-19T18:15:06.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.002+0000 7f5d0e7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5d100afeb0 0x7f5d1015e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53940/0 (socket says 172.21.15.27:53940) 2023-12-19T18:15:06.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.002+0000 7f5d0e7fc640 1 -- 172.21.15.27:0/3037796945 learned_addr learned my addr 172.21.15.27:0/3037796945 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:06.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0effd640 1 --2- 172.21.15.27:0/3037796945 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d101553a0 0x7f5d100c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675035212 0 0) 0x7f5d100b0330 con 0x7f5d100afeb0 2023-12-19T18:15:06.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5cf8003520 con 0x7f5d100afeb0 2023-12-19T18:15:06.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 866041167 0 0) 0x7f5cf8003520 con 0x7f5d100afeb0 2023-12-19T18:15:06.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d101553a0 msgr2=0x7f5d100c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 --2- 172.21.15.27:0/3037796945 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d101553a0 0x7f5d100c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 >> v1:172.21.15.192:6789/0 conn(0x7f5d1015ee30 legacy=0x7f5d100c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d100cb630 con 0x7f5d100afeb0 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f5d04001670 con 0x7f5d100afeb0 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 >> v1:172.21.15.27:6789/0 conn(0x7f5d100afeb0 legacy=0x7f5d1015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 --2- 172.21.15.27:0/3037796945 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d101553a0 0x7f5cf80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 --2- 172.21.15.27:0/3037796945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8003f70 0x7f5cf80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 --2- 172.21.15.27:0/3037796945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cf8004d00 0x7f5cf8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d100cbcf0 con 0x7f5cf8003f70 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf800bb70 con 0x7f5cf8004d00 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.003+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf800bd30 con 0x7f5d101553a0 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.004+0000 7f5d0e7fc640 1 --2- 172.21.15.27:0/3037796945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8003f70 0x7f5cf80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.004+0000 7f5d0effd640 1 --2- 172.21.15.27:0/3037796945 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d101553a0 0x7f5cf80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.004+0000 7f5d0f7fe640 1 --2- 172.21.15.27:0/3037796945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cf8004d00 0x7f5cf8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.004+0000 7f5d0e7fc640 1 -- 172.21.15.27:0/3037796945 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d101553a0 msgr2=0x7f5cf80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.004+0000 7f5d0e7fc640 1 --2- 172.21.15.27:0/3037796945 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d101553a0 0x7f5cf80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.004+0000 7f5d0e7fc640 1 -- 172.21.15.27:0/3037796945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cf8004d00 msgr2=0x7f5cf8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.004+0000 7f5d0e7fc640 1 --2- 172.21.15.27:0/3037796945 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cf8004d00 0x7f5cf8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.004+0000 7f5d0e7fc640 1 -- 172.21.15.27:0/3037796945 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf8003520 con 0x7f5cf8003f70 2023-12-19T18:15:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.005+0000 7f5d0e7fc640 1 --2- 172.21.15.27:0/3037796945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8003f70 0x7f5cf80047c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400c0e0 tx=0x7f5cf80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef1b71d02941cf5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.006+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d04005ad0 con 0x7f5cf8003f70 2023-12-19T18:15:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.006+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d04005c50 con 0x7f5cf8003f70 2023-12-19T18:15:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.006+0000 7f5d0dffb640 1 -- 172.21.15.27:0/3037796945 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0402c810 con 0x7f5cf8003f70 2023-12-19T18:15:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.006+0000 7f5d0ffff640 1 -- 172.21.15.27:0/3037796945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8003f70 msgr2=0x7f5cf80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.006+0000 7f5d0ffff640 1 --2- 172.21.15.27:0/3037796945 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8003f70 0x7f5cf80047c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5d0400c0e0 tx=0x7f5cf80057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.006+0000 7f5d0f7fe640 1 -- 172.21.15.27:0/3037796945 reap_dead start 2023-12-19T18:15:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.006+0000 7f5d0ffff640 1 -- 172.21.15.27:0/3037796945 shutdown_connections 2023-12-19T18:15:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.006+0000 7f5d0ffff640 1 -- 172.21.15.27:0/3037796945 >> 172.21.15.27:0/3037796945 conn(0x7f5d10069c70 msgr2=0x7f5d1006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.006+0000 7f5d0ffff640 1 -- 172.21.15.27:0/3037796945 shutdown_connections 2023-12-19T18:15:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.006+0000 7f5d0ffff640 1 -- 172.21.15.27:0/3037796945 wait complete. 2023-12-19T18:15:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.008+0000 7f5d0ffff640 1 Processor -- start 2023-12-19T18:15:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.008+0000 7f5d0ffff640 1 -- start start 2023-12-19T18:15:06.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.009+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8003f70 0x7f5d10151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.009+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d101412d0 0x7f5d10151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.009+0000 7f5d0ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d10143b40 0x7f5d10143fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.009+0000 7f5d0ffff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d100b4420 con 0x7f5cf8003f70 2023-12-19T18:15:06.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.009+0000 7f5d0ffff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d10140e40 con 0x7f5d101412d0 2023-12-19T18:15:06.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.009+0000 7f5d0ffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d10144500 con 0x7f5d10143b40 2023-12-19T18:15:06.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.010+0000 7f5d0effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8003f70 0x7f5d10151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.010+0000 7f5d0effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8003f70 0x7f5d10151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49474/0 (socket says 172.21.15.27:49474) 2023-12-19T18:15:06.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.010+0000 7f5d0effd640 1 -- 172.21.15.27:0/2267168932 learned_addr learned my addr 172.21.15.27:0/2267168932 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.010+0000 7f5d0e7fc640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d101412d0 0x7f5d10151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.010+0000 7f5d0f7fe640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d10143b40 0x7f5d10143fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.010+0000 7f5d0effd640 1 -- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d10143b40 msgr2=0x7f5d10143fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.010+0000 7f5d0effd640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d10143b40 0x7f5d10143fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.010+0000 7f5d0effd640 1 -- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d101412d0 msgr2=0x7f5d10151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.010+0000 7f5d0effd640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d101412d0 0x7f5d10151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.010+0000 7f5d0effd640 1 -- 172.21.15.27:0/2267168932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d100b4140 con 0x7f5cf8003f70 2023-12-19T18:15:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.010+0000 7f5d0e7fc640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d101412d0 0x7f5d10151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.010+0000 7f5d0effd640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8003f70 0x7f5d10151230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5d00011660 tx=0x7f5d00011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.011+0000 7f5cf77fe640 1 -- 172.21.15.27:0/2267168932 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000f9c0 con 0x7f5cf8003f70 2023-12-19T18:15:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.011+0000 7f5cf77fe640 1 -- 172.21.15.27:0/2267168932 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0000fb20 con 0x7f5cf8003f70 2023-12-19T18:15:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.011+0000 7f5d0ffff640 1 -- 172.21.15.27:0/2267168932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d101407f0 con 0x7f5cf8003f70 2023-12-19T18:15:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.011+0000 7f5cf77fe640 1 -- 172.21.15.27:0/2267168932 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0000fe90 con 0x7f5cf8003f70 2023-12-19T18:15:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.011+0000 7f5d0ffff640 1 -- 172.21.15.27:0/2267168932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d10144990 con 0x7f5cf8003f70 2023-12-19T18:15:06.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.012+0000 7f5d0ffff640 1 -- 172.21.15.27:0/2267168932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d100c5020 con 0x7f5cf8003f70 2023-12-19T18:15:06.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.013+0000 7f5cf77fe640 1 -- 172.21.15.27:0/2267168932 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5d00015460 con 0x7f5cf8003f70 2023-12-19T18:15:06.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.014+0000 7f5cf77fe640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5cfc06a460 0x7f5cfc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.014+0000 7f5cf77fe640 1 -- 172.21.15.27:0/2267168932 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d00088e30 con 0x7f5cf8003f70 2023-12-19T18:15:06.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.014+0000 7f5d0e7fc640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5cfc06a460 0x7f5cfc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.015+0000 7f5d0e7fc640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5cfc06a460 0x7f5cfc06c920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5d04034000 tx=0x7f5d04009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.018+0000 7f5cf77fe640 1 -- 172.21.15.27:0/2267168932 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5d00054410 con 0x7f5cf8003f70 2023-12-19T18:15:06.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.217+0000 7f5d0ffff640 1 -- 172.21.15.27:0/2267168932 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d1004e840 con 0x7f5cf8003f70 2023-12-19T18:15:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.220+0000 7f5cf77fe640 1 -- 172.21.15.27:0/2267168932 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f5d00057a30 con 0x7f5cf8003f70 2023-12-19T18:15:06.221 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:06.221 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:55.039690+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.223+0000 7f5d0ffff640 1 -- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5cfc06a460 msgr2=0x7f5cfc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.223+0000 7f5d0ffff640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5cfc06a460 0x7f5cfc06c920 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5d04034000 tx=0x7f5d04009030 comp rx=0 tx=0).stop 2023-12-19T18:15:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.223+0000 7f5d0ffff640 1 -- 172.21.15.27:0/2267168932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8003f70 msgr2=0x7f5d10151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.223+0000 7f5d0ffff640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8003f70 0x7f5d10151230 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5d00011660 tx=0x7f5d00011b20 comp rx=0 tx=0).stop 2023-12-19T18:15:06.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.223+0000 7f5d0ffff640 1 -- 172.21.15.27:0/2267168932 shutdown_connections 2023-12-19T18:15:06.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.223+0000 7f5d0ffff640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5cfc06a460 0x7f5cfc06c920 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.223+0000 7f5d0ffff640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d10143b40 0x7f5d10143fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.223+0000 7f5d0ffff640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d101412d0 0x7f5d10151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.223+0000 7f5d0ffff640 1 --2- 172.21.15.27:0/2267168932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cf8003f70 0x7f5d10151230 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.223+0000 7f5d0ffff640 1 -- 172.21.15.27:0/2267168932 >> 172.21.15.27:0/2267168932 conn(0x7f5d10069c70 msgr2=0x7f5d100c38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.223+0000 7f5d0ffff640 1 -- 172.21.15.27:0/2267168932 shutdown_connections 2023-12-19T18:15:06.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.223+0000 7f5d0ffff640 1 -- 172.21.15.27:0/2267168932 wait complete. 2023-12-19T18:15:06.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-12-19T18:15:06.242 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-12-19T18:15:06.243 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-12-19T18:15:06.243 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-12-19T18:15:06.243 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-12-19T18:15:06.244 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-12-19T18:15:06.244 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-12-19T18:15:06.245 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-12-19T18:15:06.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.572+0000 7fe75ac66640 1 Processor -- start 2023-12-19T18:15:06.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.574+0000 7fb7dbfff640 1 Processor -- start 2023-12-19T18:15:06.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.572+0000 7fe75ac66640 1 -- start start 2023-12-19T18:15:06.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.574+0000 7fe75ac66640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe75415d240 0x7fe7541618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.574+0000 7fe75ac66640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7540ce420 0x7fe7540b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.574+0000 7fe75ac66640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7540d1970 con 0x7fe7540cdf90 2023-12-19T18:15:06.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.574+0000 7fe75ac66640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe754161e10 con 0x7fe75415d240 2023-12-19T18:15:06.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.574+0000 7fe75ac66640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75406a5e0 con 0x7fe7540ce420 2023-12-19T18:15:06.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.574+0000 7fb7dbfff640 1 -- start start 2023-12-19T18:15:06.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.576+0000 7fb7dbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7dc0d0fa0 0x7fb7dc0b4820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.576+0000 7fb7dbfff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7dc0b5190 con 0x7fb7dc15cfb0 2023-12-19T18:15:06.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.576+0000 7fb7dbfff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7dc0d1580 con 0x7fb7dc0b4d60 2023-12-19T18:15:06.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.576+0000 7fb7dbfff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7dc0d1860 con 0x7fb7dc0d0fa0 2023-12-19T18:15:06.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.576+0000 7fe753fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe7540cdf90 0x7fe7540b4c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53948/0 (socket says 172.21.15.27:53948) 2023-12-19T18:15:06.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.577+0000 7fb7db7fe640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fb7dc0b4d60 0x7fb7dc0b6350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:36734/0 (socket says 172.21.15.27:36734) 2023-12-19T18:15:06.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.577+0000 7fb7db7fe640 1 -- 172.21.15.27:0/1304234364 learned_addr learned my addr 172.21.15.27:0/1304234364 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:06.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.577+0000 7fe758a02640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe75415d240 0x7fe7541618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.577+0000 7fe758a02640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe75415d240 0x7fe7541618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49478/0 (socket says 172.21.15.27:49478) 2023-12-19T18:15:06.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.577+0000 7fe758a02640 1 -- 172.21.15.27:0/2440104950 learned_addr learned my addr 172.21.15.27:0/2440104950 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.577+0000 7fe759203640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7540ce420 0x7fe7540b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.577+0000 7fb7da7fc640 1 --2- 172.21.15.27:0/1304234364 >> v2:172.21.15.27:3300/0 conn(0x7fb7dc0d0fa0 0x7fb7dc0b4820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.578+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2034329762 0 0) 0x7fb7dc0d1580 con 0x7fb7dc0b4d60 2023-12-19T18:15:06.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.578+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7c4003520 con 0x7fb7dc0b4d60 2023-12-19T18:15:06.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.579+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4015727233 0 0) 0x7fb7c4003520 con 0x7fb7dc0b4d60 2023-12-19T18:15:06.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.579+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 >> v2:172.21.15.27:3300/0 conn(0x7fb7dc0d0fa0 msgr2=0x7fb7dc0b4820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.579+0000 7fb7d9ffb640 1 --2- 172.21.15.27:0/1304234364 >> v2:172.21.15.27:3300/0 conn(0x7fb7dc0d0fa0 0x7fb7dc0b4820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.580+0000 7fe758a02640 1 -- 172.21.15.27:0/2440104950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7540ce420 msgr2=0x7fe7540b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.579+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 >> v1:172.21.15.27:6789/0 conn(0x7fb7dc15cfb0 legacy=0x7fb7dc0d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.579+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 --> v1:172.21.15.192:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc0d1b40 con 0x7fb7dc0b4d60 2023-12-19T18:15:06.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.579+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 <== mon.1 v1:172.21.15.192:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fb7d40019f0 con 0x7fb7dc0b4d60 2023-12-19T18:15:06.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.579+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 >> v1:172.21.15.192:6789/0 conn(0x7fb7dc0b4d60 legacy=0x7fb7dc0b6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.579+0000 7fb7d9ffb640 1 --2- 172.21.15.27:0/1304234364 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7c4003970 0x7fb7c4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.580+0000 7fb7d9ffb640 1 --2- 172.21.15.27:0/1304234364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7c40047c0 0x7fb7c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.580+0000 7fb7d9ffb640 1 --2- 172.21.15.27:0/1304234364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7c4005180 0x7fb7c4014190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.580+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc0d1860 con 0x7fb7c4005180 2023-12-19T18:15:06.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.580+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c40147e0 con 0x7fb7c4003970 2023-12-19T18:15:06.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.580+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c40149a0 con 0x7fb7c40047c0 2023-12-19T18:15:06.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.580+0000 7fe758a02640 1 --2- 172.21.15.27:0/2440104950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7540ce420 0x7fe7540b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.580+0000 7fe758a02640 1 -- 172.21.15.27:0/2440104950 >> v1:172.21.15.27:6789/0 conn(0x7fe7540cdf90 legacy=0x7fe7540b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.580+0000 7fb7da7fc640 1 --2- 172.21.15.27:0/1304234364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7c40047c0 0x7fb7c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.581+0000 7fe758a02640 1 -- 172.21.15.27:0/2440104950 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7540ce8b0 con 0x7fe75415d240 2023-12-19T18:15:06.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.581+0000 7fcb98241640 1 Processor -- start 2023-12-19T18:15:06.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.581+0000 7f09bcaa0640 1 Processor -- start 2023-12-19T18:15:06.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.581+0000 7f09bcaa0640 1 -- start start 2023-12-19T18:15:06.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.581+0000 7fb7da7fc640 1 -- 172.21.15.27:0/1304234364 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7c4003970 msgr2=0x7fb7c4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.581+0000 7fb7da7fc640 1 --2- 172.21.15.27:0/1304234364 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7c4003970 0x7fb7c4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.581+0000 7fb7da7fc640 1 -- 172.21.15.27:0/1304234364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7c4005180 msgr2=0x7fb7c4014190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.581+0000 7fb7da7fc640 1 --2- 172.21.15.27:0/1304234364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7c4005180 0x7fb7c4014190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.581+0000 7fb7db7fe640 1 -- 172.21.15.27:0/1304234364 reap_dead start 2023-12-19T18:15:06.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.581+0000 7fb7da7fc640 1 -- 172.21.15.27:0/1304234364 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc0d1b40 con 0x7fb7c40047c0 2023-12-19T18:15:06.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.581+0000 7fcb98241640 1 -- start start 2023-12-19T18:15:06.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.582+0000 7fe759203640 1 --2- 172.21.15.27:0/2440104950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7540ce420 0x7fe7540b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:06.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.583+0000 7f09bcaa0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f09b8161740 0x7f09b80d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.583+0000 7f09bcaa0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09b80cdd00 0x7f09b8165740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.583+0000 7fe758a02640 1 --2- 172.21.15.27:0/2440104950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75415d240 0x7fe7541618d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009870 tx=0x7fe74c02ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8152d5cbc55b3f85 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7f09bcaa0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09b80d2df0 con 0x7f09b81652d0 2023-12-19T18:15:06.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.582+0000 7f8ef239c640 1 Processor -- start 2023-12-19T18:15:06.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.582+0000 7f8ef239c640 1 -- start start 2023-12-19T18:15:06.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7f09bcaa0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09b8165c80 con 0x7f09b8161740 2023-12-19T18:15:06.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7f09bcaa0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b8161b20 con 0x7f09b80cdd00 2023-12-19T18:15:06.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7fe7537fe640 1 -- 172.21.15.27:0/2440104950 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c02f830 con 0x7fe75415d240 2023-12-19T18:15:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7f8ef239c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8eec161740 0x7f8eec0d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7f8ef239c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8eec1652d0 0x7f8eec165710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7fcb98241640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb90164a80 0x7fcb900ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7fcb98241640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb900caba0 con 0x7fcb900cbc60 2023-12-19T18:15:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7fcb98241640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb90164e60 con 0x7fcb900c9190 2023-12-19T18:15:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7fcb98241640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9006c2d0 con 0x7fcb90164a80 2023-12-19T18:15:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7f09b6d9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09b80cdd00 0x7f09b8165740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7f09b6d9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09b80cdd00 0x7f09b8165740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:42070/0 (socket says 172.21.15.27:42070) 2023-12-19T18:15:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7f09b6d9d640 1 -- 172.21.15.27:0/3924270228 learned_addr learned my addr 172.21.15.27:0/3924270228 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f09b659c640 1 --2- 172.21.15.27:0/3924270228 >> v2:172.21.15.27:3300/0 conn(0x7f09b8161740 0x7f09b80d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f09b6d9d640 1 -- 172.21.15.27:0/3924270228 >> v2:172.21.15.27:3300/0 conn(0x7f09b8161740 msgr2=0x7f09b80d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f09b6d9d640 1 --2- 172.21.15.27:0/3924270228 >> v2:172.21.15.27:3300/0 conn(0x7f09b8161740 0x7f09b80d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f09b6d9d640 1 -- 172.21.15.27:0/3924270228 >> v1:172.21.15.192:6789/0 conn(0x7f09b81652d0 legacy=0x7f09b80cd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f09b6d9d640 1 -- 172.21.15.27:0/3924270228 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b8165e70 con 0x7f09b80cdd00 2023-12-19T18:15:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f09b659c640 1 --2- 172.21.15.27:0/3924270228 >> v2:172.21.15.27:3300/0 conn(0x7f09b8161740 0x7f09b80d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7fe7537fe640 1 -- 172.21.15.27:0/2440104950 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74c02f990 con 0x7fe75415d240 2023-12-19T18:15:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7fe7537fe640 1 -- 172.21.15.27:0/2440104950 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74c02fd20 con 0x7fe75415d240 2023-12-19T18:15:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fe75ac66640 1 -- 172.21.15.27:0/2440104950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75415d240 msgr2=0x7fe7541618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7fb7da7fc640 1 --2- 172.21.15.27:0/1304234364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7c40047c0 0x7fb7c4004c40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009ac0 tx=0x7fb7c40057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c21e198583c608b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7fcb95fdd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb90164a80 0x7fcb900ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7f8ef239c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eec0d2df0 con 0x7f8eec0cd5f0 2023-12-19T18:15:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7f8ef239c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8eec165c50 con 0x7f8eec1652d0 2023-12-19T18:15:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.584+0000 7f8ef239c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec161b20 con 0x7f8eec161740 2023-12-19T18:15:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f8eeb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8eec1652d0 0x7f8eec165710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f8eeb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8eec1652d0 0x7f8eec165710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49496/0 (socket says 172.21.15.27:49496) 2023-12-19T18:15:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f8eeb7fe640 1 -- 172.21.15.27:0/40167644 learned_addr learned my addr 172.21.15.27:0/40167644 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f8eebfff640 1 --2- 172.21.15.27:0/40167644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8eec161740 0x7f8eec0d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f8eeb7fe640 1 -- 172.21.15.27:0/40167644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8eec161740 msgr2=0x7f8eec0d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f8eeb7fe640 1 --2- 172.21.15.27:0/40167644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8eec161740 0x7f8eec0d1810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f8eeb7fe640 1 -- 172.21.15.27:0/40167644 >> v1:172.21.15.192:6789/0 conn(0x7f8eec0cd5f0 legacy=0x7f8eec0b5df0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-19T18:15:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7f8eeb7fe640 1 -- 172.21.15.27:0/40167644 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eec165e70 con 0x7f8eec1652d0 2023-12-19T18:15:06.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fe75ac66640 1 --2- 172.21.15.27:0/2440104950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75415d240 0x7fe7541618d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009870 tx=0x7fe74c02ba10 comp rx=0 tx=0).stop 2023-12-19T18:15:06.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.585+0000 7fcb957dc640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fcb900c9190 0x7fcb900cb550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:36748/0 (socket says 172.21.15.27:36748) 2023-12-19T18:15:06.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fcb957dc640 1 -- 172.21.15.27:0/437916177 learned_addr learned my addr 172.21.15.27:0/437916177 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:06.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fe75ac66640 1 -- 172.21.15.27:0/2440104950 shutdown_connections 2023-12-19T18:15:06.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fe75ac66640 1 --2- 172.21.15.27:0/2440104950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7540ce420 0x7fe7540b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fe75ac66640 1 --2- 172.21.15.27:0/2440104950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe75415d240 0x7fe7541618d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fe75ac66640 1 -- 172.21.15.27:0/2440104950 >> 172.21.15.27:0/2440104950 conn(0x7fe7540d31e0 msgr2=0x7fe7540d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7f9b6a49e640 1 Processor -- start 2023-12-19T18:15:06.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.587+0000 7f09b6d9d640 1 --2- 172.21.15.27:0/3924270228 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09b80cdd00 0x7f09b8165740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009e50 tx=0x7f09b002bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f17b8eda07fad88 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fe75ac66640 1 -- 172.21.15.27:0/2440104950 shutdown_connections 2023-12-19T18:15:06.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d002c610 con 0x7fb7c40047c0 2023-12-19T18:15:06.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d002c770 con 0x7fb7c40047c0 2023-12-19T18:15:06.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fb7d9ffb640 1 -- 172.21.15.27:0/1304234364 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d002cb00 con 0x7fb7c40047c0 2023-12-19T18:15:06.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.587+0000 7fb7dbfff640 1 -- 172.21.15.27:0/1304234364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7c40047c0 msgr2=0x7fb7c4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.587+0000 7fb7dbfff640 1 --2- 172.21.15.27:0/1304234364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7c40047c0 0x7fb7c4004c40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0009ac0 tx=0x7fb7c40057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.588+0000 7fb7dbfff640 1 -- 172.21.15.27:0/1304234364 shutdown_connections 2023-12-19T18:15:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.588+0000 7fb7dbfff640 1 --2- 172.21.15.27:0/1304234364 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7c40047c0 0x7fb7c4004c40 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.588+0000 7fb7dbfff640 1 -- 172.21.15.27:0/1304234364 >> 172.21.15.27:0/1304234364 conn(0x7fb7dc0cf820 msgr2=0x7fb7dc0cfc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fcb94fdb640 1 -- 172.21.15.27:0/437916177 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2404623994 0 0) 0x7fcb90164e60 con 0x7fcb900c9190 2023-12-19T18:15:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fcb94fdb640 1 -- 172.21.15.27:0/437916177 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb80003520 con 0x7fcb900c9190 2023-12-19T18:15:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.587+0000 7fcb95fdd640 1 -- 172.21.15.27:0/437916177 >> v1:172.21.15.192:6789/0 conn(0x7fcb900c9190 legacy=0x7fcb900cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.587+0000 7fcb95fdd640 1 -- 172.21.15.27:0/437916177 >> v1:172.21.15.27:6789/0 conn(0x7fcb900cbc60 legacy=0x7fcb900cc710 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-19T18:15:06.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.587+0000 7fcb95fdd640 1 -- 172.21.15.27:0/437916177 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb900cae40 con 0x7fcb90164a80 2023-12-19T18:15:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.587+0000 7f9b6a49e640 1 -- start start 2023-12-19T18:15:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.587+0000 7f8eeb7fe640 1 --2- 172.21.15.27:0/40167644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8eec1652d0 0x7f8eec165710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009870 tx=0x7f8edc02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff953bcb3ed029e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.588+0000 7fb7dbfff640 1 -- 172.21.15.27:0/1304234364 shutdown_connections 2023-12-19T18:15:06.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.588+0000 7fb7dbfff640 1 -- 172.21.15.27:0/1304234364 wait complete. 2023-12-19T18:15:06.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.588+0000 7f09b559a640 1 -- 172.21.15.27:0/3924270228 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b002ca80 con 0x7f09b80cdd00 2023-12-19T18:15:06.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.588+0000 7f09b559a640 1 -- 172.21.15.27:0/3924270228 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b002cbe0 con 0x7f09b80cdd00 2023-12-19T18:15:06.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.588+0000 7f09b559a640 1 -- 172.21.15.27:0/3924270228 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b00343f0 con 0x7f09b80cdd00 2023-12-19T18:15:06.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.586+0000 7fe75ac66640 1 -- 172.21.15.27:0/2440104950 wait complete. 2023-12-19T18:15:06.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.588+0000 7f8eeaffd640 1 -- 172.21.15.27:0/40167644 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc031640 con 0x7f8eec1652d0 2023-12-19T18:15:06.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.589+0000 7f8eeaffd640 1 -- 172.21.15.27:0/40167644 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8edc0317a0 con 0x7f8eec1652d0 2023-12-19T18:15:06.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.589+0000 7f8eeaffd640 1 -- 172.21.15.27:0/40167644 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc031b30 con 0x7f8eec1652d0 2023-12-19T18:15:06.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.589+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3924270228 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09b80cdd00 msgr2=0x7f09b8165740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.589+0000 7f09bcaa0640 1 --2- 172.21.15.27:0/3924270228 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09b80cdd00 0x7f09b8165740 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09b0009e50 tx=0x7f09b002bee0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.589+0000 7f9b6a49e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b5c0e78b0 0x7f9b5c04f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.589+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3924270228 shutdown_connections 2023-12-19T18:15:06.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.589+0000 7f09bcaa0640 1 --2- 172.21.15.27:0/3924270228 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09b80cdd00 0x7f09b8165740 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.589+0000 7f09bcaa0640 1 --2- 172.21.15.27:0/3924270228 >> v2:172.21.15.27:3300/0 conn(0x7f09b8161740 0x7f09b80d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.589+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3924270228 >> 172.21.15.27:0/3924270228 conn(0x7f09b80d2fa0 msgr2=0x7f09b80d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.589+0000 7fcb95fdd640 1 --2- 172.21.15.27:0/437916177 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb90164a80 0x7fcb900ce520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009dc0 tx=0x7fcb8402be00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d503fdb59dfea1f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.589+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3924270228 shutdown_connections 2023-12-19T18:15:06.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.589+0000 7f9b6a49e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9b5c0eb2c0 0x7f9b5c0eb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3924270228 wait complete. 2023-12-19T18:15:06.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7fe75ac66640 1 Processor -- start 2023-12-19T18:15:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7fcb94fdb640 1 -- 172.21.15.27:0/437916177 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8402cbb0 con 0x7fcb90164a80 2023-12-19T18:15:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7fcb94fdb640 1 -- 172.21.15.27:0/437916177 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8402cd10 con 0x7fcb90164a80 2023-12-19T18:15:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7fcb94fdb640 1 -- 172.21.15.27:0/437916177 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84034660 con 0x7fcb90164a80 2023-12-19T18:15:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7f8ef239c640 1 -- 172.21.15.27:0/40167644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8eec1652d0 msgr2=0x7f8eec165710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7f8ef239c640 1 --2- 172.21.15.27:0/40167644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8eec1652d0 0x7f8eec165710 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8edc009870 tx=0x7f8edc02b8f0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7f9b6a49e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b5c050340 con 0x7f9b5c052fe0 2023-12-19T18:15:06.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7f9b6a49e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b5c006480 con 0x7f9b5c0eb2c0 2023-12-19T18:15:06.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7f9b6a49e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0ebc40 con 0x7f9b5c0e78b0 2023-12-19T18:15:06.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7fe75ac66640 1 -- start start 2023-12-19T18:15:06.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7f9b69c9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9b5c052fe0 0x7f9b5c053470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53980/0 (socket says 172.21.15.27:53980) 2023-12-19T18:15:06.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7f9b69c9d640 1 -- 172.21.15.27:0/1468957322 learned_addr learned my addr 172.21.15.27:0/1468957322 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:06.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.590+0000 7f9b6949c640 1 --2- 172.21.15.27:0/1468957322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b5c0e78b0 0x7f9b5c04f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.591+0000 7f9b68c9b640 1 --2- 172.21.15.27:0/1468957322 >> v2:172.21.15.192:3300/0 conn(0x7f9b5c0eb2c0 0x7f9b5c0eb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.591+0000 7f8ef239c640 1 -- 172.21.15.27:0/40167644 shutdown_connections 2023-12-19T18:15:06.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.591+0000 7f8ef239c640 1 --2- 172.21.15.27:0/40167644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8eec1652d0 0x7f8eec165710 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.591+0000 7f8ef239c640 1 --2- 172.21.15.27:0/40167644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8eec161740 0x7f8eec0d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.591+0000 7f8ef239c640 1 -- 172.21.15.27:0/40167644 >> 172.21.15.27:0/40167644 conn(0x7f8eec0d2fa0 msgr2=0x7f8eec0d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.591+0000 7fcb98241640 1 -- 172.21.15.27:0/437916177 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb90164a80 msgr2=0x7fcb900ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.591+0000 7fcb98241640 1 --2- 172.21.15.27:0/437916177 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb90164a80 0x7fcb900ce520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcb84009dc0 tx=0x7fcb8402be00 comp rx=0 tx=0).stop 2023-12-19T18:15:06.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.591+0000 7fb7dbfff640 1 Processor -- start 2023-12-19T18:15:06.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7f09bcaa0640 1 Processor -- start 2023-12-19T18:15:06.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7f09bcaa0640 1 -- start start 2023-12-19T18:15:06.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7f09bcaa0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09b80cdd00 0x7f09b80b7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7f09bcaa0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09b8161740 0x7f09b80b8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7f09bcaa0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09b81652d0 0x7f09b80b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7f09bcaa0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80b8f70 con 0x7f09b81652d0 2023-12-19T18:15:06.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7f09bcaa0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09b814c050 con 0x7f09b8161740 2023-12-19T18:15:06.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7f09bcaa0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09b80b5df0 con 0x7f09b80cdd00 2023-12-19T18:15:06.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7fb7dbfff640 1 -- start start 2023-12-19T18:15:06.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7fb7dbfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7c40047c0 0x7fb7dc1552f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7fb7dbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7dc0cb730 0x7fb7dc0cac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7fb7dbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7dc0b2830 0x7fb7dc0b2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7fb7dbfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc0cc8b0 con 0x7fb7dc0cb730 2023-12-19T18:15:06.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7fb7dbfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc0b3220 con 0x7fb7dc0b2830 2023-12-19T18:15:06.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7fb7dbfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7dc0cccd0 con 0x7fb7c40047c0 2023-12-19T18:15:06.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.591+0000 7fcb98241640 1 -- 172.21.15.27:0/437916177 shutdown_connections 2023-12-19T18:15:06.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.591+0000 7fcb98241640 1 --2- 172.21.15.27:0/437916177 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb90164a80 0x7fcb900ce520 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.591+0000 7fcb98241640 1 -- 172.21.15.27:0/437916177 >> 172.21.15.27:0/437916177 conn(0x7fcb9006a330 msgr2=0x7fcb900b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7fcb98241640 1 -- 172.21.15.27:0/437916177 shutdown_connections 2023-12-19T18:15:06.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7fcb98241640 1 -- 172.21.15.27:0/437916177 wait complete. 2023-12-19T18:15:06.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 896962381 0 0) 0x7f9b5c050340 con 0x7f9b5c052fe0 2023-12-19T18:15:06.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b50003520 con 0x7f9b5c052fe0 2023-12-19T18:15:06.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 760454728 0 0) 0x7f9b50003520 con 0x7f9b5c052fe0 2023-12-19T18:15:06.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b5c0e78b0 msgr2=0x7f9b5c04f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7f9b5bfff640 1 --2- 172.21.15.27:0/1468957322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b5c0e78b0 0x7f9b5c04f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 >> v2:172.21.15.192:3300/0 conn(0x7f9b5c0eb2c0 msgr2=0x7f9b5c0eb700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7f9b5bfff640 1 --2- 172.21.15.27:0/1468957322 >> v2:172.21.15.192:3300/0 conn(0x7f9b5c0eb2c0 0x7f9b5c0eb700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c007920 con 0x7f9b5c052fe0 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.592+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f9b6404e840 con 0x7f9b5c052fe0 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.593+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 >> v1:172.21.15.27:6789/0 conn(0x7f9b5c052fe0 legacy=0x7f9b5c053470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.593+0000 7f9b5bfff640 1 --2- 172.21.15.27:0/1468957322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b50003970 0x7f9b50003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.593+0000 7f9b5bfff640 1 --2- 172.21.15.27:0/1468957322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b50004310 0x7f9b500047e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.593+0000 7f9b5bfff640 1 --2- 172.21.15.27:0/1468957322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b50004db0 0x7f9b50005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.593+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c006480 con 0x7f9b50004db0 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.593+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0ebc40 con 0x7f9b50003970 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.593+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5000bd40 con 0x7f9b50004310 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.593+0000 7f9b6949c640 1 --2- 172.21.15.27:0/1468957322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b5c0e78b0 0x7f9b5c04f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.593+0000 7f9b6949c640 1 --2- 172.21.15.27:0/1468957322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b50003970 0x7f9b50003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.593+0000 7f9b69c9d640 1 --2- 172.21.15.27:0/1468957322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b50004db0 0x7f9b50005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7f9b68c9b640 1 --2- 172.21.15.27:0/1468957322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b50004310 0x7f9b500047e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.591+0000 7f8ef239c640 1 -- 172.21.15.27:0/40167644 shutdown_connections 2023-12-19T18:15:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7fb7daffd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7c40047c0 0x7fb7dc1552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7f09b659c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09b80cdd00 0x7f09b80b7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7f09b659c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09b80cdd00 0x7f09b80b7b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:42142/0 (socket says 172.21.15.27:42142) 2023-12-19T18:15:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7f09b659c640 1 -- 172.21.15.27:0/3691694587 learned_addr learned my addr 172.21.15.27:0/3691694587 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7f09b5d9b640 1 --2- 172.21.15.27:0/3691694587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09b8161740 0x7f09b80b8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7fb7daffd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7c40047c0 0x7fb7dc1552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:42130/0 (socket says 172.21.15.27:42130) 2023-12-19T18:15:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7fb7daffd640 1 -- 172.21.15.27:0/2683452048 learned_addr learned my addr 172.21.15.27:0/2683452048 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7fb7da7fc640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7dc0cb730 0x7fb7dc0cac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7fc0707cb640 1 Processor -- start 2023-12-19T18:15:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7fcb98241640 1 Processor -- start 2023-12-19T18:15:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7fc0707cb640 1 -- start start 2023-12-19T18:15:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7fb7db7fe640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7dc0b2830 0x7fb7dc0b2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7fcb98241640 1 -- start start 2023-12-19T18:15:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7fb7da7fc640 1 -- 172.21.15.27:0/2683452048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7c40047c0 msgr2=0x7fb7dc1552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7fb7da7fc640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7c40047c0 0x7fb7dc1552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7f09b659c640 1 -- 172.21.15.27:0/3691694587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09b8161740 msgr2=0x7f09b80b8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7f09b659c640 1 --2- 172.21.15.27:0/3691694587 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f09b8161740 0x7f09b80b8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7f09b659c640 1 -- 172.21.15.27:0/3691694587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09b81652d0 msgr2=0x7f09b80b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7f09b659c640 1 --2- 172.21.15.27:0/3691694587 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09b81652d0 0x7f09b80b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7f09b659c640 1 -- 172.21.15.27:0/3691694587 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09b0009b30 con 0x7f09b80cdd00 2023-12-19T18:15:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7f9b69c9d640 1 -- 172.21.15.27:0/1468957322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b50004310 msgr2=0x7f9b500047e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7f9b69c9d640 1 --2- 172.21.15.27:0/1468957322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b50004310 0x7f9b500047e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7f9b69c9d640 1 -- 172.21.15.27:0/1468957322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b50003970 msgr2=0x7f9b50003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7f9b69c9d640 1 --2- 172.21.15.27:0/1468957322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b50003970 0x7f9b50003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.595+0000 7f9b69c9d640 1 -- 172.21.15.27:0/1468957322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c007920 con 0x7f9b50004db0 2023-12-19T18:15:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.596+0000 7fb7da7fc640 1 -- 172.21.15.27:0/2683452048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7dc0b2830 msgr2=0x7fb7dc0b2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.596+0000 7fb7da7fc640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7dc0b2830 0x7fb7dc0b2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.596+0000 7fb7da7fc640 1 -- 172.21.15.27:0/2683452048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7dc0cbc30 con 0x7fb7dc0cb730 2023-12-19T18:15:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.596+0000 7fb7db7fe640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7dc0b2830 0x7fb7dc0b2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.591+0000 7f8ef239c640 1 -- 172.21.15.27:0/40167644 wait complete. 2023-12-19T18:15:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.594+0000 7f8ef239c640 1 Processor -- start 2023-12-19T18:15:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.596+0000 7f9b69c9d640 1 --2- 172.21.15.27:0/1468957322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b50004db0 0x7f9b50005230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b64063840 tx=0x7f9b500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15f043d44a629f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.596+0000 7fb7da7fc640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7dc0cb730 0x7fb7dc0cac30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4014b30 tx=0x7fb7d0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.596+0000 7f09b659c640 1 --2- 172.21.15.27:0/3691694587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09b80cdd00 0x7f09b80b7b30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f09a400c820 tx=0x7f09a400cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fc0707cb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc0680b8c30 0x7fc0680d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fc0707cb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0680ce130 0x7fc0680cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fc0707cb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0680ce560 con 0x7fc0680cf9b0 2023-12-19T18:15:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fc0707cb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0680cea80 con 0x7fc0680b8c30 2023-12-19T18:15:06.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fc0707cb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0680b5f10 con 0x7fc0680ce130 2023-12-19T18:15:06.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fb7c37fe640 1 -- 172.21.15.27:0/2683452048 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0003810 con 0x7fb7dc0cb730 2023-12-19T18:15:06.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fb7c37fe640 1 -- 172.21.15.27:0/2683452048 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7d0003970 con 0x7fb7dc0cb730 2023-12-19T18:15:06.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fb7c37fe640 1 -- 172.21.15.27:0/2683452048 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7d0003d00 con 0x7fb7dc0cb730 2023-12-19T18:15:06.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fcb98241640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb900c9190 0x7fcb900b40d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b640ab4d0 con 0x7f9b50004db0 2023-12-19T18:15:06.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fc06e567640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc0680b8c30 0x7fc0680d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fc06e567640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc0680b8c30 0x7fc0680d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49530/0 (socket says 172.21.15.27:49530) 2023-12-19T18:15:06.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fc06e567640 1 -- 172.21.15.27:0/1839360131 learned_addr learned my addr 172.21.15.27:0/1839360131 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7f09aeffd640 1 -- 172.21.15.27:0/3691694587 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4007a30 con 0x7f09b80cdd00 2023-12-19T18:15:06.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3691694587 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b80c7180 con 0x7f09b80cdd00 2023-12-19T18:15:06.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fcb98241640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb900cbc60 0x7fcb900af2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b640ab630 con 0x7f9b50004db0 2023-12-19T18:15:06.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7f9b5bfff640 1 -- 172.21.15.27:0/1468957322 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b640aba20 con 0x7f9b50004db0 2023-12-19T18:15:06.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fc06ed68640 1 --2- 172.21.15.27:0/1839360131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0680ce130 0x7fc0680cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3691694587 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09b80b8a50 con 0x7f09b80cdd00 2023-12-19T18:15:06.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fe75ac66640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7540cdf90 0x7fe754158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fe75ac66640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7540ce420 0x7fe7541564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fe75ac66640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe75415d240 0x7fe754156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fe75ac66640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7540cbc00 con 0x7fe7540ce420 2023-12-19T18:15:06.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fe75ac66640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe754153750 con 0x7fe75415d240 2023-12-19T18:15:06.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.597+0000 7fe75ac66640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe754153a30 con 0x7fe7540cdf90 2023-12-19T18:15:06.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fe758a02640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7540cdf90 0x7fe754158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fe758a02640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7540cdf90 0x7fe754158f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:42168/0 (socket says 172.21.15.27:42168) 2023-12-19T18:15:06.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fe758a02640 1 -- 172.21.15.27:0/2911925405 learned_addr learned my addr 172.21.15.27:0/2911925405 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fe759203640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe75415d240 0x7fe754156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fb7dbfff640 1 -- 172.21.15.27:0/2683452048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7dc0cbf10 con 0x7fb7dc0cb730 2023-12-19T18:15:06.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fcb95fdd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb900c9190 0x7fcb900b40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fcb98241640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb90164a80 0x7fcb90163a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fcb95fdd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb900c9190 0x7fcb900b40d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:50442/0 (socket says 172.21.15.27:50442) 2023-12-19T18:15:06.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fcb95fdd640 1 -- 172.21.15.27:0/1036956010 learned_addr learned my addr 172.21.15.27:0/1036956010 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb900c4c60 con 0x7fcb90164a80 2023-12-19T18:15:06.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb900c4f40 con 0x7fcb900c9190 2023-12-19T18:15:06.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb900c5220 con 0x7fcb900cbc60 2023-12-19T18:15:06.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fcb957dc640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb900cbc60 0x7fcb900af2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.599+0000 7fcb967de640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb90164a80 0x7fcb90163a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fc06e567640 1 -- 172.21.15.27:0/1839360131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0680ce130 msgr2=0x7fc0680cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fc06e567640 1 --2- 172.21.15.27:0/1839360131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0680ce130 0x7fc0680cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fc06e567640 1 -- 172.21.15.27:0/1839360131 >> v1:172.21.15.27:6789/0 conn(0x7fc0680cf9b0 legacy=0x7fc0680cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7fc06e567640 1 -- 172.21.15.27:0/1839360131 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0680ce7a0 con 0x7fc0680b8c30 2023-12-19T18:15:06.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.598+0000 7f8ef239c640 1 -- start start 2023-12-19T18:15:06.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.599+0000 7fe759203640 1 -- 172.21.15.27:0/2911925405 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7540cdf90 msgr2=0x7fe754158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.599+0000 7fe759203640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7540cdf90 0x7fe754158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.599+0000 7fe759203640 1 -- 172.21.15.27:0/2911925405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7540ce420 msgr2=0x7fe7541564e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-19T18:15:06.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.599+0000 7f9b6a49e640 1 -- 172.21.15.27:0/1468957322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b50004db0 msgr2=0x7f9b50005230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7f9b6a49e640 1 --2- 172.21.15.27:0/1468957322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b50004db0 0x7f9b50005230 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9b64063840 tx=0x7f9b500057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7f9b69c9d640 1 -- 172.21.15.27:0/1468957322 reap_dead start 2023-12-19T18:15:06.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.599+0000 7fb7dbfff640 1 -- 172.21.15.27:0/2683452048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7dc0cc4d0 con 0x7fb7dc0cb730 2023-12-19T18:15:06.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7f09aeffd640 1 -- 172.21.15.27:0/3691694587 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09a400ce70 con 0x7f09b80cdd00 2023-12-19T18:15:06.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7fe753fff640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7540ce420 0x7fe7541564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7f9b6a49e640 1 -- 172.21.15.27:0/1468957322 shutdown_connections 2023-12-19T18:15:06.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7f9b6a49e640 1 -- 172.21.15.27:0/1468957322 >> 172.21.15.27:0/1468957322 conn(0x7f9b5c054c10 msgr2=0x7f9b5c054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3691694587 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f09b80d1010 con 0x7f09b80cdd00 2023-12-19T18:15:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7fcb967de640 1 -- 172.21.15.27:0/1036956010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb900cbc60 msgr2=0x7fcb900af2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7fcb967de640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb900cbc60 0x7fcb900af2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7fcb967de640 1 -- 172.21.15.27:0/1036956010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb900c9190 msgr2=0x7fcb900b40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7fcb967de640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb900c9190 0x7fcb900b40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7fcb967de640 1 -- 172.21.15.27:0/1036956010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb84009a70 con 0x7fcb90164a80 2023-12-19T18:15:06.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7fe759203640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7540ce420 0x7fe7541564e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7fe759203640 1 -- 172.21.15.27:0/2911925405 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c0094d0 con 0x7fe75415d240 2023-12-19T18:15:06.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.600+0000 7fc06e567640 1 --2- 172.21.15.27:0/1839360131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0680b8c30 0x7fc0680d1fc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc064009870 tx=0x7fc06402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81cb1decf508fc8a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.601+0000 7fe753fff640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7540ce420 0x7fe7541564e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:06.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.601+0000 7f9b6a49e640 1 -- 172.21.15.27:0/1468957322 shutdown_connections 2023-12-19T18:15:06.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.601+0000 7fe759203640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe75415d240 0x7fe754156a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe744007e60 tx=0x7fe74400d4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.601+0000 7fcb967de640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb90164a80 0x7fcb90163a90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcb900717b0 tx=0x7fcb8800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.602+0000 7fb7dbfff640 1 -- 172.21.15.27:0/2683452048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7dc15d390 con 0x7fb7dc0cb730 2023-12-19T18:15:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.601+0000 7f8ef239c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8eec0cd5f0 0x7f8eec0b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.602+0000 7f8ef239c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8eec161740 0x7f8eec0c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.602+0000 7f8ef239c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8eec1652d0 0x7f8eec0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.602+0000 7f8ef239c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec0b6330 con 0x7f8eec0cd5f0 2023-12-19T18:15:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.602+0000 7f8ef239c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec15b250 con 0x7f8eec1652d0 2023-12-19T18:15:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.602+0000 7f8ef239c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eec15b530 con 0x7f8eec161740 2023-12-19T18:15:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.602+0000 7f8eebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8eec0cd5f0 0x7f8eec0b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.602+0000 7f8eebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8eec0cd5f0 0x7f8eec0b5df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49552/0 (socket says 172.21.15.27:49552) 2023-12-19T18:15:06.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.602+0000 7f8eebfff640 1 -- 172.21.15.27:0/65245229 learned_addr learned my addr 172.21.15.27:0/65245229 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.603+0000 7f8eebfff640 1 -- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8eec161740 msgr2=0x7f8eec0c8160 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-19T18:15:06.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.603+0000 7f8eebfff640 1 --2- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8eec161740 0x7f8eec0c8160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.603+0000 7f8eebfff640 1 -- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8eec1652d0 msgr2=0x7f8eec0b4b50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-19T18:15:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.603+0000 7f8eebfff640 1 --2- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8eec1652d0 0x7f8eec0b4b50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.603+0000 7f8eebfff640 1 -- 172.21.15.27:0/65245229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8edc0094d0 con 0x7f8eec0cd5f0 2023-12-19T18:15:06.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.603+0000 7fc06d565640 1 -- 172.21.15.27:0/1839360131 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064031870 con 0x7fc0680b8c30 2023-12-19T18:15:06.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.603+0000 7fc06d565640 1 -- 172.21.15.27:0/1839360131 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0640319d0 con 0x7fc0680b8c30 2023-12-19T18:15:06.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.603+0000 7fc06d565640 1 -- 172.21.15.27:0/1839360131 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc064031da0 con 0x7fc0680b8c30 2023-12-19T18:15:06.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.603+0000 7fcb8e7fc640 1 -- 172.21.15.27:0/1036956010 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88011830 con 0x7fcb90164a80 2023-12-19T18:15:06.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.601+0000 7f09aeffd640 1 -- 172.21.15.27:0/3691694587 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09a4007d90 con 0x7f09b80cdd00 2023-12-19T18:15:06.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.603+0000 7f8eebfff640 1 --2- 172.21.15.27:0/65245229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8eec0cd5f0 0x7f8eec0b5df0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000c7f0 tx=0x7f8ee000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.603+0000 7fcb8e7fc640 1 -- 172.21.15.27:0/1036956010 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb880119b0 con 0x7fcb90164a80 2023-12-19T18:15:06.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.603+0000 7fcb8e7fc640 1 -- 172.21.15.27:0/1036956010 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb88011d60 con 0x7fcb90164a80 2023-12-19T18:15:06.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.603+0000 7fb7c37fe640 1 -- 172.21.15.27:0/2683452048 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb7d003b070 con 0x7fb7dc0cb730 2023-12-19T18:15:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.604+0000 7f09aeffd640 1 -- 172.21.15.27:0/3691694587 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f09a401a450 con 0x7f09b80cdd00 2023-12-19T18:15:06.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.604+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb900c5500 con 0x7fcb90164a80 2023-12-19T18:15:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.604+0000 7f8ee8ff9640 1 -- 172.21.15.27:0/65245229 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0004220 con 0x7f8eec0cd5f0 2023-12-19T18:15:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.604+0000 7f8ee8ff9640 1 -- 172.21.15.27:0/65245229 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee0004380 con 0x7f8eec0cd5f0 2023-12-19T18:15:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.604+0000 7f8ee8ff9640 1 -- 172.21.15.27:0/65245229 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0007860 con 0x7f8eec0cd5f0 2023-12-19T18:15:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.604+0000 7f8ef239c640 1 -- 172.21.15.27:0/65245229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eec15b810 con 0x7f8eec0cd5f0 2023-12-19T18:15:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.604+0000 7fc0707cb640 1 -- 172.21.15.27:0/1839360131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0680b8c30 msgr2=0x7fc0680d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.604+0000 7fc0707cb640 1 --2- 172.21.15.27:0/1839360131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0680b8c30 0x7fc0680d1fc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc064009870 tx=0x7fc06402b910 comp rx=0 tx=0).stop 2023-12-19T18:15:06.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.604+0000 7f8ef239c640 1 -- 172.21.15.27:0/65245229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eec15bdd0 con 0x7f8eec0cd5f0 2023-12-19T18:15:06.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.605+0000 7fc0707cb640 1 -- 172.21.15.27:0/1839360131 shutdown_connections 2023-12-19T18:15:06.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.605+0000 7fc0707cb640 1 --2- 172.21.15.27:0/1839360131 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0680ce130 0x7fc0680cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.605+0000 7fc0707cb640 1 --2- 172.21.15.27:0/1839360131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0680b8c30 0x7fc0680d1fc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.605+0000 7fc0707cb640 1 -- 172.21.15.27:0/1839360131 >> 172.21.15.27:0/1839360131 conn(0x7fc0680d2fa0 msgr2=0x7fc0680d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.605+0000 7fc0707cb640 1 -- 172.21.15.27:0/1839360131 shutdown_connections 2023-12-19T18:15:06.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.602+0000 7f9b6a49e640 1 -- 172.21.15.27:0/1468957322 wait complete. 2023-12-19T18:15:06.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.605+0000 7f09aeffd640 1 --2- 172.21.15.27:0/3691694587 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f09a006a430 0x7f09a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.605+0000 7f09aeffd640 1 -- 172.21.15.27:0/3691694587 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09a4088a30 con 0x7f09b80cdd00 2023-12-19T18:15:06.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.606+0000 7f09aeffd640 1 --2- 172.21.15.27:0/3691694587 >> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] conn(0x7f09a00725d0 0x7f09a00749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.606+0000 7f09aeffd640 1 -- 172.21.15.27:0/3691694587 --> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09a00750a0 con 0x7f09a00725d0 2023-12-19T18:15:06.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.605+0000 7fb7c37fe640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb7c806a530 0x7fb7c806c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.605+0000 7fb7c37fe640 1 -- 172.21.15.27:0/2683452048 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb7d00aa810 con 0x7fb7dc0cb730 2023-12-19T18:15:06.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.606+0000 7fb7c37fe640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] conn(0x7fb7c80726d0 0x7fb7c8074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.606+0000 7fb7c37fe640 1 -- 172.21.15.27:0/2683452048 --> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb7c80751a0 con 0x7fb7c80726d0 2023-12-19T18:15:06.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.604+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb900c2890 con 0x7fcb90164a80 2023-12-19T18:15:06.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.606+0000 7f8ef239c640 1 -- 172.21.15.27:0/65245229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8ecc000fc0 con 0x7f8eec0cd5f0 2023-12-19T18:15:06.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.605+0000 7fc0707cb640 1 -- 172.21.15.27:0/1839360131 wait complete. 2023-12-19T18:15:06.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.606+0000 7fb7db7fe640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] conn(0x7fb7c80726d0 0x7fb7c8074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.606+0000 7fb7daffd640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb7c806a530 0x7fb7c806c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.606+0000 7f09b6d9d640 1 --2- 172.21.15.27:0/3691694587 >> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] conn(0x7f09a00725d0 0x7f09a00749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.607+0000 7f09b5d9b640 1 --2- 172.21.15.27:0/3691694587 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f09a006a430 0x7f09a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.607+0000 7fb7c37fe640 1 -- 172.21.15.27:0/2683452048 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb7d0075f20 con 0x7fb7dc0cb730 2023-12-19T18:15:06.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.607+0000 7fb7db7fe640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] conn(0x7fb7c80726d0 0x7fb7c8074af0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.607+0000 7f6d1cdcb640 1 Processor -- start 2023-12-19T18:15:06.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.607+0000 7f6d1cdcb640 1 -- start start 2023-12-19T18:15:06.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.607+0000 7f9b6a49e640 1 Processor -- start 2023-12-19T18:15:06.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.608+0000 7fc0707cb640 1 Processor -- start 2023-12-19T18:15:06.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.608+0000 7fb7daffd640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb7c806a530 0x7fb7c806c9f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc15b3a0 tx=0x7fb7cc007e10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.608+0000 7fe7517fa640 1 -- 172.21.15.27:0/2911925405 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe744003b90 con 0x7fe75415d240 2023-12-19T18:15:06.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.608+0000 7fe7517fa640 1 -- 172.21.15.27:0/2911925405 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe74400f040 con 0x7fe75415d240 2023-12-19T18:15:06.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.608+0000 7fe7517fa640 1 -- 172.21.15.27:0/2911925405 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe74400de80 con 0x7fe75415d240 2023-12-19T18:15:06.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.608+0000 7fe75ac66640 1 -- 172.21.15.27:0/2911925405 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe754153d10 con 0x7fe75415d240 2023-12-19T18:15:06.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.608+0000 7fe75ac66640 1 -- 172.21.15.27:0/2911925405 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7541594a0 con 0x7fe75415d240 2023-12-19T18:15:06.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.608+0000 7fc0707cb640 1 -- start start 2023-12-19T18:15:06.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.608+0000 7f09b6d9d640 1 --2- 172.21.15.27:0/3691694587 >> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] conn(0x7f09a00725d0 0x7f09a00749f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.609+0000 7fe75ac66640 1 -- 172.21.15.27:0/2911925405 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe75406ab60 con 0x7fe75415d240 2023-12-19T18:15:06.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.609+0000 7fb7c37fe640 1 -- 172.21.15.27:0/2683452048 <== osd.2 v2:172.21.15.27:6820/84138 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fb7c80751a0 con 0x7fb7c80726d0 2023-12-19T18:15:06.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.609+0000 7f6d1cdcb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6d180b8c30 0x7f6d180d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.609+0000 7f6d1cdcb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d180cf9b0 0x7f6d180cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.610+0000 7f6d1cdcb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d180d0350 con 0x7f6d180cda20 2023-12-19T18:15:06.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.610+0000 7f6d1cdcb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d180cea80 con 0x7f6d180b8c30 2023-12-19T18:15:06.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.610+0000 7f6d1cdcb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d180b5f10 con 0x7f6d180cf9b0 2023-12-19T18:15:06.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.610+0000 7f09b5d9b640 1 --2- 172.21.15.27:0/3691694587 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f09a006a430 0x7f09a006c8f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f09b80b2160 tx=0x7f09a8009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.610+0000 7f09aeffd640 1 -- 172.21.15.27:0/3691694587 <== osd.0 v2:172.21.15.27:6808/84136 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f09a00750a0 con 0x7f09a00725d0 2023-12-19T18:15:06.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.610+0000 7fc0707cb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0680b8c30 0x7fc0680ca2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.610+0000 7fc0707cb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0680ce130 0x7fc0681556b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.610+0000 7fc0707cb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0680cf9b0 0x7fc068155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.610+0000 7fc0707cb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06815a140 con 0x7fc0680ce130 2023-12-19T18:15:06.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.610+0000 7fc0707cb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc068159bc0 con 0x7fc0680cf9b0 2023-12-19T18:15:06.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.610+0000 7fc0707cb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc068157470 con 0x7fc0680b8c30 2023-12-19T18:15:06.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.611+0000 7fc06dd66640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0680ce130 0x7fc0681556b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.610+0000 7f6d15d9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d180cf9b0 0x7f6d180cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.611+0000 7f6d1659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6d180b8c30 0x7f6d180d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.611+0000 7f6d1659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6d180b8c30 0x7f6d180d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49568/0 (socket says 172.21.15.27:49568) 2023-12-19T18:15:06.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.611+0000 7f6d1659c640 1 -- 172.21.15.27:0/1486812654 learned_addr learned my addr 172.21.15.27:0/1486812654 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.611+0000 7fc06dd66640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0680ce130 0x7fc0681556b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49580/0 (socket says 172.21.15.27:49580) 2023-12-19T18:15:06.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.611+0000 7fc06dd66640 1 -- 172.21.15.27:0/3948103621 learned_addr learned my addr 172.21.15.27:0/3948103621 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.611+0000 7fc06ed68640 1 --2- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0680cf9b0 0x7fc068155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.611+0000 7f9b6a49e640 1 -- start start 2023-12-19T18:15:06.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.610+0000 7fcb8e7fc640 1 -- 172.21.15.27:0/1036956010 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcb8801a400 con 0x7fcb90164a80 2023-12-19T18:15:06.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.611+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcb900ae1b0 con 0x7fcb90164a80 2023-12-19T18:15:06.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.612+0000 7fcb8e7fc640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb7406a430 0x7fcb7406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.612+0000 7fc06dd66640 1 -- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0680b8c30 msgr2=0x7fc0680ca2a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-19T18:15:06.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.612+0000 7fc06dd66640 1 --2- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0680b8c30 0x7fc0680ca2a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.612+0000 7fc06dd66640 1 -- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0680cf9b0 msgr2=0x7fc068155bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.612+0000 7fc06dd66640 1 --2- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0680cf9b0 0x7fc068155bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.612+0000 7fc06dd66640 1 -- 172.21.15.27:0/3948103621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0640094d0 con 0x7fc0680ce130 2023-12-19T18:15:06.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.612+0000 7fc06dd66640 1 --2- 172.21.15.27:0/3948103621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0680ce130 0x7fc0681556b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c400 tx=0x7fc05800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.612+0000 7f8ee8ff9640 1 -- 172.21.15.27:0/65245229 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8ee00079c0 con 0x7f8eec0cd5f0 2023-12-19T18:15:06.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.612+0000 7f9b6a49e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b50004db0 0x7f9b5c0e49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.612+0000 7f9b6a49e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b5c0e5e30 0x7f9b5c0dc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.612+0000 7f9b6a49e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b5c17e280 0x7f9b5c038400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.612+0000 7f6d1659c640 1 -- 172.21.15.27:0/1486812654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d180cf9b0 msgr2=0x7f6d180cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7f9b6a49e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0e5a60 con 0x7f9b50004db0 2023-12-19T18:15:06.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7f9b6a49e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0e4f20 con 0x7f9b5c17e280 2023-12-19T18:15:06.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7f9b6a49e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b5c0e51c0 con 0x7f9b5c0e5e30 2023-12-19T18:15:06.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7fcb95fdd640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb7406a430 0x7fcb7406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7fcb8e7fc640 1 -- 172.21.15.27:0/1036956010 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb88056d10 con 0x7fcb90164a80 2023-12-19T18:15:06.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7fc05effd640 1 -- 172.21.15.27:0/3948103621 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0580052d0 con 0x7fc0680ce130 2023-12-19T18:15:06.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7fc05effd640 1 -- 172.21.15.27:0/3948103621 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc058005430 con 0x7fc0680ce130 2023-12-19T18:15:06.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7fc05effd640 1 -- 172.21.15.27:0/3948103621 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058003830 con 0x7fc0680ce130 2023-12-19T18:15:06.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7fcb8e7fc640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] conn(0x7fcb740725d0 0x7fcb740749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7fcb8e7fc640 1 -- 172.21.15.27:0/1036956010 --> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcb740750a0 con 0x7fcb740725d0 2023-12-19T18:15:06.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7fcb8e7fc640 1 -- 172.21.15.27:0/1036956010 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcb880090e0 con 0x7fcb90164a80 2023-12-19T18:15:06.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7f9b68c9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b5c0e5e30 0x7f9b5c0dc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7f9b69c9d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b5c17e280 0x7f9b5c038400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7f9b68c9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b5c0e5e30 0x7f9b5c0dc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:42222/0 (socket says 172.21.15.27:42222) 2023-12-19T18:15:06.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7f9b68c9b640 1 -- 172.21.15.27:0/3693707521 learned_addr learned my addr 172.21.15.27:0/3693707521 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7fc0707cb640 1 -- 172.21.15.27:0/3948103621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc068157190 con 0x7fc0680ce130 2023-12-19T18:15:06.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.614+0000 7f9b6949c640 1 --2- 172.21.15.27:0/3693707521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b50004db0 0x7f9b5c0e49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.614+0000 7f9b68c9b640 1 -- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b5c17e280 msgr2=0x7f9b5c038400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.614+0000 7f9b68c9b640 1 --2- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b5c17e280 0x7f9b5c038400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.614+0000 7f9b68c9b640 1 -- 172.21.15.27:0/3693707521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b50004db0 msgr2=0x7f9b5c0e49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.614+0000 7f9b68c9b640 1 --2- 172.21.15.27:0/3693707521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b50004db0 0x7f9b5c0e49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.614+0000 7f9b68c9b640 1 -- 172.21.15.27:0/3693707521 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b5c0e54a0 con 0x7f9b5c0e5e30 2023-12-19T18:15:06.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.614+0000 7fc0707cb640 1 -- 172.21.15.27:0/3948103621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc068156db0 con 0x7fc0680ce130 2023-12-19T18:15:06.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.614+0000 7fcb957dc640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] conn(0x7fcb740725d0 0x7fcb740749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.615+0000 7f8ee8ff9640 1 --2- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ed806a430 0x7f8ed806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.615+0000 7f8ee8ff9640 1 -- 172.21.15.27:0/65245229 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ee0088840 con 0x7f8eec0cd5f0 2023-12-19T18:15:06.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7f6d1659c640 1 --2- 172.21.15.27:0/1486812654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d180cf9b0 0x7f6d180cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7f6d1659c640 1 -- 172.21.15.27:0/1486812654 >> v1:172.21.15.27:6789/0 conn(0x7f6d180cda20 legacy=0x7f6d180cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:06.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.613+0000 7f6d1659c640 1 -- 172.21.15.27:0/1486812654 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d180ce7a0 con 0x7f6d180b8c30 2023-12-19T18:15:06.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.615+0000 7f8ee8ff9640 1 --2- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] conn(0x7f8ed8072650 0x7f8ed8074a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.615+0000 7f8ee8ff9640 1 -- 172.21.15.27:0/65245229 --> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8ed8075120 con 0x7f8ed8072650 2023-12-19T18:15:06.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.615+0000 7f8ee8ff9640 1 -- 172.21.15.27:0/65245229 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ee008d2a0 con 0x7f8eec0cd5f0 2023-12-19T18:15:06.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.615+0000 7f9b68c9b640 1 --2- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b5c0e5e30 0x7f9b5c0dc0b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b540108e0 tx=0x7f9b54010da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.615+0000 7f6d15d9b640 1 --2- 172.21.15.27:0/1486812654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d180cf9b0 0x7f6d180cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:06.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.616+0000 7fc0707cb640 1 -- 172.21.15.27:0/3948103621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc0680d17c0 con 0x7fc0680ce130 2023-12-19T18:15:06.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.616+0000 7f6d1659c640 1 --2- 172.21.15.27:0/1486812654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d180b8c30 0x7f6d180d1fc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009e50 tx=0x7f6d1002be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f9fbe8b62afdf82 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.616+0000 7f8eeb7fe640 1 --2- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ed806a430 0x7f8ed806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.616+0000 7fcb957dc640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] conn(0x7fcb740725d0 0x7fcb740749f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.616+0000 7fcb95fdd640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb7406a430 0x7fcb7406c8f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb8402be40 tx=0x7fcb84030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.616+0000 7f6d1559a640 1 -- 172.21.15.27:0/1486812654 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1002ca10 con 0x7f6d180b8c30 2023-12-19T18:15:06.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.616+0000 7f6d1559a640 1 -- 172.21.15.27:0/1486812654 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1002cb70 con 0x7f6d180b8c30 2023-12-19T18:15:06.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.616+0000 7f6d1559a640 1 -- 172.21.15.27:0/1486812654 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1002ca10 con 0x7f6d180b8c30 2023-12-19T18:15:06.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.617+0000 7f09aeffd640 1 -- 172.21.15.27:0/3691694587 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09a4053fb0 con 0x7f09b80cdd00 2023-12-19T18:15:06.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.617+0000 7f8ef0939640 1 --2- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] conn(0x7f8ed8072650 0x7f8ed8074a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.617+0000 7f9b59ffb640 1 -- 172.21.15.27:0/3693707521 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5400eb60 con 0x7f9b5c0e5e30 2023-12-19T18:15:06.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.617+0000 7f9b59ffb640 1 -- 172.21.15.27:0/3693707521 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b5400ecc0 con 0x7f9b5c0e5e30 2023-12-19T18:15:06.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.617+0000 7f9b59ffb640 1 -- 172.21.15.27:0/3693707521 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b5401c400 con 0x7f9b5c0e5e30 2023-12-19T18:15:06.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.617+0000 7fc05effd640 1 -- 172.21.15.27:0/3948103621 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc058003990 con 0x7fc0680ce130 2023-12-19T18:15:06.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.617+0000 7fcb8e7fc640 1 -- 172.21.15.27:0/1036956010 <== osd.3 v2:172.21.15.27:6816/84146 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fcb740750a0 con 0x7fcb740725d0 2023-12-19T18:15:06.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.618+0000 7f6d1cdcb640 1 -- 172.21.15.27:0/1486812654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d180b8c30 msgr2=0x7f6d180d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.618+0000 7f6d1cdcb640 1 --2- 172.21.15.27:0/1486812654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d180b8c30 0x7f6d180d1fc0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009e50 tx=0x7f6d1002be70 comp rx=0 tx=0).stop 2023-12-19T18:15:06.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.617+0000 7f9b6a49e640 1 -- 172.21.15.27:0/3693707521 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b5c0e5780 con 0x7f9b5c0e5e30 2023-12-19T18:15:06.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.618+0000 7f9b6a49e640 1 -- 172.21.15.27:0/3693707521 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b5c0dc5f0 con 0x7f9b5c0e5e30 2023-12-19T18:15:06.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.618+0000 7f8eeb7fe640 1 --2- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ed806a430 0x7f8ed806c8f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8edc036000 tx=0x7f8edc008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.618+0000 7f8ef0939640 1 --2- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] conn(0x7f8ed8072650 0x7f8ed8074a70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.619+0000 7f6d1cdcb640 1 -- 172.21.15.27:0/1486812654 shutdown_connections 2023-12-19T18:15:06.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.619+0000 7f6d1cdcb640 1 --2- 172.21.15.27:0/1486812654 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d180cf9b0 0x7f6d180cfe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.619+0000 7f6d1cdcb640 1 --2- 172.21.15.27:0/1486812654 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d180b8c30 0x7f6d180d1fc0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.619+0000 7f6d1cdcb640 1 -- 172.21.15.27:0/1486812654 >> 172.21.15.27:0/1486812654 conn(0x7f6d180d2fa0 msgr2=0x7f6d180d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.619+0000 7fc05effd640 1 --2- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc04c06a480 0x7fc04c06c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.619+0000 7f6d1cdcb640 1 -- 172.21.15.27:0/1486812654 shutdown_connections 2023-12-19T18:15:06.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.619+0000 7fc05effd640 1 -- 172.21.15.27:0/3948103621 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc058087b80 con 0x7fc0680ce130 2023-12-19T18:15:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.619+0000 7f6d1cdcb640 1 -- 172.21.15.27:0/1486812654 wait complete. 2023-12-19T18:15:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.619+0000 7fc05effd640 1 --2- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] conn(0x7fc04c0726a0 0x7fc04c074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.619+0000 7fc05effd640 1 -- 172.21.15.27:0/3948103621 --> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc04c075170 con 0x7fc04c0726a0 2023-12-19T18:15:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.619+0000 7fc05effd640 1 -- 172.21.15.27:0/3948103621 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc05808a290 con 0x7fc0680ce130 2023-12-19T18:15:06.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.619+0000 7fc06e567640 1 --2- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc04c06a480 0x7fc04c06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.620+0000 7f9b6a49e640 1 -- 172.21.15.27:0/3693707521 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9b5c050060 con 0x7f9b5c0e5e30 2023-12-19T18:15:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.620+0000 7fc06ed68640 1 --2- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] conn(0x7fc04c0726a0 0x7fc04c074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.620+0000 7f8ee8ff9640 1 -- 172.21.15.27:0/65245229 <== osd.5 v2:172.21.15.192:6808/84206 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f8ed8075120 con 0x7f8ed8072650 2023-12-19T18:15:06.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.621+0000 7fe7517fa640 1 -- 172.21.15.27:0/2911925405 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe744004030 con 0x7fe75415d240 2023-12-19T18:15:06.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.621+0000 7fc06e567640 1 --2- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc04c06a480 0x7fc04c06c940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc064036000 tx=0x7fc064008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.621+0000 7fc06ed68640 1 --2- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] conn(0x7fc04c0726a0 0x7fc04c074ac0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.622+0000 7f9b59ffb640 1 -- 172.21.15.27:0/3693707521 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9b5401c560 con 0x7f9b5c0e5e30 2023-12-19T18:15:06.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.622+0000 7f6d1cdcb640 1 Processor -- start 2023-12-19T18:15:06.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.622+0000 7fe7517fa640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe73406a480 0x7fe73406c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.623+0000 7fe7517fa640 1 -- 172.21.15.27:0/2911925405 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe744089230 con 0x7fe75415d240 2023-12-19T18:15:06.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.623+0000 7fc05effd640 1 -- 172.21.15.27:0/3948103621 <== osd.6 v2:172.21.15.192:6824/84255 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fc04c075170 con 0x7fc04c0726a0 2023-12-19T18:15:06.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.623+0000 7fe7517fa640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] conn(0x7fe7340726a0 0x7fe734074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.624+0000 7f9b59ffb640 1 --2- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9b4806a500 0x7f9b4806c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.624+0000 7f9b59ffb640 1 -- 172.21.15.27:0/3693707521 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b54088fe0 con 0x7f9b5c0e5e30 2023-12-19T18:15:06.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.622+0000 7f6d1cdcb640 1 -- start start 2023-12-19T18:15:06.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.624+0000 7f9b59ffb640 1 --2- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] conn(0x7f9b480726a0 0x7f9b48074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.624+0000 7f9b59ffb640 1 -- 172.21.15.27:0/3693707521 --> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9b48075170 con 0x7f9b480726a0 2023-12-19T18:15:06.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.623+0000 7fe758a02640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe73406a480 0x7fe73406c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.624+0000 7fe7517fa640 1 -- 172.21.15.27:0/2911925405 --> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe734075170 con 0x7fe7340726a0 2023-12-19T18:15:06.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.624+0000 7fe7517fa640 1 -- 172.21.15.27:0/2911925405 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7440896d0 con 0x7fe75415d240 2023-12-19T18:15:06.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.624+0000 7f9b6949c640 1 --2- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9b4806a500 0x7f9b4806c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.625+0000 7f6d1cdcb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d180b8c30 0x7f6d1806dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.625+0000 7fe753fff640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] conn(0x7fe7340726a0 0x7fe734074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.625+0000 7fe758a02640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe73406a480 0x7fe73406c940 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe74c02ba50 tx=0x7fe74c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.625+0000 7f6d1cdcb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d180cda20 0x7f6d1806e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.625+0000 7f9b69c9d640 1 --2- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] conn(0x7f9b480726a0 0x7f9b48074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.626+0000 7f9b59ffb640 1 -- 172.21.15.27:0/3693707521 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9b54054640 con 0x7f9b5c0e5e30 2023-12-19T18:15:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.625+0000 7f6d1659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d180b8c30 0x7f6d1806dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.626+0000 7f6d1cdcb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d180cf9b0 0x7f6d1806f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.626+0000 7f6d1cdcb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d180b4f50 con 0x7f6d180cda20 2023-12-19T18:15:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.626+0000 7f6d1cdcb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d180b5230 con 0x7f6d180b8c30 2023-12-19T18:15:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.626+0000 7f6d1cdcb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1806d000 con 0x7f6d180cf9b0 2023-12-19T18:15:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.626+0000 7f6d1659c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d180b8c30 0x7f6d1806dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:50482/0 (socket says 172.21.15.27:50482) 2023-12-19T18:15:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.626+0000 7f6d1659c640 1 -- 172.21.15.27:0/2534213687 learned_addr learned my addr 172.21.15.27:0/2534213687 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.626+0000 7f6d15d9b640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d180cda20 0x7f6d1806e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.627+0000 7f9b69c9d640 1 --2- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] conn(0x7f9b480726a0 0x7f9b48074ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.626+0000 7fe753fff640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] conn(0x7fe7340726a0 0x7fe734074ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.627+0000 7fe7517fa640 1 -- 172.21.15.27:0/2911925405 <== osd.1 v2:172.21.15.27:6800/84133 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fe734075170 con 0x7fe7340726a0 2023-12-19T18:15:06.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.627+0000 7f6d16d9d640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d180cf9b0 0x7f6d1806f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.627+0000 7f9b6949c640 1 --2- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9b4806a500 0x7f9b4806c9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c038190 tx=0x7f9b60008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.627+0000 7f6d1659c640 1 -- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d180cf9b0 msgr2=0x7f6d1806f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.627+0000 7f6d1659c640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6d180cf9b0 0x7f6d1806f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.627+0000 7f6d1659c640 1 -- 172.21.15.27:0/2534213687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d180cda20 msgr2=0x7f6d1806e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.627+0000 7f6d1659c640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d180cda20 0x7f6d1806e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.627+0000 7f6d1659c640 1 -- 172.21.15.27:0/2534213687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d10009b30 con 0x7f6d180b8c30 2023-12-19T18:15:06.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.628+0000 7f6d1659c640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d180b8c30 0x7f6d1806dae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009e50 tx=0x7f6d10004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.629+0000 7f6d0effd640 1 -- 172.21.15.27:0/2534213687 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10003ea0 con 0x7f6d180b8c30 2023-12-19T18:15:06.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.629+0000 7f6d0effd640 1 -- 172.21.15.27:0/2534213687 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d10009510 con 0x7f6d180b8c30 2023-12-19T18:15:06.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.629+0000 7f6d0effd640 1 -- 172.21.15.27:0/2534213687 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10004880 con 0x7f6d180b8c30 2023-12-19T18:15:06.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.629+0000 7f6d1cdcb640 1 -- 172.21.15.27:0/2534213687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1806d2e0 con 0x7f6d180b8c30 2023-12-19T18:15:06.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.629+0000 7f6d1cdcb640 1 -- 172.21.15.27:0/2534213687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1806f750 con 0x7f6d180b8c30 2023-12-19T18:15:06.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.629+0000 7f9b59ffb640 1 -- 172.21.15.27:0/3693707521 <== osd.4 v2:172.21.15.192:6800/84205 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f9b48075170 con 0x7f9b480726a0 2023-12-19T18:15:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.631+0000 7f6d0cff9640 1 -- 172.21.15.27:0/2534213687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6d180b50c0 con 0x7f6d180b8c30 2023-12-19T18:15:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.633+0000 7f6d0effd640 1 -- 172.21.15.27:0/2534213687 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6d1003e450 con 0x7f6d180b8c30 2023-12-19T18:15:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.635+0000 7f6d0effd640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6cf806a430 0x7f6cf806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.635+0000 7f6d0effd640 1 -- 172.21.15.27:0/2534213687 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d100abac0 con 0x7f6d180b8c30 2023-12-19T18:15:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.635+0000 7f6d0effd640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] conn(0x7f6cf80725d0 0x7f6cf80749f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.635+0000 7f6d0effd640 1 -- 172.21.15.27:0/2534213687 --> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6cf80750a0 con 0x7f6cf80725d0 2023-12-19T18:15:06.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.635+0000 7f6d0effd640 1 -- 172.21.15.27:0/2534213687 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d100abe30 con 0x7f6d180b8c30 2023-12-19T18:15:06.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.636+0000 7f6d15d9b640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6cf806a430 0x7f6cf806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.636+0000 7f6d16d9d640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] conn(0x7f6cf80725d0 0x7f6cf80749f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:06.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.638+0000 7f6d15d9b640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6cf806a430 0x7f6cf806c8f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d18160400 tx=0x7f6d00008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.638+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3691694587 --> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f09b80d0d30 con 0x7f09a00725d0 2023-12-19T18:15:06.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.638+0000 7f6d16d9d640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] conn(0x7f6cf80725d0 0x7f6cf80749f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:06.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.639+0000 7f09aeffd640 1 -- 172.21.15.27:0/3691694587 <== osd.0 v2:172.21.15.27:6808/84136 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f09b002c9a0 con 0x7f09a00725d0 2023-12-19T18:15:06.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.639+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3691694587 >> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] conn(0x7f09a00725d0 msgr2=0x7f09a00749f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.639+0000 7f09bcaa0640 1 --2- 172.21.15.27:0/3691694587 >> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] conn(0x7f09a00725d0 0x7f09a00749f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.640+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3691694587 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f09a006a430 msgr2=0x7f09a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.640+0000 7f09bcaa0640 1 --2- 172.21.15.27:0/3691694587 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f09a006a430 0x7f09a006c8f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f09b80b2160 tx=0x7f09a8009060 comp rx=0 tx=0).stop 2023-12-19T18:15:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.640+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3691694587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09b80cdd00 msgr2=0x7f09b80b7b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.640+0000 7f09bcaa0640 1 --2- 172.21.15.27:0/3691694587 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09b80cdd00 0x7f09b80b7b30 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f09a400c820 tx=0x7f09a400cc20 comp rx=0 tx=0).stop 2023-12-19T18:15:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.640+0000 7f09b6d9d640 1 -- 172.21.15.27:0/3691694587 reap_dead start 2023-12-19T18:15:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.640+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3691694587 shutdown_connections 2023-12-19T18:15:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.640+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3691694587 >> 172.21.15.27:0/3691694587 conn(0x7f09b80d2fa0 msgr2=0x7f09b815ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.640+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3691694587 shutdown_connections 2023-12-19T18:15:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.640+0000 7f09bcaa0640 1 -- 172.21.15.27:0/3691694587 wait complete. 2023-12-19T18:15:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.641+0000 7f6d0effd640 1 -- 172.21.15.27:0/2534213687 <== osd.7 v2:172.21.15.192:6816/84207 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f6cf80750a0 con 0x7f6cf80725d0 2023-12-19T18:15:06.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.642+0000 7fb7dbfff640 1 -- 172.21.15.27:0/2683452048 --> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb7dc0d1e20 con 0x7fb7c80726d0 2023-12-19T18:15:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.642+0000 7fb7c37fe640 1 -- 172.21.15.27:0/2683452048 <== osd.2 v2:172.21.15.27:6820/84138 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb7d4001d80 con 0x7fb7c80726d0 2023-12-19T18:15:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.644+0000 7fb7dbfff640 1 -- 172.21.15.27:0/2683452048 >> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] conn(0x7fb7c80726d0 msgr2=0x7fb7c8074af0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.644+0000 7fb7dbfff640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] conn(0x7fb7c80726d0 0x7fb7c8074af0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.644+0000 7fb7dbfff640 1 -- 172.21.15.27:0/2683452048 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb7c806a530 msgr2=0x7fb7c806c9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.644+0000 7fb7dbfff640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb7c806a530 0x7fb7c806c9f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb7dc15b3a0 tx=0x7fb7cc007e10 comp rx=0 tx=0).stop 2023-12-19T18:15:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.644+0000 7fb7dbfff640 1 -- 172.21.15.27:0/2683452048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7dc0cb730 msgr2=0x7fb7dc0cac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.644+0000 7fb7dbfff640 1 --2- 172.21.15.27:0/2683452048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7dc0cb730 0x7fb7dc0cac30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4014b30 tx=0x7fb7d0004730 comp rx=0 tx=0).stop 2023-12-19T18:15:06.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.644+0000 7fb7db7fe640 1 -- 172.21.15.27:0/2683452048 reap_dead start 2023-12-19T18:15:06.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.644+0000 7fb7dbfff640 1 -- 172.21.15.27:0/2683452048 shutdown_connections 2023-12-19T18:15:06.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.644+0000 7fb7dbfff640 1 -- 172.21.15.27:0/2683452048 >> 172.21.15.27:0/2683452048 conn(0x7fb7dc0cf820 msgr2=0x7fb7dc158010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.645+0000 7fb7dbfff640 1 -- 172.21.15.27:0/2683452048 shutdown_connections 2023-12-19T18:15:06.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.645+0000 7fb7dbfff640 1 -- 172.21.15.27:0/2683452048 wait complete. 2023-12-19T18:15:06.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.646+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 --> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcb900cb120 con 0x7fcb740725d0 2023-12-19T18:15:06.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.647+0000 7fcb8e7fc640 1 -- 172.21.15.27:0/1036956010 <== osd.3 v2:172.21.15.27:6816/84146 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcb7c0072a0 con 0x7fcb740725d0 2023-12-19T18:15:06.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.647+0000 7f8ef239c640 1 -- 172.21.15.27:0/65245229 --> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8ecc002cb0 con 0x7f8ed8072650 2023-12-19T18:15:06.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.648+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 >> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] conn(0x7fcb740725d0 msgr2=0x7fcb740749f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.648+0000 7fcb98241640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] conn(0x7fcb740725d0 0x7fcb740749f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.648+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb7406a430 msgr2=0x7fcb7406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.648+0000 7fcb98241640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb7406a430 0x7fcb7406c8f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb8402be40 tx=0x7fcb84030000 comp rx=0 tx=0).stop 2023-12-19T18:15:06.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.648+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb90164a80 msgr2=0x7fcb90163a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.648+0000 7fcb98241640 1 --2- 172.21.15.27:0/1036956010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb90164a80 0x7fcb90163a90 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcb900717b0 tx=0x7fcb8800b870 comp rx=0 tx=0).stop 2023-12-19T18:15:06.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.648+0000 7fcb967de640 1 -- 172.21.15.27:0/1036956010 reap_dead start 2023-12-19T18:15:06.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.649+0000 7f8ee8ff9640 1 -- 172.21.15.27:0/65245229 <== osd.5 v2:172.21.15.192:6808/84206 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8ecc002cb0 con 0x7f8ed8072650 2023-12-19T18:15:06.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.649+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 shutdown_connections 2023-12-19T18:15:06.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.649+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 >> 172.21.15.27:0/1036956010 conn(0x7fcb9006a330 msgr2=0x7fcb90070e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.649+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 shutdown_connections 2023-12-19T18:15:06.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.649+0000 7fcb98241640 1 -- 172.21.15.27:0/1036956010 wait complete. 2023-12-19T18:15:06.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.649+0000 7f8ef239c640 1 -- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] conn(0x7f8ed8072650 msgr2=0x7f8ed8074a70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.649+0000 7f8ef239c640 1 --2- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] conn(0x7f8ed8072650 0x7f8ed8074a70 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.650+0000 7f8ef239c640 1 -- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ed806a430 msgr2=0x7f8ed806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.650+0000 7f8ef239c640 1 --2- 172.21.15.27:0/65245229 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ed806a430 0x7f8ed806c8f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f8edc036000 tx=0x7f8edc008030 comp rx=0 tx=0).stop 2023-12-19T18:15:06.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.650+0000 7f8ef239c640 1 -- 172.21.15.27:0/65245229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8eec0cd5f0 msgr2=0x7f8eec0b5df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.650+0000 7f8ef239c640 1 --2- 172.21.15.27:0/65245229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8eec0cd5f0 0x7f8eec0b5df0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f8ee000c7f0 tx=0x7f8ee000ccb0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.650+0000 7f8ef0939640 1 -- 172.21.15.27:0/65245229 reap_dead start 2023-12-19T18:15:06.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.650+0000 7f8ef239c640 1 -- 172.21.15.27:0/65245229 shutdown_connections 2023-12-19T18:15:06.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.650+0000 7f8ef239c640 1 -- 172.21.15.27:0/65245229 >> 172.21.15.27:0/65245229 conn(0x7f8eec0d2fa0 msgr2=0x7f8eec163ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.650+0000 7f8ef239c640 1 -- 172.21.15.27:0/65245229 shutdown_connections 2023-12-19T18:15:06.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.650+0000 7f8ef239c640 1 -- 172.21.15.27:0/65245229 wait complete. 2023-12-19T18:15:06.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.653+0000 7fc0707cb640 1 -- 172.21.15.27:0/3948103621 --> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc0680d14e0 con 0x7fc04c0726a0 2023-12-19T18:15:06.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.655+0000 7fe75ac66640 1 -- 172.21.15.27:0/2911925405 --> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe75406a880 con 0x7fe7340726a0 2023-12-19T18:15:06.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.655+0000 7fe7517fa640 1 -- 172.21.15.27:0/2911925405 <== osd.1 v2:172.21.15.27:6800/84133 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe734075170 con 0x7fe7340726a0 2023-12-19T18:15:06.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.656+0000 7fc05effd640 1 -- 172.21.15.27:0/3948103621 <== osd.6 v2:172.21.15.192:6824/84255 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc06000f040 con 0x7fc04c0726a0 2023-12-19T18:15:06.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.656+0000 7fe75ac66640 1 -- 172.21.15.27:0/2911925405 >> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] conn(0x7fe7340726a0 msgr2=0x7fe734074ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.656+0000 7fe75ac66640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] conn(0x7fe7340726a0 0x7fe734074ac0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.656+0000 7fe75ac66640 1 -- 172.21.15.27:0/2911925405 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe73406a480 msgr2=0x7fe73406c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.656+0000 7fe75ac66640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe73406a480 0x7fe73406c940 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe74c02ba50 tx=0x7fe74c008030 comp rx=0 tx=0).stop 2023-12-19T18:15:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.657+0000 7fe75ac66640 1 -- 172.21.15.27:0/2911925405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe75415d240 msgr2=0x7fe754156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.657+0000 7fe75ac66640 1 --2- 172.21.15.27:0/2911925405 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe75415d240 0x7fe754156a20 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe744007e60 tx=0x7fe74400d4a0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.657+0000 7fe759203640 1 -- 172.21.15.27:0/2911925405 reap_dead start 2023-12-19T18:15:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.656+0000 7fc0707cb640 1 -- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] conn(0x7fc04c0726a0 msgr2=0x7fc04c074ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.656+0000 7fc0707cb640 1 --2- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] conn(0x7fc04c0726a0 0x7fc04c074ac0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.657+0000 7fc0707cb640 1 -- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc04c06a480 msgr2=0x7fc04c06c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.657+0000 7fc0707cb640 1 --2- 172.21.15.27:0/3948103621 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc04c06a480 0x7fc04c06c940 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc064036000 tx=0x7fc064008030 comp rx=0 tx=0).stop 2023-12-19T18:15:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.657+0000 7fc0707cb640 1 -- 172.21.15.27:0/3948103621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0680ce130 msgr2=0x7fc0681556b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.657+0000 7fc0707cb640 1 --2- 172.21.15.27:0/3948103621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0680ce130 0x7fc0681556b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fc05800c400 tx=0x7fc05800c8c0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.657+0000 7fc06ed68640 1 -- 172.21.15.27:0/3948103621 reap_dead start 2023-12-19T18:15:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.657+0000 7fc0707cb640 1 -- 172.21.15.27:0/3948103621 shutdown_connections 2023-12-19T18:15:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.657+0000 7fc0707cb640 1 -- 172.21.15.27:0/3948103621 >> 172.21.15.27:0/3948103621 conn(0x7fc0680d2fa0 msgr2=0x7fc0680d1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.658+0000 7fe75ac66640 1 -- 172.21.15.27:0/2911925405 shutdown_connections 2023-12-19T18:15:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.658+0000 7fc0707cb640 1 -- 172.21.15.27:0/3948103621 shutdown_connections 2023-12-19T18:15:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.658+0000 7fc0707cb640 1 -- 172.21.15.27:0/3948103621 wait complete. 2023-12-19T18:15:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.658+0000 7fe75ac66640 1 -- 172.21.15.27:0/2911925405 >> 172.21.15.27:0/2911925405 conn(0x7fe7540d31e0 msgr2=0x7fe7540b76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.659+0000 7fe75ac66640 1 -- 172.21.15.27:0/2911925405 shutdown_connections 2023-12-19T18:15:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.659+0000 7fe75ac66640 1 -- 172.21.15.27:0/2911925405 wait complete. 2023-12-19T18:15:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.663+0000 7f9b6a49e640 1 -- 172.21.15.27:0/3693707521 --> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9b5c04fd80 con 0x7f9b480726a0 2023-12-19T18:15:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.665+0000 7f9b59ffb640 1 -- 172.21.15.27:0/3693707521 <== osd.4 v2:172.21.15.192:6800/84205 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9b6404e840 con 0x7f9b480726a0 2023-12-19T18:15:06.686 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:06.687 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-19T18:15:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.665+0000 7f9b6a49e640 1 -- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] conn(0x7f9b480726a0 msgr2=0x7f9b48074ac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.666+0000 7f9b6a49e640 1 --2- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] conn(0x7f9b480726a0 0x7f9b48074ac0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.666+0000 7f9b6a49e640 1 -- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9b4806a500 msgr2=0x7f9b4806c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.666+0000 7f9b6a49e640 1 --2- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9b4806a500 0x7f9b4806c9c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c038190 tx=0x7f9b60008040 comp rx=0 tx=0).stop 2023-12-19T18:15:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.666+0000 7f9b6a49e640 1 -- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b5c0e5e30 msgr2=0x7f9b5c0dc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.666+0000 7f9b6a49e640 1 --2- 172.21.15.27:0/3693707521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b5c0e5e30 0x7f9b5c0dc0b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b540108e0 tx=0x7f9b54010da0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.666+0000 7f9b69c9d640 1 -- 172.21.15.27:0/3693707521 reap_dead start 2023-12-19T18:15:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.666+0000 7f9b6a49e640 1 -- 172.21.15.27:0/3693707521 shutdown_connections 2023-12-19T18:15:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.666+0000 7f9b6a49e640 1 -- 172.21.15.27:0/3693707521 >> 172.21.15.27:0/3693707521 conn(0x7f9b5c054c10 msgr2=0x7f9b5c04ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.667+0000 7f9b6a49e640 1 -- 172.21.15.27:0/3693707521 shutdown_connections 2023-12-19T18:15:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.667+0000 7f9b6a49e640 1 -- 172.21.15.27:0/3693707521 wait complete. 2023-12-19T18:15:06.690 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:06.690 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-19T18:15:06.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.672+0000 7f6d0cff9640 1 -- 172.21.15.27:0/2534213687 --> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6d180ce580 con 0x7f6cf80725d0 2023-12-19T18:15:06.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.673+0000 7f6d0effd640 1 -- 172.21.15.27:0/2534213687 <== osd.7 v2:172.21.15.192:6816/84207 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6d08005040 con 0x7f6cf80725d0 2023-12-19T18:15:06.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.675+0000 7f6d0cff9640 1 -- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] conn(0x7f6cf80725d0 msgr2=0x7f6cf80749f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.676+0000 7f6d0cff9640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] conn(0x7f6cf80725d0 0x7f6cf80749f0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.676+0000 7f6d0cff9640 1 -- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6cf806a430 msgr2=0x7f6cf806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.676+0000 7f6d0cff9640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6cf806a430 0x7f6cf806c8f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6d18160400 tx=0x7f6d00008040 comp rx=0 tx=0).stop 2023-12-19T18:15:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.676+0000 7f6d0cff9640 1 -- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d180b8c30 msgr2=0x7f6d1806dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.676+0000 7f6d0cff9640 1 --2- 172.21.15.27:0/2534213687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6d180b8c30 0x7f6d1806dae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6d10009e50 tx=0x7f6d10004a40 comp rx=0 tx=0).stop 2023-12-19T18:15:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.676+0000 7f6d16d9d640 1 -- 172.21.15.27:0/2534213687 reap_dead start 2023-12-19T18:15:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.677+0000 7f6d0cff9640 1 -- 172.21.15.27:0/2534213687 shutdown_connections 2023-12-19T18:15:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.677+0000 7f6d0cff9640 1 -- 172.21.15.27:0/2534213687 >> 172.21.15.27:0/2534213687 conn(0x7f6d180d2fa0 msgr2=0x7f6d180d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.677+0000 7f6d0cff9640 1 -- 172.21.15.27:0/2534213687 shutdown_connections 2023-12-19T18:15:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:06.677+0000 7f6d0cff9640 1 -- 172.21.15.27:0/2534213687 wait complete. 2023-12-19T18:15:06.693 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:06.693 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-19T18:15:06.694 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2023-12-19T18:15:06.694 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-19T18:15:06.695 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:06.695 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-19T18:15:06.696 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2023-12-19T18:15:06.696 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-19T18:15:06.697 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:06.697 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-19T18:15:06.698 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:06.698 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-19T18:15:07.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.044+0000 7fa89a896640 1 Processor -- start 2023-12-19T18:15:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.045+0000 7fa89a896640 1 -- start start 2023-12-19T18:15:07.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.047+0000 7fa89a896640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa88c0e11d0 0x7fa88c038780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.048+0000 7fa89a896640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa88c0e15b0 0x7fa88c03aff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.048+0000 7fa89a896640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa88c055480 con 0x7fa88c0542a0 2023-12-19T18:15:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.048+0000 7fa89a896640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88c056920 con 0x7fa88c0e11d0 2023-12-19T18:15:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.048+0000 7fa89a896640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa88c055290 con 0x7fa88c0e15b0 2023-12-19T18:15:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.048+0000 7fa893fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa88c0e11d0 0x7fa88c038780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.049+0000 7fa893fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa88c0e11d0 0x7fa88c038780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49610/0 (socket says 172.21.15.27:49610) 2023-12-19T18:15:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.049+0000 7fa893fff640 1 -- 172.21.15.27:0/203725758 learned_addr learned my addr 172.21.15.27:0/203725758 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.049+0000 7fa898e33640 1 --2- 172.21.15.27:0/203725758 >> v2:172.21.15.192:3300/0 conn(0x7fa88c0e15b0 0x7fa88c03aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.050+0000 7fa893fff640 1 -- 172.21.15.27:0/203725758 >> v2:172.21.15.192:3300/0 conn(0x7fa88c0e15b0 msgr2=0x7fa88c03aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:07.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.051+0000 7fa892ffd640 1 -- 172.21.15.27:0/203725758 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091705161 0 0) 0x7fa88c055480 con 0x7fa88c0542a0 2023-12-19T18:15:07.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.051+0000 7fa893fff640 1 --2- 172.21.15.27:0/203725758 >> v2:172.21.15.192:3300/0 conn(0x7fa88c0e15b0 0x7fa88c03aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.051+0000 7fa893fff640 1 -- 172.21.15.27:0/203725758 >> v1:172.21.15.27:6789/0 conn(0x7fa88c0542a0 legacy=0x7fa88c051680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.051+0000 7fa893fff640 1 -- 172.21.15.27:0/203725758 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c056540 con 0x7fa88c0e11d0 2023-12-19T18:15:07.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.053+0000 7fa898e33640 1 --2- 172.21.15.27:0/203725758 >> v2:172.21.15.192:3300/0 conn(0x7fa88c0e15b0 0x7fa88c03aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:07.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.054+0000 7fa893fff640 1 --2- 172.21.15.27:0/203725758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c0e11d0 0x7fa88c038780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0043b0 tx=0x7fa88802b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c832c21e5804d483 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.054+0000 7febf4dfd640 1 Processor -- start 2023-12-19T18:15:07.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.054+0000 7fa892ffd640 1 -- 172.21.15.27:0/203725758 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88802f810 con 0x7fa88c0e11d0 2023-12-19T18:15:07.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.054+0000 7febf4dfd640 1 -- start start 2023-12-19T18:15:07.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.055+0000 7fa892ffd640 1 -- 172.21.15.27:0/203725758 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88802f970 con 0x7fa88c0e11d0 2023-12-19T18:15:07.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.055+0000 7fa892ffd640 1 -- 172.21.15.27:0/203725758 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88802f810 con 0x7fa88c0e11d0 2023-12-19T18:15:07.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.056+0000 7fa89a896640 1 -- 172.21.15.27:0/203725758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c0e11d0 msgr2=0x7fa88c038780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.056+0000 7fa89a896640 1 --2- 172.21.15.27:0/203725758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c0e11d0 0x7fa88c038780 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fa88c0043b0 tx=0x7fa88802b9f0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.056+0000 7febf4dfd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7febe8050f00 0x7febe80eae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.056+0000 7febf4dfd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe804fe70 con 0x7febe8050300 2023-12-19T18:15:07.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.056+0000 7febf4dfd640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe80e0380 con 0x7febe80dffa0 2023-12-19T18:15:07.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.056+0000 7febf4dfd640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe804f260 con 0x7febe8050f00 2023-12-19T18:15:07.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.057+0000 7febeeffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7febe8050300 0x7febe80507f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54016/0 (socket says 172.21.15.27:54016) 2023-12-19T18:15:07.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.057+0000 7febeeffd640 1 -- 172.21.15.27:0/1513338609 learned_addr learned my addr 172.21.15.27:0/1513338609 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:07.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.057+0000 7febeffff640 1 --2- 172.21.15.27:0/1513338609 >> v2:172.21.15.192:3300/0 conn(0x7febe8050f00 0x7febe80eae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.058+0000 7febee7fc640 1 -- 172.21.15.27:0/1513338609 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 490384508 0 0) 0x7febe804fe70 con 0x7febe8050300 2023-12-19T18:15:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.058+0000 7febee7fc640 1 -- 172.21.15.27:0/1513338609 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7febe0005730 con 0x7febe8050300 2023-12-19T18:15:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.058+0000 7fa89a896640 1 -- 172.21.15.27:0/203725758 shutdown_connections 2023-12-19T18:15:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.058+0000 7fa89a896640 1 --2- 172.21.15.27:0/203725758 >> v2:172.21.15.192:3300/0 conn(0x7fa88c0e15b0 0x7fa88c03aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.058+0000 7fa89a896640 1 --2- 172.21.15.27:0/203725758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c0e11d0 0x7fa88c038780 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.058+0000 7fa89a896640 1 -- 172.21.15.27:0/203725758 >> 172.21.15.27:0/203725758 conn(0x7fa88c053780 msgr2=0x7fa88c053b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.058+0000 7fa89a896640 1 -- 172.21.15.27:0/203725758 shutdown_connections 2023-12-19T18:15:07.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.058+0000 7fa89a896640 1 -- 172.21.15.27:0/203725758 wait complete. 2023-12-19T18:15:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.058+0000 7febeffff640 1 -- 172.21.15.27:0/1513338609 >> v1:172.21.15.192:6789/0 conn(0x7febe80dffa0 legacy=0x7febe8039970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.058+0000 7febeffff640 1 -- 172.21.15.27:0/1513338609 >> v1:172.21.15.27:6789/0 conn(0x7febe8050300 legacy=0x7febe80507f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.058+0000 7febeffff640 1 -- 172.21.15.27:0/1513338609 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe804f540 con 0x7febe8050f00 2023-12-19T18:15:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.060+0000 7febeffff640 1 --2- 172.21.15.27:0/1513338609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febe8050f00 0x7febe80eae40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7febf0067c00 tx=0x7febf00a5fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d5023899b5a5406 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.061+0000 7fa89a896640 1 Processor -- start 2023-12-19T18:15:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.061+0000 7fa89a896640 1 -- start start 2023-12-19T18:15:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.061+0000 7febee7fc640 1 -- 172.21.15.27:0/1513338609 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf00a6b40 con 0x7febe8050f00 2023-12-19T18:15:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.061+0000 7f0d56054640 1 Processor -- start 2023-12-19T18:15:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.061+0000 7febee7fc640 1 -- 172.21.15.27:0/1513338609 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf00a6ca0 con 0x7febe8050f00 2023-12-19T18:15:07.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.062+0000 7febee7fc640 1 -- 172.21.15.27:0/1513338609 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf00ae680 con 0x7febe8050f00 2023-12-19T18:15:07.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.061+0000 7f0d56054640 1 -- start start 2023-12-19T18:15:07.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7fa89a896640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa88c0e11d0 0x7fa88c169920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7febf4dfd640 1 -- 172.21.15.27:0/1513338609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febe8050f00 msgr2=0x7febe80eae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7febf4dfd640 1 --2- 172.21.15.27:0/1513338609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febe8050f00 0x7febe80eae40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7febf0067c00 tx=0x7febf00a5fa0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7febf4dfd640 1 -- 172.21.15.27:0/1513338609 shutdown_connections 2023-12-19T18:15:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7febf4dfd640 1 --2- 172.21.15.27:0/1513338609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febe8050f00 0x7febe80eae40 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7febf4dfd640 1 -- 172.21.15.27:0/1513338609 >> 172.21.15.27:0/1513338609 conn(0x7febe8054c10 msgr2=0x7febe8054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.064+0000 7febf4dfd640 1 -- 172.21.15.27:0/1513338609 shutdown_connections 2023-12-19T18:15:07.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.064+0000 7febf4dfd640 1 -- 172.21.15.27:0/1513338609 wait complete. 2023-12-19T18:15:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.064+0000 7f0d56054640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0d48033e40 0x7f0d480546d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.064+0000 7f0d56054640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0d48006870 0x7f0d48006ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.064+0000 7f0d56054640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d480e3420 con 0x7f0d4803a6b0 2023-12-19T18:15:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.064+0000 7f0d56054640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d48007220 con 0x7f0d48006870 2023-12-19T18:15:07.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.064+0000 7f0d56054640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d48050240 con 0x7f0d48033e40 2023-12-19T18:15:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7fa89a896640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa88c0e15b0 0x7fa88c039fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7fa89a896640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c169e60 0x7fa88c03a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7fa89a896640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0cf950 con 0x7fa88c169e60 2023-12-19T18:15:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7fa89a896640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0cfc30 con 0x7fa88c0e15b0 2023-12-19T18:15:07.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7fa89a896640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa88c0cff10 con 0x7fa88c0e11d0 2023-12-19T18:15:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7fa898e33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c169e60 0x7fa88c03a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7fa8937fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa88c0e15b0 0x7fa88c039fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7fa898e33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c169e60 0x7fa88c03a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49622/0 (socket says 172.21.15.27:49622) 2023-12-19T18:15:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.063+0000 7fa898e33640 1 -- 172.21.15.27:0/1077621747 learned_addr learned my addr 172.21.15.27:0/1077621747 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.065+0000 7fa8937fe640 1 -- 172.21.15.27:0/1077621747 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa88c0e11d0 msgr2=0x7fa88c169920 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-19T18:15:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.065+0000 7fa8937fe640 1 --2- 172.21.15.27:0/1077621747 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa88c0e11d0 0x7fa88c169920 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.065+0000 7fa8937fe640 1 -- 172.21.15.27:0/1077621747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c169e60 msgr2=0x7fa88c03a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.065+0000 7fa8937fe640 1 --2- 172.21.15.27:0/1077621747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c169e60 0x7fa88c03a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.065+0000 7fa8937fe640 1 -- 172.21.15.27:0/1077621747 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8880094d0 con 0x7fa88c0e15b0 2023-12-19T18:15:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.065+0000 7f0d54851640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0d48033e40 0x7f0d480546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.065+0000 7f0d54851640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0d48033e40 0x7f0d480546d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:50520/0 (socket says 172.21.15.27:50520) 2023-12-19T18:15:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.065+0000 7f0d54851640 1 -- 172.21.15.27:0/1474225115 learned_addr learned my addr 172.21.15.27:0/1474225115 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.065+0000 7f0d55853640 1 --2- 172.21.15.27:0/1474225115 >> v2:172.21.15.27:3300/0 conn(0x7f0d48006870 0x7f0d48006ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.066+0000 7f0d54851640 1 -- 172.21.15.27:0/1474225115 >> v2:172.21.15.27:3300/0 conn(0x7f0d48006870 msgr2=0x7f0d48006ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.066+0000 7f0d47fff640 1 -- 172.21.15.27:0/1474225115 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2168719436 0 0) 0x7f0d480e3420 con 0x7f0d4803a6b0 2023-12-19T18:15:07.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.066+0000 7f0d54851640 1 --2- 172.21.15.27:0/1474225115 >> v2:172.21.15.27:3300/0 conn(0x7f0d48006870 0x7f0d48006ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.066+0000 7f0d54851640 1 -- 172.21.15.27:0/1474225115 >> v1:172.21.15.192:6789/0 conn(0x7f0d4803a6b0 legacy=0x7f0d48038670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.066+0000 7f0d54851640 1 -- 172.21.15.27:0/1474225115 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d48050520 con 0x7f0d48033e40 2023-12-19T18:15:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.066+0000 7fa8937fe640 1 --2- 172.21.15.27:0/1077621747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa88c0e15b0 0x7fa88c039fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa88000c400 tx=0x7fa88000c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.066+0000 7fa890ff9640 1 -- 172.21.15.27:0/1077621747 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8800035f0 con 0x7fa88c0e15b0 2023-12-19T18:15:07.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.066+0000 7f0d55853640 1 --2- 172.21.15.27:0/1474225115 >> v2:172.21.15.27:3300/0 conn(0x7f0d48006870 0x7f0d48006ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:15:07.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.067+0000 7fa89a896640 1 -- 172.21.15.27:0/1077621747 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa88c0d01f0 con 0x7fa88c0e15b0 2023-12-19T18:15:07.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.067+0000 7fa89a896640 1 -- 172.21.15.27:0/1077621747 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa88c051a60 con 0x7fa88c0e15b0 2023-12-19T18:15:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.067+0000 7fa890ff9640 1 -- 172.21.15.27:0/1077621747 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa880003750 con 0x7fa88c0e15b0 2023-12-19T18:15:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.067+0000 7fa890ff9640 1 -- 172.21.15.27:0/1077621747 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8800035f0 con 0x7fa88c0e15b0 2023-12-19T18:15:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.068+0000 7f0d54851640 1 --2- 172.21.15.27:0/1474225115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d48033e40 0x7f0d480546d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800dc70 tx=0x7f0d4002dd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de8cd032e341bc83 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.069+0000 7f0d47fff640 1 -- 172.21.15.27:0/1474225115 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4002eac0 con 0x7f0d48033e40 2023-12-19T18:15:07.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.069+0000 7fa89a896640 1 -- 172.21.15.27:0/1077621747 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa874005250 con 0x7fa88c0e15b0 2023-12-19T18:15:07.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.069+0000 7f0d47fff640 1 -- 172.21.15.27:0/1474225115 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4002ec20 con 0x7f0d48033e40 2023-12-19T18:15:07.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.069+0000 7f0d47fff640 1 -- 172.21.15.27:0/1474225115 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d40036440 con 0x7f0d48033e40 2023-12-19T18:15:07.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.067+0000 7febf4dfd640 1 Processor -- start 2023-12-19T18:15:07.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.068+0000 7febf4dfd640 1 -- start start 2023-12-19T18:15:07.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.070+0000 7f0d56054640 1 -- 172.21.15.27:0/1474225115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d48033e40 msgr2=0x7f0d480546d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.070+0000 7f0d56054640 1 --2- 172.21.15.27:0/1474225115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d48033e40 0x7f0d480546d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0d4800dc70 tx=0x7f0d4002dd30 comp rx=0 tx=0).stop 2023-12-19T18:15:07.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.071+0000 7febf4dfd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe8050300 0x7febe80458c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.071+0000 7f0d56054640 1 -- 172.21.15.27:0/1474225115 shutdown_connections 2023-12-19T18:15:07.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.071+0000 7f0d56054640 1 --2- 172.21.15.27:0/1474225115 >> v2:172.21.15.27:3300/0 conn(0x7f0d48006870 0x7f0d48006ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.071+0000 7f0d56054640 1 --2- 172.21.15.27:0/1474225115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d48033e40 0x7f0d480546d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.071+0000 7f0d56054640 1 -- 172.21.15.27:0/1474225115 >> 172.21.15.27:0/1474225115 conn(0x7f0d48054c10 msgr2=0x7f0d48054ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.071+0000 7febf4dfd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febe8050f00 0x7febe804a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.071+0000 7febf4dfd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7febe80dffa0 0x7febe80db430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.071+0000 7febf4dfd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe804a4f0 con 0x7febe8050300 2023-12-19T18:15:07.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.071+0000 7febf4dfd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe804a170 con 0x7febe8050f00 2023-12-19T18:15:07.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.071+0000 7febf4dfd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe8047810 con 0x7febe80dffa0 2023-12-19T18:15:07.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.071+0000 7f0d56054640 1 -- 172.21.15.27:0/1474225115 shutdown_connections 2023-12-19T18:15:07.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.072+0000 7febeeffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febe8050f00 0x7febe804a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.072+0000 7febeeffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febe8050f00 0x7febe804a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:50536/0 (socket says 172.21.15.27:50536) 2023-12-19T18:15:07.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.072+0000 7febeeffd640 1 -- 172.21.15.27:0/4245172404 learned_addr learned my addr 172.21.15.27:0/4245172404 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:07.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.072+0000 7febeffff640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7febe80dffa0 0x7febe80db430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.071+0000 7f0d56054640 1 -- 172.21.15.27:0/1474225115 wait complete. 2023-12-19T18:15:07.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.072+0000 7fa890ff9640 1 -- 172.21.15.27:0/1077621747 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa8800038c0 con 0x7fa88c0e15b0 2023-12-19T18:15:07.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.074+0000 7f0d56054640 1 Processor -- start 2023-12-19T18:15:07.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.074+0000 7ff1a180b640 1 Processor -- start 2023-12-19T18:15:07.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.074+0000 7f0d56054640 1 -- start start 2023-12-19T18:15:07.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.075+0000 7fa890ff9640 1 --2- 172.21.15.27:0/1077621747 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa87806a490 0x7fa87806c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.073+0000 7febeeffd640 1 -- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7febe80dffa0 msgr2=0x7febe80db430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.073+0000 7febeeffd640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7febe80dffa0 0x7febe80db430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.073+0000 7febef7fe640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe8050300 0x7febe80458c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.073+0000 7febeeffd640 1 -- 172.21.15.27:0/4245172404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe8050300 msgr2=0x7febe80458c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.073+0000 7febeeffd640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe8050300 0x7febe80458c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.073+0000 7febeeffd640 1 -- 172.21.15.27:0/4245172404 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe8047ab0 con 0x7febe8050f00 2023-12-19T18:15:07.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.073+0000 7febef7fe640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe8050300 0x7febe80458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:07.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.074+0000 7febeeffd640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febe8050f00 0x7febe804a910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c490 tx=0x7febdc00c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.075+0000 7ff1a180b640 1 -- start start 2023-12-19T18:15:07.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.075+0000 7fa890ff9640 1 -- 172.21.15.27:0/1077621747 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa880087990 con 0x7fa88c0e15b0 2023-12-19T18:15:07.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.075+0000 7fa893fff640 1 --2- 172.21.15.27:0/1077621747 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa87806a490 0x7fa87806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.076+0000 7f0d56054640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d48006870 0x7f0d4816ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.076+0000 7f0d56054640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d48033e40 0x7f0d48048710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.076+0000 7f0d56054640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4803a6b0 0x7f0d48034960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.076+0000 7f0d56054640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d48048c50 con 0x7f0d4803a6b0 2023-12-19T18:15:07.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.076+0000 7f0d56054640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d48169b70 con 0x7f0d48033e40 2023-12-19T18:15:07.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.076+0000 7f0d56054640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d48169e50 con 0x7f0d48006870 2023-12-19T18:15:07.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.076+0000 7f0d55853640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4803a6b0 0x7f0d48034960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.076+0000 7f0d55853640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4803a6b0 0x7f0d48034960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49648/0 (socket says 172.21.15.27:49648) 2023-12-19T18:15:07.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.076+0000 7f0d55853640 1 -- 172.21.15.27:0/1017806278 learned_addr learned my addr 172.21.15.27:0/1017806278 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:07.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.077+0000 7f0d54851640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d48033e40 0x7f0d48048710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.077+0000 7f0d55052640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d48006870 0x7f0d4816ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.077+0000 7ff1a180b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff19c15d260 0x7ff19c06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.077+0000 7ff1a180b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 0x7ff19c06a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.077+0000 7ff1a180b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff19c0b71b0 con 0x7ff19c0d0e70 2023-12-19T18:15:07.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.077+0000 7ff1a180b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff19c0cdaf0 con 0x7ff19c15d260 2023-12-19T18:15:07.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.077+0000 7ff1a180b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0d1a50 con 0x7ff19c0d2af0 2023-12-19T18:15:07.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.077+0000 7f690f607640 1 Processor -- start 2023-12-19T18:15:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.078+0000 7ff19affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff19c15d260 0x7ff19c06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.078+0000 7f6c120bd640 1 Processor -- start 2023-12-19T18:15:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.078+0000 7f690f607640 1 -- start start 2023-12-19T18:15:07.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.078+0000 7fa893fff640 1 --2- 172.21.15.27:0/1077621747 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa87806a490 0x7fa87806c950 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa888036000 tx=0x7fa888008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.078+0000 7ff193fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 0x7ff19c06a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.078+0000 7ff193fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 0x7ff19c06a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:42258/0 (socket says 172.21.15.27:42258) 2023-12-19T18:15:07.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.078+0000 7ff193fff640 1 -- 172.21.15.27:0/959706992 learned_addr learned my addr 172.21.15.27:0/959706992 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:07.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.079+0000 7ff193fff640 1 -- 172.21.15.27:0/959706992 >> v2:172.21.15.27:3300/0 conn(0x7ff19c15d260 msgr2=0x7ff19c06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:07.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.079+0000 7ff193fff640 1 --2- 172.21.15.27:0/959706992 >> v2:172.21.15.27:3300/0 conn(0x7ff19c15d260 0x7ff19c06a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.079+0000 7ff193fff640 1 -- 172.21.15.27:0/959706992 >> v1:172.21.15.192:6789/0 conn(0x7ff19c0d0e70 legacy=0x7ff19c0b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.079+0000 7ff193fff640 1 -- 172.21.15.27:0/959706992 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c0d1d30 con 0x7ff19c0d2af0 2023-12-19T18:15:07.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.077+0000 7f0d55853640 1 -- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d48006870 msgr2=0x7f0d4816ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.077+0000 7f0d55853640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d48006870 0x7f0d4816ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.077+0000 7f0d55853640 1 -- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d48033e40 msgr2=0x7f0d48048710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.077+0000 7f0d55853640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d48033e40 0x7f0d48048710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.078+0000 7f0d55853640 1 -- 172.21.15.27:0/1017806278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d400094d0 con 0x7f0d4803a6b0 2023-12-19T18:15:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.078+0000 7f0d55853640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4803a6b0 0x7f0d48034960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d5004f7d0 tx=0x7f0d50072460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.079+0000 7f0d45ffb640 1 -- 172.21.15.27:0/1017806278 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50092430 con 0x7f0d4803a6b0 2023-12-19T18:15:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.079+0000 7f0d45ffb640 1 -- 172.21.15.27:0/1017806278 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d50092590 con 0x7f0d4803a6b0 2023-12-19T18:15:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.079+0000 7f0d45ffb640 1 -- 172.21.15.27:0/1017806278 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d50092920 con 0x7f0d4803a6b0 2023-12-19T18:15:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.079+0000 7f0d56054640 1 -- 172.21.15.27:0/1017806278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4816a130 con 0x7f0d4803a6b0 2023-12-19T18:15:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.079+0000 7f0d56054640 1 -- 172.21.15.27:0/1017806278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4816a6f0 con 0x7f0d4803a6b0 2023-12-19T18:15:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.078+0000 7f6c120bd640 1 -- start start 2023-12-19T18:15:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.080+0000 7f6c120bd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6c0c15d280 0x7f6c0c0b8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.080+0000 7f6c120bd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c0c0ce470 0x7f6c0c0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.080+0000 7f6c120bd640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0c15d660 con 0x7f6c0c0d1360 2023-12-19T18:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.080+0000 7f6c120bd640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c0c0d1bb0 con 0x7f6c0c15d280 2023-12-19T18:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.080+0000 7f6c120bd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c0d1e90 con 0x7f6c0c0ce470 2023-12-19T18:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.079+0000 7f690f607640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f690806fc10 0x7f690806d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.079+0000 7f690f607640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6908070c00 0x7f69080712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.080+0000 7f690f607640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6908071820 con 0x7f690815adf0 2023-12-19T18:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.080+0000 7f690f607640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69080703c0 con 0x7f6908070c00 2023-12-19T18:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.080+0000 7f690f607640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f690806d8c0 con 0x7f690806fc10 2023-12-19T18:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.080+0000 7f690d3a3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f690815adf0 0x7f690806f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54022/0 (socket says 172.21.15.27:54022) 2023-12-19T18:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.080+0000 7f690dba4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6908070c00 0x7f69080712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.080+0000 7f690d3a3640 1 -- 172.21.15.27:0/225149041 learned_addr learned my addr 172.21.15.27:0/225149041 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.080+0000 7f690cba2640 1 --2- 172.21.15.27:0/225149041 >> v2:172.21.15.192:3300/0 conn(0x7f690806fc10 0x7f690806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.081+0000 7f68fffff640 1 -- 172.21.15.27:0/225149041 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3209928742 0 0) 0x7f6908071820 con 0x7f690815adf0 2023-12-19T18:15:07.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.081+0000 7f0d56054640 1 -- 172.21.15.27:0/1017806278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d480d63c0 con 0x7f0d4803a6b0 2023-12-19T18:15:07.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.081+0000 7f68fffff640 1 -- 172.21.15.27:0/225149041 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68f4003520 con 0x7f690815adf0 2023-12-19T18:15:07.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.081+0000 7f6c0b7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6c0c15d280 0x7f6c0c0b8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.081+0000 7f6c02ffd640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f6c0c0d1360 0x7f6c0c0cdd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:36806/0 (socket says 172.21.15.27:36806) 2023-12-19T18:15:07.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.081+0000 7f690dba4640 1 -- 172.21.15.27:0/225149041 >> v2:172.21.15.192:3300/0 conn(0x7f690806fc10 msgr2=0x7f690806d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:07.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.081+0000 7f690dba4640 1 --2- 172.21.15.27:0/225149041 >> v2:172.21.15.192:3300/0 conn(0x7f690806fc10 0x7f690806d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.081+0000 7f68fffff640 1 -- 172.21.15.27:0/225149041 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2655302286 0 0) 0x7f68f4003520 con 0x7f690815adf0 2023-12-19T18:15:07.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.081+0000 7f6c02ffd640 1 -- 172.21.15.27:0/1683798337 learned_addr learned my addr 172.21.15.27:0/1683798337 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:07.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.080+0000 7ff19affd640 1 --2- 172.21.15.27:0/959706992 >> v2:172.21.15.27:3300/0 conn(0x7ff19c15d260 0x7ff19c06a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:15:07.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.081+0000 7f6c0bfff640 1 --2- 172.21.15.27:0/1683798337 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c0c0ce470 0x7f6c0c0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.081+0000 7f690dba4640 1 -- 172.21.15.27:0/225149041 >> v1:172.21.15.27:6789/0 conn(0x7f690815adf0 legacy=0x7f690806f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.082+0000 7ff193fff640 1 --2- 172.21.15.27:0/959706992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 0x7ff19c06a930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff184009870 tx=0x7ff18402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d7f3e41a3a424fd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.081+0000 7f690dba4640 1 -- 172.21.15.27:0/225149041 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f690806b520 con 0x7f6908070c00 2023-12-19T18:15:07.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.081+0000 7f690cba2640 1 --2- 172.21.15.27:0/225149041 >> v2:172.21.15.192:3300/0 conn(0x7f690806fc10 0x7f690806d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:07.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.082+0000 7ff19a7fc640 1 -- 172.21.15.27:0/959706992 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff184030840 con 0x7ff19c0d2af0 2023-12-19T18:15:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.082+0000 7ff19a7fc640 1 -- 172.21.15.27:0/959706992 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1840309a0 con 0x7ff19c0d2af0 2023-12-19T18:15:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.082+0000 7ff19a7fc640 1 -- 172.21.15.27:0/959706992 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff184030d30 con 0x7ff19c0d2af0 2023-12-19T18:15:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.082+0000 7f6c0bfff640 1 -- 172.21.15.27:0/1683798337 >> v2:172.21.15.192:3300/0 conn(0x7f6c0c15d280 msgr2=0x7f6c0c0b8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.083+0000 7f0d45ffb640 1 -- 172.21.15.27:0/1017806278 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0d50061020 con 0x7f0d4803a6b0 2023-12-19T18:15:07.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.082+0000 7f6c0bfff640 1 --2- 172.21.15.27:0/1683798337 >> v2:172.21.15.192:3300/0 conn(0x7f6c0c15d280 0x7f6c0c0b8b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.082+0000 7f6c0bfff640 1 -- 172.21.15.27:0/1683798337 >> v1:172.21.15.192:6789/0 conn(0x7f6c0c0d1360 legacy=0x7f6c0c0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.082+0000 7f6c0bfff640 1 -- 172.21.15.27:0/1683798337 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c0c0d2170 con 0x7f6c0c0ce470 2023-12-19T18:15:07.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.083+0000 7ff1a180b640 1 -- 172.21.15.27:0/959706992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 msgr2=0x7ff19c06a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.083+0000 7ff1a180b640 1 --2- 172.21.15.27:0/959706992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 0x7ff19c06a930 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7ff184009870 tx=0x7ff18402b980 comp rx=0 tx=0).stop 2023-12-19T18:15:07.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.083+0000 7ff1a180b640 1 -- 172.21.15.27:0/959706992 shutdown_connections 2023-12-19T18:15:07.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.083+0000 7ff1a180b640 1 --2- 172.21.15.27:0/959706992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 0x7ff19c06a930 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.083+0000 7ff1a180b640 1 --2- 172.21.15.27:0/959706992 >> v2:172.21.15.27:3300/0 conn(0x7ff19c15d260 0x7ff19c06a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.083+0000 7ff1a180b640 1 -- 172.21.15.27:0/959706992 >> 172.21.15.27:0/959706992 conn(0x7ff19c0d2f70 msgr2=0x7ff19c0d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.083+0000 7f690dba4640 1 --2- 172.21.15.27:0/225149041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6908070c00 0x7f69080712e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6908071230 tx=0x7f690002bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be8058b2be90fa3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.084+0000 7f0d45ffb640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d3406a480 0x7f0d3406c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.084+0000 7f0d45ffb640 1 -- 172.21.15.27:0/1017806278 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d50102f10 con 0x7f0d4803a6b0 2023-12-19T18:15:07.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.084+0000 7f6c0bfff640 1 --2- 172.21.15.27:0/1683798337 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c0c0ce470 0x7f6c0c0b6130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c040097a0 tx=0x7f6c0402b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d1c9cb5ee78cae6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.084+0000 7ff1a180b640 1 -- 172.21.15.27:0/959706992 shutdown_connections 2023-12-19T18:15:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.084+0000 7ff1a180b640 1 -- 172.21.15.27:0/959706992 wait complete. 2023-12-19T18:15:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.085+0000 7f0d55052640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d3406a480 0x7f0d3406c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.085+0000 7f6c0affd640 1 -- 172.21.15.27:0/1683798337 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0402d040 con 0x7f6c0c0ce470 2023-12-19T18:15:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.085+0000 7f6c0affd640 1 -- 172.21.15.27:0/1683798337 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c04031810 con 0x7f6c0c0ce470 2023-12-19T18:15:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.085+0000 7f6c0affd640 1 -- 172.21.15.27:0/1683798337 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04031ba0 con 0x7f6c0c0ce470 2023-12-19T18:15:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.086+0000 7f6c120bd640 1 -- 172.21.15.27:0/1683798337 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c0c0ce470 msgr2=0x7f6c0c0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.086+0000 7f6c120bd640 1 --2- 172.21.15.27:0/1683798337 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c0c0ce470 0x7f6c0c0b6130 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6c040097a0 tx=0x7f6c0402b8b0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.086+0000 7f0d55052640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d3406a480 0x7f0d3406c940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c000f60 tx=0x7f0d4c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.086+0000 7f6c120bd640 1 -- 172.21.15.27:0/1683798337 shutdown_connections 2023-12-19T18:15:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.086+0000 7f6c120bd640 1 --2- 172.21.15.27:0/1683798337 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c0c0ce470 0x7f6c0c0b6130 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.086+0000 7f6c120bd640 1 --2- 172.21.15.27:0/1683798337 >> v2:172.21.15.192:3300/0 conn(0x7f6c0c15d280 0x7f6c0c0b8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.086+0000 7f6c120bd640 1 -- 172.21.15.27:0/1683798337 >> 172.21.15.27:0/1683798337 conn(0x7f6c0c0d2f90 msgr2=0x7f6c0c0d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.086+0000 7f5113473640 1 Processor -- start 2023-12-19T18:15:07.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.086+0000 7f6c120bd640 1 -- 172.21.15.27:0/1683798337 shutdown_connections 2023-12-19T18:15:07.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.086+0000 7f6c120bd640 1 -- 172.21.15.27:0/1683798337 wait complete. 2023-12-19T18:15:07.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.086+0000 7f5113473640 1 -- start start 2023-12-19T18:15:07.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.086+0000 7ff1a180b640 1 Processor -- start 2023-12-19T18:15:07.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.086+0000 7ff1a180b640 1 -- start start 2023-12-19T18:15:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.087+0000 7f9e74e21640 1 Processor -- start 2023-12-19T18:15:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7f6c120bd640 1 Processor -- start 2023-12-19T18:15:07.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7f9e74e21640 1 -- start start 2023-12-19T18:15:07.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7ff1a180b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff19c0d0e70 0x7ff19c1f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7ff1a180b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 0x7ff19c15e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7ff1a180b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff19c15d260 0x7ff19c158bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7ff1a180b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c15f520 con 0x7ff19c0d0e70 2023-12-19T18:15:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7ff1a180b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0c9fb0 con 0x7ff19c15d260 2023-12-19T18:15:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7ff1a180b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff19c0ca850 con 0x7ff19c0d2af0 2023-12-19T18:15:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7ff19affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff19c0d0e70 0x7ff19c1f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7f5113473640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f510c1449e0 0x7f510c06ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7f5113473640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f510c06d340 0x7f510c146b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7f5113473640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f510c165c40 con 0x7f510c069c70 2023-12-19T18:15:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7f5113473640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510c144710 con 0x7f510c1449e0 2023-12-19T18:15:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7f5113473640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c165030 con 0x7f510c06d340 2023-12-19T18:15:07.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7ff193fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 0x7ff19c15e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.088+0000 7f6c120bd640 1 -- start start 2023-12-19T18:15:07.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f6c120bd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c0c0ce470 0x7f6c0c14a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7ff193fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 0x7ff19c15e8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:42284/0 (socket says 172.21.15.27:42284) 2023-12-19T18:15:07.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7ff193fff640 1 -- 172.21.15.27:0/745032848 learned_addr learned my addr 172.21.15.27:0/745032848 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:07.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f5110a0e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f510c1449e0 0x7f510c06ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f6c120bd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c0c0d1360 0x7f6c0c15ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f6c120bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0c15d280 0x7f6c0c15aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f6c120bd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c15eec0 con 0x7f6c0c15d280 2023-12-19T18:15:07.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f5110a0e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f510c1449e0 0x7f510c06ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49690/0 (socket says 172.21.15.27:49690) 2023-12-19T18:15:07.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f6c120bd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c15f160 con 0x7f6c0c0ce470 2023-12-19T18:15:07.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f6c120bd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0c15f440 con 0x7f6c0c0d1360 2023-12-19T18:15:07.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f5110a0e640 1 -- 172.21.15.27:0/581235693 learned_addr learned my addr 172.21.15.27:0/581235693 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:07.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7ff19b7fe640 1 --2- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff19c15d260 0x7ff19c158bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f5111a10640 1 --2- 172.21.15.27:0/581235693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f510c06d340 0x7f510c146b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f5110a0e640 1 -- 172.21.15.27:0/581235693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f510c06d340 msgr2=0x7f510c146b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:07.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f6c0bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0c15d280 0x7f6c0c15aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f6c0b7fe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c0c0ce470 0x7f6c0c14a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.089+0000 7f6c0bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0c15d280 0x7f6c0c15aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49700/0 (socket says 172.21.15.27:49700) 2023-12-19T18:15:07.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f6c0bfff640 1 -- 172.21.15.27:0/252279841 learned_addr learned my addr 172.21.15.27:0/252279841 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:07.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f6c02ffd640 1 --2- 172.21.15.27:0/252279841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c0c0d1360 0x7f6c0c15ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f9e74e21640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e700ce470 0x7f9e70167d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f9e74e21640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e7015d660 con 0x7f9e700b8b50 2023-12-19T18:15:07.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f9e74e21640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e700d1bb0 con 0x7f9e7015d280 2023-12-19T18:15:07.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f9e74e21640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e700b8fe0 con 0x7f9e700ce470 2023-12-19T18:15:07.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7ff193fff640 1 -- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff19c15d260 msgr2=0x7ff19c158bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7ff193fff640 1 --2- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff19c15d260 0x7ff19c158bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7ff193fff640 1 -- 172.21.15.27:0/745032848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff19c0d0e70 msgr2=0x7ff19c1f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7ff193fff640 1 --2- 172.21.15.27:0/745032848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff19c0d0e70 0x7ff19c1f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7ff193fff640 1 -- 172.21.15.27:0/745032848 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff18c009400 con 0x7ff19c0d2af0 2023-12-19T18:15:07.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f6c0bfff640 1 -- 172.21.15.27:0/252279841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c0c0d1360 msgr2=0x7f6c0c15ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f6c0bfff640 1 --2- 172.21.15.27:0/252279841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c0c0d1360 0x7f6c0c15ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f6c0bfff640 1 -- 172.21.15.27:0/252279841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c0c0ce470 msgr2=0x7f6c0c14a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f6c0bfff640 1 --2- 172.21.15.27:0/252279841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c0c0ce470 0x7f6c0c14a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f6c0bfff640 1 -- 172.21.15.27:0/252279841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c04009400 con 0x7f6c0c15d280 2023-12-19T18:15:07.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f5110a0e640 1 --2- 172.21.15.27:0/581235693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f510c06d340 0x7f510c146b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f5110a0e640 1 -- 172.21.15.27:0/581235693 >> v1:172.21.15.192:6789/0 conn(0x7f510c069c70 legacy=0x7f510c06a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:07.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7f5110a0e640 1 -- 172.21.15.27:0/581235693 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c165310 con 0x7f510c1449e0 2023-12-19T18:15:07.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.090+0000 7ff193fff640 1 --2- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 0x7ff19c15e8f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff184000c00 tx=0x7ff184004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.091+0000 7f6c0bfff640 1 --2- 172.21.15.27:0/252279841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0c15d280 0x7f6c0c15aad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c040097a0 tx=0x7f6c040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.091+0000 7f9e67fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9e700b8b50 0x7f9e700cdd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54038/0 (socket says 172.21.15.27:54038) 2023-12-19T18:15:07.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.091+0000 7f9e67fff640 1 -- 172.21.15.27:0/708820728 learned_addr learned my addr 172.21.15.27:0/708820728 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:07.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.091+0000 7ff198ff9640 1 -- 172.21.15.27:0/745032848 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff184039c90 con 0x7ff19c0d2af0 2023-12-19T18:15:07.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.091+0000 7ff198ff9640 1 -- 172.21.15.27:0/745032848 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff184039df0 con 0x7ff19c0d2af0 2023-12-19T18:15:07.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.091+0000 7ff1a180b640 1 -- 172.21.15.27:0/745032848 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1840094d0 con 0x7ff19c0d2af0 2023-12-19T18:15:07.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.091+0000 7ff198ff9640 1 -- 172.21.15.27:0/745032848 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff184043600 con 0x7ff19c0d2af0 2023-12-19T18:15:07.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.091+0000 7f9e6ed9d640 1 --2- 172.21.15.27:0/708820728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e700ce470 0x7f9e70167d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.091+0000 7ff1a180b640 1 -- 172.21.15.27:0/745032848 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19c15f240 con 0x7ff19c0d2af0 2023-12-19T18:15:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.092+0000 7f5110a0e640 1 --2- 172.21.15.27:0/581235693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c1449e0 0x7f510c06ce00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009870 tx=0x7f50fc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1018627f21b08e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.092+0000 7f5103fff640 1 -- 172.21.15.27:0/581235693 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc0317c0 con 0x7f510c1449e0 2023-12-19T18:15:07.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.092+0000 7f9e6ed9d640 1 -- 172.21.15.27:0/708820728 >> v1:172.21.15.192:6789/0 conn(0x7f9e7015d280 legacy=0x7f9e700d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.092+0000 7f9e6ed9d640 1 -- 172.21.15.27:0/708820728 >> v1:172.21.15.27:6789/0 conn(0x7f9e700b8b50 legacy=0x7f9e700cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.092+0000 7f9e6ed9d640 1 -- 172.21.15.27:0/708820728 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e700d1e90 con 0x7f9e700ce470 2023-12-19T18:15:07.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.092+0000 7f6c08ff9640 1 -- 172.21.15.27:0/252279841 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04006e10 con 0x7f6c0c15d280 2023-12-19T18:15:07.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.092+0000 7f9e6dd9b640 1 -- 172.21.15.27:0/708820728 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1692793774 0 0) 0x7f9e58001110 con 0x7f9e700b8b50 2023-12-19T18:15:07.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.093+0000 7f6c08ff9640 1 -- 172.21.15.27:0/252279841 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0403ace0 con 0x7f6c0c15d280 2023-12-19T18:15:07.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.093+0000 7f6c08ff9640 1 -- 172.21.15.27:0/252279841 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c04044640 con 0x7f6c0c15d280 2023-12-19T18:15:07.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.092+0000 7f68fffff640 1 -- 172.21.15.27:0/225149041 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6904001670 con 0x7f6908070c00 2023-12-19T18:15:07.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.093+0000 7f68fffff640 1 -- 172.21.15.27:0/225149041 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690002c820 con 0x7f6908070c00 2023-12-19T18:15:07.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.093+0000 7f68fffff640 1 -- 172.21.15.27:0/225149041 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690002cbb0 con 0x7f6908070c00 2023-12-19T18:15:07.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.092+0000 7f5103fff640 1 -- 172.21.15.27:0/581235693 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc031920 con 0x7f510c1449e0 2023-12-19T18:15:07.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.092+0000 7f5103fff640 1 -- 172.21.15.27:0/581235693 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc031cb0 con 0x7f510c1449e0 2023-12-19T18:15:07.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.093+0000 7ff1a180b640 1 -- 172.21.15.27:0/745032848 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff19c066810 con 0x7ff19c0d2af0 2023-12-19T18:15:07.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.093+0000 7f6c120bd640 1 -- 172.21.15.27:0/252279841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c0c15f720 con 0x7f6c0c15d280 2023-12-19T18:15:07.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.093+0000 7f6c120bd640 1 -- 172.21.15.27:0/252279841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c0c14a650 con 0x7f6c0c15d280 2023-12-19T18:15:07.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.093+0000 7f0d45ffb640 1 -- 172.21.15.27:0/1017806278 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0d500ce590 con 0x7f0d4803a6b0 2023-12-19T18:15:07.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f690f607640 1 -- 172.21.15.27:0/225149041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6908070c00 msgr2=0x7f69080712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f690f607640 1 --2- 172.21.15.27:0/225149041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6908070c00 0x7f69080712e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f6908071230 tx=0x7f690002bcf0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f5113473640 1 -- 172.21.15.27:0/581235693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c1449e0 msgr2=0x7f510c06ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f5113473640 1 --2- 172.21.15.27:0/581235693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c1449e0 0x7f510c06ce00 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009870 tx=0x7f50fc02b860 comp rx=0 tx=0).stop 2023-12-19T18:15:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f5113473640 1 -- 172.21.15.27:0/581235693 shutdown_connections 2023-12-19T18:15:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f5113473640 1 --2- 172.21.15.27:0/581235693 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f510c06d340 0x7f510c146b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f5113473640 1 --2- 172.21.15.27:0/581235693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c1449e0 0x7f510c06ce00 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f5113473640 1 -- 172.21.15.27:0/581235693 >> 172.21.15.27:0/581235693 conn(0x7f510c0b1880 msgr2=0x7f510c0b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f690f607640 1 -- 172.21.15.27:0/225149041 shutdown_connections 2023-12-19T18:15:07.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f5113473640 1 -- 172.21.15.27:0/581235693 shutdown_connections 2023-12-19T18:15:07.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f690f607640 1 --2- 172.21.15.27:0/225149041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6908070c00 0x7f69080712e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f690f607640 1 --2- 172.21.15.27:0/225149041 >> v2:172.21.15.192:3300/0 conn(0x7f690806fc10 0x7f690806d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f690f607640 1 -- 172.21.15.27:0/225149041 >> 172.21.15.27:0/225149041 conn(0x7f69080ceb50 msgr2=0x7f69080cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.095+0000 7f690f607640 1 -- 172.21.15.27:0/225149041 shutdown_connections 2023-12-19T18:15:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.095+0000 7f690f607640 1 -- 172.21.15.27:0/225149041 wait complete. 2023-12-19T18:15:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.095+0000 7febdbfff640 1 -- 172.21.15.27:0/4245172404 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc0074e0 con 0x7febe8050f00 2023-12-19T18:15:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.095+0000 7febdbfff640 1 -- 172.21.15.27:0/4245172404 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc007640 con 0x7febe8050f00 2023-12-19T18:15:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.095+0000 7f9e6ed9d640 1 --2- 172.21.15.27:0/708820728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e700ce470 0x7f9e70167d40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009dc0 tx=0x7f9e6802bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efb5e3bf5e229be0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.094+0000 7f5113473640 1 -- 172.21.15.27:0/581235693 wait complete. 2023-12-19T18:15:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.095+0000 7febdbfff640 1 -- 172.21.15.27:0/4245172404 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc0049e0 con 0x7febe8050f00 2023-12-19T18:15:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.095+0000 7f6c120bd640 1 -- 172.21.15.27:0/252279841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0c067110 con 0x7f6c0c15d280 2023-12-19T18:15:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.096+0000 7f6c08ff9640 1 -- 172.21.15.27:0/252279841 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6c0404d440 con 0x7f6c0c15d280 2023-12-19T18:15:07.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.095+0000 7febf4dfd640 1 -- 172.21.15.27:0/4245172404 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe8047d90 con 0x7febe8050f00 2023-12-19T18:15:07.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.095+0000 7febf4dfd640 1 -- 172.21.15.27:0/4245172404 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe8045fe0 con 0x7febe8050f00 2023-12-19T18:15:07.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.096+0000 7ff198ff9640 1 -- 172.21.15.27:0/745032848 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff184043760 con 0x7ff19c0d2af0 2023-12-19T18:15:07.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.097+0000 7febf4dfd640 1 -- 172.21.15.27:0/4245172404 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd0005250 con 0x7febe8050f00 2023-12-19T18:15:07.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.097+0000 7f690f607640 1 Processor -- start 2023-12-19T18:15:07.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.097+0000 7fa890ff9640 1 -- 172.21.15.27:0/1077621747 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa880052f90 con 0x7fa88c0e15b0 2023-12-19T18:15:07.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.097+0000 7f6c08ff9640 1 --2- 172.21.15.27:0/252279841 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6bec06a480 0x7f6bec06c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.098+0000 7f6c08ff9640 1 -- 172.21.15.27:0/252279841 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c040aaaf0 con 0x7f6c0c15d280 2023-12-19T18:15:07.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.098+0000 7ff198ff9640 1 --2- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff17c06a430 0x7ff17c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.098+0000 7f5113473640 1 Processor -- start 2023-12-19T18:15:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.098+0000 7ff198ff9640 1 -- 172.21.15.27:0/745032848 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff18403e070 con 0x7ff19c0d2af0 2023-12-19T18:15:07.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.098+0000 7f5113473640 1 -- start start 2023-12-19T18:15:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.098+0000 7f6c0b7fe640 1 --2- 172.21.15.27:0/252279841 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6bec06a480 0x7f6bec06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.098+0000 7f690f607640 1 -- start start 2023-12-19T18:15:07.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7ff19affd640 1 --2- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff17c06a430 0x7ff17c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f5113473640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f510c069c70 0x7f510c1f9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f5113473640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c06d340 0x7f510c145390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f5113473640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f510c1449e0 0x7f510c0b4070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f5113473640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c15db30 con 0x7f510c06d340 2023-12-19T18:15:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f5113473640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510c15e1f0 con 0x7f510c1449e0 2023-12-19T18:15:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f5113473640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f510c15e490 con 0x7f510c069c70 2023-12-19T18:15:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f5110a0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c06d340 0x7f510c145390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f690f607640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f690806fc10 0x7f690815c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f690f607640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6908070c00 0x7f69081f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f690f607640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f690815adf0 0x7f69081f0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f5110a0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c06d340 0x7f510c145390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49710/0 (socket says 172.21.15.27:49710) 2023-12-19T18:15:07.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f5110a0e640 1 -- 172.21.15.27:0/3225614741 learned_addr learned my addr 172.21.15.27:0/3225614741 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:07.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f690f607640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69081f2580 con 0x7f690815adf0 2023-12-19T18:15:07.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f690f607640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f690815c9f0 con 0x7f690806fc10 2023-12-19T18:15:07.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.099+0000 7f690f607640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f690815ccd0 con 0x7f6908070c00 2023-12-19T18:15:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.100+0000 7f511120f640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f510c069c70 0x7f510c1f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.100+0000 7f5111a10640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f510c1449e0 0x7f510c0b4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.100+0000 7f5110a0e640 1 -- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f510c069c70 msgr2=0x7f510c1f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.100+0000 7f5110a0e640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f510c069c70 0x7f510c1f9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.100+0000 7f5110a0e640 1 -- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f510c1449e0 msgr2=0x7f510c0b4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.100+0000 7f5110a0e640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f510c1449e0 0x7f510c0b4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.100+0000 7f5110a0e640 1 -- 172.21.15.27:0/3225614741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc0094d0 con 0x7f510c06d340 2023-12-19T18:15:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.100+0000 7f690dba4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f690815adf0 0x7f69081f0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.100+0000 7f690dba4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f690815adf0 0x7f69081f0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49720/0 (socket says 172.21.15.27:49720) 2023-12-19T18:15:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.100+0000 7f690dba4640 1 -- 172.21.15.27:0/237628786 learned_addr learned my addr 172.21.15.27:0/237628786 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.100+0000 7f6c0b7fe640 1 --2- 172.21.15.27:0/252279841 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6bec06a480 0x7f6bec06c940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0113e0 tx=0x7f6bfc00b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.101+0000 7ff19affd640 1 --2- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff17c06a430 0x7ff17c06c8f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0c9c90 tx=0x7ff18c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.101+0000 7f690d3a3640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f690806fc10 0x7f690815c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.101+0000 7f690cba2640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6908070c00 0x7f69081f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.101+0000 7f690dba4640 1 -- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6908070c00 msgr2=0x7f69081f2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.101+0000 7f690dba4640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6908070c00 0x7f69081f2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.101+0000 7f690dba4640 1 -- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f690806fc10 msgr2=0x7f690815c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.101+0000 7f690dba4640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f690806fc10 0x7f690815c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.101+0000 7f5110a0e640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c06d340 0x7f510c145390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f50fc02b8a0 tx=0x7f50fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.101+0000 7f690dba4640 1 -- 172.21.15.27:0/237628786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6900009a70 con 0x7f690815adf0 2023-12-19T18:15:07.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.101+0000 7febdbfff640 1 -- 172.21.15.27:0/4245172404 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7febdc004b40 con 0x7febe8050f00 2023-12-19T18:15:07.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.102+0000 7f690dba4640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f690815adf0 0x7f69081f0df0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6900009d90 tx=0x7f6900004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.103+0000 7febdbfff640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7febd406a480 0x7febd406c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.103+0000 7febdbfff640 1 -- 172.21.15.27:0/4245172404 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7febdc087d80 con 0x7febe8050f00 2023-12-19T18:15:07.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.104+0000 7febef7fe640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7febd406a480 0x7febd406c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.105+0000 7febef7fe640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7febd406a480 0x7febd406c940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febe8037df0 tx=0x7febe4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.105+0000 7f68fdffb640 1 -- 172.21.15.27:0/237628786 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900035e90 con 0x7f690815adf0 2023-12-19T18:15:07.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.105+0000 7f68fdffb640 1 -- 172.21.15.27:0/237628786 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690002ce10 con 0x7f690815adf0 2023-12-19T18:15:07.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.105+0000 7f68fdffb640 1 -- 172.21.15.27:0/237628786 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690003e630 con 0x7f690815adf0 2023-12-19T18:15:07.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.105+0000 7f5101ffb640 1 -- 172.21.15.27:0/3225614741 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc032860 con 0x7f510c06d340 2023-12-19T18:15:07.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.105+0000 7f5101ffb640 1 -- 172.21.15.27:0/3225614741 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50fc032a00 con 0x7f510c06d340 2023-12-19T18:15:07.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.105+0000 7f5101ffb640 1 -- 172.21.15.27:0/3225614741 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50fc032db0 con 0x7f510c06d340 2023-12-19T18:15:07.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.105+0000 7f5113473640 1 -- 172.21.15.27:0/3225614741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510c15e770 con 0x7f510c06d340 2023-12-19T18:15:07.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.105+0000 7f690f607640 1 -- 172.21.15.27:0/237628786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f690815cfb0 con 0x7f690815adf0 2023-12-19T18:15:07.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.105+0000 7f5113473640 1 -- 172.21.15.27:0/3225614741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f510c15ed30 con 0x7f510c06d340 2023-12-19T18:15:07.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.106+0000 7f690f607640 1 -- 172.21.15.27:0/237628786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f690815d570 con 0x7f690815adf0 2023-12-19T18:15:07.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.106+0000 7f6c08ff9640 1 -- 172.21.15.27:0/252279841 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6c04076150 con 0x7f6c0c15d280 2023-12-19T18:15:07.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.107+0000 7f5113473640 1 -- 172.21.15.27:0/3225614741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f510c1583f0 con 0x7f510c06d340 2023-12-19T18:15:07.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.108+0000 7f690f607640 1 -- 172.21.15.27:0/237628786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f690815d790 con 0x7f690815adf0 2023-12-19T18:15:07.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.109+0000 7f68fdffb640 1 -- 172.21.15.27:0/237628786 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f690003b050 con 0x7f690815adf0 2023-12-19T18:15:07.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.110+0000 7f5101ffb640 1 -- 172.21.15.27:0/3225614741 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f50fc04d3f0 con 0x7f510c06d340 2023-12-19T18:15:07.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.110+0000 7f68fdffb640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f68ec06a430 0x7f68ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.110+0000 7f68fdffb640 1 -- 172.21.15.27:0/237628786 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f69000aa7d0 con 0x7f690815adf0 2023-12-19T18:15:07.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.111+0000 7f690d3a3640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f68ec06a430 0x7f68ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.111+0000 7f5101ffb640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f50f006a430 0x7f50f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.111+0000 7f5101ffb640 1 -- 172.21.15.27:0/3225614741 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f50fc0aaa70 con 0x7f510c06d340 2023-12-19T18:15:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.111+0000 7f9e6dd9b640 1 -- 172.21.15.27:0/708820728 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6802ccc0 con 0x7f9e700ce470 2023-12-19T18:15:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.111+0000 7f9e6dd9b640 1 -- 172.21.15.27:0/708820728 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6802ce20 con 0x7f9e700ce470 2023-12-19T18:15:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.111+0000 7f9e6dd9b640 1 -- 172.21.15.27:0/708820728 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e68034660 con 0x7f9e700ce470 2023-12-19T18:15:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.112+0000 7ff198ff9640 1 -- 172.21.15.27:0/745032848 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff1840778c0 con 0x7ff19c0d2af0 2023-12-19T18:15:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.112+0000 7f511120f640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f50f006a430 0x7f50f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.113+0000 7f9e74e21640 1 -- 172.21.15.27:0/708820728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e700ce470 msgr2=0x7f9e70167d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.113+0000 7f9e74e21640 1 --2- 172.21.15.27:0/708820728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e700ce470 0x7f9e70167d40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f9e68009dc0 tx=0x7f9e6802bf10 comp rx=0 tx=0).stop 2023-12-19T18:15:07.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.113+0000 7f9e74e21640 1 -- 172.21.15.27:0/708820728 shutdown_connections 2023-12-19T18:15:07.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.113+0000 7f690d3a3640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f68ec06a430 0x7f68ec06c8f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f69040011e0 tx=0x7f6904009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.113+0000 7f9e74e21640 1 --2- 172.21.15.27:0/708820728 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e700ce470 0x7f9e70167d40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.113+0000 7f9e74e21640 1 -- 172.21.15.27:0/708820728 >> 172.21.15.27:0/708820728 conn(0x7f9e700d2f90 msgr2=0x7f9e700d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.114+0000 7f9e74e21640 1 -- 172.21.15.27:0/708820728 shutdown_connections 2023-12-19T18:15:07.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.114+0000 7f511120f640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f50f006a430 0x7f50f006c8f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5108004ea0 tx=0x7f5108009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.114+0000 7f9e74e21640 1 -- 172.21.15.27:0/708820728 wait complete. 2023-12-19T18:15:07.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.115+0000 7febdbfff640 1 -- 172.21.15.27:0/4245172404 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7febdc056e90 con 0x7febe8050f00 2023-12-19T18:15:07.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.116+0000 7f5101ffb640 1 -- 172.21.15.27:0/3225614741 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f50fc075fc0 con 0x7f510c06d340 2023-12-19T18:15:07.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.118+0000 7f9e74e21640 1 Processor -- start 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.119+0000 7f9e74e21640 1 -- start start 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.120+0000 7f9e74e21640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e700b8b50 0x7f9e701f1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.120+0000 7f9e74e21640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e700ce470 0x7f9e700c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.121+0000 7f9e74e21640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7015d280 0x7f9e700c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.121+0000 7f9e74e21640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e701f2380 con 0x7f9e7015d280 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.121+0000 7f9e74e21640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7015d980 con 0x7f9e700b8b50 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.121+0000 7f9e74e21640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7015dc60 con 0x7f9e700ce470 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.122+0000 7f9e6ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7015d280 0x7f9e700c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.122+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e700ce470 0x7f9e700c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.122+0000 7f9e67fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e700ce470 0x7f9e700c7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:42332/0 (socket says 172.21.15.27:42332) 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.122+0000 7f9e67fff640 1 -- 172.21.15.27:0/2678555217 learned_addr learned my addr 172.21.15.27:0/2678555217 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.122+0000 7f9e6e59c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e700b8b50 0x7f9e701f1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.123+0000 7f9e6ed9d640 1 -- 172.21.15.27:0/2678555217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e700ce470 msgr2=0x7f9e700c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.123+0000 7f9e6ed9d640 1 --2- 172.21.15.27:0/2678555217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e700ce470 0x7f9e700c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.123+0000 7f9e6ed9d640 1 -- 172.21.15.27:0/2678555217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e700b8b50 msgr2=0x7f9e701f1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.123+0000 7f9e6ed9d640 1 --2- 172.21.15.27:0/2678555217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e700b8b50 0x7f9e701f1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.124+0000 7f9e6ed9d640 1 -- 172.21.15.27:0/2678555217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e68009a70 con 0x7f9e7015d280 2023-12-19T18:15:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.125+0000 7f68fdffb640 1 -- 172.21.15.27:0/237628786 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6900075dd0 con 0x7f690815adf0 2023-12-19T18:15:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.125+0000 7f9e6ed9d640 1 --2- 172.21.15.27:0/2678555217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7015d280 0x7f9e700c7840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e68033000 tx=0x7f9e68004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.126+0000 7f9e66ffd640 1 -- 172.21.15.27:0/2678555217 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6803e610 con 0x7f9e7015d280 2023-12-19T18:15:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.127+0000 7f9e66ffd640 1 -- 172.21.15.27:0/2678555217 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e6803e770 con 0x7f9e7015d280 2023-12-19T18:15:07.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.127+0000 7f9e66ffd640 1 -- 172.21.15.27:0/2678555217 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e6803eb60 con 0x7f9e7015d280 2023-12-19T18:15:07.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.127+0000 7f9e74e21640 1 -- 172.21.15.27:0/2678555217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7015df40 con 0x7f9e7015d280 2023-12-19T18:15:07.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.127+0000 7f9e74e21640 1 -- 172.21.15.27:0/2678555217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7015e500 con 0x7f9e7015d280 2023-12-19T18:15:07.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.129+0000 7f9e74e21640 1 -- 172.21.15.27:0/2678555217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7015e720 con 0x7f9e7015d280 2023-12-19T18:15:07.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.135+0000 7f9e66ffd640 1 -- 172.21.15.27:0/2678555217 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9e6803b050 con 0x7f9e7015d280 2023-12-19T18:15:07.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.136+0000 7f9e66ffd640 1 --2- 172.21.15.27:0/2678555217 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e5006a430 0x7f9e5006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:07.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.136+0000 7f9e66ffd640 1 -- 172.21.15.27:0/2678555217 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e680ab170 con 0x7f9e7015d280 2023-12-19T18:15:07.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.140+0000 7f9e6e59c640 1 --2- 172.21.15.27:0/2678555217 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e5006a430 0x7f9e5006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:07.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.141+0000 7f9e6e59c640 1 --2- 172.21.15.27:0/2678555217 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e5006a430 0x7f9e5006c8f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e60000f60 tx=0x7f9e60009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:07.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.142+0000 7f9e66ffd640 1 -- 172.21.15.27:0/2678555217 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9e68076920 con 0x7f9e7015d280 2023-12-19T18:15:07.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.521+0000 7fa89a896640 1 -- 172.21.15.27:0/1077621747 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa874005500 con 0x7fa88c0e15b0 2023-12-19T18:15:07.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.523+0000 7fa890ff9640 1 -- 172.21.15.27:0/1077621747 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa880017020 con 0x7fa88c0e15b0 2023-12-19T18:15:07.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.523+0000 7f0d56054640 1 -- 172.21.15.27:0/1017806278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f0d48005490 con 0x7f0d4803a6b0 2023-12-19T18:15:07.524 INFO:teuthology.orchestra.run.smithi027.stdout:55834574853 2023-12-19T18:15:07.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.524+0000 7f0d45ffb640 1 -- 172.21.15.27:0/1017806278 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0d50097090 con 0x7f0d4803a6b0 2023-12-19T18:15:07.525 INFO:teuthology.orchestra.run.smithi027.stdout:55834574853 2023-12-19T18:15:07.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.526+0000 7fa89a896640 1 -- 172.21.15.27:0/1077621747 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa87806a490 msgr2=0x7fa87806c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.526+0000 7fa89a896640 1 --2- 172.21.15.27:0/1077621747 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa87806a490 0x7fa87806c950 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa888036000 tx=0x7fa888008030 comp rx=0 tx=0).stop 2023-12-19T18:15:07.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.526+0000 7fa89a896640 1 -- 172.21.15.27:0/1077621747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa88c0e15b0 msgr2=0x7fa88c039fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.526+0000 7fa89a896640 1 --2- 172.21.15.27:0/1077621747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa88c0e15b0 0x7fa88c039fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa88000c400 tx=0x7fa88000c800 comp rx=0 tx=0).stop 2023-12-19T18:15:07.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.527+0000 7f0d56054640 1 -- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d3406a480 msgr2=0x7f0d3406c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.527+0000 7fa89a896640 1 -- 172.21.15.27:0/1077621747 shutdown_connections 2023-12-19T18:15:07.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.527+0000 7fa89a896640 1 --2- 172.21.15.27:0/1077621747 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa87806a490 0x7fa87806c950 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.527+0000 7fa89a896640 1 --2- 172.21.15.27:0/1077621747 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa88c169e60 0x7fa88c03a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.527+0000 7fa89a896640 1 --2- 172.21.15.27:0/1077621747 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa88c0e15b0 0x7fa88c039fe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.527+0000 7fa89a896640 1 --2- 172.21.15.27:0/1077621747 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa88c0e11d0 0x7fa88c169920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.527+0000 7fa89a896640 1 -- 172.21.15.27:0/1077621747 >> 172.21.15.27:0/1077621747 conn(0x7fa88c053780 msgr2=0x7fa88c0e24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.527+0000 7f0d56054640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d3406a480 0x7f0d3406c940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c000f60 tx=0x7f0d4c009040 comp rx=0 tx=0).stop 2023-12-19T18:15:07.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.528+0000 7fa89a896640 1 -- 172.21.15.27:0/1077621747 shutdown_connections 2023-12-19T18:15:07.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.528+0000 7fa89a896640 1 -- 172.21.15.27:0/1077621747 wait complete. 2023-12-19T18:15:07.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.527+0000 7f0d56054640 1 -- 172.21.15.27:0/1017806278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4803a6b0 msgr2=0x7f0d48034960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.527+0000 7f0d56054640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4803a6b0 0x7f0d48034960 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d5004f7d0 tx=0x7f0d50072460 comp rx=0 tx=0).stop 2023-12-19T18:15:07.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.528+0000 7f0d56054640 1 -- 172.21.15.27:0/1017806278 shutdown_connections 2023-12-19T18:15:07.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.528+0000 7f0d56054640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d3406a480 0x7f0d3406c940 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.528+0000 7f0d56054640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4803a6b0 0x7f0d48034960 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.528+0000 7f0d56054640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d48033e40 0x7f0d48048710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.528+0000 7f0d56054640 1 --2- 172.21.15.27:0/1017806278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d48006870 0x7f0d4816ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.528+0000 7f0d56054640 1 -- 172.21.15.27:0/1017806278 >> 172.21.15.27:0/1017806278 conn(0x7f0d48054c10 msgr2=0x7f0d480dfe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.529+0000 7f0d56054640 1 -- 172.21.15.27:0/1017806278 shutdown_connections 2023-12-19T18:15:07.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.529+0000 7f0d56054640 1 -- 172.21.15.27:0/1017806278 wait complete. 2023-12-19T18:15:07.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.541+0000 7ff1a180b640 1 -- 172.21.15.27:0/745032848 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff19c068e90 con 0x7ff19c0d2af0 2023-12-19T18:15:07.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.545+0000 7f6c120bd640 1 -- 172.21.15.27:0/252279841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f6c0c0b7fe0 con 0x7f6c0c15d280 2023-12-19T18:15:07.546 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:07.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.542+0000 7ff198ff9640 1 -- 172.21.15.27:0/745032848 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff184034020 con 0x7ff19c0d2af0 2023-12-19T18:15:07.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.545+0000 7f5113473640 1 -- 172.21.15.27:0/3225614741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f510c15e5d0 con 0x7f510c06d340 2023-12-19T18:15:07.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.546+0000 7febf4dfd640 1 -- 172.21.15.27:0/4245172404 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7febd00057d0 con 0x7febe8050f00 2023-12-19T18:15:07.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.546+0000 7f5101ffb640 1 -- 172.21.15.27:0/3225614741 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f510c15e5d0 con 0x7f510c06d340 2023-12-19T18:15:07.547 INFO:teuthology.orchestra.run.smithi027.stdout:55834574853 2023-12-19T18:15:07.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.546+0000 7ff1a180b640 1 -- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff17c06a430 msgr2=0x7ff17c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.546+0000 7ff1a180b640 1 --2- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff17c06a430 0x7ff17c06c8f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff19c0c9c90 tx=0x7ff18c009190 comp rx=0 tx=0).stop 2023-12-19T18:15:07.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.546+0000 7ff1a180b640 1 -- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 msgr2=0x7ff19c15e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.546+0000 7ff1a180b640 1 --2- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 0x7ff19c15e8f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff184000c00 tx=0x7ff184004730 comp rx=0 tx=0).stop 2023-12-19T18:15:07.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.546+0000 7febdbfff640 1 -- 172.21.15.27:0/4245172404 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7febdc018020 con 0x7febe8050f00 2023-12-19T18:15:07.549 INFO:teuthology.orchestra.run.smithi027.stdout:55834574853 2023-12-19T18:15:07.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.546+0000 7ff1a180b640 1 -- 172.21.15.27:0/745032848 shutdown_connections 2023-12-19T18:15:07.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.546+0000 7ff1a180b640 1 --2- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff17c06a430 0x7ff17c06c8f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.546+0000 7ff1a180b640 1 --2- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff19c15d260 0x7ff19c158bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.546+0000 7ff1a180b640 1 --2- 172.21.15.27:0/745032848 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff19c0d2af0 0x7ff19c15e8f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.546+0000 7ff1a180b640 1 --2- 172.21.15.27:0/745032848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff19c0d0e70 0x7ff19c1f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.546+0000 7ff1a180b640 1 -- 172.21.15.27:0/745032848 >> 172.21.15.27:0/745032848 conn(0x7ff19c0d2f70 msgr2=0x7ff19c0b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.547+0000 7ff1a180b640 1 -- 172.21.15.27:0/745032848 shutdown_connections 2023-12-19T18:15:07.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.547+0000 7f6c08ff9640 1 -- 172.21.15.27:0/252279841 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c04079770 con 0x7f6c0c15d280 2023-12-19T18:15:07.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.547+0000 7ff1a180b640 1 -- 172.21.15.27:0/745032848 wait complete. 2023-12-19T18:15:07.551 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:07.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7febf4dfd640 1 -- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7febd406a480 msgr2=0x7febd406c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7f5113473640 1 -- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f50f006a430 msgr2=0x7f50f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7f5113473640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f50f006a430 0x7f50f006c8f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5108004ea0 tx=0x7f5108009040 comp rx=0 tx=0).stop 2023-12-19T18:15:07.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7f5113473640 1 -- 172.21.15.27:0/3225614741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c06d340 msgr2=0x7f510c145390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7f5113473640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c06d340 0x7f510c145390 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f50fc02b8a0 tx=0x7f50fc0046c0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7febf4dfd640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7febd406a480 0x7febd406c940 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febe8037df0 tx=0x7febe4009040 comp rx=0 tx=0).stop 2023-12-19T18:15:07.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7febf4dfd640 1 -- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febe8050f00 msgr2=0x7febe804a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7febf4dfd640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febe8050f00 0x7febe804a910 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c490 tx=0x7febdc00c950 comp rx=0 tx=0).stop 2023-12-19T18:15:07.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7f5113473640 1 -- 172.21.15.27:0/3225614741 shutdown_connections 2023-12-19T18:15:07.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7f5113473640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f50f006a430 0x7f50f006c8f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7f5113473640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f510c1449e0 0x7f510c0b4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7f5113473640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f510c06d340 0x7f510c145390 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7f5113473640 1 --2- 172.21.15.27:0/3225614741 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f510c069c70 0x7f510c1f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.550+0000 7f5113473640 1 -- 172.21.15.27:0/3225614741 >> 172.21.15.27:0/3225614741 conn(0x7f510c0b1880 msgr2=0x7f510c0affc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7febf4dfd640 1 -- 172.21.15.27:0/4245172404 shutdown_connections 2023-12-19T18:15:07.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7f6c120bd640 1 -- 172.21.15.27:0/252279841 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6bec06a480 msgr2=0x7f6bec06c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7f6c120bd640 1 --2- 172.21.15.27:0/252279841 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6bec06a480 0x7f6bec06c940 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0113e0 tx=0x7f6bfc00b040 comp rx=0 tx=0).stop 2023-12-19T18:15:07.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7f6c120bd640 1 -- 172.21.15.27:0/252279841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0c15d280 msgr2=0x7f6c0c15aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7f6c120bd640 1 --2- 172.21.15.27:0/252279841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0c15d280 0x7f6c0c15aad0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f6c040097a0 tx=0x7f6c040046c0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7febf4dfd640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7febd406a480 0x7febd406c940 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7febe8037df0 tx=0x7febe4009040 comp rx=0 tx=0).stop 2023-12-19T18:15:07.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7febf4dfd640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7febe80dffa0 0x7febe80db430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7febf4dfd640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7febe8050f00 0x7febe804a910 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7febdc00c490 tx=0x7febdc00c950 comp rx=0 tx=0).stop 2023-12-19T18:15:07.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7febf4dfd640 1 --2- 172.21.15.27:0/4245172404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febe8050300 0x7febe80458c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7febf4dfd640 1 -- 172.21.15.27:0/4245172404 >> 172.21.15.27:0/4245172404 conn(0x7febe8054c10 msgr2=0x7febe804dcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7f5113473640 1 -- 172.21.15.27:0/3225614741 shutdown_connections 2023-12-19T18:15:07.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7febf4dfd640 1 -- 172.21.15.27:0/4245172404 shutdown_connections 2023-12-19T18:15:07.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7febf4dfd640 1 -- 172.21.15.27:0/4245172404 wait complete. 2023-12-19T18:15:07.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7f6c120bd640 1 -- 172.21.15.27:0/252279841 shutdown_connections 2023-12-19T18:15:07.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7f6c120bd640 1 --2- 172.21.15.27:0/252279841 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6bec06a480 0x7f6bec06c940 secure :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6bfc0113e0 tx=0x7f6bfc00b040 comp rx=0 tx=0).stop 2023-12-19T18:15:07.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7f6c120bd640 1 --2- 172.21.15.27:0/252279841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0c15d280 0x7f6c0c15aad0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7f6c120bd640 1 --2- 172.21.15.27:0/252279841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c0c0d1360 0x7f6c0c15ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7f6c120bd640 1 --2- 172.21.15.27:0/252279841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c0c0ce470 0x7f6c0c14a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7f6c120bd640 1 -- 172.21.15.27:0/252279841 >> 172.21.15.27:0/252279841 conn(0x7f6c0c0d2f90 msgr2=0x7f6c0c158ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.551+0000 7f5113473640 1 -- 172.21.15.27:0/3225614741 wait complete. 2023-12-19T18:15:07.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.552+0000 7f6c120bd640 1 -- 172.21.15.27:0/252279841 shutdown_connections 2023-12-19T18:15:07.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.552+0000 7f6c120bd640 1 -- 172.21.15.27:0/252279841 wait complete. 2023-12-19T18:15:07.558 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-12-19T18:15:07.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.558+0000 7f690f607640 1 -- 172.21.15.27:0/237628786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f6908068e90 con 0x7f690815adf0 2023-12-19T18:15:07.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2023-12-19T18:15:07.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.559+0000 7f68fdffb640 1 -- 172.21.15.27:0/237628786 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f69000793f0 con 0x7f690815adf0 2023-12-19T18:15:07.560 INFO:teuthology.orchestra.run.smithi027.stdout:55834574853 2023-12-19T18:15:07.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.562+0000 7f9e74e21640 1 -- 172.21.15.27:0/2678555217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9e7015dda0 con 0x7f9e7015d280 2023-12-19T18:15:07.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.562+0000 7f690f607640 1 -- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f68ec06a430 msgr2=0x7f68ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.562+0000 7f690f607640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f68ec06a430 0x7f68ec06c8f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f69040011e0 tx=0x7f6904009040 comp rx=0 tx=0).stop 2023-12-19T18:15:07.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.562+0000 7f690f607640 1 -- 172.21.15.27:0/237628786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f690815adf0 msgr2=0x7f69081f0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.562+0000 7f690f607640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f690815adf0 0x7f69081f0df0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6900009d90 tx=0x7f6900004a40 comp rx=0 tx=0).stop 2023-12-19T18:15:07.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.562+0000 7f690f607640 1 -- 172.21.15.27:0/237628786 shutdown_connections 2023-12-19T18:15:07.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.562+0000 7f690f607640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f68ec06a430 0x7f68ec06c8f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.562+0000 7f9e66ffd640 1 -- 172.21.15.27:0/2678555217 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9e68039020 con 0x7f9e7015d280 2023-12-19T18:15:07.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.562+0000 7f690f607640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f690815adf0 0x7f69081f0df0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.562+0000 7f690f607640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6908070c00 0x7f69081f2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.562+0000 7f690f607640 1 --2- 172.21.15.27:0/237628786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f690806fc10 0x7f690815c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.562+0000 7f690f607640 1 -- 172.21.15.27:0/237628786 >> 172.21.15.27:0/237628786 conn(0x7f69080ceb50 msgr2=0x7f6908159220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.564 INFO:teuthology.orchestra.run.smithi027.stdout:55834574853 2023-12-19T18:15:07.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.563+0000 7f690f607640 1 -- 172.21.15.27:0/237628786 shutdown_connections 2023-12-19T18:15:07.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.563+0000 7f690f607640 1 -- 172.21.15.27:0/237628786 wait complete. 2023-12-19T18:15:07.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.565+0000 7f9e74e21640 1 -- 172.21.15.27:0/2678555217 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e5006a430 msgr2=0x7f9e5006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.565+0000 7f9e74e21640 1 --2- 172.21.15.27:0/2678555217 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e5006a430 0x7f9e5006c8f0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9e60000f60 tx=0x7f9e60009040 comp rx=0 tx=0).stop 2023-12-19T18:15:07.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.565+0000 7f9e74e21640 1 -- 172.21.15.27:0/2678555217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7015d280 msgr2=0x7f9e700c7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:07.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.565+0000 7f9e74e21640 1 --2- 172.21.15.27:0/2678555217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7015d280 0x7f9e700c7840 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f9e68033000 tx=0x7f9e68004ab0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.566+0000 7f9e74e21640 1 -- 172.21.15.27:0/2678555217 shutdown_connections 2023-12-19T18:15:07.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.566+0000 7f9e74e21640 1 --2- 172.21.15.27:0/2678555217 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e5006a430 0x7f9e5006c8f0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.566+0000 7f9e74e21640 1 --2- 172.21.15.27:0/2678555217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7015d280 0x7f9e700c7840 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.566+0000 7f9e74e21640 1 --2- 172.21.15.27:0/2678555217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e700ce470 0x7f9e700c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.566+0000 7f9e74e21640 1 --2- 172.21.15.27:0/2678555217 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e700b8b50 0x7f9e701f1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:07.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.566+0000 7f9e74e21640 1 -- 172.21.15.27:0/2678555217 >> 172.21.15.27:0/2678555217 conn(0x7f9e700d2f90 msgr2=0x7f9e700cd810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:07.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.566+0000 7f9e74e21640 1 -- 172.21.15.27:0/2678555217 shutdown_connections 2023-12-19T18:15:07.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:07.566+0000 7f9e74e21640 1 -- 172.21.15.27:0/2678555217 wait complete. 2023-12-19T18:15:07.576 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-12-19T18:15:07.581 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.2 2023-12-19T18:15:07.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2023-12-19T18:15:07.584 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-12-19T18:15:07.589 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2023-12-19T18:15:07.590 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.4 2023-12-19T18:15:08.559 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-19T18:15:08.560 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-19T18:15:08.577 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-19T18:15:08.582 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-19T18:15:08.584 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-19T18:15:08.584 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-19T18:15:08.590 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-19T18:15:08.591 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-19T18:15:08.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.860+0000 7f29a08af640 1 Processor -- start 2023-12-19T18:15:08.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.860+0000 7f29a08af640 1 -- start start 2023-12-19T18:15:08.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.863+0000 7f29a08af640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f29940e75c0 0x7f2994038780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.864+0000 7f29a08af640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f29940e79a0 0x7f2994038cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.864+0000 7f29a08af640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f299404a310 0x7f29940eb320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.864+0000 7f29a08af640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2994049dc0 con 0x7f29940e75c0 2023-12-19T18:15:08.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.865+0000 7f29a08af640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2994007b30 con 0x7f29940e79a0 2023-12-19T18:15:08.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.865+0000 7f29a08af640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29940542a0 con 0x7f299404a310 2023-12-19T18:15:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.866+0000 7f299a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f29940e75c0 0x7f2994038780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.866+0000 7f299ad9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f299404a310 0x7f29940eb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.866+0000 7f299ad9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f299404a310 0x7f29940eb320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:42338/0 (socket says 172.21.15.27:42338) 2023-12-19T18:15:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.866+0000 7f299ad9d640 1 -- 172.21.15.27:0/3791296112 learned_addr learned my addr 172.21.15.27:0/3791296112 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.866+0000 7f2999d9b640 1 --2- 172.21.15.27:0/3791296112 >> v2:172.21.15.192:3300/0 conn(0x7f29940e79a0 0x7f2994038cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.867+0000 7f299ad9d640 1 -- 172.21.15.27:0/3791296112 >> v2:172.21.15.192:3300/0 conn(0x7f29940e79a0 msgr2=0x7f2994038cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.867+0000 7f299ad9d640 1 --2- 172.21.15.27:0/3791296112 >> v2:172.21.15.192:3300/0 conn(0x7f29940e79a0 0x7f2994038cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.867+0000 7f299ad9d640 1 -- 172.21.15.27:0/3791296112 >> v2:172.21.15.27:3300/0 conn(0x7f29940e75c0 msgr2=0x7f2994038780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.867+0000 7f299ad9d640 1 --2- 172.21.15.27:0/3791296112 >> v2:172.21.15.27:3300/0 conn(0x7f29940e75c0 0x7f2994038780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.867+0000 7f299ad9d640 1 -- 172.21.15.27:0/3791296112 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2994049fd0 con 0x7f299404a310 2023-12-19T18:15:08.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.867+0000 7f299a59c640 1 --2- 172.21.15.27:0/3791296112 >> v2:172.21.15.27:3300/0 conn(0x7f29940e75c0 0x7f2994038780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:08.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.869+0000 7f299ad9d640 1 --2- 172.21.15.27:0/3791296112 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f299404a310 0x7f29940eb320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f299c079de0 tx=0x7f299c0a60f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1413eee681035860 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.869+0000 7f299959a640 1 -- 172.21.15.27:0/3791296112 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c0a6bd0 con 0x7f299404a310 2023-12-19T18:15:08.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.870+0000 7f299959a640 1 -- 172.21.15.27:0/3791296112 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299c0a6d70 con 0x7f299404a310 2023-12-19T18:15:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.870+0000 7f299959a640 1 -- 172.21.15.27:0/3791296112 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299c0ae660 con 0x7f299404a310 2023-12-19T18:15:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.870+0000 7fb72751b640 1 Processor -- start 2023-12-19T18:15:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.871+0000 7f29a08af640 1 -- 172.21.15.27:0/3791296112 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f299404a310 msgr2=0x7f29940eb320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.871+0000 7f29a08af640 1 --2- 172.21.15.27:0/3791296112 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f299404a310 0x7f29940eb320 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f299c079de0 tx=0x7f299c0a60f0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.871+0000 7f29a08af640 1 -- 172.21.15.27:0/3791296112 shutdown_connections 2023-12-19T18:15:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.871+0000 7f29a08af640 1 --2- 172.21.15.27:0/3791296112 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f299404a310 0x7f29940eb320 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.871+0000 7f29a08af640 1 --2- 172.21.15.27:0/3791296112 >> v2:172.21.15.192:3300/0 conn(0x7f29940e79a0 0x7f2994038cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.871+0000 7f29a08af640 1 --2- 172.21.15.27:0/3791296112 >> v2:172.21.15.27:3300/0 conn(0x7f29940e75c0 0x7f2994038780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.871+0000 7f29a08af640 1 -- 172.21.15.27:0/3791296112 >> 172.21.15.27:0/3791296112 conn(0x7f2994053780 msgr2=0x7f2994053b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.872+0000 7f29a08af640 1 -- 172.21.15.27:0/3791296112 shutdown_connections 2023-12-19T18:15:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.872+0000 7f29a08af640 1 -- 172.21.15.27:0/3791296112 wait complete. 2023-12-19T18:15:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.871+0000 7fb72751b640 1 -- start start 2023-12-19T18:15:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.873+0000 7fb72751b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015cec0 0x7fb7200cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.873+0000 7fb72751b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb72015d2a0 0x7fb7200cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.873+0000 7fb72751b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7200cd8e0 0x7fb7200cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.873+0000 7fb72751b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7200b2610 con 0x7fb7200cd8e0 2023-12-19T18:15:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.873+0000 7fb72751b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7200b28f0 con 0x7fb72015d2a0 2023-12-19T18:15:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.873+0000 7fb72751b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200ce6a0 con 0x7fb72015cec0 2023-12-19T18:15:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.874+0000 7fb7252b7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015cec0 0x7fb7200cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.874+0000 7fb724ab6640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb72015d2a0 0x7fb7200cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.874+0000 7fb725ab8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7200cd8e0 0x7fb7200cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.874+0000 7fb724ab6640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb72015d2a0 0x7fb7200cfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:50640/0 (socket says 172.21.15.27:50640) 2023-12-19T18:15:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.874+0000 7fb724ab6640 1 -- 172.21.15.27:0/3720750715 learned_addr learned my addr 172.21.15.27:0/3720750715 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.874+0000 7fb725ab8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb7200cd8e0 0x7fb7200cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49734/0 (socket says 172.21.15.27:49734) 2023-12-19T18:15:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.875+0000 7fb724ab6640 1 -- 172.21.15.27:0/3720750715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015cec0 msgr2=0x7fb7200cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.875+0000 7fb724ab6640 1 --2- 172.21.15.27:0/3720750715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015cec0 0x7fb7200cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.875+0000 7fb724ab6640 1 -- 172.21.15.27:0/3720750715 >> v2:172.21.15.27:3300/0 conn(0x7fb7200cd8e0 msgr2=0x7fb7200cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.875+0000 7fb724ab6640 1 --2- 172.21.15.27:0/3720750715 >> v2:172.21.15.27:3300/0 conn(0x7fb7200cd8e0 0x7fb7200cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.875+0000 7fb724ab6640 1 -- 172.21.15.27:0/3720750715 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7200ce980 con 0x7fb72015d2a0 2023-12-19T18:15:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.876+0000 7f29a08af640 1 Processor -- start 2023-12-19T18:15:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.876+0000 7f29a08af640 1 -- start start 2023-12-19T18:15:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.878+0000 7f29a08af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f299404a310 0x7f29940ea970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.878+0000 7f29a08af640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29940e75c0 0x7f2994050080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.878+0000 7f29a08af640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29940e79a0 0x7f299404a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.878+0000 7f29a08af640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29940d9b00 con 0x7f299404a310 2023-12-19T18:15:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.878+0000 7f29a08af640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f299404e7e0 con 0x7f29940e75c0 2023-12-19T18:15:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.878+0000 7f29a08af640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f299404ea30 con 0x7f29940e79a0 2023-12-19T18:15:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.879+0000 7f299a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f299404a310 0x7f29940ea970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.877+0000 7fb724ab6640 1 --2- 172.21.15.27:0/3720750715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb72015d2a0 0x7fb7200cfc70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb72009c870 tx=0x7fb71802be90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bde913c12a9663c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.879+0000 7f299a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f299404a310 0x7f29940ea970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49746/0 (socket says 172.21.15.27:49746) 2023-12-19T18:15:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.879+0000 7f299a59c640 1 -- 172.21.15.27:0/4150824414 learned_addr learned my addr 172.21.15.27:0/4150824414 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.879+0000 7f2999d9b640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29940e75c0 0x7f2994050080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.879+0000 7f299ad9d640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29940e79a0 0x7f299404a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.880+0000 7f299a59c640 1 -- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29940e79a0 msgr2=0x7f299404a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.880+0000 7f299a59c640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29940e79a0 0x7f299404a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.880+0000 7f299a59c640 1 -- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29940e75c0 msgr2=0x7f2994050080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.880+0000 7f299a59c640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29940e75c0 0x7f2994050080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.880+0000 7f299a59c640 1 -- 172.21.15.27:0/4150824414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f299404ed10 con 0x7f299404a310 2023-12-19T18:15:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.880+0000 7fb717fff640 1 -- 172.21.15.27:0/3720750715 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71802ca30 con 0x7fb72015d2a0 2023-12-19T18:15:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.880+0000 7fb717fff640 1 -- 172.21.15.27:0/3720750715 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71802cb90 con 0x7fb72015d2a0 2023-12-19T18:15:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.880+0000 7fb717fff640 1 -- 172.21.15.27:0/3720750715 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7180343f0 con 0x7fb72015d2a0 2023-12-19T18:15:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.880+0000 7f299a59c640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f299404a310 0x7f29940ea970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f298c00d7d0 tx=0x7f298c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.881+0000 7f298affd640 1 -- 172.21.15.27:0/4150824414 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c004030 con 0x7f299404a310 2023-12-19T18:15:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.881+0000 7f298affd640 1 -- 172.21.15.27:0/4150824414 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f298c004190 con 0x7f299404a310 2023-12-19T18:15:08.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.881+0000 7f298affd640 1 -- 172.21.15.27:0/4150824414 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f298c004030 con 0x7f299404a310 2023-12-19T18:15:08.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.881+0000 7f29a08af640 1 -- 172.21.15.27:0/4150824414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f299404eff0 con 0x7f299404a310 2023-12-19T18:15:08.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.881+0000 7fb72751b640 1 -- 172.21.15.27:0/3720750715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb72015d2a0 msgr2=0x7fb7200cfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.881+0000 7fb72751b640 1 --2- 172.21.15.27:0/3720750715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb72015d2a0 0x7fb7200cfc70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb72009c870 tx=0x7fb71802be90 comp rx=0 tx=0).stop 2023-12-19T18:15:08.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.882+0000 7f29a08af640 1 -- 172.21.15.27:0/4150824414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f299404f5b0 con 0x7f299404a310 2023-12-19T18:15:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.882+0000 7fb72751b640 1 -- 172.21.15.27:0/3720750715 shutdown_connections 2023-12-19T18:15:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.882+0000 7fb72751b640 1 --2- 172.21.15.27:0/3720750715 >> v2:172.21.15.27:3300/0 conn(0x7fb7200cd8e0 0x7fb7200cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.882+0000 7fb72751b640 1 --2- 172.21.15.27:0/3720750715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb72015d2a0 0x7fb7200cfc70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.882+0000 7fb72751b640 1 --2- 172.21.15.27:0/3720750715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015cec0 0x7fb7200cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.882+0000 7fb72751b640 1 -- 172.21.15.27:0/3720750715 >> 172.21.15.27:0/3720750715 conn(0x7fb72009c140 msgr2=0x7fb7200d2dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:08.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.883+0000 7fb72751b640 1 -- 172.21.15.27:0/3720750715 shutdown_connections 2023-12-19T18:15:08.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.883+0000 7fb72751b640 1 -- 172.21.15.27:0/3720750715 wait complete. 2023-12-19T18:15:08.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.886+0000 7fb72751b640 1 Processor -- start 2023-12-19T18:15:08.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.886+0000 7fb72751b640 1 -- start start 2023-12-19T18:15:08.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.888+0000 7fb72751b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7200cd8e0 0x7fb72014b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.885+0000 7f29a08af640 1 -- 172.21.15.27:0/4150824414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f299404f7d0 con 0x7f299404a310 2023-12-19T18:15:08.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.885+0000 7f298affd640 1 -- 172.21.15.27:0/4150824414 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f298c004300 con 0x7f299404a310 2023-12-19T18:15:08.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.887+0000 7f298affd640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f297c06a490 0x7f297c06c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.889+0000 7fb72751b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015cec0 0x7fb7200b80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.889+0000 7fb7252b7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7200cd8e0 0x7fb72014b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.889+0000 7fb7252b7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7200cd8e0 0x7fb72014b160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:50650/0 (socket says 172.21.15.27:50650) 2023-12-19T18:15:08.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.890+0000 7fb72751b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72015d2a0 0x7fb7200cbd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.890+0000 7fb724ab6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015cec0 0x7fb7200b80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.890+0000 7fb724ab6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015cec0 0x7fb7200b80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:42360/0 (socket says 172.21.15.27:42360) 2023-12-19T18:15:08.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.890+0000 7f2999d9b640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f297c06a490 0x7f297c06c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.890+0000 7fb7252b7640 1 -- 172.21.15.27:0/3180198700 learned_addr learned my addr 172.21.15.27:0/3180198700 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:08.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.890+0000 7f298affd640 1 -- 172.21.15.27:0/4150824414 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f298c058110 con 0x7f299404a310 2023-12-19T18:15:08.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.890+0000 7fb72751b640 1 -- 172.21.15.27:0/3180198700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200cacf0 con 0x7fb72015d2a0 2023-12-19T18:15:08.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.890+0000 7fb72751b640 1 -- 172.21.15.27:0/3180198700 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200caf90 con 0x7fb7200cd8e0 2023-12-19T18:15:08.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.891+0000 7fb72751b640 1 -- 172.21.15.27:0/3180198700 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7200cb270 con 0x7fb72015cec0 2023-12-19T18:15:08.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.891+0000 7fb725ab8640 1 --2- 172.21.15.27:0/3180198700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72015d2a0 0x7fb7200cbd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.892+0000 7f2999d9b640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f297c06a490 0x7f297c06c950 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29940eaeb0 tx=0x7f2990008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.892+0000 7fb725ab8640 1 -- 172.21.15.27:0/3180198700 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015cec0 msgr2=0x7fb7200b80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.892+0000 7fb725ab8640 1 --2- 172.21.15.27:0/3180198700 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015cec0 0x7fb7200b80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.892+0000 7fb725ab8640 1 -- 172.21.15.27:0/3180198700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7200cd8e0 msgr2=0x7fb72014b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.892+0000 7fb725ab8640 1 --2- 172.21.15.27:0/3180198700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7200cd8e0 0x7fb72014b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.892+0000 7fb725ab8640 1 -- 172.21.15.27:0/3180198700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb718009b30 con 0x7fb72015d2a0 2023-12-19T18:15:08.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.893+0000 7fb725ab8640 1 --2- 172.21.15.27:0/3180198700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72015d2a0 0x7fb7200cbd50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00e7f0 tx=0x7fb71c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.896+0000 7fb715ffb640 1 -- 172.21.15.27:0/3180198700 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00c940 con 0x7fb72015d2a0 2023-12-19T18:15:08.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.896+0000 7fb72751b640 1 -- 172.21.15.27:0/3180198700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7200cb550 con 0x7fb72015d2a0 2023-12-19T18:15:08.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.897+0000 7fb72751b640 1 -- 172.21.15.27:0/3180198700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb72014b6a0 con 0x7fb72015d2a0 2023-12-19T18:15:08.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.897+0000 7f298affd640 1 -- 172.21.15.27:0/4150824414 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f298c0544e0 con 0x7f299404a310 2023-12-19T18:15:08.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.897+0000 7fb715ffb640 1 -- 172.21.15.27:0/3180198700 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb71c00caa0 con 0x7fb72015d2a0 2023-12-19T18:15:08.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.897+0000 7fb715ffb640 1 -- 172.21.15.27:0/3180198700 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb71c00c940 con 0x7fb72015d2a0 2023-12-19T18:15:08.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.900+0000 7fb72751b640 1 -- 172.21.15.27:0/3180198700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7200cba70 con 0x7fb72015d2a0 2023-12-19T18:15:08.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.900+0000 7fb715ffb640 1 -- 172.21.15.27:0/3180198700 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb71c00cc10 con 0x7fb72015d2a0 2023-12-19T18:15:08.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.902+0000 7fb715ffb640 1 --2- 172.21.15.27:0/3180198700 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb70406a490 0x7fb70406c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.902+0000 7fb715ffb640 1 -- 172.21.15.27:0/3180198700 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb71c0887b0 con 0x7fb72015d2a0 2023-12-19T18:15:08.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.910+0000 7fb7252b7640 1 --2- 172.21.15.27:0/3180198700 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb70406a490 0x7fb70406c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.911+0000 7fb715ffb640 1 -- 172.21.15.27:0/3180198700 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb7200cba70 con 0x7fb72015d2a0 2023-12-19T18:15:08.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.917+0000 7fb7252b7640 1 --2- 172.21.15.27:0/3180198700 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb70406a490 0x7fb70406c950 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb710009b10 tx=0x7fb710008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.933+0000 7f3808ace640 1 Processor -- start 2023-12-19T18:15:08.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.934+0000 7f3808ace640 1 -- start start 2023-12-19T18:15:08.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.937+0000 7f3808ace640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3804153f20 0x7f380415ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.937+0000 7f3808ace640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38040cdd50 0x7f38040cf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.937+0000 7f3808ace640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3804161530 con 0x7f38040cf9f0 2023-12-19T18:15:08.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.938+0000 7f3808ace640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3804154300 con 0x7f3804153f20 2023-12-19T18:15:08.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.938+0000 7f3808ace640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38040cfe80 con 0x7f38040cdd50 2023-12-19T18:15:08.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.938+0000 7f3802d9d640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f38040cf9f0 0x7f38040b7fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:36830/0 (socket says 172.21.15.27:36830) 2023-12-19T18:15:08.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.938+0000 7f3802d9d640 1 -- 172.21.15.27:0/624842808 learned_addr learned my addr 172.21.15.27:0/624842808 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:08.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.939+0000 7f380259c640 1 --2- 172.21.15.27:0/624842808 >> v2:172.21.15.192:3300/0 conn(0x7f3804153f20 0x7f380415ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.939+0000 7f3801d9b640 1 --2- 172.21.15.27:0/624842808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38040cdd50 0x7f38040cf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.939+0000 7f380159a640 1 -- 172.21.15.27:0/624842808 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3846083624 0 0) 0x7f3804161530 con 0x7f38040cf9f0 2023-12-19T18:15:08.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.939+0000 7f4bfcdb1640 1 Processor -- start 2023-12-19T18:15:08.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.939+0000 7f380159a640 1 -- 172.21.15.27:0/624842808 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37ec003520 con 0x7f38040cf9f0 2023-12-19T18:15:08.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.939+0000 7f3801d9b640 1 -- 172.21.15.27:0/624842808 >> v2:172.21.15.192:3300/0 conn(0x7f3804153f20 msgr2=0x7f380415ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:08.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.939+0000 7f3801d9b640 1 --2- 172.21.15.27:0/624842808 >> v2:172.21.15.192:3300/0 conn(0x7f3804153f20 0x7f380415ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.940+0000 7f3801d9b640 1 -- 172.21.15.27:0/624842808 >> v1:172.21.15.192:6789/0 conn(0x7f38040cf9f0 legacy=0x7f38040b7fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.940+0000 7f3801d9b640 1 -- 172.21.15.27:0/624842808 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38040d0320 con 0x7f38040cdd50 2023-12-19T18:15:08.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.940+0000 7f4bfcdb1640 1 -- start start 2023-12-19T18:15:08.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.942+0000 7f3801d9b640 1 --2- 172.21.15.27:0/624842808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38040cdd50 0x7f38040cf4b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009790 tx=0x7f37f802b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a317332603f8736 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.942+0000 7f4bfcdb1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bf0006b60 0x7f4bf0053c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.942+0000 7f4bfcdb1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4bf0054180 0x7f4bf0057710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.942+0000 7f4bfcdb1640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf003c060 con 0x7f4bf00e3060 2023-12-19T18:15:08.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.942+0000 7f4bfcdb1640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf0054610 con 0x7f4bf0054180 2023-12-19T18:15:08.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.942+0000 7f4bfcdb1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00070e0 con 0x7f4bf0006b60 2023-12-19T18:15:08.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.942+0000 7f380159a640 1 -- 172.21.15.27:0/624842808 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8031830 con 0x7f38040cdd50 2023-12-19T18:15:08.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.942+0000 7f380159a640 1 -- 172.21.15.27:0/624842808 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f8031990 con 0x7f38040cdd50 2023-12-19T18:15:08.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.942+0000 7f380159a640 1 -- 172.21.15.27:0/624842808 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8031d60 con 0x7f38040cdd50 2023-12-19T18:15:08.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.942+0000 7f4bf659c640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f4bf00e3060 0x7f4bf003ea30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:36832/0 (socket says 172.21.15.27:36832) 2023-12-19T18:15:08.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.943+0000 7f4bf659c640 1 -- 172.21.15.27:0/940539067 learned_addr learned my addr 172.21.15.27:0/940539067 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:08.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.943+0000 7f4bf5d9b640 1 --2- 172.21.15.27:0/940539067 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bf0006b60 0x7f4bf0053c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.943+0000 7f4bf6d9d640 1 --2- 172.21.15.27:0/940539067 >> v2:172.21.15.192:3300/0 conn(0x7f4bf0054180 0x7f4bf0057710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.943+0000 7f3808ace640 1 -- 172.21.15.27:0/624842808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38040cdd50 msgr2=0x7f38040cf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.943+0000 7f3808ace640 1 --2- 172.21.15.27:0/624842808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38040cdd50 0x7f38040cf4b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f37f8009790 tx=0x7f37f802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.944+0000 7f3808ace640 1 -- 172.21.15.27:0/624842808 shutdown_connections 2023-12-19T18:15:08.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.944+0000 7f3808ace640 1 --2- 172.21.15.27:0/624842808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38040cdd50 0x7f38040cf4b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.944+0000 7f3808ace640 1 --2- 172.21.15.27:0/624842808 >> v2:172.21.15.192:3300/0 conn(0x7f3804153f20 0x7f380415ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.944+0000 7f3808ace640 1 -- 172.21.15.27:0/624842808 >> 172.21.15.27:0/624842808 conn(0x7f380406a180 msgr2=0x7f380406a590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:08.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.944+0000 7f4bf6d9d640 1 -- 172.21.15.27:0/940539067 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bf0006b60 msgr2=0x7f4bf0053c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:08.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.944+0000 7f4bf6d9d640 1 --2- 172.21.15.27:0/940539067 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bf0006b60 0x7f4bf0053c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.944+0000 7f3808ace640 1 -- 172.21.15.27:0/624842808 shutdown_connections 2023-12-19T18:15:08.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.945+0000 7f3808ace640 1 -- 172.21.15.27:0/624842808 wait complete. 2023-12-19T18:15:08.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.944+0000 7f4bf6d9d640 1 -- 172.21.15.27:0/940539067 >> v1:172.21.15.192:6789/0 conn(0x7f4bf00e3060 legacy=0x7f4bf003ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.944+0000 7f4bf6d9d640 1 -- 172.21.15.27:0/940539067 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf00073a0 con 0x7f4bf0054180 2023-12-19T18:15:08.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.946+0000 7f4bf6d9d640 1 --2- 172.21.15.27:0/940539067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0054180 0x7f4bf0057710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8079f40 tx=0x7f4bf80a60f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc02543889c91677 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.947+0000 7f4bf559a640 1 -- 172.21.15.27:0/940539067 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf80a6c90 con 0x7f4bf0054180 2023-12-19T18:15:08.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.947+0000 7f4bf559a640 1 -- 172.21.15.27:0/940539067 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf80a6df0 con 0x7f4bf0054180 2023-12-19T18:15:08.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.947+0000 7f3808ace640 1 Processor -- start 2023-12-19T18:15:08.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.947+0000 7f4bf559a640 1 -- 172.21.15.27:0/940539067 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf80ae680 con 0x7f4bf0054180 2023-12-19T18:15:08.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.948+0000 7f3808ace640 1 -- start start 2023-12-19T18:15:08.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.948+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/940539067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0054180 msgr2=0x7f4bf0057710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.948+0000 7f4bfcdb1640 1 --2- 172.21.15.27:0/940539067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0054180 0x7f4bf0057710 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8079f40 tx=0x7f4bf80a60f0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.949+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/940539067 shutdown_connections 2023-12-19T18:15:08.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.949+0000 7f4bfcdb1640 1 --2- 172.21.15.27:0/940539067 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0054180 0x7f4bf0057710 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.949+0000 7f4bfcdb1640 1 --2- 172.21.15.27:0/940539067 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bf0006b60 0x7f4bf0053c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.949+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/940539067 >> 172.21.15.27:0/940539067 conn(0x7f4bf0055a80 msgr2=0x7f4bf0055e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.949+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/940539067 shutdown_connections 2023-12-19T18:15:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.950+0000 7f3808ace640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38040cdd50 0x7f38040c57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.950+0000 7f3808ace640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38040cf9f0 0x7f38041549c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.950+0000 7f3808ace640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3804153f20 0x7f38041566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.950+0000 7f3808ace640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3804141d90 con 0x7f38040cf9f0 2023-12-19T18:15:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.950+0000 7f3808ace640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3804141fe0 con 0x7f38040cdd50 2023-12-19T18:15:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.950+0000 7f3808ace640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38041422c0 con 0x7f3804153f20 2023-12-19T18:15:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.951+0000 7f3801d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38040cf9f0 0x7f38041549c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.951+0000 7f3801d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38040cf9f0 0x7f38041549c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49762/0 (socket says 172.21.15.27:49762) 2023-12-19T18:15:08.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.951+0000 7f3801d9b640 1 -- 172.21.15.27:0/3659389586 learned_addr learned my addr 172.21.15.27:0/3659389586 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:08.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.951+0000 7f3802d9d640 1 --2- 172.21.15.27:0/3659389586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3804153f20 0x7f38041566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.949+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/940539067 wait complete. 2023-12-19T18:15:08.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.952+0000 7f3801d9b640 1 -- 172.21.15.27:0/3659389586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3804153f20 msgr2=0x7f38041566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.952+0000 7f3801d9b640 1 --2- 172.21.15.27:0/3659389586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3804153f20 0x7f38041566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.952+0000 7f3801d9b640 1 -- 172.21.15.27:0/3659389586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38040cdd50 msgr2=0x7f38040c57e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-19T18:15:08.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.952+0000 7f3801d9b640 1 --2- 172.21.15.27:0/3659389586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38040cdd50 0x7f38040c57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.952+0000 7f3801d9b640 1 -- 172.21.15.27:0/3659389586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f80093a0 con 0x7f38040cf9f0 2023-12-19T18:15:08.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.952+0000 7fc8e6270640 1 Processor -- start 2023-12-19T18:15:08.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.952+0000 7f4bfcdb1640 1 Processor -- start 2023-12-19T18:15:08.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.952+0000 7fc8e6270640 1 -- start start 2023-12-19T18:15:08.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.953+0000 7f4bfcdb1640 1 -- start start 2023-12-19T18:15:08.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.953+0000 7f3801d9b640 1 --2- 172.21.15.27:0/3659389586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38040cf9f0 0x7f38041549c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f37f8034040 tx=0x7f37f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.952+0000 7f1c71f3e640 1 Processor -- start 2023-12-19T18:15:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.952+0000 7f1c71f3e640 1 -- start start 2023-12-19T18:15:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.954+0000 7f1c71f3e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c6c0ce470 0x7f1c6c0cd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.954+0000 7f1c71f3e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c16a300 con 0x7f1c6c0cdc70 2023-12-19T18:15:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.954+0000 7f1c71f3e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c6c0ca630 con 0x7f1c6c0cc0f0 2023-12-19T18:15:08.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.954+0000 7f1c71f3e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0ce100 con 0x7f1c6c0ce470 2023-12-19T18:15:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.954+0000 7fc8e6270640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc8e00b8b50 0x7fc8e00cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.954+0000 7fc8e6270640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e00ce2a0 0x7fc8e00b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.955+0000 7f4bfcdb1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bf0006b60 0x7f4bf00d6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.955+0000 7f4bfcdb1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0054180 0x7f4bf003aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.955+0000 7f4bfcdb1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bf00e3060 0x7f4bf0039ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.955+0000 7f4bfcdb1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00e0920 con 0x7f4bf00e3060 2023-12-19T18:15:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.955+0000 7f4bfcdb1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf004cbd0 con 0x7f4bf0054180 2023-12-19T18:15:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.955+0000 7f1c6b7fe640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f1c6c0cc0f0 0x7f1c6c169bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:36840/0 (socket says 172.21.15.27:36840) 2023-12-19T18:15:08.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.955+0000 7f1c6b7fe640 1 -- 172.21.15.27:0/3127228977 learned_addr learned my addr 172.21.15.27:0/3127228977 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:08.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.955+0000 7fc8e6270640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e015d660 con 0x7fc8e015d280 2023-12-19T18:15:08.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.955+0000 7fc8e6270640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e00b8fe0 con 0x7fc8e00b8b50 2023-12-19T18:15:08.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.955+0000 7fc8e6270640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e00ce6d0 con 0x7fc8e00ce2a0 2023-12-19T18:15:08.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.955+0000 7f4bfcdb1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf00e0b70 con 0x7f4bf0006b60 2023-12-19T18:15:08.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.955+0000 7fc8df7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc8e00b8b50 0x7fc8e00cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.955+0000 7f37f2ffd640 1 -- 172.21.15.27:0/3659389586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8005b90 con 0x7f38040cf9f0 2023-12-19T18:15:08.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7fc8df7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc8e00b8b50 0x7fc8e00cdd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:50692/0 (socket says 172.21.15.27:50692) 2023-12-19T18:15:08.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7fc8df7fe640 1 -- 172.21.15.27:0/914669956 learned_addr learned my addr 172.21.15.27:0/914669956 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:08.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7f3808ace640 1 -- 172.21.15.27:0/3659389586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38041425a0 con 0x7f38040cf9f0 2023-12-19T18:15:08.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7f3a31d47640 1 Processor -- start 2023-12-19T18:15:08.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3294078528 0 0) 0x7f1c6c0ca630 con 0x7f1c6c0cc0f0 2023-12-19T18:15:08.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7fc8e480d640 1 --2- 172.21.15.27:0/914669956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e00ce2a0 0x7fc8e00b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7f37f2ffd640 1 -- 172.21.15.27:0/3659389586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37f8005cf0 con 0x7f38040cf9f0 2023-12-19T18:15:08.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7f37f2ffd640 1 -- 172.21.15.27:0/3659389586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37f8005b90 con 0x7f38040cf9f0 2023-12-19T18:15:08.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7f3808ace640 1 -- 172.21.15.27:0/3659389586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38040c5d20 con 0x7f38040cf9f0 2023-12-19T18:15:08.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7f1c62ffd640 1 --2- 172.21.15.27:0/3127228977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c6c0ce470 0x7f1c6c0cd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c58003520 con 0x7f1c6c0cc0f0 2023-12-19T18:15:08.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251765167 0 0) 0x7f1c6c16a300 con 0x7f1c6c0cdc70 2023-12-19T18:15:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c6c0ca630 con 0x7f1c6c0cdc70 2023-12-19T18:15:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7fc8e480d640 1 -- 172.21.15.27:0/914669956 >> v2:172.21.15.192:3300/0 conn(0x7fc8e00b8b50 msgr2=0x7fc8e00cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7fc8e480d640 1 --2- 172.21.15.27:0/914669956 >> v2:172.21.15.192:3300/0 conn(0x7fc8e00b8b50 0x7fc8e00cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7fc8e480d640 1 -- 172.21.15.27:0/914669956 >> v1:172.21.15.27:6789/0 conn(0x7fc8e015d280 legacy=0x7fc8e00d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7fc8e480d640 1 -- 172.21.15.27:0/914669956 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e00d1bb0 con 0x7fc8e00ce2a0 2023-12-19T18:15:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 <== mon.1 v1:172.21.15.192:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 749979164 0 0) 0x7f1c58003520 con 0x7f1c6c0cc0f0 2023-12-19T18:15:08.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c6c0ce470 msgr2=0x7f1c6c0cd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7f3a31d47640 1 -- start start 2023-12-19T18:15:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7f1c6affd640 1 --2- 172.21.15.27:0/3127228977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c6c0ce470 0x7f1c6c0cd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 >> v1:172.21.15.27:6789/0 conn(0x7f1c6c0cdc70 legacy=0x7f1c6c0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 --> v1:172.21.15.192:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c0ca910 con 0x7f1c6c0cc0f0 2023-12-19T18:15:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 <== mon.1 v1:172.21.15.192:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f1c54001670 con 0x7f1c6c0cc0f0 2023-12-19T18:15:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.956+0000 7f4bf659c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bf0006b60 0x7f4bf00d6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7f4bf5d9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0054180 0x7f4bf003aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7f4bf6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bf00e3060 0x7f4bf0039ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7f4bf659c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bf0006b60 0x7f4bf00d6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:42392/0 (socket says 172.21.15.27:42392) 2023-12-19T18:15:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7f4bf659c640 1 -- 172.21.15.27:0/2628346074 learned_addr learned my addr 172.21.15.27:0/2628346074 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.958+0000 7f4bf659c640 1 -- 172.21.15.27:0/2628346074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0054180 msgr2=0x7f4bf003aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.958+0000 7f4bf659c640 1 --2- 172.21.15.27:0/2628346074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0054180 0x7f4bf003aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.958+0000 7f4bf659c640 1 -- 172.21.15.27:0/2628346074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bf00e3060 msgr2=0x7f4bf0039ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.958+0000 7f4bf659c640 1 --2- 172.21.15.27:0/2628346074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bf00e3060 0x7f4bf0039ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.959+0000 7f4bf659c640 1 -- 172.21.15.27:0/2628346074 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf00e0e50 con 0x7f4bf0006b60 2023-12-19T18:15:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.958+0000 7f3a31d47640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3a2c06fa20 0x7f3a2c06b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.957+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 >> v1:172.21.15.192:6789/0 conn(0x7f1c6c0cc0f0 legacy=0x7f1c6c169bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.958+0000 7f1c6affd640 1 --2- 172.21.15.27:0/3127228977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c58003b00 0x7f1c58003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.958+0000 7f1c6affd640 1 --2- 172.21.15.27:0/3127228977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c580047c0 0x7f1c58004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.958+0000 7f1c6affd640 1 --2- 172.21.15.27:0/3127228977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c58005180 0x7f1c5800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.958+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0ce100 con 0x7f1c58003b00 2023-12-19T18:15:08.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.958+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5800c020 con 0x7f1c580047c0 2023-12-19T18:15:08.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.958+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5800c1e0 con 0x7f1c58005180 2023-12-19T18:15:08.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.959+0000 7f1c6bfff640 1 --2- 172.21.15.27:0/3127228977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c58005180 0x7f1c5800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.959+0000 7fc8e480d640 1 --2- 172.21.15.27:0/914669956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e00ce2a0 0x7fc8e00b6130 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40097a0 tx=0x7fc8d402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fe2e6aed3a3a48a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.959+0000 7f4bf659c640 1 --2- 172.21.15.27:0/2628346074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bf0006b60 0x7f4bf00d6fa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4be800c4d0 tx=0x7f4be800c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.960+0000 7f1c62ffd640 1 --2- 172.21.15.27:0/3127228977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c6c0ce470 0x7f1c6c0cd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:08.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.960+0000 7f1c6b7fe640 1 --2- 172.21.15.27:0/3127228977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c58003b00 0x7f1c58003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.959+0000 7fc8deffd640 1 -- 172.21.15.27:0/914669956 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4031640 con 0x7fc8e00ce2a0 2023-12-19T18:15:08.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.960+0000 7fc8deffd640 1 -- 172.21.15.27:0/914669956 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d40317a0 con 0x7fc8e00ce2a0 2023-12-19T18:15:08.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.960+0000 7fc8deffd640 1 -- 172.21.15.27:0/914669956 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4031b70 con 0x7fc8e00ce2a0 2023-12-19T18:15:08.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.960+0000 7f3808ace640 1 -- 172.21.15.27:0/3659389586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3804142400 con 0x7f38040cf9f0 2023-12-19T18:15:08.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.960+0000 7f1c6bfff640 1 -- 172.21.15.27:0/3127228977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c580047c0 msgr2=0x7f1c58004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:08.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.960+0000 7f1c6bfff640 1 --2- 172.21.15.27:0/3127228977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c580047c0 0x7f1c58004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.960+0000 7f4be6ffd640 1 -- 172.21.15.27:0/2628346074 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8005ce0 con 0x7f4bf0006b60 2023-12-19T18:15:08.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.959+0000 7f3a31d47640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a2c06ba60 0x7f3a2c06bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.959+0000 7f3a31d47640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a2c0cad60 con 0x7f3a2c161640 2023-12-19T18:15:08.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.959+0000 7f3a31d47640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a2c161a20 con 0x7f3a2c06fa20 2023-12-19T18:15:08.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.959+0000 7f3a31d47640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c06c410 con 0x7f3a2c06ba60 2023-12-19T18:15:08.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.960+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/2628346074 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf004ce40 con 0x7f4bf0006b60 2023-12-19T18:15:08.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.959+0000 7f3a2b7fe640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f3a2c161640 0x7f3a2c0711e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:36848/0 (socket says 172.21.15.27:36848) 2023-12-19T18:15:08.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.959+0000 7f3a2b7fe640 1 -- 172.21.15.27:0/179170678 learned_addr learned my addr 172.21.15.27:0/179170678 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:08.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.960+0000 7f3a2affd640 1 --2- 172.21.15.27:0/179170678 >> v2:172.21.15.27:3300/0 conn(0x7f3a2c06fa20 0x7f3a2c06b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/2628346074 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf00e1130 con 0x7f4bf0006b60 2023-12-19T18:15:08.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7f4be6ffd640 1 -- 172.21.15.27:0/2628346074 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4be8005e40 con 0x7f4bf0006b60 2023-12-19T18:15:08.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7f4be6ffd640 1 -- 172.21.15.27:0/2628346074 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4be8005ce0 con 0x7f4bf0006b60 2023-12-19T18:15:08.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7fc8e6270640 1 -- 172.21.15.27:0/914669956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e00ce2a0 msgr2=0x7fc8e00b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7fc8e6270640 1 --2- 172.21.15.27:0/914669956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e00ce2a0 0x7fc8e00b6130 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40097a0 tx=0x7fc8d402b8f0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7f3a2bfff640 1 --2- 172.21.15.27:0/179170678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a2c06ba60 0x7f3a2c06bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7f3a2a7fc640 1 -- 172.21.15.27:0/179170678 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1749741927 0 0) 0x7f3a2c0cad60 con 0x7f3a2c161640 2023-12-19T18:15:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7f3a2a7fc640 1 -- 172.21.15.27:0/179170678 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a14003520 con 0x7f3a2c161640 2023-12-19T18:15:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7f9feeacf640 1 Processor -- start 2023-12-19T18:15:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.960+0000 7f1c6bfff640 1 -- 172.21.15.27:0/3127228977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c58003b00 msgr2=0x7f1c58003f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:08.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7f3a2affd640 1 -- 172.21.15.27:0/179170678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a2c06ba60 msgr2=0x7f3a2c06bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:08.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.962+0000 7f37f2ffd640 1 -- 172.21.15.27:0/3659389586 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f37f8042030 con 0x7f38040cf9f0 2023-12-19T18:15:08.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.963+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/2628346074 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf0005780 con 0x7f4bf0006b60 2023-12-19T18:15:08.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7f1c6bfff640 1 --2- 172.21.15.27:0/3127228977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c58003b00 0x7f1c58003f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7f1c6bfff640 1 -- 172.21.15.27:0/3127228977 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c58003520 con 0x7f1c58005180 2023-12-19T18:15:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.963+0000 7f1c6b7fe640 1 --2- 172.21.15.27:0/3127228977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c58003b00 0x7f1c58003f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.963+0000 7f1c6bfff640 1 --2- 172.21.15.27:0/3127228977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c58005180 0x7f1c5800b980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1c64001fe0 tx=0x7f1c580057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aa9c0ac32ab6b5f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.962+0000 7fc8e6270640 1 -- 172.21.15.27:0/914669956 shutdown_connections 2023-12-19T18:15:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.962+0000 7fc8e6270640 1 --2- 172.21.15.27:0/914669956 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e00ce2a0 0x7fc8e00b6130 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.962+0000 7fc8e6270640 1 --2- 172.21.15.27:0/914669956 >> v2:172.21.15.192:3300/0 conn(0x7fc8e00b8b50 0x7fc8e00cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.962+0000 7fc8e6270640 1 -- 172.21.15.27:0/914669956 >> 172.21.15.27:0/914669956 conn(0x7fc8e00d2f90 msgr2=0x7fc8e00d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.962+0000 7fc8e6270640 1 -- 172.21.15.27:0/914669956 shutdown_connections 2023-12-19T18:15:08.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.962+0000 7fc8e6270640 1 -- 172.21.15.27:0/914669956 wait complete. 2023-12-19T18:15:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7f3a2affd640 1 --2- 172.21.15.27:0/179170678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a2c06ba60 0x7f3a2c06bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7f3a2affd640 1 -- 172.21.15.27:0/179170678 >> v1:172.21.15.192:6789/0 conn(0x7f3a2c161640 legacy=0x7f3a2c0711e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.961+0000 7f3a2affd640 1 -- 172.21.15.27:0/179170678 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a2c0cb000 con 0x7f3a2c06fa20 2023-12-19T18:15:08.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.963+0000 7f3a2affd640 1 --2- 172.21.15.27:0/179170678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a2c06fa20 0x7f3a2c06b520 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3a200097a0 tx=0x7f3a2002b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7a36b46f527d27 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.962+0000 7f9feeacf640 1 -- start start 2023-12-19T18:15:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.964+0000 7f9feeacf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9fe8146a50 0x7f9fe80cd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.964+0000 7f9feeacf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80cdb20 0x7f9fe8146e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.964+0000 7f9feeacf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe80cdf80 con 0x7f9fe8147370 2023-12-19T18:15:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.964+0000 7f9feeacf640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe8165c20 con 0x7f9fe8146a50 2023-12-19T18:15:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.964+0000 7f9feeacf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe8165050 con 0x7f9fe80cdb20 2023-12-19T18:15:08.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.965+0000 7fc8e6270640 1 Processor -- start 2023-12-19T18:15:08.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.965+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c64006d50 con 0x7f1c58005180 2023-12-19T18:15:08.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.965+0000 7f9fe7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80cdb20 0x7f9fe8146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.965+0000 7f37f2ffd640 1 --2- 172.21.15.27:0/3659389586 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f37e406a490 0x7f37e406c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.965+0000 7f37f2ffd640 1 -- 172.21.15.27:0/3659389586 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37f80aa6b0 con 0x7f38040cf9f0 2023-12-19T18:15:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.965+0000 7f9fec86b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9fe8146a50 0x7f9fe80cd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.965+0000 7f9fec86b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9fe8146a50 0x7f9fe80cd5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:50714/0 (socket says 172.21.15.27:50714) 2023-12-19T18:15:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.965+0000 7f9fec86b640 1 -- 172.21.15.27:0/3764803307 learned_addr learned my addr 172.21.15.27:0/3764803307 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.965+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c64003970 con 0x7f1c58005180 2023-12-19T18:15:08.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.965+0000 7f1c6affd640 1 -- 172.21.15.27:0/3127228977 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c64003d20 con 0x7f1c58005180 2023-12-19T18:15:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.965+0000 7fc8e6270640 1 -- start start 2023-12-19T18:15:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.966+0000 7f380259c640 1 --2- 172.21.15.27:0/3659389586 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f37e406a490 0x7f37e406c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.966+0000 7f9fec86b640 1 -- 172.21.15.27:0/3764803307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80cdb20 msgr2=0x7f9fe8146e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.966+0000 7f9fec86b640 1 --2- 172.21.15.27:0/3764803307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80cdb20 0x7f9fe8146e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.966+0000 7f9fec86b640 1 -- 172.21.15.27:0/3764803307 >> v1:172.21.15.27:6789/0 conn(0x7f9fe8147370 legacy=0x7f9fe806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.966+0000 7f9fec86b640 1 -- 172.21.15.27:0/3764803307 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe81652f0 con 0x7f9fe8146a50 2023-12-19T18:15:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.966+0000 7f1c71f3e640 1 -- 172.21.15.27:0/3127228977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c58005180 msgr2=0x7f1c5800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.966+0000 7f1c71f3e640 1 --2- 172.21.15.27:0/3127228977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c58005180 0x7f1c5800b980 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1c64001fe0 tx=0x7f1c580057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.967+0000 7f1c6bfff640 1 -- 172.21.15.27:0/3127228977 reap_dead start 2023-12-19T18:15:08.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.966+0000 7f4be6ffd640 1 -- 172.21.15.27:0/2628346074 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4be8007070 con 0x7f4bf0006b60 2023-12-19T18:15:08.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.967+0000 7f1c71f3e640 1 -- 172.21.15.27:0/3127228977 shutdown_connections 2023-12-19T18:15:08.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.967+0000 7f1c71f3e640 1 -- 172.21.15.27:0/3127228977 >> 172.21.15.27:0/3127228977 conn(0x7f1c6c0cb5d0 msgr2=0x7f1c6c0cb9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:08.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.967+0000 7f1c71f3e640 1 -- 172.21.15.27:0/3127228977 shutdown_connections 2023-12-19T18:15:08.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.967+0000 7fc8e6270640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8e00b8b50 0x7fc8e014daf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.967+0000 7fc8e6270640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e00ce2a0 0x7fc8e015f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.967+0000 7fc8e6270640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8e015d280 0x7fc8e015fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.967+0000 7fc8e6270640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e015def0 con 0x7fc8e00b8b50 2023-12-19T18:15:08.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.967+0000 7fc8e6270640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e014e720 con 0x7fc8e015d280 2023-12-19T18:15:08.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.967+0000 7fc8e6270640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e01fa7a0 con 0x7fc8e00ce2a0 2023-12-19T18:15:08.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.968+0000 7f380259c640 1 --2- 172.21.15.27:0/3659389586 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f37e406a490 0x7f37e406c950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f380413fbd0 tx=0x7f37f4009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.968+0000 7fc8e480d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8e015d280 0x7fc8e015fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.968+0000 7fc8df7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e00ce2a0 0x7fc8e015f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.968+0000 7f9fec86b640 1 --2- 172.21.15.27:0/3764803307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe8146a50 0x7f9fe80cd5e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009870 tx=0x7f9fd802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=28f51fd8c627ab64 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.968+0000 7f4be6ffd640 1 --2- 172.21.15.27:0/2628346074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4bd806a490 0x7f4bd806c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.968+0000 7fc8e480d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8e015d280 0x7fc8e015fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:50722/0 (socket says 172.21.15.27:50722) 2023-12-19T18:15:08.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.968+0000 7fc8dffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8e00b8b50 0x7fc8e014daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.968+0000 7fc8e480d640 1 -- 172.21.15.27:0/1060509721 learned_addr learned my addr 172.21.15.27:0/1060509721 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.967+0000 7f1c71f3e640 1 -- 172.21.15.27:0/3127228977 wait complete. 2023-12-19T18:15:08.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.969+0000 7f9fe77fe640 1 -- 172.21.15.27:0/3764803307 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8030840 con 0x7f9fe8146a50 2023-12-19T18:15:08.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.969+0000 7f4be6ffd640 1 -- 172.21.15.27:0/2628346074 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4be8087eb0 con 0x7f4bf0006b60 2023-12-19T18:15:08.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.969+0000 7f4bf5d9b640 1 --2- 172.21.15.27:0/2628346074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4bd806a490 0x7f4bd806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.969+0000 7f9fe77fe640 1 -- 172.21.15.27:0/3764803307 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd80309a0 con 0x7f9fe8146a50 2023-12-19T18:15:08.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.969+0000 7f9fe77fe640 1 -- 172.21.15.27:0/3764803307 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8030d30 con 0x7f9fe8146a50 2023-12-19T18:15:08.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.969+0000 7fc8e480d640 1 -- 172.21.15.27:0/1060509721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e00ce2a0 msgr2=0x7fc8e015f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.969+0000 7fc8e480d640 1 --2- 172.21.15.27:0/1060509721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e00ce2a0 0x7fc8e015f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.969+0000 7fc8e480d640 1 -- 172.21.15.27:0/1060509721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8e00b8b50 msgr2=0x7fc8e014daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7fc8e480d640 1 --2- 172.21.15.27:0/1060509721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8e00b8b50 0x7fc8e014daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7f9feeacf640 1 -- 172.21.15.27:0/3764803307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe8146a50 msgr2=0x7f9fe80cd5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7f9feeacf640 1 --2- 172.21.15.27:0/3764803307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe8146a50 0x7f9fe80cd5e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8009870 tx=0x7f9fd802b980 comp rx=0 tx=0).stop 2023-12-19T18:15:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7fc8e480d640 1 -- 172.21.15.27:0/1060509721 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d4009400 con 0x7fc8e015d280 2023-12-19T18:15:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7fc8dffff640 1 --2- 172.21.15.27:0/1060509721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8e00b8b50 0x7fc8e014daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:15:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7f9feeacf640 1 -- 172.21.15.27:0/3764803307 shutdown_connections 2023-12-19T18:15:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7f9feeacf640 1 --2- 172.21.15.27:0/3764803307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80cdb20 0x7f9fe8146e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7f9feeacf640 1 --2- 172.21.15.27:0/3764803307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe8146a50 0x7f9fe80cd5e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7f9feeacf640 1 -- 172.21.15.27:0/3764803307 >> 172.21.15.27:0/3764803307 conn(0x7f9fe80d2f90 msgr2=0x7f9fe80d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:08.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7f1c71f3e640 1 Processor -- start 2023-12-19T18:15:08.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7f9feeacf640 1 -- 172.21.15.27:0/3764803307 shutdown_connections 2023-12-19T18:15:08.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7f9feeacf640 1 -- 172.21.15.27:0/3764803307 wait complete. 2023-12-19T18:15:08.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7f1c71f3e640 1 -- start start 2023-12-19T18:15:08.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.970+0000 7fc8e480d640 1 --2- 172.21.15.27:0/1060509721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8e015d280 0x7fc8e015fbc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40098c0 tx=0x7fc8d40046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.971+0000 7f4bf5d9b640 1 --2- 172.21.15.27:0/2628346074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4bd806a490 0x7f4bd806c950 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00e26d0 tx=0x7f4bec018040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.971+0000 7fc8dcff9640 1 -- 172.21.15.27:0/1060509721 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d403bb70 con 0x7fc8e015d280 2023-12-19T18:15:08.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.971+0000 7fc8dcff9640 1 -- 172.21.15.27:0/1060509721 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d403bcd0 con 0x7fc8e015d280 2023-12-19T18:15:08.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.971+0000 7fc8e6270640 1 -- 172.21.15.27:0/1060509721 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e01faa80 con 0x7fc8e015d280 2023-12-19T18:15:08.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.971+0000 7fc8dcff9640 1 -- 172.21.15.27:0/1060509721 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d4045630 con 0x7fc8e015d280 2023-12-19T18:15:08.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.971+0000 7fc8e6270640 1 -- 172.21.15.27:0/1060509721 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e015db80 con 0x7fc8e015d280 2023-12-19T18:15:08.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.972+0000 7f3a2a7fc640 1 -- 172.21.15.27:0/179170678 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c001670 con 0x7f3a2c06fa20 2023-12-19T18:15:08.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.972+0000 7f1c71f3e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c58005180 0x7f1c6c207cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.972+0000 7f1c71f3e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c6c1656d0 0x7f1c6c0c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.972+0000 7f1c71f3e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c6c0c4f60 0x7f1c6c0c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.972+0000 7f1c71f3e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c2081f0 con 0x7f1c58005180 2023-12-19T18:15:08.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.972+0000 7f1c71f3e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c0c55a0 con 0x7f1c6c0c4f60 2023-12-19T18:15:08.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.972+0000 7f1c71f3e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6c088a00 con 0x7f1c6c1656d0 2023-12-19T18:15:08.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.972+0000 7f3a2a7fc640 1 -- 172.21.15.27:0/179170678 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2002e050 con 0x7f3a2c06fa20 2023-12-19T18:15:08.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.972+0000 7f1c6b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c58005180 0x7f1c6c207cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.972+0000 7f1c6b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c58005180 0x7f1c6c207cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49788/0 (socket says 172.21.15.27:49788) 2023-12-19T18:15:08.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.972+0000 7f1c6b7fe640 1 -- 172.21.15.27:0/879715232 learned_addr learned my addr 172.21.15.27:0/879715232 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:08.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.972+0000 7f37f2ffd640 1 -- 172.21.15.27:0/3659389586 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3804142400 con 0x7f38040cf9f0 2023-12-19T18:15:08.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.972+0000 7f3a2a7fc640 1 -- 172.21.15.27:0/179170678 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a20030710 con 0x7f3a2c06fa20 2023-12-19T18:15:08.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f3a31d47640 1 -- 172.21.15.27:0/179170678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a2c06fa20 msgr2=0x7f3a2c06b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f3a31d47640 1 --2- 172.21.15.27:0/179170678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a2c06fa20 0x7f3a2c06b520 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3a200097a0 tx=0x7f3a2002b6d0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f9feeacf640 1 Processor -- start 2023-12-19T18:15:08.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f1c6b7fe640 1 -- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c6c1656d0 msgr2=0x7f1c6c0c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f1c6b7fe640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c6c1656d0 0x7f1c6c0c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f1c6b7fe640 1 -- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c6c0c4f60 msgr2=0x7f1c6c0c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:08.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f1c6bfff640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c6c0c4f60 0x7f1c6c0c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f3a31d47640 1 -- 172.21.15.27:0/179170678 shutdown_connections 2023-12-19T18:15:08.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f3a31d47640 1 --2- 172.21.15.27:0/179170678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a2c06ba60 0x7f3a2c06bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f3a31d47640 1 --2- 172.21.15.27:0/179170678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a2c06fa20 0x7f3a2c06b520 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f3a31d47640 1 -- 172.21.15.27:0/179170678 >> 172.21.15.27:0/179170678 conn(0x7f3a2c0ceb50 msgr2=0x7f3a2c0cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:08.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f9feeacf640 1 -- start start 2023-12-19T18:15:08.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f1c6b7fe640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c6c0c4f60 0x7f1c6c0c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:08.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f1c6b7fe640 1 -- 172.21.15.27:0/879715232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c6c088ce0 con 0x7f1c58005180 2023-12-19T18:15:08.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f1c6bfff640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c6c0c4f60 0x7f1c6c0c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:08.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7f3a31d47640 1 -- 172.21.15.27:0/179170678 shutdown_connections 2023-12-19T18:15:08.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.973+0000 7fc8e6270640 1 -- 172.21.15.27:0/1060509721 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e015e060 con 0x7fc8e015d280 2023-12-19T18:15:08.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.974+0000 7f1c6b7fe640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c58005180 0x7f1c6c207cb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400e7f0 tx=0x7f1c5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:08.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.974+0000 7f3a31d47640 1 -- 172.21.15.27:0/179170678 wait complete. 2023-12-19T18:15:08.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.974+0000 7f1c68ff9640 1 -- 172.21.15.27:0/879715232 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400c8d0 con 0x7f1c58005180 2023-12-19T18:15:08.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.974+0000 7f1c68ff9640 1 -- 172.21.15.27:0/879715232 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5400ca30 con 0x7f1c58005180 2023-12-19T18:15:08.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.974+0000 7f1c68ff9640 1 -- 172.21.15.27:0/879715232 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400cda0 con 0x7f1c58005180 2023-12-19T18:15:08.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.974+0000 7f1c71f3e640 1 -- 172.21.15.27:0/879715232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c6c088fc0 con 0x7f1c58005180 2023-12-19T18:15:08.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.975+0000 7f1c71f3e640 1 -- 172.21.15.27:0/879715232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c6c0c3180 con 0x7f1c58005180 2023-12-19T18:15:08.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.975+0000 7f9feeacf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80cdb20 0x7f9fe80c89b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.975+0000 7f9feeacf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe8146a50 0x7f9fe815c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.975+0000 7f9feeacf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe8147370 0x7f9fe8163c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:08.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.975+0000 7f9feeacf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe80c7cc0 con 0x7f9fe8147370 2023-12-19T18:15:08.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.975+0000 7f9feeacf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81f7810 con 0x7f9fe8146a50 2023-12-19T18:15:08.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.975+0000 7f9feeacf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe80c7ed0 con 0x7f9fe80cdb20 2023-12-19T18:15:08.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.975+0000 7f9fec86b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80cdb20 0x7f9fe80c89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.975+0000 7f9fe7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe8146a50 0x7f9fe815c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:08.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.975+0000 7f9fe7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe8146a50 0x7f9fe815c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:50730/0 (socket says 172.21.15.27:50730) 2023-12-19T18:15:08.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.975+0000 7f9fe7fff640 1 -- 172.21.15.27:0/3644229163 learned_addr learned my addr 172.21.15.27:0/3644229163 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:08.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.976+0000 7f9fed06c640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe8147370 0x7f9fe8163c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:09.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.977+0000 7fc8dcff9640 1 -- 172.21.15.27:0/1060509721 <== mon.1 v2:172.21.15.192:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc8d402f050 con 0x7fc8e015d280 2023-12-19T18:15:09.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.977+0000 7f9fed06c640 1 -- 172.21.15.27:0/3644229163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80cdb20 msgr2=0x7f9fe80c89b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.977+0000 7f9fed06c640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80cdb20 0x7f9fe80c89b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.977+0000 7f9fed06c640 1 -- 172.21.15.27:0/3644229163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe8146a50 msgr2=0x7f9fe815c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.977+0000 7f1c71f3e640 1 -- 172.21.15.27:0/879715232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c6c0c5340 con 0x7f1c58005180 2023-12-19T18:15:09.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.977+0000 7f9fed06c640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe8146a50 0x7f9fe815c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.977+0000 7f9fed06c640 1 -- 172.21.15.27:0/3644229163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd80094d0 con 0x7f9fe8147370 2023-12-19T18:15:09.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.978+0000 7f1c68ff9640 1 -- 172.21.15.27:0/879715232 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1c5401d460 con 0x7f1c58005180 2023-12-19T18:15:09.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.977+0000 7f3a31d47640 1 Processor -- start 2023-12-19T18:15:09.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.977+0000 7f3a31d47640 1 -- start start 2023-12-19T18:15:09.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.977+0000 7f9fe7fff640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe8146a50 0x7f9fe815c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:09.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.977+0000 7f9fed06c640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe8147370 0x7f9fe8163c20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c310 tx=0x7f9fdc00c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:09.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.978+0000 7f9fe57fa640 1 -- 172.21.15.27:0/3644229163 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc0052d0 con 0x7f9fe8147370 2023-12-19T18:15:09.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.978+0000 7f9fe57fa640 1 -- 172.21.15.27:0/3644229163 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fdc005430 con 0x7f9fe8147370 2023-12-19T18:15:09.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.978+0000 7f9fe57fa640 1 -- 172.21.15.27:0/3644229163 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fdc003830 con 0x7f9fe8147370 2023-12-19T18:15:09.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.978+0000 7f9feeacf640 1 -- 172.21.15.27:0/3644229163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe80c81b0 con 0x7f9fe8147370 2023-12-19T18:15:09.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.979+0000 7fc8dcff9640 1 --2- 172.21.15.27:0/1060509721 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc8cc06a430 0x7fc8cc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:09.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.979+0000 7fc8dcff9640 1 -- 172.21.15.27:0/1060509721 <== mon.1 v2:172.21.15.192:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8d40aafa0 con 0x7fc8e015d280 2023-12-19T18:15:09.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.979+0000 7f3a31d47640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a2c06ba60 0x7f3a2c0b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:09.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.980+0000 7f3a31d47640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a2c161640 0x7f3a2c0b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:09.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.979+0000 7f9feeacf640 1 -- 172.21.15.27:0/3644229163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe80c8ef0 con 0x7f9fe8147370 2023-12-19T18:15:09.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.980+0000 7f3a31d47640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c1540d0 0x7f3a2c0b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:09.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.980+0000 7fc8dffff640 1 --2- 172.21.15.27:0/1060509721 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc8cc06a430 0x7fc8cc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:09.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.980+0000 7f3a31d47640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c0b27a0 con 0x7f3a2c06ba60 2023-12-19T18:15:09.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.980+0000 7f1c68ff9640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1c4c06a480 0x7f1c4c06c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:09.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.980+0000 7f1c68ff9640 1 -- 172.21.15.27:0/879715232 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c54088970 con 0x7f1c58005180 2023-12-19T18:15:09.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.981+0000 7f1c62ffd640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1c4c06a480 0x7f1c4c06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:09.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.981+0000 7f3a2affd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a2c161640 0x7f3a2c0b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:09.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.981+0000 7f3a2affd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a2c161640 0x7f3a2c0b2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:42486/0 (socket says 172.21.15.27:42486) 2023-12-19T18:15:09.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.981+0000 7f3a2affd640 1 -- 172.21.15.27:0/3223655521 learned_addr learned my addr 172.21.15.27:0/3223655521 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:09.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.981+0000 7f3a31d47640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c1635c0 con 0x7f3a2c1540d0 2023-12-19T18:15:09.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.981+0000 7f3a2bfff640 1 --2- 172.21.15.27:0/3223655521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c1540d0 0x7f3a2c0b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:09.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.980+0000 7f3a2b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a2c06ba60 0x7f3a2c0b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:09.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.981+0000 7f3a31d47640 1 -- 172.21.15.27:0/3223655521 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c164590 con 0x7f3a2c161640 2023-12-19T18:15:09.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.981+0000 7f4be6ffd640 1 -- 172.21.15.27:0/2628346074 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4be8053530 con 0x7f4bf0006b60 2023-12-19T18:15:09.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.982+0000 7f3a2b7fe640 1 -- 172.21.15.27:0/3223655521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a2c161640 msgr2=0x7f3a2c0b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.982+0000 7f3a2b7fe640 1 --2- 172.21.15.27:0/3223655521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a2c161640 0x7f3a2c0b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.982+0000 7f3a2b7fe640 1 -- 172.21.15.27:0/3223655521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c1540d0 msgr2=0x7f3a2c0b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.982+0000 7f3a2b7fe640 1 --2- 172.21.15.27:0/3223655521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c1540d0 0x7f3a2c0b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.982+0000 7f3a2b7fe640 1 -- 172.21.15.27:0/3223655521 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a20009400 con 0x7f3a2c06ba60 2023-12-19T18:15:09.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.982+0000 7f1c62ffd640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1c4c06a480 0x7f1c4c06c940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c006fd0 tx=0x7f1c5c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:09.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.982+0000 7fc8dffff640 1 --2- 172.21.15.27:0/1060509721 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc8cc06a430 0x7fc8cc06c8f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80012e0 tx=0x7fc8d8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:09.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.983+0000 7f3a2b7fe640 1 --2- 172.21.15.27:0/3223655521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a2c06ba60 0x7f3a2c0b19e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00c700 tx=0x7f3a1c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:09.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.983+0000 7f9feeacf640 1 -- 172.21.15.27:0/3644229163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe80c86d0 con 0x7f9fe8147370 2023-12-19T18:15:09.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.985+0000 7f1c68ff9640 1 -- 172.21.15.27:0/879715232 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1c54053f40 con 0x7f1c58005180 2023-12-19T18:15:09.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.986+0000 7f3a13fff640 1 -- 172.21.15.27:0/3223655521 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c0072d0 con 0x7f3a2c06ba60 2023-12-19T18:15:09.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.986+0000 7f3a31d47640 1 -- 172.21.15.27:0/3223655521 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c0b2570 con 0x7f3a2c06ba60 2023-12-19T18:15:09.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.987+0000 7f3a13fff640 1 -- 172.21.15.27:0/3223655521 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a1c007430 con 0x7f3a2c06ba60 2023-12-19T18:15:09.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.987+0000 7f3a13fff640 1 -- 172.21.15.27:0/3223655521 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1c0072d0 con 0x7f3a2c06ba60 2023-12-19T18:15:09.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.988+0000 7f3a31d47640 1 -- 172.21.15.27:0/3223655521 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c163ab0 con 0x7f3a2c06ba60 2023-12-19T18:15:09.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.989+0000 7f9fe57fa640 1 -- 172.21.15.27:0/3644229163 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9fdc003990 con 0x7f9fe8147370 2023-12-19T18:15:09.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.990+0000 7f3a31d47640 1 -- 172.21.15.27:0/3223655521 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c068e90 con 0x7f3a2c06ba60 2023-12-19T18:15:09.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.990+0000 7f9fe57fa640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fd406a430 0x7f9fd406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:09.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.991+0000 7f9fe57fa640 1 -- 172.21.15.27:0/3644229163 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fdc088080 con 0x7f9fe8147370 2023-12-19T18:15:09.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.991+0000 7fc8dcff9640 1 -- 172.21.15.27:0/1060509721 <== mon.1 v2:172.21.15.192:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc8d4076620 con 0x7fc8e015d280 2023-12-19T18:15:09.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.991+0000 7f9fec86b640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fd406a430 0x7f9fd406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:09.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.992+0000 7f9fec86b640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fd406a430 0x7f9fd406c8f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8036000 tx=0x7f9fd8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:09.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.995+0000 7f9fe57fa640 1 -- 172.21.15.27:0/3644229163 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9fdc0537c0 con 0x7f9fe8147370 2023-12-19T18:15:09.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:08.999+0000 7f3a13fff640 1 -- 172.21.15.27:0/3223655521 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3a1c0047e0 con 0x7f3a2c06ba60 2023-12-19T18:15:09.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.001+0000 7f3a13fff640 1 --2- 172.21.15.27:0/3223655521 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3a1806a490 0x7f3a1806c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:09.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.001+0000 7f3a13fff640 1 -- 172.21.15.27:0/3223655521 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3a1c087ab0 con 0x7f3a2c06ba60 2023-12-19T18:15:09.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.002+0000 7f3a2affd640 1 --2- 172.21.15.27:0/3223655521 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3a1806a490 0x7f3a1806c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:09.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.005+0000 7f3a13fff640 1 -- 172.21.15.27:0/3223655521 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3a1c0532c0 con 0x7f3a2c06ba60 2023-12-19T18:15:09.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.007+0000 7f3a2affd640 1 --2- 172.21.15.27:0/3223655521 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3a1806a490 0x7f3a1806c950 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3a2002b710 tx=0x7f3a20008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.327+0000 7f29a08af640 1 -- 172.21.15.27:0/4150824414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2994054fe0 con 0x7f299404a310 2023-12-19T18:15:09.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.329+0000 7f298affd640 1 -- 172.21.15.27:0/4150824414 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f298c054680 con 0x7f299404a310 2023-12-19T18:15:09.329 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:09.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.333+0000 7f29a08af640 1 -- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f297c06a490 msgr2=0x7f297c06c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.333+0000 7f29a08af640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f297c06a490 0x7f297c06c950 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f29940eaeb0 tx=0x7f2990008040 comp rx=0 tx=0).stop 2023-12-19T18:15:09.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.334+0000 7f29a08af640 1 -- 172.21.15.27:0/4150824414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f299404a310 msgr2=0x7f29940ea970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.334+0000 7f29a08af640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f299404a310 0x7f29940ea970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f298c00d7d0 tx=0x7f298c00dc90 comp rx=0 tx=0).stop 2023-12-19T18:15:09.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.335+0000 7f29a08af640 1 -- 172.21.15.27:0/4150824414 shutdown_connections 2023-12-19T18:15:09.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.335+0000 7f29a08af640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f297c06a490 0x7f297c06c950 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.335+0000 7f29a08af640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f29940e79a0 0x7f299404a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.335+0000 7f29a08af640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f29940e75c0 0x7f2994050080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.335+0000 7f29a08af640 1 --2- 172.21.15.27:0/4150824414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f299404a310 0x7f29940ea970 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.335+0000 7f29a08af640 1 -- 172.21.15.27:0/4150824414 >> 172.21.15.27:0/4150824414 conn(0x7f2994053780 msgr2=0x7f29940e3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:09.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.335+0000 7f29a08af640 1 -- 172.21.15.27:0/4150824414 shutdown_connections 2023-12-19T18:15:09.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.335+0000 7f29a08af640 1 -- 172.21.15.27:0/4150824414 wait complete. 2023-12-19T18:15:09.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.341+0000 7fb72751b640 1 -- 172.21.15.27:0/3180198700 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb7200cba70 con 0x7fb72015d2a0 2023-12-19T18:15:09.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.342+0000 7fb715ffb640 1 -- 172.21.15.27:0/3180198700 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb7200cba70 con 0x7fb72015d2a0 2023-12-19T18:15:09.345 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:09.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.346+0000 7fb72751b640 1 -- 172.21.15.27:0/3180198700 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb70406a490 msgr2=0x7fb70406c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.346+0000 7fb72751b640 1 --2- 172.21.15.27:0/3180198700 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb70406a490 0x7fb70406c950 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb710009b10 tx=0x7fb710008040 comp rx=0 tx=0).stop 2023-12-19T18:15:09.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.346+0000 7fb72751b640 1 -- 172.21.15.27:0/3180198700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72015d2a0 msgr2=0x7fb7200cbd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.346+0000 7fb72751b640 1 --2- 172.21.15.27:0/3180198700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72015d2a0 0x7fb7200cbd50 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb71c00e7f0 tx=0x7fb71c00ecb0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.347+0000 7fb72751b640 1 -- 172.21.15.27:0/3180198700 shutdown_connections 2023-12-19T18:15:09.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.347+0000 7fb72751b640 1 --2- 172.21.15.27:0/3180198700 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb70406a490 0x7fb70406c950 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.347+0000 7fb72751b640 1 --2- 172.21.15.27:0/3180198700 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb72015d2a0 0x7fb7200cbd50 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.347+0000 7fb72751b640 1 --2- 172.21.15.27:0/3180198700 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb72015cec0 0x7fb7200b80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.348+0000 7fb72751b640 1 --2- 172.21.15.27:0/3180198700 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7200cd8e0 0x7fb72014b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.348+0000 7fb72751b640 1 -- 172.21.15.27:0/3180198700 >> 172.21.15.27:0/3180198700 conn(0x7fb72009c140 msgr2=0x7fb7200cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:09.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.348+0000 7fb72751b640 1 -- 172.21.15.27:0/3180198700 shutdown_connections 2023-12-19T18:15:09.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.348+0000 7fb72751b640 1 -- 172.21.15.27:0/3180198700 wait complete. 2023-12-19T18:15:09.364 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-12-19T18:15:09.364 DEBUG:teuthology.parallel:result is None 2023-12-19T18:15:09.376 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-12-19T18:15:09.376 DEBUG:teuthology.parallel:result is None 2023-12-19T18:15:09.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.381+0000 7f3808ace640 1 -- 172.21.15.27:0/3659389586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f3804142400 con 0x7f38040cf9f0 2023-12-19T18:15:09.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.382+0000 7f37f2ffd640 1 -- 172.21.15.27:0/3659389586 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3804142400 con 0x7f38040cf9f0 2023-12-19T18:15:09.383 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2023-12-19T18:15:09.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.384+0000 7f3808ace640 1 -- 172.21.15.27:0/3659389586 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f37e406a490 msgr2=0x7f37e406c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.384+0000 7f3808ace640 1 --2- 172.21.15.27:0/3659389586 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f37e406a490 0x7f37e406c950 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f380413fbd0 tx=0x7f37f4009190 comp rx=0 tx=0).stop 2023-12-19T18:15:09.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.384+0000 7f3808ace640 1 -- 172.21.15.27:0/3659389586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38040cf9f0 msgr2=0x7f38041549c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.384+0000 7f3808ace640 1 --2- 172.21.15.27:0/3659389586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38040cf9f0 0x7f38041549c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f37f8034040 tx=0x7f37f80046c0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.384+0000 7f3808ace640 1 -- 172.21.15.27:0/3659389586 shutdown_connections 2023-12-19T18:15:09.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.384+0000 7f3808ace640 1 --2- 172.21.15.27:0/3659389586 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f37e406a490 0x7f37e406c950 secure :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f380413fbd0 tx=0x7f37f4009190 comp rx=0 tx=0).stop 2023-12-19T18:15:09.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.385+0000 7f3808ace640 1 --2- 172.21.15.27:0/3659389586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3804153f20 0x7f38041566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.385+0000 7f3808ace640 1 --2- 172.21.15.27:0/3659389586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38040cf9f0 0x7f38041549c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.385+0000 7f3808ace640 1 --2- 172.21.15.27:0/3659389586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38040cdd50 0x7f38040c57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.385+0000 7f3808ace640 1 -- 172.21.15.27:0/3659389586 >> 172.21.15.27:0/3659389586 conn(0x7f380406a180 msgr2=0x7f3804151920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:09.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.385+0000 7f3808ace640 1 -- 172.21.15.27:0/3659389586 shutdown_connections 2023-12-19T18:15:09.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.385+0000 7f3808ace640 1 -- 172.21.15.27:0/3659389586 wait complete. 2023-12-19T18:15:09.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.389+0000 7fc8e6270640 1 -- 172.21.15.27:0/1060509721 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fc8e0067110 con 0x7fc8e015d280 2023-12-19T18:15:09.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.390+0000 7fc8dcff9640 1 -- 172.21.15.27:0/1060509721 <== mon.1 v2:172.21.15.192:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc8d4053020 con 0x7fc8e015d280 2023-12-19T18:15:09.391 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:09.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.393+0000 7fc8e6270640 1 -- 172.21.15.27:0/1060509721 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc8cc06a430 msgr2=0x7fc8cc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.393+0000 7fc8e6270640 1 --2- 172.21.15.27:0/1060509721 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc8cc06a430 0x7fc8cc06c8f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fc8d80012e0 tx=0x7fc8d8009040 comp rx=0 tx=0).stop 2023-12-19T18:15:09.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.393+0000 7fc8e6270640 1 -- 172.21.15.27:0/1060509721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8e015d280 msgr2=0x7fc8e015fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.393+0000 7fc8e6270640 1 --2- 172.21.15.27:0/1060509721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8e015d280 0x7fc8e015fbc0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc8d40098c0 tx=0x7fc8d40046c0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.393+0000 7fc8e6270640 1 -- 172.21.15.27:0/1060509721 shutdown_connections 2023-12-19T18:15:09.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.393+0000 7fc8e6270640 1 --2- 172.21.15.27:0/1060509721 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc8cc06a430 0x7fc8cc06c8f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.393+0000 7fc8e6270640 1 --2- 172.21.15.27:0/1060509721 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8e015d280 0x7fc8e015fbc0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.393+0000 7fc8e6270640 1 --2- 172.21.15.27:0/1060509721 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8e00ce2a0 0x7fc8e015f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.393+0000 7fc8e6270640 1 --2- 172.21.15.27:0/1060509721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8e00b8b50 0x7fc8e014daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.393+0000 7fc8e6270640 1 -- 172.21.15.27:0/1060509721 >> 172.21.15.27:0/1060509721 conn(0x7fc8e00d2f90 msgr2=0x7fc8e0154560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:09.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.393+0000 7fc8e6270640 1 -- 172.21.15.27:0/1060509721 shutdown_connections 2023-12-19T18:15:09.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.393+0000 7fc8e6270640 1 -- 172.21.15.27:0/1060509721 wait complete. 2023-12-19T18:15:09.402 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.1 2023-12-19T18:15:09.402 DEBUG:teuthology.parallel:result is None 2023-12-19T18:15:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.407+0000 7f1c71f3e640 1 -- 172.21.15.27:0/879715232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1c6c068e90 con 0x7f1c58005180 2023-12-19T18:15:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.407+0000 7f1c68ff9640 1 -- 172.21.15.27:0/879715232 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1c54057560 con 0x7f1c58005180 2023-12-19T18:15:09.407 INFO:teuthology.orchestra.run.smithi027.stdout:55834574855 2023-12-19T18:15:09.410 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-12-19T18:15:09.410 DEBUG:teuthology.parallel:result is None 2023-12-19T18:15:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.411+0000 7f1c71f3e640 1 -- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1c4c06a480 msgr2=0x7f1c4c06c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.411+0000 7f1c71f3e640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1c4c06a480 0x7f1c4c06c940 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c006fd0 tx=0x7f1c5c008040 comp rx=0 tx=0).stop 2023-12-19T18:15:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.411+0000 7f1c71f3e640 1 -- 172.21.15.27:0/879715232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c58005180 msgr2=0x7f1c6c207cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.411+0000 7f1c71f3e640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c58005180 0x7f1c6c207cb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f1c5400e7f0 tx=0x7f1c5400ecb0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.411+0000 7f1c71f3e640 1 -- 172.21.15.27:0/879715232 shutdown_connections 2023-12-19T18:15:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.411+0000 7f1c71f3e640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1c4c06a480 0x7f1c4c06c940 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.411+0000 7f1c71f3e640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c6c0c4f60 0x7f1c6c0c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.411+0000 7f1c71f3e640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c6c1656d0 0x7f1c6c0c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.411+0000 7f1c71f3e640 1 --2- 172.21.15.27:0/879715232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c58005180 0x7f1c6c207cb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.411+0000 7f1c71f3e640 1 -- 172.21.15.27:0/879715232 >> 172.21.15.27:0/879715232 conn(0x7f1c6c0cb5d0 msgr2=0x7f1c6c06f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:09.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.411+0000 7f1c71f3e640 1 -- 172.21.15.27:0/879715232 shutdown_connections 2023-12-19T18:15:09.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.411+0000 7f1c71f3e640 1 -- 172.21.15.27:0/879715232 wait complete. 2023-12-19T18:15:09.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.413+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/2628346074 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f4bf00527d0 con 0x7f4bf0006b60 2023-12-19T18:15:09.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.414+0000 7f4be6ffd640 1 -- 172.21.15.27:0/2628346074 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4be8056b50 con 0x7f4bf0006b60 2023-12-19T18:15:09.414 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:09.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.417+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/2628346074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4bd806a490 msgr2=0x7f4bd806c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.417+0000 7f4bfcdb1640 1 --2- 172.21.15.27:0/2628346074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4bd806a490 0x7f4bd806c950 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00e26d0 tx=0x7f4bec018040 comp rx=0 tx=0).stop 2023-12-19T18:15:09.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.417+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/2628346074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bf0006b60 msgr2=0x7f4bf00d6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.417+0000 7f4bfcdb1640 1 --2- 172.21.15.27:0/2628346074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bf0006b60 0x7f4bf00d6fa0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4be800c4d0 tx=0x7f4be800c990 comp rx=0 tx=0).stop 2023-12-19T18:15:09.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.417+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/2628346074 shutdown_connections 2023-12-19T18:15:09.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.417+0000 7f4bfcdb1640 1 --2- 172.21.15.27:0/2628346074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4bd806a490 0x7f4bd806c950 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.417+0000 7f4bfcdb1640 1 --2- 172.21.15.27:0/2628346074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bf00e3060 0x7f4bf0039ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.417+0000 7f4bfcdb1640 1 --2- 172.21.15.27:0/2628346074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4bf0054180 0x7f4bf003aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.417+0000 7f4bfcdb1640 1 --2- 172.21.15.27:0/2628346074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4bf0006b60 0x7f4bf00d6fa0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.417+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/2628346074 >> 172.21.15.27:0/2628346074 conn(0x7f4bf0055a80 msgr2=0x7f4bf0050fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:09.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.417+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/2628346074 shutdown_connections 2023-12-19T18:15:09.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.417+0000 7f4bfcdb1640 1 -- 172.21.15.27:0/2628346074 wait complete. 2023-12-19T18:15:09.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.425+0000 7f3a31d47640 1 -- 172.21.15.27:0/3223655521 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f3a2c0ca980 con 0x7f3a2c06ba60 2023-12-19T18:15:09.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.426+0000 7f3a13fff640 1 -- 172.21.15.27:0/3223655521 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f3a1c0568e0 con 0x7f3a2c06ba60 2023-12-19T18:15:09.426 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:09.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.428+0000 7f3a31d47640 1 -- 172.21.15.27:0/3223655521 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3a1806a490 msgr2=0x7f3a1806c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.428+0000 7f3a31d47640 1 --2- 172.21.15.27:0/3223655521 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3a1806a490 0x7f3a1806c950 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3a2002b710 tx=0x7f3a20008030 comp rx=0 tx=0).stop 2023-12-19T18:15:09.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.428+0000 7f3a31d47640 1 -- 172.21.15.27:0/3223655521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a2c06ba60 msgr2=0x7f3a2c0b19e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.428+0000 7f3a31d47640 1 --2- 172.21.15.27:0/3223655521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a2c06ba60 0x7f3a2c0b19e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3a1c00c700 tx=0x7f3a1c00cbc0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.428+0000 7f3a31d47640 1 -- 172.21.15.27:0/3223655521 shutdown_connections 2023-12-19T18:15:09.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.428+0000 7f3a31d47640 1 --2- 172.21.15.27:0/3223655521 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3a1806a490 0x7f3a1806c950 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.428+0000 7f3a31d47640 1 --2- 172.21.15.27:0/3223655521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a2c1540d0 0x7f3a2c0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.428+0000 7f3a31d47640 1 --2- 172.21.15.27:0/3223655521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a2c161640 0x7f3a2c0b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.428+0000 7f3a31d47640 1 --2- 172.21.15.27:0/3223655521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a2c06ba60 0x7f3a2c0b19e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.428+0000 7f3a31d47640 1 -- 172.21.15.27:0/3223655521 >> 172.21.15.27:0/3223655521 conn(0x7f3a2c0ceb50 msgr2=0x7f3a2c15fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:09.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.428+0000 7f3a31d47640 1 -- 172.21.15.27:0/3223655521 shutdown_connections 2023-12-19T18:15:09.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.428+0000 7f3a31d47640 1 -- 172.21.15.27:0/3223655521 wait complete. 2023-12-19T18:15:09.430 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-12-19T18:15:09.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.430+0000 7f9feeacf640 1 -- 172.21.15.27:0/3644229163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9fe80c86d0 con 0x7f9fe8147370 2023-12-19T18:15:09.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.430+0000 7f9fe57fa640 1 -- 172.21.15.27:0/3644229163 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9fdc017020 con 0x7f9fe8147370 2023-12-19T18:15:09.431 DEBUG:teuthology.parallel:result is None 2023-12-19T18:15:09.431 INFO:teuthology.orchestra.run.smithi027.stdout:55834574854 2023-12-19T18:15:09.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.432+0000 7f9feeacf640 1 -- 172.21.15.27:0/3644229163 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fd406a430 msgr2=0x7f9fd406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.432+0000 7f9feeacf640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fd406a430 0x7f9fd406c8f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8036000 tx=0x7f9fd8008030 comp rx=0 tx=0).stop 2023-12-19T18:15:09.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.432+0000 7f9feeacf640 1 -- 172.21.15.27:0/3644229163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe8147370 msgr2=0x7f9fe8163c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.432+0000 7f9feeacf640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe8147370 0x7f9fe8163c20 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc00c310 tx=0x7f9fdc00c7d0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.432+0000 7f9feeacf640 1 -- 172.21.15.27:0/3644229163 shutdown_connections 2023-12-19T18:15:09.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.432+0000 7f9feeacf640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fd406a430 0x7f9fd406c8f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.432+0000 7f9feeacf640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe8147370 0x7f9fe8163c20 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.432+0000 7f9feeacf640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe8146a50 0x7f9fe815c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.432+0000 7f9feeacf640 1 --2- 172.21.15.27:0/3644229163 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80cdb20 0x7f9fe80c89b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.432+0000 7f9feeacf640 1 -- 172.21.15.27:0/3644229163 >> 172.21.15.27:0/3644229163 conn(0x7f9fe80d2f90 msgr2=0x7f9fe80cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:09.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.432+0000 7f9feeacf640 1 -- 172.21.15.27:0/3644229163 shutdown_connections 2023-12-19T18:15:09.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.432+0000 7f9feeacf640 1 -- 172.21.15.27:0/3644229163 wait complete. 2023-12-19T18:15:09.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-12-19T18:15:09.435 DEBUG:teuthology.parallel:result is None 2023-12-19T18:15:09.446 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-12-19T18:15:09.446 DEBUG:teuthology.parallel:result is None 2023-12-19T18:15:09.449 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.4 2023-12-19T18:15:09.449 DEBUG:teuthology.parallel:result is None 2023-12-19T18:15:09.449 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-19T18:15:09.449 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:15:09.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.721+0000 7f292e4b6640 1 Processor -- start 2023-12-19T18:15:09.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.721+0000 7f292e4b6640 1 -- start start 2023-12-19T18:15:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.723+0000 7f292e4b6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f292806bbf0 0x7f29280cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.723+0000 7f292e4b6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f292806c880 0x7f29280cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.723+0000 7f292e4b6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f292815b180 con 0x7f2928159c40 2023-12-19T18:15:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.723+0000 7f292e4b6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29280ca850 con 0x7f292806c880 2023-12-19T18:15:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.723+0000 7f292e4b6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f292815a570 con 0x7f292806bbf0 2023-12-19T18:15:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.723+0000 7f2927fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2928159c40 0x7f292806b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54080/0 (socket says 172.21.15.27:54080) 2023-12-19T18:15:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.723+0000 7f2927fff640 1 -- 172.21.15.27:0/1220799199 learned_addr learned my addr 172.21.15.27:0/1220799199 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.723+0000 7f292ca53640 1 --2- 172.21.15.27:0/1220799199 >> v2:172.21.15.27:3300/0 conn(0x7f292806c880 0x7f29280cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.724+0000 7f291ffff640 1 --2- 172.21.15.27:0/1220799199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f292806bbf0 0x7f29280cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.724+0000 7f29277fe640 1 -- 172.21.15.27:0/1220799199 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427826643 0 0) 0x7f292815b180 con 0x7f2928159c40 2023-12-19T18:15:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.724+0000 7f29277fe640 1 -- 172.21.15.27:0/1220799199 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2914005730 con 0x7f2928159c40 2023-12-19T18:15:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.724+0000 7f292ca53640 1 -- 172.21.15.27:0/1220799199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f292806bbf0 msgr2=0x7f29280cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.724+0000 7f292ca53640 1 --2- 172.21.15.27:0/1220799199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f292806bbf0 0x7f29280cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.724+0000 7f292ca53640 1 -- 172.21.15.27:0/1220799199 >> v1:172.21.15.27:6789/0 conn(0x7f2928159c40 legacy=0x7f292806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.724+0000 7f292ca53640 1 -- 172.21.15.27:0/1220799199 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f292815a850 con 0x7f292806c880 2023-12-19T18:15:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.725+0000 7f292ca53640 1 --2- 172.21.15.27:0/1220799199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f292806c880 0x7f29280cdd40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2920009d90 tx=0x7f292002bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc75354d142b3178 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.725+0000 7f29277fe640 1 -- 172.21.15.27:0/1220799199 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292002cc50 con 0x7f292806c880 2023-12-19T18:15:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.725+0000 7f29277fe640 1 -- 172.21.15.27:0/1220799199 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292002cdb0 con 0x7f292806c880 2023-12-19T18:15:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.725+0000 7f29277fe640 1 -- 172.21.15.27:0/1220799199 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2920034640 con 0x7f292806c880 2023-12-19T18:15:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.726+0000 7f292e4b6640 1 -- 172.21.15.27:0/1220799199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f292806c880 msgr2=0x7f29280cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.726+0000 7f292e4b6640 1 --2- 172.21.15.27:0/1220799199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f292806c880 0x7f29280cdd40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2920009d90 tx=0x7f292002bea0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.726+0000 7f292e4b6640 1 -- 172.21.15.27:0/1220799199 shutdown_connections 2023-12-19T18:15:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.726+0000 7f292e4b6640 1 --2- 172.21.15.27:0/1220799199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f292806c880 0x7f29280cdd40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.726+0000 7f292e4b6640 1 --2- 172.21.15.27:0/1220799199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f292806bbf0 0x7f29280cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.726+0000 7f292e4b6640 1 -- 172.21.15.27:0/1220799199 >> 172.21.15.27:0/1220799199 conn(0x7f292806a330 msgr2=0x7f29280b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.726+0000 7f292e4b6640 1 -- 172.21.15.27:0/1220799199 shutdown_connections 2023-12-19T18:15:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.726+0000 7f292e4b6640 1 -- 172.21.15.27:0/1220799199 wait complete. 2023-12-19T18:15:09.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.728+0000 7f292e4b6640 1 Processor -- start 2023-12-19T18:15:09.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.728+0000 7f292e4b6640 1 -- start start 2023-12-19T18:15:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.729+0000 7f292e4b6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f292806bbf0 0x7f29280b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.729+0000 7f292e4b6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f292806c880 0x7f29280c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.729+0000 7f292e4b6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2928159c40 0x7f29280c5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.729+0000 7f292e4b6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29280b2b30 con 0x7f2928159c40 2023-12-19T18:15:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.729+0000 7f292e4b6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29280b1eb0 con 0x7f292806bbf0 2023-12-19T18:15:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.729+0000 7f292e4b6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29280b2190 con 0x7f292806c880 2023-12-19T18:15:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.729+0000 7f292ca53640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2928159c40 0x7f29280c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.729+0000 7f292ca53640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2928159c40 0x7f29280c5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49822/0 (socket says 172.21.15.27:49822) 2023-12-19T18:15:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.729+0000 7f292ca53640 1 -- 172.21.15.27:0/286221883 learned_addr learned my addr 172.21.15.27:0/286221883 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.730+0000 7f2927fff640 1 --2- 172.21.15.27:0/286221883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f292806bbf0 0x7f29280b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.730+0000 7f292ca53640 1 -- 172.21.15.27:0/286221883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f292806c880 msgr2=0x7f29280c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.730+0000 7f292ca53640 1 --2- 172.21.15.27:0/286221883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f292806c880 0x7f29280c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.730+0000 7f292ca53640 1 -- 172.21.15.27:0/286221883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f292806bbf0 msgr2=0x7f29280b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.730+0000 7f292ca53640 1 --2- 172.21.15.27:0/286221883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f292806bbf0 0x7f29280b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.730+0000 7f292ca53640 1 -- 172.21.15.27:0/286221883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2920009a70 con 0x7f2928159c40 2023-12-19T18:15:09.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.730+0000 7f292ca53640 1 --2- 172.21.15.27:0/286221883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2928159c40 0x7f29280c5300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2920009eb0 tx=0x7f2920004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.730+0000 7f29257fa640 1 -- 172.21.15.27:0/286221883 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292003f420 con 0x7f2928159c40 2023-12-19T18:15:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.731+0000 7f29257fa640 1 -- 172.21.15.27:0/286221883 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292003f5a0 con 0x7f2928159c40 2023-12-19T18:15:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.731+0000 7f292e4b6640 1 -- 172.21.15.27:0/286221883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29280b2470 con 0x7f2928159c40 2023-12-19T18:15:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.731+0000 7f29257fa640 1 -- 172.21.15.27:0/286221883 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292003f990 con 0x7f2928159c40 2023-12-19T18:15:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.731+0000 7f292e4b6640 1 -- 172.21.15.27:0/286221883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29280b3d60 con 0x7f2928159c40 2023-12-19T18:15:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.732+0000 7f292e4b6640 1 -- 172.21.15.27:0/286221883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29280b2c70 con 0x7f2928159c40 2023-12-19T18:15:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.733+0000 7f29257fa640 1 -- 172.21.15.27:0/286221883 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f292003faf0 con 0x7f2928159c40 2023-12-19T18:15:09.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.734+0000 7f29257fa640 1 --2- 172.21.15.27:0/286221883 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f290806a430 0x7f290806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:09.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.734+0000 7f29257fa640 1 -- 172.21.15.27:0/286221883 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f29200ab3b0 con 0x7f2928159c40 2023-12-19T18:15:09.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.734+0000 7f2927fff640 1 --2- 172.21.15.27:0/286221883 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f290806a430 0x7f290806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:09.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.735+0000 7f2927fff640 1 --2- 172.21.15.27:0/286221883 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f290806a430 0x7f290806c8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2918005880 tx=0x7f2918009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.738+0000 7f29257fa640 1 -- 172.21.15.27:0/286221883 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f29280b2c70 con 0x7f2928159c40 2023-12-19T18:15:09.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.919+0000 7f292e4b6640 1 -- 172.21.15.27:0/286221883 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f292815ae10 con 0x7f290806a430 2023-12-19T18:15:09.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.939+0000 7f29257fa640 1 -- 172.21.15.27:0/286221883 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260043 (secure 0 0 0) 0x7f2918001c60 con 0x7f290806a430 2023-12-19T18:15:09.943 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:09.944 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:15:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.944+0000 7f292e4b6640 1 -- 172.21.15.27:0/286221883 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f290806a430 msgr2=0x7f290806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.944+0000 7f292e4b6640 1 --2- 172.21.15.27:0/286221883 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f290806a430 0x7f290806c8f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f2918005880 tx=0x7f2918009040 comp rx=0 tx=0).stop 2023-12-19T18:15:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.944+0000 7f292e4b6640 1 -- 172.21.15.27:0/286221883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2928159c40 msgr2=0x7f29280c5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.944+0000 7f292e4b6640 1 --2- 172.21.15.27:0/286221883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2928159c40 0x7f29280c5300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2920009eb0 tx=0x7f2920004a40 comp rx=0 tx=0).stop 2023-12-19T18:15:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.944+0000 7f292e4b6640 1 -- 172.21.15.27:0/286221883 shutdown_connections 2023-12-19T18:15:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.944+0000 7f292e4b6640 1 --2- 172.21.15.27:0/286221883 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f290806a430 0x7f290806c8f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.944+0000 7f292e4b6640 1 --2- 172.21.15.27:0/286221883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2928159c40 0x7f29280c5300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.944+0000 7f292e4b6640 1 --2- 172.21.15.27:0/286221883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f292806c880 0x7f29280c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.944+0000 7f292e4b6640 1 --2- 172.21.15.27:0/286221883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f292806bbf0 0x7f29280b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.944+0000 7f292e4b6640 1 -- 172.21.15.27:0/286221883 >> 172.21.15.27:0/286221883 conn(0x7f292806a330 msgr2=0x7f292815be10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.944+0000 7f292e4b6640 1 -- 172.21.15.27:0/286221883 shutdown_connections 2023-12-19T18:15:09.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:09.944+0000 7f292e4b6640 1 -- 172.21.15.27:0/286221883 wait complete. 2023-12-19T18:15:09.962 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-19T18:15:09.707231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248636,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004644"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056823+0000","last_change":"2023-12-19T18:14:54.056711+0000","last_active":"2023-12-19T18:14:54.056823+0000","last_peered":"2023-12-19T18:14:54.056823+0000","last_clean":"2023-12-19T18:14:54.056823+0000","last_became_active":"2023-12-19T18:14:54.056335+0000","last_became_peered":"2023-12-19T18:14:54.056335+0000","last_unstale":"2023-12-19T18:14:54.056823+0000","last_undegraded":"2023-12-19T18:14:54.056823+0000","last_fullsized":"2023-12-19T18:14:54.056823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069974+0000","last_change":"2023-12-19T18:14:54.069882+0000","last_active":"2023-12-19T18:14:54.069974+0000","last_peered":"2023-12-19T18:14:54.069974+0000","last_clean":"2023-12-19T18:14:54.069974+0000","last_became_active":"2023-12-19T18:14:54.069626+0000","last_became_peered":"2023-12-19T18:14:54.069626+0000","last_unstale":"2023-12-19T18:14:54.069974+0000","last_undegraded":"2023-12-19T18:14:54.069974+0000","last_fullsized":"2023-12-19T18:14:54.069974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053578+0000","last_change":"2023-12-19T18:14:54.053111+0000","last_active":"2023-12-19T18:14:54.053578+0000","last_peered":"2023-12-19T18:14:54.053578+0000","last_clean":"2023-12-19T18:14:54.053578+0000","last_became_active":"2023-12-19T18:14:54.052858+0000","last_became_peered":"2023-12-19T18:14:54.052858+0000","last_unstale":"2023-12-19T18:14:54.053578+0000","last_undegraded":"2023-12-19T18:14:54.053578+0000","last_fullsized":"2023-12-19T18:14:54.053578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050310+0000","last_change":"2023-12-19T18:14:54.049862+0000","last_active":"2023-12-19T18:14:54.050310+0000","last_peered":"2023-12-19T18:14:54.050310+0000","last_clean":"2023-12-19T18:14:54.050310+0000","last_became_active":"2023-12-19T18:14:54.049396+0000","last_became_peered":"2023-12-19T18:14:54.049396+0000","last_unstale":"2023-12-19T18:14:54.050310+0000","last_undegraded":"2023-12-19T18:14:54.050310+0000","last_fullsized":"2023-12-19T18:14:54.050310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053036+0000","last_change":"2023-12-19T18:14:54.052387+0000","last_active":"2023-12-19T18:14:54.053036+0000","last_peered":"2023-12-19T18:14:54.053036+0000","last_clean":"2023-12-19T18:14:54.053036+0000","last_became_active":"2023-12-19T18:14:54.052008+0000","last_became_peered":"2023-12-19T18:14:54.052008+0000","last_unstale":"2023-12-19T18:14:54.053036+0000","last_undegraded":"2023-12-19T18:14:54.053036+0000","last_fullsized":"2023-12-19T18:14:54.053036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061254+0000","last_change":"2023-12-19T18:14:54.061175+0000","last_active":"2023-12-19T18:14:54.061254+0000","last_peered":"2023-12-19T18:14:54.061254+0000","last_clean":"2023-12-19T18:14:54.061254+0000","last_became_active":"2023-12-19T18:14:54.060192+0000","last_became_peered":"2023-12-19T18:14:54.060192+0000","last_unstale":"2023-12-19T18:14:54.061254+0000","last_undegraded":"2023-12-19T18:14:54.061254+0000","last_fullsized":"2023-12-19T18:14:54.061254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.054262+0000","last_change":"2023-12-19T18:14:54.053283+0000","last_active":"2023-12-19T18:14:54.054262+0000","last_peered":"2023-12-19T18:14:54.054262+0000","last_clean":"2023-12-19T18:14:54.054262+0000","last_became_active":"2023-12-19T18:14:54.052854+0000","last_became_peered":"2023-12-19T18:14:54.052854+0000","last_unstale":"2023-12-19T18:14:54.054262+0000","last_undegraded":"2023-12-19T18:14:54.054262+0000","last_fullsized":"2023-12-19T18:14:54.054262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056579+0000","last_change":"2023-12-19T18:14:54.056483+0000","last_active":"2023-12-19T18:14:54.056579+0000","last_peered":"2023-12-19T18:14:54.056579+0000","last_clean":"2023-12-19T18:14:54.056579+0000","last_became_active":"2023-12-19T18:14:54.056196+0000","last_became_peered":"2023-12-19T18:14:54.056196+0000","last_unstale":"2023-12-19T18:14:54.056579+0000","last_undegraded":"2023-12-19T18:14:54.056579+0000","last_fullsized":"2023-12-19T18:14:54.056579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075945+0000","last_change":"2023-12-19T18:14:54.074709+0000","last_active":"2023-12-19T18:14:54.075945+0000","last_peered":"2023-12-19T18:14:54.075945+0000","last_clean":"2023-12-19T18:14:54.075945+0000","last_became_active":"2023-12-19T18:14:54.074482+0000","last_became_peered":"2023-12-19T18:14:54.074482+0000","last_unstale":"2023-12-19T18:14:54.075945+0000","last_undegraded":"2023-12-19T18:14:54.075945+0000","last_fullsized":"2023-12-19T18:14:54.075945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051054+0000","last_change":"2023-12-19T18:14:54.050203+0000","last_active":"2023-12-19T18:14:54.051054+0000","last_peered":"2023-12-19T18:14:54.051054+0000","last_clean":"2023-12-19T18:14:54.051054+0000","last_became_active":"2023-12-19T18:14:54.049582+0000","last_became_peered":"2023-12-19T18:14:54.049582+0000","last_unstale":"2023-12-19T18:14:54.051054+0000","last_undegraded":"2023-12-19T18:14:54.051054+0000","last_fullsized":"2023-12-19T18:14:54.051054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056806+0000","last_change":"2023-12-19T18:14:54.056564+0000","last_active":"2023-12-19T18:14:54.056806+0000","last_peered":"2023-12-19T18:14:54.056806+0000","last_clean":"2023-12-19T18:14:54.056806+0000","last_became_active":"2023-12-19T18:14:54.056052+0000","last_became_peered":"2023-12-19T18:14:54.056052+0000","last_unstale":"2023-12-19T18:14:54.056806+0000","last_undegraded":"2023-12-19T18:14:54.056806+0000","last_fullsized":"2023-12-19T18:14:54.056806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066528+0000","last_change":"2023-12-19T18:14:54.066257+0000","last_active":"2023-12-19T18:14:54.066528+0000","last_peered":"2023-12-19T18:14:54.066528+0000","last_clean":"2023-12-19T18:14:54.066528+0000","last_became_active":"2023-12-19T18:14:54.065876+0000","last_became_peered":"2023-12-19T18:14:54.065876+0000","last_unstale":"2023-12-19T18:14:54.066528+0000","last_undegraded":"2023-12-19T18:14:54.066528+0000","last_fullsized":"2023-12-19T18:14:54.066528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070727+0000","last_change":"2023-12-19T18:14:54.070658+0000","last_active":"2023-12-19T18:14:54.070727+0000","last_peered":"2023-12-19T18:14:54.070727+0000","last_clean":"2023-12-19T18:14:54.070727+0000","last_became_active":"2023-12-19T18:14:54.065952+0000","last_became_peered":"2023-12-19T18:14:54.065952+0000","last_unstale":"2023-12-19T18:14:54.070727+0000","last_undegraded":"2023-12-19T18:14:54.070727+0000","last_fullsized":"2023-12-19T18:14:54.070727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:35:28.203585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072334+0000","last_change":"2023-12-19T18:14:54.072211+0000","last_active":"2023-12-19T18:14:54.072334+0000","last_peered":"2023-12-19T18:14:54.072334+0000","last_clean":"2023-12-19T18:14:54.072334+0000","last_became_active":"2023-12-19T18:14:54.071912+0000","last_became_peered":"2023-12-19T18:14:54.071912+0000","last_unstale":"2023-12-19T18:14:54.072334+0000","last_undegraded":"2023-12-19T18:14:54.072334+0000","last_fullsized":"2023-12-19T18:14:54.072334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056480+0000","last_change":"2023-12-19T18:14:54.055761+0000","last_active":"2023-12-19T18:14:54.056480+0000","last_peered":"2023-12-19T18:14:54.056480+0000","last_clean":"2023-12-19T18:14:54.056480+0000","last_became_active":"2023-12-19T18:14:54.055529+0000","last_became_peered":"2023-12-19T18:14:54.055529+0000","last_unstale":"2023-12-19T18:14:54.056480+0000","last_undegraded":"2023-12-19T18:14:54.056480+0000","last_fullsized":"2023-12-19T18:14:54.056480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073307+0000","last_change":"2023-12-19T18:14:54.072540+0000","last_active":"2023-12-19T18:14:54.073307+0000","last_peered":"2023-12-19T18:14:54.073307+0000","last_clean":"2023-12-19T18:14:54.073307+0000","last_became_active":"2023-12-19T18:14:54.072162+0000","last_became_peered":"2023-12-19T18:14:54.072162+0000","last_unstale":"2023-12-19T18:14:54.073307+0000","last_undegraded":"2023-12-19T18:14:54.073307+0000","last_fullsized":"2023-12-19T18:14:54.073307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075801+0000","last_change":"2023-12-19T18:14:54.075571+0000","last_active":"2023-12-19T18:14:54.075801+0000","last_peered":"2023-12-19T18:14:54.075801+0000","last_clean":"2023-12-19T18:14:54.075801+0000","last_became_active":"2023-12-19T18:14:54.074613+0000","last_became_peered":"2023-12-19T18:14:54.074613+0000","last_unstale":"2023-12-19T18:14:54.075801+0000","last_undegraded":"2023-12-19T18:14:54.075801+0000","last_fullsized":"2023-12-19T18:14:54.075801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:15.120564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.062708+0000","last_change":"2023-12-19T18:14:54.062640+0000","last_active":"2023-12-19T18:14:54.062708+0000","last_peered":"2023-12-19T18:14:54.062708+0000","last_clean":"2023-12-19T18:14:54.062708+0000","last_became_active":"2023-12-19T18:14:54.061994+0000","last_became_peered":"2023-12-19T18:14:54.061994+0000","last_unstale":"2023-12-19T18:14:54.062708+0000","last_undegraded":"2023-12-19T18:14:54.062708+0000","last_fullsized":"2023-12-19T18:14:54.062708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073330+0000","last_change":"2023-12-19T18:14:54.073175+0000","last_active":"2023-12-19T18:14:54.073330+0000","last_peered":"2023-12-19T18:14:54.073330+0000","last_clean":"2023-12-19T18:14:54.073330+0000","last_became_active":"2023-12-19T18:14:54.072223+0000","last_became_peered":"2023-12-19T18:14:54.072223+0000","last_unstale":"2023-12-19T18:14:54.073330+0000","last_undegraded":"2023-12-19T18:14:54.073330+0000","last_fullsized":"2023-12-19T18:14:54.073330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064684+0000","last_change":"2023-12-19T18:14:54.064206+0000","last_active":"2023-12-19T18:14:54.064684+0000","last_peered":"2023-12-19T18:14:54.064684+0000","last_clean":"2023-12-19T18:14:54.064684+0000","last_became_active":"2023-12-19T18:14:54.063818+0000","last_became_peered":"2023-12-19T18:14:54.063818+0000","last_unstale":"2023-12-19T18:14:54.064684+0000","last_undegraded":"2023-12-19T18:14:54.064684+0000","last_fullsized":"2023-12-19T18:14:54.064684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063666+0000","last_change":"2023-12-19T18:14:54.063234+0000","last_active":"2023-12-19T18:14:54.063666+0000","last_peered":"2023-12-19T18:14:54.063666+0000","last_clean":"2023-12-19T18:14:54.063666+0000","last_became_active":"2023-12-19T18:14:54.062885+0000","last_became_peered":"2023-12-19T18:14:54.062885+0000","last_unstale":"2023-12-19T18:14:54.063666+0000","last_undegraded":"2023-12-19T18:14:54.063666+0000","last_fullsized":"2023-12-19T18:14:54.063666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063662+0000","last_change":"2023-12-19T18:14:54.063598+0000","last_active":"2023-12-19T18:14:54.063662+0000","last_peered":"2023-12-19T18:14:54.063662+0000","last_clean":"2023-12-19T18:14:54.063662+0000","last_became_active":"2023-12-19T18:14:54.063314+0000","last_became_peered":"2023-12-19T18:14:54.063314+0000","last_unstale":"2023-12-19T18:14:54.063662+0000","last_undegraded":"2023-12-19T18:14:54.063662+0000","last_fullsized":"2023-12-19T18:14:54.063662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060228+0000","last_change":"2023-12-19T18:14:54.059917+0000","last_active":"2023-12-19T18:14:54.060228+0000","last_peered":"2023-12-19T18:14:54.060228+0000","last_clean":"2023-12-19T18:14:54.060228+0000","last_became_active":"2023-12-19T18:14:54.059674+0000","last_became_peered":"2023-12-19T18:14:54.059674+0000","last_unstale":"2023-12-19T18:14:54.060228+0000","last_undegraded":"2023-12-19T18:14:54.060228+0000","last_fullsized":"2023-12-19T18:14:54.060228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074581+0000","last_change":"2023-12-19T18:14:54.073659+0000","last_active":"2023-12-19T18:14:54.074581+0000","last_peered":"2023-12-19T18:14:54.074581+0000","last_clean":"2023-12-19T18:14:54.074581+0000","last_became_active":"2023-12-19T18:14:54.072947+0000","last_became_peered":"2023-12-19T18:14:54.072947+0000","last_unstale":"2023-12-19T18:14:54.074581+0000","last_undegraded":"2023-12-19T18:14:54.074581+0000","last_fullsized":"2023-12-19T18:14:54.074581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067036+0000","last_change":"2023-12-19T18:14:54.066189+0000","last_active":"2023-12-19T18:14:54.067036+0000","last_peered":"2023-12-19T18:14:54.067036+0000","last_clean":"2023-12-19T18:14:54.067036+0000","last_became_active":"2023-12-19T18:14:54.065921+0000","last_became_peered":"2023-12-19T18:14:54.065921+0000","last_unstale":"2023-12-19T18:14:54.067036+0000","last_undegraded":"2023-12-19T18:14:54.067036+0000","last_fullsized":"2023-12-19T18:14:54.067036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061440+0000","last_change":"2023-12-19T18:14:54.061372+0000","last_active":"2023-12-19T18:14:54.061440+0000","last_peered":"2023-12-19T18:14:54.061440+0000","last_clean":"2023-12-19T18:14:54.061440+0000","last_became_active":"2023-12-19T18:14:54.060949+0000","last_became_peered":"2023-12-19T18:14:54.060949+0000","last_unstale":"2023-12-19T18:14:54.061440+0000","last_undegraded":"2023-12-19T18:14:54.061440+0000","last_fullsized":"2023-12-19T18:14:54.061440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073948+0000","last_change":"2023-12-19T18:14:54.073038+0000","last_active":"2023-12-19T18:14:54.073948+0000","last_peered":"2023-12-19T18:14:54.073948+0000","last_clean":"2023-12-19T18:14:54.073948+0000","last_became_active":"2023-12-19T18:14:54.071844+0000","last_became_peered":"2023-12-19T18:14:54.071844+0000","last_unstale":"2023-12-19T18:14:54.073948+0000","last_undegraded":"2023-12-19T18:14:54.073948+0000","last_fullsized":"2023-12-19T18:14:54.073948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070883+0000","last_change":"2023-12-19T18:14:54.070817+0000","last_active":"2023-12-19T18:14:54.070883+0000","last_peered":"2023-12-19T18:14:54.070883+0000","last_clean":"2023-12-19T18:14:54.070883+0000","last_became_active":"2023-12-19T18:14:54.068216+0000","last_became_peered":"2023-12-19T18:14:54.068216+0000","last_unstale":"2023-12-19T18:14:54.070883+0000","last_undegraded":"2023-12-19T18:14:54.070883+0000","last_fullsized":"2023-12-19T18:14:54.070883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063273+0000","last_change":"2023-12-19T18:14:54.060522+0000","last_active":"2023-12-19T18:14:54.063273+0000","last_peered":"2023-12-19T18:14:54.063273+0000","last_clean":"2023-12-19T18:14:54.063273+0000","last_became_active":"2023-12-19T18:14:54.059641+0000","last_became_peered":"2023-12-19T18:14:54.059641+0000","last_unstale":"2023-12-19T18:14:54.063273+0000","last_undegraded":"2023-12-19T18:14:54.063273+0000","last_fullsized":"2023-12-19T18:14:54.063273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065928+0000","last_change":"2023-12-19T18:14:54.065766+0000","last_active":"2023-12-19T18:14:54.065928+0000","last_peered":"2023-12-19T18:14:54.065928+0000","last_clean":"2023-12-19T18:14:54.065928+0000","last_became_active":"2023-12-19T18:14:54.058571+0000","last_became_peered":"2023-12-19T18:14:54.058571+0000","last_unstale":"2023-12-19T18:14:54.065928+0000","last_undegraded":"2023-12-19T18:14:54.065928+0000","last_fullsized":"2023-12-19T18:14:54.065928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:47.213095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070429+0000","last_change":"2023-12-19T18:14:54.067798+0000","last_active":"2023-12-19T18:14:54.070429+0000","last_peered":"2023-12-19T18:14:54.070429+0000","last_clean":"2023-12-19T18:14:54.070429+0000","last_became_active":"2023-12-19T18:14:54.067576+0000","last_became_peered":"2023-12-19T18:14:54.067576+0000","last_unstale":"2023-12-19T18:14:54.070429+0000","last_undegraded":"2023-12-19T18:14:54.070429+0000","last_fullsized":"2023-12-19T18:14:54.070429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069889+0000","last_change":"2023-12-19T18:14:54.069741+0000","last_active":"2023-12-19T18:14:54.069889+0000","last_peered":"2023-12-19T18:14:54.069889+0000","last_clean":"2023-12-19T18:14:54.069889+0000","last_became_active":"2023-12-19T18:14:54.069513+0000","last_became_peered":"2023-12-19T18:14:54.069513+0000","last_unstale":"2023-12-19T18:14:54.069889+0000","last_undegraded":"2023-12-19T18:14:54.069889+0000","last_fullsized":"2023-12-19T18:14:54.069889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070495+0000","last_change":"2023-12-19T18:14:54.070403+0000","last_active":"2023-12-19T18:14:54.070495+0000","last_peered":"2023-12-19T18:14:54.070495+0000","last_clean":"2023-12-19T18:14:54.070495+0000","last_became_active":"2023-12-19T18:14:54.069801+0000","last_became_peered":"2023-12-19T18:14:54.069801+0000","last_unstale":"2023-12-19T18:14:54.070495+0000","last_undegraded":"2023-12-19T18:14:54.070495+0000","last_fullsized":"2023-12-19T18:14:54.070495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068551+0000","last_change":"2023-12-19T18:14:53.563064+0000","last_active":"2023-12-19T18:14:54.068551+0000","last_peered":"2023-12-19T18:14:54.068551+0000","last_clean":"2023-12-19T18:14:54.068551+0000","last_became_active":"2023-12-19T18:14:53.561534+0000","last_became_peered":"2023-12-19T18:14:53.561534+0000","last_unstale":"2023-12-19T18:14:54.068551+0000","last_undegraded":"2023-12-19T18:14:54.068551+0000","last_fullsized":"2023-12-19T18:14:54.068551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051639+0000","last_change":"2023-12-19T18:14:54.051570+0000","last_active":"2023-12-19T18:14:54.051639+0000","last_peered":"2023-12-19T18:14:54.051639+0000","last_clean":"2023-12-19T18:14:54.051639+0000","last_became_active":"2023-12-19T18:14:54.051173+0000","last_became_peered":"2023-12-19T18:14:54.051173+0000","last_unstale":"2023-12-19T18:14:54.051639+0000","last_undegraded":"2023-12-19T18:14:54.051639+0000","last_fullsized":"2023-12-19T18:14:54.051639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073383+0000","last_change":"2023-12-19T18:14:53.560763+0000","last_active":"2023-12-19T18:14:54.073383+0000","last_peered":"2023-12-19T18:14:54.073383+0000","last_clean":"2023-12-19T18:14:54.073383+0000","last_became_active":"2023-12-19T18:14:53.558648+0000","last_became_peered":"2023-12-19T18:14:53.558648+0000","last_unstale":"2023-12-19T18:14:54.073383+0000","last_undegraded":"2023-12-19T18:14:54.073383+0000","last_fullsized":"2023-12-19T18:14:54.073383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:51.179899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066389+0000","last_change":"2023-12-19T18:14:53.560483+0000","last_active":"2023-12-19T18:14:54.066389+0000","last_peered":"2023-12-19T18:14:54.066389+0000","last_clean":"2023-12-19T18:14:54.066389+0000","last_became_active":"2023-12-19T18:14:53.560244+0000","last_became_peered":"2023-12-19T18:14:53.560244+0000","last_unstale":"2023-12-19T18:14:54.066389+0000","last_undegraded":"2023-12-19T18:14:54.066389+0000","last_fullsized":"2023-12-19T18:14:54.066389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072273+0000","last_change":"2023-12-19T18:14:54.071383+0000","last_active":"2023-12-19T18:14:54.072273+0000","last_peered":"2023-12-19T18:14:54.072273+0000","last_clean":"2023-12-19T18:14:54.072273+0000","last_became_active":"2023-12-19T18:14:54.070954+0000","last_became_peered":"2023-12-19T18:14:54.070954+0000","last_unstale":"2023-12-19T18:14:54.072273+0000","last_undegraded":"2023-12-19T18:14:54.072273+0000","last_fullsized":"2023-12-19T18:14:54.072273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053551+0000","last_change":"2023-12-19T18:14:53.558346+0000","last_active":"2023-12-19T18:14:54.053551+0000","last_peered":"2023-12-19T18:14:54.053551+0000","last_clean":"2023-12-19T18:14:54.053551+0000","last_became_active":"2023-12-19T18:14:53.558124+0000","last_became_peered":"2023-12-19T18:14:53.558124+0000","last_unstale":"2023-12-19T18:14:54.053551+0000","last_undegraded":"2023-12-19T18:14:54.053551+0000","last_fullsized":"2023-12-19T18:14:54.053551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066236+0000","last_change":"2023-12-19T18:14:54.064228+0000","last_active":"2023-12-19T18:14:54.066236+0000","last_peered":"2023-12-19T18:14:54.066236+0000","last_clean":"2023-12-19T18:14:54.066236+0000","last_became_active":"2023-12-19T18:14:54.063991+0000","last_became_peered":"2023-12-19T18:14:54.063991+0000","last_unstale":"2023-12-19T18:14:54.066236+0000","last_undegraded":"2023-12-19T18:14:54.066236+0000","last_fullsized":"2023-12-19T18:14:54.066236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064163+0000","last_change":"2023-12-19T18:14:53.554048+0000","last_active":"2023-12-19T18:14:54.064163+0000","last_peered":"2023-12-19T18:14:54.064163+0000","last_clean":"2023-12-19T18:14:54.064163+0000","last_became_active":"2023-12-19T18:14:53.553811+0000","last_became_peered":"2023-12-19T18:14:53.553811+0000","last_unstale":"2023-12-19T18:14:54.064163+0000","last_undegraded":"2023-12-19T18:14:54.064163+0000","last_fullsized":"2023-12-19T18:14:54.064163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073051+0000","last_change":"2023-12-19T18:14:54.072974+0000","last_active":"2023-12-19T18:14:54.073051+0000","last_peered":"2023-12-19T18:14:54.073051+0000","last_clean":"2023-12-19T18:14:54.073051+0000","last_became_active":"2023-12-19T18:14:54.072176+0000","last_became_peered":"2023-12-19T18:14:54.072176+0000","last_unstale":"2023-12-19T18:14:54.073051+0000","last_undegraded":"2023-12-19T18:14:54.073051+0000","last_fullsized":"2023-12-19T18:14:54.073051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.043565+0000","last_change":"2023-12-19T18:14:53.557947+0000","last_active":"2023-12-19T18:14:54.043565+0000","last_peered":"2023-12-19T18:14:54.043565+0000","last_clean":"2023-12-19T18:14:54.043565+0000","last_became_active":"2023-12-19T18:14:53.557713+0000","last_became_peered":"2023-12-19T18:14:53.557713+0000","last_unstale":"2023-12-19T18:14:54.043565+0000","last_undegraded":"2023-12-19T18:14:54.043565+0000","last_fullsized":"2023-12-19T18:14:54.043565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056501+0000","last_change":"2023-12-19T18:14:54.056275+0000","last_active":"2023-12-19T18:14:54.056501+0000","last_peered":"2023-12-19T18:14:54.056501+0000","last_clean":"2023-12-19T18:14:54.056501+0000","last_became_active":"2023-12-19T18:14:54.056002+0000","last_became_peered":"2023-12-19T18:14:54.056002+0000","last_unstale":"2023-12-19T18:14:54.056501+0000","last_undegraded":"2023-12-19T18:14:54.056501+0000","last_fullsized":"2023-12-19T18:14:54.056501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:30.847801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073216+0000","last_change":"2023-12-19T18:14:53.568016+0000","last_active":"2023-12-19T18:14:54.073216+0000","last_peered":"2023-12-19T18:14:54.073216+0000","last_clean":"2023-12-19T18:14:54.073216+0000","last_became_active":"2023-12-19T18:14:53.567634+0000","last_became_peered":"2023-12-19T18:14:53.567634+0000","last_unstale":"2023-12-19T18:14:54.073216+0000","last_undegraded":"2023-12-19T18:14:54.073216+0000","last_fullsized":"2023-12-19T18:14:54.073216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075897+0000","last_change":"2023-12-19T18:14:54.075827+0000","last_active":"2023-12-19T18:14:54.075897+0000","last_peered":"2023-12-19T18:14:54.075897+0000","last_clean":"2023-12-19T18:14:54.075897+0000","last_became_active":"2023-12-19T18:14:54.074747+0000","last_became_peered":"2023-12-19T18:14:54.074747+0000","last_unstale":"2023-12-19T18:14:54.075897+0000","last_undegraded":"2023-12-19T18:14:54.075897+0000","last_fullsized":"2023-12-19T18:14:54.075897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073455+0000","last_change":"2023-12-19T18:14:53.547427+0000","last_active":"2023-12-19T18:14:54.073455+0000","last_peered":"2023-12-19T18:14:54.073455+0000","last_clean":"2023-12-19T18:14:54.073455+0000","last_became_active":"2023-12-19T18:14:53.546733+0000","last_became_peered":"2023-12-19T18:14:53.546733+0000","last_unstale":"2023-12-19T18:14:54.073455+0000","last_undegraded":"2023-12-19T18:14:54.073455+0000","last_fullsized":"2023-12-19T18:14:54.073455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:13:12.090625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069234+0000","last_change":"2023-12-19T18:14:54.069070+0000","last_active":"2023-12-19T18:14:54.069234+0000","last_peered":"2023-12-19T18:14:54.069234+0000","last_clean":"2023-12-19T18:14:54.069234+0000","last_became_active":"2023-12-19T18:14:54.066495+0000","last_became_peered":"2023-12-19T18:14:54.066495+0000","last_unstale":"2023-12-19T18:14:54.069234+0000","last_undegraded":"2023-12-19T18:14:54.069234+0000","last_fullsized":"2023-12-19T18:14:54.069234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068132+0000","last_change":"2023-12-19T18:14:53.566349+0000","last_active":"2023-12-19T18:14:54.068132+0000","last_peered":"2023-12-19T18:14:54.068132+0000","last_clean":"2023-12-19T18:14:54.068132+0000","last_became_active":"2023-12-19T18:14:53.565909+0000","last_became_peered":"2023-12-19T18:14:53.565909+0000","last_unstale":"2023-12-19T18:14:54.068132+0000","last_undegraded":"2023-12-19T18:14:54.068132+0000","last_fullsized":"2023-12-19T18:14:54.068132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073929+0000","last_change":"2023-12-19T18:14:54.072903+0000","last_active":"2023-12-19T18:14:54.073929+0000","last_peered":"2023-12-19T18:14:54.073929+0000","last_clean":"2023-12-19T18:14:54.073929+0000","last_became_active":"2023-12-19T18:14:54.072678+0000","last_became_peered":"2023-12-19T18:14:54.072678+0000","last_unstale":"2023-12-19T18:14:54.073929+0000","last_undegraded":"2023-12-19T18:14:54.073929+0000","last_fullsized":"2023-12-19T18:14:54.073929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.052955+0000","last_change":"2023-12-19T18:14:53.561604+0000","last_active":"2023-12-19T18:14:54.052955+0000","last_peered":"2023-12-19T18:14:54.052955+0000","last_clean":"2023-12-19T18:14:54.052955+0000","last_became_active":"2023-12-19T18:14:53.561372+0000","last_became_peered":"2023-12-19T18:14:53.561372+0000","last_unstale":"2023-12-19T18:14:54.052955+0000","last_undegraded":"2023-12-19T18:14:54.052955+0000","last_fullsized":"2023-12-19T18:14:54.052955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:00.255541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.057807+0000","last_change":"2023-12-19T18:14:54.057328+0000","last_active":"2023-12-19T18:14:54.057807+0000","last_peered":"2023-12-19T18:14:54.057807+0000","last_clean":"2023-12-19T18:14:54.057807+0000","last_became_active":"2023-12-19T18:14:54.055611+0000","last_became_peered":"2023-12-19T18:14:54.055611+0000","last_unstale":"2023-12-19T18:14:54.057807+0000","last_undegraded":"2023-12-19T18:14:54.057807+0000","last_fullsized":"2023-12-19T18:14:54.057807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:12.180767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.043960+0000","last_change":"2023-12-19T18:14:53.562764+0000","last_active":"2023-12-19T18:14:54.043960+0000","last_peered":"2023-12-19T18:14:54.043960+0000","last_clean":"2023-12-19T18:14:54.043960+0000","last_became_active":"2023-12-19T18:14:53.562532+0000","last_became_peered":"2023-12-19T18:14:53.562532+0000","last_unstale":"2023-12-19T18:14:54.043960+0000","last_undegraded":"2023-12-19T18:14:54.043960+0000","last_fullsized":"2023-12-19T18:14:54.043960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069588+0000","last_change":"2023-12-19T18:14:54.068780+0000","last_active":"2023-12-19T18:14:54.069588+0000","last_peered":"2023-12-19T18:14:54.069588+0000","last_clean":"2023-12-19T18:14:54.069588+0000","last_became_active":"2023-12-19T18:14:54.068556+0000","last_became_peered":"2023-12-19T18:14:54.068556+0000","last_unstale":"2023-12-19T18:14:54.069588+0000","last_undegraded":"2023-12-19T18:14:54.069588+0000","last_fullsized":"2023-12-19T18:14:54.069588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074393+0000","last_change":"2023-12-19T18:14:53.565084+0000","last_active":"2023-12-19T18:14:54.074393+0000","last_peered":"2023-12-19T18:14:54.074393+0000","last_clean":"2023-12-19T18:14:54.074393+0000","last_became_active":"2023-12-19T18:14:53.564836+0000","last_became_peered":"2023-12-19T18:14:53.564836+0000","last_unstale":"2023-12-19T18:14:54.074393+0000","last_undegraded":"2023-12-19T18:14:54.074393+0000","last_fullsized":"2023-12-19T18:14:54.074393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.076309+0000","last_change":"2023-12-19T18:14:54.075967+0000","last_active":"2023-12-19T18:14:54.076309+0000","last_peered":"2023-12-19T18:14:54.076309+0000","last_clean":"2023-12-19T18:14:54.076309+0000","last_became_active":"2023-12-19T18:14:54.074991+0000","last_became_peered":"2023-12-19T18:14:54.074991+0000","last_unstale":"2023-12-19T18:14:54.076309+0000","last_undegraded":"2023-12-19T18:14:54.076309+0000","last_fullsized":"2023-12-19T18:14:54.076309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061847+0000","last_change":"2023-12-19T18:14:53.550365+0000","last_active":"2023-12-19T18:14:54.061847+0000","last_peered":"2023-12-19T18:14:54.061847+0000","last_clean":"2023-12-19T18:14:54.061847+0000","last_became_active":"2023-12-19T18:14:53.550059+0000","last_became_peered":"2023-12-19T18:14:53.550059+0000","last_unstale":"2023-12-19T18:14:54.061847+0000","last_undegraded":"2023-12-19T18:14:54.061847+0000","last_fullsized":"2023-12-19T18:14:54.061847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050074+0000","last_change":"2023-12-19T18:14:54.048927+0000","last_active":"2023-12-19T18:14:54.050074+0000","last_peered":"2023-12-19T18:14:54.050074+0000","last_clean":"2023-12-19T18:14:54.050074+0000","last_became_active":"2023-12-19T18:14:54.048515+0000","last_became_peered":"2023-12-19T18:14:54.048515+0000","last_unstale":"2023-12-19T18:14:54.050074+0000","last_undegraded":"2023-12-19T18:14:54.050074+0000","last_fullsized":"2023-12-19T18:14:54.050074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064254+0000","last_change":"2023-12-19T18:14:53.567656+0000","last_active":"2023-12-19T18:14:54.064254+0000","last_peered":"2023-12-19T18:14:54.064254+0000","last_clean":"2023-12-19T18:14:54.064254+0000","last_became_active":"2023-12-19T18:14:53.567278+0000","last_became_peered":"2023-12-19T18:14:53.567278+0000","last_unstale":"2023-12-19T18:14:54.064254+0000","last_undegraded":"2023-12-19T18:14:54.064254+0000","last_fullsized":"2023-12-19T18:14:54.064254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065685+0000","last_change":"2023-12-19T18:14:54.065628+0000","last_active":"2023-12-19T18:14:54.065685+0000","last_peered":"2023-12-19T18:14:54.065685+0000","last_clean":"2023-12-19T18:14:54.065685+0000","last_became_active":"2023-12-19T18:14:54.065414+0000","last_became_peered":"2023-12-19T18:14:54.065414+0000","last_unstale":"2023-12-19T18:14:54.065685+0000","last_undegraded":"2023-12-19T18:14:54.065685+0000","last_fullsized":"2023-12-19T18:14:54.065685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065227+0000","last_change":"2023-12-19T18:14:53.559407+0000","last_active":"2023-12-19T18:14:54.065227+0000","last_peered":"2023-12-19T18:14:54.065227+0000","last_clean":"2023-12-19T18:14:54.065227+0000","last_became_active":"2023-12-19T18:14:53.559062+0000","last_became_peered":"2023-12-19T18:14:53.559062+0000","last_unstale":"2023-12-19T18:14:54.065227+0000","last_undegraded":"2023-12-19T18:14:54.065227+0000","last_fullsized":"2023-12-19T18:14:54.065227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061207+0000","last_change":"2023-12-19T18:14:54.061139+0000","last_active":"2023-12-19T18:14:54.061207+0000","last_peered":"2023-12-19T18:14:54.061207+0000","last_clean":"2023-12-19T18:14:54.061207+0000","last_became_active":"2023-12-19T18:14:54.060925+0000","last_became_peered":"2023-12-19T18:14:54.060925+0000","last_unstale":"2023-12-19T18:14:54.061207+0000","last_undegraded":"2023-12-19T18:14:54.061207+0000","last_fullsized":"2023-12-19T18:14:54.061207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067789+0000","last_change":"2023-12-19T18:14:53.563513+0000","last_active":"2023-12-19T18:14:54.067789+0000","last_peered":"2023-12-19T18:14:54.067789+0000","last_clean":"2023-12-19T18:14:54.067789+0000","last_became_active":"2023-12-19T18:14:53.562860+0000","last_became_peered":"2023-12-19T18:14:53.562860+0000","last_unstale":"2023-12-19T18:14:54.067789+0000","last_undegraded":"2023-12-19T18:14:54.067789+0000","last_fullsized":"2023-12-19T18:14:54.067789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:59:49.969450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072181+0000","last_change":"2023-12-19T18:14:54.071273+0000","last_active":"2023-12-19T18:14:54.072181+0000","last_peered":"2023-12-19T18:14:54.072181+0000","last_clean":"2023-12-19T18:14:54.072181+0000","last_became_active":"2023-12-19T18:14:54.070908+0000","last_became_peered":"2023-12-19T18:14:54.070908+0000","last_unstale":"2023-12-19T18:14:54.072181+0000","last_undegraded":"2023-12-19T18:14:54.072181+0000","last_fullsized":"2023-12-19T18:14:54.072181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:27:34.736465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.055663+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:14:54.055663+0000","last_peered":"2023-12-19T18:14:54.055663+0000","last_clean":"2023-12-19T18:14:54.055663+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:14:54.055663+0000","last_undegraded":"2023-12-19T18:14:54.055663+0000","last_fullsized":"2023-12-19T18:14:54.055663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053073+0000","last_change":"2023-12-19T18:14:53.557129+0000","last_active":"2023-12-19T18:14:54.053073+0000","last_peered":"2023-12-19T18:14:54.053073+0000","last_clean":"2023-12-19T18:14:54.053073+0000","last_became_active":"2023-12-19T18:14:53.556812+0000","last_became_peered":"2023-12-19T18:14:53.556812+0000","last_unstale":"2023-12-19T18:14:54.053073+0000","last_undegraded":"2023-12-19T18:14:54.053073+0000","last_fullsized":"2023-12-19T18:14:54.053073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:53.113355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051508+0000","last_change":"2023-12-19T18:14:54.051423+0000","last_active":"2023-12-19T18:14:54.051508+0000","last_peered":"2023-12-19T18:14:54.051508+0000","last_clean":"2023-12-19T18:14:54.051508+0000","last_became_active":"2023-12-19T18:14:54.050816+0000","last_became_peered":"2023-12-19T18:14:54.050816+0000","last_unstale":"2023-12-19T18:14:54.051508+0000","last_undegraded":"2023-12-19T18:14:54.051508+0000","last_fullsized":"2023-12-19T18:14:54.051508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068099+0000","last_change":"2023-12-19T18:14:53.558583+0000","last_active":"2023-12-19T18:14:54.068099+0000","last_peered":"2023-12-19T18:14:54.068099+0000","last_clean":"2023-12-19T18:14:54.068099+0000","last_became_active":"2023-12-19T18:14:53.558000+0000","last_became_peered":"2023-12-19T18:14:53.558000+0000","last_unstale":"2023-12-19T18:14:54.068099+0000","last_undegraded":"2023-12-19T18:14:54.068099+0000","last_fullsized":"2023-12-19T18:14:54.068099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:53.818952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068077+0000","last_change":"2023-12-19T18:14:54.067973+0000","last_active":"2023-12-19T18:14:54.068077+0000","last_peered":"2023-12-19T18:14:54.068077+0000","last_clean":"2023-12-19T18:14:54.068077+0000","last_became_active":"2023-12-19T18:14:54.065516+0000","last_became_peered":"2023-12-19T18:14:54.065516+0000","last_unstale":"2023-12-19T18:14:54.068077+0000","last_undegraded":"2023-12-19T18:14:54.068077+0000","last_fullsized":"2023-12-19T18:14:54.068077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.058632+0000","last_change":"2023-12-19T18:14:53.563881+0000","last_active":"2023-12-19T18:14:54.058632+0000","last_peered":"2023-12-19T18:14:54.058632+0000","last_clean":"2023-12-19T18:14:54.058632+0000","last_became_active":"2023-12-19T18:14:53.563564+0000","last_became_peered":"2023-12-19T18:14:53.563564+0000","last_unstale":"2023-12-19T18:14:54.058632+0000","last_undegraded":"2023-12-19T18:14:54.058632+0000","last_fullsized":"2023-12-19T18:14:54.058632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060261+0000","last_change":"2023-12-19T18:14:54.060208+0000","last_active":"2023-12-19T18:14:54.060261+0000","last_peered":"2023-12-19T18:14:54.060261+0000","last_clean":"2023-12-19T18:14:54.060261+0000","last_became_active":"2023-12-19T18:14:54.059235+0000","last_became_peered":"2023-12-19T18:14:54.059235+0000","last_unstale":"2023-12-19T18:14:54.060261+0000","last_undegraded":"2023-12-19T18:14:54.060261+0000","last_fullsized":"2023-12-19T18:14:54.060261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065436+0000","last_change":"2023-12-19T18:14:53.564065+0000","last_active":"2023-12-19T18:14:54.065436+0000","last_peered":"2023-12-19T18:14:54.065436+0000","last_clean":"2023-12-19T18:14:54.065436+0000","last_became_active":"2023-12-19T18:14:53.563841+0000","last_became_peered":"2023-12-19T18:14:53.563841+0000","last_unstale":"2023-12-19T18:14:54.065436+0000","last_undegraded":"2023-12-19T18:14:54.065436+0000","last_fullsized":"2023-12-19T18:14:54.065436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:46.161691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060843+0000","last_change":"2023-12-19T18:14:54.060734+0000","last_active":"2023-12-19T18:14:54.060843+0000","last_peered":"2023-12-19T18:14:54.060843+0000","last_clean":"2023-12-19T18:14:54.060843+0000","last_became_active":"2023-12-19T18:14:54.060328+0000","last_became_peered":"2023-12-19T18:14:54.060328+0000","last_unstale":"2023-12-19T18:14:54.060843+0000","last_undegraded":"2023-12-19T18:14:54.060843+0000","last_fullsized":"2023-12-19T18:14:54.060843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050368+0000","last_change":"2023-12-19T18:14:53.565164+0000","last_active":"2023-12-19T18:14:54.050368+0000","last_peered":"2023-12-19T18:14:54.050368+0000","last_clean":"2023-12-19T18:14:54.050368+0000","last_became_active":"2023-12-19T18:14:53.564538+0000","last_became_peered":"2023-12-19T18:14:53.564538+0000","last_unstale":"2023-12-19T18:14:54.050368+0000","last_undegraded":"2023-12-19T18:14:54.050368+0000","last_fullsized":"2023-12-19T18:14:54.050368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068274+0000","last_change":"2023-12-19T18:14:54.068080+0000","last_active":"2023-12-19T18:14:54.068274+0000","last_peered":"2023-12-19T18:14:54.068274+0000","last_clean":"2023-12-19T18:14:54.068274+0000","last_became_active":"2023-12-19T18:14:54.067528+0000","last_became_peered":"2023-12-19T18:14:54.067528+0000","last_unstale":"2023-12-19T18:14:54.068274+0000","last_undegraded":"2023-12-19T18:14:54.068274+0000","last_fullsized":"2023-12-19T18:14:54.068274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067999+0000","last_change":"2023-12-19T18:14:53.562234+0000","last_active":"2023-12-19T18:14:54.067999+0000","last_peered":"2023-12-19T18:14:54.067999+0000","last_clean":"2023-12-19T18:14:54.067999+0000","last_became_active":"2023-12-19T18:14:53.561326+0000","last_became_peered":"2023-12-19T18:14:53.561326+0000","last_unstale":"2023-12-19T18:14:54.067999+0000","last_undegraded":"2023-12-19T18:14:54.067999+0000","last_fullsized":"2023-12-19T18:14:54.067999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074010+0000","last_change":"2023-12-19T18:14:54.073543+0000","last_active":"2023-12-19T18:14:54.074010+0000","last_peered":"2023-12-19T18:14:54.074010+0000","last_clean":"2023-12-19T18:14:54.074010+0000","last_became_active":"2023-12-19T18:14:54.073247+0000","last_became_peered":"2023-12-19T18:14:54.073247+0000","last_unstale":"2023-12-19T18:14:54.074010+0000","last_undegraded":"2023-12-19T18:14:54.074010+0000","last_fullsized":"2023-12-19T18:14:54.074010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-19T18:14:55.050134+0000","last_change":"2023-12-19T18:14:53.551251+0000","last_active":"2023-12-19T18:14:55.050134+0000","last_peered":"2023-12-19T18:14:55.050134+0000","last_clean":"2023-12-19T18:14:55.050134+0000","last_became_active":"2023-12-19T18:14:53.550691+0000","last_became_peered":"2023-12-19T18:14:53.550691+0000","last_unstale":"2023-12-19T18:14:55.050134+0000","last_undegraded":"2023-12-19T18:14:55.050134+0000","last_fullsized":"2023-12-19T18:14:55.050134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.071102+0000","last_change":"2023-12-19T18:14:54.070766+0000","last_active":"2023-12-19T18:14:54.071102+0000","last_peered":"2023-12-19T18:14:54.071102+0000","last_clean":"2023-12-19T18:14:54.071102+0000","last_became_active":"2023-12-19T18:14:54.069495+0000","last_became_peered":"2023-12-19T18:14:54.069495+0000","last_unstale":"2023-12-19T18:14:54.071102+0000","last_undegraded":"2023-12-19T18:14:54.071102+0000","last_fullsized":"2023-12-19T18:14:54.071102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.076793+0000","last_change":"2023-12-19T18:14:53.563561+0000","last_active":"2023-12-19T18:14:54.076793+0000","last_peered":"2023-12-19T18:14:54.076793+0000","last_clean":"2023-12-19T18:14:54.076793+0000","last_became_active":"2023-12-19T18:14:53.563126+0000","last_became_peered":"2023-12-19T18:14:53.563126+0000","last_unstale":"2023-12-19T18:14:54.076793+0000","last_undegraded":"2023-12-19T18:14:54.076793+0000","last_fullsized":"2023-12-19T18:14:54.076793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.059619+0000","last_change":"2023-12-19T18:14:54.059362+0000","last_active":"2023-12-19T18:14:54.059619+0000","last_peered":"2023-12-19T18:14:54.059619+0000","last_clean":"2023-12-19T18:14:54.059619+0000","last_became_active":"2023-12-19T18:14:54.059089+0000","last_became_peered":"2023-12-19T18:14:54.059089+0000","last_unstale":"2023-12-19T18:14:54.059619+0000","last_undegraded":"2023-12-19T18:14:54.059619+0000","last_fullsized":"2023-12-19T18:14:54.059619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.045320+0000","last_change":"2023-12-19T18:14:53.551856+0000","last_active":"2023-12-19T18:14:54.045320+0000","last_peered":"2023-12-19T18:14:54.045320+0000","last_clean":"2023-12-19T18:14:54.045320+0000","last_became_active":"2023-12-19T18:14:53.551105+0000","last_became_peered":"2023-12-19T18:14:53.551105+0000","last_unstale":"2023-12-19T18:14:54.045320+0000","last_undegraded":"2023-12-19T18:14:54.045320+0000","last_fullsized":"2023-12-19T18:14:54.045320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065908+0000","last_change":"2023-12-19T18:14:54.065539+0000","last_active":"2023-12-19T18:14:54.065908+0000","last_peered":"2023-12-19T18:14:54.065908+0000","last_clean":"2023-12-19T18:14:54.065908+0000","last_became_active":"2023-12-19T18:14:54.056448+0000","last_became_peered":"2023-12-19T18:14:54.056448+0000","last_unstale":"2023-12-19T18:14:54.065908+0000","last_undegraded":"2023-12-19T18:14:54.065908+0000","last_fullsized":"2023-12-19T18:14:54.065908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.045692+0000","last_change":"2023-12-19T18:14:53.554185+0000","last_active":"2023-12-19T18:14:54.045692+0000","last_peered":"2023-12-19T18:14:54.045692+0000","last_clean":"2023-12-19T18:14:54.045692+0000","last_became_active":"2023-12-19T18:14:53.553750+0000","last_became_peered":"2023-12-19T18:14:53.553750+0000","last_unstale":"2023-12-19T18:14:54.045692+0000","last_undegraded":"2023-12-19T18:14:54.045692+0000","last_fullsized":"2023-12-19T18:14:54.045692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.059756+0000","last_change":"2023-12-19T18:14:54.059501+0000","last_active":"2023-12-19T18:14:54.059756+0000","last_peered":"2023-12-19T18:14:54.059756+0000","last_clean":"2023-12-19T18:14:54.059756+0000","last_became_active":"2023-12-19T18:14:54.059055+0000","last_became_peered":"2023-12-19T18:14:54.059055+0000","last_unstale":"2023-12-19T18:14:54.059756+0000","last_undegraded":"2023-12-19T18:14:54.059756+0000","last_fullsized":"2023-12-19T18:14:54.059756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066496+0000","last_change":"2023-12-19T18:14:53.566640+0000","last_active":"2023-12-19T18:14:54.066496+0000","last_peered":"2023-12-19T18:14:54.066496+0000","last_clean":"2023-12-19T18:14:54.066496+0000","last_became_active":"2023-12-19T18:14:53.566413+0000","last_became_peered":"2023-12-19T18:14:53.566413+0000","last_unstale":"2023-12-19T18:14:54.066496+0000","last_undegraded":"2023-12-19T18:14:54.066496+0000","last_fullsized":"2023-12-19T18:14:54.066496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074538+0000","last_change":"2023-12-19T18:14:54.073374+0000","last_active":"2023-12-19T18:14:54.074538+0000","last_peered":"2023-12-19T18:14:54.074538+0000","last_clean":"2023-12-19T18:14:54.074538+0000","last_became_active":"2023-12-19T18:14:54.072676+0000","last_became_peered":"2023-12-19T18:14:54.072676+0000","last_unstale":"2023-12-19T18:14:54.074538+0000","last_undegraded":"2023-12-19T18:14:54.074538+0000","last_fullsized":"2023-12-19T18:14:54.074538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073825+0000","last_change":"2023-12-19T18:14:53.549245+0000","last_active":"2023-12-19T18:14:54.073825+0000","last_peered":"2023-12-19T18:14:54.073825+0000","last_clean":"2023-12-19T18:14:54.073825+0000","last_became_active":"2023-12-19T18:14:53.548992+0000","last_became_peered":"2023-12-19T18:14:53.548992+0000","last_unstale":"2023-12-19T18:14:54.073825+0000","last_undegraded":"2023-12-19T18:14:54.073825+0000","last_fullsized":"2023-12-19T18:14:54.073825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074558+0000","last_change":"2023-12-19T18:14:54.073488+0000","last_active":"2023-12-19T18:14:54.074558+0000","last_peered":"2023-12-19T18:14:54.074558+0000","last_clean":"2023-12-19T18:14:54.074558+0000","last_became_active":"2023-12-19T18:14:54.072567+0000","last_became_peered":"2023-12-19T18:14:54.072567+0000","last_unstale":"2023-12-19T18:14:54.074558+0000","last_undegraded":"2023-12-19T18:14:54.074558+0000","last_fullsized":"2023-12-19T18:14:54.074558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:22.428929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.057975+0000","last_change":"2023-12-19T18:14:54.057549+0000","last_active":"2023-12-19T18:14:54.057975+0000","last_peered":"2023-12-19T18:14:54.057975+0000","last_clean":"2023-12-19T18:14:54.057975+0000","last_became_active":"2023-12-19T18:14:54.056075+0000","last_became_peered":"2023-12-19T18:14:54.056075+0000","last_unstale":"2023-12-19T18:14:54.057975+0000","last_undegraded":"2023-12-19T18:14:54.057975+0000","last_fullsized":"2023-12-19T18:14:54.057975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:47.825262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064249+0000","last_change":"2023-12-19T18:14:53.555372+0000","last_active":"2023-12-19T18:14:54.064249+0000","last_peered":"2023-12-19T18:14:54.064249+0000","last_clean":"2023-12-19T18:14:54.064249+0000","last_became_active":"2023-12-19T18:14:53.555089+0000","last_became_peered":"2023-12-19T18:14:53.555089+0000","last_unstale":"2023-12-19T18:14:54.064249+0000","last_undegraded":"2023-12-19T18:14:54.064249+0000","last_fullsized":"2023-12-19T18:14:54.064249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065405+0000","last_change":"2023-12-19T18:14:54.065119+0000","last_active":"2023-12-19T18:14:54.065405+0000","last_peered":"2023-12-19T18:14:54.065405+0000","last_clean":"2023-12-19T18:14:54.065405+0000","last_became_active":"2023-12-19T18:14:54.064697+0000","last_became_peered":"2023-12-19T18:14:54.064697+0000","last_unstale":"2023-12-19T18:14:54.065405+0000","last_undegraded":"2023-12-19T18:14:54.065405+0000","last_fullsized":"2023-12-19T18:14:54.065405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063594+0000","last_change":"2023-12-19T18:14:53.556488+0000","last_active":"2023-12-19T18:14:54.063594+0000","last_peered":"2023-12-19T18:14:54.063594+0000","last_clean":"2023-12-19T18:14:54.063594+0000","last_became_active":"2023-12-19T18:14:53.555952+0000","last_became_peered":"2023-12-19T18:14:53.555952+0000","last_unstale":"2023-12-19T18:14:54.063594+0000","last_undegraded":"2023-12-19T18:14:54.063594+0000","last_fullsized":"2023-12-19T18:14:54.063594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070434+0000","last_change":"2023-12-19T18:14:54.070014+0000","last_active":"2023-12-19T18:14:54.070434+0000","last_peered":"2023-12-19T18:14:54.070434+0000","last_clean":"2023-12-19T18:14:54.070434+0000","last_became_active":"2023-12-19T18:14:54.067648+0000","last_became_peered":"2023-12-19T18:14:54.067648+0000","last_unstale":"2023-12-19T18:14:54.070434+0000","last_undegraded":"2023-12-19T18:14:54.070434+0000","last_fullsized":"2023-12-19T18:14:54.070434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067116+0000","last_change":"2023-12-19T18:14:53.566090+0000","last_active":"2023-12-19T18:14:54.067116+0000","last_peered":"2023-12-19T18:14:54.067116+0000","last_clean":"2023-12-19T18:14:54.067116+0000","last_became_active":"2023-12-19T18:14:53.565782+0000","last_became_peered":"2023-12-19T18:14:53.565782+0000","last_unstale":"2023-12-19T18:14:54.067116+0000","last_undegraded":"2023-12-19T18:14:54.067116+0000","last_fullsized":"2023-12-19T18:14:54.067116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063312+0000","last_change":"2023-12-19T18:14:54.060686+0000","last_active":"2023-12-19T18:14:54.063312+0000","last_peered":"2023-12-19T18:14:54.063312+0000","last_clean":"2023-12-19T18:14:54.063312+0000","last_became_active":"2023-12-19T18:14:54.059977+0000","last_became_peered":"2023-12-19T18:14:54.059977+0000","last_unstale":"2023-12-19T18:14:54.063312+0000","last_undegraded":"2023-12-19T18:14:54.063312+0000","last_fullsized":"2023-12-19T18:14:54.063312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073092+0000","last_change":"2023-12-19T18:14:53.558205+0000","last_active":"2023-12-19T18:14:54.073092+0000","last_peered":"2023-12-19T18:14:54.073092+0000","last_clean":"2023-12-19T18:14:54.073092+0000","last_became_active":"2023-12-19T18:14:53.557733+0000","last_became_peered":"2023-12-19T18:14:53.557733+0000","last_unstale":"2023-12-19T18:14:54.073092+0000","last_undegraded":"2023-12-19T18:14:54.073092+0000","last_fullsized":"2023-12-19T18:14:54.073092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929138,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7210000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8180000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":71,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927026,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199928680,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.47}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4239999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927698,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7610000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2720000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.3109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.3929999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.1839999999999993}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1059999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1449999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338824,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929206,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5780000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9590000000000001}]}]}],"pool_statfs":[]}} 2023-12-19T18:15:09.966 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:15:10.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.236+0000 7f0989b4f640 1 Processor -- start 2023-12-19T18:15:10.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.236+0000 7f0989b4f640 1 -- start start 2023-12-19T18:15:10.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.238+0000 7f0989b4f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f09841553a0 0x7f09840c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.238+0000 7f0989b4f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f098415ee30 0x7f09840c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.238+0000 7f0989b4f640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09840b0360 con 0x7f09840afeb0 2023-12-19T18:15:10.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.238+0000 7f0989b4f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09840cbcf0 con 0x7f09841553a0 2023-12-19T18:15:10.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.238+0000 7f0989b4f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09840cb350 con 0x7f098415ee30 2023-12-19T18:15:10.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.239+0000 7f0988b4d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f09841553a0 0x7f09840c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:10.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.239+0000 7f0988b4d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f09841553a0 0x7f09840c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49824/0 (socket says 172.21.15.27:49824) 2023-12-19T18:15:10.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.239+0000 7f0988b4d640 1 -- 172.21.15.27:0/1959864734 learned_addr learned my addr 172.21.15.27:0/1959864734 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:10.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.239+0000 7f0988b4d640 1 -- 172.21.15.27:0/1959864734 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f098415ee30 msgr2=0x7f09840c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:10.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.239+0000 7f098934e640 1 --2- 172.21.15.27:0/1959864734 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f098415ee30 0x7f09840c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:10.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.239+0000 7f0988b4d640 1 --2- 172.21.15.27:0/1959864734 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f098415ee30 0x7f09840c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.239+0000 7f0988b4d640 1 -- 172.21.15.27:0/1959864734 >> v1:172.21.15.192:6789/0 conn(0x7f09840afeb0 legacy=0x7f098415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:10.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.239+0000 7f0988b4d640 1 -- 172.21.15.27:0/1959864734 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09840cb630 con 0x7f09841553a0 2023-12-19T18:15:10.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.239+0000 7f098934e640 1 --2- 172.21.15.27:0/1959864734 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f098415ee30 0x7f09840c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:10.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.240+0000 7f0988b4d640 1 --2- 172.21.15.27:0/1959864734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09841553a0 0x7f09840c50a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0978009870 tx=0x7f097802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96be82e5100f6b5d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:10.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.241+0000 7f09837fe640 1 -- 172.21.15.27:0/1959864734 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09780317c0 con 0x7f09841553a0 2023-12-19T18:15:10.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.241+0000 7f09837fe640 1 -- 172.21.15.27:0/1959864734 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0978031920 con 0x7f09841553a0 2023-12-19T18:15:10.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.241+0000 7f09837fe640 1 -- 172.21.15.27:0/1959864734 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0978031cb0 con 0x7f09841553a0 2023-12-19T18:15:10.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.241+0000 7f0989b4f640 1 -- 172.21.15.27:0/1959864734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09841553a0 msgr2=0x7f09840c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:10.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.241+0000 7f0989b4f640 1 --2- 172.21.15.27:0/1959864734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09841553a0 0x7f09840c50a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0978009870 tx=0x7f097802b860 comp rx=0 tx=0).stop 2023-12-19T18:15:10.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.241+0000 7f0989b4f640 1 -- 172.21.15.27:0/1959864734 shutdown_connections 2023-12-19T18:15:10.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.241+0000 7f0989b4f640 1 --2- 172.21.15.27:0/1959864734 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f098415ee30 0x7f09840c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.241+0000 7f0989b4f640 1 --2- 172.21.15.27:0/1959864734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09841553a0 0x7f09840c50a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.241+0000 7f0989b4f640 1 -- 172.21.15.27:0/1959864734 >> 172.21.15.27:0/1959864734 conn(0x7f0984069c70 msgr2=0x7f098406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:10.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.241+0000 7f0989b4f640 1 -- 172.21.15.27:0/1959864734 shutdown_connections 2023-12-19T18:15:10.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.241+0000 7f0989b4f640 1 -- 172.21.15.27:0/1959864734 wait complete. 2023-12-19T18:15:10.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.243+0000 7f0989b4f640 1 Processor -- start 2023-12-19T18:15:10.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.243+0000 7f0989b4f640 1 -- start start 2023-12-19T18:15:10.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.244+0000 7f0989b4f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09840afeb0 0x7f09841434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.244+0000 7f0989b4f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09841553a0 0x7f098415c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.244+0000 7f0989b4f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098415ee30 0x7f09840c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.244+0000 7f0989b4f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09840cd150 con 0x7f09840afeb0 2023-12-19T18:15:10.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.244+0000 7f0989b4f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09840ad950 con 0x7f098415ee30 2023-12-19T18:15:10.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.244+0000 7f0989b4f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09840ad0b0 con 0x7f09841553a0 2023-12-19T18:15:10.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.245+0000 7f0988b4d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09840afeb0 0x7f09841434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:10.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.245+0000 7f0988b4d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09840afeb0 0x7f09841434b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49832/0 (socket says 172.21.15.27:49832) 2023-12-19T18:15:10.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.245+0000 7f0988b4d640 1 -- 172.21.15.27:0/2082809832 learned_addr learned my addr 172.21.15.27:0/2082809832 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:10.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.245+0000 7f0988b4d640 1 -- 172.21.15.27:0/2082809832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09841553a0 msgr2=0x7f098415c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:10.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.245+0000 7f0988b4d640 1 --2- 172.21.15.27:0/2082809832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09841553a0 0x7f098415c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.245+0000 7f0988b4d640 1 -- 172.21.15.27:0/2082809832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098415ee30 msgr2=0x7f09840c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:10.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.245+0000 7f098934e640 1 --2- 172.21.15.27:0/2082809832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098415ee30 0x7f09840c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:10.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.245+0000 7f0988b4d640 1 --2- 172.21.15.27:0/2082809832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098415ee30 0x7f09840c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.245+0000 7f0988b4d640 1 -- 172.21.15.27:0/2082809832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09780094d0 con 0x7f09840afeb0 2023-12-19T18:15:10.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.245+0000 7f0988b4d640 1 --2- 172.21.15.27:0/2082809832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09840afeb0 0x7f09841434b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0978000c00 tx=0x7f09780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:10.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.246+0000 7f09817fa640 1 -- 172.21.15.27:0/2082809832 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09780329a0 con 0x7f09840afeb0 2023-12-19T18:15:10.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.246+0000 7f09817fa640 1 -- 172.21.15.27:0/2082809832 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0978032b20 con 0x7f09840afeb0 2023-12-19T18:15:10.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.246+0000 7f09817fa640 1 -- 172.21.15.27:0/2082809832 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09780453f0 con 0x7f09840afeb0 2023-12-19T18:15:10.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.246+0000 7f0989b4f640 1 -- 172.21.15.27:0/2082809832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09840ad390 con 0x7f09840afeb0 2023-12-19T18:15:10.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.246+0000 7f0989b4f640 1 -- 172.21.15.27:0/2082809832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098415cd20 con 0x7f09840afeb0 2023-12-19T18:15:10.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.247+0000 7f0989b4f640 1 -- 172.21.15.27:0/2082809832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09840ccf20 con 0x7f09840afeb0 2023-12-19T18:15:10.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.248+0000 7f09817fa640 1 -- 172.21.15.27:0/2082809832 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f097804e450 con 0x7f09840afeb0 2023-12-19T18:15:10.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.249+0000 7f09817fa640 1 --2- 172.21.15.27:0/2082809832 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f096806a430 0x7f096806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.249+0000 7f09817fa640 1 -- 172.21.15.27:0/2082809832 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f09780aac30 con 0x7f09840afeb0 2023-12-19T18:15:10.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.249+0000 7f0983fff640 1 --2- 172.21.15.27:0/2082809832 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f096806a430 0x7f096806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:10.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.251+0000 7f0983fff640 1 --2- 172.21.15.27:0/2082809832 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f096806a430 0x7f096806c8f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09841f5f70 tx=0x7f09700079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:10.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.253+0000 7f09817fa640 1 -- 172.21.15.27:0/2082809832 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f09840ccf20 con 0x7f09840afeb0 2023-12-19T18:15:10.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.434+0000 7f0989b4f640 1 -- 172.21.15.27:0/2082809832 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f09840c5730 con 0x7f096806a430 2023-12-19T18:15:10.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.454+0000 7f09817fa640 1 -- 172.21.15.27:0/2082809832 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+260043 (secure 0 0 0) 0x7f0970014400 con 0x7f096806a430 2023-12-19T18:15:10.458 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:10.459 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:15:10.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.459+0000 7f0989b4f640 1 -- 172.21.15.27:0/2082809832 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f096806a430 msgr2=0x7f096806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:10.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.459+0000 7f0989b4f640 1 --2- 172.21.15.27:0/2082809832 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f096806a430 0x7f096806c8f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f09841f5f70 tx=0x7f09700079b0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.459+0000 7f0989b4f640 1 -- 172.21.15.27:0/2082809832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09840afeb0 msgr2=0x7f09841434b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:10.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.459+0000 7f0989b4f640 1 --2- 172.21.15.27:0/2082809832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09840afeb0 0x7f09841434b0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0978000c00 tx=0x7f09780046c0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.460+0000 7f0989b4f640 1 -- 172.21.15.27:0/2082809832 shutdown_connections 2023-12-19T18:15:10.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.460+0000 7f0989b4f640 1 --2- 172.21.15.27:0/2082809832 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f096806a430 0x7f096806c8f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.460+0000 7f0989b4f640 1 --2- 172.21.15.27:0/2082809832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f098415ee30 0x7f09840c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.460+0000 7f0989b4f640 1 --2- 172.21.15.27:0/2082809832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f09841553a0 0x7f098415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.460+0000 7f0989b4f640 1 --2- 172.21.15.27:0/2082809832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09840afeb0 0x7f09841434b0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.460+0000 7f0989b4f640 1 -- 172.21.15.27:0/2082809832 >> 172.21.15.27:0/2082809832 conn(0x7f0984069c70 msgr2=0x7f09840c47f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:10.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.460+0000 7f0989b4f640 1 -- 172.21.15.27:0/2082809832 shutdown_connections 2023-12-19T18:15:10.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.460+0000 7f0989b4f640 1 -- 172.21.15.27:0/2082809832 wait complete. 2023-12-19T18:15:10.478 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-19T18:15:09.707231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":1710,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560789,"statfs":{"total":1600000000,"available":1598248636,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004644"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056823+0000","last_change":"2023-12-19T18:14:54.056711+0000","last_active":"2023-12-19T18:14:54.056823+0000","last_peered":"2023-12-19T18:14:54.056823+0000","last_clean":"2023-12-19T18:14:54.056823+0000","last_became_active":"2023-12-19T18:14:54.056335+0000","last_became_peered":"2023-12-19T18:14:54.056335+0000","last_unstale":"2023-12-19T18:14:54.056823+0000","last_undegraded":"2023-12-19T18:14:54.056823+0000","last_fullsized":"2023-12-19T18:14:54.056823+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069974+0000","last_change":"2023-12-19T18:14:54.069882+0000","last_active":"2023-12-19T18:14:54.069974+0000","last_peered":"2023-12-19T18:14:54.069974+0000","last_clean":"2023-12-19T18:14:54.069974+0000","last_became_active":"2023-12-19T18:14:54.069626+0000","last_became_peered":"2023-12-19T18:14:54.069626+0000","last_unstale":"2023-12-19T18:14:54.069974+0000","last_undegraded":"2023-12-19T18:14:54.069974+0000","last_fullsized":"2023-12-19T18:14:54.069974+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053578+0000","last_change":"2023-12-19T18:14:54.053111+0000","last_active":"2023-12-19T18:14:54.053578+0000","last_peered":"2023-12-19T18:14:54.053578+0000","last_clean":"2023-12-19T18:14:54.053578+0000","last_became_active":"2023-12-19T18:14:54.052858+0000","last_became_peered":"2023-12-19T18:14:54.052858+0000","last_unstale":"2023-12-19T18:14:54.053578+0000","last_undegraded":"2023-12-19T18:14:54.053578+0000","last_fullsized":"2023-12-19T18:14:54.053578+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050310+0000","last_change":"2023-12-19T18:14:54.049862+0000","last_active":"2023-12-19T18:14:54.050310+0000","last_peered":"2023-12-19T18:14:54.050310+0000","last_clean":"2023-12-19T18:14:54.050310+0000","last_became_active":"2023-12-19T18:14:54.049396+0000","last_became_peered":"2023-12-19T18:14:54.049396+0000","last_unstale":"2023-12-19T18:14:54.050310+0000","last_undegraded":"2023-12-19T18:14:54.050310+0000","last_fullsized":"2023-12-19T18:14:54.050310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053036+0000","last_change":"2023-12-19T18:14:54.052387+0000","last_active":"2023-12-19T18:14:54.053036+0000","last_peered":"2023-12-19T18:14:54.053036+0000","last_clean":"2023-12-19T18:14:54.053036+0000","last_became_active":"2023-12-19T18:14:54.052008+0000","last_became_peered":"2023-12-19T18:14:54.052008+0000","last_unstale":"2023-12-19T18:14:54.053036+0000","last_undegraded":"2023-12-19T18:14:54.053036+0000","last_fullsized":"2023-12-19T18:14:54.053036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061254+0000","last_change":"2023-12-19T18:14:54.061175+0000","last_active":"2023-12-19T18:14:54.061254+0000","last_peered":"2023-12-19T18:14:54.061254+0000","last_clean":"2023-12-19T18:14:54.061254+0000","last_became_active":"2023-12-19T18:14:54.060192+0000","last_became_peered":"2023-12-19T18:14:54.060192+0000","last_unstale":"2023-12-19T18:14:54.061254+0000","last_undegraded":"2023-12-19T18:14:54.061254+0000","last_fullsized":"2023-12-19T18:14:54.061254+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.054262+0000","last_change":"2023-12-19T18:14:54.053283+0000","last_active":"2023-12-19T18:14:54.054262+0000","last_peered":"2023-12-19T18:14:54.054262+0000","last_clean":"2023-12-19T18:14:54.054262+0000","last_became_active":"2023-12-19T18:14:54.052854+0000","last_became_peered":"2023-12-19T18:14:54.052854+0000","last_unstale":"2023-12-19T18:14:54.054262+0000","last_undegraded":"2023-12-19T18:14:54.054262+0000","last_fullsized":"2023-12-19T18:14:54.054262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:38.686221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056579+0000","last_change":"2023-12-19T18:14:54.056483+0000","last_active":"2023-12-19T18:14:54.056579+0000","last_peered":"2023-12-19T18:14:54.056579+0000","last_clean":"2023-12-19T18:14:54.056579+0000","last_became_active":"2023-12-19T18:14:54.056196+0000","last_became_peered":"2023-12-19T18:14:54.056196+0000","last_unstale":"2023-12-19T18:14:54.056579+0000","last_undegraded":"2023-12-19T18:14:54.056579+0000","last_fullsized":"2023-12-19T18:14:54.056579+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075945+0000","last_change":"2023-12-19T18:14:54.074709+0000","last_active":"2023-12-19T18:14:54.075945+0000","last_peered":"2023-12-19T18:14:54.075945+0000","last_clean":"2023-12-19T18:14:54.075945+0000","last_became_active":"2023-12-19T18:14:54.074482+0000","last_became_peered":"2023-12-19T18:14:54.074482+0000","last_unstale":"2023-12-19T18:14:54.075945+0000","last_undegraded":"2023-12-19T18:14:54.075945+0000","last_fullsized":"2023-12-19T18:14:54.075945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051054+0000","last_change":"2023-12-19T18:14:54.050203+0000","last_active":"2023-12-19T18:14:54.051054+0000","last_peered":"2023-12-19T18:14:54.051054+0000","last_clean":"2023-12-19T18:14:54.051054+0000","last_became_active":"2023-12-19T18:14:54.049582+0000","last_became_peered":"2023-12-19T18:14:54.049582+0000","last_unstale":"2023-12-19T18:14:54.051054+0000","last_undegraded":"2023-12-19T18:14:54.051054+0000","last_fullsized":"2023-12-19T18:14:54.051054+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056806+0000","last_change":"2023-12-19T18:14:54.056564+0000","last_active":"2023-12-19T18:14:54.056806+0000","last_peered":"2023-12-19T18:14:54.056806+0000","last_clean":"2023-12-19T18:14:54.056806+0000","last_became_active":"2023-12-19T18:14:54.056052+0000","last_became_peered":"2023-12-19T18:14:54.056052+0000","last_unstale":"2023-12-19T18:14:54.056806+0000","last_undegraded":"2023-12-19T18:14:54.056806+0000","last_fullsized":"2023-12-19T18:14:54.056806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066528+0000","last_change":"2023-12-19T18:14:54.066257+0000","last_active":"2023-12-19T18:14:54.066528+0000","last_peered":"2023-12-19T18:14:54.066528+0000","last_clean":"2023-12-19T18:14:54.066528+0000","last_became_active":"2023-12-19T18:14:54.065876+0000","last_became_peered":"2023-12-19T18:14:54.065876+0000","last_unstale":"2023-12-19T18:14:54.066528+0000","last_undegraded":"2023-12-19T18:14:54.066528+0000","last_fullsized":"2023-12-19T18:14:54.066528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070727+0000","last_change":"2023-12-19T18:14:54.070658+0000","last_active":"2023-12-19T18:14:54.070727+0000","last_peered":"2023-12-19T18:14:54.070727+0000","last_clean":"2023-12-19T18:14:54.070727+0000","last_became_active":"2023-12-19T18:14:54.065952+0000","last_became_peered":"2023-12-19T18:14:54.065952+0000","last_unstale":"2023-12-19T18:14:54.070727+0000","last_undegraded":"2023-12-19T18:14:54.070727+0000","last_fullsized":"2023-12-19T18:14:54.070727+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:35:28.203585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072334+0000","last_change":"2023-12-19T18:14:54.072211+0000","last_active":"2023-12-19T18:14:54.072334+0000","last_peered":"2023-12-19T18:14:54.072334+0000","last_clean":"2023-12-19T18:14:54.072334+0000","last_became_active":"2023-12-19T18:14:54.071912+0000","last_became_peered":"2023-12-19T18:14:54.071912+0000","last_unstale":"2023-12-19T18:14:54.072334+0000","last_undegraded":"2023-12-19T18:14:54.072334+0000","last_fullsized":"2023-12-19T18:14:54.072334+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056480+0000","last_change":"2023-12-19T18:14:54.055761+0000","last_active":"2023-12-19T18:14:54.056480+0000","last_peered":"2023-12-19T18:14:54.056480+0000","last_clean":"2023-12-19T18:14:54.056480+0000","last_became_active":"2023-12-19T18:14:54.055529+0000","last_became_peered":"2023-12-19T18:14:54.055529+0000","last_unstale":"2023-12-19T18:14:54.056480+0000","last_undegraded":"2023-12-19T18:14:54.056480+0000","last_fullsized":"2023-12-19T18:14:54.056480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073307+0000","last_change":"2023-12-19T18:14:54.072540+0000","last_active":"2023-12-19T18:14:54.073307+0000","last_peered":"2023-12-19T18:14:54.073307+0000","last_clean":"2023-12-19T18:14:54.073307+0000","last_became_active":"2023-12-19T18:14:54.072162+0000","last_became_peered":"2023-12-19T18:14:54.072162+0000","last_unstale":"2023-12-19T18:14:54.073307+0000","last_undegraded":"2023-12-19T18:14:54.073307+0000","last_fullsized":"2023-12-19T18:14:54.073307+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.249892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075801+0000","last_change":"2023-12-19T18:14:54.075571+0000","last_active":"2023-12-19T18:14:54.075801+0000","last_peered":"2023-12-19T18:14:54.075801+0000","last_clean":"2023-12-19T18:14:54.075801+0000","last_became_active":"2023-12-19T18:14:54.074613+0000","last_became_peered":"2023-12-19T18:14:54.074613+0000","last_unstale":"2023-12-19T18:14:54.075801+0000","last_undegraded":"2023-12-19T18:14:54.075801+0000","last_fullsized":"2023-12-19T18:14:54.075801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:15.120564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.062708+0000","last_change":"2023-12-19T18:14:54.062640+0000","last_active":"2023-12-19T18:14:54.062708+0000","last_peered":"2023-12-19T18:14:54.062708+0000","last_clean":"2023-12-19T18:14:54.062708+0000","last_became_active":"2023-12-19T18:14:54.061994+0000","last_became_peered":"2023-12-19T18:14:54.061994+0000","last_unstale":"2023-12-19T18:14:54.062708+0000","last_undegraded":"2023-12-19T18:14:54.062708+0000","last_fullsized":"2023-12-19T18:14:54.062708+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073330+0000","last_change":"2023-12-19T18:14:54.073175+0000","last_active":"2023-12-19T18:14:54.073330+0000","last_peered":"2023-12-19T18:14:54.073330+0000","last_clean":"2023-12-19T18:14:54.073330+0000","last_became_active":"2023-12-19T18:14:54.072223+0000","last_became_peered":"2023-12-19T18:14:54.072223+0000","last_unstale":"2023-12-19T18:14:54.073330+0000","last_undegraded":"2023-12-19T18:14:54.073330+0000","last_fullsized":"2023-12-19T18:14:54.073330+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:35.205536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064684+0000","last_change":"2023-12-19T18:14:54.064206+0000","last_active":"2023-12-19T18:14:54.064684+0000","last_peered":"2023-12-19T18:14:54.064684+0000","last_clean":"2023-12-19T18:14:54.064684+0000","last_became_active":"2023-12-19T18:14:54.063818+0000","last_became_peered":"2023-12-19T18:14:54.063818+0000","last_unstale":"2023-12-19T18:14:54.064684+0000","last_undegraded":"2023-12-19T18:14:54.064684+0000","last_fullsized":"2023-12-19T18:14:54.064684+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063666+0000","last_change":"2023-12-19T18:14:54.063234+0000","last_active":"2023-12-19T18:14:54.063666+0000","last_peered":"2023-12-19T18:14:54.063666+0000","last_clean":"2023-12-19T18:14:54.063666+0000","last_became_active":"2023-12-19T18:14:54.062885+0000","last_became_peered":"2023-12-19T18:14:54.062885+0000","last_unstale":"2023-12-19T18:14:54.063666+0000","last_undegraded":"2023-12-19T18:14:54.063666+0000","last_fullsized":"2023-12-19T18:14:54.063666+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063662+0000","last_change":"2023-12-19T18:14:54.063598+0000","last_active":"2023-12-19T18:14:54.063662+0000","last_peered":"2023-12-19T18:14:54.063662+0000","last_clean":"2023-12-19T18:14:54.063662+0000","last_became_active":"2023-12-19T18:14:54.063314+0000","last_became_peered":"2023-12-19T18:14:54.063314+0000","last_unstale":"2023-12-19T18:14:54.063662+0000","last_undegraded":"2023-12-19T18:14:54.063662+0000","last_fullsized":"2023-12-19T18:14:54.063662+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060228+0000","last_change":"2023-12-19T18:14:54.059917+0000","last_active":"2023-12-19T18:14:54.060228+0000","last_peered":"2023-12-19T18:14:54.060228+0000","last_clean":"2023-12-19T18:14:54.060228+0000","last_became_active":"2023-12-19T18:14:54.059674+0000","last_became_peered":"2023-12-19T18:14:54.059674+0000","last_unstale":"2023-12-19T18:14:54.060228+0000","last_undegraded":"2023-12-19T18:14:54.060228+0000","last_fullsized":"2023-12-19T18:14:54.060228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074581+0000","last_change":"2023-12-19T18:14:54.073659+0000","last_active":"2023-12-19T18:14:54.074581+0000","last_peered":"2023-12-19T18:14:54.074581+0000","last_clean":"2023-12-19T18:14:54.074581+0000","last_became_active":"2023-12-19T18:14:54.072947+0000","last_became_peered":"2023-12-19T18:14:54.072947+0000","last_unstale":"2023-12-19T18:14:54.074581+0000","last_undegraded":"2023-12-19T18:14:54.074581+0000","last_fullsized":"2023-12-19T18:14:54.074581+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067036+0000","last_change":"2023-12-19T18:14:54.066189+0000","last_active":"2023-12-19T18:14:54.067036+0000","last_peered":"2023-12-19T18:14:54.067036+0000","last_clean":"2023-12-19T18:14:54.067036+0000","last_became_active":"2023-12-19T18:14:54.065921+0000","last_became_peered":"2023-12-19T18:14:54.065921+0000","last_unstale":"2023-12-19T18:14:54.067036+0000","last_undegraded":"2023-12-19T18:14:54.067036+0000","last_fullsized":"2023-12-19T18:14:54.067036+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061440+0000","last_change":"2023-12-19T18:14:54.061372+0000","last_active":"2023-12-19T18:14:54.061440+0000","last_peered":"2023-12-19T18:14:54.061440+0000","last_clean":"2023-12-19T18:14:54.061440+0000","last_became_active":"2023-12-19T18:14:54.060949+0000","last_became_peered":"2023-12-19T18:14:54.060949+0000","last_unstale":"2023-12-19T18:14:54.061440+0000","last_undegraded":"2023-12-19T18:14:54.061440+0000","last_fullsized":"2023-12-19T18:14:54.061440+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073948+0000","last_change":"2023-12-19T18:14:54.073038+0000","last_active":"2023-12-19T18:14:54.073948+0000","last_peered":"2023-12-19T18:14:54.073948+0000","last_clean":"2023-12-19T18:14:54.073948+0000","last_became_active":"2023-12-19T18:14:54.071844+0000","last_became_peered":"2023-12-19T18:14:54.071844+0000","last_unstale":"2023-12-19T18:14:54.073948+0000","last_undegraded":"2023-12-19T18:14:54.073948+0000","last_fullsized":"2023-12-19T18:14:54.073948+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070883+0000","last_change":"2023-12-19T18:14:54.070817+0000","last_active":"2023-12-19T18:14:54.070883+0000","last_peered":"2023-12-19T18:14:54.070883+0000","last_clean":"2023-12-19T18:14:54.070883+0000","last_became_active":"2023-12-19T18:14:54.068216+0000","last_became_peered":"2023-12-19T18:14:54.068216+0000","last_unstale":"2023-12-19T18:14:54.070883+0000","last_undegraded":"2023-12-19T18:14:54.070883+0000","last_fullsized":"2023-12-19T18:14:54.070883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:52.195670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063273+0000","last_change":"2023-12-19T18:14:54.060522+0000","last_active":"2023-12-19T18:14:54.063273+0000","last_peered":"2023-12-19T18:14:54.063273+0000","last_clean":"2023-12-19T18:14:54.063273+0000","last_became_active":"2023-12-19T18:14:54.059641+0000","last_became_peered":"2023-12-19T18:14:54.059641+0000","last_unstale":"2023-12-19T18:14:54.063273+0000","last_undegraded":"2023-12-19T18:14:54.063273+0000","last_fullsized":"2023-12-19T18:14:54.063273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.260981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065928+0000","last_change":"2023-12-19T18:14:54.065766+0000","last_active":"2023-12-19T18:14:54.065928+0000","last_peered":"2023-12-19T18:14:54.065928+0000","last_clean":"2023-12-19T18:14:54.065928+0000","last_became_active":"2023-12-19T18:14:54.058571+0000","last_became_peered":"2023-12-19T18:14:54.058571+0000","last_unstale":"2023-12-19T18:14:54.065928+0000","last_undegraded":"2023-12-19T18:14:54.065928+0000","last_fullsized":"2023-12-19T18:14:54.065928+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:47.213095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070429+0000","last_change":"2023-12-19T18:14:54.067798+0000","last_active":"2023-12-19T18:14:54.070429+0000","last_peered":"2023-12-19T18:14:54.070429+0000","last_clean":"2023-12-19T18:14:54.070429+0000","last_became_active":"2023-12-19T18:14:54.067576+0000","last_became_peered":"2023-12-19T18:14:54.067576+0000","last_unstale":"2023-12-19T18:14:54.070429+0000","last_undegraded":"2023-12-19T18:14:54.070429+0000","last_fullsized":"2023-12-19T18:14:54.070429+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069889+0000","last_change":"2023-12-19T18:14:54.069741+0000","last_active":"2023-12-19T18:14:54.069889+0000","last_peered":"2023-12-19T18:14:54.069889+0000","last_clean":"2023-12-19T18:14:54.069889+0000","last_became_active":"2023-12-19T18:14:54.069513+0000","last_became_peered":"2023-12-19T18:14:54.069513+0000","last_unstale":"2023-12-19T18:14:54.069889+0000","last_undegraded":"2023-12-19T18:14:54.069889+0000","last_fullsized":"2023-12-19T18:14:54.069889+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070495+0000","last_change":"2023-12-19T18:14:54.070403+0000","last_active":"2023-12-19T18:14:54.070495+0000","last_peered":"2023-12-19T18:14:54.070495+0000","last_clean":"2023-12-19T18:14:54.070495+0000","last_became_active":"2023-12-19T18:14:54.069801+0000","last_became_peered":"2023-12-19T18:14:54.069801+0000","last_unstale":"2023-12-19T18:14:54.070495+0000","last_undegraded":"2023-12-19T18:14:54.070495+0000","last_fullsized":"2023-12-19T18:14:54.070495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:46.208613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068551+0000","last_change":"2023-12-19T18:14:53.563064+0000","last_active":"2023-12-19T18:14:54.068551+0000","last_peered":"2023-12-19T18:14:54.068551+0000","last_clean":"2023-12-19T18:14:54.068551+0000","last_became_active":"2023-12-19T18:14:53.561534+0000","last_became_peered":"2023-12-19T18:14:53.561534+0000","last_unstale":"2023-12-19T18:14:54.068551+0000","last_undegraded":"2023-12-19T18:14:54.068551+0000","last_fullsized":"2023-12-19T18:14:54.068551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051639+0000","last_change":"2023-12-19T18:14:54.051570+0000","last_active":"2023-12-19T18:14:54.051639+0000","last_peered":"2023-12-19T18:14:54.051639+0000","last_clean":"2023-12-19T18:14:54.051639+0000","last_became_active":"2023-12-19T18:14:54.051173+0000","last_became_peered":"2023-12-19T18:14:54.051173+0000","last_unstale":"2023-12-19T18:14:54.051639+0000","last_undegraded":"2023-12-19T18:14:54.051639+0000","last_fullsized":"2023-12-19T18:14:54.051639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073383+0000","last_change":"2023-12-19T18:14:53.560763+0000","last_active":"2023-12-19T18:14:54.073383+0000","last_peered":"2023-12-19T18:14:54.073383+0000","last_clean":"2023-12-19T18:14:54.073383+0000","last_became_active":"2023-12-19T18:14:53.558648+0000","last_became_peered":"2023-12-19T18:14:53.558648+0000","last_unstale":"2023-12-19T18:14:54.073383+0000","last_undegraded":"2023-12-19T18:14:54.073383+0000","last_fullsized":"2023-12-19T18:14:54.073383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:25:51.179899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066389+0000","last_change":"2023-12-19T18:14:53.560483+0000","last_active":"2023-12-19T18:14:54.066389+0000","last_peered":"2023-12-19T18:14:54.066389+0000","last_clean":"2023-12-19T18:14:54.066389+0000","last_became_active":"2023-12-19T18:14:53.560244+0000","last_became_peered":"2023-12-19T18:14:53.560244+0000","last_unstale":"2023-12-19T18:14:54.066389+0000","last_undegraded":"2023-12-19T18:14:54.066389+0000","last_fullsized":"2023-12-19T18:14:54.066389+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072273+0000","last_change":"2023-12-19T18:14:54.071383+0000","last_active":"2023-12-19T18:14:54.072273+0000","last_peered":"2023-12-19T18:14:54.072273+0000","last_clean":"2023-12-19T18:14:54.072273+0000","last_became_active":"2023-12-19T18:14:54.070954+0000","last_became_peered":"2023-12-19T18:14:54.070954+0000","last_unstale":"2023-12-19T18:14:54.072273+0000","last_undegraded":"2023-12-19T18:14:54.072273+0000","last_fullsized":"2023-12-19T18:14:54.072273+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053551+0000","last_change":"2023-12-19T18:14:53.558346+0000","last_active":"2023-12-19T18:14:54.053551+0000","last_peered":"2023-12-19T18:14:54.053551+0000","last_clean":"2023-12-19T18:14:54.053551+0000","last_became_active":"2023-12-19T18:14:53.558124+0000","last_became_peered":"2023-12-19T18:14:53.558124+0000","last_unstale":"2023-12-19T18:14:54.053551+0000","last_undegraded":"2023-12-19T18:14:54.053551+0000","last_fullsized":"2023-12-19T18:14:54.053551+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066236+0000","last_change":"2023-12-19T18:14:54.064228+0000","last_active":"2023-12-19T18:14:54.066236+0000","last_peered":"2023-12-19T18:14:54.066236+0000","last_clean":"2023-12-19T18:14:54.066236+0000","last_became_active":"2023-12-19T18:14:54.063991+0000","last_became_peered":"2023-12-19T18:14:54.063991+0000","last_unstale":"2023-12-19T18:14:54.066236+0000","last_undegraded":"2023-12-19T18:14:54.066236+0000","last_fullsized":"2023-12-19T18:14:54.066236+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:39.682156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064163+0000","last_change":"2023-12-19T18:14:53.554048+0000","last_active":"2023-12-19T18:14:54.064163+0000","last_peered":"2023-12-19T18:14:54.064163+0000","last_clean":"2023-12-19T18:14:54.064163+0000","last_became_active":"2023-12-19T18:14:53.553811+0000","last_became_peered":"2023-12-19T18:14:53.553811+0000","last_unstale":"2023-12-19T18:14:54.064163+0000","last_undegraded":"2023-12-19T18:14:54.064163+0000","last_fullsized":"2023-12-19T18:14:54.064163+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073051+0000","last_change":"2023-12-19T18:14:54.072974+0000","last_active":"2023-12-19T18:14:54.073051+0000","last_peered":"2023-12-19T18:14:54.073051+0000","last_clean":"2023-12-19T18:14:54.073051+0000","last_became_active":"2023-12-19T18:14:54.072176+0000","last_became_peered":"2023-12-19T18:14:54.072176+0000","last_unstale":"2023-12-19T18:14:54.073051+0000","last_undegraded":"2023-12-19T18:14:54.073051+0000","last_fullsized":"2023-12-19T18:14:54.073051+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.043565+0000","last_change":"2023-12-19T18:14:53.557947+0000","last_active":"2023-12-19T18:14:54.043565+0000","last_peered":"2023-12-19T18:14:54.043565+0000","last_clean":"2023-12-19T18:14:54.043565+0000","last_became_active":"2023-12-19T18:14:53.557713+0000","last_became_peered":"2023-12-19T18:14:53.557713+0000","last_unstale":"2023-12-19T18:14:54.043565+0000","last_undegraded":"2023-12-19T18:14:54.043565+0000","last_fullsized":"2023-12-19T18:14:54.043565+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.056501+0000","last_change":"2023-12-19T18:14:54.056275+0000","last_active":"2023-12-19T18:14:54.056501+0000","last_peered":"2023-12-19T18:14:54.056501+0000","last_clean":"2023-12-19T18:14:54.056501+0000","last_became_active":"2023-12-19T18:14:54.056002+0000","last_became_peered":"2023-12-19T18:14:54.056002+0000","last_unstale":"2023-12-19T18:14:54.056501+0000","last_undegraded":"2023-12-19T18:14:54.056501+0000","last_fullsized":"2023-12-19T18:14:54.056501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:30.847801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":14,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073216+0000","last_change":"2023-12-19T18:14:53.568016+0000","last_active":"2023-12-19T18:14:54.073216+0000","last_peered":"2023-12-19T18:14:54.073216+0000","last_clean":"2023-12-19T18:14:54.073216+0000","last_became_active":"2023-12-19T18:14:53.567634+0000","last_became_peered":"2023-12-19T18:14:53.567634+0000","last_unstale":"2023-12-19T18:14:54.073216+0000","last_undegraded":"2023-12-19T18:14:54.073216+0000","last_fullsized":"2023-12-19T18:14:54.073216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.075897+0000","last_change":"2023-12-19T18:14:54.075827+0000","last_active":"2023-12-19T18:14:54.075897+0000","last_peered":"2023-12-19T18:14:54.075897+0000","last_clean":"2023-12-19T18:14:54.075897+0000","last_became_active":"2023-12-19T18:14:54.074747+0000","last_became_peered":"2023-12-19T18:14:54.074747+0000","last_unstale":"2023-12-19T18:14:54.075897+0000","last_undegraded":"2023-12-19T18:14:54.075897+0000","last_fullsized":"2023-12-19T18:14:54.075897+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:44:49.908312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":17,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073455+0000","last_change":"2023-12-19T18:14:53.547427+0000","last_active":"2023-12-19T18:14:54.073455+0000","last_peered":"2023-12-19T18:14:54.073455+0000","last_clean":"2023-12-19T18:14:54.073455+0000","last_became_active":"2023-12-19T18:14:53.546733+0000","last_became_peered":"2023-12-19T18:14:53.546733+0000","last_unstale":"2023-12-19T18:14:54.073455+0000","last_undegraded":"2023-12-19T18:14:54.073455+0000","last_fullsized":"2023-12-19T18:14:54.073455+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:13:12.090625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069234+0000","last_change":"2023-12-19T18:14:54.069070+0000","last_active":"2023-12-19T18:14:54.069234+0000","last_peered":"2023-12-19T18:14:54.069234+0000","last_clean":"2023-12-19T18:14:54.069234+0000","last_became_active":"2023-12-19T18:14:54.066495+0000","last_became_peered":"2023-12-19T18:14:54.066495+0000","last_unstale":"2023-12-19T18:14:54.069234+0000","last_undegraded":"2023-12-19T18:14:54.069234+0000","last_fullsized":"2023-12-19T18:14:54.069234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068132+0000","last_change":"2023-12-19T18:14:53.566349+0000","last_active":"2023-12-19T18:14:54.068132+0000","last_peered":"2023-12-19T18:14:54.068132+0000","last_clean":"2023-12-19T18:14:54.068132+0000","last_became_active":"2023-12-19T18:14:53.565909+0000","last_became_peered":"2023-12-19T18:14:53.565909+0000","last_unstale":"2023-12-19T18:14:54.068132+0000","last_undegraded":"2023-12-19T18:14:54.068132+0000","last_fullsized":"2023-12-19T18:14:54.068132+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073929+0000","last_change":"2023-12-19T18:14:54.072903+0000","last_active":"2023-12-19T18:14:54.073929+0000","last_peered":"2023-12-19T18:14:54.073929+0000","last_clean":"2023-12-19T18:14:54.073929+0000","last_became_active":"2023-12-19T18:14:54.072678+0000","last_became_peered":"2023-12-19T18:14:54.072678+0000","last_unstale":"2023-12-19T18:14:54.073929+0000","last_undegraded":"2023-12-19T18:14:54.073929+0000","last_fullsized":"2023-12-19T18:14:54.073929+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:54.834723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.052955+0000","last_change":"2023-12-19T18:14:53.561604+0000","last_active":"2023-12-19T18:14:54.052955+0000","last_peered":"2023-12-19T18:14:54.052955+0000","last_clean":"2023-12-19T18:14:54.052955+0000","last_became_active":"2023-12-19T18:14:53.561372+0000","last_became_peered":"2023-12-19T18:14:53.561372+0000","last_unstale":"2023-12-19T18:14:54.052955+0000","last_undegraded":"2023-12-19T18:14:54.052955+0000","last_fullsized":"2023-12-19T18:14:54.052955+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:00.255541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.057807+0000","last_change":"2023-12-19T18:14:54.057328+0000","last_active":"2023-12-19T18:14:54.057807+0000","last_peered":"2023-12-19T18:14:54.057807+0000","last_clean":"2023-12-19T18:14:54.057807+0000","last_became_active":"2023-12-19T18:14:54.055611+0000","last_became_peered":"2023-12-19T18:14:54.055611+0000","last_unstale":"2023-12-19T18:14:54.057807+0000","last_undegraded":"2023-12-19T18:14:54.057807+0000","last_fullsized":"2023-12-19T18:14:54.057807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:12.180767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.043960+0000","last_change":"2023-12-19T18:14:53.562764+0000","last_active":"2023-12-19T18:14:54.043960+0000","last_peered":"2023-12-19T18:14:54.043960+0000","last_clean":"2023-12-19T18:14:54.043960+0000","last_became_active":"2023-12-19T18:14:53.562532+0000","last_became_peered":"2023-12-19T18:14:53.562532+0000","last_unstale":"2023-12-19T18:14:54.043960+0000","last_undegraded":"2023-12-19T18:14:54.043960+0000","last_fullsized":"2023-12-19T18:14:54.043960+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.069588+0000","last_change":"2023-12-19T18:14:54.068780+0000","last_active":"2023-12-19T18:14:54.069588+0000","last_peered":"2023-12-19T18:14:54.069588+0000","last_clean":"2023-12-19T18:14:54.069588+0000","last_became_active":"2023-12-19T18:14:54.068556+0000","last_became_peered":"2023-12-19T18:14:54.068556+0000","last_unstale":"2023-12-19T18:14:54.069588+0000","last_undegraded":"2023-12-19T18:14:54.069588+0000","last_fullsized":"2023-12-19T18:14:54.069588+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":19,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074393+0000","last_change":"2023-12-19T18:14:53.565084+0000","last_active":"2023-12-19T18:14:54.074393+0000","last_peered":"2023-12-19T18:14:54.074393+0000","last_clean":"2023-12-19T18:14:54.074393+0000","last_became_active":"2023-12-19T18:14:53.564836+0000","last_became_peered":"2023-12-19T18:14:53.564836+0000","last_unstale":"2023-12-19T18:14:54.074393+0000","last_undegraded":"2023-12-19T18:14:54.074393+0000","last_fullsized":"2023-12-19T18:14:54.074393+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:13:15.977501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.076309+0000","last_change":"2023-12-19T18:14:54.075967+0000","last_active":"2023-12-19T18:14:54.076309+0000","last_peered":"2023-12-19T18:14:54.076309+0000","last_clean":"2023-12-19T18:14:54.076309+0000","last_became_active":"2023-12-19T18:14:54.074991+0000","last_became_peered":"2023-12-19T18:14:54.074991+0000","last_unstale":"2023-12-19T18:14:54.076309+0000","last_undegraded":"2023-12-19T18:14:54.076309+0000","last_fullsized":"2023-12-19T18:14:54.076309+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:11.729223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061847+0000","last_change":"2023-12-19T18:14:53.550365+0000","last_active":"2023-12-19T18:14:54.061847+0000","last_peered":"2023-12-19T18:14:54.061847+0000","last_clean":"2023-12-19T18:14:54.061847+0000","last_became_active":"2023-12-19T18:14:53.550059+0000","last_became_peered":"2023-12-19T18:14:53.550059+0000","last_unstale":"2023-12-19T18:14:54.061847+0000","last_undegraded":"2023-12-19T18:14:54.061847+0000","last_fullsized":"2023-12-19T18:14:54.061847+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050074+0000","last_change":"2023-12-19T18:14:54.048927+0000","last_active":"2023-12-19T18:14:54.050074+0000","last_peered":"2023-12-19T18:14:54.050074+0000","last_clean":"2023-12-19T18:14:54.050074+0000","last_became_active":"2023-12-19T18:14:54.048515+0000","last_became_peered":"2023-12-19T18:14:54.048515+0000","last_unstale":"2023-12-19T18:14:54.050074+0000","last_undegraded":"2023-12-19T18:14:54.050074+0000","last_fullsized":"2023-12-19T18:14:54.050074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064254+0000","last_change":"2023-12-19T18:14:53.567656+0000","last_active":"2023-12-19T18:14:54.064254+0000","last_peered":"2023-12-19T18:14:54.064254+0000","last_clean":"2023-12-19T18:14:54.064254+0000","last_became_active":"2023-12-19T18:14:53.567278+0000","last_became_peered":"2023-12-19T18:14:53.567278+0000","last_unstale":"2023-12-19T18:14:54.064254+0000","last_undegraded":"2023-12-19T18:14:54.064254+0000","last_fullsized":"2023-12-19T18:14:54.064254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065685+0000","last_change":"2023-12-19T18:14:54.065628+0000","last_active":"2023-12-19T18:14:54.065685+0000","last_peered":"2023-12-19T18:14:54.065685+0000","last_clean":"2023-12-19T18:14:54.065685+0000","last_became_active":"2023-12-19T18:14:54.065414+0000","last_became_peered":"2023-12-19T18:14:54.065414+0000","last_unstale":"2023-12-19T18:14:54.065685+0000","last_undegraded":"2023-12-19T18:14:54.065685+0000","last_fullsized":"2023-12-19T18:14:54.065685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:18.713427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065227+0000","last_change":"2023-12-19T18:14:53.559407+0000","last_active":"2023-12-19T18:14:54.065227+0000","last_peered":"2023-12-19T18:14:54.065227+0000","last_clean":"2023-12-19T18:14:54.065227+0000","last_became_active":"2023-12-19T18:14:53.559062+0000","last_became_peered":"2023-12-19T18:14:53.559062+0000","last_unstale":"2023-12-19T18:14:54.065227+0000","last_undegraded":"2023-12-19T18:14:54.065227+0000","last_fullsized":"2023-12-19T18:14:54.065227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.061207+0000","last_change":"2023-12-19T18:14:54.061139+0000","last_active":"2023-12-19T18:14:54.061207+0000","last_peered":"2023-12-19T18:14:54.061207+0000","last_clean":"2023-12-19T18:14:54.061207+0000","last_became_active":"2023-12-19T18:14:54.060925+0000","last_became_peered":"2023-12-19T18:14:54.060925+0000","last_unstale":"2023-12-19T18:14:54.061207+0000","last_undegraded":"2023-12-19T18:14:54.061207+0000","last_fullsized":"2023-12-19T18:14:54.061207+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:51:59.123990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067789+0000","last_change":"2023-12-19T18:14:53.563513+0000","last_active":"2023-12-19T18:14:54.067789+0000","last_peered":"2023-12-19T18:14:54.067789+0000","last_clean":"2023-12-19T18:14:54.067789+0000","last_became_active":"2023-12-19T18:14:53.562860+0000","last_became_peered":"2023-12-19T18:14:53.562860+0000","last_unstale":"2023-12-19T18:14:54.067789+0000","last_undegraded":"2023-12-19T18:14:54.067789+0000","last_fullsized":"2023-12-19T18:14:54.067789+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:59:49.969450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.072181+0000","last_change":"2023-12-19T18:14:54.071273+0000","last_active":"2023-12-19T18:14:54.072181+0000","last_peered":"2023-12-19T18:14:54.072181+0000","last_clean":"2023-12-19T18:14:54.072181+0000","last_became_active":"2023-12-19T18:14:54.070908+0000","last_became_peered":"2023-12-19T18:14:54.070908+0000","last_unstale":"2023-12-19T18:14:54.072181+0000","last_undegraded":"2023-12-19T18:14:54.072181+0000","last_fullsized":"2023-12-19T18:14:54.072181+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:27:34.736465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.055663+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:14:54.055663+0000","last_peered":"2023-12-19T18:14:54.055663+0000","last_clean":"2023-12-19T18:14:54.055663+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:14:54.055663+0000","last_undegraded":"2023-12-19T18:14:54.055663+0000","last_fullsized":"2023-12-19T18:14:54.055663+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.053073+0000","last_change":"2023-12-19T18:14:53.557129+0000","last_active":"2023-12-19T18:14:54.053073+0000","last_peered":"2023-12-19T18:14:54.053073+0000","last_clean":"2023-12-19T18:14:54.053073+0000","last_became_active":"2023-12-19T18:14:53.556812+0000","last_became_peered":"2023-12-19T18:14:53.556812+0000","last_unstale":"2023-12-19T18:14:54.053073+0000","last_undegraded":"2023-12-19T18:14:54.053073+0000","last_fullsized":"2023-12-19T18:14:54.053073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:53.113355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.051508+0000","last_change":"2023-12-19T18:14:54.051423+0000","last_active":"2023-12-19T18:14:54.051508+0000","last_peered":"2023-12-19T18:14:54.051508+0000","last_clean":"2023-12-19T18:14:54.051508+0000","last_became_active":"2023-12-19T18:14:54.050816+0000","last_became_peered":"2023-12-19T18:14:54.050816+0000","last_unstale":"2023-12-19T18:14:54.051508+0000","last_undegraded":"2023-12-19T18:14:54.051508+0000","last_fullsized":"2023-12-19T18:14:54.051508+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:47.397945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068099+0000","last_change":"2023-12-19T18:14:53.558583+0000","last_active":"2023-12-19T18:14:54.068099+0000","last_peered":"2023-12-19T18:14:54.068099+0000","last_clean":"2023-12-19T18:14:54.068099+0000","last_became_active":"2023-12-19T18:14:53.558000+0000","last_became_peered":"2023-12-19T18:14:53.558000+0000","last_unstale":"2023-12-19T18:14:54.068099+0000","last_undegraded":"2023-12-19T18:14:54.068099+0000","last_fullsized":"2023-12-19T18:14:54.068099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:23:53.818952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068077+0000","last_change":"2023-12-19T18:14:54.067973+0000","last_active":"2023-12-19T18:14:54.068077+0000","last_peered":"2023-12-19T18:14:54.068077+0000","last_clean":"2023-12-19T18:14:54.068077+0000","last_became_active":"2023-12-19T18:14:54.065516+0000","last_became_peered":"2023-12-19T18:14:54.065516+0000","last_unstale":"2023-12-19T18:14:54.068077+0000","last_undegraded":"2023-12-19T18:14:54.068077+0000","last_fullsized":"2023-12-19T18:14:54.068077+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:57:01.271312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.058632+0000","last_change":"2023-12-19T18:14:53.563881+0000","last_active":"2023-12-19T18:14:54.058632+0000","last_peered":"2023-12-19T18:14:54.058632+0000","last_clean":"2023-12-19T18:14:54.058632+0000","last_became_active":"2023-12-19T18:14:53.563564+0000","last_became_peered":"2023-12-19T18:14:53.563564+0000","last_unstale":"2023-12-19T18:14:54.058632+0000","last_undegraded":"2023-12-19T18:14:54.058632+0000","last_fullsized":"2023-12-19T18:14:54.058632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060261+0000","last_change":"2023-12-19T18:14:54.060208+0000","last_active":"2023-12-19T18:14:54.060261+0000","last_peered":"2023-12-19T18:14:54.060261+0000","last_clean":"2023-12-19T18:14:54.060261+0000","last_became_active":"2023-12-19T18:14:54.059235+0000","last_became_peered":"2023-12-19T18:14:54.059235+0000","last_unstale":"2023-12-19T18:14:54.060261+0000","last_undegraded":"2023-12-19T18:14:54.060261+0000","last_fullsized":"2023-12-19T18:14:54.060261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:01.943407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065436+0000","last_change":"2023-12-19T18:14:53.564065+0000","last_active":"2023-12-19T18:14:54.065436+0000","last_peered":"2023-12-19T18:14:54.065436+0000","last_clean":"2023-12-19T18:14:54.065436+0000","last_became_active":"2023-12-19T18:14:53.563841+0000","last_became_peered":"2023-12-19T18:14:53.563841+0000","last_unstale":"2023-12-19T18:14:54.065436+0000","last_undegraded":"2023-12-19T18:14:54.065436+0000","last_fullsized":"2023-12-19T18:14:54.065436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:46.161691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.060843+0000","last_change":"2023-12-19T18:14:54.060734+0000","last_active":"2023-12-19T18:14:54.060843+0000","last_peered":"2023-12-19T18:14:54.060843+0000","last_clean":"2023-12-19T18:14:54.060843+0000","last_became_active":"2023-12-19T18:14:54.060328+0000","last_became_peered":"2023-12-19T18:14:54.060328+0000","last_unstale":"2023-12-19T18:14:54.060843+0000","last_undegraded":"2023-12-19T18:14:54.060843+0000","last_fullsized":"2023-12-19T18:14:54.060843+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:13.823415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.050368+0000","last_change":"2023-12-19T18:14:53.565164+0000","last_active":"2023-12-19T18:14:54.050368+0000","last_peered":"2023-12-19T18:14:54.050368+0000","last_clean":"2023-12-19T18:14:54.050368+0000","last_became_active":"2023-12-19T18:14:53.564538+0000","last_became_peered":"2023-12-19T18:14:53.564538+0000","last_unstale":"2023-12-19T18:14:54.050368+0000","last_undegraded":"2023-12-19T18:14:54.050368+0000","last_fullsized":"2023-12-19T18:14:54.050368+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:23.696073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.068274+0000","last_change":"2023-12-19T18:14:54.068080+0000","last_active":"2023-12-19T18:14:54.068274+0000","last_peered":"2023-12-19T18:14:54.068274+0000","last_clean":"2023-12-19T18:14:54.068274+0000","last_became_active":"2023-12-19T18:14:54.067528+0000","last_became_peered":"2023-12-19T18:14:54.067528+0000","last_unstale":"2023-12-19T18:14:54.068274+0000","last_undegraded":"2023-12-19T18:14:54.068274+0000","last_fullsized":"2023-12-19T18:14:54.068274+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:54.129126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067999+0000","last_change":"2023-12-19T18:14:53.562234+0000","last_active":"2023-12-19T18:14:54.067999+0000","last_peered":"2023-12-19T18:14:54.067999+0000","last_clean":"2023-12-19T18:14:54.067999+0000","last_became_active":"2023-12-19T18:14:53.561326+0000","last_became_peered":"2023-12-19T18:14:53.561326+0000","last_unstale":"2023-12-19T18:14:54.067999+0000","last_undegraded":"2023-12-19T18:14:54.067999+0000","last_fullsized":"2023-12-19T18:14:54.067999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074010+0000","last_change":"2023-12-19T18:14:54.073543+0000","last_active":"2023-12-19T18:14:54.074010+0000","last_peered":"2023-12-19T18:14:54.074010+0000","last_clean":"2023-12-19T18:14:54.074010+0000","last_became_active":"2023-12-19T18:14:54.073247+0000","last_became_peered":"2023-12-19T18:14:54.073247+0000","last_unstale":"2023-12-19T18:14:54.074010+0000","last_undegraded":"2023-12-19T18:14:54.074010+0000","last_fullsized":"2023-12-19T18:14:54.074010+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:04.631150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-19T18:14:55.050134+0000","last_change":"2023-12-19T18:14:53.551251+0000","last_active":"2023-12-19T18:14:55.050134+0000","last_peered":"2023-12-19T18:14:55.050134+0000","last_clean":"2023-12-19T18:14:55.050134+0000","last_became_active":"2023-12-19T18:14:53.550691+0000","last_became_peered":"2023-12-19T18:14:53.550691+0000","last_unstale":"2023-12-19T18:14:55.050134+0000","last_undegraded":"2023-12-19T18:14:55.050134+0000","last_fullsized":"2023-12-19T18:14:55.050134+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.071102+0000","last_change":"2023-12-19T18:14:54.070766+0000","last_active":"2023-12-19T18:14:54.071102+0000","last_peered":"2023-12-19T18:14:54.071102+0000","last_clean":"2023-12-19T18:14:54.071102+0000","last_became_active":"2023-12-19T18:14:54.069495+0000","last_became_peered":"2023-12-19T18:14:54.069495+0000","last_unstale":"2023-12-19T18:14:54.071102+0000","last_undegraded":"2023-12-19T18:14:54.071102+0000","last_fullsized":"2023-12-19T18:14:54.071102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:32:11.812301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.076793+0000","last_change":"2023-12-19T18:14:53.563561+0000","last_active":"2023-12-19T18:14:54.076793+0000","last_peered":"2023-12-19T18:14:54.076793+0000","last_clean":"2023-12-19T18:14:54.076793+0000","last_became_active":"2023-12-19T18:14:53.563126+0000","last_became_peered":"2023-12-19T18:14:53.563126+0000","last_unstale":"2023-12-19T18:14:54.076793+0000","last_undegraded":"2023-12-19T18:14:54.076793+0000","last_fullsized":"2023-12-19T18:14:54.076793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:16:29.945509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.059619+0000","last_change":"2023-12-19T18:14:54.059362+0000","last_active":"2023-12-19T18:14:54.059619+0000","last_peered":"2023-12-19T18:14:54.059619+0000","last_clean":"2023-12-19T18:14:54.059619+0000","last_became_active":"2023-12-19T18:14:54.059089+0000","last_became_peered":"2023-12-19T18:14:54.059089+0000","last_unstale":"2023-12-19T18:14:54.059619+0000","last_undegraded":"2023-12-19T18:14:54.059619+0000","last_fullsized":"2023-12-19T18:14:54.059619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.045320+0000","last_change":"2023-12-19T18:14:53.551856+0000","last_active":"2023-12-19T18:14:54.045320+0000","last_peered":"2023-12-19T18:14:54.045320+0000","last_clean":"2023-12-19T18:14:54.045320+0000","last_became_active":"2023-12-19T18:14:53.551105+0000","last_became_peered":"2023-12-19T18:14:53.551105+0000","last_unstale":"2023-12-19T18:14:54.045320+0000","last_undegraded":"2023-12-19T18:14:54.045320+0000","last_fullsized":"2023-12-19T18:14:54.045320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:45.192842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065908+0000","last_change":"2023-12-19T18:14:54.065539+0000","last_active":"2023-12-19T18:14:54.065908+0000","last_peered":"2023-12-19T18:14:54.065908+0000","last_clean":"2023-12-19T18:14:54.065908+0000","last_became_active":"2023-12-19T18:14:54.056448+0000","last_became_peered":"2023-12-19T18:14:54.056448+0000","last_unstale":"2023-12-19T18:14:54.065908+0000","last_undegraded":"2023-12-19T18:14:54.065908+0000","last_fullsized":"2023-12-19T18:14:54.065908+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:14.883403+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.045692+0000","last_change":"2023-12-19T18:14:53.554185+0000","last_active":"2023-12-19T18:14:54.045692+0000","last_peered":"2023-12-19T18:14:54.045692+0000","last_clean":"2023-12-19T18:14:54.045692+0000","last_became_active":"2023-12-19T18:14:53.553750+0000","last_became_peered":"2023-12-19T18:14:53.553750+0000","last_unstale":"2023-12-19T18:14:54.045692+0000","last_undegraded":"2023-12-19T18:14:54.045692+0000","last_fullsized":"2023-12-19T18:14:54.045692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.059756+0000","last_change":"2023-12-19T18:14:54.059501+0000","last_active":"2023-12-19T18:14:54.059756+0000","last_peered":"2023-12-19T18:14:54.059756+0000","last_clean":"2023-12-19T18:14:54.059756+0000","last_became_active":"2023-12-19T18:14:54.059055+0000","last_became_peered":"2023-12-19T18:14:54.059055+0000","last_unstale":"2023-12-19T18:14:54.059756+0000","last_undegraded":"2023-12-19T18:14:54.059756+0000","last_fullsized":"2023-12-19T18:14:54.059756+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:24.181167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.066496+0000","last_change":"2023-12-19T18:14:53.566640+0000","last_active":"2023-12-19T18:14:54.066496+0000","last_peered":"2023-12-19T18:14:54.066496+0000","last_clean":"2023-12-19T18:14:54.066496+0000","last_became_active":"2023-12-19T18:14:53.566413+0000","last_became_peered":"2023-12-19T18:14:53.566413+0000","last_unstale":"2023-12-19T18:14:54.066496+0000","last_undegraded":"2023-12-19T18:14:54.066496+0000","last_fullsized":"2023-12-19T18:14:54.066496+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:26.134698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074538+0000","last_change":"2023-12-19T18:14:54.073374+0000","last_active":"2023-12-19T18:14:54.074538+0000","last_peered":"2023-12-19T18:14:54.074538+0000","last_clean":"2023-12-19T18:14:54.074538+0000","last_became_active":"2023-12-19T18:14:54.072676+0000","last_became_peered":"2023-12-19T18:14:54.072676+0000","last_unstale":"2023-12-19T18:14:54.074538+0000","last_undegraded":"2023-12-19T18:14:54.074538+0000","last_fullsized":"2023-12-19T18:14:54.074538+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:33.484291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073825+0000","last_change":"2023-12-19T18:14:53.549245+0000","last_active":"2023-12-19T18:14:54.073825+0000","last_peered":"2023-12-19T18:14:54.073825+0000","last_clean":"2023-12-19T18:14:54.073825+0000","last_became_active":"2023-12-19T18:14:53.548992+0000","last_became_peered":"2023-12-19T18:14:53.548992+0000","last_unstale":"2023-12-19T18:14:54.073825+0000","last_undegraded":"2023-12-19T18:14:54.073825+0000","last_fullsized":"2023-12-19T18:14:54.073825+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:32.468520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.074558+0000","last_change":"2023-12-19T18:14:54.073488+0000","last_active":"2023-12-19T18:14:54.074558+0000","last_peered":"2023-12-19T18:14:54.074558+0000","last_clean":"2023-12-19T18:14:54.074558+0000","last_became_active":"2023-12-19T18:14:54.072567+0000","last_became_peered":"2023-12-19T18:14:54.072567+0000","last_unstale":"2023-12-19T18:14:54.074558+0000","last_undegraded":"2023-12-19T18:14:54.074558+0000","last_fullsized":"2023-12-19T18:14:54.074558+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:22.428929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.057975+0000","last_change":"2023-12-19T18:14:54.057549+0000","last_active":"2023-12-19T18:14:54.057975+0000","last_peered":"2023-12-19T18:14:54.057975+0000","last_clean":"2023-12-19T18:14:54.057975+0000","last_became_active":"2023-12-19T18:14:54.056075+0000","last_became_peered":"2023-12-19T18:14:54.056075+0000","last_unstale":"2023-12-19T18:14:54.057975+0000","last_undegraded":"2023-12-19T18:14:54.057975+0000","last_fullsized":"2023-12-19T18:14:54.057975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:47.825262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.064249+0000","last_change":"2023-12-19T18:14:53.555372+0000","last_active":"2023-12-19T18:14:54.064249+0000","last_peered":"2023-12-19T18:14:54.064249+0000","last_clean":"2023-12-19T18:14:54.064249+0000","last_became_active":"2023-12-19T18:14:53.555089+0000","last_became_peered":"2023-12-19T18:14:53.555089+0000","last_unstale":"2023-12-19T18:14:54.064249+0000","last_undegraded":"2023-12-19T18:14:54.064249+0000","last_fullsized":"2023-12-19T18:14:54.064249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.065405+0000","last_change":"2023-12-19T18:14:54.065119+0000","last_active":"2023-12-19T18:14:54.065405+0000","last_peered":"2023-12-19T18:14:54.065405+0000","last_clean":"2023-12-19T18:14:54.065405+0000","last_became_active":"2023-12-19T18:14:54.064697+0000","last_became_peered":"2023-12-19T18:14:54.064697+0000","last_unstale":"2023-12-19T18:14:54.065405+0000","last_undegraded":"2023-12-19T18:14:54.065405+0000","last_fullsized":"2023-12-19T18:14:54.065405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:47.177462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":13,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063594+0000","last_change":"2023-12-19T18:14:53.556488+0000","last_active":"2023-12-19T18:14:54.063594+0000","last_peered":"2023-12-19T18:14:54.063594+0000","last_clean":"2023-12-19T18:14:54.063594+0000","last_became_active":"2023-12-19T18:14:53.555952+0000","last_became_peered":"2023-12-19T18:14:53.555952+0000","last_unstale":"2023-12-19T18:14:54.063594+0000","last_undegraded":"2023-12-19T18:14:54.063594+0000","last_fullsized":"2023-12-19T18:14:54.063594+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:24.830880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.070434+0000","last_change":"2023-12-19T18:14:54.070014+0000","last_active":"2023-12-19T18:14:54.070434+0000","last_peered":"2023-12-19T18:14:54.070434+0000","last_clean":"2023-12-19T18:14:54.070434+0000","last_became_active":"2023-12-19T18:14:54.067648+0000","last_became_peered":"2023-12-19T18:14:54.067648+0000","last_unstale":"2023-12-19T18:14:54.070434+0000","last_undegraded":"2023-12-19T18:14:54.070434+0000","last_fullsized":"2023-12-19T18:14:54.070434+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:25.840099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":12,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.067116+0000","last_change":"2023-12-19T18:14:53.566090+0000","last_active":"2023-12-19T18:14:54.067116+0000","last_peered":"2023-12-19T18:14:54.067116+0000","last_clean":"2023-12-19T18:14:54.067116+0000","last_became_active":"2023-12-19T18:14:53.565782+0000","last_became_peered":"2023-12-19T18:14:53.565782+0000","last_unstale":"2023-12-19T18:14:54.067116+0000","last_undegraded":"2023-12-19T18:14:54.067116+0000","last_fullsized":"2023-12-19T18:14:54.067116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:37.329341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.063312+0000","last_change":"2023-12-19T18:14:54.060686+0000","last_active":"2023-12-19T18:14:54.063312+0000","last_peered":"2023-12-19T18:14:54.063312+0000","last_clean":"2023-12-19T18:14:54.063312+0000","last_became_active":"2023-12-19T18:14:54.059977+0000","last_became_peered":"2023-12-19T18:14:54.059977+0000","last_unstale":"2023-12-19T18:14:54.063312+0000","last_undegraded":"2023-12-19T18:14:54.063312+0000","last_fullsized":"2023-12-19T18:14:54.063312+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:53.516959+0000","last_clean_scrub_stamp":"2023-12-19T18:14:53.516959+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:18:20.426642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":15,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-19T18:14:54.073092+0000","last_change":"2023-12-19T18:14:53.558205+0000","last_active":"2023-12-19T18:14:54.073092+0000","last_peered":"2023-12-19T18:14:54.073092+0000","last_clean":"2023-12-19T18:14:54.073092+0000","last_became_active":"2023-12-19T18:14:53.557733+0000","last_became_peered":"2023-12-19T18:14:53.557733+0000","last_unstale":"2023-12-19T18:14:54.073092+0000","last_undegraded":"2023-12-19T18:14:54.073092+0000","last_fullsized":"2023-12-19T18:14:54.073092+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:52.501188+0000","last_clean_scrub_stamp":"2023-12-19T18:14:52.501188+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:39.724581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929138,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91400000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7210000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8180000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929262,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91500000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82699999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1459999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3180000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3620000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338802,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1890000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.006}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.343}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0780000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.272}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1479999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":71,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927026,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4119999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.157}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3779999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3480000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195242,"statfs":{"total":200000000,"available":199928680,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.147}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.339}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.47}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4239999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":70,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195241,"statfs":{"total":200000000,"available":199927698,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7610000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.2720000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.3109999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.3929999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":8.1839999999999993}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1059999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.1449999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":645,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194666,"statfs":{"total":200000000,"available":199338824,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.052}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2430000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.419}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.327}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":69,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195243,"statfs":{"total":200000000,"available":199929206,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1060000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1399999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.29}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2629999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5780000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9590000000000001}]}]}],"pool_statfs":[]}} 2023-12-19T18:15:10.482 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-19T18:15:10.482 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-19T18:15:10.482 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-12-19T18:15:10.482 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-12-19T18:15:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.755+0000 7f2821373640 1 Processor -- start 2023-12-19T18:15:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.756+0000 7f2821373640 1 -- start start 2023-12-19T18:15:10.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.757+0000 7f2821373640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f281c06fc10 0x7f281c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.757+0000 7f2821373640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281c06b520 con 0x7f281c070c00 2023-12-19T18:15:10.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.757+0000 7f2821373640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f281c15c1e0 con 0x7f281c15adf0 2023-12-19T18:15:10.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.757+0000 7f2821373640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281c0703c0 con 0x7f281c06fc10 2023-12-19T18:15:10.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.758+0000 7f281b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f281c070c00 0x7f281c0c91a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54094/0 (socket says 172.21.15.27:54094) 2023-12-19T18:15:10.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.758+0000 7f281b7fe640 1 -- 172.21.15.27:0/77900761 learned_addr learned my addr 172.21.15.27:0/77900761 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.758+0000 7f281a7fc640 1 --2- 172.21.15.27:0/77900761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f281c06fc10 0x7f281c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.758+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575258156 0 0) 0x7f281c06b520 con 0x7f281c070c00 2023-12-19T18:15:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.758+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2808003520 con 0x7f281c070c00 2023-12-19T18:15:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2064050842 0 0) 0x7f2808003520 con 0x7f281c070c00 2023-12-19T18:15:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f281c06fc10 msgr2=0x7f281c06d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f2819ffb640 1 --2- 172.21.15.27:0/77900761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f281c06fc10 0x7f281c06d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 >> v1:172.21.15.192:6789/0 conn(0x7f281c15adf0 legacy=0x7f281c06f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c15f3a0 con 0x7f281c070c00 2023-12-19T18:15:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f2814001670 con 0x7f281c070c00 2023-12-19T18:15:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 >> v1:172.21.15.27:6789/0 conn(0x7f281c070c00 legacy=0x7f281c0c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f2819ffb640 1 --2- 172.21.15.27:0/77900761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f281c06fc10 0x7f28080039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f2819ffb640 1 --2- 172.21.15.27:0/77900761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2808003ee0 0x7f28080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f2819ffb640 1 --2- 172.21.15.27:0/77900761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2808004d00 0x7f2808005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280800ba60 con 0x7f2808004d00 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280800bbd0 con 0x7f2808003ee0 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280800bd90 con 0x7f281c06fc10 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f281b7fe640 1 --2- 172.21.15.27:0/77900761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2808004d00 0x7f2808005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f281a7fc640 1 --2- 172.21.15.27:0/77900761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2808003ee0 0x7f28080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f281affd640 1 --2- 172.21.15.27:0/77900761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f281c06fc10 0x7f28080039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f281b7fe640 1 -- 172.21.15.27:0/77900761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f281c06fc10 msgr2=0x7f28080039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f281b7fe640 1 --2- 172.21.15.27:0/77900761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f281c06fc10 0x7f28080039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f281b7fe640 1 -- 172.21.15.27:0/77900761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2808003ee0 msgr2=0x7f28080047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.759+0000 7f281b7fe640 1 --2- 172.21.15.27:0/77900761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2808003ee0 0x7f28080047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.760+0000 7f281b7fe640 1 -- 172.21.15.27:0/77900761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2808003520 con 0x7f2808004d00 2023-12-19T18:15:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.760+0000 7f281a7fc640 1 --2- 172.21.15.27:0/77900761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2808003ee0 0x7f28080047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.761+0000 7f281b7fe640 1 --2- 172.21.15.27:0/77900761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2808004d00 0x7f2808005180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f281400c160 tx=0x7f28080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca2bf00b6c8ca5b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.761+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2814005ad0 con 0x7f2808004d00 2023-12-19T18:15:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.761+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2814005c30 con 0x7f2808004d00 2023-12-19T18:15:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.761+0000 7f2819ffb640 1 -- 172.21.15.27:0/77900761 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f281402c850 con 0x7f2808004d00 2023-12-19T18:15:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.761+0000 7f2821373640 1 -- 172.21.15.27:0/77900761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2808004d00 msgr2=0x7f2808005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.761+0000 7f2821373640 1 --2- 172.21.15.27:0/77900761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2808004d00 0x7f2808005180 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f281400c160 tx=0x7f28080057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.761+0000 7f281b7fe640 1 -- 172.21.15.27:0/77900761 reap_dead start 2023-12-19T18:15:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.762+0000 7f2821373640 1 -- 172.21.15.27:0/77900761 shutdown_connections 2023-12-19T18:15:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.762+0000 7f2821373640 1 -- 172.21.15.27:0/77900761 >> 172.21.15.27:0/77900761 conn(0x7f281c0ceb50 msgr2=0x7f281c0cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.762+0000 7f2821373640 1 -- 172.21.15.27:0/77900761 shutdown_connections 2023-12-19T18:15:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.762+0000 7f2821373640 1 -- 172.21.15.27:0/77900761 wait complete. 2023-12-19T18:15:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.763+0000 7f2821373640 1 Processor -- start 2023-12-19T18:15:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.764+0000 7f2821373640 1 -- start start 2023-12-19T18:15:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.764+0000 7f2821373640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2808004d00 0x7f281c1f3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f2821373640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f281c0c7470 0x7f281c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f2821373640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f281c0c78f0 0x7f281c0c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f2821373640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c0b2510 con 0x7f281c0c7470 2023-12-19T18:15:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f2821373640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f281c0c88e0 con 0x7f281c0c78f0 2023-12-19T18:15:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f2821373640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f281c0b1640 con 0x7f2808004d00 2023-12-19T18:15:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f281a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f281c0c7470 0x7f281c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f281a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f281c0c7470 0x7f281c0c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49840/0 (socket says 172.21.15.27:49840) 2023-12-19T18:15:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f281a7fc640 1 -- 172.21.15.27:0/3895954913 learned_addr learned my addr 172.21.15.27:0/3895954913 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f281b7fe640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f281c0c78f0 0x7f281c0c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f281affd640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2808004d00 0x7f281c1f3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f281a7fc640 1 -- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2808004d00 msgr2=0x7f281c1f3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f281a7fc640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2808004d00 0x7f281c1f3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f281a7fc640 1 -- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f281c0c78f0 msgr2=0x7f281c0c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f281a7fc640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f281c0c78f0 0x7f281c0c9900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.765+0000 7f281a7fc640 1 -- 172.21.15.27:0/3895954913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f281c1f3740 con 0x7f281c0c7470 2023-12-19T18:15:10.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.766+0000 7f281a7fc640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f281c0c7470 0x7f281c0c93c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2804010890 tx=0x7f2804010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:10.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.766+0000 7f28137fe640 1 -- 172.21.15.27:0/3895954913 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280400eb00 con 0x7f281c0c7470 2023-12-19T18:15:10.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.766+0000 7f28137fe640 1 -- 172.21.15.27:0/3895954913 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f280400ec60 con 0x7f281c0c7470 2023-12-19T18:15:10.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.766+0000 7f28137fe640 1 -- 172.21.15.27:0/3895954913 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f280401c400 con 0x7f281c0c7470 2023-12-19T18:15:10.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.766+0000 7f2821373640 1 -- 172.21.15.27:0/3895954913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f281c0b1890 con 0x7f281c0c7470 2023-12-19T18:15:10.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.766+0000 7f2821373640 1 -- 172.21.15.27:0/3895954913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f281c0b1e50 con 0x7f281c0c7470 2023-12-19T18:15:10.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.767+0000 7f2821373640 1 -- 172.21.15.27:0/3895954913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f281c1f0400 con 0x7f281c0c7470 2023-12-19T18:15:10.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.768+0000 7f28137fe640 1 -- 172.21.15.27:0/3895954913 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f280401c560 con 0x7f281c0c7470 2023-12-19T18:15:10.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.769+0000 7f28137fe640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f27fc06a460 0x7f27fc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:10.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.769+0000 7f28137fe640 1 -- 172.21.15.27:0/3895954913 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2804088df0 con 0x7f281c0c7470 2023-12-19T18:15:10.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.770+0000 7f281affd640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f27fc06a460 0x7f27fc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:10.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.771+0000 7f281affd640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f27fc06a460 0x7f27fc06c920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f280c0012e0 tx=0x7f280c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:10.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:10.773+0000 7f28137fe640 1 -- 172.21.15.27:0/3895954913 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2804054480 con 0x7f281c0c7470 2023-12-19T18:15:11.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.006+0000 7f2821373640 1 -- 172.21.15.27:0/3895954913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f281c068e90 con 0x7f281c0c7470 2023-12-19T18:15:11.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.006+0000 7f28137fe640 1 -- 172.21.15.27:0/3895954913 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f2804057aa0 con 0x7f281c0c7470 2023-12-19T18:15:11.007 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:11.007 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-19T18:15:11.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.009+0000 7f2821373640 1 -- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f27fc06a460 msgr2=0x7f27fc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.009+0000 7f2821373640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f27fc06a460 0x7f27fc06c920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f280c0012e0 tx=0x7f280c009040 comp rx=0 tx=0).stop 2023-12-19T18:15:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.009+0000 7f2821373640 1 -- 172.21.15.27:0/3895954913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f281c0c7470 msgr2=0x7f281c0c93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.009+0000 7f2821373640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f281c0c7470 0x7f281c0c93c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f2804010890 tx=0x7f2804010d50 comp rx=0 tx=0).stop 2023-12-19T18:15:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.009+0000 7f2821373640 1 -- 172.21.15.27:0/3895954913 shutdown_connections 2023-12-19T18:15:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.009+0000 7f2821373640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f27fc06a460 0x7f27fc06c920 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.009+0000 7f2821373640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f281c0c78f0 0x7f281c0c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.009+0000 7f2821373640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f281c0c7470 0x7f281c0c93c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.009+0000 7f2821373640 1 --2- 172.21.15.27:0/3895954913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2808004d00 0x7f281c1f3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.009+0000 7f2821373640 1 -- 172.21.15.27:0/3895954913 >> 172.21.15.27:0/3895954913 conn(0x7f281c0ceb50 msgr2=0x7f281c0af6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.009+0000 7f2821373640 1 -- 172.21.15.27:0/3895954913 shutdown_connections 2023-12-19T18:15:11.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.010+0000 7f2821373640 1 -- 172.21.15.27:0/3895954913 wait complete. 2023-12-19T18:15:11.028 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-12-19T18:15:11.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:11.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.263+0000 7f1c4b59e640 1 Processor -- start 2023-12-19T18:15:11.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.263+0000 7f1c4b59e640 1 -- start start 2023-12-19T18:15:11.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.265+0000 7f1c4b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c4c1553a0 0x7f1c4c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:11.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.265+0000 7f1c4b59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1c4c15ee30 0x7f1c4c0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:11.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.265+0000 7f1c4b59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c4c0b0330 con 0x7f1c4c0afeb0 2023-12-19T18:15:11.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.265+0000 7f1c4b59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c4c0cbcf0 con 0x7f1c4c15ee30 2023-12-19T18:15:11.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.265+0000 7f1c4b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0cb350 con 0x7f1c4c1553a0 2023-12-19T18:15:11.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.266+0000 7f1c4ad9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1c4c15ee30 0x7f1c4c0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:11.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.266+0000 7f1c4a59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c4c1553a0 0x7f1c4c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:11.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.266+0000 7f1c4ad9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1c4c15ee30 0x7f1c4c0c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:50792/0 (socket says 172.21.15.27:50792) 2023-12-19T18:15:11.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.266+0000 7f1c4ad9d640 1 -- 172.21.15.27:0/4195061994 learned_addr learned my addr 172.21.15.27:0/4195061994 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:11.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.267+0000 7f1c4ad9d640 1 -- 172.21.15.27:0/4195061994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c4c1553a0 msgr2=0x7f1c4c0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:11.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.267+0000 7f1c4ad9d640 1 --2- 172.21.15.27:0/4195061994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c4c1553a0 0x7f1c4c0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.267+0000 7f1c4ad9d640 1 -- 172.21.15.27:0/4195061994 >> v1:172.21.15.192:6789/0 conn(0x7f1c4c0afeb0 legacy=0x7f1c4c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.267+0000 7f1c4ad9d640 1 -- 172.21.15.27:0/4195061994 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4c0cb630 con 0x7f1c4c15ee30 2023-12-19T18:15:11.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.268+0000 7f1c4ad9d640 1 --2- 172.21.15.27:0/4195061994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c4c15ee30 0x7f1c4c0c39d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009870 tx=0x7f1c4402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c36333923e302c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:11.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.268+0000 7f1c4959a640 1 -- 172.21.15.27:0/4195061994 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c44031640 con 0x7f1c4c15ee30 2023-12-19T18:15:11.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.268+0000 7f1c4959a640 1 -- 172.21.15.27:0/4195061994 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c440317a0 con 0x7f1c4c15ee30 2023-12-19T18:15:11.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.268+0000 7f1c4959a640 1 -- 172.21.15.27:0/4195061994 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c44031b30 con 0x7f1c4c15ee30 2023-12-19T18:15:11.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.269+0000 7f1c4b59e640 1 -- 172.21.15.27:0/4195061994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c4c15ee30 msgr2=0x7f1c4c0c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.269+0000 7f1c4b59e640 1 --2- 172.21.15.27:0/4195061994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c4c15ee30 0x7f1c4c0c39d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1c44009870 tx=0x7f1c4402b8f0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.269+0000 7f1c4b59e640 1 -- 172.21.15.27:0/4195061994 shutdown_connections 2023-12-19T18:15:11.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.269+0000 7f1c4b59e640 1 --2- 172.21.15.27:0/4195061994 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c4c15ee30 0x7f1c4c0c39d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.269+0000 7f1c4b59e640 1 --2- 172.21.15.27:0/4195061994 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c4c1553a0 0x7f1c4c0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.269+0000 7f1c4b59e640 1 -- 172.21.15.27:0/4195061994 >> 172.21.15.27:0/4195061994 conn(0x7f1c4c069c70 msgr2=0x7f1c4c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:11.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.269+0000 7f1c4b59e640 1 -- 172.21.15.27:0/4195061994 shutdown_connections 2023-12-19T18:15:11.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.269+0000 7f1c4b59e640 1 -- 172.21.15.27:0/4195061994 wait complete. 2023-12-19T18:15:11.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.271+0000 7f1c4b59e640 1 Processor -- start 2023-12-19T18:15:11.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.271+0000 7f1c4b59e640 1 -- start start 2023-12-19T18:15:11.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.272+0000 7f1c4b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c4c0afeb0 0x7f1c4c15c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:11.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.272+0000 7f1c4b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c4c1553a0 0x7f1c4c0cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:11.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.272+0000 7f1c4b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c4c15ee30 0x7f1c4c141ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:11.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.272+0000 7f1c4b59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0b30e0 con 0x7f1c4c15ee30 2023-12-19T18:15:11.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.272+0000 7f1c4b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c0ce530 con 0x7f1c4c1553a0 2023-12-19T18:15:11.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.272+0000 7f1c4b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4c15d200 con 0x7f1c4c0afeb0 2023-12-19T18:15:11.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.272+0000 7f1c4ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c4c15ee30 0x7f1c4c141ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:11.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.272+0000 7f1c4ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c4c15ee30 0x7f1c4c141ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49844/0 (socket says 172.21.15.27:49844) 2023-12-19T18:15:11.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.272+0000 7f1c4ad9d640 1 -- 172.21.15.27:0/3301207828 learned_addr learned my addr 172.21.15.27:0/3301207828 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:11.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.273+0000 7f1c49d9b640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c4c1553a0 0x7f1c4c0cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:11.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.273+0000 7f1c4a59c640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c4c0afeb0 0x7f1c4c15c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:11.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.273+0000 7f1c4ad9d640 1 -- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c4c0afeb0 msgr2=0x7f1c4c15c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.273+0000 7f1c4ad9d640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c4c0afeb0 0x7f1c4c15c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.273+0000 7f1c4ad9d640 1 -- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c4c1553a0 msgr2=0x7f1c4c0cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.273+0000 7f1c4ad9d640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c4c1553a0 0x7f1c4c0cdff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.273+0000 7f1c4ad9d640 1 -- 172.21.15.27:0/3301207828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c440094d0 con 0x7f1c4c15ee30 2023-12-19T18:15:11.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.273+0000 7f1c4ad9d640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c4c15ee30 0x7f1c4c141ab0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1c440090b0 tx=0x7f1c440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:11.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.274+0000 7f1c42ffd640 1 -- 172.21.15.27:0/3301207828 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4403a930 con 0x7f1c4c15ee30 2023-12-19T18:15:11.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.274+0000 7f1c42ffd640 1 -- 172.21.15.27:0/3301207828 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c4403aad0 con 0x7f1c4c15ee30 2023-12-19T18:15:11.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.274+0000 7f1c42ffd640 1 -- 172.21.15.27:0/3301207828 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4403ae80 con 0x7f1c4c15ee30 2023-12-19T18:15:11.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.274+0000 7f1c4b59e640 1 -- 172.21.15.27:0/3301207828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4c15c960 con 0x7f1c4c15ee30 2023-12-19T18:15:11.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.274+0000 7f1c4b59e640 1 -- 172.21.15.27:0/3301207828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4c15cf20 con 0x7f1c4c15ee30 2023-12-19T18:15:11.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.275+0000 7f1c4b59e640 1 -- 172.21.15.27:0/3301207828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4c0b2eb0 con 0x7f1c4c15ee30 2023-12-19T18:15:11.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.276+0000 7f1c42ffd640 1 -- 172.21.15.27:0/3301207828 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1c4402f050 con 0x7f1c4c15ee30 2023-12-19T18:15:11.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.277+0000 7f1c42ffd640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1c2c06a430 0x7f1c2c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:11.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.277+0000 7f1c42ffd640 1 -- 172.21.15.27:0/3301207828 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1c440aa8e0 con 0x7f1c4c15ee30 2023-12-19T18:15:11.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.278+0000 7f1c4a59c640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1c2c06a430 0x7f1c2c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:11.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.278+0000 7f1c4a59c640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1c2c06a430 0x7f1c2c06c8f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0074a0 tx=0x7f1c3c00b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:11.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.281+0000 7f1c42ffd640 1 -- 172.21.15.27:0/3301207828 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1c4c0b2eb0 con 0x7f1c4c15ee30 2023-12-19T18:15:11.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.479+0000 7f1c4b59e640 1 -- 172.21.15.27:0/3301207828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1c4c0b2eb0 con 0x7f1c4c15ee30 2023-12-19T18:15:11.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.482+0000 7f1c42ffd640 1 -- 172.21.15.27:0/3301207828 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f1c4c0b2eb0 con 0x7f1c4c15ee30 2023-12-19T18:15:11.482 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:11.482 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:55.039690+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:11.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.484+0000 7f1c4b59e640 1 -- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1c2c06a430 msgr2=0x7f1c2c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.484+0000 7f1c4b59e640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1c2c06a430 0x7f1c2c06c8f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0074a0 tx=0x7f1c3c00b040 comp rx=0 tx=0).stop 2023-12-19T18:15:11.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.484+0000 7f1c4b59e640 1 -- 172.21.15.27:0/3301207828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c4c15ee30 msgr2=0x7f1c4c141ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.484+0000 7f1c4b59e640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c4c15ee30 0x7f1c4c141ab0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f1c440090b0 tx=0x7f1c440046c0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.484+0000 7f1c4b59e640 1 -- 172.21.15.27:0/3301207828 shutdown_connections 2023-12-19T18:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.484+0000 7f1c4b59e640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1c2c06a430 0x7f1c2c06c8f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.484+0000 7f1c4b59e640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c4c15ee30 0x7f1c4c141ab0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.484+0000 7f1c4b59e640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1c4c1553a0 0x7f1c4c0cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.484+0000 7f1c4b59e640 1 --2- 172.21.15.27:0/3301207828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1c4c0afeb0 0x7f1c4c15c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.484+0000 7f1c4b59e640 1 -- 172.21.15.27:0/3301207828 >> 172.21.15.27:0/3301207828 conn(0x7f1c4c069c70 msgr2=0x7f1c4c1546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.485+0000 7f1c4b59e640 1 -- 172.21.15.27:0/3301207828 shutdown_connections 2023-12-19T18:15:11.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.485+0000 7f1c4b59e640 1 -- 172.21.15.27:0/3301207828 wait complete. 2023-12-19T18:15:11.503 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:15:11.503 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:15:11.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.744+0000 7f76c8500640 1 Processor -- start 2023-12-19T18:15:11.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.744+0000 7f76c8500640 1 -- start start 2023-12-19T18:15:11.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.746+0000 7f76c8500640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76c015b1a0 0x7f76c006f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:11.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.746+0000 7f76c8500640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f76c00c2720 0x7f76c006fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:11.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.746+0000 7f76c8500640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c006bc80 con 0x7f76c0071420 2023-12-19T18:15:11.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.746+0000 7f76c8500640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c015fd90 con 0x7f76c00c2720 2023-12-19T18:15:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.746+0000 7f76c8500640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c006d690 con 0x7f76c015b1a0 2023-12-19T18:15:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.746+0000 7f76c5a9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f76c00c2720 0x7f76c006fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.746+0000 7f76c6a9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f76c0071420 0x7f76c00c9d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54102/0 (socket says 172.21.15.27:54102) 2023-12-19T18:15:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.746+0000 7f76c6a9d640 1 -- 172.21.15.27:0/3060475484 learned_addr learned my addr 172.21.15.27:0/3060475484 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.746+0000 7f76c629c640 1 --2- 172.21.15.27:0/3060475484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76c015b1a0 0x7f76c006f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:11.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.747+0000 7f76c5a9b640 1 -- 172.21.15.27:0/3060475484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76c015b1a0 msgr2=0x7f76c006f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.747+0000 7f76c5a9b640 1 --2- 172.21.15.27:0/3060475484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76c015b1a0 0x7f76c006f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.747+0000 7f76c5a9b640 1 -- 172.21.15.27:0/3060475484 >> v1:172.21.15.27:6789/0 conn(0x7f76c0071420 legacy=0x7f76c00c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.747+0000 7f76c5a9b640 1 -- 172.21.15.27:0/3060475484 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c006d0a0 con 0x7f76c00c2720 2023-12-19T18:15:11.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.747+0000 7f76c629c640 1 --2- 172.21.15.27:0/3060475484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76c015b1a0 0x7f76c006f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.748+0000 7f76c5a9b640 1 --2- 172.21.15.27:0/3060475484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c00c2720 0x7f76c006fa60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f76b00097a0 tx=0x7f76b002b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755d2bfe9a73b3b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.748+0000 7f76c529a640 1 -- 172.21.15.27:0/3060475484 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0031780 con 0x7f76c00c2720 2023-12-19T18:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.748+0000 7f76c529a640 1 -- 172.21.15.27:0/3060475484 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b00318e0 con 0x7f76c00c2720 2023-12-19T18:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.748+0000 7f76c529a640 1 -- 172.21.15.27:0/3060475484 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b0031c70 con 0x7f76c00c2720 2023-12-19T18:15:11.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.749+0000 7f76c8500640 1 -- 172.21.15.27:0/3060475484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c00c2720 msgr2=0x7f76c006fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.749+0000 7f76c8500640 1 --2- 172.21.15.27:0/3060475484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c00c2720 0x7f76c006fa60 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f76b00097a0 tx=0x7f76b002b820 comp rx=0 tx=0).stop 2023-12-19T18:15:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.749+0000 7f76c8500640 1 -- 172.21.15.27:0/3060475484 shutdown_connections 2023-12-19T18:15:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.749+0000 7f76c8500640 1 --2- 172.21.15.27:0/3060475484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c00c2720 0x7f76c006fa60 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.749+0000 7f76c8500640 1 --2- 172.21.15.27:0/3060475484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76c015b1a0 0x7f76c006f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.749+0000 7f76c8500640 1 -- 172.21.15.27:0/3060475484 >> 172.21.15.27:0/3060475484 conn(0x7f76c00cebd0 msgr2=0x7f76c00cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:11.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.749+0000 7f76c8500640 1 -- 172.21.15.27:0/3060475484 shutdown_connections 2023-12-19T18:15:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.749+0000 7f76c8500640 1 -- 172.21.15.27:0/3060475484 wait complete. 2023-12-19T18:15:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.751+0000 7f76c8500640 1 Processor -- start 2023-12-19T18:15:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.751+0000 7f76c8500640 1 -- start start 2023-12-19T18:15:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.752+0000 7f76c8500640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76c0071420 0x7f76c00c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.752+0000 7f76c8500640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76c00c2720 0x7f76c00c78d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:11.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.752+0000 7f76c8500640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c015b1a0 0x7f76c00c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:11.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.752+0000 7f76c8500640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c01f0830 con 0x7f76c015b1a0 2023-12-19T18:15:11.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.752+0000 7f76c8500640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c0152190 con 0x7f76c0071420 2023-12-19T18:15:11.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.752+0000 7f76c8500640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c00c8420 con 0x7f76c00c2720 2023-12-19T18:15:11.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.752+0000 7f76c6a9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c015b1a0 0x7f76c00c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:11.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.752+0000 7f76c6a9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c015b1a0 0x7f76c00c7e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49866/0 (socket says 172.21.15.27:49866) 2023-12-19T18:15:11.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.752+0000 7f76c6a9d640 1 -- 172.21.15.27:0/3964602107 learned_addr learned my addr 172.21.15.27:0/3964602107 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:11.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.752+0000 7f76c5a9b640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76c00c2720 0x7f76c00c78d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:11.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.752+0000 7f76c629c640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76c0071420 0x7f76c00c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:11.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.753+0000 7f76c6a9d640 1 -- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76c00c2720 msgr2=0x7f76c00c78d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.753+0000 7f76c6a9d640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76c00c2720 0x7f76c00c78d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.753+0000 7f76c6a9d640 1 -- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76c0071420 msgr2=0x7f76c00c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.753+0000 7f76c6a9d640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76c0071420 0x7f76c00c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.753+0000 7f76c6a9d640 1 -- 172.21.15.27:0/3964602107 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b0009400 con 0x7f76c015b1a0 2023-12-19T18:15:11.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.753+0000 7f76c5a9b640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76c00c2720 0x7f76c00c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:11.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.753+0000 7f76c629c640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76c0071420 0x7f76c00c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:11.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.753+0000 7f76c6a9d640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c015b1a0 0x7f76c00c7e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f76b800ca80 tx=0x7f76b800cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:11.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.753+0000 7f76b6ffd640 1 -- 172.21.15.27:0/3964602107 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b800ac30 con 0x7f76c015b1a0 2023-12-19T18:15:11.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.753+0000 7f76b6ffd640 1 -- 172.21.15.27:0/3964602107 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b800ad90 con 0x7f76c015b1a0 2023-12-19T18:15:11.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.753+0000 7f76b6ffd640 1 -- 172.21.15.27:0/3964602107 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b8012600 con 0x7f76c015b1a0 2023-12-19T18:15:11.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.753+0000 7f76c8500640 1 -- 172.21.15.27:0/3964602107 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c00c8700 con 0x7f76c015b1a0 2023-12-19T18:15:11.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.753+0000 7f76c8500640 1 -- 172.21.15.27:0/3964602107 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c00c8cc0 con 0x7f76c015b1a0 2023-12-19T18:15:11.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.754+0000 7f76c8500640 1 -- 172.21.15.27:0/3964602107 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c0068e90 con 0x7f76c015b1a0 2023-12-19T18:15:11.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.755+0000 7f76b6ffd640 1 -- 172.21.15.27:0/3964602107 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f76b8007070 con 0x7f76c015b1a0 2023-12-19T18:15:11.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.756+0000 7f76b6ffd640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f76a406a430 0x7f76a406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:11.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.756+0000 7f76b6ffd640 1 -- 172.21.15.27:0/3964602107 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f76b8088640 con 0x7f76c015b1a0 2023-12-19T18:15:11.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.757+0000 7f76c629c640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f76a406a430 0x7f76a406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:11.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.758+0000 7f76c629c640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f76a406a430 0x7f76a406c8f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f76bc006fd0 tx=0x7f76bc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:11.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.761+0000 7f76b6ffd640 1 -- 172.21.15.27:0/3964602107 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f76b8053c40 con 0x7f76c015b1a0 2023-12-19T18:15:11.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.957+0000 7f76c8500640 1 -- 172.21.15.27:0/3964602107 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76c006bc80 con 0x7f76c015b1a0 2023-12-19T18:15:11.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.958+0000 7f76b6ffd640 1 -- 172.21.15.27:0/3964602107 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76c006bc80 con 0x7f76c015b1a0 2023-12-19T18:15:11.958 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:15:11.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.960+0000 7f76c8500640 1 -- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f76a406a430 msgr2=0x7f76a406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.960+0000 7f76c8500640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f76a406a430 0x7f76a406c8f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f76bc006fd0 tx=0x7f76bc008040 comp rx=0 tx=0).stop 2023-12-19T18:15:11.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.960+0000 7f76c8500640 1 -- 172.21.15.27:0/3964602107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c015b1a0 msgr2=0x7f76c00c7e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:11.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.960+0000 7f76c8500640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c015b1a0 0x7f76c00c7e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f76b800ca80 tx=0x7f76b800cf40 comp rx=0 tx=0).stop 2023-12-19T18:15:11.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.961+0000 7f76c8500640 1 -- 172.21.15.27:0/3964602107 shutdown_connections 2023-12-19T18:15:11.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.961+0000 7f76c8500640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f76a406a430 0x7f76a406c8f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.961+0000 7f76c8500640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76c015b1a0 0x7f76c00c7e10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.961+0000 7f76c8500640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76c00c2720 0x7f76c00c78d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.961+0000 7f76c8500640 1 --2- 172.21.15.27:0/3964602107 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76c0071420 0x7f76c00c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:11.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.961+0000 7f76c8500640 1 -- 172.21.15.27:0/3964602107 >> 172.21.15.27:0/3964602107 conn(0x7f76c00cebd0 msgr2=0x7f76c00c2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:11.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.961+0000 7f76c8500640 1 -- 172.21.15.27:0/3964602107 shutdown_connections 2023-12-19T18:15:11.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:11.961+0000 7f76c8500640 1 -- 172.21.15.27:0/3964602107 wait complete. 2023-12-19T18:15:11.979 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:15:12.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.202+0000 7f6479477640 1 Processor -- start 2023-12-19T18:15:12.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.202+0000 7f6479477640 1 -- start start 2023-12-19T18:15:12.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.204+0000 7f6479477640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64740c9750 0x7f647415e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:12.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.204+0000 7f6479477640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64740ce0b0 con 0x7f64740ccfa0 2023-12-19T18:15:12.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.204+0000 7f6479477640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64740ce920 con 0x7f64740ac9d0 2023-12-19T18:15:12.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.204+0000 7f6479477640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647415e9e0 con 0x7f64740c9750 2023-12-19T18:15:12.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.205+0000 7f6472ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64740c9750 0x7f647415e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:12.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.205+0000 7f646bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f64740ccfa0 0x7f64740c9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49284/0 (socket says 172.21.15.27:49284) 2023-12-19T18:15:12.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.205+0000 7f646bfff640 1 -- 172.21.15.27:0/4251077222 learned_addr learned my addr 172.21.15.27:0/4251077222 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:12.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.205+0000 7f6472ffd640 1 -- 172.21.15.27:0/4251077222 >> v1:172.21.15.192:6789/0 conn(0x7f64740ac9d0 legacy=0x7f64740ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:12.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.205+0000 7f6472ffd640 1 -- 172.21.15.27:0/4251077222 >> v1:172.21.15.27:6789/0 conn(0x7f64740ccfa0 legacy=0x7f64740c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:12.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.205+0000 7f64727fc640 1 -- 172.21.15.27:0/4251077222 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569727240 0 0) 0x7f64740ce0b0 con 0x7f64740ccfa0 2023-12-19T18:15:12.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.205+0000 7f6472ffd640 1 -- 172.21.15.27:0/4251077222 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64740cd4a0 con 0x7f64740c9750 2023-12-19T18:15:12.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.206+0000 7f6472ffd640 1 --2- 172.21.15.27:0/4251077222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64740c9750 0x7f647415e4a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f64640097a0 tx=0x7f646402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c629bce523bd24f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:12.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.207+0000 7f64727fc640 1 -- 172.21.15.27:0/4251077222 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464030610 con 0x7f64740c9750 2023-12-19T18:15:12.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.207+0000 7f64727fc640 1 -- 172.21.15.27:0/4251077222 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6464030770 con 0x7f64740c9750 2023-12-19T18:15:12.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.207+0000 7f64727fc640 1 -- 172.21.15.27:0/4251077222 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6464030b00 con 0x7f64740c9750 2023-12-19T18:15:12.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.207+0000 7f6479477640 1 -- 172.21.15.27:0/4251077222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64740c9750 msgr2=0x7f647415e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:12.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.207+0000 7f6479477640 1 --2- 172.21.15.27:0/4251077222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64740c9750 0x7f647415e4a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f64640097a0 tx=0x7f646402b960 comp rx=0 tx=0).stop 2023-12-19T18:15:12.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.207+0000 7f6479477640 1 -- 172.21.15.27:0/4251077222 shutdown_connections 2023-12-19T18:15:12.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.208+0000 7f6479477640 1 --2- 172.21.15.27:0/4251077222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64740c9750 0x7f647415e4a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.208+0000 7f6479477640 1 -- 172.21.15.27:0/4251077222 >> 172.21.15.27:0/4251077222 conn(0x7f64740cebd0 msgr2=0x7f64740cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:12.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.208+0000 7f6479477640 1 -- 172.21.15.27:0/4251077222 shutdown_connections 2023-12-19T18:15:12.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.208+0000 7f6479477640 1 -- 172.21.15.27:0/4251077222 wait complete. 2023-12-19T18:15:12.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.209+0000 7f6479477640 1 Processor -- start 2023-12-19T18:15:12.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.210+0000 7f6479477640 1 -- start start 2023-12-19T18:15:12.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f6479477640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64740ac9d0 0x7f64740c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f6479477640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64740c9750 0x7f647406cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f6479477640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64740ccfa0 0x7f647406d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f6479477640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64741f2a90 con 0x7f64740c9750 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f6479477640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64740c5360 con 0x7f64740ccfa0 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f6479477640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64741f2d10 con 0x7f64740ac9d0 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f646bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64740c9750 0x7f647406cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f646bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64740c9750 0x7f647406cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60824/0 (socket says 172.21.15.27:60824) 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f646bfff640 1 -- 172.21.15.27:0/3588868159 learned_addr learned my addr 172.21.15.27:0/3588868159 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f6472ffd640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64740ac9d0 0x7f64740c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f64737fe640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64740ccfa0 0x7f647406d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f646bfff640 1 -- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64740ac9d0 msgr2=0x7f64740c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f646bfff640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64740ac9d0 0x7f64740c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f646bfff640 1 -- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64740ccfa0 msgr2=0x7f647406d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:12.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f646bfff640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64740ccfa0 0x7f647406d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f646bfff640 1 -- 172.21.15.27:0/3588868159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6464009400 con 0x7f64740c9750 2023-12-19T18:15:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.211+0000 7f64737fe640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64740ccfa0 0x7f647406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.212+0000 7f646bfff640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64740c9750 0x7f647406cc70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f645c00c3c0 tx=0x7f645c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:12.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.212+0000 7f646b7fe640 1 -- 172.21.15.27:0/3588868159 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c0074e0 con 0x7f64740c9750 2023-12-19T18:15:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.212+0000 7f646b7fe640 1 -- 172.21.15.27:0/3588868159 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f645c007640 con 0x7f64740c9750 2023-12-19T18:15:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.212+0000 7f6479477640 1 -- 172.21.15.27:0/3588868159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64741f2ff0 con 0x7f64740c9750 2023-12-19T18:15:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.212+0000 7f646b7fe640 1 -- 172.21.15.27:0/3588868159 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645c004a20 con 0x7f64740c9750 2023-12-19T18:15:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.212+0000 7f6479477640 1 -- 172.21.15.27:0/3588868159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64740c4fe0 con 0x7f64740c9750 2023-12-19T18:15:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.213+0000 7f6479477640 1 -- 172.21.15.27:0/3588868159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64741f3510 con 0x7f64740c9750 2023-12-19T18:15:12.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.214+0000 7f646b7fe640 1 -- 172.21.15.27:0/3588868159 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f645c004b80 con 0x7f64740c9750 2023-12-19T18:15:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.215+0000 7f646b7fe640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f645406a430 0x7f645406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:12.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.215+0000 7f646b7fe640 1 -- 172.21.15.27:0/3588868159 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f645c0887a0 con 0x7f64740c9750 2023-12-19T18:15:12.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.216+0000 7f6472ffd640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f645406a430 0x7f645406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:12.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.217+0000 7f6472ffd640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f645406a430 0x7f645406c8f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6464036000 tx=0x7f6464008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:12.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.220+0000 7f646b7fe640 1 -- 172.21.15.27:0/3588868159 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f64741f3510 con 0x7f64740c9750 2023-12-19T18:15:12.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.413+0000 7f6479477640 1 -- 172.21.15.27:0/3588868159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f64741f3510 con 0x7f64740c9750 2023-12-19T18:15:12.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.414+0000 7f646b7fe640 1 -- 172.21.15.27:0/3588868159 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f64741f3510 con 0x7f64740c9750 2023-12-19T18:15:12.414 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:15:12.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.418+0000 7f6479477640 1 -- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f645406a430 msgr2=0x7f645406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:12.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.418+0000 7f6479477640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f645406a430 0x7f645406c8f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f6464036000 tx=0x7f6464008030 comp rx=0 tx=0).stop 2023-12-19T18:15:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.418+0000 7f6479477640 1 -- 172.21.15.27:0/3588868159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64740c9750 msgr2=0x7f647406cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.418+0000 7f6479477640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64740c9750 0x7f647406cc70 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f645c00c3c0 tx=0x7f645c00c880 comp rx=0 tx=0).stop 2023-12-19T18:15:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.418+0000 7f6479477640 1 -- 172.21.15.27:0/3588868159 shutdown_connections 2023-12-19T18:15:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.418+0000 7f6479477640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f645406a430 0x7f645406c8f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.418+0000 7f6479477640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f64740ccfa0 0x7f647406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.418+0000 7f6479477640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64740c9750 0x7f647406cc70 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.418+0000 7f6479477640 1 --2- 172.21.15.27:0/3588868159 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64740ac9d0 0x7f64740c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.418+0000 7f6479477640 1 -- 172.21.15.27:0/3588868159 >> 172.21.15.27:0/3588868159 conn(0x7f64740cebd0 msgr2=0x7f64740cf1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:12.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.419+0000 7f6479477640 1 -- 172.21.15.27:0/3588868159 shutdown_connections 2023-12-19T18:15:12.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.419+0000 7f6479477640 1 -- 172.21.15.27:0/3588868159 wait complete. 2023-12-19T18:15:12.440 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:15:12.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.678+0000 7fabdafdc640 1 Processor -- start 2023-12-19T18:15:12.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.678+0000 7fabdafdc640 1 -- start start 2023-12-19T18:15:12.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.680+0000 7fabdafdc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fabd415ae80 0x7fabd40cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:12.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.680+0000 7fabdafdc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fabd40b2810 0x7fabd40b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:12.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.680+0000 7fabdafdc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd40cd6b0 con 0x7fabd40b31b0 2023-12-19T18:15:12.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.680+0000 7fabdafdc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd40cdc70 con 0x7fabd415ae80 2023-12-19T18:15:12.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.680+0000 7fabdafdc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd40b3600 con 0x7fabd40b2810 2023-12-19T18:15:12.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.680+0000 7fabd8d78640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fabd415ae80 0x7fabd40cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:12.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.680+0000 7fabd8d78640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fabd415ae80 0x7fabd40cb3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60838/0 (socket says 172.21.15.27:60838) 2023-12-19T18:15:12.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.680+0000 7fabd8d78640 1 -- 172.21.15.27:0/2369935091 learned_addr learned my addr 172.21.15.27:0/2369935091 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:12.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.681+0000 7fabcbfff640 1 --2- 172.21.15.27:0/2369935091 >> v2:172.21.15.192:3300/0 conn(0x7fabd40b2810 0x7fabd40b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:12.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.681+0000 7fabd8d78640 1 -- 172.21.15.27:0/2369935091 >> v2:172.21.15.192:3300/0 conn(0x7fabd40b2810 msgr2=0x7fabd40b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:12.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.681+0000 7fabd8d78640 1 --2- 172.21.15.27:0/2369935091 >> v2:172.21.15.192:3300/0 conn(0x7fabd40b2810 0x7fabd40b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.681+0000 7fabd8d78640 1 -- 172.21.15.27:0/2369935091 >> v1:172.21.15.192:6789/0 conn(0x7fabd40b31b0 legacy=0x7fabd40ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:12.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.681+0000 7fabd8d78640 1 -- 172.21.15.27:0/2369935091 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd40d0450 con 0x7fabd415ae80 2023-12-19T18:15:12.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.682+0000 7fabd8d78640 1 --2- 172.21.15.27:0/2369935091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabd415ae80 0x7fabd40cb3e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fabd4070160 tx=0x7fabd002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f527d8e0418faf2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:12.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.682+0000 7fabcb7fe640 1 -- 172.21.15.27:0/2369935091 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd002ca80 con 0x7fabd415ae80 2023-12-19T18:15:12.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.682+0000 7fabcb7fe640 1 -- 172.21.15.27:0/2369935091 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd002cbe0 con 0x7fabd415ae80 2023-12-19T18:15:12.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.682+0000 7fabcb7fe640 1 -- 172.21.15.27:0/2369935091 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd00343f0 con 0x7fabd415ae80 2023-12-19T18:15:12.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.683+0000 7fabdafdc640 1 -- 172.21.15.27:0/2369935091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabd415ae80 msgr2=0x7fabd40cb3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:12.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.683+0000 7fabdafdc640 1 --2- 172.21.15.27:0/2369935091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabd415ae80 0x7fabd40cb3e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fabd4070160 tx=0x7fabd002bee0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.683+0000 7fabdafdc640 1 -- 172.21.15.27:0/2369935091 shutdown_connections 2023-12-19T18:15:12.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.683+0000 7fabdafdc640 1 --2- 172.21.15.27:0/2369935091 >> v2:172.21.15.192:3300/0 conn(0x7fabd40b2810 0x7fabd40b2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.683+0000 7fabdafdc640 1 --2- 172.21.15.27:0/2369935091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabd415ae80 0x7fabd40cb3e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.683+0000 7fabdafdc640 1 -- 172.21.15.27:0/2369935091 >> 172.21.15.27:0/2369935091 conn(0x7fabd406f1f0 msgr2=0x7fabd406f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:12.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.683+0000 7fabdafdc640 1 -- 172.21.15.27:0/2369935091 shutdown_connections 2023-12-19T18:15:12.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.683+0000 7fabdafdc640 1 -- 172.21.15.27:0/2369935091 wait complete. 2023-12-19T18:15:12.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.686+0000 7fabdafdc640 1 Processor -- start 2023-12-19T18:15:12.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.686+0000 7fabdafdc640 1 -- start start 2023-12-19T18:15:12.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.687+0000 7fabdafdc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd40b2810 0x7fabd4158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:12.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.687+0000 7fabdafdc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabd40b31b0 0x7fabd40c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:12.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.687+0000 7fabdafdc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabd415ae80 0x7fabd406c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:12.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.687+0000 7fabdafdc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd41593a0 con 0x7fabd40b31b0 2023-12-19T18:15:12.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.687+0000 7fabdafdc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd41e4a00 con 0x7fabd415ae80 2023-12-19T18:15:12.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.687+0000 7fabdafdc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd41e4ce0 con 0x7fabd40b2810 2023-12-19T18:15:12.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.688+0000 7fabcbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabd40b31b0 0x7fabd40c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:12.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.688+0000 7fabcbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabd40b31b0 0x7fabd40c68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60840/0 (socket says 172.21.15.27:60840) 2023-12-19T18:15:12.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.688+0000 7fabcbfff640 1 -- 172.21.15.27:0/456755594 learned_addr learned my addr 172.21.15.27:0/456755594 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.688+0000 7fabd8d78640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd40b2810 0x7fabd4158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.688+0000 7fabd9579640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabd415ae80 0x7fabd406c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.688+0000 7fabcbfff640 1 -- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd40b2810 msgr2=0x7fabd4158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.688+0000 7fabcbfff640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd40b2810 0x7fabd4158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.688+0000 7fabcbfff640 1 -- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabd415ae80 msgr2=0x7fabd406c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:12.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.688+0000 7fabcbfff640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabd415ae80 0x7fabd406c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.688+0000 7fabcbfff640 1 -- 172.21.15.27:0/456755594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd0009b30 con 0x7fabd40b31b0 2023-12-19T18:15:12.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.688+0000 7fabd9579640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabd415ae80 0x7fabd406c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:12.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.689+0000 7fabcbfff640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabd40b31b0 0x7fabd40c68b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fabc400d800 tx=0x7fabc400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:12.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.689+0000 7fabc97fa640 1 -- 172.21.15.27:0/456755594 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc400b6c0 con 0x7fabd40b31b0 2023-12-19T18:15:12.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.689+0000 7fabc97fa640 1 -- 172.21.15.27:0/456755594 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc400b820 con 0x7fabd40b31b0 2023-12-19T18:15:12.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.689+0000 7fabc97fa640 1 -- 172.21.15.27:0/456755594 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc400bbb0 con 0x7fabd40b31b0 2023-12-19T18:15:12.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.689+0000 7fabdafdc640 1 -- 172.21.15.27:0/456755594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd41e4fc0 con 0x7fabd40b31b0 2023-12-19T18:15:12.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.689+0000 7fabdafdc640 1 -- 172.21.15.27:0/456755594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd41e5580 con 0x7fabd40b31b0 2023-12-19T18:15:12.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.690+0000 7fabdafdc640 1 -- 172.21.15.27:0/456755594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd4067110 con 0x7fabd40b31b0 2023-12-19T18:15:12.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.691+0000 7fabc97fa640 1 -- 172.21.15.27:0/456755594 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fabc400bd10 con 0x7fabd40b31b0 2023-12-19T18:15:12.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.692+0000 7fabc97fa640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fabc006a430 0x7fabc006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:12.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.692+0000 7fabc97fa640 1 -- 172.21.15.27:0/456755594 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fabc4088e70 con 0x7fabd40b31b0 2023-12-19T18:15:12.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.693+0000 7fabd8d78640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fabc006a430 0x7fabc006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:12.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.694+0000 7fabd8d78640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fabc006a430 0x7fabc006c8f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fabd0035000 tx=0x7fabd0030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:12.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.697+0000 7fabc97fa640 1 -- 172.21.15.27:0/456755594 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fabc4054450 con 0x7fabd40b31b0 2023-12-19T18:15:12.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.889+0000 7fabdafdc640 1 -- 172.21.15.27:0/456755594 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fabd40b5320 con 0x7fabd40b31b0 2023-12-19T18:15:12.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.890+0000 7fabc97fa640 1 -- 172.21.15.27:0/456755594 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fabc4057a70 con 0x7fabd40b31b0 2023-12-19T18:15:12.890 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:15:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.893+0000 7fabdafdc640 1 -- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fabc006a430 msgr2=0x7fabc006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.893+0000 7fabdafdc640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fabc006a430 0x7fabc006c8f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fabd0035000 tx=0x7fabd0030000 comp rx=0 tx=0).stop 2023-12-19T18:15:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.893+0000 7fabdafdc640 1 -- 172.21.15.27:0/456755594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabd40b31b0 msgr2=0x7fabd40c68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.893+0000 7fabdafdc640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabd40b31b0 0x7fabd40c68b0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fabc400d800 tx=0x7fabc400dcc0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.893+0000 7fabdafdc640 1 -- 172.21.15.27:0/456755594 shutdown_connections 2023-12-19T18:15:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.893+0000 7fabdafdc640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fabc006a430 0x7fabc006c8f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.893+0000 7fabdafdc640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabd415ae80 0x7fabd406c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.893+0000 7fabdafdc640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabd40b31b0 0x7fabd40c68b0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.893+0000 7fabdafdc640 1 --2- 172.21.15.27:0/456755594 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabd40b2810 0x7fabd4158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.893+0000 7fabdafdc640 1 -- 172.21.15.27:0/456755594 >> 172.21.15.27:0/456755594 conn(0x7fabd406f1f0 msgr2=0x7fabd40b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.893+0000 7fabdafdc640 1 -- 172.21.15.27:0/456755594 shutdown_connections 2023-12-19T18:15:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:12.893+0000 7fabdafdc640 1 -- 172.21.15.27:0/456755594 wait complete. 2023-12-19T18:15:12.912 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:15:13.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.146+0000 7fc60580e640 1 Processor -- start 2023-12-19T18:15:13.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.146+0000 7fc60580e640 1 -- start start 2023-12-19T18:15:13.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.148+0000 7fc60580e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc6001604d0 0x7fc6000b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:13.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.148+0000 7fc60580e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6000d2350 0x7fc6000b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:13.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.148+0000 7fc60580e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6000b5e90 con 0x7fc6000d0300 2023-12-19T18:15:13.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.148+0000 7fc60580e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6000d2b30 con 0x7fc6001604d0 2023-12-19T18:15:13.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.148+0000 7fc60580e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6000d2850 con 0x7fc6000d2350 2023-12-19T18:15:13.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.148+0000 7fc60480c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc6001604d0 0x7fc6000b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:13.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.148+0000 7fc5fffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6000d2350 0x7fc6000b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:13.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.148+0000 7fc60480c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc6001604d0 0x7fc6000b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:43820/0 (socket says 172.21.15.27:43820) 2023-12-19T18:15:13.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.149+0000 7fc60480c640 1 -- 172.21.15.27:0/2367419386 learned_addr learned my addr 172.21.15.27:0/2367419386 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:13.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.149+0000 7fc5fffff640 1 -- 172.21.15.27:0/2367419386 >> v2:172.21.15.192:3300/0 conn(0x7fc6001604d0 msgr2=0x7fc6000b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:13.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.149+0000 7fc5fffff640 1 --2- 172.21.15.27:0/2367419386 >> v2:172.21.15.192:3300/0 conn(0x7fc6001604d0 0x7fc6000b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.149+0000 7fc5fffff640 1 -- 172.21.15.27:0/2367419386 >> v1:172.21.15.192:6789/0 conn(0x7fc6000d0300 legacy=0x7fc6000d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:13.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.149+0000 7fc5fffff640 1 -- 172.21.15.27:0/2367419386 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6000cded0 con 0x7fc6000d2350 2023-12-19T18:15:13.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.149+0000 7fc60480c640 1 --2- 172.21.15.27:0/2367419386 >> v2:172.21.15.192:3300/0 conn(0x7fc6001604d0 0x7fc6000b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:15:13.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.151+0000 7fc5fffff640 1 --2- 172.21.15.27:0/2367419386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6000d2350 0x7fc6000b4d20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80097a0 tx=0x7fc5e802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6f7c0076ae4c5a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:13.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.151+0000 7fc5ff7fe640 1 -- 172.21.15.27:0/2367419386 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e802d040 con 0x7fc6000d2350 2023-12-19T18:15:13.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.151+0000 7fc5ff7fe640 1 -- 172.21.15.27:0/2367419386 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e8031810 con 0x7fc6000d2350 2023-12-19T18:15:13.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.151+0000 7fc5ff7fe640 1 -- 172.21.15.27:0/2367419386 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8031ba0 con 0x7fc6000d2350 2023-12-19T18:15:13.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.152+0000 7fc60580e640 1 -- 172.21.15.27:0/2367419386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6000d2350 msgr2=0x7fc6000b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:13.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.152+0000 7fc60580e640 1 --2- 172.21.15.27:0/2367419386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6000d2350 0x7fc6000b4d20 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fc5e80097a0 tx=0x7fc5e802b8b0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.152+0000 7fc60580e640 1 -- 172.21.15.27:0/2367419386 shutdown_connections 2023-12-19T18:15:13.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.152+0000 7fc60580e640 1 --2- 172.21.15.27:0/2367419386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6000d2350 0x7fc6000b4d20 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.152+0000 7fc60580e640 1 --2- 172.21.15.27:0/2367419386 >> v2:172.21.15.192:3300/0 conn(0x7fc6001604d0 0x7fc6000b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.152+0000 7fc60580e640 1 -- 172.21.15.27:0/2367419386 >> 172.21.15.27:0/2367419386 conn(0x7fc6000cf7e0 msgr2=0x7fc6000cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:13.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.152+0000 7fc60580e640 1 -- 172.21.15.27:0/2367419386 shutdown_connections 2023-12-19T18:15:13.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.152+0000 7fc60580e640 1 -- 172.21.15.27:0/2367419386 wait complete. 2023-12-19T18:15:13.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.154+0000 7fc60580e640 1 Processor -- start 2023-12-19T18:15:13.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.154+0000 7fc60580e640 1 -- start start 2023-12-19T18:15:13.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60580e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6000d0300 0x7fc600154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60580e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6000d2350 0x7fc60015b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60580e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6001604d0 0x7fc6000b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60580e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc600158ae0 con 0x7fc6000d0300 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60580e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6001f27d0 con 0x7fc6001604d0 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60580e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60015a050 con 0x7fc6000d2350 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60480c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6000d0300 0x7fc600154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60480c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6000d0300 0x7fc600154dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60852/0 (socket says 172.21.15.27:60852) 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60480c640 1 -- 172.21.15.27:0/849685850 learned_addr learned my addr 172.21.15.27:0/849685850 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc5fffff640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6000d2350 0x7fc60015b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60480c640 1 -- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6000d2350 msgr2=0x7fc60015b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60500d640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6001604d0 0x7fc6000b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60480c640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6000d2350 0x7fc60015b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60480c640 1 -- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6001604d0 msgr2=0x7fc6000b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:13.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60480c640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6001604d0 0x7fc6000b2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc60480c640 1 -- 172.21.15.27:0/849685850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f0009b30 con 0x7fc6000d0300 2023-12-19T18:15:13.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.155+0000 7fc5fffff640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6000d2350 0x7fc60015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:13.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.156+0000 7fc60480c640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6000d0300 0x7fc600154dd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009f20 tx=0x7fc5f000c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:13.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.156+0000 7fc5fd7fa640 1 -- 172.21.15.27:0/849685850 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0014480 con 0x7fc6000d0300 2023-12-19T18:15:13.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.156+0000 7fc5fd7fa640 1 -- 172.21.15.27:0/849685850 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5f000f040 con 0x7fc6000d0300 2023-12-19T18:15:13.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.156+0000 7fc5fd7fa640 1 -- 172.21.15.27:0/849685850 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5f00147e0 con 0x7fc6000d0300 2023-12-19T18:15:13.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.156+0000 7fc60580e640 1 -- 172.21.15.27:0/849685850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e8009400 con 0x7fc6000d0300 2023-12-19T18:15:13.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.156+0000 7fc60580e640 1 -- 172.21.15.27:0/849685850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc60015a610 con 0x7fc6000d0300 2023-12-19T18:15:13.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.157+0000 7fc60580e640 1 -- 172.21.15.27:0/849685850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc600066f80 con 0x7fc6000d0300 2023-12-19T18:15:13.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.158+0000 7fc5fd7fa640 1 -- 172.21.15.27:0/849685850 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc5f0014940 con 0x7fc6000d0300 2023-12-19T18:15:13.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.159+0000 7fc5fd7fa640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc5e006a430 0x7fc5e006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:13.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.159+0000 7fc5fd7fa640 1 -- 172.21.15.27:0/849685850 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5f00892e0 con 0x7fc6000d0300 2023-12-19T18:15:13.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.160+0000 7fc5fffff640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc5e006a430 0x7fc5e006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:13.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.161+0000 7fc5fffff640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc5e006a430 0x7fc5e006c8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8036000 tx=0x7fc5e803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:13.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.163+0000 7fc5fd7fa640 1 -- 172.21.15.27:0/849685850 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc5f0054830 con 0x7fc6000d0300 2023-12-19T18:15:13.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.394+0000 7fc60580e640 1 -- 172.21.15.27:0/849685850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc60004e840 con 0x7fc6000d0300 2023-12-19T18:15:13.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.395+0000 7fc5fd7fa640 1 -- 172.21.15.27:0/849685850 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fc5f0057e50 con 0x7fc6000d0300 2023-12-19T18:15:13.395 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:13.395 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3679393226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3679393226},{"type":"v1","addr":"172.21.15.27:6835","nonce":3679393226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/809712436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":809712436},{"type":"v1","addr":"172.21.15.192:6835","nonce":809712436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3400689212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3400689212},{"type":"v1","addr":"172.21.15.27:6833","nonce":3400689212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:54.035577+0000","modified":"2023-12-19T18:14:57.547613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/1878426383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1878426383},{"type":"v1","addr":"172.21.15.192:6837","nonce":1878426383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T18:15:13.396 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2023-12-19T18:15:13.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.398+0000 7fc60580e640 1 -- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc5e006a430 msgr2=0x7fc5e006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:13.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.398+0000 7fc60580e640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc5e006a430 0x7fc5e006c8f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc5e8036000 tx=0x7fc5e803c040 comp rx=0 tx=0).stop 2023-12-19T18:15:13.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.398+0000 7fc60580e640 1 -- 172.21.15.27:0/849685850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6000d0300 msgr2=0x7fc600154dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:13.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.398+0000 7fc60580e640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6000d0300 0x7fc600154dd0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009f20 tx=0x7fc5f000c550 comp rx=0 tx=0).stop 2023-12-19T18:15:13.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.398+0000 7fc60580e640 1 -- 172.21.15.27:0/849685850 shutdown_connections 2023-12-19T18:15:13.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.398+0000 7fc60580e640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc5e006a430 0x7fc5e006c8f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.398+0000 7fc60580e640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6001604d0 0x7fc6000b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.398+0000 7fc60580e640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6000d2350 0x7fc60015b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.398+0000 7fc60580e640 1 --2- 172.21.15.27:0/849685850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6000d0300 0x7fc600154dd0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.398+0000 7fc60580e640 1 -- 172.21.15.27:0/849685850 >> 172.21.15.27:0/849685850 conn(0x7fc6000cf7e0 msgr2=0x7fc6000b8210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:13.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.398+0000 7fc60580e640 1 -- 172.21.15.27:0/849685850 shutdown_connections 2023-12-19T18:15:13.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.398+0000 7fc60580e640 1 -- 172.21.15.27:0/849685850 wait complete. 2023-12-19T18:15:13.419 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:13.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.652+0000 7faf68cc8640 1 Processor -- start 2023-12-19T18:15:13.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.653+0000 7faf68cc8640 1 -- start start 2023-12-19T18:15:13.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.654+0000 7faf68cc8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faf6415ceb0 0x7faf640b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:13.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.654+0000 7faf68cc8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf640b5170 con 0x7faf640b4d10 2023-12-19T18:15:13.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.654+0000 7faf68cc8640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf640b6540 con 0x7faf640d2130 2023-12-19T18:15:13.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.654+0000 7faf68cc8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf640cdfb0 con 0x7faf6415ceb0 2023-12-19T18:15:13.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.654+0000 7faf62ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faf640b4d10 0x7faf640d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49288/0 (socket says 172.21.15.27:49288) 2023-12-19T18:15:13.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.654+0000 7faf62ffd640 1 -- 172.21.15.27:0/4189300019 learned_addr learned my addr 172.21.15.27:0/4189300019 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:13.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.654+0000 7faf637fe640 1 --2- 172.21.15.27:0/4189300019 >> v2:172.21.15.27:3300/0 conn(0x7faf6415ceb0 0x7faf640b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:13.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.655+0000 7faf627fc640 1 -- 172.21.15.27:0/4189300019 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3717187792 0 0) 0x7faf640b5170 con 0x7faf640b4d10 2023-12-19T18:15:13.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.655+0000 7faf627fc640 1 -- 172.21.15.27:0/4189300019 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf50003520 con 0x7faf640b4d10 2023-12-19T18:15:13.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.655+0000 7faf637fe640 1 -- 172.21.15.27:0/4189300019 >> v1:172.21.15.192:6789/0 conn(0x7faf640d2130 legacy=0x7faf640b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:13.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.655+0000 7faf637fe640 1 -- 172.21.15.27:0/4189300019 >> v1:172.21.15.27:6789/0 conn(0x7faf640b4d10 legacy=0x7faf640d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:13.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.655+0000 7faf637fe640 1 -- 172.21.15.27:0/4189300019 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf640d2600 con 0x7faf6415ceb0 2023-12-19T18:15:13.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.656+0000 7faf637fe640 1 --2- 172.21.15.27:0/4189300019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6415ceb0 0x7faf640b47d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0097a0 tx=0x7faf4c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69eed20e548796f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:13.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.656+0000 7faf627fc640 1 -- 172.21.15.27:0/4189300019 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c030610 con 0x7faf6415ceb0 2023-12-19T18:15:13.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.656+0000 7faf627fc640 1 -- 172.21.15.27:0/4189300019 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4c030770 con 0x7faf6415ceb0 2023-12-19T18:15:13.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.656+0000 7faf627fc640 1 -- 172.21.15.27:0/4189300019 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c030b00 con 0x7faf6415ceb0 2023-12-19T18:15:13.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.657+0000 7faf68cc8640 1 -- 172.21.15.27:0/4189300019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6415ceb0 msgr2=0x7faf640b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:13.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.657+0000 7faf68cc8640 1 --2- 172.21.15.27:0/4189300019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6415ceb0 0x7faf640b47d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7faf4c0097a0 tx=0x7faf4c02b960 comp rx=0 tx=0).stop 2023-12-19T18:15:13.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.657+0000 7faf68cc8640 1 -- 172.21.15.27:0/4189300019 shutdown_connections 2023-12-19T18:15:13.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.657+0000 7faf68cc8640 1 --2- 172.21.15.27:0/4189300019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf6415ceb0 0x7faf640b47d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.657+0000 7faf68cc8640 1 -- 172.21.15.27:0/4189300019 >> 172.21.15.27:0/4189300019 conn(0x7faf640cf7d0 msgr2=0x7faf640cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:13.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.657+0000 7faf68cc8640 1 -- 172.21.15.27:0/4189300019 shutdown_connections 2023-12-19T18:15:13.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.657+0000 7faf68cc8640 1 -- 172.21.15.27:0/4189300019 wait complete. 2023-12-19T18:15:13.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.659+0000 7faf68cc8640 1 Processor -- start 2023-12-19T18:15:13.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.659+0000 7faf68cc8640 1 -- start start 2023-12-19T18:15:13.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.660+0000 7faf68cc8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf640b4d10 0x7faf641564e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:13.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.660+0000 7faf68cc8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf640d2130 0x7faf64143e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:13.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.660+0000 7faf68cc8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf6415ceb0 0x7faf640c5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:13.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.660+0000 7faf68cc8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf64156ec0 con 0x7faf640b4d10 2023-12-19T18:15:13.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.660+0000 7faf68cc8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf640c6270 con 0x7faf640d2130 2023-12-19T18:15:13.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.660+0000 7faf68cc8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf64156be0 con 0x7faf6415ceb0 2023-12-19T18:15:13.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.660+0000 7faf637fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf640b4d10 0x7faf641564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:13.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.661+0000 7faf637fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf640b4d10 0x7faf641564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60872/0 (socket says 172.21.15.27:60872) 2023-12-19T18:15:13.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.661+0000 7faf637fe640 1 -- 172.21.15.27:0/775967145 learned_addr learned my addr 172.21.15.27:0/775967145 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.661+0000 7faf62ffd640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf640d2130 0x7faf64143e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.661+0000 7faf637fe640 1 -- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf6415ceb0 msgr2=0x7faf640c5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.661+0000 7faf637fe640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf6415ceb0 0x7faf640c5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.661+0000 7faf637fe640 1 -- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf640d2130 msgr2=0x7faf64143e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.661+0000 7faf637fe640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf640d2130 0x7faf64143e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.661+0000 7faf637fe640 1 -- 172.21.15.27:0/775967145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c009400 con 0x7faf640b4d10 2023-12-19T18:15:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.661+0000 7faf62ffd640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf640d2130 0x7faf64143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.661+0000 7faf637fe640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf640b4d10 0x7faf641564e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf4c02bf70 tx=0x7faf4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.662+0000 7faf5bfff640 1 -- 172.21.15.27:0/775967145 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c03e040 con 0x7faf640b4d10 2023-12-19T18:15:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.662+0000 7faf5bfff640 1 -- 172.21.15.27:0/775967145 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4c039cb0 con 0x7faf640b4d10 2023-12-19T18:15:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.662+0000 7faf5bfff640 1 -- 172.21.15.27:0/775967145 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4c044630 con 0x7faf640b4d10 2023-12-19T18:15:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.662+0000 7faf68cc8640 1 -- 172.21.15.27:0/775967145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf640c7a50 con 0x7faf640b4d10 2023-12-19T18:15:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.662+0000 7faf68cc8640 1 -- 172.21.15.27:0/775967145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf640c8010 con 0x7faf640b4d10 2023-12-19T18:15:13.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.663+0000 7faf68cc8640 1 -- 172.21.15.27:0/775967145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf640c78b0 con 0x7faf640b4d10 2023-12-19T18:15:13.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.664+0000 7faf5bfff640 1 -- 172.21.15.27:0/775967145 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faf4c043390 con 0x7faf640b4d10 2023-12-19T18:15:13.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.665+0000 7faf5bfff640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf4406a430 0x7faf4406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:13.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.665+0000 7faf5bfff640 1 -- 172.21.15.27:0/775967145 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf4c0aac40 con 0x7faf640b4d10 2023-12-19T18:15:13.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.665+0000 7faf62ffd640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf4406a430 0x7faf4406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:13.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.666+0000 7faf62ffd640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf4406a430 0x7faf4406c8f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf640d3ce0 tx=0x7faf54009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:13.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.669+0000 7faf5bfff640 1 -- 172.21.15.27:0/775967145 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7faf640c78b0 con 0x7faf640b4d10 2023-12-19T18:15:13.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.864+0000 7faf68cc8640 1 -- 172.21.15.27:0/775967145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf640c78b0 con 0x7faf640b4d10 2023-12-19T18:15:13.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.867+0000 7faf5bfff640 1 -- 172.21.15.27:0/775967145 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16041 (secure 0 0 0) 0x7faf640c78b0 con 0x7faf640b4d10 2023-12-19T18:15:13.870 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:13.870 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:55.039690+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:13.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.870+0000 7faf68cc8640 1 -- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf4406a430 msgr2=0x7faf4406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:13.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.870+0000 7faf68cc8640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf4406a430 0x7faf4406c8f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7faf640d3ce0 tx=0x7faf54009040 comp rx=0 tx=0).stop 2023-12-19T18:15:13.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.870+0000 7faf68cc8640 1 -- 172.21.15.27:0/775967145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf640b4d10 msgr2=0x7faf641564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:13.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.870+0000 7faf68cc8640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf640b4d10 0x7faf641564e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7faf4c02bf70 tx=0x7faf4c004730 comp rx=0 tx=0).stop 2023-12-19T18:15:13.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.871+0000 7faf68cc8640 1 -- 172.21.15.27:0/775967145 shutdown_connections 2023-12-19T18:15:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.871+0000 7faf68cc8640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf4406a430 0x7faf4406c8f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.871+0000 7faf68cc8640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf6415ceb0 0x7faf640c5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.871+0000 7faf68cc8640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf640d2130 0x7faf64143e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.871+0000 7faf68cc8640 1 --2- 172.21.15.27:0/775967145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf640b4d10 0x7faf641564e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.871+0000 7faf68cc8640 1 -- 172.21.15.27:0/775967145 >> 172.21.15.27:0/775967145 conn(0x7faf640cf7d0 msgr2=0x7faf6415b850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.871+0000 7faf68cc8640 1 -- 172.21.15.27:0/775967145 shutdown_connections 2023-12-19T18:15:13.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:13.871+0000 7faf68cc8640 1 -- 172.21.15.27:0/775967145 wait complete. 2023-12-19T18:15:13.890 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:15:13.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:15:14.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.123+0000 7ff5b7720640 1 Processor -- start 2023-12-19T18:15:14.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.123+0000 7ff5b7720640 1 -- start start 2023-12-19T18:15:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.125+0000 7ff5b7720640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff5b00b5160 0x7ff5b00b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.125+0000 7ff5b7720640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff5b00b55f0 0x7ff5b00b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.125+0000 7ff5b7720640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5b015d750 con 0x7ff5b015d370 2023-12-19T18:15:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.125+0000 7ff5b7720640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b00b2880 con 0x7ff5b00b55f0 2023-12-19T18:15:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.125+0000 7ff5b7720640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5b00d1750 con 0x7ff5b00b5160 2023-12-19T18:15:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.125+0000 7ff5b54bc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff5b015d370 0x7ff5b00b4a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49304/0 (socket says 172.21.15.27:49304) 2023-12-19T18:15:14.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.125+0000 7ff5b54bc640 1 -- 172.21.15.27:0/261023410 learned_addr learned my addr 172.21.15.27:0/261023410 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.125+0000 7ff5b5cbd640 1 --2- 172.21.15.27:0/261023410 >> v2:172.21.15.27:3300/0 conn(0x7ff5b00b55f0 0x7ff5b00b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.125+0000 7ff5b4cbb640 1 --2- 172.21.15.27:0/261023410 >> v2:172.21.15.192:3300/0 conn(0x7ff5b00b5160 0x7ff5b00b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.126+0000 7ff5affff640 1 -- 172.21.15.27:0/261023410 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 887810505 0 0) 0x7ff5b015d750 con 0x7ff5b015d370 2023-12-19T18:15:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.126+0000 7ff5affff640 1 -- 172.21.15.27:0/261023410 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5a0003520 con 0x7ff5b015d370 2023-12-19T18:15:14.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.126+0000 7ff5b5cbd640 1 -- 172.21.15.27:0/261023410 >> v2:172.21.15.192:3300/0 conn(0x7ff5b00b5160 msgr2=0x7ff5b00b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.126+0000 7ff5b5cbd640 1 --2- 172.21.15.27:0/261023410 >> v2:172.21.15.192:3300/0 conn(0x7ff5b00b5160 0x7ff5b00b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.126+0000 7ff5b5cbd640 1 -- 172.21.15.27:0/261023410 >> v1:172.21.15.27:6789/0 conn(0x7ff5b015d370 legacy=0x7ff5b00b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.126+0000 7ff5b5cbd640 1 -- 172.21.15.27:0/261023410 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b00d1a30 con 0x7ff5b00b55f0 2023-12-19T18:15:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.127+0000 7ff5b5cbd640 1 --2- 172.21.15.27:0/261023410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b00b55f0 0x7ff5b00b62b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80097a0 tx=0x7ff5a802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac03e2c0b62acac7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.127+0000 7ff5affff640 1 -- 172.21.15.27:0/261023410 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8030610 con 0x7ff5b00b55f0 2023-12-19T18:15:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.127+0000 7ff5affff640 1 -- 172.21.15.27:0/261023410 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5a8030770 con 0x7ff5b00b55f0 2023-12-19T18:15:14.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.127+0000 7ff5affff640 1 -- 172.21.15.27:0/261023410 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8030b00 con 0x7ff5b00b55f0 2023-12-19T18:15:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.128+0000 7ff5b7720640 1 -- 172.21.15.27:0/261023410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b00b55f0 msgr2=0x7ff5b00b62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.128+0000 7ff5b7720640 1 --2- 172.21.15.27:0/261023410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b00b55f0 0x7ff5b00b62b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5a80097a0 tx=0x7ff5a802b960 comp rx=0 tx=0).stop 2023-12-19T18:15:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.128+0000 7ff5b7720640 1 -- 172.21.15.27:0/261023410 shutdown_connections 2023-12-19T18:15:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.128+0000 7ff5b7720640 1 --2- 172.21.15.27:0/261023410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b00b55f0 0x7ff5b00b62b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.128+0000 7ff5b7720640 1 --2- 172.21.15.27:0/261023410 >> v2:172.21.15.192:3300/0 conn(0x7ff5b00b5160 0x7ff5b00b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.128+0000 7ff5b7720640 1 -- 172.21.15.27:0/261023410 >> 172.21.15.27:0/261023410 conn(0x7ff5b006a480 msgr2=0x7ff5b006a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.128+0000 7ff5b7720640 1 -- 172.21.15.27:0/261023410 shutdown_connections 2023-12-19T18:15:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.128+0000 7ff5b7720640 1 -- 172.21.15.27:0/261023410 wait complete. 2023-12-19T18:15:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.130+0000 7ff5b7720640 1 Processor -- start 2023-12-19T18:15:14.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.130+0000 7ff5b7720640 1 -- start start 2023-12-19T18:15:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.131+0000 7ff5b7720640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5b00b5160 0x7ff5b01f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.131+0000 7ff5b7720640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b00b55f0 0x7ff5b00caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.131+0000 7ff5b7720640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5b015d370 0x7ff5b0159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.131+0000 7ff5b7720640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b00cb690 con 0x7ff5b00b55f0 2023-12-19T18:15:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.131+0000 7ff5b7720640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b00cb3b0 con 0x7ff5b00b5160 2023-12-19T18:15:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.131+0000 7ff5b7720640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b015dae0 con 0x7ff5b015d370 2023-12-19T18:15:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.131+0000 7ff5b4cbb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b00b55f0 0x7ff5b00caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.131+0000 7ff5b4cbb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b00b55f0 0x7ff5b00caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60888/0 (socket says 172.21.15.27:60888) 2023-12-19T18:15:14.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.131+0000 7ff5b4cbb640 1 -- 172.21.15.27:0/2796637062 learned_addr learned my addr 172.21.15.27:0/2796637062 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.132+0000 7ff5b54bc640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5b00b5160 0x7ff5b01f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.132+0000 7ff5b5cbd640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5b015d370 0x7ff5b0159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.132+0000 7ff5b4cbb640 1 -- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5b015d370 msgr2=0x7ff5b0159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.132+0000 7ff5b4cbb640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5b015d370 0x7ff5b0159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.132+0000 7ff5b4cbb640 1 -- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5b00b5160 msgr2=0x7ff5b01f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.132+0000 7ff5b4cbb640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5b00b5160 0x7ff5b01f2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.132+0000 7ff5b4cbb640 1 -- 172.21.15.27:0/2796637062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a8009400 con 0x7ff5b00b55f0 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.132+0000 7ff5b5cbd640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5b015d370 0x7ff5b0159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.132+0000 7ff5b4cbb640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b00b55f0 0x7ff5b00caab0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00d5f0 tx=0x7ff59c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.133+0000 7ff5adffb640 1 -- 172.21.15.27:0/2796637062 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c0125f0 con 0x7ff5b00b55f0 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.133+0000 7ff5adffb640 1 -- 172.21.15.27:0/2796637062 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c012750 con 0x7ff5b00b55f0 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.133+0000 7ff5adffb640 1 -- 172.21.15.27:0/2796637062 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c012b00 con 0x7ff5b00b55f0 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.133+0000 7ff5b7720640 1 -- 172.21.15.27:0/2796637062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b015ddc0 con 0x7ff5b00b55f0 2023-12-19T18:15:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.133+0000 7ff5b7720640 1 -- 172.21.15.27:0/2796637062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b00caff0 con 0x7ff5b00b55f0 2023-12-19T18:15:14.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.134+0000 7ff5b7720640 1 -- 172.21.15.27:0/2796637062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5b015d940 con 0x7ff5b00b55f0 2023-12-19T18:15:14.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.135+0000 7ff5adffb640 1 -- 172.21.15.27:0/2796637062 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff59c012c60 con 0x7ff5b00b55f0 2023-12-19T18:15:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.136+0000 7ff5adffb640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff59406a430 0x7ff59406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.136+0000 7ff5adffb640 1 -- 172.21.15.27:0/2796637062 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff59c088d10 con 0x7ff5b00b55f0 2023-12-19T18:15:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.137+0000 7ff5b54bc640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff59406a430 0x7ff59406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.139+0000 7ff5b54bc640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff59406a430 0x7ff59406c8f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4001560 tx=0x7ff5a4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:14.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.141+0000 7ff5adffb640 1 -- 172.21.15.27:0/2796637062 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff5b015d940 con 0x7ff5b00b55f0 2023-12-19T18:15:14.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.333+0000 7ff5b7720640 1 -- 172.21.15.27:0/2796637062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5b015d940 con 0x7ff5b00b55f0 2023-12-19T18:15:14.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.334+0000 7ff5adffb640 1 -- 172.21.15.27:0/2796637062 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5b015d940 con 0x7ff5b00b55f0 2023-12-19T18:15:14.334 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:15:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.337+0000 7ff5b7720640 1 -- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff59406a430 msgr2=0x7ff59406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.337+0000 7ff5b7720640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff59406a430 0x7ff59406c8f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ff5a4001560 tx=0x7ff5a4009040 comp rx=0 tx=0).stop 2023-12-19T18:15:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.337+0000 7ff5b7720640 1 -- 172.21.15.27:0/2796637062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b00b55f0 msgr2=0x7ff5b00caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:14.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.337+0000 7ff5b7720640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b00b55f0 0x7ff5b00caab0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff59c00d5f0 tx=0x7ff59c00dab0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.337+0000 7ff5b7720640 1 -- 172.21.15.27:0/2796637062 shutdown_connections 2023-12-19T18:15:14.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.337+0000 7ff5b7720640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff59406a430 0x7ff59406c8f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.337+0000 7ff5b7720640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5b015d370 0x7ff5b0159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.337+0000 7ff5b7720640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5b00b55f0 0x7ff5b00caab0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.337+0000 7ff5b7720640 1 --2- 172.21.15.27:0/2796637062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5b00b5160 0x7ff5b01f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.337+0000 7ff5b7720640 1 -- 172.21.15.27:0/2796637062 >> 172.21.15.27:0/2796637062 conn(0x7ff5b006a480 msgr2=0x7ff5b015c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:14.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.337+0000 7ff5b7720640 1 -- 172.21.15.27:0/2796637062 shutdown_connections 2023-12-19T18:15:14.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.337+0000 7ff5b7720640 1 -- 172.21.15.27:0/2796637062 wait complete. 2023-12-19T18:15:14.352 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:15:14.362 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:15:14.363 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:15:14.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.619+0000 7f74dc9ce640 1 Processor -- start 2023-12-19T18:15:14.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.620+0000 7f74dc9ce640 1 -- start start 2023-12-19T18:15:14.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.621+0000 7f74dc9ce640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f74d80c2720 0x7f74d806fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:14.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.621+0000 7f74dc9ce640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f74d8071420 0x7f74d815ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:14.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.621+0000 7f74dc9ce640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d806bc80 con 0x7f74d815b1a0 2023-12-19T18:15:14.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.621+0000 7f74dc9ce640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d806d690 con 0x7f74d8071420 2023-12-19T18:15:14.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.621+0000 7f74dc9ce640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d806d0a0 con 0x7f74d80c2720 2023-12-19T18:15:14.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.621+0000 7f74d6d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f74d8071420 0x7f74d815ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:14.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.621+0000 7f74d6d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f74d8071420 0x7f74d815ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60900/0 (socket says 172.21.15.27:60900) 2023-12-19T18:15:14.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.621+0000 7f74d6d9d640 1 -- 172.21.15.27:0/177412397 learned_addr learned my addr 172.21.15.27:0/177412397 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:14.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.621+0000 7f74cdd9b640 1 --2- 172.21.15.27:0/177412397 >> v2:172.21.15.192:3300/0 conn(0x7f74d80c2720 0x7f74d806fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:14.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.622+0000 7f74d6d9d640 1 -- 172.21.15.27:0/177412397 >> v2:172.21.15.192:3300/0 conn(0x7f74d80c2720 msgr2=0x7f74d806fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:14.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.622+0000 7f74d6d9d640 1 --2- 172.21.15.27:0/177412397 >> v2:172.21.15.192:3300/0 conn(0x7f74d80c2720 0x7f74d806fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.622+0000 7f74d6d9d640 1 -- 172.21.15.27:0/177412397 >> v1:172.21.15.192:6789/0 conn(0x7f74d815b1a0 legacy=0x7f74d806f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:14.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.622+0000 7f74d6d9d640 1 -- 172.21.15.27:0/177412397 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d806b6c0 con 0x7f74d8071420 2023-12-19T18:15:14.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.623+0000 7f74d6d9d640 1 --2- 172.21.15.27:0/177412397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d8071420 0x7f74d815ed80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f74d806acf0 tx=0x7f74d002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cfc07bcb8d1078d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:14.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.623+0000 7f74d5d9b640 1 -- 172.21.15.27:0/177412397 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d002c9f0 con 0x7f74d8071420 2023-12-19T18:15:14.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.624+0000 7f74d5d9b640 1 -- 172.21.15.27:0/177412397 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d002cb50 con 0x7f74d8071420 2023-12-19T18:15:14.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.624+0000 7f74d5d9b640 1 -- 172.21.15.27:0/177412397 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d00343f0 con 0x7f74d8071420 2023-12-19T18:15:14.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.624+0000 7f74dc9ce640 1 -- 172.21.15.27:0/177412397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d8071420 msgr2=0x7f74d815ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:14.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.624+0000 7f74dc9ce640 1 --2- 172.21.15.27:0/177412397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d8071420 0x7f74d815ed80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f74d806acf0 tx=0x7f74d002be50 comp rx=0 tx=0).stop 2023-12-19T18:15:14.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.624+0000 7f74dc9ce640 1 -- 172.21.15.27:0/177412397 shutdown_connections 2023-12-19T18:15:14.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.624+0000 7f74dc9ce640 1 --2- 172.21.15.27:0/177412397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d8071420 0x7f74d815ed80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.624+0000 7f74dc9ce640 1 --2- 172.21.15.27:0/177412397 >> v2:172.21.15.192:3300/0 conn(0x7f74d80c2720 0x7f74d806fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.624+0000 7f74dc9ce640 1 -- 172.21.15.27:0/177412397 >> 172.21.15.27:0/177412397 conn(0x7f74d80cebd0 msgr2=0x7f74d80cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:14.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.624+0000 7f74dc9ce640 1 -- 172.21.15.27:0/177412397 shutdown_connections 2023-12-19T18:15:14.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.624+0000 7f74dc9ce640 1 -- 172.21.15.27:0/177412397 wait complete. 2023-12-19T18:15:14.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.626+0000 7f74dc9ce640 1 Processor -- start 2023-12-19T18:15:14.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.626+0000 7f74dc9ce640 1 -- start start 2023-12-19T18:15:14.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.627+0000 7f74dc9ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d8071420 0x7f74d8158390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:14.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.627+0000 7f74dc9ce640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74d80c2720 0x7f74d80c7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:14.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.627+0000 7f74dc9ce640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74d815b1a0 0x7f74d80c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:14.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.627+0000 7f74dc9ce640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d81e8150 con 0x7f74d8071420 2023-12-19T18:15:14.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.627+0000 7f74dc9ce640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d81e6dd0 con 0x7f74d80c2720 2023-12-19T18:15:14.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.627+0000 7f74dc9ce640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d81e70b0 con 0x7f74d815b1a0 2023-12-19T18:15:14.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.627+0000 7f74d659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d8071420 0x7f74d8158390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:14.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.627+0000 7f74d659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d8071420 0x7f74d8158390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60902/0 (socket says 172.21.15.27:60902) 2023-12-19T18:15:14.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.627+0000 7f74d659c640 1 -- 172.21.15.27:0/1376478660 learned_addr learned my addr 172.21.15.27:0/1376478660 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:14.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.628+0000 7f74cdd9b640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74d80c2720 0x7f74d80c7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:14.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.628+0000 7f74d6d9d640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74d815b1a0 0x7f74d80c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:14.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.628+0000 7f74d659c640 1 -- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74d815b1a0 msgr2=0x7f74d80c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:14.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.628+0000 7f74d659c640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74d815b1a0 0x7f74d80c7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.628+0000 7f74d659c640 1 -- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74d80c2720 msgr2=0x7f74d80c7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:14.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.628+0000 7f74d659c640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74d80c2720 0x7f74d80c7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.628+0000 7f74d659c640 1 -- 172.21.15.27:0/1376478660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d0009b30 con 0x7f74d8071420 2023-12-19T18:15:14.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.628+0000 7f74d659c640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d8071420 0x7f74d8158390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f74c800b3f0 tx=0x7f74c800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:14.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.628+0000 7f74cf7fe640 1 -- 172.21.15.27:0/1376478660 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c8011830 con 0x7f74d8071420 2023-12-19T18:15:14.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.628+0000 7f74cf7fe640 1 -- 172.21.15.27:0/1376478660 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c8011990 con 0x7f74d8071420 2023-12-19T18:15:14.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.628+0000 7f74cf7fe640 1 -- 172.21.15.27:0/1376478660 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c8011d40 con 0x7f74d8071420 2023-12-19T18:15:14.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.628+0000 7f74dc9ce640 1 -- 172.21.15.27:0/1376478660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d81e7390 con 0x7f74d8071420 2023-12-19T18:15:14.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.629+0000 7f74dc9ce640 1 -- 172.21.15.27:0/1376478660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d81e7950 con 0x7f74d8071420 2023-12-19T18:15:14.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.630+0000 7f74dc9ce640 1 -- 172.21.15.27:0/1376478660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d8068e90 con 0x7f74d8071420 2023-12-19T18:15:14.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.631+0000 7f74cf7fe640 1 -- 172.21.15.27:0/1376478660 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f74c801a400 con 0x7f74d8071420 2023-12-19T18:15:14.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.632+0000 7f74cf7fe640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f74b806a430 0x7f74b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:14.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.632+0000 7f74cf7fe640 1 -- 172.21.15.27:0/1376478660 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74c80887b0 con 0x7f74d8071420 2023-12-19T18:15:14.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.632+0000 7f74cdd9b640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f74b806a430 0x7f74b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:14.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.634+0000 7f74cdd9b640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f74b806a430 0x7f74b806c8f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f74d81e89f0 tx=0x7f74c0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:14.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.636+0000 7f74cf7fe640 1 -- 172.21.15.27:0/1376478660 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f74c8053d90 con 0x7f74d8071420 2023-12-19T18:15:14.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.828+0000 7f74dc9ce640 1 -- 172.21.15.27:0/1376478660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74d80702f0 con 0x7f74d8071420 2023-12-19T18:15:14.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.830+0000 7f74cf7fe640 1 -- 172.21.15.27:0/1376478660 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f74c80573b0 con 0x7f74d8071420 2023-12-19T18:15:14.829 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:15:14.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.832+0000 7f74dc9ce640 1 -- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f74b806a430 msgr2=0x7f74b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.832+0000 7f74dc9ce640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f74b806a430 0x7f74b806c8f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f74d81e89f0 tx=0x7f74c0008040 comp rx=0 tx=0).stop 2023-12-19T18:15:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.832+0000 7f74dc9ce640 1 -- 172.21.15.27:0/1376478660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d8071420 msgr2=0x7f74d8158390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.832+0000 7f74dc9ce640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d8071420 0x7f74d8158390 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f74c800b3f0 tx=0x7f74c800b8b0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.832+0000 7f74dc9ce640 1 -- 172.21.15.27:0/1376478660 shutdown_connections 2023-12-19T18:15:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.832+0000 7f74dc9ce640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f74b806a430 0x7f74b806c8f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.832+0000 7f74dc9ce640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f74d815b1a0 0x7f74d80c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.832+0000 7f74dc9ce640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f74d80c2720 0x7f74d80c7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.832+0000 7f74dc9ce640 1 --2- 172.21.15.27:0/1376478660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d8071420 0x7f74d8158390 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.832+0000 7f74dc9ce640 1 -- 172.21.15.27:0/1376478660 >> 172.21.15.27:0/1376478660 conn(0x7f74d80cebd0 msgr2=0x7f74d80afcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.832+0000 7f74dc9ce640 1 -- 172.21.15.27:0/1376478660 shutdown_connections 2023-12-19T18:15:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:14.832+0000 7f74dc9ce640 1 -- 172.21.15.27:0/1376478660 wait complete. 2023-12-19T18:15:14.850 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:15:15.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.084+0000 7ff1169bc640 1 Processor -- start 2023-12-19T18:15:15.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.085+0000 7ff1169bc640 1 -- start start 2023-12-19T18:15:15.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.086+0000 7ff1169bc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1100d1a20 0x7ff1100cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:15.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.086+0000 7ff1169bc640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1100d3850 con 0x7ff1100b3980 2023-12-19T18:15:15.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.086+0000 7ff1169bc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1100d0b10 con 0x7ff11015d120 2023-12-19T18:15:15.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.086+0000 7ff1169bc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11015d500 con 0x7ff1100d1a20 2023-12-19T18:15:15.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.087+0000 7ff1161bb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff1100b3980 0x7ff1100cd940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49312/0 (socket says 172.21.15.27:49312) 2023-12-19T18:15:15.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.087+0000 7ff1161bb640 1 -- 172.21.15.27:0/3737214505 learned_addr learned my addr 172.21.15.27:0/3737214505 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:15.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.087+0000 7ff1151b9640 1 --2- 172.21.15.27:0/3737214505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1100d1a20 0x7ff1100cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:15.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.087+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3075564227 0 0) 0x7ff1100d3850 con 0x7ff1100b3980 2023-12-19T18:15:15.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.087+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0fc003520 con 0x7ff1100b3980 2023-12-19T18:15:15.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2095007710 0 0) 0x7ff0fc003520 con 0x7ff1100b3980 2023-12-19T18:15:15.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1100d1a20 msgr2=0x7ff1100cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:15.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1149b8640 1 --2- 172.21.15.27:0/3737214505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1100d1a20 0x7ff1100cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 >> v1:172.21.15.192:6789/0 conn(0x7ff11015d120 legacy=0x7ff1100d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1100d1040 con 0x7ff1100b3980 2023-12-19T18:15:15.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7ff108001670 con 0x7ff1100b3980 2023-12-19T18:15:15.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 >> v1:172.21.15.27:6789/0 conn(0x7ff1100b3980 legacy=0x7ff1100cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1149b8640 1 --2- 172.21.15.27:0/3737214505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0fc0039a0 0x7ff0fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:15.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1149b8640 1 --2- 172.21.15.27:0/3737214505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0fc004340 0x7ff0fc004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:15.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1149b8640 1 --2- 172.21.15.27:0/3737214505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc005020 0x7ff0fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:15.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11015d500 con 0x7ff0fc005020 2023-12-19T18:15:15.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc00bf40 con 0x7ff0fc004340 2023-12-19T18:15:15.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0fc00c100 con 0x7ff0fc0039a0 2023-12-19T18:15:15.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.088+0000 7ff1161bb640 1 --2- 172.21.15.27:0/3737214505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc005020 0x7ff0fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:15.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.089+0000 7ff1159ba640 1 --2- 172.21.15.27:0/3737214505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0fc0039a0 0x7ff0fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:15.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.089+0000 7ff1161bb640 1 -- 172.21.15.27:0/3737214505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0fc0039a0 msgr2=0x7ff0fc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:15.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.089+0000 7ff1161bb640 1 --2- 172.21.15.27:0/3737214505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0fc0039a0 0x7ff0fc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.089+0000 7ff1161bb640 1 -- 172.21.15.27:0/3737214505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0fc004340 msgr2=0x7ff0fc004820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:15.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.089+0000 7ff1161bb640 1 --2- 172.21.15.27:0/3737214505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff0fc004340 0x7ff0fc004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.089+0000 7ff1161bb640 1 -- 172.21.15.27:0/3737214505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1100d1040 con 0x7ff0fc005020 2023-12-19T18:15:15.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.089+0000 7ff1159ba640 1 --2- 172.21.15.27:0/3737214505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0fc0039a0 0x7ff0fc003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:15.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.090+0000 7ff1161bb640 1 --2- 172.21.15.27:0/3737214505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc005020 0x7ff0fc00b980 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff10800c290 tx=0x7ff0fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fd7821bfbe15900 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:15.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.090+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10802cb20 con 0x7ff0fc005020 2023-12-19T18:15:15.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.090+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10802cc80 con 0x7ff0fc005020 2023-12-19T18:15:15.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.090+0000 7ff1149b8640 1 -- 172.21.15.27:0/3737214505 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1080058d0 con 0x7ff0fc005020 2023-12-19T18:15:15.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.090+0000 7ff1169bc640 1 -- 172.21.15.27:0/3737214505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc005020 msgr2=0x7ff0fc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.091+0000 7ff1169bc640 1 --2- 172.21.15.27:0/3737214505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc005020 0x7ff0fc00b980 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff10800c290 tx=0x7ff0fc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.091+0000 7ff1161bb640 1 -- 172.21.15.27:0/3737214505 reap_dead start 2023-12-19T18:15:15.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.091+0000 7ff1169bc640 1 -- 172.21.15.27:0/3737214505 shutdown_connections 2023-12-19T18:15:15.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.091+0000 7ff1169bc640 1 -- 172.21.15.27:0/3737214505 >> 172.21.15.27:0/3737214505 conn(0x7ff1100d2d30 msgr2=0x7ff1100d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:15.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.091+0000 7ff1169bc640 1 -- 172.21.15.27:0/3737214505 shutdown_connections 2023-12-19T18:15:15.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.091+0000 7ff1169bc640 1 -- 172.21.15.27:0/3737214505 wait complete. 2023-12-19T18:15:15.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.092+0000 7ff1169bc640 1 Processor -- start 2023-12-19T18:15:15.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.093+0000 7ff1169bc640 1 -- start start 2023-12-19T18:15:15.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.093+0000 7ff1169bc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc005020 0x7ff1100caf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:15.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.093+0000 7ff1169bc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1100a05d0 0x7ff1100b6430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:15.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1169bc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1100b0010 0x7ff1100b72a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:15.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1169bc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100cb4a0 con 0x7ff0fc005020 2023-12-19T18:15:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1169bc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100b0cf0 con 0x7ff1100b0010 2023-12-19T18:15:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1169bc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100b6970 con 0x7ff1100a05d0 2023-12-19T18:15:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1159ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc005020 0x7ff1100caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1159ba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc005020 0x7ff1100caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60920/0 (socket says 172.21.15.27:60920) 2023-12-19T18:15:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1159ba640 1 -- 172.21.15.27:0/3546066677 learned_addr learned my addr 172.21.15.27:0/3546066677 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1151b9640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1100a05d0 0x7ff1100b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1159ba640 1 -- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1100a05d0 msgr2=0x7ff1100b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1161bb640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1100b0010 0x7ff1100b72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1159ba640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1100a05d0 0x7ff1100b6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1159ba640 1 -- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1100b0010 msgr2=0x7ff1100b72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1159ba640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1100b0010 0x7ff1100b72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1159ba640 1 -- 172.21.15.27:0/3546066677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1100cb690 con 0x7ff0fc005020 2023-12-19T18:15:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.094+0000 7ff1159ba640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc005020 0x7ff1100caf60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00d7a0 tx=0x7ff10c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.095+0000 7ff1027fc640 1 -- 172.21.15.27:0/3546066677 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c017070 con 0x7ff0fc005020 2023-12-19T18:15:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.095+0000 7ff1027fc640 1 -- 172.21.15.27:0/3546066677 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff10c005070 con 0x7ff0fc005020 2023-12-19T18:15:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.095+0000 7ff1027fc640 1 -- 172.21.15.27:0/3546066677 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff10c005420 con 0x7ff0fc005020 2023-12-19T18:15:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.095+0000 7ff1169bc640 1 -- 172.21.15.27:0/3546066677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1100cb970 con 0x7ff0fc005020 2023-12-19T18:15:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.095+0000 7ff1169bc640 1 -- 172.21.15.27:0/3546066677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1100b5c30 con 0x7ff0fc005020 2023-12-19T18:15:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.096+0000 7ff1169bc640 1 -- 172.21.15.27:0/3546066677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11004e840 con 0x7ff0fc005020 2023-12-19T18:15:15.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.097+0000 7ff1027fc640 1 -- 172.21.15.27:0/3546066677 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff10c007030 con 0x7ff0fc005020 2023-12-19T18:15:15.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.098+0000 7ff1027fc640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff0f406a460 0x7ff0f406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:15.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.098+0000 7ff1027fc640 1 -- 172.21.15.27:0/3546066677 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff10c088130 con 0x7ff0fc005020 2023-12-19T18:15:15.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.099+0000 7ff1151b9640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff0f406a460 0x7ff0f406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:15.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.100+0000 7ff1151b9640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff0f406a460 0x7ff0f406c920 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff104004b30 tx=0x7ff10400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:15.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.102+0000 7ff1027fc640 1 -- 172.21.15.27:0/3546066677 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff10c053710 con 0x7ff0fc005020 2023-12-19T18:15:15.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.295+0000 7ff1169bc640 1 -- 172.21.15.27:0/3546066677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff1100b4540 con 0x7ff0fc005020 2023-12-19T18:15:15.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.296+0000 7ff1027fc640 1 -- 172.21.15.27:0/3546066677 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff10c056d30 con 0x7ff0fc005020 2023-12-19T18:15:15.295 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:15:15.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.299+0000 7ff1169bc640 1 -- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff0f406a460 msgr2=0x7ff0f406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.299+0000 7ff1169bc640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff0f406a460 0x7ff0f406c920 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7ff104004b30 tx=0x7ff10400b040 comp rx=0 tx=0).stop 2023-12-19T18:15:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.299+0000 7ff1169bc640 1 -- 172.21.15.27:0/3546066677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc005020 msgr2=0x7ff1100caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.299+0000 7ff1169bc640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc005020 0x7ff1100caf60 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff10c00d7a0 tx=0x7ff10c00dc60 comp rx=0 tx=0).stop 2023-12-19T18:15:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.299+0000 7ff1169bc640 1 -- 172.21.15.27:0/3546066677 shutdown_connections 2023-12-19T18:15:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.299+0000 7ff1169bc640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff0f406a460 0x7ff0f406c920 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.299+0000 7ff1169bc640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1100b0010 0x7ff1100b72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.299+0000 7ff1169bc640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1100a05d0 0x7ff1100b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.299+0000 7ff1169bc640 1 --2- 172.21.15.27:0/3546066677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0fc005020 0x7ff1100caf60 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.299+0000 7ff1169bc640 1 -- 172.21.15.27:0/3546066677 >> 172.21.15.27:0/3546066677 conn(0x7ff1100d2d30 msgr2=0x7ff1100d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.299+0000 7ff1169bc640 1 -- 172.21.15.27:0/3546066677 shutdown_connections 2023-12-19T18:15:15.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.299+0000 7ff1169bc640 1 -- 172.21.15.27:0/3546066677 wait complete. 2023-12-19T18:15:15.318 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:15:15.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.552+0000 7f7ae43cb640 1 Processor -- start 2023-12-19T18:15:15.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.552+0000 7f7ae43cb640 1 -- start start 2023-12-19T18:15:15.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.553+0000 7f7ae43cb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7adc0cdf90 0x7f7adc0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:15.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.554+0000 7f7ae43cb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7adc0d1970 con 0x7f7adc0d1e00 2023-12-19T18:15:15.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.554+0000 7f7ae43cb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7adc06a5e0 con 0x7f7adc15d240 2023-12-19T18:15:15.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.554+0000 7f7ae43cb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7adc0ce960 con 0x7f7adc0cdf90 2023-12-19T18:15:15.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.554+0000 7f7ae2968640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7adc0d1e00 0x7f7adc0d2270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49326/0 (socket says 172.21.15.27:49326) 2023-12-19T18:15:15.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.554+0000 7f7ae2968640 1 -- 172.21.15.27:0/1530922218 learned_addr learned my addr 172.21.15.27:0/1530922218 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:15.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.554+0000 7f7ae1966640 1 --2- 172.21.15.27:0/1530922218 >> v2:172.21.15.27:3300/0 conn(0x7f7adc0cdf90 0x7f7adc0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:15.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.554+0000 7f7ae1165640 1 -- 172.21.15.27:0/1530922218 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 768530205 0 0) 0x7f7adc0d1970 con 0x7f7adc0d1e00 2023-12-19T18:15:15.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.555+0000 7f7ae1966640 1 -- 172.21.15.27:0/1530922218 >> v1:172.21.15.192:6789/0 conn(0x7f7adc15d240 legacy=0x7f7adc1618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.555+0000 7f7ae1966640 1 -- 172.21.15.27:0/1530922218 >> v1:172.21.15.27:6789/0 conn(0x7f7adc0d1e00 legacy=0x7f7adc0d2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.555+0000 7f7ae1966640 1 -- 172.21.15.27:0/1530922218 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc06a880 con 0x7f7adc0cdf90 2023-12-19T18:15:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.556+0000 7f7ae1966640 1 --2- 172.21.15.27:0/1530922218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7adc0cdf90 0x7f7adc0ce420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0097a0 tx=0x7f7acc02b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1c66fd1ee978c3f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.556+0000 7f7ae1165640 1 -- 172.21.15.27:0/1530922218 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc02f810 con 0x7f7adc0cdf90 2023-12-19T18:15:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.556+0000 7f7ae1165640 1 -- 172.21.15.27:0/1530922218 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7acc02f970 con 0x7f7adc0cdf90 2023-12-19T18:15:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.556+0000 7f7ae1165640 1 -- 172.21.15.27:0/1530922218 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7acc02fd00 con 0x7f7adc0cdf90 2023-12-19T18:15:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.557+0000 7f7ae43cb640 1 -- 172.21.15.27:0/1530922218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7adc0cdf90 msgr2=0x7f7adc0ce420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.557+0000 7f7ae43cb640 1 --2- 172.21.15.27:0/1530922218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7adc0cdf90 0x7f7adc0ce420 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7acc0097a0 tx=0x7f7acc02b9f0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.557+0000 7f7ae43cb640 1 -- 172.21.15.27:0/1530922218 shutdown_connections 2023-12-19T18:15:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.557+0000 7f7ae43cb640 1 --2- 172.21.15.27:0/1530922218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7adc0cdf90 0x7f7adc0ce420 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.557+0000 7f7ae43cb640 1 -- 172.21.15.27:0/1530922218 >> 172.21.15.27:0/1530922218 conn(0x7f7adc0d31e0 msgr2=0x7f7adc0d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:15.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.557+0000 7f7ae43cb640 1 -- 172.21.15.27:0/1530922218 shutdown_connections 2023-12-19T18:15:15.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.557+0000 7f7ae43cb640 1 -- 172.21.15.27:0/1530922218 wait complete. 2023-12-19T18:15:15.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.559+0000 7f7ae43cb640 1 Processor -- start 2023-12-19T18:15:15.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.559+0000 7f7ae43cb640 1 -- start start 2023-12-19T18:15:15.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.560+0000 7f7ae43cb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7adc0cdf90 0x7f7adc14c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:15.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.560+0000 7f7ae43cb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7adc0d1e00 0x7f7adc14bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:15.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.560+0000 7f7ae43cb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7adc15d240 0x7f7adc1510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:15.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.560+0000 7f7ae43cb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc1f63e0 con 0x7f7adc15d240 2023-12-19T18:15:15.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.560+0000 7f7ae43cb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc1f66c0 con 0x7f7adc0cdf90 2023-12-19T18:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.560+0000 7f7ae43cb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7adc1f69a0 con 0x7f7adc0d1e00 2023-12-19T18:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.561+0000 7f7ae2968640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7adc15d240 0x7f7adc1510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.561+0000 7f7ae2968640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7adc15d240 0x7f7adc1510d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60948/0 (socket says 172.21.15.27:60948) 2023-12-19T18:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.561+0000 7f7ae2968640 1 -- 172.21.15.27:0/593165603 learned_addr learned my addr 172.21.15.27:0/593165603 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.561+0000 7f7ae2167640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7adc0cdf90 0x7f7adc14c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.561+0000 7f7ae1966640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7adc0d1e00 0x7f7adc14bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.561+0000 7f7ae2968640 1 -- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7adc0d1e00 msgr2=0x7f7adc14bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.561+0000 7f7ae2968640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7adc0d1e00 0x7f7adc14bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.561+0000 7f7ae2968640 1 -- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7adc0cdf90 msgr2=0x7f7adc14c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.561+0000 7f7ae2968640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7adc0cdf90 0x7f7adc14c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.561+0000 7f7ae2968640 1 -- 172.21.15.27:0/593165603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7acc009400 con 0x7f7adc15d240 2023-12-19T18:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.561+0000 7f7ae1966640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7adc0d1e00 0x7f7adc14bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.561+0000 7f7ae2968640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7adc15d240 0x7f7adc1510d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400c450 tx=0x7f7ad400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.562+0000 7f7ad2ffd640 1 -- 172.21.15.27:0/593165603 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad40074e0 con 0x7f7adc15d240 2023-12-19T18:15:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.562+0000 7f7ad2ffd640 1 -- 172.21.15.27:0/593165603 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ad4007660 con 0x7f7adc15d240 2023-12-19T18:15:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.562+0000 7f7ad2ffd640 1 -- 172.21.15.27:0/593165603 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ad4004a20 con 0x7f7adc15d240 2023-12-19T18:15:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.562+0000 7f7ae43cb640 1 -- 172.21.15.27:0/593165603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7adc1f6c80 con 0x7f7adc15d240 2023-12-19T18:15:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.562+0000 7f7ae43cb640 1 -- 172.21.15.27:0/593165603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7adc14c260 con 0x7f7adc15d240 2023-12-19T18:15:15.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.563+0000 7f7ae43cb640 1 -- 172.21.15.27:0/593165603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7adc152e70 con 0x7f7adc15d240 2023-12-19T18:15:15.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.564+0000 7f7ad2ffd640 1 -- 172.21.15.27:0/593165603 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7ad4004b80 con 0x7f7adc15d240 2023-12-19T18:15:15.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.565+0000 7f7ad2ffd640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7ac006a430 0x7f7ac006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:15.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.565+0000 7f7ad2ffd640 1 -- 172.21.15.27:0/593165603 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ad4087aa0 con 0x7f7adc15d240 2023-12-19T18:15:15.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.566+0000 7f7ae2167640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7ac006a430 0x7f7ac006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:15.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.567+0000 7f7ae2167640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7ac006a430 0x7f7ac006c8f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7adc155de0 tx=0x7f7ad8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:15.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.570+0000 7f7ad2ffd640 1 -- 172.21.15.27:0/593165603 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7adc152e70 con 0x7f7adc15d240 2023-12-19T18:15:15.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.801+0000 7f7ae43cb640 1 -- 172.21.15.27:0/593165603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7adc152e70 con 0x7f7adc15d240 2023-12-19T18:15:15.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.803+0000 7f7ad2ffd640 1 -- 172.21.15.27:0/593165603 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f7adc152e70 con 0x7f7adc15d240 2023-12-19T18:15:15.803 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:15.803 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3679393226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3679393226},{"type":"v1","addr":"172.21.15.27:6835","nonce":3679393226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/809712436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":809712436},{"type":"v1","addr":"172.21.15.192:6835","nonce":809712436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3400689212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3400689212},{"type":"v1","addr":"172.21.15.27:6833","nonce":3400689212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:54.035577+0000","modified":"2023-12-19T18:14:57.547613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/1878426383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1878426383},{"type":"v1","addr":"172.21.15.192:6837","nonce":1878426383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T18:15:15.803 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2023-12-19T18:15:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.805+0000 7f7ae43cb640 1 -- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7ac006a430 msgr2=0x7f7ac006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.805+0000 7f7ae43cb640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7ac006a430 0x7f7ac006c8f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7adc155de0 tx=0x7f7ad8009040 comp rx=0 tx=0).stop 2023-12-19T18:15:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.805+0000 7f7ae43cb640 1 -- 172.21.15.27:0/593165603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7adc15d240 msgr2=0x7f7adc1510d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:15.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.805+0000 7f7ae43cb640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7adc15d240 0x7f7adc1510d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7ad400c450 tx=0x7f7ad400c910 comp rx=0 tx=0).stop 2023-12-19T18:15:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.805+0000 7f7ae43cb640 1 -- 172.21.15.27:0/593165603 shutdown_connections 2023-12-19T18:15:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.805+0000 7f7ae43cb640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7ac006a430 0x7f7ac006c8f0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.805+0000 7f7ae43cb640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7adc15d240 0x7f7adc1510d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.805+0000 7f7ae43cb640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7adc0d1e00 0x7f7adc14bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.805+0000 7f7ae43cb640 1 --2- 172.21.15.27:0/593165603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7adc0cdf90 0x7f7adc14c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.805+0000 7f7ae43cb640 1 -- 172.21.15.27:0/593165603 >> 172.21.15.27:0/593165603 conn(0x7f7adc0d31e0 msgr2=0x7f7adc0b7c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.806+0000 7f7ae43cb640 1 -- 172.21.15.27:0/593165603 shutdown_connections 2023-12-19T18:15:15.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:15.806+0000 7f7ae43cb640 1 -- 172.21.15.27:0/593165603 wait complete. 2023-12-19T18:15:15.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:16.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.058+0000 7fa9bd937640 1 Processor -- start 2023-12-19T18:15:16.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.058+0000 7fa9bd937640 1 -- start start 2023-12-19T18:15:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.060+0000 7fa9bd937640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b80cb300 0x7fa9b80cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.060+0000 7fa9bd937640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa9b80c8f90 0x7fa9b8160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.060+0000 7fa9bd937640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9b81612d0 con 0x7fa9b8160e40 2023-12-19T18:15:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.060+0000 7fa9bd937640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9b80c4480 con 0x7fa9b80c8f90 2023-12-19T18:15:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.060+0000 7fa9bd937640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b81468c0 con 0x7fa9b80cb300 2023-12-19T18:15:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.061+0000 7fa9bd136640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa9b8160e40 0x7fa9b80ac8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49336/0 (socket says 172.21.15.27:49336) 2023-12-19T18:15:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.061+0000 7fa9bd136640 1 -- 172.21.15.27:0/1311170085 learned_addr learned my addr 172.21.15.27:0/1311170085 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.061+0000 7fa9b7fff640 1 --2- 172.21.15.27:0/1311170085 >> v2:172.21.15.192:3300/0 conn(0x7fa9b80c8f90 0x7fa9b8160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.061+0000 7fa9bc935640 1 --2- 172.21.15.27:0/1311170085 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b80cb300 0x7fa9b80cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.061+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345307000 0 0) 0x7fa9b0001490 con 0x7fa9b8160e40 2023-12-19T18:15:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.061+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9a8003520 con 0x7fa9b8160e40 2023-12-19T18:15:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 355178062 0 0) 0x7fa9a8003520 con 0x7fa9b8160e40 2023-12-19T18:15:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b80cb300 msgr2=0x7fa9b80cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 --2- 172.21.15.27:0/1311170085 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b80cb300 0x7fa9b80cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 >> v2:172.21.15.192:3300/0 conn(0x7fa9b80c8f90 msgr2=0x7fa9b8160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 --2- 172.21.15.27:0/1311170085 >> v2:172.21.15.192:3300/0 conn(0x7fa9b80c8f90 0x7fa9b8160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b80cbc20 con 0x7fa9b8160e40 2023-12-19T18:15:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fa9b0001b80 con 0x7fa9b8160e40 2023-12-19T18:15:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 >> v1:172.21.15.27:6789/0 conn(0x7fa9b8160e40 legacy=0x7fa9b80ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 --2- 172.21.15.27:0/1311170085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9a8003970 0x7fa9a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 --2- 172.21.15.27:0/1311170085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a80047c0 0x7fa9a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 --2- 172.21.15.27:0/1311170085 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b80cb300 0x7fa9a8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b81468c0 con 0x7fa9a8003970 2023-12-19T18:15:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a800b980 con 0x7fa9a80047c0 2023-12-19T18:15:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a800bac0 con 0x7fa9b80cb300 2023-12-19T18:15:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9bc935640 1 --2- 172.21.15.27:0/1311170085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9a8003970 0x7fa9a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9b7fff640 1 --2- 172.21.15.27:0/1311170085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a80047c0 0x7fa9a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.062+0000 7fa9bd136640 1 --2- 172.21.15.27:0/1311170085 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b80cb300 0x7fa9a8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.063+0000 7fa9bc935640 1 -- 172.21.15.27:0/1311170085 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b80cb300 msgr2=0x7fa9a8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.063+0000 7fa9bc935640 1 --2- 172.21.15.27:0/1311170085 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b80cb300 0x7fa9a8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.063+0000 7fa9bc935640 1 -- 172.21.15.27:0/1311170085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a80047c0 msgr2=0x7fa9a8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.063+0000 7fa9bc935640 1 --2- 172.21.15.27:0/1311170085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a80047c0 0x7fa9a8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.063+0000 7fa9bc935640 1 -- 172.21.15.27:0/1311170085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a8003520 con 0x7fa9a8003970 2023-12-19T18:15:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.064+0000 7fa9bc935640 1 --2- 172.21.15.27:0/1311170085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9a8003970 0x7fa9a8003d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009ac0 tx=0x7fa9a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0128edf247ccf61 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.064+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac02c6b0 con 0x7fa9a8003970 2023-12-19T18:15:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.064+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9ac02c810 con 0x7fa9a8003970 2023-12-19T18:15:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.064+0000 7fa9b77fe640 1 -- 172.21.15.27:0/1311170085 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac02cba0 con 0x7fa9a8003970 2023-12-19T18:15:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.064+0000 7fa9bd937640 1 -- 172.21.15.27:0/1311170085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9a8003970 msgr2=0x7fa9a8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.064+0000 7fa9bd937640 1 --2- 172.21.15.27:0/1311170085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9a8003970 0x7fa9a8003d90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac009ac0 tx=0x7fa9a80057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.064+0000 7fa9bd136640 1 -- 172.21.15.27:0/1311170085 reap_dead start 2023-12-19T18:15:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.065+0000 7fa9bd937640 1 -- 172.21.15.27:0/1311170085 shutdown_connections 2023-12-19T18:15:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.065+0000 7fa9bd937640 1 -- 172.21.15.27:0/1311170085 >> 172.21.15.27:0/1311170085 conn(0x7fa9b8069c70 msgr2=0x7fa9b806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.065+0000 7fa9bd937640 1 -- 172.21.15.27:0/1311170085 shutdown_connections 2023-12-19T18:15:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.065+0000 7fa9bd937640 1 -- 172.21.15.27:0/1311170085 wait complete. 2023-12-19T18:15:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.066+0000 7fa9bd937640 1 Processor -- start 2023-12-19T18:15:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.067+0000 7fa9bd937640 1 -- start start 2023-12-19T18:15:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9bd937640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a8003970 0x7fa9b8142530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9bd937640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9b8144c10 0x7fa9b8142c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9bd937640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b81431b0 0x7fa9b80c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9bd937640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b8142a70 con 0x7fa9b8144c10 2023-12-19T18:15:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9bd937640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b80c5f60 con 0x7fa9a8003970 2023-12-19T18:15:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9bd937640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9b80c6200 con 0x7fa9b81431b0 2023-12-19T18:15:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9b7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9b8144c10 0x7fa9b8142c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9b7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9b8144c10 0x7fa9b8142c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60980/0 (socket says 172.21.15.27:60980) 2023-12-19T18:15:16.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9b7fff640 1 -- 172.21.15.27:0/4284835841 learned_addr learned my addr 172.21.15.27:0/4284835841 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:16.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9bc935640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a8003970 0x7fa9b8142530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9bd136640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b81431b0 0x7fa9b80c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9b7fff640 1 -- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b81431b0 msgr2=0x7fa9b80c7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:16.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9b7fff640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b81431b0 0x7fa9b80c7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9b7fff640 1 -- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a8003970 msgr2=0x7fa9b8142530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:16.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9b7fff640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a8003970 0x7fa9b8142530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9b7fff640 1 -- 172.21.15.27:0/4284835841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9b80b2ce0 con 0x7fa9b8144c10 2023-12-19T18:15:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9bc935640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a8003970 0x7fa9b8142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.068+0000 7fa9bd136640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b81431b0 0x7fa9b80c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.069+0000 7fa9b7fff640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9b8144c10 0x7fa9b8142c70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40116d0 tx=0x7fa9a4011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.069+0000 7fa9b57fa640 1 -- 172.21.15.27:0/4284835841 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400f9c0 con 0x7fa9b8144c10 2023-12-19T18:15:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.069+0000 7fa9b57fa640 1 -- 172.21.15.27:0/4284835841 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a400fb20 con 0x7fa9b8144c10 2023-12-19T18:15:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.069+0000 7fa9b57fa640 1 -- 172.21.15.27:0/4284835841 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a400fe90 con 0x7fa9b8144c10 2023-12-19T18:15:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.069+0000 7fa9bd937640 1 -- 172.21.15.27:0/4284835841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9b80b2fc0 con 0x7fa9b8144c10 2023-12-19T18:15:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.069+0000 7fa9bd937640 1 -- 172.21.15.27:0/4284835841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9b80b3580 con 0x7fa9b8144c10 2023-12-19T18:15:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.071+0000 7fa9bd937640 1 -- 172.21.15.27:0/4284835841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9b804e840 con 0x7fa9b8144c10 2023-12-19T18:15:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.071+0000 7fa9b57fa640 1 -- 172.21.15.27:0/4284835841 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa9a4015460 con 0x7fa9b8144c10 2023-12-19T18:15:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.072+0000 7fa9b57fa640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa99c06a430 0x7fa99c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.073+0000 7fa9b57fa640 1 -- 172.21.15.27:0/4284835841 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa9a4088d70 con 0x7fa9b8144c10 2023-12-19T18:15:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.073+0000 7fa9bc935640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa99c06a430 0x7fa99c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.074+0000 7fa9bc935640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa99c06a430 0x7fa99c06c8f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac036000 tx=0x7fa9ac030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:16.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.077+0000 7fa9b57fa640 1 -- 172.21.15.27:0/4284835841 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa9a40543f0 con 0x7fa9b8144c10 2023-12-19T18:15:16.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.270+0000 7fa9bd937640 1 -- 172.21.15.27:0/4284835841 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9b80cc5a0 con 0x7fa9b8144c10 2023-12-19T18:15:16.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.271+0000 7fa9b57fa640 1 -- 172.21.15.27:0/4284835841 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16041 (secure 0 0 0) 0x7fa9a4057a10 con 0x7fa9b8144c10 2023-12-19T18:15:16.272 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:16.272 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:55.039690+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.274+0000 7fa9bd937640 1 -- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa99c06a430 msgr2=0x7fa99c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.274+0000 7fa9bd937640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa99c06a430 0x7fa99c06c8f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac036000 tx=0x7fa9ac030000 comp rx=0 tx=0).stop 2023-12-19T18:15:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.274+0000 7fa9bd937640 1 -- 172.21.15.27:0/4284835841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9b8144c10 msgr2=0x7fa9b8142c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.274+0000 7fa9bd937640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9b8144c10 0x7fa9b8142c70 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa9a40116d0 tx=0x7fa9a4011b90 comp rx=0 tx=0).stop 2023-12-19T18:15:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.274+0000 7fa9bd937640 1 -- 172.21.15.27:0/4284835841 shutdown_connections 2023-12-19T18:15:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.274+0000 7fa9bd937640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa99c06a430 0x7fa99c06c8f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.274+0000 7fa9bd937640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9b81431b0 0x7fa9b80c7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.274+0000 7fa9bd937640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa9b8144c10 0x7fa9b8142c70 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.274+0000 7fa9bd937640 1 --2- 172.21.15.27:0/4284835841 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9a8003970 0x7fa9b8142530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.274+0000 7fa9bd937640 1 -- 172.21.15.27:0/4284835841 >> 172.21.15.27:0/4284835841 conn(0x7fa9b8069c70 msgr2=0x7fa9b80ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:16.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.274+0000 7fa9bd937640 1 -- 172.21.15.27:0/4284835841 shutdown_connections 2023-12-19T18:15:16.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.275+0000 7fa9bd937640 1 -- 172.21.15.27:0/4284835841 wait complete. 2023-12-19T18:15:16.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:15:16.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.519+0000 7f0af7fff640 1 Processor -- start 2023-12-19T18:15:16.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.520+0000 7f0af7fff640 1 -- start start 2023-12-19T18:15:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.521+0000 7f0af7fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0af80d2350 0x7f0af80b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.521+0000 7f0af7fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af80b5e90 con 0x7f0af81604d0 2023-12-19T18:15:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.521+0000 7f0af7fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af80d2850 con 0x7f0af80b4ef0 2023-12-19T18:15:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.521+0000 7f0af7fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af80d2b30 con 0x7f0af80d2350 2023-12-19T18:15:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.521+0000 7f0af6ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0af81604d0 0x7f0af80b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49348/0 (socket says 172.21.15.27:49348) 2023-12-19T18:15:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.521+0000 7f0af6ffd640 1 -- 172.21.15.27:0/3703230218 learned_addr learned my addr 172.21.15.27:0/3703230218 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:16.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.521+0000 7f0af77fe640 1 --2- 172.21.15.27:0/3703230218 >> v2:172.21.15.192:3300/0 conn(0x7f0af80d2350 0x7f0af80b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.521+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1585064846 0 0) 0x7f0af80b5e90 con 0x7f0af81604d0 2023-12-19T18:15:16.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.521+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae4003520 con 0x7f0af81604d0 2023-12-19T18:15:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3022532815 0 0) 0x7f0ae4003520 con 0x7f0af81604d0 2023-12-19T18:15:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 >> v2:172.21.15.192:3300/0 conn(0x7f0af80d2350 msgr2=0x7f0af80b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af67fc640 1 --2- 172.21.15.27:0/3703230218 >> v2:172.21.15.192:3300/0 conn(0x7f0af80d2350 0x7f0af80b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 >> v1:172.21.15.192:6789/0 conn(0x7f0af80b4ef0 legacy=0x7f0af80b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af80cded0 con 0x7f0af81604d0 2023-12-19T18:15:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f0ae8001670 con 0x7f0af81604d0 2023-12-19T18:15:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 >> v1:172.21.15.27:6789/0 conn(0x7f0af81604d0 legacy=0x7f0af80b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af67fc640 1 --2- 172.21.15.27:0/3703230218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ae4003970 0x7f0ae4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af67fc640 1 --2- 172.21.15.27:0/3703230218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae40047c0 0x7f0ae4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af67fc640 1 --2- 172.21.15.27:0/3703230218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae4005180 0x7f0ae400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80d2b30 con 0x7f0ae4003970 2023-12-19T18:15:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae400c020 con 0x7f0ae40047c0 2023-12-19T18:15:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae400c1e0 con 0x7f0ae4005180 2023-12-19T18:15:16.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.522+0000 7f0af6ffd640 1 --2- 172.21.15.27:0/3703230218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ae4003970 0x7f0ae4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.523+0000 7f0aeffff640 1 --2- 172.21.15.27:0/3703230218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae40047c0 0x7f0ae4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.523+0000 7f0af77fe640 1 --2- 172.21.15.27:0/3703230218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae4005180 0x7f0ae400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.523+0000 7f0af6ffd640 1 -- 172.21.15.27:0/3703230218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae4005180 msgr2=0x7f0ae400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:16.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.523+0000 7f0af6ffd640 1 --2- 172.21.15.27:0/3703230218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae4005180 0x7f0ae400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.523+0000 7f0af6ffd640 1 -- 172.21.15.27:0/3703230218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae40047c0 msgr2=0x7f0ae4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:16.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.523+0000 7f0af6ffd640 1 --2- 172.21.15.27:0/3703230218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae40047c0 0x7f0ae4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.523+0000 7f0af6ffd640 1 -- 172.21.15.27:0/3703230218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae4003520 con 0x7f0ae4003970 2023-12-19T18:15:16.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.523+0000 7f0af77fe640 1 -- 172.21.15.27:0/3703230218 reap_dead start 2023-12-19T18:15:16.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.524+0000 7f0af6ffd640 1 --2- 172.21.15.27:0/3703230218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ae4003970 0x7f0ae4003d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800c240 tx=0x7f0ae40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d5a9cd45a8acf5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:16.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.524+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae8005ad0 con 0x7f0ae4003970 2023-12-19T18:15:16.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.524+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae8005c30 con 0x7f0ae4003970 2023-12-19T18:15:16.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.524+0000 7f0af67fc640 1 -- 172.21.15.27:0/3703230218 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae802cab0 con 0x7f0ae4003970 2023-12-19T18:15:16.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.525+0000 7f0af7fff640 1 -- 172.21.15.27:0/3703230218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ae4003970 msgr2=0x7f0ae4003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:16.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.525+0000 7f0af7fff640 1 --2- 172.21.15.27:0/3703230218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ae4003970 0x7f0ae4003d90 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0ae800c240 tx=0x7f0ae40057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.525+0000 7f0af7fff640 1 -- 172.21.15.27:0/3703230218 shutdown_connections 2023-12-19T18:15:16.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.525+0000 7f0af7fff640 1 --2- 172.21.15.27:0/3703230218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ae4003970 0x7f0ae4003d90 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.525+0000 7f0af7fff640 1 -- 172.21.15.27:0/3703230218 >> 172.21.15.27:0/3703230218 conn(0x7f0af80cf7e0 msgr2=0x7f0af80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:16.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.525+0000 7f0af7fff640 1 -- 172.21.15.27:0/3703230218 shutdown_connections 2023-12-19T18:15:16.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.525+0000 7f0af7fff640 1 -- 172.21.15.27:0/3703230218 wait complete. 2023-12-19T18:15:16.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.527+0000 7f0af7fff640 1 Processor -- start 2023-12-19T18:15:16.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.527+0000 7f0af7fff640 1 -- start start 2023-12-19T18:15:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0af7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae4003970 0x7f0af8155440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0af7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af8156380 0x7f0af80c87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0af7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af80c8cf0 0x7f0af8158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0af7fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af815b910 con 0x7f0af8156380 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0af7fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af815ad60 con 0x7f0af80c8cf0 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0af7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af80c9230 con 0x7f0ae4003970 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0aeffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af8156380 0x7f0af80c87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0aeffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af8156380 0x7f0af80c87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32772/0 (socket says 172.21.15.27:32772) 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0aeffff640 1 -- 172.21.15.27:0/3699286670 learned_addr learned my addr 172.21.15.27:0/3699286670 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0af6ffd640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae4003970 0x7f0af8155440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0af77fe640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af80c8cf0 0x7f0af8158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0aeffff640 1 -- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae4003970 msgr2=0x7f0af8155440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0aeffff640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae4003970 0x7f0af8155440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0aeffff640 1 -- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af80c8cf0 msgr2=0x7f0af8158910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0aeffff640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af80c8cf0 0x7f0af8158910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0aeffff640 1 -- 172.21.15.27:0/3699286670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af815af70 con 0x7f0af8156380 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0af77fe640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af80c8cf0 0x7f0af8158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.528+0000 7f0af6ffd640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae4003970 0x7f0af8155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.529+0000 7f0aeffff640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af8156380 0x7f0af80c87b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000e410 tx=0x7f0ae000e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.529+0000 7f0af4ff9640 1 -- 172.21.15.27:0/3699286670 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00052e0 con 0x7f0af8156380 2023-12-19T18:15:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.529+0000 7f0af4ff9640 1 -- 172.21.15.27:0/3699286670 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae0005440 con 0x7f0af8156380 2023-12-19T18:15:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.529+0000 7f0af7fff640 1 -- 172.21.15.27:0/3699286670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af815b250 con 0x7f0af8156380 2023-12-19T18:15:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.529+0000 7f0af4ff9640 1 -- 172.21.15.27:0/3699286670 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae00037f0 con 0x7f0af8156380 2023-12-19T18:15:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.529+0000 7f0af7fff640 1 -- 172.21.15.27:0/3699286670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af80c7a80 con 0x7f0af8156380 2023-12-19T18:15:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.531+0000 7f0af7fff640 1 -- 172.21.15.27:0/3699286670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0af8066f80 con 0x7f0af8156380 2023-12-19T18:15:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.531+0000 7f0af4ff9640 1 -- 172.21.15.27:0/3699286670 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0ae0003950 con 0x7f0af8156380 2023-12-19T18:15:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.532+0000 7f0af4ff9640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0ad806a460 0x7f0ad806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.533+0000 7f0af4ff9640 1 -- 172.21.15.27:0/3699286670 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ae0087f70 con 0x7f0af8156380 2023-12-19T18:15:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.533+0000 7f0af6ffd640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0ad806a460 0x7f0ad806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:16.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.534+0000 7f0af6ffd640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0ad806a460 0x7f0ad806c920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8034000 tx=0x7f0ae8009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:16.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.537+0000 7f0af4ff9640 1 -- 172.21.15.27:0/3699286670 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0ae00baca0 con 0x7f0af8156380 2023-12-19T18:15:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.776+0000 7f0af7fff640 1 -- 172.21.15.27:0/3699286670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0af804e840 con 0x7f0af8156380 2023-12-19T18:15:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.777+0000 7f0af4ff9640 1 -- 172.21.15.27:0/3699286670 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f0ae00536a0 con 0x7f0af8156380 2023-12-19T18:15:16.777 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:16.778 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3679393226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3679393226},{"type":"v1","addr":"172.21.15.27:6835","nonce":3679393226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/809712436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":809712436},{"type":"v1","addr":"172.21.15.192:6835","nonce":809712436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3400689212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3400689212},{"type":"v1","addr":"172.21.15.27:6833","nonce":3400689212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:54.035577+0000","modified":"2023-12-19T18:14:57.547613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/1878426383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1878426383},{"type":"v1","addr":"172.21.15.192:6837","nonce":1878426383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T18:15:16.778 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2023-12-19T18:15:16.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.780+0000 7f0af7fff640 1 -- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0ad806a460 msgr2=0x7f0ad806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:16.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.780+0000 7f0af7fff640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0ad806a460 0x7f0ad806c920 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0ae8034000 tx=0x7f0ae8009030 comp rx=0 tx=0).stop 2023-12-19T18:15:16.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.780+0000 7f0af7fff640 1 -- 172.21.15.27:0/3699286670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af8156380 msgr2=0x7f0af80c87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:16.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.780+0000 7f0af7fff640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af8156380 0x7f0af80c87b0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f0ae000e410 tx=0x7f0ae000e8d0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.780+0000 7f0af7fff640 1 -- 172.21.15.27:0/3699286670 shutdown_connections 2023-12-19T18:15:16.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.780+0000 7f0af7fff640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0ad806a460 0x7f0ad806c920 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.780+0000 7f0af7fff640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0af80c8cf0 0x7f0af8158910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.780+0000 7f0af7fff640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af8156380 0x7f0af80c87b0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.780+0000 7f0af7fff640 1 --2- 172.21.15.27:0/3699286670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae4003970 0x7f0af8155440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:16.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.780+0000 7f0af7fff640 1 -- 172.21.15.27:0/3699286670 >> 172.21.15.27:0/3699286670 conn(0x7f0af80cf7e0 msgr2=0x7f0af80c9ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:16.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.780+0000 7f0af7fff640 1 -- 172.21.15.27:0/3699286670 shutdown_connections 2023-12-19T18:15:16.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:16.780+0000 7f0af7fff640 1 -- 172.21.15.27:0/3699286670 wait complete. 2023-12-19T18:15:17.202 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:14:54.035577+0000', 'modified': '2023-12-19T18:14:57.547613+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4267}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4267': {'gid': 4267, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.192:6837/1878426383', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.192:6836', 'nonce': 1878426383}, {'type': 'v1', 'addr': '172.21.15.192:6837', 'nonce': 1878426383}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:15:17.203 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:15:17.203 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-19T18:15:17.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.445+0000 7fcdf759e640 1 Processor -- start 2023-12-19T18:15:17.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.446+0000 7fcdf759e640 1 -- start start 2023-12-19T18:15:17.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.447+0000 7fcdf759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b89b0 0x7fcdf815c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.447+0000 7fcdf759e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf80b5ca0 con 0x7fcdf80b49d0 2023-12-19T18:15:17.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.447+0000 7fcdf759e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf8161800 con 0x7fcdf80b4db0 2023-12-19T18:15:17.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.447+0000 7fcdf759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf8161050 con 0x7fcdf80b89b0 2023-12-19T18:15:17.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.447+0000 7fcdf659c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcdf80b49d0 0x7fcdf8160940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49360/0 (socket says 172.21.15.27:49360) 2023-12-19T18:15:17.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.447+0000 7fcdf659c640 1 -- 172.21.15.27:0/3227732862 learned_addr learned my addr 172.21.15.27:0/3227732862 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf5d9b640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b89b0 0x7fcdf815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2132295498 0 0) 0x7fcdf80b5ca0 con 0x7fcdf80b49d0 2023-12-19T18:15:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde4003520 con 0x7fcdf80b49d0 2023-12-19T18:15:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4061292433 0 0) 0x7fcde4003520 con 0x7fcdf80b49d0 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b89b0 msgr2=0x7fcdf815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b89b0 0x7fcdf815c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 >> v1:172.21.15.192:6789/0 conn(0x7fcdf80b4db0 legacy=0x7fcdf80d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf8161240 con 0x7fcdf80b49d0 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fcde00017f0 con 0x7fcdf80b49d0 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 >> v1:172.21.15.27:6789/0 conn(0x7fcdf80b49d0 legacy=0x7fcdf8160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcde40039a0 0x7fcde4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde40047c0 0x7fcde4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b89b0 0x7fcde4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde400bac0 con 0x7fcde40039a0 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcde400bc30 con 0x7fcde40047c0 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.448+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcde400bdf0 con 0x7fcdf80b89b0 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.449+0000 7fcdf659c640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcde40039a0 0x7fcde4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.449+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b89b0 0x7fcde4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.449+0000 7fcdf5d9b640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde40047c0 0x7fcde4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.449+0000 7fcdf659c640 1 -- 172.21.15.27:0/3227732862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b89b0 msgr2=0x7fcde4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.449+0000 7fcdf659c640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b89b0 0x7fcde4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.449+0000 7fcdf659c640 1 -- 172.21.15.27:0/3227732862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde40047c0 msgr2=0x7fcde4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.449+0000 7fcdf659c640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde40047c0 0x7fcde4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.449+0000 7fcdf659c640 1 -- 172.21.15.27:0/3227732862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde4003520 con 0x7fcde40039a0 2023-12-19T18:15:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.449+0000 7fcdf5d9b640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde40047c0 0x7fcde4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:17.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.449+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80b89b0 0x7fcde4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.450+0000 7fcdf659c640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcde40039a0 0x7fcde4003dc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcde000c160 tx=0x7fcde40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d7491910a646a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.450+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0007270 con 0x7fcde40039a0 2023-12-19T18:15:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.451+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde00073f0 con 0x7fcde40039a0 2023-12-19T18:15:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.451+0000 7fcdf559a640 1 -- 172.21.15.27:0/3227732862 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde002c850 con 0x7fcde40039a0 2023-12-19T18:15:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.451+0000 7fcdf759e640 1 -- 172.21.15.27:0/3227732862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcde40039a0 msgr2=0x7fcde4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.451+0000 7fcdf759e640 1 --2- 172.21.15.27:0/3227732862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcde40039a0 0x7fcde4003dc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fcde000c160 tx=0x7fcde40057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.451+0000 7fcdf6d9d640 1 -- 172.21.15.27:0/3227732862 reap_dead start 2023-12-19T18:15:17.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.451+0000 7fcdf759e640 1 -- 172.21.15.27:0/3227732862 shutdown_connections 2023-12-19T18:15:17.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.451+0000 7fcdf759e640 1 -- 172.21.15.27:0/3227732862 >> 172.21.15.27:0/3227732862 conn(0x7fcdf80cf760 msgr2=0x7fcdf80cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:17.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.451+0000 7fcdf759e640 1 -- 172.21.15.27:0/3227732862 shutdown_connections 2023-12-19T18:15:17.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.451+0000 7fcdf759e640 1 -- 172.21.15.27:0/3227732862 wait complete. 2023-12-19T18:15:17.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.453+0000 7fcdf759e640 1 Processor -- start 2023-12-19T18:15:17.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.453+0000 7fcdf759e640 1 -- start start 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.454+0000 7fcdf759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde40039a0 0x7fcdf8153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.454+0000 7fcdf759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdf8159bc0 0x7fcdf80cc290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.454+0000 7fcdf759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80cc7d0 0x7fcdf815f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.454+0000 7fcdf759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80ccd80 con 0x7fcdf8159bc0 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.454+0000 7fcdf759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf815da70 con 0x7fcde40039a0 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.454+0000 7fcdf759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf815d140 con 0x7fcdf80cc7d0 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.454+0000 7fcdf5d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdf8159bc0 0x7fcdf80cc290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.455+0000 7fcdf5d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdf8159bc0 0x7fcdf80cc290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32790/0 (socket says 172.21.15.27:32790) 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.455+0000 7fcdf5d9b640 1 -- 172.21.15.27:0/1898785187 learned_addr learned my addr 172.21.15.27:0/1898785187 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.455+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80cc7d0 0x7fcdf815f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.455+0000 7fcdf659c640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde40039a0 0x7fcdf8153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.455+0000 7fcdf5d9b640 1 -- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80cc7d0 msgr2=0x7fcdf815f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.455+0000 7fcdf5d9b640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80cc7d0 0x7fcdf815f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.455+0000 7fcdf5d9b640 1 -- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde40039a0 msgr2=0x7fcdf8153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:17.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.455+0000 7fcdf5d9b640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde40039a0 0x7fcdf8153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.455+0000 7fcdf5d9b640 1 -- 172.21.15.27:0/1898785187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf815d420 con 0x7fcdf8159bc0 2023-12-19T18:15:17.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.455+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80cc7d0 0x7fcdf815f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:17.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.455+0000 7fcdf5d9b640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdf8159bc0 0x7fcdf80cc290 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcde800f3b0 tx=0x7fcde800f870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:17.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.456+0000 7fcdeeffd640 1 -- 172.21.15.27:0/1898785187 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8003b90 con 0x7fcdf8159bc0 2023-12-19T18:15:17.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.456+0000 7fcdeeffd640 1 -- 172.21.15.27:0/1898785187 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde8011040 con 0x7fcdf8159bc0 2023-12-19T18:15:17.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.456+0000 7fcdeeffd640 1 -- 172.21.15.27:0/1898785187 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde8004020 con 0x7fcdf8159bc0 2023-12-19T18:15:17.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.456+0000 7fcdf759e640 1 -- 172.21.15.27:0/1898785187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf815d700 con 0x7fcdf8159bc0 2023-12-19T18:15:17.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.456+0000 7fcdf759e640 1 -- 172.21.15.27:0/1898785187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf80cb5d0 con 0x7fcdf8159bc0 2023-12-19T18:15:17.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.458+0000 7fcdeeffd640 1 -- 172.21.15.27:0/1898785187 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcde8004180 con 0x7fcdf8159bc0 2023-12-19T18:15:17.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.459+0000 7fcdeeffd640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdd806a430 0x7fcdd806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.459+0000 7fcdeeffd640 1 -- 172.21.15.27:0/1898785187 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcde8088f70 con 0x7fcdf8159bc0 2023-12-19T18:15:17.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.459+0000 7fcdf659c640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdd806a430 0x7fcdd806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.460+0000 7fcdffc03740 1 Processor -- start 2023-12-19T18:15:17.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.460+0000 7fcdffc03740 1 -- start start 2023-12-19T18:15:17.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.460+0000 7fcdf659c640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdd806a430 0x7fcdd806c8f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcde0034000 tx=0x7fcde0009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:17.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdffc03740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a80749110 0x555a807494f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdffc03740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80719090 0x555a80746380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdffc03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 0x555a807468c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdffc03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a806f4650 con 0x555a8071cc90 2023-12-19T18:15:17.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdffc03740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a80689000 con 0x555a80719090 2023-12-19T18:15:17.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdffc03740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a80686790 con 0x555a80749110 2023-12-19T18:15:17.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdf6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 0x555a807468c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdf6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 0x555a807468c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32796/0 (socket says 172.21.15.27:32796) 2023-12-19T18:15:17.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdf6d9d640 1 -- 172.21.15.27:0/4069392029 learned_addr learned my addr 172.21.15.27:0/4069392029 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:17.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdf659c640 1 --2- 172.21.15.27:0/4069392029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a80749110 0x555a807494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdf5d9b640 1 --2- 172.21.15.27:0/4069392029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80719090 0x555a80746380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdf6d9d640 1 -- 172.21.15.27:0/4069392029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a80749110 msgr2=0x555a807494f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:17.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/4069392029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a80749110 0x555a807494f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdf6d9d640 1 -- 172.21.15.27:0/4069392029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80719090 msgr2=0x555a80746380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:17.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/4069392029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80719090 0x555a80746380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.461+0000 7fcdf6d9d640 1 -- 172.21.15.27:0/4069392029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a80649710 con 0x555a8071cc90 2023-12-19T18:15:17.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.462+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/4069392029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 0x555a807468c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80d0410 tx=0x7fcdf0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfd2b78446d4abc1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:17.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.462+0000 7fcdd7fff640 1 -- 172.21.15.27:0/4069392029 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0005480 con 0x555a8071cc90 2023-12-19T18:15:17.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.462+0000 7fcdd7fff640 1 -- 172.21.15.27:0/4069392029 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf000d040 con 0x555a8071cc90 2023-12-19T18:15:17.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.462+0000 7fcdd7fff640 1 -- 172.21.15.27:0/4069392029 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0003810 con 0x555a8071cc90 2023-12-19T18:15:17.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.463+0000 7fcdffc03740 1 -- 172.21.15.27:0/4069392029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 msgr2=0x555a807468c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:17.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.463+0000 7fcdffc03740 1 --2- 172.21.15.27:0/4069392029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 0x555a807468c0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80d0410 tx=0x7fcdf0007f90 comp rx=0 tx=0).stop 2023-12-19T18:15:17.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.463+0000 7fcdffc03740 1 -- 172.21.15.27:0/4069392029 shutdown_connections 2023-12-19T18:15:17.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.463+0000 7fcdffc03740 1 --2- 172.21.15.27:0/4069392029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 0x555a807468c0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.463+0000 7fcdffc03740 1 --2- 172.21.15.27:0/4069392029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80719090 0x555a80746380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.463+0000 7fcdffc03740 1 --2- 172.21.15.27:0/4069392029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a80749110 0x555a807494f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.463+0000 7fcdffc03740 1 -- 172.21.15.27:0/4069392029 >> 172.21.15.27:0/4069392029 conn(0x555a8071b3a0 msgr2=0x555a80763f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:17.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.463+0000 7fcdffc03740 1 -- 172.21.15.27:0/4069392029 shutdown_connections 2023-12-19T18:15:17.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.463+0000 7fcdffc03740 1 -- 172.21.15.27:0/4069392029 wait complete. 2023-12-19T18:15:17.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.464+0000 7fcdffc03740 1 -- start start 2023-12-19T18:15:17.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.464+0000 7fcdffc03740 1 Processor -- start 2023-12-19T18:15:17.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.465+0000 7fcdffc03740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8071cc90 0x555a8073dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.465+0000 7fcdffc03740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a80749110 0x555a80742260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.465+0000 7fcdffc03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 0x555a8074b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.465+0000 7fcdffc03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a80682080 con 0x555a8074b3a0 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.465+0000 7fcdffc03740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a80623850 con 0x555a8071cc90 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.465+0000 7fcdffc03740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a806234f0 con 0x555a80749110 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.465+0000 7fcdffc03740 10 client.0 _ll_register_callbacks cb 0x555a8067b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.465+0000 7fcdf6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 0x555a8074b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.465+0000 7fcdf6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 0x555a8074b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32802/0 (socket says 172.21.15.27:32802) 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.465+0000 7fcdf6d9d640 1 -- 172.21.15.27:0/1531377511 learned_addr learned my addr 172.21.15.27:0/1531377511 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.465+0000 7fcdf5d9b640 1 --2- 172.21.15.27:0/1531377511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a80749110 0x555a80742260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.465+0000 7fcdf659c640 1 --2- 172.21.15.27:0/1531377511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8071cc90 0x555a8073dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.465+0000 7fcdf6d9d640 1 -- 172.21.15.27:0/1531377511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a80749110 msgr2=0x555a80742260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.466+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/1531377511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a80749110 0x555a80742260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.466+0000 7fcdf6d9d640 1 -- 172.21.15.27:0/1531377511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8071cc90 msgr2=0x555a8073dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.466+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/1531377511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8071cc90 0x555a8073dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.466+0000 7fcdf6d9d640 1 -- 172.21.15.27:0/1531377511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf8161800 con 0x555a8074b3a0 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.466+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/1531377511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 0x555a8074b820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80d0860 tx=0x7fcdf0001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721cfd85d21e9808 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.466+0000 7fcdd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.466+0000 7fcdd57fa640 1 -- 172.21.15.27:0/1531377511 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0013440 con 0x555a8074b3a0 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.466+0000 7fcdd57fa640 1 -- 172.21.15.27:0/1531377511 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdf00135a0 con 0x555a8074b3a0 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.466+0000 7fcdffc03740 1 -- 172.21.15.27:0/1531377511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555a806ac5d0 con 0x555a8074b3a0 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.466+0000 7fcdd57fa640 1 -- 172.21.15.27:0/1531377511 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdf0013760 con 0x555a8074b3a0 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.466+0000 7fcdffc03740 10 client.4662 fetch_fsmap learned FSMap version 8 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.466+0000 7fcdffc03740 1 -- 172.21.15.27:0/1531377511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x555a80680700 con 0x555a8074b3a0 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.467+0000 7fcdd57fa640 1 -- 172.21.15.27:0/1531377511 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555a806ac5d0 con 0x555a8074b3a0 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.467+0000 7fcdd57fa640 1 -- 172.21.15.27:0/1531377511 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcdf00138c0 con 0x555a8074b3a0 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.467+0000 7fcdffc03740 10 client.4662 fetch_fsmap finished waiting for FSMap version 8 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.467+0000 7fcdffc03740 10 client.4662 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.467+0000 7fcdffc03740 20 client.4662 populate_metadata read hostname 'smithi027' 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.467+0000 7fcdffc03740 1 --2- 172.21.15.27:0/1531377511 >> [v2:172.21.15.192:6836/1878426383,v1:172.21.15.192:6837/1878426383] conn(0x555a8077eed0 0x555a807812c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.467+0000 7fcdffc03740 4 client.4662 mds_command: new command op to 4267 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:15:17.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.467+0000 7fcdffc03740 1 -- 172.21.15.27:0/1531377511 --> [v2:172.21.15.192:6836/1878426383,v1:172.21.15.192:6837/1878426383] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555a806748d0 con 0x555a8077eed0 2023-12-19T18:15:17.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.467+0000 7fcdf659c640 1 --2- 172.21.15.27:0/1531377511 >> [v2:172.21.15.192:6836/1878426383,v1:172.21.15.192:6837/1878426383] conn(0x555a8077eed0 0x555a807812c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.469+0000 7fcdf659c640 1 --2- 172.21.15.27:0/1531377511 >> [v2:172.21.15.192:6836/1878426383,v1:172.21.15.192:6837/1878426383] conn(0x555a8077eed0 0x555a807812c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8cb0ed8934e82f2 server_cookie=48fd4316328af7d3 in_seq=0 out_seq=0 2023-12-19T18:15:17.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.469+0000 7fcdd57fa640 10 client.4662 ms_handle_connect on v2:172.21.15.192:6836/1878426383 2023-12-19T18:15:17.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdd57fa640 1 -- 172.21.15.27:0/1531377511 <== mds.0 v2:172.21.15.192:6836/1878426383 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x555a806748d0 con 0x555a8077eed0 2023-12-19T18:15:17.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdd57fa640 10 client.4662 handle_command_reply: tid=0 2023-12-19T18:15:17.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdffc03740 1 client.4662 shutdown 2023-12-19T18:15:17.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdffc03740 1 -- 172.21.15.27:0/1531377511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 msgr2=0x555a8074b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdffc03740 1 --2- 172.21.15.27:0/1531377511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 0x555a8074b820 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80d0860 tx=0x7fcdf0001190 comp rx=0 tx=0).stop 2023-12-19T18:15:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdffc03740 1 -- 172.21.15.27:0/1531377511 shutdown_connections 2023-12-19T18:15:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdffc03740 1 --2- 172.21.15.27:0/1531377511 >> [v2:172.21.15.192:6836/1878426383,v1:172.21.15.192:6837/1878426383] conn(0x555a8077eed0 0x555a807812c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdffc03740 1 --2- 172.21.15.27:0/1531377511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 0x555a8074b820 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdffc03740 1 --2- 172.21.15.27:0/1531377511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a80749110 0x555a80742260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdffc03740 1 --2- 172.21.15.27:0/1531377511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a8071cc90 0x555a8073dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdffc03740 1 -- 172.21.15.27:0/1531377511 >> 172.21.15.27:0/1531377511 conn(0x555a80719090 msgr2=0x555a8071d650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdd57fa640 0 client.4662 ms_handle_reset on v2:172.21.15.192:6836/1878426383 2023-12-19T18:15:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdffc03740 1 -- 172.21.15.27:0/1531377511 shutdown_connections 2023-12-19T18:15:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.471+0000 7fcdffc03740 1 -- 172.21.15.27:0/1531377511 wait complete. 2023-12-19T18:15:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.472+0000 7fcdffc03740 20 client.4662 trim_cache size 0 max 16384 2023-12-19T18:15:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.488+0000 7fcdffc03740 1 Processor -- start 2023-12-19T18:15:17.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.488+0000 7fcdffc03740 1 -- start start 2023-12-19T18:15:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.489+0000 7fcdffc03740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a8071cc90 0x555a80766a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.489+0000 7fcdffc03740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80749110 0x555a8071d190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.489+0000 7fcdffc03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 0x555a8071d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.489+0000 7fcdffc03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a803e9f20 con 0x555a8074b3a0 2023-12-19T18:15:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.489+0000 7fcdffc03740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a8073d5e0 con 0x555a80749110 2023-12-19T18:15:17.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.489+0000 7fcdffc03740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a80741960 con 0x555a8071cc90 2023-12-19T18:15:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.489+0000 7fcdf6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 0x555a8071d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.490+0000 7fcdf6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 0x555a8071d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32804/0 (socket says 172.21.15.27:32804) 2023-12-19T18:15:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.490+0000 7fcdf6d9d640 1 -- 172.21.15.27:0/2196679341 learned_addr learned my addr 172.21.15.27:0/2196679341 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.490+0000 7fcdf659c640 1 --2- 172.21.15.27:0/2196679341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a8071cc90 0x555a80766a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.490+0000 7fcdf5d9b640 1 --2- 172.21.15.27:0/2196679341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80749110 0x555a8071d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.490+0000 7fcdf6d9d640 1 -- 172.21.15.27:0/2196679341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a8071cc90 msgr2=0x555a80766a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.490+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/2196679341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a8071cc90 0x555a80766a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.490+0000 7fcdf6d9d640 1 -- 172.21.15.27:0/2196679341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80749110 msgr2=0x555a8071d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.490+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/2196679341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80749110 0x555a8071d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.490+0000 7fcdf6d9d640 1 -- 172.21.15.27:0/2196679341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555a806b6280 con 0x555a8074b3a0 2023-12-19T18:15:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.490+0000 7fcdf659c640 1 --2- 172.21.15.27:0/2196679341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a8071cc90 0x555a80766a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:17.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.490+0000 7fcdf5d9b640 1 --2- 172.21.15.27:0/2196679341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80749110 0x555a8071d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.490+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/2196679341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 0x555a8071d6d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0001eb0 tx=0x7fcdf0007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebbc37ad751d71a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.491+0000 7fcdecff9640 1 -- 172.21.15.27:0/2196679341 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0013d80 con 0x555a8074b3a0 2023-12-19T18:15:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.491+0000 7fcdecff9640 1 -- 172.21.15.27:0/2196679341 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf0003cf0 con 0x555a8074b3a0 2023-12-19T18:15:17.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.491+0000 7fcdecff9640 1 -- 172.21.15.27:0/2196679341 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf0003e60 con 0x555a8074b3a0 2023-12-19T18:15:17.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.491+0000 7fcdffc03740 1 -- 172.21.15.27:0/2196679341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 msgr2=0x555a8071d6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:17.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.491+0000 7fcdffc03740 1 --2- 172.21.15.27:0/2196679341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 0x555a8071d6d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0001eb0 tx=0x7fcdf0007f90 comp rx=0 tx=0).stop 2023-12-19T18:15:17.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.491+0000 7fcdffc03740 1 -- 172.21.15.27:0/2196679341 shutdown_connections 2023-12-19T18:15:17.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.491+0000 7fcdffc03740 1 --2- 172.21.15.27:0/2196679341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8074b3a0 0x555a8071d6d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.491+0000 7fcdffc03740 1 --2- 172.21.15.27:0/2196679341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80749110 0x555a8071d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.491+0000 7fcdffc03740 1 --2- 172.21.15.27:0/2196679341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a8071cc90 0x555a80766a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.491+0000 7fcdffc03740 1 -- 172.21.15.27:0/2196679341 >> 172.21.15.27:0/2196679341 conn(0x555a80719090 msgr2=0x555a8074c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.491+0000 7fcdffc03740 1 -- 172.21.15.27:0/2196679341 shutdown_connections 2023-12-19T18:15:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.491+0000 7fcdffc03740 1 -- 172.21.15.27:0/2196679341 wait complete. 2023-12-19T18:15:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.492+0000 7fcdffc03740 1 -- start start 2023-12-19T18:15:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.492+0000 7fcdffc03740 1 Processor -- start 2023-12-19T18:15:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.493+0000 7fcdffc03740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 0x555a80746470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.493+0000 7fcdffc03740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80749110 0x555a80740690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.493+0000 7fcdffc03740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a8074b3a0 0x555a8073c770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.493+0000 7fcdffc03740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a807469b0 con 0x555a8071cc90 2023-12-19T18:15:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.493+0000 7fcdffc03740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x555a807410a0 con 0x555a80749110 2023-12-19T18:15:17.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.493+0000 7fcdffc03740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x555a80744a20 con 0x555a8074b3a0 2023-12-19T18:15:17.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.493+0000 7fcdffc03740 10 client.0 _ll_register_callbacks cb 0x555a80742190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:15:17.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.493+0000 7fcdf659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 0x555a80746470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.493+0000 7fcdf659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 0x555a80746470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32820/0 (socket says 172.21.15.27:32820) 2023-12-19T18:15:17.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.493+0000 7fcdf659c640 1 -- 172.21.15.27:0/3659886852 learned_addr learned my addr 172.21.15.27:0/3659886852 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.494+0000 7fcdf5d9b640 1 --2- 172.21.15.27:0/3659886852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80749110 0x555a80740690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.494+0000 7fcdf6d9d640 1 --2- 172.21.15.27:0/3659886852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a8074b3a0 0x555a8073c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.494+0000 7fcdf659c640 1 -- 172.21.15.27:0/3659886852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a8074b3a0 msgr2=0x555a8073c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.494+0000 7fcdf659c640 1 --2- 172.21.15.27:0/3659886852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a8074b3a0 0x555a8073c770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.494+0000 7fcdf659c640 1 -- 172.21.15.27:0/3659886852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80749110 msgr2=0x555a80740690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.494+0000 7fcdf659c640 1 --2- 172.21.15.27:0/3659886852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80749110 0x555a80740690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.494+0000 7fcdf659c640 1 -- 172.21.15.27:0/3659886852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555a80680700 con 0x555a8071cc90 2023-12-19T18:15:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.494+0000 7fcdf659c640 1 --2- 172.21.15.27:0/3659886852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 0x555a80746470 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcde002c4c0 tx=0x7fcde002e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=454e6375a58f2e27 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.494+0000 7fcdd57fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:15:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.494+0000 7fcdd57fa640 1 -- 172.21.15.27:0/3659886852 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0007560 con 0x555a8071cc90 2023-12-19T18:15:17.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.494+0000 7fcdd57fa640 1 -- 172.21.15.27:0/3659886852 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcde0051050 con 0x555a8071cc90 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.494+0000 7fcdffc03740 1 -- 172.21.15.27:0/3659886852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcdf000d040 con 0x555a8071cc90 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.495+0000 7fcdd57fa640 1 -- 172.21.15.27:0/3659886852 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcde0051210 con 0x555a8071cc90 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.495+0000 7fcdffc03740 10 client.4668 fetch_fsmap learned FSMap version 8 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.495+0000 7fcdffc03740 1 -- 172.21.15.27:0/3659886852 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x555a80622a60 con 0x555a8071cc90 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.495+0000 7fcdd57fa640 1 -- 172.21.15.27:0/3659886852 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf000d040 con 0x555a8071cc90 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.495+0000 7fcdd57fa640 1 -- 172.21.15.27:0/3659886852 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcde004d070 con 0x555a8071cc90 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.495+0000 7fcdffc03740 10 client.4668 fetch_fsmap finished waiting for FSMap version 8 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.495+0000 7fcdffc03740 10 client.4668 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.495+0000 7fcdffc03740 20 client.4668 populate_metadata read hostname 'smithi027' 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.495+0000 7fcdffc03740 1 --2- 172.21.15.27:0/3659886852 >> [v2:172.21.15.192:6836/1878426383,v1:172.21.15.192:6837/1878426383] conn(0x555a8074c3c0 0x555a8074c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.495+0000 7fcdffc03740 4 client.4668 mds_command: new command op to 4267 tid=0[{"prefix": "status"}] 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.495+0000 7fcdffc03740 1 -- 172.21.15.27:0/3659886852 --> [v2:172.21.15.192:6836/1878426383,v1:172.21.15.192:6837/1878426383] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555a8067b100 con 0x555a8074c3c0 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.496+0000 7fcdf5d9b640 1 --2- 172.21.15.27:0/3659886852 >> [v2:172.21.15.192:6836/1878426383,v1:172.21.15.192:6837/1878426383] conn(0x555a8074c3c0 0x555a8074c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdf5d9b640 1 --2- 172.21.15.27:0/3659886852 >> [v2:172.21.15.192:6836/1878426383,v1:172.21.15.192:6837/1878426383] conn(0x555a8074c3c0 0x555a8074c7a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a493b3ed281786 server_cookie=ba0d1f677edfd673 in_seq=0 out_seq=0 2023-12-19T18:15:17.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdd57fa640 10 client.4668 ms_handle_connect on v2:172.21.15.192:6836/1878426383 2023-12-19T18:15:17.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdd57fa640 1 -- 172.21.15.27:0/3659886852 <== mds.0 v2:172.21.15.192:6836/1878426383 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x555a8067b100 con 0x555a8074c3c0 2023-12-19T18:15:17.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdd57fa640 10 client.4668 handle_command_reply: tid=0 2023-12-19T18:15:17.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdffc03740 1 client.4668 shutdown 2023-12-19T18:15:17.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdffc03740 1 -- 172.21.15.27:0/3659886852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 msgr2=0x555a80746470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:17.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdffc03740 1 --2- 172.21.15.27:0/3659886852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 0x555a80746470 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fcde002c4c0 tx=0x7fcde002e070 comp rx=0 tx=0).stop 2023-12-19T18:15:17.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdffc03740 1 -- 172.21.15.27:0/3659886852 shutdown_connections 2023-12-19T18:15:17.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdffc03740 1 --2- 172.21.15.27:0/3659886852 >> [v2:172.21.15.192:6836/1878426383,v1:172.21.15.192:6837/1878426383] conn(0x555a8074c3c0 0x555a8074c7a0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdffc03740 1 --2- 172.21.15.27:0/3659886852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x555a8074b3a0 0x555a8073c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdffc03740 1 --2- 172.21.15.27:0/3659886852 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x555a80749110 0x555a80740690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdffc03740 1 --2- 172.21.15.27:0/3659886852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x555a8071cc90 0x555a80746470 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdd57fa640 0 client.4668 ms_handle_reset on v2:172.21.15.192:6836/1878426383 2023-12-19T18:15:17.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.497+0000 7fcdffc03740 1 -- 172.21.15.27:0/3659886852 >> 172.21.15.27:0/3659886852 conn(0x555a80719090 msgr2=0x555a80719d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:17.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.498+0000 7fcdffc03740 1 -- 172.21.15.27:0/3659886852 shutdown_connections 2023-12-19T18:15:17.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.498+0000 7fcdffc03740 1 -- 172.21.15.27:0/3659886852 wait complete. 2023-12-19T18:15:17.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.498+0000 7fcdffc03740 20 client.4668 trim_cache size 0 max 16384 2023-12-19T18:15:17.501 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:15:17.501 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:15:17.501 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-12-19T18:15:17.501 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4267, 2023-12-19T18:15:17.501 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-19T18:15:17.501 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-19T18:15:17.501 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-19T18:15:17.501 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 23.454718961000001, 2023-12-19T18:15:17.501 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 8, 2023-12-19T18:15:17.502 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 20, 2023-12-19T18:15:17.502 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 20, 2023-12-19T18:15:17.502 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 28.516595746 2023-12-19T18:15:17.502 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:15:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.498+0000 7fcdf759e640 1 -- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdd806a430 msgr2=0x7fcdd806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.498+0000 7fcdf759e640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdd806a430 0x7fcdd806c8f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcde0034000 tx=0x7fcde0009030 comp rx=0 tx=0).stop 2023-12-19T18:15:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.498+0000 7fcdf759e640 1 -- 172.21.15.27:0/1898785187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdf8159bc0 msgr2=0x7fcdf80cc290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.498+0000 7fcdf759e640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdf8159bc0 0x7fcdf80cc290 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fcde800f3b0 tx=0x7fcde800f870 comp rx=0 tx=0).stop 2023-12-19T18:15:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.498+0000 7fcdf759e640 1 -- 172.21.15.27:0/1898785187 shutdown_connections 2023-12-19T18:15:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.499+0000 7fcdf759e640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdd806a430 0x7fcdd806c8f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.499+0000 7fcdf759e640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf80cc7d0 0x7fcdf815f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.499+0000 7fcdf759e640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdf8159bc0 0x7fcdf80cc290 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.499+0000 7fcdf759e640 1 --2- 172.21.15.27:0/1898785187 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcde40039a0 0x7fcdf8153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.499+0000 7fcdf759e640 1 -- 172.21.15.27:0/1898785187 >> 172.21.15.27:0/1898785187 conn(0x7fcdf80cf760 msgr2=0x7fcdf806bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.499+0000 7fcdf759e640 1 -- 172.21.15.27:0/1898785187 shutdown_connections 2023-12-19T18:15:17.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:17.499+0000 7fcdf759e640 1 -- 172.21.15.27:0/1898785187 wait complete. 2023-12-19T18:15:17.516 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-12-19T18:15:17.540 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-12-19T18:15:17.540 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-12-19T18:15:17.540 INFO:tasks.ceph_fuse:client.0 config is {} 2023-12-19T18:15:17.541 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-19T18:15:17.541 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:15:17.541 INFO:tasks.ceph_fuse:client.2 config is {} 2023-12-19T18:15:17.541 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-19T18:15:17.541 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-19T18:15:17.541 INFO:tasks.ceph_fuse:client.1 config is {} 2023-12-19T18:15:17.541 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-19T18:15:17.541 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T18:15:17.541 INFO:tasks.ceph_fuse:client.3 config is {} 2023-12-19T18:15:17.541 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-19T18:15:17.541 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-19T18:15:17.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:17.542 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:15:17.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:17.602 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2023-12-19T18:15:17.702 INFO:teuthology.orchestra.run.smithi192.stderr:Cannot find device "ceph-brx" 2023-12-19T18:15:17.706 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:15:17.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:17.707 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:15:17.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:17.775 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2023-12-19T18:15:17.893 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2023-12-19T18:15:17.898 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:15:17.899 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-12-19T18:15:17.899 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:15:18.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.143+0000 7f439ecaf640 1 Processor -- start 2023-12-19T18:15:18.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.143+0000 7f439ecaf640 1 -- start start 2023-12-19T18:15:18.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.144+0000 7f439ecaf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f439815af70 0x7f43980ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:18.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.144+0000 7f439ecaf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f43980cbce0 0x7f43980ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:18.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.144+0000 7f439ecaf640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43980cb850 con 0x7f43980ada50 2023-12-19T18:15:18.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.144+0000 7f439ecaf640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f439815b350 con 0x7f439815af70 2023-12-19T18:15:18.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.144+0000 7f439ecaf640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43980cac40 con 0x7f43980cbce0 2023-12-19T18:15:18.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.144+0000 7f439ca4b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f439815af70 0x7f43980ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:18.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.144+0000 7f439ca4b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f439815af70 0x7f43980ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32824/0 (socket says 172.21.15.27:32824) 2023-12-19T18:15:18.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.144+0000 7f439ca4b640 1 -- 172.21.15.27:0/4060138458 learned_addr learned my addr 172.21.15.27:0/4060138458 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:18.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.145+0000 7f439ca4b640 1 -- 172.21.15.27:0/4060138458 >> v2:172.21.15.192:3300/0 conn(0x7f43980cbce0 msgr2=0x7f43980ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:18.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.145+0000 7f439ca4b640 1 --2- 172.21.15.27:0/4060138458 >> v2:172.21.15.192:3300/0 conn(0x7f43980cbce0 0x7f43980ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:18.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.145+0000 7f439ca4b640 1 -- 172.21.15.27:0/4060138458 >> v1:172.21.15.192:6789/0 conn(0x7f43980ada50 legacy=0x7f43980c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:18.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.145+0000 7f439ca4b640 1 -- 172.21.15.27:0/4060138458 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43980caf20 con 0x7f439815af70 2023-12-19T18:15:18.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.146+0000 7f439ca4b640 1 --2- 172.21.15.27:0/4060138458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439815af70 0x7f43980ca540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f43980cc620 tx=0x7f438802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e5bf68275bbc2ab server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:18.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.146+0000 7f43977fe640 1 -- 172.21.15.27:0/4060138458 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f438802ca80 con 0x7f439815af70 2023-12-19T18:15:18.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.146+0000 7f43977fe640 1 -- 172.21.15.27:0/4060138458 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f438802cbe0 con 0x7f439815af70 2023-12-19T18:15:18.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.146+0000 7f43977fe640 1 -- 172.21.15.27:0/4060138458 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43880343f0 con 0x7f439815af70 2023-12-19T18:15:18.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.147+0000 7f439ecaf640 1 -- 172.21.15.27:0/4060138458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439815af70 msgr2=0x7f43980ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:18.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.147+0000 7f439ecaf640 1 --2- 172.21.15.27:0/4060138458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439815af70 0x7f43980ca540 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f43980cc620 tx=0x7f438802bee0 comp rx=0 tx=0).stop 2023-12-19T18:15:18.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.147+0000 7f439ecaf640 1 -- 172.21.15.27:0/4060138458 shutdown_connections 2023-12-19T18:15:18.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.147+0000 7f439ecaf640 1 --2- 172.21.15.27:0/4060138458 >> v2:172.21.15.192:3300/0 conn(0x7f43980cbce0 0x7f43980ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:18.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.147+0000 7f439ecaf640 1 --2- 172.21.15.27:0/4060138458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439815af70 0x7f43980ca540 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:18.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.147+0000 7f439ecaf640 1 -- 172.21.15.27:0/4060138458 >> 172.21.15.27:0/4060138458 conn(0x7f439806a330 msgr2=0x7f43980b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:18.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.147+0000 7f439ecaf640 1 -- 172.21.15.27:0/4060138458 shutdown_connections 2023-12-19T18:15:18.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.147+0000 7f439ecaf640 1 -- 172.21.15.27:0/4060138458 wait complete. 2023-12-19T18:15:18.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.149+0000 7f439ecaf640 1 Processor -- start 2023-12-19T18:15:18.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.149+0000 7f439ecaf640 1 -- start start 2023-12-19T18:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.150+0000 7f439ecaf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43980ada50 0x7f4398156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.150+0000 7f439ecaf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43980cbce0 0x7f43980cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.150+0000 7f439ecaf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f439815af70 0x7f43980ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.150+0000 7f439ecaf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43980ce780 con 0x7f43980cbce0 2023-12-19T18:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.150+0000 7f439ecaf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43980c5130 con 0x7f439815af70 2023-12-19T18:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.150+0000 7f439ecaf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43980c4800 con 0x7f43980ada50 2023-12-19T18:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.150+0000 7f4397fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43980cbce0 0x7f43980cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:18.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.150+0000 7f4397fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43980cbce0 0x7f43980cdf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32830/0 (socket says 172.21.15.27:32830) 2023-12-19T18:15:18.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.150+0000 7f4397fff640 1 -- 172.21.15.27:0/1436006873 learned_addr learned my addr 172.21.15.27:0/1436006873 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:18.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.151+0000 7f439d24c640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f439815af70 0x7f43980ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:18.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.151+0000 7f439ca4b640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43980ada50 0x7f4398156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:18.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.151+0000 7f4397fff640 1 -- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43980ada50 msgr2=0x7f4398156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:18.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.151+0000 7f4397fff640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43980ada50 0x7f4398156fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:18.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.151+0000 7f4397fff640 1 -- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f439815af70 msgr2=0x7f43980ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:18.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.151+0000 7f4397fff640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f439815af70 0x7f43980ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:18.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.151+0000 7f4397fff640 1 -- 172.21.15.27:0/1436006873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4388009b30 con 0x7f43980cbce0 2023-12-19T18:15:18.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.151+0000 7f4397fff640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43980cbce0 0x7f43980cdf50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f438000b480 tx=0x7f438000b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:18.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.151+0000 7f43957fa640 1 -- 172.21.15.27:0/1436006873 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380010870 con 0x7f43980cbce0 2023-12-19T18:15:18.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.151+0000 7f43957fa640 1 -- 172.21.15.27:0/1436006873 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43800109d0 con 0x7f43980cbce0 2023-12-19T18:15:18.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.151+0000 7f43957fa640 1 -- 172.21.15.27:0/1436006873 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4380010d80 con 0x7f43980cbce0 2023-12-19T18:15:18.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.151+0000 7f439ecaf640 1 -- 172.21.15.27:0/1436006873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43980c4ae0 con 0x7f43980cbce0 2023-12-19T18:15:18.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.152+0000 7f439ecaf640 1 -- 172.21.15.27:0/1436006873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43980ce490 con 0x7f43980cbce0 2023-12-19T18:15:18.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.153+0000 7f439ecaf640 1 -- 172.21.15.27:0/1436006873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4398068e90 con 0x7f43980cbce0 2023-12-19T18:15:18.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.154+0000 7f43957fa640 1 -- 172.21.15.27:0/1436006873 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f438000f3c0 con 0x7f43980cbce0 2023-12-19T18:15:18.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.155+0000 7f43957fa640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f437806a460 0x7f437806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:18.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.155+0000 7f43957fa640 1 -- 172.21.15.27:0/1436006873 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4380088870 con 0x7f43980cbce0 2023-12-19T18:15:18.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.155+0000 7f439ca4b640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f437806a460 0x7f437806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:18.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.156+0000 7f439ca4b640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f437806a460 0x7f437806c920 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4388035000 tx=0x7f4388030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:18.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.159+0000 7f43957fa640 1 -- 172.21.15.27:0/1436006873 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4380057980 con 0x7f43980cbce0 2023-12-19T18:15:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.393+0000 7f439ecaf640 1 -- 172.21.15.27:0/1436006873 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f43980b3850 con 0x7f43980cbce0 2023-12-19T18:15:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.394+0000 7f43957fa640 1 -- 172.21.15.27:0/1436006873 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f438001e060 con 0x7f43980cbce0 2023-12-19T18:15:18.394 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:15:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.396+0000 7f439ecaf640 1 -- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f437806a460 msgr2=0x7f437806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.396+0000 7f439ecaf640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f437806a460 0x7f437806c920 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f4388035000 tx=0x7f4388030000 comp rx=0 tx=0).stop 2023-12-19T18:15:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.396+0000 7f439ecaf640 1 -- 172.21.15.27:0/1436006873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43980cbce0 msgr2=0x7f43980cdf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.396+0000 7f439ecaf640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43980cbce0 0x7f43980cdf50 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f438000b480 tx=0x7f438000b940 comp rx=0 tx=0).stop 2023-12-19T18:15:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.397+0000 7f439ecaf640 1 -- 172.21.15.27:0/1436006873 shutdown_connections 2023-12-19T18:15:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.397+0000 7f439ecaf640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f437806a460 0x7f437806c920 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:18.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.397+0000 7f439ecaf640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f439815af70 0x7f43980ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:18.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.397+0000 7f439ecaf640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43980cbce0 0x7f43980cdf50 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:18.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.397+0000 7f439ecaf640 1 --2- 172.21.15.27:0/1436006873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43980ada50 0x7f4398156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:18.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.397+0000 7f439ecaf640 1 -- 172.21.15.27:0/1436006873 >> 172.21.15.27:0/1436006873 conn(0x7f439806a330 msgr2=0x7f43980af8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:18.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.397+0000 7f439ecaf640 1 -- 172.21.15.27:0/1436006873 shutdown_connections 2023-12-19T18:15:18.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:18.397+0000 7f439ecaf640 1 -- 172.21.15.27:0/1436006873 wait complete. 2023-12-19T18:15:18.417 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:15:18.417 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:15:18.417 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-12-19T18:15:18.417 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:15:18.418 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:15:18.418 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:15:18.418 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:15:18.418 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:15:18.418 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:15:18.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:18.418 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-12-19T18:15:18.469 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:18.470 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-12-19T18:15:18.471 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-19T18:15:18.471 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42322sec preferred_lft 42322sec 2023-12-19T18:15:18.471 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2023-12-19T18:15:18.471 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:18.471 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:15:18.471 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:18.471 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-12-19T18:15:18.472 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-19T18:15:18.472 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:18.472 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:15:18.472 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:15:18.472 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2023-12-19T18:15:18.472 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2023-12-19T18:15:18.472 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2023-12-19T18:15:18.473 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-19T18:15:18.473 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:15:18.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:18.989 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:15:19.081 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-12-19T18:15:19.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:19.085 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-12-19T18:15:19.152 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-12-19T18:15:19.153 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:15:19.153 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-19T18:15:19.153 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-19T18:15:19.153 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:15:19.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:19.153 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:15:19.153 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:15:19.154 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:15:19.154 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:15:19.154 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:15:19.154 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:15:19.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:19.944 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:15:20.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:20.008 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2023-12-19T18:15:20.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:20.068 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:15:20.068 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:15:20.068 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:15:20.068 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-19T18:15:20.068 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:15:20.236 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-19T18:15:20.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:20.236 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:15:20.237 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:15:20.237 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-19T18:15:20.237 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-19T18:15:20.237 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-19T18:15:20.237 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-19T18:15:20.237 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-19T18:15:20.237 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:15:20.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:20.565 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:15:20.565 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:15:20.566 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-12-19T18:15:20.566 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2023-12-19T18:15:20.566 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:15:20.761 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-19T18:15:20.762 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:15:20.762 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:15:20.815 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:15:20.816 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:15:20.816 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:15:20.868 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:15:20.964 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:15:20.964 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:15:21.073 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:15:21.077 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:15:21.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:21.078 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:15:21.139 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:15:21.139 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-19T18:15:21.146 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:15:21.227 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:15:21.228 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:15:21.305 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:15:21.308 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:15:21.308 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:21.309 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:15:21.353 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-12-19T18:15:21.352+0000 7f0c59bfb480 -1 init, newargv = 0x55d03956d770 newargc=15 2023-12-19T18:15:21.353 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[87789]: starting ceph client 2023-12-19T18:15:21.363 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[87789]: starting fuse 2023-12-19T18:15:22.454 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:15:22.572 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:15:22.572 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:15:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:15:22.679 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:15:22.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:22.680 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:15:22.735 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:15:22.735 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-19T18:15:22.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:15:22.736 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:15:22.737 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:15:22.737 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:15:22.737 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:15:22.737 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:15:22.737 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:15:22.737 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:15:22.737 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:15:22.737 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:15:22.737 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:15:22.737 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:15:22.737 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:15:22.930 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.87789.asok 2023-12-19T18:15:23.108 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.87789.asok 2023-12-19T18:15:23.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:23.109 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.87789.asok status 2023-12-19T18:15:23.364 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:15:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:15:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:15:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:15:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:15:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:15:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:15:23.364 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "87789", 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4680, 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 4680 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:15:23.365 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3823208656 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3823208656 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.4680 192.168.0.1:0/3823208656", 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3823208656", 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 8, 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 20, 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:15:23.366 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:15:23.367 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:15:23.380 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:15:23.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.616+0000 7fadbbfff640 1 Processor -- start 2023-12-19T18:15:23.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.616+0000 7fadbbfff640 1 -- start start 2023-12-19T18:15:23.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.617+0000 7fadbbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fadbc15ee30 0x7fadbc0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:23.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.617+0000 7fadbbfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fadbc0afeb0 0x7fadbc0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:23.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.618+0000 7fadbbfff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadbc0cb350 con 0x7fadbc1553a0 2023-12-19T18:15:23.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.618+0000 7fadbbfff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc0cbcf0 con 0x7fadbc15ee30 2023-12-19T18:15:23.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.618+0000 7fadbbfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadbc0cb630 con 0x7fadbc0afeb0 2023-12-19T18:15:23.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.618+0000 7fadba7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fadbc15ee30 0x7fadbc0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:23.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.618+0000 7fadba7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fadbc15ee30 0x7fadbc0c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42828/0 (socket says 172.21.15.27:42828) 2023-12-19T18:15:23.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.618+0000 7fadba7fc640 1 -- 172.21.15.27:0/2253970708 learned_addr learned my addr 172.21.15.27:0/2253970708 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:23.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.618+0000 7fadbb7fe640 1 --2- 172.21.15.27:0/2253970708 >> v2:172.21.15.192:3300/0 conn(0x7fadbc0afeb0 0x7fadbc0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:23.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.618+0000 7fadba7fc640 1 -- 172.21.15.27:0/2253970708 >> v2:172.21.15.192:3300/0 conn(0x7fadbc0afeb0 msgr2=0x7fadbc0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:23.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.618+0000 7fadba7fc640 1 --2- 172.21.15.27:0/2253970708 >> v2:172.21.15.192:3300/0 conn(0x7fadbc0afeb0 0x7fadbc0c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:23.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.618+0000 7fadba7fc640 1 -- 172.21.15.27:0/2253970708 >> v1:172.21.15.192:6789/0 conn(0x7fadbc1553a0 legacy=0x7fadbc15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:23.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.618+0000 7fadba7fc640 1 -- 172.21.15.27:0/2253970708 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadbc0cb910 con 0x7fadbc15ee30 2023-12-19T18:15:23.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.619+0000 7fadba7fc640 1 --2- 172.21.15.27:0/2253970708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadbc15ee30 0x7fadbc0c50a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0af670 tx=0x7fadb002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf223b4664567ac server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:23.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.620+0000 7fadb9ffb640 1 -- 172.21.15.27:0/2253970708 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb00317e0 con 0x7fadbc15ee30 2023-12-19T18:15:23.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.620+0000 7fadb9ffb640 1 -- 172.21.15.27:0/2253970708 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb0031940 con 0x7fadbc15ee30 2023-12-19T18:15:23.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.620+0000 7fadb9ffb640 1 -- 172.21.15.27:0/2253970708 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0031cd0 con 0x7fadbc15ee30 2023-12-19T18:15:23.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.620+0000 7fadbbfff640 1 -- 172.21.15.27:0/2253970708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadbc15ee30 msgr2=0x7fadbc0c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:23.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.620+0000 7fadbbfff640 1 --2- 172.21.15.27:0/2253970708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadbc15ee30 0x7fadbc0c50a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fadbc0af670 tx=0x7fadb002b880 comp rx=0 tx=0).stop 2023-12-19T18:15:23.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.620+0000 7fadbbfff640 1 -- 172.21.15.27:0/2253970708 shutdown_connections 2023-12-19T18:15:23.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.620+0000 7fadbbfff640 1 --2- 172.21.15.27:0/2253970708 >> v2:172.21.15.192:3300/0 conn(0x7fadbc0afeb0 0x7fadbc0c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:23.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.620+0000 7fadbbfff640 1 --2- 172.21.15.27:0/2253970708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadbc15ee30 0x7fadbc0c50a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:23.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.620+0000 7fadbbfff640 1 -- 172.21.15.27:0/2253970708 >> 172.21.15.27:0/2253970708 conn(0x7fadbc069c70 msgr2=0x7fadbc06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:23.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.621+0000 7fadbbfff640 1 -- 172.21.15.27:0/2253970708 shutdown_connections 2023-12-19T18:15:23.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.621+0000 7fadbbfff640 1 -- 172.21.15.27:0/2253970708 wait complete. 2023-12-19T18:15:23.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.622+0000 7fadbbfff640 1 Processor -- start 2023-12-19T18:15:23.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.623+0000 7fadbbfff640 1 -- start start 2023-12-19T18:15:23.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.623+0000 7fadbbfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadbc0afeb0 0x7fadbc1f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:23.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.623+0000 7fadbbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadbc1553a0 0x7fadbc0b3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:23.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadbbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadbc15ee30 0x7fadbc0cd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:23.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadbbfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0b4560 con 0x7fadbc1553a0 2023-12-19T18:15:23.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadbbfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc1f3830 con 0x7fadbc15ee30 2023-12-19T18:15:23.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadbbfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadbc0b42a0 con 0x7fadbc0afeb0 2023-12-19T18:15:23.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadba7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadbc1553a0 0x7fadbc0b3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:23.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadba7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadbc1553a0 0x7fadbc0b3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42844/0 (socket says 172.21.15.27:42844) 2023-12-19T18:15:23.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadba7fc640 1 -- 172.21.15.27:0/1079916325 learned_addr learned my addr 172.21.15.27:0/1079916325 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadbb7fe640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadbc15ee30 0x7fadbc0cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadbaffd640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadbc0afeb0 0x7fadbc1f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadba7fc640 1 -- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadbc0afeb0 msgr2=0x7fadbc1f32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadba7fc640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadbc0afeb0 0x7fadbc1f32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadba7fc640 1 -- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadbc15ee30 msgr2=0x7fadbc0cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadba7fc640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadbc15ee30 0x7fadbc0cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadba7fc640 1 -- 172.21.15.27:0/1079916325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadb00094d0 con 0x7fadbc1553a0 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadbb7fe640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadbc15ee30 0x7fadbc0cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.624+0000 7fadbaffd640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadbc0afeb0 0x7fadbc1f32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.625+0000 7fadba7fc640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadbc1553a0 0x7fadbc0b3d60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fadb0000c00 tx=0x7fadb00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.625+0000 7fada37fe640 1 -- 172.21.15.27:0/1079916325 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0032880 con 0x7fadbc1553a0 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.625+0000 7fada37fe640 1 -- 172.21.15.27:0/1079916325 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadb00329e0 con 0x7fadbc1553a0 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.625+0000 7fada37fe640 1 -- 172.21.15.27:0/1079916325 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadb0032d90 con 0x7fadbc1553a0 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.625+0000 7fadbbfff640 1 -- 172.21.15.27:0/1079916325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadbc0ad870 con 0x7fadbc1553a0 2023-12-19T18:15:23.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.625+0000 7fadbbfff640 1 -- 172.21.15.27:0/1079916325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadbc0ade30 con 0x7fadbc1553a0 2023-12-19T18:15:23.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.626+0000 7fadbbfff640 1 -- 172.21.15.27:0/1079916325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadbc04e840 con 0x7fadbc1553a0 2023-12-19T18:15:23.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.627+0000 7fada37fe640 1 -- 172.21.15.27:0/1079916325 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fadb00453f0 con 0x7fadbc1553a0 2023-12-19T18:15:23.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.628+0000 7fada37fe640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fada806a430 0x7fada806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:23.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.628+0000 7fada37fe640 1 -- 172.21.15.27:0/1079916325 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadb00aac20 con 0x7fadbc1553a0 2023-12-19T18:15:23.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.629+0000 7fadbaffd640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fada806a430 0x7fada806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:23.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.630+0000 7fadbaffd640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fada806a430 0x7fada806c8f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fadac003eb0 tx=0x7fadac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:23.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.633+0000 7fada37fe640 1 -- 172.21.15.27:0/1079916325 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fadb0076290 con 0x7fadbc1553a0 2023-12-19T18:15:23.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.871+0000 7fadbbfff640 1 -- 172.21.15.27:0/1079916325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fadbc0ca860 con 0x7fadbc1553a0 2023-12-19T18:15:23.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.872+0000 7fada37fe640 1 -- 172.21.15.27:0/1079916325 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fadb00798b0 con 0x7fadbc1553a0 2023-12-19T18:15:23.872 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:15:23.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.874+0000 7fadbbfff640 1 -- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fada806a430 msgr2=0x7fada806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:23.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.874+0000 7fadbbfff640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fada806a430 0x7fada806c8f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fadac003eb0 tx=0x7fadac009040 comp rx=0 tx=0).stop 2023-12-19T18:15:23.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.874+0000 7fadbbfff640 1 -- 172.21.15.27:0/1079916325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadbc1553a0 msgr2=0x7fadbc0b3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:23.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.874+0000 7fadbbfff640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadbc1553a0 0x7fadbc0b3d60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fadb0000c00 tx=0x7fadb00046c0 comp rx=0 tx=0).stop 2023-12-19T18:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.874+0000 7fadbbfff640 1 -- 172.21.15.27:0/1079916325 shutdown_connections 2023-12-19T18:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.874+0000 7fadbbfff640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fada806a430 0x7fada806c8f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.874+0000 7fadbbfff640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadbc15ee30 0x7fadbc0cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.874+0000 7fadbbfff640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadbc1553a0 0x7fadbc0b3d60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.874+0000 7fadbbfff640 1 --2- 172.21.15.27:0/1079916325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadbc0afeb0 0x7fadbc1f32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.874+0000 7fadbbfff640 1 -- 172.21.15.27:0/1079916325 >> 172.21.15.27:0/1079916325 conn(0x7fadbc069c70 msgr2=0x7fadbc1592f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.875+0000 7fadbbfff640 1 -- 172.21.15.27:0/1079916325 shutdown_connections 2023-12-19T18:15:23.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:23.875+0000 7fadbbfff640 1 -- 172.21.15.27:0/1079916325 wait complete. 2023-12-19T18:15:23.893 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:15:23.893 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:15:23.893 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-12-19T18:15:23.893 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-12-19T18:15:23.893 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-19T18:15:23.893 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:15:23.893 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:15:23.893 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:15:23.893 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-12-19T18:15:23.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:23.893 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-12-19T18:15:23.948 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:15:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:15:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:15:23.948 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42317sec preferred_lft 42317sec 2023-12-19T18:15:23.949 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::8053:76ff:fe69:c66c/64 scope link 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::781d:d6ff:feb4:7a2d/64 scope link 2023-12-19T18:15:23.950 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:23.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:23.952 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:15:24.051 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-12-19T18:15:24.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:24.056 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-12-19T18:15:24.118 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-12-19T18:15:24.118 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:15:24.118 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-19T18:15:24.118 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-19T18:15:24.118 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:15:24.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:24.119 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:15:24.119 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:15:24.119 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:15:24.119 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:15:24.119 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:15:24.119 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:15:24.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:24.333 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:15:24.390 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:15:24.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:24.392 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2023-12-19T18:15:24.444 INFO:teuthology.orchestra.run.smithi027.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-12-19T18:15:24.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:24.446 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:15:24.446 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:15:24.446 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:15:24.446 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-12-19T18:15:24.446 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:15:24.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:24.607 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-19T18:15:24.713 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:15:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:15:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:15:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:15:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:24.713 INFO:teuthology.orchestra.run.smithi027.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:15:24.713 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether b2:72:2c:04:54:2f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-19T18:15:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-19T18:15:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::b072:2cff:fe04:542f/64 scope link 2023-12-19T18:15:24.714 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:24.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:24.718 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-19T18:15:24.821 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:15:24.821 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:15:24.821 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:15:24.821 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:24.822 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:15:24.822 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:24.822 INFO:teuthology.orchestra.run.smithi027.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:15:24.822 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether b2:72:2c:04:54:2f brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-19T18:15:24.822 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-19T18:15:24.822 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:24.822 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::b072:2cff:fe04:542f/64 scope link 2023-12-19T18:15:24.822 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:24.826 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-12-19T18:15:24.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:24.826 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:15:24.826 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:15:24.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-12-19T18:15:24.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-19T18:15:24.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-12-19T18:15:24.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-12-19T18:15:24.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-12-19T18:15:24.826 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:15:25.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:25.123 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:15:25.123 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:15:25.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.1 up 2023-12-19T18:15:25.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.1 master ceph-brx 2023-12-19T18:15:25.123 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:15:25.306 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-12-19T18:15:25.306 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:15:25.306 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-12-19T18:15:25.361 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-12-19T18:15:25.361 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:15:25.361 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-12-19T18:15:25.418 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:15:25.515 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:15:25.515 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:15:25.613 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:15:25.616 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:15:25.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:25.616 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:15:25.672 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:15:25.673 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [41] 2023-12-19T18:15:25.673 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-12-19T18:15:25.678 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:15:25.767 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:15:25.767 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:15:25.848 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:15:25.851 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:15:25.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:25.851 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:15:25.889 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi027.stderr:2023-12-19T18:15:25.889+0000 7f5c8c236480 -1 init, newargv = 0x55a767f15350 newargc=15 2023-12-19T18:15:25.889 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi027.stderr:ceph-fuse[88352]: starting ceph client 2023-12-19T18:15:25.897 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:15:25.899 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi027.stderr:ceph-fuse[88352]: starting fuse 2023-12-19T18:15:26.998 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:15:27.120 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:15:27.120 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:15:27.229 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:15:27.232 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:15:27.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:27.233 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:15:27.293 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:15:27.294 INFO:teuthology.orchestra.run.smithi027.stdout:42 2023-12-19T18:15:27.294 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41, 42] 2023-12-19T18:15:27.294 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:15:27.294 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:15:27.294 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:15:27.294 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:15:27.294 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:15:27.294 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:15:27.294 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:15:27.294 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:15:27.295 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:15:27.296 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:15:27.296 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:15:27.296 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:15:27.296 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.2")) 2023-12-19T18:15:27.296 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:15:27.493 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.2.88352.asok 2023-12-19T18:15:27.672 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.88352.asok 2023-12-19T18:15:27.672 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:27.672 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.88352.asok status 2023-12-19T18:15:27.933 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:15:27.933 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:15:27.933 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:15:27.933 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:15:27.933 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "2", 2023-12-19T18:15:27.933 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:15:27.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-12-19T18:15:27.933 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "88352", 2023-12-19T18:15:27.933 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:15:27.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4695, 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 4695 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.2:0", 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1516460125 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:15:27.934 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:15:27.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:15:27.935 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.2:0", 2023-12-19T18:15:27.935 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1516460125 2023-12-19T18:15:27.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:15:27.935 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.4695 192.168.0.2:0/1516460125", 2023-12-19T18:15:27.935 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.2:0/1516460125", 2023-12-19T18:15:27.935 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-12-19T18:15:27.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 8, 2023-12-19T18:15:27.935 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 20, 2023-12-19T18:15:27.935 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:15:27.935 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:15:27.935 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:15:27.935 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:15:27.949 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:15:28.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.184+0000 7ff4f759e640 1 Processor -- start 2023-12-19T18:15:28.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.184+0000 7ff4f759e640 1 -- start start 2023-12-19T18:15:28.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.186+0000 7ff4f759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4f815ee30 0x7ff4f80c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:28.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.186+0000 7ff4f759e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff4f80afeb0 0x7ff4f80c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:28.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.186+0000 7ff4f759e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4f80b0320 con 0x7ff4f81553a0 2023-12-19T18:15:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.186+0000 7ff4f759e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4f80cbcf0 con 0x7ff4f80afeb0 2023-12-19T18:15:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.186+0000 7ff4f759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80cb350 con 0x7ff4f815ee30 2023-12-19T18:15:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.187+0000 7ff4effff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4f815ee30 0x7ff4f80c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.187+0000 7ff4f6d9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff4f80afeb0 0x7ff4f80c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.187+0000 7ff4effff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4f815ee30 0x7ff4f80c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:40074/0 (socket says 172.21.15.27:40074) 2023-12-19T18:15:28.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.187+0000 7ff4effff640 1 -- 172.21.15.27:0/2894695056 learned_addr learned my addr 172.21.15.27:0/2894695056 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.188+0000 7ff4f6d9d640 1 -- 172.21.15.27:0/2894695056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4f815ee30 msgr2=0x7ff4f80c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.188+0000 7ff4f6d9d640 1 --2- 172.21.15.27:0/2894695056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4f815ee30 0x7ff4f80c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.188+0000 7ff4f6d9d640 1 -- 172.21.15.27:0/2894695056 >> v1:172.21.15.192:6789/0 conn(0x7ff4f81553a0 legacy=0x7ff4f815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.188+0000 7ff4f6d9d640 1 -- 172.21.15.27:0/2894695056 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f80cb630 con 0x7ff4f80afeb0 2023-12-19T18:15:28.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.188+0000 7ff4effff640 1 --2- 172.21.15.27:0/2894695056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4f815ee30 0x7ff4f80c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:15:28.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.189+0000 7ff4f6d9d640 1 --2- 172.21.15.27:0/2894695056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4f80afeb0 0x7ff4f80c39d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009d90 tx=0x7ff4f002be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=faaea78f563eb141 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.189+0000 7ff4f5d9b640 1 -- 172.21.15.27:0/2894695056 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f002cbc0 con 0x7ff4f80afeb0 2023-12-19T18:15:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.190+0000 7ff4f5d9b640 1 -- 172.21.15.27:0/2894695056 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f002cd20 con 0x7ff4f80afeb0 2023-12-19T18:15:28.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.190+0000 7ff4f5d9b640 1 -- 172.21.15.27:0/2894695056 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0034660 con 0x7ff4f80afeb0 2023-12-19T18:15:28.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.190+0000 7ff4f759e640 1 -- 172.21.15.27:0/2894695056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4f80afeb0 msgr2=0x7ff4f80c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:28.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.190+0000 7ff4f759e640 1 --2- 172.21.15.27:0/2894695056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4f80afeb0 0x7ff4f80c39d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ff4f0009d90 tx=0x7ff4f002be10 comp rx=0 tx=0).stop 2023-12-19T18:15:28.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.191+0000 7ff4f759e640 1 -- 172.21.15.27:0/2894695056 shutdown_connections 2023-12-19T18:15:28.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.191+0000 7ff4f759e640 1 --2- 172.21.15.27:0/2894695056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4f80afeb0 0x7ff4f80c39d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:28.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.191+0000 7ff4f759e640 1 --2- 172.21.15.27:0/2894695056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4f815ee30 0x7ff4f80c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:28.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.191+0000 7ff4f759e640 1 -- 172.21.15.27:0/2894695056 >> 172.21.15.27:0/2894695056 conn(0x7ff4f8069c70 msgr2=0x7ff4f806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:28.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.191+0000 7ff4f759e640 1 -- 172.21.15.27:0/2894695056 shutdown_connections 2023-12-19T18:15:28.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.191+0000 7ff4f759e640 1 -- 172.21.15.27:0/2894695056 wait complete. 2023-12-19T18:15:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.193+0000 7ff4f759e640 1 Processor -- start 2023-12-19T18:15:28.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.193+0000 7ff4f759e640 1 -- start start 2023-12-19T18:15:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.195+0000 7ff4f759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4f80afeb0 0x7ff4f8158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.195+0000 7ff4f759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f81553a0 0x7ff4f80ae260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.195+0000 7ff4f759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4f815ee30 0x7ff4f8156440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.195+0000 7ff4f759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80ade40 con 0x7ff4f81553a0 2023-12-19T18:15:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.195+0000 7ff4f759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80ad1c0 con 0x7ff4f815ee30 2023-12-19T18:15:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.195+0000 7ff4f759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f80ad4a0 con 0x7ff4f80afeb0 2023-12-19T18:15:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.195+0000 7ff4effff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f81553a0 0x7ff4f80ae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:28.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.195+0000 7ff4effff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f81553a0 0x7ff4f80ae260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42850/0 (socket says 172.21.15.27:42850) 2023-12-19T18:15:28.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.195+0000 7ff4effff640 1 -- 172.21.15.27:0/4008869303 learned_addr learned my addr 172.21.15.27:0/4008869303 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:28.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.195+0000 7ff4f6d9d640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4f815ee30 0x7ff4f8156440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:28.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.195+0000 7ff4f659c640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4f80afeb0 0x7ff4f8158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:28.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.196+0000 7ff4effff640 1 -- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4f80afeb0 msgr2=0x7ff4f8158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:28.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.196+0000 7ff4effff640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4f80afeb0 0x7ff4f8158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:28.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.196+0000 7ff4effff640 1 -- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4f815ee30 msgr2=0x7ff4f8156440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:28.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.196+0000 7ff4effff640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4f815ee30 0x7ff4f8156440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:28.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.196+0000 7ff4effff640 1 -- 172.21.15.27:0/4008869303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4e00094d0 con 0x7ff4f81553a0 2023-12-19T18:15:28.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.196+0000 7ff4effff640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f81553a0 0x7ff4f80ae260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000e9b0 tx=0x7ff4e000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:28.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.196+0000 7ff4eeffd640 1 -- 172.21.15.27:0/4008869303 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e000c9a0 con 0x7ff4f81553a0 2023-12-19T18:15:28.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.196+0000 7ff4eeffd640 1 -- 172.21.15.27:0/4008869303 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4e000cb00 con 0x7ff4f81553a0 2023-12-19T18:15:28.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.196+0000 7ff4f759e640 1 -- 172.21.15.27:0/4008869303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4f0009a70 con 0x7ff4f81553a0 2023-12-19T18:15:28.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.196+0000 7ff4eeffd640 1 -- 172.21.15.27:0/4008869303 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4e000ce70 con 0x7ff4f81553a0 2023-12-19T18:15:28.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.197+0000 7ff4f759e640 1 -- 172.21.15.27:0/4008869303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4f80ada60 con 0x7ff4f81553a0 2023-12-19T18:15:28.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.198+0000 7ff4f759e640 1 -- 172.21.15.27:0/4008869303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4f804e840 con 0x7ff4f81553a0 2023-12-19T18:15:28.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.199+0000 7ff4eeffd640 1 -- 172.21.15.27:0/4008869303 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff4e0013470 con 0x7ff4f81553a0 2023-12-19T18:15:28.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.200+0000 7ff4eeffd640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff4d806a430 0x7ff4d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:28.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.200+0000 7ff4eeffd640 1 -- 172.21.15.27:0/4008869303 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff4e0088ef0 con 0x7ff4f81553a0 2023-12-19T18:15:28.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.201+0000 7ff4f659c640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff4d806a430 0x7ff4d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:28.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.202+0000 7ff4f659c640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff4d806a430 0x7ff4d806c8f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8000f60 tx=0x7ff4e8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:28.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.208+0000 7ff4eeffd640 1 -- 172.21.15.27:0/4008869303 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff4e00544d0 con 0x7ff4f81553a0 2023-12-19T18:15:28.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.441+0000 7ff4f759e640 1 -- 172.21.15.27:0/4008869303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff4f80af6a0 con 0x7ff4f81553a0 2023-12-19T18:15:28.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.442+0000 7ff4eeffd640 1 -- 172.21.15.27:0/4008869303 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff4e0017060 con 0x7ff4f81553a0 2023-12-19T18:15:28.442 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:15:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.445+0000 7ff4f759e640 1 -- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff4d806a430 msgr2=0x7ff4d806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.445+0000 7ff4f759e640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff4d806a430 0x7ff4d806c8f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff4e8000f60 tx=0x7ff4e8009040 comp rx=0 tx=0).stop 2023-12-19T18:15:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.445+0000 7ff4f759e640 1 -- 172.21.15.27:0/4008869303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f81553a0 msgr2=0x7ff4f80ae260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.445+0000 7ff4f759e640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f81553a0 0x7ff4f80ae260 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7ff4e000e9b0 tx=0x7ff4e000ee70 comp rx=0 tx=0).stop 2023-12-19T18:15:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.446+0000 7ff4f759e640 1 -- 172.21.15.27:0/4008869303 shutdown_connections 2023-12-19T18:15:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.446+0000 7ff4f759e640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff4d806a430 0x7ff4d806c8f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.446+0000 7ff4f759e640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4f815ee30 0x7ff4f8156440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.446+0000 7ff4f759e640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4f81553a0 0x7ff4f80ae260 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.446+0000 7ff4f759e640 1 --2- 172.21.15.27:0/4008869303 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4f80afeb0 0x7ff4f8158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.446+0000 7ff4f759e640 1 -- 172.21.15.27:0/4008869303 >> 172.21.15.27:0/4008869303 conn(0x7ff4f8069c70 msgr2=0x7ff4f80c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.446+0000 7ff4f759e640 1 -- 172.21.15.27:0/4008869303 shutdown_connections 2023-12-19T18:15:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:28.446+0000 7ff4f759e640 1 -- 172.21.15.27:0/4008869303 wait complete. 2023-12-19T18:15:28.466 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:15:28.466 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:15:28.466 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-19T18:15:28.466 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-19T18:15:28.466 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T18:15:28.467 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:15:28.467 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:15:28.467 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:15:28.467 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:15:28.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:28.467 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-19T18:15:28.527 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:15:28.527 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:15:28.527 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:15:28.527 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:28.527 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:15:28.527 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:28.527 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:15:28.527 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:28.527 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-19T18:15:28.527 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:15:28.527 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:28.527 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-19T18:15:28.528 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:15:28.528 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:28.528 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:15:28.528 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:28.528 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-19T18:15:28.528 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42275sec preferred_lft 42275sec 2023-12-19T18:15:28.528 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-19T18:15:28.528 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:28.529 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-19T18:15:28.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:28.529 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:15:28.529 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:15:28.529 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add name ceph-brx type bridge 2023-12-19T18:15:28.529 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr flush dev ceph-brx 2023-12-19T18:15:28.529 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set ceph-brx up 2023-12-19T18:15:28.530 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-19T18:15:28.530 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:15:28.965 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:28.965 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:15:29.046 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-19T18:15:29.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:29.052 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-19T18:15:29.103 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-19T18:15:29.103 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:15:29.103 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T18:15:29.104 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T18:15:29.104 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:15:29.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:29.105 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:15:29.105 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:15:29.105 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T18:15:29.105 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T18:15:29.105 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T18:15:29.106 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:15:29.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:29.813 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:15:29.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:29.865 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list-id 2023-12-19T18:15:29.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:29.912 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:15:29.912 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:15:29.912 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:15:29.912 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-12-19T18:15:29.912 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:15:30.087 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-12-19T18:15:30.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:30.088 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:15:30.088 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:15:30.088 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-12-19T18:15:30.088 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-19T18:15:30.088 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-12-19T18:15:30.088 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-12-19T18:15:30.088 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-12-19T18:15:30.088 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:15:30.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:30.360 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:15:30.361 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:15:30.361 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-19T18:15:30.361 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set dev brx.0 master ceph-brx 2023-12-19T18:15:30.361 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:15:30.516 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-19T18:15:30.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:15:30.517 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-19T18:15:30.558 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:15:30.558 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:15:30.558 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:15:30.651 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:15:30.748 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:15:30.748 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:15:30.833 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:15:30.835 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:15:30.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:30.836 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:15:30.880 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:15:30.881 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-19T18:15:30.929 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:15:31.007 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:15:31.007 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:15:31.076 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:15:31.079 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:15:31.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:31.080 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:15:31.165 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-19T18:15:31.166+0000 7f4fa1664480 -1 init, newargv = 0x557f8d84a720 newargc=15 2023-12-19T18:15:31.165 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[85132]: starting ceph client 2023-12-19T18:15:31.173 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[85132]: starting fuse 2023-12-19T18:15:32.225 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:15:32.321 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:15:32.321 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:15:32.413 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:15:32.416 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:15:32.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:32.417 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:15:32.472 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:15:32.473 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-12-19T18:15:32.473 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:15:32.473 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-19T18:15:32.473 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-19T18:15:32.473 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-19T18:15:32.473 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-19T18:15:32.473 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:15:32.473 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-19T18:15:32.473 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-19T18:15:32.474 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:15:32.475 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:15:32.475 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-19T18:15:32.475 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-19T18:15:32.649 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.85132.asok 2023-12-19T18:15:32.830 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.85132.asok 2023-12-19T18:15:32.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:32.830 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.85132.asok status 2023-12-19T18:15:33.073 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-19T18:15:33.073 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "85132", 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 4612, 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 4612 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:15:33.074 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3454164070 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3454164070 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.4612 192.168.0.1:0/3454164070", 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/3454164070", 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-19T18:15:33.075 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 8, 2023-12-19T18:15:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 20, 2023-12-19T18:15:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:15:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-19T18:15:33.076 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-19T18:15:33.076 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-19T18:15:33.087 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:15:33.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.329+0000 7f2304eb4640 1 Processor -- start 2023-12-19T18:15:33.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.329+0000 7f2304eb4640 1 -- start start 2023-12-19T18:15:33.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.330+0000 7f2304eb4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23001604d0 0x7f23000b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:33.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.330+0000 7f2304eb4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23000b7310 0x7f23000b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:33.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.330+0000 7f2304eb4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23000b5e90 con 0x7f23000d2350 2023-12-19T18:15:33.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.330+0000 7f2304eb4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23000d2b30 con 0x7f23001604d0 2023-12-19T18:15:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.330+0000 7f2304eb4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23000d2850 con 0x7f23000b7310 2023-12-19T18:15:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.331+0000 7f22ff7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23001604d0 0x7f23000b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.331+0000 7f22ff7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23001604d0 0x7f23000b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58506/0 (socket says 172.21.15.27:58506) 2023-12-19T18:15:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.331+0000 7f22ff7fe640 1 -- 172.21.15.27:0/1650685437 learned_addr learned my addr 172.21.15.27:0/1650685437 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.331+0000 7f22fffff640 1 --2- 172.21.15.27:0/1650685437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23000b7310 0x7f23000b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.331+0000 7f22ff7fe640 1 -- 172.21.15.27:0/1650685437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23000b7310 msgr2=0x7f23000b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.331+0000 7f22ff7fe640 1 --2- 172.21.15.27:0/1650685437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23000b7310 0x7f23000b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.331+0000 7f22ff7fe640 1 -- 172.21.15.27:0/1650685437 >> v1:172.21.15.192:6789/0 conn(0x7f23000d2350 legacy=0x7f23000d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.331+0000 7f22ff7fe640 1 -- 172.21.15.27:0/1650685437 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23000cded0 con 0x7f23001604d0 2023-12-19T18:15:33.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.332+0000 7f22ff7fe640 1 --2- 172.21.15.27:0/1650685437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23001604d0 0x7f23000b47e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009870 tx=0x7f22f002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159c65cd474ef46 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.333+0000 7f22feffd640 1 -- 172.21.15.27:0/1650685437 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0031640 con 0x7f23001604d0 2023-12-19T18:15:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.333+0000 7f22feffd640 1 -- 172.21.15.27:0/1650685437 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22f00317a0 con 0x7f23001604d0 2023-12-19T18:15:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.333+0000 7f22feffd640 1 -- 172.21.15.27:0/1650685437 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f0031b30 con 0x7f23001604d0 2023-12-19T18:15:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.333+0000 7f2304eb4640 1 -- 172.21.15.27:0/1650685437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23001604d0 msgr2=0x7f23000b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.333+0000 7f2304eb4640 1 --2- 172.21.15.27:0/1650685437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23001604d0 0x7f23000b47e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f22f0009870 tx=0x7f22f002b8f0 comp rx=0 tx=0).stop 2023-12-19T18:15:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.333+0000 7f2304eb4640 1 -- 172.21.15.27:0/1650685437 shutdown_connections 2023-12-19T18:15:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.333+0000 7f2304eb4640 1 --2- 172.21.15.27:0/1650685437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23000b7310 0x7f23000b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.333+0000 7f2304eb4640 1 --2- 172.21.15.27:0/1650685437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23001604d0 0x7f23000b47e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.333+0000 7f2304eb4640 1 -- 172.21.15.27:0/1650685437 >> 172.21.15.27:0/1650685437 conn(0x7f23000cf7e0 msgr2=0x7f23000cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.333+0000 7f2304eb4640 1 -- 172.21.15.27:0/1650685437 shutdown_connections 2023-12-19T18:15:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.333+0000 7f2304eb4640 1 -- 172.21.15.27:0/1650685437 wait complete. 2023-12-19T18:15:33.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.335+0000 7f2304eb4640 1 Processor -- start 2023-12-19T18:15:33.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.335+0000 7f2304eb4640 1 -- start start 2023-12-19T18:15:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.336+0000 7f2304eb4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23000b7310 0x7f23001f2160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.336+0000 7f2304eb4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23000d2350 0x7f2300157f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.336+0000 7f2304eb4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23001604d0 0x7f2300158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.336+0000 7f2304eb4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2300158a80 con 0x7f23000d2350 2023-12-19T18:15:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.336+0000 7f2304eb4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2300158d60 con 0x7f23000b7310 2023-12-19T18:15:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.336+0000 7f2304eb4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2300159040 con 0x7f23001604d0 2023-12-19T18:15:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.337+0000 7f22f7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23000d2350 0x7f2300157f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.337+0000 7f22f7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23000d2350 0x7f2300157f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58522/0 (socket says 172.21.15.27:58522) 2023-12-19T18:15:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.337+0000 7f22f7fff640 1 -- 172.21.15.27:0/1675932929 learned_addr learned my addr 172.21.15.27:0/1675932929 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.337+0000 7f22ff7fe640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23000b7310 0x7f23001f2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.337+0000 7f22fffff640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23001604d0 0x7f2300158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.337+0000 7f22f7fff640 1 -- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23001604d0 msgr2=0x7f2300158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.337+0000 7f22f7fff640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23001604d0 0x7f2300158490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.337+0000 7f22f7fff640 1 -- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23000b7310 msgr2=0x7f23001f2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.337+0000 7f22f7fff640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23000b7310 0x7f23001f2160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.337+0000 7f22f7fff640 1 -- 172.21.15.27:0/1675932929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f00094d0 con 0x7f23000d2350 2023-12-19T18:15:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.337+0000 7f22ff7fe640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23000b7310 0x7f23001f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.338+0000 7f22f7fff640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23000d2350 0x7f2300157f50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22e800c400 tx=0x7f22e800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.338+0000 7f22fcff9640 1 -- 172.21.15.27:0/1675932929 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8011790 con 0x7f23000d2350 2023-12-19T18:15:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.338+0000 7f22fcff9640 1 -- 172.21.15.27:0/1675932929 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e8011930 con 0x7f23000d2350 2023-12-19T18:15:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.338+0000 7f22fcff9640 1 -- 172.21.15.27:0/1675932929 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e8011ce0 con 0x7f23000d2350 2023-12-19T18:15:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.338+0000 7f2304eb4640 1 -- 172.21.15.27:0/1675932929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2300159320 con 0x7f23000d2350 2023-12-19T18:15:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.338+0000 7f2304eb4640 1 -- 172.21.15.27:0/1675932929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23001f26a0 con 0x7f23000d2350 2023-12-19T18:15:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.339+0000 7f2304eb4640 1 -- 172.21.15.27:0/1675932929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2300066f80 con 0x7f23000d2350 2023-12-19T18:15:33.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.340+0000 7f22fcff9640 1 -- 172.21.15.27:0/1675932929 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f22e801c460 con 0x7f23000d2350 2023-12-19T18:15:33.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.342+0000 7f22fcff9640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22e006a430 0x7f22e006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:33.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.342+0000 7f22fcff9640 1 -- 172.21.15.27:0/1675932929 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22e8088870 con 0x7f23000d2350 2023-12-19T18:15:33.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.342+0000 7f22ff7fe640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22e006a430 0x7f22e006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:33.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.343+0000 7f22ff7fe640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22e006a430 0x7f22e006c8f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f22f0036000 tx=0x7f22f003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:33.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.347+0000 7f22fcff9640 1 -- 172.21.15.27:0/1675932929 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f22e8053e90 con 0x7f23000d2350 2023-12-19T18:15:33.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.585+0000 7f2304eb4640 1 -- 172.21.15.27:0/1675932929 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2300145b50 con 0x7f23000d2350 2023-12-19T18:15:33.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.586+0000 7f22fcff9640 1 -- 172.21.15.27:0/1675932929 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f22e80574b0 con 0x7f23000d2350 2023-12-19T18:15:33.586 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:15:33.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.588+0000 7f2304eb4640 1 -- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22e006a430 msgr2=0x7f22e006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:33.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.588+0000 7f2304eb4640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22e006a430 0x7f22e006c8f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f22f0036000 tx=0x7f22f003c040 comp rx=0 tx=0).stop 2023-12-19T18:15:33.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.588+0000 7f2304eb4640 1 -- 172.21.15.27:0/1675932929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23000d2350 msgr2=0x7f2300157f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:33.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.588+0000 7f2304eb4640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23000d2350 0x7f2300157f50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f22e800c400 tx=0x7f22e800c8c0 comp rx=0 tx=0).stop 2023-12-19T18:15:33.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.589+0000 7f2304eb4640 1 -- 172.21.15.27:0/1675932929 shutdown_connections 2023-12-19T18:15:33.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.589+0000 7f2304eb4640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22e006a430 0x7f22e006c8f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:33.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.589+0000 7f2304eb4640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23001604d0 0x7f2300158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:33.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.589+0000 7f2304eb4640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23000d2350 0x7f2300157f50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:33.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.589+0000 7f2304eb4640 1 --2- 172.21.15.27:0/1675932929 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23000b7310 0x7f23001f2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:33.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.589+0000 7f2304eb4640 1 -- 172.21.15.27:0/1675932929 >> 172.21.15.27:0/1675932929 conn(0x7f23000cf7e0 msgr2=0x7f23000d1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:33.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.589+0000 7f2304eb4640 1 -- 172.21.15.27:0/1675932929 shutdown_connections 2023-12-19T18:15:33.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:33.589+0000 7f2304eb4640 1 -- 172.21.15.27:0/1675932929 wait complete. 2023-12-19T18:15:33.609 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:15:33.609 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:15:33.609 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-19T18:15:33.609 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-12-19T18:15:33.609 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-19T18:15:33.609 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:15:33.609 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:15:33.610 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:15:33.610 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-12-19T18:15:33.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:33.610 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-19T18:15:33.669 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:15:33.669 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:15:33.669 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:15:33.669 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:33.669 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:15:33.669 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42270sec preferred_lft 42270sec 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:15:33.670 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether a6:e1:fc:e6:56:fa brd ff:ff:ff:ff:ff:ff 2023-12-19T18:15:33.671 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:15:33.671 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:33.671 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::b892:1cff:fedc:4bed/64 scope link 2023-12-19T18:15:33.671 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:33.671 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:15:33.671 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether a6:e1:fc:e6:56:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:15:33.671 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::a4e1:fcff:fee6:56fa/64 scope link 2023-12-19T18:15:33.671 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:33.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:33.671 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:15:33.764 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-19T18:15:33.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:33.768 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-19T18:15:33.821 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-19T18:15:33.821 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:15:33.821 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T18:15:33.821 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T18:15:33.821 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:15:33.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:33.822 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:15:33.822 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:15:33.822 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T18:15:33.822 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T18:15:33.822 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T18:15:33.822 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:15:34.007 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:34.007 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:15:34.058 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T18:15:34.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:34.060 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list-id 2023-12-19T18:15:34.110 INFO:teuthology.orchestra.run.smithi192.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-12-19T18:15:34.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:34.110 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:15:34.111 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:15:34.111 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T18:15:34.111 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-12-19T18:15:34.111 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:15:34.245 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:34.245 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-19T18:15:34.341 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:15:34.341 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:15:34.341 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:15:34.341 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:34.341 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:15:34.341 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:34.341 INFO:teuthology.orchestra.run.smithi192.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:15:34.342 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:56:b5:10:c2:f6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-19T18:15:34.342 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-19T18:15:34.342 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:34.342 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::4056:b5ff:fe10:c2f6/64 scope link 2023-12-19T18:15:34.342 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:34.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:34.345 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-19T18:15:34.438 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:15:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:15:34.438 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:15:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:15:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:34.439 INFO:teuthology.orchestra.run.smithi192.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:15:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:56:b5:10:c2:f6 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-19T18:15:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-19T18:15:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::4056:b5ff:fe10:c2f6/64 scope link 2023-12-19T18:15:34.439 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:15:34.442 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-12-19T18:15:34.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:34.443 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:15:34.443 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:15:34.443 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-12-19T18:15:34.443 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-19T18:15:34.443 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-12-19T18:15:34.443 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-12-19T18:15:34.443 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-12-19T18:15:34.443 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:15:34.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:34.700 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:15:34.700 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:15:34.700 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.1 up 2023-12-19T18:15:34.700 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set dev brx.1 master ceph-brx 2023-12-19T18:15:34.700 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:15:34.862 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-12-19T18:15:34.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:15:34.863 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-12-19T18:15:34.917 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-12-19T18:15:34.917 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:15:34.917 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-12-19T18:15:34.966 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:15:35.060 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:15:35.060 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:15:35.149 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:15:35.152 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:15:35.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:35.153 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:15:35.206 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:15:35.206 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [40] 2023-12-19T18:15:35.207 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-12-19T18:15:35.212 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:15:35.288 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:15:35.289 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:15:35.356 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:15:35.359 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:15:35.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:35.360 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:15:35.395 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi192.stderr:2023-12-19T18:15:35.395+0000 7f5e4a83a480 -1 init, newargv = 0x555e4eb6c2f0 newargc=15 2023-12-19T18:15:35.395 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi192.stderr:ceph-fuse[85637]: starting ceph client 2023-12-19T18:15:35.400 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:15:35.402 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi192.stderr:ceph-fuse[85637]: starting fuse 2023-12-19T18:15:36.501 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:15:36.604 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:15:36.604 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:15:36.695 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:15:36.698 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:15:36.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:36.699 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:15:36.751 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:15:36.752 INFO:teuthology.orchestra.run.smithi192.stdout:42 2023-12-19T18:15:36.752 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40, 42] 2023-12-19T18:15:36.752 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:15:36.752 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-19T18:15:36.752 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-19T18:15:36.752 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-19T18:15:36.752 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-19T18:15:36.752 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:15:36.752 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-19T18:15:36.752 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-12-19T18:15:36.752 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:15:36.753 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:15:36.754 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.3")) 2023-12-19T18:15:36.754 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-19T18:15:36.933 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.3.85637.asok 2023-12-19T18:15:37.113 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.85637.asok 2023-12-19T18:15:37.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:37.113 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.85637.asok status 2023-12-19T18:15:37.356 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-19T18:15:37.356 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-19T18:15:37.356 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:15:37.356 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:15:37.356 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "3", 2023-12-19T18:15:37.356 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-19T18:15:37.356 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-12-19T18:15:37.356 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "85637", 2023-12-19T18:15:37.356 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 4621, 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 4621 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.2:0", 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2560229620 2023-12-19T18:15:37.357 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.2:0", 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2560229620 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.4621 192.168.0.2:0/2560229620", 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.2:0/2560229620", 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 8, 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 20, 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-19T18:15:37.358 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-19T18:15:37.371 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:37.371 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-12-19T18:15:37.371 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-19T18:15:37.437 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-12-19T18:15:37.438 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-19T18:15:37.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:37.438 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:15:37.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:37.556 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-12-19T18:15:37.556 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.2 2023-12-19T18:15:37.617 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-12-19T18:15:37.617 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-12-19T18:15:37.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:37.618 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-12-19T18:15:37.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:37.731 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-19T18:15:37.732 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-19T18:15:37.795 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-19T18:15:37.795 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-19T18:15:37.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:37.795 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:15:37.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:37.891 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-19T18:15:37.891 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.3 2023-12-19T18:15:37.945 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-19T18:15:37.945 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-12-19T18:15:37.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:37.945 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-12-19T18:15:38.038 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-12-19T18:15:38.056 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:38.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.296+0000 7f03b359e640 1 Processor -- start 2023-12-19T18:15:38.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.296+0000 7f03b359e640 1 -- start start 2023-12-19T18:15:38.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.297+0000 7f03b359e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f03b415cf00 0x7f03b4069c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.297+0000 7f03b359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b406b7d0 0x7f03b40cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.298+0000 7f03b359e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b406a370 con 0x7f03b40cf1d0 2023-12-19T18:15:38.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.298+0000 7f03b359e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b406a650 con 0x7f03b415cf00 2023-12-19T18:15:38.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.298+0000 7f03b359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b406a930 con 0x7f03b406b7d0 2023-12-19T18:15:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.298+0000 7f03abfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b406b7d0 0x7f03b40cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.298+0000 7f03abfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b406b7d0 0x7f03b40cec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:54964/0 (socket says 172.21.15.27:54964) 2023-12-19T18:15:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.298+0000 7f03abfff640 1 -- 172.21.15.27:0/1452132781 learned_addr learned my addr 172.21.15.27:0/1452132781 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.298+0000 7f03b259c640 1 --2- 172.21.15.27:0/1452132781 >> v2:172.21.15.192:3300/0 conn(0x7f03b415cf00 0x7f03b4069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.299+0000 7f03abfff640 1 -- 172.21.15.27:0/1452132781 >> v2:172.21.15.192:3300/0 conn(0x7f03b415cf00 msgr2=0x7f03b4069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:38.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.299+0000 7f03abfff640 1 --2- 172.21.15.27:0/1452132781 >> v2:172.21.15.192:3300/0 conn(0x7f03b415cf00 0x7f03b4069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.299+0000 7f03abfff640 1 -- 172.21.15.27:0/1452132781 >> v1:172.21.15.192:6789/0 conn(0x7f03b40cf1d0 legacy=0x7f03b4070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:38.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.299+0000 7f03abfff640 1 -- 172.21.15.27:0/1452132781 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b406ac10 con 0x7f03b406b7d0 2023-12-19T18:15:38.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.299+0000 7f03b1d9b640 1 -- 172.21.15.27:0/1452132781 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 704761989 0 0) 0x7f03b406a370 con 0x7f03b40cf1d0 2023-12-19T18:15:38.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.301+0000 7f03abfff640 1 --2- 172.21.15.27:0/1452132781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b406b7d0 0x7f03b40cec90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f039c009870 tx=0x7f039c02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc1ff23f3a355304 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:38.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.301+0000 7f03b1d9b640 1 -- 172.21.15.27:0/1452132781 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c030840 con 0x7f03b406b7d0 2023-12-19T18:15:38.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.301+0000 7f03b1d9b640 1 -- 172.21.15.27:0/1452132781 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039c0309a0 con 0x7f03b406b7d0 2023-12-19T18:15:38.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.301+0000 7f03b1d9b640 1 -- 172.21.15.27:0/1452132781 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039c030d30 con 0x7f03b406b7d0 2023-12-19T18:15:38.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.302+0000 7f03b359e640 1 -- 172.21.15.27:0/1452132781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b406b7d0 msgr2=0x7f03b40cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:38.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.302+0000 7f03b359e640 1 --2- 172.21.15.27:0/1452132781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b406b7d0 0x7f03b40cec90 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f039c009870 tx=0x7f039c02b980 comp rx=0 tx=0).stop 2023-12-19T18:15:38.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.302+0000 7f03b359e640 1 -- 172.21.15.27:0/1452132781 shutdown_connections 2023-12-19T18:15:38.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.302+0000 7f03b359e640 1 --2- 172.21.15.27:0/1452132781 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b406b7d0 0x7f03b40cec90 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.302+0000 7f03b359e640 1 --2- 172.21.15.27:0/1452132781 >> v2:172.21.15.192:3300/0 conn(0x7f03b415cf00 0x7f03b4069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.302+0000 7f03b359e640 1 -- 172.21.15.27:0/1452132781 >> 172.21.15.27:0/1452132781 conn(0x7f03b40cfa20 msgr2=0x7f03b40cfe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:38.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.302+0000 7f03b359e640 1 -- 172.21.15.27:0/1452132781 shutdown_connections 2023-12-19T18:15:38.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.302+0000 7f03b359e640 1 -- 172.21.15.27:0/1452132781 wait complete. 2023-12-19T18:15:38.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.304+0000 7f03b359e640 1 Processor -- start 2023-12-19T18:15:38.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.304+0000 7f03b359e640 1 -- start start 2023-12-19T18:15:38.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.305+0000 7f03b359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b406b7d0 0x7f03b4154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.305+0000 7f03b359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b40cf1d0 0x7f03b415f180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.305+0000 7f03b359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b415cf00 0x7f03b415f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.305+0000 7f03b359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b40b31e0 con 0x7f03b415cf00 2023-12-19T18:15:38.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.305+0000 7f03b359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b4154dd0 con 0x7f03b406b7d0 2023-12-19T18:15:38.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.305+0000 7f03b359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b415da90 con 0x7f03b40cf1d0 2023-12-19T18:15:38.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.306+0000 7f03b2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b415cf00 0x7f03b415f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.306+0000 7f03b2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b415cf00 0x7f03b415f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58534/0 (socket says 172.21.15.27:58534) 2023-12-19T18:15:38.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.306+0000 7f03b2d9d640 1 -- 172.21.15.27:0/3367764108 learned_addr learned my addr 172.21.15.27:0/3367764108 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:38.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.306+0000 7f03abfff640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b40cf1d0 0x7f03b415f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.306+0000 7f03b259c640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b406b7d0 0x7f03b4154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.306+0000 7f03b2d9d640 1 -- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b40cf1d0 msgr2=0x7f03b415f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:38.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.306+0000 7f03b2d9d640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b40cf1d0 0x7f03b415f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.306+0000 7f03b2d9d640 1 -- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b406b7d0 msgr2=0x7f03b4154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.306+0000 7f03b2d9d640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b406b7d0 0x7f03b4154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.306+0000 7f03b2d9d640 1 -- 172.21.15.27:0/3367764108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0094d0 con 0x7f03b415cf00 2023-12-19T18:15:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.307+0000 7f03b2d9d640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b415cf00 0x7f03b415f6c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c7e0 tx=0x7f03ac00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.307+0000 7f03aaffd640 1 -- 172.21.15.27:0/3367764108 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac003da0 con 0x7f03b415cf00 2023-12-19T18:15:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.307+0000 7f03aaffd640 1 -- 172.21.15.27:0/3367764108 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac005070 con 0x7f03b415cf00 2023-12-19T18:15:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.307+0000 7f03aaffd640 1 -- 172.21.15.27:0/3367764108 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac005420 con 0x7f03b415cf00 2023-12-19T18:15:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.307+0000 7f03b359e640 1 -- 172.21.15.27:0/3367764108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b40b2c20 con 0x7f03b415cf00 2023-12-19T18:15:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.307+0000 7f03b359e640 1 -- 172.21.15.27:0/3367764108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b415d720 con 0x7f03b415cf00 2023-12-19T18:15:38.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.308+0000 7f03b359e640 1 -- 172.21.15.27:0/3367764108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b415d580 con 0x7f03b415cf00 2023-12-19T18:15:38.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.309+0000 7f03aaffd640 1 -- 172.21.15.27:0/3367764108 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f03ac005580 con 0x7f03b415cf00 2023-12-19T18:15:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.310+0000 7f03aaffd640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f039406a430 0x7f039406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.310+0000 7f03aaffd640 1 -- 172.21.15.27:0/3367764108 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03ac088180 con 0x7f03b415cf00 2023-12-19T18:15:38.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.311+0000 7f03b259c640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f039406a430 0x7f039406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.312+0000 7f03b259c640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f039406a430 0x7f039406c8f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009ad0 tx=0x7f03a4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:38.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.315+0000 7f03aaffd640 1 -- 172.21.15.27:0/3367764108 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f03b415d580 con 0x7f03b415cf00 2023-12-19T18:15:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.510+0000 7f03b359e640 1 -- 172.21.15.27:0/3367764108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03b415d580 con 0x7f03b415cf00 2023-12-19T18:15:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.513+0000 7f03aaffd640 1 -- 172.21.15.27:0/3367764108 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f03b415d580 con 0x7f03b415cf00 2023-12-19T18:15:38.513 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:38.513 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:55.039690+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:38.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.515+0000 7f03b359e640 1 -- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f039406a430 msgr2=0x7f039406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:38.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.515+0000 7f03b359e640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f039406a430 0x7f039406c8f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f03a4009ad0 tx=0x7f03a4008040 comp rx=0 tx=0).stop 2023-12-19T18:15:38.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.516+0000 7f03b359e640 1 -- 172.21.15.27:0/3367764108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b415cf00 msgr2=0x7f03b415f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:38.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.516+0000 7f03b359e640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b415cf00 0x7f03b415f6c0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f03ac00c7e0 tx=0x7f03ac00cca0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.516+0000 7f03b359e640 1 -- 172.21.15.27:0/3367764108 shutdown_connections 2023-12-19T18:15:38.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.516+0000 7f03b359e640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f039406a430 0x7f039406c8f0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.516+0000 7f03b359e640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b415cf00 0x7f03b415f6c0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.516+0000 7f03b359e640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b40cf1d0 0x7f03b415f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.516+0000 7f03b359e640 1 --2- 172.21.15.27:0/3367764108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b406b7d0 0x7f03b4154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.516+0000 7f03b359e640 1 -- 172.21.15.27:0/3367764108 >> 172.21.15.27:0/3367764108 conn(0x7f03b40cfa20 msgr2=0x7f03b40cfe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:38.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.516+0000 7f03b359e640 1 -- 172.21.15.27:0/3367764108 shutdown_connections 2023-12-19T18:15:38.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.516+0000 7f03b359e640 1 -- 172.21.15.27:0/3367764108 wait complete. 2023-12-19T18:15:38.537 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:15:38.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:15:38.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.773+0000 7fe36e1fd640 1 Processor -- start 2023-12-19T18:15:38.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.773+0000 7fe36e1fd640 1 -- start start 2023-12-19T18:15:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.774+0000 7fe36e1fd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe36806ddc0 0x7fe3680704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.774+0000 7fe36e1fd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe36806ce80 0x7fe368070a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.775+0000 7fe36e1fd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe36815d320 con 0x7fe36815cf40 2023-12-19T18:15:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.775+0000 7fe36e1fd640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe368069fa0 con 0x7fe36806ce80 2023-12-19T18:15:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.775+0000 7fe36e1fd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36806c3c0 con 0x7fe36806ddc0 2023-12-19T18:15:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.775+0000 7fe36d1fb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe36815cf40 0x7fe3680cee30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37506/0 (socket says 172.21.15.27:37506) 2023-12-19T18:15:38.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.775+0000 7fe36d1fb640 1 -- 172.21.15.27:0/815829894 learned_addr learned my addr 172.21.15.27:0/815829894 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:38.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.775+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964710842 0 0) 0x7fe36815d320 con 0x7fe36815cf40 2023-12-19T18:15:38.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.775+0000 7fe36c9fa640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe36806ddc0 0x7fe3680704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.775+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe350003520 con 0x7fe36815cf40 2023-12-19T18:15:38.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.775+0000 7fe36d9fc640 1 --2- 172.21.15.27:0/815829894 >> v2:172.21.15.192:3300/0 conn(0x7fe36806ce80 0x7fe368070a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 984284233 0 0) 0x7fe350003520 con 0x7fe36815cf40 2023-12-19T18:15:38.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe36806ddc0 msgr2=0x7fe3680704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:38.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe36806ddc0 0x7fe3680704c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 >> v2:172.21.15.192:3300/0 conn(0x7fe36806ce80 msgr2=0x7fe368070a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:38.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 --2- 172.21.15.27:0/815829894 >> v2:172.21.15.192:3300/0 conn(0x7fe36806ce80 0x7fe368070a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3680cf660 con 0x7fe36815cf40 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe3580019f0 con 0x7fe36815cf40 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 >> v1:172.21.15.27:6789/0 conn(0x7fe36815cf40 legacy=0x7fe3680cee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe350003970 0x7fe350003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe36806ddc0 0x7fe3500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe350004d00 0x7fe350005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36806c3c0 con 0x7fe350003970 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35000ba90 con 0x7fe350004d00 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35000bc50 con 0x7fe36806ddc0 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.776+0000 7fe36d1fb640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe350003970 0x7fe350003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.777+0000 7fe36c9fa640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe36806ddc0 0x7fe3500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.777+0000 7fe36d9fc640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe350004d00 0x7fe350005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.777+0000 7fe36d1fb640 1 -- 172.21.15.27:0/815829894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe36806ddc0 msgr2=0x7fe3500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:38.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.777+0000 7fe36d1fb640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe36806ddc0 0x7fe3500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.777+0000 7fe36d1fb640 1 -- 172.21.15.27:0/815829894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe350004d00 msgr2=0x7fe350005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.777+0000 7fe36d1fb640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe350004d00 0x7fe350005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.777+0000 7fe36d1fb640 1 -- 172.21.15.27:0/815829894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe350003520 con 0x7fe350003970 2023-12-19T18:15:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.777+0000 7fe36c9fa640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe36806ddc0 0x7fe3500047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:38.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.777+0000 7fe36d9fc640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe350004d00 0x7fe350005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:38.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.778+0000 7fe36d1fb640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe350003970 0x7fe350003d90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe35800a790 tx=0x7fe3500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c7f755675e493fb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:38.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.778+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe358034630 con 0x7fe350003970 2023-12-19T18:15:38.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.778+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35802f070 con 0x7fe350003970 2023-12-19T18:15:38.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.778+0000 7fe367fff640 1 -- 172.21.15.27:0/815829894 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3580349f0 con 0x7fe350003970 2023-12-19T18:15:38.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.779+0000 7fe36e1fd640 1 -- 172.21.15.27:0/815829894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe350003970 msgr2=0x7fe350003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:38.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.779+0000 7fe36e1fd640 1 --2- 172.21.15.27:0/815829894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe350003970 0x7fe350003d90 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fe35800a790 tx=0x7fe3500057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.779+0000 7fe36d9fc640 1 -- 172.21.15.27:0/815829894 reap_dead start 2023-12-19T18:15:38.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.779+0000 7fe36e1fd640 1 -- 172.21.15.27:0/815829894 shutdown_connections 2023-12-19T18:15:38.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.779+0000 7fe36e1fd640 1 -- 172.21.15.27:0/815829894 >> 172.21.15.27:0/815829894 conn(0x7fe3680cfa60 msgr2=0x7fe3680cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:38.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.779+0000 7fe36e1fd640 1 -- 172.21.15.27:0/815829894 shutdown_connections 2023-12-19T18:15:38.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.779+0000 7fe36e1fd640 1 -- 172.21.15.27:0/815829894 wait complete. 2023-12-19T18:15:38.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.781+0000 7fe36e1fd640 1 Processor -- start 2023-12-19T18:15:38.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.781+0000 7fe36e1fd640 1 -- start start 2023-12-19T18:15:38.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.782+0000 7fe36e1fd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe350003970 0x7fe36815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.782+0000 7fe36e1fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe36815c390 0x7fe3680c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.782+0000 7fe36e1fd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3681f25f0 0x7fe3681f2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.782+0000 7fe36e1fd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36815c140 con 0x7fe36815c390 2023-12-19T18:15:38.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.782+0000 7fe36e1fd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe36815ef90 con 0x7fe3681f25f0 2023-12-19T18:15:38.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.782+0000 7fe36e1fd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe36815f270 con 0x7fe350003970 2023-12-19T18:15:38.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.783+0000 7fe36c9fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe36815c390 0x7fe3680c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.783+0000 7fe36c9fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe36815c390 0x7fe3680c6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58544/0 (socket says 172.21.15.27:58544) 2023-12-19T18:15:38.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.783+0000 7fe36c9fa640 1 -- 172.21.15.27:0/3008622790 learned_addr learned my addr 172.21.15.27:0/3008622790 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:38.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.783+0000 7fe36c9fa640 1 -- 172.21.15.27:0/3008622790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe350003970 msgr2=0x7fe36815bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:38.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.783+0000 7fe36d1fb640 1 --2- 172.21.15.27:0/3008622790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe350003970 0x7fe36815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.783+0000 7fe36c9fa640 1 --2- 172.21.15.27:0/3008622790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe350003970 0x7fe36815bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.783+0000 7fe36c9fa640 1 -- 172.21.15.27:0/3008622790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3681f25f0 msgr2=0x7fe3681f2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:38.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.783+0000 7fe36c9fa640 1 --2- 172.21.15.27:0/3008622790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3681f25f0 0x7fe3681f2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.783+0000 7fe36c9fa640 1 -- 172.21.15.27:0/3008622790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe36815f550 con 0x7fe36815c390 2023-12-19T18:15:38.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.783+0000 7fe36c9fa640 1 --2- 172.21.15.27:0/3008622790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe36815c390 0x7fe3680c6b30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00e7e0 tx=0x7fe35c00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:38.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.784+0000 7fe365ffb640 1 -- 172.21.15.27:0/3008622790 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00c6d0 con 0x7fe36815c390 2023-12-19T18:15:38.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.784+0000 7fe365ffb640 1 -- 172.21.15.27:0/3008622790 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe35c00c830 con 0x7fe36815c390 2023-12-19T18:15:38.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.784+0000 7fe365ffb640 1 -- 172.21.15.27:0/3008622790 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe35c00cba0 con 0x7fe36815c390 2023-12-19T18:15:38.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.784+0000 7fe36e1fd640 1 -- 172.21.15.27:0/3008622790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe36815f830 con 0x7fe36815c390 2023-12-19T18:15:38.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.784+0000 7fe36e1fd640 1 -- 172.21.15.27:0/3008622790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3680c7070 con 0x7fe36815c390 2023-12-19T18:15:38.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.785+0000 7fe36e1fd640 1 -- 172.21.15.27:0/3008622790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe36804e840 con 0x7fe36815c390 2023-12-19T18:15:38.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.786+0000 7fe365ffb640 1 -- 172.21.15.27:0/3008622790 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe35c00cd00 con 0x7fe36815c390 2023-12-19T18:15:38.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.787+0000 7fe365ffb640 1 --2- 172.21.15.27:0/3008622790 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe35406a430 0x7fe35406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:38.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.788+0000 7fe365ffb640 1 -- 172.21.15.27:0/3008622790 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe35c088750 con 0x7fe36815c390 2023-12-19T18:15:38.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.788+0000 7fe36d1fb640 1 --2- 172.21.15.27:0/3008622790 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe35406a430 0x7fe35406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:38.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.789+0000 7fe36d1fb640 1 --2- 172.21.15.27:0/3008622790 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe35406a430 0x7fe35406c8f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe358034000 tx=0x7fe358009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:38.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.792+0000 7fe365ffb640 1 -- 172.21.15.27:0/3008622790 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe35c053ca0 con 0x7fe36815c390 2023-12-19T18:15:38.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.983+0000 7fe36e1fd640 1 -- 172.21.15.27:0/3008622790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe36806ba50 con 0x7fe36815c390 2023-12-19T18:15:38.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.984+0000 7fe365ffb640 1 -- 172.21.15.27:0/3008622790 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe35c0572c0 con 0x7fe36815c390 2023-12-19T18:15:38.985 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:15:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.987+0000 7fe36e1fd640 1 -- 172.21.15.27:0/3008622790 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe35406a430 msgr2=0x7fe35406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.987+0000 7fe36e1fd640 1 --2- 172.21.15.27:0/3008622790 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe35406a430 0x7fe35406c8f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe358034000 tx=0x7fe358009030 comp rx=0 tx=0).stop 2023-12-19T18:15:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.987+0000 7fe36e1fd640 1 -- 172.21.15.27:0/3008622790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe36815c390 msgr2=0x7fe3680c6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:38.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.987+0000 7fe36e1fd640 1 --2- 172.21.15.27:0/3008622790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe36815c390 0x7fe3680c6b30 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fe35c00e7e0 tx=0x7fe35c00eca0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.987+0000 7fe36e1fd640 1 -- 172.21.15.27:0/3008622790 shutdown_connections 2023-12-19T18:15:38.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.987+0000 7fe36e1fd640 1 --2- 172.21.15.27:0/3008622790 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe35406a430 0x7fe35406c8f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.987+0000 7fe36e1fd640 1 --2- 172.21.15.27:0/3008622790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3681f25f0 0x7fe3681f2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.987+0000 7fe36e1fd640 1 --2- 172.21.15.27:0/3008622790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe36815c390 0x7fe3680c6b30 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.987+0000 7fe36e1fd640 1 --2- 172.21.15.27:0/3008622790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe350003970 0x7fe36815bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:38.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.987+0000 7fe36e1fd640 1 -- 172.21.15.27:0/3008622790 >> 172.21.15.27:0/3008622790 conn(0x7fe3680cfa60 msgr2=0x7fe3680ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:38.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.987+0000 7fe36e1fd640 1 -- 172.21.15.27:0/3008622790 shutdown_connections 2023-12-19T18:15:38.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:38.987+0000 7fe36e1fd640 1 -- 172.21.15.27:0/3008622790 wait complete. 2023-12-19T18:15:39.005 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:15:39.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.243+0000 7f8746bf9640 1 Processor -- start 2023-12-19T18:15:39.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.243+0000 7f8746bf9640 1 -- start start 2023-12-19T18:15:39.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.244+0000 7f8746bf9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f874015cf80 0x7f87400d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.244+0000 7f8746bf9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87400b85e0 0x7f87400cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.244+0000 7f8746bf9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f87400b8a60 0x7f87400cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.245+0000 7f8746bf9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87400d0e10 con 0x7f87400b8a60 2023-12-19T18:15:39.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.245+0000 7f8746bf9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87400d1c70 con 0x7f874015cf80 2023-12-19T18:15:39.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.245+0000 7f8746bf9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874015d360 con 0x7f87400b85e0 2023-12-19T18:15:39.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.245+0000 7f87463f8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f87400b8a60 0x7f87400cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:39.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.245+0000 7f87463f8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f87400b8a60 0x7f87400cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58558/0 (socket says 172.21.15.27:58558) 2023-12-19T18:15:39.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.245+0000 7f87463f8640 1 -- 172.21.15.27:0/552558361 learned_addr learned my addr 172.21.15.27:0/552558361 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:39.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.245+0000 7f87453f6640 1 --2- 172.21.15.27:0/552558361 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87400b85e0 0x7f87400cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:39.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.245+0000 7f87463f8640 1 -- 172.21.15.27:0/552558361 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87400b85e0 msgr2=0x7f87400cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:39.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.245+0000 7f87463f8640 1 --2- 172.21.15.27:0/552558361 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87400b85e0 0x7f87400cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.245+0000 7f8745bf7640 1 --2- 172.21.15.27:0/552558361 >> v2:172.21.15.192:3300/0 conn(0x7f874015cf80 0x7f87400d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:39.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.245+0000 7f87463f8640 1 -- 172.21.15.27:0/552558361 >> v2:172.21.15.192:3300/0 conn(0x7f874015cf80 msgr2=0x7f87400d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:39.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.246+0000 7f87463f8640 1 --2- 172.21.15.27:0/552558361 >> v2:172.21.15.192:3300/0 conn(0x7f874015cf80 0x7f87400d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.246+0000 7f87463f8640 1 -- 172.21.15.27:0/552558361 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400d1060 con 0x7f87400b8a60 2023-12-19T18:15:39.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.246+0000 7f87453f6640 1 --2- 172.21.15.27:0/552558361 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87400b85e0 0x7f87400cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:39.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.247+0000 7f87463f8640 1 --2- 172.21.15.27:0/552558361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87400b8a60 0x7f87400cdf80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f87400d3d70 tx=0x7f873802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a06b0b1525bff49 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:39.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.247+0000 7f8744bf5640 1 -- 172.21.15.27:0/552558361 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738030850 con 0x7f87400b8a60 2023-12-19T18:15:39.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.247+0000 7f8744bf5640 1 -- 172.21.15.27:0/552558361 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87380309b0 con 0x7f87400b8a60 2023-12-19T18:15:39.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.247+0000 7f8744bf5640 1 -- 172.21.15.27:0/552558361 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738030d40 con 0x7f87400b8a60 2023-12-19T18:15:39.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.248+0000 7f8746bf9640 1 -- 172.21.15.27:0/552558361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87400b8a60 msgr2=0x7f87400cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:39.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.248+0000 7f8746bf9640 1 --2- 172.21.15.27:0/552558361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87400b8a60 0x7f87400cdf80 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f87400d3d70 tx=0x7f873802b990 comp rx=0 tx=0).stop 2023-12-19T18:15:39.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.248+0000 7f8746bf9640 1 -- 172.21.15.27:0/552558361 shutdown_connections 2023-12-19T18:15:39.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.248+0000 7f8746bf9640 1 --2- 172.21.15.27:0/552558361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87400b8a60 0x7f87400cdf80 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.248+0000 7f8746bf9640 1 --2- 172.21.15.27:0/552558361 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87400b85e0 0x7f87400cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.248+0000 7f8746bf9640 1 --2- 172.21.15.27:0/552558361 >> v2:172.21.15.192:3300/0 conn(0x7f874015cf80 0x7f87400d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.248+0000 7f8746bf9640 1 -- 172.21.15.27:0/552558361 >> 172.21.15.27:0/552558361 conn(0x7f8740072e20 msgr2=0x7f8740073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:39.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.248+0000 7f8746bf9640 1 -- 172.21.15.27:0/552558361 shutdown_connections 2023-12-19T18:15:39.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.248+0000 7f8746bf9640 1 -- 172.21.15.27:0/552558361 wait complete. 2023-12-19T18:15:39.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.250+0000 7f8746bf9640 1 Processor -- start 2023-12-19T18:15:39.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.250+0000 7f8746bf9640 1 -- start start 2023-12-19T18:15:39.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f8746bf9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87400b85e0 0x7f87400a9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f8746bf9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f874015cf80 0x7f874014e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f8746bf9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f874014ef30 0x7f874015d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f8746bf9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874014d950 con 0x7f874015cf80 2023-12-19T18:15:39.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f8746bf9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874014dc30 con 0x7f87400b85e0 2023-12-19T18:15:39.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f8746bf9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f874014df10 con 0x7f874014ef30 2023-12-19T18:15:39.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f87453f6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f874015cf80 0x7f874014e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:39.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f87453f6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f874015cf80 0x7f874014e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58568/0 (socket says 172.21.15.27:58568) 2023-12-19T18:15:39.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f87453f6640 1 -- 172.21.15.27:0/4045330578 learned_addr learned my addr 172.21.15.27:0/4045330578 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:39.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f87453f6640 1 -- 172.21.15.27:0/4045330578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f874014ef30 msgr2=0x7f874015d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:39.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f87453f6640 1 --2- 172.21.15.27:0/4045330578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f874014ef30 0x7f874015d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f87453f6640 1 -- 172.21.15.27:0/4045330578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87400b85e0 msgr2=0x7f87400a9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:39.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f87453f6640 1 --2- 172.21.15.27:0/4045330578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87400b85e0 0x7f87400a9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.251+0000 7f87453f6640 1 -- 172.21.15.27:0/4045330578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87380094d0 con 0x7f874015cf80 2023-12-19T18:15:39.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.252+0000 7f87453f6640 1 --2- 172.21.15.27:0/4045330578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f874015cf80 0x7f874014e9f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f873000c7f0 tx=0x7f873000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:39.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.252+0000 7f87367fc640 1 -- 172.21.15.27:0/4045330578 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730007a30 con 0x7f874015cf80 2023-12-19T18:15:39.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.252+0000 7f87367fc640 1 -- 172.21.15.27:0/4045330578 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8730007b90 con 0x7f874015cf80 2023-12-19T18:15:39.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.252+0000 7f8746bf9640 1 -- 172.21.15.27:0/4045330578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874014e1f0 con 0x7f874015cf80 2023-12-19T18:15:39.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.252+0000 7f87367fc640 1 -- 172.21.15.27:0/4045330578 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873000ce40 con 0x7f874015cf80 2023-12-19T18:15:39.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.252+0000 7f8746bf9640 1 -- 172.21.15.27:0/4045330578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874015dcf0 con 0x7f874015cf80 2023-12-19T18:15:39.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.253+0000 7f8746bf9640 1 -- 172.21.15.27:0/4045330578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874014e710 con 0x7f874015cf80 2023-12-19T18:15:39.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.254+0000 7f87367fc640 1 -- 172.21.15.27:0/4045330578 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f87300113f0 con 0x7f874015cf80 2023-12-19T18:15:39.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.255+0000 7f87367fc640 1 --2- 172.21.15.27:0/4045330578 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f872406a430 0x7f872406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.255+0000 7f87367fc640 1 -- 172.21.15.27:0/4045330578 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8730088b00 con 0x7f874015cf80 2023-12-19T18:15:39.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.256+0000 7f8745bf7640 1 --2- 172.21.15.27:0/4045330578 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f872406a430 0x7f872406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:39.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.257+0000 7f8745bf7640 1 --2- 172.21.15.27:0/4045330578 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f872406a430 0x7f872406c8f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f873c008cd0 tx=0x7f873c008200 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:39.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.260+0000 7f87367fc640 1 -- 172.21.15.27:0/4045330578 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f87300540f0 con 0x7f874015cf80 2023-12-19T18:15:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.456+0000 7f8746bf9640 1 -- 172.21.15.27:0/4045330578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f874004e840 con 0x7f874015cf80 2023-12-19T18:15:39.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.457+0000 7f87367fc640 1 -- 172.21.15.27:0/4045330578 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f873001f020 con 0x7f874015cf80 2023-12-19T18:15:39.457 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:15:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.459+0000 7f8746bf9640 1 -- 172.21.15.27:0/4045330578 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f872406a430 msgr2=0x7f872406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.459+0000 7f8746bf9640 1 --2- 172.21.15.27:0/4045330578 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f872406a430 0x7f872406c8f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f873c008cd0 tx=0x7f873c008200 comp rx=0 tx=0).stop 2023-12-19T18:15:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.459+0000 7f8746bf9640 1 -- 172.21.15.27:0/4045330578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f874015cf80 msgr2=0x7f874014e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.459+0000 7f8746bf9640 1 --2- 172.21.15.27:0/4045330578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f874015cf80 0x7f874014e9f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f873000c7f0 tx=0x7f873000cbf0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.460+0000 7f8746bf9640 1 -- 172.21.15.27:0/4045330578 shutdown_connections 2023-12-19T18:15:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.460+0000 7f8746bf9640 1 --2- 172.21.15.27:0/4045330578 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f872406a430 0x7f872406c8f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.460+0000 7f8746bf9640 1 --2- 172.21.15.27:0/4045330578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f874014ef30 0x7f874015d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.460+0000 7f8746bf9640 1 --2- 172.21.15.27:0/4045330578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f874015cf80 0x7f874014e9f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.460+0000 7f8746bf9640 1 --2- 172.21.15.27:0/4045330578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87400b85e0 0x7f87400a9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.460+0000 7f8746bf9640 1 -- 172.21.15.27:0/4045330578 >> 172.21.15.27:0/4045330578 conn(0x7f8740072e20 msgr2=0x7f87400cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:39.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.460+0000 7f8746bf9640 1 -- 172.21.15.27:0/4045330578 shutdown_connections 2023-12-19T18:15:39.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.460+0000 7f8746bf9640 1 -- 172.21.15.27:0/4045330578 wait complete. 2023-12-19T18:15:39.481 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:15:39.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.715+0000 7f6787fff640 1 Processor -- start 2023-12-19T18:15:39.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.715+0000 7f6787fff640 1 -- start start 2023-12-19T18:15:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.716+0000 7f6787fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f67880b3980 0x7f67880cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.717+0000 7f6787fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67880d3850 con 0x7f678815d120 2023-12-19T18:15:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.717+0000 7f6787fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67880d0b10 con 0x7f67880d1a20 2023-12-19T18:15:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.717+0000 7f6787fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f678815d500 con 0x7f67880b3980 2023-12-19T18:15:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.717+0000 7f6786ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f678815d120 0x7f67880d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37510/0 (socket says 172.21.15.27:37510) 2023-12-19T18:15:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.717+0000 7f6786ffd640 1 -- 172.21.15.27:0/4207470597 learned_addr learned my addr 172.21.15.27:0/4207470597 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.717+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476076068 0 0) 0x7f67880d3850 con 0x7f678815d120 2023-12-19T18:15:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.717+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6774003520 con 0x7f678815d120 2023-12-19T18:15:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f67877fe640 1 --2- 172.21.15.27:0/4207470597 >> v2:172.21.15.192:3300/0 conn(0x7f67880b3980 0x7f67880cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2396888064 0 0) 0x7f6774003520 con 0x7f678815d120 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 >> v2:172.21.15.192:3300/0 conn(0x7f67880b3980 msgr2=0x7f67880cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6785ffb640 1 --2- 172.21.15.27:0/4207470597 >> v2:172.21.15.192:3300/0 conn(0x7f67880b3980 0x7f67880cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 >> v1:172.21.15.192:6789/0 conn(0x7f67880d1a20 legacy=0x7f67880cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67880d1040 con 0x7f678815d120 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f6770001670 con 0x7f678815d120 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 >> v1:172.21.15.27:6789/0 conn(0x7f678815d120 legacy=0x7f67880d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6785ffb640 1 --2- 172.21.15.27:0/4207470597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67740039c0 0x7f6774003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6785ffb640 1 --2- 172.21.15.27:0/4207470597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67740047c0 0x7f6774004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6785ffb640 1 --2- 172.21.15.27:0/4207470597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67740052a0 0x7f677400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678815d500 con 0x7f67740047c0 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f677400c1f0 con 0x7f67740052a0 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f677400c3b0 con 0x7f67740039c0 2023-12-19T18:15:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f67877fe640 1 --2- 172.21.15.27:0/4207470597 >> v2:172.21.15.192:3300/0 conn(0x7f67880b3980 0x7f67880cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f67867fc640 1 --2- 172.21.15.27:0/4207470597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67740047c0 0x7f6774004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.718+0000 7f6786ffd640 1 --2- 172.21.15.27:0/4207470597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67740039c0 0x7f6774003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.719+0000 7f67877fe640 1 --2- 172.21.15.27:0/4207470597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67740052a0 0x7f677400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.719+0000 7f67867fc640 1 -- 172.21.15.27:0/4207470597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67740039c0 msgr2=0x7f6774003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.719+0000 7f67867fc640 1 --2- 172.21.15.27:0/4207470597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67740039c0 0x7f6774003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.719+0000 7f67867fc640 1 -- 172.21.15.27:0/4207470597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67740052a0 msgr2=0x7f677400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.719+0000 7f67867fc640 1 --2- 172.21.15.27:0/4207470597 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67740052a0 0x7f677400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.719+0000 7f67867fc640 1 -- 172.21.15.27:0/4207470597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6774003520 con 0x7f67740047c0 2023-12-19T18:15:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.719+0000 7f67877fe640 1 -- 172.21.15.27:0/4207470597 reap_dead start 2023-12-19T18:15:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.719+0000 7f6786ffd640 1 --2- 172.21.15.27:0/4207470597 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67740039c0 0x7f6774003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.720+0000 7f67867fc640 1 --2- 172.21.15.27:0/4207470597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67740047c0 0x7f6774004c40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6778004480 tx=0x7f67740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf800140dba77169 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.720+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778031680 con 0x7f67740047c0 2023-12-19T18:15:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.720+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67780317e0 con 0x7f67740047c0 2023-12-19T18:15:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.720+0000 7f6785ffb640 1 -- 172.21.15.27:0/4207470597 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6778031bd0 con 0x7f67740047c0 2023-12-19T18:15:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.721+0000 7f6787fff640 1 -- 172.21.15.27:0/4207470597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67740047c0 msgr2=0x7f6774004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.721+0000 7f6787fff640 1 --2- 172.21.15.27:0/4207470597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67740047c0 0x7f6774004c40 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6778004480 tx=0x7f67740057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.721+0000 7f6787fff640 1 -- 172.21.15.27:0/4207470597 shutdown_connections 2023-12-19T18:15:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.721+0000 7f6787fff640 1 --2- 172.21.15.27:0/4207470597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67740047c0 0x7f6774004c40 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.721+0000 7f6787fff640 1 -- 172.21.15.27:0/4207470597 >> 172.21.15.27:0/4207470597 conn(0x7f67880d2d30 msgr2=0x7f67880d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.721+0000 7f6787fff640 1 -- 172.21.15.27:0/4207470597 shutdown_connections 2023-12-19T18:15:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.721+0000 7f6787fff640 1 -- 172.21.15.27:0/4207470597 wait complete. 2023-12-19T18:15:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.723+0000 7f6787fff640 1 Processor -- start 2023-12-19T18:15:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.723+0000 7f6787fff640 1 -- start start 2023-12-19T18:15:39.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.724+0000 7f6787fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67740047c0 0x7f67880afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.724+0000 7f6787fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67880b8430 0x7f67880b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.724+0000 7f6787fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6788159eb0 0x7f678815a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.724+0000 7f6787fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678815a870 con 0x7f6788159eb0 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.724+0000 7f6787fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f678815da00 con 0x7f67880b8430 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.724+0000 7f6787fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f678815dce0 con 0x7f67740047c0 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.724+0000 7f67877fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6788159eb0 0x7f678815a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.724+0000 7f67877fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6788159eb0 0x7f678815a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58584/0 (socket says 172.21.15.27:58584) 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.724+0000 7f67877fe640 1 -- 172.21.15.27:0/1729793437 learned_addr learned my addr 172.21.15.27:0/1729793437 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.725+0000 7f67867fc640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67880b8430 0x7f67880b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.725+0000 7f6786ffd640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67740047c0 0x7f67880afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.725+0000 7f67877fe640 1 -- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67740047c0 msgr2=0x7f67880afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.725+0000 7f67877fe640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67740047c0 0x7f67880afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.725+0000 7f67877fe640 1 -- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67880b8430 msgr2=0x7f67880b88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.725+0000 7f67877fe640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67880b8430 0x7f67880b88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.725+0000 7f67877fe640 1 -- 172.21.15.27:0/1729793437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f678815dfc0 con 0x7f6788159eb0 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.725+0000 7f67877fe640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6788159eb0 0x7f678815a330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6788071e40 tx=0x7f678000e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:39.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.725+0000 7f677ffff640 1 -- 172.21.15.27:0/1729793437 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780004240 con 0x7f6788159eb0 2023-12-19T18:15:39.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.725+0000 7f677ffff640 1 -- 172.21.15.27:0/1729793437 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67800043c0 con 0x7f6788159eb0 2023-12-19T18:15:39.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.726+0000 7f6787fff640 1 -- 172.21.15.27:0/1729793437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f678815e2a0 con 0x7f6788159eb0 2023-12-19T18:15:39.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.726+0000 7f677ffff640 1 -- 172.21.15.27:0/1729793437 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6780003850 con 0x7f6788159eb0 2023-12-19T18:15:39.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.726+0000 7f6787fff640 1 -- 172.21.15.27:0/1729793437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f678815d640 con 0x7f6788159eb0 2023-12-19T18:15:39.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.727+0000 7f6787fff640 1 -- 172.21.15.27:0/1729793437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f678804e840 con 0x7f6788159eb0 2023-12-19T18:15:39.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.728+0000 7f677ffff640 1 -- 172.21.15.27:0/1729793437 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f67800039b0 con 0x7f6788159eb0 2023-12-19T18:15:39.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.728+0000 7f677ffff640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f676806a430 0x7f676806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:39.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.729+0000 7f677ffff640 1 -- 172.21.15.27:0/1729793437 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67800886d0 con 0x7f6788159eb0 2023-12-19T18:15:39.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.729+0000 7f6786ffd640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f676806a430 0x7f676806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:39.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.730+0000 7f6786ffd640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f676806a430 0x7f676806c8f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6770001eb0 tx=0x7f6770009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:39.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.733+0000 7f677ffff640 1 -- 172.21.15.27:0/1729793437 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6780053d70 con 0x7f6788159eb0 2023-12-19T18:15:39.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.928+0000 7f6787fff640 1 -- 172.21.15.27:0/1729793437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f67880cd160 con 0x7f6788159eb0 2023-12-19T18:15:39.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.928+0000 7f677ffff640 1 -- 172.21.15.27:0/1729793437 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6780011350 con 0x7f6788159eb0 2023-12-19T18:15:39.928 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:15:39.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.931+0000 7f6787fff640 1 -- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f676806a430 msgr2=0x7f676806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:39.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.931+0000 7f6787fff640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f676806a430 0x7f676806c8f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f6770001eb0 tx=0x7f6770009040 comp rx=0 tx=0).stop 2023-12-19T18:15:39.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.931+0000 7f6787fff640 1 -- 172.21.15.27:0/1729793437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6788159eb0 msgr2=0x7f678815a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:39.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.931+0000 7f6787fff640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6788159eb0 0x7f678815a330 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6788071e40 tx=0x7f678000e910 comp rx=0 tx=0).stop 2023-12-19T18:15:39.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.931+0000 7f6787fff640 1 -- 172.21.15.27:0/1729793437 shutdown_connections 2023-12-19T18:15:39.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.931+0000 7f6787fff640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f676806a430 0x7f676806c8f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.931+0000 7f6787fff640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6788159eb0 0x7f678815a330 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.931+0000 7f6787fff640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67880b8430 0x7f67880b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.931+0000 7f6787fff640 1 --2- 172.21.15.27:0/1729793437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67740047c0 0x7f67880afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:39.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.931+0000 7f6787fff640 1 -- 172.21.15.27:0/1729793437 >> 172.21.15.27:0/1729793437 conn(0x7f67880d2d30 msgr2=0x7f678806a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:39.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.931+0000 7f6787fff640 1 -- 172.21.15.27:0/1729793437 shutdown_connections 2023-12-19T18:15:39.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:39.931+0000 7f6787fff640 1 -- 172.21.15.27:0/1729793437 wait complete. 2023-12-19T18:15:39.952 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:40.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.175+0000 7f8bc359e640 1 Processor -- start 2023-12-19T18:15:40.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.176+0000 7f8bc359e640 1 -- start start 2023-12-19T18:15:40.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.177+0000 7f8bc359e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8bc41553a0 0x7f8bc40c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.177+0000 7f8bc359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8bc40afeb0 0x7f8bc415e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.177+0000 7f8bc359e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bc415f090 con 0x7f8bc415ec60 2023-12-19T18:15:40.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.177+0000 7f8bc359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc40cbcf0 con 0x7f8bc40afeb0 2023-12-19T18:15:40.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.177+0000 7f8bc359e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bc40b0360 con 0x7f8bc41553a0 2023-12-19T18:15:40.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.178+0000 7f8bbbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8bc40afeb0 0x7f8bc415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:40.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.178+0000 7f8bbbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8bc40afeb0 0x7f8bc415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58596/0 (socket says 172.21.15.27:58596) 2023-12-19T18:15:40.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.178+0000 7f8bbbfff640 1 -- 172.21.15.27:0/4000250512 learned_addr learned my addr 172.21.15.27:0/4000250512 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:40.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.178+0000 7f8bc259c640 1 --2- 172.21.15.27:0/4000250512 >> v2:172.21.15.192:3300/0 conn(0x7f8bc41553a0 0x7f8bc40c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:40.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.178+0000 7f8bbbfff640 1 -- 172.21.15.27:0/4000250512 >> v2:172.21.15.192:3300/0 conn(0x7f8bc41553a0 msgr2=0x7f8bc40c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:40.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.178+0000 7f8bbbfff640 1 --2- 172.21.15.27:0/4000250512 >> v2:172.21.15.192:3300/0 conn(0x7f8bc41553a0 0x7f8bc40c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.178+0000 7f8bbbfff640 1 -- 172.21.15.27:0/4000250512 >> v1:172.21.15.192:6789/0 conn(0x7f8bc415ec60 legacy=0x7f8bc415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:40.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.178+0000 7f8bbbfff640 1 -- 172.21.15.27:0/4000250512 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bc40cb350 con 0x7f8bc40afeb0 2023-12-19T18:15:40.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.178+0000 7f8bc259c640 1 --2- 172.21.15.27:0/4000250512 >> v2:172.21.15.192:3300/0 conn(0x7f8bc41553a0 0x7f8bc40c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:40.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.180+0000 7f8bbbfff640 1 --2- 172.21.15.27:0/4000250512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc40afeb0 0x7f8bc415e720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8bc406d4a0 tx=0x7f8bac02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694a3cae6c848a30 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:40.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.180+0000 7f8bc1d9b640 1 -- 172.21.15.27:0/4000250512 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac0317c0 con 0x7f8bc40afeb0 2023-12-19T18:15:40.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.180+0000 7f8bc1d9b640 1 -- 172.21.15.27:0/4000250512 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bac031920 con 0x7f8bc40afeb0 2023-12-19T18:15:40.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.180+0000 7f8bc1d9b640 1 -- 172.21.15.27:0/4000250512 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bac031cb0 con 0x7f8bc40afeb0 2023-12-19T18:15:40.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.181+0000 7f8bc359e640 1 -- 172.21.15.27:0/4000250512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc40afeb0 msgr2=0x7f8bc415e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:40.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.181+0000 7f8bc359e640 1 --2- 172.21.15.27:0/4000250512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc40afeb0 0x7f8bc415e720 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f8bc406d4a0 tx=0x7f8bac02b860 comp rx=0 tx=0).stop 2023-12-19T18:15:40.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.181+0000 7f8bc359e640 1 -- 172.21.15.27:0/4000250512 shutdown_connections 2023-12-19T18:15:40.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.181+0000 7f8bc359e640 1 --2- 172.21.15.27:0/4000250512 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc40afeb0 0x7f8bc415e720 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.181+0000 7f8bc359e640 1 --2- 172.21.15.27:0/4000250512 >> v2:172.21.15.192:3300/0 conn(0x7f8bc41553a0 0x7f8bc40c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.181+0000 7f8bc359e640 1 -- 172.21.15.27:0/4000250512 >> 172.21.15.27:0/4000250512 conn(0x7f8bc4069c70 msgr2=0x7f8bc406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:40.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.181+0000 7f8bc359e640 1 -- 172.21.15.27:0/4000250512 shutdown_connections 2023-12-19T18:15:40.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.181+0000 7f8bc359e640 1 -- 172.21.15.27:0/4000250512 wait complete. 2023-12-19T18:15:40.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.183+0000 7f8bc359e640 1 Processor -- start 2023-12-19T18:15:40.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.183+0000 7f8bc359e640 1 -- start start 2023-12-19T18:15:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc40afeb0 0x7f8bc415cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bc41553a0 0x7f8bc4152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc415ec60 0x7f8bc40ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc415d850 con 0x7f8bc40afeb0 2023-12-19T18:15:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc415db30 con 0x7f8bc415ec60 2023-12-19T18:15:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bc415de10 con 0x7f8bc41553a0 2023-12-19T18:15:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc40afeb0 0x7f8bc415cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc40afeb0 0x7f8bc415cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58608/0 (socket says 172.21.15.27:58608) 2023-12-19T18:15:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc259c640 1 -- 172.21.15.27:0/4108872302 learned_addr learned my addr 172.21.15.27:0/4108872302 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bbbfff640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bc41553a0 0x7f8bc4152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:40.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc2d9d640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc415ec60 0x7f8bc40ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc259c640 1 -- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bc41553a0 msgr2=0x7f8bc4152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc259c640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bc41553a0 0x7f8bc4152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc259c640 1 -- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc415ec60 msgr2=0x7f8bc40ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc259c640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc415ec60 0x7f8bc40ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc259c640 1 -- 172.21.15.27:0/4108872302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bac0094d0 con 0x7f8bc40afeb0 2023-12-19T18:15:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bbbfff640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bc41553a0 0x7f8bc4152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.184+0000 7f8bc2d9d640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc415ec60 0x7f8bc40ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:40.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.185+0000 7f8bc259c640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc40afeb0 0x7f8bc415cf50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400c7f0 tx=0x7f8bb400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.185+0000 7f8bbaffd640 1 -- 172.21.15.27:0/4108872302 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb4007a30 con 0x7f8bc40afeb0 2023-12-19T18:15:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.185+0000 7f8bbaffd640 1 -- 172.21.15.27:0/4108872302 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bb4007b90 con 0x7f8bc40afeb0 2023-12-19T18:15:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.185+0000 7f8bbaffd640 1 -- 172.21.15.27:0/4108872302 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bb400ce40 con 0x7f8bc40afeb0 2023-12-19T18:15:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.185+0000 7f8bc359e640 1 -- 172.21.15.27:0/4108872302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bc40b2390 con 0x7f8bc40afeb0 2023-12-19T18:15:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.185+0000 7f8bc359e640 1 -- 172.21.15.27:0/4108872302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bc415d490 con 0x7f8bc40afeb0 2023-12-19T18:15:40.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.186+0000 7f8bc359e640 1 -- 172.21.15.27:0/4108872302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bc40b25d0 con 0x7f8bc40afeb0 2023-12-19T18:15:40.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.187+0000 7f8bbaffd640 1 -- 172.21.15.27:0/4108872302 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8bb40113f0 con 0x7f8bc40afeb0 2023-12-19T18:15:40.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.189+0000 7f8bbaffd640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ba406a430 0x7f8ba406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.189+0000 7f8bbaffd640 1 -- 172.21.15.27:0/4108872302 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8bb4088b70 con 0x7f8bc40afeb0 2023-12-19T18:15:40.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.189+0000 7f8bbbfff640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ba406a430 0x7f8ba406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:40.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.191+0000 7f8bbbfff640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ba406a430 0x7f8ba406c8f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8bac036000 tx=0x7f8bac03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:40.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.194+0000 7f8bbaffd640 1 -- 172.21.15.27:0/4108872302 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8bc40b25d0 con 0x7f8bc40afeb0 2023-12-19T18:15:40.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.391+0000 7f8bc359e640 1 -- 172.21.15.27:0/4108872302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bc40b25d0 con 0x7f8bc40afeb0 2023-12-19T18:15:40.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.394+0000 7f8bbaffd640 1 -- 172.21.15.27:0/4108872302 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f8bc40b25d0 con 0x7f8bc40afeb0 2023-12-19T18:15:40.394 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:40.395 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:55.039690+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:40.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.397+0000 7f8bc359e640 1 -- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ba406a430 msgr2=0x7f8ba406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:40.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.397+0000 7f8bc359e640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ba406a430 0x7f8ba406c8f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8bac036000 tx=0x7f8bac03d040 comp rx=0 tx=0).stop 2023-12-19T18:15:40.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.397+0000 7f8bc359e640 1 -- 172.21.15.27:0/4108872302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc40afeb0 msgr2=0x7f8bc415cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:40.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.397+0000 7f8bc359e640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc40afeb0 0x7f8bc415cf50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8bb400c7f0 tx=0x7f8bb400cbf0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.397+0000 7f8bc359e640 1 -- 172.21.15.27:0/4108872302 shutdown_connections 2023-12-19T18:15:40.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.397+0000 7f8bc359e640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ba406a430 0x7f8ba406c8f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.397+0000 7f8bc359e640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8bc415ec60 0x7f8bc40ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.397+0000 7f8bc359e640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8bc41553a0 0x7f8bc4152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.397+0000 7f8bc359e640 1 --2- 172.21.15.27:0/4108872302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bc40afeb0 0x7f8bc415cf50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.397+0000 7f8bc359e640 1 -- 172.21.15.27:0/4108872302 >> 172.21.15.27:0/4108872302 conn(0x7f8bc4069c70 msgr2=0x7f8bc40c4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:40.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.397+0000 7f8bc359e640 1 -- 172.21.15.27:0/4108872302 shutdown_connections 2023-12-19T18:15:40.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.397+0000 7f8bc359e640 1 -- 172.21.15.27:0/4108872302 wait complete. 2023-12-19T18:15:40.417 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:15:40.418 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:15:40.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.650+0000 7eff3359e640 1 Processor -- start 2023-12-19T18:15:40.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.650+0000 7eff3359e640 1 -- start start 2023-12-19T18:15:40.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.651+0000 7eff3359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff340ce150 0x7eff34167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.652+0000 7eff3359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff340d0e10 con 0x7eff3415cf80 2023-12-19T18:15:40.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.652+0000 7eff3359e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff340d1c70 con 0x7eff340d2100 2023-12-19T18:15:40.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.652+0000 7eff3359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340b8cf0 con 0x7eff340ce150 2023-12-19T18:15:40.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.652+0000 7eff3259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7eff3415cf80 0x7eff340b85e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37516/0 (socket says 172.21.15.27:37516) 2023-12-19T18:15:40.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.652+0000 7eff3259c640 1 -- 172.21.15.27:0/40350048 learned_addr learned my addr 172.21.15.27:0/40350048 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.652+0000 7eff32d9d640 1 --2- 172.21.15.27:0/40350048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff340ce150 0x7eff34167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.652+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2572506592 0 0) 0x7eff340d0e10 con 0x7eff3415cf80 2023-12-19T18:15:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.652+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff20003520 con 0x7eff3415cf80 2023-12-19T18:15:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3740540822 0 0) 0x7eff20003520 con 0x7eff3415cf80 2023-12-19T18:15:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff340ce150 msgr2=0x7eff34167ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3159a640 1 --2- 172.21.15.27:0/40350048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff340ce150 0x7eff34167ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 >> v1:172.21.15.192:6789/0 conn(0x7eff340d2100 legacy=0x7eff340cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff3415d360 con 0x7eff3415cf80 2023-12-19T18:15:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7eff240019f0 con 0x7eff3415cf80 2023-12-19T18:15:40.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 >> v1:172.21.15.27:6789/0 conn(0x7eff3415cf80 legacy=0x7eff340b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3159a640 1 --2- 172.21.15.27:0/40350048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff200039a0 0x7eff20003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3159a640 1 --2- 172.21.15.27:0/40350048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff200047c0 0x7eff20004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3159a640 1 --2- 172.21.15.27:0/40350048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff20005180 0x7eff2000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340b8cf0 con 0x7eff200039a0 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff2000c030 con 0x7eff20005180 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff2000c1f0 con 0x7eff200047c0 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff32d9d640 1 --2- 172.21.15.27:0/40350048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff340ce150 0x7eff34167ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.653+0000 7eff3259c640 1 --2- 172.21.15.27:0/40350048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff200039a0 0x7eff20003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.654+0000 7eff31d9b640 1 --2- 172.21.15.27:0/40350048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff200047c0 0x7eff20004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.654+0000 7eff3259c640 1 -- 172.21.15.27:0/40350048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff200047c0 msgr2=0x7eff20004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.654+0000 7eff3259c640 1 --2- 172.21.15.27:0/40350048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff200047c0 0x7eff20004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.654+0000 7eff3259c640 1 -- 172.21.15.27:0/40350048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff20005180 msgr2=0x7eff2000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.654+0000 7eff3259c640 1 --2- 172.21.15.27:0/40350048 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff20005180 0x7eff2000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.654+0000 7eff3259c640 1 -- 172.21.15.27:0/40350048 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff20003520 con 0x7eff200039a0 2023-12-19T18:15:40.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.654+0000 7eff31d9b640 1 --2- 172.21.15.27:0/40350048 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff200047c0 0x7eff20004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:40.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.655+0000 7eff3259c640 1 --2- 172.21.15.27:0/40350048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff200039a0 0x7eff20003dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff2400a820 tx=0x7eff200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f736422039ab684 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:40.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.655+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff240072d0 con 0x7eff200039a0 2023-12-19T18:15:40.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.656+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2402f070 con 0x7eff200039a0 2023-12-19T18:15:40.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.656+0000 7eff3159a640 1 -- 172.21.15.27:0/40350048 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff24007730 con 0x7eff200039a0 2023-12-19T18:15:40.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.656+0000 7eff3359e640 1 -- 172.21.15.27:0/40350048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff200039a0 msgr2=0x7eff20003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:40.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.656+0000 7eff3359e640 1 --2- 172.21.15.27:0/40350048 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff200039a0 0x7eff20003dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7eff2400a820 tx=0x7eff200057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.656+0000 7eff32d9d640 1 -- 172.21.15.27:0/40350048 reap_dead start 2023-12-19T18:15:40.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.656+0000 7eff3359e640 1 -- 172.21.15.27:0/40350048 shutdown_connections 2023-12-19T18:15:40.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.656+0000 7eff3359e640 1 -- 172.21.15.27:0/40350048 >> 172.21.15.27:0/40350048 conn(0x7eff34072e20 msgr2=0x7eff34073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:40.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.656+0000 7eff3359e640 1 -- 172.21.15.27:0/40350048 shutdown_connections 2023-12-19T18:15:40.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.656+0000 7eff3359e640 1 -- 172.21.15.27:0/40350048 wait complete. 2023-12-19T18:15:40.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.658+0000 7eff3359e640 1 Processor -- start 2023-12-19T18:15:40.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.658+0000 7eff3359e640 1 -- start start 2023-12-19T18:15:40.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.659+0000 7eff3359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff200039a0 0x7eff341f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.659+0000 7eff3359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff340aa570 0x7eff340ab2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.659+0000 7eff3359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff341f2e70 0x7eff340ab810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.659+0000 7eff3359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340aa0e0 con 0x7eff200039a0 2023-12-19T18:15:40.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.659+0000 7eff3359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff341603b0 con 0x7eff340aa570 2023-12-19T18:15:40.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.659+0000 7eff3359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340a97b0 con 0x7eff341f2e70 2023-12-19T18:15:40.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.660+0000 7eff3259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff200039a0 0x7eff341f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:40.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.660+0000 7eff3259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff200039a0 0x7eff341f2930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58630/0 (socket says 172.21.15.27:58630) 2023-12-19T18:15:40.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.660+0000 7eff3259c640 1 -- 172.21.15.27:0/3142996591 learned_addr learned my addr 172.21.15.27:0/3142996591 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:40.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.660+0000 7eff31d9b640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff340aa570 0x7eff340ab2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:40.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.660+0000 7eff32d9d640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff341f2e70 0x7eff340ab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:40.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.660+0000 7eff3259c640 1 -- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff341f2e70 msgr2=0x7eff340ab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:40.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.660+0000 7eff3259c640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff341f2e70 0x7eff340ab810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.660+0000 7eff3259c640 1 -- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff340aa570 msgr2=0x7eff340ab2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:40.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.660+0000 7eff3259c640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff340aa570 0x7eff340ab2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.660+0000 7eff3259c640 1 -- 172.21.15.27:0/3142996591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff340a9a90 con 0x7eff200039a0 2023-12-19T18:15:40.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.660+0000 7eff31d9b640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff340aa570 0x7eff340ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:40.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.661+0000 7eff3259c640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff200039a0 0x7eff341f2930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7eff2400a940 tx=0x7eff340d1be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:40.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.661+0000 7eff2affd640 1 -- 172.21.15.27:0/3142996591 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff240072e0 con 0x7eff200039a0 2023-12-19T18:15:40.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.661+0000 7eff2affd640 1 -- 172.21.15.27:0/3142996591 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff24007440 con 0x7eff200039a0 2023-12-19T18:15:40.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.661+0000 7eff2affd640 1 -- 172.21.15.27:0/3142996591 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff240077b0 con 0x7eff200039a0 2023-12-19T18:15:40.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.661+0000 7eff3359e640 1 -- 172.21.15.27:0/3142996591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff340a9d70 con 0x7eff200039a0 2023-12-19T18:15:40.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.661+0000 7eff3359e640 1 -- 172.21.15.27:0/3142996591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff3415f9b0 con 0x7eff200039a0 2023-12-19T18:15:40.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.663+0000 7eff3359e640 1 -- 172.21.15.27:0/3142996591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3404e840 con 0x7eff200039a0 2023-12-19T18:15:40.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.663+0000 7eff2affd640 1 -- 172.21.15.27:0/3142996591 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7eff24031030 con 0x7eff200039a0 2023-12-19T18:15:40.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.664+0000 7eff2affd640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff1406a460 0x7eff1406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:40.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.664+0000 7eff2affd640 1 -- 172.21.15.27:0/3142996591 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff240aa360 con 0x7eff200039a0 2023-12-19T18:15:40.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.665+0000 7eff31d9b640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff1406a460 0x7eff1406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:40.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.666+0000 7eff31d9b640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff1406a460 0x7eff1406c920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff341f6820 tx=0x7eff1c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:40.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.669+0000 7eff2affd640 1 -- 172.21.15.27:0/3142996591 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7eff24075950 con 0x7eff200039a0 2023-12-19T18:15:40.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.862+0000 7eff3359e640 1 -- 172.21.15.27:0/3142996591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff340d2730 con 0x7eff200039a0 2023-12-19T18:15:40.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.863+0000 7eff2affd640 1 -- 172.21.15.27:0/3142996591 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff24078f70 con 0x7eff200039a0 2023-12-19T18:15:40.863 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:15:40.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.865+0000 7eff3359e640 1 -- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff1406a460 msgr2=0x7eff1406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:40.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.865+0000 7eff3359e640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff1406a460 0x7eff1406c920 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7eff341f6820 tx=0x7eff1c009040 comp rx=0 tx=0).stop 2023-12-19T18:15:40.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.866+0000 7eff3359e640 1 -- 172.21.15.27:0/3142996591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff200039a0 msgr2=0x7eff341f2930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:40.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.866+0000 7eff3359e640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff200039a0 0x7eff341f2930 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7eff2400a940 tx=0x7eff340d1be0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.866+0000 7eff3359e640 1 -- 172.21.15.27:0/3142996591 shutdown_connections 2023-12-19T18:15:40.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.866+0000 7eff3359e640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff1406a460 0x7eff1406c920 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.866+0000 7eff3359e640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff341f2e70 0x7eff340ab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.866+0000 7eff3359e640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff340aa570 0x7eff340ab2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.866+0000 7eff3359e640 1 --2- 172.21.15.27:0/3142996591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff200039a0 0x7eff341f2930 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:40.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.866+0000 7eff3359e640 1 -- 172.21.15.27:0/3142996591 >> 172.21.15.27:0/3142996591 conn(0x7eff34072e20 msgr2=0x7eff340d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:40.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.866+0000 7eff3359e640 1 -- 172.21.15.27:0/3142996591 shutdown_connections 2023-12-19T18:15:40.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:40.866+0000 7eff3359e640 1 -- 172.21.15.27:0/3142996591 wait complete. 2023-12-19T18:15:40.886 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:15:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.125+0000 7fde51b69640 1 Processor -- start 2023-12-19T18:15:41.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.125+0000 7fde51b69640 1 -- start start 2023-12-19T18:15:41.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.127+0000 7fde51b69640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde4c151c90 0x7fde4c06b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.127+0000 7fde51b69640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fde4c0cf550 0x7fde4c15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.127+0000 7fde51b69640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4c15b3e0 con 0x7fde4c15e9d0 2023-12-19T18:15:41.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.127+0000 7fde51b69640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde4c15eed0 con 0x7fde4c0cf550 2023-12-19T18:15:41.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.127+0000 7fde51b69640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c15f1b0 con 0x7fde4c151c90 2023-12-19T18:15:41.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.128+0000 7fde4bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fde4c15e9d0 0x7fde4c0cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37526/0 (socket says 172.21.15.27:37526) 2023-12-19T18:15:41.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.128+0000 7fde4bfff640 1 -- 172.21.15.27:0/2501116904 learned_addr learned my addr 172.21.15.27:0/2501116904 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:41.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.128+0000 7fde50b67640 1 --2- 172.21.15.27:0/2501116904 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde4c151c90 0x7fde4c06b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:41.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.128+0000 7fde51368640 1 --2- 172.21.15.27:0/2501116904 >> v2:172.21.15.192:3300/0 conn(0x7fde4c0cf550 0x7fde4c15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:41.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.128+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035567709 0 0) 0x7fde4c15b3e0 con 0x7fde4c15e9d0 2023-12-19T18:15:41.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.128+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde3c003520 con 0x7fde4c15e9d0 2023-12-19T18:15:41.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3016933309 0 0) 0x7fde3c003520 con 0x7fde4c15e9d0 2023-12-19T18:15:41.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde4c151c90 msgr2=0x7fde4c06b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 --2- 172.21.15.27:0/2501116904 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde4c151c90 0x7fde4c06b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 >> v2:172.21.15.192:3300/0 conn(0x7fde4c0cf550 msgr2=0x7fde4c15b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 --2- 172.21.15.27:0/2501116904 >> v2:172.21.15.192:3300/0 conn(0x7fde4c0cf550 0x7fde4c15b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c15f490 con 0x7fde4c15e9d0 2023-12-19T18:15:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fde38001670 con 0x7fde4c15e9d0 2023-12-19T18:15:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 >> v1:172.21.15.27:6789/0 conn(0x7fde4c15e9d0 legacy=0x7fde4c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 --2- 172.21.15.27:0/2501116904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c003970 0x7fde3c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 --2- 172.21.15.27:0/2501116904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde3c0047c0 0x7fde3c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 --2- 172.21.15.27:0/2501116904 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde4c151c90 0x7fde3c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c15eed0 con 0x7fde3c003970 2023-12-19T18:15:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c15f1b0 con 0x7fde3c0047c0 2023-12-19T18:15:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c00b9d0 con 0x7fde4c151c90 2023-12-19T18:15:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.129+0000 7fde50b67640 1 --2- 172.21.15.27:0/2501116904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c003970 0x7fde3c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.130+0000 7fde50b67640 1 -- 172.21.15.27:0/2501116904 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde4c151c90 msgr2=0x7fde3c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.130+0000 7fde50b67640 1 --2- 172.21.15.27:0/2501116904 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde4c151c90 0x7fde3c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.130+0000 7fde4bfff640 1 --2- 172.21.15.27:0/2501116904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde3c0047c0 0x7fde3c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.130+0000 7fde50b67640 1 -- 172.21.15.27:0/2501116904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde3c0047c0 msgr2=0x7fde3c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.130+0000 7fde50b67640 1 --2- 172.21.15.27:0/2501116904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde3c0047c0 0x7fde3c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.130+0000 7fde50b67640 1 -- 172.21.15.27:0/2501116904 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c003520 con 0x7fde3c003970 2023-12-19T18:15:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.130+0000 7fde4bfff640 1 --2- 172.21.15.27:0/2501116904 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde3c0047c0 0x7fde3c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:41.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.131+0000 7fde50b67640 1 --2- 172.21.15.27:0/2501116904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c003970 0x7fde3c003d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fde40009af0 tx=0x7fde3c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17f6ebd1c80cfaf0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:41.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.131+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4002c710 con 0x7fde3c003970 2023-12-19T18:15:41.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.131+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde4002c870 con 0x7fde3c003970 2023-12-19T18:15:41.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.131+0000 7fde4b7fe640 1 -- 172.21.15.27:0/2501116904 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde4002cc00 con 0x7fde3c003970 2023-12-19T18:15:41.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.132+0000 7fde51b69640 1 -- 172.21.15.27:0/2501116904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c003970 msgr2=0x7fde3c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:41.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.132+0000 7fde51b69640 1 --2- 172.21.15.27:0/2501116904 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c003970 0x7fde3c003d90 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fde40009af0 tx=0x7fde3c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.132+0000 7fde51368640 1 -- 172.21.15.27:0/2501116904 reap_dead start 2023-12-19T18:15:41.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.132+0000 7fde51b69640 1 -- 172.21.15.27:0/2501116904 shutdown_connections 2023-12-19T18:15:41.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.132+0000 7fde51b69640 1 -- 172.21.15.27:0/2501116904 >> 172.21.15.27:0/2501116904 conn(0x7fde4c06a060 msgr2=0x7fde4c06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:41.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.132+0000 7fde51b69640 1 -- 172.21.15.27:0/2501116904 shutdown_connections 2023-12-19T18:15:41.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.132+0000 7fde51b69640 1 -- 172.21.15.27:0/2501116904 wait complete. 2023-12-19T18:15:41.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.134+0000 7fde51b69640 1 Processor -- start 2023-12-19T18:15:41.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.134+0000 7fde51b69640 1 -- start start 2023-12-19T18:15:41.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.135+0000 7fde51b69640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c003970 0x7fde4c15c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.135+0000 7fde51b69640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1e6e20 0x7fde4c152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.135+0000 7fde51b69640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde4c1e72a0 0x7fde4c0b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.135+0000 7fde51b69640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c15d350 con 0x7fde4c1e6e20 2023-12-19T18:15:41.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.135+0000 7fde51b69640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c15ca20 con 0x7fde4c1e72a0 2023-12-19T18:15:41.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.135+0000 7fde51b69640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c15cd00 con 0x7fde3c003970 2023-12-19T18:15:41.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.135+0000 7fde4bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1e6e20 0x7fde4c152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:41.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.136+0000 7fde4bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1e6e20 0x7fde4c152070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58642/0 (socket says 172.21.15.27:58642) 2023-12-19T18:15:41.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.136+0000 7fde4bfff640 1 -- 172.21.15.27:0/3780181444 learned_addr learned my addr 172.21.15.27:0/3780181444 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:41.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.136+0000 7fde50b67640 1 --2- 172.21.15.27:0/3780181444 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c003970 0x7fde4c15c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:41.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.136+0000 7fde4bfff640 1 -- 172.21.15.27:0/3780181444 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c003970 msgr2=0x7fde4c15c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:41.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.136+0000 7fde4bfff640 1 --2- 172.21.15.27:0/3780181444 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c003970 0x7fde4c15c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.136+0000 7fde4bfff640 1 -- 172.21.15.27:0/3780181444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde4c1e72a0 msgr2=0x7fde4c0b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:41.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.136+0000 7fde4bfff640 1 --2- 172.21.15.27:0/3780181444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde4c1e72a0 0x7fde4c0b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.136+0000 7fde4bfff640 1 -- 172.21.15.27:0/3780181444 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c15cfe0 con 0x7fde4c1e6e20 2023-12-19T18:15:41.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.136+0000 7fde4bfff640 1 --2- 172.21.15.27:0/3780181444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1e6e20 0x7fde4c152070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fde3800d8b0 tx=0x7fde3800dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:41.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.137+0000 7fde497fa640 1 -- 172.21.15.27:0/3780181444 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde380074e0 con 0x7fde4c1e6e20 2023-12-19T18:15:41.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.137+0000 7fde497fa640 1 -- 172.21.15.27:0/3780181444 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde38007640 con 0x7fde4c1e6e20 2023-12-19T18:15:41.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.137+0000 7fde497fa640 1 -- 172.21.15.27:0/3780181444 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde380049e0 con 0x7fde4c1e6e20 2023-12-19T18:15:41.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.137+0000 7fde51b69640 1 -- 172.21.15.27:0/3780181444 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c1e7750 con 0x7fde4c1e6e20 2023-12-19T18:15:41.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.137+0000 7fde51b69640 1 -- 172.21.15.27:0/3780181444 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c0b0f50 con 0x7fde4c1e6e20 2023-12-19T18:15:41.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.138+0000 7fde51b69640 1 -- 172.21.15.27:0/3780181444 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c04e840 con 0x7fde4c1e6e20 2023-12-19T18:15:41.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.139+0000 7fde497fa640 1 -- 172.21.15.27:0/3780181444 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fde38004b40 con 0x7fde4c1e6e20 2023-12-19T18:15:41.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.140+0000 7fde497fa640 1 --2- 172.21.15.27:0/3780181444 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde3006a430 0x7fde3006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.141+0000 7fde497fa640 1 -- 172.21.15.27:0/3780181444 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde38088340 con 0x7fde4c1e6e20 2023-12-19T18:15:41.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.141+0000 7fde50b67640 1 --2- 172.21.15.27:0/3780181444 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde3006a430 0x7fde3006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:41.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.142+0000 7fde50b67640 1 --2- 172.21.15.27:0/3780181444 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde3006a430 0x7fde3006c8f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fde40036000 tx=0x7fde40030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:41.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.145+0000 7fde497fa640 1 -- 172.21.15.27:0/3780181444 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fde38053890 con 0x7fde4c1e6e20 2023-12-19T18:15:41.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.341+0000 7fde51b69640 1 -- 172.21.15.27:0/3780181444 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fde4c0cdc00 con 0x7fde4c1e6e20 2023-12-19T18:15:41.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.342+0000 7fde497fa640 1 -- 172.21.15.27:0/3780181444 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fde38001d40 con 0x7fde4c1e6e20 2023-12-19T18:15:41.342 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:15:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.344+0000 7fde51b69640 1 -- 172.21.15.27:0/3780181444 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde3006a430 msgr2=0x7fde3006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.344+0000 7fde51b69640 1 --2- 172.21.15.27:0/3780181444 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde3006a430 0x7fde3006c8f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fde40036000 tx=0x7fde40030000 comp rx=0 tx=0).stop 2023-12-19T18:15:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.344+0000 7fde51b69640 1 -- 172.21.15.27:0/3780181444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1e6e20 msgr2=0x7fde4c152070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.344+0000 7fde51b69640 1 --2- 172.21.15.27:0/3780181444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1e6e20 0x7fde4c152070 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fde3800d8b0 tx=0x7fde3800dd70 comp rx=0 tx=0).stop 2023-12-19T18:15:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.344+0000 7fde51b69640 1 -- 172.21.15.27:0/3780181444 shutdown_connections 2023-12-19T18:15:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.345+0000 7fde51b69640 1 --2- 172.21.15.27:0/3780181444 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde3006a430 0x7fde3006c8f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.345+0000 7fde51b69640 1 --2- 172.21.15.27:0/3780181444 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde4c1e72a0 0x7fde4c0b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.345+0000 7fde51b69640 1 --2- 172.21.15.27:0/3780181444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4c1e6e20 0x7fde4c152070 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.345+0000 7fde51b69640 1 --2- 172.21.15.27:0/3780181444 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c003970 0x7fde4c15c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.345+0000 7fde51b69640 1 -- 172.21.15.27:0/3780181444 >> 172.21.15.27:0/3780181444 conn(0x7fde4c06a060 msgr2=0x7fde4c0b3570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.345+0000 7fde51b69640 1 -- 172.21.15.27:0/3780181444 shutdown_connections 2023-12-19T18:15:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.345+0000 7fde51b69640 1 -- 172.21.15.27:0/3780181444 wait complete. 2023-12-19T18:15:41.366 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:15:41.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.595+0000 7f785b927640 1 Processor -- start 2023-12-19T18:15:41.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.595+0000 7f785b927640 1 -- start start 2023-12-19T18:15:41.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.597+0000 7f785b927640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785415b1a0 0x7f785406f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.597+0000 7f785b927640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f78540c2720 0x7f785406fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.597+0000 7f785b927640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f785406bc80 con 0x7f7854071420 2023-12-19T18:15:41.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.597+0000 7f785b927640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f785415fd90 con 0x7f78540c2720 2023-12-19T18:15:41.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.597+0000 7f785b927640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785406d690 con 0x7f785415b1a0 2023-12-19T18:15:41.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.597+0000 7f7858ec2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f78540c2720 0x7f785406fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:41.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.597+0000 7f7858ec2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f78540c2720 0x7f785406fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58654/0 (socket says 172.21.15.27:58654) 2023-12-19T18:15:41.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.597+0000 7f7858ec2640 1 -- 172.21.15.27:0/3727061284 learned_addr learned my addr 172.21.15.27:0/3727061284 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:41.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.597+0000 7f7858ec2640 1 -- 172.21.15.27:0/3727061284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785415b1a0 msgr2=0x7f785406f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:41.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.597+0000 7f78596c3640 1 --2- 172.21.15.27:0/3727061284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785415b1a0 0x7f785406f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:41.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.598+0000 7f7858ec2640 1 --2- 172.21.15.27:0/3727061284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785415b1a0 0x7f785406f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.598+0000 7f7858ec2640 1 -- 172.21.15.27:0/3727061284 >> v1:172.21.15.192:6789/0 conn(0x7f7854071420 legacy=0x7f78540c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:41.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.598+0000 7f7858ec2640 1 -- 172.21.15.27:0/3727061284 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f785406d0a0 con 0x7f78540c2720 2023-12-19T18:15:41.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.598+0000 7f78596c3640 1 --2- 172.21.15.27:0/3727061284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785415b1a0 0x7f785406f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:41.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.599+0000 7f7858ec2640 1 --2- 172.21.15.27:0/3727061284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78540c2720 0x7f785406fa60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7844009870 tx=0x7f784402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2757c3404848dc27 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:41.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.599+0000 7f784bfff640 1 -- 172.21.15.27:0/3727061284 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78440317c0 con 0x7f78540c2720 2023-12-19T18:15:41.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.599+0000 7f784bfff640 1 -- 172.21.15.27:0/3727061284 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7844031920 con 0x7f78540c2720 2023-12-19T18:15:41.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.599+0000 7f784bfff640 1 -- 172.21.15.27:0/3727061284 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7844031cb0 con 0x7f78540c2720 2023-12-19T18:15:41.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.600+0000 7f785b927640 1 -- 172.21.15.27:0/3727061284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78540c2720 msgr2=0x7f785406fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:41.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.600+0000 7f785b927640 1 --2- 172.21.15.27:0/3727061284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78540c2720 0x7f785406fa60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f7844009870 tx=0x7f784402b860 comp rx=0 tx=0).stop 2023-12-19T18:15:41.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.600+0000 7f785b927640 1 -- 172.21.15.27:0/3727061284 shutdown_connections 2023-12-19T18:15:41.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.600+0000 7f785b927640 1 --2- 172.21.15.27:0/3727061284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78540c2720 0x7f785406fa60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.600+0000 7f785b927640 1 --2- 172.21.15.27:0/3727061284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785415b1a0 0x7f785406f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.600+0000 7f785b927640 1 -- 172.21.15.27:0/3727061284 >> 172.21.15.27:0/3727061284 conn(0x7f78540cebd0 msgr2=0x7f78540cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:41.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.600+0000 7f785b927640 1 -- 172.21.15.27:0/3727061284 shutdown_connections 2023-12-19T18:15:41.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.600+0000 7f785b927640 1 -- 172.21.15.27:0/3727061284 wait complete. 2023-12-19T18:15:41.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.602+0000 7f785b927640 1 Processor -- start 2023-12-19T18:15:41.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.602+0000 7f785b927640 1 -- start start 2023-12-19T18:15:41.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.603+0000 7f785b927640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7854071420 0x7f78540af7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.603+0000 7f785b927640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78540c2720 0x7f78540afd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.603+0000 7f785b927640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785415b1a0 0x7f785414b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.603+0000 7f785b927640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78540a87f0 con 0x7f7854071420 2023-12-19T18:15:41.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.603+0000 7f785b927640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78540a84c0 con 0x7f78540c2720 2023-12-19T18:15:41.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.603+0000 7f785b927640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f785414c010 con 0x7f785415b1a0 2023-12-19T18:15:41.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.603+0000 7f78596c3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7854071420 0x7f78540af7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:41.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.603+0000 7f78596c3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7854071420 0x7f78540af7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58658/0 (socket says 172.21.15.27:58658) 2023-12-19T18:15:41.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.603+0000 7f78596c3640 1 -- 172.21.15.27:0/532080068 learned_addr learned my addr 172.21.15.27:0/532080068 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:41.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.604+0000 7f7859ec4640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785415b1a0 0x7f785414b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:41.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.604+0000 7f7858ec2640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78540c2720 0x7f78540afd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:41.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.604+0000 7f78596c3640 1 -- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785415b1a0 msgr2=0x7f785414b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:41.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.604+0000 7f78596c3640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785415b1a0 0x7f785414b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.604+0000 7f78596c3640 1 -- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78540c2720 msgr2=0x7f78540afd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:41.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.604+0000 7f78596c3640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78540c2720 0x7f78540afd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.604+0000 7f78596c3640 1 -- 172.21.15.27:0/532080068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78440094d0 con 0x7f7854071420 2023-12-19T18:15:41.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.604+0000 7f78596c3640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7854071420 0x7f78540af7f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f785000c7f0 tx=0x7f785000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:41.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.604+0000 7f7849ffb640 1 -- 172.21.15.27:0/532080068 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7850007820 con 0x7f7854071420 2023-12-19T18:15:41.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.605+0000 7f7849ffb640 1 -- 172.21.15.27:0/532080068 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7850007980 con 0x7f7854071420 2023-12-19T18:15:41.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.605+0000 7f7849ffb640 1 -- 172.21.15.27:0/532080068 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7850007cf0 con 0x7f7854071420 2023-12-19T18:15:41.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.605+0000 7f785b927640 1 -- 172.21.15.27:0/532080068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f785414c2f0 con 0x7f7854071420 2023-12-19T18:15:41.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.605+0000 7f785b927640 1 -- 172.21.15.27:0/532080068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78540ac010 con 0x7f7854071420 2023-12-19T18:15:41.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.606+0000 7f785b927640 1 -- 172.21.15.27:0/532080068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78540a8630 con 0x7f7854071420 2023-12-19T18:15:41.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.607+0000 7f7849ffb640 1 -- 172.21.15.27:0/532080068 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f78500113f0 con 0x7f7854071420 2023-12-19T18:15:41.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.608+0000 7f7849ffb640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f783806a460 0x7f783806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:41.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.608+0000 7f7849ffb640 1 -- 172.21.15.27:0/532080068 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7850088c40 con 0x7f7854071420 2023-12-19T18:15:41.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.609+0000 7f7858ec2640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f783806a460 0x7f783806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:41.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.610+0000 7f7858ec2640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f783806a460 0x7f783806c920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7844036000 tx=0x7f784403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:41.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.613+0000 7f7849ffb640 1 -- 172.21.15.27:0/532080068 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f78540a8630 con 0x7f7854071420 2023-12-19T18:15:41.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.808+0000 7f785b927640 1 -- 172.21.15.27:0/532080068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78540a8630 con 0x7f7854071420 2023-12-19T18:15:41.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.809+0000 7f7849ffb640 1 -- 172.21.15.27:0/532080068 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78540a8630 con 0x7f7854071420 2023-12-19T18:15:41.809 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:15:41.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.811+0000 7f785b927640 1 -- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f783806a460 msgr2=0x7f783806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:41.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.811+0000 7f785b927640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f783806a460 0x7f783806c920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7844036000 tx=0x7f784403d040 comp rx=0 tx=0).stop 2023-12-19T18:15:41.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.812+0000 7f785b927640 1 -- 172.21.15.27:0/532080068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7854071420 msgr2=0x7f78540af7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:41.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.812+0000 7f785b927640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7854071420 0x7f78540af7f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f785000c7f0 tx=0x7f785000ccb0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.812+0000 7f785b927640 1 -- 172.21.15.27:0/532080068 shutdown_connections 2023-12-19T18:15:41.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.812+0000 7f785b927640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f783806a460 0x7f783806c920 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.812+0000 7f785b927640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f785415b1a0 0x7f785414b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.812+0000 7f785b927640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f78540c2720 0x7f78540afd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.812+0000 7f785b927640 1 --2- 172.21.15.27:0/532080068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7854071420 0x7f78540af7f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:41.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.812+0000 7f785b927640 1 -- 172.21.15.27:0/532080068 >> 172.21.15.27:0/532080068 conn(0x7f78540cebd0 msgr2=0x7f78540cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:41.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.812+0000 7f785b927640 1 -- 172.21.15.27:0/532080068 shutdown_connections 2023-12-19T18:15:41.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:41.812+0000 7f785b927640 1 -- 172.21.15.27:0/532080068 wait complete. 2023-12-19T18:15:41.832 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:42.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.067+0000 7f1edb59e640 1 Processor -- start 2023-12-19T18:15:42.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.067+0000 7f1edb59e640 1 -- start start 2023-12-19T18:15:42.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.069+0000 7f1edb59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1604d0 0x7f1edc0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.069+0000 7f1edb59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1edc0d2350 0x7f1edc0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.069+0000 7f1edb59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1edc0b5e90 con 0x7f1edc0d0300 2023-12-19T18:15:42.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.069+0000 7f1edb59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1edc0d2b30 con 0x7f1edc0d2350 2023-12-19T18:15:42.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.069+0000 7f1edb59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc0d2850 con 0x7f1edc1604d0 2023-12-19T18:15:42.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.069+0000 7f1ed3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1edc0d2350 0x7f1edc0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.069+0000 7f1ed3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1edc0d2350 0x7f1edc0b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47788/0 (socket says 172.21.15.27:47788) 2023-12-19T18:15:42.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.069+0000 7f1ed3fff640 1 -- 172.21.15.27:0/4270514184 learned_addr learned my addr 172.21.15.27:0/4270514184 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:42.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.069+0000 7f1eda59c640 1 --2- 172.21.15.27:0/4270514184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1604d0 0x7f1edc0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.070+0000 7f1ed3fff640 1 -- 172.21.15.27:0/4270514184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1604d0 msgr2=0x7f1edc0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:42.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.070+0000 7f1ed3fff640 1 --2- 172.21.15.27:0/4270514184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1604d0 0x7f1edc0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.070+0000 7f1ed3fff640 1 -- 172.21.15.27:0/4270514184 >> v1:172.21.15.192:6789/0 conn(0x7f1edc0d0300 legacy=0x7f1edc0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:42.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.070+0000 7f1ed3fff640 1 -- 172.21.15.27:0/4270514184 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1edc0cded0 con 0x7f1edc0d2350 2023-12-19T18:15:42.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.071+0000 7f1ed3fff640 1 --2- 172.21.15.27:0/4270514184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1edc0d2350 0x7f1edc0b4d20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009870 tx=0x7f1ec402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=907663fe576996a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.071+0000 7f1ed9d9b640 1 -- 172.21.15.27:0/4270514184 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4031640 con 0x7f1edc0d2350 2023-12-19T18:15:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.071+0000 7f1ed9d9b640 1 -- 172.21.15.27:0/4270514184 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec40317a0 con 0x7f1edc0d2350 2023-12-19T18:15:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.071+0000 7f1ed9d9b640 1 -- 172.21.15.27:0/4270514184 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec4031b30 con 0x7f1edc0d2350 2023-12-19T18:15:42.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.072+0000 7f1edb59e640 1 -- 172.21.15.27:0/4270514184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1edc0d2350 msgr2=0x7f1edc0b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.072+0000 7f1edb59e640 1 --2- 172.21.15.27:0/4270514184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1edc0d2350 0x7f1edc0b4d20 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4009870 tx=0x7f1ec402b8f0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.072+0000 7f1edb59e640 1 -- 172.21.15.27:0/4270514184 shutdown_connections 2023-12-19T18:15:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.072+0000 7f1edb59e640 1 --2- 172.21.15.27:0/4270514184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1edc0d2350 0x7f1edc0b4d20 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.072+0000 7f1edb59e640 1 --2- 172.21.15.27:0/4270514184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1604d0 0x7f1edc0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.072+0000 7f1edb59e640 1 -- 172.21.15.27:0/4270514184 >> 172.21.15.27:0/4270514184 conn(0x7f1edc0cf7e0 msgr2=0x7f1edc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.072+0000 7f1edb59e640 1 -- 172.21.15.27:0/4270514184 shutdown_connections 2023-12-19T18:15:42.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.072+0000 7f1edb59e640 1 -- 172.21.15.27:0/4270514184 wait complete. 2023-12-19T18:15:42.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.074+0000 7f1edb59e640 1 Processor -- start 2023-12-19T18:15:42.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.074+0000 7f1edb59e640 1 -- start start 2023-12-19T18:15:42.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.075+0000 7f1edb59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1edc0d0300 0x7f1edc1f2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.075+0000 7f1edb59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1edc0d2350 0x7f1edc0b7750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.075+0000 7f1edb59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1604d0 0x7f1edc0b7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.075+0000 7f1edb59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc0ccbc0 con 0x7f1edc0d0300 2023-12-19T18:15:42.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.075+0000 7f1edb59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc14d9d0 con 0x7f1edc0d2350 2023-12-19T18:15:42.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.075+0000 7f1edb59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1edc14dcb0 con 0x7f1edc1604d0 2023-12-19T18:15:42.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.076+0000 7f1eda59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1edc0d0300 0x7f1edc1f2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.076+0000 7f1eda59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1edc0d0300 0x7f1edc1f2030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47804/0 (socket says 172.21.15.27:47804) 2023-12-19T18:15:42.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.076+0000 7f1eda59c640 1 -- 172.21.15.27:0/1300729950 learned_addr learned my addr 172.21.15.27:0/1300729950 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:42.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.076+0000 7f1ed3fff640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1edc0d2350 0x7f1edc0b7750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.076+0000 7f1edad9d640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1604d0 0x7f1edc0b7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.076+0000 7f1eda59c640 1 -- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1604d0 msgr2=0x7f1edc0b7c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:42.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.076+0000 7f1eda59c640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1604d0 0x7f1edc0b7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.076+0000 7f1eda59c640 1 -- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1edc0d2350 msgr2=0x7f1edc0b7750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.076+0000 7f1eda59c640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1edc0d2350 0x7f1edc0b7750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.076+0000 7f1eda59c640 1 -- 172.21.15.27:0/1300729950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec40094d0 con 0x7f1edc0d0300 2023-12-19T18:15:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.076+0000 7f1ed3fff640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1edc0d2350 0x7f1edc0b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:42.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.077+0000 7f1eda59c640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1edc0d0300 0x7f1edc1f2030 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00d800 tx=0x7f1ecc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.077+0000 7f1ed2ffd640 1 -- 172.21.15.27:0/1300729950 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00b6c0 con 0x7f1edc0d0300 2023-12-19T18:15:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.077+0000 7f1ed2ffd640 1 -- 172.21.15.27:0/1300729950 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ecc00b820 con 0x7f1edc0d0300 2023-12-19T18:15:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.077+0000 7f1edb59e640 1 -- 172.21.15.27:0/1300729950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1edc14df90 con 0x7f1edc0d0300 2023-12-19T18:15:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.077+0000 7f1ed2ffd640 1 -- 172.21.15.27:0/1300729950 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ecc00bbb0 con 0x7f1edc0d0300 2023-12-19T18:15:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.077+0000 7f1edb59e640 1 -- 172.21.15.27:0/1300729950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1edc14e790 con 0x7f1edc0d0300 2023-12-19T18:15:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.078+0000 7f1edb59e640 1 -- 172.21.15.27:0/1300729950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1edc066f80 con 0x7f1edc0d0300 2023-12-19T18:15:42.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.079+0000 7f1ed2ffd640 1 -- 172.21.15.27:0/1300729950 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1ecc00bd10 con 0x7f1edc0d0300 2023-12-19T18:15:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.080+0000 7f1ed2ffd640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ebc06a430 0x7f1ebc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.080+0000 7f1ed2ffd640 1 -- 172.21.15.27:0/1300729950 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1ecc088e70 con 0x7f1edc0d0300 2023-12-19T18:15:42.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.081+0000 7f1ed3fff640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ebc06a430 0x7f1ebc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.082+0000 7f1ed3fff640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ebc06a430 0x7f1ebc06c8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4036000 tx=0x7f1ec403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:42.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.085+0000 7f1ed2ffd640 1 -- 172.21.15.27:0/1300729950 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1ecc054450 con 0x7f1edc0d0300 2023-12-19T18:15:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.286+0000 7f1edb59e640 1 -- 172.21.15.27:0/1300729950 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1edc04e840 con 0x7f1edc0d0300 2023-12-19T18:15:42.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.289+0000 7f1ed2ffd640 1 -- 172.21.15.27:0/1300729950 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f1ecc057a70 con 0x7f1edc0d0300 2023-12-19T18:15:42.289 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:42.289 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":20,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:14:55.039690+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.291+0000 7f1edb59e640 1 -- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ebc06a430 msgr2=0x7f1ebc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.291+0000 7f1edb59e640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ebc06a430 0x7f1ebc06c8f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1ec4036000 tx=0x7f1ec403c040 comp rx=0 tx=0).stop 2023-12-19T18:15:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.291+0000 7f1edb59e640 1 -- 172.21.15.27:0/1300729950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1edc0d0300 msgr2=0x7f1edc1f2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.291+0000 7f1edb59e640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1edc0d0300 0x7f1edc1f2030 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1ecc00d800 tx=0x7f1ecc00dcc0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.292+0000 7f1edb59e640 1 -- 172.21.15.27:0/1300729950 shutdown_connections 2023-12-19T18:15:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.292+0000 7f1edb59e640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ebc06a430 0x7f1ebc06c8f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.292+0000 7f1edb59e640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1edc1604d0 0x7f1edc0b7c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.292+0000 7f1edb59e640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1edc0d2350 0x7f1edc0b7750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.292+0000 7f1edb59e640 1 --2- 172.21.15.27:0/1300729950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1edc0d0300 0x7f1edc1f2030 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.292+0000 7f1edb59e640 1 -- 172.21.15.27:0/1300729950 >> 172.21.15.27:0/1300729950 conn(0x7f1edc0cf7e0 msgr2=0x7f1edc15ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:42.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.292+0000 7f1edb59e640 1 -- 172.21.15.27:0/1300729950 shutdown_connections 2023-12-19T18:15:42.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.292+0000 7f1edb59e640 1 -- 172.21.15.27:0/1300729950 wait complete. 2023-12-19T18:15:42.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:15:42.311 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:15:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.543+0000 7fc7825dc640 1 Processor -- start 2023-12-19T18:15:42.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.543+0000 7fc7825dc640 1 -- start start 2023-12-19T18:15:42.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.544+0000 7fc7825dc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc77c0cb3e0 0x7fc77c0b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.545+0000 7fc7825dc640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77c0cd6b0 con 0x7fc77c15ae80 2023-12-19T18:15:42.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.545+0000 7fc7825dc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc77c0cdc70 con 0x7fc77c0b3be0 2023-12-19T18:15:42.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.545+0000 7fc7825dc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc77c0d0450 con 0x7fc77c0cb3e0 2023-12-19T18:15:42.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.545+0000 7fc77bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc77c15ae80 0x7fc77c0b2810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49274/0 (socket says 172.21.15.27:49274) 2023-12-19T18:15:42.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.545+0000 7fc77bfff640 1 -- 172.21.15.27:0/1263277864 learned_addr learned my addr 172.21.15.27:0/1263277864 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.545+0000 7fc77b7fe640 1 --2- 172.21.15.27:0/1263277864 >> v2:172.21.15.192:3300/0 conn(0x7fc77c0cb3e0 0x7fc77c0b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.545+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513360541 0 0) 0x7fc77c0cd6b0 con 0x7fc77c15ae80 2023-12-19T18:15:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.545+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc764003520 con 0x7fc77c15ae80 2023-12-19T18:15:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3013580586 0 0) 0x7fc764003520 con 0x7fc77c15ae80 2023-12-19T18:15:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 >> v2:172.21.15.192:3300/0 conn(0x7fc77c0cb3e0 msgr2=0x7fc77c0b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77affd640 1 --2- 172.21.15.27:0/1263277864 >> v2:172.21.15.192:3300/0 conn(0x7fc77c0cb3e0 0x7fc77c0b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 >> v1:172.21.15.192:6789/0 conn(0x7fc77c0b3be0 legacy=0x7fc77c0b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c0d0730 con 0x7fc77c15ae80 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fc76c001670 con 0x7fc77c15ae80 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 >> v1:172.21.15.27:6789/0 conn(0x7fc77c15ae80 legacy=0x7fc77c0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77affd640 1 --2- 172.21.15.27:0/1263277864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7640039c0 0x7fc764003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77affd640 1 --2- 172.21.15.27:0/1263277864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7640047c0 0x7fc764004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77affd640 1 --2- 172.21.15.27:0/1263277864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc764005180 0x7fc76400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0d0450 con 0x7fc7640039c0 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc76400c080 con 0x7fc764005180 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc76400c240 con 0x7fc7640047c0 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.546+0000 7fc77bfff640 1 --2- 172.21.15.27:0/1263277864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7640039c0 0x7fc764003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.547+0000 7fc77b7fe640 1 --2- 172.21.15.27:0/1263277864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7640047c0 0x7fc764004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.547+0000 7fc780b79640 1 --2- 172.21.15.27:0/1263277864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc764005180 0x7fc76400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.547+0000 7fc77bfff640 1 -- 172.21.15.27:0/1263277864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7640047c0 msgr2=0x7fc764004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:42.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.547+0000 7fc77bfff640 1 --2- 172.21.15.27:0/1263277864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7640047c0 0x7fc764004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.547+0000 7fc77bfff640 1 -- 172.21.15.27:0/1263277864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc764005180 msgr2=0x7fc76400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:42.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.547+0000 7fc77bfff640 1 --2- 172.21.15.27:0/1263277864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc764005180 0x7fc76400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.547+0000 7fc77bfff640 1 -- 172.21.15.27:0/1263277864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc764003520 con 0x7fc7640039c0 2023-12-19T18:15:42.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.547+0000 7fc780b79640 1 -- 172.21.15.27:0/1263277864 reap_dead start 2023-12-19T18:15:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.548+0000 7fc77bfff640 1 --2- 172.21.15.27:0/1263277864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7640039c0 0x7fc764003de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00c0e0 tx=0x7fc7640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1917cfdc706dec1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.548+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c005ad0 con 0x7fc7640039c0 2023-12-19T18:15:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.548+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc76c005c30 con 0x7fc7640039c0 2023-12-19T18:15:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.548+0000 7fc77affd640 1 -- 172.21.15.27:0/1263277864 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc76c02ca50 con 0x7fc7640039c0 2023-12-19T18:15:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.549+0000 7fc7825dc640 1 -- 172.21.15.27:0/1263277864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7640039c0 msgr2=0x7fc764003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.549+0000 7fc7825dc640 1 --2- 172.21.15.27:0/1263277864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7640039c0 0x7fc764003de0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00c0e0 tx=0x7fc7640057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.549+0000 7fc7825dc640 1 -- 172.21.15.27:0/1263277864 shutdown_connections 2023-12-19T18:15:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.549+0000 7fc7825dc640 1 --2- 172.21.15.27:0/1263277864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7640039c0 0x7fc764003de0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.549+0000 7fc7825dc640 1 -- 172.21.15.27:0/1263277864 >> 172.21.15.27:0/1263277864 conn(0x7fc77c06f1f0 msgr2=0x7fc77c06f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.549+0000 7fc7825dc640 1 -- 172.21.15.27:0/1263277864 shutdown_connections 2023-12-19T18:15:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.549+0000 7fc7825dc640 1 -- 172.21.15.27:0/1263277864 wait complete. 2023-12-19T18:15:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.551+0000 7fc7825dc640 1 Processor -- start 2023-12-19T18:15:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.551+0000 7fc7825dc640 1 -- start start 2023-12-19T18:15:42.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.552+0000 7fc7825dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7640039c0 0x7fc77c15c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.552+0000 7fc7825dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc77c15c6c0 0x7fc77c0c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.552+0000 7fc7825dc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77c0c9900 0x7fc77c15dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.552+0000 7fc7825dc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c0c9d80 con 0x7fc77c0c9900 2023-12-19T18:15:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.552+0000 7fc7825dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c15cc30 con 0x7fc7640039c0 2023-12-19T18:15:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.552+0000 7fc7825dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc77c15cf10 con 0x7fc77c15c6c0 2023-12-19T18:15:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.552+0000 7fc780b79640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77c0c9900 0x7fc77c15dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.552+0000 7fc780b79640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77c0c9900 0x7fc77c15dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47830/0 (socket says 172.21.15.27:47830) 2023-12-19T18:15:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.552+0000 7fc780b79640 1 -- 172.21.15.27:0/1393986338 learned_addr learned my addr 172.21.15.27:0/1393986338 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:42.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.553+0000 7fc77b7fe640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc77c15c6c0 0x7fc77c0c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.553+0000 7fc77bfff640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7640039c0 0x7fc77c15c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.553+0000 7fc780b79640 1 -- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc77c15c6c0 msgr2=0x7fc77c0c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.553+0000 7fc780b79640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc77c15c6c0 0x7fc77c0c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.553+0000 7fc780b79640 1 -- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7640039c0 msgr2=0x7fc77c15c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.553+0000 7fc780b79640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7640039c0 0x7fc77c15c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.553+0000 7fc780b79640 1 -- 172.21.15.27:0/1393986338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77c15d1f0 con 0x7fc77c0c9900 2023-12-19T18:15:42.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.553+0000 7fc780b79640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77c0c9900 0x7fc77c15dcd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0744e0 tx=0x7fc77400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:42.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.553+0000 7fc778ff9640 1 -- 172.21.15.27:0/1393986338 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774003b70 con 0x7fc77c0c9900 2023-12-19T18:15:42.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.553+0000 7fc778ff9640 1 -- 172.21.15.27:0/1393986338 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc774003cd0 con 0x7fc77c0c9900 2023-12-19T18:15:42.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.553+0000 7fc778ff9640 1 -- 172.21.15.27:0/1393986338 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc774005290 con 0x7fc77c0c9900 2023-12-19T18:15:42.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.554+0000 7fc7825dc640 1 -- 172.21.15.27:0/1393986338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc77c15d4d0 con 0x7fc77c0c9900 2023-12-19T18:15:42.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.554+0000 7fc7825dc640 1 -- 172.21.15.27:0/1393986338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77c0ad500 con 0x7fc77c0c9900 2023-12-19T18:15:42.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.555+0000 7fc7825dc640 1 -- 172.21.15.27:0/1393986338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc77c067110 con 0x7fc77c0c9900 2023-12-19T18:15:42.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.556+0000 7fc778ff9640 1 -- 172.21.15.27:0/1393986338 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc7740053f0 con 0x7fc77c0c9900 2023-12-19T18:15:42.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.557+0000 7fc778ff9640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc76806a460 0x7fc76806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:42.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.557+0000 7fc778ff9640 1 -- 172.21.15.27:0/1393986338 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc774088280 con 0x7fc77c0c9900 2023-12-19T18:15:42.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.558+0000 7fc77bfff640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc76806a460 0x7fc76806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:42.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.559+0000 7fc77bfff640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc76806a460 0x7fc76806c920 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc76c034000 tx=0x7fc76c009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:42.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.562+0000 7fc778ff9640 1 -- 172.21.15.27:0/1393986338 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc774053870 con 0x7fc77c0c9900 2023-12-19T18:15:42.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.764+0000 7fc7825dc640 1 -- 172.21.15.27:0/1393986338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc77c06bfb0 con 0x7fc77c0c9900 2023-12-19T18:15:42.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.764+0000 7fc778ff9640 1 -- 172.21.15.27:0/1393986338 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc7740180a0 con 0x7fc77c0c9900 2023-12-19T18:15:42.765 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:15:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.768+0000 7fc7825dc640 1 -- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc76806a460 msgr2=0x7fc76806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.768+0000 7fc7825dc640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc76806a460 0x7fc76806c920 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc76c034000 tx=0x7fc76c009030 comp rx=0 tx=0).stop 2023-12-19T18:15:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.768+0000 7fc7825dc640 1 -- 172.21.15.27:0/1393986338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77c0c9900 msgr2=0x7fc77c15dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.768+0000 7fc7825dc640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77c0c9900 0x7fc77c15dcd0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fc77c0744e0 tx=0x7fc77400dc00 comp rx=0 tx=0).stop 2023-12-19T18:15:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.768+0000 7fc7825dc640 1 -- 172.21.15.27:0/1393986338 shutdown_connections 2023-12-19T18:15:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.768+0000 7fc7825dc640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc76806a460 0x7fc76806c920 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.768+0000 7fc7825dc640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc77c0c9900 0x7fc77c15dcd0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.768+0000 7fc7825dc640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc77c15c6c0 0x7fc77c0c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.768+0000 7fc7825dc640 1 --2- 172.21.15.27:0/1393986338 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7640039c0 0x7fc77c15c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:42.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.768+0000 7fc7825dc640 1 -- 172.21.15.27:0/1393986338 >> 172.21.15.27:0/1393986338 conn(0x7fc77c06f1f0 msgr2=0x7fc77c06c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:42.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.768+0000 7fc7825dc640 1 -- 172.21.15.27:0/1393986338 shutdown_connections 2023-12-19T18:15:42.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:42.768+0000 7fc7825dc640 1 -- 172.21.15.27:0/1393986338 wait complete. 2023-12-19T18:15:42.788 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:15:43.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.024+0000 7f272e399640 1 Processor -- start 2023-12-19T18:15:43.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.025+0000 7f272e399640 1 -- start start 2023-12-19T18:15:43.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.026+0000 7f272e399640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f27280cf9b0 0x7f27280cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.026+0000 7f272e399640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f27280cda20 0x7f27280cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.026+0000 7f272e399640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27280d26d0 con 0x7f27280b8c30 2023-12-19T18:15:43.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.026+0000 7f272e399640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27280cea80 con 0x7f27280cda20 2023-12-19T18:15:43.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.026+0000 7f272e399640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27280b5f10 con 0x7f27280cf9b0 2023-12-19T18:15:43.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.027+0000 7f2727fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f27280b8c30 0x7f27280d1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49282/0 (socket says 172.21.15.27:49282) 2023-12-19T18:15:43.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.027+0000 7f2727fff640 1 -- 172.21.15.27:0/3364548023 learned_addr learned my addr 172.21.15.27:0/3364548023 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:43.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.027+0000 7f27277fe640 1 --2- 172.21.15.27:0/3364548023 >> v2:172.21.15.192:3300/0 conn(0x7f27280cf9b0 0x7f27280cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.027+0000 7f272c936640 1 --2- 172.21.15.27:0/3364548023 >> v2:172.21.15.27:3300/0 conn(0x7f27280cda20 0x7f27280cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.027+0000 7f2726ffd640 1 -- 172.21.15.27:0/3364548023 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3524218097 0 0) 0x7f27280d26d0 con 0x7f27280b8c30 2023-12-19T18:15:43.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.027+0000 7f2726ffd640 1 -- 172.21.15.27:0/3364548023 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2714003520 con 0x7f27280b8c30 2023-12-19T18:15:43.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.028+0000 7f272c936640 1 -- 172.21.15.27:0/3364548023 >> v2:172.21.15.192:3300/0 conn(0x7f27280cf9b0 msgr2=0x7f27280cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:43.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.028+0000 7f272c936640 1 --2- 172.21.15.27:0/3364548023 >> v2:172.21.15.192:3300/0 conn(0x7f27280cf9b0 0x7f27280cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.028+0000 7f272c936640 1 -- 172.21.15.27:0/3364548023 >> v1:172.21.15.27:6789/0 conn(0x7f27280b8c30 legacy=0x7f27280d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.028+0000 7f2726ffd640 1 -- 172.21.15.27:0/3364548023 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 916048310 0 0) 0x7f2714003520 con 0x7f27280b8c30 2023-12-19T18:15:43.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.028+0000 7f272c936640 1 -- 172.21.15.27:0/3364548023 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27280ce7a0 con 0x7f27280cda20 2023-12-19T18:15:43.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.029+0000 7f272c936640 1 --2- 172.21.15.27:0/3364548023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27280cda20 0x7f27280cde30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f27280ca000 tx=0x7f272002b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53756d87b6ff9d85 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:43.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.030+0000 7f2726ffd640 1 -- 172.21.15.27:0/3364548023 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27180019f0 con 0x7f27280cda20 2023-12-19T18:15:43.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.030+0000 7f2726ffd640 1 -- 172.21.15.27:0/3364548023 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272002e050 con 0x7f27280cda20 2023-12-19T18:15:43.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.030+0000 7f2726ffd640 1 -- 172.21.15.27:0/3364548023 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2720030710 con 0x7f27280cda20 2023-12-19T18:15:43.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.030+0000 7f272e399640 1 -- 172.21.15.27:0/3364548023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27280cda20 msgr2=0x7f27280cde30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.030+0000 7f272e399640 1 --2- 172.21.15.27:0/3364548023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27280cda20 0x7f27280cde30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f27280ca000 tx=0x7f272002b710 comp rx=0 tx=0).stop 2023-12-19T18:15:43.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.031+0000 7f272e399640 1 -- 172.21.15.27:0/3364548023 shutdown_connections 2023-12-19T18:15:43.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.031+0000 7f272e399640 1 --2- 172.21.15.27:0/3364548023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27280cda20 0x7f27280cde30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.031+0000 7f272e399640 1 --2- 172.21.15.27:0/3364548023 >> v2:172.21.15.192:3300/0 conn(0x7f27280cf9b0 0x7f27280cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.031+0000 7f272e399640 1 -- 172.21.15.27:0/3364548023 >> 172.21.15.27:0/3364548023 conn(0x7f27280d2fa0 msgr2=0x7f27280d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:43.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.031+0000 7f272e399640 1 -- 172.21.15.27:0/3364548023 shutdown_connections 2023-12-19T18:15:43.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.031+0000 7f272e399640 1 -- 172.21.15.27:0/3364548023 wait complete. 2023-12-19T18:15:43.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.032+0000 7f272e399640 1 Processor -- start 2023-12-19T18:15:43.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.033+0000 7f272e399640 1 -- start start 2023-12-19T18:15:43.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f272e399640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27280b8c30 0x7f272815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f272e399640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27280cda20 0x7f27280b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f272e399640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27280cf9b0 0x7f27280a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f272e399640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272815bae0 con 0x7f27280b8c30 2023-12-19T18:15:43.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f272e399640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27280b4a80 con 0x7f27280cf9b0 2023-12-19T18:15:43.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f272e399640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27280b1400 con 0x7f27280cda20 2023-12-19T18:15:43.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f2727fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27280b8c30 0x7f272815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f2727fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27280b8c30 0x7f272815b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47848/0 (socket says 172.21.15.27:47848) 2023-12-19T18:15:43.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f2727fff640 1 -- 172.21.15.27:0/3587027434 learned_addr learned my addr 172.21.15.27:0/3587027434 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:43.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f272c936640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27280cf9b0 0x7f27280a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f27277fe640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27280cda20 0x7f27280b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f2727fff640 1 -- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27280cda20 msgr2=0x7f27280b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f2727fff640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27280cda20 0x7f27280b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f2727fff640 1 -- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27280cf9b0 msgr2=0x7f27280a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f2727fff640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27280cf9b0 0x7f27280a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f2727fff640 1 -- 172.21.15.27:0/3587027434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2720009400 con 0x7f27280b8c30 2023-12-19T18:15:43.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f272c936640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27280cf9b0 0x7f27280a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:43.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.034+0000 7f27277fe640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27280cda20 0x7f27280b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:43.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.035+0000 7f2727fff640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27280b8c30 0x7f272815b5a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f27180018d0 tx=0x7f271800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:43.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.035+0000 7f2724ff9640 1 -- 172.21.15.27:0/3587027434 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718012400 con 0x7f27280b8c30 2023-12-19T18:15:43.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.035+0000 7f2724ff9640 1 -- 172.21.15.27:0/3587027434 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2718012560 con 0x7f27280b8c30 2023-12-19T18:15:43.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.035+0000 7f2724ff9640 1 -- 172.21.15.27:0/3587027434 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2718012950 con 0x7f27280b8c30 2023-12-19T18:15:43.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.035+0000 7f272e399640 1 -- 172.21.15.27:0/3587027434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27280b16e0 con 0x7f27280b8c30 2023-12-19T18:15:43.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.035+0000 7f272e399640 1 -- 172.21.15.27:0/3587027434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27280b5490 con 0x7f27280b8c30 2023-12-19T18:15:43.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.036+0000 7f272e399640 1 -- 172.21.15.27:0/3587027434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27280b4570 con 0x7f27280b8c30 2023-12-19T18:15:43.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.038+0000 7f2724ff9640 1 -- 172.21.15.27:0/3587027434 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2718012ab0 con 0x7f27280b8c30 2023-12-19T18:15:43.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.039+0000 7f2724ff9640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f270806a430 0x7f270806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.039+0000 7f2724ff9640 1 -- 172.21.15.27:0/3587027434 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f27180891b0 con 0x7f27280b8c30 2023-12-19T18:15:43.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.039+0000 7f27277fe640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f270806a430 0x7f270806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.040+0000 7f27277fe640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f270806a430 0x7f270806c8f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f272815be40 tx=0x7f2710008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:43.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.043+0000 7f2724ff9640 1 -- 172.21.15.27:0/3587027434 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f27280b4570 con 0x7f27280b8c30 2023-12-19T18:15:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.235+0000 7f272e399640 1 -- 172.21.15.27:0/3587027434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f27280b4570 con 0x7f27280b8c30 2023-12-19T18:15:43.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.237+0000 7f2724ff9640 1 -- 172.21.15.27:0/3587027434 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f27280b4570 con 0x7f27280b8c30 2023-12-19T18:15:43.237 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:15:43.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.239+0000 7f272e399640 1 -- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f270806a430 msgr2=0x7f270806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.239+0000 7f272e399640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f270806a430 0x7f270806c8f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f272815be40 tx=0x7f2710008040 comp rx=0 tx=0).stop 2023-12-19T18:15:43.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.239+0000 7f272e399640 1 -- 172.21.15.27:0/3587027434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27280b8c30 msgr2=0x7f272815b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.239+0000 7f272e399640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27280b8c30 0x7f272815b5a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f27180018d0 tx=0x7f271800a430 comp rx=0 tx=0).stop 2023-12-19T18:15:43.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.240+0000 7f272e399640 1 -- 172.21.15.27:0/3587027434 shutdown_connections 2023-12-19T18:15:43.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.240+0000 7f272e399640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f270806a430 0x7f270806c8f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.240+0000 7f272e399640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27280cf9b0 0x7f27280a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.240+0000 7f272e399640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27280cda20 0x7f27280b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.240+0000 7f272e399640 1 --2- 172.21.15.27:0/3587027434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27280b8c30 0x7f272815b5a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.240+0000 7f272e399640 1 -- 172.21.15.27:0/3587027434 >> 172.21.15.27:0/3587027434 conn(0x7f27280d2fa0 msgr2=0x7f27280d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:43.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.240+0000 7f272e399640 1 -- 172.21.15.27:0/3587027434 shutdown_connections 2023-12-19T18:15:43.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.240+0000 7f272e399640 1 -- 172.21.15.27:0/3587027434 wait complete. 2023-12-19T18:15:43.261 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:15:43.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.499+0000 7f4c9ecf3640 1 Processor -- start 2023-12-19T18:15:43.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.499+0000 7f4c9ecf3640 1 -- start start 2023-12-19T18:15:43.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9ecf3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4c9815d120 0x7f4c980cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9ecf3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c980d1310 0x7f4c980d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9ecf3640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4c980b2970 0x7f4c980b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9ecf3640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c980d3850 con 0x7f4c9815d120 2023-12-19T18:15:43.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9ecf3640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c980d0b10 con 0x7f4c980b2970 2023-12-19T18:15:43.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9ecf3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c980d1c90 con 0x7f4c980d1310 2023-12-19T18:15:43.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9dcf1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4c9815d120 0x7f4c980cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9dcf1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4c9815d120 0x7f4c980cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47852/0 (socket says 172.21.15.27:47852) 2023-12-19T18:15:43.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9dcf1640 1 -- 172.21.15.27:0/3933443237 learned_addr learned my addr 172.21.15.27:0/3933443237 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:43.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9d4f0640 1 --2- 172.21.15.27:0/3933443237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c980d1310 0x7f4c980d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9e4f2640 1 --2- 172.21.15.27:0/3933443237 >> v2:172.21.15.192:3300/0 conn(0x7f4c980b2970 0x7f4c980b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9dcf1640 1 -- 172.21.15.27:0/3933443237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c980d1310 msgr2=0x7f4c980d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:43.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9dcf1640 1 --2- 172.21.15.27:0/3933443237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c980d1310 0x7f4c980d1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9dcf1640 1 -- 172.21.15.27:0/3933443237 >> v2:172.21.15.192:3300/0 conn(0x7f4c980b2970 msgr2=0x7f4c980b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:43.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.501+0000 7f4c9dcf1640 1 --2- 172.21.15.27:0/3933443237 >> v2:172.21.15.192:3300/0 conn(0x7f4c980b2970 0x7f4c980b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.502+0000 7f4c9dcf1640 1 -- 172.21.15.27:0/3933443237 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9815d500 con 0x7f4c9815d120 2023-12-19T18:15:43.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.502+0000 7f4c9d4f0640 1 --2- 172.21.15.27:0/3933443237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c980d1310 0x7f4c980d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:43.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.502+0000 7f4c9e4f2640 1 --2- 172.21.15.27:0/3933443237 >> v2:172.21.15.192:3300/0 conn(0x7f4c980b2970 0x7f4c980b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:43.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.503+0000 7f4c9dcf1640 1 --2- 172.21.15.27:0/3933443237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c9815d120 0x7f4c980cfd70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c9808bd20 tx=0x7f4c9402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=197fdc3d4ca97e16 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:43.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.503+0000 7f4c9ccef640 1 -- 172.21.15.27:0/3933443237 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9402ca80 con 0x7f4c9815d120 2023-12-19T18:15:43.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.503+0000 7f4c9ccef640 1 -- 172.21.15.27:0/3933443237 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9402cbe0 con 0x7f4c9815d120 2023-12-19T18:15:43.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.503+0000 7f4c9ccef640 1 -- 172.21.15.27:0/3933443237 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c940343f0 con 0x7f4c9815d120 2023-12-19T18:15:43.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.504+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/3933443237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c9815d120 msgr2=0x7f4c980cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.504+0000 7f4c9ecf3640 1 --2- 172.21.15.27:0/3933443237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c9815d120 0x7f4c980cfd70 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f4c9808bd20 tx=0x7f4c9402bee0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.504+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/3933443237 shutdown_connections 2023-12-19T18:15:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.504+0000 7f4c9ecf3640 1 --2- 172.21.15.27:0/3933443237 >> v2:172.21.15.192:3300/0 conn(0x7f4c980b2970 0x7f4c980b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.504+0000 7f4c9ecf3640 1 --2- 172.21.15.27:0/3933443237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c980d1310 0x7f4c980d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.504+0000 7f4c9ecf3640 1 --2- 172.21.15.27:0/3933443237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c9815d120 0x7f4c980cfd70 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.504+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/3933443237 >> 172.21.15.27:0/3933443237 conn(0x7f4c980d2d30 msgr2=0x7f4c980d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.504+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/3933443237 shutdown_connections 2023-12-19T18:15:43.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.504+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/3933443237 wait complete. 2023-12-19T18:15:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.506+0000 7f4c9ecf3640 1 Processor -- start 2023-12-19T18:15:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.506+0000 7f4c9ecf3640 1 -- start start 2023-12-19T18:15:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.507+0000 7f4c9ecf3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c980b2970 0x7f4c98158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.507+0000 7f4c9ecf3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c980d1310 0x7f4c9814b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.507+0000 7f4c9ecf3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9815d120 0x7f4c98160130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.507+0000 7f4c9ecf3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c98158580 con 0x7f4c980b2970 2023-12-19T18:15:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.507+0000 7f4c9ecf3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9815a070 con 0x7f4c980d1310 2023-12-19T18:15:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.507+0000 7f4c9ecf3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c9814b690 con 0x7f4c9815d120 2023-12-19T18:15:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.507+0000 7f4c9dcf1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c980b2970 0x7f4c98158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.508+0000 7f4c9dcf1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c980b2970 0x7f4c98158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47860/0 (socket says 172.21.15.27:47860) 2023-12-19T18:15:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.508+0000 7f4c9dcf1640 1 -- 172.21.15.27:0/4260970545 learned_addr learned my addr 172.21.15.27:0/4260970545 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.508+0000 7f4c9e4f2640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9815d120 0x7f4c98160130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.508+0000 7f4c9d4f0640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c980d1310 0x7f4c9814b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.508+0000 7f4c9dcf1640 1 -- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9815d120 msgr2=0x7f4c98160130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.508+0000 7f4c9dcf1640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9815d120 0x7f4c98160130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.508+0000 7f4c9dcf1640 1 -- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c980d1310 msgr2=0x7f4c9814b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.508+0000 7f4c9dcf1640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c980d1310 0x7f4c9814b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.508+0000 7f4c9dcf1640 1 -- 172.21.15.27:0/4260970545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c94009b30 con 0x7f4c980b2970 2023-12-19T18:15:43.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.508+0000 7f4c9dcf1640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c980b2970 0x7f4c98158e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4c940093e0 tx=0x7f4c94004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:43.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.509+0000 7f4c8e7fc640 1 -- 172.21.15.27:0/4260970545 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9403e450 con 0x7f4c980b2970 2023-12-19T18:15:43.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.509+0000 7f4c8e7fc640 1 -- 172.21.15.27:0/4260970545 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c9403e5b0 con 0x7f4c980b2970 2023-12-19T18:15:43.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.509+0000 7f4c8e7fc640 1 -- 172.21.15.27:0/4260970545 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c9403e960 con 0x7f4c980b2970 2023-12-19T18:15:43.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.509+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/4260970545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c98160670 con 0x7f4c980b2970 2023-12-19T18:15:43.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.509+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/4260970545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c98159740 con 0x7f4c980b2970 2023-12-19T18:15:43.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.510+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/4260970545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c980d1020 con 0x7f4c980b2970 2023-12-19T18:15:43.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.511+0000 7f4c8e7fc640 1 -- 172.21.15.27:0/4260970545 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4c9402cd50 con 0x7f4c980b2970 2023-12-19T18:15:43.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.512+0000 7f4c8e7fc640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4c7c06a430 0x7f4c7c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.512+0000 7f4c8e7fc640 1 -- 172.21.15.27:0/4260970545 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4c940ab6f0 con 0x7f4c980b2970 2023-12-19T18:15:43.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.513+0000 7f4c9d4f0640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4c7c06a430 0x7f4c7c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.514+0000 7f4c9d4f0640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4c7c06a430 0x7f4c7c06c8f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4c980ceaf0 tx=0x7f4c88008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:43.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.517+0000 7f4c8e7fc640 1 -- 172.21.15.27:0/4260970545 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4c980d1020 con 0x7f4c980b2970 2023-12-19T18:15:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.712+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/4260970545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4c980d0ff0 con 0x7f4c980b2970 2023-12-19T18:15:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.713+0000 7f4c8e7fc640 1 -- 172.21.15.27:0/4260970545 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4c94076c20 con 0x7f4c980b2970 2023-12-19T18:15:43.713 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:15:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.716+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4c7c06a430 msgr2=0x7f4c7c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.716+0000 7f4c9ecf3640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4c7c06a430 0x7f4c7c06c8f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4c980ceaf0 tx=0x7f4c88008040 comp rx=0 tx=0).stop 2023-12-19T18:15:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.716+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/4260970545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c980b2970 msgr2=0x7f4c98158e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.716+0000 7f4c9ecf3640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c980b2970 0x7f4c98158e10 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4c940093e0 tx=0x7f4c94004a40 comp rx=0 tx=0).stop 2023-12-19T18:15:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.716+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/4260970545 shutdown_connections 2023-12-19T18:15:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.716+0000 7f4c9ecf3640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4c7c06a430 0x7f4c7c06c8f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.716+0000 7f4c9ecf3640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4c9815d120 0x7f4c98160130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.716+0000 7f4c9ecf3640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4c980d1310 0x7f4c9814b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.716+0000 7f4c9ecf3640 1 --2- 172.21.15.27:0/4260970545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c980b2970 0x7f4c98158e10 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.716+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/4260970545 >> 172.21.15.27:0/4260970545 conn(0x7f4c980d2d30 msgr2=0x7f4c980b3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:43.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.716+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/4260970545 shutdown_connections 2023-12-19T18:15:43.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.716+0000 7f4c9ecf3640 1 -- 172.21.15.27:0/4260970545 wait complete. 2023-12-19T18:15:43.737 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:43.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.968+0000 7fa93dafa640 1 Processor -- start 2023-12-19T18:15:43.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.969+0000 7fa93dafa640 1 -- start start 2023-12-19T18:15:43.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.970+0000 7fa93dafa640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9380b3170 0x7fa9380b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.970+0000 7fa93dafa640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa9380b2b10 0x7fa9380cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.970+0000 7fa93dafa640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa93815b850 con 0x7fa93815ad60 2023-12-19T18:15:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.970+0000 7fa93dafa640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9380b2fa0 con 0x7fa9380b2b10 2023-12-19T18:15:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.970+0000 7fa93dafa640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93815bb20 con 0x7fa9380b3170 2023-12-19T18:15:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.971+0000 7fa93caf8640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa93815ad60 0x7fa93815b140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49292/0 (socket says 172.21.15.27:49292) 2023-12-19T18:15:43.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.971+0000 7fa93caf8640 1 -- 172.21.15.27:0/571659519 learned_addr learned my addr 172.21.15.27:0/571659519 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.971+0000 7fa937fff640 1 --2- 172.21.15.27:0/571659519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9380b3170 0x7fa9380b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.971+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1868283033 0 0) 0x7fa93815b850 con 0x7fa93815ad60 2023-12-19T18:15:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.971+0000 7fa93d2f9640 1 --2- 172.21.15.27:0/571659519 >> v2:172.21.15.192:3300/0 conn(0x7fa9380b2b10 0x7fa9380cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.971+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa928003520 con 0x7fa93815ad60 2023-12-19T18:15:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2705092430 0 0) 0x7fa928003520 con 0x7fa93815ad60 2023-12-19T18:15:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9380b3170 msgr2=0x7fa9380b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 --2- 172.21.15.27:0/571659519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9380b3170 0x7fa9380b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 >> v2:172.21.15.192:3300/0 conn(0x7fa9380b2b10 msgr2=0x7fa9380cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:43.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 --2- 172.21.15.27:0/571659519 >> v2:172.21.15.192:3300/0 conn(0x7fa9380b2b10 0x7fa9380cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93815be00 con 0x7fa93815ad60 2023-12-19T18:15:43.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fa92c001670 con 0x7fa93815ad60 2023-12-19T18:15:43.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa93d2f9640 1 --2- 172.21.15.27:0/571659519 >> v2:172.21.15.192:3300/0 conn(0x7fa9380b2b10 0x7fa9380cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:43.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 >> v1:172.21.15.27:6789/0 conn(0x7fa93815ad60 legacy=0x7fa93815b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 --2- 172.21.15.27:0/571659519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa928003970 0x7fa928003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 --2- 172.21.15.27:0/571659519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9280047c0 0x7fa928004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 --2- 172.21.15.27:0/571659519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9380b3170 0x7fa928005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380b2fa0 con 0x7fa928003970 2023-12-19T18:15:43.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93815bb20 con 0x7fa9280047c0 2023-12-19T18:15:43.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa92800ba00 con 0x7fa9380b3170 2023-12-19T18:15:43.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa93caf8640 1 --2- 172.21.15.27:0/571659519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa928003970 0x7fa928003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa937fff640 1 --2- 172.21.15.27:0/571659519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9280047c0 0x7fa928004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa93caf8640 1 -- 172.21.15.27:0/571659519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9380b3170 msgr2=0x7fa928005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:43.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.972+0000 7fa93caf8640 1 --2- 172.21.15.27:0/571659519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9380b3170 0x7fa928005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.973+0000 7fa93caf8640 1 -- 172.21.15.27:0/571659519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9280047c0 msgr2=0x7fa928004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:43.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.973+0000 7fa93caf8640 1 --2- 172.21.15.27:0/571659519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa9280047c0 0x7fa928004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.973+0000 7fa93caf8640 1 -- 172.21.15.27:0/571659519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa928003520 con 0x7fa928003970 2023-12-19T18:15:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.974+0000 7fa93caf8640 1 --2- 172.21.15.27:0/571659519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa928003970 0x7fa928003d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c160 tx=0x7fa9280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecca138dda6f1612 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.974+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c005ad0 con 0x7fa928003970 2023-12-19T18:15:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.974+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92c005c30 con 0x7fa928003970 2023-12-19T18:15:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.974+0000 7fa9377fe640 1 -- 172.21.15.27:0/571659519 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c02c810 con 0x7fa928003970 2023-12-19T18:15:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.974+0000 7fa93dafa640 1 -- 172.21.15.27:0/571659519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa928003970 msgr2=0x7fa928003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.974+0000 7fa93dafa640 1 --2- 172.21.15.27:0/571659519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa928003970 0x7fa928003d90 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fa92c00c160 tx=0x7fa9280057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.974+0000 7fa93d2f9640 1 -- 172.21.15.27:0/571659519 reap_dead start 2023-12-19T18:15:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.975+0000 7fa93dafa640 1 -- 172.21.15.27:0/571659519 shutdown_connections 2023-12-19T18:15:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.975+0000 7fa93dafa640 1 -- 172.21.15.27:0/571659519 >> 172.21.15.27:0/571659519 conn(0x7fa93806d5f0 msgr2=0x7fa93806d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.975+0000 7fa93dafa640 1 -- 172.21.15.27:0/571659519 shutdown_connections 2023-12-19T18:15:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.975+0000 7fa93dafa640 1 -- 172.21.15.27:0/571659519 wait complete. 2023-12-19T18:15:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.977+0000 7fa93dafa640 1 Processor -- start 2023-12-19T18:15:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.977+0000 7fa93dafa640 1 -- start start 2023-12-19T18:15:43.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.978+0000 7fa93dafa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa928003970 0x7fa93815c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.978+0000 7fa93dafa640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa93806c6a0 0x7fa9381f1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.978+0000 7fa93dafa640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9380c9a90 0x7fa9380c9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.978+0000 7fa93dafa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9380ca480 con 0x7fa928003970 2023-12-19T18:15:43.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.978+0000 7fa93dafa640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa93815eb40 con 0x7fa93806c6a0 2023-12-19T18:15:43.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.978+0000 7fa93dafa640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa93815ee20 con 0x7fa9380c9a90 2023-12-19T18:15:43.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.978+0000 7fa93caf8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa928003970 0x7fa93815c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.978+0000 7fa93caf8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa928003970 0x7fa93815c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47882/0 (socket says 172.21.15.27:47882) 2023-12-19T18:15:43.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.978+0000 7fa93caf8640 1 -- 172.21.15.27:0/3282733860 learned_addr learned my addr 172.21.15.27:0/3282733860 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:43.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.979+0000 7fa93d2f9640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9380c9a90 0x7fa9380c9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.979+0000 7fa937fff640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa93806c6a0 0x7fa9381f1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.979+0000 7fa93caf8640 1 -- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9380c9a90 msgr2=0x7fa9380c9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.979+0000 7fa93caf8640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9380c9a90 0x7fa9380c9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.979+0000 7fa93caf8640 1 -- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa93806c6a0 msgr2=0x7fa9381f1a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:43.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.979+0000 7fa93caf8640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa93806c6a0 0x7fa9381f1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:43.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.979+0000 7fa93caf8640 1 -- 172.21.15.27:0/3282733860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9380c7f40 con 0x7fa928003970 2023-12-19T18:15:43.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.979+0000 7fa93d2f9640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9380c9a90 0x7fa9380c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:43.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.979+0000 7fa93caf8640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa928003970 0x7fa93815c390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa92c000fb0 tx=0x7fa92c000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:43.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.979+0000 7fa9357fa640 1 -- 172.21.15.27:0/3282733860 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c039070 con 0x7fa928003970 2023-12-19T18:15:43.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.979+0000 7fa9357fa640 1 -- 172.21.15.27:0/3282733860 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa92c001680 con 0x7fa928003970 2023-12-19T18:15:43.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.979+0000 7fa9357fa640 1 -- 172.21.15.27:0/3282733860 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa92c007290 con 0x7fa928003970 2023-12-19T18:15:43.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.979+0000 7fa93dafa640 1 -- 172.21.15.27:0/3282733860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9380c8220 con 0x7fa928003970 2023-12-19T18:15:43.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.980+0000 7fa93dafa640 1 -- 172.21.15.27:0/3282733860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9380ce6c0 con 0x7fa928003970 2023-12-19T18:15:43.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.981+0000 7fa93dafa640 1 -- 172.21.15.27:0/3282733860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa93804e840 con 0x7fa928003970 2023-12-19T18:15:43.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.982+0000 7fa9357fa640 1 -- 172.21.15.27:0/3282733860 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa92c032070 con 0x7fa928003970 2023-12-19T18:15:43.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.983+0000 7fa9357fa640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa91c06a460 0x7fa91c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:43.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.983+0000 7fa9357fa640 1 -- 172.21.15.27:0/3282733860 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa92c03c040 con 0x7fa928003970 2023-12-19T18:15:43.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.984+0000 7fa937fff640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa91c06a460 0x7fa91c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:43.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.985+0000 7fa937fff640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa91c06a460 0x7fa91c06c920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa9380d02e0 tx=0x7fa924008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:43.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:43.988+0000 7fa9357fa640 1 -- 172.21.15.27:0/3282733860 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa92c075060 con 0x7fa928003970 2023-12-19T18:15:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.184+0000 7fa93dafa640 1 -- 172.21.15.27:0/3282733860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa93806f120 con 0x7fa928003970 2023-12-19T18:15:44.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.187+0000 7fa9357fa640 1 -- 172.21.15.27:0/3282733860 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16090 (secure 0 0 0) 0x7fa92c045070 con 0x7fa928003970 2023-12-19T18:15:44.188 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:44.188 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:15:43.580746+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:44.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.190+0000 7fa93dafa640 1 -- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa91c06a460 msgr2=0x7fa91c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.190+0000 7fa93dafa640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa91c06a460 0x7fa91c06c920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fa9380d02e0 tx=0x7fa924008040 comp rx=0 tx=0).stop 2023-12-19T18:15:44.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.190+0000 7fa93dafa640 1 -- 172.21.15.27:0/3282733860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa928003970 msgr2=0x7fa93815c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.190+0000 7fa93dafa640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa928003970 0x7fa93815c390 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fa92c000fb0 tx=0x7fa92c000fe0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.190+0000 7fa93dafa640 1 -- 172.21.15.27:0/3282733860 shutdown_connections 2023-12-19T18:15:44.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.190+0000 7fa93dafa640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa91c06a460 0x7fa91c06c920 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.190+0000 7fa93dafa640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa9380c9a90 0x7fa9380c9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.190+0000 7fa93dafa640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa93806c6a0 0x7fa9381f1a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.190+0000 7fa93dafa640 1 --2- 172.21.15.27:0/3282733860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa928003970 0x7fa93815c390 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.190+0000 7fa93dafa640 1 -- 172.21.15.27:0/3282733860 >> 172.21.15.27:0/3282733860 conn(0x7fa93806d5f0 msgr2=0x7fa9380b07c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:44.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.190+0000 7fa93dafa640 1 -- 172.21.15.27:0/3282733860 shutdown_connections 2023-12-19T18:15:44.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.190+0000 7fa93dafa640 1 -- 172.21.15.27:0/3282733860 wait complete. 2023-12-19T18:15:44.209 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:15:44.210 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:15:44.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.433+0000 7f50236ab640 1 Processor -- start 2023-12-19T18:15:44.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.433+0000 7f50236ab640 1 -- start start 2023-12-19T18:15:44.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.435+0000 7f50236ab640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f501c0adc60 0x7f501c0cb410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.435+0000 7f50236ab640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f501c0c9e80 con 0x7f501c160b70 2023-12-19T18:15:44.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.435+0000 7f50236ab640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f501c160f50 con 0x7f501c0cb950 2023-12-19T18:15:44.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.435+0000 7f50236ab640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f501c0c9be0 con 0x7f501c0adc60 2023-12-19T18:15:44.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.435+0000 7f5021447640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f501c160b70 0x7f501c0ae0f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49308/0 (socket says 172.21.15.27:49308) 2023-12-19T18:15:44.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.435+0000 7f5021447640 1 -- 172.21.15.27:0/3345757346 learned_addr learned my addr 172.21.15.27:0/3345757346 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:44.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.435+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529526553 0 0) 0x7f501c0c9e80 con 0x7f501c160b70 2023-12-19T18:15:44.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5008003520 con 0x7f501c160b70 2023-12-19T18:15:44.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5020c46640 1 --2- 172.21.15.27:0/3345757346 >> v2:172.21.15.192:3300/0 conn(0x7f501c0adc60 0x7f501c0cb410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1997577818 0 0) 0x7f5008003520 con 0x7f501c160b70 2023-12-19T18:15:44.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 >> v2:172.21.15.192:3300/0 conn(0x7f501c0adc60 msgr2=0x7f501c0cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:44.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 --2- 172.21.15.27:0/3345757346 >> v2:172.21.15.192:3300/0 conn(0x7f501c0adc60 0x7f501c0cb410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 >> v1:172.21.15.192:6789/0 conn(0x7f501c0cb950 legacy=0x7f501c0ccfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c0ad180 con 0x7f501c160b70 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f5018001670 con 0x7f501c160b70 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 >> v1:172.21.15.27:6789/0 conn(0x7f501c160b70 legacy=0x7f501c0ae0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 --2- 172.21.15.27:0/3345757346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50080039c0 0x7f5008003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 --2- 172.21.15.27:0/3345757346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50080047c0 0x7f5008004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 --2- 172.21.15.27:0/3345757346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50080052a0 0x7f500800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c0c9be0 con 0x7f50080052a0 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500800c1c0 con 0x7f50080039c0 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500800c380 con 0x7f50080047c0 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.436+0000 7f5021c48640 1 --2- 172.21.15.27:0/3345757346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50080052a0 0x7f500800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.437+0000 7f5021447640 1 --2- 172.21.15.27:0/3345757346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50080039c0 0x7f5008003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.437+0000 7f5020c46640 1 --2- 172.21.15.27:0/3345757346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50080047c0 0x7f5008004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.437+0000 7f5021c48640 1 -- 172.21.15.27:0/3345757346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50080047c0 msgr2=0x7f5008004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:44.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.437+0000 7f5021c48640 1 --2- 172.21.15.27:0/3345757346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50080047c0 0x7f5008004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.437+0000 7f5021c48640 1 -- 172.21.15.27:0/3345757346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50080039c0 msgr2=0x7f5008003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:44.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.437+0000 7f5021c48640 1 --2- 172.21.15.27:0/3345757346 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50080039c0 0x7f5008003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.437+0000 7f5021c48640 1 -- 172.21.15.27:0/3345757346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5008003520 con 0x7f50080052a0 2023-12-19T18:15:44.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.437+0000 7f5020c46640 1 --2- 172.21.15.27:0/3345757346 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50080047c0 0x7f5008004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:44.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.439+0000 7f5021c48640 1 -- 172.21.15.27:0/3345757346 reap_dead start 2023-12-19T18:15:44.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.439+0000 7f5021c48640 1 --2- 172.21.15.27:0/3345757346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50080052a0 0x7f500800b980 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5014001e50 tx=0x7f50080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cc38df90247cb1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:44.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.439+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5014003b70 con 0x7f50080052a0 2023-12-19T18:15:44.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.439+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5014003d10 con 0x7f50080052a0 2023-12-19T18:15:44.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.439+0000 7f5013fff640 1 -- 172.21.15.27:0/3345757346 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5014005310 con 0x7f50080052a0 2023-12-19T18:15:44.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.440+0000 7f50236ab640 1 -- 172.21.15.27:0/3345757346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50080052a0 msgr2=0x7f500800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.440+0000 7f50236ab640 1 --2- 172.21.15.27:0/3345757346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50080052a0 0x7f500800b980 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f5014001e50 tx=0x7f50080057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.440+0000 7f50236ab640 1 -- 172.21.15.27:0/3345757346 shutdown_connections 2023-12-19T18:15:44.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.440+0000 7f50236ab640 1 --2- 172.21.15.27:0/3345757346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50080052a0 0x7f500800b980 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.440+0000 7f50236ab640 1 -- 172.21.15.27:0/3345757346 >> 172.21.15.27:0/3345757346 conn(0x7f501c0cf3f0 msgr2=0x7f501c0cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:44.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.441+0000 7f50236ab640 1 -- 172.21.15.27:0/3345757346 shutdown_connections 2023-12-19T18:15:44.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.441+0000 7f50236ab640 1 -- 172.21.15.27:0/3345757346 wait complete. 2023-12-19T18:15:44.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.444+0000 7f50236ab640 1 Processor -- start 2023-12-19T18:15:44.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.444+0000 7f50236ab640 1 -- start start 2023-12-19T18:15:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.446+0000 7f50236ab640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50080052a0 0x7f501c0b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.446+0000 7f50236ab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f501c0c9070 0x7f501c0b0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.446+0000 7f50236ab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f501c0ccf80 0x7f501c0cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.446+0000 7f50236ab640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c0b15e0 con 0x7f501c0c9070 2023-12-19T18:15:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.446+0000 7f50236ab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f501c0cd940 con 0x7f501c0ccf80 2023-12-19T18:15:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.446+0000 7f50236ab640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f501c0b0960 con 0x7f50080052a0 2023-12-19T18:15:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.446+0000 7f5020c46640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f501c0c9070 0x7f501c0b0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.446+0000 7f5020c46640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f501c0c9070 0x7f501c0b0110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47908/0 (socket says 172.21.15.27:47908) 2023-12-19T18:15:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.446+0000 7f5020c46640 1 -- 172.21.15.27:0/1787253481 learned_addr learned my addr 172.21.15.27:0/1787253481 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.447+0000 7f5021447640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50080052a0 0x7f501c0b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.447+0000 7f5020c46640 1 -- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50080052a0 msgr2=0x7f501c0b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.447+0000 7f5020c46640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50080052a0 0x7f501c0b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.447+0000 7f5020c46640 1 -- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f501c0ccf80 msgr2=0x7f501c0cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.447+0000 7f5020c46640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f501c0ccf80 0x7f501c0cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.447+0000 7f5020c46640 1 -- 172.21.15.27:0/1787253481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f501c0b0c40 con 0x7f501c0c9070 2023-12-19T18:15:44.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.447+0000 7f5021447640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50080052a0 0x7f501c0b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:44.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.447+0000 7f5020c46640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f501c0c9070 0x7f501c0b0110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f500c00f550 tx=0x7f500c00fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:44.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.447+0000 7f5011ffb640 1 -- 172.21.15.27:0/1787253481 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c013810 con 0x7f501c0c9070 2023-12-19T18:15:44.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.448+0000 7f5011ffb640 1 -- 172.21.15.27:0/1787253481 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500c013970 con 0x7f501c0c9070 2023-12-19T18:15:44.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.448+0000 7f50236ab640 1 -- 172.21.15.27:0/1787253481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f501c0b0f20 con 0x7f501c0c9070 2023-12-19T18:15:44.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.448+0000 7f5011ffb640 1 -- 172.21.15.27:0/1787253481 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500c013ce0 con 0x7f501c0c9070 2023-12-19T18:15:44.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.448+0000 7f50236ab640 1 -- 172.21.15.27:0/1787253481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f501c0b0650 con 0x7f501c0c9070 2023-12-19T18:15:44.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.449+0000 7f50236ab640 1 -- 172.21.15.27:0/1787253481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f501c0b1720 con 0x7f501c0c9070 2023-12-19T18:15:44.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.450+0000 7f5011ffb640 1 -- 172.21.15.27:0/1787253481 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f500c01f460 con 0x7f501c0c9070 2023-12-19T18:15:44.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.451+0000 7f5011ffb640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f500006a460 0x7f500006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.452+0000 7f5011ffb640 1 -- 172.21.15.27:0/1787253481 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f500c088d10 con 0x7f501c0c9070 2023-12-19T18:15:44.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.452+0000 7f5021447640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f500006a460 0x7f500006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.453+0000 7f5021447640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f500006a460 0x7f500006c920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5018001eb0 tx=0x7f5018009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:44.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.456+0000 7f5011ffb640 1 -- 172.21.15.27:0/1787253481 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f501c0b1720 con 0x7f501c0c9070 2023-12-19T18:15:44.524 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:15:44.533 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:15:44.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.659+0000 7f50236ab640 1 -- 172.21.15.27:0/1787253481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f501c0b1720 con 0x7f501c0c9070 2023-12-19T18:15:44.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.659+0000 7f5011ffb640 1 -- 172.21.15.27:0/1787253481 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f501c0b1720 con 0x7f501c0c9070 2023-12-19T18:15:44.659 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:15:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.662+0000 7f50236ab640 1 -- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f500006a460 msgr2=0x7f500006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.662+0000 7f50236ab640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f500006a460 0x7f500006c920 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5018001eb0 tx=0x7f5018009040 comp rx=0 tx=0).stop 2023-12-19T18:15:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.662+0000 7f50236ab640 1 -- 172.21.15.27:0/1787253481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f501c0c9070 msgr2=0x7f501c0b0110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.662+0000 7f50236ab640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f501c0c9070 0x7f501c0b0110 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f500c00f550 tx=0x7f500c00fa10 comp rx=0 tx=0).stop 2023-12-19T18:15:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.662+0000 7f50236ab640 1 -- 172.21.15.27:0/1787253481 shutdown_connections 2023-12-19T18:15:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.662+0000 7f50236ab640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f500006a460 0x7f500006c920 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.662+0000 7f50236ab640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f501c0ccf80 0x7f501c0cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.662+0000 7f50236ab640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f501c0c9070 0x7f501c0b0110 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.662+0000 7f50236ab640 1 --2- 172.21.15.27:0/1787253481 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50080052a0 0x7f501c0b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.662+0000 7f50236ab640 1 -- 172.21.15.27:0/1787253481 >> 172.21.15.27:0/1787253481 conn(0x7f501c0cf3f0 msgr2=0x7f501c0ca3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.662+0000 7f50236ab640 1 -- 172.21.15.27:0/1787253481 shutdown_connections 2023-12-19T18:15:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.662+0000 7f50236ab640 1 -- 172.21.15.27:0/1787253481 wait complete. 2023-12-19T18:15:44.681 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:15:44.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.940+0000 7f02ae7b1640 1 Processor -- start 2023-12-19T18:15:44.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.940+0000 7f02ae7b1640 1 -- start start 2023-12-19T18:15:44.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.941+0000 7f02ae7b1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a80c2720 0x7f02a806fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.942+0000 7f02ae7b1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a806bc80 con 0x7f02a8071420 2023-12-19T18:15:44.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.942+0000 7f02ae7b1640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a806d690 con 0x7f02a815b1a0 2023-12-19T18:15:44.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.942+0000 7f02ae7b1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a806d0a0 con 0x7f02a80c2720 2023-12-19T18:15:44.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.942+0000 7f02acd4e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f02a8071420 0x7f02a815ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49318/0 (socket says 172.21.15.27:49318) 2023-12-19T18:15:44.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.942+0000 7f02acd4e640 1 -- 172.21.15.27:0/2563117641 learned_addr learned my addr 172.21.15.27:0/2563117641 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:44.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.942+0000 7f029ffff640 1 --2- 172.21.15.27:0/2563117641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a80c2720 0x7f02a806fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.942+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890628172 0 0) 0x7f02a0001490 con 0x7f02a8071420 2023-12-19T18:15:44.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0294003520 con 0x7f02a8071420 2023-12-19T18:15:44.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1727161181 0 0) 0x7f0294003520 con 0x7f02a8071420 2023-12-19T18:15:44.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a80c2720 msgr2=0x7f02a806fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:44.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 --2- 172.21.15.27:0/2563117641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a80c2720 0x7f02a806fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 >> v1:172.21.15.192:6789/0 conn(0x7f02a815b1a0 legacy=0x7f02a806f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a806b6c0 con 0x7f02a8071420 2023-12-19T18:15:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f02a0001b80 con 0x7f02a8071420 2023-12-19T18:15:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 >> v1:172.21.15.27:6789/0 conn(0x7f02a8071420 legacy=0x7f02a815ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 --2- 172.21.15.27:0/2563117641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a80c2720 0x7f02940039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 --2- 172.21.15.27:0/2563117641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294003f20 0x7f02940047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 --2- 172.21.15.27:0/2563117641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0294004d00 0x7f0294005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a806d0a0 con 0x7f0294003f20 2023-12-19T18:15:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029400bb10 con 0x7f0294004d00 2023-12-19T18:15:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029400bcd0 con 0x7f02a80c2720 2023-12-19T18:15:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.943+0000 7f029ffff640 1 --2- 172.21.15.27:0/2563117641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294003f20 0x7f02940047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.944+0000 7f02a7fff640 1 --2- 172.21.15.27:0/2563117641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a80c2720 0x7f02940039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.944+0000 7f029ffff640 1 -- 172.21.15.27:0/2563117641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a80c2720 msgr2=0x7f02940039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.944+0000 7f029ffff640 1 --2- 172.21.15.27:0/2563117641 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02a80c2720 0x7f02940039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.944+0000 7f029ffff640 1 -- 172.21.15.27:0/2563117641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0294004d00 msgr2=0x7f0294005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:44.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.944+0000 7f029ffff640 1 --2- 172.21.15.27:0/2563117641 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0294004d00 0x7f0294005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.944+0000 7f029ffff640 1 -- 172.21.15.27:0/2563117641 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294003520 con 0x7f0294003f20 2023-12-19T18:15:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.945+0000 7f029ffff640 1 --2- 172.21.15.27:0/2563117641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294003f20 0x7f02940047c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0290009a90 tx=0x7f02940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2df32314333c1236 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.945+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029002c680 con 0x7f0294003f20 2023-12-19T18:15:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.945+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f029002c7e0 con 0x7f0294003f20 2023-12-19T18:15:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.945+0000 7f02a77fe640 1 -- 172.21.15.27:0/2563117641 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f029002cb70 con 0x7f0294003f20 2023-12-19T18:15:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.946+0000 7f02ae7b1640 1 -- 172.21.15.27:0/2563117641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294003f20 msgr2=0x7f02940047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.946+0000 7f02ae7b1640 1 --2- 172.21.15.27:0/2563117641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0294003f20 0x7f02940047c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f0290009a90 tx=0x7f02940057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.946+0000 7f02acd4e640 1 -- 172.21.15.27:0/2563117641 reap_dead start 2023-12-19T18:15:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.946+0000 7f02ae7b1640 1 -- 172.21.15.27:0/2563117641 shutdown_connections 2023-12-19T18:15:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.946+0000 7f02ae7b1640 1 -- 172.21.15.27:0/2563117641 >> 172.21.15.27:0/2563117641 conn(0x7f02a80cebd0 msgr2=0x7f02a80cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.946+0000 7f02ae7b1640 1 -- 172.21.15.27:0/2563117641 shutdown_connections 2023-12-19T18:15:44.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.946+0000 7f02ae7b1640 1 -- 172.21.15.27:0/2563117641 wait complete. 2023-12-19T18:15:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.948+0000 7f02ae7b1640 1 Processor -- start 2023-12-19T18:15:44.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.948+0000 7f02ae7b1640 1 -- start start 2023-12-19T18:15:44.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.949+0000 7f02ae7b1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0294003f20 0x7f02a80c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.949+0000 7f02ae7b1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a814a120 0x7f02a81e6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.949+0000 7f02ae7b1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02a80cd8a0 0x7f02a80cdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.949+0000 7f02ae7b1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a80ce260 con 0x7f02a80cd8a0 2023-12-19T18:15:44.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.949+0000 7f02ae7b1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a80c6c60 con 0x7f02a814a120 2023-12-19T18:15:44.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.949+0000 7f02ae7b1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a80c6f40 con 0x7f0294003f20 2023-12-19T18:15:44.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.949+0000 7f02acd4e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02a80cd8a0 0x7f02a80cdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.950+0000 7f02acd4e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02a80cd8a0 0x7f02a80cdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47924/0 (socket says 172.21.15.27:47924) 2023-12-19T18:15:44.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.950+0000 7f02acd4e640 1 -- 172.21.15.27:0/3093166433 learned_addr learned my addr 172.21.15.27:0/3093166433 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:44.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.950+0000 7f02a7fff640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0294003f20 0x7f02a80c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.950+0000 7f029ffff640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a814a120 0x7f02a81e6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.950+0000 7f02acd4e640 1 -- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0294003f20 msgr2=0x7f02a80c7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.950+0000 7f02acd4e640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0294003f20 0x7f02a80c7d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.950+0000 7f02acd4e640 1 -- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a814a120 msgr2=0x7f02a81e6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.950+0000 7f02acd4e640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a814a120 0x7f02a81e6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.950+0000 7f02acd4e640 1 -- 172.21.15.27:0/3093166433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a80c7220 con 0x7f02a80cd8a0 2023-12-19T18:15:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.950+0000 7f02acd4e640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02a80cd8a0 0x7f02a80cdd20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f02a80cea10 tx=0x7f02a000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.951+0000 7f02a57fa640 1 -- 172.21.15.27:0/3093166433 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a000ce60 con 0x7f02a80cd8a0 2023-12-19T18:15:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.951+0000 7f02a57fa640 1 -- 172.21.15.27:0/3093166433 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02a000a5b0 con 0x7f02a80cd8a0 2023-12-19T18:15:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.951+0000 7f02ae7b1640 1 -- 172.21.15.27:0/3093166433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a80c7500 con 0x7f02a80cd8a0 2023-12-19T18:15:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.951+0000 7f02a57fa640 1 -- 172.21.15.27:0/3093166433 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a000a960 con 0x7f02a80cd8a0 2023-12-19T18:15:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.951+0000 7f02ae7b1640 1 -- 172.21.15.27:0/3093166433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a80c6920 con 0x7f02a80cd8a0 2023-12-19T18:15:44.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.953+0000 7f02ae7b1640 1 -- 172.21.15.27:0/3093166433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a80c7a20 con 0x7f02a80cd8a0 2023-12-19T18:15:44.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.954+0000 7f02a57fa640 1 -- 172.21.15.27:0/3093166433 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f02a000aac0 con 0x7f02a80cd8a0 2023-12-19T18:15:44.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.955+0000 7f02a57fa640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f028806a4b0 0x7f028806c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:44.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.956+0000 7f02a57fa640 1 -- 172.21.15.27:0/3093166433 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f02a0088430 con 0x7f02a80cd8a0 2023-12-19T18:15:44.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.956+0000 7f02a7fff640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f028806a4b0 0x7f028806c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:44.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.957+0000 7f02a7fff640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f028806a4b0 0x7f028806c970 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0298007fd0 tx=0x7f0298009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:44.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:44.959+0000 7f02a57fa640 1 -- 172.21.15.27:0/3093166433 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f02a80c7a20 con 0x7f02a80cd8a0 2023-12-19T18:15:45.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.152+0000 7f02ae7b1640 1 -- 172.21.15.27:0/3093166433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f02a806bc80 con 0x7f02a80cd8a0 2023-12-19T18:15:45.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.153+0000 7f02a57fa640 1 -- 172.21.15.27:0/3093166433 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f02a806bc80 con 0x7f02a80cd8a0 2023-12-19T18:15:45.153 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:15:45.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.156+0000 7f02ae7b1640 1 -- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f028806a4b0 msgr2=0x7f028806c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:45.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.156+0000 7f02ae7b1640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f028806a4b0 0x7f028806c970 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0298007fd0 tx=0x7f0298009040 comp rx=0 tx=0).stop 2023-12-19T18:15:45.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.156+0000 7f02ae7b1640 1 -- 172.21.15.27:0/3093166433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02a80cd8a0 msgr2=0x7f02a80cdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:45.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.156+0000 7f02ae7b1640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02a80cd8a0 0x7f02a80cdd20 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f02a80cea10 tx=0x7f02a000cc10 comp rx=0 tx=0).stop 2023-12-19T18:15:45.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.156+0000 7f02ae7b1640 1 -- 172.21.15.27:0/3093166433 shutdown_connections 2023-12-19T18:15:45.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.156+0000 7f02ae7b1640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f028806a4b0 0x7f028806c970 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.156+0000 7f02ae7b1640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02a80cd8a0 0x7f02a80cdd20 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.156+0000 7f02ae7b1640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02a814a120 0x7f02a81e6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.156+0000 7f02ae7b1640 1 --2- 172.21.15.27:0/3093166433 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0294003f20 0x7f02a80c7d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.156+0000 7f02ae7b1640 1 -- 172.21.15.27:0/3093166433 >> 172.21.15.27:0/3093166433 conn(0x7f02a80cebd0 msgr2=0x7f02a80cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:45.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.156+0000 7f02ae7b1640 1 -- 172.21.15.27:0/3093166433 shutdown_connections 2023-12-19T18:15:45.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.156+0000 7f02ae7b1640 1 -- 172.21.15.27:0/3093166433 wait complete. 2023-12-19T18:15:45.175 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:15:45.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.395+0000 7fb7623a9640 1 Processor -- start 2023-12-19T18:15:45.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.396+0000 7fb7623a9640 1 -- start start 2023-12-19T18:15:45.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.397+0000 7fb7623a9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb75c0b3980 0x7fb75c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.397+0000 7fb7623a9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb75c0d3850 con 0x7fb75c15d120 2023-12-19T18:15:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.397+0000 7fb7623a9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb75c0d0b10 con 0x7fb75c0d1a20 2023-12-19T18:15:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.397+0000 7fb7623a9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb75c15d500 con 0x7fb75c0b3980 2023-12-19T18:15:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.398+0000 7fb7613a7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb75c15d120 0x7fb75c0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49328/0 (socket says 172.21.15.27:49328) 2023-12-19T18:15:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.398+0000 7fb7613a7640 1 -- 172.21.15.27:0/1420274629 learned_addr learned my addr 172.21.15.27:0/1420274629 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.398+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080055531 0 0) 0x7fb75c0d3850 con 0x7fb75c15d120 2023-12-19T18:15:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.398+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb748003520 con 0x7fb75c15d120 2023-12-19T18:15:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.398+0000 7fb761ba8640 1 --2- 172.21.15.27:0/1420274629 >> v2:172.21.15.192:3300/0 conn(0x7fb75c0b3980 0x7fb75c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.398+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4136347413 0 0) 0x7fb748003520 con 0x7fb75c15d120 2023-12-19T18:15:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.398+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 >> v2:172.21.15.192:3300/0 conn(0x7fb75c0b3980 msgr2=0x7fb75c0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.398+0000 7fb753fff640 1 --2- 172.21.15.27:0/1420274629 >> v2:172.21.15.192:3300/0 conn(0x7fb75c0b3980 0x7fb75c0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.398+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 >> v1:172.21.15.192:6789/0 conn(0x7fb75c0d1a20 legacy=0x7fb75c0cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.398+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c0d1040 con 0x7fb75c15d120 2023-12-19T18:15:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.398+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fb758001670 con 0x7fb75c15d120 2023-12-19T18:15:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.399+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 >> v1:172.21.15.27:6789/0 conn(0x7fb75c15d120 legacy=0x7fb75c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:45.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.399+0000 7fb753fff640 1 --2- 172.21.15.27:0/1420274629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7480039c0 0x7fb748003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.399+0000 7fb753fff640 1 --2- 172.21.15.27:0/1420274629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7480047c0 0x7fb748004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.399+0000 7fb753fff640 1 --2- 172.21.15.27:0/1420274629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7480051d0 0x7fb74800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.399+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c15d500 con 0x7fb7480051d0 2023-12-19T18:15:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.399+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74800c120 con 0x7fb7480047c0 2023-12-19T18:15:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.399+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74800c2e0 con 0x7fb7480039c0 2023-12-19T18:15:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.399+0000 7fb761ba8640 1 --2- 172.21.15.27:0/1420274629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7480051d0 0x7fb74800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.399+0000 7fb7613a7640 1 --2- 172.21.15.27:0/1420274629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7480039c0 0x7fb748003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.399+0000 7fb760ba6640 1 --2- 172.21.15.27:0/1420274629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7480047c0 0x7fb748004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.399+0000 7fb761ba8640 1 -- 172.21.15.27:0/1420274629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7480039c0 msgr2=0x7fb748003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.399+0000 7fb761ba8640 1 --2- 172.21.15.27:0/1420274629 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7480039c0 0x7fb748003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.399+0000 7fb761ba8640 1 -- 172.21.15.27:0/1420274629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7480047c0 msgr2=0x7fb748004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.400+0000 7fb761ba8640 1 --2- 172.21.15.27:0/1420274629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7480047c0 0x7fb748004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.400+0000 7fb761ba8640 1 -- 172.21.15.27:0/1420274629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb748003520 con 0x7fb7480051d0 2023-12-19T18:15:45.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.400+0000 7fb760ba6640 1 --2- 172.21.15.27:0/1420274629 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7480047c0 0x7fb748004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.401+0000 7fb761ba8640 1 -- 172.21.15.27:0/1420274629 reap_dead start 2023-12-19T18:15:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.401+0000 7fb761ba8640 1 --2- 172.21.15.27:0/1420274629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7480051d0 0x7fb74800b980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb754007330 tx=0x7fb7480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72a12ece9d8cf104 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.401+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75402ccd0 con 0x7fb7480051d0 2023-12-19T18:15:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.402+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75402ce30 con 0x7fb7480051d0 2023-12-19T18:15:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.402+0000 7fb753fff640 1 -- 172.21.15.27:0/1420274629 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754034680 con 0x7fb7480051d0 2023-12-19T18:15:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.402+0000 7fb7623a9640 1 -- 172.21.15.27:0/1420274629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7480051d0 msgr2=0x7fb74800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.402+0000 7fb7623a9640 1 --2- 172.21.15.27:0/1420274629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7480051d0 0x7fb74800b980 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb754007330 tx=0x7fb7480057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.402+0000 7fb7623a9640 1 -- 172.21.15.27:0/1420274629 shutdown_connections 2023-12-19T18:15:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.402+0000 7fb7623a9640 1 --2- 172.21.15.27:0/1420274629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7480051d0 0x7fb74800b980 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.402+0000 7fb7623a9640 1 -- 172.21.15.27:0/1420274629 >> 172.21.15.27:0/1420274629 conn(0x7fb75c0d2d30 msgr2=0x7fb75c0d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.402+0000 7fb7623a9640 1 -- 172.21.15.27:0/1420274629 shutdown_connections 2023-12-19T18:15:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.402+0000 7fb7623a9640 1 -- 172.21.15.27:0/1420274629 wait complete. 2023-12-19T18:15:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.404+0000 7fb7623a9640 1 Processor -- start 2023-12-19T18:15:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.405+0000 7fb7623a9640 1 -- start start 2023-12-19T18:15:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.405+0000 7fb7623a9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7480051d0 0x7fb75c0b8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb7623a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb75c0b60d0 0x7fb75c0b6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb7623a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb75c06b5e0 0x7fb75c06ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb7623a9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c06bf80 con 0x7fb7480051d0 2023-12-19T18:15:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb7623a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c15d990 con 0x7fb75c0b60d0 2023-12-19T18:15:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb7623a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c15dc70 con 0x7fb75c06b5e0 2023-12-19T18:15:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb7613a7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7480051d0 0x7fb75c0b8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb7613a7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7480051d0 0x7fb75c0b8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47942/0 (socket says 172.21.15.27:47942) 2023-12-19T18:15:45.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb7613a7640 1 -- 172.21.15.27:0/300144431 learned_addr learned my addr 172.21.15.27:0/300144431 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb761ba8640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb75c06b5e0 0x7fb75c06ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb760ba6640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb75c0b60d0 0x7fb75c0b6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb7613a7640 1 -- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb75c06b5e0 msgr2=0x7fb75c06ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb7613a7640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb75c06b5e0 0x7fb75c06ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb7613a7640 1 -- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb75c0b60d0 msgr2=0x7fb75c0b6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb7613a7640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb75c0b60d0 0x7fb75c0b6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb7613a7640 1 -- 172.21.15.27:0/300144431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c15df50 con 0x7fb7480051d0 2023-12-19T18:15:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.406+0000 7fb760ba6640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb75c0b60d0 0x7fb75c0b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.407+0000 7fb7613a7640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7480051d0 0x7fb75c0b8510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb75800e700 tx=0x7fb75800eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.407+0000 7fb751ffb640 1 -- 172.21.15.27:0/300144431 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75800c620 con 0x7fb7480051d0 2023-12-19T18:15:45.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.407+0000 7fb751ffb640 1 -- 172.21.15.27:0/300144431 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb75800c7a0 con 0x7fb7480051d0 2023-12-19T18:15:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.407+0000 7fb7623a9640 1 -- 172.21.15.27:0/300144431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75c15e230 con 0x7fb7480051d0 2023-12-19T18:15:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.407+0000 7fb751ffb640 1 -- 172.21.15.27:0/300144431 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75800cb50 con 0x7fb7480051d0 2023-12-19T18:15:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.407+0000 7fb7623a9640 1 -- 172.21.15.27:0/300144431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75c15d640 con 0x7fb7480051d0 2023-12-19T18:15:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.409+0000 7fb7623a9640 1 -- 172.21.15.27:0/300144431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75c04e840 con 0x7fb7480051d0 2023-12-19T18:15:45.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.409+0000 7fb751ffb640 1 -- 172.21.15.27:0/300144431 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb75800ccb0 con 0x7fb7480051d0 2023-12-19T18:15:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.411+0000 7fb751ffb640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb74006a460 0x7fb74006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.411+0000 7fb751ffb640 1 -- 172.21.15.27:0/300144431 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb758088510 con 0x7fb7480051d0 2023-12-19T18:15:45.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.412+0000 7fb760ba6640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb74006a460 0x7fb74006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:45.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.413+0000 7fb760ba6640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb74006a460 0x7fb74006c920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb75c06a0f0 tx=0x7fb74c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:45.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.415+0000 7fb751ffb640 1 -- 172.21.15.27:0/300144431 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb758053ae0 con 0x7fb7480051d0 2023-12-19T18:15:45.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.611+0000 7fb7623a9640 1 -- 172.21.15.27:0/300144431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb75c06c180 con 0x7fb7480051d0 2023-12-19T18:15:45.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.612+0000 7fb751ffb640 1 -- 172.21.15.27:0/300144431 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb758057100 con 0x7fb7480051d0 2023-12-19T18:15:45.612 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:15:45.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.614+0000 7fb7623a9640 1 -- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb74006a460 msgr2=0x7fb74006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:45.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.614+0000 7fb7623a9640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb74006a460 0x7fb74006c920 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb75c06a0f0 tx=0x7fb74c009040 comp rx=0 tx=0).stop 2023-12-19T18:15:45.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.614+0000 7fb7623a9640 1 -- 172.21.15.27:0/300144431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7480051d0 msgr2=0x7fb75c0b8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:45.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.614+0000 7fb7623a9640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7480051d0 0x7fb75c0b8510 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb75800e700 tx=0x7fb75800eb00 comp rx=0 tx=0).stop 2023-12-19T18:15:45.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.615+0000 7fb7623a9640 1 -- 172.21.15.27:0/300144431 shutdown_connections 2023-12-19T18:15:45.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.615+0000 7fb7623a9640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb74006a460 0x7fb74006c920 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.615+0000 7fb7623a9640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb75c06b5e0 0x7fb75c06ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.615+0000 7fb7623a9640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb75c0b60d0 0x7fb75c0b6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.615+0000 7fb7623a9640 1 --2- 172.21.15.27:0/300144431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7480051d0 0x7fb75c0b8510 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.615+0000 7fb7623a9640 1 -- 172.21.15.27:0/300144431 >> 172.21.15.27:0/300144431 conn(0x7fb75c0d2d30 msgr2=0x7fb75c0b2870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:45.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.615+0000 7fb7623a9640 1 -- 172.21.15.27:0/300144431 shutdown_connections 2023-12-19T18:15:45.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.615+0000 7fb7623a9640 1 -- 172.21.15.27:0/300144431 wait complete. 2023-12-19T18:15:45.661 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestClusterFull) That when an OSD epoch barrier is set on an MDS, subsequently 2023-12-19T18:15:45.661 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestClusterFull) 2023-12-19T18:15:45.661 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-12-19T18:15:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.887+0000 7fef2c96c640 1 Processor -- start 2023-12-19T18:15:45.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.887+0000 7fef2c96c640 1 -- start start 2023-12-19T18:15:45.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.889+0000 7fef2c96c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef2815ee30 0x7fef280c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.889+0000 7fef2c96c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef280afeb0 0x7fef280c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.889+0000 7fef2c96c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef280b0320 con 0x7fef281553a0 2023-12-19T18:15:45.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.889+0000 7fef2c96c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef280cbcf0 con 0x7fef280afeb0 2023-12-19T18:15:45.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.889+0000 7fef2c96c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef280cb350 con 0x7fef2815ee30 2023-12-19T18:15:45.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.890+0000 7fef27fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef280afeb0 0x7fef280c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:45.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.890+0000 7fef27fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef280afeb0 0x7fef280c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47956/0 (socket says 172.21.15.27:47956) 2023-12-19T18:15:45.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.890+0000 7fef27fff640 1 -- 172.21.15.27:0/2326343293 learned_addr learned my addr 172.21.15.27:0/2326343293 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:45.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.890+0000 7fef27fff640 1 -- 172.21.15.27:0/2326343293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef2815ee30 msgr2=0x7fef280c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:45.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.890+0000 7fef1ffff640 1 --2- 172.21.15.27:0/2326343293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef2815ee30 0x7fef280c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:45.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.890+0000 7fef27fff640 1 --2- 172.21.15.27:0/2326343293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef2815ee30 0x7fef280c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.890+0000 7fef27fff640 1 -- 172.21.15.27:0/2326343293 >> v1:172.21.15.27:6789/0 conn(0x7fef281553a0 legacy=0x7fef2815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:45.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.890+0000 7fef26ffd640 1 -- 172.21.15.27:0/2326343293 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228438762 0 0) 0x7fef280b0320 con 0x7fef281553a0 2023-12-19T18:15:45.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.890+0000 7fef27fff640 1 -- 172.21.15.27:0/2326343293 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef280cb630 con 0x7fef280afeb0 2023-12-19T18:15:45.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.890+0000 7fef1ffff640 1 --2- 172.21.15.27:0/2326343293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef2815ee30 0x7fef280c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:45.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.891+0000 7fef27fff640 1 --2- 172.21.15.27:0/2326343293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef280afeb0 0x7fef280c39d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fef20009e50 tx=0x7fef2002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b363087387f65115 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:45.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.892+0000 7fef26ffd640 1 -- 172.21.15.27:0/2326343293 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2002ca70 con 0x7fef280afeb0 2023-12-19T18:15:45.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.892+0000 7fef26ffd640 1 -- 172.21.15.27:0/2326343293 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2002cbd0 con 0x7fef280afeb0 2023-12-19T18:15:45.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.892+0000 7fef26ffd640 1 -- 172.21.15.27:0/2326343293 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef200343f0 con 0x7fef280afeb0 2023-12-19T18:15:45.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.892+0000 7fef2c96c640 1 -- 172.21.15.27:0/2326343293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef280afeb0 msgr2=0x7fef280c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:45.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.892+0000 7fef2c96c640 1 --2- 172.21.15.27:0/2326343293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef280afeb0 0x7fef280c39d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fef20009e50 tx=0x7fef2002bed0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.892+0000 7fef2c96c640 1 -- 172.21.15.27:0/2326343293 shutdown_connections 2023-12-19T18:15:45.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.892+0000 7fef2c96c640 1 --2- 172.21.15.27:0/2326343293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef280afeb0 0x7fef280c39d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.892+0000 7fef2c96c640 1 --2- 172.21.15.27:0/2326343293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef2815ee30 0x7fef280c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.892+0000 7fef2c96c640 1 -- 172.21.15.27:0/2326343293 >> 172.21.15.27:0/2326343293 conn(0x7fef28069c70 msgr2=0x7fef2806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:45.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.892+0000 7fef2c96c640 1 -- 172.21.15.27:0/2326343293 shutdown_connections 2023-12-19T18:15:45.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.892+0000 7fef2c96c640 1 -- 172.21.15.27:0/2326343293 wait complete. 2023-12-19T18:15:45.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.894+0000 7fef2c96c640 1 Processor -- start 2023-12-19T18:15:45.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.894+0000 7fef2c96c640 1 -- start start 2023-12-19T18:15:45.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.895+0000 7fef2c96c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef280afeb0 0x7fef281f26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.895+0000 7fef2c96c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef281553a0 0x7fef280ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.895+0000 7fef2c96c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef2815ee30 0x7fef280cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.895+0000 7fef2c96c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef280cc510 con 0x7fef2815ee30 2023-12-19T18:15:45.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.895+0000 7fef2c96c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef281f2fa0 con 0x7fef280afeb0 2023-12-19T18:15:45.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.895+0000 7fef2c96c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef281f3280 con 0x7fef281553a0 2023-12-19T18:15:45.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.895+0000 7fef27fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef2815ee30 0x7fef280cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:45.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.895+0000 7fef27fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef2815ee30 0x7fef280cccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47964/0 (socket says 172.21.15.27:47964) 2023-12-19T18:15:45.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.895+0000 7fef27fff640 1 -- 172.21.15.27:0/4005153811 learned_addr learned my addr 172.21.15.27:0/4005153811 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:45.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.895+0000 7fef277fe640 1 --2- 172.21.15.27:0/4005153811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef280afeb0 0x7fef281f26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:45.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.896+0000 7fef27fff640 1 -- 172.21.15.27:0/4005153811 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef281553a0 msgr2=0x7fef280ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:45.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.896+0000 7fef27fff640 1 --2- 172.21.15.27:0/4005153811 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef281553a0 0x7fef280ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.896+0000 7fef27fff640 1 -- 172.21.15.27:0/4005153811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef280afeb0 msgr2=0x7fef281f26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:45.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.896+0000 7fef27fff640 1 --2- 172.21.15.27:0/4005153811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef280afeb0 0x7fef281f26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:45.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.896+0000 7fef27fff640 1 -- 172.21.15.27:0/4005153811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef20009b30 con 0x7fef2815ee30 2023-12-19T18:15:45.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.896+0000 7fef27fff640 1 --2- 172.21.15.27:0/4005153811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef2815ee30 0x7fef280cccb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fef20009e50 tx=0x7fef20004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:45.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.896+0000 7fef24ff9640 1 -- 172.21.15.27:0/4005153811 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef20009510 con 0x7fef2815ee30 2023-12-19T18:15:45.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.896+0000 7fef24ff9640 1 -- 172.21.15.27:0/4005153811 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef2003e450 con 0x7fef2815ee30 2023-12-19T18:15:45.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.896+0000 7fef24ff9640 1 -- 172.21.15.27:0/4005153811 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef2003e800 con 0x7fef2815ee30 2023-12-19T18:15:45.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.896+0000 7fef2c96c640 1 -- 172.21.15.27:0/4005153811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef281f3560 con 0x7fef2815ee30 2023-12-19T18:15:45.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.896+0000 7fef2c96c640 1 -- 172.21.15.27:0/4005153811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef281f2c20 con 0x7fef2815ee30 2023-12-19T18:15:45.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.898+0000 7fef2c96c640 1 -- 172.21.15.27:0/4005153811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef2804e840 con 0x7fef2815ee30 2023-12-19T18:15:45.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.898+0000 7fef24ff9640 1 -- 172.21.15.27:0/4005153811 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fef2003e960 con 0x7fef2815ee30 2023-12-19T18:15:45.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.900+0000 7fef24ff9640 1 --2- 172.21.15.27:0/4005153811 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fef0806a430 0x7fef0806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:45.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.900+0000 7fef24ff9640 1 -- 172.21.15.27:0/4005153811 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fef200ab870 con 0x7fef2815ee30 2023-12-19T18:15:45.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.900+0000 7fef277fe640 1 --2- 172.21.15.27:0/4005153811 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fef0806a430 0x7fef0806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:45.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.901+0000 7fef277fe640 1 --2- 172.21.15.27:0/4005153811 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fef0806a430 0x7fef0806c8f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fef18000f60 tx=0x7fef18009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:45.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:45.904+0000 7fef24ff9640 1 -- 172.21.15.27:0/4005153811 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fef20076e50 con 0x7fef2815ee30 2023-12-19T18:15:46.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.165+0000 7fef2c96c640 1 -- 172.21.15.27:0/4005153811 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7fef280c9ae0 con 0x7fef2815ee30 2023-12-19T18:15:46.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.622+0000 7fef24ff9640 1 -- 172.21.15.27:0/4005153811 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v59) v1 ==== 133+0+0 (secure 0 0 0) 0x7fef2007a470 con 0x7fef2815ee30 2023-12-19T18:15:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.626+0000 7fef2c96c640 1 -- 172.21.15.27:0/4005153811 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fef0806a430 msgr2=0x7fef0806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.626+0000 7fef2c96c640 1 --2- 172.21.15.27:0/4005153811 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fef0806a430 0x7fef0806c8f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fef18000f60 tx=0x7fef18009040 comp rx=0 tx=0).stop 2023-12-19T18:15:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.626+0000 7fef2c96c640 1 -- 172.21.15.27:0/4005153811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef2815ee30 msgr2=0x7fef280cccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.626+0000 7fef2c96c640 1 --2- 172.21.15.27:0/4005153811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef2815ee30 0x7fef280cccb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fef20009e50 tx=0x7fef20004a40 comp rx=0 tx=0).stop 2023-12-19T18:15:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.626+0000 7fef2c96c640 1 -- 172.21.15.27:0/4005153811 shutdown_connections 2023-12-19T18:15:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.626+0000 7fef2c96c640 1 --2- 172.21.15.27:0/4005153811 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fef0806a430 0x7fef0806c8f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.626+0000 7fef2c96c640 1 --2- 172.21.15.27:0/4005153811 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef2815ee30 0x7fef280cccb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.626+0000 7fef2c96c640 1 --2- 172.21.15.27:0/4005153811 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef281553a0 0x7fef280ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.626+0000 7fef2c96c640 1 --2- 172.21.15.27:0/4005153811 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef280afeb0 0x7fef281f26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.626+0000 7fef2c96c640 1 -- 172.21.15.27:0/4005153811 >> 172.21.15.27:0/4005153811 conn(0x7fef28069c70 msgr2=0x7fef28158d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.626+0000 7fef2c96c640 1 -- 172.21.15.27:0/4005153811 shutdown_connections 2023-12-19T18:15:46.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:46.626+0000 7fef2c96c640 1 -- 172.21.15.27:0/4005153811 wait complete. 2023-12-19T18:15:46.647 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-12-19T18:15:46.901 INFO:teuthology.orchestra.run.smithi027.stdout:{"osd_objectstore":"memstore"} 2023-12-19T18:15:46.902 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-12-19T18:15:46.902 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:15:47.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.141+0000 7fcdba2a8640 1 Processor -- start 2023-12-19T18:15:47.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.142+0000 7fcdba2a8640 1 -- start start 2023-12-19T18:15:47.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.143+0000 7fcdba2a8640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcdb40d1a20 0x7fcdb40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:47.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.143+0000 7fcdba2a8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb40d3850 con 0x7fcdb40b3980 2023-12-19T18:15:47.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.143+0000 7fcdba2a8640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdb40d0b10 con 0x7fcdb415d120 2023-12-19T18:15:47.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.143+0000 7fcdba2a8640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdb415d500 con 0x7fcdb40d1a20 2023-12-19T18:15:47.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.144+0000 7fcdb9aa7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcdb40b3980 0x7fcdb40cd940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49340/0 (socket says 172.21.15.27:49340) 2023-12-19T18:15:47.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.144+0000 7fcdb9aa7640 1 -- 172.21.15.27:0/3694897262 learned_addr learned my addr 172.21.15.27:0/3694897262 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:47.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.144+0000 7fcdb8aa5640 1 --2- 172.21.15.27:0/3694897262 >> v2:172.21.15.192:3300/0 conn(0x7fcdb40d1a20 0x7fcdb40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:47.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.144+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1878678304 0 0) 0x7fcdb40d3850 con 0x7fcdb40b3980 2023-12-19T18:15:47.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.144+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcda0003520 con 0x7fcdb40b3980 2023-12-19T18:15:47.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2811033246 0 0) 0x7fcda0003520 con 0x7fcdb40b3980 2023-12-19T18:15:47.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 >> v2:172.21.15.192:3300/0 conn(0x7fcdb40d1a20 msgr2=0x7fcdb40cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:47.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdabfff640 1 --2- 172.21.15.27:0/3694897262 >> v2:172.21.15.192:3300/0 conn(0x7fcdb40d1a20 0x7fcdb40cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:47.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 >> v1:172.21.15.192:6789/0 conn(0x7fcdb415d120 legacy=0x7fcdb40d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:47.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb40d1040 con 0x7fcdb40b3980 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fcdac001670 con 0x7fcdb40b3980 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 >> v1:172.21.15.27:6789/0 conn(0x7fcdb40b3980 legacy=0x7fcdb40cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdabfff640 1 --2- 172.21.15.27:0/3694897262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcda00039c0 0x7fcda0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdabfff640 1 --2- 172.21.15.27:0/3694897262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcda00047c0 0x7fcda0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdabfff640 1 --2- 172.21.15.27:0/3694897262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcda0005180 0x7fcda000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb415d500 con 0x7fcda00047c0 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcda000c080 con 0x7fcda00039c0 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcda000c240 con 0x7fcda0005180 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.145+0000 7fcdb8aa5640 1 --2- 172.21.15.27:0/3694897262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcda00047c0 0x7fcda0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.146+0000 7fcdb92a6640 1 --2- 172.21.15.27:0/3694897262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcda00039c0 0x7fcda0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.146+0000 7fcdb8aa5640 1 -- 172.21.15.27:0/3694897262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcda0005180 msgr2=0x7fcda000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.146+0000 7fcdb8aa5640 1 --2- 172.21.15.27:0/3694897262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcda0005180 0x7fcda000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.146+0000 7fcdb8aa5640 1 -- 172.21.15.27:0/3694897262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcda00039c0 msgr2=0x7fcda0003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:47.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.146+0000 7fcdb8aa5640 1 --2- 172.21.15.27:0/3694897262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcda00039c0 0x7fcda0003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:47.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.146+0000 7fcdb8aa5640 1 -- 172.21.15.27:0/3694897262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda0003520 con 0x7fcda00047c0 2023-12-19T18:15:47.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.146+0000 7fcdb9aa7640 1 -- 172.21.15.27:0/3694897262 reap_dead start 2023-12-19T18:15:47.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.147+0000 7fcdb8aa5640 1 --2- 172.21.15.27:0/3694897262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcda00047c0 0x7fcda0004c40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009ac0 tx=0x7fcda00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56572b659095383 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:47.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.147+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda402c6b0 con 0x7fcda00047c0 2023-12-19T18:15:47.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.147+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda402c810 con 0x7fcda00047c0 2023-12-19T18:15:47.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.147+0000 7fcdabfff640 1 -- 172.21.15.27:0/3694897262 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda402cba0 con 0x7fcda00047c0 2023-12-19T18:15:47.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.148+0000 7fcdba2a8640 1 -- 172.21.15.27:0/3694897262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcda00047c0 msgr2=0x7fcda0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:47.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.148+0000 7fcdba2a8640 1 --2- 172.21.15.27:0/3694897262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcda00047c0 0x7fcda0004c40 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fcda4009ac0 tx=0x7fcda00057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:47.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.148+0000 7fcdba2a8640 1 -- 172.21.15.27:0/3694897262 shutdown_connections 2023-12-19T18:15:47.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.148+0000 7fcdba2a8640 1 --2- 172.21.15.27:0/3694897262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcda00047c0 0x7fcda0004c40 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:47.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.148+0000 7fcdba2a8640 1 -- 172.21.15.27:0/3694897262 >> 172.21.15.27:0/3694897262 conn(0x7fcdb40d2d30 msgr2=0x7fcdb40d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:47.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.148+0000 7fcdba2a8640 1 -- 172.21.15.27:0/3694897262 shutdown_connections 2023-12-19T18:15:47.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.148+0000 7fcdba2a8640 1 -- 172.21.15.27:0/3694897262 wait complete. 2023-12-19T18:15:47.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.150+0000 7fcdba2a8640 1 Processor -- start 2023-12-19T18:15:47.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.150+0000 7fcdba2a8640 1 -- start start 2023-12-19T18:15:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.151+0000 7fcdba2a8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcda00047c0 0x7fcdb415e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.151+0000 7fcdba2a8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdb40b5950 0x7fcdb415f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.151+0000 7fcdba2a8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdb415a900 0x7fcdb41f58a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.151+0000 7fcdba2a8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb415b1b0 con 0x7fcdb40b5950 2023-12-19T18:15:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.151+0000 7fcdba2a8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb415e760 con 0x7fcda00047c0 2023-12-19T18:15:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.151+0000 7fcdba2a8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb40b5ec0 con 0x7fcdb415a900 2023-12-19T18:15:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.151+0000 7fcdb8aa5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdb40b5950 0x7fcdb415f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:47.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.151+0000 7fcdb8aa5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdb40b5950 0x7fcdb415f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47978/0 (socket says 172.21.15.27:47978) 2023-12-19T18:15:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.151+0000 7fcdb8aa5640 1 -- 172.21.15.27:0/1807792215 learned_addr learned my addr 172.21.15.27:0/1807792215 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.152+0000 7fcdb92a6640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcda00047c0 0x7fcdb415e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.152+0000 7fcdb9aa7640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdb415a900 0x7fcdb41f58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.152+0000 7fcdb8aa5640 1 -- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdb415a900 msgr2=0x7fcdb41f58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.152+0000 7fcdb8aa5640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdb415a900 0x7fcdb41f58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.152+0000 7fcdb8aa5640 1 -- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcda00047c0 msgr2=0x7fcdb415e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.152+0000 7fcdb8aa5640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcda00047c0 0x7fcdb415e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:47.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.152+0000 7fcdb8aa5640 1 -- 172.21.15.27:0/1807792215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb40b61a0 con 0x7fcdb40b5950 2023-12-19T18:15:47.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.152+0000 7fcdb9aa7640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdb415a900 0x7fcdb41f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:47.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.152+0000 7fcdb8aa5640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdb40b5950 0x7fcdb415f170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcda402c250 tx=0x7fcda4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:47.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.153+0000 7fcda9ffb640 1 -- 172.21.15.27:0/1807792215 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4003810 con 0x7fcdb40b5950 2023-12-19T18:15:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.153+0000 7fcda9ffb640 1 -- 172.21.15.27:0/1807792215 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda4003970 con 0x7fcdb40b5950 2023-12-19T18:15:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.153+0000 7fcda9ffb640 1 -- 172.21.15.27:0/1807792215 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda4003d00 con 0x7fcdb40b5950 2023-12-19T18:15:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.153+0000 7fcdba2a8640 1 -- 172.21.15.27:0/1807792215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb40b6480 con 0x7fcdb40b5950 2023-12-19T18:15:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.153+0000 7fcdba2a8640 1 -- 172.21.15.27:0/1807792215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb40b6760 con 0x7fcdb40b5950 2023-12-19T18:15:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.154+0000 7fcdba2a8640 1 -- 172.21.15.27:0/1807792215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdb404e840 con 0x7fcdb40b5950 2023-12-19T18:15:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.155+0000 7fcda9ffb640 1 -- 172.21.15.27:0/1807792215 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcda4047450 con 0x7fcdb40b5950 2023-12-19T18:15:47.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.156+0000 7fcda9ffb640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcd9806a430 0x7fcd9806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:47.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.156+0000 7fcda9ffb640 1 -- 172.21.15.27:0/1807792215 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcda40aabe0 con 0x7fcdb40b5950 2023-12-19T18:15:47.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.157+0000 7fcdb92a6640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcd9806a430 0x7fcd9806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:47.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.158+0000 7fcdb92a6640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcd9806a430 0x7fcd9806c8f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0007fd0 tx=0x7fcdb0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:47.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.161+0000 7fcda9ffb640 1 -- 172.21.15.27:0/1807792215 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcda40762f0 con 0x7fcdb40b5950 2023-12-19T18:15:47.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.354+0000 7fcdba2a8640 1 -- 172.21.15.27:0/1807792215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcdb406bcf0 con 0x7fcdb40b5950 2023-12-19T18:15:47.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.359+0000 7fcda9ffb640 1 -- 172.21.15.27:0/1807792215 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcda4003e60 con 0x7fcdb40b5950 2023-12-19T18:15:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.362+0000 7fcdba2a8640 1 -- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcd9806a430 msgr2=0x7fcd9806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.362+0000 7fcdba2a8640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcd9806a430 0x7fcd9806c8f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fcdb0007fd0 tx=0x7fcdb0009040 comp rx=0 tx=0).stop 2023-12-19T18:15:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.362+0000 7fcdba2a8640 1 -- 172.21.15.27:0/1807792215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdb40b5950 msgr2=0x7fcdb415f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.362+0000 7fcdba2a8640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdb40b5950 0x7fcdb415f170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fcda402c250 tx=0x7fcda4004730 comp rx=0 tx=0).stop 2023-12-19T18:15:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.362+0000 7fcdba2a8640 1 -- 172.21.15.27:0/1807792215 shutdown_connections 2023-12-19T18:15:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.362+0000 7fcdba2a8640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcd9806a430 0x7fcd9806c8f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.362+0000 7fcdba2a8640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdb415a900 0x7fcdb41f58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.362+0000 7fcdba2a8640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdb40b5950 0x7fcdb415f170 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.362+0000 7fcdba2a8640 1 --2- 172.21.15.27:0/1807792215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcda00047c0 0x7fcdb415e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.362+0000 7fcdba2a8640 1 -- 172.21.15.27:0/1807792215 >> 172.21.15.27:0/1807792215 conn(0x7fcdb40d2d30 msgr2=0x7fcdb40cfd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:47.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.362+0000 7fcdba2a8640 1 -- 172.21.15.27:0/1807792215 shutdown_connections 2023-12-19T18:15:47.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.362+0000 7fcdba2a8640 1 -- 172.21.15.27:0/1807792215 wait complete. 2023-12-19T18:15:47.380 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:15:47.380 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:15:47.510 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:15:47.633 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:15:47.634 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:15:47.634 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:15:47.701 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:15:47.701 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:15:47.761 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:47.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.993+0000 7f3aa296c640 1 Processor -- start 2023-12-19T18:15:47.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.993+0000 7f3aa296c640 1 -- start start 2023-12-19T18:15:47.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.995+0000 7f3aa296c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3a9c0cdf90 0x7f3a9c164d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:47.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.995+0000 7f3aa296c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3a9c165290 0x7f3a9c06f340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:47.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.995+0000 7f3aa296c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a9c0c4a20 con 0x7f3a9c15b870 2023-12-19T18:15:47.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.995+0000 7f3aa296c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c165740 con 0x7f3a9c165290 2023-12-19T18:15:47.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.995+0000 7f3aa296c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a9c0ce750 con 0x7f3a9c0cdf90 2023-12-19T18:15:47.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.996+0000 7f3aa216b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3a9c165290 0x7f3a9c06f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:47.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.996+0000 7f3aa216b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3a9c165290 0x7f3a9c06f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47990/0 (socket says 172.21.15.27:47990) 2023-12-19T18:15:47.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.996+0000 7f3aa216b640 1 -- 172.21.15.27:0/3573132825 learned_addr learned my addr 172.21.15.27:0/3573132825 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:47.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.996+0000 7f3aa1169640 1 --2- 172.21.15.27:0/3573132825 >> v2:172.21.15.192:3300/0 conn(0x7f3a9c0cdf90 0x7f3a9c164d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:47.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.996+0000 7f3aa216b640 1 -- 172.21.15.27:0/3573132825 >> v2:172.21.15.192:3300/0 conn(0x7f3a9c0cdf90 msgr2=0x7f3a9c164d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:47.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.996+0000 7f3aa216b640 1 --2- 172.21.15.27:0/3573132825 >> v2:172.21.15.192:3300/0 conn(0x7f3a9c0cdf90 0x7f3a9c164d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:47.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.996+0000 7f3aa216b640 1 -- 172.21.15.27:0/3573132825 >> v1:172.21.15.192:6789/0 conn(0x7f3a9c15b870 legacy=0x7f3a9c0c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:47.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.996+0000 7f3aa216b640 1 -- 172.21.15.27:0/3573132825 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c15bc50 con 0x7f3a9c165290 2023-12-19T18:15:47.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.998+0000 7f3aa216b640 1 --2- 172.21.15.27:0/3573132825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c165290 0x7f3a9c06f340 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0ccb90 tx=0x7f3a9402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6beffc5f0185476c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:47.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.998+0000 7f3aa0968640 1 -- 172.21.15.27:0/3573132825 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a9402ca80 con 0x7f3a9c165290 2023-12-19T18:15:47.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.998+0000 7f3aa0968640 1 -- 172.21.15.27:0/3573132825 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a9402cbe0 con 0x7f3a9c165290 2023-12-19T18:15:47.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.998+0000 7f3aa0968640 1 -- 172.21.15.27:0/3573132825 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a940343f0 con 0x7f3a9c165290 2023-12-19T18:15:47.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.999+0000 7f3aa296c640 1 -- 172.21.15.27:0/3573132825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c165290 msgr2=0x7f3a9c06f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.999+0000 7f3aa296c640 1 --2- 172.21.15.27:0/3573132825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c165290 0x7f3a9c06f340 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0ccb90 tx=0x7f3a9402bee0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.999+0000 7f3aa296c640 1 -- 172.21.15.27:0/3573132825 shutdown_connections 2023-12-19T18:15:48.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.999+0000 7f3aa296c640 1 --2- 172.21.15.27:0/3573132825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c165290 0x7f3a9c06f340 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.999+0000 7f3aa296c640 1 --2- 172.21.15.27:0/3573132825 >> v2:172.21.15.192:3300/0 conn(0x7f3a9c0cdf90 0x7f3a9c164d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.999+0000 7f3aa296c640 1 -- 172.21.15.27:0/3573132825 >> 172.21.15.27:0/3573132825 conn(0x7f3a9c069c70 msgr2=0x7f3a9c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:48.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.999+0000 7f3aa296c640 1 -- 172.21.15.27:0/3573132825 shutdown_connections 2023-12-19T18:15:48.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:47.999+0000 7f3aa296c640 1 -- 172.21.15.27:0/3573132825 wait complete. 2023-12-19T18:15:48.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.001+0000 7f3aa296c640 1 Processor -- start 2023-12-19T18:15:48.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.001+0000 7f3aa296c640 1 -- start start 2023-12-19T18:15:48.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.002+0000 7f3aa296c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0cdf90 0x7f3a9c1f7880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.002+0000 7f3aa296c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a9c15b870 0x7f3a9c14fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.002+0000 7f3aa296c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a9c165290 0x7f3a9c0c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.002+0000 7f3aa296c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0c6a30 con 0x7f3a9c0cdf90 2023-12-19T18:15:48.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.002+0000 7f3aa296c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c150140 con 0x7f3a9c165290 2023-12-19T18:15:48.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.002+0000 7f3aa296c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c0c5f50 con 0x7f3a9c15b870 2023-12-19T18:15:48.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.003+0000 7f3aa196a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0cdf90 0x7f3a9c1f7880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.003+0000 7f3aa196a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0cdf90 0x7f3a9c1f7880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48000/0 (socket says 172.21.15.27:48000) 2023-12-19T18:15:48.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.003+0000 7f3aa196a640 1 -- 172.21.15.27:0/2508509442 learned_addr learned my addr 172.21.15.27:0/2508509442 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:48.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.003+0000 7f3aa1169640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a9c15b870 0x7f3a9c14fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.003+0000 7f3aa216b640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a9c165290 0x7f3a9c0c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.003+0000 7f3aa196a640 1 -- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a9c15b870 msgr2=0x7f3a9c14fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.003+0000 7f3aa196a640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a9c15b870 0x7f3a9c14fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.003+0000 7f3aa196a640 1 -- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a9c165290 msgr2=0x7f3a9c0c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.003+0000 7f3aa196a640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a9c165290 0x7f3a9c0c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.003+0000 7f3aa196a640 1 -- 172.21.15.27:0/2508509442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a94009b30 con 0x7f3a9c0cdf90 2023-12-19T18:15:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.004+0000 7f3aa196a640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0cdf90 0x7f3a9c1f7880 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800c430 tx=0x7f3a9800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.004+0000 7f3a927fc640 1 -- 172.21.15.27:0/2508509442 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98011790 con 0x7f3a9c0cdf90 2023-12-19T18:15:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.004+0000 7f3a927fc640 1 -- 172.21.15.27:0/2508509442 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a980118f0 con 0x7f3a9c0cdf90 2023-12-19T18:15:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.004+0000 7f3a927fc640 1 -- 172.21.15.27:0/2508509442 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a98011ca0 con 0x7f3a9c0cdf90 2023-12-19T18:15:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.004+0000 7f3aa296c640 1 -- 172.21.15.27:0/2508509442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a9c0c6190 con 0x7f3a9c0cdf90 2023-12-19T18:15:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.004+0000 7f3aa296c640 1 -- 172.21.15.27:0/2508509442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a9c0c6750 con 0x7f3a9c0cdf90 2023-12-19T18:15:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.005+0000 7f3aa296c640 1 -- 172.21.15.27:0/2508509442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a9c0c4e90 con 0x7f3a9c0cdf90 2023-12-19T18:15:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.006+0000 7f3a927fc640 1 -- 172.21.15.27:0/2508509442 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3a9801c460 con 0x7f3a9c0cdf90 2023-12-19T18:15:48.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.007+0000 7f3a927fc640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3a8006a430 0x7f3a8006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.007+0000 7f3a927fc640 1 -- 172.21.15.27:0/2508509442 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3a980888a0 con 0x7f3a9c0cdf90 2023-12-19T18:15:48.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.008+0000 7f3aa1169640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3a8006a430 0x7f3a8006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.009+0000 7f3aa1169640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3a8006a430 0x7f3a8006c8f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0b47b0 tx=0x7f3a8c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:48.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.012+0000 7f3a927fc640 1 -- 172.21.15.27:0/2508509442 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3a9c0c4e90 con 0x7f3a9c0cdf90 2023-12-19T18:15:48.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.205+0000 7f3aa296c640 1 -- 172.21.15.27:0/2508509442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a9c0c4e90 con 0x7f3a9c0cdf90 2023-12-19T18:15:48.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.208+0000 7f3a927fc640 1 -- 172.21.15.27:0/2508509442 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16090 (secure 0 0 0) 0x7f3a9c0c4e90 con 0x7f3a9c0cdf90 2023-12-19T18:15:48.209 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:48.209 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:15:43.580746+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:48.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.211+0000 7f3aa296c640 1 -- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3a8006a430 msgr2=0x7f3a8006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.211+0000 7f3aa296c640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3a8006a430 0x7f3a8006c8f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3a9c0b47b0 tx=0x7f3a8c008040 comp rx=0 tx=0).stop 2023-12-19T18:15:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.211+0000 7f3aa296c640 1 -- 172.21.15.27:0/2508509442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0cdf90 msgr2=0x7f3a9c1f7880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.211+0000 7f3aa296c640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0cdf90 0x7f3a9c1f7880 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3a9800c430 tx=0x7f3a9800c8f0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.211+0000 7f3aa296c640 1 -- 172.21.15.27:0/2508509442 shutdown_connections 2023-12-19T18:15:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.211+0000 7f3aa296c640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3a8006a430 0x7f3a8006c8f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.211+0000 7f3aa296c640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a9c165290 0x7f3a9c0c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.211+0000 7f3aa296c640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a9c15b870 0x7f3a9c14fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.211+0000 7f3aa296c640 1 --2- 172.21.15.27:0/2508509442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0cdf90 0x7f3a9c1f7880 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.211+0000 7f3aa296c640 1 -- 172.21.15.27:0/2508509442 >> 172.21.15.27:0/2508509442 conn(0x7f3a9c069c70 msgr2=0x7f3a9c162960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.211+0000 7f3aa296c640 1 -- 172.21.15.27:0/2508509442 shutdown_connections 2023-12-19T18:15:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.211+0000 7f3aa296c640 1 -- 172.21.15.27:0/2508509442 wait complete. 2023-12-19T18:15:48.232 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:15:48.233 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:15:48.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.468+0000 7f401ec0e640 1 Processor -- start 2023-12-19T18:15:48.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.468+0000 7f401ec0e640 1 -- start start 2023-12-19T18:15:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.470+0000 7f401ec0e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4018143450 0x7f40180ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.470+0000 7f401ec0e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40180b27d0 con 0x7f401806c850 2023-12-19T18:15:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.470+0000 7f401ec0e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40180cf850 con 0x7f40180cf430 2023-12-19T18:15:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.470+0000 7f401ec0e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40180b33e0 con 0x7f4018143450 2023-12-19T18:15:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.470+0000 7f401e40d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f401806c850 0x7f40180cbdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49344/0 (socket says 172.21.15.27:49344) 2023-12-19T18:15:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.470+0000 7f401e40d640 1 -- 172.21.15.27:0/712145509 learned_addr learned my addr 172.21.15.27:0/712145509 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401dc0c640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4018143450 0x7f40180ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3606978931 0 0) 0x7f40180b27d0 con 0x7f401806c850 2023-12-19T18:15:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4004005730 con 0x7f401806c850 2023-12-19T18:15:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3622519489 0 0) 0x7f4004005730 con 0x7f401806c850 2023-12-19T18:15:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4018143450 msgr2=0x7f40180ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4018143450 0x7f40180ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 >> v1:172.21.15.192:6789/0 conn(0x7f40180cf430 legacy=0x7f401806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40180b2a40 con 0x7f401806c850 2023-12-19T18:15:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f40100019f0 con 0x7f401806c850 2023-12-19T18:15:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 >> v1:172.21.15.27:6789/0 conn(0x7f401806c850 legacy=0x7f40180cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4004005bb0 0x7f4004005f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4018143450 0x7f40040069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4004006f10 0x7f4004007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40180b33e0 con 0x7f4004005bb0 2023-12-19T18:15:48.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f400400dc40 con 0x7f4004006f10 2023-12-19T18:15:48.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.471+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f400400de00 con 0x7f4018143450 2023-12-19T18:15:48.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.472+0000 7f401dc0c640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4004005bb0 0x7f4004005f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.472+0000 7f401e40d640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4004006f10 0x7f4004007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.472+0000 7f401d40b640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4018143450 0x7f40040069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.472+0000 7f401dc0c640 1 -- 172.21.15.27:0/712145509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4018143450 msgr2=0x7f40040069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.472+0000 7f401dc0c640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4018143450 0x7f40040069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.472+0000 7f401dc0c640 1 -- 172.21.15.27:0/712145509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4004006f10 msgr2=0x7f4004007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.472+0000 7f401dc0c640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4004006f10 0x7f4004007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.472+0000 7f401dc0c640 1 -- 172.21.15.27:0/712145509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4004005730 con 0x7f4004005bb0 2023-12-19T18:15:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.472+0000 7f401d40b640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4018143450 0x7f40040069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.472+0000 7f401e40d640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4004006f10 0x7f4004007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.473+0000 7f401dc0c640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4004005bb0 0x7f4004005f90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f400c009ac0 tx=0x7f4004007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5859c33d7908a6b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.474+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c02c6b0 con 0x7f4004005bb0 2023-12-19T18:15:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.474+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f400c02c810 con 0x7f4004005bb0 2023-12-19T18:15:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.474+0000 7f401cc0a640 1 -- 172.21.15.27:0/712145509 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f400c02cba0 con 0x7f4004005bb0 2023-12-19T18:15:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.474+0000 7f401ec0e640 1 -- 172.21.15.27:0/712145509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4004005bb0 msgr2=0x7f4004005f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.474+0000 7f401ec0e640 1 --2- 172.21.15.27:0/712145509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4004005bb0 0x7f4004005f90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f400c009ac0 tx=0x7f4004007a00 comp rx=0 tx=0).stop 2023-12-19T18:15:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.474+0000 7f401e40d640 1 -- 172.21.15.27:0/712145509 reap_dead start 2023-12-19T18:15:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.474+0000 7f401ec0e640 1 -- 172.21.15.27:0/712145509 shutdown_connections 2023-12-19T18:15:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.474+0000 7f401ec0e640 1 -- 172.21.15.27:0/712145509 >> 172.21.15.27:0/712145509 conn(0x7f401806a060 msgr2=0x7f401806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.474+0000 7f401ec0e640 1 -- 172.21.15.27:0/712145509 shutdown_connections 2023-12-19T18:15:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.474+0000 7f401ec0e640 1 -- 172.21.15.27:0/712145509 wait complete. 2023-12-19T18:15:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.476+0000 7f401ec0e640 1 Processor -- start 2023-12-19T18:15:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.476+0000 7f401ec0e640 1 -- start start 2023-12-19T18:15:48.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.477+0000 7f401ec0e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4004005bb0 0x7f4018159370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.477+0000 7f401ec0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f401815b8b0 0x7f401815bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.477+0000 7f401ec0e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40180cc820 0x7f40180ccca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.477+0000 7f401ec0e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40181598b0 con 0x7f401815b8b0 2023-12-19T18:15:48.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.477+0000 7f401ec0e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40180cd1e0 con 0x7f4004005bb0 2023-12-19T18:15:48.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.477+0000 7f401ec0e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f401815cf80 con 0x7f40180cc820 2023-12-19T18:15:48.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.478+0000 7f401d40b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f401815b8b0 0x7f401815bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.478+0000 7f401d40b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f401815b8b0 0x7f401815bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48014/0 (socket says 172.21.15.27:48014) 2023-12-19T18:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.478+0000 7f401d40b640 1 -- 172.21.15.27:0/1158436447 learned_addr learned my addr 172.21.15.27:0/1158436447 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.478+0000 7f401e40d640 1 --2- 172.21.15.27:0/1158436447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40180cc820 0x7f40180ccca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.478+0000 7f401d40b640 1 -- 172.21.15.27:0/1158436447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40180cc820 msgr2=0x7f40180ccca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.478+0000 7f401d40b640 1 --2- 172.21.15.27:0/1158436447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40180cc820 0x7f40180ccca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.478+0000 7f401d40b640 1 -- 172.21.15.27:0/1158436447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4004005bb0 msgr2=0x7f4018159370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.478+0000 7f401d40b640 1 --2- 172.21.15.27:0/1158436447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4004005bb0 0x7f4018159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.478+0000 7f401d40b640 1 -- 172.21.15.27:0/1158436447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401815c370 con 0x7f401815b8b0 2023-12-19T18:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.479+0000 7f401d40b640 1 --2- 172.21.15.27:0/1158436447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f401815b8b0 0x7f401815bd30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f401400d820 tx=0x7f401400dce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.479+0000 7f400a7fc640 1 -- 172.21.15.27:0/1158436447 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4014005480 con 0x7f401815b8b0 2023-12-19T18:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.479+0000 7f400a7fc640 1 -- 172.21.15.27:0/1158436447 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401400de10 con 0x7f401815b8b0 2023-12-19T18:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.479+0000 7f400a7fc640 1 -- 172.21.15.27:0/1158436447 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401400b8d0 con 0x7f401815b8b0 2023-12-19T18:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.479+0000 7f401ec0e640 1 -- 172.21.15.27:0/1158436447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f401815c650 con 0x7f401815b8b0 2023-12-19T18:15:48.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.479+0000 7f401ec0e640 1 -- 172.21.15.27:0/1158436447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f401815cc10 con 0x7f401815b8b0 2023-12-19T18:15:48.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.480+0000 7f401ec0e640 1 -- 172.21.15.27:0/1158436447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f401804e840 con 0x7f401815b8b0 2023-12-19T18:15:48.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.481+0000 7f400a7fc640 1 -- 172.21.15.27:0/1158436447 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f401400ba30 con 0x7f401815b8b0 2023-12-19T18:15:48.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.482+0000 7f400a7fc640 1 --2- 172.21.15.27:0/1158436447 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3ffc06a430 0x7f3ffc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.482+0000 7f400a7fc640 1 -- 172.21.15.27:0/1158436447 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f4014088a60 con 0x7f401815b8b0 2023-12-19T18:15:48.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.483+0000 7f401dc0c640 1 --2- 172.21.15.27:0/1158436447 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3ffc06a430 0x7f3ffc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.484+0000 7f401dc0c640 1 --2- 172.21.15.27:0/1158436447 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3ffc06a430 0x7f3ffc06c8f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f400c036000 tx=0x7f400c02e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:48.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.486+0000 7f400a7fc640 1 -- 172.21.15.27:0/1158436447 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4014054030 con 0x7f401815b8b0 2023-12-19T18:15:48.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.682+0000 7f401ec0e640 1 -- 172.21.15.27:0/1158436447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f401815cdd0 con 0x7f401815b8b0 2023-12-19T18:15:48.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.683+0000 7f400a7fc640 1 -- 172.21.15.27:0/1158436447 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4014057650 con 0x7f401815b8b0 2023-12-19T18:15:48.683 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:15:48.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.686+0000 7f401ec0e640 1 -- 172.21.15.27:0/1158436447 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3ffc06a430 msgr2=0x7f3ffc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.686+0000 7f401ec0e640 1 --2- 172.21.15.27:0/1158436447 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3ffc06a430 0x7f3ffc06c8f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f400c036000 tx=0x7f400c02e000 comp rx=0 tx=0).stop 2023-12-19T18:15:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.686+0000 7f401ec0e640 1 -- 172.21.15.27:0/1158436447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f401815b8b0 msgr2=0x7f401815bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.686+0000 7f401ec0e640 1 --2- 172.21.15.27:0/1158436447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f401815b8b0 0x7f401815bd30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f401400d820 tx=0x7f401400dce0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.686+0000 7f401ec0e640 1 -- 172.21.15.27:0/1158436447 shutdown_connections 2023-12-19T18:15:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.686+0000 7f401ec0e640 1 --2- 172.21.15.27:0/1158436447 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3ffc06a430 0x7f3ffc06c8f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.686+0000 7f401ec0e640 1 --2- 172.21.15.27:0/1158436447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40180cc820 0x7f40180ccca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.686+0000 7f401ec0e640 1 --2- 172.21.15.27:0/1158436447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f401815b8b0 0x7f401815bd30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.686+0000 7f401ec0e640 1 --2- 172.21.15.27:0/1158436447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4004005bb0 0x7f4018159370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.686+0000 7f401ec0e640 1 -- 172.21.15.27:0/1158436447 >> 172.21.15.27:0/1158436447 conn(0x7f401806a060 msgr2=0x7f401806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.686+0000 7f401ec0e640 1 -- 172.21.15.27:0/1158436447 shutdown_connections 2023-12-19T18:15:48.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.686+0000 7f401ec0e640 1 -- 172.21.15.27:0/1158436447 wait complete. 2023-12-19T18:15:48.704 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:15:48.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.915+0000 7fe08a8dc640 1 Processor -- start 2023-12-19T18:15:48.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.915+0000 7fe08a8dc640 1 -- start start 2023-12-19T18:15:48.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.917+0000 7fe08a8dc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe084143450 0x7fe0840ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.917+0000 7fe08a8dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0840cf430 0x7fe08406c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.917+0000 7fe08a8dc640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0840b27d0 con 0x7fe08406c680 2023-12-19T18:15:48.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.917+0000 7fe08a8dc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0840b33e0 con 0x7fe084143450 2023-12-19T18:15:48.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.917+0000 7fe08a8dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840cf850 con 0x7fe0840cf430 2023-12-19T18:15:48.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.917+0000 7fe08a0db640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe08406c680 0x7fe08406b000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49346/0 (socket says 172.21.15.27:49346) 2023-12-19T18:15:48.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.917+0000 7fe08a0db640 1 -- 172.21.15.27:0/1525175897 learned_addr learned my addr 172.21.15.27:0/1525175897 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:48.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.917+0000 7fe0898da640 1 --2- 172.21.15.27:0/1525175897 >> v2:172.21.15.192:3300/0 conn(0x7fe084143450 0x7fe0840ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.918+0000 7fe0890d9640 1 --2- 172.21.15.27:0/1525175897 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0840cf430 0x7fe08406c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.918+0000 7fe0888d8640 1 -- 172.21.15.27:0/1525175897 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3938115234 0 0) 0x7fe0840b27d0 con 0x7fe08406c680 2023-12-19T18:15:48.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.918+0000 7fe0888d8640 1 -- 172.21.15.27:0/1525175897 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe070003520 con 0x7fe08406c680 2023-12-19T18:15:48.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.918+0000 7fe0888d8640 1 -- 172.21.15.27:0/1525175897 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4030725200 0 0) 0x7fe070003520 con 0x7fe08406c680 2023-12-19T18:15:48.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.918+0000 7fe0898da640 1 -- 172.21.15.27:0/1525175897 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0840cf430 msgr2=0x7fe08406c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:48.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.918+0000 7fe0898da640 1 --2- 172.21.15.27:0/1525175897 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0840cf430 0x7fe08406c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.918+0000 7fe0898da640 1 -- 172.21.15.27:0/1525175897 >> v1:172.21.15.27:6789/0 conn(0x7fe08406c680 legacy=0x7fe08406b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.918+0000 7fe0898da640 1 -- 172.21.15.27:0/1525175897 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0840cce00 con 0x7fe084143450 2023-12-19T18:15:48.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.920+0000 7fe0898da640 1 --2- 172.21.15.27:0/1525175897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe084143450 0x7fe0840ceef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe0800097a0 tx=0x7fe08002b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eec3ac5725dc79d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:48.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.920+0000 7fe0888d8640 1 -- 172.21.15.27:0/1525175897 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0019f0 con 0x7fe084143450 2023-12-19T18:15:48.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.920+0000 7fe0888d8640 1 -- 172.21.15.27:0/1525175897 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08002e050 con 0x7fe084143450 2023-12-19T18:15:48.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.920+0000 7fe0888d8640 1 -- 172.21.15.27:0/1525175897 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe080030730 con 0x7fe084143450 2023-12-19T18:15:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.921+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1525175897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe084143450 msgr2=0x7fe0840ceef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.921+0000 7fe08a8dc640 1 --2- 172.21.15.27:0/1525175897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe084143450 0x7fe0840ceef0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fe0800097a0 tx=0x7fe08002b710 comp rx=0 tx=0).stop 2023-12-19T18:15:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.921+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1525175897 shutdown_connections 2023-12-19T18:15:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.921+0000 7fe08a8dc640 1 --2- 172.21.15.27:0/1525175897 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0840cf430 0x7fe08406c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.921+0000 7fe08a8dc640 1 --2- 172.21.15.27:0/1525175897 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe084143450 0x7fe0840ceef0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.921+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1525175897 >> 172.21.15.27:0/1525175897 conn(0x7fe08406a060 msgr2=0x7fe08406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.921+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1525175897 shutdown_connections 2023-12-19T18:15:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.921+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1525175897 wait complete. 2023-12-19T18:15:48.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.923+0000 7fe08a8dc640 1 Processor -- start 2023-12-19T18:15:48.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.923+0000 7fe08a8dc640 1 -- start start 2023-12-19T18:15:48.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.924+0000 7fe08a8dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe08406c680 0x7fe084139d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.924+0000 7fe08a8dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0840cf430 0x7fe0840b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.924+0000 7fe08a8dc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe084143450 0x7fe0840b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.924+0000 7fe08a8dc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08415aa70 con 0x7fe084143450 2023-12-19T18:15:48.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.924+0000 7fe08a8dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08413a820 con 0x7fe08406c680 2023-12-19T18:15:48.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.924+0000 7fe08a8dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0841581b0 con 0x7fe0840cf430 2023-12-19T18:15:48.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.924+0000 7fe08a0db640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe084143450 0x7fe0840b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.924+0000 7fe08a0db640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe084143450 0x7fe0840b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48026/0 (socket says 172.21.15.27:48026) 2023-12-19T18:15:48.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.924+0000 7fe08a0db640 1 -- 172.21.15.27:0/1600144341 learned_addr learned my addr 172.21.15.27:0/1600144341 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:48.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.924+0000 7fe0898da640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe08406c680 0x7fe084139d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.924+0000 7fe0890d9640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0840cf430 0x7fe0840b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.925+0000 7fe08a0db640 1 -- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0840cf430 msgr2=0x7fe0840b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.925+0000 7fe08a0db640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0840cf430 0x7fe0840b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.925+0000 7fe08a0db640 1 -- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe08406c680 msgr2=0x7fe084139d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.925+0000 7fe08a0db640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe08406c680 0x7fe084139d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.925+0000 7fe08a0db640 1 -- 172.21.15.27:0/1600144341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe080009400 con 0x7fe084143450 2023-12-19T18:15:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.925+0000 7fe0898da640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe08406c680 0x7fe084139d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.925+0000 7fe08a0db640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe084143450 0x7fe0840b0bb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00cca0 tx=0x7fe07c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.925+0000 7fe07a7fc640 1 -- 172.21.15.27:0/1600144341 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c012400 con 0x7fe084143450 2023-12-19T18:15:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.925+0000 7fe07a7fc640 1 -- 172.21.15.27:0/1600144341 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c012560 con 0x7fe084143450 2023-12-19T18:15:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.925+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1600144341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe084158490 con 0x7fe084143450 2023-12-19T18:15:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.925+0000 7fe07a7fc640 1 -- 172.21.15.27:0/1600144341 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c012910 con 0x7fe084143450 2023-12-19T18:15:48.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.926+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1600144341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe084158a50 con 0x7fe084143450 2023-12-19T18:15:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.927+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1600144341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08415abe0 con 0x7fe084143450 2023-12-19T18:15:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.928+0000 7fe07a7fc640 1 -- 172.21.15.27:0/1600144341 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe07c012a70 con 0x7fe084143450 2023-12-19T18:15:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.928+0000 7fe07a7fc640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe06806a430 0x7fe06806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.929+0000 7fe07a7fc640 1 -- 172.21.15.27:0/1600144341 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe07c089100 con 0x7fe084143450 2023-12-19T18:15:48.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.929+0000 7fe0898da640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe06806a430 0x7fe06806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:48.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.930+0000 7fe0898da640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe06806a430 0x7fe06806c8f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe080034000 tx=0x7fe080008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:48.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:48.933+0000 7fe07a7fc640 1 -- 172.21.15.27:0/1600144341 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe07c0546d0 con 0x7fe084143450 2023-12-19T18:15:49.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.127+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1600144341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe08404e840 con 0x7fe084143450 2023-12-19T18:15:49.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.128+0000 7fe07a7fc640 1 -- 172.21.15.27:0/1600144341 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe07c057cf0 con 0x7fe084143450 2023-12-19T18:15:49.129 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:15:49.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.131+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe06806a430 msgr2=0x7fe06806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:49.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.131+0000 7fe08a8dc640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe06806a430 0x7fe06806c8f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fe080034000 tx=0x7fe080008030 comp rx=0 tx=0).stop 2023-12-19T18:15:49.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.131+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1600144341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe084143450 msgr2=0x7fe0840b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:49.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.131+0000 7fe08a8dc640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe084143450 0x7fe0840b0bb0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00cca0 tx=0x7fe07c00a430 comp rx=0 tx=0).stop 2023-12-19T18:15:49.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.131+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1600144341 shutdown_connections 2023-12-19T18:15:49.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.131+0000 7fe08a8dc640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe06806a430 0x7fe06806c8f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.131+0000 7fe08a8dc640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe084143450 0x7fe0840b0bb0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.131+0000 7fe08a8dc640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0840cf430 0x7fe0840b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.131+0000 7fe08a8dc640 1 --2- 172.21.15.27:0/1600144341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe08406c680 0x7fe084139d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.131+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1600144341 >> 172.21.15.27:0/1600144341 conn(0x7fe08406a060 msgr2=0x7fe0840ae500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:49.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.132+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1600144341 shutdown_connections 2023-12-19T18:15:49.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.132+0000 7fe08a8dc640 1 -- 172.21.15.27:0/1600144341 wait complete. 2023-12-19T18:15:49.151 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:15:49.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.388+0000 7fe5ab06c640 1 Processor -- start 2023-12-19T18:15:49.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.388+0000 7fe5ab06c640 1 -- start start 2023-12-19T18:15:49.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.390+0000 7fe5ab06c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe5a40d2100 0x7fe5a40cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.390+0000 7fe5ab06c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe5a40cdf80 0x7fe5a40b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.390+0000 7fe5ab06c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5a40d0e10 con 0x7fe5a415cf80 2023-12-19T18:15:49.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.390+0000 7fe5ab06c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a40b8cf0 con 0x7fe5a40cdf80 2023-12-19T18:15:49.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.390+0000 7fe5ab06c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5a40ce3f0 con 0x7fe5a40d2100 2023-12-19T18:15:49.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.390+0000 7fe5aa06a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe5a415cf80 0x7fe5a40b85e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49354/0 (socket says 172.21.15.27:49354) 2023-12-19T18:15:49.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.390+0000 7fe5aa06a640 1 -- 172.21.15.27:0/4273196720 learned_addr learned my addr 172.21.15.27:0/4273196720 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.390+0000 7fe5aa86b640 1 --2- 172.21.15.27:0/4273196720 >> v2:172.21.15.27:3300/0 conn(0x7fe5a40cdf80 0x7fe5a40b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.390+0000 7fe5a9869640 1 --2- 172.21.15.27:0/4273196720 >> v2:172.21.15.192:3300/0 conn(0x7fe5a40d2100 0x7fe5a40cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3768310899 0 0) 0x7fe5a40d0e10 con 0x7fe5a415cf80 2023-12-19T18:15:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe594003520 con 0x7fe5a415cf80 2023-12-19T18:15:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 520154231 0 0) 0x7fe594003520 con 0x7fe5a415cf80 2023-12-19T18:15:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 >> v2:172.21.15.192:3300/0 conn(0x7fe5a40d2100 msgr2=0x7fe5a40cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:49.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 --2- 172.21.15.27:0/4273196720 >> v2:172.21.15.192:3300/0 conn(0x7fe5a40d2100 0x7fe5a40cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 >> v2:172.21.15.27:3300/0 conn(0x7fe5a40cdf80 msgr2=0x7fe5a40b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 --2- 172.21.15.27:0/4273196720 >> v2:172.21.15.27:3300/0 conn(0x7fe5a40cdf80 0x7fe5a40b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a40d1c70 con 0x7fe5a415cf80 2023-12-19T18:15:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe5980019f0 con 0x7fe5a415cf80 2023-12-19T18:15:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 >> v1:172.21.15.27:6789/0 conn(0x7fe5a415cf80 legacy=0x7fe5a40b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 --2- 172.21.15.27:0/4273196720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5940039c0 0x7fe594003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 --2- 172.21.15.27:0/4273196720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5940047c0 0x7fe594004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 --2- 172.21.15.27:0/4273196720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe594005180 0x7fe59400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a40b8cf0 con 0x7fe594005180 2023-12-19T18:15:49.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a40ce3f0 con 0x7fe5940047c0 2023-12-19T18:15:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.391+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe59400bf40 con 0x7fe5940039c0 2023-12-19T18:15:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.392+0000 7fe5aa86b640 1 --2- 172.21.15.27:0/4273196720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe594005180 0x7fe59400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.392+0000 7fe5a9869640 1 --2- 172.21.15.27:0/4273196720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5940047c0 0x7fe594004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.392+0000 7fe5aa86b640 1 -- 172.21.15.27:0/4273196720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5940039c0 msgr2=0x7fe594003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.392+0000 7fe5aa06a640 1 --2- 172.21.15.27:0/4273196720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5940039c0 0x7fe594003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.392+0000 7fe5aa86b640 1 --2- 172.21.15.27:0/4273196720 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5940039c0 0x7fe594003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.392+0000 7fe5aa86b640 1 -- 172.21.15.27:0/4273196720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5940047c0 msgr2=0x7fe594004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.392+0000 7fe5aa86b640 1 --2- 172.21.15.27:0/4273196720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5940047c0 0x7fe594004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.392+0000 7fe5aa86b640 1 -- 172.21.15.27:0/4273196720 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe594003520 con 0x7fe594005180 2023-12-19T18:15:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.392+0000 7fe5a9869640 1 --2- 172.21.15.27:0/4273196720 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5940047c0 0x7fe594004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:49.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.393+0000 7fe5aa86b640 1 -- 172.21.15.27:0/4273196720 reap_dead start 2023-12-19T18:15:49.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.393+0000 7fe5aa86b640 1 --2- 172.21.15.27:0/4273196720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe594005180 0x7fe59400b980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009ac0 tx=0x7fe5940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0e7232369e3e19 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:49.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.394+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c004020 con 0x7fe594005180 2023-12-19T18:15:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.394+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59c004180 con 0x7fe594005180 2023-12-19T18:15:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.394+0000 7fe5a9068640 1 -- 172.21.15.27:0/4273196720 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59c02c710 con 0x7fe594005180 2023-12-19T18:15:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.394+0000 7fe5ab06c640 1 -- 172.21.15.27:0/4273196720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe594005180 msgr2=0x7fe59400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.394+0000 7fe5ab06c640 1 --2- 172.21.15.27:0/4273196720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe594005180 0x7fe59400b980 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe59c009ac0 tx=0x7fe5940057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.394+0000 7fe5ab06c640 1 -- 172.21.15.27:0/4273196720 shutdown_connections 2023-12-19T18:15:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.394+0000 7fe5ab06c640 1 --2- 172.21.15.27:0/4273196720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe594005180 0x7fe59400b980 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.394+0000 7fe5ab06c640 1 -- 172.21.15.27:0/4273196720 >> 172.21.15.27:0/4273196720 conn(0x7fe5a4072e20 msgr2=0x7fe5a4073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.394+0000 7fe5ab06c640 1 -- 172.21.15.27:0/4273196720 shutdown_connections 2023-12-19T18:15:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.395+0000 7fe5ab06c640 1 -- 172.21.15.27:0/4273196720 wait complete. 2023-12-19T18:15:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.396+0000 7fe5ab06c640 1 Processor -- start 2023-12-19T18:15:49.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.397+0000 7fe5ab06c640 1 -- start start 2023-12-19T18:15:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.397+0000 7fe5ab06c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe594005180 0x7fe5a415a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.397+0000 7fe5ab06c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a40c92f0 0x7fe5a40c8b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.397+0000 7fe5ab06c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5a40ca3c0 0x7fe5a40ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.397+0000 7fe5ab06c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a40cad80 con 0x7fe5a40c92f0 2023-12-19T18:15:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.397+0000 7fe5ab06c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a40b1c80 con 0x7fe5a40ca3c0 2023-12-19T18:15:49.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.397+0000 7fe5ab06c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5a415a750 con 0x7fe594005180 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.398+0000 7fe5a9869640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a40c92f0 0x7fe5a40c8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.398+0000 7fe5a9869640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a40c92f0 0x7fe5a40c8b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48054/0 (socket says 172.21.15.27:48054) 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.398+0000 7fe5a9869640 1 -- 172.21.15.27:0/3095885971 learned_addr learned my addr 172.21.15.27:0/3095885971 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.398+0000 7fe5aa06a640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe594005180 0x7fe5a415a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.398+0000 7fe5aa86b640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5a40ca3c0 0x7fe5a40ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.398+0000 7fe5a9869640 1 -- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe594005180 msgr2=0x7fe5a415a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.398+0000 7fe5a9869640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe594005180 0x7fe5a415a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.398+0000 7fe5a9869640 1 -- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5a40ca3c0 msgr2=0x7fe5a40ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.398+0000 7fe5a9869640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5a40ca3c0 0x7fe5a40ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.398+0000 7fe5a9869640 1 -- 172.21.15.27:0/3095885971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a40c90b0 con 0x7fe5a40c92f0 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.399+0000 7fe5a9869640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a40c92f0 0x7fe5a40c8b70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe5900108b0 tx=0x7fe590010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.399+0000 7fe5a2ffd640 1 -- 172.21.15.27:0/3095885971 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59000eb60 con 0x7fe5a40c92f0 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.399+0000 7fe5a2ffd640 1 -- 172.21.15.27:0/3095885971 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe59000ecc0 con 0x7fe5a40c92f0 2023-12-19T18:15:49.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.399+0000 7fe5ab06c640 1 -- 172.21.15.27:0/3095885971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5a40c7db0 con 0x7fe5a40c92f0 2023-12-19T18:15:49.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.399+0000 7fe5a2ffd640 1 -- 172.21.15.27:0/3095885971 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe59001c400 con 0x7fe5a40c92f0 2023-12-19T18:15:49.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.399+0000 7fe5ab06c640 1 -- 172.21.15.27:0/3095885971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5a40c8370 con 0x7fe5a40c92f0 2023-12-19T18:15:49.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.400+0000 7fe5ab06c640 1 -- 172.21.15.27:0/3095885971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5a40d0020 con 0x7fe5a40c92f0 2023-12-19T18:15:49.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.401+0000 7fe5a2ffd640 1 -- 172.21.15.27:0/3095885971 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe59001c560 con 0x7fe5a40c92f0 2023-12-19T18:15:49.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.402+0000 7fe5a2ffd640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe58806a460 0x7fe58806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.402+0000 7fe5a2ffd640 1 -- 172.21.15.27:0/3095885971 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe590088f60 con 0x7fe5a40c92f0 2023-12-19T18:15:49.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.403+0000 7fe5aa06a640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe58806a460 0x7fe58806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.404+0000 7fe5aa06a640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe58806a460 0x7fe58806c920 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe598001350 tx=0x7fe598009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:49.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.406+0000 7fe5a2ffd640 1 -- 172.21.15.27:0/3095885971 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe5900545d0 con 0x7fe5a40c92f0 2023-12-19T18:15:49.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.608+0000 7fe5ab06c640 1 -- 172.21.15.27:0/3095885971 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe5a404e840 con 0x7fe5a40c92f0 2023-12-19T18:15:49.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.610+0000 7fe5a2ffd640 1 -- 172.21.15.27:0/3095885971 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe590057bf0 con 0x7fe5a40c92f0 2023-12-19T18:15:49.610 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:15:49.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.612+0000 7fe5ab06c640 1 -- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe58806a460 msgr2=0x7fe58806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:49.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.612+0000 7fe5ab06c640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe58806a460 0x7fe58806c920 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fe598001350 tx=0x7fe598009040 comp rx=0 tx=0).stop 2023-12-19T18:15:49.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.612+0000 7fe5ab06c640 1 -- 172.21.15.27:0/3095885971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a40c92f0 msgr2=0x7fe5a40c8b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:49.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.612+0000 7fe5ab06c640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a40c92f0 0x7fe5a40c8b70 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe5900108b0 tx=0x7fe590010d70 comp rx=0 tx=0).stop 2023-12-19T18:15:49.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.612+0000 7fe5ab06c640 1 -- 172.21.15.27:0/3095885971 shutdown_connections 2023-12-19T18:15:49.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.612+0000 7fe5ab06c640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe58806a460 0x7fe58806c920 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.612+0000 7fe5ab06c640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5a40ca3c0 0x7fe5a40ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.612+0000 7fe5ab06c640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5a40c92f0 0x7fe5a40c8b70 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.612+0000 7fe5ab06c640 1 --2- 172.21.15.27:0/3095885971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe594005180 0x7fe5a415a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.613+0000 7fe5ab06c640 1 -- 172.21.15.27:0/3095885971 >> 172.21.15.27:0/3095885971 conn(0x7fe5a4072e20 msgr2=0x7fe5a415c020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:49.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.613+0000 7fe5ab06c640 1 -- 172.21.15.27:0/3095885971 shutdown_connections 2023-12-19T18:15:49.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.613+0000 7fe5ab06c640 1 -- 172.21.15.27:0/3095885971 wait complete. 2023-12-19T18:15:49.634 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:15:49.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.871+0000 7fd897fff640 1 Processor -- start 2023-12-19T18:15:49.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.872+0000 7fd897fff640 1 -- start start 2023-12-19T18:15:49.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.873+0000 7fd897fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8980b49d0 0x7fd898160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.873+0000 7fd897fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd8980b89b0 0x7fd89815c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.873+0000 7fd897fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd8980b4db0 0x7fd89815cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.873+0000 7fd897fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89815d300 con 0x7fd8980b4db0 2023-12-19T18:15:49.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.873+0000 7fd897fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8980b5ca0 con 0x7fd8980b89b0 2023-12-19T18:15:49.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.873+0000 7fd897fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd898161800 con 0x7fd8980b49d0 2023-12-19T18:15:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.874+0000 7fd8977fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd8980b4db0 0x7fd89815cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.874+0000 7fd8977fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd8980b4db0 0x7fd89815cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48058/0 (socket says 172.21.15.27:48058) 2023-12-19T18:15:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.874+0000 7fd8977fe640 1 -- 172.21.15.27:0/3153912353 learned_addr learned my addr 172.21.15.27:0/3153912353 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.874+0000 7fd88e7fc640 1 --2- 172.21.15.27:0/3153912353 >> v2:172.21.15.192:3300/0 conn(0x7fd8980b89b0 0x7fd89815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.874+0000 7fd896ffd640 1 --2- 172.21.15.27:0/3153912353 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8980b49d0 0x7fd898160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.875+0000 7fd8977fe640 1 -- 172.21.15.27:0/3153912353 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8980b49d0 msgr2=0x7fd898160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.875+0000 7fd8977fe640 1 --2- 172.21.15.27:0/3153912353 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8980b49d0 0x7fd898160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.875+0000 7fd8977fe640 1 -- 172.21.15.27:0/3153912353 >> v2:172.21.15.192:3300/0 conn(0x7fd8980b89b0 msgr2=0x7fd89815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:49.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.875+0000 7fd8977fe640 1 --2- 172.21.15.27:0/3153912353 >> v2:172.21.15.192:3300/0 conn(0x7fd8980b89b0 0x7fd89815c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.875+0000 7fd8977fe640 1 -- 172.21.15.27:0/3153912353 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd898161240 con 0x7fd8980b4db0 2023-12-19T18:15:49.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.877+0000 7fd8977fe640 1 --2- 172.21.15.27:0/3153912353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8980b4db0 0x7fd89815cdc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd89806cc10 tx=0x7fd89002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=109c7163db3f4599 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:49.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.877+0000 7fd8967fc640 1 -- 172.21.15.27:0/3153912353 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89002cb00 con 0x7fd8980b4db0 2023-12-19T18:15:49.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.877+0000 7fd8967fc640 1 -- 172.21.15.27:0/3153912353 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89002cc60 con 0x7fd8980b4db0 2023-12-19T18:15:49.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.877+0000 7fd8967fc640 1 -- 172.21.15.27:0/3153912353 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8900343f0 con 0x7fd8980b4db0 2023-12-19T18:15:49.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.877+0000 7fd897fff640 1 -- 172.21.15.27:0/3153912353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8980b4db0 msgr2=0x7fd89815cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:49.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.877+0000 7fd897fff640 1 --2- 172.21.15.27:0/3153912353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8980b4db0 0x7fd89815cdc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd89806cc10 tx=0x7fd89002bf60 comp rx=0 tx=0).stop 2023-12-19T18:15:49.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.878+0000 7fd897fff640 1 -- 172.21.15.27:0/3153912353 shutdown_connections 2023-12-19T18:15:49.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.878+0000 7fd897fff640 1 --2- 172.21.15.27:0/3153912353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8980b4db0 0x7fd89815cdc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.878+0000 7fd897fff640 1 --2- 172.21.15.27:0/3153912353 >> v2:172.21.15.192:3300/0 conn(0x7fd8980b89b0 0x7fd89815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.878+0000 7fd897fff640 1 --2- 172.21.15.27:0/3153912353 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8980b49d0 0x7fd898160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.878+0000 7fd897fff640 1 -- 172.21.15.27:0/3153912353 >> 172.21.15.27:0/3153912353 conn(0x7fd8980cf760 msgr2=0x7fd8980cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:49.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.878+0000 7fd897fff640 1 -- 172.21.15.27:0/3153912353 shutdown_connections 2023-12-19T18:15:49.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.878+0000 7fd897fff640 1 -- 172.21.15.27:0/3153912353 wait complete. 2023-12-19T18:15:49.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.879+0000 7fd897fff640 1 Processor -- start 2023-12-19T18:15:49.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.880+0000 7fd897fff640 1 -- start start 2023-12-19T18:15:49.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd897fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8980b49d0 0x7fd89815d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd897fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8980b4db0 0x7fd898154e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd897fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8980b89b0 0x7fd8981f1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd897fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89815dc90 con 0x7fd8980b4db0 2023-12-19T18:15:49.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd897fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89815e920 con 0x7fd8980b49d0 2023-12-19T18:15:49.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd897fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89815edb0 con 0x7fd8980b89b0 2023-12-19T18:15:49.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd88e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8980b4db0 0x7fd898154e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd88e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8980b4db0 0x7fd898154e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48074/0 (socket says 172.21.15.27:48074) 2023-12-19T18:15:49.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd88e7fc640 1 -- 172.21.15.27:0/803638033 learned_addr learned my addr 172.21.15.27:0/803638033 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd896ffd640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8980b49d0 0x7fd89815d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd88e7fc640 1 -- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8980b89b0 msgr2=0x7fd8981f1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd8977fe640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8980b89b0 0x7fd8981f1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd88e7fc640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8980b89b0 0x7fd8981f1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd88e7fc640 1 -- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8980b49d0 msgr2=0x7fd89815d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd88e7fc640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8980b49d0 0x7fd89815d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd88e7fc640 1 -- 172.21.15.27:0/803638033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd890009b30 con 0x7fd8980b4db0 2023-12-19T18:15:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.881+0000 7fd896ffd640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8980b49d0 0x7fd89815d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.882+0000 7fd88e7fc640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8980b4db0 0x7fd898154e40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd88000e840 tx=0x7fd88000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.882+0000 7fd88ffff640 1 -- 172.21.15.27:0/803638033 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8800094f0 con 0x7fd8980b4db0 2023-12-19T18:15:49.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.882+0000 7fd88ffff640 1 -- 172.21.15.27:0/803638033 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88000c940 con 0x7fd8980b4db0 2023-12-19T18:15:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.882+0000 7fd88ffff640 1 -- 172.21.15.27:0/803638033 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88000ccd0 con 0x7fd8980b4db0 2023-12-19T18:15:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.882+0000 7fd897fff640 1 -- 172.21.15.27:0/803638033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89815dff0 con 0x7fd8980b4db0 2023-12-19T18:15:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.882+0000 7fd897fff640 1 -- 172.21.15.27:0/803638033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89815e5b0 con 0x7fd8980b4db0 2023-12-19T18:15:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.883+0000 7fd897fff640 1 -- 172.21.15.27:0/803638033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89804e840 con 0x7fd8980b4db0 2023-12-19T18:15:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.884+0000 7fd88ffff640 1 -- 172.21.15.27:0/803638033 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd880013470 con 0x7fd8980b4db0 2023-12-19T18:15:49.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.885+0000 7fd88ffff640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd87806a430 0x7fd87806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:49.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.885+0000 7fd88ffff640 1 -- 172.21.15.27:0/803638033 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd880088a10 con 0x7fd8980b4db0 2023-12-19T18:15:49.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.886+0000 7fd896ffd640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd87806a430 0x7fd87806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:49.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.887+0000 7fd896ffd640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd87806a430 0x7fd87806c8f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd888009ab0 tx=0x7fd888008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:49.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:49.890+0000 7fd88ffff640 1 -- 172.21.15.27:0/803638033 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd880053f40 con 0x7fd8980b4db0 2023-12-19T18:15:50.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.101+0000 7fd897fff640 1 -- 172.21.15.27:0/803638033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd8980d08b0 con 0x7fd8980b4db0 2023-12-19T18:15:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.101+0000 7fd88ffff640 1 -- 172.21.15.27:0/803638033 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fd880017080 con 0x7fd8980b4db0 2023-12-19T18:15:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.104+0000 7fd897fff640 1 -- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd87806a430 msgr2=0x7fd87806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.104+0000 7fd897fff640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd87806a430 0x7fd87806c8f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd888009ab0 tx=0x7fd888008040 comp rx=0 tx=0).stop 2023-12-19T18:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.104+0000 7fd897fff640 1 -- 172.21.15.27:0/803638033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8980b4db0 msgr2=0x7fd898154e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.104+0000 7fd897fff640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8980b4db0 0x7fd898154e40 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd88000e840 tx=0x7fd88000ed00 comp rx=0 tx=0).stop 2023-12-19T18:15:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.104+0000 7fd897fff640 1 -- 172.21.15.27:0/803638033 shutdown_connections 2023-12-19T18:15:50.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.104+0000 7fd897fff640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd87806a430 0x7fd87806c8f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.104+0000 7fd897fff640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8980b89b0 0x7fd8981f1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.104+0000 7fd897fff640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8980b4db0 0x7fd898154e40 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.104+0000 7fd897fff640 1 --2- 172.21.15.27:0/803638033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8980b49d0 0x7fd89815d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.104+0000 7fd897fff640 1 -- 172.21.15.27:0/803638033 >> 172.21.15.27:0/803638033 conn(0x7fd8980cf760 msgr2=0x7fd8980d13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:50.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.104+0000 7fd897fff640 1 -- 172.21.15.27:0/803638033 shutdown_connections 2023-12-19T18:15:50.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.104+0000 7fd897fff640 1 -- 172.21.15.27:0/803638033 wait complete. 2023-12-19T18:15:50.122 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:15:50.123 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:15:50.176 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:50.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.409+0000 7fecdb5cb640 1 Processor -- start 2023-12-19T18:15:50.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.409+0000 7fecdb5cb640 1 -- start start 2023-12-19T18:15:50.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.411+0000 7fecdb5cb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fecd415af70 0x7fecd40ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:50.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.411+0000 7fecdb5cb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecd40cbce0 0x7fecd40ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:50.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.411+0000 7fecdb5cb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecd40cb850 con 0x7fecd40ada50 2023-12-19T18:15:50.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.411+0000 7fecdb5cb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecd415b350 con 0x7fecd415af70 2023-12-19T18:15:50.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.411+0000 7fecdb5cb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40cac40 con 0x7fecd40cbce0 2023-12-19T18:15:50.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.412+0000 7fecd8b66640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecd40cbce0 0x7fecd40ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:50.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.412+0000 7fecd8b66640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecd40cbce0 0x7fecd40ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:33896/0 (socket says 172.21.15.27:33896) 2023-12-19T18:15:50.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.412+0000 7fecd9367640 1 --2- 172.21.15.27:0/1439741529 >> v2:172.21.15.192:3300/0 conn(0x7fecd415af70 0x7fecd40ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:50.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.412+0000 7fecd8b66640 1 -- 172.21.15.27:0/1439741529 learned_addr learned my addr 172.21.15.27:0/1439741529 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:50.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.412+0000 7fecd9b68640 1 --1- 172.21.15.27:0/1439741529 >> v1:172.21.15.192:6789/0 conn(0x7fecd40ada50 0x7fecd40c9190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:52414/0 (socket says 172.21.15.27:52414) 2023-12-19T18:15:50.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.413+0000 7fecd8b66640 1 -- 172.21.15.27:0/1439741529 >> v2:172.21.15.192:3300/0 conn(0x7fecd415af70 msgr2=0x7fecd40ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:50.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.413+0000 7fecd8b66640 1 --2- 172.21.15.27:0/1439741529 >> v2:172.21.15.192:3300/0 conn(0x7fecd415af70 0x7fecd40ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.413+0000 7fecd8b66640 1 -- 172.21.15.27:0/1439741529 >> v1:172.21.15.192:6789/0 conn(0x7fecd40ada50 legacy=0x7fecd40c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:50.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.413+0000 7fecd8b66640 1 -- 172.21.15.27:0/1439741529 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd40caf20 con 0x7fecd40cbce0 2023-12-19T18:15:50.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.414+0000 7fecd8b66640 1 --2- 172.21.15.27:0/1439741529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecd40cbce0 0x7fecd40ad510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009870 tx=0x7fecc402b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13cb73c8d50fa90f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:50.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.415+0000 7feccbfff640 1 -- 172.21.15.27:0/1439741529 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4030840 con 0x7fecd40cbce0 2023-12-19T18:15:50.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.415+0000 7feccbfff640 1 -- 172.21.15.27:0/1439741529 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecc40309a0 con 0x7fecd40cbce0 2023-12-19T18:15:50.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.415+0000 7feccbfff640 1 -- 172.21.15.27:0/1439741529 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecc4030d30 con 0x7fecd40cbce0 2023-12-19T18:15:50.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.416+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1439741529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecd40cbce0 msgr2=0x7fecd40ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:50.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.416+0000 7fecdb5cb640 1 --2- 172.21.15.27:0/1439741529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecd40cbce0 0x7fecd40ad510 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fecc4009870 tx=0x7fecc402b980 comp rx=0 tx=0).stop 2023-12-19T18:15:50.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.416+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1439741529 shutdown_connections 2023-12-19T18:15:50.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.416+0000 7fecdb5cb640 1 --2- 172.21.15.27:0/1439741529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecd40cbce0 0x7fecd40ad510 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.416+0000 7fecdb5cb640 1 --2- 172.21.15.27:0/1439741529 >> v2:172.21.15.192:3300/0 conn(0x7fecd415af70 0x7fecd40ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.416+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1439741529 >> 172.21.15.27:0/1439741529 conn(0x7fecd406a330 msgr2=0x7fecd40b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:50.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.416+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1439741529 shutdown_connections 2023-12-19T18:15:50.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.416+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1439741529 wait complete. 2023-12-19T18:15:50.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.418+0000 7fecdb5cb640 1 Processor -- start 2023-12-19T18:15:50.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.418+0000 7fecdb5cb640 1 -- start start 2023-12-19T18:15:50.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.419+0000 7fecdb5cb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd40ada50 0x7fecd415d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:50.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.419+0000 7fecdb5cb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecd40cbce0 0x7fecd414fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:50.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.419+0000 7fecdb5cb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecd415af70 0x7fecd40cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:50.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.419+0000 7fecdb5cb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40ce7b0 con 0x7fecd40ada50 2023-12-19T18:15:50.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.419+0000 7fecdb5cb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecd40ce4d0 con 0x7fecd415af70 2023-12-19T18:15:50.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.419+0000 7fecdb5cb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecd415c680 con 0x7fecd40cbce0 2023-12-19T18:15:50.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.419+0000 7fecd9367640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd40ada50 0x7fecd415d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:50.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.419+0000 7fecd9367640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd40ada50 0x7fecd415d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48084/0 (socket says 172.21.15.27:48084) 2023-12-19T18:15:50.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.419+0000 7fecd9367640 1 -- 172.21.15.27:0/1354630128 learned_addr learned my addr 172.21.15.27:0/1354630128 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:50.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.420+0000 7fecd8b66640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecd40cbce0 0x7fecd414fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:50.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.420+0000 7fecd9b68640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecd415af70 0x7fecd40cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:50.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.420+0000 7fecd9367640 1 -- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecd40cbce0 msgr2=0x7fecd414fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:50.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.420+0000 7fecd9367640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecd40cbce0 0x7fecd414fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.420+0000 7fecd9367640 1 -- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecd415af70 msgr2=0x7fecd40cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:50.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.420+0000 7fecd9367640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecd415af70 0x7fecd40cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.420+0000 7fecd9367640 1 -- 172.21.15.27:0/1354630128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc40094d0 con 0x7fecd40ada50 2023-12-19T18:15:50.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.420+0000 7fecd9367640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd40ada50 0x7fecd415d160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fecd000e870 tx=0x7fecd000ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:50.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.420+0000 7fecc9ffb640 1 -- 172.21.15.27:0/1354630128 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd00094f0 con 0x7fecd40ada50 2023-12-19T18:15:50.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.420+0000 7fecc9ffb640 1 -- 172.21.15.27:0/1354630128 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd000c940 con 0x7fecd40ada50 2023-12-19T18:15:50.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.420+0000 7fecc9ffb640 1 -- 172.21.15.27:0/1354630128 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd000ccf0 con 0x7fecd40ada50 2023-12-19T18:15:50.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.420+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1354630128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecd415c960 con 0x7fecd40ada50 2023-12-19T18:15:50.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.421+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1354630128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecd4150530 con 0x7fecd40ada50 2023-12-19T18:15:50.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.422+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1354630128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecd415c4e0 con 0x7fecd40ada50 2023-12-19T18:15:50.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.423+0000 7fecc9ffb640 1 -- 172.21.15.27:0/1354630128 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fecd0013470 con 0x7fecd40ada50 2023-12-19T18:15:50.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.424+0000 7fecc9ffb640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fecb806a430 0x7fecb806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:50.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.424+0000 7fecc9ffb640 1 -- 172.21.15.27:0/1354630128 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fecd0088ae0 con 0x7fecd40ada50 2023-12-19T18:15:50.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.425+0000 7fecd8b66640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fecb806a430 0x7fecb806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:50.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.426+0000 7fecd8b66640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fecb806a430 0x7fecb806c8f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fecc4036000 tx=0x7fecc4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:50.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.429+0000 7fecc9ffb640 1 -- 172.21.15.27:0/1354630128 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fecd415c4e0 con 0x7fecd40ada50 2023-12-19T18:15:50.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.624+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1354630128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fecd415c4e0 con 0x7fecd40ada50 2023-12-19T18:15:50.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.627+0000 7fecc9ffb640 1 -- 172.21.15.27:0/1354630128 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16090 (secure 0 0 0) 0x7fecd415c4e0 con 0x7fecd40ada50 2023-12-19T18:15:50.627 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:50.628 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:15:43.580746+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:50.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.630+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fecb806a430 msgr2=0x7fecb806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:50.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.630+0000 7fecdb5cb640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fecb806a430 0x7fecb806c8f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fecc4036000 tx=0x7fecc4008030 comp rx=0 tx=0).stop 2023-12-19T18:15:50.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.630+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1354630128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd40ada50 msgr2=0x7fecd415d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:50.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.630+0000 7fecdb5cb640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd40ada50 0x7fecd415d160 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fecd000e870 tx=0x7fecd000ed30 comp rx=0 tx=0).stop 2023-12-19T18:15:50.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.630+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1354630128 shutdown_connections 2023-12-19T18:15:50.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.630+0000 7fecdb5cb640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fecb806a430 0x7fecb806c8f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.630+0000 7fecdb5cb640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecd415af70 0x7fecd40cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.630+0000 7fecdb5cb640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecd40cbce0 0x7fecd414fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.630+0000 7fecdb5cb640 1 --2- 172.21.15.27:0/1354630128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecd40ada50 0x7fecd415d160 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.630+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1354630128 >> 172.21.15.27:0/1354630128 conn(0x7fecd406a330 msgr2=0x7fecd4070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:50.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.630+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1354630128 shutdown_connections 2023-12-19T18:15:50.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.630+0000 7fecdb5cb640 1 -- 172.21.15.27:0/1354630128 wait complete. 2023-12-19T18:15:50.649 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:15:50.649 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:15:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.869+0000 7fd33df3a640 1 Processor -- start 2023-12-19T18:15:50.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.869+0000 7fd33df3a640 1 -- start start 2023-12-19T18:15:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.871+0000 7fd33df3a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd338151c90 0x7fd33806b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.871+0000 7fd33df3a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd3380cf550 0x7fd33815b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.871+0000 7fd33df3a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd33815b3e0 con 0x7fd33815e9d0 2023-12-19T18:15:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.871+0000 7fd33df3a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd33815eed0 con 0x7fd3380cf550 2023-12-19T18:15:50.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.871+0000 7fd33df3a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd33815f1b0 con 0x7fd338151c90 2023-12-19T18:15:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.872+0000 7fd33cf38640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd338151c90 0x7fd33806b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.872+0000 7fd33cf38640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd338151c90 0x7fd33806b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:33912/0 (socket says 172.21.15.27:33912) 2023-12-19T18:15:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.872+0000 7fd33d739640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd3380cf550 0x7fd33815b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.872+0000 7fd33cf38640 1 -- 172.21.15.27:0/1429223198 learned_addr learned my addr 172.21.15.27:0/1429223198 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.872+0000 7fd33cf38640 1 -- 172.21.15.27:0/1429223198 >> v2:172.21.15.192:3300/0 conn(0x7fd3380cf550 msgr2=0x7fd33815b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:50.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.872+0000 7fd33cf38640 1 --2- 172.21.15.27:0/1429223198 >> v2:172.21.15.192:3300/0 conn(0x7fd3380cf550 0x7fd33815b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.872+0000 7fd33cf38640 1 -- 172.21.15.27:0/1429223198 >> v1:172.21.15.192:6789/0 conn(0x7fd33815e9d0 legacy=0x7fd3380cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:50.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.872+0000 7fd33cf38640 1 -- 172.21.15.27:0/1429223198 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33815f490 con 0x7fd338151c90 2023-12-19T18:15:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.874+0000 7fd33cf38640 1 --2- 172.21.15.27:0/1429223198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd338151c90 0x7fd33806b4d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd334009870 tx=0x7fd33402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ff83756c212ed30 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.874+0000 7fd32f7fe640 1 -- 172.21.15.27:0/1429223198 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334031640 con 0x7fd338151c90 2023-12-19T18:15:50.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.874+0000 7fd32f7fe640 1 -- 172.21.15.27:0/1429223198 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3340317a0 con 0x7fd338151c90 2023-12-19T18:15:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.875+0000 7fd32f7fe640 1 -- 172.21.15.27:0/1429223198 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd334031b30 con 0x7fd338151c90 2023-12-19T18:15:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.875+0000 7fd33df3a640 1 -- 172.21.15.27:0/1429223198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd338151c90 msgr2=0x7fd33806b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.875+0000 7fd33df3a640 1 --2- 172.21.15.27:0/1429223198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd338151c90 0x7fd33806b4d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd334009870 tx=0x7fd33402b8f0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.875+0000 7fd33df3a640 1 -- 172.21.15.27:0/1429223198 shutdown_connections 2023-12-19T18:15:50.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.875+0000 7fd33df3a640 1 --2- 172.21.15.27:0/1429223198 >> v2:172.21.15.192:3300/0 conn(0x7fd3380cf550 0x7fd33815b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.875+0000 7fd33df3a640 1 --2- 172.21.15.27:0/1429223198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd338151c90 0x7fd33806b4d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.875+0000 7fd33df3a640 1 -- 172.21.15.27:0/1429223198 >> 172.21.15.27:0/1429223198 conn(0x7fd33806a060 msgr2=0x7fd33806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:50.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.875+0000 7fd33df3a640 1 -- 172.21.15.27:0/1429223198 shutdown_connections 2023-12-19T18:15:50.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.875+0000 7fd33df3a640 1 -- 172.21.15.27:0/1429223198 wait complete. 2023-12-19T18:15:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.877+0000 7fd33df3a640 1 Processor -- start 2023-12-19T18:15:50.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.877+0000 7fd33df3a640 1 -- start start 2023-12-19T18:15:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.878+0000 7fd33df3a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3380cf550 0x7fd33814d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.878+0000 7fd33df3a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd338151c90 0x7fd3380b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:50.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.878+0000 7fd33df3a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd33815e9d0 0x7fd3380b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.878+0000 7fd33df3a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd338157ef0 con 0x7fd3380cf550 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.878+0000 7fd33df3a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33814df30 con 0x7fd33815e9d0 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.878+0000 7fd33df3a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd338154280 con 0x7fd338151c90 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.879+0000 7fd33cf38640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3380cf550 0x7fd33814d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.879+0000 7fd33cf38640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3380cf550 0x7fd33814d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48086/0 (socket says 172.21.15.27:48086) 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.879+0000 7fd33cf38640 1 -- 172.21.15.27:0/1458841352 learned_addr learned my addr 172.21.15.27:0/1458841352 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.879+0000 7fd32ffff640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd338151c90 0x7fd3380b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.879+0000 7fd33cf38640 1 -- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd338151c90 msgr2=0x7fd3380b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.879+0000 7fd33cf38640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd338151c90 0x7fd3380b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.879+0000 7fd33cf38640 1 -- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd33815e9d0 msgr2=0x7fd3380b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.879+0000 7fd33cf38640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd33815e9d0 0x7fd3380b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.879+0000 7fd33cf38640 1 -- 172.21.15.27:0/1458841352 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3340094d0 con 0x7fd3380cf550 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.879+0000 7fd32ffff640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd338151c90 0x7fd3380b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:50.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.880+0000 7fd33cf38640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3380cf550 0x7fd33814d9f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd334000c00 tx=0x7fd3340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.880+0000 7fd32d7fa640 1 -- 172.21.15.27:0/1458841352 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33403a930 con 0x7fd3380cf550 2023-12-19T18:15:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.880+0000 7fd32d7fa640 1 -- 172.21.15.27:0/1458841352 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33403aa90 con 0x7fd3380cf550 2023-12-19T18:15:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.880+0000 7fd33df3a640 1 -- 172.21.15.27:0/1458841352 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd338154560 con 0x7fd3380cf550 2023-12-19T18:15:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.880+0000 7fd32d7fa640 1 -- 172.21.15.27:0/1458841352 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33403ae40 con 0x7fd3380cf550 2023-12-19T18:15:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.880+0000 7fd33df3a640 1 -- 172.21.15.27:0/1458841352 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd338154b20 con 0x7fd3380cf550 2023-12-19T18:15:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.881+0000 7fd33df3a640 1 -- 172.21.15.27:0/1458841352 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd338158060 con 0x7fd3380cf550 2023-12-19T18:15:50.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.882+0000 7fd32d7fa640 1 -- 172.21.15.27:0/1458841352 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd33402f050 con 0x7fd3380cf550 2023-12-19T18:15:50.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.883+0000 7fd32d7fa640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd32406a430 0x7fd32406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:50.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.883+0000 7fd32d7fa640 1 -- 172.21.15.27:0/1458841352 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fd3340aa650 con 0x7fd3380cf550 2023-12-19T18:15:50.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.884+0000 7fd32ffff640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd32406a430 0x7fd32406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:50.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.885+0000 7fd32ffff640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd32406a430 0x7fd32406c8f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd3380ad9e0 tx=0x7fd328009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:50.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:50.888+0000 7fd32d7fa640 1 -- 172.21.15.27:0/1458841352 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd334075cb0 con 0x7fd3380cf550 2023-12-19T18:15:51.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.083+0000 7fd33df3a640 1 -- 172.21.15.27:0/1458841352 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd33804e840 con 0x7fd3380cf550 2023-12-19T18:15:51.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.085+0000 7fd32d7fa640 1 -- 172.21.15.27:0/1458841352 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd3340792d0 con 0x7fd3380cf550 2023-12-19T18:15:51.085 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:15:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.087+0000 7fd33df3a640 1 -- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd32406a430 msgr2=0x7fd32406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.087+0000 7fd33df3a640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd32406a430 0x7fd32406c8f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd3380ad9e0 tx=0x7fd328009040 comp rx=0 tx=0).stop 2023-12-19T18:15:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.087+0000 7fd33df3a640 1 -- 172.21.15.27:0/1458841352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3380cf550 msgr2=0x7fd33814d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.087+0000 7fd33df3a640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3380cf550 0x7fd33814d9f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fd334000c00 tx=0x7fd3340046c0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.087+0000 7fd33df3a640 1 -- 172.21.15.27:0/1458841352 shutdown_connections 2023-12-19T18:15:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.087+0000 7fd33df3a640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd32406a430 0x7fd32406c8f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.087+0000 7fd33df3a640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd33815e9d0 0x7fd3380b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.087+0000 7fd33df3a640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd338151c90 0x7fd3380b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.087+0000 7fd33df3a640 1 --2- 172.21.15.27:0/1458841352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3380cf550 0x7fd33814d9f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.087+0000 7fd33df3a640 1 -- 172.21.15.27:0/1458841352 >> 172.21.15.27:0/1458841352 conn(0x7fd33806a060 msgr2=0x7fd33815d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.088+0000 7fd33df3a640 1 -- 172.21.15.27:0/1458841352 shutdown_connections 2023-12-19T18:15:51.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.088+0000 7fd33df3a640 1 -- 172.21.15.27:0/1458841352 wait complete. 2023-12-19T18:15:51.108 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:15:51.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.344+0000 7f1557fff640 1 Processor -- start 2023-12-19T18:15:51.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.345+0000 7f1557fff640 1 -- start start 2023-12-19T18:15:51.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.346+0000 7f1557fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f155815cf80 0x7f15580d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:51.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.346+0000 7f1557fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f15580b85e0 0x7f15580cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:51.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.347+0000 7f1557fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15580d0e10 con 0x7f15580b8a60 2023-12-19T18:15:51.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.347+0000 7f1557fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15580d1c70 con 0x7f155815cf80 2023-12-19T18:15:51.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.347+0000 7f1557fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f155815d360 con 0x7f15580b85e0 2023-12-19T18:15:51.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.347+0000 7f1556ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f155815cf80 0x7f15580d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:51.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.347+0000 7f1556ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f155815cf80 0x7f15580d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48098/0 (socket says 172.21.15.27:48098) 2023-12-19T18:15:51.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.347+0000 7f1556ffd640 1 -- 172.21.15.27:0/681587509 learned_addr learned my addr 172.21.15.27:0/681587509 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:51.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.347+0000 7f1556ffd640 1 -- 172.21.15.27:0/681587509 >> v2:172.21.15.192:3300/0 conn(0x7f15580b85e0 msgr2=0x7f15580cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:51.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.347+0000 7f1556ffd640 1 --2- 172.21.15.27:0/681587509 >> v2:172.21.15.192:3300/0 conn(0x7f15580b85e0 0x7f15580cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.347+0000 7f1556ffd640 1 -- 172.21.15.27:0/681587509 >> v1:172.21.15.192:6789/0 conn(0x7f15580b8a60 legacy=0x7f15580cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:51.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.347+0000 7f1556ffd640 1 -- 172.21.15.27:0/681587509 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15580d1060 con 0x7f155815cf80 2023-12-19T18:15:51.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.348+0000 7f1556ffd640 1 --2- 172.21.15.27:0/681587509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f155815cf80 0x7f15580d2100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15580cd270 tx=0x7f154802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6af482d409a2033c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.349+0000 7f15567fc640 1 -- 172.21.15.27:0/681587509 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548030860 con 0x7f155815cf80 2023-12-19T18:15:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.349+0000 7f15567fc640 1 -- 172.21.15.27:0/681587509 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15480309c0 con 0x7f155815cf80 2023-12-19T18:15:51.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.349+0000 7f15567fc640 1 -- 172.21.15.27:0/681587509 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1548030d50 con 0x7f155815cf80 2023-12-19T18:15:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.349+0000 7f1557fff640 1 -- 172.21.15.27:0/681587509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f155815cf80 msgr2=0x7f15580d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.350+0000 7f1557fff640 1 --2- 172.21.15.27:0/681587509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f155815cf80 0x7f15580d2100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f15580cd270 tx=0x7f154802b9a0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.350+0000 7f1557fff640 1 -- 172.21.15.27:0/681587509 shutdown_connections 2023-12-19T18:15:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.350+0000 7f1557fff640 1 --2- 172.21.15.27:0/681587509 >> v2:172.21.15.192:3300/0 conn(0x7f15580b85e0 0x7f15580cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.350+0000 7f1557fff640 1 --2- 172.21.15.27:0/681587509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f155815cf80 0x7f15580d2100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.350+0000 7f1557fff640 1 -- 172.21.15.27:0/681587509 >> 172.21.15.27:0/681587509 conn(0x7f1558072e20 msgr2=0x7f1558073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.350+0000 7f1557fff640 1 -- 172.21.15.27:0/681587509 shutdown_connections 2023-12-19T18:15:51.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.350+0000 7f1557fff640 1 -- 172.21.15.27:0/681587509 wait complete. 2023-12-19T18:15:51.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.352+0000 7f1557fff640 1 Processor -- start 2023-12-19T18:15:51.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.352+0000 7f1557fff640 1 -- start start 2023-12-19T18:15:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.354+0000 7f1557fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15580b85e0 0x7f15580cc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.354+0000 7f1557fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f155815cf80 0x7f15580a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.354+0000 7f1557fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15580cc620 0x7f15580a9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.354+0000 7f1557fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155814cb60 con 0x7f155815cf80 2023-12-19T18:15:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.354+0000 7f1557fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f155814bee0 con 0x7f15580cc620 2023-12-19T18:15:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.354+0000 7f1557fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f155814c1c0 con 0x7f15580b85e0 2023-12-19T18:15:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.354+0000 7f154ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f155815cf80 0x7f15580a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.354+0000 7f154ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f155815cf80 0x7f15580a96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48110/0 (socket says 172.21.15.27:48110) 2023-12-19T18:15:51.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.354+0000 7f154ffff640 1 -- 172.21.15.27:0/1114943278 learned_addr learned my addr 172.21.15.27:0/1114943278 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:51.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.354+0000 7f1556ffd640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15580b85e0 0x7f15580cc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:51.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.354+0000 7f15577fe640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15580cc620 0x7f15580a9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:51.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.354+0000 7f154ffff640 1 -- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15580b85e0 msgr2=0x7f15580cc0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:51.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.355+0000 7f154ffff640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15580b85e0 0x7f15580cc0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.355+0000 7f154ffff640 1 -- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15580cc620 msgr2=0x7f15580a9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:51.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.355+0000 7f154ffff640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15580cc620 0x7f15580a9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.355+0000 7f154ffff640 1 -- 172.21.15.27:0/1114943278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15480094d0 con 0x7f155815cf80 2023-12-19T18:15:51.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.355+0000 7f15577fe640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15580cc620 0x7f15580a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:51.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.355+0000 7f154ffff640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f155815cf80 0x7f15580a96a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f154000b4a0 tx=0x7f154000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:51.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.355+0000 7f1554ff9640 1 -- 172.21.15.27:0/1114943278 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540010610 con 0x7f155815cf80 2023-12-19T18:15:51.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.355+0000 7f1554ff9640 1 -- 172.21.15.27:0/1114943278 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1540010770 con 0x7f155815cf80 2023-12-19T18:15:51.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.355+0000 7f1554ff9640 1 -- 172.21.15.27:0/1114943278 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540010b20 con 0x7f155815cf80 2023-12-19T18:15:51.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.355+0000 7f1557fff640 1 -- 172.21.15.27:0/1114943278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f155814c4a0 con 0x7f155815cf80 2023-12-19T18:15:51.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.356+0000 7f1557fff640 1 -- 172.21.15.27:0/1114943278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f155815df40 con 0x7f155815cf80 2023-12-19T18:15:51.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.357+0000 7f1557fff640 1 -- 172.21.15.27:0/1114943278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f155804e840 con 0x7f155815cf80 2023-12-19T18:15:51.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.358+0000 7f1554ff9640 1 -- 172.21.15.27:0/1114943278 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1540010c80 con 0x7f155815cf80 2023-12-19T18:15:51.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.359+0000 7f1554ff9640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f153806a430 0x7f153806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:51.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.359+0000 7f1554ff9640 1 -- 172.21.15.27:0/1114943278 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1540088540 con 0x7f155815cf80 2023-12-19T18:15:51.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.359+0000 7f1556ffd640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f153806a430 0x7f153806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:51.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.361+0000 7f1556ffd640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f153806a430 0x7f153806c8f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1548036000 tx=0x7f154803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:51.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.363+0000 7f1554ff9640 1 -- 172.21.15.27:0/1114943278 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1540053b00 con 0x7f155815cf80 2023-12-19T18:15:51.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.556+0000 7f1557fff640 1 -- 172.21.15.27:0/1114943278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f155804e050 con 0x7f155815cf80 2023-12-19T18:15:51.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.557+0000 7f1554ff9640 1 -- 172.21.15.27:0/1114943278 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1540057120 con 0x7f155815cf80 2023-12-19T18:15:51.557 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:15:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.559+0000 7f1557fff640 1 -- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f153806a430 msgr2=0x7f153806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.559+0000 7f1557fff640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f153806a430 0x7f153806c8f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1548036000 tx=0x7f154803b040 comp rx=0 tx=0).stop 2023-12-19T18:15:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.559+0000 7f1557fff640 1 -- 172.21.15.27:0/1114943278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f155815cf80 msgr2=0x7f15580a96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.559+0000 7f1557fff640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f155815cf80 0x7f15580a96a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f154000b4a0 tx=0x7f154000b960 comp rx=0 tx=0).stop 2023-12-19T18:15:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.560+0000 7f1557fff640 1 -- 172.21.15.27:0/1114943278 shutdown_connections 2023-12-19T18:15:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.560+0000 7f1557fff640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f153806a430 0x7f153806c8f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.560+0000 7f1557fff640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15580cc620 0x7f15580a9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.560+0000 7f1557fff640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f155815cf80 0x7f15580a96a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.560+0000 7f1557fff640 1 --2- 172.21.15.27:0/1114943278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f15580b85e0 0x7f15580cc0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.560+0000 7f1557fff640 1 -- 172.21.15.27:0/1114943278 >> 172.21.15.27:0/1114943278 conn(0x7f1558072e20 msgr2=0x7f15580d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.560+0000 7f1557fff640 1 -- 172.21.15.27:0/1114943278 shutdown_connections 2023-12-19T18:15:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.560+0000 7f1557fff640 1 -- 172.21.15.27:0/1114943278 wait complete. 2023-12-19T18:15:51.579 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:15:51.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.822+0000 7f033ab9b640 1 Processor -- start 2023-12-19T18:15:51.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.822+0000 7f033ab9b640 1 -- start start 2023-12-19T18:15:51.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.824+0000 7f033ab9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0334159330 0x7f033415da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:51.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.824+0000 7f033ab9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03340b2fc0 0x7f03340b3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:51.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.824+0000 7f033ab9b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03340b2b30 con 0x7f033406f320 2023-12-19T18:15:51.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.824+0000 7f033ab9b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f033415df90 con 0x7f03340b2fc0 2023-12-19T18:15:51.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.824+0000 7f033ab9b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f033406b000 con 0x7f0334159330 2023-12-19T18:15:51.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.825+0000 7f0339398640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03340b2fc0 0x7f03340b3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:51.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.825+0000 7f033a39a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f033406f320 0x7f033406b860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49364/0 (socket says 172.21.15.27:49364) 2023-12-19T18:15:51.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.825+0000 7f033a39a640 1 -- 172.21.15.27:0/2291207430 learned_addr learned my addr 172.21.15.27:0/2291207430 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:51.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.825+0000 7f0339b99640 1 --2- 172.21.15.27:0/2291207430 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0334159330 0x7f033415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:51.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.825+0000 7f0339398640 1 -- 172.21.15.27:0/2291207430 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0334159330 msgr2=0x7f033415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:51.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.825+0000 7f0339398640 1 --2- 172.21.15.27:0/2291207430 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0334159330 0x7f033415da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.825+0000 7f0339398640 1 -- 172.21.15.27:0/2291207430 >> v1:172.21.15.27:6789/0 conn(0x7f033406f320 legacy=0x7f033406b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:51.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.825+0000 7f0339398640 1 -- 172.21.15.27:0/2291207430 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f033406b2a0 con 0x7f03340b2fc0 2023-12-19T18:15:51.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.825+0000 7f0338b97640 1 -- 172.21.15.27:0/2291207430 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298220284 0 0) 0x7f03340b2b30 con 0x7f033406f320 2023-12-19T18:15:51.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.826+0000 7f0339398640 1 --2- 172.21.15.27:0/2291207430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03340b2fc0 0x7f03340b3440 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f03280097a0 tx=0x7f032802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c93806d9d7d920 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:51.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.827+0000 7f0338b97640 1 -- 172.21.15.27:0/2291207430 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032802d040 con 0x7f03340b2fc0 2023-12-19T18:15:51.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.827+0000 7f0338b97640 1 -- 172.21.15.27:0/2291207430 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0328031810 con 0x7f03340b2fc0 2023-12-19T18:15:51.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.827+0000 7f0338b97640 1 -- 172.21.15.27:0/2291207430 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0328031ba0 con 0x7f03340b2fc0 2023-12-19T18:15:51.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.827+0000 7f033ab9b640 1 -- 172.21.15.27:0/2291207430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03340b2fc0 msgr2=0x7f03340b3440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:51.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.827+0000 7f033ab9b640 1 --2- 172.21.15.27:0/2291207430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03340b2fc0 0x7f03340b3440 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f03280097a0 tx=0x7f032802b8b0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.827+0000 7f033ab9b640 1 -- 172.21.15.27:0/2291207430 shutdown_connections 2023-12-19T18:15:51.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.827+0000 7f033ab9b640 1 --2- 172.21.15.27:0/2291207430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03340b2fc0 0x7f03340b3440 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.827+0000 7f033ab9b640 1 --2- 172.21.15.27:0/2291207430 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0334159330 0x7f033415da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.827+0000 7f033ab9b640 1 -- 172.21.15.27:0/2291207430 >> 172.21.15.27:0/2291207430 conn(0x7f033406a060 msgr2=0x7f033406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:51.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.828+0000 7f033ab9b640 1 -- 172.21.15.27:0/2291207430 shutdown_connections 2023-12-19T18:15:51.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.828+0000 7f033ab9b640 1 -- 172.21.15.27:0/2291207430 wait complete. 2023-12-19T18:15:51.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.829+0000 7f033ab9b640 1 Processor -- start 2023-12-19T18:15:51.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.830+0000 7f033ab9b640 1 -- start start 2023-12-19T18:15:51.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033ab9b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f033406f320 0x7f033413b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:51.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033ab9b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03340b2fc0 0x7f03340cb5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:51.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033ab9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0334159330 0x7f03340cbae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:51.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033ab9b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0334150d80 con 0x7f0334159330 2023-12-19T18:15:51.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033ab9b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033413b900 con 0x7f03340b2fc0 2023-12-19T18:15:51.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033ab9b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0334150aa0 con 0x7f033406f320 2023-12-19T18:15:51.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033a39a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0334159330 0x7f03340cbae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:51.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033a39a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0334159330 0x7f03340cbae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48142/0 (socket says 172.21.15.27:48142) 2023-12-19T18:15:51.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033a39a640 1 -- 172.21.15.27:0/1680039882 learned_addr learned my addr 172.21.15.27:0/1680039882 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:51.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f0339b99640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f033406f320 0x7f033413b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:51.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f0339398640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03340b2fc0 0x7f03340cb5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:51.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033a39a640 1 -- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f033406f320 msgr2=0x7f033413b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:51.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033a39a640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f033406f320 0x7f033413b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033a39a640 1 -- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03340b2fc0 msgr2=0x7f03340cb5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033a39a640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03340b2fc0 0x7f03340cb5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f033a39a640 1 -- 172.21.15.27:0/1680039882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0328009400 con 0x7f0334159330 2023-12-19T18:15:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.831+0000 7f0339b99640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f033406f320 0x7f033413b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.832+0000 7f033a39a640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0334159330 0x7f03340cbae0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f032c00c780 tx=0x7f032c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.832+0000 7f03267fc640 1 -- 172.21.15.27:0/1680039882 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00ce30 con 0x7f0334159330 2023-12-19T18:15:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.832+0000 7f03267fc640 1 -- 172.21.15.27:0/1680039882 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f032c00a640 con 0x7f0334159330 2023-12-19T18:15:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.832+0000 7f03267fc640 1 -- 172.21.15.27:0/1680039882 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00a9f0 con 0x7f0334159330 2023-12-19T18:15:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.832+0000 7f033ab9b640 1 -- 172.21.15.27:0/1680039882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0334153490 con 0x7f0334159330 2023-12-19T18:15:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.832+0000 7f033ab9b640 1 -- 172.21.15.27:0/1680039882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f033414cc10 con 0x7f0334159330 2023-12-19T18:15:51.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.833+0000 7f033ab9b640 1 -- 172.21.15.27:0/1680039882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03340cc070 con 0x7f0334159330 2023-12-19T18:15:51.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.834+0000 7f03267fc640 1 -- 172.21.15.27:0/1680039882 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f032c00ab50 con 0x7f0334159330 2023-12-19T18:15:51.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.835+0000 7f03267fc640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f031806a430 0x7f031806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:51.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.835+0000 7f03267fc640 1 -- 172.21.15.27:0/1680039882 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f032c087e80 con 0x7f0334159330 2023-12-19T18:15:51.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.836+0000 7f0339b99640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f031806a430 0x7f031806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:51.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.837+0000 7f0339b99640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f031806a430 0x7f031806c8f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0330006fd0 tx=0x7f0330008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:51.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:51.840+0000 7f03267fc640 1 -- 172.21.15.27:0/1680039882 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f03340cc070 con 0x7f0334159330 2023-12-19T18:15:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.036+0000 7f033ab9b640 1 -- 172.21.15.27:0/1680039882 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03340cc070 con 0x7f0334159330 2023-12-19T18:15:52.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.037+0000 7f03267fc640 1 -- 172.21.15.27:0/1680039882 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f03340cc070 con 0x7f0334159330 2023-12-19T18:15:52.037 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:15:52.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.040+0000 7f033ab9b640 1 -- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f031806a430 msgr2=0x7f031806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:52.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.040+0000 7f033ab9b640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f031806a430 0x7f031806c8f0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0330006fd0 tx=0x7f0330008040 comp rx=0 tx=0).stop 2023-12-19T18:15:52.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.040+0000 7f033ab9b640 1 -- 172.21.15.27:0/1680039882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0334159330 msgr2=0x7f03340cbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:52.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.040+0000 7f033ab9b640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0334159330 0x7f03340cbae0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f032c00c780 tx=0x7f032c00cc40 comp rx=0 tx=0).stop 2023-12-19T18:15:52.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.040+0000 7f033ab9b640 1 -- 172.21.15.27:0/1680039882 shutdown_connections 2023-12-19T18:15:52.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.040+0000 7f033ab9b640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f031806a430 0x7f031806c8f0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.040+0000 7f033ab9b640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0334159330 0x7f03340cbae0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.040+0000 7f033ab9b640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03340b2fc0 0x7f03340cb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.040+0000 7f033ab9b640 1 --2- 172.21.15.27:0/1680039882 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f033406f320 0x7f033413b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.040+0000 7f033ab9b640 1 -- 172.21.15.27:0/1680039882 >> 172.21.15.27:0/1680039882 conn(0x7f033406a060 msgr2=0x7f033415cb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:52.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.040+0000 7f033ab9b640 1 -- 172.21.15.27:0/1680039882 shutdown_connections 2023-12-19T18:15:52.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.040+0000 7f033ab9b640 1 -- 172.21.15.27:0/1680039882 wait complete. 2023-12-19T18:15:52.061 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:15:52.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.294+0000 7f1fa984d640 1 Processor -- start 2023-12-19T18:15:52.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.294+0000 7f1fa984d640 1 -- start start 2023-12-19T18:15:52.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.296+0000 7f1fa984d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fa40b49d0 0x7f1fa4160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:52.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.296+0000 7f1fa984d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa40b5ca0 con 0x7f1fa40b89b0 2023-12-19T18:15:52.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.296+0000 7f1fa984d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fa4161800 con 0x7f1fa415cf90 2023-12-19T18:15:52.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.296+0000 7f1fa984d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa4161240 con 0x7f1fa40b49d0 2023-12-19T18:15:52.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.296+0000 7f1fa3fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1fa40b89b0 0x7f1fa415c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46232/0 (socket says 172.21.15.27:46232) 2023-12-19T18:15:52.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.296+0000 7f1fa3fff640 1 -- 172.21.15.27:0/2808574688 learned_addr learned my addr 172.21.15.27:0/2808574688 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:52.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.296+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1220867997 0 0) 0x7f1fa40b5ca0 con 0x7f1fa40b89b0 2023-12-19T18:15:52.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.296+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f8c003520 con 0x7f1fa40b89b0 2023-12-19T18:15:52.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.296+0000 7f1fa884b640 1 --2- 172.21.15.27:0/2808574688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fa40b49d0 0x7f1fa4160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:52.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 493119416 0 0) 0x7f1f8c003520 con 0x7f1fa40b89b0 2023-12-19T18:15:52.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fa40b49d0 msgr2=0x7f1fa4160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:52.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa37fe640 1 --2- 172.21.15.27:0/2808574688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fa40b49d0 0x7f1fa4160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 >> v1:172.21.15.192:6789/0 conn(0x7f1fa415cf90 legacy=0x7f1fa40d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa4161520 con 0x7f1fa40b89b0 2023-12-19T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f1f98001670 con 0x7f1fa40b89b0 2023-12-19T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 >> v1:172.21.15.27:6789/0 conn(0x7f1fa40b89b0 legacy=0x7f1fa415c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa37fe640 1 --2- 172.21.15.27:0/2808574688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fa40b49d0 0x7f1f8c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa37fe640 1 --2- 172.21.15.27:0/2808574688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8c003fd0 0x7f1f8c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa37fe640 1 --2- 172.21.15.27:0/2808574688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8c004d00 0x7f1f8c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa4161240 con 0x7f1f8c003fd0 2023-12-19T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c00bbd0 con 0x7f1f8c004d00 2023-12-19T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8c00bd90 con 0x7f1fa40b49d0 2023-12-19T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.297+0000 7f1fa3fff640 1 --2- 172.21.15.27:0/2808574688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8c003fd0 0x7f1f8c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.298+0000 7f1fa884b640 1 --2- 172.21.15.27:0/2808574688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fa40b49d0 0x7f1f8c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.298+0000 7f1fa904c640 1 --2- 172.21.15.27:0/2808574688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8c004d00 0x7f1f8c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.298+0000 7f1fa3fff640 1 -- 172.21.15.27:0/2808574688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fa40b49d0 msgr2=0x7f1f8c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:52.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.298+0000 7f1fa3fff640 1 --2- 172.21.15.27:0/2808574688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fa40b49d0 0x7f1f8c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.298+0000 7f1fa3fff640 1 -- 172.21.15.27:0/2808574688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8c004d00 msgr2=0x7f1f8c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:52.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.298+0000 7f1fa3fff640 1 --2- 172.21.15.27:0/2808574688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8c004d00 0x7f1f8c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.298+0000 7f1fa3fff640 1 -- 172.21.15.27:0/2808574688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c003520 con 0x7f1f8c003fd0 2023-12-19T18:15:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.299+0000 7f1fa3fff640 1 --2- 172.21.15.27:0/2808574688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8c003fd0 0x7f1f8c0047c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f98007fd0 tx=0x7f1f8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79125a5b638cb562 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.299+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f98005ad0 con 0x7f1f8c003fd0 2023-12-19T18:15:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.299+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f98005c30 con 0x7f1f8c003fd0 2023-12-19T18:15:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.299+0000 7f1fa37fe640 1 -- 172.21.15.27:0/2808574688 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9802cab0 con 0x7f1f8c003fd0 2023-12-19T18:15:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.300+0000 7f1fa984d640 1 -- 172.21.15.27:0/2808574688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8c003fd0 msgr2=0x7f1f8c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.300+0000 7f1fa984d640 1 --2- 172.21.15.27:0/2808574688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8c003fd0 0x7f1f8c0047c0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1f98007fd0 tx=0x7f1f8c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.300+0000 7f1fa904c640 1 -- 172.21.15.27:0/2808574688 reap_dead start 2023-12-19T18:15:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.300+0000 7f1fa984d640 1 -- 172.21.15.27:0/2808574688 shutdown_connections 2023-12-19T18:15:52.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.300+0000 7f1fa984d640 1 -- 172.21.15.27:0/2808574688 >> 172.21.15.27:0/2808574688 conn(0x7f1fa40cf760 msgr2=0x7f1fa40cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:52.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.300+0000 7f1fa984d640 1 -- 172.21.15.27:0/2808574688 shutdown_connections 2023-12-19T18:15:52.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.300+0000 7f1fa984d640 1 -- 172.21.15.27:0/2808574688 wait complete. 2023-12-19T18:15:52.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.302+0000 7f1fa984d640 1 Processor -- start 2023-12-19T18:15:52.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.302+0000 7f1fa984d640 1 -- start start 2023-12-19T18:15:52.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.303+0000 7f1fa984d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8c003fd0 0x7f1fa4150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:52.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.303+0000 7f1fa984d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fa41f2f90 0x7f1fa414e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:52.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.303+0000 7f1fa984d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa415e490 0x7f1fa415e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:52.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.303+0000 7f1fa984d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa415ee50 con 0x7f1fa415e490 2023-12-19T18:15:52.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.303+0000 7f1fa984d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa41f2b00 con 0x7f1f8c003fd0 2023-12-19T18:15:52.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.303+0000 7f1fa984d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fa41f3410 con 0x7f1fa41f2f90 2023-12-19T18:15:52.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.303+0000 7f1fa904c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa415e490 0x7f1fa415e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:52.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.303+0000 7f1fa904c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa415e490 0x7f1fa415e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48454/0 (socket says 172.21.15.27:48454) 2023-12-19T18:15:52.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.303+0000 7f1fa904c640 1 -- 172.21.15.27:0/3539172210 learned_addr learned my addr 172.21.15.27:0/3539172210 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:52.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.304+0000 7f1fa884b640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8c003fd0 0x7f1fa4150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:52.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.304+0000 7f1fa904c640 1 -- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fa41f2f90 msgr2=0x7f1fa414e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:52.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.304+0000 7f1fa904c640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fa41f2f90 0x7f1fa414e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.304+0000 7f1fa904c640 1 -- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8c003fd0 msgr2=0x7f1fa4150f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:52.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.304+0000 7f1fa904c640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8c003fd0 0x7f1fa4150f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.304+0000 7f1fa904c640 1 -- 172.21.15.27:0/3539172210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa41f24d0 con 0x7f1fa415e490 2023-12-19T18:15:52.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.304+0000 7f1fa884b640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8c003fd0 0x7f1fa4150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:52.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.304+0000 7f1fa904c640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa415e490 0x7f1fa415e910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1fa415d370 tx=0x7f1f9c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:52.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.304+0000 7f1fa17fa640 1 -- 172.21.15.27:0/3539172210 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c003970 con 0x7f1fa415e490 2023-12-19T18:15:52.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.304+0000 7f1fa17fa640 1 -- 172.21.15.27:0/3539172210 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9c003ad0 con 0x7f1fa415e490 2023-12-19T18:15:52.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.304+0000 7f1fa17fa640 1 -- 172.21.15.27:0/3539172210 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9c003e80 con 0x7f1fa415e490 2023-12-19T18:15:52.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.305+0000 7f1fa984d640 1 -- 172.21.15.27:0/3539172210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fa414e300 con 0x7f1fa415e490 2023-12-19T18:15:52.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.305+0000 7f1fa984d640 1 -- 172.21.15.27:0/3539172210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fa41f2790 con 0x7f1fa415e490 2023-12-19T18:15:52.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.306+0000 7f1fa984d640 1 -- 172.21.15.27:0/3539172210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fa41f2920 con 0x7f1fa415e490 2023-12-19T18:15:52.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.307+0000 7f1fa17fa640 1 -- 172.21.15.27:0/3539172210 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1f9c0050d0 con 0x7f1fa415e490 2023-12-19T18:15:52.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.308+0000 7f1fa17fa640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f9006a430 0x7f1f9006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:52.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.308+0000 7f1fa17fa640 1 -- 172.21.15.27:0/3539172210 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1f9c088640 con 0x7f1fa415e490 2023-12-19T18:15:52.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.308+0000 7f1fa884b640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f9006a430 0x7f1f9006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:52.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.309+0000 7f1fa884b640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f9006a430 0x7f1f9006c8f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400b8a0 tx=0x7f1f94008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:52.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.312+0000 7f1fa17fa640 1 -- 172.21.15.27:0/3539172210 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1f9c053ca0 con 0x7f1fa415e490 2023-12-19T18:15:52.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.519+0000 7f1fa984d640 1 -- 172.21.15.27:0/3539172210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1fa404e840 con 0x7f1fa415e490 2023-12-19T18:15:52.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.520+0000 7f1fa17fa640 1 -- 172.21.15.27:0/3539172210 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1f9c0572c0 con 0x7f1fa415e490 2023-12-19T18:15:52.520 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:15:52.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.522+0000 7f1fa984d640 1 -- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f9006a430 msgr2=0x7f1f9006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:52.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.522+0000 7f1fa984d640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f9006a430 0x7f1f9006c8f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1f9400b8a0 tx=0x7f1f94008040 comp rx=0 tx=0).stop 2023-12-19T18:15:52.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.522+0000 7f1fa984d640 1 -- 172.21.15.27:0/3539172210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa415e490 msgr2=0x7f1fa415e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:52.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.522+0000 7f1fa984d640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa415e490 0x7f1fa415e910 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1fa415d370 tx=0x7f1f9c00ec10 comp rx=0 tx=0).stop 2023-12-19T18:15:52.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.523+0000 7f1fa984d640 1 -- 172.21.15.27:0/3539172210 shutdown_connections 2023-12-19T18:15:52.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.523+0000 7f1fa984d640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f9006a430 0x7f1f9006c8f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.523+0000 7f1fa984d640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fa415e490 0x7f1fa415e910 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.523+0000 7f1fa984d640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fa41f2f90 0x7f1fa414e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.523+0000 7f1fa984d640 1 --2- 172.21.15.27:0/3539172210 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8c003fd0 0x7f1fa4150f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:52.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.523+0000 7f1fa984d640 1 -- 172.21.15.27:0/3539172210 >> 172.21.15.27:0/3539172210 conn(0x7f1fa40cf760 msgr2=0x7f1fa406bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:52.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.523+0000 7f1fa984d640 1 -- 172.21.15.27:0/3539172210 shutdown_connections 2023-12-19T18:15:52.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:52.523+0000 7f1fa984d640 1 -- 172.21.15.27:0/3539172210 wait complete. 2023-12-19T18:15:52.542 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-12-19T18:15:52.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:52.543 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-19T18:15:52.658 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:15:52.658 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:15:52.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:52.659 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:15:52.729 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:15:52.729 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:15:52.788 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:15:52.789 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:15:52.840 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:53.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.084+0000 7f1fdd3ab640 1 Processor -- start 2023-12-19T18:15:53.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.084+0000 7f1fdd3ab640 1 -- start start 2023-12-19T18:15:53.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.086+0000 7f1fdd3ab640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd8163b00 0x7f1fd8165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.086+0000 7f1fdd3ab640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd80cd6b0 con 0x7f1fd815ae80 2023-12-19T18:15:53.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.086+0000 7f1fdd3ab640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fd80cdc70 con 0x7f1fd80cb3e0 2023-12-19T18:15:53.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.086+0000 7f1fdd3ab640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80b3630 con 0x7f1fd8163b00 2023-12-19T18:15:53.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.086+0000 7f1fd6ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1fd815ae80 0x7f1fd80b2810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46240/0 (socket says 172.21.15.27:46240) 2023-12-19T18:15:53.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.086+0000 7f1fd6ffd640 1 -- 172.21.15.27:0/3854482715 learned_addr learned my addr 172.21.15.27:0/3854482715 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:53.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121967168 0 0) 0x7f1fd80cd6b0 con 0x7f1fd815ae80 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fc4003520 con 0x7f1fd815ae80 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd77fe640 1 --2- 172.21.15.27:0/3854482715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd8163b00 0x7f1fd8165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1338731889 0 0) 0x7f1fc4003520 con 0x7f1fd815ae80 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd8163b00 msgr2=0x7f1fd8165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 --2- 172.21.15.27:0/3854482715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd8163b00 0x7f1fd8165ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 >> v1:172.21.15.192:6789/0 conn(0x7f1fd80cb3e0 legacy=0x7f1fd80b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd80d0450 con 0x7f1fd815ae80 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f1fc8001670 con 0x7f1fd815ae80 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 >> v1:172.21.15.27:6789/0 conn(0x7f1fd815ae80 legacy=0x7f1fd80b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 --2- 172.21.15.27:0/3854482715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd8163b00 0x7f1fc40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 --2- 172.21.15.27:0/3854482715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fc4004000 0x7f1fc40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 --2- 172.21.15.27:0/3854482715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fc4004d00 0x7f1fc4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80b3630 con 0x7f1fc4004d00 2023-12-19T18:15:53.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc400bb70 con 0x7f1fc4004000 2023-12-19T18:15:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc400bd30 con 0x7f1fd8163b00 2023-12-19T18:15:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd77fe640 1 --2- 172.21.15.27:0/3854482715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fc4004d00 0x7f1fc4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd6ffd640 1 --2- 172.21.15.27:0/3854482715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd8163b00 0x7f1fc40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd67fc640 1 --2- 172.21.15.27:0/3854482715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fc4004000 0x7f1fc40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd77fe640 1 -- 172.21.15.27:0/3854482715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd8163b00 msgr2=0x7f1fc40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd77fe640 1 --2- 172.21.15.27:0/3854482715 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd8163b00 0x7f1fc40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd77fe640 1 -- 172.21.15.27:0/3854482715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fc4004000 msgr2=0x7f1fc40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd77fe640 1 --2- 172.21.15.27:0/3854482715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fc4004000 0x7f1fc40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd77fe640 1 -- 172.21.15.27:0/3854482715 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc4003520 con 0x7f1fc4004d00 2023-12-19T18:15:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.088+0000 7f1fd67fc640 1 --2- 172.21.15.27:0/3854482715 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fc4004000 0x7f1fc40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.089+0000 7f1fd77fe640 1 --2- 172.21.15.27:0/3854482715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fc4004d00 0x7f1fc4005180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000ba00 tx=0x7f1fc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6646ddedd444bdd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.090+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0032620 con 0x7f1fc4004d00 2023-12-19T18:15:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.090+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd0032780 con 0x7f1fc4004d00 2023-12-19T18:15:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.090+0000 7f1fd5ffb640 1 -- 172.21.15.27:0/3854482715 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0032b10 con 0x7f1fc4004d00 2023-12-19T18:15:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.090+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3854482715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fc4004d00 msgr2=0x7f1fc4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.090+0000 7f1fdd3ab640 1 --2- 172.21.15.27:0/3854482715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fc4004d00 0x7f1fc4005180 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000ba00 tx=0x7f1fc40057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.090+0000 7f1fd77fe640 1 -- 172.21.15.27:0/3854482715 reap_dead start 2023-12-19T18:15:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.090+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3854482715 shutdown_connections 2023-12-19T18:15:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.090+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3854482715 >> 172.21.15.27:0/3854482715 conn(0x7f1fd806f1f0 msgr2=0x7f1fd806f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.090+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3854482715 shutdown_connections 2023-12-19T18:15:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.090+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3854482715 wait complete. 2023-12-19T18:15:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.092+0000 7f1fdd3ab640 1 Processor -- start 2023-12-19T18:15:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.092+0000 7f1fdd3ab640 1 -- start start 2023-12-19T18:15:53.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.093+0000 7f1fdd3ab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fc4004d00 0x7f1fd8149600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.093+0000 7f1fdd3ab640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd814b480 0x7f1fd80b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.093+0000 7f1fdd3ab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd80b3cb0 0x7f1fd81522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.093+0000 7f1fdd3ab640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd80b4130 con 0x7f1fc4004d00 2023-12-19T18:15:53.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.093+0000 7f1fdd3ab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8145ce0 con 0x7f1fd80b3cb0 2023-12-19T18:15:53.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.093+0000 7f1fdd3ab640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fd8148cd0 con 0x7f1fd814b480 2023-12-19T18:15:53.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.094+0000 7f1fd6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fc4004d00 0x7f1fd8149600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.094+0000 7f1fd6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fc4004d00 0x7f1fd8149600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48468/0 (socket says 172.21.15.27:48468) 2023-12-19T18:15:53.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.094+0000 7f1fd6ffd640 1 -- 172.21.15.27:0/3164369077 learned_addr learned my addr 172.21.15.27:0/3164369077 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:53.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.094+0000 7f1fd6ffd640 1 -- 172.21.15.27:0/3164369077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd814b480 msgr2=0x7f1fd80b3770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.094+0000 7f1fd6ffd640 1 --2- 172.21.15.27:0/3164369077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd814b480 0x7f1fd80b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.094+0000 7f1fd6ffd640 1 -- 172.21.15.27:0/3164369077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd80b3cb0 msgr2=0x7f1fd81522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.094+0000 7f1fd6ffd640 1 --2- 172.21.15.27:0/3164369077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd80b3cb0 0x7f1fd81522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.094+0000 7f1fd6ffd640 1 -- 172.21.15.27:0/3164369077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd814bc60 con 0x7f1fc4004d00 2023-12-19T18:15:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.094+0000 7f1fd6ffd640 1 --2- 172.21.15.27:0/3164369077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fc4004d00 0x7f1fd8149600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800e860 tx=0x7f1fc800ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.095+0000 7f1fcf7fe640 1 -- 172.21.15.27:0/3164369077 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800c850 con 0x7f1fc4004d00 2023-12-19T18:15:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.095+0000 7f1fcf7fe640 1 -- 172.21.15.27:0/3164369077 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc800c9b0 con 0x7f1fc4004d00 2023-12-19T18:15:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.095+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3164369077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fd8147f10 con 0x7f1fc4004d00 2023-12-19T18:15:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.095+0000 7f1fcf7fe640 1 -- 172.21.15.27:0/3164369077 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc800cd60 con 0x7f1fc4004d00 2023-12-19T18:15:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.095+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3164369077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fd81484d0 con 0x7f1fc4004d00 2023-12-19T18:15:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.097+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3164369077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fd8067110 con 0x7f1fc4004d00 2023-12-19T18:15:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.098+0000 7f1fcf7fe640 1 -- 172.21.15.27:0/3164369077 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1fc801d3b0 con 0x7f1fc4004d00 2023-12-19T18:15:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.099+0000 7f1fcf7fe640 1 --2- 172.21.15.27:0/3164369077 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fb806a460 0x7f1fb806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.099+0000 7f1fcf7fe640 1 -- 172.21.15.27:0/3164369077 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1fc8088480 con 0x7f1fc4004d00 2023-12-19T18:15:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.099+0000 7f1fd67fc640 1 --2- 172.21.15.27:0/3164369077 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fb806a460 0x7f1fb806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.100+0000 7f1fd67fc640 1 --2- 172.21.15.27:0/3164369077 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fb806a460 0x7f1fb806c920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0007fd0 tx=0x7f1fc0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:53.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.102+0000 7f1fcf7fe640 1 -- 172.21.15.27:0/3164369077 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1fc8053ae0 con 0x7f1fc4004d00 2023-12-19T18:15:53.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.303+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3164369077 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1fd80b4cd0 con 0x7f1fc4004d00 2023-12-19T18:15:53.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.306+0000 7f1fcf7fe640 1 -- 172.21.15.27:0/3164369077 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16090 (secure 0 0 0) 0x7f1fc8019090 con 0x7f1fc4004d00 2023-12-19T18:15:53.305 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:53.305 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:15:43.580746+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:53.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.308+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3164369077 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fb806a460 msgr2=0x7f1fb806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:53.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.308+0000 7f1fdd3ab640 1 --2- 172.21.15.27:0/3164369077 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fb806a460 0x7f1fb806c920 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1fc0007fd0 tx=0x7f1fc0009040 comp rx=0 tx=0).stop 2023-12-19T18:15:53.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.308+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3164369077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fc4004d00 msgr2=0x7f1fd8149600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:53.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.308+0000 7f1fdd3ab640 1 --2- 172.21.15.27:0/3164369077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fc4004d00 0x7f1fd8149600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1fc800e860 tx=0x7f1fc800ed20 comp rx=0 tx=0).stop 2023-12-19T18:15:53.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.308+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3164369077 shutdown_connections 2023-12-19T18:15:53.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.308+0000 7f1fdd3ab640 1 --2- 172.21.15.27:0/3164369077 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fb806a460 0x7f1fb806c920 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.308+0000 7f1fdd3ab640 1 --2- 172.21.15.27:0/3164369077 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fd80b3cb0 0x7f1fd81522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.308+0000 7f1fdd3ab640 1 --2- 172.21.15.27:0/3164369077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fd814b480 0x7f1fd80b3770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.308+0000 7f1fdd3ab640 1 --2- 172.21.15.27:0/3164369077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fc4004d00 0x7f1fd8149600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.308+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3164369077 >> 172.21.15.27:0/3164369077 conn(0x7f1fd806f1f0 msgr2=0x7f1fd814acc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:53.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.308+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3164369077 shutdown_connections 2023-12-19T18:15:53.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.309+0000 7f1fdd3ab640 1 -- 172.21.15.27:0/3164369077 wait complete. 2023-12-19T18:15:53.329 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:15:53.329 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:15:53.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.572+0000 7f3c649d6640 1 Processor -- start 2023-12-19T18:15:53.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.572+0000 7f3c649d6640 1 -- start start 2023-12-19T18:15:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.574+0000 7f3c649d6640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3c6015e9d0 0x7f3c600cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.574+0000 7f3c649d6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6015b3e0 con 0x7f3c60151c90 2023-12-19T18:15:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.574+0000 7f3c649d6640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c6015eed0 con 0x7f3c60152070 2023-12-19T18:15:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.574+0000 7f3c649d6640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c6015f1b0 con 0x7f3c6015e9d0 2023-12-19T18:15:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.574+0000 7f3c5f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3c60151c90 0x7f3c6006b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46242/0 (socket says 172.21.15.27:46242) 2023-12-19T18:15:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.574+0000 7f3c5f7fe640 1 -- 172.21.15.27:0/184719234 learned_addr learned my addr 172.21.15.27:0/184719234 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5effd640 1 --2- 172.21.15.27:0/184719234 >> v2:172.21.15.192:3300/0 conn(0x7f3c6015e9d0 0x7f3c600cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 375652761 0 0) 0x7f3c6015b3e0 con 0x7f3c60151c90 2023-12-19T18:15:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c48003520 con 0x7f3c60151c90 2023-12-19T18:15:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3056741066 0 0) 0x7f3c48003520 con 0x7f3c60151c90 2023-12-19T18:15:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 >> v2:172.21.15.192:3300/0 conn(0x7f3c6015e9d0 msgr2=0x7f3c600cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 --2- 172.21.15.27:0/184719234 >> v2:172.21.15.192:3300/0 conn(0x7f3c6015e9d0 0x7f3c600cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 >> v1:172.21.15.192:6789/0 conn(0x7f3c60152070 legacy=0x7f3c6015b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6015f490 con 0x7f3c60151c90 2023-12-19T18:15:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f3c50001670 con 0x7f3c60151c90 2023-12-19T18:15:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 >> v1:172.21.15.27:6789/0 conn(0x7f3c60151c90 legacy=0x7f3c6006b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 --2- 172.21.15.27:0/184719234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c480039f0 0x7f3c48003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 --2- 172.21.15.27:0/184719234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c480047c0 0x7f3c48004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 --2- 172.21.15.27:0/184719234 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c480052a0 0x7f3c4800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6015f1b0 con 0x7f3c480047c0 2023-12-19T18:15:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4800c1f0 con 0x7f3c480039f0 2023-12-19T18:15:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.575+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c4800c3b0 con 0x7f3c480052a0 2023-12-19T18:15:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.576+0000 7f3c5f7fe640 1 --2- 172.21.15.27:0/184719234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c480039f0 0x7f3c48003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.576+0000 7f3c5ffff640 1 --2- 172.21.15.27:0/184719234 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c480052a0 0x7f3c4800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.576+0000 7f3c5effd640 1 --2- 172.21.15.27:0/184719234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c480047c0 0x7f3c48004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.577+0000 7f3c5f7fe640 1 -- 172.21.15.27:0/184719234 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c480052a0 msgr2=0x7f3c4800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:53.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.577+0000 7f3c5f7fe640 1 --2- 172.21.15.27:0/184719234 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c480052a0 0x7f3c4800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.577+0000 7f3c5f7fe640 1 -- 172.21.15.27:0/184719234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c480047c0 msgr2=0x7f3c48004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:53.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.577+0000 7f3c5f7fe640 1 --2- 172.21.15.27:0/184719234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c480047c0 0x7f3c48004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.577+0000 7f3c5ffff640 1 -- 172.21.15.27:0/184719234 reap_dead start 2023-12-19T18:15:53.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.577+0000 7f3c5f7fe640 1 -- 172.21.15.27:0/184719234 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c48003520 con 0x7f3c480039f0 2023-12-19T18:15:53.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.578+0000 7f3c5f7fe640 1 --2- 172.21.15.27:0/184719234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c480039f0 0x7f3c48003e10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3c50007f70 tx=0x7f3c480057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bdb81335090c7c1d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:53.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.579+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50005ad0 con 0x7f3c480039f0 2023-12-19T18:15:53.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.579+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c50005c50 con 0x7f3c480039f0 2023-12-19T18:15:53.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.579+0000 7f3c5e7fc640 1 -- 172.21.15.27:0/184719234 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c50007070 con 0x7f3c480039f0 2023-12-19T18:15:53.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.579+0000 7f3c649d6640 1 -- 172.21.15.27:0/184719234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c480039f0 msgr2=0x7f3c48003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:53.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.579+0000 7f3c649d6640 1 --2- 172.21.15.27:0/184719234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c480039f0 0x7f3c48003e10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f3c50007f70 tx=0x7f3c480057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.579+0000 7f3c649d6640 1 -- 172.21.15.27:0/184719234 shutdown_connections 2023-12-19T18:15:53.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.579+0000 7f3c649d6640 1 --2- 172.21.15.27:0/184719234 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c480039f0 0x7f3c48003e10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.579+0000 7f3c649d6640 1 -- 172.21.15.27:0/184719234 >> 172.21.15.27:0/184719234 conn(0x7f3c6006a060 msgr2=0x7f3c6006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:53.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.579+0000 7f3c649d6640 1 -- 172.21.15.27:0/184719234 shutdown_connections 2023-12-19T18:15:53.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.580+0000 7f3c649d6640 1 -- 172.21.15.27:0/184719234 wait complete. 2023-12-19T18:15:53.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.581+0000 7f3c649d6640 1 Processor -- start 2023-12-19T18:15:53.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.581+0000 7f3c649d6640 1 -- start start 2023-12-19T18:15:53.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.582+0000 7f3c649d6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c480039f0 0x7f3c600acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.582+0000 7f3c649d6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c60143680 0x7f3c60141400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.582+0000 7f3c649d6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c6006b000 0x7f3c6006b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.582+0000 7f3c649d6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6006b9f0 con 0x7f3c60143680 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.582+0000 7f3c649d6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c60144c10 con 0x7f3c6006b000 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.582+0000 7f3c649d6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6014fb60 con 0x7f3c480039f0 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.583+0000 7f3c5effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c60143680 0x7f3c60141400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.583+0000 7f3c5effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c60143680 0x7f3c60141400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48478/0 (socket says 172.21.15.27:48478) 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.583+0000 7f3c5effd640 1 -- 172.21.15.27:0/1645265638 learned_addr learned my addr 172.21.15.27:0/1645265638 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.583+0000 7f3c5ffff640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c6006b000 0x7f3c6006b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.583+0000 7f3c5f7fe640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c480039f0 0x7f3c600acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.583+0000 7f3c5effd640 1 -- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c480039f0 msgr2=0x7f3c600acdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.583+0000 7f3c5effd640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c480039f0 0x7f3c600acdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.583+0000 7f3c5effd640 1 -- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c6006b000 msgr2=0x7f3c6006b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.583+0000 7f3c5effd640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c6006b000 0x7f3c6006b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.583+0000 7f3c5effd640 1 -- 172.21.15.27:0/1645265638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c600ad300 con 0x7f3c60143680 2023-12-19T18:15:53.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.583+0000 7f3c5ffff640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c6006b000 0x7f3c6006b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:53.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.584+0000 7f3c5effd640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c60143680 0x7f3c60141400 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400bfd0 tx=0x7f3c5400f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:53.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.584+0000 7f3c47fff640 1 -- 172.21.15.27:0/1645265638 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54014870 con 0x7f3c60143680 2023-12-19T18:15:53.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.584+0000 7f3c47fff640 1 -- 172.21.15.27:0/1645265638 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c540149d0 con 0x7f3c60143680 2023-12-19T18:15:53.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.584+0000 7f3c47fff640 1 -- 172.21.15.27:0/1645265638 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54014d60 con 0x7f3c60143680 2023-12-19T18:15:53.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.584+0000 7f3c649d6640 1 -- 172.21.15.27:0/1645265638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c60143b70 con 0x7f3c60143680 2023-12-19T18:15:53.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.584+0000 7f3c649d6640 1 -- 172.21.15.27:0/1645265638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c60144130 con 0x7f3c60143680 2023-12-19T18:15:53.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.585+0000 7f3c649d6640 1 -- 172.21.15.27:0/1645265638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6004e840 con 0x7f3c60143680 2023-12-19T18:15:53.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.586+0000 7f3c47fff640 1 -- 172.21.15.27:0/1645265638 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3c54015410 con 0x7f3c60143680 2023-12-19T18:15:53.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.587+0000 7f3c47fff640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3c4c06a430 0x7f3c4c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:53.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.587+0000 7f3c47fff640 1 -- 172.21.15.27:0/1645265638 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3c54088ff0 con 0x7f3c60143680 2023-12-19T18:15:53.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.588+0000 7f3c5f7fe640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3c4c06a430 0x7f3c4c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:53.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.589+0000 7f3c5f7fe640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3c4c06a430 0x7f3c4c06c8f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c50034000 tx=0x7f3c50007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:53.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.591+0000 7f3c47fff640 1 -- 172.21.15.27:0/1645265638 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3c54054520 con 0x7f3c60143680 2023-12-19T18:15:53.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.789+0000 7f3c649d6640 1 -- 172.21.15.27:0/1645265638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c6006a5e0 con 0x7f3c60143680 2023-12-19T18:15:53.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.790+0000 7f3c47fff640 1 -- 172.21.15.27:0/1645265638 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c54006c30 con 0x7f3c60143680 2023-12-19T18:15:53.789 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:15:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.793+0000 7f3c649d6640 1 -- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3c4c06a430 msgr2=0x7f3c4c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.793+0000 7f3c649d6640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3c4c06a430 0x7f3c4c06c8f0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c50034000 tx=0x7f3c50007000 comp rx=0 tx=0).stop 2023-12-19T18:15:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.793+0000 7f3c649d6640 1 -- 172.21.15.27:0/1645265638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c60143680 msgr2=0x7f3c60141400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.793+0000 7f3c649d6640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c60143680 0x7f3c60141400 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400bfd0 tx=0x7f3c5400f920 comp rx=0 tx=0).stop 2023-12-19T18:15:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.793+0000 7f3c649d6640 1 -- 172.21.15.27:0/1645265638 shutdown_connections 2023-12-19T18:15:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.793+0000 7f3c649d6640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3c4c06a430 0x7f3c4c06c8f0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.793+0000 7f3c649d6640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c6006b000 0x7f3c6006b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.793+0000 7f3c649d6640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c60143680 0x7f3c60141400 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.793+0000 7f3c649d6640 1 --2- 172.21.15.27:0/1645265638 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c480039f0 0x7f3c600acdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.793+0000 7f3c649d6640 1 -- 172.21.15.27:0/1645265638 >> 172.21.15.27:0/1645265638 conn(0x7f3c6006a060 msgr2=0x7f3c601512f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.794+0000 7f3c649d6640 1 -- 172.21.15.27:0/1645265638 shutdown_connections 2023-12-19T18:15:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:53.794+0000 7f3c649d6640 1 -- 172.21.15.27:0/1645265638 wait complete. 2023-12-19T18:15:53.812 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:15:54.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.051+0000 7f7d2a3bc640 1 Processor -- start 2023-12-19T18:15:54.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.051+0000 7f7d2a3bc640 1 -- start start 2023-12-19T18:15:54.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.053+0000 7f7d2a3bc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7d24151c90 0x7f7d2406b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.053+0000 7f7d2a3bc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7d2415e9d0 0x7f7d240cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.053+0000 7f7d2a3bc640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2415b3e0 con 0x7f7d24152070 2023-12-19T18:15:54.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.053+0000 7f7d2a3bc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2415eed0 con 0x7f7d2415e9d0 2023-12-19T18:15:54.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.053+0000 7f7d2a3bc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2415f1b0 con 0x7f7d24151c90 2023-12-19T18:15:54.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.054+0000 7f7d28bb9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7d2415e9d0 0x7f7d240cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.054+0000 7f7d29bbb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7d24152070 0x7f7d2415b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46258/0 (socket says 172.21.15.27:46258) 2023-12-19T18:15:54.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.054+0000 7f7d29bbb640 1 -- 172.21.15.27:0/1978172126 learned_addr learned my addr 172.21.15.27:0/1978172126 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:54.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.054+0000 7f7d293ba640 1 --2- 172.21.15.27:0/1978172126 >> v2:172.21.15.192:3300/0 conn(0x7f7d24151c90 0x7f7d2406b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.054+0000 7f7d28bb9640 1 -- 172.21.15.27:0/1978172126 >> v2:172.21.15.192:3300/0 conn(0x7f7d24151c90 msgr2=0x7f7d2406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:54.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.054+0000 7f7d28bb9640 1 --2- 172.21.15.27:0/1978172126 >> v2:172.21.15.192:3300/0 conn(0x7f7d24151c90 0x7f7d2406b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.054+0000 7f7d28bb9640 1 -- 172.21.15.27:0/1978172126 >> v1:172.21.15.27:6789/0 conn(0x7f7d24152070 legacy=0x7f7d2415b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.054+0000 7f7d1bfff640 1 -- 172.21.15.27:0/1978172126 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2387322033 0 0) 0x7f7d2415b3e0 con 0x7f7d24152070 2023-12-19T18:15:54.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.054+0000 7f7d28bb9640 1 -- 172.21.15.27:0/1978172126 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2415f490 con 0x7f7d2415e9d0 2023-12-19T18:15:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.055+0000 7f7d28bb9640 1 --2- 172.21.15.27:0/1978172126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2415e9d0 0x7f7d240cee40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7d140097a0 tx=0x7f7d1402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abd95d6c67a64454 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.056+0000 7f7d1bfff640 1 -- 172.21.15.27:0/1978172126 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1402e070 con 0x7f7d2415e9d0 2023-12-19T18:15:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.056+0000 7f7d1bfff640 1 -- 172.21.15.27:0/1978172126 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d140315a0 con 0x7f7d2415e9d0 2023-12-19T18:15:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.056+0000 7f7d1bfff640 1 -- 172.21.15.27:0/1978172126 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d14031930 con 0x7f7d2415e9d0 2023-12-19T18:15:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.056+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1978172126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2415e9d0 msgr2=0x7f7d240cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.056+0000 7f7d2a3bc640 1 --2- 172.21.15.27:0/1978172126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2415e9d0 0x7f7d240cee40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f7d140097a0 tx=0x7f7d1402b840 comp rx=0 tx=0).stop 2023-12-19T18:15:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.056+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1978172126 shutdown_connections 2023-12-19T18:15:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.056+0000 7f7d2a3bc640 1 --2- 172.21.15.27:0/1978172126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2415e9d0 0x7f7d240cee40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.056+0000 7f7d2a3bc640 1 --2- 172.21.15.27:0/1978172126 >> v2:172.21.15.192:3300/0 conn(0x7f7d24151c90 0x7f7d2406b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.056+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1978172126 >> 172.21.15.27:0/1978172126 conn(0x7f7d2406a060 msgr2=0x7f7d2406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.056+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1978172126 shutdown_connections 2023-12-19T18:15:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.056+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1978172126 wait complete. 2023-12-19T18:15:54.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.058+0000 7f7d2a3bc640 1 Processor -- start 2023-12-19T18:15:54.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.058+0000 7f7d2a3bc640 1 -- start start 2023-12-19T18:15:54.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.059+0000 7f7d2a3bc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d24151c90 0x7f7d2413f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.059+0000 7f7d2a3bc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d24152070 0x7f7d2413fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.059+0000 7f7d2a3bc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2415e9d0 0x7f7d2413bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.059+0000 7f7d2a3bc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2413c8d0 con 0x7f7d2415e9d0 2023-12-19T18:15:54.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.059+0000 7f7d2a3bc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2413cb20 con 0x7f7d24151c90 2023-12-19T18:15:54.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.059+0000 7f7d2a3bc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2413ce00 con 0x7f7d24152070 2023-12-19T18:15:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.060+0000 7f7d29bbb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2415e9d0 0x7f7d2413bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.060+0000 7f7d29bbb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2415e9d0 0x7f7d2413bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48500/0 (socket says 172.21.15.27:48500) 2023-12-19T18:15:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.060+0000 7f7d29bbb640 1 -- 172.21.15.27:0/1750713421 learned_addr learned my addr 172.21.15.27:0/1750713421 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.060+0000 7f7d28bb9640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d24152070 0x7f7d2413fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.060+0000 7f7d293ba640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d24151c90 0x7f7d2413f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.060+0000 7f7d29bbb640 1 -- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d24152070 msgr2=0x7f7d2413fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.060+0000 7f7d29bbb640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d24152070 0x7f7d2413fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.060+0000 7f7d29bbb640 1 -- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d24151c90 msgr2=0x7f7d2413f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.060+0000 7f7d29bbb640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d24151c90 0x7f7d2413f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.060+0000 7f7d29bbb640 1 -- 172.21.15.27:0/1750713421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d14009400 con 0x7f7d2415e9d0 2023-12-19T18:15:54.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.060+0000 7f7d293ba640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d24151c90 0x7f7d2413f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:54.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.060+0000 7f7d29bbb640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2415e9d0 0x7f7d2413bfd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c002230 tx=0x7f7d1c00cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:54.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.061+0000 7f7d19ffb640 1 -- 172.21.15.27:0/1750713421 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00ce30 con 0x7f7d2415e9d0 2023-12-19T18:15:54.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.061+0000 7f7d19ffb640 1 -- 172.21.15.27:0/1750713421 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c00a640 con 0x7f7d2415e9d0 2023-12-19T18:15:54.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.061+0000 7f7d19ffb640 1 -- 172.21.15.27:0/1750713421 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c00a9f0 con 0x7f7d2415e9d0 2023-12-19T18:15:54.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.061+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1750713421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2413d0e0 con 0x7f7d2415e9d0 2023-12-19T18:15:54.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.061+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1750713421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2413d6a0 con 0x7f7d2415e9d0 2023-12-19T18:15:54.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.062+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1750713421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2413d890 con 0x7f7d2415e9d0 2023-12-19T18:15:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.063+0000 7f7d19ffb640 1 -- 172.21.15.27:0/1750713421 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7d1c00ab50 con 0x7f7d2415e9d0 2023-12-19T18:15:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.064+0000 7f7d19ffb640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d0806a460 0x7f7d0806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.064+0000 7f7d19ffb640 1 -- 172.21.15.27:0/1750713421 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7d1c087db0 con 0x7f7d2415e9d0 2023-12-19T18:15:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.065+0000 7f7d293ba640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d0806a460 0x7f7d0806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.066+0000 7f7d293ba640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d0806a460 0x7f7d0806c920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006fd0 tx=0x7f7d20008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:54.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.069+0000 7f7d19ffb640 1 -- 172.21.15.27:0/1750713421 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7d1c0532f0 con 0x7f7d2415e9d0 2023-12-19T18:15:54.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.264+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1750713421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d2404e840 con 0x7f7d2415e9d0 2023-12-19T18:15:54.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.265+0000 7f7d19ffb640 1 -- 172.21.15.27:0/1750713421 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d1c018020 con 0x7f7d2415e9d0 2023-12-19T18:15:54.265 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:15:54.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.267+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d0806a460 msgr2=0x7f7d0806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.267+0000 7f7d2a3bc640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d0806a460 0x7f7d0806c920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7d20006fd0 tx=0x7f7d20008040 comp rx=0 tx=0).stop 2023-12-19T18:15:54.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.267+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1750713421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2415e9d0 msgr2=0x7f7d2413bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.267+0000 7f7d2a3bc640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2415e9d0 0x7f7d2413bfd0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c002230 tx=0x7f7d1c00cbe0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.268+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1750713421 shutdown_connections 2023-12-19T18:15:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.268+0000 7f7d2a3bc640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d0806a460 0x7f7d0806c920 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.268+0000 7f7d2a3bc640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2415e9d0 0x7f7d2413bfd0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.268+0000 7f7d2a3bc640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d24152070 0x7f7d2413fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.268+0000 7f7d2a3bc640 1 --2- 172.21.15.27:0/1750713421 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d24151c90 0x7f7d2413f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.268+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1750713421 >> 172.21.15.27:0/1750713421 conn(0x7f7d2406a060 msgr2=0x7f7d241507a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:54.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.268+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1750713421 shutdown_connections 2023-12-19T18:15:54.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.268+0000 7f7d2a3bc640 1 -- 172.21.15.27:0/1750713421 wait complete. 2023-12-19T18:15:54.286 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:15:54.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.522+0000 7f9fecda2640 1 Processor -- start 2023-12-19T18:15:54.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.523+0000 7f9fecda2640 1 -- start start 2023-12-19T18:15:54.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.524+0000 7f9fecda2640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9fe815d120 0x7f9fe80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.524+0000 7f9fecda2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9fe80ce950 0x7f9fe80d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.524+0000 7f9fecda2640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe80d3850 con 0x7f9fe80d1310 2023-12-19T18:15:54.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.525+0000 7f9fecda2640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe80d0b10 con 0x7f9fe80ce950 2023-12-19T18:15:54.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.525+0000 7f9fecda2640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fe80d1c90 con 0x7f9fe815d120 2023-12-19T18:15:54.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.525+0000 7f9fe7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9fe80ce950 0x7f9fe80d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.525+0000 7f9fe7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9fe80ce950 0x7f9fe80d1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48512/0 (socket says 172.21.15.27:48512) 2023-12-19T18:15:54.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.525+0000 7f9fe7fff640 1 -- 172.21.15.27:0/339287375 learned_addr learned my addr 172.21.15.27:0/339287375 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:54.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.525+0000 7f9fe77fe640 1 --2- 172.21.15.27:0/339287375 >> v2:172.21.15.192:3300/0 conn(0x7f9fe815d120 0x7f9fe80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.526+0000 7f9fe7fff640 1 -- 172.21.15.27:0/339287375 >> v2:172.21.15.192:3300/0 conn(0x7f9fe815d120 msgr2=0x7f9fe80cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:54.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.526+0000 7f9fe7fff640 1 --2- 172.21.15.27:0/339287375 >> v2:172.21.15.192:3300/0 conn(0x7f9fe815d120 0x7f9fe80cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.526+0000 7f9fe7fff640 1 -- 172.21.15.27:0/339287375 >> v1:172.21.15.192:6789/0 conn(0x7f9fe80d1310 legacy=0x7f9fe80b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.526+0000 7f9fe7fff640 1 -- 172.21.15.27:0/339287375 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe815d500 con 0x7f9fe80ce950 2023-12-19T18:15:54.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.527+0000 7f9fe7fff640 1 --2- 172.21.15.27:0/339287375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe80ce950 0x7f9fe80d1750 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8071e70 tx=0x7f9fe002bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0191e89c1e73a4f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:54.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.527+0000 7f9fe67fc640 1 -- 172.21.15.27:0/339287375 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe002cb10 con 0x7f9fe80ce950 2023-12-19T18:15:54.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.528+0000 7f9fe67fc640 1 -- 172.21.15.27:0/339287375 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe002cc70 con 0x7f9fe80ce950 2023-12-19T18:15:54.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.528+0000 7f9fe67fc640 1 -- 172.21.15.27:0/339287375 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe00343f0 con 0x7f9fe80ce950 2023-12-19T18:15:54.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.528+0000 7f9fecda2640 1 -- 172.21.15.27:0/339287375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe80ce950 msgr2=0x7f9fe80d1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.528+0000 7f9fecda2640 1 --2- 172.21.15.27:0/339287375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe80ce950 0x7f9fe80d1750 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8071e70 tx=0x7f9fe002bf70 comp rx=0 tx=0).stop 2023-12-19T18:15:54.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.528+0000 7f9fecda2640 1 -- 172.21.15.27:0/339287375 shutdown_connections 2023-12-19T18:15:54.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.528+0000 7f9fecda2640 1 --2- 172.21.15.27:0/339287375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe80ce950 0x7f9fe80d1750 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.528+0000 7f9fecda2640 1 --2- 172.21.15.27:0/339287375 >> v2:172.21.15.192:3300/0 conn(0x7f9fe815d120 0x7f9fe80cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.528+0000 7f9fecda2640 1 -- 172.21.15.27:0/339287375 >> 172.21.15.27:0/339287375 conn(0x7f9fe80d2d30 msgr2=0x7f9fe80d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:54.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.528+0000 7f9fecda2640 1 -- 172.21.15.27:0/339287375 shutdown_connections 2023-12-19T18:15:54.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.528+0000 7f9fecda2640 1 -- 172.21.15.27:0/339287375 wait complete. 2023-12-19T18:15:54.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.530+0000 7f9fecda2640 1 Processor -- start 2023-12-19T18:15:54.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.530+0000 7f9fecda2640 1 -- start start 2023-12-19T18:15:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.531+0000 7f9fecda2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe80ce950 0x7f9fe814b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.531+0000 7f9fecda2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe80d1310 0x7f9fe814bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.531+0000 7f9fecda2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe815d120 0x7f9fe80b0590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.531+0000 7f9fecda2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe815f190 con 0x7f9fe80ce950 2023-12-19T18:15:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.531+0000 7f9fecda2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe80cb510 con 0x7f9fe80d1310 2023-12-19T18:15:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.531+0000 7f9fecda2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe80cb7f0 con 0x7f9fe815d120 2023-12-19T18:15:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.532+0000 7f9fe77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe80ce950 0x7f9fe814b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.532+0000 7f9fe77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe80ce950 0x7f9fe814b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48518/0 (socket says 172.21.15.27:48518) 2023-12-19T18:15:54.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.532+0000 7f9fe77fe640 1 -- 172.21.15.27:0/1032323035 learned_addr learned my addr 172.21.15.27:0/1032323035 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:54.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.532+0000 7f9fe7fff640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe815d120 0x7f9fe80b0590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.532+0000 7f9fe77fe640 1 -- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe815d120 msgr2=0x7f9fe80b0590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.532+0000 7f9fe6ffd640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe80d1310 0x7f9fe814bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.532+0000 7f9fe77fe640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe815d120 0x7f9fe80b0590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.532+0000 7f9fe77fe640 1 -- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe80d1310 msgr2=0x7f9fe814bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.532+0000 7f9fe77fe640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe80d1310 0x7f9fe814bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.532+0000 7f9fe77fe640 1 -- 172.21.15.27:0/1032323035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe0009b30 con 0x7f9fe80ce950 2023-12-19T18:15:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.532+0000 7f9fe6ffd640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe80d1310 0x7f9fe814bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.533+0000 7f9fe77fe640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe80ce950 0x7f9fe814b050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800e870 tx=0x7f9fd800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.533+0000 7f9fcffff640 1 -- 172.21.15.27:0/1032323035 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd80094f0 con 0x7f9fe80ce950 2023-12-19T18:15:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.533+0000 7f9fcffff640 1 -- 172.21.15.27:0/1032323035 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd800c940 con 0x7f9fe80ce950 2023-12-19T18:15:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.533+0000 7f9fcffff640 1 -- 172.21.15.27:0/1032323035 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd800ccf0 con 0x7f9fe80ce950 2023-12-19T18:15:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.533+0000 7f9fecda2640 1 -- 172.21.15.27:0/1032323035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe80cbad0 con 0x7f9fe80ce950 2023-12-19T18:15:54.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.533+0000 7f9fecda2640 1 -- 172.21.15.27:0/1032323035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe814b590 con 0x7f9fe80ce950 2023-12-19T18:15:54.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.534+0000 7f9fecda2640 1 -- 172.21.15.27:0/1032323035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe804e840 con 0x7f9fe80ce950 2023-12-19T18:15:54.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.535+0000 7f9fcffff640 1 -- 172.21.15.27:0/1032323035 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9fd8013470 con 0x7f9fe80ce950 2023-12-19T18:15:54.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.537+0000 7f9fcffff640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fd406a460 0x7f9fd406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.537+0000 7f9fcffff640 1 -- 172.21.15.27:0/1032323035 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9fd8088950 con 0x7f9fe80ce950 2023-12-19T18:15:54.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.538+0000 7f9fe6ffd640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fd406a460 0x7f9fd406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.539+0000 7f9fe6ffd640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fd406a460 0x7f9fd406c920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9fe806bc10 tx=0x7f9fdc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:54.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.541+0000 7f9fcffff640 1 -- 172.21.15.27:0/1032323035 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9fd8053fb0 con 0x7f9fe80ce950 2023-12-19T18:15:54.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.740+0000 7f9fecda2640 1 -- 172.21.15.27:0/1032323035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9fe80afc10 con 0x7f9fe80ce950 2023-12-19T18:15:54.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.741+0000 7f9fcffff640 1 -- 172.21.15.27:0/1032323035 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9fd80170a0 con 0x7f9fe80ce950 2023-12-19T18:15:54.741 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:15:54.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.744+0000 7f9fecda2640 1 -- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fd406a460 msgr2=0x7f9fd406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.744+0000 7f9fecda2640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fd406a460 0x7f9fd406c920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f9fe806bc10 tx=0x7f9fdc009040 comp rx=0 tx=0).stop 2023-12-19T18:15:54.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.744+0000 7f9fecda2640 1 -- 172.21.15.27:0/1032323035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe80ce950 msgr2=0x7f9fe814b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.744+0000 7f9fecda2640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe80ce950 0x7f9fe814b050 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9fd800e870 tx=0x7f9fd800ed30 comp rx=0 tx=0).stop 2023-12-19T18:15:54.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.744+0000 7f9fecda2640 1 -- 172.21.15.27:0/1032323035 shutdown_connections 2023-12-19T18:15:54.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.744+0000 7f9fecda2640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fd406a460 0x7f9fd406c920 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.744+0000 7f9fecda2640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe815d120 0x7f9fe80b0590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.744+0000 7f9fecda2640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fe80d1310 0x7f9fe814bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.744+0000 7f9fecda2640 1 --2- 172.21.15.27:0/1032323035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe80ce950 0x7f9fe814b050 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.744+0000 7f9fecda2640 1 -- 172.21.15.27:0/1032323035 >> 172.21.15.27:0/1032323035 conn(0x7f9fe80d2d30 msgr2=0x7f9fe815b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:54.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.744+0000 7f9fecda2640 1 -- 172.21.15.27:0/1032323035 shutdown_connections 2023-12-19T18:15:54.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.744+0000 7f9fecda2640 1 -- 172.21.15.27:0/1032323035 wait complete. 2023-12-19T18:15:54.765 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:15:54.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.991+0000 7f7b7cb49640 1 Processor -- start 2023-12-19T18:15:54.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.991+0000 7f7b7cb49640 1 -- start start 2023-12-19T18:15:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.992+0000 7f7b7cb49640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7b78070fb0 0x7f7b7806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.992+0000 7f7b7cb49640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b780adf10 con 0x7f7b7806b850 2023-12-19T18:15:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.993+0000 7f7b7cb49640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b780adac0 con 0x7f7b7815ac50 2023-12-19T18:15:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.993+0000 7f7b7cb49640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b780af630 con 0x7f7b78070fb0 2023-12-19T18:15:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.993+0000 7f7b77fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7b7806b850 0x7f7b7806f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46270/0 (socket says 172.21.15.27:46270) 2023-12-19T18:15:54.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.993+0000 7f7b77fff640 1 -- 172.21.15.27:0/1579397156 learned_addr learned my addr 172.21.15.27:0/1579397156 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:54.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.993+0000 7f7b76ffd640 1 --2- 172.21.15.27:0/1579397156 >> v2:172.21.15.192:3300/0 conn(0x7f7b78070fb0 0x7f7b7806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.993+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2126857906 0 0) 0x7f7b780adf10 con 0x7f7b7806b850 2023-12-19T18:15:54.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.993+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b64003520 con 0x7f7b7806b850 2023-12-19T18:15:54.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2313481368 0 0) 0x7f7b64003520 con 0x7f7b7806b850 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 >> v2:172.21.15.192:3300/0 conn(0x7f7b78070fb0 msgr2=0x7f7b7806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b767fc640 1 --2- 172.21.15.27:0/1579397156 >> v2:172.21.15.192:3300/0 conn(0x7f7b78070fb0 0x7f7b7806b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 >> v1:172.21.15.192:6789/0 conn(0x7f7b7815ac50 legacy=0x7f7b780aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b780713f0 con 0x7f7b7806b850 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f7b700019f0 con 0x7f7b7806b850 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 >> v1:172.21.15.27:6789/0 conn(0x7f7b7806b850 legacy=0x7f7b7806f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b767fc640 1 --2- 172.21.15.27:0/1579397156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b640039c0 0x7f7b64003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b767fc640 1 --2- 172.21.15.27:0/1579397156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b640047c0 0x7f7b64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b767fc640 1 --2- 172.21.15.27:0/1579397156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b640052a0 0x7f7b6400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780af630 con 0x7f7b640039c0 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6400c1a0 con 0x7f7b640047c0 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6400c360 con 0x7f7b640052a0 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.994+0000 7f7b777fe640 1 --2- 172.21.15.27:0/1579397156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b640039c0 0x7f7b64003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.995+0000 7f7b76ffd640 1 --2- 172.21.15.27:0/1579397156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b640047c0 0x7f7b64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.995+0000 7f7b77fff640 1 --2- 172.21.15.27:0/1579397156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b640052a0 0x7f7b6400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:54.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.995+0000 7f7b777fe640 1 -- 172.21.15.27:0/1579397156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b640052a0 msgr2=0x7f7b6400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:54.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.995+0000 7f7b777fe640 1 --2- 172.21.15.27:0/1579397156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b640052a0 0x7f7b6400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.995+0000 7f7b777fe640 1 -- 172.21.15.27:0/1579397156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b640047c0 msgr2=0x7f7b64004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:54.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.995+0000 7f7b777fe640 1 --2- 172.21.15.27:0/1579397156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b640047c0 0x7f7b64004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.995+0000 7f7b777fe640 1 -- 172.21.15.27:0/1579397156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b64003520 con 0x7f7b640039c0 2023-12-19T18:15:54.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.995+0000 7f7b77fff640 1 -- 172.21.15.27:0/1579397156 reap_dead start 2023-12-19T18:15:54.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.996+0000 7f7b777fe640 1 --2- 172.21.15.27:0/1579397156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b640039c0 0x7f7b64003de0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7b68001750 tx=0x7f7b640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d365a4141e8083d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:54.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.996+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b680052d0 con 0x7f7b640039c0 2023-12-19T18:15:54.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.996+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6802e050 con 0x7f7b640039c0 2023-12-19T18:15:54.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.996+0000 7f7b767fc640 1 -- 172.21.15.27:0/1579397156 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b68003830 con 0x7f7b640039c0 2023-12-19T18:15:54.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.997+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1579397156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b640039c0 msgr2=0x7f7b64003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:54.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.997+0000 7f7b7cb49640 1 --2- 172.21.15.27:0/1579397156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b640039c0 0x7f7b64003de0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7b68001750 tx=0x7f7b640057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.997+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1579397156 shutdown_connections 2023-12-19T18:15:54.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.997+0000 7f7b7cb49640 1 --2- 172.21.15.27:0/1579397156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b640039c0 0x7f7b64003de0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:54.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.997+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1579397156 >> 172.21.15.27:0/1579397156 conn(0x7f7b7806a7d0 msgr2=0x7f7b780ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:54.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.997+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1579397156 shutdown_connections 2023-12-19T18:15:54.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.997+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1579397156 wait complete. 2023-12-19T18:15:54.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.999+0000 7f7b7cb49640 1 Processor -- start 2023-12-19T18:15:54.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:54.999+0000 7f7b7cb49640 1 -- start start 2023-12-19T18:15:55.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.000+0000 7f7b7cb49640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b640039c0 0x7f7b781f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:55.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.000+0000 7f7b7cb49640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b78152a30 0x7f7b780c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:55.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.000+0000 7f7b7cb49640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815c550 0x7f7b780c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:55.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.000+0000 7f7b7cb49640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b781f2510 con 0x7f7b7815c550 2023-12-19T18:15:55.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.000+0000 7f7b7cb49640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78149f50 con 0x7f7b640039c0 2023-12-19T18:15:55.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.000+0000 7f7b7cb49640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780c8f00 con 0x7f7b78152a30 2023-12-19T18:15:55.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.000+0000 7f7b77fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815c550 0x7f7b780c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:55.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.000+0000 7f7b77fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815c550 0x7f7b780c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48522/0 (socket says 172.21.15.27:48522) 2023-12-19T18:15:55.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.000+0000 7f7b77fff640 1 -- 172.21.15.27:0/1286813701 learned_addr learned my addr 172.21.15.27:0/1286813701 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:55.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b777fe640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b640039c0 0x7f7b781f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:55.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b76ffd640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b78152a30 0x7f7b780c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:55.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b77fff640 1 -- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b78152a30 msgr2=0x7f7b780c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:55.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b77fff640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b78152a30 0x7f7b780c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b77fff640 1 -- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b640039c0 msgr2=0x7f7b781f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:55.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b77fff640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b640039c0 0x7f7b781f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b77fff640 1 -- 172.21.15.27:0/1286813701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b78152f10 con 0x7f7b7815c550 2023-12-19T18:15:55.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b76ffd640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b78152a30 0x7f7b780c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:55.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b777fe640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b640039c0 0x7f7b781f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:55.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b77fff640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815c550 0x7f7b780c8100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7b7806d790 tx=0x7f7b70007bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:55.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b6ffff640 1 -- 172.21.15.27:0/1286813701 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000ead0 con 0x7f7b7815c550 2023-12-19T18:15:55.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b6ffff640 1 -- 172.21.15.27:0/1286813701 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7000ec30 con 0x7f7b7815c550 2023-12-19T18:15:55.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1286813701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b7814a1a0 con 0x7f7b7815c550 2023-12-19T18:15:55.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.001+0000 7f7b6ffff640 1 -- 172.21.15.27:0/1286813701 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b70014430 con 0x7f7b7815c550 2023-12-19T18:15:55.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.002+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1286813701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b7815b030 con 0x7f7b7815c550 2023-12-19T18:15:55.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.003+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1286813701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b78067170 con 0x7f7b7815c550 2023-12-19T18:15:55.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.004+0000 7f7b6ffff640 1 -- 172.21.15.27:0/1286813701 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7b70014590 con 0x7f7b7815c550 2023-12-19T18:15:55.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.005+0000 7f7b6ffff640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5806a460 0x7f7b5806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:55.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.005+0000 7f7b6ffff640 1 -- 172.21.15.27:0/1286813701 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7b700885c0 con 0x7f7b7815c550 2023-12-19T18:15:55.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.006+0000 7f7b777fe640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5806a460 0x7f7b5806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:55.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.007+0000 7f7b777fe640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5806a460 0x7f7b5806c920 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7b68033000 tx=0x7f7b68009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:55.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.009+0000 7f7b6ffff640 1 -- 172.21.15.27:0/1286813701 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7b70057610 con 0x7f7b7815c550 2023-12-19T18:15:55.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.219+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1286813701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7b7804e840 con 0x7f7b7815c550 2023-12-19T18:15:55.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.220+0000 7f7b6ffff640 1 -- 172.21.15.27:0/1286813701 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7b700571a0 con 0x7f7b7815c550 2023-12-19T18:15:55.220 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:15:55.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.223+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5806a460 msgr2=0x7f7b5806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:55.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.223+0000 7f7b7cb49640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5806a460 0x7f7b5806c920 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7b68033000 tx=0x7f7b68009030 comp rx=0 tx=0).stop 2023-12-19T18:15:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.223+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1286813701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815c550 msgr2=0x7f7b780c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.223+0000 7f7b7cb49640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815c550 0x7f7b780c8100 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f7b7806d790 tx=0x7f7b70007bc0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.223+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1286813701 shutdown_connections 2023-12-19T18:15:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.223+0000 7f7b7cb49640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5806a460 0x7f7b5806c920 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.223+0000 7f7b7cb49640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815c550 0x7f7b780c8100 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.223+0000 7f7b7cb49640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b78152a30 0x7f7b780c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.223+0000 7f7b7cb49640 1 --2- 172.21.15.27:0/1286813701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b640039c0 0x7f7b781f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.223+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1286813701 >> 172.21.15.27:0/1286813701 conn(0x7f7b7806a7d0 msgr2=0x7f7b7806c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.223+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1286813701 shutdown_connections 2023-12-19T18:15:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.223+0000 7f7b7cb49640 1 -- 172.21.15.27:0/1286813701 wait complete. 2023-12-19T18:15:55.243 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:15:55.243 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:15:55.306 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:55.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.540+0000 7f9c6b59e640 1 Processor -- start 2023-12-19T18:15:55.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.540+0000 7f9c6b59e640 1 -- start start 2023-12-19T18:15:55.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.542+0000 7f9c6b59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9c6c1553a0 0x7f9c6c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:55.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.542+0000 7f9c6b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c6c15ee30 0x7f9c6c0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:55.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.542+0000 7f9c6b59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c6c0b0360 con 0x7f9c6c0afeb0 2023-12-19T18:15:55.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.542+0000 7f9c6b59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c6c0cbcf0 con 0x7f9c6c1553a0 2023-12-19T18:15:55.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.542+0000 7f9c6b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c0cb350 con 0x7f9c6c15ee30 2023-12-19T18:15:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.543+0000 7f9c6ad9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c6c15ee30 0x7f9c6c0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.543+0000 7f9c6ad9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c6c15ee30 0x7f9c6c0c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:40962/0 (socket says 172.21.15.27:40962) 2023-12-19T18:15:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.543+0000 7f9c6ad9d640 1 -- 172.21.15.27:0/3391024476 learned_addr learned my addr 172.21.15.27:0/3391024476 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.543+0000 7f9c6a59c640 1 --2- 172.21.15.27:0/3391024476 >> v2:172.21.15.192:3300/0 conn(0x7f9c6c1553a0 0x7f9c6c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.543+0000 7f9c6ad9d640 1 -- 172.21.15.27:0/3391024476 >> v2:172.21.15.192:3300/0 conn(0x7f9c6c1553a0 msgr2=0x7f9c6c0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.543+0000 7f9c6ad9d640 1 --2- 172.21.15.27:0/3391024476 >> v2:172.21.15.192:3300/0 conn(0x7f9c6c1553a0 0x7f9c6c0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.543+0000 7f9c6ad9d640 1 -- 172.21.15.27:0/3391024476 >> v1:172.21.15.192:6789/0 conn(0x7f9c6c0afeb0 legacy=0x7f9c6c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:55.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.543+0000 7f9c6ad9d640 1 -- 172.21.15.27:0/3391024476 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c0cb630 con 0x7f9c6c15ee30 2023-12-19T18:15:55.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.545+0000 7f9c6ad9d640 1 --2- 172.21.15.27:0/3391024476 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c6c15ee30 0x7f9c6c0c39d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009e50 tx=0x7f9c6402bee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5d8822b5b34aacb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:55.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.545+0000 7f9c69d9b640 1 -- 172.21.15.27:0/3391024476 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6402ca80 con 0x7f9c6c15ee30 2023-12-19T18:15:55.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.545+0000 7f9c69d9b640 1 -- 172.21.15.27:0/3391024476 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6402cbe0 con 0x7f9c6c15ee30 2023-12-19T18:15:55.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.545+0000 7f9c69d9b640 1 -- 172.21.15.27:0/3391024476 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c640343f0 con 0x7f9c6c15ee30 2023-12-19T18:15:55.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.546+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3391024476 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c6c15ee30 msgr2=0x7f9c6c0c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:55.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.546+0000 7f9c6b59e640 1 --2- 172.21.15.27:0/3391024476 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c6c15ee30 0x7f9c6c0c39d0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009e50 tx=0x7f9c6402bee0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.546+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3391024476 shutdown_connections 2023-12-19T18:15:55.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.546+0000 7f9c6b59e640 1 --2- 172.21.15.27:0/3391024476 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c6c15ee30 0x7f9c6c0c39d0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.546+0000 7f9c6b59e640 1 --2- 172.21.15.27:0/3391024476 >> v2:172.21.15.192:3300/0 conn(0x7f9c6c1553a0 0x7f9c6c0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.546+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3391024476 >> 172.21.15.27:0/3391024476 conn(0x7f9c6c069c70 msgr2=0x7f9c6c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:55.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.546+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3391024476 shutdown_connections 2023-12-19T18:15:55.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.546+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3391024476 wait complete. 2023-12-19T18:15:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.548+0000 7f9c6b59e640 1 Processor -- start 2023-12-19T18:15:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.548+0000 7f9c6b59e640 1 -- start start 2023-12-19T18:15:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.549+0000 7f9c6b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c6c0afeb0 0x7f9c6c0b17b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.549+0000 7f9c6b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c6c1553a0 0x7f9c6c1f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.549+0000 7f9c6b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c15ee30 0x7f9c6c1f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.549+0000 7f9c6b59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c14f720 con 0x7f9c6c15ee30 2023-12-19T18:15:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.549+0000 7f9c6b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c15c370 con 0x7f9c6c0afeb0 2023-12-19T18:15:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.549+0000 7f9c6b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c15c650 con 0x7f9c6c1553a0 2023-12-19T18:15:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.550+0000 7f9c6ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c15ee30 0x7f9c6c1f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.550+0000 7f9c6ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c15ee30 0x7f9c6c1f6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48538/0 (socket says 172.21.15.27:48538) 2023-12-19T18:15:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.550+0000 7f9c6ad9d640 1 -- 172.21.15.27:0/3060473198 learned_addr learned my addr 172.21.15.27:0/3060473198 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:55.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.550+0000 7f9c6a59c640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c6c0afeb0 0x7f9c6c0b17b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.550+0000 7f9c63fff640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c6c1553a0 0x7f9c6c1f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.550+0000 7f9c6ad9d640 1 -- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c6c1553a0 msgr2=0x7f9c6c1f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.550+0000 7f9c6ad9d640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c6c1553a0 0x7f9c6c1f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.550+0000 7f9c6ad9d640 1 -- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c6c0afeb0 msgr2=0x7f9c6c0b17b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.550+0000 7f9c6ad9d640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c6c0afeb0 0x7f9c6c0b17b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.550+0000 7f9c6ad9d640 1 -- 172.21.15.27:0/3060473198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c64009b30 con 0x7f9c6c15ee30 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.550+0000 7f9c6a59c640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c6c0afeb0 0x7f9c6c0b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.550+0000 7f9c63fff640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c6c1553a0 0x7f9c6c1f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.551+0000 7f9c6ad9d640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c15ee30 0x7f9c6c1f6300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009e50 tx=0x7f9c64004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.551+0000 7f9c62ffd640 1 -- 172.21.15.27:0/3060473198 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c64040670 con 0x7f9c6c15ee30 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.551+0000 7f9c62ffd640 1 -- 172.21.15.27:0/3060473198 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c640407d0 con 0x7f9c6c15ee30 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.551+0000 7f9c62ffd640 1 -- 172.21.15.27:0/3060473198 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c64040b80 con 0x7f9c6c15ee30 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.551+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3060473198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6c15c930 con 0x7f9c6c15ee30 2023-12-19T18:15:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.551+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3060473198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6c0b3500 con 0x7f9c6c15ee30 2023-12-19T18:15:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.552+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3060473198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6c0c3ed0 con 0x7f9c6c15ee30 2023-12-19T18:15:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.553+0000 7f9c62ffd640 1 -- 172.21.15.27:0/3060473198 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9c6402cd50 con 0x7f9c6c15ee30 2023-12-19T18:15:55.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.554+0000 7f9c62ffd640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c4c06a430 0x7f9c4c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:55.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.554+0000 7f9c62ffd640 1 -- 172.21.15.27:0/3060473198 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9c640ab6c0 con 0x7f9c6c15ee30 2023-12-19T18:15:55.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.555+0000 7f9c6a59c640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c4c06a430 0x7f9c4c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.556+0000 7f9c6a59c640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c4c06a430 0x7f9c4c06c8f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009ac0 tx=0x7f9c5c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:55.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.559+0000 7f9c62ffd640 1 -- 172.21.15.27:0/3060473198 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9c64076bf0 con 0x7f9c6c15ee30 2023-12-19T18:15:55.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.756+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3060473198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c6c04e840 con 0x7f9c6c15ee30 2023-12-19T18:15:55.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.758+0000 7f9c62ffd640 1 -- 172.21.15.27:0/3060473198 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16090 (secure 0 0 0) 0x7f9c6407a210 con 0x7f9c6c15ee30 2023-12-19T18:15:55.758 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:55.759 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:15:43.580746+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:55.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.761+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c4c06a430 msgr2=0x7f9c4c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:55.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.761+0000 7f9c6b59e640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c4c06a430 0x7f9c4c06c8f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9c5c009ac0 tx=0x7f9c5c008040 comp rx=0 tx=0).stop 2023-12-19T18:15:55.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.761+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3060473198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c15ee30 msgr2=0x7f9c6c1f6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:55.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.761+0000 7f9c6b59e640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c15ee30 0x7f9c6c1f6300 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f9c64009e50 tx=0x7f9c64004a40 comp rx=0 tx=0).stop 2023-12-19T18:15:55.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.761+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3060473198 shutdown_connections 2023-12-19T18:15:55.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.761+0000 7f9c6b59e640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c4c06a430 0x7f9c4c06c8f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.761+0000 7f9c6b59e640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c15ee30 0x7f9c6c1f6300 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.761+0000 7f9c6b59e640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c6c1553a0 0x7f9c6c1f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.761+0000 7f9c6b59e640 1 --2- 172.21.15.27:0/3060473198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c6c0afeb0 0x7f9c6c0b17b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:55.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.761+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3060473198 >> 172.21.15.27:0/3060473198 conn(0x7f9c6c069c70 msgr2=0x7f9c6c0c4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:55.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.761+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3060473198 shutdown_connections 2023-12-19T18:15:55.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:55.761+0000 7f9c6b59e640 1 -- 172.21.15.27:0/3060473198 wait complete. 2023-12-19T18:15:55.782 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:15:55.783 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:15:56.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.016+0000 7f7c7a170640 1 Processor -- start 2023-12-19T18:15:56.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.016+0000 7f7c7a170640 1 -- start start 2023-12-19T18:15:56.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.017+0000 7f7c7a170640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c7415cf80 0x7f7c740d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.017+0000 7f7c7a170640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7c740b85e0 0x7f7c740cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.017+0000 7f7c7a170640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c740d0e10 con 0x7f7c740b8a60 2023-12-19T18:15:56.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.017+0000 7f7c7a170640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c740d1c70 con 0x7f7c740b85e0 2023-12-19T18:15:56.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.017+0000 7f7c7a170640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7415d360 con 0x7f7c7415cf80 2023-12-19T18:15:56.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.018+0000 7f7c7896d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7c740b85e0 0x7f7c740cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.018+0000 7f7c7896d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7c740b85e0 0x7f7c740cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48540/0 (socket says 172.21.15.27:48540) 2023-12-19T18:15:56.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.018+0000 7f7c7896d640 1 -- 172.21.15.27:0/3819770584 learned_addr learned my addr 172.21.15.27:0/3819770584 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:56.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.018+0000 7f7c7916e640 1 --2- 172.21.15.27:0/3819770584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c7415cf80 0x7f7c740d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.018+0000 7f7c7896d640 1 -- 172.21.15.27:0/3819770584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c7415cf80 msgr2=0x7f7c740d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:56.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.018+0000 7f7c7896d640 1 --2- 172.21.15.27:0/3819770584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c7415cf80 0x7f7c740d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.018+0000 7f7c7896d640 1 -- 172.21.15.27:0/3819770584 >> v1:172.21.15.192:6789/0 conn(0x7f7c740b8a60 legacy=0x7f7c740cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:56.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.018+0000 7f7c7896d640 1 -- 172.21.15.27:0/3819770584 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c740d1060 con 0x7f7c740b85e0 2023-12-19T18:15:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.019+0000 7f7c7896d640 1 --2- 172.21.15.27:0/3819770584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c740b85e0 0x7f7c740cda40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009870 tx=0x7f7c6402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba30fd60e42e6f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.020+0000 7f7c6bfff640 1 -- 172.21.15.27:0/3819770584 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64030840 con 0x7f7c740b85e0 2023-12-19T18:15:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.020+0000 7f7c6bfff640 1 -- 172.21.15.27:0/3819770584 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c640309a0 con 0x7f7c740b85e0 2023-12-19T18:15:56.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.020+0000 7f7c6bfff640 1 -- 172.21.15.27:0/3819770584 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c64030d30 con 0x7f7c740b85e0 2023-12-19T18:15:56.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.020+0000 7f7c7a170640 1 -- 172.21.15.27:0/3819770584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c740b85e0 msgr2=0x7f7c740cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.020+0000 7f7c7a170640 1 --2- 172.21.15.27:0/3819770584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c740b85e0 0x7f7c740cda40 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f7c64009870 tx=0x7f7c6402b980 comp rx=0 tx=0).stop 2023-12-19T18:15:56.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.021+0000 7f7c7a170640 1 -- 172.21.15.27:0/3819770584 shutdown_connections 2023-12-19T18:15:56.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.021+0000 7f7c7a170640 1 --2- 172.21.15.27:0/3819770584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c740b85e0 0x7f7c740cda40 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.021+0000 7f7c7a170640 1 --2- 172.21.15.27:0/3819770584 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c7415cf80 0x7f7c740d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.021+0000 7f7c7a170640 1 -- 172.21.15.27:0/3819770584 >> 172.21.15.27:0/3819770584 conn(0x7f7c74072e20 msgr2=0x7f7c74073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:56.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.021+0000 7f7c7a170640 1 -- 172.21.15.27:0/3819770584 shutdown_connections 2023-12-19T18:15:56.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.021+0000 7f7c7a170640 1 -- 172.21.15.27:0/3819770584 wait complete. 2023-12-19T18:15:56.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.023+0000 7f7c7a170640 1 Processor -- start 2023-12-19T18:15:56.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.023+0000 7f7c7a170640 1 -- start start 2023-12-19T18:15:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.024+0000 7f7c7a170640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c740b85e0 0x7f7c74152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.024+0000 7f7c7a170640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c7415cf80 0x7f7c740c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.024+0000 7f7c7a170640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c740c93c0 0x7f7c740b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.024+0000 7f7c7a170640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741559a0 con 0x7f7c740b85e0 2023-12-19T18:15:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.024+0000 7f7c7a170640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740c89f0 con 0x7f7c7415cf80 2023-12-19T18:15:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.024+0000 7f7c7a170640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c740c80c0 con 0x7f7c740c93c0 2023-12-19T18:15:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.024+0000 7f7c7916e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c740b85e0 0x7f7c74152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.024+0000 7f7c7916e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c740b85e0 0x7f7c74152120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48556/0 (socket says 172.21.15.27:48556) 2023-12-19T18:15:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.024+0000 7f7c7916e640 1 -- 172.21.15.27:0/3270127301 learned_addr learned my addr 172.21.15.27:0/3270127301 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.024+0000 7f7c7896d640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c7415cf80 0x7f7c740c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.024+0000 7f7c7996f640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c740c93c0 0x7f7c740b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.024+0000 7f7c7916e640 1 -- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c740c93c0 msgr2=0x7f7c740b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.025+0000 7f7c7916e640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c740c93c0 0x7f7c740b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.025+0000 7f7c7916e640 1 -- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c7415cf80 msgr2=0x7f7c740c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.025+0000 7f7c7916e640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c7415cf80 0x7f7c740c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.025+0000 7f7c7916e640 1 -- 172.21.15.27:0/3270127301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c640094d0 con 0x7f7c740b85e0 2023-12-19T18:15:56.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.025+0000 7f7c7916e640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c740b85e0 0x7f7c74152120 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c7f0 tx=0x7f7c7000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:56.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.025+0000 7f7c69ffb640 1 -- 172.21.15.27:0/3270127301 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70004220 con 0x7f7c740b85e0 2023-12-19T18:15:56.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.025+0000 7f7c69ffb640 1 -- 172.21.15.27:0/3270127301 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c70004380 con 0x7f7c740b85e0 2023-12-19T18:15:56.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.025+0000 7f7c69ffb640 1 -- 172.21.15.27:0/3270127301 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70007840 con 0x7f7c740b85e0 2023-12-19T18:15:56.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.025+0000 7f7c7a170640 1 -- 172.21.15.27:0/3270127301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c740c83a0 con 0x7f7c740b85e0 2023-12-19T18:15:56.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.025+0000 7f7c7a170640 1 -- 172.21.15.27:0/3270127301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c74152660 con 0x7f7c740b85e0 2023-12-19T18:15:56.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.026+0000 7f7c7a170640 1 -- 172.21.15.27:0/3270127301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c7404e840 con 0x7f7c740b85e0 2023-12-19T18:15:56.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.027+0000 7f7c69ffb640 1 -- 172.21.15.27:0/3270127301 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7c70007bb0 con 0x7f7c740b85e0 2023-12-19T18:15:56.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.028+0000 7f7c69ffb640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c5806a430 0x7f7c5806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.028+0000 7f7c69ffb640 1 -- 172.21.15.27:0/3270127301 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7c700888e0 con 0x7f7c740b85e0 2023-12-19T18:15:56.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.029+0000 7f7c7896d640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c5806a430 0x7f7c5806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.030+0000 7f7c7896d640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c5806a430 0x7f7c5806c8f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7c64036000 tx=0x7f7c64008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:56.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.033+0000 7f7c69ffb640 1 -- 172.21.15.27:0/3270127301 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7c70053e10 con 0x7f7c740b85e0 2023-12-19T18:15:56.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.225+0000 7f7c7a170640 1 -- 172.21.15.27:0/3270127301 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7c740c6130 con 0x7f7c740b85e0 2023-12-19T18:15:56.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.226+0000 7f7c69ffb640 1 -- 172.21.15.27:0/3270127301 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7c7001e020 con 0x7f7c740b85e0 2023-12-19T18:15:56.226 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:15:56.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.229+0000 7f7c7a170640 1 -- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c5806a430 msgr2=0x7f7c5806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.229+0000 7f7c7a170640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c5806a430 0x7f7c5806c8f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f7c64036000 tx=0x7f7c64008030 comp rx=0 tx=0).stop 2023-12-19T18:15:56.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.229+0000 7f7c7a170640 1 -- 172.21.15.27:0/3270127301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c740b85e0 msgr2=0x7f7c74152120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.229+0000 7f7c7a170640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c740b85e0 0x7f7c74152120 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7c7000c7f0 tx=0x7f7c7000ccb0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.229+0000 7f7c7a170640 1 -- 172.21.15.27:0/3270127301 shutdown_connections 2023-12-19T18:15:56.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.229+0000 7f7c7a170640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c5806a430 0x7f7c5806c8f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.229+0000 7f7c7a170640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c740c93c0 0x7f7c740b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.229+0000 7f7c7a170640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c7415cf80 0x7f7c740c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.229+0000 7f7c7a170640 1 --2- 172.21.15.27:0/3270127301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c740b85e0 0x7f7c74152120 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.229+0000 7f7c7a170640 1 -- 172.21.15.27:0/3270127301 >> 172.21.15.27:0/3270127301 conn(0x7f7c74072e20 msgr2=0x7f7c740b4950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:56.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.230+0000 7f7c7a170640 1 -- 172.21.15.27:0/3270127301 shutdown_connections 2023-12-19T18:15:56.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.230+0000 7f7c7a170640 1 -- 172.21.15.27:0/3270127301 wait complete. 2023-12-19T18:15:56.248 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:15:56.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.478+0000 7f2e0aba5640 1 Processor -- start 2023-12-19T18:15:56.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.479+0000 7f2e0aba5640 1 -- start start 2023-12-19T18:15:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.480+0000 7f2e0aba5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e9d0 0x7f2e040cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.481+0000 7f2e0aba5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0415b3e0 con 0x7f2e04151c90 2023-12-19T18:15:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.481+0000 7f2e0aba5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e0415eed0 con 0x7f2e04152070 2023-12-19T18:15:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.481+0000 7f2e0aba5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0415f1b0 con 0x7f2e0415e9d0 2023-12-19T18:15:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.481+0000 7f2e09ba3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2e04151c90 0x7f2e0406b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46276/0 (socket says 172.21.15.27:46276) 2023-12-19T18:15:56.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.481+0000 7f2e09ba3640 1 -- 172.21.15.27:0/1686444806 learned_addr learned my addr 172.21.15.27:0/1686444806 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.481+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 762690284 0 0) 0x7f2e0415b3e0 con 0x7f2e04151c90 2023-12-19T18:15:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.481+0000 7f2e093a2640 1 --2- 172.21.15.27:0/1686444806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e9d0 0x7f2e040cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.481+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2df0003520 con 0x7f2e04151c90 2023-12-19T18:15:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 920412994 0 0) 0x7f2df0003520 con 0x7f2e04151c90 2023-12-19T18:15:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e9d0 msgr2=0x7f2e040cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:56.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e08ba1640 1 --2- 172.21.15.27:0/1686444806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e9d0 0x7f2e040cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 >> v1:172.21.15.192:6789/0 conn(0x7f2e04152070 legacy=0x7f2e0415b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0415f490 con 0x7f2e04151c90 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f2e00001670 con 0x7f2e04151c90 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 >> v1:172.21.15.27:6789/0 conn(0x7f2e04151c90 legacy=0x7f2e0406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e08ba1640 1 --2- 172.21.15.27:0/1686444806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e9d0 0x7f2df00039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e08ba1640 1 --2- 172.21.15.27:0/1686444806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df0003f70 0x7f2df00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e08ba1640 1 --2- 172.21.15.27:0/1686444806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0004d00 0x7f2df0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0415f1b0 con 0x7f2df0004d00 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df000bb70 con 0x7f2df0003f70 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df000bd30 con 0x7f2e0415e9d0 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.482+0000 7f2e0a3a4640 1 --2- 172.21.15.27:0/1686444806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0004d00 0x7f2df0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.483+0000 7f2e093a2640 1 --2- 172.21.15.27:0/1686444806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df0003f70 0x7f2df00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.483+0000 7f2e09ba3640 1 --2- 172.21.15.27:0/1686444806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e9d0 0x7f2df00039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.483+0000 7f2e0a3a4640 1 -- 172.21.15.27:0/1686444806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e9d0 msgr2=0x7f2df00039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.483+0000 7f2e0a3a4640 1 --2- 172.21.15.27:0/1686444806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e9d0 0x7f2df00039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.483+0000 7f2e0a3a4640 1 -- 172.21.15.27:0/1686444806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df0003f70 msgr2=0x7f2df00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.483+0000 7f2e0a3a4640 1 --2- 172.21.15.27:0/1686444806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df0003f70 0x7f2df00047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.483+0000 7f2e0a3a4640 1 -- 172.21.15.27:0/1686444806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df0003520 con 0x7f2df0004d00 2023-12-19T18:15:56.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.484+0000 7f2e0a3a4640 1 --2- 172.21.15.27:0/1686444806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0004d00 0x7f2df0005180 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc001e50 tx=0x7f2df00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcdbb2e9846390 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:56.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.484+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc003b70 con 0x7f2df0004d00 2023-12-19T18:15:56.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.484+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dfc003cd0 con 0x7f2df0004d00 2023-12-19T18:15:56.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.484+0000 7f2e08ba1640 1 -- 172.21.15.27:0/1686444806 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc005310 con 0x7f2df0004d00 2023-12-19T18:15:56.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.485+0000 7f2e0aba5640 1 -- 172.21.15.27:0/1686444806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0004d00 msgr2=0x7f2df0005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.485+0000 7f2e0aba5640 1 --2- 172.21.15.27:0/1686444806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df0004d00 0x7f2df0005180 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc001e50 tx=0x7f2df00057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.485+0000 7f2e0a3a4640 1 -- 172.21.15.27:0/1686444806 reap_dead start 2023-12-19T18:15:56.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.485+0000 7f2e0aba5640 1 -- 172.21.15.27:0/1686444806 shutdown_connections 2023-12-19T18:15:56.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.485+0000 7f2e0aba5640 1 -- 172.21.15.27:0/1686444806 >> 172.21.15.27:0/1686444806 conn(0x7f2e0406a060 msgr2=0x7f2e0406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:56.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.485+0000 7f2e0aba5640 1 -- 172.21.15.27:0/1686444806 shutdown_connections 2023-12-19T18:15:56.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.485+0000 7f2e0aba5640 1 -- 172.21.15.27:0/1686444806 wait complete. 2023-12-19T18:15:56.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.487+0000 7f2e0aba5640 1 Processor -- start 2023-12-19T18:15:56.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.487+0000 7f2e0aba5640 1 -- start start 2023-12-19T18:15:56.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.488+0000 7f2e0aba5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df0004d00 0x7f2e041f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.488+0000 7f2e0aba5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e280 0x7f2e041f53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.488+0000 7f2e0aba5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e040c47e0 0x7f2e040c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.488+0000 7f2e0aba5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e040c5310 con 0x7f2e040c47e0 2023-12-19T18:15:56.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.488+0000 7f2e0aba5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e040c5030 con 0x7f2df0004d00 2023-12-19T18:15:56.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.488+0000 7f2e0aba5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e0415b8d0 con 0x7f2e0415e280 2023-12-19T18:15:56.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.488+0000 7f2e0a3a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e040c47e0 0x7f2e040c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.489+0000 7f2e0a3a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e040c47e0 0x7f2e040c5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48572/0 (socket says 172.21.15.27:48572) 2023-12-19T18:15:56.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.489+0000 7f2e0a3a4640 1 -- 172.21.15.27:0/909020806 learned_addr learned my addr 172.21.15.27:0/909020806 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:56.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.489+0000 7f2e09ba3640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df0004d00 0x7f2e041f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.489+0000 7f2e093a2640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e280 0x7f2e041f53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.489+0000 7f2e0a3a4640 1 -- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e280 msgr2=0x7f2e041f53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.489+0000 7f2e0a3a4640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e280 0x7f2e041f53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.489+0000 7f2e0a3a4640 1 -- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df0004d00 msgr2=0x7f2e041f3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.489+0000 7f2e0a3a4640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df0004d00 0x7f2e041f3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.489+0000 7f2e0a3a4640 1 -- 172.21.15.27:0/909020806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e0415bbb0 con 0x7f2e040c47e0 2023-12-19T18:15:56.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.489+0000 7f2e09ba3640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df0004d00 0x7f2e041f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:56.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.489+0000 7f2e093a2640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e280 0x7f2e041f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:56.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.489+0000 7f2e0a3a4640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e040c47e0 0x7f2e040c5a80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc005040 tx=0x7f2dfc0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:56.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.490+0000 7f2dfa7fc640 1 -- 172.21.15.27:0/909020806 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc03e070 con 0x7f2e040c47e0 2023-12-19T18:15:56.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.490+0000 7f2dfa7fc640 1 -- 172.21.15.27:0/909020806 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dfc006b50 con 0x7f2e040c47e0 2023-12-19T18:15:56.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.490+0000 7f2e0aba5640 1 -- 172.21.15.27:0/909020806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e0415be90 con 0x7f2e040c47e0 2023-12-19T18:15:56.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.490+0000 7f2dfa7fc640 1 -- 172.21.15.27:0/909020806 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc007030 con 0x7f2e040c47e0 2023-12-19T18:15:56.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.490+0000 7f2e0aba5640 1 -- 172.21.15.27:0/909020806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e040c5fc0 con 0x7f2e040c47e0 2023-12-19T18:15:56.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.491+0000 7f2e0aba5640 1 -- 172.21.15.27:0/909020806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e04066f60 con 0x7f2e040c47e0 2023-12-19T18:15:56.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.492+0000 7f2dfa7fc640 1 -- 172.21.15.27:0/909020806 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2dfc007190 con 0x7f2e040c47e0 2023-12-19T18:15:56.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.493+0000 7f2dfa7fc640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2de806a460 0x7f2de806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.493+0000 7f2dfa7fc640 1 -- 172.21.15.27:0/909020806 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2dfc0a9c10 con 0x7f2e040c47e0 2023-12-19T18:15:56.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.494+0000 7f2e09ba3640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2de806a460 0x7f2de806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.495+0000 7f2e09ba3640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2de806a460 0x7f2de806c920 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2e00001bc0 tx=0x7f2e00000ef0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:56.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.498+0000 7f2dfa7fc640 1 -- 172.21.15.27:0/909020806 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2dfc075280 con 0x7f2e040c47e0 2023-12-19T18:15:56.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.700+0000 7f2e0aba5640 1 -- 172.21.15.27:0/909020806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2e040c5310 con 0x7f2e040c47e0 2023-12-19T18:15:56.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.701+0000 7f2dfa7fc640 1 -- 172.21.15.27:0/909020806 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2dfc048020 con 0x7f2e040c47e0 2023-12-19T18:15:56.701 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:15:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.703+0000 7f2e0aba5640 1 -- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2de806a460 msgr2=0x7f2de806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.703+0000 7f2e0aba5640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2de806a460 0x7f2de806c920 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2e00001bc0 tx=0x7f2e00000ef0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.703+0000 7f2e0aba5640 1 -- 172.21.15.27:0/909020806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e040c47e0 msgr2=0x7f2e040c5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.703+0000 7f2e0aba5640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e040c47e0 0x7f2e040c5a80 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc005040 tx=0x7f2dfc0014a0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.704+0000 7f2e0aba5640 1 -- 172.21.15.27:0/909020806 shutdown_connections 2023-12-19T18:15:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.704+0000 7f2e0aba5640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2de806a460 0x7f2de806c920 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.704+0000 7f2e0aba5640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e040c47e0 0x7f2e040c5a80 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.704+0000 7f2e0aba5640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2e0415e280 0x7f2e041f53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.704+0000 7f2e0aba5640 1 --2- 172.21.15.27:0/909020806 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df0004d00 0x7f2e041f3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.704+0000 7f2e0aba5640 1 -- 172.21.15.27:0/909020806 >> 172.21.15.27:0/909020806 conn(0x7f2e0406a060 msgr2=0x7f2e041511d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.704+0000 7f2e0aba5640 1 -- 172.21.15.27:0/909020806 shutdown_connections 2023-12-19T18:15:56.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.704+0000 7f2e0aba5640 1 -- 172.21.15.27:0/909020806 wait complete. 2023-12-19T18:15:56.724 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:15:56.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.950+0000 7efd558fb640 1 Processor -- start 2023-12-19T18:15:56.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.950+0000 7efd558fb640 1 -- start start 2023-12-19T18:15:56.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.952+0000 7efd558fb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efd500c9750 0x7efd5015e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.952+0000 7efd558fb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd500ce0b0 con 0x7efd500ccfa0 2023-12-19T18:15:56.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.952+0000 7efd558fb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd500ce920 con 0x7efd500ac9d0 2023-12-19T18:15:56.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.952+0000 7efd558fb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd5015e9e0 con 0x7efd500c9750 2023-12-19T18:15:56.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.952+0000 7efd47fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efd500ccfa0 0x7efd500c9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46290/0 (socket says 172.21.15.27:46290) 2023-12-19T18:15:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.952+0000 7efd47fff640 1 -- 172.21.15.27:0/2278596641 learned_addr learned my addr 172.21.15.27:0/2278596641 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.952+0000 7efd4effd640 1 --2- 172.21.15.27:0/2278596641 >> v2:172.21.15.27:3300/0 conn(0x7efd500c9750 0x7efd5015e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.953+0000 7efd4effd640 1 -- 172.21.15.27:0/2278596641 >> v1:172.21.15.192:6789/0 conn(0x7efd500ac9d0 legacy=0x7efd500ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.953+0000 7efd4effd640 1 -- 172.21.15.27:0/2278596641 >> v1:172.21.15.27:6789/0 conn(0x7efd500ccfa0 legacy=0x7efd500c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.953+0000 7efd4effd640 1 -- 172.21.15.27:0/2278596641 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd500cd4a0 con 0x7efd500c9750 2023-12-19T18:15:56.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.954+0000 7efd4effd640 1 --2- 172.21.15.27:0/2278596641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd500c9750 0x7efd5015e4a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efd400097a0 tx=0x7efd4002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c899d932b6a0fe5a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:56.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.954+0000 7efd4e7fc640 1 -- 172.21.15.27:0/2278596641 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40031830 con 0x7efd500c9750 2023-12-19T18:15:56.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.954+0000 7efd4e7fc640 1 -- 172.21.15.27:0/2278596641 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd40031990 con 0x7efd500c9750 2023-12-19T18:15:56.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.954+0000 7efd4e7fc640 1 -- 172.21.15.27:0/2278596641 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd40031d20 con 0x7efd500c9750 2023-12-19T18:15:56.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.955+0000 7efd558fb640 1 -- 172.21.15.27:0/2278596641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd500c9750 msgr2=0x7efd5015e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.955+0000 7efd558fb640 1 --2- 172.21.15.27:0/2278596641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd500c9750 0x7efd5015e4a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7efd400097a0 tx=0x7efd4002b8d0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.955+0000 7efd558fb640 1 -- 172.21.15.27:0/2278596641 shutdown_connections 2023-12-19T18:15:56.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.955+0000 7efd558fb640 1 --2- 172.21.15.27:0/2278596641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd500c9750 0x7efd5015e4a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.955+0000 7efd558fb640 1 -- 172.21.15.27:0/2278596641 >> 172.21.15.27:0/2278596641 conn(0x7efd500cebd0 msgr2=0x7efd500cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:56.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.955+0000 7efd558fb640 1 -- 172.21.15.27:0/2278596641 shutdown_connections 2023-12-19T18:15:56.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.955+0000 7efd558fb640 1 -- 172.21.15.27:0/2278596641 wait complete. 2023-12-19T18:15:56.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.957+0000 7efd558fb640 1 Processor -- start 2023-12-19T18:15:56.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.957+0000 7efd558fb640 1 -- start start 2023-12-19T18:15:56.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.958+0000 7efd558fb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd500ac9d0 0x7efd500c3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.958+0000 7efd558fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd500c9750 0x7efd50149ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.958+0000 7efd558fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd500ccfa0 0x7efd500ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.958+0000 7efd558fb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd500c2780 con 0x7efd500ccfa0 2023-12-19T18:15:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.958+0000 7efd558fb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd500c2a60 con 0x7efd500ac9d0 2023-12-19T18:15:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.958+0000 7efd558fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd500c2d40 con 0x7efd500c9750 2023-12-19T18:15:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.958+0000 7efd4f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd500ccfa0 0x7efd500ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.958+0000 7efd4f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd500ccfa0 0x7efd500ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48592/0 (socket says 172.21.15.27:48592) 2023-12-19T18:15:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.959+0000 7efd4f7fe640 1 -- 172.21.15.27:0/3310761645 learned_addr learned my addr 172.21.15.27:0/3310761645 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.959+0000 7efd4effd640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd500ac9d0 0x7efd500c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.959+0000 7efd47fff640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd500c9750 0x7efd50149ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.959+0000 7efd4f7fe640 1 -- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd500c9750 msgr2=0x7efd50149ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.959+0000 7efd4f7fe640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd500c9750 0x7efd50149ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.959+0000 7efd4f7fe640 1 -- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd500ac9d0 msgr2=0x7efd500c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.959+0000 7efd4f7fe640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd500ac9d0 0x7efd500c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.959+0000 7efd4f7fe640 1 -- 172.21.15.27:0/3310761645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd40009400 con 0x7efd500ccfa0 2023-12-19T18:15:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.959+0000 7efd4f7fe640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd500ccfa0 0x7efd500ace00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efd4800c780 tx=0x7efd4800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.960+0000 7efd4cff9640 1 -- 172.21.15.27:0/3310761645 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd48004b60 con 0x7efd500ccfa0 2023-12-19T18:15:56.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.960+0000 7efd4cff9640 1 -- 172.21.15.27:0/3310761645 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd4800ce90 con 0x7efd500ccfa0 2023-12-19T18:15:56.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.960+0000 7efd4cff9640 1 -- 172.21.15.27:0/3310761645 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd4800a850 con 0x7efd500ccfa0 2023-12-19T18:15:56.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.960+0000 7efd558fb640 1 -- 172.21.15.27:0/3310761645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd500c3020 con 0x7efd500ccfa0 2023-12-19T18:15:56.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.960+0000 7efd558fb640 1 -- 172.21.15.27:0/3310761645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd5014a430 con 0x7efd500ccfa0 2023-12-19T18:15:56.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.961+0000 7efd558fb640 1 -- 172.21.15.27:0/3310761645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd500c3540 con 0x7efd500ccfa0 2023-12-19T18:15:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.962+0000 7efd4cff9640 1 -- 172.21.15.27:0/3310761645 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efd4800a9b0 con 0x7efd500ccfa0 2023-12-19T18:15:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.963+0000 7efd4cff9640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd3006a430 0x7efd3006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:56.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.964+0000 7efd4cff9640 1 -- 172.21.15.27:0/3310761645 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7efd48088b60 con 0x7efd500ccfa0 2023-12-19T18:15:56.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.964+0000 7efd4effd640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd3006a430 0x7efd3006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:56.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.966+0000 7efd4effd640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd3006a430 0x7efd3006c8f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efd40036000 tx=0x7efd4003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:56.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:56.969+0000 7efd4cff9640 1 -- 172.21.15.27:0/3310761645 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efd500c3540 con 0x7efd500ccfa0 2023-12-19T18:15:57.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.174+0000 7efd558fb640 1 -- 172.21.15.27:0/3310761645 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efd500c3540 con 0x7efd500ccfa0 2023-12-19T18:15:57.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.175+0000 7efd4cff9640 1 -- 172.21.15.27:0/3310761645 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7efd500c3540 con 0x7efd500ccfa0 2023-12-19T18:15:57.175 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:15:57.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.177+0000 7efd558fb640 1 -- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd3006a430 msgr2=0x7efd3006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:57.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.177+0000 7efd558fb640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd3006a430 0x7efd3006c8f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7efd40036000 tx=0x7efd4003c040 comp rx=0 tx=0).stop 2023-12-19T18:15:57.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.177+0000 7efd558fb640 1 -- 172.21.15.27:0/3310761645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd500ccfa0 msgr2=0x7efd500ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:57.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.177+0000 7efd558fb640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd500ccfa0 0x7efd500ace00 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7efd4800c780 tx=0x7efd4800cc40 comp rx=0 tx=0).stop 2023-12-19T18:15:57.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.178+0000 7efd558fb640 1 -- 172.21.15.27:0/3310761645 shutdown_connections 2023-12-19T18:15:57.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.178+0000 7efd558fb640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd3006a430 0x7efd3006c8f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:57.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.178+0000 7efd558fb640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd500ccfa0 0x7efd500ace00 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:57.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.178+0000 7efd558fb640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd500c9750 0x7efd50149ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:57.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.178+0000 7efd558fb640 1 --2- 172.21.15.27:0/3310761645 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd500ac9d0 0x7efd500c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:57.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.178+0000 7efd558fb640 1 -- 172.21.15.27:0/3310761645 >> 172.21.15.27:0/3310761645 conn(0x7efd500cebd0 msgr2=0x7efd500c9b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:57.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.178+0000 7efd558fb640 1 -- 172.21.15.27:0/3310761645 shutdown_connections 2023-12-19T18:15:57.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.178+0000 7efd558fb640 1 -- 172.21.15.27:0/3310761645 wait complete. 2023-12-19T18:15:57.196 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:15:57.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.430+0000 7f43306ec640 1 Processor -- start 2023-12-19T18:15:57.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.430+0000 7f43306ec640 1 -- start start 2023-12-19T18:15:57.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.432+0000 7f43306ec640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f432806fcf0 0x7f43280cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:57.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.432+0000 7f43306ec640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4328070160 con 0x7f43280cff30 2023-12-19T18:15:57.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.432+0000 7f43306ec640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f432815c200 con 0x7f43280b3210 2023-12-19T18:15:57.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.432+0000 7f43306ec640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43280cb6e0 con 0x7f432806fcf0 2023-12-19T18:15:57.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.432+0000 7f432e488640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f43280cff30 0x7f43280d0310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46302/0 (socket says 172.21.15.27:46302) 2023-12-19T18:15:57.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.432+0000 7f432e488640 1 -- 172.21.15.27:0/3871167395 learned_addr learned my addr 172.21.15.27:0/3871167395 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:57.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.432+0000 7f432ec89640 1 --2- 172.21.15.27:0/3871167395 >> v2:172.21.15.192:3300/0 conn(0x7f432806fcf0 0x7f43280cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:57.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.433+0000 7f432d486640 1 -- 172.21.15.27:0/3871167395 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756626405 0 0) 0x7f4328070160 con 0x7f43280cff30 2023-12-19T18:15:57.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.433+0000 7f432d486640 1 -- 172.21.15.27:0/3871167395 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4310003520 con 0x7f43280cff30 2023-12-19T18:15:57.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.433+0000 7f432d486640 1 -- 172.21.15.27:0/3871167395 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342987143 0 0) 0x7f4310003520 con 0x7f43280cff30 2023-12-19T18:15:57.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.433+0000 7f432ec89640 1 -- 172.21.15.27:0/3871167395 >> v1:172.21.15.192:6789/0 conn(0x7f43280b3210 legacy=0x7f43280cccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:57.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.433+0000 7f432ec89640 1 -- 172.21.15.27:0/3871167395 >> v1:172.21.15.27:6789/0 conn(0x7f43280cff30 legacy=0x7f43280d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:57.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.433+0000 7f432ec89640 1 -- 172.21.15.27:0/3871167395 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432815b610 con 0x7f432806fcf0 2023-12-19T18:15:57.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.434+0000 7f432ec89640 1 --2- 172.21.15.27:0/3871167395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f432806fcf0 0x7f43280cd3d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f43200097d0 tx=0x7f432002b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1406f9a75bc5371d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:57.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.435+0000 7f432d486640 1 -- 172.21.15.27:0/3871167395 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43180019f0 con 0x7f432806fcf0 2023-12-19T18:15:57.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.435+0000 7f432d486640 1 -- 172.21.15.27:0/3871167395 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432002e050 con 0x7f432806fcf0 2023-12-19T18:15:57.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.435+0000 7f432d486640 1 -- 172.21.15.27:0/3871167395 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4320030730 con 0x7f432806fcf0 2023-12-19T18:15:57.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.435+0000 7f43306ec640 1 -- 172.21.15.27:0/3871167395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f432806fcf0 msgr2=0x7f43280cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:57.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.435+0000 7f43306ec640 1 --2- 172.21.15.27:0/3871167395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f432806fcf0 0x7f43280cd3d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f43200097d0 tx=0x7f432002b710 comp rx=0 tx=0).stop 2023-12-19T18:15:57.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.436+0000 7f43306ec640 1 -- 172.21.15.27:0/3871167395 shutdown_connections 2023-12-19T18:15:57.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.436+0000 7f43306ec640 1 --2- 172.21.15.27:0/3871167395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f432806fcf0 0x7f43280cd3d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:57.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.436+0000 7f43306ec640 1 -- 172.21.15.27:0/3871167395 >> 172.21.15.27:0/3871167395 conn(0x7f432806f1d0 msgr2=0x7f432806f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:57.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.436+0000 7f43306ec640 1 -- 172.21.15.27:0/3871167395 shutdown_connections 2023-12-19T18:15:57.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.436+0000 7f43306ec640 1 -- 172.21.15.27:0/3871167395 wait complete. 2023-12-19T18:15:57.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.438+0000 7f43306ec640 1 Processor -- start 2023-12-19T18:15:57.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.438+0000 7f43306ec640 1 -- start start 2023-12-19T18:15:57.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.439+0000 7f43306ec640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f432806fcf0 0x7f4328156fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:57.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.439+0000 7f43306ec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43280b3210 0x7f43281513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:57.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.439+0000 7f43306ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43280cff30 0x7f43280cec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:57.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.439+0000 7f43306ec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43281518f0 con 0x7f43280cff30 2023-12-19T18:15:57.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.439+0000 7f43306ec640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4328157630 con 0x7f432806fcf0 2023-12-19T18:15:57.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.439+0000 7f43306ec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43280b08f0 con 0x7f43280b3210 2023-12-19T18:15:57.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.439+0000 7f432ec89640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43280cff30 0x7f43280cec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:57.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.439+0000 7f432ec89640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43280cff30 0x7f43280cec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48604/0 (socket says 172.21.15.27:48604) 2023-12-19T18:15:57.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.439+0000 7f432ec89640 1 -- 172.21.15.27:0/1460712602 learned_addr learned my addr 172.21.15.27:0/1460712602 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.440+0000 7f432e488640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f432806fcf0 0x7f4328156fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.440+0000 7f432dc87640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43280b3210 0x7f43281513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.440+0000 7f432ec89640 1 -- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43280b3210 msgr2=0x7f43281513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.440+0000 7f432ec89640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43280b3210 0x7f43281513b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.440+0000 7f432ec89640 1 -- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f432806fcf0 msgr2=0x7f4328156fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.440+0000 7f432ec89640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f432806fcf0 0x7f4328156fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.440+0000 7f432ec89640 1 -- 172.21.15.27:0/1460712602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4320009400 con 0x7f43280cff30 2023-12-19T18:15:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.440+0000 7f432ec89640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43280cff30 0x7f43280cec80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4320035040 tx=0x7f43200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.440+0000 7f4326ffd640 1 -- 172.21.15.27:0/1460712602 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432003c880 con 0x7f43280cff30 2023-12-19T18:15:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.441+0000 7f4326ffd640 1 -- 172.21.15.27:0/1460712602 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432003ca00 con 0x7f43280cff30 2023-12-19T18:15:57.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.441+0000 7f4326ffd640 1 -- 172.21.15.27:0/1460712602 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432003cdb0 con 0x7f43280cff30 2023-12-19T18:15:57.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.441+0000 7f43306ec640 1 -- 172.21.15.27:0/1460712602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43280b0bd0 con 0x7f43280cff30 2023-12-19T18:15:57.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.441+0000 7f43306ec640 1 -- 172.21.15.27:0/1460712602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43280b1190 con 0x7f43280cff30 2023-12-19T18:15:57.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.442+0000 7f43306ec640 1 -- 172.21.15.27:0/1460712602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43280cb1b0 con 0x7f43280cff30 2023-12-19T18:15:57.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.443+0000 7f4326ffd640 1 -- 172.21.15.27:0/1460712602 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4320039040 con 0x7f43280cff30 2023-12-19T18:15:57.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.444+0000 7f4326ffd640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f431406a430 0x7f431406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:57.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.445+0000 7f4326ffd640 1 -- 172.21.15.27:0/1460712602 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f43200aa760 con 0x7f43280cff30 2023-12-19T18:15:57.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.445+0000 7f432e488640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f431406a430 0x7f431406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:57.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.446+0000 7f432e488640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f431406a430 0x7f431406c8f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4318001b20 tx=0x7f4318009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:57.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.449+0000 7f4326ffd640 1 -- 172.21.15.27:0/1460712602 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4320075db0 con 0x7f43280cff30 2023-12-19T18:15:57.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.659+0000 7f43306ec640 1 -- 172.21.15.27:0/1460712602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4328067110 con 0x7f43280cff30 2023-12-19T18:15:57.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.660+0000 7f4326ffd640 1 -- 172.21.15.27:0/1460712602 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4320037080 con 0x7f43280cff30 2023-12-19T18:15:57.660 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:15:57.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.663+0000 7f43306ec640 1 -- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f431406a430 msgr2=0x7f431406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:57.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.663+0000 7f43306ec640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f431406a430 0x7f431406c8f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4318001b20 tx=0x7f4318009040 comp rx=0 tx=0).stop 2023-12-19T18:15:57.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.663+0000 7f43306ec640 1 -- 172.21.15.27:0/1460712602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43280cff30 msgr2=0x7f43280cec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:57.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.663+0000 7f43306ec640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43280cff30 0x7f43280cec80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f4320035040 tx=0x7f43200046c0 comp rx=0 tx=0).stop 2023-12-19T18:15:57.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.663+0000 7f43306ec640 1 -- 172.21.15.27:0/1460712602 shutdown_connections 2023-12-19T18:15:57.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.663+0000 7f43306ec640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f431406a430 0x7f431406c8f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:57.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.663+0000 7f43306ec640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43280cff30 0x7f43280cec80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:57.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.663+0000 7f43306ec640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43280b3210 0x7f43281513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:57.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.663+0000 7f43306ec640 1 --2- 172.21.15.27:0/1460712602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f432806fcf0 0x7f4328156fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:57.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.663+0000 7f43306ec640 1 -- 172.21.15.27:0/1460712602 >> 172.21.15.27:0/1460712602 conn(0x7f432806f1d0 msgr2=0x7f432806c700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:57.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.663+0000 7f43306ec640 1 -- 172.21.15.27:0/1460712602 shutdown_connections 2023-12-19T18:15:57.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:57.663+0000 7f43306ec640 1 -- 172.21.15.27:0/1460712602 wait complete. 2023-12-19T18:15:57.681 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-19T18:15:57.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:57.682 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-19T18:15:57.788 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:15:57.788 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:15:57.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:15:57.788 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-19T18:15:57.852 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:15:57.852 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:15:57.918 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:15:57.918 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:15:57.977 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:15:58.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.207+0000 7f186e93e640 1 Processor -- start 2023-12-19T18:15:58.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.207+0000 7f186e93e640 1 -- start start 2023-12-19T18:15:58.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.209+0000 7f186e93e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18680cbce0 0x7f18680ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.209+0000 7f186e93e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f18680adc20 0x7f186806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.209+0000 7f186e93e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18680cb850 con 0x7f186815af70 2023-12-19T18:15:58.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.209+0000 7f186e93e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186815b350 con 0x7f18680adc20 2023-12-19T18:15:58.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.209+0000 7f186e93e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18680cac40 con 0x7f18680cbce0 2023-12-19T18:15:58.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.210+0000 7f1867fff640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f186815af70 0x7f18680ad510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:40696/0 (socket says 172.21.15.27:40696) 2023-12-19T18:15:58.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.210+0000 7f1867fff640 1 -- 172.21.15.27:0/103909289 learned_addr learned my addr 172.21.15.27:0/103909289 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:58.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.210+0000 7f186cedb640 1 --2- 172.21.15.27:0/103909289 >> v2:172.21.15.192:3300/0 conn(0x7f18680adc20 0x7f186806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.210+0000 7f18677fe640 1 --2- 172.21.15.27:0/103909289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18680cbce0 0x7f18680ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.210+0000 7f1866ffd640 1 -- 172.21.15.27:0/103909289 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257310700 0 0) 0x7f18680cb850 con 0x7f186815af70 2023-12-19T18:15:58.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.210+0000 7f1866ffd640 1 -- 172.21.15.27:0/103909289 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1850003520 con 0x7f186815af70 2023-12-19T18:15:58.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.211+0000 7f18677fe640 1 -- 172.21.15.27:0/103909289 >> v2:172.21.15.192:3300/0 conn(0x7f18680adc20 msgr2=0x7f186806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:58.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.211+0000 7f18677fe640 1 --2- 172.21.15.27:0/103909289 >> v2:172.21.15.192:3300/0 conn(0x7f18680adc20 0x7f186806b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.211+0000 7f18677fe640 1 -- 172.21.15.27:0/103909289 >> v1:172.21.15.192:6789/0 conn(0x7f186815af70 legacy=0x7f18680ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:58.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.211+0000 7f18677fe640 1 -- 172.21.15.27:0/103909289 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18680caf20 con 0x7f18680cbce0 2023-12-19T18:15:58.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.211+0000 7f186cedb640 1 --2- 172.21.15.27:0/103909289 >> v2:172.21.15.192:3300/0 conn(0x7f18680adc20 0x7f186806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:15:58.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.212+0000 7f18677fe640 1 --2- 172.21.15.27:0/103909289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18680cbce0 0x7f18680ca540 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f185c0097a0 tx=0x7f185c02b8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35c4b97735c67d7f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:58.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.213+0000 7f1866ffd640 1 -- 172.21.15.27:0/103909289 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c031830 con 0x7f18680cbce0 2023-12-19T18:15:58.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.213+0000 7f1866ffd640 1 -- 172.21.15.27:0/103909289 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c031990 con 0x7f18680cbce0 2023-12-19T18:15:58.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.213+0000 7f1866ffd640 1 -- 172.21.15.27:0/103909289 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c031d00 con 0x7f18680cbce0 2023-12-19T18:15:58.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.213+0000 7f186e93e640 1 -- 172.21.15.27:0/103909289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18680cbce0 msgr2=0x7f18680ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:58.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.213+0000 7f186e93e640 1 --2- 172.21.15.27:0/103909289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18680cbce0 0x7f18680ca540 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f185c0097a0 tx=0x7f185c02b8d0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.213+0000 7f186e93e640 1 -- 172.21.15.27:0/103909289 shutdown_connections 2023-12-19T18:15:58.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.213+0000 7f186e93e640 1 --2- 172.21.15.27:0/103909289 >> v2:172.21.15.192:3300/0 conn(0x7f18680adc20 0x7f186806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.213+0000 7f186e93e640 1 --2- 172.21.15.27:0/103909289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18680cbce0 0x7f18680ca540 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.213+0000 7f186e93e640 1 -- 172.21.15.27:0/103909289 >> 172.21.15.27:0/103909289 conn(0x7f186806a330 msgr2=0x7f18680b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:58.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.213+0000 7f186e93e640 1 -- 172.21.15.27:0/103909289 shutdown_connections 2023-12-19T18:15:58.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.214+0000 7f186e93e640 1 -- 172.21.15.27:0/103909289 wait complete. 2023-12-19T18:15:58.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.215+0000 7f186e93e640 1 Processor -- start 2023-12-19T18:15:58.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.216+0000 7f186e93e640 1 -- start start 2023-12-19T18:15:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.217+0000 7f186e93e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18680adc20 0x7f186815cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.217+0000 7f186e93e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18680cbce0 0x7f186815dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.217+0000 7f186e93e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f186815af70 0x7f18681f1c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.217+0000 7f186e93e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186815c640 con 0x7f18680cbce0 2023-12-19T18:15:58.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.217+0000 7f186e93e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186815d010 con 0x7f18680adc20 2023-12-19T18:15:58.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.217+0000 7f186e93e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186815ba30 con 0x7f186815af70 2023-12-19T18:15:58.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.217+0000 7f18677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18680cbce0 0x7f186815dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.217+0000 7f18677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18680cbce0 0x7f186815dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48612/0 (socket says 172.21.15.27:48612) 2023-12-19T18:15:58.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.217+0000 7f18677fe640 1 -- 172.21.15.27:0/3533446581 learned_addr learned my addr 172.21.15.27:0/3533446581 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:58.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.217+0000 7f1867fff640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18680adc20 0x7f186815cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.217+0000 7f186cedb640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f186815af70 0x7f18681f1c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.218+0000 7f18677fe640 1 -- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f186815af70 msgr2=0x7f18681f1c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.218+0000 7f18677fe640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f186815af70 0x7f18681f1c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.218+0000 7f18677fe640 1 -- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18680adc20 msgr2=0x7f186815cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.218+0000 7f18677fe640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18680adc20 0x7f186815cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.218+0000 7f18677fe640 1 -- 172.21.15.27:0/3533446581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1860009400 con 0x7f18680cbce0 2023-12-19T18:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.218+0000 7f18677fe640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18680cbce0 0x7f186815dd30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f185c0098c0 tx=0x7f185c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.218+0000 7f1864ff9640 1 -- 172.21.15.27:0/3533446581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c03a630 con 0x7f18680cbce0 2023-12-19T18:15:58.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.218+0000 7f1864ff9640 1 -- 172.21.15.27:0/3533446581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c03a790 con 0x7f18680cbce0 2023-12-19T18:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.218+0000 7f1864ff9640 1 -- 172.21.15.27:0/3533446581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c03ab40 con 0x7f18680cbce0 2023-12-19T18:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.219+0000 7f186e93e640 1 -- 172.21.15.27:0/3533446581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185c009400 con 0x7f18680cbce0 2023-12-19T18:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.219+0000 7f186e93e640 1 -- 172.21.15.27:0/3533446581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186815bff0 con 0x7f18680cbce0 2023-12-19T18:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.220+0000 7f186e93e640 1 -- 172.21.15.27:0/3533446581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1868068e90 con 0x7f18680cbce0 2023-12-19T18:15:58.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.221+0000 7f1864ff9640 1 -- 172.21.15.27:0/3533446581 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f185c03aca0 con 0x7f18680cbce0 2023-12-19T18:15:58.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.222+0000 7f1864ff9640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f185406a430 0x7f185406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.222+0000 7f1864ff9640 1 -- 172.21.15.27:0/3533446581 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f185c0aa100 con 0x7f18680cbce0 2023-12-19T18:15:58.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.222+0000 7f1867fff640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f185406a430 0x7f185406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.224+0000 7f1867fff640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f185406a430 0x7f185406c8f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1858005c00 tx=0x7f1858009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:58.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.226+0000 7f1864ff9640 1 -- 172.21.15.27:0/3533446581 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f185c0756e0 con 0x7f18680cbce0 2023-12-19T18:15:58.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.424+0000 7f186e93e640 1 -- 172.21.15.27:0/3533446581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f18680cb4b0 con 0x7f18680cbce0 2023-12-19T18:15:58.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.427+0000 7f1864ff9640 1 -- 172.21.15.27:0/3533446581 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16090 (secure 0 0 0) 0x7f185c07de60 con 0x7f18680cbce0 2023-12-19T18:15:58.428 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:15:58.428 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:15:43.580746+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:15:58.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.430+0000 7f186e93e640 1 -- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f185406a430 msgr2=0x7f185406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:58.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.430+0000 7f186e93e640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f185406a430 0x7f185406c8f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f1858005c00 tx=0x7f1858009040 comp rx=0 tx=0).stop 2023-12-19T18:15:58.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.430+0000 7f186e93e640 1 -- 172.21.15.27:0/3533446581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18680cbce0 msgr2=0x7f186815dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:58.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.430+0000 7f186e93e640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18680cbce0 0x7f186815dd30 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f185c0098c0 tx=0x7f185c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.430+0000 7f186e93e640 1 -- 172.21.15.27:0/3533446581 shutdown_connections 2023-12-19T18:15:58.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.430+0000 7f186e93e640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f185406a430 0x7f185406c8f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.430+0000 7f186e93e640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f186815af70 0x7f18681f1c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.430+0000 7f186e93e640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18680cbce0 0x7f186815dd30 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.430+0000 7f186e93e640 1 --2- 172.21.15.27:0/3533446581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18680adc20 0x7f186815cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.430+0000 7f186e93e640 1 -- 172.21.15.27:0/3533446581 >> 172.21.15.27:0/3533446581 conn(0x7f186806a330 msgr2=0x7f1868158fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:58.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.431+0000 7f186e93e640 1 -- 172.21.15.27:0/3533446581 shutdown_connections 2023-12-19T18:15:58.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.431+0000 7f186e93e640 1 -- 172.21.15.27:0/3533446581 wait complete. 2023-12-19T18:15:58.451 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:15:58.451 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:15:58.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.677+0000 7f7e12173640 1 Processor -- start 2023-12-19T18:15:58.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.677+0000 7f7e12173640 1 -- start start 2023-12-19T18:15:58.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.678+0000 7f7e12173640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e0c0b5160 0x7f7e0c0b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.678+0000 7f7e12173640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0c15d750 con 0x7f7e0c15d370 2023-12-19T18:15:58.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.679+0000 7f7e12173640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0c0d1750 con 0x7f7e0c0b55f0 2023-12-19T18:15:58.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.679+0000 7f7e12173640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c0b2880 con 0x7f7e0c0b5160 2023-12-19T18:15:58.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.679+0000 7f7e0b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7e0c15d370 0x7f7e0c0b4a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46304/0 (socket says 172.21.15.27:46304) 2023-12-19T18:15:58.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.679+0000 7f7e0b7fe640 1 -- 172.21.15.27:0/1309306304 learned_addr learned my addr 172.21.15.27:0/1309306304 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:58.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.679+0000 7f7e0affd640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e0c0b5160 0x7f7e0c0b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.679+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420452765 0 0) 0x7f7e0c15d750 con 0x7f7e0c15d370 2023-12-19T18:15:58.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.679+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7df8003520 con 0x7f7e0c15d370 2023-12-19T18:15:58.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2381760598 0 0) 0x7f7df8003520 con 0x7f7e0c15d370 2023-12-19T18:15:58.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e0c0b5160 msgr2=0x7f7e0c0b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:58.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0a7fc640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e0c0b5160 0x7f7e0c0b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 >> v1:172.21.15.192:6789/0 conn(0x7f7e0c0b55f0 legacy=0x7f7e0c0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c0d1a30 con 0x7f7e0c15d370 2023-12-19T18:15:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f7dfc0019f0 con 0x7f7e0c15d370 2023-12-19T18:15:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 >> v1:172.21.15.27:6789/0 conn(0x7f7e0c15d370 legacy=0x7f7e0c0b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0a7fc640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df80039a0 0x7f7df8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0a7fc640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df80047c0 0x7f7df8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0a7fc640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e0c0b5160 0x7f7df8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df800bac0 con 0x7f7df80047c0 2023-12-19T18:15:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7df800bc30 con 0x7f7df80039a0 2023-12-19T18:15:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7df800bdf0 con 0x7f7e0c0b5160 2023-12-19T18:15:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.680+0000 7f7e0affd640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df80047c0 0x7f7df8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.681+0000 7f7e0b7fe640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df80039a0 0x7f7df8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.681+0000 7f7e0bfff640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e0c0b5160 0x7f7df8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.681+0000 7f7e0affd640 1 -- 172.21.15.27:0/1309306304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e0c0b5160 msgr2=0x7f7df8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:58.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.681+0000 7f7e0affd640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e0c0b5160 0x7f7df8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.681+0000 7f7e0affd640 1 -- 172.21.15.27:0/1309306304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df80039a0 msgr2=0x7f7df8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:58.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.681+0000 7f7e0affd640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df80039a0 0x7f7df8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.681+0000 7f7e0affd640 1 -- 172.21.15.27:0/1309306304 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7df8003520 con 0x7f7df80047c0 2023-12-19T18:15:58.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.681+0000 7f7e0b7fe640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7df80039a0 0x7f7df8003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:58.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.681+0000 7f7e0bfff640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7e0c0b5160 0x7f7df8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:58.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.682+0000 7f7e0affd640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df80047c0 0x7f7df8004c40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009ac0 tx=0x7f7df80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb6cc13754531494 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:58.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.682+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df402c6b0 con 0x7f7df80047c0 2023-12-19T18:15:58.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.682+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df402c810 con 0x7f7df80047c0 2023-12-19T18:15:58.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.682+0000 7f7e0a7fc640 1 -- 172.21.15.27:0/1309306304 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df402cba0 con 0x7f7df80047c0 2023-12-19T18:15:58.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.683+0000 7f7e12173640 1 -- 172.21.15.27:0/1309306304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df80047c0 msgr2=0x7f7df8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:58.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.683+0000 7f7e12173640 1 --2- 172.21.15.27:0/1309306304 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7df80047c0 0x7f7df8004c40 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7df4009ac0 tx=0x7f7df80057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.683+0000 7f7e0bfff640 1 -- 172.21.15.27:0/1309306304 reap_dead start 2023-12-19T18:15:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.683+0000 7f7e12173640 1 -- 172.21.15.27:0/1309306304 shutdown_connections 2023-12-19T18:15:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.683+0000 7f7e12173640 1 -- 172.21.15.27:0/1309306304 >> 172.21.15.27:0/1309306304 conn(0x7f7e0c06a480 msgr2=0x7f7e0c06a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.683+0000 7f7e12173640 1 -- 172.21.15.27:0/1309306304 shutdown_connections 2023-12-19T18:15:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.683+0000 7f7e12173640 1 -- 172.21.15.27:0/1309306304 wait complete. 2023-12-19T18:15:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.685+0000 7f7e12173640 1 Processor -- start 2023-12-19T18:15:58.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.685+0000 7f7e12173640 1 -- start start 2023-12-19T18:15:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.686+0000 7f7e12173640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7df80047c0 0x7f7e0c1505d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.686+0000 7f7e12173640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e0c155ac0 0x7f7e0c15eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.686+0000 7f7e12173640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e0c15f0e0 0x7f7e0c0b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.686+0000 7f7e12173640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c0b4ff0 con 0x7f7e0c155ac0 2023-12-19T18:15:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.686+0000 7f7e12173640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c15f570 con 0x7f7e0c15f0e0 2023-12-19T18:15:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.686+0000 7f7e12173640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c0b4a30 con 0x7f7df80047c0 2023-12-19T18:15:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.686+0000 7f7e0affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e0c155ac0 0x7f7e0c15eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.686+0000 7f7e0affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e0c155ac0 0x7f7e0c15eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48628/0 (socket says 172.21.15.27:48628) 2023-12-19T18:15:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.686+0000 7f7e0affd640 1 -- 172.21.15.27:0/2951178506 learned_addr learned my addr 172.21.15.27:0/2951178506 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.687+0000 7f7e0b7fe640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7df80047c0 0x7f7e0c1505d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.687+0000 7f7e0bfff640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e0c15f0e0 0x7f7e0c0b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.687+0000 7f7e0affd640 1 -- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7df80047c0 msgr2=0x7f7e0c1505d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.687+0000 7f7e0affd640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7df80047c0 0x7f7e0c1505d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.687+0000 7f7e0affd640 1 -- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e0c15f0e0 msgr2=0x7f7e0c0b3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.687+0000 7f7e0affd640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e0c15f0e0 0x7f7e0c0b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.687+0000 7f7e0affd640 1 -- 172.21.15.27:0/2951178506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c0b4d10 con 0x7f7e0c155ac0 2023-12-19T18:15:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.687+0000 7f7e0affd640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e0c155ac0 0x7f7e0c15eba0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7df402c250 tx=0x7f7df4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:58.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.688+0000 7f7e03fff640 1 -- 172.21.15.27:0/2951178506 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df4003810 con 0x7f7e0c155ac0 2023-12-19T18:15:58.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.688+0000 7f7e03fff640 1 -- 172.21.15.27:0/2951178506 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df4003970 con 0x7f7e0c155ac0 2023-12-19T18:15:58.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.688+0000 7f7e03fff640 1 -- 172.21.15.27:0/2951178506 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df4003ce0 con 0x7f7e0c155ac0 2023-12-19T18:15:58.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.688+0000 7f7e12173640 1 -- 172.21.15.27:0/2951178506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0c0b2140 con 0x7f7e0c155ac0 2023-12-19T18:15:58.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.688+0000 7f7e12173640 1 -- 172.21.15.27:0/2951178506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c0b4240 con 0x7f7e0c155ac0 2023-12-19T18:15:58.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.689+0000 7f7e12173640 1 -- 172.21.15.27:0/2951178506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c0b1f80 con 0x7f7e0c155ac0 2023-12-19T18:15:58.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.692+0000 7f7e03fff640 1 -- 172.21.15.27:0/2951178506 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7df403e400 con 0x7f7e0c155ac0 2023-12-19T18:15:58.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.693+0000 7f7e03fff640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7dec06a430 0x7f7dec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:58.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.693+0000 7f7e03fff640 1 -- 172.21.15.27:0/2951178506 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7df40aadd0 con 0x7f7e0c155ac0 2023-12-19T18:15:58.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.694+0000 7f7e0b7fe640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7dec06a430 0x7f7dec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:58.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.695+0000 7f7e0b7fe640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7dec06a430 0x7f7dec06c8f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc002230 tx=0x7f7dfc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:58.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.696+0000 7f7e03fff640 1 -- 172.21.15.27:0/2951178506 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7df4076430 con 0x7f7e0c155ac0 2023-12-19T18:15:58.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.892+0000 7f7e12173640 1 -- 172.21.15.27:0/2951178506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7e0c068e90 con 0x7f7e0c155ac0 2023-12-19T18:15:58.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.893+0000 7f7e03fff640 1 -- 172.21.15.27:0/2951178506 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7df40044e0 con 0x7f7e0c155ac0 2023-12-19T18:15:58.894 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:15:58.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.896+0000 7f7e12173640 1 -- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7dec06a430 msgr2=0x7f7dec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:58.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.896+0000 7f7e12173640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7dec06a430 0x7f7dec06c8f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc002230 tx=0x7f7dfc009040 comp rx=0 tx=0).stop 2023-12-19T18:15:58.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.896+0000 7f7e12173640 1 -- 172.21.15.27:0/2951178506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e0c155ac0 msgr2=0x7f7e0c15eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:58.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.896+0000 7f7e12173640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e0c155ac0 0x7f7e0c15eba0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7df402c250 tx=0x7f7df4004730 comp rx=0 tx=0).stop 2023-12-19T18:15:58.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.896+0000 7f7e12173640 1 -- 172.21.15.27:0/2951178506 shutdown_connections 2023-12-19T18:15:58.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.896+0000 7f7e12173640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7dec06a430 0x7f7dec06c8f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.896+0000 7f7e12173640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7e0c15f0e0 0x7f7e0c0b3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.896+0000 7f7e12173640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7e0c155ac0 0x7f7e0c15eba0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.896+0000 7f7e12173640 1 --2- 172.21.15.27:0/2951178506 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7df80047c0 0x7f7e0c1505d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:58.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.896+0000 7f7e12173640 1 -- 172.21.15.27:0/2951178506 >> 172.21.15.27:0/2951178506 conn(0x7f7e0c06a480 msgr2=0x7f7e0c15b350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:58.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.897+0000 7f7e12173640 1 -- 172.21.15.27:0/2951178506 shutdown_connections 2023-12-19T18:15:58.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:58.897+0000 7f7e12173640 1 -- 172.21.15.27:0/2951178506 wait complete. 2023-12-19T18:15:58.915 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:15:59.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.149+0000 7fed66ba3640 1 Processor -- start 2023-12-19T18:15:59.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.149+0000 7fed66ba3640 1 -- start start 2023-12-19T18:15:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.150+0000 7fed66ba3640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fed6015cf80 0x7fed600d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.151+0000 7fed66ba3640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed600d0e10 con 0x7fed600b8a60 2023-12-19T18:15:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.151+0000 7fed66ba3640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed600d1c70 con 0x7fed600b85e0 2023-12-19T18:15:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.151+0000 7fed66ba3640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed6015d360 con 0x7fed6015cf80 2023-12-19T18:15:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.151+0000 7fed663a2640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fed600b8a60 0x7fed600b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46316/0 (socket says 172.21.15.27:46316) 2023-12-19T18:15:59.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.151+0000 7fed663a2640 1 -- 172.21.15.27:0/984471789 learned_addr learned my addr 172.21.15.27:0/984471789 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:59.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.151+0000 7fed65ba1640 1 --2- 172.21.15.27:0/984471789 >> v2:172.21.15.192:3300/0 conn(0x7fed6015cf80 0x7fed600d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874649800 0 0) 0x7fed600d0e10 con 0x7fed600b8a60 2023-12-19T18:15:59.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed4c003520 con 0x7fed600b8a60 2023-12-19T18:15:59.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2974221460 0 0) 0x7fed4c003520 con 0x7fed600b8a60 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 >> v2:172.21.15.192:3300/0 conn(0x7fed6015cf80 msgr2=0x7fed600d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 --2- 172.21.15.27:0/984471789 >> v2:172.21.15.192:3300/0 conn(0x7fed6015cf80 0x7fed600d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 >> v1:172.21.15.192:6789/0 conn(0x7fed600b85e0 legacy=0x7fed600cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed600d1060 con 0x7fed600b8a60 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fed580019f0 con 0x7fed600b8a60 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 >> v1:172.21.15.27:6789/0 conn(0x7fed600b8a60 legacy=0x7fed600b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 --2- 172.21.15.27:0/984471789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed4c0039c0 0x7fed4c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 --2- 172.21.15.27:0/984471789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed4c0047c0 0x7fed4c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 --2- 172.21.15.27:0/984471789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed4c005180 0x7fed4c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6015d360 con 0x7fed4c005180 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c00c060 con 0x7fed4c0047c0 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.152+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed4c00c220 con 0x7fed4c0039c0 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.153+0000 7fed663a2640 1 --2- 172.21.15.27:0/984471789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed4c005180 0x7fed4c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.153+0000 7fed653a0640 1 --2- 172.21.15.27:0/984471789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed4c0047c0 0x7fed4c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.153+0000 7fed65ba1640 1 --2- 172.21.15.27:0/984471789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed4c0039c0 0x7fed4c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.153+0000 7fed663a2640 1 -- 172.21.15.27:0/984471789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed4c0039c0 msgr2=0x7fed4c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.153+0000 7fed663a2640 1 --2- 172.21.15.27:0/984471789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed4c0039c0 0x7fed4c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.153+0000 7fed663a2640 1 -- 172.21.15.27:0/984471789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed4c0047c0 msgr2=0x7fed4c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.153+0000 7fed663a2640 1 --2- 172.21.15.27:0/984471789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed4c0047c0 0x7fed4c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.153+0000 7fed663a2640 1 -- 172.21.15.27:0/984471789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed4c003520 con 0x7fed4c005180 2023-12-19T18:15:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.153+0000 7fed653a0640 1 --2- 172.21.15.27:0/984471789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed4c0047c0 0x7fed4c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:15:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.154+0000 7fed663a2640 1 -- 172.21.15.27:0/984471789 reap_dead start 2023-12-19T18:15:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.154+0000 7fed663a2640 1 --2- 172.21.15.27:0/984471789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed4c005180 0x7fed4c00b980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fed5800a750 tx=0x7fed4c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c941d8445eb6f03b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.155+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed58034630 con 0x7fed4c005180 2023-12-19T18:15:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.155+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5802f070 con 0x7fed4c005180 2023-12-19T18:15:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.155+0000 7fed64b9f640 1 -- 172.21.15.27:0/984471789 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed580349f0 con 0x7fed4c005180 2023-12-19T18:15:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.155+0000 7fed66ba3640 1 -- 172.21.15.27:0/984471789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed4c005180 msgr2=0x7fed4c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.155+0000 7fed66ba3640 1 --2- 172.21.15.27:0/984471789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed4c005180 0x7fed4c00b980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fed5800a750 tx=0x7fed4c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.155+0000 7fed66ba3640 1 -- 172.21.15.27:0/984471789 shutdown_connections 2023-12-19T18:15:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.155+0000 7fed66ba3640 1 --2- 172.21.15.27:0/984471789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed4c005180 0x7fed4c00b980 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.155+0000 7fed66ba3640 1 -- 172.21.15.27:0/984471789 >> 172.21.15.27:0/984471789 conn(0x7fed60072e20 msgr2=0x7fed60073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.155+0000 7fed66ba3640 1 -- 172.21.15.27:0/984471789 shutdown_connections 2023-12-19T18:15:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.155+0000 7fed66ba3640 1 -- 172.21.15.27:0/984471789 wait complete. 2023-12-19T18:15:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.157+0000 7fed66ba3640 1 Processor -- start 2023-12-19T18:15:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.157+0000 7fed66ba3640 1 -- start start 2023-12-19T18:15:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.158+0000 7fed66ba3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed4c005180 0x7fed600c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.158+0000 7fed66ba3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed601f2870 0x7fed600cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.158+0000 7fed66ba3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed600b4bc0 0x7fed60151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.158+0000 7fed66ba3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6015dfc0 con 0x7fed4c005180 2023-12-19T18:15:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.158+0000 7fed66ba3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed600cceb0 con 0x7fed601f2870 2023-12-19T18:15:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.159+0000 7fed66ba3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed600ca510 con 0x7fed600b4bc0 2023-12-19T18:15:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.159+0000 7fed65ba1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed4c005180 0x7fed600c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.159+0000 7fed65ba1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed4c005180 0x7fed600c9fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48650/0 (socket says 172.21.15.27:48650) 2023-12-19T18:15:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.159+0000 7fed65ba1640 1 -- 172.21.15.27:0/3882768815 learned_addr learned my addr 172.21.15.27:0/3882768815 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.159+0000 7fed653a0640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed601f2870 0x7fed600cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.159+0000 7fed663a2640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed600b4bc0 0x7fed60151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.159+0000 7fed65ba1640 1 -- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed600b4bc0 msgr2=0x7fed60151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.159+0000 7fed65ba1640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed600b4bc0 0x7fed60151d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.159+0000 7fed65ba1640 1 -- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed601f2870 msgr2=0x7fed600cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.159+0000 7fed65ba1640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed601f2870 0x7fed600cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.159+0000 7fed65ba1640 1 -- 172.21.15.27:0/3882768815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed600cc970 con 0x7fed4c005180 2023-12-19T18:15:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.160+0000 7fed65ba1640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed4c005180 0x7fed600c9fd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fed5c011570 tx=0x7fed5c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.160+0000 7fed567fc640 1 -- 172.21.15.27:0/3882768815 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c015930 con 0x7fed4c005180 2023-12-19T18:15:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.160+0000 7fed567fc640 1 -- 172.21.15.27:0/3882768815 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed5c015a90 con 0x7fed4c005180 2023-12-19T18:15:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.160+0000 7fed567fc640 1 -- 172.21.15.27:0/3882768815 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed5c015e20 con 0x7fed4c005180 2023-12-19T18:15:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.160+0000 7fed66ba3640 1 -- 172.21.15.27:0/3882768815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6015d940 con 0x7fed4c005180 2023-12-19T18:15:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.160+0000 7fed66ba3640 1 -- 172.21.15.27:0/3882768815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed6015dbe0 con 0x7fed4c005180 2023-12-19T18:15:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.161+0000 7fed66ba3640 1 -- 172.21.15.27:0/3882768815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed6015d7a0 con 0x7fed4c005180 2023-12-19T18:15:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.162+0000 7fed567fc640 1 -- 172.21.15.27:0/3882768815 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fed5c00f460 con 0x7fed4c005180 2023-12-19T18:15:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.163+0000 7fed567fc640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed4406a430 0x7fed4406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.163+0000 7fed567fc640 1 -- 172.21.15.27:0/3882768815 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fed5c0890e0 con 0x7fed4c005180 2023-12-19T18:15:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.164+0000 7fed653a0640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed4406a430 0x7fed4406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.165+0000 7fed653a0640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed4406a430 0x7fed4406c8f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fed50000c00 tx=0x7fed50009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:59.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.168+0000 7fed567fc640 1 -- 172.21.15.27:0/3882768815 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fed6015d7a0 con 0x7fed4c005180 2023-12-19T18:15:59.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.361+0000 7fed66ba3640 1 -- 172.21.15.27:0/3882768815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed6015d7a0 con 0x7fed4c005180 2023-12-19T18:15:59.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.362+0000 7fed567fc640 1 -- 172.21.15.27:0/3882768815 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed6015d7a0 con 0x7fed4c005180 2023-12-19T18:15:59.363 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:15:59.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.365+0000 7fed66ba3640 1 -- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed4406a430 msgr2=0x7fed4406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:59.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.365+0000 7fed66ba3640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed4406a430 0x7fed4406c8f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fed50000c00 tx=0x7fed50009040 comp rx=0 tx=0).stop 2023-12-19T18:15:59.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.365+0000 7fed66ba3640 1 -- 172.21.15.27:0/3882768815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed4c005180 msgr2=0x7fed600c9fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:59.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.365+0000 7fed66ba3640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed4c005180 0x7fed600c9fd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fed5c011570 tx=0x7fed5c011a30 comp rx=0 tx=0).stop 2023-12-19T18:15:59.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.365+0000 7fed66ba3640 1 -- 172.21.15.27:0/3882768815 shutdown_connections 2023-12-19T18:15:59.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.365+0000 7fed66ba3640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed4406a430 0x7fed4406c8f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.365+0000 7fed66ba3640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed600b4bc0 0x7fed60151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.365+0000 7fed66ba3640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed601f2870 0x7fed600cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.365+0000 7fed66ba3640 1 --2- 172.21.15.27:0/3882768815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed4c005180 0x7fed600c9fd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.365+0000 7fed66ba3640 1 -- 172.21.15.27:0/3882768815 >> 172.21.15.27:0/3882768815 conn(0x7fed60072e20 msgr2=0x7fed60073200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:59.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.365+0000 7fed66ba3640 1 -- 172.21.15.27:0/3882768815 shutdown_connections 2023-12-19T18:15:59.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.365+0000 7fed66ba3640 1 -- 172.21.15.27:0/3882768815 wait complete. 2023-12-19T18:15:59.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:15:59.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.625+0000 7fdac759e640 1 Processor -- start 2023-12-19T18:15:59.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.625+0000 7fdac759e640 1 -- start start 2023-12-19T18:15:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.627+0000 7fdac759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdac80d2100 0x7fdac80cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.627+0000 7fdac759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdac80cdf80 0x7fdac80b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.627+0000 7fdac759e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac80d0e10 con 0x7fdac815cf80 2023-12-19T18:15:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.627+0000 7fdac759e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac80b8cf0 con 0x7fdac80cdf80 2023-12-19T18:15:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.627+0000 7fdac759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac80ce410 con 0x7fdac80d2100 2023-12-19T18:15:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.627+0000 7fdac659c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdac815cf80 0x7fdac80b85e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46318/0 (socket says 172.21.15.27:46318) 2023-12-19T18:15:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.627+0000 7fdac659c640 1 -- 172.21.15.27:0/943909525 learned_addr learned my addr 172.21.15.27:0/943909525 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:15:59.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.627+0000 7fdac6d9d640 1 --2- 172.21.15.27:0/943909525 >> v2:172.21.15.27:3300/0 conn(0x7fdac80cdf80 0x7fdac80b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.627+0000 7fdac5d9b640 1 --2- 172.21.15.27:0/943909525 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdac80d2100 0x7fdac80cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.628+0000 7fdac559a640 1 -- 172.21.15.27:0/943909525 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2372963333 0 0) 0x7fdac80d0e10 con 0x7fdac815cf80 2023-12-19T18:15:59.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.628+0000 7fdac559a640 1 -- 172.21.15.27:0/943909525 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab4003520 con 0x7fdac815cf80 2023-12-19T18:15:59.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.628+0000 7fdac6d9d640 1 -- 172.21.15.27:0/943909525 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdac80d2100 msgr2=0x7fdac80cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:15:59.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.628+0000 7fdac6d9d640 1 --2- 172.21.15.27:0/943909525 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdac80d2100 0x7fdac80cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.628+0000 7fdac6d9d640 1 -- 172.21.15.27:0/943909525 >> v1:172.21.15.27:6789/0 conn(0x7fdac815cf80 legacy=0x7fdac80b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:59.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.628+0000 7fdac6d9d640 1 -- 172.21.15.27:0/943909525 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac80d1c70 con 0x7fdac80cdf80 2023-12-19T18:15:59.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.629+0000 7fdac6d9d640 1 --2- 172.21.15.27:0/943909525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdac80cdf80 0x7fdac80b5d60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdac00097a0 tx=0x7fdac002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d5beec647ce206 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:59.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.629+0000 7fdac559a640 1 -- 172.21.15.27:0/943909525 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0030800 con 0x7fdac80cdf80 2023-12-19T18:15:59.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.629+0000 7fdac559a640 1 -- 172.21.15.27:0/943909525 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac0030960 con 0x7fdac80cdf80 2023-12-19T18:15:59.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.629+0000 7fdac559a640 1 -- 172.21.15.27:0/943909525 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0030cf0 con 0x7fdac80cdf80 2023-12-19T18:15:59.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.630+0000 7fdac759e640 1 -- 172.21.15.27:0/943909525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdac80cdf80 msgr2=0x7fdac80b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:59.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.630+0000 7fdac759e640 1 --2- 172.21.15.27:0/943909525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdac80cdf80 0x7fdac80b5d60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdac00097a0 tx=0x7fdac002b940 comp rx=0 tx=0).stop 2023-12-19T18:15:59.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.630+0000 7fdac759e640 1 -- 172.21.15.27:0/943909525 shutdown_connections 2023-12-19T18:15:59.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.630+0000 7fdac759e640 1 --2- 172.21.15.27:0/943909525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdac80cdf80 0x7fdac80b5d60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.630+0000 7fdac759e640 1 --2- 172.21.15.27:0/943909525 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdac80d2100 0x7fdac80cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.630+0000 7fdac759e640 1 -- 172.21.15.27:0/943909525 >> 172.21.15.27:0/943909525 conn(0x7fdac8072e20 msgr2=0x7fdac8073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.630+0000 7fdac759e640 1 -- 172.21.15.27:0/943909525 shutdown_connections 2023-12-19T18:15:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.630+0000 7fdac759e640 1 -- 172.21.15.27:0/943909525 wait complete. 2023-12-19T18:15:59.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.632+0000 7fdac759e640 1 Processor -- start 2023-12-19T18:15:59.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.632+0000 7fdac759e640 1 -- start start 2023-12-19T18:15:59.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.633+0000 7fdac759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdac80cdf80 0x7fdac8149e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.633+0000 7fdac759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdac80d2100 0x7fdac80b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.633+0000 7fdac759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdac815cf80 0x7fdac80a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.633+0000 7fdac759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac80cb4c0 con 0x7fdac815cf80 2023-12-19T18:15:59.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.633+0000 7fdac759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac814a340 con 0x7fdac80cdf80 2023-12-19T18:15:59.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.633+0000 7fdac759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac814cd50 con 0x7fdac80d2100 2023-12-19T18:15:59.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.633+0000 7fdac6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdac815cf80 0x7fdac80a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.634+0000 7fdac6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdac815cf80 0x7fdac80a96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48674/0 (socket says 172.21.15.27:48674) 2023-12-19T18:15:59.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.634+0000 7fdac6d9d640 1 -- 172.21.15.27:0/2158179093 learned_addr learned my addr 172.21.15.27:0/2158179093 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:15:59.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.634+0000 7fdac5d9b640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdac80d2100 0x7fdac80b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.634+0000 7fdac6d9d640 1 -- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdac80d2100 msgr2=0x7fdac80b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.634+0000 7fdac659c640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdac80cdf80 0x7fdac8149e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.634+0000 7fdac6d9d640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdac80d2100 0x7fdac80b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.634+0000 7fdac6d9d640 1 -- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdac80cdf80 msgr2=0x7fdac8149e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.634+0000 7fdac6d9d640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdac80cdf80 0x7fdac8149e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.634+0000 7fdac6d9d640 1 -- 172.21.15.27:0/2158179093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac0009400 con 0x7fdac815cf80 2023-12-19T18:15:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.634+0000 7fdac5d9b640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdac80d2100 0x7fdac80b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.634+0000 7fdac659c640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdac80cdf80 0x7fdac8149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:15:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.634+0000 7fdac6d9d640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdac815cf80 0x7fdac80a96a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdac00098c0 tx=0x7fdac0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.635+0000 7fdabeffd640 1 -- 172.21.15.27:0/2158179093 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac003e040 con 0x7fdac815cf80 2023-12-19T18:15:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.635+0000 7fdabeffd640 1 -- 172.21.15.27:0/2158179093 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdac0039c90 con 0x7fdac815cf80 2023-12-19T18:15:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.635+0000 7fdac759e640 1 -- 172.21.15.27:0/2158179093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac80cab20 con 0x7fdac815cf80 2023-12-19T18:15:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.635+0000 7fdabeffd640 1 -- 172.21.15.27:0/2158179093 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdac0006e10 con 0x7fdac815cf80 2023-12-19T18:15:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.635+0000 7fdac759e640 1 -- 172.21.15.27:0/2158179093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac80cb0e0 con 0x7fdac815cf80 2023-12-19T18:15:59.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.636+0000 7fdac759e640 1 -- 172.21.15.27:0/2158179093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac80cb600 con 0x7fdac815cf80 2023-12-19T18:15:59.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.637+0000 7fdabeffd640 1 -- 172.21.15.27:0/2158179093 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdac00423e0 con 0x7fdac815cf80 2023-12-19T18:15:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.638+0000 7fdabeffd640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdaa806a430 0x7fdaa806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:15:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.638+0000 7fdabeffd640 1 -- 172.21.15.27:0/2158179093 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fdac00aacb0 con 0x7fdac815cf80 2023-12-19T18:15:59.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.639+0000 7fdac659c640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdaa806a430 0x7fdaa806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:15:59.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.640+0000 7fdac659c640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdaa806a430 0x7fdaa806c8f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005c00 tx=0x7fdab8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:15:59.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.642+0000 7fdabeffd640 1 -- 172.21.15.27:0/2158179093 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdac80cb600 con 0x7fdac815cf80 2023-12-19T18:15:59.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.844+0000 7fdac759e640 1 -- 172.21.15.27:0/2158179093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdac80cb600 con 0x7fdac815cf80 2023-12-19T18:15:59.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.845+0000 7fdabeffd640 1 -- 172.21.15.27:0/2158179093 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdac0048020 con 0x7fdac815cf80 2023-12-19T18:15:59.845 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:15:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.847+0000 7fdac759e640 1 -- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdaa806a430 msgr2=0x7fdaa806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.847+0000 7fdac759e640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdaa806a430 0x7fdaa806c8f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fdab8005c00 tx=0x7fdab8009040 comp rx=0 tx=0).stop 2023-12-19T18:15:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.847+0000 7fdac759e640 1 -- 172.21.15.27:0/2158179093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdac815cf80 msgr2=0x7fdac80a96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:15:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.847+0000 7fdac759e640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdac815cf80 0x7fdac80a96a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdac00098c0 tx=0x7fdac0004730 comp rx=0 tx=0).stop 2023-12-19T18:15:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.847+0000 7fdac759e640 1 -- 172.21.15.27:0/2158179093 shutdown_connections 2023-12-19T18:15:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.847+0000 7fdac759e640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdaa806a430 0x7fdaa806c8f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.847+0000 7fdac759e640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdac815cf80 0x7fdac80a96a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.847+0000 7fdac759e640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdac80d2100 0x7fdac80b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.847+0000 7fdac759e640 1 --2- 172.21.15.27:0/2158179093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdac80cdf80 0x7fdac8149e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:15:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.847+0000 7fdac759e640 1 -- 172.21.15.27:0/2158179093 >> 172.21.15.27:0/2158179093 conn(0x7fdac8072e20 msgr2=0x7fdac806a270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:15:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.848+0000 7fdac759e640 1 -- 172.21.15.27:0/2158179093 shutdown_connections 2023-12-19T18:15:59.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:15:59.848+0000 7fdac759e640 1 -- 172.21.15.27:0/2158179093 wait complete. 2023-12-19T18:15:59.869 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:16:00.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.106+0000 7f8965299640 1 Processor -- start 2023-12-19T18:16:00.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.106+0000 7f8965299640 1 -- start start 2023-12-19T18:16:00.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.107+0000 7f8965299640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f896015ac50 0x7f8960070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:00.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.107+0000 7f8965299640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89600af360 0x7f896006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:00.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.107+0000 7f8965299640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89600adf10 con 0x7f89600aef20 2023-12-19T18:16:00.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.107+0000 7f8965299640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89600adac0 con 0x7f896015ac50 2023-12-19T18:16:00.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.107+0000 7f8965299640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89600ae160 con 0x7f89600af360 2023-12-19T18:16:00.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.108+0000 7f895ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f896015ac50 0x7f8960070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.108+0000 7f895ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f896015ac50 0x7f8960070fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48688/0 (socket says 172.21.15.27:48688) 2023-12-19T18:16:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.108+0000 7f895ffff640 1 -- 172.21.15.27:0/3929188254 learned_addr learned my addr 172.21.15.27:0/3929188254 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.108+0000 7f8964a98640 1 --2- 172.21.15.27:0/3929188254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89600af360 0x7f896006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.108+0000 7f895ffff640 1 -- 172.21.15.27:0/3929188254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89600af360 msgr2=0x7f896006ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.108+0000 7f895ffff640 1 --2- 172.21.15.27:0/3929188254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89600af360 0x7f896006ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.108+0000 7f895ffff640 1 -- 172.21.15.27:0/3929188254 >> v1:172.21.15.192:6789/0 conn(0x7f89600aef20 legacy=0x7f896006b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.108+0000 7f895ffff640 1 -- 172.21.15.27:0/3929188254 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89600ae440 con 0x7f896015ac50 2023-12-19T18:16:00.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.110+0000 7f895ffff640 1 --2- 172.21.15.27:0/3929188254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896015ac50 0x7f8960070fb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8950009e50 tx=0x7f895002bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9919a1fd5f4ff45d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.110+0000 7f895effd640 1 -- 172.21.15.27:0/3929188254 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895002c950 con 0x7f896015ac50 2023-12-19T18:16:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.110+0000 7f895effd640 1 -- 172.21.15.27:0/3929188254 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895002cab0 con 0x7f896015ac50 2023-12-19T18:16:00.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.110+0000 7f895effd640 1 -- 172.21.15.27:0/3929188254 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895002ce40 con 0x7f896015ac50 2023-12-19T18:16:00.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.111+0000 7f8965299640 1 -- 172.21.15.27:0/3929188254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896015ac50 msgr2=0x7f8960070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:00.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.111+0000 7f8965299640 1 --2- 172.21.15.27:0/3929188254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896015ac50 0x7f8960070fb0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f8950009e50 tx=0x7f895002bdb0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.111+0000 7f8965299640 1 -- 172.21.15.27:0/3929188254 shutdown_connections 2023-12-19T18:16:00.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.111+0000 7f8965299640 1 --2- 172.21.15.27:0/3929188254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89600af360 0x7f896006ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.111+0000 7f8965299640 1 --2- 172.21.15.27:0/3929188254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896015ac50 0x7f8960070fb0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.111+0000 7f8965299640 1 -- 172.21.15.27:0/3929188254 >> 172.21.15.27:0/3929188254 conn(0x7f896006a7d0 msgr2=0x7f89600ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:00.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.111+0000 7f8965299640 1 -- 172.21.15.27:0/3929188254 shutdown_connections 2023-12-19T18:16:00.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.111+0000 7f8965299640 1 -- 172.21.15.27:0/3929188254 wait complete. 2023-12-19T18:16:00.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.113+0000 7f8965299640 1 Processor -- start 2023-12-19T18:16:00.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.113+0000 7f8965299640 1 -- start start 2023-12-19T18:16:00.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.114+0000 7f8965299640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89600aef20 0x7f89601590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:00.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.114+0000 7f8965299640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89600af360 0x7f89601f2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:00.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.114+0000 7f8965299640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896015ac50 0x7f89600b1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:00.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.114+0000 7f8965299640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89600b32e0 con 0x7f896015ac50 2023-12-19T18:16:00.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.114+0000 7f8965299640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89600b21e0 con 0x7f89600aef20 2023-12-19T18:16:00.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.114+0000 7f8965299640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89600b24c0 con 0x7f89600af360 2023-12-19T18:16:00.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.114+0000 7f8964a98640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896015ac50 0x7f89600b1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:00.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.114+0000 7f8964a98640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896015ac50 0x7f89600b1ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48700/0 (socket says 172.21.15.27:48700) 2023-12-19T18:16:00.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.115+0000 7f8964a98640 1 -- 172.21.15.27:0/2876281189 learned_addr learned my addr 172.21.15.27:0/2876281189 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:00.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.115+0000 7f8964a98640 1 -- 172.21.15.27:0/2876281189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89600af360 msgr2=0x7f89601f2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:00.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.115+0000 7f8964a98640 1 --2- 172.21.15.27:0/2876281189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89600af360 0x7f89601f2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.115+0000 7f8964a98640 1 -- 172.21.15.27:0/2876281189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89600aef20 msgr2=0x7f89601590a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:00.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.115+0000 7f8964a98640 1 --2- 172.21.15.27:0/2876281189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89600aef20 0x7f89601590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.115+0000 7f8964a98640 1 -- 172.21.15.27:0/2876281189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8950009b30 con 0x7f896015ac50 2023-12-19T18:16:00.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.115+0000 7f8964a98640 1 --2- 172.21.15.27:0/2876281189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896015ac50 0x7f89600b1ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f895800d5d0 tx=0x7f895800da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:00.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.115+0000 7f895cff9640 1 -- 172.21.15.27:0/2876281189 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89580125f0 con 0x7f896015ac50 2023-12-19T18:16:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.116+0000 7f895cff9640 1 -- 172.21.15.27:0/2876281189 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8958012770 con 0x7f896015ac50 2023-12-19T18:16:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.116+0000 7f895cff9640 1 -- 172.21.15.27:0/2876281189 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958012b20 con 0x7f896015ac50 2023-12-19T18:16:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.116+0000 7f8965299640 1 -- 172.21.15.27:0/2876281189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89600b27a0 con 0x7f896015ac50 2023-12-19T18:16:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.116+0000 7f8965299640 1 -- 172.21.15.27:0/2876281189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89600b2a80 con 0x7f896015ac50 2023-12-19T18:16:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.117+0000 7f8965299640 1 -- 172.21.15.27:0/2876281189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8960156ea0 con 0x7f896015ac50 2023-12-19T18:16:00.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.118+0000 7f895cff9640 1 -- 172.21.15.27:0/2876281189 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8958012c80 con 0x7f896015ac50 2023-12-19T18:16:00.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.119+0000 7f895cff9640 1 --2- 172.21.15.27:0/2876281189 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f894006a430 0x7f894006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:00.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.119+0000 7f895cff9640 1 -- 172.21.15.27:0/2876281189 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f8958088d70 con 0x7f896015ac50 2023-12-19T18:16:00.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.119+0000 7f895ffff640 1 --2- 172.21.15.27:0/2876281189 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f894006a430 0x7f894006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:00.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.120+0000 7f895ffff640 1 --2- 172.21.15.27:0/2876281189 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f894006a430 0x7f894006c8f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8950035000 tx=0x7f895002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:00.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.123+0000 7f895cff9640 1 -- 172.21.15.27:0/2876281189 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8958054350 con 0x7f896015ac50 2023-12-19T18:16:00.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.329+0000 7f8965299640 1 -- 172.21.15.27:0/2876281189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f896004e840 con 0x7f896015ac50 2023-12-19T18:16:00.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.330+0000 7f895cff9640 1 -- 172.21.15.27:0/2876281189 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8958057970 con 0x7f896015ac50 2023-12-19T18:16:00.330 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:16:00.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.332+0000 7f8965299640 1 -- 172.21.15.27:0/2876281189 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f894006a430 msgr2=0x7f894006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:00.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.332+0000 7f8965299640 1 --2- 172.21.15.27:0/2876281189 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f894006a430 0x7f894006c8f0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f8950035000 tx=0x7f895002e000 comp rx=0 tx=0).stop 2023-12-19T18:16:00.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.332+0000 7f8965299640 1 -- 172.21.15.27:0/2876281189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896015ac50 msgr2=0x7f89600b1ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:00.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.332+0000 7f8965299640 1 --2- 172.21.15.27:0/2876281189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896015ac50 0x7f89600b1ae0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f895800d5d0 tx=0x7f895800da90 comp rx=0 tx=0).stop 2023-12-19T18:16:00.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.333+0000 7f8965299640 1 -- 172.21.15.27:0/2876281189 shutdown_connections 2023-12-19T18:16:00.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.333+0000 7f8965299640 1 --2- 172.21.15.27:0/2876281189 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f894006a430 0x7f894006c8f0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.333+0000 7f8965299640 1 --2- 172.21.15.27:0/2876281189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f896015ac50 0x7f89600b1ae0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.333+0000 7f8965299640 1 --2- 172.21.15.27:0/2876281189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f89600af360 0x7f89601f2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.333+0000 7f8965299640 1 --2- 172.21.15.27:0/2876281189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f89600aef20 0x7f89601590a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.333+0000 7f8965299640 1 -- 172.21.15.27:0/2876281189 >> 172.21.15.27:0/2876281189 conn(0x7f896006a7d0 msgr2=0x7f89600ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:00.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.333+0000 7f8965299640 1 -- 172.21.15.27:0/2876281189 shutdown_connections 2023-12-19T18:16:00.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.333+0000 7f8965299640 1 -- 172.21.15.27:0/2876281189 wait complete. 2023-12-19T18:16:00.354 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:16:00.354 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:00.406 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:00.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.688+0000 7fefb535d640 1 Processor -- start 2023-12-19T18:16:00.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.688+0000 7fefb535d640 1 -- start start 2023-12-19T18:16:00.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.690+0000 7fefb535d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fefb01553a0 0x7fefb00c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:00.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.690+0000 7fefb535d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb00b0360 con 0x7fefb00afeb0 2023-12-19T18:16:00.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.690+0000 7fefb535d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefb00cb350 con 0x7fefb015ee30 2023-12-19T18:16:00.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.690+0000 7fefb535d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefb00cbcf0 con 0x7fefb01553a0 2023-12-19T18:16:00.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.691+0000 7fefa7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fefb00afeb0 0x7fefb015e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46330/0 (socket says 172.21.15.27:46330) 2023-12-19T18:16:00.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.691+0000 7fefa7fff640 1 -- 172.21.15.27:0/474417394 learned_addr learned my addr 172.21.15.27:0/474417394 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:00.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.691+0000 7fefaffff640 1 --2- 172.21.15.27:0/474417394 >> v2:172.21.15.27:3300/0 conn(0x7fefb01553a0 0x7fefb00c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:00.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.691+0000 7fefaf7fe640 1 -- 172.21.15.27:0/474417394 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412833644 0 0) 0x7fefb00b0360 con 0x7fefb00afeb0 2023-12-19T18:16:00.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.691+0000 7fefaf7fe640 1 -- 172.21.15.27:0/474417394 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef9c003520 con 0x7fefb00afeb0 2023-12-19T18:16:00.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.691+0000 7fefaffff640 1 -- 172.21.15.27:0/474417394 >> v1:172.21.15.192:6789/0 conn(0x7fefb015ee30 legacy=0x7fefb00c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:00.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.692+0000 7fefaffff640 1 -- 172.21.15.27:0/474417394 >> v1:172.21.15.27:6789/0 conn(0x7fefb00afeb0 legacy=0x7fefb015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:00.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.692+0000 7fefaf7fe640 1 -- 172.21.15.27:0/474417394 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3382431907 0 0) 0x7fef9c003520 con 0x7fefb00afeb0 2023-12-19T18:16:00.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.692+0000 7fefaffff640 1 -- 172.21.15.27:0/474417394 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefb00cb630 con 0x7fefb01553a0 2023-12-19T18:16:00.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.693+0000 7fefaffff640 1 --2- 172.21.15.27:0/474417394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefb01553a0 0x7fefb00c50a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fefa00097a0 tx=0x7fefa002b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cd11b52356e3118 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:00.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.693+0000 7fefaf7fe640 1 -- 172.21.15.27:0/474417394 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef98001670 con 0x7fefb01553a0 2023-12-19T18:16:00.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.693+0000 7fefaf7fe640 1 -- 172.21.15.27:0/474417394 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa002e050 con 0x7fefb01553a0 2023-12-19T18:16:00.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.693+0000 7fefaf7fe640 1 -- 172.21.15.27:0/474417394 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa0030730 con 0x7fefb01553a0 2023-12-19T18:16:00.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.694+0000 7fefb535d640 1 -- 172.21.15.27:0/474417394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefb01553a0 msgr2=0x7fefb00c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:00.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.694+0000 7fefb535d640 1 --2- 172.21.15.27:0/474417394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefb01553a0 0x7fefb00c50a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fefa00097a0 tx=0x7fefa002b790 comp rx=0 tx=0).stop 2023-12-19T18:16:00.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.694+0000 7fefb535d640 1 -- 172.21.15.27:0/474417394 shutdown_connections 2023-12-19T18:16:00.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.694+0000 7fefb535d640 1 --2- 172.21.15.27:0/474417394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefb01553a0 0x7fefb00c50a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.694+0000 7fefb535d640 1 -- 172.21.15.27:0/474417394 >> 172.21.15.27:0/474417394 conn(0x7fefb0069c70 msgr2=0x7fefb006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:00.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.694+0000 7fefb535d640 1 -- 172.21.15.27:0/474417394 shutdown_connections 2023-12-19T18:16:00.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.694+0000 7fefb535d640 1 -- 172.21.15.27:0/474417394 wait complete. 2023-12-19T18:16:00.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.696+0000 7fefb535d640 1 Processor -- start 2023-12-19T18:16:00.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.696+0000 7fefb535d640 1 -- start start 2023-12-19T18:16:00.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.697+0000 7fefb535d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefb00afeb0 0x7fefb013d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:00.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.697+0000 7fefb535d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefb01553a0 0x7fefb015d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:00.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.697+0000 7fefb535d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefb015ee30 0x7fefb006c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:00.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.697+0000 7fefb535d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb015e6c0 con 0x7fefb015ee30 2023-12-19T18:16:00.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.697+0000 7fefb535d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefb015dab0 con 0x7fefb00afeb0 2023-12-19T18:16:00.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.697+0000 7fefb535d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefb015dd90 con 0x7fefb01553a0 2023-12-19T18:16:00.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.697+0000 7fefb4b5c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefb015ee30 0x7fefb006c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:00.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.697+0000 7fefb4b5c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefb015ee30 0x7fefb006c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48718/0 (socket says 172.21.15.27:48718) 2023-12-19T18:16:00.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.697+0000 7fefb4b5c640 1 -- 172.21.15.27:0/2726925242 learned_addr learned my addr 172.21.15.27:0/2726925242 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:00.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefaffff640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefb00afeb0 0x7fefb013d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:00.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefa7fff640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefb01553a0 0x7fefb015d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:00.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefb4b5c640 1 -- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefb01553a0 msgr2=0x7fefb015d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:00.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefb4b5c640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefb01553a0 0x7fefb015d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefb4b5c640 1 -- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefb00afeb0 msgr2=0x7fefb013d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:00.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefb4b5c640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefb00afeb0 0x7fefb013d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefb4b5c640 1 -- 172.21.15.27:0/2726925242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefa0009400 con 0x7fefb015ee30 2023-12-19T18:16:00.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefaffff640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefb00afeb0 0x7fefb013d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:00.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefa7fff640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefb01553a0 0x7fefb015d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:00.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefb4b5c640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefb015ee30 0x7fefb006c420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fefa800c6b0 tx=0x7fefa800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:00.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefad7fa640 1 -- 172.21.15.27:0/2726925242 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8003b70 con 0x7fefb015ee30 2023-12-19T18:16:00.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefad7fa640 1 -- 172.21.15.27:0/2726925242 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefa8003cf0 con 0x7fefb015ee30 2023-12-19T18:16:00.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefad7fa640 1 -- 172.21.15.27:0/2726925242 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefa8003ea0 con 0x7fefb015ee30 2023-12-19T18:16:00.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.698+0000 7fefb535d640 1 -- 172.21.15.27:0/2726925242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefb015e070 con 0x7fefb015ee30 2023-12-19T18:16:00.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.699+0000 7fefb535d640 1 -- 172.21.15.27:0/2726925242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefb013d820 con 0x7fefb015ee30 2023-12-19T18:16:00.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.700+0000 7fefb535d640 1 -- 172.21.15.27:0/2726925242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefb015e870 con 0x7fefb015ee30 2023-12-19T18:16:00.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.701+0000 7fefad7fa640 1 -- 172.21.15.27:0/2726925242 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fefa8005310 con 0x7fefb015ee30 2023-12-19T18:16:00.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.702+0000 7fefad7fa640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fef9006a460 0x7fef9006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:00.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.702+0000 7fefad7fa640 1 -- 172.21.15.27:0/2726925242 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fefa80887c0 con 0x7fefb015ee30 2023-12-19T18:16:00.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.703+0000 7fefaffff640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fef9006a460 0x7fef9006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:00.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.705+0000 7fefaffff640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fef9006a460 0x7fef9006c920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fefa0034000 tx=0x7fefa002f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:00.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.708+0000 7fefad7fa640 1 -- 172.21.15.27:0/2726925242 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fefb015e870 con 0x7fefb015ee30 2023-12-19T18:16:00.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.905+0000 7fefb535d640 1 -- 172.21.15.27:0/2726925242 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fefb004e840 con 0x7fefb015ee30 2023-12-19T18:16:00.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.908+0000 7fefad7fa640 1 -- 172.21.15.27:0/2726925242 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16090 (secure 0 0 0) 0x7fefa8053c60 con 0x7fefb015ee30 2023-12-19T18:16:00.908 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:00.908 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:15:43.580746+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.910+0000 7fefb535d640 1 -- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fef9006a460 msgr2=0x7fef9006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.910+0000 7fefb535d640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fef9006a460 0x7fef9006c920 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fefa0034000 tx=0x7fefa002f000 comp rx=0 tx=0).stop 2023-12-19T18:16:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.911+0000 7fefb535d640 1 -- 172.21.15.27:0/2726925242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefb015ee30 msgr2=0x7fefb006c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.911+0000 7fefb535d640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefb015ee30 0x7fefb006c420 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fefa800c6b0 tx=0x7fefa800cb70 comp rx=0 tx=0).stop 2023-12-19T18:16:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.911+0000 7fefb535d640 1 -- 172.21.15.27:0/2726925242 shutdown_connections 2023-12-19T18:16:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.911+0000 7fefb535d640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fef9006a460 0x7fef9006c920 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.911+0000 7fefb535d640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fefb015ee30 0x7fefb006c420 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.911+0000 7fefb535d640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fefb01553a0 0x7fefb015d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.911+0000 7fefb535d640 1 --2- 172.21.15.27:0/2726925242 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fefb00afeb0 0x7fefb013d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:00.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.911+0000 7fefb535d640 1 -- 172.21.15.27:0/2726925242 >> 172.21.15.27:0/2726925242 conn(0x7fefb0069c70 msgr2=0x7fefb00cf290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:00.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.911+0000 7fefb535d640 1 -- 172.21.15.27:0/2726925242 shutdown_connections 2023-12-19T18:16:00.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:00.911+0000 7fefb535d640 1 -- 172.21.15.27:0/2726925242 wait complete. 2023-12-19T18:16:00.931 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:16:00.931 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:16:01.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.169+0000 7f6f80cd7640 1 Processor -- start 2023-12-19T18:16:01.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.169+0000 7f6f80cd7640 1 -- start start 2023-12-19T18:16:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.170+0000 7f6f80cd7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6f7c15cf80 0x7f6f7c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.170+0000 7f6f80cd7640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6f7c0b85e0 0x7f6f7c0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.170+0000 7f6f80cd7640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f7c0d0e10 con 0x7f6f7c0b8a60 2023-12-19T18:16:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.170+0000 7f6f80cd7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7c0d1c70 con 0x7f6f7c15cf80 2023-12-19T18:16:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.170+0000 7f6f80cd7640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f7c15d360 con 0x7f6f7c0b85e0 2023-12-19T18:16:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.171+0000 7f6f7b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6f7c15cf80 0x7f6f7c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:01.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.171+0000 7f6f7affd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6f7c0b85e0 0x7f6f7c0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.171+0000 7f6f7affd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6f7c0b85e0 0x7f6f7c0cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:43940/0 (socket says 172.21.15.27:43940) 2023-12-19T18:16:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.171+0000 7f6f7affd640 1 -- 172.21.15.27:0/1616150844 learned_addr learned my addr 172.21.15.27:0/1616150844 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.171+0000 7f6f7b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6f7c15cf80 0x7f6f7c0d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48724/0 (socket says 172.21.15.27:48724) 2023-12-19T18:16:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.172+0000 7f6f7b7fe640 1 -- 172.21.15.27:0/1616150844 >> v2:172.21.15.192:3300/0 conn(0x7f6f7c0b85e0 msgr2=0x7f6f7c0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.172+0000 7f6f7b7fe640 1 --2- 172.21.15.27:0/1616150844 >> v2:172.21.15.192:3300/0 conn(0x7f6f7c0b85e0 0x7f6f7c0cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.172+0000 7f6f7b7fe640 1 -- 172.21.15.27:0/1616150844 >> v1:172.21.15.192:6789/0 conn(0x7f6f7c0b8a60 legacy=0x7f6f7c0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:01.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.172+0000 7f6f7b7fe640 1 -- 172.21.15.27:0/1616150844 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f7c0d1060 con 0x7f6f7c15cf80 2023-12-19T18:16:01.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.173+0000 7f6f7b7fe640 1 --2- 172.21.15.27:0/1616150844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f7c15cf80 0x7f6f7c0d2100 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0cd270 tx=0x7f6f6c02bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27dd7a99dbf6b7c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:01.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.174+0000 7f6f7a7fc640 1 -- 172.21.15.27:0/1616150844 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c02cd00 con 0x7f6f7c15cf80 2023-12-19T18:16:01.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.174+0000 7f6f7a7fc640 1 -- 172.21.15.27:0/1616150844 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6c02ce60 con 0x7f6f7c15cf80 2023-12-19T18:16:01.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.174+0000 7f6f7a7fc640 1 -- 172.21.15.27:0/1616150844 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6c034660 con 0x7f6f7c15cf80 2023-12-19T18:16:01.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.174+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1616150844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f7c15cf80 msgr2=0x7f6f7c0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:01.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.174+0000 7f6f80cd7640 1 --2- 172.21.15.27:0/1616150844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f7c15cf80 0x7f6f7c0d2100 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c0cd270 tx=0x7f6f6c02bf50 comp rx=0 tx=0).stop 2023-12-19T18:16:01.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.174+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1616150844 shutdown_connections 2023-12-19T18:16:01.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.174+0000 7f6f80cd7640 1 --2- 172.21.15.27:0/1616150844 >> v2:172.21.15.192:3300/0 conn(0x7f6f7c0b85e0 0x7f6f7c0cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.174+0000 7f6f80cd7640 1 --2- 172.21.15.27:0/1616150844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f7c15cf80 0x7f6f7c0d2100 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.174+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1616150844 >> 172.21.15.27:0/1616150844 conn(0x7f6f7c072e20 msgr2=0x7f6f7c073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:01.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.175+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1616150844 shutdown_connections 2023-12-19T18:16:01.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.175+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1616150844 wait complete. 2023-12-19T18:16:01.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.177+0000 7f6f80cd7640 1 Processor -- start 2023-12-19T18:16:01.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.177+0000 7f6f80cd7640 1 -- start start 2023-12-19T18:16:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.178+0000 7f6f80cd7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f7c0b85e0 0x7f6f7c152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.178+0000 7f6f80cd7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f7c15cf80 0x7f6f7c151940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.178+0000 7f6f80cd7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f7c152fa0 0x7f6f7c151e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.178+0000 7f6f80cd7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c0ca1e0 con 0x7f6f7c15cf80 2023-12-19T18:16:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.178+0000 7f6f80cd7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c1556a0 con 0x7f6f7c152fa0 2023-12-19T18:16:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.178+0000 7f6f80cd7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c155980 con 0x7f6f7c0b85e0 2023-12-19T18:16:01.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.178+0000 7f6f7affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f7c15cf80 0x7f6f7c151940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:01.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.178+0000 7f6f7affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f7c15cf80 0x7f6f7c151940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48736/0 (socket says 172.21.15.27:48736) 2023-12-19T18:16:01.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.178+0000 7f6f7affd640 1 -- 172.21.15.27:0/1375123247 learned_addr learned my addr 172.21.15.27:0/1375123247 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:01.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.178+0000 7f6f7b7fe640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f7c0b85e0 0x7f6f7c152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:01.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.179+0000 7f6f7affd640 1 -- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f7c0b85e0 msgr2=0x7f6f7c152a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:01.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.179+0000 7f6f7affd640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f7c0b85e0 0x7f6f7c152a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.179+0000 7f6f7affd640 1 -- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f7c152fa0 msgr2=0x7f6f7c151e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:01.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.179+0000 7f6f7bfff640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f7c152fa0 0x7f6f7c151e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:01.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.179+0000 7f6f7affd640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f7c152fa0 0x7f6f7c151e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.179+0000 7f6f7affd640 1 -- 172.21.15.27:0/1375123247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c009a70 con 0x7f6f7c15cf80 2023-12-19T18:16:01.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.179+0000 7f6f7bfff640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f7c152fa0 0x7f6f7c151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:01.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.179+0000 7f6f7affd640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f7c15cf80 0x7f6f7c151940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009fd0 tx=0x7f6f6400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:01.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.179+0000 7f6f73fff640 1 -- 172.21.15.27:0/1375123247 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400c940 con 0x7f6f7c15cf80 2023-12-19T18:16:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.179+0000 7f6f73fff640 1 -- 172.21.15.27:0/1375123247 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6400caa0 con 0x7f6f7c15cf80 2023-12-19T18:16:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.179+0000 7f6f73fff640 1 -- 172.21.15.27:0/1375123247 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400ce30 con 0x7f6f7c15cf80 2023-12-19T18:16:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.179+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1375123247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7c155c60 con 0x7f6f7c15cf80 2023-12-19T18:16:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.179+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1375123247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7c154a00 con 0x7f6f7c15cf80 2023-12-19T18:16:01.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.180+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1375123247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7c04e840 con 0x7f6f7c15cf80 2023-12-19T18:16:01.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.182+0000 7f6f73fff640 1 -- 172.21.15.27:0/1375123247 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6f64013470 con 0x7f6f7c15cf80 2023-12-19T18:16:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.183+0000 7f6f73fff640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5c06a430 0x7f6f5c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.183+0000 7f6f73fff640 1 -- 172.21.15.27:0/1375123247 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f6f64088a80 con 0x7f6f7c15cf80 2023-12-19T18:16:01.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.183+0000 7f6f7b7fe640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5c06a430 0x7f6f5c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:01.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.184+0000 7f6f7b7fe640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5c06a430 0x7f6f5c06c8f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c036000 tx=0x7f6f6c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:01.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.187+0000 7f6f73fff640 1 -- 172.21.15.27:0/1375123247 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6f64054050 con 0x7f6f7c15cf80 2023-12-19T18:16:01.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.382+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1375123247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6f7c0c95d0 con 0x7f6f7c15cf80 2023-12-19T18:16:01.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.383+0000 7f6f73fff640 1 -- 172.21.15.27:0/1375123247 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6f640170a0 con 0x7f6f7c15cf80 2023-12-19T18:16:01.383 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:16:01.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.385+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5c06a430 msgr2=0x7f6f5c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:01.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.385+0000 7f6f80cd7640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5c06a430 0x7f6f5c06c8f0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c036000 tx=0x7f6f6c030000 comp rx=0 tx=0).stop 2023-12-19T18:16:01.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.385+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1375123247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f7c15cf80 msgr2=0x7f6f7c151940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:01.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.385+0000 7f6f80cd7640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f7c15cf80 0x7f6f7c151940 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f6f64009fd0 tx=0x7f6f6400ed40 comp rx=0 tx=0).stop 2023-12-19T18:16:01.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.385+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1375123247 shutdown_connections 2023-12-19T18:16:01.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.385+0000 7f6f80cd7640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5c06a430 0x7f6f5c06c8f0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.385+0000 7f6f80cd7640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f7c152fa0 0x7f6f7c151e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.385+0000 7f6f80cd7640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f7c15cf80 0x7f6f7c151940 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.385+0000 7f6f80cd7640 1 --2- 172.21.15.27:0/1375123247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f7c0b85e0 0x7f6f7c152a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.385+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1375123247 >> 172.21.15.27:0/1375123247 conn(0x7f6f7c072e20 msgr2=0x7f6f7c0cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:01.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.386+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1375123247 shutdown_connections 2023-12-19T18:16:01.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.386+0000 7f6f80cd7640 1 -- 172.21.15.27:0/1375123247 wait complete. 2023-12-19T18:16:01.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:16:01.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.644+0000 7f2162e6d640 1 Processor -- start 2023-12-19T18:16:01.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.644+0000 7f2162e6d640 1 -- start start 2023-12-19T18:16:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.645+0000 7f2162e6d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f215c151c90 0x7f215c06b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.645+0000 7f2162e6d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f215c15e9d0 0x7f215c0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:01.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.645+0000 7f2162e6d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f215c15b3e0 con 0x7f215c152070 2023-12-19T18:16:01.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.645+0000 7f2162e6d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215c15eed0 con 0x7f215c151c90 2023-12-19T18:16:01.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.645+0000 7f2162e6d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f215c15f1b0 con 0x7f215c15e9d0 2023-12-19T18:16:01.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.646+0000 7f216266c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f215c152070 0x7f215c15b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:46342/0 (socket says 172.21.15.27:46342) 2023-12-19T18:16:01.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.646+0000 7f2161e6b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f215c151c90 0x7f215c06b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:01.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.646+0000 7f216266c640 1 -- 172.21.15.27:0/687799404 learned_addr learned my addr 172.21.15.27:0/687799404 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:01.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.646+0000 7f216166a640 1 --2- 172.21.15.27:0/687799404 >> v2:172.21.15.192:3300/0 conn(0x7f215c15e9d0 0x7f215c0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:01.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.646+0000 7f2161e6b640 1 -- 172.21.15.27:0/687799404 >> v2:172.21.15.192:3300/0 conn(0x7f215c15e9d0 msgr2=0x7f215c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:01.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.646+0000 7f2161e6b640 1 --2- 172.21.15.27:0/687799404 >> v2:172.21.15.192:3300/0 conn(0x7f215c15e9d0 0x7f215c0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.646+0000 7f2161e6b640 1 -- 172.21.15.27:0/687799404 >> v1:172.21.15.27:6789/0 conn(0x7f215c152070 legacy=0x7f215c15b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:01.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.646+0000 7f2161e6b640 1 -- 172.21.15.27:0/687799404 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c15f490 con 0x7f215c151c90 2023-12-19T18:16:01.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.647+0000 7f2161e6b640 1 --2- 172.21.15.27:0/687799404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f215c151c90 0x7f215c06b4d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f21580097a0 tx=0x7f215802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b331854023a6f05 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.648+0000 7f2160e69640 1 -- 172.21.15.27:0/687799404 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215802e070 con 0x7f215c151c90 2023-12-19T18:16:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.648+0000 7f2160e69640 1 -- 172.21.15.27:0/687799404 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21580315a0 con 0x7f215c151c90 2023-12-19T18:16:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.648+0000 7f2160e69640 1 -- 172.21.15.27:0/687799404 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158031930 con 0x7f215c151c90 2023-12-19T18:16:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.648+0000 7f2162e6d640 1 -- 172.21.15.27:0/687799404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f215c151c90 msgr2=0x7f215c06b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.648+0000 7f2162e6d640 1 --2- 172.21.15.27:0/687799404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f215c151c90 0x7f215c06b4d0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f21580097a0 tx=0x7f215802b840 comp rx=0 tx=0).stop 2023-12-19T18:16:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.648+0000 7f2162e6d640 1 -- 172.21.15.27:0/687799404 shutdown_connections 2023-12-19T18:16:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.648+0000 7f2162e6d640 1 --2- 172.21.15.27:0/687799404 >> v2:172.21.15.192:3300/0 conn(0x7f215c15e9d0 0x7f215c0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.648+0000 7f2162e6d640 1 --2- 172.21.15.27:0/687799404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f215c151c90 0x7f215c06b4d0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.648+0000 7f2162e6d640 1 -- 172.21.15.27:0/687799404 >> 172.21.15.27:0/687799404 conn(0x7f215c06a060 msgr2=0x7f215c06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.648+0000 7f2162e6d640 1 -- 172.21.15.27:0/687799404 shutdown_connections 2023-12-19T18:16:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.648+0000 7f2162e6d640 1 -- 172.21.15.27:0/687799404 wait complete. 2023-12-19T18:16:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.650+0000 7f2162e6d640 1 Processor -- start 2023-12-19T18:16:01.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.650+0000 7f2162e6d640 1 -- start start 2023-12-19T18:16:01.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.651+0000 7f2162e6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f215c151c90 0x7f215c14d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.651+0000 7f2162e6d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f215c152070 0x7f215c0c3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.651+0000 7f2162e6d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f215c15e9d0 0x7f215c0aa1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.651+0000 7f2162e6d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c0a9150 con 0x7f215c151c90 2023-12-19T18:16:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.651+0000 7f2162e6d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f215c14dde0 con 0x7f215c15e9d0 2023-12-19T18:16:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.651+0000 7f2162e6d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f215c0a93f0 con 0x7f215c152070 2023-12-19T18:16:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.652+0000 7f2161e6b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f215c151c90 0x7f215c14d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.652+0000 7f2161e6b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f215c151c90 0x7f215c14d8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48752/0 (socket says 172.21.15.27:48752) 2023-12-19T18:16:01.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.652+0000 7f2161e6b640 1 -- 172.21.15.27:0/2047224684 learned_addr learned my addr 172.21.15.27:0/2047224684 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:01.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.652+0000 7f216266c640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f215c15e9d0 0x7f215c0aa1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:01.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.652+0000 7f2161e6b640 1 -- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f215c152070 msgr2=0x7f215c0c3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:01.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.652+0000 7f2161e6b640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f215c152070 0x7f215c0c3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.652+0000 7f2161e6b640 1 -- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f215c15e9d0 msgr2=0x7f215c0aa1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.652+0000 7f2161e6b640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f215c15e9d0 0x7f215c0aa1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.652+0000 7f2161e6b640 1 -- 172.21.15.27:0/2047224684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2158009400 con 0x7f215c151c90 2023-12-19T18:16:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.652+0000 7f216266c640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f215c15e9d0 0x7f215c0aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.652+0000 7f2161e6b640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f215c151c90 0x7f215c14d8a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2158034040 tx=0x7f21580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.653+0000 7f21527fc640 1 -- 172.21.15.27:0/2047224684 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158032840 con 0x7f215c151c90 2023-12-19T18:16:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.653+0000 7f21527fc640 1 -- 172.21.15.27:0/2047224684 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21580329a0 con 0x7f215c151c90 2023-12-19T18:16:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.653+0000 7f21527fc640 1 -- 172.21.15.27:0/2047224684 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2158032d50 con 0x7f215c151c90 2023-12-19T18:16:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.653+0000 7f2162e6d640 1 -- 172.21.15.27:0/2047224684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f215c0a96d0 con 0x7f215c151c90 2023-12-19T18:16:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.653+0000 7f2162e6d640 1 -- 172.21.15.27:0/2047224684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f215c0a9c90 con 0x7f215c151c90 2023-12-19T18:16:01.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.654+0000 7f2162e6d640 1 -- 172.21.15.27:0/2047224684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f215c0a9e80 con 0x7f215c151c90 2023-12-19T18:16:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.655+0000 7f21527fc640 1 -- 172.21.15.27:0/2047224684 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f215804e430 con 0x7f215c151c90 2023-12-19T18:16:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.656+0000 7f21527fc640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f214006a460 0x7f214006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:01.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.657+0000 7f21527fc640 1 -- 172.21.15.27:0/2047224684 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f21580aa920 con 0x7f215c151c90 2023-12-19T18:16:01.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.657+0000 7f216166a640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f214006a460 0x7f214006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:01.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.658+0000 7f216166a640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f214006a460 0x7f214006c920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f214c006fd0 tx=0x7f214c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:01.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.661+0000 7f21527fc640 1 -- 172.21.15.27:0/2047224684 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2158075ee0 con 0x7f215c151c90 2023-12-19T18:16:01.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.856+0000 7f2162e6d640 1 -- 172.21.15.27:0/2047224684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f215c04e840 con 0x7f215c151c90 2023-12-19T18:16:01.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.857+0000 7f21527fc640 1 -- 172.21.15.27:0/2047224684 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2158079500 con 0x7f215c151c90 2023-12-19T18:16:01.857 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:16:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.859+0000 7f2162e6d640 1 -- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f214006a460 msgr2=0x7f214006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.859+0000 7f2162e6d640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f214006a460 0x7f214006c920 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f214c006fd0 tx=0x7f214c008040 comp rx=0 tx=0).stop 2023-12-19T18:16:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.859+0000 7f2162e6d640 1 -- 172.21.15.27:0/2047224684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f215c151c90 msgr2=0x7f215c14d8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.859+0000 7f2162e6d640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f215c151c90 0x7f215c14d8a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f2158034040 tx=0x7f21580046c0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.859+0000 7f2162e6d640 1 -- 172.21.15.27:0/2047224684 shutdown_connections 2023-12-19T18:16:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.859+0000 7f2162e6d640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f214006a460 0x7f214006c920 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.859+0000 7f2162e6d640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f215c15e9d0 0x7f215c0aa1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.859+0000 7f2162e6d640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f215c152070 0x7f215c0c3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.859+0000 7f2162e6d640 1 --2- 172.21.15.27:0/2047224684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f215c151c90 0x7f215c14d8a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.859+0000 7f2162e6d640 1 -- 172.21.15.27:0/2047224684 >> 172.21.15.27:0/2047224684 conn(0x7f215c06a060 msgr2=0x7f215c151260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.860+0000 7f2162e6d640 1 -- 172.21.15.27:0/2047224684 shutdown_connections 2023-12-19T18:16:01.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:01.860+0000 7f2162e6d640 1 -- 172.21.15.27:0/2047224684 wait complete. 2023-12-19T18:16:01.880 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:16:02.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.120+0000 7f77048ec640 1 Processor -- start 2023-12-19T18:16:02.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.120+0000 7f77048ec640 1 -- start start 2023-12-19T18:16:02.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.122+0000 7f77048ec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f77000b3980 0x7f77000cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.122+0000 7f77048ec640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77000d3850 con 0x7f77000d1a20 2023-12-19T18:16:02.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.122+0000 7f77048ec640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77000d0b10 con 0x7f770015d120 2023-12-19T18:16:02.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.122+0000 7f77048ec640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f770015d500 con 0x7f77000b3980 2023-12-19T18:16:02.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.123+0000 7f76feffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f77000d1a20 0x7f77000cd940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45094/0 (socket says 172.21.15.27:45094) 2023-12-19T18:16:02.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.123+0000 7f76feffd640 1 -- 172.21.15.27:0/2368372078 learned_addr learned my addr 172.21.15.27:0/2368372078 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:02.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.123+0000 7f76fffff640 1 --2- 172.21.15.27:0/2368372078 >> v2:172.21.15.27:3300/0 conn(0x7f77000b3980 0x7f77000cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.123+0000 7f76fe7fc640 1 -- 172.21.15.27:0/2368372078 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2262442573 0 0) 0x7f77000d3850 con 0x7f77000d1a20 2023-12-19T18:16:02.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.123+0000 7f76fe7fc640 1 -- 172.21.15.27:0/2368372078 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76ec003520 con 0x7f77000d1a20 2023-12-19T18:16:02.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.123+0000 7f76fffff640 1 -- 172.21.15.27:0/2368372078 >> v1:172.21.15.192:6789/0 conn(0x7f770015d120 legacy=0x7f77000d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.123+0000 7f76fffff640 1 -- 172.21.15.27:0/2368372078 >> v1:172.21.15.27:6789/0 conn(0x7f77000d1a20 legacy=0x7f77000cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.123+0000 7f76fffff640 1 -- 172.21.15.27:0/2368372078 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77000d1040 con 0x7f77000b3980 2023-12-19T18:16:02.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.124+0000 7f76fffff640 1 --2- 172.21.15.27:0/2368372078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77000b3980 0x7f77000cfd70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f76f80097a0 tx=0x7f76f802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8600a54890c63574 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:02.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.125+0000 7f76fe7fc640 1 -- 172.21.15.27:0/2368372078 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8030610 con 0x7f77000b3980 2023-12-19T18:16:02.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.125+0000 7f76fe7fc640 1 -- 172.21.15.27:0/2368372078 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8030770 con 0x7f77000b3980 2023-12-19T18:16:02.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.125+0000 7f76fe7fc640 1 -- 172.21.15.27:0/2368372078 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8030b00 con 0x7f77000b3980 2023-12-19T18:16:02.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.125+0000 7f77048ec640 1 -- 172.21.15.27:0/2368372078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77000b3980 msgr2=0x7f77000cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.126+0000 7f77048ec640 1 --2- 172.21.15.27:0/2368372078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77000b3980 0x7f77000cfd70 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f76f80097a0 tx=0x7f76f802b960 comp rx=0 tx=0).stop 2023-12-19T18:16:02.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.126+0000 7f77048ec640 1 -- 172.21.15.27:0/2368372078 shutdown_connections 2023-12-19T18:16:02.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.126+0000 7f77048ec640 1 --2- 172.21.15.27:0/2368372078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77000b3980 0x7f77000cfd70 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.126+0000 7f77048ec640 1 -- 172.21.15.27:0/2368372078 >> 172.21.15.27:0/2368372078 conn(0x7f77000d2d30 msgr2=0x7f77000d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:02.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.126+0000 7f77048ec640 1 -- 172.21.15.27:0/2368372078 shutdown_connections 2023-12-19T18:16:02.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.126+0000 7f77048ec640 1 -- 172.21.15.27:0/2368372078 wait complete. 2023-12-19T18:16:02.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.128+0000 7f77048ec640 1 Processor -- start 2023-12-19T18:16:02.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.128+0000 7f77048ec640 1 -- start start 2023-12-19T18:16:02.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.130+0000 7f77048ec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77000b3980 0x7f77000c93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.130+0000 7f77048ec640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77000d1a20 0x7f7700159230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.130+0000 7f77048ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770015d120 0x7f7700159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.130+0000 7f77048ec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77000b8aa0 con 0x7f770015d120 2023-12-19T18:16:02.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.130+0000 7f77048ec640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77000b84e0 con 0x7f77000d1a20 2023-12-19T18:16:02.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.130+0000 7f77048ec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77000b87c0 con 0x7f77000b3980 2023-12-19T18:16:02.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.130+0000 7f76fffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770015d120 0x7f7700159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.130+0000 7f76fffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770015d120 0x7f7700159b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45442/0 (socket says 172.21.15.27:45442) 2023-12-19T18:16:02.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.130+0000 7f76fffff640 1 -- 172.21.15.27:0/1162386595 learned_addr learned my addr 172.21.15.27:0/1162386595 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:02.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.130+0000 7f76feffd640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77000d1a20 0x7f7700159230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.130+0000 7f76ff7fe640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77000b3980 0x7f77000c93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.130+0000 7f76fffff640 1 -- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77000b3980 msgr2=0x7f77000c93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.131+0000 7f76fffff640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77000b3980 0x7f77000c93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.131+0000 7f76fffff640 1 -- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77000d1a20 msgr2=0x7f7700159230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.131+0000 7f76fffff640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77000d1a20 0x7f7700159230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.131+0000 7f76fffff640 1 -- 172.21.15.27:0/1162386595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f8009400 con 0x7f770015d120 2023-12-19T18:16:02.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.131+0000 7f76feffd640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77000d1a20 0x7f7700159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:02.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.131+0000 7f76ff7fe640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77000b3980 0x7f77000c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:02.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.131+0000 7f76fffff640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770015d120 0x7f7700159b70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f76f80098c0 tx=0x7f76f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:02.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.131+0000 7f76f7fff640 1 -- 172.21.15.27:0/1162386595 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f803e070 con 0x7f770015d120 2023-12-19T18:16:02.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.131+0000 7f76f7fff640 1 -- 172.21.15.27:0/1162386595 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76f8030e10 con 0x7f770015d120 2023-12-19T18:16:02.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.131+0000 7f76f7fff640 1 -- 172.21.15.27:0/1162386595 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f8039bd0 con 0x7f770015d120 2023-12-19T18:16:02.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.131+0000 7f77048ec640 1 -- 172.21.15.27:0/1162386595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77000c6830 con 0x7f770015d120 2023-12-19T18:16:02.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.131+0000 7f77048ec640 1 -- 172.21.15.27:0/1162386595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77000b2fa0 con 0x7f770015d120 2023-12-19T18:16:02.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.133+0000 7f77048ec640 1 -- 172.21.15.27:0/1162386595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77000c6690 con 0x7f770015d120 2023-12-19T18:16:02.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.134+0000 7f76f7fff640 1 -- 172.21.15.27:0/1162386595 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f76f8039d30 con 0x7f770015d120 2023-12-19T18:16:02.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.135+0000 7f76f7fff640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f76e006a460 0x7f76e006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.135+0000 7f76f7fff640 1 -- 172.21.15.27:0/1162386595 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f76f80aabb0 con 0x7f770015d120 2023-12-19T18:16:02.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.136+0000 7f76ff7fe640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f76e006a460 0x7f76e006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.137+0000 7f76ff7fe640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f76e006a460 0x7f76e006c920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f76f00012e0 tx=0x7f76f0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:02.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.140+0000 7f76f7fff640 1 -- 172.21.15.27:0/1162386595 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f77000c6690 con 0x7f770015d120 2023-12-19T18:16:02.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.331+0000 7f77048ec640 1 -- 172.21.15.27:0/1162386595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f77000c6690 con 0x7f770015d120 2023-12-19T18:16:02.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.332+0000 7f76f7fff640 1 -- 172.21.15.27:0/1162386595 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f77000c6690 con 0x7f770015d120 2023-12-19T18:16:02.332 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:16:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.335+0000 7f77048ec640 1 -- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f76e006a460 msgr2=0x7f76e006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.335+0000 7f77048ec640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f76e006a460 0x7f76e006c920 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f76f00012e0 tx=0x7f76f0009040 comp rx=0 tx=0).stop 2023-12-19T18:16:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.335+0000 7f77048ec640 1 -- 172.21.15.27:0/1162386595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770015d120 msgr2=0x7f7700159b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.335+0000 7f77048ec640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770015d120 0x7f7700159b70 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f76f80098c0 tx=0x7f76f80046c0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.335+0000 7f77048ec640 1 -- 172.21.15.27:0/1162386595 shutdown_connections 2023-12-19T18:16:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.335+0000 7f77048ec640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f76e006a460 0x7f76e006c920 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.335+0000 7f77048ec640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f770015d120 0x7f7700159b70 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.335+0000 7f77048ec640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77000d1a20 0x7f7700159230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.335+0000 7f77048ec640 1 --2- 172.21.15.27:0/1162386595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77000b3980 0x7f77000c93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.335+0000 7f77048ec640 1 -- 172.21.15.27:0/1162386595 >> 172.21.15.27:0/1162386595 conn(0x7f77000d2d30 msgr2=0x7f77000b1320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.335+0000 7f77048ec640 1 -- 172.21.15.27:0/1162386595 shutdown_connections 2023-12-19T18:16:02.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.335+0000 7f77048ec640 1 -- 172.21.15.27:0/1162386595 wait complete. 2023-12-19T18:16:02.355 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:16:02.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.589+0000 7fc416cdc640 1 Processor -- start 2023-12-19T18:16:02.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.589+0000 7fc416cdc640 1 -- start start 2023-12-19T18:16:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.591+0000 7fc416cdc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc41015ac50 0x7fc410070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.591+0000 7fc416cdc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc4100aef20 0x7fc41006b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.591+0000 7fc416cdc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc4100af360 0x7fc41006b850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.591+0000 7fc416cdc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100adf10 con 0x7fc4100aef20 2023-12-19T18:16:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.591+0000 7fc416cdc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100adac0 con 0x7fc4100af360 2023-12-19T18:16:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.591+0000 7fc416cdc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41006bd90 con 0x7fc41015ac50 2023-12-19T18:16:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.591+0000 7fc4154d9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc4100aef20 0x7fc41006b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.591+0000 7fc4154d9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc4100aef20 0x7fc41006b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45452/0 (socket says 172.21.15.27:45452) 2023-12-19T18:16:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.591+0000 7fc4154d9640 1 -- 172.21.15.27:0/3170668708 learned_addr learned my addr 172.21.15.27:0/3170668708 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.592+0000 7fc4154d9640 1 -- 172.21.15.27:0/3170668708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc41015ac50 msgr2=0x7fc410070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.592+0000 7fc4154d9640 1 --2- 172.21.15.27:0/3170668708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc41015ac50 0x7fc410070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.592+0000 7fc4154d9640 1 -- 172.21.15.27:0/3170668708 >> v2:172.21.15.192:3300/0 conn(0x7fc4100af360 msgr2=0x7fc41006b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.592+0000 7fc4154d9640 1 --2- 172.21.15.27:0/3170668708 >> v2:172.21.15.192:3300/0 conn(0x7fc4100af360 0x7fc41006b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.592+0000 7fc4154d9640 1 -- 172.21.15.27:0/3170668708 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4100ae160 con 0x7fc4100aef20 2023-12-19T18:16:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.593+0000 7fc4154d9640 1 --2- 172.21.15.27:0/3170668708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100aef20 0x7fc41006b310 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc4100adec0 tx=0x7fc40002b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2560519097952ee8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.593+0000 7fc414cd8640 1 -- 172.21.15.27:0/3170668708 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc400031750 con 0x7fc4100aef20 2023-12-19T18:16:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.593+0000 7fc414cd8640 1 -- 172.21.15.27:0/3170668708 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4000318b0 con 0x7fc4100aef20 2023-12-19T18:16:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.593+0000 7fc414cd8640 1 -- 172.21.15.27:0/3170668708 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc400031c40 con 0x7fc4100aef20 2023-12-19T18:16:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.594+0000 7fc416cdc640 1 -- 172.21.15.27:0/3170668708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100aef20 msgr2=0x7fc41006b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.594+0000 7fc416cdc640 1 --2- 172.21.15.27:0/3170668708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100aef20 0x7fc41006b310 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fc4100adec0 tx=0x7fc40002b7f0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.594+0000 7fc416cdc640 1 -- 172.21.15.27:0/3170668708 shutdown_connections 2023-12-19T18:16:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.594+0000 7fc416cdc640 1 --2- 172.21.15.27:0/3170668708 >> v2:172.21.15.192:3300/0 conn(0x7fc4100af360 0x7fc41006b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.594+0000 7fc416cdc640 1 --2- 172.21.15.27:0/3170668708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100aef20 0x7fc41006b310 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.594+0000 7fc416cdc640 1 --2- 172.21.15.27:0/3170668708 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc41015ac50 0x7fc410070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.594+0000 7fc416cdc640 1 -- 172.21.15.27:0/3170668708 >> 172.21.15.27:0/3170668708 conn(0x7fc41006a7d0 msgr2=0x7fc4100ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.594+0000 7fc416cdc640 1 -- 172.21.15.27:0/3170668708 shutdown_connections 2023-12-19T18:16:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.594+0000 7fc416cdc640 1 -- 172.21.15.27:0/3170668708 wait complete. 2023-12-19T18:16:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.596+0000 7fc416cdc640 1 Processor -- start 2023-12-19T18:16:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.596+0000 7fc416cdc640 1 -- start start 2023-12-19T18:16:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.597+0000 7fc416cdc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100aef20 0x7fc410150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.597+0000 7fc416cdc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4100af360 0x7fc41014ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.597+0000 7fc416cdc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc41015ac50 0x7fc4101498d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.597+0000 7fc416cdc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100ca930 con 0x7fc4100aef20 2023-12-19T18:16:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.597+0000 7fc416cdc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc410150780 con 0x7fc4100af360 2023-12-19T18:16:02.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.597+0000 7fc416cdc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100cabd0 con 0x7fc41015ac50 2023-12-19T18:16:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.598+0000 7fc415cda640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100aef20 0x7fc410150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.598+0000 7fc4164db640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc41015ac50 0x7fc4101498d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.598+0000 7fc415cda640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100aef20 0x7fc410150c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45468/0 (socket says 172.21.15.27:45468) 2023-12-19T18:16:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.598+0000 7fc415cda640 1 -- 172.21.15.27:0/3515875581 learned_addr learned my addr 172.21.15.27:0/3515875581 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.598+0000 7fc4154d9640 1 --2- 172.21.15.27:0/3515875581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4100af360 0x7fc41014ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.598+0000 7fc415cda640 1 -- 172.21.15.27:0/3515875581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc41015ac50 msgr2=0x7fc4101498d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.598+0000 7fc415cda640 1 --2- 172.21.15.27:0/3515875581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc41015ac50 0x7fc4101498d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.598+0000 7fc415cda640 1 -- 172.21.15.27:0/3515875581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4100af360 msgr2=0x7fc41014ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.598+0000 7fc415cda640 1 --2- 172.21.15.27:0/3515875581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4100af360 0x7fc41014ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.598+0000 7fc415cda640 1 -- 172.21.15.27:0/3515875581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4000094d0 con 0x7fc4100aef20 2023-12-19T18:16:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.599+0000 7fc415cda640 1 --2- 172.21.15.27:0/3515875581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100aef20 0x7fc410150c60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00b4a0 tx=0x7fc40c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.599+0000 7fc4067fc640 1 -- 172.21.15.27:0/3515875581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c010610 con 0x7fc4100aef20 2023-12-19T18:16:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.599+0000 7fc4067fc640 1 -- 172.21.15.27:0/3515875581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc40c010770 con 0x7fc4100aef20 2023-12-19T18:16:02.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.599+0000 7fc416cdc640 1 -- 172.21.15.27:0/3515875581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4100caeb0 con 0x7fc4100aef20 2023-12-19T18:16:02.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.599+0000 7fc4067fc640 1 -- 172.21.15.27:0/3515875581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40c010b20 con 0x7fc4100aef20 2023-12-19T18:16:02.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.599+0000 7fc416cdc640 1 -- 172.21.15.27:0/3515875581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41014e630 con 0x7fc4100aef20 2023-12-19T18:16:02.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.600+0000 7fc416cdc640 1 -- 172.21.15.27:0/3515875581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc41004e840 con 0x7fc4100aef20 2023-12-19T18:16:02.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.601+0000 7fc4067fc640 1 -- 172.21.15.27:0/3515875581 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc40c010c80 con 0x7fc4100aef20 2023-12-19T18:16:02.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.602+0000 7fc4067fc640 1 --2- 172.21.15.27:0/3515875581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc3f406a460 0x7fc3f406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:02.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.602+0000 7fc4067fc640 1 -- 172.21.15.27:0/3515875581 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fc40c088540 con 0x7fc4100aef20 2023-12-19T18:16:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.603+0000 7fc4154d9640 1 --2- 172.21.15.27:0/3515875581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc3f406a460 0x7fc3f406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.604+0000 7fc4154d9640 1 --2- 172.21.15.27:0/3515875581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc3f406a460 0x7fc3f406c920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc400035000 tx=0x7fc400008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:02.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.607+0000 7fc4067fc640 1 -- 172.21.15.27:0/3515875581 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc40c053b20 con 0x7fc4100aef20 2023-12-19T18:16:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.819+0000 7fc416cdc640 1 -- 172.21.15.27:0/3515875581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc41006f430 con 0x7fc4100aef20 2023-12-19T18:16:02.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.820+0000 7fc4067fc640 1 -- 172.21.15.27:0/3515875581 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc40c057140 con 0x7fc4100aef20 2023-12-19T18:16:02.820 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:16:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.822+0000 7fc416cdc640 1 -- 172.21.15.27:0/3515875581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc3f406a460 msgr2=0x7fc3f406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.822+0000 7fc416cdc640 1 --2- 172.21.15.27:0/3515875581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc3f406a460 0x7fc3f406c920 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fc400035000 tx=0x7fc400008030 comp rx=0 tx=0).stop 2023-12-19T18:16:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.822+0000 7fc416cdc640 1 -- 172.21.15.27:0/3515875581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100aef20 msgr2=0x7fc410150c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.822+0000 7fc416cdc640 1 --2- 172.21.15.27:0/3515875581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100aef20 0x7fc410150c60 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00b4a0 tx=0x7fc40c00b960 comp rx=0 tx=0).stop 2023-12-19T18:16:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.823+0000 7fc416cdc640 1 -- 172.21.15.27:0/3515875581 shutdown_connections 2023-12-19T18:16:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.823+0000 7fc416cdc640 1 --2- 172.21.15.27:0/3515875581 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc3f406a460 0x7fc3f406c920 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.823+0000 7fc416cdc640 1 --2- 172.21.15.27:0/3515875581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc41015ac50 0x7fc4101498d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.823+0000 7fc416cdc640 1 --2- 172.21.15.27:0/3515875581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4100af360 0x7fc41014ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.823+0000 7fc416cdc640 1 --2- 172.21.15.27:0/3515875581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100aef20 0x7fc410150c60 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:02.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.823+0000 7fc416cdc640 1 -- 172.21.15.27:0/3515875581 >> 172.21.15.27:0/3515875581 conn(0x7fc41006a7d0 msgr2=0x7fc4100ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:02.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.823+0000 7fc416cdc640 1 -- 172.21.15.27:0/3515875581 shutdown_connections 2023-12-19T18:16:02.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:02.823+0000 7fc416cdc640 1 -- 172.21.15.27:0/3515875581 wait complete. 2023-12-19T18:16:02.843 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-12-19T18:16:02.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:02.844 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-12-19T18:16:02.945 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:16:02.945 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:16:02.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:02.945 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-19T18:16:03.001 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:16:03.001 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:03.061 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:16:03.062 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:03.116 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:03.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.360+0000 7fcf7359e640 1 Processor -- start 2023-12-19T18:16:03.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.360+0000 7fcf7359e640 1 -- start start 2023-12-19T18:16:03.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.362+0000 7fcf7359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcf740af630 0x7fcf740afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.362+0000 7fcf7359e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcf740c4550 0x7fcf740c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.362+0000 7fcf7359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf740cc190 con 0x7fcf74154030 2023-12-19T18:16:03.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.362+0000 7fcf7359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf740b0000 con 0x7fcf740af630 2023-12-19T18:16:03.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.362+0000 7fcf7359e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf740cdc00 con 0x7fcf740c4550 2023-12-19T18:16:03.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.362+0000 7fcf7259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcf74154030 0x7fcf740cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45096/0 (socket says 172.21.15.27:45096) 2023-12-19T18:16:03.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.362+0000 7fcf7259c640 1 -- 172.21.15.27:0/2484786037 learned_addr learned my addr 172.21.15.27:0/2484786037 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:03.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.362+0000 7fcf71d9b640 1 --2- 172.21.15.27:0/2484786037 >> v2:172.21.15.27:3300/0 conn(0x7fcf740af630 0x7fcf740afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.363+0000 7fcf72d9d640 1 --2- 172.21.15.27:0/2484786037 >> v2:172.21.15.192:3300/0 conn(0x7fcf740c4550 0x7fcf740c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.363+0000 7fcf7159a640 1 -- 172.21.15.27:0/2484786037 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1094520637 0 0) 0x7fcf740cc190 con 0x7fcf74154030 2023-12-19T18:16:03.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.363+0000 7fcf7159a640 1 -- 172.21.15.27:0/2484786037 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf5c003520 con 0x7fcf74154030 2023-12-19T18:16:03.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.363+0000 7fcf71d9b640 1 -- 172.21.15.27:0/2484786037 >> v2:172.21.15.192:3300/0 conn(0x7fcf740c4550 msgr2=0x7fcf740c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:03.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.363+0000 7fcf71d9b640 1 --2- 172.21.15.27:0/2484786037 >> v2:172.21.15.192:3300/0 conn(0x7fcf740c4550 0x7fcf740c49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.363+0000 7fcf71d9b640 1 -- 172.21.15.27:0/2484786037 >> v1:172.21.15.27:6789/0 conn(0x7fcf74154030 legacy=0x7fcf740cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.363+0000 7fcf71d9b640 1 -- 172.21.15.27:0/2484786037 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf740cb2a0 con 0x7fcf740af630 2023-12-19T18:16:03.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.364+0000 7fcf71d9b640 1 --2- 172.21.15.27:0/2484786037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf740af630 0x7fcf740afac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcf680097a0 tx=0x7fcf6802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a5315477ffe9a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:03.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.365+0000 7fcf7159a640 1 -- 172.21.15.27:0/2484786037 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68031830 con 0x7fcf740af630 2023-12-19T18:16:03.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.365+0000 7fcf7159a640 1 -- 172.21.15.27:0/2484786037 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf68031990 con 0x7fcf740af630 2023-12-19T18:16:03.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.365+0000 7fcf7159a640 1 -- 172.21.15.27:0/2484786037 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68031d20 con 0x7fcf740af630 2023-12-19T18:16:03.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.365+0000 7fcf7359e640 1 -- 172.21.15.27:0/2484786037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf740af630 msgr2=0x7fcf740afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.365+0000 7fcf7359e640 1 --2- 172.21.15.27:0/2484786037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf740af630 0x7fcf740afac0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fcf680097a0 tx=0x7fcf6802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.366+0000 7fcf7359e640 1 -- 172.21.15.27:0/2484786037 shutdown_connections 2023-12-19T18:16:03.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.366+0000 7fcf7359e640 1 --2- 172.21.15.27:0/2484786037 >> v2:172.21.15.192:3300/0 conn(0x7fcf740c4550 0x7fcf740c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.366+0000 7fcf7359e640 1 --2- 172.21.15.27:0/2484786037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf740af630 0x7fcf740afac0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.366+0000 7fcf7359e640 1 -- 172.21.15.27:0/2484786037 >> 172.21.15.27:0/2484786037 conn(0x7fcf74069c70 msgr2=0x7fcf7406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:03.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.366+0000 7fcf7359e640 1 -- 172.21.15.27:0/2484786037 shutdown_connections 2023-12-19T18:16:03.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.366+0000 7fcf7359e640 1 -- 172.21.15.27:0/2484786037 wait complete. 2023-12-19T18:16:03.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.367+0000 7fcf7359e640 1 Processor -- start 2023-12-19T18:16:03.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.368+0000 7fcf7359e640 1 -- start start 2023-12-19T18:16:03.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.368+0000 7fcf7359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf740af630 0x7fcf74070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf7359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf740c4550 0x7fcf7406f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf7359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf74154030 0x7fcf74142d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf7359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf74143870 con 0x7fcf74154030 2023-12-19T18:16:03.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf7359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf74071360 con 0x7fcf740af630 2023-12-19T18:16:03.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf7359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf7406d250 con 0x7fcf740c4550 2023-12-19T18:16:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf72d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf74154030 0x7fcf74142d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf72d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf74154030 0x7fcf74142d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45492/0 (socket says 172.21.15.27:45492) 2023-12-19T18:16:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf72d9d640 1 -- 172.21.15.27:0/3354142073 learned_addr learned my addr 172.21.15.27:0/3354142073 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf7259c640 1 --2- 172.21.15.27:0/3354142073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf740af630 0x7fcf74070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf72d9d640 1 -- 172.21.15.27:0/3354142073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf740c4550 msgr2=0x7fcf7406f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf72d9d640 1 --2- 172.21.15.27:0/3354142073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf740c4550 0x7fcf7406f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf72d9d640 1 -- 172.21.15.27:0/3354142073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf740af630 msgr2=0x7fcf74070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf72d9d640 1 --2- 172.21.15.27:0/3354142073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf740af630 0x7fcf74070cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.369+0000 7fcf72d9d640 1 -- 172.21.15.27:0/3354142073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf68009400 con 0x7fcf74154030 2023-12-19T18:16:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.370+0000 7fcf72d9d640 1 --2- 172.21.15.27:0/3354142073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf74154030 0x7fcf74142d10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c007e40 tx=0x7fcf6c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.370+0000 7fcf62ffd640 1 -- 172.21.15.27:0/3354142073 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c014620 con 0x7fcf74154030 2023-12-19T18:16:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.370+0000 7fcf62ffd640 1 -- 172.21.15.27:0/3354142073 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c00ee90 con 0x7fcf74154030 2023-12-19T18:16:03.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.370+0000 7fcf62ffd640 1 -- 172.21.15.27:0/3354142073 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0149e0 con 0x7fcf74154030 2023-12-19T18:16:03.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.370+0000 7fcf7359e640 1 -- 172.21.15.27:0/3354142073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf7406f860 con 0x7fcf74154030 2023-12-19T18:16:03.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.370+0000 7fcf7359e640 1 -- 172.21.15.27:0/3354142073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf7406cf70 con 0x7fcf74154030 2023-12-19T18:16:03.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.371+0000 7fcf7359e640 1 -- 172.21.15.27:0/3354142073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7404e840 con 0x7fcf74154030 2023-12-19T18:16:03.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.372+0000 7fcf62ffd640 1 -- 172.21.15.27:0/3354142073 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcf6c014b40 con 0x7fcf74154030 2023-12-19T18:16:03.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.373+0000 7fcf62ffd640 1 --2- 172.21.15.27:0/3354142073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf5406a460 0x7fcf5406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.373+0000 7fcf62ffd640 1 -- 172.21.15.27:0/3354142073 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fcf6c0891d0 con 0x7fcf74154030 2023-12-19T18:16:03.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.374+0000 7fcf7259c640 1 --2- 172.21.15.27:0/3354142073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf5406a460 0x7fcf5406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.375+0000 7fcf7259c640 1 --2- 172.21.15.27:0/3354142073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf5406a460 0x7fcf5406c920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcf640011e0 tx=0x7fcf64009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:03.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.378+0000 7fcf62ffd640 1 -- 172.21.15.27:0/3354142073 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcf6c054830 con 0x7fcf74154030 2023-12-19T18:16:03.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.578+0000 7fcf7359e640 1 -- 172.21.15.27:0/3354142073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf740cbe10 con 0x7fcf74154030 2023-12-19T18:16:03.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.581+0000 7fcf62ffd640 1 -- 172.21.15.27:0/3354142073 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16090 (secure 0 0 0) 0x7fcf6c057e50 con 0x7fcf74154030 2023-12-19T18:16:03.582 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:03.582 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:15:43.580746+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.584+0000 7fcf7359e640 1 -- 172.21.15.27:0/3354142073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf5406a460 msgr2=0x7fcf5406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.584+0000 7fcf7359e640 1 --2- 172.21.15.27:0/3354142073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf5406a460 0x7fcf5406c920 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcf640011e0 tx=0x7fcf64009040 comp rx=0 tx=0).stop 2023-12-19T18:16:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.584+0000 7fcf7359e640 1 -- 172.21.15.27:0/3354142073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf74154030 msgr2=0x7fcf74142d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.584+0000 7fcf7359e640 1 --2- 172.21.15.27:0/3354142073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf74154030 0x7fcf74142d10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c007e40 tx=0x7fcf6c00e4b0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.584+0000 7fcf7359e640 1 -- 172.21.15.27:0/3354142073 shutdown_connections 2023-12-19T18:16:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.584+0000 7fcf7359e640 1 --2- 172.21.15.27:0/3354142073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf5406a460 0x7fcf5406c920 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.584+0000 7fcf7359e640 1 --2- 172.21.15.27:0/3354142073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf74154030 0x7fcf74142d10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.584+0000 7fcf7359e640 1 --2- 172.21.15.27:0/3354142073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf740c4550 0x7fcf7406f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.584+0000 7fcf7359e640 1 --2- 172.21.15.27:0/3354142073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf740af630 0x7fcf74070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.584+0000 7fcf7359e640 1 -- 172.21.15.27:0/3354142073 >> 172.21.15.27:0/3354142073 conn(0x7fcf74069c70 msgr2=0x7fcf740c49c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.584+0000 7fcf7359e640 1 -- 172.21.15.27:0/3354142073 shutdown_connections 2023-12-19T18:16:03.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.584+0000 7fcf7359e640 1 -- 172.21.15.27:0/3354142073 wait complete. 2023-12-19T18:16:03.606 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:16:03.606 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:16:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.844+0000 7f7cbe882640 1 Processor -- start 2023-12-19T18:16:03.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.845+0000 7f7cbe882640 1 -- start start 2023-12-19T18:16:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.846+0000 7f7cbe882640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7cb80ce210 0x7f7cb80cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.846+0000 7f7cbe882640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb80c9880 con 0x7f7cb815a0b0 2023-12-19T18:16:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.846+0000 7f7cbe882640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb80c9b20 con 0x7f7cb806cee0 2023-12-19T18:16:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.846+0000 7f7cbe882640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb80c9de0 con 0x7f7cb80ce210 2023-12-19T18:16:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.846+0000 7f7cb7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7cb815a0b0 0x7f7cb80cd400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45098/0 (socket says 172.21.15.27:45098) 2023-12-19T18:16:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.847+0000 7f7cb7fff640 1 -- 172.21.15.27:0/584762197 learned_addr learned my addr 172.21.15.27:0/584762197 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.847+0000 7f7caffff640 1 --2- 172.21.15.27:0/584762197 >> v2:172.21.15.27:3300/0 conn(0x7f7cb80ce210 0x7f7cb80cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.847+0000 7f7cb77fe640 1 -- 172.21.15.27:0/584762197 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1561342421 0 0) 0x7f7cb80c9880 con 0x7f7cb815a0b0 2023-12-19T18:16:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.847+0000 7f7caffff640 1 -- 172.21.15.27:0/584762197 >> v1:172.21.15.192:6789/0 conn(0x7f7cb806cee0 legacy=0x7f7cb8070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:03.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.847+0000 7f7caffff640 1 -- 172.21.15.27:0/584762197 >> v1:172.21.15.27:6789/0 conn(0x7f7cb815a0b0 legacy=0x7f7cb80cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.847+0000 7f7caffff640 1 -- 172.21.15.27:0/584762197 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb80cc920 con 0x7f7cb80ce210 2023-12-19T18:16:03.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.848+0000 7f7caffff640 1 --2- 172.21.15.27:0/584762197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb80ce210 0x7f7cb80cdc80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00097a0 tx=0x7f7ca002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b873b508071836b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:03.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.849+0000 7f7cb77fe640 1 -- 172.21.15.27:0/584762197 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0031830 con 0x7f7cb80ce210 2023-12-19T18:16:03.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.849+0000 7f7cb77fe640 1 -- 172.21.15.27:0/584762197 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca0031990 con 0x7f7cb80ce210 2023-12-19T18:16:03.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.849+0000 7f7cb77fe640 1 -- 172.21.15.27:0/584762197 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0031d20 con 0x7f7cb80ce210 2023-12-19T18:16:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.849+0000 7f7cbe882640 1 -- 172.21.15.27:0/584762197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb80ce210 msgr2=0x7f7cb80cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.849+0000 7f7cbe882640 1 --2- 172.21.15.27:0/584762197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb80ce210 0x7f7cb80cdc80 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00097a0 tx=0x7f7ca002b8d0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.850+0000 7f7cbe882640 1 -- 172.21.15.27:0/584762197 shutdown_connections 2023-12-19T18:16:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.850+0000 7f7cbe882640 1 --2- 172.21.15.27:0/584762197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb80ce210 0x7f7cb80cdc80 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.850+0000 7f7cbe882640 1 -- 172.21.15.27:0/584762197 >> 172.21.15.27:0/584762197 conn(0x7f7cb80cebd0 msgr2=0x7f7cb80cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.850+0000 7f7cbe882640 1 -- 172.21.15.27:0/584762197 shutdown_connections 2023-12-19T18:16:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.850+0000 7f7cbe882640 1 -- 172.21.15.27:0/584762197 wait complete. 2023-12-19T18:16:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.851+0000 7f7cbe882640 1 Processor -- start 2023-12-19T18:16:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.852+0000 7f7cbe882640 1 -- start start 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7cbe882640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb806cee0 0x7f7cb80c92a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7cbe882640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb80ce210 0x7f7cb814ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7cbe882640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cb815a0b0 0x7f7cb8158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7cbe882640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb81513b0 con 0x7f7cb80ce210 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7cbe882640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb81594c0 con 0x7f7cb806cee0 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7cbe882640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb8150b10 con 0x7f7cb815a0b0 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7caffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb80ce210 0x7f7cb814ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7caffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb80ce210 0x7f7cb814ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45518/0 (socket says 172.21.15.27:45518) 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7caffff640 1 -- 172.21.15.27:0/921131954 learned_addr learned my addr 172.21.15.27:0/921131954 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7cb7fff640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb806cee0 0x7f7cb80c92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7cbce1f640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cb815a0b0 0x7f7cb8158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7caffff640 1 -- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cb815a0b0 msgr2=0x7f7cb8158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7caffff640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cb815a0b0 0x7f7cb8158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7caffff640 1 -- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb806cee0 msgr2=0x7f7cb80c92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7caffff640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb806cee0 0x7f7cb80c92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7caffff640 1 -- 172.21.15.27:0/921131954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca0009400 con 0x7f7cb80ce210 2023-12-19T18:16:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.853+0000 7f7cb7fff640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb806cee0 0x7f7cb80c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.854+0000 7f7caffff640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb80ce210 0x7f7cb814ecf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00098c0 tx=0x7f7ca00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.854+0000 7f7cb57fa640 1 -- 172.21.15.27:0/921131954 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca003aa60 con 0x7f7cb80ce210 2023-12-19T18:16:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.854+0000 7f7cb57fa640 1 -- 172.21.15.27:0/921131954 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca003abc0 con 0x7f7cb80ce210 2023-12-19T18:16:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.854+0000 7f7cb57fa640 1 -- 172.21.15.27:0/921131954 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca00423f0 con 0x7f7cb80ce210 2023-12-19T18:16:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.854+0000 7f7cbe882640 1 -- 172.21.15.27:0/921131954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb8150df0 con 0x7f7cb80ce210 2023-12-19T18:16:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.854+0000 7f7cbe882640 1 -- 172.21.15.27:0/921131954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb80c28a0 con 0x7f7cb80ce210 2023-12-19T18:16:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.856+0000 7f7cbe882640 1 -- 172.21.15.27:0/921131954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cb8068e90 con 0x7f7cb80ce210 2023-12-19T18:16:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.857+0000 7f7cb57fa640 1 -- 172.21.15.27:0/921131954 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7ca0030380 con 0x7f7cb80ce210 2023-12-19T18:16:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.857+0000 7f7cb57fa640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c9806a430 0x7f7c9806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.858+0000 7f7cb57fa640 1 -- 172.21.15.27:0/921131954 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7ca00aa480 con 0x7f7cb80ce210 2023-12-19T18:16:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.858+0000 7f7cb7fff640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c9806a430 0x7f7c9806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:03.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.859+0000 7f7cb7fff640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c9806a430 0x7f7c9806c8f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8000f60 tx=0x7f7ca8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:03.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:03.862+0000 7f7cb57fa640 1 -- 172.21.15.27:0/921131954 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7ca0075ae0 con 0x7f7cb80ce210 2023-12-19T18:16:04.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.055+0000 7f7cbe882640 1 -- 172.21.15.27:0/921131954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7cb80ccbd0 con 0x7f7cb80ce210 2023-12-19T18:16:04.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.056+0000 7f7cb57fa640 1 -- 172.21.15.27:0/921131954 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7ca0079100 con 0x7f7cb80ce210 2023-12-19T18:16:04.056 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:16:04.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.058+0000 7f7cbe882640 1 -- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c9806a430 msgr2=0x7f7c9806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.058+0000 7f7cbe882640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c9806a430 0x7f7c9806c8f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7ca8000f60 tx=0x7f7ca8009040 comp rx=0 tx=0).stop 2023-12-19T18:16:04.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.058+0000 7f7cbe882640 1 -- 172.21.15.27:0/921131954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb80ce210 msgr2=0x7f7cb814ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.058+0000 7f7cbe882640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb80ce210 0x7f7cb814ecf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f7ca00098c0 tx=0x7f7ca00046c0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.058+0000 7f7cbe882640 1 -- 172.21.15.27:0/921131954 shutdown_connections 2023-12-19T18:16:04.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.058+0000 7f7cbe882640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c9806a430 0x7f7c9806c8f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.058+0000 7f7cbe882640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7cb815a0b0 0x7f7cb8158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.058+0000 7f7cbe882640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cb80ce210 0x7f7cb814ecf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.058+0000 7f7cbe882640 1 --2- 172.21.15.27:0/921131954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7cb806cee0 0x7f7cb80c92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.058+0000 7f7cbe882640 1 -- 172.21.15.27:0/921131954 >> 172.21.15.27:0/921131954 conn(0x7f7cb80cebd0 msgr2=0x7f7cb8151d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:04.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.058+0000 7f7cbe882640 1 -- 172.21.15.27:0/921131954 shutdown_connections 2023-12-19T18:16:04.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.059+0000 7f7cbe882640 1 -- 172.21.15.27:0/921131954 wait complete. 2023-12-19T18:16:04.077 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:16:04.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.306+0000 7f8505d5c640 1 Processor -- start 2023-12-19T18:16:04.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.306+0000 7f8505d5c640 1 -- start start 2023-12-19T18:16:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.307+0000 7f8505d5c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85000ce210 0x7f85000cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.308+0000 7f8505d5c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f850006cee0 0x7f850006d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.308+0000 7f8505d5c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85000c9880 con 0x7f850015a0b0 2023-12-19T18:16:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.308+0000 7f8505d5c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850006d8d0 con 0x7f850006cee0 2023-12-19T18:16:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.308+0000 7f8505d5c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85000c9de0 con 0x7f85000ce210 2023-12-19T18:16:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.308+0000 7f84ff7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f850015a0b0 0x7f85000cd400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45112/0 (socket says 172.21.15.27:45112) 2023-12-19T18:16:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.308+0000 7f84ff7fe640 1 -- 172.21.15.27:0/701836664 learned_addr learned my addr 172.21.15.27:0/701836664 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.308+0000 7f84fffff640 1 --2- 172.21.15.27:0/701836664 >> v2:172.21.15.27:3300/0 conn(0x7f850006cee0 0x7f850006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.308+0000 7f84f7fff640 1 --2- 172.21.15.27:0/701836664 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85000ce210 0x7f85000cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.308+0000 7f84feffd640 1 -- 172.21.15.27:0/701836664 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262093883 0 0) 0x7f85000c9880 con 0x7f850015a0b0 2023-12-19T18:16:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.309+0000 7f84feffd640 1 -- 172.21.15.27:0/701836664 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f84ec003520 con 0x7f850015a0b0 2023-12-19T18:16:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.309+0000 7f84fffff640 1 -- 172.21.15.27:0/701836664 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85000ce210 msgr2=0x7f85000cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:04.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.309+0000 7f84fffff640 1 --2- 172.21.15.27:0/701836664 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85000ce210 0x7f85000cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.309+0000 7f84fffff640 1 -- 172.21.15.27:0/701836664 >> v1:172.21.15.27:6789/0 conn(0x7f850015a0b0 legacy=0x7f85000cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.309+0000 7f84fffff640 1 -- 172.21.15.27:0/701836664 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85000c9b20 con 0x7f850006cee0 2023-12-19T18:16:04.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.309+0000 7f84f7fff640 1 --2- 172.21.15.27:0/701836664 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85000ce210 0x7f85000cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:04.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.310+0000 7f84fffff640 1 --2- 172.21.15.27:0/701836664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850006cee0 0x7f850006d390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009d90 tx=0x7f84f802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcae2df0ebdf0d84 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:04.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.310+0000 7f84feffd640 1 -- 172.21.15.27:0/701836664 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f802cc50 con 0x7f850006cee0 2023-12-19T18:16:04.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.310+0000 7f84feffd640 1 -- 172.21.15.27:0/701836664 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f802cdb0 con 0x7f850006cee0 2023-12-19T18:16:04.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.310+0000 7f84feffd640 1 -- 172.21.15.27:0/701836664 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f8034660 con 0x7f850006cee0 2023-12-19T18:16:04.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.311+0000 7f8505d5c640 1 -- 172.21.15.27:0/701836664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850006cee0 msgr2=0x7f850006d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.311+0000 7f8505d5c640 1 --2- 172.21.15.27:0/701836664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850006cee0 0x7f850006d390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009d90 tx=0x7f84f802bea0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.311+0000 7f8505d5c640 1 -- 172.21.15.27:0/701836664 shutdown_connections 2023-12-19T18:16:04.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.311+0000 7f8505d5c640 1 --2- 172.21.15.27:0/701836664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850006cee0 0x7f850006d390 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.311+0000 7f8505d5c640 1 --2- 172.21.15.27:0/701836664 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85000ce210 0x7f85000cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.311+0000 7f8505d5c640 1 -- 172.21.15.27:0/701836664 >> 172.21.15.27:0/701836664 conn(0x7f85000cebd0 msgr2=0x7f85000cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:04.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.311+0000 7f8505d5c640 1 -- 172.21.15.27:0/701836664 shutdown_connections 2023-12-19T18:16:04.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.311+0000 7f8505d5c640 1 -- 172.21.15.27:0/701836664 wait complete. 2023-12-19T18:16:04.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.313+0000 7f8505d5c640 1 Processor -- start 2023-12-19T18:16:04.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.313+0000 7f8505d5c640 1 -- start start 2023-12-19T18:16:04.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.314+0000 7f8505d5c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f850006cee0 0x7f850015d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.314+0000 7f8505d5c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85000ce210 0x7f85000b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.314+0000 7f8505d5c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850015a0b0 0x7f850014df30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.314+0000 7f8505d5c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85000b1eb0 con 0x7f85000ce210 2023-12-19T18:16:04.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.314+0000 7f8505d5c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85000b20c0 con 0x7f850015a0b0 2023-12-19T18:16:04.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.314+0000 7f8505d5c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85000b23a0 con 0x7f850006cee0 2023-12-19T18:16:04.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.314+0000 7f84f7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85000ce210 0x7f85000b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.314+0000 7f84f7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85000ce210 0x7f85000b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45536/0 (socket says 172.21.15.27:45536) 2023-12-19T18:16:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.314+0000 7f84f7fff640 1 -- 172.21.15.27:0/255113829 learned_addr learned my addr 172.21.15.27:0/255113829 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.314+0000 7f84ff7fe640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f850006cee0 0x7f850015d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.315+0000 7f84fffff640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850015a0b0 0x7f850014df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.315+0000 7f84f7fff640 1 -- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f850006cee0 msgr2=0x7f850015d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.315+0000 7f84f7fff640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f850006cee0 0x7f850015d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.315+0000 7f84f7fff640 1 -- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850015a0b0 msgr2=0x7f850014df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.315+0000 7f84f7fff640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850015a0b0 0x7f850014df30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.315+0000 7f84f7fff640 1 -- 172.21.15.27:0/255113829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f8009a70 con 0x7f85000ce210 2023-12-19T18:16:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.315+0000 7f84f7fff640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85000ce210 0x7f85000b1970 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f84e800e7e0 tx=0x7f84e800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.315+0000 7f84fcff9640 1 -- 172.21.15.27:0/255113829 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e800c6d0 con 0x7f85000ce210 2023-12-19T18:16:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.315+0000 7f84fcff9640 1 -- 172.21.15.27:0/255113829 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e800c830 con 0x7f85000ce210 2023-12-19T18:16:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.315+0000 7f84fcff9640 1 -- 172.21.15.27:0/255113829 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e800cbe0 con 0x7f85000ce210 2023-12-19T18:16:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.315+0000 7f8505d5c640 1 -- 172.21.15.27:0/255113829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850014fae0 con 0x7f85000ce210 2023-12-19T18:16:04.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.315+0000 7f8505d5c640 1 -- 172.21.15.27:0/255113829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850014e470 con 0x7f85000ce210 2023-12-19T18:16:04.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.317+0000 7f8505d5c640 1 -- 172.21.15.27:0/255113829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85000b25c0 con 0x7f85000ce210 2023-12-19T18:16:04.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.318+0000 7f84fcff9640 1 -- 172.21.15.27:0/255113829 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f84e800cd40 con 0x7f85000ce210 2023-12-19T18:16:04.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.318+0000 7f84fcff9640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f84e006a430 0x7f84e006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.319+0000 7f84fcff9640 1 -- 172.21.15.27:0/255113829 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f84e8088a50 con 0x7f85000ce210 2023-12-19T18:16:04.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.319+0000 7f84ff7fe640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f84e006a430 0x7f84e006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.320+0000 7f84ff7fe640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f84e006a430 0x7f84e006c8f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f84f00011e0 tx=0x7f84f0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:04.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.323+0000 7f84fcff9640 1 -- 172.21.15.27:0/255113829 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f84e80540b0 con 0x7f85000ce210 2023-12-19T18:16:04.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.514+0000 7f8505d5c640 1 -- 172.21.15.27:0/255113829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8500068e90 con 0x7f85000ce210 2023-12-19T18:16:04.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.515+0000 7f84fcff9640 1 -- 172.21.15.27:0/255113829 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f84e8021020 con 0x7f85000ce210 2023-12-19T18:16:04.515 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:16:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.517+0000 7f8505d5c640 1 -- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f84e006a430 msgr2=0x7f84e006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.517+0000 7f8505d5c640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f84e006a430 0x7f84e006c8f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f84f00011e0 tx=0x7f84f0009040 comp rx=0 tx=0).stop 2023-12-19T18:16:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.517+0000 7f8505d5c640 1 -- 172.21.15.27:0/255113829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85000ce210 msgr2=0x7f85000b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.517+0000 7f8505d5c640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85000ce210 0x7f85000b1970 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f84e800e7e0 tx=0x7f84e800eca0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.518+0000 7f8505d5c640 1 -- 172.21.15.27:0/255113829 shutdown_connections 2023-12-19T18:16:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.518+0000 7f8505d5c640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f84e006a430 0x7f84e006c8f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.518+0000 7f8505d5c640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850015a0b0 0x7f850014df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.518+0000 7f8505d5c640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85000ce210 0x7f85000b1970 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.518+0000 7f8505d5c640 1 --2- 172.21.15.27:0/255113829 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f850006cee0 0x7f850015d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.518+0000 7f8505d5c640 1 -- 172.21.15.27:0/255113829 >> 172.21.15.27:0/255113829 conn(0x7f85000cebd0 msgr2=0x7f8500151920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:04.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.518+0000 7f8505d5c640 1 -- 172.21.15.27:0/255113829 shutdown_connections 2023-12-19T18:16:04.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.518+0000 7f8505d5c640 1 -- 172.21.15.27:0/255113829 wait complete. 2023-12-19T18:16:04.536 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:16:04.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.762+0000 7ff55e526640 1 Processor -- start 2023-12-19T18:16:04.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.762+0000 7ff55e526640 1 -- start start 2023-12-19T18:16:04.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.764+0000 7ff55e526640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff558151c90 0x7ff55806b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.764+0000 7ff55e526640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5580cf550 0x7ff55815b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.764+0000 7ff55e526640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff55815b3e0 con 0x7ff55815e9d0 2023-12-19T18:16:04.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.764+0000 7ff55e526640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff55815eed0 con 0x7ff558151c90 2023-12-19T18:16:04.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.764+0000 7ff55e526640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff55815f1b0 con 0x7ff5580cf550 2023-12-19T18:16:04.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.765+0000 7ff55d524640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff558151c90 0x7ff55806b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.765+0000 7ff55dd25640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5580cf550 0x7ff55815b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.765+0000 7ff55d524640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff558151c90 0x7ff55806b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:38938/0 (socket says 172.21.15.27:38938) 2023-12-19T18:16:04.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.765+0000 7ff55dd25640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5580cf550 0x7ff55815b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:46846/0 (socket says 172.21.15.27:46846) 2023-12-19T18:16:04.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.765+0000 7ff55d524640 1 -- 172.21.15.27:0/943751491 learned_addr learned my addr 172.21.15.27:0/943751491 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.765+0000 7ff55d524640 1 -- 172.21.15.27:0/943751491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5580cf550 msgr2=0x7ff55815b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.765+0000 7ff55d524640 1 --2- 172.21.15.27:0/943751491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5580cf550 0x7ff55815b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.765+0000 7ff55d524640 1 -- 172.21.15.27:0/943751491 >> v1:172.21.15.192:6789/0 conn(0x7ff55815e9d0 legacy=0x7ff5580cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.766+0000 7ff55d524640 1 -- 172.21.15.27:0/943751491 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff55815f490 con 0x7ff558151c90 2023-12-19T18:16:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.767+0000 7ff55d524640 1 --2- 172.21.15.27:0/943751491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff558151c90 0x7ff55806b4d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff5540098a0 tx=0x7ff55402b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86f4ad7f378479af server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.767+0000 7ff54ffff640 1 -- 172.21.15.27:0/943751491 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554031640 con 0x7ff558151c90 2023-12-19T18:16:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.767+0000 7ff54ffff640 1 -- 172.21.15.27:0/943751491 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5540317a0 con 0x7ff558151c90 2023-12-19T18:16:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.767+0000 7ff54ffff640 1 -- 172.21.15.27:0/943751491 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff554031b30 con 0x7ff558151c90 2023-12-19T18:16:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.768+0000 7ff55e526640 1 -- 172.21.15.27:0/943751491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff558151c90 msgr2=0x7ff55806b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.768+0000 7ff55e526640 1 --2- 172.21.15.27:0/943751491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff558151c90 0x7ff55806b4d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff5540098a0 tx=0x7ff55402b8f0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.768+0000 7ff55e526640 1 -- 172.21.15.27:0/943751491 shutdown_connections 2023-12-19T18:16:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.768+0000 7ff55e526640 1 --2- 172.21.15.27:0/943751491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5580cf550 0x7ff55815b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.768+0000 7ff55e526640 1 --2- 172.21.15.27:0/943751491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff558151c90 0x7ff55806b4d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.768+0000 7ff55e526640 1 -- 172.21.15.27:0/943751491 >> 172.21.15.27:0/943751491 conn(0x7ff55806a060 msgr2=0x7ff55806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:04.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.768+0000 7ff55e526640 1 -- 172.21.15.27:0/943751491 shutdown_connections 2023-12-19T18:16:04.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.768+0000 7ff55e526640 1 -- 172.21.15.27:0/943751491 wait complete. 2023-12-19T18:16:04.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.770+0000 7ff55e526640 1 Processor -- start 2023-12-19T18:16:04.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.770+0000 7ff55e526640 1 -- start start 2023-12-19T18:16:04.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.771+0000 7ff55e526640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5580cf550 0x7ff5581f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.771+0000 7ff55e526640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff558151c90 0x7ff55815ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.771+0000 7ff55e526640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff55815e9d0 0x7ff5581442a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.771+0000 7ff55e526640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5581f2720 con 0x7ff5580cf550 2023-12-19T18:16:04.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.771+0000 7ff55e526640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff558145dc0 con 0x7ff55815e9d0 2023-12-19T18:16:04.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.771+0000 7ff55e526640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5581460a0 con 0x7ff558151c90 2023-12-19T18:16:04.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.772+0000 7ff55d524640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5580cf550 0x7ff5581f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.772+0000 7ff55d524640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5580cf550 0x7ff5581f21e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45540/0 (socket says 172.21.15.27:45540) 2023-12-19T18:16:04.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.772+0000 7ff55d524640 1 -- 172.21.15.27:0/1865692023 learned_addr learned my addr 172.21.15.27:0/1865692023 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:04.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.772+0000 7ff55dd25640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff55815e9d0 0x7ff5581442a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.772+0000 7ff55cd23640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff558151c90 0x7ff55815ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.772+0000 7ff55d524640 1 -- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff558151c90 msgr2=0x7ff55815ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.772+0000 7ff55d524640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff558151c90 0x7ff55815ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.772+0000 7ff55d524640 1 -- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff55815e9d0 msgr2=0x7ff5581442a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.772+0000 7ff55d524640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff55815e9d0 0x7ff5581442a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.772+0000 7ff55d524640 1 -- 172.21.15.27:0/1865692023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5540094d0 con 0x7ff5580cf550 2023-12-19T18:16:04.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.773+0000 7ff55d524640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5580cf550 0x7ff5581f21e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff5540098a0 tx=0x7ff5540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:04.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.773+0000 7ff54dffb640 1 -- 172.21.15.27:0/1865692023 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55403a930 con 0x7ff5580cf550 2023-12-19T18:16:04.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.773+0000 7ff54dffb640 1 -- 172.21.15.27:0/1865692023 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff55403aa90 con 0x7ff5580cf550 2023-12-19T18:16:04.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.773+0000 7ff54dffb640 1 -- 172.21.15.27:0/1865692023 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff55403ae40 con 0x7ff5580cf550 2023-12-19T18:16:04.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.773+0000 7ff55e526640 1 -- 172.21.15.27:0/1865692023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff558146380 con 0x7ff5580cf550 2023-12-19T18:16:04.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.773+0000 7ff55e526640 1 -- 172.21.15.27:0/1865692023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff558146940 con 0x7ff5580cf550 2023-12-19T18:16:04.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.775+0000 7ff55e526640 1 -- 172.21.15.27:0/1865692023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff558146b30 con 0x7ff5580cf550 2023-12-19T18:16:04.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.776+0000 7ff54dffb640 1 -- 172.21.15.27:0/1865692023 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff55402f050 con 0x7ff5580cf550 2023-12-19T18:16:04.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.777+0000 7ff54dffb640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff53c06a430 0x7ff53c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:04.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.777+0000 7ff54dffb640 1 -- 172.21.15.27:0/1865692023 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff5540aa930 con 0x7ff5580cf550 2023-12-19T18:16:04.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.777+0000 7ff55cd23640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff53c06a430 0x7ff53c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:04.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.778+0000 7ff55cd23640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff53c06a430 0x7ff53c06c8f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5580ce170 tx=0x7ff548009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:04.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.781+0000 7ff54dffb640 1 -- 172.21.15.27:0/1865692023 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff554075f00 con 0x7ff5580cf550 2023-12-19T18:16:04.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.975+0000 7ff55e526640 1 -- 172.21.15.27:0/1865692023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff55804e840 con 0x7ff5580cf550 2023-12-19T18:16:04.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.976+0000 7ff54dffb640 1 -- 172.21.15.27:0/1865692023 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff554035020 con 0x7ff5580cf550 2023-12-19T18:16:04.977 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:16:04.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.979+0000 7ff55e526640 1 -- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff53c06a430 msgr2=0x7ff53c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.979+0000 7ff55e526640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff53c06a430 0x7ff53c06c8f0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7ff5580ce170 tx=0x7ff548009040 comp rx=0 tx=0).stop 2023-12-19T18:16:04.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.979+0000 7ff55e526640 1 -- 172.21.15.27:0/1865692023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5580cf550 msgr2=0x7ff5581f21e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:04.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.979+0000 7ff55e526640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5580cf550 0x7ff5581f21e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7ff5540098a0 tx=0x7ff5540046c0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.979+0000 7ff55e526640 1 -- 172.21.15.27:0/1865692023 shutdown_connections 2023-12-19T18:16:04.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.979+0000 7ff55e526640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff53c06a430 0x7ff53c06c8f0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.979+0000 7ff55e526640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff55815e9d0 0x7ff5581442a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.979+0000 7ff55e526640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff558151c90 0x7ff55815ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.979+0000 7ff55e526640 1 --2- 172.21.15.27:0/1865692023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5580cf550 0x7ff5581f21e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:04.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.979+0000 7ff55e526640 1 -- 172.21.15.27:0/1865692023 >> 172.21.15.27:0/1865692023 conn(0x7ff55806a060 msgr2=0x7ff558150810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:04.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.979+0000 7ff55e526640 1 -- 172.21.15.27:0/1865692023 shutdown_connections 2023-12-19T18:16:04.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:04.979+0000 7ff55e526640 1 -- 172.21.15.27:0/1865692023 wait complete. 2023-12-19T18:16:04.999 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:16:05.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.233+0000 7f7d2bfff640 1 Processor -- start 2023-12-19T18:16:05.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.234+0000 7f7d2bfff640 1 -- start start 2023-12-19T18:16:05.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.235+0000 7f7d2bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7d2c15ac50 0x7f7d2c070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.235+0000 7f7d2bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d2c0af360 0x7f7d2c06ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.235+0000 7f7d2bfff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d2c0adf10 con 0x7f7d2c0aef20 2023-12-19T18:16:05.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.235+0000 7f7d2bfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d2c0adac0 con 0x7f7d2c15ac50 2023-12-19T18:16:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.235+0000 7f7d2bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0ae160 con 0x7f7d2c0af360 2023-12-19T18:16:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.236+0000 7f7d2affd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7d2c15ac50 0x7f7d2c070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.236+0000 7f7d2affd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7d2c15ac50 0x7f7d2c070fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:38954/0 (socket says 172.21.15.27:38954) 2023-12-19T18:16:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.236+0000 7f7d2affd640 1 -- 172.21.15.27:0/1143706989 learned_addr learned my addr 172.21.15.27:0/1143706989 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.236+0000 7f7d2b7fe640 1 --2- 172.21.15.27:0/1143706989 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d2c0af360 0x7f7d2c06ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.237+0000 7f7d2affd640 1 -- 172.21.15.27:0/1143706989 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d2c0af360 msgr2=0x7f7d2c06ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.237+0000 7f7d2affd640 1 --2- 172.21.15.27:0/1143706989 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d2c0af360 0x7f7d2c06ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.237+0000 7f7d2affd640 1 -- 172.21.15.27:0/1143706989 >> v1:172.21.15.192:6789/0 conn(0x7f7d2c0aef20 legacy=0x7f7d2c06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.237+0000 7f7d2affd640 1 -- 172.21.15.27:0/1143706989 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c0ae440 con 0x7f7d2c15ac50 2023-12-19T18:16:05.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.238+0000 7f7d2affd640 1 --2- 172.21.15.27:0/1143706989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d2c15ac50 0x7f7d2c070fb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009e50 tx=0x7f7d1c02be40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4fa05642907e4eff server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:05.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.239+0000 7f7d29ffb640 1 -- 172.21.15.27:0/1143706989 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c02c9e0 con 0x7f7d2c15ac50 2023-12-19T18:16:05.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.239+0000 7f7d29ffb640 1 -- 172.21.15.27:0/1143706989 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c02cb40 con 0x7f7d2c15ac50 2023-12-19T18:16:05.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.239+0000 7f7d29ffb640 1 -- 172.21.15.27:0/1143706989 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c0343f0 con 0x7f7d2c15ac50 2023-12-19T18:16:05.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.239+0000 7f7d2bfff640 1 -- 172.21.15.27:0/1143706989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d2c15ac50 msgr2=0x7f7d2c070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.239+0000 7f7d2bfff640 1 --2- 172.21.15.27:0/1143706989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d2c15ac50 0x7f7d2c070fb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c009e50 tx=0x7f7d1c02be40 comp rx=0 tx=0).stop 2023-12-19T18:16:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.239+0000 7f7d2bfff640 1 -- 172.21.15.27:0/1143706989 shutdown_connections 2023-12-19T18:16:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.239+0000 7f7d2bfff640 1 --2- 172.21.15.27:0/1143706989 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d2c0af360 0x7f7d2c06ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.239+0000 7f7d2bfff640 1 --2- 172.21.15.27:0/1143706989 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d2c15ac50 0x7f7d2c070fb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.239+0000 7f7d2bfff640 1 -- 172.21.15.27:0/1143706989 >> 172.21.15.27:0/1143706989 conn(0x7f7d2c06a7d0 msgr2=0x7f7d2c0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.240+0000 7f7d2bfff640 1 -- 172.21.15.27:0/1143706989 shutdown_connections 2023-12-19T18:16:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.240+0000 7f7d2bfff640 1 -- 172.21.15.27:0/1143706989 wait complete. 2023-12-19T18:16:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.241+0000 7f7d2bfff640 1 Processor -- start 2023-12-19T18:16:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.242+0000 7f7d2bfff640 1 -- start start 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.242+0000 7f7d2bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c0aef20 0x7f7d2c0b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d2c0af360 0x7f7d2c1504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d2c15ac50 0x7f7d2c1584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2bfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c157d00 con 0x7f7d2c0aef20 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2bfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0b1b40 con 0x7f7d2c15ac50 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c0b1e20 con 0x7f7d2c0af360 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c0aef20 0x7f7d2c0b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c0aef20 0x7f7d2c0b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45544/0 (socket says 172.21.15.27:45544) 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2affd640 1 -- 172.21.15.27:0/2929174370 learned_addr learned my addr 172.21.15.27:0/2929174370 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2a7fc640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d2c0af360 0x7f7d2c1504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2b7fe640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d2c15ac50 0x7f7d2c1584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2affd640 1 -- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d2c0af360 msgr2=0x7f7d2c1504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2affd640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d2c0af360 0x7f7d2c1504f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2affd640 1 -- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d2c15ac50 msgr2=0x7f7d2c1584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2affd640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d2c15ac50 0x7f7d2c1584e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.243+0000 7f7d2affd640 1 -- 172.21.15.27:0/2929174370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d1c009b30 con 0x7f7d2c0aef20 2023-12-19T18:16:05.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.244+0000 7f7d2affd640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c0aef20 0x7f7d2c0b14a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c000f80 tx=0x7f7d1c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:05.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.244+0000 7f7d23fff640 1 -- 172.21.15.27:0/2929174370 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c009510 con 0x7f7d2c0aef20 2023-12-19T18:16:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.244+0000 7f7d23fff640 1 -- 172.21.15.27:0/2929174370 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d1c03e450 con 0x7f7d2c0aef20 2023-12-19T18:16:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.244+0000 7f7d23fff640 1 -- 172.21.15.27:0/2929174370 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d1c03e800 con 0x7f7d2c0aef20 2023-12-19T18:16:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.244+0000 7f7d2bfff640 1 -- 172.21.15.27:0/2929174370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c0b2100 con 0x7f7d2c0aef20 2023-12-19T18:16:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.244+0000 7f7d2bfff640 1 -- 172.21.15.27:0/2929174370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c1f2e50 con 0x7f7d2c0aef20 2023-12-19T18:16:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.245+0000 7f7d2bfff640 1 -- 172.21.15.27:0/2929174370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2c04e840 con 0x7f7d2c0aef20 2023-12-19T18:16:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.246+0000 7f7d23fff640 1 -- 172.21.15.27:0/2929174370 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7d1c03e960 con 0x7f7d2c0aef20 2023-12-19T18:16:05.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.248+0000 7f7d23fff640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d0c06a430 0x7f7d0c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.248+0000 7f7d23fff640 1 -- 172.21.15.27:0/2929174370 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7d1c0ab690 con 0x7f7d2c0aef20 2023-12-19T18:16:05.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.248+0000 7f7d2a7fc640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d0c06a430 0x7f7d0c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.249+0000 7f7d2a7fc640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d0c06a430 0x7f7d0c06c8f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c1580b0 tx=0x7f7d14009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:05.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.253+0000 7f7d23fff640 1 -- 172.21.15.27:0/2929174370 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7d1c076c70 con 0x7f7d2c0aef20 2023-12-19T18:16:05.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.466+0000 7f7d2bfff640 1 -- 172.21.15.27:0/2929174370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f7d2c0c9030 con 0x7f7d2c0aef20 2023-12-19T18:16:05.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.467+0000 7f7d23fff640 1 -- 172.21.15.27:0/2929174370 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f7d1c07a290 con 0x7f7d2c0aef20 2023-12-19T18:16:05.467 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:16:05.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.470+0000 7f7d2bfff640 1 -- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d0c06a430 msgr2=0x7f7d0c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.470+0000 7f7d2bfff640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d0c06a430 0x7f7d0c06c8f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7d2c1580b0 tx=0x7f7d14009040 comp rx=0 tx=0).stop 2023-12-19T18:16:05.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.470+0000 7f7d2bfff640 1 -- 172.21.15.27:0/2929174370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c0aef20 msgr2=0x7f7d2c0b14a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.470+0000 7f7d2bfff640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c0aef20 0x7f7d2c0b14a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7d1c000f80 tx=0x7f7d1c004a40 comp rx=0 tx=0).stop 2023-12-19T18:16:05.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.470+0000 7f7d2bfff640 1 -- 172.21.15.27:0/2929174370 shutdown_connections 2023-12-19T18:16:05.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.470+0000 7f7d2bfff640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d0c06a430 0x7f7d0c06c8f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.470+0000 7f7d2bfff640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7d2c15ac50 0x7f7d2c1584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.470+0000 7f7d2bfff640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7d2c0af360 0x7f7d2c1504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.470+0000 7f7d2bfff640 1 --2- 172.21.15.27:0/2929174370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c0aef20 0x7f7d2c0b14a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.470+0000 7f7d2bfff640 1 -- 172.21.15.27:0/2929174370 >> 172.21.15.27:0/2929174370 conn(0x7f7d2c06a7d0 msgr2=0x7f7d2c06b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:05.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.470+0000 7f7d2bfff640 1 -- 172.21.15.27:0/2929174370 shutdown_connections 2023-12-19T18:16:05.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.470+0000 7f7d2bfff640 1 -- 172.21.15.27:0/2929174370 wait complete. 2023-12-19T18:16:05.491 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:16:05.491 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:05.553 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:05.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.794+0000 7fb369900640 1 Processor -- start 2023-12-19T18:16:05.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.794+0000 7fb369900640 1 -- start start 2023-12-19T18:16:05.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.795+0000 7fb369900640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb3640cbce0 0x7fb3640ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.795+0000 7fb369900640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3640adc20 0x7fb36406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.795+0000 7fb369900640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3640cb850 con 0x7fb36415af70 2023-12-19T18:16:05.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.795+0000 7fb369900640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb36415b350 con 0x7fb3640cbce0 2023-12-19T18:16:05.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.795+0000 7fb369900640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640cac40 con 0x7fb3640adc20 2023-12-19T18:16:05.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.796+0000 7fb35bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb3640cbce0 0x7fb3640ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.796+0000 7fb3637fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3640adc20 0x7fb36406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.796+0000 7fb35bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb3640cbce0 0x7fb3640ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:38970/0 (socket says 172.21.15.27:38970) 2023-12-19T18:16:05.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.796+0000 7fb35bfff640 1 -- 172.21.15.27:0/1642967355 learned_addr learned my addr 172.21.15.27:0/1642967355 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:05.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.796+0000 7fb362ffd640 1 --1- 172.21.15.27:0/1642967355 >> v1:172.21.15.192:6789/0 conn(0x7fb36415af70 0x7fb3640ad510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:43540/0 (socket says 172.21.15.27:43540) 2023-12-19T18:16:05.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.797+0000 7fb35bfff640 1 -- 172.21.15.27:0/1642967355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3640adc20 msgr2=0x7fb36406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:05.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.797+0000 7fb35bfff640 1 --2- 172.21.15.27:0/1642967355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3640adc20 0x7fb36406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.797+0000 7fb35bfff640 1 -- 172.21.15.27:0/1642967355 >> v1:172.21.15.192:6789/0 conn(0x7fb36415af70 legacy=0x7fb3640ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.797+0000 7fb35bfff640 1 -- 172.21.15.27:0/1642967355 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3640caf20 con 0x7fb3640cbce0 2023-12-19T18:16:05.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.799+0000 7fb35bfff640 1 --2- 172.21.15.27:0/1642967355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3640cbce0 0x7fb3640ca540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009870 tx=0x7fb34c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a23a5cd8c27db6e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:05.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.800+0000 7fb3627fc640 1 -- 172.21.15.27:0/1642967355 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c030840 con 0x7fb3640cbce0 2023-12-19T18:16:05.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.800+0000 7fb3627fc640 1 -- 172.21.15.27:0/1642967355 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c0309a0 con 0x7fb3640cbce0 2023-12-19T18:16:05.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.800+0000 7fb3627fc640 1 -- 172.21.15.27:0/1642967355 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c030d30 con 0x7fb3640cbce0 2023-12-19T18:16:05.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.800+0000 7fb369900640 1 -- 172.21.15.27:0/1642967355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3640cbce0 msgr2=0x7fb3640ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.800+0000 7fb369900640 1 --2- 172.21.15.27:0/1642967355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3640cbce0 0x7fb3640ca540 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb34c009870 tx=0x7fb34c02b980 comp rx=0 tx=0).stop 2023-12-19T18:16:05.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.800+0000 7fb369900640 1 -- 172.21.15.27:0/1642967355 shutdown_connections 2023-12-19T18:16:05.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.800+0000 7fb369900640 1 --2- 172.21.15.27:0/1642967355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb3640adc20 0x7fb36406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.800+0000 7fb369900640 1 --2- 172.21.15.27:0/1642967355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3640cbce0 0x7fb3640ca540 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.800+0000 7fb369900640 1 -- 172.21.15.27:0/1642967355 >> 172.21.15.27:0/1642967355 conn(0x7fb36406a330 msgr2=0x7fb3640b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:05.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.800+0000 7fb369900640 1 -- 172.21.15.27:0/1642967355 shutdown_connections 2023-12-19T18:16:05.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.800+0000 7fb369900640 1 -- 172.21.15.27:0/1642967355 wait complete. 2023-12-19T18:16:05.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.802+0000 7fb369900640 1 Processor -- start 2023-12-19T18:16:05.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.802+0000 7fb369900640 1 -- start start 2023-12-19T18:16:05.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.803+0000 7fb369900640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3640adc20 0x7fb36415ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.803+0000 7fb369900640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3640cbce0 0x7fb3640c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.803+0000 7fb369900640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb36415af70 0x7fb3640b2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.803+0000 7fb369900640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36415e410 con 0x7fb3640cbce0 2023-12-19T18:16:05.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.803+0000 7fb369900640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36415c380 con 0x7fb3640adc20 2023-12-19T18:16:05.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.803+0000 7fb369900640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3640c2cc0 con 0x7fb36415af70 2023-12-19T18:16:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.804+0000 7fb35bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3640cbce0 0x7fb3640c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.804+0000 7fb35bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3640cbce0 0x7fb3640c3a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45552/0 (socket says 172.21.15.27:45552) 2023-12-19T18:16:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.804+0000 7fb35bfff640 1 -- 172.21.15.27:0/256593999 learned_addr learned my addr 172.21.15.27:0/256593999 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.804+0000 7fb362ffd640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3640adc20 0x7fb36415ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.804+0000 7fb3637fe640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb36415af70 0x7fb3640b2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.804+0000 7fb35bfff640 1 -- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb36415af70 msgr2=0x7fb3640b2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.804+0000 7fb35bfff640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb36415af70 0x7fb3640b2020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.804+0000 7fb35bfff640 1 -- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3640adc20 msgr2=0x7fb36415ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.804+0000 7fb35bfff640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3640adc20 0x7fb36415ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.804+0000 7fb35bfff640 1 -- 172.21.15.27:0/256593999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb34c0094d0 con 0x7fb3640cbce0 2023-12-19T18:16:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.805+0000 7fb35bfff640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3640cbce0 0x7fb3640c3a80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb34c000c00 tx=0x7fb34c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.805+0000 7fb360ff9640 1 -- 172.21.15.27:0/256593999 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c039c90 con 0x7fb3640cbce0 2023-12-19T18:16:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.805+0000 7fb360ff9640 1 -- 172.21.15.27:0/256593999 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c039df0 con 0x7fb3640cbce0 2023-12-19T18:16:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.805+0000 7fb369900640 1 -- 172.21.15.27:0/256593999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3640c2fa0 con 0x7fb3640cbce0 2023-12-19T18:16:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.805+0000 7fb360ff9640 1 -- 172.21.15.27:0/256593999 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c043640 con 0x7fb3640cbce0 2023-12-19T18:16:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.805+0000 7fb369900640 1 -- 172.21.15.27:0/256593999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3640c3560 con 0x7fb3640cbce0 2023-12-19T18:16:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.806+0000 7fb369900640 1 -- 172.21.15.27:0/256593999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3640c3750 con 0x7fb3640cbce0 2023-12-19T18:16:05.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.807+0000 7fb360ff9640 1 -- 172.21.15.27:0/256593999 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb34c0437a0 con 0x7fb3640cbce0 2023-12-19T18:16:05.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.809+0000 7fb360ff9640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb34406a430 0x7fb34406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:05.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.809+0000 7fb360ff9640 1 -- 172.21.15.27:0/256593999 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb34c03e070 con 0x7fb3640cbce0 2023-12-19T18:16:05.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.810+0000 7fb362ffd640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb34406a430 0x7fb34406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:05.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.811+0000 7fb362ffd640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb34406a430 0x7fb34406c8f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb354000f60 tx=0x7fb354007ca0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:05.813+0000 7fb360ff9640 1 -- 172.21.15.27:0/256593999 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb34c076f70 con 0x7fb3640cbce0 2023-12-19T18:16:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.008+0000 7fb369900640 1 -- 172.21.15.27:0/256593999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb364068e90 con 0x7fb3640cbce0 2023-12-19T18:16:06.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.010+0000 7fb360ff9640 1 -- 172.21.15.27:0/256593999 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16090 (secure 0 0 0) 0x7fb34c07a590 con 0x7fb3640cbce0 2023-12-19T18:16:06.011 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:06.011 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:15:43.580746+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:06.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.013+0000 7fb369900640 1 -- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb34406a430 msgr2=0x7fb34406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.013+0000 7fb369900640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb34406a430 0x7fb34406c8f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fb354000f60 tx=0x7fb354007ca0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.013+0000 7fb369900640 1 -- 172.21.15.27:0/256593999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3640cbce0 msgr2=0x7fb3640c3a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.013+0000 7fb369900640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3640cbce0 0x7fb3640c3a80 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fb34c000c00 tx=0x7fb34c004730 comp rx=0 tx=0).stop 2023-12-19T18:16:06.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.013+0000 7fb369900640 1 -- 172.21.15.27:0/256593999 shutdown_connections 2023-12-19T18:16:06.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.013+0000 7fb369900640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb34406a430 0x7fb34406c8f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.013+0000 7fb369900640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb36415af70 0x7fb3640b2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.013+0000 7fb369900640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3640cbce0 0x7fb3640c3a80 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.013+0000 7fb369900640 1 --2- 172.21.15.27:0/256593999 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb3640adc20 0x7fb36415ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.013+0000 7fb369900640 1 -- 172.21.15.27:0/256593999 >> 172.21.15.27:0/256593999 conn(0x7fb36406a330 msgr2=0x7fb3640af6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:06.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.013+0000 7fb369900640 1 -- 172.21.15.27:0/256593999 shutdown_connections 2023-12-19T18:16:06.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.013+0000 7fb369900640 1 -- 172.21.15.27:0/256593999 wait complete. 2023-12-19T18:16:06.034 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:16:06.034 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:16:06.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.271+0000 7fe879a8c640 1 Processor -- start 2023-12-19T18:16:06.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.271+0000 7fe879a8c640 1 -- start start 2023-12-19T18:16:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.273+0000 7fe879a8c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe87415cf80 0x7fe8740d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.273+0000 7fe879a8c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8740d0e10 con 0x7fe8740b85e0 2023-12-19T18:16:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.273+0000 7fe879a8c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8740d1c70 con 0x7fe8740b8a60 2023-12-19T18:16:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.273+0000 7fe879a8c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87415d360 con 0x7fe87415cf80 2023-12-19T18:16:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.273+0000 7fe873fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe8740b85e0 0x7fe8740cda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45122/0 (socket says 172.21.15.27:45122) 2023-12-19T18:16:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.273+0000 7fe873fff640 1 -- 172.21.15.27:0/4011664491 learned_addr learned my addr 172.21.15.27:0/4011664491 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.273+0000 7fe878a8a640 1 --2- 172.21.15.27:0/4011664491 >> v2:172.21.15.192:3300/0 conn(0x7fe87415cf80 0x7fe8740d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.273+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2819129849 0 0) 0x7fe8740d0e10 con 0x7fe8740b85e0 2023-12-19T18:16:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.273+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe85c003520 con 0x7fe8740b85e0 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4269020120 0 0) 0x7fe85c003520 con 0x7fe8740b85e0 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 >> v2:172.21.15.192:3300/0 conn(0x7fe87415cf80 msgr2=0x7fe8740d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe8737fe640 1 --2- 172.21.15.27:0/4011664491 >> v2:172.21.15.192:3300/0 conn(0x7fe87415cf80 0x7fe8740d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 >> v1:172.21.15.192:6789/0 conn(0x7fe8740b8a60 legacy=0x7fe8740b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8740d1060 con 0x7fe8740b85e0 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe868001670 con 0x7fe8740b85e0 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 >> v1:172.21.15.27:6789/0 conn(0x7fe8740b85e0 legacy=0x7fe8740cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe8737fe640 1 --2- 172.21.15.27:0/4011664491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe85c0039c0 0x7fe85c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe8737fe640 1 --2- 172.21.15.27:0/4011664491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe85c0047c0 0x7fe85c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe8737fe640 1 --2- 172.21.15.27:0/4011664491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe85c005180 0x7fe85c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87415d360 con 0x7fe85c0047c0 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c00c060 con 0x7fe85c0039c0 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe85c00c220 con 0x7fe85c005180 2023-12-19T18:16:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.274+0000 7fe873fff640 1 --2- 172.21.15.27:0/4011664491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe85c0047c0 0x7fe85c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.275+0000 7fe878a8a640 1 --2- 172.21.15.27:0/4011664491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe85c0039c0 0x7fe85c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.275+0000 7fe873fff640 1 -- 172.21.15.27:0/4011664491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe85c005180 msgr2=0x7fe85c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.275+0000 7fe873fff640 1 --2- 172.21.15.27:0/4011664491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe85c005180 0x7fe85c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.275+0000 7fe873fff640 1 -- 172.21.15.27:0/4011664491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe85c0039c0 msgr2=0x7fe85c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.275+0000 7fe873fff640 1 --2- 172.21.15.27:0/4011664491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe85c0039c0 0x7fe85c003de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.275+0000 7fe873fff640 1 -- 172.21.15.27:0/4011664491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe85c003520 con 0x7fe85c0047c0 2023-12-19T18:16:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.275+0000 7fe87928b640 1 -- 172.21.15.27:0/4011664491 reap_dead start 2023-12-19T18:16:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.276+0000 7fe873fff640 1 --2- 172.21.15.27:0/4011664491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe85c0047c0 0x7fe85c004c40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe86800c0e0 tx=0x7fe85c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9387a9f7115dbc9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.276+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe868005ad0 con 0x7fe85c0047c0 2023-12-19T18:16:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.276+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe868005c30 con 0x7fe85c0047c0 2023-12-19T18:16:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.276+0000 7fe8737fe640 1 -- 172.21.15.27:0/4011664491 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86802c7d0 con 0x7fe85c0047c0 2023-12-19T18:16:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.277+0000 7fe879a8c640 1 -- 172.21.15.27:0/4011664491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe85c0047c0 msgr2=0x7fe85c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.277+0000 7fe879a8c640 1 --2- 172.21.15.27:0/4011664491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe85c0047c0 0x7fe85c004c40 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe86800c0e0 tx=0x7fe85c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.277+0000 7fe879a8c640 1 -- 172.21.15.27:0/4011664491 shutdown_connections 2023-12-19T18:16:06.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.277+0000 7fe879a8c640 1 --2- 172.21.15.27:0/4011664491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe85c0047c0 0x7fe85c004c40 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.277+0000 7fe879a8c640 1 -- 172.21.15.27:0/4011664491 >> 172.21.15.27:0/4011664491 conn(0x7fe874072e20 msgr2=0x7fe874073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:06.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.277+0000 7fe879a8c640 1 -- 172.21.15.27:0/4011664491 shutdown_connections 2023-12-19T18:16:06.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.277+0000 7fe879a8c640 1 -- 172.21.15.27:0/4011664491 wait complete. 2023-12-19T18:16:06.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.279+0000 7fe879a8c640 1 Processor -- start 2023-12-19T18:16:06.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.279+0000 7fe879a8c640 1 -- start start 2023-12-19T18:16:06.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.280+0000 7fe879a8c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe85c0047c0 0x7fe8740cc5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.280+0000 7fe879a8c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe874151e40 0x7fe8740c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.280+0000 7fe879a8c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8740ccb00 0x7fe8740c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.280+0000 7fe879a8c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe874152560 con 0x7fe8740ccb00 2023-12-19T18:16:06.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.280+0000 7fe879a8c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87415d5d0 con 0x7fe85c0047c0 2023-12-19T18:16:06.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.280+0000 7fe879a8c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8740ca100 con 0x7fe874151e40 2023-12-19T18:16:06.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.281+0000 7fe87928b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8740ccb00 0x7fe8740c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.281+0000 7fe87928b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8740ccb00 0x7fe8740c8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45570/0 (socket says 172.21.15.27:45570) 2023-12-19T18:16:06.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.281+0000 7fe87928b640 1 -- 172.21.15.27:0/4281251347 learned_addr learned my addr 172.21.15.27:0/4281251347 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:06.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.281+0000 7fe878a8a640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe85c0047c0 0x7fe8740cc5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.281+0000 7fe873fff640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe874151e40 0x7fe8740c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.281+0000 7fe87928b640 1 -- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe874151e40 msgr2=0x7fe8740c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.281+0000 7fe87928b640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe874151e40 0x7fe8740c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.281+0000 7fe87928b640 1 -- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe85c0047c0 msgr2=0x7fe8740cc5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.281+0000 7fe87928b640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe85c0047c0 0x7fe8740cc5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.281+0000 7fe87928b640 1 -- 172.21.15.27:0/4281251347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87415d870 con 0x7fe8740ccb00 2023-12-19T18:16:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.281+0000 7fe873fff640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe874151e40 0x7fe8740c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:06.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.281+0000 7fe87928b640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8740ccb00 0x7fe8740c8eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe8740b58c0 tx=0x7fe86c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:06.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.282+0000 7fe8717fa640 1 -- 172.21.15.27:0/4281251347 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c004220 con 0x7fe8740ccb00 2023-12-19T18:16:06.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.282+0000 7fe8717fa640 1 -- 172.21.15.27:0/4281251347 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c004380 con 0x7fe8740ccb00 2023-12-19T18:16:06.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.282+0000 7fe8717fa640 1 -- 172.21.15.27:0/4281251347 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c007310 con 0x7fe8740ccb00 2023-12-19T18:16:06.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.282+0000 7fe879a8c640 1 -- 172.21.15.27:0/4281251347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87415db50 con 0x7fe8740ccb00 2023-12-19T18:16:06.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.282+0000 7fe879a8c640 1 -- 172.21.15.27:0/4281251347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8740c9d40 con 0x7fe8740ccb00 2023-12-19T18:16:06.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.283+0000 7fe879a8c640 1 -- 172.21.15.27:0/4281251347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8740ca5c0 con 0x7fe8740ccb00 2023-12-19T18:16:06.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.284+0000 7fe8717fa640 1 -- 172.21.15.27:0/4281251347 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe86c007470 con 0x7fe8740ccb00 2023-12-19T18:16:06.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.285+0000 7fe8717fa640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe86006a430 0x7fe86006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.285+0000 7fe8717fa640 1 -- 172.21.15.27:0/4281251347 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe86c0879b0 con 0x7fe8740ccb00 2023-12-19T18:16:06.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.286+0000 7fe878a8a640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe86006a430 0x7fe86006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.287+0000 7fe878a8a640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe86006a430 0x7fe86006c8f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe87404f8f0 tx=0x7fe86400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:06.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.289+0000 7fe8717fa640 1 -- 172.21.15.27:0/4281251347 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe86c052f80 con 0x7fe8740ccb00 2023-12-19T18:16:06.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.485+0000 7fe879a8c640 1 -- 172.21.15.27:0/4281251347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8740d18e0 con 0x7fe8740ccb00 2023-12-19T18:16:06.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.486+0000 7fe8717fa640 1 -- 172.21.15.27:0/4281251347 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe8740d18e0 con 0x7fe8740ccb00 2023-12-19T18:16:06.486 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:16:06.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.489+0000 7fe879a8c640 1 -- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe86006a430 msgr2=0x7fe86006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.489+0000 7fe879a8c640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe86006a430 0x7fe86006c8f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe87404f8f0 tx=0x7fe86400b040 comp rx=0 tx=0).stop 2023-12-19T18:16:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.489+0000 7fe879a8c640 1 -- 172.21.15.27:0/4281251347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8740ccb00 msgr2=0x7fe8740c8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.489+0000 7fe879a8c640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8740ccb00 0x7fe8740c8eb0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe8740b58c0 tx=0x7fe86c00c820 comp rx=0 tx=0).stop 2023-12-19T18:16:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.489+0000 7fe879a8c640 1 -- 172.21.15.27:0/4281251347 shutdown_connections 2023-12-19T18:16:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.489+0000 7fe879a8c640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe86006a430 0x7fe86006c8f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.489+0000 7fe879a8c640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8740ccb00 0x7fe8740c8eb0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.489+0000 7fe879a8c640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe874151e40 0x7fe8740c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.489+0000 7fe879a8c640 1 --2- 172.21.15.27:0/4281251347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe85c0047c0 0x7fe8740cc5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.489+0000 7fe879a8c640 1 -- 172.21.15.27:0/4281251347 >> 172.21.15.27:0/4281251347 conn(0x7fe874072e20 msgr2=0x7fe874065910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.489+0000 7fe879a8c640 1 -- 172.21.15.27:0/4281251347 shutdown_connections 2023-12-19T18:16:06.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.489+0000 7fe879a8c640 1 -- 172.21.15.27:0/4281251347 wait complete. 2023-12-19T18:16:06.510 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:16:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.742+0000 7f396828f640 1 Processor -- start 2023-12-19T18:16:06.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.742+0000 7f396828f640 1 -- start start 2023-12-19T18:16:06.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.744+0000 7f396828f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f396006a380 0x7f39600cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.744+0000 7f396828f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39600713e0 con 0x7f3960070d90 2023-12-19T18:16:06.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.744+0000 7f396828f640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39600cdd20 con 0x7f39600af0c0 2023-12-19T18:16:06.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.744+0000 7f396828f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3960071910 con 0x7f396006a380 2023-12-19T18:16:06.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.744+0000 7f396682c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3960070d90 0x7f39600cad50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45134/0 (socket says 172.21.15.27:45134) 2023-12-19T18:16:06.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.744+0000 7f396682c640 1 -- 172.21.15.27:0/1778247092 learned_addr learned my addr 172.21.15.27:0/1778247092 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:06.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.744+0000 7f396582a640 1 --2- 172.21.15.27:0/1778247092 >> v2:172.21.15.192:3300/0 conn(0x7f396006a380 0x7f39600cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 756959930 0 0) 0x7f39600713e0 con 0x7f3960070d90 2023-12-19T18:16:06.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3948003520 con 0x7f3960070d90 2023-12-19T18:16:06.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4077905439 0 0) 0x7f3948003520 con 0x7f3960070d90 2023-12-19T18:16:06.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 >> v2:172.21.15.192:3300/0 conn(0x7f396006a380 msgr2=0x7f39600cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:06.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 --2- 172.21.15.27:0/1778247092 >> v2:172.21.15.192:3300/0 conn(0x7f396006a380 0x7f39600cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 >> v1:172.21.15.192:6789/0 conn(0x7f39600af0c0 legacy=0x7f3960069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39600cd9f0 con 0x7f3960070d90 2023-12-19T18:16:06.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f39580019f0 con 0x7f3960070d90 2023-12-19T18:16:06.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 >> v1:172.21.15.27:6789/0 conn(0x7f3960070d90 legacy=0x7f39600cad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 --2- 172.21.15.27:0/1778247092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39480039c0 0x7f3948003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 --2- 172.21.15.27:0/1778247092 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39480047c0 0x7f3948004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 --2- 172.21.15.27:0/1778247092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39480052a0 0x7f394800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3960071910 con 0x7f39480039c0 2023-12-19T18:16:06.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f394800c1a0 con 0x7f39480052a0 2023-12-19T18:16:06.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.745+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f394800c360 con 0x7f39480047c0 2023-12-19T18:16:06.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.746+0000 7f396602b640 1 --2- 172.21.15.27:0/1778247092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39480039c0 0x7f3948003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.746+0000 7f396682c640 1 --2- 172.21.15.27:0/1778247092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39480052a0 0x7f394800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.746+0000 7f396602b640 1 -- 172.21.15.27:0/1778247092 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39480047c0 msgr2=0x7f3948004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:06.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.746+0000 7f396602b640 1 --2- 172.21.15.27:0/1778247092 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39480047c0 0x7f3948004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.746+0000 7f396602b640 1 -- 172.21.15.27:0/1778247092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39480052a0 msgr2=0x7f394800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:06.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.746+0000 7f396602b640 1 --2- 172.21.15.27:0/1778247092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39480052a0 0x7f394800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.746+0000 7f396682c640 1 -- 172.21.15.27:0/1778247092 reap_dead start 2023-12-19T18:16:06.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.746+0000 7f396602b640 1 -- 172.21.15.27:0/1778247092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3948003520 con 0x7f39480039c0 2023-12-19T18:16:06.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.747+0000 7f396602b640 1 --2- 172.21.15.27:0/1778247092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39480039c0 0x7f3948003de0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3950001750 tx=0x7f39480057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f166e38b6e31f294 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:06.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.749+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39500052d0 con 0x7f39480039c0 2023-12-19T18:16:06.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.749+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f395002e050 con 0x7f39480039c0 2023-12-19T18:16:06.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.749+0000 7f3965029640 1 -- 172.21.15.27:0/1778247092 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950003830 con 0x7f39480039c0 2023-12-19T18:16:06.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.749+0000 7f396828f640 1 -- 172.21.15.27:0/1778247092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39480039c0 msgr2=0x7f3948003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.749+0000 7f396828f640 1 --2- 172.21.15.27:0/1778247092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39480039c0 0x7f3948003de0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f3950001750 tx=0x7f39480057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.750+0000 7f396828f640 1 -- 172.21.15.27:0/1778247092 shutdown_connections 2023-12-19T18:16:06.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.750+0000 7f396828f640 1 --2- 172.21.15.27:0/1778247092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39480039c0 0x7f3948003de0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.750+0000 7f396828f640 1 -- 172.21.15.27:0/1778247092 >> 172.21.15.27:0/1778247092 conn(0x7f396006ad80 msgr2=0x7f396006b190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:06.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.750+0000 7f396828f640 1 -- 172.21.15.27:0/1778247092 shutdown_connections 2023-12-19T18:16:06.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.750+0000 7f396828f640 1 -- 172.21.15.27:0/1778247092 wait complete. 2023-12-19T18:16:06.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.752+0000 7f396828f640 1 Processor -- start 2023-12-19T18:16:06.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.752+0000 7f396828f640 1 -- start start 2023-12-19T18:16:06.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396828f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39480039c0 0x7f39601e7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396828f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39600c9790 0x7f39600c9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396828f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f396015cce0 0x7f396015d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396828f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39601f5730 con 0x7f39480039c0 2023-12-19T18:16:06.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396828f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39601f5a10 con 0x7f39600c9790 2023-12-19T18:16:06.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396828f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39601f5cf0 con 0x7f396015cce0 2023-12-19T18:16:06.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396602b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39480039c0 0x7f39601e7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396602b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39480039c0 0x7f39601e7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45594/0 (socket says 172.21.15.27:45594) 2023-12-19T18:16:06.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396602b640 1 -- 172.21.15.27:0/1296658448 learned_addr learned my addr 172.21.15.27:0/1296658448 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:06.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396682c640 1 --2- 172.21.15.27:0/1296658448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f396015cce0 0x7f396015d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396602b640 1 -- 172.21.15.27:0/1296658448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f396015cce0 msgr2=0x7f396015d140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396602b640 1 --2- 172.21.15.27:0/1296658448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f396015cce0 0x7f396015d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396602b640 1 -- 172.21.15.27:0/1296658448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39600c9790 msgr2=0x7f39600c9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396602b640 1 --2- 172.21.15.27:0/1296658448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39600c9790 0x7f39600c9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.753+0000 7f396602b640 1 -- 172.21.15.27:0/1296658448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f396006d690 con 0x7f39480039c0 2023-12-19T18:16:06.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.754+0000 7f396602b640 1 --2- 172.21.15.27:0/1296658448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39480039c0 0x7f39601e7f60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3950005040 tx=0x7f3950001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:06.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.754+0000 7f395effd640 1 -- 172.21.15.27:0/1296658448 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39500076c0 con 0x7f39480039c0 2023-12-19T18:16:06.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.754+0000 7f395effd640 1 -- 172.21.15.27:0/1296658448 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39500069e0 con 0x7f39480039c0 2023-12-19T18:16:06.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.754+0000 7f395effd640 1 -- 172.21.15.27:0/1296658448 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3950006d90 con 0x7f39480039c0 2023-12-19T18:16:06.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.754+0000 7f396828f640 1 -- 172.21.15.27:0/1296658448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39601f5fd0 con 0x7f39480039c0 2023-12-19T18:16:06.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.754+0000 7f396828f640 1 -- 172.21.15.27:0/1296658448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f396015d680 con 0x7f39480039c0 2023-12-19T18:16:06.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.756+0000 7f396828f640 1 -- 172.21.15.27:0/1296658448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3960068e90 con 0x7f39480039c0 2023-12-19T18:16:06.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.757+0000 7f395effd640 1 -- 172.21.15.27:0/1296658448 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f39500035f0 con 0x7f39480039c0 2023-12-19T18:16:06.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.758+0000 7f395effd640 1 --2- 172.21.15.27:0/1296658448 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f394c06a430 0x7f394c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:06.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.758+0000 7f395effd640 1 -- 172.21.15.27:0/1296658448 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f39500aa710 con 0x7f39480039c0 2023-12-19T18:16:06.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.758+0000 7f396582a640 1 --2- 172.21.15.27:0/1296658448 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f394c06a430 0x7f394c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:06.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.759+0000 7f396582a640 1 --2- 172.21.15.27:0/1296658448 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f394c06a430 0x7f394c06c8f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3954006e80 tx=0x7f3954006dd0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:06.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.763+0000 7f395effd640 1 -- 172.21.15.27:0/1296658448 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3950075cd0 con 0x7f39480039c0 2023-12-19T18:16:06.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.961+0000 7f396828f640 1 -- 172.21.15.27:0/1296658448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3960065d80 con 0x7f39480039c0 2023-12-19T18:16:06.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.962+0000 7f395effd640 1 -- 172.21.15.27:0/1296658448 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f39500792f0 con 0x7f39480039c0 2023-12-19T18:16:06.962 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:16:06.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.964+0000 7f396828f640 1 -- 172.21.15.27:0/1296658448 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f394c06a430 msgr2=0x7f394c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.964+0000 7f396828f640 1 --2- 172.21.15.27:0/1296658448 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f394c06a430 0x7f394c06c8f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3954006e80 tx=0x7f3954006dd0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.964+0000 7f396828f640 1 -- 172.21.15.27:0/1296658448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39480039c0 msgr2=0x7f39601e7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:06.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.964+0000 7f396828f640 1 --2- 172.21.15.27:0/1296658448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39480039c0 0x7f39601e7f60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f3950005040 tx=0x7f3950001120 comp rx=0 tx=0).stop 2023-12-19T18:16:06.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.964+0000 7f396828f640 1 -- 172.21.15.27:0/1296658448 shutdown_connections 2023-12-19T18:16:06.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.964+0000 7f396828f640 1 --2- 172.21.15.27:0/1296658448 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f394c06a430 0x7f394c06c8f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.964+0000 7f396828f640 1 --2- 172.21.15.27:0/1296658448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f396015cce0 0x7f396015d140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.964+0000 7f396828f640 1 --2- 172.21.15.27:0/1296658448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39600c9790 0x7f39600c9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.964+0000 7f396828f640 1 --2- 172.21.15.27:0/1296658448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39480039c0 0x7f39601e7f60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:06.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.964+0000 7f396828f640 1 -- 172.21.15.27:0/1296658448 >> 172.21.15.27:0/1296658448 conn(0x7f396006ad80 msgr2=0x7f39600b08d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:06.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.964+0000 7f396828f640 1 -- 172.21.15.27:0/1296658448 shutdown_connections 2023-12-19T18:16:06.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:06.965+0000 7f396828f640 1 -- 172.21.15.27:0/1296658448 wait complete. 2023-12-19T18:16:06.985 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:16:07.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.225+0000 7f41a60cc640 1 Processor -- start 2023-12-19T18:16:07.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.225+0000 7f41a60cc640 1 -- start start 2023-12-19T18:16:07.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.227+0000 7f41a60cc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f41a0151c90 0x7f41a006b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.227+0000 7f41a60cc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f41a015e9d0 0x7f41a00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.227+0000 7f41a60cc640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41a015b3e0 con 0x7f41a0152070 2023-12-19T18:16:07.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.227+0000 7f41a60cc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a015eed0 con 0x7f41a0151c90 2023-12-19T18:16:07.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.227+0000 7f41a60cc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41a015f1b0 con 0x7f41a015e9d0 2023-12-19T18:16:07.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.227+0000 7f41a50ca640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f41a0151c90 0x7f41a006b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.227+0000 7f41a58cb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f41a0152070 0x7f41a015b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45136/0 (socket says 172.21.15.27:45136) 2023-12-19T18:16:07.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.227+0000 7f41a58cb640 1 -- 172.21.15.27:0/3862138897 learned_addr learned my addr 172.21.15.27:0/3862138897 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:07.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.228+0000 7f41a48c9640 1 --2- 172.21.15.27:0/3862138897 >> v2:172.21.15.192:3300/0 conn(0x7f41a015e9d0 0x7f41a00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.228+0000 7f41a50ca640 1 -- 172.21.15.27:0/3862138897 >> v2:172.21.15.192:3300/0 conn(0x7f41a015e9d0 msgr2=0x7f41a00cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:07.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.228+0000 7f41a50ca640 1 --2- 172.21.15.27:0/3862138897 >> v2:172.21.15.192:3300/0 conn(0x7f41a015e9d0 0x7f41a00cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.228+0000 7f41a50ca640 1 -- 172.21.15.27:0/3862138897 >> v1:172.21.15.27:6789/0 conn(0x7f41a0152070 legacy=0x7f41a015b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.228+0000 7f41a50ca640 1 -- 172.21.15.27:0/3862138897 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a015f490 con 0x7f41a0151c90 2023-12-19T18:16:07.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.228+0000 7f4197fff640 1 -- 172.21.15.27:0/3862138897 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1091972815 0 0) 0x7f41a015b3e0 con 0x7f41a0152070 2023-12-19T18:16:07.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.229+0000 7f41a50ca640 1 --2- 172.21.15.27:0/3862138897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0151c90 0x7f41a006b4d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f419c009d90 tx=0x7f419c02be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b0476a4f80314d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:07.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.230+0000 7f4197fff640 1 -- 172.21.15.27:0/3862138897 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c02cbe0 con 0x7f41a0151c90 2023-12-19T18:16:07.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.230+0000 7f4197fff640 1 -- 172.21.15.27:0/3862138897 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c02cd40 con 0x7f41a0151c90 2023-12-19T18:16:07.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.230+0000 7f4197fff640 1 -- 172.21.15.27:0/3862138897 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c034660 con 0x7f41a0151c90 2023-12-19T18:16:07.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.231+0000 7f41a60cc640 1 -- 172.21.15.27:0/3862138897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0151c90 msgr2=0x7f41a006b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.231+0000 7f41a60cc640 1 --2- 172.21.15.27:0/3862138897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0151c90 0x7f41a006b4d0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f419c009d90 tx=0x7f419c02be30 comp rx=0 tx=0).stop 2023-12-19T18:16:07.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.231+0000 7f41a60cc640 1 -- 172.21.15.27:0/3862138897 shutdown_connections 2023-12-19T18:16:07.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.231+0000 7f41a60cc640 1 --2- 172.21.15.27:0/3862138897 >> v2:172.21.15.192:3300/0 conn(0x7f41a015e9d0 0x7f41a00cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.231+0000 7f41a60cc640 1 --2- 172.21.15.27:0/3862138897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0151c90 0x7f41a006b4d0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.231+0000 7f41a60cc640 1 -- 172.21.15.27:0/3862138897 >> 172.21.15.27:0/3862138897 conn(0x7f41a006a060 msgr2=0x7f41a006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:07.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.231+0000 7f41a60cc640 1 -- 172.21.15.27:0/3862138897 shutdown_connections 2023-12-19T18:16:07.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.231+0000 7f41a60cc640 1 -- 172.21.15.27:0/3862138897 wait complete. 2023-12-19T18:16:07.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.233+0000 7f41a60cc640 1 Processor -- start 2023-12-19T18:16:07.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.233+0000 7f41a60cc640 1 -- start start 2023-12-19T18:16:07.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.234+0000 7f41a60cc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0151c90 0x7f41a0141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.234+0000 7f41a60cc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41a0152070 0x7f41a00af150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.234+0000 7f41a60cc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41a015e9d0 0x7f41a01e67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.234+0000 7f41a60cc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a00aecc0 con 0x7f41a0151c90 2023-12-19T18:16:07.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.234+0000 7f41a60cc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41a01419b0 con 0x7f41a0152070 2023-12-19T18:16:07.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.234+0000 7f41a60cc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41a0154540 con 0x7f41a015e9d0 2023-12-19T18:16:07.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.234+0000 7f41a50ca640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0151c90 0x7f41a0141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.234+0000 7f41a50ca640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0151c90 0x7f41a0141470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45604/0 (socket says 172.21.15.27:45604) 2023-12-19T18:16:07.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.234+0000 7f41a50ca640 1 -- 172.21.15.27:0/2305542794 learned_addr learned my addr 172.21.15.27:0/2305542794 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.235+0000 7f41a58cb640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41a015e9d0 0x7f41a01e67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.235+0000 7f41a48c9640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41a0152070 0x7f41a00af150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.235+0000 7f41a50ca640 1 -- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41a015e9d0 msgr2=0x7f41a01e67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.235+0000 7f41a50ca640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41a015e9d0 0x7f41a01e67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.235+0000 7f41a50ca640 1 -- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41a0152070 msgr2=0x7f41a00af150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.235+0000 7f41a50ca640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41a0152070 0x7f41a00af150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.235+0000 7f41a50ca640 1 -- 172.21.15.27:0/2305542794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f419c009a70 con 0x7f41a0151c90 2023-12-19T18:16:07.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.235+0000 7f41a48c9640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41a0152070 0x7f41a00af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:07.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.235+0000 7f41a50ca640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0151c90 0x7f41a0141470 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f419c009d90 tx=0x7f419c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:07.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.235+0000 7f4195ffb640 1 -- 172.21.15.27:0/2305542794 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c03e610 con 0x7f41a0151c90 2023-12-19T18:16:07.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.235+0000 7f4195ffb640 1 -- 172.21.15.27:0/2305542794 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419c03e770 con 0x7f41a0151c90 2023-12-19T18:16:07.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.235+0000 7f4195ffb640 1 -- 172.21.15.27:0/2305542794 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419c03eb00 con 0x7f41a0151c90 2023-12-19T18:16:07.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.236+0000 7f41a60cc640 1 -- 172.21.15.27:0/2305542794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41a0154260 con 0x7f41a0151c90 2023-12-19T18:16:07.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.236+0000 7f41a60cc640 1 -- 172.21.15.27:0/2305542794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41a00ae950 con 0x7f41a0151c90 2023-12-19T18:16:07.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.237+0000 7f41a60cc640 1 -- 172.21.15.27:0/2305542794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41a004e840 con 0x7f41a0151c90 2023-12-19T18:16:07.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.238+0000 7f4195ffb640 1 -- 172.21.15.27:0/2305542794 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f419c03b050 con 0x7f41a0151c90 2023-12-19T18:16:07.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.239+0000 7f4195ffb640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f418406a460 0x7f418406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.239+0000 7f4195ffb640 1 -- 172.21.15.27:0/2305542794 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f419c0aaca0 con 0x7f41a0151c90 2023-12-19T18:16:07.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.239+0000 7f41a48c9640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f418406a460 0x7f418406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.240+0000 7f41a48c9640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f418406a460 0x7f418406c920 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4190006fd0 tx=0x7f4190008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:07.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.243+0000 7f4195ffb640 1 -- 172.21.15.27:0/2305542794 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f419c076270 con 0x7f41a0151c90 2023-12-19T18:16:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.435+0000 7f41a60cc640 1 -- 172.21.15.27:0/2305542794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f41a00cd1a0 con 0x7f41a0151c90 2023-12-19T18:16:07.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.436+0000 7f4195ffb640 1 -- 172.21.15.27:0/2305542794 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f419c079890 con 0x7f41a0151c90 2023-12-19T18:16:07.436 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:16:07.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.439+0000 7f41a60cc640 1 -- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f418406a460 msgr2=0x7f418406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.439+0000 7f41a60cc640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f418406a460 0x7f418406c920 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f4190006fd0 tx=0x7f4190008040 comp rx=0 tx=0).stop 2023-12-19T18:16:07.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.439+0000 7f41a60cc640 1 -- 172.21.15.27:0/2305542794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0151c90 msgr2=0x7f41a0141470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.439+0000 7f41a60cc640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0151c90 0x7f41a0141470 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f419c009d90 tx=0x7f419c004a40 comp rx=0 tx=0).stop 2023-12-19T18:16:07.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.439+0000 7f41a60cc640 1 -- 172.21.15.27:0/2305542794 shutdown_connections 2023-12-19T18:16:07.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.439+0000 7f41a60cc640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f418406a460 0x7f418406c920 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.439+0000 7f41a60cc640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f41a015e9d0 0x7f41a01e67d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.439+0000 7f41a60cc640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41a0152070 0x7f41a00af150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.439+0000 7f41a60cc640 1 --2- 172.21.15.27:0/2305542794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41a0151c90 0x7f41a0141470 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.439+0000 7f41a60cc640 1 -- 172.21.15.27:0/2305542794 >> 172.21.15.27:0/2305542794 conn(0x7f41a006a060 msgr2=0x7f41a01513d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:07.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.439+0000 7f41a60cc640 1 -- 172.21.15.27:0/2305542794 shutdown_connections 2023-12-19T18:16:07.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.439+0000 7f41a60cc640 1 -- 172.21.15.27:0/2305542794 wait complete. 2023-12-19T18:16:07.460 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:16:07.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.696+0000 7f93db269640 1 Processor -- start 2023-12-19T18:16:07.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.696+0000 7f93db269640 1 -- start start 2023-12-19T18:16:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.698+0000 7f93db269640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93d415d1d0 0x7f93d40ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.698+0000 7f93db269640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f93d40ac8d0 0x7f93d40acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.698+0000 7f93db269640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93d40ce7c0 con 0x7f93d4071140 2023-12-19T18:16:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.698+0000 7f93db269640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93d4070150 con 0x7f93d40ac8d0 2023-12-19T18:16:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.698+0000 7f93db269640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d415d5b0 con 0x7f93d415d1d0 2023-12-19T18:16:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.698+0000 7f93d9806640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f93d40ac8d0 0x7f93d40acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.698+0000 7f93d8804640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f93d4071140 0x7f93d40cd250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:43568/0 (socket says 172.21.15.27:43568) 2023-12-19T18:16:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.699+0000 7f93d8804640 1 -- 172.21.15.27:0/2975884999 learned_addr learned my addr 172.21.15.27:0/2975884999 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.698+0000 7f93d9005640 1 --2- 172.21.15.27:0/2975884999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93d415d1d0 0x7f93d40ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.699+0000 7f93d9806640 1 -- 172.21.15.27:0/2975884999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93d415d1d0 msgr2=0x7f93d40ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.699+0000 7f93d9806640 1 --2- 172.21.15.27:0/2975884999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93d415d1d0 0x7f93d40ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.699+0000 7f93d9806640 1 -- 172.21.15.27:0/2975884999 >> v1:172.21.15.192:6789/0 conn(0x7f93d4071140 legacy=0x7f93d40cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.699+0000 7f93cbfff640 1 -- 172.21.15.27:0/2975884999 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1080239432 0 0) 0x7f93d40ce7c0 con 0x7f93d4071140 2023-12-19T18:16:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.699+0000 7f93d9806640 1 -- 172.21.15.27:0/2975884999 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d4070360 con 0x7f93d40ac8d0 2023-12-19T18:16:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.700+0000 7f93d9806640 1 --2- 172.21.15.27:0/2975884999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d40ac8d0 0x7f93d40acd60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009d90 tx=0x7f93cc02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db9f008d5942032b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.701+0000 7f93cbfff640 1 -- 172.21.15.27:0/2975884999 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc02cbd0 con 0x7f93d40ac8d0 2023-12-19T18:16:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.701+0000 7f93cbfff640 1 -- 172.21.15.27:0/2975884999 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93cc02cd30 con 0x7f93d40ac8d0 2023-12-19T18:16:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.701+0000 7f93cbfff640 1 -- 172.21.15.27:0/2975884999 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc02cea0 con 0x7f93d40ac8d0 2023-12-19T18:16:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.701+0000 7f93db269640 1 -- 172.21.15.27:0/2975884999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d40ac8d0 msgr2=0x7f93d40acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.701+0000 7f93db269640 1 --2- 172.21.15.27:0/2975884999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d40ac8d0 0x7f93d40acd60 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009d90 tx=0x7f93cc02be20 comp rx=0 tx=0).stop 2023-12-19T18:16:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.701+0000 7f93db269640 1 -- 172.21.15.27:0/2975884999 shutdown_connections 2023-12-19T18:16:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.701+0000 7f93db269640 1 --2- 172.21.15.27:0/2975884999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d40ac8d0 0x7f93d40acd60 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.701+0000 7f93db269640 1 --2- 172.21.15.27:0/2975884999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93d415d1d0 0x7f93d40ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.701+0000 7f93db269640 1 -- 172.21.15.27:0/2975884999 >> 172.21.15.27:0/2975884999 conn(0x7f93d40ceb50 msgr2=0x7f93d40cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.702+0000 7f93db269640 1 -- 172.21.15.27:0/2975884999 shutdown_connections 2023-12-19T18:16:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.702+0000 7f93db269640 1 -- 172.21.15.27:0/2975884999 wait complete. 2023-12-19T18:16:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.703+0000 7f93db269640 1 Processor -- start 2023-12-19T18:16:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.704+0000 7f93db269640 1 -- start start 2023-12-19T18:16:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.704+0000 7f93db269640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93d4071140 0x7f93d415f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.704+0000 7f93db269640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93d40ac8d0 0x7f93d4156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.704+0000 7f93db269640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d415d1d0 0x7f93d406b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.704+0000 7f93db269640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d415f370 con 0x7f93d415d1d0 2023-12-19T18:16:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.704+0000 7f93db269640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d40b2d00 con 0x7f93d4071140 2023-12-19T18:16:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.704+0000 7f93db269640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93d4157c90 con 0x7f93d40ac8d0 2023-12-19T18:16:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.705+0000 7f93d9806640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d415d1d0 0x7f93d406b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.705+0000 7f93d9806640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d415d1d0 0x7f93d406b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45612/0 (socket says 172.21.15.27:45612) 2023-12-19T18:16:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.705+0000 7f93d9806640 1 -- 172.21.15.27:0/468033690 learned_addr learned my addr 172.21.15.27:0/468033690 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.705+0000 7f93d9806640 1 -- 172.21.15.27:0/468033690 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93d40ac8d0 msgr2=0x7f93d4156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.705+0000 7f93d9806640 1 --2- 172.21.15.27:0/468033690 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93d40ac8d0 0x7f93d4156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.705+0000 7f93d9806640 1 -- 172.21.15.27:0/468033690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93d4071140 msgr2=0x7f93d415f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.705+0000 7f93d9005640 1 --2- 172.21.15.27:0/468033690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93d4071140 0x7f93d415f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.705+0000 7f93d9806640 1 --2- 172.21.15.27:0/468033690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93d4071140 0x7f93d415f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.705+0000 7f93d9806640 1 -- 172.21.15.27:0/468033690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93cc009a70 con 0x7f93d415d1d0 2023-12-19T18:16:07.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.706+0000 7f93d9806640 1 --2- 172.21.15.27:0/468033690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d415d1d0 0x7f93d406b5b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009d90 tx=0x7f93cc004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:07.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.706+0000 7f93c9ffb640 1 -- 172.21.15.27:0/468033690 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc02cea0 con 0x7f93d415d1d0 2023-12-19T18:16:07.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.706+0000 7f93c9ffb640 1 -- 172.21.15.27:0/468033690 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93cc003b90 con 0x7f93d415d1d0 2023-12-19T18:16:07.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.706+0000 7f93c9ffb640 1 -- 172.21.15.27:0/468033690 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93cc009450 con 0x7f93d415d1d0 2023-12-19T18:16:07.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.706+0000 7f93db269640 1 -- 172.21.15.27:0/468033690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d4157f70 con 0x7f93d415d1d0 2023-12-19T18:16:07.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.706+0000 7f93db269640 1 -- 172.21.15.27:0/468033690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d4158530 con 0x7f93d415d1d0 2023-12-19T18:16:07.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.707+0000 7f93db269640 1 -- 172.21.15.27:0/468033690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d40b2e70 con 0x7f93d415d1d0 2023-12-19T18:16:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.708+0000 7f93c9ffb640 1 -- 172.21.15.27:0/468033690 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f93cc03b050 con 0x7f93d415d1d0 2023-12-19T18:16:07.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.709+0000 7f93c9ffb640 1 --2- 172.21.15.27:0/468033690 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f93b806a430 0x7f93b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:07.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.709+0000 7f93c9ffb640 1 -- 172.21.15.27:0/468033690 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f93cc0aadb0 con 0x7f93d415d1d0 2023-12-19T18:16:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.710+0000 7f93d9005640 1 --2- 172.21.15.27:0/468033690 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f93b806a430 0x7f93b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:07.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.711+0000 7f93d9005640 1 --2- 172.21.15.27:0/468033690 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f93b806a430 0x7f93b806c8f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009a10 tx=0x7f93d0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:07.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.713+0000 7f93c9ffb640 1 -- 172.21.15.27:0/468033690 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f93cc0af070 con 0x7f93d415d1d0 2023-12-19T18:16:07.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.930+0000 7f93db269640 1 -- 172.21.15.27:0/468033690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f93d4160960 con 0x7f93d415d1d0 2023-12-19T18:16:07.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.931+0000 7f93c9ffb640 1 -- 172.21.15.27:0/468033690 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f93cc039080 con 0x7f93d415d1d0 2023-12-19T18:16:07.931 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:16:07.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.934+0000 7f93db269640 1 -- 172.21.15.27:0/468033690 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f93b806a430 msgr2=0x7f93b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.934+0000 7f93db269640 1 --2- 172.21.15.27:0/468033690 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f93b806a430 0x7f93b806c8f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009a10 tx=0x7f93d0008040 comp rx=0 tx=0).stop 2023-12-19T18:16:07.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.934+0000 7f93db269640 1 -- 172.21.15.27:0/468033690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d415d1d0 msgr2=0x7f93d406b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:07.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.934+0000 7f93db269640 1 --2- 172.21.15.27:0/468033690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d415d1d0 0x7f93d406b5b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f93cc009d90 tx=0x7f93cc004a40 comp rx=0 tx=0).stop 2023-12-19T18:16:07.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.934+0000 7f93db269640 1 -- 172.21.15.27:0/468033690 shutdown_connections 2023-12-19T18:16:07.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.934+0000 7f93db269640 1 --2- 172.21.15.27:0/468033690 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f93b806a430 0x7f93b806c8f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.934+0000 7f93db269640 1 --2- 172.21.15.27:0/468033690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d415d1d0 0x7f93d406b5b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.934+0000 7f93db269640 1 --2- 172.21.15.27:0/468033690 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f93d40ac8d0 0x7f93d4156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.934+0000 7f93db269640 1 --2- 172.21.15.27:0/468033690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f93d4071140 0x7f93d415f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:07.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.934+0000 7f93db269640 1 -- 172.21.15.27:0/468033690 >> 172.21.15.27:0/468033690 conn(0x7f93d40ceb50 msgr2=0x7f93d40cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:07.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.934+0000 7f93db269640 1 -- 172.21.15.27:0/468033690 shutdown_connections 2023-12-19T18:16:07.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:07.935+0000 7f93db269640 1 -- 172.21.15.27:0/468033690 wait complete. 2023-12-19T18:16:07.955 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-19T18:16:07.955 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:07.955 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-12-19T18:16:08.058 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:16:08.059 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:16:08.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:08.059 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-19T18:16:08.109 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:16:08.110 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:08.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.352+0000 7feb0bfff640 1 Processor -- start 2023-12-19T18:16:08.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.352+0000 7feb0bfff640 1 -- start start 2023-12-19T18:16:08.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.353+0000 7feb0bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feb0c1604d0 0x7feb0c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.354+0000 7feb0bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7feb0c0d2350 0x7feb0c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.354+0000 7feb0bfff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c0b5e90 con 0x7feb0c0d0300 2023-12-19T18:16:08.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.354+0000 7feb0bfff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c0d2b30 con 0x7feb0c1604d0 2023-12-19T18:16:08.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.354+0000 7feb0bfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c0d2850 con 0x7feb0c0d2350 2023-12-19T18:16:08.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.354+0000 7feb0affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feb0c1604d0 0x7feb0c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.354+0000 7feb0b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7feb0c0d0300 0x7feb0c0d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45144/0 (socket says 172.21.15.27:45144) 2023-12-19T18:16:08.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.354+0000 7feb0b7fe640 1 -- 172.21.15.27:0/3132961608 learned_addr learned my addr 172.21.15.27:0/3132961608 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:08.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.354+0000 7feb0a7fc640 1 --2- 172.21.15.27:0/3132961608 >> v2:172.21.15.192:3300/0 conn(0x7feb0c0d2350 0x7feb0c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.355+0000 7feb09ffb640 1 -- 172.21.15.27:0/3132961608 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 892326919 0 0) 0x7feb0c0b5e90 con 0x7feb0c0d0300 2023-12-19T18:16:08.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.355+0000 7feb09ffb640 1 -- 172.21.15.27:0/3132961608 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaf8003520 con 0x7feb0c0d0300 2023-12-19T18:16:08.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.355+0000 7feb0affd640 1 -- 172.21.15.27:0/3132961608 >> v2:172.21.15.192:3300/0 conn(0x7feb0c0d2350 msgr2=0x7feb0c0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:08.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.355+0000 7feb0affd640 1 --2- 172.21.15.27:0/3132961608 >> v2:172.21.15.192:3300/0 conn(0x7feb0c0d2350 0x7feb0c0b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.355+0000 7feb0affd640 1 -- 172.21.15.27:0/3132961608 >> v1:172.21.15.27:6789/0 conn(0x7feb0c0d0300 legacy=0x7feb0c0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.355+0000 7feb0affd640 1 -- 172.21.15.27:0/3132961608 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c0cded0 con 0x7feb0c1604d0 2023-12-19T18:16:08.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.356+0000 7feb0affd640 1 --2- 172.21.15.27:0/3132961608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c1604d0 0x7feb0c0b47e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feb0c072120 tx=0x7feaf402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abb7057e4ab9cdbf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.356+0000 7feb09ffb640 1 -- 172.21.15.27:0/3132961608 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4031830 con 0x7feb0c1604d0 2023-12-19T18:16:08.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.356+0000 7feb09ffb640 1 -- 172.21.15.27:0/3132961608 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaf4031990 con 0x7feb0c1604d0 2023-12-19T18:16:08.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.356+0000 7feb09ffb640 1 -- 172.21.15.27:0/3132961608 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaf4031d20 con 0x7feb0c1604d0 2023-12-19T18:16:08.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.357+0000 7feb0bfff640 1 -- 172.21.15.27:0/3132961608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c1604d0 msgr2=0x7feb0c0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.357+0000 7feb0bfff640 1 --2- 172.21.15.27:0/3132961608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c1604d0 0x7feb0c0b47e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7feb0c072120 tx=0x7feaf402b8d0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.357+0000 7feb0bfff640 1 -- 172.21.15.27:0/3132961608 shutdown_connections 2023-12-19T18:16:08.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.357+0000 7feb0bfff640 1 --2- 172.21.15.27:0/3132961608 >> v2:172.21.15.192:3300/0 conn(0x7feb0c0d2350 0x7feb0c0b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.357+0000 7feb0bfff640 1 --2- 172.21.15.27:0/3132961608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c1604d0 0x7feb0c0b47e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.357+0000 7feb0bfff640 1 -- 172.21.15.27:0/3132961608 >> 172.21.15.27:0/3132961608 conn(0x7feb0c0cf7e0 msgr2=0x7feb0c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.357+0000 7feb0bfff640 1 -- 172.21.15.27:0/3132961608 shutdown_connections 2023-12-19T18:16:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.357+0000 7feb0bfff640 1 -- 172.21.15.27:0/3132961608 wait complete. 2023-12-19T18:16:08.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.359+0000 7feb0bfff640 1 Processor -- start 2023-12-19T18:16:08.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.359+0000 7feb0bfff640 1 -- start start 2023-12-19T18:16:08.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.360+0000 7feb0bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb0c0d0300 0x7feb0c1f35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.360+0000 7feb0bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c0d2350 0x7feb0c1f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.360+0000 7feb0bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb0c1604d0 0x7feb0c1f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.360+0000 7feb0bfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1f3410 con 0x7feb0c0d2350 2023-12-19T18:16:08.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.360+0000 7feb0bfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0c9e30 con 0x7feb0c1604d0 2023-12-19T18:16:08.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.360+0000 7feb0bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c0ca110 con 0x7feb0c0d0300 2023-12-19T18:16:08.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.360+0000 7feb0a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c0d2350 0x7feb0c1f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.360+0000 7feb0a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c0d2350 0x7feb0c1f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45624/0 (socket says 172.21.15.27:45624) 2023-12-19T18:16:08.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.360+0000 7feb0a7fc640 1 -- 172.21.15.27:0/2044635998 learned_addr learned my addr 172.21.15.27:0/2044635998 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:08.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.361+0000 7feb0affd640 1 --2- 172.21.15.27:0/2044635998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb0c0d0300 0x7feb0c1f35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.361+0000 7feb0a7fc640 1 -- 172.21.15.27:0/2044635998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb0c0d0300 msgr2=0x7feb0c1f35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.361+0000 7feb0a7fc640 1 --2- 172.21.15.27:0/2044635998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb0c0d0300 0x7feb0c1f35d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.361+0000 7feb0a7fc640 1 -- 172.21.15.27:0/2044635998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb0c1604d0 msgr2=0x7feb0c1f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.361+0000 7feb0a7fc640 1 --2- 172.21.15.27:0/2044635998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb0c1604d0 0x7feb0c1f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.361+0000 7feb0a7fc640 1 -- 172.21.15.27:0/2044635998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf4009400 con 0x7feb0c0d2350 2023-12-19T18:16:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.361+0000 7feb0a7fc640 1 --2- 172.21.15.27:0/2044635998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c0d2350 0x7feb0c1f2ed0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feafc009fd0 tx=0x7feafc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.361+0000 7feb037fe640 1 -- 172.21.15.27:0/2044635998 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc014620 con 0x7feb0c0d2350 2023-12-19T18:16:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.361+0000 7feb037fe640 1 -- 172.21.15.27:0/2044635998 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc014780 con 0x7feb0c0d2350 2023-12-19T18:16:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.361+0000 7feb037fe640 1 -- 172.21.15.27:0/2044635998 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc014b30 con 0x7feb0c0d2350 2023-12-19T18:16:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.361+0000 7feb0bfff640 1 -- 172.21.15.27:0/2044635998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c0ca3f0 con 0x7feb0c0d2350 2023-12-19T18:16:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.362+0000 7feb0bfff640 1 -- 172.21.15.27:0/2044635998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c0ca9b0 con 0x7feb0c0d2350 2023-12-19T18:16:08.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.363+0000 7feb0bfff640 1 -- 172.21.15.27:0/2044635998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c066f80 con 0x7feb0c0d2350 2023-12-19T18:16:08.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.364+0000 7feb037fe640 1 -- 172.21.15.27:0/2044635998 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7feafc014c90 con 0x7feb0c0d2350 2023-12-19T18:16:08.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.365+0000 7feb037fe640 1 --2- 172.21.15.27:0/2044635998 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feaec06a430 0x7feaec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.365+0000 7feb037fe640 1 -- 172.21.15.27:0/2044635998 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7feafc089310 con 0x7feb0c0d2350 2023-12-19T18:16:08.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.365+0000 7feb0affd640 1 --2- 172.21.15.27:0/2044635998 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feaec06a430 0x7feaec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.366+0000 7feb0affd640 1 --2- 172.21.15.27:0/2044635998 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feaec06a430 0x7feaec06c8f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feaf4036000 tx=0x7feaf403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.369+0000 7feb037fe640 1 -- 172.21.15.27:0/2044635998 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7feafc0548d0 con 0x7feb0c0d2350 2023-12-19T18:16:08.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.603+0000 7feb0bfff640 1 -- 172.21.15.27:0/2044635998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb0c04e840 con 0x7feb0c0d2350 2023-12-19T18:16:08.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.604+0000 7feb037fe640 1 -- 172.21.15.27:0/2044635998 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7feafc057ef0 con 0x7feb0c0d2350 2023-12-19T18:16:08.604 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:08.604 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3679393226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3679393226},{"type":"v1","addr":"172.21.15.27:6835","nonce":3679393226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/809712436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":809712436},{"type":"v1","addr":"172.21.15.192:6835","nonce":809712436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3400689212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3400689212},{"type":"v1","addr":"172.21.15.27:6833","nonce":3400689212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:54.035577+0000","modified":"2023-12-19T18:14:57.547613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/1878426383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1878426383},{"type":"v1","addr":"172.21.15.192:6837","nonce":1878426383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T18:16:08.604 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2023-12-19T18:16:08.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.606+0000 7feb0bfff640 1 -- 172.21.15.27:0/2044635998 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feaec06a430 msgr2=0x7feaec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.606+0000 7feb0bfff640 1 --2- 172.21.15.27:0/2044635998 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feaec06a430 0x7feaec06c8f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feaf4036000 tx=0x7feaf403c040 comp rx=0 tx=0).stop 2023-12-19T18:16:08.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.606+0000 7feb0bfff640 1 -- 172.21.15.27:0/2044635998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c0d2350 msgr2=0x7feb0c1f2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.606+0000 7feb0bfff640 1 --2- 172.21.15.27:0/2044635998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c0d2350 0x7feb0c1f2ed0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feafc009fd0 tx=0x7feafc00e4b0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.606+0000 7feb0bfff640 1 -- 172.21.15.27:0/2044635998 shutdown_connections 2023-12-19T18:16:08.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.606+0000 7feb0bfff640 1 --2- 172.21.15.27:0/2044635998 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feaec06a430 0x7feaec06c8f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.606+0000 7feb0bfff640 1 --2- 172.21.15.27:0/2044635998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feb0c1604d0 0x7feb0c1f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.606+0000 7feb0bfff640 1 --2- 172.21.15.27:0/2044635998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb0c0d2350 0x7feb0c1f2ed0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.606+0000 7feb0bfff640 1 --2- 172.21.15.27:0/2044635998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feb0c0d0300 0x7feb0c1f35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.606+0000 7feb0bfff640 1 -- 172.21.15.27:0/2044635998 >> 172.21.15.27:0/2044635998 conn(0x7feb0c0cf7e0 msgr2=0x7feb0c0b4660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:08.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.606+0000 7feb0bfff640 1 -- 172.21.15.27:0/2044635998 shutdown_connections 2023-12-19T18:16:08.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.607+0000 7feb0bfff640 1 -- 172.21.15.27:0/2044635998 wait complete. 2023-12-19T18:16:08.628 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.851+0000 7efd49462640 1 Processor -- start 2023-12-19T18:16:08.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.851+0000 7efd49462640 1 -- start start 2023-12-19T18:16:08.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.852+0000 7efd49462640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efd440d2350 0x7efd440b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.852+0000 7efd49462640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd440b5e90 con 0x7efd440b4ef0 2023-12-19T18:16:08.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.852+0000 7efd49462640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd440d2850 con 0x7efd441604d0 2023-12-19T18:16:08.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.852+0000 7efd49462640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd440d2b30 con 0x7efd440d2350 2023-12-19T18:16:08.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.853+0000 7efd437fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efd440b4ef0 0x7efd440b6300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45152/0 (socket says 172.21.15.27:45152) 2023-12-19T18:16:08.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.853+0000 7efd437fe640 1 -- 172.21.15.27:0/3299340257 learned_addr learned my addr 172.21.15.27:0/3299340257 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:08.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.853+0000 7efd48c61640 1 --2- 172.21.15.27:0/3299340257 >> v2:172.21.15.192:3300/0 conn(0x7efd440d2350 0x7efd440b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.853+0000 7efd42ffd640 1 -- 172.21.15.27:0/3299340257 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4172993975 0 0) 0x7efd440b5e90 con 0x7efd440b4ef0 2023-12-19T18:16:08.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.853+0000 7efd42ffd640 1 -- 172.21.15.27:0/3299340257 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd30003520 con 0x7efd440b4ef0 2023-12-19T18:16:08.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.854+0000 7efd42ffd640 1 -- 172.21.15.27:0/3299340257 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1560519740 0 0) 0x7efd30003520 con 0x7efd440b4ef0 2023-12-19T18:16:08.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.854+0000 7efd48c61640 1 -- 172.21.15.27:0/3299340257 >> v1:172.21.15.192:6789/0 conn(0x7efd441604d0 legacy=0x7efd440b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.854+0000 7efd48c61640 1 -- 172.21.15.27:0/3299340257 >> v1:172.21.15.27:6789/0 conn(0x7efd440b4ef0 legacy=0x7efd440b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.854+0000 7efd48c61640 1 -- 172.21.15.27:0/3299340257 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd440cded0 con 0x7efd440d2350 2023-12-19T18:16:08.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.855+0000 7efd48c61640 1 --2- 172.21.15.27:0/3299340257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd440d2350 0x7efd440b6a10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0097a0 tx=0x7efd3c02b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3017ee8da5fe1208 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.856+0000 7efd42ffd640 1 -- 172.21.15.27:0/3299340257 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c02f050 con 0x7efd440d2350 2023-12-19T18:16:08.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.856+0000 7efd42ffd640 1 -- 172.21.15.27:0/3299340257 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c031500 con 0x7efd440d2350 2023-12-19T18:16:08.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.856+0000 7efd42ffd640 1 -- 172.21.15.27:0/3299340257 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd2c001670 con 0x7efd440d2350 2023-12-19T18:16:08.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.856+0000 7efd49462640 1 -- 172.21.15.27:0/3299340257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd440d2350 msgr2=0x7efd440b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.856+0000 7efd49462640 1 --2- 172.21.15.27:0/3299340257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd440d2350 0x7efd440b6a10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7efd3c0097a0 tx=0x7efd3c02b820 comp rx=0 tx=0).stop 2023-12-19T18:16:08.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.857+0000 7efd49462640 1 -- 172.21.15.27:0/3299340257 shutdown_connections 2023-12-19T18:16:08.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.857+0000 7efd49462640 1 --2- 172.21.15.27:0/3299340257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd440d2350 0x7efd440b6a10 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.857+0000 7efd49462640 1 -- 172.21.15.27:0/3299340257 >> 172.21.15.27:0/3299340257 conn(0x7efd440cf7e0 msgr2=0x7efd440cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:08.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.857+0000 7efd49462640 1 -- 172.21.15.27:0/3299340257 shutdown_connections 2023-12-19T18:16:08.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.857+0000 7efd49462640 1 -- 172.21.15.27:0/3299340257 wait complete. 2023-12-19T18:16:08.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.858+0000 7efd49462640 1 Processor -- start 2023-12-19T18:16:08.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.858+0000 7efd49462640 1 -- start start 2023-12-19T18:16:08.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.859+0000 7efd49462640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd440b4ef0 0x7efd441550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.859+0000 7efd49462640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd440d2350 0x7efd44154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.859+0000 7efd49462640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd441604d0 0x7efd441f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd49462640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd44158520 con 0x7efd441604d0 2023-12-19T18:16:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd49462640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd44154a50 con 0x7efd440b4ef0 2023-12-19T18:16:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd49462640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd44158730 con 0x7efd440d2350 2023-12-19T18:16:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd48c61640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd441604d0 0x7efd441f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd48c61640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd441604d0 0x7efd441f62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45628/0 (socket says 172.21.15.27:45628) 2023-12-19T18:16:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd48c61640 1 -- 172.21.15.27:0/736703341 learned_addr learned my addr 172.21.15.27:0/736703341 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd43fff640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd440b4ef0 0x7efd441550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd437fe640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd440d2350 0x7efd44154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd48c61640 1 -- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd440d2350 msgr2=0x7efd44154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd48c61640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd440d2350 0x7efd44154280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd48c61640 1 -- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd440b4ef0 msgr2=0x7efd441550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd48c61640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd440b4ef0 0x7efd441550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:08.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd48c61640 1 -- 172.21.15.27:0/736703341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd3c009400 con 0x7efd441604d0 2023-12-19T18:16:08.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.860+0000 7efd43fff640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd440b4ef0 0x7efd441550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:08.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.861+0000 7efd48c61640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd441604d0 0x7efd441f62b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd3c036040 tx=0x7efd3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.861+0000 7efd40ff9640 1 -- 172.21.15.27:0/736703341 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c004230 con 0x7efd441604d0 2023-12-19T18:16:08.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.861+0000 7efd40ff9640 1 -- 172.21.15.27:0/736703341 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd3c004390 con 0x7efd441604d0 2023-12-19T18:16:08.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.861+0000 7efd40ff9640 1 -- 172.21.15.27:0/736703341 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd3c03d960 con 0x7efd441604d0 2023-12-19T18:16:08.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.861+0000 7efd49462640 1 -- 172.21.15.27:0/736703341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd44158a10 con 0x7efd441604d0 2023-12-19T18:16:08.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.861+0000 7efd49462640 1 -- 172.21.15.27:0/736703341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd44158fd0 con 0x7efd441604d0 2023-12-19T18:16:08.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.862+0000 7efd49462640 1 -- 172.21.15.27:0/736703341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd44066f80 con 0x7efd441604d0 2023-12-19T18:16:08.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.863+0000 7efd40ff9640 1 -- 172.21.15.27:0/736703341 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efd3c03a040 con 0x7efd441604d0 2023-12-19T18:16:08.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.864+0000 7efd40ff9640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd2406a430 0x7efd2406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:08.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.864+0000 7efd40ff9640 1 -- 172.21.15.27:0/736703341 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7efd3c0aa3d0 con 0x7efd441604d0 2023-12-19T18:16:08.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.865+0000 7efd43fff640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd2406a430 0x7efd2406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.866+0000 7efd43fff640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd2406a430 0x7efd2406c8f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efd440b8ac0 tx=0x7efd34009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:08.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:08.869+0000 7efd40ff9640 1 -- 172.21.15.27:0/736703341 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efd3c075900 con 0x7efd441604d0 2023-12-19T18:16:09.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.068+0000 7efd49462640 1 -- 172.21.15.27:0/736703341 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd441591c0 con 0x7efd441604d0 2023-12-19T18:16:09.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.071+0000 7efd40ff9640 1 -- 172.21.15.27:0/736703341 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16090 (secure 0 0 0) 0x7efd3c078f20 con 0x7efd441604d0 2023-12-19T18:16:09.072 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:09.072 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:15:43.580746+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:09.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.075+0000 7efd49462640 1 -- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd2406a430 msgr2=0x7efd2406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.075+0000 7efd49462640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd2406a430 0x7efd2406c8f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7efd440b8ac0 tx=0x7efd34009040 comp rx=0 tx=0).stop 2023-12-19T18:16:09.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.075+0000 7efd49462640 1 -- 172.21.15.27:0/736703341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd441604d0 msgr2=0x7efd441f62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.075+0000 7efd49462640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd441604d0 0x7efd441f62b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efd3c036040 tx=0x7efd3c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.075+0000 7efd49462640 1 -- 172.21.15.27:0/736703341 shutdown_connections 2023-12-19T18:16:09.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.075+0000 7efd49462640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd2406a430 0x7efd2406c8f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.075+0000 7efd49462640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd441604d0 0x7efd441f62b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.075+0000 7efd49462640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd440d2350 0x7efd44154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.075+0000 7efd49462640 1 --2- 172.21.15.27:0/736703341 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd440b4ef0 0x7efd441550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.075+0000 7efd49462640 1 -- 172.21.15.27:0/736703341 >> 172.21.15.27:0/736703341 conn(0x7efd440cf7e0 msgr2=0x7efd440b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:09.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.075+0000 7efd49462640 1 -- 172.21.15.27:0/736703341 shutdown_connections 2023-12-19T18:16:09.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.075+0000 7efd49462640 1 -- 172.21.15.27:0/736703341 wait complete. 2023-12-19T18:16:09.096 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:14:52.124331+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:14:53.068734+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:16:09.096 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:16:09.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.333+0000 7f527759e640 1 Processor -- start 2023-12-19T18:16:09.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.333+0000 7f527759e640 1 -- start start 2023-12-19T18:16:09.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.335+0000 7f527759e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52780cf540 0x7f527815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.335+0000 7f527759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f527815aaf0 0x7f527806b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.335+0000 7f527759e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f527806b4c0 con 0x7f52780cf160 2023-12-19T18:16:09.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.335+0000 7f527759e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5278161bd0 con 0x7f527815aaf0 2023-12-19T18:16:09.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.335+0000 7f527759e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5278070080 con 0x7f52780cf540 2023-12-19T18:16:09.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.335+0000 7f5276d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f527815aaf0 0x7f527806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.336+0000 7f5276d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f527815aaf0 0x7f527806b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45640/0 (socket says 172.21.15.27:45640) 2023-12-19T18:16:09.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.336+0000 7f5276d9d640 1 -- 172.21.15.27:0/2463756425 learned_addr learned my addr 172.21.15.27:0/2463756425 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:09.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.336+0000 7f526dd9b640 1 --2- 172.21.15.27:0/2463756425 >> v2:172.21.15.192:3300/0 conn(0x7f52780cf540 0x7f527815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.336+0000 7f5276d9d640 1 -- 172.21.15.27:0/2463756425 >> v2:172.21.15.192:3300/0 conn(0x7f52780cf540 msgr2=0x7f527815a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:09.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.336+0000 7f5276d9d640 1 --2- 172.21.15.27:0/2463756425 >> v2:172.21.15.192:3300/0 conn(0x7f52780cf540 0x7f527815a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.336+0000 7f5276d9d640 1 -- 172.21.15.27:0/2463756425 >> v1:172.21.15.192:6789/0 conn(0x7f52780cf160 legacy=0x7f527815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:09.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.336+0000 7f5276d9d640 1 -- 172.21.15.27:0/2463756425 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f527806d140 con 0x7f527815aaf0 2023-12-19T18:16:09.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.337+0000 7f5276d9d640 1 --2- 172.21.15.27:0/2463756425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f527815aaf0 0x7f527806b930 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f527806d830 tx=0x7f527002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d4fd50ad31aa316 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.337+0000 7f5275d9b640 1 -- 172.21.15.27:0/2463756425 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527002c9f0 con 0x7f527815aaf0 2023-12-19T18:16:09.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.338+0000 7f5275d9b640 1 -- 172.21.15.27:0/2463756425 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527002cb50 con 0x7f527815aaf0 2023-12-19T18:16:09.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.338+0000 7f5275d9b640 1 -- 172.21.15.27:0/2463756425 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52700343f0 con 0x7f527815aaf0 2023-12-19T18:16:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.338+0000 7f527759e640 1 -- 172.21.15.27:0/2463756425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f527815aaf0 msgr2=0x7f527806b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.338+0000 7f527759e640 1 --2- 172.21.15.27:0/2463756425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f527815aaf0 0x7f527806b930 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f527806d830 tx=0x7f527002be50 comp rx=0 tx=0).stop 2023-12-19T18:16:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.338+0000 7f527759e640 1 -- 172.21.15.27:0/2463756425 shutdown_connections 2023-12-19T18:16:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.338+0000 7f527759e640 1 --2- 172.21.15.27:0/2463756425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f527815aaf0 0x7f527806b930 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.338+0000 7f527759e640 1 --2- 172.21.15.27:0/2463756425 >> v2:172.21.15.192:3300/0 conn(0x7f52780cf540 0x7f527815a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.338+0000 7f527759e640 1 -- 172.21.15.27:0/2463756425 >> 172.21.15.27:0/2463756425 conn(0x7f527806a060 msgr2=0x7f527806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.338+0000 7f527759e640 1 -- 172.21.15.27:0/2463756425 shutdown_connections 2023-12-19T18:16:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.338+0000 7f527759e640 1 -- 172.21.15.27:0/2463756425 wait complete. 2023-12-19T18:16:09.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.340+0000 7f527759e640 1 Processor -- start 2023-12-19T18:16:09.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.340+0000 7f527759e640 1 -- start start 2023-12-19T18:16:09.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.341+0000 7f527759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52780cf160 0x7f5278147560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.341+0000 7f527759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52780cf540 0x7f52780b0820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.341+0000 7f527759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f527815aaf0 0x7f52780b0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.341+0000 7f527759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5278147aa0 con 0x7f52780cf540 2023-12-19T18:16:09.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.341+0000 7f527759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52780cd380 con 0x7f52780cf160 2023-12-19T18:16:09.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.341+0000 7f527759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52780cd660 con 0x7f527815aaf0 2023-12-19T18:16:09.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.342+0000 7f526dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52780cf540 0x7f52780b0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.342+0000 7f526dd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52780cf540 0x7f52780b0820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45650/0 (socket says 172.21.15.27:45650) 2023-12-19T18:16:09.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.342+0000 7f526dd9b640 1 -- 172.21.15.27:0/455746990 learned_addr learned my addr 172.21.15.27:0/455746990 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.342+0000 7f526dd9b640 1 -- 172.21.15.27:0/455746990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f527815aaf0 msgr2=0x7f52780b0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.342+0000 7f526dd9b640 1 --2- 172.21.15.27:0/455746990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f527815aaf0 0x7f52780b0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.342+0000 7f526dd9b640 1 -- 172.21.15.27:0/455746990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52780cf160 msgr2=0x7f5278147560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.342+0000 7f527659c640 1 --2- 172.21.15.27:0/455746990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52780cf160 0x7f5278147560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.342+0000 7f526dd9b640 1 --2- 172.21.15.27:0/455746990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52780cf160 0x7f5278147560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.342+0000 7f526dd9b640 1 -- 172.21.15.27:0/455746990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5270009b30 con 0x7f52780cf540 2023-12-19T18:16:09.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.343+0000 7f526dd9b640 1 --2- 172.21.15.27:0/455746990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52780cf540 0x7f52780b0820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f526000d800 tx=0x7f526000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.343+0000 7f526f7fe640 1 -- 172.21.15.27:0/455746990 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000b6c0 con 0x7f52780cf540 2023-12-19T18:16:09.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.343+0000 7f526f7fe640 1 -- 172.21.15.27:0/455746990 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f526000b820 con 0x7f52780cf540 2023-12-19T18:16:09.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.343+0000 7f526f7fe640 1 -- 172.21.15.27:0/455746990 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f526000bbb0 con 0x7f52780cf540 2023-12-19T18:16:09.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.343+0000 7f527759e640 1 -- 172.21.15.27:0/455746990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52780cd940 con 0x7f52780cf540 2023-12-19T18:16:09.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.343+0000 7f527759e640 1 -- 172.21.15.27:0/455746990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52780cc650 con 0x7f52780cf540 2023-12-19T18:16:09.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.344+0000 7f527759e640 1 -- 172.21.15.27:0/455746990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5278160240 con 0x7f52780cf540 2023-12-19T18:16:09.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.345+0000 7f526f7fe640 1 -- 172.21.15.27:0/455746990 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f526000bd10 con 0x7f52780cf540 2023-12-19T18:16:09.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.346+0000 7f526f7fe640 1 --2- 172.21.15.27:0/455746990 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f525806a430 0x7f525806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.346+0000 7f526f7fe640 1 -- 172.21.15.27:0/455746990 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5260088f00 con 0x7f52780cf540 2023-12-19T18:16:09.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.347+0000 7f527659c640 1 --2- 172.21.15.27:0/455746990 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f525806a430 0x7f525806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.348+0000 7f527659c640 1 --2- 172.21.15.27:0/455746990 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f525806a430 0x7f525806c8f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f526800a750 tx=0x7f5268008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.350+0000 7f526f7fe640 1 -- 172.21.15.27:0/455746990 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5260054430 con 0x7f52780cf540 2023-12-19T18:16:09.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.544+0000 7f527759e640 1 -- 172.21.15.27:0/455746990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52780b2d60 con 0x7f52780cf540 2023-12-19T18:16:09.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.545+0000 7f526f7fe640 1 -- 172.21.15.27:0/455746990 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5260057a50 con 0x7f52780cf540 2023-12-19T18:16:09.546 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:16:09.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.548+0000 7f527759e640 1 -- 172.21.15.27:0/455746990 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f525806a430 msgr2=0x7f525806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.548+0000 7f527759e640 1 --2- 172.21.15.27:0/455746990 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f525806a430 0x7f525806c8f0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f526800a750 tx=0x7f5268008040 comp rx=0 tx=0).stop 2023-12-19T18:16:09.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.548+0000 7f527759e640 1 -- 172.21.15.27:0/455746990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52780cf540 msgr2=0x7f52780b0820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.548+0000 7f527759e640 1 --2- 172.21.15.27:0/455746990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52780cf540 0x7f52780b0820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f526000d800 tx=0x7f526000dcc0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.548+0000 7f527759e640 1 -- 172.21.15.27:0/455746990 shutdown_connections 2023-12-19T18:16:09.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.548+0000 7f527759e640 1 --2- 172.21.15.27:0/455746990 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f525806a430 0x7f525806c8f0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.548+0000 7f527759e640 1 --2- 172.21.15.27:0/455746990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f527815aaf0 0x7f52780b0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.548+0000 7f527759e640 1 --2- 172.21.15.27:0/455746990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52780cf540 0x7f52780b0820 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.548+0000 7f527759e640 1 --2- 172.21.15.27:0/455746990 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52780cf160 0x7f5278147560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.548+0000 7f527759e640 1 -- 172.21.15.27:0/455746990 >> 172.21.15.27:0/455746990 conn(0x7f527806a060 msgr2=0x7f52780b1750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:09.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.548+0000 7f527759e640 1 -- 172.21.15.27:0/455746990 shutdown_connections 2023-12-19T18:16:09.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.548+0000 7f527759e640 1 -- 172.21.15.27:0/455746990 wait complete. 2023-12-19T18:16:09.567 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:16:09.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.798+0000 7f7222496640 1 Processor -- start 2023-12-19T18:16:09.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.798+0000 7f7222496640 1 -- start start 2023-12-19T18:16:09.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.799+0000 7f7222496640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f721c0affc0 0x7f721c0cd780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.799+0000 7f7222496640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f721c06c210 0x7f721c0cdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.799+0000 7f7222496640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f721c0d0710 con 0x7f721c161810 2023-12-19T18:16:09.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.799+0000 7f7222496640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f721c0cf2c0 con 0x7f721c0affc0 2023-12-19T18:16:09.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.799+0000 7f7222496640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c0cb3e0 con 0x7f721c06c210 2023-12-19T18:16:09.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.800+0000 7f721bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f721c0affc0 0x7f721c0cd780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.800+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f721c06c210 0x7f721c0cdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.800+0000 7f721b7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f721c06c210 0x7f721c0cdcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:47004/0 (socket says 172.21.15.27:47004) 2023-12-19T18:16:09.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.800+0000 7f721bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f721c0affc0 0x7f721c0cd780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:39112/0 (socket says 172.21.15.27:39112) 2023-12-19T18:16:09.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.800+0000 7f721b7fe640 1 -- 172.21.15.27:0/1612291287 learned_addr learned my addr 172.21.15.27:0/1612291287 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:09.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.801+0000 7f721b7fe640 1 -- 172.21.15.27:0/1612291287 >> v2:172.21.15.192:3300/0 conn(0x7f721c0affc0 msgr2=0x7f721c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:09.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.801+0000 7f721b7fe640 1 --2- 172.21.15.27:0/1612291287 >> v2:172.21.15.192:3300/0 conn(0x7f721c0affc0 0x7f721c0cd780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.801+0000 7f721b7fe640 1 -- 172.21.15.27:0/1612291287 >> v1:172.21.15.192:6789/0 conn(0x7f721c161810 legacy=0x7f721c0b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.801+0000 7f721b7fe640 1 -- 172.21.15.27:0/1612291287 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c06bd90 con 0x7f721c06c210 2023-12-19T18:16:09.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.801+0000 7f721bfff640 1 --2- 172.21.15.27:0/1612291287 >> v2:172.21.15.192:3300/0 conn(0x7f721c0affc0 0x7f721c0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.802+0000 7f721b7fe640 1 --2- 172.21.15.27:0/1612291287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f721c06c210 0x7f721c0cdcc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7204009870 tx=0x7f720402b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5642846af95515 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.803+0000 7f721affd640 1 -- 172.21.15.27:0/1612291287 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204031640 con 0x7f721c06c210 2023-12-19T18:16:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.803+0000 7f721affd640 1 -- 172.21.15.27:0/1612291287 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72040317a0 con 0x7f721c06c210 2023-12-19T18:16:09.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.803+0000 7f721affd640 1 -- 172.21.15.27:0/1612291287 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7204031b30 con 0x7f721c06c210 2023-12-19T18:16:09.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.803+0000 7f7222496640 1 -- 172.21.15.27:0/1612291287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f721c06c210 msgr2=0x7f721c0cdcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.803+0000 7f7222496640 1 --2- 172.21.15.27:0/1612291287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f721c06c210 0x7f721c0cdcc0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7204009870 tx=0x7f720402b8f0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.803+0000 7f7222496640 1 -- 172.21.15.27:0/1612291287 shutdown_connections 2023-12-19T18:16:09.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.803+0000 7f7222496640 1 --2- 172.21.15.27:0/1612291287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f721c06c210 0x7f721c0cdcc0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.803+0000 7f7222496640 1 --2- 172.21.15.27:0/1612291287 >> v2:172.21.15.192:3300/0 conn(0x7f721c0affc0 0x7f721c0cd780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.803+0000 7f7222496640 1 -- 172.21.15.27:0/1612291287 >> 172.21.15.27:0/1612291287 conn(0x7f721c06f1f0 msgr2=0x7f721c06f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:09.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.803+0000 7f7222496640 1 -- 172.21.15.27:0/1612291287 shutdown_connections 2023-12-19T18:16:09.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.804+0000 7f7222496640 1 -- 172.21.15.27:0/1612291287 wait complete. 2023-12-19T18:16:09.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.805+0000 7f7222496640 1 Processor -- start 2023-12-19T18:16:09.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.806+0000 7f7222496640 1 -- start start 2023-12-19T18:16:09.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.806+0000 7f7222496640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f721c06c210 0x7f721c1f0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.806+0000 7f7222496640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c0affc0 0x7f721c1f39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f7222496640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f721c161810 0x7f721c157990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f7222496640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c1f3ef0 con 0x7f721c06c210 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f7222496640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f721c155fb0 con 0x7f721c0affc0 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f7222496640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f721c156290 con 0x7f721c161810 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f721bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f721c06c210 0x7f721c1f0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f721bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f721c06c210 0x7f721c1f0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45662/0 (socket says 172.21.15.27:45662) 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f721bfff640 1 -- 172.21.15.27:0/207538267 learned_addr learned my addr 172.21.15.27:0/207538267 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f721b7fe640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c0affc0 0x7f721c1f39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f721bfff640 1 -- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f721c161810 msgr2=0x7f721c157990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f7220a33640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f721c161810 0x7f721c157990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f721bfff640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f721c161810 0x7f721c157990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f721bfff640 1 -- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c0affc0 msgr2=0x7f721c1f39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f721bfff640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c0affc0 0x7f721c1f39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:09.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.807+0000 7f721bfff640 1 -- 172.21.15.27:0/207538267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f720c009a70 con 0x7f721c06c210 2023-12-19T18:16:09.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.808+0000 7f721bfff640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f721c06c210 0x7f721c1f0530 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f720c00efc0 tx=0x7f720c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.808+0000 7f7218ff9640 1 -- 172.21.15.27:0/207538267 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c014680 con 0x7f721c06c210 2023-12-19T18:16:09.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.808+0000 7f7218ff9640 1 -- 172.21.15.27:0/207538267 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f720c00f040 con 0x7f721c06c210 2023-12-19T18:16:09.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.808+0000 7f7222496640 1 -- 172.21.15.27:0/207538267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72040094d0 con 0x7f721c06c210 2023-12-19T18:16:09.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.808+0000 7f7218ff9640 1 -- 172.21.15.27:0/207538267 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f720c014a00 con 0x7f721c06c210 2023-12-19T18:16:09.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.808+0000 7f7222496640 1 -- 172.21.15.27:0/207538267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f721c156850 con 0x7f721c06c210 2023-12-19T18:16:09.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.810+0000 7f7222496640 1 -- 172.21.15.27:0/207538267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f721c067110 con 0x7f721c06c210 2023-12-19T18:16:09.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.811+0000 7f7218ff9640 1 -- 172.21.15.27:0/207538267 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f720c014b60 con 0x7f721c06c210 2023-12-19T18:16:09.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.811+0000 7f7218ff9640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f71fc06a460 0x7f71fc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:09.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.812+0000 7f7218ff9640 1 -- 172.21.15.27:0/207538267 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f720c089970 con 0x7f721c06c210 2023-12-19T18:16:09.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.812+0000 7f721b7fe640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f71fc06a460 0x7f71fc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:09.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.813+0000 7f721b7fe640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f71fc06a460 0x7f71fc06c920 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7204036000 tx=0x7f720403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:09.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:09.816+0000 7f7218ff9640 1 -- 172.21.15.27:0/207538267 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f720c054f30 con 0x7f721c06c210 2023-12-19T18:16:10.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.011+0000 7f7222496640 1 -- 172.21.15.27:0/207538267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f721c0b6d10 con 0x7f721c06c210 2023-12-19T18:16:10.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.012+0000 7f7218ff9640 1 -- 172.21.15.27:0/207538267 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f720c058550 con 0x7f721c06c210 2023-12-19T18:16:10.012 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:16:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.015+0000 7f7222496640 1 -- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f71fc06a460 msgr2=0x7f71fc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.015+0000 7f7222496640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f71fc06a460 0x7f71fc06c920 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f7204036000 tx=0x7f720403c040 comp rx=0 tx=0).stop 2023-12-19T18:16:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.015+0000 7f7222496640 1 -- 172.21.15.27:0/207538267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f721c06c210 msgr2=0x7f721c1f0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.015+0000 7f7222496640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f721c06c210 0x7f721c1f0530 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f720c00efc0 tx=0x7f720c00c490 comp rx=0 tx=0).stop 2023-12-19T18:16:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.015+0000 7f7222496640 1 -- 172.21.15.27:0/207538267 shutdown_connections 2023-12-19T18:16:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.015+0000 7f7222496640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f71fc06a460 0x7f71fc06c920 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.015+0000 7f7222496640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f721c161810 0x7f721c157990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.015+0000 7f7222496640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f721c0affc0 0x7f721c1f39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.015+0000 7f7222496640 1 --2- 172.21.15.27:0/207538267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f721c06c210 0x7f721c1f0530 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.015+0000 7f7222496640 1 -- 172.21.15.27:0/207538267 >> 172.21.15.27:0/207538267 conn(0x7f721c06f1f0 msgr2=0x7f721c0c8600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.015+0000 7f7222496640 1 -- 172.21.15.27:0/207538267 shutdown_connections 2023-12-19T18:16:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.015+0000 7f7222496640 1 -- 172.21.15.27:0/207538267 wait complete. 2023-12-19T18:16:10.032 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:16:10.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.266+0000 7f4991adc640 1 Processor -- start 2023-12-19T18:16:10.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.266+0000 7f4991adc640 1 -- start start 2023-12-19T18:16:10.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.267+0000 7f4991adc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f498c15cf80 0x7f498c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:10.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.267+0000 7f4991adc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f498c0b85e0 0x7f498c0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:10.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.268+0000 7f4991adc640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f498c0d0e10 con 0x7f498c0b8a60 2023-12-19T18:16:10.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.268+0000 7f4991adc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f498c0d1c70 con 0x7f498c15cf80 2023-12-19T18:16:10.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.268+0000 7f4991adc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c15d360 con 0x7f498c0b85e0 2023-12-19T18:16:10.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.268+0000 7f49912db640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f498c0b8a60 0x7f498c0cf500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45164/0 (socket says 172.21.15.27:45164) 2023-12-19T18:16:10.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.268+0000 7f4990ada640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f498c15cf80 0x7f498c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:10.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.268+0000 7f49912db640 1 -- 172.21.15.27:0/2560128067 learned_addr learned my addr 172.21.15.27:0/2560128067 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:10.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.268+0000 7f498bfff640 1 --2- 172.21.15.27:0/2560128067 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f498c0b85e0 0x7f498c0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:10.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.268+0000 7f4990ada640 1 -- 172.21.15.27:0/2560128067 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f498c0b85e0 msgr2=0x7f498c0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:10.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.268+0000 7f4990ada640 1 --2- 172.21.15.27:0/2560128067 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f498c0b85e0 0x7f498c0cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.268+0000 7f498b7fe640 1 -- 172.21.15.27:0/2560128067 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1103530546 0 0) 0x7f498c0d0e10 con 0x7f498c0b8a60 2023-12-19T18:16:10.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.268+0000 7f4990ada640 1 -- 172.21.15.27:0/2560128067 >> v1:172.21.15.27:6789/0 conn(0x7f498c0b8a60 legacy=0x7f498c0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.268+0000 7f4990ada640 1 -- 172.21.15.27:0/2560128067 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f498c0d1060 con 0x7f498c15cf80 2023-12-19T18:16:10.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.270+0000 7f4990ada640 1 --2- 172.21.15.27:0/2560128067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498c15cf80 0x7f498c0d2100 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f497c009d90 tx=0x7f497c02bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5431de71dc04f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:10.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.270+0000 7f498b7fe640 1 -- 172.21.15.27:0/2560128067 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c02cce0 con 0x7f498c15cf80 2023-12-19T18:16:10.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.270+0000 7f498b7fe640 1 -- 172.21.15.27:0/2560128067 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c02ce40 con 0x7f498c15cf80 2023-12-19T18:16:10.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.270+0000 7f498b7fe640 1 -- 172.21.15.27:0/2560128067 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c034660 con 0x7f498c15cf80 2023-12-19T18:16:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.271+0000 7f4991adc640 1 -- 172.21.15.27:0/2560128067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498c15cf80 msgr2=0x7f498c0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.271+0000 7f4991adc640 1 --2- 172.21.15.27:0/2560128067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498c15cf80 0x7f498c0d2100 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f497c009d90 tx=0x7f497c02bf30 comp rx=0 tx=0).stop 2023-12-19T18:16:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.271+0000 7f4991adc640 1 -- 172.21.15.27:0/2560128067 shutdown_connections 2023-12-19T18:16:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.271+0000 7f4991adc640 1 --2- 172.21.15.27:0/2560128067 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f498c0b85e0 0x7f498c0cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.271+0000 7f4991adc640 1 --2- 172.21.15.27:0/2560128067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498c15cf80 0x7f498c0d2100 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.271+0000 7f4991adc640 1 -- 172.21.15.27:0/2560128067 >> 172.21.15.27:0/2560128067 conn(0x7f498c072e20 msgr2=0x7f498c073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.271+0000 7f4991adc640 1 -- 172.21.15.27:0/2560128067 shutdown_connections 2023-12-19T18:16:10.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.271+0000 7f4991adc640 1 -- 172.21.15.27:0/2560128067 wait complete. 2023-12-19T18:16:10.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.273+0000 7f4991adc640 1 Processor -- start 2023-12-19T18:16:10.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.273+0000 7f4991adc640 1 -- start start 2023-12-19T18:16:10.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4991adc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498c0b85e0 0x7f498c15ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:10.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4991adc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f498c15cf80 0x7f498c15d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:10.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4991adc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f498c0ca210 0x7f498c15dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:10.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4991adc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0aa420 con 0x7f498c0b85e0 2023-12-19T18:16:10.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4991adc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0ca780 con 0x7f498c15cf80 2023-12-19T18:16:10.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4991adc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f498c0caa60 con 0x7f498c0ca210 2023-12-19T18:16:10.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4990ada640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498c0b85e0 0x7f498c15ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:10.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4990ada640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498c0b85e0 0x7f498c15ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45682/0 (socket says 172.21.15.27:45682) 2023-12-19T18:16:10.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4990ada640 1 -- 172.21.15.27:0/2893966366 learned_addr learned my addr 172.21.15.27:0/2893966366 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:10.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f49912db640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f498c0ca210 0x7f498c15dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:10.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f498bfff640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f498c15cf80 0x7f498c15d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:10.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4990ada640 1 -- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f498c0ca210 msgr2=0x7f498c15dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4990ada640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f498c0ca210 0x7f498c15dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4990ada640 1 -- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f498c15cf80 msgr2=0x7f498c15d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4990ada640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f498c15cf80 0x7f498c15d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.274+0000 7f4990ada640 1 -- 172.21.15.27:0/2893966366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c009a70 con 0x7f498c0b85e0 2023-12-19T18:16:10.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.275+0000 7f4990ada640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498c0b85e0 0x7f498c15ef90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f497c037040 tx=0x7f497c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:10.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.275+0000 7f49897fa640 1 -- 172.21.15.27:0/2893966366 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c03d600 con 0x7f498c0b85e0 2023-12-19T18:16:10.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.275+0000 7f49897fa640 1 -- 172.21.15.27:0/2893966366 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f497c03d760 con 0x7f498c0b85e0 2023-12-19T18:16:10.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.275+0000 7f49897fa640 1 -- 172.21.15.27:0/2893966366 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f497c03db10 con 0x7f498c0b85e0 2023-12-19T18:16:10.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.275+0000 7f4991adc640 1 -- 172.21.15.27:0/2893966366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f498c0cad40 con 0x7f498c0b85e0 2023-12-19T18:16:10.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.276+0000 7f4991adc640 1 -- 172.21.15.27:0/2893966366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f498c0cb300 con 0x7f498c0b85e0 2023-12-19T18:16:10.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.277+0000 7f4991adc640 1 -- 172.21.15.27:0/2893966366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f498c04e840 con 0x7f498c0b85e0 2023-12-19T18:16:10.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.278+0000 7f49897fa640 1 -- 172.21.15.27:0/2893966366 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f497c03b040 con 0x7f498c0b85e0 2023-12-19T18:16:10.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.279+0000 7f49897fa640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f496c06a430 0x7f496c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:10.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.279+0000 7f49897fa640 1 -- 172.21.15.27:0/2893966366 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f497c0aac50 con 0x7f498c0b85e0 2023-12-19T18:16:10.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.280+0000 7f498bfff640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f496c06a430 0x7f496c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:10.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.281+0000 7f498bfff640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f496c06a430 0x7f496c06c8f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f498c15bd90 tx=0x7f4974008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.285+0000 7f49897fa640 1 -- 172.21.15.27:0/2893966366 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f497c0b0020 con 0x7f498c0b85e0 2023-12-19T18:16:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.478+0000 7f4991adc640 1 -- 172.21.15.27:0/2893966366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f498c1e7720 con 0x7f498c0b85e0 2023-12-19T18:16:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.479+0000 7f49897fa640 1 -- 172.21.15.27:0/2893966366 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f497c0390a0 con 0x7f498c0b85e0 2023-12-19T18:16:10.480 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:16:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.482+0000 7f4991adc640 1 -- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f496c06a430 msgr2=0x7f496c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.482+0000 7f4991adc640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f496c06a430 0x7f496c06c8f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f498c15bd90 tx=0x7f4974008040 comp rx=0 tx=0).stop 2023-12-19T18:16:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.482+0000 7f4991adc640 1 -- 172.21.15.27:0/2893966366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498c0b85e0 msgr2=0x7f498c15ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.482+0000 7f4991adc640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498c0b85e0 0x7f498c15ef90 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f497c037040 tx=0x7f497c004a40 comp rx=0 tx=0).stop 2023-12-19T18:16:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.482+0000 7f4991adc640 1 -- 172.21.15.27:0/2893966366 shutdown_connections 2023-12-19T18:16:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.482+0000 7f4991adc640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f496c06a430 0x7f496c06c8f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.482+0000 7f4991adc640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f498c0ca210 0x7f498c15dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.482+0000 7f4991adc640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f498c15cf80 0x7f498c15d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.482+0000 7f4991adc640 1 --2- 172.21.15.27:0/2893966366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f498c0b85e0 0x7f498c15ef90 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.482+0000 7f4991adc640 1 -- 172.21.15.27:0/2893966366 >> 172.21.15.27:0/2893966366 conn(0x7f498c072e20 msgr2=0x7f498c15afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.482+0000 7f4991adc640 1 -- 172.21.15.27:0/2893966366 shutdown_connections 2023-12-19T18:16:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.482+0000 7f4991adc640 1 -- 172.21.15.27:0/2893966366 wait complete. 2023-12-19T18:16:10.504 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.742+0000 7f7187c1e640 1 Processor -- start 2023-12-19T18:16:10.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.742+0000 7f7187c1e640 1 -- start start 2023-12-19T18:16:10.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.743+0000 7f7187c1e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71800cdf90 0x7f71800ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:10.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.743+0000 7f7187c1e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f71800d1e00 0x7f71800d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:10.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.743+0000 7f7187c1e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71800ce930 con 0x7f718015d240 2023-12-19T18:16:10.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.743+0000 7f7187c1e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71800d1970 con 0x7f71800d1e00 2023-12-19T18:16:10.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.743+0000 7f7187c1e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f718006a5e0 con 0x7f71800cdf90 2023-12-19T18:16:10.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.744+0000 7f71851b9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71800cdf90 0x7f71800ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:10.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.744+0000 7f71861bb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f71800d1e00 0x7f71800d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:10.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.744+0000 7f71851b9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71800cdf90 0x7f71800ce3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:47032/0 (socket says 172.21.15.27:47032) 2023-12-19T18:16:10.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.744+0000 7f71861bb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f71800d1e00 0x7f71800d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:39142/0 (socket says 172.21.15.27:39142) 2023-12-19T18:16:10.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.744+0000 7f71851b9640 1 -- 172.21.15.27:0/2852590240 learned_addr learned my addr 172.21.15.27:0/2852590240 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:10.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.744+0000 7f71859ba640 1 --1- 172.21.15.27:0/2852590240 >> v1:172.21.15.192:6789/0 conn(0x7f718015d240 0x7f71801618d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:43600/0 (socket says 172.21.15.27:43600) 2023-12-19T18:16:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.745+0000 7f71851b9640 1 -- 172.21.15.27:0/2852590240 >> v2:172.21.15.192:3300/0 conn(0x7f71800d1e00 msgr2=0x7f71800d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.745+0000 7f71849b8640 1 -- 172.21.15.27:0/2852590240 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 20283754 0 0) 0x7f71800ce930 con 0x7f718015d240 2023-12-19T18:16:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.745+0000 7f71851b9640 1 --2- 172.21.15.27:0/2852590240 >> v2:172.21.15.192:3300/0 conn(0x7f71800d1e00 0x7f71800d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.745+0000 7f71851b9640 1 -- 172.21.15.27:0/2852590240 >> v1:172.21.15.192:6789/0 conn(0x7f718015d240 legacy=0x7f71801618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.745+0000 7f71851b9640 1 -- 172.21.15.27:0/2852590240 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f718006a880 con 0x7f71800cdf90 2023-12-19T18:16:10.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.746+0000 7f71851b9640 1 --2- 172.21.15.27:0/2852590240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71800cdf90 0x7f71800ce3f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7170009870 tx=0x7f717002ba20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=292af77b631069ba server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.747+0000 7f71849b8640 1 -- 172.21.15.27:0/2852590240 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717002f840 con 0x7f71800cdf90 2023-12-19T18:16:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.747+0000 7f71849b8640 1 -- 172.21.15.27:0/2852590240 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f717002f9a0 con 0x7f71800cdf90 2023-12-19T18:16:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.747+0000 7f71849b8640 1 -- 172.21.15.27:0/2852590240 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717002fd30 con 0x7f71800cdf90 2023-12-19T18:16:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.748+0000 7f7187c1e640 1 -- 172.21.15.27:0/2852590240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71800cdf90 msgr2=0x7f71800ce3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.748+0000 7f7187c1e640 1 --2- 172.21.15.27:0/2852590240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71800cdf90 0x7f71800ce3f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7170009870 tx=0x7f717002ba20 comp rx=0 tx=0).stop 2023-12-19T18:16:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.748+0000 7f7187c1e640 1 -- 172.21.15.27:0/2852590240 shutdown_connections 2023-12-19T18:16:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.748+0000 7f7187c1e640 1 --2- 172.21.15.27:0/2852590240 >> v2:172.21.15.192:3300/0 conn(0x7f71800d1e00 0x7f71800d2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.748+0000 7f7187c1e640 1 --2- 172.21.15.27:0/2852590240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71800cdf90 0x7f71800ce3f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.748+0000 7f7187c1e640 1 -- 172.21.15.27:0/2852590240 >> 172.21.15.27:0/2852590240 conn(0x7f71800d31e0 msgr2=0x7f71800d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.748+0000 7f7187c1e640 1 -- 172.21.15.27:0/2852590240 shutdown_connections 2023-12-19T18:16:10.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.748+0000 7f7187c1e640 1 -- 172.21.15.27:0/2852590240 wait complete. 2023-12-19T18:16:10.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.750+0000 7f7187c1e640 1 Processor -- start 2023-12-19T18:16:10.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.750+0000 7f7187c1e640 1 -- start start 2023-12-19T18:16:10.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.751+0000 7f7187c1e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71800cdf90 0x7f7180155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:10.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.751+0000 7f7187c1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71800d1e00 0x7f71801531b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:10.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.751+0000 7f7187c1e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f718015d240 0x7f71801f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:10.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.751+0000 7f7187c1e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7180156320 con 0x7f71800d1e00 2023-12-19T18:16:10.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.751+0000 7f7187c1e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f718014ded0 con 0x7f718015d240 2023-12-19T18:16:10.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.751+0000 7f7187c1e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71800cb780 con 0x7f71800cdf90 2023-12-19T18:16:10.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.751+0000 7f71851b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71800d1e00 0x7f71801531b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:10.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.751+0000 7f71851b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71800d1e00 0x7f71801531b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45690/0 (socket says 172.21.15.27:45690) 2023-12-19T18:16:10.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.751+0000 7f71851b9640 1 -- 172.21.15.27:0/2306287099 learned_addr learned my addr 172.21.15.27:0/2306287099 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:10.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.752+0000 7f71859ba640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71800cdf90 0x7f7180155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:10.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.752+0000 7f71861bb640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f718015d240 0x7f71801f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.752+0000 7f71851b9640 1 -- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71800cdf90 msgr2=0x7f7180155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.752+0000 7f71851b9640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71800cdf90 0x7f7180155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.752+0000 7f71851b9640 1 -- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f718015d240 msgr2=0x7f71801f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.752+0000 7f71851b9640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f718015d240 0x7f71801f2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.752+0000 7f71851b9640 1 -- 172.21.15.27:0/2306287099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71700094d0 con 0x7f71800d1e00 2023-12-19T18:16:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.752+0000 7f71861bb640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f718015d240 0x7f71801f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.752+0000 7f71851b9640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71800d1e00 0x7f71801531b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7170000c00 tx=0x7f71700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.753+0000 7f71767fc640 1 -- 172.21.15.27:0/2306287099 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f717003b040 con 0x7f71800d1e00 2023-12-19T18:16:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.753+0000 7f71767fc640 1 -- 172.21.15.27:0/2306287099 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7170037c60 con 0x7f71800d1e00 2023-12-19T18:16:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.753+0000 7f71767fc640 1 -- 172.21.15.27:0/2306287099 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7170042480 con 0x7f71800d1e00 2023-12-19T18:16:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.753+0000 7f7187c1e640 1 -- 172.21.15.27:0/2306287099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7180152ca0 con 0x7f71800d1e00 2023-12-19T18:16:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.753+0000 7f7187c1e640 1 -- 172.21.15.27:0/2306287099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7180156040 con 0x7f71800d1e00 2023-12-19T18:16:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.754+0000 7f7187c1e640 1 -- 172.21.15.27:0/2306287099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71800d1580 con 0x7f71800d1e00 2023-12-19T18:16:10.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.755+0000 7f71767fc640 1 -- 172.21.15.27:0/2306287099 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f71700425e0 con 0x7f71800d1e00 2023-12-19T18:16:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.756+0000 7f71767fc640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f716406a430 0x7f716406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.756+0000 7f71767fc640 1 -- 172.21.15.27:0/2306287099 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f71700ab9a0 con 0x7f71800d1e00 2023-12-19T18:16:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.757+0000 7f71859ba640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f716406a430 0x7f716406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:10.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.758+0000 7f71859ba640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f716406a430 0x7f716406c8f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f717c0012e0 tx=0x7f717c001630 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.760+0000 7f71767fc640 1 -- 172.21.15.27:0/2306287099 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7170076f60 con 0x7f71800d1e00 2023-12-19T18:16:10.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.993+0000 7f7187c1e640 1 -- 172.21.15.27:0/2306287099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7180068e90 con 0x7f71800d1e00 2023-12-19T18:16:10.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.994+0000 7f71767fc640 1 -- 172.21.15.27:0/2306287099 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7f717007a580 con 0x7f71800d1e00 2023-12-19T18:16:10.994 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:10.994 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3679393226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3679393226},{"type":"v1","addr":"172.21.15.27:6835","nonce":3679393226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/809712436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":809712436},{"type":"v1","addr":"172.21.15.192:6835","nonce":809712436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3400689212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3400689212},{"type":"v1","addr":"172.21.15.27:6833","nonce":3400689212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:54.035577+0000","modified":"2023-12-19T18:14:57.547613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/1878426383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1878426383},{"type":"v1","addr":"172.21.15.192:6837","nonce":1878426383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T18:16:10.994 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2023-12-19T18:16:10.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.996+0000 7f7187c1e640 1 -- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f716406a430 msgr2=0x7f716406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.996+0000 7f7187c1e640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f716406a430 0x7f716406c8f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f717c0012e0 tx=0x7f717c001630 comp rx=0 tx=0).stop 2023-12-19T18:16:10.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.996+0000 7f7187c1e640 1 -- 172.21.15.27:0/2306287099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71800d1e00 msgr2=0x7f71801531b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:10.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.996+0000 7f7187c1e640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71800d1e00 0x7f71801531b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7170000c00 tx=0x7f71700046c0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.996+0000 7f7187c1e640 1 -- 172.21.15.27:0/2306287099 shutdown_connections 2023-12-19T18:16:10.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.997+0000 7f7187c1e640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f716406a430 0x7f716406c8f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.997+0000 7f7187c1e640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f718015d240 0x7f71801f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.997+0000 7f7187c1e640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71800d1e00 0x7f71801531b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.997+0000 7f7187c1e640 1 --2- 172.21.15.27:0/2306287099 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71800cdf90 0x7f7180155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:10.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.997+0000 7f7187c1e640 1 -- 172.21.15.27:0/2306287099 >> 172.21.15.27:0/2306287099 conn(0x7f71800d31e0 msgr2=0x7f71800d0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:10.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.997+0000 7f7187c1e640 1 -- 172.21.15.27:0/2306287099 shutdown_connections 2023-12-19T18:16:10.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:10.997+0000 7f7187c1e640 1 -- 172.21.15.27:0/2306287099 wait complete. 2023-12-19T18:16:11.017 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:11.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.249+0000 7f6989327640 1 Processor -- start 2023-12-19T18:16:11.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.249+0000 7f6989327640 1 -- start start 2023-12-19T18:16:11.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.250+0000 7f6989327640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69841604d0 0x7f69840b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.250+0000 7f6989327640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840d2350 0x7f69840b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.250+0000 7f6989327640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69840b5e90 con 0x7f69840d0300 2023-12-19T18:16:11.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.250+0000 7f6989327640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69840d2b30 con 0x7f69841604d0 2023-12-19T18:16:11.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.250+0000 7f6989327640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840d2850 con 0x7f69840d2350 2023-12-19T18:16:11.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.251+0000 7f6983fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f69841604d0 0x7f69840b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.251+0000 7f6988b26640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f69840d0300 0x7f69840d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45174/0 (socket says 172.21.15.27:45174) 2023-12-19T18:16:11.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.251+0000 7f6988b26640 1 -- 172.21.15.27:0/1627551330 learned_addr learned my addr 172.21.15.27:0/1627551330 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:11.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.251+0000 7f69837fe640 1 --2- 172.21.15.27:0/1627551330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840d2350 0x7f69840b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.251+0000 7f6982ffd640 1 -- 172.21.15.27:0/1627551330 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3886520735 0 0) 0x7f69840b5e90 con 0x7f69840d0300 2023-12-19T18:16:11.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.252+0000 7f6983fff640 1 -- 172.21.15.27:0/1627551330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840d2350 msgr2=0x7f69840b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:11.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.252+0000 7f6983fff640 1 --2- 172.21.15.27:0/1627551330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840d2350 0x7f69840b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.252+0000 7f6983fff640 1 -- 172.21.15.27:0/1627551330 >> v1:172.21.15.27:6789/0 conn(0x7f69840d0300 legacy=0x7f69840d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.252+0000 7f6983fff640 1 -- 172.21.15.27:0/1627551330 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840cded0 con 0x7f69841604d0 2023-12-19T18:16:11.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.252+0000 7f69837fe640 1 --2- 172.21.15.27:0/1627551330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840d2350 0x7f69840b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:11.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.253+0000 7f6983fff640 1 --2- 172.21.15.27:0/1627551330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69841604d0 0x7f69840b47e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f69740097a0 tx=0x7f697402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6481e1cd84191d3e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:11.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.254+0000 7f6982ffd640 1 -- 172.21.15.27:0/1627551330 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697402d040 con 0x7f69841604d0 2023-12-19T18:16:11.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.254+0000 7f6982ffd640 1 -- 172.21.15.27:0/1627551330 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6974031810 con 0x7f69841604d0 2023-12-19T18:16:11.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.254+0000 7f6982ffd640 1 -- 172.21.15.27:0/1627551330 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974031ba0 con 0x7f69841604d0 2023-12-19T18:16:11.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.254+0000 7f6989327640 1 -- 172.21.15.27:0/1627551330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69841604d0 msgr2=0x7f69840b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.254+0000 7f6989327640 1 --2- 172.21.15.27:0/1627551330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69841604d0 0x7f69840b47e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f69740097a0 tx=0x7f697402b8b0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.255+0000 7f6989327640 1 -- 172.21.15.27:0/1627551330 shutdown_connections 2023-12-19T18:16:11.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.255+0000 7f6989327640 1 --2- 172.21.15.27:0/1627551330 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840d2350 0x7f69840b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.255+0000 7f6989327640 1 --2- 172.21.15.27:0/1627551330 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69841604d0 0x7f69840b47e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.255+0000 7f6989327640 1 -- 172.21.15.27:0/1627551330 >> 172.21.15.27:0/1627551330 conn(0x7f69840cf7e0 msgr2=0x7f69840cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:11.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.255+0000 7f6989327640 1 -- 172.21.15.27:0/1627551330 shutdown_connections 2023-12-19T18:16:11.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.255+0000 7f6989327640 1 -- 172.21.15.27:0/1627551330 wait complete. 2023-12-19T18:16:11.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.257+0000 7f6989327640 1 Processor -- start 2023-12-19T18:16:11.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.257+0000 7f6989327640 1 -- start start 2023-12-19T18:16:11.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.258+0000 7f6989327640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69840d0300 0x7f69840c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.258+0000 7f6989327640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840d2350 0x7f69840b7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.258+0000 7f6989327640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69841604d0 0x7f69840b8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.258+0000 7f6989327640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840c7ee0 con 0x7f69841604d0 2023-12-19T18:16:11.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.258+0000 7f6989327640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840cb520 con 0x7f69840d0300 2023-12-19T18:16:11.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.258+0000 7f6989327640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840cb800 con 0x7f69840d2350 2023-12-19T18:16:11.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.258+0000 7f6988b26640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69841604d0 0x7f69840b8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.258+0000 7f6988b26640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69841604d0 0x7f69840b8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45716/0 (socket says 172.21.15.27:45716) 2023-12-19T18:16:11.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.258+0000 7f6988b26640 1 -- 172.21.15.27:0/3253717631 learned_addr learned my addr 172.21.15.27:0/3253717631 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:11.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.258+0000 7f6983fff640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69840d0300 0x7f69840c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.258+0000 7f69837fe640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840d2350 0x7f69840b7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.259+0000 7f6988b26640 1 -- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840d2350 msgr2=0x7f69840b7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.259+0000 7f6988b26640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840d2350 0x7f69840b7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.259+0000 7f6988b26640 1 -- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69840d0300 msgr2=0x7f69840c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.259+0000 7f6988b26640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69840d0300 0x7f69840c79a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.259+0000 7f6988b26640 1 -- 172.21.15.27:0/3253717631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6974009400 con 0x7f69841604d0 2023-12-19T18:16:11.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.259+0000 7f6988b26640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69841604d0 0x7f69840b8380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f697c00c780 tx=0x7f697c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:11.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.259+0000 7f6980ff9640 1 -- 172.21.15.27:0/3253717631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c00ce90 con 0x7f69841604d0 2023-12-19T18:16:11.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.259+0000 7f6980ff9640 1 -- 172.21.15.27:0/3253717631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697c00a640 con 0x7f69841604d0 2023-12-19T18:16:11.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.259+0000 7f6980ff9640 1 -- 172.21.15.27:0/3253717631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697c00a9d0 con 0x7f69841604d0 2023-12-19T18:16:11.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.259+0000 7f6989327640 1 -- 172.21.15.27:0/3253717631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69840cbae0 con 0x7f69841604d0 2023-12-19T18:16:11.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.260+0000 7f6989327640 1 -- 172.21.15.27:0/3253717631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69840cc0a0 con 0x7f69841604d0 2023-12-19T18:16:11.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.261+0000 7f6989327640 1 -- 172.21.15.27:0/3253717631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6984066f80 con 0x7f69841604d0 2023-12-19T18:16:11.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.262+0000 7f6980ff9640 1 -- 172.21.15.27:0/3253717631 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f697c00ab30 con 0x7f69841604d0 2023-12-19T18:16:11.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.263+0000 7f6980ff9640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f697006a430 0x7f697006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.263+0000 7f6980ff9640 1 -- 172.21.15.27:0/3253717631 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f697c087ec0 con 0x7f69841604d0 2023-12-19T18:16:11.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.263+0000 7f6983fff640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f697006a430 0x7f697006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.265+0000 7f6983fff640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f697006a430 0x7f697006c8f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6974036000 tx=0x7f697403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:11.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.267+0000 7f6980ff9640 1 -- 172.21.15.27:0/3253717631 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f697c053480 con 0x7f69841604d0 2023-12-19T18:16:11.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.461+0000 7f6989327640 1 -- 172.21.15.27:0/3253717631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f69840cc2c0 con 0x7f69841604d0 2023-12-19T18:16:11.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.463+0000 7f6980ff9640 1 -- 172.21.15.27:0/3253717631 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16090 (secure 0 0 0) 0x7f697c018020 con 0x7f69841604d0 2023-12-19T18:16:11.463 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:11.463 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:15:43.580746+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:11.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.465+0000 7f6989327640 1 -- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f697006a430 msgr2=0x7f697006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.465+0000 7f6989327640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f697006a430 0x7f697006c8f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6974036000 tx=0x7f697403c040 comp rx=0 tx=0).stop 2023-12-19T18:16:11.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.465+0000 7f6989327640 1 -- 172.21.15.27:0/3253717631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69841604d0 msgr2=0x7f69840b8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.465+0000 7f6989327640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69841604d0 0x7f69840b8380 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f697c00c780 tx=0x7f697c00cc40 comp rx=0 tx=0).stop 2023-12-19T18:16:11.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.466+0000 7f6989327640 1 -- 172.21.15.27:0/3253717631 shutdown_connections 2023-12-19T18:16:11.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.466+0000 7f6989327640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f697006a430 0x7f697006c8f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.466+0000 7f6989327640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69841604d0 0x7f69840b8380 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.466+0000 7f6989327640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840d2350 0x7f69840b7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.466+0000 7f6989327640 1 --2- 172.21.15.27:0/3253717631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f69840d0300 0x7f69840c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.466+0000 7f6989327640 1 -- 172.21.15.27:0/3253717631 >> 172.21.15.27:0/3253717631 conn(0x7f69840cf7e0 msgr2=0x7f69840d1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:11.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.466+0000 7f6989327640 1 -- 172.21.15.27:0/3253717631 shutdown_connections 2023-12-19T18:16:11.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.466+0000 7f6989327640 1 -- 172.21.15.27:0/3253717631 wait complete. 2023-12-19T18:16:11.486 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:16:11.487 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:11.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.708+0000 7fa108bd6640 1 Processor -- start 2023-12-19T18:16:11.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.708+0000 7fa108bd6640 1 -- start start 2023-12-19T18:16:11.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.710+0000 7fa108bd6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa1041553a0 0x7fa1040c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.710+0000 7fa108bd6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1040b0360 con 0x7fa10415ee30 2023-12-19T18:16:11.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.710+0000 7fa108bd6640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1040cb350 con 0x7fa1040afeb0 2023-12-19T18:16:11.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.710+0000 7fa108bd6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1040cbcf0 con 0x7fa1041553a0 2023-12-19T18:16:11.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.711+0000 7fa103fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa10415ee30 0x7fa1040c39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45182/0 (socket says 172.21.15.27:45182) 2023-12-19T18:16:11.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.711+0000 7fa103fff640 1 -- 172.21.15.27:0/4278811917 learned_addr learned my addr 172.21.15.27:0/4278811917 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:11.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.711+0000 7fa1037fe640 1 --2- 172.21.15.27:0/4278811917 >> v2:172.21.15.27:3300/0 conn(0x7fa1041553a0 0x7fa1040c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.711+0000 7fa1037fe640 1 -- 172.21.15.27:0/4278811917 >> v1:172.21.15.192:6789/0 conn(0x7fa1040afeb0 legacy=0x7fa10415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:11.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.711+0000 7fa1037fe640 1 -- 172.21.15.27:0/4278811917 >> v1:172.21.15.27:6789/0 conn(0x7fa10415ee30 legacy=0x7fa1040c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.711+0000 7fa1037fe640 1 -- 172.21.15.27:0/4278811917 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1040cb630 con 0x7fa1041553a0 2023-12-19T18:16:11.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.711+0000 7fa102ffd640 1 -- 172.21.15.27:0/4278811917 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1369139930 0 0) 0x7fa1040b0360 con 0x7fa10415ee30 2023-12-19T18:16:11.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.712+0000 7fa1037fe640 1 --2- 172.21.15.27:0/4278811917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1041553a0 0x7fa1040c50a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40097a0 tx=0x7fa0f402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=759e8e6222b30f38 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:11.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.713+0000 7fa102ffd640 1 -- 172.21.15.27:0/4278811917 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f402e070 con 0x7fa1041553a0 2023-12-19T18:16:11.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.713+0000 7fa102ffd640 1 -- 172.21.15.27:0/4278811917 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f40315a0 con 0x7fa1041553a0 2023-12-19T18:16:11.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.713+0000 7fa102ffd640 1 -- 172.21.15.27:0/4278811917 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4031930 con 0x7fa1041553a0 2023-12-19T18:16:11.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.713+0000 7fa108bd6640 1 -- 172.21.15.27:0/4278811917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1041553a0 msgr2=0x7fa1040c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.713+0000 7fa108bd6640 1 --2- 172.21.15.27:0/4278811917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1041553a0 0x7fa1040c50a0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa0f40097a0 tx=0x7fa0f402b840 comp rx=0 tx=0).stop 2023-12-19T18:16:11.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.713+0000 7fa108bd6640 1 -- 172.21.15.27:0/4278811917 shutdown_connections 2023-12-19T18:16:11.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.713+0000 7fa108bd6640 1 --2- 172.21.15.27:0/4278811917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1041553a0 0x7fa1040c50a0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.713+0000 7fa108bd6640 1 -- 172.21.15.27:0/4278811917 >> 172.21.15.27:0/4278811917 conn(0x7fa104069c70 msgr2=0x7fa10406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:11.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.713+0000 7fa108bd6640 1 -- 172.21.15.27:0/4278811917 shutdown_connections 2023-12-19T18:16:11.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.713+0000 7fa108bd6640 1 -- 172.21.15.27:0/4278811917 wait complete. 2023-12-19T18:16:11.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.715+0000 7fa108bd6640 1 Processor -- start 2023-12-19T18:16:11.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.715+0000 7fa108bd6640 1 -- start start 2023-12-19T18:16:11.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.716+0000 7fa108bd6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1040afeb0 0x7fa104151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.716+0000 7fa108bd6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1041553a0 0x7fa10415dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.716+0000 7fa108bd6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa10415ee30 0x7fa1040ad030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.716+0000 7fa108bd6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa10415e180 con 0x7fa1040afeb0 2023-12-19T18:16:11.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.717+0000 7fa108bd6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1041521e0 con 0x7fa10415ee30 2023-12-19T18:16:11.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.717+0000 7fa108bd6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1040ac8c0 con 0x7fa1041553a0 2023-12-19T18:16:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.717+0000 7fa1037fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1040afeb0 0x7fa104151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.717+0000 7fa1037fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1040afeb0 0x7fa104151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45734/0 (socket says 172.21.15.27:45734) 2023-12-19T18:16:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.717+0000 7fa1037fe640 1 -- 172.21.15.27:0/1911081791 learned_addr learned my addr 172.21.15.27:0/1911081791 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.717+0000 7fa0fbfff640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1041553a0 0x7fa10415dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.717+0000 7fa103fff640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa10415ee30 0x7fa1040ad030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.717+0000 7fa1037fe640 1 -- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1041553a0 msgr2=0x7fa10415dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.717+0000 7fa1037fe640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1041553a0 0x7fa10415dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.717+0000 7fa1037fe640 1 -- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa10415ee30 msgr2=0x7fa1040ad030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.717+0000 7fa1037fe640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa10415ee30 0x7fa1040ad030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.717+0000 7fa1037fe640 1 -- 172.21.15.27:0/1911081791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f4009400 con 0x7fa1040afeb0 2023-12-19T18:16:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.717+0000 7fa0fbfff640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1041553a0 0x7fa10415dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:11.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.718+0000 7fa1037fe640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1040afeb0 0x7fa104151340 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4034040 tx=0x7fa0f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.718+0000 7fa100ff9640 1 -- 172.21.15.27:0/1911081791 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4004230 con 0x7fa1040afeb0 2023-12-19T18:16:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.718+0000 7fa100ff9640 1 -- 172.21.15.27:0/1911081791 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f40043b0 con 0x7fa1040afeb0 2023-12-19T18:16:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.718+0000 7fa108bd6640 1 -- 172.21.15.27:0/1911081791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1040acba0 con 0x7fa1040afeb0 2023-12-19T18:16:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.718+0000 7fa100ff9640 1 -- 172.21.15.27:0/1911081791 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f4004520 con 0x7fa1040afeb0 2023-12-19T18:16:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.718+0000 7fa108bd6640 1 -- 172.21.15.27:0/1911081791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa10415cf90 con 0x7fa1040afeb0 2023-12-19T18:16:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.720+0000 7fa108bd6640 1 -- 172.21.15.27:0/1911081791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1040c4060 con 0x7fa1040afeb0 2023-12-19T18:16:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.721+0000 7fa100ff9640 1 -- 172.21.15.27:0/1911081791 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa0f4032880 con 0x7fa1040afeb0 2023-12-19T18:16:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.722+0000 7fa100ff9640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0e406a430 0x7fa0e406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.722+0000 7fa100ff9640 1 -- 172.21.15.27:0/1911081791 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa0f40aaa60 con 0x7fa1040afeb0 2023-12-19T18:16:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.723+0000 7fa0fbfff640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0e406a430 0x7fa0e406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:11.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.724+0000 7fa0fbfff640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0e406a430 0x7fa0e406c8f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa104159080 tx=0x7fa0ec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:11.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.727+0000 7fa100ff9640 1 -- 172.21.15.27:0/1911081791 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa0f40760c0 con 0x7fa1040afeb0 2023-12-19T18:16:11.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.962+0000 7fa108bd6640 1 -- 172.21.15.27:0/1911081791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1040c4060 con 0x7fa1040afeb0 2023-12-19T18:16:11.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.963+0000 7fa100ff9640 1 -- 172.21.15.27:0/1911081791 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fa0f403ae60 con 0x7fa1040afeb0 2023-12-19T18:16:11.963 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:11.964 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3679393226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3679393226},{"type":"v1","addr":"172.21.15.27:6835","nonce":3679393226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/809712436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":809712436},{"type":"v1","addr":"172.21.15.192:6835","nonce":809712436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3400689212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3400689212},{"type":"v1","addr":"172.21.15.27:6833","nonce":3400689212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:54.035577+0000","modified":"2023-12-19T18:14:57.547613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/1878426383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1878426383},{"type":"v1","addr":"172.21.15.192:6837","nonce":1878426383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T18:16:11.964 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2023-12-19T18:16:11.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.966+0000 7fa108bd6640 1 -- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0e406a430 msgr2=0x7fa0e406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.966+0000 7fa108bd6640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0e406a430 0x7fa0e406c8f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa104159080 tx=0x7fa0ec009040 comp rx=0 tx=0).stop 2023-12-19T18:16:11.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.966+0000 7fa108bd6640 1 -- 172.21.15.27:0/1911081791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1040afeb0 msgr2=0x7fa104151340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:11.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.966+0000 7fa108bd6640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1040afeb0 0x7fa104151340 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4034040 tx=0x7fa0f40046c0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.966+0000 7fa108bd6640 1 -- 172.21.15.27:0/1911081791 shutdown_connections 2023-12-19T18:16:11.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.966+0000 7fa108bd6640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0e406a430 0x7fa0e406c8f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.966+0000 7fa108bd6640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa10415ee30 0x7fa1040ad030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.966+0000 7fa108bd6640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1041553a0 0x7fa10415dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.966+0000 7fa108bd6640 1 --2- 172.21.15.27:0/1911081791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1040afeb0 0x7fa104151340 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:11.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.966+0000 7fa108bd6640 1 -- 172.21.15.27:0/1911081791 >> 172.21.15.27:0/1911081791 conn(0x7fa104069c70 msgr2=0x7fa1040ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:11.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.966+0000 7fa108bd6640 1 -- 172.21.15.27:0/1911081791 shutdown_connections 2023-12-19T18:16:11.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:11.966+0000 7fa108bd6640 1 -- 172.21.15.27:0/1911081791 wait complete. 2023-12-19T18:16:11.984 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.219+0000 7fb6b21e1640 1 Processor -- start 2023-12-19T18:16:12.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.219+0000 7fb6b21e1640 1 -- start start 2023-12-19T18:16:12.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.221+0000 7fb6b21e1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb6ac15ee30 0x7fb6ac0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.221+0000 7fb6b21e1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ac0cb350 con 0x7fb6ac0afeb0 2023-12-19T18:16:12.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.221+0000 7fb6b21e1640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6ac0cb630 con 0x7fb6ac1553a0 2023-12-19T18:16:12.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.221+0000 7fb6b21e1640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6ac0cbcf0 con 0x7fb6ac15ee30 2023-12-19T18:16:12.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.221+0000 7fb6b19e0640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb6ac0afeb0 0x7fb6ac0c39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60074/0 (socket says 172.21.15.27:60074) 2023-12-19T18:16:12.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.221+0000 7fb6b19e0640 1 -- 172.21.15.27:0/3152794057 learned_addr learned my addr 172.21.15.27:0/3152794057 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6b09de640 1 --2- 172.21.15.27:0/3152794057 >> v2:172.21.15.192:3300/0 conn(0x7fb6ac15ee30 0x7fb6ac0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 369432325 0 0) 0x7fb6ac0cb350 con 0x7fb6ac0afeb0 2023-12-19T18:16:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb698003520 con 0x7fb6ac0afeb0 2023-12-19T18:16:12.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 550168375 0 0) 0x7fb698003520 con 0x7fb6ac0afeb0 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 >> v2:172.21.15.192:3300/0 conn(0x7fb6ac15ee30 msgr2=0x7fb6ac0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6a3fff640 1 --2- 172.21.15.27:0/3152794057 >> v2:172.21.15.192:3300/0 conn(0x7fb6ac15ee30 0x7fb6ac0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 >> v1:172.21.15.192:6789/0 conn(0x7fb6ac1553a0 legacy=0x7fb6ac15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac0cb910 con 0x7fb6ac0afeb0 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fb6a40019f0 con 0x7fb6ac0afeb0 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6b09de640 1 --2- 172.21.15.27:0/3152794057 >> v2:172.21.15.192:3300/0 conn(0x7fb6ac15ee30 0x7fb6ac0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 >> v1:172.21.15.27:6789/0 conn(0x7fb6ac0afeb0 legacy=0x7fb6ac0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6a3fff640 1 --2- 172.21.15.27:0/3152794057 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb698003970 0x7fb698003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6a3fff640 1 --2- 172.21.15.27:0/3152794057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6980047c0 0x7fb698004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.222+0000 7fb6a3fff640 1 --2- 172.21.15.27:0/3152794057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb698005180 0x7fb69800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.223+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0cbcf0 con 0x7fb698005180 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.223+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69800c120 con 0x7fb6980047c0 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.223+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69800c2e0 con 0x7fb698003970 2023-12-19T18:16:12.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.223+0000 7fb6b19e0640 1 --2- 172.21.15.27:0/3152794057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb698005180 0x7fb69800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.223+0000 7fb6b09de640 1 --2- 172.21.15.27:0/3152794057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6980047c0 0x7fb698004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.223+0000 7fb6b11df640 1 --2- 172.21.15.27:0/3152794057 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb698003970 0x7fb698003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.223+0000 7fb6b19e0640 1 -- 172.21.15.27:0/3152794057 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb698003970 msgr2=0x7fb698003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:12.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.223+0000 7fb6b19e0640 1 --2- 172.21.15.27:0/3152794057 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb698003970 0x7fb698003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.223+0000 7fb6b19e0640 1 -- 172.21.15.27:0/3152794057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6980047c0 msgr2=0x7fb698004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:12.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.223+0000 7fb6b19e0640 1 --2- 172.21.15.27:0/3152794057 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6980047c0 0x7fb698004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.223+0000 7fb6b19e0640 1 -- 172.21.15.27:0/3152794057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb698003520 con 0x7fb698005180 2023-12-19T18:16:12.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.224+0000 7fb6b19e0640 1 -- 172.21.15.27:0/3152794057 reap_dead start 2023-12-19T18:16:12.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.224+0000 7fb6b19e0640 1 --2- 172.21.15.27:0/3152794057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb698005180 0x7fb69800b980 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400a830 tx=0x7fb6980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c32b061e6e94775a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:12.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.225+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a40072d0 con 0x7fb698005180 2023-12-19T18:16:12.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.225+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a402f070 con 0x7fb698005180 2023-12-19T18:16:12.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.225+0000 7fb6a3fff640 1 -- 172.21.15.27:0/3152794057 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a40076b0 con 0x7fb698005180 2023-12-19T18:16:12.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.225+0000 7fb6b21e1640 1 -- 172.21.15.27:0/3152794057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb698005180 msgr2=0x7fb69800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.225+0000 7fb6b21e1640 1 --2- 172.21.15.27:0/3152794057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb698005180 0x7fb69800b980 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fb6a400a830 tx=0x7fb6980057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.225+0000 7fb6b21e1640 1 -- 172.21.15.27:0/3152794057 shutdown_connections 2023-12-19T18:16:12.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.225+0000 7fb6b21e1640 1 --2- 172.21.15.27:0/3152794057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb698005180 0x7fb69800b980 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.225+0000 7fb6b21e1640 1 -- 172.21.15.27:0/3152794057 >> 172.21.15.27:0/3152794057 conn(0x7fb6ac069c70 msgr2=0x7fb6ac06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:12.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.225+0000 7fb6b21e1640 1 -- 172.21.15.27:0/3152794057 shutdown_connections 2023-12-19T18:16:12.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.226+0000 7fb6b21e1640 1 -- 172.21.15.27:0/3152794057 wait complete. 2023-12-19T18:16:12.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.227+0000 7fb6b21e1640 1 Processor -- start 2023-12-19T18:16:12.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.228+0000 7fb6b21e1640 1 -- start start 2023-12-19T18:16:12.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.229+0000 7fb6b21e1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb698005180 0x7fb6ac1f1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.229+0000 7fb6b21e1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6ac15dfa0 0x7fb6ac0ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.229+0000 7fb6b21e1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6ac0ace00 0x7fb6ac06f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.229+0000 7fb6b21e1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac14cd10 con 0x7fb698005180 2023-12-19T18:16:12.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.229+0000 7fb6b21e1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac1f2340 con 0x7fb6ac15dfa0 2023-12-19T18:16:12.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.229+0000 7fb6b21e1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6ac0ad660 con 0x7fb6ac0ace00 2023-12-19T18:16:12.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.229+0000 7fb6b11df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb698005180 0x7fb6ac1f1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.229+0000 7fb6b11df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb698005180 0x7fb6ac1f1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59024/0 (socket says 172.21.15.27:59024) 2023-12-19T18:16:12.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.229+0000 7fb6b11df640 1 -- 172.21.15.27:0/401165174 learned_addr learned my addr 172.21.15.27:0/401165174 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:12.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.229+0000 7fb6b19e0640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6ac0ace00 0x7fb6ac06f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.229+0000 7fb6b09de640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6ac15dfa0 0x7fb6ac0ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.230+0000 7fb6b11df640 1 -- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6ac0ace00 msgr2=0x7fb6ac06f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.230+0000 7fb6b11df640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6ac0ace00 0x7fb6ac06f320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.230+0000 7fb6b11df640 1 -- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6ac15dfa0 msgr2=0x7fb6ac0ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.230+0000 7fb6b11df640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6ac15dfa0 0x7fb6ac0ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.230+0000 7fb6b11df640 1 -- 172.21.15.27:0/401165174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6ac0ad380 con 0x7fb698005180 2023-12-19T18:16:12.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.230+0000 7fb6b11df640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb698005180 0x7fb6ac1f1e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800e4d0 tx=0x7fb6a800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:12.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.230+0000 7fb6a1ffb640 1 -- 172.21.15.27:0/401165174 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a80052e0 con 0x7fb698005180 2023-12-19T18:16:12.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.230+0000 7fb6a1ffb640 1 -- 172.21.15.27:0/401165174 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6a8005440 con 0x7fb698005180 2023-12-19T18:16:12.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.230+0000 7fb6a1ffb640 1 -- 172.21.15.27:0/401165174 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6a8003810 con 0x7fb698005180 2023-12-19T18:16:12.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.230+0000 7fb6b21e1640 1 -- 172.21.15.27:0/401165174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6ac15d4c0 con 0x7fb698005180 2023-12-19T18:16:12.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.230+0000 7fb6b21e1640 1 -- 172.21.15.27:0/401165174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6ac15da80 con 0x7fb698005180 2023-12-19T18:16:12.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.232+0000 7fb6b21e1640 1 -- 172.21.15.27:0/401165174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6ac15d320 con 0x7fb698005180 2023-12-19T18:16:12.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.232+0000 7fb6a1ffb640 1 -- 172.21.15.27:0/401165174 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb6a8003970 con 0x7fb698005180 2023-12-19T18:16:12.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.233+0000 7fb6a1ffb640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb69006a430 0x7fb69006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.233+0000 7fb6a1ffb640 1 -- 172.21.15.27:0/401165174 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb6a8088720 con 0x7fb698005180 2023-12-19T18:16:12.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.234+0000 7fb6b09de640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb69006a430 0x7fb69006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.235+0000 7fb6b09de640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb69006a430 0x7fb69006c8f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac070300 tx=0x7fb69c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:12.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.238+0000 7fb6a1ffb640 1 -- 172.21.15.27:0/401165174 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb6ac15d320 con 0x7fb698005180 2023-12-19T18:16:12.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.472+0000 7fb6b21e1640 1 -- 172.21.15.27:0/401165174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6ac15d320 con 0x7fb698005180 2023-12-19T18:16:12.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.474+0000 7fb6a1ffb640 1 -- 172.21.15.27:0/401165174 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4706 (secure 0 0 0) 0x7fb6ac15d320 con 0x7fb698005180 2023-12-19T18:16:12.474 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:12.475 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3679393226","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3679393226},{"type":"v1","addr":"172.21.15.27:6835","nonce":3679393226}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4264,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/809712436","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":809712436},{"type":"v1","addr":"172.21.15.192:6835","nonce":809712436}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3400689212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3400689212},{"type":"v1","addr":"172.21.15.27:6833","nonce":3400689212}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:14:54.035577+0000","modified":"2023-12-19T18:14:57.547613+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4267},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4267":{"gid":4267,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.192:6837/1878426383","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1878426383},{"type":"v1","addr":"172.21.15.192:6837","nonce":1878426383}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-19T18:16:12.475 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 8 2023-12-19T18:16:12.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.476+0000 7fb6b21e1640 1 -- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb69006a430 msgr2=0x7fb69006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.476+0000 7fb6b21e1640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb69006a430 0x7fb69006c8f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac070300 tx=0x7fb69c008040 comp rx=0 tx=0).stop 2023-12-19T18:16:12.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.476+0000 7fb6b21e1640 1 -- 172.21.15.27:0/401165174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb698005180 msgr2=0x7fb6ac1f1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.476+0000 7fb6b21e1640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb698005180 0x7fb6ac1f1e00 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb6a800e4d0 tx=0x7fb6a800e990 comp rx=0 tx=0).stop 2023-12-19T18:16:12.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.476+0000 7fb6b21e1640 1 -- 172.21.15.27:0/401165174 shutdown_connections 2023-12-19T18:16:12.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.476+0000 7fb6b21e1640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb69006a430 0x7fb69006c8f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.476+0000 7fb6b21e1640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6ac0ace00 0x7fb6ac06f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.476+0000 7fb6b21e1640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6ac15dfa0 0x7fb6ac0ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.476+0000 7fb6b21e1640 1 --2- 172.21.15.27:0/401165174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb698005180 0x7fb6ac1f1e00 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.476+0000 7fb6b21e1640 1 -- 172.21.15.27:0/401165174 >> 172.21.15.27:0/401165174 conn(0x7fb6ac069c70 msgr2=0x7fb6ac0c92c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:12.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.477+0000 7fb6b21e1640 1 -- 172.21.15.27:0/401165174 shutdown_connections 2023-12-19T18:16:12.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.477+0000 7fb6b21e1640 1 -- 172.21.15.27:0/401165174 wait complete. 2023-12-19T18:16:12.497 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:12.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.735+0000 7f1485dab640 1 Processor -- start 2023-12-19T18:16:12.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.735+0000 7f1485dab640 1 -- start start 2023-12-19T18:16:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.737+0000 7f1485dab640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14800cbce0 0x7f14800ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.737+0000 7f1485dab640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14800adc20 0x7f148006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.737+0000 7f1485dab640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14800cb850 con 0x7f148015af70 2023-12-19T18:16:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.737+0000 7f1485dab640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f148015b350 con 0x7f14800cbce0 2023-12-19T18:16:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.737+0000 7f1485dab640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14800cac40 con 0x7f14800adc20 2023-12-19T18:16:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.737+0000 7f1477fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14800cbce0 0x7f14800ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.737+0000 7f1477fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14800cbce0 0x7f14800ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59038/0 (socket says 172.21.15.27:59038) 2023-12-19T18:16:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.737+0000 7f1477fff640 1 -- 172.21.15.27:0/521471742 learned_addr learned my addr 172.21.15.27:0/521471742 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.738+0000 7f147ffff640 1 --2- 172.21.15.27:0/521471742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14800adc20 0x7f148006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.738+0000 7f1477fff640 1 -- 172.21.15.27:0/521471742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14800adc20 msgr2=0x7f148006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.738+0000 7f1477fff640 1 --2- 172.21.15.27:0/521471742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14800adc20 0x7f148006b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.738+0000 7f1477fff640 1 -- 172.21.15.27:0/521471742 >> v1:172.21.15.192:6789/0 conn(0x7f148015af70 legacy=0x7f14800ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:12.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.738+0000 7f1477fff640 1 -- 172.21.15.27:0/521471742 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14800caf20 con 0x7f14800cbce0 2023-12-19T18:16:12.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.739+0000 7f1477fff640 1 --2- 172.21.15.27:0/521471742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14800cbce0 0x7f14800ca540 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1468009870 tx=0x7f146802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb065bc250ce9fc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:12.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.739+0000 7f147effd640 1 -- 172.21.15.27:0/521471742 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468031640 con 0x7f14800cbce0 2023-12-19T18:16:12.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.739+0000 7f147effd640 1 -- 172.21.15.27:0/521471742 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14680317a0 con 0x7f14800cbce0 2023-12-19T18:16:12.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.739+0000 7f147effd640 1 -- 172.21.15.27:0/521471742 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1468031b30 con 0x7f14800cbce0 2023-12-19T18:16:12.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.740+0000 7f1485dab640 1 -- 172.21.15.27:0/521471742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14800cbce0 msgr2=0x7f14800ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.740+0000 7f1485dab640 1 --2- 172.21.15.27:0/521471742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14800cbce0 0x7f14800ca540 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f1468009870 tx=0x7f146802b8f0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.740+0000 7f1485dab640 1 -- 172.21.15.27:0/521471742 shutdown_connections 2023-12-19T18:16:12.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.740+0000 7f1485dab640 1 --2- 172.21.15.27:0/521471742 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14800adc20 0x7f148006b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.740+0000 7f1485dab640 1 --2- 172.21.15.27:0/521471742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14800cbce0 0x7f14800ca540 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.740+0000 7f1485dab640 1 -- 172.21.15.27:0/521471742 >> 172.21.15.27:0/521471742 conn(0x7f148006a330 msgr2=0x7f14800b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:12.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.740+0000 7f1485dab640 1 -- 172.21.15.27:0/521471742 shutdown_connections 2023-12-19T18:16:12.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.740+0000 7f1485dab640 1 -- 172.21.15.27:0/521471742 wait complete. 2023-12-19T18:16:12.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.742+0000 7f1485dab640 1 Processor -- start 2023-12-19T18:16:12.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.742+0000 7f1485dab640 1 -- start start 2023-12-19T18:16:12.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.743+0000 7f1485dab640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14800adc20 0x7f14801590f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.743+0000 7f1485dab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14800cbce0 0x7f14800aec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.743+0000 7f1485dab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148015af70 0x7f14800ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.743+0000 7f1485dab640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148015d500 con 0x7f148015af70 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.743+0000 7f1485dab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f148015c610 con 0x7f14800cbce0 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.743+0000 7f1485dab640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f148015c8f0 con 0x7f14800adc20 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.743+0000 7f147ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148015af70 0x7f14800ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.744+0000 7f147ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148015af70 0x7f14800ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59040/0 (socket says 172.21.15.27:59040) 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.744+0000 7f147ffff640 1 -- 172.21.15.27:0/2926374422 learned_addr learned my addr 172.21.15.27:0/2926374422 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.744+0000 7f1477fff640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14800cbce0 0x7f14800aec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.744+0000 7f147f7fe640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14800adc20 0x7f14801590f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.744+0000 7f147ffff640 1 -- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14800adc20 msgr2=0x7f14801590f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.744+0000 7f147ffff640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14800adc20 0x7f14801590f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.744+0000 7f147ffff640 1 -- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14800cbce0 msgr2=0x7f14800aec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.744+0000 7f147ffff640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14800cbce0 0x7f14800aec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.744+0000 7f147ffff640 1 -- 172.21.15.27:0/2926374422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14680094d0 con 0x7f148015af70 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.744+0000 7f1477fff640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14800cbce0 0x7f14800aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.744+0000 7f147f7fe640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14800adc20 0x7f14801590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:12.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.744+0000 7f147ffff640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148015af70 0x7f14800ac8d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1478007e40 tx=0x7f147800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:12.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.745+0000 7f147cff9640 1 -- 172.21.15.27:0/2926374422 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478014620 con 0x7f148015af70 2023-12-19T18:16:12.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.745+0000 7f147cff9640 1 -- 172.21.15.27:0/2926374422 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f147800ee90 con 0x7f148015af70 2023-12-19T18:16:12.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.745+0000 7f147cff9640 1 -- 172.21.15.27:0/2926374422 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14780149e0 con 0x7f148015af70 2023-12-19T18:16:12.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.745+0000 7f1485dab640 1 -- 172.21.15.27:0/2926374422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f148015cbd0 con 0x7f148015af70 2023-12-19T18:16:12.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.745+0000 7f1485dab640 1 -- 172.21.15.27:0/2926374422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f148015d190 con 0x7f148015af70 2023-12-19T18:16:12.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.746+0000 7f1485dab640 1 -- 172.21.15.27:0/2926374422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f148015d670 con 0x7f148015af70 2023-12-19T18:16:12.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.747+0000 7f147cff9640 1 -- 172.21.15.27:0/2926374422 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1478014b40 con 0x7f148015af70 2023-12-19T18:16:12.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.748+0000 7f147cff9640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f146006a430 0x7f146006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:12.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.748+0000 7f147cff9640 1 -- 172.21.15.27:0/2926374422 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f14780892e0 con 0x7f148015af70 2023-12-19T18:16:12.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.748+0000 7f147f7fe640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f146006a430 0x7f146006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:12.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.750+0000 7f147f7fe640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f146006a430 0x7f146006c8f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f147000a830 tx=0x7f1470008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:12.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.752+0000 7f147cff9640 1 -- 172.21.15.27:0/2926374422 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f14780548b0 con 0x7f148015af70 2023-12-19T18:16:12.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.948+0000 7f1485dab640 1 -- 172.21.15.27:0/2926374422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1480068e90 con 0x7f148015af70 2023-12-19T18:16:12.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.951+0000 7f147cff9640 1 -- 172.21.15.27:0/2926374422 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16090 (secure 0 0 0) 0x7f1478057ed0 con 0x7f148015af70 2023-12-19T18:16:12.951 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:12.952 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":22,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:15:43.580746+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:14:52.124331+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-19T18:14:53.068734+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.15","mappings":[{"from":0,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:12.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.953+0000 7f1485dab640 1 -- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f146006a430 msgr2=0x7f146006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.953+0000 7f1485dab640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f146006a430 0x7f146006c8f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f147000a830 tx=0x7f1470008040 comp rx=0 tx=0).stop 2023-12-19T18:16:12.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.953+0000 7f1485dab640 1 -- 172.21.15.27:0/2926374422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148015af70 msgr2=0x7f14800ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:12.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.953+0000 7f1485dab640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148015af70 0x7f14800ac8d0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1478007e40 tx=0x7f147800e4b0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.953+0000 7f1485dab640 1 -- 172.21.15.27:0/2926374422 shutdown_connections 2023-12-19T18:16:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.953+0000 7f1485dab640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f146006a430 0x7f146006c8f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.953+0000 7f1485dab640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f148015af70 0x7f14800ac8d0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.953+0000 7f1485dab640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14800cbce0 0x7f14800aec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.953+0000 7f1485dab640 1 --2- 172.21.15.27:0/2926374422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14800adc20 0x7f14801590f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:12.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.953+0000 7f1485dab640 1 -- 172.21.15.27:0/2926374422 >> 172.21.15.27:0/2926374422 conn(0x7f148006a330 msgr2=0x7f14800b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:12.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.954+0000 7f1485dab640 1 -- 172.21.15.27:0/2926374422 shutdown_connections 2023-12-19T18:16:12.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:12.954+0000 7f1485dab640 1 -- 172.21.15.27:0/2926374422 wait complete. 2023-12-19T18:16:12.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:16:13.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.208+0000 7fe15042c640 1 Processor -- start 2023-12-19T18:16:13.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.208+0000 7fe15042c640 1 -- start start 2023-12-19T18:16:13.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.209+0000 7fe15042c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe14815d260 0x7fe14806a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.209+0000 7fe15042c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1480d1a50 con 0x7fe1480d1580 2023-12-19T18:16:13.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.209+0000 7fe15042c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1480d1d30 con 0x7fe1480d2af0 2023-12-19T18:16:13.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.209+0000 7fe15042c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1480cdaf0 con 0x7fe14815d260 2023-12-19T18:16:13.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.210+0000 7fe14e9c9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe1480d1580 0x7fe1480b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60088/0 (socket says 172.21.15.27:60088) 2023-12-19T18:16:13.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.210+0000 7fe14e9c9640 1 -- 172.21.15.27:0/965327033 learned_addr learned my addr 172.21.15.27:0/965327033 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:13.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.210+0000 7fe14e1c8640 1 --2- 172.21.15.27:0/965327033 >> v2:172.21.15.192:3300/0 conn(0x7fe14815d260 0x7fe14806a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.210+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1985184434 0 0) 0x7fe1480d1a50 con 0x7fe1480d1580 2023-12-19T18:16:13.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.210+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe130003520 con 0x7fe1480d1580 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2308615763 0 0) 0x7fe130003520 con 0x7fe1480d1580 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 >> v2:172.21.15.192:3300/0 conn(0x7fe14815d260 msgr2=0x7fe14806a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d1c6640 1 --2- 172.21.15.27:0/965327033 >> v2:172.21.15.192:3300/0 conn(0x7fe14815d260 0x7fe14806a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 >> v1:172.21.15.192:6789/0 conn(0x7fe1480d2af0 legacy=0x7fe1480d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480d2010 con 0x7fe1480d1580 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe140001670 con 0x7fe1480d1580 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 >> v1:172.21.15.27:6789/0 conn(0x7fe1480d1580 legacy=0x7fe1480b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d1c6640 1 --2- 172.21.15.27:0/965327033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe130003970 0x7fe130003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d1c6640 1 --2- 172.21.15.27:0/965327033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1300047c0 0x7fe130004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d1c6640 1 --2- 172.21.15.27:0/965327033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe130005180 0x7fe13000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1480cdaf0 con 0x7fe130003970 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13000c020 con 0x7fe130005180 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe13000c1e0 con 0x7fe1300047c0 2023-12-19T18:16:13.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14e1c8640 1 --2- 172.21.15.27:0/965327033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe130003970 0x7fe130003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14d9c7640 1 --2- 172.21.15.27:0/965327033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1300047c0 0x7fe130004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.211+0000 7fe14e1c8640 1 -- 172.21.15.27:0/965327033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1300047c0 msgr2=0x7fe130004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:13.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.212+0000 7fe14e1c8640 1 --2- 172.21.15.27:0/965327033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1300047c0 0x7fe130004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.212+0000 7fe14e1c8640 1 -- 172.21.15.27:0/965327033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe130005180 msgr2=0x7fe13000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:13.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.212+0000 7fe14e1c8640 1 --2- 172.21.15.27:0/965327033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe130005180 0x7fe13000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.212+0000 7fe14e1c8640 1 -- 172.21.15.27:0/965327033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe130003520 con 0x7fe130003970 2023-12-19T18:16:13.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.212+0000 7fe14e9c9640 1 -- 172.21.15.27:0/965327033 reap_dead start 2023-12-19T18:16:13.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.213+0000 7fe14e1c8640 1 --2- 172.21.15.27:0/965327033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe130003970 0x7fe130003d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe138009fa0 tx=0x7fe1300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe80865b55eb8024 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.213+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13802cce0 con 0x7fe130003970 2023-12-19T18:16:13.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.213+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13800f040 con 0x7fe130003970 2023-12-19T18:16:13.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.213+0000 7fe14d1c6640 1 -- 172.21.15.27:0/965327033 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe138034660 con 0x7fe130003970 2023-12-19T18:16:13.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.214+0000 7fe15042c640 1 -- 172.21.15.27:0/965327033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe130003970 msgr2=0x7fe130003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.214+0000 7fe15042c640 1 --2- 172.21.15.27:0/965327033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe130003970 0x7fe130003d90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe138009fa0 tx=0x7fe1300057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.214+0000 7fe15042c640 1 -- 172.21.15.27:0/965327033 shutdown_connections 2023-12-19T18:16:13.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.214+0000 7fe15042c640 1 --2- 172.21.15.27:0/965327033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe130003970 0x7fe130003d90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.214+0000 7fe15042c640 1 -- 172.21.15.27:0/965327033 >> 172.21.15.27:0/965327033 conn(0x7fe1480d2f70 msgr2=0x7fe1480d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:13.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.214+0000 7fe15042c640 1 -- 172.21.15.27:0/965327033 shutdown_connections 2023-12-19T18:16:13.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.214+0000 7fe15042c640 1 -- 172.21.15.27:0/965327033 wait complete. 2023-12-19T18:16:13.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.216+0000 7fe15042c640 1 Processor -- start 2023-12-19T18:16:13.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.216+0000 7fe15042c640 1 -- start start 2023-12-19T18:16:13.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.217+0000 7fe15042c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe130003970 0x7fe1480ca1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.217+0000 7fe15042c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1481604f0 0x7fe1480c90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.217+0000 7fe15042c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1480cbbe0 0x7fe1480cc070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.217+0000 7fe15042c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe14815e1d0 con 0x7fe130003970 2023-12-19T18:16:13.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.217+0000 7fe15042c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1480c95f0 con 0x7fe1480cbbe0 2023-12-19T18:16:13.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.217+0000 7fe15042c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe14815dba0 con 0x7fe1481604f0 2023-12-19T18:16:13.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.217+0000 7fe14e1c8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe130003970 0x7fe1480ca1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.217+0000 7fe14e1c8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe130003970 0x7fe1480ca1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59058/0 (socket says 172.21.15.27:59058) 2023-12-19T18:16:13.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.217+0000 7fe14e1c8640 1 -- 172.21.15.27:0/2387904305 learned_addr learned my addr 172.21.15.27:0/2387904305 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:13.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.217+0000 7fe14d9c7640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1481604f0 0x7fe1480c90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.218+0000 7fe14e9c9640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1480cbbe0 0x7fe1480cc070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.218+0000 7fe14e1c8640 1 -- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1481604f0 msgr2=0x7fe1480c90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.218+0000 7fe14e1c8640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1481604f0 0x7fe1480c90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.218+0000 7fe14e1c8640 1 -- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1480cbbe0 msgr2=0x7fe1480cc070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.218+0000 7fe14e1c8640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1480cbbe0 0x7fe1480cc070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.218+0000 7fe14e1c8640 1 -- 172.21.15.27:0/2387904305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1480cca60 con 0x7fe130003970 2023-12-19T18:16:13.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.218+0000 7fe14e9c9640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1480cbbe0 0x7fe1480cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:13.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.218+0000 7fe14e1c8640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe130003970 0x7fe1480ca1f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe13802c5c0 tx=0x7fe138009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.218+0000 7fe146ffd640 1 -- 172.21.15.27:0/2387904305 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13803e610 con 0x7fe130003970 2023-12-19T18:16:13.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.218+0000 7fe146ffd640 1 -- 172.21.15.27:0/2387904305 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe13803e770 con 0x7fe130003970 2023-12-19T18:16:13.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.218+0000 7fe146ffd640 1 -- 172.21.15.27:0/2387904305 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe13803eb20 con 0x7fe130003970 2023-12-19T18:16:13.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.218+0000 7fe15042c640 1 -- 172.21.15.27:0/2387904305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1480ccd40 con 0x7fe130003970 2023-12-19T18:16:13.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.219+0000 7fe15042c640 1 -- 172.21.15.27:0/2387904305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1480cc5b0 con 0x7fe130003970 2023-12-19T18:16:13.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.220+0000 7fe15042c640 1 -- 172.21.15.27:0/2387904305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1480ca730 con 0x7fe130003970 2023-12-19T18:16:13.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.221+0000 7fe146ffd640 1 -- 172.21.15.27:0/2387904305 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe13803b050 con 0x7fe130003970 2023-12-19T18:16:13.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.222+0000 7fe146ffd640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe13406a430 0x7fe13406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.222+0000 7fe146ffd640 1 -- 172.21.15.27:0/2387904305 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe1380ab0b0 con 0x7fe130003970 2023-12-19T18:16:13.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.223+0000 7fe14d9c7640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe13406a430 0x7fe13406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.224+0000 7fe14d9c7640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe13406a430 0x7fe13406c8f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe13c007fd0 tx=0x7fe13c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.227+0000 7fe146ffd640 1 -- 172.21.15.27:0/2387904305 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe1380766a0 con 0x7fe130003970 2023-12-19T18:16:13.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.460+0000 7fe15042c640 1 -- 172.21.15.27:0/2387904305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe14806afc0 con 0x7fe130003970 2023-12-19T18:16:13.461 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:16:13.461+0000 7f8f3865c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:16:13.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.475+0000 7fe146ffd640 1 -- 172.21.15.27:0/2387904305 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe13804d090 con 0x7fe130003970 2023-12-19T18:16:13.475 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:16:13.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.478+0000 7fe15042c640 1 -- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe13406a430 msgr2=0x7fe13406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.478+0000 7fe15042c640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe13406a430 0x7fe13406c8f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fe13c007fd0 tx=0x7fe13c009040 comp rx=0 tx=0).stop 2023-12-19T18:16:13.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.478+0000 7fe15042c640 1 -- 172.21.15.27:0/2387904305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe130003970 msgr2=0x7fe1480ca1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.478+0000 7fe15042c640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe130003970 0x7fe1480ca1f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fe13802c5c0 tx=0x7fe138009750 comp rx=0 tx=0).stop 2023-12-19T18:16:13.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.478+0000 7fe15042c640 1 -- 172.21.15.27:0/2387904305 shutdown_connections 2023-12-19T18:16:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.478+0000 7fe15042c640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe13406a430 0x7fe13406c8f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.478+0000 7fe15042c640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1480cbbe0 0x7fe1480cc070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.478+0000 7fe15042c640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1481604f0 0x7fe1480c90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.478+0000 7fe15042c640 1 --2- 172.21.15.27:0/2387904305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe130003970 0x7fe1480ca1f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.478+0000 7fe15042c640 1 -- 172.21.15.27:0/2387904305 >> 172.21.15.27:0/2387904305 conn(0x7fe1480d2f70 msgr2=0x7fe1480b7610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.478+0000 7fe15042c640 1 -- 172.21.15.27:0/2387904305 shutdown_connections 2023-12-19T18:16:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.478+0000 7fe15042c640 1 -- 172.21.15.27:0/2387904305 wait complete. 2023-12-19T18:16:13.499 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:16:13.501 INFO:tasks.ceph.mds.d.smithi192.stderr: -4029> 2023-12-19T18:14:59.979+0000 7fa91ec33640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:16:13.598 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-19T18:16:13.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.719+0000 7f8b64dd4640 1 Processor -- start 2023-12-19T18:16:13.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.720+0000 7f8b64dd4640 1 -- start start 2023-12-19T18:16:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.721+0000 7f8b64dd4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b60158790 0x7f8b600ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.721+0000 7f8b64dd4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b600adc50 0x7f8b6015b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.721+0000 7f8b64dd4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8b6015b810 0x7f8b6015e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.721+0000 7f8b64dd4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b600c99d0 con 0x7f8b60158790 2023-12-19T18:16:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.721+0000 7f8b64dd4640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b600ae100 con 0x7f8b6015b810 2023-12-19T18:16:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.721+0000 7f8b64dd4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b60158b70 con 0x7f8b600adc50 2023-12-19T18:16:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.721+0000 7f8b5f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b60158790 0x7f8b600ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.721+0000 7f8b5f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b60158790 0x7f8b600ad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59066/0 (socket says 172.21.15.27:59066) 2023-12-19T18:16:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.721+0000 7f8b5f7fe640 1 -- 172.21.15.27:0/1017100836 learned_addr learned my addr 172.21.15.27:0/1017100836 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.722+0000 7f8b5f7fe640 1 -- 172.21.15.27:0/1017100836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b600adc50 msgr2=0x7f8b6015b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.722+0000 7f8b5ffff640 1 --2- 172.21.15.27:0/1017100836 >> v2:172.21.15.192:3300/0 conn(0x7f8b6015b810 0x7f8b6015e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.722+0000 7f8b5f7fe640 1 --2- 172.21.15.27:0/1017100836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b600adc50 0x7f8b6015b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.722+0000 7f8b5f7fe640 1 -- 172.21.15.27:0/1017100836 >> v2:172.21.15.192:3300/0 conn(0x7f8b6015b810 msgr2=0x7f8b6015e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.722+0000 7f8b5f7fe640 1 --2- 172.21.15.27:0/1017100836 >> v2:172.21.15.192:3300/0 conn(0x7f8b6015b810 0x7f8b6015e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.722+0000 7f8b5f7fe640 1 -- 172.21.15.27:0/1017100836 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b6015c030 con 0x7f8b60158790 2023-12-19T18:16:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.723+0000 7f8b5f7fe640 1 --2- 172.21.15.27:0/1017100836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60158790 0x7f8b600ad710 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8b600c4b30 tx=0x7f8b5002b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58bfbcf5451fa119 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.723+0000 7f8b5e7fc640 1 -- 172.21.15.27:0/1017100836 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50031860 con 0x7f8b60158790 2023-12-19T18:16:13.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.724+0000 7f8b5e7fc640 1 -- 172.21.15.27:0/1017100836 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b500319c0 con 0x7f8b60158790 2023-12-19T18:16:13.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.724+0000 7f8b5e7fc640 1 -- 172.21.15.27:0/1017100836 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b50031d50 con 0x7f8b60158790 2023-12-19T18:16:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.724+0000 7f8b64dd4640 1 -- 172.21.15.27:0/1017100836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60158790 msgr2=0x7f8b600ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.724+0000 7f8b64dd4640 1 --2- 172.21.15.27:0/1017100836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60158790 0x7f8b600ad710 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f8b600c4b30 tx=0x7f8b5002b900 comp rx=0 tx=0).stop 2023-12-19T18:16:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.725+0000 7f8b64dd4640 1 -- 172.21.15.27:0/1017100836 shutdown_connections 2023-12-19T18:16:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.725+0000 7f8b64dd4640 1 --2- 172.21.15.27:0/1017100836 >> v2:172.21.15.192:3300/0 conn(0x7f8b6015b810 0x7f8b6015e730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.725+0000 7f8b64dd4640 1 --2- 172.21.15.27:0/1017100836 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b600adc50 0x7f8b6015b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.725+0000 7f8b64dd4640 1 --2- 172.21.15.27:0/1017100836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60158790 0x7f8b600ad710 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.725+0000 7f8b64dd4640 1 -- 172.21.15.27:0/1017100836 >> 172.21.15.27:0/1017100836 conn(0x7f8b600b05b0 msgr2=0x7f8b600b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.725+0000 7f8b64dd4640 1 -- 172.21.15.27:0/1017100836 shutdown_connections 2023-12-19T18:16:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.725+0000 7f8b64dd4640 1 -- 172.21.15.27:0/1017100836 wait complete. 2023-12-19T18:16:13.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.727+0000 7f8b64dd4640 1 Processor -- start 2023-12-19T18:16:13.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.727+0000 7f8b64dd4640 1 -- start start 2023-12-19T18:16:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.728+0000 7f8b64dd4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b600adc50 0x7f8b60155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.728+0000 7f8b64dd4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60158790 0x7f8b600c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.728+0000 7f8b64dd4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b6015b810 0x7f8b601492e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.728+0000 7f8b64dd4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600c5f50 con 0x7f8b60158790 2023-12-19T18:16:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.728+0000 7f8b64dd4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600ca100 con 0x7f8b600adc50 2023-12-19T18:16:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.728+0000 7f8b64dd4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600ca3c0 con 0x7f8b6015b810 2023-12-19T18:16:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.728+0000 7f8b5effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60158790 0x7f8b600c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.728+0000 7f8b5effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60158790 0x7f8b600c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59078/0 (socket says 172.21.15.27:59078) 2023-12-19T18:16:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.728+0000 7f8b5effd640 1 -- 172.21.15.27:0/193638074 learned_addr learned my addr 172.21.15.27:0/193638074 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b5f7fe640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b600adc50 0x7f8b60155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b5ffff640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b6015b810 0x7f8b601492e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b5effd640 1 -- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b6015b810 msgr2=0x7f8b601492e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b5effd640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b6015b810 0x7f8b601492e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b5effd640 1 -- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b600adc50 msgr2=0x7f8b60155130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:13.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b5effd640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b600adc50 0x7f8b60155130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:13.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b5effd640 1 -- 172.21.15.27:0/193638074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b500094d0 con 0x7f8b60158790 2023-12-19T18:16:13.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b5ffff640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b6015b810 0x7f8b601492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:13.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b5f7fe640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b600adc50 0x7f8b60155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:13.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b5effd640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60158790 0x7f8b600c5a10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400b4a0 tx=0x7f8b5400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b47fff640 1 -- 172.21.15.27:0/193638074 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54010610 con 0x7f8b60158790 2023-12-19T18:16:13.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b47fff640 1 -- 172.21.15.27:0/193638074 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b54010770 con 0x7f8b60158790 2023-12-19T18:16:13.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b47fff640 1 -- 172.21.15.27:0/193638074 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54010b20 con 0x7f8b60158790 2023-12-19T18:16:13.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.729+0000 7f8b64dd4640 1 -- 172.21.15.27:0/193638074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b600ca6a0 con 0x7f8b60158790 2023-12-19T18:16:13.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.730+0000 7f8b64dd4640 1 -- 172.21.15.27:0/193638074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b600cac60 con 0x7f8b60158790 2023-12-19T18:16:13.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.731+0000 7f8b64dd4640 1 -- 172.21.15.27:0/193638074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b6004e840 con 0x7f8b60158790 2023-12-19T18:16:13.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.732+0000 7f8b47fff640 1 -- 172.21.15.27:0/193638074 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8b54010c80 con 0x7f8b60158790 2023-12-19T18:16:13.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.733+0000 7f8b47fff640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b4c06a430 0x7f8b4c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:13.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.733+0000 7f8b47fff640 1 -- 172.21.15.27:0/193638074 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f8b54088580 con 0x7f8b60158790 2023-12-19T18:16:13.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.734+0000 7f8b5f7fe640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b4c06a430 0x7f8b4c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:13.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.735+0000 7f8b5f7fe640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b4c06a430 0x7f8b4c06c8f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8b50036000 tx=0x7f8b5003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:13.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.738+0000 7f8b47fff640 1 -- 172.21.15.27:0/193638074 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8b54053b00 con 0x7f8b60158790 2023-12-19T18:16:13.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:13.975+0000 7f8b64dd4640 1 -- 172.21.15.27:0/193638074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f8b600c4050 con 0x7f8b60158790 2023-12-19T18:16:14.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.488+0000 7f8b47fff640 1 -- 172.21.15.27:0/193638074 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8b54086e60 con 0x7f8b60158790 2023-12-19T18:16:14.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.491+0000 7f8b64dd4640 1 -- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b4c06a430 msgr2=0x7f8b4c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:14.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.491+0000 7f8b64dd4640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b4c06a430 0x7f8b4c06c8f0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8b50036000 tx=0x7f8b5003c040 comp rx=0 tx=0).stop 2023-12-19T18:16:14.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.491+0000 7f8b64dd4640 1 -- 172.21.15.27:0/193638074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60158790 msgr2=0x7f8b600c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:14.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.491+0000 7f8b64dd4640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60158790 0x7f8b600c5a10 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400b4a0 tx=0x7f8b5400b960 comp rx=0 tx=0).stop 2023-12-19T18:16:14.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.491+0000 7f8b64dd4640 1 -- 172.21.15.27:0/193638074 shutdown_connections 2023-12-19T18:16:14.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.491+0000 7f8b64dd4640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b4c06a430 0x7f8b4c06c8f0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.491+0000 7f8b64dd4640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b6015b810 0x7f8b601492e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.491+0000 7f8b64dd4640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b60158790 0x7f8b600c5a10 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.491+0000 7f8b64dd4640 1 --2- 172.21.15.27:0/193638074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b600adc50 0x7f8b60155130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.491+0000 7f8b64dd4640 1 -- 172.21.15.27:0/193638074 >> 172.21.15.27:0/193638074 conn(0x7f8b600b05b0 msgr2=0x7f8b600c3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:14.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.491+0000 7f8b64dd4640 1 -- 172.21.15.27:0/193638074 shutdown_connections 2023-12-19T18:16:14.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.491+0000 7f8b64dd4640 1 -- 172.21.15.27:0/193638074 wait complete. 2023-12-19T18:16:14.511 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:16:14.675 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:16:14.684 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:16:14.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.748+0000 7fe60f1ec640 1 Processor -- start 2023-12-19T18:16:14.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.748+0000 7fe60f1ec640 1 -- start start 2023-12-19T18:16:14.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.749+0000 7fe60f1ec640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe60815d000 0x7fe6080b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:14.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.749+0000 7fe60f1ec640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6080d1760 con 0x7fe6080cde80 2023-12-19T18:16:14.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.749+0000 7fe60f1ec640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6081610c0 con 0x7fe60815d3e0 2023-12-19T18:16:14.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.749+0000 7fe60f1ec640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6080ce350 con 0x7fe60815d000 2023-12-19T18:16:14.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.750+0000 7fe60d789640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe6080cde80 0x7fe6080d1bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60092/0 (socket says 172.21.15.27:60092) 2023-12-19T18:16:14.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.750+0000 7fe60d789640 1 -- 172.21.15.27:0/770879152 learned_addr learned my addr 172.21.15.27:0/770879152 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:14.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.750+0000 7fe60cf88640 1 --2- 172.21.15.27:0/770879152 >> v2:172.21.15.192:3300/0 conn(0x7fe60815d000 0x7fe6080b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:14.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.750+0000 7fe6077fe640 1 -- 172.21.15.27:0/770879152 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1526988377 0 0) 0x7fe6080d1760 con 0x7fe6080cde80 2023-12-19T18:16:14.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.750+0000 7fe6077fe640 1 -- 172.21.15.27:0/770879152 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5f8003520 con 0x7fe6080cde80 2023-12-19T18:16:14.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.750+0000 7fe60cf88640 1 -- 172.21.15.27:0/770879152 >> v1:172.21.15.192:6789/0 conn(0x7fe60815d3e0 legacy=0x7fe6080cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:14.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.750+0000 7fe60cf88640 1 -- 172.21.15.27:0/770879152 >> v1:172.21.15.27:6789/0 conn(0x7fe6080cde80 legacy=0x7fe6080d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:14.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.751+0000 7fe60cf88640 1 -- 172.21.15.27:0/770879152 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6080ce630 con 0x7fe60815d000 2023-12-19T18:16:14.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.752+0000 7fe60cf88640 1 --2- 172.21.15.27:0/770879152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60815d000 0x7fe6080b4990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0097c0 tx=0x7fe5fc02b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34c003ff818d9fc3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:14.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.752+0000 7fe6077fe640 1 -- 172.21.15.27:0/770879152 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc031890 con 0x7fe60815d000 2023-12-19T18:16:14.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.752+0000 7fe6077fe640 1 -- 172.21.15.27:0/770879152 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc0319f0 con 0x7fe60815d000 2023-12-19T18:16:14.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.752+0000 7fe6077fe640 1 -- 172.21.15.27:0/770879152 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc031d60 con 0x7fe60815d000 2023-12-19T18:16:14.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.753+0000 7fe60f1ec640 1 -- 172.21.15.27:0/770879152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60815d000 msgr2=0x7fe6080b4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:14.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.753+0000 7fe60f1ec640 1 --2- 172.21.15.27:0/770879152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60815d000 0x7fe6080b4990 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0097c0 tx=0x7fe5fc02b930 comp rx=0 tx=0).stop 2023-12-19T18:16:14.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.753+0000 7fe60f1ec640 1 -- 172.21.15.27:0/770879152 shutdown_connections 2023-12-19T18:16:14.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.753+0000 7fe60f1ec640 1 --2- 172.21.15.27:0/770879152 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60815d000 0x7fe6080b4990 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.753+0000 7fe60f1ec640 1 -- 172.21.15.27:0/770879152 >> 172.21.15.27:0/770879152 conn(0x7fe6080cf990 msgr2=0x7fe6080cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:14.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.753+0000 7fe60f1ec640 1 -- 172.21.15.27:0/770879152 shutdown_connections 2023-12-19T18:16:14.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.753+0000 7fe60f1ec640 1 -- 172.21.15.27:0/770879152 wait complete. 2023-12-19T18:16:14.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.755+0000 7fe60f1ec640 1 Processor -- start 2023-12-19T18:16:14.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.755+0000 7fe60f1ec640 1 -- start start 2023-12-19T18:16:14.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.756+0000 7fe60f1ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6080cde80 0x7fe6080caab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:14.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.756+0000 7fe60f1ec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe60815d000 0x7fe6080a9860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:14.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.756+0000 7fe60f1ec640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60815d3e0 0x7fe6080a9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:14.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.756+0000 7fe60f1ec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6080ab390 con 0x7fe6080cde80 2023-12-19T18:16:14.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.756+0000 7fe60f1ec640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe60814a160 con 0x7fe60815d3e0 2023-12-19T18:16:14.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.756+0000 7fe60f1ec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe608149570 con 0x7fe60815d000 2023-12-19T18:16:14.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.756+0000 7fe60cf88640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6080cde80 0x7fe6080caab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:14.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.756+0000 7fe60cf88640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6080cde80 0x7fe6080caab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59092/0 (socket says 172.21.15.27:59092) 2023-12-19T18:16:14.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.756+0000 7fe60cf88640 1 -- 172.21.15.27:0/1045313569 learned_addr learned my addr 172.21.15.27:0/1045313569 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:14.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.756+0000 7fe60d789640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60815d3e0 0x7fe6080a9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:14.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.756+0000 7fe607fff640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe60815d000 0x7fe6080a9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:14.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.757+0000 7fe60cf88640 1 -- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe60815d000 msgr2=0x7fe6080a9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:14.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.757+0000 7fe60cf88640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe60815d000 0x7fe6080a9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.757+0000 7fe60cf88640 1 -- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60815d3e0 msgr2=0x7fe6080a9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:14.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.757+0000 7fe60cf88640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60815d3e0 0x7fe6080a9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:14.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.757+0000 7fe60cf88640 1 -- 172.21.15.27:0/1045313569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5fc0093a0 con 0x7fe6080cde80 2023-12-19T18:16:14.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.757+0000 7fe60cf88640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6080cde80 0x7fe6080caab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0098e0 tx=0x7fe5fc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:14.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.757+0000 7fe6057fa640 1 -- 172.21.15.27:0/1045313569 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc03cc80 con 0x7fe6080cde80 2023-12-19T18:16:14.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.758+0000 7fe6057fa640 1 -- 172.21.15.27:0/1045313569 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5fc03cde0 con 0x7fe6080cde80 2023-12-19T18:16:14.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.758+0000 7fe60f1ec640 1 -- 172.21.15.27:0/1045313569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe608149830 con 0x7fe6080cde80 2023-12-19T18:16:14.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.758+0000 7fe6057fa640 1 -- 172.21.15.27:0/1045313569 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5fc005b90 con 0x7fe6080cde80 2023-12-19T18:16:14.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.758+0000 7fe60f1ec640 1 -- 172.21.15.27:0/1045313569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe608149df0 con 0x7fe6080cde80 2023-12-19T18:16:14.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.759+0000 7fe60f1ec640 1 -- 172.21.15.27:0/1045313569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6080b86c0 con 0x7fe6080cde80 2023-12-19T18:16:14.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.760+0000 7fe6057fa640 1 -- 172.21.15.27:0/1045313569 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe5fc005cf0 con 0x7fe6080cde80 2023-12-19T18:16:14.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.761+0000 7fe6057fa640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5ec06a430 0x7fe5ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:14.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.761+0000 7fe6057fa640 1 -- 172.21.15.27:0/1045313569 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5824+0+0 (secure 0 0 0) 0x7fe5fc0ab480 con 0x7fe6080cde80 2023-12-19T18:16:14.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.761+0000 7fe607fff640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5ec06a430 0x7fe5ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:14.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.762+0000 7fe607fff640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5ec06a430 0x7fe5ec06c8f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe60815b610 tx=0x7fe5f4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.767+0000 7fe6057fa640 1 -- 172.21.15.27:0/1045313569 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe5fc076ac0 con 0x7fe6080cde80 2023-12-19T18:16:14.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:14.968+0000 7fe60f1ec640 1 -- 172.21.15.27:0/1045313569 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe6080d0c20 con 0x7fe6080cde80 2023-12-19T18:16:15.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:15.808+0000 7fe6057fa640 1 -- 172.21.15.27:0/1045313569 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe5fc07a0e0 con 0x7fe6080cde80 2023-12-19T18:16:15.808 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:16:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:15.811+0000 7fe60f1ec640 1 -- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5ec06a430 msgr2=0x7fe5ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:15.811+0000 7fe60f1ec640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5ec06a430 0x7fe5ec06c8f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe60815b610 tx=0x7fe5f4009040 comp rx=0 tx=0).stop 2023-12-19T18:16:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:15.811+0000 7fe60f1ec640 1 -- 172.21.15.27:0/1045313569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6080cde80 msgr2=0x7fe6080caab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:15.811+0000 7fe60f1ec640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6080cde80 0x7fe6080caab0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fe5fc0098e0 tx=0x7fe5fc0046c0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:15.811+0000 7fe60f1ec640 1 -- 172.21.15.27:0/1045313569 shutdown_connections 2023-12-19T18:16:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:15.811+0000 7fe60f1ec640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5ec06a430 0x7fe5ec06c8f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:15.811+0000 7fe60f1ec640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe60815d3e0 0x7fe6080a9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:15.811+0000 7fe60f1ec640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe60815d000 0x7fe6080a9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:15.811+0000 7fe60f1ec640 1 --2- 172.21.15.27:0/1045313569 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6080cde80 0x7fe6080caab0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:15.811+0000 7fe60f1ec640 1 -- 172.21.15.27:0/1045313569 >> 172.21.15.27:0/1045313569 conn(0x7fe6080cf990 msgr2=0x7fe6080d1af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:15.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:15.812+0000 7fe60f1ec640 1 -- 172.21.15.27:0/1045313569 shutdown_connections 2023-12-19T18:16:15.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:15.812+0000 7fe60f1ec640 1 -- 172.21.15.27:0/1045313569 wait complete. 2023-12-19T18:16:15.838 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:16:16.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.062+0000 7f9a7ccee640 1 Processor -- start 2023-12-19T18:16:16.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.062+0000 7f9a7ccee640 1 -- start start 2023-12-19T18:16:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.064+0000 7f9a7ccee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a7815d000 0x7f9a780b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.064+0000 7f9a7ccee640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9a780cde80 0x7f9a780b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.064+0000 7f9a7ccee640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a780d1760 con 0x7f9a7815d3e0 2023-12-19T18:16:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.064+0000 7f9a7ccee640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a781610c0 con 0x7f9a780cde80 2023-12-19T18:16:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.064+0000 7f9a7ccee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780d2c00 con 0x7f9a7815d000 2023-12-19T18:16:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.064+0000 7f9a75d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9a7815d3e0 0x7f9a780cd770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60108/0 (socket says 172.21.15.27:60108) 2023-12-19T18:16:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.064+0000 7f9a75d9b640 1 -- 172.21.15.27:0/2243434185 learned_addr learned my addr 172.21.15.27:0/2243434185 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.064+0000 7f9a76d9d640 1 --2- 172.21.15.27:0/2243434185 >> v2:172.21.15.27:3300/0 conn(0x7f9a780cde80 0x7f9a780b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.065+0000 7f9a7659c640 1 --2- 172.21.15.27:0/2243434185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a7815d000 0x7f9a780b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.065+0000 7f9a7559a640 1 -- 172.21.15.27:0/2243434185 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 38212818 0 0) 0x7f9a780d1760 con 0x7f9a7815d3e0 2023-12-19T18:16:16.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.065+0000 7f9a76d9d640 1 -- 172.21.15.27:0/2243434185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a7815d000 msgr2=0x7f9a780b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.065+0000 7f9a76d9d640 1 --2- 172.21.15.27:0/2243434185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a7815d000 0x7f9a780b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.065+0000 7f9a76d9d640 1 -- 172.21.15.27:0/2243434185 >> v1:172.21.15.27:6789/0 conn(0x7f9a7815d3e0 legacy=0x7f9a780cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.065+0000 7f9a76d9d640 1 -- 172.21.15.27:0/2243434185 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a780ce350 con 0x7f9a780cde80 2023-12-19T18:16:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.067+0000 7f9a76d9d640 1 --2- 172.21.15.27:0/2243434185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a780cde80 0x7f9a780b4ed0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009d90 tx=0x7f9a7002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e2298eeea57e9e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.067+0000 7f9a7559a640 1 -- 172.21.15.27:0/2243434185 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7002cc60 con 0x7f9a780cde80 2023-12-19T18:16:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.067+0000 7f9a7559a640 1 -- 172.21.15.27:0/2243434185 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7002cdc0 con 0x7f9a780cde80 2023-12-19T18:16:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.067+0000 7f9a7559a640 1 -- 172.21.15.27:0/2243434185 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a70034660 con 0x7f9a780cde80 2023-12-19T18:16:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.067+0000 7f9a7ccee640 1 -- 172.21.15.27:0/2243434185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a780cde80 msgr2=0x7f9a780b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.067+0000 7f9a7ccee640 1 --2- 172.21.15.27:0/2243434185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a780cde80 0x7f9a780b4ed0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9a70009d90 tx=0x7f9a7002beb0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.068+0000 7f9a7ccee640 1 -- 172.21.15.27:0/2243434185 shutdown_connections 2023-12-19T18:16:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.068+0000 7f9a7ccee640 1 --2- 172.21.15.27:0/2243434185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a780cde80 0x7f9a780b4ed0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.068+0000 7f9a7ccee640 1 --2- 172.21.15.27:0/2243434185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a7815d000 0x7f9a780b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.068+0000 7f9a7ccee640 1 -- 172.21.15.27:0/2243434185 >> 172.21.15.27:0/2243434185 conn(0x7f9a780cf990 msgr2=0x7f9a780cfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.068+0000 7f9a7ccee640 1 -- 172.21.15.27:0/2243434185 shutdown_connections 2023-12-19T18:16:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.068+0000 7f9a7ccee640 1 -- 172.21.15.27:0/2243434185 wait complete. 2023-12-19T18:16:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.070+0000 7f9a7ccee640 1 Processor -- start 2023-12-19T18:16:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.071+0000 7f9a7ccee640 1 -- start start 2023-12-19T18:16:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.072+0000 7f9a7ccee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a780cde80 0x7f9a780caaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.072+0000 7f9a7ccee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a7815d000 0x7f9a780d3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.072+0000 7f9a7ccee640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a7815d3e0 0x7f9a7815ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.072+0000 7f9a7ccee640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780a9b40 con 0x7f9a7815d000 2023-12-19T18:16:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.072+0000 7f9a7ccee640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780cb1a0 con 0x7f9a7815d3e0 2023-12-19T18:16:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.072+0000 7f9a7ccee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a780cb480 con 0x7f9a780cde80 2023-12-19T18:16:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.072+0000 7f9a75d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a7815d000 0x7f9a780d3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.072+0000 7f9a75d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a7815d000 0x7f9a780d3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59112/0 (socket says 172.21.15.27:59112) 2023-12-19T18:16:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.072+0000 7f9a75d9b640 1 -- 172.21.15.27:0/1436808593 learned_addr learned my addr 172.21.15.27:0/1436808593 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:16.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.073+0000 7f9a7659c640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a780cde80 0x7f9a780caaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.073+0000 7f9a76d9d640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a7815d3e0 0x7f9a7815ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.073+0000 7f9a75d9b640 1 -- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a780cde80 msgr2=0x7f9a780caaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.073+0000 7f9a75d9b640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a780cde80 0x7f9a780caaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.073+0000 7f9a75d9b640 1 -- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a7815d3e0 msgr2=0x7f9a7815ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.073+0000 7f9a75d9b640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a7815d3e0 0x7f9a7815ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.073+0000 7f9a75d9b640 1 -- 172.21.15.27:0/1436808593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a70009a70 con 0x7f9a7815d000 2023-12-19T18:16:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.073+0000 7f9a75d9b640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a7815d000 0x7f9a780d3110 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00c4d0 tx=0x7f9a6c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.073+0000 7f9a66ffd640 1 -- 172.21.15.27:0/1436808593 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c0074e0 con 0x7f9a7815d000 2023-12-19T18:16:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.073+0000 7f9a66ffd640 1 -- 172.21.15.27:0/1436808593 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a6c007660 con 0x7f9a7815d000 2023-12-19T18:16:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.073+0000 7f9a66ffd640 1 -- 172.21.15.27:0/1436808593 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a6c004a20 con 0x7f9a7815d000 2023-12-19T18:16:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.073+0000 7f9a7ccee640 1 -- 172.21.15.27:0/1436808593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a780cb760 con 0x7f9a7815d000 2023-12-19T18:16:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.074+0000 7f9a7ccee640 1 -- 172.21.15.27:0/1436808593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a780cbd20 con 0x7f9a7815d000 2023-12-19T18:16:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.075+0000 7f9a7ccee640 1 -- 172.21.15.27:0/1436808593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a78067110 con 0x7f9a7815d000 2023-12-19T18:16:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.076+0000 7f9a66ffd640 1 -- 172.21.15.27:0/1436808593 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9a6c004b80 con 0x7f9a7815d000 2023-12-19T18:16:16.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.077+0000 7f9a66ffd640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a5806a430 0x7f9a5806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:16.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.077+0000 7f9a66ffd640 1 -- 172.21.15.27:0/1436808593 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5386+0+0 (secure 0 0 0) 0x7f9a6c087aa0 con 0x7f9a7815d000 2023-12-19T18:16:16.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.077+0000 7f9a7659c640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a5806a430 0x7f9a5806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:16.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.078+0000 7f9a7659c640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a5806a430 0x7f9a5806c8f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9a780c9a20 tx=0x7f9a68008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:16.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.081+0000 7f9a66ffd640 1 -- 172.21.15.27:0/1436808593 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9a6c053130 con 0x7f9a7815d000 2023-12-19T18:16:16.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.271+0000 7f9a7ccee640 1 -- 172.21.15.27:0/1436808593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9a780d13c0 con 0x7f9a7815d000 2023-12-19T18:16:16.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.818+0000 7f9a66ffd640 1 -- 172.21.15.27:0/1436808593 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9a6c018020 con 0x7f9a7815d000 2023-12-19T18:16:16.822 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-12-19T18:16:16.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.825+0000 7f9a7ccee640 1 -- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a5806a430 msgr2=0x7f9a5806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.825+0000 7f9a7ccee640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a5806a430 0x7f9a5806c8f0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f9a780c9a20 tx=0x7f9a68008040 comp rx=0 tx=0).stop 2023-12-19T18:16:16.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.825+0000 7f9a7ccee640 1 -- 172.21.15.27:0/1436808593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a7815d000 msgr2=0x7f9a780d3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:16.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.825+0000 7f9a7ccee640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a7815d000 0x7f9a780d3110 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9a6c00c4d0 tx=0x7f9a6c00c990 comp rx=0 tx=0).stop 2023-12-19T18:16:16.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.826+0000 7f9a7ccee640 1 -- 172.21.15.27:0/1436808593 shutdown_connections 2023-12-19T18:16:16.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.826+0000 7f9a7ccee640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a5806a430 0x7f9a5806c8f0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.826+0000 7f9a7ccee640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a7815d3e0 0x7f9a7815ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.826+0000 7f9a7ccee640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a7815d000 0x7f9a780d3110 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.826+0000 7f9a7ccee640 1 --2- 172.21.15.27:0/1436808593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a780cde80 0x7f9a780caaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.826+0000 7f9a7ccee640 1 -- 172.21.15.27:0/1436808593 >> 172.21.15.27:0/1436808593 conn(0x7f9a780cf990 msgr2=0x7f9a780b6140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.826+0000 7f9a7ccee640 1 -- 172.21.15.27:0/1436808593 shutdown_connections 2023-12-19T18:16:16.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:16.826+0000 7f9a7ccee640 1 -- 172.21.15.27:0/1436808593 wait complete. 2023-12-19T18:16:16.848 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:16:16.848 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:16:16.849 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:16:16.849 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:16:16.849 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:16:16.849 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:16:16.849 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:16:16.850 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:16:16.850 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:16:16.850 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:16:16.850 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:16:16.850 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:16:16.851 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:16:16.851 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:16:16.851 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:16:16.851 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:16:16.872 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:16:16.873 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:16:16.874 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:16:16.875 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:16:16.876 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:16:16.877 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:16:16.879 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:16:16.879 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:16:16.922 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:16:16.922 DEBUG:teuthology.parallel:result is None 2023-12-19T18:16:16.925 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:16:16.925 DEBUG:teuthology.parallel:result is None 2023-12-19T18:16:16.928 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:16:16.929 DEBUG:teuthology.parallel:result is None 2023-12-19T18:16:16.932 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:16:16.932 DEBUG:teuthology.parallel:result is None 2023-12-19T18:16:16.932 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:16:17.089 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-19T18:16:17.090 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:16:17.091 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-19T18:16:17.092 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:16:17.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.135+0000 7f308acaf640 1 Processor -- start 2023-12-19T18:16:17.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.136+0000 7f308acaf640 1 -- start start 2023-12-19T18:16:17.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.137+0000 7f308acaf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f308415d1d0 0x7f30840ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:17.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.137+0000 7f308acaf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3084071140 0x7f30840cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:17.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.137+0000 7f308acaf640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30840ad8e0 con 0x7f30840cd790 2023-12-19T18:16:17.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.137+0000 7f308acaf640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30840cdbc0 con 0x7f3084071140 2023-12-19T18:16:17.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.137+0000 7f308acaf640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3084070150 con 0x7f308415d1d0 2023-12-19T18:16:17.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.137+0000 7f3083fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3084071140 0x7f30840cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:17.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.137+0000 7f3083fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3084071140 0x7f30840cd250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59136/0 (socket says 172.21.15.27:59136) 2023-12-19T18:16:17.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.137+0000 7f3083fff640 1 -- 172.21.15.27:0/609127618 learned_addr learned my addr 172.21.15.27:0/609127618 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:17.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.137+0000 7f3088a4b640 1 --2- 172.21.15.27:0/609127618 >> v2:172.21.15.192:3300/0 conn(0x7f308415d1d0 0x7f30840ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:17.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.138+0000 7f3083fff640 1 -- 172.21.15.27:0/609127618 >> v2:172.21.15.192:3300/0 conn(0x7f308415d1d0 msgr2=0x7f30840ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:17.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.138+0000 7f3083fff640 1 --2- 172.21.15.27:0/609127618 >> v2:172.21.15.192:3300/0 conn(0x7f308415d1d0 0x7f30840ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:17.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.138+0000 7f3083fff640 1 -- 172.21.15.27:0/609127618 >> v1:172.21.15.192:6789/0 conn(0x7f30840cd790 legacy=0x7f30840c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:17.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.138+0000 7f3083fff640 1 -- 172.21.15.27:0/609127618 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30840ce7c0 con 0x7f3084071140 2023-12-19T18:16:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.139+0000 7f3083fff640 1 --2- 172.21.15.27:0/609127618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3084071140 0x7f30840cd250 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f30840cc610 tx=0x7f307002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec4a554657258ba server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.139+0000 7f30837fe640 1 -- 172.21.15.27:0/609127618 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070031880 con 0x7f3084071140 2023-12-19T18:16:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.139+0000 7f30837fe640 1 -- 172.21.15.27:0/609127618 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30700319e0 con 0x7f3084071140 2023-12-19T18:16:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.139+0000 7f30837fe640 1 -- 172.21.15.27:0/609127618 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3070031d70 con 0x7f3084071140 2023-12-19T18:16:17.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.140+0000 7f308acaf640 1 -- 172.21.15.27:0/609127618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3084071140 msgr2=0x7f30840cd250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:17.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.140+0000 7f308acaf640 1 --2- 172.21.15.27:0/609127618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3084071140 0x7f30840cd250 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f30840cc610 tx=0x7f307002b920 comp rx=0 tx=0).stop 2023-12-19T18:16:17.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.140+0000 7f308acaf640 1 -- 172.21.15.27:0/609127618 shutdown_connections 2023-12-19T18:16:17.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.140+0000 7f308acaf640 1 --2- 172.21.15.27:0/609127618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3084071140 0x7f30840cd250 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:17.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.140+0000 7f308acaf640 1 --2- 172.21.15.27:0/609127618 >> v2:172.21.15.192:3300/0 conn(0x7f308415d1d0 0x7f30840ccd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:17.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.140+0000 7f308acaf640 1 -- 172.21.15.27:0/609127618 >> 172.21.15.27:0/609127618 conn(0x7f30840ceb50 msgr2=0x7f30840cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:17.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.140+0000 7f308acaf640 1 -- 172.21.15.27:0/609127618 shutdown_connections 2023-12-19T18:16:17.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.140+0000 7f308acaf640 1 -- 172.21.15.27:0/609127618 wait complete. 2023-12-19T18:16:17.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.142+0000 7f308acaf640 1 Processor -- start 2023-12-19T18:16:17.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.142+0000 7f308acaf640 1 -- start start 2023-12-19T18:16:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f308acaf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3084071140 0x7f30841f5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f308acaf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30840cd790 0x7f30841f3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f308acaf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f308415d1d0 0x7f30841e68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f308acaf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30841e64c0 con 0x7f3084071140 2023-12-19T18:16:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f308acaf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30840b19b0 con 0x7f308415d1d0 2023-12-19T18:16:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f308acaf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30841e58d0 con 0x7f30840cd790 2023-12-19T18:16:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f3088a4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3084071140 0x7f30841f5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f3088a4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3084071140 0x7f30841f5980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59142/0 (socket says 172.21.15.27:59142) 2023-12-19T18:16:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f3088a4b640 1 -- 172.21.15.27:0/1551993508 learned_addr learned my addr 172.21.15.27:0/1551993508 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f308924c640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f308415d1d0 0x7f30841e68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f3083fff640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30840cd790 0x7f30841f3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f3088a4b640 1 -- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30840cd790 msgr2=0x7f30841f3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f3088a4b640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30840cd790 0x7f30841f3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f3088a4b640 1 -- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f308415d1d0 msgr2=0x7f30841e68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f3088a4b640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f308415d1d0 0x7f30841e68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f3088a4b640 1 -- 172.21.15.27:0/1551993508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30700094d0 con 0x7f3084071140 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f3083fff640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30840cd790 0x7f30841f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.143+0000 7f308924c640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f308415d1d0 0x7f30841e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.144+0000 7f3088a4b640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3084071140 0x7f30841f5980 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f307800e870 tx=0x7f307800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.144+0000 7f30817fa640 1 -- 172.21.15.27:0/1551993508 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3078009510 con 0x7f3084071140 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.144+0000 7f30817fa640 1 -- 172.21.15.27:0/1551993508 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f307800eec0 con 0x7f3084071140 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.144+0000 7f30817fa640 1 -- 172.21.15.27:0/1551993508 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f307800cb20 con 0x7f3084071140 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.144+0000 7f308acaf640 1 -- 172.21.15.27:0/1551993508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30841e5b20 con 0x7f3084071140 2023-12-19T18:16:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.144+0000 7f308acaf640 1 -- 172.21.15.27:0/1551993508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30841e60e0 con 0x7f3084071140 2023-12-19T18:16:17.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.145+0000 7f308acaf640 1 -- 172.21.15.27:0/1551993508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30841e6600 con 0x7f3084071140 2023-12-19T18:16:17.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.146+0000 7f30817fa640 1 -- 172.21.15.27:0/1551993508 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f307800ccc0 con 0x7f3084071140 2023-12-19T18:16:17.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.147+0000 7f30817fa640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f306806a430 0x7f306806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.147+0000 7f30817fa640 1 -- 172.21.15.27:0/1551993508 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3078088770 con 0x7f3084071140 2023-12-19T18:16:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.147+0000 7f3083fff640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f306806a430 0x7f306806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.148+0000 7f3083fff640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f306806a430 0x7f306806c8f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3070036000 tx=0x7f307003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:17.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.152+0000 7f30817fa640 1 -- 172.21.15.27:0/1551993508 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3078054050 con 0x7f3084071140 2023-12-19T18:16:17.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:17.359+0000 7f308acaf640 1 -- 172.21.15.27:0/1551993508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3084070610 con 0x7f3084071140 2023-12-19T18:16:18.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.138+0000 7f30817fa640 1 -- 172.21.15.27:0/1551993508 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3078057670 con 0x7f3084071140 2023-12-19T18:16:18.139 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2023-12-19T18:16:18.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.141+0000 7f308acaf640 1 -- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f306806a430 msgr2=0x7f306806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:18.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.141+0000 7f308acaf640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f306806a430 0x7f306806c8f0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3070036000 tx=0x7f307003c040 comp rx=0 tx=0).stop 2023-12-19T18:16:18.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.141+0000 7f308acaf640 1 -- 172.21.15.27:0/1551993508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3084071140 msgr2=0x7f30841f5980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:18.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.141+0000 7f308acaf640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3084071140 0x7f30841f5980 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f307800e870 tx=0x7f307800ec70 comp rx=0 tx=0).stop 2023-12-19T18:16:18.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.141+0000 7f308acaf640 1 -- 172.21.15.27:0/1551993508 shutdown_connections 2023-12-19T18:16:18.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.141+0000 7f308acaf640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f306806a430 0x7f306806c8f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.141+0000 7f308acaf640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f308415d1d0 0x7f30841e68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.141+0000 7f308acaf640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30840cd790 0x7f30841f3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.141+0000 7f308acaf640 1 --2- 172.21.15.27:0/1551993508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3084071140 0x7f30841f5980 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.141+0000 7f308acaf640 1 -- 172.21.15.27:0/1551993508 >> 172.21.15.27:0/1551993508 conn(0x7f30840ceb50 msgr2=0x7f308406ba10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:18.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.142+0000 7f308acaf640 1 -- 172.21.15.27:0/1551993508 shutdown_connections 2023-12-19T18:16:18.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.142+0000 7f308acaf640 1 -- 172.21.15.27:0/1551993508 wait complete. 2023-12-19T18:16:18.163 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:16:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.397+0000 7fc24af11640 1 Processor -- start 2023-12-19T18:16:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.397+0000 7fc24af11640 1 -- start start 2023-12-19T18:16:18.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.399+0000 7fc24af11640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc24415aab0 0x7fc244070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:18.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.399+0000 7fc24af11640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24415aed0 con 0x7fc244070aa0 2023-12-19T18:16:18.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.399+0000 7fc24af11640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24415b110 con 0x7fc244154170 2023-12-19T18:16:18.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.399+0000 7fc24af11640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc244071470 con 0x7fc24415aab0 2023-12-19T18:16:18.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.399+0000 7fc24a710640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc24415aab0 0x7fc244070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:18.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.399+0000 7fc24a710640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc24415aab0 0x7fc244070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59144/0 (socket says 172.21.15.27:59144) 2023-12-19T18:16:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.399+0000 7fc24a710640 1 -- 172.21.15.27:0/1419552349 learned_addr learned my addr 172.21.15.27:0/1419552349 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.399+0000 7fc24970e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc244070aa0 0x7fc24415a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60110/0 (socket says 172.21.15.27:60110) 2023-12-19T18:16:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.400+0000 7fc24a710640 1 -- 172.21.15.27:0/1419552349 >> v1:172.21.15.192:6789/0 conn(0x7fc244154170 legacy=0x7fc244070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.400+0000 7fc24a710640 1 -- 172.21.15.27:0/1419552349 >> v1:172.21.15.27:6789/0 conn(0x7fc244070aa0 legacy=0x7fc24415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.400+0000 7fc24a710640 1 -- 172.21.15.27:0/1419552349 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24406fce0 con 0x7fc24415aab0 2023-12-19T18:16:18.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.400+0000 7fc248f0d640 1 -- 172.21.15.27:0/1419552349 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223802501 0 0) 0x7fc234001110 con 0x7fc244070aa0 2023-12-19T18:16:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.401+0000 7fc24a710640 1 --2- 172.21.15.27:0/1419552349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc24415aab0 0x7fc244070f30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009e50 tx=0x7fc23c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44bfb9aefb7186b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.401+0000 7fc248f0d640 1 -- 172.21.15.27:0/1419552349 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c02ca80 con 0x7fc24415aab0 2023-12-19T18:16:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.401+0000 7fc248f0d640 1 -- 172.21.15.27:0/1419552349 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c02cbe0 con 0x7fc24415aab0 2023-12-19T18:16:18.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.401+0000 7fc248f0d640 1 -- 172.21.15.27:0/1419552349 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0343f0 con 0x7fc24415aab0 2023-12-19T18:16:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.402+0000 7fc24af11640 1 -- 172.21.15.27:0/1419552349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc24415aab0 msgr2=0x7fc244070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.402+0000 7fc24af11640 1 --2- 172.21.15.27:0/1419552349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc24415aab0 0x7fc244070f30 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009e50 tx=0x7fc23c02bee0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.402+0000 7fc24af11640 1 -- 172.21.15.27:0/1419552349 shutdown_connections 2023-12-19T18:16:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.402+0000 7fc24af11640 1 --2- 172.21.15.27:0/1419552349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc24415aab0 0x7fc244070f30 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.402+0000 7fc24af11640 1 -- 172.21.15.27:0/1419552349 >> 172.21.15.27:0/1419552349 conn(0x7fc24406a7d0 msgr2=0x7fc2440ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:18.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.402+0000 7fc24af11640 1 -- 172.21.15.27:0/1419552349 shutdown_connections 2023-12-19T18:16:18.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.402+0000 7fc24af11640 1 -- 172.21.15.27:0/1419552349 wait complete. 2023-12-19T18:16:18.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.404+0000 7fc24af11640 1 Processor -- start 2023-12-19T18:16:18.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.404+0000 7fc24af11640 1 -- start start 2023-12-19T18:16:18.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.405+0000 7fc24af11640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc244070aa0 0x7fc2441582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:18.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.405+0000 7fc24af11640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc244154170 0x7fc2440b2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:18.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.405+0000 7fc24af11640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc24415aab0 0x7fc2440b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:18.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.405+0000 7fc24af11640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24414cc60 con 0x7fc24415aab0 2023-12-19T18:16:18.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.405+0000 7fc24af11640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2441ee150 con 0x7fc244070aa0 2023-12-19T18:16:18.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.405+0000 7fc24af11640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2440cd2d0 con 0x7fc244154170 2023-12-19T18:16:18.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.405+0000 7fc24a710640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc24415aab0 0x7fc2440b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:18.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.405+0000 7fc24a710640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc24415aab0 0x7fc2440b2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59152/0 (socket says 172.21.15.27:59152) 2023-12-19T18:16:18.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.405+0000 7fc24a710640 1 -- 172.21.15.27:0/1513506585 learned_addr learned my addr 172.21.15.27:0/1513506585 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:18.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.406+0000 7fc249f0f640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc244070aa0 0x7fc2441582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:18.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.406+0000 7fc24970e640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc244154170 0x7fc2440b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:18.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.406+0000 7fc24a710640 1 -- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc244154170 msgr2=0x7fc2440b2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:18.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.406+0000 7fc24a710640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc244154170 0x7fc2440b2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.406+0000 7fc24a710640 1 -- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc244070aa0 msgr2=0x7fc2441582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:18.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.406+0000 7fc24a710640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc244070aa0 0x7fc2441582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.406+0000 7fc24a710640 1 -- 172.21.15.27:0/1513506585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c009b30 con 0x7fc24415aab0 2023-12-19T18:16:18.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.406+0000 7fc24a710640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc24415aab0 0x7fc2440b2890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009e50 tx=0x7fc23c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.407+0000 7fc23a7fc640 1 -- 172.21.15.27:0/1513506585 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c009510 con 0x7fc24415aab0 2023-12-19T18:16:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.407+0000 7fc23a7fc640 1 -- 172.21.15.27:0/1513506585 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c040470 con 0x7fc24415aab0 2023-12-19T18:16:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.407+0000 7fc23a7fc640 1 -- 172.21.15.27:0/1513506585 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c040800 con 0x7fc24415aab0 2023-12-19T18:16:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.407+0000 7fc24af11640 1 -- 172.21.15.27:0/1513506585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2440cd5b0 con 0x7fc24415aab0 2023-12-19T18:16:18.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.407+0000 7fc24af11640 1 -- 172.21.15.27:0/1513506585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2440cdb70 con 0x7fc24415aab0 2023-12-19T18:16:18.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.408+0000 7fc24af11640 1 -- 172.21.15.27:0/1513506585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2441ee290 con 0x7fc24415aab0 2023-12-19T18:16:18.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.409+0000 7fc23a7fc640 1 -- 172.21.15.27:0/1513506585 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc23c02cd50 con 0x7fc24415aab0 2023-12-19T18:16:18.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.410+0000 7fc23a7fc640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc22806a430 0x7fc22806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:18.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.410+0000 7fc23a7fc640 1 -- 172.21.15.27:0/1513506585 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fc23c0ab080 con 0x7fc24415aab0 2023-12-19T18:16:18.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.410+0000 7fc249f0f640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc22806a430 0x7fc22806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.412+0000 7fc249f0f640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc22806a430 0x7fc22806c8f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc240004ea0 tx=0x7fc240009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:18.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.415+0000 7fc23a7fc640 1 -- 172.21.15.27:0/1513506585 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc23c0ddca0 con 0x7fc24415aab0 2023-12-19T18:16:18.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.678+0000 7fc24af11640 1 -- 172.21.15.27:0/1513506585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc24404e840 con 0x7fc24415aab0 2023-12-19T18:16:18.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.680+0000 7fc23a7fc640 1 -- 172.21.15.27:0/1513506585 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v4) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fc23c076920 con 0x7fc24415aab0 2023-12-19T18:16:18.682 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:18.682 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:16:18.682 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2023-12-19T18:16:18.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.682 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2023-12-19T18:16:18.682 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFl+4ruHBAAXwbpmT43SmlvNbuZj5e+bQ==", 2023-12-19T18:16:18.682 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.683 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:16:18.683 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:16:18.683 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:16:18.683 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.683 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.683 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2023-12-19T18:16:18.683 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmZyUBhAAG3kZkXvJIo7N9f1j+DPCSQ==", 2023-12-19T18:16:18.683 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.683 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:16:18.683 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlChNOMBAAX4GV3iIGkSQdAsYu9oCPDw==", 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.685 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2023-12-19T18:16:18.685 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlTmFXFhAABA0NMUxKBV1ExAhYc5RFbQ==", 2023-12-19T18:16:18.685 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.685 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:16:18.685 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:16:18.685 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:16:18.685 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.685 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFluleFIBAAs1goFVMOX4CbjwFhg0KoBg==", 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFlInW3MxAAI4hY1GBxXYXDswKKrEyDTg==", 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:16:18.686 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFlAWZ5CRAA82/2r9/sT0L9VlGHbXksQw==", 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2023-12-19T18:16:18.687 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFl1IvaGRAAqVacDDPLx+NqBOGp6MeX2w==", 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlr0BBHRAA+7gfs/RG6DtABnYPg059Tw==", 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.688 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlXZbELRAAKD0qOkhk7skfc4xcAC8Lhg==", 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlF07MARAA2TJJ+5C1HIU+SVhqmKaVHA==", 2023-12-19T18:16:18.689 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:16:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2023-12-19T18:16:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlISPGEhAAdFIRRkXlLyOx/oMZ4MgL9Q==", 2023-12-19T18:16:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:16:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:16:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmNM9JRAARgS20O8zpm6H70WDv+lgGw==", 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2023-12-19T18:16:18.691 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlWoZ8AhAADWvKVzeDhDGHtYNUFIOWsA==", 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFllWFVMhAAD9ebDOppHTODVQjg6ppP3g==", 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:16:18.692 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlwbIQDRAAt/h55zBNCEId0vkYSUQ3TQ==", 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2023-12-19T18:16:18.693 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB53YFloCb6CxAAmZQyz5XOEqDOOiROJHLzxQ==", 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-mds", 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCM3YFlvupYHxAA4wqfFAyQa7Owo7vTR/FVjw==", 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-mds" 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-mgr", 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCM3YFll/5YHxAA+Ni+LIjPHjqHbeipjpNK6A==", 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-mgr" 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-osd", 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCM3YFl7QxZHxAAgibkN7S4L8Iz6iQHQOvAJw==", 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-osd" 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.695 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-rbd", 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCM3YFlDRdZHxAAIhBJMMU1c799AbqwJCirWQ==", 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-rbd" 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCM3YFloSFZHxAAkobcGtpFauyEW6f6X1sbNg==", 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.696 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.bootstrap-rgw", 2023-12-19T18:16:18.697 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCM3YFlnytZHxAAKMzlAxhSK9r4E2kBmYiHdQ==", 2023-12-19T18:16:18.697 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.697 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile bootstrap-rgw" 2023-12-19T18:16:18.697 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.697 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2023-12-19T18:16:18.697 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFltBhxDxAAiAS9CYPY86IMmJ7NGgA2lw==", 2023-12-19T18:16:18.697 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.697 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:16:18.697 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlUnAhBBAAi++W9XK3o3Y/185hB3LuIg==", 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:16:18.698 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:16:18.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.683+0000 7fc24af11640 1 -- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc22806a430 msgr2=0x7fc22806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:18.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.683+0000 7fc24af11640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc22806a430 0x7fc22806c8f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc240004ea0 tx=0x7fc240009040 comp rx=0 tx=0).stop 2023-12-19T18:16:18.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.683+0000 7fc24af11640 1 -- 172.21.15.27:0/1513506585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc24415aab0 msgr2=0x7fc2440b2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:18.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.683+0000 7fc24af11640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc24415aab0 0x7fc2440b2890 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009e50 tx=0x7fc23c004a40 comp rx=0 tx=0).stop 2023-12-19T18:16:18.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.683+0000 7fc24af11640 1 -- 172.21.15.27:0/1513506585 shutdown_connections 2023-12-19T18:16:18.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.683+0000 7fc24af11640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc22806a430 0x7fc22806c8f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.683+0000 7fc24af11640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc24415aab0 0x7fc2440b2890 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.683+0000 7fc24af11640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc244154170 0x7fc2440b2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.683+0000 7fc24af11640 1 --2- 172.21.15.27:0/1513506585 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc244070aa0 0x7fc2441582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.683+0000 7fc24af11640 1 -- 172.21.15.27:0/1513506585 >> 172.21.15.27:0/1513506585 conn(0x7fc24406a7d0 msgr2=0x7fc2440ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:18.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.683+0000 7fc24af11640 1 -- 172.21.15.27:0/1513506585 shutdown_connections 2023-12-19T18:16:18.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.684+0000 7fc24af11640 1 -- 172.21.15.27:0/1513506585 wait complete. 2023-12-19T18:16:18.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-12-19T18:16:18.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.939+0000 7f945f59e640 1 Processor -- start 2023-12-19T18:16:18.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.940+0000 7f945f59e640 1 -- start start 2023-12-19T18:16:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.941+0000 7f945f59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f946015ac50 0x7f9460070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.941+0000 7f945f59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f94600aef20 0x7f946006b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.941+0000 7f945f59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94600adf10 con 0x7f94600af360 2023-12-19T18:16:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.941+0000 7f945f59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94600adac0 con 0x7f94600aef20 2023-12-19T18:16:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.941+0000 7f945f59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94600ae160 con 0x7f946015ac50 2023-12-19T18:16:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.942+0000 7f9457fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f94600aef20 0x7f946006b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.942+0000 7f9457fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f94600aef20 0x7f946006b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59158/0 (socket says 172.21.15.27:59158) 2023-12-19T18:16:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.942+0000 7f9457fff640 1 -- 172.21.15.27:0/3932538574 learned_addr learned my addr 172.21.15.27:0/3932538574 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.942+0000 7f945e59c640 1 --2- 172.21.15.27:0/3932538574 >> v2:172.21.15.192:3300/0 conn(0x7f946015ac50 0x7f9460070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.942+0000 7f9457fff640 1 -- 172.21.15.27:0/3932538574 >> v2:172.21.15.192:3300/0 conn(0x7f946015ac50 msgr2=0x7f9460070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.942+0000 7f9457fff640 1 --2- 172.21.15.27:0/3932538574 >> v2:172.21.15.192:3300/0 conn(0x7f946015ac50 0x7f9460070fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.942+0000 7f9457fff640 1 -- 172.21.15.27:0/3932538574 >> v1:172.21.15.27:6789/0 conn(0x7f94600af360 legacy=0x7f946006b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.943+0000 7f9457fff640 1 -- 172.21.15.27:0/3932538574 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94600ae440 con 0x7f94600aef20 2023-12-19T18:16:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.943+0000 7f945dd9b640 1 -- 172.21.15.27:0/3932538574 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2248866091 0 0) 0x7f94600adf10 con 0x7f94600af360 2023-12-19T18:16:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.944+0000 7f9457fff640 1 --2- 172.21.15.27:0/3932538574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94600aef20 0x7f946006b310 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f946006cfb0 tx=0x7f944802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=232257cebadcb487 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.944+0000 7f945dd9b640 1 -- 172.21.15.27:0/3932538574 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94480317e0 con 0x7f94600aef20 2023-12-19T18:16:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.944+0000 7f945dd9b640 1 -- 172.21.15.27:0/3932538574 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9448031940 con 0x7f94600aef20 2023-12-19T18:16:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.944+0000 7f945dd9b640 1 -- 172.21.15.27:0/3932538574 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9448031cd0 con 0x7f94600aef20 2023-12-19T18:16:18.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.945+0000 7f945f59e640 1 -- 172.21.15.27:0/3932538574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94600aef20 msgr2=0x7f946006b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:18.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.945+0000 7f945f59e640 1 --2- 172.21.15.27:0/3932538574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94600aef20 0x7f946006b310 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f946006cfb0 tx=0x7f944802b880 comp rx=0 tx=0).stop 2023-12-19T18:16:18.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.945+0000 7f945f59e640 1 -- 172.21.15.27:0/3932538574 shutdown_connections 2023-12-19T18:16:18.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.945+0000 7f945f59e640 1 --2- 172.21.15.27:0/3932538574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94600aef20 0x7f946006b310 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.945+0000 7f945f59e640 1 --2- 172.21.15.27:0/3932538574 >> v2:172.21.15.192:3300/0 conn(0x7f946015ac50 0x7f9460070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.945+0000 7f945f59e640 1 -- 172.21.15.27:0/3932538574 >> 172.21.15.27:0/3932538574 conn(0x7f946006a7d0 msgr2=0x7f94600ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:18.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.945+0000 7f945f59e640 1 -- 172.21.15.27:0/3932538574 shutdown_connections 2023-12-19T18:16:18.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.945+0000 7f945f59e640 1 -- 172.21.15.27:0/3932538574 wait complete. 2023-12-19T18:16:18.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.947+0000 7f945f59e640 1 Processor -- start 2023-12-19T18:16:18.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.947+0000 7f945f59e640 1 -- start start 2023-12-19T18:16:18.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.948+0000 7f945f59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94600aef20 0x7f94600c9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:18.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.948+0000 7f945f59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94600af360 0x7f94600b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.948+0000 7f945f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f946015ac50 0x7f94601f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.948+0000 7f945f59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94600cb260 con 0x7f946015ac50 2023-12-19T18:16:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.948+0000 7f945f59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9460157e60 con 0x7f94600aef20 2023-12-19T18:16:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.948+0000 7f945f59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94600b2280 con 0x7f94600af360 2023-12-19T18:16:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.948+0000 7f945ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f946015ac50 0x7f94601f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.948+0000 7f945ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f946015ac50 0x7f94601f5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59166/0 (socket says 172.21.15.27:59166) 2023-12-19T18:16:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.948+0000 7f945ed9d640 1 -- 172.21.15.27:0/2495335976 learned_addr learned my addr 172.21.15.27:0/2495335976 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.948+0000 7f945e59c640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94600aef20 0x7f94600c9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f9457fff640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94600af360 0x7f94600b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f945ed9d640 1 -- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94600af360 msgr2=0x7f94600b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f945ed9d640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94600af360 0x7f94600b1d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f945ed9d640 1 -- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94600aef20 msgr2=0x7f94600c9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f945ed9d640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94600aef20 0x7f94600c9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f945ed9d640 1 -- 172.21.15.27:0/2495335976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94480094d0 con 0x7f946015ac50 2023-12-19T18:16:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f9457fff640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94600af360 0x7f94600b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f945e59c640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94600aef20 0x7f94600c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f945ed9d640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f946015ac50 0x7f94601f5bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f945800c780 tx=0x7f945800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f9456ffd640 1 -- 172.21.15.27:0/2495335976 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458003da0 con 0x7f946015ac50 2023-12-19T18:16:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f9456ffd640 1 -- 172.21.15.27:0/2495335976 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f945800ce30 con 0x7f946015ac50 2023-12-19T18:16:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f9456ffd640 1 -- 172.21.15.27:0/2495335976 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458005290 con 0x7f946015ac50 2023-12-19T18:16:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f945f59e640 1 -- 172.21.15.27:0/2495335976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94600ca5e0 con 0x7f946015ac50 2023-12-19T18:16:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.949+0000 7f945f59e640 1 -- 172.21.15.27:0/2495335976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94600caba0 con 0x7f946015ac50 2023-12-19T18:16:18.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.950+0000 7f945f59e640 1 -- 172.21.15.27:0/2495335976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f946004e840 con 0x7f946015ac50 2023-12-19T18:16:18.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.952+0000 7f9456ffd640 1 -- 172.21.15.27:0/2495335976 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f94580053f0 con 0x7f946015ac50 2023-12-19T18:16:18.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.953+0000 7f9456ffd640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f944006a430 0x7f944006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:18.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.953+0000 7f9456ffd640 1 -- 172.21.15.27:0/2495335976 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9458087d60 con 0x7f946015ac50 2023-12-19T18:16:18.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.954+0000 7f945e59c640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f944006a430 0x7f944006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:18.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.955+0000 7f945e59c640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f944006a430 0x7f944006c8f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9450006fd0 tx=0x7f9450008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:18.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:18.958+0000 7f9456ffd640 1 -- 172.21.15.27:0/2495335976 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9458053690 con 0x7f946015ac50 2023-12-19T18:16:19.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.233+0000 7f945f59e640 1 -- 172.21.15.27:0/2495335976 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f946006fbf0 con 0x7f946015ac50 2023-12-19T18:16:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.242+0000 7f9456ffd640 1 -- 172.21.15.27:0/2495335976 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9458017020 con 0x7f946015ac50 2023-12-19T18:16:19.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.245+0000 7f945f59e640 1 -- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f944006a430 msgr2=0x7f944006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:19.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.245+0000 7f945f59e640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f944006a430 0x7f944006c8f0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f9450006fd0 tx=0x7f9450008040 comp rx=0 tx=0).stop 2023-12-19T18:16:19.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.245+0000 7f945f59e640 1 -- 172.21.15.27:0/2495335976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f946015ac50 msgr2=0x7f94601f5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:19.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.245+0000 7f945f59e640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f946015ac50 0x7f94601f5bf0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f945800c780 tx=0x7f945800cc40 comp rx=0 tx=0).stop 2023-12-19T18:16:19.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.245+0000 7f945f59e640 1 -- 172.21.15.27:0/2495335976 shutdown_connections 2023-12-19T18:16:19.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.245+0000 7f945f59e640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f944006a430 0x7f944006c8f0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.245+0000 7f945f59e640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f946015ac50 0x7f94601f5bf0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.245+0000 7f945f59e640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f94600af360 0x7f94600b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.245+0000 7f945f59e640 1 --2- 172.21.15.27:0/2495335976 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f94600aef20 0x7f94600c9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.245+0000 7f945f59e640 1 -- 172.21.15.27:0/2495335976 >> 172.21.15.27:0/2495335976 conn(0x7f946006a7d0 msgr2=0x7f9460159ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.245+0000 7f945f59e640 1 -- 172.21.15.27:0/2495335976 shutdown_connections 2023-12-19T18:16:19.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.245+0000 7f945f59e640 1 -- 172.21.15.27:0/2495335976 wait complete. 2023-12-19T18:16:19.266 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-12-19T18:16:19.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.504+0000 7f8699737640 1 Processor -- start 2023-12-19T18:16:19.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.504+0000 7f8699737640 1 -- start start 2023-12-19T18:16:19.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.505+0000 7f8699737640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f869415ac50 0x7f8694070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:19.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.505+0000 7f8699737640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f86940aef20 0x7f869406b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:19.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.505+0000 7f8699737640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86940adf10 con 0x7f86940af360 2023-12-19T18:16:19.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.505+0000 7f8699737640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86940adac0 con 0x7f86940aef20 2023-12-19T18:16:19.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.505+0000 7f8699737640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86940ae160 con 0x7f869415ac50 2023-12-19T18:16:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.506+0000 7f86937fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f86940aef20 0x7f869406b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.506+0000 7f86937fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f86940aef20 0x7f869406b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59170/0 (socket says 172.21.15.27:59170) 2023-12-19T18:16:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.506+0000 7f86937fe640 1 -- 172.21.15.27:0/3608869999 learned_addr learned my addr 172.21.15.27:0/3608869999 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.506+0000 7f8693fff640 1 --2- 172.21.15.27:0/3608869999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f869415ac50 0x7f8694070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.506+0000 7f86937fe640 1 -- 172.21.15.27:0/3608869999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f869415ac50 msgr2=0x7f8694070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.506+0000 7f86937fe640 1 --2- 172.21.15.27:0/3608869999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f869415ac50 0x7f8694070fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.506+0000 7f86937fe640 1 -- 172.21.15.27:0/3608869999 >> v1:172.21.15.192:6789/0 conn(0x7f86940af360 legacy=0x7f869406b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.506+0000 7f86937fe640 1 -- 172.21.15.27:0/3608869999 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86940ae440 con 0x7f86940aef20 2023-12-19T18:16:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.507+0000 7f86937fe640 1 --2- 172.21.15.27:0/3608869999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86940aef20 0x7f869406b310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8688009870 tx=0x7f868802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3de829ef32927088 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.508+0000 7f8692ffd640 1 -- 172.21.15.27:0/3608869999 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688031740 con 0x7f86940aef20 2023-12-19T18:16:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.508+0000 7f8692ffd640 1 -- 172.21.15.27:0/3608869999 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86880318a0 con 0x7f86940aef20 2023-12-19T18:16:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.508+0000 7f8692ffd640 1 -- 172.21.15.27:0/3608869999 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8688031c30 con 0x7f86940aef20 2023-12-19T18:16:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.508+0000 7f8699737640 1 -- 172.21.15.27:0/3608869999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86940aef20 msgr2=0x7f869406b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.508+0000 7f8699737640 1 --2- 172.21.15.27:0/3608869999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86940aef20 0x7f869406b310 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8688009870 tx=0x7f868802b7e0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.508+0000 7f8699737640 1 -- 172.21.15.27:0/3608869999 shutdown_connections 2023-12-19T18:16:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.508+0000 7f8699737640 1 --2- 172.21.15.27:0/3608869999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86940aef20 0x7f869406b310 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.509+0000 7f8699737640 1 --2- 172.21.15.27:0/3608869999 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f869415ac50 0x7f8694070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.509+0000 7f8699737640 1 -- 172.21.15.27:0/3608869999 >> 172.21.15.27:0/3608869999 conn(0x7f869406a7d0 msgr2=0x7f86940ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:19.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.509+0000 7f8699737640 1 -- 172.21.15.27:0/3608869999 shutdown_connections 2023-12-19T18:16:19.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.509+0000 7f8699737640 1 -- 172.21.15.27:0/3608869999 wait complete. 2023-12-19T18:16:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.510+0000 7f8699737640 1 Processor -- start 2023-12-19T18:16:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.511+0000 7f8699737640 1 -- start start 2023-12-19T18:16:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.511+0000 7f8699737640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86940aef20 0x7f869414e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.511+0000 7f8699737640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86940af360 0x7f86941f0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.511+0000 7f8699737640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f869415ac50 0x7f869406bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.511+0000 7f8699737640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869414e8c0 con 0x7f86940aef20 2023-12-19T18:16:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.512+0000 7f8699737640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f869414eba0 con 0x7f869415ac50 2023-12-19T18:16:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.512+0000 7f8699737640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f869414ee80 con 0x7f86940af360 2023-12-19T18:16:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.512+0000 7f8693fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86940aef20 0x7f869414e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.512+0000 7f8693fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86940aef20 0x7f869414e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59182/0 (socket says 172.21.15.27:59182) 2023-12-19T18:16:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.512+0000 7f8693fff640 1 -- 172.21.15.27:0/2191087036 learned_addr learned my addr 172.21.15.27:0/2191087036 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:19.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.512+0000 7f86937fe640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86940af360 0x7f86941f0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:19.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.512+0000 7f8693fff640 1 -- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86940af360 msgr2=0x7f86941f0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:19.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.512+0000 7f8693fff640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86940af360 0x7f86941f0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.512+0000 7f8698f36640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f869415ac50 0x7f869406bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:19.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.512+0000 7f8693fff640 1 -- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f869415ac50 msgr2=0x7f869406bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:19.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.512+0000 7f8693fff640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f869415ac50 0x7f869406bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.512+0000 7f8693fff640 1 -- 172.21.15.27:0/2191087036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86880094d0 con 0x7f86940aef20 2023-12-19T18:16:19.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.512+0000 7f86937fe640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86940af360 0x7f86941f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:19.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.513+0000 7f8693fff640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86940aef20 0x7f869414e1c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8684007ef0 tx=0x7f868400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:19.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.513+0000 7f8690ff9640 1 -- 172.21.15.27:0/2191087036 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684014620 con 0x7f86940aef20 2023-12-19T18:16:19.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.513+0000 7f8690ff9640 1 -- 172.21.15.27:0/2191087036 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8684014780 con 0x7f86940aef20 2023-12-19T18:16:19.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.513+0000 7f8690ff9640 1 -- 172.21.15.27:0/2191087036 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8684014b30 con 0x7f86940aef20 2023-12-19T18:16:19.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.513+0000 7f8699737640 1 -- 172.21.15.27:0/2191087036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f869414f160 con 0x7f86940aef20 2023-12-19T18:16:19.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.513+0000 7f8699737640 1 -- 172.21.15.27:0/2191087036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86941f1460 con 0x7f86940aef20 2023-12-19T18:16:19.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.514+0000 7f8699737640 1 -- 172.21.15.27:0/2191087036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f869406c9b0 con 0x7f86940aef20 2023-12-19T18:16:19.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.515+0000 7f8690ff9640 1 -- 172.21.15.27:0/2191087036 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8684014c90 con 0x7f86940aef20 2023-12-19T18:16:19.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.516+0000 7f8690ff9640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f868006a430 0x7f868006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:19.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.516+0000 7f8690ff9640 1 -- 172.21.15.27:0/2191087036 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8684088fa0 con 0x7f86940aef20 2023-12-19T18:16:19.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.517+0000 7f86937fe640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f868006a430 0x7f868006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:19.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.518+0000 7f86937fe640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f868006a430 0x7f868006c8f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8688035000 tx=0x7f8688008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:19.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.520+0000 7f8690ff9640 1 -- 172.21.15.27:0/2191087036 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f86840bbca0 con 0x7f86940aef20 2023-12-19T18:16:19.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.784+0000 7f8699737640 1 -- 172.21.15.27:0/2191087036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f869415b380 con 0x7f86940aef20 2023-12-19T18:16:19.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.791+0000 7f8690ff9640 1 -- 172.21.15.27:0/2191087036 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f86840548e0 con 0x7f86940aef20 2023-12-19T18:16:19.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.793+0000 7f8699737640 1 -- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f868006a430 msgr2=0x7f868006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.793+0000 7f8699737640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f868006a430 0x7f868006c8f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f8688035000 tx=0x7f8688008030 comp rx=0 tx=0).stop 2023-12-19T18:16:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.793+0000 7f8699737640 1 -- 172.21.15.27:0/2191087036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86940aef20 msgr2=0x7f869414e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.793+0000 7f8699737640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86940aef20 0x7f869414e1c0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8684007ef0 tx=0x7f868400e4b0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.793+0000 7f8699737640 1 -- 172.21.15.27:0/2191087036 shutdown_connections 2023-12-19T18:16:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.793+0000 7f8699737640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f868006a430 0x7f868006c8f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.793+0000 7f8699737640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f869415ac50 0x7f869406bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.793+0000 7f8699737640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86940af360 0x7f86941f0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.793+0000 7f8699737640 1 --2- 172.21.15.27:0/2191087036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86940aef20 0x7f869414e1c0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.793+0000 7f8699737640 1 -- 172.21.15.27:0/2191087036 >> 172.21.15.27:0/2191087036 conn(0x7f869406a7d0 msgr2=0x7f8694070d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.794+0000 7f8699737640 1 -- 172.21.15.27:0/2191087036 shutdown_connections 2023-12-19T18:16:19.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:19.794+0000 7f8699737640 1 -- 172.21.15.27:0/2191087036 wait complete. 2023-12-19T18:16:19.815 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-12-19T18:16:20.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.040+0000 7f530bfff640 1 Processor -- start 2023-12-19T18:16:20.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.040+0000 7f530bfff640 1 -- start start 2023-12-19T18:16:20.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.042+0000 7f530bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f530c15b9e0 0x7f530c0c9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.042+0000 7f530bfff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c06b310 con 0x7f530c06f8e0 2023-12-19T18:16:20.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.042+0000 7f530bfff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c0c9950 con 0x7f530c162150 2023-12-19T18:16:20.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.042+0000 7f530bfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f530c162570 con 0x7f530c15b9e0 2023-12-19T18:16:20.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.043+0000 7f5303fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f530c06f8e0 0x7f530c161a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60132/0 (socket says 172.21.15.27:60132) 2023-12-19T18:16:20.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.043+0000 7f5303fff640 1 -- 172.21.15.27:0/820415351 learned_addr learned my addr 172.21.15.27:0/820415351 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:20.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.043+0000 7f530affd640 1 --2- 172.21.15.27:0/820415351 >> v2:172.21.15.192:3300/0 conn(0x7f530c15b9e0 0x7f530c0c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.043+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2072213452 0 0) 0x7f530c06b310 con 0x7f530c06f8e0 2023-12-19T18:16:20.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f8003520 con 0x7f530c06f8e0 2023-12-19T18:16:20.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3836226536 0 0) 0x7f52f8003520 con 0x7f530c06f8e0 2023-12-19T18:16:20.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 >> v2:172.21.15.192:3300/0 conn(0x7f530c15b9e0 msgr2=0x7f530c0c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:20.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 --2- 172.21.15.27:0/820415351 >> v2:172.21.15.192:3300/0 conn(0x7f530c15b9e0 0x7f530c0c9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 >> v1:172.21.15.192:6789/0 conn(0x7f530c162150 legacy=0x7f530c06c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c162790 con 0x7f530c06f8e0 2023-12-19T18:16:20.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f52f4001670 con 0x7f530c06f8e0 2023-12-19T18:16:20.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 >> v1:172.21.15.27:6789/0 conn(0x7f530c06f8e0 legacy=0x7f530c161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 --2- 172.21.15.27:0/820415351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52f80039c0 0x7f52f8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 --2- 172.21.15.27:0/820415351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80047c0 0x7f52f8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 --2- 172.21.15.27:0/820415351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f8005180 0x7f52f800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c162570 con 0x7f52f80047c0 2023-12-19T18:16:20.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f800c080 con 0x7f52f8005180 2023-12-19T18:16:20.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.044+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f800c240 con 0x7f52f80039c0 2023-12-19T18:16:20.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.045+0000 7f5303fff640 1 --2- 172.21.15.27:0/820415351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80047c0 0x7f52f8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.045+0000 7f530affd640 1 --2- 172.21.15.27:0/820415351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52f80039c0 0x7f52f8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.045+0000 7f530b7fe640 1 --2- 172.21.15.27:0/820415351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f8005180 0x7f52f800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.045+0000 7f5303fff640 1 -- 172.21.15.27:0/820415351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52f80039c0 msgr2=0x7f52f8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:20.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.045+0000 7f5303fff640 1 --2- 172.21.15.27:0/820415351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52f80039c0 0x7f52f8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.045+0000 7f5303fff640 1 -- 172.21.15.27:0/820415351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f8005180 msgr2=0x7f52f800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:20.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.045+0000 7f5303fff640 1 --2- 172.21.15.27:0/820415351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f8005180 0x7f52f800b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.045+0000 7f5303fff640 1 -- 172.21.15.27:0/820415351 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f8003520 con 0x7f52f80047c0 2023-12-19T18:16:20.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.045+0000 7f530b7fe640 1 --2- 172.21.15.27:0/820415351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f8005180 0x7f52f800b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:20.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.045+0000 7f530b7fe640 1 -- 172.21.15.27:0/820415351 reap_dead start 2023-12-19T18:16:20.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.046+0000 7f5303fff640 1 --2- 172.21.15.27:0/820415351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80047c0 0x7f52f8004c40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f52f400c0e0 tx=0x7f52f80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29de584ee176a215 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:20.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.047+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f4005ad0 con 0x7f52f80047c0 2023-12-19T18:16:20.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.047+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f4005c50 con 0x7f52f80047c0 2023-12-19T18:16:20.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.047+0000 7f530a7fc640 1 -- 172.21.15.27:0/820415351 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f402c810 con 0x7f52f80047c0 2023-12-19T18:16:20.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.047+0000 7f530bfff640 1 -- 172.21.15.27:0/820415351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80047c0 msgr2=0x7f52f8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.047+0000 7f530bfff640 1 --2- 172.21.15.27:0/820415351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80047c0 0x7f52f8004c40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f52f400c0e0 tx=0x7f52f80057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.047+0000 7f530bfff640 1 -- 172.21.15.27:0/820415351 shutdown_connections 2023-12-19T18:16:20.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.047+0000 7f530bfff640 1 --2- 172.21.15.27:0/820415351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80047c0 0x7f52f8004c40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.047+0000 7f530bfff640 1 -- 172.21.15.27:0/820415351 >> 172.21.15.27:0/820415351 conn(0x7f530c06a7d0 msgr2=0x7f530c0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:20.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.047+0000 7f530bfff640 1 -- 172.21.15.27:0/820415351 shutdown_connections 2023-12-19T18:16:20.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.047+0000 7f530bfff640 1 -- 172.21.15.27:0/820415351 wait complete. 2023-12-19T18:16:20.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.049+0000 7f530bfff640 1 Processor -- start 2023-12-19T18:16:20.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.049+0000 7f530bfff640 1 -- start start 2023-12-19T18:16:20.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.050+0000 7f530bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80047c0 0x7f530c0a9b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.050+0000 7f530bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c0a84b0 0x7f530c0b2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.050+0000 7f530bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f530c0b3ec0 0x7f530c14b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.050+0000 7f530bfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0a96f0 con 0x7f52f80047c0 2023-12-19T18:16:20.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.051+0000 7f530bfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0a8a70 con 0x7f530c0a84b0 2023-12-19T18:16:20.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.051+0000 7f530bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0a8d50 con 0x7f530c0b3ec0 2023-12-19T18:16:20.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.051+0000 7f530affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80047c0 0x7f530c0a9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.051+0000 7f530affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80047c0 0x7f530c0a9b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59200/0 (socket says 172.21.15.27:59200) 2023-12-19T18:16:20.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.051+0000 7f530affd640 1 -- 172.21.15.27:0/3968720866 learned_addr learned my addr 172.21.15.27:0/3968720866 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.051+0000 7f5303fff640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c0a84b0 0x7f530c0b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.051+0000 7f530b7fe640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f530c0b3ec0 0x7f530c14b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.051+0000 7f530affd640 1 -- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f530c0b3ec0 msgr2=0x7f530c14b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.051+0000 7f530affd640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f530c0b3ec0 0x7f530c14b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.051+0000 7f530affd640 1 -- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c0a84b0 msgr2=0x7f530c0b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.051+0000 7f530affd640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c0a84b0 0x7f530c0b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.051+0000 7f530affd640 1 -- 172.21.15.27:0/3968720866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c0a9030 con 0x7f52f80047c0 2023-12-19T18:16:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.052+0000 7f530affd640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80047c0 0x7f530c0a9b10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0116d0 tx=0x7f52fc011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.052+0000 7f5308ff9640 1 -- 172.21.15.27:0/3968720866 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc00f9c0 con 0x7f52f80047c0 2023-12-19T18:16:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.052+0000 7f5308ff9640 1 -- 172.21.15.27:0/3968720866 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc00fb20 con 0x7f52f80047c0 2023-12-19T18:16:20.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.052+0000 7f5308ff9640 1 -- 172.21.15.27:0/3968720866 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc00fe90 con 0x7f52f80047c0 2023-12-19T18:16:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.052+0000 7f530bfff640 1 -- 172.21.15.27:0/3968720866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530c0a9310 con 0x7f52f80047c0 2023-12-19T18:16:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.052+0000 7f530bfff640 1 -- 172.21.15.27:0/3968720866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530c14bdd0 con 0x7f52f80047c0 2023-12-19T18:16:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.053+0000 7f530bfff640 1 -- 172.21.15.27:0/3968720866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530c04e840 con 0x7f52f80047c0 2023-12-19T18:16:20.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.055+0000 7f5308ff9640 1 -- 172.21.15.27:0/3968720866 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f52fc015460 con 0x7f52f80047c0 2023-12-19T18:16:20.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.055+0000 7f5308ff9640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f52ec06a430 0x7f52ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.056+0000 7f5308ff9640 1 -- 172.21.15.27:0/3968720866 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f52fc088bf0 con 0x7f52f80047c0 2023-12-19T18:16:20.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.056+0000 7f5303fff640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f52ec06a430 0x7f52ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.057+0000 7f5303fff640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f52ec06a430 0x7f52ec06c8f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f52f4034000 tx=0x7f52f4009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:20.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.060+0000 7f5308ff9640 1 -- 172.21.15.27:0/3968720866 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f52fc054490 con 0x7f52f80047c0 2023-12-19T18:16:20.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.325+0000 7f530bfff640 1 -- 172.21.15.27:0/3968720866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f530c065f60 con 0x7f52f80047c0 2023-12-19T18:16:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.333+0000 7f5308ff9640 1 -- 172.21.15.27:0/3968720866 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7f52fc057ab0 con 0x7f52f80047c0 2023-12-19T18:16:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.336+0000 7f530bfff640 1 -- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f52ec06a430 msgr2=0x7f52ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.336+0000 7f530bfff640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f52ec06a430 0x7f52ec06c8f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f52f4034000 tx=0x7f52f4009030 comp rx=0 tx=0).stop 2023-12-19T18:16:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.336+0000 7f530bfff640 1 -- 172.21.15.27:0/3968720866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80047c0 msgr2=0x7f530c0a9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.336+0000 7f530bfff640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80047c0 0x7f530c0a9b10 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f52fc0116d0 tx=0x7f52fc011b90 comp rx=0 tx=0).stop 2023-12-19T18:16:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.336+0000 7f530bfff640 1 -- 172.21.15.27:0/3968720866 shutdown_connections 2023-12-19T18:16:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.336+0000 7f530bfff640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f52ec06a430 0x7f52ec06c8f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.336+0000 7f530bfff640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f530c0b3ec0 0x7f530c14b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.336+0000 7f530bfff640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f530c0a84b0 0x7f530c0b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.336+0000 7f530bfff640 1 --2- 172.21.15.27:0/3968720866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80047c0 0x7f530c0a9b10 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.336+0000 7f530bfff640 1 -- 172.21.15.27:0/3968720866 >> 172.21.15.27:0/3968720866 conn(0x7f530c06a7d0 msgr2=0x7f530c06fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.336+0000 7f530bfff640 1 -- 172.21.15.27:0/3968720866 shutdown_connections 2023-12-19T18:16:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.336+0000 7f530bfff640 1 -- 172.21.15.27:0/3968720866 wait complete. 2023-12-19T18:16:20.354 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-12-19T18:16:20.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.588+0000 7f5f3ffff640 1 Processor -- start 2023-12-19T18:16:20.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.588+0000 7f5f3ffff640 1 -- start start 2023-12-19T18:16:20.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.590+0000 7f5f3ffff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5f4015aab0 0x7f5f40070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.590+0000 7f5f3ffff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4015aed0 con 0x7f5f40154170 2023-12-19T18:16:20.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.590+0000 7f5f3ffff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f4015b110 con 0x7f5f40070aa0 2023-12-19T18:16:20.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.590+0000 7f5f3ffff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f40071470 con 0x7f5f4015aab0 2023-12-19T18:16:20.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.591+0000 7f5f3effd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5f40154170 0x7f5f40070170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60146/0 (socket says 172.21.15.27:60146) 2023-12-19T18:16:20.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.591+0000 7f5f3effd640 1 -- 172.21.15.27:0/1135314800 learned_addr learned my addr 172.21.15.27:0/1135314800 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:20.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.591+0000 7f5f3f7fe640 1 --2- 172.21.15.27:0/1135314800 >> v2:172.21.15.192:3300/0 conn(0x7f5f4015aab0 0x7f5f40070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.591+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969944616 0 0) 0x7f5f30001110 con 0x7f5f40154170 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.591+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f2c005730 con 0x7f5f40154170 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.591+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4252438431 0 0) 0x7f5f2c005730 con 0x7f5f40154170 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.591+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 >> v2:172.21.15.192:3300/0 conn(0x7f5f4015aab0 msgr2=0x7f5f40070f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.591+0000 7f5f3e7fc640 1 --2- 172.21.15.27:0/1135314800 >> v2:172.21.15.192:3300/0 conn(0x7f5f4015aab0 0x7f5f40070f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.591+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 >> v1:172.21.15.192:6789/0 conn(0x7f5f40070aa0 legacy=0x7f5f4015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.591+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f4006fce0 con 0x7f5f40154170 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.591+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f5f30001800 con 0x7f5f40154170 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 >> v1:172.21.15.27:6789/0 conn(0x7f5f40154170 legacy=0x7f5f40070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3e7fc640 1 --2- 172.21.15.27:0/1135314800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f2c005b80 0x7f5f2c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3e7fc640 1 --2- 172.21.15.27:0/1135314800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f2c0069d0 0x7f5f2c006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3e7fc640 1 --2- 172.21.15.27:0/1135314800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f2c007390 0x7f5f2c00db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f40071470 con 0x7f5f2c007390 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c00e330 con 0x7f5f2c005b80 2023-12-19T18:16:20.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f2c00e4f0 con 0x7f5f2c0069d0 2023-12-19T18:16:20.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3f7fe640 1 --2- 172.21.15.27:0/1135314800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f2c007390 0x7f5f2c00db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3effd640 1 --2- 172.21.15.27:0/1135314800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f2c005b80 0x7f5f2c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3f7fe640 1 -- 172.21.15.27:0/1135314800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f2c0069d0 msgr2=0x7f5f2c006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:20.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f37fff640 1 --2- 172.21.15.27:0/1135314800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f2c0069d0 0x7f5f2c006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3f7fe640 1 --2- 172.21.15.27:0/1135314800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f2c0069d0 0x7f5f2c006e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3f7fe640 1 -- 172.21.15.27:0/1135314800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f2c005b80 msgr2=0x7f5f2c005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:20.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3f7fe640 1 --2- 172.21.15.27:0/1135314800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f2c005b80 0x7f5f2c005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.592+0000 7f5f3f7fe640 1 -- 172.21.15.27:0/1135314800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f2c005730 con 0x7f5f2c007390 2023-12-19T18:16:20.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.593+0000 7f5f3f7fe640 1 -- 172.21.15.27:0/1135314800 reap_dead start 2023-12-19T18:16:20.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.594+0000 7f5f3f7fe640 1 --2- 172.21.15.27:0/1135314800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f2c007390 0x7f5f2c00db90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009fc0 tx=0x7f5f2c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=423b14a5e0a5f2df server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:20.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.594+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3802cc60 con 0x7f5f2c007390 2023-12-19T18:16:20.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.594+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3800f040 con 0x7f5f2c007390 2023-12-19T18:16:20.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.594+0000 7f5f3e7fc640 1 -- 172.21.15.27:0/1135314800 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38003990 con 0x7f5f2c007390 2023-12-19T18:16:20.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.594+0000 7f5f3ffff640 1 -- 172.21.15.27:0/1135314800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f2c007390 msgr2=0x7f5f2c00db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.594+0000 7f5f3ffff640 1 --2- 172.21.15.27:0/1135314800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f2c007390 0x7f5f2c00db90 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f5f38009fc0 tx=0x7f5f2c007a00 comp rx=0 tx=0).stop 2023-12-19T18:16:20.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.595+0000 7f5f3ffff640 1 -- 172.21.15.27:0/1135314800 shutdown_connections 2023-12-19T18:16:20.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.595+0000 7f5f3ffff640 1 --2- 172.21.15.27:0/1135314800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f2c007390 0x7f5f2c00db90 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.595+0000 7f5f3ffff640 1 -- 172.21.15.27:0/1135314800 >> 172.21.15.27:0/1135314800 conn(0x7f5f4006a7d0 msgr2=0x7f5f400ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:20.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.595+0000 7f5f3ffff640 1 -- 172.21.15.27:0/1135314800 shutdown_connections 2023-12-19T18:16:20.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.595+0000 7f5f3ffff640 1 -- 172.21.15.27:0/1135314800 wait complete. 2023-12-19T18:16:20.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.597+0000 7f5f3ffff640 1 Processor -- start 2023-12-19T18:16:20.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.597+0000 7f5f3ffff640 1 -- start start 2023-12-19T18:16:20.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.598+0000 7f5f3ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f2c007390 0x7f5f401598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.598+0000 7f5f3ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f401f5c20 0x7f5f40152510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.598+0000 7f5f3ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f400b1790 0x7f5f400b1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.598+0000 7f5f3ffff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f40152a50 con 0x7f5f400b1790 2023-12-19T18:16:20.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.598+0000 7f5f3ffff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f400b1020 con 0x7f5f401f5c20 2023-12-19T18:16:20.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.598+0000 7f5f3ffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f401f6390 con 0x7f5f2c007390 2023-12-19T18:16:20.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.598+0000 7f5f3f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f400b1790 0x7f5f400b1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.598+0000 7f5f3f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f400b1790 0x7f5f400b1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59208/0 (socket says 172.21.15.27:59208) 2023-12-19T18:16:20.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.598+0000 7f5f3f7fe640 1 -- 172.21.15.27:0/3630828969 learned_addr learned my addr 172.21.15.27:0/3630828969 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:20.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.598+0000 7f5f3effd640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f2c007390 0x7f5f401598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.598+0000 7f5f37fff640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f401f5c20 0x7f5f40152510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.599+0000 7f5f3f7fe640 1 -- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f2c007390 msgr2=0x7f5f401598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.599+0000 7f5f3f7fe640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f2c007390 0x7f5f401598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.599+0000 7f5f3f7fe640 1 -- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f401f5c20 msgr2=0x7f5f40152510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.599+0000 7f5f3f7fe640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f401f5c20 0x7f5f40152510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.599+0000 7f5f3f7fe640 1 -- 172.21.15.27:0/3630828969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f401f6670 con 0x7f5f400b1790 2023-12-19T18:16:20.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.599+0000 7f5f3f7fe640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f400b1790 0x7f5f400b1c10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5f3802c6d0 tx=0x7f5f38009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:20.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.599+0000 7f5f3cff9640 1 -- 172.21.15.27:0/3630828969 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38003da0 con 0x7f5f400b1790 2023-12-19T18:16:20.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.599+0000 7f5f3cff9640 1 -- 172.21.15.27:0/3630828969 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f38035950 con 0x7f5f400b1790 2023-12-19T18:16:20.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.599+0000 7f5f3cff9640 1 -- 172.21.15.27:0/3630828969 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38035d40 con 0x7f5f400b1790 2023-12-19T18:16:20.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.599+0000 7f5f3ffff640 1 -- 172.21.15.27:0/3630828969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f401f6950 con 0x7f5f400b1790 2023-12-19T18:16:20.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.600+0000 7f5f3ffff640 1 -- 172.21.15.27:0/3630828969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f401f60a0 con 0x7f5f400b1790 2023-12-19T18:16:20.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.601+0000 7f5f3ffff640 1 -- 172.21.15.27:0/3630828969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f400b14b0 con 0x7f5f400b1790 2023-12-19T18:16:20.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.602+0000 7f5f3cff9640 1 -- 172.21.15.27:0/3630828969 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5f38004030 con 0x7f5f400b1790 2023-12-19T18:16:20.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.603+0000 7f5f3cff9640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5f2006a460 0x7f5f2006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:20.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.603+0000 7f5f3cff9640 1 -- 172.21.15.27:0/3630828969 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5f380ab1d0 con 0x7f5f400b1790 2023-12-19T18:16:20.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.603+0000 7f5f3effd640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5f2006a460 0x7f5f2006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:20.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.604+0000 7f5f3effd640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5f2006a460 0x7f5f2006c920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5f30001270 tx=0x7f5f30009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:20.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.607+0000 7f5f3cff9640 1 -- 172.21.15.27:0/3630828969 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5f400b14b0 con 0x7f5f400b1790 2023-12-19T18:16:20.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.869+0000 7f5f3ffff640 1 -- 172.21.15.27:0/3630828969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f5f400b14b0 con 0x7f5f400b1790 2023-12-19T18:16:20.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.875+0000 7f5f3cff9640 1 -- 172.21.15.27:0/3630828969 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f5f38076c20 con 0x7f5f400b1790 2023-12-19T18:16:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.878+0000 7f5f3ffff640 1 -- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5f2006a460 msgr2=0x7f5f2006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.878+0000 7f5f3ffff640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5f2006a460 0x7f5f2006c920 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5f30001270 tx=0x7f5f30009040 comp rx=0 tx=0).stop 2023-12-19T18:16:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.878+0000 7f5f3ffff640 1 -- 172.21.15.27:0/3630828969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f400b1790 msgr2=0x7f5f400b1c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.878+0000 7f5f3ffff640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f400b1790 0x7f5f400b1c10 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f5f3802c6d0 tx=0x7f5f38009750 comp rx=0 tx=0).stop 2023-12-19T18:16:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.878+0000 7f5f3ffff640 1 -- 172.21.15.27:0/3630828969 shutdown_connections 2023-12-19T18:16:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.878+0000 7f5f3ffff640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5f2006a460 0x7f5f2006c920 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.878+0000 7f5f3ffff640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f400b1790 0x7f5f400b1c10 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.878+0000 7f5f3ffff640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5f401f5c20 0x7f5f40152510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.878+0000 7f5f3ffff640 1 --2- 172.21.15.27:0/3630828969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5f2c007390 0x7f5f401598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.878+0000 7f5f3ffff640 1 -- 172.21.15.27:0/3630828969 >> 172.21.15.27:0/3630828969 conn(0x7f5f4006a7d0 msgr2=0x7f5f4015d540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.878+0000 7f5f3ffff640 1 -- 172.21.15.27:0/3630828969 shutdown_connections 2023-12-19T18:16:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:20.878+0000 7f5f3ffff640 1 -- 172.21.15.27:0/3630828969 wait complete. 2023-12-19T18:16:20.896 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-12-19T18:16:21.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.124+0000 7f128b59e640 1 Processor -- start 2023-12-19T18:16:21.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.124+0000 7f128b59e640 1 -- start start 2023-12-19T18:16:21.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.125+0000 7f128b59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f128c15b3a0 0x7f128c06c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.126+0000 7f128b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f128c0cad90 0x7f128c06b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.126+0000 7f128b59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f128c0cabc0 con 0x7f128c15afc0 2023-12-19T18:16:21.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.126+0000 7f128b59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f128c0c2ba0 con 0x7f128c15b3a0 2023-12-19T18:16:21.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.126+0000 7f128b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128c0ca040 con 0x7f128c0cad90 2023-12-19T18:16:21.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.126+0000 7f128a59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f128c15afc0 0x7f128c0c2490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60160/0 (socket says 172.21.15.27:60160) 2023-12-19T18:16:21.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.126+0000 7f128a59c640 1 -- 172.21.15.27:0/2177884333 learned_addr learned my addr 172.21.15.27:0/2177884333 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.126+0000 7f128ad9d640 1 --2- 172.21.15.27:0/2177884333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f128c0cad90 0x7f128c06b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.126+0000 7f1283fff640 1 --2- 172.21.15.27:0/2177884333 >> v2:172.21.15.192:3300/0 conn(0x7f128c15b3a0 0x7f128c06c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.126+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4063027588 0 0) 0x7f128c0cabc0 con 0x7f128c15afc0 2023-12-19T18:16:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.126+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1278003520 con 0x7f128c15afc0 2023-12-19T18:16:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1474904852 0 0) 0x7f1278003520 con 0x7f128c15afc0 2023-12-19T18:16:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f128c0cad90 msgr2=0x7f128c06b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 --2- 172.21.15.27:0/2177884333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f128c0cad90 0x7f128c06b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 >> v2:172.21.15.192:3300/0 conn(0x7f128c15b3a0 msgr2=0x7f128c06c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 --2- 172.21.15.27:0/2177884333 >> v2:172.21.15.192:3300/0 conn(0x7f128c15b3a0 0x7f128c06c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c0ca320 con 0x7f128c15afc0 2023-12-19T18:16:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f127c001670 con 0x7f128c15afc0 2023-12-19T18:16:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 >> v1:172.21.15.27:6789/0 conn(0x7f128c15afc0 legacy=0x7f128c0c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 --2- 172.21.15.27:0/2177884333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f128c0cad90 0x7f1278003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 --2- 172.21.15.27:0/2177884333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1278003f40 0x7f12780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 --2- 172.21.15.27:0/2177884333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1278004d00 0x7f1278005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c0c2ba0 con 0x7f1278003f40 2023-12-19T18:16:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c0ca040 con 0x7f1278004d00 2023-12-19T18:16:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f127800ba30 con 0x7f128c0cad90 2023-12-19T18:16:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.127+0000 7f1283fff640 1 --2- 172.21.15.27:0/2177884333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1278003f40 0x7f12780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.128+0000 7f1283fff640 1 -- 172.21.15.27:0/2177884333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f128c0cad90 msgr2=0x7f1278003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.128+0000 7f1283fff640 1 --2- 172.21.15.27:0/2177884333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f128c0cad90 0x7f1278003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.128+0000 7f1283fff640 1 -- 172.21.15.27:0/2177884333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1278004d00 msgr2=0x7f1278005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:21.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.128+0000 7f1283fff640 1 --2- 172.21.15.27:0/2177884333 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1278004d00 0x7f1278005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.128+0000 7f1283fff640 1 -- 172.21.15.27:0/2177884333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1278003520 con 0x7f1278003f40 2023-12-19T18:16:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.129+0000 7f1283fff640 1 --2- 172.21.15.27:0/2177884333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1278003f40 0x7f12780047c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1274009ac0 tx=0x7f12780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc6277a377c7eac server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.129+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127402c710 con 0x7f1278003f40 2023-12-19T18:16:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.129+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127402c870 con 0x7f1278003f40 2023-12-19T18:16:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.129+0000 7f1289d9b640 1 -- 172.21.15.27:0/2177884333 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127402cc00 con 0x7f1278003f40 2023-12-19T18:16:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.130+0000 7f128b59e640 1 -- 172.21.15.27:0/2177884333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1278003f40 msgr2=0x7f12780047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.130+0000 7f128b59e640 1 --2- 172.21.15.27:0/2177884333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1278003f40 0x7f12780047c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f1274009ac0 tx=0x7f12780057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.130+0000 7f128ad9d640 1 -- 172.21.15.27:0/2177884333 reap_dead start 2023-12-19T18:16:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.130+0000 7f128b59e640 1 -- 172.21.15.27:0/2177884333 shutdown_connections 2023-12-19T18:16:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.130+0000 7f128b59e640 1 -- 172.21.15.27:0/2177884333 >> 172.21.15.27:0/2177884333 conn(0x7f128c0cb450 msgr2=0x7f128c0cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.130+0000 7f128b59e640 1 -- 172.21.15.27:0/2177884333 shutdown_connections 2023-12-19T18:16:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.130+0000 7f128b59e640 1 -- 172.21.15.27:0/2177884333 wait complete. 2023-12-19T18:16:21.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.132+0000 7f128b59e640 1 Processor -- start 2023-12-19T18:16:21.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.132+0000 7f128b59e640 1 -- start start 2023-12-19T18:16:21.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.133+0000 7f128b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1278003f40 0x7f128c157120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.133+0000 7f128b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f128c0b3d50 0x7f128c0c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.133+0000 7f128b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f128c14c560 0x7f128c14c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.133+0000 7f128b59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c14d460 con 0x7f1278003f40 2023-12-19T18:16:21.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.133+0000 7f128b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f128c06f350 con 0x7f128c14c560 2023-12-19T18:16:21.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.133+0000 7f128b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f128c06fed0 con 0x7f128c0b3d50 2023-12-19T18:16:21.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.133+0000 7f128a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1278003f40 0x7f128c157120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:21.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.133+0000 7f128a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1278003f40 0x7f128c157120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59224/0 (socket says 172.21.15.27:59224) 2023-12-19T18:16:21.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.133+0000 7f128a59c640 1 -- 172.21.15.27:0/3230129476 learned_addr learned my addr 172.21.15.27:0/3230129476 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:21.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.134+0000 7f1283fff640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f128c0b3d50 0x7f128c0c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:21.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.134+0000 7f128a59c640 1 -- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f128c0b3d50 msgr2=0x7f128c0c4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:21.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.134+0000 7f128a59c640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f128c0b3d50 0x7f128c0c4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.134+0000 7f128a59c640 1 -- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f128c14c560 msgr2=0x7f128c14c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.134+0000 7f128ad9d640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f128c14c560 0x7f128c14c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.134+0000 7f128a59c640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f128c14c560 0x7f128c14c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.134+0000 7f128a59c640 1 -- 172.21.15.27:0/3230129476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f128c14cf20 con 0x7f1278003f40 2023-12-19T18:16:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.134+0000 7f128ad9d640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f128c14c560 0x7f128c14c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.134+0000 7f128a59c640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1278003f40 0x7f128c157120 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f127c00c7d0 tx=0x7f127c00cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.134+0000 7f1282ffd640 1 -- 172.21.15.27:0/3230129476 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c0074e0 con 0x7f1278003f40 2023-12-19T18:16:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.134+0000 7f1282ffd640 1 -- 172.21.15.27:0/3230129476 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f127c007640 con 0x7f1278003f40 2023-12-19T18:16:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.134+0000 7f1282ffd640 1 -- 172.21.15.27:0/3230129476 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f127c004a20 con 0x7f1278003f40 2023-12-19T18:16:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.134+0000 7f128b59e640 1 -- 172.21.15.27:0/3230129476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f128c06f5a0 con 0x7f1278003f40 2023-12-19T18:16:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.135+0000 7f128b59e640 1 -- 172.21.15.27:0/3230129476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f128c06fb60 con 0x7f1278003f40 2023-12-19T18:16:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.136+0000 7f128b59e640 1 -- 172.21.15.27:0/3230129476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f128c04e840 con 0x7f1278003f40 2023-12-19T18:16:21.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.137+0000 7f1282ffd640 1 -- 172.21.15.27:0/3230129476 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f127c004b80 con 0x7f1278003f40 2023-12-19T18:16:21.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.138+0000 7f1282ffd640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f126c06a430 0x7f126c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.138+0000 7f1282ffd640 1 -- 172.21.15.27:0/3230129476 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f127c087af0 con 0x7f1278003f40 2023-12-19T18:16:21.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.139+0000 7f1283fff640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f126c06a430 0x7f126c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:21.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.140+0000 7f1283fff640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f126c06a430 0x7f126c06c8f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1274036000 tx=0x7f1274030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:21.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.142+0000 7f1282ffd640 1 -- 172.21.15.27:0/3230129476 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f127c053390 con 0x7f1278003f40 2023-12-19T18:16:21.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.403+0000 7f128b59e640 1 -- 172.21.15.27:0/3230129476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f128c0ce350 con 0x7f1278003f40 2023-12-19T18:16:21.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.411+0000 7f1282ffd640 1 -- 172.21.15.27:0/3230129476 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v9) v1 ==== 97+0+0 (secure 0 0 0) 0x7f127c0180a0 con 0x7f1278003f40 2023-12-19T18:16:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.414+0000 7f128b59e640 1 -- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f126c06a430 msgr2=0x7f126c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.414+0000 7f128b59e640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f126c06a430 0x7f126c06c8f0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f1274036000 tx=0x7f1274030000 comp rx=0 tx=0).stop 2023-12-19T18:16:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.414+0000 7f128b59e640 1 -- 172.21.15.27:0/3230129476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1278003f40 msgr2=0x7f128c157120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:21.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.414+0000 7f128b59e640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1278003f40 0x7f128c157120 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f127c00c7d0 tx=0x7f127c00cc90 comp rx=0 tx=0).stop 2023-12-19T18:16:21.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.414+0000 7f128b59e640 1 -- 172.21.15.27:0/3230129476 shutdown_connections 2023-12-19T18:16:21.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.414+0000 7f128b59e640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f126c06a430 0x7f126c06c8f0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.414+0000 7f128b59e640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f128c14c560 0x7f128c14c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.414+0000 7f128b59e640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f128c0b3d50 0x7f128c0c4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.414+0000 7f128b59e640 1 --2- 172.21.15.27:0/3230129476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1278003f40 0x7f128c157120 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.414+0000 7f128b59e640 1 -- 172.21.15.27:0/3230129476 >> 172.21.15.27:0/3230129476 conn(0x7f128c0cb450 msgr2=0x7f128c0c3560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:21.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.414+0000 7f128b59e640 1 -- 172.21.15.27:0/3230129476 shutdown_connections 2023-12-19T18:16:21.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.414+0000 7f128b59e640 1 -- 172.21.15.27:0/3230129476 wait complete. 2023-12-19T18:16:21.434 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-12-19T18:16:21.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.666+0000 7f40f2a22640 1 Processor -- start 2023-12-19T18:16:21.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.666+0000 7f40f2a22640 1 -- start start 2023-12-19T18:16:21.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.667+0000 7f40f2a22640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f40ec15ac50 0x7f40ec070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.667+0000 7f40f2a22640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ec0aef20 0x7f40ec06b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.667+0000 7f40f2a22640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ec0adf10 con 0x7f40ec0af360 2023-12-19T18:16:21.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.667+0000 7f40f2a22640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ec0adac0 con 0x7f40ec15ac50 2023-12-19T18:16:21.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.667+0000 7f40f2a22640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec0ae160 con 0x7f40ec0aef20 2023-12-19T18:16:21.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.668+0000 7f40f1a20640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f40ec15ac50 0x7f40ec070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:21.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.668+0000 7f40f1a20640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f40ec15ac50 0x7f40ec070fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59236/0 (socket says 172.21.15.27:59236) 2023-12-19T18:16:21.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.668+0000 7f40f1a20640 1 -- 172.21.15.27:0/2375005841 learned_addr learned my addr 172.21.15.27:0/2375005841 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:21.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.668+0000 7f40f121f640 1 --2- 172.21.15.27:0/2375005841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ec0aef20 0x7f40ec06b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:21.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.668+0000 7f40f1a20640 1 -- 172.21.15.27:0/2375005841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ec0aef20 msgr2=0x7f40ec06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:21.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.668+0000 7f40f1a20640 1 --2- 172.21.15.27:0/2375005841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ec0aef20 0x7f40ec06b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.668+0000 7f40f1a20640 1 -- 172.21.15.27:0/2375005841 >> v1:172.21.15.192:6789/0 conn(0x7f40ec0af360 legacy=0x7f40ec06b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:21.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.668+0000 7f40f1a20640 1 -- 172.21.15.27:0/2375005841 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ec0ae440 con 0x7f40ec15ac50 2023-12-19T18:16:21.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.670+0000 7f40f1a20640 1 --2- 172.21.15.27:0/2375005841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40ec15ac50 0x7f40ec070fb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009870 tx=0x7f40e802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb1cab410226992 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:21.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.670+0000 7f40f0a1e640 1 -- 172.21.15.27:0/2375005841 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8031740 con 0x7f40ec15ac50 2023-12-19T18:16:21.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.670+0000 7f40f0a1e640 1 -- 172.21.15.27:0/2375005841 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e80318a0 con 0x7f40ec15ac50 2023-12-19T18:16:21.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.670+0000 7f40f0a1e640 1 -- 172.21.15.27:0/2375005841 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8031c30 con 0x7f40ec15ac50 2023-12-19T18:16:21.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.670+0000 7f40f2a22640 1 -- 172.21.15.27:0/2375005841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40ec15ac50 msgr2=0x7f40ec070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:21.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.670+0000 7f40f2a22640 1 --2- 172.21.15.27:0/2375005841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40ec15ac50 0x7f40ec070fb0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009870 tx=0x7f40e802b7e0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.671+0000 7f40f2a22640 1 -- 172.21.15.27:0/2375005841 shutdown_connections 2023-12-19T18:16:21.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.671+0000 7f40f2a22640 1 --2- 172.21.15.27:0/2375005841 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ec0aef20 0x7f40ec06b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.671+0000 7f40f2a22640 1 --2- 172.21.15.27:0/2375005841 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40ec15ac50 0x7f40ec070fb0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.671+0000 7f40f2a22640 1 -- 172.21.15.27:0/2375005841 >> 172.21.15.27:0/2375005841 conn(0x7f40ec06a7d0 msgr2=0x7f40ec0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:21.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.671+0000 7f40f2a22640 1 -- 172.21.15.27:0/2375005841 shutdown_connections 2023-12-19T18:16:21.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.671+0000 7f40f2a22640 1 -- 172.21.15.27:0/2375005841 wait complete. 2023-12-19T18:16:21.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.672+0000 7f40f2a22640 1 Processor -- start 2023-12-19T18:16:21.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.673+0000 7f40f2a22640 1 -- start start 2023-12-19T18:16:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.673+0000 7f40f2a22640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40ec0aef20 0x7f40ec155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.674+0000 7f40f2a22640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40ec0af360 0x7f40ec146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.674+0000 7f40f2a22640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ec15ac50 0x7f40ec147450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.674+0000 7f40f2a22640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec156c30 con 0x7f40ec0aef20 2023-12-19T18:16:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.674+0000 7f40f2a22640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec156e40 con 0x7f40ec0af360 2023-12-19T18:16:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.674+0000 7f40f2a22640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ec157120 con 0x7f40ec15ac50 2023-12-19T18:16:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.674+0000 7f40f1a20640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40ec0aef20 0x7f40ec155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.674+0000 7f40f1a20640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40ec0aef20 0x7f40ec155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59240/0 (socket says 172.21.15.27:59240) 2023-12-19T18:16:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.674+0000 7f40f1a20640 1 -- 172.21.15.27:0/890461832 learned_addr learned my addr 172.21.15.27:0/890461832 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.674+0000 7f40f1a20640 1 -- 172.21.15.27:0/890461832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ec15ac50 msgr2=0x7f40ec147450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.674+0000 7f40f1a20640 1 --2- 172.21.15.27:0/890461832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ec15ac50 0x7f40ec147450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.674+0000 7f40f1a20640 1 -- 172.21.15.27:0/890461832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40ec0af360 msgr2=0x7f40ec146f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.674+0000 7f40f1a20640 1 --2- 172.21.15.27:0/890461832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40ec0af360 0x7f40ec146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.674+0000 7f40f1a20640 1 -- 172.21.15.27:0/890461832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40e80094d0 con 0x7f40ec0aef20 2023-12-19T18:16:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.675+0000 7f40f1a20640 1 --2- 172.21.15.27:0/890461832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40ec0aef20 0x7f40ec155e20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009870 tx=0x7f40e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.675+0000 7f40e27fc640 1 -- 172.21.15.27:0/890461832 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e8033c70 con 0x7f40ec0aef20 2023-12-19T18:16:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.675+0000 7f40e27fc640 1 -- 172.21.15.27:0/890461832 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40e8033dd0 con 0x7f40ec0aef20 2023-12-19T18:16:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.675+0000 7f40e27fc640 1 -- 172.21.15.27:0/890461832 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40e803c640 con 0x7f40ec0aef20 2023-12-19T18:16:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.675+0000 7f40f2a22640 1 -- 172.21.15.27:0/890461832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ec157400 con 0x7f40ec0aef20 2023-12-19T18:16:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.675+0000 7f40f2a22640 1 -- 172.21.15.27:0/890461832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ec156360 con 0x7f40ec0aef20 2023-12-19T18:16:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.676+0000 7f40f2a22640 1 -- 172.21.15.27:0/890461832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ec1564f0 con 0x7f40ec0aef20 2023-12-19T18:16:21.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.677+0000 7f40e27fc640 1 -- 172.21.15.27:0/890461832 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f40e803a040 con 0x7f40ec0aef20 2023-12-19T18:16:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.678+0000 7f40e27fc640 1 --2- 172.21.15.27:0/890461832 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f40d006a430 0x7f40d006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.678+0000 7f40e27fc640 1 -- 172.21.15.27:0/890461832 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f40e80aa090 con 0x7f40ec0aef20 2023-12-19T18:16:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.679+0000 7f40f121f640 1 --2- 172.21.15.27:0/890461832 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f40d006a430 0x7f40d006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:21.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.680+0000 7f40f121f640 1 --2- 172.21.15.27:0/890461832 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f40d006a430 0x7f40d006c8f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f40dc006fd0 tx=0x7f40dc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:21.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.683+0000 7f40e27fc640 1 -- 172.21.15.27:0/890461832 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f40e8075ae0 con 0x7f40ec0aef20 2023-12-19T18:16:21.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.953+0000 7f40f2a22640 1 -- 172.21.15.27:0/890461832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f40ec0c8f00 con 0x7f40ec0aef20 2023-12-19T18:16:21.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.960+0000 7f40e27fc640 1 -- 172.21.15.27:0/890461832 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f40e802f0a0 con 0x7f40ec0aef20 2023-12-19T18:16:21.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.962+0000 7f40f2a22640 1 -- 172.21.15.27:0/890461832 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f40d006a430 msgr2=0x7f40d006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:21.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.962+0000 7f40f2a22640 1 --2- 172.21.15.27:0/890461832 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f40d006a430 0x7f40d006c8f0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f40dc006fd0 tx=0x7f40dc008040 comp rx=0 tx=0).stop 2023-12-19T18:16:21.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.962+0000 7f40f2a22640 1 -- 172.21.15.27:0/890461832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40ec0aef20 msgr2=0x7f40ec155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:21.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.962+0000 7f40f2a22640 1 --2- 172.21.15.27:0/890461832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40ec0aef20 0x7f40ec155e20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f40e8009870 tx=0x7f40e80046c0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.962+0000 7f40f2a22640 1 -- 172.21.15.27:0/890461832 shutdown_connections 2023-12-19T18:16:21.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.962+0000 7f40f2a22640 1 --2- 172.21.15.27:0/890461832 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f40d006a430 0x7f40d006c8f0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.962+0000 7f40f2a22640 1 --2- 172.21.15.27:0/890461832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ec15ac50 0x7f40ec147450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.962+0000 7f40f2a22640 1 --2- 172.21.15.27:0/890461832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40ec0af360 0x7f40ec146f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.962+0000 7f40f2a22640 1 --2- 172.21.15.27:0/890461832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40ec0aef20 0x7f40ec155e20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:21.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.962+0000 7f40f2a22640 1 -- 172.21.15.27:0/890461832 >> 172.21.15.27:0/890461832 conn(0x7f40ec06a7d0 msgr2=0x7f40ec06c400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:21.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.962+0000 7f40f2a22640 1 -- 172.21.15.27:0/890461832 shutdown_connections 2023-12-19T18:16:21.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:21.962+0000 7f40f2a22640 1 -- 172.21.15.27:0/890461832 wait complete. 2023-12-19T18:16:21.982 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:22.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.214+0000 7fa2a7505640 1 Processor -- start 2023-12-19T18:16:22.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.214+0000 7fa2a7505640 1 -- start start 2023-12-19T18:16:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.215+0000 7fa2a7505640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa2a00ca540 0x7fa2a00b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.215+0000 7fa2a7505640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a00cb850 con 0x7fa2a00cbce0 2023-12-19T18:16:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.216+0000 7fa2a7505640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2a00ae330 con 0x7fa2a015af70 2023-12-19T18:16:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.216+0000 7fa2a7505640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2a015b350 con 0x7fa2a00ca540 2023-12-19T18:16:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.216+0000 7fa2a5aa2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa2a00ca540 0x7fa2a00b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.216+0000 7fa2a5aa2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa2a00ca540 0x7fa2a00b1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53990/0 (socket says 172.21.15.27:53990) 2023-12-19T18:16:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.216+0000 7fa2a5aa2640 1 -- 172.21.15.27:0/816768871 learned_addr learned my addr 172.21.15.27:0/816768871 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:22.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.216+0000 7fa2a5aa2640 1 -- 172.21.15.27:0/816768871 >> v1:172.21.15.192:6789/0 conn(0x7fa2a015af70 legacy=0x7fa2a00ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:22.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.216+0000 7fa2a5aa2640 1 -- 172.21.15.27:0/816768871 >> v1:172.21.15.27:6789/0 conn(0x7fa2a00cbce0 legacy=0x7fa2a00adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:22.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.216+0000 7fa2a5aa2640 1 -- 172.21.15.27:0/816768871 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2a00ca990 con 0x7fa2a00ca540 2023-12-19T18:16:22.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.216+0000 7fa297fff640 1 -- 172.21.15.27:0/816768871 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385894662 0 0) 0x7fa2a00cb850 con 0x7fa2a00cbce0 2023-12-19T18:16:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.217+0000 7fa2a5aa2640 1 --2- 172.21.15.27:0/816768871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2a00ca540 0x7fa2a00b1990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa298009e50 tx=0x7fa29802bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1826e0a4e06d7a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.218+0000 7fa297fff640 1 -- 172.21.15.27:0/816768871 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29802cb20 con 0x7fa2a00ca540 2023-12-19T18:16:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.218+0000 7fa297fff640 1 -- 172.21.15.27:0/816768871 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa29802cc80 con 0x7fa2a00ca540 2023-12-19T18:16:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.218+0000 7fa297fff640 1 -- 172.21.15.27:0/816768871 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2980343f0 con 0x7fa2a00ca540 2023-12-19T18:16:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.218+0000 7fa2a7505640 1 -- 172.21.15.27:0/816768871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2a00ca540 msgr2=0x7fa2a00b1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.218+0000 7fa2a7505640 1 --2- 172.21.15.27:0/816768871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2a00ca540 0x7fa2a00b1990 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fa298009e50 tx=0x7fa29802bf80 comp rx=0 tx=0).stop 2023-12-19T18:16:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.218+0000 7fa2a7505640 1 -- 172.21.15.27:0/816768871 shutdown_connections 2023-12-19T18:16:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.218+0000 7fa2a7505640 1 --2- 172.21.15.27:0/816768871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2a00ca540 0x7fa2a00b1990 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.218+0000 7fa2a7505640 1 -- 172.21.15.27:0/816768871 >> 172.21.15.27:0/816768871 conn(0x7fa2a006a330 msgr2=0x7fa2a00b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.218+0000 7fa2a7505640 1 -- 172.21.15.27:0/816768871 shutdown_connections 2023-12-19T18:16:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.218+0000 7fa2a7505640 1 -- 172.21.15.27:0/816768871 wait complete. 2023-12-19T18:16:22.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.220+0000 7fa2a7505640 1 Processor -- start 2023-12-19T18:16:22.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.220+0000 7fa2a7505640 1 -- start start 2023-12-19T18:16:22.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.221+0000 7fa2a7505640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2a00ca540 0x7fa2a01f23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.221+0000 7fa2a7505640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2a00cbce0 0x7fa2a014d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.221+0000 7fa2a7505640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2a015af70 0x7fa2a0070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.221+0000 7fa2a7505640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a01f2930 con 0x7fa2a015af70 2023-12-19T18:16:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.221+0000 7fa2a7505640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a00c54b0 con 0x7fa2a00cbce0 2023-12-19T18:16:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.221+0000 7fa2a7505640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a00c4c10 con 0x7fa2a00ca540 2023-12-19T18:16:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.222+0000 7fa2a5aa2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2a015af70 0x7fa2a0070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.222+0000 7fa2a5aa2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2a015af70 0x7fa2a0070dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54000/0 (socket says 172.21.15.27:54000) 2023-12-19T18:16:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.222+0000 7fa2a5aa2640 1 -- 172.21.15.27:0/3000456855 learned_addr learned my addr 172.21.15.27:0/3000456855 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.222+0000 7fa2a4aa0640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2a00cbce0 0x7fa2a014d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.222+0000 7fa2a52a1640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2a00ca540 0x7fa2a01f23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.222+0000 7fa2a5aa2640 1 -- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2a00ca540 msgr2=0x7fa2a01f23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.222+0000 7fa2a5aa2640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2a00ca540 0x7fa2a01f23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.222+0000 7fa2a5aa2640 1 -- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2a00cbce0 msgr2=0x7fa2a014d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.222+0000 7fa2a5aa2640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2a00cbce0 0x7fa2a014d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.222+0000 7fa2a5aa2640 1 -- 172.21.15.27:0/3000456855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa298009b30 con 0x7fa2a015af70 2023-12-19T18:16:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.222+0000 7fa2a5aa2640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2a015af70 0x7fa2a0070dd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa298009e50 tx=0x7fa298004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.223+0000 7fa295ffb640 1 -- 172.21.15.27:0/3000456855 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298009510 con 0x7fa2a015af70 2023-12-19T18:16:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.223+0000 7fa295ffb640 1 -- 172.21.15.27:0/3000456855 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa298007e50 con 0x7fa2a015af70 2023-12-19T18:16:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.223+0000 7fa2a7505640 1 -- 172.21.15.27:0/3000456855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a00c4ef0 con 0x7fa2a015af70 2023-12-19T18:16:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.223+0000 7fa295ffb640 1 -- 172.21.15.27:0/3000456855 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29803e690 con 0x7fa2a015af70 2023-12-19T18:16:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.223+0000 7fa2a7505640 1 -- 172.21.15.27:0/3000456855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a0147b50 con 0x7fa2a015af70 2023-12-19T18:16:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.224+0000 7fa2a7505640 1 -- 172.21.15.27:0/3000456855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a0068e90 con 0x7fa2a015af70 2023-12-19T18:16:22.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.225+0000 7fa295ffb640 1 -- 172.21.15.27:0/3000456855 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa29803e7f0 con 0x7fa2a015af70 2023-12-19T18:16:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.226+0000 7fa295ffb640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa28406a430 0x7fa28406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.226+0000 7fa295ffb640 1 -- 172.21.15.27:0/3000456855 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa2980ab310 con 0x7fa2a015af70 2023-12-19T18:16:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.226+0000 7fa2a52a1640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa28406a430 0x7fa28406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:22.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.228+0000 7fa2a52a1640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa28406a430 0x7fa28406c8f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa29c006fd0 tx=0x7fa29c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:22.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.230+0000 7fa295ffb640 1 -- 172.21.15.27:0/3000456855 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa298076bb0 con 0x7fa2a015af70 2023-12-19T18:16:22.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.429+0000 7fa2a7505640 1 -- 172.21.15.27:0/3000456855 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2a00c8dd0 con 0x7fa2a015af70 2023-12-19T18:16:22.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.430+0000 7fa295ffb640 1 -- 172.21.15.27:0/3000456855 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+12389 (secure 0 0 0) 0x7fa29807a1d0 con 0x7fa2a015af70 2023-12-19T18:16:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:22.431 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:16:18.124295+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:22.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.433+0000 7fa2a7505640 1 -- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa28406a430 msgr2=0x7fa28406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:22.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.433+0000 7fa2a7505640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa28406a430 0x7fa28406c8f0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fa29c006fd0 tx=0x7fa29c008040 comp rx=0 tx=0).stop 2023-12-19T18:16:22.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.433+0000 7fa2a7505640 1 -- 172.21.15.27:0/3000456855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2a015af70 msgr2=0x7fa2a0070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:22.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.433+0000 7fa2a7505640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2a015af70 0x7fa2a0070dd0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fa298009e50 tx=0x7fa298004a40 comp rx=0 tx=0).stop 2023-12-19T18:16:22.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.434+0000 7fa2a7505640 1 -- 172.21.15.27:0/3000456855 shutdown_connections 2023-12-19T18:16:22.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.434+0000 7fa2a7505640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa28406a430 0x7fa28406c8f0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.434+0000 7fa2a7505640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2a015af70 0x7fa2a0070dd0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.434+0000 7fa2a7505640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2a00cbce0 0x7fa2a014d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.434+0000 7fa2a7505640 1 --2- 172.21.15.27:0/3000456855 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2a00ca540 0x7fa2a01f23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.434+0000 7fa2a7505640 1 -- 172.21.15.27:0/3000456855 >> 172.21.15.27:0/3000456855 conn(0x7fa2a006a330 msgr2=0x7fa2a00b17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:22.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.434+0000 7fa2a7505640 1 -- 172.21.15.27:0/3000456855 shutdown_connections 2023-12-19T18:16:22.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.434+0000 7fa2a7505640 1 -- 172.21.15.27:0/3000456855 wait complete. 2023-12-19T18:16:22.454 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:16:22.455 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:16:22.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.701+0000 7f0766799640 1 Processor -- start 2023-12-19T18:16:22.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.701+0000 7f0766799640 1 -- start start 2023-12-19T18:16:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.703+0000 7f0766799640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f076015a0b0 0x7f07600cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.703+0000 7f0766799640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f076006cee0 0x7f076006d390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.703+0000 7f0766799640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07600c9880 con 0x7f07600ce210 2023-12-19T18:16:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.703+0000 7f0766799640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076006d8d0 con 0x7f076015a0b0 2023-12-19T18:16:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.703+0000 7f0766799640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07600c9de0 con 0x7f076006cee0 2023-12-19T18:16:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.703+0000 7f075ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f076015a0b0 0x7f07600cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.704+0000 7f075ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f076015a0b0 0x7f07600cdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54014/0 (socket says 172.21.15.27:54014) 2023-12-19T18:16:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.704+0000 7f075ffff640 1 -- 172.21.15.27:0/1199425363 learned_addr learned my addr 172.21.15.27:0/1199425363 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:22.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.704+0000 7f0764d36640 1 --2- 172.21.15.27:0/1199425363 >> v2:172.21.15.192:3300/0 conn(0x7f076006cee0 0x7f076006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.704+0000 7f075ffff640 1 -- 172.21.15.27:0/1199425363 >> v2:172.21.15.192:3300/0 conn(0x7f076006cee0 msgr2=0x7f076006d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.704+0000 7f075ffff640 1 --2- 172.21.15.27:0/1199425363 >> v2:172.21.15.192:3300/0 conn(0x7f076006cee0 0x7f076006d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.704+0000 7f075ffff640 1 -- 172.21.15.27:0/1199425363 >> v1:172.21.15.27:6789/0 conn(0x7f07600ce210 legacy=0x7f07600cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.704+0000 7f075ffff640 1 -- 172.21.15.27:0/1199425363 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07600c9b20 con 0x7f076015a0b0 2023-12-19T18:16:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.704+0000 7f0764d36640 1 --2- 172.21.15.27:0/1199425363 >> v2:172.21.15.192:3300/0 conn(0x7f076006cee0 0x7f076006d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.705+0000 7f075ffff640 1 --2- 172.21.15.27:0/1199425363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015a0b0 0x7f07600cdc80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f076006bda0 tx=0x7f075002e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678a4599af996a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.706+0000 7f075effd640 1 -- 172.21.15.27:0/1199425363 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075000b0d0 con 0x7f076015a0b0 2023-12-19T18:16:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.706+0000 7f075effd640 1 -- 172.21.15.27:0/1199425363 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075000b230 con 0x7f076015a0b0 2023-12-19T18:16:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.706+0000 7f075effd640 1 -- 172.21.15.27:0/1199425363 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0750009600 con 0x7f076015a0b0 2023-12-19T18:16:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.706+0000 7f0766799640 1 -- 172.21.15.27:0/1199425363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015a0b0 msgr2=0x7f07600cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.706+0000 7f0766799640 1 --2- 172.21.15.27:0/1199425363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015a0b0 0x7f07600cdc80 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f076006bda0 tx=0x7f075002e180 comp rx=0 tx=0).stop 2023-12-19T18:16:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.706+0000 7f0766799640 1 -- 172.21.15.27:0/1199425363 shutdown_connections 2023-12-19T18:16:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.706+0000 7f0766799640 1 --2- 172.21.15.27:0/1199425363 >> v2:172.21.15.192:3300/0 conn(0x7f076006cee0 0x7f076006d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.706+0000 7f0766799640 1 --2- 172.21.15.27:0/1199425363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015a0b0 0x7f07600cdc80 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.706+0000 7f0766799640 1 -- 172.21.15.27:0/1199425363 >> 172.21.15.27:0/1199425363 conn(0x7f07600cebd0 msgr2=0x7f07600cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.706+0000 7f0766799640 1 -- 172.21.15.27:0/1199425363 shutdown_connections 2023-12-19T18:16:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.706+0000 7f0766799640 1 -- 172.21.15.27:0/1199425363 wait complete. 2023-12-19T18:16:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.708+0000 7f0766799640 1 Processor -- start 2023-12-19T18:16:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.708+0000 7f0766799640 1 -- start start 2023-12-19T18:16:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.709+0000 7f0766799640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f076006cee0 0x7f07600ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.709+0000 7f0766799640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07600ce210 0x7f0760145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.709+0000 7f0766799640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015a0b0 0x7f07600c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.709+0000 7f0766799640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07601595f0 con 0x7f076015a0b0 2023-12-19T18:16:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.709+0000 7f0766799640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07601f2aa0 con 0x7f076006cee0 2023-12-19T18:16:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.709+0000 7f0766799640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07600ae680 con 0x7f07600ce210 2023-12-19T18:16:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.710+0000 7f0764d36640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015a0b0 0x7f07600c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.710+0000 7f0764d36640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015a0b0 0x7f07600c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54024/0 (socket says 172.21.15.27:54024) 2023-12-19T18:16:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.710+0000 7f0764d36640 1 -- 172.21.15.27:0/1222503236 learned_addr learned my addr 172.21.15.27:0/1222503236 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:22.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.710+0000 7f075ffff640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f076006cee0 0x7f07600ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:22.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.710+0000 7f0764d36640 1 -- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07600ce210 msgr2=0x7f0760145c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:22.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.710+0000 7f075f7fe640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07600ce210 0x7f0760145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:22.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.710+0000 7f0764d36640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07600ce210 0x7f0760145c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.710+0000 7f0764d36640 1 -- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f076006cee0 msgr2=0x7f07600ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:22.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.710+0000 7f0764d36640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f076006cee0 0x7f07600ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.710+0000 7f0764d36640 1 -- 172.21.15.27:0/1222503236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075000bd40 con 0x7f076015a0b0 2023-12-19T18:16:22.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.710+0000 7f075ffff640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f076006cee0 0x7f07600ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:22.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.710+0000 7f075f7fe640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07600ce210 0x7f0760145c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:22.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.710+0000 7f0764d36640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015a0b0 0x7f07600c8870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f075800c7c0 tx=0x7f075800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:22.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.711+0000 7f075cff9640 1 -- 172.21.15.27:0/1222503236 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758004230 con 0x7f076015a0b0 2023-12-19T18:16:22.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.711+0000 7f075cff9640 1 -- 172.21.15.27:0/1222503236 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0758004390 con 0x7f076015a0b0 2023-12-19T18:16:22.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.711+0000 7f075cff9640 1 -- 172.21.15.27:0/1222503236 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0758007860 con 0x7f076015a0b0 2023-12-19T18:16:22.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.711+0000 7f0766799640 1 -- 172.21.15.27:0/1222503236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07600ae960 con 0x7f076015a0b0 2023-12-19T18:16:22.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.711+0000 7f0766799640 1 -- 172.21.15.27:0/1222503236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07600aef20 con 0x7f076015a0b0 2023-12-19T18:16:22.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.712+0000 7f0766799640 1 -- 172.21.15.27:0/1222503236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0760068e90 con 0x7f076015a0b0 2023-12-19T18:16:22.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.713+0000 7f075cff9640 1 -- 172.21.15.27:0/1222503236 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f07580079c0 con 0x7f076015a0b0 2023-12-19T18:16:22.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.714+0000 7f075cff9640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f074006a430 0x7f074006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:22.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.714+0000 7f075cff9640 1 -- 172.21.15.27:0/1222503236 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0758088840 con 0x7f076015a0b0 2023-12-19T18:16:22.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.715+0000 7f075ffff640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f074006a430 0x7f074006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.716+0000 7f075ffff640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f074006a430 0x7f074006c8f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0750037000 tx=0x7f0750032000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.718+0000 7f075cff9640 1 -- 172.21.15.27:0/1222503236 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0758054220 con 0x7f076015a0b0 2023-12-19T18:16:22.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.912+0000 7f0766799640 1 -- 172.21.15.27:0/1222503236 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f07600ae7c0 con 0x7f076015a0b0 2023-12-19T18:16:22.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.913+0000 7f075cff9640 1 -- 172.21.15.27:0/1222503236 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7f07600ae7c0 con 0x7f076015a0b0 2023-12-19T18:16:22.913 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:16:22.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.916+0000 7f0766799640 1 -- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f074006a430 msgr2=0x7f074006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:22.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.916+0000 7f0766799640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f074006a430 0x7f074006c8f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f0750037000 tx=0x7f0750032000 comp rx=0 tx=0).stop 2023-12-19T18:16:22.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.916+0000 7f0766799640 1 -- 172.21.15.27:0/1222503236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015a0b0 msgr2=0x7f07600c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:22.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.916+0000 7f0766799640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015a0b0 0x7f07600c8870 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f075800c7c0 tx=0x7f075800cc80 comp rx=0 tx=0).stop 2023-12-19T18:16:22.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.916+0000 7f0766799640 1 -- 172.21.15.27:0/1222503236 shutdown_connections 2023-12-19T18:16:22.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.916+0000 7f0766799640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f074006a430 0x7f074006c8f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.916+0000 7f0766799640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015a0b0 0x7f07600c8870 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.916+0000 7f0766799640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07600ce210 0x7f0760145c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.916+0000 7f0766799640 1 --2- 172.21.15.27:0/1222503236 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f076006cee0 0x7f07600ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:22.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.916+0000 7f0766799640 1 -- 172.21.15.27:0/1222503236 >> 172.21.15.27:0/1222503236 conn(0x7f07600cebd0 msgr2=0x7f076015c740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:22.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.916+0000 7f0766799640 1 -- 172.21.15.27:0/1222503236 shutdown_connections 2023-12-19T18:16:22.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:22.916+0000 7f0766799640 1 -- 172.21.15.27:0/1222503236 wait complete. 2023-12-19T18:16:22.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:16:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.162+0000 7f98d4b67640 1 Processor -- start 2023-12-19T18:16:23.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.162+0000 7f98d4b67640 1 -- start start 2023-12-19T18:16:23.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.164+0000 7f98d4b67640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f98d015ad80 0x7f98d015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.164+0000 7f98d4b67640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98d00b3100 0x7f98d00cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.164+0000 7f98d4b67640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d00cffc0 con 0x7f98d00b2560 2023-12-19T18:16:23.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.164+0000 7f98d4b67640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d015f8a0 con 0x7f98d015ad80 2023-12-19T18:16:23.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.164+0000 7f98d4b67640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d006e470 con 0x7f98d00b3100 2023-12-19T18:16:23.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.164+0000 7f98cf7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f98d015ad80 0x7f98d015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.164+0000 7f98ceffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f98d00b2560 0x7f98d00b29f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56622/0 (socket says 172.21.15.27:56622) 2023-12-19T18:16:23.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.164+0000 7f98ceffd640 1 -- 172.21.15.27:0/971345942 learned_addr learned my addr 172.21.15.27:0/971345942 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:23.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.164+0000 7f98cf7fe640 1 -- 172.21.15.27:0/971345942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98d00b3100 msgr2=0x7f98d00cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.164+0000 7f98cffff640 1 --2- 172.21.15.27:0/971345942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98d00b3100 0x7f98d00cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.165+0000 7f98cf7fe640 1 --2- 172.21.15.27:0/971345942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98d00b3100 0x7f98d00cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.165+0000 7f98cf7fe640 1 -- 172.21.15.27:0/971345942 >> v1:172.21.15.27:6789/0 conn(0x7f98d00b2560 legacy=0x7f98d00b29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.165+0000 7f98ce7fc640 1 -- 172.21.15.27:0/971345942 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053938485 0 0) 0x7f98d00cffc0 con 0x7f98d00b2560 2023-12-19T18:16:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.165+0000 7f98cf7fe640 1 -- 172.21.15.27:0/971345942 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d00cfa00 con 0x7f98d015ad80 2023-12-19T18:16:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.165+0000 7f98cffff640 1 --2- 172.21.15.27:0/971345942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98d00b3100 0x7f98d00cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:23.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.166+0000 7f98cf7fe640 1 --2- 172.21.15.27:0/971345942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98d015ad80 0x7f98d015f360 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009d90 tx=0x7f98c002be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd113f8c9a84a6f3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.166+0000 7f98ce7fc640 1 -- 172.21.15.27:0/971345942 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c002cbc0 con 0x7f98d015ad80 2023-12-19T18:16:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.166+0000 7f98ce7fc640 1 -- 172.21.15.27:0/971345942 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c002cd20 con 0x7f98d015ad80 2023-12-19T18:16:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.166+0000 7f98ce7fc640 1 -- 172.21.15.27:0/971345942 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c0034660 con 0x7f98d015ad80 2023-12-19T18:16:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.167+0000 7f98d4b67640 1 -- 172.21.15.27:0/971345942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98d015ad80 msgr2=0x7f98d015f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.167+0000 7f98d4b67640 1 --2- 172.21.15.27:0/971345942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98d015ad80 0x7f98d015f360 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f98c0009d90 tx=0x7f98c002be10 comp rx=0 tx=0).stop 2023-12-19T18:16:23.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.167+0000 7f98d4b67640 1 -- 172.21.15.27:0/971345942 shutdown_connections 2023-12-19T18:16:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.167+0000 7f98d4b67640 1 --2- 172.21.15.27:0/971345942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98d00b3100 0x7f98d00cc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.167+0000 7f98d4b67640 1 --2- 172.21.15.27:0/971345942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98d015ad80 0x7f98d015f360 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.167+0000 7f98d4b67640 1 -- 172.21.15.27:0/971345942 >> 172.21.15.27:0/971345942 conn(0x7f98d006d4e0 msgr2=0x7f98d006d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.167+0000 7f98d4b67640 1 -- 172.21.15.27:0/971345942 shutdown_connections 2023-12-19T18:16:23.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.167+0000 7f98d4b67640 1 -- 172.21.15.27:0/971345942 wait complete. 2023-12-19T18:16:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.169+0000 7f98d4b67640 1 Processor -- start 2023-12-19T18:16:23.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.169+0000 7f98d4b67640 1 -- start start 2023-12-19T18:16:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.170+0000 7f98d4b67640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98d00b2560 0x7f98d00c9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.170+0000 7f98d4b67640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98d00b3100 0x7f98d00c7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.170+0000 7f98d4b67640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98d015ad80 0x7f98d00c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.170+0000 7f98d4b67640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00caa30 con 0x7f98d015ad80 2023-12-19T18:16:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.170+0000 7f98d4b67640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d015e260 con 0x7f98d00b3100 2023-12-19T18:16:23.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.170+0000 7f98d4b67640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00c9990 con 0x7f98d00b2560 2023-12-19T18:16:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.170+0000 7f98cffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98d015ad80 0x7f98d00c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.170+0000 7f98cffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98d015ad80 0x7f98d00c4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54052/0 (socket says 172.21.15.27:54052) 2023-12-19T18:16:23.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.170+0000 7f98cffff640 1 -- 172.21.15.27:0/3628649182 learned_addr learned my addr 172.21.15.27:0/3628649182 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98cf7fe640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98d00b2560 0x7f98d00c9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98ceffd640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98d00b3100 0x7f98d00c7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98cffff640 1 -- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98d00b2560 msgr2=0x7f98d00c9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98cffff640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98d00b2560 0x7f98d00c9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98cffff640 1 -- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98d00b3100 msgr2=0x7f98d00c7190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98cffff640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98d00b3100 0x7f98d00c7190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98cffff640 1 -- 172.21.15.27:0/3628649182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0009a70 con 0x7f98d015ad80 2023-12-19T18:16:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98ceffd640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98d00b3100 0x7f98d00c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98cf7fe640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98d00b2560 0x7f98d00c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:23.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98cffff640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98d015ad80 0x7f98d00c4e40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c7c0 tx=0x7f98c800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98b7fff640 1 -- 172.21.15.27:0/3628649182 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8004230 con 0x7f98d015ad80 2023-12-19T18:16:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98b7fff640 1 -- 172.21.15.27:0/3628649182 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c8004390 con 0x7f98d015ad80 2023-12-19T18:16:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98b7fff640 1 -- 172.21.15.27:0/3628649182 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c8007860 con 0x7f98d015ad80 2023-12-19T18:16:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98d4b67640 1 -- 172.21.15.27:0/3628649182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d00c9c70 con 0x7f98d015ad80 2023-12-19T18:16:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.171+0000 7f98d4b67640 1 -- 172.21.15.27:0/3628649182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d00ca230 con 0x7f98d015ad80 2023-12-19T18:16:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.173+0000 7f98d4b67640 1 -- 172.21.15.27:0/3628649182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d004e840 con 0x7f98d015ad80 2023-12-19T18:16:23.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.174+0000 7f98b7fff640 1 -- 172.21.15.27:0/3628649182 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f98c80079c0 con 0x7f98d015ad80 2023-12-19T18:16:23.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.175+0000 7f98b7fff640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f98bc06a430 0x7f98bc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.175+0000 7f98b7fff640 1 -- 172.21.15.27:0/3628649182 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f98c80888a0 con 0x7f98d015ad80 2023-12-19T18:16:23.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.176+0000 7f98cf7fe640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f98bc06a430 0x7f98bc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.177+0000 7f98cf7fe640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f98bc06a430 0x7f98bc06c8f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f98c0036000 tx=0x7f98c0030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:23.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.180+0000 7f98b7fff640 1 -- 172.21.15.27:0/3628649182 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f98c80542b0 con 0x7f98d015ad80 2023-12-19T18:16:23.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.377+0000 7f98d4b67640 1 -- 172.21.15.27:0/3628649182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f98d00ce090 con 0x7f98d015ad80 2023-12-19T18:16:23.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.377+0000 7f98b7fff640 1 -- 172.21.15.27:0/3628649182 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+65 (secure 0 0 0) 0x7f98c8015090 con 0x7f98d015ad80 2023-12-19T18:16:23.377 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:23.378 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-12-19T18:16:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-12-19T18:16:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-12-19T18:16:23.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:23.378 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-12-19T18:16:23.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.380+0000 7f98d4b67640 1 -- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f98bc06a430 msgr2=0x7f98bc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.380+0000 7f98d4b67640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f98bc06a430 0x7f98bc06c8f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f98c0036000 tx=0x7f98c0030000 comp rx=0 tx=0).stop 2023-12-19T18:16:23.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.380+0000 7f98d4b67640 1 -- 172.21.15.27:0/3628649182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98d015ad80 msgr2=0x7f98d00c4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.380+0000 7f98d4b67640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98d015ad80 0x7f98d00c4e40 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f98c800c7c0 tx=0x7f98c800cc80 comp rx=0 tx=0).stop 2023-12-19T18:16:23.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.380+0000 7f98d4b67640 1 -- 172.21.15.27:0/3628649182 shutdown_connections 2023-12-19T18:16:23.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.380+0000 7f98d4b67640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f98bc06a430 0x7f98bc06c8f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.380+0000 7f98d4b67640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98d015ad80 0x7f98d00c4e40 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.380+0000 7f98d4b67640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f98d00b3100 0x7f98d00c7190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.380+0000 7f98d4b67640 1 --2- 172.21.15.27:0/3628649182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f98d00b2560 0x7f98d00c9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.380+0000 7f98d4b67640 1 -- 172.21.15.27:0/3628649182 >> 172.21.15.27:0/3628649182 conn(0x7f98d006d4e0 msgr2=0x7f98d015e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:23.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.381+0000 7f98d4b67640 1 -- 172.21.15.27:0/3628649182 shutdown_connections 2023-12-19T18:16:23.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.381+0000 7f98d4b67640 1 -- 172.21.15.27:0/3628649182 wait complete. 2023-12-19T18:16:23.401 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:16:23.402 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:16:23.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.645+0000 7f45437d2640 1 Processor -- start 2023-12-19T18:16:23.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.645+0000 7f45437d2640 1 -- start start 2023-12-19T18:16:23.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.646+0000 7f45437d2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f453c1619c0 0x7f453c165db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.647+0000 7f45437d2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c161da0 con 0x7f453c0cdf70 2023-12-19T18:16:23.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.647+0000 7f45437d2640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f453c1662f0 con 0x7f453c0c90c0 2023-12-19T18:16:23.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.647+0000 7f45437d2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0ce900 con 0x7f453c1619c0 2023-12-19T18:16:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.647+0000 7f4541d6f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f453c0cdf70 0x7f453c0caf80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56638/0 (socket says 172.21.15.27:56638) 2023-12-19T18:16:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.647+0000 7f4541d6f640 1 -- 172.21.15.27:0/1418544890 learned_addr learned my addr 172.21.15.27:0/1418544890 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.647+0000 7f454156e640 1 --2- 172.21.15.27:0/1418544890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f453c1619c0 0x7f453c165db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.647+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 917618809 0 0) 0x7f453c161da0 con 0x7f453c0cdf70 2023-12-19T18:16:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4528003520 con 0x7f453c0cdf70 2023-12-19T18:16:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 215284993 0 0) 0x7f4528003520 con 0x7f453c0cdf70 2023-12-19T18:16:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f453c1619c0 msgr2=0x7f453c165db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 --2- 172.21.15.27:0/1418544890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f453c1619c0 0x7f453c165db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 >> v1:172.21.15.192:6789/0 conn(0x7f453c0c90c0 legacy=0x7f453c0cd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c162be0 con 0x7f453c0cdf70 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f4534001a10 con 0x7f453c0cdf70 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 >> v1:172.21.15.27:6789/0 conn(0x7f453c0cdf70 legacy=0x7f453c0caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 --2- 172.21.15.27:0/1418544890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f453c1619c0 0x7f45280039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 --2- 172.21.15.27:0/1418544890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4528003f70 0x7f45280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 --2- 172.21.15.27:0/1418544890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4528004d00 0x7f4528005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0ce900 con 0x7f4528003f70 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452800bac0 con 0x7f4528004d00 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452800bc80 con 0x7f453c1619c0 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.648+0000 7f4540d6d640 1 --2- 172.21.15.27:0/1418544890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4528003f70 0x7f45280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.649+0000 7f454156e640 1 --2- 172.21.15.27:0/1418544890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f453c1619c0 0x7f45280039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.649+0000 7f4540d6d640 1 -- 172.21.15.27:0/1418544890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f453c1619c0 msgr2=0x7f45280039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.649+0000 7f4540d6d640 1 --2- 172.21.15.27:0/1418544890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f453c1619c0 0x7f45280039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.649+0000 7f4540d6d640 1 -- 172.21.15.27:0/1418544890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4528004d00 msgr2=0x7f4528005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:23.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.649+0000 7f4540d6d640 1 --2- 172.21.15.27:0/1418544890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4528004d00 0x7f4528005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.649+0000 7f4540d6d640 1 -- 172.21.15.27:0/1418544890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4528003520 con 0x7f4528003f70 2023-12-19T18:16:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.650+0000 7f4540d6d640 1 --2- 172.21.15.27:0/1418544890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4528003f70 0x7f45280047c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f452c0015b0 tx=0x7f45280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=568ce8e81feb6a70 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.650+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c0052d0 con 0x7f4528003f70 2023-12-19T18:16:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.650+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f452c02e050 con 0x7f4528003f70 2023-12-19T18:16:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.650+0000 7f4533fff640 1 -- 172.21.15.27:0/1418544890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f452c003830 con 0x7f4528003f70 2023-12-19T18:16:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.651+0000 7f45437d2640 1 -- 172.21.15.27:0/1418544890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4528003f70 msgr2=0x7f45280047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.651+0000 7f45437d2640 1 --2- 172.21.15.27:0/1418544890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4528003f70 0x7f45280047c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f452c0015b0 tx=0x7f45280057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.651+0000 7f4541d6f640 1 -- 172.21.15.27:0/1418544890 reap_dead start 2023-12-19T18:16:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.651+0000 7f45437d2640 1 -- 172.21.15.27:0/1418544890 shutdown_connections 2023-12-19T18:16:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.651+0000 7f45437d2640 1 -- 172.21.15.27:0/1418544890 >> 172.21.15.27:0/1418544890 conn(0x7f453c0b0620 msgr2=0x7f453c0b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.651+0000 7f45437d2640 1 -- 172.21.15.27:0/1418544890 shutdown_connections 2023-12-19T18:16:23.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.651+0000 7f45437d2640 1 -- 172.21.15.27:0/1418544890 wait complete. 2023-12-19T18:16:23.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.653+0000 7f45437d2640 1 Processor -- start 2023-12-19T18:16:23.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.653+0000 7f45437d2640 1 -- start start 2023-12-19T18:16:23.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.654+0000 7f45437d2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4528003f70 0x7f453c15f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.654+0000 7f45437d2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f453c0c5270 0x7f453c0b2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.654+0000 7f45437d2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f453c0af2e0 0x7f453c0af790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.654+0000 7f45437d2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0b2530 con 0x7f453c0af2e0 2023-12-19T18:16:23.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.654+0000 7f45437d2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0afcd0 con 0x7f453c0c5270 2023-12-19T18:16:23.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.654+0000 7f45437d2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f453c0b16a0 con 0x7f4528003f70 2023-12-19T18:16:23.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.654+0000 7f4541d6f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f453c0af2e0 0x7f453c0af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.654+0000 7f4541d6f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f453c0af2e0 0x7f453c0af790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54078/0 (socket says 172.21.15.27:54078) 2023-12-19T18:16:23.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.654+0000 7f4541d6f640 1 -- 172.21.15.27:0/4281751789 learned_addr learned my addr 172.21.15.27:0/4281751789 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:23.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.655+0000 7f454156e640 1 --2- 172.21.15.27:0/4281751789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4528003f70 0x7f453c15f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.655+0000 7f4541d6f640 1 -- 172.21.15.27:0/4281751789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4528003f70 msgr2=0x7f453c15f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.655+0000 7f4541d6f640 1 --2- 172.21.15.27:0/4281751789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4528003f70 0x7f453c15f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.655+0000 7f4541d6f640 1 -- 172.21.15.27:0/4281751789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f453c0c5270 msgr2=0x7f453c0b2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:23.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.655+0000 7f4541d6f640 1 --2- 172.21.15.27:0/4281751789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f453c0c5270 0x7f453c0b2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:23.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.655+0000 7f4541d6f640 1 -- 172.21.15.27:0/4281751789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f453c15f5e0 con 0x7f453c0af2e0 2023-12-19T18:16:23.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.655+0000 7f4541d6f640 1 --2- 172.21.15.27:0/4281751789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f453c0af2e0 0x7f453c0af790 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4534001560 tx=0x7f453400cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:23.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.655+0000 7f4531ffb640 1 -- 172.21.15.27:0/4281751789 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453400cea0 con 0x7f453c0af2e0 2023-12-19T18:16:23.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.655+0000 7f4531ffb640 1 -- 172.21.15.27:0/4281751789 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f453400a610 con 0x7f453c0af2e0 2023-12-19T18:16:23.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.655+0000 7f45437d2640 1 -- 172.21.15.27:0/4281751789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f453c0b3170 con 0x7f453c0af2e0 2023-12-19T18:16:23.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.656+0000 7f4531ffb640 1 -- 172.21.15.27:0/4281751789 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f453400a9c0 con 0x7f453c0af2e0 2023-12-19T18:16:23.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.656+0000 7f45437d2640 1 -- 172.21.15.27:0/4281751789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f453c0b1b90 con 0x7f453c0af2e0 2023-12-19T18:16:23.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.657+0000 7f45437d2640 1 -- 172.21.15.27:0/4281751789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f453c068e90 con 0x7f453c0af2e0 2023-12-19T18:16:23.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.658+0000 7f4531ffb640 1 -- 172.21.15.27:0/4281751789 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f453400ab20 con 0x7f453c0af2e0 2023-12-19T18:16:23.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.659+0000 7f4531ffb640 1 --2- 172.21.15.27:0/4281751789 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f452006a430 0x7f452006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:23.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.659+0000 7f4531ffb640 1 -- 172.21.15.27:0/4281751789 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f45340882c0 con 0x7f453c0af2e0 2023-12-19T18:16:23.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.659+0000 7f454156e640 1 --2- 172.21.15.27:0/4281751789 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f452006a430 0x7f452006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:23.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.661+0000 7f454156e640 1 --2- 172.21.15.27:0/4281751789 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f452006a430 0x7f452006c8f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4538009a10 tx=0x7f4538008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:23.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.663+0000 7f4531ffb640 1 -- 172.21.15.27:0/4281751789 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4534053c10 con 0x7f453c0af2e0 2023-12-19T18:16:23.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:23.858+0000 7f45437d2640 1 -- 172.21.15.27:0/4281751789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f453c0adc90 con 0x7f453c0af2e0 2023-12-19T18:16:24.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.272+0000 7f4531ffb640 1 -- 172.21.15.27:0/4281751789 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4534057230 con 0x7f453c0af2e0 2023-12-19T18:16:24.273 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-12-19T18:16:24.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.279+0000 7f45437d2640 1 -- 172.21.15.27:0/4281751789 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f452006a430 msgr2=0x7f452006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.279+0000 7f45437d2640 1 --2- 172.21.15.27:0/4281751789 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f452006a430 0x7f452006c8f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f4538009a10 tx=0x7f4538008040 comp rx=0 tx=0).stop 2023-12-19T18:16:24.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.279+0000 7f45437d2640 1 -- 172.21.15.27:0/4281751789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f453c0af2e0 msgr2=0x7f453c0af790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.279+0000 7f45437d2640 1 --2- 172.21.15.27:0/4281751789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f453c0af2e0 0x7f453c0af790 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f4534001560 tx=0x7f453400cc50 comp rx=0 tx=0).stop 2023-12-19T18:16:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.279+0000 7f45437d2640 1 -- 172.21.15.27:0/4281751789 shutdown_connections 2023-12-19T18:16:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.279+0000 7f45437d2640 1 --2- 172.21.15.27:0/4281751789 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f452006a430 0x7f452006c8f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.279+0000 7f45437d2640 1 --2- 172.21.15.27:0/4281751789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f453c0af2e0 0x7f453c0af790 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.279+0000 7f45437d2640 1 --2- 172.21.15.27:0/4281751789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f453c0c5270 0x7f453c0b2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.279+0000 7f45437d2640 1 --2- 172.21.15.27:0/4281751789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4528003f70 0x7f453c15f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.279+0000 7f45437d2640 1 -- 172.21.15.27:0/4281751789 >> 172.21.15.27:0/4281751789 conn(0x7f453c0b0620 msgr2=0x7f453c0cb480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.280+0000 7f45437d2640 1 -- 172.21.15.27:0/4281751789 shutdown_connections 2023-12-19T18:16:24.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.280+0000 7f45437d2640 1 -- 172.21.15.27:0/4281751789 wait complete. 2023-12-19T18:16:24.300 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:16:24.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.544+0000 7f3d389d8640 1 Processor -- start 2023-12-19T18:16:24.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.544+0000 7f3d389d8640 1 -- start start 2023-12-19T18:16:24.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.546+0000 7f3d389d8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d3415ae60 0x7f3d34070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.546+0000 7f3d389d8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3d340ad9d0 0x7f3d34071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.546+0000 7f3d389d8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d340b03f0 con 0x7f3d340cc870 2023-12-19T18:16:24.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.546+0000 7f3d389d8640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d340b0c90 con 0x7f3d340ad9d0 2023-12-19T18:16:24.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.546+0000 7f3d389d8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d34071550 con 0x7f3d3415ae60 2023-12-19T18:16:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.546+0000 7f3d2bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3d340cc870 0x7f3d3415e9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56652/0 (socket says 172.21.15.27:56652) 2023-12-19T18:16:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.547+0000 7f3d2bfff640 1 -- 172.21.15.27:0/1562252062 learned_addr learned my addr 172.21.15.27:0/1562252062 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.547+0000 7f3d3359e640 1 --2- 172.21.15.27:0/1562252062 >> v2:172.21.15.27:3300/0 conn(0x7f3d340ad9d0 0x7f3d34071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.547+0000 7f3d32d9d640 1 --2- 172.21.15.27:0/1562252062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d3415ae60 0x7f3d34070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.547+0000 7f3d3259c640 1 -- 172.21.15.27:0/1562252062 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1893145291 0 0) 0x7f3d340b03f0 con 0x7f3d340cc870 2023-12-19T18:16:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.547+0000 7f3d3259c640 1 -- 172.21.15.27:0/1562252062 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3d20003520 con 0x7f3d340cc870 2023-12-19T18:16:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.547+0000 7f3d3359e640 1 -- 172.21.15.27:0/1562252062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d3415ae60 msgr2=0x7f3d34070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.547+0000 7f3d3359e640 1 --2- 172.21.15.27:0/1562252062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d3415ae60 0x7f3d34070ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.547+0000 7f3d3359e640 1 -- 172.21.15.27:0/1562252062 >> v1:172.21.15.27:6789/0 conn(0x7f3d340cc870 legacy=0x7f3d3415e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.547+0000 7f3d3359e640 1 -- 172.21.15.27:0/1562252062 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d340b4740 con 0x7f3d340ad9d0 2023-12-19T18:16:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.548+0000 7f3d3359e640 1 --2- 172.21.15.27:0/1562252062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d340ad9d0 0x7f3d34071010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009d90 tx=0x7f3d2c02bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d829935bc48d32 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.549+0000 7f3d3259c640 1 -- 172.21.15.27:0/1562252062 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c02cb30 con 0x7f3d340ad9d0 2023-12-19T18:16:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.549+0000 7f3d3259c640 1 -- 172.21.15.27:0/1562252062 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2c02cc90 con 0x7f3d340ad9d0 2023-12-19T18:16:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.549+0000 7f3d3259c640 1 -- 172.21.15.27:0/1562252062 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c034440 con 0x7f3d340ad9d0 2023-12-19T18:16:24.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.549+0000 7f3d389d8640 1 -- 172.21.15.27:0/1562252062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d340ad9d0 msgr2=0x7f3d34071010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.549+0000 7f3d389d8640 1 --2- 172.21.15.27:0/1562252062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d340ad9d0 0x7f3d34071010 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009d90 tx=0x7f3d2c02bd80 comp rx=0 tx=0).stop 2023-12-19T18:16:24.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.549+0000 7f3d389d8640 1 -- 172.21.15.27:0/1562252062 shutdown_connections 2023-12-19T18:16:24.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.549+0000 7f3d389d8640 1 --2- 172.21.15.27:0/1562252062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d340ad9d0 0x7f3d34071010 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.549+0000 7f3d389d8640 1 --2- 172.21.15.27:0/1562252062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d3415ae60 0x7f3d34070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.549+0000 7f3d389d8640 1 -- 172.21.15.27:0/1562252062 >> 172.21.15.27:0/1562252062 conn(0x7f3d340cb3f0 msgr2=0x7f3d340cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:24.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.549+0000 7f3d389d8640 1 -- 172.21.15.27:0/1562252062 shutdown_connections 2023-12-19T18:16:24.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.550+0000 7f3d389d8640 1 -- 172.21.15.27:0/1562252062 wait complete. 2023-12-19T18:16:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.551+0000 7f3d389d8640 1 Processor -- start 2023-12-19T18:16:24.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.551+0000 7f3d389d8640 1 -- start start 2023-12-19T18:16:24.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.552+0000 7f3d389d8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d340ad9d0 0x7f3d341efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.552+0000 7f3d389d8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d340cc870 0x7f3d3415bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.552+0000 7f3d389d8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d3415ae60 0x7f3d3415df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.552+0000 7f3d389d8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d340c1ea0 con 0x7f3d3415ae60 2023-12-19T18:16:24.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.552+0000 7f3d389d8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d340c2d50 con 0x7f3d340ad9d0 2023-12-19T18:16:24.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.552+0000 7f3d389d8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d340c3030 con 0x7f3d340cc870 2023-12-19T18:16:24.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.553+0000 7f3d3359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d3415ae60 0x7f3d3415df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.553+0000 7f3d3359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d3415ae60 0x7f3d3415df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54100/0 (socket says 172.21.15.27:54100) 2023-12-19T18:16:24.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.553+0000 7f3d3359e640 1 -- 172.21.15.27:0/2365049091 learned_addr learned my addr 172.21.15.27:0/2365049091 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:24.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.553+0000 7f3d2bfff640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d340cc870 0x7f3d3415bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.553+0000 7f3d32d9d640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d340ad9d0 0x7f3d341efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.553+0000 7f3d3359e640 1 -- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d340cc870 msgr2=0x7f3d3415bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.553+0000 7f3d3359e640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d340cc870 0x7f3d3415bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.553+0000 7f3d3359e640 1 -- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d340ad9d0 msgr2=0x7f3d341efdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:24.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.553+0000 7f3d3359e640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d340ad9d0 0x7f3d341efdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:24.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.553+0000 7f3d3359e640 1 -- 172.21.15.27:0/2365049091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c009a70 con 0x7f3d3415ae60 2023-12-19T18:16:24.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.554+0000 7f3d3359e640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d3415ae60 0x7f3d3415df60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009d90 tx=0x7f3d2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:24.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.554+0000 7f3d2b7fe640 1 -- 172.21.15.27:0/2365049091 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c009450 con 0x7f3d3415ae60 2023-12-19T18:16:24.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.554+0000 7f3d2b7fe640 1 -- 172.21.15.27:0/2365049091 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2c040450 con 0x7f3d3415ae60 2023-12-19T18:16:24.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.554+0000 7f3d2b7fe640 1 -- 172.21.15.27:0/2365049091 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c040800 con 0x7f3d3415ae60 2023-12-19T18:16:24.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.554+0000 7f3d389d8640 1 -- 172.21.15.27:0/2365049091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d340c3310 con 0x7f3d3415ae60 2023-12-19T18:16:24.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.554+0000 7f3d389d8640 1 -- 172.21.15.27:0/2365049091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d340c38d0 con 0x7f3d3415ae60 2023-12-19T18:16:24.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.556+0000 7f3d389d8640 1 -- 172.21.15.27:0/2365049091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d34066fe0 con 0x7f3d3415ae60 2023-12-19T18:16:24.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.557+0000 7f3d2b7fe640 1 -- 172.21.15.27:0/2365049091 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3d2c040960 con 0x7f3d3415ae60 2023-12-19T18:16:24.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.558+0000 7f3d2b7fe640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3d1406a430 0x7f3d1406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:24.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.559+0000 7f3d2b7fe640 1 -- 172.21.15.27:0/2365049091 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f3d2c0ab8b0 con 0x7f3d3415ae60 2023-12-19T18:16:24.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.559+0000 7f3d32d9d640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3d1406a430 0x7f3d1406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:24.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.560+0000 7f3d32d9d640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3d1406a430 0x7f3d1406c8f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d24006fd0 tx=0x7f3d24008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:24.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.562+0000 7f3d2b7fe640 1 -- 172.21.15.27:0/2365049091 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3d2c077180 con 0x7f3d3415ae60 2023-12-19T18:16:24.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:24.758+0000 7f3d389d8640 1 -- 172.21.15.27:0/2365049091 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3d340c3af0 con 0x7f3d3415ae60 2023-12-19T18:16:25.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.282+0000 7f3d2b7fe640 1 -- 172.21.15.27:0/2365049091 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3d2c07a7a0 con 0x7f3d3415ae60 2023-12-19T18:16:25.287 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-12-19T18:16:25.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.291+0000 7f3d389d8640 1 -- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3d1406a430 msgr2=0x7f3d1406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.291+0000 7f3d389d8640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3d1406a430 0x7f3d1406c8f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f3d24006fd0 tx=0x7f3d24008040 comp rx=0 tx=0).stop 2023-12-19T18:16:25.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.291+0000 7f3d389d8640 1 -- 172.21.15.27:0/2365049091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d3415ae60 msgr2=0x7f3d3415df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.291+0000 7f3d389d8640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d3415ae60 0x7f3d3415df60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009d90 tx=0x7f3d2c004a40 comp rx=0 tx=0).stop 2023-12-19T18:16:25.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.291+0000 7f3d389d8640 1 -- 172.21.15.27:0/2365049091 shutdown_connections 2023-12-19T18:16:25.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.291+0000 7f3d389d8640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3d1406a430 0x7f3d1406c8f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.291+0000 7f3d389d8640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d3415ae60 0x7f3d3415df60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.291+0000 7f3d389d8640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d340cc870 0x7f3d3415bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.291+0000 7f3d389d8640 1 --2- 172.21.15.27:0/2365049091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d340ad9d0 0x7f3d341efdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.291+0000 7f3d389d8640 1 -- 172.21.15.27:0/2365049091 >> 172.21.15.27:0/2365049091 conn(0x7f3d340cb3f0 msgr2=0x7f3d340aec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:25.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.292+0000 7f3d389d8640 1 -- 172.21.15.27:0/2365049091 shutdown_connections 2023-12-19T18:16:25.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.292+0000 7f3d389d8640 1 -- 172.21.15.27:0/2365049091 wait complete. 2023-12-19T18:16:25.318 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:16:25.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.522+0000 7f0015428640 1 Processor -- start 2023-12-19T18:16:25.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.522+0000 7f0015428640 1 -- start start 2023-12-19T18:16:25.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.523+0000 7f0015428640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f00100c2720 0x7f001006fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.523+0000 7f0015428640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0010071420 0x7f001015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.524+0000 7f0015428640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f001006bc80 con 0x7f001015b1a0 2023-12-19T18:16:25.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.524+0000 7f0015428640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001006d690 con 0x7f0010071420 2023-12-19T18:16:25.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.524+0000 7f0015428640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001006d0a0 con 0x7f00100c2720 2023-12-19T18:16:25.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.524+0000 7f000effd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f001015b1a0 0x7f001006f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56658/0 (socket says 172.21.15.27:56658) 2023-12-19T18:16:25.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.524+0000 7f000effd640 1 -- 172.21.15.27:0/3125769481 learned_addr learned my addr 172.21.15.27:0/3125769481 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:25.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.524+0000 7f000f7fe640 1 --2- 172.21.15.27:0/3125769481 >> v2:172.21.15.27:3300/0 conn(0x7f0010071420 0x7f001015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.524+0000 7f000e7fc640 1 --2- 172.21.15.27:0/3125769481 >> v2:172.21.15.192:3300/0 conn(0x7f00100c2720 0x7f001006fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.524+0000 7f000dffb640 1 -- 172.21.15.27:0/3125769481 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 879038183 0 0) 0x7f0000001110 con 0x7f001015b1a0 2023-12-19T18:16:25.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.524+0000 7f000dffb640 1 -- 172.21.15.27:0/3125769481 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efff8003520 con 0x7f001015b1a0 2023-12-19T18:16:25.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.525+0000 7f000f7fe640 1 -- 172.21.15.27:0/3125769481 >> v2:172.21.15.192:3300/0 conn(0x7f00100c2720 msgr2=0x7f001006fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:25.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.525+0000 7f000f7fe640 1 --2- 172.21.15.27:0/3125769481 >> v2:172.21.15.192:3300/0 conn(0x7f00100c2720 0x7f001006fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.525+0000 7f000f7fe640 1 -- 172.21.15.27:0/3125769481 >> v1:172.21.15.27:6789/0 conn(0x7f001015b1a0 legacy=0x7f001006f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.525+0000 7f000f7fe640 1 -- 172.21.15.27:0/3125769481 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001006b6c0 con 0x7f0010071420 2023-12-19T18:16:25.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.526+0000 7f000f7fe640 1 --2- 172.21.15.27:0/3125769481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0010071420 0x7f001015ed80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f001006acf0 tx=0x7f000802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85dd333363263a98 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:25.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.526+0000 7f000dffb640 1 -- 172.21.15.27:0/3125769481 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f000802cbf0 con 0x7f0010071420 2023-12-19T18:16:25.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.526+0000 7f000dffb640 1 -- 172.21.15.27:0/3125769481 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f000802cd50 con 0x7f0010071420 2023-12-19T18:16:25.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.526+0000 7f000dffb640 1 -- 172.21.15.27:0/3125769481 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008034660 con 0x7f0010071420 2023-12-19T18:16:25.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.526+0000 7f0015428640 1 -- 172.21.15.27:0/3125769481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0010071420 msgr2=0x7f001015ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.527+0000 7f0015428640 1 --2- 172.21.15.27:0/3125769481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0010071420 0x7f001015ed80 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f001006acf0 tx=0x7f000802be40 comp rx=0 tx=0).stop 2023-12-19T18:16:25.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.527+0000 7f0015428640 1 -- 172.21.15.27:0/3125769481 shutdown_connections 2023-12-19T18:16:25.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.527+0000 7f0015428640 1 --2- 172.21.15.27:0/3125769481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0010071420 0x7f001015ed80 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.527+0000 7f0015428640 1 --2- 172.21.15.27:0/3125769481 >> v2:172.21.15.192:3300/0 conn(0x7f00100c2720 0x7f001006fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.527+0000 7f0015428640 1 -- 172.21.15.27:0/3125769481 >> 172.21.15.27:0/3125769481 conn(0x7f00100cebd0 msgr2=0x7f00100cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:25.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.527+0000 7f0015428640 1 -- 172.21.15.27:0/3125769481 shutdown_connections 2023-12-19T18:16:25.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.527+0000 7f0015428640 1 -- 172.21.15.27:0/3125769481 wait complete. 2023-12-19T18:16:25.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.529+0000 7f0015428640 1 Processor -- start 2023-12-19T18:16:25.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.529+0000 7f0015428640 1 -- start start 2023-12-19T18:16:25.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f0015428640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0010071420 0x7f00101e1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f0015428640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00100c2720 0x7f00101e04a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f0015428640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f001015b1a0 0x7f00100ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f0015428640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00101e22b0 con 0x7f001015b1a0 2023-12-19T18:16:25.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f0015428640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00101e0cd0 con 0x7f00100c2720 2023-12-19T18:16:25.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f0015428640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00101e0fb0 con 0x7f0010071420 2023-12-19T18:16:25.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f000f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f001015b1a0 0x7f00100ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f000f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f001015b1a0 0x7f00100ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54122/0 (socket says 172.21.15.27:54122) 2023-12-19T18:16:25.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f000f7fe640 1 -- 172.21.15.27:0/1618266829 learned_addr learned my addr 172.21.15.27:0/1618266829 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:25.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f000e7fc640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00100c2720 0x7f00101e04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f000effd640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0010071420 0x7f00101e1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f000f7fe640 1 -- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0010071420 msgr2=0x7f00101e1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f000f7fe640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0010071420 0x7f00101e1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f000f7fe640 1 -- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00100c2720 msgr2=0x7f00101e04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.530+0000 7f000f7fe640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00100c2720 0x7f00101e04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.531+0000 7f000f7fe640 1 -- 172.21.15.27:0/1618266829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0008009a70 con 0x7f001015b1a0 2023-12-19T18:16:25.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.531+0000 7f000f7fe640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f001015b1a0 0x7f00100ac8d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0008009e80 tx=0x7f0008004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:25.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.531+0000 7efff77fe640 1 -- 172.21.15.27:0/1618266829 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008040650 con 0x7f001015b1a0 2023-12-19T18:16:25.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.531+0000 7efff77fe640 1 -- 172.21.15.27:0/1618266829 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00080407b0 con 0x7f001015b1a0 2023-12-19T18:16:25.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.531+0000 7efff77fe640 1 -- 172.21.15.27:0/1618266829 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0008040b60 con 0x7f001015b1a0 2023-12-19T18:16:25.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.531+0000 7f0015428640 1 -- 172.21.15.27:0/1618266829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00101e1290 con 0x7f001015b1a0 2023-12-19T18:16:25.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.531+0000 7f0015428640 1 -- 172.21.15.27:0/1618266829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00101e1850 con 0x7f001015b1a0 2023-12-19T18:16:25.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.533+0000 7f0015428640 1 -- 172.21.15.27:0/1618266829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00100ad5d0 con 0x7f001015b1a0 2023-12-19T18:16:25.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.535+0000 7efff77fe640 1 -- 172.21.15.27:0/1618266829 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0008040cc0 con 0x7f001015b1a0 2023-12-19T18:16:25.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.536+0000 7efff77fe640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efffc06a430 0x7efffc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:25.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.536+0000 7efff77fe640 1 -- 172.21.15.27:0/1618266829 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f00080abaa0 con 0x7f001015b1a0 2023-12-19T18:16:25.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.536+0000 7f000effd640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efffc06a430 0x7efffc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:25.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.538+0000 7f000effd640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efffc06a430 0x7efffc06c8f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0000007fd0 tx=0x7f0000009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:25.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.540+0000 7efff77fe640 1 -- 172.21.15.27:0/1618266829 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f00080771f0 con 0x7f001015b1a0 2023-12-19T18:16:25.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.777+0000 7f0015428640 1 -- 172.21.15.27:0/1618266829 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f0010068e90 con 0x7f001015b1a0 2023-12-19T18:16:25.788 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 4 and data pool 5 2023-12-19T18:16:25.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.786+0000 7efff77fe640 1 -- 172.21.15.27:0/1618266829 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0008009450 con 0x7f001015b1a0 2023-12-19T18:16:25.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.792+0000 7f0015428640 1 -- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efffc06a430 msgr2=0x7efffc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.792+0000 7f0015428640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efffc06a430 0x7efffc06c8f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0000007fd0 tx=0x7f0000009040 comp rx=0 tx=0).stop 2023-12-19T18:16:25.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.792+0000 7f0015428640 1 -- 172.21.15.27:0/1618266829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f001015b1a0 msgr2=0x7f00100ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:25.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.792+0000 7f0015428640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f001015b1a0 0x7f00100ac8d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f0008009e80 tx=0x7f0008004a40 comp rx=0 tx=0).stop 2023-12-19T18:16:25.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.792+0000 7f0015428640 1 -- 172.21.15.27:0/1618266829 shutdown_connections 2023-12-19T18:16:25.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.792+0000 7f0015428640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efffc06a430 0x7efffc06c8f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.792+0000 7f0015428640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f001015b1a0 0x7f00100ac8d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.792+0000 7f0015428640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f00100c2720 0x7f00101e04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.792+0000 7f0015428640 1 --2- 172.21.15.27:0/1618266829 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0010071420 0x7f00101e1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:25.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.792+0000 7f0015428640 1 -- 172.21.15.27:0/1618266829 >> 172.21.15.27:0/1618266829 conn(0x7f00100cebd0 msgr2=0x7f00100cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:25.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.793+0000 7f0015428640 1 -- 172.21.15.27:0/1618266829 shutdown_connections 2023-12-19T18:16:25.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:25.793+0000 7f0015428640 1 -- 172.21.15.27:0/1618266829 wait complete. 2023-12-19T18:16:25.828 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:26.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.021+0000 7f0a50a22640 1 Processor -- start 2023-12-19T18:16:26.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.021+0000 7f0a50a22640 1 -- start start 2023-12-19T18:16:26.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.022+0000 7f0a50a22640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a4c1553a0 0x7f0a4c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.023+0000 7f0a50a22640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4c0b0330 con 0x7f0a4c15ee30 2023-12-19T18:16:26.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.023+0000 7f0a50a22640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4c0cb350 con 0x7f0a4c0afeb0 2023-12-19T18:16:26.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.023+0000 7f0a50a22640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0cbcf0 con 0x7f0a4c1553a0 2023-12-19T18:16:26.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.023+0000 7f0a4bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0a4c15ee30 0x7f0a4c0c39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56668/0 (socket says 172.21.15.27:56668) 2023-12-19T18:16:26.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.023+0000 7f0a4bfff640 1 -- 172.21.15.27:0/4120509413 learned_addr learned my addr 172.21.15.27:0/4120509413 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:26.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.023+0000 7f0a4b7fe640 1 --2- 172.21.15.27:0/4120509413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a4c1553a0 0x7f0a4c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.023+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2552736739 0 0) 0x7f0a4c0b0330 con 0x7f0a4c15ee30 2023-12-19T18:16:26.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.023+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a34003520 con 0x7f0a4c15ee30 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1612994443 0 0) 0x7f0a34003520 con 0x7f0a4c15ee30 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a4c1553a0 msgr2=0x7f0a4c0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4a7fc640 1 --2- 172.21.15.27:0/4120509413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a4c1553a0 0x7f0a4c0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 >> v1:172.21.15.192:6789/0 conn(0x7f0a4c0afeb0 legacy=0x7f0a4c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c0cb630 con 0x7f0a4c15ee30 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f0a440019f0 con 0x7f0a4c15ee30 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 >> v1:172.21.15.27:6789/0 conn(0x7f0a4c15ee30 legacy=0x7f0a4c0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4a7fc640 1 --2- 172.21.15.27:0/4120509413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a340039a0 0x7f0a34003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4a7fc640 1 --2- 172.21.15.27:0/4120509413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a340047c0 0x7f0a34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4a7fc640 1 --2- 172.21.15.27:0/4120509413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a4c1553a0 0x7f0a34005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0cbcf0 con 0x7f0a340039a0 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3400bb10 con 0x7f0a340047c0 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3400bcd0 con 0x7f0a4c1553a0 2023-12-19T18:16:26.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.024+0000 7f0a4b7fe640 1 --2- 172.21.15.27:0/4120509413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a340039a0 0x7f0a34003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.025+0000 7f0a4affd640 1 --2- 172.21.15.27:0/4120509413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a340047c0 0x7f0a34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.025+0000 7f0a4b7fe640 1 -- 172.21.15.27:0/4120509413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a4c1553a0 msgr2=0x7f0a34005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.025+0000 7f0a4b7fe640 1 --2- 172.21.15.27:0/4120509413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a4c1553a0 0x7f0a34005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.025+0000 7f0a4b7fe640 1 -- 172.21.15.27:0/4120509413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a340047c0 msgr2=0x7f0a34004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.025+0000 7f0a4b7fe640 1 --2- 172.21.15.27:0/4120509413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a340047c0 0x7f0a34004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.025+0000 7f0a4b7fe640 1 -- 172.21.15.27:0/4120509413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a34003520 con 0x7f0a340039a0 2023-12-19T18:16:26.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.025+0000 7f0a4affd640 1 --2- 172.21.15.27:0/4120509413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a340047c0 0x7f0a34004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:26.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.026+0000 7f0a4b7fe640 1 --2- 172.21.15.27:0/4120509413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a340039a0 0x7f0a34003dc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009ac0 tx=0x7f0a340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f00e4dbcb86cc3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:26.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.026+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c02c710 con 0x7f0a340039a0 2023-12-19T18:16:26.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.026+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c02c870 con 0x7f0a340039a0 2023-12-19T18:16:26.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.026+0000 7f0a4a7fc640 1 -- 172.21.15.27:0/4120509413 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c02cc00 con 0x7f0a340039a0 2023-12-19T18:16:26.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.027+0000 7f0a50a22640 1 -- 172.21.15.27:0/4120509413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a340039a0 msgr2=0x7f0a34003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.027+0000 7f0a50a22640 1 --2- 172.21.15.27:0/4120509413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a340039a0 0x7f0a34003dc0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009ac0 tx=0x7f0a340057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.027+0000 7f0a4bfff640 1 -- 172.21.15.27:0/4120509413 reap_dead start 2023-12-19T18:16:26.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.027+0000 7f0a50a22640 1 -- 172.21.15.27:0/4120509413 shutdown_connections 2023-12-19T18:16:26.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.027+0000 7f0a50a22640 1 -- 172.21.15.27:0/4120509413 >> 172.21.15.27:0/4120509413 conn(0x7f0a4c069c70 msgr2=0x7f0a4c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:26.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.027+0000 7f0a50a22640 1 -- 172.21.15.27:0/4120509413 shutdown_connections 2023-12-19T18:16:26.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.027+0000 7f0a50a22640 1 -- 172.21.15.27:0/4120509413 wait complete. 2023-12-19T18:16:26.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.029+0000 7f0a50a22640 1 Processor -- start 2023-12-19T18:16:26.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.029+0000 7f0a50a22640 1 -- start start 2023-12-19T18:16:26.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a50a22640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a340039a0 0x7f0a4c154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a50a22640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a4c15cb10 0x7f0a4c0b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a50a22640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c0b23c0 0x7f0a4c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a50a22640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0b2840 con 0x7f0a4c0b23c0 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a50a22640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c154a50 con 0x7f0a4c15cb10 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a50a22640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a4c0b42d0 con 0x7f0a340039a0 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a4bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c0b23c0 0x7f0a4c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a4bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c0b23c0 0x7f0a4c0cd560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54142/0 (socket says 172.21.15.27:54142) 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a4bfff640 1 -- 172.21.15.27:0/705008109 learned_addr learned my addr 172.21.15.27:0/705008109 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a4b7fe640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a340039a0 0x7f0a4c154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a4bfff640 1 -- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a340039a0 msgr2=0x7f0a4c154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a4affd640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a4c15cb10 0x7f0a4c0b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a4bfff640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a340039a0 0x7f0a4c154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a4bfff640 1 -- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a4c15cb10 msgr2=0x7f0a4c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a4bfff640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a4c15cb10 0x7f0a4c0b1e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a4bfff640 1 -- 172.21.15.27:0/705008109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a4c15d6a0 con 0x7f0a4c0b23c0 2023-12-19T18:16:26.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.030+0000 7f0a4b7fe640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a340039a0 0x7f0a4c154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:26.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.031+0000 7f0a4bfff640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c0b23c0 0x7f0a4c0cd560 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0a44001370 tx=0x7f0a4400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:26.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.031+0000 7f0a33fff640 1 -- 172.21.15.27:0/705008109 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400a5b0 con 0x7f0a4c0b23c0 2023-12-19T18:16:26.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.031+0000 7f0a33fff640 1 -- 172.21.15.27:0/705008109 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4400a710 con 0x7f0a4c0b23c0 2023-12-19T18:16:26.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.031+0000 7f0a33fff640 1 -- 172.21.15.27:0/705008109 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4400aac0 con 0x7f0a4c0b23c0 2023-12-19T18:16:26.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.031+0000 7f0a50a22640 1 -- 172.21.15.27:0/705008109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4c15d980 con 0x7f0a4c0b23c0 2023-12-19T18:16:26.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.031+0000 7f0a50a22640 1 -- 172.21.15.27:0/705008109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a4c15df40 con 0x7f0a4c0b23c0 2023-12-19T18:16:26.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.032+0000 7f0a50a22640 1 -- 172.21.15.27:0/705008109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a4c04e840 con 0x7f0a4c0b23c0 2023-12-19T18:16:26.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.033+0000 7f0a33fff640 1 -- 172.21.15.27:0/705008109 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0a4400ac20 con 0x7f0a4c0b23c0 2023-12-19T18:16:26.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.035+0000 7f0a33fff640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a3806a430 0x7f0a3806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.035+0000 7f0a33fff640 1 -- 172.21.15.27:0/705008109 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a44088150 con 0x7f0a4c0b23c0 2023-12-19T18:16:26.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.036+0000 7f0a4b7fe640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a3806a430 0x7f0a3806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.037+0000 7f0a4b7fe640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a3806a430 0x7f0a3806c8f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009ac0 tx=0x7f0a3c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:26.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.040+0000 7f0a33fff640 1 -- 172.21.15.27:0/705008109 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0a440536a0 con 0x7f0a4c0b23c0 2023-12-19T18:16:26.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.240+0000 7f0a50a22640 1 -- 172.21.15.27:0/705008109 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a4c0c5470 con 0x7f0a4c0b23c0 2023-12-19T18:16:26.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.243+0000 7f0a33fff640 1 -- 172.21.15.27:0/705008109 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16054 (secure 0 0 0) 0x7f0a44056cc0 con 0x7f0a4c0b23c0 2023-12-19T18:16:26.244 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:26.244 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":33,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:16:25.779358+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:23.925853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:24.827898+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:26.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.246+0000 7f0a50a22640 1 -- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a3806a430 msgr2=0x7f0a3806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.246+0000 7f0a50a22640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a3806a430 0x7f0a3806c8f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c009ac0 tx=0x7f0a3c030000 comp rx=0 tx=0).stop 2023-12-19T18:16:26.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.246+0000 7f0a50a22640 1 -- 172.21.15.27:0/705008109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c0b23c0 msgr2=0x7f0a4c0cd560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.246+0000 7f0a50a22640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c0b23c0 0x7f0a4c0cd560 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f0a44001370 tx=0x7f0a4400ccd0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.246+0000 7f0a50a22640 1 -- 172.21.15.27:0/705008109 shutdown_connections 2023-12-19T18:16:26.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.246+0000 7f0a50a22640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a3806a430 0x7f0a3806c8f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.246+0000 7f0a50a22640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a4c0b23c0 0x7f0a4c0cd560 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.246+0000 7f0a50a22640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a4c15cb10 0x7f0a4c0b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.246+0000 7f0a50a22640 1 --2- 172.21.15.27:0/705008109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a340039a0 0x7f0a4c154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.246+0000 7f0a50a22640 1 -- 172.21.15.27:0/705008109 >> 172.21.15.27:0/705008109 conn(0x7f0a4c069c70 msgr2=0x7f0a4c159f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:26.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.246+0000 7f0a50a22640 1 -- 172.21.15.27:0/705008109 shutdown_connections 2023-12-19T18:16:26.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.246+0000 7f0a50a22640 1 -- 172.21.15.27:0/705008109 wait complete. 2023-12-19T18:16:26.267 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:26.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.490+0000 7fadb3e1e640 1 Processor -- start 2023-12-19T18:16:26.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.490+0000 7fadb3e1e640 1 -- start start 2023-12-19T18:16:26.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.492+0000 7fadb3e1e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fadac15af70 0x7fadac0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.492+0000 7fadb3e1e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fadac0cbce0 0x7fadac0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.492+0000 7fadb3e1e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadac0cb850 con 0x7fadac0ada50 2023-12-19T18:16:26.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.492+0000 7fadb3e1e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadac15b350 con 0x7fadac15af70 2023-12-19T18:16:26.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.492+0000 7fadb3e1e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadac0cac40 con 0x7fadac0cbce0 2023-12-19T18:16:26.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.492+0000 7fadb1bba640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fadac15af70 0x7fadac0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.492+0000 7fadb23bb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fadac0ada50 0x7fadac0c9190 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56680/0 (socket says 172.21.15.27:56680) 2023-12-19T18:16:26.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.492+0000 7fadb23bb640 1 -- 172.21.15.27:0/3121776680 learned_addr learned my addr 172.21.15.27:0/3121776680 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:26.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.493+0000 7fadb13b9640 1 --2- 172.21.15.27:0/3121776680 >> v2:172.21.15.192:3300/0 conn(0x7fadac0cbce0 0x7fadac0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.493+0000 7fadb1bba640 1 -- 172.21.15.27:0/3121776680 >> v2:172.21.15.192:3300/0 conn(0x7fadac0cbce0 msgr2=0x7fadac0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:26.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.493+0000 7fadb1bba640 1 --2- 172.21.15.27:0/3121776680 >> v2:172.21.15.192:3300/0 conn(0x7fadac0cbce0 0x7fadac0ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.493+0000 7fadb1bba640 1 -- 172.21.15.27:0/3121776680 >> v1:172.21.15.27:6789/0 conn(0x7fadac0ada50 legacy=0x7fadac0c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.493+0000 7fadb1bba640 1 -- 172.21.15.27:0/3121776680 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadac0caf20 con 0x7fadac15af70 2023-12-19T18:16:26.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.493+0000 7fadb0bb8640 1 -- 172.21.15.27:0/3121776680 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 845485010 0 0) 0x7fadac0cb850 con 0x7fadac0ada50 2023-12-19T18:16:26.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.494+0000 7fadb1bba640 1 --2- 172.21.15.27:0/3121776680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadac15af70 0x7fadac0ca540 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fadac0cc620 tx=0x7fada802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53735cee281e608 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:26.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.494+0000 7fadb0bb8640 1 -- 172.21.15.27:0/3121776680 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada8031830 con 0x7fadac15af70 2023-12-19T18:16:26.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.494+0000 7fadb0bb8640 1 -- 172.21.15.27:0/3121776680 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada8031990 con 0x7fadac15af70 2023-12-19T18:16:26.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.494+0000 7fadb0bb8640 1 -- 172.21.15.27:0/3121776680 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada8031d20 con 0x7fadac15af70 2023-12-19T18:16:26.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.495+0000 7fadb3e1e640 1 -- 172.21.15.27:0/3121776680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadac15af70 msgr2=0x7fadac0ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.495+0000 7fadb3e1e640 1 --2- 172.21.15.27:0/3121776680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadac15af70 0x7fadac0ca540 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fadac0cc620 tx=0x7fada802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.495+0000 7fadb3e1e640 1 -- 172.21.15.27:0/3121776680 shutdown_connections 2023-12-19T18:16:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.495+0000 7fadb3e1e640 1 --2- 172.21.15.27:0/3121776680 >> v2:172.21.15.192:3300/0 conn(0x7fadac0cbce0 0x7fadac0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.495+0000 7fadb3e1e640 1 --2- 172.21.15.27:0/3121776680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadac15af70 0x7fadac0ca540 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.495+0000 7fadb3e1e640 1 -- 172.21.15.27:0/3121776680 >> 172.21.15.27:0/3121776680 conn(0x7fadac06a330 msgr2=0x7fadac0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.495+0000 7fadb3e1e640 1 -- 172.21.15.27:0/3121776680 shutdown_connections 2023-12-19T18:16:26.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.495+0000 7fadb3e1e640 1 -- 172.21.15.27:0/3121776680 wait complete. 2023-12-19T18:16:26.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.497+0000 7fadb3e1e640 1 Processor -- start 2023-12-19T18:16:26.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.497+0000 7fadb3e1e640 1 -- start start 2023-12-19T18:16:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.498+0000 7fadb3e1e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadac0ada50 0x7fadac0c4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.498+0000 7fadb3e1e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadac0cbce0 0x7fadac0c8470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.498+0000 7fadb3e1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadac15af70 0x7fadac1f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.498+0000 7fadb3e1e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac0cd660 con 0x7fadac15af70 2023-12-19T18:16:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.498+0000 7fadb3e1e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadac0cc9e0 con 0x7fadac0ada50 2023-12-19T18:16:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.498+0000 7fadb3e1e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadac0cccc0 con 0x7fadac0cbce0 2023-12-19T18:16:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.499+0000 7fadb23bb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadac15af70 0x7fadac1f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.499+0000 7fadb23bb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadac15af70 0x7fadac1f2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54164/0 (socket says 172.21.15.27:54164) 2023-12-19T18:16:26.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.499+0000 7fadb23bb640 1 -- 172.21.15.27:0/630632306 learned_addr learned my addr 172.21.15.27:0/630632306 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.499+0000 7fadb1bba640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadac0ada50 0x7fadac0c4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.499+0000 7fadb13b9640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadac0cbce0 0x7fadac0c8470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.499+0000 7fadb23bb640 1 -- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadac0cbce0 msgr2=0x7fadac0c8470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.499+0000 7fadb23bb640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadac0cbce0 0x7fadac0c8470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.499+0000 7fadb23bb640 1 -- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadac0ada50 msgr2=0x7fadac0c4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.499+0000 7fadb23bb640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadac0ada50 0x7fadac0c4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.499+0000 7fadb23bb640 1 -- 172.21.15.27:0/630632306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fada8009400 con 0x7fadac15af70 2023-12-19T18:16:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.499+0000 7fadb1bba640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadac0ada50 0x7fadac0c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.500+0000 7fadb23bb640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadac15af70 0x7fadac1f2b00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fada400c7a0 tx=0x7fada400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.500+0000 7fada27fc640 1 -- 172.21.15.27:0/630632306 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada400a640 con 0x7fadac15af70 2023-12-19T18:16:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.500+0000 7fada27fc640 1 -- 172.21.15.27:0/630632306 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fada400a7a0 con 0x7fadac15af70 2023-12-19T18:16:26.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.500+0000 7fada27fc640 1 -- 172.21.15.27:0/630632306 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fada400abd0 con 0x7fadac15af70 2023-12-19T18:16:26.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.500+0000 7fadb3e1e640 1 -- 172.21.15.27:0/630632306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadac0ccfa0 con 0x7fadac15af70 2023-12-19T18:16:26.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.500+0000 7fadb3e1e640 1 -- 172.21.15.27:0/630632306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadac0c5260 con 0x7fadac15af70 2023-12-19T18:16:26.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.502+0000 7fadb3e1e640 1 -- 172.21.15.27:0/630632306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadac0cd7a0 con 0x7fadac15af70 2023-12-19T18:16:26.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.503+0000 7fada27fc640 1 -- 172.21.15.27:0/630632306 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fada400ad30 con 0x7fadac15af70 2023-12-19T18:16:26.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.503+0000 7fada27fc640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fad9006a430 0x7fad9006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.504+0000 7fada27fc640 1 -- 172.21.15.27:0/630632306 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fada4087e30 con 0x7fadac15af70 2023-12-19T18:16:26.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.504+0000 7fadb1bba640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fad9006a430 0x7fad9006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.505+0000 7fadb1bba640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fad9006a430 0x7fad9006c8f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fada8036000 tx=0x7fada8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:26.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.508+0000 7fada27fc640 1 -- 172.21.15.27:0/630632306 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fadac0cd7a0 con 0x7fadac15af70 2023-12-19T18:16:26.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.702+0000 7fadb3e1e640 1 -- 172.21.15.27:0/630632306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadac0cd7a0 con 0x7fadac15af70 2023-12-19T18:16:26.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.703+0000 7fada27fc640 1 -- 172.21.15.27:0/630632306 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16054 (secure 0 0 0) 0x7fadac0cd7a0 con 0x7fadac15af70 2023-12-19T18:16:26.704 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:26.704 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":33,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:16:25.779358+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:23.925853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:24.827898+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:26.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.706+0000 7fadb3e1e640 1 -- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fad9006a430 msgr2=0x7fad9006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.706+0000 7fadb3e1e640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fad9006a430 0x7fad9006c8f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fada8036000 tx=0x7fada8008030 comp rx=0 tx=0).stop 2023-12-19T18:16:26.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.706+0000 7fadb3e1e640 1 -- 172.21.15.27:0/630632306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadac15af70 msgr2=0x7fadac1f2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.706+0000 7fadb3e1e640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadac15af70 0x7fadac1f2b00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fada400c7a0 tx=0x7fada400cc60 comp rx=0 tx=0).stop 2023-12-19T18:16:26.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.706+0000 7fadb3e1e640 1 -- 172.21.15.27:0/630632306 shutdown_connections 2023-12-19T18:16:26.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.706+0000 7fadb3e1e640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fad9006a430 0x7fad9006c8f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.706+0000 7fadb3e1e640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadac15af70 0x7fadac1f2b00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.706+0000 7fadb3e1e640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadac0cbce0 0x7fadac0c8470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.706+0000 7fadb3e1e640 1 --2- 172.21.15.27:0/630632306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadac0ada50 0x7fadac0c4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.706+0000 7fadb3e1e640 1 -- 172.21.15.27:0/630632306 >> 172.21.15.27:0/630632306 conn(0x7fadac06a330 msgr2=0x7fadac15a240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:26.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.706+0000 7fadb3e1e640 1 -- 172.21.15.27:0/630632306 shutdown_connections 2023-12-19T18:16:26.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.706+0000 7fadb3e1e640 1 -- 172.21.15.27:0/630632306 wait complete. 2023-12-19T18:16:26.723 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:16:26.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.915+0000 7f421d5c9640 1 Processor -- start 2023-12-19T18:16:26.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.915+0000 7f421d5c9640 1 -- start start 2023-12-19T18:16:26.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.917+0000 7f421d5c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421815b1a0 0x7f421806f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.917+0000 7f421d5c9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4218071420 0x7f421815ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.917+0000 7f421d5c9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f421806bc80 con 0x7f42180c2720 2023-12-19T18:16:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.917+0000 7f421d5c9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f421806d690 con 0x7f4218071420 2023-12-19T18:16:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.917+0000 7f421d5c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f421806d0a0 con 0x7f421815b1a0 2023-12-19T18:16:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.917+0000 7f42177fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4218071420 0x7f421815ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.917+0000 7f42177fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4218071420 0x7f421815ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54170/0 (socket says 172.21.15.27:54170) 2023-12-19T18:16:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.917+0000 7f42177fe640 1 -- 172.21.15.27:0/140703515 learned_addr learned my addr 172.21.15.27:0/140703515 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.918+0000 7f4216ffd640 1 --2- 172.21.15.27:0/140703515 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421815b1a0 0x7f421806f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.918+0000 7f42177fe640 1 -- 172.21.15.27:0/140703515 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421815b1a0 msgr2=0x7f421806f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:26.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.918+0000 7f42177fe640 1 --2- 172.21.15.27:0/140703515 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421815b1a0 0x7f421806f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.918+0000 7f42177fe640 1 -- 172.21.15.27:0/140703515 >> v1:172.21.15.192:6789/0 conn(0x7f42180c2720 legacy=0x7f421806fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:26.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.918+0000 7f42177fe640 1 -- 172.21.15.27:0/140703515 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f421806b6c0 con 0x7f4218071420 2023-12-19T18:16:26.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.919+0000 7f42177fe640 1 --2- 172.21.15.27:0/140703515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4218071420 0x7f421815ed80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4210009e50 tx=0x7f421002be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d15ad5720298e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:26.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.919+0000 7f4215ffb640 1 -- 172.21.15.27:0/140703515 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421002c9e0 con 0x7f4218071420 2023-12-19T18:16:26.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.919+0000 7f4215ffb640 1 -- 172.21.15.27:0/140703515 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421002cb40 con 0x7f4218071420 2023-12-19T18:16:26.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.919+0000 7f4215ffb640 1 -- 172.21.15.27:0/140703515 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42100343f0 con 0x7f4218071420 2023-12-19T18:16:26.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.920+0000 7f421d5c9640 1 -- 172.21.15.27:0/140703515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4218071420 msgr2=0x7f421815ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.920+0000 7f421d5c9640 1 --2- 172.21.15.27:0/140703515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4218071420 0x7f421815ed80 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f4210009e50 tx=0x7f421002be40 comp rx=0 tx=0).stop 2023-12-19T18:16:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.920+0000 7f421d5c9640 1 -- 172.21.15.27:0/140703515 shutdown_connections 2023-12-19T18:16:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.920+0000 7f421d5c9640 1 --2- 172.21.15.27:0/140703515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4218071420 0x7f421815ed80 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.920+0000 7f421d5c9640 1 --2- 172.21.15.27:0/140703515 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f421815b1a0 0x7f421806f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.920+0000 7f421d5c9640 1 -- 172.21.15.27:0/140703515 >> 172.21.15.27:0/140703515 conn(0x7f42180cebd0 msgr2=0x7f42180cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.920+0000 7f421d5c9640 1 -- 172.21.15.27:0/140703515 shutdown_connections 2023-12-19T18:16:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.920+0000 7f421d5c9640 1 -- 172.21.15.27:0/140703515 wait complete. 2023-12-19T18:16:26.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.922+0000 7f421d5c9640 1 Processor -- start 2023-12-19T18:16:26.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.922+0000 7f421d5c9640 1 -- start start 2023-12-19T18:16:26.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.923+0000 7f421d5c9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4218071420 0x7f42181f2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.923+0000 7f421d5c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42180c2720 0x7f42181f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.923+0000 7f421d5c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f421815b1a0 0x7f42180c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.923+0000 7f421d5c9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42181f3be0 con 0x7f421815b1a0 2023-12-19T18:16:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.923+0000 7f421d5c9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42180c89c0 con 0x7f4218071420 2023-12-19T18:16:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.923+0000 7f421d5c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42181f2f80 con 0x7f42180c2720 2023-12-19T18:16:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.924+0000 7f42177fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f421815b1a0 0x7f42180c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.924+0000 7f42177fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f421815b1a0 0x7f42180c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54176/0 (socket says 172.21.15.27:54176) 2023-12-19T18:16:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.924+0000 7f42177fe640 1 -- 172.21.15.27:0/399059537 learned_addr learned my addr 172.21.15.27:0/399059537 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.924+0000 7f4216ffd640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4218071420 0x7f42181f2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.924+0000 7f42167fc640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42180c2720 0x7f42181f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.924+0000 7f42177fe640 1 -- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42180c2720 msgr2=0x7f42181f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.924+0000 7f42177fe640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42180c2720 0x7f42181f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.924+0000 7f42177fe640 1 -- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4218071420 msgr2=0x7f42181f2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.924+0000 7f42177fe640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4218071420 0x7f42181f2880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.924+0000 7f42177fe640 1 -- 172.21.15.27:0/399059537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4210009b30 con 0x7f421815b1a0 2023-12-19T18:16:26.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.924+0000 7f4216ffd640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4218071420 0x7f42181f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:26.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.924+0000 7f42167fc640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42180c2720 0x7f42181f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:26.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.925+0000 7f42177fe640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f421815b1a0 0x7f42180c8480 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4210036040 tx=0x7f4210004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:26.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.925+0000 7f41ff7fe640 1 -- 172.21.15.27:0/399059537 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4210009510 con 0x7f421815b1a0 2023-12-19T18:16:26.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.925+0000 7f41ff7fe640 1 -- 172.21.15.27:0/399059537 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f421003e450 con 0x7f421815b1a0 2023-12-19T18:16:26.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.925+0000 7f41ff7fe640 1 -- 172.21.15.27:0/399059537 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f421003e7e0 con 0x7f421815b1a0 2023-12-19T18:16:26.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.925+0000 7f421d5c9640 1 -- 172.21.15.27:0/399059537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42181f3260 con 0x7f421815b1a0 2023-12-19T18:16:26.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.925+0000 7f421d5c9640 1 -- 172.21.15.27:0/399059537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42181f3820 con 0x7f421815b1a0 2023-12-19T18:16:26.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.926+0000 7f421d5c9640 1 -- 172.21.15.27:0/399059537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42181f39b0 con 0x7f421815b1a0 2023-12-19T18:16:26.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.927+0000 7f41ff7fe640 1 -- 172.21.15.27:0/399059537 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f421003e940 con 0x7f421815b1a0 2023-12-19T18:16:26.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.928+0000 7f41ff7fe640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f420406a430 0x7f420406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:26.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.928+0000 7f41ff7fe640 1 -- 172.21.15.27:0/399059537 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f42100ab550 con 0x7f421815b1a0 2023-12-19T18:16:26.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.929+0000 7f4216ffd640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f420406a430 0x7f420406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:26.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.930+0000 7f4216ffd640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f420406a430 0x7f420406c8f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4208006fd0 tx=0x7f4208008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:26.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:26.933+0000 7f41ff7fe640 1 -- 172.21.15.27:0/399059537 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f42181f39b0 con 0x7f421815b1a0 2023-12-19T18:16:27.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.167+0000 7f421d5c9640 1 -- 172.21.15.27:0/399059537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f421806bc80 con 0x7f421815b1a0 2023-12-19T18:16:27.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.792+0000 7f41ff7fe640 1 -- 172.21.15.27:0/399059537 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v14) v1 ==== 118+0+0 (secure 0 0 0) 0x7f421806bc80 con 0x7f421815b1a0 2023-12-19T18:16:27.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.796+0000 7f421d5c9640 1 -- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f420406a430 msgr2=0x7f420406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:27.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.796+0000 7f421d5c9640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f420406a430 0x7f420406c8f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f4208006fd0 tx=0x7f4208008040 comp rx=0 tx=0).stop 2023-12-19T18:16:27.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.796+0000 7f421d5c9640 1 -- 172.21.15.27:0/399059537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f421815b1a0 msgr2=0x7f42180c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:27.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.796+0000 7f421d5c9640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f421815b1a0 0x7f42180c8480 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4210036040 tx=0x7f4210004a40 comp rx=0 tx=0).stop 2023-12-19T18:16:27.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.796+0000 7f421d5c9640 1 -- 172.21.15.27:0/399059537 shutdown_connections 2023-12-19T18:16:27.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.796+0000 7f421d5c9640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f420406a430 0x7f420406c8f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.796+0000 7f421d5c9640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f421815b1a0 0x7f42180c8480 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.796+0000 7f421d5c9640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42180c2720 0x7f42181f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.796+0000 7f421d5c9640 1 --2- 172.21.15.27:0/399059537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4218071420 0x7f42181f2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:27.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.796+0000 7f421d5c9640 1 -- 172.21.15.27:0/399059537 >> 172.21.15.27:0/399059537 conn(0x7f42180cebd0 msgr2=0x7f42180c1b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:27.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.797+0000 7f421d5c9640 1 -- 172.21.15.27:0/399059537 shutdown_connections 2023-12-19T18:16:27.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:27.797+0000 7f421d5c9640 1 -- 172.21.15.27:0/399059537 wait complete. 2023-12-19T18:16:27.818 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:16:27.818 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:16:27.832 INFO:tasks.ceph.mds.a.smithi027.stderr:2023-12-19T18:16:27.831+0000 7f3e248aa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:16:28.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.056+0000 7fbcdd654640 1 Processor -- start 2023-12-19T18:16:28.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.056+0000 7fbcdd654640 1 -- start start 2023-12-19T18:16:28.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.058+0000 7fbcdd654640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8158790 0x7fbcd80ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.058+0000 7fbcdd654640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbcd80adc50 0x7fbcd815b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.058+0000 7fbcdd654640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd815c030 con 0x7fbcd815b810 2023-12-19T18:16:28.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.058+0000 7fbcdd654640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd80c99d0 con 0x7fbcd80adc50 2023-12-19T18:16:28.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.058+0000 7fbcdd654640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd8158b70 con 0x7fbcd8158790 2023-12-19T18:16:28.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.058+0000 7fbcd7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8158790 0x7fbcd80ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.059+0000 7fbcd7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8158790 0x7fbcd80ad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:41300/0 (socket says 172.21.15.27:41300) 2023-12-19T18:16:28.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.059+0000 7fbcd7fff640 1 -- 172.21.15.27:0/1202934833 learned_addr learned my addr 172.21.15.27:0/1202934833 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:28.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.059+0000 7fbcd77fe640 1 --2- 172.21.15.27:0/1202934833 >> v2:172.21.15.192:3300/0 conn(0x7fbcd80adc50 0x7fbcd815b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.059+0000 7fbcd7fff640 1 -- 172.21.15.27:0/1202934833 >> v2:172.21.15.192:3300/0 conn(0x7fbcd80adc50 msgr2=0x7fbcd815b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:28.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.059+0000 7fbcd7fff640 1 --2- 172.21.15.27:0/1202934833 >> v2:172.21.15.192:3300/0 conn(0x7fbcd80adc50 0x7fbcd815b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.059+0000 7fbcd7fff640 1 -- 172.21.15.27:0/1202934833 >> v1:172.21.15.192:6789/0 conn(0x7fbcd815b810 legacy=0x7fbcd815e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.059+0000 7fbcd7fff640 1 -- 172.21.15.27:0/1202934833 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd80c9030 con 0x7fbcd8158790 2023-12-19T18:16:28.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.061+0000 7fbcd7fff640 1 --2- 172.21.15.27:0/1202934833 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8158790 0x7fbcd80ad710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009e50 tx=0x7fbcc802bf60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c86a1a0dba03ce77 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:28.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.061+0000 7fbcd6ffd640 1 -- 172.21.15.27:0/1202934833 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc802cb00 con 0x7fbcd8158790 2023-12-19T18:16:28.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.061+0000 7fbcd6ffd640 1 -- 172.21.15.27:0/1202934833 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc802cc60 con 0x7fbcd8158790 2023-12-19T18:16:28.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.061+0000 7fbcd6ffd640 1 -- 172.21.15.27:0/1202934833 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc80343f0 con 0x7fbcd8158790 2023-12-19T18:16:28.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.062+0000 7fbcdd654640 1 -- 172.21.15.27:0/1202934833 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8158790 msgr2=0x7fbcd80ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.062+0000 7fbcdd654640 1 --2- 172.21.15.27:0/1202934833 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8158790 0x7fbcd80ad710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8009e50 tx=0x7fbcc802bf60 comp rx=0 tx=0).stop 2023-12-19T18:16:28.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.062+0000 7fbcdd654640 1 -- 172.21.15.27:0/1202934833 shutdown_connections 2023-12-19T18:16:28.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.062+0000 7fbcdd654640 1 --2- 172.21.15.27:0/1202934833 >> v2:172.21.15.192:3300/0 conn(0x7fbcd80adc50 0x7fbcd815b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.062+0000 7fbcdd654640 1 --2- 172.21.15.27:0/1202934833 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8158790 0x7fbcd80ad710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.062+0000 7fbcdd654640 1 -- 172.21.15.27:0/1202934833 >> 172.21.15.27:0/1202934833 conn(0x7fbcd80b05b0 msgr2=0x7fbcd80b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:28.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.062+0000 7fbcdd654640 1 -- 172.21.15.27:0/1202934833 shutdown_connections 2023-12-19T18:16:28.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.062+0000 7fbcdd654640 1 -- 172.21.15.27:0/1202934833 wait complete. 2023-12-19T18:16:28.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.064+0000 7fbcdd654640 1 Processor -- start 2023-12-19T18:16:28.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.064+0000 7fbcdd654640 1 -- start start 2023-12-19T18:16:28.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.065+0000 7fbcdd654640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80adc50 0x7fbcd80cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.065+0000 7fbcdd654640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8158790 0x7fbcd806b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.065+0000 7fbcdd654640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcd815b810 0x7fbcd80cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.065+0000 7fbcdd654640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd814bf10 con 0x7fbcd80adc50 2023-12-19T18:16:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.065+0000 7fbcdd654640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd8152ef0 con 0x7fbcd815b810 2023-12-19T18:16:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.065+0000 7fbcdd654640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd80ccf90 con 0x7fbcd8158790 2023-12-19T18:16:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.065+0000 7fbcd7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80adc50 0x7fbcd80cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.066+0000 7fbcd7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80adc50 0x7fbcd80cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54178/0 (socket says 172.21.15.27:54178) 2023-12-19T18:16:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.066+0000 7fbcd7fff640 1 -- 172.21.15.27:0/3248155495 learned_addr learned my addr 172.21.15.27:0/3248155495 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.066+0000 7fbcd77fe640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8158790 0x7fbcd806b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.066+0000 7fbcdce53640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcd815b810 0x7fbcd80cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.066+0000 7fbcd7fff640 1 -- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8158790 msgr2=0x7fbcd806b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.066+0000 7fbcd7fff640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8158790 0x7fbcd806b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.066+0000 7fbcd7fff640 1 -- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcd815b810 msgr2=0x7fbcd80cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.066+0000 7fbcd7fff640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcd815b810 0x7fbcd80cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.066+0000 7fbcd7fff640 1 -- 172.21.15.27:0/3248155495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc8009b30 con 0x7fbcd80adc50 2023-12-19T18:16:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.066+0000 7fbcd77fe640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8158790 0x7fbcd806b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.066+0000 7fbcd7fff640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80adc50 0x7fbcd80cdf70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8036040 tx=0x7fbcc8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.067+0000 7fbcd4ff9640 1 -- 172.21.15.27:0/3248155495 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc8009510 con 0x7fbcd80adc50 2023-12-19T18:16:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.067+0000 7fbcd4ff9640 1 -- 172.21.15.27:0/3248155495 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc803e450 con 0x7fbcd80adc50 2023-12-19T18:16:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.067+0000 7fbcd4ff9640 1 -- 172.21.15.27:0/3248155495 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc803e800 con 0x7fbcd80adc50 2023-12-19T18:16:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.067+0000 7fbcdd654640 1 -- 172.21.15.27:0/3248155495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd80cd270 con 0x7fbcd80adc50 2023-12-19T18:16:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.067+0000 7fbcdd654640 1 -- 172.21.15.27:0/3248155495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd8152b80 con 0x7fbcd80adc50 2023-12-19T18:16:28.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.068+0000 7fbcdd654640 1 -- 172.21.15.27:0/3248155495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd804e840 con 0x7fbcd80adc50 2023-12-19T18:16:28.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.069+0000 7fbcd4ff9640 1 -- 172.21.15.27:0/3248155495 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbcc803e960 con 0x7fbcd80adc50 2023-12-19T18:16:28.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.070+0000 7fbcd4ff9640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbcb806a460 0x7fbcb806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:28.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.070+0000 7fbcd4ff9640 1 -- 172.21.15.27:0/3248155495 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbcc80ab470 con 0x7fbcd80adc50 2023-12-19T18:16:28.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.071+0000 7fbcd77fe640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbcb806a460 0x7fbcb806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:28.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.072+0000 7fbcd77fe640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbcb806a460 0x7fbcb806c920 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbcd806c610 tx=0x7fbcc0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:28.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.074+0000 7fbcd4ff9640 1 -- 172.21.15.27:0/3248155495 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbcc80769d0 con 0x7fbcd80adc50 2023-12-19T18:16:28.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:28.305+0000 7fbcdd654640 1 -- 172.21.15.27:0/3248155495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbcd815aaf0 con 0x7fbcd80adc50 2023-12-19T18:16:29.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.310+0000 7fbcd4ff9640 1 -- 172.21.15.27:0/3248155495 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v15) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbcc8079ff0 con 0x7fbcd80adc50 2023-12-19T18:16:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.313+0000 7fbcdd654640 1 -- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbcb806a460 msgr2=0x7fbcb806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.313+0000 7fbcdd654640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbcb806a460 0x7fbcb806c920 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fbcd806c610 tx=0x7fbcc0008040 comp rx=0 tx=0).stop 2023-12-19T18:16:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.313+0000 7fbcdd654640 1 -- 172.21.15.27:0/3248155495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80adc50 msgr2=0x7fbcd80cdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.313+0000 7fbcdd654640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80adc50 0x7fbcd80cdf70 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8036040 tx=0x7fbcc8004a40 comp rx=0 tx=0).stop 2023-12-19T18:16:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.313+0000 7fbcdd654640 1 -- 172.21.15.27:0/3248155495 shutdown_connections 2023-12-19T18:16:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.313+0000 7fbcdd654640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbcb806a460 0x7fbcb806c920 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.313+0000 7fbcdd654640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcd815b810 0x7fbcd80cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.313+0000 7fbcdd654640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcd8158790 0x7fbcd806b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.313+0000 7fbcdd654640 1 --2- 172.21.15.27:0/3248155495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80adc50 0x7fbcd80cdf70 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.313+0000 7fbcdd654640 1 -- 172.21.15.27:0/3248155495 >> 172.21.15.27:0/3248155495 conn(0x7fbcd80b05b0 msgr2=0x7fbcd80af0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:29.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.314+0000 7fbcdd654640 1 -- 172.21.15.27:0/3248155495 shutdown_connections 2023-12-19T18:16:29.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.314+0000 7fbcdd654640 1 -- 172.21.15.27:0/3248155495 wait complete. 2023-12-19T18:16:29.338 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:16:29.338 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:29.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.623+0000 7fe8ca999640 1 Processor -- start 2023-12-19T18:16:29.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.623+0000 7fe8ca999640 1 -- start start 2023-12-19T18:16:29.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.625+0000 7fe8ca999640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe8c41604d0 0x7fe8c40b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:29.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.625+0000 7fe8ca999640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c40b5e90 con 0x7fe8c40b7310 2023-12-19T18:16:29.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.625+0000 7fe8ca999640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8c40d1750 con 0x7fe8c40d2350 2023-12-19T18:16:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.625+0000 7fe8ca999640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8c40d2b30 con 0x7fe8c41604d0 2023-12-19T18:16:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.625+0000 7fe8c9997640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe8c41604d0 0x7fe8c40b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.625+0000 7fe8c9997640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe8c41604d0 0x7fe8c40b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54186/0 (socket says 172.21.15.27:54186) 2023-12-19T18:16:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.625+0000 7fe8c9997640 1 -- 172.21.15.27:0/526834635 learned_addr learned my addr 172.21.15.27:0/526834635 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.626+0000 7fe8c9997640 1 -- 172.21.15.27:0/526834635 >> v1:172.21.15.192:6789/0 conn(0x7fe8c40d2350 legacy=0x7fe8c40d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.626+0000 7fe8c9997640 1 -- 172.21.15.27:0/526834635 >> v1:172.21.15.27:6789/0 conn(0x7fe8c40b7310 legacy=0x7fe8c40d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.626+0000 7fe8c9997640 1 -- 172.21.15.27:0/526834635 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c40d2850 con 0x7fe8c41604d0 2023-12-19T18:16:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.627+0000 7fe8c9997640 1 --2- 172.21.15.27:0/526834635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c41604d0 0x7fe8c40b47e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009e50 tx=0x7fe8b802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c3978deb183485 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.627+0000 7fe8c8995640 1 -- 172.21.15.27:0/526834635 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b802ca80 con 0x7fe8c41604d0 2023-12-19T18:16:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.627+0000 7fe8c8995640 1 -- 172.21.15.27:0/526834635 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b802cbe0 con 0x7fe8c41604d0 2023-12-19T18:16:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.627+0000 7fe8c8995640 1 -- 172.21.15.27:0/526834635 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b80343f0 con 0x7fe8c41604d0 2023-12-19T18:16:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.628+0000 7fe8ca999640 1 -- 172.21.15.27:0/526834635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c41604d0 msgr2=0x7fe8c40b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.628+0000 7fe8ca999640 1 --2- 172.21.15.27:0/526834635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c41604d0 0x7fe8c40b47e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8009e50 tx=0x7fe8b802bee0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.628+0000 7fe8ca999640 1 -- 172.21.15.27:0/526834635 shutdown_connections 2023-12-19T18:16:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.628+0000 7fe8ca999640 1 --2- 172.21.15.27:0/526834635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c41604d0 0x7fe8c40b47e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.628+0000 7fe8ca999640 1 -- 172.21.15.27:0/526834635 >> 172.21.15.27:0/526834635 conn(0x7fe8c40cf7e0 msgr2=0x7fe8c40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:29.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.628+0000 7fe8ca999640 1 -- 172.21.15.27:0/526834635 shutdown_connections 2023-12-19T18:16:29.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.628+0000 7fe8ca999640 1 -- 172.21.15.27:0/526834635 wait complete. 2023-12-19T18:16:29.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.630+0000 7fe8ca999640 1 Processor -- start 2023-12-19T18:16:29.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.630+0000 7fe8ca999640 1 -- start start 2023-12-19T18:16:29.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.631+0000 7fe8ca999640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8c40b7310 0x7fe8c415e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:29.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.631+0000 7fe8ca999640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8c40d2350 0x7fe8c40ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:29.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.631+0000 7fe8ca999640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c41604d0 0x7fe8c40b49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:29.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.631+0000 7fe8ca999640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c415ebf0 con 0x7fe8c41604d0 2023-12-19T18:16:29.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.631+0000 7fe8ca999640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c40a9840 con 0x7fe8c40b7310 2023-12-19T18:16:29.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.631+0000 7fe8ca999640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c40a9b20 con 0x7fe8c40d2350 2023-12-19T18:16:29.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.632+0000 7fe8ca198640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c41604d0 0x7fe8c40b49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:29.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.632+0000 7fe8ca198640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c41604d0 0x7fe8c40b49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54202/0 (socket says 172.21.15.27:54202) 2023-12-19T18:16:29.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.632+0000 7fe8ca198640 1 -- 172.21.15.27:0/1313115575 learned_addr learned my addr 172.21.15.27:0/1313115575 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:29.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.632+0000 7fe8c9997640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8c40b7310 0x7fe8c415e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:29.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.632+0000 7fe8c9196640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8c40d2350 0x7fe8c40ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:29.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.632+0000 7fe8ca198640 1 -- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8c40d2350 msgr2=0x7fe8c40ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:29.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.632+0000 7fe8ca198640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8c40d2350 0x7fe8c40ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.632+0000 7fe8ca198640 1 -- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8c40b7310 msgr2=0x7fe8c415e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:29.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.632+0000 7fe8ca198640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8c40b7310 0x7fe8c415e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.632+0000 7fe8ca198640 1 -- 172.21.15.27:0/1313115575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8009b30 con 0x7fe8c41604d0 2023-12-19T18:16:29.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.633+0000 7fe8ca198640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c41604d0 0x7fe8c40b49d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00c390 tx=0x7fe8bc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:29.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.633+0000 7fe8c27fc640 1 -- 172.21.15.27:0/1313115575 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc0052d0 con 0x7fe8c41604d0 2023-12-19T18:16:29.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.633+0000 7fe8c27fc640 1 -- 172.21.15.27:0/1313115575 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8bc005430 con 0x7fe8c41604d0 2023-12-19T18:16:29.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.633+0000 7fe8c27fc640 1 -- 172.21.15.27:0/1313115575 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc003810 con 0x7fe8c41604d0 2023-12-19T18:16:29.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.633+0000 7fe8ca999640 1 -- 172.21.15.27:0/1313115575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c40a9e00 con 0x7fe8c41604d0 2023-12-19T18:16:29.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.633+0000 7fe8ca999640 1 -- 172.21.15.27:0/1313115575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c40aa3c0 con 0x7fe8c41604d0 2023-12-19T18:16:29.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.634+0000 7fe8ca999640 1 -- 172.21.15.27:0/1313115575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c4066f80 con 0x7fe8c41604d0 2023-12-19T18:16:29.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.635+0000 7fe8c27fc640 1 -- 172.21.15.27:0/1313115575 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe8bc003970 con 0x7fe8c41604d0 2023-12-19T18:16:29.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.636+0000 7fe8c27fc640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe8a806a430 0x7fe8a806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:29.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.636+0000 7fe8c27fc640 1 -- 172.21.15.27:0/1313115575 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8bc087d80 con 0x7fe8c41604d0 2023-12-19T18:16:29.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.637+0000 7fe8c9997640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe8a806a430 0x7fe8a806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:29.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.638+0000 7fe8c9997640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe8a806a430 0x7fe8a806c8f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8035000 tx=0x7fe8b8030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:29.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.640+0000 7fe8c27fc640 1 -- 172.21.15.27:0/1313115575 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe8bc053400 con 0x7fe8c41604d0 2023-12-19T18:16:29.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.873+0000 7fe8ca999640 1 -- 172.21.15.27:0/1313115575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe8c40aa5e0 con 0x7fe8c41604d0 2023-12-19T18:16:29.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.874+0000 7fe8c27fc640 1 -- 172.21.15.27:0/1313115575 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fe8bc056a20 con 0x7fe8c41604d0 2023-12-19T18:16:29.875 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:29.875 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3178168231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3178168231},{"type":"v1","addr":"172.21.15.192:6837","nonce":3178168231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3961149394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3961149394},{"type":"v1","addr":"172.21.15.192:6836","nonce":3961149394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5190,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/752784926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":752784926},{"type":"v1","addr":"172.21.15.27:6833","nonce":752784926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:25.779852+0000","modified":"2023-12-19T18:16:29.303970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/3463287343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3463287343},{"type":"v1","addr":"172.21.15.27:6835","nonce":3463287343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T18:16:29.875 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 15 2023-12-19T18:16:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.877+0000 7fe8ca999640 1 -- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe8a806a430 msgr2=0x7fe8a806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.877+0000 7fe8ca999640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe8a806a430 0x7fe8a806c8f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8035000 tx=0x7fe8b8030000 comp rx=0 tx=0).stop 2023-12-19T18:16:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.877+0000 7fe8ca999640 1 -- 172.21.15.27:0/1313115575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c41604d0 msgr2=0x7fe8c40b49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.877+0000 7fe8ca999640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c41604d0 0x7fe8c40b49d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00c390 tx=0x7fe8bc00c850 comp rx=0 tx=0).stop 2023-12-19T18:16:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.877+0000 7fe8ca999640 1 -- 172.21.15.27:0/1313115575 shutdown_connections 2023-12-19T18:16:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.877+0000 7fe8ca999640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe8a806a430 0x7fe8a806c8f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.877+0000 7fe8ca999640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8c41604d0 0x7fe8c40b49d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.877+0000 7fe8ca999640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8c40d2350 0x7fe8c40ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.877+0000 7fe8ca999640 1 --2- 172.21.15.27:0/1313115575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8c40b7310 0x7fe8c415e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:29.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.877+0000 7fe8ca999640 1 -- 172.21.15.27:0/1313115575 >> 172.21.15.27:0/1313115575 conn(0x7fe8c40cf7e0 msgr2=0x7fe8c40d0f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:29.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.877+0000 7fe8ca999640 1 -- 172.21.15.27:0/1313115575 shutdown_connections 2023-12-19T18:16:29.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:29.877+0000 7fe8ca999640 1 -- 172.21.15.27:0/1313115575 wait complete. 2023-12-19T18:16:29.895 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:30.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.117+0000 7f1f8720e640 1 Processor -- start 2023-12-19T18:16:30.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.117+0000 7f1f8720e640 1 -- start start 2023-12-19T18:16:30.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.119+0000 7f1f8720e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1f800cdf90 0x7f1f80164d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.119+0000 7f1f8720e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f800c4a20 con 0x7f1f80165290 2023-12-19T18:16:30.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.119+0000 7f1f8720e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8015bc50 con 0x7f1f8015b870 2023-12-19T18:16:30.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.119+0000 7f1f8720e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f80165740 con 0x7f1f800cdf90 2023-12-19T18:16:30.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.119+0000 7f1f86a0d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1f80165290 0x7f1f8006f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56686/0 (socket says 172.21.15.27:56686) 2023-12-19T18:16:30.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.119+0000 7f1f86a0d640 1 -- 172.21.15.27:0/2367902397 learned_addr learned my addr 172.21.15.27:0/2367902397 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:30.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.119+0000 7f1f85a0b640 1 --2- 172.21.15.27:0/2367902397 >> v2:172.21.15.27:3300/0 conn(0x7f1f800cdf90 0x7f1f80164d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.120+0000 7f1f8520a640 1 -- 172.21.15.27:0/2367902397 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050851771 0 0) 0x7f1f800c4a20 con 0x7f1f80165290 2023-12-19T18:16:30.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.120+0000 7f1f8520a640 1 -- 172.21.15.27:0/2367902397 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f6c003520 con 0x7f1f80165290 2023-12-19T18:16:30.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.120+0000 7f1f85a0b640 1 -- 172.21.15.27:0/2367902397 >> v1:172.21.15.192:6789/0 conn(0x7f1f8015b870 legacy=0x7f1f800c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.120+0000 7f1f85a0b640 1 -- 172.21.15.27:0/2367902397 >> v1:172.21.15.27:6789/0 conn(0x7f1f80165290 legacy=0x7f1f8006f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.120+0000 7f1f85a0b640 1 -- 172.21.15.27:0/2367902397 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f800c4eb0 con 0x7f1f800cdf90 2023-12-19T18:16:30.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.121+0000 7f1f85a0b640 1 --2- 172.21.15.27:0/2367902397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f800cdf90 0x7f1f80164d50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1f700097a0 tx=0x7f1f7002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22289506a434ace server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:30.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.121+0000 7f1f8520a640 1 -- 172.21.15.27:0/2367902397 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70030610 con 0x7f1f800cdf90 2023-12-19T18:16:30.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.121+0000 7f1f8520a640 1 -- 172.21.15.27:0/2367902397 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f70030770 con 0x7f1f800cdf90 2023-12-19T18:16:30.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.121+0000 7f1f8520a640 1 -- 172.21.15.27:0/2367902397 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70030ae0 con 0x7f1f800cdf90 2023-12-19T18:16:30.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.122+0000 7f1f8720e640 1 -- 172.21.15.27:0/2367902397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f800cdf90 msgr2=0x7f1f80164d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.122+0000 7f1f8720e640 1 --2- 172.21.15.27:0/2367902397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f800cdf90 0x7f1f80164d50 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f1f700097a0 tx=0x7f1f7002b960 comp rx=0 tx=0).stop 2023-12-19T18:16:30.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.122+0000 7f1f8720e640 1 -- 172.21.15.27:0/2367902397 shutdown_connections 2023-12-19T18:16:30.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.122+0000 7f1f8720e640 1 --2- 172.21.15.27:0/2367902397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f800cdf90 0x7f1f80164d50 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.122+0000 7f1f8720e640 1 -- 172.21.15.27:0/2367902397 >> 172.21.15.27:0/2367902397 conn(0x7f1f80069c70 msgr2=0x7f1f8006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:30.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.122+0000 7f1f8720e640 1 -- 172.21.15.27:0/2367902397 shutdown_connections 2023-12-19T18:16:30.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.122+0000 7f1f8720e640 1 -- 172.21.15.27:0/2367902397 wait complete. 2023-12-19T18:16:30.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.124+0000 7f1f8720e640 1 Processor -- start 2023-12-19T18:16:30.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.124+0000 7f1f8720e640 1 -- start start 2023-12-19T18:16:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.125+0000 7f1f8720e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f800cdf90 0x7f1f801f7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.125+0000 7f1f8720e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8015b870 0x7f1f801f99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.125+0000 7f1f8720e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f80165290 0x7f1f801598c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.125+0000 7f1f8720e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f801f7ba0 con 0x7f1f8015b870 2023-12-19T18:16:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.125+0000 7f1f8720e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f800c8070 con 0x7f1f80165290 2023-12-19T18:16:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.125+0000 7f1f8720e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f800c8350 con 0x7f1f800cdf90 2023-12-19T18:16:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.125+0000 7f1f85a0b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8015b870 0x7f1f801f99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.125+0000 7f1f85a0b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8015b870 0x7f1f801f99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54216/0 (socket says 172.21.15.27:54216) 2023-12-19T18:16:30.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.125+0000 7f1f85a0b640 1 -- 172.21.15.27:0/2363017649 learned_addr learned my addr 172.21.15.27:0/2363017649 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.125+0000 7f1f8620c640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f800cdf90 0x7f1f801f7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.126+0000 7f1f86a0d640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f80165290 0x7f1f801598c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.126+0000 7f1f85a0b640 1 -- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f800cdf90 msgr2=0x7f1f801f7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.126+0000 7f1f85a0b640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f800cdf90 0x7f1f801f7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.126+0000 7f1f85a0b640 1 -- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f80165290 msgr2=0x7f1f801598c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.126+0000 7f1f85a0b640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f80165290 0x7f1f801598c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.126+0000 7f1f85a0b640 1 -- 172.21.15.27:0/2363017649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f70009400 con 0x7f1f8015b870 2023-12-19T18:16:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.126+0000 7f1f85a0b640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8015b870 0x7f1f801f99b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1f70000c00 tx=0x7f1f700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.126+0000 7f1f76ffd640 1 -- 172.21.15.27:0/2363017649 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70039c90 con 0x7f1f8015b870 2023-12-19T18:16:30.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.126+0000 7f1f76ffd640 1 -- 172.21.15.27:0/2363017649 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7003f040 con 0x7f1f8015b870 2023-12-19T18:16:30.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.126+0000 7f1f8720e640 1 -- 172.21.15.27:0/2363017649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f800c8630 con 0x7f1f8015b870 2023-12-19T18:16:30.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.126+0000 7f1f76ffd640 1 -- 172.21.15.27:0/2363017649 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70006e10 con 0x7f1f8015b870 2023-12-19T18:16:30.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.127+0000 7f1f8720e640 1 -- 172.21.15.27:0/2363017649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f800c5e00 con 0x7f1f8015b870 2023-12-19T18:16:30.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.128+0000 7f1f8720e640 1 -- 172.21.15.27:0/2363017649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8004e840 con 0x7f1f8015b870 2023-12-19T18:16:30.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.129+0000 7f1f76ffd640 1 -- 172.21.15.27:0/2363017649 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1f70004030 con 0x7f1f8015b870 2023-12-19T18:16:30.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.130+0000 7f1f76ffd640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6406a430 0x7f1f6406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.130+0000 7f1f76ffd640 1 -- 172.21.15.27:0/2363017649 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f700aa8e0 con 0x7f1f8015b870 2023-12-19T18:16:30.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.131+0000 7f1f8620c640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6406a430 0x7f1f6406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.132+0000 7f1f8620c640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6406a430 0x7f1f6406c8f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f8015a4d0 tx=0x7f1f7c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:30.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.134+0000 7f1f76ffd640 1 -- 172.21.15.27:0/2363017649 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1f70075ec0 con 0x7f1f8015b870 2023-12-19T18:16:30.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.326+0000 7f1f8720e640 1 -- 172.21.15.27:0/2363017649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f801f7da0 con 0x7f1f8015b870 2023-12-19T18:16:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.329+0000 7f1f76ffd640 1 -- 172.21.15.27:0/2363017649 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f1f70048090 con 0x7f1f8015b870 2023-12-19T18:16:30.329 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:30.330 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":34,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:16:26.783935+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:23.925853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:24.827898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.331+0000 7f1f8720e640 1 -- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6406a430 msgr2=0x7f1f6406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.331+0000 7f1f8720e640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6406a430 0x7f1f6406c8f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1f8015a4d0 tx=0x7f1f7c009040 comp rx=0 tx=0).stop 2023-12-19T18:16:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.331+0000 7f1f8720e640 1 -- 172.21.15.27:0/2363017649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8015b870 msgr2=0x7f1f801f99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.331+0000 7f1f8720e640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8015b870 0x7f1f801f99b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f1f70000c00 tx=0x7f1f700046c0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.332+0000 7f1f8720e640 1 -- 172.21.15.27:0/2363017649 shutdown_connections 2023-12-19T18:16:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.332+0000 7f1f8720e640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6406a430 0x7f1f6406c8f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.332+0000 7f1f8720e640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f80165290 0x7f1f801598c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.332+0000 7f1f8720e640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f8015b870 0x7f1f801f99b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.332+0000 7f1f8720e640 1 --2- 172.21.15.27:0/2363017649 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f800cdf90 0x7f1f801f7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.332+0000 7f1f8720e640 1 -- 172.21.15.27:0/2363017649 >> 172.21.15.27:0/2363017649 conn(0x7f1f80069c70 msgr2=0x7f1f8006c140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:30.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.332+0000 7f1f8720e640 1 -- 172.21.15.27:0/2363017649 shutdown_connections 2023-12-19T18:16:30.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.332+0000 7f1f8720e640 1 -- 172.21.15.27:0/2363017649 wait complete. 2023-12-19T18:16:30.349 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:16:30.349 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:16:30.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.579+0000 7f4f2242c640 1 Processor -- start 2023-12-19T18:16:30.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.579+0000 7f4f2242c640 1 -- start start 2023-12-19T18:16:30.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.580+0000 7f4f2242c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4f1c15d240 0x7f4f1c1618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.581+0000 7f4f2242c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1c0d1970 con 0x7f4f1c0ce420 2023-12-19T18:16:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.581+0000 7f4f2242c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f1c161e10 con 0x7f4f1c0cdf90 2023-12-19T18:16:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.581+0000 7f4f2242c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f1c0b5a80 con 0x7f4f1c15d240 2023-12-19T18:16:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.581+0000 7f4f1bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4f1c15d240 0x7f4f1c1618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.581+0000 7f4f1bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4f1c15d240 0x7f4f1c1618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54218/0 (socket says 172.21.15.27:54218) 2023-12-19T18:16:30.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.581+0000 7f4f1bfff640 1 -- 172.21.15.27:0/2745286476 learned_addr learned my addr 172.21.15.27:0/2745286476 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.581+0000 7f4f1bfff640 1 -- 172.21.15.27:0/2745286476 >> v1:172.21.15.192:6789/0 conn(0x7f4f1c0cdf90 legacy=0x7f4f1c0b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.582+0000 7f4f1bfff640 1 -- 172.21.15.27:0/2745286476 >> v1:172.21.15.27:6789/0 conn(0x7f4f1c0ce420 legacy=0x7f4f1c0b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.582+0000 7f4f1bfff640 1 -- 172.21.15.27:0/2745286476 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c06a5e0 con 0x7f4f1c15d240 2023-12-19T18:16:30.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.582+0000 7f4f1affd640 1 -- 172.21.15.27:0/2745286476 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148770011 0 0) 0x7f4f1c0d1970 con 0x7f4f1c0ce420 2023-12-19T18:16:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.583+0000 7f4f1bfff640 1 --2- 172.21.15.27:0/2745286476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f1c15d240 0x7f4f1c1618d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009e50 tx=0x7f4f0c02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d228939183476a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.583+0000 7f4f1affd640 1 -- 172.21.15.27:0/2745286476 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c02cbb0 con 0x7f4f1c15d240 2023-12-19T18:16:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.583+0000 7f4f1affd640 1 -- 172.21.15.27:0/2745286476 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f0c02cd10 con 0x7f4f1c15d240 2023-12-19T18:16:30.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.583+0000 7f4f1affd640 1 -- 172.21.15.27:0/2745286476 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c034610 con 0x7f4f1c15d240 2023-12-19T18:16:30.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.584+0000 7f4f2242c640 1 -- 172.21.15.27:0/2745286476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f1c15d240 msgr2=0x7f4f1c1618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.584+0000 7f4f2242c640 1 --2- 172.21.15.27:0/2745286476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f1c15d240 0x7f4f1c1618d0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c009e50 tx=0x7f4f0c02c010 comp rx=0 tx=0).stop 2023-12-19T18:16:30.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.584+0000 7f4f2242c640 1 -- 172.21.15.27:0/2745286476 shutdown_connections 2023-12-19T18:16:30.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.584+0000 7f4f2242c640 1 --2- 172.21.15.27:0/2745286476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f1c15d240 0x7f4f1c1618d0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.584+0000 7f4f2242c640 1 -- 172.21.15.27:0/2745286476 >> 172.21.15.27:0/2745286476 conn(0x7f4f1c0d31e0 msgr2=0x7f4f1c0d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:30.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.584+0000 7f4f2242c640 1 -- 172.21.15.27:0/2745286476 shutdown_connections 2023-12-19T18:16:30.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.584+0000 7f4f2242c640 1 -- 172.21.15.27:0/2745286476 wait complete. 2023-12-19T18:16:30.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.586+0000 7f4f2242c640 1 Processor -- start 2023-12-19T18:16:30.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.586+0000 7f4f2242c640 1 -- start start 2023-12-19T18:16:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.587+0000 7f4f2242c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f1c0cdf90 0x7f4f1c0a9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.587+0000 7f4f2242c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f1c0ce420 0x7f4f1c0a9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.587+0000 7f4f2242c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f1c15d240 0x7f4f1c149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.587+0000 7f4f2242c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c1f2850 con 0x7f4f1c15d240 2023-12-19T18:16:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.587+0000 7f4f2242c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c14bba0 con 0x7f4f1c0ce420 2023-12-19T18:16:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.587+0000 7f4f2242c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1c14be80 con 0x7f4f1c0cdf90 2023-12-19T18:16:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.587+0000 7f4f209c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f1c15d240 0x7f4f1c149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.587+0000 7f4f209c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f1c15d240 0x7f4f1c149020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54230/0 (socket says 172.21.15.27:54230) 2023-12-19T18:16:30.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.587+0000 7f4f209c9640 1 -- 172.21.15.27:0/1989450542 learned_addr learned my addr 172.21.15.27:0/1989450542 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:30.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.587+0000 7f4f1bfff640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f1c0cdf90 0x7f4f1c0a9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.587+0000 7f4f1b7fe640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f1c0ce420 0x7f4f1c0a9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.587+0000 7f4f209c9640 1 -- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f1c0cdf90 msgr2=0x7f4f1c0a9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.588+0000 7f4f209c9640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f1c0cdf90 0x7f4f1c0a9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.588+0000 7f4f209c9640 1 -- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f1c0ce420 msgr2=0x7f4f1c0a9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.588+0000 7f4f209c9640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f1c0ce420 0x7f4f1c0a9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.588+0000 7f4f209c9640 1 -- 172.21.15.27:0/1989450542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c009b30 con 0x7f4f1c15d240 2023-12-19T18:16:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.588+0000 7f4f209c9640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f1c15d240 0x7f4f1c149020 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400c430 tx=0x7f4f1400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.588+0000 7f4f18ff9640 1 -- 172.21.15.27:0/1989450542 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f140035f0 con 0x7f4f1c15d240 2023-12-19T18:16:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.588+0000 7f4f18ff9640 1 -- 172.21.15.27:0/1989450542 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f14003770 con 0x7f4f1c15d240 2023-12-19T18:16:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.588+0000 7f4f18ff9640 1 -- 172.21.15.27:0/1989450542 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f14003b20 con 0x7f4f1c15d240 2023-12-19T18:16:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.588+0000 7f4f2242c640 1 -- 172.21.15.27:0/1989450542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f1c1f2a60 con 0x7f4f1c15d240 2023-12-19T18:16:30.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.588+0000 7f4f2242c640 1 -- 172.21.15.27:0/1989450542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f1c149560 con 0x7f4f1c15d240 2023-12-19T18:16:30.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.589+0000 7f4f2242c640 1 -- 172.21.15.27:0/1989450542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f1c1f2f80 con 0x7f4f1c15d240 2023-12-19T18:16:30.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.590+0000 7f4f18ff9640 1 -- 172.21.15.27:0/1989450542 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4f14003c80 con 0x7f4f1c15d240 2023-12-19T18:16:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.591+0000 7f4f18ff9640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4efc06a430 0x7f4efc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.591+0000 7f4f18ff9640 1 -- 172.21.15.27:0/1989450542 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4f14087ff0 con 0x7f4f1c15d240 2023-12-19T18:16:30.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.592+0000 7f4f1bfff640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4efc06a430 0x7f4efc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:30.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.593+0000 7f4f1bfff640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4efc06a430 0x7f4efc06c8f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c035000 tx=0x7f4f0c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:30.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.595+0000 7f4f18ff9640 1 -- 172.21.15.27:0/1989450542 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4f140baca0 con 0x7f4f1c15d240 2023-12-19T18:16:30.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.784+0000 7f4f2242c640 1 -- 172.21.15.27:0/1989450542 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4f1c06a880 con 0x7f4efc06a430 2023-12-19T18:16:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.810+0000 7f4f18ff9640 1 -- 172.21.15.27:0/1989450542 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283011 (secure 0 0 0) 0x7f4f0c009510 con 0x7f4efc06a430 2023-12-19T18:16:30.815 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:16:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.816+0000 7f4f2242c640 1 -- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4efc06a430 msgr2=0x7f4efc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.816+0000 7f4f2242c640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4efc06a430 0x7f4efc06c8f0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c035000 tx=0x7f4f0c030000 comp rx=0 tx=0).stop 2023-12-19T18:16:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.816+0000 7f4f2242c640 1 -- 172.21.15.27:0/1989450542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f1c15d240 msgr2=0x7f4f1c149020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.816+0000 7f4f2242c640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f1c15d240 0x7f4f1c149020 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f4f1400c430 tx=0x7f4f1400c8f0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.816+0000 7f4f2242c640 1 -- 172.21.15.27:0/1989450542 shutdown_connections 2023-12-19T18:16:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.816+0000 7f4f2242c640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4efc06a430 0x7f4efc06c8f0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.816+0000 7f4f2242c640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f1c15d240 0x7f4f1c149020 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.816+0000 7f4f2242c640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f1c0ce420 0x7f4f1c0a9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.816+0000 7f4f2242c640 1 --2- 172.21.15.27:0/1989450542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f1c0cdf90 0x7f4f1c0a9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:30.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.816+0000 7f4f2242c640 1 -- 172.21.15.27:0/1989450542 >> 172.21.15.27:0/1989450542 conn(0x7f4f1c0d31e0 msgr2=0x7f4f1c15ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:30.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.817+0000 7f4f2242c640 1 -- 172.21.15.27:0/1989450542 shutdown_connections 2023-12-19T18:16:30.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:30.817+0000 7f4f2242c640 1 -- 172.21.15.27:0/1989450542 wait complete. 2023-12-19T18:16:30.837 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-12-19T18:16:29.736476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2339,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560160,"statfs":{"total":1600000000,"available":1597604612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.466566"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258081+0000","last_change":"2023-12-19T18:16:25.799217+0000","last_active":"2023-12-19T18:16:27.258081+0000","last_peered":"2023-12-19T18:16:27.258081+0000","last_clean":"2023-12-19T18:16:27.258081+0000","last_became_active":"2023-12-19T18:16:25.798957+0000","last_became_peered":"2023-12-19T18:16:25.798957+0000","last_unstale":"2023-12-19T18:16:27.258081+0000","last_undegraded":"2023-12-19T18:16:27.258081+0000","last_fullsized":"2023-12-19T18:16:27.258081+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:04.824215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789904+0000","last_change":"2023-12-19T18:16:25.800188+0000","last_active":"2023-12-19T18:16:26.789904+0000","last_peered":"2023-12-19T18:16:26.789904+0000","last_clean":"2023-12-19T18:16:26.789904+0000","last_became_active":"2023-12-19T18:16:25.799488+0000","last_became_peered":"2023-12-19T18:16:25.799488+0000","last_unstale":"2023-12-19T18:16:26.789904+0000","last_undegraded":"2023-12-19T18:16:26.789904+0000","last_fullsized":"2023-12-19T18:16:26.789904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:16.864011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.515741+0000","last_change":"2023-12-19T18:16:25.800800+0000","last_active":"2023-12-19T18:16:27.515741+0000","last_peered":"2023-12-19T18:16:27.515741+0000","last_clean":"2023-12-19T18:16:27.515741+0000","last_became_active":"2023-12-19T18:16:25.800362+0000","last_became_peered":"2023-12-19T18:16:25.800362+0000","last_unstale":"2023-12-19T18:16:27.515741+0000","last_undegraded":"2023-12-19T18:16:27.515741+0000","last_fullsized":"2023-12-19T18:16:27.515741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:36:59.266648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001304+0000","last_change":"2023-12-19T18:16:25.816293+0000","last_active":"2023-12-19T18:16:27.001304+0000","last_peered":"2023-12-19T18:16:27.001304+0000","last_clean":"2023-12-19T18:16:27.001304+0000","last_became_active":"2023-12-19T18:16:25.806265+0000","last_became_peered":"2023-12-19T18:16:25.806265+0000","last_unstale":"2023-12-19T18:16:27.001304+0000","last_undegraded":"2023-12-19T18:16:27.001304+0000","last_fullsized":"2023-12-19T18:16:27.001304+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:14:09.527590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.792213+0000","last_change":"2023-12-19T18:16:25.819182+0000","last_active":"2023-12-19T18:16:26.792213+0000","last_peered":"2023-12-19T18:16:26.792213+0000","last_clean":"2023-12-19T18:16:26.792213+0000","last_became_active":"2023-12-19T18:16:25.818780+0000","last_became_peered":"2023-12-19T18:16:25.818780+0000","last_unstale":"2023-12-19T18:16:26.792213+0000","last_undegraded":"2023-12-19T18:16:26.792213+0000","last_fullsized":"2023-12-19T18:16:26.792213+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:36:06.247999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791142+0000","last_change":"2023-12-19T18:16:25.815652+0000","last_active":"2023-12-19T18:16:26.791142+0000","last_peered":"2023-12-19T18:16:26.791142+0000","last_clean":"2023-12-19T18:16:26.791142+0000","last_became_active":"2023-12-19T18:16:25.813468+0000","last_became_peered":"2023-12-19T18:16:25.813468+0000","last_unstale":"2023-12-19T18:16:26.791142+0000","last_undegraded":"2023-12-19T18:16:26.791142+0000","last_fullsized":"2023-12-19T18:16:26.791142+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:08:47.017890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.257960+0000","last_change":"2023-12-19T18:16:25.799296+0000","last_active":"2023-12-19T18:16:27.257960+0000","last_peered":"2023-12-19T18:16:27.257960+0000","last_clean":"2023-12-19T18:16:27.257960+0000","last_became_active":"2023-12-19T18:16:25.799061+0000","last_became_peered":"2023-12-19T18:16:25.799061+0000","last_unstale":"2023-12-19T18:16:27.257960+0000","last_undegraded":"2023-12-19T18:16:27.257960+0000","last_fullsized":"2023-12-19T18:16:27.257960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:51.757166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.257538+0000","last_change":"2023-12-19T18:16:25.813342+0000","last_active":"2023-12-19T18:16:27.257538+0000","last_peered":"2023-12-19T18:16:27.257538+0000","last_clean":"2023-12-19T18:16:27.257538+0000","last_became_active":"2023-12-19T18:16:25.812678+0000","last_became_peered":"2023-12-19T18:16:25.812678+0000","last_unstale":"2023-12-19T18:16:27.257538+0000","last_undegraded":"2023-12-19T18:16:27.257538+0000","last_fullsized":"2023-12-19T18:16:27.257538+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:13.683668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.515526+0000","last_change":"2023-12-19T18:16:25.804595+0000","last_active":"2023-12-19T18:16:27.515526+0000","last_peered":"2023-12-19T18:16:27.515526+0000","last_clean":"2023-12-19T18:16:27.515526+0000","last_became_active":"2023-12-19T18:16:25.804370+0000","last_became_peered":"2023-12-19T18:16:25.804370+0000","last_unstale":"2023-12-19T18:16:27.515526+0000","last_undegraded":"2023-12-19T18:16:27.515526+0000","last_fullsized":"2023-12-19T18:16:27.515526+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:16:56.118989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000407+0000","last_change":"2023-12-19T18:16:25.816468+0000","last_active":"2023-12-19T18:16:27.000407+0000","last_peered":"2023-12-19T18:16:27.000407+0000","last_clean":"2023-12-19T18:16:27.000407+0000","last_became_active":"2023-12-19T18:16:25.816160+0000","last_became_peered":"2023-12-19T18:16:25.816160+0000","last_unstale":"2023-12-19T18:16:27.000407+0000","last_undegraded":"2023-12-19T18:16:27.000407+0000","last_fullsized":"2023-12-19T18:16:27.000407+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:16.864011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789577+0000","last_change":"2023-12-19T18:16:25.809408+0000","last_active":"2023-12-19T18:16:26.789577+0000","last_peered":"2023-12-19T18:16:26.789577+0000","last_clean":"2023-12-19T18:16:26.789577+0000","last_became_active":"2023-12-19T18:16:25.809063+0000","last_became_peered":"2023-12-19T18:16:25.809063+0000","last_unstale":"2023-12-19T18:16:26.789577+0000","last_undegraded":"2023-12-19T18:16:26.789577+0000","last_fullsized":"2023-12-19T18:16:26.789577+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:30.260299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.516112+0000","last_change":"2023-12-19T18:16:25.815221+0000","last_active":"2023-12-19T18:16:27.516112+0000","last_peered":"2023-12-19T18:16:27.516112+0000","last_clean":"2023-12-19T18:16:27.516112+0000","last_became_active":"2023-12-19T18:16:25.814984+0000","last_became_peered":"2023-12-19T18:16:25.814984+0000","last_unstale":"2023-12-19T18:16:27.516112+0000","last_undegraded":"2023-12-19T18:16:27.516112+0000","last_fullsized":"2023-12-19T18:16:27.516112+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:21:44.079593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031321+0000","last_change":"2023-12-19T18:16:25.822160+0000","last_active":"2023-12-19T18:16:27.031321+0000","last_peered":"2023-12-19T18:16:27.031321+0000","last_clean":"2023-12-19T18:16:27.031321+0000","last_became_active":"2023-12-19T18:16:25.821845+0000","last_became_peered":"2023-12-19T18:16:25.821845+0000","last_unstale":"2023-12-19T18:16:27.031321+0000","last_undegraded":"2023-12-19T18:16:27.031321+0000","last_fullsized":"2023-12-19T18:16:27.031321+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:05.588891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000780+0000","last_change":"2023-12-19T18:16:25.816670+0000","last_active":"2023-12-19T18:16:27.000780+0000","last_peered":"2023-12-19T18:16:27.000780+0000","last_clean":"2023-12-19T18:16:27.000780+0000","last_became_active":"2023-12-19T18:16:25.807372+0000","last_became_peered":"2023-12-19T18:16:25.807372+0000","last_unstale":"2023-12-19T18:16:27.000780+0000","last_undegraded":"2023-12-19T18:16:27.000780+0000","last_fullsized":"2023-12-19T18:16:27.000780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:04:34.044220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030505+0000","last_change":"2023-12-19T18:16:25.798674+0000","last_active":"2023-12-19T18:16:27.030505+0000","last_peered":"2023-12-19T18:16:27.030505+0000","last_clean":"2023-12-19T18:16:27.030505+0000","last_became_active":"2023-12-19T18:16:25.798449+0000","last_became_peered":"2023-12-19T18:16:25.798449+0000","last_unstale":"2023-12-19T18:16:27.030505+0000","last_undegraded":"2023-12-19T18:16:27.030505+0000","last_fullsized":"2023-12-19T18:16:27.030505+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:36:06.247999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.029967+0000","last_change":"2023-12-19T18:16:25.813830+0000","last_active":"2023-12-19T18:16:27.029967+0000","last_peered":"2023-12-19T18:16:27.029967+0000","last_clean":"2023-12-19T18:16:27.029967+0000","last_became_active":"2023-12-19T18:16:25.813619+0000","last_became_peered":"2023-12-19T18:16:25.813619+0000","last_unstale":"2023-12-19T18:16:27.029967+0000","last_undegraded":"2023-12-19T18:16:27.029967+0000","last_fullsized":"2023-12-19T18:16:27.029967+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:55.031822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789808+0000","last_change":"2023-12-19T18:16:25.805286+0000","last_active":"2023-12-19T18:16:26.789808+0000","last_peered":"2023-12-19T18:16:26.789808+0000","last_clean":"2023-12-19T18:16:26.789808+0000","last_became_active":"2023-12-19T18:16:25.804757+0000","last_became_peered":"2023-12-19T18:16:25.804757+0000","last_unstale":"2023-12-19T18:16:26.789808+0000","last_undegraded":"2023-12-19T18:16:26.789808+0000","last_fullsized":"2023-12-19T18:16:26.789808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:37.892797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790843+0000","last_change":"2023-12-19T18:16:25.819729+0000","last_active":"2023-12-19T18:16:26.790843+0000","last_peered":"2023-12-19T18:16:26.790843+0000","last_clean":"2023-12-19T18:16:26.790843+0000","last_became_active":"2023-12-19T18:16:25.819481+0000","last_became_peered":"2023-12-19T18:16:25.819481+0000","last_unstale":"2023-12-19T18:16:26.790843+0000","last_undegraded":"2023-12-19T18:16:26.790843+0000","last_fullsized":"2023-12-19T18:16:26.790843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:05:40.068118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790389+0000","last_change":"2023-12-19T18:16:25.818368+0000","last_active":"2023-12-19T18:16:26.790389+0000","last_peered":"2023-12-19T18:16:26.790389+0000","last_clean":"2023-12-19T18:16:26.790389+0000","last_became_active":"2023-12-19T18:16:25.816858+0000","last_became_peered":"2023-12-19T18:16:25.816858+0000","last_unstale":"2023-12-19T18:16:26.790389+0000","last_undegraded":"2023-12-19T18:16:26.790389+0000","last_fullsized":"2023-12-19T18:16:26.790389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:48.430697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031430+0000","last_change":"2023-12-19T18:16:25.816240+0000","last_active":"2023-12-19T18:16:27.031430+0000","last_peered":"2023-12-19T18:16:27.031430+0000","last_clean":"2023-12-19T18:16:27.031430+0000","last_became_active":"2023-12-19T18:16:25.816014+0000","last_became_peered":"2023-12-19T18:16:25.816014+0000","last_unstale":"2023-12-19T18:16:27.031430+0000","last_undegraded":"2023-12-19T18:16:27.031430+0000","last_fullsized":"2023-12-19T18:16:27.031430+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:21:18.095663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030165+0000","last_change":"2023-12-19T18:16:25.816700+0000","last_active":"2023-12-19T18:16:27.030165+0000","last_peered":"2023-12-19T18:16:27.030165+0000","last_clean":"2023-12-19T18:16:27.030165+0000","last_became_active":"2023-12-19T18:16:25.816358+0000","last_became_peered":"2023-12-19T18:16:25.816358+0000","last_unstale":"2023-12-19T18:16:27.030165+0000","last_undegraded":"2023-12-19T18:16:27.030165+0000","last_fullsized":"2023-12-19T18:16:27.030165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:05.588891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258157+0000","last_change":"2023-12-19T18:16:25.818226+0000","last_active":"2023-12-19T18:16:27.258157+0000","last_peered":"2023-12-19T18:16:27.258157+0000","last_clean":"2023-12-19T18:16:27.258157+0000","last_became_active":"2023-12-19T18:16:25.817898+0000","last_became_peered":"2023-12-19T18:16:25.817898+0000","last_unstale":"2023-12-19T18:16:27.258157+0000","last_undegraded":"2023-12-19T18:16:27.258157+0000","last_fullsized":"2023-12-19T18:16:27.258157+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:16.864011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031564+0000","last_change":"2023-12-19T18:16:25.818733+0000","last_active":"2023-12-19T18:16:27.031564+0000","last_peered":"2023-12-19T18:16:27.031564+0000","last_clean":"2023-12-19T18:16:27.031564+0000","last_became_active":"2023-12-19T18:16:25.817452+0000","last_became_peered":"2023-12-19T18:16:25.817452+0000","last_unstale":"2023-12-19T18:16:27.031564+0000","last_undegraded":"2023-12-19T18:16:27.031564+0000","last_fullsized":"2023-12-19T18:16:27.031564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:04.824215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258013+0000","last_change":"2023-12-19T18:16:25.805800+0000","last_active":"2023-12-19T18:16:27.258013+0000","last_peered":"2023-12-19T18:16:27.258013+0000","last_clean":"2023-12-19T18:16:27.258013+0000","last_became_active":"2023-12-19T18:16:25.805374+0000","last_became_peered":"2023-12-19T18:16:25.805374+0000","last_unstale":"2023-12-19T18:16:27.258013+0000","last_undegraded":"2023-12-19T18:16:27.258013+0000","last_fullsized":"2023-12-19T18:16:27.258013+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:04:34.044220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000649+0000","last_change":"2023-12-19T18:16:25.807744+0000","last_active":"2023-12-19T18:16:27.000649+0000","last_peered":"2023-12-19T18:16:27.000649+0000","last_clean":"2023-12-19T18:16:27.000649+0000","last_became_active":"2023-12-19T18:16:25.806951+0000","last_became_peered":"2023-12-19T18:16:25.806951+0000","last_unstale":"2023-12-19T18:16:27.000649+0000","last_undegraded":"2023-12-19T18:16:27.000649+0000","last_fullsized":"2023-12-19T18:16:27.000649+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:02:12.534299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000887+0000","last_change":"2023-12-19T18:16:25.805230+0000","last_active":"2023-12-19T18:16:27.000887+0000","last_peered":"2023-12-19T18:16:27.000887+0000","last_clean":"2023-12-19T18:16:27.000887+0000","last_became_active":"2023-12-19T18:16:25.800565+0000","last_became_peered":"2023-12-19T18:16:25.800565+0000","last_unstale":"2023-12-19T18:16:27.000887+0000","last_undegraded":"2023-12-19T18:16:27.000887+0000","last_fullsized":"2023-12-19T18:16:27.000887+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:51:53.950034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790137+0000","last_change":"2023-12-19T18:16:25.794897+0000","last_active":"2023-12-19T18:16:26.790137+0000","last_peered":"2023-12-19T18:16:26.790137+0000","last_clean":"2023-12-19T18:16:26.790137+0000","last_became_active":"2023-12-19T18:16:25.793719+0000","last_became_peered":"2023-12-19T18:16:25.793719+0000","last_unstale":"2023-12-19T18:16:26.790137+0000","last_undegraded":"2023-12-19T18:16:26.790137+0000","last_fullsized":"2023-12-19T18:16:26.790137+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:05.588891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.257793+0000","last_change":"2023-12-19T18:16:25.812892+0000","last_active":"2023-12-19T18:16:27.257793+0000","last_peered":"2023-12-19T18:16:27.257793+0000","last_clean":"2023-12-19T18:16:27.257793+0000","last_became_active":"2023-12-19T18:16:25.812339+0000","last_became_peered":"2023-12-19T18:16:25.812339+0000","last_unstale":"2023-12-19T18:16:27.257793+0000","last_undegraded":"2023-12-19T18:16:27.257793+0000","last_fullsized":"2023-12-19T18:16:27.257793+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:55.031822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030253+0000","last_change":"2023-12-19T18:16:25.804795+0000","last_active":"2023-12-19T18:16:27.030253+0000","last_peered":"2023-12-19T18:16:27.030253+0000","last_clean":"2023-12-19T18:16:27.030253+0000","last_became_active":"2023-12-19T18:16:25.804576+0000","last_became_peered":"2023-12-19T18:16:25.804576+0000","last_unstale":"2023-12-19T18:16:27.030253+0000","last_undegraded":"2023-12-19T18:16:27.030253+0000","last_fullsized":"2023-12-19T18:16:27.030253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:13.683668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030640+0000","last_change":"2023-12-19T18:16:25.802268+0000","last_active":"2023-12-19T18:16:27.030640+0000","last_peered":"2023-12-19T18:16:27.030640+0000","last_clean":"2023-12-19T18:16:27.030640+0000","last_became_active":"2023-12-19T18:16:25.802002+0000","last_became_peered":"2023-12-19T18:16:25.802002+0000","last_unstale":"2023-12-19T18:16:27.030640+0000","last_undegraded":"2023-12-19T18:16:27.030640+0000","last_fullsized":"2023-12-19T18:16:27.030640+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:16.864011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258367+0000","last_change":"2023-12-19T18:16:25.812387+0000","last_active":"2023-12-19T18:16:27.258367+0000","last_peered":"2023-12-19T18:16:27.258367+0000","last_clean":"2023-12-19T18:16:27.258367+0000","last_became_active":"2023-12-19T18:16:25.811120+0000","last_became_peered":"2023-12-19T18:16:25.811120+0000","last_unstale":"2023-12-19T18:16:27.258367+0000","last_undegraded":"2023-12-19T18:16:27.258367+0000","last_fullsized":"2023-12-19T18:16:27.258367+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:21:18.095663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789888+0000","last_change":"2023-12-19T18:16:25.805143+0000","last_active":"2023-12-19T18:16:26.789888+0000","last_peered":"2023-12-19T18:16:26.789888+0000","last_clean":"2023-12-19T18:16:26.789888+0000","last_became_active":"2023-12-19T18:16:25.804465+0000","last_became_peered":"2023-12-19T18:16:25.804465+0000","last_unstale":"2023-12-19T18:16:26.789888+0000","last_undegraded":"2023-12-19T18:16:26.789888+0000","last_fullsized":"2023-12-19T18:16:26.789888+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:38.959609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030903+0000","last_change":"2023-12-19T18:16:25.821547+0000","last_active":"2023-12-19T18:16:27.030903+0000","last_peered":"2023-12-19T18:16:27.030903+0000","last_clean":"2023-12-19T18:16:27.030903+0000","last_became_active":"2023-12-19T18:16:25.820666+0000","last_became_peered":"2023-12-19T18:16:25.820666+0000","last_unstale":"2023-12-19T18:16:27.030903+0000","last_undegraded":"2023-12-19T18:16:27.030903+0000","last_fullsized":"2023-12-19T18:16:27.030903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:56.429563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789747+0000","last_change":"2023-12-19T18:16:25.285814+0000","last_active":"2023-12-19T18:16:26.789747+0000","last_peered":"2023-12-19T18:16:26.789747+0000","last_clean":"2023-12-19T18:16:26.789747+0000","last_became_active":"2023-12-19T18:16:25.285070+0000","last_became_peered":"2023-12-19T18:16:25.285070+0000","last_unstale":"2023-12-19T18:16:26.789747+0000","last_undegraded":"2023-12-19T18:16:26.789747+0000","last_fullsized":"2023-12-19T18:16:26.789747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:17:46.704163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.257937+0000","last_change":"2023-12-19T18:16:25.804575+0000","last_active":"2023-12-19T18:16:27.257937+0000","last_peered":"2023-12-19T18:16:27.257937+0000","last_clean":"2023-12-19T18:16:27.257937+0000","last_became_active":"2023-12-19T18:16:25.804374+0000","last_became_peered":"2023-12-19T18:16:25.804374+0000","last_unstale":"2023-12-19T18:16:27.257937+0000","last_undegraded":"2023-12-19T18:16:27.257937+0000","last_fullsized":"2023-12-19T18:16:27.257937+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:36:06.247999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001672+0000","last_change":"2023-12-19T18:16:25.303358+0000","last_active":"2023-12-19T18:16:27.001672+0000","last_peered":"2023-12-19T18:16:27.001672+0000","last_clean":"2023-12-19T18:16:27.001672+0000","last_became_active":"2023-12-19T18:16:25.302800+0000","last_became_peered":"2023-12-19T18:16:25.302800+0000","last_unstale":"2023-12-19T18:16:27.001672+0000","last_undegraded":"2023-12-19T18:16:27.001672+0000","last_fullsized":"2023-12-19T18:16:27.001672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:21:39.863938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258657+0000","last_change":"2023-12-19T18:16:25.808487+0000","last_active":"2023-12-19T18:16:27.258657+0000","last_peered":"2023-12-19T18:16:27.258657+0000","last_clean":"2023-12-19T18:16:27.258657+0000","last_became_active":"2023-12-19T18:16:25.807952+0000","last_became_peered":"2023-12-19T18:16:25.807952+0000","last_unstale":"2023-12-19T18:16:27.258657+0000","last_undegraded":"2023-12-19T18:16:27.258657+0000","last_fullsized":"2023-12-19T18:16:27.258657+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:44.371578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791778+0000","last_change":"2023-12-19T18:16:25.296997+0000","last_active":"2023-12-19T18:16:26.791778+0000","last_peered":"2023-12-19T18:16:26.791778+0000","last_clean":"2023-12-19T18:16:26.791778+0000","last_became_active":"2023-12-19T18:16:25.296601+0000","last_became_peered":"2023-12-19T18:16:25.296601+0000","last_unstale":"2023-12-19T18:16:26.791778+0000","last_undegraded":"2023-12-19T18:16:26.791778+0000","last_fullsized":"2023-12-19T18:16:26.791778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:06.239206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001320+0000","last_change":"2023-12-19T18:16:25.822975+0000","last_active":"2023-12-19T18:16:27.001320+0000","last_peered":"2023-12-19T18:16:27.001320+0000","last_clean":"2023-12-19T18:16:27.001320+0000","last_became_active":"2023-12-19T18:16:25.822582+0000","last_became_peered":"2023-12-19T18:16:25.822582+0000","last_unstale":"2023-12-19T18:16:27.001320+0000","last_undegraded":"2023-12-19T18:16:27.001320+0000","last_fullsized":"2023-12-19T18:16:27.001320+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:34.334812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000661+0000","last_change":"2023-12-19T18:16:25.305719+0000","last_active":"2023-12-19T18:16:27.000661+0000","last_peered":"2023-12-19T18:16:27.000661+0000","last_clean":"2023-12-19T18:16:27.000661+0000","last_became_active":"2023-12-19T18:16:25.303632+0000","last_became_peered":"2023-12-19T18:16:25.303632+0000","last_unstale":"2023-12-19T18:16:27.000661+0000","last_undegraded":"2023-12-19T18:16:27.000661+0000","last_fullsized":"2023-12-19T18:16:27.000661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:16:55.105888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.516004+0000","last_change":"2023-12-19T18:16:25.799430+0000","last_active":"2023-12-19T18:16:27.516004+0000","last_peered":"2023-12-19T18:16:27.516004+0000","last_clean":"2023-12-19T18:16:27.516004+0000","last_became_active":"2023-12-19T18:16:25.799212+0000","last_became_peered":"2023-12-19T18:16:25.799212+0000","last_unstale":"2023-12-19T18:16:27.516004+0000","last_undegraded":"2023-12-19T18:16:27.516004+0000","last_fullsized":"2023-12-19T18:16:27.516004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:21:40.877038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258447+0000","last_change":"2023-12-19T18:16:25.296622+0000","last_active":"2023-12-19T18:16:27.258447+0000","last_peered":"2023-12-19T18:16:27.258447+0000","last_clean":"2023-12-19T18:16:27.258447+0000","last_became_active":"2023-12-19T18:16:25.296393+0000","last_became_peered":"2023-12-19T18:16:25.296393+0000","last_unstale":"2023-12-19T18:16:27.258447+0000","last_undegraded":"2023-12-19T18:16:27.258447+0000","last_fullsized":"2023-12-19T18:16:27.258447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:39:27.598226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258305+0000","last_change":"2023-12-19T18:16:25.812846+0000","last_active":"2023-12-19T18:16:27.258305+0000","last_peered":"2023-12-19T18:16:27.258305+0000","last_clean":"2023-12-19T18:16:27.258305+0000","last_became_active":"2023-12-19T18:16:25.812626+0000","last_became_peered":"2023-12-19T18:16:25.812626+0000","last_unstale":"2023-12-19T18:16:27.258305+0000","last_undegraded":"2023-12-19T18:16:27.258305+0000","last_fullsized":"2023-12-19T18:16:27.258305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:56.429563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258392+0000","last_change":"2023-12-19T18:16:25.299972+0000","last_active":"2023-12-19T18:16:27.258392+0000","last_peered":"2023-12-19T18:16:27.258392+0000","last_clean":"2023-12-19T18:16:27.258392+0000","last_became_active":"2023-12-19T18:16:25.299396+0000","last_became_peered":"2023-12-19T18:16:25.299396+0000","last_unstale":"2023-12-19T18:16:27.258392+0000","last_undegraded":"2023-12-19T18:16:27.258392+0000","last_fullsized":"2023-12-19T18:16:27.258392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:22:32.420308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.515879+0000","last_change":"2023-12-19T18:16:25.792494+0000","last_active":"2023-12-19T18:16:27.515879+0000","last_peered":"2023-12-19T18:16:27.515879+0000","last_clean":"2023-12-19T18:16:27.515879+0000","last_became_active":"2023-12-19T18:16:25.792101+0000","last_became_peered":"2023-12-19T18:16:25.792101+0000","last_unstale":"2023-12-19T18:16:27.515879+0000","last_undegraded":"2023-12-19T18:16:27.515879+0000","last_fullsized":"2023-12-19T18:16:27.515879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:21:38.087736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790857+0000","last_change":"2023-12-19T18:16:25.301512+0000","last_active":"2023-12-19T18:16:26.790857+0000","last_peered":"2023-12-19T18:16:26.790857+0000","last_clean":"2023-12-19T18:16:26.790857+0000","last_became_active":"2023-12-19T18:16:25.301203+0000","last_became_peered":"2023-12-19T18:16:25.301203+0000","last_unstale":"2023-12-19T18:16:26.790857+0000","last_undegraded":"2023-12-19T18:16:26.790857+0000","last_fullsized":"2023-12-19T18:16:26.790857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:02.452027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001063+0000","last_change":"2023-12-19T18:16:25.804967+0000","last_active":"2023-12-19T18:16:27.001063+0000","last_peered":"2023-12-19T18:16:27.001063+0000","last_clean":"2023-12-19T18:16:27.001063+0000","last_became_active":"2023-12-19T18:16:25.803372+0000","last_became_peered":"2023-12-19T18:16:25.803372+0000","last_unstale":"2023-12-19T18:16:27.001063+0000","last_undegraded":"2023-12-19T18:16:27.001063+0000","last_fullsized":"2023-12-19T18:16:27.001063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:51.514257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258064+0000","last_change":"2023-12-19T18:16:25.305691+0000","last_active":"2023-12-19T18:16:27.258064+0000","last_peered":"2023-12-19T18:16:27.258064+0000","last_clean":"2023-12-19T18:16:27.258064+0000","last_became_active":"2023-12-19T18:16:25.305398+0000","last_became_peered":"2023-12-19T18:16:25.305398+0000","last_unstale":"2023-12-19T18:16:27.258064+0000","last_undegraded":"2023-12-19T18:16:27.258064+0000","last_fullsized":"2023-12-19T18:16:27.258064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:25:59.735910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790009+0000","last_change":"2023-12-19T18:16:25.817402+0000","last_active":"2023-12-19T18:16:26.790009+0000","last_peered":"2023-12-19T18:16:26.790009+0000","last_clean":"2023-12-19T18:16:26.790009+0000","last_became_active":"2023-12-19T18:16:25.817153+0000","last_became_peered":"2023-12-19T18:16:25.817153+0000","last_unstale":"2023-12-19T18:16:26.790009+0000","last_undegraded":"2023-12-19T18:16:26.790009+0000","last_fullsized":"2023-12-19T18:16:26.790009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:03:27.920655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790900+0000","last_change":"2023-12-19T18:16:25.300172+0000","last_active":"2023-12-19T18:16:26.790900+0000","last_peered":"2023-12-19T18:16:26.790900+0000","last_clean":"2023-12-19T18:16:26.790900+0000","last_became_active":"2023-12-19T18:16:25.299795+0000","last_became_peered":"2023-12-19T18:16:25.299795+0000","last_unstale":"2023-12-19T18:16:26.790900+0000","last_undegraded":"2023-12-19T18:16:26.790900+0000","last_fullsized":"2023-12-19T18:16:26.790900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:30:12.514619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000392+0000","last_change":"2023-12-19T18:16:25.816487+0000","last_active":"2023-12-19T18:16:27.000392+0000","last_peered":"2023-12-19T18:16:27.000392+0000","last_clean":"2023-12-19T18:16:27.000392+0000","last_became_active":"2023-12-19T18:16:25.806464+0000","last_became_peered":"2023-12-19T18:16:25.806464+0000","last_unstale":"2023-12-19T18:16:27.000392+0000","last_undegraded":"2023-12-19T18:16:27.000392+0000","last_fullsized":"2023-12-19T18:16:27.000392+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:13.683668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791908+0000","last_change":"2023-12-19T18:16:25.293350+0000","last_active":"2023-12-19T18:16:26.791908+0000","last_peered":"2023-12-19T18:16:26.791908+0000","last_clean":"2023-12-19T18:16:26.791908+0000","last_became_active":"2023-12-19T18:16:25.292964+0000","last_became_peered":"2023-12-19T18:16:25.292964+0000","last_unstale":"2023-12-19T18:16:26.791908+0000","last_undegraded":"2023-12-19T18:16:26.791908+0000","last_fullsized":"2023-12-19T18:16:26.791908+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:45:46.239581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001788+0000","last_change":"2023-12-19T18:16:25.815758+0000","last_active":"2023-12-19T18:16:27.001788+0000","last_peered":"2023-12-19T18:16:27.001788+0000","last_clean":"2023-12-19T18:16:27.001788+0000","last_became_active":"2023-12-19T18:16:25.804470+0000","last_became_peered":"2023-12-19T18:16:25.804470+0000","last_unstale":"2023-12-19T18:16:27.001788+0000","last_undegraded":"2023-12-19T18:16:27.001788+0000","last_fullsized":"2023-12-19T18:16:27.001788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:17:47.717264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789936+0000","last_change":"2023-12-19T18:16:25.293616+0000","last_active":"2023-12-19T18:16:26.789936+0000","last_peered":"2023-12-19T18:16:26.789936+0000","last_clean":"2023-12-19T18:16:26.789936+0000","last_became_active":"2023-12-19T18:16:25.293377+0000","last_became_peered":"2023-12-19T18:16:25.293377+0000","last_unstale":"2023-12-19T18:16:26.789936+0000","last_undegraded":"2023-12-19T18:16:26.789936+0000","last_fullsized":"2023-12-19T18:16:26.789936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:04:33.031119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789650+0000","last_change":"2023-12-19T18:16:25.809579+0000","last_active":"2023-12-19T18:16:26.789650+0000","last_peered":"2023-12-19T18:16:26.789650+0000","last_clean":"2023-12-19T18:16:26.789650+0000","last_became_active":"2023-12-19T18:16:25.809340+0000","last_became_peered":"2023-12-19T18:16:25.809340+0000","last_unstale":"2023-12-19T18:16:26.789650+0000","last_undegraded":"2023-12-19T18:16:26.789650+0000","last_fullsized":"2023-12-19T18:16:26.789650+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:16:07.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031645+0000","last_change":"2023-12-19T18:16:25.305151+0000","last_active":"2023-12-19T18:16:27.031645+0000","last_peered":"2023-12-19T18:16:27.031645+0000","last_clean":"2023-12-19T18:16:27.031645+0000","last_became_active":"2023-12-19T18:16:25.304897+0000","last_became_peered":"2023-12-19T18:16:25.304897+0000","last_unstale":"2023-12-19T18:16:27.031645+0000","last_undegraded":"2023-12-19T18:16:27.031645+0000","last_fullsized":"2023-12-19T18:16:27.031645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:21:43.066492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031113+0000","last_change":"2023-12-19T18:16:25.817157+0000","last_active":"2023-12-19T18:16:27.031113+0000","last_peered":"2023-12-19T18:16:27.031113+0000","last_clean":"2023-12-19T18:16:27.031113+0000","last_became_active":"2023-12-19T18:16:25.816939+0000","last_became_peered":"2023-12-19T18:16:25.816939+0000","last_unstale":"2023-12-19T18:16:27.031113+0000","last_undegraded":"2023-12-19T18:16:27.031113+0000","last_fullsized":"2023-12-19T18:16:27.031113+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:51.757166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258364+0000","last_change":"2023-12-19T18:16:25.301625+0000","last_active":"2023-12-19T18:16:27.258364+0000","last_peered":"2023-12-19T18:16:27.258364+0000","last_clean":"2023-12-19T18:16:27.258364+0000","last_became_active":"2023-12-19T18:16:25.301421+0000","last_became_peered":"2023-12-19T18:16:25.301421+0000","last_unstale":"2023-12-19T18:16:27.258364+0000","last_undegraded":"2023-12-19T18:16:27.258364+0000","last_fullsized":"2023-12-19T18:16:27.258364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:02.452027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000905+0000","last_change":"2023-12-19T18:16:25.822825+0000","last_active":"2023-12-19T18:16:27.000905+0000","last_peered":"2023-12-19T18:16:27.000905+0000","last_clean":"2023-12-19T18:16:27.000905+0000","last_became_active":"2023-12-19T18:16:25.822228+0000","last_became_peered":"2023-12-19T18:16:25.822228+0000","last_unstale":"2023-12-19T18:16:27.000905+0000","last_undegraded":"2023-12-19T18:16:27.000905+0000","last_fullsized":"2023-12-19T18:16:27.000905+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:44.371578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000992+0000","last_change":"2023-12-19T18:16:25.300602+0000","last_active":"2023-12-19T18:16:27.000992+0000","last_peered":"2023-12-19T18:16:27.000992+0000","last_clean":"2023-12-19T18:16:27.000992+0000","last_became_active":"2023-12-19T18:16:25.295746+0000","last_became_peered":"2023-12-19T18:16:25.295746+0000","last_unstale":"2023-12-19T18:16:27.000992+0000","last_undegraded":"2023-12-19T18:16:27.000992+0000","last_fullsized":"2023-12-19T18:16:27.000992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:58.274841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258127+0000","last_change":"2023-12-19T18:16:25.800219+0000","last_active":"2023-12-19T18:16:27.258127+0000","last_peered":"2023-12-19T18:16:27.258127+0000","last_clean":"2023-12-19T18:16:27.258127+0000","last_became_active":"2023-12-19T18:16:25.800012+0000","last_became_peered":"2023-12-19T18:16:25.800012+0000","last_unstale":"2023-12-19T18:16:27.258127+0000","last_undegraded":"2023-12-19T18:16:27.258127+0000","last_fullsized":"2023-12-19T18:16:27.258127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:05.588891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000761+0000","last_change":"2023-12-19T18:16:25.298404+0000","last_active":"2023-12-19T18:16:27.000761+0000","last_peered":"2023-12-19T18:16:27.000761+0000","last_clean":"2023-12-19T18:16:27.000761+0000","last_became_active":"2023-12-19T18:16:25.298118+0000","last_became_peered":"2023-12-19T18:16:25.298118+0000","last_unstale":"2023-12-19T18:16:27.000761+0000","last_undegraded":"2023-12-19T18:16:27.000761+0000","last_fullsized":"2023-12-19T18:16:27.000761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:21:43.066492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790095+0000","last_change":"2023-12-19T18:16:25.804979+0000","last_active":"2023-12-19T18:16:26.790095+0000","last_peered":"2023-12-19T18:16:26.790095+0000","last_clean":"2023-12-19T18:16:26.790095+0000","last_became_active":"2023-12-19T18:16:25.804125+0000","last_became_peered":"2023-12-19T18:16:25.804125+0000","last_unstale":"2023-12-19T18:16:26.790095+0000","last_undegraded":"2023-12-19T18:16:26.790095+0000","last_fullsized":"2023-12-19T18:16:26.790095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:52:45.062755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791410+0000","last_change":"2023-12-19T18:16:25.299632+0000","last_active":"2023-12-19T18:16:26.791410+0000","last_peered":"2023-12-19T18:16:26.791410+0000","last_clean":"2023-12-19T18:16:26.791410+0000","last_became_active":"2023-12-19T18:16:25.299314+0000","last_became_peered":"2023-12-19T18:16:25.299314+0000","last_unstale":"2023-12-19T18:16:26.791410+0000","last_undegraded":"2023-12-19T18:16:26.791410+0000","last_fullsized":"2023-12-19T18:16:26.791410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:45:17.663195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789864+0000","last_change":"2023-12-19T18:16:25.811696+0000","last_active":"2023-12-19T18:16:26.789864+0000","last_peered":"2023-12-19T18:16:26.789864+0000","last_clean":"2023-12-19T18:16:26.789864+0000","last_became_active":"2023-12-19T18:16:25.811485+0000","last_became_peered":"2023-12-19T18:16:25.811485+0000","last_unstale":"2023-12-19T18:16:26.789864+0000","last_undegraded":"2023-12-19T18:16:26.789864+0000","last_fullsized":"2023-12-19T18:16:26.789864+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:44.630670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791077+0000","last_change":"2023-12-19T18:16:25.303692+0000","last_active":"2023-12-19T18:16:26.791077+0000","last_peered":"2023-12-19T18:16:26.791077+0000","last_clean":"2023-12-19T18:16:26.791077+0000","last_became_active":"2023-12-19T18:16:25.303474+0000","last_became_peered":"2023-12-19T18:16:25.303474+0000","last_unstale":"2023-12-19T18:16:26.791077+0000","last_undegraded":"2023-12-19T18:16:26.791077+0000","last_fullsized":"2023-12-19T18:16:26.791077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:22:32.420308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.515811+0000","last_change":"2023-12-19T18:16:25.791106+0000","last_active":"2023-12-19T18:16:27.515811+0000","last_peered":"2023-12-19T18:16:27.515811+0000","last_clean":"2023-12-19T18:16:27.515811+0000","last_became_active":"2023-12-19T18:16:25.790874+0000","last_became_peered":"2023-12-19T18:16:25.790874+0000","last_unstale":"2023-12-19T18:16:27.515811+0000","last_undegraded":"2023-12-19T18:16:27.515811+0000","last_fullsized":"2023-12-19T18:16:27.515811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:45:18.676296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258639+0000","last_change":"2023-12-19T18:16:25.300831+0000","last_active":"2023-12-19T18:16:27.258639+0000","last_peered":"2023-12-19T18:16:27.258639+0000","last_clean":"2023-12-19T18:16:27.258639+0000","last_became_active":"2023-12-19T18:16:25.300591+0000","last_became_peered":"2023-12-19T18:16:25.300591+0000","last_unstale":"2023-12-19T18:16:27.258639+0000","last_undegraded":"2023-12-19T18:16:27.258639+0000","last_fullsized":"2023-12-19T18:16:27.258639+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:45:17.663195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790382+0000","last_change":"2023-12-19T18:16:25.794378+0000","last_active":"2023-12-19T18:16:26.790382+0000","last_peered":"2023-12-19T18:16:26.790382+0000","last_clean":"2023-12-19T18:16:26.790382+0000","last_became_active":"2023-12-19T18:16:25.792644+0000","last_became_peered":"2023-12-19T18:16:25.792644+0000","last_unstale":"2023-12-19T18:16:26.790382+0000","last_undegraded":"2023-12-19T18:16:26.790382+0000","last_fullsized":"2023-12-19T18:16:26.790382+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:13.683668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789664+0000","last_change":"2023-12-19T18:16:25.303530+0000","last_active":"2023-12-19T18:16:26.789664+0000","last_peered":"2023-12-19T18:16:26.789664+0000","last_clean":"2023-12-19T18:16:26.789664+0000","last_became_active":"2023-12-19T18:16:25.300741+0000","last_became_peered":"2023-12-19T18:16:25.300741+0000","last_unstale":"2023-12-19T18:16:26.789664+0000","last_undegraded":"2023-12-19T18:16:26.789664+0000","last_fullsized":"2023-12-19T18:16:26.789664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:25.153944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791488+0000","last_change":"2023-12-19T18:16:25.816543+0000","last_active":"2023-12-19T18:16:26.791488+0000","last_peered":"2023-12-19T18:16:26.791488+0000","last_clean":"2023-12-19T18:16:26.791488+0000","last_became_active":"2023-12-19T18:16:25.815988+0000","last_became_peered":"2023-12-19T18:16:25.815988+0000","last_unstale":"2023-12-19T18:16:26.791488+0000","last_undegraded":"2023-12-19T18:16:26.791488+0000","last_fullsized":"2023-12-19T18:16:26.791488+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:30:13.527720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030516+0000","last_change":"2023-12-19T18:16:25.301845+0000","last_active":"2023-12-19T18:16:27.030516+0000","last_peered":"2023-12-19T18:16:27.030516+0000","last_clean":"2023-12-19T18:16:27.030516+0000","last_became_active":"2023-12-19T18:16:25.301613+0000","last_became_peered":"2023-12-19T18:16:25.301613+0000","last_unstale":"2023-12-19T18:16:27.030516+0000","last_undegraded":"2023-12-19T18:16:27.030516+0000","last_fullsized":"2023-12-19T18:16:27.030516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:20:01.412379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031171+0000","last_change":"2023-12-19T18:16:25.812183+0000","last_active":"2023-12-19T18:16:27.031171+0000","last_peered":"2023-12-19T18:16:27.031171+0000","last_clean":"2023-12-19T18:16:27.031171+0000","last_became_active":"2023-12-19T18:16:25.811950+0000","last_became_peered":"2023-12-19T18:16:25.811950+0000","last_unstale":"2023-12-19T18:16:27.031171+0000","last_undegraded":"2023-12-19T18:16:27.031171+0000","last_fullsized":"2023-12-19T18:16:27.031171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:46:47.615522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791258+0000","last_change":"2023-12-19T18:16:25.296835+0000","last_active":"2023-12-19T18:16:26.791258+0000","last_peered":"2023-12-19T18:16:26.791258+0000","last_clean":"2023-12-19T18:16:26.791258+0000","last_became_active":"2023-12-19T18:16:25.295833+0000","last_became_peered":"2023-12-19T18:16:25.295833+0000","last_unstale":"2023-12-19T18:16:26.791258+0000","last_undegraded":"2023-12-19T18:16:26.791258+0000","last_fullsized":"2023-12-19T18:16:26.791258+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:08:46.004789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.788988+0000","last_change":"2023-12-19T18:16:25.813000+0000","last_active":"2023-12-19T18:16:26.788988+0000","last_peered":"2023-12-19T18:16:26.788988+0000","last_clean":"2023-12-19T18:16:26.788988+0000","last_became_active":"2023-12-19T18:16:25.812692+0000","last_became_peered":"2023-12-19T18:16:25.812692+0000","last_unstale":"2023-12-19T18:16:26.788988+0000","last_undegraded":"2023-12-19T18:16:26.788988+0000","last_fullsized":"2023-12-19T18:16:26.788988+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:55.031822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258292+0000","last_change":"2023-12-19T18:16:25.299441+0000","last_active":"2023-12-19T18:16:27.258292+0000","last_peered":"2023-12-19T18:16:27.258292+0000","last_clean":"2023-12-19T18:16:27.258292+0000","last_became_active":"2023-12-19T18:16:25.299206+0000","last_became_peered":"2023-12-19T18:16:25.299206+0000","last_unstale":"2023-12-19T18:16:27.258292+0000","last_undegraded":"2023-12-19T18:16:27.258292+0000","last_fullsized":"2023-12-19T18:16:27.258292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:06.239206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031441+0000","last_change":"2023-12-19T18:16:25.818715+0000","last_active":"2023-12-19T18:16:27.031441+0000","last_peered":"2023-12-19T18:16:27.031441+0000","last_clean":"2023-12-19T18:16:27.031441+0000","last_became_active":"2023-12-19T18:16:25.818477+0000","last_became_peered":"2023-12-19T18:16:25.818477+0000","last_unstale":"2023-12-19T18:16:27.031441+0000","last_undegraded":"2023-12-19T18:16:27.031441+0000","last_fullsized":"2023-12-19T18:16:27.031441+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:36:06.247999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.516343+0000","last_change":"2023-12-19T18:16:25.298779+0000","last_active":"2023-12-19T18:16:27.516343+0000","last_peered":"2023-12-19T18:16:27.516343+0000","last_clean":"2023-12-19T18:16:27.516343+0000","last_became_active":"2023-12-19T18:16:25.298544+0000","last_became_peered":"2023-12-19T18:16:25.298544+0000","last_unstale":"2023-12-19T18:16:27.516343+0000","last_undegraded":"2023-12-19T18:16:27.516343+0000","last_fullsized":"2023-12-19T18:16:27.516343+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:27.527197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.516200+0000","last_change":"2023-12-19T18:16:25.792323+0000","last_active":"2023-12-19T18:16:27.516200+0000","last_peered":"2023-12-19T18:16:27.516200+0000","last_clean":"2023-12-19T18:16:27.516200+0000","last_became_active":"2023-12-19T18:16:25.791781+0000","last_became_peered":"2023-12-19T18:16:25.791781+0000","last_unstale":"2023-12-19T18:16:27.516200+0000","last_undegraded":"2023-12-19T18:16:27.516200+0000","last_fullsized":"2023-12-19T18:16:27.516200+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:03.465127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789783+0000","last_change":"2023-12-19T18:16:25.301169+0000","last_active":"2023-12-19T18:16:26.789783+0000","last_peered":"2023-12-19T18:16:26.789783+0000","last_clean":"2023-12-19T18:16:26.789783+0000","last_became_active":"2023-12-19T18:16:25.300283+0000","last_became_peered":"2023-12-19T18:16:25.300283+0000","last_unstale":"2023-12-19T18:16:26.789783+0000","last_undegraded":"2023-12-19T18:16:26.789783+0000","last_fullsized":"2023-12-19T18:16:26.789783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:43.358477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791080+0000","last_change":"2023-12-19T18:16:25.813222+0000","last_active":"2023-12-19T18:16:26.791080+0000","last_peered":"2023-12-19T18:16:26.791080+0000","last_clean":"2023-12-19T18:16:26.791080+0000","last_became_active":"2023-12-19T18:16:25.812840+0000","last_became_peered":"2023-12-19T18:16:25.812840+0000","last_unstale":"2023-12-19T18:16:26.791080+0000","last_undegraded":"2023-12-19T18:16:26.791080+0000","last_fullsized":"2023-12-19T18:16:26.791080+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:45:47.252681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031148+0000","last_change":"2023-12-19T18:16:25.299513+0000","last_active":"2023-12-19T18:16:27.031148+0000","last_peered":"2023-12-19T18:16:27.031148+0000","last_clean":"2023-12-19T18:16:27.031148+0000","last_became_active":"2023-12-19T18:16:25.299123+0000","last_became_peered":"2023-12-19T18:16:25.299123+0000","last_unstale":"2023-12-19T18:16:27.031148+0000","last_undegraded":"2023-12-19T18:16:27.031148+0000","last_fullsized":"2023-12-19T18:16:27.031148+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:36:58.253547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790166+0000","last_change":"2023-12-19T18:16:25.813903+0000","last_active":"2023-12-19T18:16:26.790166+0000","last_peered":"2023-12-19T18:16:26.790166+0000","last_clean":"2023-12-19T18:16:26.790166+0000","last_became_active":"2023-12-19T18:16:25.813689+0000","last_became_peered":"2023-12-19T18:16:25.813689+0000","last_unstale":"2023-12-19T18:16:26.790166+0000","last_undegraded":"2023-12-19T18:16:26.790166+0000","last_fullsized":"2023-12-19T18:16:26.790166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:05.570774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.107414+0000","last_change":"2023-12-19T18:16:25.292674+0000","last_active":"2023-12-19T18:16:27.107414+0000","last_peered":"2023-12-19T18:16:27.107414+0000","last_clean":"2023-12-19T18:16:27.107414+0000","last_became_active":"2023-12-19T18:16:25.292217+0000","last_became_peered":"2023-12-19T18:16:25.292217+0000","last_unstale":"2023-12-19T18:16:27.107414+0000","last_undegraded":"2023-12-19T18:16:27.107414+0000","last_fullsized":"2023-12-19T18:16:27.107414+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:47.417596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790641+0000","last_change":"2023-12-19T18:16:25.823013+0000","last_active":"2023-12-19T18:16:26.790641+0000","last_peered":"2023-12-19T18:16:26.790641+0000","last_clean":"2023-12-19T18:16:26.790641+0000","last_became_active":"2023-12-19T18:16:25.822752+0000","last_became_peered":"2023-12-19T18:16:25.822752+0000","last_unstale":"2023-12-19T18:16:26.790641+0000","last_undegraded":"2023-12-19T18:16:26.790641+0000","last_fullsized":"2023-12-19T18:16:26.790641+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:46:47.615522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031571+0000","last_change":"2023-12-19T18:16:25.303266+0000","last_active":"2023-12-19T18:16:27.031571+0000","last_peered":"2023-12-19T18:16:27.031571+0000","last_clean":"2023-12-19T18:16:27.031571+0000","last_became_active":"2023-12-19T18:16:25.303057+0000","last_became_peered":"2023-12-19T18:16:25.303057+0000","last_unstale":"2023-12-19T18:16:27.031571+0000","last_undegraded":"2023-12-19T18:16:27.031571+0000","last_fullsized":"2023-12-19T18:16:27.031571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:21:39.863938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030398+0000","last_change":"2023-12-19T18:16:25.811981+0000","last_active":"2023-12-19T18:16:27.030398+0000","last_peered":"2023-12-19T18:16:27.030398+0000","last_clean":"2023-12-19T18:16:27.030398+0000","last_became_active":"2023-12-19T18:16:25.811760+0000","last_became_peered":"2023-12-19T18:16:25.811760+0000","last_unstale":"2023-12-19T18:16:27.030398+0000","last_undegraded":"2023-12-19T18:16:27.030398+0000","last_fullsized":"2023-12-19T18:16:27.030398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:04.824215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031979+0000","last_change":"2023-12-19T18:16:25.300070+0000","last_active":"2023-12-19T18:16:27.031979+0000","last_peered":"2023-12-19T18:16:27.031979+0000","last_clean":"2023-12-19T18:16:27.031979+0000","last_became_active":"2023-12-19T18:16:25.299188+0000","last_became_peered":"2023-12-19T18:16:25.299188+0000","last_unstale":"2023-12-19T18:16:27.031979+0000","last_undegraded":"2023-12-19T18:16:27.031979+0000","last_fullsized":"2023-12-19T18:16:27.031979+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:02.452027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000532+0000","last_change":"2023-12-19T18:16:25.821432+0000","last_active":"2023-12-19T18:16:27.000532+0000","last_peered":"2023-12-19T18:16:27.000532+0000","last_clean":"2023-12-19T18:16:27.000532+0000","last_became_active":"2023-12-19T18:16:25.817640+0000","last_became_peered":"2023-12-19T18:16:25.817640+0000","last_unstale":"2023-12-19T18:16:27.000532+0000","last_undegraded":"2023-12-19T18:16:27.000532+0000","last_fullsized":"2023-12-19T18:16:27.000532+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:46:41.127939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031637+0000","last_change":"2023-12-19T18:16:25.305031+0000","last_active":"2023-12-19T18:16:27.031637+0000","last_peered":"2023-12-19T18:16:27.031637+0000","last_clean":"2023-12-19T18:16:27.031637+0000","last_became_active":"2023-12-19T18:16:25.304747+0000","last_became_peered":"2023-12-19T18:16:25.304747+0000","last_unstale":"2023-12-19T18:16:27.031637+0000","last_undegraded":"2023-12-19T18:16:27.031637+0000","last_fullsized":"2023-12-19T18:16:27.031637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:21:37.074635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001470+0000","last_change":"2023-12-19T18:16:25.815954+0000","last_active":"2023-12-19T18:16:27.001470+0000","last_peered":"2023-12-19T18:16:27.001470+0000","last_clean":"2023-12-19T18:16:27.001470+0000","last_became_active":"2023-12-19T18:16:25.805500+0000","last_became_peered":"2023-12-19T18:16:25.805500+0000","last_unstale":"2023-12-19T18:16:27.001470+0000","last_undegraded":"2023-12-19T18:16:27.001470+0000","last_fullsized":"2023-12-19T18:16:27.001470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:26.167044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258528+0000","last_change":"2023-12-19T18:16:25.300623+0000","last_active":"2023-12-19T18:16:27.258528+0000","last_peered":"2023-12-19T18:16:27.258528+0000","last_clean":"2023-12-19T18:16:27.258528+0000","last_became_active":"2023-12-19T18:16:25.299737+0000","last_became_peered":"2023-12-19T18:16:25.299737+0000","last_unstale":"2023-12-19T18:16:27.258528+0000","last_undegraded":"2023-12-19T18:16:27.258528+0000","last_fullsized":"2023-12-19T18:16:27.258528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:21:37.074635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789848+0000","last_change":"2023-12-19T18:16:25.816192+0000","last_active":"2023-12-19T18:16:26.789848+0000","last_peered":"2023-12-19T18:16:26.789848+0000","last_clean":"2023-12-19T18:16:26.789848+0000","last_became_active":"2023-12-19T18:16:25.815227+0000","last_became_peered":"2023-12-19T18:16:25.815227+0000","last_unstale":"2023-12-19T18:16:26.789848+0000","last_undegraded":"2023-12-19T18:16:26.789848+0000","last_fullsized":"2023-12-19T18:16:26.789848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:03:43.583733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001887+0000","last_change":"2023-12-19T18:16:25.303503+0000","last_active":"2023-12-19T18:16:27.001887+0000","last_peered":"2023-12-19T18:16:27.001887+0000","last_clean":"2023-12-19T18:16:27.001887+0000","last_became_active":"2023-12-19T18:16:25.303128+0000","last_became_peered":"2023-12-19T18:16:25.303128+0000","last_unstale":"2023-12-19T18:16:27.001887+0000","last_undegraded":"2023-12-19T18:16:27.001887+0000","last_fullsized":"2023-12-19T18:16:27.001887+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:20:01.412379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030756+0000","last_change":"2023-12-19T18:16:25.807977+0000","last_active":"2023-12-19T18:16:27.030756+0000","last_peered":"2023-12-19T18:16:27.030756+0000","last_clean":"2023-12-19T18:16:27.030756+0000","last_became_active":"2023-12-19T18:16:25.807755+0000","last_became_peered":"2023-12-19T18:16:25.807755+0000","last_unstale":"2023-12-19T18:16:27.030756+0000","last_undegraded":"2023-12-19T18:16:27.030756+0000","last_fullsized":"2023-12-19T18:16:27.030756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:51.757166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789662+0000","last_change":"2023-12-19T18:16:25.303304+0000","last_active":"2023-12-19T18:16:26.789662+0000","last_peered":"2023-12-19T18:16:26.789662+0000","last_clean":"2023-12-19T18:16:26.789662+0000","last_became_active":"2023-12-19T18:16:25.302774+0000","last_became_peered":"2023-12-19T18:16:25.302774+0000","last_unstale":"2023-12-19T18:16:26.789662+0000","last_undegraded":"2023-12-19T18:16:26.789662+0000","last_fullsized":"2023-12-19T18:16:26.789662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:15.850910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-19T18:16:25.788595+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:16:25.788595+0000","last_peered":"2023-12-19T18:16:25.788595+0000","last_clean":"2023-12-19T18:16:25.788595+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:16:25.788595+0000","last_undegraded":"2023-12-19T18:16:25.788595+0000","last_fullsized":"2023-12-19T18:16:25.788595+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":147,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195164,"statfs":{"total":200000000,"available":199848737,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":1.194},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.1499999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.327},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":1.236}]},{"osd":2,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.4710000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":1.5720000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.2769999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.754,"5min":2.754,"15min":2.754},"last":1.51}]},{"osd":6,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.0720000000000001,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.5429999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":149,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195163,"statfs":{"total":200000000,"available":199847195,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.123},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.70699999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.84099999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.8340000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.95799999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.2629999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.024},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.2030000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.224}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194587,"statfs":{"total":200000000,"available":199257809,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":1.351},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":1.631}]},{"osd":1,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.843}]},{"osd":2,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.754},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.1100000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.1899999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.302}]},{"osd":6,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.5}]},{"osd":7,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.9359999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.411}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":149,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195163,"statfs":{"total":200000000,"available":199847173,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.0669999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.1719999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.399},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.80500000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.645},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.7050000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.518},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.3440000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.296},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.2450000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.4750000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":148,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195164,"statfs":{"total":200000000,"available":199847983,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":3.4350000000000001,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.177},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.82999999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.288}]},{"osd":2,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.6890000000000001,"5min":3.6890000000000001,"15min":3.6890000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.466}]},{"osd":4,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":1.101},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.0209999999999999,"5min":4.0209999999999999,"15min":4.0209999999999999},"last":1.448}]},{"osd":5,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.456,"15min":1.456},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":3.3370000000000002,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":1.0589999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":3.9399999999999999,"5min":3.9399999999999999,"15min":3.9399999999999999},"last":1.369},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":3.2229999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.825}]},{"osd":7,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.135},{"interface":"front","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":3.9849999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.2270000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":147,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195164,"statfs":{"total":200000000,"available":199848725,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.7610000000000001,"5min":4.7610000000000001,"15min":4.7610000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":4.7850000000000001,"5min":4.7850000000000001,"15min":4.7850000000000001},"last":1.502}]},{"osd":1,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":8.2720000000000002,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":5.0620000000000003,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":1.1439999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":8.3109999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":8.0690000000000008,"5min":8.0690000000000008,"15min":8.0690000000000008},"last":1.3759999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":5.3929999999999998,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":7.3570000000000002,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":1.611}]},{"osd":5,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":8.1839999999999993,"5min":8.1839999999999993,"15min":8.1839999999999993},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":5.165,"5min":5.165,"15min":5.165},"last":1.8540000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":5.1059999999999999,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.635,"15min":1.635},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":8.1370000000000005,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":1.637}]},{"osd":7,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":5.1449999999999996,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":1.7050000000000001},{"interface":"front","average":{"1min":1.744,"5min":1.744,"15min":1.744},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":8.2300000000000004,"5min":8.2300000000000004,"15min":8.2300000000000004},"last":1.4419999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258321,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.167},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.254}]},{"osd":2,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.5700000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.597}]},{"osd":3,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":2.1019999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.4950000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.821},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":2.2469999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.7030000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.948},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.772}]},{"osd":7,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.3380000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":2.3610000000000002},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":2,"5min":2,"15min":2},"last":1.6579999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":147,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195164,"statfs":{"total":200000000,"available":199848669,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.72299999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.89400000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.641},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.069}]},{"osd":4,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.7509999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.494}]},{"osd":6,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.5910000000000002,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.8049999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.9379999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":3.1429999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":1.345}]}]}],"pool_statfs":[]}} 2023-12-19T18:16:30.843 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:16:31.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.120+0000 7ffb849c9640 1 Processor -- start 2023-12-19T18:16:31.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.120+0000 7ffb849c9640 1 -- start start 2023-12-19T18:16:31.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb849c9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ffb80164a80 0x7ffb800ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb849c9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffb800cbc60 0x7ffb800cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb849c9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb800caba0 con 0x7ffb800c9190 2023-12-19T18:16:31.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb849c9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb80164e60 con 0x7ffb800cbc60 2023-12-19T18:16:31.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb849c9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb8006c2d0 con 0x7ffb80164a80 2023-12-19T18:16:31.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb7ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffb800cbc60 0x7ffb800cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb7ed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffb800cbc60 0x7ffb800cc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54244/0 (socket says 172.21.15.27:54244) 2023-12-19T18:16:31.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb7ed9d640 1 -- 172.21.15.27:0/3452648212 learned_addr learned my addr 172.21.15.27:0/3452648212 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb7e59c640 1 --2- 172.21.15.27:0/3452648212 >> v2:172.21.15.192:3300/0 conn(0x7ffb80164a80 0x7ffb800ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb7ed9d640 1 -- 172.21.15.27:0/3452648212 >> v2:172.21.15.192:3300/0 conn(0x7ffb80164a80 msgr2=0x7ffb800ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb7ed9d640 1 --2- 172.21.15.27:0/3452648212 >> v2:172.21.15.192:3300/0 conn(0x7ffb80164a80 0x7ffb800ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb7ed9d640 1 -- 172.21.15.27:0/3452648212 >> v1:172.21.15.27:6789/0 conn(0x7ffb800c9190 legacy=0x7ffb800cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb7ed9d640 1 -- 172.21.15.27:0/3452648212 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb800cae40 con 0x7ffb800cbc60 2023-12-19T18:16:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.122+0000 7ffb7e59c640 1 --2- 172.21.15.27:0/3452648212 >> v2:172.21.15.192:3300/0 conn(0x7ffb80164a80 0x7ffb800ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:31.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.124+0000 7ffb7ed9d640 1 --2- 172.21.15.27:0/3452648212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb800cbc60 0x7ffb800cc710 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ffb8006d730 tx=0x7ffb7802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10680551f8cad22f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.124+0000 7ffb7dd9b640 1 -- 172.21.15.27:0/3452648212 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb7802ca80 con 0x7ffb800cbc60 2023-12-19T18:16:31.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.124+0000 7ffb7dd9b640 1 -- 172.21.15.27:0/3452648212 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb7802cbe0 con 0x7ffb800cbc60 2023-12-19T18:16:31.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.124+0000 7ffb7dd9b640 1 -- 172.21.15.27:0/3452648212 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb780343f0 con 0x7ffb800cbc60 2023-12-19T18:16:31.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.124+0000 7ffb849c9640 1 -- 172.21.15.27:0/3452648212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb800cbc60 msgr2=0x7ffb800cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.124+0000 7ffb849c9640 1 --2- 172.21.15.27:0/3452648212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb800cbc60 0x7ffb800cc710 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ffb8006d730 tx=0x7ffb7802bee0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.125+0000 7ffb849c9640 1 -- 172.21.15.27:0/3452648212 shutdown_connections 2023-12-19T18:16:31.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.125+0000 7ffb849c9640 1 --2- 172.21.15.27:0/3452648212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb800cbc60 0x7ffb800cc710 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.125+0000 7ffb849c9640 1 --2- 172.21.15.27:0/3452648212 >> v2:172.21.15.192:3300/0 conn(0x7ffb80164a80 0x7ffb800ce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.125+0000 7ffb849c9640 1 -- 172.21.15.27:0/3452648212 >> 172.21.15.27:0/3452648212 conn(0x7ffb8006a330 msgr2=0x7ffb800b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:31.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.125+0000 7ffb849c9640 1 -- 172.21.15.27:0/3452648212 shutdown_connections 2023-12-19T18:16:31.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.125+0000 7ffb849c9640 1 -- 172.21.15.27:0/3452648212 wait complete. 2023-12-19T18:16:31.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.127+0000 7ffb849c9640 1 Processor -- start 2023-12-19T18:16:31.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.127+0000 7ffb849c9640 1 -- start start 2023-12-19T18:16:31.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.128+0000 7ffb849c9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb800c9190 0x7ffb800c2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.128+0000 7ffb849c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb800cbc60 0x7ffb800c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.128+0000 7ffb849c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb80164a80 0x7ffb800c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.128+0000 7ffb849c9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800c39c0 con 0x7ffb800cbc60 2023-12-19T18:16:31.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.128+0000 7ffb849c9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800c2db0 con 0x7ffb800c9190 2023-12-19T18:16:31.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.128+0000 7ffb849c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb800c3090 con 0x7ffb80164a80 2023-12-19T18:16:31.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.128+0000 7ffb77fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb800cbc60 0x7ffb800c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.128+0000 7ffb77fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb800cbc60 0x7ffb800c4af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54246/0 (socket says 172.21.15.27:54246) 2023-12-19T18:16:31.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.128+0000 7ffb77fff640 1 -- 172.21.15.27:0/2883440992 learned_addr learned my addr 172.21.15.27:0/2883440992 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:31.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.128+0000 7ffb7e59c640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb800c9190 0x7ffb800c2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.129+0000 7ffb7ed9d640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb80164a80 0x7ffb800c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.129+0000 7ffb77fff640 1 -- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb80164a80 msgr2=0x7ffb800c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.129+0000 7ffb77fff640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb80164a80 0x7ffb800c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.129+0000 7ffb77fff640 1 -- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb800c9190 msgr2=0x7ffb800c2720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.129+0000 7ffb77fff640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb800c9190 0x7ffb800c2720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.129+0000 7ffb77fff640 1 -- 172.21.15.27:0/2883440992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb78009b30 con 0x7ffb800cbc60 2023-12-19T18:16:31.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.129+0000 7ffb77fff640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb800cbc60 0x7ffb800c4af0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800c400 tx=0x7ffb6800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.129+0000 7ffb76ffd640 1 -- 172.21.15.27:0/2883440992 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb680052d0 con 0x7ffb800cbc60 2023-12-19T18:16:31.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.129+0000 7ffb76ffd640 1 -- 172.21.15.27:0/2883440992 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb68005430 con 0x7ffb800cbc60 2023-12-19T18:16:31.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.129+0000 7ffb76ffd640 1 -- 172.21.15.27:0/2883440992 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb68003830 con 0x7ffb800cbc60 2023-12-19T18:16:31.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.129+0000 7ffb849c9640 1 -- 172.21.15.27:0/2883440992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb800c3370 con 0x7ffb800cbc60 2023-12-19T18:16:31.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.130+0000 7ffb849c9640 1 -- 172.21.15.27:0/2883440992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb800c5840 con 0x7ffb800cbc60 2023-12-19T18:16:31.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.131+0000 7ffb849c9640 1 -- 172.21.15.27:0/2883440992 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb80068e90 con 0x7ffb800cbc60 2023-12-19T18:16:31.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.132+0000 7ffb76ffd640 1 -- 172.21.15.27:0/2883440992 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ffb68003990 con 0x7ffb800cbc60 2023-12-19T18:16:31.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.133+0000 7ffb76ffd640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffb6006a430 0x7ffb6006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.133+0000 7ffb76ffd640 1 -- 172.21.15.27:0/2883440992 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb68087f50 con 0x7ffb800cbc60 2023-12-19T18:16:31.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.133+0000 7ffb7e59c640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffb6006a430 0x7ffb6006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.134+0000 7ffb7e59c640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffb6006a430 0x7ffb6006c8f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000a990 tx=0x7ffb70008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.137+0000 7ffb76ffd640 1 -- 172.21.15.27:0/2883440992 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ffb680534a0 con 0x7ffb800cbc60 2023-12-19T18:16:31.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.329+0000 7ffb849c9640 1 -- 172.21.15.27:0/2883440992 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ffb800ae1b0 con 0x7ffb6006a430 2023-12-19T18:16:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.351+0000 7ffb76ffd640 1 -- 172.21.15.27:0/2883440992 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283011 (secure 0 0 0) 0x7ffb7000a710 con 0x7ffb6006a430 2023-12-19T18:16:31.355 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:31.357 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:16:31.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.358+0000 7ffb849c9640 1 -- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffb6006a430 msgr2=0x7ffb6006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.358+0000 7ffb849c9640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffb6006a430 0x7ffb6006c8f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ffb7000a990 tx=0x7ffb70008040 comp rx=0 tx=0).stop 2023-12-19T18:16:31.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.358+0000 7ffb849c9640 1 -- 172.21.15.27:0/2883440992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb800cbc60 msgr2=0x7ffb800c4af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.358+0000 7ffb849c9640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb800cbc60 0x7ffb800c4af0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7ffb6800c400 tx=0x7ffb6800c8c0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.358+0000 7ffb849c9640 1 -- 172.21.15.27:0/2883440992 shutdown_connections 2023-12-19T18:16:31.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.358+0000 7ffb849c9640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffb6006a430 0x7ffb6006c8f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.358+0000 7ffb849c9640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffb80164a80 0x7ffb800c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.358+0000 7ffb849c9640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb800cbc60 0x7ffb800c4af0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.358+0000 7ffb849c9640 1 --2- 172.21.15.27:0/2883440992 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffb800c9190 0x7ffb800c2720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.358+0000 7ffb849c9640 1 -- 172.21.15.27:0/2883440992 >> 172.21.15.27:0/2883440992 conn(0x7ffb8006a330 msgr2=0x7ffb800aee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:31.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.358+0000 7ffb849c9640 1 -- 172.21.15.27:0/2883440992 shutdown_connections 2023-12-19T18:16:31.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.359+0000 7ffb849c9640 1 -- 172.21.15.27:0/2883440992 wait complete. 2023-12-19T18:16:31.381 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":79,"stamp":"2023-12-19T18:16:29.736476+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2339,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1560160,"statfs":{"total":1600000000,"available":1597604612,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.466566"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258081+0000","last_change":"2023-12-19T18:16:25.799217+0000","last_active":"2023-12-19T18:16:27.258081+0000","last_peered":"2023-12-19T18:16:27.258081+0000","last_clean":"2023-12-19T18:16:27.258081+0000","last_became_active":"2023-12-19T18:16:25.798957+0000","last_became_peered":"2023-12-19T18:16:25.798957+0000","last_unstale":"2023-12-19T18:16:27.258081+0000","last_undegraded":"2023-12-19T18:16:27.258081+0000","last_fullsized":"2023-12-19T18:16:27.258081+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:04.824215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789904+0000","last_change":"2023-12-19T18:16:25.800188+0000","last_active":"2023-12-19T18:16:26.789904+0000","last_peered":"2023-12-19T18:16:26.789904+0000","last_clean":"2023-12-19T18:16:26.789904+0000","last_became_active":"2023-12-19T18:16:25.799488+0000","last_became_peered":"2023-12-19T18:16:25.799488+0000","last_unstale":"2023-12-19T18:16:26.789904+0000","last_undegraded":"2023-12-19T18:16:26.789904+0000","last_fullsized":"2023-12-19T18:16:26.789904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:16.864011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.515741+0000","last_change":"2023-12-19T18:16:25.800800+0000","last_active":"2023-12-19T18:16:27.515741+0000","last_peered":"2023-12-19T18:16:27.515741+0000","last_clean":"2023-12-19T18:16:27.515741+0000","last_became_active":"2023-12-19T18:16:25.800362+0000","last_became_peered":"2023-12-19T18:16:25.800362+0000","last_unstale":"2023-12-19T18:16:27.515741+0000","last_undegraded":"2023-12-19T18:16:27.515741+0000","last_fullsized":"2023-12-19T18:16:27.515741+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:36:59.266648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001304+0000","last_change":"2023-12-19T18:16:25.816293+0000","last_active":"2023-12-19T18:16:27.001304+0000","last_peered":"2023-12-19T18:16:27.001304+0000","last_clean":"2023-12-19T18:16:27.001304+0000","last_became_active":"2023-12-19T18:16:25.806265+0000","last_became_peered":"2023-12-19T18:16:25.806265+0000","last_unstale":"2023-12-19T18:16:27.001304+0000","last_undegraded":"2023-12-19T18:16:27.001304+0000","last_fullsized":"2023-12-19T18:16:27.001304+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:14:09.527590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.792213+0000","last_change":"2023-12-19T18:16:25.819182+0000","last_active":"2023-12-19T18:16:26.792213+0000","last_peered":"2023-12-19T18:16:26.792213+0000","last_clean":"2023-12-19T18:16:26.792213+0000","last_became_active":"2023-12-19T18:16:25.818780+0000","last_became_peered":"2023-12-19T18:16:25.818780+0000","last_unstale":"2023-12-19T18:16:26.792213+0000","last_undegraded":"2023-12-19T18:16:26.792213+0000","last_fullsized":"2023-12-19T18:16:26.792213+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:36:06.247999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791142+0000","last_change":"2023-12-19T18:16:25.815652+0000","last_active":"2023-12-19T18:16:26.791142+0000","last_peered":"2023-12-19T18:16:26.791142+0000","last_clean":"2023-12-19T18:16:26.791142+0000","last_became_active":"2023-12-19T18:16:25.813468+0000","last_became_peered":"2023-12-19T18:16:25.813468+0000","last_unstale":"2023-12-19T18:16:26.791142+0000","last_undegraded":"2023-12-19T18:16:26.791142+0000","last_fullsized":"2023-12-19T18:16:26.791142+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:08:47.017890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.257960+0000","last_change":"2023-12-19T18:16:25.799296+0000","last_active":"2023-12-19T18:16:27.257960+0000","last_peered":"2023-12-19T18:16:27.257960+0000","last_clean":"2023-12-19T18:16:27.257960+0000","last_became_active":"2023-12-19T18:16:25.799061+0000","last_became_peered":"2023-12-19T18:16:25.799061+0000","last_unstale":"2023-12-19T18:16:27.257960+0000","last_undegraded":"2023-12-19T18:16:27.257960+0000","last_fullsized":"2023-12-19T18:16:27.257960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:51.757166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.257538+0000","last_change":"2023-12-19T18:16:25.813342+0000","last_active":"2023-12-19T18:16:27.257538+0000","last_peered":"2023-12-19T18:16:27.257538+0000","last_clean":"2023-12-19T18:16:27.257538+0000","last_became_active":"2023-12-19T18:16:25.812678+0000","last_became_peered":"2023-12-19T18:16:25.812678+0000","last_unstale":"2023-12-19T18:16:27.257538+0000","last_undegraded":"2023-12-19T18:16:27.257538+0000","last_fullsized":"2023-12-19T18:16:27.257538+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:13.683668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.515526+0000","last_change":"2023-12-19T18:16:25.804595+0000","last_active":"2023-12-19T18:16:27.515526+0000","last_peered":"2023-12-19T18:16:27.515526+0000","last_clean":"2023-12-19T18:16:27.515526+0000","last_became_active":"2023-12-19T18:16:25.804370+0000","last_became_peered":"2023-12-19T18:16:25.804370+0000","last_unstale":"2023-12-19T18:16:27.515526+0000","last_undegraded":"2023-12-19T18:16:27.515526+0000","last_fullsized":"2023-12-19T18:16:27.515526+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:16:56.118989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000407+0000","last_change":"2023-12-19T18:16:25.816468+0000","last_active":"2023-12-19T18:16:27.000407+0000","last_peered":"2023-12-19T18:16:27.000407+0000","last_clean":"2023-12-19T18:16:27.000407+0000","last_became_active":"2023-12-19T18:16:25.816160+0000","last_became_peered":"2023-12-19T18:16:25.816160+0000","last_unstale":"2023-12-19T18:16:27.000407+0000","last_undegraded":"2023-12-19T18:16:27.000407+0000","last_fullsized":"2023-12-19T18:16:27.000407+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:16.864011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789577+0000","last_change":"2023-12-19T18:16:25.809408+0000","last_active":"2023-12-19T18:16:26.789577+0000","last_peered":"2023-12-19T18:16:26.789577+0000","last_clean":"2023-12-19T18:16:26.789577+0000","last_became_active":"2023-12-19T18:16:25.809063+0000","last_became_peered":"2023-12-19T18:16:25.809063+0000","last_unstale":"2023-12-19T18:16:26.789577+0000","last_undegraded":"2023-12-19T18:16:26.789577+0000","last_fullsized":"2023-12-19T18:16:26.789577+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:30.260299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.516112+0000","last_change":"2023-12-19T18:16:25.815221+0000","last_active":"2023-12-19T18:16:27.516112+0000","last_peered":"2023-12-19T18:16:27.516112+0000","last_clean":"2023-12-19T18:16:27.516112+0000","last_became_active":"2023-12-19T18:16:25.814984+0000","last_became_peered":"2023-12-19T18:16:25.814984+0000","last_unstale":"2023-12-19T18:16:27.516112+0000","last_undegraded":"2023-12-19T18:16:27.516112+0000","last_fullsized":"2023-12-19T18:16:27.516112+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:21:44.079593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031321+0000","last_change":"2023-12-19T18:16:25.822160+0000","last_active":"2023-12-19T18:16:27.031321+0000","last_peered":"2023-12-19T18:16:27.031321+0000","last_clean":"2023-12-19T18:16:27.031321+0000","last_became_active":"2023-12-19T18:16:25.821845+0000","last_became_peered":"2023-12-19T18:16:25.821845+0000","last_unstale":"2023-12-19T18:16:27.031321+0000","last_undegraded":"2023-12-19T18:16:27.031321+0000","last_fullsized":"2023-12-19T18:16:27.031321+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:05.588891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000780+0000","last_change":"2023-12-19T18:16:25.816670+0000","last_active":"2023-12-19T18:16:27.000780+0000","last_peered":"2023-12-19T18:16:27.000780+0000","last_clean":"2023-12-19T18:16:27.000780+0000","last_became_active":"2023-12-19T18:16:25.807372+0000","last_became_peered":"2023-12-19T18:16:25.807372+0000","last_unstale":"2023-12-19T18:16:27.000780+0000","last_undegraded":"2023-12-19T18:16:27.000780+0000","last_fullsized":"2023-12-19T18:16:27.000780+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:04:34.044220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030505+0000","last_change":"2023-12-19T18:16:25.798674+0000","last_active":"2023-12-19T18:16:27.030505+0000","last_peered":"2023-12-19T18:16:27.030505+0000","last_clean":"2023-12-19T18:16:27.030505+0000","last_became_active":"2023-12-19T18:16:25.798449+0000","last_became_peered":"2023-12-19T18:16:25.798449+0000","last_unstale":"2023-12-19T18:16:27.030505+0000","last_undegraded":"2023-12-19T18:16:27.030505+0000","last_fullsized":"2023-12-19T18:16:27.030505+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:36:06.247999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.029967+0000","last_change":"2023-12-19T18:16:25.813830+0000","last_active":"2023-12-19T18:16:27.029967+0000","last_peered":"2023-12-19T18:16:27.029967+0000","last_clean":"2023-12-19T18:16:27.029967+0000","last_became_active":"2023-12-19T18:16:25.813619+0000","last_became_peered":"2023-12-19T18:16:25.813619+0000","last_unstale":"2023-12-19T18:16:27.029967+0000","last_undegraded":"2023-12-19T18:16:27.029967+0000","last_fullsized":"2023-12-19T18:16:27.029967+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:55.031822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789808+0000","last_change":"2023-12-19T18:16:25.805286+0000","last_active":"2023-12-19T18:16:26.789808+0000","last_peered":"2023-12-19T18:16:26.789808+0000","last_clean":"2023-12-19T18:16:26.789808+0000","last_became_active":"2023-12-19T18:16:25.804757+0000","last_became_peered":"2023-12-19T18:16:25.804757+0000","last_unstale":"2023-12-19T18:16:26.789808+0000","last_undegraded":"2023-12-19T18:16:26.789808+0000","last_fullsized":"2023-12-19T18:16:26.789808+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:37.892797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790843+0000","last_change":"2023-12-19T18:16:25.819729+0000","last_active":"2023-12-19T18:16:26.790843+0000","last_peered":"2023-12-19T18:16:26.790843+0000","last_clean":"2023-12-19T18:16:26.790843+0000","last_became_active":"2023-12-19T18:16:25.819481+0000","last_became_peered":"2023-12-19T18:16:25.819481+0000","last_unstale":"2023-12-19T18:16:26.790843+0000","last_undegraded":"2023-12-19T18:16:26.790843+0000","last_fullsized":"2023-12-19T18:16:26.790843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:05:40.068118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790389+0000","last_change":"2023-12-19T18:16:25.818368+0000","last_active":"2023-12-19T18:16:26.790389+0000","last_peered":"2023-12-19T18:16:26.790389+0000","last_clean":"2023-12-19T18:16:26.790389+0000","last_became_active":"2023-12-19T18:16:25.816858+0000","last_became_peered":"2023-12-19T18:16:25.816858+0000","last_unstale":"2023-12-19T18:16:26.790389+0000","last_undegraded":"2023-12-19T18:16:26.790389+0000","last_fullsized":"2023-12-19T18:16:26.790389+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:48.430697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031430+0000","last_change":"2023-12-19T18:16:25.816240+0000","last_active":"2023-12-19T18:16:27.031430+0000","last_peered":"2023-12-19T18:16:27.031430+0000","last_clean":"2023-12-19T18:16:27.031430+0000","last_became_active":"2023-12-19T18:16:25.816014+0000","last_became_peered":"2023-12-19T18:16:25.816014+0000","last_unstale":"2023-12-19T18:16:27.031430+0000","last_undegraded":"2023-12-19T18:16:27.031430+0000","last_fullsized":"2023-12-19T18:16:27.031430+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:21:18.095663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030165+0000","last_change":"2023-12-19T18:16:25.816700+0000","last_active":"2023-12-19T18:16:27.030165+0000","last_peered":"2023-12-19T18:16:27.030165+0000","last_clean":"2023-12-19T18:16:27.030165+0000","last_became_active":"2023-12-19T18:16:25.816358+0000","last_became_peered":"2023-12-19T18:16:25.816358+0000","last_unstale":"2023-12-19T18:16:27.030165+0000","last_undegraded":"2023-12-19T18:16:27.030165+0000","last_fullsized":"2023-12-19T18:16:27.030165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:05.588891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258157+0000","last_change":"2023-12-19T18:16:25.818226+0000","last_active":"2023-12-19T18:16:27.258157+0000","last_peered":"2023-12-19T18:16:27.258157+0000","last_clean":"2023-12-19T18:16:27.258157+0000","last_became_active":"2023-12-19T18:16:25.817898+0000","last_became_peered":"2023-12-19T18:16:25.817898+0000","last_unstale":"2023-12-19T18:16:27.258157+0000","last_undegraded":"2023-12-19T18:16:27.258157+0000","last_fullsized":"2023-12-19T18:16:27.258157+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:16.864011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031564+0000","last_change":"2023-12-19T18:16:25.818733+0000","last_active":"2023-12-19T18:16:27.031564+0000","last_peered":"2023-12-19T18:16:27.031564+0000","last_clean":"2023-12-19T18:16:27.031564+0000","last_became_active":"2023-12-19T18:16:25.817452+0000","last_became_peered":"2023-12-19T18:16:25.817452+0000","last_unstale":"2023-12-19T18:16:27.031564+0000","last_undegraded":"2023-12-19T18:16:27.031564+0000","last_fullsized":"2023-12-19T18:16:27.031564+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:04.824215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258013+0000","last_change":"2023-12-19T18:16:25.805800+0000","last_active":"2023-12-19T18:16:27.258013+0000","last_peered":"2023-12-19T18:16:27.258013+0000","last_clean":"2023-12-19T18:16:27.258013+0000","last_became_active":"2023-12-19T18:16:25.805374+0000","last_became_peered":"2023-12-19T18:16:25.805374+0000","last_unstale":"2023-12-19T18:16:27.258013+0000","last_undegraded":"2023-12-19T18:16:27.258013+0000","last_fullsized":"2023-12-19T18:16:27.258013+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:04:34.044220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000649+0000","last_change":"2023-12-19T18:16:25.807744+0000","last_active":"2023-12-19T18:16:27.000649+0000","last_peered":"2023-12-19T18:16:27.000649+0000","last_clean":"2023-12-19T18:16:27.000649+0000","last_became_active":"2023-12-19T18:16:25.806951+0000","last_became_peered":"2023-12-19T18:16:25.806951+0000","last_unstale":"2023-12-19T18:16:27.000649+0000","last_undegraded":"2023-12-19T18:16:27.000649+0000","last_fullsized":"2023-12-19T18:16:27.000649+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:02:12.534299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000887+0000","last_change":"2023-12-19T18:16:25.805230+0000","last_active":"2023-12-19T18:16:27.000887+0000","last_peered":"2023-12-19T18:16:27.000887+0000","last_clean":"2023-12-19T18:16:27.000887+0000","last_became_active":"2023-12-19T18:16:25.800565+0000","last_became_peered":"2023-12-19T18:16:25.800565+0000","last_unstale":"2023-12-19T18:16:27.000887+0000","last_undegraded":"2023-12-19T18:16:27.000887+0000","last_fullsized":"2023-12-19T18:16:27.000887+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:51:53.950034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790137+0000","last_change":"2023-12-19T18:16:25.794897+0000","last_active":"2023-12-19T18:16:26.790137+0000","last_peered":"2023-12-19T18:16:26.790137+0000","last_clean":"2023-12-19T18:16:26.790137+0000","last_became_active":"2023-12-19T18:16:25.793719+0000","last_became_peered":"2023-12-19T18:16:25.793719+0000","last_unstale":"2023-12-19T18:16:26.790137+0000","last_undegraded":"2023-12-19T18:16:26.790137+0000","last_fullsized":"2023-12-19T18:16:26.790137+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:05.588891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.257793+0000","last_change":"2023-12-19T18:16:25.812892+0000","last_active":"2023-12-19T18:16:27.257793+0000","last_peered":"2023-12-19T18:16:27.257793+0000","last_clean":"2023-12-19T18:16:27.257793+0000","last_became_active":"2023-12-19T18:16:25.812339+0000","last_became_peered":"2023-12-19T18:16:25.812339+0000","last_unstale":"2023-12-19T18:16:27.257793+0000","last_undegraded":"2023-12-19T18:16:27.257793+0000","last_fullsized":"2023-12-19T18:16:27.257793+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:55.031822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030253+0000","last_change":"2023-12-19T18:16:25.804795+0000","last_active":"2023-12-19T18:16:27.030253+0000","last_peered":"2023-12-19T18:16:27.030253+0000","last_clean":"2023-12-19T18:16:27.030253+0000","last_became_active":"2023-12-19T18:16:25.804576+0000","last_became_peered":"2023-12-19T18:16:25.804576+0000","last_unstale":"2023-12-19T18:16:27.030253+0000","last_undegraded":"2023-12-19T18:16:27.030253+0000","last_fullsized":"2023-12-19T18:16:27.030253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:13.683668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030640+0000","last_change":"2023-12-19T18:16:25.802268+0000","last_active":"2023-12-19T18:16:27.030640+0000","last_peered":"2023-12-19T18:16:27.030640+0000","last_clean":"2023-12-19T18:16:27.030640+0000","last_became_active":"2023-12-19T18:16:25.802002+0000","last_became_peered":"2023-12-19T18:16:25.802002+0000","last_unstale":"2023-12-19T18:16:27.030640+0000","last_undegraded":"2023-12-19T18:16:27.030640+0000","last_fullsized":"2023-12-19T18:16:27.030640+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:16.864011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258367+0000","last_change":"2023-12-19T18:16:25.812387+0000","last_active":"2023-12-19T18:16:27.258367+0000","last_peered":"2023-12-19T18:16:27.258367+0000","last_clean":"2023-12-19T18:16:27.258367+0000","last_became_active":"2023-12-19T18:16:25.811120+0000","last_became_peered":"2023-12-19T18:16:25.811120+0000","last_unstale":"2023-12-19T18:16:27.258367+0000","last_undegraded":"2023-12-19T18:16:27.258367+0000","last_fullsized":"2023-12-19T18:16:27.258367+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:21:18.095663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789888+0000","last_change":"2023-12-19T18:16:25.805143+0000","last_active":"2023-12-19T18:16:26.789888+0000","last_peered":"2023-12-19T18:16:26.789888+0000","last_clean":"2023-12-19T18:16:26.789888+0000","last_became_active":"2023-12-19T18:16:25.804465+0000","last_became_peered":"2023-12-19T18:16:25.804465+0000","last_unstale":"2023-12-19T18:16:26.789888+0000","last_undegraded":"2023-12-19T18:16:26.789888+0000","last_fullsized":"2023-12-19T18:16:26.789888+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:38.959609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030903+0000","last_change":"2023-12-19T18:16:25.821547+0000","last_active":"2023-12-19T18:16:27.030903+0000","last_peered":"2023-12-19T18:16:27.030903+0000","last_clean":"2023-12-19T18:16:27.030903+0000","last_became_active":"2023-12-19T18:16:25.820666+0000","last_became_peered":"2023-12-19T18:16:25.820666+0000","last_unstale":"2023-12-19T18:16:27.030903+0000","last_undegraded":"2023-12-19T18:16:27.030903+0000","last_fullsized":"2023-12-19T18:16:27.030903+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:56.429563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"33'4","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789747+0000","last_change":"2023-12-19T18:16:25.285814+0000","last_active":"2023-12-19T18:16:26.789747+0000","last_peered":"2023-12-19T18:16:26.789747+0000","last_clean":"2023-12-19T18:16:26.789747+0000","last_became_active":"2023-12-19T18:16:25.285070+0000","last_became_peered":"2023-12-19T18:16:25.285070+0000","last_unstale":"2023-12-19T18:16:26.789747+0000","last_undegraded":"2023-12-19T18:16:26.789747+0000","last_fullsized":"2023-12-19T18:16:26.789747+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:17:46.704163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.257937+0000","last_change":"2023-12-19T18:16:25.804575+0000","last_active":"2023-12-19T18:16:27.257937+0000","last_peered":"2023-12-19T18:16:27.257937+0000","last_clean":"2023-12-19T18:16:27.257937+0000","last_became_active":"2023-12-19T18:16:25.804374+0000","last_became_peered":"2023-12-19T18:16:25.804374+0000","last_unstale":"2023-12-19T18:16:27.257937+0000","last_undegraded":"2023-12-19T18:16:27.257937+0000","last_fullsized":"2023-12-19T18:16:27.257937+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:36:06.247999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001672+0000","last_change":"2023-12-19T18:16:25.303358+0000","last_active":"2023-12-19T18:16:27.001672+0000","last_peered":"2023-12-19T18:16:27.001672+0000","last_clean":"2023-12-19T18:16:27.001672+0000","last_became_active":"2023-12-19T18:16:25.302800+0000","last_became_peered":"2023-12-19T18:16:25.302800+0000","last_unstale":"2023-12-19T18:16:27.001672+0000","last_undegraded":"2023-12-19T18:16:27.001672+0000","last_fullsized":"2023-12-19T18:16:27.001672+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:21:39.863938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258657+0000","last_change":"2023-12-19T18:16:25.808487+0000","last_active":"2023-12-19T18:16:27.258657+0000","last_peered":"2023-12-19T18:16:27.258657+0000","last_clean":"2023-12-19T18:16:27.258657+0000","last_became_active":"2023-12-19T18:16:25.807952+0000","last_became_peered":"2023-12-19T18:16:25.807952+0000","last_unstale":"2023-12-19T18:16:27.258657+0000","last_undegraded":"2023-12-19T18:16:27.258657+0000","last_fullsized":"2023-12-19T18:16:27.258657+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:44.371578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791778+0000","last_change":"2023-12-19T18:16:25.296997+0000","last_active":"2023-12-19T18:16:26.791778+0000","last_peered":"2023-12-19T18:16:26.791778+0000","last_clean":"2023-12-19T18:16:26.791778+0000","last_became_active":"2023-12-19T18:16:25.296601+0000","last_became_peered":"2023-12-19T18:16:25.296601+0000","last_unstale":"2023-12-19T18:16:26.791778+0000","last_undegraded":"2023-12-19T18:16:26.791778+0000","last_fullsized":"2023-12-19T18:16:26.791778+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:06.239206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001320+0000","last_change":"2023-12-19T18:16:25.822975+0000","last_active":"2023-12-19T18:16:27.001320+0000","last_peered":"2023-12-19T18:16:27.001320+0000","last_clean":"2023-12-19T18:16:27.001320+0000","last_became_active":"2023-12-19T18:16:25.822582+0000","last_became_peered":"2023-12-19T18:16:25.822582+0000","last_unstale":"2023-12-19T18:16:27.001320+0000","last_undegraded":"2023-12-19T18:16:27.001320+0000","last_fullsized":"2023-12-19T18:16:27.001320+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:12:34.334812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000661+0000","last_change":"2023-12-19T18:16:25.305719+0000","last_active":"2023-12-19T18:16:27.000661+0000","last_peered":"2023-12-19T18:16:27.000661+0000","last_clean":"2023-12-19T18:16:27.000661+0000","last_became_active":"2023-12-19T18:16:25.303632+0000","last_became_peered":"2023-12-19T18:16:25.303632+0000","last_unstale":"2023-12-19T18:16:27.000661+0000","last_undegraded":"2023-12-19T18:16:27.000661+0000","last_fullsized":"2023-12-19T18:16:27.000661+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:16:55.105888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.516004+0000","last_change":"2023-12-19T18:16:25.799430+0000","last_active":"2023-12-19T18:16:27.516004+0000","last_peered":"2023-12-19T18:16:27.516004+0000","last_clean":"2023-12-19T18:16:27.516004+0000","last_became_active":"2023-12-19T18:16:25.799212+0000","last_became_peered":"2023-12-19T18:16:25.799212+0000","last_unstale":"2023-12-19T18:16:27.516004+0000","last_undegraded":"2023-12-19T18:16:27.516004+0000","last_fullsized":"2023-12-19T18:16:27.516004+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:21:40.877038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258447+0000","last_change":"2023-12-19T18:16:25.296622+0000","last_active":"2023-12-19T18:16:27.258447+0000","last_peered":"2023-12-19T18:16:27.258447+0000","last_clean":"2023-12-19T18:16:27.258447+0000","last_became_active":"2023-12-19T18:16:25.296393+0000","last_became_peered":"2023-12-19T18:16:25.296393+0000","last_unstale":"2023-12-19T18:16:27.258447+0000","last_undegraded":"2023-12-19T18:16:27.258447+0000","last_fullsized":"2023-12-19T18:16:27.258447+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:39:27.598226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258305+0000","last_change":"2023-12-19T18:16:25.812846+0000","last_active":"2023-12-19T18:16:27.258305+0000","last_peered":"2023-12-19T18:16:27.258305+0000","last_clean":"2023-12-19T18:16:27.258305+0000","last_became_active":"2023-12-19T18:16:25.812626+0000","last_became_peered":"2023-12-19T18:16:25.812626+0000","last_unstale":"2023-12-19T18:16:27.258305+0000","last_undegraded":"2023-12-19T18:16:27.258305+0000","last_fullsized":"2023-12-19T18:16:27.258305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:56.429563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258392+0000","last_change":"2023-12-19T18:16:25.299972+0000","last_active":"2023-12-19T18:16:27.258392+0000","last_peered":"2023-12-19T18:16:27.258392+0000","last_clean":"2023-12-19T18:16:27.258392+0000","last_became_active":"2023-12-19T18:16:25.299396+0000","last_became_peered":"2023-12-19T18:16:25.299396+0000","last_unstale":"2023-12-19T18:16:27.258392+0000","last_undegraded":"2023-12-19T18:16:27.258392+0000","last_fullsized":"2023-12-19T18:16:27.258392+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:22:32.420308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.515879+0000","last_change":"2023-12-19T18:16:25.792494+0000","last_active":"2023-12-19T18:16:27.515879+0000","last_peered":"2023-12-19T18:16:27.515879+0000","last_clean":"2023-12-19T18:16:27.515879+0000","last_became_active":"2023-12-19T18:16:25.792101+0000","last_became_peered":"2023-12-19T18:16:25.792101+0000","last_unstale":"2023-12-19T18:16:27.515879+0000","last_undegraded":"2023-12-19T18:16:27.515879+0000","last_fullsized":"2023-12-19T18:16:27.515879+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:21:38.087736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790857+0000","last_change":"2023-12-19T18:16:25.301512+0000","last_active":"2023-12-19T18:16:26.790857+0000","last_peered":"2023-12-19T18:16:26.790857+0000","last_clean":"2023-12-19T18:16:26.790857+0000","last_became_active":"2023-12-19T18:16:25.301203+0000","last_became_peered":"2023-12-19T18:16:25.301203+0000","last_unstale":"2023-12-19T18:16:26.790857+0000","last_undegraded":"2023-12-19T18:16:26.790857+0000","last_fullsized":"2023-12-19T18:16:26.790857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:02.452027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001063+0000","last_change":"2023-12-19T18:16:25.804967+0000","last_active":"2023-12-19T18:16:27.001063+0000","last_peered":"2023-12-19T18:16:27.001063+0000","last_clean":"2023-12-19T18:16:27.001063+0000","last_became_active":"2023-12-19T18:16:25.803372+0000","last_became_peered":"2023-12-19T18:16:25.803372+0000","last_unstale":"2023-12-19T18:16:27.001063+0000","last_undegraded":"2023-12-19T18:16:27.001063+0000","last_fullsized":"2023-12-19T18:16:27.001063+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:51.514257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258064+0000","last_change":"2023-12-19T18:16:25.305691+0000","last_active":"2023-12-19T18:16:27.258064+0000","last_peered":"2023-12-19T18:16:27.258064+0000","last_clean":"2023-12-19T18:16:27.258064+0000","last_became_active":"2023-12-19T18:16:25.305398+0000","last_became_peered":"2023-12-19T18:16:25.305398+0000","last_unstale":"2023-12-19T18:16:27.258064+0000","last_undegraded":"2023-12-19T18:16:27.258064+0000","last_fullsized":"2023-12-19T18:16:27.258064+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:25:59.735910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790009+0000","last_change":"2023-12-19T18:16:25.817402+0000","last_active":"2023-12-19T18:16:26.790009+0000","last_peered":"2023-12-19T18:16:26.790009+0000","last_clean":"2023-12-19T18:16:26.790009+0000","last_became_active":"2023-12-19T18:16:25.817153+0000","last_became_peered":"2023-12-19T18:16:25.817153+0000","last_unstale":"2023-12-19T18:16:26.790009+0000","last_undegraded":"2023-12-19T18:16:26.790009+0000","last_fullsized":"2023-12-19T18:16:26.790009+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:03:27.920655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790900+0000","last_change":"2023-12-19T18:16:25.300172+0000","last_active":"2023-12-19T18:16:26.790900+0000","last_peered":"2023-12-19T18:16:26.790900+0000","last_clean":"2023-12-19T18:16:26.790900+0000","last_became_active":"2023-12-19T18:16:25.299795+0000","last_became_peered":"2023-12-19T18:16:25.299795+0000","last_unstale":"2023-12-19T18:16:26.790900+0000","last_undegraded":"2023-12-19T18:16:26.790900+0000","last_fullsized":"2023-12-19T18:16:26.790900+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:30:12.514619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000392+0000","last_change":"2023-12-19T18:16:25.816487+0000","last_active":"2023-12-19T18:16:27.000392+0000","last_peered":"2023-12-19T18:16:27.000392+0000","last_clean":"2023-12-19T18:16:27.000392+0000","last_became_active":"2023-12-19T18:16:25.806464+0000","last_became_peered":"2023-12-19T18:16:25.806464+0000","last_unstale":"2023-12-19T18:16:27.000392+0000","last_undegraded":"2023-12-19T18:16:27.000392+0000","last_fullsized":"2023-12-19T18:16:27.000392+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:13.683668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791908+0000","last_change":"2023-12-19T18:16:25.293350+0000","last_active":"2023-12-19T18:16:26.791908+0000","last_peered":"2023-12-19T18:16:26.791908+0000","last_clean":"2023-12-19T18:16:26.791908+0000","last_became_active":"2023-12-19T18:16:25.292964+0000","last_became_peered":"2023-12-19T18:16:25.292964+0000","last_unstale":"2023-12-19T18:16:26.791908+0000","last_undegraded":"2023-12-19T18:16:26.791908+0000","last_fullsized":"2023-12-19T18:16:26.791908+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:45:46.239581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001788+0000","last_change":"2023-12-19T18:16:25.815758+0000","last_active":"2023-12-19T18:16:27.001788+0000","last_peered":"2023-12-19T18:16:27.001788+0000","last_clean":"2023-12-19T18:16:27.001788+0000","last_became_active":"2023-12-19T18:16:25.804470+0000","last_became_peered":"2023-12-19T18:16:25.804470+0000","last_unstale":"2023-12-19T18:16:27.001788+0000","last_undegraded":"2023-12-19T18:16:27.001788+0000","last_fullsized":"2023-12-19T18:16:27.001788+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:17:47.717264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"33'3","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789936+0000","last_change":"2023-12-19T18:16:25.293616+0000","last_active":"2023-12-19T18:16:26.789936+0000","last_peered":"2023-12-19T18:16:26.789936+0000","last_clean":"2023-12-19T18:16:26.789936+0000","last_became_active":"2023-12-19T18:16:25.293377+0000","last_became_peered":"2023-12-19T18:16:25.293377+0000","last_unstale":"2023-12-19T18:16:26.789936+0000","last_undegraded":"2023-12-19T18:16:26.789936+0000","last_fullsized":"2023-12-19T18:16:26.789936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:04:33.031119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789650+0000","last_change":"2023-12-19T18:16:25.809579+0000","last_active":"2023-12-19T18:16:26.789650+0000","last_peered":"2023-12-19T18:16:26.789650+0000","last_clean":"2023-12-19T18:16:26.789650+0000","last_became_active":"2023-12-19T18:16:25.809340+0000","last_became_peered":"2023-12-19T18:16:25.809340+0000","last_unstale":"2023-12-19T18:16:26.789650+0000","last_undegraded":"2023-12-19T18:16:26.789650+0000","last_fullsized":"2023-12-19T18:16:26.789650+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:16:07.795706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"33'6","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031645+0000","last_change":"2023-12-19T18:16:25.305151+0000","last_active":"2023-12-19T18:16:27.031645+0000","last_peered":"2023-12-19T18:16:27.031645+0000","last_clean":"2023-12-19T18:16:27.031645+0000","last_became_active":"2023-12-19T18:16:25.304897+0000","last_became_peered":"2023-12-19T18:16:25.304897+0000","last_unstale":"2023-12-19T18:16:27.031645+0000","last_undegraded":"2023-12-19T18:16:27.031645+0000","last_fullsized":"2023-12-19T18:16:27.031645+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:21:43.066492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031113+0000","last_change":"2023-12-19T18:16:25.817157+0000","last_active":"2023-12-19T18:16:27.031113+0000","last_peered":"2023-12-19T18:16:27.031113+0000","last_clean":"2023-12-19T18:16:27.031113+0000","last_became_active":"2023-12-19T18:16:25.816939+0000","last_became_peered":"2023-12-19T18:16:25.816939+0000","last_unstale":"2023-12-19T18:16:27.031113+0000","last_undegraded":"2023-12-19T18:16:27.031113+0000","last_fullsized":"2023-12-19T18:16:27.031113+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:51.757166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258364+0000","last_change":"2023-12-19T18:16:25.301625+0000","last_active":"2023-12-19T18:16:27.258364+0000","last_peered":"2023-12-19T18:16:27.258364+0000","last_clean":"2023-12-19T18:16:27.258364+0000","last_became_active":"2023-12-19T18:16:25.301421+0000","last_became_peered":"2023-12-19T18:16:25.301421+0000","last_unstale":"2023-12-19T18:16:27.258364+0000","last_undegraded":"2023-12-19T18:16:27.258364+0000","last_fullsized":"2023-12-19T18:16:27.258364+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:02.452027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000905+0000","last_change":"2023-12-19T18:16:25.822825+0000","last_active":"2023-12-19T18:16:27.000905+0000","last_peered":"2023-12-19T18:16:27.000905+0000","last_clean":"2023-12-19T18:16:27.000905+0000","last_became_active":"2023-12-19T18:16:25.822228+0000","last_became_peered":"2023-12-19T18:16:25.822228+0000","last_unstale":"2023-12-19T18:16:27.000905+0000","last_undegraded":"2023-12-19T18:16:27.000905+0000","last_fullsized":"2023-12-19T18:16:27.000905+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:44.371578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000992+0000","last_change":"2023-12-19T18:16:25.300602+0000","last_active":"2023-12-19T18:16:27.000992+0000","last_peered":"2023-12-19T18:16:27.000992+0000","last_clean":"2023-12-19T18:16:27.000992+0000","last_became_active":"2023-12-19T18:16:25.295746+0000","last_became_peered":"2023-12-19T18:16:25.295746+0000","last_unstale":"2023-12-19T18:16:27.000992+0000","last_undegraded":"2023-12-19T18:16:27.000992+0000","last_fullsized":"2023-12-19T18:16:27.000992+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:58.274841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258127+0000","last_change":"2023-12-19T18:16:25.800219+0000","last_active":"2023-12-19T18:16:27.258127+0000","last_peered":"2023-12-19T18:16:27.258127+0000","last_clean":"2023-12-19T18:16:27.258127+0000","last_became_active":"2023-12-19T18:16:25.800012+0000","last_became_peered":"2023-12-19T18:16:25.800012+0000","last_unstale":"2023-12-19T18:16:27.258127+0000","last_undegraded":"2023-12-19T18:16:27.258127+0000","last_fullsized":"2023-12-19T18:16:27.258127+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:05.588891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000761+0000","last_change":"2023-12-19T18:16:25.298404+0000","last_active":"2023-12-19T18:16:27.000761+0000","last_peered":"2023-12-19T18:16:27.000761+0000","last_clean":"2023-12-19T18:16:27.000761+0000","last_became_active":"2023-12-19T18:16:25.298118+0000","last_became_peered":"2023-12-19T18:16:25.298118+0000","last_unstale":"2023-12-19T18:16:27.000761+0000","last_undegraded":"2023-12-19T18:16:27.000761+0000","last_fullsized":"2023-12-19T18:16:27.000761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:21:43.066492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790095+0000","last_change":"2023-12-19T18:16:25.804979+0000","last_active":"2023-12-19T18:16:26.790095+0000","last_peered":"2023-12-19T18:16:26.790095+0000","last_clean":"2023-12-19T18:16:26.790095+0000","last_became_active":"2023-12-19T18:16:25.804125+0000","last_became_peered":"2023-12-19T18:16:25.804125+0000","last_unstale":"2023-12-19T18:16:26.790095+0000","last_undegraded":"2023-12-19T18:16:26.790095+0000","last_fullsized":"2023-12-19T18:16:26.790095+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:52:45.062755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"33'8","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791410+0000","last_change":"2023-12-19T18:16:25.299632+0000","last_active":"2023-12-19T18:16:26.791410+0000","last_peered":"2023-12-19T18:16:26.791410+0000","last_clean":"2023-12-19T18:16:26.791410+0000","last_became_active":"2023-12-19T18:16:25.299314+0000","last_became_peered":"2023-12-19T18:16:25.299314+0000","last_unstale":"2023-12-19T18:16:26.791410+0000","last_undegraded":"2023-12-19T18:16:26.791410+0000","last_fullsized":"2023-12-19T18:16:26.791410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:45:17.663195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789864+0000","last_change":"2023-12-19T18:16:25.811696+0000","last_active":"2023-12-19T18:16:26.789864+0000","last_peered":"2023-12-19T18:16:26.789864+0000","last_clean":"2023-12-19T18:16:26.789864+0000","last_became_active":"2023-12-19T18:16:25.811485+0000","last_became_peered":"2023-12-19T18:16:25.811485+0000","last_unstale":"2023-12-19T18:16:26.789864+0000","last_undegraded":"2023-12-19T18:16:26.789864+0000","last_fullsized":"2023-12-19T18:16:26.789864+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:44.630670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791077+0000","last_change":"2023-12-19T18:16:25.303692+0000","last_active":"2023-12-19T18:16:26.791077+0000","last_peered":"2023-12-19T18:16:26.791077+0000","last_clean":"2023-12-19T18:16:26.791077+0000","last_became_active":"2023-12-19T18:16:25.303474+0000","last_became_peered":"2023-12-19T18:16:25.303474+0000","last_unstale":"2023-12-19T18:16:26.791077+0000","last_undegraded":"2023-12-19T18:16:26.791077+0000","last_fullsized":"2023-12-19T18:16:26.791077+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:22:32.420308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.515811+0000","last_change":"2023-12-19T18:16:25.791106+0000","last_active":"2023-12-19T18:16:27.515811+0000","last_peered":"2023-12-19T18:16:27.515811+0000","last_clean":"2023-12-19T18:16:27.515811+0000","last_became_active":"2023-12-19T18:16:25.790874+0000","last_became_peered":"2023-12-19T18:16:25.790874+0000","last_unstale":"2023-12-19T18:16:27.515811+0000","last_undegraded":"2023-12-19T18:16:27.515811+0000","last_fullsized":"2023-12-19T18:16:27.515811+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:45:18.676296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258639+0000","last_change":"2023-12-19T18:16:25.300831+0000","last_active":"2023-12-19T18:16:27.258639+0000","last_peered":"2023-12-19T18:16:27.258639+0000","last_clean":"2023-12-19T18:16:27.258639+0000","last_became_active":"2023-12-19T18:16:25.300591+0000","last_became_peered":"2023-12-19T18:16:25.300591+0000","last_unstale":"2023-12-19T18:16:27.258639+0000","last_undegraded":"2023-12-19T18:16:27.258639+0000","last_fullsized":"2023-12-19T18:16:27.258639+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:45:17.663195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790382+0000","last_change":"2023-12-19T18:16:25.794378+0000","last_active":"2023-12-19T18:16:26.790382+0000","last_peered":"2023-12-19T18:16:26.790382+0000","last_clean":"2023-12-19T18:16:26.790382+0000","last_became_active":"2023-12-19T18:16:25.792644+0000","last_became_peered":"2023-12-19T18:16:25.792644+0000","last_unstale":"2023-12-19T18:16:26.790382+0000","last_undegraded":"2023-12-19T18:16:26.790382+0000","last_fullsized":"2023-12-19T18:16:26.790382+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:13.683668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"33'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789664+0000","last_change":"2023-12-19T18:16:25.303530+0000","last_active":"2023-12-19T18:16:26.789664+0000","last_peered":"2023-12-19T18:16:26.789664+0000","last_clean":"2023-12-19T18:16:26.789664+0000","last_became_active":"2023-12-19T18:16:25.300741+0000","last_became_peered":"2023-12-19T18:16:25.300741+0000","last_unstale":"2023-12-19T18:16:26.789664+0000","last_undegraded":"2023-12-19T18:16:26.789664+0000","last_fullsized":"2023-12-19T18:16:26.789664+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:25.153944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791488+0000","last_change":"2023-12-19T18:16:25.816543+0000","last_active":"2023-12-19T18:16:26.791488+0000","last_peered":"2023-12-19T18:16:26.791488+0000","last_clean":"2023-12-19T18:16:26.791488+0000","last_became_active":"2023-12-19T18:16:25.815988+0000","last_became_peered":"2023-12-19T18:16:25.815988+0000","last_unstale":"2023-12-19T18:16:26.791488+0000","last_undegraded":"2023-12-19T18:16:26.791488+0000","last_fullsized":"2023-12-19T18:16:26.791488+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:30:13.527720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030516+0000","last_change":"2023-12-19T18:16:25.301845+0000","last_active":"2023-12-19T18:16:27.030516+0000","last_peered":"2023-12-19T18:16:27.030516+0000","last_clean":"2023-12-19T18:16:27.030516+0000","last_became_active":"2023-12-19T18:16:25.301613+0000","last_became_peered":"2023-12-19T18:16:25.301613+0000","last_unstale":"2023-12-19T18:16:27.030516+0000","last_undegraded":"2023-12-19T18:16:27.030516+0000","last_fullsized":"2023-12-19T18:16:27.030516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:20:01.412379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031171+0000","last_change":"2023-12-19T18:16:25.812183+0000","last_active":"2023-12-19T18:16:27.031171+0000","last_peered":"2023-12-19T18:16:27.031171+0000","last_clean":"2023-12-19T18:16:27.031171+0000","last_became_active":"2023-12-19T18:16:25.811950+0000","last_became_peered":"2023-12-19T18:16:25.811950+0000","last_unstale":"2023-12-19T18:16:27.031171+0000","last_undegraded":"2023-12-19T18:16:27.031171+0000","last_fullsized":"2023-12-19T18:16:27.031171+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:46:47.615522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791258+0000","last_change":"2023-12-19T18:16:25.296835+0000","last_active":"2023-12-19T18:16:26.791258+0000","last_peered":"2023-12-19T18:16:26.791258+0000","last_clean":"2023-12-19T18:16:26.791258+0000","last_became_active":"2023-12-19T18:16:25.295833+0000","last_became_peered":"2023-12-19T18:16:25.295833+0000","last_unstale":"2023-12-19T18:16:26.791258+0000","last_undegraded":"2023-12-19T18:16:26.791258+0000","last_fullsized":"2023-12-19T18:16:26.791258+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:08:46.004789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.788988+0000","last_change":"2023-12-19T18:16:25.813000+0000","last_active":"2023-12-19T18:16:26.788988+0000","last_peered":"2023-12-19T18:16:26.788988+0000","last_clean":"2023-12-19T18:16:26.788988+0000","last_became_active":"2023-12-19T18:16:25.812692+0000","last_became_peered":"2023-12-19T18:16:25.812692+0000","last_unstale":"2023-12-19T18:16:26.788988+0000","last_undegraded":"2023-12-19T18:16:26.788988+0000","last_fullsized":"2023-12-19T18:16:26.788988+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:55.031822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258292+0000","last_change":"2023-12-19T18:16:25.299441+0000","last_active":"2023-12-19T18:16:27.258292+0000","last_peered":"2023-12-19T18:16:27.258292+0000","last_clean":"2023-12-19T18:16:27.258292+0000","last_became_active":"2023-12-19T18:16:25.299206+0000","last_became_peered":"2023-12-19T18:16:25.299206+0000","last_unstale":"2023-12-19T18:16:27.258292+0000","last_undegraded":"2023-12-19T18:16:27.258292+0000","last_fullsized":"2023-12-19T18:16:27.258292+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:06.239206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031441+0000","last_change":"2023-12-19T18:16:25.818715+0000","last_active":"2023-12-19T18:16:27.031441+0000","last_peered":"2023-12-19T18:16:27.031441+0000","last_clean":"2023-12-19T18:16:27.031441+0000","last_became_active":"2023-12-19T18:16:25.818477+0000","last_became_peered":"2023-12-19T18:16:25.818477+0000","last_unstale":"2023-12-19T18:16:27.031441+0000","last_undegraded":"2023-12-19T18:16:27.031441+0000","last_fullsized":"2023-12-19T18:16:27.031441+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:36:06.247999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.516343+0000","last_change":"2023-12-19T18:16:25.298779+0000","last_active":"2023-12-19T18:16:27.516343+0000","last_peered":"2023-12-19T18:16:27.516343+0000","last_clean":"2023-12-19T18:16:27.516343+0000","last_became_active":"2023-12-19T18:16:25.298544+0000","last_became_peered":"2023-12-19T18:16:25.298544+0000","last_unstale":"2023-12-19T18:16:27.516343+0000","last_undegraded":"2023-12-19T18:16:27.516343+0000","last_fullsized":"2023-12-19T18:16:27.516343+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:44:27.527197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.516200+0000","last_change":"2023-12-19T18:16:25.792323+0000","last_active":"2023-12-19T18:16:27.516200+0000","last_peered":"2023-12-19T18:16:27.516200+0000","last_clean":"2023-12-19T18:16:27.516200+0000","last_became_active":"2023-12-19T18:16:25.791781+0000","last_became_peered":"2023-12-19T18:16:25.791781+0000","last_unstale":"2023-12-19T18:16:27.516200+0000","last_undegraded":"2023-12-19T18:16:27.516200+0000","last_fullsized":"2023-12-19T18:16:27.516200+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:03.465127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789783+0000","last_change":"2023-12-19T18:16:25.301169+0000","last_active":"2023-12-19T18:16:26.789783+0000","last_peered":"2023-12-19T18:16:26.789783+0000","last_clean":"2023-12-19T18:16:26.789783+0000","last_became_active":"2023-12-19T18:16:25.300283+0000","last_became_peered":"2023-12-19T18:16:25.300283+0000","last_unstale":"2023-12-19T18:16:26.789783+0000","last_undegraded":"2023-12-19T18:16:26.789783+0000","last_fullsized":"2023-12-19T18:16:26.789783+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:43.358477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.791080+0000","last_change":"2023-12-19T18:16:25.813222+0000","last_active":"2023-12-19T18:16:26.791080+0000","last_peered":"2023-12-19T18:16:26.791080+0000","last_clean":"2023-12-19T18:16:26.791080+0000","last_became_active":"2023-12-19T18:16:25.812840+0000","last_became_peered":"2023-12-19T18:16:25.812840+0000","last_unstale":"2023-12-19T18:16:26.791080+0000","last_undegraded":"2023-12-19T18:16:26.791080+0000","last_fullsized":"2023-12-19T18:16:26.791080+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:45:47.252681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"33'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031148+0000","last_change":"2023-12-19T18:16:25.299513+0000","last_active":"2023-12-19T18:16:27.031148+0000","last_peered":"2023-12-19T18:16:27.031148+0000","last_clean":"2023-12-19T18:16:27.031148+0000","last_became_active":"2023-12-19T18:16:25.299123+0000","last_became_peered":"2023-12-19T18:16:25.299123+0000","last_unstale":"2023-12-19T18:16:27.031148+0000","last_undegraded":"2023-12-19T18:16:27.031148+0000","last_fullsized":"2023-12-19T18:16:27.031148+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:36:58.253547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790166+0000","last_change":"2023-12-19T18:16:25.813903+0000","last_active":"2023-12-19T18:16:26.790166+0000","last_peered":"2023-12-19T18:16:26.790166+0000","last_clean":"2023-12-19T18:16:26.790166+0000","last_became_active":"2023-12-19T18:16:25.813689+0000","last_became_peered":"2023-12-19T18:16:25.813689+0000","last_unstale":"2023-12-19T18:16:26.790166+0000","last_undegraded":"2023-12-19T18:16:26.790166+0000","last_fullsized":"2023-12-19T18:16:26.790166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:33:05.570774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.107414+0000","last_change":"2023-12-19T18:16:25.292674+0000","last_active":"2023-12-19T18:16:27.107414+0000","last_peered":"2023-12-19T18:16:27.107414+0000","last_clean":"2023-12-19T18:16:27.107414+0000","last_became_active":"2023-12-19T18:16:25.292217+0000","last_became_peered":"2023-12-19T18:16:25.292217+0000","last_unstale":"2023-12-19T18:16:27.107414+0000","last_undegraded":"2023-12-19T18:16:27.107414+0000","last_fullsized":"2023-12-19T18:16:27.107414+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:47.417596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.790641+0000","last_change":"2023-12-19T18:16:25.823013+0000","last_active":"2023-12-19T18:16:26.790641+0000","last_peered":"2023-12-19T18:16:26.790641+0000","last_clean":"2023-12-19T18:16:26.790641+0000","last_became_active":"2023-12-19T18:16:25.822752+0000","last_became_peered":"2023-12-19T18:16:25.822752+0000","last_unstale":"2023-12-19T18:16:26.790641+0000","last_undegraded":"2023-12-19T18:16:26.790641+0000","last_fullsized":"2023-12-19T18:16:26.790641+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:46:47.615522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031571+0000","last_change":"2023-12-19T18:16:25.303266+0000","last_active":"2023-12-19T18:16:27.031571+0000","last_peered":"2023-12-19T18:16:27.031571+0000","last_clean":"2023-12-19T18:16:27.031571+0000","last_became_active":"2023-12-19T18:16:25.303057+0000","last_became_peered":"2023-12-19T18:16:25.303057+0000","last_unstale":"2023-12-19T18:16:27.031571+0000","last_undegraded":"2023-12-19T18:16:27.031571+0000","last_fullsized":"2023-12-19T18:16:27.031571+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:21:39.863938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030398+0000","last_change":"2023-12-19T18:16:25.811981+0000","last_active":"2023-12-19T18:16:27.030398+0000","last_peered":"2023-12-19T18:16:27.030398+0000","last_clean":"2023-12-19T18:16:27.030398+0000","last_became_active":"2023-12-19T18:16:25.811760+0000","last_became_peered":"2023-12-19T18:16:25.811760+0000","last_unstale":"2023-12-19T18:16:27.030398+0000","last_undegraded":"2023-12-19T18:16:27.030398+0000","last_fullsized":"2023-12-19T18:16:27.030398+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:04.824215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031979+0000","last_change":"2023-12-19T18:16:25.300070+0000","last_active":"2023-12-19T18:16:27.031979+0000","last_peered":"2023-12-19T18:16:27.031979+0000","last_clean":"2023-12-19T18:16:27.031979+0000","last_became_active":"2023-12-19T18:16:25.299188+0000","last_became_peered":"2023-12-19T18:16:25.299188+0000","last_unstale":"2023-12-19T18:16:27.031979+0000","last_undegraded":"2023-12-19T18:16:27.031979+0000","last_fullsized":"2023-12-19T18:16:27.031979+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:02.452027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.000532+0000","last_change":"2023-12-19T18:16:25.821432+0000","last_active":"2023-12-19T18:16:27.000532+0000","last_peered":"2023-12-19T18:16:27.000532+0000","last_clean":"2023-12-19T18:16:27.000532+0000","last_became_active":"2023-12-19T18:16:25.817640+0000","last_became_peered":"2023-12-19T18:16:25.817640+0000","last_unstale":"2023-12-19T18:16:27.000532+0000","last_undegraded":"2023-12-19T18:16:27.000532+0000","last_fullsized":"2023-12-19T18:16:27.000532+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:46:41.127939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.031637+0000","last_change":"2023-12-19T18:16:25.305031+0000","last_active":"2023-12-19T18:16:27.031637+0000","last_peered":"2023-12-19T18:16:27.031637+0000","last_clean":"2023-12-19T18:16:27.031637+0000","last_became_active":"2023-12-19T18:16:25.304747+0000","last_became_peered":"2023-12-19T18:16:25.304747+0000","last_unstale":"2023-12-19T18:16:27.031637+0000","last_undegraded":"2023-12-19T18:16:27.031637+0000","last_fullsized":"2023-12-19T18:16:27.031637+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:21:37.074635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001470+0000","last_change":"2023-12-19T18:16:25.815954+0000","last_active":"2023-12-19T18:16:27.001470+0000","last_peered":"2023-12-19T18:16:27.001470+0000","last_clean":"2023-12-19T18:16:27.001470+0000","last_became_active":"2023-12-19T18:16:25.805500+0000","last_became_peered":"2023-12-19T18:16:25.805500+0000","last_unstale":"2023-12-19T18:16:27.001470+0000","last_undegraded":"2023-12-19T18:16:27.001470+0000","last_fullsized":"2023-12-19T18:16:27.001470+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:06:26.167044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.258528+0000","last_change":"2023-12-19T18:16:25.300623+0000","last_active":"2023-12-19T18:16:27.258528+0000","last_peered":"2023-12-19T18:16:27.258528+0000","last_clean":"2023-12-19T18:16:27.258528+0000","last_became_active":"2023-12-19T18:16:25.299737+0000","last_became_peered":"2023-12-19T18:16:25.299737+0000","last_unstale":"2023-12-19T18:16:27.258528+0000","last_undegraded":"2023-12-19T18:16:27.258528+0000","last_fullsized":"2023-12-19T18:16:27.258528+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:21:37.074635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789848+0000","last_change":"2023-12-19T18:16:25.816192+0000","last_active":"2023-12-19T18:16:26.789848+0000","last_peered":"2023-12-19T18:16:26.789848+0000","last_clean":"2023-12-19T18:16:26.789848+0000","last_became_active":"2023-12-19T18:16:25.815227+0000","last_became_peered":"2023-12-19T18:16:25.815227+0000","last_unstale":"2023-12-19T18:16:26.789848+0000","last_undegraded":"2023-12-19T18:16:26.789848+0000","last_fullsized":"2023-12-19T18:16:26.789848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:03:43.583733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.001887+0000","last_change":"2023-12-19T18:16:25.303503+0000","last_active":"2023-12-19T18:16:27.001887+0000","last_peered":"2023-12-19T18:16:27.001887+0000","last_clean":"2023-12-19T18:16:27.001887+0000","last_became_active":"2023-12-19T18:16:25.303128+0000","last_became_peered":"2023-12-19T18:16:25.303128+0000","last_unstale":"2023-12-19T18:16:27.001887+0000","last_undegraded":"2023-12-19T18:16:27.001887+0000","last_fullsized":"2023-12-19T18:16:27.001887+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:20:01.412379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:27.030756+0000","last_change":"2023-12-19T18:16:25.807977+0000","last_active":"2023-12-19T18:16:27.030756+0000","last_peered":"2023-12-19T18:16:27.030756+0000","last_clean":"2023-12-19T18:16:27.030756+0000","last_became_active":"2023-12-19T18:16:25.807755+0000","last_became_peered":"2023-12-19T18:16:25.807755+0000","last_unstale":"2023-12-19T18:16:27.030756+0000","last_undegraded":"2023-12-19T18:16:27.030756+0000","last_fullsized":"2023-12-19T18:16:27.030756+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:25.265641+0000","last_clean_scrub_stamp":"2023-12-19T18:16:25.265641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:51.757166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-19T18:16:26.789662+0000","last_change":"2023-12-19T18:16:25.303304+0000","last_active":"2023-12-19T18:16:26.789662+0000","last_peered":"2023-12-19T18:16:26.789662+0000","last_clean":"2023-12-19T18:16:26.789662+0000","last_became_active":"2023-12-19T18:16:25.302774+0000","last_became_peered":"2023-12-19T18:16:25.302774+0000","last_unstale":"2023-12-19T18:16:26.789662+0000","last_undegraded":"2023-12-19T18:16:26.789662+0000","last_fullsized":"2023-12-19T18:16:26.789662+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:16:24.252540+0000","last_clean_scrub_stamp":"2023-12-19T18:16:24.252540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:22:15.850910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":176,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-12-19T18:16:25.788595+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:16:25.788595+0000","last_peered":"2023-12-19T18:16:25.788595+0000","last_clean":"2023-12-19T18:16:25.788595+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:16:25.788595+0000","last_undegraded":"2023-12-19T18:16:25.788595+0000","last_fullsized":"2023-12-19T18:16:25.788595+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":147,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195164,"statfs":{"total":200000000,"available":199848737,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.639,"5min":1.639,"15min":1.639},"last":1.194},{"interface":"front","average":{"1min":1.141,"5min":1.141,"15min":1.141},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.677,"5min":1.677,"15min":1.677},"last":1.1499999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.8450000000000002,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.327},{"interface":"front","average":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":1.236}]},{"osd":2,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.7629999999999999,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.4710000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.748,"5min":0.748,"15min":0.748},"max":{"1min":3.0110000000000001,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":1.5720000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.042,"15min":1.042},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.206,"5min":1.206,"15min":1.206},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.2769999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.286,"15min":1.286},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.284,"5min":1.284,"15min":1.284},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":2.754,"5min":2.754,"15min":2.754},"last":1.51}]},{"osd":6,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.0720000000000001,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.5429999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574871,"num_pgs":25,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":149,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195163,"statfs":{"total":200000000,"available":199847195,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.123},{"interface":"front","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.70699999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.119,"15min":1.119},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":2.5870000000000002,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":0.80900000000000005},{"interface":"front","average":{"1min":1.147,"5min":1.147,"15min":1.147},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.84099999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.248,"15min":1.248},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":2.8340000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":0.89600000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.1579999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.95799999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":1.6499999999999999},"last":1.2629999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.131,"15min":1.131},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.6819999999999999},"last":1.024},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.2030000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.6419999999999999,"5min":1.6419999999999999,"15min":1.6419999999999999},"last":1.3300000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.5649999999999999,"5min":1.5649999999999999,"15min":1.5649999999999999},"last":1.224}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194587,"statfs":{"total":200000000,"available":199257809,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.863,"5min":1.863,"15min":1.863},"last":1.351},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1319999999999999},"min":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":1.631}]},{"osd":1,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"min":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.93400000000000005},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.4259999999999999,"5min":1.4259999999999999,"15min":1.4259999999999999},"last":1.843}]},{"osd":2,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4670000000000001,"5min":1.4670000000000001,"15min":1.4670000000000001},"last":1.754},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.698,"5min":1.698,"15min":1.698},"last":1.1100000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.1899999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":1.302}]},{"osd":6,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.9530000000000001,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.5}]},{"osd":7,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.0030000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.9359999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":0.623,"5min":0.623,"15min":0.623},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.411}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":35,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":149,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195163,"statfs":{"total":200000000,"available":199847173,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":0.998,"5min":0.998,"15min":0.998},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.0669999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.1719999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.6850000000000001,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":1.399},{"interface":"front","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.80500000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.238,"15min":1.238},"min":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.645},{"interface":"front","average":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.7050000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":1.518},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"min":{"1min":0.504,"5min":0.504,"15min":0.504},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.3440000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.296},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.2450000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:15:48 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.6890000000000001,"5min":1.6890000000000001,"15min":1.6890000000000001},"last":1.5680000000000001},{"interface":"front","average":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.9330000000000001,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.4750000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":148,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195164,"statfs":{"total":200000000,"available":199847983,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":3.4350000000000001,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.177},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4990000000000001,"5min":1.4990000000000001,"15min":1.4990000000000001},"last":0.82999999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.087,"15min":1.087},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":4.5129999999999999,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":1.105,"5min":1.105,"15min":1.105},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.288}]},{"osd":2,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":3.6890000000000001,"5min":3.6890000000000001,"15min":3.6890000000000001},"last":0.89100000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.202,"15min":1.202},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.905,"5min":1.905,"15min":1.905},"last":1.466}]},{"osd":4,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.171,"15min":1.171},"min":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.774,"5min":1.774,"15min":1.774},"last":1.101},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":4.0209999999999999,"5min":4.0209999999999999,"15min":4.0209999999999999},"last":1.448}]},{"osd":5,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.456,"15min":1.456},"min":{"1min":0.879,"5min":0.879,"15min":0.879},"max":{"1min":3.3370000000000002,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":1.5860000000000001},{"interface":"front","average":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.966,"5min":1.966,"15min":1.966},"last":1.0589999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.502,"5min":1.502,"15min":1.502},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":3.9399999999999999,"5min":3.9399999999999999,"15min":3.9399999999999999},"last":1.369},{"interface":"front","average":{"1min":1.516,"5min":1.516,"15min":1.516},"min":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"max":{"1min":3.2229999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.825}]},{"osd":7,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.0960000000000001,"5min":2.0960000000000001,"15min":2.0960000000000001},"last":1.135},{"interface":"front","average":{"1min":1.556,"5min":1.556,"15min":1.556},"min":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"max":{"1min":3.9849999999999999,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.2270000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":147,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195164,"statfs":{"total":200000000,"available":199848725,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":4.7610000000000001,"5min":4.7610000000000001,"15min":4.7610000000000001},"last":1.0209999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":4.7850000000000001,"5min":4.7850000000000001,"15min":4.7850000000000001},"last":1.502}]},{"osd":1,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":8.2720000000000002,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.212,"5min":1.212,"15min":1.212},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":5.0620000000000003,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":1.1439999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":8.3109999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"last":1.2470000000000001},{"interface":"front","average":{"1min":1.45,"5min":1.45,"15min":1.45},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":8.0690000000000008,"5min":8.0690000000000008,"15min":8.0690000000000008},"last":1.3759999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.506,"15min":1.506},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":5.3929999999999998,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.625,"5min":1.625,"15min":1.625},"min":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"max":{"1min":7.3570000000000002,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":1.611}]},{"osd":5,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.659,"5min":1.659,"15min":1.659},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"max":{"1min":8.1839999999999993,"5min":8.1839999999999993,"15min":8.1839999999999993},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4239999999999999,"15min":1.4239999999999999},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":5.165,"5min":5.165,"15min":5.165},"last":1.8540000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.476,"5min":1.476,"15min":1.476},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":5.1059999999999999,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.635,"5min":1.635,"15min":1.635},"min":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":8.1370000000000005,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":1.637}]},{"osd":7,"last update":"Tue Dec 19 18:15:44 2023","interfaces":[{"interface":"back","average":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":5.1449999999999996,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":1.7050000000000001},{"interface":"front","average":{"1min":1.744,"5min":1.744,"15min":1.744},"min":{"1min":0.878,"5min":0.878,"15min":0.878},"max":{"1min":8.2300000000000004,"5min":8.2300000000000004,"15min":8.2300000000000004},"last":1.4419999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":724,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194588,"statfs":{"total":200000000,"available":199258321,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.167},{"interface":"front","average":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.5349999999999999,"5min":1.5349999999999999,"15min":1.5349999999999999},"last":1.254}]},{"osd":2,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":2.5739999999999998,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":1.5700000000000001},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":1.597}]},{"osd":3,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":2.1019999999999999},{"interface":"front","average":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.3959999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.4950000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"max":{"1min":2.0419999999999998,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.821},{"interface":"front","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":2.2469999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.381,"15min":1.381},"min":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.7030000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"max":{"1min":2.1699999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.948},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.772}]},{"osd":7,"last update":"Tue Dec 19 18:15:43 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.3380000000000001,"5min":2.3380000000000001,"15min":2.3380000000000001},"last":2.3610000000000002},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.4339999999999999,"15min":1.4339999999999999},"min":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"max":{"1min":2,"5min":2,"15min":2},"last":1.6579999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":147,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195164,"statfs":{"total":200000000,"available":199848669,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.94199999999999995},{"interface":"front","average":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":2.4180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.72299999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.5169999999999999,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.89400000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.8959999999999999,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.641},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.9550000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.069}]},{"osd":4,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":2.9300000000000002,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.7509999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3500000000000001,"15min":1.3500000000000001},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.7989999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.494}]},{"osd":6,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.5910000000000002,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.8049999999999999},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"min":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":2.1640000000000001,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":1.9379999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:15:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.9930000000000001,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.427,"5min":1.427,"15min":1.427},"min":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"max":{"1min":3.1429999999999998,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":1.345}]}]}],"pool_statfs":[]}} 2023-12-19T18:16:31.385 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:16:31.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:16:31.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.677+0000 7f21dde7d640 1 Processor -- start 2023-12-19T18:16:31.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.677+0000 7f21dde7d640 1 -- start start 2023-12-19T18:16:31.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.679+0000 7f21dde7d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f21d815a0b0 0x7f21d80cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.679+0000 7f21dde7d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d80c9880 con 0x7f21d80ce210 2023-12-19T18:16:31.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.679+0000 7f21dde7d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d80c9b20 con 0x7f21d806cee0 2023-12-19T18:16:31.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.679+0000 7f21dde7d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d80c9de0 con 0x7f21d815a0b0 2023-12-19T18:16:31.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.679+0000 7f21d6ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f21d80ce210 0x7f21d80cd400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56726/0 (socket says 172.21.15.27:56726) 2023-12-19T18:16:31.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.679+0000 7f21d6ffd640 1 -- 172.21.15.27:0/1077499357 learned_addr learned my addr 172.21.15.27:0/1077499357 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:31.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.679+0000 7f21d77fe640 1 --2- 172.21.15.27:0/1077499357 >> v2:172.21.15.192:3300/0 conn(0x7f21d815a0b0 0x7f21d80cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830855454 0 0) 0x7f21d80c9880 con 0x7f21d80ce210 2023-12-19T18:16:31.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21c0005730 con 0x7f21d80ce210 2023-12-19T18:16:31.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1271858928 0 0) 0x7f21c0005730 con 0x7f21d80ce210 2023-12-19T18:16:31.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 >> v2:172.21.15.192:3300/0 conn(0x7f21d815a0b0 msgr2=0x7f21d80cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:31.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 --2- 172.21.15.27:0/1077499357 >> v2:172.21.15.192:3300/0 conn(0x7f21d815a0b0 0x7f21d80cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 >> v1:172.21.15.192:6789/0 conn(0x7f21d806cee0 legacy=0x7f21d8070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d80cc920 con 0x7f21d80ce210 2023-12-19T18:16:31.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f21cc001670 con 0x7f21d80ce210 2023-12-19T18:16:31.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 >> v1:172.21.15.27:6789/0 conn(0x7f21d80ce210 legacy=0x7f21d80cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 --2- 172.21.15.27:0/1077499357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21c0005bd0 0x7f21c0005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 --2- 172.21.15.27:0/1077499357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c00069d0 0x7f21c0006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 --2- 172.21.15.27:0/1077499357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21c0007390 0x7f21c000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80c9de0 con 0x7f21c00069d0 2023-12-19T18:16:31.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c000e290 con 0x7f21c0005bd0 2023-12-19T18:16:31.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21c000e450 con 0x7f21c0007390 2023-12-19T18:16:31.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.680+0000 7f21d6ffd640 1 --2- 172.21.15.27:0/1077499357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c00069d0 0x7f21c0006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.681+0000 7f21d77fe640 1 --2- 172.21.15.27:0/1077499357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21c0005bd0 0x7f21c0005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.681+0000 7f21d7fff640 1 --2- 172.21.15.27:0/1077499357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21c0007390 0x7f21c000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.681+0000 7f21d6ffd640 1 -- 172.21.15.27:0/1077499357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21c0007390 msgr2=0x7f21c000db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:31.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.681+0000 7f21d6ffd640 1 --2- 172.21.15.27:0/1077499357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21c0007390 0x7f21c000db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.681+0000 7f21d6ffd640 1 -- 172.21.15.27:0/1077499357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21c0005bd0 msgr2=0x7f21c0005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:31.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.681+0000 7f21d6ffd640 1 --2- 172.21.15.27:0/1077499357 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21c0005bd0 0x7f21c0005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.681+0000 7f21d6ffd640 1 -- 172.21.15.27:0/1077499357 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c0005730 con 0x7f21c00069d0 2023-12-19T18:16:31.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.681+0000 7f21d7fff640 1 --2- 172.21.15.27:0/1077499357 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21c0007390 0x7f21c000db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:31.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.681+0000 7f21d7fff640 1 -- 172.21.15.27:0/1077499357 reap_dead start 2023-12-19T18:16:31.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.682+0000 7f21d6ffd640 1 --2- 172.21.15.27:0/1077499357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c00069d0 0x7f21c0006e50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f21cc007fb0 tx=0x7f21c0007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d331454ec2b8def5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.683+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc005ad0 con 0x7f21c00069d0 2023-12-19T18:16:31.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.683+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc005c30 con 0x7f21c00069d0 2023-12-19T18:16:31.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.683+0000 7f21d67fc640 1 -- 172.21.15.27:0/1077499357 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc02c7c0 con 0x7f21c00069d0 2023-12-19T18:16:31.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.683+0000 7f21dde7d640 1 -- 172.21.15.27:0/1077499357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c00069d0 msgr2=0x7f21c0006e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.683+0000 7f21dde7d640 1 --2- 172.21.15.27:0/1077499357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c00069d0 0x7f21c0006e50 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f21cc007fb0 tx=0x7f21c0007a00 comp rx=0 tx=0).stop 2023-12-19T18:16:31.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.683+0000 7f21dde7d640 1 -- 172.21.15.27:0/1077499357 shutdown_connections 2023-12-19T18:16:31.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.683+0000 7f21dde7d640 1 --2- 172.21.15.27:0/1077499357 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c00069d0 0x7f21c0006e50 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.683+0000 7f21dde7d640 1 -- 172.21.15.27:0/1077499357 >> 172.21.15.27:0/1077499357 conn(0x7f21d80cebd0 msgr2=0x7f21d80cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:31.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.683+0000 7f21dde7d640 1 -- 172.21.15.27:0/1077499357 shutdown_connections 2023-12-19T18:16:31.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.684+0000 7f21dde7d640 1 -- 172.21.15.27:0/1077499357 wait complete. 2023-12-19T18:16:31.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.685+0000 7f21dde7d640 1 Processor -- start 2023-12-19T18:16:31.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.686+0000 7f21dde7d640 1 -- start start 2023-12-19T18:16:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.686+0000 7f21dde7d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c00069d0 0x7f21d80b41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21dde7d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21d80b15b0 0x7f21d8156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21dde7d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21d806fff0 0x7f21d80704a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21dde7d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d80709e0 con 0x7f21c00069d0 2023-12-19T18:16:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21dde7d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d81f2e30 con 0x7f21d806fff0 2023-12-19T18:16:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21dde7d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d81f3110 con 0x7f21d80b15b0 2023-12-19T18:16:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21d77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c00069d0 0x7f21d80b41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21d77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c00069d0 0x7f21d80b41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54260/0 (socket says 172.21.15.27:54260) 2023-12-19T18:16:31.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21d77fe640 1 -- 172.21.15.27:0/70651194 learned_addr learned my addr 172.21.15.27:0/70651194 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:31.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21d7fff640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21d806fff0 0x7f21d80704a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21d6ffd640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21d80b15b0 0x7f21d8156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21d77fe640 1 -- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21d80b15b0 msgr2=0x7f21d8156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21d77fe640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21d80b15b0 0x7f21d8156d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21d77fe640 1 -- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21d806fff0 msgr2=0x7f21d80704a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21d77fe640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21d806fff0 0x7f21d80704a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21d77fe640 1 -- 172.21.15.27:0/70651194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d81f33f0 con 0x7f21c00069d0 2023-12-19T18:16:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.687+0000 7f21d7fff640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21d806fff0 0x7f21d80704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.688+0000 7f21d77fe640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c00069d0 0x7f21d80b41c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f21c80116d0 tx=0x7f21c8011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.688+0000 7f21bffff640 1 -- 172.21.15.27:0/70651194 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c800f9c0 con 0x7f21c00069d0 2023-12-19T18:16:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.688+0000 7f21bffff640 1 -- 172.21.15.27:0/70651194 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c800fb20 con 0x7f21c00069d0 2023-12-19T18:16:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.688+0000 7f21dde7d640 1 -- 172.21.15.27:0/70651194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d81f36d0 con 0x7f21c00069d0 2023-12-19T18:16:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.688+0000 7f21bffff640 1 -- 172.21.15.27:0/70651194 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c800fe90 con 0x7f21c00069d0 2023-12-19T18:16:31.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.688+0000 7f21dde7d640 1 -- 172.21.15.27:0/70651194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d81f2ae0 con 0x7f21c00069d0 2023-12-19T18:16:31.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.690+0000 7f21dde7d640 1 -- 172.21.15.27:0/70651194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d8068e90 con 0x7f21c00069d0 2023-12-19T18:16:31.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.690+0000 7f21bffff640 1 -- 172.21.15.27:0/70651194 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f21c8015460 con 0x7f21c00069d0 2023-12-19T18:16:31.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.691+0000 7f21bffff640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f21c406a460 0x7f21c406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:31.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.692+0000 7f21bffff640 1 -- 172.21.15.27:0/70651194 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f21c8089200 con 0x7f21c00069d0 2023-12-19T18:16:31.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.692+0000 7f21d6ffd640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f21c406a460 0x7f21c406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:31.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.693+0000 7f21d6ffd640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f21c406a460 0x7f21c406c920 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f21cc034000 tx=0x7f21cc009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:31.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.696+0000 7f21bffff640 1 -- 172.21.15.27:0/70651194 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f21c80547f0 con 0x7f21c00069d0 2023-12-19T18:16:31.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.963+0000 7f21dde7d640 1 -- 172.21.15.27:0/70651194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f21d806b6f0 con 0x7f21c00069d0 2023-12-19T18:16:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.973+0000 7f21bffff640 1 -- 172.21.15.27:0/70651194 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) v1 ==== 190+0+0 (secure 0 0 0) 0x7f21c8057e10 con 0x7f21c00069d0 2023-12-19T18:16:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.976+0000 7f21dde7d640 1 -- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f21c406a460 msgr2=0x7f21c406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.976+0000 7f21dde7d640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f21c406a460 0x7f21c406c920 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f21cc034000 tx=0x7f21cc009030 comp rx=0 tx=0).stop 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.976+0000 7f21dde7d640 1 -- 172.21.15.27:0/70651194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c00069d0 msgr2=0x7f21d80b41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.976+0000 7f21dde7d640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c00069d0 0x7f21d80b41c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f21c80116d0 tx=0x7f21c8011b90 comp rx=0 tx=0).stop 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.976+0000 7f21dde7d640 1 -- 172.21.15.27:0/70651194 shutdown_connections 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.976+0000 7f21dde7d640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f21c406a460 0x7f21c406c920 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.976+0000 7f21dde7d640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f21d806fff0 0x7f21d80704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.976+0000 7f21dde7d640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f21d80b15b0 0x7f21d8156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.976+0000 7f21dde7d640 1 --2- 172.21.15.27:0/70651194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c00069d0 0x7f21d80b41c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.976+0000 7f21dde7d640 1 -- 172.21.15.27:0/70651194 >> 172.21.15.27:0/70651194 conn(0x7f21d80cebd0 msgr2=0x7f21d80cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:31.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.976+0000 7f21dde7d640 1 -- 172.21.15.27:0/70651194 shutdown_connections 2023-12-19T18:16:31.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:31.976+0000 7f21dde7d640 1 -- 172.21.15.27:0/70651194 wait complete. 2023-12-19T18:16:31.995 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:32.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.209+0000 7f0ff759e640 1 Processor -- start 2023-12-19T18:16:32.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.209+0000 7f0ff759e640 1 -- start start 2023-12-19T18:16:32.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.211+0000 7f0ff759e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0ff81553a0 0x7f0ff80c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.211+0000 7f0ff759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0ff80afeb0 0x7f0ff815e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.211+0000 7f0ff759e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff815f090 con 0x7f0ff815ec60 2023-12-19T18:16:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.211+0000 7f0ff759e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff80cbcf0 con 0x7f0ff80afeb0 2023-12-19T18:16:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.211+0000 7f0ff759e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff80b0360 con 0x7f0ff81553a0 2023-12-19T18:16:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.211+0000 7f0ff5d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0ff80afeb0 0x7f0ff815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.211+0000 7f0ff5d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0ff80afeb0 0x7f0ff815e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42672/0 (socket says 172.21.15.27:42672) 2023-12-19T18:16:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.211+0000 7f0ff5d9b640 1 -- 172.21.15.27:0/2789460038 learned_addr learned my addr 172.21.15.27:0/2789460038 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.212+0000 7f0ff659c640 1 --2- 172.21.15.27:0/2789460038 >> v2:172.21.15.192:3300/0 conn(0x7f0ff81553a0 0x7f0ff80c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.212+0000 7f0ff5d9b640 1 -- 172.21.15.27:0/2789460038 >> v2:172.21.15.192:3300/0 conn(0x7f0ff81553a0 msgr2=0x7f0ff80c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:32.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.212+0000 7f0ff5d9b640 1 --2- 172.21.15.27:0/2789460038 >> v2:172.21.15.192:3300/0 conn(0x7f0ff81553a0 0x7f0ff80c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.212+0000 7f0ff5d9b640 1 -- 172.21.15.27:0/2789460038 >> v1:172.21.15.192:6789/0 conn(0x7f0ff815ec60 legacy=0x7f0ff815b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:32.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.212+0000 7f0ff5d9b640 1 -- 172.21.15.27:0/2789460038 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff80cb350 con 0x7f0ff80afeb0 2023-12-19T18:16:32.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.213+0000 7f0ff5d9b640 1 --2- 172.21.15.27:0/2789460038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ff80afeb0 0x7f0ff815e720 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0ff806d4a0 tx=0x7f0fec02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4a2970a2a4d317d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:32.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.213+0000 7f0ff559a640 1 -- 172.21.15.27:0/2789460038 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec0317c0 con 0x7f0ff80afeb0 2023-12-19T18:16:32.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.213+0000 7f0ff559a640 1 -- 172.21.15.27:0/2789460038 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fec031920 con 0x7f0ff80afeb0 2023-12-19T18:16:32.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.213+0000 7f0ff559a640 1 -- 172.21.15.27:0/2789460038 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec031cb0 con 0x7f0ff80afeb0 2023-12-19T18:16:32.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.214+0000 7f0ff759e640 1 -- 172.21.15.27:0/2789460038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ff80afeb0 msgr2=0x7f0ff815e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.214+0000 7f0ff759e640 1 --2- 172.21.15.27:0/2789460038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ff80afeb0 0x7f0ff815e720 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f0ff806d4a0 tx=0x7f0fec02b860 comp rx=0 tx=0).stop 2023-12-19T18:16:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.214+0000 7f0ff759e640 1 -- 172.21.15.27:0/2789460038 shutdown_connections 2023-12-19T18:16:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.214+0000 7f0ff759e640 1 --2- 172.21.15.27:0/2789460038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ff80afeb0 0x7f0ff815e720 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.214+0000 7f0ff759e640 1 --2- 172.21.15.27:0/2789460038 >> v2:172.21.15.192:3300/0 conn(0x7f0ff81553a0 0x7f0ff80c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.214+0000 7f0ff759e640 1 -- 172.21.15.27:0/2789460038 >> 172.21.15.27:0/2789460038 conn(0x7f0ff8069c70 msgr2=0x7f0ff806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.214+0000 7f0ff759e640 1 -- 172.21.15.27:0/2789460038 shutdown_connections 2023-12-19T18:16:32.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.214+0000 7f0ff759e640 1 -- 172.21.15.27:0/2789460038 wait complete. 2023-12-19T18:16:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.216+0000 7f0ff759e640 1 Processor -- start 2023-12-19T18:16:32.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.216+0000 7f0ff759e640 1 -- start start 2023-12-19T18:16:32.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.217+0000 7f0ff759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ff80afeb0 0x7f0ff81f0740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.217+0000 7f0ff759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ff81553a0 0x7f0ff8155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.217+0000 7f0ff759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ff815ec60 0x7f0ff81561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.217+0000 7f0ff759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff80b3070 con 0x7f0ff80afeb0 2023-12-19T18:16:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.217+0000 7f0ff759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8155f50 con 0x7f0ff81553a0 2023-12-19T18:16:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.217+0000 7f0ff759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff815c750 con 0x7f0ff815ec60 2023-12-19T18:16:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.217+0000 7f0ff659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ff80afeb0 0x7f0ff81f0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.217+0000 7f0ff659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ff80afeb0 0x7f0ff81f0740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42680/0 (socket says 172.21.15.27:42680) 2023-12-19T18:16:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.217+0000 7f0ff659c640 1 -- 172.21.15.27:0/1542574137 learned_addr learned my addr 172.21.15.27:0/1542574137 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.218+0000 7f0ff5d9b640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ff81553a0 0x7f0ff8155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.218+0000 7f0ff659c640 1 -- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ff815ec60 msgr2=0x7f0ff81561b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.218+0000 7f0ff6d9d640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ff815ec60 0x7f0ff81561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.218+0000 7f0ff659c640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ff815ec60 0x7f0ff81561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.218+0000 7f0ff659c640 1 -- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ff81553a0 msgr2=0x7f0ff8155a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.218+0000 7f0ff659c640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ff81553a0 0x7f0ff8155a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.218+0000 7f0ff659c640 1 -- 172.21.15.27:0/1542574137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec0094d0 con 0x7f0ff80afeb0 2023-12-19T18:16:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.218+0000 7f0ff5d9b640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ff81553a0 0x7f0ff8155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.218+0000 7f0ff659c640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ff80afeb0 0x7f0ff81f0740 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800d800 tx=0x7f0fe800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.218+0000 7f0fe6ffd640 1 -- 172.21.15.27:0/1542574137 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe800b6c0 con 0x7f0ff80afeb0 2023-12-19T18:16:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.219+0000 7f0fe6ffd640 1 -- 172.21.15.27:0/1542574137 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe800b820 con 0x7f0ff80afeb0 2023-12-19T18:16:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.219+0000 7f0fe6ffd640 1 -- 172.21.15.27:0/1542574137 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe800bbb0 con 0x7f0ff80afeb0 2023-12-19T18:16:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.219+0000 7f0ff759e640 1 -- 172.21.15.27:0/1542574137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff81f0d90 con 0x7f0ff80afeb0 2023-12-19T18:16:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.219+0000 7f0ff759e640 1 -- 172.21.15.27:0/1542574137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff81f1350 con 0x7f0ff80afeb0 2023-12-19T18:16:32.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.220+0000 7f0ff759e640 1 -- 172.21.15.27:0/1542574137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff804e840 con 0x7f0ff80afeb0 2023-12-19T18:16:32.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.221+0000 7f0fe6ffd640 1 -- 172.21.15.27:0/1542574137 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0fe800bd10 con 0x7f0ff80afeb0 2023-12-19T18:16:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.222+0000 7f0fe6ffd640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0fd806a430 0x7f0fd806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:32.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.222+0000 7f0fe6ffd640 1 -- 172.21.15.27:0/1542574137 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0fe8088dd0 con 0x7f0ff80afeb0 2023-12-19T18:16:32.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.223+0000 7f0ff5d9b640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0fd806a430 0x7f0fd806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:32.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.224+0000 7f0ff5d9b640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0fd806a430 0x7f0fd806c8f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0fec036000 tx=0x7f0fec03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:32.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.226+0000 7f0fe6ffd640 1 -- 172.21.15.27:0/1542574137 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0fe8054450 con 0x7f0ff80afeb0 2023-12-19T18:16:32.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.456+0000 7f0ff759e640 1 -- 172.21.15.27:0/1542574137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ff80af6a0 con 0x7f0ff80afeb0 2023-12-19T18:16:32.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.457+0000 7f0fe6ffd640 1 -- 172.21.15.27:0/1542574137 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f0fe8057a70 con 0x7f0ff80afeb0 2023-12-19T18:16:32.457 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:32.457 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3178168231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3178168231},{"type":"v1","addr":"172.21.15.192:6837","nonce":3178168231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3961149394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3961149394},{"type":"v1","addr":"172.21.15.192:6836","nonce":3961149394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5190,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/752784926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":752784926},{"type":"v1","addr":"172.21.15.27:6833","nonce":752784926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:25.779852+0000","modified":"2023-12-19T18:16:29.303970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/3463287343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3463287343},{"type":"v1","addr":"172.21.15.27:6835","nonce":3463287343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T18:16:32.457 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 15 2023-12-19T18:16:32.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.461+0000 7f0ff759e640 1 -- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0fd806a430 msgr2=0x7f0fd806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.461+0000 7f0ff759e640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0fd806a430 0x7f0fd806c8f0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0fec036000 tx=0x7f0fec03d040 comp rx=0 tx=0).stop 2023-12-19T18:16:32.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.461+0000 7f0ff759e640 1 -- 172.21.15.27:0/1542574137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ff80afeb0 msgr2=0x7f0ff81f0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:32.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.461+0000 7f0ff759e640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ff80afeb0 0x7f0ff81f0740 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800d800 tx=0x7f0fe800dcc0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.461+0000 7f0ff759e640 1 -- 172.21.15.27:0/1542574137 shutdown_connections 2023-12-19T18:16:32.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.461+0000 7f0ff759e640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0fd806a430 0x7f0fd806c8f0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.461+0000 7f0ff759e640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ff815ec60 0x7f0ff81561b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.461+0000 7f0ff759e640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ff81553a0 0x7f0ff8155a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.461+0000 7f0ff759e640 1 --2- 172.21.15.27:0/1542574137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ff80afeb0 0x7f0ff81f0740 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:32.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.461+0000 7f0ff759e640 1 -- 172.21.15.27:0/1542574137 >> 172.21.15.27:0/1542574137 conn(0x7f0ff8069c70 msgr2=0x7f0ff80c4d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:32.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.461+0000 7f0ff759e640 1 -- 172.21.15.27:0/1542574137 shutdown_connections 2023-12-19T18:16:32.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:32.461+0000 7f0ff759e640 1 -- 172.21.15.27:0/1542574137 wait complete. 2023-12-19T18:16:32.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 15, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:16:25.779852+0000', 'modified': '2023-12-19T18:16:29.303970+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5196}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5196': {'gid': 5196, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.27:6835/3463287343', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6834', 'nonce': 3463287343}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 3463287343}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:16:32.882 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:16:32.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-19T18:16:33.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.120+0000 7f67a6d9f640 1 Processor -- start 2023-12-19T18:16:33.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.120+0000 7f67a6d9f640 1 -- start start 2023-12-19T18:16:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.121+0000 7f67a6d9f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a006ab00 0x7f67a00d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.121+0000 7f67a6d9f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f67a00d2af0 0x7f67a00d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.121+0000 7f67a6d9f640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67a00d1a50 con 0x7f67a015d260 2023-12-19T18:16:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.121+0000 7f67a6d9f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67a00cdaf0 con 0x7f67a00d2af0 2023-12-19T18:16:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.121+0000 7f67a6d9f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a00b71b0 con 0x7f67a006ab00 2023-12-19T18:16:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.122+0000 7f67a533c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f67a00d2af0 0x7f67a00d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.122+0000 7f679ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a006ab00 0x7f67a00d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.122+0000 7f67a533c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f67a00d2af0 0x7f67a00d13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:34452/0 (socket says 172.21.15.27:34452) 2023-12-19T18:16:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.122+0000 7f67a533c640 1 -- 172.21.15.27:0/1480478583 learned_addr learned my addr 172.21.15.27:0/1480478583 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:33.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.122+0000 7f67a4b3b640 1 --1- 172.21.15.27:0/1480478583 >> v1:172.21.15.192:6789/0 conn(0x7f67a015d260 0x7f67a006a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:44312/0 (socket says 172.21.15.27:44312) 2023-12-19T18:16:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.123+0000 7f679ffff640 1 -- 172.21.15.27:0/1480478583 >> v2:172.21.15.192:3300/0 conn(0x7f67a00d2af0 msgr2=0x7f67a00d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.123+0000 7f679ffff640 1 --2- 172.21.15.27:0/1480478583 >> v2:172.21.15.192:3300/0 conn(0x7f67a00d2af0 0x7f67a00d13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.123+0000 7f679ffff640 1 -- 172.21.15.27:0/1480478583 >> v1:172.21.15.192:6789/0 conn(0x7f67a015d260 legacy=0x7f67a006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.123+0000 7f679ffff640 1 -- 172.21.15.27:0/1480478583 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67a00d1d30 con 0x7f67a006ab00 2023-12-19T18:16:33.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.123+0000 7f67a533c640 1 --2- 172.21.15.27:0/1480478583 >> v2:172.21.15.192:3300/0 conn(0x7f67a00d2af0 0x7f67a00d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:16:33.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.125+0000 7f679ffff640 1 --2- 172.21.15.27:0/1480478583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a006ab00 0x7f67a00d0e70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67880097a0 tx=0x7f678802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20b35c75bc297f4e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.125+0000 7f679f7fe640 1 -- 172.21.15.27:0/1480478583 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678802d040 con 0x7f67a006ab00 2023-12-19T18:16:33.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.125+0000 7f679f7fe640 1 -- 172.21.15.27:0/1480478583 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6788031810 con 0x7f67a006ab00 2023-12-19T18:16:33.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.125+0000 7f679f7fe640 1 -- 172.21.15.27:0/1480478583 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6788031ba0 con 0x7f67a006ab00 2023-12-19T18:16:33.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.126+0000 7f67a6d9f640 1 -- 172.21.15.27:0/1480478583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a006ab00 msgr2=0x7f67a00d0e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.126+0000 7f67a6d9f640 1 --2- 172.21.15.27:0/1480478583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a006ab00 0x7f67a00d0e70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f67880097a0 tx=0x7f678802b8b0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.126+0000 7f67a6d9f640 1 -- 172.21.15.27:0/1480478583 shutdown_connections 2023-12-19T18:16:33.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.126+0000 7f67a6d9f640 1 --2- 172.21.15.27:0/1480478583 >> v2:172.21.15.192:3300/0 conn(0x7f67a00d2af0 0x7f67a00d13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.126+0000 7f67a6d9f640 1 --2- 172.21.15.27:0/1480478583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a006ab00 0x7f67a00d0e70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.126+0000 7f67a6d9f640 1 -- 172.21.15.27:0/1480478583 >> 172.21.15.27:0/1480478583 conn(0x7f67a00d2f70 msgr2=0x7f67a00d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.126+0000 7f67a6d9f640 1 -- 172.21.15.27:0/1480478583 shutdown_connections 2023-12-19T18:16:33.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.126+0000 7f67a6d9f640 1 -- 172.21.15.27:0/1480478583 wait complete. 2023-12-19T18:16:33.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.128+0000 7f67a6d9f640 1 Processor -- start 2023-12-19T18:16:33.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.129+0000 7f67a6d9f640 1 -- start start 2023-12-19T18:16:33.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a6d9f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a006ab00 0x7f67a01f2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a6d9f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a00d2af0 0x7f67a015f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a6d9f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67a015d260 0x7f67a015fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a6d9f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a01f2790 con 0x7f67a006ab00 2023-12-19T18:16:33.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a6d9f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a015e1f0 con 0x7f67a015d260 2023-12-19T18:16:33.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a6d9f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67a015e4d0 con 0x7f67a00d2af0 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a4b3b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a006ab00 0x7f67a01f2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a4b3b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a006ab00 0x7f67a01f2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42686/0 (socket says 172.21.15.27:42686) 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a4b3b640 1 -- 172.21.15.27:0/283654096 learned_addr learned my addr 172.21.15.27:0/283654096 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f679ffff640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a00d2af0 0x7f67a015f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a533c640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67a015d260 0x7f67a015fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a4b3b640 1 -- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a00d2af0 msgr2=0x7f67a015f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a00d2af0 0x7f67a015f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a4b3b640 1 -- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67a015d260 msgr2=0x7f67a015fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.130+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67a015d260 0x7f67a015fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.131+0000 7f67a4b3b640 1 -- 172.21.15.27:0/283654096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67980094d0 con 0x7f67a006ab00 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.131+0000 7f67a533c640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67a015d260 0x7f67a015fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.131+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a006ab00 0x7f67a01f2250 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f679000c7a0 tx=0x7f679000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.131+0000 7f679d7fa640 1 -- 172.21.15.27:0/283654096 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679000a640 con 0x7f67a006ab00 2023-12-19T18:16:33.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.131+0000 7f679d7fa640 1 -- 172.21.15.27:0/283654096 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f679000a7a0 con 0x7f67a006ab00 2023-12-19T18:16:33.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.131+0000 7f679d7fa640 1 -- 172.21.15.27:0/283654096 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679000ab10 con 0x7f67a006ab00 2023-12-19T18:16:33.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.131+0000 7f67a6d9f640 1 -- 172.21.15.27:0/283654096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6788009400 con 0x7f67a006ab00 2023-12-19T18:16:33.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.131+0000 7f67a6d9f640 1 -- 172.21.15.27:0/283654096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a014c1e0 con 0x7f67a006ab00 2023-12-19T18:16:33.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.134+0000 7f679d7fa640 1 -- 172.21.15.27:0/283654096 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f679000ac70 con 0x7f67a006ab00 2023-12-19T18:16:33.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.135+0000 7f679d7fa640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f678006a430 0x7f678006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.136+0000 7f679d7fa640 1 -- 172.21.15.27:0/283654096 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6790087ff0 con 0x7f67a006ab00 2023-12-19T18:16:33.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.136+0000 7f67a99ef740 1 Processor -- start 2023-12-19T18:16:33.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.136+0000 7f67a99ef740 1 -- start start 2023-12-19T18:16:33.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.136+0000 7f679ffff640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f678006a430 0x7f678006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.137+0000 7f67a99ef740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 0x557f49946330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.137+0000 7f67a99ef740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49915ed0 0x557f499431c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.137+0000 7f67a99ef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f49919ad0 0x557f49943700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.137+0000 7f67a99ef740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f498f1650 con 0x557f49945f50 2023-12-19T18:16:33.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.137+0000 7f67a99ef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f49886000 con 0x557f49919ad0 2023-12-19T18:16:33.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.137+0000 7f67a99ef740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f49883790 con 0x557f49915ed0 2023-12-19T18:16:33.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.137+0000 7f679ffff640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f678006a430 0x7f678006c8f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6788036000 tx=0x7f678803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.137+0000 7f67a4b3b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 0x557f49946330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.137+0000 7f67a4b3b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 0x557f49946330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42702/0 (socket says 172.21.15.27:42702) 2023-12-19T18:16:33.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.137+0000 7f67a4b3b640 1 -- 172.21.15.27:0/185889508 learned_addr learned my addr 172.21.15.27:0/185889508 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:33.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.137+0000 7f679ffff640 1 --2- 172.21.15.27:0/185889508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49915ed0 0x557f499431c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.138+0000 7f67a533c640 1 --2- 172.21.15.27:0/185889508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f49919ad0 0x557f49943700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.138+0000 7f67a4b3b640 1 -- 172.21.15.27:0/185889508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49915ed0 msgr2=0x557f499431c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.138+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/185889508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49915ed0 0x557f499431c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.138+0000 7f67a4b3b640 1 -- 172.21.15.27:0/185889508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f49919ad0 msgr2=0x557f49943700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.138+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/185889508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f49919ad0 0x557f49943700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.138+0000 7f67a4b3b640 1 -- 172.21.15.27:0/185889508 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f49846710 con 0x557f49945f50 2023-12-19T18:16:33.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.138+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/185889508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 0x557f49946330 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6790053880 tx=0x7f6790056ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83bd5ae1fa403e01 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.138+0000 7f67967fc640 1 -- 172.21.15.27:0/185889508 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679005f2b0 con 0x557f49945f50 2023-12-19T18:16:33.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.138+0000 7f67967fc640 1 -- 172.21.15.27:0/185889508 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6790055720 con 0x557f49945f50 2023-12-19T18:16:33.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.138+0000 7f67967fc640 1 -- 172.21.15.27:0/185889508 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6790053a90 con 0x557f49945f50 2023-12-19T18:16:33.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.139+0000 7f67a99ef740 1 -- 172.21.15.27:0/185889508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 msgr2=0x557f49946330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.139+0000 7f67a99ef740 1 --2- 172.21.15.27:0/185889508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 0x557f49946330 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6790053880 tx=0x7f6790056ce0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.139+0000 7f67a99ef740 1 -- 172.21.15.27:0/185889508 shutdown_connections 2023-12-19T18:16:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.139+0000 7f67a99ef740 1 --2- 172.21.15.27:0/185889508 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f49919ad0 0x557f49943700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.139+0000 7f67a99ef740 1 --2- 172.21.15.27:0/185889508 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49915ed0 0x557f499431c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.139+0000 7f67a99ef740 1 --2- 172.21.15.27:0/185889508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 0x557f49946330 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.139+0000 7f67a99ef740 1 -- 172.21.15.27:0/185889508 >> 172.21.15.27:0/185889508 conn(0x557f499181e0 msgr2=0x557f49960d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.139+0000 7f67a99ef740 1 -- 172.21.15.27:0/185889508 shutdown_connections 2023-12-19T18:16:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.139+0000 7f67a99ef740 1 -- 172.21.15.27:0/185889508 wait complete. 2023-12-19T18:16:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.140+0000 7f67a99ef740 1 -- start start 2023-12-19T18:16:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.140+0000 7f67a99ef740 1 Processor -- start 2023-12-19T18:16:33.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.141+0000 7f67a99ef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f49919ad0 0x557f4993f960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.141+0000 7f67a99ef740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49945f50 0x557f4995e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.141+0000 7f67a99ef740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f4992c0e0 0x557f4992c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.141+0000 7f67a99ef740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4987f080 con 0x557f4992c0e0 2023-12-19T18:16:33.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.141+0000 7f67a99ef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f49820850 con 0x557f49919ad0 2023-12-19T18:16:33.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.141+0000 7f67a99ef740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f498204f0 con 0x557f49945f50 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.141+0000 7f67a99ef740 10 client.0 _ll_register_callbacks cb 0x557f498789c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.142+0000 7f67a533c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f4992c0e0 0x557f4992c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.142+0000 7f67a533c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f4992c0e0 0x557f4992c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42718/0 (socket says 172.21.15.27:42718) 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.142+0000 7f67a533c640 1 -- 172.21.15.27:0/2305979896 learned_addr learned my addr 172.21.15.27:0/2305979896 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.142+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/2305979896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f49919ad0 0x557f4993f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.142+0000 7f679ffff640 1 --2- 172.21.15.27:0/2305979896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49945f50 0x557f4995e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.142+0000 7f67a533c640 1 -- 172.21.15.27:0/2305979896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49945f50 msgr2=0x557f4995e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.142+0000 7f67a533c640 1 --2- 172.21.15.27:0/2305979896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49945f50 0x557f4995e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.142+0000 7f67a533c640 1 -- 172.21.15.27:0/2305979896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f49919ad0 msgr2=0x557f4993f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.142+0000 7f67a533c640 1 --2- 172.21.15.27:0/2305979896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f49919ad0 0x557f4993f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.142+0000 7f67a533c640 1 -- 172.21.15.27:0/2305979896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a014c1e0 con 0x557f4992c0e0 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.143+0000 7f67a533c640 1 --2- 172.21.15.27:0/2305979896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f4992c0e0 0x557f4992c540 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6798012570 tx=0x7f6798012a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5ea4c01faca093 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.143+0000 7f677f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:16:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.143+0000 7f677f7fe640 1 -- 172.21.15.27:0/2305979896 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798014820 con 0x557f4992c0e0 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.143+0000 7f677f7fe640 1 -- 172.21.15.27:0/2305979896 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6798014980 con 0x557f4992c0e0 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.143+0000 7f67a99ef740 1 -- 172.21.15.27:0/2305979896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f498a95d0 con 0x557f4992c0e0 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.143+0000 7f677f7fe640 1 -- 172.21.15.27:0/2305979896 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6798014bd0 con 0x557f4992c0e0 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.143+0000 7f67a99ef740 10 client.5382 fetch_fsmap learned FSMap version 15 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.143+0000 7f67a99ef740 1 -- 172.21.15.27:0/2305979896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x557f4987d700 con 0x557f4992c0e0 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.144+0000 7f677f7fe640 1 -- 172.21.15.27:0/2305979896 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557f498a95d0 con 0x557f4992c0e0 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.144+0000 7f677f7fe640 1 -- 172.21.15.27:0/2305979896 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6798020440 con 0x557f4992c0e0 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.144+0000 7f67a99ef740 10 client.5382 fetch_fsmap finished waiting for FSMap version 15 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.144+0000 7f67a99ef740 10 client.5382 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.144+0000 7f67a99ef740 20 client.5382 populate_metadata read hostname 'smithi027' 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.144+0000 7f67a99ef740 1 --2- 172.21.15.27:0/2305979896 >> [v2:172.21.15.27:6834/3463287343,v1:172.21.15.27:6835/3463287343] conn(0x557f49960820 0x557f49960c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.144+0000 7f67a99ef740 4 client.5382 mds_command: new command op to 5196 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.144+0000 7f67a99ef740 1 -- 172.21.15.27:0/2305979896 --> [v2:172.21.15.27:6834/3463287343,v1:172.21.15.27:6835/3463287343] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f498718d0 con 0x557f49960820 2023-12-19T18:16:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.144+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/2305979896 >> [v2:172.21.15.27:6834/3463287343,v1:172.21.15.27:6835/3463287343] conn(0x557f49960820 0x557f49960c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.145+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/2305979896 >> [v2:172.21.15.27:6834/3463287343,v1:172.21.15.27:6835/3463287343] conn(0x557f49960820 0x557f49960c00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=522dcd51b14f68f3 server_cookie=d225260f413a5707 in_seq=0 out_seq=0 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.145+0000 7f677f7fe640 10 client.5382 ms_handle_connect on v2:172.21.15.27:6834/3463287343 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f677f7fe640 1 -- 172.21.15.27:0/2305979896 <== mds.0 v2:172.21.15.27:6834/3463287343 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557f498718d0 con 0x557f49960820 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f677f7fe640 10 client.5382 handle_command_reply: tid=0 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f67a99ef740 1 client.5382 shutdown 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f67a99ef740 1 -- 172.21.15.27:0/2305979896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f4992c0e0 msgr2=0x557f4992c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f67a99ef740 1 --2- 172.21.15.27:0/2305979896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f4992c0e0 0x557f4992c540 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6798012570 tx=0x7f6798012a30 comp rx=0 tx=0).stop 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f67a99ef740 1 -- 172.21.15.27:0/2305979896 shutdown_connections 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f67a99ef740 1 --2- 172.21.15.27:0/2305979896 >> [v2:172.21.15.27:6834/3463287343,v1:172.21.15.27:6835/3463287343] conn(0x557f49960820 0x557f49960c00 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f67a99ef740 1 --2- 172.21.15.27:0/2305979896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f4992c0e0 0x557f4992c540 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f67a99ef740 1 --2- 172.21.15.27:0/2305979896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49945f50 0x557f4995e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f67a99ef740 1 --2- 172.21.15.27:0/2305979896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f49919ad0 0x557f4993f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f67a99ef740 1 -- 172.21.15.27:0/2305979896 >> 172.21.15.27:0/2305979896 conn(0x557f49915ed0 msgr2=0x557f4991a490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f677f7fe640 0 client.5382 ms_handle_reset on v2:172.21.15.27:6834/3463287343 2023-12-19T18:16:33.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f67a99ef740 1 -- 172.21.15.27:0/2305979896 shutdown_connections 2023-12-19T18:16:33.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.146+0000 7f67a99ef740 1 -- 172.21.15.27:0/2305979896 wait complete. 2023-12-19T18:16:33.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.147+0000 7f67a99ef740 20 client.5382 trim_cache size 0 max 16384 2023-12-19T18:16:33.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.164+0000 7f67a99ef740 1 Processor -- start 2023-12-19T18:16:33.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.164+0000 7f67a99ef740 1 -- start start 2023-12-19T18:16:33.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f67a99ef740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49919ad0 0x557f4993f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f67a99ef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f4992c0e0 0x557f4993a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f67a99ef740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49945f50 0x557f49964460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f67a99ef740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f495e6f20 con 0x557f49919ad0 2023-12-19T18:16:33.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f67a99ef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f4991cd60 con 0x557f4992c0e0 2023-12-19T18:16:33.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f67a99ef740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f49963c70 con 0x557f49945f50 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f67a4b3b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49919ad0 0x557f4993f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f67a4b3b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49919ad0 0x557f4993f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42730/0 (socket says 172.21.15.27:42730) 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f67a4b3b640 1 -- 172.21.15.27:0/4253500832 learned_addr learned my addr 172.21.15.27:0/4253500832 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f67a533c640 1 --2- 172.21.15.27:0/4253500832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49945f50 0x557f49964460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f67a4b3b640 1 -- 172.21.15.27:0/4253500832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49945f50 msgr2=0x557f49964460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/4253500832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49945f50 0x557f49964460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f67a4b3b640 1 -- 172.21.15.27:0/4253500832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f4992c0e0 msgr2=0x557f4993a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.165+0000 7f679ffff640 1 --2- 172.21.15.27:0/4253500832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f4992c0e0 0x557f4993a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.166+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/4253500832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f4992c0e0 0x557f4993a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.166+0000 7f67a4b3b640 1 -- 172.21.15.27:0/4253500832 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f498b3280 con 0x557f49919ad0 2023-12-19T18:16:33.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.166+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/4253500832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49919ad0 0x557f4993f770 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6790057080 tx=0x7f679001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b894efbd11bdce28 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.166+0000 7f6796ffd640 1 -- 172.21.15.27:0/4253500832 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679005f330 con 0x557f49919ad0 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.166+0000 7f6796ffd640 1 -- 172.21.15.27:0/4253500832 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67900862b0 con 0x557f49919ad0 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.166+0000 7f6796ffd640 1 -- 172.21.15.27:0/4253500832 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679005d0c0 con 0x557f49919ad0 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.167+0000 7f67a99ef740 1 -- 172.21.15.27:0/4253500832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49919ad0 msgr2=0x557f4993f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.167+0000 7f67a99ef740 1 --2- 172.21.15.27:0/4253500832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49919ad0 0x557f4993f770 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6790057080 tx=0x7f679001d000 comp rx=0 tx=0).stop 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.167+0000 7f67a99ef740 1 -- 172.21.15.27:0/4253500832 shutdown_connections 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.167+0000 7f67a99ef740 1 --2- 172.21.15.27:0/4253500832 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49945f50 0x557f49964460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.167+0000 7f67a99ef740 1 --2- 172.21.15.27:0/4253500832 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f4992c0e0 0x557f4993a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.167+0000 7f67a99ef740 1 --2- 172.21.15.27:0/4253500832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49919ad0 0x557f4993f770 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.167+0000 7f67a99ef740 1 -- 172.21.15.27:0/4253500832 >> 172.21.15.27:0/4253500832 conn(0x557f49915ed0 msgr2=0x557f49949b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.167+0000 7f67a99ef740 1 -- 172.21.15.27:0/4253500832 shutdown_connections 2023-12-19T18:16:33.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.167+0000 7f67a99ef740 1 -- 172.21.15.27:0/4253500832 wait complete. 2023-12-19T18:16:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.168+0000 7f67a99ef740 1 -- start start 2023-12-19T18:16:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.168+0000 7f67a99ef740 1 Processor -- start 2023-12-19T18:16:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a99ef740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49919ad0 0x557f49943200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a99ef740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f4992c0e0 0x557f4993dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a99ef740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 0x557f4991b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a99ef740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f49943740 con 0x557f49945f50 2023-12-19T18:16:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a99ef740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f49982df0 con 0x557f4992c0e0 2023-12-19T18:16:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a99ef740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x557f4993e100 con 0x557f49919ad0 2023-12-19T18:16:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a99ef740 10 client.0 _ll_register_callbacks cb 0x7f6798009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:16:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a533c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 0x557f4991b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a533c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 0x557f4991b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42746/0 (socket says 172.21.15.27:42746) 2023-12-19T18:16:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a533c640 1 -- 172.21.15.27:0/2810075461 learned_addr learned my addr 172.21.15.27:0/2810075461 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:33.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/2810075461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49919ad0 0x557f49943200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f679ffff640 1 --2- 172.21.15.27:0/2810075461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f4992c0e0 0x557f4993dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a533c640 1 -- 172.21.15.27:0/2810075461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49919ad0 msgr2=0x557f49943200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a533c640 1 --2- 172.21.15.27:0/2810075461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49919ad0 0x557f49943200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a533c640 1 -- 172.21.15.27:0/2810075461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f4992c0e0 msgr2=0x557f4993dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a533c640 1 --2- 172.21.15.27:0/2810075461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f4992c0e0 0x557f4993dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.169+0000 7f67a533c640 1 -- 172.21.15.27:0/2810075461 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6790084040 con 0x557f49945f50 2023-12-19T18:16:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.170+0000 7f67a533c640 1 --2- 172.21.15.27:0/2810075461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 0x557f4991b5b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6798000c00 tx=0x7f67980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3db95369b0afa7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.170+0000 7f677f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:16:33.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.170+0000 7f677f7fe640 1 -- 172.21.15.27:0/2810075461 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6798014620 con 0x557f49945f50 2023-12-19T18:16:33.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.170+0000 7f677f7fe640 1 -- 172.21.15.27:0/2810075461 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6798014780 con 0x557f49945f50 2023-12-19T18:16:33.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.170+0000 7f67a99ef740 1 -- 172.21.15.27:0/2810075461 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6790055720 con 0x557f49945f50 2023-12-19T18:16:33.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.170+0000 7f677f7fe640 1 -- 172.21.15.27:0/2810075461 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=15) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6798014980 con 0x557f49945f50 2023-12-19T18:16:33.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.170+0000 7f67a99ef740 10 client.5388 fetch_fsmap learned FSMap version 15 2023-12-19T18:16:33.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.170+0000 7f67a99ef740 1 -- 172.21.15.27:0/2810075461 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=15,monmap=2+}) v3 -- 0x557f4981fa60 con 0x557f49945f50 2023-12-19T18:16:33.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.171+0000 7f677f7fe640 1 -- 172.21.15.27:0/2810075461 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6790055720 con 0x557f49945f50 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.171+0000 7f677f7fe640 1 -- 172.21.15.27:0/2810075461 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 15) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6798014ae0 con 0x557f49945f50 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.171+0000 7f67a99ef740 10 client.5388 fetch_fsmap finished waiting for FSMap version 15 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.171+0000 7f67a99ef740 10 client.5388 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.171+0000 7f67a99ef740 20 client.5388 populate_metadata read hostname 'smithi027' 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.171+0000 7f67a99ef740 1 --2- 172.21.15.27:0/2810075461 >> [v2:172.21.15.27:6834/3463287343,v1:172.21.15.27:6835/3463287343] conn(0x557f499c6960 0x557f4991baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.171+0000 7f67a99ef740 4 client.5388 mds_command: new command op to 5196 tid=0[{"prefix": "status"}] 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.171+0000 7f67a99ef740 1 -- 172.21.15.27:0/2810075461 --> [v2:172.21.15.27:6834/3463287343,v1:172.21.15.27:6835/3463287343] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557f49878100 con 0x557f499c6960 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.171+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/2810075461 >> [v2:172.21.15.27:6834/3463287343,v1:172.21.15.27:6835/3463287343] conn(0x557f499c6960 0x557f4991baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f67a4b3b640 1 --2- 172.21.15.27:0/2810075461 >> [v2:172.21.15.27:6834/3463287343,v1:172.21.15.27:6835/3463287343] conn(0x557f499c6960 0x557f4991baf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=55c47897d4820283 server_cookie=f0f7d99c7c03c60c in_seq=0 out_seq=0 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f677f7fe640 10 client.5388 ms_handle_connect on v2:172.21.15.27:6834/3463287343 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f677f7fe640 1 -- 172.21.15.27:0/2810075461 <== mds.0 v2:172.21.15.27:6834/3463287343 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x557f49878100 con 0x557f499c6960 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f677f7fe640 10 client.5388 handle_command_reply: tid=0 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f67a99ef740 1 client.5388 shutdown 2023-12-19T18:16:33.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f67a99ef740 1 -- 172.21.15.27:0/2810075461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 msgr2=0x557f4991b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f67a99ef740 1 --2- 172.21.15.27:0/2810075461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 0x557f4991b5b0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6798000c00 tx=0x7f67980046c0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f67a99ef740 1 -- 172.21.15.27:0/2810075461 shutdown_connections 2023-12-19T18:16:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f67a99ef740 1 --2- 172.21.15.27:0/2810075461 >> [v2:172.21.15.27:6834/3463287343,v1:172.21.15.27:6835/3463287343] conn(0x557f499c6960 0x557f4991baf0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f67a99ef740 1 --2- 172.21.15.27:0/2810075461 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557f49945f50 0x557f4991b5b0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f67a99ef740 1 --2- 172.21.15.27:0/2810075461 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557f4992c0e0 0x557f4993dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f67a99ef740 1 --2- 172.21.15.27:0/2810075461 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557f49919ad0 0x557f49943200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f677f7fe640 0 client.5388 ms_handle_reset on v2:172.21.15.27:6834/3463287343 2023-12-19T18:16:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f67a99ef740 1 -- 172.21.15.27:0/2810075461 >> 172.21.15.27:0/2810075461 conn(0x557f49915ed0 msgr2=0x557f49960c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f67a99ef740 1 -- 172.21.15.27:0/2810075461 shutdown_connections 2023-12-19T18:16:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.172+0000 7f67a99ef740 1 -- 172.21.15.27:0/2810075461 wait complete. 2023-12-19T18:16:33.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.173+0000 7f67a99ef740 20 client.5388 trim_cache size 0 max 16384 2023-12-19T18:16:33.175 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:16:33.176 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:16:33.176 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-12-19T18:16:33.176 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5196, 2023-12-19T18:16:33.176 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-19T18:16:33.176 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-19T18:16:33.176 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-19T18:16:33.176 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 7.3877141719999999, 2023-12-19T18:16:33.176 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 15, 2023-12-19T18:16:33.176 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 34, 2023-12-19T18:16:33.176 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 34, 2023-12-19T18:16:33.176 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 16.072818871999999 2023-12-19T18:16:33.176 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:16:33.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.173+0000 7f67a6d9f640 1 -- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f678006a430 msgr2=0x7f678006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.173+0000 7f67a6d9f640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f678006a430 0x7f678006c8f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f6788036000 tx=0x7f678803c040 comp rx=0 tx=0).stop 2023-12-19T18:16:33.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.173+0000 7f67a6d9f640 1 -- 172.21.15.27:0/283654096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a006ab00 msgr2=0x7f67a01f2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.173+0000 7f67a6d9f640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a006ab00 0x7f67a01f2250 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f679000c7a0 tx=0x7f679000cc60 comp rx=0 tx=0).stop 2023-12-19T18:16:33.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.173+0000 7f67a6d9f640 1 -- 172.21.15.27:0/283654096 shutdown_connections 2023-12-19T18:16:33.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.173+0000 7f67a6d9f640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f678006a430 0x7f678006c8f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.173+0000 7f67a6d9f640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f67a015d260 0x7f67a015fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.173+0000 7f67a6d9f640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67a00d2af0 0x7f67a015f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.173+0000 7f67a6d9f640 1 --2- 172.21.15.27:0/283654096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a006ab00 0x7f67a01f2250 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.173+0000 7f67a6d9f640 1 -- 172.21.15.27:0/283654096 >> 172.21.15.27:0/283654096 conn(0x7f67a00d2f70 msgr2=0x7f67a00b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.173+0000 7f67a6d9f640 1 -- 172.21.15.27:0/283654096 shutdown_connections 2023-12-19T18:16:33.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.173+0000 7f67a6d9f640 1 -- 172.21.15.27:0/283654096 wait complete. 2023-12-19T18:16:33.191 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:16:33.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.426+0000 7f49c5db0640 1 Processor -- start 2023-12-19T18:16:33.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.426+0000 7f49c5db0640 1 -- start start 2023-12-19T18:16:33.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.428+0000 7f49c5db0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f49c01553a0 0x7f49c00c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.428+0000 7f49c5db0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c00b0360 con 0x7f49c015ee30 2023-12-19T18:16:33.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.428+0000 7f49c5db0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49c00cb350 con 0x7f49c00afeb0 2023-12-19T18:16:33.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.428+0000 7f49c5db0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49c00cbcf0 con 0x7f49c01553a0 2023-12-19T18:16:33.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.428+0000 7f49c55af640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f49c015ee30 0x7f49c00c39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47094/0 (socket says 172.21.15.27:47094) 2023-12-19T18:16:33.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.428+0000 7f49c55af640 1 -- 172.21.15.27:0/814779510 learned_addr learned my addr 172.21.15.27:0/814779510 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:33.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49c4dae640 1 --2- 172.21.15.27:0/814779510 >> v2:172.21.15.192:3300/0 conn(0x7f49c01553a0 0x7f49c00c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 271487109 0 0) 0x7f49c00b0360 con 0x7f49c015ee30 2023-12-19T18:16:33.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b0003520 con 0x7f49c015ee30 2023-12-19T18:16:33.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2142355248 0 0) 0x7f49b0003520 con 0x7f49c015ee30 2023-12-19T18:16:33.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 >> v2:172.21.15.192:3300/0 conn(0x7f49c01553a0 msgr2=0x7f49c00c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 --2- 172.21.15.27:0/814779510 >> v2:172.21.15.192:3300/0 conn(0x7f49c01553a0 0x7f49c00c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 >> v1:172.21.15.192:6789/0 conn(0x7f49c00afeb0 legacy=0x7f49c015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00cb630 con 0x7f49c015ee30 2023-12-19T18:16:33.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f49b80019f0 con 0x7f49c015ee30 2023-12-19T18:16:33.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 >> v1:172.21.15.27:6789/0 conn(0x7f49c015ee30 legacy=0x7f49c00c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 --2- 172.21.15.27:0/814779510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49b0003970 0x7f49b0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 --2- 172.21.15.27:0/814779510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b00047c0 0x7f49b0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 --2- 172.21.15.27:0/814779510 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49b0005180 0x7f49b000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00cbcf0 con 0x7f49b0003970 2023-12-19T18:16:33.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49b000c120 con 0x7f49b00047c0 2023-12-19T18:16:33.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.429+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49b000c2e0 con 0x7f49b0005180 2023-12-19T18:16:33.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.430+0000 7f49c4dae640 1 --2- 172.21.15.27:0/814779510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49b0003970 0x7f49b0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.430+0000 7f49c4dae640 1 -- 172.21.15.27:0/814779510 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49b0005180 msgr2=0x7f49b000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.430+0000 7f49c4dae640 1 --2- 172.21.15.27:0/814779510 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49b0005180 0x7f49b000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.430+0000 7f49bffff640 1 --2- 172.21.15.27:0/814779510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b00047c0 0x7f49b0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.430+0000 7f49c4dae640 1 -- 172.21.15.27:0/814779510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b00047c0 msgr2=0x7f49b0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:33.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.430+0000 7f49c4dae640 1 --2- 172.21.15.27:0/814779510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b00047c0 0x7f49b0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.430+0000 7f49c4dae640 1 -- 172.21.15.27:0/814779510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b0003520 con 0x7f49b0003970 2023-12-19T18:16:33.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.430+0000 7f49c55af640 1 -- 172.21.15.27:0/814779510 reap_dead start 2023-12-19T18:16:33.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.430+0000 7f49bffff640 1 --2- 172.21.15.27:0/814779510 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b00047c0 0x7f49b0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:33.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.431+0000 7f49c4dae640 1 --2- 172.21.15.27:0/814779510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49b0003970 0x7f49b0003d90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009ac0 tx=0x7f49b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def89fdeadb0c8aa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.432+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b402c6b0 con 0x7f49b0003970 2023-12-19T18:16:33.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.432+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b402c810 con 0x7f49b0003970 2023-12-19T18:16:33.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.432+0000 7f49bf7fe640 1 -- 172.21.15.27:0/814779510 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b402cba0 con 0x7f49b0003970 2023-12-19T18:16:33.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.432+0000 7f49c5db0640 1 -- 172.21.15.27:0/814779510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49b0003970 msgr2=0x7f49b0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.432+0000 7f49c5db0640 1 --2- 172.21.15.27:0/814779510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49b0003970 0x7f49b0003d90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009ac0 tx=0x7f49b00057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.432+0000 7f49c5db0640 1 -- 172.21.15.27:0/814779510 shutdown_connections 2023-12-19T18:16:33.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.432+0000 7f49c5db0640 1 --2- 172.21.15.27:0/814779510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49b0003970 0x7f49b0003d90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.432+0000 7f49c5db0640 1 -- 172.21.15.27:0/814779510 >> 172.21.15.27:0/814779510 conn(0x7f49c0069c70 msgr2=0x7f49c006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.432+0000 7f49c5db0640 1 -- 172.21.15.27:0/814779510 shutdown_connections 2023-12-19T18:16:33.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.432+0000 7f49c5db0640 1 -- 172.21.15.27:0/814779510 wait complete. 2023-12-19T18:16:33.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.434+0000 7f49c5db0640 1 Processor -- start 2023-12-19T18:16:33.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.434+0000 7f49c5db0640 1 -- start start 2023-12-19T18:16:33.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.435+0000 7f49c5db0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b0003970 0x7f49c00cdff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.435+0000 7f49c5db0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49c00c8880 0x7f49c00b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.435+0000 7f49c5db0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49c00b4380 0x7f49c006fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.435+0000 7f49c5db0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00b48b0 con 0x7f49c00b4380 2023-12-19T18:16:33.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.435+0000 7f49c5db0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c01f69b0 con 0x7f49b0003970 2023-12-19T18:16:33.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.435+0000 7f49c5db0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c00c8460 con 0x7f49c00c8880 2023-12-19T18:16:33.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.436+0000 7f49c55af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49c00b4380 0x7f49c006fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.436+0000 7f49c55af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49c00b4380 0x7f49c006fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42770/0 (socket says 172.21.15.27:42770) 2023-12-19T18:16:33.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.436+0000 7f49c55af640 1 -- 172.21.15.27:0/1846412354 learned_addr learned my addr 172.21.15.27:0/1846412354 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:33.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.436+0000 7f49c55af640 1 -- 172.21.15.27:0/1846412354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49c00c8880 msgr2=0x7f49c00b3e40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-19T18:16:33.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.436+0000 7f49c55af640 1 --2- 172.21.15.27:0/1846412354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49c00c8880 0x7f49c00b3e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.436+0000 7f49c55af640 1 -- 172.21.15.27:0/1846412354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b0003970 msgr2=0x7f49c00cdff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.436+0000 7f49c55af640 1 --2- 172.21.15.27:0/1846412354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b0003970 0x7f49c00cdff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.436+0000 7f49c55af640 1 -- 172.21.15.27:0/1846412354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49c00ce530 con 0x7f49c00b4380 2023-12-19T18:16:33.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.436+0000 7f49c55af640 1 --2- 172.21.15.27:0/1846412354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49c00b4380 0x7f49c006fd50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f49c0071090 tx=0x7f49b800cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.437+0000 7f49bd7fa640 1 -- 172.21.15.27:0/1846412354 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b800a5b0 con 0x7f49c00b4380 2023-12-19T18:16:33.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.437+0000 7f49bd7fa640 1 -- 172.21.15.27:0/1846412354 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b800a710 con 0x7f49c00b4380 2023-12-19T18:16:33.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.437+0000 7f49bd7fa640 1 -- 172.21.15.27:0/1846412354 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b800aac0 con 0x7f49c00b4380 2023-12-19T18:16:33.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.437+0000 7f49c5db0640 1 -- 172.21.15.27:0/1846412354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c00cc920 con 0x7f49c00b4380 2023-12-19T18:16:33.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.437+0000 7f49c5db0640 1 -- 172.21.15.27:0/1846412354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c01f5ed0 con 0x7f49c00b4380 2023-12-19T18:16:33.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.438+0000 7f49c5db0640 1 -- 172.21.15.27:0/1846412354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c00c5020 con 0x7f49c00b4380 2023-12-19T18:16:33.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.439+0000 7f49bd7fa640 1 -- 172.21.15.27:0/1846412354 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f49b800ac20 con 0x7f49c00b4380 2023-12-19T18:16:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.440+0000 7f49bd7fa640 1 --2- 172.21.15.27:0/1846412354 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f49a406a430 0x7f49a406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.440+0000 7f49bd7fa640 1 -- 172.21.15.27:0/1846412354 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49b8088630 con 0x7f49c00b4380 2023-12-19T18:16:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.441+0000 7f49c4dae640 1 --2- 172.21.15.27:0/1846412354 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f49a406a430 0x7f49a406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.442+0000 7f49c4dae640 1 --2- 172.21.15.27:0/1846412354 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f49a406a430 0x7f49a406c8f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f49b4036000 tx=0x7f49b402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.444+0000 7f49bd7fa640 1 -- 172.21.15.27:0/1846412354 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f49b8053b80 con 0x7f49c00b4380 2023-12-19T18:16:33.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.685+0000 7f49c5db0640 1 -- 172.21.15.27:0/1846412354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f49c004e840 con 0x7f49c00b4380 2023-12-19T18:16:33.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.686+0000 7f49bd7fa640 1 -- 172.21.15.27:0/1846412354 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f49b80571a0 con 0x7f49c00b4380 2023-12-19T18:16:33.686 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:16:33.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.689+0000 7f49c5db0640 1 -- 172.21.15.27:0/1846412354 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f49a406a430 msgr2=0x7f49a406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.689+0000 7f49c5db0640 1 --2- 172.21.15.27:0/1846412354 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f49a406a430 0x7f49a406c8f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f49b4036000 tx=0x7f49b402e000 comp rx=0 tx=0).stop 2023-12-19T18:16:33.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.689+0000 7f49c5db0640 1 -- 172.21.15.27:0/1846412354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49c00b4380 msgr2=0x7f49c006fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:33.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.689+0000 7f49c5db0640 1 --2- 172.21.15.27:0/1846412354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49c00b4380 0x7f49c006fd50 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f49c0071090 tx=0x7f49b800cf00 comp rx=0 tx=0).stop 2023-12-19T18:16:33.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.689+0000 7f49c5db0640 1 -- 172.21.15.27:0/1846412354 shutdown_connections 2023-12-19T18:16:33.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.689+0000 7f49c5db0640 1 --2- 172.21.15.27:0/1846412354 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f49a406a430 0x7f49a406c8f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.689+0000 7f49c5db0640 1 --2- 172.21.15.27:0/1846412354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49c00b4380 0x7f49c006fd50 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.689+0000 7f49c5db0640 1 --2- 172.21.15.27:0/1846412354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49c00c8880 0x7f49c00b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.689+0000 7f49c5db0640 1 --2- 172.21.15.27:0/1846412354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49b0003970 0x7f49c00cdff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:33.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.689+0000 7f49c5db0640 1 -- 172.21.15.27:0/1846412354 >> 172.21.15.27:0/1846412354 conn(0x7f49c0069c70 msgr2=0x7f49c006b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:33.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.689+0000 7f49c5db0640 1 -- 172.21.15.27:0/1846412354 shutdown_connections 2023-12-19T18:16:33.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:33.690+0000 7f49c5db0640 1 -- 172.21.15.27:0/1846412354 wait complete. 2023-12-19T18:16:33.709 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:16:33.709 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:16:33.709 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-12-19T18:16:33.709 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:16:33.709 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:16:33.709 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:16:33.709 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:16:33.709 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:16:33.709 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:16:33.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:33.710 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-12-19T18:16:33.766 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:16:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:16:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:16:33.766 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:16:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:33.767 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:16:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-12-19T18:16:33.767 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:16:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-12-19T18:16:33.767 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:16:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-12-19T18:16:33.767 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-19T18:16:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42247sec preferred_lft 42247sec 2023-12-19T18:16:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2023-12-19T18:16:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:33.768 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:16:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-12-19T18:16:33.768 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:16:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:16:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::8053:76ff:fe69:c66c/64 scope link 2023-12-19T18:16:33.768 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:33.769 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:16:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:16:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::781d:d6ff:feb4:7a2d/64 scope link 2023-12-19T18:16:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:33.769 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:16:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether f2:29:63:cc:14:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:16:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::f029:63ff:fecc:140e/64 scope link 2023-12-19T18:16:33.769 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:33.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:33.770 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:16:33.864 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-12-19T18:16:33.868 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:33.869 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-12-19T18:16:33.925 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-12-19T18:16:33.925 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:16:33.926 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-19T18:16:33.926 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-19T18:16:33.926 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:16:33.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:33.926 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:16:33.926 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:16:33.926 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:16:33.927 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:16:33.927 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:16:33.927 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:16:34.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:34.134 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:16:34.191 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:16:34.192 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:16:34.192 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:16:34.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:34.192 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-12-19T18:16:34.304 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-19T18:16:34.304 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:16:34.305 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:16:34.365 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:16:34.365 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:16:34.365 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:16:34.423 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:16:34.536 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:16:34.536 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:16:34.637 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:16:34.641 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:16:34.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:34.642 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:16:34.705 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:16:34.705 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-19T18:16:34.714 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:16:34.805 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:16:34.805 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:16:34.886 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:16:34.889 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:16:34.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:34.890 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:16:34.927 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-12-19T18:16:34.927+0000 7fc396841480 -1 init, newargv = 0x56440509e7a0 newargc=15 2023-12-19T18:16:34.928 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[93397]: starting ceph client 2023-12-19T18:16:34.936 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[93397]: starting fuse 2023-12-19T18:16:36.034 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:16:36.155 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:16:36.156 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:16:36.261 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:16:36.266 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:16:36.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:36.266 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:16:36.329 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:16:36.329 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-19T18:16:36.329 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:16:36.329 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:16:36.330 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:16:36.331 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:16:36.331 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:16:36.331 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:16:36.331 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:16:36.331 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:16:36.331 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:16:36.331 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:36.331 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:16:36.331 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:16:36.533 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.93397.asok 2023-12-19T18:16:36.712 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.93397.asok 2023-12-19T18:16:36.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:36.713 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.93397.asok status 2023-12-19T18:16:36.975 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:16:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:16:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:16:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:16:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:16:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:16:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:16:36.975 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "93397", 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5400, 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 5400 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:16:36.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3963972847 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3963972847 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.5400 192.168.0.1:0/3963972847", 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3963972847", 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-12-19T18:16:36.977 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 15, 2023-12-19T18:16:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 34, 2023-12-19T18:16:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:16:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:16:36.978 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:16:36.978 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:16:36.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:36.991 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-12-19T18:16:36.991 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-19T18:16:37.048 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-12-19T18:16:37.048 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-19T18:16:37.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:37.048 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:16:37.152 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:16:37.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.385+0000 7f79d7fff640 1 Processor -- start 2023-12-19T18:16:37.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.385+0000 7f79d7fff640 1 -- start start 2023-12-19T18:16:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.387+0000 7f79d7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79d81553a0 0x7f79d80c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.387+0000 7f79d7fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d80b0360 con 0x7f79d815ee30 2023-12-19T18:16:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.387+0000 7f79d7fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79d80cb350 con 0x7f79d80afeb0 2023-12-19T18:16:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.387+0000 7f79d7fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79d80cbcf0 con 0x7f79d81553a0 2023-12-19T18:16:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.387+0000 7f79d6ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79d81553a0 0x7f79d80c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.387+0000 7f79d77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f79d815ee30 0x7f79d80c39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47100/0 (socket says 172.21.15.27:47100) 2023-12-19T18:16:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.387+0000 7f79d77fe640 1 -- 172.21.15.27:0/1880696337 learned_addr learned my addr 172.21.15.27:0/1880696337 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.388+0000 7f79d5ffb640 1 -- 172.21.15.27:0/1880696337 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743339838 0 0) 0x7f79d80b0360 con 0x7f79d815ee30 2023-12-19T18:16:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.388+0000 7f79d5ffb640 1 -- 172.21.15.27:0/1880696337 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79c4003520 con 0x7f79d815ee30 2023-12-19T18:16:37.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.388+0000 7f79d6ffd640 1 -- 172.21.15.27:0/1880696337 >> v1:172.21.15.192:6789/0 conn(0x7f79d80afeb0 legacy=0x7f79d815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.388+0000 7f79d6ffd640 1 -- 172.21.15.27:0/1880696337 >> v1:172.21.15.27:6789/0 conn(0x7f79d815ee30 legacy=0x7f79d80c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.388+0000 7f79d6ffd640 1 -- 172.21.15.27:0/1880696337 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d80cb630 con 0x7f79d81553a0 2023-12-19T18:16:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.389+0000 7f79d6ffd640 1 --2- 172.21.15.27:0/1880696337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d81553a0 0x7f79d80c50a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f79c80097a0 tx=0x7f79c802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba95623c60a516 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.389+0000 7f79d5ffb640 1 -- 172.21.15.27:0/1880696337 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c802e070 con 0x7f79d81553a0 2023-12-19T18:16:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.389+0000 7f79d5ffb640 1 -- 172.21.15.27:0/1880696337 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79c80315a0 con 0x7f79d81553a0 2023-12-19T18:16:37.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.389+0000 7f79d5ffb640 1 -- 172.21.15.27:0/1880696337 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79c8031930 con 0x7f79d81553a0 2023-12-19T18:16:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.390+0000 7f79d7fff640 1 -- 172.21.15.27:0/1880696337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d81553a0 msgr2=0x7f79d80c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.390+0000 7f79d7fff640 1 --2- 172.21.15.27:0/1880696337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d81553a0 0x7f79d80c50a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f79c80097a0 tx=0x7f79c802b840 comp rx=0 tx=0).stop 2023-12-19T18:16:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.390+0000 7f79d7fff640 1 -- 172.21.15.27:0/1880696337 shutdown_connections 2023-12-19T18:16:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.390+0000 7f79d7fff640 1 --2- 172.21.15.27:0/1880696337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d81553a0 0x7f79d80c50a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.390+0000 7f79d7fff640 1 -- 172.21.15.27:0/1880696337 >> 172.21.15.27:0/1880696337 conn(0x7f79d8069c70 msgr2=0x7f79d806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:37.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.390+0000 7f79d7fff640 1 -- 172.21.15.27:0/1880696337 shutdown_connections 2023-12-19T18:16:37.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.390+0000 7f79d7fff640 1 -- 172.21.15.27:0/1880696337 wait complete. 2023-12-19T18:16:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.392+0000 7f79d7fff640 1 Processor -- start 2023-12-19T18:16:37.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.392+0000 7f79d7fff640 1 -- start start 2023-12-19T18:16:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.393+0000 7f79d7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d80afeb0 0x7f79d8158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.393+0000 7f79d7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d81553a0 0x7f79d814e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.393+0000 7f79d7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d815ee30 0x7f79d80b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.393+0000 7f79d7fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8155ef0 con 0x7f79d815ee30 2023-12-19T18:16:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.393+0000 7f79d7fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8155930 con 0x7f79d81553a0 2023-12-19T18:16:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.393+0000 7f79d7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79d8155c10 con 0x7f79d80afeb0 2023-12-19T18:16:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.393+0000 7f79d77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d815ee30 0x7f79d80b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.393+0000 7f79d77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d815ee30 0x7f79d80b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42790/0 (socket says 172.21.15.27:42790) 2023-12-19T18:16:37.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.394+0000 7f79d77fe640 1 -- 172.21.15.27:0/769325808 learned_addr learned my addr 172.21.15.27:0/769325808 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.394+0000 7f79d6ffd640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d80afeb0 0x7f79d8158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.394+0000 7f79d77fe640 1 -- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d80afeb0 msgr2=0x7f79d8158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.394+0000 7f79d77fe640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d80afeb0 0x7f79d8158100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.394+0000 7f79d67fc640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d81553a0 0x7f79d814e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.394+0000 7f79d77fe640 1 -- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d81553a0 msgr2=0x7f79d814e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.394+0000 7f79d77fe640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d81553a0 0x7f79d814e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.394+0000 7f79d77fe640 1 -- 172.21.15.27:0/769325808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79c8009400 con 0x7f79d815ee30 2023-12-19T18:16:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.394+0000 7f79d67fc640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d81553a0 0x7f79d814e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.394+0000 7f79d77fe640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d815ee30 0x7f79d80b3f70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f79d000caa0 tx=0x7f79d000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.395+0000 7f79cf7fe640 1 -- 172.21.15.27:0/769325808 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d000ac00 con 0x7f79d815ee30 2023-12-19T18:16:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.395+0000 7f79cf7fe640 1 -- 172.21.15.27:0/769325808 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79d000ad60 con 0x7f79d815ee30 2023-12-19T18:16:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.395+0000 7f79cf7fe640 1 -- 172.21.15.27:0/769325808 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79d0012620 con 0x7f79d815ee30 2023-12-19T18:16:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.395+0000 7f79d7fff640 1 -- 172.21.15.27:0/769325808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79d80b3a50 con 0x7f79d815ee30 2023-12-19T18:16:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.395+0000 7f79d7fff640 1 -- 172.21.15.27:0/769325808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79d80b44b0 con 0x7f79d815ee30 2023-12-19T18:16:37.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.396+0000 7f79d7fff640 1 -- 172.21.15.27:0/769325808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79d804e840 con 0x7f79d815ee30 2023-12-19T18:16:37.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.397+0000 7f79cf7fe640 1 -- 172.21.15.27:0/769325808 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f79d0007070 con 0x7f79d815ee30 2023-12-19T18:16:37.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.398+0000 7f79cf7fe640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79b806a430 0x7f79b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:37.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.398+0000 7f79cf7fe640 1 -- 172.21.15.27:0/769325808 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79d00888a0 con 0x7f79d815ee30 2023-12-19T18:16:37.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.399+0000 7f79d6ffd640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79b806a430 0x7f79b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:37.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.400+0000 7f79d6ffd640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79b806a430 0x7f79b806c8f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f79c8036000 tx=0x7f79c803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:37.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.403+0000 7f79cf7fe640 1 -- 172.21.15.27:0/769325808 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f79d0053e90 con 0x7f79d815ee30 2023-12-19T18:16:37.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.637+0000 7f79d7fff640 1 -- 172.21.15.27:0/769325808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f79d80afbb0 con 0x7f79d815ee30 2023-12-19T18:16:37.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.638+0000 7f79cf7fe640 1 -- 172.21.15.27:0/769325808 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v15) v1 ==== 53+0+73 (secure 0 0 0) 0x7f79d0017020 con 0x7f79d815ee30 2023-12-19T18:16:37.638 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:16:37.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.640+0000 7f79d7fff640 1 -- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79b806a430 msgr2=0x7f79b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.640+0000 7f79d7fff640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79b806a430 0x7f79b806c8f0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f79c8036000 tx=0x7f79c803d040 comp rx=0 tx=0).stop 2023-12-19T18:16:37.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.641+0000 7f79d7fff640 1 -- 172.21.15.27:0/769325808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d815ee30 msgr2=0x7f79d80b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:37.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.641+0000 7f79d7fff640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d815ee30 0x7f79d80b3f70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f79d000caa0 tx=0x7f79d000cf60 comp rx=0 tx=0).stop 2023-12-19T18:16:37.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.641+0000 7f79d7fff640 1 -- 172.21.15.27:0/769325808 shutdown_connections 2023-12-19T18:16:37.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.641+0000 7f79d7fff640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79b806a430 0x7f79b806c8f0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.641+0000 7f79d7fff640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79d815ee30 0x7f79d80b3f70 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.641+0000 7f79d7fff640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79d81553a0 0x7f79d814e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.641+0000 7f79d7fff640 1 --2- 172.21.15.27:0/769325808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79d80afeb0 0x7f79d8158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.641+0000 7f79d7fff640 1 -- 172.21.15.27:0/769325808 >> 172.21.15.27:0/769325808 conn(0x7f79d8069c70 msgr2=0x7f79d80c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.641+0000 7f79d7fff640 1 -- 172.21.15.27:0/769325808 shutdown_connections 2023-12-19T18:16:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:37.641+0000 7f79d7fff640 1 -- 172.21.15.27:0/769325808 wait complete. 2023-12-19T18:16:37.659 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:16:37.659 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:16:37.659 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-19T18:16:37.659 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-19T18:16:37.659 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T18:16:37.659 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:16:37.659 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:16:37.659 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:16:37.660 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:16:37.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:37.660 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-19T18:16:37.721 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:16:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:16:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:16:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:16:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:37.722 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:16:37.722 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-19T18:16:37.723 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:16:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-19T18:16:37.723 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:16:37.723 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:37.723 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:16:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-19T18:16:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42206sec preferred_lft 42206sec 2023-12-19T18:16:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-19T18:16:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:37.724 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:16:37.724 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:16:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:16:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::b892:1cff:fedc:4bed/64 scope link 2023-12-19T18:16:37.725 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:37.725 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:16:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether a6:e1:fc:e6:56:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:16:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::a4e1:fcff:fee6:56fa/64 scope link 2023-12-19T18:16:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:37.726 INFO:teuthology.orchestra.run.smithi192.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:16:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T18:16:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::404f:6cff:fe73:9aa2/64 scope link 2023-12-19T18:16:37.726 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:16:37.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:37.728 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:16:37.820 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-19T18:16:37.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:37.824 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-19T18:16:37.880 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-19T18:16:37.880 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:16:37.880 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T18:16:37.880 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T18:16:37.880 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:16:37.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:37.880 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:16:37.881 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:16:37.881 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T18:16:37.881 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T18:16:37.881 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T18:16:37.881 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:16:38.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:38.050 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:16:38.100 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T18:16:38.100 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-19T18:16:38.101 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:16:38.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:38.102 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-19T18:16:38.196 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-19T18:16:38.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:16:38.196 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-19T18:16:38.252 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:16:38.253 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:16:38.253 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:16:38.305 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:16:38.395 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:16:38.395 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:16:38.491 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:16:38.494 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:16:38.495 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:38.495 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:16:38.545 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:16:38.546 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-19T18:16:38.551 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:16:38.629 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:16:38.630 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:16:38.701 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:16:38.704 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:16:38.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:38.704 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:16:38.740 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-19T18:16:38.740+0000 7fe03823b480 -1 init, newargv = 0x55c3e80ce420 newargc=15 2023-12-19T18:16:38.741 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[86432]: starting ceph client 2023-12-19T18:16:38.750 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[86432]: starting fuse 2023-12-19T18:16:39.850 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:16:39.954 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:16:39.955 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:16:40.047 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:16:40.050 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:16:40.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:40.051 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:16:40.104 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:16:40.105 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-12-19T18:16:40.105 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:16:40.105 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-19T18:16:40.105 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-19T18:16:40.105 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-19T18:16:40.105 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-19T18:16:40.105 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:16:40.105 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-19T18:16:40.105 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-19T18:16:40.105 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-19T18:16:40.105 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-19T18:16:40.105 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-19T18:16:40.106 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-19T18:16:40.284 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.86432.asok 2023-12-19T18:16:40.464 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.86432.asok 2023-12-19T18:16:40.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:40.465 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.86432.asok status 2023-12-19T18:16:40.713 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "86432", 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5050, 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-19T18:16:40.714 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5050 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1320886399 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1320886399 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5050 192.168.0.1:0/1320886399", 2023-12-19T18:16:40.715 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/1320886399", 2023-12-19T18:16:40.716 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-19T18:16:40.716 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 15, 2023-12-19T18:16:40.716 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 34, 2023-12-19T18:16:40.716 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:16:40.716 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-19T18:16:40.716 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-19T18:16:40.716 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-19T18:16:40.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:40.728 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-19T18:16:40.728 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-19T18:16:40.785 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-19T18:16:40.785 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-19T18:16:40.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:40.785 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:16:40.875 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:41.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.118+0000 7f4ff8282640 1 Processor -- start 2023-12-19T18:16:41.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.118+0000 7f4ff8282640 1 -- start start 2023-12-19T18:16:41.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.119+0000 7f4ff8282640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff00cea40 0x7f4ff00d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.119+0000 7f4ff8282640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4ff006b390 0x7f4ff006bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.119+0000 7f4ff8282640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ff015e4c0 con 0x7f4ff015ae80 2023-12-19T18:16:41.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.119+0000 7f4ff8282640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ff006c3e0 con 0x7f4ff006b390 2023-12-19T18:16:41.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.119+0000 7f4ff8282640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff015e7a0 con 0x7f4ff00cea40 2023-12-19T18:16:41.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.120+0000 7f4ff601e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4ff015ae80 0x7f4ff006ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47106/0 (socket says 172.21.15.27:47106) 2023-12-19T18:16:41.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.120+0000 7f4ff601e640 1 -- 172.21.15.27:0/242134782 learned_addr learned my addr 172.21.15.27:0/242134782 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.120+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81866215 0 0) 0x7f4ff015e4c0 con 0x7f4ff015ae80 2023-12-19T18:16:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.120+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fe0003520 con 0x7f4ff015ae80 2023-12-19T18:16:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.120+0000 7f4ff681f640 1 --2- 172.21.15.27:0/242134782 >> v2:172.21.15.192:3300/0 conn(0x7f4ff006b390 0x7f4ff006bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 190267179 0 0) 0x7f4fe0003520 con 0x7f4ff015ae80 2023-12-19T18:16:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff00cea40 msgr2=0x7f4ff00d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff581d640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff00cea40 0x7f4ff00d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff00cea40 0x7f4ff00d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 >> v2:172.21.15.192:3300/0 conn(0x7f4ff006b390 msgr2=0x7f4ff006bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 --2- 172.21.15.27:0/242134782 >> v2:172.21.15.192:3300/0 conn(0x7f4ff006b390 0x7f4ff006bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff006c940 con 0x7f4ff015ae80 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f4fe40019f0 con 0x7f4ff015ae80 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 >> v1:172.21.15.27:6789/0 conn(0x7f4ff015ae80 legacy=0x7f4ff006ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fe0003970 0x7f4fe0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe00047c0 0x7f4fe0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fe00051d0 0x7f4fe000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff015e7a0 con 0x7f4fe00047c0 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe000c100 con 0x7f4fe00051d0 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fe000c2c0 con 0x7f4fe0003970 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff681f640 1 --2- 172.21.15.27:0/242134782 >> v2:172.21.15.192:3300/0 conn(0x7f4ff006b390 0x7f4ff006bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff581d640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ff00cea40 0x7f4ff00d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.121+0000 7f4ff581d640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe00047c0 0x7f4fe0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.122+0000 7f4ff601e640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fe0003970 0x7f4fe0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.122+0000 7f4ff681f640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fe00051d0 0x7f4fe000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.122+0000 7f4ff581d640 1 -- 172.21.15.27:0/242134782 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fe0003970 msgr2=0x7f4fe0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.122+0000 7f4ff581d640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fe0003970 0x7f4fe0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.122+0000 7f4ff581d640 1 -- 172.21.15.27:0/242134782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fe00051d0 msgr2=0x7f4fe000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.122+0000 7f4ff581d640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fe00051d0 0x7f4fe000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.122+0000 7f4ff581d640 1 -- 172.21.15.27:0/242134782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fe0003520 con 0x7f4fe00047c0 2023-12-19T18:16:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.122+0000 7f4ff681f640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fe00051d0 0x7f4fe000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.123+0000 7f4ff581d640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe00047c0 0x7f4fe0004c40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4ff006c550 tx=0x7f4fe00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf4c29dae5728f6e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.123+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc032620 con 0x7f4fe00047c0 2023-12-19T18:16:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.123+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc0327a0 con 0x7f4fe00047c0 2023-12-19T18:16:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.123+0000 7f4ff501c640 1 -- 172.21.15.27:0/242134782 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc032b90 con 0x7f4fe00047c0 2023-12-19T18:16:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.124+0000 7f4ff8282640 1 -- 172.21.15.27:0/242134782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe00047c0 msgr2=0x7f4fe0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.124+0000 7f4ff8282640 1 --2- 172.21.15.27:0/242134782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fe00047c0 0x7f4fe0004c40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f4ff006c550 tx=0x7f4fe00057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.124+0000 7f4ff681f640 1 -- 172.21.15.27:0/242134782 reap_dead start 2023-12-19T18:16:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.124+0000 7f4ff8282640 1 -- 172.21.15.27:0/242134782 shutdown_connections 2023-12-19T18:16:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.124+0000 7f4ff8282640 1 -- 172.21.15.27:0/242134782 >> 172.21.15.27:0/242134782 conn(0x7f4ff00cd810 msgr2=0x7f4ff00cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:41.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.124+0000 7f4ff8282640 1 -- 172.21.15.27:0/242134782 shutdown_connections 2023-12-19T18:16:41.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.124+0000 7f4ff8282640 1 -- 172.21.15.27:0/242134782 wait complete. 2023-12-19T18:16:41.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.126+0000 7f4ff8282640 1 Processor -- start 2023-12-19T18:16:41.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.126+0000 7f4ff8282640 1 -- start start 2023-12-19T18:16:41.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.127+0000 7f4ff8282640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fe00047c0 0x7f4ff00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.127+0000 7f4ff8282640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ff00b4d50 0x7f4ff01582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.127+0000 7f4ff8282640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ff0158fb0 0x7f4ff0159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.127+0000 7f4ff8282640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00b5e60 con 0x7f4ff00b4d50 2023-12-19T18:16:41.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.127+0000 7f4ff8282640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff01599a0 con 0x7f4ff0158fb0 2023-12-19T18:16:41.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.127+0000 7f4ff8282640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ff00af7b0 con 0x7f4fe00047c0 2023-12-19T18:16:41.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.127+0000 7f4ff581d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ff00b4d50 0x7f4ff01582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.127+0000 7f4ff581d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ff00b4d50 0x7f4ff01582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42798/0 (socket says 172.21.15.27:42798) 2023-12-19T18:16:41.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.127+0000 7f4ff581d640 1 -- 172.21.15.27:0/455543155 learned_addr learned my addr 172.21.15.27:0/455543155 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:41.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.128+0000 7f4ff681f640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ff0158fb0 0x7f4ff0159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.128+0000 7f4ff601e640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fe00047c0 0x7f4ff00ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.128+0000 7f4ff581d640 1 -- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fe00047c0 msgr2=0x7f4ff00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.128+0000 7f4ff581d640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fe00047c0 0x7f4ff00ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.128+0000 7f4ff581d640 1 -- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ff0158fb0 msgr2=0x7f4ff0159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.128+0000 7f4ff581d640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ff0158fb0 0x7f4ff0159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.128+0000 7f4ff581d640 1 -- 172.21.15.27:0/455543155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff00ace10 con 0x7f4ff00b4d50 2023-12-19T18:16:41.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.128+0000 7f4ff681f640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ff0158fb0 0x7f4ff0159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:41.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.128+0000 7f4ff581d640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ff00b4d50 0x7f4ff01582a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc034040 tx=0x7f4fdc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:41.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.129+0000 7f4feeffd640 1 -- 172.21.15.27:0/455543155 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc044410 con 0x7f4ff00b4d50 2023-12-19T18:16:41.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.129+0000 7f4feeffd640 1 -- 172.21.15.27:0/455543155 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fdc044590 con 0x7f4ff00b4d50 2023-12-19T18:16:41.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.129+0000 7f4feeffd640 1 -- 172.21.15.27:0/455543155 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fdc044940 con 0x7f4ff00b4d50 2023-12-19T18:16:41.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.129+0000 7f4ff8282640 1 -- 172.21.15.27:0/455543155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ff00b51e0 con 0x7f4ff00b4d50 2023-12-19T18:16:41.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.129+0000 7f4ff8282640 1 -- 172.21.15.27:0/455543155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ff00b57a0 con 0x7f4ff00b4d50 2023-12-19T18:16:41.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.130+0000 7f4ff8282640 1 -- 172.21.15.27:0/455543155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ff0068e90 con 0x7f4ff00b4d50 2023-12-19T18:16:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.131+0000 7f4feeffd640 1 -- 172.21.15.27:0/455543155 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4fdc03b3f0 con 0x7f4ff00b4d50 2023-12-19T18:16:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.132+0000 7f4feeffd640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4fd406a430 0x7f4fd406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.132+0000 7f4feeffd640 1 -- 172.21.15.27:0/455543155 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fdc0aa510 con 0x7f4ff00b4d50 2023-12-19T18:16:41.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.133+0000 7f4ff601e640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4fd406a430 0x7f4fd406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.134+0000 7f4ff601e640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4fd406a430 0x7f4fd406c8f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40013d0 tx=0x7f4fe4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:41.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.137+0000 7f4feeffd640 1 -- 172.21.15.27:0/455543155 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4fdc075cb0 con 0x7f4ff00b4d50 2023-12-19T18:16:41.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.378+0000 7f4ff8282640 1 -- 172.21.15.27:0/455543155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4ff00d1c10 con 0x7f4ff00b4d50 2023-12-19T18:16:41.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.379+0000 7f4feeffd640 1 -- 172.21.15.27:0/455543155 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f4fdc049090 con 0x7f4ff00b4d50 2023-12-19T18:16:41.380 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:41.380 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3178168231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3178168231},{"type":"v1","addr":"172.21.15.192:6837","nonce":3178168231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3961149394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3961149394},{"type":"v1","addr":"172.21.15.192:6836","nonce":3961149394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5190,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/752784926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":752784926},{"type":"v1","addr":"172.21.15.27:6833","nonce":752784926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:25.779852+0000","modified":"2023-12-19T18:16:29.303970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/3463287343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3463287343},{"type":"v1","addr":"172.21.15.27:6835","nonce":3463287343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T18:16:41.380 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 15 2023-12-19T18:16:41.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.382+0000 7f4ff8282640 1 -- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4fd406a430 msgr2=0x7f4fd406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.382+0000 7f4ff8282640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4fd406a430 0x7f4fd406c8f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f4fe40013d0 tx=0x7f4fe4009040 comp rx=0 tx=0).stop 2023-12-19T18:16:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.382+0000 7f4ff8282640 1 -- 172.21.15.27:0/455543155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ff00b4d50 msgr2=0x7f4ff01582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.382+0000 7f4ff8282640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ff00b4d50 0x7f4ff01582a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f4fdc034040 tx=0x7f4fdc004730 comp rx=0 tx=0).stop 2023-12-19T18:16:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.382+0000 7f4ff8282640 1 -- 172.21.15.27:0/455543155 shutdown_connections 2023-12-19T18:16:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.382+0000 7f4ff8282640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4fd406a430 0x7f4fd406c8f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.382+0000 7f4ff8282640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ff0158fb0 0x7f4ff0159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.382+0000 7f4ff8282640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ff00b4d50 0x7f4ff01582a0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.382+0000 7f4ff8282640 1 --2- 172.21.15.27:0/455543155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fe00047c0 0x7f4ff00ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.382+0000 7f4ff8282640 1 -- 172.21.15.27:0/455543155 >> 172.21.15.27:0/455543155 conn(0x7f4ff00cd810 msgr2=0x7f4ff00d0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.383+0000 7f4ff8282640 1 -- 172.21.15.27:0/455543155 shutdown_connections 2023-12-19T18:16:41.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.383+0000 7f4ff8282640 1 -- 172.21.15.27:0/455543155 wait complete. 2023-12-19T18:16:41.402 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-12-19T18:16:41.633 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":5196,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.831801885000001,"mdsmap_epoch":15,"osdmap_epoch":34,"osdmap_epoch_barrier":34,"uptime":24.516919714} 2023-12-19T18:16:41.633 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 5196, "mdsmap_epoch": 15, "osdmap_epoch": 34, "osdmap_epoch_barrier": 34, "rank_uptime": 15.831801885, "state": "up:active", "uptime": 24.516919714, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:16:41.634 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph df --format=json-pretty 2023-12-19T18:16:41.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.872+0000 7f3d40228640 1 Processor -- start 2023-12-19T18:16:41.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.872+0000 7f3d40228640 1 -- start start 2023-12-19T18:16:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.874+0000 7f3d40228640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d3806cec0 0x7f3d3806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.874+0000 7f3d40228640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3d380c9770 0x7f3d380cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.874+0000 7f3d40228640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3d380cd990 0x7f3d380cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.874+0000 7f3d40228640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d3815a330 con 0x7f3d380c9770 2023-12-19T18:16:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.874+0000 7f3d40228640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d380cc720 con 0x7f3d380cd990 2023-12-19T18:16:41.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.874+0000 7f3d40228640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380b1a00 con 0x7f3d3806cec0 2023-12-19T18:16:41.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.874+0000 7f3d3d7c3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3d380c9770 0x7f3d380cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.874+0000 7f3d3d7c3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3d380c9770 0x7f3d380cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42810/0 (socket says 172.21.15.27:42810) 2023-12-19T18:16:41.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.874+0000 7f3d3d7c3640 1 -- 172.21.15.27:0/1278922878 learned_addr learned my addr 172.21.15.27:0/1278922878 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:41.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.874+0000 7f3d3e7c5640 1 --2- 172.21.15.27:0/1278922878 >> v2:172.21.15.192:3300/0 conn(0x7f3d380cd990 0x7f3d380cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.874+0000 7f3d3dfc4640 1 --2- 172.21.15.27:0/1278922878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d3806cec0 0x7f3d3806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.875+0000 7f3d3d7c3640 1 -- 172.21.15.27:0/1278922878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d3806cec0 msgr2=0x7f3d3806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:41.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.875+0000 7f3d3d7c3640 1 --2- 172.21.15.27:0/1278922878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d3806cec0 0x7f3d3806d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.875+0000 7f3d3d7c3640 1 -- 172.21.15.27:0/1278922878 >> v2:172.21.15.192:3300/0 conn(0x7f3d380cd990 msgr2=0x7f3d380cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.875+0000 7f3d3d7c3640 1 --2- 172.21.15.27:0/1278922878 >> v2:172.21.15.192:3300/0 conn(0x7f3d380cd990 0x7f3d380cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.875+0000 7f3d3d7c3640 1 -- 172.21.15.27:0/1278922878 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d3806d7e0 con 0x7f3d380c9770 2023-12-19T18:16:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.875+0000 7f3d3dfc4640 1 --2- 172.21.15.27:0/1278922878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d3806cec0 0x7f3d3806d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:41.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.875+0000 7f3d3e7c5640 1 --2- 172.21.15.27:0/1278922878 >> v2:172.21.15.192:3300/0 conn(0x7f3d380cd990 0x7f3d380cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.876+0000 7f3d3d7c3640 1 --2- 172.21.15.27:0/1278922878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d380c9770 0x7f3d380cd450 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3d38074190 tx=0x7f3d2c02b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0082ff7932740f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.876+0000 7f3d3cfc2640 1 -- 172.21.15.27:0/1278922878 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c031860 con 0x7f3d380c9770 2023-12-19T18:16:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.876+0000 7f3d3cfc2640 1 -- 172.21.15.27:0/1278922878 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2c0319c0 con 0x7f3d380c9770 2023-12-19T18:16:41.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.876+0000 7f3d3cfc2640 1 -- 172.21.15.27:0/1278922878 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c031d50 con 0x7f3d380c9770 2023-12-19T18:16:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.877+0000 7f3d40228640 1 -- 172.21.15.27:0/1278922878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d380c9770 msgr2=0x7f3d380cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.877+0000 7f3d40228640 1 --2- 172.21.15.27:0/1278922878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d380c9770 0x7f3d380cd450 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f3d38074190 tx=0x7f3d2c02b900 comp rx=0 tx=0).stop 2023-12-19T18:16:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.877+0000 7f3d40228640 1 -- 172.21.15.27:0/1278922878 shutdown_connections 2023-12-19T18:16:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.877+0000 7f3d40228640 1 --2- 172.21.15.27:0/1278922878 >> v2:172.21.15.192:3300/0 conn(0x7f3d380cd990 0x7f3d380cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.877+0000 7f3d40228640 1 --2- 172.21.15.27:0/1278922878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d380c9770 0x7f3d380cd450 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.877+0000 7f3d40228640 1 --2- 172.21.15.27:0/1278922878 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d3806cec0 0x7f3d3806d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.877+0000 7f3d40228640 1 -- 172.21.15.27:0/1278922878 >> 172.21.15.27:0/1278922878 conn(0x7f3d380ceb50 msgr2=0x7f3d380cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.877+0000 7f3d40228640 1 -- 172.21.15.27:0/1278922878 shutdown_connections 2023-12-19T18:16:41.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.877+0000 7f3d40228640 1 -- 172.21.15.27:0/1278922878 wait complete. 2023-12-19T18:16:41.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.879+0000 7f3d40228640 1 Processor -- start 2023-12-19T18:16:41.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.879+0000 7f3d40228640 1 -- start start 2023-12-19T18:16:41.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d40228640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d3806cec0 0x7f3d38157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d40228640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d380c9770 0x7f3d380b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d40228640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d380cd990 0x7f3d381f2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d40228640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380addf0 con 0x7f3d380c9770 2023-12-19T18:16:41.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d40228640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380b3250 con 0x7f3d3806cec0 2023-12-19T18:16:41.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d40228640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d380b34a0 con 0x7f3d380cd990 2023-12-19T18:16:41.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d3d7c3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d380c9770 0x7f3d380b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d3d7c3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d380c9770 0x7f3d380b4260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42816/0 (socket says 172.21.15.27:42816) 2023-12-19T18:16:41.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d3d7c3640 1 -- 172.21.15.27:0/1584275896 learned_addr learned my addr 172.21.15.27:0/1584275896 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:41.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d3d7c3640 1 -- 172.21.15.27:0/1584275896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d380cd990 msgr2=0x7f3d381f2650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d3d7c3640 1 --2- 172.21.15.27:0/1584275896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d380cd990 0x7f3d381f2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d3d7c3640 1 -- 172.21.15.27:0/1584275896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d3806cec0 msgr2=0x7f3d38157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:41.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d3d7c3640 1 --2- 172.21.15.27:0/1584275896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d3806cec0 0x7f3d38157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:41.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.881+0000 7f3d3d7c3640 1 -- 172.21.15.27:0/1584275896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d2c0094d0 con 0x7f3d380c9770 2023-12-19T18:16:41.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.882+0000 7f3d3d7c3640 1 --2- 172.21.15.27:0/1584275896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d380c9770 0x7f3d380b4260 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c000c00 tx=0x7f3d2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:41.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.882+0000 7f3d2a7fc640 1 -- 172.21.15.27:0/1584275896 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c03ad10 con 0x7f3d380c9770 2023-12-19T18:16:41.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.882+0000 7f3d2a7fc640 1 -- 172.21.15.27:0/1584275896 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d2c03ae70 con 0x7f3d380c9770 2023-12-19T18:16:41.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.882+0000 7f3d40228640 1 -- 172.21.15.27:0/1584275896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d380b3780 con 0x7f3d380c9770 2023-12-19T18:16:41.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.882+0000 7f3d2a7fc640 1 -- 172.21.15.27:0/1584275896 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c044610 con 0x7f3d380c9770 2023-12-19T18:16:41.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.882+0000 7f3d40228640 1 -- 172.21.15.27:0/1584275896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d380b3d40 con 0x7f3d380c9770 2023-12-19T18:16:41.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.883+0000 7f3d40228640 1 -- 172.21.15.27:0/1584275896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d380b3f60 con 0x7f3d380c9770 2023-12-19T18:16:41.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.884+0000 7f3d2a7fc640 1 -- 172.21.15.27:0/1584275896 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3d2c044770 con 0x7f3d380c9770 2023-12-19T18:16:41.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.886+0000 7f3d2a7fc640 1 --2- 172.21.15.27:0/1584275896 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3d1c06a430 0x7f3d1c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:41.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.886+0000 7f3d2a7fc640 1 -- 172.21.15.27:0/1584275896 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3d2c0ab0d0 con 0x7f3d380c9770 2023-12-19T18:16:41.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.886+0000 7f3d3dfc4640 1 --2- 172.21.15.27:0/1584275896 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3d1c06a430 0x7f3d1c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:41.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.887+0000 7f3d3dfc4640 1 --2- 172.21.15.27:0/1584275896 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3d1c06a430 0x7f3d1c06c8f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3d34006fd0 tx=0x7f3d34008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:41.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:41.890+0000 7f3d2a7fc640 1 -- 172.21.15.27:0/1584275896 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3d2c076620 con 0x7f3d380c9770 2023-12-19T18:16:42.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.154+0000 7f3d40228640 1 -- 172.21.15.27:0/1584275896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "df", "format": "json-pretty"} v 0) v1 -- 0x7f3d38068e90 con 0x7f3d380c9770 2023-12-19T18:16:42.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.155+0000 7f3d2a7fc640 1 -- 172.21.15.27:0/1584275896 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "df", "format": "json-pretty"}]=0 v0) v1 ==== 75+0+1620 (secure 0 0 0) 0x7f3d2c079c40 con 0x7f3d380c9770 2023-12-19T18:16:42.155 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:42.155 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "stats": { 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "total_bytes": 1600000000, 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "total_avail_bytes": 1597603728, 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "total_used_bytes": 2396272, 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "total_used_raw_bytes": 2396272, 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "total_used_raw_ratio": 0.0014976699603721499, 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "num_osds": 8, 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "num_per_pool_osds": 0, 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "num_per_pool_omap_osds": 0 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "stats_by_class": { 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "hdd": { 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "total_bytes": 1600000000, 2023-12-19T18:16:42.156 INFO:teuthology.orchestra.run.smithi027.stdout: "total_avail_bytes": 1597603728, 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "total_used_bytes": 2396272, 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "total_used_raw_bytes": 2396272, 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "total_used_raw_ratio": 0.0014976699603721499 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "name": ".mgr", 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 1, 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "stats": { 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "stored": 590368, 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "objects": 2, 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "kb_used": 577, 2023-12-19T18:16:42.157 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_used": 590368, 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "percent_used": 0.00052964512724429369, 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "max_avail": 557028928 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephfs_metadata", 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 4, 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "stats": { 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "stored": 2804, 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "objects": 22, 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "kb_used": 3, 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_used": 2804, 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "percent_used": 2.5169185846607434e-06, 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: "max_avail": 557028928 2023-12-19T18:16:42.158 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "name": "cephfs_data", 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5, 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "stats": { 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "stored": 0, 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "objects": 0, 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "kb_used": 0, 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "bytes_used": 0, 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "percent_used": 0, 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: "max_avail": 557028928 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:16:42.159 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:16:42.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.158+0000 7f3d40228640 1 -- 172.21.15.27:0/1584275896 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3d1c06a430 msgr2=0x7f3d1c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:42.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.158+0000 7f3d40228640 1 --2- 172.21.15.27:0/1584275896 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3d1c06a430 0x7f3d1c06c8f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3d34006fd0 tx=0x7f3d34008040 comp rx=0 tx=0).stop 2023-12-19T18:16:42.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.158+0000 7f3d40228640 1 -- 172.21.15.27:0/1584275896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d380c9770 msgr2=0x7f3d380b4260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:42.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.158+0000 7f3d40228640 1 --2- 172.21.15.27:0/1584275896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d380c9770 0x7f3d380b4260 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c000c00 tx=0x7f3d2c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.158+0000 7f3d40228640 1 -- 172.21.15.27:0/1584275896 shutdown_connections 2023-12-19T18:16:42.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.158+0000 7f3d40228640 1 --2- 172.21.15.27:0/1584275896 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3d1c06a430 0x7f3d1c06c8f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.158+0000 7f3d40228640 1 --2- 172.21.15.27:0/1584275896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3d380cd990 0x7f3d381f2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.158+0000 7f3d40228640 1 --2- 172.21.15.27:0/1584275896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d380c9770 0x7f3d380b4260 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.158+0000 7f3d40228640 1 --2- 172.21.15.27:0/1584275896 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3d3806cec0 0x7f3d38157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:42.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.158+0000 7f3d40228640 1 -- 172.21.15.27:0/1584275896 >> 172.21.15.27:0/1584275896 conn(0x7f3d380ceb50 msgr2=0x7f3d380c3100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:42.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.158+0000 7f3d40228640 1 -- 172.21.15.27:0/1584275896 shutdown_connections 2023-12-19T18:16:42.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:42.158+0000 7f3d40228640 1 -- 172.21.15.27:0/1584275896 wait complete. 2023-12-19T18:16:42.177 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:16:42.177 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:16:42.177 DEBUG:teuthology.orchestra.run.smithi027:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-12-19T18:16:42.177 DEBUG:teuthology.orchestra.run.smithi027:> os.fchmod(fd, 0O666) 2023-12-19T18:16:42.177 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(fd) 2023-12-19T18:16:42.178 DEBUG:teuthology.orchestra.run.smithi027:> os.close(fd) 2023-12-19T18:16:42.178 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:16:42.509 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:16:42.509 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:16:42.509 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:16:42.509 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:16:42.509 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:16:42.509 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:42.509 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:16:42.509 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:16:42.509 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:16:42.509 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:16:42.509 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:16:42.510 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:16:42.756 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.93397.asok 2023-12-19T18:16:42.936 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.93397.asok 2023-12-19T18:16:42.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:42.937 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.93397.asok status 2023-12-19T18:16:43.197 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:16:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:16:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:16:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:16:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:16:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:16:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:16:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "93397", 2023-12-19T18:16:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 1, 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5400, 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 5400 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3963972847 2023-12-19T18:16:43.198 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:16:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:16:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:16:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3963972847 2023-12-19T18:16:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.5400 192.168.0.1:0/3963972847", 2023-12-19T18:16:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3963972847", 2023-12-19T18:16:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 2, 2023-12-19T18:16:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 15, 2023-12-19T18:16:43.199 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 34, 2023-12-19T18:16:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 34, 2023-12-19T18:16:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:16:43.200 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:16:43.200 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:16:43.214 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-12-19T18:16:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.457+0000 7f40dd48a640 1 Processor -- start 2023-12-19T18:16:43.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.457+0000 7f40dd48a640 1 -- start start 2023-12-19T18:16:43.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.459+0000 7f40dd48a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40d80b49d0 0x7f40d8160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:43.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.459+0000 7f40dd48a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f40d815cf90 0x7f40d80d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:43.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.459+0000 7f40dd48a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d80b5ca0 con 0x7f40d80b89b0 2023-12-19T18:16:43.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.459+0000 7f40dd48a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d8161800 con 0x7f40d815cf90 2023-12-19T18:16:43.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.459+0000 7f40dd48a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d8161240 con 0x7f40d80b49d0 2023-12-19T18:16:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.459+0000 7f40d7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40d80b49d0 0x7f40d8160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.460+0000 7f40d7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40d80b49d0 0x7f40d8160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:58934/0 (socket says 172.21.15.27:58934) 2023-12-19T18:16:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.460+0000 7f40d7fff640 1 -- 172.21.15.27:0/3952953801 learned_addr learned my addr 172.21.15.27:0/3952953801 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.460+0000 7f40dcc89640 1 --2- 172.21.15.27:0/3952953801 >> v2:172.21.15.192:3300/0 conn(0x7f40d815cf90 0x7f40d80d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:43.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.460+0000 7f40d7fff640 1 -- 172.21.15.27:0/3952953801 >> v2:172.21.15.192:3300/0 conn(0x7f40d815cf90 msgr2=0x7f40d80d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.460+0000 7f40d7fff640 1 --2- 172.21.15.27:0/3952953801 >> v2:172.21.15.192:3300/0 conn(0x7f40d815cf90 0x7f40d80d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.460+0000 7f40d7fff640 1 -- 172.21.15.27:0/3952953801 >> v1:172.21.15.192:6789/0 conn(0x7f40d80b89b0 legacy=0x7f40d815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.460+0000 7f40d7fff640 1 -- 172.21.15.27:0/3952953801 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d8161520 con 0x7f40d80b49d0 2023-12-19T18:16:43.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.462+0000 7f40d7fff640 1 --2- 172.21.15.27:0/3952953801 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40d80b49d0 0x7f40d8160940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009850 tx=0x7f40c802b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d848242f6f6a25b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.462+0000 7f40d6ffd640 1 -- 172.21.15.27:0/3952953801 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8004230 con 0x7f40d80b49d0 2023-12-19T18:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.462+0000 7f40d6ffd640 1 -- 172.21.15.27:0/3952953801 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c8004390 con 0x7f40d80b49d0 2023-12-19T18:16:43.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.462+0000 7f40d6ffd640 1 -- 172.21.15.27:0/3952953801 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8003810 con 0x7f40d80b49d0 2023-12-19T18:16:43.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.463+0000 7f40dd48a640 1 -- 172.21.15.27:0/3952953801 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40d80b49d0 msgr2=0x7f40d8160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:43.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.463+0000 7f40dd48a640 1 --2- 172.21.15.27:0/3952953801 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40d80b49d0 0x7f40d8160940 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f40c8009850 tx=0x7f40c802b9f0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.463+0000 7f40dd48a640 1 -- 172.21.15.27:0/3952953801 shutdown_connections 2023-12-19T18:16:43.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.463+0000 7f40dd48a640 1 --2- 172.21.15.27:0/3952953801 >> v2:172.21.15.192:3300/0 conn(0x7f40d815cf90 0x7f40d80d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.463+0000 7f40dd48a640 1 --2- 172.21.15.27:0/3952953801 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40d80b49d0 0x7f40d8160940 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.463+0000 7f40dd48a640 1 -- 172.21.15.27:0/3952953801 >> 172.21.15.27:0/3952953801 conn(0x7f40d80cf760 msgr2=0x7f40d80cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:43.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.463+0000 7f40dd48a640 1 -- 172.21.15.27:0/3952953801 shutdown_connections 2023-12-19T18:16:43.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.463+0000 7f40dd48a640 1 -- 172.21.15.27:0/3952953801 wait complete. 2023-12-19T18:16:43.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.465+0000 7f40dd48a640 1 Processor -- start 2023-12-19T18:16:43.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.465+0000 7f40dd48a640 1 -- start start 2023-12-19T18:16:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.466+0000 7f40dd48a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40d80b49d0 0x7f40d8150ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.466+0000 7f40dd48a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40d80b89b0 0x7f40d80cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.466+0000 7f40dd48a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d815cf90 0x7f40d80c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.466+0000 7f40dd48a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d8154c80 con 0x7f40d815cf90 2023-12-19T18:16:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.466+0000 7f40dd48a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d8154f60 con 0x7f40d80b49d0 2023-12-19T18:16:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.466+0000 7f40dd48a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d8155240 con 0x7f40d80b89b0 2023-12-19T18:16:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.466+0000 7f40dcc89640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d815cf90 0x7f40d80c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.467+0000 7f40dcc89640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d815cf90 0x7f40d80c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55324/0 (socket says 172.21.15.27:55324) 2023-12-19T18:16:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.467+0000 7f40dcc89640 1 -- 172.21.15.27:0/2967486714 learned_addr learned my addr 172.21.15.27:0/2967486714 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.467+0000 7f40d77fe640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40d80b89b0 0x7f40d80cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:43.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.467+0000 7f40d7fff640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40d80b49d0 0x7f40d8150ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:43.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.467+0000 7f40dcc89640 1 -- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40d80b89b0 msgr2=0x7f40d80cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:43.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.467+0000 7f40dcc89640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40d80b89b0 0x7f40d80cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.467+0000 7f40dcc89640 1 -- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40d80b49d0 msgr2=0x7f40d8150ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:43.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.467+0000 7f40dcc89640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40d80b49d0 0x7f40d8150ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:43.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.467+0000 7f40dcc89640 1 -- 172.21.15.27:0/2967486714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40c8009460 con 0x7f40d815cf90 2023-12-19T18:16:43.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.467+0000 7f40d7fff640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40d80b49d0 0x7f40d8150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:43.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.468+0000 7f40dcc89640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d815cf90 0x7f40d80c72e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f40d000efc0 tx=0x7f40d000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:43.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.468+0000 7f40d4ff9640 1 -- 172.21.15.27:0/2967486714 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d0009ba0 con 0x7f40d815cf90 2023-12-19T18:16:43.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.468+0000 7f40d4ff9640 1 -- 172.21.15.27:0/2967486714 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d000f040 con 0x7f40d815cf90 2023-12-19T18:16:43.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.468+0000 7f40dd48a640 1 -- 172.21.15.27:0/2967486714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d8155520 con 0x7f40d815cf90 2023-12-19T18:16:43.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.468+0000 7f40d4ff9640 1 -- 172.21.15.27:0/2967486714 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d00146c0 con 0x7f40d815cf90 2023-12-19T18:16:43.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.468+0000 7f40dd48a640 1 -- 172.21.15.27:0/2967486714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d8151020 con 0x7f40d815cf90 2023-12-19T18:16:43.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.469+0000 7f40dd48a640 1 -- 172.21.15.27:0/2967486714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d8155a40 con 0x7f40d815cf90 2023-12-19T18:16:43.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.470+0000 7f40d4ff9640 1 -- 172.21.15.27:0/2967486714 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f40d0014820 con 0x7f40d815cf90 2023-12-19T18:16:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.471+0000 7f40d4ff9640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f40c406a430 0x7f40c406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:43.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.471+0000 7f40d4ff9640 1 -- 172.21.15.27:0/2967486714 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f40d00894a0 con 0x7f40d815cf90 2023-12-19T18:16:43.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.472+0000 7f40d7fff640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f40c406a430 0x7f40c406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:43.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.473+0000 7f40d7fff640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f40c406a430 0x7f40c406c8f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f40c8035000 tx=0x7f40c8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:43.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.476+0000 7f40d4ff9640 1 -- 172.21.15.27:0/2967486714 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f40d8155a40 con 0x7f40d815cf90 2023-12-19T18:16:43.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:43.688+0000 7f40dd48a640 1 -- 172.21.15.27:0/2967486714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7f40d8155a40 con 0x7f40d815cf90 2023-12-19T18:16:44.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.662+0000 7f40d4ff9640 1 -- 172.21.15.27:0/2967486714 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v37) v1 ==== 93+0+0 (secure 0 0 0) 0x7f40d8155a40 con 0x7f40d815cf90 2023-12-19T18:16:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:pauserd,pausewr is set 2023-12-19T18:16:44.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.665+0000 7f40dd48a640 1 -- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f40c406a430 msgr2=0x7f40c406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.666+0000 7f40dd48a640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f40c406a430 0x7f40c406c8f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f40c8035000 tx=0x7f40c8008030 comp rx=0 tx=0).stop 2023-12-19T18:16:44.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.666+0000 7f40dd48a640 1 -- 172.21.15.27:0/2967486714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d815cf90 msgr2=0x7f40d80c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.666+0000 7f40dd48a640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d815cf90 0x7f40d80c72e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f40d000efc0 tx=0x7f40d000c490 comp rx=0 tx=0).stop 2023-12-19T18:16:44.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.666+0000 7f40dd48a640 1 -- 172.21.15.27:0/2967486714 shutdown_connections 2023-12-19T18:16:44.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.666+0000 7f40dd48a640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f40c406a430 0x7f40c406c8f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.666+0000 7f40dd48a640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40d815cf90 0x7f40d80c72e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.666+0000 7f40dd48a640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40d80b89b0 0x7f40d80cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.666+0000 7f40dd48a640 1 --2- 172.21.15.27:0/2967486714 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40d80b49d0 0x7f40d8150ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.666+0000 7f40dd48a640 1 -- 172.21.15.27:0/2967486714 >> 172.21.15.27:0/2967486714 conn(0x7f40d80cf760 msgr2=0x7f40d80cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:44.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.666+0000 7f40dd48a640 1 -- 172.21.15.27:0/2967486714 shutdown_connections 2023-12-19T18:16:44.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.666+0000 7f40dd48a640 1 -- 172.21.15.27:0/2967486714 wait complete. 2023-12-19T18:16:44.690 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-12-19T18:16:44.824 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:16:44.834 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:16:44.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.924+0000 7f1ec2a21640 1 Processor -- start 2023-12-19T18:16:44.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.924+0000 7f1ec2a21640 1 -- start start 2023-12-19T18:16:44.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.926+0000 7f1ec2a21640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ebc0b8b50 0x7f1ebc0cdd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.926+0000 7f1ec2a21640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1ebc0ce2a0 0x7f1ebc0b6130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.926+0000 7f1ec2a21640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ebc15d660 con 0x7f1ebc15d280 2023-12-19T18:16:44.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.926+0000 7f1ec2a21640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ebc0b8fb0 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.926+0000 7f1ec2a21640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc0ce6f0 con 0x7f1ebc0b8b50 2023-12-19T18:16:44.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.926+0000 7f1ebbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1ebc15d280 0x7f1ebc0d1360 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42236/0 (socket says 172.21.15.27:42236) 2023-12-19T18:16:44.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.926+0000 7f1ebbfff640 1 -- 172.21.15.27:0/2853406217 learned_addr learned my addr 172.21.15.27:0/2853406217 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:44.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.926+0000 7f1ec0fbe640 1 --2- 172.21.15.27:0/2853406217 >> v2:172.21.15.27:3300/0 conn(0x7f1ebc0ce2a0 0x7f1ebc0b6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.926+0000 7f1ebb7fe640 1 --2- 172.21.15.27:0/2853406217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ebc0b8b50 0x7f1ebc0cdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.927+0000 7f1ec0fbe640 1 -- 172.21.15.27:0/2853406217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ebc0b8b50 msgr2=0x7f1ebc0cdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:44.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.927+0000 7f1ec0fbe640 1 --2- 172.21.15.27:0/2853406217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ebc0b8b50 0x7f1ebc0cdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.927+0000 7f1ec0fbe640 1 -- 172.21.15.27:0/2853406217 >> v1:172.21.15.27:6789/0 conn(0x7f1ebc15d280 legacy=0x7f1ebc0d1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.927+0000 7f1ec0fbe640 1 -- 172.21.15.27:0/2853406217 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ebc0d1bb0 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.928+0000 7f1ec0fbe640 1 --2- 172.21.15.27:0/2853406217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ebc0ce2a0 0x7f1ebc0b6130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009d90 tx=0x7f1eb402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59ce56eb8c21054a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.928+0000 7f1ebaffd640 1 -- 172.21.15.27:0/2853406217 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb402cc50 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.928+0000 7f1ebaffd640 1 -- 172.21.15.27:0/2853406217 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb402cdb0 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.928+0000 7f1ebaffd640 1 -- 172.21.15.27:0/2853406217 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4034660 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.929+0000 7f1ec2a21640 1 -- 172.21.15.27:0/2853406217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ebc0ce2a0 msgr2=0x7f1ebc0b6130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.929+0000 7f1ec2a21640 1 --2- 172.21.15.27:0/2853406217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ebc0ce2a0 0x7f1ebc0b6130 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f1eb4009d90 tx=0x7f1eb402bea0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.929+0000 7f1ec2a21640 1 -- 172.21.15.27:0/2853406217 shutdown_connections 2023-12-19T18:16:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.929+0000 7f1ec2a21640 1 --2- 172.21.15.27:0/2853406217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ebc0ce2a0 0x7f1ebc0b6130 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.929+0000 7f1ec2a21640 1 --2- 172.21.15.27:0/2853406217 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ebc0b8b50 0x7f1ebc0cdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.929+0000 7f1ec2a21640 1 -- 172.21.15.27:0/2853406217 >> 172.21.15.27:0/2853406217 conn(0x7f1ebc0d2f90 msgr2=0x7f1ebc0d33a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.929+0000 7f1ec2a21640 1 -- 172.21.15.27:0/2853406217 shutdown_connections 2023-12-19T18:16:44.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.929+0000 7f1ec2a21640 1 -- 172.21.15.27:0/2853406217 wait complete. 2023-12-19T18:16:44.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.931+0000 7f1ec2a21640 1 Processor -- start 2023-12-19T18:16:44.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.931+0000 7f1ec2a21640 1 -- start start 2023-12-19T18:16:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.932+0000 7f1ec2a21640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ebc0b8b50 0x7f1ebc1f1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.932+0000 7f1ec2a21640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ebc0ce2a0 0x7f1ebc15e130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.932+0000 7f1ec2a21640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ebc15d280 0x7f1ebc0b7140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.932+0000 7f1ec2a21640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc1f21f0 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.932+0000 7f1ec2a21640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc160580 con 0x7f1ebc15d280 2023-12-19T18:16:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.932+0000 7f1ec2a21640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ebc156610 con 0x7f1ebc0b8b50 2023-12-19T18:16:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.932+0000 7f1ebb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ebc0ce2a0 0x7f1ebc15e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.933+0000 7f1ebb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ebc0ce2a0 0x7f1ebc15e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55338/0 (socket says 172.21.15.27:55338) 2023-12-19T18:16:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.933+0000 7f1ebb7fe640 1 -- 172.21.15.27:0/1369960047 learned_addr learned my addr 172.21.15.27:0/1369960047 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.933+0000 7f1ebbfff640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ebc0b8b50 0x7f1ebc1f1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.933+0000 7f1ec0fbe640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ebc15d280 0x7f1ebc0b7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.933+0000 7f1ebb7fe640 1 -- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ebc0b8b50 msgr2=0x7f1ebc1f1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.933+0000 7f1ebb7fe640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ebc0b8b50 0x7f1ebc1f1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.933+0000 7f1ebb7fe640 1 -- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ebc15d280 msgr2=0x7f1ebc0b7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.933+0000 7f1ebb7fe640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ebc15d280 0x7f1ebc0b7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:44.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.933+0000 7f1ebb7fe640 1 -- 172.21.15.27:0/1369960047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb4009a70 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.933+0000 7f1ebb7fe640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ebc0ce2a0 0x7f1ebc15e130 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000d5f0 tx=0x7f1eb000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:44.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.934+0000 7f1eb8ff9640 1 -- 172.21.15.27:0/1369960047 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00125f0 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.934+0000 7f1eb8ff9640 1 -- 172.21.15.27:0/1369960047 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0012750 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.934+0000 7f1eb8ff9640 1 -- 172.21.15.27:0/1369960047 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0012b00 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.934+0000 7f1ec2a21640 1 -- 172.21.15.27:0/1369960047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ebc15f970 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.934+0000 7f1ec2a21640 1 -- 172.21.15.27:0/1369960047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ebc15ff30 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.935+0000 7f1ec2a21640 1 -- 172.21.15.27:0/1369960047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ebc067110 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.936+0000 7f1eb8ff9640 1 -- 172.21.15.27:0/1369960047 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1eb0012c60 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.937+0000 7f1eb8ff9640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ea806a430 0x7f1ea806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:44.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.937+0000 7f1eb8ff9640 1 -- 172.21.15.27:0/1369960047 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1eb0088e20 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.937+0000 7f1eb8ff9640 1 -- 172.21.15.27:0/1369960047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=38+}) v3 -- 0x7f1ea80720e0 con 0x7f1ebc0ce2a0 2023-12-19T18:16:44.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.938+0000 7f1ebbfff640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ea806a430 0x7f1ea806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:44.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.939+0000 7f1ebbfff640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ea806a430 0x7f1ea806c8f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1eac005880 tx=0x7f1eac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:44.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:44.941+0000 7f1eb8ff9640 1 -- 172.21.15.27:0/1369960047 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1eb00542f0 con 0x7f1ebc0ce2a0 2023-12-19T18:16:45.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.149+0000 7f1ec2a21640 1 -- 172.21.15.27:0/1369960047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7f1ebc0d0640 con 0x7f1ebc0ce2a0 2023-12-19T18:16:45.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.671+0000 7f1eb8ff9640 1 -- 172.21.15.27:0/1369960047 <== mon.0 v2:172.21.15.27:3300/0 7 ==== osd_map(38..38 src has 1..38) v4 ==== 288+0+0 (secure 0 0 0) 0x7f1eb00586b0 con 0x7f1ebc0ce2a0 2023-12-19T18:16:45.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.671+0000 7f1eb8ff9640 1 -- 172.21.15.27:0/1369960047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=39}) v3 -- 0x7f1ea80729b0 con 0x7f1ebc0ce2a0 2023-12-19T18:16:45.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.677+0000 7f1eb8ff9640 1 -- 172.21.15.27:0/1369960047 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v38) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1eb0057910 con 0x7f1ebc0ce2a0 2023-12-19T18:16:45.679 INFO:teuthology.orchestra.run.smithi027.stderr:pauserd,pausewr is unset 2023-12-19T18:16:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.681+0000 7f1ec2a21640 1 -- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ea806a430 msgr2=0x7f1ea806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.681+0000 7f1ec2a21640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ea806a430 0x7f1ea806c8f0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1eac005880 tx=0x7f1eac009040 comp rx=0 tx=0).stop 2023-12-19T18:16:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.681+0000 7f1ec2a21640 1 -- 172.21.15.27:0/1369960047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ebc0ce2a0 msgr2=0x7f1ebc15e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.681+0000 7f1ec2a21640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ebc0ce2a0 0x7f1ebc15e130 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000d5f0 tx=0x7f1eb000dab0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.681+0000 7f1ec2a21640 1 -- 172.21.15.27:0/1369960047 shutdown_connections 2023-12-19T18:16:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.681+0000 7f1ec2a21640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ea806a430 0x7f1ea806c8f0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.681+0000 7f1ec2a21640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ebc15d280 0x7f1ebc0b7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.681+0000 7f1ec2a21640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ebc0ce2a0 0x7f1ebc15e130 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.681+0000 7f1ec2a21640 1 --2- 172.21.15.27:0/1369960047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ebc0b8b50 0x7f1ebc1f1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.682+0000 7f1ec2a21640 1 -- 172.21.15.27:0/1369960047 >> 172.21.15.27:0/1369960047 conn(0x7f1ebc0d2f90 msgr2=0x7f1ebc0cf7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.682+0000 7f1ec2a21640 1 -- 172.21.15.27:0/1369960047 shutdown_connections 2023-12-19T18:16:45.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.682+0000 7f1ec2a21640 1 -- 172.21.15.27:0/1369960047 wait complete. 2023-12-19T18:16:45.704 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:45.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.929+0000 7fee24e7e640 1 Processor -- start 2023-12-19T18:16:45.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.929+0000 7fee24e7e640 1 -- start start 2023-12-19T18:16:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.931+0000 7fee24e7e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee20154030 0x7fee200cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.931+0000 7fee24e7e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fee200af630 0x7fee200afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.931+0000 7fee24e7e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee200cc190 con 0x7fee2015d620 2023-12-19T18:16:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.931+0000 7fee24e7e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee200b0000 con 0x7fee20154030 2023-12-19T18:16:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.931+0000 7fee24e7e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee200cb2a0 con 0x7fee200af630 2023-12-19T18:16:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.931+0000 7fee1f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee20154030 0x7fee200cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.931+0000 7fee1f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fee20154030 0x7fee200cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55352/0 (socket says 172.21.15.27:55352) 2023-12-19T18:16:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.931+0000 7fee1f7fe640 1 -- 172.21.15.27:0/3481908454 learned_addr learned my addr 172.21.15.27:0/3481908454 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.932+0000 7fee1f7fe640 1 -- 172.21.15.27:0/3481908454 >> v2:172.21.15.192:3300/0 conn(0x7fee200af630 msgr2=0x7fee200afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.932+0000 7fee1f7fe640 1 --2- 172.21.15.27:0/3481908454 >> v2:172.21.15.192:3300/0 conn(0x7fee200af630 0x7fee200afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.932+0000 7fee1f7fe640 1 -- 172.21.15.27:0/3481908454 >> v1:172.21.15.192:6789/0 conn(0x7fee2015d620 legacy=0x7fee2015da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:45.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.932+0000 7fee1f7fe640 1 -- 172.21.15.27:0/3481908454 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee200cb580 con 0x7fee20154030 2023-12-19T18:16:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.933+0000 7fee1f7fe640 1 --2- 172.21.15.27:0/3481908454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee20154030 0x7fee200cd4f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fee1000bb10 tx=0x7fee1002e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f918e1c1bcd125e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.933+0000 7fee1e7fc640 1 -- 172.21.15.27:0/3481908454 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1002ee90 con 0x7fee20154030 2023-12-19T18:16:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.933+0000 7fee1e7fc640 1 -- 172.21.15.27:0/3481908454 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee10036440 con 0x7fee20154030 2023-12-19T18:16:45.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.933+0000 7fee1e7fc640 1 -- 172.21.15.27:0/3481908454 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee100367d0 con 0x7fee20154030 2023-12-19T18:16:45.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.934+0000 7fee24e7e640 1 -- 172.21.15.27:0/3481908454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee20154030 msgr2=0x7fee200cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.934+0000 7fee24e7e640 1 --2- 172.21.15.27:0/3481908454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee20154030 0x7fee200cd4f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fee1000bb10 tx=0x7fee1002e0e0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.934+0000 7fee24e7e640 1 -- 172.21.15.27:0/3481908454 shutdown_connections 2023-12-19T18:16:45.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.934+0000 7fee24e7e640 1 --2- 172.21.15.27:0/3481908454 >> v2:172.21.15.192:3300/0 conn(0x7fee200af630 0x7fee200afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.934+0000 7fee24e7e640 1 --2- 172.21.15.27:0/3481908454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee20154030 0x7fee200cd4f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.934+0000 7fee24e7e640 1 -- 172.21.15.27:0/3481908454 >> 172.21.15.27:0/3481908454 conn(0x7fee20069c70 msgr2=0x7fee2006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:45.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.934+0000 7fee24e7e640 1 -- 172.21.15.27:0/3481908454 shutdown_connections 2023-12-19T18:16:45.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.934+0000 7fee24e7e640 1 -- 172.21.15.27:0/3481908454 wait complete. 2023-12-19T18:16:45.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.936+0000 7fee24e7e640 1 Processor -- start 2023-12-19T18:16:45.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.936+0000 7fee24e7e640 1 -- start start 2023-12-19T18:16:45.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.937+0000 7fee24e7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee200af630 0x7fee200b4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.937+0000 7fee24e7e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee20154030 0x7fee200c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.937+0000 7fee24e7e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2015d620 0x7fee200b3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.937+0000 7fee24e7e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2014c9b0 con 0x7fee2015d620 2023-12-19T18:16:45.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.937+0000 7fee24e7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee2014c0a0 con 0x7fee200af630 2023-12-19T18:16:45.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.937+0000 7fee24e7e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee200b1ef0 con 0x7fee20154030 2023-12-19T18:16:45.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.937+0000 7fee1ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2015d620 0x7fee200b3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.937+0000 7fee1ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2015d620 0x7fee200b3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55368/0 (socket says 172.21.15.27:55368) 2023-12-19T18:16:45.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.937+0000 7fee1ffff640 1 -- 172.21.15.27:0/4089238801 learned_addr learned my addr 172.21.15.27:0/4089238801 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:45.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.938+0000 7fee1effd640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee20154030 0x7fee200c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.938+0000 7fee1f7fe640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee200af630 0x7fee200b4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.938+0000 7fee1ffff640 1 -- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee20154030 msgr2=0x7fee200c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.938+0000 7fee1ffff640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee20154030 0x7fee200c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.938+0000 7fee1ffff640 1 -- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee200af630 msgr2=0x7fee200b4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:45.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.938+0000 7fee1ffff640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee200af630 0x7fee200b4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:45.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.938+0000 7fee1ffff640 1 -- 172.21.15.27:0/4089238801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee1000b770 con 0x7fee2015d620 2023-12-19T18:16:45.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.938+0000 7fee1f7fe640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee200af630 0x7fee200b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:45.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.938+0000 7fee1ffff640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2015d620 0x7fee200b3330 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fee1800b990 tx=0x7fee1800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:45.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.938+0000 7fee07fff640 1 -- 172.21.15.27:0/4089238801 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1800cba0 con 0x7fee2015d620 2023-12-19T18:16:45.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.938+0000 7fee07fff640 1 -- 172.21.15.27:0/4089238801 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1800cd00 con 0x7fee2015d620 2023-12-19T18:16:45.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.938+0000 7fee07fff640 1 -- 172.21.15.27:0/4089238801 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee18014680 con 0x7fee2015d620 2023-12-19T18:16:45.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.938+0000 7fee24e7e640 1 -- 172.21.15.27:0/4089238801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee200c8870 con 0x7fee2015d620 2023-12-19T18:16:45.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.939+0000 7fee24e7e640 1 -- 172.21.15.27:0/4089238801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee2014c2f0 con 0x7fee2015d620 2023-12-19T18:16:45.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.940+0000 7fee24e7e640 1 -- 172.21.15.27:0/4089238801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee2014caf0 con 0x7fee2015d620 2023-12-19T18:16:45.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.941+0000 7fee07fff640 1 -- 172.21.15.27:0/4089238801 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fee1801d3b0 con 0x7fee2015d620 2023-12-19T18:16:45.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.942+0000 7fee07fff640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fee0c06a430 0x7fee0c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:45.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.942+0000 7fee07fff640 1 -- 172.21.15.27:0/4089238801 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fee180888c0 con 0x7fee2015d620 2023-12-19T18:16:45.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.942+0000 7fee1f7fe640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fee0c06a430 0x7fee0c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:45.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.944+0000 7fee1f7fe640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fee0c06a430 0x7fee0c06c8f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fee10038000 tx=0x7fee10008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:45.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:45.946+0000 7fee07fff640 1 -- 172.21.15.27:0/4089238801 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fee2014caf0 con 0x7fee2015d620 2023-12-19T18:16:46.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.141+0000 7fee24e7e640 1 -- 172.21.15.27:0/4089238801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee2014caf0 con 0x7fee2015d620 2023-12-19T18:16:46.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.144+0000 7fee07fff640 1 -- 172.21.15.27:0/4089238801 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16291 (secure 0 0 0) 0x7fee2014caf0 con 0x7fee2015d620 2023-12-19T18:16:46.145 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:46.145 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:16:45.666993+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:23.925853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:24.827898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]},{"pgid":"5.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:46.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.147+0000 7fee24e7e640 1 -- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fee0c06a430 msgr2=0x7fee0c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:46.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.147+0000 7fee24e7e640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fee0c06a430 0x7fee0c06c8f0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fee10038000 tx=0x7fee10008040 comp rx=0 tx=0).stop 2023-12-19T18:16:46.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.147+0000 7fee24e7e640 1 -- 172.21.15.27:0/4089238801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2015d620 msgr2=0x7fee200b3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:46.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.147+0000 7fee24e7e640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2015d620 0x7fee200b3330 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fee1800b990 tx=0x7fee1800be50 comp rx=0 tx=0).stop 2023-12-19T18:16:46.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.147+0000 7fee24e7e640 1 -- 172.21.15.27:0/4089238801 shutdown_connections 2023-12-19T18:16:46.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.147+0000 7fee24e7e640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fee0c06a430 0x7fee0c06c8f0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.147+0000 7fee24e7e640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee2015d620 0x7fee200b3330 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.147+0000 7fee24e7e640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fee20154030 0x7fee200c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.147+0000 7fee24e7e640 1 --2- 172.21.15.27:0/4089238801 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fee200af630 0x7fee200b4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:46.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.147+0000 7fee24e7e640 1 -- 172.21.15.27:0/4089238801 >> 172.21.15.27:0/4089238801 conn(0x7fee20069c70 msgr2=0x7fee2015da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:46.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.147+0000 7fee24e7e640 1 -- 172.21.15.27:0/4089238801 shutdown_connections 2023-12-19T18:16:46.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:46.147+0000 7fee24e7e640 1 -- 172.21.15.27:0/4089238801 wait complete. 2023-12-19T18:16:46.169 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:16:46.169 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:16:46.169 DEBUG:teuthology.orchestra.run.smithi027:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-12-19T18:16:46.169 DEBUG:teuthology.orchestra.run.smithi027:> os.fchmod(fd, 0O666) 2023-12-19T18:16:46.169 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(fd) 2023-12-19T18:16:46.170 DEBUG:teuthology.orchestra.run.smithi027:> os.close(fd) 2023-12-19T18:16:46.170 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:16:46.499 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:16:46.500 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:16:46.501 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:46.501 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:16:46.501 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:16:46.501 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:16:46.501 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:16:46.501 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:16:46.501 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:16:46.501 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:16:46.501 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:16:46.501 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:46.501 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:16:46.501 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:16:46.755 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.93397.asok 2023-12-19T18:16:46.934 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.93397.asok 2023-12-19T18:16:46.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:46.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.93397.asok status 2023-12-19T18:16:47.195 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:16:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:16:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "93397", 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 1, 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5400, 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:16:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 5400 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3963972847 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3963972847 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.5400 192.168.0.1:0/3963972847", 2023-12-19T18:16:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3963972847", 2023-12-19T18:16:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 2, 2023-12-19T18:16:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 15, 2023-12-19T18:16:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 34, 2023-12-19T18:16:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 34, 2023-12-19T18:16:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:16:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:16:47.198 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:16:47.212 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:47.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.445+0000 7fe46ab57640 1 Processor -- start 2023-12-19T18:16:47.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.446+0000 7fe46ab57640 1 -- start start 2023-12-19T18:16:47.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.447+0000 7fe46ab57640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4641604d0 0x7fe4640b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:47.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.447+0000 7fe46ab57640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4640b5e90 con 0x7fe4640d2350 2023-12-19T18:16:47.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.447+0000 7fe46ab57640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4640d1750 con 0x7fe4640b7310 2023-12-19T18:16:47.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.447+0000 7fe46ab57640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640d2b30 con 0x7fe4641604d0 2023-12-19T18:16:47.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.448+0000 7fe469354640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe4640d2350 0x7fe4640d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42250/0 (socket says 172.21.15.27:42250) 2023-12-19T18:16:47.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.448+0000 7fe469354640 1 -- 172.21.15.27:0/2827369066 learned_addr learned my addr 172.21.15.27:0/2827369066 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:47.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.448+0000 7fe469b55640 1 --2- 172.21.15.27:0/2827369066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4641604d0 0x7fe4640b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:47.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.448+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651984234 0 0) 0x7fe4640b5e90 con 0x7fe4640d2350 2023-12-19T18:16:47.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.448+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe450003520 con 0x7fe4640d2350 2023-12-19T18:16:47.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1512357112 0 0) 0x7fe450003520 con 0x7fe4640d2350 2023-12-19T18:16:47.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4641604d0 msgr2=0x7fe4640b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:47.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe468b53640 1 --2- 172.21.15.27:0/2827369066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4641604d0 0x7fe4640b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:47.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 >> v1:172.21.15.192:6789/0 conn(0x7fe4640b7310 legacy=0x7fe4640d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:47.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4640d2850 con 0x7fe4640d2350 2023-12-19T18:16:47.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe454001670 con 0x7fe4640d2350 2023-12-19T18:16:47.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 >> v1:172.21.15.27:6789/0 conn(0x7fe4640d2350 legacy=0x7fe4640d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:47.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe468b53640 1 --2- 172.21.15.27:0/2827369066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4641604d0 0x7fe4500039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe468b53640 1 --2- 172.21.15.27:0/2827369066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe450004020 0x7fe4500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe468b53640 1 --2- 172.21.15.27:0/2827369066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe450004d00 0x7fe450005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640d2b30 con 0x7fe450004020 2023-12-19T18:16:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45000bbc0 con 0x7fe450004d00 2023-12-19T18:16:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45000bd80 con 0x7fe4641604d0 2023-12-19T18:16:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe469354640 1 --2- 172.21.15.27:0/2827369066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe450004020 0x7fe4500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe469b55640 1 --2- 172.21.15.27:0/2827369066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4641604d0 0x7fe4500039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.449+0000 7fe46a356640 1 --2- 172.21.15.27:0/2827369066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe450004d00 0x7fe450005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.450+0000 7fe469354640 1 -- 172.21.15.27:0/2827369066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4641604d0 msgr2=0x7fe4500039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.450+0000 7fe469354640 1 --2- 172.21.15.27:0/2827369066 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4641604d0 0x7fe4500039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.450+0000 7fe469354640 1 -- 172.21.15.27:0/2827369066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe450004d00 msgr2=0x7fe450005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.450+0000 7fe469354640 1 --2- 172.21.15.27:0/2827369066 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe450004d00 0x7fe450005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.450+0000 7fe469354640 1 -- 172.21.15.27:0/2827369066 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe450003520 con 0x7fe450004020 2023-12-19T18:16:47.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.451+0000 7fe469354640 1 --2- 172.21.15.27:0/2827369066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe450004020 0x7fe4500047c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe45400c0e0 tx=0x7fe4500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc37302b45adb8f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:47.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.451+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454005ad0 con 0x7fe450004020 2023-12-19T18:16:47.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.451+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe454005c30 con 0x7fe450004020 2023-12-19T18:16:47.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.451+0000 7fe468b53640 1 -- 172.21.15.27:0/2827369066 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45402ca50 con 0x7fe450004020 2023-12-19T18:16:47.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.452+0000 7fe46ab57640 1 -- 172.21.15.27:0/2827369066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe450004020 msgr2=0x7fe4500047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:47.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.452+0000 7fe46ab57640 1 --2- 172.21.15.27:0/2827369066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe450004020 0x7fe4500047c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe45400c0e0 tx=0x7fe4500057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:47.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.452+0000 7fe46a356640 1 -- 172.21.15.27:0/2827369066 reap_dead start 2023-12-19T18:16:47.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.452+0000 7fe46ab57640 1 -- 172.21.15.27:0/2827369066 shutdown_connections 2023-12-19T18:16:47.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.452+0000 7fe46ab57640 1 -- 172.21.15.27:0/2827369066 >> 172.21.15.27:0/2827369066 conn(0x7fe4640cf7e0 msgr2=0x7fe4640cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:47.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.452+0000 7fe46ab57640 1 -- 172.21.15.27:0/2827369066 shutdown_connections 2023-12-19T18:16:47.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.452+0000 7fe46ab57640 1 -- 172.21.15.27:0/2827369066 wait complete. 2023-12-19T18:16:47.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.454+0000 7fe46ab57640 1 Processor -- start 2023-12-19T18:16:47.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.454+0000 7fe46ab57640 1 -- start start 2023-12-19T18:16:47.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.455+0000 7fe46ab57640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe450004020 0x7fe4641f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:47.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.455+0000 7fe46ab57640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe464158fc0 0x7fe4640c8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.455+0000 7fe46ab57640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4641561b0 0x7fe464156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.455+0000 7fe46ab57640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe464156b70 con 0x7fe4641561b0 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.455+0000 7fe46ab57640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4641f60a0 con 0x7fe464158fc0 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.455+0000 7fe46ab57640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4640b30e0 con 0x7fe450004020 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.455+0000 7fe46a356640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4641561b0 0x7fe464156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.455+0000 7fe46a356640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4641561b0 0x7fe464156630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55388/0 (socket says 172.21.15.27:55388) 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.455+0000 7fe46a356640 1 -- 172.21.15.27:0/1171598872 learned_addr learned my addr 172.21.15.27:0/1171598872 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.456+0000 7fe469354640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe464158fc0 0x7fe4640c8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.456+0000 7fe469b55640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe450004020 0x7fe4641f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.456+0000 7fe46a356640 1 -- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe450004020 msgr2=0x7fe4641f6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.456+0000 7fe46a356640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe450004020 0x7fe4641f6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.456+0000 7fe46a356640 1 -- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe464158fc0 msgr2=0x7fe4640c8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.456+0000 7fe46a356640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe464158fc0 0x7fe4640c8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:47.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.456+0000 7fe46a356640 1 -- 172.21.15.27:0/1171598872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4640b33c0 con 0x7fe4641561b0 2023-12-19T18:16:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.456+0000 7fe46a356640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4641561b0 0x7fe464156630 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe4640d3e60 tx=0x7fe45c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.456+0000 7fe45a7fc640 1 -- 172.21.15.27:0/1171598872 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c003970 con 0x7fe4641561b0 2023-12-19T18:16:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.456+0000 7fe45a7fc640 1 -- 172.21.15.27:0/1171598872 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45c003ad0 con 0x7fe4641561b0 2023-12-19T18:16:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.456+0000 7fe46ab57640 1 -- 172.21.15.27:0/1171598872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4641f5a50 con 0x7fe4641561b0 2023-12-19T18:16:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.456+0000 7fe45a7fc640 1 -- 172.21.15.27:0/1171598872 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe45c003e80 con 0x7fe4641561b0 2023-12-19T18:16:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.457+0000 7fe46ab57640 1 -- 172.21.15.27:0/1171598872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4640c91c0 con 0x7fe4641561b0 2023-12-19T18:16:47.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.458+0000 7fe46ab57640 1 -- 172.21.15.27:0/1171598872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe464066f80 con 0x7fe4641561b0 2023-12-19T18:16:47.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.459+0000 7fe45a7fc640 1 -- 172.21.15.27:0/1171598872 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe45c0050d0 con 0x7fe4641561b0 2023-12-19T18:16:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.460+0000 7fe45a7fc640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe44806a460 0x7fe44806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.460+0000 7fe45a7fc640 1 -- 172.21.15.27:0/1171598872 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fe45c088810 con 0x7fe4641561b0 2023-12-19T18:16:47.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.460+0000 7fe469b55640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe44806a460 0x7fe44806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:47.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.462+0000 7fe469b55640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe44806a460 0x7fe44806c920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe460006fd0 tx=0x7fe460008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:47.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.464+0000 7fe45a7fc640 1 -- 172.21.15.27:0/1171598872 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe45c053ce0 con 0x7fe4641561b0 2023-12-19T18:16:47.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.699+0000 7fe46ab57640 1 -- 172.21.15.27:0/1171598872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4640ce180 con 0x7fe4641561b0 2023-12-19T18:16:47.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.701+0000 7fe45a7fc640 1 -- 172.21.15.27:0/1171598872 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fe45c057300 con 0x7fe4641561b0 2023-12-19T18:16:47.702 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:47.702 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3178168231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3178168231},{"type":"v1","addr":"172.21.15.192:6837","nonce":3178168231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3961149394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3961149394},{"type":"v1","addr":"172.21.15.192:6836","nonce":3961149394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5190,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/752784926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":752784926},{"type":"v1","addr":"172.21.15.27:6833","nonce":752784926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:25.779852+0000","modified":"2023-12-19T18:16:29.303970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/3463287343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3463287343},{"type":"v1","addr":"172.21.15.27:6835","nonce":3463287343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T18:16:47.702 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 15 2023-12-19T18:16:47.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.704+0000 7fe46ab57640 1 -- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe44806a460 msgr2=0x7fe44806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:47.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.704+0000 7fe46ab57640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe44806a460 0x7fe44806c920 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe460006fd0 tx=0x7fe460008040 comp rx=0 tx=0).stop 2023-12-19T18:16:47.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.704+0000 7fe46ab57640 1 -- 172.21.15.27:0/1171598872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4641561b0 msgr2=0x7fe464156630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.704+0000 7fe46ab57640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4641561b0 0x7fe464156630 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe4640d3e60 tx=0x7fe45c00ec10 comp rx=0 tx=0).stop 2023-12-19T18:16:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.704+0000 7fe46ab57640 1 -- 172.21.15.27:0/1171598872 shutdown_connections 2023-12-19T18:16:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.704+0000 7fe46ab57640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe44806a460 0x7fe44806c920 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.704+0000 7fe46ab57640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4641561b0 0x7fe464156630 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.704+0000 7fe46ab57640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe464158fc0 0x7fe4640c8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.704+0000 7fe46ab57640 1 --2- 172.21.15.27:0/1171598872 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe450004020 0x7fe4641f6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.704+0000 7fe46ab57640 1 -- 172.21.15.27:0/1171598872 >> 172.21.15.27:0/1171598872 conn(0x7fe4640cf7e0 msgr2=0x7fe4640caf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.704+0000 7fe46ab57640 1 -- 172.21.15.27:0/1171598872 shutdown_connections 2023-12-19T18:16:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:47.704+0000 7fe46ab57640 1 -- 172.21.15.27:0/1171598872 wait complete. 2023-12-19T18:16:47.725 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json osdmap barrier 38 2023-12-19T18:16:47.977 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-12-19T18:16:47.977 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:16:47.977 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:16:47.978 DEBUG:teuthology.orchestra.run.smithi027:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-12-19T18:16:47.978 DEBUG:teuthology.orchestra.run.smithi027:> os.fchmod(fd, 0O666) 2023-12-19T18:16:47.978 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(fd) 2023-12-19T18:16:47.978 DEBUG:teuthology.orchestra.run.smithi027:> os.close(fd) 2023-12-19T18:16:47.978 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:16:48.293 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:16:48.294 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:48.295 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:16:48.295 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:16:48.295 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:16:48.295 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:16:48.295 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:16:48.295 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:16:48.295 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:16:48.295 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:16:48.295 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:48.295 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:16:48.295 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:16:48.540 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.93397.asok 2023-12-19T18:16:48.720 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.93397.asok 2023-12-19T18:16:48.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:48.720 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.93397.asok status 2023-12-19T18:16:48.978 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:16:48.978 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:16:48.978 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:16:48.979 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:16:48.979 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:16:48.979 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:16:48.979 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:16:48.979 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "93397", 2023-12-19T18:16:48.979 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:16:48.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:48.979 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 2, 2023-12-19T18:16:48.980 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:16:48.980 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5400, 2023-12-19T18:16:48.980 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:16:48.980 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:16:48.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:16:48.980 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 5400 2023-12-19T18:16:48.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:48.980 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:16:48.981 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:16:48.981 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:16:48.981 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3963972847 2023-12-19T18:16:48.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:48.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:48.981 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:16:48.981 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:16:48.981 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:16:48.981 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3963972847 2023-12-19T18:16:48.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:48.982 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.5400 192.168.0.1:0/3963972847", 2023-12-19T18:16:48.982 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3963972847", 2023-12-19T18:16:48.982 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 3, 2023-12-19T18:16:48.982 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 15, 2023-12-19T18:16:48.982 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 38, 2023-12-19T18:16:48.982 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 38, 2023-12-19T18:16:48.982 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:16:48.982 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:16:48.982 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:16:48.995 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:16:48.995 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:16:48.996 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:16:48.996 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:16:48.996 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:16:48.996 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:48.996 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:16:48.996 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:16:48.996 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:16:48.996 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:16:48.996 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:48.996 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:16:48.997 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:16:48.997 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:16:48.997 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:16:48.997 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:48.997 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:16:48.997 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:16:48.997 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:16:48.997 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:16:48.997 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:16:48.998 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:16:48.998 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:16:48.998 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:16:48.998 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:16:48.998 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:16:48.998 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:16:49.203 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.93397.asok 2023-12-19T18:16:49.383 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.93397.asok 2023-12-19T18:16:49.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:49.383 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.93397.asok status 2023-12-19T18:16:49.647 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:16:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:16:49.647 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "93397", 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 2, 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5400, 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:16:49.648 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 5400 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3963972847 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3963972847 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.5400 192.168.0.1:0/3963972847", 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3963972847", 2023-12-19T18:16:49.649 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 3, 2023-12-19T18:16:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 15, 2023-12-19T18:16:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 38, 2023-12-19T18:16:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 38, 2023-12-19T18:16:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:16:49.650 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:16:49.650 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:16:49.663 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-12-19T18:16:49.663 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:16:49.664 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:16:49.802 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:16:49.922 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:16:49.922 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:16:49.922 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:49.988 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:50.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.226+0000 7f33a5f4f640 1 Processor -- start 2023-12-19T18:16:50.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.226+0000 7f33a5f4f640 1 -- start start 2023-12-19T18:16:50.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.228+0000 7f33a5f4f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33a00c50a0 0x7f33a00afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.228+0000 7f33a5f4f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a00c54f0 con 0x7f33a015ee30 2023-12-19T18:16:50.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.228+0000 7f33a5f4f640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33a00cb350 con 0x7f33a01553a0 2023-12-19T18:16:50.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.228+0000 7f33a5f4f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00cbcf0 con 0x7f33a00c50a0 2023-12-19T18:16:50.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.229+0000 7f339ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f33a015ee30 0x7f33a00c39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42256/0 (socket says 172.21.15.27:42256) 2023-12-19T18:16:50.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.229+0000 7f339ffff640 1 -- 172.21.15.27:0/2455734732 learned_addr learned my addr 172.21.15.27:0/2455734732 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:50.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.229+0000 7f33a574e640 1 --2- 172.21.15.27:0/2455734732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33a00c50a0 0x7f33a00afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.229+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1388635196 0 0) 0x7f3394001110 con 0x7f33a015ee30 2023-12-19T18:16:50.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.229+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3388003520 con 0x7f33a015ee30 2023-12-19T18:16:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3028612517 0 0) 0x7f3388003520 con 0x7f33a015ee30 2023-12-19T18:16:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33a00c50a0 msgr2=0x7f33a00afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339f7fe640 1 --2- 172.21.15.27:0/2455734732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33a00c50a0 0x7f33a00afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 >> v1:172.21.15.192:6789/0 conn(0x7f33a01553a0 legacy=0x7f33a015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a00cb630 con 0x7f33a015ee30 2023-12-19T18:16:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f3394001800 con 0x7f33a015ee30 2023-12-19T18:16:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 >> v1:172.21.15.27:6789/0 conn(0x7f33a015ee30 legacy=0x7f33a00c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339f7fe640 1 --2- 172.21.15.27:0/2455734732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880039a0 0x7f3388003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339f7fe640 1 --2- 172.21.15.27:0/2455734732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33880047c0 0x7f3388004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339f7fe640 1 --2- 172.21.15.27:0/2455734732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33a00c50a0 0x7f3388005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00cbcf0 con 0x7f33880047c0 2023-12-19T18:16:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f338800bb40 con 0x7f33880039a0 2023-12-19T18:16:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f338800bd00 con 0x7f33a00c50a0 2023-12-19T18:16:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.230+0000 7f339ffff640 1 --2- 172.21.15.27:0/2455734732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33880047c0 0x7f3388004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.231+0000 7f33a4f4d640 1 --2- 172.21.15.27:0/2455734732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880039a0 0x7f3388003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.231+0000 7f33a574e640 1 --2- 172.21.15.27:0/2455734732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33a00c50a0 0x7f3388005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.231+0000 7f339ffff640 1 -- 172.21.15.27:0/2455734732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33a00c50a0 msgr2=0x7f3388005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.231+0000 7f339ffff640 1 --2- 172.21.15.27:0/2455734732 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33a00c50a0 0x7f3388005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.231+0000 7f339ffff640 1 -- 172.21.15.27:0/2455734732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880039a0 msgr2=0x7f3388003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.231+0000 7f339ffff640 1 --2- 172.21.15.27:0/2455734732 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880039a0 0x7f3388003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.231+0000 7f339ffff640 1 -- 172.21.15.27:0/2455734732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3388003520 con 0x7f33880047c0 2023-12-19T18:16:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.232+0000 7f339ffff640 1 --2- 172.21.15.27:0/2455734732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33880047c0 0x7f3388004c40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3394007fc0 tx=0x7f33880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aecb9ed8b9277491 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.233+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3394005ad0 con 0x7f33880047c0 2023-12-19T18:16:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.233+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3394005c50 con 0x7f33880047c0 2023-12-19T18:16:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.233+0000 7f339f7fe640 1 -- 172.21.15.27:0/2455734732 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f339402c7e0 con 0x7f33880047c0 2023-12-19T18:16:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.233+0000 7f33a5f4f640 1 -- 172.21.15.27:0/2455734732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33880047c0 msgr2=0x7f3388004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.233+0000 7f33a5f4f640 1 --2- 172.21.15.27:0/2455734732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33880047c0 0x7f3388004c40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3394007fc0 tx=0x7f33880057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.234+0000 7f33a574e640 1 -- 172.21.15.27:0/2455734732 reap_dead start 2023-12-19T18:16:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.234+0000 7f33a5f4f640 1 -- 172.21.15.27:0/2455734732 shutdown_connections 2023-12-19T18:16:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.234+0000 7f33a5f4f640 1 -- 172.21.15.27:0/2455734732 >> 172.21.15.27:0/2455734732 conn(0x7f33a0069c70 msgr2=0x7f33a006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.234+0000 7f33a5f4f640 1 -- 172.21.15.27:0/2455734732 shutdown_connections 2023-12-19T18:16:50.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.234+0000 7f33a5f4f640 1 -- 172.21.15.27:0/2455734732 wait complete. 2023-12-19T18:16:50.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.236+0000 7f33a5f4f640 1 Processor -- start 2023-12-19T18:16:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.237+0000 7f33a5f4f640 1 -- start start 2023-12-19T18:16:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.238+0000 7f33a5f4f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880047c0 0x7f33a0150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.238+0000 7f33a5f4f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33a01f3260 0x7f33a015d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.238+0000 7f33a5f4f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33a00adab0 0x7f33a006b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.238+0000 7f33a5f4f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a015e4d0 con 0x7f33a00adab0 2023-12-19T18:16:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.238+0000 7f33a5f4f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33a00cc510 con 0x7f33880047c0 2023-12-19T18:16:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.238+0000 7f33a5f4f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33a015dce0 con 0x7f33a01f3260 2023-12-19T18:16:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.238+0000 7f33a574e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33a00adab0 0x7f33a006b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.238+0000 7f33a574e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33a00adab0 0x7f33a006b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55412/0 (socket says 172.21.15.27:55412) 2023-12-19T18:16:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.238+0000 7f33a574e640 1 -- 172.21.15.27:0/4052694491 learned_addr learned my addr 172.21.15.27:0/4052694491 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.239+0000 7f33a4f4d640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880047c0 0x7f33a0150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.239+0000 7f339ffff640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33a01f3260 0x7f33a015d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.239+0000 7f33a574e640 1 -- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33a01f3260 msgr2=0x7f33a015d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.239+0000 7f33a574e640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33a01f3260 0x7f33a015d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.239+0000 7f33a574e640 1 -- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880047c0 msgr2=0x7f33a0150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.239+0000 7f33a574e640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880047c0 0x7f33a0150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.239+0000 7f33a574e640 1 -- 172.21.15.27:0/4052694491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a01f36e0 con 0x7f33a00adab0 2023-12-19T18:16:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.239+0000 7f33a574e640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33a00adab0 0x7f33a006b3f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f33a0071090 tx=0x7f3398011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.240+0000 7f339d7fa640 1 -- 172.21.15.27:0/4052694491 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3398004500 con 0x7f33a00adab0 2023-12-19T18:16:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.240+0000 7f339d7fa640 1 -- 172.21.15.27:0/4052694491 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3398003610 con 0x7f33a00adab0 2023-12-19T18:16:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.240+0000 7f339d7fa640 1 -- 172.21.15.27:0/4052694491 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33980039e0 con 0x7f33a00adab0 2023-12-19T18:16:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.240+0000 7f33a5f4f640 1 -- 172.21.15.27:0/4052694491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33a015df10 con 0x7f33a00adab0 2023-12-19T18:16:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.240+0000 7f33a5f4f640 1 -- 172.21.15.27:0/4052694491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33a00cc720 con 0x7f33a00adab0 2023-12-19T18:16:50.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.241+0000 7f33a5f4f640 1 -- 172.21.15.27:0/4052694491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33a00cb8f0 con 0x7f33a00adab0 2023-12-19T18:16:50.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.242+0000 7f339d7fa640 1 -- 172.21.15.27:0/4052694491 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3398003b40 con 0x7f33a00adab0 2023-12-19T18:16:50.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.243+0000 7f339d7fa640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f338c06a430 0x7f338c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.243+0000 7f339d7fa640 1 -- 172.21.15.27:0/4052694491 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f33980889f0 con 0x7f33a00adab0 2023-12-19T18:16:50.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.244+0000 7f33a4f4d640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f338c06a430 0x7f338c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.245+0000 7f33a4f4d640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f338c06a430 0x7f338c06c8f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3390000f80 tx=0x7f3390009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.247+0000 7f339d7fa640 1 -- 172.21.15.27:0/4052694491 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f33a00cb8f0 con 0x7f33a00adab0 2023-12-19T18:16:50.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.445+0000 7f33a5f4f640 1 -- 172.21.15.27:0/4052694491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33a00cb8f0 con 0x7f33a00adab0 2023-12-19T18:16:50.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.448+0000 7f339d7fa640 1 -- 172.21.15.27:0/4052694491 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16291 (secure 0 0 0) 0x7f33a00cb8f0 con 0x7f33a00adab0 2023-12-19T18:16:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:50.448 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:16:45.666993+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:23.925853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:24.827898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]},{"pgid":"5.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:50.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.450+0000 7f33a5f4f640 1 -- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f338c06a430 msgr2=0x7f338c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.450+0000 7f33a5f4f640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f338c06a430 0x7f338c06c8f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3390000f80 tx=0x7f3390009040 comp rx=0 tx=0).stop 2023-12-19T18:16:50.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.450+0000 7f33a5f4f640 1 -- 172.21.15.27:0/4052694491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33a00adab0 msgr2=0x7f33a006b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.450+0000 7f33a5f4f640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33a00adab0 0x7f33a006b3f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f33a0071090 tx=0x7f3398011920 comp rx=0 tx=0).stop 2023-12-19T18:16:50.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.450+0000 7f33a5f4f640 1 -- 172.21.15.27:0/4052694491 shutdown_connections 2023-12-19T18:16:50.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.450+0000 7f33a5f4f640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f338c06a430 0x7f338c06c8f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.450+0000 7f33a5f4f640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33a00adab0 0x7f33a006b3f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.450+0000 7f33a5f4f640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33a01f3260 0x7f33a015d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.450+0000 7f33a5f4f640 1 --2- 172.21.15.27:0/4052694491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33880047c0 0x7f33a0150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.450+0000 7f33a5f4f640 1 -- 172.21.15.27:0/4052694491 >> 172.21.15.27:0/4052694491 conn(0x7f33a0069c70 msgr2=0x7f33a0158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:50.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.451+0000 7f33a5f4f640 1 -- 172.21.15.27:0/4052694491 shutdown_connections 2023-12-19T18:16:50.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.451+0000 7f33a5f4f640 1 -- 172.21.15.27:0/4052694491 wait complete. 2023-12-19T18:16:50.471 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:16:23.925853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:16:24.827898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:16:50.471 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:16:50.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.696+0000 7fccabfff640 1 Processor -- start 2023-12-19T18:16:50.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.696+0000 7fccabfff640 1 -- start start 2023-12-19T18:16:50.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.698+0000 7fccabfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fccac151c90 0x7fccac06b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.698+0000 7fccabfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccac15e9d0 0x7fccac0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.698+0000 7fccabfff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccac15b3e0 con 0x7fccac152070 2023-12-19T18:16:50.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.698+0000 7fccabfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccac15eed0 con 0x7fccac151c90 2023-12-19T18:16:50.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.698+0000 7fccabfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac15f1b0 con 0x7fccac15e9d0 2023-12-19T18:16:50.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.698+0000 7fccab7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fccac152070 0x7fccac15b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42262/0 (socket says 172.21.15.27:42262) 2023-12-19T18:16:50.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.698+0000 7fccab7fe640 1 -- 172.21.15.27:0/2166353147 learned_addr learned my addr 172.21.15.27:0/2166353147 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:50.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.698+0000 7fccaaffd640 1 --2- 172.21.15.27:0/2166353147 >> v2:172.21.15.192:3300/0 conn(0x7fccac151c90 0x7fccac06b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.699+0000 7fcca27fc640 1 --2- 172.21.15.27:0/2166353147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccac15e9d0 0x7fccac0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.699+0000 7fccaa7fc640 1 -- 172.21.15.27:0/2166353147 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108356869 0 0) 0x7fccac15b3e0 con 0x7fccac152070 2023-12-19T18:16:50.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.699+0000 7fccaa7fc640 1 -- 172.21.15.27:0/2166353147 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc98003520 con 0x7fccac152070 2023-12-19T18:16:50.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.699+0000 7fccaaffd640 1 -- 172.21.15.27:0/2166353147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccac15e9d0 msgr2=0x7fccac0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:50.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.699+0000 7fccaaffd640 1 --2- 172.21.15.27:0/2166353147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccac15e9d0 0x7fccac0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.699+0000 7fccaaffd640 1 -- 172.21.15.27:0/2166353147 >> v1:172.21.15.27:6789/0 conn(0x7fccac152070 legacy=0x7fccac15b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.699+0000 7fccaaffd640 1 -- 172.21.15.27:0/2166353147 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac15f490 con 0x7fccac151c90 2023-12-19T18:16:50.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.700+0000 7fccaaffd640 1 --2- 172.21.15.27:0/2166353147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccac151c90 0x7fccac06b4d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009d90 tx=0x7fcc9c02be10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e9d1e2b1fdc10aa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.701+0000 7fccaa7fc640 1 -- 172.21.15.27:0/2166353147 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c02cbc0 con 0x7fccac151c90 2023-12-19T18:16:50.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.701+0000 7fccaa7fc640 1 -- 172.21.15.27:0/2166353147 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c02cd20 con 0x7fccac151c90 2023-12-19T18:16:50.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.701+0000 7fccaa7fc640 1 -- 172.21.15.27:0/2166353147 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c034640 con 0x7fccac151c90 2023-12-19T18:16:50.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.702+0000 7fccabfff640 1 -- 172.21.15.27:0/2166353147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccac151c90 msgr2=0x7fccac06b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.702+0000 7fccabfff640 1 --2- 172.21.15.27:0/2166353147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccac151c90 0x7fccac06b4d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009d90 tx=0x7fcc9c02be10 comp rx=0 tx=0).stop 2023-12-19T18:16:50.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.702+0000 7fccabfff640 1 -- 172.21.15.27:0/2166353147 shutdown_connections 2023-12-19T18:16:50.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.702+0000 7fccabfff640 1 --2- 172.21.15.27:0/2166353147 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccac15e9d0 0x7fccac0cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.702+0000 7fccabfff640 1 --2- 172.21.15.27:0/2166353147 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccac151c90 0x7fccac06b4d0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.702+0000 7fccabfff640 1 -- 172.21.15.27:0/2166353147 >> 172.21.15.27:0/2166353147 conn(0x7fccac06a060 msgr2=0x7fccac06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:50.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.702+0000 7fccabfff640 1 -- 172.21.15.27:0/2166353147 shutdown_connections 2023-12-19T18:16:50.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.702+0000 7fccabfff640 1 -- 172.21.15.27:0/2166353147 wait complete. 2023-12-19T18:16:50.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.704+0000 7fccabfff640 1 Processor -- start 2023-12-19T18:16:50.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.704+0000 7fccabfff640 1 -- start start 2023-12-19T18:16:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.705+0000 7fccabfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccac151c90 0x7fccac1f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.705+0000 7fccabfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccac152070 0x7fccac153250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.705+0000 7fccabfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccac15e9d0 0x7fccac153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.705+0000 7fccabfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac1e6b30 con 0x7fccac151c90 2023-12-19T18:16:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.705+0000 7fccabfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac1e5c80 con 0x7fccac15e9d0 2023-12-19T18:16:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.705+0000 7fccabfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac1e5f20 con 0x7fccac152070 2023-12-19T18:16:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.705+0000 7fccaaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccac151c90 0x7fccac1f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.705+0000 7fccaaffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccac151c90 0x7fccac1f52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55424/0 (socket says 172.21.15.27:55424) 2023-12-19T18:16:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.705+0000 7fccaaffd640 1 -- 172.21.15.27:0/629383118 learned_addr learned my addr 172.21.15.27:0/629383118 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.705+0000 7fccab7fe640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccac15e9d0 0x7fccac153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.705+0000 7fccaaffd640 1 -- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccac152070 msgr2=0x7fccac153250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.705+0000 7fccaaffd640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccac152070 0x7fccac153250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.706+0000 7fccaaffd640 1 -- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccac15e9d0 msgr2=0x7fccac153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.706+0000 7fccaaffd640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccac15e9d0 0x7fccac153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.706+0000 7fccaaffd640 1 -- 172.21.15.27:0/629383118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9c009a70 con 0x7fccac151c90 2023-12-19T18:16:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.706+0000 7fccab7fe640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccac15e9d0 0x7fccac153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.706+0000 7fccaaffd640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccac151c90 0x7fccac1f52e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009eb0 tx=0x7fcc9c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.706+0000 7fcca3fff640 1 -- 172.21.15.27:0/629383118 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c03e610 con 0x7fccac151c90 2023-12-19T18:16:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.706+0000 7fcca3fff640 1 -- 172.21.15.27:0/629383118 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c02ce90 con 0x7fccac151c90 2023-12-19T18:16:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.706+0000 7fcca3fff640 1 -- 172.21.15.27:0/629383118 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c03e9b0 con 0x7fccac151c90 2023-12-19T18:16:50.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.706+0000 7fccabfff640 1 -- 172.21.15.27:0/629383118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccac1e6200 con 0x7fccac151c90 2023-12-19T18:16:50.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.707+0000 7fccabfff640 1 -- 172.21.15.27:0/629383118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccac1e67c0 con 0x7fccac151c90 2023-12-19T18:16:50.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.708+0000 7fccabfff640 1 -- 172.21.15.27:0/629383118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccac1e6980 con 0x7fccac151c90 2023-12-19T18:16:50.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.708+0000 7fcca3fff640 1 -- 172.21.15.27:0/629383118 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcc9c03b050 con 0x7fccac151c90 2023-12-19T18:16:50.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.709+0000 7fcca3fff640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcc8c06a430 0x7fcc8c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:50.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.710+0000 7fcca3fff640 1 -- 172.21.15.27:0/629383118 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fcc9c0ab120 con 0x7fccac151c90 2023-12-19T18:16:50.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.710+0000 7fcca27fc640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcc8c06a430 0x7fcc8c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:50.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.711+0000 7fcca27fc640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcc8c06a430 0x7fcc8c06c8f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcc94006a90 tx=0x7fcc94006a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:50.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.714+0000 7fcca3fff640 1 -- 172.21.15.27:0/629383118 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcc9c0767a0 con 0x7fccac151c90 2023-12-19T18:16:50.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.910+0000 7fccabfff640 1 -- 172.21.15.27:0/629383118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fccac15e750 con 0x7fccac151c90 2023-12-19T18:16:50.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.911+0000 7fcca3fff640 1 -- 172.21.15.27:0/629383118 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcc9c009450 con 0x7fccac151c90 2023-12-19T18:16:50.911 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:16:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.913+0000 7fccabfff640 1 -- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcc8c06a430 msgr2=0x7fcc8c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.913+0000 7fccabfff640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcc8c06a430 0x7fcc8c06c8f0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fcc94006a90 tx=0x7fcc94006a20 comp rx=0 tx=0).stop 2023-12-19T18:16:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.914+0000 7fccabfff640 1 -- 172.21.15.27:0/629383118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccac151c90 msgr2=0x7fccac1f52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:50.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.914+0000 7fccabfff640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccac151c90 0x7fccac1f52e0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c009eb0 tx=0x7fcc9c004a40 comp rx=0 tx=0).stop 2023-12-19T18:16:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.914+0000 7fccabfff640 1 -- 172.21.15.27:0/629383118 shutdown_connections 2023-12-19T18:16:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.914+0000 7fccabfff640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcc8c06a430 0x7fcc8c06c8f0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.914+0000 7fccabfff640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccac15e9d0 0x7fccac153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.914+0000 7fccabfff640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccac152070 0x7fccac153250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.914+0000 7fccabfff640 1 --2- 172.21.15.27:0/629383118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccac151c90 0x7fccac1f52e0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.914+0000 7fccabfff640 1 -- 172.21.15.27:0/629383118 >> 172.21.15.27:0/629383118 conn(0x7fccac06a060 msgr2=0x7fccac15dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.914+0000 7fccabfff640 1 -- 172.21.15.27:0/629383118 shutdown_connections 2023-12-19T18:16:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:50.914+0000 7fccabfff640 1 -- 172.21.15.27:0/629383118 wait complete. 2023-12-19T18:16:50.932 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:16:51.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.171+0000 7f9585527640 1 Processor -- start 2023-12-19T18:16:51.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.172+0000 7f9585527640 1 -- start start 2023-12-19T18:16:51.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.173+0000 7f9585527640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9580151c90 0x7f958006b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.173+0000 7f9585527640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f958015e9d0 0x7f95800cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.173+0000 7f9585527640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9580152070 0x7f95800cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.173+0000 7f9585527640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f958015b3e0 con 0x7f9580152070 2023-12-19T18:16:51.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.173+0000 7f9585527640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f958015eed0 con 0x7f958015e9d0 2023-12-19T18:16:51.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.173+0000 7f9585527640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f958015f1b0 con 0x7f9580151c90 2023-12-19T18:16:51.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.174+0000 7f9584d26640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9580152070 0x7f95800cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.174+0000 7f9584d26640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9580152070 0x7f95800cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55430/0 (socket says 172.21.15.27:55430) 2023-12-19T18:16:51.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.174+0000 7f9584d26640 1 -- 172.21.15.27:0/539239089 learned_addr learned my addr 172.21.15.27:0/539239089 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.174+0000 7f9577fff640 1 --2- 172.21.15.27:0/539239089 >> v2:172.21.15.192:3300/0 conn(0x7f958015e9d0 0x7f95800cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.174+0000 7f9584d26640 1 -- 172.21.15.27:0/539239089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9580151c90 msgr2=0x7f958006b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.174+0000 7f957ffff640 1 --2- 172.21.15.27:0/539239089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9580151c90 0x7f958006b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.174+0000 7f9584d26640 1 --2- 172.21.15.27:0/539239089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9580151c90 0x7f958006b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.174+0000 7f9584d26640 1 -- 172.21.15.27:0/539239089 >> v2:172.21.15.192:3300/0 conn(0x7f958015e9d0 msgr2=0x7f95800cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.174+0000 7f9584d26640 1 --2- 172.21.15.27:0/539239089 >> v2:172.21.15.192:3300/0 conn(0x7f958015e9d0 0x7f95800cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.174+0000 7f9584d26640 1 -- 172.21.15.27:0/539239089 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f958015f490 con 0x7f9580152070 2023-12-19T18:16:51.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.174+0000 7f9577fff640 1 --2- 172.21.15.27:0/539239089 >> v2:172.21.15.192:3300/0 conn(0x7f958015e9d0 0x7f95800cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:51.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.175+0000 7f9584d26640 1 --2- 172.21.15.27:0/539239089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9580152070 0x7f95800cf380 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f95800cfa80 tx=0x7f957802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44211e442410d823 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:51.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.176+0000 7f957f7fe640 1 -- 172.21.15.27:0/539239089 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957802c9f0 con 0x7f9580152070 2023-12-19T18:16:51.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.176+0000 7f957f7fe640 1 -- 172.21.15.27:0/539239089 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957802cb50 con 0x7f9580152070 2023-12-19T18:16:51.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.176+0000 7f957f7fe640 1 -- 172.21.15.27:0/539239089 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95780343f0 con 0x7f9580152070 2023-12-19T18:16:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.176+0000 7f9585527640 1 -- 172.21.15.27:0/539239089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9580152070 msgr2=0x7f95800cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.176+0000 7f9585527640 1 --2- 172.21.15.27:0/539239089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9580152070 0x7f95800cf380 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f95800cfa80 tx=0x7f957802be50 comp rx=0 tx=0).stop 2023-12-19T18:16:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.177+0000 7f9585527640 1 -- 172.21.15.27:0/539239089 shutdown_connections 2023-12-19T18:16:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.177+0000 7f9585527640 1 --2- 172.21.15.27:0/539239089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9580152070 0x7f95800cf380 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.177+0000 7f9585527640 1 --2- 172.21.15.27:0/539239089 >> v2:172.21.15.192:3300/0 conn(0x7f958015e9d0 0x7f95800cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.177+0000 7f9585527640 1 --2- 172.21.15.27:0/539239089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9580151c90 0x7f958006b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.177+0000 7f9585527640 1 -- 172.21.15.27:0/539239089 >> 172.21.15.27:0/539239089 conn(0x7f958006a060 msgr2=0x7f958006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.177+0000 7f9585527640 1 -- 172.21.15.27:0/539239089 shutdown_connections 2023-12-19T18:16:51.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.177+0000 7f9585527640 1 -- 172.21.15.27:0/539239089 wait complete. 2023-12-19T18:16:51.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.178+0000 7f9585527640 1 Processor -- start 2023-12-19T18:16:51.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.179+0000 7f9585527640 1 -- start start 2023-12-19T18:16:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9585527640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9580151c90 0x7f95801e6700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9585527640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9580152070 0x7f95801e72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9585527640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958015e9d0 0x7f9580147e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9585527640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95801483a0 con 0x7f958015e9d0 2023-12-19T18:16:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9585527640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f958014c570 con 0x7f9580151c90 2023-12-19T18:16:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9585527640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95801e6d90 con 0x7f9580152070 2023-12-19T18:16:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9584d26640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958015e9d0 0x7f9580147e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9584d26640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958015e9d0 0x7f9580147e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55434/0 (socket says 172.21.15.27:55434) 2023-12-19T18:16:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9584d26640 1 -- 172.21.15.27:0/2828633758 learned_addr learned my addr 172.21.15.27:0/2828633758 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9577fff640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9580152070 0x7f95801e72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f957ffff640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9580151c90 0x7f95801e6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9584d26640 1 -- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9580152070 msgr2=0x7f95801e72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9584d26640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9580152070 0x7f95801e72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9584d26640 1 -- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9580151c90 msgr2=0x7f95801e6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9584d26640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9580151c90 0x7f95801e6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9584d26640 1 -- 172.21.15.27:0/2828633758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9578009b30 con 0x7f958015e9d0 2023-12-19T18:16:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f9577fff640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9580152070 0x7f95801e72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.180+0000 7f957ffff640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9580151c90 0x7f95801e6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.181+0000 7f9584d26640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958015e9d0 0x7f9580147e60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f957802c460 tx=0x7f9578004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.181+0000 7f957d7fa640 1 -- 172.21.15.27:0/2828633758 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9578009510 con 0x7f958015e9d0 2023-12-19T18:16:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.181+0000 7f957d7fa640 1 -- 172.21.15.27:0/2828633758 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f957803e450 con 0x7f958015e9d0 2023-12-19T18:16:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.181+0000 7f957d7fa640 1 -- 172.21.15.27:0/2828633758 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f957803e7e0 con 0x7f958015e9d0 2023-12-19T18:16:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.181+0000 7f9585527640 1 -- 172.21.15.27:0/2828633758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95801f5a20 con 0x7f958015e9d0 2023-12-19T18:16:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.181+0000 7f9585527640 1 -- 172.21.15.27:0/2828633758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95801f5fe0 con 0x7f958015e9d0 2023-12-19T18:16:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.183+0000 7f9585527640 1 -- 172.21.15.27:0/2828633758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f958004e840 con 0x7f958015e9d0 2023-12-19T18:16:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.183+0000 7f957d7fa640 1 -- 172.21.15.27:0/2828633758 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f957803e940 con 0x7f958015e9d0 2023-12-19T18:16:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.184+0000 7f957d7fa640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f956006a430 0x7f956006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.185+0000 7f957d7fa640 1 -- 172.21.15.27:0/2828633758 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f95780abbc0 con 0x7f958015e9d0 2023-12-19T18:16:51.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.185+0000 7f957ffff640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f956006a430 0x7f956006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.186+0000 7f957ffff640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f956006a430 0x7f956006c8f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9570006fd0 tx=0x7f9570008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:51.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.189+0000 7f957d7fa640 1 -- 172.21.15.27:0/2828633758 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9578077090 con 0x7f958015e9d0 2023-12-19T18:16:51.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.383+0000 7f9585527640 1 -- 172.21.15.27:0/2828633758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f958015f740 con 0x7f958015e9d0 2023-12-19T18:16:51.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.384+0000 7f957d7fa640 1 -- 172.21.15.27:0/2828633758 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9578033e80 con 0x7f958015e9d0 2023-12-19T18:16:51.385 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:16:51.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.387+0000 7f9585527640 1 -- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f956006a430 msgr2=0x7f956006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.387+0000 7f9585527640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f956006a430 0x7f956006c8f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9570006fd0 tx=0x7f9570008040 comp rx=0 tx=0).stop 2023-12-19T18:16:51.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.387+0000 7f9585527640 1 -- 172.21.15.27:0/2828633758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958015e9d0 msgr2=0x7f9580147e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.387+0000 7f9585527640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958015e9d0 0x7f9580147e60 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f957802c460 tx=0x7f9578004a40 comp rx=0 tx=0).stop 2023-12-19T18:16:51.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.388+0000 7f9585527640 1 -- 172.21.15.27:0/2828633758 shutdown_connections 2023-12-19T18:16:51.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.388+0000 7f9585527640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f956006a430 0x7f956006c8f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.388+0000 7f9585527640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f958015e9d0 0x7f9580147e60 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.388+0000 7f9585527640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9580152070 0x7f95801e72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.388+0000 7f9585527640 1 --2- 172.21.15.27:0/2828633758 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9580151c90 0x7f95801e6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.388+0000 7f9585527640 1 -- 172.21.15.27:0/2828633758 >> 172.21.15.27:0/2828633758 conn(0x7f958006a060 msgr2=0x7f95800cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:51.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.388+0000 7f9585527640 1 -- 172.21.15.27:0/2828633758 shutdown_connections 2023-12-19T18:16:51.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.388+0000 7f9585527640 1 -- 172.21.15.27:0/2828633758 wait complete. 2023-12-19T18:16:51.409 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:16:51.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.645+0000 7fb51655c640 1 Processor -- start 2023-12-19T18:16:51.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.645+0000 7fb51655c640 1 -- start start 2023-12-19T18:16:51.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.646+0000 7fb51655c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb5100ccfa0 0x7fb5100c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.647+0000 7fb51655c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5100ce0b0 con 0x7fb5100ac9d0 2023-12-19T18:16:51.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.647+0000 7fb51655c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5100ce920 con 0x7fb5100c9750 2023-12-19T18:16:51.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.647+0000 7fb51655c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5100ca070 con 0x7fb5100ccfa0 2023-12-19T18:16:51.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.647+0000 7fb507fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb5100ccfa0 0x7fb5100c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.647+0000 7fb514af9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb5100ac9d0 0x7fb5100ace20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:42270/0 (socket says 172.21.15.27:42270) 2023-12-19T18:16:51.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.647+0000 7fb514af9640 1 -- 172.21.15.27:0/4264673427 learned_addr learned my addr 172.21.15.27:0/4264673427 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:51.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.647+0000 7fb507fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb5100ccfa0 0x7fb5100c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55446/0 (socket says 172.21.15.27:55446) 2023-12-19T18:16:51.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.647+0000 7fb507fff640 1 -- 172.21.15.27:0/4264673427 >> v1:172.21.15.192:6789/0 conn(0x7fb5100c9750 legacy=0x7fb51015e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:51.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.647+0000 7fb507fff640 1 -- 172.21.15.27:0/4264673427 >> v1:172.21.15.27:6789/0 conn(0x7fb5100ac9d0 legacy=0x7fb5100ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.647+0000 7fb507fff640 1 -- 172.21.15.27:0/4264673427 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5100cd4a0 con 0x7fb5100ccfa0 2023-12-19T18:16:51.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.649+0000 7fb507fff640 1 --2- 172.21.15.27:0/4264673427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5100ccfa0 0x7fb5100c9b30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80097a0 tx=0x7fb4f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=924928d7b90eb809 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:51.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.649+0000 7fb50f7fe640 1 -- 172.21.15.27:0/4264673427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8031830 con 0x7fb5100ccfa0 2023-12-19T18:16:51.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.649+0000 7fb50f7fe640 1 -- 172.21.15.27:0/4264673427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f8031990 con 0x7fb5100ccfa0 2023-12-19T18:16:51.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.649+0000 7fb50f7fe640 1 -- 172.21.15.27:0/4264673427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f8031d20 con 0x7fb5100ccfa0 2023-12-19T18:16:51.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.650+0000 7fb51655c640 1 -- 172.21.15.27:0/4264673427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5100ccfa0 msgr2=0x7fb5100c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.650+0000 7fb51655c640 1 --2- 172.21.15.27:0/4264673427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5100ccfa0 0x7fb5100c9b30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fb4f80097a0 tx=0x7fb4f802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.650+0000 7fb51655c640 1 -- 172.21.15.27:0/4264673427 shutdown_connections 2023-12-19T18:16:51.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.650+0000 7fb51655c640 1 --2- 172.21.15.27:0/4264673427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5100ccfa0 0x7fb5100c9b30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.650+0000 7fb51655c640 1 -- 172.21.15.27:0/4264673427 >> 172.21.15.27:0/4264673427 conn(0x7fb5100cebd0 msgr2=0x7fb5100cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:51.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.650+0000 7fb51655c640 1 -- 172.21.15.27:0/4264673427 shutdown_connections 2023-12-19T18:16:51.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.650+0000 7fb51655c640 1 -- 172.21.15.27:0/4264673427 wait complete. 2023-12-19T18:16:51.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.652+0000 7fb51655c640 1 Processor -- start 2023-12-19T18:16:51.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.652+0000 7fb51655c640 1 -- start start 2023-12-19T18:16:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.653+0000 7fb51655c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5100ac9d0 0x7fb5100b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.653+0000 7fb51655c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5100c9750 0x7fb51006c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.653+0000 7fb51655c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5100ccfa0 0x7fb51006cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.653+0000 7fb51655c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5101f30f0 con 0x7fb5100ac9d0 2023-12-19T18:16:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.653+0000 7fb51655c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100c26e0 con 0x7fb5100c9750 2023-12-19T18:16:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.653+0000 7fb51655c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5100c2ca0 con 0x7fb5100ccfa0 2023-12-19T18:16:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.653+0000 7fb50ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5100ac9d0 0x7fb5100b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.653+0000 7fb50ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5100ac9d0 0x7fb5100b3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55462/0 (socket says 172.21.15.27:55462) 2023-12-19T18:16:51.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.653+0000 7fb50ffff640 1 -- 172.21.15.27:0/3383777660 learned_addr learned my addr 172.21.15.27:0/3383777660 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:51.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.654+0000 7fb507fff640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5100c9750 0x7fb51006c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.654+0000 7fb50ffff640 1 -- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5100ccfa0 msgr2=0x7fb51006cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.654+0000 7fb50ffff640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5100ccfa0 0x7fb51006cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.654+0000 7fb50ffff640 1 -- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5100c9750 msgr2=0x7fb51006c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.654+0000 7fb50ffff640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5100c9750 0x7fb51006c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.654+0000 7fb50ffff640 1 -- 172.21.15.27:0/3383777660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f8009400 con 0x7fb5100ac9d0 2023-12-19T18:16:51.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.654+0000 7fb507fff640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5100c9750 0x7fb51006c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:51.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.654+0000 7fb50ffff640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5100ac9d0 0x7fb5100b3f90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb50000c430 tx=0x7fb50000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:51.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.654+0000 7fb50d7fa640 1 -- 172.21.15.27:0/3383777660 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500011790 con 0x7fb5100ac9d0 2023-12-19T18:16:51.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.654+0000 7fb50d7fa640 1 -- 172.21.15.27:0/3383777660 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5000118f0 con 0x7fb5100ac9d0 2023-12-19T18:16:51.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.654+0000 7fb50d7fa640 1 -- 172.21.15.27:0/3383777660 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb500011ce0 con 0x7fb5100ac9d0 2023-12-19T18:16:51.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.654+0000 7fb51655c640 1 -- 172.21.15.27:0/3383777660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb510152d50 con 0x7fb5100ac9d0 2023-12-19T18:16:51.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.655+0000 7fb51655c640 1 -- 172.21.15.27:0/3383777660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5100c2930 con 0x7fb5100ac9d0 2023-12-19T18:16:51.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.656+0000 7fb51655c640 1 -- 172.21.15.27:0/3383777660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5100c2ac0 con 0x7fb5100ac9d0 2023-12-19T18:16:51.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.657+0000 7fb50d7fa640 1 -- 172.21.15.27:0/3383777660 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb50001c460 con 0x7fb5100ac9d0 2023-12-19T18:16:51.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.658+0000 7fb50d7fa640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb4f006a430 0x7fb4f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:51.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.658+0000 7fb50d7fa640 1 -- 172.21.15.27:0/3383777660 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb500088860 con 0x7fb5100ac9d0 2023-12-19T18:16:51.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.659+0000 7fb507fff640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb4f006a430 0x7fb4f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:51.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.660+0000 7fb507fff640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb4f006a430 0x7fb4f006c8f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8036000 tx=0x7fb4f8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:51.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.663+0000 7fb50d7fa640 1 -- 172.21.15.27:0/3383777660 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb500053e60 con 0x7fb5100ac9d0 2023-12-19T18:16:51.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.860+0000 7fb51655c640 1 -- 172.21.15.27:0/3383777660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb510068e90 con 0x7fb5100ac9d0 2023-12-19T18:16:51.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.861+0000 7fb50d7fa640 1 -- 172.21.15.27:0/3383777660 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb500057480 con 0x7fb5100ac9d0 2023-12-19T18:16:51.861 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:16:51.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.863+0000 7fb51655c640 1 -- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb4f006a430 msgr2=0x7fb4f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.863+0000 7fb51655c640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb4f006a430 0x7fb4f006c8f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb4f8036000 tx=0x7fb4f8008030 comp rx=0 tx=0).stop 2023-12-19T18:16:51.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.863+0000 7fb51655c640 1 -- 172.21.15.27:0/3383777660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5100ac9d0 msgr2=0x7fb5100b3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:51.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.863+0000 7fb51655c640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5100ac9d0 0x7fb5100b3f90 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb50000c430 tx=0x7fb50000c8f0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.864+0000 7fb51655c640 1 -- 172.21.15.27:0/3383777660 shutdown_connections 2023-12-19T18:16:51.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.864+0000 7fb51655c640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb4f006a430 0x7fb4f006c8f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.864+0000 7fb51655c640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5100ccfa0 0x7fb51006cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.864+0000 7fb51655c640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5100c9750 0x7fb51006c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.864+0000 7fb51655c640 1 --2- 172.21.15.27:0/3383777660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5100ac9d0 0x7fb5100b3f90 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:51.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.864+0000 7fb51655c640 1 -- 172.21.15.27:0/3383777660 >> 172.21.15.27:0/3383777660 conn(0x7fb5100cebd0 msgr2=0x7fb510070b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:51.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.864+0000 7fb51655c640 1 -- 172.21.15.27:0/3383777660 shutdown_connections 2023-12-19T18:16:51.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:51.864+0000 7fb51655c640 1 -- 172.21.15.27:0/3383777660 wait complete. 2023-12-19T18:16:51.885 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:16:52.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.122+0000 7f6dca12c640 1 Processor -- start 2023-12-19T18:16:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.123+0000 7f6dca12c640 1 -- start start 2023-12-19T18:16:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.124+0000 7f6dca12c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6dc41540e0 0x7f6dc4160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.124+0000 7f6dca12c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6dc406c6e0 0x7f6dc415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.125+0000 7f6dca12c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dc415d600 con 0x7f6dc415e2e0 2023-12-19T18:16:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.125+0000 7f6dca12c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc4161130 con 0x7f6dc41540e0 2023-12-19T18:16:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.125+0000 7f6dca12c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dc41613d0 con 0x7f6dc406c6e0 2023-12-19T18:16:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.125+0000 7f6dc912a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6dc41540e0 0x7f6dc4160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.125+0000 7f6dc912a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6dc41540e0 0x7f6dc4160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41158/0 (socket says 172.21.15.27:41158) 2023-12-19T18:16:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.125+0000 7f6dc912a640 1 -- 172.21.15.27:0/3147759335 learned_addr learned my addr 172.21.15.27:0/3147759335 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.125+0000 7f6dc992b640 1 --1- 172.21.15.27:0/3147759335 >> v1:172.21.15.27:6789/0 conn(0x7f6dc415e2e0 0x7f6dc406b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55162/0 (socket says 172.21.15.27:55162) 2023-12-19T18:16:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.125+0000 7f6dc8929640 1 --2- 172.21.15.27:0/3147759335 >> v2:172.21.15.192:3300/0 conn(0x7f6dc406c6e0 0x7f6dc415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.126+0000 7f6dc912a640 1 -- 172.21.15.27:0/3147759335 >> v2:172.21.15.192:3300/0 conn(0x7f6dc406c6e0 msgr2=0x7f6dc415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.126+0000 7f6dc912a640 1 --2- 172.21.15.27:0/3147759335 >> v2:172.21.15.192:3300/0 conn(0x7f6dc406c6e0 0x7f6dc415dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.126+0000 7f6dc912a640 1 -- 172.21.15.27:0/3147759335 >> v1:172.21.15.27:6789/0 conn(0x7f6dc415e2e0 legacy=0x7f6dc406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.126+0000 7f6dc912a640 1 -- 172.21.15.27:0/3147759335 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc41616b0 con 0x7f6dc41540e0 2023-12-19T18:16:52.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.126+0000 7f6dbbfff640 1 -- 172.21.15.27:0/3147759335 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1487365271 0 0) 0x7f6dc415d600 con 0x7f6dc415e2e0 2023-12-19T18:16:52.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.126+0000 7f6dc8929640 1 --2- 172.21.15.27:0/3147759335 >> v2:172.21.15.192:3300/0 conn(0x7f6dc406c6e0 0x7f6dc415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:16:52.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.127+0000 7f6dc912a640 1 --2- 172.21.15.27:0/3147759335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dc41540e0 0x7f6dc4160bf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6dc406d3d0 tx=0x7f6dc002b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf9620ddd44d79a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:52.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.127+0000 7f6dbbfff640 1 -- 172.21.15.27:0/3147759335 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0030860 con 0x7f6dc41540e0 2023-12-19T18:16:52.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.127+0000 7f6dbbfff640 1 -- 172.21.15.27:0/3147759335 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc00309c0 con 0x7f6dc41540e0 2023-12-19T18:16:52.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.127+0000 7f6dbbfff640 1 -- 172.21.15.27:0/3147759335 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0030d50 con 0x7f6dc41540e0 2023-12-19T18:16:52.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.128+0000 7f6dca12c640 1 -- 172.21.15.27:0/3147759335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dc41540e0 msgr2=0x7f6dc4160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.128+0000 7f6dca12c640 1 --2- 172.21.15.27:0/3147759335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dc41540e0 0x7f6dc4160bf0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f6dc406d3d0 tx=0x7f6dc002b9a0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.128+0000 7f6dca12c640 1 -- 172.21.15.27:0/3147759335 shutdown_connections 2023-12-19T18:16:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.128+0000 7f6dca12c640 1 --2- 172.21.15.27:0/3147759335 >> v2:172.21.15.192:3300/0 conn(0x7f6dc406c6e0 0x7f6dc415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.128+0000 7f6dca12c640 1 --2- 172.21.15.27:0/3147759335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dc41540e0 0x7f6dc4160bf0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.128+0000 7f6dca12c640 1 -- 172.21.15.27:0/3147759335 >> 172.21.15.27:0/3147759335 conn(0x7f6dc406a7d0 msgr2=0x7f6dc40ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.128+0000 7f6dca12c640 1 -- 172.21.15.27:0/3147759335 shutdown_connections 2023-12-19T18:16:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.128+0000 7f6dca12c640 1 -- 172.21.15.27:0/3147759335 wait complete. 2023-12-19T18:16:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.130+0000 7f6dca12c640 1 Processor -- start 2023-12-19T18:16:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.130+0000 7f6dca12c640 1 -- start start 2023-12-19T18:16:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.131+0000 7f6dca12c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dc406c6e0 0x7f6dc41f28b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.131+0000 7f6dca12c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dc41540e0 0x7f6dc40b3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.131+0000 7f6dca12c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dc415e2e0 0x7f6dc40b3b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.131+0000 7f6dca12c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc4159c10 con 0x7f6dc415e2e0 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.131+0000 7f6dca12c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc4159650 con 0x7f6dc406c6e0 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.131+0000 7f6dca12c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dc4159930 con 0x7f6dc41540e0 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.132+0000 7f6dc992b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dc415e2e0 0x7f6dc40b3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.132+0000 7f6dc992b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dc415e2e0 0x7f6dc40b3b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41164/0 (socket says 172.21.15.27:41164) 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.132+0000 7f6dc992b640 1 -- 172.21.15.27:0/4207017777 learned_addr learned my addr 172.21.15.27:0/4207017777 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.132+0000 7f6dc912a640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dc406c6e0 0x7f6dc41f28b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.132+0000 7f6dc8929640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dc41540e0 0x7f6dc40b3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.132+0000 7f6dc992b640 1 -- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dc41540e0 msgr2=0x7f6dc40b3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.132+0000 7f6dc992b640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dc41540e0 0x7f6dc40b3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.132+0000 7f6dc992b640 1 -- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dc406c6e0 msgr2=0x7f6dc41f28b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.132+0000 7f6dc992b640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dc406c6e0 0x7f6dc41f28b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.132+0000 7f6dc992b640 1 -- 172.21.15.27:0/4207017777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00094d0 con 0x7f6dc415e2e0 2023-12-19T18:16:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.132+0000 7f6dc992b640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dc415e2e0 0x7f6dc40b3b90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00c7a0 tx=0x7f6dbc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.133+0000 7f6db9ffb640 1 -- 172.21.15.27:0/4207017777 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00a640 con 0x7f6dc415e2e0 2023-12-19T18:16:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.133+0000 7f6db9ffb640 1 -- 172.21.15.27:0/4207017777 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dbc00a7a0 con 0x7f6dc415e2e0 2023-12-19T18:16:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.133+0000 7f6db9ffb640 1 -- 172.21.15.27:0/4207017777 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dbc00abd0 con 0x7f6dc415e2e0 2023-12-19T18:16:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.133+0000 7f6dca12c640 1 -- 172.21.15.27:0/4207017777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dc41f3880 con 0x7f6dc415e2e0 2023-12-19T18:16:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.133+0000 7f6dca12c640 1 -- 172.21.15.27:0/4207017777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dc41f2df0 con 0x7f6dc415e2e0 2023-12-19T18:16:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.134+0000 7f6dca12c640 1 -- 172.21.15.27:0/4207017777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dc41f36c0 con 0x7f6dc415e2e0 2023-12-19T18:16:52.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.135+0000 7f6db9ffb640 1 -- 172.21.15.27:0/4207017777 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6dbc00ad30 con 0x7f6dc415e2e0 2023-12-19T18:16:52.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.136+0000 7f6db9ffb640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6da806a430 0x7f6da806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.136+0000 7f6db9ffb640 1 -- 172.21.15.27:0/4207017777 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f6dbc087fa0 con 0x7f6dc415e2e0 2023-12-19T18:16:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.137+0000 7f6dc912a640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6da806a430 0x7f6da806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.138+0000 7f6dc912a640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6da806a430 0x7f6da806c8f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0036000 tx=0x7f6dc0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:52.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.141+0000 7f6db9ffb640 1 -- 172.21.15.27:0/4207017777 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6dbc053470 con 0x7f6dc415e2e0 2023-12-19T18:16:52.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.344+0000 7f6dca12c640 1 -- 172.21.15.27:0/4207017777 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6dc404e840 con 0x7f6dc415e2e0 2023-12-19T18:16:52.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.345+0000 7f6db9ffb640 1 -- 172.21.15.27:0/4207017777 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6dbc018020 con 0x7f6dc415e2e0 2023-12-19T18:16:52.346 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:16:52.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.347+0000 7f6dca12c640 1 -- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6da806a430 msgr2=0x7f6da806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.347+0000 7f6dca12c640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6da806a430 0x7f6da806c8f0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0036000 tx=0x7f6dc0008030 comp rx=0 tx=0).stop 2023-12-19T18:16:52.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.347+0000 7f6dca12c640 1 -- 172.21.15.27:0/4207017777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dc415e2e0 msgr2=0x7f6dc40b3b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.347+0000 7f6dca12c640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dc415e2e0 0x7f6dc40b3b90 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6dbc00c7a0 tx=0x7f6dbc00cc60 comp rx=0 tx=0).stop 2023-12-19T18:16:52.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.348+0000 7f6dca12c640 1 -- 172.21.15.27:0/4207017777 shutdown_connections 2023-12-19T18:16:52.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.348+0000 7f6dca12c640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6da806a430 0x7f6da806c8f0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.348+0000 7f6dca12c640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6dc415e2e0 0x7f6dc40b3b90 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.348+0000 7f6dca12c640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6dc41540e0 0x7f6dc40b3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.348+0000 7f6dca12c640 1 --2- 172.21.15.27:0/4207017777 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6dc406c6e0 0x7f6dc41f28b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.348+0000 7f6dca12c640 1 -- 172.21.15.27:0/4207017777 >> 172.21.15.27:0/4207017777 conn(0x7f6dc406a7d0 msgr2=0x7f6dc415b220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:52.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.348+0000 7f6dca12c640 1 -- 172.21.15.27:0/4207017777 shutdown_connections 2023-12-19T18:16:52.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.348+0000 7f6dca12c640 1 -- 172.21.15.27:0/4207017777 wait complete. 2023-12-19T18:16:52.367 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-12-19T18:16:52.367 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:52.368 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-19T18:16:52.472 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[93397]: fuse finished with error 0 and tester_r 0 2023-12-19T18:16:52.482 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:16:52.483 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:52.483 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:16:52.553 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:16:52.553 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:52.596 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:52.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.802+0000 7f877b59e640 1 Processor -- start 2023-12-19T18:16:52.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.802+0000 7f877b59e640 1 -- start start 2023-12-19T18:16:52.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.804+0000 7f877b59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f877c1553a0 0x7f877c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.804+0000 7f877b59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877c0b0360 con 0x7f877c15ee30 2023-12-19T18:16:52.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.804+0000 7f877b59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f877c0cb350 con 0x7f877c0afeb0 2023-12-19T18:16:52.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.804+0000 7f877b59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f877c0cbcf0 con 0x7f877c1553a0 2023-12-19T18:16:52.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.804+0000 7f877a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f877c1553a0 0x7f877c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.804+0000 7f877ad9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f877c15ee30 0x7f877c0c39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55178/0 (socket says 172.21.15.27:55178) 2023-12-19T18:16:52.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.804+0000 7f877ad9d640 1 -- 172.21.15.27:0/3042054233 learned_addr learned my addr 172.21.15.27:0/3042054233 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:52.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.805+0000 7f877a59c640 1 -- 172.21.15.27:0/3042054233 >> v1:172.21.15.192:6789/0 conn(0x7f877c0afeb0 legacy=0x7f877c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:52.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.805+0000 7f877a59c640 1 -- 172.21.15.27:0/3042054233 >> v1:172.21.15.27:6789/0 conn(0x7f877c15ee30 legacy=0x7f877c0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.805+0000 7f877a59c640 1 -- 172.21.15.27:0/3042054233 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f877c0cb630 con 0x7f877c1553a0 2023-12-19T18:16:52.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.805+0000 7f877959a640 1 -- 172.21.15.27:0/3042054233 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537607246 0 0) 0x7f877c0b0360 con 0x7f877c15ee30 2023-12-19T18:16:52.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.806+0000 7f877a59c640 1 --2- 172.21.15.27:0/3042054233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f877c1553a0 0x7f877c0c50a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f876c0097a0 tx=0x7f876c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdae65661250997b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:52.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.806+0000 7f877959a640 1 -- 172.21.15.27:0/3042054233 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c02e070 con 0x7f877c1553a0 2023-12-19T18:16:52.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.806+0000 7f877959a640 1 -- 172.21.15.27:0/3042054233 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f876c0315a0 con 0x7f877c1553a0 2023-12-19T18:16:52.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.806+0000 7f877959a640 1 -- 172.21.15.27:0/3042054233 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f876c031930 con 0x7f877c1553a0 2023-12-19T18:16:52.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.807+0000 7f877b59e640 1 -- 172.21.15.27:0/3042054233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f877c1553a0 msgr2=0x7f877c0c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.807+0000 7f877b59e640 1 --2- 172.21.15.27:0/3042054233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f877c1553a0 0x7f877c0c50a0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f876c0097a0 tx=0x7f876c02b840 comp rx=0 tx=0).stop 2023-12-19T18:16:52.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.807+0000 7f877b59e640 1 -- 172.21.15.27:0/3042054233 shutdown_connections 2023-12-19T18:16:52.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.807+0000 7f877b59e640 1 --2- 172.21.15.27:0/3042054233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f877c1553a0 0x7f877c0c50a0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.807+0000 7f877b59e640 1 -- 172.21.15.27:0/3042054233 >> 172.21.15.27:0/3042054233 conn(0x7f877c069c70 msgr2=0x7f877c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:52.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.807+0000 7f877b59e640 1 -- 172.21.15.27:0/3042054233 shutdown_connections 2023-12-19T18:16:52.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.807+0000 7f877b59e640 1 -- 172.21.15.27:0/3042054233 wait complete. 2023-12-19T18:16:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.809+0000 7f877b59e640 1 Processor -- start 2023-12-19T18:16:52.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.809+0000 7f877b59e640 1 -- start start 2023-12-19T18:16:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.810+0000 7f877b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f877c0afeb0 0x7f877c0cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.810+0000 7f877b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f877c1553a0 0x7f877c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.810+0000 7f877b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f877c15ee30 0x7f877c152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.810+0000 7f877b59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0ce5d0 con 0x7f877c1553a0 2023-12-19T18:16:52.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.810+0000 7f877b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f877c152d60 con 0x7f877c0afeb0 2023-12-19T18:16:52.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.810+0000 7f877b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f877c0ce010 con 0x7f877c15ee30 2023-12-19T18:16:52.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.810+0000 7f8779d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f877c1553a0 0x7f877c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.810+0000 7f8779d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f877c1553a0 0x7f877c0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41184/0 (socket says 172.21.15.27:41184) 2023-12-19T18:16:52.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.810+0000 7f8779d9b640 1 -- 172.21.15.27:0/1800253575 learned_addr learned my addr 172.21.15.27:0/1800253575 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:52.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.810+0000 7f877a59c640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f877c0afeb0 0x7f877c0cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.811+0000 7f877ad9d640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f877c15ee30 0x7f877c152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.811+0000 7f8779d9b640 1 -- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f877c15ee30 msgr2=0x7f877c152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.811+0000 7f8779d9b640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f877c15ee30 0x7f877c152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.811+0000 7f8779d9b640 1 -- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f877c0afeb0 msgr2=0x7f877c0cd9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:52.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.811+0000 7f8779d9b640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f877c0afeb0 0x7f877c0cd9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:52.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.811+0000 7f8779d9b640 1 -- 172.21.15.27:0/1800253575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f876c009400 con 0x7f877c1553a0 2023-12-19T18:16:52.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.811+0000 7f877ad9d640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f877c15ee30 0x7f877c152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:52.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.811+0000 7f8779d9b640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f877c1553a0 0x7f877c0cceb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f877000c430 tx=0x7f877000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:52.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.811+0000 7f876affd640 1 -- 172.21.15.27:0/1800253575 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770011790 con 0x7f877c1553a0 2023-12-19T18:16:52.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.811+0000 7f876affd640 1 -- 172.21.15.27:0/1800253575 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87700118f0 con 0x7f877c1553a0 2023-12-19T18:16:52.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.811+0000 7f877b59e640 1 -- 172.21.15.27:0/1800253575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f877c0ce2f0 con 0x7f877c1553a0 2023-12-19T18:16:52.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.811+0000 7f876affd640 1 -- 172.21.15.27:0/1800253575 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8770011ce0 con 0x7f877c1553a0 2023-12-19T18:16:52.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.811+0000 7f877b59e640 1 -- 172.21.15.27:0/1800253575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f877c0cc990 con 0x7f877c1553a0 2023-12-19T18:16:52.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.813+0000 7f877b59e640 1 -- 172.21.15.27:0/1800253575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f877c0cc510 con 0x7f877c1553a0 2023-12-19T18:16:52.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.814+0000 7f876affd640 1 -- 172.21.15.27:0/1800253575 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f877001c460 con 0x7f877c1553a0 2023-12-19T18:16:52.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.814+0000 7f876affd640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f875c06a430 0x7f875c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:52.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.815+0000 7f876affd640 1 -- 172.21.15.27:0/1800253575 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f8770088920 con 0x7f877c1553a0 2023-12-19T18:16:52.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.815+0000 7f877a59c640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f875c06a430 0x7f875c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:52.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.816+0000 7f877a59c640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f875c06a430 0x7f875c06c8f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f876c036000 tx=0x7f876c03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:52.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:52.819+0000 7f876affd640 1 -- 172.21.15.27:0/1800253575 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f877c0cc510 con 0x7f877c1553a0 2023-12-19T18:16:53.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.014+0000 7f877b59e640 1 -- 172.21.15.27:0/1800253575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f877c0cc510 con 0x7f877c1553a0 2023-12-19T18:16:53.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.017+0000 7f876affd640 1 -- 172.21.15.27:0/1800253575 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16291 (secure 0 0 0) 0x7f877c0cc510 con 0x7f877c1553a0 2023-12-19T18:16:53.017 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:53.017 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:16:45.666993+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:23.925853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:24.827898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]},{"pgid":"5.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:53.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.019+0000 7f877b59e640 1 -- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f875c06a430 msgr2=0x7f875c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.019+0000 7f877b59e640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f875c06a430 0x7f875c06c8f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f876c036000 tx=0x7f876c03d040 comp rx=0 tx=0).stop 2023-12-19T18:16:53.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.019+0000 7f877b59e640 1 -- 172.21.15.27:0/1800253575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f877c1553a0 msgr2=0x7f877c0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.019+0000 7f877b59e640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f877c1553a0 0x7f877c0cceb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f877000c430 tx=0x7f877000c8f0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.019+0000 7f877b59e640 1 -- 172.21.15.27:0/1800253575 shutdown_connections 2023-12-19T18:16:53.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.019+0000 7f877b59e640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f875c06a430 0x7f875c06c8f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.019+0000 7f877b59e640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f877c15ee30 0x7f877c152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.019+0000 7f877b59e640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f877c1553a0 0x7f877c0cceb0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.019+0000 7f877b59e640 1 --2- 172.21.15.27:0/1800253575 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f877c0afeb0 0x7f877c0cd9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.019+0000 7f877b59e640 1 -- 172.21.15.27:0/1800253575 >> 172.21.15.27:0/1800253575 conn(0x7f877c069c70 msgr2=0x7f877c15e620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:53.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.020+0000 7f877b59e640 1 -- 172.21.15.27:0/1800253575 shutdown_connections 2023-12-19T18:16:53.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.020+0000 7f877b59e640 1 -- 172.21.15.27:0/1800253575 wait complete. 2023-12-19T18:16:53.040 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:16:23.925853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:16:24.827898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:16:53.040 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:16:53.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.276+0000 7fde45a5b640 1 Processor -- start 2023-12-19T18:16:53.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.276+0000 7fde45a5b640 1 -- start start 2023-12-19T18:16:53.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.277+0000 7fde45a5b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fde400d2100 0x7fde400cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.277+0000 7fde45a5b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fde400cdf80 0x7fde400b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.277+0000 7fde45a5b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde400d0e10 con 0x7fde4015cf80 2023-12-19T18:16:53.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.277+0000 7fde45a5b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde400b8cf0 con 0x7fde400d2100 2023-12-19T18:16:53.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.277+0000 7fde45a5b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde400ce3f0 con 0x7fde400cdf80 2023-12-19T18:16:53.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.278+0000 7fde44a59640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fde4015cf80 0x7fde400b85e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55192/0 (socket says 172.21.15.27:55192) 2023-12-19T18:16:53.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.278+0000 7fde44a59640 1 -- 172.21.15.27:0/601891717 learned_addr learned my addr 172.21.15.27:0/601891717 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:53.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.278+0000 7fde3ffff640 1 --2- 172.21.15.27:0/601891717 >> v2:172.21.15.27:3300/0 conn(0x7fde400d2100 0x7fde400cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.278+0000 7fde4525a640 1 --2- 172.21.15.27:0/601891717 >> v2:172.21.15.192:3300/0 conn(0x7fde400cdf80 0x7fde400b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.278+0000 7fde3ffff640 1 -- 172.21.15.27:0/601891717 >> v2:172.21.15.192:3300/0 conn(0x7fde400cdf80 msgr2=0x7fde400b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:53.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.278+0000 7fde3ffff640 1 --2- 172.21.15.27:0/601891717 >> v2:172.21.15.192:3300/0 conn(0x7fde400cdf80 0x7fde400b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.278+0000 7fde3ffff640 1 -- 172.21.15.27:0/601891717 >> v1:172.21.15.27:6789/0 conn(0x7fde4015cf80 legacy=0x7fde400b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.278+0000 7fde3f7fe640 1 -- 172.21.15.27:0/601891717 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3810676667 0 0) 0x7fde400d0e10 con 0x7fde4015cf80 2023-12-19T18:16:53.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.278+0000 7fde3ffff640 1 -- 172.21.15.27:0/601891717 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde400d1c70 con 0x7fde400d2100 2023-12-19T18:16:53.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.280+0000 7fde3ffff640 1 --2- 172.21.15.27:0/601891717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde400d2100 0x7fde400cda40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fde400c9cd0 tx=0x7fde3402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=453c6d1b6f4dea11 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:53.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.280+0000 7fde3f7fe640 1 -- 172.21.15.27:0/601891717 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34030610 con 0x7fde400d2100 2023-12-19T18:16:53.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.280+0000 7fde3f7fe640 1 -- 172.21.15.27:0/601891717 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde34030770 con 0x7fde400d2100 2023-12-19T18:16:53.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.280+0000 7fde3f7fe640 1 -- 172.21.15.27:0/601891717 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34030b00 con 0x7fde400d2100 2023-12-19T18:16:53.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.280+0000 7fde45a5b640 1 -- 172.21.15.27:0/601891717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde400d2100 msgr2=0x7fde400cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.280+0000 7fde45a5b640 1 --2- 172.21.15.27:0/601891717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde400d2100 0x7fde400cda40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fde400c9cd0 tx=0x7fde3402b960 comp rx=0 tx=0).stop 2023-12-19T18:16:53.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.281+0000 7fde45a5b640 1 -- 172.21.15.27:0/601891717 shutdown_connections 2023-12-19T18:16:53.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.281+0000 7fde45a5b640 1 --2- 172.21.15.27:0/601891717 >> v2:172.21.15.192:3300/0 conn(0x7fde400cdf80 0x7fde400b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.281+0000 7fde45a5b640 1 --2- 172.21.15.27:0/601891717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde400d2100 0x7fde400cda40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.281+0000 7fde45a5b640 1 -- 172.21.15.27:0/601891717 >> 172.21.15.27:0/601891717 conn(0x7fde40072e20 msgr2=0x7fde40073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:53.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.281+0000 7fde45a5b640 1 -- 172.21.15.27:0/601891717 shutdown_connections 2023-12-19T18:16:53.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.281+0000 7fde45a5b640 1 -- 172.21.15.27:0/601891717 wait complete. 2023-12-19T18:16:53.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.282+0000 7fde45a5b640 1 Processor -- start 2023-12-19T18:16:53.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.283+0000 7fde45a5b640 1 -- start start 2023-12-19T18:16:53.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde45a5b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde400cdf80 0x7fde400c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde45a5b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde400d2100 0x7fde40159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde45a5b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4015cf80 0x7fde400b3290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde45a5b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde400ca280 con 0x7fde4015cf80 2023-12-19T18:16:53.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde45a5b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde400b1f70 con 0x7fde400d2100 2023-12-19T18:16:53.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde45a5b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde40159840 con 0x7fde400cdf80 2023-12-19T18:16:53.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde4525a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4015cf80 0x7fde400b3290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde4525a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4015cf80 0x7fde400b3290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41210/0 (socket says 172.21.15.27:41210) 2023-12-19T18:16:53.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde4525a640 1 -- 172.21.15.27:0/4139140677 learned_addr learned my addr 172.21.15.27:0/4139140677 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:53.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde3ffff640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde400d2100 0x7fde40159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde44a59640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde400cdf80 0x7fde400c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde4525a640 1 -- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde400cdf80 msgr2=0x7fde400c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde4525a640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde400cdf80 0x7fde400c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde4525a640 1 -- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde400d2100 msgr2=0x7fde40159300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde4525a640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde400d2100 0x7fde40159300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde4525a640 1 -- 172.21.15.27:0/4139140677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde34009400 con 0x7fde4015cf80 2023-12-19T18:16:53.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde44a59640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde400cdf80 0x7fde400c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:53.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.284+0000 7fde3ffff640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde400d2100 0x7fde40159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:53.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.285+0000 7fde4525a640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4015cf80 0x7fde400b3290 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fde3800d580 tx=0x7fde3800da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:53.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.285+0000 7fde3d7fa640 1 -- 172.21.15.27:0/4139140677 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde380125f0 con 0x7fde4015cf80 2023-12-19T18:16:53.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.285+0000 7fde3d7fa640 1 -- 172.21.15.27:0/4139140677 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde38012770 con 0x7fde4015cf80 2023-12-19T18:16:53.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.285+0000 7fde3d7fa640 1 -- 172.21.15.27:0/4139140677 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde38012b20 con 0x7fde4015cf80 2023-12-19T18:16:53.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.285+0000 7fde45a5b640 1 -- 172.21.15.27:0/4139140677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde400ca490 con 0x7fde4015cf80 2023-12-19T18:16:53.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.285+0000 7fde45a5b640 1 -- 172.21.15.27:0/4139140677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde400caa50 con 0x7fde4015cf80 2023-12-19T18:16:53.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.286+0000 7fde45a5b640 1 -- 172.21.15.27:0/4139140677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4004e840 con 0x7fde4015cf80 2023-12-19T18:16:53.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.287+0000 7fde3d7fa640 1 -- 172.21.15.27:0/4139140677 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fde38012c80 con 0x7fde4015cf80 2023-12-19T18:16:53.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.288+0000 7fde3d7fa640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde2c06a430 0x7fde2c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.288+0000 7fde3d7fa640 1 -- 172.21.15.27:0/4139140677 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fde38088e70 con 0x7fde4015cf80 2023-12-19T18:16:53.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.289+0000 7fde44a59640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde2c06a430 0x7fde2c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.290+0000 7fde44a59640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde2c06a430 0x7fde2c06c8f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fde30007fd0 tx=0x7fde30009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:53.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.293+0000 7fde3d7fa640 1 -- 172.21.15.27:0/4139140677 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fde38054350 con 0x7fde4015cf80 2023-12-19T18:16:53.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.490+0000 7fde45a5b640 1 -- 172.21.15.27:0/4139140677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fde400b2d70 con 0x7fde4015cf80 2023-12-19T18:16:53.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.491+0000 7fde3d7fa640 1 -- 172.21.15.27:0/4139140677 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fde38057970 con 0x7fde4015cf80 2023-12-19T18:16:53.491 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:16:53.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.494+0000 7fde45a5b640 1 -- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde2c06a430 msgr2=0x7fde2c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.494+0000 7fde45a5b640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde2c06a430 0x7fde2c06c8f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fde30007fd0 tx=0x7fde30009040 comp rx=0 tx=0).stop 2023-12-19T18:16:53.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.494+0000 7fde45a5b640 1 -- 172.21.15.27:0/4139140677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4015cf80 msgr2=0x7fde400b3290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.494+0000 7fde45a5b640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4015cf80 0x7fde400b3290 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fde3800d580 tx=0x7fde3800da40 comp rx=0 tx=0).stop 2023-12-19T18:16:53.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.494+0000 7fde45a5b640 1 -- 172.21.15.27:0/4139140677 shutdown_connections 2023-12-19T18:16:53.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.494+0000 7fde45a5b640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde2c06a430 0x7fde2c06c8f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.494+0000 7fde45a5b640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde4015cf80 0x7fde400b3290 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.494+0000 7fde45a5b640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde400d2100 0x7fde40159300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.494+0000 7fde45a5b640 1 --2- 172.21.15.27:0/4139140677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde400cdf80 0x7fde400c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.494+0000 7fde45a5b640 1 -- 172.21.15.27:0/4139140677 >> 172.21.15.27:0/4139140677 conn(0x7fde40072e20 msgr2=0x7fde400b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:53.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.494+0000 7fde45a5b640 1 -- 172.21.15.27:0/4139140677 shutdown_connections 2023-12-19T18:16:53.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.494+0000 7fde45a5b640 1 -- 172.21.15.27:0/4139140677 wait complete. 2023-12-19T18:16:53.512 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:16:53.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.741+0000 7f54d7fff640 1 Processor -- start 2023-12-19T18:16:53.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.742+0000 7f54d7fff640 1 -- start start 2023-12-19T18:16:53.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.743+0000 7f54d7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f54d80d1a20 0x7f54d80cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.743+0000 7f54d7fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f54d80b3980 0x7f54d80cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.743+0000 7f54d7fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d80d3850 con 0x7f54d815d120 2023-12-19T18:16:53.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.743+0000 7f54d7fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d80d0b10 con 0x7f54d80d1a20 2023-12-19T18:16:53.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.743+0000 7f54d7fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d815d500 con 0x7f54d80b3980 2023-12-19T18:16:53.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.744+0000 7f54d67fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f54d80d1a20 0x7f54d80cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.744+0000 7f54d67fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f54d80d1a20 0x7f54d80cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41212/0 (socket says 172.21.15.27:41212) 2023-12-19T18:16:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.744+0000 7f54d67fc640 1 -- 172.21.15.27:0/296860565 learned_addr learned my addr 172.21.15.27:0/296860565 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.744+0000 7f54d67fc640 1 -- 172.21.15.27:0/296860565 >> v2:172.21.15.192:3300/0 conn(0x7f54d80b3980 msgr2=0x7f54d80cd940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.744+0000 7f54d77fe640 1 --2- 172.21.15.27:0/296860565 >> v2:172.21.15.192:3300/0 conn(0x7f54d80b3980 0x7f54d80cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.744+0000 7f54d67fc640 1 --2- 172.21.15.27:0/296860565 >> v2:172.21.15.192:3300/0 conn(0x7f54d80b3980 0x7f54d80cd940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.744+0000 7f54d67fc640 1 -- 172.21.15.27:0/296860565 >> v1:172.21.15.192:6789/0 conn(0x7f54d815d120 legacy=0x7f54d80d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.744+0000 7f54d67fc640 1 -- 172.21.15.27:0/296860565 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d80d1040 con 0x7f54d80d1a20 2023-12-19T18:16:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.744+0000 7f54d77fe640 1 --2- 172.21.15.27:0/296860565 >> v2:172.21.15.192:3300/0 conn(0x7f54d80b3980 0x7f54d80cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:53.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.746+0000 7f54d67fc640 1 --2- 172.21.15.27:0/296860565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80d1a20 0x7f54d80cfd70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f54d808d520 tx=0x7f54cc02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b84c6b0be0e0dfb4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:53.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.746+0000 7f54d5ffb640 1 -- 172.21.15.27:0/296860565 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc031870 con 0x7f54d80d1a20 2023-12-19T18:16:53.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.746+0000 7f54d5ffb640 1 -- 172.21.15.27:0/296860565 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc0319d0 con 0x7f54d80d1a20 2023-12-19T18:16:53.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.746+0000 7f54d5ffb640 1 -- 172.21.15.27:0/296860565 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc031d60 con 0x7f54d80d1a20 2023-12-19T18:16:53.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.747+0000 7f54d7fff640 1 -- 172.21.15.27:0/296860565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80d1a20 msgr2=0x7f54d80cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.747+0000 7f54d7fff640 1 --2- 172.21.15.27:0/296860565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80d1a20 0x7f54d80cfd70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f54d808d520 tx=0x7f54cc02b910 comp rx=0 tx=0).stop 2023-12-19T18:16:53.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.747+0000 7f54d7fff640 1 -- 172.21.15.27:0/296860565 shutdown_connections 2023-12-19T18:16:53.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.747+0000 7f54d7fff640 1 --2- 172.21.15.27:0/296860565 >> v2:172.21.15.192:3300/0 conn(0x7f54d80b3980 0x7f54d80cd940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.747+0000 7f54d7fff640 1 --2- 172.21.15.27:0/296860565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80d1a20 0x7f54d80cfd70 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.747+0000 7f54d7fff640 1 -- 172.21.15.27:0/296860565 >> 172.21.15.27:0/296860565 conn(0x7f54d80d2d30 msgr2=0x7f54d80d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:53.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.747+0000 7f54d7fff640 1 -- 172.21.15.27:0/296860565 shutdown_connections 2023-12-19T18:16:53.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.747+0000 7f54d7fff640 1 -- 172.21.15.27:0/296860565 wait complete. 2023-12-19T18:16:53.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.749+0000 7f54d7fff640 1 Processor -- start 2023-12-19T18:16:53.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.749+0000 7f54d7fff640 1 -- start start 2023-12-19T18:16:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.750+0000 7f54d7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80b3980 0x7f54d8149ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.750+0000 7f54d7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d80d1a20 0x7f54d80cc060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.750+0000 7f54d7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d815d120 0x7f54d80af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.750+0000 7f54d7fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d80cd100 con 0x7f54d80b3980 2023-12-19T18:16:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.750+0000 7f54d7fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d80ccb40 con 0x7f54d815d120 2023-12-19T18:16:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.750+0000 7f54d7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d80cce20 con 0x7f54d80d1a20 2023-12-19T18:16:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.750+0000 7f54d6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80b3980 0x7f54d8149ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.750+0000 7f54d6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80b3980 0x7f54d8149ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41216/0 (socket says 172.21.15.27:41216) 2023-12-19T18:16:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.750+0000 7f54d6ffd640 1 -- 172.21.15.27:0/3390658627 learned_addr learned my addr 172.21.15.27:0/3390658627 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.751+0000 7f54d77fe640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d815d120 0x7f54d80af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.751+0000 7f54d67fc640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d80d1a20 0x7f54d80cc060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.751+0000 7f54d6ffd640 1 -- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d80d1a20 msgr2=0x7f54d80cc060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.751+0000 7f54d6ffd640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d80d1a20 0x7f54d80cc060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.751+0000 7f54d6ffd640 1 -- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d815d120 msgr2=0x7f54d80af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.751+0000 7f54d6ffd640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d815d120 0x7f54d80af900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.751+0000 7f54d6ffd640 1 -- 172.21.15.27:0/3390658627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc0094d0 con 0x7f54d80b3980 2023-12-19T18:16:53.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.751+0000 7f54d77fe640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d815d120 0x7f54d80af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.751+0000 7f54d6ffd640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80b3980 0x7f54d8149ac0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f54c800b3f0 tx=0x7f54c800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.751+0000 7f54bffff640 1 -- 172.21.15.27:0/3390658627 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8011830 con 0x7f54d80b3980 2023-12-19T18:16:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.751+0000 7f54bffff640 1 -- 172.21.15.27:0/3390658627 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c8011990 con 0x7f54d80b3980 2023-12-19T18:16:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.751+0000 7f54bffff640 1 -- 172.21.15.27:0/3390658627 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8011d40 con 0x7f54d80b3980 2023-12-19T18:16:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.751+0000 7f54d7fff640 1 -- 172.21.15.27:0/3390658627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d814a300 con 0x7f54d80b3980 2023-12-19T18:16:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.752+0000 7f54d7fff640 1 -- 172.21.15.27:0/3390658627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d80b0ba0 con 0x7f54d80b3980 2023-12-19T18:16:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.753+0000 7f54d7fff640 1 -- 172.21.15.27:0/3390658627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d80b0d90 con 0x7f54d80b3980 2023-12-19T18:16:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.754+0000 7f54bffff640 1 -- 172.21.15.27:0/3390658627 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f54c801a400 con 0x7f54d80b3980 2023-12-19T18:16:53.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.755+0000 7f54bffff640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f54c406a460 0x7f54c406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:53.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.755+0000 7f54bffff640 1 -- 172.21.15.27:0/3390658627 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f54c80887c0 con 0x7f54d80b3980 2023-12-19T18:16:53.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.755+0000 7f54d67fc640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f54c406a460 0x7f54c406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:53.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.756+0000 7f54d67fc640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f54c406a460 0x7f54c406c920 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f54cc036000 tx=0x7f54cc03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:53.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.759+0000 7f54bffff640 1 -- 172.21.15.27:0/3390658627 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f54d80b0d90 con 0x7f54d80b3980 2023-12-19T18:16:53.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.951+0000 7f54d7fff640 1 -- 172.21.15.27:0/3390658627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f54d814a160 con 0x7f54d80b3980 2023-12-19T18:16:53.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.952+0000 7f54bffff640 1 -- 172.21.15.27:0/3390658627 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f54d814a160 con 0x7f54d80b3980 2023-12-19T18:16:53.953 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:16:53.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.955+0000 7f54d7fff640 1 -- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f54c406a460 msgr2=0x7f54c406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.955+0000 7f54d7fff640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f54c406a460 0x7f54c406c920 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f54cc036000 tx=0x7f54cc03c040 comp rx=0 tx=0).stop 2023-12-19T18:16:53.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.955+0000 7f54d7fff640 1 -- 172.21.15.27:0/3390658627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80b3980 msgr2=0x7f54d8149ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:53.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.955+0000 7f54d7fff640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80b3980 0x7f54d8149ac0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f54c800b3f0 tx=0x7f54c800b8b0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.955+0000 7f54d7fff640 1 -- 172.21.15.27:0/3390658627 shutdown_connections 2023-12-19T18:16:53.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.955+0000 7f54d7fff640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f54c406a460 0x7f54c406c920 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.955+0000 7f54d7fff640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f54d815d120 0x7f54d80af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.955+0000 7f54d7fff640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f54d80d1a20 0x7f54d80cc060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.955+0000 7f54d7fff640 1 --2- 172.21.15.27:0/3390658627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80b3980 0x7f54d8149ac0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:53.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.955+0000 7f54d7fff640 1 -- 172.21.15.27:0/3390658627 >> 172.21.15.27:0/3390658627 conn(0x7f54d80d2d30 msgr2=0x7f54d80d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:53.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.955+0000 7f54d7fff640 1 -- 172.21.15.27:0/3390658627 shutdown_connections 2023-12-19T18:16:53.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:53.956+0000 7f54d7fff640 1 -- 172.21.15.27:0/3390658627 wait complete. 2023-12-19T18:16:53.976 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:16:54.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.213+0000 7f6e07fff640 1 Processor -- start 2023-12-19T18:16:54.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.213+0000 7f6e07fff640 1 -- start start 2023-12-19T18:16:54.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.215+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e0815cf80 0x7f6e080d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:54.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.215+0000 7f6e07fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e080d0e10 con 0x7f6e080b8a60 2023-12-19T18:16:54.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.215+0000 7f6e07fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e080d1c70 con 0x7f6e080b85e0 2023-12-19T18:16:54.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.215+0000 7f6e07fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e0815d360 con 0x7f6e0815cf80 2023-12-19T18:16:54.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.215+0000 7f6e077fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6e080b8a60 0x7f6e080b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55194/0 (socket says 172.21.15.27:55194) 2023-12-19T18:16:54.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.215+0000 7f6e077fe640 1 -- 172.21.15.27:0/473417526 learned_addr learned my addr 172.21.15.27:0/473417526 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:54.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.216+0000 7f6e06ffd640 1 --2- 172.21.15.27:0/473417526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e0815cf80 0x7f6e080d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:54.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.216+0000 7f6e05ffb640 1 -- 172.21.15.27:0/473417526 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219293067 0 0) 0x7f6e080d0e10 con 0x7f6e080b8a60 2023-12-19T18:16:54.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.216+0000 7f6e05ffb640 1 -- 172.21.15.27:0/473417526 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6df0003520 con 0x7f6e080b8a60 2023-12-19T18:16:54.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.216+0000 7f6e05ffb640 1 -- 172.21.15.27:0/473417526 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044943173 0 0) 0x7f6e080d1c70 con 0x7f6e080b85e0 2023-12-19T18:16:54.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.216+0000 7f6e05ffb640 1 -- 172.21.15.27:0/473417526 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e080d0e10 con 0x7f6e080b85e0 2023-12-19T18:16:54.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.216+0000 7f6e06ffd640 1 -- 172.21.15.27:0/473417526 >> v1:172.21.15.192:6789/0 conn(0x7f6e080b85e0 legacy=0x7f6e080cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:54.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.216+0000 7f6e06ffd640 1 -- 172.21.15.27:0/473417526 >> v1:172.21.15.27:6789/0 conn(0x7f6e080b8a60 legacy=0x7f6e080b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:54.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.217+0000 7f6e05ffb640 1 -- 172.21.15.27:0/473417526 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2508353603 0 0) 0x7f6df0003520 con 0x7f6e080b8a60 2023-12-19T18:16:54.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.217+0000 7f6e06ffd640 1 -- 172.21.15.27:0/473417526 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e080d1060 con 0x7f6e0815cf80 2023-12-19T18:16:54.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.218+0000 7f6e06ffd640 1 --2- 172.21.15.27:0/473417526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e0815cf80 0x7f6e080d2100 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009dc0 tx=0x7f6df802be70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6cc4990d0b279c92 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:54.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.218+0000 7f6e05ffb640 1 -- 172.21.15.27:0/473417526 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e00001670 con 0x7f6e0815cf80 2023-12-19T18:16:54.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.218+0000 7f6e05ffb640 1 -- 172.21.15.27:0/473417526 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6df802c9a0 con 0x7f6e0815cf80 2023-12-19T18:16:54.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.218+0000 7f6e05ffb640 1 -- 172.21.15.27:0/473417526 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6df802cd30 con 0x7f6e0815cf80 2023-12-19T18:16:54.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.219+0000 7f6e07fff640 1 -- 172.21.15.27:0/473417526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e0815cf80 msgr2=0x7f6e080d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:54.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.219+0000 7f6e07fff640 1 --2- 172.21.15.27:0/473417526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e0815cf80 0x7f6e080d2100 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f6df8009dc0 tx=0x7f6df802be70 comp rx=0 tx=0).stop 2023-12-19T18:16:54.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.219+0000 7f6e07fff640 1 -- 172.21.15.27:0/473417526 shutdown_connections 2023-12-19T18:16:54.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.219+0000 7f6e07fff640 1 --2- 172.21.15.27:0/473417526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e0815cf80 0x7f6e080d2100 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.219+0000 7f6e07fff640 1 -- 172.21.15.27:0/473417526 >> 172.21.15.27:0/473417526 conn(0x7f6e08072e20 msgr2=0x7f6e08073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:54.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.219+0000 7f6e07fff640 1 -- 172.21.15.27:0/473417526 shutdown_connections 2023-12-19T18:16:54.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.219+0000 7f6e07fff640 1 -- 172.21.15.27:0/473417526 wait complete. 2023-12-19T18:16:54.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.221+0000 7f6e07fff640 1 Processor -- start 2023-12-19T18:16:54.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.221+0000 7f6e07fff640 1 -- start start 2023-12-19T18:16:54.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.222+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e080b85e0 0x7f6e08152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:54.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.222+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e0815cf80 0x7f6e08154ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:54.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.222+0000 7f6e07fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e081550e0 0x7f6e080cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:54.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.222+0000 7f6e07fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e081527b0 con 0x7f6e0815cf80 2023-12-19T18:16:54.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.222+0000 7f6e07fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e08155b40 con 0x7f6e081550e0 2023-12-19T18:16:54.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.222+0000 7f6e07fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e08153160 con 0x7f6e080b85e0 2023-12-19T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.222+0000 7f6e067fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e0815cf80 0x7f6e08154ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.222+0000 7f6e067fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e0815cf80 0x7f6e08154ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41226/0 (socket says 172.21.15.27:41226) 2023-12-19T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.222+0000 7f6e067fc640 1 -- 172.21.15.27:0/3623945250 learned_addr learned my addr 172.21.15.27:0/3623945250 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.223+0000 7f6e06ffd640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e080b85e0 0x7f6e08152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.223+0000 7f6e077fe640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e081550e0 0x7f6e080cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.223+0000 7f6e067fc640 1 -- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e080b85e0 msgr2=0x7f6e08152270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.223+0000 7f6e067fc640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e080b85e0 0x7f6e08152270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.223+0000 7f6e067fc640 1 -- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e081550e0 msgr2=0x7f6e080cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.223+0000 7f6e067fc640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e081550e0 0x7f6e080cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.223+0000 7f6e067fc640 1 -- 172.21.15.27:0/3623945250 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6df8009a70 con 0x7f6e0815cf80 2023-12-19T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.223+0000 7f6e067fc640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e0815cf80 0x7f6e08154ba0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00c530 tx=0x7f6dfc00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.223+0000 7f6def7fe640 1 -- 172.21.15.27:0/3623945250 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc0052d0 con 0x7f6e0815cf80 2023-12-19T18:16:54.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.224+0000 7f6def7fe640 1 -- 172.21.15.27:0/3623945250 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dfc005450 con 0x7f6e0815cf80 2023-12-19T18:16:54.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.224+0000 7f6e07fff640 1 -- 172.21.15.27:0/3623945250 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e08152e30 con 0x7f6e0815cf80 2023-12-19T18:16:54.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.224+0000 7f6def7fe640 1 -- 172.21.15.27:0/3623945250 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dfc012870 con 0x7f6e0815cf80 2023-12-19T18:16:54.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.224+0000 7f6e07fff640 1 -- 172.21.15.27:0/3623945250 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e08155860 con 0x7f6e0815cf80 2023-12-19T18:16:54.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.225+0000 7f6e07fff640 1 -- 172.21.15.27:0/3623945250 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e0804e840 con 0x7f6e0815cf80 2023-12-19T18:16:54.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.226+0000 7f6def7fe640 1 -- 172.21.15.27:0/3623945250 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6dfc0129d0 con 0x7f6e0815cf80 2023-12-19T18:16:54.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.227+0000 7f6def7fe640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6df406a430 0x7f6df406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:54.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.227+0000 7f6def7fe640 1 -- 172.21.15.27:0/3623945250 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f6dfc0884e0 con 0x7f6e0815cf80 2023-12-19T18:16:54.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.227+0000 7f6e06ffd640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6df406a430 0x7f6df406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:54.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.229+0000 7f6e06ffd640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6df406a430 0x7f6df406c8f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6df8036000 tx=0x7f6df8030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:54.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.231+0000 7f6def7fe640 1 -- 172.21.15.27:0/3623945250 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6dfc053a60 con 0x7f6e0815cf80 2023-12-19T18:16:54.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.424+0000 7f6e07fff640 1 -- 172.21.15.27:0/3623945250 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f6e081532a0 con 0x7f6e0815cf80 2023-12-19T18:16:54.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.425+0000 7f6def7fe640 1 -- 172.21.15.27:0/3623945250 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7f6dfc057080 con 0x7f6e0815cf80 2023-12-19T18:16:54.425 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:16:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.427+0000 7f6e07fff640 1 -- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6df406a430 msgr2=0x7f6df406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.427+0000 7f6e07fff640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6df406a430 0x7f6df406c8f0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6df8036000 tx=0x7f6df8030000 comp rx=0 tx=0).stop 2023-12-19T18:16:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.427+0000 7f6e07fff640 1 -- 172.21.15.27:0/3623945250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e0815cf80 msgr2=0x7f6e08154ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.427+0000 7f6e07fff640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e0815cf80 0x7f6e08154ba0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f6dfc00c530 tx=0x7f6dfc00c9f0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.428+0000 7f6e07fff640 1 -- 172.21.15.27:0/3623945250 shutdown_connections 2023-12-19T18:16:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.428+0000 7f6e07fff640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6df406a430 0x7f6df406c8f0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.428+0000 7f6e07fff640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e081550e0 0x7f6e080cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.428+0000 7f6e07fff640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e0815cf80 0x7f6e08154ba0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.428+0000 7f6e07fff640 1 --2- 172.21.15.27:0/3623945250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e080b85e0 0x7f6e08152270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.428+0000 7f6e07fff640 1 -- 172.21.15.27:0/3623945250 >> 172.21.15.27:0/3623945250 conn(0x7f6e08072e20 msgr2=0x7f6e080cfd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:54.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.428+0000 7f6e07fff640 1 -- 172.21.15.27:0/3623945250 shutdown_connections 2023-12-19T18:16:54.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.428+0000 7f6e07fff640 1 -- 172.21.15.27:0/3623945250 wait complete. 2023-12-19T18:16:54.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:16:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.676+0000 7f456a841640 1 Processor -- start 2023-12-19T18:16:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.676+0000 7f456a841640 1 -- start start 2023-12-19T18:16:54.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.678+0000 7f456a841640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f456406fc10 0x7f456406d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:54.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.678+0000 7f456a841640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4564070c00 0x7f45640712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:54.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.678+0000 7f456a841640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4564071820 con 0x7f456415adf0 2023-12-19T18:16:54.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.678+0000 7f456a841640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45640703c0 con 0x7f4564070c00 2023-12-19T18:16:54.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.678+0000 7f456a841640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456406d8c0 con 0x7f456406fc10 2023-12-19T18:16:54.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.678+0000 7f4568dde640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4564070c00 0x7f45640712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:54.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.678+0000 7f4568dde640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4564070c00 0x7f45640712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41228/0 (socket says 172.21.15.27:41228) 2023-12-19T18:16:54.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.678+0000 7f4568dde640 1 -- 172.21.15.27:0/4192188148 learned_addr learned my addr 172.21.15.27:0/4192188148 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:54.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.679+0000 7f4568dde640 1 -- 172.21.15.27:0/4192188148 >> v2:172.21.15.192:3300/0 conn(0x7f456406fc10 msgr2=0x7f456406d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:54.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.679+0000 7f4568dde640 1 --2- 172.21.15.27:0/4192188148 >> v2:172.21.15.192:3300/0 conn(0x7f456406fc10 0x7f456406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.679+0000 7f4568dde640 1 -- 172.21.15.27:0/4192188148 >> v1:172.21.15.192:6789/0 conn(0x7f456415adf0 legacy=0x7f456406f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:54.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.679+0000 7f4568dde640 1 -- 172.21.15.27:0/4192188148 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456406b520 con 0x7f4564070c00 2023-12-19T18:16:54.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.680+0000 7f4568dde640 1 --2- 172.21.15.27:0/4192188148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4564070c00 0x7f45640712e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4564071230 tx=0x7f455c02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28ba4567975cb37e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:54.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.680+0000 7f45637fe640 1 -- 172.21.15.27:0/4192188148 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c02c9f0 con 0x7f4564070c00 2023-12-19T18:16:54.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.680+0000 7f45637fe640 1 -- 172.21.15.27:0/4192188148 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455c02cb50 con 0x7f4564070c00 2023-12-19T18:16:54.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.680+0000 7f45637fe640 1 -- 172.21.15.27:0/4192188148 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455c0343f0 con 0x7f4564070c00 2023-12-19T18:16:54.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.681+0000 7f456a841640 1 -- 172.21.15.27:0/4192188148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4564070c00 msgr2=0x7f45640712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:54.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.681+0000 7f456a841640 1 --2- 172.21.15.27:0/4192188148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4564070c00 0x7f45640712e0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f4564071230 tx=0x7f455c02be50 comp rx=0 tx=0).stop 2023-12-19T18:16:54.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.681+0000 7f456a841640 1 -- 172.21.15.27:0/4192188148 shutdown_connections 2023-12-19T18:16:54.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.681+0000 7f456a841640 1 --2- 172.21.15.27:0/4192188148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4564070c00 0x7f45640712e0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.681+0000 7f456a841640 1 --2- 172.21.15.27:0/4192188148 >> v2:172.21.15.192:3300/0 conn(0x7f456406fc10 0x7f456406d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.681+0000 7f456a841640 1 -- 172.21.15.27:0/4192188148 >> 172.21.15.27:0/4192188148 conn(0x7f45640ceb50 msgr2=0x7f45640cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:54.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.681+0000 7f456a841640 1 -- 172.21.15.27:0/4192188148 shutdown_connections 2023-12-19T18:16:54.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.681+0000 7f456a841640 1 -- 172.21.15.27:0/4192188148 wait complete. 2023-12-19T18:16:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.683+0000 7f456a841640 1 Processor -- start 2023-12-19T18:16:54.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.683+0000 7f456a841640 1 -- start start 2023-12-19T18:16:54.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f456a841640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f456406fc10 0x7f456414e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:54.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f456a841640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4564070c00 0x7f45641f35a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:54.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f456a841640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456415adf0 0x7f45640c5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:54.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f456a841640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456414e590 con 0x7f456406fc10 2023-12-19T18:16:54.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f456a841640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45641f2b20 con 0x7f4564070c00 2023-12-19T18:16:54.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f456a841640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45640c66e0 con 0x7f456415adf0 2023-12-19T18:16:54.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f4563fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f456406fc10 0x7f456414e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:54.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f4563fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f456406fc10 0x7f456414e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41240/0 (socket says 172.21.15.27:41240) 2023-12-19T18:16:54.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f4563fff640 1 -- 172.21.15.27:0/991189095 learned_addr learned my addr 172.21.15.27:0/991189095 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:54.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f4563fff640 1 -- 172.21.15.27:0/991189095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456415adf0 msgr2=0x7f45640c5c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:54.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f4563fff640 1 --2- 172.21.15.27:0/991189095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456415adf0 0x7f45640c5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f4563fff640 1 -- 172.21.15.27:0/991189095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4564070c00 msgr2=0x7f45641f35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:54.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f4563fff640 1 --2- 172.21.15.27:0/991189095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4564070c00 0x7f45641f35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.684+0000 7f4563fff640 1 -- 172.21.15.27:0/991189095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c009b30 con 0x7f456406fc10 2023-12-19T18:16:54.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.685+0000 7f4563fff640 1 --2- 172.21.15.27:0/991189095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f456406fc10 0x7f456414e050 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f455400b3f0 tx=0x7f455400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:54.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.685+0000 7f45617fa640 1 -- 172.21.15.27:0/991189095 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554011830 con 0x7f456406fc10 2023-12-19T18:16:54.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.685+0000 7f45617fa640 1 -- 172.21.15.27:0/991189095 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4554011990 con 0x7f456406fc10 2023-12-19T18:16:54.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.685+0000 7f456a841640 1 -- 172.21.15.27:0/991189095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45640c69c0 con 0x7f456406fc10 2023-12-19T18:16:54.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.685+0000 7f45617fa640 1 -- 172.21.15.27:0/991189095 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554011d40 con 0x7f456406fc10 2023-12-19T18:16:54.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.685+0000 7f456a841640 1 -- 172.21.15.27:0/991189095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45641f3080 con 0x7f456406fc10 2023-12-19T18:16:54.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.686+0000 7f456a841640 1 -- 172.21.15.27:0/991189095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45640c6bb0 con 0x7f456406fc10 2023-12-19T18:16:54.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.687+0000 7f45617fa640 1 -- 172.21.15.27:0/991189095 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f455401a400 con 0x7f456406fc10 2023-12-19T18:16:54.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.688+0000 7f45617fa640 1 --2- 172.21.15.27:0/991189095 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f454406a460 0x7f454406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:54.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.688+0000 7f45617fa640 1 -- 172.21.15.27:0/991189095 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f4554088800 con 0x7f456406fc10 2023-12-19T18:16:54.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.689+0000 7f455bfff640 1 --2- 172.21.15.27:0/991189095 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f454406a460 0x7f454406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:54.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.690+0000 7f455bfff640 1 --2- 172.21.15.27:0/991189095 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f454406a460 0x7f454406c920 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f45640b05d0 tx=0x7f454c006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:54.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.692+0000 7f45617fa640 1 -- 172.21.15.27:0/991189095 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4554053e10 con 0x7f456406fc10 2023-12-19T18:16:54.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.903+0000 7f456a841640 1 -- 172.21.15.27:0/991189095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f45641f3270 con 0x7f456406fc10 2023-12-19T18:16:54.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.904+0000 7f45617fa640 1 -- 172.21.15.27:0/991189095 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v38) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4554057430 con 0x7f456406fc10 2023-12-19T18:16:54.904 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:16:54.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.906+0000 7f456a841640 1 -- 172.21.15.27:0/991189095 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f454406a460 msgr2=0x7f454406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:54.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.907+0000 7f456a841640 1 --2- 172.21.15.27:0/991189095 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f454406a460 0x7f454406c920 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f45640b05d0 tx=0x7f454c006c90 comp rx=0 tx=0).stop 2023-12-19T18:16:54.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.907+0000 7f456a841640 1 -- 172.21.15.27:0/991189095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f456406fc10 msgr2=0x7f456414e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:54.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.907+0000 7f456a841640 1 --2- 172.21.15.27:0/991189095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f456406fc10 0x7f456414e050 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f455400b3f0 tx=0x7f455400b8b0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.907+0000 7f456a841640 1 -- 172.21.15.27:0/991189095 shutdown_connections 2023-12-19T18:16:54.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.907+0000 7f456a841640 1 --2- 172.21.15.27:0/991189095 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f454406a460 0x7f454406c920 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.907+0000 7f456a841640 1 --2- 172.21.15.27:0/991189095 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f456415adf0 0x7f45640c5c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.907+0000 7f456a841640 1 --2- 172.21.15.27:0/991189095 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4564070c00 0x7f45641f35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.907+0000 7f456a841640 1 --2- 172.21.15.27:0/991189095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f456406fc10 0x7f456414e050 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:54.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.907+0000 7f456a841640 1 -- 172.21.15.27:0/991189095 >> 172.21.15.27:0/991189095 conn(0x7f45640ceb50 msgr2=0x7f456415e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:54.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.907+0000 7f456a841640 1 -- 172.21.15.27:0/991189095 shutdown_connections 2023-12-19T18:16:54.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:54.907+0000 7f456a841640 1 -- 172.21.15.27:0/991189095 wait complete. 2023-12-19T18:16:54.925 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-19T18:16:54.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:54.925 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-19T18:16:55.026 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[86432]: fuse finished with error 0 and tester_r 0 2023-12-19T18:16:55.029 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:16:55.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:55.030 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-19T18:16:55.155 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:16:55.155 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:55.223 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:16:55.223 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:16:55.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:55.223 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-19T18:16:55.283 INFO:teuthology.orchestra.run.smithi027.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-19T18:16:55.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:16:55.284 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:16:55.284 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:16:55.344 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:16:55.344 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:16:55.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:16:55.344 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-19T18:16:55.398 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-19T18:16:55.399 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:16:55.400 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:55.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.638+0000 7f2079251640 1 Processor -- start 2023-12-19T18:16:55.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.639+0000 7f2079251640 1 -- start start 2023-12-19T18:16:55.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.640+0000 7f2079251640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207415af70 0x7f20740ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:55.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.640+0000 7f2079251640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f20740adc20 0x7f207406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:55.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.640+0000 7f2079251640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20740cb850 con 0x7f20740cbce0 2023-12-19T18:16:55.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.640+0000 7f2079251640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f207415b350 con 0x7f20740adc20 2023-12-19T18:16:55.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.640+0000 7f2079251640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20740cac40 con 0x7f207415af70 2023-12-19T18:16:55.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.640+0000 7f207259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f20740cbce0 0x7f20740ad510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55202/0 (socket says 172.21.15.27:55202) 2023-12-19T18:16:55.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.640+0000 7f207259c640 1 -- 172.21.15.27:0/681356128 learned_addr learned my addr 172.21.15.27:0/681356128 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:55.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2072d9d640 1 --2- 172.21.15.27:0/681356128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207415af70 0x7f20740ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:55.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f207359e640 1 --2- 172.21.15.27:0/681356128 >> v2:172.21.15.192:3300/0 conn(0x7f20740adc20 0x7f207406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251413252 0 0) 0x7f20740cb850 con 0x7f20740cbce0 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2060003520 con 0x7f20740cbce0 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2083394034 0 0) 0x7f2060003520 con 0x7f20740cbce0 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207415af70 msgr2=0x7f20740ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 --2- 172.21.15.27:0/681356128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207415af70 0x7f20740ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 >> v2:172.21.15.192:3300/0 conn(0x7f20740adc20 msgr2=0x7f207406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 --2- 172.21.15.27:0/681356128 >> v2:172.21.15.192:3300/0 conn(0x7f20740adc20 0x7f207406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20740caf20 con 0x7f20740cbce0 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f205c001670 con 0x7f20740cbce0 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 >> v1:172.21.15.27:6789/0 conn(0x7f20740cbce0 legacy=0x7f20740ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 --2- 172.21.15.27:0/681356128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2060003970 0x7f2060003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 --2- 172.21.15.27:0/681356128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20600047c0 0x7f2060004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 --2- 172.21.15.27:0/681356128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207415af70 0x7f2060005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:55.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207415b350 con 0x7f2060003970 2023-12-19T18:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20740cac40 con 0x7f20600047c0 2023-12-19T18:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.641+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f206000b9d0 con 0x7f207415af70 2023-12-19T18:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.642+0000 7f2072d9d640 1 --2- 172.21.15.27:0/681356128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2060003970 0x7f2060003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.642+0000 7f207259c640 1 --2- 172.21.15.27:0/681356128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20600047c0 0x7f2060004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.642+0000 7f2072d9d640 1 -- 172.21.15.27:0/681356128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207415af70 msgr2=0x7f2060005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.642+0000 7f2072d9d640 1 --2- 172.21.15.27:0/681356128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207415af70 0x7f2060005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.642+0000 7f2072d9d640 1 -- 172.21.15.27:0/681356128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20600047c0 msgr2=0x7f2060004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.642+0000 7f2072d9d640 1 --2- 172.21.15.27:0/681356128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20600047c0 0x7f2060004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.642+0000 7f2072d9d640 1 -- 172.21.15.27:0/681356128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2060003520 con 0x7f2060003970 2023-12-19T18:16:55.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.642+0000 7f207259c640 1 --2- 172.21.15.27:0/681356128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20600047c0 0x7f2060004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:55.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.643+0000 7f2072d9d640 1 --2- 172.21.15.27:0/681356128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2060003970 0x7f2060003d90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f206400ba30 tx=0x7f20600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1ad4f668bbe312a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:55.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.643+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064032680 con 0x7f2060003970 2023-12-19T18:16:55.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.644+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20640327e0 con 0x7f2060003970 2023-12-19T18:16:55.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.644+0000 7f2071d9b640 1 -- 172.21.15.27:0/681356128 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064032b70 con 0x7f2060003970 2023-12-19T18:16:55.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.644+0000 7f2079251640 1 -- 172.21.15.27:0/681356128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2060003970 msgr2=0x7f2060003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:55.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.644+0000 7f2079251640 1 --2- 172.21.15.27:0/681356128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2060003970 0x7f2060003d90 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f206400ba30 tx=0x7f20600057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:55.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.644+0000 7f207359e640 1 -- 172.21.15.27:0/681356128 reap_dead start 2023-12-19T18:16:55.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.644+0000 7f2079251640 1 -- 172.21.15.27:0/681356128 shutdown_connections 2023-12-19T18:16:55.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.644+0000 7f2079251640 1 -- 172.21.15.27:0/681356128 >> 172.21.15.27:0/681356128 conn(0x7f207406a330 msgr2=0x7f20740b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:55.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.644+0000 7f2079251640 1 -- 172.21.15.27:0/681356128 shutdown_connections 2023-12-19T18:16:55.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.644+0000 7f2079251640 1 -- 172.21.15.27:0/681356128 wait complete. 2023-12-19T18:16:55.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.646+0000 7f2079251640 1 Processor -- start 2023-12-19T18:16:55.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.646+0000 7f2079251640 1 -- start start 2023-12-19T18:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.647+0000 7f2079251640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2060003970 0x7f207414f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.647+0000 7f2079251640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207415cab0 0x7f20740c5c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f2079251640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20740c2b60 0x7f20740cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f2079251640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f207414f5b0 con 0x7f2060003970 2023-12-19T18:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f2079251640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20740c6150 con 0x7f20740c2b60 2023-12-19T18:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f2079251640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20740c6c90 con 0x7f207415cab0 2023-12-19T18:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f2072d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2060003970 0x7f207414f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f2072d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2060003970 0x7f207414f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41246/0 (socket says 172.21.15.27:41246) 2023-12-19T18:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f2072d9d640 1 -- 172.21.15.27:0/3922491426 learned_addr learned my addr 172.21.15.27:0/3922491426 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f207259c640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207415cab0 0x7f20740c5c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f207359e640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20740c2b60 0x7f20740cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f2072d9d640 1 -- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207415cab0 msgr2=0x7f20740c5c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f2072d9d640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207415cab0 0x7f20740c5c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f2072d9d640 1 -- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20740c2b60 msgr2=0x7f20740cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f2072d9d640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20740c2b60 0x7f20740cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.648+0000 7f2072d9d640 1 -- 172.21.15.27:0/3922491426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20740c63f0 con 0x7f2060003970 2023-12-19T18:16:55.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.649+0000 7f2072d9d640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2060003970 0x7f207414f070 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f206402bf00 tx=0x7f2064004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:55.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.649+0000 7f206b7fe640 1 -- 172.21.15.27:0/3922491426 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20640308a0 con 0x7f2060003970 2023-12-19T18:16:55.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.649+0000 7f206b7fe640 1 -- 172.21.15.27:0/3922491426 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2064030a20 con 0x7f2060003970 2023-12-19T18:16:55.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.649+0000 7f2079251640 1 -- 172.21.15.27:0/3922491426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20740c66d0 con 0x7f2060003970 2023-12-19T18:16:55.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.649+0000 7f206b7fe640 1 -- 172.21.15.27:0/3922491426 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2064030dd0 con 0x7f2060003970 2023-12-19T18:16:55.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.649+0000 7f2079251640 1 -- 172.21.15.27:0/3922491426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207415b5c0 con 0x7f2060003970 2023-12-19T18:16:55.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.650+0000 7f2079251640 1 -- 172.21.15.27:0/3922491426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2074068e90 con 0x7f2060003970 2023-12-19T18:16:55.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.651+0000 7f206b7fe640 1 -- 172.21.15.27:0/3922491426 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f206403b420 con 0x7f2060003970 2023-12-19T18:16:55.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.652+0000 7f206b7fe640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f205406a430 0x7f205406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:55.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.652+0000 7f206b7fe640 1 -- 172.21.15.27:0/3922491426 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f20640aaeb0 con 0x7f2060003970 2023-12-19T18:16:55.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.653+0000 7f207259c640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f205406a430 0x7f205406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:55.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.654+0000 7f207259c640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f205406a430 0x7f205406c8f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f205c000c00 tx=0x7f205c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:55.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.657+0000 7f206b7fe640 1 -- 172.21.15.27:0/3922491426 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2064076410 con 0x7f2060003970 2023-12-19T18:16:55.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.893+0000 7f2079251640 1 -- 172.21.15.27:0/3922491426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f20740cf960 con 0x7f2060003970 2023-12-19T18:16:55.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.895+0000 7f206b7fe640 1 -- 172.21.15.27:0/3922491426 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f2064079a30 con 0x7f2060003970 2023-12-19T18:16:55.895 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3178168231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3178168231},{"type":"v1","addr":"172.21.15.192:6837","nonce":3178168231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3961149394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3961149394},{"type":"v1","addr":"172.21.15.192:6836","nonce":3961149394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5190,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/752784926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":752784926},{"type":"v1","addr":"172.21.15.27:6833","nonce":752784926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:25.779852+0000","modified":"2023-12-19T18:16:29.303970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/3463287343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3463287343},{"type":"v1","addr":"172.21.15.27:6835","nonce":3463287343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T18:16:55.896 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 15 2023-12-19T18:16:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.898+0000 7f2079251640 1 -- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f205406a430 msgr2=0x7f205406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.898+0000 7f2079251640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f205406a430 0x7f205406c8f0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f205c000c00 tx=0x7f205c009040 comp rx=0 tx=0).stop 2023-12-19T18:16:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.898+0000 7f2079251640 1 -- 172.21.15.27:0/3922491426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2060003970 msgr2=0x7f207414f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.898+0000 7f2079251640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2060003970 0x7f207414f070 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f206402bf00 tx=0x7f2064004730 comp rx=0 tx=0).stop 2023-12-19T18:16:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.898+0000 7f2079251640 1 -- 172.21.15.27:0/3922491426 shutdown_connections 2023-12-19T18:16:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.898+0000 7f2079251640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f205406a430 0x7f205406c8f0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.898+0000 7f2079251640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f20740c2b60 0x7f20740cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.898+0000 7f2079251640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f207415cab0 0x7f20740c5c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:55.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.898+0000 7f2079251640 1 --2- 172.21.15.27:0/3922491426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2060003970 0x7f207414f070 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:55.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.898+0000 7f2079251640 1 -- 172.21.15.27:0/3922491426 >> 172.21.15.27:0/3922491426 conn(0x7f207406a330 msgr2=0x7f2074070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:55.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.898+0000 7f2079251640 1 -- 172.21.15.27:0/3922491426 shutdown_connections 2023-12-19T18:16:55.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:55.898+0000 7f2079251640 1 -- 172.21.15.27:0/3922491426 wait complete. 2023-12-19T18:16:55.918 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:56.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.141+0000 7f552b277640 1 Processor -- start 2023-12-19T18:16:56.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.141+0000 7f552b277640 1 -- start start 2023-12-19T18:16:56.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.143+0000 7f552b277640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55240cbc60 0x7f55240cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.143+0000 7f552b277640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55240ce980 con 0x7f55240ce520 2023-12-19T18:16:56.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.143+0000 7f552b277640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55240caba0 con 0x7f5524164a80 2023-12-19T18:16:56.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.143+0000 7f552b277640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5524164e60 con 0x7f55240cbc60 2023-12-19T18:16:56.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.143+0000 7f5528812640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f55240ce520 0x7f55240cb550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55210/0 (socket says 172.21.15.27:55210) 2023-12-19T18:16:56.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.143+0000 7f5528812640 1 -- 172.21.15.27:0/1761645557 learned_addr learned my addr 172.21.15.27:0/1761645557 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.144+0000 7f5529814640 1 --2- 172.21.15.27:0/1761645557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55240cbc60 0x7f55240cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.144+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1984082015 0 0) 0x7f55240ce980 con 0x7f55240ce520 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.144+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5510003520 con 0x7f55240ce520 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.144+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2257096836 0 0) 0x7f5510003520 con 0x7f55240ce520 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.144+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55240cbc60 msgr2=0x7f55240cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.144+0000 7f551bfff640 1 --2- 172.21.15.27:0/1761645557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55240cbc60 0x7f55240cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.144+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 >> v1:172.21.15.192:6789/0 conn(0x7f5524164a80 legacy=0x7f55240c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.144+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f552406c2d0 con 0x7f55240ce520 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.144+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f5514001670 con 0x7f55240ce520 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.144+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 >> v1:172.21.15.27:6789/0 conn(0x7f55240ce520 legacy=0x7f55240cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.144+0000 7f551bfff640 1 --2- 172.21.15.27:0/1761645557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55100039a0 0x7f5510003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.144+0000 7f551bfff640 1 --2- 172.21.15.27:0/1761645557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55240cbc60 0x7f55100047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.144+0000 7f551bfff640 1 --2- 172.21.15.27:0/1761645557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5510004d00 0x7f5510005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.145+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524164e60 con 0x7f55100039a0 2023-12-19T18:16:56.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.145+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551000bae0 con 0x7f5510004d00 2023-12-19T18:16:56.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.145+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551000bca0 con 0x7f55240cbc60 2023-12-19T18:16:56.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.145+0000 7f5529013640 1 --2- 172.21.15.27:0/1761645557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55100039a0 0x7f5510003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.145+0000 7f5529814640 1 --2- 172.21.15.27:0/1761645557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5510004d00 0x7f5510005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.145+0000 7f5528812640 1 --2- 172.21.15.27:0/1761645557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55240cbc60 0x7f55100047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.145+0000 7f5529013640 1 -- 172.21.15.27:0/1761645557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55240cbc60 msgr2=0x7f55100047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:56.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.145+0000 7f5529013640 1 --2- 172.21.15.27:0/1761645557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55240cbc60 0x7f55100047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.145+0000 7f5529013640 1 -- 172.21.15.27:0/1761645557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5510004d00 msgr2=0x7f5510005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:56.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.145+0000 7f5529013640 1 --2- 172.21.15.27:0/1761645557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5510004d00 0x7f5510005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.145+0000 7f5529013640 1 -- 172.21.15.27:0/1761645557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5510003520 con 0x7f55100039a0 2023-12-19T18:16:56.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.145+0000 7f5529814640 1 --2- 172.21.15.27:0/1761645557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5510004d00 0x7f5510005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:56.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.146+0000 7f5529013640 1 --2- 172.21.15.27:0/1761645557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55100039a0 0x7f5510003dc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5520001750 tx=0x7f55100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa921d7793536732 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:56.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.147+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55200052d0 con 0x7f55100039a0 2023-12-19T18:16:56.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.147+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f552002e050 con 0x7f55100039a0 2023-12-19T18:16:56.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.147+0000 7f551bfff640 1 -- 172.21.15.27:0/1761645557 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55200035f0 con 0x7f55100039a0 2023-12-19T18:16:56.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.147+0000 7f552b277640 1 -- 172.21.15.27:0/1761645557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55100039a0 msgr2=0x7f5510003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.147+0000 7f552b277640 1 --2- 172.21.15.27:0/1761645557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55100039a0 0x7f5510003dc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5520001750 tx=0x7f55100057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.147+0000 7f5529814640 1 -- 172.21.15.27:0/1761645557 reap_dead start 2023-12-19T18:16:56.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.147+0000 7f552b277640 1 -- 172.21.15.27:0/1761645557 shutdown_connections 2023-12-19T18:16:56.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.147+0000 7f552b277640 1 -- 172.21.15.27:0/1761645557 >> 172.21.15.27:0/1761645557 conn(0x7f552406a330 msgr2=0x7f55240b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:56.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.148+0000 7f552b277640 1 -- 172.21.15.27:0/1761645557 shutdown_connections 2023-12-19T18:16:56.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.148+0000 7f552b277640 1 -- 172.21.15.27:0/1761645557 wait complete. 2023-12-19T18:16:56.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.149+0000 7f552b277640 1 Processor -- start 2023-12-19T18:16:56.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.150+0000 7f552b277640 1 -- start start 2023-12-19T18:16:56.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f552b277640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55100039a0 0x7f5524158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f552b277640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55240ae600 0x7f55240aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f552b277640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55240c26e0 0x7f55240c2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f552b277640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55241f6ef0 con 0x7f55240ae600 2023-12-19T18:16:56.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f552b277640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55240aefc0 con 0x7f55240c26e0 2023-12-19T18:16:56.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f552b277640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55240c3cd0 con 0x7f55100039a0 2023-12-19T18:16:56.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f5528812640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55240ae600 0x7f55240aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f5528812640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55240ae600 0x7f55240aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41254/0 (socket says 172.21.15.27:41254) 2023-12-19T18:16:56.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f5528812640 1 -- 172.21.15.27:0/2436063684 learned_addr learned my addr 172.21.15.27:0/2436063684 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:56.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f5528812640 1 -- 172.21.15.27:0/2436063684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55100039a0 msgr2=0x7f5524158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f5528812640 1 --2- 172.21.15.27:0/2436063684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55100039a0 0x7f5524158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f5528812640 1 -- 172.21.15.27:0/2436063684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55240c26e0 msgr2=0x7f55240c2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f5528812640 1 --2- 172.21.15.27:0/2436063684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55240c26e0 0x7f55240c2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.151+0000 7f5528812640 1 -- 172.21.15.27:0/2436063684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55240c3430 con 0x7f55240ae600 2023-12-19T18:16:56.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.152+0000 7f5528812640 1 --2- 172.21.15.27:0/2436063684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55240ae600 0x7f55240aea80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f551400d720 tx=0x7f551400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:56.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.152+0000 7f5519ffb640 1 -- 172.21.15.27:0/2436063684 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55140074e0 con 0x7f55240ae600 2023-12-19T18:16:56.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.152+0000 7f5519ffb640 1 -- 172.21.15.27:0/2436063684 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5514007640 con 0x7f55240ae600 2023-12-19T18:16:56.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.152+0000 7f5519ffb640 1 -- 172.21.15.27:0/2436063684 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5514004a00 con 0x7f55240ae600 2023-12-19T18:16:56.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.152+0000 7f552b277640 1 -- 172.21.15.27:0/2436063684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55240c3710 con 0x7f55240ae600 2023-12-19T18:16:56.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.152+0000 7f552b277640 1 -- 172.21.15.27:0/2436063684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5524159400 con 0x7f55240ae600 2023-12-19T18:16:56.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.153+0000 7f552b277640 1 -- 172.21.15.27:0/2436063684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55240ae170 con 0x7f55240ae600 2023-12-19T18:16:56.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.154+0000 7f5519ffb640 1 -- 172.21.15.27:0/2436063684 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5514004b60 con 0x7f55240ae600 2023-12-19T18:16:56.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.156+0000 7f5519ffb640 1 --2- 172.21.15.27:0/2436063684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f550806a430 0x7f550806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.156+0000 7f5519ffb640 1 -- 172.21.15.27:0/2436063684 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f55140882d0 con 0x7f55240ae600 2023-12-19T18:16:56.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.156+0000 7f5529013640 1 --2- 172.21.15.27:0/2436063684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f550806a430 0x7f550806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.158+0000 7f5529013640 1 --2- 172.21.15.27:0/2436063684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f550806a430 0x7f550806c8f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5520033000 tx=0x7f552002f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:56.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.160+0000 7f5519ffb640 1 -- 172.21.15.27:0/2436063684 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f55140537a0 con 0x7f55240ae600 2023-12-19T18:16:56.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.357+0000 7f552b277640 1 -- 172.21.15.27:0/2436063684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5524068e90 con 0x7f55240ae600 2023-12-19T18:16:56.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.359+0000 7f5519ffb640 1 -- 172.21.15.27:0/2436063684 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16291 (secure 0 0 0) 0x7f5514056dc0 con 0x7f55240ae600 2023-12-19T18:16:56.359 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:56.359 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:16:45.666993+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:23.925853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:24.827898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]},{"pgid":"5.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:56.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.362+0000 7f552b277640 1 -- 172.21.15.27:0/2436063684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f550806a430 msgr2=0x7f550806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.362+0000 7f552b277640 1 --2- 172.21.15.27:0/2436063684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f550806a430 0x7f550806c8f0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f5520033000 tx=0x7f552002f000 comp rx=0 tx=0).stop 2023-12-19T18:16:56.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.362+0000 7f552b277640 1 -- 172.21.15.27:0/2436063684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55240ae600 msgr2=0x7f55240aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.362+0000 7f552b277640 1 --2- 172.21.15.27:0/2436063684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55240ae600 0x7f55240aea80 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f551400d720 tx=0x7f551400dbe0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.362+0000 7f552b277640 1 -- 172.21.15.27:0/2436063684 shutdown_connections 2023-12-19T18:16:56.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.362+0000 7f552b277640 1 --2- 172.21.15.27:0/2436063684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f550806a430 0x7f550806c8f0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.362+0000 7f552b277640 1 --2- 172.21.15.27:0/2436063684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55240c26e0 0x7f55240c2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.362+0000 7f552b277640 1 --2- 172.21.15.27:0/2436063684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55240ae600 0x7f55240aea80 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.362+0000 7f552b277640 1 --2- 172.21.15.27:0/2436063684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55100039a0 0x7f5524158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.362+0000 7f552b277640 1 -- 172.21.15.27:0/2436063684 >> 172.21.15.27:0/2436063684 conn(0x7f552406a330 msgr2=0x7f55240af430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:56.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.362+0000 7f552b277640 1 -- 172.21.15.27:0/2436063684 shutdown_connections 2023-12-19T18:16:56.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.362+0000 7f552b277640 1 -- 172.21.15.27:0/2436063684 wait complete. 2023-12-19T18:16:56.381 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:16:23.925853+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:16:24.827898+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:16:56.381 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:16:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.602+0000 7fb14cac4640 1 Processor -- start 2023-12-19T18:16:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.603+0000 7fb14cac4640 1 -- start start 2023-12-19T18:16:56.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.604+0000 7fb14cac4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1480ca280 0x7fb14815be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.604+0000 7fb14cac4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14815c380 con 0x7fb14815b130 2023-12-19T18:16:56.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.604+0000 7fb14cac4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb14806e8d0 con 0x7fb148070ae0 2023-12-19T18:16:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.604+0000 7fb14cac4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb14806ee90 con 0x7fb1480ca280 2023-12-19T18:16:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.605+0000 7fb145d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb14815b130 0x7fb14806dff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55226/0 (socket says 172.21.15.27:55226) 2023-12-19T18:16:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.605+0000 7fb145d9b640 1 -- 172.21.15.27:0/2667967137 learned_addr learned my addr 172.21.15.27:0/2667967137 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.605+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2081529497 0 0) 0x7fb14815c380 con 0x7fb14815b130 2023-12-19T18:16:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.605+0000 7fb146d9d640 1 --2- 172.21.15.27:0/2667967137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1480ca280 0x7fb14815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.605+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb134003520 con 0x7fb14815b130 2023-12-19T18:16:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.605+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2982960172 0 0) 0x7fb134003520 con 0x7fb14815b130 2023-12-19T18:16:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.605+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1480ca280 msgr2=0x7fb14815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.605+0000 7fb14559a640 1 --2- 172.21.15.27:0/2667967137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1480ca280 0x7fb14815be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.605+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 >> v1:172.21.15.192:6789/0 conn(0x7fb148070ae0 legacy=0x7fb148070ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.605+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14806ebb0 con 0x7fb14815b130 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.605+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fb130001670 con 0x7fb14815b130 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 >> v1:172.21.15.27:6789/0 conn(0x7fb14815b130 legacy=0x7fb14806dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb14559a640 1 --2- 172.21.15.27:0/2667967137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1480ca280 0x7fb1340039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb14559a640 1 --2- 172.21.15.27:0/2667967137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb134004020 0x7fb1340047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb14559a640 1 --2- 172.21.15.27:0/2667967137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb134004d00 0x7fb134005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb14806ee90 con 0x7fb134004020 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb13400bbc0 con 0x7fb134004d00 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13400bd80 con 0x7fb1480ca280 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb145d9b640 1 --2- 172.21.15.27:0/2667967137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb134004020 0x7fb1340047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb146d9d640 1 --2- 172.21.15.27:0/2667967137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb134004d00 0x7fb134005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb14659c640 1 --2- 172.21.15.27:0/2667967137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1480ca280 0x7fb1340039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb145d9b640 1 -- 172.21.15.27:0/2667967137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1480ca280 msgr2=0x7fb1340039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb145d9b640 1 --2- 172.21.15.27:0/2667967137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1480ca280 0x7fb1340039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb145d9b640 1 -- 172.21.15.27:0/2667967137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb134004d00 msgr2=0x7fb134005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb145d9b640 1 --2- 172.21.15.27:0/2667967137 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb134004d00 0x7fb134005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.606+0000 7fb145d9b640 1 -- 172.21.15.27:0/2667967137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb134003520 con 0x7fb134004020 2023-12-19T18:16:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.607+0000 7fb14659c640 1 --2- 172.21.15.27:0/2667967137 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1480ca280 0x7fb1340039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.608+0000 7fb145d9b640 1 --2- 172.21.15.27:0/2667967137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb134004020 0x7fb1340047c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb13000c0e0 tx=0x7fb1340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ea521bd5c001a5d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.608+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb130005ad0 con 0x7fb134004020 2023-12-19T18:16:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.608+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb130005c30 con 0x7fb134004020 2023-12-19T18:16:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.608+0000 7fb14559a640 1 -- 172.21.15.27:0/2667967137 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13002ca30 con 0x7fb134004020 2023-12-19T18:16:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.609+0000 7fb14cac4640 1 -- 172.21.15.27:0/2667967137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb134004020 msgr2=0x7fb1340047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.609+0000 7fb14cac4640 1 --2- 172.21.15.27:0/2667967137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb134004020 0x7fb1340047c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fb13000c0e0 tx=0x7fb1340057f0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.609+0000 7fb146d9d640 1 -- 172.21.15.27:0/2667967137 reap_dead start 2023-12-19T18:16:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.609+0000 7fb14cac4640 1 -- 172.21.15.27:0/2667967137 shutdown_connections 2023-12-19T18:16:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.609+0000 7fb14cac4640 1 -- 172.21.15.27:0/2667967137 >> 172.21.15.27:0/2667967137 conn(0x7fb1480cc6d0 msgr2=0x7fb1480ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.609+0000 7fb14cac4640 1 -- 172.21.15.27:0/2667967137 shutdown_connections 2023-12-19T18:16:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.609+0000 7fb14cac4640 1 -- 172.21.15.27:0/2667967137 wait complete. 2023-12-19T18:16:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.611+0000 7fb14cac4640 1 Processor -- start 2023-12-19T18:16:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.611+0000 7fb14cac4640 1 -- start start 2023-12-19T18:16:56.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.612+0000 7fb14cac4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb134004020 0x7fb14814ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.612+0000 7fb14cac4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1481ee850 0x7fb148155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.612+0000 7fb14cac4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1480c36f0 0x7fb1481562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.612+0000 7fb14cac4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1481522c0 con 0x7fb1480c36f0 2023-12-19T18:16:56.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.612+0000 7fb14cac4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1481525a0 con 0x7fb1481ee850 2023-12-19T18:16:56.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.612+0000 7fb14cac4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1480c3c20 con 0x7fb134004020 2023-12-19T18:16:56.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.612+0000 7fb146d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1480c36f0 0x7fb1481562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.612+0000 7fb146d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1480c36f0 0x7fb1481562d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41280/0 (socket says 172.21.15.27:41280) 2023-12-19T18:16:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.612+0000 7fb146d9d640 1 -- 172.21.15.27:0/3393391050 learned_addr learned my addr 172.21.15.27:0/3393391050 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.613+0000 7fb14659c640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb134004020 0x7fb14814ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.613+0000 7fb145d9b640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1481ee850 0x7fb148155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.613+0000 7fb146d9d640 1 -- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb134004020 msgr2=0x7fb14814ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.613+0000 7fb146d9d640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb134004020 0x7fb14814ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.613+0000 7fb146d9d640 1 -- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1481ee850 msgr2=0x7fb148155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.613+0000 7fb146d9d640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1481ee850 0x7fb148155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.613+0000 7fb146d9d640 1 -- 172.21.15.27:0/3393391050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1480c3f00 con 0x7fb1480c36f0 2023-12-19T18:16:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.613+0000 7fb146d9d640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1480c36f0 0x7fb1481562d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb14806c070 tx=0x7fb14000f920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.614+0000 7fb13effd640 1 -- 172.21.15.27:0/3393391050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb140004440 con 0x7fb1480c36f0 2023-12-19T18:16:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.614+0000 7fb13effd640 1 -- 172.21.15.27:0/3393391050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb140003610 con 0x7fb1480c36f0 2023-12-19T18:16:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.614+0000 7fb13effd640 1 -- 172.21.15.27:0/3393391050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1400039c0 con 0x7fb1480c36f0 2023-12-19T18:16:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.614+0000 7fb14cac4640 1 -- 172.21.15.27:0/3393391050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1480c41e0 con 0x7fb1480c36f0 2023-12-19T18:16:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.614+0000 7fb14cac4640 1 -- 172.21.15.27:0/3393391050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1480cf800 con 0x7fb1480c36f0 2023-12-19T18:16:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.615+0000 7fb14cac4640 1 -- 172.21.15.27:0/3393391050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1481526e0 con 0x7fb1480c36f0 2023-12-19T18:16:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.616+0000 7fb13effd640 1 -- 172.21.15.27:0/3393391050 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb140003b20 con 0x7fb1480c36f0 2023-12-19T18:16:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.617+0000 7fb13effd640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb12806a430 0x7fb12806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.617+0000 7fb13effd640 1 -- 172.21.15.27:0/3393391050 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fb1400886f0 con 0x7fb1480c36f0 2023-12-19T18:16:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.618+0000 7fb14659c640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb12806a430 0x7fb12806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.619+0000 7fb14659c640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb12806a430 0x7fb12806c8f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb138000f80 tx=0x7fb138009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.622+0000 7fb13effd640 1 -- 172.21.15.27:0/3393391050 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb1481526e0 con 0x7fb1480c36f0 2023-12-19T18:16:56.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.813+0000 7fb14cac4640 1 -- 172.21.15.27:0/3393391050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1481526e0 con 0x7fb1480c36f0 2023-12-19T18:16:56.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.814+0000 7fb13effd640 1 -- 172.21.15.27:0/3393391050 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb1481526e0 con 0x7fb1480c36f0 2023-12-19T18:16:56.814 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:16:56.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.817+0000 7fb14cac4640 1 -- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb12806a430 msgr2=0x7fb12806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.817+0000 7fb14cac4640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb12806a430 0x7fb12806c8f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb138000f80 tx=0x7fb138009040 comp rx=0 tx=0).stop 2023-12-19T18:16:56.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.817+0000 7fb14cac4640 1 -- 172.21.15.27:0/3393391050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1480c36f0 msgr2=0x7fb1481562d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:56.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.817+0000 7fb14cac4640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1480c36f0 0x7fb1481562d0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fb14806c070 tx=0x7fb14000f920 comp rx=0 tx=0).stop 2023-12-19T18:16:56.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.817+0000 7fb14cac4640 1 -- 172.21.15.27:0/3393391050 shutdown_connections 2023-12-19T18:16:56.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.817+0000 7fb14cac4640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb12806a430 0x7fb12806c8f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.817+0000 7fb14cac4640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1480c36f0 0x7fb1481562d0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.817+0000 7fb14cac4640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb1481ee850 0x7fb148155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.817+0000 7fb14cac4640 1 --2- 172.21.15.27:0/3393391050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb134004020 0x7fb14814ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:56.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.817+0000 7fb14cac4640 1 -- 172.21.15.27:0/3393391050 >> 172.21.15.27:0/3393391050 conn(0x7fb1480cc6d0 msgr2=0x7fb1480c2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:56.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.817+0000 7fb14cac4640 1 -- 172.21.15.27:0/3393391050 shutdown_connections 2023-12-19T18:16:56.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:56.817+0000 7fb14cac4640 1 -- 172.21.15.27:0/3393391050 wait complete. 2023-12-19T18:16:56.835 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:16:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.067+0000 7f5c759c6640 1 Processor -- start 2023-12-19T18:16:57.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.068+0000 7f5c759c6640 1 -- start start 2023-12-19T18:16:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.069+0000 7f5c759c6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c7015cf80 0x7f5c700d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.070+0000 7f5c759c6640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5c700b85e0 0x7f5c700cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.070+0000 7f5c759c6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b8a60 0x7f5c700cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.070+0000 7f5c759c6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c700d0e10 con 0x7f5c7015cf80 2023-12-19T18:16:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.070+0000 7f5c759c6640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c700d1c70 con 0x7f5c700b85e0 2023-12-19T18:16:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.070+0000 7f5c759c6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7015d360 con 0x7f5c700b8a60 2023-12-19T18:16:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.070+0000 7f5c749c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c7015cf80 0x7f5c700d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.070+0000 7f5c749c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c7015cf80 0x7f5c700d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41296/0 (socket says 172.21.15.27:41296) 2023-12-19T18:16:57.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.070+0000 7f5c749c4640 1 -- 172.21.15.27:0/184365442 learned_addr learned my addr 172.21.15.27:0/184365442 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:57.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.070+0000 7f5c751c5640 1 --2- 172.21.15.27:0/184365442 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b8a60 0x7f5c700cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.070+0000 7f5c6ffff640 1 --2- 172.21.15.27:0/184365442 >> v2:172.21.15.192:3300/0 conn(0x7f5c700b85e0 0x7f5c700cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.070+0000 7f5c749c4640 1 -- 172.21.15.27:0/184365442 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b8a60 msgr2=0x7f5c700cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:57.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.071+0000 7f5c749c4640 1 --2- 172.21.15.27:0/184365442 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b8a60 0x7f5c700cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.071+0000 7f5c749c4640 1 -- 172.21.15.27:0/184365442 >> v2:172.21.15.192:3300/0 conn(0x7f5c700b85e0 msgr2=0x7f5c700cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:57.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.071+0000 7f5c749c4640 1 --2- 172.21.15.27:0/184365442 >> v2:172.21.15.192:3300/0 conn(0x7f5c700b85e0 0x7f5c700cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.071+0000 7f5c749c4640 1 -- 172.21.15.27:0/184365442 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c700d1060 con 0x7f5c7015cf80 2023-12-19T18:16:57.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.071+0000 7f5c751c5640 1 --2- 172.21.15.27:0/184365442 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b8a60 0x7f5c700cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:57.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.071+0000 7f5c6ffff640 1 --2- 172.21.15.27:0/184365442 >> v2:172.21.15.192:3300/0 conn(0x7f5c700b85e0 0x7f5c700cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:57.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.072+0000 7f5c749c4640 1 --2- 172.21.15.27:0/184365442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c7015cf80 0x7f5c700d2100 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5c700cd270 tx=0x7f5c6002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23482b3a20fbd441 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.072+0000 7f5c6f7fe640 1 -- 172.21.15.27:0/184365442 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6002cb00 con 0x7f5c7015cf80 2023-12-19T18:16:57.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.072+0000 7f5c6f7fe640 1 -- 172.21.15.27:0/184365442 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6002cc60 con 0x7f5c7015cf80 2023-12-19T18:16:57.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.072+0000 7f5c6f7fe640 1 -- 172.21.15.27:0/184365442 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c600343f0 con 0x7f5c7015cf80 2023-12-19T18:16:57.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.073+0000 7f5c759c6640 1 -- 172.21.15.27:0/184365442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c7015cf80 msgr2=0x7f5c700d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.073+0000 7f5c759c6640 1 --2- 172.21.15.27:0/184365442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c7015cf80 0x7f5c700d2100 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f5c700cd270 tx=0x7f5c6002bf60 comp rx=0 tx=0).stop 2023-12-19T18:16:57.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.073+0000 7f5c759c6640 1 -- 172.21.15.27:0/184365442 shutdown_connections 2023-12-19T18:16:57.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.073+0000 7f5c759c6640 1 --2- 172.21.15.27:0/184365442 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b8a60 0x7f5c700cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.073+0000 7f5c759c6640 1 --2- 172.21.15.27:0/184365442 >> v2:172.21.15.192:3300/0 conn(0x7f5c700b85e0 0x7f5c700cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.073+0000 7f5c759c6640 1 --2- 172.21.15.27:0/184365442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c7015cf80 0x7f5c700d2100 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.073+0000 7f5c759c6640 1 -- 172.21.15.27:0/184365442 >> 172.21.15.27:0/184365442 conn(0x7f5c70072e20 msgr2=0x7f5c70073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:57.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.073+0000 7f5c759c6640 1 -- 172.21.15.27:0/184365442 shutdown_connections 2023-12-19T18:16:57.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.073+0000 7f5c759c6640 1 -- 172.21.15.27:0/184365442 wait complete. 2023-12-19T18:16:57.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.075+0000 7f5c759c6640 1 Processor -- start 2023-12-19T18:16:57.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.075+0000 7f5c759c6640 1 -- start start 2023-12-19T18:16:57.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.076+0000 7f5c759c6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b85e0 0x7f5c70157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.076+0000 7f5c759c6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c7015cf80 0x7f5c70160020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.076+0000 7f5c759c6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c70157e40 0x7f5c701f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.076+0000 7f5c759c6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c7015fb90 con 0x7f5c70157e40 2023-12-19T18:16:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.076+0000 7f5c759c6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c701f6150 con 0x7f5c7015cf80 2023-12-19T18:16:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.076+0000 7f5c759c6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c701f6430 con 0x7f5c700b85e0 2023-12-19T18:16:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.076+0000 7f5c751c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c70157e40 0x7f5c701f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.077+0000 7f5c751c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c70157e40 0x7f5c701f5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41306/0 (socket says 172.21.15.27:41306) 2023-12-19T18:16:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.077+0000 7f5c751c5640 1 -- 172.21.15.27:0/3110489167 learned_addr learned my addr 172.21.15.27:0/3110489167 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.077+0000 7f5c749c4640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b85e0 0x7f5c70157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.077+0000 7f5c6ffff640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c7015cf80 0x7f5c70160020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.077+0000 7f5c751c5640 1 -- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b85e0 msgr2=0x7f5c70157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.077+0000 7f5c751c5640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b85e0 0x7f5c70157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.077+0000 7f5c751c5640 1 -- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c7015cf80 msgr2=0x7f5c70160020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.077+0000 7f5c751c5640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c7015cf80 0x7f5c70160020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.077+0000 7f5c751c5640 1 -- 172.21.15.27:0/3110489167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c60009b30 con 0x7f5c70157e40 2023-12-19T18:16:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.077+0000 7f5c751c5640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c70157e40 0x7f5c701f5700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800c570 tx=0x7f5c6800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.078+0000 7f5c6d7fa640 1 -- 172.21.15.27:0/3110489167 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68011800 con 0x7f5c70157e40 2023-12-19T18:16:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.078+0000 7f5c6d7fa640 1 -- 172.21.15.27:0/3110489167 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c68011980 con 0x7f5c70157e40 2023-12-19T18:16:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.078+0000 7f5c759c6640 1 -- 172.21.15.27:0/3110489167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c7015f820 con 0x7f5c70157e40 2023-12-19T18:16:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.078+0000 7f5c6d7fa640 1 -- 172.21.15.27:0/3110489167 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68011d30 con 0x7f5c70157e40 2023-12-19T18:16:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.078+0000 7f5c759c6640 1 -- 172.21.15.27:0/3110489167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c70160560 con 0x7f5c70157e40 2023-12-19T18:16:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.079+0000 7f5c759c6640 1 -- 172.21.15.27:0/3110489167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c701607e0 con 0x7f5c70157e40 2023-12-19T18:16:57.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.080+0000 7f5c6d7fa640 1 -- 172.21.15.27:0/3110489167 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5c68018360 con 0x7f5c70157e40 2023-12-19T18:16:57.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.081+0000 7f5c6d7fa640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c5006a430 0x7f5c5006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.081+0000 7f5c6d7fa640 1 -- 172.21.15.27:0/3110489167 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f5c68088920 con 0x7f5c70157e40 2023-12-19T18:16:57.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.082+0000 7f5c749c4640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c5006a430 0x7f5c5006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.083+0000 7f5c749c4640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c5006a430 0x7f5c5006c8f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c60035000 tx=0x7f5c60030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.085+0000 7f5c6d7fa640 1 -- 172.21.15.27:0/3110489167 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5c701607e0 con 0x7f5c70157e40 2023-12-19T18:16:57.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.271+0000 7f5c759c6640 1 -- 172.21.15.27:0/3110489167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c701607e0 con 0x7f5c70157e40 2023-12-19T18:16:57.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.273+0000 7f5c6d7fa640 1 -- 172.21.15.27:0/3110489167 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v38) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c701607e0 con 0x7f5c70157e40 2023-12-19T18:16:57.273 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:16:57.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.275+0000 7f5c759c6640 1 -- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c5006a430 msgr2=0x7f5c5006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.275+0000 7f5c759c6640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c5006a430 0x7f5c5006c8f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f5c60035000 tx=0x7f5c60030000 comp rx=0 tx=0).stop 2023-12-19T18:16:57.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.275+0000 7f5c759c6640 1 -- 172.21.15.27:0/3110489167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c70157e40 msgr2=0x7f5c701f5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.275+0000 7f5c759c6640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c70157e40 0x7f5c701f5700 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5c6800c570 tx=0x7f5c6800ca30 comp rx=0 tx=0).stop 2023-12-19T18:16:57.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.275+0000 7f5c759c6640 1 -- 172.21.15.27:0/3110489167 shutdown_connections 2023-12-19T18:16:57.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.275+0000 7f5c759c6640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c5006a430 0x7f5c5006c8f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.275+0000 7f5c759c6640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c70157e40 0x7f5c701f5700 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.275+0000 7f5c759c6640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c7015cf80 0x7f5c70160020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.275+0000 7f5c759c6640 1 --2- 172.21.15.27:0/3110489167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c700b85e0 0x7f5c70157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.275+0000 7f5c759c6640 1 -- 172.21.15.27:0/3110489167 >> 172.21.15.27:0/3110489167 conn(0x7f5c70072e20 msgr2=0x7f5c700b32e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:57.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.275+0000 7f5c759c6640 1 -- 172.21.15.27:0/3110489167 shutdown_connections 2023-12-19T18:16:57.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.275+0000 7f5c759c6640 1 -- 172.21.15.27:0/3110489167 wait complete. 2023-12-19T18:16:57.292 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:16:57.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.527+0000 7eff20217640 1 Processor -- start 2023-12-19T18:16:57.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.527+0000 7eff20217640 1 -- start start 2023-12-19T18:16:57.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.529+0000 7eff20217640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7eff180c2720 0x7eff1806fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.529+0000 7eff20217640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7eff18071420 0x7eff1815ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.529+0000 7eff20217640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff1806bc80 con 0x7eff1815b1a0 2023-12-19T18:16:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.529+0000 7eff20217640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff1806d690 con 0x7eff180c2720 2023-12-19T18:16:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.529+0000 7eff20217640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff1806d0a0 con 0x7eff18071420 2023-12-19T18:16:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.529+0000 7eff1dfb3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7eff1815b1a0 0x7eff1806f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55238/0 (socket says 172.21.15.27:55238) 2023-12-19T18:16:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.529+0000 7eff1dfb3640 1 -- 172.21.15.27:0/2429254812 learned_addr learned my addr 172.21.15.27:0/2429254812 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:57.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.529+0000 7eff1d7b2640 1 --2- 172.21.15.27:0/2429254812 >> v2:172.21.15.27:3300/0 conn(0x7eff180c2720 0x7eff1806fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.530+0000 7eff1e7b4640 1 --2- 172.21.15.27:0/2429254812 >> v2:172.21.15.192:3300/0 conn(0x7eff18071420 0x7eff1815ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.530+0000 7eff1cfb1640 1 -- 172.21.15.27:0/2429254812 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821876598 0 0) 0x7eff14001110 con 0x7eff1815b1a0 2023-12-19T18:16:57.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.530+0000 7eff1cfb1640 1 -- 172.21.15.27:0/2429254812 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff04003520 con 0x7eff1815b1a0 2023-12-19T18:16:57.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.530+0000 7eff1d7b2640 1 -- 172.21.15.27:0/2429254812 >> v2:172.21.15.192:3300/0 conn(0x7eff18071420 msgr2=0x7eff1815ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:57.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.530+0000 7eff1d7b2640 1 --2- 172.21.15.27:0/2429254812 >> v2:172.21.15.192:3300/0 conn(0x7eff18071420 0x7eff1815ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.530+0000 7eff1d7b2640 1 -- 172.21.15.27:0/2429254812 >> v1:172.21.15.27:6789/0 conn(0x7eff1815b1a0 legacy=0x7eff1806f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.530+0000 7eff1d7b2640 1 -- 172.21.15.27:0/2429254812 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1806b6c0 con 0x7eff180c2720 2023-12-19T18:16:57.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.531+0000 7eff1d7b2640 1 --2- 172.21.15.27:0/2429254812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff180c2720 0x7eff1806fc30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7eff1806d600 tx=0x7eff0802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9dc248f405d4bf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.532+0000 7eff1cfb1640 1 -- 172.21.15.27:0/2429254812 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0802e070 con 0x7eff180c2720 2023-12-19T18:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.532+0000 7eff1cfb1640 1 -- 172.21.15.27:0/2429254812 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff080315a0 con 0x7eff180c2720 2023-12-19T18:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.532+0000 7eff1cfb1640 1 -- 172.21.15.27:0/2429254812 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08031930 con 0x7eff180c2720 2023-12-19T18:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.532+0000 7eff20217640 1 -- 172.21.15.27:0/2429254812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff180c2720 msgr2=0x7eff1806fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.532+0000 7eff20217640 1 --2- 172.21.15.27:0/2429254812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff180c2720 0x7eff1806fc30 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7eff1806d600 tx=0x7eff0802b840 comp rx=0 tx=0).stop 2023-12-19T18:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.532+0000 7eff20217640 1 -- 172.21.15.27:0/2429254812 shutdown_connections 2023-12-19T18:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.532+0000 7eff20217640 1 --2- 172.21.15.27:0/2429254812 >> v2:172.21.15.192:3300/0 conn(0x7eff18071420 0x7eff1815ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.532+0000 7eff20217640 1 --2- 172.21.15.27:0/2429254812 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff180c2720 0x7eff1806fc30 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.532+0000 7eff20217640 1 -- 172.21.15.27:0/2429254812 >> 172.21.15.27:0/2429254812 conn(0x7eff180cebd0 msgr2=0x7eff180cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.532+0000 7eff20217640 1 -- 172.21.15.27:0/2429254812 shutdown_connections 2023-12-19T18:16:57.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.532+0000 7eff20217640 1 -- 172.21.15.27:0/2429254812 wait complete. 2023-12-19T18:16:57.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.534+0000 7eff20217640 1 Processor -- start 2023-12-19T18:16:57.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.534+0000 7eff20217640 1 -- start start 2023-12-19T18:16:57.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.535+0000 7eff20217640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff18071420 0x7eff18149fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.535+0000 7eff20217640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff180c2720 0x7eff18156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.535+0000 7eff20217640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff1815b1a0 0x7eff18156770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.535+0000 7eff20217640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff18156000 con 0x7eff180c2720 2023-12-19T18:16:57.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.535+0000 7eff20217640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff180a85f0 con 0x7eff1815b1a0 2023-12-19T18:16:57.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.535+0000 7eff20217640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff18155d20 con 0x7eff18071420 2023-12-19T18:16:57.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.536+0000 7eff1d7b2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff180c2720 0x7eff18156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.536+0000 7eff1d7b2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff180c2720 0x7eff18156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41338/0 (socket says 172.21.15.27:41338) 2023-12-19T18:16:57.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.536+0000 7eff1d7b2640 1 -- 172.21.15.27:0/201700323 learned_addr learned my addr 172.21.15.27:0/201700323 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:57.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.536+0000 7eff1d7b2640 1 -- 172.21.15.27:0/201700323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff18071420 msgr2=0x7eff18149fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.536+0000 7eff1d7b2640 1 --2- 172.21.15.27:0/201700323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff18071420 0x7eff18149fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.536+0000 7eff1e7b4640 1 --2- 172.21.15.27:0/201700323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff1815b1a0 0x7eff18156770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.536+0000 7eff1d7b2640 1 -- 172.21.15.27:0/201700323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff1815b1a0 msgr2=0x7eff18156770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.536+0000 7eff1d7b2640 1 --2- 172.21.15.27:0/201700323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff1815b1a0 0x7eff18156770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.536+0000 7eff1d7b2640 1 -- 172.21.15.27:0/201700323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff08009400 con 0x7eff180c2720 2023-12-19T18:16:57.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.536+0000 7eff1d7b2640 1 --2- 172.21.15.27:0/201700323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff180c2720 0x7eff18156230 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7eff08009890 tx=0x7eff080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.537+0000 7eff0e7fc640 1 -- 172.21.15.27:0/201700323 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08032840 con 0x7eff180c2720 2023-12-19T18:16:57.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.537+0000 7eff0e7fc640 1 -- 172.21.15.27:0/201700323 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff08031f00 con 0x7eff180c2720 2023-12-19T18:16:57.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.537+0000 7eff20217640 1 -- 172.21.15.27:0/201700323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff180a8890 con 0x7eff180c2720 2023-12-19T18:16:57.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.537+0000 7eff0e7fc640 1 -- 172.21.15.27:0/201700323 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff08032be0 con 0x7eff180c2720 2023-12-19T18:16:57.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.537+0000 7eff20217640 1 -- 172.21.15.27:0/201700323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff1814a4f0 con 0x7eff180c2720 2023-12-19T18:16:57.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.538+0000 7eff20217640 1 -- 172.21.15.27:0/201700323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff180a8db0 con 0x7eff180c2720 2023-12-19T18:16:57.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.539+0000 7eff0e7fc640 1 -- 172.21.15.27:0/201700323 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7eff08032d40 con 0x7eff180c2720 2023-12-19T18:16:57.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.541+0000 7eff0e7fc640 1 --2- 172.21.15.27:0/201700323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efefc06a430 0x7efefc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:57.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.541+0000 7eff0e7fc640 1 -- 172.21.15.27:0/201700323 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7eff080aa8d0 con 0x7eff180c2720 2023-12-19T18:16:57.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.542+0000 7eff1dfb3640 1 --2- 172.21.15.27:0/201700323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efefc06a430 0x7efefc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:57.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.543+0000 7eff1dfb3640 1 --2- 172.21.15.27:0/201700323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efefc06a430 0x7efefc06c8f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7eff14005880 tx=0x7eff14001150 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:57.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.545+0000 7eff0e7fc640 1 -- 172.21.15.27:0/201700323 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7eff180a8db0 con 0x7eff180c2720 2023-12-19T18:16:57.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.745+0000 7eff20217640 1 -- 172.21.15.27:0/201700323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7eff180a8db0 con 0x7eff180c2720 2023-12-19T18:16:57.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.746+0000 7eff0e7fc640 1 -- 172.21.15.27:0/201700323 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v38) v1 ==== 100+0+11 (secure 0 0 0) 0x7eff180a8db0 con 0x7eff180c2720 2023-12-19T18:16:57.746 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:16:57.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.749+0000 7eff20217640 1 -- 172.21.15.27:0/201700323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efefc06a430 msgr2=0x7efefc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.749+0000 7eff20217640 1 --2- 172.21.15.27:0/201700323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efefc06a430 0x7efefc06c8f0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7eff14005880 tx=0x7eff14001150 comp rx=0 tx=0).stop 2023-12-19T18:16:57.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.749+0000 7eff20217640 1 -- 172.21.15.27:0/201700323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff180c2720 msgr2=0x7eff18156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:57.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.749+0000 7eff20217640 1 --2- 172.21.15.27:0/201700323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff180c2720 0x7eff18156230 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7eff08009890 tx=0x7eff080046c0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.749+0000 7eff20217640 1 -- 172.21.15.27:0/201700323 shutdown_connections 2023-12-19T18:16:57.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.749+0000 7eff20217640 1 --2- 172.21.15.27:0/201700323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efefc06a430 0x7efefc06c8f0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.749+0000 7eff20217640 1 --2- 172.21.15.27:0/201700323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff1815b1a0 0x7eff18156770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.749+0000 7eff20217640 1 --2- 172.21.15.27:0/201700323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff180c2720 0x7eff18156230 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.749+0000 7eff20217640 1 --2- 172.21.15.27:0/201700323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff18071420 0x7eff18149fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:57.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.749+0000 7eff20217640 1 -- 172.21.15.27:0/201700323 >> 172.21.15.27:0/201700323 conn(0x7eff180cebd0 msgr2=0x7eff180cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:57.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.749+0000 7eff20217640 1 -- 172.21.15.27:0/201700323 shutdown_connections 2023-12-19T18:16:57.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.749+0000 7eff20217640 1 -- 172.21.15.27:0/201700323 wait complete. 2023-12-19T18:16:57.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:57.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.999+0000 7fab20e0b640 1 Processor -- start 2023-12-19T18:16:57.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:57.999+0000 7fab20e0b640 1 -- start start 2023-12-19T18:16:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.000+0000 7fab20e0b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab1c15af70 0x7fab1c0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.000+0000 7fab20e0b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fab1c0cbce0 0x7fab1c0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.000+0000 7fab20e0b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fab1c0ada50 0x7fab1c0adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.000+0000 7fab20e0b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1c0cb850 con 0x7fab1c0ada50 2023-12-19T18:16:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.000+0000 7fab20e0b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab1c15b350 con 0x7fab1c0cbce0 2023-12-19T18:16:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.000+0000 7fab20e0b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0cac40 con 0x7fab1c15af70 2023-12-19T18:16:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.001+0000 7fab1ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fab1c0ada50 0x7fab1c0adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.001+0000 7fab1ad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fab1c0ada50 0x7fab1c0adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41340/0 (socket says 172.21.15.27:41340) 2023-12-19T18:16:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.001+0000 7fab1ad9d640 1 -- 172.21.15.27:0/3135364738 learned_addr learned my addr 172.21.15.27:0/3135364738 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.001+0000 7fab1a59c640 1 --2- 172.21.15.27:0/3135364738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab1c15af70 0x7fab1c0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.001+0000 7fab19d9b640 1 --2- 172.21.15.27:0/3135364738 >> v2:172.21.15.192:3300/0 conn(0x7fab1c0cbce0 0x7fab1c0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.001+0000 7fab1ad9d640 1 -- 172.21.15.27:0/3135364738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab1c15af70 msgr2=0x7fab1c0ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.001+0000 7fab1ad9d640 1 --2- 172.21.15.27:0/3135364738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab1c15af70 0x7fab1c0ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.001+0000 7fab1ad9d640 1 -- 172.21.15.27:0/3135364738 >> v2:172.21.15.192:3300/0 conn(0x7fab1c0cbce0 msgr2=0x7fab1c0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.001+0000 7fab1ad9d640 1 --2- 172.21.15.27:0/3135364738 >> v2:172.21.15.192:3300/0 conn(0x7fab1c0cbce0 0x7fab1c0ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.001+0000 7fab1ad9d640 1 -- 172.21.15.27:0/3135364738 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c0caf20 con 0x7fab1c0ada50 2023-12-19T18:16:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.003+0000 7fab1ad9d640 1 --2- 172.21.15.27:0/3135364738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1c0ada50 0x7fab1c0adf00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fab1c06d620 tx=0x7fab1402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f38ace126dad65e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.003+0000 7fab1959a640 1 -- 172.21.15.27:0/3135364738 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1402ca70 con 0x7fab1c0ada50 2023-12-19T18:16:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.003+0000 7fab1959a640 1 -- 172.21.15.27:0/3135364738 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1402cbd0 con 0x7fab1c0ada50 2023-12-19T18:16:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.003+0000 7fab1959a640 1 -- 172.21.15.27:0/3135364738 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab140343f0 con 0x7fab1c0ada50 2023-12-19T18:16:58.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.004+0000 7fab20e0b640 1 -- 172.21.15.27:0/3135364738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1c0ada50 msgr2=0x7fab1c0adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.004+0000 7fab20e0b640 1 --2- 172.21.15.27:0/3135364738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1c0ada50 0x7fab1c0adf00 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fab1c06d620 tx=0x7fab1402bed0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.004+0000 7fab20e0b640 1 -- 172.21.15.27:0/3135364738 shutdown_connections 2023-12-19T18:16:58.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.004+0000 7fab20e0b640 1 --2- 172.21.15.27:0/3135364738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1c0ada50 0x7fab1c0adf00 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.004+0000 7fab20e0b640 1 --2- 172.21.15.27:0/3135364738 >> v2:172.21.15.192:3300/0 conn(0x7fab1c0cbce0 0x7fab1c0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.004+0000 7fab20e0b640 1 --2- 172.21.15.27:0/3135364738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab1c15af70 0x7fab1c0ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.004+0000 7fab20e0b640 1 -- 172.21.15.27:0/3135364738 >> 172.21.15.27:0/3135364738 conn(0x7fab1c06a330 msgr2=0x7fab1c0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:58.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.004+0000 7fab20e0b640 1 -- 172.21.15.27:0/3135364738 shutdown_connections 2023-12-19T18:16:58.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.004+0000 7fab20e0b640 1 -- 172.21.15.27:0/3135364738 wait complete. 2023-12-19T18:16:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.006+0000 7fab20e0b640 1 Processor -- start 2023-12-19T18:16:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.006+0000 7fab20e0b640 1 -- start start 2023-12-19T18:16:58.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.007+0000 7fab20e0b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1c0ada50 0x7fab1c14b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.007+0000 7fab20e0b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab1c0cbce0 0x7fab1c0af3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.007+0000 7fab20e0b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab1c15af70 0x7fab1c0af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.007+0000 7fab20e0b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c14b9c0 con 0x7fab1c0ada50 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.007+0000 7fab20e0b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0b3f10 con 0x7fab1c15af70 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.007+0000 7fab20e0b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab1c0b41f0 con 0x7fab1c0cbce0 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.007+0000 7fab1a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1c0ada50 0x7fab1c14b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.007+0000 7fab1a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1c0ada50 0x7fab1c14b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41354/0 (socket says 172.21.15.27:41354) 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.007+0000 7fab1a59c640 1 -- 172.21.15.27:0/1693548502 learned_addr learned my addr 172.21.15.27:0/1693548502 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.008+0000 7fab19d9b640 1 --2- 172.21.15.27:0/1693548502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab1c0cbce0 0x7fab1c0af3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.008+0000 7fab1a59c640 1 -- 172.21.15.27:0/1693548502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab1c0cbce0 msgr2=0x7fab1c0af3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.008+0000 7fab1a59c640 1 --2- 172.21.15.27:0/1693548502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab1c0cbce0 0x7fab1c0af3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.008+0000 7fab1a59c640 1 -- 172.21.15.27:0/1693548502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab1c15af70 msgr2=0x7fab1c0af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.008+0000 7fab1a59c640 1 --2- 172.21.15.27:0/1693548502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab1c15af70 0x7fab1c0af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.008+0000 7fab1a59c640 1 -- 172.21.15.27:0/1693548502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab14009b30 con 0x7fab1c0ada50 2023-12-19T18:16:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.008+0000 7fab1a59c640 1 --2- 172.21.15.27:0/1693548502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1c0ada50 0x7fab1c14b480 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00d5f0 tx=0x7fab0c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.008+0000 7fab0affd640 1 -- 172.21.15.27:0/1693548502 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c0125f0 con 0x7fab1c0ada50 2023-12-19T18:16:58.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.008+0000 7fab0affd640 1 -- 172.21.15.27:0/1693548502 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab0c012750 con 0x7fab1c0ada50 2023-12-19T18:16:58.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.008+0000 7fab0affd640 1 -- 172.21.15.27:0/1693548502 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c012b00 con 0x7fab1c0ada50 2023-12-19T18:16:58.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.008+0000 7fab20e0b640 1 -- 172.21.15.27:0/1693548502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab1c0b44d0 con 0x7fab1c0ada50 2023-12-19T18:16:58.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.009+0000 7fab20e0b640 1 -- 172.21.15.27:0/1693548502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab1c14bf60 con 0x7fab1c0ada50 2023-12-19T18:16:58.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.010+0000 7fab20e0b640 1 -- 172.21.15.27:0/1693548502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab1c068e90 con 0x7fab1c0ada50 2023-12-19T18:16:58.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.011+0000 7fab0affd640 1 -- 172.21.15.27:0/1693548502 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fab0c012c60 con 0x7fab1c0ada50 2023-12-19T18:16:58.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.012+0000 7fab0affd640 1 --2- 172.21.15.27:0/1693548502 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faafc06a430 0x7faafc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.012+0000 7fab0affd640 1 -- 172.21.15.27:0/1693548502 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fab0c088ec0 con 0x7fab1c0ada50 2023-12-19T18:16:58.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.012+0000 7fab19d9b640 1 --2- 172.21.15.27:0/1693548502 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faafc06a430 0x7faafc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.014+0000 7fab19d9b640 1 --2- 172.21.15.27:0/1693548502 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faafc06a430 0x7faafc06c8f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0cd9d0 tx=0x7fab10008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.016+0000 7fab0affd640 1 -- 172.21.15.27:0/1693548502 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fab0c054390 con 0x7fab1c0ada50 2023-12-19T18:16:58.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.248+0000 7fab20e0b640 1 -- 172.21.15.27:0/1693548502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab1c0cb1d0 con 0x7fab1c0ada50 2023-12-19T18:16:58.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.249+0000 7fab0affd640 1 -- 172.21.15.27:0/1693548502 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fab0c0579b0 con 0x7fab1c0ada50 2023-12-19T18:16:58.249 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:58.249 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3178168231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3178168231},{"type":"v1","addr":"172.21.15.192:6837","nonce":3178168231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3961149394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3961149394},{"type":"v1","addr":"172.21.15.192:6836","nonce":3961149394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5190,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/752784926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":752784926},{"type":"v1","addr":"172.21.15.27:6833","nonce":752784926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:25.779852+0000","modified":"2023-12-19T18:16:29.303970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/3463287343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3463287343},{"type":"v1","addr":"172.21.15.27:6835","nonce":3463287343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T18:16:58.250 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 15 2023-12-19T18:16:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.252+0000 7fab20e0b640 1 -- 172.21.15.27:0/1693548502 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faafc06a430 msgr2=0x7faafc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.252+0000 7fab20e0b640 1 --2- 172.21.15.27:0/1693548502 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faafc06a430 0x7faafc06c8f0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0cd9d0 tx=0x7fab10008040 comp rx=0 tx=0).stop 2023-12-19T18:16:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.252+0000 7fab20e0b640 1 -- 172.21.15.27:0/1693548502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1c0ada50 msgr2=0x7fab1c14b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.252+0000 7fab20e0b640 1 --2- 172.21.15.27:0/1693548502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1c0ada50 0x7fab1c14b480 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fab0c00d5f0 tx=0x7fab0c00dab0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.252+0000 7fab20e0b640 1 -- 172.21.15.27:0/1693548502 shutdown_connections 2023-12-19T18:16:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.252+0000 7fab20e0b640 1 --2- 172.21.15.27:0/1693548502 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faafc06a430 0x7faafc06c8f0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.252+0000 7fab20e0b640 1 --2- 172.21.15.27:0/1693548502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fab1c15af70 0x7fab1c0af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.252+0000 7fab20e0b640 1 --2- 172.21.15.27:0/1693548502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fab1c0cbce0 0x7fab1c0af3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.252+0000 7fab20e0b640 1 --2- 172.21.15.27:0/1693548502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab1c0ada50 0x7fab1c14b480 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.252+0000 7fab20e0b640 1 -- 172.21.15.27:0/1693548502 >> 172.21.15.27:0/1693548502 conn(0x7fab1c06a330 msgr2=0x7fab1c070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.252+0000 7fab20e0b640 1 -- 172.21.15.27:0/1693548502 shutdown_connections 2023-12-19T18:16:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.252+0000 7fab20e0b640 1 -- 172.21.15.27:0/1693548502 wait complete. 2023-12-19T18:16:58.271 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:58.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.502+0000 7f6767b16640 1 Processor -- start 2023-12-19T18:16:58.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.502+0000 7f6767b16640 1 -- start start 2023-12-19T18:16:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.504+0000 7f6767b16640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f676015d270 0x7f67600cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.504+0000 7f6767b16640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f67600cce20 0x7f67600cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.504+0000 7f6767b16640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f676006b4e0 con 0x7f676015ce90 2023-12-19T18:16:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.504+0000 7f6767b16640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f676006ce90 con 0x7f67600cce20 2023-12-19T18:16:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.504+0000 7f6767b16640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67600704f0 con 0x7f676015d270 2023-12-19T18:16:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.504+0000 7f67660b3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f67600cce20 0x7f67600cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.504+0000 7f67658b2640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f676015ce90 0x7f67600cc710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55252/0 (socket says 172.21.15.27:55252) 2023-12-19T18:16:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.504+0000 7f67658b2640 1 -- 172.21.15.27:0/2371032292 learned_addr learned my addr 172.21.15.27:0/2371032292 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:58.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.505+0000 7f67650b1640 1 --2- 172.21.15.27:0/2371032292 >> v2:172.21.15.192:3300/0 conn(0x7f676015d270 0x7f67600cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.505+0000 7f67660b3640 1 -- 172.21.15.27:0/2371032292 >> v2:172.21.15.192:3300/0 conn(0x7f676015d270 msgr2=0x7f67600cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:58.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.505+0000 7f67660b3640 1 --2- 172.21.15.27:0/2371032292 >> v2:172.21.15.192:3300/0 conn(0x7f676015d270 0x7f67600cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.505+0000 7f67660b3640 1 -- 172.21.15.27:0/2371032292 >> v1:172.21.15.27:6789/0 conn(0x7f676015ce90 legacy=0x7f67600cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.505+0000 7f67660b3640 1 -- 172.21.15.27:0/2371032292 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676006b7c0 con 0x7f67600cce20 2023-12-19T18:16:58.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.505+0000 7f67650b1640 1 --2- 172.21.15.27:0/2371032292 >> v2:172.21.15.192:3300/0 conn(0x7f676015d270 0x7f67600cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:58.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.506+0000 7f67660b3640 1 --2- 172.21.15.27:0/2371032292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67600cce20 0x7f67600cb7f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f67600cea20 tx=0x7f675802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93269ff94d094d1b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.506+0000 7f67648b0640 1 -- 172.21.15.27:0/2371032292 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675802cc70 con 0x7f67600cce20 2023-12-19T18:16:58.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.506+0000 7f67648b0640 1 -- 172.21.15.27:0/2371032292 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675802cdd0 con 0x7f67600cce20 2023-12-19T18:16:58.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.506+0000 7f67648b0640 1 -- 172.21.15.27:0/2371032292 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6758034660 con 0x7f67600cce20 2023-12-19T18:16:58.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.507+0000 7f6767b16640 1 -- 172.21.15.27:0/2371032292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67600cce20 msgr2=0x7f67600cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.507+0000 7f6767b16640 1 --2- 172.21.15.27:0/2371032292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67600cce20 0x7f67600cb7f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f67600cea20 tx=0x7f675802bec0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.507+0000 7f6767b16640 1 -- 172.21.15.27:0/2371032292 shutdown_connections 2023-12-19T18:16:58.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.507+0000 7f6767b16640 1 --2- 172.21.15.27:0/2371032292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67600cce20 0x7f67600cb7f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.507+0000 7f6767b16640 1 --2- 172.21.15.27:0/2371032292 >> v2:172.21.15.192:3300/0 conn(0x7f676015d270 0x7f67600cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.507+0000 7f6767b16640 1 -- 172.21.15.27:0/2371032292 >> 172.21.15.27:0/2371032292 conn(0x7f676006a330 msgr2=0x7f67600b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:58.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.507+0000 7f6767b16640 1 -- 172.21.15.27:0/2371032292 shutdown_connections 2023-12-19T18:16:58.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.507+0000 7f6767b16640 1 -- 172.21.15.27:0/2371032292 wait complete. 2023-12-19T18:16:58.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.509+0000 7f6767b16640 1 Processor -- start 2023-12-19T18:16:58.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.509+0000 7f6767b16640 1 -- start start 2023-12-19T18:16:58.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.510+0000 7f6767b16640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67600cce20 0x7f676014b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.510+0000 7f6767b16640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f676015ce90 0x7f67600b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.510+0000 7f6767b16640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67600b3e20 0x7f67600c2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.510+0000 7f6767b16640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676014b0c0 con 0x7f67600b3e20 2023-12-19T18:16:58.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.510+0000 7f6767b16640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67600b42a0 con 0x7f676015ce90 2023-12-19T18:16:58.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.510+0000 7f6767b16640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67600c32f0 con 0x7f67600cce20 2023-12-19T18:16:58.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.511+0000 7f67660b3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67600b3e20 0x7f67600c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.511+0000 7f67660b3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67600b3e20 0x7f67600c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41376/0 (socket says 172.21.15.27:41376) 2023-12-19T18:16:58.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.511+0000 7f67660b3640 1 -- 172.21.15.27:0/3526706847 learned_addr learned my addr 172.21.15.27:0/3526706847 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.511+0000 7f67650b1640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f676015ce90 0x7f67600b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.511+0000 7f67658b2640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67600cce20 0x7f676014b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.511+0000 7f67660b3640 1 -- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67600cce20 msgr2=0x7f676014b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.511+0000 7f67660b3640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67600cce20 0x7f676014b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.511+0000 7f67660b3640 1 -- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f676015ce90 msgr2=0x7f67600b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.511+0000 7f67660b3640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f676015ce90 0x7f67600b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.511+0000 7f67660b3640 1 -- 172.21.15.27:0/3526706847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6758009a70 con 0x7f67600b3e20 2023-12-19T18:16:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.512+0000 7f67660b3640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67600b3e20 0x7f67600c2670 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f675802c4d0 tx=0x7f6758004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.512+0000 7f67567fc640 1 -- 172.21.15.27:0/3526706847 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675803e610 con 0x7f67600b3e20 2023-12-19T18:16:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.512+0000 7f67567fc640 1 -- 172.21.15.27:0/3526706847 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f675803e770 con 0x7f67600b3e20 2023-12-19T18:16:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.512+0000 7f67567fc640 1 -- 172.21.15.27:0/3526706847 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f675803eae0 con 0x7f67600b3e20 2023-12-19T18:16:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.512+0000 7f6767b16640 1 -- 172.21.15.27:0/3526706847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67600c35d0 con 0x7f67600b3e20 2023-12-19T18:16:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.512+0000 7f6767b16640 1 -- 172.21.15.27:0/3526706847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67600c3b90 con 0x7f67600b3e20 2023-12-19T18:16:58.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.513+0000 7f6767b16640 1 -- 172.21.15.27:0/3526706847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6760148be0 con 0x7f67600b3e20 2023-12-19T18:16:58.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.514+0000 7f67567fc640 1 -- 172.21.15.27:0/3526706847 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f675803b050 con 0x7f67600b3e20 2023-12-19T18:16:58.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.515+0000 7f67567fc640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f674406a430 0x7f674406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.515+0000 7f67567fc640 1 -- 172.21.15.27:0/3526706847 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f67580ab490 con 0x7f67600b3e20 2023-12-19T18:16:58.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.516+0000 7f67658b2640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f674406a430 0x7f674406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.517+0000 7f67658b2640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f674406a430 0x7f674406c8f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f675c000f60 tx=0x7f675c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.519+0000 7f67567fc640 1 -- 172.21.15.27:0/3526706847 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6758076b10 con 0x7f67600b3e20 2023-12-19T18:16:58.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.718+0000 7f6767b16640 1 -- 172.21.15.27:0/3526706847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6760068e90 con 0x7f67600b3e20 2023-12-19T18:16:58.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.719+0000 7f67567fc640 1 -- 172.21.15.27:0/3526706847 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16291 (secure 0 0 0) 0x7f675807a130 con 0x7f67600b3e20 2023-12-19T18:16:58.720 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:58.720 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:16:45.666993+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:23.925853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:24.827898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]},{"pgid":"5.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:16:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.723+0000 7f6767b16640 1 -- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f674406a430 msgr2=0x7f674406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.723+0000 7f6767b16640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f674406a430 0x7f674406c8f0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f675c000f60 tx=0x7f675c009040 comp rx=0 tx=0).stop 2023-12-19T18:16:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.723+0000 7f6767b16640 1 -- 172.21.15.27:0/3526706847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67600b3e20 msgr2=0x7f67600c2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.723+0000 7f6767b16640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67600b3e20 0x7f67600c2670 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f675802c4d0 tx=0x7f6758004a40 comp rx=0 tx=0).stop 2023-12-19T18:16:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.723+0000 7f6767b16640 1 -- 172.21.15.27:0/3526706847 shutdown_connections 2023-12-19T18:16:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.723+0000 7f6767b16640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f674406a430 0x7f674406c8f0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.723+0000 7f6767b16640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67600b3e20 0x7f67600c2670 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.723+0000 7f6767b16640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f676015ce90 0x7f67600b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.723+0000 7f6767b16640 1 --2- 172.21.15.27:0/3526706847 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f67600cce20 0x7f676014b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.723+0000 7f6767b16640 1 -- 172.21.15.27:0/3526706847 >> 172.21.15.27:0/3526706847 conn(0x7f676006a330 msgr2=0x7f67600ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.723+0000 7f6767b16640 1 -- 172.21.15.27:0/3526706847 shutdown_connections 2023-12-19T18:16:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.723+0000 7f6767b16640 1 -- 172.21.15.27:0/3526706847 wait complete. 2023-12-19T18:16:58.742 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:16:58.743 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:58.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.977+0000 7f1a5092d640 1 Processor -- start 2023-12-19T18:16:58.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.977+0000 7f1a5092d640 1 -- start start 2023-12-19T18:16:58.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.979+0000 7f1a5092d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a4c0cb300 0x7f1a4c0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.979+0000 7f1a5092d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1a4c0c8f90 0x7f1a4c160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.979+0000 7f1a5092d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4c1612d0 con 0x7f1a4c160e40 2023-12-19T18:16:58.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.979+0000 7f1a5092d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a4c0c4480 con 0x7f1a4c0c8f90 2023-12-19T18:16:58.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.979+0000 7f1a5092d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c1468c0 con 0x7f1a4c0cb300 2023-12-19T18:16:58.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.980+0000 7f1a4b59e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1a4c160e40 0x7f1a4c0ac8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55268/0 (socket says 172.21.15.27:55268) 2023-12-19T18:16:58.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.980+0000 7f1a4b59e640 1 -- 172.21.15.27:0/4258465404 learned_addr learned my addr 172.21.15.27:0/4258465404 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:16:58.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.980+0000 7f1a4259c640 1 --2- 172.21.15.27:0/4258465404 >> v2:172.21.15.192:3300/0 conn(0x7f1a4c0c8f90 0x7f1a4c160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.980+0000 7f1a4ad9d640 1 --2- 172.21.15.27:0/4258465404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a4c0cb300 0x7f1a4c0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.980+0000 7f1a4a59c640 1 -- 172.21.15.27:0/4258465404 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1513384936 0 0) 0x7f1a44001110 con 0x7f1a4c160e40 2023-12-19T18:16:58.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.980+0000 7f1a4a59c640 1 -- 172.21.15.27:0/4258465404 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a38003520 con 0x7f1a4c160e40 2023-12-19T18:16:58.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.981+0000 7f1a4259c640 1 -- 172.21.15.27:0/4258465404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a4c0cb300 msgr2=0x7f1a4c0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:58.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.981+0000 7f1a4259c640 1 --2- 172.21.15.27:0/4258465404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a4c0cb300 0x7f1a4c0cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.981+0000 7f1a4259c640 1 -- 172.21.15.27:0/4258465404 >> v1:172.21.15.27:6789/0 conn(0x7f1a4c160e40 legacy=0x7f1a4c0ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.981+0000 7f1a4259c640 1 -- 172.21.15.27:0/4258465404 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c0cbc20 con 0x7f1a4c0c8f90 2023-12-19T18:16:58.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.982+0000 7f1a4259c640 1 --2- 172.21.15.27:0/4258465404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c0c8f90 0x7f1a4c160900 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a340097a0 tx=0x7f1a3402b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bcd439df2f0ba06 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.983+0000 7f1a4a59c640 1 -- 172.21.15.27:0/4258465404 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3402d040 con 0x7f1a4c0c8f90 2023-12-19T18:16:58.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.983+0000 7f1a4a59c640 1 -- 172.21.15.27:0/4258465404 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a34031810 con 0x7f1a4c0c8f90 2023-12-19T18:16:58.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.983+0000 7f1a4a59c640 1 -- 172.21.15.27:0/4258465404 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a34031ba0 con 0x7f1a4c0c8f90 2023-12-19T18:16:58.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.983+0000 7f1a5092d640 1 -- 172.21.15.27:0/4258465404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c0c8f90 msgr2=0x7f1a4c160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.983+0000 7f1a5092d640 1 --2- 172.21.15.27:0/4258465404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c0c8f90 0x7f1a4c160900 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1a340097a0 tx=0x7f1a3402b8b0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.983+0000 7f1a5092d640 1 -- 172.21.15.27:0/4258465404 shutdown_connections 2023-12-19T18:16:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.983+0000 7f1a5092d640 1 --2- 172.21.15.27:0/4258465404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c0c8f90 0x7f1a4c160900 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.983+0000 7f1a5092d640 1 --2- 172.21.15.27:0/4258465404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a4c0cb300 0x7f1a4c0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.983+0000 7f1a5092d640 1 -- 172.21.15.27:0/4258465404 >> 172.21.15.27:0/4258465404 conn(0x7f1a4c069c70 msgr2=0x7f1a4c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.984+0000 7f1a5092d640 1 -- 172.21.15.27:0/4258465404 shutdown_connections 2023-12-19T18:16:58.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.984+0000 7f1a5092d640 1 -- 172.21.15.27:0/4258465404 wait complete. 2023-12-19T18:16:58.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.985+0000 7f1a5092d640 1 Processor -- start 2023-12-19T18:16:58.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.986+0000 7f1a5092d640 1 -- start start 2023-12-19T18:16:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a5092d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a4c0c8f90 0x7f1a4c15dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a5092d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c0cb300 0x7f1a4c0c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a5092d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c160e40 0x7f1a4c1f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a5092d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0c5a10 con 0x7f1a4c160e40 2023-12-19T18:16:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a5092d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c131a70 con 0x7f1a4c0cb300 2023-12-19T18:16:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a5092d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c131140 con 0x7f1a4c0c8f90 2023-12-19T18:16:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a4b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c160e40 0x7f1a4c1f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a4b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c160e40 0x7f1a4c1f60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41386/0 (socket says 172.21.15.27:41386) 2023-12-19T18:16:58.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a4b59e640 1 -- 172.21.15.27:0/3341779169 learned_addr learned my addr 172.21.15.27:0/3341779169 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a4ad9d640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a4c0c8f90 0x7f1a4c15dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a4259c640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c0cb300 0x7f1a4c0c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a4b59e640 1 -- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a4c0c8f90 msgr2=0x7f1a4c15dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a4b59e640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a4c0c8f90 0x7f1a4c15dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a4b59e640 1 -- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c0cb300 msgr2=0x7f1a4c0c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a4b59e640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c0cb300 0x7f1a4c0c9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a4b59e640 1 -- 172.21.15.27:0/3341779169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a34009400 con 0x7f1a4c160e40 2023-12-19T18:16:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.987+0000 7f1a4ad9d640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a4c0c8f90 0x7f1a4c15dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:16:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.988+0000 7f1a4b59e640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c160e40 0x7f1a4c1f60a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400c700 tx=0x7f1a4400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.988+0000 7f1a43fff640 1 -- 172.21.15.27:0/3341779169 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44005ab0 con 0x7f1a4c160e40 2023-12-19T18:16:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.988+0000 7f1a43fff640 1 -- 172.21.15.27:0/3341779169 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a44005c10 con 0x7f1a4c160e40 2023-12-19T18:16:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.988+0000 7f1a5092d640 1 -- 172.21.15.27:0/3341779169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4c131420 con 0x7f1a4c160e40 2023-12-19T18:16:58.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.988+0000 7f1a43fff640 1 -- 172.21.15.27:0/3341779169 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44001d50 con 0x7f1a4c160e40 2023-12-19T18:16:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.988+0000 7f1a5092d640 1 -- 172.21.15.27:0/3341779169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c1434b0 con 0x7f1a4c160e40 2023-12-19T18:16:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.989+0000 7f1a5092d640 1 -- 172.21.15.27:0/3341779169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4c131be0 con 0x7f1a4c160e40 2023-12-19T18:16:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.990+0000 7f1a43fff640 1 -- 172.21.15.27:0/3341779169 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1a44007070 con 0x7f1a4c160e40 2023-12-19T18:16:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.991+0000 7f1a43fff640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1a2c06a460 0x7f1a2c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.991+0000 7f1a43fff640 1 -- 172.21.15.27:0/3341779169 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f1a44087df0 con 0x7f1a4c160e40 2023-12-19T18:16:58.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.992+0000 7f1a4ad9d640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1a2c06a460 0x7f1a2c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:58.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.993+0000 7f1a4ad9d640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1a2c06a460 0x7f1a2c06c920 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c007440 tx=0x7f1a3c00b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:58.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:58.996+0000 7f1a43fff640 1 -- 172.21.15.27:0/3341779169 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1a440532c0 con 0x7f1a4c160e40 2023-12-19T18:16:59.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.227+0000 7f1a5092d640 1 -- 172.21.15.27:0/3341779169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1a4c1612d0 con 0x7f1a4c160e40 2023-12-19T18:16:59.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.228+0000 7f1a43fff640 1 -- 172.21.15.27:0/3341779169 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f1a4c1612d0 con 0x7f1a4c160e40 2023-12-19T18:16:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:59.228 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3178168231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3178168231},{"type":"v1","addr":"172.21.15.192:6837","nonce":3178168231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3961149394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3961149394},{"type":"v1","addr":"172.21.15.192:6836","nonce":3961149394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5190,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/752784926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":752784926},{"type":"v1","addr":"172.21.15.27:6833","nonce":752784926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:25.779852+0000","modified":"2023-12-19T18:16:29.303970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/3463287343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3463287343},{"type":"v1","addr":"172.21.15.27:6835","nonce":3463287343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T18:16:59.228 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 15 2023-12-19T18:16:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.231+0000 7f1a5092d640 1 -- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1a2c06a460 msgr2=0x7f1a2c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.231+0000 7f1a5092d640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1a2c06a460 0x7f1a2c06c920 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c007440 tx=0x7f1a3c00b040 comp rx=0 tx=0).stop 2023-12-19T18:16:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.231+0000 7f1a5092d640 1 -- 172.21.15.27:0/3341779169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c160e40 msgr2=0x7f1a4c1f60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.231+0000 7f1a5092d640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c160e40 0x7f1a4c1f60a0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f1a4400c700 tx=0x7f1a4400cbc0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.231+0000 7f1a5092d640 1 -- 172.21.15.27:0/3341779169 shutdown_connections 2023-12-19T18:16:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.231+0000 7f1a5092d640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1a2c06a460 0x7f1a2c06c920 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.231+0000 7f1a5092d640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a4c160e40 0x7f1a4c1f60a0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.231+0000 7f1a5092d640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1a4c0cb300 0x7f1a4c0c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.231+0000 7f1a5092d640 1 --2- 172.21.15.27:0/3341779169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1a4c0c8f90 0x7f1a4c15dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.231+0000 7f1a5092d640 1 -- 172.21.15.27:0/3341779169 >> 172.21.15.27:0/3341779169 conn(0x7f1a4c069c70 msgr2=0x7f1a4c143f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:59.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.231+0000 7f1a5092d640 1 -- 172.21.15.27:0/3341779169 shutdown_connections 2023-12-19T18:16:59.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.231+0000 7f1a5092d640 1 -- 172.21.15.27:0/3341779169 wait complete. 2023-12-19T18:16:59.251 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:16:59.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.470+0000 7f0dbbfff640 1 Processor -- start 2023-12-19T18:16:59.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.470+0000 7f0dbbfff640 1 -- start start 2023-12-19T18:16:59.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.472+0000 7f0dbbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0dbc15ee30 0x7f0dbc0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.472+0000 7f0dbbfff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc0cb350 con 0x7f0dbc0afeb0 2023-12-19T18:16:59.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.472+0000 7f0dbbfff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0dbc0cb630 con 0x7f0dbc1553a0 2023-12-19T18:16:59.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.473+0000 7f0dbbfff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0dbc0cbcf0 con 0x7f0dbc15ee30 2023-12-19T18:16:59.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.473+0000 7f0dba7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0dbc15ee30 0x7f0dbc0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.473+0000 7f0dba7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0dbc15ee30 0x7f0dbc0c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41396/0 (socket says 172.21.15.27:41396) 2023-12-19T18:16:59.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.473+0000 7f0dba7fc640 1 -- 172.21.15.27:0/1481841864 learned_addr learned my addr 172.21.15.27:0/1481841864 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:59.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.473+0000 7f0dba7fc640 1 -- 172.21.15.27:0/1481841864 >> v1:172.21.15.192:6789/0 conn(0x7f0dbc1553a0 legacy=0x7f0dbc15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:59.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.473+0000 7f0dba7fc640 1 -- 172.21.15.27:0/1481841864 >> v1:172.21.15.27:6789/0 conn(0x7f0dbc0afeb0 legacy=0x7f0dbc0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.474+0000 7f0dba7fc640 1 -- 172.21.15.27:0/1481841864 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dbc0cb910 con 0x7f0dbc15ee30 2023-12-19T18:16:59.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.475+0000 7f0dba7fc640 1 --2- 172.21.15.27:0/1481841864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc15ee30 0x7f0dbc0c50a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009870 tx=0x7f0da402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81f389df302a5d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:59.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.475+0000 7f0db9ffb640 1 -- 172.21.15.27:0/1481841864 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da4030850 con 0x7f0dbc15ee30 2023-12-19T18:16:59.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.475+0000 7f0db9ffb640 1 -- 172.21.15.27:0/1481841864 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da40309b0 con 0x7f0dbc15ee30 2023-12-19T18:16:59.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.475+0000 7f0db9ffb640 1 -- 172.21.15.27:0/1481841864 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da4030d40 con 0x7f0dbc15ee30 2023-12-19T18:16:59.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.476+0000 7f0dbbfff640 1 -- 172.21.15.27:0/1481841864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc15ee30 msgr2=0x7f0dbc0c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.476+0000 7f0dbbfff640 1 --2- 172.21.15.27:0/1481841864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc15ee30 0x7f0dbc0c50a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f0da4009870 tx=0x7f0da402b990 comp rx=0 tx=0).stop 2023-12-19T18:16:59.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.476+0000 7f0dbbfff640 1 -- 172.21.15.27:0/1481841864 shutdown_connections 2023-12-19T18:16:59.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.476+0000 7f0dbbfff640 1 --2- 172.21.15.27:0/1481841864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc15ee30 0x7f0dbc0c50a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.476+0000 7f0dbbfff640 1 -- 172.21.15.27:0/1481841864 >> 172.21.15.27:0/1481841864 conn(0x7f0dbc069c70 msgr2=0x7f0dbc06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:59.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.476+0000 7f0dbbfff640 1 -- 172.21.15.27:0/1481841864 shutdown_connections 2023-12-19T18:16:59.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.476+0000 7f0dbbfff640 1 -- 172.21.15.27:0/1481841864 wait complete. 2023-12-19T18:16:59.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.477+0000 7f0dbbfff640 1 Processor -- start 2023-12-19T18:16:59.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.478+0000 7f0dbbfff640 1 -- start start 2023-12-19T18:16:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.478+0000 7f0dbbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dbc0afeb0 0x7f0dbc154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dbbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1553a0 0x7f0dbc0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dbbfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dbc15ee30 0x7f0dbc158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dbbfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0b4470 con 0x7f0dbc1553a0 2023-12-19T18:16:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dbbfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0ad100 con 0x7f0dbc0afeb0 2023-12-19T18:16:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dbbfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0dbc0ad3e0 con 0x7f0dbc15ee30 2023-12-19T18:16:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dba7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1553a0 0x7f0dbc0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dba7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1553a0 0x7f0dbc0b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41410/0 (socket says 172.21.15.27:41410) 2023-12-19T18:16:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dba7fc640 1 -- 172.21.15.27:0/3625702979 learned_addr learned my addr 172.21.15.27:0/3625702979 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dbaffd640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dbc0afeb0 0x7f0dbc154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dbb7fe640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dbc15ee30 0x7f0dbc158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dba7fc640 1 -- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dbc15ee30 msgr2=0x7f0dbc158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dba7fc640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dbc15ee30 0x7f0dbc158f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dba7fc640 1 -- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dbc0afeb0 msgr2=0x7f0dbc154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dba7fc640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dbc0afeb0 0x7f0dbc154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.479+0000 7f0dba7fc640 1 -- 172.21.15.27:0/3625702979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da40094d0 con 0x7f0dbc1553a0 2023-12-19T18:16:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.480+0000 7f0dba7fc640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1553a0 0x7f0dbc0b15c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0da4000c00 tx=0x7f0da4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:59.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.480+0000 7f0db37fe640 1 -- 172.21.15.27:0/3625702979 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da4030820 con 0x7f0dbc1553a0 2023-12-19T18:16:59.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.480+0000 7f0db37fe640 1 -- 172.21.15.27:0/3625702979 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da4039d90 con 0x7f0dbc1553a0 2023-12-19T18:16:59.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.480+0000 7f0dbbfff640 1 -- 172.21.15.27:0/3625702979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dbc0ad6c0 con 0x7f0dbc1553a0 2023-12-19T18:16:59.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.480+0000 7f0db37fe640 1 -- 172.21.15.27:0/3625702979 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da4043670 con 0x7f0dbc1553a0 2023-12-19T18:16:59.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.480+0000 7f0dbbfff640 1 -- 172.21.15.27:0/3625702979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dbc0adc80 con 0x7f0dbc1553a0 2023-12-19T18:16:59.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.481+0000 7f0dbbfff640 1 -- 172.21.15.27:0/3625702979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dbc04e840 con 0x7f0dbc1553a0 2023-12-19T18:16:59.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.483+0000 7f0db37fe640 1 -- 172.21.15.27:0/3625702979 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0da402e070 con 0x7f0dbc1553a0 2023-12-19T18:16:59.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.483+0000 7f0db37fe640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d9c06a430 0x7f0d9c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.484+0000 7f0db37fe640 1 -- 172.21.15.27:0/3625702979 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f0da40aadd0 con 0x7f0dbc1553a0 2023-12-19T18:16:59.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.484+0000 7f0dbaffd640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d9c06a430 0x7f0d9c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.485+0000 7f0dbaffd640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d9c06a430 0x7f0d9c06c8f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0012e0 tx=0x7f0dac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.488+0000 7f0db37fe640 1 -- 172.21.15.27:0/3625702979 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0da4076330 con 0x7f0dbc1553a0 2023-12-19T18:16:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.724+0000 7f0dbbfff640 1 -- 172.21.15.27:0/3625702979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0dbc0af6a0 con 0x7f0dbc1553a0 2023-12-19T18:16:59.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.725+0000 7f0db37fe640 1 -- 172.21.15.27:0/3625702979 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 15 v15) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f0da4079950 con 0x7f0dbc1553a0 2023-12-19T18:16:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:16:59.725 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":15,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3178168231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3178168231},{"type":"v1","addr":"172.21.15.192:6837","nonce":3178168231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3961149394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3961149394},{"type":"v1","addr":"172.21.15.192:6836","nonce":3961149394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5190,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/752784926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":752784926},{"type":"v1","addr":"172.21.15.27:6833","nonce":752784926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":15,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:16:25.779852+0000","modified":"2023-12-19T18:16:29.303970+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5196},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5196":{"gid":5196,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":4,"addr":"172.21.15.27:6835/3463287343","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3463287343},{"type":"v1","addr":"172.21.15.27:6835","nonce":3463287343}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-19T18:16:59.726 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 15 2023-12-19T18:16:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.727+0000 7f0dbbfff640 1 -- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d9c06a430 msgr2=0x7f0d9c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.727+0000 7f0dbbfff640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d9c06a430 0x7f0d9c06c8f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f0dac0012e0 tx=0x7f0dac009040 comp rx=0 tx=0).stop 2023-12-19T18:16:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.727+0000 7f0dbbfff640 1 -- 172.21.15.27:0/3625702979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1553a0 msgr2=0x7f0dbc0b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.727+0000 7f0dbbfff640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1553a0 0x7f0dbc0b15c0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f0da4000c00 tx=0x7f0da4004730 comp rx=0 tx=0).stop 2023-12-19T18:16:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.728+0000 7f0dbbfff640 1 -- 172.21.15.27:0/3625702979 shutdown_connections 2023-12-19T18:16:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.728+0000 7f0dbbfff640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d9c06a430 0x7f0d9c06c8f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.728+0000 7f0dbbfff640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0dbc15ee30 0x7f0dbc158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.728+0000 7f0dbbfff640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dbc1553a0 0x7f0dbc0b15c0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.728+0000 7f0dbbfff640 1 --2- 172.21.15.27:0/3625702979 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0dbc0afeb0 0x7f0dbc154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.728+0000 7f0dbbfff640 1 -- 172.21.15.27:0/3625702979 >> 172.21.15.27:0/3625702979 conn(0x7f0dbc069c70 msgr2=0x7f0dbc15e6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:59.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.728+0000 7f0dbbfff640 1 -- 172.21.15.27:0/3625702979 shutdown_connections 2023-12-19T18:16:59.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.728+0000 7f0dbbfff640 1 -- 172.21.15.27:0/3625702979 wait complete. 2023-12-19T18:16:59.748 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:16:59.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.982+0000 7f99c5bf9640 1 Processor -- start 2023-12-19T18:16:59.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.982+0000 7f99c5bf9640 1 -- start start 2023-12-19T18:16:59.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.984+0000 7f99c5bf9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99c0154030 0x7f99c00cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.984+0000 7f99c5bf9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f99c00c5560 0x7f99c00afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.984+0000 7f99c5bf9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99c00affd0 con 0x7f99c00af630 2023-12-19T18:16:59.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.984+0000 7f99c5bf9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99c00cc190 con 0x7f99c00c5560 2023-12-19T18:16:59.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.984+0000 7f99c5bf9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c00cb2a0 con 0x7f99c0154030 2023-12-19T18:16:59.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.985+0000 7f99c53f8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f99c00c5560 0x7f99c00afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.985+0000 7f99c53f8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f99c00c5560 0x7f99c00afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41422/0 (socket says 172.21.15.27:41422) 2023-12-19T18:16:59.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.985+0000 7f99c53f8640 1 -- 172.21.15.27:0/299991437 learned_addr learned my addr 172.21.15.27:0/299991437 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:59.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.985+0000 7f99c4bf7640 1 --2- 172.21.15.27:0/299991437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99c0154030 0x7f99c00cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.985+0000 7f99c53f8640 1 -- 172.21.15.27:0/299991437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99c0154030 msgr2=0x7f99c00cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:16:59.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.985+0000 7f99c53f8640 1 --2- 172.21.15.27:0/299991437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99c0154030 0x7f99c00cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.985+0000 7f99c53f8640 1 -- 172.21.15.27:0/299991437 >> v1:172.21.15.27:6789/0 conn(0x7f99c00af630 legacy=0x7f99c015d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.985+0000 7f99c53f8640 1 -- 172.21.15.27:0/299991437 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c00cb580 con 0x7f99c00c5560 2023-12-19T18:16:59.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.985+0000 7f99bf7fe640 1 -- 172.21.15.27:0/299991437 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1646188510 0 0) 0x7f99c00affd0 con 0x7f99c00af630 2023-12-19T18:16:59.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.986+0000 7f99c53f8640 1 --2- 172.21.15.27:0/299991437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c00c5560 0x7f99c00afa90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009e50 tx=0x7f99b802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707f2af397bec086 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:59.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.987+0000 7f99bf7fe640 1 -- 172.21.15.27:0/299991437 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b802cb00 con 0x7f99c00c5560 2023-12-19T18:16:59.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.987+0000 7f99bf7fe640 1 -- 172.21.15.27:0/299991437 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b802cc60 con 0x7f99c00c5560 2023-12-19T18:16:59.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.987+0000 7f99bf7fe640 1 -- 172.21.15.27:0/299991437 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b80343f0 con 0x7f99c00c5560 2023-12-19T18:16:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.987+0000 7f99c5bf9640 1 -- 172.21.15.27:0/299991437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c00c5560 msgr2=0x7f99c00afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.987+0000 7f99c5bf9640 1 --2- 172.21.15.27:0/299991437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c00c5560 0x7f99c00afa90 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f99b8009e50 tx=0x7f99b802bf60 comp rx=0 tx=0).stop 2023-12-19T18:16:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.987+0000 7f99c5bf9640 1 -- 172.21.15.27:0/299991437 shutdown_connections 2023-12-19T18:16:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.987+0000 7f99c5bf9640 1 --2- 172.21.15.27:0/299991437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c00c5560 0x7f99c00afa90 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.987+0000 7f99c5bf9640 1 --2- 172.21.15.27:0/299991437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99c0154030 0x7f99c00cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.987+0000 7f99c5bf9640 1 -- 172.21.15.27:0/299991437 >> 172.21.15.27:0/299991437 conn(0x7f99c0069c70 msgr2=0x7f99c006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:16:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.987+0000 7f99c5bf9640 1 -- 172.21.15.27:0/299991437 shutdown_connections 2023-12-19T18:16:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.987+0000 7f99c5bf9640 1 -- 172.21.15.27:0/299991437 wait complete. 2023-12-19T18:16:59.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.989+0000 7f99c5bf9640 1 Processor -- start 2023-12-19T18:16:59.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.989+0000 7f99c5bf9640 1 -- start start 2023-12-19T18:16:59.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.990+0000 7f99c5bf9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c00af630 0x7f99c01497e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.990+0000 7f99c5bf9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99c00c5560 0x7f99c014b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.990+0000 7f99c5bf9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99c0154030 0x7f99c0148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.990+0000 7f99c5bf9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0149d20 con 0x7f99c00af630 2023-12-19T18:16:59.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.990+0000 7f99c5bf9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c014bb40 con 0x7f99c0154030 2023-12-19T18:16:59.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.990+0000 7f99c5bf9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99c0149fc0 con 0x7f99c00c5560 2023-12-19T18:16:59.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.991+0000 7f99c4bf7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c00af630 0x7f99c01497e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.991+0000 7f99c4bf7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c00af630 0x7f99c01497e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41432/0 (socket says 172.21.15.27:41432) 2023-12-19T18:16:59.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.991+0000 7f99c4bf7640 1 -- 172.21.15.27:0/3981612474 learned_addr learned my addr 172.21.15.27:0/3981612474 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:16:59.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.991+0000 7f99c4bf7640 1 -- 172.21.15.27:0/3981612474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99c00c5560 msgr2=0x7f99c014b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.991+0000 7f99c4bf7640 1 --2- 172.21.15.27:0/3981612474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99c00c5560 0x7f99c014b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.991+0000 7f99c4bf7640 1 -- 172.21.15.27:0/3981612474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99c0154030 msgr2=0x7f99c0148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:16:59.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.991+0000 7f99c4bf7640 1 --2- 172.21.15.27:0/3981612474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99c0154030 0x7f99c0148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:16:59.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.991+0000 7f99c4bf7640 1 -- 172.21.15.27:0/3981612474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b8009b30 con 0x7f99c00af630 2023-12-19T18:16:59.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.991+0000 7f99c4bf7640 1 --2- 172.21.15.27:0/3981612474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c00af630 0x7f99c01497e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f99b400d5f0 tx=0x7f99b400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:16:59.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.992+0000 7f99bd7fa640 1 -- 172.21.15.27:0/3981612474 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b40125f0 con 0x7f99c00af630 2023-12-19T18:16:59.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.992+0000 7f99bd7fa640 1 -- 172.21.15.27:0/3981612474 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99b4012750 con 0x7f99c00af630 2023-12-19T18:16:59.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.992+0000 7f99bd7fa640 1 -- 172.21.15.27:0/3981612474 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99b4012b00 con 0x7f99c00af630 2023-12-19T18:16:59.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.992+0000 7f99c5bf9640 1 -- 172.21.15.27:0/3981612474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c014a840 con 0x7f99c00af630 2023-12-19T18:16:59.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.992+0000 7f99c5bf9640 1 -- 172.21.15.27:0/3981612474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c014ae00 con 0x7f99c00af630 2023-12-19T18:16:59.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.993+0000 7f99c5bf9640 1 -- 172.21.15.27:0/3981612474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c004e840 con 0x7f99c00af630 2023-12-19T18:16:59.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.994+0000 7f99bd7fa640 1 -- 172.21.15.27:0/3981612474 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f99b4012c60 con 0x7f99c00af630 2023-12-19T18:16:59.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.995+0000 7f99bd7fa640 1 --2- 172.21.15.27:0/3981612474 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99a406a480 0x7f99a406c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:16:59.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.995+0000 7f99bd7fa640 1 -- 172.21.15.27:0/3981612474 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f99b4088cb0 con 0x7f99c00af630 2023-12-19T18:16:59.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.996+0000 7f99bffff640 1 --2- 172.21.15.27:0/3981612474 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99a406a480 0x7f99a406c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:16:59.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:16:59.997+0000 7f99bffff640 1 --2- 172.21.15.27:0/3981612474 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99a406a480 0x7f99a406c940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f99c00c8d00 tx=0x7f99ac007910 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:00.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.000+0000 7f99bd7fa640 1 -- 172.21.15.27:0/3981612474 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f99b4054390 con 0x7f99c00af630 2023-12-19T18:17:00.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.196+0000 7f99c5bf9640 1 -- 172.21.15.27:0/3981612474 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99c00cf730 con 0x7f99c00af630 2023-12-19T18:17:00.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.198+0000 7f99bd7fa640 1 -- 172.21.15.27:0/3981612474 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) v1 ==== 74+0+16291 (secure 0 0 0) 0x7f99b40579b0 con 0x7f99c00af630 2023-12-19T18:17:00.199 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:00.199 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:16:45.666993+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:16:23.925853+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-19T18:16:24.827898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"5.13","mappings":[{"from":4,"to":5}]},{"pgid":"5.15","mappings":[{"from":4,"to":5}]},{"pgid":"5.2a","mappings":[{"from":4,"to":5}]},{"pgid":"5.32","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:00.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.201+0000 7f99c5bf9640 1 -- 172.21.15.27:0/3981612474 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99a406a480 msgr2=0x7f99a406c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.201+0000 7f99c5bf9640 1 --2- 172.21.15.27:0/3981612474 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99a406a480 0x7f99a406c940 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f99c00c8d00 tx=0x7f99ac007910 comp rx=0 tx=0).stop 2023-12-19T18:17:00.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.201+0000 7f99c5bf9640 1 -- 172.21.15.27:0/3981612474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c00af630 msgr2=0x7f99c01497e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.201+0000 7f99c5bf9640 1 --2- 172.21.15.27:0/3981612474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c00af630 0x7f99c01497e0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f99b400d5f0 tx=0x7f99b400dab0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.201+0000 7f99c5bf9640 1 -- 172.21.15.27:0/3981612474 shutdown_connections 2023-12-19T18:17:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.201+0000 7f99c5bf9640 1 --2- 172.21.15.27:0/3981612474 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99a406a480 0x7f99a406c940 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.201+0000 7f99c5bf9640 1 --2- 172.21.15.27:0/3981612474 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99c0154030 0x7f99c0148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.201+0000 7f99c5bf9640 1 --2- 172.21.15.27:0/3981612474 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99c00c5560 0x7f99c014b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.201+0000 7f99c5bf9640 1 --2- 172.21.15.27:0/3981612474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c00af630 0x7f99c01497e0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.201+0000 7f99c5bf9640 1 -- 172.21.15.27:0/3981612474 >> 172.21.15.27:0/3981612474 conn(0x7f99c0069c70 msgr2=0x7f99c014fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.201+0000 7f99c5bf9640 1 -- 172.21.15.27:0/3981612474 shutdown_connections 2023-12-19T18:17:00.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.201+0000 7f99c5bf9640 1 -- 172.21.15.27:0/3981612474 wait complete. 2023-12-19T18:17:00.223 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:17:00.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.446+0000 7f30e96ce640 1 Processor -- start 2023-12-19T18:17:00.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.446+0000 7f30e96ce640 1 -- start start 2023-12-19T18:17:00.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.448+0000 7f30e96ce640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f30e415adf0 0x7f30e406d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.448+0000 7f30e96ce640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f30e406fc10 0x7f30e40712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.448+0000 7f30e96ce640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30e4071820 con 0x7f30e4070c00 2023-12-19T18:17:00.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.448+0000 7f30e96ce640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e40703c0 con 0x7f30e415adf0 2023-12-19T18:17:00.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.448+0000 7f30e96ce640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30e406d8c0 con 0x7f30e406fc10 2023-12-19T18:17:00.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.448+0000 7f30e2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f30e415adf0 0x7f30e406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.449+0000 7f30e2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f30e415adf0 0x7f30e406d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41444/0 (socket says 172.21.15.27:41444) 2023-12-19T18:17:00.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.449+0000 7f30e2ffd640 1 -- 172.21.15.27:0/3205106383 learned_addr learned my addr 172.21.15.27:0/3205106383 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:00.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.449+0000 7f30e37fe640 1 --2- 172.21.15.27:0/3205106383 >> v2:172.21.15.192:3300/0 conn(0x7f30e406fc10 0x7f30e40712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.449+0000 7f30e2ffd640 1 -- 172.21.15.27:0/3205106383 >> v2:172.21.15.192:3300/0 conn(0x7f30e406fc10 msgr2=0x7f30e40712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:00.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.449+0000 7f30e2ffd640 1 --2- 172.21.15.27:0/3205106383 >> v2:172.21.15.192:3300/0 conn(0x7f30e406fc10 0x7f30e40712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.449+0000 7f30e2ffd640 1 -- 172.21.15.27:0/3205106383 >> v1:172.21.15.192:6789/0 conn(0x7f30e4070c00 legacy=0x7f30e406f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.449+0000 7f30e2ffd640 1 -- 172.21.15.27:0/3205106383 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30e406b520 con 0x7f30e415adf0 2023-12-19T18:17:00.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.449+0000 7f30e37fe640 1 --2- 172.21.15.27:0/3205106383 >> v2:172.21.15.192:3300/0 conn(0x7f30e406fc10 0x7f30e40712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:00.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.451+0000 7f30e2ffd640 1 --2- 172.21.15.27:0/3205106383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30e415adf0 0x7f30e406d380 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f30e406aff0 tx=0x7f30d402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4f0626aafdcaf41 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:00.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.451+0000 7f30e27fc640 1 -- 172.21.15.27:0/3205106383 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4031860 con 0x7f30e415adf0 2023-12-19T18:17:00.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.451+0000 7f30e27fc640 1 -- 172.21.15.27:0/3205106383 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d40319c0 con 0x7f30e415adf0 2023-12-19T18:17:00.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.451+0000 7f30e27fc640 1 -- 172.21.15.27:0/3205106383 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d4031d50 con 0x7f30e415adf0 2023-12-19T18:17:00.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.452+0000 7f30e96ce640 1 -- 172.21.15.27:0/3205106383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30e415adf0 msgr2=0x7f30e406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.452+0000 7f30e96ce640 1 --2- 172.21.15.27:0/3205106383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30e415adf0 0x7f30e406d380 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f30e406aff0 tx=0x7f30d402b900 comp rx=0 tx=0).stop 2023-12-19T18:17:00.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.452+0000 7f30e96ce640 1 -- 172.21.15.27:0/3205106383 shutdown_connections 2023-12-19T18:17:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.452+0000 7f30e96ce640 1 --2- 172.21.15.27:0/3205106383 >> v2:172.21.15.192:3300/0 conn(0x7f30e406fc10 0x7f30e40712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.452+0000 7f30e96ce640 1 --2- 172.21.15.27:0/3205106383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30e415adf0 0x7f30e406d380 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.452+0000 7f30e96ce640 1 -- 172.21.15.27:0/3205106383 >> 172.21.15.27:0/3205106383 conn(0x7f30e40ceb50 msgr2=0x7f30e40cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.452+0000 7f30e96ce640 1 -- 172.21.15.27:0/3205106383 shutdown_connections 2023-12-19T18:17:00.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.452+0000 7f30e96ce640 1 -- 172.21.15.27:0/3205106383 wait complete. 2023-12-19T18:17:00.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.454+0000 7f30e96ce640 1 Processor -- start 2023-12-19T18:17:00.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.454+0000 7f30e96ce640 1 -- start start 2023-12-19T18:17:00.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.455+0000 7f30e96ce640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30e406fc10 0x7f30e414df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.455+0000 7f30e96ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30e4070c00 0x7f30e40ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.455+0000 7f30e96ce640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30e415adf0 0x7f30e40c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.455+0000 7f30e96ce640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40af020 con 0x7f30e4070c00 2023-12-19T18:17:00.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.455+0000 7f30e96ce640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40ae410 con 0x7f30e406fc10 2023-12-19T18:17:00.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.455+0000 7f30e96ce640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30e40ae6f0 con 0x7f30e415adf0 2023-12-19T18:17:00.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.455+0000 7f30da7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30e4070c00 0x7f30e40ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.456+0000 7f30da7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30e4070c00 0x7f30e40ada00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41458/0 (socket says 172.21.15.27:41458) 2023-12-19T18:17:00.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.456+0000 7f30da7fc640 1 -- 172.21.15.27:0/1470729794 learned_addr learned my addr 172.21.15.27:0/1470729794 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:00.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.456+0000 7f30e37fe640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30e415adf0 0x7f30e40c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.456+0000 7f30e2ffd640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30e406fc10 0x7f30e414df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.456+0000 7f30da7fc640 1 -- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30e415adf0 msgr2=0x7f30e40c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.456+0000 7f30da7fc640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30e415adf0 0x7f30e40c3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.456+0000 7f30da7fc640 1 -- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30e406fc10 msgr2=0x7f30e414df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.456+0000 7f30da7fc640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30e406fc10 0x7f30e414df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.456+0000 7f30da7fc640 1 -- 172.21.15.27:0/1470729794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d40094d0 con 0x7f30e4070c00 2023-12-19T18:17:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.456+0000 7f30e37fe640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30e415adf0 0x7f30e40c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.456+0000 7f30e2ffd640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30e406fc10 0x7f30e414df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.456+0000 7f30da7fc640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30e4070c00 0x7f30e40ada00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00c450 tx=0x7f30cc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.457+0000 7f30dbfff640 1 -- 172.21.15.27:0/1470729794 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc0052d0 con 0x7f30e4070c00 2023-12-19T18:17:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.457+0000 7f30dbfff640 1 -- 172.21.15.27:0/1470729794 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30cc005430 con 0x7f30e4070c00 2023-12-19T18:17:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.457+0000 7f30e96ce640 1 -- 172.21.15.27:0/1470729794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e40ae9d0 con 0x7f30e4070c00 2023-12-19T18:17:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.457+0000 7f30dbfff640 1 -- 172.21.15.27:0/1470729794 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30cc012870 con 0x7f30e4070c00 2023-12-19T18:17:00.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.457+0000 7f30e96ce640 1 -- 172.21.15.27:0/1470729794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e414e4b0 con 0x7f30e4070c00 2023-12-19T18:17:00.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.458+0000 7f30e96ce640 1 -- 172.21.15.27:0/1470729794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30e40af1d0 con 0x7f30e4070c00 2023-12-19T18:17:00.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.459+0000 7f30dbfff640 1 -- 172.21.15.27:0/1470729794 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f30cc0129d0 con 0x7f30e4070c00 2023-12-19T18:17:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.460+0000 7f30dbfff640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f30c406a430 0x7f30c406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.460+0000 7f30dbfff640 1 -- 172.21.15.27:0/1470729794 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f30cc088570 con 0x7f30e4070c00 2023-12-19T18:17:00.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.461+0000 7f30e2ffd640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f30c406a430 0x7f30c406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.462+0000 7f30e2ffd640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f30c406a430 0x7f30c406c8f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f30d4036000 tx=0x7f30d403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:00.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.464+0000 7f30dbfff640 1 -- 172.21.15.27:0/1470729794 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f30e40af1d0 con 0x7f30e4070c00 2023-12-19T18:17:00.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.693+0000 7f30e96ce640 1 -- 172.21.15.27:0/1470729794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f30e4068e90 con 0x7f30e4070c00 2023-12-19T18:17:00.695 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:17:00.695+0000 7f8f3865c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:17:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.713+0000 7f30dbfff640 1 -- 172.21.15.27:0/1470729794 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v16) v1 ==== 161+0+0 (secure 0 0 0) 0x7f30cc053a40 con 0x7f30e4070c00 2023-12-19T18:17:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:17:00.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.716+0000 7f30e96ce640 1 -- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f30c406a430 msgr2=0x7f30c406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.716+0000 7f30e96ce640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f30c406a430 0x7f30c406c8f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f30d4036000 tx=0x7f30d403c040 comp rx=0 tx=0).stop 2023-12-19T18:17:00.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.716+0000 7f30e96ce640 1 -- 172.21.15.27:0/1470729794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30e4070c00 msgr2=0x7f30e40ada00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.716+0000 7f30e96ce640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30e4070c00 0x7f30e40ada00 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f30cc00c450 tx=0x7f30cc00c910 comp rx=0 tx=0).stop 2023-12-19T18:17:00.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.716+0000 7f30e96ce640 1 -- 172.21.15.27:0/1470729794 shutdown_connections 2023-12-19T18:17:00.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.716+0000 7f30e96ce640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f30c406a430 0x7f30c406c8f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.716+0000 7f30e96ce640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30e415adf0 0x7f30e40c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.716+0000 7f30e96ce640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30e4070c00 0x7f30e40ada00 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.716+0000 7f30e96ce640 1 --2- 172.21.15.27:0/1470729794 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30e406fc10 0x7f30e414df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.716+0000 7f30e96ce640 1 -- 172.21.15.27:0/1470729794 >> 172.21.15.27:0/1470729794 conn(0x7f30e40ceb50 msgr2=0x7f30e40cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:00.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.716+0000 7f30e96ce640 1 -- 172.21.15.27:0/1470729794 shutdown_connections 2023-12-19T18:17:00.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.717+0000 7f30e96ce640 1 -- 172.21.15.27:0/1470729794 wait complete. 2023-12-19T18:17:00.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:17:00.743 INFO:tasks.ceph.mds.a.smithi027.stderr: -3783> 2023-12-19T18:16:27.831+0000 7f3e248aa640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:17:00.811 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:17:00.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.949+0000 7f888d069640 1 Processor -- start 2023-12-19T18:17:00.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.949+0000 7f888d069640 1 -- start start 2023-12-19T18:17:00.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.951+0000 7f888d069640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f888815cf80 0x7f88880cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.951+0000 7f888d069640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f88880b05b0 0x7f88880b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.951+0000 7f888d069640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88880cb3c0 con 0x7f88880ca6b0 2023-12-19T18:17:00.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.951+0000 7f888d069640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88880b0f80 con 0x7f88880b05b0 2023-12-19T18:17:00.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.951+0000 7f888d069640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f888806cb20 con 0x7f888815cf80 2023-12-19T18:17:00.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.951+0000 7f888759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f88880b05b0 0x7f88880b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.951+0000 7f888759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f88880b05b0 0x7f88880b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41476/0 (socket says 172.21.15.27:41476) 2023-12-19T18:17:00.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.951+0000 7f888759e640 1 -- 172.21.15.27:0/800747619 learned_addr learned my addr 172.21.15.27:0/800747619 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:00.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.951+0000 7f888659c640 1 --1- 172.21.15.27:0/800747619 >> v1:172.21.15.27:6789/0 conn(0x7f88880ca6b0 0x7f8888161310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55308/0 (socket says 172.21.15.27:55308) 2023-12-19T18:17:00.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.951+0000 7f8886d9d640 1 --2- 172.21.15.27:0/800747619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f888815cf80 0x7f88880cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.952+0000 7f888759e640 1 -- 172.21.15.27:0/800747619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f888815cf80 msgr2=0x7f88880cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:00.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.952+0000 7f888759e640 1 --2- 172.21.15.27:0/800747619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f888815cf80 0x7f88880cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.952+0000 7f888759e640 1 -- 172.21.15.27:0/800747619 >> v1:172.21.15.27:6789/0 conn(0x7f88880ca6b0 legacy=0x7f8888161310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.952+0000 7f888759e640 1 -- 172.21.15.27:0/800747619 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8888160ba0 con 0x7f88880b05b0 2023-12-19T18:17:00.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.952+0000 7f8885d9b640 1 -- 172.21.15.27:0/800747619 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1899025539 0 0) 0x7f88880cb3c0 con 0x7f88880ca6b0 2023-12-19T18:17:00.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.953+0000 7f888759e640 1 --2- 172.21.15.27:0/800747619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88880b05b0 0x7f88880b0a40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8880009e50 tx=0x7f888002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efdf07fff794182f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:00.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.953+0000 7f8885d9b640 1 -- 172.21.15.27:0/800747619 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888002cb00 con 0x7f88880b05b0 2023-12-19T18:17:00.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.953+0000 7f8885d9b640 1 -- 172.21.15.27:0/800747619 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f888002cc60 con 0x7f88880b05b0 2023-12-19T18:17:00.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.953+0000 7f8885d9b640 1 -- 172.21.15.27:0/800747619 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88800343f0 con 0x7f88880b05b0 2023-12-19T18:17:00.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.954+0000 7f888d069640 1 -- 172.21.15.27:0/800747619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88880b05b0 msgr2=0x7f88880b0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.954+0000 7f888d069640 1 --2- 172.21.15.27:0/800747619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88880b05b0 0x7f88880b0a40 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f8880009e50 tx=0x7f888002bf60 comp rx=0 tx=0).stop 2023-12-19T18:17:00.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.954+0000 7f888d069640 1 -- 172.21.15.27:0/800747619 shutdown_connections 2023-12-19T18:17:00.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.954+0000 7f888d069640 1 --2- 172.21.15.27:0/800747619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88880b05b0 0x7f88880b0a40 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.954+0000 7f888d069640 1 --2- 172.21.15.27:0/800747619 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f888815cf80 0x7f88880cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.954+0000 7f888d069640 1 -- 172.21.15.27:0/800747619 >> 172.21.15.27:0/800747619 conn(0x7f88880cb5b0 msgr2=0x7f88880cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.954+0000 7f888d069640 1 -- 172.21.15.27:0/800747619 shutdown_connections 2023-12-19T18:17:00.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.954+0000 7f888d069640 1 -- 172.21.15.27:0/800747619 wait complete. 2023-12-19T18:17:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.956+0000 7f888d069640 1 Processor -- start 2023-12-19T18:17:00.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.956+0000 7f888d069640 1 -- start start 2023-12-19T18:17:00.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.957+0000 7f888d069640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88880b05b0 0x7f888814b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.957+0000 7f888d069640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88880ca6b0 0x7f888815fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.957+0000 7f888d069640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f888815cf80 0x7f88880b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.957+0000 7f888d069640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88880c4100 con 0x7f88880ca6b0 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.957+0000 7f888d069640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88880b1aa0 con 0x7f888815cf80 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.957+0000 7f888d069640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88880c3550 con 0x7f88880b05b0 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.957+0000 7f888659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88880ca6b0 0x7f888815fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.957+0000 7f888659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88880ca6b0 0x7f888815fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:41488/0 (socket says 172.21.15.27:41488) 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.957+0000 7f888659c640 1 -- 172.21.15.27:0/3926741544 learned_addr learned my addr 172.21.15.27:0/3926741544 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.958+0000 7f888659c640 1 -- 172.21.15.27:0/3926741544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88880b05b0 msgr2=0x7f888814b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.958+0000 7f8886d9d640 1 --2- 172.21.15.27:0/3926741544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88880b05b0 0x7f888814b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.958+0000 7f888659c640 1 --2- 172.21.15.27:0/3926741544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88880b05b0 0x7f888814b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.958+0000 7f888659c640 1 -- 172.21.15.27:0/3926741544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f888815cf80 msgr2=0x7f88880b1270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.958+0000 7f888659c640 1 --2- 172.21.15.27:0/3926741544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f888815cf80 0x7f88880b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.958+0000 7f888659c640 1 -- 172.21.15.27:0/3926741544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8880009b30 con 0x7f88880ca6b0 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.958+0000 7f888659c640 1 --2- 172.21.15.27:0/3926741544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88880ca6b0 0x7f888815fc90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c4d0 tx=0x7f887c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:00.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.958+0000 7f886f7fe640 1 -- 172.21.15.27:0/3926741544 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c0074e0 con 0x7f88880ca6b0 2023-12-19T18:17:00.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.958+0000 7f886f7fe640 1 -- 172.21.15.27:0/3926741544 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f887c007660 con 0x7f88880ca6b0 2023-12-19T18:17:00.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.958+0000 7f886f7fe640 1 -- 172.21.15.27:0/3926741544 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f887c004a20 con 0x7f88880ca6b0 2023-12-19T18:17:00.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.958+0000 7f888d069640 1 -- 172.21.15.27:0/3926741544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88881601d0 con 0x7f88880ca6b0 2023-12-19T18:17:00.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.959+0000 7f888d069640 1 -- 172.21.15.27:0/3926741544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88880c3a40 con 0x7f88880ca6b0 2023-12-19T18:17:00.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.959+0000 7f888d069640 1 -- 172.21.15.27:0/3926741544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88880c4240 con 0x7f88880ca6b0 2023-12-19T18:17:00.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.961+0000 7f886f7fe640 1 -- 172.21.15.27:0/3926741544 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f887c004b80 con 0x7f88880ca6b0 2023-12-19T18:17:00.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.962+0000 7f886f7fe640 1 --2- 172.21.15.27:0/3926741544 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f887406a430 0x7f887406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:00.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.962+0000 7f886f7fe640 1 -- 172.21.15.27:0/3926741544 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f887c087bd0 con 0x7f88880ca6b0 2023-12-19T18:17:00.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.962+0000 7f8886d9d640 1 --2- 172.21.15.27:0/3926741544 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f887406a430 0x7f887406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:00.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.963+0000 7f8886d9d640 1 --2- 172.21.15.27:0/3926741544 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f887406a430 0x7f887406c8f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8878006fd0 tx=0x7f8878008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:00.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:00.966+0000 7f886f7fe640 1 -- 172.21.15.27:0/3926741544 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f88880c4240 con 0x7f88880ca6b0 2023-12-19T18:17:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.201+0000 7f888d069640 1 -- 172.21.15.27:0/3926741544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f88880c4240 con 0x7f88880ca6b0 2023-12-19T18:17:01.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.797+0000 7f886f7fe640 1 -- 172.21.15.27:0/3926741544 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f88880c4240 con 0x7f88880ca6b0 2023-12-19T18:17:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.800+0000 7f888d069640 1 -- 172.21.15.27:0/3926741544 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f887406a430 msgr2=0x7f887406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.800+0000 7f888d069640 1 --2- 172.21.15.27:0/3926741544 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f887406a430 0x7f887406c8f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8878006fd0 tx=0x7f8878008040 comp rx=0 tx=0).stop 2023-12-19T18:17:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.800+0000 7f888d069640 1 -- 172.21.15.27:0/3926741544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88880ca6b0 msgr2=0x7f888815fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.800+0000 7f888d069640 1 --2- 172.21.15.27:0/3926741544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88880ca6b0 0x7f888815fc90 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f887c00c4d0 tx=0x7f887c00c990 comp rx=0 tx=0).stop 2023-12-19T18:17:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.800+0000 7f888d069640 1 -- 172.21.15.27:0/3926741544 shutdown_connections 2023-12-19T18:17:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.800+0000 7f888d069640 1 --2- 172.21.15.27:0/3926741544 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f887406a430 0x7f887406c8f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.800+0000 7f888d069640 1 --2- 172.21.15.27:0/3926741544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f888815cf80 0x7f88880b1270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.800+0000 7f888d069640 1 --2- 172.21.15.27:0/3926741544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88880ca6b0 0x7f888815fc90 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.800+0000 7f888d069640 1 --2- 172.21.15.27:0/3926741544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f88880b05b0 0x7f888814b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.800+0000 7f888d069640 1 -- 172.21.15.27:0/3926741544 >> 172.21.15.27:0/3926741544 conn(0x7f88880cb5b0 msgr2=0x7f88880cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:01.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.801+0000 7f888d069640 1 -- 172.21.15.27:0/3926741544 shutdown_connections 2023-12-19T18:17:01.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:01.801+0000 7f888d069640 1 -- 172.21.15.27:0/3926741544 wait complete. 2023-12-19T18:17:01.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:17:02.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.058+0000 7fed3c973640 1 Processor -- start 2023-12-19T18:17:02.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.058+0000 7fed3c973640 1 -- start start 2023-12-19T18:17:02.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.060+0000 7fed3c973640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed3815dfe0 0x7fed380cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.060+0000 7fed3c973640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fed380cb670 0x7fed380c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.060+0000 7fed3c973640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed38160920 con 0x7fed3815dc00 2023-12-19T18:17:02.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.060+0000 7fed3c973640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed380cbae0 con 0x7fed380cb670 2023-12-19T18:17:02.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.060+0000 7fed3c973640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed380cd220 con 0x7fed3815dfe0 2023-12-19T18:17:02.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.061+0000 7fed377fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fed3815dc00 0x7fed380cc6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55026/0 (socket says 172.21.15.27:55026) 2023-12-19T18:17:02.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.061+0000 7fed377fe640 1 -- 172.21.15.27:0/2450549191 learned_addr learned my addr 172.21.15.27:0/2450549191 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:02.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.061+0000 7fed37fff640 1 --2- 172.21.15.27:0/2450549191 >> v2:172.21.15.27:3300/0 conn(0x7fed380cb670 0x7fed380c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.061+0000 7fed36ffd640 1 --2- 172.21.15.27:0/2450549191 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed3815dfe0 0x7fed380cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.061+0000 7fed367fc640 1 -- 172.21.15.27:0/2450549191 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807611555 0 0) 0x7fed38160920 con 0x7fed3815dc00 2023-12-19T18:17:02.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.061+0000 7fed367fc640 1 -- 172.21.15.27:0/2450549191 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed20003520 con 0x7fed3815dc00 2023-12-19T18:17:02.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.061+0000 7fed37fff640 1 -- 172.21.15.27:0/2450549191 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed3815dfe0 msgr2=0x7fed380cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:02.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.061+0000 7fed37fff640 1 --2- 172.21.15.27:0/2450549191 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed3815dfe0 0x7fed380cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.061+0000 7fed37fff640 1 -- 172.21.15.27:0/2450549191 >> v1:172.21.15.27:6789/0 conn(0x7fed3815dc00 legacy=0x7fed380cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.061+0000 7fed37fff640 1 -- 172.21.15.27:0/2450549191 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed38160be0 con 0x7fed380cb670 2023-12-19T18:17:02.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.063+0000 7fed37fff640 1 --2- 172.21.15.27:0/2450549191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed380cb670 0x7fed380c8f90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fed30009d90 tx=0x7fed3002bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0f2c26b3fc81d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:02.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.063+0000 7fed367fc640 1 -- 172.21.15.27:0/2450549191 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed3002cb40 con 0x7fed380cb670 2023-12-19T18:17:02.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.063+0000 7fed367fc640 1 -- 172.21.15.27:0/2450549191 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed3002cca0 con 0x7fed380cb670 2023-12-19T18:17:02.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.063+0000 7fed367fc640 1 -- 172.21.15.27:0/2450549191 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed30034660 con 0x7fed380cb670 2023-12-19T18:17:02.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.064+0000 7fed3c973640 1 -- 172.21.15.27:0/2450549191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed380cb670 msgr2=0x7fed380c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.064+0000 7fed3c973640 1 --2- 172.21.15.27:0/2450549191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed380cb670 0x7fed380c8f90 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7fed30009d90 tx=0x7fed3002bd90 comp rx=0 tx=0).stop 2023-12-19T18:17:02.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.064+0000 7fed3c973640 1 -- 172.21.15.27:0/2450549191 shutdown_connections 2023-12-19T18:17:02.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.064+0000 7fed3c973640 1 --2- 172.21.15.27:0/2450549191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed380cb670 0x7fed380c8f90 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.064+0000 7fed3c973640 1 --2- 172.21.15.27:0/2450549191 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed3815dfe0 0x7fed380cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.064+0000 7fed3c973640 1 -- 172.21.15.27:0/2450549191 >> 172.21.15.27:0/2450549191 conn(0x7fed380b05b0 msgr2=0x7fed380b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:02.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.064+0000 7fed3c973640 1 -- 172.21.15.27:0/2450549191 shutdown_connections 2023-12-19T18:17:02.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.064+0000 7fed3c973640 1 -- 172.21.15.27:0/2450549191 wait complete. 2023-12-19T18:17:02.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.066+0000 7fed3c973640 1 Processor -- start 2023-12-19T18:17:02.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.066+0000 7fed3c973640 1 -- start start 2023-12-19T18:17:02.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.067+0000 7fed3c973640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed380cb670 0x7fed38156d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.067+0000 7fed3c973640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed3815dc00 0x7fed380c4c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.067+0000 7fed3c973640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3815dfe0 0x7fed38154a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.067+0000 7fed3c973640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed38150400 con 0x7fed3815dc00 2023-12-19T18:17:02.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.067+0000 7fed3c973640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed381509c0 con 0x7fed3815dfe0 2023-12-19T18:17:02.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.067+0000 7fed3c973640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed380c5170 con 0x7fed380cb670 2023-12-19T18:17:02.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.067+0000 7fed36ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed3815dc00 0x7fed380c4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.067+0000 7fed36ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed3815dc00 0x7fed380c4c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38046/0 (socket says 172.21.15.27:38046) 2023-12-19T18:17:02.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.067+0000 7fed36ffd640 1 -- 172.21.15.27:0/1618165554 learned_addr learned my addr 172.21.15.27:0/1618165554 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:02.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.068+0000 7fed377fe640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed380cb670 0x7fed38156d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.068+0000 7fed37fff640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3815dfe0 0x7fed38154a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.068+0000 7fed36ffd640 1 -- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed380cb670 msgr2=0x7fed38156d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.068+0000 7fed36ffd640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed380cb670 0x7fed38156d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.068+0000 7fed36ffd640 1 -- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3815dfe0 msgr2=0x7fed38154a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.068+0000 7fed36ffd640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3815dfe0 0x7fed38154a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.068+0000 7fed36ffd640 1 -- 172.21.15.27:0/1618165554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed30009a70 con 0x7fed3815dc00 2023-12-19T18:17:02.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.068+0000 7fed36ffd640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed3815dc00 0x7fed380c4c30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00d620 tx=0x7fed2c00dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:02.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.068+0000 7fed1ffff640 1 -- 172.21.15.27:0/1618165554 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c00b620 con 0x7fed3815dc00 2023-12-19T18:17:02.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.068+0000 7fed1ffff640 1 -- 172.21.15.27:0/1618165554 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2c00b780 con 0x7fed3815dc00 2023-12-19T18:17:02.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.068+0000 7fed1ffff640 1 -- 172.21.15.27:0/1618165554 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2c00bb30 con 0x7fed3815dc00 2023-12-19T18:17:02.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.068+0000 7fed3c973640 1 -- 172.21.15.27:0/1618165554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed380c3ae0 con 0x7fed3815dc00 2023-12-19T18:17:02.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.069+0000 7fed3c973640 1 -- 172.21.15.27:0/1618165554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed38150650 con 0x7fed3815dc00 2023-12-19T18:17:02.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.070+0000 7fed3c973640 1 -- 172.21.15.27:0/1618165554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3804e840 con 0x7fed3815dc00 2023-12-19T18:17:02.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.071+0000 7fed1ffff640 1 -- 172.21.15.27:0/1618165554 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fed2c00bc90 con 0x7fed3815dc00 2023-12-19T18:17:02.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.072+0000 7fed1ffff640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed2406a430 0x7fed2406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:02.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.072+0000 7fed1ffff640 1 -- 172.21.15.27:0/1618165554 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 5911+0+0 (secure 0 0 0) 0x7fed2c088f80 con 0x7fed3815dc00 2023-12-19T18:17:02.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.072+0000 7fed377fe640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed2406a430 0x7fed2406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:02.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.073+0000 7fed377fe640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed2406a430 0x7fed2406c8f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fed280011e0 tx=0x7fed28009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:02.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.076+0000 7fed1ffff640 1 -- 172.21.15.27:0/1618165554 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fed2c054400 con 0x7fed3815dc00 2023-12-19T18:17:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.272+0000 7fed3c973640 1 -- 172.21.15.27:0/1618165554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fed380c4060 con 0x7fed3815dc00 2023-12-19T18:17:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.822+0000 7fed1ffff640 1 -- 172.21.15.27:0/1618165554 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v40) v1 ==== 181+0+0 (secure 0 0 0) 0x7fed2c057a20 con 0x7fed3815dc00 2023-12-19T18:17:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:17:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.828+0000 7fed3c973640 1 -- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed2406a430 msgr2=0x7fed2406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.828+0000 7fed3c973640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed2406a430 0x7fed2406c8f0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fed280011e0 tx=0x7fed28009040 comp rx=0 tx=0).stop 2023-12-19T18:17:02.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.828+0000 7fed3c973640 1 -- 172.21.15.27:0/1618165554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed3815dc00 msgr2=0x7fed380c4c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:02.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.828+0000 7fed3c973640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed3815dc00 0x7fed380c4c30 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fed2c00d620 tx=0x7fed2c00dae0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.828+0000 7fed3c973640 1 -- 172.21.15.27:0/1618165554 shutdown_connections 2023-12-19T18:17:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.828+0000 7fed3c973640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed2406a430 0x7fed2406c8f0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.828+0000 7fed3c973640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed3815dfe0 0x7fed38154a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.828+0000 7fed3c973640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed3815dc00 0x7fed380c4c30 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.828+0000 7fed3c973640 1 --2- 172.21.15.27:0/1618165554 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed380cb670 0x7fed38156d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.828+0000 7fed3c973640 1 -- 172.21.15.27:0/1618165554 >> 172.21.15.27:0/1618165554 conn(0x7fed380b05b0 msgr2=0x7fed3806c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.828+0000 7fed3c973640 1 -- 172.21.15.27:0/1618165554 shutdown_connections 2023-12-19T18:17:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:02.828+0000 7fed3c973640 1 -- 172.21.15.27:0/1618165554 wait complete. 2023-12-19T18:17:02.850 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:17:03.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.072+0000 7f75a34af640 1 Processor -- start 2023-12-19T18:17:03.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.073+0000 7f75a34af640 1 -- start start 2023-12-19T18:17:03.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.074+0000 7f75a34af640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f759c15b0d0 0x7f759c0cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.074+0000 7f75a34af640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f759c0cbb60 0x7f759c0c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.074+0000 7f75a34af640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759c070fb0 con 0x7f759c15b4b0 2023-12-19T18:17:03.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.075+0000 7f75a34af640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759c06fab0 con 0x7f759c0cbb60 2023-12-19T18:17:03.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.075+0000 7f75a34af640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759c06f4f0 con 0x7f759c15b0d0 2023-12-19T18:17:03.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.075+0000 7f75a1a4c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f759c0cbb60 0x7f759c0c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.075+0000 7f75a0a4a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f759c15b4b0 0x7f759c0cb450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55028/0 (socket says 172.21.15.27:55028) 2023-12-19T18:17:03.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.075+0000 7f75a0a4a640 1 -- 172.21.15.27:0/2482388387 learned_addr learned my addr 172.21.15.27:0/2482388387 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:03.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.075+0000 7f75a124b640 1 --2- 172.21.15.27:0/2482388387 >> v2:172.21.15.192:3300/0 conn(0x7f759c15b0d0 0x7f759c0cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.075+0000 7f7593fff640 1 -- 172.21.15.27:0/2482388387 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637629 0 0) 0x7f759c070fb0 con 0x7f759c15b4b0 2023-12-19T18:17:03.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.075+0000 7f75a1a4c640 1 -- 172.21.15.27:0/2482388387 >> v2:172.21.15.192:3300/0 conn(0x7f759c15b0d0 msgr2=0x7f759c0cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:03.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.075+0000 7f75a1a4c640 1 --2- 172.21.15.27:0/2482388387 >> v2:172.21.15.192:3300/0 conn(0x7f759c15b0d0 0x7f759c0cabf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.075+0000 7f75a1a4c640 1 -- 172.21.15.27:0/2482388387 >> v1:172.21.15.27:6789/0 conn(0x7f759c15b4b0 legacy=0x7f759c0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.076+0000 7f75a1a4c640 1 -- 172.21.15.27:0/2482388387 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c06f7d0 con 0x7f759c0cbb60 2023-12-19T18:17:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.077+0000 7f75a1a4c640 1 --2- 172.21.15.27:0/2482388387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f759c0cbb60 0x7f759c0c91c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7594009d90 tx=0x7f759402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d2779c440f21e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.077+0000 7f7593fff640 1 -- 172.21.15.27:0/2482388387 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759402cd00 con 0x7f759c0cbb60 2023-12-19T18:17:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.077+0000 7f7593fff640 1 -- 172.21.15.27:0/2482388387 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759402ce60 con 0x7f759c0cbb60 2023-12-19T18:17:03.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.077+0000 7f7593fff640 1 -- 172.21.15.27:0/2482388387 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7594034660 con 0x7f759c0cbb60 2023-12-19T18:17:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.078+0000 7f75a34af640 1 -- 172.21.15.27:0/2482388387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f759c0cbb60 msgr2=0x7f759c0c91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.078+0000 7f75a34af640 1 --2- 172.21.15.27:0/2482388387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f759c0cbb60 0x7f759c0c91c0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f7594009d90 tx=0x7f759402bf50 comp rx=0 tx=0).stop 2023-12-19T18:17:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.078+0000 7f75a34af640 1 -- 172.21.15.27:0/2482388387 shutdown_connections 2023-12-19T18:17:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.078+0000 7f75a34af640 1 --2- 172.21.15.27:0/2482388387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f759c0cbb60 0x7f759c0c91c0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.078+0000 7f75a34af640 1 --2- 172.21.15.27:0/2482388387 >> v2:172.21.15.192:3300/0 conn(0x7f759c15b0d0 0x7f759c0cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.078+0000 7f75a34af640 1 -- 172.21.15.27:0/2482388387 >> 172.21.15.27:0/2482388387 conn(0x7f759c0b0620 msgr2=0x7f759c0b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:03.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.078+0000 7f75a34af640 1 -- 172.21.15.27:0/2482388387 shutdown_connections 2023-12-19T18:17:03.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.078+0000 7f75a34af640 1 -- 172.21.15.27:0/2482388387 wait complete. 2023-12-19T18:17:03.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.080+0000 7f75a34af640 1 Processor -- start 2023-12-19T18:17:03.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.080+0000 7f75a34af640 1 -- start start 2023-12-19T18:17:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.081+0000 7f75a34af640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c0cbb60 0x7f759c0cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.081+0000 7f75a34af640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f759c15b0d0 0x7f759c152250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.081+0000 7f75a34af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f759c15b4b0 0x7f759c152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.081+0000 7f75a34af640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0c6df0 con 0x7f759c15b4b0 2023-12-19T18:17:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.081+0000 7f75a34af640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0c70d0 con 0x7f759c15b0d0 2023-12-19T18:17:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.081+0000 7f75a34af640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0c73b0 con 0x7f759c0cbb60 2023-12-19T18:17:03.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.081+0000 7f75a1a4c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f759c15b4b0 0x7f759c152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.081+0000 7f75a1a4c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f759c15b4b0 0x7f759c152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38072/0 (socket says 172.21.15.27:38072) 2023-12-19T18:17:03.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.081+0000 7f75a1a4c640 1 -- 172.21.15.27:0/1370661511 learned_addr learned my addr 172.21.15.27:0/1370661511 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:03.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.081+0000 7f75a124b640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c0cbb60 0x7f759c0cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.082+0000 7f75a1a4c640 1 -- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c0cbb60 msgr2=0x7f759c0cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.082+0000 7f75a1a4c640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c0cbb60 0x7f759c0cd790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.082+0000 7f75a1a4c640 1 -- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f759c15b0d0 msgr2=0x7f759c152250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.082+0000 7f75a0a4a640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f759c15b0d0 0x7f759c152250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.082+0000 7f75a1a4c640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f759c15b0d0 0x7f759c152250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.082+0000 7f75a1a4c640 1 -- 172.21.15.27:0/1370661511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7594009a70 con 0x7f759c15b4b0 2023-12-19T18:17:03.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.082+0000 7f75a0a4a640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f759c15b0d0 0x7f759c152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:03.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.082+0000 7f75a1a4c640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f759c15b4b0 0x7f759c152790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7594009d90 tx=0x7f7594004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:03.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.082+0000 7f7591ffb640 1 -- 172.21.15.27:0/1370661511 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759403d600 con 0x7f759c15b4b0 2023-12-19T18:17:03.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.082+0000 7f7591ffb640 1 -- 172.21.15.27:0/1370661511 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f759403d780 con 0x7f759c15b4b0 2023-12-19T18:17:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.082+0000 7f7591ffb640 1 -- 172.21.15.27:0/1370661511 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f759403db30 con 0x7f759c15b4b0 2023-12-19T18:17:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.082+0000 7f75a34af640 1 -- 172.21.15.27:0/1370661511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759c0c7690 con 0x7f759c15b4b0 2023-12-19T18:17:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.083+0000 7f75a34af640 1 -- 172.21.15.27:0/1370661511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c0c7c50 con 0x7f759c15b4b0 2023-12-19T18:17:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.084+0000 7f75a34af640 1 -- 172.21.15.27:0/1370661511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759c068e90 con 0x7f759c15b4b0 2023-12-19T18:17:03.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.085+0000 7f7591ffb640 1 -- 172.21.15.27:0/1370661511 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f759403b040 con 0x7f759c15b4b0 2023-12-19T18:17:03.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.086+0000 7f7591ffb640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f758006a430 0x7f758006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:03.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.086+0000 7f7591ffb640 1 -- 172.21.15.27:0/1370661511 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f75940aae70 con 0x7f759c15b4b0 2023-12-19T18:17:03.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.087+0000 7f75a124b640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f758006a430 0x7f758006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:03.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.088+0000 7f75a124b640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f758006a430 0x7f758006c8f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7598006fd0 tx=0x7f7598008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:03.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.091+0000 7f7591ffb640 1 -- 172.21.15.27:0/1370661511 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7594076540 con 0x7f759c15b4b0 2023-12-19T18:17:03.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.284+0000 7f75a34af640 1 -- 172.21.15.27:0/1370661511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f759c14a4c0 con 0x7f759c15b4b0 2023-12-19T18:17:03.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.826+0000 7f7591ffb640 1 -- 172.21.15.27:0/1370661511 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v41) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7594079b60 con 0x7f759c15b4b0 2023-12-19T18:17:03.828 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-12-19T18:17:03.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.833+0000 7f75a34af640 1 -- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f758006a430 msgr2=0x7f758006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.833+0000 7f75a34af640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f758006a430 0x7f758006c8f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7598006fd0 tx=0x7f7598008040 comp rx=0 tx=0).stop 2023-12-19T18:17:03.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.833+0000 7f75a34af640 1 -- 172.21.15.27:0/1370661511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f759c15b4b0 msgr2=0x7f759c152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:03.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.833+0000 7f75a34af640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f759c15b4b0 0x7f759c152790 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f7594009d90 tx=0x7f7594004a40 comp rx=0 tx=0).stop 2023-12-19T18:17:03.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.834+0000 7f75a34af640 1 -- 172.21.15.27:0/1370661511 shutdown_connections 2023-12-19T18:17:03.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.834+0000 7f75a34af640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f758006a430 0x7f758006c8f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.834+0000 7f75a34af640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f759c15b4b0 0x7f759c152790 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.834+0000 7f75a34af640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f759c15b0d0 0x7f759c152250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.834+0000 7f75a34af640 1 --2- 172.21.15.27:0/1370661511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f759c0cbb60 0x7f759c0cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:03.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.834+0000 7f75a34af640 1 -- 172.21.15.27:0/1370661511 >> 172.21.15.27:0/1370661511 conn(0x7f759c0b0620 msgr2=0x7f759c0c3e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:03.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.834+0000 7f75a34af640 1 -- 172.21.15.27:0/1370661511 shutdown_connections 2023-12-19T18:17:03.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:03.835+0000 7f75a34af640 1 -- 172.21.15.27:0/1370661511 wait complete. 2023-12-19T18:17:03.856 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:17:04.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.083+0000 7f3c3ffff640 1 Processor -- start 2023-12-19T18:17:04.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.083+0000 7f3c3ffff640 1 -- start start 2023-12-19T18:17:04.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.085+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c40158790 0x7f3c400ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.085+0000 7f3c3ffff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c4015c030 con 0x7f3c400adc50 2023-12-19T18:17:04.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.085+0000 7f3c3ffff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c400c9030 con 0x7f3c4015b9e0 2023-12-19T18:17:04.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.085+0000 7f3c3ffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400c99d0 con 0x7f3c40158790 2023-12-19T18:17:04.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.086+0000 7f3c3e7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3c400adc50 0x7f3c4015b2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55030/0 (socket says 172.21.15.27:55030) 2023-12-19T18:17:04.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.086+0000 7f3c3e7fc640 1 -- 172.21.15.27:0/768708397 learned_addr learned my addr 172.21.15.27:0/768708397 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:04.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.086+0000 7f3c3effd640 1 --2- 172.21.15.27:0/768708397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c40158790 0x7f3c400ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.086+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3854031312 0 0) 0x7f3c4015c030 con 0x7f3c400adc50 2023-12-19T18:17:04.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.086+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c28003520 con 0x7f3c400adc50 2023-12-19T18:17:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.086+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3142942252 0 0) 0x7f3c28003520 con 0x7f3c400adc50 2023-12-19T18:17:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.086+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c40158790 msgr2=0x7f3c400ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.086+0000 7f3c3dffb640 1 --2- 172.21.15.27:0/768708397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c40158790 0x7f3c400ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.086+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 >> v1:172.21.15.192:6789/0 conn(0x7f3c4015b9e0 legacy=0x7f3c400c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.086+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400c9310 con 0x7f3c400adc50 2023-12-19T18:17:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.087+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f3c34001670 con 0x7f3c400adc50 2023-12-19T18:17:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.087+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 >> v1:172.21.15.27:6789/0 conn(0x7f3c400adc50 legacy=0x7f3c4015b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.087+0000 7f3c3dffb640 1 --2- 172.21.15.27:0/768708397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c40158790 0x7f3c280039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.087+0000 7f3c3dffb640 1 --2- 172.21.15.27:0/768708397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c28003f20 0x7f3c280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.087+0000 7f3c3dffb640 1 --2- 172.21.15.27:0/768708397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c28004d00 0x7f3c28005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.087+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400c99d0 con 0x7f3c28004d00 2023-12-19T18:17:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.087+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2800ba60 con 0x7f3c28003f20 2023-12-19T18:17:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.087+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c2800bc20 con 0x7f3c40158790 2023-12-19T18:17:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.087+0000 7f3c3f7fe640 1 --2- 172.21.15.27:0/768708397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c28004d00 0x7f3c28005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.087+0000 7f3c3e7fc640 1 --2- 172.21.15.27:0/768708397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c28003f20 0x7f3c280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.087+0000 7f3c3effd640 1 --2- 172.21.15.27:0/768708397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c40158790 0x7f3c280039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.087+0000 7f3c3f7fe640 1 -- 172.21.15.27:0/768708397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c40158790 msgr2=0x7f3c280039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.087+0000 7f3c3f7fe640 1 --2- 172.21.15.27:0/768708397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c40158790 0x7f3c280039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.088+0000 7f3c3f7fe640 1 -- 172.21.15.27:0/768708397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c28003f20 msgr2=0x7f3c280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.088+0000 7f3c3f7fe640 1 --2- 172.21.15.27:0/768708397 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c28003f20 0x7f3c280047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.088+0000 7f3c3f7fe640 1 -- 172.21.15.27:0/768708397 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c28003520 con 0x7f3c28004d00 2023-12-19T18:17:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.088+0000 7f3c3effd640 1 --2- 172.21.15.27:0/768708397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c40158790 0x7f3c280039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:04.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.089+0000 7f3c3f7fe640 1 --2- 172.21.15.27:0/768708397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c28004d00 0x7f3c28005180 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3c38001e50 tx=0x7f3c280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e380db42b3762438 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:04.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.089+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38003b70 con 0x7f3c28004d00 2023-12-19T18:17:04.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.089+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c38003cd0 con 0x7f3c28004d00 2023-12-19T18:17:04.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.089+0000 7f3c3dffb640 1 -- 172.21.15.27:0/768708397 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c38005310 con 0x7f3c28004d00 2023-12-19T18:17:04.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.090+0000 7f3c3ffff640 1 -- 172.21.15.27:0/768708397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c28004d00 msgr2=0x7f3c28005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.090+0000 7f3c3ffff640 1 --2- 172.21.15.27:0/768708397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c28004d00 0x7f3c28005180 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f3c38001e50 tx=0x7f3c280057f0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.090+0000 7f3c3f7fe640 1 -- 172.21.15.27:0/768708397 reap_dead start 2023-12-19T18:17:04.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.090+0000 7f3c3ffff640 1 -- 172.21.15.27:0/768708397 shutdown_connections 2023-12-19T18:17:04.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.090+0000 7f3c3ffff640 1 -- 172.21.15.27:0/768708397 >> 172.21.15.27:0/768708397 conn(0x7f3c400b05b0 msgr2=0x7f3c400b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:04.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.090+0000 7f3c3ffff640 1 -- 172.21.15.27:0/768708397 shutdown_connections 2023-12-19T18:17:04.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.090+0000 7f3c3ffff640 1 -- 172.21.15.27:0/768708397 wait complete. 2023-12-19T18:17:04.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.092+0000 7f3c3ffff640 1 Processor -- start 2023-12-19T18:17:04.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.093+0000 7f3c3ffff640 1 -- start start 2023-12-19T18:17:04.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.094+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4014c320 0x7f3c4006b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.094+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c4014c700 0x7f3c4006b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.094+0000 7f3c3ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c400c8680 0x7f3c4006bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.094+0000 7f3c3ffff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400c63b0 con 0x7f3c4014c320 2023-12-19T18:17:04.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.094+0000 7f3c3ffff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400b4860 con 0x7f3c4014c700 2023-12-19T18:17:04.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.094+0000 7f3c3ffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c400b4580 con 0x7f3c400c8680 2023-12-19T18:17:04.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.094+0000 7f3c3effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4014c320 0x7f3c4006b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.094+0000 7f3c3effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4014c320 0x7f3c4006b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38092/0 (socket says 172.21.15.27:38092) 2023-12-19T18:17:04.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.094+0000 7f3c3effd640 1 -- 172.21.15.27:0/2403577396 learned_addr learned my addr 172.21.15.27:0/2403577396 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:04.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.095+0000 7f3c3effd640 1 -- 172.21.15.27:0/2403577396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c400c8680 msgr2=0x7f3c4006bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.095+0000 7f3c3effd640 1 --2- 172.21.15.27:0/2403577396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c400c8680 0x7f3c4006bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.095+0000 7f3c3effd640 1 -- 172.21.15.27:0/2403577396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c4014c700 msgr2=0x7f3c4006b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.095+0000 7f3c3effd640 1 --2- 172.21.15.27:0/2403577396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c4014c700 0x7f3c4006b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.095+0000 7f3c3effd640 1 -- 172.21.15.27:0/2403577396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400c5d60 con 0x7f3c4014c320 2023-12-19T18:17:04.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.095+0000 7f3c3effd640 1 --2- 172.21.15.27:0/2403577396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4014c320 0x7f3c4006b2e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3c300108b0 tx=0x7f3c30010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:04.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.095+0000 7f3c277fe640 1 -- 172.21.15.27:0/2403577396 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3000eae0 con 0x7f3c4014c320 2023-12-19T18:17:04.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.095+0000 7f3c277fe640 1 -- 172.21.15.27:0/2403577396 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c3000ec40 con 0x7f3c4014c320 2023-12-19T18:17:04.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.095+0000 7f3c277fe640 1 -- 172.21.15.27:0/2403577396 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3001c400 con 0x7f3c4014c320 2023-12-19T18:17:04.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.095+0000 7f3c3ffff640 1 -- 172.21.15.27:0/2403577396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c400c6040 con 0x7f3c4014c320 2023-12-19T18:17:04.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.096+0000 7f3c3ffff640 1 -- 172.21.15.27:0/2403577396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c400b3300 con 0x7f3c4014c320 2023-12-19T18:17:04.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.097+0000 7f3c3ffff640 1 -- 172.21.15.27:0/2403577396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c4004e840 con 0x7f3c4014c320 2023-12-19T18:17:04.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.098+0000 7f3c277fe640 1 -- 172.21.15.27:0/2403577396 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3c3001c560 con 0x7f3c4014c320 2023-12-19T18:17:04.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.099+0000 7f3c277fe640 1 --2- 172.21.15.27:0/2403577396 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3c2c06a460 0x7f3c2c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.099+0000 7f3c277fe640 1 -- 172.21.15.27:0/2403577396 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3c30088b50 con 0x7f3c4014c320 2023-12-19T18:17:04.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.100+0000 7f3c3e7fc640 1 --2- 172.21.15.27:0/2403577396 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3c2c06a460 0x7f3c2c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.101+0000 7f3c3e7fc640 1 --2- 172.21.15.27:0/2403577396 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3c2c06a460 0x7f3c2c06c920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3c400c3ff0 tx=0x7f3c34009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.103+0000 7f3c277fe640 1 -- 172.21.15.27:0/2403577396 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3c300544f0 con 0x7f3c4014c320 2023-12-19T18:17:04.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.303+0000 7f3c3ffff640 1 -- 172.21.15.27:0/2403577396 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3c400c4e20 con 0x7f3c4014c320 2023-12-19T18:17:04.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.311+0000 7f3c277fe640 1 -- 172.21.15.27:0/2403577396 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v3) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3c30057b10 con 0x7f3c4014c320 2023-12-19T18:17:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.314+0000 7f3c3ffff640 1 -- 172.21.15.27:0/2403577396 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3c2c06a460 msgr2=0x7f3c2c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.314+0000 7f3c3ffff640 1 --2- 172.21.15.27:0/2403577396 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3c2c06a460 0x7f3c2c06c920 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3c400c3ff0 tx=0x7f3c34009040 comp rx=0 tx=0).stop 2023-12-19T18:17:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.314+0000 7f3c3ffff640 1 -- 172.21.15.27:0/2403577396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4014c320 msgr2=0x7f3c4006b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.314+0000 7f3c3ffff640 1 --2- 172.21.15.27:0/2403577396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4014c320 0x7f3c4006b2e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3c300108b0 tx=0x7f3c30010d70 comp rx=0 tx=0).stop 2023-12-19T18:17:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.314+0000 7f3c3ffff640 1 -- 172.21.15.27:0/2403577396 shutdown_connections 2023-12-19T18:17:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.314+0000 7f3c3ffff640 1 --2- 172.21.15.27:0/2403577396 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3c2c06a460 0x7f3c2c06c920 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.314+0000 7f3c3ffff640 1 --2- 172.21.15.27:0/2403577396 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3c400c8680 0x7f3c4006bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.314+0000 7f3c3ffff640 1 --2- 172.21.15.27:0/2403577396 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3c4014c700 0x7f3c4006b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.314+0000 7f3c3ffff640 1 --2- 172.21.15.27:0/2403577396 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c4014c320 0x7f3c4006b2e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.314+0000 7f3c3ffff640 1 -- 172.21.15.27:0/2403577396 >> 172.21.15.27:0/2403577396 conn(0x7f3c400b05b0 msgr2=0x7f3c400cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.315+0000 7f3c3ffff640 1 -- 172.21.15.27:0/2403577396 shutdown_connections 2023-12-19T18:17:04.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.315+0000 7f3c3ffff640 1 -- 172.21.15.27:0/2403577396 wait complete. 2023-12-19T18:17:04.333 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier' 2023-12-19T18:17:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.561+0000 7f33ea4ee640 1 Processor -- start 2023-12-19T18:17:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.561+0000 7f33ea4ee640 1 -- start start 2023-12-19T18:17:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.562+0000 7f33ea4ee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33e41604d0 0x7f33e40b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.562+0000 7f33ea4ee640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f33e40d2350 0x7f33e40b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.562+0000 7f33ea4ee640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e40b5e90 con 0x7f33e40d0300 2023-12-19T18:17:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.562+0000 7f33ea4ee640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e40d2b30 con 0x7f33e40d2350 2023-12-19T18:17:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.562+0000 7f33ea4ee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40d2850 con 0x7f33e41604d0 2023-12-19T18:17:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.563+0000 7f33e8ceb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f33e40d2350 0x7f33e40b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.563+0000 7f33e94ec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33e41604d0 0x7f33e40b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.563+0000 7f33e8ceb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f33e40d2350 0x7f33e40b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:33602/0 (socket says 172.21.15.27:33602) 2023-12-19T18:17:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.563+0000 7f33e8ceb640 1 -- 172.21.15.27:0/1903198757 learned_addr learned my addr 172.21.15.27:0/1903198757 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.564+0000 7f33e8ceb640 1 -- 172.21.15.27:0/1903198757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33e41604d0 msgr2=0x7f33e40b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.564+0000 7f33e8ceb640 1 --2- 172.21.15.27:0/1903198757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33e41604d0 0x7f33e40b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.564+0000 7f33e8ceb640 1 -- 172.21.15.27:0/1903198757 >> v1:172.21.15.192:6789/0 conn(0x7f33e40d0300 legacy=0x7f33e40d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.564+0000 7f33e8ceb640 1 -- 172.21.15.27:0/1903198757 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e40cded0 con 0x7f33e40d2350 2023-12-19T18:17:04.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.566+0000 7f33e8ceb640 1 --2- 172.21.15.27:0/1903198757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e40d2350 0x7f33e40b4d20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009870 tx=0x7f33d802b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57a07ca1742c0baa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:04.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.566+0000 7f33e3fff640 1 -- 172.21.15.27:0/1903198757 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8030840 con 0x7f33e40d2350 2023-12-19T18:17:04.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.566+0000 7f33e3fff640 1 -- 172.21.15.27:0/1903198757 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d80309a0 con 0x7f33e40d2350 2023-12-19T18:17:04.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.566+0000 7f33e3fff640 1 -- 172.21.15.27:0/1903198757 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8030d30 con 0x7f33e40d2350 2023-12-19T18:17:04.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.567+0000 7f33ea4ee640 1 -- 172.21.15.27:0/1903198757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e40d2350 msgr2=0x7f33e40b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.567+0000 7f33ea4ee640 1 --2- 172.21.15.27:0/1903198757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e40d2350 0x7f33e40b4d20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f33d8009870 tx=0x7f33d802b980 comp rx=0 tx=0).stop 2023-12-19T18:17:04.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.567+0000 7f33ea4ee640 1 -- 172.21.15.27:0/1903198757 shutdown_connections 2023-12-19T18:17:04.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.567+0000 7f33ea4ee640 1 --2- 172.21.15.27:0/1903198757 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e40d2350 0x7f33e40b4d20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.567+0000 7f33ea4ee640 1 --2- 172.21.15.27:0/1903198757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33e41604d0 0x7f33e40b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.567+0000 7f33ea4ee640 1 -- 172.21.15.27:0/1903198757 >> 172.21.15.27:0/1903198757 conn(0x7f33e40cf7e0 msgr2=0x7f33e40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:04.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.567+0000 7f33ea4ee640 1 -- 172.21.15.27:0/1903198757 shutdown_connections 2023-12-19T18:17:04.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.567+0000 7f33ea4ee640 1 -- 172.21.15.27:0/1903198757 wait complete. 2023-12-19T18:17:04.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.569+0000 7f33ea4ee640 1 Processor -- start 2023-12-19T18:17:04.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.569+0000 7f33ea4ee640 1 -- start start 2023-12-19T18:17:04.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.570+0000 7f33ea4ee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33e40d0300 0x7f33e40b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.570+0000 7f33ea4ee640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e40d2350 0x7f33e40c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.570+0000 7f33ea4ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33e41604d0 0x7f33e41f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.570+0000 7f33ea4ee640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e41f2c90 con 0x7f33e41604d0 2023-12-19T18:17:04.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.570+0000 7f33ea4ee640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40c6cc0 con 0x7f33e40d2350 2023-12-19T18:17:04.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.570+0000 7f33ea4ee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e40c6fa0 con 0x7f33e40d0300 2023-12-19T18:17:04.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.570+0000 7f33e9ced640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33e41604d0 0x7f33e41f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.570+0000 7f33e9ced640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33e41604d0 0x7f33e41f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38106/0 (socket says 172.21.15.27:38106) 2023-12-19T18:17:04.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.570+0000 7f33e9ced640 1 -- 172.21.15.27:0/3687993468 learned_addr learned my addr 172.21.15.27:0/3687993468 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:04.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.571+0000 7f33e94ec640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33e40d0300 0x7f33e40b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.571+0000 7f33e8ceb640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e40d2350 0x7f33e40c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.571+0000 7f33e9ced640 1 -- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33e40d0300 msgr2=0x7f33e40b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.571+0000 7f33e9ced640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33e40d0300 0x7f33e40b81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.571+0000 7f33e9ced640 1 -- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e40d2350 msgr2=0x7f33e40c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:04.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.571+0000 7f33e9ced640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e40d2350 0x7f33e40c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:04.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.571+0000 7f33e9ced640 1 -- 172.21.15.27:0/3687993468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d80094d0 con 0x7f33e41604d0 2023-12-19T18:17:04.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.571+0000 7f33e8ceb640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e40d2350 0x7f33e40c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.571+0000 7f33e9ced640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33e41604d0 0x7f33e41f62e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00c390 tx=0x7f33dc00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.571+0000 7f33e1ffb640 1 -- 172.21.15.27:0/3687993468 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0052d0 con 0x7f33e41604d0 2023-12-19T18:17:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.572+0000 7f33e1ffb640 1 -- 172.21.15.27:0/3687993468 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc005430 con 0x7f33e41604d0 2023-12-19T18:17:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.572+0000 7f33e1ffb640 1 -- 172.21.15.27:0/3687993468 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc003810 con 0x7f33e41604d0 2023-12-19T18:17:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.572+0000 7f33ea4ee640 1 -- 172.21.15.27:0/3687993468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33e40c7280 con 0x7f33e41604d0 2023-12-19T18:17:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.572+0000 7f33ea4ee640 1 -- 172.21.15.27:0/3687993468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e4159a00 con 0x7f33e41604d0 2023-12-19T18:17:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.573+0000 7f33ea4ee640 1 -- 172.21.15.27:0/3687993468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e4066f80 con 0x7f33e41604d0 2023-12-19T18:17:04.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.574+0000 7f33e1ffb640 1 -- 172.21.15.27:0/3687993468 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f33dc003970 con 0x7f33e41604d0 2023-12-19T18:17:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.575+0000 7f33e1ffb640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f33d006a430 0x7f33d006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.575+0000 7f33e1ffb640 1 -- 172.21.15.27:0/3687993468 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f33dc087b30 con 0x7f33e41604d0 2023-12-19T18:17:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.575+0000 7f33e94ec640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f33d006a430 0x7f33d006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.577+0000 7f33e94ec640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f33d006a430 0x7f33d006c8f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f33d40074a0 tx=0x7f33d400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:04.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.579+0000 7f33e1ffb640 1 -- 172.21.15.27:0/3687993468 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f33dc053420 con 0x7f33e41604d0 2023-12-19T18:17:04.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:04.846+0000 7f33ea4ee640 1 -- 172.21.15.27:0/3687993468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]} v 0) v1 -- 0x7f33e404e840 con 0x7f33e41604d0 2023-12-19T18:17:05.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:05.846+0000 7f33e1ffb640 1 -- 172.21.15.27:0/3687993468 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_barrier"]}]=0 v134) v1 ==== 130+0+0 (secure 0 0 0) 0x7f33dc056a40 con 0x7f33e41604d0 2023-12-19T18:17:05.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:05.849+0000 7f33ea4ee640 1 -- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f33d006a430 msgr2=0x7f33d006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:05.849+0000 7f33ea4ee640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f33d006a430 0x7f33d006c8f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f33d40074a0 tx=0x7f33d400b040 comp rx=0 tx=0).stop 2023-12-19T18:17:05.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:05.849+0000 7f33ea4ee640 1 -- 172.21.15.27:0/3687993468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33e41604d0 msgr2=0x7f33e41f62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:05.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:05.849+0000 7f33ea4ee640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33e41604d0 0x7f33e41f62e0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f33dc00c390 tx=0x7f33dc00c850 comp rx=0 tx=0).stop 2023-12-19T18:17:05.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:05.850+0000 7f33ea4ee640 1 -- 172.21.15.27:0/3687993468 shutdown_connections 2023-12-19T18:17:05.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:05.850+0000 7f33ea4ee640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f33d006a430 0x7f33d006c8f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:05.850+0000 7f33ea4ee640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33e41604d0 0x7f33e41f62e0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:05.850+0000 7f33ea4ee640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33e40d2350 0x7f33e40c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:05.850+0000 7f33ea4ee640 1 --2- 172.21.15.27:0/3687993468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33e40d0300 0x7f33e40b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:05.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:05.850+0000 7f33ea4ee640 1 -- 172.21.15.27:0/3687993468 >> 172.21.15.27:0/3687993468 conn(0x7f33e40cf7e0 msgr2=0x7f33e40d13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:05.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:05.850+0000 7f33ea4ee640 1 -- 172.21.15.27:0/3687993468 shutdown_connections 2023-12-19T18:17:05.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:05.850+0000 7f33ea4ee640 1 -- 172.21.15.27:0/3687993468 wait complete. 2023-12-19T18:17:05.869 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2023-12-19T18:17:05.870 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestClusterFull) 2023-12-19T18:17:05.870 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-12-19T18:17:06.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.102+0000 7f53ca01f640 1 Processor -- start 2023-12-19T18:17:06.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.102+0000 7f53ca01f640 1 -- start start 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.104+0000 7f53ca01f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c41540e0 0x7f53c4160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.104+0000 7f53ca01f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f53c406c6e0 0x7f53c415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.104+0000 7f53ca01f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f53c415e2e0 0x7f53c406b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.104+0000 7f53ca01f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c415d600 con 0x7f53c415e2e0 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.104+0000 7f53ca01f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c4161130 con 0x7f53c406c6e0 2023-12-19T18:17:06.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.104+0000 7f53ca01f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c41613d0 con 0x7f53c41540e0 2023-12-19T18:17:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.104+0000 7f53c981e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f53c415e2e0 0x7f53c406b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.104+0000 7f53c981e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f53c415e2e0 0x7f53c406b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38110/0 (socket says 172.21.15.27:38110) 2023-12-19T18:17:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.104+0000 7f53c981e640 1 -- 172.21.15.27:0/3933325254 learned_addr learned my addr 172.21.15.27:0/3933325254 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.105+0000 7f53c881c640 1 --2- 172.21.15.27:0/3933325254 >> v2:172.21.15.192:3300/0 conn(0x7f53c406c6e0 0x7f53c415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.105+0000 7f53c901d640 1 --2- 172.21.15.27:0/3933325254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c41540e0 0x7f53c4160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.105+0000 7f53c981e640 1 -- 172.21.15.27:0/3933325254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c41540e0 msgr2=0x7f53c4160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:06.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.105+0000 7f53c981e640 1 --2- 172.21.15.27:0/3933325254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c41540e0 0x7f53c4160bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.105+0000 7f53c981e640 1 -- 172.21.15.27:0/3933325254 >> v2:172.21.15.192:3300/0 conn(0x7f53c406c6e0 msgr2=0x7f53c415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:06.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.105+0000 7f53c981e640 1 --2- 172.21.15.27:0/3933325254 >> v2:172.21.15.192:3300/0 conn(0x7f53c406c6e0 0x7f53c415dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.105+0000 7f53c981e640 1 -- 172.21.15.27:0/3933325254 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c41616b0 con 0x7f53c415e2e0 2023-12-19T18:17:06.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.106+0000 7f53c981e640 1 --2- 172.21.15.27:0/3933325254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c415e2e0 0x7f53c406b310 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f53c406d540 tx=0x7f53bc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a2ba878a7d55cf2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:06.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.106+0000 7f53bbfff640 1 -- 172.21.15.27:0/3933325254 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc02ca70 con 0x7f53c415e2e0 2023-12-19T18:17:06.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.106+0000 7f53bbfff640 1 -- 172.21.15.27:0/3933325254 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc02cbd0 con 0x7f53c415e2e0 2023-12-19T18:17:06.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.106+0000 7f53bbfff640 1 -- 172.21.15.27:0/3933325254 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0343f0 con 0x7f53c415e2e0 2023-12-19T18:17:06.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.107+0000 7f53ca01f640 1 -- 172.21.15.27:0/3933325254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c415e2e0 msgr2=0x7f53c406b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.107+0000 7f53ca01f640 1 --2- 172.21.15.27:0/3933325254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c415e2e0 0x7f53c406b310 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f53c406d540 tx=0x7f53bc02bed0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.107+0000 7f53ca01f640 1 -- 172.21.15.27:0/3933325254 shutdown_connections 2023-12-19T18:17:06.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.107+0000 7f53ca01f640 1 --2- 172.21.15.27:0/3933325254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c415e2e0 0x7f53c406b310 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.107+0000 7f53ca01f640 1 --2- 172.21.15.27:0/3933325254 >> v2:172.21.15.192:3300/0 conn(0x7f53c406c6e0 0x7f53c415dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.107+0000 7f53ca01f640 1 --2- 172.21.15.27:0/3933325254 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c41540e0 0x7f53c4160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.107+0000 7f53ca01f640 1 -- 172.21.15.27:0/3933325254 >> 172.21.15.27:0/3933325254 conn(0x7f53c406a7d0 msgr2=0x7f53c40ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:06.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.107+0000 7f53ca01f640 1 -- 172.21.15.27:0/3933325254 shutdown_connections 2023-12-19T18:17:06.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.107+0000 7f53ca01f640 1 -- 172.21.15.27:0/3933325254 wait complete. 2023-12-19T18:17:06.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.109+0000 7f53ca01f640 1 Processor -- start 2023-12-19T18:17:06.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.109+0000 7f53ca01f640 1 -- start start 2023-12-19T18:17:06.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.110+0000 7f53ca01f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c406c6e0 0x7f53c4150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.110+0000 7f53ca01f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c41540e0 0x7f53c4151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.110+0000 7f53ca01f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c415e2e0 0x7f53c41568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.110+0000 7f53ca01f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40b0740 con 0x7f53c41540e0 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.110+0000 7f53ca01f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40afe30 con 0x7f53c406c6e0 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.110+0000 7f53ca01f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c4151360 con 0x7f53c415e2e0 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.110+0000 7f53c881c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c41540e0 0x7f53c4151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.110+0000 7f53c881c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c41540e0 0x7f53c4151d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38126/0 (socket says 172.21.15.27:38126) 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.110+0000 7f53c881c640 1 -- 172.21.15.27:0/3344262650 learned_addr learned my addr 172.21.15.27:0/3344262650 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.111+0000 7f53c901d640 1 --2- 172.21.15.27:0/3344262650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c406c6e0 0x7f53c4150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.111+0000 7f53c881c640 1 -- 172.21.15.27:0/3344262650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c415e2e0 msgr2=0x7f53c41568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.111+0000 7f53c881c640 1 --2- 172.21.15.27:0/3344262650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c415e2e0 0x7f53c41568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.111+0000 7f53c881c640 1 -- 172.21.15.27:0/3344262650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c406c6e0 msgr2=0x7f53c4150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.111+0000 7f53c881c640 1 --2- 172.21.15.27:0/3344262650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c406c6e0 0x7f53c4150e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.111+0000 7f53c881c640 1 -- 172.21.15.27:0/3344262650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53bc009b30 con 0x7f53c41540e0 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.111+0000 7f53c881c640 1 --2- 172.21.15.27:0/3344262650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c41540e0 0x7f53c4151d20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f53b400d800 tx=0x7f53b400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.111+0000 7f53b9ffb640 1 -- 172.21.15.27:0/3344262650 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b40096a0 con 0x7f53c41540e0 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.111+0000 7f53b9ffb640 1 -- 172.21.15.27:0/3344262650 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53b4009800 con 0x7f53c41540e0 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.111+0000 7f53b9ffb640 1 -- 172.21.15.27:0/3344262650 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53b4009990 con 0x7f53c41540e0 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.111+0000 7f53ca01f640 1 -- 172.21.15.27:0/3344262650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c41527c0 con 0x7f53c41540e0 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.111+0000 7f53ca01f640 1 -- 172.21.15.27:0/3344262650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c40b0360 con 0x7f53c41540e0 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.113+0000 7f53ca01f640 1 -- 172.21.15.27:0/3344262650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c40b0880 con 0x7f53c41540e0 2023-12-19T18:17:06.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.114+0000 7f53b9ffb640 1 -- 172.21.15.27:0/3344262650 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f53b4009af0 con 0x7f53c41540e0 2023-12-19T18:17:06.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.114+0000 7f53b9ffb640 1 --2- 172.21.15.27:0/3344262650 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f53a806a430 0x7f53a806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:06.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.115+0000 7f53b9ffb640 1 -- 172.21.15.27:0/3344262650 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f53b40889d0 con 0x7f53c41540e0 2023-12-19T18:17:06.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.115+0000 7f53c901d640 1 --2- 172.21.15.27:0/3344262650 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f53a806a430 0x7f53a806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:06.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.116+0000 7f53c901d640 1 --2- 172.21.15.27:0/3344262650 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f53a806a430 0x7f53a806c8f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53c0006fd0 tx=0x7f53c0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:06.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.119+0000 7f53b9ffb640 1 -- 172.21.15.27:0/3344262650 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f53c40b0880 con 0x7f53c41540e0 2023-12-19T18:17:06.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.381+0000 7f53ca01f640 1 -- 172.21.15.27:0/3344262650 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f53c40b0880 con 0x7f53c41540e0 2023-12-19T18:17:06.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.849+0000 7f53b9ffb640 1 -- 172.21.15.27:0/3344262650 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v135) v1 ==== 145+0+0 (secure 0 0 0) 0x7f53c40b0880 con 0x7f53c41540e0 2023-12-19T18:17:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.852+0000 7f53ca01f640 1 -- 172.21.15.27:0/3344262650 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f53a806a430 msgr2=0x7f53a806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.852+0000 7f53ca01f640 1 --2- 172.21.15.27:0/3344262650 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f53a806a430 0x7f53a806c8f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53c0006fd0 tx=0x7f53c0008040 comp rx=0 tx=0).stop 2023-12-19T18:17:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.852+0000 7f53ca01f640 1 -- 172.21.15.27:0/3344262650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c41540e0 msgr2=0x7f53c4151d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.852+0000 7f53ca01f640 1 --2- 172.21.15.27:0/3344262650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c41540e0 0x7f53c4151d20 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f53b400d800 tx=0x7f53b400dcc0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.853+0000 7f53ca01f640 1 -- 172.21.15.27:0/3344262650 shutdown_connections 2023-12-19T18:17:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.853+0000 7f53ca01f640 1 --2- 172.21.15.27:0/3344262650 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f53a806a430 0x7f53a806c8f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.853+0000 7f53ca01f640 1 --2- 172.21.15.27:0/3344262650 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c415e2e0 0x7f53c41568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.853+0000 7f53ca01f640 1 --2- 172.21.15.27:0/3344262650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c41540e0 0x7f53c4151d20 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.853+0000 7f53ca01f640 1 --2- 172.21.15.27:0/3344262650 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c406c6e0 0x7f53c4150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:06.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.853+0000 7f53ca01f640 1 -- 172.21.15.27:0/3344262650 >> 172.21.15.27:0/3344262650 conn(0x7f53c406a7d0 msgr2=0x7f53c40cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:06.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.853+0000 7f53ca01f640 1 -- 172.21.15.27:0/3344262650 shutdown_connections 2023-12-19T18:17:06.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:06.853+0000 7f53ca01f640 1 -- 172.21.15.27:0/3344262650 wait complete. 2023-12-19T18:17:06.872 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-12-19T18:17:07.111 INFO:teuthology.orchestra.run.smithi027.stdout:{"osd_objectstore":"memstore"} 2023-12-19T18:17:07.225 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-12-19T18:17:07.225 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:17:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.470+0000 7fa0c2099640 1 Processor -- start 2023-12-19T18:17:07.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.470+0000 7fa0c2099640 1 -- start start 2023-12-19T18:17:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.472+0000 7fa0c2099640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0bc0d2100 0x7fa0bc0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.472+0000 7fa0c2099640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa0bc0cdf80 0x7fa0bc0b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.472+0000 7fa0c2099640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0bc0d0e10 con 0x7fa0bc15cf80 2023-12-19T18:17:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.472+0000 7fa0c2099640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0bc0b8cf0 con 0x7fa0bc0cdf80 2023-12-19T18:17:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.472+0000 7fa0c2099640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc0ce410 con 0x7fa0bc0d2100 2023-12-19T18:17:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.472+0000 7fa0c1898640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa0bc0cdf80 0x7fa0bc0b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.472+0000 7fa0c1898640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa0bc0cdf80 0x7fa0bc0b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38132/0 (socket says 172.21.15.27:38132) 2023-12-19T18:17:07.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.472+0000 7fa0c1898640 1 -- 172.21.15.27:0/2010687100 learned_addr learned my addr 172.21.15.27:0/2010687100 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:07.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.472+0000 7fa0c0896640 1 --2- 172.21.15.27:0/2010687100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0bc0d2100 0x7fa0bc0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.472+0000 7fa0c1898640 1 -- 172.21.15.27:0/2010687100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0bc0d2100 msgr2=0x7fa0bc0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:07.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.472+0000 7fa0c1898640 1 --2- 172.21.15.27:0/2010687100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0bc0d2100 0x7fa0bc0cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.472+0000 7fa0c1898640 1 -- 172.21.15.27:0/2010687100 >> v1:172.21.15.27:6789/0 conn(0x7fa0bc15cf80 legacy=0x7fa0bc0b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.472+0000 7fa0c1898640 1 -- 172.21.15.27:0/2010687100 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0bc0d1c70 con 0x7fa0bc0cdf80 2023-12-19T18:17:07.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.473+0000 7fa0c0896640 1 --2- 172.21.15.27:0/2010687100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0bc0d2100 0x7fa0bc0cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:07.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.474+0000 7fa0c1898640 1 --2- 172.21.15.27:0/2010687100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc0cdf80 0x7fa0bc0b5d60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009e50 tx=0x7fa0b402bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b53cde80897fd37 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:07.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.474+0000 7fa0bbfff640 1 -- 172.21.15.27:0/2010687100 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b402cb90 con 0x7fa0bc0cdf80 2023-12-19T18:17:07.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.474+0000 7fa0bbfff640 1 -- 172.21.15.27:0/2010687100 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b402ccf0 con 0x7fa0bc0cdf80 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.474+0000 7fa0bbfff640 1 -- 172.21.15.27:0/2010687100 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4034610 con 0x7fa0bc0cdf80 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.474+0000 7fa0c2099640 1 -- 172.21.15.27:0/2010687100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc0cdf80 msgr2=0x7fa0bc0b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.474+0000 7fa0c2099640 1 --2- 172.21.15.27:0/2010687100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc0cdf80 0x7fa0bc0b5d60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4009e50 tx=0x7fa0b402bff0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.475+0000 7fa0c2099640 1 -- 172.21.15.27:0/2010687100 shutdown_connections 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.475+0000 7fa0c2099640 1 --2- 172.21.15.27:0/2010687100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc0cdf80 0x7fa0bc0b5d60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.475+0000 7fa0c2099640 1 --2- 172.21.15.27:0/2010687100 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0bc0d2100 0x7fa0bc0cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.475+0000 7fa0c2099640 1 -- 172.21.15.27:0/2010687100 >> 172.21.15.27:0/2010687100 conn(0x7fa0bc072e20 msgr2=0x7fa0bc073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.475+0000 7fa0c2099640 1 -- 172.21.15.27:0/2010687100 shutdown_connections 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.475+0000 7fa0c2099640 1 -- 172.21.15.27:0/2010687100 wait complete. 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.476+0000 7fa0c2099640 1 Processor -- start 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.477+0000 7fa0c2099640 1 -- start start 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c2099640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0bc0cdf80 0x7fa0bc1f5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c2099640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc0d2100 0x7fa0bc0b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c2099640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0bc15cf80 0x7fa0bc0b4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c2099640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc15d570 con 0x7fa0bc0d2100 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c2099640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc15daf0 con 0x7fa0bc0cdf80 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c2099640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0bc1f6770 con 0x7fa0bc15cf80 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c0896640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc0d2100 0x7fa0bc0b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c0896640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc0d2100 0x7fa0bc0b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38136/0 (socket says 172.21.15.27:38136) 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c0896640 1 -- 172.21.15.27:0/1492108502 learned_addr learned my addr 172.21.15.27:0/1492108502 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c1898640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0bc15cf80 0x7fa0bc0b4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c1097640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0bc0cdf80 0x7fa0bc1f5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c0896640 1 -- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0bc15cf80 msgr2=0x7fa0bc0b4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c0896640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0bc15cf80 0x7fa0bc0b4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c0896640 1 -- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0bc0cdf80 msgr2=0x7fa0bc1f5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c0896640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0bc0cdf80 0x7fa0bc1f5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c0896640 1 -- 172.21.15.27:0/1492108502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b4009b30 con 0x7fa0bc0d2100 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c1898640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0bc15cf80 0x7fa0bc0b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:07.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.478+0000 7fa0c1097640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0bc0cdf80 0x7fa0bc1f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.479+0000 7fa0c0896640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc0d2100 0x7fa0bc0b3b10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c7f0 tx=0x7fa0b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.479+0000 7fa0b9ffb640 1 -- 172.21.15.27:0/1492108502 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0007a30 con 0x7fa0bc0d2100 2023-12-19T18:17:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.479+0000 7fa0b9ffb640 1 -- 172.21.15.27:0/1492108502 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b0007b90 con 0x7fa0bc0d2100 2023-12-19T18:17:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.479+0000 7fa0b9ffb640 1 -- 172.21.15.27:0/1492108502 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000ce40 con 0x7fa0bc0d2100 2023-12-19T18:17:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.479+0000 7fa0c2099640 1 -- 172.21.15.27:0/1492108502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0bc1f6a50 con 0x7fa0bc0d2100 2023-12-19T18:17:07.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.479+0000 7fa0c2099640 1 -- 172.21.15.27:0/1492108502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0bc0c86f0 con 0x7fa0bc0d2100 2023-12-19T18:17:07.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.480+0000 7fa0c2099640 1 -- 172.21.15.27:0/1492108502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0bc0b2c70 con 0x7fa0bc0d2100 2023-12-19T18:17:07.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.481+0000 7fa0b9ffb640 1 -- 172.21.15.27:0/1492108502 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa0b00113f0 con 0x7fa0bc0d2100 2023-12-19T18:17:07.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.483+0000 7fa0b9ffb640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0a806a430 0x7fa0a806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:07.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.483+0000 7fa0b9ffb640 1 -- 172.21.15.27:0/1492108502 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa0b0088750 con 0x7fa0bc0d2100 2023-12-19T18:17:07.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.483+0000 7fa0c1097640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0a806a430 0x7fa0a806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:07.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.485+0000 7fa0c1097640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0a806a430 0x7fa0a806c8f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac000f60 tx=0x7fa0ac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:07.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.488+0000 7fa0b9ffb640 1 -- 172.21.15.27:0/1492108502 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa0b00540d0 con 0x7fa0bc0d2100 2023-12-19T18:17:07.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.682+0000 7fa0c2099640 1 -- 172.21.15.27:0/1492108502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa0bc1f6c40 con 0x7fa0bc0d2100 2023-12-19T18:17:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.690+0000 7fa0b9ffb640 1 -- 172.21.15.27:0/1492108502 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v4)=0 v4) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa0b00576f0 con 0x7fa0bc0d2100 2023-12-19T18:17:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.693+0000 7fa0c2099640 1 -- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0a806a430 msgr2=0x7fa0a806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.693+0000 7fa0c2099640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0a806a430 0x7fa0a806c8f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac000f60 tx=0x7fa0ac009040 comp rx=0 tx=0).stop 2023-12-19T18:17:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.693+0000 7fa0c2099640 1 -- 172.21.15.27:0/1492108502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc0d2100 msgr2=0x7fa0bc0b3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.693+0000 7fa0c2099640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc0d2100 0x7fa0bc0b3b10 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000c7f0 tx=0x7fa0b000cbf0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.693+0000 7fa0c2099640 1 -- 172.21.15.27:0/1492108502 shutdown_connections 2023-12-19T18:17:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.693+0000 7fa0c2099640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0a806a430 0x7fa0a806c8f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.693+0000 7fa0c2099640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0bc15cf80 0x7fa0bc0b4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.693+0000 7fa0c2099640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0bc0d2100 0x7fa0bc0b3b10 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.693+0000 7fa0c2099640 1 --2- 172.21.15.27:0/1492108502 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0bc0cdf80 0x7fa0bc1f5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:07.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.693+0000 7fa0c2099640 1 -- 172.21.15.27:0/1492108502 >> 172.21.15.27:0/1492108502 conn(0x7fa0bc072e20 msgr2=0x7fa0bc0b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:07.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.694+0000 7fa0c2099640 1 -- 172.21.15.27:0/1492108502 shutdown_connections 2023-12-19T18:17:07.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:07.694+0000 7fa0c2099640 1 -- 172.21.15.27:0/1492108502 wait complete. 2023-12-19T18:17:07.712 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:17:07.712 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:17:07.841 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:17:07.964 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:17:07.964 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:17:07.964 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:08.030 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:17:08.030 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:17:08.030 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:08.089 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:17:08.089 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:17:08.089 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:08.153 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:17:08.153 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:17:08.153 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:08.214 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:17:08.214 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:17:08.215 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:08.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.456+0000 7f022cf3a640 1 Processor -- start 2023-12-19T18:17:08.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.456+0000 7f022cf3a640 1 -- start start 2023-12-19T18:17:08.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.457+0000 7f022cf3a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f02281553a0 0x7f02280c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.457+0000 7f022cf3a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02280afeb0 0x7f022815e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.457+0000 7f022cf3a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02280b0330 con 0x7f022815ec60 2023-12-19T18:17:08.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.457+0000 7f022cf3a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02280cbcf0 con 0x7f02281553a0 2023-12-19T18:17:08.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.457+0000 7f022cf3a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022815f0d0 con 0x7f02280afeb0 2023-12-19T18:17:08.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.458+0000 7f02277fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f02281553a0 0x7f02280c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.458+0000 7f02277fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f02281553a0 0x7f02280c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:33650/0 (socket says 172.21.15.27:33650) 2023-12-19T18:17:08.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.458+0000 7f0226ffd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02280afeb0 0x7f022815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.458+0000 7f02277fe640 1 -- 172.21.15.27:0/4189246644 learned_addr learned my addr 172.21.15.27:0/4189246644 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:08.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.459+0000 7f02277fe640 1 -- 172.21.15.27:0/4189246644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02280afeb0 msgr2=0x7f022815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:08.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.459+0000 7f02277fe640 1 --2- 172.21.15.27:0/4189246644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02280afeb0 0x7f022815e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.459+0000 7f02277fe640 1 -- 172.21.15.27:0/4189246644 >> v1:172.21.15.192:6789/0 conn(0x7f022815ec60 legacy=0x7f022815b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.459+0000 7f02277fe640 1 -- 172.21.15.27:0/4189246644 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02280cb350 con 0x7f02281553a0 2023-12-19T18:17:08.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.459+0000 7f0226ffd640 1 --2- 172.21.15.27:0/4189246644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02280afeb0 0x7f022815e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:08.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.460+0000 7f02277fe640 1 --2- 172.21.15.27:0/4189246644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02281553a0 0x7f02280c50a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f02180098a0 tx=0x7f021802b8e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b690ad79ce9a5bb9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:08.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.461+0000 7f02267fc640 1 -- 172.21.15.27:0/4189246644 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218031640 con 0x7f02281553a0 2023-12-19T18:17:08.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.461+0000 7f02267fc640 1 -- 172.21.15.27:0/4189246644 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02180317a0 con 0x7f02281553a0 2023-12-19T18:17:08.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.461+0000 7f02267fc640 1 -- 172.21.15.27:0/4189246644 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0218031b30 con 0x7f02281553a0 2023-12-19T18:17:08.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.461+0000 7f022cf3a640 1 -- 172.21.15.27:0/4189246644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02281553a0 msgr2=0x7f02280c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.461+0000 7f022cf3a640 1 --2- 172.21.15.27:0/4189246644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02281553a0 0x7f02280c50a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f02180098a0 tx=0x7f021802b8e0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.461+0000 7f022cf3a640 1 -- 172.21.15.27:0/4189246644 shutdown_connections 2023-12-19T18:17:08.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.461+0000 7f022cf3a640 1 --2- 172.21.15.27:0/4189246644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f02280afeb0 0x7f022815e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.461+0000 7f022cf3a640 1 --2- 172.21.15.27:0/4189246644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02281553a0 0x7f02280c50a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.461+0000 7f022cf3a640 1 -- 172.21.15.27:0/4189246644 >> 172.21.15.27:0/4189246644 conn(0x7f0228069c70 msgr2=0x7f022806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:08.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.462+0000 7f022cf3a640 1 -- 172.21.15.27:0/4189246644 shutdown_connections 2023-12-19T18:17:08.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.462+0000 7f022cf3a640 1 -- 172.21.15.27:0/4189246644 wait complete. 2023-12-19T18:17:08.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.463+0000 7f022cf3a640 1 Processor -- start 2023-12-19T18:17:08.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.464+0000 7f022cf3a640 1 -- start start 2023-12-19T18:17:08.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.465+0000 7f022cf3a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02280afeb0 0x7f02281efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.465+0000 7f022cf3a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02281553a0 0x7f02281f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.465+0000 7f022cf3a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022815ec60 0x7f0228157b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.465+0000 7f022cf3a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022815c2d0 con 0x7f02281553a0 2023-12-19T18:17:08.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.465+0000 7f022cf3a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02281f3630 con 0x7f02280afeb0 2023-12-19T18:17:08.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.465+0000 7f022cf3a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0228148420 con 0x7f022815ec60 2023-12-19T18:17:08.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.465+0000 7f0226ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02281553a0 0x7f02281f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.465+0000 7f0226ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02281553a0 0x7f02281f30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38148/0 (socket says 172.21.15.27:38148) 2023-12-19T18:17:08.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.465+0000 7f0226ffd640 1 -- 172.21.15.27:0/320488496 learned_addr learned my addr 172.21.15.27:0/320488496 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.465+0000 7f0227fff640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022815ec60 0x7f0228157b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.465+0000 7f02277fe640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02280afeb0 0x7f02281efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.466+0000 7f0226ffd640 1 -- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022815ec60 msgr2=0x7f0228157b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.466+0000 7f0226ffd640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022815ec60 0x7f0228157b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.466+0000 7f0226ffd640 1 -- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02280afeb0 msgr2=0x7f02281efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.466+0000 7f0226ffd640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02280afeb0 0x7f02281efbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.466+0000 7f0226ffd640 1 -- 172.21.15.27:0/320488496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f021c009a70 con 0x7f02281553a0 2023-12-19T18:17:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.466+0000 7f02277fe640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02280afeb0 0x7f02281efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.466+0000 7f0226ffd640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02281553a0 0x7f02281f30f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f021c00efc0 tx=0x7f021c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.466+0000 7f020ffff640 1 -- 172.21.15.27:0/320488496 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c014680 con 0x7f02281553a0 2023-12-19T18:17:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.466+0000 7f020ffff640 1 -- 172.21.15.27:0/320488496 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021c00f040 con 0x7f02281553a0 2023-12-19T18:17:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.466+0000 7f020ffff640 1 -- 172.21.15.27:0/320488496 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021c014a00 con 0x7f02281553a0 2023-12-19T18:17:08.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.466+0000 7f022cf3a640 1 -- 172.21.15.27:0/320488496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02180094d0 con 0x7f02281553a0 2023-12-19T18:17:08.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.467+0000 7f022cf3a640 1 -- 172.21.15.27:0/320488496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02281489e0 con 0x7f02281553a0 2023-12-19T18:17:08.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.468+0000 7f022cf3a640 1 -- 172.21.15.27:0/320488496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022815c440 con 0x7f02281553a0 2023-12-19T18:17:08.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.469+0000 7f020ffff640 1 -- 172.21.15.27:0/320488496 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f021c014b60 con 0x7f02281553a0 2023-12-19T18:17:08.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.470+0000 7f020ffff640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f021406a430 0x7f021406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:08.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.470+0000 7f020ffff640 1 -- 172.21.15.27:0/320488496 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f021c089630 con 0x7f02281553a0 2023-12-19T18:17:08.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.470+0000 7f02277fe640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f021406a430 0x7f021406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:08.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.471+0000 7f02277fe640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f021406a430 0x7f021406c8f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0218036000 tx=0x7f021803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.474+0000 7f020ffff640 1 -- 172.21.15.27:0/320488496 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f022815c440 con 0x7f02281553a0 2023-12-19T18:17:08.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.707+0000 7f022cf3a640 1 -- 172.21.15.27:0/320488496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f022815c410 con 0x7f02281553a0 2023-12-19T18:17:08.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.708+0000 7f020ffff640 1 -- 172.21.15.27:0/320488496 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f021c054e80 con 0x7f02281553a0 2023-12-19T18:17:08.708 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:08.709 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":17,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4691,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3178168231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3178168231},{"type":"v1","addr":"172.21.15.192:6837","nonce":3178168231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3961149394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3961149394},{"type":"v1","addr":"172.21.15.192:6836","nonce":3961149394}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5190,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/752784926","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":752784926},{"type":"v1","addr":"172.21.15.27:6833","nonce":752784926}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5601,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/722609446","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":722609446},{"type":"v1","addr":"172.21.15.27:6835","nonce":722609446}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":17}],"filesystems":[]} 2023-12-19T18:17:08.709 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 17 2023-12-19T18:17:08.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.711+0000 7f022cf3a640 1 -- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f021406a430 msgr2=0x7f021406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.711+0000 7f022cf3a640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f021406a430 0x7f021406c8f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f0218036000 tx=0x7f021803c040 comp rx=0 tx=0).stop 2023-12-19T18:17:08.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.711+0000 7f022cf3a640 1 -- 172.21.15.27:0/320488496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02281553a0 msgr2=0x7f02281f30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:08.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.711+0000 7f022cf3a640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02281553a0 0x7f02281f30f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f021c00efc0 tx=0x7f021c00c490 comp rx=0 tx=0).stop 2023-12-19T18:17:08.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.712+0000 7f022cf3a640 1 -- 172.21.15.27:0/320488496 shutdown_connections 2023-12-19T18:17:08.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.712+0000 7f022cf3a640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f021406a430 0x7f021406c8f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.712+0000 7f022cf3a640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022815ec60 0x7f0228157b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.712+0000 7f022cf3a640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02281553a0 0x7f02281f30f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.712+0000 7f022cf3a640 1 --2- 172.21.15.27:0/320488496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02280afeb0 0x7f02281efbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:08.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.712+0000 7f022cf3a640 1 -- 172.21.15.27:0/320488496 >> 172.21.15.27:0/320488496 conn(0x7f0228069c70 msgr2=0x7f022815a230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:08.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.712+0000 7f022cf3a640 1 -- 172.21.15.27:0/320488496 shutdown_connections 2023-12-19T18:17:08.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:08.712+0000 7f022cf3a640 1 -- 172.21.15.27:0/320488496 wait complete. 2023-12-19T18:17:08.733 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:17:08.733 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:17:08.733 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:17:08.734 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:17:08.734 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:17:08.734 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:17:08.734 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:17:08.734 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:17:08.734 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:17:08.734 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:17:08.735 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:17:08.735 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:17:08.735 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:17:08.735 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:17:08.735 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:17:08.735 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:17:08.752 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:17:08.752 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:17:08.755 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:17:08.755 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:17:08.756 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:17:08.756 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:17:08.757 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:17:08.757 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:17:08.803 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:17:08.804 DEBUG:teuthology.parallel:result is None 2023-12-19T18:17:08.805 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:17:08.805 DEBUG:teuthology.parallel:result is None 2023-12-19T18:17:08.813 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:17:08.813 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:17:08.813 DEBUG:teuthology.parallel:result is None 2023-12-19T18:17:08.814 DEBUG:teuthology.parallel:result is None 2023-12-19T18:17:08.814 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:17:08.968 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-19T18:17:08.969 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-19T18:17:08.974 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:17:08.977 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:17:09.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.019+0000 7fbe4cd85640 1 Processor -- start 2023-12-19T18:17:09.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.019+0000 7fbe4cd85640 1 -- start start 2023-12-19T18:17:09.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.020+0000 7fbe4cd85640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbe480b89b0 0x7fbe4815c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.020+0000 7fbe4cd85640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe480b5ca0 con 0x7fbe480b4db0 2023-12-19T18:17:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.020+0000 7fbe4cd85640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe48161800 con 0x7fbe480b49d0 2023-12-19T18:17:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.020+0000 7fbe4cd85640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe48161050 con 0x7fbe480b89b0 2023-12-19T18:17:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.021+0000 7fbe47fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbe480b4db0 0x7fbe480d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55046/0 (socket says 172.21.15.27:55046) 2023-12-19T18:17:09.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.021+0000 7fbe47fff640 1 -- 172.21.15.27:0/3393403233 learned_addr learned my addr 172.21.15.27:0/3393403233 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:09.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.021+0000 7fbe3ffff640 1 --2- 172.21.15.27:0/3393403233 >> v2:172.21.15.192:3300/0 conn(0x7fbe480b89b0 0x7fbe4815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:09.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.021+0000 7fbe46ffd640 1 -- 172.21.15.27:0/3393403233 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 724579290 0 0) 0x7fbe480b5ca0 con 0x7fbe480b4db0 2023-12-19T18:17:09.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.021+0000 7fbe46ffd640 1 -- 172.21.15.27:0/3393403233 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe34003520 con 0x7fbe480b4db0 2023-12-19T18:17:09.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.021+0000 7fbe46ffd640 1 -- 172.21.15.27:0/3393403233 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2567149188 0 0) 0x7fbe48161800 con 0x7fbe480b49d0 2023-12-19T18:17:09.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.021+0000 7fbe46ffd640 1 -- 172.21.15.27:0/3393403233 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe480b5ca0 con 0x7fbe480b49d0 2023-12-19T18:17:09.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.022+0000 7fbe3ffff640 1 -- 172.21.15.27:0/3393403233 >> v1:172.21.15.192:6789/0 conn(0x7fbe480b49d0 legacy=0x7fbe48160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:09.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.022+0000 7fbe3ffff640 1 -- 172.21.15.27:0/3393403233 >> v1:172.21.15.27:6789/0 conn(0x7fbe480b4db0 legacy=0x7fbe480d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:09.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.022+0000 7fbe3ffff640 1 -- 172.21.15.27:0/3393403233 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe48161240 con 0x7fbe480b89b0 2023-12-19T18:17:09.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.023+0000 7fbe3ffff640 1 --2- 172.21.15.27:0/3393403233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480b89b0 0x7fbe4815c880 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbe300097a0 tx=0x7fbe3002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=758393034c450a4c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.024+0000 7fbe46ffd640 1 -- 172.21.15.27:0/3393403233 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30030610 con 0x7fbe480b89b0 2023-12-19T18:17:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.024+0000 7fbe46ffd640 1 -- 172.21.15.27:0/3393403233 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe30030770 con 0x7fbe480b89b0 2023-12-19T18:17:09.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.024+0000 7fbe46ffd640 1 -- 172.21.15.27:0/3393403233 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe30030b00 con 0x7fbe480b89b0 2023-12-19T18:17:09.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.024+0000 7fbe4cd85640 1 -- 172.21.15.27:0/3393403233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480b89b0 msgr2=0x7fbe4815c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:09.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.024+0000 7fbe4cd85640 1 --2- 172.21.15.27:0/3393403233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480b89b0 0x7fbe4815c880 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fbe300097a0 tx=0x7fbe3002b960 comp rx=0 tx=0).stop 2023-12-19T18:17:09.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.025+0000 7fbe4cd85640 1 -- 172.21.15.27:0/3393403233 shutdown_connections 2023-12-19T18:17:09.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.025+0000 7fbe4cd85640 1 --2- 172.21.15.27:0/3393403233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480b89b0 0x7fbe4815c880 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:09.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.025+0000 7fbe4cd85640 1 -- 172.21.15.27:0/3393403233 >> 172.21.15.27:0/3393403233 conn(0x7fbe480cf760 msgr2=0x7fbe480cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:09.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.025+0000 7fbe4cd85640 1 -- 172.21.15.27:0/3393403233 shutdown_connections 2023-12-19T18:17:09.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.025+0000 7fbe4cd85640 1 -- 172.21.15.27:0/3393403233 wait complete. 2023-12-19T18:17:09.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.026+0000 7fbe4cd85640 1 Processor -- start 2023-12-19T18:17:09.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.027+0000 7fbe4cd85640 1 -- start start 2023-12-19T18:17:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.028+0000 7fbe4cd85640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe480b49d0 0x7fbe481f53c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.028+0000 7fbe4cd85640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480b4db0 0x7fbe48158090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.028+0000 7fbe4cd85640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe480b89b0 0x7fbe481585d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.028+0000 7fbe4cd85640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe481f5900 con 0x7fbe480b89b0 2023-12-19T18:17:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.028+0000 7fbe4cd85640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe48159810 con 0x7fbe480b4db0 2023-12-19T18:17:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.028+0000 7fbe4cd85640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe48151aa0 con 0x7fbe480b49d0 2023-12-19T18:17:09.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.028+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe480b89b0 0x7fbe481585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.028+0000 7fbe47fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe480b89b0 0x7fbe481585d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38170/0 (socket says 172.21.15.27:38170) 2023-12-19T18:17:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.028+0000 7fbe47fff640 1 -- 172.21.15.27:0/4121869710 learned_addr learned my addr 172.21.15.27:0/4121869710 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.028+0000 7fbe477fe640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe480b49d0 0x7fbe481f53c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:09.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.028+0000 7fbe3ffff640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480b4db0 0x7fbe48158090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.029+0000 7fbe47fff640 1 -- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe480b49d0 msgr2=0x7fbe481f53c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.029+0000 7fbe47fff640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe480b49d0 0x7fbe481f53c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.029+0000 7fbe47fff640 1 -- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480b4db0 msgr2=0x7fbe48158090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.029+0000 7fbe47fff640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480b4db0 0x7fbe48158090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.029+0000 7fbe47fff640 1 -- 172.21.15.27:0/4121869710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe30009400 con 0x7fbe480b89b0 2023-12-19T18:17:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.029+0000 7fbe477fe640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe480b49d0 0x7fbe481f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.029+0000 7fbe47fff640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe480b89b0 0x7fbe481585d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000cb40 tx=0x7fbe4000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.029+0000 7fbe44ff9640 1 -- 172.21.15.27:0/4121869710 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe4000ac40 con 0x7fbe480b89b0 2023-12-19T18:17:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.029+0000 7fbe44ff9640 1 -- 172.21.15.27:0/4121869710 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe4000ada0 con 0x7fbe480b89b0 2023-12-19T18:17:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.029+0000 7fbe44ff9640 1 -- 172.21.15.27:0/4121869710 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe40012620 con 0x7fbe480b89b0 2023-12-19T18:17:09.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.029+0000 7fbe4cd85640 1 -- 172.21.15.27:0/4121869710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe480caa80 con 0x7fbe480b89b0 2023-12-19T18:17:09.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.029+0000 7fbe4cd85640 1 -- 172.21.15.27:0/4121869710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe48158ee0 con 0x7fbe480b89b0 2023-12-19T18:17:09.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.031+0000 7fbe4cd85640 1 -- 172.21.15.27:0/4121869710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe481599c0 con 0x7fbe480b89b0 2023-12-19T18:17:09.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.032+0000 7fbe44ff9640 1 -- 172.21.15.27:0/4121869710 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbe40007070 con 0x7fbe480b89b0 2023-12-19T18:17:09.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.033+0000 7fbe44ff9640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbe2806a480 0x7fbe2806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:09.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.033+0000 7fbe44ff9640 1 -- 172.21.15.27:0/4121869710 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbe400884b0 con 0x7fbe480b89b0 2023-12-19T18:17:09.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.033+0000 7fbe477fe640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbe2806a480 0x7fbe2806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:09.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.034+0000 7fbe477fe640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbe2806a480 0x7fbe2806c940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbe38000fb0 tx=0x7fbe38009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:09.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.037+0000 7fbe44ff9640 1 -- 172.21.15.27:0/4121869710 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbe481599c0 con 0x7fbe480b89b0 2023-12-19T18:17:09.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:09.242+0000 7fbe4cd85640 1 -- 172.21.15.27:0/4121869710 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fbe481599c0 con 0x7fbe480b89b0 2023-12-19T18:17:10.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.011+0000 7fbe44ff9640 1 -- 172.21.15.27:0/4121869710 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v46) v1 ==== 97+0+0 (secure 0 0 0) 0x7fbe481599c0 con 0x7fbe480b89b0 2023-12-19T18:17:10.010 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2023-12-19T18:17:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.014+0000 7fbe4cd85640 1 -- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbe2806a480 msgr2=0x7fbe2806c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.014+0000 7fbe4cd85640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbe2806a480 0x7fbe2806c940 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbe38000fb0 tx=0x7fbe38009040 comp rx=0 tx=0).stop 2023-12-19T18:17:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.014+0000 7fbe4cd85640 1 -- 172.21.15.27:0/4121869710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe480b89b0 msgr2=0x7fbe481585d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.014+0000 7fbe4cd85640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe480b89b0 0x7fbe481585d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000cb40 tx=0x7fbe4000a060 comp rx=0 tx=0).stop 2023-12-19T18:17:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.015+0000 7fbe4cd85640 1 -- 172.21.15.27:0/4121869710 shutdown_connections 2023-12-19T18:17:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.015+0000 7fbe4cd85640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbe2806a480 0x7fbe2806c940 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.015+0000 7fbe4cd85640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbe480b89b0 0x7fbe481585d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.015+0000 7fbe4cd85640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe480b4db0 0x7fbe48158090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.015+0000 7fbe4cd85640 1 --2- 172.21.15.27:0/4121869710 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe480b49d0 0x7fbe481f53c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.015+0000 7fbe4cd85640 1 -- 172.21.15.27:0/4121869710 >> 172.21.15.27:0/4121869710 conn(0x7fbe480cf760 msgr2=0x7fbe480b6160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.015+0000 7fbe4cd85640 1 -- 172.21.15.27:0/4121869710 shutdown_connections 2023-12-19T18:17:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.015+0000 7fbe4cd85640 1 -- 172.21.15.27:0/4121869710 wait complete. 2023-12-19T18:17:10.037 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:17:10.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.282+0000 7f120ffff640 1 Processor -- start 2023-12-19T18:17:10.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.282+0000 7f120ffff640 1 -- start start 2023-12-19T18:17:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.284+0000 7f120ffff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1210154170 0x7f1210070170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.285+0000 7f120ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1210070aa0 0x7f1210070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.285+0000 7f120ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f121015a3a0 0x7f121015a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.285+0000 7f120ffff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1210071440 con 0x7f121015a3a0 2023-12-19T18:17:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.285+0000 7f120ffff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f121015b110 con 0x7f1210154170 2023-12-19T18:17:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.285+0000 7f120ffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f121006fce0 con 0x7f1210070aa0 2023-12-19T18:17:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.285+0000 7f120f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f121015a3a0 0x7f121015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.285+0000 7f120f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f121015a3a0 0x7f121015a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38184/0 (socket says 172.21.15.27:38184) 2023-12-19T18:17:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.285+0000 7f120f7fe640 1 -- 172.21.15.27:0/2002432167 learned_addr learned my addr 172.21.15.27:0/2002432167 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.286+0000 7f120effd640 1 --2- 172.21.15.27:0/2002432167 >> v2:172.21.15.192:3300/0 conn(0x7f1210154170 0x7f1210070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.285+0000 7f1207fff640 1 --2- 172.21.15.27:0/2002432167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1210070aa0 0x7f1210070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.286+0000 7f120f7fe640 1 -- 172.21.15.27:0/2002432167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1210070aa0 msgr2=0x7f1210070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.286+0000 7f120f7fe640 1 --2- 172.21.15.27:0/2002432167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1210070aa0 0x7f1210070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.286+0000 7f120f7fe640 1 -- 172.21.15.27:0/2002432167 >> v2:172.21.15.192:3300/0 conn(0x7f1210154170 msgr2=0x7f1210070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:10.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.286+0000 7f120f7fe640 1 --2- 172.21.15.27:0/2002432167 >> v2:172.21.15.192:3300/0 conn(0x7f1210154170 0x7f1210070170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.286+0000 7f120f7fe640 1 -- 172.21.15.27:0/2002432167 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f121006f3b0 con 0x7f121015a3a0 2023-12-19T18:17:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.286+0000 7f120effd640 1 --2- 172.21.15.27:0/2002432167 >> v2:172.21.15.192:3300/0 conn(0x7f1210154170 0x7f1210070170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.287+0000 7f120f7fe640 1 --2- 172.21.15.27:0/2002432167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f121015a3a0 0x7f121015a830 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f121006d770 tx=0x7f120802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dddfb1619a82cd4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.287+0000 7f120e7fc640 1 -- 172.21.15.27:0/2002432167 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120802c9e0 con 0x7f121015a3a0 2023-12-19T18:17:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.287+0000 7f120e7fc640 1 -- 172.21.15.27:0/2002432167 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120802cb40 con 0x7f121015a3a0 2023-12-19T18:17:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.287+0000 7f120e7fc640 1 -- 172.21.15.27:0/2002432167 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12080343f0 con 0x7f121015a3a0 2023-12-19T18:17:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.288+0000 7f120ffff640 1 -- 172.21.15.27:0/2002432167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f121015a3a0 msgr2=0x7f121015a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.288+0000 7f120ffff640 1 --2- 172.21.15.27:0/2002432167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f121015a3a0 0x7f121015a830 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f121006d770 tx=0x7f120802be40 comp rx=0 tx=0).stop 2023-12-19T18:17:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.288+0000 7f120ffff640 1 -- 172.21.15.27:0/2002432167 shutdown_connections 2023-12-19T18:17:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.288+0000 7f120ffff640 1 --2- 172.21.15.27:0/2002432167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f121015a3a0 0x7f121015a830 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.288+0000 7f120ffff640 1 --2- 172.21.15.27:0/2002432167 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1210070aa0 0x7f1210070f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.288+0000 7f120ffff640 1 --2- 172.21.15.27:0/2002432167 >> v2:172.21.15.192:3300/0 conn(0x7f1210154170 0x7f1210070170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.288+0000 7f120ffff640 1 -- 172.21.15.27:0/2002432167 >> 172.21.15.27:0/2002432167 conn(0x7f121006a7d0 msgr2=0x7f12100ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.288+0000 7f120ffff640 1 -- 172.21.15.27:0/2002432167 shutdown_connections 2023-12-19T18:17:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.288+0000 7f120ffff640 1 -- 172.21.15.27:0/2002432167 wait complete. 2023-12-19T18:17:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.290+0000 7f120ffff640 1 Processor -- start 2023-12-19T18:17:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.290+0000 7f120ffff640 1 -- start start 2023-12-19T18:17:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.291+0000 7f120ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1210070aa0 0x7f12101f2fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.291+0000 7f120ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1210154170 0x7f12100a92e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.291+0000 7f120ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f121015a3a0 0x7f12100cd590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.291+0000 7f120ffff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12101f0180 con 0x7f121015a3a0 2023-12-19T18:17:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.291+0000 7f120ffff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1210158fb0 con 0x7f1210154170 2023-12-19T18:17:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.291+0000 7f120ffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1210159290 con 0x7f1210070aa0 2023-12-19T18:17:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.291+0000 7f120f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f121015a3a0 0x7f12100cd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.291+0000 7f120f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f121015a3a0 0x7f12100cd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38200/0 (socket says 172.21.15.27:38200) 2023-12-19T18:17:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.291+0000 7f120f7fe640 1 -- 172.21.15.27:0/4270234355 learned_addr learned my addr 172.21.15.27:0/4270234355 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.292+0000 7f1207fff640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1210154170 0x7f12100a92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.292+0000 7f120effd640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1210070aa0 0x7f12101f2fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.292+0000 7f120f7fe640 1 -- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1210070aa0 msgr2=0x7f12101f2fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.292+0000 7f120f7fe640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1210070aa0 0x7f12101f2fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.292+0000 7f120f7fe640 1 -- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1210154170 msgr2=0x7f12100a92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.292+0000 7f120f7fe640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1210154170 0x7f12100a92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.292+0000 7f120f7fe640 1 -- 172.21.15.27:0/4270234355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1208009b30 con 0x7f121015a3a0 2023-12-19T18:17:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.292+0000 7f120f7fe640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f121015a3a0 0x7f12100cd590 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1208036040 tx=0x7f1208004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.292+0000 7f12077fe640 1 -- 172.21.15.27:0/4270234355 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1208009510 con 0x7f121015a3a0 2023-12-19T18:17:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.292+0000 7f12077fe640 1 -- 172.21.15.27:0/4270234355 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120803e450 con 0x7f121015a3a0 2023-12-19T18:17:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.292+0000 7f12077fe640 1 -- 172.21.15.27:0/4270234355 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120803e800 con 0x7f121015a3a0 2023-12-19T18:17:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.293+0000 7f120ffff640 1 -- 172.21.15.27:0/4270234355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1210159570 con 0x7f121015a3a0 2023-12-19T18:17:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.293+0000 7f120ffff640 1 -- 172.21.15.27:0/4270234355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1210159b30 con 0x7f121015a3a0 2023-12-19T18:17:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.294+0000 7f120ffff640 1 -- 172.21.15.27:0/4270234355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121004e840 con 0x7f121015a3a0 2023-12-19T18:17:10.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.295+0000 7f12077fe640 1 -- 172.21.15.27:0/4270234355 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f120803e960 con 0x7f121015a3a0 2023-12-19T18:17:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.296+0000 7f12077fe640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11f006a430 0x7f11f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.296+0000 7f12077fe640 1 -- 172.21.15.27:0/4270234355 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f12080ab3a0 con 0x7f121015a3a0 2023-12-19T18:17:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.297+0000 7f120effd640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11f006a430 0x7f11f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.298+0000 7f120effd640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11f006a430 0x7f11f006c8f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f120000a950 tx=0x7f1200008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:10.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.301+0000 7f12077fe640 1 -- 172.21.15.27:0/4270234355 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1208076cd0 con 0x7f121015a3a0 2023-12-19T18:17:10.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.563+0000 7f120ffff640 1 -- 172.21.15.27:0/4270234355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f12100cabf0 con 0x7f121015a3a0 2023-12-19T18:17:10.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.564+0000 7f12077fe640 1 -- 172.21.15.27:0/4270234355 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f120807a2f0 con 0x7f121015a3a0 2023-12-19T18:17:10.563 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:10.563 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:17:10.563 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2023-12-19T18:17:10.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.563 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2023-12-19T18:17:10.563 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFl+4ruHBAAXwbpmT43SmlvNbuZj5e+bQ==", 2023-12-19T18:17:10.563 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.563 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:17:10.563 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:17:10.563 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:17:10.564 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.564 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2023-12-19T18:17:10.564 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmZyUBhAAG3kZkXvJIo7N9f1j+DPCSQ==", 2023-12-19T18:17:10.564 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.564 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:17:10.564 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:17:10.564 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:17:10.564 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.564 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlChNOMBAAX4GV3iIGkSQdAsYu9oCPDw==", 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2023-12-19T18:17:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlTmFXFhAABA0NMUxKBV1ExAhYc5RFbQ==", 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFluleFIBAAs1goFVMOX4CbjwFhg0KoBg==", 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFlInW3MxAAI4hY1GBxXYXDswKKrEyDTg==", 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFlAWZ5CRAA82/2r9/sT0L9VlGHbXksQw==", 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:10.567 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFl1IvaGRAAqVacDDPLx+NqBOGp6MeX2w==", 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2023-12-19T18:17:10.568 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlr0BBHRAA+7gfs/RG6DtABnYPg059Tw==", 2023-12-19T18:17:10.569 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.569 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:10.569 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:17:10.569 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.569 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.569 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2023-12-19T18:17:10.569 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlXZbELRAAKD0qOkhk7skfc4xcAC8Lhg==", 2023-12-19T18:17:10.569 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.569 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:10.569 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlF07MARAA2TJJ+5C1HIU+SVhqmKaVHA==", 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2023-12-19T18:17:10.570 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlISPGEhAAdFIRRkXlLyOx/oMZ4MgL9Q==", 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmNM9JRAARgS20O8zpm6H70WDv+lgGw==", 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlWoZ8AhAADWvKVzeDhDGHtYNUFIOWsA==", 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFllWFVMhAAD9ebDOppHTODVQjg6ppP3g==", 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:17:10.572 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlwbIQDRAAt/h55zBNCEId0vkYSUQ3TQ==", 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB53YFloCb6CxAAmZQyz5XOEqDOOiROJHLzxQ==", 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFltBhxDxAAiAS9CYPY86IMmJ7NGgA2lw==", 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:17:10.574 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlUnAhBBAAi++W9XK3o3Y/185hB3LuIg==", 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:10.575 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:17:10.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.566+0000 7f120ffff640 1 -- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11f006a430 msgr2=0x7f11f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.566+0000 7f120ffff640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11f006a430 0x7f11f006c8f0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f120000a950 tx=0x7f1200008040 comp rx=0 tx=0).stop 2023-12-19T18:17:10.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.566+0000 7f120ffff640 1 -- 172.21.15.27:0/4270234355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f121015a3a0 msgr2=0x7f12100cd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.566+0000 7f120ffff640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f121015a3a0 0x7f12100cd590 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1208036040 tx=0x7f1208004a40 comp rx=0 tx=0).stop 2023-12-19T18:17:10.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.566+0000 7f120ffff640 1 -- 172.21.15.27:0/4270234355 shutdown_connections 2023-12-19T18:17:10.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.566+0000 7f120ffff640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11f006a430 0x7f11f006c8f0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.566+0000 7f120ffff640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f121015a3a0 0x7f12100cd590 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.566+0000 7f120ffff640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1210154170 0x7f12100a92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.566+0000 7f120ffff640 1 --2- 172.21.15.27:0/4270234355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1210070aa0 0x7f12101f2fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.566+0000 7f120ffff640 1 -- 172.21.15.27:0/4270234355 >> 172.21.15.27:0/4270234355 conn(0x7f121006a7d0 msgr2=0x7f12100af9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:10.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.566+0000 7f120ffff640 1 -- 172.21.15.27:0/4270234355 shutdown_connections 2023-12-19T18:17:10.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.566+0000 7f120ffff640 1 -- 172.21.15.27:0/4270234355 wait complete. 2023-12-19T18:17:10.586 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:10.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.826+0000 7f04ccac4640 1 Processor -- start 2023-12-19T18:17:10.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.827+0000 7f04ccac4640 1 -- start start 2023-12-19T18:17:10.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.828+0000 7f04ccac4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f04c8159c40 0x7f04c806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.828+0000 7f04ccac4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f04c815a020 0x7f04c806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.828+0000 7f04ccac4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04c815b180 con 0x7f04c806c880 2023-12-19T18:17:10.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.828+0000 7f04ccac4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c80ca850 con 0x7f04c8159c40 2023-12-19T18:17:10.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.828+0000 7f04ccac4640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04c815a570 con 0x7f04c815a020 2023-12-19T18:17:10.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.829+0000 7f04c659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f04c8159c40 0x7f04c806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.829+0000 7f04c659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f04c8159c40 0x7f04c806b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38208/0 (socket says 172.21.15.27:38208) 2023-12-19T18:17:10.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.829+0000 7f04c659c640 1 -- 172.21.15.27:0/469176911 learned_addr learned my addr 172.21.15.27:0/469176911 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:10.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.829+0000 7f04c659c640 1 -- 172.21.15.27:0/469176911 >> v2:172.21.15.192:3300/0 conn(0x7f04c815a020 msgr2=0x7f04c806ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:10.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.829+0000 7f04c659c640 1 --2- 172.21.15.27:0/469176911 >> v2:172.21.15.192:3300/0 conn(0x7f04c815a020 0x7f04c806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.829+0000 7f04c659c640 1 -- 172.21.15.27:0/469176911 >> v1:172.21.15.27:6789/0 conn(0x7f04c806c880 legacy=0x7f04c80cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:10.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.829+0000 7f04c659c640 1 -- 172.21.15.27:0/469176911 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04c815a850 con 0x7f04c8159c40 2023-12-19T18:17:10.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.831+0000 7f04c659c640 1 --2- 172.21.15.27:0/469176911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04c8159c40 0x7f04c806b4e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f04c80ae580 tx=0x7f04b802bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f66bcfd466879fd4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.831+0000 7f04c559a640 1 -- 172.21.15.27:0/469176911 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b802cb10 con 0x7f04c8159c40 2023-12-19T18:17:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.831+0000 7f04c559a640 1 -- 172.21.15.27:0/469176911 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04b802cc70 con 0x7f04c8159c40 2023-12-19T18:17:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.831+0000 7f04c559a640 1 -- 172.21.15.27:0/469176911 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04b80343f0 con 0x7f04c8159c40 2023-12-19T18:17:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.831+0000 7f04ccac4640 1 -- 172.21.15.27:0/469176911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04c8159c40 msgr2=0x7f04c806b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.831+0000 7f04ccac4640 1 --2- 172.21.15.27:0/469176911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04c8159c40 0x7f04c806b4e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f04c80ae580 tx=0x7f04b802bf70 comp rx=0 tx=0).stop 2023-12-19T18:17:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.831+0000 7f04ccac4640 1 -- 172.21.15.27:0/469176911 shutdown_connections 2023-12-19T18:17:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.832+0000 7f04ccac4640 1 --2- 172.21.15.27:0/469176911 >> v2:172.21.15.192:3300/0 conn(0x7f04c815a020 0x7f04c806ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.832+0000 7f04ccac4640 1 --2- 172.21.15.27:0/469176911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04c8159c40 0x7f04c806b4e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.832+0000 7f04ccac4640 1 -- 172.21.15.27:0/469176911 >> 172.21.15.27:0/469176911 conn(0x7f04c806a330 msgr2=0x7f04c80b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:10.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.832+0000 7f04ccac4640 1 -- 172.21.15.27:0/469176911 shutdown_connections 2023-12-19T18:17:10.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.832+0000 7f04ccac4640 1 -- 172.21.15.27:0/469176911 wait complete. 2023-12-19T18:17:10.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.833+0000 7f04ccac4640 1 Processor -- start 2023-12-19T18:17:10.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.834+0000 7f04ccac4640 1 -- start start 2023-12-19T18:17:10.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04ccac4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04c806c880 0x7f04c8157d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04ccac4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04c8159c40 0x7f04c80cc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04ccac4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04c815a020 0x7f04c80c4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04ccac4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c80b1210 con 0x7f04c815a020 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04ccac4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04c80cccf0 con 0x7f04c806c880 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04ccac4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04c80b1460 con 0x7f04c8159c40 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04c6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04c815a020 0x7f04c80c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04c6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04c815a020 0x7f04c80c4840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38224/0 (socket says 172.21.15.27:38224) 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04c6d9d640 1 -- 172.21.15.27:0/2581656600 learned_addr learned my addr 172.21.15.27:0/2581656600 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04c659c640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04c806c880 0x7f04c8157d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04c5d9b640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04c8159c40 0x7f04c80cc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04c6d9d640 1 -- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04c8159c40 msgr2=0x7f04c80cc330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04c6d9d640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04c8159c40 0x7f04c80cc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04c6d9d640 1 -- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04c806c880 msgr2=0x7f04c8157d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:10.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04c6d9d640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04c806c880 0x7f04c8157d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:10.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04c6d9d640 1 -- 172.21.15.27:0/2581656600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04b8009b30 con 0x7f04c815a020 2023-12-19T18:17:10.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.835+0000 7f04c659c640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04c806c880 0x7f04c8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:10.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.836+0000 7f04c6d9d640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04c815a020 0x7f04c80c4840 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f04c000b410 tx=0x7f04c000b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:10.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.836+0000 7f04beffd640 1 -- 172.21.15.27:0/2581656600 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0011830 con 0x7f04c815a020 2023-12-19T18:17:10.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.836+0000 7f04beffd640 1 -- 172.21.15.27:0/2581656600 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04c00119b0 con 0x7f04c815a020 2023-12-19T18:17:10.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.836+0000 7f04beffd640 1 -- 172.21.15.27:0/2581656600 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04c0011d60 con 0x7f04c815a020 2023-12-19T18:17:10.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.836+0000 7f04ccac4640 1 -- 172.21.15.27:0/2581656600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04c80b1740 con 0x7f04c815a020 2023-12-19T18:17:10.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.836+0000 7f04ccac4640 1 -- 172.21.15.27:0/2581656600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04c80cc870 con 0x7f04c815a020 2023-12-19T18:17:10.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.837+0000 7f04ccac4640 1 -- 172.21.15.27:0/2581656600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04c80b1c60 con 0x7f04c815a020 2023-12-19T18:17:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.838+0000 7f04beffd640 1 -- 172.21.15.27:0/2581656600 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f04c001a400 con 0x7f04c815a020 2023-12-19T18:17:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.839+0000 7f04beffd640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f04a806a480 0x7f04a806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:10.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.839+0000 7f04beffd640 1 -- 172.21.15.27:0/2581656600 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f04c0088670 con 0x7f04c815a020 2023-12-19T18:17:10.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.840+0000 7f04c659c640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f04a806a480 0x7f04a806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:10.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.841+0000 7f04c659c640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f04a806a480 0x7f04a806c940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04b8035000 tx=0x7f04b8030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:10.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:10.843+0000 7f04beffd640 1 -- 172.21.15.27:0/2581656600 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f04c80b1c60 con 0x7f04c815a020 2023-12-19T18:17:11.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.040+0000 7f04ccac4640 1 -- 172.21.15.27:0/2581656600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f04c80b1c60 con 0x7f04c815a020 2023-12-19T18:17:11.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.042+0000 7f04beffd640 1 -- 172.21.15.27:0/2581656600 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+12389 (secure 0 0 0) 0x7f04c80b1c60 con 0x7f04c815a020 2023-12-19T18:17:11.043 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:11.043 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:17:10.002864+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":35,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:11.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.045+0000 7f04ccac4640 1 -- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f04a806a480 msgr2=0x7f04a806c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.045+0000 7f04ccac4640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f04a806a480 0x7f04a806c940 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f04b8035000 tx=0x7f04b8030000 comp rx=0 tx=0).stop 2023-12-19T18:17:11.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.045+0000 7f04ccac4640 1 -- 172.21.15.27:0/2581656600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04c815a020 msgr2=0x7f04c80c4840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.045+0000 7f04ccac4640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04c815a020 0x7f04c80c4840 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f04c000b410 tx=0x7f04c000b8d0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.046+0000 7f04ccac4640 1 -- 172.21.15.27:0/2581656600 shutdown_connections 2023-12-19T18:17:11.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.046+0000 7f04ccac4640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f04a806a480 0x7f04a806c940 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.046+0000 7f04ccac4640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04c815a020 0x7f04c80c4840 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.046+0000 7f04ccac4640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f04c8159c40 0x7f04c80cc330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.046+0000 7f04ccac4640 1 --2- 172.21.15.27:0/2581656600 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f04c806c880 0x7f04c8157d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.046+0000 7f04ccac4640 1 -- 172.21.15.27:0/2581656600 >> 172.21.15.27:0/2581656600 conn(0x7f04c806a330 msgr2=0x7f04c815c350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:11.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.046+0000 7f04ccac4640 1 -- 172.21.15.27:0/2581656600 shutdown_connections 2023-12-19T18:17:11.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.046+0000 7f04ccac4640 1 -- 172.21.15.27:0/2581656600 wait complete. 2023-12-19T18:17:11.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:17:11.067 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:17:11.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.308+0000 7f46091b1640 1 Processor -- start 2023-12-19T18:17:11.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.308+0000 7f46091b1640 1 -- start start 2023-12-19T18:17:11.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.310+0000 7f46091b1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f460415b1a0 0x7f460406f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.310+0000 7f46091b1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4604071420 0x7f460415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.310+0000 7f46091b1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f460406bc80 con 0x7f46040c2720 2023-12-19T18:17:11.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.310+0000 7f46091b1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460406d690 con 0x7f460415b1a0 2023-12-19T18:17:11.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.310+0000 7f46091b1640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f460406d0a0 con 0x7f4604071420 2023-12-19T18:17:11.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.310+0000 7f4602d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f460415b1a0 0x7f460406f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.310+0000 7f4602d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f460415b1a0 0x7f460406f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38230/0 (socket says 172.21.15.27:38230) 2023-12-19T18:17:11.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.310+0000 7f4602d9d640 1 -- 172.21.15.27:0/1432411020 learned_addr learned my addr 172.21.15.27:0/1432411020 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:11.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.311+0000 7f4602d9d640 1 -- 172.21.15.27:0/1432411020 >> v2:172.21.15.192:3300/0 conn(0x7f4604071420 msgr2=0x7f460415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:11.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.311+0000 7f460359e640 1 --2- 172.21.15.27:0/1432411020 >> v2:172.21.15.192:3300/0 conn(0x7f4604071420 0x7f460415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.311+0000 7f4602d9d640 1 --2- 172.21.15.27:0/1432411020 >> v2:172.21.15.192:3300/0 conn(0x7f4604071420 0x7f460415ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.311+0000 7f4602d9d640 1 -- 172.21.15.27:0/1432411020 >> v1:172.21.15.27:6789/0 conn(0x7f46040c2720 legacy=0x7f460406fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.311+0000 7f4602d9d640 1 -- 172.21.15.27:0/1432411020 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f460406b6c0 con 0x7f460415b1a0 2023-12-19T18:17:11.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.312+0000 7f4602d9d640 1 --2- 172.21.15.27:0/1432411020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f460415b1a0 0x7f460406f520 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f460406d600 tx=0x7f45f402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a86b6f07aa281c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:11.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.312+0000 7f4601d9b640 1 -- 172.21.15.27:0/1432411020 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4031880 con 0x7f460415b1a0 2023-12-19T18:17:11.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.312+0000 7f4601d9b640 1 -- 172.21.15.27:0/1432411020 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f40319e0 con 0x7f460415b1a0 2023-12-19T18:17:11.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.312+0000 7f4601d9b640 1 -- 172.21.15.27:0/1432411020 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f4031d70 con 0x7f460415b1a0 2023-12-19T18:17:11.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.313+0000 7f46091b1640 1 -- 172.21.15.27:0/1432411020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f460415b1a0 msgr2=0x7f460406f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.313+0000 7f46091b1640 1 --2- 172.21.15.27:0/1432411020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f460415b1a0 0x7f460406f520 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f460406d600 tx=0x7f45f402b920 comp rx=0 tx=0).stop 2023-12-19T18:17:11.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.313+0000 7f46091b1640 1 -- 172.21.15.27:0/1432411020 shutdown_connections 2023-12-19T18:17:11.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.313+0000 7f46091b1640 1 --2- 172.21.15.27:0/1432411020 >> v2:172.21.15.192:3300/0 conn(0x7f4604071420 0x7f460415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.313+0000 7f46091b1640 1 --2- 172.21.15.27:0/1432411020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f460415b1a0 0x7f460406f520 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.313+0000 7f46091b1640 1 -- 172.21.15.27:0/1432411020 >> 172.21.15.27:0/1432411020 conn(0x7f46040cebd0 msgr2=0x7f46040cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:11.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.313+0000 7f46091b1640 1 -- 172.21.15.27:0/1432411020 shutdown_connections 2023-12-19T18:17:11.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.314+0000 7f46091b1640 1 -- 172.21.15.27:0/1432411020 wait complete. 2023-12-19T18:17:11.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.316+0000 7f46091b1640 1 Processor -- start 2023-12-19T18:17:11.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.316+0000 7f46091b1640 1 -- start start 2023-12-19T18:17:11.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.317+0000 7f46091b1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4604071420 0x7f4604151c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.317+0000 7f46091b1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46040c2720 0x7f46041e1790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f46091b1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f460415b1a0 0x7f46040c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f46091b1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46041e04a0 con 0x7f46040c2720 2023-12-19T18:17:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f46091b1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46041e06f0 con 0x7f4604071420 2023-12-19T18:17:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f46091b1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46041e09d0 con 0x7f460415b1a0 2023-12-19T18:17:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f460259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46040c2720 0x7f46041e1790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f460259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46040c2720 0x7f46041e1790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38236/0 (socket says 172.21.15.27:38236) 2023-12-19T18:17:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f460259c640 1 -- 172.21.15.27:0/1657010285 learned_addr learned my addr 172.21.15.27:0/1657010285 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f460359e640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f460415b1a0 0x7f46040c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f4602d9d640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4604071420 0x7f4604151c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f460259c640 1 -- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f460415b1a0 msgr2=0x7f46040c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f460259c640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f460415b1a0 0x7f46040c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f460259c640 1 -- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4604071420 msgr2=0x7f4604151c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f460259c640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4604071420 0x7f4604151c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f460259c640 1 -- 172.21.15.27:0/1657010285 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f40094d0 con 0x7f46040c2720 2023-12-19T18:17:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.318+0000 7f460359e640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f460415b1a0 0x7f46040c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.319+0000 7f460259c640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46040c2720 0x7f46041e1790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f45f800c450 tx=0x7f45f800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.319+0000 7f45eb7fe640 1 -- 172.21.15.27:0/1657010285 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f80052d0 con 0x7f46040c2720 2023-12-19T18:17:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.319+0000 7f45eb7fe640 1 -- 172.21.15.27:0/1657010285 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f8005430 con 0x7f46040c2720 2023-12-19T18:17:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.319+0000 7f45eb7fe640 1 -- 172.21.15.27:0/1657010285 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f8012870 con 0x7f46040c2720 2023-12-19T18:17:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.319+0000 7f46091b1640 1 -- 172.21.15.27:0/1657010285 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46041e0cb0 con 0x7f46040c2720 2023-12-19T18:17:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.319+0000 7f46091b1640 1 -- 172.21.15.27:0/1657010285 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46041e1270 con 0x7f46040c2720 2023-12-19T18:17:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.321+0000 7f46091b1640 1 -- 172.21.15.27:0/1657010285 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4604068e90 con 0x7f46040c2720 2023-12-19T18:17:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.321+0000 7f45eb7fe640 1 -- 172.21.15.27:0/1657010285 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f45f80129d0 con 0x7f46040c2720 2023-12-19T18:17:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.323+0000 7f45eb7fe640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45f006a460 0x7f45f006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.323+0000 7f45eb7fe640 1 -- 172.21.15.27:0/1657010285 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f45f8087f00 con 0x7f46040c2720 2023-12-19T18:17:11.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.324+0000 7f4602d9d640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45f006a460 0x7f45f006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.325+0000 7f4602d9d640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45f006a460 0x7f45f006c920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f45f4036000 tx=0x7f45f403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:11.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.328+0000 7f45eb7fe640 1 -- 172.21.15.27:0/1657010285 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f45f8053830 con 0x7f46040c2720 2023-12-19T18:17:11.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.521+0000 7f46091b1640 1 -- 172.21.15.27:0/1657010285 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f460406bc80 con 0x7f46040c2720 2023-12-19T18:17:11.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.522+0000 7f45eb7fe640 1 -- 172.21.15.27:0/1657010285 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7f460406bc80 con 0x7f46040c2720 2023-12-19T18:17:11.522 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:17:11.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.525+0000 7f46091b1640 1 -- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45f006a460 msgr2=0x7f45f006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.525+0000 7f46091b1640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45f006a460 0x7f45f006c920 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f45f4036000 tx=0x7f45f403c040 comp rx=0 tx=0).stop 2023-12-19T18:17:11.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.525+0000 7f46091b1640 1 -- 172.21.15.27:0/1657010285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46040c2720 msgr2=0x7f46041e1790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.525+0000 7f46091b1640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46040c2720 0x7f46041e1790 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f45f800c450 tx=0x7f45f800c910 comp rx=0 tx=0).stop 2023-12-19T18:17:11.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.526+0000 7f46091b1640 1 -- 172.21.15.27:0/1657010285 shutdown_connections 2023-12-19T18:17:11.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.526+0000 7f46091b1640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45f006a460 0x7f45f006c920 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.526+0000 7f46091b1640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f460415b1a0 0x7f46040c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.526+0000 7f46091b1640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46040c2720 0x7f46041e1790 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.526+0000 7f46091b1640 1 --2- 172.21.15.27:0/1657010285 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4604071420 0x7f4604151c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.526+0000 7f46091b1640 1 -- 172.21.15.27:0/1657010285 >> 172.21.15.27:0/1657010285 conn(0x7f46040cebd0 msgr2=0x7f46040c3870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:11.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.526+0000 7f46091b1640 1 -- 172.21.15.27:0/1657010285 shutdown_connections 2023-12-19T18:17:11.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.526+0000 7f46091b1640 1 -- 172.21.15.27:0/1657010285 wait complete. 2023-12-19T18:17:11.545 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:17:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.771+0000 7feadb59e640 1 Processor -- start 2023-12-19T18:17:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.771+0000 7feadb59e640 1 -- start start 2023-12-19T18:17:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.772+0000 7feadb59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc0b49d0 0x7feadc160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.773+0000 7feadb59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc0b5ca0 con 0x7feadc0b89b0 2023-12-19T18:17:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.773+0000 7feadb59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feadc161800 con 0x7feadc15cf90 2023-12-19T18:17:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.773+0000 7feadb59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc161240 con 0x7feadc0b49d0 2023-12-19T18:17:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.773+0000 7fead9d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7feadc0b89b0 0x7feadc15c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55074/0 (socket says 172.21.15.27:55074) 2023-12-19T18:17:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.773+0000 7fead9d9b640 1 -- 172.21.15.27:0/3675449701 learned_addr learned my addr 172.21.15.27:0/3675449701 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.773+0000 7feada59c640 1 --2- 172.21.15.27:0/3675449701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc0b49d0 0x7feadc160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.773+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2756727328 0 0) 0x7feadc0b5ca0 con 0x7feadc0b89b0 2023-12-19T18:17:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.773+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feac8003520 con 0x7feadc0b89b0 2023-12-19T18:17:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3377315987 0 0) 0x7feac8003520 con 0x7feadc0b89b0 2023-12-19T18:17:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc0b49d0 msgr2=0x7feadc160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7fead959a640 1 --2- 172.21.15.27:0/3675449701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc0b49d0 0x7feadc160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 >> v1:172.21.15.192:6789/0 conn(0x7feadc15cf90 legacy=0x7feadc0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc161520 con 0x7feadc0b89b0 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7feac4001670 con 0x7feadc0b89b0 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 >> v1:172.21.15.27:6789/0 conn(0x7feadc0b89b0 legacy=0x7feadc15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7fead959a640 1 --2- 172.21.15.27:0/3675449701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc0b49d0 0x7feac80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7fead959a640 1 --2- 172.21.15.27:0/3675449701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac8003f20 0x7feac80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7fead959a640 1 --2- 172.21.15.27:0/3675449701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac8004d00 0x7feac8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc161240 con 0x7feac8004d00 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac800baf0 con 0x7feac8003f20 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac800bcb0 con 0x7feadc0b49d0 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.774+0000 7feadad9d640 1 --2- 172.21.15.27:0/3675449701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac8004d00 0x7feac8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.775+0000 7fead9d9b640 1 --2- 172.21.15.27:0/3675449701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac8003f20 0x7feac80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.775+0000 7feada59c640 1 --2- 172.21.15.27:0/3675449701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc0b49d0 0x7feac80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.775+0000 7feadad9d640 1 -- 172.21.15.27:0/3675449701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc0b49d0 msgr2=0x7feac80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.775+0000 7feadad9d640 1 --2- 172.21.15.27:0/3675449701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc0b49d0 0x7feac80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.775+0000 7feadad9d640 1 -- 172.21.15.27:0/3675449701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac8003f20 msgr2=0x7feac80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.775+0000 7feadad9d640 1 --2- 172.21.15.27:0/3675449701 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac8003f20 0x7feac80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.775+0000 7feadad9d640 1 -- 172.21.15.27:0/3675449701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac8003520 con 0x7feac8004d00 2023-12-19T18:17:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.775+0000 7feada59c640 1 --2- 172.21.15.27:0/3675449701 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc0b49d0 0x7feac80039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.776+0000 7feadad9d640 1 --2- 172.21.15.27:0/3675449701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac8004d00 0x7feac8005180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fead4001750 tx=0x7feac80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7686d0c73fd031fe server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.776+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead40052d0 con 0x7feac8004d00 2023-12-19T18:17:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.776+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead402e050 con 0x7feac8004d00 2023-12-19T18:17:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.776+0000 7fead959a640 1 -- 172.21.15.27:0/3675449701 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4003830 con 0x7feac8004d00 2023-12-19T18:17:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.777+0000 7feadb59e640 1 -- 172.21.15.27:0/3675449701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac8004d00 msgr2=0x7feac8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.777+0000 7feadb59e640 1 --2- 172.21.15.27:0/3675449701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac8004d00 0x7feac8005180 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fead4001750 tx=0x7feac80057f0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.777+0000 7feadad9d640 1 -- 172.21.15.27:0/3675449701 reap_dead start 2023-12-19T18:17:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.777+0000 7feadb59e640 1 -- 172.21.15.27:0/3675449701 shutdown_connections 2023-12-19T18:17:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.777+0000 7feadb59e640 1 -- 172.21.15.27:0/3675449701 >> 172.21.15.27:0/3675449701 conn(0x7feadc0cf760 msgr2=0x7feadc0cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.777+0000 7feadb59e640 1 -- 172.21.15.27:0/3675449701 shutdown_connections 2023-12-19T18:17:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.777+0000 7feadb59e640 1 -- 172.21.15.27:0/3675449701 wait complete. 2023-12-19T18:17:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.779+0000 7feadb59e640 1 Processor -- start 2023-12-19T18:17:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.779+0000 7feadb59e640 1 -- start start 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.780+0000 7feadb59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac8004d00 0x7feadc0c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.780+0000 7feadb59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc1501e0 0x7feadc06bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.780+0000 7feadb59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feadc150640 0x7feadc06c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.780+0000 7feadb59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc0cbc50 con 0x7feadc150640 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.780+0000 7feadb59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feadc14f420 con 0x7feac8004d00 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.780+0000 7feadb59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feadc14f700 con 0x7feadc1501e0 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.780+0000 7feadad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feadc150640 0x7feadc06c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.780+0000 7feadad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feadc150640 0x7feadc06c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38258/0 (socket says 172.21.15.27:38258) 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.780+0000 7feadad9d640 1 -- 172.21.15.27:0/3130268408 learned_addr learned my addr 172.21.15.27:0/3130268408 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.780+0000 7fead9d9b640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc1501e0 0x7feadc06bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.780+0000 7feada59c640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac8004d00 0x7feadc0c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.781+0000 7feadad9d640 1 -- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc1501e0 msgr2=0x7feadc06bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.781+0000 7feadad9d640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc1501e0 0x7feadc06bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.781+0000 7feadad9d640 1 -- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac8004d00 msgr2=0x7feadc0c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.781+0000 7feadad9d640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac8004d00 0x7feadc0c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.781+0000 7feadad9d640 1 -- 172.21.15.27:0/3130268408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feadc14f9e0 con 0x7feadc150640 2023-12-19T18:17:11.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.781+0000 7feada59c640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac8004d00 0x7feadc0c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:11.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.781+0000 7feadad9d640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feadc150640 0x7feadc06c140 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fead4034040 tx=0x7fead4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:11.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.781+0000 7fead2ffd640 1 -- 172.21.15.27:0/3130268408 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead40054c0 con 0x7feadc150640 2023-12-19T18:17:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.781+0000 7fead2ffd640 1 -- 172.21.15.27:0/3130268408 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fead4007a30 con 0x7feadc150640 2023-12-19T18:17:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.781+0000 7fead2ffd640 1 -- 172.21.15.27:0/3130268408 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fead4006a10 con 0x7feadc150640 2023-12-19T18:17:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.781+0000 7feadb59e640 1 -- 172.21.15.27:0/3130268408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feadc14fcc0 con 0x7feadc150640 2023-12-19T18:17:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.781+0000 7feadb59e640 1 -- 172.21.15.27:0/3130268408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feadc0c77b0 con 0x7feadc150640 2023-12-19T18:17:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.783+0000 7feadb59e640 1 -- 172.21.15.27:0/3130268408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feadc14feb0 con 0x7feadc150640 2023-12-19T18:17:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.784+0000 7fead2ffd640 1 -- 172.21.15.27:0/3130268408 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fead4006b70 con 0x7feadc150640 2023-12-19T18:17:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.784+0000 7fead2ffd640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feabc06a430 0x7feabc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.785+0000 7fead2ffd640 1 -- 172.21.15.27:0/3130268408 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fead40a9300 con 0x7feadc150640 2023-12-19T18:17:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.785+0000 7feada59c640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feabc06a430 0x7feabc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:11.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.786+0000 7feada59c640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feabc06a430 0x7feabc06c8f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feacc007580 tx=0x7feacc00d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:11.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.789+0000 7fead2ffd640 1 -- 172.21.15.27:0/3130268408 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fead4074bb0 con 0x7feadc150640 2023-12-19T18:17:11.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.984+0000 7feadb59e640 1 -- 172.21.15.27:0/3130268408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7feadc04e840 con 0x7feadc150640 2023-12-19T18:17:11.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.985+0000 7fead2ffd640 1 -- 172.21.15.27:0/3130268408 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v46) v1 ==== 84+0+65 (secure 0 0 0) 0x7fead40781d0 con 0x7feadc150640 2023-12-19T18:17:11.986 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:11.986 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-12-19T18:17:11.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:11.986 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-12-19T18:17:11.986 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-12-19T18:17:11.986 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:11.986 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-12-19T18:17:11.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.987+0000 7feadb59e640 1 -- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feabc06a430 msgr2=0x7feabc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.987+0000 7feadb59e640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feabc06a430 0x7feabc06c8f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7feacc007580 tx=0x7feacc00d040 comp rx=0 tx=0).stop 2023-12-19T18:17:11.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.987+0000 7feadb59e640 1 -- 172.21.15.27:0/3130268408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feadc150640 msgr2=0x7feadc06c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:11.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.987+0000 7feadb59e640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feadc150640 0x7feadc06c140 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fead4034040 tx=0x7fead4001120 comp rx=0 tx=0).stop 2023-12-19T18:17:11.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.987+0000 7feadb59e640 1 -- 172.21.15.27:0/3130268408 shutdown_connections 2023-12-19T18:17:11.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.987+0000 7feadb59e640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feabc06a430 0x7feabc06c8f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.987+0000 7feadb59e640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feadc150640 0x7feadc06c140 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.987+0000 7feadb59e640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feadc1501e0 0x7feadc06bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.987+0000 7feadb59e640 1 --2- 172.21.15.27:0/3130268408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feac8004d00 0x7feadc0c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:11.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.987+0000 7feadb59e640 1 -- 172.21.15.27:0/3130268408 >> 172.21.15.27:0/3130268408 conn(0x7feadc0cf760 msgr2=0x7feadc15c6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:11.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.988+0000 7feadb59e640 1 -- 172.21.15.27:0/3130268408 shutdown_connections 2023-12-19T18:17:11.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:11.988+0000 7feadb59e640 1 -- 172.21.15.27:0/3130268408 wait complete. 2023-12-19T18:17:12.008 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:17:12.009 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:17:12.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.238+0000 7f1169def640 1 Processor -- start 2023-12-19T18:17:12.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.238+0000 7f1169def640 1 -- start start 2023-12-19T18:17:12.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.240+0000 7f1169def640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f11640cb840 0x7f1164167cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.240+0000 7f1169def640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1164160920 con 0x7f116415dfe0 2023-12-19T18:17:12.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.240+0000 7f1169def640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11640cd220 con 0x7f116415dc00 2023-12-19T18:17:12.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.240+0000 7f1169def640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11640c9fa0 con 0x7f11640cb840 2023-12-19T18:17:12.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.241+0000 7f1163fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f116415dfe0 0x7f11640cb130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41764/0 (socket says 172.21.15.27:41764) 2023-12-19T18:17:12.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.241+0000 7f1163fff640 1 -- 172.21.15.27:0/345150056 learned_addr learned my addr 172.21.15.27:0/345150056 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:12.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.241+0000 7f11695ee640 1 --2- 172.21.15.27:0/345150056 >> v2:172.21.15.192:3300/0 conn(0x7f11640cb840 0x7f1164167cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.241+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3959968866 0 0) 0x7f1164160920 con 0x7f116415dfe0 2023-12-19T18:17:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.241+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1150003520 con 0x7f116415dfe0 2023-12-19T18:17:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 261334217 0 0) 0x7f1150003520 con 0x7f116415dfe0 2023-12-19T18:17:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 >> v2:172.21.15.192:3300/0 conn(0x7f11640cb840 msgr2=0x7f1164167cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f11637fe640 1 --2- 172.21.15.27:0/345150056 >> v2:172.21.15.192:3300/0 conn(0x7f11640cb840 0x7f1164167cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 >> v1:172.21.15.192:6789/0 conn(0x7f116415dc00 legacy=0x7f11640cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1164160be0 con 0x7f116415dfe0 2023-12-19T18:17:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f114c001670 con 0x7f116415dfe0 2023-12-19T18:17:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 >> v1:172.21.15.27:6789/0 conn(0x7f116415dfe0 legacy=0x7f11640cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f11637fe640 1 --2- 172.21.15.27:0/345150056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1150003970 0x7f1150003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f11637fe640 1 --2- 172.21.15.27:0/345150056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11500047c0 0x7f1150004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f11637fe640 1 --2- 172.21.15.27:0/345150056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11500052a0 0x7f115000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11640c9fa0 con 0x7f1150003970 2023-12-19T18:17:12.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f115000c170 con 0x7f11500047c0 2023-12-19T18:17:12.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f115000c330 con 0x7f11500052a0 2023-12-19T18:17:12.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.242+0000 7f1168ded640 1 --2- 172.21.15.27:0/345150056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1150003970 0x7f1150003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.243+0000 7f1163fff640 1 --2- 172.21.15.27:0/345150056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11500047c0 0x7f1150004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.243+0000 7f11695ee640 1 --2- 172.21.15.27:0/345150056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11500052a0 0x7f115000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.243+0000 7f1168ded640 1 -- 172.21.15.27:0/345150056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11500052a0 msgr2=0x7f115000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.243+0000 7f1168ded640 1 --2- 172.21.15.27:0/345150056 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11500052a0 0x7f115000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.243+0000 7f1168ded640 1 -- 172.21.15.27:0/345150056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11500047c0 msgr2=0x7f1150004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.243+0000 7f1168ded640 1 --2- 172.21.15.27:0/345150056 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11500047c0 0x7f1150004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.243+0000 7f1168ded640 1 -- 172.21.15.27:0/345150056 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1150003520 con 0x7f1150003970 2023-12-19T18:17:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.243+0000 7f11695ee640 1 -- 172.21.15.27:0/345150056 reap_dead start 2023-12-19T18:17:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.245+0000 7f1168ded640 1 --2- 172.21.15.27:0/345150056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1150003970 0x7f1150003d90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1154001e20 tx=0x7f11500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26db03ed50689 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.245+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154003b70 con 0x7f1150003970 2023-12-19T18:17:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.245+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1154003cd0 con 0x7f1150003970 2023-12-19T18:17:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.245+0000 7f11637fe640 1 -- 172.21.15.27:0/345150056 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1154005310 con 0x7f1150003970 2023-12-19T18:17:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.245+0000 7f1169def640 1 -- 172.21.15.27:0/345150056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1150003970 msgr2=0x7f1150003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.246+0000 7f1169def640 1 --2- 172.21.15.27:0/345150056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1150003970 0x7f1150003d90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1154001e20 tx=0x7f11500057f0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.246+0000 7f1169def640 1 -- 172.21.15.27:0/345150056 shutdown_connections 2023-12-19T18:17:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.246+0000 7f1169def640 1 --2- 172.21.15.27:0/345150056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1150003970 0x7f1150003d90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.246+0000 7f1169def640 1 -- 172.21.15.27:0/345150056 >> 172.21.15.27:0/345150056 conn(0x7f11640b05b0 msgr2=0x7f11640b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:12.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.246+0000 7f1169def640 1 -- 172.21.15.27:0/345150056 shutdown_connections 2023-12-19T18:17:12.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.246+0000 7f1169def640 1 -- 172.21.15.27:0/345150056 wait complete. 2023-12-19T18:17:12.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.248+0000 7f1169def640 1 Processor -- start 2023-12-19T18:17:12.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.248+0000 7f1169def640 1 -- start start 2023-12-19T18:17:12.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.249+0000 7f1169def640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1150003970 0x7f116414c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.249+0000 7f1169def640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f116415f610 0x7f116415fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.249+0000 7f1169def640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11641f3ae0 0x7f11641f3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.249+0000 7f1169def640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11640c4880 con 0x7f11641f3ae0 2023-12-19T18:17:12.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.249+0000 7f1169def640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11640c3c00 con 0x7f116415f610 2023-12-19T18:17:12.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.249+0000 7f1169def640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11640c3ee0 con 0x7f1150003970 2023-12-19T18:17:12.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.249+0000 7f11695ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11641f3ae0 0x7f11641f3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.249+0000 7f11695ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11641f3ae0 0x7f11641f3f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33988/0 (socket says 172.21.15.27:33988) 2023-12-19T18:17:12.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.249+0000 7f11695ee640 1 -- 172.21.15.27:0/2153277123 learned_addr learned my addr 172.21.15.27:0/2153277123 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:12.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.249+0000 7f1168ded640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1150003970 0x7f116414c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.249+0000 7f1163fff640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f116415f610 0x7f116415fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.250+0000 7f11695ee640 1 -- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1150003970 msgr2=0x7f116414c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.250+0000 7f11695ee640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1150003970 0x7f116414c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.250+0000 7f11695ee640 1 -- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f116415f610 msgr2=0x7f116415fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.250+0000 7f11695ee640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f116415f610 0x7f116415fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.250+0000 7f11695ee640 1 -- 172.21.15.27:0/2153277123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11640c41c0 con 0x7f11641f3ae0 2023-12-19T18:17:12.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.250+0000 7f1163fff640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f116415f610 0x7f116415fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:12.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.250+0000 7f11695ee640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11641f3ae0 0x7f11641f3f60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f11640cda30 tx=0x7f115c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:12.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.250+0000 7f11617fa640 1 -- 172.21.15.27:0/2153277123 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c004440 con 0x7f11641f3ae0 2023-12-19T18:17:12.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.250+0000 7f11617fa640 1 -- 172.21.15.27:0/2153277123 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f115c003610 con 0x7f11641f3ae0 2023-12-19T18:17:12.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.250+0000 7f11617fa640 1 -- 172.21.15.27:0/2153277123 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f115c0039c0 con 0x7f11641f3ae0 2023-12-19T18:17:12.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.250+0000 7f1169def640 1 -- 172.21.15.27:0/2153277123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11640c44a0 con 0x7f11641f3ae0 2023-12-19T18:17:12.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.250+0000 7f1169def640 1 -- 172.21.15.27:0/2153277123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11640c4ca0 con 0x7f11641f3ae0 2023-12-19T18:17:12.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.252+0000 7f1169def640 1 -- 172.21.15.27:0/2153277123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f116404e840 con 0x7f11641f3ae0 2023-12-19T18:17:12.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.252+0000 7f11617fa640 1 -- 172.21.15.27:0/2153277123 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f115c003b20 con 0x7f11641f3ae0 2023-12-19T18:17:12.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.253+0000 7f11617fa640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f114406a430 0x7f114406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:12.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.253+0000 7f11617fa640 1 -- 172.21.15.27:0/2153277123 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f115c0883f0 con 0x7f11641f3ae0 2023-12-19T18:17:12.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.254+0000 7f1168ded640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f114406a430 0x7f114406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:12.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.255+0000 7f1168ded640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f114406a430 0x7f114406c8f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1154034000 tx=0x7f1154009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:12.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.258+0000 7f11617fa640 1 -- 172.21.15.27:0/2153277123 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f115c053dc0 con 0x7f11641f3ae0 2023-12-19T18:17:12.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.457+0000 7f1169def640 1 -- 172.21.15.27:0/2153277123 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1164154230 con 0x7f11641f3ae0 2023-12-19T18:17:12.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.741+0000 7f11617fa640 1 -- 172.21.15.27:0/2153277123 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f115c0573e0 con 0x7f11641f3ae0 2023-12-19T18:17:12.742 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-12-19T18:17:12.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.747+0000 7f1169def640 1 -- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f114406a430 msgr2=0x7f114406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.747+0000 7f1169def640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f114406a430 0x7f114406c8f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f1154034000 tx=0x7f1154009030 comp rx=0 tx=0).stop 2023-12-19T18:17:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.747+0000 7f1169def640 1 -- 172.21.15.27:0/2153277123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11641f3ae0 msgr2=0x7f11641f3f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.747+0000 7f1169def640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11641f3ae0 0x7f11641f3f60 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f11640cda30 tx=0x7f115c009f90 comp rx=0 tx=0).stop 2023-12-19T18:17:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.747+0000 7f1169def640 1 -- 172.21.15.27:0/2153277123 shutdown_connections 2023-12-19T18:17:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.747+0000 7f1169def640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f114406a430 0x7f114406c8f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.747+0000 7f1169def640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11641f3ae0 0x7f11641f3f60 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.747+0000 7f1169def640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f116415f610 0x7f116415fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.747+0000 7f1169def640 1 --2- 172.21.15.27:0/2153277123 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1150003970 0x7f116414c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:12.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.747+0000 7f1169def640 1 -- 172.21.15.27:0/2153277123 >> 172.21.15.27:0/2153277123 conn(0x7f11640b05b0 msgr2=0x7f116415c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:12.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.749+0000 7f1169def640 1 -- 172.21.15.27:0/2153277123 shutdown_connections 2023-12-19T18:17:12.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:12.749+0000 7f1169def640 1 -- 172.21.15.27:0/2153277123 wait complete. 2023-12-19T18:17:12.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:17:13.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.011+0000 7f8b7759e640 1 Processor -- start 2023-12-19T18:17:13.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.011+0000 7f8b7759e640 1 -- start start 2023-12-19T18:17:13.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.013+0000 7f8b7759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b7815ce50 0x7f8b780d24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.013+0000 7f8b7759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b780d0d60 0x7f8b780cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.013+0000 7f8b7759e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b780d0100 con 0x7f8b780cfcb0 2023-12-19T18:17:13.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.013+0000 7f8b7759e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b780d1ee0 con 0x7f8b780d0d60 2023-12-19T18:17:13.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.013+0000 7f8b7759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b780d12d0 con 0x7f8b7815ce50 2023-12-19T18:17:13.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.013+0000 7f8b76d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8b780cfcb0 0x7f8b780b17e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41770/0 (socket says 172.21.15.27:41770) 2023-12-19T18:17:13.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.013+0000 7f8b75d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8b780d0d60 0x7f8b780cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.013+0000 7f8b76d9d640 1 -- 172.21.15.27:0/2812901984 learned_addr learned my addr 172.21.15.27:0/2812901984 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:13.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.014+0000 7f8b7659c640 1 --2- 172.21.15.27:0/2812901984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b7815ce50 0x7f8b780d24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.014+0000 7f8b7559a640 1 -- 172.21.15.27:0/2812901984 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1738057645 0 0) 0x7f8b70001110 con 0x7f8b780cfcb0 2023-12-19T18:17:13.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.014+0000 7f8b7559a640 1 -- 172.21.15.27:0/2812901984 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b64003520 con 0x7f8b780cfcb0 2023-12-19T18:17:13.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.014+0000 7f8b75d9b640 1 -- 172.21.15.27:0/2812901984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b7815ce50 msgr2=0x7f8b780d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:13.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.014+0000 7f8b75d9b640 1 --2- 172.21.15.27:0/2812901984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b7815ce50 0x7f8b780d24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.014+0000 7f8b75d9b640 1 -- 172.21.15.27:0/2812901984 >> v1:172.21.15.27:6789/0 conn(0x7f8b780cfcb0 legacy=0x7f8b780b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.014+0000 7f8b75d9b640 1 -- 172.21.15.27:0/2812901984 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780d15b0 con 0x7f8b780d0d60 2023-12-19T18:17:13.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.014+0000 7f8b7659c640 1 --2- 172.21.15.27:0/2812901984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b7815ce50 0x7f8b780d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:17:13.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.016+0000 7f8b75d9b640 1 --2- 172.21.15.27:0/2812901984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b780d0d60 0x7f8b780cf770 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8b600097a0 tx=0x7f8b6002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afb6a98ec393f1c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:13.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.016+0000 7f8b7559a640 1 -- 172.21.15.27:0/2812901984 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60031830 con 0x7f8b780d0d60 2023-12-19T18:17:13.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.016+0000 7f8b7559a640 1 -- 172.21.15.27:0/2812901984 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b60031990 con 0x7f8b780d0d60 2023-12-19T18:17:13.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.016+0000 7f8b7559a640 1 -- 172.21.15.27:0/2812901984 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b60031d20 con 0x7f8b780d0d60 2023-12-19T18:17:13.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.017+0000 7f8b7759e640 1 -- 172.21.15.27:0/2812901984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b780d0d60 msgr2=0x7f8b780cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.017+0000 7f8b7759e640 1 --2- 172.21.15.27:0/2812901984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b780d0d60 0x7f8b780cf770 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8b600097a0 tx=0x7f8b6002b8d0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.017+0000 7f8b7759e640 1 -- 172.21.15.27:0/2812901984 shutdown_connections 2023-12-19T18:17:13.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.017+0000 7f8b7759e640 1 --2- 172.21.15.27:0/2812901984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b780d0d60 0x7f8b780cf770 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.017+0000 7f8b7759e640 1 --2- 172.21.15.27:0/2812901984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b7815ce50 0x7f8b780d24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.017+0000 7f8b7759e640 1 -- 172.21.15.27:0/2812901984 >> 172.21.15.27:0/2812901984 conn(0x7f8b7806a2d0 msgr2=0x7f8b7806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:13.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.017+0000 7f8b7759e640 1 -- 172.21.15.27:0/2812901984 shutdown_connections 2023-12-19T18:17:13.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.017+0000 7f8b7759e640 1 -- 172.21.15.27:0/2812901984 wait complete. 2023-12-19T18:17:13.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.019+0000 7f8b7759e640 1 Processor -- start 2023-12-19T18:17:13.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.019+0000 7f8b7759e640 1 -- start start 2023-12-19T18:17:13.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.020+0000 7f8b7759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b780cfcb0 0x7f8b781574b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.020+0000 7f8b7759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b780d0d60 0x7f8b7815f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.020+0000 7f8b7759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7815ce50 0x7f8b781f2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.020+0000 7f8b7759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b78154b70 con 0x7f8b7815ce50 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.020+0000 7f8b7759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b78154d80 con 0x7f8b780cfcb0 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.020+0000 7f8b7759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b78155060 con 0x7f8b780d0d60 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.020+0000 7f8b76d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7815ce50 0x7f8b781f2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.020+0000 7f8b76d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7815ce50 0x7f8b781f2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33996/0 (socket says 172.21.15.27:33996) 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.020+0000 7f8b76d9d640 1 -- 172.21.15.27:0/885101624 learned_addr learned my addr 172.21.15.27:0/885101624 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.020+0000 7f8b75d9b640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b780d0d60 0x7f8b7815f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.020+0000 7f8b7659c640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b780cfcb0 0x7f8b781574b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.021+0000 7f8b76d9d640 1 -- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b780d0d60 msgr2=0x7f8b7815f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.021+0000 7f8b76d9d640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b780d0d60 0x7f8b7815f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.021+0000 7f8b76d9d640 1 -- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b780cfcb0 msgr2=0x7f8b781574b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.021+0000 7f8b76d9d640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b780cfcb0 0x7f8b781574b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.021+0000 7f8b76d9d640 1 -- 172.21.15.27:0/885101624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b60009400 con 0x7f8b7815ce50 2023-12-19T18:17:13.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.021+0000 7f8b76d9d640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7815ce50 0x7f8b781f2670 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000c390 tx=0x7f8b7000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:13.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.021+0000 7f8b6effd640 1 -- 172.21.15.27:0/885101624 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70005ce0 con 0x7f8b7815ce50 2023-12-19T18:17:13.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.021+0000 7f8b6effd640 1 -- 172.21.15.27:0/885101624 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b70005e40 con 0x7f8b7815ce50 2023-12-19T18:17:13.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.021+0000 7f8b7759e640 1 -- 172.21.15.27:0/885101624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b78155340 con 0x7f8b7815ce50 2023-12-19T18:17:13.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.021+0000 7f8b6effd640 1 -- 172.21.15.27:0/885101624 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b700072f0 con 0x7f8b7815ce50 2023-12-19T18:17:13.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.021+0000 7f8b7759e640 1 -- 172.21.15.27:0/885101624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b781579f0 con 0x7f8b7815ce50 2023-12-19T18:17:13.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.022+0000 7f8b7759e640 1 -- 172.21.15.27:0/885101624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b780b8d70 con 0x7f8b7815ce50 2023-12-19T18:17:13.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.024+0000 7f8b6effd640 1 -- 172.21.15.27:0/885101624 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8b70007450 con 0x7f8b7815ce50 2023-12-19T18:17:13.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.025+0000 7f8b6effd640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b5806a460 0x7f8b5806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.025+0000 7f8b6effd640 1 -- 172.21.15.27:0/885101624 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f8b70087f80 con 0x7f8b7815ce50 2023-12-19T18:17:13.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.026+0000 7f8b7659c640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b5806a460 0x7f8b5806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.027+0000 7f8b7659c640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b5806a460 0x7f8b5806c920 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8b68003fd0 tx=0x7f8b68003990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:13.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.029+0000 7f8b6effd640 1 -- 172.21.15.27:0/885101624 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8b700537c0 con 0x7f8b7815ce50 2023-12-19T18:17:13.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.224+0000 7f8b7759e640 1 -- 172.21.15.27:0/885101624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8b780d0100 con 0x7f8b7815ce50 2023-12-19T18:17:13.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.748+0000 7f8b6effd640 1 -- 172.21.15.27:0/885101624 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v48) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8b780d0100 con 0x7f8b7815ce50 2023-12-19T18:17:13.755 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-12-19T18:17:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.760+0000 7f8b7759e640 1 -- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b5806a460 msgr2=0x7f8b5806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.760+0000 7f8b7759e640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b5806a460 0x7f8b5806c920 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f8b68003fd0 tx=0x7f8b68003990 comp rx=0 tx=0).stop 2023-12-19T18:17:13.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.760+0000 7f8b7759e640 1 -- 172.21.15.27:0/885101624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7815ce50 msgr2=0x7f8b781f2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.760+0000 7f8b7759e640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7815ce50 0x7f8b781f2670 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f8b7000c390 tx=0x7f8b7000c850 comp rx=0 tx=0).stop 2023-12-19T18:17:13.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.763+0000 7f8b7759e640 1 -- 172.21.15.27:0/885101624 shutdown_connections 2023-12-19T18:17:13.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.763+0000 7f8b7759e640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b5806a460 0x7f8b5806c920 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.763+0000 7f8b7759e640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b7815ce50 0x7f8b781f2670 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.763+0000 7f8b7759e640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b780d0d60 0x7f8b7815f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.763+0000 7f8b7759e640 1 --2- 172.21.15.27:0/885101624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b780cfcb0 0x7f8b781574b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.763+0000 7f8b7759e640 1 -- 172.21.15.27:0/885101624 >> 172.21.15.27:0/885101624 conn(0x7f8b7806a2d0 msgr2=0x7f8b780d3580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:13.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.764+0000 7f8b7759e640 1 -- 172.21.15.27:0/885101624 shutdown_connections 2023-12-19T18:17:13.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.764+0000 7f8b7759e640 1 -- 172.21.15.27:0/885101624 wait complete. 2023-12-19T18:17:13.787 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:17:13.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.994+0000 7f72ff59e640 1 Processor -- start 2023-12-19T18:17:13.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.994+0000 7f72ff59e640 1 -- start start 2023-12-19T18:17:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.995+0000 7f72ff59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f730015cf80 0x7f73000d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.995+0000 7f72ff59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f73000b8a60 0x7f73000b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.996+0000 7f72ff59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73000d0e10 con 0x7f73000b85e0 2023-12-19T18:17:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.996+0000 7f72ff59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73000d1c70 con 0x7f73000b8a60 2023-12-19T18:17:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.996+0000 7f72ff59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730015d360 con 0x7f730015cf80 2023-12-19T18:17:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.996+0000 7f72fe59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f730015cf80 0x7f73000d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.996+0000 7f72fed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f73000b8a60 0x7f73000b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.996+0000 7f72fe59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f730015cf80 0x7f73000d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:54494/0 (socket says 172.21.15.27:54494) 2023-12-19T18:17:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.996+0000 7f72fe59c640 1 -- 172.21.15.27:0/46645340 learned_addr learned my addr 172.21.15.27:0/46645340 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.996+0000 7f72fed9d640 1 --2- 172.21.15.27:0/46645340 >> v2:172.21.15.27:3300/0 conn(0x7f73000b8a60 0x7f73000b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34000/0 (socket says 172.21.15.27:34000) 2023-12-19T18:17:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.997+0000 7f72fed9d640 1 -- 172.21.15.27:0/46645340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f730015cf80 msgr2=0x7f73000d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.997+0000 7f72fed9d640 1 --2- 172.21.15.27:0/46645340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f730015cf80 0x7f73000d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.997+0000 7f72fed9d640 1 -- 172.21.15.27:0/46645340 >> v1:172.21.15.192:6789/0 conn(0x7f73000b85e0 legacy=0x7f73000cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.997+0000 7f72fed9d640 1 -- 172.21.15.27:0/46645340 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73000d1060 con 0x7f73000b8a60 2023-12-19T18:17:13.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.998+0000 7f72fed9d640 1 --2- 172.21.15.27:0/46645340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73000b8a60 0x7f73000b5d60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f72f80097a0 tx=0x7f72f802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35f6412e1c8a1bf2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:13.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.998+0000 7f72fd59a640 1 -- 172.21.15.27:0/46645340 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8030610 con 0x7f73000b8a60 2023-12-19T18:17:13.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.998+0000 7f72fd59a640 1 -- 172.21.15.27:0/46645340 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f8030770 con 0x7f73000b8a60 2023-12-19T18:17:13.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.998+0000 7f72fd59a640 1 -- 172.21.15.27:0/46645340 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f8030b00 con 0x7f73000b8a60 2023-12-19T18:17:13.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.999+0000 7f72ff59e640 1 -- 172.21.15.27:0/46645340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73000b8a60 msgr2=0x7f73000b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:13.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.999+0000 7f72ff59e640 1 --2- 172.21.15.27:0/46645340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73000b8a60 0x7f73000b5d60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f72f80097a0 tx=0x7f72f802b960 comp rx=0 tx=0).stop 2023-12-19T18:17:14.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.999+0000 7f72ff59e640 1 -- 172.21.15.27:0/46645340 shutdown_connections 2023-12-19T18:17:14.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.999+0000 7f72ff59e640 1 --2- 172.21.15.27:0/46645340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73000b8a60 0x7f73000b5d60 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.999+0000 7f72ff59e640 1 --2- 172.21.15.27:0/46645340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f730015cf80 0x7f73000d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.999+0000 7f72ff59e640 1 -- 172.21.15.27:0/46645340 >> 172.21.15.27:0/46645340 conn(0x7f7300072e20 msgr2=0x7f7300073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:14.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.999+0000 7f72ff59e640 1 -- 172.21.15.27:0/46645340 shutdown_connections 2023-12-19T18:17:14.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:13.999+0000 7f72ff59e640 1 -- 172.21.15.27:0/46645340 wait complete. 2023-12-19T18:17:14.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.001+0000 7f72ff59e640 1 Processor -- start 2023-12-19T18:17:14.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.001+0000 7f72ff59e640 1 -- start start 2023-12-19T18:17:14.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.002+0000 7f72ff59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73000b85e0 0x7f73001f59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.002+0000 7f72ff59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f730015cf80 0x7f73001f5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.002+0000 7f72ff59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73000c8b40 0x7f73000c8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.002+0000 7f72ff59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73000cc9b0 con 0x7f730015cf80 2023-12-19T18:17:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.002+0000 7f72ff59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730015dba0 con 0x7f73000c8b40 2023-12-19T18:17:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.002+0000 7f72ff59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730015de80 con 0x7f73000b85e0 2023-12-19T18:17:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.002+0000 7f72fdd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f730015cf80 0x7f73001f5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.003+0000 7f72fdd9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f730015cf80 0x7f73001f5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34002/0 (socket says 172.21.15.27:34002) 2023-12-19T18:17:14.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.003+0000 7f72fdd9b640 1 -- 172.21.15.27:0/2870600854 learned_addr learned my addr 172.21.15.27:0/2870600854 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.003+0000 7f72fed9d640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73000c8b40 0x7f73000c8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.003+0000 7f72fdd9b640 1 -- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73000b85e0 msgr2=0x7f73001f59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.003+0000 7f72fdd9b640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73000b85e0 0x7f73001f59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.003+0000 7f72fdd9b640 1 -- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73000c8b40 msgr2=0x7f73000c8ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.003+0000 7f72fdd9b640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73000c8b40 0x7f73000c8ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.003+0000 7f72fdd9b640 1 -- 172.21.15.27:0/2870600854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f8009400 con 0x7f730015cf80 2023-12-19T18:17:14.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.003+0000 7f72fed9d640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73000c8b40 0x7f73000c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:14.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.003+0000 7f72fdd9b640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f730015cf80 0x7f73001f5f30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f72f000c870 tx=0x7f72f000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.004+0000 7f72f6ffd640 1 -- 172.21.15.27:0/2870600854 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f0003b70 con 0x7f730015cf80 2023-12-19T18:17:14.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.004+0000 7f72f6ffd640 1 -- 172.21.15.27:0/2870600854 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72f0003cf0 con 0x7f730015cf80 2023-12-19T18:17:14.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.004+0000 7f72f6ffd640 1 -- 172.21.15.27:0/2870600854 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72f000aa90 con 0x7f730015cf80 2023-12-19T18:17:14.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.004+0000 7f72ff59e640 1 -- 172.21.15.27:0/2870600854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730015e160 con 0x7f730015cf80 2023-12-19T18:17:14.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.004+0000 7f72ff59e640 1 -- 172.21.15.27:0/2870600854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73000c9530 con 0x7f730015cf80 2023-12-19T18:17:14.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.005+0000 7f72ff59e640 1 -- 172.21.15.27:0/2870600854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730015e380 con 0x7f730015cf80 2023-12-19T18:17:14.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.006+0000 7f72f6ffd640 1 -- 172.21.15.27:0/2870600854 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f72f000abf0 con 0x7f730015cf80 2023-12-19T18:17:14.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.007+0000 7f72f6ffd640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f72e006a460 0x7f72e006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.007+0000 7f72f6ffd640 1 -- 172.21.15.27:0/2870600854 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f72f00887e0 con 0x7f730015cf80 2023-12-19T18:17:14.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.008+0000 7f72fe59c640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f72e006a460 0x7f72e006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.009+0000 7f72fe59c640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f72e006a460 0x7f72e006c920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f72e8006f40 tx=0x7f72e8006e90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.012+0000 7f72f6ffd640 1 -- 172.21.15.27:0/2870600854 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f72f0053d90 con 0x7f730015cf80 2023-12-19T18:17:14.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.247+0000 7f72ff59e640 1 -- 172.21.15.27:0/2870600854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f730004e840 con 0x7f730015cf80 2023-12-19T18:17:14.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.264+0000 7f72f6ffd640 1 -- 172.21.15.27:0/2870600854 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v19) v1 ==== 172+0+0 (secure 0 0 0) 0x7f72f0003e60 con 0x7f730015cf80 2023-12-19T18:17:14.265 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 6 and data pool 7 2023-12-19T18:17:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.269+0000 7f72ff59e640 1 -- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f72e006a460 msgr2=0x7f72e006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.269+0000 7f72ff59e640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f72e006a460 0x7f72e006c920 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f72e8006f40 tx=0x7f72e8006e90 comp rx=0 tx=0).stop 2023-12-19T18:17:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.270+0000 7f72ff59e640 1 -- 172.21.15.27:0/2870600854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f730015cf80 msgr2=0x7f73001f5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.270+0000 7f72ff59e640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f730015cf80 0x7f73001f5f30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f72f000c870 tx=0x7f72f000cd30 comp rx=0 tx=0).stop 2023-12-19T18:17:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.272+0000 7f72ff59e640 1 -- 172.21.15.27:0/2870600854 shutdown_connections 2023-12-19T18:17:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.272+0000 7f72ff59e640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f72e006a460 0x7f72e006c920 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.272+0000 7f72ff59e640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73000c8b40 0x7f73000c8ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.272+0000 7f72ff59e640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f730015cf80 0x7f73001f5f30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.272+0000 7f72ff59e640 1 --2- 172.21.15.27:0/2870600854 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f73000b85e0 0x7f73001f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.272+0000 7f72ff59e640 1 -- 172.21.15.27:0/2870600854 >> 172.21.15.27:0/2870600854 conn(0x7f7300072e20 msgr2=0x7f73000d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:14.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.273+0000 7f72ff59e640 1 -- 172.21.15.27:0/2870600854 shutdown_connections 2023-12-19T18:17:14.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.274+0000 7f72ff59e640 1 -- 172.21.15.27:0/2870600854 wait complete. 2023-12-19T18:17:14.299 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.491+0000 7fdd21817640 1 Processor -- start 2023-12-19T18:17:14.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.491+0000 7fdd21817640 1 -- start start 2023-12-19T18:17:14.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.493+0000 7fdd21817640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd1c164a80 0x7fdd1c0ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.493+0000 7fdd21817640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdd1c0c9190 0x7fdd1c0cb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.493+0000 7fdd21817640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd1c0caba0 con 0x7fdd1c0cba90 2023-12-19T18:17:14.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.493+0000 7fdd21817640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd1c0cbf00 con 0x7fdd1c0c9190 2023-12-19T18:17:14.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.493+0000 7fdd21817640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c164e60 con 0x7fdd1c164a80 2023-12-19T18:17:14.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.493+0000 7fdd1b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdd1c0cba90 0x7fdd1c0cc710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41776/0 (socket says 172.21.15.27:41776) 2023-12-19T18:17:14.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.493+0000 7fdd1b7fe640 1 -- 172.21.15.27:0/1833750689 learned_addr learned my addr 172.21.15.27:0/1833750689 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:14.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 805052561 0 0) 0x7fdd1c0caba0 con 0x7fdd1c0cba90 2023-12-19T18:17:14.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd1a7fc640 1 --2- 172.21.15.27:0/1833750689 >> v2:172.21.15.192:3300/0 conn(0x7fdd1c0c9190 0x7fdd1c0cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd1affd640 1 --2- 172.21.15.27:0/1833750689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd1c164a80 0x7fdd1c0ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd04003520 con 0x7fdd1c0cba90 2023-12-19T18:17:14.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2650690325 0 0) 0x7fdd04003520 con 0x7fdd1c0cba90 2023-12-19T18:17:14.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd1c164a80 msgr2=0x7fdd1c0ce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 --2- 172.21.15.27:0/1833750689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd1c164a80 0x7fdd1c0ce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 >> v2:172.21.15.192:3300/0 conn(0x7fdd1c0c9190 msgr2=0x7fdd1c0cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 --2- 172.21.15.27:0/1833750689 >> v2:172.21.15.192:3300/0 conn(0x7fdd1c0c9190 0x7fdd1c0cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c06c2d0 con 0x7fdd1c0cba90 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fdd140019f0 con 0x7fdd1c0cba90 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 >> v1:172.21.15.27:6789/0 conn(0x7fdd1c0cba90 legacy=0x7fdd1c0cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 --2- 172.21.15.27:0/1833750689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd04003970 0x7fdd04003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 --2- 172.21.15.27:0/1833750689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd040047c0 0x7fdd04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 --2- 172.21.15.27:0/1833750689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd1c164a80 0x7fdd04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c0cbf00 con 0x7fdd04003970 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c164e60 con 0x7fdd040047c0 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.494+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0400ba30 con 0x7fdd1c164a80 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.495+0000 7fdd1affd640 1 --2- 172.21.15.27:0/1833750689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd04003970 0x7fdd04003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.495+0000 7fdd1a7fc640 1 --2- 172.21.15.27:0/1833750689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd040047c0 0x7fdd04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.495+0000 7fdd1affd640 1 -- 172.21.15.27:0/1833750689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd1c164a80 msgr2=0x7fdd04005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:14.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.495+0000 7fdd1affd640 1 --2- 172.21.15.27:0/1833750689 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd1c164a80 0x7fdd04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.495+0000 7fdd1affd640 1 -- 172.21.15.27:0/1833750689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd040047c0 msgr2=0x7fdd04004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:14.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.495+0000 7fdd1affd640 1 --2- 172.21.15.27:0/1833750689 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd040047c0 0x7fdd04004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.495+0000 7fdd1affd640 1 -- 172.21.15.27:0/1833750689 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd04003520 con 0x7fdd04003970 2023-12-19T18:17:14.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.496+0000 7fdd1affd640 1 --2- 172.21.15.27:0/1833750689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd04003970 0x7fdd04003d90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00ba00 tx=0x7fdd040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dc0ee0773945d7e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.496+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c02d040 con 0x7fdd04003970 2023-12-19T18:17:14.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.496+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd0c033610 con 0x7fdd04003970 2023-12-19T18:17:14.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.496+0000 7fdd19ffb640 1 -- 172.21.15.27:0/1833750689 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c0339a0 con 0x7fdd04003970 2023-12-19T18:17:14.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.497+0000 7fdd21817640 1 -- 172.21.15.27:0/1833750689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd04003970 msgr2=0x7fdd04003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.497+0000 7fdd21817640 1 --2- 172.21.15.27:0/1833750689 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd04003970 0x7fdd04003d90 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c00ba00 tx=0x7fdd040057f0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.497+0000 7fdd1b7fe640 1 -- 172.21.15.27:0/1833750689 reap_dead start 2023-12-19T18:17:14.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.497+0000 7fdd21817640 1 -- 172.21.15.27:0/1833750689 shutdown_connections 2023-12-19T18:17:14.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.497+0000 7fdd21817640 1 -- 172.21.15.27:0/1833750689 >> 172.21.15.27:0/1833750689 conn(0x7fdd1c06a330 msgr2=0x7fdd1c0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:14.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.497+0000 7fdd21817640 1 -- 172.21.15.27:0/1833750689 shutdown_connections 2023-12-19T18:17:14.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.497+0000 7fdd21817640 1 -- 172.21.15.27:0/1833750689 wait complete. 2023-12-19T18:17:14.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.499+0000 7fdd21817640 1 Processor -- start 2023-12-19T18:17:14.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.499+0000 7fdd21817640 1 -- start start 2023-12-19T18:17:14.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.500+0000 7fdd21817640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd04003970 0x7fdd1c15ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.500+0000 7fdd21817640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd1c0c3420 0x7fdd1c0b2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.500+0000 7fdd21817640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c0ae600 0x7fdd1c0aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.500+0000 7fdd21817640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c0aeff0 con 0x7fdd1c0ae600 2023-12-19T18:17:14.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.500+0000 7fdd21817640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c0b3270 con 0x7fdd1c0c3420 2023-12-19T18:17:14.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.500+0000 7fdd21817640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c0b29d0 con 0x7fdd04003970 2023-12-19T18:17:14.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.500+0000 7fdd1b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c0ae600 0x7fdd1c0aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.501+0000 7fdd1b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c0ae600 0x7fdd1c0aeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34024/0 (socket says 172.21.15.27:34024) 2023-12-19T18:17:14.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.501+0000 7fdd1b7fe640 1 -- 172.21.15.27:0/2697069087 learned_addr learned my addr 172.21.15.27:0/2697069087 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.501+0000 7fdd1a7fc640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd1c0c3420 0x7fdd1c0b2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.501+0000 7fdd1affd640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd04003970 0x7fdd1c15ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.501+0000 7fdd1b7fe640 1 -- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd04003970 msgr2=0x7fdd1c15ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.501+0000 7fdd1b7fe640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd04003970 0x7fdd1c15ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.501+0000 7fdd1b7fe640 1 -- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd1c0c3420 msgr2=0x7fdd1c0b2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.501+0000 7fdd1b7fe640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd1c0c3420 0x7fdd1c0b2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.501+0000 7fdd1b7fe640 1 -- 172.21.15.27:0/2697069087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd1c0b2cb0 con 0x7fdd1c0ae600 2023-12-19T18:17:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.501+0000 7fdd1a7fc640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd1c0c3420 0x7fdd1c0b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.501+0000 7fdd1b7fe640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c0ae600 0x7fdd1c0aeab0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdd140013d0 tx=0x7fdd1400cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.501+0000 7fdd03fff640 1 -- 172.21.15.27:0/2697069087 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1400a610 con 0x7fdd1c0ae600 2023-12-19T18:17:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.502+0000 7fdd03fff640 1 -- 172.21.15.27:0/2697069087 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd1400a770 con 0x7fdd1c0ae600 2023-12-19T18:17:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.502+0000 7fdd03fff640 1 -- 172.21.15.27:0/2697069087 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1400ab20 con 0x7fdd1c0ae600 2023-12-19T18:17:14.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.502+0000 7fdd21817640 1 -- 172.21.15.27:0/2697069087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c0b2f90 con 0x7fdd1c0ae600 2023-12-19T18:17:14.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.502+0000 7fdd21817640 1 -- 172.21.15.27:0/2697069087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c0c5c10 con 0x7fdd1c0ae600 2023-12-19T18:17:14.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.503+0000 7fdd21817640 1 -- 172.21.15.27:0/2697069087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1c0c4010 con 0x7fdd1c0ae600 2023-12-19T18:17:14.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.504+0000 7fdd03fff640 1 -- 172.21.15.27:0/2697069087 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdd1400ac80 con 0x7fdd1c0ae600 2023-12-19T18:17:14.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.505+0000 7fdd03fff640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd0806a430 0x7fdd0806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.505+0000 7fdd03fff640 1 -- 172.21.15.27:0/2697069087 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd14088720 con 0x7fdd1c0ae600 2023-12-19T18:17:14.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.506+0000 7fdd1affd640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd0806a430 0x7fdd0806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.507+0000 7fdd1affd640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd0806a430 0x7fdd0806c8f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdd040057a0 tx=0x7fdd0c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.509+0000 7fdd03fff640 1 -- 172.21.15.27:0/2697069087 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdd14053d00 con 0x7fdd1c0ae600 2023-12-19T18:17:14.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.704+0000 7fdd21817640 1 -- 172.21.15.27:0/2697069087 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd1c0ae180 con 0x7fdd1c0ae600 2023-12-19T18:17:14.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.708+0000 7fdd03fff640 1 -- 172.21.15.27:0/2697069087 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16054 (secure 0 0 0) 0x7fdd14057320 con 0x7fdd1c0ae600 2023-12-19T18:17:14.708 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:14.708 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":49,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:17:14.249844+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:12.527441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:13.291755+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:14.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.710+0000 7fdd21817640 1 -- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd0806a430 msgr2=0x7fdd0806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.710+0000 7fdd21817640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd0806a430 0x7fdd0806c8f0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fdd040057a0 tx=0x7fdd0c030000 comp rx=0 tx=0).stop 2023-12-19T18:17:14.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.710+0000 7fdd21817640 1 -- 172.21.15.27:0/2697069087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c0ae600 msgr2=0x7fdd1c0aeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.710+0000 7fdd21817640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c0ae600 0x7fdd1c0aeab0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fdd140013d0 tx=0x7fdd1400cf00 comp rx=0 tx=0).stop 2023-12-19T18:17:14.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.710+0000 7fdd21817640 1 -- 172.21.15.27:0/2697069087 shutdown_connections 2023-12-19T18:17:14.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.710+0000 7fdd21817640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd0806a430 0x7fdd0806c8f0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.710+0000 7fdd21817640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd1c0ae600 0x7fdd1c0aeab0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.710+0000 7fdd21817640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd1c0c3420 0x7fdd1c0b2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.710+0000 7fdd21817640 1 --2- 172.21.15.27:0/2697069087 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd04003970 0x7fdd1c15ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.711+0000 7fdd21817640 1 -- 172.21.15.27:0/2697069087 >> 172.21.15.27:0/2697069087 conn(0x7fdd1c06a330 msgr2=0x7fdd1c0af840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.711+0000 7fdd21817640 1 -- 172.21.15.27:0/2697069087 shutdown_connections 2023-12-19T18:17:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.711+0000 7fdd21817640 1 -- 172.21.15.27:0/2697069087 wait complete. 2023-12-19T18:17:14.731 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:14.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.961+0000 7f95f6e2c640 1 Processor -- start 2023-12-19T18:17:14.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.961+0000 7f95f6e2c640 1 -- start start 2023-12-19T18:17:14.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.963+0000 7f95f6e2c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95f0164a80 0x7f95f00ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.963+0000 7f95f6e2c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f95f00cbc60 0x7f95f00cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.963+0000 7f95f6e2c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95f00caba0 con 0x7f95f00c9190 2023-12-19T18:17:14.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.963+0000 7f95f6e2c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f0164e60 con 0x7f95f0164a80 2023-12-19T18:17:14.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.963+0000 7f95f6e2c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f006c2d0 con 0x7f95f00cbc60 2023-12-19T18:17:14.964 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:17:14.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.963+0000 7f95f4bc8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95f0164a80 0x7f95f00ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.964+0000 7f95f4bc8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95f0164a80 0x7f95f00ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34030/0 (socket says 172.21.15.27:34030) 2023-12-19T18:17:14.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.964+0000 7f95f4bc8640 1 -- 172.21.15.27:0/3376184332 learned_addr learned my addr 172.21.15.27:0/3376184332 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:14.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.964+0000 7f95f4bc8640 1 -- 172.21.15.27:0/3376184332 >> v2:172.21.15.192:3300/0 conn(0x7f95f00cbc60 msgr2=0x7f95f00cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:14.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.964+0000 7f95f53c9640 1 --2- 172.21.15.27:0/3376184332 >> v2:172.21.15.192:3300/0 conn(0x7f95f00cbc60 0x7f95f00cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.964+0000 7f95f4bc8640 1 --2- 172.21.15.27:0/3376184332 >> v2:172.21.15.192:3300/0 conn(0x7f95f00cbc60 0x7f95f00cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.964+0000 7f95f4bc8640 1 -- 172.21.15.27:0/3376184332 >> v1:172.21.15.192:6789/0 conn(0x7f95f00c9190 legacy=0x7f95f00cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:14.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.964+0000 7f95f4bc8640 1 -- 172.21.15.27:0/3376184332 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f00cae40 con 0x7f95f0164a80 2023-12-19T18:17:14.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.964+0000 7f95f53c9640 1 --2- 172.21.15.27:0/3376184332 >> v2:172.21.15.192:3300/0 conn(0x7f95f00cbc60 0x7f95f00cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:14.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.965+0000 7f95f4bc8640 1 --2- 172.21.15.27:0/3376184332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f0164a80 0x7f95f00ce520 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f95f00ad9d0 tx=0x7f95e402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bf8dafe2e0049e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.969 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:17:14.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.966+0000 7f95ef7fe640 1 -- 172.21.15.27:0/3376184332 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e40317e0 con 0x7f95f0164a80 2023-12-19T18:17:14.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.966+0000 7f95ef7fe640 1 -- 172.21.15.27:0/3376184332 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e4031940 con 0x7f95f0164a80 2023-12-19T18:17:14.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.966+0000 7f95ef7fe640 1 -- 172.21.15.27:0/3376184332 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e4031cd0 con 0x7f95f0164a80 2023-12-19T18:17:14.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.966+0000 7f95f6e2c640 1 -- 172.21.15.27:0/3376184332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f0164a80 msgr2=0x7f95f00ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.966+0000 7f95f6e2c640 1 --2- 172.21.15.27:0/3376184332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f0164a80 0x7f95f00ce520 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f95f00ad9d0 tx=0x7f95e402b880 comp rx=0 tx=0).stop 2023-12-19T18:17:14.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.966+0000 7f95f6e2c640 1 -- 172.21.15.27:0/3376184332 shutdown_connections 2023-12-19T18:17:14.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.966+0000 7f95f6e2c640 1 --2- 172.21.15.27:0/3376184332 >> v2:172.21.15.192:3300/0 conn(0x7f95f00cbc60 0x7f95f00cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.966+0000 7f95f6e2c640 1 --2- 172.21.15.27:0/3376184332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f0164a80 0x7f95f00ce520 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.966+0000 7f95f6e2c640 1 -- 172.21.15.27:0/3376184332 >> 172.21.15.27:0/3376184332 conn(0x7f95f006a330 msgr2=0x7f95f00b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:14.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.966+0000 7f95f6e2c640 1 -- 172.21.15.27:0/3376184332 shutdown_connections 2023-12-19T18:17:14.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.966+0000 7f95f6e2c640 1 -- 172.21.15.27:0/3376184332 wait complete. 2023-12-19T18:17:14.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.968+0000 7f95f6e2c640 1 Processor -- start 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.968+0000 7f95f6e2c640 1 -- start start 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.969+0000 7f95f6e2c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f00c9190 0x7f95f00b1780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f6e2c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95f00cbc60 0x7f95f01f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f6e2c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95f0164a80 0x7f95f01fa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f6e2c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f00b4230 con 0x7f95f00c9190 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f6e2c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f00b4510 con 0x7f95f0164a80 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f6e2c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f00b47f0 con 0x7f95f00cbc60 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f4bc8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f00c9190 0x7f95f00b1780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f4bc8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f00c9190 0x7f95f00b1780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34038/0 (socket says 172.21.15.27:34038) 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f4bc8640 1 -- 172.21.15.27:0/1463393660 learned_addr learned my addr 172.21.15.27:0/1463393660 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95effff640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95f00cbc60 0x7f95f01f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f53c9640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95f0164a80 0x7f95f01fa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f4bc8640 1 -- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95f00cbc60 msgr2=0x7f95f01f9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f4bc8640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95f00cbc60 0x7f95f01f9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f4bc8640 1 -- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95f0164a80 msgr2=0x7f95f01fa4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f4bc8640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95f0164a80 0x7f95f01fa4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.970+0000 7f95f4bc8640 1 -- 172.21.15.27:0/1463393660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e40094d0 con 0x7f95f00c9190 2023-12-19T18:17:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.971+0000 7f95f4bc8640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f00c9190 0x7f95f00b1780 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f95e4000c00 tx=0x7f95e40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.971+0000 7f95ed7fa640 1 -- 172.21.15.27:0/1463393660 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e4032980 con 0x7f95f00c9190 2023-12-19T18:17:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.971+0000 7f95ed7fa640 1 -- 172.21.15.27:0/1463393660 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e4032ae0 con 0x7f95f00c9190 2023-12-19T18:17:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.971+0000 7f95f6e2c640 1 -- 172.21.15.27:0/1463393660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f00c6d00 con 0x7f95f00c9190 2023-12-19T18:17:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.971+0000 7f95ed7fa640 1 -- 172.21.15.27:0/1463393660 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e4032e90 con 0x7f95f00c9190 2023-12-19T18:17:14.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.972+0000 7f95f6e2c640 1 -- 172.21.15.27:0/1463393660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f0151260 con 0x7f95f00c9190 2023-12-19T18:17:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.973+0000 7f95f6e2c640 1 -- 172.21.15.27:0/1463393660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f0068e90 con 0x7f95f00c9190 2023-12-19T18:17:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.974+0000 7f95ed7fa640 1 -- 172.21.15.27:0/1463393660 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f95e404d3f0 con 0x7f95f00c9190 2023-12-19T18:17:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.975+0000 7f95ed7fa640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f95d406a430 0x7f95d406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.975+0000 7f95ed7fa640 1 -- 172.21.15.27:0/1463393660 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f95e40aa8c0 con 0x7f95f00c9190 2023-12-19T18:17:14.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.975+0000 7f95effff640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f95d406a430 0x7f95d406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:14.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.977+0000 7f95effff640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f95d406a430 0x7f95d406c8f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f95dc006fd0 tx=0x7f95dc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:14.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:14.979+0000 7f95ed7fa640 1 -- 172.21.15.27:0/1463393660 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f95e4075f60 con 0x7f95f00c9190 2023-12-19T18:17:15.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.181+0000 7f95f6e2c640 1 -- 172.21.15.27:0/1463393660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95f00b4ac0 con 0x7f95f00c9190 2023-12-19T18:17:15.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.183+0000 7f95ed7fa640 1 -- 172.21.15.27:0/1463393660 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) v1 ==== 74+0+16054 (secure 0 0 0) 0x7f95e4079580 con 0x7f95f00c9190 2023-12-19T18:17:15.183 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:15.184 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":49,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:17:14.249844+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:12.527441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:13.291755+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.186+0000 7f95f6e2c640 1 -- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f95d406a430 msgr2=0x7f95d406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.186+0000 7f95f6e2c640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f95d406a430 0x7f95d406c8f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f95dc006fd0 tx=0x7f95dc008040 comp rx=0 tx=0).stop 2023-12-19T18:17:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.186+0000 7f95f6e2c640 1 -- 172.21.15.27:0/1463393660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f00c9190 msgr2=0x7f95f00b1780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.186+0000 7f95f6e2c640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f00c9190 0x7f95f00b1780 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f95e4000c00 tx=0x7f95e40046c0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.186+0000 7f95f6e2c640 1 -- 172.21.15.27:0/1463393660 shutdown_connections 2023-12-19T18:17:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.186+0000 7f95f6e2c640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f95d406a430 0x7f95d406c8f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.186+0000 7f95f6e2c640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95f0164a80 0x7f95f01fa4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.186+0000 7f95f6e2c640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95f00cbc60 0x7f95f01f9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.186+0000 7f95f6e2c640 1 --2- 172.21.15.27:0/1463393660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95f00c9190 0x7f95f00b1780 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.186+0000 7f95f6e2c640 1 -- 172.21.15.27:0/1463393660 >> 172.21.15.27:0/1463393660 conn(0x7f95f006a330 msgr2=0x7f95f00ae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.186+0000 7f95f6e2c640 1 -- 172.21.15.27:0/1463393660 shutdown_connections 2023-12-19T18:17:15.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.186+0000 7f95f6e2c640 1 -- 172.21.15.27:0/1463393660 wait complete. 2023-12-19T18:17:15.207 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:17:15.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.435+0000 7f7edffff640 1 Processor -- start 2023-12-19T18:17:15.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.436+0000 7f7edffff640 1 -- start start 2023-12-19T18:17:15.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.438+0000 7f7edffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ee00cbe20 0x7f7ee00c4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.438+0000 7f7edffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7ee00c4e70 0x7f7ee00ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.438+0000 7f7edffff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7ee00adc50 0x7f7ee00cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.438+0000 7f7edffff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee00c52d0 con 0x7f7ee00c4e70 2023-12-19T18:17:15.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.438+0000 7f7edffff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ee015d780 con 0x7f7ee00adc50 2023-12-19T18:17:15.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.438+0000 7f7edffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee00cc200 con 0x7f7ee00cbe20 2023-12-19T18:17:15.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.438+0000 7f7ed7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7ee00c4e70 0x7f7ee00ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.438+0000 7f7ed7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7ee00c4e70 0x7f7ee00ad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34046/0 (socket says 172.21.15.27:34046) 2023-12-19T18:17:15.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.439+0000 7f7ed7fff640 1 -- 172.21.15.27:0/167762534 learned_addr learned my addr 172.21.15.27:0/167762534 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:15.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.439+0000 7f7edeffd640 1 --2- 172.21.15.27:0/167762534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ee00cbe20 0x7f7ee00c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.439+0000 7f7edf7fe640 1 --2- 172.21.15.27:0/167762534 >> v2:172.21.15.192:3300/0 conn(0x7f7ee00adc50 0x7f7ee00cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.439+0000 7f7ed7fff640 1 -- 172.21.15.27:0/167762534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ee00cbe20 msgr2=0x7f7ee00c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:15.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.439+0000 7f7ed7fff640 1 --2- 172.21.15.27:0/167762534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ee00cbe20 0x7f7ee00c4930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.439+0000 7f7ed7fff640 1 -- 172.21.15.27:0/167762534 >> v2:172.21.15.192:3300/0 conn(0x7f7ee00adc50 msgr2=0x7f7ee00cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:15.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.439+0000 7f7ed7fff640 1 --2- 172.21.15.27:0/167762534 >> v2:172.21.15.192:3300/0 conn(0x7f7ee00adc50 0x7f7ee00cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.439+0000 7f7ed7fff640 1 -- 172.21.15.27:0/167762534 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee015d9d0 con 0x7f7ee00c4e70 2023-12-19T18:17:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.440+0000 7f7ed7fff640 1 --2- 172.21.15.27:0/167762534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ee00c4e70 0x7f7ee00ad710 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00c8f40 tx=0x7f7ec802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f983e9f0a0f2143 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.441+0000 7f7ede7fc640 1 -- 172.21.15.27:0/167762534 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec80317e0 con 0x7f7ee00c4e70 2023-12-19T18:17:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.441+0000 7f7ede7fc640 1 -- 172.21.15.27:0/167762534 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec8031940 con 0x7f7ee00c4e70 2023-12-19T18:17:15.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.441+0000 7f7ede7fc640 1 -- 172.21.15.27:0/167762534 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec8031cd0 con 0x7f7ee00c4e70 2023-12-19T18:17:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.441+0000 7f7edffff640 1 -- 172.21.15.27:0/167762534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ee00c4e70 msgr2=0x7f7ee00ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.441+0000 7f7edffff640 1 --2- 172.21.15.27:0/167762534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ee00c4e70 0x7f7ee00ad710 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7ee00c8f40 tx=0x7f7ec802b880 comp rx=0 tx=0).stop 2023-12-19T18:17:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.441+0000 7f7edffff640 1 -- 172.21.15.27:0/167762534 shutdown_connections 2023-12-19T18:17:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.441+0000 7f7edffff640 1 --2- 172.21.15.27:0/167762534 >> v2:172.21.15.192:3300/0 conn(0x7f7ee00adc50 0x7f7ee00cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.441+0000 7f7edffff640 1 --2- 172.21.15.27:0/167762534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ee00c4e70 0x7f7ee00ad710 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.441+0000 7f7edffff640 1 --2- 172.21.15.27:0/167762534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ee00cbe20 0x7f7ee00c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.441+0000 7f7edffff640 1 -- 172.21.15.27:0/167762534 >> 172.21.15.27:0/167762534 conn(0x7f7ee00b05b0 msgr2=0x7f7ee00b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.441+0000 7f7edffff640 1 -- 172.21.15.27:0/167762534 shutdown_connections 2023-12-19T18:17:15.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.442+0000 7f7edffff640 1 -- 172.21.15.27:0/167762534 wait complete. 2023-12-19T18:17:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.443+0000 7f7edffff640 1 Processor -- start 2023-12-19T18:17:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.444+0000 7f7edffff640 1 -- start start 2023-12-19T18:17:15.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ee00adc50 0x7f7ee015a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ee00c4e70 0x7f7ee015bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ee00cbe20 0x7f7ee015c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edffff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee01f5cf0 con 0x7f7ee00adc50 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edffff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee015c6b0 con 0x7f7ee00c4e70 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ee0155fb0 con 0x7f7ee00cbe20 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ee00adc50 0x7f7ee015a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ee00adc50 0x7f7ee015a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34058/0 (socket says 172.21.15.27:34058) 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edeffd640 1 -- 172.21.15.27:0/46585977 learned_addr learned my addr 172.21.15.27:0/46585977 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7ed7fff640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ee00c4e70 0x7f7ee015bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edeffd640 1 -- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ee00cbe20 msgr2=0x7f7ee015c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edf7fe640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ee00cbe20 0x7f7ee015c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edeffd640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ee00cbe20 0x7f7ee015c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edeffd640 1 -- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ee00c4e70 msgr2=0x7f7ee015bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edeffd640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ee00c4e70 0x7f7ee015bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:15.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7edeffd640 1 -- 172.21.15.27:0/46585977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec80094d0 con 0x7f7ee00adc50 2023-12-19T18:17:15.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.445+0000 7f7ed7fff640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ee00c4e70 0x7f7ee015bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:15.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.446+0000 7f7edeffd640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ee00adc50 0x7f7ee015a030 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000d5f0 tx=0x7f7ed000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:15.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.446+0000 7f7ed77fe640 1 -- 172.21.15.27:0/46585977 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00125f0 con 0x7f7ee00adc50 2023-12-19T18:17:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.446+0000 7f7ed77fe640 1 -- 172.21.15.27:0/46585977 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed0012750 con 0x7f7ee00adc50 2023-12-19T18:17:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.446+0000 7f7ed77fe640 1 -- 172.21.15.27:0/46585977 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0012b00 con 0x7f7ee00adc50 2023-12-19T18:17:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.446+0000 7f7edffff640 1 -- 172.21.15.27:0/46585977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ee015c940 con 0x7f7ee00adc50 2023-12-19T18:17:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.446+0000 7f7edffff640 1 -- 172.21.15.27:0/46585977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ee015cc20 con 0x7f7ee00adc50 2023-12-19T18:17:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.448+0000 7f7edffff640 1 -- 172.21.15.27:0/46585977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ee004e840 con 0x7f7ee00adc50 2023-12-19T18:17:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.448+0000 7f7ed77fe640 1 -- 172.21.15.27:0/46585977 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7ed0012c60 con 0x7f7ee00adc50 2023-12-19T18:17:15.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.449+0000 7f7ed77fe640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7ec006a430 0x7f7ec006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:15.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.450+0000 7f7ed77fe640 1 -- 172.21.15.27:0/46585977 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ed0088da0 con 0x7f7ee00adc50 2023-12-19T18:17:15.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.450+0000 7f7ed7fff640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7ec006a430 0x7f7ec006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:15.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.451+0000 7f7ed7fff640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7ec006a430 0x7f7ec006c8f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8036000 tx=0x7f7ec803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:15.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.454+0000 7f7ed77fe640 1 -- 172.21.15.27:0/46585977 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7ed0054390 con 0x7f7ee00adc50 2023-12-19T18:17:15.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:15.692+0000 7f7edffff640 1 -- 172.21.15.27:0/46585977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7ee00712f0 con 0x7f7ee00adc50 2023-12-19T18:17:16.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.273+0000 7f7ed77fe640 1 -- 172.21.15.27:0/46585977 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v22) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7ed00579b0 con 0x7f7ee00adc50 2023-12-19T18:17:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.277+0000 7f7edffff640 1 -- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7ec006a430 msgr2=0x7f7ec006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.277+0000 7f7edffff640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7ec006a430 0x7f7ec006c8f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8036000 tx=0x7f7ec803d040 comp rx=0 tx=0).stop 2023-12-19T18:17:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.277+0000 7f7edffff640 1 -- 172.21.15.27:0/46585977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ee00adc50 msgr2=0x7f7ee015a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:16.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.277+0000 7f7edffff640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ee00adc50 0x7f7ee015a030 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000d5f0 tx=0x7f7ed000dab0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.277+0000 7f7edffff640 1 -- 172.21.15.27:0/46585977 shutdown_connections 2023-12-19T18:17:16.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.277+0000 7f7edffff640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7ec006a430 0x7f7ec006c8f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.277+0000 7f7edffff640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ee00cbe20 0x7f7ee015c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.277+0000 7f7edffff640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ee00c4e70 0x7f7ee015bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.277+0000 7f7edffff640 1 --2- 172.21.15.27:0/46585977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ee00adc50 0x7f7ee015a030 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.277+0000 7f7edffff640 1 -- 172.21.15.27:0/46585977 >> 172.21.15.27:0/46585977 conn(0x7f7ee00b05b0 msgr2=0x7f7ee006fe90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.277+0000 7f7edffff640 1 -- 172.21.15.27:0/46585977 shutdown_connections 2023-12-19T18:17:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.277+0000 7f7edffff640 1 -- 172.21.15.27:0/46585977 wait complete. 2023-12-19T18:17:16.300 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:17:16.300 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:17:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.526+0000 7f5209f11640 1 Processor -- start 2023-12-19T18:17:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.526+0000 7f5209f11640 1 -- start start 2023-12-19T18:17:16.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.528+0000 7f5209f11640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5204158790 0x7f52040ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:16.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.528+0000 7f5209f11640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52040adc50 0x7f520415b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:16.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.528+0000 7f5209f11640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520415c030 con 0x7f520415b810 2023-12-19T18:17:16.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.528+0000 7f5209f11640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52040c99d0 con 0x7f52040adc50 2023-12-19T18:17:16.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.528+0000 7f5209f11640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52040ae100 con 0x7f5204158790 2023-12-19T18:17:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.529+0000 7f5203fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52040adc50 0x7f520415b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.529+0000 7f5203fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52040adc50 0x7f520415b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34066/0 (socket says 172.21.15.27:34066) 2023-12-19T18:17:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.529+0000 7f5203fff640 1 -- 172.21.15.27:0/3758379920 learned_addr learned my addr 172.21.15.27:0/3758379920 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.529+0000 7f5208f0f640 1 --2- 172.21.15.27:0/3758379920 >> v2:172.21.15.192:3300/0 conn(0x7f5204158790 0x7f52040ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.529+0000 7f5203fff640 1 -- 172.21.15.27:0/3758379920 >> v2:172.21.15.192:3300/0 conn(0x7f5204158790 msgr2=0x7f52040ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.529+0000 7f5203fff640 1 --2- 172.21.15.27:0/3758379920 >> v2:172.21.15.192:3300/0 conn(0x7f5204158790 0x7f52040ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.529+0000 7f5203fff640 1 -- 172.21.15.27:0/3758379920 >> v1:172.21.15.192:6789/0 conn(0x7f520415b810 legacy=0x7f520415e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.529+0000 7f5203fff640 1 -- 172.21.15.27:0/3758379920 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52040c9030 con 0x7f52040adc50 2023-12-19T18:17:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.531+0000 7f5203fff640 1 --2- 172.21.15.27:0/3758379920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52040adc50 0x7f520415b2d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009870 tx=0x7f51f002b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eac469d65ef31e74 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.531+0000 7f52037fe640 1 -- 172.21.15.27:0/3758379920 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0030850 con 0x7f52040adc50 2023-12-19T18:17:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.531+0000 7f52037fe640 1 -- 172.21.15.27:0/3758379920 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f00309b0 con 0x7f52040adc50 2023-12-19T18:17:16.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.531+0000 7f52037fe640 1 -- 172.21.15.27:0/3758379920 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f0030d40 con 0x7f52040adc50 2023-12-19T18:17:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.532+0000 7f5209f11640 1 -- 172.21.15.27:0/3758379920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52040adc50 msgr2=0x7f520415b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.532+0000 7f5209f11640 1 --2- 172.21.15.27:0/3758379920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52040adc50 0x7f520415b2d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009870 tx=0x7f51f002b990 comp rx=0 tx=0).stop 2023-12-19T18:17:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.532+0000 7f5209f11640 1 -- 172.21.15.27:0/3758379920 shutdown_connections 2023-12-19T18:17:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.532+0000 7f5209f11640 1 --2- 172.21.15.27:0/3758379920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52040adc50 0x7f520415b2d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.532+0000 7f5209f11640 1 --2- 172.21.15.27:0/3758379920 >> v2:172.21.15.192:3300/0 conn(0x7f5204158790 0x7f52040ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.532+0000 7f5209f11640 1 -- 172.21.15.27:0/3758379920 >> 172.21.15.27:0/3758379920 conn(0x7f52040b05b0 msgr2=0x7f52040b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.532+0000 7f5209f11640 1 -- 172.21.15.27:0/3758379920 shutdown_connections 2023-12-19T18:17:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.532+0000 7f5209f11640 1 -- 172.21.15.27:0/3758379920 wait complete. 2023-12-19T18:17:16.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.535+0000 7f5209f11640 1 Processor -- start 2023-12-19T18:17:16.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.535+0000 7f5209f11640 1 -- start start 2023-12-19T18:17:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.536+0000 7f5209f11640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52040adc50 0x7f52041f3280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.536+0000 7f5209f11640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5204158790 0x7f520406b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.536+0000 7f5209f11640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f520415b810 0x7f5204158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.536+0000 7f5209f11640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520406b820 con 0x7f52040adc50 2023-12-19T18:17:16.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.536+0000 7f5209f11640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f520406c660 con 0x7f520415b810 2023-12-19T18:17:16.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.536+0000 7f5209f11640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520406ba50 con 0x7f5204158790 2023-12-19T18:17:16.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.536+0000 7f5208f0f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52040adc50 0x7f52041f3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:16.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.536+0000 7f5208f0f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52040adc50 0x7f52041f3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34068/0 (socket says 172.21.15.27:34068) 2023-12-19T18:17:16.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.536+0000 7f5208f0f640 1 -- 172.21.15.27:0/4279291983 learned_addr learned my addr 172.21.15.27:0/4279291983 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:16.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.537+0000 7f5209710640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f520415b810 0x7f5204158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:16.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.537+0000 7f5203fff640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5204158790 0x7f520406b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:16.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.537+0000 7f5208f0f640 1 -- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5204158790 msgr2=0x7f520406b2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:16.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.537+0000 7f5208f0f640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5204158790 0x7f520406b2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.537+0000 7f5208f0f640 1 -- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f520415b810 msgr2=0x7f5204158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:16.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.537+0000 7f5208f0f640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f520415b810 0x7f5204158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:16.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.537+0000 7f5208f0f640 1 -- 172.21.15.27:0/4279291983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f00094d0 con 0x7f52040adc50 2023-12-19T18:17:16.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.537+0000 7f5203fff640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5204158790 0x7f520406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:16.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.538+0000 7f5208f0f640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52040adc50 0x7f52041f3280 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f51f800d800 tx=0x7f51f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:16.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.538+0000 7f52017fa640 1 -- 172.21.15.27:0/4279291983 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f800b6c0 con 0x7f52040adc50 2023-12-19T18:17:16.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.538+0000 7f52017fa640 1 -- 172.21.15.27:0/4279291983 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f800b840 con 0x7f52040adc50 2023-12-19T18:17:16.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.538+0000 7f52017fa640 1 -- 172.21.15.27:0/4279291983 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f800bbf0 con 0x7f52040adc50 2023-12-19T18:17:16.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.538+0000 7f5209f11640 1 -- 172.21.15.27:0/4279291983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520406bd30 con 0x7f52040adc50 2023-12-19T18:17:16.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.538+0000 7f5209f11640 1 -- 172.21.15.27:0/4279291983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520406c2f0 con 0x7f52040adc50 2023-12-19T18:17:16.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.539+0000 7f5209f11640 1 -- 172.21.15.27:0/4279291983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52040c56e0 con 0x7f52040adc50 2023-12-19T18:17:16.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.540+0000 7f52017fa640 1 -- 172.21.15.27:0/4279291983 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f51f800bd50 con 0x7f52040adc50 2023-12-19T18:17:16.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.541+0000 7f52017fa640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f51e806a430 0x7f51e806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:16.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.542+0000 7f52017fa640 1 -- 172.21.15.27:0/4279291983 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51f8088eb0 con 0x7f52040adc50 2023-12-19T18:17:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.543+0000 7f5203fff640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f51e806a430 0x7f51e806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:16.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.544+0000 7f5203fff640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f51e806a430 0x7f51e806c8f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f51f0036000 tx=0x7f51f003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:16.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.546+0000 7f52017fa640 1 -- 172.21.15.27:0/4279291983 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f51f80544a0 con 0x7f52040adc50 2023-12-19T18:17:16.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:16.788+0000 7f5209f11640 1 -- 172.21.15.27:0/4279291983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f520404e840 con 0x7f52040adc50 2023-12-19T18:17:17.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:17.765+0000 7f52017fa640 1 -- 172.21.15.27:0/4279291983 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v23) v1 ==== 122+0+0 (secure 0 0 0) 0x7f51f8057ac0 con 0x7f52040adc50 2023-12-19T18:17:17.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:17.768+0000 7f5209f11640 1 -- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f51e806a430 msgr2=0x7f51e806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:17.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:17.768+0000 7f5209f11640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f51e806a430 0x7f51e806c8f0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f51f0036000 tx=0x7f51f003b040 comp rx=0 tx=0).stop 2023-12-19T18:17:17.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:17.768+0000 7f5209f11640 1 -- 172.21.15.27:0/4279291983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52040adc50 msgr2=0x7f52041f3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:17.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:17.768+0000 7f5209f11640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52040adc50 0x7f52041f3280 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f51f800d800 tx=0x7f51f800dcc0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:17.768+0000 7f5209f11640 1 -- 172.21.15.27:0/4279291983 shutdown_connections 2023-12-19T18:17:17.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:17.768+0000 7f5209f11640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f51e806a430 0x7f51e806c8f0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:17.768+0000 7f5209f11640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f520415b810 0x7f5204158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:17.768+0000 7f5209f11640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5204158790 0x7f520406b2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:17.768+0000 7f5209f11640 1 --2- 172.21.15.27:0/4279291983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52040adc50 0x7f52041f3280 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:17.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:17.768+0000 7f5209f11640 1 -- 172.21.15.27:0/4279291983 >> 172.21.15.27:0/4279291983 conn(0x7f52040b05b0 msgr2=0x7f5204156020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:17.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:17.769+0000 7f5209f11640 1 -- 172.21.15.27:0/4279291983 shutdown_connections 2023-12-19T18:17:17.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:17.769+0000 7f5209f11640 1 -- 172.21.15.27:0/4279291983 wait complete. 2023-12-19T18:17:17.790 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:17:17.790 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:18.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.028+0000 7f182c9f9640 1 Processor -- start 2023-12-19T18:17:18.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.028+0000 7f182c9f9640 1 -- start start 2023-12-19T18:17:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.030+0000 7f182c9f9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f18280af630 0x7f18280afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.030+0000 7f182c9f9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18280c4550 0x7f18280c49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.030+0000 7f182c9f9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18280cc190 con 0x7f1828154030 2023-12-19T18:17:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.030+0000 7f182c9f9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18280b0000 con 0x7f18280af630 2023-12-19T18:17:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.030+0000 7f182c9f9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18280cdc00 con 0x7f18280c4550 2023-12-19T18:17:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.031+0000 7f1826ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f18280af630 0x7f18280afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.031+0000 7f1826ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f18280af630 0x7f18280afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34070/0 (socket says 172.21.15.27:34070) 2023-12-19T18:17:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.031+0000 7f1826ffd640 1 -- 172.21.15.27:0/1408129440 learned_addr learned my addr 172.21.15.27:0/1408129440 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.031+0000 7f1827fff640 1 --2- 172.21.15.27:0/1408129440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18280c4550 0x7f18280c49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.031+0000 7f1826ffd640 1 -- 172.21.15.27:0/1408129440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18280c4550 msgr2=0x7f18280c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.031+0000 7f1826ffd640 1 --2- 172.21.15.27:0/1408129440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18280c4550 0x7f18280c49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.031+0000 7f1826ffd640 1 -- 172.21.15.27:0/1408129440 >> v1:172.21.15.192:6789/0 conn(0x7f1828154030 legacy=0x7f18280cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.031+0000 7f1826ffd640 1 -- 172.21.15.27:0/1408129440 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18280cb2a0 con 0x7f18280af630 2023-12-19T18:17:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.031+0000 7f1827fff640 1 --2- 172.21.15.27:0/1408129440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18280c4550 0x7f18280c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.032+0000 7f1826ffd640 1 --2- 172.21.15.27:0/1408129440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18280af630 0x7f18280afac0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f181c009870 tx=0x7f181c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b29d7ab437529a15 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.033+0000 7f18267fc640 1 -- 172.21.15.27:0/1408129440 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c031640 con 0x7f18280af630 2023-12-19T18:17:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.033+0000 7f18267fc640 1 -- 172.21.15.27:0/1408129440 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c0317a0 con 0x7f18280af630 2023-12-19T18:17:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.033+0000 7f18267fc640 1 -- 172.21.15.27:0/1408129440 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c031b30 con 0x7f18280af630 2023-12-19T18:17:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.033+0000 7f182c9f9640 1 -- 172.21.15.27:0/1408129440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18280af630 msgr2=0x7f18280afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.033+0000 7f182c9f9640 1 --2- 172.21.15.27:0/1408129440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18280af630 0x7f18280afac0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f181c009870 tx=0x7f181c02b8f0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.033+0000 7f182c9f9640 1 -- 172.21.15.27:0/1408129440 shutdown_connections 2023-12-19T18:17:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.033+0000 7f182c9f9640 1 --2- 172.21.15.27:0/1408129440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18280c4550 0x7f18280c49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.033+0000 7f182c9f9640 1 --2- 172.21.15.27:0/1408129440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18280af630 0x7f18280afac0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.033+0000 7f182c9f9640 1 -- 172.21.15.27:0/1408129440 >> 172.21.15.27:0/1408129440 conn(0x7f1828069c70 msgr2=0x7f182806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.033+0000 7f182c9f9640 1 -- 172.21.15.27:0/1408129440 shutdown_connections 2023-12-19T18:17:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.034+0000 7f182c9f9640 1 -- 172.21.15.27:0/1408129440 wait complete. 2023-12-19T18:17:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.035+0000 7f182c9f9640 1 Processor -- start 2023-12-19T18:17:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.035+0000 7f182c9f9640 1 -- start start 2023-12-19T18:17:18.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.036+0000 7f182c9f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18280af630 0x7f182806ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:18.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.036+0000 7f182c9f9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18280c4550 0x7f18281492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:18.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.036+0000 7f182c9f9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1828154030 0x7f1828143180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:18.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.036+0000 7f182c9f9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18281436c0 con 0x7f18280af630 2023-12-19T18:17:18.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.036+0000 7f182c9f9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18281438b0 con 0x7f18280c4550 2023-12-19T18:17:18.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.036+0000 7f182c9f9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f182806c990 con 0x7f1828154030 2023-12-19T18:17:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.037+0000 7f18277fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18280af630 0x7f182806ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.037+0000 7f18277fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18280af630 0x7f182806ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34082/0 (socket says 172.21.15.27:34082) 2023-12-19T18:17:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.037+0000 7f18277fe640 1 -- 172.21.15.27:0/4159022440 learned_addr learned my addr 172.21.15.27:0/4159022440 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.037+0000 7f1826ffd640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18280c4550 0x7f18281492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.037+0000 7f1827fff640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1828154030 0x7f1828143180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.037+0000 7f18277fe640 1 -- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1828154030 msgr2=0x7f1828143180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.037+0000 7f18277fe640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1828154030 0x7f1828143180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.037+0000 7f18277fe640 1 -- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18280c4550 msgr2=0x7f18281492b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.037+0000 7f18277fe640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18280c4550 0x7f18281492b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.037+0000 7f18277fe640 1 -- 172.21.15.27:0/4159022440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f181c0094d0 con 0x7f18280af630 2023-12-19T18:17:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.037+0000 7f1826ffd640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18280c4550 0x7f18281492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.038+0000 7f18277fe640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18280af630 0x7f182806ce20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f181800b3f0 tx=0x7f181800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.038+0000 7f180ffff640 1 -- 172.21.15.27:0/4159022440 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1818011830 con 0x7f18280af630 2023-12-19T18:17:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.038+0000 7f180ffff640 1 -- 172.21.15.27:0/4159022440 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1818011990 con 0x7f18280af630 2023-12-19T18:17:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.038+0000 7f180ffff640 1 -- 172.21.15.27:0/4159022440 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1818011d40 con 0x7f18280af630 2023-12-19T18:17:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.038+0000 7f182c9f9640 1 -- 172.21.15.27:0/4159022440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f182806c060 con 0x7f18280af630 2023-12-19T18:17:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.038+0000 7f182c9f9640 1 -- 172.21.15.27:0/4159022440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f182806c620 con 0x7f18280af630 2023-12-19T18:17:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.039+0000 7f182c9f9640 1 -- 172.21.15.27:0/4159022440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f182804e840 con 0x7f18280af630 2023-12-19T18:17:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.040+0000 7f180ffff640 1 -- 172.21.15.27:0/4159022440 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f181801a400 con 0x7f18280af630 2023-12-19T18:17:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.041+0000 7f180ffff640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f181406a460 0x7f181406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.041+0000 7f180ffff640 1 -- 172.21.15.27:0/4159022440 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1818088680 con 0x7f18280af630 2023-12-19T18:17:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.042+0000 7f1826ffd640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f181406a460 0x7f181406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.043+0000 7f1826ffd640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f181406a460 0x7f181406c920 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f181c036000 tx=0x7f181c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:18.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.046+0000 7f180ffff640 1 -- 172.21.15.27:0/4159022440 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1818053d00 con 0x7f18280af630 2023-12-19T18:17:18.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.279+0000 7f182c9f9640 1 -- 172.21.15.27:0/4159022440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18280c3c50 con 0x7f18280af630 2023-12-19T18:17:18.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.280+0000 7f180ffff640 1 -- 172.21.15.27:0/4159022440 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f1818057320 con 0x7f18280af630 2023-12-19T18:17:18.281 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:18.281 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/930391176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":930391176},{"type":"v1","addr":"172.21.15.192:6837","nonce":930391176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5179,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3606564673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3606564673},{"type":"v1","addr":"172.21.15.192:6836","nonce":3606564673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3604117097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3604117097},{"type":"v1","addr":"172.21.15.27:6834","nonce":3604117097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:14.250689+0000","modified":"2023-12-19T18:17:17.758670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3814458529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3814458529},{"type":"v1","addr":"172.21.15.27:6835","nonce":3814458529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T18:17:18.281 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-19T18:17:18.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.283+0000 7f182c9f9640 1 -- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f181406a460 msgr2=0x7f181406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:18.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.283+0000 7f182c9f9640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f181406a460 0x7f181406c920 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f181c036000 tx=0x7f181c03c040 comp rx=0 tx=0).stop 2023-12-19T18:17:18.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.283+0000 7f182c9f9640 1 -- 172.21.15.27:0/4159022440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18280af630 msgr2=0x7f182806ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:18.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.283+0000 7f182c9f9640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18280af630 0x7f182806ce20 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f181800b3f0 tx=0x7f181800b8b0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.283+0000 7f182c9f9640 1 -- 172.21.15.27:0/4159022440 shutdown_connections 2023-12-19T18:17:18.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.283+0000 7f182c9f9640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f181406a460 0x7f181406c920 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.283+0000 7f182c9f9640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1828154030 0x7f1828143180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.283+0000 7f182c9f9640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18280c4550 0x7f18281492b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.283+0000 7f182c9f9640 1 --2- 172.21.15.27:0/4159022440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18280af630 0x7f182806ce20 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.283+0000 7f182c9f9640 1 -- 172.21.15.27:0/4159022440 >> 172.21.15.27:0/4159022440 conn(0x7f1828069c70 msgr2=0x7f18280ca270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:18.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.283+0000 7f182c9f9640 1 -- 172.21.15.27:0/4159022440 shutdown_connections 2023-12-19T18:17:18.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.284+0000 7f182c9f9640 1 -- 172.21.15.27:0/4159022440 wait complete. 2023-12-19T18:17:18.302 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:17:18.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.527+0000 7f5c9a099640 1 Processor -- start 2023-12-19T18:17:18.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.527+0000 7f5c9a099640 1 -- start start 2023-12-19T18:17:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.529+0000 7f5c9a099640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5c9415ee30 0x7f5c940c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.529+0000 7f5c9a099640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5c940afeb0 0x7f5c940c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.529+0000 7f5c9a099640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c940cb350 con 0x7f5c941553a0 2023-12-19T18:17:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.529+0000 7f5c9a099640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c940cbcf0 con 0x7f5c940afeb0 2023-12-19T18:17:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.529+0000 7f5c9a099640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c940cb630 con 0x7f5c9415ee30 2023-12-19T18:17:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.529+0000 7f5c99097640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5c941553a0 0x7f5c9415e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41786/0 (socket says 172.21.15.27:41786) 2023-12-19T18:17:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.529+0000 7f5c99097640 1 -- 172.21.15.27:0/3946213965 learned_addr learned my addr 172.21.15.27:0/3946213965 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.529+0000 7f5c99898640 1 --2- 172.21.15.27:0/3946213965 >> v2:172.21.15.27:3300/0 conn(0x7f5c940afeb0 0x7f5c940c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.529+0000 7f5c8bfff640 1 -- 172.21.15.27:0/3946213965 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3263192081 0 0) 0x7f5c940cb350 con 0x7f5c941553a0 2023-12-19T18:17:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.529+0000 7f5c98896640 1 --2- 172.21.15.27:0/3946213965 >> v2:172.21.15.192:3300/0 conn(0x7f5c9415ee30 0x7f5c940c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.530+0000 7f5c8bfff640 1 -- 172.21.15.27:0/3946213965 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c80003520 con 0x7f5c941553a0 2023-12-19T18:17:18.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.530+0000 7f5c99898640 1 -- 172.21.15.27:0/3946213965 >> v2:172.21.15.192:3300/0 conn(0x7f5c9415ee30 msgr2=0x7f5c940c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:18.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.530+0000 7f5c99898640 1 --2- 172.21.15.27:0/3946213965 >> v2:172.21.15.192:3300/0 conn(0x7f5c9415ee30 0x7f5c940c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.530+0000 7f5c99898640 1 -- 172.21.15.27:0/3946213965 >> v1:172.21.15.27:6789/0 conn(0x7f5c941553a0 legacy=0x7f5c9415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:18.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.530+0000 7f5c8bfff640 1 -- 172.21.15.27:0/3946213965 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2638695419 0 0) 0x7f5c80003520 con 0x7f5c941553a0 2023-12-19T18:17:18.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.530+0000 7f5c99898640 1 -- 172.21.15.27:0/3946213965 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c940cb910 con 0x7f5c940afeb0 2023-12-19T18:17:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.531+0000 7f5c99898640 1 --2- 172.21.15.27:0/3946213965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c940afeb0 0x7f5c940c39d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5c9406d4a0 tx=0x7f5c8c02bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f28c572bed40264 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.532+0000 7f5c8bfff640 1 -- 172.21.15.27:0/3946213965 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c90001670 con 0x7f5c940afeb0 2023-12-19T18:17:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.532+0000 7f5c8bfff640 1 -- 172.21.15.27:0/3946213965 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8c02c830 con 0x7f5c940afeb0 2023-12-19T18:17:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.532+0000 7f5c8bfff640 1 -- 172.21.15.27:0/3946213965 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8c02cbc0 con 0x7f5c940afeb0 2023-12-19T18:17:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.532+0000 7f5c9a099640 1 -- 172.21.15.27:0/3946213965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c940afeb0 msgr2=0x7f5c940c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.532+0000 7f5c9a099640 1 --2- 172.21.15.27:0/3946213965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c940afeb0 0x7f5c940c39d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f5c9406d4a0 tx=0x7f5c8c02bd00 comp rx=0 tx=0).stop 2023-12-19T18:17:18.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.532+0000 7f5c9a099640 1 -- 172.21.15.27:0/3946213965 shutdown_connections 2023-12-19T18:17:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.532+0000 7f5c9a099640 1 --2- 172.21.15.27:0/3946213965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c940afeb0 0x7f5c940c39d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.532+0000 7f5c9a099640 1 --2- 172.21.15.27:0/3946213965 >> v2:172.21.15.192:3300/0 conn(0x7f5c9415ee30 0x7f5c940c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.532+0000 7f5c9a099640 1 -- 172.21.15.27:0/3946213965 >> 172.21.15.27:0/3946213965 conn(0x7f5c94069c70 msgr2=0x7f5c9406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.533+0000 7f5c9a099640 1 -- 172.21.15.27:0/3946213965 shutdown_connections 2023-12-19T18:17:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.533+0000 7f5c9a099640 1 -- 172.21.15.27:0/3946213965 wait complete. 2023-12-19T18:17:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.534+0000 7f5c9a099640 1 Processor -- start 2023-12-19T18:17:18.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.535+0000 7f5c9a099640 1 -- start start 2023-12-19T18:17:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c9a099640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c940afeb0 0x7f5c940b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c9a099640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c941553a0 0x7f5c94150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c9a099640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c9415ee30 0x7f5c9415c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c9a099640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c940ae2e0 con 0x7f5c941553a0 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c9a099640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9415cc40 con 0x7f5c940afeb0 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c9a099640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9415cf20 con 0x7f5c9415ee30 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c98896640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c941553a0 0x7f5c94150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c98896640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c941553a0 0x7f5c94150610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34104/0 (socket says 172.21.15.27:34104) 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c98896640 1 -- 172.21.15.27:0/2973576413 learned_addr learned my addr 172.21.15.27:0/2973576413 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c99097640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c940afeb0 0x7f5c940b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c98896640 1 -- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c9415ee30 msgr2=0x7f5c9415c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c98896640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c9415ee30 0x7f5c9415c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c98896640 1 -- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c940afeb0 msgr2=0x7f5c940b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c98896640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c940afeb0 0x7f5c940b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c98896640 1 -- 172.21.15.27:0/2973576413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c8c009a70 con 0x7f5c941553a0 2023-12-19T18:17:18.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.536+0000 7f5c99097640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c940afeb0 0x7f5c940b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:18.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.537+0000 7f5c98896640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c941553a0 0x7f5c94150610 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400d5f0 tx=0x7f5c8400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:18.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.537+0000 7f5c89ffb640 1 -- 172.21.15.27:0/2973576413 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c840125f0 con 0x7f5c941553a0 2023-12-19T18:17:18.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.537+0000 7f5c89ffb640 1 -- 172.21.15.27:0/2973576413 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c84012750 con 0x7f5c941553a0 2023-12-19T18:17:18.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.537+0000 7f5c9a099640 1 -- 172.21.15.27:0/2973576413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c9415d200 con 0x7f5c941553a0 2023-12-19T18:17:18.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.537+0000 7f5c89ffb640 1 -- 172.21.15.27:0/2973576413 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c84012b00 con 0x7f5c941553a0 2023-12-19T18:17:18.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.537+0000 7f5c9a099640 1 -- 172.21.15.27:0/2973576413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c9415c810 con 0x7f5c941553a0 2023-12-19T18:17:18.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.538+0000 7f5c9a099640 1 -- 172.21.15.27:0/2973576413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c940ae420 con 0x7f5c941553a0 2023-12-19T18:17:18.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.539+0000 7f5c89ffb640 1 -- 172.21.15.27:0/2973576413 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5c84012c60 con 0x7f5c941553a0 2023-12-19T18:17:18.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.540+0000 7f5c89ffb640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c7806a460 0x7f5c7806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:18.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.541+0000 7f5c89ffb640 1 -- 172.21.15.27:0/2973576413 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c84088d70 con 0x7f5c941553a0 2023-12-19T18:17:18.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.541+0000 7f5c99097640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c7806a460 0x7f5c7806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:18.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.542+0000 7f5c99097640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c7806a460 0x7f5c7806c920 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5c900011e0 tx=0x7f5c90009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:18.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.545+0000 7f5c89ffb640 1 -- 172.21.15.27:0/2973576413 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5c940ae420 con 0x7f5c941553a0 2023-12-19T18:17:18.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.742+0000 7f5c9a099640 1 -- 172.21.15.27:0/2973576413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c940ae420 con 0x7f5c941553a0 2023-12-19T18:17:18.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.745+0000 7f5c89ffb640 1 -- 172.21.15.27:0/2973576413 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+16041 (secure 0 0 0) 0x7f5c940ae420 con 0x7f5c941553a0 2023-12-19T18:17:18.746 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:18.746 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":50,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:17:15.255587+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:12.527441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"49","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:13.291755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:17:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.749+0000 7f5c9a099640 1 -- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c7806a460 msgr2=0x7f5c7806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.749+0000 7f5c9a099640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c7806a460 0x7f5c7806c920 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5c900011e0 tx=0x7f5c90009040 comp rx=0 tx=0).stop 2023-12-19T18:17:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.749+0000 7f5c9a099640 1 -- 172.21.15.27:0/2973576413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c941553a0 msgr2=0x7f5c94150610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.749+0000 7f5c9a099640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c941553a0 0x7f5c94150610 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5c8400d5f0 tx=0x7f5c8400dab0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.749+0000 7f5c9a099640 1 -- 172.21.15.27:0/2973576413 shutdown_connections 2023-12-19T18:17:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.749+0000 7f5c9a099640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c7806a460 0x7f5c7806c920 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.749+0000 7f5c9a099640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c9415ee30 0x7f5c9415c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.749+0000 7f5c9a099640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c941553a0 0x7f5c94150610 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.749+0000 7f5c9a099640 1 --2- 172.21.15.27:0/2973576413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c940afeb0 0x7f5c940b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.749+0000 7f5c9a099640 1 -- 172.21.15.27:0/2973576413 >> 172.21.15.27:0/2973576413 conn(0x7f5c94069c70 msgr2=0x7f5c94158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.749+0000 7f5c9a099640 1 -- 172.21.15.27:0/2973576413 shutdown_connections 2023-12-19T18:17:18.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:18.749+0000 7f5c9a099640 1 -- 172.21.15.27:0/2973576413 wait complete. 2023-12-19T18:17:18.770 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:17:18.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:17:19.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.004+0000 7f81dcfca640 1 Processor -- start 2023-12-19T18:17:19.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.004+0000 7f81dcfca640 1 -- start start 2023-12-19T18:17:19.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.006+0000 7f81dcfca640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81d81604d0 0x7f81d80b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.006+0000 7f81dcfca640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f81d80d2350 0x7f81d80b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.006+0000 7f81dcfca640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f81d80d0300 0x7f81d80d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.006+0000 7f81dcfca640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d80d2b30 con 0x7f81d80d2350 2023-12-19T18:17:19.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.006+0000 7f81dcfca640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d80b5e90 con 0x7f81d80d0300 2023-12-19T18:17:19.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.006+0000 7f81dcfca640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80d2850 con 0x7f81d81604d0 2023-12-19T18:17:19.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.006+0000 7f81d6ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f81d80d2350 0x7f81d80b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.006+0000 7f81d6ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f81d80d2350 0x7f81d80b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34118/0 (socket says 172.21.15.27:34118) 2023-12-19T18:17:19.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.006+0000 7f81d6ffd640 1 -- 172.21.15.27:0/1455437340 learned_addr learned my addr 172.21.15.27:0/1455437340 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:19.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.006+0000 7f81d77fe640 1 --2- 172.21.15.27:0/1455437340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81d81604d0 0x7f81d80b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.007+0000 7f81d7fff640 1 --2- 172.21.15.27:0/1455437340 >> v2:172.21.15.192:3300/0 conn(0x7f81d80d0300 0x7f81d80d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.007+0000 7f81d6ffd640 1 -- 172.21.15.27:0/1455437340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81d81604d0 msgr2=0x7f81d80b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:19.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.007+0000 7f81d6ffd640 1 --2- 172.21.15.27:0/1455437340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81d81604d0 0x7f81d80b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.007+0000 7f81d6ffd640 1 -- 172.21.15.27:0/1455437340 >> v2:172.21.15.192:3300/0 conn(0x7f81d80d0300 msgr2=0x7f81d80d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:19.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.007+0000 7f81d6ffd640 1 --2- 172.21.15.27:0/1455437340 >> v2:172.21.15.192:3300/0 conn(0x7f81d80d0300 0x7f81d80d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.007+0000 7f81d6ffd640 1 -- 172.21.15.27:0/1455437340 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d80cded0 con 0x7f81d80d2350 2023-12-19T18:17:19.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.007+0000 7f81d77fe640 1 --2- 172.21.15.27:0/1455437340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81d81604d0 0x7f81d80b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:19.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.007+0000 7f81d7fff640 1 --2- 172.21.15.27:0/1455437340 >> v2:172.21.15.192:3300/0 conn(0x7f81d80d0300 0x7f81d80d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:19.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.008+0000 7f81d6ffd640 1 --2- 172.21.15.27:0/1455437340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81d80d2350 0x7f81d80b4d20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f81d8071e80 tx=0x7f81c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa94d9537a4ff40 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.008+0000 7f81d67fc640 1 -- 172.21.15.27:0/1455437340 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c0031870 con 0x7f81d80d2350 2023-12-19T18:17:19.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.008+0000 7f81d67fc640 1 -- 172.21.15.27:0/1455437340 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c00319d0 con 0x7f81d80d2350 2023-12-19T18:17:19.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.008+0000 7f81d67fc640 1 -- 172.21.15.27:0/1455437340 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c0031d60 con 0x7f81d80d2350 2023-12-19T18:17:19.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.009+0000 7f81dcfca640 1 -- 172.21.15.27:0/1455437340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81d80d2350 msgr2=0x7f81d80b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.009+0000 7f81dcfca640 1 --2- 172.21.15.27:0/1455437340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81d80d2350 0x7f81d80b4d20 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f81d8071e80 tx=0x7f81c002b910 comp rx=0 tx=0).stop 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.009+0000 7f81dcfca640 1 -- 172.21.15.27:0/1455437340 shutdown_connections 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.009+0000 7f81dcfca640 1 --2- 172.21.15.27:0/1455437340 >> v2:172.21.15.192:3300/0 conn(0x7f81d80d0300 0x7f81d80d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.009+0000 7f81dcfca640 1 --2- 172.21.15.27:0/1455437340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81d80d2350 0x7f81d80b4d20 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.009+0000 7f81dcfca640 1 --2- 172.21.15.27:0/1455437340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81d81604d0 0x7f81d80b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.009+0000 7f81dcfca640 1 -- 172.21.15.27:0/1455437340 >> 172.21.15.27:0/1455437340 conn(0x7f81d80cf7e0 msgr2=0x7f81d80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.009+0000 7f81dcfca640 1 -- 172.21.15.27:0/1455437340 shutdown_connections 2023-12-19T18:17:19.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.009+0000 7f81dcfca640 1 -- 172.21.15.27:0/1455437340 wait complete. 2023-12-19T18:17:19.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.011+0000 7f81dcfca640 1 Processor -- start 2023-12-19T18:17:19.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.011+0000 7f81dcfca640 1 -- start start 2023-12-19T18:17:19.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.012+0000 7f81dcfca640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81d80d0300 0x7f81d80cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.012+0000 7f81dcfca640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81d80d2350 0x7f81d8152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.012+0000 7f81dcfca640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81d81604d0 0x7f81d80c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.012+0000 7f81dcfca640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d80c7ad0 con 0x7f81d81604d0 2023-12-19T18:17:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.012+0000 7f81dcfca640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d815ad60 con 0x7f81d80d2350 2023-12-19T18:17:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.012+0000 7f81dcfca640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d815b040 con 0x7f81d80d0300 2023-12-19T18:17:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.012+0000 7f81d7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81d81604d0 0x7f81d80c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.012+0000 7f81d7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81d81604d0 0x7f81d80c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34126/0 (socket says 172.21.15.27:34126) 2023-12-19T18:17:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.012+0000 7f81d7fff640 1 -- 172.21.15.27:0/1389223529 learned_addr learned my addr 172.21.15.27:0/1389223529 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.013+0000 7f81d6ffd640 1 --2- 172.21.15.27:0/1389223529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81d80d2350 0x7f81d8152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.013+0000 7f81d7fff640 1 -- 172.21.15.27:0/1389223529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81d80d0300 msgr2=0x7f81d80cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.013+0000 7f81d7fff640 1 --2- 172.21.15.27:0/1389223529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81d80d0300 0x7f81d80cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.013+0000 7f81d7fff640 1 -- 172.21.15.27:0/1389223529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81d80d2350 msgr2=0x7f81d8152b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.013+0000 7f81d7fff640 1 --2- 172.21.15.27:0/1389223529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81d80d2350 0x7f81d8152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.013+0000 7f81d7fff640 1 -- 172.21.15.27:0/1389223529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c00094d0 con 0x7f81d81604d0 2023-12-19T18:17:19.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.013+0000 7f81d7fff640 1 --2- 172.21.15.27:0/1389223529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81d81604d0 0x7f81d80c7280 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f81d000afd0 tx=0x7f81d000ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.013+0000 7f81cffff640 1 -- 172.21.15.27:0/1389223529 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d0004440 con 0x7f81d81604d0 2023-12-19T18:17:19.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.013+0000 7f81cffff640 1 -- 172.21.15.27:0/1389223529 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d0003610 con 0x7f81d81604d0 2023-12-19T18:17:19.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.013+0000 7f81cffff640 1 -- 172.21.15.27:0/1389223529 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d00039c0 con 0x7f81d81604d0 2023-12-19T18:17:19.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.014+0000 7f81dcfca640 1 -- 172.21.15.27:0/1389223529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d815b320 con 0x7f81d81604d0 2023-12-19T18:17:19.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.014+0000 7f81dcfca640 1 -- 172.21.15.27:0/1389223529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d80c7db0 con 0x7f81d81604d0 2023-12-19T18:17:19.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.015+0000 7f81dcfca640 1 -- 172.21.15.27:0/1389223529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d815abc0 con 0x7f81d81604d0 2023-12-19T18:17:19.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.016+0000 7f81cffff640 1 -- 172.21.15.27:0/1389223529 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f81d0003b20 con 0x7f81d81604d0 2023-12-19T18:17:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.017+0000 7f81cffff640 1 --2- 172.21.15.27:0/1389223529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f81b806a430 0x7f81b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.017+0000 7f81cffff640 1 -- 172.21.15.27:0/1389223529 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81d0088ac0 con 0x7f81d81604d0 2023-12-19T18:17:19.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.017+0000 7f81d77fe640 1 --2- 172.21.15.27:0/1389223529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f81b806a430 0x7f81b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.019+0000 7f81d77fe640 1 --2- 172.21.15.27:0/1389223529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f81b806a430 0x7f81b806c8f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f81c8008ea0 tx=0x7f81c80083d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.021+0000 7f81cffff640 1 -- 172.21.15.27:0/1389223529 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f81d815abc0 con 0x7f81d81604d0 2023-12-19T18:17:19.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.210+0000 7f81dcfca640 1 -- 172.21.15.27:0/1389223529 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f81d80ce490 con 0x7f81b806a430 2023-12-19T18:17:19.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.234+0000 7f81cffff640 1 -- 172.21.15.27:0/1389223529 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+273772 (secure 0 0 0) 0x7f81c8012070 con 0x7f81b806a430 2023-12-19T18:17:19.239 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:19.240 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:17:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.241+0000 7f81dcfca640 1 -- 172.21.15.27:0/1389223529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f81b806a430 msgr2=0x7f81b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.241+0000 7f81dcfca640 1 --2- 172.21.15.27:0/1389223529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f81b806a430 0x7f81b806c8f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f81c8008ea0 tx=0x7f81c80083d0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.241+0000 7f81dcfca640 1 -- 172.21.15.27:0/1389223529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81d81604d0 msgr2=0x7f81d80c7280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.241+0000 7f81dcfca640 1 --2- 172.21.15.27:0/1389223529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81d81604d0 0x7f81d80c7280 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f81d000afd0 tx=0x7f81d000ca20 comp rx=0 tx=0).stop 2023-12-19T18:17:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.241+0000 7f81dcfca640 1 -- 172.21.15.27:0/1389223529 shutdown_connections 2023-12-19T18:17:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.241+0000 7f81dcfca640 1 --2- 172.21.15.27:0/1389223529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f81b806a430 0x7f81b806c8f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.241+0000 7f81dcfca640 1 --2- 172.21.15.27:0/1389223529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81d81604d0 0x7f81d80c7280 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.241+0000 7f81dcfca640 1 --2- 172.21.15.27:0/1389223529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f81d80d2350 0x7f81d8152b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.241+0000 7f81dcfca640 1 --2- 172.21.15.27:0/1389223529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f81d80d0300 0x7f81d80cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.241+0000 7f81dcfca640 1 -- 172.21.15.27:0/1389223529 >> 172.21.15.27:0/1389223529 conn(0x7f81d80cf7e0 msgr2=0x7f81d80b4460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.241+0000 7f81dcfca640 1 -- 172.21.15.27:0/1389223529 shutdown_connections 2023-12-19T18:17:19.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.241+0000 7f81dcfca640 1 -- 172.21.15.27:0/1389223529 wait complete. 2023-12-19T18:17:19.262 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":119,"stamp":"2023-12-19T18:17:17.753413+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":19,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":2910,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559589,"statfs":{"total":1600000000,"available":1597019696,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.013908"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.746309+0000","last_change":"2023-12-19T18:17:12.746309+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.746309+0000","last_undegraded":"2023-12-19T18:17:12.746309+0000","last_fullsized":"2023-12-19T18:17:12.746309+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:10.250166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.744436+0000","last_change":"2023-12-19T18:17:12.744436+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.744436+0000","last_undegraded":"2023-12-19T18:17:12.744436+0000","last_fullsized":"2023-12-19T18:17:12.744436+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:52:34.850640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.745206+0000","last_change":"2023-12-19T18:17:12.745206+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.745206+0000","last_undegraded":"2023-12-19T18:17:12.745206+0000","last_fullsized":"2023-12-19T18:17:12.745206+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:37:11.023952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.744321+0000","last_change":"2023-12-19T18:17:12.744321+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.744321+0000","last_undegraded":"2023-12-19T18:17:12.744321+0000","last_fullsized":"2023-12-19T18:17:12.744321+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:33.389677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.748370+0000","last_change":"2023-12-19T18:17:12.748370+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.748370+0000","last_undegraded":"2023-12-19T18:17:12.748370+0000","last_fullsized":"2023-12-19T18:17:12.748370+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:57:35.015037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.742333+0000","last_change":"2023-12-19T18:17:12.742333+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.742333+0000","last_undegraded":"2023-12-19T18:17:12.742333+0000","last_fullsized":"2023-12-19T18:17:12.742333+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:21.833075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.748741+0000","last_change":"2023-12-19T18:17:12.748741+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.748741+0000","last_undegraded":"2023-12-19T18:17:12.748741+0000","last_fullsized":"2023-12-19T18:17:12.748741+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:54:51.030454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.745729+0000","last_change":"2023-12-19T18:17:12.745729+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.745729+0000","last_undegraded":"2023-12-19T18:17:12.745729+0000","last_fullsized":"2023-12-19T18:17:12.745729+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:58:12.447837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.746474+0000","last_change":"2023-12-19T18:17:12.746474+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.746474+0000","last_undegraded":"2023-12-19T18:17:12.746474+0000","last_fullsized":"2023-12-19T18:17:12.746474+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:48:46.625951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.744155+0000","last_change":"2023-12-19T18:17:12.744155+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.744155+0000","last_undegraded":"2023-12-19T18:17:12.744155+0000","last_fullsized":"2023-12-19T18:17:12.744155+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:15.322827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.748612+0000","last_change":"2023-12-19T18:17:12.748612+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.748612+0000","last_undegraded":"2023-12-19T18:17:12.748612+0000","last_fullsized":"2023-12-19T18:17:12.748612+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:54:51.030454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.743946+0000","last_change":"2023-12-19T18:17:12.743946+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.743946+0000","last_undegraded":"2023-12-19T18:17:12.743946+0000","last_fullsized":"2023-12-19T18:17:12.743946+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:39.026592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.742567+0000","last_change":"2023-12-19T18:17:12.742567+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.742567+0000","last_undegraded":"2023-12-19T18:17:12.742567+0000","last_fullsized":"2023-12-19T18:17:12.742567+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:16.564267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.747708+0000","last_change":"2023-12-19T18:17:12.747708+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.747708+0000","last_undegraded":"2023-12-19T18:17:12.747708+0000","last_fullsized":"2023-12-19T18:17:12.747708+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:18.395505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.747495+0000","last_change":"2023-12-19T18:17:12.747495+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.747495+0000","last_undegraded":"2023-12-19T18:17:12.747495+0000","last_fullsized":"2023-12-19T18:17:12.747495+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:18.395505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.746623+0000","last_change":"2023-12-19T18:17:12.746623+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.746623+0000","last_undegraded":"2023-12-19T18:17:12.746623+0000","last_fullsized":"2023-12-19T18:17:12.746623+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:42:34.077752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.746729+0000","last_change":"2023-12-19T18:17:12.746729+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.746729+0000","last_undegraded":"2023-12-19T18:17:12.746729+0000","last_fullsized":"2023-12-19T18:17:12.746729+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:26:54.451411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.746450+0000","last_change":"2023-12-19T18:17:12.746450+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.746450+0000","last_undegraded":"2023-12-19T18:17:12.746450+0000","last_fullsized":"2023-12-19T18:17:12.746450+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:14:31.695686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.744490+0000","last_change":"2023-12-19T18:17:12.744490+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.744490+0000","last_undegraded":"2023-12-19T18:17:12.744490+0000","last_fullsized":"2023-12-19T18:17:12.744490+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:19.073949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.745730+0000","last_change":"2023-12-19T18:17:12.745730+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.745730+0000","last_undegraded":"2023-12-19T18:17:12.745730+0000","last_fullsized":"2023-12-19T18:17:12.745730+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:43:50.372916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.744004+0000","last_change":"2023-12-19T18:17:12.744004+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.744004+0000","last_undegraded":"2023-12-19T18:17:12.744004+0000","last_fullsized":"2023-12-19T18:17:12.744004+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:12.876293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.746926+0000","last_change":"2023-12-19T18:17:12.746926+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.746926+0000","last_undegraded":"2023-12-19T18:17:12.746926+0000","last_fullsized":"2023-12-19T18:17:12.746926+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:26:54.451411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.744848+0000","last_change":"2023-12-19T18:17:12.744848+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.744848+0000","last_undegraded":"2023-12-19T18:17:12.744848+0000","last_fullsized":"2023-12-19T18:17:12.744848+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:26:49.228013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.742200+0000","last_change":"2023-12-19T18:17:12.742200+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.742200+0000","last_undegraded":"2023-12-19T18:17:12.742200+0000","last_fullsized":"2023-12-19T18:17:12.742200+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:55:36.933023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.746664+0000","last_change":"2023-12-19T18:17:12.746664+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.746664+0000","last_undegraded":"2023-12-19T18:17:12.746664+0000","last_fullsized":"2023-12-19T18:17:12.746664+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:47.048524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.745787+0000","last_change":"2023-12-19T18:17:12.745787+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.745787+0000","last_undegraded":"2023-12-19T18:17:12.745787+0000","last_fullsized":"2023-12-19T18:17:12.745787+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:47.048524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.741333+0000","last_change":"2023-12-19T18:17:12.741333+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.741333+0000","last_undegraded":"2023-12-19T18:17:12.741333+0000","last_fullsized":"2023-12-19T18:17:12.741333+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:57:40.505431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.746943+0000","last_change":"2023-12-19T18:17:12.746943+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.746943+0000","last_undegraded":"2023-12-19T18:17:12.746943+0000","last_fullsized":"2023-12-19T18:17:12.746943+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:57:35.015037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.741019+0000","last_change":"2023-12-19T18:17:12.741019+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.741019+0000","last_undegraded":"2023-12-19T18:17:12.741019+0000","last_fullsized":"2023-12-19T18:17:12.741019+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:40.945820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.745373+0000","last_change":"2023-12-19T18:17:12.745373+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.745373+0000","last_undegraded":"2023-12-19T18:17:12.745373+0000","last_fullsized":"2023-12-19T18:17:12.745373+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:30:19.514717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.741227+0000","last_change":"2023-12-19T18:17:12.741227+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.741227+0000","last_undegraded":"2023-12-19T18:17:12.741227+0000","last_fullsized":"2023-12-19T18:17:12.741227+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:58:44.448587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-19T18:17:13.736698+0000","last_change":"2023-12-19T18:17:13.736698+0000","last_active":"2023-12-19T18:17:13.736698+0000","last_peered":"2023-12-19T18:17:13.736698+0000","last_clean":"2023-12-19T18:17:13.736698+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:13.736698+0000","last_undegraded":"2023-12-19T18:17:13.736698+0000","last_fullsized":"2023-12-19T18:17:13.736698+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":1,"reported_epoch":47,"state":"creating+peering","last_fresh":"2023-12-19T18:17:12.742571+0000","last_change":"2023-12-19T18:17:12.742571+0000","last_active":"2023-12-19T18:17:12.720765+0000","last_peered":"2023-12-19T18:17:12.720765+0000","last_clean":"2023-12-19T18:17:12.720765+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:17:12.742571+0000","last_undegraded":"2023-12-19T18:17:12.742571+0000","last_fullsized":"2023-12-19T18:17:12.742571+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:19:01.049904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":187,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-12-19T18:17:10.287015+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:17:10.287015+0000","last_peered":"2023-12-19T18:17:10.287015+0000","last_clean":"2023-12-19T18:17:10.287015+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:17:10.287015+0000","last_undegraded":"2023-12-19T18:17:10.287015+0000","last_fullsized":"2023-12-19T18:17:10.287015+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195092,"statfs":{"total":200000000,"available":199775054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.117,"15min":1.1100000000000001},"min":{"1min":0.73499999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.5640000000000001,"5min":1.639,"15min":1.639},"last":0.745},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.149,"15min":1.143},"min":{"1min":0.79900000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4910000000000001,"5min":1.677,"15min":1.677},"last":0.79600000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.175,"15min":1.1699999999999999},"min":{"1min":0.71099999999999997,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.6759999999999999,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":1.006},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1759999999999999,"15min":1.1859999999999999},"min":{"1min":0.83499999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.367,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":1.2649999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0680000000000001,"15min":1.077},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.373,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.198,"5min":1.161,"15min":1.155},"min":{"1min":0.72199999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.486,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.319}]},{"osd":3,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1140000000000001,"15min":1.1000000000000001},"min":{"1min":0.84399999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.96699999999999997},{"interface":"front","average":{"1min":1.381,"5min":1.2490000000000001,"15min":1.2270000000000001},"min":{"1min":0.94899999999999995,"5min":0.748,"15min":0.748},"max":{"1min":1.635,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":1.1599999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.0700000000000001,"15min":1.0509999999999999},"min":{"1min":0.77900000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.413},{"interface":"front","average":{"1min":1.25,"5min":1.214,"15min":1.208},"min":{"1min":0.92600000000000005,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.7370000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.351}]},{"osd":5,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.3129999999999999,"15min":1.2949999999999999},"min":{"1min":0.90300000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7370000000000001,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":1.466},{"interface":"front","average":{"1min":1.331,"5min":1.2929999999999999,"15min":1.2869999999999999},"min":{"1min":0.88800000000000001,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6539999999999999,"5min":2.754,"15min":2.754},"last":1.5229999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.431,"15min":1.417},"min":{"1min":1.254,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7729999999999999,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.8400000000000001},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.3360000000000001,"15min":1.3220000000000001},"min":{"1min":1.0009999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.661,"5min":2.149,"15min":2.149},"last":1.655}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195092,"statfs":{"total":200000000,"available":199775054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":0.96999999999999997,"15min":0.95399999999999996},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.028,"15min":1.0149999999999999},"min":{"1min":0.70099999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.1970000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.107,"15min":1.115},"min":{"1min":0.78000000000000003,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.548,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.1359999999999999,"15min":1.143},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.028}]},{"osd":2,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1080000000000001,"15min":1.0960000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.163},{"interface":"front","average":{"1min":1.155,"5min":1.2290000000000001,"15min":1.2410000000000001},"min":{"1min":0.81499999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.6850000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.411}]},{"osd":3,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.163,"15min":1.155},"min":{"1min":0.94799999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.48,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.331},{"interface":"front","average":{"1min":1.153,"5min":1.147,"15min":1.1459999999999999},"min":{"1min":0.872,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.468}]},{"osd":4,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.216,"15min":1.22},"min":{"1min":0.81799999999999995,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6180000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.3180000000000001,"15min":1.3009999999999999},"min":{"1min":0.96199999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":1.0760000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.1519999999999999,"15min":1.1379999999999999},"min":{"1min":0.75900000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.232},{"interface":"front","average":{"1min":1.387,"5min":1.319,"15min":1.3080000000000001},"min":{"1min":1.0389999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7869999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.544}]},{"osd":7,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3109999999999999,"15min":1.2989999999999999},"min":{"1min":0.77200000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":1.512},{"interface":"front","average":{"1min":1.389,"5min":1.3200000000000001,"15min":1.3080000000000001},"min":{"1min":0.92300000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":1.3520000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":796,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194516,"statfs":{"total":200000000,"available":199184686,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1419999999999999,"15min":1.135},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.4199999999999999,"5min":1.863,"15min":1.863},"last":0.67200000000000004},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.157,"15min":1.1399999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":1.014}]},{"osd":1,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.99399999999999999,"15min":0.98799999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.03,"15min":1.0149999999999999},"min":{"1min":0.68700000000000006,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.2030000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.0660000000000001,"15min":1.046},"min":{"1min":0.75700000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":1.284},{"interface":"front","average":{"1min":1.081,"5min":1.0900000000000001,"15min":1.0920000000000001},"min":{"1min":0.76800000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6040000000000001,"5min":1.698,"15min":1.698},"last":1.3879999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.0569999999999999,"15min":1.042},"min":{"1min":0.69399999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.1950000000000001,"15min":1.1739999999999999},"min":{"1min":0.88300000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.2390000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.1379999999999999,"15min":1.1080000000000001},"min":{"1min":0.77600000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.716,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.097},{"interface":"front","average":{"1min":1.165,"5min":1.105,"15min":1.095},"min":{"1min":0.85599999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5109999999999999,"5min":1.901,"15min":1.901},"last":0.96899999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.3120000000000001,"15min":1.282},"min":{"1min":1.1799999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.853,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.327},{"interface":"front","average":{"1min":1.298,"5min":1.2210000000000001,"15min":1.208},"min":{"1min":1.044,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5600000000000001,"5min":1.976,"15min":1.976},"last":1.2669999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.226,"15min":1.2},"min":{"1min":0.85399999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.9359999999999999,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.143},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.399,"15min":1.387},"min":{"1min":1.21,"5min":0.623,"15min":0.623},"max":{"1min":1.806,"5min":1.875,"15min":1.875},"last":1.1659999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195092,"statfs":{"total":200000000,"available":199775054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.008,"15min":1.0009999999999999},"min":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.4419999999999999,"5min":1.454,"15min":1.454},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.089,"5min":1.0369999999999999,"15min":1.028},"min":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.73699999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.137,"15min":1.141},"min":{"1min":0.875,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.393,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.327},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.161,"15min":1.1459999999999999},"min":{"1min":0.88200000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.92200000000000004}]},{"osd":2,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1100000000000001,"15min":1.1180000000000001},"min":{"1min":0.747,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5129999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.85799999999999998},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.075,"15min":1.075},"min":{"1min":0.80500000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.3160000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.4470000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2470000000000001,"15min":1.2410000000000001},"min":{"1min":0.88300000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.333,"5min":1.294,"15min":1.288},"min":{"1min":0.85499999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.7050000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.4039999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.165,"15min":1.155},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.3360000000000001,"15min":1.3180000000000001},"min":{"1min":0.69499999999999995,"5min":0.504,"15min":0.504},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.2290000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2989999999999999,"15min":1.2969999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.369,"15min":1.361},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.532}]},{"osd":7,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.331,"15min":1.319},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.006},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.8999999999999999,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.2689999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195092,"statfs":{"total":200000000,"available":199775054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.95599999999999996,"15min":0.97999999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2450000000000001,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":1.1339999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.49299999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.91100000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":1.0369999999999999,"15min":1.0700000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.488,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":1.0760000000000001},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.073,"15min":1.0940000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.512,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.292}]},{"osd":2,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.1579999999999999,"15min":1.1950000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6240000000000001,"5min":3.6890000000000001,"15min":3.6890000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1819999999999999,"15min":1.1950000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.726,"5min":1.905,"15min":1.905},"last":1.5900000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.167,"15min":1.169},"min":{"1min":0.79500000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.758,"5min":1.774,"15min":1.774},"last":1.115},{"interface":"front","average":{"1min":1.264,"5min":1.3859999999999999,"15min":1.4059999999999999},"min":{"1min":0.86399999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.742,"5min":4.0209999999999999,"15min":4.0209999999999999},"last":1.345}]},{"osd":5,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.421,"15min":1.444},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.804,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":1.8340000000000001},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2370000000000001,"15min":1.234},"min":{"1min":0.76600000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.1990000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.462,"15min":1.488},"min":{"1min":0.91000000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.8320000000000001,"5min":3.9399999999999999,"15min":3.9399999999999999},"last":1.246},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.4790000000000001,"15min":1.5029999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.871,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.95}]},{"osd":7,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.335,"15min":1.341},"min":{"1min":0.75900000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.5209999999999999,"15min":1.544},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":1.8180000000000001,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.4630000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195092,"statfs":{"total":200000000,"available":199775054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":0.98399999999999999},"min":{"1min":0.373,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.575,"5min":4.7610000000000001,"15min":4.7610000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.90200000000000002,"5min":1.0229999999999999,"15min":1.0429999999999999},"min":{"1min":0.435,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.645,"5min":4.7850000000000001,"15min":4.7850000000000001},"last":1.02}]},{"osd":1,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.2270000000000001,"15min":1.28},"min":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1919999999999999,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.999,"5min":1.169,"15min":1.1970000000000001},"min":{"1min":0.51400000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.623,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":1.145}]},{"osd":3,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.401,"15min":1.4670000000000001},"min":{"1min":0.61799999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5840000000000001,"5min":8.3109999999999999,"15min":8.3109999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.05,"5min":1.3700000000000001,"15min":1.423},"min":{"1min":0.51900000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.802,"5min":8.0690000000000008,"15min":8.0690000000000008},"last":1.232}]},{"osd":4,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.46,"15min":1.49},"min":{"1min":0.82599999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.702,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":1.2929999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.554,"15min":1.601},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.6819999999999999,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":1.3480000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.5840000000000001,"15min":1.6339999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.177,"5min":8.1839999999999993,"15min":8.1839999999999993},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.4039999999999999,"15min":1.417},"min":{"1min":0.90400000000000003,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8540000000000001,"5min":5.165,"15min":5.165},"last":1.085}]},{"osd":6,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.452,"15min":1.468},"min":{"1min":0.82999999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.7929999999999999,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.579,"15min":1.6160000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.1309999999999998,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":1.6200000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.5580000000000001,"15min":1.575},"min":{"1min":0.878,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.877,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":1.306},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.6870000000000001,"15min":1.7250000000000001},"min":{"1min":0.93999999999999995,"5min":0.878,"15min":0.878},"max":{"1min":2.2170000000000001,"5min":8.2300000000000004,"15min":8.2300000000000004},"last":1.6950000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":796,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194516,"statfs":{"total":200000000,"available":199184686,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.84499999999999997,"15min":0.84699999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.2849999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.75800000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.0469999999999999,"15min":1.0589999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.5700000000000001,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.89600000000000002,"15min":0.89000000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.85799999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.115,"15min":1.1020000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.27},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1779999999999999,"15min":1.1830000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.641,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.6200000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2689999999999999,"15min":1.274},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.821,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.264,"15min":1.262},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.3939999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.256,"15min":1.2509999999999999},"min":{"1min":0.91900000000000004,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.7290000000000001,"5min":1.821,"15min":1.821},"last":1.337},{"interface":"front","average":{"1min":1.397,"5min":1.3839999999999999,"15min":1.3819999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0249999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.4399999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3560000000000001,"15min":1.3580000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.948,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.48},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.425,"15min":1.419},"min":{"1min":0.98799999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.1280000000000001,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.5109999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.319,"15min":1.3069999999999999},"min":{"1min":0.82399999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.3610000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.105},{"interface":"front","average":{"1min":1.427,"5min":1.4319999999999999,"15min":1.4330000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.855,"5min":2,"15min":2},"last":1.5609999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":219,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195092,"statfs":{"total":200000000,"available":199775054,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.871,"15min":0.85899999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.639,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.998,"5min":0.99399999999999999,"15min":0.99299999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.593,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.56999999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.90100000000000002,"15min":0.91500000000000004},"min":{"1min":0.47999999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0980000000000001,"5min":1.577,"15min":1.577},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90200000000000002,"15min":0.90100000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.466,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.42399999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.98099999999999998,"15min":0.98899999999999999},"min":{"1min":0.53100000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.669,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":1.04,"5min":1.1639999999999999,"15min":1.1839999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.7230000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":1.0069999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.1719999999999999,"15min":1.1519999999999999},"min":{"1min":0.84099999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":1.196,"5min":1.264,"15min":1.2749999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7509999999999999,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.74399999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.319,"15min":1.339},"min":{"1min":0.88600000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6240000000000001,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.25,"15min":1.2410000000000001},"min":{"1min":0.81000000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6519999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":0.72199999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3620000000000001,"15min":1.363},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8049999999999999,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.335,"15min":1.319},"min":{"1min":0.99299999999999999,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.9379999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.77000000000000002}]},{"osd":7,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.3580000000000001,"15min":1.3420000000000001},"min":{"1min":0.95599999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.897,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":0.89200000000000002},{"interface":"front","average":{"1min":1.325,"5min":1.4059999999999999,"15min":1.4199999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6930000000000001,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":0.873}]}]}],"pool_statfs":[]}} 2023-12-19T18:17:19.269 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:17:19.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.542+0000 7f3ab9f8a640 1 Processor -- start 2023-12-19T18:17:19.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.542+0000 7f3ab9f8a640 1 -- start start 2023-12-19T18:17:19.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.543+0000 7f3ab9f8a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3ab41553a0 0x7f3ab40c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.543+0000 7f3ab9f8a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab40b0360 con 0x7f3ab40afeb0 2023-12-19T18:17:19.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.543+0000 7f3ab9f8a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ab40cb350 con 0x7f3ab415ee30 2023-12-19T18:17:19.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.544+0000 7f3ab9f8a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ab40cbcf0 con 0x7f3ab41553a0 2023-12-19T18:17:19.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.544+0000 7f3ab3fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3ab40afeb0 0x7f3ab415e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41802/0 (socket says 172.21.15.27:41802) 2023-12-19T18:17:19.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.544+0000 7f3ab3fff640 1 -- 172.21.15.27:0/2380789671 learned_addr learned my addr 172.21.15.27:0/2380789671 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:19.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.544+0000 7f3ab8f88640 1 --2- 172.21.15.27:0/2380789671 >> v2:172.21.15.192:3300/0 conn(0x7f3ab41553a0 0x7f3ab40c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.544+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14916822 0 0) 0x7f3ab40b0360 con 0x7f3ab40afeb0 2023-12-19T18:17:19.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.544+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a9c003520 con 0x7f3ab40afeb0 2023-12-19T18:17:19.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 654751887 0 0) 0x7f3a9c003520 con 0x7f3ab40afeb0 2023-12-19T18:17:19.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 >> v2:172.21.15.192:3300/0 conn(0x7f3ab41553a0 msgr2=0x7f3ab40c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:19.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab37fe640 1 --2- 172.21.15.27:0/2380789671 >> v2:172.21.15.192:3300/0 conn(0x7f3ab41553a0 0x7f3ab40c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 >> v1:172.21.15.192:6789/0 conn(0x7f3ab415ee30 legacy=0x7f3ab40c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab40cb630 con 0x7f3ab40afeb0 2023-12-19T18:17:19.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f3aa80019f0 con 0x7f3ab40afeb0 2023-12-19T18:17:19.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 >> v1:172.21.15.27:6789/0 conn(0x7f3ab40afeb0 legacy=0x7f3ab415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab37fe640 1 --2- 172.21.15.27:0/2380789671 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a9c0039c0 0x7f3a9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab37fe640 1 --2- 172.21.15.27:0/2380789671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a9c0047c0 0x7f3a9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab37fe640 1 --2- 172.21.15.27:0/2380789671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0051d0 0x7f3a9c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab40cbcf0 con 0x7f3a9c0051d0 2023-12-19T18:17:19.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c00c120 con 0x7f3a9c0047c0 2023-12-19T18:17:19.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a9c00c2e0 con 0x7f3a9c0039c0 2023-12-19T18:17:19.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab9789640 1 --2- 172.21.15.27:0/2380789671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0051d0 0x7f3a9c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.545+0000 7f3ab8f88640 1 --2- 172.21.15.27:0/2380789671 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a9c0039c0 0x7f3a9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.546+0000 7f3ab3fff640 1 --2- 172.21.15.27:0/2380789671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a9c0047c0 0x7f3a9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.546+0000 7f3ab9789640 1 -- 172.21.15.27:0/2380789671 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a9c0039c0 msgr2=0x7f3a9c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:19.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.546+0000 7f3ab9789640 1 --2- 172.21.15.27:0/2380789671 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3a9c0039c0 0x7f3a9c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.546+0000 7f3ab9789640 1 -- 172.21.15.27:0/2380789671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a9c0047c0 msgr2=0x7f3a9c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:19.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.546+0000 7f3ab9789640 1 --2- 172.21.15.27:0/2380789671 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3a9c0047c0 0x7f3a9c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.546+0000 7f3ab9789640 1 -- 172.21.15.27:0/2380789671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a9c003520 con 0x7f3a9c0051d0 2023-12-19T18:17:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.547+0000 7f3ab9789640 1 -- 172.21.15.27:0/2380789671 reap_dead start 2023-12-19T18:17:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.547+0000 7f3ab9789640 1 --2- 172.21.15.27:0/2380789671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0051d0 0x7f3a9c00b980 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3aac001720 tx=0x7f3a9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded147eb004f7ff9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.547+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0052d0 con 0x7f3a9c0051d0 2023-12-19T18:17:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.547+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aac02e050 con 0x7f3a9c0051d0 2023-12-19T18:17:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.547+0000 7f3ab37fe640 1 -- 172.21.15.27:0/2380789671 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aac0035f0 con 0x7f3a9c0051d0 2023-12-19T18:17:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.548+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2380789671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0051d0 msgr2=0x7f3a9c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.548+0000 7f3ab9f8a640 1 --2- 172.21.15.27:0/2380789671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0051d0 0x7f3a9c00b980 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f3aac001720 tx=0x7f3a9c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.548+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2380789671 shutdown_connections 2023-12-19T18:17:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.548+0000 7f3ab9f8a640 1 --2- 172.21.15.27:0/2380789671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0051d0 0x7f3a9c00b980 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.548+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2380789671 >> 172.21.15.27:0/2380789671 conn(0x7f3ab4069c70 msgr2=0x7f3ab406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.548+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2380789671 shutdown_connections 2023-12-19T18:17:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.548+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2380789671 wait complete. 2023-12-19T18:17:19.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.550+0000 7f3ab9f8a640 1 Processor -- start 2023-12-19T18:17:19.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.550+0000 7f3ab9f8a640 1 -- start start 2023-12-19T18:17:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.551+0000 7f3ab9f8a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0051d0 0x7f3ab40c8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.551+0000 7f3ab9f8a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ab4159e30 0x7f3ab41ec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.551+0000 7f3ab9f8a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ab415a2b0 0x7f3ab41ecb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.551+0000 7f3ab9f8a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab40c6390 con 0x7f3a9c0051d0 2023-12-19T18:17:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.551+0000 7f3ab9f8a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab40c7780 con 0x7f3ab415a2b0 2023-12-19T18:17:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.551+0000 7f3ab9f8a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab41f3ab0 con 0x7f3ab4159e30 2023-12-19T18:17:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.551+0000 7f3ab8f88640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0051d0 0x7f3ab40c8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.551+0000 7f3ab8f88640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0051d0 0x7f3ab40c8490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34150/0 (socket says 172.21.15.27:34150) 2023-12-19T18:17:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.551+0000 7f3ab8f88640 1 -- 172.21.15.27:0/2363267793 learned_addr learned my addr 172.21.15.27:0/2363267793 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.552+0000 7f3ab9789640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ab415a2b0 0x7f3ab41ecb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.552+0000 7f3ab3fff640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ab4159e30 0x7f3ab41ec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.552+0000 7f3ab8f88640 1 -- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ab4159e30 msgr2=0x7f3ab41ec5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.552+0000 7f3ab8f88640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ab4159e30 0x7f3ab41ec5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.552+0000 7f3ab8f88640 1 -- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ab415a2b0 msgr2=0x7f3ab41ecb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.552+0000 7f3ab8f88640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ab415a2b0 0x7f3ab41ecb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.552+0000 7f3ab8f88640 1 -- 172.21.15.27:0/2363267793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab41f37d0 con 0x7f3a9c0051d0 2023-12-19T18:17:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.552+0000 7f3ab8f88640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0051d0 0x7f3ab40c8490 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4011540 tx=0x7f3aa4011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.552+0000 7f3ab17fa640 1 -- 172.21.15.27:0/2363267793 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4015930 con 0x7f3a9c0051d0 2023-12-19T18:17:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.553+0000 7f3ab17fa640 1 -- 172.21.15.27:0/2363267793 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3aa4015a90 con 0x7f3a9c0051d0 2023-12-19T18:17:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.553+0000 7f3ab17fa640 1 -- 172.21.15.27:0/2363267793 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3aa4015e60 con 0x7f3a9c0051d0 2023-12-19T18:17:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.553+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2363267793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab40c79b0 con 0x7f3a9c0051d0 2023-12-19T18:17:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.553+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2363267793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab40c7f70 con 0x7f3a9c0051d0 2023-12-19T18:17:19.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.554+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2363267793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab40c8100 con 0x7f3a9c0051d0 2023-12-19T18:17:19.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.555+0000 7f3ab17fa640 1 -- 172.21.15.27:0/2363267793 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3aa400f460 con 0x7f3a9c0051d0 2023-12-19T18:17:19.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.556+0000 7f3ab17fa640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3aa006a430 0x7f3aa006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:19.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.556+0000 7f3ab17fa640 1 -- 172.21.15.27:0/2363267793 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3aa4089140 con 0x7f3a9c0051d0 2023-12-19T18:17:19.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.557+0000 7f3ab3fff640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3aa006a430 0x7f3aa006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.558+0000 7f3ab3fff640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3aa006a430 0x7f3aa006c8f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8002230 tx=0x7f3aa8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:19.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.560+0000 7f3ab17fa640 1 -- 172.21.15.27:0/2363267793 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3aa4054730 con 0x7f3a9c0051d0 2023-12-19T18:17:19.718 INFO:tasks.ceph.mds.c.smithi027.stderr:2023-12-19T18:17:19.717+0000 7f60cd376640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:17:19.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.754+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2363267793 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ab40c5730 con 0x7f3aa006a430 2023-12-19T18:17:19.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.778+0000 7f3ab17fa640 1 -- 172.21.15.27:0/2363267793 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282570 (secure 0 0 0) 0x7f3aa8005d30 con 0x7f3aa006a430 2023-12-19T18:17:19.781 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:19.783 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:17:19.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.783+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3aa006a430 msgr2=0x7f3aa006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.783+0000 7f3ab9f8a640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3aa006a430 0x7f3aa006c8f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f3aa8002230 tx=0x7f3aa8009040 comp rx=0 tx=0).stop 2023-12-19T18:17:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.783+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2363267793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0051d0 msgr2=0x7f3ab40c8490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.783+0000 7f3ab9f8a640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0051d0 0x7f3ab40c8490 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4011540 tx=0x7f3aa4011a00 comp rx=0 tx=0).stop 2023-12-19T18:17:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.783+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2363267793 shutdown_connections 2023-12-19T18:17:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.783+0000 7f3ab9f8a640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3aa006a430 0x7f3aa006c8f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.783+0000 7f3ab9f8a640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3ab415a2b0 0x7f3ab41ecb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.783+0000 7f3ab9f8a640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3ab4159e30 0x7f3ab41ec5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.783+0000 7f3ab9f8a640 1 --2- 172.21.15.27:0/2363267793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a9c0051d0 0x7f3ab40c8490 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.783+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2363267793 >> 172.21.15.27:0/2363267793 conn(0x7f3ab4069c70 msgr2=0x7f3ab4152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.783+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2363267793 shutdown_connections 2023-12-19T18:17:19.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:19.783+0000 7f3ab9f8a640 1 -- 172.21.15.27:0/2363267793 wait complete. 2023-12-19T18:17:19.805 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2023-12-19T18:17:19.753976+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":3066,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1559433,"statfs":{"total":1600000000,"available":1596860172,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.008444"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.823338+0000","last_change":"2023-12-19T18:17:14.270791+0000","last_active":"2023-12-19T18:17:15.823338+0000","last_peered":"2023-12-19T18:17:15.823338+0000","last_clean":"2023-12-19T18:17:15.823338+0000","last_became_active":"2023-12-19T18:17:14.270264+0000","last_became_peered":"2023-12-19T18:17:14.270264+0000","last_unstale":"2023-12-19T18:17:15.823338+0000","last_undegraded":"2023-12-19T18:17:15.823338+0000","last_fullsized":"2023-12-19T18:17:15.823338+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:40.042526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.668178+0000","last_change":"2023-12-19T18:17:14.283814+0000","last_active":"2023-12-19T18:17:15.668178+0000","last_peered":"2023-12-19T18:17:15.668178+0000","last_clean":"2023-12-19T18:17:15.668178+0000","last_became_active":"2023-12-19T18:17:14.283555+0000","last_became_peered":"2023-12-19T18:17:14.283555+0000","last_unstale":"2023-12-19T18:17:15.668178+0000","last_undegraded":"2023-12-19T18:17:15.668178+0000","last_fullsized":"2023-12-19T18:17:15.668178+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:18.819941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.268548+0000","last_change":"2023-12-19T18:17:14.283205+0000","last_active":"2023-12-19T18:17:15.268548+0000","last_peered":"2023-12-19T18:17:15.268548+0000","last_clean":"2023-12-19T18:17:15.268548+0000","last_became_active":"2023-12-19T18:17:14.282970+0000","last_became_peered":"2023-12-19T18:17:14.282970+0000","last_unstale":"2023-12-19T18:17:15.268548+0000","last_undegraded":"2023-12-19T18:17:15.268548+0000","last_fullsized":"2023-12-19T18:17:15.268548+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:14:16.979907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.322468+0000","last_change":"2023-12-19T18:17:14.282796+0000","last_active":"2023-12-19T18:17:15.322468+0000","last_peered":"2023-12-19T18:17:15.322468+0000","last_clean":"2023-12-19T18:17:15.322468+0000","last_became_active":"2023-12-19T18:17:14.277828+0000","last_became_peered":"2023-12-19T18:17:14.277828+0000","last_unstale":"2023-12-19T18:17:15.322468+0000","last_undegraded":"2023-12-19T18:17:15.322468+0000","last_fullsized":"2023-12-19T18:17:15.322468+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:53:07.633993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.823266+0000","last_change":"2023-12-19T18:17:14.282290+0000","last_active":"2023-12-19T18:17:15.823266+0000","last_peered":"2023-12-19T18:17:15.823266+0000","last_clean":"2023-12-19T18:17:15.823266+0000","last_became_active":"2023-12-19T18:17:14.282114+0000","last_became_peered":"2023-12-19T18:17:14.282114+0000","last_unstale":"2023-12-19T18:17:15.823266+0000","last_undegraded":"2023-12-19T18:17:15.823266+0000","last_fullsized":"2023-12-19T18:17:15.823266+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:18.819941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.669740+0000","last_change":"2023-12-19T18:17:14.287266+0000","last_active":"2023-12-19T18:17:15.669740+0000","last_peered":"2023-12-19T18:17:15.669740+0000","last_clean":"2023-12-19T18:17:15.669740+0000","last_became_active":"2023-12-19T18:17:14.286592+0000","last_became_peered":"2023-12-19T18:17:14.286592+0000","last_unstale":"2023-12-19T18:17:15.669740+0000","last_undegraded":"2023-12-19T18:17:15.669740+0000","last_fullsized":"2023-12-19T18:17:15.669740+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:41:07.539062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.271918+0000","last_change":"2023-12-19T18:17:14.288164+0000","last_active":"2023-12-19T18:17:15.271918+0000","last_peered":"2023-12-19T18:17:15.271918+0000","last_clean":"2023-12-19T18:17:15.271918+0000","last_became_active":"2023-12-19T18:17:14.287921+0000","last_became_peered":"2023-12-19T18:17:14.287921+0000","last_unstale":"2023-12-19T18:17:15.271918+0000","last_undegraded":"2023-12-19T18:17:15.271918+0000","last_fullsized":"2023-12-19T18:17:15.271918+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:12:14.448174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.321941+0000","last_change":"2023-12-19T18:17:14.271708+0000","last_active":"2023-12-19T18:17:15.321941+0000","last_peered":"2023-12-19T18:17:15.321941+0000","last_clean":"2023-12-19T18:17:15.321941+0000","last_became_active":"2023-12-19T18:17:14.270235+0000","last_became_peered":"2023-12-19T18:17:14.270235+0000","last_unstale":"2023-12-19T18:17:15.321941+0000","last_undegraded":"2023-12-19T18:17:15.321941+0000","last_fullsized":"2023-12-19T18:17:15.321941+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:22.849008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.669207+0000","last_change":"2023-12-19T18:17:14.292883+0000","last_active":"2023-12-19T18:17:15.669207+0000","last_peered":"2023-12-19T18:17:15.669207+0000","last_clean":"2023-12-19T18:17:15.669207+0000","last_became_active":"2023-12-19T18:17:14.292499+0000","last_became_peered":"2023-12-19T18:17:14.292499+0000","last_unstale":"2023-12-19T18:17:15.669207+0000","last_undegraded":"2023-12-19T18:17:15.669207+0000","last_fullsized":"2023-12-19T18:17:15.669207+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:00:54.488849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.273210+0000","last_change":"2023-12-19T18:17:14.289806+0000","last_active":"2023-12-19T18:17:15.273210+0000","last_peered":"2023-12-19T18:17:15.273210+0000","last_clean":"2023-12-19T18:17:15.273210+0000","last_became_active":"2023-12-19T18:17:14.289104+0000","last_became_peered":"2023-12-19T18:17:14.289104+0000","last_unstale":"2023-12-19T18:17:15.273210+0000","last_undegraded":"2023-12-19T18:17:15.273210+0000","last_fullsized":"2023-12-19T18:17:15.273210+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:48.461208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.322420+0000","last_change":"2023-12-19T18:17:14.287998+0000","last_active":"2023-12-19T18:17:15.322420+0000","last_peered":"2023-12-19T18:17:15.322420+0000","last_clean":"2023-12-19T18:17:15.322420+0000","last_became_active":"2023-12-19T18:17:14.286039+0000","last_became_peered":"2023-12-19T18:17:14.286039+0000","last_unstale":"2023-12-19T18:17:15.322420+0000","last_undegraded":"2023-12-19T18:17:15.322420+0000","last_fullsized":"2023-12-19T18:17:15.322420+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:48.461208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.668423+0000","last_change":"2023-12-19T18:17:14.281664+0000","last_active":"2023-12-19T18:17:15.668423+0000","last_peered":"2023-12-19T18:17:15.668423+0000","last_clean":"2023-12-19T18:17:15.668423+0000","last_became_active":"2023-12-19T18:17:14.281484+0000","last_became_peered":"2023-12-19T18:17:14.281484+0000","last_unstale":"2023-12-19T18:17:15.668423+0000","last_undegraded":"2023-12-19T18:17:15.668423+0000","last_fullsized":"2023-12-19T18:17:15.668423+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:42:35.093685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.668885+0000","last_change":"2023-12-19T18:17:14.286089+0000","last_active":"2023-12-19T18:17:15.668885+0000","last_peered":"2023-12-19T18:17:15.668885+0000","last_clean":"2023-12-19T18:17:15.668885+0000","last_became_active":"2023-12-19T18:17:14.285866+0000","last_became_peered":"2023-12-19T18:17:14.285866+0000","last_unstale":"2023-12-19T18:17:15.668885+0000","last_undegraded":"2023-12-19T18:17:15.668885+0000","last_fullsized":"2023-12-19T18:17:15.668885+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:58:13.463770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.323066+0000","last_change":"2023-12-19T18:17:14.290774+0000","last_active":"2023-12-19T18:17:15.323066+0000","last_peered":"2023-12-19T18:17:15.323066+0000","last_clean":"2023-12-19T18:17:15.323066+0000","last_became_active":"2023-12-19T18:17:14.290387+0000","last_became_peered":"2023-12-19T18:17:14.290387+0000","last_unstale":"2023-12-19T18:17:15.323066+0000","last_undegraded":"2023-12-19T18:17:15.323066+0000","last_fullsized":"2023-12-19T18:17:15.323066+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:44.539319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.321904+0000","last_change":"2023-12-19T18:17:14.269222+0000","last_active":"2023-12-19T18:17:15.321904+0000","last_peered":"2023-12-19T18:17:15.321904+0000","last_clean":"2023-12-19T18:17:15.321904+0000","last_became_active":"2023-12-19T18:17:14.268515+0000","last_became_peered":"2023-12-19T18:17:14.268515+0000","last_unstale":"2023-12-19T18:17:15.321904+0000","last_undegraded":"2023-12-19T18:17:15.321904+0000","last_fullsized":"2023-12-19T18:17:15.321904+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:06.887188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.823767+0000","last_change":"2023-12-19T18:17:14.278868+0000","last_active":"2023-12-19T18:17:15.823767+0000","last_peered":"2023-12-19T18:17:15.823767+0000","last_clean":"2023-12-19T18:17:15.823767+0000","last_became_active":"2023-12-19T18:17:14.278634+0000","last_became_peered":"2023-12-19T18:17:14.278634+0000","last_unstale":"2023-12-19T18:17:15.823767+0000","last_undegraded":"2023-12-19T18:17:15.823767+0000","last_fullsized":"2023-12-19T18:17:15.823767+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:37.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.668620+0000","last_change":"2023-12-19T18:17:14.269997+0000","last_active":"2023-12-19T18:17:15.668620+0000","last_peered":"2023-12-19T18:17:15.668620+0000","last_clean":"2023-12-19T18:17:15.668620+0000","last_became_active":"2023-12-19T18:17:14.269630+0000","last_became_peered":"2023-12-19T18:17:14.269630+0000","last_unstale":"2023-12-19T18:17:15.668620+0000","last_undegraded":"2023-12-19T18:17:15.668620+0000","last_fullsized":"2023-12-19T18:17:15.668620+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:40.042526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.268357+0000","last_change":"2023-12-19T18:17:14.277161+0000","last_active":"2023-12-19T18:17:15.268357+0000","last_peered":"2023-12-19T18:17:15.268357+0000","last_clean":"2023-12-19T18:17:15.268357+0000","last_became_active":"2023-12-19T18:17:14.276915+0000","last_became_peered":"2023-12-19T18:17:14.276915+0000","last_unstale":"2023-12-19T18:17:15.268357+0000","last_undegraded":"2023-12-19T18:17:15.268357+0000","last_fullsized":"2023-12-19T18:17:15.268357+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:40.776126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.823448+0000","last_change":"2023-12-19T18:17:14.285138+0000","last_active":"2023-12-19T18:17:15.823448+0000","last_peered":"2023-12-19T18:17:15.823448+0000","last_clean":"2023-12-19T18:17:15.823448+0000","last_became_active":"2023-12-19T18:17:14.284907+0000","last_became_peered":"2023-12-19T18:17:14.284907+0000","last_unstale":"2023-12-19T18:17:15.823448+0000","last_undegraded":"2023-12-19T18:17:15.823448+0000","last_fullsized":"2023-12-19T18:17:15.823448+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:11:38.937188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.273798+0000","last_change":"2023-12-19T18:17:14.289654+0000","last_active":"2023-12-19T18:17:15.273798+0000","last_peered":"2023-12-19T18:17:15.273798+0000","last_clean":"2023-12-19T18:17:15.273798+0000","last_became_active":"2023-12-19T18:17:14.288800+0000","last_became_peered":"2023-12-19T18:17:14.288800+0000","last_unstale":"2023-12-19T18:17:15.273798+0000","last_undegraded":"2023-12-19T18:17:15.273798+0000","last_fullsized":"2023-12-19T18:17:15.273798+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:12:45.339741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.323320+0000","last_change":"2023-12-19T18:17:14.279298+0000","last_active":"2023-12-19T18:17:15.323320+0000","last_peered":"2023-12-19T18:17:15.323320+0000","last_clean":"2023-12-19T18:17:15.323320+0000","last_became_active":"2023-12-19T18:17:14.273615+0000","last_became_peered":"2023-12-19T18:17:14.273615+0000","last_unstale":"2023-12-19T18:17:15.323320+0000","last_undegraded":"2023-12-19T18:17:15.323320+0000","last_fullsized":"2023-12-19T18:17:15.323320+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:42:54.135928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.670115+0000","last_change":"2023-12-19T18:17:14.281999+0000","last_active":"2023-12-19T18:17:15.670115+0000","last_peered":"2023-12-19T18:17:15.670115+0000","last_clean":"2023-12-19T18:17:15.670115+0000","last_became_active":"2023-12-19T18:17:14.280543+0000","last_became_peered":"2023-12-19T18:17:14.280543+0000","last_unstale":"2023-12-19T18:17:15.670115+0000","last_undegraded":"2023-12-19T18:17:15.670115+0000","last_fullsized":"2023-12-19T18:17:15.670115+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:48.064458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.272522+0000","last_change":"2023-12-19T18:17:14.276400+0000","last_active":"2023-12-19T18:17:15.272522+0000","last_peered":"2023-12-19T18:17:15.272522+0000","last_clean":"2023-12-19T18:17:15.272522+0000","last_became_active":"2023-12-19T18:17:14.276192+0000","last_became_peered":"2023-12-19T18:17:14.276192+0000","last_unstale":"2023-12-19T18:17:15.272522+0000","last_undegraded":"2023-12-19T18:17:15.272522+0000","last_fullsized":"2023-12-19T18:17:15.272522+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:40:04.985616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.667727+0000","last_change":"2023-12-19T18:17:14.268347+0000","last_active":"2023-12-19T18:17:15.667727+0000","last_peered":"2023-12-19T18:17:15.667727+0000","last_clean":"2023-12-19T18:17:15.667727+0000","last_became_active":"2023-12-19T18:17:14.268126+0000","last_became_peered":"2023-12-19T18:17:14.268126+0000","last_unstale":"2023-12-19T18:17:15.667727+0000","last_undegraded":"2023-12-19T18:17:15.667727+0000","last_fullsized":"2023-12-19T18:17:15.667727+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:01:50.547764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.823414+0000","last_change":"2023-12-19T18:17:14.272506+0000","last_active":"2023-12-19T18:17:15.823414+0000","last_peered":"2023-12-19T18:17:15.823414+0000","last_clean":"2023-12-19T18:17:15.823414+0000","last_became_active":"2023-12-19T18:17:14.272269+0000","last_became_peered":"2023-12-19T18:17:14.272269+0000","last_unstale":"2023-12-19T18:17:15.823414+0000","last_undegraded":"2023-12-19T18:17:15.823414+0000","last_fullsized":"2023-12-19T18:17:15.823414+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:26:50.243947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.322655+0000","last_change":"2023-12-19T18:17:14.284214+0000","last_active":"2023-12-19T18:17:15.322655+0000","last_peered":"2023-12-19T18:17:15.322655+0000","last_clean":"2023-12-19T18:17:15.322655+0000","last_became_active":"2023-12-19T18:17:14.281746+0000","last_became_peered":"2023-12-19T18:17:14.281746+0000","last_unstale":"2023-12-19T18:17:15.322655+0000","last_undegraded":"2023-12-19T18:17:15.322655+0000","last_fullsized":"2023-12-19T18:17:15.322655+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:25:50.158021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.272082+0000","last_change":"2023-12-19T18:17:14.278469+0000","last_active":"2023-12-19T18:17:15.272082+0000","last_peered":"2023-12-19T18:17:15.272082+0000","last_clean":"2023-12-19T18:17:15.272082+0000","last_became_active":"2023-12-19T18:17:14.277958+0000","last_became_peered":"2023-12-19T18:17:14.277958+0000","last_unstale":"2023-12-19T18:17:15.272082+0000","last_undegraded":"2023-12-19T18:17:15.272082+0000","last_fullsized":"2023-12-19T18:17:15.272082+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:08:08.399481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.272011+0000","last_change":"2023-12-19T18:17:14.274961+0000","last_active":"2023-12-19T18:17:15.272011+0000","last_peered":"2023-12-19T18:17:15.272011+0000","last_clean":"2023-12-19T18:17:15.272011+0000","last_became_active":"2023-12-19T18:17:14.274725+0000","last_became_peered":"2023-12-19T18:17:14.274725+0000","last_unstale":"2023-12-19T18:17:15.272011+0000","last_undegraded":"2023-12-19T18:17:15.272011+0000","last_fullsized":"2023-12-19T18:17:15.272011+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:47:59.752575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.822979+0000","last_change":"2023-12-19T18:17:14.287574+0000","last_active":"2023-12-19T18:17:15.822979+0000","last_peered":"2023-12-19T18:17:15.822979+0000","last_clean":"2023-12-19T18:17:15.822979+0000","last_became_active":"2023-12-19T18:17:14.287304+0000","last_became_peered":"2023-12-19T18:17:14.287304+0000","last_unstale":"2023-12-19T18:17:15.822979+0000","last_undegraded":"2023-12-19T18:17:15.822979+0000","last_fullsized":"2023-12-19T18:17:15.822979+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:42:35.093685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.668366+0000","last_change":"2023-12-19T18:17:14.277685+0000","last_active":"2023-12-19T18:17:15.668366+0000","last_peered":"2023-12-19T18:17:15.668366+0000","last_clean":"2023-12-19T18:17:15.668366+0000","last_became_active":"2023-12-19T18:17:14.277433+0000","last_became_peered":"2023-12-19T18:17:14.277433+0000","last_unstale":"2023-12-19T18:17:15.668366+0000","last_undegraded":"2023-12-19T18:17:15.668366+0000","last_fullsized":"2023-12-19T18:17:15.668366+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:37.843081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.669992+0000","last_change":"2023-12-19T18:17:14.278398+0000","last_active":"2023-12-19T18:17:15.669992+0000","last_peered":"2023-12-19T18:17:15.669992+0000","last_clean":"2023-12-19T18:17:15.669992+0000","last_became_active":"2023-12-19T18:17:14.278071+0000","last_became_peered":"2023-12-19T18:17:14.278071+0000","last_unstale":"2023-12-19T18:17:15.669992+0000","last_undegraded":"2023-12-19T18:17:15.669992+0000","last_fullsized":"2023-12-19T18:17:15.669992+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:16.338760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.322681+0000","last_change":"2023-12-19T18:17:14.287777+0000","last_active":"2023-12-19T18:17:15.322681+0000","last_peered":"2023-12-19T18:17:15.322681+0000","last_clean":"2023-12-19T18:17:15.322681+0000","last_became_active":"2023-12-19T18:17:14.286007+0000","last_became_peered":"2023-12-19T18:17:14.286007+0000","last_unstale":"2023-12-19T18:17:15.322681+0000","last_undegraded":"2023-12-19T18:17:15.322681+0000","last_fullsized":"2023-12-19T18:17:15.322681+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:45:05.976616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.668884+0000","last_change":"2023-12-19T18:17:14.288580+0000","last_active":"2023-12-19T18:17:15.668884+0000","last_peered":"2023-12-19T18:17:15.668884+0000","last_clean":"2023-12-19T18:17:15.668884+0000","last_became_active":"2023-12-19T18:17:14.288016+0000","last_became_peered":"2023-12-19T18:17:14.288016+0000","last_unstale":"2023-12-19T18:17:15.668884+0000","last_undegraded":"2023-12-19T18:17:15.668884+0000","last_fullsized":"2023-12-19T18:17:15.668884+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:57:36.030970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"49'4","reported_seq":15,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.288828+0000","last_change":"2023-12-19T18:17:13.774067+0000","last_active":"2023-12-19T18:17:14.288828+0000","last_peered":"2023-12-19T18:17:14.288828+0000","last_clean":"2023-12-19T18:17:14.288828+0000","last_became_active":"2023-12-19T18:17:13.773621+0000","last_became_peered":"2023-12-19T18:17:13.773621+0000","last_unstale":"2023-12-19T18:17:14.288828+0000","last_undegraded":"2023-12-19T18:17:14.288828+0000","last_fullsized":"2023-12-19T18:17:14.288828+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:10.250166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.823769+0000","last_change":"2023-12-19T18:17:14.286530+0000","last_active":"2023-12-19T18:17:15.823769+0000","last_peered":"2023-12-19T18:17:15.823769+0000","last_clean":"2023-12-19T18:17:15.823769+0000","last_became_active":"2023-12-19T18:17:14.285903+0000","last_became_peered":"2023-12-19T18:17:14.285903+0000","last_unstale":"2023-12-19T18:17:15.823769+0000","last_undegraded":"2023-12-19T18:17:15.823769+0000","last_fullsized":"2023-12-19T18:17:15.823769+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:05.627747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"49'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.277658+0000","last_change":"2023-12-19T18:17:13.763824+0000","last_active":"2023-12-19T18:17:14.277658+0000","last_peered":"2023-12-19T18:17:14.277658+0000","last_clean":"2023-12-19T18:17:14.277658+0000","last_became_active":"2023-12-19T18:17:13.763534+0000","last_became_peered":"2023-12-19T18:17:13.763534+0000","last_unstale":"2023-12-19T18:17:14.277658+0000","last_undegraded":"2023-12-19T18:17:14.277658+0000","last_fullsized":"2023-12-19T18:17:14.277658+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:52:34.850640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.667912+0000","last_change":"2023-12-19T18:17:14.283540+0000","last_active":"2023-12-19T18:17:15.667912+0000","last_peered":"2023-12-19T18:17:15.667912+0000","last_clean":"2023-12-19T18:17:15.667912+0000","last_became_active":"2023-12-19T18:17:14.283151+0000","last_became_peered":"2023-12-19T18:17:14.283151+0000","last_unstale":"2023-12-19T18:17:15.667912+0000","last_undegraded":"2023-12-19T18:17:15.667912+0000","last_fullsized":"2023-12-19T18:17:15.667912+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:11:38.937188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"49'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.282182+0000","last_change":"2023-12-19T18:17:13.772634+0000","last_active":"2023-12-19T18:17:14.282182+0000","last_peered":"2023-12-19T18:17:14.282182+0000","last_clean":"2023-12-19T18:17:14.282182+0000","last_became_active":"2023-12-19T18:17:13.772252+0000","last_became_peered":"2023-12-19T18:17:13.772252+0000","last_unstale":"2023-12-19T18:17:14.282182+0000","last_undegraded":"2023-12-19T18:17:14.282182+0000","last_fullsized":"2023-12-19T18:17:14.282182+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:37:11.023952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.322068+0000","last_change":"2023-12-19T18:17:14.282663+0000","last_active":"2023-12-19T18:17:15.322068+0000","last_peered":"2023-12-19T18:17:15.322068+0000","last_clean":"2023-12-19T18:17:15.322068+0000","last_became_active":"2023-12-19T18:17:14.280528+0000","last_became_peered":"2023-12-19T18:17:14.280528+0000","last_unstale":"2023-12-19T18:17:15.322068+0000","last_undegraded":"2023-12-19T18:17:15.322068+0000","last_fullsized":"2023-12-19T18:17:15.322068+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:12:43.364068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"49'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.278236+0000","last_change":"2023-12-19T18:17:13.763934+0000","last_active":"2023-12-19T18:17:14.278236+0000","last_peered":"2023-12-19T18:17:14.278236+0000","last_clean":"2023-12-19T18:17:14.278236+0000","last_became_active":"2023-12-19T18:17:13.763677+0000","last_became_peered":"2023-12-19T18:17:13.763677+0000","last_unstale":"2023-12-19T18:17:14.278236+0000","last_undegraded":"2023-12-19T18:17:14.278236+0000","last_fullsized":"2023-12-19T18:17:14.278236+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:33.389677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.272194+0000","last_change":"2023-12-19T18:17:14.278358+0000","last_active":"2023-12-19T18:17:15.272194+0000","last_peered":"2023-12-19T18:17:15.272194+0000","last_clean":"2023-12-19T18:17:15.272194+0000","last_became_active":"2023-12-19T18:17:14.277709+0000","last_became_peered":"2023-12-19T18:17:14.277709+0000","last_unstale":"2023-12-19T18:17:15.272194+0000","last_undegraded":"2023-12-19T18:17:15.272194+0000","last_fullsized":"2023-12-19T18:17:15.272194+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:32:36.563620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"49'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.284931+0000","last_change":"2023-12-19T18:17:13.772384+0000","last_active":"2023-12-19T18:17:14.284931+0000","last_peered":"2023-12-19T18:17:14.284931+0000","last_clean":"2023-12-19T18:17:14.284931+0000","last_became_active":"2023-12-19T18:17:13.771801+0000","last_became_peered":"2023-12-19T18:17:13.771801+0000","last_unstale":"2023-12-19T18:17:14.284931+0000","last_undegraded":"2023-12-19T18:17:14.284931+0000","last_fullsized":"2023-12-19T18:17:14.284931+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:57:35.015037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.272977+0000","last_change":"2023-12-19T18:17:14.292065+0000","last_active":"2023-12-19T18:17:15.272977+0000","last_peered":"2023-12-19T18:17:15.272977+0000","last_clean":"2023-12-19T18:17:15.272977+0000","last_became_active":"2023-12-19T18:17:14.291657+0000","last_became_peered":"2023-12-19T18:17:14.291657+0000","last_unstale":"2023-12-19T18:17:15.272977+0000","last_undegraded":"2023-12-19T18:17:15.272977+0000","last_fullsized":"2023-12-19T18:17:15.272977+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:53:07.633993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"49'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.275897+0000","last_change":"2023-12-19T18:17:13.764169+0000","last_active":"2023-12-19T18:17:14.275897+0000","last_peered":"2023-12-19T18:17:14.275897+0000","last_clean":"2023-12-19T18:17:14.275897+0000","last_became_active":"2023-12-19T18:17:13.763875+0000","last_became_peered":"2023-12-19T18:17:13.763875+0000","last_unstale":"2023-12-19T18:17:14.275897+0000","last_undegraded":"2023-12-19T18:17:14.275897+0000","last_fullsized":"2023-12-19T18:17:14.275897+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:21.833075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.273474+0000","last_change":"2023-12-19T18:17:14.289498+0000","last_active":"2023-12-19T18:17:15.273474+0000","last_peered":"2023-12-19T18:17:15.273474+0000","last_clean":"2023-12-19T18:17:15.273474+0000","last_became_active":"2023-12-19T18:17:14.288444+0000","last_became_peered":"2023-12-19T18:17:14.288444+0000","last_unstale":"2023-12-19T18:17:15.273474+0000","last_undegraded":"2023-12-19T18:17:15.273474+0000","last_fullsized":"2023-12-19T18:17:15.273474+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:06.887188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.265893+0000","last_change":"2023-12-19T18:17:13.756673+0000","last_active":"2023-12-19T18:17:14.265893+0000","last_peered":"2023-12-19T18:17:14.265893+0000","last_clean":"2023-12-19T18:17:14.265893+0000","last_became_active":"2023-12-19T18:17:13.756255+0000","last_became_peered":"2023-12-19T18:17:13.756255+0000","last_unstale":"2023-12-19T18:17:14.265893+0000","last_undegraded":"2023-12-19T18:17:14.265893+0000","last_fullsized":"2023-12-19T18:17:14.265893+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:54:51.030454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.271873+0000","last_change":"2023-12-19T18:17:14.285471+0000","last_active":"2023-12-19T18:17:15.271873+0000","last_peered":"2023-12-19T18:17:15.271873+0000","last_clean":"2023-12-19T18:17:15.271873+0000","last_became_active":"2023-12-19T18:17:14.285101+0000","last_became_peered":"2023-12-19T18:17:14.285101+0000","last_unstale":"2023-12-19T18:17:15.271873+0000","last_undegraded":"2023-12-19T18:17:15.271873+0000","last_fullsized":"2023-12-19T18:17:15.271873+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:57.225554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.285078+0000","last_change":"2023-12-19T18:17:13.769856+0000","last_active":"2023-12-19T18:17:14.285078+0000","last_peered":"2023-12-19T18:17:14.285078+0000","last_clean":"2023-12-19T18:17:14.285078+0000","last_became_active":"2023-12-19T18:17:13.768768+0000","last_became_peered":"2023-12-19T18:17:13.768768+0000","last_unstale":"2023-12-19T18:17:14.285078+0000","last_undegraded":"2023-12-19T18:17:14.285078+0000","last_fullsized":"2023-12-19T18:17:14.285078+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:58:12.447837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.268599+0000","last_change":"2023-12-19T18:17:14.280093+0000","last_active":"2023-12-19T18:17:15.268599+0000","last_peered":"2023-12-19T18:17:15.268599+0000","last_clean":"2023-12-19T18:17:15.268599+0000","last_became_active":"2023-12-19T18:17:14.279876+0000","last_became_peered":"2023-12-19T18:17:14.279876+0000","last_unstale":"2023-12-19T18:17:15.268599+0000","last_undegraded":"2023-12-19T18:17:15.268599+0000","last_fullsized":"2023-12-19T18:17:15.268599+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:30:20.530651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.262943+0000","last_change":"2023-12-19T18:17:13.765898+0000","last_active":"2023-12-19T18:17:14.262943+0000","last_peered":"2023-12-19T18:17:14.262943+0000","last_clean":"2023-12-19T18:17:14.262943+0000","last_became_active":"2023-12-19T18:17:13.765675+0000","last_became_peered":"2023-12-19T18:17:13.765675+0000","last_unstale":"2023-12-19T18:17:14.262943+0000","last_undegraded":"2023-12-19T18:17:14.262943+0000","last_fullsized":"2023-12-19T18:17:14.262943+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:48:46.625951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.268513+0000","last_change":"2023-12-19T18:17:14.269115+0000","last_active":"2023-12-19T18:17:15.268513+0000","last_peered":"2023-12-19T18:17:15.268513+0000","last_clean":"2023-12-19T18:17:15.268513+0000","last_became_active":"2023-12-19T18:17:14.268897+0000","last_became_peered":"2023-12-19T18:17:14.268897+0000","last_unstale":"2023-12-19T18:17:15.268513+0000","last_undegraded":"2023-12-19T18:17:15.268513+0000","last_fullsized":"2023-12-19T18:17:15.268513+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:00:55.570421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.279968+0000","last_change":"2023-12-19T18:17:13.764727+0000","last_active":"2023-12-19T18:17:14.279968+0000","last_peered":"2023-12-19T18:17:14.279968+0000","last_clean":"2023-12-19T18:17:14.279968+0000","last_became_active":"2023-12-19T18:17:13.764177+0000","last_became_peered":"2023-12-19T18:17:13.764177+0000","last_unstale":"2023-12-19T18:17:14.279968+0000","last_undegraded":"2023-12-19T18:17:14.279968+0000","last_fullsized":"2023-12-19T18:17:14.279968+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:55:15.322827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.321959+0000","last_change":"2023-12-19T18:17:14.290615+0000","last_active":"2023-12-19T18:17:15.321959+0000","last_peered":"2023-12-19T18:17:15.321959+0000","last_clean":"2023-12-19T18:17:15.321959+0000","last_became_active":"2023-12-19T18:17:14.290019+0000","last_became_peered":"2023-12-19T18:17:14.290019+0000","last_unstale":"2023-12-19T18:17:15.321959+0000","last_undegraded":"2023-12-19T18:17:15.321959+0000","last_fullsized":"2023-12-19T18:17:15.321959+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:20:18.819941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"49'3","reported_seq":14,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.276472+0000","last_change":"2023-12-19T18:17:13.760693+0000","last_active":"2023-12-19T18:17:14.276472+0000","last_peered":"2023-12-19T18:17:14.276472+0000","last_clean":"2023-12-19T18:17:14.276472+0000","last_became_active":"2023-12-19T18:17:13.760403+0000","last_became_peered":"2023-12-19T18:17:13.760403+0000","last_unstale":"2023-12-19T18:17:14.276472+0000","last_undegraded":"2023-12-19T18:17:14.276472+0000","last_fullsized":"2023-12-19T18:17:14.276472+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:54:51.030454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.272331+0000","last_change":"2023-12-19T18:17:14.283209+0000","last_active":"2023-12-19T18:17:15.272331+0000","last_peered":"2023-12-19T18:17:15.272331+0000","last_clean":"2023-12-19T18:17:15.272331+0000","last_became_active":"2023-12-19T18:17:14.282966+0000","last_became_peered":"2023-12-19T18:17:14.282966+0000","last_unstale":"2023-12-19T18:17:15.272331+0000","last_undegraded":"2023-12-19T18:17:15.272331+0000","last_fullsized":"2023-12-19T18:17:15.272331+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:42:26.470901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"49'6","reported_seq":17,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.295210+0000","last_change":"2023-12-19T18:17:13.768098+0000","last_active":"2023-12-19T18:17:14.295210+0000","last_peered":"2023-12-19T18:17:14.295210+0000","last_clean":"2023-12-19T18:17:14.295210+0000","last_became_active":"2023-12-19T18:17:13.766452+0000","last_became_peered":"2023-12-19T18:17:13.766452+0000","last_unstale":"2023-12-19T18:17:14.295210+0000","last_undegraded":"2023-12-19T18:17:14.295210+0000","last_fullsized":"2023-12-19T18:17:14.295210+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:39.026592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.268658+0000","last_change":"2023-12-19T18:17:14.267375+0000","last_active":"2023-12-19T18:17:15.268658+0000","last_peered":"2023-12-19T18:17:15.268658+0000","last_clean":"2023-12-19T18:17:15.268658+0000","last_became_active":"2023-12-19T18:17:14.267172+0000","last_became_peered":"2023-12-19T18:17:14.267172+0000","last_unstale":"2023-12-19T18:17:15.268658+0000","last_undegraded":"2023-12-19T18:17:15.268658+0000","last_fullsized":"2023-12-19T18:17:15.268658+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:37:12.039885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"49'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.291452+0000","last_change":"2023-12-19T18:17:13.774237+0000","last_active":"2023-12-19T18:17:14.291452+0000","last_peered":"2023-12-19T18:17:14.291452+0000","last_clean":"2023-12-19T18:17:14.291452+0000","last_became_active":"2023-12-19T18:17:13.773986+0000","last_became_peered":"2023-12-19T18:17:13.773986+0000","last_unstale":"2023-12-19T18:17:14.291452+0000","last_undegraded":"2023-12-19T18:17:14.291452+0000","last_fullsized":"2023-12-19T18:17:14.291452+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:16.564267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.669828+0000","last_change":"2023-12-19T18:17:14.283499+0000","last_active":"2023-12-19T18:17:15.669828+0000","last_peered":"2023-12-19T18:17:15.669828+0000","last_clean":"2023-12-19T18:17:15.669828+0000","last_became_active":"2023-12-19T18:17:14.281600+0000","last_became_peered":"2023-12-19T18:17:14.281600+0000","last_unstale":"2023-12-19T18:17:15.669828+0000","last_undegraded":"2023-12-19T18:17:15.669828+0000","last_fullsized":"2023-12-19T18:17:15.669828+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:19.411438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.266579+0000","last_change":"2023-12-19T18:17:13.765537+0000","last_active":"2023-12-19T18:17:14.266579+0000","last_peered":"2023-12-19T18:17:14.266579+0000","last_clean":"2023-12-19T18:17:14.266579+0000","last_became_active":"2023-12-19T18:17:13.765124+0000","last_became_peered":"2023-12-19T18:17:13.765124+0000","last_unstale":"2023-12-19T18:17:14.266579+0000","last_undegraded":"2023-12-19T18:17:14.266579+0000","last_fullsized":"2023-12-19T18:17:14.266579+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:18.395505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.668937+0000","last_change":"2023-12-19T18:17:14.290269+0000","last_active":"2023-12-19T18:17:15.668937+0000","last_peered":"2023-12-19T18:17:15.668937+0000","last_clean":"2023-12-19T18:17:15.668937+0000","last_became_active":"2023-12-19T18:17:14.290024+0000","last_became_peered":"2023-12-19T18:17:14.290024+0000","last_unstale":"2023-12-19T18:17:15.668937+0000","last_undegraded":"2023-12-19T18:17:15.668937+0000","last_fullsized":"2023-12-19T18:17:15.668937+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:11.266100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.269580+0000","last_change":"2023-12-19T18:17:13.772209+0000","last_active":"2023-12-19T18:17:14.269580+0000","last_peered":"2023-12-19T18:17:14.269580+0000","last_clean":"2023-12-19T18:17:14.269580+0000","last_became_active":"2023-12-19T18:17:13.771518+0000","last_became_peered":"2023-12-19T18:17:13.771518+0000","last_unstale":"2023-12-19T18:17:14.269580+0000","last_undegraded":"2023-12-19T18:17:14.269580+0000","last_fullsized":"2023-12-19T18:17:14.269580+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:14:18.395505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.668737+0000","last_change":"2023-12-19T18:17:14.269852+0000","last_active":"2023-12-19T18:17:15.668737+0000","last_peered":"2023-12-19T18:17:15.668737+0000","last_clean":"2023-12-19T18:17:15.668737+0000","last_became_active":"2023-12-19T18:17:14.269333+0000","last_became_peered":"2023-12-19T18:17:14.269333+0000","last_unstale":"2023-12-19T18:17:15.668737+0000","last_undegraded":"2023-12-19T18:17:15.668737+0000","last_fullsized":"2023-12-19T18:17:15.668737+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:26:50.243947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"49'8","reported_seq":19,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.291347+0000","last_change":"2023-12-19T18:17:13.759248+0000","last_active":"2023-12-19T18:17:14.291347+0000","last_peered":"2023-12-19T18:17:14.291347+0000","last_clean":"2023-12-19T18:17:14.291347+0000","last_became_active":"2023-12-19T18:17:13.758874+0000","last_became_peered":"2023-12-19T18:17:13.758874+0000","last_unstale":"2023-12-19T18:17:14.291347+0000","last_undegraded":"2023-12-19T18:17:14.291347+0000","last_fullsized":"2023-12-19T18:17:14.291347+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:42:34.077752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.669952+0000","last_change":"2023-12-19T18:17:14.281199+0000","last_active":"2023-12-19T18:17:15.669952+0000","last_peered":"2023-12-19T18:17:15.669952+0000","last_clean":"2023-12-19T18:17:15.669952+0000","last_became_active":"2023-12-19T18:17:14.280058+0000","last_became_peered":"2023-12-19T18:17:14.280058+0000","last_unstale":"2023-12-19T18:17:15.669952+0000","last_undegraded":"2023-12-19T18:17:15.669952+0000","last_fullsized":"2023-12-19T18:17:15.669952+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:54:52.046387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"49'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.290907+0000","last_change":"2023-12-19T18:17:13.773369+0000","last_active":"2023-12-19T18:17:14.290907+0000","last_peered":"2023-12-19T18:17:14.290907+0000","last_clean":"2023-12-19T18:17:14.290907+0000","last_became_active":"2023-12-19T18:17:13.772949+0000","last_became_peered":"2023-12-19T18:17:13.772949+0000","last_unstale":"2023-12-19T18:17:14.290907+0000","last_undegraded":"2023-12-19T18:17:14.290907+0000","last_fullsized":"2023-12-19T18:17:14.290907+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:26:54.451411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.669136+0000","last_change":"2023-12-19T18:17:14.288785+0000","last_active":"2023-12-19T18:17:15.669136+0000","last_peered":"2023-12-19T18:17:15.669136+0000","last_clean":"2023-12-19T18:17:15.669136+0000","last_became_active":"2023-12-19T18:17:14.288336+0000","last_became_peered":"2023-12-19T18:17:14.288336+0000","last_unstale":"2023-12-19T18:17:15.669136+0000","last_undegraded":"2023-12-19T18:17:15.669136+0000","last_fullsized":"2023-12-19T18:17:15.669136+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:43:51.388849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"49'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.283567+0000","last_change":"2023-12-19T18:17:13.767539+0000","last_active":"2023-12-19T18:17:14.283567+0000","last_peered":"2023-12-19T18:17:14.283567+0000","last_clean":"2023-12-19T18:17:14.283567+0000","last_became_active":"2023-12-19T18:17:13.765294+0000","last_became_peered":"2023-12-19T18:17:13.765294+0000","last_unstale":"2023-12-19T18:17:14.283567+0000","last_undegraded":"2023-12-19T18:17:14.283567+0000","last_fullsized":"2023-12-19T18:17:14.283567+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:14:31.695686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.668848+0000","last_change":"2023-12-19T18:17:14.268792+0000","last_active":"2023-12-19T18:17:15.668848+0000","last_peered":"2023-12-19T18:17:15.668848+0000","last_clean":"2023-12-19T18:17:15.668848+0000","last_became_active":"2023-12-19T18:17:14.268583+0000","last_became_peered":"2023-12-19T18:17:14.268583+0000","last_unstale":"2023-12-19T18:17:15.668848+0000","last_undegraded":"2023-12-19T18:17:15.668848+0000","last_fullsized":"2023-12-19T18:17:15.668848+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:05.627747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"49'1","reported_seq":12,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.287734+0000","last_change":"2023-12-19T18:17:13.761979+0000","last_active":"2023-12-19T18:17:14.287734+0000","last_peered":"2023-12-19T18:17:14.287734+0000","last_clean":"2023-12-19T18:17:14.287734+0000","last_became_active":"2023-12-19T18:17:13.756276+0000","last_became_peered":"2023-12-19T18:17:13.756276+0000","last_unstale":"2023-12-19T18:17:14.287734+0000","last_undegraded":"2023-12-19T18:17:14.287734+0000","last_fullsized":"2023-12-19T18:17:14.287734+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:19.073949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.268774+0000","last_change":"2023-12-19T18:17:14.283187+0000","last_active":"2023-12-19T18:17:15.268774+0000","last_peered":"2023-12-19T18:17:15.268774+0000","last_clean":"2023-12-19T18:17:15.268774+0000","last_became_active":"2023-12-19T18:17:14.282953+0000","last_became_peered":"2023-12-19T18:17:14.282953+0000","last_unstale":"2023-12-19T18:17:15.268774+0000","last_undegraded":"2023-12-19T18:17:15.268774+0000","last_fullsized":"2023-12-19T18:17:15.268774+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:11.266100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.269837+0000","last_change":"2023-12-19T18:17:13.772032+0000","last_active":"2023-12-19T18:17:14.269837+0000","last_peered":"2023-12-19T18:17:14.269837+0000","last_clean":"2023-12-19T18:17:14.269837+0000","last_became_active":"2023-12-19T18:17:13.771169+0000","last_became_peered":"2023-12-19T18:17:13.771169+0000","last_unstale":"2023-12-19T18:17:14.269837+0000","last_undegraded":"2023-12-19T18:17:14.269837+0000","last_fullsized":"2023-12-19T18:17:14.269837+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:43:50.372916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.273702+0000","last_change":"2023-12-19T18:17:14.288709+0000","last_active":"2023-12-19T18:17:15.273702+0000","last_peered":"2023-12-19T18:17:15.273702+0000","last_clean":"2023-12-19T18:17:15.273702+0000","last_became_active":"2023-12-19T18:17:14.287208+0000","last_became_peered":"2023-12-19T18:17:14.287208+0000","last_unstale":"2023-12-19T18:17:15.273702+0000","last_undegraded":"2023-12-19T18:17:15.273702+0000","last_fullsized":"2023-12-19T18:17:15.273702+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:44.539319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"49'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.292677+0000","last_change":"2023-12-19T18:17:13.766353+0000","last_active":"2023-12-19T18:17:14.292677+0000","last_peered":"2023-12-19T18:17:14.292677+0000","last_clean":"2023-12-19T18:17:14.292677+0000","last_became_active":"2023-12-19T18:17:13.765729+0000","last_became_peered":"2023-12-19T18:17:13.765729+0000","last_unstale":"2023-12-19T18:17:14.292677+0000","last_undegraded":"2023-12-19T18:17:14.292677+0000","last_fullsized":"2023-12-19T18:17:14.292677+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:12.876293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.669149+0000","last_change":"2023-12-19T18:17:14.282168+0000","last_active":"2023-12-19T18:17:15.669149+0000","last_peered":"2023-12-19T18:17:15.669149+0000","last_clean":"2023-12-19T18:17:15.669149+0000","last_became_active":"2023-12-19T18:17:14.281948+0000","last_became_peered":"2023-12-19T18:17:14.281948+0000","last_unstale":"2023-12-19T18:17:15.669149+0000","last_undegraded":"2023-12-19T18:17:15.669149+0000","last_fullsized":"2023-12-19T18:17:15.669149+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:13.892227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.266069+0000","last_change":"2023-12-19T18:17:13.766178+0000","last_active":"2023-12-19T18:17:14.266069+0000","last_peered":"2023-12-19T18:17:14.266069+0000","last_clean":"2023-12-19T18:17:14.266069+0000","last_became_active":"2023-12-19T18:17:13.765723+0000","last_became_peered":"2023-12-19T18:17:13.765723+0000","last_unstale":"2023-12-19T18:17:14.266069+0000","last_undegraded":"2023-12-19T18:17:14.266069+0000","last_fullsized":"2023-12-19T18:17:14.266069+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:26:54.451411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.322872+0000","last_change":"2023-12-19T18:17:14.283883+0000","last_active":"2023-12-19T18:17:15.322872+0000","last_peered":"2023-12-19T18:17:15.322872+0000","last_clean":"2023-12-19T18:17:15.322872+0000","last_became_active":"2023-12-19T18:17:14.279625+0000","last_became_peered":"2023-12-19T18:17:14.279625+0000","last_unstale":"2023-12-19T18:17:15.322872+0000","last_undegraded":"2023-12-19T18:17:15.322872+0000","last_fullsized":"2023-12-19T18:17:15.322872+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:27:03.643980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.286364+0000","last_change":"2023-12-19T18:17:13.765819+0000","last_active":"2023-12-19T18:17:14.286364+0000","last_peered":"2023-12-19T18:17:14.286364+0000","last_clean":"2023-12-19T18:17:14.286364+0000","last_became_active":"2023-12-19T18:17:13.765306+0000","last_became_peered":"2023-12-19T18:17:13.765306+0000","last_unstale":"2023-12-19T18:17:14.286364+0000","last_undegraded":"2023-12-19T18:17:14.286364+0000","last_fullsized":"2023-12-19T18:17:14.286364+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:26:49.228013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.322241+0000","last_change":"2023-12-19T18:17:14.287927+0000","last_active":"2023-12-19T18:17:15.322241+0000","last_peered":"2023-12-19T18:17:15.322241+0000","last_clean":"2023-12-19T18:17:15.322241+0000","last_became_active":"2023-12-19T18:17:14.287669+0000","last_became_peered":"2023-12-19T18:17:14.287669+0000","last_unstale":"2023-12-19T18:17:15.322241+0000","last_undegraded":"2023-12-19T18:17:15.322241+0000","last_fullsized":"2023-12-19T18:17:15.322241+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:28:47.810319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.274223+0000","last_change":"2023-12-19T18:17:13.773874+0000","last_active":"2023-12-19T18:17:14.274223+0000","last_peered":"2023-12-19T18:17:14.274223+0000","last_clean":"2023-12-19T18:17:14.274223+0000","last_became_active":"2023-12-19T18:17:13.773683+0000","last_became_peered":"2023-12-19T18:17:13.773683+0000","last_unstale":"2023-12-19T18:17:14.274223+0000","last_undegraded":"2023-12-19T18:17:14.274223+0000","last_fullsized":"2023-12-19T18:17:14.274223+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:55:36.933023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.272255+0000","last_change":"2023-12-19T18:17:14.285300+0000","last_active":"2023-12-19T18:17:15.272255+0000","last_peered":"2023-12-19T18:17:15.272255+0000","last_clean":"2023-12-19T18:17:15.272255+0000","last_became_active":"2023-12-19T18:17:14.284998+0000","last_became_peered":"2023-12-19T18:17:14.284998+0000","last_unstale":"2023-12-19T18:17:15.272255+0000","last_undegraded":"2023-12-19T18:17:15.272255+0000","last_fullsized":"2023-12-19T18:17:15.272255+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:19:31.299155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.259541+0000","last_change":"2023-12-19T18:17:13.772832+0000","last_active":"2023-12-19T18:17:14.259541+0000","last_peered":"2023-12-19T18:17:14.259541+0000","last_clean":"2023-12-19T18:17:14.259541+0000","last_became_active":"2023-12-19T18:17:13.772503+0000","last_became_peered":"2023-12-19T18:17:13.772503+0000","last_unstale":"2023-12-19T18:17:14.259541+0000","last_undegraded":"2023-12-19T18:17:14.259541+0000","last_fullsized":"2023-12-19T18:17:14.259541+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:47.048524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.273657+0000","last_change":"2023-12-19T18:17:14.289351+0000","last_active":"2023-12-19T18:17:15.273657+0000","last_peered":"2023-12-19T18:17:15.273657+0000","last_clean":"2023-12-19T18:17:15.273657+0000","last_became_active":"2023-12-19T18:17:14.287054+0000","last_became_peered":"2023-12-19T18:17:14.287054+0000","last_unstale":"2023-12-19T18:17:15.273657+0000","last_undegraded":"2023-12-19T18:17:15.273657+0000","last_fullsized":"2023-12-19T18:17:15.273657+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:25:50.158021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.266681+0000","last_change":"2023-12-19T18:17:13.766915+0000","last_active":"2023-12-19T18:17:14.266681+0000","last_peered":"2023-12-19T18:17:14.266681+0000","last_clean":"2023-12-19T18:17:14.266681+0000","last_became_active":"2023-12-19T18:17:13.766670+0000","last_became_peered":"2023-12-19T18:17:13.766670+0000","last_unstale":"2023-12-19T18:17:14.266681+0000","last_undegraded":"2023-12-19T18:17:14.266681+0000","last_fullsized":"2023-12-19T18:17:14.266681+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:47.048524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.271982+0000","last_change":"2023-12-19T18:17:14.285331+0000","last_active":"2023-12-19T18:17:15.271982+0000","last_peered":"2023-12-19T18:17:15.271982+0000","last_clean":"2023-12-19T18:17:15.271982+0000","last_became_active":"2023-12-19T18:17:14.284751+0000","last_became_peered":"2023-12-19T18:17:14.284751+0000","last_unstale":"2023-12-19T18:17:15.271982+0000","last_undegraded":"2023-12-19T18:17:15.271982+0000","last_fullsized":"2023-12-19T18:17:15.271982+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:03:38.026851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.278121+0000","last_change":"2023-12-19T18:17:13.772081+0000","last_active":"2023-12-19T18:17:14.278121+0000","last_peered":"2023-12-19T18:17:14.278121+0000","last_clean":"2023-12-19T18:17:14.278121+0000","last_became_active":"2023-12-19T18:17:13.771681+0000","last_became_peered":"2023-12-19T18:17:13.771681+0000","last_unstale":"2023-12-19T18:17:14.278121+0000","last_undegraded":"2023-12-19T18:17:14.278121+0000","last_fullsized":"2023-12-19T18:17:14.278121+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:57:40.505431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.273516+0000","last_change":"2023-12-19T18:17:14.291879+0000","last_active":"2023-12-19T18:17:15.273516+0000","last_peered":"2023-12-19T18:17:15.273516+0000","last_clean":"2023-12-19T18:17:15.273516+0000","last_became_active":"2023-12-19T18:17:14.291348+0000","last_became_peered":"2023-12-19T18:17:14.291348+0000","last_unstale":"2023-12-19T18:17:15.273516+0000","last_undegraded":"2023-12-19T18:17:15.273516+0000","last_fullsized":"2023-12-19T18:17:15.273516+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:42:54.135928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.266675+0000","last_change":"2023-12-19T18:17:13.761837+0000","last_active":"2023-12-19T18:17:14.266675+0000","last_peered":"2023-12-19T18:17:14.266675+0000","last_clean":"2023-12-19T18:17:14.266675+0000","last_became_active":"2023-12-19T18:17:13.760174+0000","last_became_peered":"2023-12-19T18:17:13.760174+0000","last_unstale":"2023-12-19T18:17:14.266675+0000","last_undegraded":"2023-12-19T18:17:14.266675+0000","last_fullsized":"2023-12-19T18:17:14.266675+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:57:35.015037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.323083+0000","last_change":"2023-12-19T18:17:14.275156+0000","last_active":"2023-12-19T18:17:15.323083+0000","last_peered":"2023-12-19T18:17:15.323083+0000","last_clean":"2023-12-19T18:17:15.323083+0000","last_became_active":"2023-12-19T18:17:14.270450+0000","last_became_peered":"2023-12-19T18:17:14.270450+0000","last_unstale":"2023-12-19T18:17:15.323083+0000","last_undegraded":"2023-12-19T18:17:15.323083+0000","last_fullsized":"2023-12-19T18:17:15.323083+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:01:50.547764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.272565+0000","last_change":"2023-12-19T18:17:13.769448+0000","last_active":"2023-12-19T18:17:14.272565+0000","last_peered":"2023-12-19T18:17:14.272565+0000","last_clean":"2023-12-19T18:17:14.272565+0000","last_became_active":"2023-12-19T18:17:13.769219+0000","last_became_peered":"2023-12-19T18:17:13.769219+0000","last_unstale":"2023-12-19T18:17:14.272565+0000","last_undegraded":"2023-12-19T18:17:14.272565+0000","last_fullsized":"2023-12-19T18:17:14.272565+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:15:40.945820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.268395+0000","last_change":"2023-12-19T18:17:14.266873+0000","last_active":"2023-12-19T18:17:15.268395+0000","last_peered":"2023-12-19T18:17:15.268395+0000","last_clean":"2023-12-19T18:17:15.268395+0000","last_became_active":"2023-12-19T18:17:14.266646+0000","last_became_peered":"2023-12-19T18:17:14.266646+0000","last_unstale":"2023-12-19T18:17:15.268395+0000","last_undegraded":"2023-12-19T18:17:15.268395+0000","last_fullsized":"2023-12-19T18:17:15.268395+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:57:41.521365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.281689+0000","last_change":"2023-12-19T18:17:13.773885+0000","last_active":"2023-12-19T18:17:14.281689+0000","last_peered":"2023-12-19T18:17:14.281689+0000","last_clean":"2023-12-19T18:17:14.281689+0000","last_became_active":"2023-12-19T18:17:13.773032+0000","last_became_peered":"2023-12-19T18:17:13.773032+0000","last_unstale":"2023-12-19T18:17:14.281689+0000","last_undegraded":"2023-12-19T18:17:14.281689+0000","last_fullsized":"2023-12-19T18:17:14.281689+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:30:19.514717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.668244+0000","last_change":"2023-12-19T18:17:14.265090+0000","last_active":"2023-12-19T18:17:15.668244+0000","last_peered":"2023-12-19T18:17:15.668244+0000","last_clean":"2023-12-19T18:17:15.668244+0000","last_became_active":"2023-12-19T18:17:14.264823+0000","last_became_peered":"2023-12-19T18:17:14.264823+0000","last_unstale":"2023-12-19T18:17:15.668244+0000","last_undegraded":"2023-12-19T18:17:15.668244+0000","last_fullsized":"2023-12-19T18:17:15.668244+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:00:54.488849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.274133+0000","last_change":"2023-12-19T18:17:13.769016+0000","last_active":"2023-12-19T18:17:14.274133+0000","last_peered":"2023-12-19T18:17:14.274133+0000","last_clean":"2023-12-19T18:17:14.274133+0000","last_became_active":"2023-12-19T18:17:13.768494+0000","last_became_peered":"2023-12-19T18:17:13.768494+0000","last_unstale":"2023-12-19T18:17:14.274133+0000","last_undegraded":"2023-12-19T18:17:14.274133+0000","last_fullsized":"2023-12-19T18:17:14.274133+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:58:44.448587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-12-19T18:17:15.322096+0000","last_change":"2023-12-19T18:17:14.272089+0000","last_active":"2023-12-19T18:17:15.322096+0000","last_peered":"2023-12-19T18:17:15.322096+0000","last_clean":"2023-12-19T18:17:15.322096+0000","last_became_active":"2023-12-19T18:17:14.271086+0000","last_became_peered":"2023-12-19T18:17:14.271086+0000","last_unstale":"2023-12-19T18:17:15.322096+0000","last_undegraded":"2023-12-19T18:17:15.322096+0000","last_fullsized":"2023-12-19T18:17:15.322096+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":48,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:13.736698+0000","last_clean_scrub_stamp":"2023-12-19T18:17:13.736698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:12:45.339741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"49'2","reported_seq":13,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.284473+0000","last_change":"2023-12-19T18:17:13.775721+0000","last_active":"2023-12-19T18:17:14.284473+0000","last_peered":"2023-12-19T18:17:14.284473+0000","last_clean":"2023-12-19T18:17:14.284473+0000","last_became_active":"2023-12-19T18:17:13.775489+0000","last_became_peered":"2023-12-19T18:17:13.775489+0000","last_unstale":"2023-12-19T18:17:14.284473+0000","last_undegraded":"2023-12-19T18:17:14.284473+0000","last_fullsized":"2023-12-19T18:17:14.284473+0000","mapping_epoch":47,"log_start":"0'0","ondisk_log_start":"0'0","created":47,"last_epoch_clean":48,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:17:12.720765+0000","last_clean_scrub_stamp":"2023-12-19T18:17:12.720765+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:19:01.049904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":190,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-12-19T18:17:14.261972+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:17:14.261972+0000","last_peered":"2023-12-19T18:17:14.261972+0000","last_clean":"2023-12-19T18:17:14.261972+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:17:14.261972+0000","last_undegraded":"2023-12-19T18:17:14.261972+0000","last_fullsized":"2023-12-19T18:17:14.261972+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":238,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195073,"statfs":{"total":200000000,"available":199755670,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.117,"15min":1.1100000000000001},"min":{"1min":0.73499999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.5640000000000001,"5min":1.639,"15min":1.639},"last":1.02},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.149,"15min":1.143},"min":{"1min":0.79900000000000004,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.4910000000000001,"5min":1.677,"15min":1.677},"last":0.97399999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.175,"15min":1.1699999999999999},"min":{"1min":0.71099999999999997,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.6759999999999999,"5min":2.8450000000000002,"15min":2.8450000000000002},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.1759999999999999,"15min":1.1859999999999999},"min":{"1min":0.83499999999999996,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.367,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":0.877}]},{"osd":2,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0680000000000001,"15min":1.077},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.373,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":1.198,"5min":1.161,"15min":1.155},"min":{"1min":0.72199999999999998,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.486,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":1.26}]},{"osd":3,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.196,"5min":1.1140000000000001,"15min":1.1000000000000001},"min":{"1min":0.84399999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.2490000000000001,"15min":1.2270000000000001},"min":{"1min":0.94899999999999995,"5min":0.748,"15min":0.748},"max":{"1min":1.635,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":1.135}]},{"osd":4,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.0700000000000001,"15min":1.0509999999999999},"min":{"1min":0.77900000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.25,"5min":1.214,"15min":1.208},"min":{"1min":0.92600000000000005,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.7370000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":0.94599999999999995}]},{"osd":5,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.3129999999999999,"15min":1.2949999999999999},"min":{"1min":0.90300000000000002,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7370000000000001,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":1.335},{"interface":"front","average":{"1min":1.331,"5min":1.2929999999999999,"15min":1.2869999999999999},"min":{"1min":0.88800000000000001,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.6539999999999999,"5min":2.754,"15min":2.754},"last":1.0740000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:16:47 2023","interfaces":[{"interface":"back","average":{"1min":1.5149999999999999,"5min":1.431,"15min":1.417},"min":{"1min":1.254,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.7729999999999999,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.3360000000000001,"15min":1.3220000000000001},"min":{"1min":1.0009999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.661,"5min":2.149,"15min":2.149},"last":1.246}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":239,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195073,"statfs":{"total":200000000,"available":199755088,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,4],"upper_bound":32},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0669999999999999,"5min":0.96999999999999997,"15min":0.95399999999999996},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":1.7270000000000001},{"interface":"front","average":{"1min":1.107,"5min":1.028,"15min":1.0149999999999999},"min":{"1min":0.70099999999999996,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.573,"5min":1.573,"15min":1.573},"last":1.0449999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":1.107,"15min":1.115},"min":{"1min":0.78000000000000003,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.548,"5min":2.5870000000000002,"15min":2.5870000000000002},"last":4.5540000000000003},{"interface":"front","average":{"1min":1.095,"5min":1.1359999999999999,"15min":1.143},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.671,"5min":1.671,"15min":1.671},"last":1.776}]},{"osd":2,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1080000000000001,"15min":1.0960000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":1.821},{"interface":"front","average":{"1min":1.155,"5min":1.2290000000000001,"15min":1.2410000000000001},"min":{"1min":0.81499999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.6850000000000001,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.0880000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.163,"15min":1.155},"min":{"1min":0.94799999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.48,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.123},{"interface":"front","average":{"1min":1.153,"5min":1.147,"15min":1.1459999999999999},"min":{"1min":0.872,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":4.6189999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.216,"15min":1.22},"min":{"1min":0.81799999999999995,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":1.6180000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":3.0680000000000001},{"interface":"front","average":{"1min":1.421,"5min":1.3180000000000001,"15min":1.3009999999999999},"min":{"1min":0.96199999999999997,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7969999999999999,"5min":1.7969999999999999,"15min":1.7969999999999999},"last":2.859}]},{"osd":5,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.1519999999999999,"15min":1.1379999999999999},"min":{"1min":0.75900000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":4.6680000000000001},{"interface":"front","average":{"1min":1.387,"5min":1.319,"15min":1.3080000000000001},"min":{"1min":1.0389999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.7869999999999999,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":3.3580000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.383,"5min":1.3109999999999999,"15min":1.2989999999999999},"min":{"1min":0.77200000000000002,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.7110000000000001,"5min":1.7110000000000001,"15min":1.7110000000000001},"last":2.0699999999999998},{"interface":"front","average":{"1min":1.389,"5min":1.3200000000000001,"15min":1.3080000000000001},"min":{"1min":0.92300000000000004,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":1.798,"5min":1.798,"15min":1.798},"last":5.6639999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":815,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194497,"statfs":{"total":200000000,"available":199165314,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.1419999999999999,"15min":1.135},"min":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.4199999999999999,"5min":1.863,"15min":1.863},"last":1.482},{"interface":"front","average":{"1min":1.2589999999999999,"5min":1.157,"15min":1.1399999999999999},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":1.3380000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.034,"5min":0.99399999999999999,"15min":0.98799999999999999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.4770000000000001,"5min":1.4770000000000001,"15min":1.4770000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.1220000000000001,"5min":1.03,"15min":1.0149999999999999},"min":{"1min":0.68700000000000006,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.843,"5min":1.843,"15min":1.843},"last":1.446}]},{"osd":2,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.0660000000000001,"15min":1.046},"min":{"1min":0.75700000000000001,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.0900000000000001,"15min":1.0920000000000001},"min":{"1min":0.76800000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.6040000000000001,"5min":1.698,"15min":1.698},"last":1.2470000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.0569999999999999,"15min":1.042},"min":{"1min":0.69399999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.1950000000000001,"15min":1.1739999999999999},"min":{"1min":0.88300000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.6739999999999999,"5min":1.6739999999999999,"15min":1.6739999999999999},"last":1.419}]},{"osd":4,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.1379999999999999,"15min":1.1080000000000001},"min":{"1min":0.77600000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.716,"5min":2.0289999999999999,"15min":2.0289999999999999},"last":1.659},{"interface":"front","average":{"1min":1.165,"5min":1.105,"15min":1.095},"min":{"1min":0.85599999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.5109999999999999,"5min":1.901,"15min":1.901},"last":0.93400000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.3120000000000001,"15min":1.282},"min":{"1min":1.1799999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.853,"5min":1.9530000000000001,"15min":1.9530000000000001},"last":1.5309999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.2210000000000001,"15min":1.208},"min":{"1min":1.044,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.5600000000000001,"5min":1.976,"15min":1.976},"last":1.1639999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:16:45 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.226,"15min":1.2},"min":{"1min":0.85399999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.9359999999999999,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.0860000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.399,"15min":1.387},"min":{"1min":1.21,"5min":0.623,"15min":0.623},"max":{"1min":1.806,"5min":1.875,"15min":1.875},"last":1.29}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":240,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195072,"statfs":{"total":200000000,"available":199754048,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.052,"5min":1.008,"15min":1.0009999999999999},"min":{"1min":0.70299999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.4419999999999999,"5min":1.454,"15min":1.454},"last":1.579},{"interface":"front","average":{"1min":1.089,"5min":1.0369999999999999,"15min":1.028},"min":{"1min":0.72899999999999998,"5min":0.71399999999999997,"15min":0.71399999999999997},"max":{"1min":1.7909999999999999,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":1.0649999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.137,"15min":1.141},"min":{"1min":0.875,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.393,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.115},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.161,"15min":1.1459999999999999},"min":{"1min":0.88200000000000001,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.6479999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0640000000000001,"5min":1.1100000000000001,"15min":1.1180000000000001},"min":{"1min":0.747,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.5129999999999999,"5min":1.6850000000000001,"15min":1.6850000000000001},"last":0.95299999999999996},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.075,"15min":1.075},"min":{"1min":0.80500000000000005,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.3160000000000001,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":1.234}]},{"osd":3,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.2470000000000001,"15min":1.2410000000000001},"min":{"1min":0.88300000000000001,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":1.4590000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.294,"15min":1.288},"min":{"1min":0.85499999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.7050000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.2829999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.222,"5min":1.165,"15min":1.155},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.8740000000000001,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.345},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.3360000000000001,"15min":1.3180000000000001},"min":{"1min":0.69499999999999995,"5min":0.504,"15min":0.504},"max":{"1min":1.72,"5min":1.72,"15min":1.72},"last":1.696}]},{"osd":6,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.2989999999999999,"15min":1.2969999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.853,"5min":1.853,"15min":1.853},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.369,"15min":1.361},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.78,"5min":1.78,"15min":1.78},"last":1.512}]},{"osd":7,"last update":"Tue Dec 19 18:16:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.331,"15min":1.319},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.6930000000000001,"5min":1.6930000000000001,"15min":1.6930000000000001},"last":1.744},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.8999999999999999,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.3680000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574882,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":240,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195072,"statfs":{"total":200000000,"available":199754140,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.95599999999999996,"15min":0.97999999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2450000000000001,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":0.98899999999999999},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.91900000000000004,"15min":0.91700000000000004},"min":{"1min":0.49299999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":1.4890000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":1.0369999999999999,"15min":1.0700000000000001},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.488,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":1.107},{"interface":"front","average":{"1min":0.94499999999999995,"5min":1.073,"15min":1.0940000000000001},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.512,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":1.169}]},{"osd":2,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":1.1579999999999999,"15min":1.1950000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.6240000000000001,"5min":3.6890000000000001,"15min":3.6890000000000001},"last":1.577},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1819999999999999,"15min":1.1950000000000001},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.726,"5min":1.905,"15min":1.905},"last":1.3169999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.155,"5min":1.167,"15min":1.169},"min":{"1min":0.79500000000000004,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.758,"5min":1.774,"15min":1.774},"last":1.367},{"interface":"front","average":{"1min":1.264,"5min":1.3859999999999999,"15min":1.4059999999999999},"min":{"1min":0.86399999999999999,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.742,"5min":4.0209999999999999,"15min":4.0209999999999999},"last":1.27}]},{"osd":5,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.421,"15min":1.444},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.804,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":1.425},{"interface":"front","average":{"1min":1.2569999999999999,"5min":1.2370000000000001,"15min":1.234},"min":{"1min":0.76600000000000001,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":1.2290000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.462,"15min":1.488},"min":{"1min":0.91000000000000003,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.8320000000000001,"5min":3.9399999999999999,"15min":3.9399999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.3320000000000001,"5min":1.4790000000000001,"15min":1.5029999999999999},"min":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.871,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.5589999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.335,"15min":1.341},"min":{"1min":0.75900000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.627},{"interface":"front","average":{"1min":1.385,"5min":1.5209999999999999,"15min":1.544},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":1.8180000000000001,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.734}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":238,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195073,"statfs":{"total":200000000,"available":199755614,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.96399999999999997,"15min":0.98399999999999999},"min":{"1min":0.373,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.575,"5min":4.7610000000000001,"15min":4.7610000000000001},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.90200000000000002,"5min":1.0229999999999999,"15min":1.0429999999999999},"min":{"1min":0.435,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.645,"5min":4.7850000000000001,"15min":4.7850000000000001},"last":1.143}]},{"osd":1,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":1.2270000000000001,"15min":1.28},"min":{"1min":0.49199999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.1919999999999999,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.999,"5min":1.169,"15min":1.1970000000000001},"min":{"1min":0.51400000000000001,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.623,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":1.256}]},{"osd":3,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0049999999999999,"5min":1.401,"15min":1.4670000000000001},"min":{"1min":0.61799999999999999,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.5840000000000001,"5min":8.3109999999999999,"15min":8.3109999999999999},"last":1.3999999999999999},{"interface":"front","average":{"1min":1.05,"5min":1.3700000000000001,"15min":1.423},"min":{"1min":0.51900000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.802,"5min":8.0690000000000008,"15min":8.0690000000000008},"last":0.97099999999999997}]},{"osd":4,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.46,"15min":1.49},"min":{"1min":0.82599999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.702,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.554,"15min":1.601},"min":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.6819999999999999,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":1.49}]},{"osd":5,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.2869999999999999,"5min":1.5840000000000001,"15min":1.6339999999999999},"min":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.177,"5min":8.1839999999999993,"15min":8.1839999999999993},"last":1.331},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.4039999999999999,"15min":1.417},"min":{"1min":0.90400000000000003,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8540000000000001,"5min":5.165,"15min":5.165},"last":1.452}]},{"osd":6,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.357,"5min":1.452,"15min":1.468},"min":{"1min":0.82999999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.7929999999999999,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":1.4319999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.579,"15min":1.6160000000000001},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.1309999999999998,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":1.2090000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:16:44 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.5580000000000001,"15min":1.575},"min":{"1min":0.878,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.877,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":1.351},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.6870000000000001,"15min":1.7250000000000001},"min":{"1min":0.93999999999999995,"5min":0.878,"15min":0.878},"max":{"1min":2.2170000000000001,"5min":8.2300000000000004,"15min":8.2300000000000004},"last":1.53}]}]},{"osd":1,"up_from":13,"seq":55834574882,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":815,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194497,"statfs":{"total":200000000,"available":199165266,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.84499999999999997,"15min":0.84699999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.2849999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.95799999999999996,"15min":0.96799999999999997},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.024}]},{"osd":2,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":0.97699999999999998,"5min":1.0469999999999999,"15min":1.0589999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.5700000000000001,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.89600000000000002,"15min":0.89000000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.0780000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.115,"15min":1.1020000000000001},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":2.1019999999999999,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1779999999999999,"15min":1.1830000000000001},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.641,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.2949999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.2390000000000001,"5min":1.2689999999999999,"15min":1.274},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.821,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.264,"15min":1.262},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.502}]},{"osd":5,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.284,"5min":1.256,"15min":1.2509999999999999},"min":{"1min":0.91900000000000004,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":1.7290000000000001,"5min":1.821,"15min":1.821},"last":1.4379999999999999},{"interface":"front","average":{"1min":1.397,"5min":1.3839999999999999,"15min":1.3819999999999999},"min":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"max":{"1min":2.0249999999999999,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.3520000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3560000000000001,"15min":1.3580000000000001},"min":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.948,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.425,"15min":1.419},"min":{"1min":0.98799999999999999,"5min":0.84399999999999997,"15min":0.84399999999999997},"max":{"1min":2.1280000000000001,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.478}]},{"osd":7,"last update":"Tue Dec 19 18:16:46 2023","interfaces":[{"interface":"back","average":{"1min":1.393,"5min":1.319,"15min":1.3069999999999999},"min":{"1min":0.82399999999999995,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.3610000000000002,"5min":2.3610000000000002,"15min":2.3610000000000002},"last":1.607},{"interface":"front","average":{"1min":1.427,"5min":1.4319999999999999,"15min":1.4330000000000001},"min":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"max":{"1min":1.855,"5min":2,"15min":2},"last":1.6499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":239,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":195073,"statfs":{"total":200000000,"available":199755032,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.871,"15min":0.85899999999999999},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.639,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.998,"5min":0.99399999999999999,"15min":0.99299999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.593,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.75900000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.90100000000000002,"15min":0.91500000000000004},"min":{"1min":0.47999999999999998,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0980000000000001,"5min":1.577,"15min":1.577},"last":1.042},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.90200000000000002,"15min":0.90100000000000002},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.466,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.73099999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.98099999999999998,"15min":0.98899999999999999},"min":{"1min":0.53100000000000003,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.669,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.629},{"interface":"front","average":{"1min":1.04,"5min":1.1639999999999999,"15min":1.1839999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.7230000000000001,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.89200000000000002}]},{"osd":4,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.296,"5min":1.1719999999999999,"15min":1.1519999999999999},"min":{"1min":0.84099999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"max":{"1min":1.8979999999999999,"5min":1.8979999999999999,"15min":1.8979999999999999},"last":1.101},{"interface":"front","average":{"1min":1.196,"5min":1.264,"15min":1.2749999999999999},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.7509999999999999,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":0.85899999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.319,"15min":1.339},"min":{"1min":0.88600000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6240000000000001,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.0629999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.25,"15min":1.2410000000000001},"min":{"1min":0.81000000000000005,"5min":0.72699999999999998,"15min":0.72699999999999998},"max":{"1min":1.6519999999999999,"5min":1.7989999999999999,"15min":1.7989999999999999},"last":1.1319999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3620000000000001,"15min":1.363},"min":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8049999999999999,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.335,"15min":1.319},"min":{"1min":0.99299999999999999,"5min":0.82799999999999996,"15min":0.82799999999999996},"max":{"1min":1.9379999999999999,"5min":2.1640000000000001,"15min":2.1640000000000001},"last":0.92900000000000005}]},{"osd":7,"last update":"Tue Dec 19 18:16:49 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.3580000000000001,"15min":1.3420000000000001},"min":{"1min":0.95599999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.897,"5min":1.9930000000000001,"15min":1.9930000000000001},"last":1.165},{"interface":"front","average":{"1min":1.325,"5min":1.4059999999999999,"15min":1.4199999999999999},"min":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6930000000000001,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":1.014}]}]}],"pool_statfs":[]}} 2023-12-19T18:17:19.810 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:17:19.810 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:17:20.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.095+0000 7f860335c640 1 Processor -- start 2023-12-19T18:17:20.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.096+0000 7f860335c640 1 -- start start 2023-12-19T18:17:20.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.097+0000 7f860335c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f85fc0b8c30 0x7f85fc0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.097+0000 7f860335c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85fc0ce130 0x7f85fc0cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.097+0000 7f860335c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85fc0ce560 con 0x7f85fc0cf9b0 2023-12-19T18:17:20.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.097+0000 7f860335c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85fc0cea80 con 0x7f85fc0b8c30 2023-12-19T18:17:20.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.097+0000 7f860335c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc0b5f10 con 0x7f85fc0ce130 2023-12-19T18:17:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.098+0000 7f86010f8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f85fc0b8c30 0x7f85fc0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.098+0000 7f86008f7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f85fc0cf9b0 0x7f85fc0cda20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41816/0 (socket says 172.21.15.27:41816) 2023-12-19T18:17:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.098+0000 7f86008f7640 1 -- 172.21.15.27:0/2021483986 learned_addr learned my addr 172.21.15.27:0/2021483986 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.098+0000 7f85f3fff640 1 -- 172.21.15.27:0/2021483986 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3934191481 0 0) 0x7f85ec001110 con 0x7f85fc0cf9b0 2023-12-19T18:17:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.098+0000 7f85f3fff640 1 -- 172.21.15.27:0/2021483986 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85e8003520 con 0x7f85fc0cf9b0 2023-12-19T18:17:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.098+0000 7f86018f9640 1 --2- 172.21.15.27:0/2021483986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85fc0ce130 0x7f85fc0cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.098+0000 7f86010f8640 1 -- 172.21.15.27:0/2021483986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85fc0ce130 msgr2=0x7f85fc0cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.098+0000 7f86010f8640 1 --2- 172.21.15.27:0/2021483986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85fc0ce130 0x7f85fc0cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.098+0000 7f86010f8640 1 -- 172.21.15.27:0/2021483986 >> v1:172.21.15.27:6789/0 conn(0x7f85fc0cf9b0 legacy=0x7f85fc0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.098+0000 7f86010f8640 1 -- 172.21.15.27:0/2021483986 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc0ce7a0 con 0x7f85fc0b8c30 2023-12-19T18:17:20.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.099+0000 7f86010f8640 1 --2- 172.21.15.27:0/2021483986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc0b8c30 0x7f85fc0d1fc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f85f80097a0 tx=0x7f85f802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d780bbc62b50486c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:20.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.100+0000 7f85f3fff640 1 -- 172.21.15.27:0/2021483986 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8031780 con 0x7f85fc0b8c30 2023-12-19T18:17:20.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.100+0000 7f85f3fff640 1 -- 172.21.15.27:0/2021483986 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f80318e0 con 0x7f85fc0b8c30 2023-12-19T18:17:20.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.100+0000 7f85f3fff640 1 -- 172.21.15.27:0/2021483986 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f8031c50 con 0x7f85fc0b8c30 2023-12-19T18:17:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.100+0000 7f860335c640 1 -- 172.21.15.27:0/2021483986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc0b8c30 msgr2=0x7f85fc0d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.100+0000 7f860335c640 1 --2- 172.21.15.27:0/2021483986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc0b8c30 0x7f85fc0d1fc0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f85f80097a0 tx=0x7f85f802b820 comp rx=0 tx=0).stop 2023-12-19T18:17:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.101+0000 7f860335c640 1 -- 172.21.15.27:0/2021483986 shutdown_connections 2023-12-19T18:17:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.101+0000 7f860335c640 1 --2- 172.21.15.27:0/2021483986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85fc0ce130 0x7f85fc0cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.101+0000 7f860335c640 1 --2- 172.21.15.27:0/2021483986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc0b8c30 0x7f85fc0d1fc0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.101+0000 7f860335c640 1 -- 172.21.15.27:0/2021483986 >> 172.21.15.27:0/2021483986 conn(0x7f85fc0d2fa0 msgr2=0x7f85fc0d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.101+0000 7f860335c640 1 -- 172.21.15.27:0/2021483986 shutdown_connections 2023-12-19T18:17:20.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.101+0000 7f860335c640 1 -- 172.21.15.27:0/2021483986 wait complete. 2023-12-19T18:17:20.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.103+0000 7f860335c640 1 Processor -- start 2023-12-19T18:17:20.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.103+0000 7f860335c640 1 -- start start 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.104+0000 7f860335c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85fc0b8c30 0x7f85fc0ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.104+0000 7f860335c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc0ce130 0x7f85fc0cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.104+0000 7f860335c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85fc0cf9b0 0x7f85fc0cb830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.104+0000 7f860335c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc149090 con 0x7f85fc0ce130 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.104+0000 7f860335c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc149350 con 0x7f85fc0cf9b0 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.104+0000 7f860335c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85fc149630 con 0x7f85fc0b8c30 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.104+0000 7f86008f7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc0ce130 0x7f85fc0cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.104+0000 7f86008f7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc0ce130 0x7f85fc0cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34168/0 (socket says 172.21.15.27:34168) 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.104+0000 7f86008f7640 1 -- 172.21.15.27:0/3797150687 learned_addr learned my addr 172.21.15.27:0/3797150687 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.105+0000 7f86010f8640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85fc0b8c30 0x7f85fc0ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.105+0000 7f86018f9640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85fc0cf9b0 0x7f85fc0cb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.105+0000 7f86008f7640 1 -- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85fc0b8c30 msgr2=0x7f85fc0ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.105+0000 7f86008f7640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85fc0b8c30 0x7f85fc0ca070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.105+0000 7f86008f7640 1 -- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85fc0cf9b0 msgr2=0x7f85fc0cb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.105+0000 7f86008f7640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85fc0cf9b0 0x7f85fc0cb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.105+0000 7f86008f7640 1 -- 172.21.15.27:0/3797150687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85f8009400 con 0x7f85fc0ce130 2023-12-19T18:17:20.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.105+0000 7f86018f9640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85fc0cf9b0 0x7f85fc0cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:20.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.105+0000 7f86008f7640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc0ce130 0x7f85fc0cc680 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00c390 tx=0x7f85ec00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:20.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.105+0000 7f85f1ffb640 1 -- 172.21.15.27:0/3797150687 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec005ce0 con 0x7f85fc0ce130 2023-12-19T18:17:20.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.105+0000 7f85f1ffb640 1 -- 172.21.15.27:0/3797150687 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ec005e60 con 0x7f85fc0ce130 2023-12-19T18:17:20.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.105+0000 7f860335c640 1 -- 172.21.15.27:0/3797150687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85fc149910 con 0x7f85fc0ce130 2023-12-19T18:17:20.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.105+0000 7f85f1ffb640 1 -- 172.21.15.27:0/3797150687 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ec007310 con 0x7f85fc0ce130 2023-12-19T18:17:20.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.106+0000 7f860335c640 1 -- 172.21.15.27:0/3797150687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85fc0ccbc0 con 0x7f85fc0ce130 2023-12-19T18:17:20.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.107+0000 7f860335c640 1 -- 172.21.15.27:0/3797150687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85fc068e90 con 0x7f85fc0ce130 2023-12-19T18:17:20.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.108+0000 7f85f1ffb640 1 -- 172.21.15.27:0/3797150687 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f85ec007470 con 0x7f85fc0ce130 2023-12-19T18:17:20.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.109+0000 7f85f1ffb640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f85e006a460 0x7f85e006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.109+0000 7f85f1ffb640 1 -- 172.21.15.27:0/3797150687 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85ec087e00 con 0x7f85fc0ce130 2023-12-19T18:17:20.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.109+0000 7f86010f8640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f85e006a460 0x7f85e006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.111+0000 7f86010f8640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f85e006a460 0x7f85e006c920 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f85f8036000 tx=0x7f85f803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:20.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.113+0000 7f85f1ffb640 1 -- 172.21.15.27:0/3797150687 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f85ec053350 con 0x7f85fc0ce130 2023-12-19T18:17:20.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.382+0000 7f860335c640 1 -- 172.21.15.27:0/3797150687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f85fc0ce560 con 0x7f85fc0ce130 2023-12-19T18:17:20.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.390+0000 7f85f1ffb640 1 -- 172.21.15.27:0/3797150687 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7f85fc0ce560 con 0x7f85fc0ce130 2023-12-19T18:17:20.390 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2023-12-19T18:17:20.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.393+0000 7f860335c640 1 -- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f85e006a460 msgr2=0x7f85e006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.393+0000 7f860335c640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f85e006a460 0x7f85e006c920 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f85f8036000 tx=0x7f85f803d040 comp rx=0 tx=0).stop 2023-12-19T18:17:20.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.393+0000 7f860335c640 1 -- 172.21.15.27:0/3797150687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc0ce130 msgr2=0x7f85fc0cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.393+0000 7f860335c640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc0ce130 0x7f85fc0cc680 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f85ec00c390 tx=0x7f85ec00c850 comp rx=0 tx=0).stop 2023-12-19T18:17:20.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.393+0000 7f860335c640 1 -- 172.21.15.27:0/3797150687 shutdown_connections 2023-12-19T18:17:20.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.393+0000 7f860335c640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f85e006a460 0x7f85e006c920 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.393+0000 7f860335c640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85fc0cf9b0 0x7f85fc0cb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.393+0000 7f860335c640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85fc0ce130 0x7f85fc0cc680 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.393+0000 7f860335c640 1 --2- 172.21.15.27:0/3797150687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85fc0b8c30 0x7f85fc0ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.393+0000 7f860335c640 1 -- 172.21.15.27:0/3797150687 >> 172.21.15.27:0/3797150687 conn(0x7f85fc0d2fa0 msgr2=0x7f85fc0d1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:20.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.393+0000 7f860335c640 1 -- 172.21.15.27:0/3797150687 shutdown_connections 2023-12-19T18:17:20.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.394+0000 7f860335c640 1 -- 172.21.15.27:0/3797150687 wait complete. 2023-12-19T18:17:20.415 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:20.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.639+0000 7fc9edf7e640 1 Processor -- start 2023-12-19T18:17:20.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.639+0000 7fc9edf7e640 1 -- start start 2023-12-19T18:17:20.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.641+0000 7fc9edf7e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc9e815af70 0x7fc9e80ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.641+0000 7fc9edf7e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e80cb850 con 0x7fc9e80adc20 2023-12-19T18:17:20.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.641+0000 7fc9edf7e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9e815b350 con 0x7fc9e80cbce0 2023-12-19T18:17:20.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.641+0000 7fc9edf7e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9e80cac40 con 0x7fc9e815af70 2023-12-19T18:17:20.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.641+0000 7fc9e7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc9e80adc20 0x7fc9e806b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41822/0 (socket says 172.21.15.27:41822) 2023-12-19T18:17:20.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.641+0000 7fc9e7fff640 1 -- 172.21.15.27:0/2041068197 learned_addr learned my addr 172.21.15.27:0/2041068197 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:20.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.642+0000 7fc9e77fe640 1 --2- 172.21.15.27:0/2041068197 >> v2:172.21.15.192:3300/0 conn(0x7fc9e815af70 0x7fc9e80ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.642+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1590268520 0 0) 0x7fc9e80cb850 con 0x7fc9e80adc20 2023-12-19T18:17:20.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.642+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9d4003520 con 0x7fc9e80adc20 2023-12-19T18:17:20.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.642+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1369645665 0 0) 0x7fc9d4003520 con 0x7fc9e80adc20 2023-12-19T18:17:20.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.642+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 >> v2:172.21.15.192:3300/0 conn(0x7fc9e815af70 msgr2=0x7fc9e80ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:20.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.642+0000 7fc9e6ffd640 1 --2- 172.21.15.27:0/2041068197 >> v2:172.21.15.192:3300/0 conn(0x7fc9e815af70 0x7fc9e80ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.642+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 >> v1:172.21.15.192:6789/0 conn(0x7fc9e80cbce0 legacy=0x7fc9e80ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.642+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e80caf20 con 0x7fc9e80adc20 2023-12-19T18:17:20.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.642+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fc9e00019f0 con 0x7fc9e80adc20 2023-12-19T18:17:20.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.642+0000 7fc9e77fe640 1 --2- 172.21.15.27:0/2041068197 >> v2:172.21.15.192:3300/0 conn(0x7fc9e815af70 0x7fc9e80ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:20.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.642+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 >> v1:172.21.15.27:6789/0 conn(0x7fc9e80adc20 legacy=0x7fc9e806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.642+0000 7fc9e6ffd640 1 --2- 172.21.15.27:0/2041068197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9d4003970 0x7fc9d4003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.643+0000 7fc9e6ffd640 1 --2- 172.21.15.27:0/2041068197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9d40047c0 0x7fc9d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.643+0000 7fc9e6ffd640 1 --2- 172.21.15.27:0/2041068197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d4005180 0x7fc9d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.643+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80cac40 con 0x7fc9d4005180 2023-12-19T18:17:20.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.643+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d400c020 con 0x7fc9d40047c0 2023-12-19T18:17:20.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.643+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d400c1e0 con 0x7fc9d4003970 2023-12-19T18:17:20.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.643+0000 7fc9e7fff640 1 --2- 172.21.15.27:0/2041068197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d4005180 0x7fc9d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.643+0000 7fc9deffd640 1 --2- 172.21.15.27:0/2041068197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9d40047c0 0x7fc9d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.643+0000 7fc9e77fe640 1 --2- 172.21.15.27:0/2041068197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9d4003970 0x7fc9d4003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.643+0000 7fc9e7fff640 1 -- 172.21.15.27:0/2041068197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9d4003970 msgr2=0x7fc9d4003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.643+0000 7fc9e7fff640 1 --2- 172.21.15.27:0/2041068197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9d4003970 0x7fc9d4003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.644+0000 7fc9e7fff640 1 -- 172.21.15.27:0/2041068197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9d40047c0 msgr2=0x7fc9d4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.644+0000 7fc9e7fff640 1 --2- 172.21.15.27:0/2041068197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9d40047c0 0x7fc9d4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.644+0000 7fc9e7fff640 1 -- 172.21.15.27:0/2041068197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d4003520 con 0x7fc9d4005180 2023-12-19T18:17:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.644+0000 7fc9e7fff640 1 -- 172.21.15.27:0/2041068197 reap_dead start 2023-12-19T18:17:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.645+0000 7fc9e7fff640 1 --2- 172.21.15.27:0/2041068197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d4005180 0x7fc9d400b980 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000a830 tx=0x7fc9d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdea190ab07eae2d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.645+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00072d0 con 0x7fc9d4005180 2023-12-19T18:17:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.645+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9e002f070 con 0x7fc9d4005180 2023-12-19T18:17:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.645+0000 7fc9e6ffd640 1 -- 172.21.15.27:0/2041068197 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00076b0 con 0x7fc9d4005180 2023-12-19T18:17:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.646+0000 7fc9edf7e640 1 -- 172.21.15.27:0/2041068197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d4005180 msgr2=0x7fc9d400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.646+0000 7fc9edf7e640 1 --2- 172.21.15.27:0/2041068197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d4005180 0x7fc9d400b980 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000a830 tx=0x7fc9d40057f0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.646+0000 7fc9edf7e640 1 -- 172.21.15.27:0/2041068197 shutdown_connections 2023-12-19T18:17:20.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.646+0000 7fc9edf7e640 1 --2- 172.21.15.27:0/2041068197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9d4005180 0x7fc9d400b980 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.646+0000 7fc9edf7e640 1 -- 172.21.15.27:0/2041068197 >> 172.21.15.27:0/2041068197 conn(0x7fc9e806a330 msgr2=0x7fc9e80b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:20.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.646+0000 7fc9edf7e640 1 -- 172.21.15.27:0/2041068197 shutdown_connections 2023-12-19T18:17:20.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.646+0000 7fc9edf7e640 1 -- 172.21.15.27:0/2041068197 wait complete. 2023-12-19T18:17:20.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.648+0000 7fc9edf7e640 1 Processor -- start 2023-12-19T18:17:20.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.648+0000 7fc9edf7e640 1 -- start start 2023-12-19T18:17:20.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9edf7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9d4005180 0x7fc9e80c5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9edf7e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9e815b540 0x7fc9e81569c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9edf7e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9e80ac8d0 0x7fc9e80acd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9edf7e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80ad290 con 0x7fc9e815b540 2023-12-19T18:17:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9edf7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80c5910 con 0x7fc9d4005180 2023-12-19T18:17:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9edf7e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e80cc710 con 0x7fc9e80ac8d0 2023-12-19T18:17:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9e815b540 0x7fc9e81569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9deffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9e815b540 0x7fc9e81569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34188/0 (socket says 172.21.15.27:34188) 2023-12-19T18:17:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9deffd640 1 -- 172.21.15.27:0/177357709 learned_addr learned my addr 172.21.15.27:0/177357709 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9e77fe640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9d4005180 0x7fc9e80c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9e7fff640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9e80ac8d0 0x7fc9e80acd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9deffd640 1 -- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9e80ac8d0 msgr2=0x7fc9e80acd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9deffd640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9e80ac8d0 0x7fc9e80acd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9deffd640 1 -- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9d4005180 msgr2=0x7fc9e80c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9deffd640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9d4005180 0x7fc9e80c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9deffd640 1 -- 172.21.15.27:0/177357709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e815c050 con 0x7fc9e815b540 2023-12-19T18:17:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.650+0000 7fc9e7fff640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9e80ac8d0 0x7fc9e80acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.651+0000 7fc9e77fe640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9d4005180 0x7fc9e80c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:20.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.651+0000 7fc9deffd640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9e815b540 0x7fc9e81569c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000f5f0 tx=0x7fc9d000fab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.651+0000 7fc9e4ff9640 1 -- 172.21.15.27:0/177357709 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d00052e0 con 0x7fc9e815b540 2023-12-19T18:17:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.651+0000 7fc9e4ff9640 1 -- 172.21.15.27:0/177357709 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d0005440 con 0x7fc9e815b540 2023-12-19T18:17:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.651+0000 7fc9e4ff9640 1 -- 172.21.15.27:0/177357709 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0003830 con 0x7fc9e815b540 2023-12-19T18:17:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.651+0000 7fc9edf7e640 1 -- 172.21.15.27:0/177357709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9e815c2f0 con 0x7fc9e815b540 2023-12-19T18:17:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.651+0000 7fc9edf7e640 1 -- 172.21.15.27:0/177357709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e8156f00 con 0x7fc9e815b540 2023-12-19T18:17:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.653+0000 7fc9edf7e640 1 -- 172.21.15.27:0/177357709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9e8068e90 con 0x7fc9e815b540 2023-12-19T18:17:20.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.654+0000 7fc9e4ff9640 1 -- 172.21.15.27:0/177357709 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc9d0003990 con 0x7fc9e815b540 2023-12-19T18:17:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.654+0000 7fc9e4ff9640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc9c806a430 0x7fc9c806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:20.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.655+0000 7fc9e4ff9640 1 -- 172.21.15.27:0/177357709 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9d0088670 con 0x7fc9e815b540 2023-12-19T18:17:20.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.655+0000 7fc9e77fe640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc9c806a430 0x7fc9c806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:20.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.656+0000 7fc9e77fe640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc9c806a430 0x7fc9c806c8f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009b10 tx=0x7fc9d8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:20.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.659+0000 7fc9e4ff9640 1 -- 172.21.15.27:0/177357709 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc9d0053cf0 con 0x7fc9e815b540 2023-12-19T18:17:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.892+0000 7fc9edf7e640 1 -- 172.21.15.27:0/177357709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9e815a430 con 0x7fc9e815b540 2023-12-19T18:17:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.894+0000 7fc9e4ff9640 1 -- 172.21.15.27:0/177357709 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fc9d0057310 con 0x7fc9e815b540 2023-12-19T18:17:20.894 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:20.894 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/930391176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":930391176},{"type":"v1","addr":"172.21.15.192:6837","nonce":930391176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5179,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3606564673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3606564673},{"type":"v1","addr":"172.21.15.192:6836","nonce":3606564673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3604117097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3604117097},{"type":"v1","addr":"172.21.15.27:6834","nonce":3604117097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:14.250689+0000","modified":"2023-12-19T18:17:17.758670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3814458529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3814458529},{"type":"v1","addr":"172.21.15.27:6835","nonce":3814458529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T18:17:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-19T18:17:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.896+0000 7fc9edf7e640 1 -- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc9c806a430 msgr2=0x7fc9c806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.896+0000 7fc9edf7e640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc9c806a430 0x7fc9c806c8f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fc9d8009b10 tx=0x7fc9d8008040 comp rx=0 tx=0).stop 2023-12-19T18:17:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.896+0000 7fc9edf7e640 1 -- 172.21.15.27:0/177357709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9e815b540 msgr2=0x7fc9e81569c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.896+0000 7fc9edf7e640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9e815b540 0x7fc9e81569c0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc9d000f5f0 tx=0x7fc9d000fab0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.896+0000 7fc9edf7e640 1 -- 172.21.15.27:0/177357709 shutdown_connections 2023-12-19T18:17:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.896+0000 7fc9edf7e640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc9c806a430 0x7fc9c806c8f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.896+0000 7fc9edf7e640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc9e80ac8d0 0x7fc9e80acd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.896+0000 7fc9edf7e640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9e815b540 0x7fc9e81569c0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.896+0000 7fc9edf7e640 1 --2- 172.21.15.27:0/177357709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc9d4005180 0x7fc9e80c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:20.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.896+0000 7fc9edf7e640 1 -- 172.21.15.27:0/177357709 >> 172.21.15.27:0/177357709 conn(0x7fc9e806a330 msgr2=0x7fc9e80af970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:20.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.896+0000 7fc9edf7e640 1 -- 172.21.15.27:0/177357709 shutdown_connections 2023-12-19T18:17:20.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:20.897+0000 7fc9edf7e640 1 -- 172.21.15.27:0/177357709 wait complete. 2023-12-19T18:17:21.318 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:17:14.250689+0000', 'modified': '2023-12-19T18:17:17.758670+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5679}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5679': {'gid': 5679, 'name': 'c', 'rank': 0, 'incarnation': 19, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/3814458529', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6833', 'nonce': 3814458529}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 3814458529}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:17:21.318 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:17:21.318 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-19T18:17:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.555+0000 7fcf767a9640 1 Processor -- start 2023-12-19T18:17:21.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.556+0000 7fcf767a9640 1 -- start start 2023-12-19T18:17:21.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.557+0000 7fcf767a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf7015b9e0 0x7fcf700c9410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.558+0000 7fcf767a9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcf7006f8e0 0x7fcf7006fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.558+0000 7fcf767a9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf7006b310 con 0x7fcf70161a40 2023-12-19T18:17:21.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.558+0000 7fcf767a9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf700702a0 con 0x7fcf7006f8e0 2023-12-19T18:17:21.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.558+0000 7fcf767a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf700c9950 con 0x7fcf7015b9e0 2023-12-19T18:17:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.558+0000 7fcf74fa6640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcf7006f8e0 0x7fcf7006fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.558+0000 7fcf75fa8640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fcf70161a40 0x7fcf70161ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:55562/0 (socket says 172.21.15.27:55562) 2023-12-19T18:17:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.558+0000 7fcf75fa8640 1 -- 172.21.15.27:0/1210821656 learned_addr learned my addr 172.21.15.27:0/1210821656 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.558+0000 7fcf757a7640 1 --2- 172.21.15.27:0/1210821656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf7015b9e0 0x7fcf700c9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.559+0000 7fcf757a7640 1 -- 172.21.15.27:0/1210821656 >> v2:172.21.15.192:3300/0 conn(0x7fcf7006f8e0 msgr2=0x7fcf7006fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:21.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.559+0000 7fcf757a7640 1 --2- 172.21.15.27:0/1210821656 >> v2:172.21.15.192:3300/0 conn(0x7fcf7006f8e0 0x7fcf7006fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.559+0000 7fcf757a7640 1 -- 172.21.15.27:0/1210821656 >> v1:172.21.15.192:6789/0 conn(0x7fcf70161a40 legacy=0x7fcf70161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.559+0000 7fcf757a7640 1 -- 172.21.15.27:0/1210821656 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf70162790 con 0x7fcf7015b9e0 2023-12-19T18:17:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.559+0000 7fcf74fa6640 1 --2- 172.21.15.27:0/1210821656 >> v2:172.21.15.192:3300/0 conn(0x7fcf7006f8e0 0x7fcf7006fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:17:21.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.561+0000 7fcf757a7640 1 --2- 172.21.15.27:0/1210821656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf7015b9e0 0x7fcf700c9410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0097a0 tx=0x7fcf6c02b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d58a88b19e0dd0d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.562+0000 7fcf67fff640 1 -- 172.21.15.27:0/1210821656 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c02d040 con 0x7fcf7015b9e0 2023-12-19T18:17:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.562+0000 7fcf67fff640 1 -- 172.21.15.27:0/1210821656 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c031810 con 0x7fcf7015b9e0 2023-12-19T18:17:21.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.562+0000 7fcf67fff640 1 -- 172.21.15.27:0/1210821656 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c031ba0 con 0x7fcf7015b9e0 2023-12-19T18:17:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.562+0000 7fcf767a9640 1 -- 172.21.15.27:0/1210821656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf7015b9e0 msgr2=0x7fcf700c9410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.562+0000 7fcf767a9640 1 --2- 172.21.15.27:0/1210821656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf7015b9e0 0x7fcf700c9410 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c0097a0 tx=0x7fcf6c02b8b0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.562+0000 7fcf767a9640 1 -- 172.21.15.27:0/1210821656 shutdown_connections 2023-12-19T18:17:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.562+0000 7fcf767a9640 1 --2- 172.21.15.27:0/1210821656 >> v2:172.21.15.192:3300/0 conn(0x7fcf7006f8e0 0x7fcf7006fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.562+0000 7fcf767a9640 1 --2- 172.21.15.27:0/1210821656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf7015b9e0 0x7fcf700c9410 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.562+0000 7fcf767a9640 1 -- 172.21.15.27:0/1210821656 >> 172.21.15.27:0/1210821656 conn(0x7fcf7006a7d0 msgr2=0x7fcf700ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.563+0000 7fcf767a9640 1 -- 172.21.15.27:0/1210821656 shutdown_connections 2023-12-19T18:17:21.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.563+0000 7fcf767a9640 1 -- 172.21.15.27:0/1210821656 wait complete. 2023-12-19T18:17:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.564+0000 7fcf767a9640 1 Processor -- start 2023-12-19T18:17:21.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.565+0000 7fcf767a9640 1 -- start start 2023-12-19T18:17:21.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf767a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf7006f8e0 0x7fcf70159c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf767a9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf7015b9e0 0x7fcf70155670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf767a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70161a40 0x7fcf70157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf767a9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf70158980 con 0x7fcf7015b9e0 2023-12-19T18:17:21.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf767a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf701597e0 con 0x7fcf70161a40 2023-12-19T18:17:21.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf767a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf70158bd0 con 0x7fcf7006f8e0 2023-12-19T18:17:21.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf74fa6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf7015b9e0 0x7fcf70155670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf74fa6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf7015b9e0 0x7fcf70155670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34204/0 (socket says 172.21.15.27:34204) 2023-12-19T18:17:21.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf74fa6640 1 -- 172.21.15.27:0/4180185295 learned_addr learned my addr 172.21.15.27:0/4180185295 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf757a7640 1 --2- 172.21.15.27:0/4180185295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf7006f8e0 0x7fcf70159c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf74fa6640 1 -- 172.21.15.27:0/4180185295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf7006f8e0 msgr2=0x7fcf70159c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf74fa6640 1 --2- 172.21.15.27:0/4180185295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf7006f8e0 0x7fcf70159c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf74fa6640 1 -- 172.21.15.27:0/4180185295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70161a40 msgr2=0x7fcf70157e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf74fa6640 1 --2- 172.21.15.27:0/4180185295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70161a40 0x7fcf70157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.566+0000 7fcf74fa6640 1 -- 172.21.15.27:0/4180185295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf60009400 con 0x7fcf7015b9e0 2023-12-19T18:17:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.567+0000 7fcf74fa6640 1 --2- 172.21.15.27:0/4180185295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf7015b9e0 0x7fcf70155670 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000e990 tx=0x7fcf6000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.567+0000 7fcf65ffb640 1 -- 172.21.15.27:0/4180185295 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6000c9e0 con 0x7fcf7015b9e0 2023-12-19T18:17:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.567+0000 7fcf65ffb640 1 -- 172.21.15.27:0/4180185295 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6000cb40 con 0x7fcf7015b9e0 2023-12-19T18:17:21.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.567+0000 7fcf65ffb640 1 -- 172.21.15.27:0/4180185295 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6000ced0 con 0x7fcf7015b9e0 2023-12-19T18:17:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.567+0000 7fcf767a9640 1 -- 172.21.15.27:0/4180185295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf6c009400 con 0x7fcf7015b9e0 2023-12-19T18:17:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.567+0000 7fcf767a9640 1 -- 172.21.15.27:0/4180185295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf70159190 con 0x7fcf7015b9e0 2023-12-19T18:17:21.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.569+0000 7fcf65ffb640 1 -- 172.21.15.27:0/4180185295 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcf60013470 con 0x7fcf7015b9e0 2023-12-19T18:17:21.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.571+0000 7fcf65ffb640 1 --2- 172.21.15.27:0/4180185295 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf5406a430 0x7fcf5406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.571+0000 7fcf65ffb640 1 -- 172.21.15.27:0/4180185295 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf60088bd0 con 0x7fcf7015b9e0 2023-12-19T18:17:21.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.572+0000 7fcf757a7640 1 --2- 172.21.15.27:0/4180185295 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf5406a430 0x7fcf5406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.572+0000 7fcf7a57e740 1 Processor -- start 2023-12-19T18:17:21.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.572+0000 7fcf7a57e740 1 -- start start 2023-12-19T18:17:21.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.573+0000 7fcf7a57e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 0x5639a9c8e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.573+0000 7fcf7a57e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c5e090 0x5639a9c8b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.573+0000 7fcf7a57e740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9c8b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.573+0000 7fcf7a57e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a9c39650 con 0x5639a9c5e090 2023-12-19T18:17:21.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.573+0000 7fcf7a57e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a9bce000 con 0x5639a9c8e110 2023-12-19T18:17:21.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.573+0000 7fcf7a57e740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639a9bcb790 con 0x5639a9c61c90 2023-12-19T18:17:21.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.573+0000 7fcf757a7640 1 --2- 172.21.15.27:0/4180185295 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf5406a430 0x7fcf5406c8f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c036000 tx=0x7fcf6c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:21.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.573+0000 7fcf74fa6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c5e090 0x5639a9c8b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.573+0000 7fcf74fa6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c5e090 0x5639a9c8b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34210/0 (socket says 172.21.15.27:34210) 2023-12-19T18:17:21.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.573+0000 7fcf74fa6640 1 -- 172.21.15.27:0/703231193 learned_addr learned my addr 172.21.15.27:0/703231193 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:21.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.574+0000 7fcf757a7640 1 --2- 172.21.15.27:0/703231193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 0x5639a9c8e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.574+0000 7fcf75fa8640 1 --2- 172.21.15.27:0/703231193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9c8b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.574+0000 7fcf74fa6640 1 -- 172.21.15.27:0/703231193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 msgr2=0x5639a9c8b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:21.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.574+0000 7fcf74fa6640 1 --2- 172.21.15.27:0/703231193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9c8b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.574+0000 7fcf74fa6640 1 -- 172.21.15.27:0/703231193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 msgr2=0x5639a9c8e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:21.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.574+0000 7fcf74fa6640 1 --2- 172.21.15.27:0/703231193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 0x5639a9c8e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.574+0000 7fcf74fa6640 1 -- 172.21.15.27:0/703231193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639a9b8e710 con 0x5639a9c5e090 2023-12-19T18:17:21.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.574+0000 7fcf74fa6640 1 --2- 172.21.15.27:0/703231193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c5e090 0x5639a9c8b380 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcf6005f920 tx=0x7fcf60057e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d072261d2a8341af server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:21.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.574+0000 7fcf52ffd640 1 -- 172.21.15.27:0/703231193 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf60056210 con 0x5639a9c5e090 2023-12-19T18:17:21.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.574+0000 7fcf52ffd640 1 -- 172.21.15.27:0/703231193 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf60054580 con 0x5639a9c5e090 2023-12-19T18:17:21.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.574+0000 7fcf52ffd640 1 -- 172.21.15.27:0/703231193 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6005fbc0 con 0x5639a9c5e090 2023-12-19T18:17:21.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.575+0000 7fcf7a57e740 1 -- 172.21.15.27:0/703231193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c5e090 msgr2=0x5639a9c8b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.575+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/703231193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c5e090 0x5639a9c8b380 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcf6005f920 tx=0x7fcf60057e70 comp rx=0 tx=0).stop 2023-12-19T18:17:21.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.575+0000 7fcf7a57e740 1 -- 172.21.15.27:0/703231193 shutdown_connections 2023-12-19T18:17:21.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.575+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/703231193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9c8b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.575+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/703231193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c5e090 0x5639a9c8b380 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.575+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/703231193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 0x5639a9c8e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.575+0000 7fcf7a57e740 1 -- 172.21.15.27:0/703231193 >> 172.21.15.27:0/703231193 conn(0x5639a9c603a0 msgr2=0x5639a9ca8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:21.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.575+0000 7fcf7a57e740 1 -- 172.21.15.27:0/703231193 shutdown_connections 2023-12-19T18:17:21.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.575+0000 7fcf7a57e740 1 -- 172.21.15.27:0/703231193 wait complete. 2023-12-19T18:17:21.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.576+0000 7fcf7a57e740 1 -- start start 2023-12-19T18:17:21.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.576+0000 7fcf7a57e740 1 Processor -- start 2023-12-19T18:17:21.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.577+0000 7fcf7a57e740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9c9f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.577+0000 7fcf7a57e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 0x5639a9c83e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.577+0000 7fcf7a57e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 0x5639a9c88530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.577+0000 7fcf7a57e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a9bc7080 con 0x5639a9c720a0 2023-12-19T18:17:21.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.577+0000 7fcf7a57e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a9b68850 con 0x5639a9c8e110 2023-12-19T18:17:21.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.577+0000 7fcf7a57e740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639a9b684f0 con 0x5639a9c61c90 2023-12-19T18:17:21.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.578+0000 7fcf7a57e740 10 client.0 _ll_register_callbacks cb 0x5639a9bc09c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:17:21.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.578+0000 7fcf75fa8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 0x5639a9c88530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.578+0000 7fcf75fa8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 0x5639a9c88530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34216/0 (socket says 172.21.15.27:34216) 2023-12-19T18:17:21.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.578+0000 7fcf75fa8640 1 -- 172.21.15.27:0/1306334441 learned_addr learned my addr 172.21.15.27:0/1306334441 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:21.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.578+0000 7fcf757a7640 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9c9f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.578+0000 7fcf74fa6640 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 0x5639a9c83e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.578+0000 7fcf75fa8640 1 -- 172.21.15.27:0/1306334441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 msgr2=0x5639a9c9f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:21.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.578+0000 7fcf75fa8640 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9c9f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.578+0000 7fcf75fa8640 1 -- 172.21.15.27:0/1306334441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 msgr2=0x5639a9c83e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:21.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.578+0000 7fcf75fa8640 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 0x5639a9c83e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.578+0000 7fcf75fa8640 1 -- 172.21.15.27:0/1306334441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf70159190 con 0x5639a9c720a0 2023-12-19T18:17:21.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.578+0000 7fcf757a7640 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9c9f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:21.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.579+0000 7fcf75fa8640 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 0x5639a9c88530 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800d800 tx=0x7fcf6800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb887bec4d39c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:21.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.579+0000 7fcf4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:17:21.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.579+0000 7fcf4bfff640 1 -- 172.21.15.27:0/1306334441 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6800bac0 con 0x5639a9c720a0 2023-12-19T18:17:21.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.579+0000 7fcf4bfff640 1 -- 172.21.15.27:0/1306334441 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf6800bc20 con 0x5639a9c720a0 2023-12-19T18:17:21.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.579+0000 7fcf7a57e740 1 -- 172.21.15.27:0/1306334441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639a9bf15d0 con 0x5639a9c720a0 2023-12-19T18:17:21.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.579+0000 7fcf4bfff640 1 -- 172.21.15.27:0/1306334441 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf68001b30 con 0x5639a9c720a0 2023-12-19T18:17:21.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.579+0000 7fcf7a57e740 10 client.5811 fetch_fsmap learned FSMap version 23 2023-12-19T18:17:21.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.579+0000 7fcf7a57e740 1 -- 172.21.15.27:0/1306334441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5639a9bc5700 con 0x5639a9c720a0 2023-12-19T18:17:21.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.579+0000 7fcf4bfff640 1 -- 172.21.15.27:0/1306334441 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5639a9bf15d0 con 0x5639a9c720a0 2023-12-19T18:17:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.580+0000 7fcf4bfff640 1 -- 172.21.15.27:0/1306334441 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcf68012410 con 0x5639a9c720a0 2023-12-19T18:17:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.580+0000 7fcf7a57e740 10 client.5811 fetch_fsmap finished waiting for FSMap version 23 2023-12-19T18:17:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.580+0000 7fcf7a57e740 10 client.5811 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:17:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.580+0000 7fcf7a57e740 20 client.5811 populate_metadata read hostname 'smithi027' 2023-12-19T18:17:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.580+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.27:6833/3814458529,v1:172.21.15.27:6835/3814458529] conn(0x5639a9c74100 0x5639a9c744e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.580+0000 7fcf7a57e740 4 client.5811 mds_command: new command op to 5679 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:17:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.580+0000 7fcf7a57e740 1 -- 172.21.15.27:0/1306334441 --> [v2:172.21.15.27:6833/3814458529,v1:172.21.15.27:6835/3814458529] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639a9bb98d0 con 0x5639a9c74100 2023-12-19T18:17:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.580+0000 7fcf757a7640 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.27:6833/3814458529,v1:172.21.15.27:6835/3814458529] conn(0x5639a9c74100 0x5639a9c744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.581+0000 7fcf757a7640 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.27:6833/3814458529,v1:172.21.15.27:6835/3814458529] conn(0x5639a9c74100 0x5639a9c744e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c2f4b18f91a30d9 server_cookie=f69c9ca06d6f36fe in_seq=0 out_seq=0 2023-12-19T18:17:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.581+0000 7fcf4bfff640 10 client.5811 ms_handle_connect on v2:172.21.15.27:6833/3814458529 2023-12-19T18:17:21.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.582+0000 7fcf4bfff640 1 -- 172.21.15.27:0/1306334441 <== mds.0 v2:172.21.15.27:6833/3814458529 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5639a9bb98d0 con 0x5639a9c74100 2023-12-19T18:17:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.582+0000 7fcf4bfff640 10 client.5811 handle_command_reply: tid=0 2023-12-19T18:17:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.582+0000 7fcf7a57e740 1 client.5811 shutdown 2023-12-19T18:17:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.582+0000 7fcf7a57e740 1 -- 172.21.15.27:0/1306334441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 msgr2=0x5639a9c88530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.582+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 0x5639a9c88530 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fcf6800d800 tx=0x7fcf6800dcc0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.582+0000 7fcf7a57e740 1 -- 172.21.15.27:0/1306334441 shutdown_connections 2023-12-19T18:17:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.582+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 0x5639a9c88530 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.582+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 0x5639a9c83e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.582+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.27:6833/3814458529,v1:172.21.15.27:6835/3814458529] conn(0x5639a9c74100 0x5639a9c744e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.582+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/1306334441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9c9f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.582+0000 7fcf4bfff640 0 client.5811 ms_handle_reset on v2:172.21.15.27:6833/3814458529 2023-12-19T18:17:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.582+0000 7fcf7a57e740 1 -- 172.21.15.27:0/1306334441 >> 172.21.15.27:0/1306334441 conn(0x5639a9c5e090 msgr2=0x5639a9c62540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.582+0000 7fcf7a57e740 1 -- 172.21.15.27:0/1306334441 shutdown_connections 2023-12-19T18:17:21.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.583+0000 7fcf7a57e740 1 -- 172.21.15.27:0/1306334441 wait complete. 2023-12-19T18:17:21.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.583+0000 7fcf7a57e740 20 client.5811 trim_cache size 0 max 16384 2023-12-19T18:17:21.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.599+0000 7fcf7a57e740 1 Processor -- start 2023-12-19T18:17:21.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.599+0000 7fcf7a57e740 1 -- start start 2023-12-19T18:17:21.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.600+0000 7fcf7a57e740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9c9ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.600+0000 7fcf7a57e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 0x5639a9c73b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.600+0000 7fcf7a57e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 0x5639a9c62160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.600+0000 7fcf7a57e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a992ef20 con 0x5639a9c720a0 2023-12-19T18:17:21.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.600+0000 7fcf7a57e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a9bc6220 con 0x5639a9c8e110 2023-12-19T18:17:21.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.600+0000 7fcf7a57e740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639a9c91720 con 0x5639a9c61c90 2023-12-19T18:17:21.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.600+0000 7fcf74fa6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 0x5639a9c73b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.601+0000 7fcf74fa6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 0x5639a9c73b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34220/0 (socket says 172.21.15.27:34220) 2023-12-19T18:17:21.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.601+0000 7fcf74fa6640 1 -- 172.21.15.27:0/2019210142 learned_addr learned my addr 172.21.15.27:0/2019210142 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:21.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.601+0000 7fcf74fa6640 1 -- 172.21.15.27:0/2019210142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 msgr2=0x5639a9c9ea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:21.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.601+0000 7fcf74fa6640 1 --2- 172.21.15.27:0/2019210142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9c9ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.601+0000 7fcf74fa6640 1 -- 172.21.15.27:0/2019210142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 msgr2=0x5639a9c62160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:21.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.601+0000 7fcf75fa8640 1 --2- 172.21.15.27:0/2019210142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 0x5639a9c62160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.601+0000 7fcf74fa6640 1 --2- 172.21.15.27:0/2019210142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 0x5639a9c62160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.601+0000 7fcf74fa6640 1 -- 172.21.15.27:0/2019210142 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639a9bfb280 con 0x5639a9c720a0 2023-12-19T18:17:21.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.601+0000 7fcf74fa6640 1 --2- 172.21.15.27:0/2019210142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 0x5639a9c73b00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcf6005be60 tx=0x7fcf60059000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a84b5d8b36b08fe server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:21.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.602+0000 7fcf537fe640 1 -- 172.21.15.27:0/2019210142 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf600546f0 con 0x5639a9c720a0 2023-12-19T18:17:21.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.602+0000 7fcf537fe640 1 -- 172.21.15.27:0/2019210142 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6005fe50 con 0x5639a9c720a0 2023-12-19T18:17:21.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.602+0000 7fcf537fe640 1 -- 172.21.15.27:0/2019210142 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6005f9c0 con 0x5639a9c720a0 2023-12-19T18:17:21.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.602+0000 7fcf7a57e740 1 -- 172.21.15.27:0/2019210142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 msgr2=0x5639a9c73b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.602+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/2019210142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 0x5639a9c73b00 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcf6005be60 tx=0x7fcf60059000 comp rx=0 tx=0).stop 2023-12-19T18:17:21.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.602+0000 7fcf7a57e740 1 -- 172.21.15.27:0/2019210142 shutdown_connections 2023-12-19T18:17:21.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.602+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/2019210142 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c8e110 0x5639a9c62160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.602+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/2019210142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c720a0 0x5639a9c73b00 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.602+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/2019210142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9c9ea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.602+0000 7fcf7a57e740 1 -- 172.21.15.27:0/2019210142 >> 172.21.15.27:0/2019210142 conn(0x5639a9c5e090 msgr2=0x5639a9c82830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:21.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.602+0000 7fcf7a57e740 1 -- 172.21.15.27:0/2019210142 shutdown_connections 2023-12-19T18:17:21.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.602+0000 7fcf7a57e740 1 -- 172.21.15.27:0/2019210142 wait complete. 2023-12-19T18:17:21.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.603+0000 7fcf7a57e740 1 -- start start 2023-12-19T18:17:21.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.603+0000 7fcf7a57e740 1 Processor -- start 2023-12-19T18:17:21.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.604+0000 7fcf7a57e740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9d099c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.604+0000 7fcf7a57e740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c720a0 0x5639a9cbb0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.604+0000 7fcf7a57e740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c8e110 0x5639a9d0f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.604+0000 7fcf7a57e740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a9c5d8b0 con 0x5639a9c8e110 2023-12-19T18:17:21.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.604+0000 7fcf7a57e740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639a9d10450 con 0x5639a9c720a0 2023-12-19T18:17:21.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.604+0000 7fcf7a57e740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639a9c627d0 con 0x5639a9c61c90 2023-12-19T18:17:21.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.604+0000 7fcf7a57e740 10 client.0 _ll_register_callbacks cb 0x7fcf680018e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:17:21.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.605+0000 7fcf75fa8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c8e110 0x5639a9d0f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.605+0000 7fcf75fa8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c8e110 0x5639a9d0f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34234/0 (socket says 172.21.15.27:34234) 2023-12-19T18:17:21.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.605+0000 7fcf75fa8640 1 -- 172.21.15.27:0/3513608676 learned_addr learned my addr 172.21.15.27:0/3513608676 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:21.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.605+0000 7fcf74fa6640 1 --2- 172.21.15.27:0/3513608676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c720a0 0x5639a9cbb0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.605+0000 7fcf757a7640 1 --2- 172.21.15.27:0/3513608676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9d099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.605+0000 7fcf75fa8640 1 -- 172.21.15.27:0/3513608676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 msgr2=0x5639a9d099c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:21.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.605+0000 7fcf75fa8640 1 --2- 172.21.15.27:0/3513608676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9d099c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.605+0000 7fcf75fa8640 1 -- 172.21.15.27:0/3513608676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c720a0 msgr2=0x5639a9cbb0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.605+0000 7fcf75fa8640 1 --2- 172.21.15.27:0/3513608676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c720a0 0x5639a9cbb0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.605+0000 7fcf75fa8640 1 -- 172.21.15.27:0/3513608676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5639a9b8e710 con 0x5639a9c8e110 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.606+0000 7fcf75fa8640 1 --2- 172.21.15.27:0/3513608676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c8e110 0x5639a9d0f890 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcf68012650 tx=0x7fcf68005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=913e6a20ec5bf111 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.606+0000 7fcf4bfff640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.606+0000 7fcf4bfff640 1 -- 172.21.15.27:0/3513608676 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf68028070 con 0x5639a9c8e110 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.606+0000 7fcf7a57e740 1 -- 172.21.15.27:0/3513608676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf60054580 con 0x5639a9c8e110 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.606+0000 7fcf4bfff640 1 -- 172.21.15.27:0/3513608676 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf68012970 con 0x5639a9c8e110 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.606+0000 7fcf4bfff640 1 -- 172.21.15.27:0/3513608676 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf6800b4a0 con 0x5639a9c8e110 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.606+0000 7fcf7a57e740 10 client.5817 fetch_fsmap learned FSMap version 23 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.606+0000 7fcf7a57e740 1 -- 172.21.15.27:0/3513608676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=2+}) v3 -- 0x5639a9b67a60 con 0x5639a9c8e110 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.606+0000 7fcf4bfff640 1 -- 172.21.15.27:0/3513608676 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf60054580 con 0x5639a9c8e110 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.607+0000 7fcf4bfff640 1 -- 172.21.15.27:0/3513608676 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 23) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcf6800b8c0 con 0x5639a9c8e110 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.607+0000 7fcf7a57e740 10 client.5817 fetch_fsmap finished waiting for FSMap version 23 2023-12-19T18:17:21.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.607+0000 7fcf7a57e740 10 client.5817 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:17:21.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.607+0000 7fcf7a57e740 20 client.5817 populate_metadata read hostname 'smithi027' 2023-12-19T18:17:21.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.607+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/3513608676 >> [v2:172.21.15.27:6833/3814458529,v1:172.21.15.27:6835/3814458529] conn(0x5639a9c905c0 0x5639a9c909a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.607+0000 7fcf7a57e740 4 client.5817 mds_command: new command op to 5679 tid=0[{"prefix": "status"}] 2023-12-19T18:17:21.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.607+0000 7fcf7a57e740 1 -- 172.21.15.27:0/3513608676 --> [v2:172.21.15.27:6833/3814458529,v1:172.21.15.27:6835/3814458529] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5639a9bc0100 con 0x5639a9c905c0 2023-12-19T18:17:21.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.607+0000 7fcf757a7640 1 --2- 172.21.15.27:0/3513608676 >> [v2:172.21.15.27:6833/3814458529,v1:172.21.15.27:6835/3814458529] conn(0x5639a9c905c0 0x5639a9c909a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf757a7640 1 --2- 172.21.15.27:0/3513608676 >> [v2:172.21.15.27:6833/3814458529,v1:172.21.15.27:6835/3814458529] conn(0x5639a9c905c0 0x5639a9c909a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e38028c657f02c3 server_cookie=7901a71cd2ffe9c0 in_seq=0 out_seq=0 2023-12-19T18:17:21.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf4bfff640 10 client.5817 ms_handle_connect on v2:172.21.15.27:6833/3814458529 2023-12-19T18:17:21.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf4bfff640 1 -- 172.21.15.27:0/3513608676 <== mds.0 v2:172.21.15.27:6833/3814458529 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x5639a9bc0100 con 0x5639a9c905c0 2023-12-19T18:17:21.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf4bfff640 10 client.5817 handle_command_reply: tid=0 2023-12-19T18:17:21.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf7a57e740 1 client.5817 shutdown 2023-12-19T18:17:21.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf7a57e740 1 -- 172.21.15.27:0/3513608676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c8e110 msgr2=0x5639a9d0f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/3513608676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c8e110 0x5639a9d0f890 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcf68012650 tx=0x7fcf68005b70 comp rx=0 tx=0).stop 2023-12-19T18:17:21.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf7a57e740 1 -- 172.21.15.27:0/3513608676 shutdown_connections 2023-12-19T18:17:21.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/3513608676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5639a9c8e110 0x5639a9d0f890 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/3513608676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5639a9c720a0 0x5639a9cbb0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/3513608676 >> [v2:172.21.15.27:6833/3814458529,v1:172.21.15.27:6835/3814458529] conn(0x5639a9c905c0 0x5639a9c909a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf7a57e740 1 --2- 172.21.15.27:0/3513608676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5639a9c61c90 0x5639a9d099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf7a57e740 1 -- 172.21.15.27:0/3513608676 >> 172.21.15.27:0/3513608676 conn(0x5639a9c5e090 msgr2=0x5639a9ca9260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:21.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf4bfff640 0 client.5817 ms_handle_reset on v2:172.21.15.27:6833/3814458529 2023-12-19T18:17:21.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf7a57e740 1 -- 172.21.15.27:0/3513608676 shutdown_connections 2023-12-19T18:17:21.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf7a57e740 1 -- 172.21.15.27:0/3513608676 wait complete. 2023-12-19T18:17:21.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.608+0000 7fcf7a57e740 20 client.5817 trim_cache size 0 max 16384 2023-12-19T18:17:21.613 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:17:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:17:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-12-19T18:17:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5679, 2023-12-19T18:17:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-19T18:17:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-19T18:17:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-19T18:17:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 7.3525118440000004, 2023-12-19T18:17:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 23, 2023-12-19T18:17:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 50, 2023-12-19T18:17:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 50, 2023-12-19T18:17:21.613 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 12.624568606 2023-12-19T18:17:21.613 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:17:21.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.609+0000 7fcf767a9640 1 -- 172.21.15.27:0/4180185295 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf5406a430 msgr2=0x7fcf5406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.609+0000 7fcf767a9640 1 --2- 172.21.15.27:0/4180185295 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf5406a430 0x7fcf5406c8f0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c036000 tx=0x7fcf6c03c040 comp rx=0 tx=0).stop 2023-12-19T18:17:21.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.609+0000 7fcf767a9640 1 -- 172.21.15.27:0/4180185295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf7015b9e0 msgr2=0x7fcf70155670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.609+0000 7fcf767a9640 1 --2- 172.21.15.27:0/4180185295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf7015b9e0 0x7fcf70155670 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000e990 tx=0x7fcf6000ee50 comp rx=0 tx=0).stop 2023-12-19T18:17:21.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.609+0000 7fcf767a9640 1 -- 172.21.15.27:0/4180185295 shutdown_connections 2023-12-19T18:17:21.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.609+0000 7fcf767a9640 1 --2- 172.21.15.27:0/4180185295 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf5406a430 0x7fcf5406c8f0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.609+0000 7fcf767a9640 1 --2- 172.21.15.27:0/4180185295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf70161a40 0x7fcf70157e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.609+0000 7fcf767a9640 1 --2- 172.21.15.27:0/4180185295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf7015b9e0 0x7fcf70155670 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.609+0000 7fcf767a9640 1 --2- 172.21.15.27:0/4180185295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf7006f8e0 0x7fcf70159c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.609+0000 7fcf767a9640 1 -- 172.21.15.27:0/4180185295 >> 172.21.15.27:0/4180185295 conn(0x7fcf7006a7d0 msgr2=0x7fcf70164d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:21.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.609+0000 7fcf767a9640 1 -- 172.21.15.27:0/4180185295 shutdown_connections 2023-12-19T18:17:21.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.609+0000 7fcf767a9640 1 -- 172.21.15.27:0/4180185295 wait complete. 2023-12-19T18:17:21.626 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:17:21.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.837+0000 7f194bfff640 1 Processor -- start 2023-12-19T18:17:21.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.837+0000 7f194bfff640 1 -- start start 2023-12-19T18:17:21.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.839+0000 7f194bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f194c0c50a0 0x7f194c0afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.839+0000 7f194bfff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c0c54f0 con 0x7f194c1553a0 2023-12-19T18:17:21.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.839+0000 7f194bfff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f194c0cb350 con 0x7f194c15ee30 2023-12-19T18:17:21.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.839+0000 7f194bfff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f194c0cbcf0 con 0x7f194c0c50a0 2023-12-19T18:17:21.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.839+0000 7f194b7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f194c0c50a0 0x7f194c0afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.839+0000 7f194affd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f194c1553a0 0x7f194c15e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41824/0 (socket says 172.21.15.27:41824) 2023-12-19T18:17:21.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.839+0000 7f194affd640 1 -- 172.21.15.27:0/2779669742 learned_addr learned my addr 172.21.15.27:0/2779669742 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:21.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.840+0000 7f194b7fe640 1 -- 172.21.15.27:0/2779669742 >> v1:172.21.15.192:6789/0 conn(0x7f194c15ee30 legacy=0x7f194c0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:21.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.840+0000 7f194b7fe640 1 -- 172.21.15.27:0/2779669742 >> v1:172.21.15.27:6789/0 conn(0x7f194c1553a0 legacy=0x7f194c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.840+0000 7f194b7fe640 1 -- 172.21.15.27:0/2779669742 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f194c0cb630 con 0x7f194c0c50a0 2023-12-19T18:17:21.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.841+0000 7f194b7fe640 1 --2- 172.21.15.27:0/2779669742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f194c0c50a0 0x7f194c0afeb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1944009d90 tx=0x7f194402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7516847a5e8cca3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:21.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.841+0000 7f194a7fc640 1 -- 172.21.15.27:0/2779669742 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194402cbd0 con 0x7f194c0c50a0 2023-12-19T18:17:21.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.841+0000 7f194a7fc640 1 -- 172.21.15.27:0/2779669742 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194402cd30 con 0x7f194c0c50a0 2023-12-19T18:17:21.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.841+0000 7f194a7fc640 1 -- 172.21.15.27:0/2779669742 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194402cea0 con 0x7f194c0c50a0 2023-12-19T18:17:21.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.842+0000 7f194bfff640 1 -- 172.21.15.27:0/2779669742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f194c0c50a0 msgr2=0x7f194c0afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.842+0000 7f194bfff640 1 --2- 172.21.15.27:0/2779669742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f194c0c50a0 0x7f194c0afeb0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f1944009d90 tx=0x7f194402be20 comp rx=0 tx=0).stop 2023-12-19T18:17:21.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.842+0000 7f194bfff640 1 -- 172.21.15.27:0/2779669742 shutdown_connections 2023-12-19T18:17:21.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.842+0000 7f194bfff640 1 --2- 172.21.15.27:0/2779669742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f194c0c50a0 0x7f194c0afeb0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.842+0000 7f194bfff640 1 -- 172.21.15.27:0/2779669742 >> 172.21.15.27:0/2779669742 conn(0x7f194c069c70 msgr2=0x7f194c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:21.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.842+0000 7f194bfff640 1 -- 172.21.15.27:0/2779669742 shutdown_connections 2023-12-19T18:17:21.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.842+0000 7f194bfff640 1 -- 172.21.15.27:0/2779669742 wait complete. 2023-12-19T18:17:21.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.844+0000 7f194bfff640 1 Processor -- start 2023-12-19T18:17:21.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.844+0000 7f194bfff640 1 -- start start 2023-12-19T18:17:21.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.845+0000 7f194bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f194c0c50a0 0x7f194c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.845+0000 7f194bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f194c1553a0 0x7f194c0aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.845+0000 7f194bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f194c15ee30 0x7f194c157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.845+0000 7f194bfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0b1b00 con 0x7f194c15ee30 2023-12-19T18:17:21.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.845+0000 7f194bfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0af090 con 0x7f194c0c50a0 2023-12-19T18:17:21.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.845+0000 7f194bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f194c0ade20 con 0x7f194c1553a0 2023-12-19T18:17:21.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.845+0000 7f194b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f194c15ee30 0x7f194c157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.845+0000 7f194b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f194c15ee30 0x7f194c157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34248/0 (socket says 172.21.15.27:34248) 2023-12-19T18:17:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.845+0000 7f194b7fe640 1 -- 172.21.15.27:0/4101714674 learned_addr learned my addr 172.21.15.27:0/4101714674 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.846+0000 7f194affd640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f194c0c50a0 0x7f194c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.846+0000 7f1943fff640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f194c1553a0 0x7f194c0aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.846+0000 7f194b7fe640 1 -- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f194c1553a0 msgr2=0x7f194c0aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.846+0000 7f194b7fe640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f194c1553a0 0x7f194c0aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.846+0000 7f194b7fe640 1 -- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f194c0c50a0 msgr2=0x7f194c0b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.846+0000 7f194b7fe640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f194c0c50a0 0x7f194c0b15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:21.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.846+0000 7f194b7fe640 1 -- 172.21.15.27:0/4101714674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1944009a70 con 0x7f194c15ee30 2023-12-19T18:17:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.846+0000 7f194b7fe640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f194c15ee30 0x7f194c157f00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1944009d90 tx=0x7f1944004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.846+0000 7f1948ff9640 1 -- 172.21.15.27:0/4101714674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194402cea0 con 0x7f194c15ee30 2023-12-19T18:17:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.846+0000 7f1948ff9640 1 -- 172.21.15.27:0/4101714674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1944040450 con 0x7f194c15ee30 2023-12-19T18:17:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.846+0000 7f1948ff9640 1 -- 172.21.15.27:0/4101714674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19440407e0 con 0x7f194c15ee30 2023-12-19T18:17:21.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.847+0000 7f194bfff640 1 -- 172.21.15.27:0/4101714674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f194c15c2d0 con 0x7f194c15ee30 2023-12-19T18:17:21.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.847+0000 7f194bfff640 1 -- 172.21.15.27:0/4101714674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f194c15c890 con 0x7f194c15ee30 2023-12-19T18:17:21.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.848+0000 7f194bfff640 1 -- 172.21.15.27:0/4101714674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f194c04e840 con 0x7f194c15ee30 2023-12-19T18:17:21.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.849+0000 7f1948ff9640 1 -- 172.21.15.27:0/4101714674 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1944040940 con 0x7f194c15ee30 2023-12-19T18:17:21.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.850+0000 7f1948ff9640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f192c06a430 0x7f192c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:21.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.850+0000 7f1948ff9640 1 -- 172.21.15.27:0/4101714674 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f19440ab3e0 con 0x7f194c15ee30 2023-12-19T18:17:21.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.851+0000 7f194affd640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f192c06a430 0x7f192c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:21.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.852+0000 7f194affd640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f192c06a430 0x7f192c06c8f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f193c005880 tx=0x7f193c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:21.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:21.855+0000 7f1948ff9640 1 -- 172.21.15.27:0/4101714674 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1944076a90 con 0x7f194c15ee30 2023-12-19T18:17:22.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.087+0000 7f194bfff640 1 -- 172.21.15.27:0/4101714674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f194c0af6a0 con 0x7f194c15ee30 2023-12-19T18:17:22.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.087+0000 7f1948ff9640 1 -- 172.21.15.27:0/4101714674 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f194407a0b0 con 0x7f194c15ee30 2023-12-19T18:17:22.088 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:17:22.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.090+0000 7f194bfff640 1 -- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f192c06a430 msgr2=0x7f192c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:22.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.090+0000 7f194bfff640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f192c06a430 0x7f192c06c8f0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f193c005880 tx=0x7f193c009040 comp rx=0 tx=0).stop 2023-12-19T18:17:22.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.090+0000 7f194bfff640 1 -- 172.21.15.27:0/4101714674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f194c15ee30 msgr2=0x7f194c157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:22.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.090+0000 7f194bfff640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f194c15ee30 0x7f194c157f00 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1944009d90 tx=0x7f1944004a40 comp rx=0 tx=0).stop 2023-12-19T18:17:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.090+0000 7f194bfff640 1 -- 172.21.15.27:0/4101714674 shutdown_connections 2023-12-19T18:17:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.090+0000 7f194bfff640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f192c06a430 0x7f192c06c8f0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.090+0000 7f194bfff640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f194c15ee30 0x7f194c157f00 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.090+0000 7f194bfff640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f194c1553a0 0x7f194c0aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.090+0000 7f194bfff640 1 --2- 172.21.15.27:0/4101714674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f194c0c50a0 0x7f194c0b15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.090+0000 7f194bfff640 1 -- 172.21.15.27:0/4101714674 >> 172.21.15.27:0/4101714674 conn(0x7f194c069c70 msgr2=0x7f194c159f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:22.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.091+0000 7f194bfff640 1 -- 172.21.15.27:0/4101714674 shutdown_connections 2023-12-19T18:17:22.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:22.091+0000 7f194bfff640 1 -- 172.21.15.27:0/4101714674 wait complete. 2023-12-19T18:17:22.111 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:17:22.111 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:17:22.111 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-12-19T18:17:22.111 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:17:22.111 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:17:22.112 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:17:22.112 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:17:22.112 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:17:22.112 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:17:22.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:22.113 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-12-19T18:17:22.170 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:17:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:17:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:17:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:22.170 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-19T18:17:22.171 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42198sec preferred_lft 42198sec 2023-12-19T18:17:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2023-12-19T18:17:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:22.172 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:17:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-12-19T18:17:22.172 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:17:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:17:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::8053:76ff:fe69:c66c/64 scope link 2023-12-19T18:17:22.172 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:22.173 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:17:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:17:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::781d:d6ff:feb4:7a2d/64 scope link 2023-12-19T18:17:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:22.173 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:17:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether f2:29:63:cc:14:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:17:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::f029:63ff:fecc:140e/64 scope link 2023-12-19T18:17:22.173 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:22.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:22.174 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:17:22.282 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-12-19T18:17:22.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:22.287 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-12-19T18:17:22.345 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-12-19T18:17:22.345 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:17:22.346 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-19T18:17:22.346 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-19T18:17:22.346 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:17:22.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:22.346 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:17:22.346 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:17:22.346 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:17:22.346 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:17:22.346 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:17:22.346 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:17:22.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:22.551 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:17:22.606 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:17:22.606 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:17:22.607 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:17:22.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:22.607 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-12-19T18:17:22.720 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-19T18:17:22.720 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:17:22.720 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:17:22.779 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:17:22.780 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:17:22.780 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:17:22.840 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:17:22.945 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:17:22.946 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:17:23.055 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:17:23.060 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:17:23.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:23.061 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:17:23.124 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:17:23.124 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-19T18:17:23.130 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:17:23.216 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:17:23.216 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:17:23.293 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:17:23.296 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:17:23.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:23.297 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:17:23.330 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-12-19T18:17:23.330+0000 7f86128f4480 -1 init, newargv = 0x56306d7ef470 newargc=15 2023-12-19T18:17:23.331 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[96772]: starting ceph client 2023-12-19T18:17:23.338 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[96772]: starting fuse 2023-12-19T18:17:23.343 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:17:23.343 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-19T18:17:23.343 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:17:23.343 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:17:23.343 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:17:23.343 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:17:23.343 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:17:23.343 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:17:23.343 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:17:23.343 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:17:23.343 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:17:23.344 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:17:23.345 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:17:23.550 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.96772.asok 2023-12-19T18:17:23.729 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.96772.asok 2023-12-19T18:17:23.729 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:23.729 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.96772.asok status 2023-12-19T18:17:23.987 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:17:23.987 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "96772", 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 5832, 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:17:23.988 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 5832 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3699225783 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3699225783 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.5832 192.168.0.1:0/3699225783", 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3699225783", 2023-12-19T18:17:23.989 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-12-19T18:17:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 23, 2023-12-19T18:17:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 50, 2023-12-19T18:17:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:17:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:17:23.990 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:17:23.990 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:17:24.004 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:24.005 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-12-19T18:17:24.005 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-19T18:17:24.064 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-12-19T18:17:24.064 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-19T18:17:24.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:24.064 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:17:24.175 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:17:24.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.399+0000 7fade9efd640 1 Processor -- start 2023-12-19T18:17:24.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.399+0000 7fade9efd640 1 -- start start 2023-12-19T18:17:24.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.400+0000 7fade9efd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fade40b4ef0 0x7fade40b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.401+0000 7fade9efd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fade40d2350 0x7fade40b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.401+0000 7fade9efd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade40b5e90 con 0x7fade41604d0 2023-12-19T18:17:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.401+0000 7fade9efd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade40d2b30 con 0x7fade40b4ef0 2023-12-19T18:17:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.401+0000 7fade9efd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade40d2850 con 0x7fade40d2350 2023-12-19T18:17:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.401+0000 7fade8efb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fade41604d0 0x7fade40b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40038/0 (socket says 172.21.15.27:40038) 2023-12-19T18:17:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.401+0000 7fade8efb640 1 -- 172.21.15.27:0/592049655 learned_addr learned my addr 172.21.15.27:0/592049655 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.401+0000 7fade3fff640 1 --2- 172.21.15.27:0/592049655 >> v2:172.21.15.27:3300/0 conn(0x7fade40b4ef0 0x7fade40b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.401+0000 7fade96fc640 1 --2- 172.21.15.27:0/592049655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fade40d2350 0x7fade40b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.401+0000 7fade37fe640 1 -- 172.21.15.27:0/592049655 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511762621 0 0) 0x7fade40b5e90 con 0x7fade41604d0 2023-12-19T18:17:24.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.401+0000 7fade37fe640 1 -- 172.21.15.27:0/592049655 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadcc003520 con 0x7fade41604d0 2023-12-19T18:17:24.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.401+0000 7fade3fff640 1 -- 172.21.15.27:0/592049655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fade40d2350 msgr2=0x7fade40b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:24.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.401+0000 7fade3fff640 1 --2- 172.21.15.27:0/592049655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fade40d2350 0x7fade40b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.401+0000 7fade3fff640 1 -- 172.21.15.27:0/592049655 >> v1:172.21.15.27:6789/0 conn(0x7fade41604d0 legacy=0x7fade40b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.402+0000 7fade3fff640 1 -- 172.21.15.27:0/592049655 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade40cded0 con 0x7fade40b4ef0 2023-12-19T18:17:24.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.403+0000 7fade3fff640 1 --2- 172.21.15.27:0/592049655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fade40b4ef0 0x7fade40b6300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fadd80097a0 tx=0x7fadd802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9042e093ec43480 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:24.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.403+0000 7fade37fe640 1 -- 172.21.15.27:0/592049655 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8031830 con 0x7fade40b4ef0 2023-12-19T18:17:24.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.403+0000 7fade37fe640 1 -- 172.21.15.27:0/592049655 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd8031990 con 0x7fade40b4ef0 2023-12-19T18:17:24.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.403+0000 7fade37fe640 1 -- 172.21.15.27:0/592049655 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8031d20 con 0x7fade40b4ef0 2023-12-19T18:17:24.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.404+0000 7fade9efd640 1 -- 172.21.15.27:0/592049655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fade40b4ef0 msgr2=0x7fade40b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.404+0000 7fade9efd640 1 --2- 172.21.15.27:0/592049655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fade40b4ef0 0x7fade40b6300 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fadd80097a0 tx=0x7fadd802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.404+0000 7fade9efd640 1 -- 172.21.15.27:0/592049655 shutdown_connections 2023-12-19T18:17:24.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.404+0000 7fade9efd640 1 --2- 172.21.15.27:0/592049655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fade40d2350 0x7fade40b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.404+0000 7fade9efd640 1 --2- 172.21.15.27:0/592049655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fade40b4ef0 0x7fade40b6300 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.404+0000 7fade9efd640 1 -- 172.21.15.27:0/592049655 >> 172.21.15.27:0/592049655 conn(0x7fade40cf7e0 msgr2=0x7fade40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:24.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.404+0000 7fade9efd640 1 -- 172.21.15.27:0/592049655 shutdown_connections 2023-12-19T18:17:24.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.404+0000 7fade9efd640 1 -- 172.21.15.27:0/592049655 wait complete. 2023-12-19T18:17:24.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.406+0000 7fade9efd640 1 Processor -- start 2023-12-19T18:17:24.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.406+0000 7fade9efd640 1 -- start start 2023-12-19T18:17:24.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade9efd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fade40b4ef0 0x7fade40ca840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade9efd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fade40d2350 0x7fade415b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade9efd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade41604d0 0x7fade40cc770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade9efd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade415bc40 con 0x7fade40d2350 2023-12-19T18:17:24.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade9efd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40cccb0 con 0x7fade41604d0 2023-12-19T18:17:24.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade9efd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade4159480 con 0x7fade40b4ef0 2023-12-19T18:17:24.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fade40d2350 0x7fade415b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fade40d2350 0x7fade415b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46328/0 (socket says 172.21.15.27:46328) 2023-12-19T18:17:24.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade3fff640 1 -- 172.21.15.27:0/3839918380 learned_addr learned my addr 172.21.15.27:0/3839918380 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:24.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade8efb640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fade40b4ef0 0x7fade40ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade3fff640 1 -- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fade40b4ef0 msgr2=0x7fade40ca840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade3fff640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fade40b4ef0 0x7fade40ca840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade3fff640 1 -- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade41604d0 msgr2=0x7fade40cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade3fff640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade41604d0 0x7fade40cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade3fff640 1 -- 172.21.15.27:0/3839918380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd8009400 con 0x7fade40d2350 2023-12-19T18:17:24.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.407+0000 7fade8efb640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fade40b4ef0 0x7fade40ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:24.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.408+0000 7fade3fff640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fade40d2350 0x7fade415b700 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fadd8034040 tx=0x7fadd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:24.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.408+0000 7fade17fa640 1 -- 172.21.15.27:0/3839918380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd803a8f0 con 0x7fade40d2350 2023-12-19T18:17:24.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.408+0000 7fade17fa640 1 -- 172.21.15.27:0/3839918380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd803aa70 con 0x7fade40d2350 2023-12-19T18:17:24.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.408+0000 7fade17fa640 1 -- 172.21.15.27:0/3839918380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd803ae20 con 0x7fade40d2350 2023-12-19T18:17:24.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.408+0000 7fade9efd640 1 -- 172.21.15.27:0/3839918380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade40caed0 con 0x7fade40d2350 2023-12-19T18:17:24.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.408+0000 7fade9efd640 1 -- 172.21.15.27:0/3839918380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade40cb490 con 0x7fade40d2350 2023-12-19T18:17:24.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.409+0000 7fade9efd640 1 -- 172.21.15.27:0/3839918380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade4066f80 con 0x7fade40d2350 2023-12-19T18:17:24.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.410+0000 7fade17fa640 1 -- 172.21.15.27:0/3839918380 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fadd804d440 con 0x7fade40d2350 2023-12-19T18:17:24.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.411+0000 7fade17fa640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fadd006a430 0x7fadd006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:24.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.411+0000 7fade17fa640 1 -- 172.21.15.27:0/3839918380 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fadd80aaa90 con 0x7fade40d2350 2023-12-19T18:17:24.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.412+0000 7fade8efb640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fadd006a430 0x7fadd006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:24.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.413+0000 7fade8efb640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fadd006a430 0x7fadd006c8f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fadd4001560 tx=0x7fadd4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:24.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.416+0000 7fade17fa640 1 -- 172.21.15.27:0/3839918380 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fadd8075fe0 con 0x7fade40d2350 2023-12-19T18:17:24.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.651+0000 7fade9efd640 1 -- 172.21.15.27:0/3839918380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fade40ce460 con 0x7fade40d2350 2023-12-19T18:17:24.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.652+0000 7fade17fa640 1 -- 172.21.15.27:0/3839918380 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7fadd8035020 con 0x7fade40d2350 2023-12-19T18:17:24.652 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:17:24.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.655+0000 7fade9efd640 1 -- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fadd006a430 msgr2=0x7fadd006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.655+0000 7fade9efd640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fadd006a430 0x7fadd006c8f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fadd4001560 tx=0x7fadd4009040 comp rx=0 tx=0).stop 2023-12-19T18:17:24.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.655+0000 7fade9efd640 1 -- 172.21.15.27:0/3839918380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fade40d2350 msgr2=0x7fade415b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:24.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.655+0000 7fade9efd640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fade40d2350 0x7fade415b700 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fadd8034040 tx=0x7fadd80046c0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.655+0000 7fade9efd640 1 -- 172.21.15.27:0/3839918380 shutdown_connections 2023-12-19T18:17:24.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.655+0000 7fade9efd640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fadd006a430 0x7fadd006c8f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.655+0000 7fade9efd640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fade41604d0 0x7fade40cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.655+0000 7fade9efd640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fade40d2350 0x7fade415b700 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.655+0000 7fade9efd640 1 --2- 172.21.15.27:0/3839918380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fade40b4ef0 0x7fade40ca840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:24.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.655+0000 7fade9efd640 1 -- 172.21.15.27:0/3839918380 >> 172.21.15.27:0/3839918380 conn(0x7fade40cf7e0 msgr2=0x7fade40b8210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:24.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.655+0000 7fade9efd640 1 -- 172.21.15.27:0/3839918380 shutdown_connections 2023-12-19T18:17:24.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:24.655+0000 7fade9efd640 1 -- 172.21.15.27:0/3839918380 wait complete. 2023-12-19T18:17:24.675 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:17:24.675 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:17:24.675 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-19T18:17:24.675 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-19T18:17:24.675 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T18:17:24.675 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:17:24.676 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:17:24.676 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:17:24.676 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:17:24.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:24.676 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-19T18:17:24.737 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:17:24.737 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:17:24.737 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:17:24.737 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:24.737 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:17:24.737 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:24.737 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:17:24.737 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:24.737 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-19T18:17:24.737 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:17:24.737 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:24.737 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42158sec preferred_lft 42158sec 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::b892:1cff:fedc:4bed/64 scope link 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:17:24.738 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether a6:e1:fc:e6:56:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:17:24.739 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::a4e1:fcff:fee6:56fa/64 scope link 2023-12-19T18:17:24.739 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:24.739 INFO:teuthology.orchestra.run.smithi192.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:17:24.739 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T18:17:24.739 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::404f:6cff:fe73:9aa2/64 scope link 2023-12-19T18:17:24.739 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:17:24.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:24.740 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:17:24.835 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-19T18:17:24.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:24.839 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-19T18:17:24.894 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-19T18:17:24.894 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:17:24.894 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T18:17:24.894 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T18:17:24.894 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:17:24.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:24.895 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:17:24.895 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:17:24.895 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T18:17:24.895 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T18:17:24.896 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T18:17:24.896 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:17:25.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:25.086 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:17:25.138 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T18:17:25.138 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-19T18:17:25.139 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:17:25.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:25.140 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-19T18:17:25.234 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-19T18:17:25.234 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:17:25.234 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-19T18:17:25.290 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:17:25.291 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:17:25.291 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:17:25.343 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:17:25.431 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:17:25.431 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:17:25.523 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:17:25.528 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:17:25.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:25.528 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:17:25.580 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:17:25.580 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-19T18:17:25.585 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:17:25.659 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:17:25.660 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:17:25.728 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:17:25.731 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:17:25.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:25.732 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:17:25.767 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-19T18:17:25.767+0000 7f18a503a480 -1 init, newargv = 0x55d23241b380 newargc=15 2023-12-19T18:17:25.767 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[87132]: starting ceph client 2023-12-19T18:17:25.776 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[87132]: starting fuse 2023-12-19T18:17:26.875 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:17:26.980 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:17:26.980 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:17:27.070 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:17:27.074 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:17:27.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:27.075 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:17:27.130 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:17:27.131 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-12-19T18:17:27.131 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:17:27.131 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-19T18:17:27.131 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-19T18:17:27.132 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-19T18:17:27.132 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-19T18:17:27.132 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:17:27.132 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-19T18:17:27.132 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-19T18:17:27.132 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-19T18:17:27.132 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-19T18:17:27.132 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:17:27.133 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-19T18:17:27.133 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-19T18:17:27.133 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-19T18:17:27.133 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-19T18:17:27.133 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:17:27.133 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-19T18:17:27.133 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:17:27.134 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:17:27.134 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:17:27.135 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-19T18:17:27.135 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-19T18:17:27.135 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-19T18:17:27.136 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:17:27.136 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:17:27.136 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-19T18:17:27.136 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-19T18:17:27.308 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.87132.asok 2023-12-19T18:17:27.488 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.87132.asok 2023-12-19T18:17:27.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:27.488 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.87132.asok status 2023-12-19T18:17:27.728 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-19T18:17:27.729 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-19T18:17:27.729 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:17:27.729 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:17:27.729 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-19T18:17:27.729 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-19T18:17:27.729 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-19T18:17:27.729 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "87132", 2023-12-19T18:17:27.729 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-19T18:17:27.729 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:17:27.730 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-19T18:17:27.730 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-19T18:17:27.730 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 4940, 2023-12-19T18:17:27.730 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-19T18:17:27.730 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-19T18:17:27.730 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-19T18:17:27.730 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 4940 2023-12-19T18:17:27.730 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:17:27.730 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:17:27.730 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:17:27.730 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:17:27.730 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1259068788 2023-12-19T18:17:27.730 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-19T18:17:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:17:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:17:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:17:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:17:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 1259068788 2023-12-19T18:17:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:17:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.4940 192.168.0.1:0/1259068788", 2023-12-19T18:17:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/1259068788", 2023-12-19T18:17:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-19T18:17:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 23, 2023-12-19T18:17:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 50, 2023-12-19T18:17:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:17:27.731 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-19T18:17:27.732 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-19T18:17:27.732 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-19T18:17:27.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:27.744 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-19T18:17:27.744 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-19T18:17:27.797 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-19T18:17:27.797 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-19T18:17:27.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:27.798 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:17:27.892 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:28.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.147+0000 7f9e97c28640 1 Processor -- start 2023-12-19T18:17:28.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.147+0000 7f9e97c28640 1 -- start start 2023-12-19T18:17:28.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.148+0000 7f9e97c28640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9e9015d240 0x7f9e901618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:28.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.149+0000 7f9e97c28640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e900d1970 con 0x7f9e900cdf90 2023-12-19T18:17:28.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.149+0000 7f9e97c28640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e90161e10 con 0x7f9e900ce420 2023-12-19T18:17:28.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.149+0000 7f9e97c28640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e900b5a80 con 0x7f9e9015d240 2023-12-19T18:17:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.149+0000 7f9e951c3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9e900cdf90 0x7f9e900b4c60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40040/0 (socket says 172.21.15.27:40040) 2023-12-19T18:17:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.149+0000 7f9e951c3640 1 -- 172.21.15.27:0/231996075 learned_addr learned my addr 172.21.15.27:0/231996075 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.149+0000 7f9e959c4640 1 --2- 172.21.15.27:0/231996075 >> v2:172.21.15.192:3300/0 conn(0x7f9e9015d240 0x7f9e901618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.149+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3755174302 0 0) 0x7f9e900d1970 con 0x7f9e900cdf90 2023-12-19T18:17:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e7c003520 con 0x7f9e900cdf90 2023-12-19T18:17:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1055352310 0 0) 0x7f9e7c003520 con 0x7f9e900cdf90 2023-12-19T18:17:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 >> v2:172.21.15.192:3300/0 conn(0x7f9e9015d240 msgr2=0x7f9e901618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 --2- 172.21.15.27:0/231996075 >> v2:172.21.15.192:3300/0 conn(0x7f9e9015d240 0x7f9e901618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 >> v1:172.21.15.192:6789/0 conn(0x7f9e900ce420 legacy=0x7f9e900b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e9006a5e0 con 0x7f9e900cdf90 2023-12-19T18:17:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f9e80001670 con 0x7f9e900cdf90 2023-12-19T18:17:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 >> v1:172.21.15.27:6789/0 conn(0x7f9e900cdf90 legacy=0x7f9e900b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 --2- 172.21.15.27:0/231996075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7c0039c0 0x7f9e7c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 --2- 172.21.15.27:0/231996075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c0047c0 0x7f9e7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 --2- 172.21.15.27:0/231996075 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e7c005180 0x7f9e7c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e900b5a80 con 0x7f9e7c0039c0 2023-12-19T18:17:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c00c060 con 0x7f9e7c0047c0 2023-12-19T18:17:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7c00c220 con 0x7f9e7c005180 2023-12-19T18:17:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.150+0000 7f9e959c4640 1 --2- 172.21.15.27:0/231996075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7c0039c0 0x7f9e7c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:28.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.151+0000 7f9e951c3640 1 --2- 172.21.15.27:0/231996075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c0047c0 0x7f9e7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:28.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.151+0000 7f9e959c4640 1 -- 172.21.15.27:0/231996075 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e7c005180 msgr2=0x7f9e7c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:28.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.151+0000 7f9e959c4640 1 --2- 172.21.15.27:0/231996075 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e7c005180 0x7f9e7c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.151+0000 7f9e959c4640 1 -- 172.21.15.27:0/231996075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c0047c0 msgr2=0x7f9e7c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:28.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.151+0000 7f9e959c4640 1 --2- 172.21.15.27:0/231996075 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c0047c0 0x7f9e7c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.151+0000 7f9e959c4640 1 -- 172.21.15.27:0/231996075 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e7c003520 con 0x7f9e7c0039c0 2023-12-19T18:17:28.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.151+0000 7f9e961c5640 1 -- 172.21.15.27:0/231996075 reap_dead start 2023-12-19T18:17:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.152+0000 7f9e959c4640 1 --2- 172.21.15.27:0/231996075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7c0039c0 0x7f9e7c003de0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009ac0 tx=0x7f9e7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11604bbd4c9830c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.152+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c02c6d0 con 0x7f9e7c0039c0 2023-12-19T18:17:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.152+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8c02c830 con 0x7f9e7c0039c0 2023-12-19T18:17:28.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.152+0000 7f9e949c2640 1 -- 172.21.15.27:0/231996075 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8c02cbc0 con 0x7f9e7c0039c0 2023-12-19T18:17:28.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.153+0000 7f9e97c28640 1 -- 172.21.15.27:0/231996075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7c0039c0 msgr2=0x7f9e7c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.153+0000 7f9e97c28640 1 --2- 172.21.15.27:0/231996075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7c0039c0 0x7f9e7c003de0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c009ac0 tx=0x7f9e7c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.153+0000 7f9e97c28640 1 -- 172.21.15.27:0/231996075 shutdown_connections 2023-12-19T18:17:28.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.153+0000 7f9e97c28640 1 --2- 172.21.15.27:0/231996075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e7c0039c0 0x7f9e7c003de0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.153+0000 7f9e97c28640 1 -- 172.21.15.27:0/231996075 >> 172.21.15.27:0/231996075 conn(0x7f9e900d31e0 msgr2=0x7f9e900d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:28.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.153+0000 7f9e97c28640 1 -- 172.21.15.27:0/231996075 shutdown_connections 2023-12-19T18:17:28.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.153+0000 7f9e97c28640 1 -- 172.21.15.27:0/231996075 wait complete. 2023-12-19T18:17:28.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.155+0000 7f9e97c28640 1 Processor -- start 2023-12-19T18:17:28.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.155+0000 7f9e97c28640 1 -- start start 2023-12-19T18:17:28.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.156+0000 7f9e97c28640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c0039c0 0x7f9e9015ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:28.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.156+0000 7f9e97c28640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e90151560 0x7f9e901f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:28.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.156+0000 7f9e97c28640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901519e0 0x7f9e901f6340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:28.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.156+0000 7f9e97c28640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e90160460 con 0x7f9e901519e0 2023-12-19T18:17:28.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.156+0000 7f9e97c28640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e9015d6c0 con 0x7f9e7c0039c0 2023-12-19T18:17:28.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.156+0000 7f9e97c28640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e900cc7b0 con 0x7f9e90151560 2023-12-19T18:17:28.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.156+0000 7f9e961c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901519e0 0x7f9e901f6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:28.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.157+0000 7f9e961c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901519e0 0x7f9e901f6340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:46354/0 (socket says 172.21.15.27:46354) 2023-12-19T18:17:28.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.157+0000 7f9e961c5640 1 -- 172.21.15.27:0/997991834 learned_addr learned my addr 172.21.15.27:0/997991834 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:28.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.157+0000 7f9e959c4640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c0039c0 0x7f9e9015ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.157+0000 7f9e951c3640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e90151560 0x7f9e901f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.157+0000 7f9e961c5640 1 -- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e90151560 msgr2=0x7f9e901f5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.157+0000 7f9e961c5640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e90151560 0x7f9e901f5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.157+0000 7f9e961c5640 1 -- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c0039c0 msgr2=0x7f9e9015ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.157+0000 7f9e961c5640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c0039c0 0x7f9e9015ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.157+0000 7f9e961c5640 1 -- 172.21.15.27:0/997991834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e900cca90 con 0x7f9e901519e0 2023-12-19T18:17:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.157+0000 7f9e951c3640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e90151560 0x7f9e901f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.157+0000 7f9e961c5640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901519e0 0x7f9e901f6340 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9e900b5d00 tx=0x7f9e8800cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.158+0000 7f9e867fc640 1 -- 172.21.15.27:0/997991834 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e88005d40 con 0x7f9e901519e0 2023-12-19T18:17:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.158+0000 7f9e867fc640 1 -- 172.21.15.27:0/997991834 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e8800ced0 con 0x7f9e901519e0 2023-12-19T18:17:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.158+0000 7f9e867fc640 1 -- 172.21.15.27:0/997991834 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e8800a850 con 0x7f9e901519e0 2023-12-19T18:17:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.158+0000 7f9e97c28640 1 -- 172.21.15.27:0/997991834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e9015d960 con 0x7f9e901519e0 2023-12-19T18:17:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.158+0000 7f9e97c28640 1 -- 172.21.15.27:0/997991834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e900ccd70 con 0x7f9e901519e0 2023-12-19T18:17:28.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.159+0000 7f9e97c28640 1 -- 172.21.15.27:0/997991834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e900665d0 con 0x7f9e901519e0 2023-12-19T18:17:28.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.160+0000 7f9e867fc640 1 -- 172.21.15.27:0/997991834 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9e8800a9b0 con 0x7f9e901519e0 2023-12-19T18:17:28.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.161+0000 7f9e867fc640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e7406a430 0x7f9e7406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:28.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.162+0000 7f9e867fc640 1 -- 172.21.15.27:0/997991834 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9e880878f0 con 0x7f9e901519e0 2023-12-19T18:17:28.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.162+0000 7f9e959c4640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e7406a430 0x7f9e7406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:28.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.163+0000 7f9e959c4640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e7406a430 0x7f9e7406c8f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c036000 tx=0x7f9e8c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:28.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.167+0000 7f9e867fc640 1 -- 172.21.15.27:0/997991834 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9e88052f70 con 0x7f9e901519e0 2023-12-19T18:17:28.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.402+0000 7f9e97c28640 1 -- 172.21.15.27:0/997991834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9e90068e90 con 0x7f9e901519e0 2023-12-19T18:17:28.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.403+0000 7f9e867fc640 1 -- 172.21.15.27:0/997991834 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f9e88017090 con 0x7f9e901519e0 2023-12-19T18:17:28.403 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:28.403 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/930391176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":930391176},{"type":"v1","addr":"172.21.15.192:6837","nonce":930391176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5179,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3606564673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3606564673},{"type":"v1","addr":"172.21.15.192:6836","nonce":3606564673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3604117097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3604117097},{"type":"v1","addr":"172.21.15.27:6834","nonce":3604117097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:14.250689+0000","modified":"2023-12-19T18:17:17.758670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3814458529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3814458529},{"type":"v1","addr":"172.21.15.27:6835","nonce":3814458529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T18:17:28.403 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-19T18:17:28.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.405+0000 7f9e97c28640 1 -- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e7406a430 msgr2=0x7f9e7406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.405+0000 7f9e97c28640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e7406a430 0x7f9e7406c8f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9e8c036000 tx=0x7f9e8c030000 comp rx=0 tx=0).stop 2023-12-19T18:17:28.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.405+0000 7f9e97c28640 1 -- 172.21.15.27:0/997991834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901519e0 msgr2=0x7f9e901f6340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:28.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.405+0000 7f9e97c28640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901519e0 0x7f9e901f6340 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f9e900b5d00 tx=0x7f9e8800cb90 comp rx=0 tx=0).stop 2023-12-19T18:17:28.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.406+0000 7f9e97c28640 1 -- 172.21.15.27:0/997991834 shutdown_connections 2023-12-19T18:17:28.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.406+0000 7f9e97c28640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e7406a430 0x7f9e7406c8f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.406+0000 7f9e97c28640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e901519e0 0x7f9e901f6340 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.406+0000 7f9e97c28640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9e90151560 0x7f9e901f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.406+0000 7f9e97c28640 1 --2- 172.21.15.27:0/997991834 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9e7c0039c0 0x7f9e9015ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:28.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.406+0000 7f9e97c28640 1 -- 172.21.15.27:0/997991834 >> 172.21.15.27:0/997991834 conn(0x7f9e900d31e0 msgr2=0x7f9e900c7530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:28.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.406+0000 7f9e97c28640 1 -- 172.21.15.27:0/997991834 shutdown_connections 2023-12-19T18:17:28.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:28.406+0000 7f9e97c28640 1 -- 172.21.15.27:0/997991834 wait complete. 2023-12-19T18:17:28.427 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-12-19T18:17:28.672 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":5679,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.399785581,"mdsmap_epoch":23,"osdmap_epoch":50,"osdmap_epoch_barrier":50,"uptime":19.671854753000002} 2023-12-19T18:17:28.672 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 5679, "mdsmap_epoch": 23, "osdmap_epoch": 50, "osdmap_epoch_barrier": 50, "rank_uptime": 14.399785581, "state": "up:active", "uptime": 19.671854753, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:17:28.672 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-12-19T18:17:28.925 INFO:teuthology.orchestra.run.smithi027.stdout:{"osd_mon_report_interval":"5"} 2023-12-19T18:17:28.926 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-12-19T18:17:28.926 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-12-19T18:17:28.926 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:17:28.926 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:28.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:28.981 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-12-19T18:17:35.117 INFO:teuthology.orchestra.run.smithi027.stderr:265+0 records in 2023-12-19T18:17:35.117 INFO:teuthology.orchestra.run.smithi027.stderr:265+0 records out 2023-12-19T18:17:35.117 INFO:teuthology.orchestra.run.smithi027.stderr:277872640 bytes (278 MB, 265 MiB) copied, 6.069 s, 45.8 MB/s 2023-12-19T18:17:35.119 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:17:35.119 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:35.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:35.182 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-12-19T18:17:39.576 INFO:teuthology.orchestra.run.smithi027.stderr:292+0 records in 2023-12-19T18:17:39.576 INFO:teuthology.orchestra.run.smithi027.stderr:292+0 records out 2023-12-19T18:17:39.577 INFO:teuthology.orchestra.run.smithi027.stderr:306184192 bytes (306 MB, 292 MiB) copied, 4.33361 s, 70.7 MB/s 2023-12-19T18:17:39.578 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-12-19T18:17:39.578 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:17:39.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.817+0000 7fb147882640 1 Processor -- start 2023-12-19T18:17:39.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.818+0000 7fb147882640 1 -- start start 2023-12-19T18:17:39.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.819+0000 7fb147882640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb14006ab00 0x7fb1400d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:39.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.819+0000 7fb147882640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1400d2af0 0x7fb1400d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:39.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.819+0000 7fb147882640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1400d1a50 con 0x7fb14015d260 2023-12-19T18:17:39.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.819+0000 7fb147882640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1400cdaf0 con 0x7fb1400d2af0 2023-12-19T18:17:39.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.819+0000 7fb147882640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1400b71b0 con 0x7fb14006ab00 2023-12-19T18:17:39.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.820+0000 7fb145e1f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1400d2af0 0x7fb1400d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.820+0000 7fb145e1f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb1400d2af0 0x7fb1400d13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48926/0 (socket says 172.21.15.27:48926) 2023-12-19T18:17:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.820+0000 7fb145e1f640 1 -- 172.21.15.27:0/1971779009 learned_addr learned my addr 172.21.15.27:0/1971779009 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.820+0000 7fb144e1d640 1 --2- 172.21.15.27:0/1971779009 >> v2:172.21.15.192:3300/0 conn(0x7fb14006ab00 0x7fb1400d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.820+0000 7fb145e1f640 1 -- 172.21.15.27:0/1971779009 >> v2:172.21.15.192:3300/0 conn(0x7fb14006ab00 msgr2=0x7fb1400d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.820+0000 7fb145e1f640 1 --2- 172.21.15.27:0/1971779009 >> v2:172.21.15.192:3300/0 conn(0x7fb14006ab00 0x7fb1400d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.820+0000 7fb145e1f640 1 -- 172.21.15.27:0/1971779009 >> v1:172.21.15.192:6789/0 conn(0x7fb14015d260 legacy=0x7fb14006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:39.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.820+0000 7fb145e1f640 1 -- 172.21.15.27:0/1971779009 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1400d1d30 con 0x7fb1400d2af0 2023-12-19T18:17:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.821+0000 7fb145e1f640 1 --2- 172.21.15.27:0/1971779009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1400d2af0 0x7fb1400d13b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb1400b29a0 tx=0x7fb13802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7843370528977bb2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.822+0000 7fb137fff640 1 -- 172.21.15.27:0/1971779009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb138030860 con 0x7fb1400d2af0 2023-12-19T18:17:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.822+0000 7fb137fff640 1 -- 172.21.15.27:0/1971779009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1380309c0 con 0x7fb1400d2af0 2023-12-19T18:17:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.822+0000 7fb137fff640 1 -- 172.21.15.27:0/1971779009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb138030d50 con 0x7fb1400d2af0 2023-12-19T18:17:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.822+0000 7fb147882640 1 -- 172.21.15.27:0/1971779009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1400d2af0 msgr2=0x7fb1400d13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:39.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.822+0000 7fb147882640 1 --2- 172.21.15.27:0/1971779009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1400d2af0 0x7fb1400d13b0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fb1400b29a0 tx=0x7fb13802b9a0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.822+0000 7fb147882640 1 -- 172.21.15.27:0/1971779009 shutdown_connections 2023-12-19T18:17:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.822+0000 7fb147882640 1 --2- 172.21.15.27:0/1971779009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1400d2af0 0x7fb1400d13b0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.822+0000 7fb147882640 1 --2- 172.21.15.27:0/1971779009 >> v2:172.21.15.192:3300/0 conn(0x7fb14006ab00 0x7fb1400d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.822+0000 7fb147882640 1 -- 172.21.15.27:0/1971779009 >> 172.21.15.27:0/1971779009 conn(0x7fb1400d2f70 msgr2=0x7fb1400d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.823+0000 7fb147882640 1 -- 172.21.15.27:0/1971779009 shutdown_connections 2023-12-19T18:17:39.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.823+0000 7fb147882640 1 -- 172.21.15.27:0/1971779009 wait complete. 2023-12-19T18:17:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.824+0000 7fb147882640 1 Processor -- start 2023-12-19T18:17:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.825+0000 7fb147882640 1 -- start start 2023-12-19T18:17:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb147882640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14006ab00 0x7fb1400c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb147882640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1400d2af0 0x7fb1401e6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb147882640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb14015d260 0x7fb1400bc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb147882640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1400bce10 con 0x7fb14006ab00 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb147882640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1401e6240 con 0x7fb14015d260 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb147882640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb140157810 con 0x7fb1400d2af0 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb14561e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14006ab00 0x7fb1400c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb14561e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14006ab00 0x7fb1400c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:48934/0 (socket says 172.21.15.27:48934) 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb14561e640 1 -- 172.21.15.27:0/574964328 learned_addr learned my addr 172.21.15.27:0/574964328 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb145e1f640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb14015d260 0x7fb1400bc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb144e1d640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1400d2af0 0x7fb1401e6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb14561e640 1 -- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1400d2af0 msgr2=0x7fb1401e6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb14561e640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1400d2af0 0x7fb1401e6d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb14561e640 1 -- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb14015d260 msgr2=0x7fb1400bc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb14561e640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb14015d260 0x7fb1400bc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.826+0000 7fb14561e640 1 -- 172.21.15.27:0/574964328 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1380094d0 con 0x7fb14006ab00 2023-12-19T18:17:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.827+0000 7fb14561e640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14006ab00 0x7fb1400c9070 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00c710 tx=0x7fb13c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.827+0000 7fb135ffb640 1 -- 172.21.15.27:0/574964328 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c003b90 con 0x7fb14006ab00 2023-12-19T18:17:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.827+0000 7fb135ffb640 1 -- 172.21.15.27:0/574964328 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb13c003cf0 con 0x7fb14006ab00 2023-12-19T18:17:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.827+0000 7fb147882640 1 -- 172.21.15.27:0/574964328 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb140157af0 con 0x7fb14006ab00 2023-12-19T18:17:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.827+0000 7fb135ffb640 1 -- 172.21.15.27:0/574964328 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb13c00aa90 con 0x7fb14006ab00 2023-12-19T18:17:39.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.827+0000 7fb147882640 1 -- 172.21.15.27:0/574964328 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1401e67e0 con 0x7fb14006ab00 2023-12-19T18:17:39.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.829+0000 7fb147882640 1 -- 172.21.15.27:0/574964328 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb140066810 con 0x7fb14006ab00 2023-12-19T18:17:39.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.829+0000 7fb135ffb640 1 -- 172.21.15.27:0/574964328 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb13c00abf0 con 0x7fb14006ab00 2023-12-19T18:17:39.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.830+0000 7fb135ffb640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb12406a430 0x7fb12406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:39.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.831+0000 7fb135ffb640 1 -- 172.21.15.27:0/574964328 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb13c088690 con 0x7fb14006ab00 2023-12-19T18:17:39.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.831+0000 7fb144e1d640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb12406a430 0x7fb12406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:39.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.832+0000 7fb144e1d640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb12406a430 0x7fb12406c8f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb14006a840 tx=0x7fb130008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:39.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:39.835+0000 7fb135ffb640 1 -- 172.21.15.27:0/574964328 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb13c053cf0 con 0x7fb14006ab00 2023-12-19T18:17:40.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.034+0000 7fb147882640 1 -- 172.21.15.27:0/574964328 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fb140068e90 con 0x7fb14006ab00 2023-12-19T18:17:40.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.038+0000 7fb135ffb640 1 -- 172.21.15.27:0/574964328 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v51) v1 ==== 81+0+33556 (secure 0 0 0) 0x7fb13c057310 con 0x7fb14006ab00 2023-12-19T18:17:40.039 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:40.039 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:17:40.039 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 51, 2023-12-19T18:17:40.039 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:17:40.039 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:17:40.039 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:17:38.605151+0000", 2023-12-19T18:17:40.039 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:17:40.039 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:17:40.039 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:17:40.039 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:17:40.039 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 7, 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:17:40.040 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:17:40.041 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:17:40.041 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:17:40.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.041 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:17:40.041 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:17:40.041 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:17:40.041 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 4097, 2023-12-19T18:17:40.042 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,backfillfull", 2023-12-19T18:17:40.042 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:40.042 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:40.042 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:40.042 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:40.042 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:40.042 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:40.043 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:40.043 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:40.043 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:40.043 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:40.043 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:17:40.043 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:17:40.044 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:17:40.044 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:17:40.044 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:17:40.044 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:40.044 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:40.044 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:40.045 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:40.045 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:40.045 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:40.045 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:40.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.045 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "51", 2023-12-19T18:17:40.046 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:40.046 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:40.046 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:40.046 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:40.046 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:40.046 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:40.046 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:40.047 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:40.047 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:40.047 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:40.047 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:40.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:40.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:40.048 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:40.048 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:40.048 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:40.048 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:40.048 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:40.048 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:40.049 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:40.049 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:40.049 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:40.049 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:40.049 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:40.049 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:40.049 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:40.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.050 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:40.050 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:40.050 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:40.050 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:40.050 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:40.050 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:40.051 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:40.051 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:40.051 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:40.051 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:40.051 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:40.051 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:40.051 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:17:40.051 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:17:40.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.052 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:40.052 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:17:40.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.052 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:40.052 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:17:40.052 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:17:40.053 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:17:40.053 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:17:40.053 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:17:40.053 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:40.053 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:40.053 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:40.053 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.054 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.054 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 6, 2023-12-19T18:17:40.054 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:17:40.054 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:12.527441+0000", 2023-12-19T18:17:40.054 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 4097, 2023-12-19T18:17:40.054 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,backfillfull", 2023-12-19T18:17:40.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:40.055 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:40.055 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:40.055 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:40.055 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:40.055 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:40.055 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:40.056 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:40.056 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:40.056 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:40.056 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:17:40.056 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:17:40.056 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:17:40.056 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:17:40.056 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:17:40.057 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:40.057 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:40.057 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:40.057 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:40.057 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:40.057 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:40.057 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:40.058 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.058 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "51", 2023-12-19T18:17:40.058 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:40.058 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:40.058 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:40.058 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:40.058 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:40.058 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:40.058 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:40.059 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:40.059 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:40.059 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:40.059 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:40.059 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:40.059 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:40.059 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:40.059 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:40.060 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:40.060 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:40.060 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:40.060 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:40.060 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:40.060 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:40.060 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:40.061 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:40.061 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:40.061 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:40.061 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:40.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.061 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:40.061 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:40.062 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:40.062 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:40.062 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:40.062 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:40.062 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:40.062 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:40.062 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:40.063 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:40.063 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:40.063 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:40.063 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:17:40.063 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:17:40.063 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:17:40.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.064 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:40.064 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:40.064 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:17:40.064 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.064 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.064 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:40.065 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:17:40.065 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:17:40.065 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:40.065 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:17:40.065 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:17:40.065 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:40.066 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:40.066 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:40.066 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.066 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 7, 2023-12-19T18:17:40.066 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:17:40.066 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:13.291755+0000", 2023-12-19T18:17:40.066 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 4097, 2023-12-19T18:17:40.067 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,backfillfull", 2023-12-19T18:17:40.067 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:40.067 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:40.067 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:40.067 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:40.067 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:40.067 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:40.067 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:40.067 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:40.068 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:40.068 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:40.068 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:17:40.068 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:17:40.068 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:17:40.068 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:17:40.068 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:17:40.068 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:40.068 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:40.069 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:40.069 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:40.069 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:40.069 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:40.069 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:40.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.069 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "51", 2023-12-19T18:17:40.069 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:40.069 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:40.070 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:40.070 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:40.070 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:40.070 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:40.070 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:40.070 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:40.070 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:40.071 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:40.071 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:40.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:40.071 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:40.071 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:40.071 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:40.071 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:40.072 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:40.072 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:40.072 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:40.072 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:40.072 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:40.072 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:40.072 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:40.073 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:40.073 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:40.073 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:40.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.073 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:40.073 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:40.073 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:40.073 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:40.073 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:40.073 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.074 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.75, 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.75, 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.75, 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.75, 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:40.075 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:40.076 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:40.076 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:40.076 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:40.076 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:40.076 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:40.076 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:40.076 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:40.076 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:40.077 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:40.077 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:40.077 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.077 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:17:40.077 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:40.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.077 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:17:40.077 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:40.077 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.078 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.079 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:40.080 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.080 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.080 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:17:40.080 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:40.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.080 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.080 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:17:40.080 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:40.080 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.080 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:17:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:17:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:17:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:17:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:40.081 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:17:40.082 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:40.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.083 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.083 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:17:40.083 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:40.083 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.083 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.083 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:40.084 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.084 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.084 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:17:40.084 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:40.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.084 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.084 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.085 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:40.086 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.087 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.087 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:17:40.087 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:17:40.087 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:17:40.087 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:17:40.087 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:40.087 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:40.087 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:40.087 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.092 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:17:40.092 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:40.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.092 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.092 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:17:40.092 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:40.092 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.092 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.092 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:17:40.092 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:17:40.092 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:40.093 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:40.094 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:40.094 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:40.094 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:40.094 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:40.094 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:40.094 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:40.094 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.094 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.094 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:17:40.094 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:40.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.095 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.095 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:17:40.095 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:40.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.095 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.095 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:40.095 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.095 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.096 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.097 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:40.098 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:40.099 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:40.099 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.099 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:40.099 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:40.100 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:40.100 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:40.100 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:40.100 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:40.100 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:40.100 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:40.100 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:17:40.101 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:40.102 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.102 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.102 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:40.102 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.102 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.102 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:17:40.102 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:40.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.103 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:17:40.103 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:40.103 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.103 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.105 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:40.105 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.105 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:17:40.105 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:40.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.106 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.106 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:17:40.106 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:40.106 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.106 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.107 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:17:40.107 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:17:40.107 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:17:40.107 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:17:40.107 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:40.107 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:40.107 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:40.107 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:40.108 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:40.108 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:40.108 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:40.108 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:40.108 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:40.108 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:40.108 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:40.108 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:40.109 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:40.109 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:40.109 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:40.109 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:40.109 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.111 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:17:40.112 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull", 2023-12-19T18:17:40.113 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:40.114 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.116 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:40.117 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:17:40.118 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:40.119 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:17:40.120 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:40.121 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.122 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.122 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:40.122 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.122 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.122 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:17:40.122 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:40.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.122 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.122 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:17:40.123 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:17:40.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:40.125 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.126 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:40.127 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.128 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:40.129 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:17:40.130 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:17:40.131 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:17:40.131 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:17:40.131 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:17:40.131 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:40.131 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:17:40.131 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:17:40.131 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:17:40.131 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:17:40.131 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:17:40.131 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:17:40.131 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:17:40.131 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:17:40.132 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:17:40.132 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:17:40.132 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:17:40.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:40.132 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:17:40.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.041+0000 7fb147882640 1 -- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb12406a430 msgr2=0x7fb12406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.041+0000 7fb147882640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb12406a430 0x7fb12406c8f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb14006a840 tx=0x7fb130008040 comp rx=0 tx=0).stop 2023-12-19T18:17:40.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.041+0000 7fb147882640 1 -- 172.21.15.27:0/574964328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14006ab00 msgr2=0x7fb1400c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:40.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.041+0000 7fb147882640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14006ab00 0x7fb1400c9070 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fb13c00c710 tx=0x7fb13c00cbd0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.041+0000 7fb147882640 1 -- 172.21.15.27:0/574964328 shutdown_connections 2023-12-19T18:17:40.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.041+0000 7fb147882640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb12406a430 0x7fb12406c8f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.041+0000 7fb147882640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb14015d260 0x7fb1400bc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.041+0000 7fb147882640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb1400d2af0 0x7fb1401e6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.041+0000 7fb147882640 1 --2- 172.21.15.27:0/574964328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb14006ab00 0x7fb1400c9070 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:40.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.041+0000 7fb147882640 1 -- 172.21.15.27:0/574964328 >> 172.21.15.27:0/574964328 conn(0x7fb1400d2f70 msgr2=0x7fb140153ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:40.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.041+0000 7fb147882640 1 -- 172.21.15.27:0/574964328 shutdown_connections 2023-12-19T18:17:40.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:40.041+0000 7fb147882640 1 -- 172.21.15.27:0/574964328 wait complete. 2023-12-19T18:17:40.135 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-12-19T18:17:40.662 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:17:40.662+0000 7f8f3ae61640 -1 log_channel(cluster) log [ERR] : Health check failed: 3 full osd(s) (OSD_FULL) 2023-12-19T18:17:45.103 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:17:45.113 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:17:45.136 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:17:45.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.372+0000 7fc4849d6640 1 Processor -- start 2023-12-19T18:17:45.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.372+0000 7fc4849d6640 1 -- start start 2023-12-19T18:17:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.374+0000 7fc4849d6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc480070aa0 0x7fc480070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.374+0000 7fc4849d6640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc48015a3a0 0x7fc48015a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.374+0000 7fc4849d6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc480071440 con 0x7fc480154170 2023-12-19T18:17:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.374+0000 7fc4849d6640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc48015b110 con 0x7fc48015a3a0 2023-12-19T18:17:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.374+0000 7fc4849d6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc48006fce0 con 0x7fc480070aa0 2023-12-19T18:17:45.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.374+0000 7fc47f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc480154170 0x7fc480070170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43378/0 (socket says 172.21.15.27:43378) 2023-12-19T18:17:45.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.374+0000 7fc47f7fe640 1 -- 172.21.15.27:0/51570775 learned_addr learned my addr 172.21.15.27:0/51570775 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521856765 0 0) 0x7fc480071440 con 0x7fc480154170 2023-12-19T18:17:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47effd640 1 --2- 172.21.15.27:0/51570775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc480070aa0 0x7fc480070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47ffff640 1 --2- 172.21.15.27:0/51570775 >> v2:172.21.15.192:3300/0 conn(0x7fc48015a3a0 0x7fc48015a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc468003520 con 0x7fc480154170 2023-12-19T18:17:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3173558433 0 0) 0x7fc468003520 con 0x7fc480154170 2023-12-19T18:17:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc480070aa0 msgr2=0x7fc480070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 --2- 172.21.15.27:0/51570775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc480070aa0 0x7fc480070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 >> v2:172.21.15.192:3300/0 conn(0x7fc48015a3a0 msgr2=0x7fc48015a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 --2- 172.21.15.27:0/51570775 >> v2:172.21.15.192:3300/0 conn(0x7fc48015a3a0 0x7fc48015a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc480070880 con 0x7fc480154170 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fc470001670 con 0x7fc480154170 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 >> v1:172.21.15.27:6789/0 conn(0x7fc480154170 legacy=0x7fc480070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 --2- 172.21.15.27:0/51570775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc480070aa0 0x7fc468003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 --2- 172.21.15.27:0/51570775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc468003fa0 0x7fc4680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 --2- 172.21.15.27:0/51570775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc468004d00 0x7fc468005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48015b110 con 0x7fc468004d00 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48006fce0 con 0x7fc468003fa0 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.375+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46800bb40 con 0x7fc480070aa0 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.376+0000 7fc47ffff640 1 --2- 172.21.15.27:0/51570775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc468004d00 0x7fc468005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.376+0000 7fc47f7fe640 1 --2- 172.21.15.27:0/51570775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc480070aa0 0x7fc468003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.376+0000 7fc47effd640 1 --2- 172.21.15.27:0/51570775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc468003fa0 0x7fc4680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.376+0000 7fc47ffff640 1 -- 172.21.15.27:0/51570775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc480070aa0 msgr2=0x7fc468003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.376+0000 7fc47ffff640 1 --2- 172.21.15.27:0/51570775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc480070aa0 0x7fc468003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.376+0000 7fc47ffff640 1 -- 172.21.15.27:0/51570775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc468003fa0 msgr2=0x7fc4680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:45.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.376+0000 7fc47ffff640 1 --2- 172.21.15.27:0/51570775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc468003fa0 0x7fc4680047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.376+0000 7fc47ffff640 1 -- 172.21.15.27:0/51570775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc468003520 con 0x7fc468004d00 2023-12-19T18:17:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.377+0000 7fc47ffff640 1 --2- 172.21.15.27:0/51570775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc468004d00 0x7fc468005180 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc478007360 tx=0x7fc4680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15876b0084e68761 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.378+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47802ccf0 con 0x7fc468004d00 2023-12-19T18:17:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.378+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47802ce50 con 0x7fc468004d00 2023-12-19T18:17:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.378+0000 7fc47e7fc640 1 -- 172.21.15.27:0/51570775 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc478034660 con 0x7fc468004d00 2023-12-19T18:17:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.378+0000 7fc4849d6640 1 -- 172.21.15.27:0/51570775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc468004d00 msgr2=0x7fc468005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.378+0000 7fc4849d6640 1 --2- 172.21.15.27:0/51570775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc468004d00 0x7fc468005180 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc478007360 tx=0x7fc4680057f0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.378+0000 7fc47ffff640 1 -- 172.21.15.27:0/51570775 reap_dead start 2023-12-19T18:17:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.378+0000 7fc4849d6640 1 -- 172.21.15.27:0/51570775 shutdown_connections 2023-12-19T18:17:45.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.378+0000 7fc4849d6640 1 -- 172.21.15.27:0/51570775 >> 172.21.15.27:0/51570775 conn(0x7fc48006a7d0 msgr2=0x7fc4800ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.379+0000 7fc4849d6640 1 -- 172.21.15.27:0/51570775 shutdown_connections 2023-12-19T18:17:45.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.379+0000 7fc4849d6640 1 -- 172.21.15.27:0/51570775 wait complete. 2023-12-19T18:17:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.380+0000 7fc4849d6640 1 Processor -- start 2023-12-19T18:17:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.381+0000 7fc4849d6640 1 -- start start 2023-12-19T18:17:45.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.381+0000 7fc4849d6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc468004d00 0x7fc4800b40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.382+0000 7fc4849d6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4801591e0 0x7fc480155870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.382+0000 7fc4849d6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc48014c5b0 0x7fc480152440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.382+0000 7fc4849d6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4800b3c90 con 0x7fc468004d00 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.382+0000 7fc4849d6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4801575a0 con 0x7fc4801591e0 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.382+0000 7fc4849d6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc480157880 con 0x7fc48014c5b0 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.382+0000 7fc47f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc468004d00 0x7fc4800b40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.382+0000 7fc47f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc468004d00 0x7fc4800b40b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42614/0 (socket says 172.21.15.27:42614) 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.382+0000 7fc47f7fe640 1 -- 172.21.15.27:0/2429558080 learned_addr learned my addr 172.21.15.27:0/2429558080 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.382+0000 7fc47f7fe640 1 -- 172.21.15.27:0/2429558080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc48014c5b0 msgr2=0x7fc480152440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.382+0000 7fc47f7fe640 1 --2- 172.21.15.27:0/2429558080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc48014c5b0 0x7fc480152440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.382+0000 7fc47f7fe640 1 -- 172.21.15.27:0/2429558080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4801591e0 msgr2=0x7fc480155870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.382+0000 7fc47f7fe640 1 --2- 172.21.15.27:0/2429558080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4801591e0 0x7fc480155870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.382+0000 7fc47f7fe640 1 -- 172.21.15.27:0/2429558080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4800b38b0 con 0x7fc468004d00 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.383+0000 7fc47f7fe640 1 --2- 172.21.15.27:0/2429558080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc468004d00 0x7fc4800b40b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc47000e840 tx=0x7fc47000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.383+0000 7fc467fff640 1 -- 172.21.15.27:0/2429558080 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47000c850 con 0x7fc468004d00 2023-12-19T18:17:45.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.383+0000 7fc467fff640 1 -- 172.21.15.27:0/2429558080 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc47000c9d0 con 0x7fc468004d00 2023-12-19T18:17:45.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.383+0000 7fc4849d6640 1 -- 172.21.15.27:0/2429558080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48014ca60 con 0x7fc468004d00 2023-12-19T18:17:45.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.383+0000 7fc467fff640 1 -- 172.21.15.27:0/2429558080 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc47000cd80 con 0x7fc468004d00 2023-12-19T18:17:45.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.383+0000 7fc4849d6640 1 -- 172.21.15.27:0/2429558080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc480157310 con 0x7fc468004d00 2023-12-19T18:17:45.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.385+0000 7fc4849d6640 1 -- 172.21.15.27:0/2429558080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4800b3dd0 con 0x7fc468004d00 2023-12-19T18:17:45.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.385+0000 7fc467fff640 1 -- 172.21.15.27:0/2429558080 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc470013420 con 0x7fc468004d00 2023-12-19T18:17:45.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.387+0000 7fc467fff640 1 --2- 172.21.15.27:0/2429558080 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc46c06a460 0x7fc46c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:45.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.387+0000 7fc467fff640 1 -- 172.21.15.27:0/2429558080 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fc470088550 con 0x7fc468004d00 2023-12-19T18:17:45.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.387+0000 7fc467fff640 1 -- 172.21.15.27:0/2429558080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=57}) v3 -- 0x7fc46c072170 con 0x7fc468004d00 2023-12-19T18:17:45.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.387+0000 7fc47effd640 1 --2- 172.21.15.27:0/2429558080 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc46c06a460 0x7fc46c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:45.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.389+0000 7fc47effd640 1 --2- 172.21.15.27:0/2429558080 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc46c06a460 0x7fc46c06c920 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc4800ae6f0 tx=0x7fc47400b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:45.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.392+0000 7fc467fff640 1 -- 172.21.15.27:0/2429558080 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc4800b3dd0 con 0x7fc468004d00 2023-12-19T18:17:45.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.586+0000 7fc4849d6640 1 -- 172.21.15.27:0/2429558080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc4800b3dd0 con 0x7fc468004d00 2023-12-19T18:17:45.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.590+0000 7fc467fff640 1 -- 172.21.15.27:0/2429558080 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v56) v1 ==== 81+0+34614 (secure 0 0 0) 0x7fc4800b3dd0 con 0x7fc468004d00 2023-12-19T18:17:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:45.594 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:17:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 56, 2023-12-19T18:17:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:17:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:17:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:17:44.720262+0000", 2023-12-19T18:17:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:17:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:17:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:17:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:17:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:17:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:17:45.594 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 7, 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "luminous", 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:17:45.595 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:45.596 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:45.597 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "52", 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:45.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:17:45.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 6, 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:12.527441+0000", 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:45.602 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.603 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "52", 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:45.604 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:45.605 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:45.606 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.607 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 7, 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:13.291755+0000", 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:45.608 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "52", 2023-12-19T18:17:45.609 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:45.610 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:45.611 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:45.612 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.25, 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.25, 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.25, 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.25, 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.613 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:45.614 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:45.616 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:45.617 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:17:45.618 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull", 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:45.619 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.620 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.621 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:17:45.622 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:45.623 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull", 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:45.624 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.625 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:17:45.626 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:17:45.627 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:45.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:45.629 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:45.630 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.631 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.632 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:17:45.634 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull", 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:45.635 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.636 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.637 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:45.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.639 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:45.640 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.641 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.642 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:17:45.643 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:17:45.644 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:45.645 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:45.646 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:17:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.647 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:17:45.648 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:45.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:45.650 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:45.651 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:17:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:17:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.653 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:45.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:17:45.656 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull", 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:45.657 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:45.658 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:17:45.659 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:45.660 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:17:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:45.661 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:17:45.662 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [ 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.5", 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.663 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.6", 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.8", 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.664 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.9", 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.20", 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:45.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 0 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:17:45.666 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:17:45.667 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:17:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:17:45.668 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:45.668 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:17:45.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.592+0000 7fc4849d6640 1 -- 172.21.15.27:0/2429558080 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc46c06a460 msgr2=0x7fc46c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.592+0000 7fc4849d6640 1 --2- 172.21.15.27:0/2429558080 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc46c06a460 0x7fc46c06c920 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc4800ae6f0 tx=0x7fc47400b2e0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.592+0000 7fc4849d6640 1 -- 172.21.15.27:0/2429558080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc468004d00 msgr2=0x7fc4800b40b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:45.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.592+0000 7fc4849d6640 1 --2- 172.21.15.27:0/2429558080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc468004d00 0x7fc4800b40b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc47000e840 tx=0x7fc47000ed00 comp rx=0 tx=0).stop 2023-12-19T18:17:45.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.593+0000 7fc4849d6640 1 -- 172.21.15.27:0/2429558080 shutdown_connections 2023-12-19T18:17:45.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.593+0000 7fc4849d6640 1 --2- 172.21.15.27:0/2429558080 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc46c06a460 0x7fc46c06c920 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.593+0000 7fc4849d6640 1 --2- 172.21.15.27:0/2429558080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc48014c5b0 0x7fc480152440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.593+0000 7fc4849d6640 1 --2- 172.21.15.27:0/2429558080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc4801591e0 0x7fc480155870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.593+0000 7fc4849d6640 1 --2- 172.21.15.27:0/2429558080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc468004d00 0x7fc4800b40b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:45.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.593+0000 7fc4849d6640 1 -- 172.21.15.27:0/2429558080 >> 172.21.15.27:0/2429558080 conn(0x7fc48006a7d0 msgr2=0x7fc4800b0790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:45.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.593+0000 7fc4849d6640 1 -- 172.21.15.27:0/2429558080 shutdown_connections 2023-12-19T18:17:45.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:45.593+0000 7fc4849d6640 1 -- 172.21.15.27:0/2429558080 wait complete. 2023-12-19T18:17:45.671 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-12-19T18:17:45.671 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:17:45.671 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:45.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:45.737 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-12-19T18:17:45.994 INFO:teuthology.orchestra.run.smithi027.stderr:dd: error writing 'large_file_b': No space left on device 2023-12-19T18:17:45.994 INFO:teuthology.orchestra.run.smithi027.stderr:dd: closing output file 'large_file_b': No space left on device 2023-12-19T18:17:45.995 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:17:45.996 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:17:46.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.228+0000 7f06dcd3d640 1 Processor -- start 2023-12-19T18:17:46.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.228+0000 7f06dcd3d640 1 -- start start 2023-12-19T18:17:46.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.229+0000 7f06dcd3d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d80b49d0 0x7f06d8160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.230+0000 7f06dcd3d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d80b5ca0 con 0x7f06d815cf90 2023-12-19T18:17:46.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.230+0000 7f06dcd3d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06d8161800 con 0x7f06d80b89b0 2023-12-19T18:17:46.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.230+0000 7f06dcd3d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d8161240 con 0x7f06d80b49d0 2023-12-19T18:17:46.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.230+0000 7f06d7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f06d815cf90 0x7f06d80d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43388/0 (socket says 172.21.15.27:43388) 2023-12-19T18:17:46.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.230+0000 7f06d7fff640 1 -- 172.21.15.27:0/2365871968 learned_addr learned my addr 172.21.15.27:0/2365871968 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:17:46.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.230+0000 7f06d77fe640 1 --2- 172.21.15.27:0/2365871968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d80b49d0 0x7f06d8160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 450930950 0 0) 0x7f06d80b5ca0 con 0x7f06d815cf90 2023-12-19T18:17:46.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06c4003520 con 0x7f06d815cf90 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1206348315 0 0) 0x7f06c4003520 con 0x7f06d815cf90 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d80b49d0 msgr2=0x7f06d8160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 --2- 172.21.15.27:0/2365871968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d80b49d0 0x7f06d8160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 >> v1:172.21.15.192:6789/0 conn(0x7f06d80b89b0 legacy=0x7f06d815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d8161520 con 0x7f06d815cf90 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f06d0001670 con 0x7f06d815cf90 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 >> v1:172.21.15.27:6789/0 conn(0x7f06d815cf90 legacy=0x7f06d80d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 --2- 172.21.15.27:0/2365871968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40039a0 0x7f06c4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 --2- 172.21.15.27:0/2365871968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06c40047c0 0x7f06c4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 --2- 172.21.15.27:0/2365871968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d80b49d0 0x7f06c4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d8161240 con 0x7f06c40039a0 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c400baf0 con 0x7f06c40047c0 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c400bcb0 con 0x7f06d80b49d0 2023-12-19T18:17:46.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.231+0000 7f06d77fe640 1 --2- 172.21.15.27:0/2365871968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40039a0 0x7f06c4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.232+0000 7f06d7fff640 1 --2- 172.21.15.27:0/2365871968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d80b49d0 0x7f06c4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.232+0000 7f06d6ffd640 1 --2- 172.21.15.27:0/2365871968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06c40047c0 0x7f06c4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.232+0000 7f06d77fe640 1 -- 172.21.15.27:0/2365871968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d80b49d0 msgr2=0x7f06c4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:46.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.232+0000 7f06d77fe640 1 --2- 172.21.15.27:0/2365871968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d80b49d0 0x7f06c4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.232+0000 7f06d77fe640 1 -- 172.21.15.27:0/2365871968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06c40047c0 msgr2=0x7f06c4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:46.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.232+0000 7f06d77fe640 1 --2- 172.21.15.27:0/2365871968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06c40047c0 0x7f06c4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.232+0000 7f06d77fe640 1 -- 172.21.15.27:0/2365871968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c4003520 con 0x7f06c40039a0 2023-12-19T18:17:46.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.232+0000 7f06d6ffd640 1 --2- 172.21.15.27:0/2365871968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06c40047c0 0x7f06c4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:46.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.234+0000 7f06d77fe640 1 --2- 172.21.15.27:0/2365871968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40039a0 0x7f06c4003dc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009a80 tx=0x7f06c40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5113bbb488c2f970 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:46.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.234+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8005990 con 0x7f06c40039a0 2023-12-19T18:17:46.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.234+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c8005af0 con 0x7f06c40039a0 2023-12-19T18:17:46.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.234+0000 7f06d67fc640 1 -- 172.21.15.27:0/2365871968 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8005ea0 con 0x7f06c40039a0 2023-12-19T18:17:46.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.234+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2365871968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40039a0 msgr2=0x7f06c4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.234+0000 7f06dcd3d640 1 --2- 172.21.15.27:0/2365871968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40039a0 0x7f06c4003dc0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f06c8009a80 tx=0x7f06c40057f0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.234+0000 7f06d7fff640 1 -- 172.21.15.27:0/2365871968 reap_dead start 2023-12-19T18:17:46.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.235+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2365871968 shutdown_connections 2023-12-19T18:17:46.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.235+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2365871968 >> 172.21.15.27:0/2365871968 conn(0x7f06d80cf760 msgr2=0x7f06d80cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:46.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.235+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2365871968 shutdown_connections 2023-12-19T18:17:46.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.235+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2365871968 wait complete. 2023-12-19T18:17:46.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.237+0000 7f06dcd3d640 1 Processor -- start 2023-12-19T18:17:46.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.237+0000 7f06dcd3d640 1 -- start start 2023-12-19T18:17:46.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.238+0000 7f06dcd3d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40039a0 0x7f06d80ccb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.238+0000 7f06dcd3d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d81f1d20 0x7f06d80d14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.238+0000 7f06dcd3d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06d81f21a0 0x7f06d80d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.238+0000 7f06dcd3d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80cd050 con 0x7f06c40039a0 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.238+0000 7f06dcd3d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06d80cc680 con 0x7f06d81f21a0 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.238+0000 7f06dcd3d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06d815f690 con 0x7f06d81f1d20 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.238+0000 7f06d77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40039a0 0x7f06d80ccb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.238+0000 7f06d77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40039a0 0x7f06d80ccb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42630/0 (socket says 172.21.15.27:42630) 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.238+0000 7f06d77fe640 1 -- 172.21.15.27:0/2880159537 learned_addr learned my addr 172.21.15.27:0/2880159537 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.238+0000 7f06d6ffd640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d81f1d20 0x7f06d80d14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.239+0000 7f06d7fff640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06d81f21a0 0x7f06d80d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.239+0000 7f06d77fe640 1 -- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d81f1d20 msgr2=0x7f06d80d14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.239+0000 7f06d77fe640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d81f1d20 0x7f06d80d14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.239+0000 7f06d77fe640 1 -- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06d81f21a0 msgr2=0x7f06d80d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.239+0000 7f06d77fe640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06d81f21a0 0x7f06d80d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.239+0000 7f06d77fe640 1 -- 172.21.15.27:0/2880159537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06d80cbd50 con 0x7f06c40039a0 2023-12-19T18:17:46.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.239+0000 7f06d77fe640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40039a0 0x7f06d80ccb10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f06c8036040 tx=0x7f06c8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.239+0000 7f06cffff640 1 -- 172.21.15.27:0/2880159537 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c8005e70 con 0x7f06c40039a0 2023-12-19T18:17:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.239+0000 7f06cffff640 1 -- 172.21.15.27:0/2880159537 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c8004020 con 0x7f06c40039a0 2023-12-19T18:17:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.239+0000 7f06cffff640 1 -- 172.21.15.27:0/2880159537 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c80043d0 con 0x7f06c40039a0 2023-12-19T18:17:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.239+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2880159537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06d80cc030 con 0x7f06c40039a0 2023-12-19T18:17:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.239+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2880159537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06d815e8f0 con 0x7f06c40039a0 2023-12-19T18:17:46.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.241+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2880159537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06d804e840 con 0x7f06c40039a0 2023-12-19T18:17:46.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.241+0000 7f06cffff640 1 -- 172.21.15.27:0/2880159537 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f06c8003610 con 0x7f06c40039a0 2023-12-19T18:17:46.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.243+0000 7f06cffff640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f06b806a430 0x7f06b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.243+0000 7f06cffff640 1 -- 172.21.15.27:0/2880159537 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f06c80aab70 con 0x7f06c40039a0 2023-12-19T18:17:46.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.243+0000 7f06cffff640 1 -- 172.21.15.27:0/2880159537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=57}) v3 -- 0x7f06b8072140 con 0x7f06c40039a0 2023-12-19T18:17:46.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.243+0000 7f06d6ffd640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f06b806a430 0x7f06b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.245+0000 7f06d6ffd640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f06b806a430 0x7f06b806c8f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f06d815b910 tx=0x7f06c0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:46.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.247+0000 7f06cffff640 1 -- 172.21.15.27:0/2880159537 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f06c80760d0 con 0x7f06c40039a0 2023-12-19T18:17:46.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.450+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2880159537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f06d80d08b0 con 0x7f06c40039a0 2023-12-19T18:17:46.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.454+0000 7f06cffff640 1 -- 172.21.15.27:0/2880159537 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v56) v1 ==== 81+0+34614 (secure 0 0 0) 0x7f06c80796f0 con 0x7f06c40039a0 2023-12-19T18:17:46.454 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:46.455 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:17:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 56, 2023-12-19T18:17:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:17:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:17:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:17:44.720262+0000", 2023-12-19T18:17:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:17:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:17:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:17:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:17:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:17:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:17:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:17:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:17:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:17:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:17:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:17:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:17:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:17:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:17:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 7, 2023-12-19T18:17:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:17:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:17:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "luminous", 2023-12-19T18:17:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:17:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:17:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:17:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:17:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:17:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:17:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:17:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:17:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:46.461 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:17:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:17:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:17:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:17:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:17:46.462 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:46.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "52", 2023-12-19T18:17:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:46.464 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:46.465 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:46.466 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.467 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:46.468 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:17:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:17:46.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:17:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.470 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:17:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:17:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:17:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:17:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:17:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:46.471 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 6, 2023-12-19T18:17:46.472 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:17:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:12.527441+0000", 2023-12-19T18:17:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:17:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:17:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:46.473 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:17:46.474 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:46.475 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "52", 2023-12-19T18:17:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:46.476 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:46.477 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:46.478 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.479 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:46.480 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:17:46.481 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:17:46.482 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 7, 2023-12-19T18:17:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:17:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:13.291755+0000", 2023-12-19T18:17:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:17:46.483 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:17:46.484 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:17:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "52", 2023-12-19T18:17:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:46.485 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:46.486 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:46.487 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:17:46.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.25, 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.25, 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.25, 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.25, 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:46.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:46.490 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.491 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:17:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:17:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:17:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:17:46.494 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:17:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.495 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:17:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:17:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:17:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:17:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:17:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull", 2023-12-19T18:17:46.496 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:46.497 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:17:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:17:46.499 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:46.500 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:46.501 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull", 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:46.502 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:46.503 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:17:46.504 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:17:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.505 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:17:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:17:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:46.506 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:17:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.507 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:17:46.508 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.509 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.512 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.513 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:17:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:17:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:17:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:17:46.514 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:17:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:17:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull", 2023-12-19T18:17:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:46.515 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:46.516 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.517 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:17:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.518 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:17:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:46.519 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:17:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:17:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.521 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:17:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:46.522 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:17:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.523 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:17:46.524 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:17:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.525 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:17:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:17:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:17:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:17:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:46.526 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:17:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:46.527 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:46.528 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.529 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:17:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:17:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.530 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:17:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:46.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:17:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:17:46.533 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:17:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:46.534 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.535 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:17:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:17:46.536 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:17:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:17:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:17:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:17:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:46.537 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:17:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:46.538 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:46.539 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:17:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:17:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.541 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:17:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.542 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:17:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.543 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:46.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:17:46.545 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:17:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:17:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:17:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:46.546 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:17:46.547 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:46.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:46.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull", 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.551 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:46.552 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:17:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:46.553 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:17:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:46.554 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:17:46.555 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:17:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:17:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:46.557 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [ 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.5", 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.558 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.6", 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.8", 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.9", 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.560 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.20", 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 0 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:17:46.561 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:17:46.562 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:17:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:17:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:17:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:17:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:17:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:17:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:17:46.563 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:46.563 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:17:46.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.457+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f06b806a430 msgr2=0x7f06b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.457+0000 7f06dcd3d640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f06b806a430 0x7f06b806c8f0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f06d815b910 tx=0x7f06c0009040 comp rx=0 tx=0).stop 2023-12-19T18:17:46.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.457+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2880159537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40039a0 msgr2=0x7f06d80ccb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.457+0000 7f06dcd3d640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40039a0 0x7f06d80ccb10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f06c8036040 tx=0x7f06c8004730 comp rx=0 tx=0).stop 2023-12-19T18:17:46.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.457+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2880159537 shutdown_connections 2023-12-19T18:17:46.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.457+0000 7f06dcd3d640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f06b806a430 0x7f06b806c8f0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.457+0000 7f06dcd3d640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f06d81f21a0 0x7f06d80d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.457+0000 7f06dcd3d640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f06d81f1d20 0x7f06d80d14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.457+0000 7f06dcd3d640 1 --2- 172.21.15.27:0/2880159537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06c40039a0 0x7f06d80ccb10 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.457+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2880159537 >> 172.21.15.27:0/2880159537 conn(0x7f06d80cf760 msgr2=0x7f06d8157790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:46.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.457+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2880159537 shutdown_connections 2023-12-19T18:17:46.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.457+0000 7f06dcd3d640 1 -- 172.21.15.27:0/2880159537 wait complete. 2023-12-19T18:17:46.566 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:17:46.735 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:17:46.735+0000 7f8f3ae61640 -1 log_channel(cluster) log [ERR] : Health check update: 4 full osd(s) (OSD_FULL) 2023-12-19T18:17:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.808+0000 7f6b2413c640 1 Processor -- start 2023-12-19T18:17:46.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.808+0000 7f6b2413c640 1 -- start start 2023-12-19T18:17:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.809+0000 7f6b2413c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6b1c06bbf0 0x7f6b1c0cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.809+0000 7f6b2413c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6b1c06c880 0x7f6b1c0cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.809+0000 7f6b2413c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b1c15b180 con 0x7f6b1c159c40 2023-12-19T18:17:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.809+0000 7f6b2413c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c0ca850 con 0x7f6b1c06c880 2023-12-19T18:17:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.809+0000 7f6b2413c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1c15a570 con 0x7f6b1c06bbf0 2023-12-19T18:17:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.810+0000 7f6b226d9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6b1c06c880 0x7f6b1c0cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.810+0000 7f6b226d9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6b1c06c880 0x7f6b1c0cdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42642/0 (socket says 172.21.15.27:42642) 2023-12-19T18:17:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.810+0000 7f6b226d9640 1 -- 172.21.15.27:0/2951495991 learned_addr learned my addr 172.21.15.27:0/2951495991 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:46.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.810+0000 7f6b216d7640 1 --2- 172.21.15.27:0/2951495991 >> v2:172.21.15.192:3300/0 conn(0x7f6b1c06bbf0 0x7f6b1c0cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.810+0000 7f6b226d9640 1 -- 172.21.15.27:0/2951495991 >> v2:172.21.15.192:3300/0 conn(0x7f6b1c06bbf0 msgr2=0x7f6b1c0cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.810+0000 7f6b226d9640 1 --2- 172.21.15.27:0/2951495991 >> v2:172.21.15.192:3300/0 conn(0x7f6b1c06bbf0 0x7f6b1c0cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.810+0000 7f6b226d9640 1 -- 172.21.15.27:0/2951495991 >> v1:172.21.15.192:6789/0 conn(0x7f6b1c159c40 legacy=0x7f6b1c06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:46.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.810+0000 7f6b226d9640 1 -- 172.21.15.27:0/2951495991 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1c15a850 con 0x7f6b1c06c880 2023-12-19T18:17:46.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.812+0000 7f6b226d9640 1 --2- 172.21.15.27:0/2951495991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c06c880 0x7f6b1c0cdd40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c06cf60 tx=0x7f6b1402e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af325819e7a5866a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:46.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.812+0000 7f6b20ed6640 1 -- 172.21.15.27:0/2951495991 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b1402ea90 con 0x7f6b1c06c880 2023-12-19T18:17:46.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.812+0000 7f6b20ed6640 1 -- 172.21.15.27:0/2951495991 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b1402ebf0 con 0x7f6b1c06c880 2023-12-19T18:17:46.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.812+0000 7f6b20ed6640 1 -- 172.21.15.27:0/2951495991 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b14009600 con 0x7f6b1c06c880 2023-12-19T18:17:46.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.813+0000 7f6b2413c640 1 -- 172.21.15.27:0/2951495991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c06c880 msgr2=0x7f6b1c0cdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.813+0000 7f6b2413c640 1 --2- 172.21.15.27:0/2951495991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c06c880 0x7f6b1c0cdd40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c06cf60 tx=0x7f6b1402e0f0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.813+0000 7f6b2413c640 1 -- 172.21.15.27:0/2951495991 shutdown_connections 2023-12-19T18:17:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.813+0000 7f6b2413c640 1 --2- 172.21.15.27:0/2951495991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c06c880 0x7f6b1c0cdd40 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.813+0000 7f6b2413c640 1 --2- 172.21.15.27:0/2951495991 >> v2:172.21.15.192:3300/0 conn(0x7f6b1c06bbf0 0x7f6b1c0cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.813+0000 7f6b2413c640 1 -- 172.21.15.27:0/2951495991 >> 172.21.15.27:0/2951495991 conn(0x7f6b1c06a330 msgr2=0x7f6b1c0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:46.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.813+0000 7f6b2413c640 1 -- 172.21.15.27:0/2951495991 shutdown_connections 2023-12-19T18:17:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.813+0000 7f6b2413c640 1 -- 172.21.15.27:0/2951495991 wait complete. 2023-12-19T18:17:46.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.815+0000 7f6b2413c640 1 Processor -- start 2023-12-19T18:17:46.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.815+0000 7f6b2413c640 1 -- start start 2023-12-19T18:17:46.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.816+0000 7f6b2413c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c06bbf0 0x7f6b1c0b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.816+0000 7f6b2413c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b1c06c880 0x7f6b1c158f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.816+0000 7f6b2413c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b1c159c40 0x7f6b1c1f1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.817+0000 7f6b2413c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c159490 con 0x7f6b1c06bbf0 2023-12-19T18:17:46.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.817+0000 7f6b2413c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0b1d70 con 0x7f6b1c06c880 2023-12-19T18:17:46.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.817+0000 7f6b2413c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b1c0b2050 con 0x7f6b1c159c40 2023-12-19T18:17:46.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.817+0000 7f6b21ed8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c06bbf0 0x7f6b1c0b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.817+0000 7f6b21ed8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c06bbf0 0x7f6b1c0b16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42658/0 (socket says 172.21.15.27:42658) 2023-12-19T18:17:46.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.817+0000 7f6b21ed8640 1 -- 172.21.15.27:0/3657748678 learned_addr learned my addr 172.21.15.27:0/3657748678 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:46.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.817+0000 7f6b21ed8640 1 -- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b1c159c40 msgr2=0x7f6b1c1f1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.817+0000 7f6b226d9640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b1c159c40 0x7f6b1c1f1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.817+0000 7f6b216d7640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b1c06c880 0x7f6b1c158f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.817+0000 7f6b21ed8640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b1c159c40 0x7f6b1c1f1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.817+0000 7f6b21ed8640 1 -- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b1c06c880 msgr2=0x7f6b1c158f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.817+0000 7f6b21ed8640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b1c06c880 0x7f6b1c158f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.817+0000 7f6b21ed8640 1 -- 172.21.15.27:0/3657748678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1400bd40 con 0x7f6b1c06bbf0 2023-12-19T18:17:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.818+0000 7f6b21ed8640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c06bbf0 0x7f6b1c0b16e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800b3f0 tx=0x7f6b1800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.818+0000 7f6b127fc640 1 -- 172.21.15.27:0/3657748678 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18011830 con 0x7f6b1c06bbf0 2023-12-19T18:17:46.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.818+0000 7f6b127fc640 1 -- 172.21.15.27:0/3657748678 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b18011990 con 0x7f6b1c06bbf0 2023-12-19T18:17:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.818+0000 7f6b127fc640 1 -- 172.21.15.27:0/3657748678 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b18011d40 con 0x7f6b1c06bbf0 2023-12-19T18:17:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.818+0000 7f6b2413c640 1 -- 172.21.15.27:0/3657748678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b1c0b2330 con 0x7f6b1c06bbf0 2023-12-19T18:17:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.818+0000 7f6b2413c640 1 -- 172.21.15.27:0/3657748678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b1c0b28f0 con 0x7f6b1c06bbf0 2023-12-19T18:17:46.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.819+0000 7f6b2413c640 1 -- 172.21.15.27:0/3657748678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b1c068e90 con 0x7f6b1c06bbf0 2023-12-19T18:17:46.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.820+0000 7f6b127fc640 1 -- 172.21.15.27:0/3657748678 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6b1801a400 con 0x7f6b1c06bbf0 2023-12-19T18:17:46.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.821+0000 7f6b127fc640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6b0006a430 0x7f6b0006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:46.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.822+0000 7f6b127fc640 1 -- 172.21.15.27:0/3657748678 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6b180888c0 con 0x7f6b1c06bbf0 2023-12-19T18:17:46.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.822+0000 7f6b127fc640 1 -- 172.21.15.27:0/3657748678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=58}) v3 -- 0x7f6b00072140 con 0x7f6b1c06bbf0 2023-12-19T18:17:46.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.822+0000 7f6b216d7640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6b0006a430 0x7f6b0006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:46.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.823+0000 7f6b216d7640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6b0006a430 0x7f6b0006c8f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c1f17a0 tx=0x7f6b0c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:46.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:46.826+0000 7f6b127fc640 1 -- 172.21.15.27:0/3657748678 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6b18015060 con 0x7f6b1c06bbf0 2023-12-19T18:17:47.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.063+0000 7f6b2413c640 1 -- 172.21.15.27:0/3657748678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b1c0c9980 con 0x7f6b1c06bbf0 2023-12-19T18:17:47.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.064+0000 7f6b127fc640 1 -- 172.21.15.27:0/3657748678 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f6b18053e10 con 0x7f6b1c06bbf0 2023-12-19T18:17:47.065 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:47.065 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/930391176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":930391176},{"type":"v1","addr":"172.21.15.192:6837","nonce":930391176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5179,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3606564673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3606564673},{"type":"v1","addr":"172.21.15.192:6836","nonce":3606564673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3604117097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3604117097},{"type":"v1","addr":"172.21.15.27:6834","nonce":3604117097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:14.250689+0000","modified":"2023-12-19T18:17:17.758670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3814458529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3814458529},{"type":"v1","addr":"172.21.15.27:6835","nonce":3814458529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T18:17:47.065 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.067+0000 7f6b2413c640 1 -- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6b0006a430 msgr2=0x7f6b0006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.067+0000 7f6b2413c640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6b0006a430 0x7f6b0006c8f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c1f17a0 tx=0x7f6b0c008040 comp rx=0 tx=0).stop 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.067+0000 7f6b2413c640 1 -- 172.21.15.27:0/3657748678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c06bbf0 msgr2=0x7f6b1c0b16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.067+0000 7f6b2413c640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c06bbf0 0x7f6b1c0b16e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6b1800b3f0 tx=0x7f6b1800b8b0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.068+0000 7f6b2413c640 1 -- 172.21.15.27:0/3657748678 shutdown_connections 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.068+0000 7f6b2413c640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6b0006a430 0x7f6b0006c8f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.068+0000 7f6b2413c640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6b1c159c40 0x7f6b1c1f1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.068+0000 7f6b2413c640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6b1c06c880 0x7f6b1c158f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.068+0000 7f6b2413c640 1 --2- 172.21.15.27:0/3657748678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b1c06bbf0 0x7f6b1c0b16e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.068+0000 7f6b2413c640 1 -- 172.21.15.27:0/3657748678 >> 172.21.15.27:0/3657748678 conn(0x7f6b1c06a330 msgr2=0x7f6b1c15b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.068+0000 7f6b2413c640 1 -- 172.21.15.27:0/3657748678 shutdown_connections 2023-12-19T18:17:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.068+0000 7f6b2413c640 1 -- 172.21.15.27:0/3657748678 wait complete. 2023-12-19T18:17:47.089 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-12-19T18:17:47.343 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":5679,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":33.070862621000003,"mdsmap_epoch":23,"osdmap_epoch":57,"osdmap_epoch_barrier":50,"uptime":38.342930330000002} 2023-12-19T18:17:47.343 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 5679, "mdsmap_epoch": 23, "osdmap_epoch": 57, "osdmap_epoch_barrier": 50, "rank_uptime": 33.070862621, "state": "up:active", "uptime": 38.34293033, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:17:47.344 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-12-19T18:17:47.344 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:17:47.344 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:17:47.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:47.397 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-12-19T18:17:47.453 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-12-19T18:17:47.453+0000 7f85f3fff640 -1 client.5832 handle_client_reply no pending request on tid 14 safe is:1 2023-12-19T18:17:47.454 INFO:teuthology.orchestra.run.smithi027.stderr:dd: failed to open 'small_file_1': Permission denied 2023-12-19T18:17:47.454 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:17:47.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:47.455 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-12-19T18:17:47.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:17:47.570 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-12-19T18:17:47.642 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:17:47.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.866+0000 7f51b19a7640 1 Processor -- start 2023-12-19T18:17:47.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.866+0000 7f51b19a7640 1 -- start start 2023-12-19T18:17:47.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.867+0000 7f51b19a7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51ac15ac50 0x7f51ac070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:47.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.867+0000 7f51b19a7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f51ac0af360 0x7f51ac06ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.867+0000 7f51b19a7640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac0adf10 con 0x7f51ac0aef20 2023-12-19T18:17:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.867+0000 7f51b19a7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ac0adac0 con 0x7f51ac0af360 2023-12-19T18:17:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.867+0000 7f51b19a7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0ae160 con 0x7f51ac15ac50 2023-12-19T18:17:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.867+0000 7f51b11a6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f51ac0af360 0x7f51ac06ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.867+0000 7f51b11a6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f51ac0af360 0x7f51ac06ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42664/0 (socket says 172.21.15.27:42664) 2023-12-19T18:17:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.867+0000 7f51b11a6640 1 -- 172.21.15.27:0/3770525637 learned_addr learned my addr 172.21.15.27:0/3770525637 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.867+0000 7f51b09a5640 1 --2- 172.21.15.27:0/3770525637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51ac15ac50 0x7f51ac070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.867+0000 7f51b11a6640 1 -- 172.21.15.27:0/3770525637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51ac15ac50 msgr2=0x7f51ac070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.867+0000 7f51b11a6640 1 --2- 172.21.15.27:0/3770525637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51ac15ac50 0x7f51ac070fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.867+0000 7f51b11a6640 1 -- 172.21.15.27:0/3770525637 >> v1:172.21.15.192:6789/0 conn(0x7f51ac0aef20 legacy=0x7f51ac06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.868+0000 7f51b11a6640 1 -- 172.21.15.27:0/3770525637 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac0ae440 con 0x7f51ac0af360 2023-12-19T18:17:47.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.868+0000 7f51b09a5640 1 --2- 172.21.15.27:0/3770525637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51ac15ac50 0x7f51ac070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:47.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.869+0000 7f51b11a6640 1 --2- 172.21.15.27:0/3770525637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51ac0af360 0x7f51ac06ba20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009e50 tx=0x7f51a402bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7aa626ec7a20aac server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:47.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.869+0000 7f51ab7fe640 1 -- 172.21.15.27:0/3770525637 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a402c950 con 0x7f51ac0af360 2023-12-19T18:17:47.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.869+0000 7f51ab7fe640 1 -- 172.21.15.27:0/3770525637 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a402cab0 con 0x7f51ac0af360 2023-12-19T18:17:47.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.869+0000 7f51ab7fe640 1 -- 172.21.15.27:0/3770525637 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a402ce40 con 0x7f51ac0af360 2023-12-19T18:17:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.870+0000 7f51b19a7640 1 -- 172.21.15.27:0/3770525637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51ac0af360 msgr2=0x7f51ac06ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.870+0000 7f51b19a7640 1 --2- 172.21.15.27:0/3770525637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51ac0af360 0x7f51ac06ba20 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f51a4009e50 tx=0x7f51a402bdb0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.870+0000 7f51b19a7640 1 -- 172.21.15.27:0/3770525637 shutdown_connections 2023-12-19T18:17:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.870+0000 7f51b19a7640 1 --2- 172.21.15.27:0/3770525637 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51ac0af360 0x7f51ac06ba20 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.870+0000 7f51b19a7640 1 --2- 172.21.15.27:0/3770525637 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51ac15ac50 0x7f51ac070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.870+0000 7f51b19a7640 1 -- 172.21.15.27:0/3770525637 >> 172.21.15.27:0/3770525637 conn(0x7f51ac06a7d0 msgr2=0x7f51ac0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.870+0000 7f51b19a7640 1 -- 172.21.15.27:0/3770525637 shutdown_connections 2023-12-19T18:17:47.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.870+0000 7f51b19a7640 1 -- 172.21.15.27:0/3770525637 wait complete. 2023-12-19T18:17:47.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.872+0000 7f51b19a7640 1 Processor -- start 2023-12-19T18:17:47.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.872+0000 7f51b19a7640 1 -- start start 2023-12-19T18:17:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.873+0000 7f51b19a7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51ac0aef20 0x7f51ac14d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.873+0000 7f51b19a7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51ac0af360 0x7f51ac0ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.873+0000 7f51b19a7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51ac15ac50 0x7f51ac0c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.873+0000 7f51b19a7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac15cee0 con 0x7f51ac0aef20 2023-12-19T18:17:47.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.873+0000 7f51b19a7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac150220 con 0x7f51ac0af360 2023-12-19T18:17:47.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.873+0000 7f51b19a7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac14f6a0 con 0x7f51ac15ac50 2023-12-19T18:17:47.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.873+0000 7f51b09a5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51ac0aef20 0x7f51ac14d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:47.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.873+0000 7f51b09a5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51ac0aef20 0x7f51ac14d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42676/0 (socket says 172.21.15.27:42676) 2023-12-19T18:17:47.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.873+0000 7f51b09a5640 1 -- 172.21.15.27:0/1313652294 learned_addr learned my addr 172.21.15.27:0/1313652294 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:47.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.874+0000 7f51b11a6640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51ac15ac50 0x7f51ac0c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:47.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.874+0000 7f51abfff640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51ac0af360 0x7f51ac0ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:47.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.874+0000 7f51b09a5640 1 -- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51ac15ac50 msgr2=0x7f51ac0c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:47.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.874+0000 7f51b09a5640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51ac15ac50 0x7f51ac0c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.874+0000 7f51b09a5640 1 -- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51ac0af360 msgr2=0x7f51ac0ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:47.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.874+0000 7f51b09a5640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51ac0af360 0x7f51ac0ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:47.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.874+0000 7f51b09a5640 1 -- 172.21.15.27:0/1313652294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4009b30 con 0x7f51ac0aef20 2023-12-19T18:17:47.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.874+0000 7f51b11a6640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51ac15ac50 0x7f51ac0c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:47.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.874+0000 7f51b09a5640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51ac0aef20 0x7f51ac14d4c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f51a000c7f0 tx=0x7f51a000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:47.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.875+0000 7f51a97fa640 1 -- 172.21.15.27:0/1313652294 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0007820 con 0x7f51ac0aef20 2023-12-19T18:17:47.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.875+0000 7f51a97fa640 1 -- 172.21.15.27:0/1313652294 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a0007980 con 0x7f51ac0aef20 2023-12-19T18:17:47.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.875+0000 7f51a97fa640 1 -- 172.21.15.27:0/1313652294 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a0007cf0 con 0x7f51ac0aef20 2023-12-19T18:17:47.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.875+0000 7f51b19a7640 1 -- 172.21.15.27:0/1313652294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac14f980 con 0x7f51ac0aef20 2023-12-19T18:17:47.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.875+0000 7f51b19a7640 1 -- 172.21.15.27:0/1313652294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac14ff40 con 0x7f51ac0aef20 2023-12-19T18:17:47.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.876+0000 7f51b19a7640 1 -- 172.21.15.27:0/1313652294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac04e840 con 0x7f51ac0aef20 2023-12-19T18:17:47.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.877+0000 7f51a97fa640 1 -- 172.21.15.27:0/1313652294 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f51a00113f0 con 0x7f51ac0aef20 2023-12-19T18:17:47.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.878+0000 7f51a97fa640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f519006a430 0x7f519006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:47.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.878+0000 7f51a97fa640 1 -- 172.21.15.27:0/1313652294 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f51a0088ce0 con 0x7f51ac0aef20 2023-12-19T18:17:47.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.878+0000 7f51a97fa640 1 -- 172.21.15.27:0/1313652294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=59}) v3 -- 0x7f5190072140 con 0x7f51ac0aef20 2023-12-19T18:17:47.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.879+0000 7f51abfff640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f519006a430 0x7f519006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:47.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.880+0000 7f51abfff640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f519006a430 0x7f519006c8f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0b4000 tx=0x7f5198008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:47.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:47.882+0000 7f51a97fa640 1 -- 172.21.15.27:0/1313652294 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f51a001f060 con 0x7f51ac0aef20 2023-12-19T18:17:48.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.079+0000 7f51b19a7640 1 -- 172.21.15.27:0/1313652294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f51ac06f430 con 0x7f51ac0aef20 2023-12-19T18:17:48.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.080+0000 7f51a97fa640 1 -- 172.21.15.27:0/1313652294 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v58) v1 ==== 81+0+34542 (secure 0 0 0) 0x7f51a0054230 con 0x7f51ac0aef20 2023-12-19T18:17:48.081 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:48.081 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:17:48.081 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 58, 2023-12-19T18:17:48.081 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:17:48.082 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:17:48.082 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:17:47.744407+0000", 2023-12-19T18:17:48.082 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:17:48.082 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:17:48.082 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:17:48.082 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:17:48.082 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:17:48.082 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:17:48.082 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:17:48.082 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:17:48.082 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:17:48.082 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:48.082 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:17:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:17:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:17:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:17:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:17:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 7, 2023-12-19T18:17:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:17:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:17:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "luminous", 2023-12-19T18:17:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:17:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:17:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:17:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.083 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:48.084 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "52", 2023-12-19T18:17:48.085 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:48.086 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:48.087 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.088 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 6, 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:12.527441+0000", 2023-12-19T18:17:48.089 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:17:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "52", 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:48.093 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.094 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 7, 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:13.291755+0000", 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:17:48.095 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:17:48.096 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "52", 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:48.097 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:48.098 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:48.099 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.25, 2023-12-19T18:17:48.100 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.25, 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.25, 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.25, 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:48.101 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:48.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.104 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.105 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:48.106 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:48.107 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:48.107 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:48.107 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:48.107 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:48.107 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:48.107 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:48.107 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:48.107 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:48.107 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:48.107 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:48.107 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.107 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.108 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.110 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:17:48.110 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:48.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.110 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:17:48.110 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:48.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.110 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.110 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:48.110 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:17:48.111 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:17:48.112 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:17:48.112 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:48.112 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull", 2023-12-19T18:17:48.112 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:48.112 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:48.112 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.112 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:48.112 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:48.112 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:17:48.113 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.115 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:17:48.115 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:48.115 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.115 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.115 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:48.115 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.115 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:17:48.115 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:48.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.117 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:17:48.117 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:48.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.117 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.117 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:17:48.117 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:17:48.117 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:17:48.117 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:17:48.117 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:48.117 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:48.117 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:48.118 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.119 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.122 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:48.123 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.124 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.125 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.126 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.127 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.128 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.129 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:17:48.130 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:48.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:48.133 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:48.134 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.135 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.136 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:17:48.137 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:48.138 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:48.139 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.140 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.141 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:48.142 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:17:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:17:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:17:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:17:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:17:48.143 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:48.144 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:48.145 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:17:48.145 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:48.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.145 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.145 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:48.145 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:48.145 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:48.145 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:48.145 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:48.145 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:48.145 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:17:48.145 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:48.145 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:48.146 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:17:48.147 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:48.148 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:48.149 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [ 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.5", 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.150 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.6", 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.8", 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:48.151 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.9", 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.20", 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:48.152 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 0 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:17:48.153 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:17:48.154 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:17:48.155 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:17:48.155 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:17:48.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:48.155 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:17:48.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.083+0000 7f51b19a7640 1 -- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f519006a430 msgr2=0x7f519006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:48.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.083+0000 7f51b19a7640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f519006a430 0x7f519006c8f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f51ac0b4000 tx=0x7f5198008040 comp rx=0 tx=0).stop 2023-12-19T18:17:48.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.083+0000 7f51b19a7640 1 -- 172.21.15.27:0/1313652294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51ac0aef20 msgr2=0x7f51ac14d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:48.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.083+0000 7f51b19a7640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51ac0aef20 0x7f51ac14d4c0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f51a000c7f0 tx=0x7f51a000ccb0 comp rx=0 tx=0).stop 2023-12-19T18:17:48.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.083+0000 7f51b19a7640 1 -- 172.21.15.27:0/1313652294 shutdown_connections 2023-12-19T18:17:48.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.083+0000 7f51b19a7640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f519006a430 0x7f519006c8f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:48.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.083+0000 7f51b19a7640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51ac15ac50 0x7f51ac0c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:48.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.083+0000 7f51b19a7640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51ac0af360 0x7f51ac0ca7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:48.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.083+0000 7f51b19a7640 1 --2- 172.21.15.27:0/1313652294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51ac0aef20 0x7f51ac14d4c0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:48.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.083+0000 7f51b19a7640 1 -- 172.21.15.27:0/1313652294 >> 172.21.15.27:0/1313652294 conn(0x7f51ac06a7d0 msgr2=0x7f51ac0ccd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:48.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.083+0000 7f51b19a7640 1 -- 172.21.15.27:0/1313652294 shutdown_connections 2023-12-19T18:17:48.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:48.083+0000 7f51b19a7640 1 -- 172.21.15.27:0/1313652294 wait complete. 2023-12-19T18:17:48.158 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-12-19T18:17:49.774 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:17:49.773+0000 7f8f3ae61640 -1 log_channel(cluster) log [ERR] : Health check failed: Full OSDs blocking recovery: 1 pg recovery_toofull (PG_RECOVERY_FULL) 2023-12-19T18:17:53.159 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:17:53.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.406+0000 7fa311270640 1 Processor -- start 2023-12-19T18:17:53.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.406+0000 7fa311270640 1 -- start start 2023-12-19T18:17:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.408+0000 7fa311270640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa30c15ab90 0x7fa30c06b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.408+0000 7fa311270640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa30c15af70 0x7fa30c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.408+0000 7fa311270640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa30c0ad7a0 0x7fa30c0cbad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.408+0000 7fa311270640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30c0ce870 con 0x7fa30c15ab90 2023-12-19T18:17:53.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.408+0000 7fa311270640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30c0ce3e0 con 0x7fa30c0ad7a0 2023-12-19T18:17:53.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.408+0000 7fa311270640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c15f7b0 con 0x7fa30c15af70 2023-12-19T18:17:53.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.408+0000 7fa30affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa30c15ab90 0x7fa30c06b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.409+0000 7fa30affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa30c15ab90 0x7fa30c06b500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47290/0 (socket says 172.21.15.27:47290) 2023-12-19T18:17:53.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.409+0000 7fa30affd640 1 -- 172.21.15.27:0/1365267456 learned_addr learned my addr 172.21.15.27:0/1365267456 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:53.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.409+0000 7fa30a7fc640 1 --2- 172.21.15.27:0/1365267456 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa30c15af70 0x7fa30c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.409+0000 7fa30b7fe640 1 --2- 172.21.15.27:0/1365267456 >> v2:172.21.15.192:3300/0 conn(0x7fa30c0ad7a0 0x7fa30c0cbad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.409+0000 7fa30affd640 1 -- 172.21.15.27:0/1365267456 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa30c15af70 msgr2=0x7fa30c0cb590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:53.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.409+0000 7fa30affd640 1 --2- 172.21.15.27:0/1365267456 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa30c15af70 0x7fa30c0cb590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.409+0000 7fa30affd640 1 -- 172.21.15.27:0/1365267456 >> v2:172.21.15.192:3300/0 conn(0x7fa30c0ad7a0 msgr2=0x7fa30c0cbad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:53.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.409+0000 7fa30affd640 1 --2- 172.21.15.27:0/1365267456 >> v2:172.21.15.192:3300/0 conn(0x7fa30c0ad7a0 0x7fa30c0cbad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.409+0000 7fa30affd640 1 -- 172.21.15.27:0/1365267456 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c0c9150 con 0x7fa30c15ab90 2023-12-19T18:17:53.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.411+0000 7fa30affd640 1 --2- 172.21.15.27:0/1365267456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa30c15ab90 0x7fa30c06b500 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0ade10 tx=0x7fa2f402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4548d2824ac6ad3e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:53.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.411+0000 7fa309ffb640 1 -- 172.21.15.27:0/1365267456 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4031640 con 0x7fa30c15ab90 2023-12-19T18:17:53.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.411+0000 7fa309ffb640 1 -- 172.21.15.27:0/1365267456 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f40317a0 con 0x7fa30c15ab90 2023-12-19T18:17:53.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.411+0000 7fa309ffb640 1 -- 172.21.15.27:0/1365267456 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f4031b30 con 0x7fa30c15ab90 2023-12-19T18:17:53.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.412+0000 7fa311270640 1 -- 172.21.15.27:0/1365267456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa30c15ab90 msgr2=0x7fa30c06b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.412+0000 7fa311270640 1 --2- 172.21.15.27:0/1365267456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa30c15ab90 0x7fa30c06b500 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0ade10 tx=0x7fa2f402b8f0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.412+0000 7fa311270640 1 -- 172.21.15.27:0/1365267456 shutdown_connections 2023-12-19T18:17:53.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.412+0000 7fa311270640 1 --2- 172.21.15.27:0/1365267456 >> v2:172.21.15.192:3300/0 conn(0x7fa30c0ad7a0 0x7fa30c0cbad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.412+0000 7fa311270640 1 --2- 172.21.15.27:0/1365267456 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa30c15af70 0x7fa30c0cb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.412+0000 7fa311270640 1 --2- 172.21.15.27:0/1365267456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa30c15ab90 0x7fa30c06b500 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.412+0000 7fa311270640 1 -- 172.21.15.27:0/1365267456 >> 172.21.15.27:0/1365267456 conn(0x7fa30c06a2a0 msgr2=0x7fa30c0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:53.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.412+0000 7fa311270640 1 -- 172.21.15.27:0/1365267456 shutdown_connections 2023-12-19T18:17:53.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.412+0000 7fa311270640 1 -- 172.21.15.27:0/1365267456 wait complete. 2023-12-19T18:17:53.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.414+0000 7fa311270640 1 Processor -- start 2023-12-19T18:17:53.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.414+0000 7fa311270640 1 -- start start 2023-12-19T18:17:53.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.415+0000 7fa311270640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa30c0ad7a0 0x7fa30c0b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.415+0000 7fa311270640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa30c15ab90 0x7fa30c070310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.415+0000 7fa311270640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa30c15af70 0x7fa30c1f3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.415+0000 7fa311270640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0b4120 con 0x7fa30c15ab90 2023-12-19T18:17:53.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.415+0000 7fa311270640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c070a90 con 0x7fa30c15af70 2023-12-19T18:17:53.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.415+0000 7fa311270640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c070850 con 0x7fa30c0ad7a0 2023-12-19T18:17:53.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.415+0000 7fa30a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa30c15ab90 0x7fa30c070310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.416+0000 7fa30a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa30c15ab90 0x7fa30c070310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47296/0 (socket says 172.21.15.27:47296) 2023-12-19T18:17:53.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.416+0000 7fa30a7fc640 1 -- 172.21.15.27:0/2713471401 learned_addr learned my addr 172.21.15.27:0/2713471401 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:53.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.416+0000 7fa30affd640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa30c0ad7a0 0x7fa30c0b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.416+0000 7fa30a7fc640 1 -- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa30c0ad7a0 msgr2=0x7fa30c0b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.416+0000 7fa30a7fc640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa30c0ad7a0 0x7fa30c0b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.416+0000 7fa30a7fc640 1 -- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa30c15af70 msgr2=0x7fa30c1f3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.416+0000 7fa30a7fc640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa30c15af70 0x7fa30c1f3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.416+0000 7fa30a7fc640 1 -- 172.21.15.27:0/2713471401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f4009470 con 0x7fa30c15ab90 2023-12-19T18:17:53.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.416+0000 7fa30affd640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa30c0ad7a0 0x7fa30c0b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:53.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.416+0000 7fa30a7fc640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa30c15ab90 0x7fa30c070310 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00d5c0 tx=0x7fa2fc00da80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:53.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.417+0000 7fa3037fe640 1 -- 172.21.15.27:0/2713471401 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc0125f0 con 0x7fa30c15ab90 2023-12-19T18:17:53.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.417+0000 7fa3037fe640 1 -- 172.21.15.27:0/2713471401 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc012750 con 0x7fa30c15ab90 2023-12-19T18:17:53.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.417+0000 7fa3037fe640 1 -- 172.21.15.27:0/2713471401 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc012b00 con 0x7fa30c15ab90 2023-12-19T18:17:53.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.417+0000 7fa311270640 1 -- 172.21.15.27:0/2713471401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30c0b17b0 con 0x7fa30c15ab90 2023-12-19T18:17:53.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.417+0000 7fa311270640 1 -- 172.21.15.27:0/2713471401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30c0b1d70 con 0x7fa30c15ab90 2023-12-19T18:17:53.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.418+0000 7fa311270640 1 -- 172.21.15.27:0/2713471401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30c068e90 con 0x7fa30c15ab90 2023-12-19T18:17:53.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.419+0000 7fa3037fe640 1 -- 172.21.15.27:0/2713471401 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa2fc012c60 con 0x7fa30c15ab90 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.420+0000 7fa3037fe640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa2ec06a430 0x7fa2ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.420+0000 7fa3037fe640 1 -- 172.21.15.27:0/2713471401 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa2fc088da0 con 0x7fa30c15ab90 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.420+0000 7fa3037fe640 1 -- 172.21.15.27:0/2713471401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=60}) v3 -- 0x7fa2ec072140 con 0x7fa30c15ab90 2023-12-19T18:17:53.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.421+0000 7fa30affd640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa2ec06a430 0x7fa2ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:53.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.422+0000 7fa30affd640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa2ec06a430 0x7fa2ec06c8f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4036000 tx=0x7fa2f403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:53.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.425+0000 7fa3037fe640 1 -- 172.21.15.27:0/2713471401 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa2fc054260 con 0x7fa30c15ab90 2023-12-19T18:17:53.574 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:17:53.574+0000 7f8f3ae61640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2023-12-19T18:17:53.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.585+0000 7fa3037fe640 1 -- 172.21.15.27:0/2713471401 <== mon.0 v2:172.21.15.27:3300/0 7 ==== osd_map(60..60 src has 1..60) v4 ==== 296+0+0 (secure 0 0 0) 0x7fa2fc058620 con 0x7fa30c15ab90 2023-12-19T18:17:53.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.585+0000 7fa3037fe640 1 -- 172.21.15.27:0/2713471401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=61}) v3 -- 0x7fa2ec0728a0 con 0x7fa30c15ab90 2023-12-19T18:17:53.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.624+0000 7fa311270640 1 -- 172.21.15.27:0/2713471401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fa30c0b1160 con 0x7fa30c15ab90 2023-12-19T18:17:53.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.628+0000 7fa3037fe640 1 -- 172.21.15.27:0/2713471401 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v60) v1 ==== 81+0+34486 (secure 0 0 0) 0x7fa2fc057880 con 0x7fa30c15ab90 2023-12-19T18:17:53.629 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:53.629 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:17:53.629 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 60, 2023-12-19T18:17:53.629 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:17:53.629 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:17:53.629 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:17:53.574917+0000", 2023-12-19T18:17:53.629 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:17:53.629 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:17:53.629 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:17:53.629 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:17:53.629 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:17:53.629 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:17:53.629 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 7, 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "luminous", 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:17:53.630 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:53.631 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:53.632 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:53.632 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:17:53.632 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:17:53.632 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:17:53.632 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:17:53.632 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:17:53.632 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:53.632 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:53.632 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "52", 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:53.633 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:53.634 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:53.635 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:53.636 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 6, 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:12.527441+0000", 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:53.637 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:53.638 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "52", 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:53.639 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:53.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:17:53.641 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:17:53.642 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 7, 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:13.291755+0000", 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:53.643 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:53.644 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:53.645 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:53.645 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:53.645 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:53.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.645 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "52", 2023-12-19T18:17:53.645 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:53.645 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:53.645 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:53.645 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:53.645 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:53.645 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:53.645 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:53.645 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:53.646 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:53.647 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.25, 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.25, 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.25, 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.25, 2023-12-19T18:17:53.648 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:53.649 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:17:53.650 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.651 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.652 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:53.654 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.656 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.657 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:17:53.658 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:53.659 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.660 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:17:53.661 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:53.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.661 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:17:53.661 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:53.661 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.661 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.661 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:53.661 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.661 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.662 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:17:53.663 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:53.664 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:53.665 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.667 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:53.668 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:17:53.669 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:53.670 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:17:53.671 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:17:53.672 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:53.673 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.674 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:53.675 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.677 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.678 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.679 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:53.681 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.682 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:53.683 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.684 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:17:53.685 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:53.686 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.687 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:17:53.688 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:17:53.689 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:53.690 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.691 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:53.692 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.693 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:53.694 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.695 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:53.696 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [ 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.5", 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.697 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.6", 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.8", 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:53.698 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.9", 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.20", 2023-12-19T18:17:53.699 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 0 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:17:53.700 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:17:53.701 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:17:53.702 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:17:53.702 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:17:53.702 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:17:53.702 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:53.702 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:17:53.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.631+0000 7fa311270640 1 -- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa2ec06a430 msgr2=0x7fa2ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.631+0000 7fa311270640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa2ec06a430 0x7fa2ec06c8f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fa2f4036000 tx=0x7fa2f403c040 comp rx=0 tx=0).stop 2023-12-19T18:17:53.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.631+0000 7fa311270640 1 -- 172.21.15.27:0/2713471401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa30c15ab90 msgr2=0x7fa30c070310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:53.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.631+0000 7fa311270640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa30c15ab90 0x7fa30c070310 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc00d5c0 tx=0x7fa2fc00da80 comp rx=0 tx=0).stop 2023-12-19T18:17:53.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.631+0000 7fa311270640 1 -- 172.21.15.27:0/2713471401 shutdown_connections 2023-12-19T18:17:53.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.631+0000 7fa311270640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa2ec06a430 0x7fa2ec06c8f0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.631+0000 7fa311270640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa30c15af70 0x7fa30c1f3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.631+0000 7fa311270640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa30c15ab90 0x7fa30c070310 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.631+0000 7fa311270640 1 --2- 172.21.15.27:0/2713471401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa30c0ad7a0 0x7fa30c0b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:53.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.631+0000 7fa311270640 1 -- 172.21.15.27:0/2713471401 >> 172.21.15.27:0/2713471401 conn(0x7fa30c06a2a0 msgr2=0x7fa30c06be00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:53.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.631+0000 7fa311270640 1 -- 172.21.15.27:0/2713471401 shutdown_connections 2023-12-19T18:17:53.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:53.631+0000 7fa311270640 1 -- 172.21.15.27:0/2713471401 wait complete. 2023-12-19T18:17:53.705 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-12-19T18:17:58.706 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:17:58.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.951+0000 7f25c37d9640 1 Processor -- start 2023-12-19T18:17:58.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.952+0000 7f25c37d9640 1 -- start start 2023-12-19T18:17:58.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.953+0000 7f25c37d9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f25bc06cec0 0x7f25bc06d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.953+0000 7f25c37d9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f25bc0c9770 0x7f25bc0cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.953+0000 7f25c37d9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25bc0cd990 0x7f25bc0cde20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.953+0000 7f25c37d9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25bc15a330 con 0x7f25bc0c9770 2023-12-19T18:17:58.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.953+0000 7f25c37d9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25bc0cc720 con 0x7f25bc06cec0 2023-12-19T18:17:58.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.953+0000 7f25c37d9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0b1a00 con 0x7f25bc0cd990 2023-12-19T18:17:58.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.954+0000 7f25c0d74640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f25bc0c9770 0x7f25bc0cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.954+0000 7f25c0d74640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f25bc0c9770 0x7f25bc0cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47312/0 (socket says 172.21.15.27:47312) 2023-12-19T18:17:58.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.954+0000 7f25c0d74640 1 -- 172.21.15.27:0/3692473631 learned_addr learned my addr 172.21.15.27:0/3692473631 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:58.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.954+0000 7f25c1575640 1 --2- 172.21.15.27:0/3692473631 >> v2:172.21.15.192:3300/0 conn(0x7f25bc06cec0 0x7f25bc06d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.954+0000 7f25c1d76640 1 --2- 172.21.15.27:0/3692473631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25bc0cd990 0x7f25bc0cde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.955+0000 7f25c0d74640 1 -- 172.21.15.27:0/3692473631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25bc0cd990 msgr2=0x7f25bc0cde20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:58.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.955+0000 7f25c0d74640 1 --2- 172.21.15.27:0/3692473631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25bc0cd990 0x7f25bc0cde20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.955+0000 7f25c0d74640 1 -- 172.21.15.27:0/3692473631 >> v2:172.21.15.192:3300/0 conn(0x7f25bc06cec0 msgr2=0x7f25bc06d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:58.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.955+0000 7f25c0d74640 1 --2- 172.21.15.27:0/3692473631 >> v2:172.21.15.192:3300/0 conn(0x7f25bc06cec0 0x7f25bc06d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.955+0000 7f25c0d74640 1 -- 172.21.15.27:0/3692473631 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc06d7e0 con 0x7f25bc0c9770 2023-12-19T18:17:58.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.956+0000 7f25c0d74640 1 --2- 172.21.15.27:0/3692473631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0c9770 0x7f25bc0cd450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f25bc066860 tx=0x7f25ac02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d0a06f4467d939 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:58.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.956+0000 7f25b3fff640 1 -- 172.21.15.27:0/3692473631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac031870 con 0x7f25bc0c9770 2023-12-19T18:17:58.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.956+0000 7f25b3fff640 1 -- 172.21.15.27:0/3692473631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25ac0319d0 con 0x7f25bc0c9770 2023-12-19T18:17:58.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.956+0000 7f25b3fff640 1 -- 172.21.15.27:0/3692473631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac031d60 con 0x7f25bc0c9770 2023-12-19T18:17:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.957+0000 7f25c37d9640 1 -- 172.21.15.27:0/3692473631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0c9770 msgr2=0x7f25bc0cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.957+0000 7f25c37d9640 1 --2- 172.21.15.27:0/3692473631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0c9770 0x7f25bc0cd450 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f25bc066860 tx=0x7f25ac02b910 comp rx=0 tx=0).stop 2023-12-19T18:17:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.957+0000 7f25c37d9640 1 -- 172.21.15.27:0/3692473631 shutdown_connections 2023-12-19T18:17:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.957+0000 7f25c37d9640 1 --2- 172.21.15.27:0/3692473631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25bc0cd990 0x7f25bc0cde20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.957+0000 7f25c37d9640 1 --2- 172.21.15.27:0/3692473631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0c9770 0x7f25bc0cd450 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.957+0000 7f25c37d9640 1 --2- 172.21.15.27:0/3692473631 >> v2:172.21.15.192:3300/0 conn(0x7f25bc06cec0 0x7f25bc06d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.957+0000 7f25c37d9640 1 -- 172.21.15.27:0/3692473631 >> 172.21.15.27:0/3692473631 conn(0x7f25bc0ceb50 msgr2=0x7f25bc0cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.957+0000 7f25c37d9640 1 -- 172.21.15.27:0/3692473631 shutdown_connections 2023-12-19T18:17:58.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.957+0000 7f25c37d9640 1 -- 172.21.15.27:0/3692473631 wait complete. 2023-12-19T18:17:58.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.959+0000 7f25c37d9640 1 Processor -- start 2023-12-19T18:17:58.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.959+0000 7f25c37d9640 1 -- start start 2023-12-19T18:17:58.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.960+0000 7f25c37d9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc06cec0 0x7f25bc0c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.960+0000 7f25c37d9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25bc0c9770 0x7f25bc150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.960+0000 7f25c37d9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0cd990 0x7f25bc1448a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.960+0000 7f25c37d9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc0c20e0 con 0x7f25bc0cd990 2023-12-19T18:17:58.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.960+0000 7f25c37d9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc14fba0 con 0x7f25bc06cec0 2023-12-19T18:17:58.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.960+0000 7f25c37d9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc14ef90 con 0x7f25bc0c9770 2023-12-19T18:17:58.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.961+0000 7f25c1d76640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0cd990 0x7f25bc1448a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.961+0000 7f25c1d76640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0cd990 0x7f25bc1448a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47328/0 (socket says 172.21.15.27:47328) 2023-12-19T18:17:58.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.961+0000 7f25c1d76640 1 -- 172.21.15.27:0/771337428 learned_addr learned my addr 172.21.15.27:0/771337428 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:58.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.961+0000 7f25c1575640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc06cec0 0x7f25bc0c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.961+0000 7f25c1d76640 1 -- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25bc0c9770 msgr2=0x7f25bc150030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.961+0000 7f25c0d74640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25bc0c9770 0x7f25bc150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.961+0000 7f25c1d76640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25bc0c9770 0x7f25bc150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.961+0000 7f25c1d76640 1 -- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc06cec0 msgr2=0x7f25bc0c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:58.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.961+0000 7f25c1d76640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc06cec0 0x7f25bc0c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:58.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.961+0000 7f25c1d76640 1 -- 172.21.15.27:0/771337428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac0094d0 con 0x7f25bc0cd990 2023-12-19T18:17:58.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.961+0000 7f25c1575640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc06cec0 0x7f25bc0c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:17:58.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.961+0000 7f25c1d76640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0cd990 0x7f25bc1448a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f25b400d600 tx=0x7f25b400dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:58.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.962+0000 7f25b1ffb640 1 -- 172.21.15.27:0/771337428 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b40125f0 con 0x7f25bc0cd990 2023-12-19T18:17:58.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.962+0000 7f25b1ffb640 1 -- 172.21.15.27:0/771337428 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b4012770 con 0x7f25bc0cd990 2023-12-19T18:17:58.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.962+0000 7f25b1ffb640 1 -- 172.21.15.27:0/771337428 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b4012b20 con 0x7f25bc0cd990 2023-12-19T18:17:58.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.962+0000 7f25c37d9640 1 -- 172.21.15.27:0/771337428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc14f270 con 0x7f25bc0cd990 2023-12-19T18:17:58.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.962+0000 7f25c37d9640 1 -- 172.21.15.27:0/771337428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc14f830 con 0x7f25bc0cd990 2023-12-19T18:17:58.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.963+0000 7f25c37d9640 1 -- 172.21.15.27:0/771337428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25bc0ccc30 con 0x7f25bc0cd990 2023-12-19T18:17:58.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.964+0000 7f25b1ffb640 1 -- 172.21.15.27:0/771337428 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f25b4012c80 con 0x7f25bc0cd990 2023-12-19T18:17:58.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.965+0000 7f25b1ffb640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f25a006a430 0x7f25a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:58.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.965+0000 7f25b1ffb640 1 -- 172.21.15.27:0/771337428 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f25b4088e40 con 0x7f25bc0cd990 2023-12-19T18:17:58.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.966+0000 7f25c1575640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f25a006a430 0x7f25a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:58.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.967+0000 7f25c1575640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f25a006a430 0x7f25a006c8f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f25bc14b880 tx=0x7f25b800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:58.970+0000 7f25b1ffb640 1 -- 172.21.15.27:0/771337428 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f25bc0ccc30 con 0x7f25bc0cd990 2023-12-19T18:17:59.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.168+0000 7f25c37d9640 1 -- 172.21.15.27:0/771337428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f25bc0ccc00 con 0x7f25bc0cd990 2023-12-19T18:17:59.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.172+0000 7f25b1ffb640 1 -- 172.21.15.27:0/771337428 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v63) v1 ==== 81+0+34399 (secure 0 0 0) 0x7f25b4054310 con 0x7f25bc0cd990 2023-12-19T18:17:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:59.173 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:17:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 63, 2023-12-19T18:17:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:17:56.605695+0000", 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:17:59.174 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 7, 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "luminous", 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:17:59.175 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:17:59.176 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "63", 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:59.177 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:59.178 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:59.179 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:17:59.180 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 6, 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:12.527441+0000", 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:17:59.181 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:17:59.182 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "63", 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:59.183 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:59.184 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:59.185 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:17:59.186 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:17:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:17:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:17:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 7, 2023-12-19T18:17:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:17:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:13.291755+0000", 2023-12-19T18:17:59.187 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:17:59.188 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "63", 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:59.189 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:59.190 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:59.191 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.25, 2023-12-19T18:17:59.192 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.25, 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.25, 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.25, 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.193 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:17:59.194 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.197 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:59.198 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.199 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.200 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.201 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.204 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:59.204 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.204 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.204 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.204 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.204 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.204 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.204 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.204 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.204 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:59.204 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.205 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.205 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.205 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.205 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:17:59.205 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.205 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.205 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:17:59.205 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.205 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.206 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.209 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.211 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.212 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.213 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.214 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.215 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.216 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:17:59.217 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.218 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.219 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.220 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.221 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.222 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:17:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:17:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.224 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:59.225 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.226 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:17:59.229 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.230 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:17:59.231 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.232 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.232 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.232 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.232 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.232 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:17:59.232 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.232 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.233 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:17:59.234 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:17:59.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.236 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.238 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.239 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.240 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [ 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.5", 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.6", 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.8", 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:59.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.9", 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.20", 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:59.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 0 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:17:59.244 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:17:59.245 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:17:59.246 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:17:59.246 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:17:59.246 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.246 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:17:59.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.174+0000 7f25c37d9640 1 -- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f25a006a430 msgr2=0x7f25a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.174+0000 7f25c37d9640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f25a006a430 0x7f25a006c8f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f25bc14b880 tx=0x7f25b800b040 comp rx=0 tx=0).stop 2023-12-19T18:17:59.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.174+0000 7f25c37d9640 1 -- 172.21.15.27:0/771337428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0cd990 msgr2=0x7f25bc1448a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.174+0000 7f25c37d9640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0cd990 0x7f25bc1448a0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f25b400d600 tx=0x7f25b400dac0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.175+0000 7f25c37d9640 1 -- 172.21.15.27:0/771337428 shutdown_connections 2023-12-19T18:17:59.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.175+0000 7f25c37d9640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f25a006a430 0x7f25a006c8f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.175+0000 7f25c37d9640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0cd990 0x7f25bc1448a0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.175+0000 7f25c37d9640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f25bc0c9770 0x7f25bc150030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.175+0000 7f25c37d9640 1 --2- 172.21.15.27:0/771337428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f25bc06cec0 0x7f25bc0c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.175+0000 7f25c37d9640 1 -- 172.21.15.27:0/771337428 >> 172.21.15.27:0/771337428 conn(0x7f25bc0ceb50 msgr2=0x7f25bc147090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:59.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.175+0000 7f25c37d9640 1 -- 172.21.15.27:0/771337428 shutdown_connections 2023-12-19T18:17:59.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.175+0000 7f25c37d9640 1 -- 172.21.15.27:0/771337428 wait complete. 2023-12-19T18:17:59.249 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-12-19T18:17:59.249 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:17:59.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.492+0000 7f0b4ea4d640 1 Processor -- start 2023-12-19T18:17:59.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.492+0000 7f0b4ea4d640 1 -- start start 2023-12-19T18:17:59.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.494+0000 7f0b4ea4d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b4815ac50 0x7f0b48070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.494+0000 7f0b4ea4d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0b480aef20 0x7f0b4806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.494+0000 7f0b4ea4d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b480adf10 con 0x7f0b480af360 2023-12-19T18:17:59.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.494+0000 7f0b4ea4d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b480adac0 con 0x7f0b480aef20 2023-12-19T18:17:59.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.494+0000 7f0b4ea4d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b480ae160 con 0x7f0b4815ac50 2023-12-19T18:17:59.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.494+0000 7f0b4da4b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b4815ac50 0x7f0b48070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.494+0000 7f0b4d24a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0b480aef20 0x7f0b4806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.494+0000 7f0b4d24a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0b480aef20 0x7f0b4806b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:49916/0 (socket says 172.21.15.27:49916) 2023-12-19T18:17:59.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.494+0000 7f0b4d24a640 1 -- 172.21.15.27:0/1717804441 learned_addr learned my addr 172.21.15.27:0/1717804441 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:59.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.495+0000 7f0b4d24a640 1 -- 172.21.15.27:0/1717804441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b4815ac50 msgr2=0x7f0b48070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:17:59.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.495+0000 7f0b4d24a640 1 --2- 172.21.15.27:0/1717804441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b4815ac50 0x7f0b48070fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.495+0000 7f0b4d24a640 1 -- 172.21.15.27:0/1717804441 >> v1:172.21.15.192:6789/0 conn(0x7f0b480af360 legacy=0x7f0b4806b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.495+0000 7f0b4d24a640 1 -- 172.21.15.27:0/1717804441 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b480ae440 con 0x7f0b480aef20 2023-12-19T18:17:59.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.497+0000 7f0b4d24a640 1 --2- 172.21.15.27:0/1717804441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b480aef20 0x7f0b4806b310 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009870 tx=0x7f0b4402b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=981502303633db10 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.497+0000 7f0b4ca49640 1 -- 172.21.15.27:0/1717804441 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b440317c0 con 0x7f0b480aef20 2023-12-19T18:17:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.497+0000 7f0b4ca49640 1 -- 172.21.15.27:0/1717804441 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b44031920 con 0x7f0b480aef20 2023-12-19T18:17:59.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.497+0000 7f0b4ca49640 1 -- 172.21.15.27:0/1717804441 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b44031cb0 con 0x7f0b480aef20 2023-12-19T18:17:59.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.498+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/1717804441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b480aef20 msgr2=0x7f0b4806b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.498+0000 7f0b4ea4d640 1 --2- 172.21.15.27:0/1717804441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b480aef20 0x7f0b4806b310 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b44009870 tx=0x7f0b4402b860 comp rx=0 tx=0).stop 2023-12-19T18:17:59.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.498+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/1717804441 shutdown_connections 2023-12-19T18:17:59.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.498+0000 7f0b4ea4d640 1 --2- 172.21.15.27:0/1717804441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b480aef20 0x7f0b4806b310 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.498+0000 7f0b4ea4d640 1 --2- 172.21.15.27:0/1717804441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b4815ac50 0x7f0b48070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.498+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/1717804441 >> 172.21.15.27:0/1717804441 conn(0x7f0b4806a7d0 msgr2=0x7f0b480ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:59.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.498+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/1717804441 shutdown_connections 2023-12-19T18:17:59.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.498+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/1717804441 wait complete. 2023-12-19T18:17:59.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.500+0000 7f0b4ea4d640 1 Processor -- start 2023-12-19T18:17:59.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.500+0000 7f0b4ea4d640 1 -- start start 2023-12-19T18:17:59.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.501+0000 7f0b4ea4d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b480aef20 0x7f0b48155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.501+0000 7f0b4ea4d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b480af360 0x7f0b481562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.501+0000 7f0b4ea4d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b4815ac50 0x7f0b4813ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.501+0000 7f0b4ea4d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b48157700 con 0x7f0b480aef20 2023-12-19T18:17:59.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.501+0000 7f0b4ea4d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b4813dc30 con 0x7f0b480af360 2023-12-19T18:17:59.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.501+0000 7f0b4ea4d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b48156af0 con 0x7f0b4815ac50 2023-12-19T18:17:59.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.501+0000 7f0b4da4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b480aef20 0x7f0b48155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.501+0000 7f0b4da4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b480aef20 0x7f0b48155d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47330/0 (socket says 172.21.15.27:47330) 2023-12-19T18:17:59.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.502+0000 7f0b4da4b640 1 -- 172.21.15.27:0/4294452448 learned_addr learned my addr 172.21.15.27:0/4294452448 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:17:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.502+0000 7f0b4da4b640 1 -- 172.21.15.27:0/4294452448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b4815ac50 msgr2=0x7f0b4813ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.502+0000 7f0b4da4b640 1 --2- 172.21.15.27:0/4294452448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b4815ac50 0x7f0b4813ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.502+0000 7f0b4da4b640 1 -- 172.21.15.27:0/4294452448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b480af360 msgr2=0x7f0b481562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.502+0000 7f0b4da4b640 1 --2- 172.21.15.27:0/4294452448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b480af360 0x7f0b481562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.502+0000 7f0b4da4b640 1 -- 172.21.15.27:0/4294452448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b440094d0 con 0x7f0b480aef20 2023-12-19T18:17:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.502+0000 7f0b4da4b640 1 --2- 172.21.15.27:0/4294452448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b480aef20 0x7f0b48155d80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00e9b0 tx=0x7f0b3c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.502+0000 7f0b3a7fc640 1 -- 172.21.15.27:0/4294452448 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c00c9a0 con 0x7f0b480aef20 2023-12-19T18:17:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.503+0000 7f0b3a7fc640 1 -- 172.21.15.27:0/4294452448 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b3c00cb00 con 0x7f0b480aef20 2023-12-19T18:17:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.503+0000 7f0b3a7fc640 1 -- 172.21.15.27:0/4294452448 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b3c013470 con 0x7f0b480aef20 2023-12-19T18:17:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.503+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/4294452448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b48156dd0 con 0x7f0b480aef20 2023-12-19T18:17:59.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.503+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/4294452448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b48157390 con 0x7f0b480aef20 2023-12-19T18:17:59.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.504+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/4294452448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b4804e840 con 0x7f0b480aef20 2023-12-19T18:17:59.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.505+0000 7f0b3a7fc640 1 -- 172.21.15.27:0/4294452448 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0b3c0135d0 con 0x7f0b480aef20 2023-12-19T18:17:59.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.506+0000 7f0b3a7fc640 1 --2- 172.21.15.27:0/4294452448 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0b2c06a430 0x7f0b2c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:17:59.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.506+0000 7f0b3a7fc640 1 -- 172.21.15.27:0/4294452448 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0b3c088c70 con 0x7f0b480aef20 2023-12-19T18:17:59.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.507+0000 7f0b4d24a640 1 --2- 172.21.15.27:0/4294452448 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0b2c06a430 0x7f0b2c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:17:59.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.508+0000 7f0b4d24a640 1 --2- 172.21.15.27:0/4294452448 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0b2c06a430 0x7f0b2c06c8f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0b44036000 tx=0x7f0b44030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:17:59.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.510+0000 7f0b3a7fc640 1 -- 172.21.15.27:0/4294452448 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0b3c0541c0 con 0x7f0b480aef20 2023-12-19T18:17:59.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.708+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/4294452448 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f0b4806f430 con 0x7f0b480aef20 2023-12-19T18:17:59.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.712+0000 7f0b3a7fc640 1 -- 172.21.15.27:0/4294452448 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v63) v1 ==== 81+0+34399 (secure 0 0 0) 0x7f0b3c017090 con 0x7f0b480aef20 2023-12-19T18:17:59.712 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:17:59.713 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:17:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 63, 2023-12-19T18:17:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:17:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:17:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:17:56.605695+0000", 2023-12-19T18:17:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:17:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:17:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:17:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:17:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:17:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:17:59.713 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 7, 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "luminous", 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:17:59.714 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:17:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:17:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:17:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:17:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:17:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:17:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:59.715 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:59.716 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "63", 2023-12-19T18:17:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:59.717 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:59.718 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:59.719 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.720 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 6, 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:12.527441+0000", 2023-12-19T18:17:59.721 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:17:59.722 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:17:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:17:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "63", 2023-12-19T18:17:59.723 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:59.724 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:59.725 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:17:59.726 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:59.727 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 7, 2023-12-19T18:17:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:17:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:17:13.291755+0000", 2023-12-19T18:17:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:17:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:17:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:17:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:17:59.728 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:17:59.729 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:17:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:17:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:17:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:17:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:17:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:17:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "63", 2023-12-19T18:17:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:17:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:17:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:17:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:17:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:17:59.730 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:17:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:17:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:17:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:17:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:17:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:17:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:17:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:17:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:17:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:17:59.731 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:17:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:17:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:17:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:17:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:17:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:17:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:17:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:17:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:17:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:17:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:17:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:17:59.732 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.733 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.25, 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.25, 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.25, 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.25, 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.734 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.735 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.736 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.737 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:17:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:17:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.738 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.739 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.740 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.741 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.742 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.743 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.744 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.745 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.746 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.747 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.748 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:17:59.749 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.750 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:59.751 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.752 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.753 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.754 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:17:59.755 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.756 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.757 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:17:59.758 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:17:59.759 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:17:59.760 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:59.761 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:17:59.762 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:17:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.763 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:17:59.764 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.765 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.766 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 48, 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.767 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.768 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:17:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:17:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.769 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.770 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:17:59.771 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:17:59.772 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 54, 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:17:59.773 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.774 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.776 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.777 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.778 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:17:59.779 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.780 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.781 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:17:59.782 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [ 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.5", 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:59.783 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.6", 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.8", 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:59.784 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.9", 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 7 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.785 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "7.20", 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 0 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:17:59.786 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:17:59.787 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:17:59.788 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:17:59.788 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:17:59.788 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:17:59.788 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:17:59.788 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:17:59.788 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:17:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.714+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/4294452448 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0b2c06a430 msgr2=0x7f0b2c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.714+0000 7f0b4ea4d640 1 --2- 172.21.15.27:0/4294452448 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0b2c06a430 0x7f0b2c06c8f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0b44036000 tx=0x7f0b44030000 comp rx=0 tx=0).stop 2023-12-19T18:17:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.714+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/4294452448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b480aef20 msgr2=0x7f0b48155d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:17:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.714+0000 7f0b4ea4d640 1 --2- 172.21.15.27:0/4294452448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b480aef20 0x7f0b48155d80 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00e9b0 tx=0x7f0b3c00ee70 comp rx=0 tx=0).stop 2023-12-19T18:17:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.714+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/4294452448 shutdown_connections 2023-12-19T18:17:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.714+0000 7f0b4ea4d640 1 --2- 172.21.15.27:0/4294452448 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0b2c06a430 0x7f0b2c06c8f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.714+0000 7f0b4ea4d640 1 --2- 172.21.15.27:0/4294452448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0b4815ac50 0x7f0b4813ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.714+0000 7f0b4ea4d640 1 --2- 172.21.15.27:0/4294452448 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0b480af360 0x7f0b481562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.714+0000 7f0b4ea4d640 1 --2- 172.21.15.27:0/4294452448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b480aef20 0x7f0b48155d80 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:17:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.714+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/4294452448 >> 172.21.15.27:0/4294452448 conn(0x7f0b4806a7d0 msgr2=0x7f0b480cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:17:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.715+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/4294452448 shutdown_connections 2023-12-19T18:17:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:17:59.715+0000 7f0b4ea4d640 1 -- 172.21.15.27:0/4294452448 wait complete. 2023-12-19T18:17:59.791 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.035+0000 7fb785737640 1 Processor -- start 2023-12-19T18:18:00.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.036+0000 7fb785737640 1 -- start start 2023-12-19T18:18:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.037+0000 7fb785737640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb78015ee30 0x7fb7800c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.037+0000 7fb785737640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb7800afeb0 0x7fb7800c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.037+0000 7fb785737640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7800cb350 con 0x7fb7801553a0 2023-12-19T18:18:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.038+0000 7fb785737640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800cbcf0 con 0x7fb78015ee30 2023-12-19T18:18:00.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.038+0000 7fb785737640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800cb630 con 0x7fb7800afeb0 2023-12-19T18:18:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.038+0000 7fb77f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb78015ee30 0x7fb7800c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.038+0000 7fb77f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb78015ee30 0x7fb7800c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47344/0 (socket says 172.21.15.27:47344) 2023-12-19T18:18:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.038+0000 7fb77f7fe640 1 -- 172.21.15.27:0/534691372 learned_addr learned my addr 172.21.15.27:0/534691372 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.038+0000 7fb784f36640 1 --2- 172.21.15.27:0/534691372 >> v2:172.21.15.192:3300/0 conn(0x7fb7800afeb0 0x7fb7800c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.038+0000 7fb77f7fe640 1 -- 172.21.15.27:0/534691372 >> v2:172.21.15.192:3300/0 conn(0x7fb7800afeb0 msgr2=0x7fb7800c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.038+0000 7fb77f7fe640 1 --2- 172.21.15.27:0/534691372 >> v2:172.21.15.192:3300/0 conn(0x7fb7800afeb0 0x7fb7800c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.038+0000 7fb77f7fe640 1 -- 172.21.15.27:0/534691372 >> v1:172.21.15.192:6789/0 conn(0x7fb7801553a0 legacy=0x7fb78015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.038+0000 7fb77f7fe640 1 -- 172.21.15.27:0/534691372 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7800cb910 con 0x7fb78015ee30 2023-12-19T18:18:00.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.040+0000 7fb77f7fe640 1 --2- 172.21.15.27:0/534691372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78015ee30 0x7fb7800c50a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb7800af670 tx=0x7fb77402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63f326a2510fbf98 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:00.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.040+0000 7fb77effd640 1 -- 172.21.15.27:0/534691372 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7740317e0 con 0x7fb78015ee30 2023-12-19T18:18:00.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.040+0000 7fb77effd640 1 -- 172.21.15.27:0/534691372 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb774031940 con 0x7fb78015ee30 2023-12-19T18:18:00.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.040+0000 7fb77effd640 1 -- 172.21.15.27:0/534691372 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb774031cd0 con 0x7fb78015ee30 2023-12-19T18:18:00.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.040+0000 7fb785737640 1 -- 172.21.15.27:0/534691372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78015ee30 msgr2=0x7fb7800c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:00.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.040+0000 7fb785737640 1 --2- 172.21.15.27:0/534691372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78015ee30 0x7fb7800c50a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fb7800af670 tx=0x7fb77402b880 comp rx=0 tx=0).stop 2023-12-19T18:18:00.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.041+0000 7fb785737640 1 -- 172.21.15.27:0/534691372 shutdown_connections 2023-12-19T18:18:00.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.041+0000 7fb785737640 1 --2- 172.21.15.27:0/534691372 >> v2:172.21.15.192:3300/0 conn(0x7fb7800afeb0 0x7fb7800c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.041+0000 7fb785737640 1 --2- 172.21.15.27:0/534691372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78015ee30 0x7fb7800c50a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.041+0000 7fb785737640 1 -- 172.21.15.27:0/534691372 >> 172.21.15.27:0/534691372 conn(0x7fb780069c70 msgr2=0x7fb78006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:00.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.041+0000 7fb785737640 1 -- 172.21.15.27:0/534691372 shutdown_connections 2023-12-19T18:18:00.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.041+0000 7fb785737640 1 -- 172.21.15.27:0/534691372 wait complete. 2023-12-19T18:18:00.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.043+0000 7fb785737640 1 Processor -- start 2023-12-19T18:18:00.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.043+0000 7fb785737640 1 -- start start 2023-12-19T18:18:00.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.044+0000 7fb785737640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7800afeb0 0x7fb7800c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:00.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.044+0000 7fb785737640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7801553a0 0x7fb7800c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:00.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.044+0000 7fb785737640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78015ee30 0x7fb780159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:00.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.044+0000 7fb785737640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7801f3c50 con 0x7fb78015ee30 2023-12-19T18:18:00.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.045+0000 7fb785737640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7801f3080 con 0x7fb7800afeb0 2023-12-19T18:18:00.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.045+0000 7fb785737640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7801f40e0 con 0x7fb7801553a0 2023-12-19T18:18:00.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.045+0000 7fb784f36640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78015ee30 0x7fb780159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:00.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.045+0000 7fb784f36640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78015ee30 0x7fb780159db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:47358/0 (socket says 172.21.15.27:47358) 2023-12-19T18:18:00.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.045+0000 7fb784f36640 1 -- 172.21.15.27:0/1892646997 learned_addr learned my addr 172.21.15.27:0/1892646997 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:00.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.045+0000 7fb77ffff640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7800afeb0 0x7fb7800c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:00.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.045+0000 7fb77f7fe640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7801553a0 0x7fb7800c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:00.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.045+0000 7fb784f36640 1 -- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7801553a0 msgr2=0x7fb7800c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:00.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.045+0000 7fb784f36640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7801553a0 0x7fb7800c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.045+0000 7fb784f36640 1 -- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7800afeb0 msgr2=0x7fb7800c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:00.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.045+0000 7fb784f36640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7800afeb0 0x7fb7800c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.045+0000 7fb784f36640 1 -- 172.21.15.27:0/1892646997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7740094d0 con 0x7fb78015ee30 2023-12-19T18:18:00.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.045+0000 7fb77f7fe640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7801553a0 0x7fb7800c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:00.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.046+0000 7fb784f36640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78015ee30 0x7fb780159db0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb778007e40 tx=0x7fb77800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:00.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.046+0000 7fb77cff9640 1 -- 172.21.15.27:0/1892646997 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778014620 con 0x7fb78015ee30 2023-12-19T18:18:00.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.046+0000 7fb77cff9640 1 -- 172.21.15.27:0/1892646997 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb77800ee90 con 0x7fb78015ee30 2023-12-19T18:18:00.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.046+0000 7fb77cff9640 1 -- 172.21.15.27:0/1892646997 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7780149e0 con 0x7fb78015ee30 2023-12-19T18:18:00.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.046+0000 7fb785737640 1 -- 172.21.15.27:0/1892646997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7801f3320 con 0x7fb78015ee30 2023-12-19T18:18:00.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.046+0000 7fb785737640 1 -- 172.21.15.27:0/1892646997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7801f38e0 con 0x7fb78015ee30 2023-12-19T18:18:00.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.047+0000 7fb785737640 1 -- 172.21.15.27:0/1892646997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb78004e840 con 0x7fb78015ee30 2023-12-19T18:18:00.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.048+0000 7fb77cff9640 1 -- 172.21.15.27:0/1892646997 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb778014b40 con 0x7fb78015ee30 2023-12-19T18:18:00.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.050+0000 7fb77cff9640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb76c06a430 0x7fb76c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:00.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.050+0000 7fb77cff9640 1 -- 172.21.15.27:0/1892646997 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb778089350 con 0x7fb78015ee30 2023-12-19T18:18:00.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.050+0000 7fb77ffff640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb76c06a430 0x7fb76c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:00.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.051+0000 7fb77ffff640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb76c06a430 0x7fb76c06c8f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb770004ea0 tx=0x7fb770009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:00.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.054+0000 7fb77cff9640 1 -- 172.21.15.27:0/1892646997 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb7780548b0 con 0x7fb78015ee30 2023-12-19T18:18:00.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.285+0000 7fb785737640 1 -- 172.21.15.27:0/1892646997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7801f3d90 con 0x7fb78015ee30 2023-12-19T18:18:00.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.286+0000 7fb77cff9640 1 -- 172.21.15.27:0/1892646997 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fb778057ed0 con 0x7fb78015ee30 2023-12-19T18:18:00.286 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:00.286 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/930391176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":930391176},{"type":"v1","addr":"172.21.15.192:6837","nonce":930391176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5179,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3606564673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3606564673},{"type":"v1","addr":"172.21.15.192:6836","nonce":3606564673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3604117097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3604117097},{"type":"v1","addr":"172.21.15.27:6834","nonce":3604117097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:14.250689+0000","modified":"2023-12-19T18:17:17.758670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3814458529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3814458529},{"type":"v1","addr":"172.21.15.27:6835","nonce":3814458529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T18:18:00.287 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-19T18:18:00.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.289+0000 7fb785737640 1 -- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb76c06a430 msgr2=0x7fb76c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:00.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.289+0000 7fb785737640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb76c06a430 0x7fb76c06c8f0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb770004ea0 tx=0x7fb770009040 comp rx=0 tx=0).stop 2023-12-19T18:18:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.289+0000 7fb785737640 1 -- 172.21.15.27:0/1892646997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78015ee30 msgr2=0x7fb780159db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.289+0000 7fb785737640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78015ee30 0x7fb780159db0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fb778007e40 tx=0x7fb77800e4b0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.289+0000 7fb785737640 1 -- 172.21.15.27:0/1892646997 shutdown_connections 2023-12-19T18:18:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.289+0000 7fb785737640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb76c06a430 0x7fb76c06c8f0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.289+0000 7fb785737640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb78015ee30 0x7fb780159db0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.289+0000 7fb785737640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb7801553a0 0x7fb7800c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.289+0000 7fb785737640 1 --2- 172.21.15.27:0/1892646997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb7800afeb0 0x7fb7800c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.289+0000 7fb785737640 1 -- 172.21.15.27:0/1892646997 >> 172.21.15.27:0/1892646997 conn(0x7fb780069c70 msgr2=0x7fb780065910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.289+0000 7fb785737640 1 -- 172.21.15.27:0/1892646997 shutdown_connections 2023-12-19T18:18:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:00.289+0000 7fb785737640 1 -- 172.21.15.27:0/1892646997 wait complete. 2023-12-19T18:18:00.310 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-12-19T18:18:00.566 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":5679,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":46.292694128999997,"mdsmap_epoch":23,"osdmap_epoch":63,"osdmap_epoch_barrier":56,"uptime":51.564765113999997} 2023-12-19T18:18:00.566 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 5679, "mdsmap_epoch": 23, "osdmap_epoch": 63, "osdmap_epoch_barrier": 56, "rank_uptime": 46.292694129, "state": "up:active", "uptime": 51.564765114, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:18:00.566 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-12-19T18:18:00.566 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:18:00.567 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:00.624 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:00.624 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-12-19T18:18:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:50+0 records in 2023-12-19T18:18:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:50+0 records out 2023-12-19T18:18:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:52428800 bytes (52 MB, 50 MiB) copied, 1.00622 s, 52.1 MB/s 2023-12-19T18:18:01.692 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:18:01.692 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:18:01.833 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:18:01.957 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:18:01.958 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:18:01.958 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:02.023 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:02.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.256+0000 7ff9d733d640 1 Processor -- start 2023-12-19T18:18:02.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.257+0000 7ff9d733d640 1 -- start start 2023-12-19T18:18:02.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.258+0000 7ff9d733d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff9d01604d0 0x7ff9d00b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.258+0000 7ff9d733d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d00b5e90 con 0x7ff9d00b7310 2023-12-19T18:18:02.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.258+0000 7ff9d733d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9d00d1750 con 0x7ff9d00d2350 2023-12-19T18:18:02.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.258+0000 7ff9d733d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9d00d2b30 con 0x7ff9d01604d0 2023-12-19T18:18:02.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.259+0000 7ff9d6b3c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff9d00b7310 0x7ff9d00d1040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33538/0 (socket says 172.21.15.27:33538) 2023-12-19T18:18:02.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.259+0000 7ff9d6b3c640 1 -- 172.21.15.27:0/1562536440 learned_addr learned my addr 172.21.15.27:0/1562536440 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:02.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.259+0000 7ff9d633b640 1 --2- 172.21.15.27:0/1562536440 >> v2:172.21.15.192:3300/0 conn(0x7ff9d01604d0 0x7ff9d00b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.259+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142486120 0 0) 0x7ff9d00b5e90 con 0x7ff9d00b7310 2023-12-19T18:18:02.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.259+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9bc003520 con 0x7ff9d00b7310 2023-12-19T18:18:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 203157151 0 0) 0x7ff9bc003520 con 0x7ff9d00b7310 2023-12-19T18:18:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 >> v2:172.21.15.192:3300/0 conn(0x7ff9d01604d0 msgr2=0x7ff9d00b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d5339640 1 --2- 172.21.15.27:0/1562536440 >> v2:172.21.15.192:3300/0 conn(0x7ff9d01604d0 0x7ff9d00b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 >> v1:172.21.15.192:6789/0 conn(0x7ff9d00d2350 legacy=0x7ff9d00d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d00d2850 con 0x7ff9d00b7310 2023-12-19T18:18:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7ff9c80019f0 con 0x7ff9d00b7310 2023-12-19T18:18:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 >> v1:172.21.15.27:6789/0 conn(0x7ff9d00b7310 legacy=0x7ff9d00d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d5339640 1 --2- 172.21.15.27:0/1562536440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0039c0 0x7ff9bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d5339640 1 --2- 172.21.15.27:0/1562536440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9bc0047c0 0x7ff9bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d5339640 1 --2- 172.21.15.27:0/1562536440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9bc0051d0 0x7ff9bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d00d2b30 con 0x7ff9bc0039c0 2023-12-19T18:18:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc00c120 con 0x7ff9bc0051d0 2023-12-19T18:18:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc00c2e0 con 0x7ff9bc0047c0 2023-12-19T18:18:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.260+0000 7ff9d633b640 1 --2- 172.21.15.27:0/1562536440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0039c0 0x7ff9bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.261+0000 7ff9d5b3a640 1 --2- 172.21.15.27:0/1562536440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9bc0047c0 0x7ff9bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.261+0000 7ff9d633b640 1 -- 172.21.15.27:0/1562536440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9bc0047c0 msgr2=0x7ff9bc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.261+0000 7ff9d633b640 1 --2- 172.21.15.27:0/1562536440 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9bc0047c0 0x7ff9bc004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.261+0000 7ff9d633b640 1 -- 172.21.15.27:0/1562536440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9bc0051d0 msgr2=0x7ff9bc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.261+0000 7ff9d6b3c640 1 --2- 172.21.15.27:0/1562536440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9bc0051d0 0x7ff9bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.261+0000 7ff9d633b640 1 --2- 172.21.15.27:0/1562536440 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9bc0051d0 0x7ff9bc00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.261+0000 7ff9d633b640 1 -- 172.21.15.27:0/1562536440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc003520 con 0x7ff9bc0039c0 2023-12-19T18:18:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.261+0000 7ff9d6b3c640 1 -- 172.21.15.27:0/1562536440 reap_dead start 2023-12-19T18:18:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.262+0000 7ff9d633b640 1 --2- 172.21.15.27:0/1562536440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0039c0 0x7ff9bc003de0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009a90 tx=0x7ff9bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bca6e12ded951b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.262+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc02c680 con 0x7ff9bc0039c0 2023-12-19T18:18:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.262+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc02c7e0 con 0x7ff9bc0039c0 2023-12-19T18:18:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.262+0000 7ff9d5339640 1 -- 172.21.15.27:0/1562536440 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc02cb70 con 0x7ff9bc0039c0 2023-12-19T18:18:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.263+0000 7ff9d733d640 1 -- 172.21.15.27:0/1562536440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0039c0 msgr2=0x7ff9bc003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.263+0000 7ff9d733d640 1 --2- 172.21.15.27:0/1562536440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0039c0 0x7ff9bc003de0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009a90 tx=0x7ff9bc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.263+0000 7ff9d733d640 1 -- 172.21.15.27:0/1562536440 shutdown_connections 2023-12-19T18:18:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.263+0000 7ff9d733d640 1 --2- 172.21.15.27:0/1562536440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0039c0 0x7ff9bc003de0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.263+0000 7ff9d733d640 1 -- 172.21.15.27:0/1562536440 >> 172.21.15.27:0/1562536440 conn(0x7ff9d00cf7e0 msgr2=0x7ff9d00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.263+0000 7ff9d733d640 1 -- 172.21.15.27:0/1562536440 shutdown_connections 2023-12-19T18:18:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.263+0000 7ff9d733d640 1 -- 172.21.15.27:0/1562536440 wait complete. 2023-12-19T18:18:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.265+0000 7ff9d733d640 1 Processor -- start 2023-12-19T18:18:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.265+0000 7ff9d733d640 1 -- start start 2023-12-19T18:18:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.266+0000 7ff9d733d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0039c0 0x7ff9d015c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.266+0000 7ff9d733d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9d015c680 0x7ff9d00c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.266+0000 7ff9d733d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9d01f2980 0x7ff9d01f2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.266+0000 7ff9d733d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d01e6bb0 con 0x7ff9bc0039c0 2023-12-19T18:18:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.266+0000 7ff9d733d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d00b3890 con 0x7ff9d01f2980 2023-12-19T18:18:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.266+0000 7ff9d733d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9d00b3b70 con 0x7ff9d015c680 2023-12-19T18:18:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.266+0000 7ff9d633b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0039c0 0x7ff9d015c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.266+0000 7ff9d633b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0039c0 0x7ff9d015c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57956/0 (socket says 172.21.15.27:57956) 2023-12-19T18:18:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.266+0000 7ff9d633b640 1 -- 172.21.15.27:0/1929391551 learned_addr learned my addr 172.21.15.27:0/1929391551 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.267+0000 7ff9d6b3c640 1 --2- 172.21.15.27:0/1929391551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9d01f2980 0x7ff9d01f2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.267+0000 7ff9d633b640 1 -- 172.21.15.27:0/1929391551 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9d015c680 msgr2=0x7ff9d00c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.267+0000 7ff9d633b640 1 --2- 172.21.15.27:0/1929391551 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9d015c680 0x7ff9d00c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.267+0000 7ff9d633b640 1 -- 172.21.15.27:0/1929391551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9d01f2980 msgr2=0x7ff9d01f2e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.267+0000 7ff9d633b640 1 --2- 172.21.15.27:0/1929391551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9d01f2980 0x7ff9d01f2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.267+0000 7ff9d633b640 1 -- 172.21.15.27:0/1929391551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d00b3e50 con 0x7ff9bc0039c0 2023-12-19T18:18:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.267+0000 7ff9d633b640 1 --2- 172.21.15.27:0/1929391551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0039c0 0x7ff9d015c0c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009a90 tx=0x7ff9cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.268+0000 7ff9c2ffd640 1 -- 172.21.15.27:0/1929391551 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc003810 con 0x7ff9bc0039c0 2023-12-19T18:18:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.268+0000 7ff9c2ffd640 1 -- 172.21.15.27:0/1929391551 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc003970 con 0x7ff9bc0039c0 2023-12-19T18:18:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.268+0000 7ff9d733d640 1 -- 172.21.15.27:0/1929391551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9d00b4130 con 0x7ff9bc0039c0 2023-12-19T18:18:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.268+0000 7ff9c2ffd640 1 -- 172.21.15.27:0/1929391551 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc003d00 con 0x7ff9bc0039c0 2023-12-19T18:18:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.268+0000 7ff9d733d640 1 -- 172.21.15.27:0/1929391551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9d00b3520 con 0x7ff9bc0039c0 2023-12-19T18:18:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.269+0000 7ff9d733d640 1 -- 172.21.15.27:0/1929391551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9d0066f80 con 0x7ff9bc0039c0 2023-12-19T18:18:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.270+0000 7ff9c2ffd640 1 -- 172.21.15.27:0/1929391551 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff9cc040450 con 0x7ff9bc0039c0 2023-12-19T18:18:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.272+0000 7ff9c2ffd640 1 --2- 172.21.15.27:0/1929391551 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff9b406a430 0x7ff9b406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.272+0000 7ff9c2ffd640 1 -- 172.21.15.27:0/1929391551 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7ff9cc0aa950 con 0x7ff9bc0039c0 2023-12-19T18:18:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.272+0000 7ff9d5b3a640 1 --2- 172.21.15.27:0/1929391551 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff9b406a430 0x7ff9b406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.274+0000 7ff9d5b3a640 1 --2- 172.21.15.27:0/1929391551 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff9b406a430 0x7ff9b406c8f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4000f60 tx=0x7ff9c4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:02.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.276+0000 7ff9c2ffd640 1 -- 172.21.15.27:0/1929391551 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff9cc075e10 con 0x7ff9bc0039c0 2023-12-19T18:18:02.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.476+0000 7ff9d733d640 1 -- 172.21.15.27:0/1929391551 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9d004e840 con 0x7ff9bc0039c0 2023-12-19T18:18:02.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.479+0000 7ff9c2ffd640 1 -- 172.21.15.27:0/1929391551 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16274 (secure 0 0 0) 0x7ff9cc0aa020 con 0x7ff9bc0039c0 2023-12-19T18:18:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:02.480 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":63,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:17:56.605695+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:12.527441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:13.291755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.8","mappings":[{"from":5,"to":7}]},{"pgid":"7.9","mappings":[{"from":5,"to":7}]},{"pgid":"7.20","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:02.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.482+0000 7ff9d733d640 1 -- 172.21.15.27:0/1929391551 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff9b406a430 msgr2=0x7ff9b406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.482+0000 7ff9d733d640 1 --2- 172.21.15.27:0/1929391551 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff9b406a430 0x7ff9b406c8f0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4000f60 tx=0x7ff9c4009040 comp rx=0 tx=0).stop 2023-12-19T18:18:02.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.482+0000 7ff9d733d640 1 -- 172.21.15.27:0/1929391551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0039c0 msgr2=0x7ff9d015c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.482+0000 7ff9d733d640 1 --2- 172.21.15.27:0/1929391551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0039c0 0x7ff9d015c0c0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc009a90 tx=0x7ff9cc004730 comp rx=0 tx=0).stop 2023-12-19T18:18:02.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.482+0000 7ff9d733d640 1 -- 172.21.15.27:0/1929391551 shutdown_connections 2023-12-19T18:18:02.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.482+0000 7ff9d733d640 1 --2- 172.21.15.27:0/1929391551 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff9b406a430 0x7ff9b406c8f0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.482+0000 7ff9d733d640 1 --2- 172.21.15.27:0/1929391551 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9d01f2980 0x7ff9d01f2e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.482+0000 7ff9d733d640 1 --2- 172.21.15.27:0/1929391551 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9d015c680 0x7ff9d00c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.482+0000 7ff9d733d640 1 --2- 172.21.15.27:0/1929391551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0039c0 0x7ff9d015c0c0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.482+0000 7ff9d733d640 1 -- 172.21.15.27:0/1929391551 >> 172.21.15.27:0/1929391551 conn(0x7ff9d00cf7e0 msgr2=0x7ff9d00cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:02.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.482+0000 7ff9d733d640 1 -- 172.21.15.27:0/1929391551 shutdown_connections 2023-12-19T18:18:02.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.482+0000 7ff9d733d640 1 -- 172.21.15.27:0/1929391551 wait complete. 2023-12-19T18:18:02.502 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:17:12.527441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:17:13.291755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:18:02.502 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:18:02.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.745+0000 7f7b7855c640 1 Processor -- start 2023-12-19T18:18:02.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.745+0000 7f7b7855c640 1 -- start start 2023-12-19T18:18:02.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.746+0000 7f7b7855c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7b700b8c30 0x7f7b700d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.746+0000 7f7b7855c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b700ce130 0x7f7b700cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.747+0000 7f7b7855c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b700ce560 con 0x7f7b700cf9b0 2023-12-19T18:18:02.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.747+0000 7f7b7855c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b700cea80 con 0x7f7b700b8c30 2023-12-19T18:18:02.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.747+0000 7f7b7855c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b700b5f10 con 0x7f7b700ce130 2023-12-19T18:18:02.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.747+0000 7f7b762f8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7b700b8c30 0x7f7b700d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.747+0000 7f7b75af7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7b700cf9b0 0x7f7b700cda20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33542/0 (socket says 172.21.15.27:33542) 2023-12-19T18:18:02.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.747+0000 7f7b75af7640 1 -- 172.21.15.27:0/1656715458 learned_addr learned my addr 172.21.15.27:0/1656715458 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:02.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.747+0000 7f7b76af9640 1 --2- 172.21.15.27:0/1656715458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b700ce130 0x7f7b700cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.747+0000 7f7b762f8640 1 -- 172.21.15.27:0/1656715458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b700ce130 msgr2=0x7f7b700cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:02.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.747+0000 7f7b762f8640 1 --2- 172.21.15.27:0/1656715458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b700ce130 0x7f7b700cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.747+0000 7f7b762f8640 1 -- 172.21.15.27:0/1656715458 >> v1:172.21.15.27:6789/0 conn(0x7f7b700cf9b0 legacy=0x7f7b700cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.747+0000 7f7b762f8640 1 -- 172.21.15.27:0/1656715458 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b700ce7a0 con 0x7f7b700b8c30 2023-12-19T18:18:02.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.747+0000 7f7b76af9640 1 --2- 172.21.15.27:0/1656715458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b700ce130 0x7f7b700cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:02.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.749+0000 7f7b762f8640 1 --2- 172.21.15.27:0/1656715458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b700b8c30 0x7f7b700d1fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0097a0 tx=0x7f7b6c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3cdbbee40a62411 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:02.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.749+0000 7f7b752f6640 1 -- 172.21.15.27:0/1656715458 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c02e070 con 0x7f7b700b8c30 2023-12-19T18:18:02.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.749+0000 7f7b752f6640 1 -- 172.21.15.27:0/1656715458 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c0315a0 con 0x7f7b700b8c30 2023-12-19T18:18:02.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.749+0000 7f7b752f6640 1 -- 172.21.15.27:0/1656715458 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c031930 con 0x7f7b700b8c30 2023-12-19T18:18:02.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.750+0000 7f7b7855c640 1 -- 172.21.15.27:0/1656715458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b700b8c30 msgr2=0x7f7b700d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.750+0000 7f7b7855c640 1 --2- 172.21.15.27:0/1656715458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b700b8c30 0x7f7b700d1fc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c0097a0 tx=0x7f7b6c02b840 comp rx=0 tx=0).stop 2023-12-19T18:18:02.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.750+0000 7f7b7855c640 1 -- 172.21.15.27:0/1656715458 shutdown_connections 2023-12-19T18:18:02.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.750+0000 7f7b7855c640 1 --2- 172.21.15.27:0/1656715458 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b700ce130 0x7f7b700cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.750+0000 7f7b7855c640 1 --2- 172.21.15.27:0/1656715458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b700b8c30 0x7f7b700d1fc0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.750+0000 7f7b7855c640 1 -- 172.21.15.27:0/1656715458 >> 172.21.15.27:0/1656715458 conn(0x7f7b700d2fa0 msgr2=0x7f7b700d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:02.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.750+0000 7f7b7855c640 1 -- 172.21.15.27:0/1656715458 shutdown_connections 2023-12-19T18:18:02.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.750+0000 7f7b7855c640 1 -- 172.21.15.27:0/1656715458 wait complete. 2023-12-19T18:18:02.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.752+0000 7f7b7855c640 1 Processor -- start 2023-12-19T18:18:02.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.752+0000 7f7b7855c640 1 -- start start 2023-12-19T18:18:02.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.753+0000 7f7b7855c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b700b8c30 0x7f7b700ae390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.753+0000 7f7b7855c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b700ce130 0x7f7b700aa470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.753+0000 7f7b7855c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b700cf9b0 0x7f7b701f3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.753+0000 7f7b7855c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b700ad330 con 0x7f7b700b8c30 2023-12-19T18:18:02.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.753+0000 7f7b7855c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b700ad5d0 con 0x7f7b700ce130 2023-12-19T18:18:02.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.753+0000 7f7b7855c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b700ad8b0 con 0x7f7b700cf9b0 2023-12-19T18:18:02.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.753+0000 7f7b762f8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b700b8c30 0x7f7b700ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.753+0000 7f7b762f8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b700b8c30 0x7f7b700ae390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57970/0 (socket says 172.21.15.27:57970) 2023-12-19T18:18:02.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.753+0000 7f7b762f8640 1 -- 172.21.15.27:0/1871593884 learned_addr learned my addr 172.21.15.27:0/1871593884 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:02.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.753+0000 7f7b76af9640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b700cf9b0 0x7f7b701f3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.754+0000 7f7b75af7640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b700ce130 0x7f7b700aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.754+0000 7f7b762f8640 1 -- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b700cf9b0 msgr2=0x7f7b701f3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.754+0000 7f7b762f8640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b700cf9b0 0x7f7b701f3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.754+0000 7f7b762f8640 1 -- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b700ce130 msgr2=0x7f7b700aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.754+0000 7f7b762f8640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b700ce130 0x7f7b700aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.754+0000 7f7b762f8640 1 -- 172.21.15.27:0/1871593884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b6c009400 con 0x7f7b700b8c30 2023-12-19T18:18:02.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.754+0000 7f7b762f8640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b700b8c30 0x7f7b700ae390 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c034040 tx=0x7f7b6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:02.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.754+0000 7f7b66ffd640 1 -- 172.21.15.27:0/1871593884 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c032900 con 0x7f7b700b8c30 2023-12-19T18:18:02.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.755+0000 7f7b66ffd640 1 -- 172.21.15.27:0/1871593884 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6c032a60 con 0x7f7b700b8c30 2023-12-19T18:18:02.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.755+0000 7f7b66ffd640 1 -- 172.21.15.27:0/1871593884 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6c032e10 con 0x7f7b700b8c30 2023-12-19T18:18:02.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.755+0000 7f7b7855c640 1 -- 172.21.15.27:0/1871593884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b700adb90 con 0x7f7b700b8c30 2023-12-19T18:18:02.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.755+0000 7f7b7855c640 1 -- 172.21.15.27:0/1871593884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b701f3c60 con 0x7f7b700b8c30 2023-12-19T18:18:02.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.756+0000 7f7b7855c640 1 -- 172.21.15.27:0/1871593884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b70068e90 con 0x7f7b700b8c30 2023-12-19T18:18:02.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.757+0000 7f7b66ffd640 1 -- 172.21.15.27:0/1871593884 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7b6c04e430 con 0x7f7b700b8c30 2023-12-19T18:18:02.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.758+0000 7f7b66ffd640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5406a430 0x7f7b5406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:02.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.758+0000 7f7b66ffd640 1 -- 172.21.15.27:0/1871593884 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7b6c0aab40 con 0x7f7b700b8c30 2023-12-19T18:18:02.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.758+0000 7f7b75af7640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5406a430 0x7f7b5406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:02.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.760+0000 7f7b75af7640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5406a430 0x7f7b5406c8f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7b700b5240 tx=0x7f7b60009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:02.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.762+0000 7f7b66ffd640 1 -- 172.21.15.27:0/1871593884 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7b6c076090 con 0x7f7b700b8c30 2023-12-19T18:18:02.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.954+0000 7f7b7855c640 1 -- 172.21.15.27:0/1871593884 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f7b700d0ef0 con 0x7f7b700b8c30 2023-12-19T18:18:02.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.955+0000 7f7b66ffd640 1 -- 172.21.15.27:0/1871593884 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7b6c0796b0 con 0x7f7b700b8c30 2023-12-19T18:18:02.955 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:18:02.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.957+0000 7f7b7855c640 1 -- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5406a430 msgr2=0x7f7b5406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.957+0000 7f7b7855c640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5406a430 0x7f7b5406c8f0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7b700b5240 tx=0x7f7b60009040 comp rx=0 tx=0).stop 2023-12-19T18:18:02.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.957+0000 7f7b7855c640 1 -- 172.21.15.27:0/1871593884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b700b8c30 msgr2=0x7f7b700ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:02.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.957+0000 7f7b7855c640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b700b8c30 0x7f7b700ae390 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c034040 tx=0x7f7b6c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.957+0000 7f7b7855c640 1 -- 172.21.15.27:0/1871593884 shutdown_connections 2023-12-19T18:18:02.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.957+0000 7f7b7855c640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5406a430 0x7f7b5406c8f0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.957+0000 7f7b7855c640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b700cf9b0 0x7f7b701f3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.957+0000 7f7b7855c640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b700ce130 0x7f7b700aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.957+0000 7f7b7855c640 1 --2- 172.21.15.27:0/1871593884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b700b8c30 0x7f7b700ae390 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:02.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.957+0000 7f7b7855c640 1 -- 172.21.15.27:0/1871593884 >> 172.21.15.27:0/1871593884 conn(0x7f7b700d2fa0 msgr2=0x7f7b700b2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:02.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.957+0000 7f7b7855c640 1 -- 172.21.15.27:0/1871593884 shutdown_connections 2023-12-19T18:18:02.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:02.957+0000 7f7b7855c640 1 -- 172.21.15.27:0/1871593884 wait complete. 2023-12-19T18:18:02.978 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:18:03.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.219+0000 7f5235527640 1 Processor -- start 2023-12-19T18:18:03.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.219+0000 7f5235527640 1 -- start start 2023-12-19T18:18:03.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.221+0000 7f5235527640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52300ce130 0x7f52300cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.221+0000 7f5235527640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52300ce540 con 0x7f52300cf9b0 2023-12-19T18:18:03.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.221+0000 7f5235527640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52300d26d0 con 0x7f52300b8c30 2023-12-19T18:18:03.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.221+0000 7f5235527640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52300cea80 con 0x7f52300ce130 2023-12-19T18:18:03.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.221+0000 7f522f7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52300ce130 0x7f52300cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.221+0000 7f522e7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f52300cf9b0 0x7f52300cda20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33550/0 (socket says 172.21.15.27:33550) 2023-12-19T18:18:03.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.221+0000 7f522e7fc640 1 -- 172.21.15.27:0/2904424632 learned_addr learned my addr 172.21.15.27:0/2904424632 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:03.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.222+0000 7f522dffb640 1 -- 172.21.15.27:0/2904424632 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4036068828 0 0) 0x7f52300ce540 con 0x7f52300cf9b0 2023-12-19T18:18:03.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.222+0000 7f522dffb640 1 -- 172.21.15.27:0/2904424632 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f521c003520 con 0x7f52300cf9b0 2023-12-19T18:18:03.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.222+0000 7f522f7fe640 1 -- 172.21.15.27:0/2904424632 >> v1:172.21.15.192:6789/0 conn(0x7f52300b8c30 legacy=0x7f52300d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.222+0000 7f522f7fe640 1 -- 172.21.15.27:0/2904424632 >> v1:172.21.15.27:6789/0 conn(0x7f52300cf9b0 legacy=0x7f52300cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.222+0000 7f522f7fe640 1 -- 172.21.15.27:0/2904424632 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52300ce7a0 con 0x7f52300ce130 2023-12-19T18:18:03.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.224+0000 7f522f7fe640 1 --2- 172.21.15.27:0/2904424632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52300ce130 0x7f52300cfe40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f52280097a0 tx=0x7f522802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d583134fcaeb4d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:03.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.224+0000 7f522dffb640 1 -- 172.21.15.27:0/2904424632 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228031830 con 0x7f52300ce130 2023-12-19T18:18:03.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.224+0000 7f522dffb640 1 -- 172.21.15.27:0/2904424632 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5228031990 con 0x7f52300ce130 2023-12-19T18:18:03.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.224+0000 7f522dffb640 1 -- 172.21.15.27:0/2904424632 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228031d20 con 0x7f52300ce130 2023-12-19T18:18:03.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.225+0000 7f5235527640 1 -- 172.21.15.27:0/2904424632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52300ce130 msgr2=0x7f52300cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.225+0000 7f5235527640 1 --2- 172.21.15.27:0/2904424632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52300ce130 0x7f52300cfe40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f52280097a0 tx=0x7f522802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.225+0000 7f5235527640 1 -- 172.21.15.27:0/2904424632 shutdown_connections 2023-12-19T18:18:03.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.225+0000 7f5235527640 1 --2- 172.21.15.27:0/2904424632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52300ce130 0x7f52300cfe40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.225+0000 7f5235527640 1 -- 172.21.15.27:0/2904424632 >> 172.21.15.27:0/2904424632 conn(0x7f52300d2fa0 msgr2=0x7f52300d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:03.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.225+0000 7f5235527640 1 -- 172.21.15.27:0/2904424632 shutdown_connections 2023-12-19T18:18:03.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.225+0000 7f5235527640 1 -- 172.21.15.27:0/2904424632 wait complete. 2023-12-19T18:18:03.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.227+0000 7f5235527640 1 Processor -- start 2023-12-19T18:18:03.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.227+0000 7f5235527640 1 -- start start 2023-12-19T18:18:03.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.228+0000 7f5235527640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52300b8c30 0x7f52300b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.228+0000 7f5235527640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52300ce130 0x7f52300b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.228+0000 7f5235527640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52300cf9b0 0x7f5230157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.228+0000 7f5235527640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5230158b70 con 0x7f52300b8c30 2023-12-19T18:18:03.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.228+0000 7f5235527640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52300b30c0 con 0x7f52300ce130 2023-12-19T18:18:03.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.228+0000 7f5235527640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52300b33a0 con 0x7f52300cf9b0 2023-12-19T18:18:03.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.229+0000 7f522effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52300b8c30 0x7f52300b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.229+0000 7f522effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52300b8c30 0x7f52300b4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57984/0 (socket says 172.21.15.27:57984) 2023-12-19T18:18:03.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.229+0000 7f522effd640 1 -- 172.21.15.27:0/2786354874 learned_addr learned my addr 172.21.15.27:0/2786354874 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:03.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.229+0000 7f522f7fe640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52300cf9b0 0x7f5230157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.229+0000 7f522e7fc640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52300ce130 0x7f52300b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.229+0000 7f522effd640 1 -- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52300cf9b0 msgr2=0x7f5230157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.229+0000 7f522effd640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52300cf9b0 0x7f5230157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.229+0000 7f522effd640 1 -- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52300ce130 msgr2=0x7f52300b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.229+0000 7f522effd640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52300ce130 0x7f52300b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.229+0000 7f522effd640 1 -- 172.21.15.27:0/2786354874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5228009400 con 0x7f52300b8c30 2023-12-19T18:18:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.229+0000 7f522e7fc640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52300ce130 0x7f52300b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.230+0000 7f522effd640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52300b8c30 0x7f52300b4160 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f522000c870 tx=0x7f522000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.230+0000 7f52277fe640 1 -- 172.21.15.27:0/2786354874 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5220003b70 con 0x7f52300b8c30 2023-12-19T18:18:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.230+0000 7f52277fe640 1 -- 172.21.15.27:0/2786354874 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5220003cf0 con 0x7f52300b8c30 2023-12-19T18:18:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.230+0000 7f52277fe640 1 -- 172.21.15.27:0/2786354874 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522000aa90 con 0x7f52300b8c30 2023-12-19T18:18:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.230+0000 7f5235527640 1 -- 172.21.15.27:0/2786354874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52300b3680 con 0x7f52300b8c30 2023-12-19T18:18:03.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.230+0000 7f5235527640 1 -- 172.21.15.27:0/2786354874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52300b3c40 con 0x7f52300b8c30 2023-12-19T18:18:03.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.231+0000 7f5235527640 1 -- 172.21.15.27:0/2786354874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5230067110 con 0x7f52300b8c30 2023-12-19T18:18:03.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.232+0000 7f52277fe640 1 -- 172.21.15.27:0/2786354874 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f522000abf0 con 0x7f52300b8c30 2023-12-19T18:18:03.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.234+0000 7f52277fe640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f521006a430 0x7f521006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.234+0000 7f52277fe640 1 -- 172.21.15.27:0/2786354874 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5220088920 con 0x7f52300b8c30 2023-12-19T18:18:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.235+0000 7f522e7fc640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f521006a430 0x7f521006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.236+0000 7f522e7fc640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f521006a430 0x7f521006c8f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f52300b4d90 tx=0x7f5218009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:03.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.238+0000 7f52277fe640 1 -- 172.21.15.27:0/2786354874 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5220053de0 con 0x7f52300b8c30 2023-12-19T18:18:03.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.431+0000 7f5235527640 1 -- 172.21.15.27:0/2786354874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f52300cd360 con 0x7f52300b8c30 2023-12-19T18:18:03.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.432+0000 7f52277fe640 1 -- 172.21.15.27:0/2786354874 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5220003e60 con 0x7f52300b8c30 2023-12-19T18:18:03.433 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:18:03.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.434+0000 7f5235527640 1 -- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f521006a430 msgr2=0x7f521006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.434+0000 7f5235527640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f521006a430 0x7f521006c8f0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f52300b4d90 tx=0x7f5218009040 comp rx=0 tx=0).stop 2023-12-19T18:18:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.434+0000 7f5235527640 1 -- 172.21.15.27:0/2786354874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52300b8c30 msgr2=0x7f52300b4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.434+0000 7f5235527640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52300b8c30 0x7f52300b4160 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f522000c870 tx=0x7f522000cd30 comp rx=0 tx=0).stop 2023-12-19T18:18:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.434+0000 7f5235527640 1 -- 172.21.15.27:0/2786354874 shutdown_connections 2023-12-19T18:18:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.434+0000 7f5235527640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f521006a430 0x7f521006c8f0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.434+0000 7f5235527640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52300cf9b0 0x7f5230157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.434+0000 7f5235527640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52300ce130 0x7f52300b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.434+0000 7f5235527640 1 --2- 172.21.15.27:0/2786354874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52300b8c30 0x7f52300b4160 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.434+0000 7f5235527640 1 -- 172.21.15.27:0/2786354874 >> 172.21.15.27:0/2786354874 conn(0x7f52300d2fa0 msgr2=0x7f52300d0260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:03.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.435+0000 7f5235527640 1 -- 172.21.15.27:0/2786354874 shutdown_connections 2023-12-19T18:18:03.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.435+0000 7f5235527640 1 -- 172.21.15.27:0/2786354874 wait complete. 2023-12-19T18:18:03.458 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:18:03.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.688+0000 7fce489df640 1 Processor -- start 2023-12-19T18:18:03.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.689+0000 7fce489df640 1 -- start start 2023-12-19T18:18:03.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.690+0000 7fce489df640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fce4415ce80 0x7fce44161400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.690+0000 7fce489df640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce440b2660 0x7fce440b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.690+0000 7fce489df640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fce4406ad80 0x7fce4406b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.690+0000 7fce489df640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce440b3000 con 0x7fce4415ce80 2023-12-19T18:18:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.690+0000 7fce489df640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce44161940 con 0x7fce4406ad80 2023-12-19T18:18:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.690+0000 7fce489df640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce440b00e0 con 0x7fce440b2660 2023-12-19T18:18:03.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.690+0000 7fce437fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fce4415ce80 0x7fce44161400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.691+0000 7fce437fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fce4415ce80 0x7fce44161400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57992/0 (socket says 172.21.15.27:57992) 2023-12-19T18:18:03.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.691+0000 7fce437fe640 1 -- 172.21.15.27:0/445954498 learned_addr learned my addr 172.21.15.27:0/445954498 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:03.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.691+0000 7fce437fe640 1 -- 172.21.15.27:0/445954498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce440b2660 msgr2=0x7fce440b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:03.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.691+0000 7fce3bfff640 1 --2- 172.21.15.27:0/445954498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce440b2660 0x7fce440b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.691+0000 7fce437fe640 1 --2- 172.21.15.27:0/445954498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce440b2660 0x7fce440b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.691+0000 7fce437fe640 1 -- 172.21.15.27:0/445954498 >> v2:172.21.15.192:3300/0 conn(0x7fce4406ad80 msgr2=0x7fce4406b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:03.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.691+0000 7fce437fe640 1 --2- 172.21.15.27:0/445954498 >> v2:172.21.15.192:3300/0 conn(0x7fce4406ad80 0x7fce4406b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.691+0000 7fce437fe640 1 -- 172.21.15.27:0/445954498 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4415d260 con 0x7fce4415ce80 2023-12-19T18:18:03.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.691+0000 7fce3bfff640 1 --2- 172.21.15.27:0/445954498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce440b2660 0x7fce440b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:03.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.692+0000 7fce437fe640 1 --2- 172.21.15.27:0/445954498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4415ce80 0x7fce44161400 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fce44067020 tx=0x7fce3402b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6decdeff7c96ed5b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:03.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.693+0000 7fce42ffd640 1 -- 172.21.15.27:0/445954498 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34031880 con 0x7fce4415ce80 2023-12-19T18:18:03.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.693+0000 7fce42ffd640 1 -- 172.21.15.27:0/445954498 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce340319e0 con 0x7fce4415ce80 2023-12-19T18:18:03.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.693+0000 7fce42ffd640 1 -- 172.21.15.27:0/445954498 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce34031d70 con 0x7fce4415ce80 2023-12-19T18:18:03.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.693+0000 7fce489df640 1 -- 172.21.15.27:0/445954498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4415ce80 msgr2=0x7fce44161400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.693+0000 7fce489df640 1 --2- 172.21.15.27:0/445954498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4415ce80 0x7fce44161400 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fce44067020 tx=0x7fce3402b920 comp rx=0 tx=0).stop 2023-12-19T18:18:03.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.693+0000 7fce489df640 1 -- 172.21.15.27:0/445954498 shutdown_connections 2023-12-19T18:18:03.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.693+0000 7fce489df640 1 --2- 172.21.15.27:0/445954498 >> v2:172.21.15.192:3300/0 conn(0x7fce4406ad80 0x7fce4406b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.693+0000 7fce489df640 1 --2- 172.21.15.27:0/445954498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce440b2660 0x7fce440b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.693+0000 7fce489df640 1 --2- 172.21.15.27:0/445954498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4415ce80 0x7fce44161400 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.693+0000 7fce489df640 1 -- 172.21.15.27:0/445954498 >> 172.21.15.27:0/445954498 conn(0x7fce4406f120 msgr2=0x7fce4406f500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:03.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.693+0000 7fce489df640 1 -- 172.21.15.27:0/445954498 shutdown_connections 2023-12-19T18:18:03.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.693+0000 7fce489df640 1 -- 172.21.15.27:0/445954498 wait complete. 2023-12-19T18:18:03.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.695+0000 7fce489df640 1 Processor -- start 2023-12-19T18:18:03.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.695+0000 7fce489df640 1 -- start start 2023-12-19T18:18:03.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.696+0000 7fce489df640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce4406ad80 0x7fce4415ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.696+0000 7fce489df640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce440b2660 0x7fce440af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.696+0000 7fce489df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4415ce80 0x7fce440c6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.696+0000 7fce489df640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4415d680 con 0x7fce4415ce80 2023-12-19T18:18:03.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.696+0000 7fce489df640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce440c7e30 con 0x7fce4406ad80 2023-12-19T18:18:03.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.696+0000 7fce489df640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4415fb10 con 0x7fce440b2660 2023-12-19T18:18:03.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.697+0000 7fce43fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4415ce80 0x7fce440c6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.697+0000 7fce43fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4415ce80 0x7fce440c6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57994/0 (socket says 172.21.15.27:57994) 2023-12-19T18:18:03.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.697+0000 7fce43fff640 1 -- 172.21.15.27:0/3413556426 learned_addr learned my addr 172.21.15.27:0/3413556426 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:03.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.697+0000 7fce3bfff640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce440b2660 0x7fce440af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.697+0000 7fce437fe640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce4406ad80 0x7fce4415ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.697+0000 7fce43fff640 1 -- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce440b2660 msgr2=0x7fce440af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.697+0000 7fce43fff640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce440b2660 0x7fce440af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.697+0000 7fce43fff640 1 -- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce4406ad80 msgr2=0x7fce4415ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.697+0000 7fce43fff640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce4406ad80 0x7fce4415ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.697+0000 7fce43fff640 1 -- 172.21.15.27:0/3413556426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce340094d0 con 0x7fce4415ce80 2023-12-19T18:18:03.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.697+0000 7fce437fe640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce4406ad80 0x7fce4415ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:03.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.697+0000 7fce43fff640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4415ce80 0x7fce440c6270 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00b970 tx=0x7fce3c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:03.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.698+0000 7fce40ff9640 1 -- 172.21.15.27:0/3413556426 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c00cb40 con 0x7fce4415ce80 2023-12-19T18:18:03.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.698+0000 7fce40ff9640 1 -- 172.21.15.27:0/3413556426 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c00ccc0 con 0x7fce4415ce80 2023-12-19T18:18:03.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.698+0000 7fce40ff9640 1 -- 172.21.15.27:0/3413556426 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c014680 con 0x7fce4415ce80 2023-12-19T18:18:03.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.698+0000 7fce489df640 1 -- 172.21.15.27:0/3413556426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4415f270 con 0x7fce4415ce80 2023-12-19T18:18:03.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.698+0000 7fce489df640 1 -- 172.21.15.27:0/3413556426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4415f830 con 0x7fce4415ce80 2023-12-19T18:18:03.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.699+0000 7fce489df640 1 -- 172.21.15.27:0/3413556426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4415d7c0 con 0x7fce4415ce80 2023-12-19T18:18:03.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.700+0000 7fce40ff9640 1 -- 172.21.15.27:0/3413556426 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fce3c01d3b0 con 0x7fce4415ce80 2023-12-19T18:18:03.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.701+0000 7fce40ff9640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce2406a430 0x7fce2406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:03.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.701+0000 7fce40ff9640 1 -- 172.21.15.27:0/3413556426 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fce3c088970 con 0x7fce4415ce80 2023-12-19T18:18:03.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.702+0000 7fce437fe640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce2406a430 0x7fce2406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:03.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.703+0000 7fce437fe640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce2406a430 0x7fce2406c8f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce34036000 tx=0x7fce3403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:03.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.706+0000 7fce40ff9640 1 -- 172.21.15.27:0/3413556426 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fce4415d7c0 con 0x7fce4415ce80 2023-12-19T18:18:03.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.895+0000 7fce489df640 1 -- 172.21.15.27:0/3413556426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce4415d7c0 con 0x7fce4415ce80 2023-12-19T18:18:03.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.896+0000 7fce40ff9640 1 -- 172.21.15.27:0/3413556426 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7fce4415d7c0 con 0x7fce4415ce80 2023-12-19T18:18:03.897 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:18:03.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.898+0000 7fce489df640 1 -- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce2406a430 msgr2=0x7fce2406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.898+0000 7fce489df640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce2406a430 0x7fce2406c8f0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce34036000 tx=0x7fce3403c040 comp rx=0 tx=0).stop 2023-12-19T18:18:03.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.898+0000 7fce489df640 1 -- 172.21.15.27:0/3413556426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4415ce80 msgr2=0x7fce440c6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:03.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.898+0000 7fce489df640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4415ce80 0x7fce440c6270 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fce3c00b970 tx=0x7fce3c00be30 comp rx=0 tx=0).stop 2023-12-19T18:18:03.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.899+0000 7fce489df640 1 -- 172.21.15.27:0/3413556426 shutdown_connections 2023-12-19T18:18:03.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.899+0000 7fce489df640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce2406a430 0x7fce2406c8f0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.899+0000 7fce489df640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4415ce80 0x7fce440c6270 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.899+0000 7fce489df640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce440b2660 0x7fce440af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.899+0000 7fce489df640 1 --2- 172.21.15.27:0/3413556426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce4406ad80 0x7fce4415ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:03.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.899+0000 7fce489df640 1 -- 172.21.15.27:0/3413556426 >> 172.21.15.27:0/3413556426 conn(0x7fce4406f120 msgr2=0x7fce440b3840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:03.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.899+0000 7fce489df640 1 -- 172.21.15.27:0/3413556426 shutdown_connections 2023-12-19T18:18:03.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:03.899+0000 7fce489df640 1 -- 172.21.15.27:0/3413556426 wait complete. 2023-12-19T18:18:03.916 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:18:04.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.150+0000 7fde44228640 1 Processor -- start 2023-12-19T18:18:04.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.150+0000 7fde44228640 1 -- start start 2023-12-19T18:18:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.152+0000 7fde44228640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fde3c06fc10 0x7fde3c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.152+0000 7fde44228640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c070c00 0x7fde3c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.152+0000 7fde44228640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3c071820 con 0x7fde3c15adf0 2023-12-19T18:18:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.152+0000 7fde44228640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde3c0703c0 con 0x7fde3c06fc10 2023-12-19T18:18:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.152+0000 7fde44228640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c06d8c0 con 0x7fde3c070c00 2023-12-19T18:18:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.152+0000 7fde41fc4640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fde3c15adf0 0x7fde3c06f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33558/0 (socket says 172.21.15.27:33558) 2023-12-19T18:18:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.152+0000 7fde41fc4640 1 -- 172.21.15.27:0/2000323632 learned_addr learned my addr 172.21.15.27:0/2000323632 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.152+0000 7fde417c3640 1 --2- 172.21.15.27:0/2000323632 >> v2:172.21.15.27:3300/0 conn(0x7fde3c06fc10 0x7fde3c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.152+0000 7fde427c5640 1 --2- 172.21.15.27:0/2000323632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c070c00 0x7fde3c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.152+0000 7fde417c3640 1 -- 172.21.15.27:0/2000323632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c070c00 msgr2=0x7fde3c0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.152+0000 7fde40fc2640 1 -- 172.21.15.27:0/2000323632 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948560470 0 0) 0x7fde3c071820 con 0x7fde3c15adf0 2023-12-19T18:18:04.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.153+0000 7fde417c3640 1 --2- 172.21.15.27:0/2000323632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c070c00 0x7fde3c0712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.153+0000 7fde417c3640 1 -- 172.21.15.27:0/2000323632 >> v1:172.21.15.27:6789/0 conn(0x7fde3c15adf0 legacy=0x7fde3c06f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.153+0000 7fde417c3640 1 -- 172.21.15.27:0/2000323632 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c06b520 con 0x7fde3c06fc10 2023-12-19T18:18:04.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.154+0000 7fde417c3640 1 --2- 172.21.15.27:0/2000323632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c06fc10 0x7fde3c06d380 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0097a0 tx=0x7fde2c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21936a29a3f0923c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.154+0000 7fde40fc2640 1 -- 172.21.15.27:0/2000323632 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c02e070 con 0x7fde3c06fc10 2023-12-19T18:18:04.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.154+0000 7fde40fc2640 1 -- 172.21.15.27:0/2000323632 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c0315a0 con 0x7fde3c06fc10 2023-12-19T18:18:04.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.154+0000 7fde40fc2640 1 -- 172.21.15.27:0/2000323632 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c031930 con 0x7fde3c06fc10 2023-12-19T18:18:04.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.155+0000 7fde44228640 1 -- 172.21.15.27:0/2000323632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c06fc10 msgr2=0x7fde3c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.155+0000 7fde44228640 1 --2- 172.21.15.27:0/2000323632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c06fc10 0x7fde3c06d380 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fde2c0097a0 tx=0x7fde2c02b840 comp rx=0 tx=0).stop 2023-12-19T18:18:04.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.155+0000 7fde44228640 1 -- 172.21.15.27:0/2000323632 shutdown_connections 2023-12-19T18:18:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.155+0000 7fde44228640 1 --2- 172.21.15.27:0/2000323632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c070c00 0x7fde3c0712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.155+0000 7fde44228640 1 --2- 172.21.15.27:0/2000323632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c06fc10 0x7fde3c06d380 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.155+0000 7fde44228640 1 -- 172.21.15.27:0/2000323632 >> 172.21.15.27:0/2000323632 conn(0x7fde3c0ceb50 msgr2=0x7fde3c0cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.155+0000 7fde44228640 1 -- 172.21.15.27:0/2000323632 shutdown_connections 2023-12-19T18:18:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.155+0000 7fde44228640 1 -- 172.21.15.27:0/2000323632 wait complete. 2023-12-19T18:18:04.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.157+0000 7fde44228640 1 Processor -- start 2023-12-19T18:18:04.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.157+0000 7fde44228640 1 -- start start 2023-12-19T18:18:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.158+0000 7fde44228640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c06fc10 0x7fde3c0c5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.158+0000 7fde44228640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c070c00 0x7fde3c0b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.158+0000 7fde44228640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde3c15adf0 0x7fde3c0b4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.158+0000 7fde44228640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c1584d0 con 0x7fde3c070c00 2023-12-19T18:18:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.158+0000 7fde44228640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c158720 con 0x7fde3c15adf0 2023-12-19T18:18:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.158+0000 7fde44228640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c158a00 con 0x7fde3c06fc10 2023-12-19T18:18:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.158+0000 7fde417c3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c070c00 0x7fde3c0b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.158+0000 7fde417c3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c070c00 0x7fde3c0b28d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58012/0 (socket says 172.21.15.27:58012) 2023-12-19T18:18:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.158+0000 7fde417c3640 1 -- 172.21.15.27:0/348884115 learned_addr learned my addr 172.21.15.27:0/348884115 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.158+0000 7fde41fc4640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c06fc10 0x7fde3c0c5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.158+0000 7fde417c3640 1 -- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c06fc10 msgr2=0x7fde3c0c5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.158+0000 7fde417c3640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c06fc10 0x7fde3c0c5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.159+0000 7fde417c3640 1 -- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde3c15adf0 msgr2=0x7fde3c0b4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.159+0000 7fde417c3640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde3c15adf0 0x7fde3c0b4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.159+0000 7fde417c3640 1 -- 172.21.15.27:0/348884115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c009400 con 0x7fde3c070c00 2023-12-19T18:18:04.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.159+0000 7fde41fc4640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c06fc10 0x7fde3c0c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:04.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.159+0000 7fde417c3640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c070c00 0x7fde3c0b28d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fde2c034040 tx=0x7fde2c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.159+0000 7fde327fc640 1 -- 172.21.15.27:0/348884115 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c032900 con 0x7fde3c070c00 2023-12-19T18:18:04.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.159+0000 7fde327fc640 1 -- 172.21.15.27:0/348884115 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c032a80 con 0x7fde3c070c00 2023-12-19T18:18:04.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.159+0000 7fde44228640 1 -- 172.21.15.27:0/348884115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3c158ce0 con 0x7fde3c070c00 2023-12-19T18:18:04.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.159+0000 7fde327fc640 1 -- 172.21.15.27:0/348884115 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c032e30 con 0x7fde3c070c00 2023-12-19T18:18:04.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.159+0000 7fde44228640 1 -- 172.21.15.27:0/348884115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3c0b46a0 con 0x7fde3c070c00 2023-12-19T18:18:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.160+0000 7fde44228640 1 -- 172.21.15.27:0/348884115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3c159200 con 0x7fde3c070c00 2023-12-19T18:18:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.161+0000 7fde327fc640 1 -- 172.21.15.27:0/348884115 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fde2c04e430 con 0x7fde3c070c00 2023-12-19T18:18:04.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.163+0000 7fde327fc640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde2006a430 0x7fde2006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.163+0000 7fde327fc640 1 -- 172.21.15.27:0/348884115 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fde2c0aaac0 con 0x7fde3c070c00 2023-12-19T18:18:04.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.163+0000 7fde41fc4640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde2006a430 0x7fde2006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.164+0000 7fde41fc4640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde2006a430 0x7fde2006c8f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fde38000f60 tx=0x7fde38009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.166+0000 7fde327fc640 1 -- 172.21.15.27:0/348884115 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fde3c159200 con 0x7fde3c070c00 2023-12-19T18:18:04.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.368+0000 7fde44228640 1 -- 172.21.15.27:0/348884115 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fde3c159200 con 0x7fde3c070c00 2023-12-19T18:18:04.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.369+0000 7fde327fc640 1 -- 172.21.15.27:0/348884115 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) v1 ==== 80+0+0 (secure 0 0 0) 0x7fde3c159200 con 0x7fde3c070c00 2023-12-19T18:18:04.369 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:18:04.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.372+0000 7fde44228640 1 -- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde2006a430 msgr2=0x7fde2006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.372+0000 7fde44228640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde2006a430 0x7fde2006c8f0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fde38000f60 tx=0x7fde38009040 comp rx=0 tx=0).stop 2023-12-19T18:18:04.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.372+0000 7fde44228640 1 -- 172.21.15.27:0/348884115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c070c00 msgr2=0x7fde3c0b28d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.372+0000 7fde44228640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c070c00 0x7fde3c0b28d0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fde2c034040 tx=0x7fde2c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.372+0000 7fde44228640 1 -- 172.21.15.27:0/348884115 shutdown_connections 2023-12-19T18:18:04.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.372+0000 7fde44228640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde2006a430 0x7fde2006c8f0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.372+0000 7fde44228640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde3c15adf0 0x7fde3c0b4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.372+0000 7fde44228640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde3c070c00 0x7fde3c0b28d0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.372+0000 7fde44228640 1 --2- 172.21.15.27:0/348884115 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde3c06fc10 0x7fde3c0c5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.372+0000 7fde44228640 1 -- 172.21.15.27:0/348884115 >> 172.21.15.27:0/348884115 conn(0x7fde3c0ceb50 msgr2=0x7fde3c0cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:04.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.372+0000 7fde44228640 1 -- 172.21.15.27:0/348884115 shutdown_connections 2023-12-19T18:18:04.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.372+0000 7fde44228640 1 -- 172.21.15.27:0/348884115 wait complete. 2023-12-19T18:18:04.388 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-12-19T18:18:04.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:04.389 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-19T18:18:04.497 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[96772]: fuse finished with error 0 and tester_r 0 2023-12-19T18:18:04.500 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:18:04.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:04.500 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:18:04.573 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:18:04.574 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:04.616 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:04.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.813+0000 7f9cea4bd640 1 Processor -- start 2023-12-19T18:18:04.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.813+0000 7f9cea4bd640 1 -- start start 2023-12-19T18:18:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.815+0000 7f9cea4bd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9ce415af70 0x7f9ce40ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.815+0000 7f9cea4bd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ce40adc20 0x7f9ce406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.815+0000 7f9cea4bd640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ce40cb850 con 0x7f9ce40cbce0 2023-12-19T18:18:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.815+0000 7f9cea4bd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ce415b350 con 0x7f9ce415af70 2023-12-19T18:18:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.815+0000 7f9cea4bd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce40cac40 con 0x7f9ce40adc20 2023-12-19T18:18:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.815+0000 7f9ce3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9ce415af70 0x7f9ce40ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.816+0000 7f9ce3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9ce415af70 0x7f9ce40ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58020/0 (socket says 172.21.15.27:58020) 2023-12-19T18:18:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.816+0000 7f9ce3fff640 1 -- 172.21.15.27:0/1754943990 learned_addr learned my addr 172.21.15.27:0/1754943990 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.816+0000 7f9ce3fff640 1 -- 172.21.15.27:0/1754943990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ce40adc20 msgr2=0x7f9ce406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.816+0000 7f9ce3fff640 1 --2- 172.21.15.27:0/1754943990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ce40adc20 0x7f9ce406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.816+0000 7f9ce3fff640 1 -- 172.21.15.27:0/1754943990 >> v1:172.21.15.192:6789/0 conn(0x7f9ce40cbce0 legacy=0x7f9ce40ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:04.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.816+0000 7f9ce3fff640 1 -- 172.21.15.27:0/1754943990 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ce40caf20 con 0x7f9ce415af70 2023-12-19T18:18:04.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.817+0000 7f9ce3fff640 1 --2- 172.21.15.27:0/1754943990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce415af70 0x7f9ce40ca540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009870 tx=0x7f9cd402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f65560e66547c4f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.817+0000 7f9ce37fe640 1 -- 172.21.15.27:0/1754943990 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4031640 con 0x7f9ce415af70 2023-12-19T18:18:04.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.817+0000 7f9ce37fe640 1 -- 172.21.15.27:0/1754943990 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cd40317a0 con 0x7f9ce415af70 2023-12-19T18:18:04.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.817+0000 7f9ce37fe640 1 -- 172.21.15.27:0/1754943990 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cd4031b30 con 0x7f9ce415af70 2023-12-19T18:18:04.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.818+0000 7f9cea4bd640 1 -- 172.21.15.27:0/1754943990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce415af70 msgr2=0x7f9ce40ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.818+0000 7f9cea4bd640 1 --2- 172.21.15.27:0/1754943990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce415af70 0x7f9ce40ca540 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4009870 tx=0x7f9cd402b8f0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.818+0000 7f9cea4bd640 1 -- 172.21.15.27:0/1754943990 shutdown_connections 2023-12-19T18:18:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.818+0000 7f9cea4bd640 1 --2- 172.21.15.27:0/1754943990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ce40adc20 0x7f9ce406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.818+0000 7f9cea4bd640 1 --2- 172.21.15.27:0/1754943990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce415af70 0x7f9ce40ca540 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.818+0000 7f9cea4bd640 1 -- 172.21.15.27:0/1754943990 >> 172.21.15.27:0/1754943990 conn(0x7f9ce406a330 msgr2=0x7f9ce40b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.818+0000 7f9cea4bd640 1 -- 172.21.15.27:0/1754943990 shutdown_connections 2023-12-19T18:18:04.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.818+0000 7f9cea4bd640 1 -- 172.21.15.27:0/1754943990 wait complete. 2023-12-19T18:18:04.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.820+0000 7f9cea4bd640 1 Processor -- start 2023-12-19T18:18:04.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.820+0000 7f9cea4bd640 1 -- start start 2023-12-19T18:18:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.821+0000 7f9cea4bd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ce40adc20 0x7f9ce41f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.821+0000 7f9cea4bd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce40cbce0 0x7f9ce4156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.821+0000 7f9cea4bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce415af70 0x7f9ce40c2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.821+0000 7f9cea4bd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce41f63f0 con 0x7f9ce415af70 2023-12-19T18:18:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.821+0000 7f9cea4bd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce4156a90 con 0x7f9ce40cbce0 2023-12-19T18:18:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.821+0000 7f9cea4bd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ce41f6690 con 0x7f9ce40adc20 2023-12-19T18:18:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.821+0000 7f9ce8a5a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce415af70 0x7f9ce40c2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.821+0000 7f9ce8a5a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce415af70 0x7f9ce40c2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58026/0 (socket says 172.21.15.27:58026) 2023-12-19T18:18:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.821+0000 7f9ce8a5a640 1 -- 172.21.15.27:0/376492507 learned_addr learned my addr 172.21.15.27:0/376492507 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.821+0000 7f9ce3fff640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ce40adc20 0x7f9ce41f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.822+0000 7f9cdbfff640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce40cbce0 0x7f9ce4156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.822+0000 7f9ce8a5a640 1 -- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ce40adc20 msgr2=0x7f9ce41f5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.822+0000 7f9ce8a5a640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ce40adc20 0x7f9ce41f5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.822+0000 7f9ce8a5a640 1 -- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce40cbce0 msgr2=0x7f9ce4156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:04.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.822+0000 7f9ce8a5a640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce40cbce0 0x7f9ce4156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:04.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.822+0000 7f9ce8a5a640 1 -- 172.21.15.27:0/376492507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd40094d0 con 0x7f9ce415af70 2023-12-19T18:18:04.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.822+0000 7f9ce8a5a640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce415af70 0x7f9ce40c2650 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00b970 tx=0x7f9cdc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.822+0000 7f9ce17fa640 1 -- 172.21.15.27:0/376492507 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc00cb40 con 0x7f9ce415af70 2023-12-19T18:18:04.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.822+0000 7f9ce17fa640 1 -- 172.21.15.27:0/376492507 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc00ccc0 con 0x7f9ce415af70 2023-12-19T18:18:04.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.822+0000 7f9ce17fa640 1 -- 172.21.15.27:0/376492507 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc014680 con 0x7f9ce415af70 2023-12-19T18:18:04.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.822+0000 7f9cea4bd640 1 -- 172.21.15.27:0/376492507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ce41f6970 con 0x7f9ce415af70 2023-12-19T18:18:04.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.823+0000 7f9cea4bd640 1 -- 172.21.15.27:0/376492507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ce4156d50 con 0x7f9ce415af70 2023-12-19T18:18:04.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.824+0000 7f9cea4bd640 1 -- 172.21.15.27:0/376492507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ce4068e90 con 0x7f9ce415af70 2023-12-19T18:18:04.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.825+0000 7f9ce17fa640 1 -- 172.21.15.27:0/376492507 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9cdc01d3b0 con 0x7f9ce415af70 2023-12-19T18:18:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.825+0000 7f9ce17fa640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cc406a430 0x7f9cc406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:04.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.826+0000 7f9ce17fa640 1 -- 172.21.15.27:0/376492507 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9cdc0889e0 con 0x7f9ce415af70 2023-12-19T18:18:04.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.826+0000 7f9ce3fff640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cc406a430 0x7f9cc406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:04.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.827+0000 7f9ce3fff640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cc406a430 0x7f9cc406c8f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4036000 tx=0x7f9cd403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:04.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:04.830+0000 7f9ce17fa640 1 -- 172.21.15.27:0/376492507 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9cdc053ea0 con 0x7f9ce415af70 2023-12-19T18:18:05.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.019+0000 7f9cea4bd640 1 -- 172.21.15.27:0/376492507 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ce415a490 con 0x7f9ce415af70 2023-12-19T18:18:05.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.021+0000 7f9ce17fa640 1 -- 172.21.15.27:0/376492507 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f9cdc019020 con 0x7f9ce415af70 2023-12-19T18:18:05.021 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:05.021 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":63,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:17:56.605695+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:12.527441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:13.291755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.8","mappings":[{"from":5,"to":7}]},{"pgid":"7.9","mappings":[{"from":5,"to":7}]},{"pgid":"7.20","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:05.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.024+0000 7f9cea4bd640 1 -- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cc406a430 msgr2=0x7f9cc406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.024+0000 7f9cea4bd640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cc406a430 0x7f9cc406c8f0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9cd4036000 tx=0x7f9cd403c040 comp rx=0 tx=0).stop 2023-12-19T18:18:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.024+0000 7f9cea4bd640 1 -- 172.21.15.27:0/376492507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce415af70 msgr2=0x7f9ce40c2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.024+0000 7f9cea4bd640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce415af70 0x7f9ce40c2650 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc00b970 tx=0x7f9cdc00be30 comp rx=0 tx=0).stop 2023-12-19T18:18:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.024+0000 7f9cea4bd640 1 -- 172.21.15.27:0/376492507 shutdown_connections 2023-12-19T18:18:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.024+0000 7f9cea4bd640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cc406a430 0x7f9cc406c8f0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.024+0000 7f9cea4bd640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ce415af70 0x7f9ce40c2650 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.024+0000 7f9cea4bd640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ce40cbce0 0x7f9ce4156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.024+0000 7f9cea4bd640 1 --2- 172.21.15.27:0/376492507 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ce40adc20 0x7f9ce41f5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.024+0000 7f9cea4bd640 1 -- 172.21.15.27:0/376492507 >> 172.21.15.27:0/376492507 conn(0x7f9ce406a330 msgr2=0x7f9ce40c9170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.024+0000 7f9cea4bd640 1 -- 172.21.15.27:0/376492507 shutdown_connections 2023-12-19T18:18:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.024+0000 7f9cea4bd640 1 -- 172.21.15.27:0/376492507 wait complete. 2023-12-19T18:18:05.044 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:17:12.527441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:17:13.291755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:18:05.044 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:18:05.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.282+0000 7f18a6496640 1 Processor -- start 2023-12-19T18:18:05.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.282+0000 7f18a6496640 1 -- start start 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.284+0000 7f18a6496640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f18a015b1a0 0x7f18a006f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.284+0000 7f18a6496640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f18a00c2720 0x7f18a006fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.284+0000 7f18a6496640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18a0071420 0x7f18a006ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.284+0000 7f18a6496640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a006bc80 con 0x7f18a00c2720 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.284+0000 7f18a6496640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18a006d690 con 0x7f18a015b1a0 2023-12-19T18:18:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.284+0000 7f18a6496640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a006d0a0 con 0x7f18a0071420 2023-12-19T18:18:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.285+0000 7f1897fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f18a00c2720 0x7f18a006fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.285+0000 7f1897fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f18a00c2720 0x7f18a006fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58028/0 (socket says 172.21.15.27:58028) 2023-12-19T18:18:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.285+0000 7f1897fff640 1 -- 172.21.15.27:0/587663207 learned_addr learned my addr 172.21.15.27:0/587663207 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.285+0000 7f189ffff640 1 --2- 172.21.15.27:0/587663207 >> v2:172.21.15.192:3300/0 conn(0x7f18a015b1a0 0x7f18a006f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.285+0000 7f18a4a33640 1 --2- 172.21.15.27:0/587663207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18a0071420 0x7f18a006ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.285+0000 7f1897fff640 1 -- 172.21.15.27:0/587663207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18a0071420 msgr2=0x7f18a006ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.285+0000 7f1897fff640 1 --2- 172.21.15.27:0/587663207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18a0071420 0x7f18a006ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.285+0000 7f1897fff640 1 -- 172.21.15.27:0/587663207 >> v2:172.21.15.192:3300/0 conn(0x7f18a015b1a0 msgr2=0x7f18a006f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.285+0000 7f1897fff640 1 --2- 172.21.15.27:0/587663207 >> v2:172.21.15.192:3300/0 conn(0x7f18a015b1a0 0x7f18a006f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.285+0000 7f1897fff640 1 -- 172.21.15.27:0/587663207 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a006b6c0 con 0x7f18a00c2720 2023-12-19T18:18:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.285+0000 7f189ffff640 1 --2- 172.21.15.27:0/587663207 >> v2:172.21.15.192:3300/0 conn(0x7f18a015b1a0 0x7f18a006f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.286+0000 7f1897fff640 1 --2- 172.21.15.27:0/587663207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a00c2720 0x7f18a006fa60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f18a00cea40 tx=0x7f188802b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5011a189aac366f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.287+0000 7f189f7fe640 1 -- 172.21.15.27:0/587663207 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18880317d0 con 0x7f18a00c2720 2023-12-19T18:18:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.287+0000 7f189f7fe640 1 -- 172.21.15.27:0/587663207 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1888031930 con 0x7f18a00c2720 2023-12-19T18:18:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.287+0000 7f189f7fe640 1 -- 172.21.15.27:0/587663207 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1888031cc0 con 0x7f18a00c2720 2023-12-19T18:18:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.287+0000 7f18a6496640 1 -- 172.21.15.27:0/587663207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a00c2720 msgr2=0x7f18a006fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.287+0000 7f18a6496640 1 --2- 172.21.15.27:0/587663207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a00c2720 0x7f18a006fa60 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f18a00cea40 tx=0x7f188802b870 comp rx=0 tx=0).stop 2023-12-19T18:18:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.288+0000 7f18a6496640 1 -- 172.21.15.27:0/587663207 shutdown_connections 2023-12-19T18:18:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.288+0000 7f18a6496640 1 --2- 172.21.15.27:0/587663207 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18a0071420 0x7f18a006ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.288+0000 7f18a6496640 1 --2- 172.21.15.27:0/587663207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a00c2720 0x7f18a006fa60 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.288+0000 7f18a6496640 1 --2- 172.21.15.27:0/587663207 >> v2:172.21.15.192:3300/0 conn(0x7f18a015b1a0 0x7f18a006f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.288+0000 7f18a6496640 1 -- 172.21.15.27:0/587663207 >> 172.21.15.27:0/587663207 conn(0x7f18a00cebd0 msgr2=0x7f18a00cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.288+0000 7f18a6496640 1 -- 172.21.15.27:0/587663207 shutdown_connections 2023-12-19T18:18:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.288+0000 7f18a6496640 1 -- 172.21.15.27:0/587663207 wait complete. 2023-12-19T18:18:05.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.289+0000 7f18a6496640 1 Processor -- start 2023-12-19T18:18:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.290+0000 7f18a6496640 1 -- start start 2023-12-19T18:18:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a6496640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18a0071420 0x7f18a0156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a6496640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18a00c2720 0x7f18a014cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a6496640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a015b1a0 0x7f18a00af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a6496640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a01574f0 con 0x7f18a015b1a0 2023-12-19T18:18:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a6496640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a014d460 con 0x7f18a0071420 2023-12-19T18:18:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a6496640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18a00c6080 con 0x7f18a00c2720 2023-12-19T18:18:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a4a33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a015b1a0 0x7f18a00af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a4a33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a015b1a0 0x7f18a00af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58040/0 (socket says 172.21.15.27:58040) 2023-12-19T18:18:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a4a33640 1 -- 172.21.15.27:0/1131001807 learned_addr learned my addr 172.21.15.27:0/1131001807 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f1897fff640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18a00c2720 0x7f18a014cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f189ffff640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18a0071420 0x7f18a0156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a4a33640 1 -- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18a00c2720 msgr2=0x7f18a014cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a4a33640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18a00c2720 0x7f18a014cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a4a33640 1 -- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18a0071420 msgr2=0x7f18a0156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a4a33640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18a0071420 0x7f18a0156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.291+0000 7f18a4a33640 1 -- 172.21.15.27:0/1131001807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18880094d0 con 0x7f18a015b1a0 2023-12-19T18:18:05.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.292+0000 7f18a4a33640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a015b1a0 0x7f18a00af9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1898007d10 tx=0x7f189800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.292+0000 7f189d7fa640 1 -- 172.21.15.27:0/1131001807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898014420 con 0x7f18a015b1a0 2023-12-19T18:18:05.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.292+0000 7f189d7fa640 1 -- 172.21.15.27:0/1131001807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1898014580 con 0x7f18a015b1a0 2023-12-19T18:18:05.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.292+0000 7f189d7fa640 1 -- 172.21.15.27:0/1131001807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1898014930 con 0x7f18a015b1a0 2023-12-19T18:18:05.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.292+0000 7f18a6496640 1 -- 172.21.15.27:0/1131001807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a00c6360 con 0x7f18a015b1a0 2023-12-19T18:18:05.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.292+0000 7f18a6496640 1 -- 172.21.15.27:0/1131001807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a00c6920 con 0x7f18a015b1a0 2023-12-19T18:18:05.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.294+0000 7f18a6496640 1 -- 172.21.15.27:0/1131001807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a0068e90 con 0x7f18a015b1a0 2023-12-19T18:18:05.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.294+0000 7f189d7fa640 1 -- 172.21.15.27:0/1131001807 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1898014a90 con 0x7f18a015b1a0 2023-12-19T18:18:05.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.295+0000 7f189d7fa640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f188006a430 0x7f188006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.295+0000 7f189d7fa640 1 -- 172.21.15.27:0/1131001807 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1898089210 con 0x7f18a015b1a0 2023-12-19T18:18:05.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.296+0000 7f189ffff640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f188006a430 0x7f188006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.297+0000 7f189ffff640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f188006a430 0x7f188006c8f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1890006fd0 tx=0x7f1890008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.300+0000 7f189d7fa640 1 -- 172.21.15.27:0/1131001807 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1898054770 con 0x7f18a015b1a0 2023-12-19T18:18:05.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.487+0000 7f18a6496640 1 -- 172.21.15.27:0/1131001807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f18a00cc4c0 con 0x7f18a015b1a0 2023-12-19T18:18:05.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.488+0000 7f189d7fa640 1 -- 172.21.15.27:0/1131001807 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18980098c0 con 0x7f18a015b1a0 2023-12-19T18:18:05.488 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:18:05.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.490+0000 7f18a6496640 1 -- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f188006a430 msgr2=0x7f188006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.490+0000 7f18a6496640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f188006a430 0x7f188006c8f0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1890006fd0 tx=0x7f1890008040 comp rx=0 tx=0).stop 2023-12-19T18:18:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.490+0000 7f18a6496640 1 -- 172.21.15.27:0/1131001807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a015b1a0 msgr2=0x7f18a00af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.490+0000 7f18a6496640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a015b1a0 0x7f18a00af9c0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1898007d10 tx=0x7f189800e4b0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.490+0000 7f18a6496640 1 -- 172.21.15.27:0/1131001807 shutdown_connections 2023-12-19T18:18:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.490+0000 7f18a6496640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f188006a430 0x7f188006c8f0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.490+0000 7f18a6496640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a015b1a0 0x7f18a00af9c0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.490+0000 7f18a6496640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f18a00c2720 0x7f18a014cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.490+0000 7f18a6496640 1 --2- 172.21.15.27:0/1131001807 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f18a0071420 0x7f18a0156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.490+0000 7f18a6496640 1 -- 172.21.15.27:0/1131001807 >> 172.21.15.27:0/1131001807 conn(0x7f18a00cebd0 msgr2=0x7f18a0153c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.491+0000 7f18a6496640 1 -- 172.21.15.27:0/1131001807 shutdown_connections 2023-12-19T18:18:05.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.491+0000 7f18a6496640 1 -- 172.21.15.27:0/1131001807 wait complete. 2023-12-19T18:18:05.508 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:18:05.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.744+0000 7f3232dcb640 1 Processor -- start 2023-12-19T18:18:05.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.744+0000 7f3232dcb640 1 -- start start 2023-12-19T18:18:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.745+0000 7f3232dcb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f322c06d5f0 0x7f322c0cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.745+0000 7f3232dcb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c0c9880 con 0x7f322c0ce210 2023-12-19T18:18:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.745+0000 7f3232dcb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f322c0c9b20 con 0x7f322c15a0b0 2023-12-19T18:18:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.745+0000 7f3232dcb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f322c0c9de0 con 0x7f322c06d5f0 2023-12-19T18:18:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.746+0000 7f3231368640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f322c06d5f0 0x7f322c0cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.746+0000 7f3231368640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f322c06d5f0 0x7f322c0cdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58054/0 (socket says 172.21.15.27:58054) 2023-12-19T18:18:05.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.746+0000 7f3231368640 1 -- 172.21.15.27:0/3137483980 learned_addr learned my addr 172.21.15.27:0/3137483980 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:05.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.746+0000 7f3231368640 1 -- 172.21.15.27:0/3137483980 >> v1:172.21.15.192:6789/0 conn(0x7f322c15a0b0 legacy=0x7f322c0cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:05.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.746+0000 7f3231368640 1 -- 172.21.15.27:0/3137483980 >> v1:172.21.15.27:6789/0 conn(0x7f322c0ce210 legacy=0x7f322c06cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.746+0000 7f3231368640 1 -- 172.21.15.27:0/3137483980 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f322c0cc920 con 0x7f322c06d5f0 2023-12-19T18:18:05.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.747+0000 7f3231368640 1 --2- 172.21.15.27:0/3137483980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f322c06d5f0 0x7f322c0cdc80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3224009870 tx=0x7f322402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8b52022b3d04fd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.748+0000 7f322b7fe640 1 -- 172.21.15.27:0/3137483980 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224030860 con 0x7f322c06d5f0 2023-12-19T18:18:05.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.748+0000 7f322b7fe640 1 -- 172.21.15.27:0/3137483980 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32240309c0 con 0x7f322c06d5f0 2023-12-19T18:18:05.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.748+0000 7f322b7fe640 1 -- 172.21.15.27:0/3137483980 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224030d50 con 0x7f322c06d5f0 2023-12-19T18:18:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.748+0000 7f3232dcb640 1 -- 172.21.15.27:0/3137483980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f322c06d5f0 msgr2=0x7f322c0cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.748+0000 7f3232dcb640 1 --2- 172.21.15.27:0/3137483980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f322c06d5f0 0x7f322c0cdc80 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f3224009870 tx=0x7f322402b9a0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.748+0000 7f3232dcb640 1 -- 172.21.15.27:0/3137483980 shutdown_connections 2023-12-19T18:18:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.748+0000 7f3232dcb640 1 --2- 172.21.15.27:0/3137483980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f322c06d5f0 0x7f322c0cdc80 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.748+0000 7f3232dcb640 1 -- 172.21.15.27:0/3137483980 >> 172.21.15.27:0/3137483980 conn(0x7f322c0cebd0 msgr2=0x7f322c0cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.749+0000 7f3232dcb640 1 -- 172.21.15.27:0/3137483980 shutdown_connections 2023-12-19T18:18:05.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.749+0000 7f3232dcb640 1 -- 172.21.15.27:0/3137483980 wait complete. 2023-12-19T18:18:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.750+0000 7f3232dcb640 1 Processor -- start 2023-12-19T18:18:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.750+0000 7f3232dcb640 1 -- start start 2023-12-19T18:18:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.751+0000 7f3232dcb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f322c06d5f0 0x7f322c145dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.751+0000 7f3232dcb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f322c0ce210 0x7f322c0b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.751+0000 7f3232dcb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f322c15a0b0 0x7f322c0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.751+0000 7f3232dcb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b48a0 con 0x7f322c15a0b0 2023-12-19T18:18:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.751+0000 7f3232dcb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b4320 con 0x7f322c0ce210 2023-12-19T18:18:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.751+0000 7f3232dcb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f322c0b45c0 con 0x7f322c06d5f0 2023-12-19T18:18:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.752+0000 7f3231368640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f322c15a0b0 0x7f322c0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.752+0000 7f3231368640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f322c15a0b0 0x7f322c0b3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58056/0 (socket says 172.21.15.27:58056) 2023-12-19T18:18:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.752+0000 7f3231368640 1 -- 172.21.15.27:0/835126655 learned_addr learned my addr 172.21.15.27:0/835126655 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.752+0000 7f322bfff640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f322c0ce210 0x7f322c0b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.752+0000 7f3230b67640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f322c06d5f0 0x7f322c145dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.752+0000 7f3231368640 1 -- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f322c06d5f0 msgr2=0x7f322c145dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.752+0000 7f3231368640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f322c06d5f0 0x7f322c145dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.752+0000 7f3231368640 1 -- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f322c0ce210 msgr2=0x7f322c0b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.752+0000 7f3231368640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f322c0ce210 0x7f322c0b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.752+0000 7f3231368640 1 -- 172.21.15.27:0/835126655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32240094d0 con 0x7f322c15a0b0 2023-12-19T18:18:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.752+0000 7f322bfff640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f322c0ce210 0x7f322c0b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.752+0000 7f3230b67640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f322c06d5f0 0x7f322c145dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.753+0000 7f3231368640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f322c15a0b0 0x7f322c0b3de0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f322402b9e0 tx=0x7f3224004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.753+0000 7f32297fa640 1 -- 172.21.15.27:0/835126655 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3224039d90 con 0x7f322c15a0b0 2023-12-19T18:18:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.753+0000 7f32297fa640 1 -- 172.21.15.27:0/835126655 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32240433f0 con 0x7f322c15a0b0 2023-12-19T18:18:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.753+0000 7f32297fa640 1 -- 172.21.15.27:0/835126655 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32240437a0 con 0x7f322c15a0b0 2023-12-19T18:18:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.753+0000 7f3232dcb640 1 -- 172.21.15.27:0/835126655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322c0c6e20 con 0x7f322c15a0b0 2023-12-19T18:18:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.753+0000 7f3232dcb640 1 -- 172.21.15.27:0/835126655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322c0c73e0 con 0x7f322c15a0b0 2023-12-19T18:18:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.754+0000 7f3232dcb640 1 -- 172.21.15.27:0/835126655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f322c0c7620 con 0x7f322c15a0b0 2023-12-19T18:18:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.755+0000 7f32297fa640 1 -- 172.21.15.27:0/835126655 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f322402f3b0 con 0x7f322c15a0b0 2023-12-19T18:18:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.756+0000 7f32297fa640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f320c06a430 0x7f320c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.756+0000 7f32297fa640 1 -- 172.21.15.27:0/835126655 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f32240aadb0 con 0x7f322c15a0b0 2023-12-19T18:18:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.757+0000 7f3230b67640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f320c06a430 0x7f320c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.758+0000 7f3230b67640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f320c06a430 0x7f320c06c8f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f321c004520 tx=0x7f321c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:05.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.761+0000 7f32297fa640 1 -- 172.21.15.27:0/835126655 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f322c0c7620 con 0x7f322c15a0b0 2023-12-19T18:18:05.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.949+0000 7f3232dcb640 1 -- 172.21.15.27:0/835126655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f322c0c7620 con 0x7f322c15a0b0 2023-12-19T18:18:05.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.950+0000 7f32297fa640 1 -- 172.21.15.27:0/835126655 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7f322c0c7620 con 0x7f322c15a0b0 2023-12-19T18:18:05.950 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:18:05.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.953+0000 7f3232dcb640 1 -- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f320c06a430 msgr2=0x7f320c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.953+0000 7f3232dcb640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f320c06a430 0x7f320c06c8f0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f321c004520 tx=0x7f321c009040 comp rx=0 tx=0).stop 2023-12-19T18:18:05.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.953+0000 7f3232dcb640 1 -- 172.21.15.27:0/835126655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f322c15a0b0 msgr2=0x7f322c0b3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:05.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.953+0000 7f3232dcb640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f322c15a0b0 0x7f322c0b3de0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f322402b9e0 tx=0x7f3224004730 comp rx=0 tx=0).stop 2023-12-19T18:18:05.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.954+0000 7f3232dcb640 1 -- 172.21.15.27:0/835126655 shutdown_connections 2023-12-19T18:18:05.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.954+0000 7f3232dcb640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f320c06a430 0x7f320c06c8f0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.954+0000 7f3232dcb640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f322c15a0b0 0x7f322c0b3de0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.954+0000 7f3232dcb640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f322c0ce210 0x7f322c0b1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.954+0000 7f3232dcb640 1 --2- 172.21.15.27:0/835126655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f322c06d5f0 0x7f322c145dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:05.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.954+0000 7f3232dcb640 1 -- 172.21.15.27:0/835126655 >> 172.21.15.27:0/835126655 conn(0x7f322c0cebd0 msgr2=0x7f322c06fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.954+0000 7f3232dcb640 1 -- 172.21.15.27:0/835126655 shutdown_connections 2023-12-19T18:18:05.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:05.954+0000 7f3232dcb640 1 -- 172.21.15.27:0/835126655 wait complete. 2023-12-19T18:18:05.972 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:18:06.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.208+0000 7fdc97084640 1 Processor -- start 2023-12-19T18:18:06.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.208+0000 7fdc97084640 1 -- start start 2023-12-19T18:18:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.209+0000 7fdc97084640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdc900ce130 0x7fdc900cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.209+0000 7fdc97084640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc900ce540 con 0x7fdc900cf9b0 2023-12-19T18:18:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.209+0000 7fdc97084640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc900d26d0 con 0x7fdc900b8c30 2023-12-19T18:18:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.210+0000 7fdc97084640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc900cea80 con 0x7fdc900ce130 2023-12-19T18:18:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.210+0000 7fdc8ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdc900cf9b0 0x7fdc900cda20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33564/0 (socket says 172.21.15.27:33564) 2023-12-19T18:18:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.210+0000 7fdc8ffff640 1 -- 172.21.15.27:0/1041164427 learned_addr learned my addr 172.21.15.27:0/1041164427 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:06.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.210+0000 7fdc95621640 1 --2- 172.21.15.27:0/1041164427 >> v2:172.21.15.192:3300/0 conn(0x7fdc900ce130 0x7fdc900cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.210+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1350470140 0 0) 0x7fdc900ce540 con 0x7fdc900cf9b0 2023-12-19T18:18:06.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.210+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc7c003520 con 0x7fdc900cf9b0 2023-12-19T18:18:06.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 327328409 0 0) 0x7fdc7c003520 con 0x7fdc900cf9b0 2023-12-19T18:18:06.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 >> v2:172.21.15.192:3300/0 conn(0x7fdc900ce130 msgr2=0x7fdc900cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:06.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8f7fe640 1 --2- 172.21.15.27:0/1041164427 >> v2:172.21.15.192:3300/0 conn(0x7fdc900ce130 0x7fdc900cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 >> v1:172.21.15.192:6789/0 conn(0x7fdc900b8c30 legacy=0x7fdc900d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc900ce7a0 con 0x7fdc900cf9b0 2023-12-19T18:18:06.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fdc78001670 con 0x7fdc900cf9b0 2023-12-19T18:18:06.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 >> v1:172.21.15.27:6789/0 conn(0x7fdc900cf9b0 legacy=0x7fdc900cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8f7fe640 1 --2- 172.21.15.27:0/1041164427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc7c0039c0 0x7fdc7c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8f7fe640 1 --2- 172.21.15.27:0/1041164427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc7c0047c0 0x7fdc7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8f7fe640 1 --2- 172.21.15.27:0/1041164427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc7c0051d0 0x7fdc7c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc900cea80 con 0x7fdc7c0047c0 2023-12-19T18:18:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c00c120 con 0x7fdc7c0039c0 2023-12-19T18:18:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7c00c2e0 con 0x7fdc7c0051d0 2023-12-19T18:18:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc8ffff640 1 --2- 172.21.15.27:0/1041164427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc7c0047c0 0x7fdc7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.211+0000 7fdc94e20640 1 --2- 172.21.15.27:0/1041164427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc7c0039c0 0x7fdc7c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.212+0000 7fdc8ffff640 1 -- 172.21.15.27:0/1041164427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc7c0051d0 msgr2=0x7fdc7c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.212+0000 7fdc8ffff640 1 --2- 172.21.15.27:0/1041164427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc7c0051d0 0x7fdc7c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.212+0000 7fdc8ffff640 1 -- 172.21.15.27:0/1041164427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc7c0039c0 msgr2=0x7fdc7c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.212+0000 7fdc8ffff640 1 --2- 172.21.15.27:0/1041164427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc7c0039c0 0x7fdc7c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.212+0000 7fdc8ffff640 1 -- 172.21.15.27:0/1041164427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7c003520 con 0x7fdc7c0047c0 2023-12-19T18:18:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.212+0000 7fdc94e20640 1 --2- 172.21.15.27:0/1041164427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc7c0039c0 0x7fdc7c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:06.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.212+0000 7fdc95621640 1 -- 172.21.15.27:0/1041164427 reap_dead start 2023-12-19T18:18:06.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.213+0000 7fdc8ffff640 1 --2- 172.21.15.27:0/1041164427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc7c0047c0 0x7fdc7c004c40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdc78001eb0 tx=0x7fdc7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d247327db76e7a9b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.213+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc78005ad0 con 0x7fdc7c0047c0 2023-12-19T18:18:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.213+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc78005c30 con 0x7fdc7c0047c0 2023-12-19T18:18:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.213+0000 7fdc8f7fe640 1 -- 172.21.15.27:0/1041164427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7802c9d0 con 0x7fdc7c0047c0 2023-12-19T18:18:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.213+0000 7fdc97084640 1 -- 172.21.15.27:0/1041164427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc7c0047c0 msgr2=0x7fdc7c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.213+0000 7fdc97084640 1 --2- 172.21.15.27:0/1041164427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc7c0047c0 0x7fdc7c004c40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fdc78001eb0 tx=0x7fdc7c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.214+0000 7fdc97084640 1 -- 172.21.15.27:0/1041164427 shutdown_connections 2023-12-19T18:18:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.214+0000 7fdc97084640 1 --2- 172.21.15.27:0/1041164427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc7c0047c0 0x7fdc7c004c40 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.214+0000 7fdc97084640 1 -- 172.21.15.27:0/1041164427 >> 172.21.15.27:0/1041164427 conn(0x7fdc900d2fa0 msgr2=0x7fdc900d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.214+0000 7fdc97084640 1 -- 172.21.15.27:0/1041164427 shutdown_connections 2023-12-19T18:18:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.214+0000 7fdc97084640 1 -- 172.21.15.27:0/1041164427 wait complete. 2023-12-19T18:18:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.215+0000 7fdc97084640 1 Processor -- start 2023-12-19T18:18:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.216+0000 7fdc97084640 1 -- start start 2023-12-19T18:18:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.216+0000 7fdc97084640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc7c0047c0 0x7fdc900cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.216+0000 7fdc97084640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc901f6d60 0x7fdc900cbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc97084640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc90158d60 0x7fdc901591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc97084640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc900b4d80 con 0x7fdc90158d60 2023-12-19T18:18:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc97084640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc901f37f0 con 0x7fdc7c0047c0 2023-12-19T18:18:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc97084640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc901f6280 con 0x7fdc901f6d60 2023-12-19T18:18:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc95621640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc90158d60 0x7fdc901591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc95621640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc90158d60 0x7fdc901591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58074/0 (socket says 172.21.15.27:58074) 2023-12-19T18:18:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc95621640 1 -- 172.21.15.27:0/4267613345 learned_addr learned my addr 172.21.15.27:0/4267613345 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc94e20640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc7c0047c0 0x7fdc900cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc8ffff640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc901f6d60 0x7fdc900cbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc95621640 1 -- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc901f6d60 msgr2=0x7fdc900cbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc95621640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc901f6d60 0x7fdc900cbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc95621640 1 -- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc7c0047c0 msgr2=0x7fdc900cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc95621640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc7c0047c0 0x7fdc900cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc95621640 1 -- 172.21.15.27:0/4267613345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc901f6560 con 0x7fdc90158d60 2023-12-19T18:18:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc8ffff640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc901f6d60 0x7fdc900cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.217+0000 7fdc94e20640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc7c0047c0 0x7fdc900cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.218+0000 7fdc95621640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc90158d60 0x7fdc901591e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdc900b29b0 tx=0x7fdc8800ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.218+0000 7fdc8d7fa640 1 -- 172.21.15.27:0/4267613345 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8800c620 con 0x7fdc90158d60 2023-12-19T18:18:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.218+0000 7fdc8d7fa640 1 -- 172.21.15.27:0/4267613345 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc8800c780 con 0x7fdc90158d60 2023-12-19T18:18:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.218+0000 7fdc8d7fa640 1 -- 172.21.15.27:0/4267613345 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc8800cb70 con 0x7fdc90158d60 2023-12-19T18:18:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.218+0000 7fdc97084640 1 -- 172.21.15.27:0/4267613345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc901f6840 con 0x7fdc90158d60 2023-12-19T18:18:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.218+0000 7fdc97084640 1 -- 172.21.15.27:0/4267613345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc90159720 con 0x7fdc90158d60 2023-12-19T18:18:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.219+0000 7fdc97084640 1 -- 172.21.15.27:0/4267613345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc901f6a30 con 0x7fdc90158d60 2023-12-19T18:18:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.220+0000 7fdc8d7fa640 1 -- 172.21.15.27:0/4267613345 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdc8800ccd0 con 0x7fdc90158d60 2023-12-19T18:18:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.222+0000 7fdc8d7fa640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc7006a430 0x7fdc7006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.222+0000 7fdc8d7fa640 1 -- 172.21.15.27:0/4267613345 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fdc88088870 con 0x7fdc90158d60 2023-12-19T18:18:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.222+0000 7fdc94e20640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc7006a430 0x7fdc7006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.223+0000 7fdc94e20640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc7006a430 0x7fdc7006c8f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdc80007fd0 tx=0x7fdc80009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.226+0000 7fdc8d7fa640 1 -- 172.21.15.27:0/4267613345 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdc88053d30 con 0x7fdc90158d60 2023-12-19T18:18:06.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.420+0000 7fdc97084640 1 -- 172.21.15.27:0/4267613345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc90068e90 con 0x7fdc90158d60 2023-12-19T18:18:06.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.421+0000 7fdc8d7fa640 1 -- 172.21.15.27:0/4267613345 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc88057350 con 0x7fdc90158d60 2023-12-19T18:18:06.421 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:18:06.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.423+0000 7fdc97084640 1 -- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc7006a430 msgr2=0x7fdc7006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.423+0000 7fdc97084640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc7006a430 0x7fdc7006c8f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdc80007fd0 tx=0x7fdc80009040 comp rx=0 tx=0).stop 2023-12-19T18:18:06.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.424+0000 7fdc97084640 1 -- 172.21.15.27:0/4267613345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc90158d60 msgr2=0x7fdc901591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.424+0000 7fdc97084640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc90158d60 0x7fdc901591e0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fdc900b29b0 tx=0x7fdc8800ec00 comp rx=0 tx=0).stop 2023-12-19T18:18:06.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.424+0000 7fdc97084640 1 -- 172.21.15.27:0/4267613345 shutdown_connections 2023-12-19T18:18:06.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.424+0000 7fdc97084640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc7006a430 0x7fdc7006c8f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.424+0000 7fdc97084640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc90158d60 0x7fdc901591e0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.424+0000 7fdc97084640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc901f6d60 0x7fdc900cbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.424+0000 7fdc97084640 1 --2- 172.21.15.27:0/4267613345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc7c0047c0 0x7fdc900cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.424+0000 7fdc97084640 1 -- 172.21.15.27:0/4267613345 >> 172.21.15.27:0/4267613345 conn(0x7fdc900d2fa0 msgr2=0x7fdc900cc580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:06.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.424+0000 7fdc97084640 1 -- 172.21.15.27:0/4267613345 shutdown_connections 2023-12-19T18:18:06.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.424+0000 7fdc97084640 1 -- 172.21.15.27:0/4267613345 wait complete. 2023-12-19T18:18:06.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:18:06.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.682+0000 7f9ec0684640 1 Processor -- start 2023-12-19T18:18:06.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.683+0000 7f9ec0684640 1 -- start start 2023-12-19T18:18:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.684+0000 7f9ec0684640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9eb815adf0 0x7f9eb806d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.684+0000 7f9ec0684640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb806b520 con 0x7f9eb806fc10 2023-12-19T18:18:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.684+0000 7f9ec0684640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9eb815c1e0 con 0x7f9eb8070c00 2023-12-19T18:18:06.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.684+0000 7f9ec0684640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9eb80703c0 con 0x7f9eb815adf0 2023-12-19T18:18:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.684+0000 7f9ebec21640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9eb806fc10 0x7f9eb80c91a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33576/0 (socket says 172.21.15.27:33576) 2023-12-19T18:18:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.684+0000 7f9ebec21640 1 -- 172.21.15.27:0/2041447012 learned_addr learned my addr 172.21.15.27:0/2041447012 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.685+0000 7f9ebe420640 1 --2- 172.21.15.27:0/2041447012 >> v2:172.21.15.192:3300/0 conn(0x7f9eb815adf0 0x7f9eb806d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.685+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1752286882 0 0) 0x7f9eb806b520 con 0x7f9eb806fc10 2023-12-19T18:18:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.685+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ea4003520 con 0x7f9eb806fc10 2023-12-19T18:18:06.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.685+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2712851757 0 0) 0x7f9ea4003520 con 0x7f9eb806fc10 2023-12-19T18:18:06.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.685+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 >> v2:172.21.15.192:3300/0 conn(0x7f9eb815adf0 msgr2=0x7f9eb806d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:06.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.685+0000 7f9ebd41e640 1 --2- 172.21.15.27:0/2041447012 >> v2:172.21.15.192:3300/0 conn(0x7f9eb815adf0 0x7f9eb806d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.685+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 >> v1:172.21.15.192:6789/0 conn(0x7f9eb8070c00 legacy=0x7f9eb806f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.685+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb815f3a0 con 0x7f9eb806fc10 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f9eb00019f0 con 0x7f9eb806fc10 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 >> v1:172.21.15.27:6789/0 conn(0x7f9eb806fc10 legacy=0x7f9eb80c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebd41e640 1 --2- 172.21.15.27:0/2041447012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea40039c0 0x7f9ea4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebd41e640 1 --2- 172.21.15.27:0/2041447012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea40047c0 0x7f9ea4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebd41e640 1 --2- 172.21.15.27:0/2041447012 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ea40051d0 0x7f9ea400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea400c0f0 con 0x7f9ea40039c0 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea400c260 con 0x7f9ea40047c0 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea400c420 con 0x7f9ea40051d0 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebe420640 1 --2- 172.21.15.27:0/2041447012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea40039c0 0x7f9ea4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebdc1f640 1 --2- 172.21.15.27:0/2041447012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea40047c0 0x7f9ea4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebec21640 1 --2- 172.21.15.27:0/2041447012 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ea40051d0 0x7f9ea400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebe420640 1 -- 172.21.15.27:0/2041447012 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ea40051d0 msgr2=0x7f9ea400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebe420640 1 --2- 172.21.15.27:0/2041447012 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ea40051d0 0x7f9ea400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebe420640 1 -- 172.21.15.27:0/2041447012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea40047c0 msgr2=0x7f9ea4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.686+0000 7f9ebe420640 1 --2- 172.21.15.27:0/2041447012 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ea40047c0 0x7f9ea4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.687+0000 7f9ebe420640 1 -- 172.21.15.27:0/2041447012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4003520 con 0x7f9ea40039c0 2023-12-19T18:18:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.687+0000 7f9ebec21640 1 --2- 172.21.15.27:0/2041447012 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ea40051d0 0x7f9ea400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.687+0000 7f9ebec21640 1 -- 172.21.15.27:0/2041447012 reap_dead start 2023-12-19T18:18:06.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.688+0000 7f9ebe420640 1 --2- 172.21.15.27:0/2041447012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea40039c0 0x7f9ea4003de0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009a90 tx=0x7f9ea40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5b7bb0d8bd6c1fd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.688+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb402c680 con 0x7f9ea40039c0 2023-12-19T18:18:06.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.688+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb402c7e0 con 0x7f9ea40039c0 2023-12-19T18:18:06.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.688+0000 7f9ebd41e640 1 -- 172.21.15.27:0/2041447012 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb402cb70 con 0x7f9ea40039c0 2023-12-19T18:18:06.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.688+0000 7f9ec0684640 1 -- 172.21.15.27:0/2041447012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea40039c0 msgr2=0x7f9ea4003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.688+0000 7f9ec0684640 1 --2- 172.21.15.27:0/2041447012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea40039c0 0x7f9ea4003de0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009a90 tx=0x7f9ea40057f0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.689+0000 7f9ec0684640 1 -- 172.21.15.27:0/2041447012 shutdown_connections 2023-12-19T18:18:06.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.689+0000 7f9ec0684640 1 --2- 172.21.15.27:0/2041447012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ea40039c0 0x7f9ea4003de0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.689+0000 7f9ec0684640 1 -- 172.21.15.27:0/2041447012 >> 172.21.15.27:0/2041447012 conn(0x7f9eb80ceb50 msgr2=0x7f9eb80cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:06.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.689+0000 7f9ec0684640 1 -- 172.21.15.27:0/2041447012 shutdown_connections 2023-12-19T18:18:06.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.689+0000 7f9ec0684640 1 -- 172.21.15.27:0/2041447012 wait complete. 2023-12-19T18:18:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.690+0000 7f9ec0684640 1 Processor -- start 2023-12-19T18:18:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.691+0000 7f9ec0684640 1 -- start start 2023-12-19T18:18:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.691+0000 7f9ec0684640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ea40039c0 0x7f9eb80cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ec0684640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9eb80c2680 0x7f9eb80c3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ec0684640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eb814ae10 0x7f9eb80af250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ec0684640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80ce590 con 0x7f9eb814ae10 2023-12-19T18:18:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ec0684640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80c2e00 con 0x7f9eb80c2680 2023-12-19T18:18:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ec0684640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9eb80c2b20 con 0x7f9ea40039c0 2023-12-19T18:18:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ebec21640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eb814ae10 0x7f9eb80af250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ebec21640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eb814ae10 0x7f9eb80af250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58092/0 (socket says 172.21.15.27:58092) 2023-12-19T18:18:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ebec21640 1 -- 172.21.15.27:0/4251012379 learned_addr learned my addr 172.21.15.27:0/4251012379 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ebe420640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ea40039c0 0x7f9eb80cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ebdc1f640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9eb80c2680 0x7f9eb80c3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ebec21640 1 -- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ea40039c0 msgr2=0x7f9eb80cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ebec21640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ea40039c0 0x7f9eb80cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ebec21640 1 -- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9eb80c2680 msgr2=0x7f9eb80c3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ebec21640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9eb80c2680 0x7f9eb80c3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ebec21640 1 -- 172.21.15.27:0/4251012379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb80cc830 con 0x7f9eb814ae10 2023-12-19T18:18:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.692+0000 7f9ebdc1f640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9eb80c2680 0x7f9eb80c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.693+0000 7f9ebec21640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eb814ae10 0x7f9eb80af250 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80b4ca0 tx=0x7f9eb000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.693+0000 7f9eaeffd640 1 -- 172.21.15.27:0/4251012379 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0004d60 con 0x7f9eb814ae10 2023-12-19T18:18:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.693+0000 7f9eaeffd640 1 -- 172.21.15.27:0/4251012379 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9eb000bb80 con 0x7f9eb814ae10 2023-12-19T18:18:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.693+0000 7f9eaeffd640 1 -- 172.21.15.27:0/4251012379 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0007070 con 0x7f9eb814ae10 2023-12-19T18:18:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.693+0000 7f9ec0684640 1 -- 172.21.15.27:0/4251012379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eb80ccb10 con 0x7f9eb814ae10 2023-12-19T18:18:06.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.693+0000 7f9ec0684640 1 -- 172.21.15.27:0/4251012379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb80c35a0 con 0x7f9eb814ae10 2023-12-19T18:18:06.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.694+0000 7f9ec0684640 1 -- 172.21.15.27:0/4251012379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9eb80cd030 con 0x7f9eb814ae10 2023-12-19T18:18:06.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.695+0000 7f9eaeffd640 1 -- 172.21.15.27:0/4251012379 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9eb00071d0 con 0x7f9eb814ae10 2023-12-19T18:18:06.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.696+0000 7f9eaeffd640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e9c06a460 0x7f9e9c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:06.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.696+0000 7f9eaeffd640 1 -- 172.21.15.27:0/4251012379 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f9eb0088ce0 con 0x7f9eb814ae10 2023-12-19T18:18:06.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.697+0000 7f9ebe420640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e9c06a460 0x7f9e9c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:06.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.698+0000 7f9ebe420640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e9c06a460 0x7f9e9c06c920 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4036000 tx=0x7f9eb4030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:06.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.701+0000 7f9eaeffd640 1 -- 172.21.15.27:0/4251012379 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9eb80cd030 con 0x7f9eb814ae10 2023-12-19T18:18:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.908+0000 7f9ec0684640 1 -- 172.21.15.27:0/4251012379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9eb80cd030 con 0x7f9eb814ae10 2023-12-19T18:18:06.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.909+0000 7f9eaeffd640 1 -- 172.21.15.27:0/4251012379 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v63) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9eb80cd030 con 0x7f9eb814ae10 2023-12-19T18:18:06.909 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:18:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.911+0000 7f9ec0684640 1 -- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e9c06a460 msgr2=0x7f9e9c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.911+0000 7f9ec0684640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e9c06a460 0x7f9e9c06c920 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4036000 tx=0x7f9eb4030000 comp rx=0 tx=0).stop 2023-12-19T18:18:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.911+0000 7f9ec0684640 1 -- 172.21.15.27:0/4251012379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eb814ae10 msgr2=0x7f9eb80af250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.911+0000 7f9ec0684640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eb814ae10 0x7f9eb80af250 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80b4ca0 tx=0x7f9eb000b4a0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.911+0000 7f9ec0684640 1 -- 172.21.15.27:0/4251012379 shutdown_connections 2023-12-19T18:18:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.911+0000 7f9ec0684640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9e9c06a460 0x7f9e9c06c920 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.911+0000 7f9ec0684640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9eb814ae10 0x7f9eb80af250 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.911+0000 7f9ec0684640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9eb80c2680 0x7f9eb80c3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.911+0000 7f9ec0684640 1 --2- 172.21.15.27:0/4251012379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ea40039c0 0x7f9eb80cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.911+0000 7f9ec0684640 1 -- 172.21.15.27:0/4251012379 >> 172.21.15.27:0/4251012379 conn(0x7f9eb80ceb50 msgr2=0x7f9eb8070030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.911+0000 7f9ec0684640 1 -- 172.21.15.27:0/4251012379 shutdown_connections 2023-12-19T18:18:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:06.912+0000 7f9ec0684640 1 -- 172.21.15.27:0/4251012379 wait complete. 2023-12-19T18:18:06.930 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-19T18:18:06.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:06.931 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-19T18:18:07.030 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[87132]: fuse finished with error 0 and tester_r 0 2023-12-19T18:18:07.034 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:18:07.034 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:07.034 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-19T18:18:07.215 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:18:07.215 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:07.279 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:18:07.279 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:18:07.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:07.279 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-19T18:18:07.337 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:18:07.338 INFO:teuthology.orchestra.run.smithi027.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-19T18:18:07.338 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:18:07.338 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:07.397 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:18:07.398 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:18:07.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:07.398 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-19T18:18:07.450 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-19T18:18:07.451 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:18:07.452 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.699+0000 7fd46b367640 1 Processor -- start 2023-12-19T18:18:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.699+0000 7fd46b367640 1 -- start start 2023-12-19T18:18:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.701+0000 7fd46b367640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd4640c8f90 0x7fd464160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.701+0000 7fd46b367640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4641612b0 con 0x7fd4640cb300 2023-12-19T18:18:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.701+0000 7fd46b367640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd464145d60 con 0x7fd464160e40 2023-12-19T18:18:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.701+0000 7fd46b367640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4640c4480 con 0x7fd4640c8f90 2023-12-19T18:18:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.701+0000 7fd469b64640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd4640c8f90 0x7fd464160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.701+0000 7fd469b64640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd4640c8f90 0x7fd464160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58096/0 (socket says 172.21.15.27:58096) 2023-12-19T18:18:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.701+0000 7fd469b64640 1 -- 172.21.15.27:0/2721528289 learned_addr learned my addr 172.21.15.27:0/2721528289 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.702+0000 7fd469b64640 1 -- 172.21.15.27:0/2721528289 >> v1:172.21.15.192:6789/0 conn(0x7fd464160e40 legacy=0x7fd4640cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.702+0000 7fd469b64640 1 -- 172.21.15.27:0/2721528289 >> v1:172.21.15.27:6789/0 conn(0x7fd4640cb300 legacy=0x7fd4640cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.702+0000 7fd469b64640 1 -- 172.21.15.27:0/2721528289 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd464161510 con 0x7fd4640c8f90 2023-12-19T18:18:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.703+0000 7fd469b64640 1 --2- 172.21.15.27:0/2721528289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4640c8f90 0x7fd464160900 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd4540098a0 tx=0x7fd45402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9c81b36f9c8c4d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.703+0000 7fd469363640 1 -- 172.21.15.27:0/2721528289 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454030840 con 0x7fd4640c8f90 2023-12-19T18:18:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.703+0000 7fd469363640 1 -- 172.21.15.27:0/2721528289 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4540309a0 con 0x7fd4640c8f90 2023-12-19T18:18:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.703+0000 7fd469363640 1 -- 172.21.15.27:0/2721528289 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd454030d30 con 0x7fd4640c8f90 2023-12-19T18:18:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.704+0000 7fd46b367640 1 -- 172.21.15.27:0/2721528289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4640c8f90 msgr2=0x7fd464160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.704+0000 7fd46b367640 1 --2- 172.21.15.27:0/2721528289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4640c8f90 0x7fd464160900 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd4540098a0 tx=0x7fd45402b980 comp rx=0 tx=0).stop 2023-12-19T18:18:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.704+0000 7fd46b367640 1 -- 172.21.15.27:0/2721528289 shutdown_connections 2023-12-19T18:18:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.704+0000 7fd46b367640 1 --2- 172.21.15.27:0/2721528289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4640c8f90 0x7fd464160900 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.704+0000 7fd46b367640 1 -- 172.21.15.27:0/2721528289 >> 172.21.15.27:0/2721528289 conn(0x7fd464069c70 msgr2=0x7fd46406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.704+0000 7fd46b367640 1 -- 172.21.15.27:0/2721528289 shutdown_connections 2023-12-19T18:18:07.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.704+0000 7fd46b367640 1 -- 172.21.15.27:0/2721528289 wait complete. 2023-12-19T18:18:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.706+0000 7fd46b367640 1 Processor -- start 2023-12-19T18:18:07.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.706+0000 7fd46b367640 1 -- start start 2023-12-19T18:18:07.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.707+0000 7fd46b367640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4640c8f90 0x7fd46415c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.707+0000 7fd46b367640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4640cb300 0x7fd46415edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.707+0000 7fd46b367640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd464160e40 0x7fd46406c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.707+0000 7fd46b367640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4640b1b10 con 0x7fd4640c8f90 2023-12-19T18:18:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.707+0000 7fd46b367640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4640c6a70 con 0x7fd464160e40 2023-12-19T18:18:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.707+0000 7fd46b367640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4640c5ef0 con 0x7fd4640cb300 2023-12-19T18:18:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.708+0000 7fd46a365640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4640c8f90 0x7fd46415c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.708+0000 7fd46a365640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4640c8f90 0x7fd46415c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58110/0 (socket says 172.21.15.27:58110) 2023-12-19T18:18:07.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.708+0000 7fd46a365640 1 -- 172.21.15.27:0/202438977 learned_addr learned my addr 172.21.15.27:0/202438977 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:07.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.708+0000 7fd469b64640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4640cb300 0x7fd46415edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.708+0000 7fd46a365640 1 -- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4640cb300 msgr2=0x7fd46415edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.708+0000 7fd46a365640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4640cb300 0x7fd46415edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.708+0000 7fd46ab66640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd464160e40 0x7fd46406c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.708+0000 7fd46a365640 1 -- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd464160e40 msgr2=0x7fd46406c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.708+0000 7fd46a365640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd464160e40 0x7fd46406c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.708+0000 7fd46a365640 1 -- 172.21.15.27:0/202438977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4540094d0 con 0x7fd4640c8f90 2023-12-19T18:18:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.708+0000 7fd46ab66640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd464160e40 0x7fd46406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.708+0000 7fd46a365640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4640c8f90 0x7fd46415c370 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd46000c400 tx=0x7fd46000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.709+0000 7fd45affd640 1 -- 172.21.15.27:0/202438977 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4600052d0 con 0x7fd4640c8f90 2023-12-19T18:18:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.709+0000 7fd45affd640 1 -- 172.21.15.27:0/202438977 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd460005430 con 0x7fd4640c8f90 2023-12-19T18:18:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.709+0000 7fd45affd640 1 -- 172.21.15.27:0/202438977 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd460003830 con 0x7fd4640c8f90 2023-12-19T18:18:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.709+0000 7fd46b367640 1 -- 172.21.15.27:0/202438977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4640c61d0 con 0x7fd4640c8f90 2023-12-19T18:18:07.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.709+0000 7fd46b367640 1 -- 172.21.15.27:0/202438977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4640c6790 con 0x7fd4640c8f90 2023-12-19T18:18:07.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.710+0000 7fd46b367640 1 -- 172.21.15.27:0/202438977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd46404e840 con 0x7fd4640c8f90 2023-12-19T18:18:07.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.711+0000 7fd45affd640 1 -- 172.21.15.27:0/202438977 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd460003990 con 0x7fd4640c8f90 2023-12-19T18:18:07.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.712+0000 7fd45affd640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd44806a430 0x7fd44806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:07.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.712+0000 7fd45affd640 1 -- 172.21.15.27:0/202438977 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd460087ff0 con 0x7fd4640c8f90 2023-12-19T18:18:07.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.713+0000 7fd469b64640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd44806a430 0x7fd44806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:07.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.714+0000 7fd469b64640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd44806a430 0x7fd44806c8f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd454036000 tx=0x7fd45403b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:07.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.716+0000 7fd45affd640 1 -- 172.21.15.27:0/202438977 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd4600534b0 con 0x7fd4640c8f90 2023-12-19T18:18:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.950+0000 7fd46b367640 1 -- 172.21.15.27:0/202438977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd4640adc50 con 0x7fd4640c8f90 2023-12-19T18:18:07.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.951+0000 7fd45affd640 1 -- 172.21.15.27:0/202438977 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4712 (secure 0 0 0) 0x7fd460056ad0 con 0x7fd4640c8f90 2023-12-19T18:18:07.954 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:07.954 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/930391176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":930391176},{"type":"v1","addr":"172.21.15.192:6837","nonce":930391176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5179,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3606564673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3606564673},{"type":"v1","addr":"172.21.15.192:6836","nonce":3606564673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3604117097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3604117097},{"type":"v1","addr":"172.21.15.27:6834","nonce":3604117097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:14.250689+0000","modified":"2023-12-19T18:17:17.758670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3814458529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3814458529},{"type":"v1","addr":"172.21.15.27:6835","nonce":3814458529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T18:18:07.955 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-19T18:18:07.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.954+0000 7fd46b367640 1 -- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd44806a430 msgr2=0x7fd44806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:07.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.954+0000 7fd46b367640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd44806a430 0x7fd44806c8f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd454036000 tx=0x7fd45403b040 comp rx=0 tx=0).stop 2023-12-19T18:18:07.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.954+0000 7fd46b367640 1 -- 172.21.15.27:0/202438977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4640c8f90 msgr2=0x7fd46415c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:07.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.954+0000 7fd46b367640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4640c8f90 0x7fd46415c370 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd46000c400 tx=0x7fd46000c8c0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.954+0000 7fd46b367640 1 -- 172.21.15.27:0/202438977 shutdown_connections 2023-12-19T18:18:07.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.954+0000 7fd46b367640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd44806a430 0x7fd44806c8f0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.954+0000 7fd46b367640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd464160e40 0x7fd46406c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.954+0000 7fd46b367640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4640cb300 0x7fd46415edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.954+0000 7fd46b367640 1 --2- 172.21.15.27:0/202438977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4640c8f90 0x7fd46415c370 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:07.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.954+0000 7fd46b367640 1 -- 172.21.15.27:0/202438977 >> 172.21.15.27:0/202438977 conn(0x7fd464069c70 msgr2=0x7fd4640cb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:07.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.954+0000 7fd46b367640 1 -- 172.21.15.27:0/202438977 shutdown_connections 2023-12-19T18:18:07.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:07.955+0000 7fd46b367640 1 -- 172.21.15.27:0/202438977 wait complete. 2023-12-19T18:18:07.972 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.198+0000 7f03b3fff640 1 Processor -- start 2023-12-19T18:18:08.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.199+0000 7f03b3fff640 1 -- start start 2023-12-19T18:18:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.200+0000 7f03b3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03b40b4ef0 0x7f03b40b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.200+0000 7f03b3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b40d2350 0x7f03b40b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.200+0000 7f03b3fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b40b5e90 con 0x7f03b41604d0 2023-12-19T18:18:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.200+0000 7f03b3fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b40d2b30 con 0x7f03b40b4ef0 2023-12-19T18:18:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.200+0000 7f03b3fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b40d2850 con 0x7f03b40d2350 2023-12-19T18:18:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.201+0000 7f03b27fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03b40b4ef0 0x7f03b40b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.201+0000 7f03b27fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03b40b4ef0 0x7f03b40b6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58120/0 (socket says 172.21.15.27:58120) 2023-12-19T18:18:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.201+0000 7f03b27fc640 1 -- 172.21.15.27:0/4190521685 learned_addr learned my addr 172.21.15.27:0/4190521685 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.201+0000 7f03b37fe640 1 --2- 172.21.15.27:0/4190521685 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b40d2350 0x7f03b40b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.201+0000 7f03b27fc640 1 -- 172.21.15.27:0/4190521685 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b40d2350 msgr2=0x7f03b40b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.201+0000 7f03b27fc640 1 --2- 172.21.15.27:0/4190521685 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b40d2350 0x7f03b40b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.201+0000 7f03b27fc640 1 -- 172.21.15.27:0/4190521685 >> v1:172.21.15.192:6789/0 conn(0x7f03b41604d0 legacy=0x7f03b40b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.201+0000 7f03b27fc640 1 -- 172.21.15.27:0/4190521685 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b40cded0 con 0x7f03b40b4ef0 2023-12-19T18:18:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.203+0000 7f03b27fc640 1 --2- 172.21.15.27:0/4190521685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b40b4ef0 0x7f03b40b6300 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009870 tx=0x7f03a802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff486bd5da2185da server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.203+0000 7f03b1ffb640 1 -- 172.21.15.27:0/4190521685 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8030840 con 0x7f03b40b4ef0 2023-12-19T18:18:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.203+0000 7f03b1ffb640 1 -- 172.21.15.27:0/4190521685 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a80309a0 con 0x7f03b40b4ef0 2023-12-19T18:18:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.203+0000 7f03b1ffb640 1 -- 172.21.15.27:0/4190521685 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a8030d30 con 0x7f03b40b4ef0 2023-12-19T18:18:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.203+0000 7f03b3fff640 1 -- 172.21.15.27:0/4190521685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b40b4ef0 msgr2=0x7f03b40b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.203+0000 7f03b3fff640 1 --2- 172.21.15.27:0/4190521685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b40b4ef0 0x7f03b40b6300 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f03a8009870 tx=0x7f03a802b980 comp rx=0 tx=0).stop 2023-12-19T18:18:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.204+0000 7f03b3fff640 1 -- 172.21.15.27:0/4190521685 shutdown_connections 2023-12-19T18:18:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.204+0000 7f03b3fff640 1 --2- 172.21.15.27:0/4190521685 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b40d2350 0x7f03b40b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.204+0000 7f03b3fff640 1 --2- 172.21.15.27:0/4190521685 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b40b4ef0 0x7f03b40b6300 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.204+0000 7f03b3fff640 1 -- 172.21.15.27:0/4190521685 >> 172.21.15.27:0/4190521685 conn(0x7f03b40cf7e0 msgr2=0x7f03b40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.204+0000 7f03b3fff640 1 -- 172.21.15.27:0/4190521685 shutdown_connections 2023-12-19T18:18:08.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.204+0000 7f03b3fff640 1 -- 172.21.15.27:0/4190521685 wait complete. 2023-12-19T18:18:08.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.205+0000 7f03b3fff640 1 Processor -- start 2023-12-19T18:18:08.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.206+0000 7f03b3fff640 1 -- start start 2023-12-19T18:18:08.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.206+0000 7f03b3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b40b4ef0 0x7f03b40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.207+0000 7f03b3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b40d2350 0x7f03b41f2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.207+0000 7f03b3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b41604d0 0x7f03b40ca500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.207+0000 7f03b3fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b40cb820 con 0x7f03b40b4ef0 2023-12-19T18:18:08.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.207+0000 7f03b3fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b40cbb00 con 0x7f03b40d2350 2023-12-19T18:18:08.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.207+0000 7f03b3fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b40cbde0 con 0x7f03b41604d0 2023-12-19T18:18:08.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.207+0000 7f03b2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b40b4ef0 0x7f03b40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.207+0000 7f03b2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b40b4ef0 0x7f03b40cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58128/0 (socket says 172.21.15.27:58128) 2023-12-19T18:18:08.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.207+0000 7f03b2ffd640 1 -- 172.21.15.27:0/1916488877 learned_addr learned my addr 172.21.15.27:0/1916488877 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:08.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.207+0000 7f03b2ffd640 1 -- 172.21.15.27:0/1916488877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b41604d0 msgr2=0x7f03b40ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.207+0000 7f03b2ffd640 1 --2- 172.21.15.27:0/1916488877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b41604d0 0x7f03b40ca500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.207+0000 7f03b2ffd640 1 -- 172.21.15.27:0/1916488877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b40d2350 msgr2=0x7f03b41f2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.207+0000 7f03b2ffd640 1 --2- 172.21.15.27:0/1916488877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b40d2350 0x7f03b41f2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.207+0000 7f03b2ffd640 1 -- 172.21.15.27:0/1916488877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a80094d0 con 0x7f03b40b4ef0 2023-12-19T18:18:08.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.208+0000 7f03b2ffd640 1 --2- 172.21.15.27:0/1916488877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b40b4ef0 0x7f03b40cc8c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f03a400c450 tx=0x7f03a400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:08.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.208+0000 7f039b7fe640 1 -- 172.21.15.27:0/1916488877 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a40052d0 con 0x7f03b40b4ef0 2023-12-19T18:18:08.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.208+0000 7f039b7fe640 1 -- 172.21.15.27:0/1916488877 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03a4005430 con 0x7f03b40b4ef0 2023-12-19T18:18:08.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.208+0000 7f039b7fe640 1 -- 172.21.15.27:0/1916488877 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4012870 con 0x7f03b40b4ef0 2023-12-19T18:18:08.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.208+0000 7f03b3fff640 1 -- 172.21.15.27:0/1916488877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b40cc0c0 con 0x7f03b40b4ef0 2023-12-19T18:18:08.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.208+0000 7f03b3fff640 1 -- 172.21.15.27:0/1916488877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b41f3120 con 0x7f03b40b4ef0 2023-12-19T18:18:08.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.209+0000 7f03b3fff640 1 -- 172.21.15.27:0/1916488877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b41f3310 con 0x7f03b40b4ef0 2023-12-19T18:18:08.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.210+0000 7f039b7fe640 1 -- 172.21.15.27:0/1916488877 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f03a40129d0 con 0x7f03b40b4ef0 2023-12-19T18:18:08.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.211+0000 7f039b7fe640 1 --2- 172.21.15.27:0/1916488877 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f03a006a430 0x7f03a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.211+0000 7f039b7fe640 1 -- 172.21.15.27:0/1916488877 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f03a40884d0 con 0x7f03b40b4ef0 2023-12-19T18:18:08.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.212+0000 7f03b27fc640 1 --2- 172.21.15.27:0/1916488877 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f03a006a430 0x7f03a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.213+0000 7f03b27fc640 1 --2- 172.21.15.27:0/1916488877 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f03a006a430 0x7f03a006c8f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03a8036000 tx=0x7f03a803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:08.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.216+0000 7f039b7fe640 1 -- 172.21.15.27:0/1916488877 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f03b41f3310 con 0x7f03b40b4ef0 2023-12-19T18:18:08.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.411+0000 7f03b3fff640 1 -- 172.21.15.27:0/1916488877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03b40cc5e0 con 0x7f03b40b4ef0 2023-12-19T18:18:08.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.413+0000 7f039b7fe640 1 -- 172.21.15.27:0/1916488877 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f03b40cc5e0 con 0x7f03b40b4ef0 2023-12-19T18:18:08.414 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:08.414 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":63,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:17:56.605695+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:12.527441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:13.291755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.8","mappings":[{"from":5,"to":7}]},{"pgid":"7.9","mappings":[{"from":5,"to":7}]},{"pgid":"7.20","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:08.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.416+0000 7f03b3fff640 1 -- 172.21.15.27:0/1916488877 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f03a006a430 msgr2=0x7f03a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.416+0000 7f03b3fff640 1 --2- 172.21.15.27:0/1916488877 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f03a006a430 0x7f03a006c8f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03a8036000 tx=0x7f03a803b040 comp rx=0 tx=0).stop 2023-12-19T18:18:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.416+0000 7f03b3fff640 1 -- 172.21.15.27:0/1916488877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b40b4ef0 msgr2=0x7f03b40cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.416+0000 7f03b3fff640 1 --2- 172.21.15.27:0/1916488877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b40b4ef0 0x7f03b40cc8c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f03a400c450 tx=0x7f03a400c910 comp rx=0 tx=0).stop 2023-12-19T18:18:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.416+0000 7f03b3fff640 1 -- 172.21.15.27:0/1916488877 shutdown_connections 2023-12-19T18:18:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.416+0000 7f03b3fff640 1 --2- 172.21.15.27:0/1916488877 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f03a006a430 0x7f03a006c8f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.416+0000 7f03b3fff640 1 --2- 172.21.15.27:0/1916488877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03b41604d0 0x7f03b40ca500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.416+0000 7f03b3fff640 1 --2- 172.21.15.27:0/1916488877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03b40d2350 0x7f03b41f2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.416+0000 7f03b3fff640 1 --2- 172.21.15.27:0/1916488877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03b40b4ef0 0x7f03b40cc8c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.416+0000 7f03b3fff640 1 -- 172.21.15.27:0/1916488877 >> 172.21.15.27:0/1916488877 conn(0x7f03b40cf7e0 msgr2=0x7f03b40b3720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.416+0000 7f03b3fff640 1 -- 172.21.15.27:0/1916488877 shutdown_connections 2023-12-19T18:18:08.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.416+0000 7f03b3fff640 1 -- 172.21.15.27:0/1916488877 wait complete. 2023-12-19T18:18:08.437 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:17:12.527441+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:17:13.291755+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:18:08.438 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:18:08.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.675+0000 7f3925c12640 1 Processor -- start 2023-12-19T18:18:08.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.675+0000 7f3925c12640 1 -- start start 2023-12-19T18:18:08.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.676+0000 7f3925c12640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f39200cf9b0 0x7f39200cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.676+0000 7f3925c12640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f39200cda20 0x7f39200cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.676+0000 7f3925c12640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39200d26d0 con 0x7f39200b8c30 2023-12-19T18:18:08.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.677+0000 7f3925c12640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39200cea80 con 0x7f39200cf9b0 2023-12-19T18:18:08.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.677+0000 7f3925c12640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39200b5f10 con 0x7f39200cda20 2023-12-19T18:18:08.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.677+0000 7f3917fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f39200cf9b0 0x7f39200cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.677+0000 7f391ffff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f39200cda20 0x7f39200cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.677+0000 7f391ffff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f39200cda20 0x7f39200cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:36208/0 (socket says 172.21.15.27:36208) 2023-12-19T18:18:08.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.677+0000 7f391ffff640 1 -- 172.21.15.27:0/3942917717 learned_addr learned my addr 172.21.15.27:0/3942917717 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:08.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.677+0000 7f3917fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f39200cf9b0 0x7f39200cfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58144/0 (socket says 172.21.15.27:58144) 2023-12-19T18:18:08.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.678+0000 7f3917fff640 1 -- 172.21.15.27:0/3942917717 >> v2:172.21.15.192:3300/0 conn(0x7f39200cda20 msgr2=0x7f39200cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:08.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.678+0000 7f3917fff640 1 --2- 172.21.15.27:0/3942917717 >> v2:172.21.15.192:3300/0 conn(0x7f39200cda20 0x7f39200cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.678+0000 7f3917fff640 1 -- 172.21.15.27:0/3942917717 >> v1:172.21.15.192:6789/0 conn(0x7f39200b8c30 legacy=0x7f39200d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.678+0000 7f3917fff640 1 -- 172.21.15.27:0/3942917717 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39200ce7a0 con 0x7f39200cf9b0 2023-12-19T18:18:08.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.679+0000 7f3917fff640 1 --2- 172.21.15.27:0/3942917717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39200cf9b0 0x7f39200cfe40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3920161390 tx=0x7f390802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e110570a6cecd8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:08.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.680+0000 7f391effd640 1 -- 172.21.15.27:0/3942917717 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f390802e070 con 0x7f39200cf9b0 2023-12-19T18:18:08.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.680+0000 7f391effd640 1 -- 172.21.15.27:0/3942917717 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39080315a0 con 0x7f39200cf9b0 2023-12-19T18:18:08.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.680+0000 7f391effd640 1 -- 172.21.15.27:0/3942917717 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3908031930 con 0x7f39200cf9b0 2023-12-19T18:18:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.681+0000 7f3925c12640 1 -- 172.21.15.27:0/3942917717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39200cf9b0 msgr2=0x7f39200cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.681+0000 7f3925c12640 1 --2- 172.21.15.27:0/3942917717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39200cf9b0 0x7f39200cfe40 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3920161390 tx=0x7f390802b840 comp rx=0 tx=0).stop 2023-12-19T18:18:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.681+0000 7f3925c12640 1 -- 172.21.15.27:0/3942917717 shutdown_connections 2023-12-19T18:18:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.681+0000 7f3925c12640 1 --2- 172.21.15.27:0/3942917717 >> v2:172.21.15.192:3300/0 conn(0x7f39200cda20 0x7f39200cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.681+0000 7f3925c12640 1 --2- 172.21.15.27:0/3942917717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39200cf9b0 0x7f39200cfe40 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.681+0000 7f3925c12640 1 -- 172.21.15.27:0/3942917717 >> 172.21.15.27:0/3942917717 conn(0x7f39200d2fa0 msgr2=0x7f39200d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.681+0000 7f3925c12640 1 -- 172.21.15.27:0/3942917717 shutdown_connections 2023-12-19T18:18:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.681+0000 7f3925c12640 1 -- 172.21.15.27:0/3942917717 wait complete. 2023-12-19T18:18:08.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.683+0000 7f3925c12640 1 Processor -- start 2023-12-19T18:18:08.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.683+0000 7f3925c12640 1 -- start start 2023-12-19T18:18:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.684+0000 7f3925c12640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39200b8c30 0x7f39200cbaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.684+0000 7f3925c12640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39200cda20 0x7f39200ca070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.684+0000 7f3925c12640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39200cf9b0 0x7f39201f2a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.684+0000 7f3925c12640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39200caa50 con 0x7f39200b8c30 2023-12-19T18:18:08.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.684+0000 7f3925c12640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39200cad30 con 0x7f39200cf9b0 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.684+0000 7f3925c12640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39200cb010 con 0x7f39200cda20 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.684+0000 7f391f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39200b8c30 0x7f39200cbaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.684+0000 7f391f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39200b8c30 0x7f39200cbaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58160/0 (socket says 172.21.15.27:58160) 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.684+0000 7f391f7fe640 1 -- 172.21.15.27:0/3453604709 learned_addr learned my addr 172.21.15.27:0/3453604709 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.684+0000 7f391ffff640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39200cf9b0 0x7f39201f2a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.684+0000 7f3917fff640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39200cda20 0x7f39200ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.685+0000 7f391f7fe640 1 -- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39200cda20 msgr2=0x7f39200ca070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.685+0000 7f391f7fe640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39200cda20 0x7f39200ca070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.685+0000 7f391f7fe640 1 -- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39200cf9b0 msgr2=0x7f39201f2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.685+0000 7f391f7fe640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39200cf9b0 0x7f39201f2a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.685+0000 7f391f7fe640 1 -- 172.21.15.27:0/3453604709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3908009400 con 0x7f39200b8c30 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.685+0000 7f391f7fe640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39200b8c30 0x7f39200cbaf0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f391000c450 tx=0x7f391000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.685+0000 7f391cff9640 1 -- 172.21.15.27:0/3453604709 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39100052d0 con 0x7f39200b8c30 2023-12-19T18:18:08.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.685+0000 7f391cff9640 1 -- 172.21.15.27:0/3453604709 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3910005430 con 0x7f39200b8c30 2023-12-19T18:18:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.685+0000 7f391cff9640 1 -- 172.21.15.27:0/3453604709 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3910012870 con 0x7f39200b8c30 2023-12-19T18:18:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.685+0000 7f3925c12640 1 -- 172.21.15.27:0/3453604709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39200cb2f0 con 0x7f39200b8c30 2023-12-19T18:18:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.685+0000 7f3925c12640 1 -- 172.21.15.27:0/3453604709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39200cc030 con 0x7f39200b8c30 2023-12-19T18:18:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.687+0000 7f3925c12640 1 -- 172.21.15.27:0/3453604709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3920068e90 con 0x7f39200b8c30 2023-12-19T18:18:08.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.687+0000 7f391cff9640 1 -- 172.21.15.27:0/3453604709 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f39100129d0 con 0x7f39200b8c30 2023-12-19T18:18:08.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.688+0000 7f391cff9640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f390006a430 0x7f390006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:08.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.688+0000 7f391cff9640 1 -- 172.21.15.27:0/3453604709 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f3910088510 con 0x7f39200b8c30 2023-12-19T18:18:08.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.689+0000 7f3917fff640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f390006a430 0x7f390006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:08.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.690+0000 7f3917fff640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f390006a430 0x7f390006c8f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3908036000 tx=0x7f390803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:08.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.694+0000 7f391cff9640 1 -- 172.21.15.27:0/3453604709 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3910053a70 con 0x7f39200b8c30 2023-12-19T18:18:08.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.891+0000 7f3925c12640 1 -- 172.21.15.27:0/3453604709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f39200cd360 con 0x7f39200b8c30 2023-12-19T18:18:08.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.892+0000 7f391cff9640 1 -- 172.21.15.27:0/3453604709 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v63) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3910057090 con 0x7f39200b8c30 2023-12-19T18:18:08.892 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:18:08.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.894+0000 7f3925c12640 1 -- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f390006a430 msgr2=0x7f390006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.894+0000 7f3925c12640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f390006a430 0x7f390006c8f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f3908036000 tx=0x7f390803d040 comp rx=0 tx=0).stop 2023-12-19T18:18:08.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.894+0000 7f3925c12640 1 -- 172.21.15.27:0/3453604709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39200b8c30 msgr2=0x7f39200cbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:08.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.894+0000 7f3925c12640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39200b8c30 0x7f39200cbaf0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f391000c450 tx=0x7f391000c910 comp rx=0 tx=0).stop 2023-12-19T18:18:08.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.894+0000 7f3925c12640 1 -- 172.21.15.27:0/3453604709 shutdown_connections 2023-12-19T18:18:08.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.894+0000 7f3925c12640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f390006a430 0x7f390006c8f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.894+0000 7f3925c12640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f39200cf9b0 0x7f39201f2a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.894+0000 7f3925c12640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f39200cda20 0x7f39200ca070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.894+0000 7f3925c12640 1 --2- 172.21.15.27:0/3453604709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39200b8c30 0x7f39200cbaf0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:08.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.894+0000 7f3925c12640 1 -- 172.21.15.27:0/3453604709 >> 172.21.15.27:0/3453604709 conn(0x7f39200d2fa0 msgr2=0x7f39200d0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:08.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.894+0000 7f3925c12640 1 -- 172.21.15.27:0/3453604709 shutdown_connections 2023-12-19T18:18:08.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:08.895+0000 7f3925c12640 1 -- 172.21.15.27:0/3453604709 wait complete. 2023-12-19T18:18:08.915 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:18:09.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.157+0000 7f75733c6640 1 Processor -- start 2023-12-19T18:18:09.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.157+0000 7f75733c6640 1 -- start start 2023-12-19T18:18:09.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.158+0000 7f75733c6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f756c15cf80 0x7f756c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:09.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.158+0000 7f75733c6640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f756c0b85e0 0x7f756c0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:09.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.158+0000 7f75733c6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f756c0b8a60 0x7f756c0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:09.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.158+0000 7f75733c6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756c0d0e10 con 0x7f756c15cf80 2023-12-19T18:18:09.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f75733c6640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f756c0d1c70 con 0x7f756c0b85e0 2023-12-19T18:18:09.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f75733c6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756c15d360 con 0x7f756c0b8a60 2023-12-19T18:18:09.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f75723c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f756c15cf80 0x7f756c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:09.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f75723c4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f756c15cf80 0x7f756c0d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58176/0 (socket says 172.21.15.27:58176) 2023-12-19T18:18:09.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f75723c4640 1 -- 172.21.15.27:0/3333863829 learned_addr learned my addr 172.21.15.27:0/3333863829 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:09.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f7572bc5640 1 --2- 172.21.15.27:0/3333863829 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f756c0b8a60 0x7f756c0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:09.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f7571bc3640 1 --2- 172.21.15.27:0/3333863829 >> v2:172.21.15.192:3300/0 conn(0x7f756c0b85e0 0x7f756c0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:09.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f75723c4640 1 -- 172.21.15.27:0/3333863829 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f756c0b8a60 msgr2=0x7f756c0cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:09.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f75723c4640 1 --2- 172.21.15.27:0/3333863829 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f756c0b8a60 0x7f756c0cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f75723c4640 1 -- 172.21.15.27:0/3333863829 >> v2:172.21.15.192:3300/0 conn(0x7f756c0b85e0 msgr2=0x7f756c0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:09.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f75723c4640 1 --2- 172.21.15.27:0/3333863829 >> v2:172.21.15.192:3300/0 conn(0x7f756c0b85e0 0x7f756c0cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f75723c4640 1 -- 172.21.15.27:0/3333863829 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c0d1060 con 0x7f756c15cf80 2023-12-19T18:18:09.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f7571bc3640 1 --2- 172.21.15.27:0/3333863829 >> v2:172.21.15.192:3300/0 conn(0x7f756c0b85e0 0x7f756c0cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.159+0000 7f7572bc5640 1 --2- 172.21.15.27:0/3333863829 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f756c0b8a60 0x7f756c0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.161+0000 7f75723c4640 1 --2- 172.21.15.27:0/3333863829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f756c15cf80 0x7f756c0d2100 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f756c0cd270 tx=0x7f756002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75614878b6369ea0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.161+0000 7f75713c2640 1 -- 172.21.15.27:0/3333863829 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756002cb00 con 0x7f756c15cf80 2023-12-19T18:18:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.161+0000 7f75713c2640 1 -- 172.21.15.27:0/3333863829 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756002cc60 con 0x7f756c15cf80 2023-12-19T18:18:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.161+0000 7f75713c2640 1 -- 172.21.15.27:0/3333863829 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75600343f0 con 0x7f756c15cf80 2023-12-19T18:18:09.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.162+0000 7f75733c6640 1 -- 172.21.15.27:0/3333863829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f756c15cf80 msgr2=0x7f756c0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:09.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.162+0000 7f75733c6640 1 --2- 172.21.15.27:0/3333863829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f756c15cf80 0x7f756c0d2100 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f756c0cd270 tx=0x7f756002bf60 comp rx=0 tx=0).stop 2023-12-19T18:18:09.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.162+0000 7f75733c6640 1 -- 172.21.15.27:0/3333863829 shutdown_connections 2023-12-19T18:18:09.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.162+0000 7f75733c6640 1 --2- 172.21.15.27:0/3333863829 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f756c0b8a60 0x7f756c0cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.162+0000 7f75733c6640 1 --2- 172.21.15.27:0/3333863829 >> v2:172.21.15.192:3300/0 conn(0x7f756c0b85e0 0x7f756c0cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.162+0000 7f75733c6640 1 --2- 172.21.15.27:0/3333863829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f756c15cf80 0x7f756c0d2100 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.162+0000 7f75733c6640 1 -- 172.21.15.27:0/3333863829 >> 172.21.15.27:0/3333863829 conn(0x7f756c072e20 msgr2=0x7f756c073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:09.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.162+0000 7f75733c6640 1 -- 172.21.15.27:0/3333863829 shutdown_connections 2023-12-19T18:18:09.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.162+0000 7f75733c6640 1 -- 172.21.15.27:0/3333863829 wait complete. 2023-12-19T18:18:09.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.164+0000 7f75733c6640 1 Processor -- start 2023-12-19T18:18:09.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.164+0000 7f75733c6640 1 -- start start 2023-12-19T18:18:09.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.165+0000 7f75733c6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f756c0b85e0 0x7f756c1f2150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:09.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.165+0000 7f75733c6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f756c15cf80 0x7f756c14d720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:09.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.165+0000 7f75733c6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f756c14ac80 0x7f756c14c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:09.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.165+0000 7f75733c6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c150b00 con 0x7f756c14ac80 2023-12-19T18:18:09.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.165+0000 7f75733c6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f756c14dc60 con 0x7f756c0b85e0 2023-12-19T18:18:09.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.165+0000 7f75733c6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f756c14b190 con 0x7f756c15cf80 2023-12-19T18:18:09.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.165+0000 7f7572bc5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f756c14ac80 0x7f756c14c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:09.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.166+0000 7f7572bc5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f756c14ac80 0x7f756c14c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58178/0 (socket says 172.21.15.27:58178) 2023-12-19T18:18:09.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.166+0000 7f7572bc5640 1 -- 172.21.15.27:0/3529369773 learned_addr learned my addr 172.21.15.27:0/3529369773 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:09.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.166+0000 7f7571bc3640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f756c15cf80 0x7f756c14d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:09.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.166+0000 7f75723c4640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f756c0b85e0 0x7f756c1f2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:09.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.166+0000 7f7572bc5640 1 -- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f756c15cf80 msgr2=0x7f756c14d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:09.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.166+0000 7f7572bc5640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f756c15cf80 0x7f756c14d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.166+0000 7f7572bc5640 1 -- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f756c0b85e0 msgr2=0x7f756c1f2150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:09.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.166+0000 7f7572bc5640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f756c0b85e0 0x7f756c1f2150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.166+0000 7f7572bc5640 1 -- 172.21.15.27:0/3529369773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7560009b30 con 0x7f756c14ac80 2023-12-19T18:18:09.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.166+0000 7f7572bc5640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f756c14ac80 0x7f756c14c230 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f756400c570 tx=0x7f756400ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:09.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.167+0000 7f756affd640 1 -- 172.21.15.27:0/3529369773 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564011800 con 0x7f756c14ac80 2023-12-19T18:18:09.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.167+0000 7f756affd640 1 -- 172.21.15.27:0/3529369773 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7564011980 con 0x7f756c14ac80 2023-12-19T18:18:09.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.167+0000 7f75733c6640 1 -- 172.21.15.27:0/3529369773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f756c14b470 con 0x7f756c14ac80 2023-12-19T18:18:09.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.167+0000 7f756affd640 1 -- 172.21.15.27:0/3529369773 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7564011d30 con 0x7f756c14ac80 2023-12-19T18:18:09.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.167+0000 7f75733c6640 1 -- 172.21.15.27:0/3529369773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f756c14ba30 con 0x7f756c14ac80 2023-12-19T18:18:09.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.168+0000 7f75733c6640 1 -- 172.21.15.27:0/3529369773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f756c04e840 con 0x7f756c14ac80 2023-12-19T18:18:09.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.169+0000 7f756affd640 1 -- 172.21.15.27:0/3529369773 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7564018360 con 0x7f756c14ac80 2023-12-19T18:18:09.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.171+0000 7f756affd640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f755006a430 0x7f755006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:09.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.171+0000 7f756affd640 1 -- 172.21.15.27:0/3529369773 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f7564088980 con 0x7f756c14ac80 2023-12-19T18:18:09.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.171+0000 7f75723c4640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f755006a430 0x7f755006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:09.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.172+0000 7f75723c4640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f755006a430 0x7f755006c8f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7560035000 tx=0x7f7560030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:09.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.175+0000 7f756affd640 1 -- 172.21.15.27:0/3529369773 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7564053ed0 con 0x7f756c14ac80 2023-12-19T18:18:09.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.370+0000 7f75733c6640 1 -- 172.21.15.27:0/3529369773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f756c15c7a0 con 0x7f756c14ac80 2023-12-19T18:18:09.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.371+0000 7f756affd640 1 -- 172.21.15.27:0/3529369773 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v63) v1 ==== 104+0+11 (secure 0 0 0) 0x7f756401f020 con 0x7f756c14ac80 2023-12-19T18:18:09.371 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:18:09.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.373+0000 7f75733c6640 1 -- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f755006a430 msgr2=0x7f755006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:09.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.374+0000 7f75733c6640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f755006a430 0x7f755006c8f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f7560035000 tx=0x7f7560030000 comp rx=0 tx=0).stop 2023-12-19T18:18:09.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.374+0000 7f75733c6640 1 -- 172.21.15.27:0/3529369773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f756c14ac80 msgr2=0x7f756c14c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:09.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.374+0000 7f75733c6640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f756c14ac80 0x7f756c14c230 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f756400c570 tx=0x7f756400ca30 comp rx=0 tx=0).stop 2023-12-19T18:18:09.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.374+0000 7f75733c6640 1 -- 172.21.15.27:0/3529369773 shutdown_connections 2023-12-19T18:18:09.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.374+0000 7f75733c6640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f755006a430 0x7f755006c8f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.374+0000 7f75733c6640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f756c14ac80 0x7f756c14c230 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.374+0000 7f75733c6640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f756c15cf80 0x7f756c14d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.374+0000 7f75733c6640 1 --2- 172.21.15.27:0/3529369773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f756c0b85e0 0x7f756c1f2150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.374+0000 7f75733c6640 1 -- 172.21.15.27:0/3529369773 >> 172.21.15.27:0/3529369773 conn(0x7f756c072e20 msgr2=0x7f756c0cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.374+0000 7f75733c6640 1 -- 172.21.15.27:0/3529369773 shutdown_connections 2023-12-19T18:18:09.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.374+0000 7f75733c6640 1 -- 172.21.15.27:0/3529369773 wait complete. 2023-12-19T18:18:09.392 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:18:09.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.620+0000 7f030ffff640 1 Processor -- start 2023-12-19T18:18:09.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.621+0000 7f030ffff640 1 -- start start 2023-12-19T18:18:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.623+0000 7f030ffff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0310151c90 0x7f031006b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.623+0000 7f030ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03100cf550 0x7f031015b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.623+0000 7f030ffff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f031015b3e0 con 0x7f031015e9d0 2023-12-19T18:18:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.623+0000 7f030ffff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031015eed0 con 0x7f03100cf550 2023-12-19T18:18:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.623+0000 7f030ffff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f031015f1b0 con 0x7f0310151c90 2023-12-19T18:18:09.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.623+0000 7f0307fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f031015e9d0 0x7f03100cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33592/0 (socket says 172.21.15.27:33592) 2023-12-19T18:18:09.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.623+0000 7f0307fff640 1 -- 172.21.15.27:0/3455617664 learned_addr learned my addr 172.21.15.27:0/3455617664 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:09.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.623+0000 7f030f7fe640 1 --2- 172.21.15.27:0/3455617664 >> v2:172.21.15.27:3300/0 conn(0x7f03100cf550 0x7f031015b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:09.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.623+0000 7f030effd640 1 --2- 172.21.15.27:0/3455617664 >> v2:172.21.15.192:3300/0 conn(0x7f0310151c90 0x7f031006b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:09.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.624+0000 7f030e7fc640 1 -- 172.21.15.27:0/3455617664 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470100572 0 0) 0x7f031015b3e0 con 0x7f031015e9d0 2023-12-19T18:18:09.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.624+0000 7f030e7fc640 1 -- 172.21.15.27:0/3455617664 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02fc003520 con 0x7f031015e9d0 2023-12-19T18:18:09.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.624+0000 7f030f7fe640 1 -- 172.21.15.27:0/3455617664 >> v2:172.21.15.192:3300/0 conn(0x7f0310151c90 msgr2=0x7f031006b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:09.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.624+0000 7f030f7fe640 1 --2- 172.21.15.27:0/3455617664 >> v2:172.21.15.192:3300/0 conn(0x7f0310151c90 0x7f031006b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.624+0000 7f030f7fe640 1 -- 172.21.15.27:0/3455617664 >> v1:172.21.15.27:6789/0 conn(0x7f031015e9d0 legacy=0x7f03100cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:09.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.624+0000 7f030f7fe640 1 -- 172.21.15.27:0/3455617664 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f031015f490 con 0x7f03100cf550 2023-12-19T18:18:09.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.626+0000 7f030f7fe640 1 --2- 172.21.15.27:0/3455617664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03100cf550 0x7f031015b8a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0308009d90 tx=0x7f030802bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8aafd94c1340b38 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:09.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.626+0000 7f030e7fc640 1 -- 172.21.15.27:0/3455617664 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f030802cb40 con 0x7f03100cf550 2023-12-19T18:18:09.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.626+0000 7f030e7fc640 1 -- 172.21.15.27:0/3455617664 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f030802cca0 con 0x7f03100cf550 2023-12-19T18:18:09.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.626+0000 7f030e7fc640 1 -- 172.21.15.27:0/3455617664 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308034660 con 0x7f03100cf550 2023-12-19T18:18:09.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.627+0000 7f030ffff640 1 -- 172.21.15.27:0/3455617664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03100cf550 msgr2=0x7f031015b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:09.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.627+0000 7f030ffff640 1 --2- 172.21.15.27:0/3455617664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03100cf550 0x7f031015b8a0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f0308009d90 tx=0x7f030802bd90 comp rx=0 tx=0).stop 2023-12-19T18:18:09.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.627+0000 7f030ffff640 1 -- 172.21.15.27:0/3455617664 shutdown_connections 2023-12-19T18:18:09.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.627+0000 7f030ffff640 1 --2- 172.21.15.27:0/3455617664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03100cf550 0x7f031015b8a0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.627+0000 7f030ffff640 1 --2- 172.21.15.27:0/3455617664 >> v2:172.21.15.192:3300/0 conn(0x7f0310151c90 0x7f031006b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.627+0000 7f030ffff640 1 -- 172.21.15.27:0/3455617664 >> 172.21.15.27:0/3455617664 conn(0x7f031006a060 msgr2=0x7f031006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:09.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.627+0000 7f030ffff640 1 -- 172.21.15.27:0/3455617664 shutdown_connections 2023-12-19T18:18:09.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.627+0000 7f030ffff640 1 -- 172.21.15.27:0/3455617664 wait complete. 2023-12-19T18:18:09.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.629+0000 7f030ffff640 1 Processor -- start 2023-12-19T18:18:09.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.629+0000 7f030ffff640 1 -- start start 2023-12-19T18:18:09.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.630+0000 7f030ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03100cf550 0x7f0310152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:09.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.630+0000 7f030ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0310151c90 0x7f0310153d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:09.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.630+0000 7f030ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f031015e9d0 0x7f031014ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:09.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.630+0000 7f030ffff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0310154560 con 0x7f0310151c90 2023-12-19T18:18:09.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.630+0000 7f030ffff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0310154840 con 0x7f031015e9d0 2023-12-19T18:18:09.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.630+0000 7f030ffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0310154b20 con 0x7f03100cf550 2023-12-19T18:18:09.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.630+0000 7f0307fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0310151c90 0x7f0310153d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.630+0000 7f0307fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0310151c90 0x7f0310153d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58194/0 (socket says 172.21.15.27:58194) 2023-12-19T18:18:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.630+0000 7f0307fff640 1 -- 172.21.15.27:0/3433183412 learned_addr learned my addr 172.21.15.27:0/3433183412 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.631+0000 7f030f7fe640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f031015e9d0 0x7f031014ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.631+0000 7f0307fff640 1 -- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03100cf550 msgr2=0x7f0310152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.631+0000 7f0307fff640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03100cf550 0x7f0310152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.631+0000 7f0307fff640 1 -- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f031015e9d0 msgr2=0x7f031014ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.631+0000 7f0307fff640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f031015e9d0 0x7f031014ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.631+0000 7f0307fff640 1 -- 172.21.15.27:0/3433183412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0308009a70 con 0x7f0310151c90 2023-12-19T18:18:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.631+0000 7f030f7fe640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f031015e9d0 0x7f031014ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.631+0000 7f0307fff640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0310151c90 0x7f0310153d10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f02f800c3c0 tx=0x7f02f800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:09.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.631+0000 7f03077fe640 1 -- 172.21.15.27:0/3433183412 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f8005ce0 con 0x7f0310151c90 2023-12-19T18:18:09.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.631+0000 7f03077fe640 1 -- 172.21.15.27:0/3433183412 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f8005e60 con 0x7f0310151c90 2023-12-19T18:18:09.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.631+0000 7f03077fe640 1 -- 172.21.15.27:0/3433183412 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f80072f0 con 0x7f0310151c90 2023-12-19T18:18:09.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.632+0000 7f030ffff640 1 -- 172.21.15.27:0/3433183412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0310154e00 con 0x7f0310151c90 2023-12-19T18:18:09.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.632+0000 7f030ffff640 1 -- 172.21.15.27:0/3433183412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0310154250 con 0x7f0310151c90 2023-12-19T18:18:09.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.633+0000 7f030ffff640 1 -- 172.21.15.27:0/3433183412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f031004e840 con 0x7f0310151c90 2023-12-19T18:18:09.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.634+0000 7f03077fe640 1 -- 172.21.15.27:0/3433183412 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f02f8007450 con 0x7f0310151c90 2023-12-19T18:18:09.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.635+0000 7f03077fe640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f02f006a460 0x7f02f006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:09.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.635+0000 7f03077fe640 1 -- 172.21.15.27:0/3433183412 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f02f8087f70 con 0x7f0310151c90 2023-12-19T18:18:09.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.636+0000 7f030effd640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f02f006a460 0x7f02f006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:09.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.637+0000 7f030effd640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f02f006a460 0x7f02f006c920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0300006a90 tx=0x7f0300006a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:09.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.642+0000 7f03077fe640 1 -- 172.21.15.27:0/3433183412 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f02f8053580 con 0x7f0310151c90 2023-12-19T18:18:09.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.842+0000 7f030ffff640 1 -- 172.21.15.27:0/3433183412 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f031015baa0 con 0x7f0310151c90 2023-12-19T18:18:09.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.843+0000 7f03077fe640 1 -- 172.21.15.27:0/3433183412 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v63) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02f80170a0 con 0x7f0310151c90 2023-12-19T18:18:09.843 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:18:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.846+0000 7f030ffff640 1 -- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f02f006a460 msgr2=0x7f02f006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.846+0000 7f030ffff640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f02f006a460 0x7f02f006c920 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0300006a90 tx=0x7f0300006a20 comp rx=0 tx=0).stop 2023-12-19T18:18:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.846+0000 7f030ffff640 1 -- 172.21.15.27:0/3433183412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0310151c90 msgr2=0x7f0310153d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.846+0000 7f030ffff640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0310151c90 0x7f0310153d10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f02f800c3c0 tx=0x7f02f800c880 comp rx=0 tx=0).stop 2023-12-19T18:18:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.846+0000 7f030ffff640 1 -- 172.21.15.27:0/3433183412 shutdown_connections 2023-12-19T18:18:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.846+0000 7f030ffff640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f02f006a460 0x7f02f006c920 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.846+0000 7f030ffff640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f031015e9d0 0x7f031014ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.846+0000 7f030ffff640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0310151c90 0x7f0310153d10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.846+0000 7f030ffff640 1 --2- 172.21.15.27:0/3433183412 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03100cf550 0x7f0310152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:09.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.846+0000 7f030ffff640 1 -- 172.21.15.27:0/3433183412 >> 172.21.15.27:0/3433183412 conn(0x7f031006a060 msgr2=0x7f03101552b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:09.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.846+0000 7f030ffff640 1 -- 172.21.15.27:0/3433183412 shutdown_connections 2023-12-19T18:18:09.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:09.846+0000 7f030ffff640 1 -- 172.21.15.27:0/3433183412 wait complete. 2023-12-19T18:18:09.866 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:10.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.089+0000 7f0a38e8a640 1 Processor -- start 2023-12-19T18:18:10.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.090+0000 7f0a38e8a640 1 -- start start 2023-12-19T18:18:10.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.091+0000 7f0a38e8a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a3415ceb0 0x7f0a340b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.091+0000 7f0a38e8a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a340b5170 con 0x7f0a340d2130 2023-12-19T18:18:10.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.091+0000 7f0a38e8a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a340b6540 con 0x7f0a340b4d10 2023-12-19T18:18:10.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.091+0000 7f0a38e8a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340cdfb0 con 0x7f0a3415ceb0 2023-12-19T18:18:10.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.092+0000 7f0a33fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0a340d2130 0x7f0a340b69d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33602/0 (socket says 172.21.15.27:33602) 2023-12-19T18:18:10.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.092+0000 7f0a33fff640 1 -- 172.21.15.27:0/1799637581 learned_addr learned my addr 172.21.15.27:0/1799637581 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:10.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.092+0000 7f0a337fe640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a3415ceb0 0x7f0a340b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.092+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731326944 0 0) 0x7f0a340b5170 con 0x7f0a340d2130 2023-12-19T18:18:10.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.092+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a20003520 con 0x7f0a340d2130 2023-12-19T18:18:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.092+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2287100981 0 0) 0x7f0a20003520 con 0x7f0a340d2130 2023-12-19T18:18:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.092+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a3415ceb0 msgr2=0x7f0a340b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.092+0000 7f0a32ffd640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a3415ceb0 0x7f0a340b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 >> v1:172.21.15.192:6789/0 conn(0x7f0a340b4d10 legacy=0x7f0a340d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a340d2600 con 0x7f0a340d2130 2023-12-19T18:18:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f0a2c0019f0 con 0x7f0a340d2130 2023-12-19T18:18:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 >> v1:172.21.15.27:6789/0 conn(0x7f0a340d2130 legacy=0x7f0a340b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a32ffd640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a3415ceb0 0x7f0a200039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a32ffd640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a20003f70 0x7f0a200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a32ffd640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20004d00 0x7f0a20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340cdfb0 con 0x7f0a20004d00 2023-12-19T18:18:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2000bac0 con 0x7f0a20003f70 2023-12-19T18:18:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2000bc80 con 0x7f0a3415ceb0 2023-12-19T18:18:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a33fff640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20004d00 0x7f0a20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a2bfff640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a20003f70 0x7f0a200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a337fe640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a3415ceb0 0x7f0a200039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a33fff640 1 -- 172.21.15.27:0/1799637581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a3415ceb0 msgr2=0x7f0a200039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a33fff640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a3415ceb0 0x7f0a200039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a33fff640 1 -- 172.21.15.27:0/1799637581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a20003f70 msgr2=0x7f0a200047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a33fff640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a20003f70 0x7f0a200047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a33fff640 1 -- 172.21.15.27:0/1799637581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a20003520 con 0x7f0a20004d00 2023-12-19T18:18:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a2bfff640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a20003f70 0x7f0a200047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.093+0000 7f0a337fe640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a3415ceb0 0x7f0a200039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.095+0000 7f0a33fff640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20004d00 0x7f0a20005180 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0a3408c1d0 tx=0x7f0a200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41df3c63466f67c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.095+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c034630 con 0x7f0a20004d00 2023-12-19T18:18:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.095+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2c02f070 con 0x7f0a20004d00 2023-12-19T18:18:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.095+0000 7f0a32ffd640 1 -- 172.21.15.27:0/1799637581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2c034a50 con 0x7f0a20004d00 2023-12-19T18:18:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.095+0000 7f0a38e8a640 1 -- 172.21.15.27:0/1799637581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20004d00 msgr2=0x7f0a20005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.095+0000 7f0a38e8a640 1 --2- 172.21.15.27:0/1799637581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20004d00 0x7f0a20005180 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f0a3408c1d0 tx=0x7f0a200057f0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.095+0000 7f0a33fff640 1 -- 172.21.15.27:0/1799637581 reap_dead start 2023-12-19T18:18:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.095+0000 7f0a38e8a640 1 -- 172.21.15.27:0/1799637581 shutdown_connections 2023-12-19T18:18:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.095+0000 7f0a38e8a640 1 -- 172.21.15.27:0/1799637581 >> 172.21.15.27:0/1799637581 conn(0x7f0a340cf7d0 msgr2=0x7f0a340cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.096+0000 7f0a38e8a640 1 -- 172.21.15.27:0/1799637581 shutdown_connections 2023-12-19T18:18:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.096+0000 7f0a38e8a640 1 -- 172.21.15.27:0/1799637581 wait complete. 2023-12-19T18:18:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.097+0000 7f0a38e8a640 1 Processor -- start 2023-12-19T18:18:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.097+0000 7f0a38e8a640 1 -- start start 2023-12-19T18:18:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.098+0000 7f0a38e8a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20004d00 0x7f0a3414a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.098+0000 7f0a38e8a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a340c6860 0x7f0a340c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.098+0000 7f0a38e8a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a340c6cc0 0x7f0a340c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.098+0000 7f0a38e8a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340c9270 con 0x7f0a20004d00 2023-12-19T18:18:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.098+0000 7f0a38e8a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340c9550 con 0x7f0a340c6860 2023-12-19T18:18:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.098+0000 7f0a38e8a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a340c9830 con 0x7f0a340c6cc0 2023-12-19T18:18:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.099+0000 7f0a337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20004d00 0x7f0a3414a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.099+0000 7f0a337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20004d00 0x7f0a3414a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58214/0 (socket says 172.21.15.27:58214) 2023-12-19T18:18:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.099+0000 7f0a337fe640 1 -- 172.21.15.27:0/4110577185 learned_addr learned my addr 172.21.15.27:0/4110577185 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.099+0000 7f0a337fe640 1 -- 172.21.15.27:0/4110577185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a340c6cc0 msgr2=0x7f0a340c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.099+0000 7f0a337fe640 1 --2- 172.21.15.27:0/4110577185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a340c6cc0 0x7f0a340c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.099+0000 7f0a337fe640 1 -- 172.21.15.27:0/4110577185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a340c6860 msgr2=0x7f0a340c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.099+0000 7f0a337fe640 1 --2- 172.21.15.27:0/4110577185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a340c6860 0x7f0a340c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.099+0000 7f0a337fe640 1 -- 172.21.15.27:0/4110577185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a340c9b10 con 0x7f0a20004d00 2023-12-19T18:18:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.099+0000 7f0a337fe640 1 --2- 172.21.15.27:0/4110577185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20004d00 0x7f0a3414a030 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0a240108b0 tx=0x7f0a24010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.100+0000 7f0a30ff9640 1 -- 172.21.15.27:0/4110577185 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2400eae0 con 0x7f0a20004d00 2023-12-19T18:18:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.100+0000 7f0a30ff9640 1 -- 172.21.15.27:0/4110577185 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2400ec40 con 0x7f0a20004d00 2023-12-19T18:18:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.100+0000 7f0a30ff9640 1 -- 172.21.15.27:0/4110577185 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2401c400 con 0x7f0a20004d00 2023-12-19T18:18:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.100+0000 7f0a38e8a640 1 -- 172.21.15.27:0/4110577185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a340ca210 con 0x7f0a20004d00 2023-12-19T18:18:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.100+0000 7f0a38e8a640 1 -- 172.21.15.27:0/4110577185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a340c8ee0 con 0x7f0a20004d00 2023-12-19T18:18:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.101+0000 7f0a38e8a640 1 -- 172.21.15.27:0/4110577185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a3404e840 con 0x7f0a20004d00 2023-12-19T18:18:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.102+0000 7f0a30ff9640 1 -- 172.21.15.27:0/4110577185 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0a2401c560 con 0x7f0a20004d00 2023-12-19T18:18:10.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.104+0000 7f0a30ff9640 1 --2- 172.21.15.27:0/4110577185 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a1406a430 0x7f0a1406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.104+0000 7f0a30ff9640 1 -- 172.21.15.27:0/4110577185 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0a24088fd0 con 0x7f0a20004d00 2023-12-19T18:18:10.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.104+0000 7f0a2bfff640 1 --2- 172.21.15.27:0/4110577185 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a1406a430 0x7f0a1406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.106+0000 7f0a2bfff640 1 --2- 172.21.15.27:0/4110577185 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a1406a430 0x7f0a1406c8f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0a340ccfd0 tx=0x7f0a1c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:10.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.108+0000 7f0a30ff9640 1 -- 172.21.15.27:0/4110577185 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0a24054540 con 0x7f0a20004d00 2023-12-19T18:18:10.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.343+0000 7f0a38e8a640 1 -- 172.21.15.27:0/4110577185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a340c93b0 con 0x7f0a20004d00 2023-12-19T18:18:10.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.344+0000 7f0a30ff9640 1 -- 172.21.15.27:0/4110577185 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f0a340c93b0 con 0x7f0a20004d00 2023-12-19T18:18:10.345 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:10.345 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/930391176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":930391176},{"type":"v1","addr":"172.21.15.192:6837","nonce":930391176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5179,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3606564673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3606564673},{"type":"v1","addr":"172.21.15.192:6836","nonce":3606564673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3604117097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3604117097},{"type":"v1","addr":"172.21.15.27:6834","nonce":3604117097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:14.250689+0000","modified":"2023-12-19T18:17:17.758670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3814458529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3814458529},{"type":"v1","addr":"172.21.15.27:6835","nonce":3814458529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T18:18:10.345 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-19T18:18:10.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.347+0000 7f0a38e8a640 1 -- 172.21.15.27:0/4110577185 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a1406a430 msgr2=0x7f0a1406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.347+0000 7f0a38e8a640 1 --2- 172.21.15.27:0/4110577185 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a1406a430 0x7f0a1406c8f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0a340ccfd0 tx=0x7f0a1c009040 comp rx=0 tx=0).stop 2023-12-19T18:18:10.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.347+0000 7f0a38e8a640 1 -- 172.21.15.27:0/4110577185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20004d00 msgr2=0x7f0a3414a030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.347+0000 7f0a38e8a640 1 --2- 172.21.15.27:0/4110577185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20004d00 0x7f0a3414a030 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f0a240108b0 tx=0x7f0a24010d70 comp rx=0 tx=0).stop 2023-12-19T18:18:10.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.347+0000 7f0a38e8a640 1 -- 172.21.15.27:0/4110577185 shutdown_connections 2023-12-19T18:18:10.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.347+0000 7f0a38e8a640 1 --2- 172.21.15.27:0/4110577185 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a1406a430 0x7f0a1406c8f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.347+0000 7f0a38e8a640 1 --2- 172.21.15.27:0/4110577185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a340c6cc0 0x7f0a340c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.347+0000 7f0a38e8a640 1 --2- 172.21.15.27:0/4110577185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a340c6860 0x7f0a340c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.347+0000 7f0a38e8a640 1 --2- 172.21.15.27:0/4110577185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a20004d00 0x7f0a3414a030 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.347+0000 7f0a38e8a640 1 -- 172.21.15.27:0/4110577185 >> 172.21.15.27:0/4110577185 conn(0x7f0a340cf7d0 msgr2=0x7f0a340d1010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:10.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.347+0000 7f0a38e8a640 1 -- 172.21.15.27:0/4110577185 shutdown_connections 2023-12-19T18:18:10.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.347+0000 7f0a38e8a640 1 -- 172.21.15.27:0/4110577185 wait complete. 2023-12-19T18:18:10.368 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:10.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.599+0000 7f8812207640 1 Processor -- start 2023-12-19T18:18:10.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.599+0000 7f8812207640 1 -- start start 2023-12-19T18:18:10.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.601+0000 7f8812207640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f880c154030 0x7f880c0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.601+0000 7f8812207640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880c0af630 0x7f880c0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.601+0000 7f8812207640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f880c0affd0 con 0x7f880c15d620 2023-12-19T18:18:10.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.601+0000 7f8812207640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f880c0cc190 con 0x7f880c154030 2023-12-19T18:18:10.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.601+0000 7f8812207640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880c0cb2a0 con 0x7f880c0af630 2023-12-19T18:18:10.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.601+0000 7f8811205640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f880c154030 0x7f880c0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.601+0000 7f8811205640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f880c154030 0x7f880c0cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58218/0 (socket says 172.21.15.27:58218) 2023-12-19T18:18:10.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.601+0000 7f8811205640 1 -- 172.21.15.27:0/3578485777 learned_addr learned my addr 172.21.15.27:0/3578485777 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:10.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.601+0000 7f8811a06640 1 --1- 172.21.15.27:0/3578485777 >> v1:172.21.15.27:6789/0 conn(0x7f880c15d620 0x7f880c15da60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33604/0 (socket says 172.21.15.27:33604) 2023-12-19T18:18:10.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.602+0000 7f8810a04640 1 --2- 172.21.15.27:0/3578485777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880c0af630 0x7f880c0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.602+0000 7f8811205640 1 -- 172.21.15.27:0/3578485777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880c0af630 msgr2=0x7f880c0afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:10.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.602+0000 7f8811205640 1 --2- 172.21.15.27:0/3578485777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880c0af630 0x7f880c0afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.602+0000 7f8811205640 1 -- 172.21.15.27:0/3578485777 >> v1:172.21.15.27:6789/0 conn(0x7f880c15d620 legacy=0x7f880c15da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.602+0000 7f8811205640 1 -- 172.21.15.27:0/3578485777 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880c0cb580 con 0x7f880c154030 2023-12-19T18:18:10.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.602+0000 7f8803fff640 1 -- 172.21.15.27:0/3578485777 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 985539794 0 0) 0x7f880c0affd0 con 0x7f880c15d620 2023-12-19T18:18:10.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.603+0000 7f8811205640 1 --2- 172.21.15.27:0/3578485777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c154030 0x7f880c0cd4f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8808009870 tx=0x7f880802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f93b66b470bddf76 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.604+0000 7f8803fff640 1 -- 172.21.15.27:0/3578485777 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808030840 con 0x7f880c154030 2023-12-19T18:18:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.604+0000 7f8803fff640 1 -- 172.21.15.27:0/3578485777 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88080309a0 con 0x7f880c154030 2023-12-19T18:18:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.604+0000 7f8803fff640 1 -- 172.21.15.27:0/3578485777 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8808030d30 con 0x7f880c154030 2023-12-19T18:18:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.604+0000 7f8812207640 1 -- 172.21.15.27:0/3578485777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c154030 msgr2=0x7f880c0cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.604+0000 7f8812207640 1 --2- 172.21.15.27:0/3578485777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c154030 0x7f880c0cd4f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f8808009870 tx=0x7f880802b980 comp rx=0 tx=0).stop 2023-12-19T18:18:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.604+0000 7f8812207640 1 -- 172.21.15.27:0/3578485777 shutdown_connections 2023-12-19T18:18:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.604+0000 7f8812207640 1 --2- 172.21.15.27:0/3578485777 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880c0af630 0x7f880c0afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.604+0000 7f8812207640 1 --2- 172.21.15.27:0/3578485777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c154030 0x7f880c0cd4f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.604+0000 7f8812207640 1 -- 172.21.15.27:0/3578485777 >> 172.21.15.27:0/3578485777 conn(0x7f880c069c70 msgr2=0x7f880c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:10.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.605+0000 7f8812207640 1 -- 172.21.15.27:0/3578485777 shutdown_connections 2023-12-19T18:18:10.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.605+0000 7f8812207640 1 -- 172.21.15.27:0/3578485777 wait complete. 2023-12-19T18:18:10.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.606+0000 7f8812207640 1 Processor -- start 2023-12-19T18:18:10.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.606+0000 7f8812207640 1 -- start start 2023-12-19T18:18:10.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.607+0000 7f8812207640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880c0af630 0x7f880c0ca4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.607+0000 7f8812207640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c154030 0x7f880c0c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.607+0000 7f8812207640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f880c15d620 0x7f880c1452a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.607+0000 7f8812207640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c0c68f0 con 0x7f880c154030 2023-12-19T18:18:10.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.607+0000 7f8812207640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f880c06f790 con 0x7f880c15d620 2023-12-19T18:18:10.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.607+0000 7f8812207640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880c06fa50 con 0x7f880c0af630 2023-12-19T18:18:10.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.608+0000 7f8810a04640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c154030 0x7f880c0c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.608+0000 7f8810a04640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c154030 0x7f880c0c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58220/0 (socket says 172.21.15.27:58220) 2023-12-19T18:18:10.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.608+0000 7f8810a04640 1 -- 172.21.15.27:0/3173997022 learned_addr learned my addr 172.21.15.27:0/3173997022 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:10.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.608+0000 7f8811a06640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f880c15d620 0x7f880c1452a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.608+0000 7f8811205640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880c0af630 0x7f880c0ca4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.608+0000 7f8810a04640 1 -- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880c0af630 msgr2=0x7f880c0ca4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.608+0000 7f8810a04640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880c0af630 0x7f880c0ca4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.608+0000 7f8810a04640 1 -- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f880c15d620 msgr2=0x7f880c1452a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.608+0000 7f8810a04640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f880c15d620 0x7f880c1452a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.608+0000 7f8810a04640 1 -- 172.21.15.27:0/3173997022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88080094d0 con 0x7f880c154030 2023-12-19T18:18:10.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.608+0000 7f8811a06640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f880c15d620 0x7f880c1452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:10.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.609+0000 7f8810a04640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c154030 0x7f880c0c63b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00e870 tx=0x7f87fc00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:10.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.609+0000 7f8801ffb640 1 -- 172.21.15.27:0/3173997022 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc0094f0 con 0x7f880c154030 2023-12-19T18:18:10.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.609+0000 7f8801ffb640 1 -- 172.21.15.27:0/3173997022 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87fc00c940 con 0x7f880c154030 2023-12-19T18:18:10.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.609+0000 7f8812207640 1 -- 172.21.15.27:0/3173997022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f880c06fd30 con 0x7f880c154030 2023-12-19T18:18:10.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.609+0000 7f8801ffb640 1 -- 172.21.15.27:0/3173997022 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87fc00ccf0 con 0x7f880c154030 2023-12-19T18:18:10.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.609+0000 7f8812207640 1 -- 172.21.15.27:0/3173997022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f880c0702f0 con 0x7f880c154030 2023-12-19T18:18:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.610+0000 7f8812207640 1 -- 172.21.15.27:0/3173997022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f880c070480 con 0x7f880c154030 2023-12-19T18:18:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.611+0000 7f8801ffb640 1 -- 172.21.15.27:0/3173997022 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f87fc013470 con 0x7f880c154030 2023-12-19T18:18:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.612+0000 7f8801ffb640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f87f006a430 0x7f87f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.612+0000 7f8801ffb640 1 -- 172.21.15.27:0/3173997022 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f87fc088b50 con 0x7f880c154030 2023-12-19T18:18:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.613+0000 7f8811205640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f87f006a430 0x7f87f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.614+0000 7f8811205640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f87f006a430 0x7f87f006c8f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8808036000 tx=0x7f880803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:10.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.617+0000 7f8801ffb640 1 -- 172.21.15.27:0/3173997022 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f87fc054010 con 0x7f880c154030 2023-12-19T18:18:10.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.812+0000 7f8812207640 1 -- 172.21.15.27:0/3173997022 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f880c04e840 con 0x7f880c154030 2023-12-19T18:18:10.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.814+0000 7f8801ffb640 1 -- 172.21.15.27:0/3173997022 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f87fc017090 con 0x7f880c154030 2023-12-19T18:18:10.815 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:10.815 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":63,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:17:56.605695+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:12.527441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:13.291755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.8","mappings":[{"from":5,"to":7}]},{"pgid":"7.9","mappings":[{"from":5,"to":7}]},{"pgid":"7.20","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:10.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.817+0000 7f8812207640 1 -- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f87f006a430 msgr2=0x7f87f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.817+0000 7f8812207640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f87f006a430 0x7f87f006c8f0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8808036000 tx=0x7f880803b040 comp rx=0 tx=0).stop 2023-12-19T18:18:10.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.817+0000 7f8812207640 1 -- 172.21.15.27:0/3173997022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c154030 msgr2=0x7f880c0c63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:10.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.817+0000 7f8812207640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c154030 0x7f880c0c63b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f87fc00e870 tx=0x7f87fc00ed30 comp rx=0 tx=0).stop 2023-12-19T18:18:10.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.817+0000 7f8812207640 1 -- 172.21.15.27:0/3173997022 shutdown_connections 2023-12-19T18:18:10.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.817+0000 7f8812207640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f87f006a430 0x7f87f006c8f0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.817+0000 7f8812207640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f880c15d620 0x7f880c1452a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.817+0000 7f8812207640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f880c154030 0x7f880c0c63b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.817+0000 7f8812207640 1 --2- 172.21.15.27:0/3173997022 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880c0af630 0x7f880c0ca4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:10.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.817+0000 7f8812207640 1 -- 172.21.15.27:0/3173997022 >> 172.21.15.27:0/3173997022 conn(0x7f880c069c70 msgr2=0x7f880c0c4550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:10.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.817+0000 7f8812207640 1 -- 172.21.15.27:0/3173997022 shutdown_connections 2023-12-19T18:18:10.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:10.817+0000 7f8812207640 1 -- 172.21.15.27:0/3173997022 wait complete. 2023-12-19T18:18:10.836 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:18:10.836 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:11.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.057+0000 7f5461a9e640 1 Processor -- start 2023-12-19T18:18:11.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.058+0000 7f5461a9e640 1 -- start start 2023-12-19T18:18:11.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.059+0000 7f5461a9e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f545c0b4ef0 0x7f545c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.059+0000 7f5461a9e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0b5e90 con 0x7f545c1604d0 2023-12-19T18:18:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.059+0000 7f5461a9e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f545c0d2850 con 0x7f545c0d2350 2023-12-19T18:18:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.059+0000 7f5461a9e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f545c0d2b30 con 0x7f545c0b4ef0 2023-12-19T18:18:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.059+0000 7f545bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f545c0b4ef0 0x7f545c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.059+0000 7f545bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f545c0b4ef0 0x7f545c0b6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58230/0 (socket says 172.21.15.27:58230) 2023-12-19T18:18:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.059+0000 7f545bfff640 1 -- 172.21.15.27:0/943892662 learned_addr learned my addr 172.21.15.27:0/943892662 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.059+0000 7f5460a9c640 1 --1- 172.21.15.27:0/943892662 >> v1:172.21.15.27:6789/0 conn(0x7f545c1604d0 0x7f545c0b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:33618/0 (socket says 172.21.15.27:33618) 2023-12-19T18:18:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.060+0000 7f545bfff640 1 -- 172.21.15.27:0/943892662 >> v1:172.21.15.192:6789/0 conn(0x7f545c0d2350 legacy=0x7f545c0b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:11.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.060+0000 7f545bfff640 1 -- 172.21.15.27:0/943892662 >> v1:172.21.15.27:6789/0 conn(0x7f545c1604d0 legacy=0x7f545c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.060+0000 7f545bfff640 1 -- 172.21.15.27:0/943892662 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f545c0cded0 con 0x7f545c0b4ef0 2023-12-19T18:18:11.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.061+0000 7f545bfff640 1 --2- 172.21.15.27:0/943892662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f545c0b4ef0 0x7f545c0b6300 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5448009870 tx=0x7f544802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d442f8f3a0e48c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:11.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.062+0000 7f545b7fe640 1 -- 172.21.15.27:0/943892662 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448030860 con 0x7f545c0b4ef0 2023-12-19T18:18:11.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.062+0000 7f545b7fe640 1 -- 172.21.15.27:0/943892662 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54480309c0 con 0x7f545c0b4ef0 2023-12-19T18:18:11.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.062+0000 7f545b7fe640 1 -- 172.21.15.27:0/943892662 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448030d50 con 0x7f545c0b4ef0 2023-12-19T18:18:11.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.062+0000 7f5461a9e640 1 -- 172.21.15.27:0/943892662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f545c0b4ef0 msgr2=0x7f545c0b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.062+0000 7f5461a9e640 1 --2- 172.21.15.27:0/943892662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f545c0b4ef0 0x7f545c0b6300 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f5448009870 tx=0x7f544802b9a0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.062+0000 7f5461a9e640 1 -- 172.21.15.27:0/943892662 shutdown_connections 2023-12-19T18:18:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.062+0000 7f5461a9e640 1 --2- 172.21.15.27:0/943892662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f545c0b4ef0 0x7f545c0b6300 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.062+0000 7f5461a9e640 1 -- 172.21.15.27:0/943892662 >> 172.21.15.27:0/943892662 conn(0x7f545c0cf7e0 msgr2=0x7f545c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.062+0000 7f5461a9e640 1 -- 172.21.15.27:0/943892662 shutdown_connections 2023-12-19T18:18:11.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.062+0000 7f5461a9e640 1 -- 172.21.15.27:0/943892662 wait complete. 2023-12-19T18:18:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.064+0000 7f5461a9e640 1 Processor -- start 2023-12-19T18:18:11.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.064+0000 7f5461a9e640 1 -- start start 2023-12-19T18:18:11.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.065+0000 7f5461a9e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f545c0b4ef0 0x7f545c0cc120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.065+0000 7f5461a9e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f545c0d2350 0x7f545c0b7c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.066+0000 7f5461a9e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f545c1604d0 0x7f545c0b81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.066+0000 7f5461a9e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c154910 con 0x7f545c0d2350 2023-12-19T18:18:11.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.066+0000 7f5461a9e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0cb070 con 0x7f545c0b4ef0 2023-12-19T18:18:11.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.066+0000 7f5461a9e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f545c0cc660 con 0x7f545c1604d0 2023-12-19T18:18:11.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.066+0000 7f545bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f545c0d2350 0x7f545c0b7c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.066+0000 7f545bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f545c0d2350 0x7f545c0b7c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58232/0 (socket says 172.21.15.27:58232) 2023-12-19T18:18:11.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.066+0000 7f545bfff640 1 -- 172.21.15.27:0/2550329463 learned_addr learned my addr 172.21.15.27:0/2550329463 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:11.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.066+0000 7f545bfff640 1 -- 172.21.15.27:0/2550329463 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f545c1604d0 msgr2=0x7f545c0b81b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.066+0000 7f545bfff640 1 --2- 172.21.15.27:0/2550329463 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f545c1604d0 0x7f545c0b81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.066+0000 7f545bfff640 1 -- 172.21.15.27:0/2550329463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f545c0b4ef0 msgr2=0x7f545c0cc120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.066+0000 7f545bfff640 1 --2- 172.21.15.27:0/2550329463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f545c0b4ef0 0x7f545c0cc120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.066+0000 7f545bfff640 1 -- 172.21.15.27:0/2550329463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54480094d0 con 0x7f545c0d2350 2023-12-19T18:18:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.067+0000 7f545bfff640 1 --2- 172.21.15.27:0/2550329463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f545c0d2350 0x7f545c0b7c70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5448009870 tx=0x7f5448004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.067+0000 7f54597fa640 1 -- 172.21.15.27:0/2550329463 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5448030810 con 0x7f545c0d2350 2023-12-19T18:18:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.067+0000 7f54597fa640 1 -- 172.21.15.27:0/2550329463 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5448043400 con 0x7f545c0d2350 2023-12-19T18:18:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.067+0000 7f54597fa640 1 -- 172.21.15.27:0/2550329463 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54480437b0 con 0x7f545c0d2350 2023-12-19T18:18:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.067+0000 7f5461a9e640 1 -- 172.21.15.27:0/2550329463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f545c0caab0 con 0x7f545c0d2350 2023-12-19T18:18:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.067+0000 7f5461a9e640 1 -- 172.21.15.27:0/2550329463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f545c0c6a10 con 0x7f545c0d2350 2023-12-19T18:18:11.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.068+0000 7f5461a9e640 1 -- 172.21.15.27:0/2550329463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f545c0c6870 con 0x7f545c0d2350 2023-12-19T18:18:11.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.069+0000 7f54597fa640 1 -- 172.21.15.27:0/2550329463 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5448043910 con 0x7f545c0d2350 2023-12-19T18:18:11.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.070+0000 7f54597fa640 1 --2- 172.21.15.27:0/2550329463 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f544006a430 0x7f544006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.070+0000 7f54597fa640 1 -- 172.21.15.27:0/2550329463 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f544803e070 con 0x7f545c0d2350 2023-12-19T18:18:11.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.071+0000 7f5460a9c640 1 --2- 172.21.15.27:0/2550329463 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f544006a430 0x7f544006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.072+0000 7f5460a9c640 1 --2- 172.21.15.27:0/2550329463 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f544006a430 0x7f544006c8f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5450000f60 tx=0x7f5450007a00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:11.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.074+0000 7f54597fa640 1 -- 172.21.15.27:0/2550329463 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f545c0c6870 con 0x7f545c0d2350 2023-12-19T18:18:11.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.304+0000 7f5461a9e640 1 -- 172.21.15.27:0/2550329463 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f545c0c6870 con 0x7f545c0d2350 2023-12-19T18:18:11.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.305+0000 7f54597fa640 1 -- 172.21.15.27:0/2550329463 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f545c0c6870 con 0x7f545c0d2350 2023-12-19T18:18:11.305 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:11.305 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/930391176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":930391176},{"type":"v1","addr":"172.21.15.192:6837","nonce":930391176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5179,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3606564673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3606564673},{"type":"v1","addr":"172.21.15.192:6836","nonce":3606564673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3604117097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3604117097},{"type":"v1","addr":"172.21.15.27:6834","nonce":3604117097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:14.250689+0000","modified":"2023-12-19T18:17:17.758670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3814458529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3814458529},{"type":"v1","addr":"172.21.15.27:6835","nonce":3814458529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T18:18:11.306 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-19T18:18:11.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.308+0000 7f5461a9e640 1 -- 172.21.15.27:0/2550329463 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f544006a430 msgr2=0x7f544006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.308+0000 7f5461a9e640 1 --2- 172.21.15.27:0/2550329463 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f544006a430 0x7f544006c8f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5450000f60 tx=0x7f5450007a00 comp rx=0 tx=0).stop 2023-12-19T18:18:11.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.308+0000 7f5461a9e640 1 -- 172.21.15.27:0/2550329463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f545c0d2350 msgr2=0x7f545c0b7c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.308+0000 7f5461a9e640 1 --2- 172.21.15.27:0/2550329463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f545c0d2350 0x7f545c0b7c70 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f5448009870 tx=0x7f5448004730 comp rx=0 tx=0).stop 2023-12-19T18:18:11.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.308+0000 7f5461a9e640 1 -- 172.21.15.27:0/2550329463 shutdown_connections 2023-12-19T18:18:11.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.308+0000 7f5461a9e640 1 --2- 172.21.15.27:0/2550329463 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f544006a430 0x7f544006c8f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.308+0000 7f5461a9e640 1 --2- 172.21.15.27:0/2550329463 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f545c1604d0 0x7f545c0b81b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.308+0000 7f5461a9e640 1 --2- 172.21.15.27:0/2550329463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f545c0d2350 0x7f545c0b7c70 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.308+0000 7f5461a9e640 1 --2- 172.21.15.27:0/2550329463 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f545c0b4ef0 0x7f545c0cc120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.308+0000 7f5461a9e640 1 -- 172.21.15.27:0/2550329463 >> 172.21.15.27:0/2550329463 conn(0x7f545c0cf7e0 msgr2=0x7f545c0b2910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:11.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.308+0000 7f5461a9e640 1 -- 172.21.15.27:0/2550329463 shutdown_connections 2023-12-19T18:18:11.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.308+0000 7f5461a9e640 1 -- 172.21.15.27:0/2550329463 wait complete. 2023-12-19T18:18:11.327 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:11.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.547+0000 7f666751b640 1 Processor -- start 2023-12-19T18:18:11.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.547+0000 7f666751b640 1 -- start start 2023-12-19T18:18:11.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.550+0000 7f666751b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6660161010 0x7f6660167c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.550+0000 7f666751b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6660145d60 con 0x7f66600c8f90 2023-12-19T18:18:11.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.550+0000 7f666751b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6660161510 con 0x7f66600cb300 2023-12-19T18:18:11.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.550+0000 7f666751b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66600c4480 con 0x7f6660161010 2023-12-19T18:18:11.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.550+0000 7f6666d1a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6660161010 0x7f6660167c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.550+0000 7f6666d1a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6660161010 0x7f6660167c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58240/0 (socket says 172.21.15.27:58240) 2023-12-19T18:18:11.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.550+0000 7f6666d1a640 1 -- 172.21.15.27:0/1528204836 learned_addr learned my addr 172.21.15.27:0/1528204836 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:11.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.551+0000 7f6666d1a640 1 -- 172.21.15.27:0/1528204836 >> v1:172.21.15.192:6789/0 conn(0x7f66600cb300 legacy=0x7f66600cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.551+0000 7f6666d1a640 1 -- 172.21.15.27:0/1528204836 >> v1:172.21.15.27:6789/0 conn(0x7f66600c8f90 legacy=0x7f6660160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.551+0000 7f6666d1a640 1 -- 172.21.15.27:0/1528204836 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66601617f0 con 0x7f6660161010 2023-12-19T18:18:11.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.552+0000 7f6666d1a640 1 --2- 172.21.15.27:0/1528204836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6660161010 0x7f6660167c90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6658009e80 tx=0x7f665802bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d95a8a802cd8d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:11.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.552+0000 7f6665517640 1 -- 172.21.15.27:0/1528204836 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665802cb90 con 0x7f6660161010 2023-12-19T18:18:11.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.552+0000 7f6665517640 1 -- 172.21.15.27:0/1528204836 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665802ccf0 con 0x7f6660161010 2023-12-19T18:18:11.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.552+0000 7f6665517640 1 -- 172.21.15.27:0/1528204836 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6658034610 con 0x7f6660161010 2023-12-19T18:18:11.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.553+0000 7f666751b640 1 -- 172.21.15.27:0/1528204836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6660161010 msgr2=0x7f6660167c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.553+0000 7f666751b640 1 --2- 172.21.15.27:0/1528204836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6660161010 0x7f6660167c90 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f6658009e80 tx=0x7f665802bff0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.553+0000 7f666751b640 1 -- 172.21.15.27:0/1528204836 shutdown_connections 2023-12-19T18:18:11.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.553+0000 7f666751b640 1 --2- 172.21.15.27:0/1528204836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6660161010 0x7f6660167c90 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.553+0000 7f666751b640 1 -- 172.21.15.27:0/1528204836 >> 172.21.15.27:0/1528204836 conn(0x7f6660069c70 msgr2=0x7f666006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:11.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.553+0000 7f666751b640 1 -- 172.21.15.27:0/1528204836 shutdown_connections 2023-12-19T18:18:11.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.553+0000 7f666751b640 1 -- 172.21.15.27:0/1528204836 wait complete. 2023-12-19T18:18:11.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.555+0000 7f666751b640 1 Processor -- start 2023-12-19T18:18:11.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.555+0000 7f666751b640 1 -- start start 2023-12-19T18:18:11.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.556+0000 7f666751b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66600c8f90 0x7f666015c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.556+0000 7f666751b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66600cb300 0x7f66600b1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.556+0000 7f666751b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6660161010 0x7f666015f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.556+0000 7f666751b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666015d690 con 0x7f66600c8f90 2023-12-19T18:18:11.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.556+0000 7f666751b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666015dc10 con 0x7f66600cb300 2023-12-19T18:18:11.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.556+0000 7f666751b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66600b2a90 con 0x7f6660161010 2023-12-19T18:18:11.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.556+0000 7f6666519640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66600c8f90 0x7f666015c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.557+0000 7f6666519640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66600c8f90 0x7f666015c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58248/0 (socket says 172.21.15.27:58248) 2023-12-19T18:18:11.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.557+0000 7f6666519640 1 -- 172.21.15.27:0/2044739406 learned_addr learned my addr 172.21.15.27:0/2044739406 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:11.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.557+0000 7f6666d1a640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6660161010 0x7f666015f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.557+0000 7f6666519640 1 -- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6660161010 msgr2=0x7f666015f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.557+0000 7f6666519640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6660161010 0x7f666015f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.557+0000 7f6665d18640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66600cb300 0x7f66600b1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.557+0000 7f6666519640 1 -- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66600cb300 msgr2=0x7f66600b1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.557+0000 7f6666519640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66600cb300 0x7f66600b1fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.557+0000 7f6666519640 1 -- 172.21.15.27:0/2044739406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6658009b30 con 0x7f66600c8f90 2023-12-19T18:18:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.557+0000 7f6666d1a640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6660161010 0x7f666015f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.557+0000 7f6666519640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66600c8f90 0x7f666015c300 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f665c00c400 tx=0x7f665c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.558+0000 7f6656ffd640 1 -- 172.21.15.27:0/2044739406 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c0052d0 con 0x7f66600c8f90 2023-12-19T18:18:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.558+0000 7f6656ffd640 1 -- 172.21.15.27:0/2044739406 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665c005430 con 0x7f66600c8f90 2023-12-19T18:18:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.558+0000 7f6656ffd640 1 -- 172.21.15.27:0/2044739406 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665c003830 con 0x7f66600c8f90 2023-12-19T18:18:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.558+0000 7f666751b640 1 -- 172.21.15.27:0/2044739406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66600b2d70 con 0x7f66600c8f90 2023-12-19T18:18:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.558+0000 7f666751b640 1 -- 172.21.15.27:0/2044739406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666015c840 con 0x7f66600c8f90 2023-12-19T18:18:11.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.559+0000 7f666751b640 1 -- 172.21.15.27:0/2044739406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666004e840 con 0x7f66600c8f90 2023-12-19T18:18:11.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.560+0000 7f6656ffd640 1 -- 172.21.15.27:0/2044739406 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f665c003990 con 0x7f66600c8f90 2023-12-19T18:18:11.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.561+0000 7f6656ffd640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f664406a430 0x7f664406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:11.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.561+0000 7f6656ffd640 1 -- 172.21.15.27:0/2044739406 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f665c087f50 con 0x7f66600c8f90 2023-12-19T18:18:11.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.562+0000 7f6665d18640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f664406a430 0x7f664406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:11.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.563+0000 7f6665d18640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f664406a430 0x7f664406c8f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f666013b180 tx=0x7f6650009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:11.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.565+0000 7f6656ffd640 1 -- 172.21.15.27:0/2044739406 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f665c0534b0 con 0x7f66600c8f90 2023-12-19T18:18:11.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.800+0000 7f666751b640 1 -- 172.21.15.27:0/2044739406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66600adc50 con 0x7f66600c8f90 2023-12-19T18:18:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.801+0000 7f6656ffd640 1 -- 172.21.15.27:0/2044739406 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+4712 (secure 0 0 0) 0x7f665c056ad0 con 0x7f66600c8f90 2023-12-19T18:18:11.801 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:11.801 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":23,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/930391176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":930391176},{"type":"v1","addr":"172.21.15.192:6837","nonce":930391176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5179,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3606564673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3606564673},{"type":"v1","addr":"172.21.15.192:6836","nonce":3606564673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3604117097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3604117097},{"type":"v1","addr":"172.21.15.27:6834","nonce":3604117097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:17:14.250689+0000","modified":"2023-12-19T18:17:17.758670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5679},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5679":{"gid":5679,"name":"c","rank":0,"incarnation":19,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/3814458529","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":3814458529},{"type":"v1","addr":"172.21.15.27:6835","nonce":3814458529}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-19T18:18:11.801 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 23 2023-12-19T18:18:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.803+0000 7f666751b640 1 -- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f664406a430 msgr2=0x7f664406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.803+0000 7f666751b640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f664406a430 0x7f664406c8f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f666013b180 tx=0x7f6650009040 comp rx=0 tx=0).stop 2023-12-19T18:18:11.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.803+0000 7f666751b640 1 -- 172.21.15.27:0/2044739406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66600c8f90 msgr2=0x7f666015c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.803+0000 7f666751b640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66600c8f90 0x7f666015c300 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f665c00c400 tx=0x7f665c00c8c0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.803+0000 7f666751b640 1 -- 172.21.15.27:0/2044739406 shutdown_connections 2023-12-19T18:18:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.803+0000 7f666751b640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f664406a430 0x7f664406c8f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.803+0000 7f666751b640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6660161010 0x7f666015f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.803+0000 7f666751b640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66600cb300 0x7f66600b1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.803+0000 7f666751b640 1 --2- 172.21.15.27:0/2044739406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66600c8f90 0x7f666015c300 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.803+0000 7f666751b640 1 -- 172.21.15.27:0/2044739406 >> 172.21.15.27:0/2044739406 conn(0x7f6660069c70 msgr2=0x7f6660070c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.803+0000 7f666751b640 1 -- 172.21.15.27:0/2044739406 shutdown_connections 2023-12-19T18:18:11.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:11.803+0000 7f666751b640 1 -- 172.21.15.27:0/2044739406 wait complete. 2023-12-19T18:18:11.824 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:12.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.048+0000 7f189bfff640 1 Processor -- start 2023-12-19T18:18:12.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.048+0000 7f189bfff640 1 -- start start 2023-12-19T18:18:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.050+0000 7f189bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f189c1553a0 0x7f189c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.050+0000 7f189bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f189c15ee30 0x7f189c0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.050+0000 7f189bfff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f189c0b0330 con 0x7f189c0afeb0 2023-12-19T18:18:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.050+0000 7f189bfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189c0cbcf0 con 0x7f189c15ee30 2023-12-19T18:18:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.050+0000 7f189bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c0cb350 con 0x7f189c1553a0 2023-12-19T18:18:12.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.051+0000 7f189affd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f189c1553a0 0x7f189c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.051+0000 7f189b7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f189c15ee30 0x7f189c0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.051+0000 7f189b7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f189c15ee30 0x7f189c0c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:54816/0 (socket says 172.21.15.27:54816) 2023-12-19T18:18:12.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.051+0000 7f189b7fe640 1 -- 172.21.15.27:0/1719064182 learned_addr learned my addr 172.21.15.27:0/1719064182 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.052+0000 7f189b7fe640 1 -- 172.21.15.27:0/1719064182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f189c1553a0 msgr2=0x7f189c0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.052+0000 7f189b7fe640 1 --2- 172.21.15.27:0/1719064182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f189c1553a0 0x7f189c0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.052+0000 7f189b7fe640 1 -- 172.21.15.27:0/1719064182 >> v1:172.21.15.192:6789/0 conn(0x7f189c0afeb0 legacy=0x7f189c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.052+0000 7f189b7fe640 1 -- 172.21.15.27:0/1719064182 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f189c0cb630 con 0x7f189c15ee30 2023-12-19T18:18:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.053+0000 7f189b7fe640 1 --2- 172.21.15.27:0/1719064182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f189c15ee30 0x7f189c0c39d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1894009870 tx=0x7f189402b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b13f3308937818a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.053+0000 7f189a7fc640 1 -- 172.21.15.27:0/1719064182 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894031870 con 0x7f189c15ee30 2023-12-19T18:18:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.053+0000 7f189a7fc640 1 -- 172.21.15.27:0/1719064182 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18940319d0 con 0x7f189c15ee30 2023-12-19T18:18:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.053+0000 7f189a7fc640 1 -- 172.21.15.27:0/1719064182 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894031d60 con 0x7f189c15ee30 2023-12-19T18:18:12.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.054+0000 7f189bfff640 1 -- 172.21.15.27:0/1719064182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f189c15ee30 msgr2=0x7f189c0c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.054+0000 7f189bfff640 1 --2- 172.21.15.27:0/1719064182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f189c15ee30 0x7f189c0c39d0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f1894009870 tx=0x7f189402b910 comp rx=0 tx=0).stop 2023-12-19T18:18:12.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.054+0000 7f189bfff640 1 -- 172.21.15.27:0/1719064182 shutdown_connections 2023-12-19T18:18:12.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.054+0000 7f189bfff640 1 --2- 172.21.15.27:0/1719064182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f189c15ee30 0x7f189c0c39d0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.054+0000 7f189bfff640 1 --2- 172.21.15.27:0/1719064182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f189c1553a0 0x7f189c0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.054+0000 7f189bfff640 1 -- 172.21.15.27:0/1719064182 >> 172.21.15.27:0/1719064182 conn(0x7f189c069c70 msgr2=0x7f189c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.054+0000 7f189bfff640 1 -- 172.21.15.27:0/1719064182 shutdown_connections 2023-12-19T18:18:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.054+0000 7f189bfff640 1 -- 172.21.15.27:0/1719064182 wait complete. 2023-12-19T18:18:12.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.056+0000 7f189bfff640 1 Processor -- start 2023-12-19T18:18:12.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.056+0000 7f189bfff640 1 -- start start 2023-12-19T18:18:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.057+0000 7f189bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f189c0afeb0 0x7f189c0c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.057+0000 7f189bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f189c1553a0 0x7f189c1472f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.057+0000 7f189bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c15ee30 0x7f189c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.057+0000 7f189bfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c148820 con 0x7f189c15ee30 2023-12-19T18:18:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.057+0000 7f189bfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c0c6860 con 0x7f189c1553a0 2023-12-19T18:18:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.057+0000 7f189bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f189c147c10 con 0x7f189c0afeb0 2023-12-19T18:18:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.057+0000 7f189b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c15ee30 0x7f189c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.057+0000 7f189b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c15ee30 0x7f189c15d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52804/0 (socket says 172.21.15.27:52804) 2023-12-19T18:18:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.057+0000 7f189b7fe640 1 -- 172.21.15.27:0/1198291618 learned_addr learned my addr 172.21.15.27:0/1198291618 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.058+0000 7f189affd640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f189c0afeb0 0x7f189c0c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.058+0000 7f1893fff640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f189c1553a0 0x7f189c1472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.058+0000 7f189b7fe640 1 -- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f189c0afeb0 msgr2=0x7f189c0c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.058+0000 7f189b7fe640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f189c0afeb0 0x7f189c0c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.058+0000 7f189b7fe640 1 -- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f189c1553a0 msgr2=0x7f189c1472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.058+0000 7f189b7fe640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f189c1553a0 0x7f189c1472f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.058+0000 7f189b7fe640 1 -- 172.21.15.27:0/1198291618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18940094d0 con 0x7f189c15ee30 2023-12-19T18:18:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.058+0000 7f189b7fe640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c15ee30 0x7f189c15d320 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f18940090b0 tx=0x7f18940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.058+0000 7f18937fe640 1 -- 172.21.15.27:0/1198291618 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189403abe0 con 0x7f189c15ee30 2023-12-19T18:18:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.059+0000 7f18937fe640 1 -- 172.21.15.27:0/1198291618 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f189403ad40 con 0x7f189c15ee30 2023-12-19T18:18:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.059+0000 7f18937fe640 1 -- 172.21.15.27:0/1198291618 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1894044630 con 0x7f189c15ee30 2023-12-19T18:18:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.059+0000 7f189bfff640 1 -- 172.21.15.27:0/1198291618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189c147ef0 con 0x7f189c15ee30 2023-12-19T18:18:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.059+0000 7f189bfff640 1 -- 172.21.15.27:0/1198291618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189c1484b0 con 0x7f189c15ee30 2023-12-19T18:18:12.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.060+0000 7f189bfff640 1 -- 172.21.15.27:0/1198291618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189c06c780 con 0x7f189c15ee30 2023-12-19T18:18:12.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.061+0000 7f18937fe640 1 -- 172.21.15.27:0/1198291618 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f189404d450 con 0x7f189c15ee30 2023-12-19T18:18:12.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.062+0000 7f18937fe640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f187c06a430 0x7f187c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.062+0000 7f18937fe640 1 -- 172.21.15.27:0/1198291618 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f189402d040 con 0x7f189c15ee30 2023-12-19T18:18:12.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.062+0000 7f189affd640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f187c06a430 0x7f187c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.064+0000 7f189affd640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f187c06a430 0x7f187c06c8f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f188c009ad0 tx=0x7f188c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:12.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.066+0000 7f18937fe640 1 -- 172.21.15.27:0/1198291618 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1894076400 con 0x7f189c15ee30 2023-12-19T18:18:12.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.258+0000 7f189bfff640 1 -- 172.21.15.27:0/1198291618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f189c04e840 con 0x7f189c15ee30 2023-12-19T18:18:12.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.261+0000 7f18937fe640 1 -- 172.21.15.27:0/1198291618 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v63) v1 ==== 74+0+16274 (secure 0 0 0) 0x7f1894035070 con 0x7f189c15ee30 2023-12-19T18:18:12.261 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:12.261 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":63,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:17:56.605695+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:17:12.527441+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-19T18:17:13.291755+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.6","mappings":[{"from":5,"to":7}]},{"pgid":"7.8","mappings":[{"from":5,"to":7}]},{"pgid":"7.9","mappings":[{"from":5,"to":7}]},{"pgid":"7.20","mappings":[{"from":5,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:12.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.263+0000 7f189bfff640 1 -- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f187c06a430 msgr2=0x7f187c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.263+0000 7f189bfff640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f187c06a430 0x7f187c06c8f0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f188c009ad0 tx=0x7f188c008040 comp rx=0 tx=0).stop 2023-12-19T18:18:12.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.263+0000 7f189bfff640 1 -- 172.21.15.27:0/1198291618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c15ee30 msgr2=0x7f189c15d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.263+0000 7f189bfff640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c15ee30 0x7f189c15d320 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f18940090b0 tx=0x7f18940046c0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.263+0000 7f189bfff640 1 -- 172.21.15.27:0/1198291618 shutdown_connections 2023-12-19T18:18:12.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.263+0000 7f189bfff640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f187c06a430 0x7f187c06c8f0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.263+0000 7f189bfff640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c15ee30 0x7f189c15d320 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.263+0000 7f189bfff640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f189c1553a0 0x7f189c1472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.263+0000 7f189bfff640 1 --2- 172.21.15.27:0/1198291618 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f189c0afeb0 0x7f189c0c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.263+0000 7f189bfff640 1 -- 172.21.15.27:0/1198291618 >> 172.21.15.27:0/1198291618 conn(0x7f189c069c70 msgr2=0x7f189c06c070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:12.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.263+0000 7f189bfff640 1 -- 172.21.15.27:0/1198291618 shutdown_connections 2023-12-19T18:18:12.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.263+0000 7f189bfff640 1 -- 172.21.15.27:0/1198291618 wait complete. 2023-12-19T18:18:12.285 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:18:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.522+0000 7fcc697f0640 1 Processor -- start 2023-12-19T18:18:12.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.523+0000 7fcc697f0640 1 -- start start 2023-12-19T18:18:12.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.524+0000 7fcc697f0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcc6415d260 0x7fcc6406a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.524+0000 7fcc697f0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc640d2af0 0x7fcc6406a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.524+0000 7fcc697f0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc640b71b0 con 0x7fcc640d0e70 2023-12-19T18:18:12.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.524+0000 7fcc697f0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc640cdaf0 con 0x7fcc6415d260 2023-12-19T18:18:12.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.524+0000 7fcc697f0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640d1a50 con 0x7fcc640d2af0 2023-12-19T18:18:12.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.524+0000 7fcc637fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcc640d0e70 0x7fcc640b49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58146/0 (socket says 172.21.15.27:58146) 2023-12-19T18:18:12.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.524+0000 7fcc637fe640 1 -- 172.21.15.27:0/2846664262 learned_addr learned my addr 172.21.15.27:0/2846664262 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:12.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.525+0000 7fcc627fc640 1 --2- 172.21.15.27:0/2846664262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc640d2af0 0x7fcc6406a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.525+0000 7fcc62ffd640 1 --2- 172.21.15.27:0/2846664262 >> v2:172.21.15.192:3300/0 conn(0x7fcc6415d260 0x7fcc6406a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.525+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4180193870 0 0) 0x7fcc5c001110 con 0x7fcc640d0e70 2023-12-19T18:18:12.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.525+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc50003520 con 0x7fcc640d0e70 2023-12-19T18:18:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.525+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1430260344 0 0) 0x7fcc50003520 con 0x7fcc640d0e70 2023-12-19T18:18:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.525+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc640d2af0 msgr2=0x7fcc6406a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.525+0000 7fcc61ffb640 1 --2- 172.21.15.27:0/2846664262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc640d2af0 0x7fcc6406a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.525+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 >> v2:172.21.15.192:3300/0 conn(0x7fcc6415d260 msgr2=0x7fcc6406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.525+0000 7fcc61ffb640 1 --2- 172.21.15.27:0/2846664262 >> v2:172.21.15.192:3300/0 conn(0x7fcc6415d260 0x7fcc6406a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.525+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640d1d30 con 0x7fcc640d0e70 2023-12-19T18:18:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.525+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fcc5c001800 con 0x7fcc640d0e70 2023-12-19T18:18:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.525+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 >> v1:172.21.15.27:6789/0 conn(0x7fcc640d0e70 legacy=0x7fcc640b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc61ffb640 1 --2- 172.21.15.27:0/2846664262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc50003970 0x7fcc50003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc61ffb640 1 --2- 172.21.15.27:0/2846664262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc640d2af0 0x7fcc500047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc61ffb640 1 --2- 172.21.15.27:0/2846664262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50004d00 0x7fcc50005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640cdaf0 con 0x7fcc50004d00 2023-12-19T18:18:12.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640d1a50 con 0x7fcc50003970 2023-12-19T18:18:12.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc5000b980 con 0x7fcc640d2af0 2023-12-19T18:18:12.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc637fe640 1 --2- 172.21.15.27:0/2846664262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50004d00 0x7fcc50005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc62ffd640 1 --2- 172.21.15.27:0/2846664262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc50003970 0x7fcc50003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc627fc640 1 --2- 172.21.15.27:0/2846664262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc640d2af0 0x7fcc500047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc637fe640 1 -- 172.21.15.27:0/2846664262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc640d2af0 msgr2=0x7fcc500047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:12.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc637fe640 1 --2- 172.21.15.27:0/2846664262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc640d2af0 0x7fcc500047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc637fe640 1 -- 172.21.15.27:0/2846664262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc50003970 msgr2=0x7fcc50003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:12.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc637fe640 1 --2- 172.21.15.27:0/2846664262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc50003970 0x7fcc50003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.526+0000 7fcc637fe640 1 -- 172.21.15.27:0/2846664262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc50003520 con 0x7fcc50004d00 2023-12-19T18:18:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.527+0000 7fcc637fe640 1 --2- 172.21.15.27:0/2846664262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50004d00 0x7fcc50005180 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00c210 tx=0x7fcc500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aeefc02140a3bdb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:12.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.528+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c02cae0 con 0x7fcc50004d00 2023-12-19T18:18:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.528+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c02cc60 con 0x7fcc50004d00 2023-12-19T18:18:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.528+0000 7fcc61ffb640 1 -- 172.21.15.27:0/2846664262 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c0058d0 con 0x7fcc50004d00 2023-12-19T18:18:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.528+0000 7fcc697f0640 1 -- 172.21.15.27:0/2846664262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50004d00 msgr2=0x7fcc50005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.528+0000 7fcc697f0640 1 --2- 172.21.15.27:0/2846664262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc50004d00 0x7fcc50005180 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00c210 tx=0x7fcc500057f0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.528+0000 7fcc637fe640 1 -- 172.21.15.27:0/2846664262 reap_dead start 2023-12-19T18:18:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.528+0000 7fcc697f0640 1 -- 172.21.15.27:0/2846664262 shutdown_connections 2023-12-19T18:18:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.528+0000 7fcc697f0640 1 -- 172.21.15.27:0/2846664262 >> 172.21.15.27:0/2846664262 conn(0x7fcc640d2f70 msgr2=0x7fcc640d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.528+0000 7fcc697f0640 1 -- 172.21.15.27:0/2846664262 shutdown_connections 2023-12-19T18:18:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.529+0000 7fcc697f0640 1 -- 172.21.15.27:0/2846664262 wait complete. 2023-12-19T18:18:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.530+0000 7fcc697f0640 1 Processor -- start 2023-12-19T18:18:12.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.530+0000 7fcc697f0640 1 -- start start 2023-12-19T18:18:12.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.531+0000 7fcc697f0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc50004d00 0x7fcc641535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.531+0000 7fcc697f0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc641f2990 0x7fcc641f6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.531+0000 7fcc697f0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cb5d0 0x7fcc640cba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.531+0000 7fcc697f0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc641f3a10 con 0x7fcc640cb5d0 2023-12-19T18:18:12.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.531+0000 7fcc697f0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc641f6ed0 con 0x7fcc641f2990 2023-12-19T18:18:12.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.531+0000 7fcc697f0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640cbfa0 con 0x7fcc50004d00 2023-12-19T18:18:12.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.532+0000 7fcc637fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cb5d0 0x7fcc640cba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.532+0000 7fcc637fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cb5d0 0x7fcc640cba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52830/0 (socket says 172.21.15.27:52830) 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.532+0000 7fcc637fe640 1 -- 172.21.15.27:0/487129221 learned_addr learned my addr 172.21.15.27:0/487129221 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.532+0000 7fcc62ffd640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc50004d00 0x7fcc641535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.532+0000 7fcc627fc640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc641f2990 0x7fcc641f6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.532+0000 7fcc637fe640 1 -- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc50004d00 msgr2=0x7fcc641535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.532+0000 7fcc637fe640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc50004d00 0x7fcc641535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.532+0000 7fcc637fe640 1 -- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc641f2990 msgr2=0x7fcc641f6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.532+0000 7fcc637fe640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc641f2990 0x7fcc641f6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.532+0000 7fcc637fe640 1 -- 172.21.15.27:0/487129221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc641f2e90 con 0x7fcc640cb5d0 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.532+0000 7fcc62ffd640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc50004d00 0x7fcc641535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.532+0000 7fcc627fc640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc641f2990 0x7fcc641f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.532+0000 7fcc637fe640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cb5d0 0x7fcc640cba60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00c210 tx=0x7fcc5c001c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.533+0000 7fcc5b7fe640 1 -- 172.21.15.27:0/487129221 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c03c040 con 0x7fcc640cb5d0 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.533+0000 7fcc5b7fe640 1 -- 172.21.15.27:0/487129221 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5c02caf0 con 0x7fcc640cb5d0 2023-12-19T18:18:12.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.533+0000 7fcc5b7fe640 1 -- 172.21.15.27:0/487129221 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5c007290 con 0x7fcc640cb5d0 2023-12-19T18:18:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.533+0000 7fcc697f0640 1 -- 172.21.15.27:0/487129221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc641f3170 con 0x7fcc640cb5d0 2023-12-19T18:18:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.533+0000 7fcc697f0640 1 -- 172.21.15.27:0/487129221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc641f3730 con 0x7fcc640cb5d0 2023-12-19T18:18:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.534+0000 7fcc697f0640 1 -- 172.21.15.27:0/487129221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc64066810 con 0x7fcc640cb5d0 2023-12-19T18:18:12.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.535+0000 7fcc5b7fe640 1 -- 172.21.15.27:0/487129221 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcc5c032070 con 0x7fcc640cb5d0 2023-12-19T18:18:12.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.536+0000 7fcc5b7fe640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcc4406a460 0x7fcc4406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:12.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.536+0000 7fcc5b7fe640 1 -- 172.21.15.27:0/487129221 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fcc5c0a9280 con 0x7fcc640cb5d0 2023-12-19T18:18:12.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.537+0000 7fcc62ffd640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcc4406a460 0x7fcc4406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:12.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.538+0000 7fcc62ffd640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcc4406a460 0x7fcc4406c920 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcc54007440 tx=0x7fcc5400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:12.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.540+0000 7fcc5b7fe640 1 -- 172.21.15.27:0/487129221 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcc5c074740 con 0x7fcc640cb5d0 2023-12-19T18:18:12.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.780+0000 7fcc697f0640 1 -- 172.21.15.27:0/487129221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcc640b8330 con 0x7fcc640cb5d0 2023-12-19T18:18:12.781 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:18:12.781+0000 7f8f3865c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:18:12.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.799+0000 7fcc5b7fe640 1 -- 172.21.15.27:0/487129221 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v24) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcc640b8330 con 0x7fcc640cb5d0 2023-12-19T18:18:12.799 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:18:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.802+0000 7fcc697f0640 1 -- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcc4406a460 msgr2=0x7fcc4406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.802+0000 7fcc697f0640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcc4406a460 0x7fcc4406c920 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fcc54007440 tx=0x7fcc5400b040 comp rx=0 tx=0).stop 2023-12-19T18:18:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.802+0000 7fcc697f0640 1 -- 172.21.15.27:0/487129221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cb5d0 msgr2=0x7fcc640cba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.802+0000 7fcc697f0640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cb5d0 0x7fcc640cba60 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c00c210 tx=0x7fcc5c001c00 comp rx=0 tx=0).stop 2023-12-19T18:18:12.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.802+0000 7fcc697f0640 1 -- 172.21.15.27:0/487129221 shutdown_connections 2023-12-19T18:18:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.802+0000 7fcc697f0640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcc4406a460 0x7fcc4406c920 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.802+0000 7fcc697f0640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc640cb5d0 0x7fcc640cba60 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.802+0000 7fcc697f0640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcc641f2990 0x7fcc641f6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.802+0000 7fcc697f0640 1 --2- 172.21.15.27:0/487129221 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcc50004d00 0x7fcc641535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.802+0000 7fcc697f0640 1 -- 172.21.15.27:0/487129221 >> 172.21.15.27:0/487129221 conn(0x7fcc640d2f70 msgr2=0x7fcc6415ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.802+0000 7fcc697f0640 1 -- 172.21.15.27:0/487129221 shutdown_connections 2023-12-19T18:18:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:12.802+0000 7fcc697f0640 1 -- 172.21.15.27:0/487129221 wait complete. 2023-12-19T18:18:12.822 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:18:12.884 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:18:13.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.029+0000 7fda8c85e640 1 Processor -- start 2023-12-19T18:18:13.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.029+0000 7fda8c85e640 1 -- start start 2023-12-19T18:18:13.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.031+0000 7fda8c85e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fda8815cfc0 0x7fda88160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.031+0000 7fda8c85e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fda880d27b0 0x7fda880cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.031+0000 7fda8c85e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda88161a50 con 0x7fda880cdb30 2023-12-19T18:18:13.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.031+0000 7fda8c85e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda880ce500 con 0x7fda880d27b0 2023-12-19T18:18:13.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.031+0000 7fda8c85e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda88161490 con 0x7fda8815cfc0 2023-12-19T18:18:13.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.031+0000 7fda8759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fda880d27b0 0x7fda880cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.031+0000 7fda8759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fda880d27b0 0x7fda880cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52848/0 (socket says 172.21.15.27:52848) 2023-12-19T18:18:13.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.031+0000 7fda8759e640 1 -- 172.21.15.27:0/3949742372 learned_addr learned my addr 172.21.15.27:0/3949742372 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:13.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.032+0000 7fda86d9d640 1 --2- 172.21.15.27:0/3949742372 >> v2:172.21.15.192:3300/0 conn(0x7fda8815cfc0 0x7fda88160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.032+0000 7fda8759e640 1 -- 172.21.15.27:0/3949742372 >> v2:172.21.15.192:3300/0 conn(0x7fda8815cfc0 msgr2=0x7fda88160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:13.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.032+0000 7fda8759e640 1 --2- 172.21.15.27:0/3949742372 >> v2:172.21.15.192:3300/0 conn(0x7fda8815cfc0 0x7fda88160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.032+0000 7fda8759e640 1 -- 172.21.15.27:0/3949742372 >> v1:172.21.15.192:6789/0 conn(0x7fda880cdb30 legacy=0x7fda880d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:13.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.032+0000 7fda8759e640 1 -- 172.21.15.27:0/3949742372 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda88161770 con 0x7fda880d27b0 2023-12-19T18:18:13.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.033+0000 7fda8759e640 1 --2- 172.21.15.27:0/3949742372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda880d27b0 0x7fda880cdfc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fda880b4fc0 tx=0x7fda8002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7f33846a21d6367 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:13.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.034+0000 7fda8659c640 1 -- 172.21.15.27:0/3949742372 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8002c9f0 con 0x7fda880d27b0 2023-12-19T18:18:13.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.034+0000 7fda8659c640 1 -- 172.21.15.27:0/3949742372 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8002cb50 con 0x7fda880d27b0 2023-12-19T18:18:13.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.034+0000 7fda8659c640 1 -- 172.21.15.27:0/3949742372 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda800343f0 con 0x7fda880d27b0 2023-12-19T18:18:13.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.034+0000 7fda8c85e640 1 -- 172.21.15.27:0/3949742372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda880d27b0 msgr2=0x7fda880cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.034+0000 7fda8c85e640 1 --2- 172.21.15.27:0/3949742372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda880d27b0 0x7fda880cdfc0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fda880b4fc0 tx=0x7fda8002be50 comp rx=0 tx=0).stop 2023-12-19T18:18:13.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.034+0000 7fda8c85e640 1 -- 172.21.15.27:0/3949742372 shutdown_connections 2023-12-19T18:18:13.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.034+0000 7fda8c85e640 1 --2- 172.21.15.27:0/3949742372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda880d27b0 0x7fda880cdfc0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.034+0000 7fda8c85e640 1 --2- 172.21.15.27:0/3949742372 >> v2:172.21.15.192:3300/0 conn(0x7fda8815cfc0 0x7fda88160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.034+0000 7fda8c85e640 1 -- 172.21.15.27:0/3949742372 >> 172.21.15.27:0/3949742372 conn(0x7fda88072e20 msgr2=0x7fda88073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:13.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.035+0000 7fda8c85e640 1 -- 172.21.15.27:0/3949742372 shutdown_connections 2023-12-19T18:18:13.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.035+0000 7fda8c85e640 1 -- 172.21.15.27:0/3949742372 wait complete. 2023-12-19T18:18:13.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.037+0000 7fda8c85e640 1 Processor -- start 2023-12-19T18:18:13.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.037+0000 7fda8c85e640 1 -- start start 2023-12-19T18:18:13.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.038+0000 7fda8c85e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda880cdb30 0x7fda880cc5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.038+0000 7fda8c85e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda880d27b0 0x7fda8815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.038+0000 7fda8c85e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda8815cfc0 0x7fda8815b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.038+0000 7fda8c85e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda880c96c0 con 0x7fda880cdb30 2023-12-19T18:18:13.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.038+0000 7fda8c85e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda8815a6c0 con 0x7fda880d27b0 2023-12-19T18:18:13.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.038+0000 7fda8c85e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda8815a9a0 con 0x7fda8815cfc0 2023-12-19T18:18:13.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.038+0000 7fda86d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda880cdb30 0x7fda880cc5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.038+0000 7fda86d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda880cdb30 0x7fda880cc5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52858/0 (socket says 172.21.15.27:52858) 2023-12-19T18:18:13.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.038+0000 7fda86d9d640 1 -- 172.21.15.27:0/581487555 learned_addr learned my addr 172.21.15.27:0/581487555 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:13.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.039+0000 7fda7e59c640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda880d27b0 0x7fda8815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.039+0000 7fda8759e640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda8815cfc0 0x7fda8815b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.039+0000 7fda86d9d640 1 -- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda8815cfc0 msgr2=0x7fda8815b760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.039+0000 7fda86d9d640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda8815cfc0 0x7fda8815b760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.039+0000 7fda86d9d640 1 -- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda880d27b0 msgr2=0x7fda8815fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.039+0000 7fda86d9d640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda880d27b0 0x7fda8815fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.039+0000 7fda86d9d640 1 -- 172.21.15.27:0/581487555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda80009b30 con 0x7fda880cdb30 2023-12-19T18:18:13.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.039+0000 7fda86d9d640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda880cdb30 0x7fda880cc5d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fda7800e870 tx=0x7fda7800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:13.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.040+0000 7fda7ffff640 1 -- 172.21.15.27:0/581487555 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda780094f0 con 0x7fda880cdb30 2023-12-19T18:18:13.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.040+0000 7fda7ffff640 1 -- 172.21.15.27:0/581487555 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda7800c940 con 0x7fda880cdb30 2023-12-19T18:18:13.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.040+0000 7fda7ffff640 1 -- 172.21.15.27:0/581487555 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda7800ccf0 con 0x7fda880cdb30 2023-12-19T18:18:13.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.040+0000 7fda8c85e640 1 -- 172.21.15.27:0/581487555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda8815ac80 con 0x7fda880cdb30 2023-12-19T18:18:13.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.040+0000 7fda8c85e640 1 -- 172.21.15.27:0/581487555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda8815b240 con 0x7fda880cdb30 2023-12-19T18:18:13.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.041+0000 7fda8c85e640 1 -- 172.21.15.27:0/581487555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda8804e840 con 0x7fda880cdb30 2023-12-19T18:18:13.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.042+0000 7fda7ffff640 1 -- 172.21.15.27:0/581487555 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fda78013470 con 0x7fda880cdb30 2023-12-19T18:18:13.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.043+0000 7fda7ffff640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda6c06a430 0x7fda6c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:13.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.043+0000 7fda7ffff640 1 -- 172.21.15.27:0/581487555 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5940+0+0 (secure 0 0 0) 0x7fda78088b10 con 0x7fda880cdb30 2023-12-19T18:18:13.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.044+0000 7fda7e59c640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda6c06a430 0x7fda6c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:13.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.045+0000 7fda7e59c640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda6c06a430 0x7fda6c06c8f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fda70006fd0 tx=0x7fda70008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:13.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.047+0000 7fda7ffff640 1 -- 172.21.15.27:0/581487555 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fda78054010 con 0x7fda880cdb30 2023-12-19T18:18:13.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.275+0000 7fda8c85e640 1 -- 172.21.15.27:0/581487555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fda880b4a50 con 0x7fda880cdb30 2023-12-19T18:18:13.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.802+0000 7fda7ffff640 1 -- 172.21.15.27:0/581487555 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7fda78057630 con 0x7fda880cdb30 2023-12-19T18:18:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.806+0000 7fda8c85e640 1 -- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda6c06a430 msgr2=0x7fda6c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.806+0000 7fda8c85e640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda6c06a430 0x7fda6c06c8f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fda70006fd0 tx=0x7fda70008040 comp rx=0 tx=0).stop 2023-12-19T18:18:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.806+0000 7fda8c85e640 1 -- 172.21.15.27:0/581487555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda880cdb30 msgr2=0x7fda880cc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.806+0000 7fda8c85e640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda880cdb30 0x7fda880cc5d0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fda7800e870 tx=0x7fda7800ed30 comp rx=0 tx=0).stop 2023-12-19T18:18:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.806+0000 7fda8c85e640 1 -- 172.21.15.27:0/581487555 shutdown_connections 2023-12-19T18:18:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.806+0000 7fda8c85e640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda6c06a430 0x7fda6c06c8f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.806+0000 7fda8c85e640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda8815cfc0 0x7fda8815b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.806+0000 7fda8c85e640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda880d27b0 0x7fda8815fe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.806+0000 7fda8c85e640 1 --2- 172.21.15.27:0/581487555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda880cdb30 0x7fda880cc5d0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.806+0000 7fda8c85e640 1 -- 172.21.15.27:0/581487555 >> 172.21.15.27:0/581487555 conn(0x7fda88072e20 msgr2=0x7fda880b83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.807+0000 7fda8c85e640 1 -- 172.21.15.27:0/581487555 shutdown_connections 2023-12-19T18:18:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:13.807+0000 7fda8c85e640 1 -- 172.21.15.27:0/581487555 wait complete. 2023-12-19T18:18:13.831 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:18:14.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.064+0000 7f4207fff640 1 Processor -- start 2023-12-19T18:18:14.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.064+0000 7f4207fff640 1 -- start start 2023-12-19T18:18:14.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.066+0000 7f4207fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4208161030 0x7f42081657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:14.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.066+0000 7f4207fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42080cba60 0x7f42080cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.066+0000 7f4207fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4208165d00 con 0x7f42080c55b0 2023-12-19T18:18:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.066+0000 7f4207fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42080c95f0 con 0x7f42080cba60 2023-12-19T18:18:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.066+0000 7f4207fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42080cb100 con 0x7f4208161030 2023-12-19T18:18:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.066+0000 7f42077fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42080cba60 0x7f42080cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.066+0000 7f42077fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42080cba60 0x7f42080cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52870/0 (socket says 172.21.15.27:52870) 2023-12-19T18:18:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.066+0000 7f42077fe640 1 -- 172.21.15.27:0/3386431211 learned_addr learned my addr 172.21.15.27:0/3386431211 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:14.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.067+0000 7f4206ffd640 1 --2- 172.21.15.27:0/3386431211 >> v2:172.21.15.192:3300/0 conn(0x7f4208161030 0x7f42081657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:14.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.067+0000 7f42077fe640 1 -- 172.21.15.27:0/3386431211 >> v2:172.21.15.192:3300/0 conn(0x7f4208161030 msgr2=0x7f42081657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:14.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.067+0000 7f42077fe640 1 --2- 172.21.15.27:0/3386431211 >> v2:172.21.15.192:3300/0 conn(0x7f4208161030 0x7f42081657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.067+0000 7f42077fe640 1 -- 172.21.15.27:0/3386431211 >> v1:172.21.15.192:6789/0 conn(0x7f42080c55b0 legacy=0x7f42080cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:14.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.067+0000 7f42077fe640 1 -- 172.21.15.27:0/3386431211 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42080cae20 con 0x7f42080cba60 2023-12-19T18:18:14.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.068+0000 7f42077fe640 1 --2- 172.21.15.27:0/3386431211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42080cba60 0x7f42080cc510 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f42080cb320 tx=0x7f420002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c91c0aaf7baea0a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:14.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.068+0000 7f4205ffb640 1 -- 172.21.15.27:0/3386431211 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420002ca80 con 0x7f42080cba60 2023-12-19T18:18:14.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.068+0000 7f4205ffb640 1 -- 172.21.15.27:0/3386431211 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420002cbe0 con 0x7f42080cba60 2023-12-19T18:18:14.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.069+0000 7f4205ffb640 1 -- 172.21.15.27:0/3386431211 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42000343f0 con 0x7f42080cba60 2023-12-19T18:18:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.069+0000 7f4207fff640 1 -- 172.21.15.27:0/3386431211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42080cba60 msgr2=0x7f42080cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.069+0000 7f4207fff640 1 --2- 172.21.15.27:0/3386431211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42080cba60 0x7f42080cc510 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f42080cb320 tx=0x7f420002bee0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.069+0000 7f4207fff640 1 -- 172.21.15.27:0/3386431211 shutdown_connections 2023-12-19T18:18:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.069+0000 7f4207fff640 1 --2- 172.21.15.27:0/3386431211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42080cba60 0x7f42080cc510 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.069+0000 7f4207fff640 1 --2- 172.21.15.27:0/3386431211 >> v2:172.21.15.192:3300/0 conn(0x7f4208161030 0x7f42081657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.069+0000 7f4207fff640 1 -- 172.21.15.27:0/3386431211 >> 172.21.15.27:0/3386431211 conn(0x7f42080b05b0 msgr2=0x7f42080b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:14.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.069+0000 7f4207fff640 1 -- 172.21.15.27:0/3386431211 shutdown_connections 2023-12-19T18:18:14.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.069+0000 7f4207fff640 1 -- 172.21.15.27:0/3386431211 wait complete. 2023-12-19T18:18:14.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.071+0000 7f4207fff640 1 Processor -- start 2023-12-19T18:18:14.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.071+0000 7f4207fff640 1 -- start start 2023-12-19T18:18:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.072+0000 7f4207fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42080c55b0 0x7f4208151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.072+0000 7f4207fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42080cba60 0x7f4208070ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.072+0000 7f4207fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4208161030 0x7f420815f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.072+0000 7f4207fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42080c7360 con 0x7f42080cba60 2023-12-19T18:18:14.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.072+0000 7f4207fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42081522d0 con 0x7f42080c55b0 2023-12-19T18:18:14.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.072+0000 7f4207fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4208161410 con 0x7f4208161030 2023-12-19T18:18:14.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.072+0000 7f42067fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42080cba60 0x7f4208070ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:14.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.072+0000 7f42067fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42080cba60 0x7f4208070ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52884/0 (socket says 172.21.15.27:52884) 2023-12-19T18:18:14.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.072+0000 7f42067fc640 1 -- 172.21.15.27:0/2029139517 learned_addr learned my addr 172.21.15.27:0/2029139517 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.073+0000 7f4206ffd640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42080c55b0 0x7f4208151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.073+0000 7f42067fc640 1 -- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4208161030 msgr2=0x7f420815f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.073+0000 7f42067fc640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4208161030 0x7f420815f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.073+0000 7f42067fc640 1 -- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42080c55b0 msgr2=0x7f4208151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.073+0000 7f42067fc640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42080c55b0 0x7f4208151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.073+0000 7f42067fc640 1 -- 172.21.15.27:0/2029139517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4200009b30 con 0x7f42080cba60 2023-12-19T18:18:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.073+0000 7f4206ffd640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42080c55b0 0x7f4208151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.073+0000 7f42067fc640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42080cba60 0x7f4208070ea0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00b3f0 tx=0x7f41fc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.073+0000 7f41effff640 1 -- 172.21.15.27:0/2029139517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc011830 con 0x7f42080cba60 2023-12-19T18:18:14.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.073+0000 7f41effff640 1 -- 172.21.15.27:0/2029139517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41fc011990 con 0x7f42080cba60 2023-12-19T18:18:14.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.073+0000 7f41effff640 1 -- 172.21.15.27:0/2029139517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41fc011d40 con 0x7f42080cba60 2023-12-19T18:18:14.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.073+0000 7f4207fff640 1 -- 172.21.15.27:0/2029139517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420806b7e0 con 0x7f42080cba60 2023-12-19T18:18:14.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.074+0000 7f4207fff640 1 -- 172.21.15.27:0/2029139517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420806bda0 con 0x7f42080cba60 2023-12-19T18:18:14.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.075+0000 7f4207fff640 1 -- 172.21.15.27:0/2029139517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42080c74a0 con 0x7f42080cba60 2023-12-19T18:18:14.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.076+0000 7f41effff640 1 -- 172.21.15.27:0/2029139517 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f41fc01a400 con 0x7f42080cba60 2023-12-19T18:18:14.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.076+0000 7f41effff640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f41f406a430 0x7f41f406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:14.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.077+0000 7f41effff640 1 -- 172.21.15.27:0/2029139517 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f41fc0887d0 con 0x7f42080cba60 2023-12-19T18:18:14.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.077+0000 7f4206ffd640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f41f406a430 0x7f41f406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:14.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.079+0000 7f4206ffd640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f41f406a430 0x7f41f406c8f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f41f8006fd0 tx=0x7f41f8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:14.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.082+0000 7f41effff640 1 -- 172.21.15.27:0/2029139517 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f42080c74a0 con 0x7f42080cba60 2023-12-19T18:18:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.270+0000 7f4207fff640 1 -- 172.21.15.27:0/2029139517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f42080c74a0 con 0x7f42080cba60 2023-12-19T18:18:14.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.806+0000 7f41effff640 1 -- 172.21.15.27:0/2029139517 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v65) v1 ==== 181+0+0 (secure 0 0 0) 0x7f42080c74a0 con 0x7f42080cba60 2023-12-19T18:18:14.807 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:18:14.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.809+0000 7f4207fff640 1 -- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f41f406a430 msgr2=0x7f41f406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:14.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.809+0000 7f4207fff640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f41f406a430 0x7f41f406c8f0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f41f8006fd0 tx=0x7f41f8008040 comp rx=0 tx=0).stop 2023-12-19T18:18:14.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.810+0000 7f4207fff640 1 -- 172.21.15.27:0/2029139517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42080cba60 msgr2=0x7f4208070ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:14.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.810+0000 7f4207fff640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42080cba60 0x7f4208070ea0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00b3f0 tx=0x7f41fc00b8b0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.810+0000 7f4207fff640 1 -- 172.21.15.27:0/2029139517 shutdown_connections 2023-12-19T18:18:14.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.810+0000 7f4207fff640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f41f406a430 0x7f41f406c8f0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.810+0000 7f4207fff640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4208161030 0x7f420815f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.810+0000 7f4207fff640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42080cba60 0x7f4208070ea0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.810+0000 7f4207fff640 1 --2- 172.21.15.27:0/2029139517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42080c55b0 0x7f4208151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:14.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.810+0000 7f4207fff640 1 -- 172.21.15.27:0/2029139517 >> 172.21.15.27:0/2029139517 conn(0x7f42080b05b0 msgr2=0x7f42080ae3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:14.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.811+0000 7f4207fff640 1 -- 172.21.15.27:0/2029139517 shutdown_connections 2023-12-19T18:18:14.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:14.811+0000 7f4207fff640 1 -- 172.21.15.27:0/2029139517 wait complete. 2023-12-19T18:18:14.833 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:18:15.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.064+0000 7ff9911a2640 1 Processor -- start 2023-12-19T18:18:15.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.064+0000 7ff9911a2640 1 -- start start 2023-12-19T18:18:15.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.065+0000 7ff9911a2640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff98c15aed0 0x7ff98c0cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:15.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.065+0000 7ff9911a2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff98c0cdaa0 0x7ff98c0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:15.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.066+0000 7ff9911a2640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff98c0b3fd0 con 0x7ff98c0d1450 2023-12-19T18:18:15.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.066+0000 7ff9911a2640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff98c0cf6f0 con 0x7ff98c0cdaa0 2023-12-19T18:18:15.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.066+0000 7ff9911a2640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff98c0cf9d0 con 0x7ff98c15aed0 2023-12-19T18:18:15.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.066+0000 7ff9909a1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff98c0cdaa0 0x7ff98c0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:15.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.066+0000 7ff9909a1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff98c0cdaa0 0x7ff98c0b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52898/0 (socket says 172.21.15.27:52898) 2023-12-19T18:18:15.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.066+0000 7ff9909a1640 1 -- 172.21.15.27:0/1658590900 learned_addr learned my addr 172.21.15.27:0/1658590900 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:15.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.066+0000 7ff98bfff640 1 --2- 172.21.15.27:0/1658590900 >> v2:172.21.15.192:3300/0 conn(0x7ff98c15aed0 0x7ff98c0cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:15.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.066+0000 7ff9909a1640 1 -- 172.21.15.27:0/1658590900 >> v2:172.21.15.192:3300/0 conn(0x7ff98c15aed0 msgr2=0x7ff98c0cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:15.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.066+0000 7ff9909a1640 1 --2- 172.21.15.27:0/1658590900 >> v2:172.21.15.192:3300/0 conn(0x7ff98c15aed0 0x7ff98c0cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.066+0000 7ff9909a1640 1 -- 172.21.15.27:0/1658590900 >> v1:172.21.15.192:6789/0 conn(0x7ff98c0d1450 legacy=0x7ff98c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:15.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.066+0000 7ff9909a1640 1 -- 172.21.15.27:0/1658590900 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c0b3980 con 0x7ff98c0cdaa0 2023-12-19T18:18:15.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.068+0000 7ff9909a1640 1 --2- 172.21.15.27:0/1658590900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98c0cdaa0 0x7ff98c0b4460 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0ce6b0 tx=0x7ff98402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0eb8eea4e62db5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:15.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.068+0000 7ff98affd640 1 -- 172.21.15.27:0/1658590900 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff98402c9f0 con 0x7ff98c0cdaa0 2023-12-19T18:18:15.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.068+0000 7ff98affd640 1 -- 172.21.15.27:0/1658590900 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff98402cb50 con 0x7ff98c0cdaa0 2023-12-19T18:18:15.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.068+0000 7ff98affd640 1 -- 172.21.15.27:0/1658590900 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9840343f0 con 0x7ff98c0cdaa0 2023-12-19T18:18:15.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.069+0000 7ff9911a2640 1 -- 172.21.15.27:0/1658590900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98c0cdaa0 msgr2=0x7ff98c0b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:15.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.069+0000 7ff9911a2640 1 --2- 172.21.15.27:0/1658590900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98c0cdaa0 0x7ff98c0b4460 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0ce6b0 tx=0x7ff98402be50 comp rx=0 tx=0).stop 2023-12-19T18:18:15.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.069+0000 7ff9911a2640 1 -- 172.21.15.27:0/1658590900 shutdown_connections 2023-12-19T18:18:15.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.069+0000 7ff9911a2640 1 --2- 172.21.15.27:0/1658590900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98c0cdaa0 0x7ff98c0b4460 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.069+0000 7ff9911a2640 1 --2- 172.21.15.27:0/1658590900 >> v2:172.21.15.192:3300/0 conn(0x7ff98c15aed0 0x7ff98c0cf110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.069+0000 7ff9911a2640 1 -- 172.21.15.27:0/1658590900 >> 172.21.15.27:0/1658590900 conn(0x7ff98c099ee0 msgr2=0x7ff98c0d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:15.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.069+0000 7ff9911a2640 1 -- 172.21.15.27:0/1658590900 shutdown_connections 2023-12-19T18:18:15.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.069+0000 7ff9911a2640 1 -- 172.21.15.27:0/1658590900 wait complete. 2023-12-19T18:18:15.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.071+0000 7ff9911a2640 1 Processor -- start 2023-12-19T18:18:15.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.071+0000 7ff9911a2640 1 -- start start 2023-12-19T18:18:15.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff9911a2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff98c0cdaa0 0x7ff98c1571c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:15.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff9911a2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98c0d1450 0x7ff98c0c8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:15.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff9911a2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff98c1e5800 0x7ff98c1f6840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:15.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff9911a2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c0c8bc0 con 0x7ff98c0d1450 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff9911a2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c1f6d80 con 0x7ff98c1e5800 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff9911a2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff98c1e5f00 con 0x7ff98c0cdaa0 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff98b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98c0d1450 0x7ff98c0c8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff98b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98c0d1450 0x7ff98c0c8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52908/0 (socket says 172.21.15.27:52908) 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff98b7fe640 1 -- 172.21.15.27:0/4022321690 learned_addr learned my addr 172.21.15.27:0/4022321690 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff98bfff640 1 --2- 172.21.15.27:0/4022321690 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff98c0cdaa0 0x7ff98c1571c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff98b7fe640 1 -- 172.21.15.27:0/4022321690 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff98c0cdaa0 msgr2=0x7ff98c1571c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff98b7fe640 1 --2- 172.21.15.27:0/4022321690 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff98c0cdaa0 0x7ff98c1571c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff98b7fe640 1 -- 172.21.15.27:0/4022321690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff98c1e5800 msgr2=0x7ff98c1f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff98b7fe640 1 --2- 172.21.15.27:0/4022321690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff98c1e5800 0x7ff98c1f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.072+0000 7ff98b7fe640 1 -- 172.21.15.27:0/4022321690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff984009b30 con 0x7ff98c0d1450 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.073+0000 7ff98b7fe640 1 --2- 172.21.15.27:0/4022321690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98c0d1450 0x7ff98c0c8fe0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff97400c460 tx=0x7ff97400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.073+0000 7ff988ff9640 1 -- 172.21.15.27:0/4022321690 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff974012830 con 0x7ff98c0d1450 2023-12-19T18:18:15.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.073+0000 7ff988ff9640 1 -- 172.21.15.27:0/4022321690 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9740129d0 con 0x7ff98c0d1450 2023-12-19T18:18:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.073+0000 7ff9911a2640 1 -- 172.21.15.27:0/4022321690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff98c1e5c80 con 0x7ff98c0d1450 2023-12-19T18:18:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.073+0000 7ff988ff9640 1 -- 172.21.15.27:0/4022321690 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff974012d80 con 0x7ff98c0d1450 2023-12-19T18:18:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.073+0000 7ff9911a2640 1 -- 172.21.15.27:0/4022321690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff98c0c8500 con 0x7ff98c0d1450 2023-12-19T18:18:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.074+0000 7ff9911a2640 1 -- 172.21.15.27:0/4022321690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff98c0c8d00 con 0x7ff98c0d1450 2023-12-19T18:18:15.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.075+0000 7ff988ff9640 1 -- 172.21.15.27:0/4022321690 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff97401b400 con 0x7ff98c0d1450 2023-12-19T18:18:15.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.076+0000 7ff988ff9640 1 --2- 172.21.15.27:0/4022321690 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff96c06a430 0x7ff96c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:15.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.076+0000 7ff988ff9640 1 -- 172.21.15.27:0/4022321690 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 5502+0+0 (secure 0 0 0) 0x7ff9740887a0 con 0x7ff98c0d1450 2023-12-19T18:18:15.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.077+0000 7ff98bfff640 1 --2- 172.21.15.27:0/4022321690 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff96c06a430 0x7ff96c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:15.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.078+0000 7ff98bfff640 1 --2- 172.21.15.27:0/4022321690 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff96c06a430 0x7ff96c06c8f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff97c006fd0 tx=0x7ff97c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:15.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.080+0000 7ff988ff9640 1 -- 172.21.15.27:0/4022321690 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff98c0c8d00 con 0x7ff98c0d1450 2023-12-19T18:18:15.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.264+0000 7ff9911a2640 1 -- 172.21.15.27:0/4022321690 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff98c0c8d00 con 0x7ff98c0d1450 2023-12-19T18:18:15.277 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:18:15.285 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:18:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.811+0000 7ff988ff9640 1 -- 172.21.15.27:0/4022321690 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v66) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff98c0c8d00 con 0x7ff98c0d1450 2023-12-19T18:18:15.811 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-12-19T18:18:15.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.814+0000 7ff9911a2640 1 -- 172.21.15.27:0/4022321690 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff96c06a430 msgr2=0x7ff96c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:15.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.814+0000 7ff9911a2640 1 --2- 172.21.15.27:0/4022321690 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff96c06a430 0x7ff96c06c8f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ff97c006fd0 tx=0x7ff97c008040 comp rx=0 tx=0).stop 2023-12-19T18:18:15.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.814+0000 7ff9911a2640 1 -- 172.21.15.27:0/4022321690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98c0d1450 msgr2=0x7ff98c0c8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:15.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.814+0000 7ff9911a2640 1 --2- 172.21.15.27:0/4022321690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98c0d1450 0x7ff98c0c8fe0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7ff97400c460 tx=0x7ff97400c920 comp rx=0 tx=0).stop 2023-12-19T18:18:15.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.815+0000 7ff9911a2640 1 -- 172.21.15.27:0/4022321690 shutdown_connections 2023-12-19T18:18:15.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.815+0000 7ff9911a2640 1 --2- 172.21.15.27:0/4022321690 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff96c06a430 0x7ff96c06c8f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.815+0000 7ff9911a2640 1 --2- 172.21.15.27:0/4022321690 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff98c1e5800 0x7ff98c1f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.815+0000 7ff9911a2640 1 --2- 172.21.15.27:0/4022321690 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff98c0d1450 0x7ff98c0c8fe0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.815+0000 7ff9911a2640 1 --2- 172.21.15.27:0/4022321690 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff98c0cdaa0 0x7ff98c1571c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:15.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.815+0000 7ff9911a2640 1 -- 172.21.15.27:0/4022321690 >> 172.21.15.27:0/4022321690 conn(0x7ff98c099ee0 msgr2=0x7ff98c159fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:15.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.815+0000 7ff9911a2640 1 -- 172.21.15.27:0/4022321690 shutdown_connections 2023-12-19T18:18:15.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:15.815+0000 7ff9911a2640 1 -- 172.21.15.27:0/4022321690 wait complete. 2023-12-19T18:18:15.839 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:18:16.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.063+0000 7fa36d24d640 1 Processor -- start 2023-12-19T18:18:16.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.064+0000 7fa36d24d640 1 -- start start 2023-12-19T18:18:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.065+0000 7fa36d24d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3680b0a80 0x7fa3680cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.065+0000 7fa36d24d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa36806cd30 0x7fa3680ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.065+0000 7fa36d24d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa368161380 con 0x7fa36815cd40 2023-12-19T18:18:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.065+0000 7fa36d24d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa368161810 con 0x7fa36806cd30 2023-12-19T18:18:16.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.065+0000 7fa36d24d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36815d120 con 0x7fa3680b0a80 2023-12-19T18:18:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.066+0000 7fa35ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3680b0a80 0x7fa3680cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.066+0000 7fa36ca4c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa36806cd30 0x7fa3680ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.066+0000 7fa35ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3680b0a80 0x7fa3680cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:60608/0 (socket says 172.21.15.27:60608) 2023-12-19T18:18:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.066+0000 7fa35ffff640 1 -- 172.21.15.27:0/1407451778 learned_addr learned my addr 172.21.15.27:0/1407451778 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.067+0000 7fa35ffff640 1 -- 172.21.15.27:0/1407451778 >> v2:172.21.15.192:3300/0 conn(0x7fa36806cd30 msgr2=0x7fa3680ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.067+0000 7fa35ffff640 1 --2- 172.21.15.27:0/1407451778 >> v2:172.21.15.192:3300/0 conn(0x7fa36806cd30 0x7fa3680ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.067+0000 7fa35ffff640 1 -- 172.21.15.27:0/1407451778 >> v1:172.21.15.192:6789/0 conn(0x7fa36815cd40 legacy=0x7fa3680b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.067+0000 7fa35ffff640 1 -- 172.21.15.27:0/1407451778 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa368160a50 con 0x7fa3680b0a80 2023-12-19T18:18:16.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.068+0000 7fa35ffff640 1 --2- 172.21.15.27:0/1407451778 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3680b0a80 0x7fa3680cadd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa350009870 tx=0x7fa35002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=122b4d716925cb99 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.069+0000 7fa3677fe640 1 -- 172.21.15.27:0/1407451778 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350031640 con 0x7fa3680b0a80 2023-12-19T18:18:16.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.069+0000 7fa3677fe640 1 -- 172.21.15.27:0/1407451778 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3500317a0 con 0x7fa3680b0a80 2023-12-19T18:18:16.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.069+0000 7fa3677fe640 1 -- 172.21.15.27:0/1407451778 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa350031b30 con 0x7fa3680b0a80 2023-12-19T18:18:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.070+0000 7fa36d24d640 1 -- 172.21.15.27:0/1407451778 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3680b0a80 msgr2=0x7fa3680cadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.070+0000 7fa36d24d640 1 --2- 172.21.15.27:0/1407451778 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3680b0a80 0x7fa3680cadd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa350009870 tx=0x7fa35002b8f0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.070+0000 7fa36d24d640 1 -- 172.21.15.27:0/1407451778 shutdown_connections 2023-12-19T18:18:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.070+0000 7fa36d24d640 1 --2- 172.21.15.27:0/1407451778 >> v2:172.21.15.192:3300/0 conn(0x7fa36806cd30 0x7fa3680ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.070+0000 7fa36d24d640 1 --2- 172.21.15.27:0/1407451778 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3680b0a80 0x7fa3680cadd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.070+0000 7fa36d24d640 1 -- 172.21.15.27:0/1407451778 >> 172.21.15.27:0/1407451778 conn(0x7fa3680cb370 msgr2=0x7fa3680cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:16.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.070+0000 7fa36d24d640 1 -- 172.21.15.27:0/1407451778 shutdown_connections 2023-12-19T18:18:16.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.070+0000 7fa36d24d640 1 -- 172.21.15.27:0/1407451778 wait complete. 2023-12-19T18:18:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.072+0000 7fa36d24d640 1 Processor -- start 2023-12-19T18:18:16.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.072+0000 7fa36d24d640 1 -- start start 2023-12-19T18:18:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.073+0000 7fa36d24d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa36806cd30 0x7fa368150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.073+0000 7fa36d24d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3680b0a80 0x7fa3680c2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.073+0000 7fa36d24d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa36815cd40 0x7fa3680c2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.073+0000 7fa36d24d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3680c2f80 con 0x7fa36815cd40 2023-12-19T18:18:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.073+0000 7fa36d24d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3680c4830 con 0x7fa3680b0a80 2023-12-19T18:18:16.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.073+0000 7fa36d24d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3680c2ca0 con 0x7fa36806cd30 2023-12-19T18:18:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.074+0000 7fa36ca4c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa36815cd40 0x7fa3680c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.074+0000 7fa36ca4c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa36815cd40 0x7fa3680c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52924/0 (socket says 172.21.15.27:52924) 2023-12-19T18:18:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.074+0000 7fa36ca4c640 1 -- 172.21.15.27:0/1456964007 learned_addr learned my addr 172.21.15.27:0/1456964007 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.074+0000 7fa367fff640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa36806cd30 0x7fa368150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.074+0000 7fa35ffff640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3680b0a80 0x7fa3680c2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.074+0000 7fa36ca4c640 1 -- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa36806cd30 msgr2=0x7fa368150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.074+0000 7fa36ca4c640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa36806cd30 0x7fa368150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.074+0000 7fa36ca4c640 1 -- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3680b0a80 msgr2=0x7fa3680c2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.074+0000 7fa36ca4c640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3680b0a80 0x7fa3680c2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.074+0000 7fa36ca4c640 1 -- 172.21.15.27:0/1456964007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3500094d0 con 0x7fa36815cd40 2023-12-19T18:18:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.074+0000 7fa367fff640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa36806cd30 0x7fa368150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.075+0000 7fa36ca4c640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa36815cd40 0x7fa3680c2670 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa36000efc0 tx=0x7fa36000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.075+0000 7fa3657fa640 1 -- 172.21.15.27:0/1456964007 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360014480 con 0x7fa36815cd40 2023-12-19T18:18:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.075+0000 7fa3657fa640 1 -- 172.21.15.27:0/1456964007 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36000f040 con 0x7fa36815cd40 2023-12-19T18:18:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.075+0000 7fa3657fa640 1 -- 172.21.15.27:0/1456964007 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa360014820 con 0x7fa36815cd40 2023-12-19T18:18:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.075+0000 7fa36d24d640 1 -- 172.21.15.27:0/1456964007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3680c6ad0 con 0x7fa36815cd40 2023-12-19T18:18:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.075+0000 7fa36d24d640 1 -- 172.21.15.27:0/1456964007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3680c7090 con 0x7fa36815cd40 2023-12-19T18:18:16.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.076+0000 7fa36d24d640 1 -- 172.21.15.27:0/1456964007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3680c6930 con 0x7fa36815cd40 2023-12-19T18:18:16.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.077+0000 7fa3657fa640 1 -- 172.21.15.27:0/1456964007 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa360014980 con 0x7fa36815cd40 2023-12-19T18:18:16.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.078+0000 7fa3657fa640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa34806a430 0x7fa34806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.078+0000 7fa3657fa640 1 -- 172.21.15.27:0/1456964007 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa360089220 con 0x7fa36815cd40 2023-12-19T18:18:16.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.079+0000 7fa367fff640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa34806a430 0x7fa34806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.080+0000 7fa367fff640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa34806a430 0x7fa34806c8f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa358007fd0 tx=0x7fa358009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.083+0000 7fa3657fa640 1 -- 172.21.15.27:0/1456964007 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa3680c6930 con 0x7fa36815cd40 2023-12-19T18:18:16.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.275+0000 7fa36d24d640 1 -- 172.21.15.27:0/1456964007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa3680c6930 con 0x7fa36815cd40 2023-12-19T18:18:16.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.281+0000 7fa3657fa640 1 -- 172.21.15.27:0/1456964007 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa3680c6930 con 0x7fa36815cd40 2023-12-19T18:18:16.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.284+0000 7fa36d24d640 1 -- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa34806a430 msgr2=0x7fa34806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.284+0000 7fa36d24d640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa34806a430 0x7fa34806c8f0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fa358007fd0 tx=0x7fa358009040 comp rx=0 tx=0).stop 2023-12-19T18:18:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.284+0000 7fa36d24d640 1 -- 172.21.15.27:0/1456964007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa36815cd40 msgr2=0x7fa3680c2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.284+0000 7fa36d24d640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa36815cd40 0x7fa3680c2670 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fa36000efc0 tx=0x7fa36000c490 comp rx=0 tx=0).stop 2023-12-19T18:18:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.285+0000 7fa36d24d640 1 -- 172.21.15.27:0/1456964007 shutdown_connections 2023-12-19T18:18:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.285+0000 7fa36d24d640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa34806a430 0x7fa34806c8f0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.285+0000 7fa36d24d640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa36815cd40 0x7fa3680c2670 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.285+0000 7fa36d24d640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3680b0a80 0x7fa3680c2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.285+0000 7fa36d24d640 1 --2- 172.21.15.27:0/1456964007 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa36806cd30 0x7fa368150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.285+0000 7fa36d24d640 1 -- 172.21.15.27:0/1456964007 >> 172.21.15.27:0/1456964007 conn(0x7fa3680cb370 msgr2=0x7fa3680c8fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.285+0000 7fa36d24d640 1 -- 172.21.15.27:0/1456964007 shutdown_connections 2023-12-19T18:18:16.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.285+0000 7fa36d24d640 1 -- 172.21.15.27:0/1456964007 wait complete. 2023-12-19T18:18:16.303 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file' 2023-12-19T18:18:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.536+0000 7f62e759e640 1 Processor -- start 2023-12-19T18:18:16.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.536+0000 7f62e759e640 1 -- start start 2023-12-19T18:18:16.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.537+0000 7f62e759e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f62e80b49d0 0x7f62e8160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.538+0000 7f62e759e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e80b5ca0 con 0x7f62e80b89b0 2023-12-19T18:18:16.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.538+0000 7f62e759e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e8161800 con 0x7f62e815cf90 2023-12-19T18:18:16.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.538+0000 7f62e759e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e8161240 con 0x7f62e80b49d0 2023-12-19T18:18:16.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.538+0000 7f62e5d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f62e80b89b0 0x7f62e815c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58148/0 (socket says 172.21.15.27:58148) 2023-12-19T18:18:16.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.538+0000 7f62e5d9b640 1 -- 172.21.15.27:0/2808850606 learned_addr learned my addr 172.21.15.27:0/2808850606 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:16.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.538+0000 7f62e659c640 1 --2- 172.21.15.27:0/2808850606 >> v2:172.21.15.192:3300/0 conn(0x7f62e80b49d0 0x7f62e8160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.538+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1031617091 0 0) 0x7f62e80b5ca0 con 0x7f62e80b89b0 2023-12-19T18:18:16.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62d4003520 con 0x7f62e80b89b0 2023-12-19T18:18:16.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1286131171 0 0) 0x7f62d4003520 con 0x7f62e80b89b0 2023-12-19T18:18:16.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 >> v2:172.21.15.192:3300/0 conn(0x7f62e80b49d0 msgr2=0x7f62e8160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:16.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 --2- 172.21.15.27:0/2808850606 >> v2:172.21.15.192:3300/0 conn(0x7f62e80b49d0 0x7f62e8160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 >> v1:172.21.15.192:6789/0 conn(0x7f62e815cf90 legacy=0x7f62e80d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e8161520 con 0x7f62e80b89b0 2023-12-19T18:18:16.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f62d0001670 con 0x7f62e80b89b0 2023-12-19T18:18:16.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 >> v1:172.21.15.27:6789/0 conn(0x7f62e80b89b0 legacy=0x7f62e815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 --2- 172.21.15.27:0/2808850606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d40039c0 0x7f62d4003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 --2- 172.21.15.27:0/2808850606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d40047c0 0x7f62d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 --2- 172.21.15.27:0/2808850606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62d4005180 0x7f62d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e8161240 con 0x7f62d40047c0 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d400c060 con 0x7f62d40039c0 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.539+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d400c220 con 0x7f62d4005180 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.540+0000 7f62e5d9b640 1 --2- 172.21.15.27:0/2808850606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d40047c0 0x7f62d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.540+0000 7f62e659c640 1 --2- 172.21.15.27:0/2808850606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d40039c0 0x7f62d4003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.540+0000 7f62e5d9b640 1 -- 172.21.15.27:0/2808850606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62d4005180 msgr2=0x7f62d400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.540+0000 7f62e5d9b640 1 --2- 172.21.15.27:0/2808850606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62d4005180 0x7f62d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.540+0000 7f62e5d9b640 1 -- 172.21.15.27:0/2808850606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d40039c0 msgr2=0x7f62d4003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.540+0000 7f62e5d9b640 1 --2- 172.21.15.27:0/2808850606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d40039c0 0x7f62d4003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.540+0000 7f62e5d9b640 1 -- 172.21.15.27:0/2808850606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d4003520 con 0x7f62d40047c0 2023-12-19T18:18:16.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.540+0000 7f62e659c640 1 --2- 172.21.15.27:0/2808850606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d40039c0 0x7f62d4003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:16.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.540+0000 7f62e6d9d640 1 -- 172.21.15.27:0/2808850606 reap_dead start 2023-12-19T18:18:16.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.541+0000 7f62e5d9b640 1 --2- 172.21.15.27:0/2808850606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d40047c0 0x7f62d4004c40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c0e0 tx=0x7f62d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c9a5c0c61e1301 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.541+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0005ad0 con 0x7f62d40047c0 2023-12-19T18:18:16.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.542+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d0005c30 con 0x7f62d40047c0 2023-12-19T18:18:16.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.542+0000 7f62e559a640 1 -- 172.21.15.27:0/2808850606 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d002ca50 con 0x7f62d40047c0 2023-12-19T18:18:16.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.542+0000 7f62e759e640 1 -- 172.21.15.27:0/2808850606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d40047c0 msgr2=0x7f62d4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.542+0000 7f62e759e640 1 --2- 172.21.15.27:0/2808850606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d40047c0 0x7f62d4004c40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c0e0 tx=0x7f62d40057f0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.542+0000 7f62e759e640 1 -- 172.21.15.27:0/2808850606 shutdown_connections 2023-12-19T18:18:16.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.542+0000 7f62e759e640 1 --2- 172.21.15.27:0/2808850606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d40047c0 0x7f62d4004c40 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.542+0000 7f62e759e640 1 -- 172.21.15.27:0/2808850606 >> 172.21.15.27:0/2808850606 conn(0x7f62e80cf760 msgr2=0x7f62e80cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:16.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.542+0000 7f62e759e640 1 -- 172.21.15.27:0/2808850606 shutdown_connections 2023-12-19T18:18:16.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.542+0000 7f62e759e640 1 -- 172.21.15.27:0/2808850606 wait complete. 2023-12-19T18:18:16.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.544+0000 7f62e759e640 1 Processor -- start 2023-12-19T18:18:16.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.544+0000 7f62e759e640 1 -- start start 2023-12-19T18:18:16.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.545+0000 7f62e759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d40047c0 0x7f62e815ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.545+0000 7f62e759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62e81f6b00 0x7f62e815b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.545+0000 7f62e759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62e80c8420 0x7f62e80c8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.545+0000 7f62e759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e8160230 con 0x7f62d40047c0 2023-12-19T18:18:16.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.545+0000 7f62e759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e80c8dd0 con 0x7f62e81f6b00 2023-12-19T18:18:16.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.545+0000 7f62e759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e815f6b0 con 0x7f62e80c8420 2023-12-19T18:18:16.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.546+0000 7f62e659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d40047c0 0x7f62e815ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.546+0000 7f62e659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d40047c0 0x7f62e815ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52938/0 (socket says 172.21.15.27:52938) 2023-12-19T18:18:16.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.546+0000 7f62e659c640 1 -- 172.21.15.27:0/3887513408 learned_addr learned my addr 172.21.15.27:0/3887513408 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:16.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.546+0000 7f62e5d9b640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62e81f6b00 0x7f62e815b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.546+0000 7f62e6d9d640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62e80c8420 0x7f62e80c8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.546+0000 7f62e659c640 1 -- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62e80c8420 msgr2=0x7f62e80c8890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.546+0000 7f62e659c640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62e80c8420 0x7f62e80c8890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.546+0000 7f62e659c640 1 -- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62e81f6b00 msgr2=0x7f62e815b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.546+0000 7f62e659c640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62e81f6b00 0x7f62e815b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.546+0000 7f62e659c640 1 -- 172.21.15.27:0/3887513408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e815f990 con 0x7f62d40047c0 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.546+0000 7f62e5d9b640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62e81f6b00 0x7f62e815b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.546+0000 7f62e6d9d640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62e80c8420 0x7f62e80c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.547+0000 7f62e659c640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d40047c0 0x7f62e815ee60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f62d8007e70 tx=0x7f62d8010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.547+0000 7f62deffd640 1 -- 172.21.15.27:0/3887513408 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d800e050 con 0x7f62d40047c0 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.547+0000 7f62deffd640 1 -- 172.21.15.27:0/3887513408 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d800f040 con 0x7f62d40047c0 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.547+0000 7f62deffd640 1 -- 172.21.15.27:0/3887513408 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8003bd0 con 0x7f62d40047c0 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.547+0000 7f62e759e640 1 -- 172.21.15.27:0/3887513408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e815fc70 con 0x7f62d40047c0 2023-12-19T18:18:16.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.547+0000 7f62e759e640 1 -- 172.21.15.27:0/3887513408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e815f3a0 con 0x7f62d40047c0 2023-12-19T18:18:16.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.548+0000 7f62e759e640 1 -- 172.21.15.27:0/3887513408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e804e840 con 0x7f62d40047c0 2023-12-19T18:18:16.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.549+0000 7f62deffd640 1 -- 172.21.15.27:0/3887513408 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f62d8003d30 con 0x7f62d40047c0 2023-12-19T18:18:16.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.550+0000 7f62deffd640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62c806a460 0x7f62c806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:16.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.550+0000 7f62deffd640 1 -- 172.21.15.27:0/3887513408 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f62d80893c0 con 0x7f62d40047c0 2023-12-19T18:18:16.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.551+0000 7f62e5d9b640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62c806a460 0x7f62c806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:16.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.552+0000 7f62e5d9b640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62c806a460 0x7f62c806c920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f62d0034000 tx=0x7f62d0009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:16.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.554+0000 7f62deffd640 1 -- 172.21.15.27:0/3887513408 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f62d8054ca0 con 0x7f62d40047c0 2023-12-19T18:18:16.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:16.819+0000 7f62e759e640 1 -- 172.21.15.27:0/3887513408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]} v 0) v1 -- 0x7f62e80c9640 con 0x7f62d40047c0 2023-12-19T18:18:17.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:17.823+0000 7f62deffd640 1 -- 172.21.15.27:0/3887513408 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_different_file"]}]=0 v194) v1 ==== 142+0+0 (secure 0 0 0) 0x7f62d80582c0 con 0x7f62d40047c0 2023-12-19T18:18:17.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:17.827+0000 7f62e759e640 1 -- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62c806a460 msgr2=0x7f62c806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:17.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:17.827+0000 7f62e759e640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62c806a460 0x7f62c806c920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f62d0034000 tx=0x7f62d0009030 comp rx=0 tx=0).stop 2023-12-19T18:18:17.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:17.827+0000 7f62e759e640 1 -- 172.21.15.27:0/3887513408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d40047c0 msgr2=0x7f62e815ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:17.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:17.827+0000 7f62e759e640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d40047c0 0x7f62e815ee60 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f62d8007e70 tx=0x7f62d8010590 comp rx=0 tx=0).stop 2023-12-19T18:18:17.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:17.827+0000 7f62e759e640 1 -- 172.21.15.27:0/3887513408 shutdown_connections 2023-12-19T18:18:17.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:17.827+0000 7f62e759e640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62c806a460 0x7f62c806c920 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:17.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:17.827+0000 7f62e759e640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62e80c8420 0x7f62e80c8890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:17.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:17.827+0000 7f62e759e640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62e81f6b00 0x7f62e815b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:17.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:17.827+0000 7f62e759e640 1 --2- 172.21.15.27:0/3887513408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d40047c0 0x7f62e815ee60 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:17.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:17.827+0000 7f62e759e640 1 -- 172.21.15.27:0/3887513408 >> 172.21.15.27:0/3887513408 conn(0x7f62e80cf760 msgr2=0x7f62e80cbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:17.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:17.828+0000 7f62e759e640 1 -- 172.21.15.27:0/3887513408 shutdown_connections 2023-12-19T18:18:17.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:17.828+0000 7f62e759e640 1 -- 172.21.15.27:0/3887513408 wait complete. 2023-12-19T18:18:17.850 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-12-19T18:18:17.850 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestClusterFull) 2023-12-19T18:18:17.851 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-12-19T18:18:18.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.087+0000 7f633882b640 1 Processor -- start 2023-12-19T18:18:18.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.087+0000 7f633882b640 1 -- start start 2023-12-19T18:18:18.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.089+0000 7f633882b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63341553a0 0x7f63340c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.089+0000 7f633882b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f63340afeb0 0x7f633415e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.089+0000 7f633882b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633415f090 con 0x7f633415ec60 2023-12-19T18:18:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.089+0000 7f633882b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63340cbcf0 con 0x7f63341553a0 2023-12-19T18:18:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.089+0000 7f633882b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63340b0360 con 0x7f63340afeb0 2023-12-19T18:18:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.089+0000 7f6332d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63341553a0 0x7f63340c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.089+0000 7f6332d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63341553a0 0x7f63340c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52946/0 (socket says 172.21.15.27:52946) 2023-12-19T18:18:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.089+0000 7f6332d9d640 1 -- 172.21.15.27:0/2453136292 learned_addr learned my addr 172.21.15.27:0/2453136292 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.090+0000 7f6332d9d640 1 -- 172.21.15.27:0/2453136292 >> v2:172.21.15.192:3300/0 conn(0x7f63340afeb0 msgr2=0x7f633415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.090+0000 7f6332d9d640 1 --2- 172.21.15.27:0/2453136292 >> v2:172.21.15.192:3300/0 conn(0x7f63340afeb0 0x7f633415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.090+0000 7f6332d9d640 1 -- 172.21.15.27:0/2453136292 >> v1:172.21.15.192:6789/0 conn(0x7f633415ec60 legacy=0x7f633415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:18.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.090+0000 7f6332d9d640 1 -- 172.21.15.27:0/2453136292 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340cb350 con 0x7f63341553a0 2023-12-19T18:18:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.091+0000 7f6332d9d640 1 --2- 172.21.15.27:0/2453136292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63341553a0 0x7f63340c50a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f63340cc1c0 tx=0x7f632402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcf45aa2a9066b21 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.091+0000 7f6331d9b640 1 -- 172.21.15.27:0/2453136292 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63240317e0 con 0x7f63341553a0 2023-12-19T18:18:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.091+0000 7f6331d9b640 1 -- 172.21.15.27:0/2453136292 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6324031940 con 0x7f63341553a0 2023-12-19T18:18:18.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.091+0000 7f6331d9b640 1 -- 172.21.15.27:0/2453136292 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324031cd0 con 0x7f63341553a0 2023-12-19T18:18:18.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.092+0000 7f633882b640 1 -- 172.21.15.27:0/2453136292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63341553a0 msgr2=0x7f63340c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:18.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.092+0000 7f633882b640 1 --2- 172.21.15.27:0/2453136292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63341553a0 0x7f63340c50a0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f63340cc1c0 tx=0x7f632402b880 comp rx=0 tx=0).stop 2023-12-19T18:18:18.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.092+0000 7f633882b640 1 -- 172.21.15.27:0/2453136292 shutdown_connections 2023-12-19T18:18:18.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.092+0000 7f633882b640 1 --2- 172.21.15.27:0/2453136292 >> v2:172.21.15.192:3300/0 conn(0x7f63340afeb0 0x7f633415e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.092+0000 7f633882b640 1 --2- 172.21.15.27:0/2453136292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63341553a0 0x7f63340c50a0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.092+0000 7f633882b640 1 -- 172.21.15.27:0/2453136292 >> 172.21.15.27:0/2453136292 conn(0x7f6334069c70 msgr2=0x7f633406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:18.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.092+0000 7f633882b640 1 -- 172.21.15.27:0/2453136292 shutdown_connections 2023-12-19T18:18:18.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.092+0000 7f633882b640 1 -- 172.21.15.27:0/2453136292 wait complete. 2023-12-19T18:18:18.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.094+0000 7f633882b640 1 Processor -- start 2023-12-19T18:18:18.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.094+0000 7f633882b640 1 -- start start 2023-12-19T18:18:18.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.095+0000 7f633882b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63340afeb0 0x7f633415c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:18.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.095+0000 7f633882b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63341553a0 0x7f633413c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:18.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.095+0000 7f633882b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633415ec60 0x7f63341f5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:18.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.095+0000 7f633882b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6334142ee0 con 0x7f63341553a0 2023-12-19T18:18:18.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.095+0000 7f633882b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63341f68c0 con 0x7f63340afeb0 2023-12-19T18:18:18.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.095+0000 7f633882b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63341f6ba0 con 0x7f633415ec60 2023-12-19T18:18:18.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.095+0000 7f633259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63341553a0 0x7f633413c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:18.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.095+0000 7f633259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63341553a0 0x7f633413c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52956/0 (socket says 172.21.15.27:52956) 2023-12-19T18:18:18.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.095+0000 7f633259c640 1 -- 172.21.15.27:0/3920513064 learned_addr learned my addr 172.21.15.27:0/3920513064 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:18.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.095+0000 7f6332d9d640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63340afeb0 0x7f633415c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:18.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.095+0000 7f633359e640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633415ec60 0x7f63341f5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:18.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.096+0000 7f633259c640 1 -- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633415ec60 msgr2=0x7f63341f5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:18.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.096+0000 7f633259c640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633415ec60 0x7f63341f5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.096+0000 7f633259c640 1 -- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63340afeb0 msgr2=0x7f633415c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:18.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.096+0000 7f633259c640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63340afeb0 0x7f633415c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.096+0000 7f633259c640 1 -- 172.21.15.27:0/3920513064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63240094d0 con 0x7f63341553a0 2023-12-19T18:18:18.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.096+0000 7f633259c640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63341553a0 0x7f633413c1f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f632800ba50 tx=0x7f632800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:18.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.096+0000 7f631b7fe640 1 -- 172.21.15.27:0/3920513064 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632800cbe0 con 0x7f63341553a0 2023-12-19T18:18:18.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.096+0000 7f631b7fe640 1 -- 172.21.15.27:0/3920513064 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f632800cd60 con 0x7f63341553a0 2023-12-19T18:18:18.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.096+0000 7f631b7fe640 1 -- 172.21.15.27:0/3920513064 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328014680 con 0x7f63341553a0 2023-12-19T18:18:18.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.096+0000 7f633882b640 1 -- 172.21.15.27:0/3920513064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63340ccdc0 con 0x7f63341553a0 2023-12-19T18:18:18.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.096+0000 7f633882b640 1 -- 172.21.15.27:0/3920513064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63340cd380 con 0x7f63341553a0 2023-12-19T18:18:18.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.098+0000 7f633882b640 1 -- 172.21.15.27:0/3920513064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63340cd5c0 con 0x7f63341553a0 2023-12-19T18:18:18.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.099+0000 7f631b7fe640 1 -- 172.21.15.27:0/3920513064 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f632801d3b0 con 0x7f63341553a0 2023-12-19T18:18:18.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.100+0000 7f631b7fe640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f632006a430 0x7f632006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:18.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.100+0000 7f631b7fe640 1 -- 172.21.15.27:0/3920513064 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6328088610 con 0x7f63341553a0 2023-12-19T18:18:18.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.100+0000 7f6332d9d640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f632006a430 0x7f632006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:18.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.101+0000 7f6332d9d640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f632006a430 0x7f632006c8f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6324036000 tx=0x7f632403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:18.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.104+0000 7f631b7fe640 1 -- 172.21.15.27:0/3920513064 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f63340cd5c0 con 0x7f63341553a0 2023-12-19T18:18:18.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.367+0000 7f633882b640 1 -- 172.21.15.27:0/3920513064 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f63340cd5c0 con 0x7f63341553a0 2023-12-19T18:18:18.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.827+0000 7f631b7fe640 1 -- 172.21.15.27:0/3920513064 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v195) v1 ==== 137+0+0 (secure 0 0 0) 0x7f63340cd5c0 con 0x7f63341553a0 2023-12-19T18:18:18.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.831+0000 7f633882b640 1 -- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f632006a430 msgr2=0x7f632006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:18.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.831+0000 7f633882b640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f632006a430 0x7f632006c8f0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6324036000 tx=0x7f632403d040 comp rx=0 tx=0).stop 2023-12-19T18:18:18.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.831+0000 7f633882b640 1 -- 172.21.15.27:0/3920513064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63341553a0 msgr2=0x7f633413c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:18.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.831+0000 7f633882b640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63341553a0 0x7f633413c1f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f632800ba50 tx=0x7f632800bf10 comp rx=0 tx=0).stop 2023-12-19T18:18:18.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.831+0000 7f633882b640 1 -- 172.21.15.27:0/3920513064 shutdown_connections 2023-12-19T18:18:18.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.831+0000 7f633882b640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f632006a430 0x7f632006c8f0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.831+0000 7f633882b640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633415ec60 0x7f63341f5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.831+0000 7f633882b640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63341553a0 0x7f633413c1f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.831+0000 7f633882b640 1 --2- 172.21.15.27:0/3920513064 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63340afeb0 0x7f633415c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:18.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.831+0000 7f633882b640 1 -- 172.21.15.27:0/3920513064 >> 172.21.15.27:0/3920513064 conn(0x7f6334069c70 msgr2=0x7f63340c4980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:18.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.831+0000 7f633882b640 1 -- 172.21.15.27:0/3920513064 shutdown_connections 2023-12-19T18:18:18.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:18.831+0000 7f633882b640 1 -- 172.21.15.27:0/3920513064 wait complete. 2023-12-19T18:18:18.851 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-12-19T18:18:19.100 INFO:teuthology.orchestra.run.smithi027.stdout:{"osd_objectstore":"memstore"} 2023-12-19T18:18:19.100 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-12-19T18:18:19.100 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:18:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.342+0000 7fd01ee62640 1 Processor -- start 2023-12-19T18:18:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.342+0000 7fd01ee62640 1 -- start start 2023-12-19T18:18:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.344+0000 7fd01ee62640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0180cf540 0x7fd01815a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.344+0000 7fd01ee62640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd01806b4c0 con 0x7fd01815aaf0 2023-12-19T18:18:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.344+0000 7fd01ee62640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd018161bd0 con 0x7fd0180cf160 2023-12-19T18:18:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.344+0000 7fd01ee62640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd018070080 con 0x7fd0180cf540 2023-12-19T18:18:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.344+0000 7fd01e661640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd01815aaf0 0x7fd01806b930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:58152/0 (socket says 172.21.15.27:58152) 2023-12-19T18:18:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.344+0000 7fd01e661640 1 -- 172.21.15.27:0/1408097181 learned_addr learned my addr 172.21.15.27:0/1408097181 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.344+0000 7fd01d65f640 1 --2- 172.21.15.27:0/1408097181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0180cf540 0x7fd01815a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.345+0000 7fd01ce5e640 1 -- 172.21.15.27:0/1408097181 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4003266243 0 0) 0x7fd01806b4c0 con 0x7fd01815aaf0 2023-12-19T18:18:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.345+0000 7fd01ce5e640 1 -- 172.21.15.27:0/1408097181 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd004003520 con 0x7fd01815aaf0 2023-12-19T18:18:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.345+0000 7fd01ce5e640 1 -- 172.21.15.27:0/1408097181 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 736744880 0 0) 0x7fd018161bd0 con 0x7fd0180cf160 2023-12-19T18:18:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.345+0000 7fd01ce5e640 1 -- 172.21.15.27:0/1408097181 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd01806b4c0 con 0x7fd0180cf160 2023-12-19T18:18:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.345+0000 7fd01ce5e640 1 -- 172.21.15.27:0/1408097181 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2823170406 0 0) 0x7fd004003520 con 0x7fd01815aaf0 2023-12-19T18:18:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.345+0000 7fd01d65f640 1 -- 172.21.15.27:0/1408097181 >> v1:172.21.15.192:6789/0 conn(0x7fd0180cf160 legacy=0x7fd01815e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.345+0000 7fd01d65f640 1 -- 172.21.15.27:0/1408097181 >> v1:172.21.15.27:6789/0 conn(0x7fd01815aaf0 legacy=0x7fd01806b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.345+0000 7fd01d65f640 1 -- 172.21.15.27:0/1408097181 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01806d140 con 0x7fd0180cf540 2023-12-19T18:18:19.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.347+0000 7fd01d65f640 1 --2- 172.21.15.27:0/1408097181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0180cf540 0x7fd01815a5b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0097d0 tx=0x7fd00c02b800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3f9fc519263fd92 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:19.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.347+0000 7fd01ce5e640 1 -- 172.21.15.27:0/1408097181 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0100019f0 con 0x7fd0180cf540 2023-12-19T18:18:19.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.348+0000 7fd01ce5e640 1 -- 172.21.15.27:0/1408097181 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd00c02e050 con 0x7fd0180cf540 2023-12-19T18:18:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.348+0000 7fd01ce5e640 1 -- 172.21.15.27:0/1408097181 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd00c030710 con 0x7fd0180cf540 2023-12-19T18:18:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.348+0000 7fd01ee62640 1 -- 172.21.15.27:0/1408097181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0180cf540 msgr2=0x7fd01815a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.348+0000 7fd01ee62640 1 --2- 172.21.15.27:0/1408097181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0180cf540 0x7fd01815a5b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fd00c0097d0 tx=0x7fd00c02b800 comp rx=0 tx=0).stop 2023-12-19T18:18:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.348+0000 7fd01ee62640 1 -- 172.21.15.27:0/1408097181 shutdown_connections 2023-12-19T18:18:19.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.348+0000 7fd01ee62640 1 --2- 172.21.15.27:0/1408097181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0180cf540 0x7fd01815a5b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.348+0000 7fd01ee62640 1 -- 172.21.15.27:0/1408097181 >> 172.21.15.27:0/1408097181 conn(0x7fd01806a060 msgr2=0x7fd01806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:19.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.348+0000 7fd01ee62640 1 -- 172.21.15.27:0/1408097181 shutdown_connections 2023-12-19T18:18:19.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.348+0000 7fd01ee62640 1 -- 172.21.15.27:0/1408097181 wait complete. 2023-12-19T18:18:19.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.350+0000 7fd01ee62640 1 Processor -- start 2023-12-19T18:18:19.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.350+0000 7fd01ee62640 1 -- start start 2023-12-19T18:18:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.351+0000 7fd01ee62640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0180cf160 0x7fd018155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.351+0000 7fd01ee62640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0180cf540 0x7fd0181424f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.351+0000 7fd01ee62640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd01815aaf0 0x7fd01815dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.351+0000 7fd01ee62640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0181556b0 con 0x7fd0180cf160 2023-12-19T18:18:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.351+0000 7fd01ee62640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd018156010 con 0x7fd01815aaf0 2023-12-19T18:18:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.351+0000 7fd01ee62640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd018142a30 con 0x7fd0180cf540 2023-12-19T18:18:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.352+0000 7fd01de60640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0180cf160 0x7fd018155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.352+0000 7fd01de60640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0180cf160 0x7fd018155ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52972/0 (socket says 172.21.15.27:52972) 2023-12-19T18:18:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.352+0000 7fd01de60640 1 -- 172.21.15.27:0/4004213162 learned_addr learned my addr 172.21.15.27:0/4004213162 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:19.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.352+0000 7fd01d65f640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0180cf540 0x7fd0181424f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.352+0000 7fd01de60640 1 -- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0180cf540 msgr2=0x7fd0181424f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.352+0000 7fd01de60640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0180cf540 0x7fd0181424f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.352+0000 7fd01de60640 1 -- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd01815aaf0 msgr2=0x7fd01815dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.352+0000 7fd01e661640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd01815aaf0 0x7fd01815dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.352+0000 7fd01de60640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd01815aaf0 0x7fd01815dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.352+0000 7fd01de60640 1 -- 172.21.15.27:0/4004213162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd00c009400 con 0x7fd0180cf160 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.353+0000 7fd01d65f640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0180cf540 0x7fd0181424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.353+0000 7fd01e661640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd01815aaf0 0x7fd01815dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.353+0000 7fd01de60640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0180cf160 0x7fd018155ad0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd01400c490 tx=0x7fd01400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.353+0000 7fd00a7fc640 1 -- 172.21.15.27:0/4004213162 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0140052d0 con 0x7fd0180cf160 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.353+0000 7fd00a7fc640 1 -- 172.21.15.27:0/4004213162 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd014005430 con 0x7fd0180cf160 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.353+0000 7fd00a7fc640 1 -- 172.21.15.27:0/4004213162 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd014012870 con 0x7fd0180cf160 2023-12-19T18:18:19.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.353+0000 7fd01ee62640 1 -- 172.21.15.27:0/4004213162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0181547e0 con 0x7fd0180cf160 2023-12-19T18:18:19.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.353+0000 7fd01ee62640 1 -- 172.21.15.27:0/4004213162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd018154d10 con 0x7fd0180cf160 2023-12-19T18:18:19.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.354+0000 7fd01ee62640 1 -- 172.21.15.27:0/4004213162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0181557f0 con 0x7fd0180cf160 2023-12-19T18:18:19.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.356+0000 7fd00a7fc640 1 -- 172.21.15.27:0/4004213162 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd0140129d0 con 0x7fd0180cf160 2023-12-19T18:18:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.356+0000 7fd00a7fc640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcffc06a460 0x7fcffc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:19.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.357+0000 7fd00a7fc640 1 -- 172.21.15.27:0/4004213162 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd014088150 con 0x7fd0180cf160 2023-12-19T18:18:19.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.357+0000 7fd01d65f640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcffc06a460 0x7fcffc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:19.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.358+0000 7fd01d65f640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcffc06a460 0x7fcffc06c920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd00c034000 tx=0x7fd00c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:19.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.362+0000 7fd00a7fc640 1 -- 172.21.15.27:0/4004213162 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd014053ab0 con 0x7fd0180cf160 2023-12-19T18:18:19.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.561+0000 7fd01ee62640 1 -- 172.21.15.27:0/4004213162 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd01804e840 con 0x7fd0180cf160 2023-12-19T18:18:19.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.567+0000 7fd00a7fc640 1 -- 172.21.15.27:0/4004213162 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd0140570d0 con 0x7fd0180cf160 2023-12-19T18:18:19.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.570+0000 7fd01ee62640 1 -- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcffc06a460 msgr2=0x7fcffc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.570+0000 7fd01ee62640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcffc06a460 0x7fcffc06c920 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fd00c034000 tx=0x7fd00c008030 comp rx=0 tx=0).stop 2023-12-19T18:18:19.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.570+0000 7fd01ee62640 1 -- 172.21.15.27:0/4004213162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0180cf160 msgr2=0x7fd018155ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:19.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.570+0000 7fd01ee62640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0180cf160 0x7fd018155ad0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fd01400c490 tx=0x7fd01400c950 comp rx=0 tx=0).stop 2023-12-19T18:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.570+0000 7fd01ee62640 1 -- 172.21.15.27:0/4004213162 shutdown_connections 2023-12-19T18:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.570+0000 7fd01ee62640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcffc06a460 0x7fcffc06c920 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.570+0000 7fd01ee62640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd01815aaf0 0x7fd01815dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.570+0000 7fd01ee62640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0180cf540 0x7fd0181424f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.570+0000 7fd01ee62640 1 --2- 172.21.15.27:0/4004213162 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0180cf160 0x7fd018155ad0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.570+0000 7fd01ee62640 1 -- 172.21.15.27:0/4004213162 >> 172.21.15.27:0/4004213162 conn(0x7fd01806a060 msgr2=0x7fd018156890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.570+0000 7fd01ee62640 1 -- 172.21.15.27:0/4004213162 shutdown_connections 2023-12-19T18:18:19.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:19.570+0000 7fd01ee62640 1 -- 172.21.15.27:0/4004213162 wait complete. 2023-12-19T18:18:19.591 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:18:19.591 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:18:19.710 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:18:19.831 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:18:19.831 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:18:19.832 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:19.897 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:18:19.897 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:18:19.897 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:19.958 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:18:19.958 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:18:19.958 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:20.022 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:18:20.023 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:18:20.023 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:18:20.084 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:18:20.084 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:18:20.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.327+0000 7f3bc62bd640 1 Processor -- start 2023-12-19T18:18:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.327+0000 7f3bc62bd640 1 -- start start 2023-12-19T18:18:20.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.329+0000 7f3bc62bd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3bc0159c40 0x7f3bc006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.329+0000 7f3bc62bd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3bc015a020 0x7f3bc006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.329+0000 7f3bc62bd640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bc015b180 con 0x7f3bc006c880 2023-12-19T18:18:20.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.329+0000 7f3bc62bd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc00ca850 con 0x7f3bc0159c40 2023-12-19T18:18:20.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.329+0000 7f3bc62bd640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bc015a570 con 0x7f3bc015a020 2023-12-19T18:18:20.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.329+0000 7f3bbffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3bc0159c40 0x7f3bc006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.329+0000 7f3bbffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3bc0159c40 0x7f3bc006b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52974/0 (socket says 172.21.15.27:52974) 2023-12-19T18:18:20.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.329+0000 7f3bbffff640 1 -- 172.21.15.27:0/2746204742 learned_addr learned my addr 172.21.15.27:0/2746204742 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:20.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.330+0000 7f3bc485a640 1 --2- 172.21.15.27:0/2746204742 >> v2:172.21.15.192:3300/0 conn(0x7f3bc015a020 0x7f3bc006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.330+0000 7f3bbffff640 1 -- 172.21.15.27:0/2746204742 >> v2:172.21.15.192:3300/0 conn(0x7f3bc015a020 msgr2=0x7f3bc006ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.330+0000 7f3bbffff640 1 --2- 172.21.15.27:0/2746204742 >> v2:172.21.15.192:3300/0 conn(0x7f3bc015a020 0x7f3bc006ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.330+0000 7f3bbffff640 1 -- 172.21.15.27:0/2746204742 >> v1:172.21.15.192:6789/0 conn(0x7f3bc006c880 legacy=0x7f3bc00cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.330+0000 7f3bbffff640 1 -- 172.21.15.27:0/2746204742 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bc015a850 con 0x7f3bc0159c40 2023-12-19T18:18:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.331+0000 7f3bbffff640 1 --2- 172.21.15.27:0/2746204742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc0159c40 0x7f3bc006b4e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00ae580 tx=0x7f3bb002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51dfa40e1b99f0a0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.331+0000 7f3bbf7fe640 1 -- 172.21.15.27:0/2746204742 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0031870 con 0x7f3bc0159c40 2023-12-19T18:18:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.331+0000 7f3bbf7fe640 1 -- 172.21.15.27:0/2746204742 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb00319d0 con 0x7f3bc0159c40 2023-12-19T18:18:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.331+0000 7f3bbf7fe640 1 -- 172.21.15.27:0/2746204742 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb0031d60 con 0x7f3bc0159c40 2023-12-19T18:18:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.332+0000 7f3bc62bd640 1 -- 172.21.15.27:0/2746204742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc0159c40 msgr2=0x7f3bc006b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.332+0000 7f3bc62bd640 1 --2- 172.21.15.27:0/2746204742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc0159c40 0x7f3bc006b4e0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3bc00ae580 tx=0x7f3bb002b910 comp rx=0 tx=0).stop 2023-12-19T18:18:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.332+0000 7f3bc62bd640 1 -- 172.21.15.27:0/2746204742 shutdown_connections 2023-12-19T18:18:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.332+0000 7f3bc62bd640 1 --2- 172.21.15.27:0/2746204742 >> v2:172.21.15.192:3300/0 conn(0x7f3bc015a020 0x7f3bc006ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.332+0000 7f3bc62bd640 1 --2- 172.21.15.27:0/2746204742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc0159c40 0x7f3bc006b4e0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.332+0000 7f3bc62bd640 1 -- 172.21.15.27:0/2746204742 >> 172.21.15.27:0/2746204742 conn(0x7f3bc006a330 msgr2=0x7f3bc00b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.332+0000 7f3bc62bd640 1 -- 172.21.15.27:0/2746204742 shutdown_connections 2023-12-19T18:18:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.332+0000 7f3bc62bd640 1 -- 172.21.15.27:0/2746204742 wait complete. 2023-12-19T18:18:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.334+0000 7f3bc62bd640 1 Processor -- start 2023-12-19T18:18:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.334+0000 7f3bc62bd640 1 -- start start 2023-12-19T18:18:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.335+0000 7f3bc62bd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bc006c880 0x7f3bc014e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.335+0000 7f3bc62bd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc0159c40 0x7f3bc014ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.335+0000 7f3bc62bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc015a020 0x7f3bc00b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.335+0000 7f3bc62bd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc00b40f0 con 0x7f3bc015a020 2023-12-19T18:18:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.335+0000 7f3bc62bd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc014fca0 con 0x7f3bc006c880 2023-12-19T18:18:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.335+0000 7f3bc62bd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bc00b3b30 con 0x7f3bc0159c40 2023-12-19T18:18:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.335+0000 7f3bc485a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc015a020 0x7f3bc00b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.335+0000 7f3bc485a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc015a020 0x7f3bc00b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52982/0 (socket says 172.21.15.27:52982) 2023-12-19T18:18:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.335+0000 7f3bc485a640 1 -- 172.21.15.27:0/722381028 learned_addr learned my addr 172.21.15.27:0/722381028 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.336+0000 7f3bbffff640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bc006c880 0x7f3bc014e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.336+0000 7f3bb7fff640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc0159c40 0x7f3bc014ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.336+0000 7f3bc485a640 1 -- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc0159c40 msgr2=0x7f3bc014ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.336+0000 7f3bc485a640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc0159c40 0x7f3bc014ee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.336+0000 7f3bc485a640 1 -- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bc006c880 msgr2=0x7f3bc014e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.336+0000 7f3bc485a640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bc006c880 0x7f3bc014e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.336+0000 7f3bc485a640 1 -- 172.21.15.27:0/722381028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bb00094d0 con 0x7f3bc015a020 2023-12-19T18:18:20.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.336+0000 7f3bc485a640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc015a020 0x7f3bc00b2b00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800cc60 tx=0x7f3bb800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:20.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.336+0000 7f3bbd7fa640 1 -- 172.21.15.27:0/722381028 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb800ce70 con 0x7f3bc015a020 2023-12-19T18:18:20.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.336+0000 7f3bbd7fa640 1 -- 172.21.15.27:0/722381028 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bb8003990 con 0x7f3bc015a020 2023-12-19T18:18:20.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.336+0000 7f3bbd7fa640 1 -- 172.21.15.27:0/722381028 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bb8003d20 con 0x7f3bc015a020 2023-12-19T18:18:20.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.336+0000 7f3bc62bd640 1 -- 172.21.15.27:0/722381028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bc00b3e10 con 0x7f3bc015a020 2023-12-19T18:18:20.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.337+0000 7f3bc62bd640 1 -- 172.21.15.27:0/722381028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bc0153ef0 con 0x7f3bc015a020 2023-12-19T18:18:20.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.338+0000 7f3bc62bd640 1 -- 172.21.15.27:0/722381028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bc014f790 con 0x7f3bc015a020 2023-12-19T18:18:20.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.339+0000 7f3bbd7fa640 1 -- 172.21.15.27:0/722381028 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3bb8003e80 con 0x7f3bc015a020 2023-12-19T18:18:20.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.340+0000 7f3bbd7fa640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3ba006a430 0x7f3ba006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.340+0000 7f3bbd7fa640 1 -- 172.21.15.27:0/722381028 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3bb8088ae0 con 0x7f3bc015a020 2023-12-19T18:18:20.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.340+0000 7f3bbffff640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3ba006a430 0x7f3ba006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.342+0000 7f3bbffff640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3ba006a430 0x7f3ba006c8f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0036000 tx=0x7f3bb003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:20.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.344+0000 7f3bbd7fa640 1 -- 172.21.15.27:0/722381028 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3bc014f790 con 0x7f3bc015a020 2023-12-19T18:18:20.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.579+0000 7f3bc62bd640 1 -- 172.21.15.27:0/722381028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bc014f790 con 0x7f3bc015a020 2023-12-19T18:18:20.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.580+0000 7f3bbd7fa640 1 -- 172.21.15.27:0/722381028 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f3bc014f790 con 0x7f3bc015a020 2023-12-19T18:18:20.580 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:20.580 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":25,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4883,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/930391176","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":930391176},{"type":"v1","addr":"172.21.15.192:6837","nonce":930391176}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5179,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3606564673","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3606564673},{"type":"v1","addr":"172.21.15.192:6836","nonce":3606564673}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":5676,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/3604117097","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3604117097},{"type":"v1","addr":"172.21.15.27:6834","nonce":3604117097}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18},{"gid":6051,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1803467184","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":1803467184},{"type":"v1","addr":"172.21.15.27:6835","nonce":1803467184}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":25}],"filesystems":[]} 2023-12-19T18:18:20.580 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 25 2023-12-19T18:18:20.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.583+0000 7f3bc62bd640 1 -- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3ba006a430 msgr2=0x7f3ba006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.583+0000 7f3bc62bd640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3ba006a430 0x7f3ba006c8f0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f3bb0036000 tx=0x7f3bb003c040 comp rx=0 tx=0).stop 2023-12-19T18:18:20.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.583+0000 7f3bc62bd640 1 -- 172.21.15.27:0/722381028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc015a020 msgr2=0x7f3bc00b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.583+0000 7f3bc62bd640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc015a020 0x7f3bc00b2b00 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3bb800cc60 tx=0x7f3bb800a5c0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.583+0000 7f3bc62bd640 1 -- 172.21.15.27:0/722381028 shutdown_connections 2023-12-19T18:18:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.583+0000 7f3bc62bd640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3ba006a430 0x7f3ba006c8f0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.583+0000 7f3bc62bd640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bc015a020 0x7f3bc00b2b00 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.583+0000 7f3bc62bd640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bc0159c40 0x7f3bc014ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.583+0000 7f3bc62bd640 1 --2- 172.21.15.27:0/722381028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bc006c880 0x7f3bc014e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.583+0000 7f3bc62bd640 1 -- 172.21.15.27:0/722381028 >> 172.21.15.27:0/722381028 conn(0x7f3bc006a330 msgr2=0x7f3bc00c9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.583+0000 7f3bc62bd640 1 -- 172.21.15.27:0/722381028 shutdown_connections 2023-12-19T18:18:20.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.583+0000 7f3bc62bd640 1 -- 172.21.15.27:0/722381028 wait complete. 2023-12-19T18:18:20.602 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:18:20.603 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:18:20.603 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:18:20.603 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:18:20.603 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:18:20.603 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:18:20.603 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:18:20.603 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:18:20.604 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:18:20.604 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:18:20.604 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:18:20.604 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:18:20.604 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:18:20.604 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:18:20.604 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:18:20.605 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:18:20.618 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:18:20.618 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:18:20.623 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:18:20.623 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:18:20.624 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:18:20.624 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:18:20.628 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:18:20.629 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:18:20.671 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:18:20.671 DEBUG:teuthology.parallel:result is None 2023-12-19T18:18:20.676 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:18:20.676 DEBUG:teuthology.parallel:result is None 2023-12-19T18:18:20.679 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:18:20.679 DEBUG:teuthology.parallel:result is None 2023-12-19T18:18:20.684 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:18:20.684 DEBUG:teuthology.parallel:result is None 2023-12-19T18:18:20.684 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:18:20.839 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-19T18:18:20.839 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:18:20.840 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-19T18:18:20.844 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:18:20.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.885+0000 7f9d093d5640 1 Processor -- start 2023-12-19T18:18:20.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.885+0000 7f9d093d5640 1 -- start start 2023-12-19T18:18:20.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.886+0000 7f9d093d5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9d040d1580 0x7f9d04167e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.886+0000 7f9d093d5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d040b71b0 con 0x7f9d0406ab00 2023-12-19T18:18:20.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.886+0000 7f9d093d5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d040d1a50 con 0x7f9d0415d260 2023-12-19T18:18:20.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.886+0000 7f9d093d5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d040cdaf0 con 0x7f9d040d1580 2023-12-19T18:18:20.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.887+0000 7f9d037fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9d040d1580 0x7f9d04167e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.887+0000 7f9d037fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9d040d1580 0x7f9d04167e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53022/0 (socket says 172.21.15.27:53022) 2023-12-19T18:18:20.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.887+0000 7f9d037fe640 1 -- 172.21.15.27:0/3694467820 learned_addr learned my addr 172.21.15.27:0/3694467820 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:20.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.887+0000 7f9d037fe640 1 -- 172.21.15.27:0/3694467820 >> v1:172.21.15.192:6789/0 conn(0x7f9d0415d260 legacy=0x7f9d0406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.887+0000 7f9d037fe640 1 -- 172.21.15.27:0/3694467820 >> v1:172.21.15.27:6789/0 conn(0x7f9d0406ab00 legacy=0x7f9d040d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.887+0000 7f9d037fe640 1 -- 172.21.15.27:0/3694467820 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d040d1d30 con 0x7f9d040d1580 2023-12-19T18:18:20.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.888+0000 7f9d037fe640 1 --2- 172.21.15.27:0/3694467820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d040d1580 0x7f9d04167e20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009e50 tx=0x7f9cfc02c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e64f6e223854fa34 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:20.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.889+0000 7f9d01ffb640 1 -- 172.21.15.27:0/3694467820 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc02cbb0 con 0x7f9d040d1580 2023-12-19T18:18:20.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.889+0000 7f9d01ffb640 1 -- 172.21.15.27:0/3694467820 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cfc02cd10 con 0x7f9d040d1580 2023-12-19T18:18:20.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.889+0000 7f9d01ffb640 1 -- 172.21.15.27:0/3694467820 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cfc034610 con 0x7f9d040d1580 2023-12-19T18:18:20.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.889+0000 7f9d093d5640 1 -- 172.21.15.27:0/3694467820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d040d1580 msgr2=0x7f9d04167e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.889+0000 7f9d093d5640 1 --2- 172.21.15.27:0/3694467820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d040d1580 0x7f9d04167e20 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f9cfc009e50 tx=0x7f9cfc02c010 comp rx=0 tx=0).stop 2023-12-19T18:18:20.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.889+0000 7f9d093d5640 1 -- 172.21.15.27:0/3694467820 shutdown_connections 2023-12-19T18:18:20.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.889+0000 7f9d093d5640 1 --2- 172.21.15.27:0/3694467820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d040d1580 0x7f9d04167e20 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.889+0000 7f9d093d5640 1 -- 172.21.15.27:0/3694467820 >> 172.21.15.27:0/3694467820 conn(0x7f9d040d2f70 msgr2=0x7f9d040d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:20.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.889+0000 7f9d093d5640 1 -- 172.21.15.27:0/3694467820 shutdown_connections 2023-12-19T18:18:20.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.889+0000 7f9d093d5640 1 -- 172.21.15.27:0/3694467820 wait complete. 2023-12-19T18:18:20.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.891+0000 7f9d093d5640 1 Processor -- start 2023-12-19T18:18:20.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.891+0000 7f9d093d5640 1 -- start start 2023-12-19T18:18:20.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.892+0000 7f9d093d5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0406ab00 0x7f9d040c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.892+0000 7f9d093d5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d040d1580 0x7f9d041f29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.892+0000 7f9d093d5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d0415d260 0x7f9d041f2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.892+0000 7f9d093d5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041f3510 con 0x7f9d0406ab00 2023-12-19T18:18:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.892+0000 7f9d093d5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041f37f0 con 0x7f9d0415d260 2023-12-19T18:18:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.892+0000 7f9d093d5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d041f3ad0 con 0x7f9d040d1580 2023-12-19T18:18:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.893+0000 7f9d02ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0406ab00 0x7f9d040c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.893+0000 7f9d02ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0406ab00 0x7f9d040c8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53030/0 (socket says 172.21.15.27:53030) 2023-12-19T18:18:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.893+0000 7f9d02ffd640 1 -- 172.21.15.27:0/1385735698 learned_addr learned my addr 172.21.15.27:0/1385735698 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.893+0000 7f9d027fc640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d040d1580 0x7f9d041f29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.893+0000 7f9d037fe640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d0415d260 0x7f9d041f2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.893+0000 7f9d02ffd640 1 -- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d040d1580 msgr2=0x7f9d041f29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.893+0000 7f9d02ffd640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d040d1580 0x7f9d041f29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.893+0000 7f9d02ffd640 1 -- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d0415d260 msgr2=0x7f9d041f2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.893+0000 7f9d02ffd640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d0415d260 0x7f9d041f2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.893+0000 7f9d02ffd640 1 -- 172.21.15.27:0/1385735698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cfc009b30 con 0x7f9d0406ab00 2023-12-19T18:18:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.893+0000 7f9d02ffd640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0406ab00 0x7f9d040c8a80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400c400 tx=0x7f9cf400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.894+0000 7f9ceb7fe640 1 -- 172.21.15.27:0/1385735698 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf40052d0 con 0x7f9d0406ab00 2023-12-19T18:18:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.894+0000 7f9ceb7fe640 1 -- 172.21.15.27:0/1385735698 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf4005430 con 0x7f9d0406ab00 2023-12-19T18:18:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.894+0000 7f9d093d5640 1 -- 172.21.15.27:0/1385735698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d041f3db0 con 0x7f9d0406ab00 2023-12-19T18:18:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.894+0000 7f9ceb7fe640 1 -- 172.21.15.27:0/1385735698 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4003830 con 0x7f9d0406ab00 2023-12-19T18:18:20.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.894+0000 7f9d093d5640 1 -- 172.21.15.27:0/1385735698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d040c8fc0 con 0x7f9d0406ab00 2023-12-19T18:18:20.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.895+0000 7f9d093d5640 1 -- 172.21.15.27:0/1385735698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d04066810 con 0x7f9d0406ab00 2023-12-19T18:18:20.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.896+0000 7f9ceb7fe640 1 -- 172.21.15.27:0/1385735698 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9cf4003990 con 0x7f9d0406ab00 2023-12-19T18:18:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.897+0000 7f9ceb7fe640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cf006a430 0x7f9cf006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.897+0000 7f9ceb7fe640 1 -- 172.21.15.27:0/1385735698 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9cf4087c50 con 0x7f9d0406ab00 2023-12-19T18:18:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.898+0000 7f9d027fc640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cf006a430 0x7f9cf006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:20.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.898+0000 7f9d027fc640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cf006a430 0x7f9cf006c8f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d04153910 tx=0x7f9cf8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:20.901+0000 7f9ceb7fe640 1 -- 172.21.15.27:0/1385735698 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9cf40534a0 con 0x7f9d0406ab00 2023-12-19T18:18:21.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.107+0000 7f9d093d5640 1 -- 172.21.15.27:0/1385735698 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f9d041f42d0 con 0x7f9d0406ab00 2023-12-19T18:18:21.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.877+0000 7f9ceb7fe640 1 -- 172.21.15.27:0/1385735698 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v71) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9d041f42d0 con 0x7f9d0406ab00 2023-12-19T18:18:21.877 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2023-12-19T18:18:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.881+0000 7f9d093d5640 1 -- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cf006a430 msgr2=0x7f9cf006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.881+0000 7f9d093d5640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cf006a430 0x7f9cf006c8f0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9d04153910 tx=0x7f9cf8009040 comp rx=0 tx=0).stop 2023-12-19T18:18:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.881+0000 7f9d093d5640 1 -- 172.21.15.27:0/1385735698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0406ab00 msgr2=0x7f9d040c8a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.881+0000 7f9d093d5640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0406ab00 0x7f9d040c8a80 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400c400 tx=0x7f9cf400c8c0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.881+0000 7f9d093d5640 1 -- 172.21.15.27:0/1385735698 shutdown_connections 2023-12-19T18:18:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.881+0000 7f9d093d5640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cf006a430 0x7f9cf006c8f0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.881+0000 7f9d093d5640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9d0415d260 0x7f9d041f2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.881+0000 7f9d093d5640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9d040d1580 0x7f9d041f29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.881+0000 7f9d093d5640 1 --2- 172.21.15.27:0/1385735698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d0406ab00 0x7f9d040c8a80 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.881+0000 7f9d093d5640 1 -- 172.21.15.27:0/1385735698 >> 172.21.15.27:0/1385735698 conn(0x7f9d040d2f70 msgr2=0x7f9d040b7350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.882+0000 7f9d093d5640 1 -- 172.21.15.27:0/1385735698 shutdown_connections 2023-12-19T18:18:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:21.882+0000 7f9d093d5640 1 -- 172.21.15.27:0/1385735698 wait complete. 2023-12-19T18:18:21.905 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:18:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.139+0000 7fa64575c640 1 Processor -- start 2023-12-19T18:18:22.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.140+0000 7fa64575c640 1 -- start start 2023-12-19T18:18:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.142+0000 7fa64575c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64015adf0 0x7fa64006d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.142+0000 7fa64575c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa640070c00 0x7fa6400712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.142+0000 7fa64575c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa64006f500 0x7fa64006f990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.142+0000 7fa64575c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6400703c0 con 0x7fa640070c00 2023-12-19T18:18:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.142+0000 7fa64575c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa64006fed0 con 0x7fa64006f500 2023-12-19T18:18:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.142+0000 7fa64575c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa64006d8c0 con 0x7fa64015adf0 2023-12-19T18:18:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.142+0000 7fa6367fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa640070c00 0x7fa6400712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.142+0000 7fa6367fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa640070c00 0x7fa6400712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33922/0 (socket says 172.21.15.27:33922) 2023-12-19T18:18:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.142+0000 7fa6367fc640 1 -- 172.21.15.27:0/4097881351 learned_addr learned my addr 172.21.15.27:0/4097881351 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:22.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.142+0000 7fa63effd640 1 --2- 172.21.15.27:0/4097881351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64015adf0 0x7fa64006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.142+0000 7fa6367fc640 1 -- 172.21.15.27:0/4097881351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64015adf0 msgr2=0x7fa64006d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.143+0000 7fa6367fc640 1 --2- 172.21.15.27:0/4097881351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64015adf0 0x7fa64006d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.143+0000 7fa6367fc640 1 -- 172.21.15.27:0/4097881351 >> v2:172.21.15.192:3300/0 conn(0x7fa64006f500 msgr2=0x7fa64006f990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.143+0000 7fa6367fc640 1 --2- 172.21.15.27:0/4097881351 >> v2:172.21.15.192:3300/0 conn(0x7fa64006f500 0x7fa64006f990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.143+0000 7fa6367fc640 1 -- 172.21.15.27:0/4097881351 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa640071820 con 0x7fa640070c00 2023-12-19T18:18:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.143+0000 7fa63effd640 1 --2- 172.21.15.27:0/4097881351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64015adf0 0x7fa64006d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:22.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.144+0000 7fa6367fc640 1 --2- 172.21.15.27:0/4097881351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa640070c00 0x7fa6400712e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa6400cb230 tx=0x7fa62802b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fb1f5acfa2a563c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:22.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.144+0000 7fa63e7fc640 1 -- 172.21.15.27:0/4097881351 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628031740 con 0x7fa640070c00 2023-12-19T18:18:22.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.144+0000 7fa63e7fc640 1 -- 172.21.15.27:0/4097881351 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6280318a0 con 0x7fa640070c00 2023-12-19T18:18:22.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.144+0000 7fa63e7fc640 1 -- 172.21.15.27:0/4097881351 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa628031c30 con 0x7fa640070c00 2023-12-19T18:18:22.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.145+0000 7fa64575c640 1 -- 172.21.15.27:0/4097881351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa640070c00 msgr2=0x7fa6400712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.145+0000 7fa64575c640 1 --2- 172.21.15.27:0/4097881351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa640070c00 0x7fa6400712e0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fa6400cb230 tx=0x7fa62802b7e0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.145+0000 7fa64575c640 1 -- 172.21.15.27:0/4097881351 shutdown_connections 2023-12-19T18:18:22.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.145+0000 7fa64575c640 1 --2- 172.21.15.27:0/4097881351 >> v2:172.21.15.192:3300/0 conn(0x7fa64006f500 0x7fa64006f990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.145+0000 7fa64575c640 1 --2- 172.21.15.27:0/4097881351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa640070c00 0x7fa6400712e0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.145+0000 7fa64575c640 1 --2- 172.21.15.27:0/4097881351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64015adf0 0x7fa64006d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.145+0000 7fa64575c640 1 -- 172.21.15.27:0/4097881351 >> 172.21.15.27:0/4097881351 conn(0x7fa6400ceb50 msgr2=0x7fa6400cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:22.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.145+0000 7fa64575c640 1 -- 172.21.15.27:0/4097881351 shutdown_connections 2023-12-19T18:18:22.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.145+0000 7fa64575c640 1 -- 172.21.15.27:0/4097881351 wait complete. 2023-12-19T18:18:22.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.147+0000 7fa64575c640 1 Processor -- start 2023-12-19T18:18:22.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.147+0000 7fa64575c640 1 -- start start 2023-12-19T18:18:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.148+0000 7fa64575c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64006f500 0x7fa640145050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.148+0000 7fa64575c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa640070c00 0x7fa6401e4700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.148+0000 7fa64575c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa64015adf0 0x7fa6400c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.148+0000 7fa64575c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6400c4680 con 0x7fa64015adf0 2023-12-19T18:18:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.148+0000 7fa64575c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6401e4c40 con 0x7fa640070c00 2023-12-19T18:18:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.148+0000 7fa64575c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa640152480 con 0x7fa64006f500 2023-12-19T18:18:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.148+0000 7fa63f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa64015adf0 0x7fa6400c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.148+0000 7fa63f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa64015adf0 0x7fa6400c4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33938/0 (socket says 172.21.15.27:33938) 2023-12-19T18:18:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.148+0000 7fa63f7fe640 1 -- 172.21.15.27:0/259808156 learned_addr learned my addr 172.21.15.27:0/259808156 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.148+0000 7fa63effd640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64006f500 0x7fa640145050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.149+0000 7fa6367fc640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa640070c00 0x7fa6401e4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.149+0000 7fa63f7fe640 1 -- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64006f500 msgr2=0x7fa640145050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.149+0000 7fa63f7fe640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64006f500 0x7fa640145050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.149+0000 7fa63f7fe640 1 -- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa640070c00 msgr2=0x7fa6401e4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.149+0000 7fa63f7fe640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa640070c00 0x7fa6401e4700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.149+0000 7fa63f7fe640 1 -- 172.21.15.27:0/259808156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6280094d0 con 0x7fa64015adf0 2023-12-19T18:18:22.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.149+0000 7fa63f7fe640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa64015adf0 0x7fa6400c4b10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa63800b970 tx=0x7fa63800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:22.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.149+0000 7fa637fff640 1 -- 172.21.15.27:0/259808156 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63800cb40 con 0x7fa64015adf0 2023-12-19T18:18:22.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.149+0000 7fa637fff640 1 -- 172.21.15.27:0/259808156 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63800ccc0 con 0x7fa64015adf0 2023-12-19T18:18:22.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.149+0000 7fa64575c640 1 -- 172.21.15.27:0/259808156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6400c3a70 con 0x7fa64015adf0 2023-12-19T18:18:22.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.149+0000 7fa637fff640 1 -- 172.21.15.27:0/259808156 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638014680 con 0x7fa64015adf0 2023-12-19T18:18:22.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.149+0000 7fa64575c640 1 -- 172.21.15.27:0/259808156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6400c4030 con 0x7fa64015adf0 2023-12-19T18:18:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.151+0000 7fa64575c640 1 -- 172.21.15.27:0/259808156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6400c47f0 con 0x7fa64015adf0 2023-12-19T18:18:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.151+0000 7fa637fff640 1 -- 172.21.15.27:0/259808156 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa63801d3b0 con 0x7fa64015adf0 2023-12-19T18:18:22.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.153+0000 7fa637fff640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa62006a460 0x7fa62006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.153+0000 7fa637fff640 1 -- 172.21.15.27:0/259808156 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa638088590 con 0x7fa64015adf0 2023-12-19T18:18:22.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.153+0000 7fa63effd640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa62006a460 0x7fa62006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.154+0000 7fa63effd640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa62006a460 0x7fa62006c920 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa63000b840 tx=0x7fa630008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:22.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.157+0000 7fa637fff640 1 -- 172.21.15.27:0/259808156 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa638053e30 con 0x7fa64015adf0 2023-12-19T18:18:22.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.421+0000 7fa64575c640 1 -- 172.21.15.27:0/259808156 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fa640068e90 con 0x7fa64015adf0 2023-12-19T18:18:22.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.423+0000 7fa637fff640 1 -- 172.21.15.27:0/259808156 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fa638019090 con 0x7fa64015adf0 2023-12-19T18:18:22.423 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:22.423 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:18:22.423 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2023-12-19T18:18:22.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.423 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2023-12-19T18:18:22.423 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFl+4ruHBAAXwbpmT43SmlvNbuZj5e+bQ==", 2023-12-19T18:18:22.423 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.423 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:18:22.423 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:18:22.423 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmZyUBhAAG3kZkXvJIo7N9f1j+DPCSQ==", 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.425 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2023-12-19T18:18:22.425 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlChNOMBAAX4GV3iIGkSQdAsYu9oCPDw==", 2023-12-19T18:18:22.425 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.425 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:18:22.425 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:18:22.425 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:18:22.425 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.425 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.425 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2023-12-19T18:18:22.425 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlTmFXFhAABA0NMUxKBV1ExAhYc5RFbQ==", 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFluleFIBAAs1goFVMOX4CbjwFhg0KoBg==", 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFlInW3MxAAI4hY1GBxXYXDswKKrEyDTg==", 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFlAWZ5CRAA82/2r9/sT0L9VlGHbXksQw==", 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.427 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:22.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:18:22.428 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.428 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.428 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2023-12-19T18:18:22.428 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFl1IvaGRAAqVacDDPLx+NqBOGp6MeX2w==", 2023-12-19T18:18:22.428 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:22.428 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:18:22.428 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.428 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlr0BBHRAA+7gfs/RG6DtABnYPg059Tw==", 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlXZbELRAAKD0qOkhk7skfc4xcAC8Lhg==", 2023-12-19T18:18:22.429 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlF07MARAA2TJJ+5C1HIU+SVhqmKaVHA==", 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlISPGEhAAdFIRRkXlLyOx/oMZ4MgL9Q==", 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmNM9JRAARgS20O8zpm6H70WDv+lgGw==", 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T18:18:22.431 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlWoZ8AhAADWvKVzeDhDGHtYNUFIOWsA==", 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFllWFVMhAAD9ebDOppHTODVQjg6ppP3g==", 2023-12-19T18:18:22.432 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlwbIQDRAAt/h55zBNCEId0vkYSUQ3TQ==", 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:18:22.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB53YFloCb6CxAAmZQyz5XOEqDOOiROJHLzxQ==", 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFltBhxDxAAiAS9CYPY86IMmJ7NGgA2lw==", 2023-12-19T18:18:22.434 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlUnAhBBAAi++W9XK3o3Y/185hB3LuIg==", 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:22.435 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:18:22.436 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:18:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.425+0000 7fa64575c640 1 -- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa62006a460 msgr2=0x7fa62006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.425+0000 7fa64575c640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa62006a460 0x7fa62006c920 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa63000b840 tx=0x7fa630008040 comp rx=0 tx=0).stop 2023-12-19T18:18:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.425+0000 7fa64575c640 1 -- 172.21.15.27:0/259808156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa64015adf0 msgr2=0x7fa6400c4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.425+0000 7fa64575c640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa64015adf0 0x7fa6400c4b10 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fa63800b970 tx=0x7fa63800be30 comp rx=0 tx=0).stop 2023-12-19T18:18:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.425+0000 7fa64575c640 1 -- 172.21.15.27:0/259808156 shutdown_connections 2023-12-19T18:18:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.425+0000 7fa64575c640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa62006a460 0x7fa62006c920 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.425+0000 7fa64575c640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa64015adf0 0x7fa6400c4b10 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.425+0000 7fa64575c640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa640070c00 0x7fa6401e4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.425+0000 7fa64575c640 1 --2- 172.21.15.27:0/259808156 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa64006f500 0x7fa640145050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.425+0000 7fa64575c640 1 -- 172.21.15.27:0/259808156 >> 172.21.15.27:0/259808156 conn(0x7fa6400ceb50 msgr2=0x7fa6400b1ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.425+0000 7fa64575c640 1 -- 172.21.15.27:0/259808156 shutdown_connections 2023-12-19T18:18:22.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.425+0000 7fa64575c640 1 -- 172.21.15.27:0/259808156 wait complete. 2023-12-19T18:18:22.446 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.661+0000 7f1f9c88c640 1 Processor -- start 2023-12-19T18:18:22.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.661+0000 7f1f9c88c640 1 -- start start 2023-12-19T18:18:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.662+0000 7f1f9c88c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1f9815ee30 0x7f1f980c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.662+0000 7f1f9c88c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1f980afeb0 0x7f1f980c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.662+0000 7f1f9c88c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f980cb350 con 0x7f1f981553a0 2023-12-19T18:18:22.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.662+0000 7f1f9c88c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f980cbcf0 con 0x7f1f980afeb0 2023-12-19T18:18:22.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.662+0000 7f1f9c88c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f980cb630 con 0x7f1f9815ee30 2023-12-19T18:18:22.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.663+0000 7f1f97fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1f980afeb0 0x7f1f980c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.663+0000 7f1f97fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1f980afeb0 0x7f1f980c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33950/0 (socket says 172.21.15.27:33950) 2023-12-19T18:18:22.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.663+0000 7f1f97fff640 1 -- 172.21.15.27:0/2288002704 learned_addr learned my addr 172.21.15.27:0/2288002704 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:22.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.663+0000 7f1f8ffff640 1 --2- 172.21.15.27:0/2288002704 >> v2:172.21.15.192:3300/0 conn(0x7f1f9815ee30 0x7f1f980c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.663+0000 7f1f97fff640 1 -- 172.21.15.27:0/2288002704 >> v2:172.21.15.192:3300/0 conn(0x7f1f9815ee30 msgr2=0x7f1f980c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.663+0000 7f1f97fff640 1 --2- 172.21.15.27:0/2288002704 >> v2:172.21.15.192:3300/0 conn(0x7f1f9815ee30 0x7f1f980c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.663+0000 7f1f97fff640 1 -- 172.21.15.27:0/2288002704 >> v1:172.21.15.192:6789/0 conn(0x7f1f981553a0 legacy=0x7f1f9815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.663+0000 7f1f97fff640 1 -- 172.21.15.27:0/2288002704 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f980cb910 con 0x7f1f980afeb0 2023-12-19T18:18:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.664+0000 7f1f97fff640 1 --2- 172.21.15.27:0/2288002704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f980afeb0 0x7f1f980c39d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1f9806d4a0 tx=0x7f1f9002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4136e0a3ef09890 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:22.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.664+0000 7f1f96ffd640 1 -- 172.21.15.27:0/2288002704 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9002c9f0 con 0x7f1f980afeb0 2023-12-19T18:18:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.665+0000 7f1f96ffd640 1 -- 172.21.15.27:0/2288002704 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9002cb50 con 0x7f1f980afeb0 2023-12-19T18:18:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.665+0000 7f1f96ffd640 1 -- 172.21.15.27:0/2288002704 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f900343f0 con 0x7f1f980afeb0 2023-12-19T18:18:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.665+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2288002704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f980afeb0 msgr2=0x7f1f980c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.665+0000 7f1f9c88c640 1 --2- 172.21.15.27:0/2288002704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f980afeb0 0x7f1f980c39d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f1f9806d4a0 tx=0x7f1f9002be50 comp rx=0 tx=0).stop 2023-12-19T18:18:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.665+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2288002704 shutdown_connections 2023-12-19T18:18:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.665+0000 7f1f9c88c640 1 --2- 172.21.15.27:0/2288002704 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f980afeb0 0x7f1f980c39d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.665+0000 7f1f9c88c640 1 --2- 172.21.15.27:0/2288002704 >> v2:172.21.15.192:3300/0 conn(0x7f1f9815ee30 0x7f1f980c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.665+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2288002704 >> 172.21.15.27:0/2288002704 conn(0x7f1f98069c70 msgr2=0x7f1f9806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.665+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2288002704 shutdown_connections 2023-12-19T18:18:22.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.665+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2288002704 wait complete. 2023-12-19T18:18:22.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.667+0000 7f1f9c88c640 1 Processor -- start 2023-12-19T18:18:22.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.667+0000 7f1f9c88c640 1 -- start start 2023-12-19T18:18:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.668+0000 7f1f9c88c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f980afeb0 0x7f1f980b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.668+0000 7f1f9c88c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f981553a0 0x7f1f980c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.668+0000 7f1f9c88c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9815ee30 0x7f1f98149960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.668+0000 7f1f9c88c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f98144960 con 0x7f1f9815ee30 2023-12-19T18:18:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.668+0000 7f1f9c88c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f98143d90 con 0x7f1f980afeb0 2023-12-19T18:18:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.668+0000 7f1f9c88c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f980c6e20 con 0x7f1f981553a0 2023-12-19T18:18:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.668+0000 7f1f97fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9815ee30 0x7f1f98149960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.668+0000 7f1f97fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9815ee30 0x7f1f98149960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33966/0 (socket says 172.21.15.27:33966) 2023-12-19T18:18:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.668+0000 7f1f97fff640 1 -- 172.21.15.27:0/2206257487 learned_addr learned my addr 172.21.15.27:0/2206257487 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.669+0000 7f1f8ffff640 1 --2- 172.21.15.27:0/2206257487 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f981553a0 0x7f1f980c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.669+0000 7f1f97fff640 1 -- 172.21.15.27:0/2206257487 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f981553a0 msgr2=0x7f1f980c68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.669+0000 7f1f97fff640 1 --2- 172.21.15.27:0/2206257487 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f981553a0 0x7f1f980c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.669+0000 7f1f97fff640 1 -- 172.21.15.27:0/2206257487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f980afeb0 msgr2=0x7f1f980b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.669+0000 7f1f97fff640 1 --2- 172.21.15.27:0/2206257487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f980afeb0 0x7f1f980b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.669+0000 7f1f97fff640 1 -- 172.21.15.27:0/2206257487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f90009b30 con 0x7f1f9815ee30 2023-12-19T18:18:22.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.669+0000 7f1f97fff640 1 --2- 172.21.15.27:0/2206257487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9815ee30 0x7f1f98149960 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1f90036040 tx=0x7f1f90004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:22.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.669+0000 7f1f94ff9640 1 -- 172.21.15.27:0/2206257487 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f90009510 con 0x7f1f9815ee30 2023-12-19T18:18:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.669+0000 7f1f94ff9640 1 -- 172.21.15.27:0/2206257487 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f9003e450 con 0x7f1f9815ee30 2023-12-19T18:18:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.669+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2206257487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f98144030 con 0x7f1f9815ee30 2023-12-19T18:18:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.669+0000 7f1f94ff9640 1 -- 172.21.15.27:0/2206257487 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f9003e7e0 con 0x7f1f9815ee30 2023-12-19T18:18:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.669+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2206257487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f981445f0 con 0x7f1f9815ee30 2023-12-19T18:18:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.670+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2206257487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f98144780 con 0x7f1f9815ee30 2023-12-19T18:18:22.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.671+0000 7f1f94ff9640 1 -- 172.21.15.27:0/2206257487 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1f9003e940 con 0x7f1f9815ee30 2023-12-19T18:18:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.673+0000 7f1f94ff9640 1 --2- 172.21.15.27:0/2206257487 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f7806a430 0x7f1f7806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.673+0000 7f1f94ff9640 1 -- 172.21.15.27:0/2206257487 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1f900ab3f0 con 0x7f1f9815ee30 2023-12-19T18:18:22.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.673+0000 7f1f977fe640 1 --2- 172.21.15.27:0/2206257487 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f7806a430 0x7f1f7806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:22.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.674+0000 7f1f977fe640 1 --2- 172.21.15.27:0/2206257487 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f7806a430 0x7f1f7806c8f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f88001560 tx=0x7f1f88007990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:22.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.677+0000 7f1f94ff9640 1 -- 172.21.15.27:0/2206257487 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1f90076d20 con 0x7f1f9815ee30 2023-12-19T18:18:22.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.874+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2206257487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f9804e840 con 0x7f1f9815ee30 2023-12-19T18:18:22.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.875+0000 7f1f94ff9640 1 -- 172.21.15.27:0/2206257487 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+12389 (secure 0 0 0) 0x7f1f9007a340 con 0x7f1f9815ee30 2023-12-19T18:18:22.876 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:22.876 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":71,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:18:21.869596+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":54,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:22.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.878+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2206257487 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f7806a430 msgr2=0x7f1f7806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.878+0000 7f1f9c88c640 1 --2- 172.21.15.27:0/2206257487 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f7806a430 0x7f1f7806c8f0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1f88001560 tx=0x7f1f88007990 comp rx=0 tx=0).stop 2023-12-19T18:18:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.878+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2206257487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9815ee30 msgr2=0x7f1f98149960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.878+0000 7f1f9c88c640 1 --2- 172.21.15.27:0/2206257487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9815ee30 0x7f1f98149960 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f1f90036040 tx=0x7f1f90004a40 comp rx=0 tx=0).stop 2023-12-19T18:18:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.878+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2206257487 shutdown_connections 2023-12-19T18:18:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.878+0000 7f1f9c88c640 1 --2- 172.21.15.27:0/2206257487 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f7806a430 0x7f1f7806c8f0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.878+0000 7f1f9c88c640 1 --2- 172.21.15.27:0/2206257487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9815ee30 0x7f1f98149960 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.878+0000 7f1f9c88c640 1 --2- 172.21.15.27:0/2206257487 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f981553a0 0x7f1f980c68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.878+0000 7f1f9c88c640 1 --2- 172.21.15.27:0/2206257487 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f980afeb0 0x7f1f980b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.878+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2206257487 >> 172.21.15.27:0/2206257487 conn(0x7f1f98069c70 msgr2=0x7f1f98159d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.879+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2206257487 shutdown_connections 2023-12-19T18:18:22.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:22.879+0000 7f1f9c88c640 1 -- 172.21.15.27:0/2206257487 wait complete. 2023-12-19T18:18:22.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '63', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:18:22.900 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:18:23.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.123+0000 7fddaf59e640 1 Processor -- start 2023-12-19T18:18:23.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.123+0000 7fddaf59e640 1 -- start start 2023-12-19T18:18:23.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.125+0000 7fddaf59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddb015d120 0x7fddb00cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.125+0000 7fddaf59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fddb00ce950 0x7fddb00d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.125+0000 7fddaf59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb00d3850 con 0x7fddb00d1310 2023-12-19T18:18:23.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.125+0000 7fddaf59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb00d0b10 con 0x7fddb00ce950 2023-12-19T18:18:23.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.125+0000 7fddaf59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00d1c90 con 0x7fddb015d120 2023-12-19T18:18:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.125+0000 7fddaed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fddb00ce950 0x7fddb00d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.125+0000 7fddaed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fddb00ce950 0x7fddb00d1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33978/0 (socket says 172.21.15.27:33978) 2023-12-19T18:18:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.125+0000 7fddaed9d640 1 -- 172.21.15.27:0/877922675 learned_addr learned my addr 172.21.15.27:0/877922675 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.125+0000 7fddae59c640 1 --2- 172.21.15.27:0/877922675 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddb015d120 0x7fddb00cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.126+0000 7fddaed9d640 1 -- 172.21.15.27:0/877922675 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddb015d120 msgr2=0x7fddb00cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.126+0000 7fddaed9d640 1 --2- 172.21.15.27:0/877922675 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddb015d120 0x7fddb00cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.126+0000 7fddaed9d640 1 -- 172.21.15.27:0/877922675 >> v1:172.21.15.192:6789/0 conn(0x7fddb00d1310 legacy=0x7fddb00b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.126+0000 7fddaed9d640 1 -- 172.21.15.27:0/877922675 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb015d500 con 0x7fddb00ce950 2023-12-19T18:18:23.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.127+0000 7fddaed9d640 1 --2- 172.21.15.27:0/877922675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb00ce950 0x7fddb00d1750 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009870 tx=0x7fdda802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb090ee1533ed0ca server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.127+0000 7fddad59a640 1 -- 172.21.15.27:0/877922675 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda8030840 con 0x7fddb00ce950 2023-12-19T18:18:23.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.127+0000 7fddad59a640 1 -- 172.21.15.27:0/877922675 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda80309a0 con 0x7fddb00ce950 2023-12-19T18:18:23.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.127+0000 7fddad59a640 1 -- 172.21.15.27:0/877922675 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda8030d30 con 0x7fddb00ce950 2023-12-19T18:18:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.128+0000 7fddaf59e640 1 -- 172.21.15.27:0/877922675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb00ce950 msgr2=0x7fddb00d1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.128+0000 7fddaf59e640 1 --2- 172.21.15.27:0/877922675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb00ce950 0x7fddb00d1750 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fdda8009870 tx=0x7fdda802b980 comp rx=0 tx=0).stop 2023-12-19T18:18:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.128+0000 7fddaf59e640 1 -- 172.21.15.27:0/877922675 shutdown_connections 2023-12-19T18:18:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.128+0000 7fddaf59e640 1 --2- 172.21.15.27:0/877922675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb00ce950 0x7fddb00d1750 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.128+0000 7fddaf59e640 1 --2- 172.21.15.27:0/877922675 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddb015d120 0x7fddb00cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.128+0000 7fddaf59e640 1 -- 172.21.15.27:0/877922675 >> 172.21.15.27:0/877922675 conn(0x7fddb00d2d30 msgr2=0x7fddb00d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:23.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.128+0000 7fddaf59e640 1 -- 172.21.15.27:0/877922675 shutdown_connections 2023-12-19T18:18:23.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.128+0000 7fddaf59e640 1 -- 172.21.15.27:0/877922675 wait complete. 2023-12-19T18:18:23.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.130+0000 7fddaf59e640 1 Processor -- start 2023-12-19T18:18:23.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.130+0000 7fddaf59e640 1 -- start start 2023-12-19T18:18:23.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.131+0000 7fddaf59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddb00ce950 0x7fddb014e180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.131+0000 7fddaf59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddb00d1310 0x7fddb00c7010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.131+0000 7fddaf59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb015d120 0x7fddb00c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.131+0000 7fddaf59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00c95f0 con 0x7fddb015d120 2023-12-19T18:18:23.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.131+0000 7fddaf59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb014e6c0 con 0x7fddb00ce950 2023-12-19T18:18:23.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.131+0000 7fddaf59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb00c9310 con 0x7fddb00d1310 2023-12-19T18:18:23.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.131+0000 7fddaed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb015d120 0x7fddb00c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.131+0000 7fddaed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb015d120 0x7fddb00c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33992/0 (socket says 172.21.15.27:33992) 2023-12-19T18:18:23.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.131+0000 7fddaed9d640 1 -- 172.21.15.27:0/2420699257 learned_addr learned my addr 172.21.15.27:0/2420699257 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.132+0000 7fddae59c640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddb00ce950 0x7fddb014e180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.132+0000 7fddadd9b640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddb00d1310 0x7fddb00c7010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.132+0000 7fddaed9d640 1 -- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddb00d1310 msgr2=0x7fddb00c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.132+0000 7fddaed9d640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddb00d1310 0x7fddb00c7010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.132+0000 7fddaed9d640 1 -- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddb00ce950 msgr2=0x7fddb014e180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.132+0000 7fddaed9d640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddb00ce950 0x7fddb014e180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.132+0000 7fddaed9d640 1 -- 172.21.15.27:0/2420699257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda80094d0 con 0x7fddb015d120 2023-12-19T18:18:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.132+0000 7fddaed9d640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb015d120 0x7fddb00c7550 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdda802b9c0 tx=0x7fdda8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.133+0000 7fdda6ffd640 1 -- 172.21.15.27:0/2420699257 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda8039a90 con 0x7fddb015d120 2023-12-19T18:18:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.133+0000 7fdda6ffd640 1 -- 172.21.15.27:0/2420699257 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdda8039bf0 con 0x7fddb015d120 2023-12-19T18:18:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.133+0000 7fdda6ffd640 1 -- 172.21.15.27:0/2420699257 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdda80433f0 con 0x7fddb015d120 2023-12-19T18:18:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.133+0000 7fddaf59e640 1 -- 172.21.15.27:0/2420699257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb00cb170 con 0x7fddb015d120 2023-12-19T18:18:23.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.133+0000 7fddaf59e640 1 -- 172.21.15.27:0/2420699257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb00cb730 con 0x7fddb015d120 2023-12-19T18:18:23.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.134+0000 7fddaf59e640 1 -- 172.21.15.27:0/2420699257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb00cafd0 con 0x7fddb015d120 2023-12-19T18:18:23.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.135+0000 7fdda6ffd640 1 -- 172.21.15.27:0/2420699257 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdda804c3b0 con 0x7fddb015d120 2023-12-19T18:18:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.136+0000 7fdda6ffd640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd9006a430 0x7fdd9006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.136+0000 7fdda6ffd640 1 -- 172.21.15.27:0/2420699257 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fdda80aaa60 con 0x7fddb015d120 2023-12-19T18:18:23.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.137+0000 7fddae59c640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd9006a430 0x7fdd9006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.138+0000 7fddae59c640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd9006a430 0x7fdd9006c8f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdda00099c0 tx=0x7fdda0003990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.140+0000 7fdda6ffd640 1 -- 172.21.15.27:0/2420699257 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fddb00cafd0 con 0x7fddb015d120 2023-12-19T18:18:23.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.332+0000 7fddaf59e640 1 -- 172.21.15.27:0/2420699257 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fddb00cafd0 con 0x7fddb015d120 2023-12-19T18:18:23.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.333+0000 7fdda6ffd640 1 -- 172.21.15.27:0/2420699257 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v71) v1 ==== 93+0+10 (secure 0 0 0) 0x7fddb00cafd0 con 0x7fddb015d120 2023-12-19T18:18:23.333 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:18:23.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.335+0000 7fddaf59e640 1 -- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd9006a430 msgr2=0x7fdd9006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.335+0000 7fddaf59e640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd9006a430 0x7fdd9006c8f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdda00099c0 tx=0x7fdda0003990 comp rx=0 tx=0).stop 2023-12-19T18:18:23.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.335+0000 7fddaf59e640 1 -- 172.21.15.27:0/2420699257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb015d120 msgr2=0x7fddb00c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.335+0000 7fddaf59e640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb015d120 0x7fddb00c7550 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fdda802b9c0 tx=0x7fdda8004730 comp rx=0 tx=0).stop 2023-12-19T18:18:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.335+0000 7fddaf59e640 1 -- 172.21.15.27:0/2420699257 shutdown_connections 2023-12-19T18:18:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.335+0000 7fddaf59e640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd9006a430 0x7fdd9006c8f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.335+0000 7fddaf59e640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fddb015d120 0x7fddb00c7550 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.335+0000 7fddaf59e640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fddb00d1310 0x7fddb00c7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.335+0000 7fddaf59e640 1 --2- 172.21.15.27:0/2420699257 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fddb00ce950 0x7fddb014e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.335+0000 7fddaf59e640 1 -- 172.21.15.27:0/2420699257 >> 172.21.15.27:0/2420699257 conn(0x7fddb00d2d30 msgr2=0x7fddb006a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:23.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.335+0000 7fddaf59e640 1 -- 172.21.15.27:0/2420699257 shutdown_connections 2023-12-19T18:18:23.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.335+0000 7fddaf59e640 1 -- 172.21.15.27:0/2420699257 wait complete. 2023-12-19T18:18:23.355 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:18:23.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.591+0000 7f95b5766640 1 Processor -- start 2023-12-19T18:18:23.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.591+0000 7f95b5766640 1 -- start start 2023-12-19T18:18:23.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.593+0000 7f95b5766640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95b006fc10 0x7f95b006d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.593+0000 7f95b5766640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95b0070c00 0x7f95b00712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.593+0000 7f95b5766640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b0071820 con 0x7f95b015adf0 2023-12-19T18:18:23.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.593+0000 7f95b5766640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b00703c0 con 0x7f95b0070c00 2023-12-19T18:18:23.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.593+0000 7f95b5766640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b006d8c0 con 0x7f95b006fc10 2023-12-19T18:18:23.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.593+0000 7f95af7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95b0070c00 0x7f95b00712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.593+0000 7f95af7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95b0070c00 0x7f95b00712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33996/0 (socket says 172.21.15.27:33996) 2023-12-19T18:18:23.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.593+0000 7f95af7fe640 1 -- 172.21.15.27:0/3953433808 learned_addr learned my addr 172.21.15.27:0/3953433808 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:23.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.594+0000 7f95a7fff640 1 --2- 172.21.15.27:0/3953433808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95b006fc10 0x7f95b006d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.594+0000 7f95af7fe640 1 -- 172.21.15.27:0/3953433808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95b006fc10 msgr2=0x7f95b006d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:23.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.594+0000 7f95af7fe640 1 --2- 172.21.15.27:0/3953433808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95b006fc10 0x7f95b006d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.594+0000 7f95af7fe640 1 -- 172.21.15.27:0/3953433808 >> v1:172.21.15.192:6789/0 conn(0x7f95b015adf0 legacy=0x7f95b006f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:23.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.594+0000 7f95af7fe640 1 -- 172.21.15.27:0/3953433808 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b006b520 con 0x7f95b0070c00 2023-12-19T18:18:23.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.595+0000 7f95af7fe640 1 --2- 172.21.15.27:0/3953433808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95b0070c00 0x7f95b00712e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009e80 tx=0x7f95a802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309a90a924b50101 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.595+0000 7f95ae7fc640 1 -- 172.21.15.27:0/3953433808 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a802c9c0 con 0x7f95b0070c00 2023-12-19T18:18:23.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.595+0000 7f95ae7fc640 1 -- 172.21.15.27:0/3953433808 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a802cb20 con 0x7f95b0070c00 2023-12-19T18:18:23.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.596+0000 7f95ae7fc640 1 -- 172.21.15.27:0/3953433808 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a80343f0 con 0x7f95b0070c00 2023-12-19T18:18:23.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.596+0000 7f95b5766640 1 -- 172.21.15.27:0/3953433808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95b0070c00 msgr2=0x7f95b00712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.596+0000 7f95b5766640 1 --2- 172.21.15.27:0/3953433808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95b0070c00 0x7f95b00712e0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f95a8009e80 tx=0x7f95a802be20 comp rx=0 tx=0).stop 2023-12-19T18:18:23.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.596+0000 7f95b5766640 1 -- 172.21.15.27:0/3953433808 shutdown_connections 2023-12-19T18:18:23.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.596+0000 7f95b5766640 1 --2- 172.21.15.27:0/3953433808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95b0070c00 0x7f95b00712e0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.596+0000 7f95b5766640 1 --2- 172.21.15.27:0/3953433808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95b006fc10 0x7f95b006d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.596+0000 7f95b5766640 1 -- 172.21.15.27:0/3953433808 >> 172.21.15.27:0/3953433808 conn(0x7f95b00ceb50 msgr2=0x7f95b00cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:23.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.596+0000 7f95b5766640 1 -- 172.21.15.27:0/3953433808 shutdown_connections 2023-12-19T18:18:23.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.596+0000 7f95b5766640 1 -- 172.21.15.27:0/3953433808 wait complete. 2023-12-19T18:18:23.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.598+0000 7f95b5766640 1 Processor -- start 2023-12-19T18:18:23.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.598+0000 7f95b5766640 1 -- start start 2023-12-19T18:18:23.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95b5766640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95b006fc10 0x7f95b00c8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95b5766640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95b0070c00 0x7f95b00c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95b5766640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95b015adf0 0x7f95b01fa760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95b5766640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00c7780 con 0x7f95b015adf0 2023-12-19T18:18:23.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95b5766640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00c80e0 con 0x7f95b0070c00 2023-12-19T18:18:23.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95b5766640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00ce520 con 0x7f95b006fc10 2023-12-19T18:18:23.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95b015adf0 0x7f95b01fa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95af7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95b015adf0 0x7f95b01fa760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34002/0 (socket says 172.21.15.27:34002) 2023-12-19T18:18:23.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95af7fe640 1 -- 172.21.15.27:0/388462573 learned_addr learned my addr 172.21.15.27:0/388462573 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:23.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95a7fff640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95b0070c00 0x7f95b00c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95aeffd640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95b006fc10 0x7f95b00c8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95af7fe640 1 -- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95b006fc10 msgr2=0x7f95b00c8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95af7fe640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95b006fc10 0x7f95b00c8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95af7fe640 1 -- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95b0070c00 msgr2=0x7f95b00c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95af7fe640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95b0070c00 0x7f95b00c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95af7fe640 1 -- 172.21.15.27:0/388462573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a8009b30 con 0x7f95b015adf0 2023-12-19T18:18:23.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.600+0000 7f95aeffd640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95b006fc10 0x7f95b00c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:23.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.601+0000 7f95af7fe640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95b015adf0 0x7f95b01fa760 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f95a8036040 tx=0x7f95a8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.601+0000 7f95a77fe640 1 -- 172.21.15.27:0/388462573 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8009510 con 0x7f95b015adf0 2023-12-19T18:18:23.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.601+0000 7f95a77fe640 1 -- 172.21.15.27:0/388462573 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a803e450 con 0x7f95b015adf0 2023-12-19T18:18:23.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.601+0000 7f95a77fe640 1 -- 172.21.15.27:0/388462573 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a803e800 con 0x7f95b015adf0 2023-12-19T18:18:23.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.601+0000 7f95b5766640 1 -- 172.21.15.27:0/388462573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b00c6de0 con 0x7f95b015adf0 2023-12-19T18:18:23.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.601+0000 7f95b5766640 1 -- 172.21.15.27:0/388462573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b00c73a0 con 0x7f95b015adf0 2023-12-19T18:18:23.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.603+0000 7f95b5766640 1 -- 172.21.15.27:0/388462573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b00c78c0 con 0x7f95b015adf0 2023-12-19T18:18:23.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.604+0000 7f95a77fe640 1 -- 172.21.15.27:0/388462573 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f95a803e960 con 0x7f95b015adf0 2023-12-19T18:18:23.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.604+0000 7f95a77fe640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f959006a430 0x7f959006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:23.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.605+0000 7f95a77fe640 1 -- 172.21.15.27:0/388462573 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f95a80ab2f0 con 0x7f95b015adf0 2023-12-19T18:18:23.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.605+0000 7f95aeffd640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f959006a430 0x7f959006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:23.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.606+0000 7f95aeffd640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f959006a430 0x7f959006c8f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f95a000a830 tx=0x7f95a0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:23.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.609+0000 7f95a77fe640 1 -- 172.21.15.27:0/388462573 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f95b00c78c0 con 0x7f95b015adf0 2023-12-19T18:18:23.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.808+0000 7f95b5766640 1 -- 172.21.15.27:0/388462573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f95b00c78c0 con 0x7f95b015adf0 2023-12-19T18:18:23.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.809+0000 7f95a77fe640 1 -- 172.21.15.27:0/388462573 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v71) v1 ==== 84+0+65 (secure 0 0 0) 0x7f95b00c78c0 con 0x7f95b015adf0 2023-12-19T18:18:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:23.809 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-12-19T18:18:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:18:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-12-19T18:18:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-12-19T18:18:23.809 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:23.809 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-12-19T18:18:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.811+0000 7f95b5766640 1 -- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f959006a430 msgr2=0x7f959006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.811+0000 7f95b5766640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f959006a430 0x7f959006c8f0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f95a000a830 tx=0x7f95a0008040 comp rx=0 tx=0).stop 2023-12-19T18:18:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.811+0000 7f95b5766640 1 -- 172.21.15.27:0/388462573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95b015adf0 msgr2=0x7f95b01fa760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.811+0000 7f95b5766640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95b015adf0 0x7f95b01fa760 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f95a8036040 tx=0x7f95a8004a40 comp rx=0 tx=0).stop 2023-12-19T18:18:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.812+0000 7f95b5766640 1 -- 172.21.15.27:0/388462573 shutdown_connections 2023-12-19T18:18:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.812+0000 7f95b5766640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f959006a430 0x7f959006c8f0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.812+0000 7f95b5766640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95b015adf0 0x7f95b01fa760 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.812+0000 7f95b5766640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95b0070c00 0x7f95b00c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.812+0000 7f95b5766640 1 --2- 172.21.15.27:0/388462573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95b006fc10 0x7f95b00c8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:23.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.812+0000 7f95b5766640 1 -- 172.21.15.27:0/388462573 >> 172.21.15.27:0/388462573 conn(0x7f95b00ceb50 msgr2=0x7f95b00cf050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:23.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.812+0000 7f95b5766640 1 -- 172.21.15.27:0/388462573 shutdown_connections 2023-12-19T18:18:23.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:23.812+0000 7f95b5766640 1 -- 172.21.15.27:0/388462573 wait complete. 2023-12-19T18:18:23.832 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:18:23.833 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:18:24.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.072+0000 7f7b7e862640 1 Processor -- start 2023-12-19T18:18:24.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.072+0000 7f7b7e862640 1 -- start start 2023-12-19T18:18:24.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.074+0000 7f7b7e862640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b7815abf0 0x7f7b780cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.074+0000 7f7b7e862640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7b780cbbd0 0x7f7b780704c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.074+0000 7f7b7e862640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b780cbff0 con 0x7f7b78070a00 2023-12-19T18:18:24.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.074+0000 7f7b7e862640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b780ce640 con 0x7f7b780cbbd0 2023-12-19T18:18:24.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.074+0000 7f7b7e862640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78070e50 con 0x7f7b7815abf0 2023-12-19T18:18:24.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.075+0000 7f7b7d05f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7b780cbbd0 0x7f7b780704c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.075+0000 7f7b7d05f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7b780cbbd0 0x7f7b780704c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34014/0 (socket says 172.21.15.27:34014) 2023-12-19T18:18:24.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.075+0000 7f7b7d05f640 1 -- 172.21.15.27:0/1065755285 learned_addr learned my addr 172.21.15.27:0/1065755285 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:24.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.075+0000 7f7b7e061640 1 --1- 172.21.15.27:0/1065755285 >> v1:172.21.15.27:6789/0 conn(0x7f7b78070a00 0x7f7b78069c70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34636/0 (socket says 172.21.15.27:34636) 2023-12-19T18:18:24.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.075+0000 7f7b7d860640 1 --2- 172.21.15.27:0/1065755285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b7815abf0 0x7f7b780cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.075+0000 7f7b7d05f640 1 -- 172.21.15.27:0/1065755285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b7815abf0 msgr2=0x7f7b780cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:24.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.075+0000 7f7b7d05f640 1 --2- 172.21.15.27:0/1065755285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b7815abf0 0x7f7b780cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.075+0000 7f7b7d05f640 1 -- 172.21.15.27:0/1065755285 >> v1:172.21.15.27:6789/0 conn(0x7f7b78070a00 legacy=0x7f7b78069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.075+0000 7f7b7d05f640 1 -- 172.21.15.27:0/1065755285 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b7806b1e0 con 0x7f7b780cbbd0 2023-12-19T18:18:24.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.075+0000 7f7b7d860640 1 --2- 172.21.15.27:0/1065755285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b7815abf0 0x7f7b780cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:24.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.075+0000 7f7b7c85e640 1 -- 172.21.15.27:0/1065755285 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1790552970 0 0) 0x7f7b70001490 con 0x7f7b78070a00 2023-12-19T18:18:24.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.077+0000 7f7b7d05f640 1 --2- 172.21.15.27:0/1065755285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b780cbbd0 0x7f7b780704c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009870 tx=0x7f7b6802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db2857c2feb14ab server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:24.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.077+0000 7f7b7c85e640 1 -- 172.21.15.27:0/1065755285 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6802f830 con 0x7f7b780cbbd0 2023-12-19T18:18:24.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.077+0000 7f7b7c85e640 1 -- 172.21.15.27:0/1065755285 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b6802f990 con 0x7f7b780cbbd0 2023-12-19T18:18:24.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.077+0000 7f7b7c85e640 1 -- 172.21.15.27:0/1065755285 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b6802fd20 con 0x7f7b780cbbd0 2023-12-19T18:18:24.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.077+0000 7f7b7e862640 1 -- 172.21.15.27:0/1065755285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b780cbbd0 msgr2=0x7f7b780704c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.077+0000 7f7b7e862640 1 --2- 172.21.15.27:0/1065755285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b780cbbd0 0x7f7b780704c0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009870 tx=0x7f7b6802ba10 comp rx=0 tx=0).stop 2023-12-19T18:18:24.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.077+0000 7f7b7e862640 1 -- 172.21.15.27:0/1065755285 shutdown_connections 2023-12-19T18:18:24.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.078+0000 7f7b7e862640 1 --2- 172.21.15.27:0/1065755285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b780cbbd0 0x7f7b780704c0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.078+0000 7f7b7e862640 1 --2- 172.21.15.27:0/1065755285 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b7815abf0 0x7f7b780cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.078+0000 7f7b7e862640 1 -- 172.21.15.27:0/1065755285 >> 172.21.15.27:0/1065755285 conn(0x7f7b780cfa60 msgr2=0x7f7b780cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:24.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.078+0000 7f7b7e862640 1 -- 172.21.15.27:0/1065755285 shutdown_connections 2023-12-19T18:18:24.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.078+0000 7f7b7e862640 1 -- 172.21.15.27:0/1065755285 wait complete. 2023-12-19T18:18:24.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.080+0000 7f7b7e862640 1 Processor -- start 2023-12-19T18:18:24.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.080+0000 7f7b7e862640 1 -- start start 2023-12-19T18:18:24.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.081+0000 7f7b7e862640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b78070a00 0x7f7b78147e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.081+0000 7f7b7e862640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b780cbbd0 0x7f7b780c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.081+0000 7f7b7e862640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815abf0 0x7f7b780c8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.081+0000 7f7b7e862640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780c9dc0 con 0x7f7b7815abf0 2023-12-19T18:18:24.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.081+0000 7f7b7e862640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b781483c0 con 0x7f7b78070a00 2023-12-19T18:18:24.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.081+0000 7f7b7e862640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b780c9140 con 0x7f7b780cbbd0 2023-12-19T18:18:24.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.081+0000 7f7b7e061640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815abf0 0x7f7b780c8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.081+0000 7f7b7e061640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815abf0 0x7f7b780c8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34024/0 (socket says 172.21.15.27:34024) 2023-12-19T18:18:24.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.081+0000 7f7b7e061640 1 -- 172.21.15.27:0/4133927400 learned_addr learned my addr 172.21.15.27:0/4133927400 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:24.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.082+0000 7f7b7d05f640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b780cbbd0 0x7f7b780c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.082+0000 7f7b7d860640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b78070a00 0x7f7b78147e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.082+0000 7f7b7e061640 1 -- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b780cbbd0 msgr2=0x7f7b780c85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.082+0000 7f7b7e061640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b780cbbd0 0x7f7b780c85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.082+0000 7f7b7e061640 1 -- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b78070a00 msgr2=0x7f7b78147e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.082+0000 7f7b7e061640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b78070a00 0x7f7b78147e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.082+0000 7f7b7e061640 1 -- 172.21.15.27:0/4133927400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b680094d0 con 0x7f7b7815abf0 2023-12-19T18:18:24.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.082+0000 7f7b7e061640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815abf0 0x7f7b780c8b10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000c7d0 tx=0x7f7b7000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:24.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.082+0000 7f7b6e7fc640 1 -- 172.21.15.27:0/4133927400 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000a640 con 0x7f7b7815abf0 2023-12-19T18:18:24.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.082+0000 7f7b6e7fc640 1 -- 172.21.15.27:0/4133927400 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b7000a7c0 con 0x7f7b7815abf0 2023-12-19T18:18:24.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.082+0000 7f7b6e7fc640 1 -- 172.21.15.27:0/4133927400 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b7000ab70 con 0x7f7b7815abf0 2023-12-19T18:18:24.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.082+0000 7f7b7e862640 1 -- 172.21.15.27:0/4133927400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b780c9420 con 0x7f7b7815abf0 2023-12-19T18:18:24.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.083+0000 7f7b7e862640 1 -- 172.21.15.27:0/4133927400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b780c99e0 con 0x7f7b7815abf0 2023-12-19T18:18:24.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.084+0000 7f7b7e862640 1 -- 172.21.15.27:0/4133927400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b780c9f00 con 0x7f7b7815abf0 2023-12-19T18:18:24.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.085+0000 7f7b6e7fc640 1 -- 172.21.15.27:0/4133927400 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7b7000acd0 con 0x7f7b7815abf0 2023-12-19T18:18:24.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.086+0000 7f7b6e7fc640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5c06a430 0x7f7b5c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.086+0000 7f7b6e7fc640 1 -- 172.21.15.27:0/4133927400 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7b70087ae0 con 0x7f7b7815abf0 2023-12-19T18:18:24.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.086+0000 7f7b7d860640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5c06a430 0x7f7b5c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.087+0000 7f7b7d860640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5c06a430 0x7f7b5c06c8f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7b780c58e0 tx=0x7f7b74008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:24.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.090+0000 7f7b6e7fc640 1 -- 172.21.15.27:0/4133927400 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7b780c9f00 con 0x7f7b7815abf0 2023-12-19T18:18:24.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.283+0000 7f7b7e862640 1 -- 172.21.15.27:0/4133927400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f7b780cbff0 con 0x7f7b7815abf0 2023-12-19T18:18:24.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.618+0000 7f7b6e7fc640 1 -- 172.21.15.27:0/4133927400 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v72) v1 ==== 138+0+0 (secure 0 0 0) 0x7f7b780cbff0 con 0x7f7b7815abf0 2023-12-19T18:18:24.619 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-12-19T18:18:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.623+0000 7f7b7e862640 1 -- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5c06a430 msgr2=0x7f7b5c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.623+0000 7f7b7e862640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5c06a430 0x7f7b5c06c8f0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f7b780c58e0 tx=0x7f7b74008040 comp rx=0 tx=0).stop 2023-12-19T18:18:24.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.623+0000 7f7b7e862640 1 -- 172.21.15.27:0/4133927400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815abf0 msgr2=0x7f7b780c8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.623+0000 7f7b7e862640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815abf0 0x7f7b780c8b10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f7b7000c7d0 tx=0x7f7b7000cc90 comp rx=0 tx=0).stop 2023-12-19T18:18:24.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.625+0000 7f7b7e862640 1 -- 172.21.15.27:0/4133927400 shutdown_connections 2023-12-19T18:18:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.625+0000 7f7b7e862640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b5c06a430 0x7f7b5c06c8f0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.625+0000 7f7b7e862640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b7815abf0 0x7f7b780c8b10 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.625+0000 7f7b7e862640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7b780cbbd0 0x7f7b780c85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.625+0000 7f7b7e862640 1 --2- 172.21.15.27:0/4133927400 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7b78070a00 0x7f7b78147e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.625+0000 7f7b7e862640 1 -- 172.21.15.27:0/4133927400 >> 172.21.15.27:0/4133927400 conn(0x7f7b780cfa60 msgr2=0x7f7b780b2720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.626+0000 7f7b7e862640 1 -- 172.21.15.27:0/4133927400 shutdown_connections 2023-12-19T18:18:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.626+0000 7f7b7e862640 1 -- 172.21.15.27:0/4133927400 wait complete. 2023-12-19T18:18:24.647 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:18:24.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.866+0000 7ff437207640 1 Processor -- start 2023-12-19T18:18:24.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.866+0000 7ff437207640 1 -- start start 2023-12-19T18:18:24.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.868+0000 7ff437207640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff4300d0d60 0x7ff4300cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.868+0000 7ff437207640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4300d2bc0 con 0x7ff43015ce50 2023-12-19T18:18:24.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.868+0000 7ff437207640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4300d1ee0 con 0x7ff4300cfcb0 2023-12-19T18:18:24.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.868+0000 7ff437207640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4300d00c0 con 0x7ff4300d0d60 2023-12-19T18:18:24.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.868+0000 7ff435a04640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff4300d0d60 0x7ff4300cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.868+0000 7ff435a04640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff4300d0d60 0x7ff4300cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34026/0 (socket says 172.21.15.27:34026) 2023-12-19T18:18:24.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.868+0000 7ff435a04640 1 -- 172.21.15.27:0/1349714615 learned_addr learned my addr 172.21.15.27:0/1349714615 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:24.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.869+0000 7ff435a04640 1 -- 172.21.15.27:0/1349714615 >> v1:172.21.15.192:6789/0 conn(0x7ff4300cfcb0 legacy=0x7ff4300b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:24.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.869+0000 7ff435a04640 1 -- 172.21.15.27:0/1349714615 >> v1:172.21.15.27:6789/0 conn(0x7ff43015ce50 legacy=0x7ff4300d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.869+0000 7ff435a04640 1 -- 172.21.15.27:0/1349714615 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4300d12d0 con 0x7ff4300d0d60 2023-12-19T18:18:24.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.869+0000 7ff435203640 1 -- 172.21.15.27:0/1349714615 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 856427891 0 0) 0x7ff420001490 con 0x7ff43015ce50 2023-12-19T18:18:24.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.870+0000 7ff435a04640 1 --2- 172.21.15.27:0/1349714615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4300d0d60 0x7ff4300cf770 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff424009870 tx=0x7ff42402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253fda6b713acf15 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:24.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.871+0000 7ff435203640 1 -- 172.21.15.27:0/1349714615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424030860 con 0x7ff4300d0d60 2023-12-19T18:18:24.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.871+0000 7ff435203640 1 -- 172.21.15.27:0/1349714615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4240309c0 con 0x7ff4300d0d60 2023-12-19T18:18:24.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.871+0000 7ff435203640 1 -- 172.21.15.27:0/1349714615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff424030d50 con 0x7ff4300d0d60 2023-12-19T18:18:24.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.871+0000 7ff437207640 1 -- 172.21.15.27:0/1349714615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4300d0d60 msgr2=0x7ff4300cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.871+0000 7ff437207640 1 --2- 172.21.15.27:0/1349714615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4300d0d60 0x7ff4300cf770 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7ff424009870 tx=0x7ff42402b9a0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.871+0000 7ff437207640 1 -- 172.21.15.27:0/1349714615 shutdown_connections 2023-12-19T18:18:24.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.871+0000 7ff437207640 1 --2- 172.21.15.27:0/1349714615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4300d0d60 0x7ff4300cf770 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.871+0000 7ff437207640 1 -- 172.21.15.27:0/1349714615 >> 172.21.15.27:0/1349714615 conn(0x7ff43006a2d0 msgr2=0x7ff43006a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:24.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.871+0000 7ff437207640 1 -- 172.21.15.27:0/1349714615 shutdown_connections 2023-12-19T18:18:24.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.871+0000 7ff437207640 1 -- 172.21.15.27:0/1349714615 wait complete. 2023-12-19T18:18:24.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.873+0000 7ff437207640 1 Processor -- start 2023-12-19T18:18:24.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.873+0000 7ff437207640 1 -- start start 2023-12-19T18:18:24.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.874+0000 7ff437207640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4300cfcb0 0x7ff4300d2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.874+0000 7ff437207640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4300d0d60 0x7ff4301431d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.874+0000 7ff437207640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff43015ce50 0x7ff430143710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.874+0000 7ff437207640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff430154570 con 0x7ff4300cfcb0 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.874+0000 7ff437207640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff430143d90 con 0x7ff43015ce50 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.874+0000 7ff437207640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff430144070 con 0x7ff4300d0d60 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.875+0000 7ff436205640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4300cfcb0 0x7ff4300d2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.875+0000 7ff436205640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4300cfcb0 0x7ff4300d2300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34040/0 (socket says 172.21.15.27:34040) 2023-12-19T18:18:24.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.875+0000 7ff436205640 1 -- 172.21.15.27:0/2682600019 learned_addr learned my addr 172.21.15.27:0/2682600019 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:24.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.875+0000 7ff436205640 1 -- 172.21.15.27:0/2682600019 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4300d0d60 msgr2=0x7ff4301431d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.875+0000 7ff436205640 1 --2- 172.21.15.27:0/2682600019 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4300d0d60 0x7ff4301431d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.875+0000 7ff436205640 1 -- 172.21.15.27:0/2682600019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff43015ce50 msgr2=0x7ff430143710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:24.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.875+0000 7ff436a06640 1 --2- 172.21.15.27:0/2682600019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff43015ce50 0x7ff430143710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.875+0000 7ff436205640 1 --2- 172.21.15.27:0/2682600019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff43015ce50 0x7ff430143710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:24.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.875+0000 7ff436205640 1 -- 172.21.15.27:0/2682600019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4240094d0 con 0x7ff4300cfcb0 2023-12-19T18:18:24.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.876+0000 7ff436205640 1 --2- 172.21.15.27:0/2682600019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4300cfcb0 0x7ff4300d2300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff42000c7f0 tx=0x7ff42000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:24.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.876+0000 7ff42effd640 1 -- 172.21.15.27:0/2682600019 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff420003da0 con 0x7ff4300cfcb0 2023-12-19T18:18:24.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.876+0000 7ff42effd640 1 -- 172.21.15.27:0/2682600019 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff420005070 con 0x7ff4300cfcb0 2023-12-19T18:18:24.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.876+0000 7ff42effd640 1 -- 172.21.15.27:0/2682600019 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff420005400 con 0x7ff4300cfcb0 2023-12-19T18:18:24.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.876+0000 7ff437207640 1 -- 172.21.15.27:0/2682600019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff430141190 con 0x7ff4300cfcb0 2023-12-19T18:18:24.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.876+0000 7ff437207640 1 -- 172.21.15.27:0/2682600019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff430141750 con 0x7ff4300cfcb0 2023-12-19T18:18:24.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.877+0000 7ff437207640 1 -- 172.21.15.27:0/2682600019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4300721f0 con 0x7ff4300cfcb0 2023-12-19T18:18:24.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.880+0000 7ff42effd640 1 -- 172.21.15.27:0/2682600019 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff420005560 con 0x7ff4300cfcb0 2023-12-19T18:18:24.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.881+0000 7ff42effd640 1 --2- 172.21.15.27:0/2682600019 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff41c06a430 0x7ff41c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:24.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.881+0000 7ff42effd640 1 -- 172.21.15.27:0/2682600019 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 5227+0+0 (secure 0 0 0) 0x7ff4200882d0 con 0x7ff4300cfcb0 2023-12-19T18:18:24.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.881+0000 7ff435a04640 1 --2- 172.21.15.27:0/2682600019 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff41c06a430 0x7ff41c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:24.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.882+0000 7ff435a04640 1 --2- 172.21.15.27:0/2682600019 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff41c06a430 0x7ff41c06c8f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff424036000 tx=0x7ff42403b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:24.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:24.884+0000 7ff42effd640 1 -- 172.21.15.27:0/2682600019 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff420053a80 con 0x7ff4300cfcb0 2023-12-19T18:18:25.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.080+0000 7ff437207640 1 -- 172.21.15.27:0/2682600019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff43004e840 con 0x7ff4300cfcb0 2023-12-19T18:18:25.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.634+0000 7ff42effd640 1 -- 172.21.15.27:0/2682600019 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff420005810 con 0x7ff4300cfcb0 2023-12-19T18:18:25.638 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-12-19T18:18:25.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.645+0000 7ff437207640 1 -- 172.21.15.27:0/2682600019 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff41c06a430 msgr2=0x7ff41c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.645+0000 7ff437207640 1 --2- 172.21.15.27:0/2682600019 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff41c06a430 0x7ff41c06c8f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ff424036000 tx=0x7ff42403b040 comp rx=0 tx=0).stop 2023-12-19T18:18:25.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.645+0000 7ff437207640 1 -- 172.21.15.27:0/2682600019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4300cfcb0 msgr2=0x7ff4300d2300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.645+0000 7ff437207640 1 --2- 172.21.15.27:0/2682600019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4300cfcb0 0x7ff4300d2300 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7ff42000c7f0 tx=0x7ff42000ccb0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.647+0000 7ff437207640 1 -- 172.21.15.27:0/2682600019 shutdown_connections 2023-12-19T18:18:25.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.647+0000 7ff437207640 1 --2- 172.21.15.27:0/2682600019 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff41c06a430 0x7ff41c06c8f0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.647+0000 7ff437207640 1 --2- 172.21.15.27:0/2682600019 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff43015ce50 0x7ff430143710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.647+0000 7ff437207640 1 --2- 172.21.15.27:0/2682600019 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4300d0d60 0x7ff4301431d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.647+0000 7ff437207640 1 --2- 172.21.15.27:0/2682600019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4300cfcb0 0x7ff4300d2300 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.647+0000 7ff437207640 1 -- 172.21.15.27:0/2682600019 >> 172.21.15.27:0/2682600019 conn(0x7ff43006a2d0 msgr2=0x7ff43015fcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:25.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.648+0000 7ff437207640 1 -- 172.21.15.27:0/2682600019 shutdown_connections 2023-12-19T18:18:25.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.651+0000 7ff437207640 1 -- 172.21.15.27:0/2682600019 wait complete. 2023-12-19T18:18:25.674 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:18:25.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.893+0000 7f022bfff640 1 Processor -- start 2023-12-19T18:18:25.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.893+0000 7f022bfff640 1 -- start start 2023-12-19T18:18:25.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.894+0000 7f022bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f022c15cf80 0x7f022c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.894+0000 7f022bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022c0b8a60 0x7f022c0b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.895+0000 7f022bfff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022c0d0e10 con 0x7f022c0b85e0 2023-12-19T18:18:25.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.895+0000 7f022bfff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022c0d1c70 con 0x7f022c15cf80 2023-12-19T18:18:25.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.895+0000 7f022bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c15d360 con 0x7f022c0b8a60 2023-12-19T18:18:25.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.895+0000 7f022affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f022c15cf80 0x7f022c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.895+0000 7f022affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f022c15cf80 0x7f022c0d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34054/0 (socket says 172.21.15.27:34054) 2023-12-19T18:18:25.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.895+0000 7f022affd640 1 -- 172.21.15.27:0/1440171540 learned_addr learned my addr 172.21.15.27:0/1440171540 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:25.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.895+0000 7f022b7fe640 1 --2- 172.21.15.27:0/1440171540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022c0b8a60 0x7f022c0b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.895+0000 7f022affd640 1 -- 172.21.15.27:0/1440171540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022c0b8a60 msgr2=0x7f022c0b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:25.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.895+0000 7f022affd640 1 --2- 172.21.15.27:0/1440171540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022c0b8a60 0x7f022c0b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.895+0000 7f022affd640 1 -- 172.21.15.27:0/1440171540 >> v1:172.21.15.192:6789/0 conn(0x7f022c0b85e0 legacy=0x7f022c0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:25.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.895+0000 7f022affd640 1 -- 172.21.15.27:0/1440171540 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f022c0d1060 con 0x7f022c15cf80 2023-12-19T18:18:25.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.897+0000 7f022affd640 1 --2- 172.21.15.27:0/1440171540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022c15cf80 0x7f022c0d2100 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0214009870 tx=0x7f021402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6689893404f8f674 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:25.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.897+0000 7f0229ffb640 1 -- 172.21.15.27:0/1440171540 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214030840 con 0x7f022c15cf80 2023-12-19T18:18:25.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.897+0000 7f0229ffb640 1 -- 172.21.15.27:0/1440171540 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02140309a0 con 0x7f022c15cf80 2023-12-19T18:18:25.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.897+0000 7f0229ffb640 1 -- 172.21.15.27:0/1440171540 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0214030d30 con 0x7f022c15cf80 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.898+0000 7f022bfff640 1 -- 172.21.15.27:0/1440171540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022c15cf80 msgr2=0x7f022c0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.898+0000 7f022bfff640 1 --2- 172.21.15.27:0/1440171540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022c15cf80 0x7f022c0d2100 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0214009870 tx=0x7f021402b980 comp rx=0 tx=0).stop 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.898+0000 7f022bfff640 1 -- 172.21.15.27:0/1440171540 shutdown_connections 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.898+0000 7f022bfff640 1 --2- 172.21.15.27:0/1440171540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022c0b8a60 0x7f022c0b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.898+0000 7f022bfff640 1 --2- 172.21.15.27:0/1440171540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022c15cf80 0x7f022c0d2100 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.898+0000 7f022bfff640 1 -- 172.21.15.27:0/1440171540 >> 172.21.15.27:0/1440171540 conn(0x7f022c072e20 msgr2=0x7f022c073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.898+0000 7f022bfff640 1 -- 172.21.15.27:0/1440171540 shutdown_connections 2023-12-19T18:18:25.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.898+0000 7f022bfff640 1 -- 172.21.15.27:0/1440171540 wait complete. 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.900+0000 7f022bfff640 1 Processor -- start 2023-12-19T18:18:25.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.900+0000 7f022bfff640 1 -- start start 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.901+0000 7f022bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f022c0b85e0 0x7f022c1428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.901+0000 7f022bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022c15cf80 0x7f022c15ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.901+0000 7f022bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022c153090 0x7f022c155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.901+0000 7f022bfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0caee0 con 0x7f022c153090 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.901+0000 7f022bfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f022c0ca610 con 0x7f022c0b85e0 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.901+0000 7f022bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f022c155c70 con 0x7f022c15cf80 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.901+0000 7f022b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022c153090 0x7f022c155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.901+0000 7f022b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022c153090 0x7f022c155300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34070/0 (socket says 172.21.15.27:34070) 2023-12-19T18:18:25.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.901+0000 7f022b7fe640 1 -- 172.21.15.27:0/399772635 learned_addr learned my addr 172.21.15.27:0/399772635 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.901+0000 7f022affd640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f022c0b85e0 0x7f022c1428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.901+0000 7f022a7fc640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022c15cf80 0x7f022c15ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.902+0000 7f022b7fe640 1 -- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022c15cf80 msgr2=0x7f022c15ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.902+0000 7f022b7fe640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022c15cf80 0x7f022c15ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.902+0000 7f022b7fe640 1 -- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f022c0b85e0 msgr2=0x7f022c1428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.902+0000 7f022b7fe640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f022c0b85e0 0x7f022c1428b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.902+0000 7f022b7fe640 1 -- 172.21.15.27:0/399772635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02140094d0 con 0x7f022c153090 2023-12-19T18:18:25.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.902+0000 7f022affd640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f022c0b85e0 0x7f022c1428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:25.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.902+0000 7f022a7fc640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022c15cf80 0x7f022c15ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:25.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.902+0000 7f022b7fe640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022c153090 0x7f022c155300 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f022400d4f0 tx=0x7f022400d9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:25.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.902+0000 7f02237fe640 1 -- 172.21.15.27:0/399772635 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02240125f0 con 0x7f022c153090 2023-12-19T18:18:25.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.902+0000 7f02237fe640 1 -- 172.21.15.27:0/399772635 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0224012770 con 0x7f022c153090 2023-12-19T18:18:25.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.902+0000 7f022bfff640 1 -- 172.21.15.27:0/399772635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f022c0ca890 con 0x7f022c153090 2023-12-19T18:18:25.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.902+0000 7f02237fe640 1 -- 172.21.15.27:0/399772635 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224012b20 con 0x7f022c153090 2023-12-19T18:18:25.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.902+0000 7f022bfff640 1 -- 172.21.15.27:0/399772635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022c160510 con 0x7f022c153090 2023-12-19T18:18:25.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.904+0000 7f022bfff640 1 -- 172.21.15.27:0/399772635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f022c0cb090 con 0x7f022c153090 2023-12-19T18:18:25.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.905+0000 7f02237fe640 1 -- 172.21.15.27:0/399772635 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0224012c80 con 0x7f022c153090 2023-12-19T18:18:25.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.906+0000 7f02237fe640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020c06a460 0x7f020c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:25.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.906+0000 7f02237fe640 1 -- 172.21.15.27:0/399772635 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f0224088ce0 con 0x7f022c153090 2023-12-19T18:18:25.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.907+0000 7f022affd640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020c06a460 0x7f020c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:25.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.908+0000 7f022affd640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020c06a460 0x7f020c06c920 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0214036000 tx=0x7f021403b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:25.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:25.911+0000 7f02237fe640 1 -- 172.21.15.27:0/399772635 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f022c0cb090 con 0x7f022c153090 2023-12-19T18:18:26.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.144+0000 7f022bfff640 1 -- 172.21.15.27:0/399772635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f022c04e840 con 0x7f022c153090 2023-12-19T18:18:26.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.157+0000 7f02237fe640 1 -- 172.21.15.27:0/399772635 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v27) v1 ==== 172+0+0 (secure 0 0 0) 0x7f0224054320 con 0x7f022c153090 2023-12-19T18:18:26.158 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 8 and data pool 9 2023-12-19T18:18:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.161+0000 7f022bfff640 1 -- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020c06a460 msgr2=0x7f020c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.161+0000 7f022bfff640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020c06a460 0x7f020c06c920 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f0214036000 tx=0x7f021403b040 comp rx=0 tx=0).stop 2023-12-19T18:18:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.161+0000 7f022bfff640 1 -- 172.21.15.27:0/399772635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022c153090 msgr2=0x7f022c155300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.161+0000 7f022bfff640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022c153090 0x7f022c155300 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f022400d4f0 tx=0x7f022400d9b0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.161+0000 7f022bfff640 1 -- 172.21.15.27:0/399772635 shutdown_connections 2023-12-19T18:18:26.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.161+0000 7f022bfff640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020c06a460 0x7f020c06c920 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.161+0000 7f022bfff640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022c153090 0x7f022c155300 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.161+0000 7f022bfff640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022c15cf80 0x7f022c15ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.161+0000 7f022bfff640 1 --2- 172.21.15.27:0/399772635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f022c0b85e0 0x7f022c1428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.161+0000 7f022bfff640 1 -- 172.21.15.27:0/399772635 >> 172.21.15.27:0/399772635 conn(0x7f022c072e20 msgr2=0x7f022c0b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:26.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.162+0000 7f022bfff640 1 -- 172.21.15.27:0/399772635 shutdown_connections 2023-12-19T18:18:26.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.162+0000 7f022bfff640 1 -- 172.21.15.27:0/399772635 wait complete. 2023-12-19T18:18:26.190 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:26.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.383+0000 7f2a23a7a640 1 Processor -- start 2023-12-19T18:18:26.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.383+0000 7f2a23a7a640 1 -- start start 2023-12-19T18:18:26.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.385+0000 7f2a23a7a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2a1c15d270 0x7f2a1c0cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.385+0000 7f2a23a7a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2a1c0cce20 0x7f2a1c0cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.385+0000 7f2a23a7a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a1c06b4e0 con 0x7f2a1c15ce90 2023-12-19T18:18:26.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.385+0000 7f2a23a7a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a1c06ce90 con 0x7f2a1c15d270 2023-12-19T18:18:26.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.385+0000 7f2a23a7a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a1c0704f0 con 0x7f2a1c0cce20 2023-12-19T18:18:26.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.385+0000 7f2a21816640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2a1c15ce90 0x7f2a1c0cc710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34654/0 (socket says 172.21.15.27:34654) 2023-12-19T18:18:26.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.385+0000 7f2a21816640 1 -- 172.21.15.27:0/3456512934 learned_addr learned my addr 172.21.15.27:0/3456512934 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:26.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.385+0000 7f2a21015640 1 --2- 172.21.15.27:0/3456512934 >> v2:172.21.15.27:3300/0 conn(0x7f2a1c15d270 0x7f2a1c0cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.385+0000 7f2a22017640 1 --2- 172.21.15.27:0/3456512934 >> v2:172.21.15.192:3300/0 conn(0x7f2a1c0cce20 0x7f2a1c0cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.385+0000 7f2a21015640 1 -- 172.21.15.27:0/3456512934 >> v2:172.21.15.192:3300/0 conn(0x7f2a1c0cce20 msgr2=0x7f2a1c0cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.385+0000 7f2a21015640 1 --2- 172.21.15.27:0/3456512934 >> v2:172.21.15.192:3300/0 conn(0x7f2a1c0cce20 0x7f2a1c0cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.385+0000 7f2a21015640 1 -- 172.21.15.27:0/3456512934 >> v1:172.21.15.27:6789/0 conn(0x7f2a1c15ce90 legacy=0x7f2a1c0cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.385+0000 7f2a21015640 1 -- 172.21.15.27:0/3456512934 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a1c06b7c0 con 0x7f2a1c15d270 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.386+0000 7f2a21015640 1 --2- 172.21.15.27:0/3456512934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a1c15d270 0x7f2a1c0cb2b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0c2380 tx=0x7f2a0c02b710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0f270df95dab7fb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.386+0000 7f2a20814640 1 -- 172.21.15.27:0/3456512934 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c030710 con 0x7f2a1c15d270 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.386+0000 7f2a20814640 1 -- 172.21.15.27:0/3456512934 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0c02e050 con 0x7f2a1c15d270 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.387+0000 7f2a20814640 1 -- 172.21.15.27:0/3456512934 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c030a90 con 0x7f2a1c15d270 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.387+0000 7f2a23a7a640 1 -- 172.21.15.27:0/3456512934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a1c15d270 msgr2=0x7f2a1c0cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.387+0000 7f2a23a7a640 1 --2- 172.21.15.27:0/3456512934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a1c15d270 0x7f2a1c0cb2b0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c0c2380 tx=0x7f2a0c02b710 comp rx=0 tx=0).stop 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.387+0000 7f2a23a7a640 1 -- 172.21.15.27:0/3456512934 shutdown_connections 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.387+0000 7f2a23a7a640 1 --2- 172.21.15.27:0/3456512934 >> v2:172.21.15.192:3300/0 conn(0x7f2a1c0cce20 0x7f2a1c0cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.387+0000 7f2a23a7a640 1 --2- 172.21.15.27:0/3456512934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a1c15d270 0x7f2a1c0cb2b0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.387+0000 7f2a23a7a640 1 -- 172.21.15.27:0/3456512934 >> 172.21.15.27:0/3456512934 conn(0x7f2a1c06a330 msgr2=0x7f2a1c0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.387+0000 7f2a23a7a640 1 -- 172.21.15.27:0/3456512934 shutdown_connections 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.387+0000 7f2a23a7a640 1 -- 172.21.15.27:0/3456512934 wait complete. 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.389+0000 7f2a23a7a640 1 Processor -- start 2023-12-19T18:18:26.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.389+0000 7f2a23a7a640 1 -- start start 2023-12-19T18:18:26.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.390+0000 7f2a23a7a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a1c0cce20 0x7f2a1c0c2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.390+0000 7f2a23a7a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a1c15ce90 0x7f2a1c157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.390+0000 7f2a23a7a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a1c157de0 0x7f2a1c159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.390+0000 7f2a23a7a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c0c37d0 con 0x7f2a1c15ce90 2023-12-19T18:18:26.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.390+0000 7f2a23a7a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c0c2c50 con 0x7f2a1c157de0 2023-12-19T18:18:26.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.390+0000 7f2a23a7a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1c0c2f30 con 0x7f2a1c0cce20 2023-12-19T18:18:26.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.391+0000 7f2a21015640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a1c15ce90 0x7f2a1c157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.391+0000 7f2a21015640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a1c15ce90 0x7f2a1c157520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34098/0 (socket says 172.21.15.27:34098) 2023-12-19T18:18:26.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.391+0000 7f2a21015640 1 -- 172.21.15.27:0/1835478062 learned_addr learned my addr 172.21.15.27:0/1835478062 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.391+0000 7f2a21816640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a1c0cce20 0x7f2a1c0c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.391+0000 7f2a21015640 1 -- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a1c0cce20 msgr2=0x7f2a1c0c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.391+0000 7f2a22017640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a1c157de0 0x7f2a1c159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.391+0000 7f2a21015640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a1c0cce20 0x7f2a1c0c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.391+0000 7f2a21015640 1 -- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a1c157de0 msgr2=0x7f2a1c159cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.391+0000 7f2a21015640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a1c157de0 0x7f2a1c159cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.391+0000 7f2a21015640 1 -- 172.21.15.27:0/1835478062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c009400 con 0x7f2a1c15ce90 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.391+0000 7f2a22017640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a1c157de0 0x7f2a1c159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.392+0000 7f2a21015640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a1c15ce90 0x7f2a1c157520 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c000c00 tx=0x7f2a0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.392+0000 7f2a127fc640 1 -- 172.21.15.27:0/1835478062 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c03b8d0 con 0x7f2a1c15ce90 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.392+0000 7f2a127fc640 1 -- 172.21.15.27:0/1835478062 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0c03ba50 con 0x7f2a1c15ce90 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.392+0000 7f2a23a7a640 1 -- 172.21.15.27:0/1835478062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a1c0c3210 con 0x7f2a1c15ce90 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.392+0000 7f2a127fc640 1 -- 172.21.15.27:0/1835478062 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c03be00 con 0x7f2a1c15ce90 2023-12-19T18:18:26.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.392+0000 7f2a23a7a640 1 -- 172.21.15.27:0/1835478062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a1c160390 con 0x7f2a1c15ce90 2023-12-19T18:18:26.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.394+0000 7f2a23a7a640 1 -- 172.21.15.27:0/1835478062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a1c068e90 con 0x7f2a1c15ce90 2023-12-19T18:18:26.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.395+0000 7f2a127fc640 1 -- 172.21.15.27:0/1835478062 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2a0c039040 con 0x7f2a1c15ce90 2023-12-19T18:18:26.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.396+0000 7f2a127fc640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2a0006a460 0x7f2a0006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.396+0000 7f2a127fc640 1 -- 172.21.15.27:0/1835478062 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2a0c0aa1a0 con 0x7f2a1c15ce90 2023-12-19T18:18:26.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.397+0000 7f2a21816640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2a0006a460 0x7f2a0006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.398+0000 7f2a21816640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2a0006a460 0x7f2a0006c920 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2a18000f60 tx=0x7f2a18009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:26.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.401+0000 7f2a127fc640 1 -- 172.21.15.27:0/1835478062 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2a0c0758b0 con 0x7f2a1c15ce90 2023-12-19T18:18:26.591 INFO:tasks.ceph.mds.c.smithi027.stderr:2023-12-19T18:18:26.591+0000 7fe717f61640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:18:26.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.597+0000 7f2a23a7a640 1 -- 172.21.15.27:0/1835478062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a1c15c640 con 0x7f2a1c15ce90 2023-12-19T18:18:26.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.599+0000 7f2a127fc640 1 -- 172.21.15.27:0/1835478062 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16110 (secure 0 0 0) 0x7f2a0c078ed0 con 0x7f2a1c15ce90 2023-12-19T18:18:26.599 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:26.599 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":74,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:18:26.146510+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:24.350728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:25.147723+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.601+0000 7f2a23a7a640 1 -- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2a0006a460 msgr2=0x7f2a0006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.601+0000 7f2a23a7a640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2a0006a460 0x7f2a0006c920 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f2a18000f60 tx=0x7f2a18009040 comp rx=0 tx=0).stop 2023-12-19T18:18:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.601+0000 7f2a23a7a640 1 -- 172.21.15.27:0/1835478062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a1c15ce90 msgr2=0x7f2a1c157520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.601+0000 7f2a23a7a640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a1c15ce90 0x7f2a1c157520 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c000c00 tx=0x7f2a0c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.602+0000 7f2a23a7a640 1 -- 172.21.15.27:0/1835478062 shutdown_connections 2023-12-19T18:18:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.602+0000 7f2a23a7a640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2a0006a460 0x7f2a0006c920 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.602+0000 7f2a23a7a640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2a1c157de0 0x7f2a1c159cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.602+0000 7f2a23a7a640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a1c15ce90 0x7f2a1c157520 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.602+0000 7f2a23a7a640 1 --2- 172.21.15.27:0/1835478062 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2a1c0cce20 0x7f2a1c0c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.602+0000 7f2a23a7a640 1 -- 172.21.15.27:0/1835478062 >> 172.21.15.27:0/1835478062 conn(0x7f2a1c06a330 msgr2=0x7f2a1c0cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:26.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.602+0000 7f2a23a7a640 1 -- 172.21.15.27:0/1835478062 shutdown_connections 2023-12-19T18:18:26.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.602+0000 7f2a23a7a640 1 -- 172.21.15.27:0/1835478062 wait complete. 2023-12-19T18:18:26.622 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:26.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.855+0000 7f12b7fff640 1 Processor -- start 2023-12-19T18:18:26.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.856+0000 7f12b7fff640 1 -- start start 2023-12-19T18:18:26.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.857+0000 7f12b7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b815ceb0 0x7f12b80b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.857+0000 7f12b7fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f12b80b4d10 0x7f12b80d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.857+0000 7f12b7fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12b80b5170 con 0x7f12b80d1f60 2023-12-19T18:18:26.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.857+0000 7f12b7fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12b80d23f0 con 0x7f12b80b4d10 2023-12-19T18:18:26.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.857+0000 7f12b7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b80b6540 con 0x7f12b815ceb0 2023-12-19T18:18:26.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.857+0000 7f12affff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f12b80b4d10 0x7f12b80d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.857+0000 7f12b6ffd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b815ceb0 0x7f12b80b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.857+0000 7f12affff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f12b80b4d10 0x7f12b80d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:41112/0 (socket says 172.21.15.27:41112) 2023-12-19T18:18:26.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.857+0000 7f12affff640 1 -- 172.21.15.27:0/2218475029 learned_addr learned my addr 172.21.15.27:0/2218475029 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:26.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.858+0000 7f12b6ffd640 1 -- 172.21.15.27:0/2218475029 >> v2:172.21.15.192:3300/0 conn(0x7f12b80b4d10 msgr2=0x7f12b80d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:26.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.858+0000 7f12b6ffd640 1 --2- 172.21.15.27:0/2218475029 >> v2:172.21.15.192:3300/0 conn(0x7f12b80b4d10 0x7f12b80d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.858+0000 7f12b6ffd640 1 -- 172.21.15.27:0/2218475029 >> v1:172.21.15.192:6789/0 conn(0x7f12b80d1f60 legacy=0x7f12b806bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.858+0000 7f12b6ffd640 1 -- 172.21.15.27:0/2218475029 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b80cdfb0 con 0x7f12b815ceb0 2023-12-19T18:18:26.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.858+0000 7f12affff640 1 --2- 172.21.15.27:0/2218475029 >> v2:172.21.15.192:3300/0 conn(0x7f12b80b4d10 0x7f12b80d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:18:26.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.859+0000 7f12b6ffd640 1 --2- 172.21.15.27:0/2218475029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b815ceb0 0x7f12b80b47d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f12a80097a0 tx=0x7f12a802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6b929ad31dcef56 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:26.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.860+0000 7f12b67fc640 1 -- 172.21.15.27:0/2218475029 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a802d040 con 0x7f12b815ceb0 2023-12-19T18:18:26.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.860+0000 7f12b67fc640 1 -- 172.21.15.27:0/2218475029 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a8031810 con 0x7f12b815ceb0 2023-12-19T18:18:26.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.860+0000 7f12b67fc640 1 -- 172.21.15.27:0/2218475029 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8031ba0 con 0x7f12b815ceb0 2023-12-19T18:18:26.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.861+0000 7f12b7fff640 1 -- 172.21.15.27:0/2218475029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b815ceb0 msgr2=0x7f12b80b47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.861+0000 7f12b7fff640 1 --2- 172.21.15.27:0/2218475029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b815ceb0 0x7f12b80b47d0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f12a80097a0 tx=0x7f12a802b8b0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.861+0000 7f12b7fff640 1 -- 172.21.15.27:0/2218475029 shutdown_connections 2023-12-19T18:18:26.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.861+0000 7f12b7fff640 1 --2- 172.21.15.27:0/2218475029 >> v2:172.21.15.192:3300/0 conn(0x7f12b80b4d10 0x7f12b80d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.861+0000 7f12b7fff640 1 --2- 172.21.15.27:0/2218475029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b815ceb0 0x7f12b80b47d0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.861+0000 7f12b7fff640 1 -- 172.21.15.27:0/2218475029 >> 172.21.15.27:0/2218475029 conn(0x7f12b80cf7d0 msgr2=0x7f12b80cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:26.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.861+0000 7f12b7fff640 1 -- 172.21.15.27:0/2218475029 shutdown_connections 2023-12-19T18:18:26.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.861+0000 7f12b7fff640 1 -- 172.21.15.27:0/2218475029 wait complete. 2023-12-19T18:18:26.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.863+0000 7f12b7fff640 1 Processor -- start 2023-12-19T18:18:26.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.863+0000 7f12b7fff640 1 -- start start 2023-12-19T18:18:26.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.864+0000 7f12b7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b80b4d10 0x7f12b814d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.864+0000 7f12b7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12b80d1f60 0x7f12b80cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.864+0000 7f12b7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b815ceb0 0x7f12b8151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.864+0000 7f12b7fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8156a30 con 0x7f12b80b4d10 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.864+0000 7f12b7fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8156ec0 con 0x7f12b80d1f60 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.864+0000 7f12b7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12b8153160 con 0x7f12b815ceb0 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.864+0000 7f12b6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b80b4d10 0x7f12b814d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.864+0000 7f12b6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b80b4d10 0x7f12b814d990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34110/0 (socket says 172.21.15.27:34110) 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.864+0000 7f12b6ffd640 1 -- 172.21.15.27:0/3870548755 learned_addr learned my addr 172.21.15.27:0/3870548755 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.864+0000 7f12affff640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12b80d1f60 0x7f12b80cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.864+0000 7f12b6ffd640 1 -- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b815ceb0 msgr2=0x7f12b8151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.864+0000 7f12b77fe640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b815ceb0 0x7f12b8151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.865+0000 7f12b6ffd640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b815ceb0 0x7f12b8151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.865+0000 7f12b6ffd640 1 -- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12b80d1f60 msgr2=0x7f12b80cc8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.865+0000 7f12b6ffd640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12b80d1f60 0x7f12b80cc8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.865+0000 7f12b6ffd640 1 -- 172.21.15.27:0/3870548755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a00094d0 con 0x7f12b80b4d10 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.865+0000 7f12affff640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12b80d1f60 0x7f12b80cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:26.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.865+0000 7f12b77fe640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b815ceb0 0x7f12b8151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:26.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.865+0000 7f12b6ffd640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b80b4d10 0x7f12b814d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f12a802b8f0 tx=0x7f12a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:26.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.865+0000 7f12af7fe640 1 -- 172.21.15.27:0/3870548755 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8006e10 con 0x7f12b80b4d10 2023-12-19T18:18:26.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.865+0000 7f12af7fe640 1 -- 172.21.15.27:0/3870548755 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12a803ad60 con 0x7f12b80b4d10 2023-12-19T18:18:26.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.865+0000 7f12b7fff640 1 -- 172.21.15.27:0/3870548755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12a8009400 con 0x7f12b80b4d10 2023-12-19T18:18:26.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.865+0000 7f12af7fe640 1 -- 172.21.15.27:0/3870548755 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a8044630 con 0x7f12b80b4d10 2023-12-19T18:18:26.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.865+0000 7f12b7fff640 1 -- 172.21.15.27:0/3870548755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b81563e0 con 0x7f12b80b4d10 2023-12-19T18:18:26.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.867+0000 7f12b7fff640 1 -- 172.21.15.27:0/3870548755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12b804e840 con 0x7f12b80b4d10 2023-12-19T18:18:26.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.867+0000 7f12af7fe640 1 -- 172.21.15.27:0/3870548755 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f12a804d450 con 0x7f12b80b4d10 2023-12-19T18:18:26.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.868+0000 7f12af7fe640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f129806a430 0x7f129806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:26.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.868+0000 7f12af7fe640 1 -- 172.21.15.27:0/3870548755 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f12a80aab70 con 0x7f12b80b4d10 2023-12-19T18:18:26.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.869+0000 7f12affff640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f129806a430 0x7f129806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:26.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.870+0000 7f12affff640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f129806a430 0x7f129806c8f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f12b80cf490 tx=0x7f12a0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:26.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:26.873+0000 7f12af7fe640 1 -- 172.21.15.27:0/3870548755 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f12a8076150 con 0x7f12b80b4d10 2023-12-19T18:18:27.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.061+0000 7f12b7fff640 1 -- 172.21.15.27:0/3870548755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12b8159f90 con 0x7f12b80b4d10 2023-12-19T18:18:27.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.064+0000 7f12af7fe640 1 -- 172.21.15.27:0/3870548755 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) v1 ==== 74+0+16110 (secure 0 0 0) 0x7f12a8035020 con 0x7f12b80b4d10 2023-12-19T18:18:27.063 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:27.063 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":74,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:18:26.146510+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:24.350728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:25.147723+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.066+0000 7f12b7fff640 1 -- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f129806a430 msgr2=0x7f129806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.066+0000 7f12b7fff640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f129806a430 0x7f129806c8f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f12b80cf490 tx=0x7f12a0008040 comp rx=0 tx=0).stop 2023-12-19T18:18:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.066+0000 7f12b7fff640 1 -- 172.21.15.27:0/3870548755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b80b4d10 msgr2=0x7f12b814d990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.066+0000 7f12b7fff640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b80b4d10 0x7f12b814d990 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f12a802b8f0 tx=0x7f12a80046c0 comp rx=0 tx=0).stop 2023-12-19T18:18:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.067+0000 7f12b7fff640 1 -- 172.21.15.27:0/3870548755 shutdown_connections 2023-12-19T18:18:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.067+0000 7f12b7fff640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f129806a430 0x7f129806c8f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.067+0000 7f12b7fff640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12b815ceb0 0x7f12b8151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.067+0000 7f12b7fff640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12b80d1f60 0x7f12b80cc8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.067+0000 7f12b7fff640 1 --2- 172.21.15.27:0/3870548755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12b80b4d10 0x7f12b814d990 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.067+0000 7f12b7fff640 1 -- 172.21.15.27:0/3870548755 >> 172.21.15.27:0/3870548755 conn(0x7f12b80cf7d0 msgr2=0x7f12b80d2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.067+0000 7f12b7fff640 1 -- 172.21.15.27:0/3870548755 shutdown_connections 2023-12-19T18:18:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.067+0000 7f12b7fff640 1 -- 172.21.15.27:0/3870548755 wait complete. 2023-12-19T18:18:27.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:18:27.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.315+0000 7fb9d22dc640 1 Processor -- start 2023-12-19T18:18:27.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.316+0000 7fb9d22dc640 1 -- start start 2023-12-19T18:18:27.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.317+0000 7fb9d22dc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb9cc0cdb30 0x7fb9cc0cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:27.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.317+0000 7fb9d22dc640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9cc161250 con 0x7fb9cc15cfc0 2023-12-19T18:18:27.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.317+0000 7fb9d22dc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9cc161a50 con 0x7fb9cc0d17a0 2023-12-19T18:18:27.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.317+0000 7fb9d22dc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9cc0ce500 con 0x7fb9cc0cdb30 2023-12-19T18:18:27.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.318+0000 7fb9d0ad9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb9cc0cdb30 0x7fb9cc0cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:27.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.318+0000 7fb9d0ad9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb9cc0cdb30 0x7fb9cc0cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34112/0 (socket says 172.21.15.27:34112) 2023-12-19T18:18:27.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.318+0000 7fb9d12da640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb9cc15cfc0 0x7fb9cc160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34666/0 (socket says 172.21.15.27:34666) 2023-12-19T18:18:27.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.318+0000 7fb9d0ad9640 1 -- 172.21.15.27:0/2003479044 learned_addr learned my addr 172.21.15.27:0/2003479044 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:27.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.318+0000 7fb9d0ad9640 1 -- 172.21.15.27:0/2003479044 >> v1:172.21.15.192:6789/0 conn(0x7fb9cc0d17a0 legacy=0x7fb9cc0d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:27.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.318+0000 7fb9d0ad9640 1 -- 172.21.15.27:0/2003479044 >> v1:172.21.15.27:6789/0 conn(0x7fb9cc15cfc0 legacy=0x7fb9cc160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:27.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.318+0000 7fb9d0ad9640 1 -- 172.21.15.27:0/2003479044 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9cc161490 con 0x7fb9cc0cdb30 2023-12-19T18:18:27.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.319+0000 7fb9d0ad9640 1 --2- 172.21.15.27:0/2003479044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9cc0cdb30 0x7fb9cc0cdfc0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009870 tx=0x7fb9c002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79b14d7de7e9a57e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:27.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.320+0000 7fb9cbfff640 1 -- 172.21.15.27:0/2003479044 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0031870 con 0x7fb9cc0cdb30 2023-12-19T18:18:27.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.320+0000 7fb9cbfff640 1 -- 172.21.15.27:0/2003479044 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c00319d0 con 0x7fb9cc0cdb30 2023-12-19T18:18:27.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.320+0000 7fb9cbfff640 1 -- 172.21.15.27:0/2003479044 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0031d60 con 0x7fb9cc0cdb30 2023-12-19T18:18:27.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.320+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2003479044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9cc0cdb30 msgr2=0x7fb9cc0cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:27.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.320+0000 7fb9d22dc640 1 --2- 172.21.15.27:0/2003479044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9cc0cdb30 0x7fb9cc0cdfc0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009870 tx=0x7fb9c002b910 comp rx=0 tx=0).stop 2023-12-19T18:18:27.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.320+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2003479044 shutdown_connections 2023-12-19T18:18:27.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.320+0000 7fb9d22dc640 1 --2- 172.21.15.27:0/2003479044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9cc0cdb30 0x7fb9cc0cdfc0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:27.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.320+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2003479044 >> 172.21.15.27:0/2003479044 conn(0x7fb9cc072e20 msgr2=0x7fb9cc073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:27.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.321+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2003479044 shutdown_connections 2023-12-19T18:18:27.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.321+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2003479044 wait complete. 2023-12-19T18:18:27.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.322+0000 7fb9d22dc640 1 Processor -- start 2023-12-19T18:18:27.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.322+0000 7fb9d22dc640 1 -- start start 2023-12-19T18:18:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.323+0000 7fb9d22dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9cc0cdb30 0x7fb9cc0cc860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.323+0000 7fb9d22dc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9cc0d17a0 0x7fb9cc0b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.323+0000 7fb9d22dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9cc15cfc0 0x7fb9cc0b6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.323+0000 7fb9d22dc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc0ccda0 con 0x7fb9cc0d17a0 2023-12-19T18:18:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.323+0000 7fb9d22dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc141b10 con 0x7fb9cc0cdb30 2023-12-19T18:18:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.323+0000 7fb9d22dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9cc141df0 con 0x7fb9cc15cfc0 2023-12-19T18:18:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.324+0000 7fb9d0ad9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9cc0d17a0 0x7fb9cc0b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.324+0000 7fb9d0ad9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9cc0d17a0 0x7fb9cc0b5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34128/0 (socket says 172.21.15.27:34128) 2023-12-19T18:18:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.324+0000 7fb9d0ad9640 1 -- 172.21.15.27:0/2664938322 learned_addr learned my addr 172.21.15.27:0/2664938322 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:27.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.324+0000 7fb9d12da640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9cc0cdb30 0x7fb9cc0cc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.324+0000 7fb9d0ad9640 1 -- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9cc15cfc0 msgr2=0x7fb9cc0b6400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.324+0000 7fb9d0ad9640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9cc15cfc0 0x7fb9cc0b6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.324+0000 7fb9d0ad9640 1 -- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9cc0cdb30 msgr2=0x7fb9cc0cc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.324+0000 7fb9d0ad9640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9cc0cdb30 0x7fb9cc0cc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.324+0000 7fb9d0ad9640 1 -- 172.21.15.27:0/2664938322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c00094d0 con 0x7fb9cc0d17a0 2023-12-19T18:18:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.324+0000 7fb9d12da640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9cc0cdb30 0x7fb9cc0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.325+0000 7fb9d0ad9640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9cc0d17a0 0x7fb9cc0b5ec0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009870 tx=0x7fb9c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.325+0000 7fb9c9ffb640 1 -- 172.21.15.27:0/2664938322 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0031820 con 0x7fb9cc0d17a0 2023-12-19T18:18:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.325+0000 7fb9c9ffb640 1 -- 172.21.15.27:0/2664938322 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c00443f0 con 0x7fb9cc0d17a0 2023-12-19T18:18:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.325+0000 7fb9c9ffb640 1 -- 172.21.15.27:0/2664938322 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c00447a0 con 0x7fb9cc0d17a0 2023-12-19T18:18:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.325+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2664938322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9cc1420d0 con 0x7fb9cc0d17a0 2023-12-19T18:18:27.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.325+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2664938322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9cc142690 con 0x7fb9cc0d17a0 2023-12-19T18:18:27.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.326+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2664938322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9cc142880 con 0x7fb9cc0d17a0 2023-12-19T18:18:27.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.327+0000 7fb9c9ffb640 1 -- 172.21.15.27:0/2664938322 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb9c004d450 con 0x7fb9cc0d17a0 2023-12-19T18:18:27.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.328+0000 7fb9c9ffb640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb9b806a430 0x7fb9b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:27.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.328+0000 7fb9c9ffb640 1 -- 172.21.15.27:0/2664938322 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9c002d040 con 0x7fb9cc0d17a0 2023-12-19T18:18:27.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.329+0000 7fb9d12da640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb9b806a430 0x7fb9b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:27.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.330+0000 7fb9d12da640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb9b806a430 0x7fb9b806c8f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc007fd0 tx=0x7fb9bc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:27.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.333+0000 7fb9c9ffb640 1 -- 172.21.15.27:0/2664938322 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb9c00766c0 con 0x7fb9cc0d17a0 2023-12-19T18:18:27.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:27.564+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2664938322 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb9cc04e840 con 0x7fb9cc0d17a0 2023-12-19T18:18:28.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.161+0000 7fb9c9ffb640 1 -- 172.21.15.27:0/2664938322 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v29) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb9c00350b0 con 0x7fb9cc0d17a0 2023-12-19T18:18:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.164+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb9b806a430 msgr2=0x7fb9b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.164+0000 7fb9d22dc640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb9b806a430 0x7fb9b806c8f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fb9bc007fd0 tx=0x7fb9bc009040 comp rx=0 tx=0).stop 2023-12-19T18:18:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.164+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2664938322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9cc0d17a0 msgr2=0x7fb9cc0b5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.164+0000 7fb9d22dc640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9cc0d17a0 0x7fb9cc0b5ec0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009870 tx=0x7fb9c00046c0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.164+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2664938322 shutdown_connections 2023-12-19T18:18:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.164+0000 7fb9d22dc640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb9b806a430 0x7fb9b806c8f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.164+0000 7fb9d22dc640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9cc15cfc0 0x7fb9cc0b6400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.164+0000 7fb9d22dc640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb9cc0d17a0 0x7fb9cc0b5ec0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.164+0000 7fb9d22dc640 1 --2- 172.21.15.27:0/2664938322 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9cc0cdb30 0x7fb9cc0cc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.164+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2664938322 >> 172.21.15.27:0/2664938322 conn(0x7fb9cc072e20 msgr2=0x7fb9cc0b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:28.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.165+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2664938322 shutdown_connections 2023-12-19T18:18:28.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.165+0000 7fb9d22dc640 1 -- 172.21.15.27:0/2664938322 wait complete. 2023-12-19T18:18:28.186 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:18:28.187 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:18:28.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.421+0000 7f4925363640 1 Processor -- start 2023-12-19T18:18:28.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.421+0000 7f4925363640 1 -- start start 2023-12-19T18:18:28.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.423+0000 7f4925363640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f492015b1a0 0x7f492006f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.423+0000 7f4925363640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492006bc80 con 0x7f4920071420 2023-12-19T18:18:28.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.423+0000 7f4925363640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492006d690 con 0x7f49200c2720 2023-12-19T18:18:28.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.423+0000 7f4925363640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492006d0a0 con 0x7f492015b1a0 2023-12-19T18:18:28.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.423+0000 7f491f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4920071420 0x7f492015ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34678/0 (socket says 172.21.15.27:34678) 2023-12-19T18:18:28.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.423+0000 7f491f7fe640 1 -- 172.21.15.27:0/939007978 learned_addr learned my addr 172.21.15.27:0/939007978 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:28.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491effd640 1 --2- 172.21.15.27:0/939007978 >> v2:172.21.15.192:3300/0 conn(0x7f492015b1a0 0x7f492006f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880791332 0 0) 0x7f4918001490 con 0x7f4920071420 2023-12-19T18:18:28.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f490c003520 con 0x7f4920071420 2023-12-19T18:18:28.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 913394467 0 0) 0x7f490c003520 con 0x7f4920071420 2023-12-19T18:18:28.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 >> v2:172.21.15.192:3300/0 conn(0x7f492015b1a0 msgr2=0x7f492006f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:28.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491e7fc640 1 --2- 172.21.15.27:0/939007978 >> v2:172.21.15.192:3300/0 conn(0x7f492015b1a0 0x7f492006f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 >> v1:172.21.15.192:6789/0 conn(0x7f49200c2720 legacy=0x7f492006fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:28.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492006b6c0 con 0x7f4920071420 2023-12-19T18:18:28.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f4918001b80 con 0x7f4920071420 2023-12-19T18:18:28.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 >> v1:172.21.15.27:6789/0 conn(0x7f4920071420 legacy=0x7f492015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491e7fc640 1 --2- 172.21.15.27:0/939007978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f490c0039c0 0x7f490c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491e7fc640 1 --2- 172.21.15.27:0/939007978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f490c0047c0 0x7f490c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491e7fc640 1 --2- 172.21.15.27:0/939007978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f490c0052a0 0x7f490c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.424+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492006d0a0 con 0x7f490c0039c0 2023-12-19T18:18:28.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.425+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c00c1c0 con 0x7f490c0052a0 2023-12-19T18:18:28.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.425+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c00c380 con 0x7f490c0047c0 2023-12-19T18:18:28.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.425+0000 7f491effd640 1 --2- 172.21.15.27:0/939007978 >> v2:172.21.15.192:3300/0 conn(0x7f492015b1a0 0x7f492006f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:18:28.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.425+0000 7f491effd640 1 --2- 172.21.15.27:0/939007978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f490c0039c0 0x7f490c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.425+0000 7f491f7fe640 1 --2- 172.21.15.27:0/939007978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f490c0052a0 0x7f490c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.425+0000 7f4917fff640 1 --2- 172.21.15.27:0/939007978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f490c0047c0 0x7f490c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.425+0000 7f491effd640 1 -- 172.21.15.27:0/939007978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f490c0047c0 msgr2=0x7f490c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:28.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.425+0000 7f491effd640 1 --2- 172.21.15.27:0/939007978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f490c0047c0 0x7f490c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.425+0000 7f491effd640 1 -- 172.21.15.27:0/939007978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f490c0052a0 msgr2=0x7f490c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:28.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.425+0000 7f491effd640 1 --2- 172.21.15.27:0/939007978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f490c0052a0 0x7f490c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.425+0000 7f491effd640 1 -- 172.21.15.27:0/939007978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c003520 con 0x7f490c0039c0 2023-12-19T18:18:28.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.425+0000 7f491f7fe640 1 -- 172.21.15.27:0/939007978 reap_dead start 2023-12-19T18:18:28.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.426+0000 7f491effd640 1 --2- 172.21.15.27:0/939007978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f490c0039c0 0x7f490c003de0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f49100098a0 tx=0x7f490c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d374f2ebf825035 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:28.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.427+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910030630 con 0x7f490c0039c0 2023-12-19T18:18:28.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.427+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4910030790 con 0x7f490c0039c0 2023-12-19T18:18:28.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.427+0000 7f491e7fc640 1 -- 172.21.15.27:0/939007978 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910030b00 con 0x7f490c0039c0 2023-12-19T18:18:28.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.427+0000 7f4925363640 1 -- 172.21.15.27:0/939007978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f490c0039c0 msgr2=0x7f490c003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.427+0000 7f4925363640 1 --2- 172.21.15.27:0/939007978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f490c0039c0 0x7f490c003de0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f49100098a0 tx=0x7f490c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.427+0000 7f4925363640 1 -- 172.21.15.27:0/939007978 shutdown_connections 2023-12-19T18:18:28.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.427+0000 7f4925363640 1 --2- 172.21.15.27:0/939007978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f490c0039c0 0x7f490c003de0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.427+0000 7f4925363640 1 -- 172.21.15.27:0/939007978 >> 172.21.15.27:0/939007978 conn(0x7f49200cebd0 msgr2=0x7f49200cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:28.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.427+0000 7f4925363640 1 -- 172.21.15.27:0/939007978 shutdown_connections 2023-12-19T18:18:28.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.427+0000 7f4925363640 1 -- 172.21.15.27:0/939007978 wait complete. 2023-12-19T18:18:28.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.429+0000 7f4925363640 1 Processor -- start 2023-12-19T18:18:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.430+0000 7f4925363640 1 -- start start 2023-12-19T18:18:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.430+0000 7f4925363640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f490c0039c0 0x7f49201e27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.431+0000 7f4925363640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49201e1fa0 0x7f49200c1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.431+0000 7f4925363640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49201e3f20 0x7f49200c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.431+0000 7f4925363640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49201e3a90 con 0x7f490c0039c0 2023-12-19T18:18:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.431+0000 7f4925363640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49200c1ac0 con 0x7f49201e3f20 2023-12-19T18:18:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.431+0000 7f4925363640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49201e2e80 con 0x7f49201e1fa0 2023-12-19T18:18:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.431+0000 7f491effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f490c0039c0 0x7f49201e27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.431+0000 7f491effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f490c0039c0 0x7f49201e27f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34144/0 (socket says 172.21.15.27:34144) 2023-12-19T18:18:28.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.431+0000 7f491effd640 1 -- 172.21.15.27:0/2061474306 learned_addr learned my addr 172.21.15.27:0/2061474306 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.431+0000 7f491effd640 1 -- 172.21.15.27:0/2061474306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49201e1fa0 msgr2=0x7f49200c1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.431+0000 7f491effd640 1 --2- 172.21.15.27:0/2061474306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49201e1fa0 0x7f49200c1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.431+0000 7f491effd640 1 -- 172.21.15.27:0/2061474306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49201e3f20 msgr2=0x7f49200c79a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.431+0000 7f491effd640 1 --2- 172.21.15.27:0/2061474306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49201e3f20 0x7f49200c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.431+0000 7f491effd640 1 -- 172.21.15.27:0/2061474306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49201e3160 con 0x7f490c0039c0 2023-12-19T18:18:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.432+0000 7f491effd640 1 --2- 172.21.15.27:0/2061474306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f490c0039c0 0x7f49201e27f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f49100099c0 tx=0x7f4910004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.432+0000 7f49177fe640 1 -- 172.21.15.27:0/2061474306 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910039960 con 0x7f490c0039c0 2023-12-19T18:18:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.432+0000 7f49177fe640 1 -- 172.21.15.27:0/2061474306 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f491003f040 con 0x7f490c0039c0 2023-12-19T18:18:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.432+0000 7f49177fe640 1 -- 172.21.15.27:0/2061474306 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4910039d20 con 0x7f490c0039c0 2023-12-19T18:18:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.432+0000 7f4925363640 1 -- 172.21.15.27:0/2061474306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49201e3440 con 0x7f490c0039c0 2023-12-19T18:18:28.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.432+0000 7f4925363640 1 -- 172.21.15.27:0/2061474306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49201e2400 con 0x7f490c0039c0 2023-12-19T18:18:28.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.433+0000 7f4925363640 1 -- 172.21.15.27:0/2061474306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4920068e90 con 0x7f490c0039c0 2023-12-19T18:18:28.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.434+0000 7f49177fe640 1 -- 172.21.15.27:0/2061474306 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f491004c400 con 0x7f490c0039c0 2023-12-19T18:18:28.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.435+0000 7f49177fe640 1 --2- 172.21.15.27:0/2061474306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f490006a460 0x7f490006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:28.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.435+0000 7f49177fe640 1 -- 172.21.15.27:0/2061474306 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f49100aae80 con 0x7f490c0039c0 2023-12-19T18:18:28.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.436+0000 7f4917fff640 1 --2- 172.21.15.27:0/2061474306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f490006a460 0x7f490006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:28.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.437+0000 7f4917fff640 1 --2- 172.21.15.27:0/2061474306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f490006a460 0x7f490006c920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f49200a8680 tx=0x7f4908009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:28.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.439+0000 7f49177fe640 1 -- 172.21.15.27:0/2061474306 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4910076340 con 0x7f490c0039c0 2023-12-19T18:18:28.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:28.671+0000 7f4925363640 1 -- 172.21.15.27:0/2061474306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f49200ccc40 con 0x7f490c0039c0 2023-12-19T18:18:29.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.670+0000 7f49177fe640 1 -- 172.21.15.27:0/2061474306 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4910034070 con 0x7f490c0039c0 2023-12-19T18:18:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.673+0000 7f4925363640 1 -- 172.21.15.27:0/2061474306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f490006a460 msgr2=0x7f490006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.673+0000 7f4925363640 1 --2- 172.21.15.27:0/2061474306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f490006a460 0x7f490006c920 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f49200a8680 tx=0x7f4908009040 comp rx=0 tx=0).stop 2023-12-19T18:18:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.673+0000 7f4925363640 1 -- 172.21.15.27:0/2061474306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f490c0039c0 msgr2=0x7f49201e27f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.673+0000 7f4925363640 1 --2- 172.21.15.27:0/2061474306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f490c0039c0 0x7f49201e27f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f49100099c0 tx=0x7f4910004730 comp rx=0 tx=0).stop 2023-12-19T18:18:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.674+0000 7f4925363640 1 -- 172.21.15.27:0/2061474306 shutdown_connections 2023-12-19T18:18:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.674+0000 7f4925363640 1 --2- 172.21.15.27:0/2061474306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f490006a460 0x7f490006c920 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.674+0000 7f4925363640 1 --2- 172.21.15.27:0/2061474306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49201e3f20 0x7f49200c79a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.674+0000 7f4925363640 1 --2- 172.21.15.27:0/2061474306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f49201e1fa0 0x7f49200c1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.674+0000 7f4925363640 1 --2- 172.21.15.27:0/2061474306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f490c0039c0 0x7f49201e27f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.674+0000 7f4925363640 1 -- 172.21.15.27:0/2061474306 >> 172.21.15.27:0/2061474306 conn(0x7f49200cebd0 msgr2=0x7f49200cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.674+0000 7f4925363640 1 -- 172.21.15.27:0/2061474306 shutdown_connections 2023-12-19T18:18:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.674+0000 7f4925363640 1 -- 172.21.15.27:0/2061474306 wait complete. 2023-12-19T18:18:29.696 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:18:29.696 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:29.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.929+0000 7f0ea5e70640 1 Processor -- start 2023-12-19T18:18:29.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.930+0000 7f0ea5e70640 1 -- start start 2023-12-19T18:18:29.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.931+0000 7f0ea5e70640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0ea015d270 0x7f0ea00cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:29.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.931+0000 7f0ea5e70640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea00cce20 0x7f0ea00cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:29.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.931+0000 7f0ea5e70640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ea006b4e0 con 0x7f0ea015ce90 2023-12-19T18:18:29.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.931+0000 7f0ea5e70640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ea006ce90 con 0x7f0ea015d270 2023-12-19T18:18:29.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.931+0000 7f0ea5e70640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea00704f0 con 0x7f0ea00cce20 2023-12-19T18:18:29.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.931+0000 7f0e97fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0ea015d270 0x7f0ea00cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:29.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.931+0000 7f0e9f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0ea015ce90 0x7f0ea00cc710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34690/0 (socket says 172.21.15.27:34690) 2023-12-19T18:18:29.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.931+0000 7f0e9f7fe640 1 -- 172.21.15.27:0/4233736989 learned_addr learned my addr 172.21.15.27:0/4233736989 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:29.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.932+0000 7f0e9ffff640 1 --2- 172.21.15.27:0/4233736989 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea00cce20 0x7f0ea00cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:29.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.932+0000 7f0e97fff640 1 -- 172.21.15.27:0/4233736989 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea00cce20 msgr2=0x7f0ea00cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:29.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.932+0000 7f0e97fff640 1 --2- 172.21.15.27:0/4233736989 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea00cce20 0x7f0ea00cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:29.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.932+0000 7f0e97fff640 1 -- 172.21.15.27:0/4233736989 >> v1:172.21.15.27:6789/0 conn(0x7f0ea015ce90 legacy=0x7f0ea00cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:29.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.932+0000 7f0e9effd640 1 -- 172.21.15.27:0/4233736989 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821144183 0 0) 0x7f0ea006b4e0 con 0x7f0ea015ce90 2023-12-19T18:18:29.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.932+0000 7f0e97fff640 1 -- 172.21.15.27:0/4233736989 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ea006b7c0 con 0x7f0ea015d270 2023-12-19T18:18:29.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.933+0000 7f0e97fff640 1 --2- 172.21.15.27:0/4233736989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea015d270 0x7f0ea00cb2b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0e880097a0 tx=0x7f0e8802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1c3e84922824a3a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:29.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.933+0000 7f0e9effd640 1 -- 172.21.15.27:0/4233736989 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8802d040 con 0x7f0ea015d270 2023-12-19T18:18:29.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.934+0000 7f0e9effd640 1 -- 172.21.15.27:0/4233736989 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e88031810 con 0x7f0ea015d270 2023-12-19T18:18:29.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.934+0000 7f0e9effd640 1 -- 172.21.15.27:0/4233736989 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e88031ba0 con 0x7f0ea015d270 2023-12-19T18:18:29.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.934+0000 7f0ea5e70640 1 -- 172.21.15.27:0/4233736989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea015d270 msgr2=0x7f0ea00cb2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:29.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.934+0000 7f0ea5e70640 1 --2- 172.21.15.27:0/4233736989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea015d270 0x7f0ea00cb2b0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f0e880097a0 tx=0x7f0e8802b8b0 comp rx=0 tx=0).stop 2023-12-19T18:18:29.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.934+0000 7f0ea5e70640 1 -- 172.21.15.27:0/4233736989 shutdown_connections 2023-12-19T18:18:29.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.934+0000 7f0ea5e70640 1 --2- 172.21.15.27:0/4233736989 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea00cce20 0x7f0ea00cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:29.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.934+0000 7f0ea5e70640 1 --2- 172.21.15.27:0/4233736989 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea015d270 0x7f0ea00cb2b0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:29.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.934+0000 7f0ea5e70640 1 -- 172.21.15.27:0/4233736989 >> 172.21.15.27:0/4233736989 conn(0x7f0ea006a330 msgr2=0x7f0ea00b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:29.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.934+0000 7f0ea5e70640 1 -- 172.21.15.27:0/4233736989 shutdown_connections 2023-12-19T18:18:29.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.934+0000 7f0ea5e70640 1 -- 172.21.15.27:0/4233736989 wait complete. 2023-12-19T18:18:29.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.936+0000 7f0ea5e70640 1 Processor -- start 2023-12-19T18:18:29.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.936+0000 7f0ea5e70640 1 -- start start 2023-12-19T18:18:29.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.937+0000 7f0ea5e70640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea00cce20 0x7f0ea01f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:29.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.937+0000 7f0ea5e70640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea015ce90 0x7f0ea01f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:29.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.937+0000 7f0ea5e70640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00b1d20 0x7f0ea00b21a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:29.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.937+0000 7f0ea5e70640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea00b2890 con 0x7f0ea00b1d20 2023-12-19T18:18:29.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.937+0000 7f0ea5e70640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea00b2b70 con 0x7f0ea00cce20 2023-12-19T18:18:29.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.937+0000 7f0ea5e70640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea00c3760 con 0x7f0ea015ce90 2023-12-19T18:18:29.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.938+0000 7f0e9ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00b1d20 0x7f0ea00b21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:29.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.938+0000 7f0e9ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00b1d20 0x7f0ea00b21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34164/0 (socket says 172.21.15.27:34164) 2023-12-19T18:18:29.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.938+0000 7f0e9ffff640 1 -- 172.21.15.27:0/753157526 learned_addr learned my addr 172.21.15.27:0/753157526 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:29.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.938+0000 7f0e97fff640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea015ce90 0x7f0ea01f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:29.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.938+0000 7f0e9f7fe640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea00cce20 0x7f0ea01f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:29.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.938+0000 7f0e9ffff640 1 -- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea015ce90 msgr2=0x7f0ea01f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:29.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.938+0000 7f0e9ffff640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea015ce90 0x7f0ea01f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:29.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.938+0000 7f0e9ffff640 1 -- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea00cce20 msgr2=0x7f0ea01f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:29.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.938+0000 7f0e9ffff640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea00cce20 0x7f0ea01f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:29.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.938+0000 7f0e9ffff640 1 -- 172.21.15.27:0/753157526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e88009400 con 0x7f0ea00b1d20 2023-12-19T18:18:29.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.938+0000 7f0e9f7fe640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea00cce20 0x7f0ea01f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:29.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.938+0000 7f0e97fff640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea015ce90 0x7f0ea01f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:29.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.939+0000 7f0e9ffff640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00b1d20 0x7f0ea00b21a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800cd90 tx=0x7f0e9800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:29.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.939+0000 7f0e9cff9640 1 -- 172.21.15.27:0/753157526 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e98003b90 con 0x7f0ea00b1d20 2023-12-19T18:18:29.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.939+0000 7f0e9cff9640 1 -- 172.21.15.27:0/753157526 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e98003d10 con 0x7f0ea00b1d20 2023-12-19T18:18:29.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.939+0000 7f0e9cff9640 1 -- 172.21.15.27:0/753157526 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e98004260 con 0x7f0ea00b1d20 2023-12-19T18:18:29.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.939+0000 7f0ea5e70640 1 -- 172.21.15.27:0/753157526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea00c3a40 con 0x7f0ea00b1d20 2023-12-19T18:18:29.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.939+0000 7f0ea5e70640 1 -- 172.21.15.27:0/753157526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea01f6960 con 0x7f0ea00b1d20 2023-12-19T18:18:29.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.940+0000 7f0ea5e70640 1 -- 172.21.15.27:0/753157526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea0068e90 con 0x7f0ea00b1d20 2023-12-19T18:18:29.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.941+0000 7f0e9cff9640 1 -- 172.21.15.27:0/753157526 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0e980043c0 con 0x7f0ea00b1d20 2023-12-19T18:18:29.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.942+0000 7f0e9cff9640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0e8006a430 0x7f0e8006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:29.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.942+0000 7f0e9cff9640 1 -- 172.21.15.27:0/753157526 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e98012070 con 0x7f0ea00b1d20 2023-12-19T18:18:29.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.943+0000 7f0e9f7fe640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0e8006a430 0x7f0e8006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:29.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.944+0000 7f0e9f7fe640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0e8006a430 0x7f0e8006c8f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0e90000f60 tx=0x7f0e90009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:29.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:29.946+0000 7f0e9cff9640 1 -- 172.21.15.27:0/753157526 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0e980548a0 con 0x7f0ea00b1d20 2023-12-19T18:18:30.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.176+0000 7f0ea5e70640 1 -- 172.21.15.27:0/753157526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ea00b4ac0 con 0x7f0ea00b1d20 2023-12-19T18:18:30.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.178+0000 7f0e9cff9640 1 -- 172.21.15.27:0/753157526 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f0e98057ec0 con 0x7f0ea00b1d20 2023-12-19T18:18:30.178 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:30.178 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1624358030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1624358030},{"type":"v1","addr":"172.21.15.192:6836","nonce":1624358030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5030,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3025339386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3025339386},{"type":"v1","addr":"172.21.15.192:6837","nonce":3025339386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/1685117850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1685117850},{"type":"v1","addr":"172.21.15.27:6833","nonce":1685117850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:18:26.147336+0000","modified":"2023-12-19T18:18:29.663526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2772294131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2772294131},{"type":"v1","addr":"172.21.15.27:6835","nonce":2772294131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-19T18:18:30.178 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2023-12-19T18:18:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.180+0000 7f0ea5e70640 1 -- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0e8006a430 msgr2=0x7f0e8006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:30.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.180+0000 7f0ea5e70640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0e8006a430 0x7f0e8006c8f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f0e90000f60 tx=0x7f0e90009040 comp rx=0 tx=0).stop 2023-12-19T18:18:30.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.180+0000 7f0ea5e70640 1 -- 172.21.15.27:0/753157526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00b1d20 msgr2=0x7f0ea00b21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:30.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.180+0000 7f0ea5e70640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00b1d20 0x7f0ea00b21a0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f0e9800cd90 tx=0x7f0e9800a660 comp rx=0 tx=0).stop 2023-12-19T18:18:30.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.181+0000 7f0ea5e70640 1 -- 172.21.15.27:0/753157526 shutdown_connections 2023-12-19T18:18:30.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.181+0000 7f0ea5e70640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0e8006a430 0x7f0e8006c8f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.181+0000 7f0ea5e70640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea00b1d20 0x7f0ea00b21a0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.181+0000 7f0ea5e70640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ea015ce90 0x7f0ea01f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.181+0000 7f0ea5e70640 1 --2- 172.21.15.27:0/753157526 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ea00cce20 0x7f0ea01f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.181+0000 7f0ea5e70640 1 -- 172.21.15.27:0/753157526 >> 172.21.15.27:0/753157526 conn(0x7f0ea006a330 msgr2=0x7f0ea00ae4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:30.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.181+0000 7f0ea5e70640 1 -- 172.21.15.27:0/753157526 shutdown_connections 2023-12-19T18:18:30.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.181+0000 7f0ea5e70640 1 -- 172.21.15.27:0/753157526 wait complete. 2023-12-19T18:18:30.201 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:18:30.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.423+0000 7f5d6ccc5640 1 Processor -- start 2023-12-19T18:18:30.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.424+0000 7f5d6ccc5640 1 -- start start 2023-12-19T18:18:30.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.425+0000 7f5d6ccc5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d680cb300 0x7f5d680cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:30.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.425+0000 7f5d6ccc5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5d680c8f90 0x7f5d68160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:30.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.425+0000 7f5d6ccc5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d681612d0 con 0x7f5d68160e40 2023-12-19T18:18:30.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.426+0000 7f5d6ccc5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d680c4480 con 0x7f5d680c8f90 2023-12-19T18:18:30.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.426+0000 7f5d6ccc5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d681468c0 con 0x7f5d680cb300 2023-12-19T18:18:30.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.426+0000 7f5d67fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5d68160e40 0x7f5d680ac8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34692/0 (socket says 172.21.15.27:34692) 2023-12-19T18:18:30.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.426+0000 7f5d67fff640 1 -- 172.21.15.27:0/3221095181 learned_addr learned my addr 172.21.15.27:0/3221095181 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.426+0000 7f5d677fe640 1 --2- 172.21.15.27:0/3221095181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d680cb300 0x7f5d680cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.426+0000 7f5d5ffff640 1 --2- 172.21.15.27:0/3221095181 >> v2:172.21.15.192:3300/0 conn(0x7f5d680c8f90 0x7f5d68160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.427+0000 7f5d66ffd640 1 -- 172.21.15.27:0/3221095181 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4050540326 0 0) 0x7f5d60001490 con 0x7f5d68160e40 2023-12-19T18:18:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.427+0000 7f5d66ffd640 1 -- 172.21.15.27:0/3221095181 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5d54003520 con 0x7f5d68160e40 2023-12-19T18:18:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.427+0000 7f5d677fe640 1 -- 172.21.15.27:0/3221095181 >> v2:172.21.15.192:3300/0 conn(0x7f5d680c8f90 msgr2=0x7f5d68160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:30.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.427+0000 7f5d677fe640 1 --2- 172.21.15.27:0/3221095181 >> v2:172.21.15.192:3300/0 conn(0x7f5d680c8f90 0x7f5d68160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.427+0000 7f5d677fe640 1 -- 172.21.15.27:0/3221095181 >> v1:172.21.15.27:6789/0 conn(0x7f5d68160e40 legacy=0x7f5d680ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:30.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.427+0000 7f5d66ffd640 1 -- 172.21.15.27:0/3221095181 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1433932121 0 0) 0x7f5d54003520 con 0x7f5d68160e40 2023-12-19T18:18:30.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.427+0000 7f5d677fe640 1 -- 172.21.15.27:0/3221095181 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d680cbc20 con 0x7f5d680cb300 2023-12-19T18:18:30.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.429+0000 7f5d677fe640 1 --2- 172.21.15.27:0/3221095181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d680cb300 0x7f5d680cb6e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5d580097a0 tx=0x7f5d5802b8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=529a8fcbf844504c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:30.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.429+0000 7f5d66ffd640 1 -- 172.21.15.27:0/3221095181 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d5802d040 con 0x7f5d680cb300 2023-12-19T18:18:30.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.429+0000 7f5d66ffd640 1 -- 172.21.15.27:0/3221095181 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d58031810 con 0x7f5d680cb300 2023-12-19T18:18:30.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.429+0000 7f5d66ffd640 1 -- 172.21.15.27:0/3221095181 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58031b80 con 0x7f5d680cb300 2023-12-19T18:18:30.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.430+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/3221095181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d680cb300 msgr2=0x7f5d680cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:30.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.430+0000 7f5d6ccc5640 1 --2- 172.21.15.27:0/3221095181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d680cb300 0x7f5d680cb6e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f5d580097a0 tx=0x7f5d5802b8b0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.430+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/3221095181 shutdown_connections 2023-12-19T18:18:30.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.430+0000 7f5d6ccc5640 1 --2- 172.21.15.27:0/3221095181 >> v2:172.21.15.192:3300/0 conn(0x7f5d680c8f90 0x7f5d68160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.430+0000 7f5d6ccc5640 1 --2- 172.21.15.27:0/3221095181 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d680cb300 0x7f5d680cb6e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.430+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/3221095181 >> 172.21.15.27:0/3221095181 conn(0x7f5d68069c70 msgr2=0x7f5d6806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:30.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.430+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/3221095181 shutdown_connections 2023-12-19T18:18:30.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.430+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/3221095181 wait complete. 2023-12-19T18:18:30.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.432+0000 7f5d6ccc5640 1 Processor -- start 2023-12-19T18:18:30.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.432+0000 7f5d6ccc5640 1 -- start start 2023-12-19T18:18:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.433+0000 7f5d6ccc5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d680c8f90 0x7f5d68160090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.433+0000 7f5d6ccc5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d680cb300 0x7f5d681f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.433+0000 7f5d6ccc5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d68160e40 0x7f5d681f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.433+0000 7f5d6ccc5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d681f3dc0 con 0x7f5d680c8f90 2023-12-19T18:18:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.433+0000 7f5d6ccc5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d68144c70 con 0x7f5d680cb300 2023-12-19T18:18:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.433+0000 7f5d6ccc5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d681f3130 con 0x7f5d68160e40 2023-12-19T18:18:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.433+0000 7f5d677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d680c8f90 0x7f5d68160090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:30.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.434+0000 7f5d677fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d680c8f90 0x7f5d68160090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34180/0 (socket says 172.21.15.27:34180) 2023-12-19T18:18:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.434+0000 7f5d677fe640 1 -- 172.21.15.27:0/1179319049 learned_addr learned my addr 172.21.15.27:0/1179319049 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.434+0000 7f5d677fe640 1 -- 172.21.15.27:0/1179319049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d68160e40 msgr2=0x7f5d681f2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.434+0000 7f5d677fe640 1 --2- 172.21.15.27:0/1179319049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d68160e40 0x7f5d681f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.434+0000 7f5d677fe640 1 -- 172.21.15.27:0/1179319049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d680cb300 msgr2=0x7f5d681f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.434+0000 7f5d677fe640 1 --2- 172.21.15.27:0/1179319049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d680cb300 0x7f5d681f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.434+0000 7f5d677fe640 1 -- 172.21.15.27:0/1179319049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d58009400 con 0x7f5d680c8f90 2023-12-19T18:18:30.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.434+0000 7f5d677fe640 1 --2- 172.21.15.27:0/1179319049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d680c8f90 0x7f5d68160090 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5d5802b8f0 tx=0x7f5d580046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:30.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.435+0000 7f5d64ff9640 1 -- 172.21.15.27:0/1179319049 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58003820 con 0x7f5d680c8f90 2023-12-19T18:18:30.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.435+0000 7f5d64ff9640 1 -- 172.21.15.27:0/1179319049 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d5803e070 con 0x7f5d680c8f90 2023-12-19T18:18:30.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.435+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/1179319049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d680b2ec0 con 0x7f5d680c8f90 2023-12-19T18:18:30.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.435+0000 7f5d64ff9640 1 -- 172.21.15.27:0/1179319049 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d58003ba0 con 0x7f5d680c8f90 2023-12-19T18:18:30.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.435+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/1179319049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d681f39e0 con 0x7f5d680c8f90 2023-12-19T18:18:30.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.436+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/1179319049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d681f3f00 con 0x7f5d680c8f90 2023-12-19T18:18:30.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.437+0000 7f5d64ff9640 1 -- 172.21.15.27:0/1179319049 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5d58003d00 con 0x7f5d680c8f90 2023-12-19T18:18:30.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.438+0000 7f5d64ff9640 1 --2- 172.21.15.27:0/1179319049 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5d4806a430 0x7f5d4806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:30.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.438+0000 7f5d64ff9640 1 -- 172.21.15.27:0/1179319049 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5d580aa6d0 con 0x7f5d680c8f90 2023-12-19T18:18:30.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.438+0000 7f5d5ffff640 1 --2- 172.21.15.27:0/1179319049 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5d4806a430 0x7f5d4806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:30.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.439+0000 7f5d5ffff640 1 --2- 172.21.15.27:0/1179319049 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5d4806a430 0x7f5d4806c8f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5d50006a90 tx=0x7f5d50006a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:30.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.442+0000 7f5d64ff9640 1 -- 172.21.15.27:0/1179319049 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5d681f3f00 con 0x7f5d680c8f90 2023-12-19T18:18:30.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.640+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/1179319049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d681f3f00 con 0x7f5d680c8f90 2023-12-19T18:18:30.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.643+0000 7f5d64ff9640 1 -- 172.21.15.27:0/1179319049 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+16097 (secure 0 0 0) 0x7f5d58040090 con 0x7f5d680c8f90 2023-12-19T18:18:30.643 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:30.643 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":75,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:18:27.151545+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"63","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:24.350728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"74","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:25.147723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:18:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.645+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/1179319049 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5d4806a430 msgr2=0x7f5d4806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.645+0000 7f5d6ccc5640 1 --2- 172.21.15.27:0/1179319049 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5d4806a430 0x7f5d4806c8f0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5d50006a90 tx=0x7f5d50006a20 comp rx=0 tx=0).stop 2023-12-19T18:18:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.645+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/1179319049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d680c8f90 msgr2=0x7f5d68160090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.645+0000 7f5d6ccc5640 1 --2- 172.21.15.27:0/1179319049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d680c8f90 0x7f5d68160090 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f5d5802b8f0 tx=0x7f5d580046c0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.646+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/1179319049 shutdown_connections 2023-12-19T18:18:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.646+0000 7f5d6ccc5640 1 --2- 172.21.15.27:0/1179319049 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5d4806a430 0x7f5d4806c8f0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.646+0000 7f5d6ccc5640 1 --2- 172.21.15.27:0/1179319049 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d68160e40 0x7f5d681f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.646+0000 7f5d6ccc5640 1 --2- 172.21.15.27:0/1179319049 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d680cb300 0x7f5d681f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.646+0000 7f5d6ccc5640 1 --2- 172.21.15.27:0/1179319049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d680c8f90 0x7f5d68160090 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.646+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/1179319049 >> 172.21.15.27:0/1179319049 conn(0x7f5d68069c70 msgr2=0x7f5d680afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.646+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/1179319049 shutdown_connections 2023-12-19T18:18:30.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.646+0000 7f5d6ccc5640 1 -- 172.21.15.27:0/1179319049 wait complete. 2023-12-19T18:18:30.664 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:18:30.664 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:18:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.894+0000 7ff7561a5640 1 Processor -- start 2023-12-19T18:18:30.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.894+0000 7ff7561a5640 1 -- start start 2023-12-19T18:18:30.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.896+0000 7ff7561a5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff7500b4ef0 0x7ff7500b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:30.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.896+0000 7ff7561a5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff7500d2350 0x7ff7500b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:30.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.896+0000 7ff7561a5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7500b5e90 con 0x7ff7501604d0 2023-12-19T18:18:30.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.896+0000 7ff7561a5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7500d2b30 con 0x7ff7500b4ef0 2023-12-19T18:18:30.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.896+0000 7ff7561a5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7500d2850 con 0x7ff7500d2350 2023-12-19T18:18:30.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.896+0000 7ff7549a2640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff7500b4ef0 0x7ff7500b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:30.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.896+0000 7ff7551a3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff7501604d0 0x7ff7500b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34708/0 (socket says 172.21.15.27:34708) 2023-12-19T18:18:30.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.896+0000 7ff7551a3640 1 -- 172.21.15.27:0/3916022297 learned_addr learned my addr 172.21.15.27:0/3916022297 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:30.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.896+0000 7ff7559a4640 1 --2- 172.21.15.27:0/3916022297 >> v2:172.21.15.192:3300/0 conn(0x7ff7500d2350 0x7ff7500b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:30.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.897+0000 7ff7549a2640 1 -- 172.21.15.27:0/3916022297 >> v2:172.21.15.192:3300/0 conn(0x7ff7500d2350 msgr2=0x7ff7500b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:30.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.897+0000 7ff7549a2640 1 --2- 172.21.15.27:0/3916022297 >> v2:172.21.15.192:3300/0 conn(0x7ff7500d2350 0x7ff7500b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.897+0000 7ff7549a2640 1 -- 172.21.15.27:0/3916022297 >> v1:172.21.15.27:6789/0 conn(0x7ff7501604d0 legacy=0x7ff7500b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:30.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.897+0000 7ff7549a2640 1 -- 172.21.15.27:0/3916022297 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7500cded0 con 0x7ff7500b4ef0 2023-12-19T18:18:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.898+0000 7ff7549a2640 1 --2- 172.21.15.27:0/3916022297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7500b4ef0 0x7ff7500b6300 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff750071e80 tx=0x7ff74002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69dbf3c42753777c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.898+0000 7ff747fff640 1 -- 172.21.15.27:0/3916022297 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740031830 con 0x7ff7500b4ef0 2023-12-19T18:18:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.898+0000 7ff747fff640 1 -- 172.21.15.27:0/3916022297 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff740031990 con 0x7ff7500b4ef0 2023-12-19T18:18:30.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.898+0000 7ff747fff640 1 -- 172.21.15.27:0/3916022297 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff740031d20 con 0x7ff7500b4ef0 2023-12-19T18:18:30.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.899+0000 7ff7561a5640 1 -- 172.21.15.27:0/3916022297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7500b4ef0 msgr2=0x7ff7500b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:30.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.899+0000 7ff7561a5640 1 --2- 172.21.15.27:0/3916022297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7500b4ef0 0x7ff7500b6300 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7ff750071e80 tx=0x7ff74002b8d0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.899+0000 7ff7561a5640 1 -- 172.21.15.27:0/3916022297 shutdown_connections 2023-12-19T18:18:30.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.899+0000 7ff7561a5640 1 --2- 172.21.15.27:0/3916022297 >> v2:172.21.15.192:3300/0 conn(0x7ff7500d2350 0x7ff7500b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.899+0000 7ff7561a5640 1 --2- 172.21.15.27:0/3916022297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7500b4ef0 0x7ff7500b6300 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.899+0000 7ff7561a5640 1 -- 172.21.15.27:0/3916022297 >> 172.21.15.27:0/3916022297 conn(0x7ff7500cf7e0 msgr2=0x7ff7500cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:30.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.899+0000 7ff7561a5640 1 -- 172.21.15.27:0/3916022297 shutdown_connections 2023-12-19T18:18:30.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.899+0000 7ff7561a5640 1 -- 172.21.15.27:0/3916022297 wait complete. 2023-12-19T18:18:30.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.901+0000 7ff7561a5640 1 Processor -- start 2023-12-19T18:18:30.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.901+0000 7ff7561a5640 1 -- start start 2023-12-19T18:18:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.902+0000 7ff7561a5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7500b4ef0 0x7ff7500d1420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.902+0000 7ff7561a5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7500d2350 0x7ff750155720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.902+0000 7ff7561a5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7501604d0 0x7ff7500c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.902+0000 7ff7561a5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7500c6a10 con 0x7ff7501604d0 2023-12-19T18:18:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.902+0000 7ff7561a5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7500c6c60 con 0x7ff7500d2350 2023-12-19T18:18:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.902+0000 7ff7561a5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff750156130 con 0x7ff7500b4ef0 2023-12-19T18:18:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.902+0000 7ff7559a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7501604d0 0x7ff7500c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.902+0000 7ff7559a4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7501604d0 0x7ff7500c9e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34204/0 (socket says 172.21.15.27:34204) 2023-12-19T18:18:30.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.902+0000 7ff7559a4640 1 -- 172.21.15.27:0/340122437 learned_addr learned my addr 172.21.15.27:0/340122437 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.903+0000 7ff7549a2640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7500d2350 0x7ff750155720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.903+0000 7ff7551a3640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7500b4ef0 0x7ff7500d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.903+0000 7ff7559a4640 1 -- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7500b4ef0 msgr2=0x7ff7500d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.903+0000 7ff7559a4640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7500b4ef0 0x7ff7500d1420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.903+0000 7ff7559a4640 1 -- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7500d2350 msgr2=0x7ff750155720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.903+0000 7ff7559a4640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7500d2350 0x7ff750155720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.903+0000 7ff7559a4640 1 -- 172.21.15.27:0/340122437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff740009400 con 0x7ff7501604d0 2023-12-19T18:18:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.903+0000 7ff7551a3640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7500b4ef0 0x7ff7500d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.903+0000 7ff7559a4640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7501604d0 0x7ff7500c9e30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff74800d800 tx=0x7ff74800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:30.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.903+0000 7ff745ffb640 1 -- 172.21.15.27:0/340122437 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74800b6c0 con 0x7ff7501604d0 2023-12-19T18:18:30.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.903+0000 7ff745ffb640 1 -- 172.21.15.27:0/340122437 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74800b840 con 0x7ff7501604d0 2023-12-19T18:18:30.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.903+0000 7ff745ffb640 1 -- 172.21.15.27:0/340122437 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74800bbf0 con 0x7ff7501604d0 2023-12-19T18:18:30.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.903+0000 7ff7561a5640 1 -- 172.21.15.27:0/340122437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff750156440 con 0x7ff7501604d0 2023-12-19T18:18:30.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.904+0000 7ff7561a5640 1 -- 172.21.15.27:0/340122437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7500d1960 con 0x7ff7501604d0 2023-12-19T18:18:30.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.905+0000 7ff7561a5640 1 -- 172.21.15.27:0/340122437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7500d1b50 con 0x7ff7501604d0 2023-12-19T18:18:30.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.906+0000 7ff745ffb640 1 -- 172.21.15.27:0/340122437 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff74800bd50 con 0x7ff7501604d0 2023-12-19T18:18:30.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.907+0000 7ff745ffb640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff73406a430 0x7ff73406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:30.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.907+0000 7ff745ffb640 1 -- 172.21.15.27:0/340122437 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff748088ec0 con 0x7ff7501604d0 2023-12-19T18:18:30.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.907+0000 7ff7551a3640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff73406a430 0x7ff73406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:30.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.909+0000 7ff7551a3640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff73406a430 0x7ff73406c8f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff74c005c00 tx=0x7ff74c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:30.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:30.911+0000 7ff745ffb640 1 -- 172.21.15.27:0/340122437 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff748054410 con 0x7ff7501604d0 2023-12-19T18:18:31.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.101+0000 7ff7561a5640 1 -- 172.21.15.27:0/340122437 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff7500ce1b0 con 0x7ff73406a430 2023-12-19T18:18:31.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.124+0000 7ff745ffb640 1 -- 172.21.15.27:0/340122437 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282493 (secure 0 0 0) 0x7ff74c00dc30 con 0x7ff73406a430 2023-12-19T18:18:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:31.130 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:18:31.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.130+0000 7ff7561a5640 1 -- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff73406a430 msgr2=0x7ff73406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:31.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.130+0000 7ff7561a5640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff73406a430 0x7ff73406c8f0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff74c005c00 tx=0x7ff74c009040 comp rx=0 tx=0).stop 2023-12-19T18:18:31.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.130+0000 7ff7561a5640 1 -- 172.21.15.27:0/340122437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7501604d0 msgr2=0x7ff7500c9e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:31.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.130+0000 7ff7561a5640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7501604d0 0x7ff7500c9e30 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7ff74800d800 tx=0x7ff74800dcc0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.131+0000 7ff7561a5640 1 -- 172.21.15.27:0/340122437 shutdown_connections 2023-12-19T18:18:31.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.131+0000 7ff7561a5640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff73406a430 0x7ff73406c8f0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.131+0000 7ff7561a5640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7501604d0 0x7ff7500c9e30 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.131+0000 7ff7561a5640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff7500d2350 0x7ff750155720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.131+0000 7ff7561a5640 1 --2- 172.21.15.27:0/340122437 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff7500b4ef0 0x7ff7500d1420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.131+0000 7ff7561a5640 1 -- 172.21.15.27:0/340122437 >> 172.21.15.27:0/340122437 conn(0x7ff7500cf7e0 msgr2=0x7ff75015ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:31.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.131+0000 7ff7561a5640 1 -- 172.21.15.27:0/340122437 shutdown_connections 2023-12-19T18:18:31.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.131+0000 7ff7561a5640 1 -- 172.21.15.27:0/340122437 wait complete. 2023-12-19T18:18:31.149 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2023-12-19T18:18:29.781159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4245,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558254,"statfs":{"total":1600000000,"available":1595652412,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.162180"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387197+0000","last_change":"2023-12-19T18:18:26.189381+0000","last_active":"2023-12-19T18:18:27.387197+0000","last_peered":"2023-12-19T18:18:27.387197+0000","last_clean":"2023-12-19T18:18:27.387197+0000","last_became_active":"2023-12-19T18:18:26.189131+0000","last_became_peered":"2023-12-19T18:18:26.189131+0000","last_unstale":"2023-12-19T18:18:27.387197+0000","last_undegraded":"2023-12-19T18:18:27.387197+0000","last_fullsized":"2023-12-19T18:18:27.387197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:24.891680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160097+0000","last_change":"2023-12-19T18:18:26.186884+0000","last_active":"2023-12-19T18:18:27.160097+0000","last_peered":"2023-12-19T18:18:27.160097+0000","last_clean":"2023-12-19T18:18:27.160097+0000","last_became_active":"2023-12-19T18:18:26.186128+0000","last_became_peered":"2023-12-19T18:18:26.186128+0000","last_unstale":"2023-12-19T18:18:27.160097+0000","last_undegraded":"2023-12-19T18:18:27.160097+0000","last_fullsized":"2023-12-19T18:18:27.160097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:14:33.249521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385253+0000","last_change":"2023-12-19T18:18:26.170465+0000","last_active":"2023-12-19T18:18:27.385253+0000","last_peered":"2023-12-19T18:18:27.385253+0000","last_clean":"2023-12-19T18:18:27.385253+0000","last_became_active":"2023-12-19T18:18:26.170240+0000","last_became_peered":"2023-12-19T18:18:26.170240+0000","last_unstale":"2023-12-19T18:18:27.385253+0000","last_undegraded":"2023-12-19T18:18:27.385253+0000","last_fullsized":"2023-12-19T18:18:27.385253+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:07.344354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159435+0000","last_change":"2023-12-19T18:18:26.184974+0000","last_active":"2023-12-19T18:18:27.159435+0000","last_peered":"2023-12-19T18:18:27.159435+0000","last_clean":"2023-12-19T18:18:27.159435+0000","last_became_active":"2023-12-19T18:18:26.184801+0000","last_became_peered":"2023-12-19T18:18:26.184801+0000","last_unstale":"2023-12-19T18:18:27.159435+0000","last_undegraded":"2023-12-19T18:18:27.159435+0000","last_fullsized":"2023-12-19T18:18:27.159435+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:13:02.427340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386795+0000","last_change":"2023-12-19T18:18:26.186303+0000","last_active":"2023-12-19T18:18:27.386795+0000","last_peered":"2023-12-19T18:18:27.386795+0000","last_clean":"2023-12-19T18:18:27.386795+0000","last_became_active":"2023-12-19T18:18:26.185558+0000","last_became_peered":"2023-12-19T18:18:26.185558+0000","last_unstale":"2023-12-19T18:18:27.386795+0000","last_undegraded":"2023-12-19T18:18:27.386795+0000","last_fullsized":"2023-12-19T18:18:27.386795+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:14:33.249521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387475+0000","last_change":"2023-12-19T18:18:26.185934+0000","last_active":"2023-12-19T18:18:27.387475+0000","last_peered":"2023-12-19T18:18:27.387475+0000","last_clean":"2023-12-19T18:18:27.387475+0000","last_became_active":"2023-12-19T18:18:26.185209+0000","last_became_peered":"2023-12-19T18:18:26.185209+0000","last_unstale":"2023-12-19T18:18:27.387475+0000","last_undegraded":"2023-12-19T18:18:27.387475+0000","last_fullsized":"2023-12-19T18:18:27.387475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:16:36.724138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159456+0000","last_change":"2023-12-19T18:18:26.175772+0000","last_active":"2023-12-19T18:18:27.159456+0000","last_peered":"2023-12-19T18:18:27.159456+0000","last_clean":"2023-12-19T18:18:27.159456+0000","last_became_active":"2023-12-19T18:18:26.175429+0000","last_became_peered":"2023-12-19T18:18:26.175429+0000","last_unstale":"2023-12-19T18:18:27.159456+0000","last_undegraded":"2023-12-19T18:18:27.159456+0000","last_fullsized":"2023-12-19T18:18:27.159456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:01:22.524186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.156740+0000","last_change":"2023-12-19T18:18:26.169375+0000","last_active":"2023-12-19T18:18:27.156740+0000","last_peered":"2023-12-19T18:18:27.156740+0000","last_clean":"2023-12-19T18:18:27.156740+0000","last_became_active":"2023-12-19T18:18:26.168092+0000","last_became_peered":"2023-12-19T18:18:26.168092+0000","last_unstale":"2023-12-19T18:18:27.156740+0000","last_undegraded":"2023-12-19T18:18:27.156740+0000","last_fullsized":"2023-12-19T18:18:27.156740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:55:54.904265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405359+0000","last_change":"2023-12-19T18:18:26.180745+0000","last_active":"2023-12-19T18:18:27.405359+0000","last_peered":"2023-12-19T18:18:27.405359+0000","last_clean":"2023-12-19T18:18:27.405359+0000","last_became_active":"2023-12-19T18:18:26.180147+0000","last_became_peered":"2023-12-19T18:18:26.180147+0000","last_unstale":"2023-12-19T18:18:27.405359+0000","last_undegraded":"2023-12-19T18:18:27.405359+0000","last_fullsized":"2023-12-19T18:18:27.405359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:12.995484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387000+0000","last_change":"2023-12-19T18:18:26.188142+0000","last_active":"2023-12-19T18:18:27.387000+0000","last_peered":"2023-12-19T18:18:27.387000+0000","last_clean":"2023-12-19T18:18:27.387000+0000","last_became_active":"2023-12-19T18:18:26.186633+0000","last_became_peered":"2023-12-19T18:18:26.186633+0000","last_unstale":"2023-12-19T18:18:27.387000+0000","last_undegraded":"2023-12-19T18:18:27.387000+0000","last_fullsized":"2023-12-19T18:18:27.387000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:17:16.473679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405395+0000","last_change":"2023-12-19T18:18:26.191246+0000","last_active":"2023-12-19T18:18:27.405395+0000","last_peered":"2023-12-19T18:18:27.405395+0000","last_clean":"2023-12-19T18:18:27.405395+0000","last_became_active":"2023-12-19T18:18:26.190893+0000","last_became_peered":"2023-12-19T18:18:26.190893+0000","last_unstale":"2023-12-19T18:18:27.405395+0000","last_undegraded":"2023-12-19T18:18:27.405395+0000","last_fullsized":"2023-12-19T18:18:27.405395+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:21:16.081794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405464+0000","last_change":"2023-12-19T18:18:26.173892+0000","last_active":"2023-12-19T18:18:27.405464+0000","last_peered":"2023-12-19T18:18:27.405464+0000","last_clean":"2023-12-19T18:18:27.405464+0000","last_became_active":"2023-12-19T18:18:26.173667+0000","last_became_peered":"2023-12-19T18:18:26.173667+0000","last_unstale":"2023-12-19T18:18:27.405464+0000","last_undegraded":"2023-12-19T18:18:27.405464+0000","last_fullsized":"2023-12-19T18:18:27.405464+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:31:41.207311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404652+0000","last_change":"2023-12-19T18:18:26.178392+0000","last_active":"2023-12-19T18:18:27.404652+0000","last_peered":"2023-12-19T18:18:27.404652+0000","last_clean":"2023-12-19T18:18:27.404652+0000","last_became_active":"2023-12-19T18:18:26.178135+0000","last_became_peered":"2023-12-19T18:18:26.178135+0000","last_unstale":"2023-12-19T18:18:27.404652+0000","last_undegraded":"2023-12-19T18:18:27.404652+0000","last_fullsized":"2023-12-19T18:18:27.404652+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:03.870971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385473+0000","last_change":"2023-12-19T18:18:26.183937+0000","last_active":"2023-12-19T18:18:27.385473+0000","last_peered":"2023-12-19T18:18:27.385473+0000","last_clean":"2023-12-19T18:18:27.385473+0000","last_became_active":"2023-12-19T18:18:26.183679+0000","last_became_peered":"2023-12-19T18:18:26.183679+0000","last_unstale":"2023-12-19T18:18:27.385473+0000","last_undegraded":"2023-12-19T18:18:27.385473+0000","last_fullsized":"2023-12-19T18:18:27.385473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:21:16.081794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404824+0000","last_change":"2023-12-19T18:18:26.185740+0000","last_active":"2023-12-19T18:18:27.404824+0000","last_peered":"2023-12-19T18:18:27.404824+0000","last_clean":"2023-12-19T18:18:27.404824+0000","last_became_active":"2023-12-19T18:18:26.185525+0000","last_became_peered":"2023-12-19T18:18:26.185525+0000","last_unstale":"2023-12-19T18:18:27.404824+0000","last_undegraded":"2023-12-19T18:18:27.404824+0000","last_fullsized":"2023-12-19T18:18:27.404824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:16:36.724138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387695+0000","last_change":"2023-12-19T18:18:26.186077+0000","last_active":"2023-12-19T18:18:27.387695+0000","last_peered":"2023-12-19T18:18:27.387695+0000","last_clean":"2023-12-19T18:18:27.387695+0000","last_became_active":"2023-12-19T18:18:26.185678+0000","last_became_peered":"2023-12-19T18:18:26.185678+0000","last_unstale":"2023-12-19T18:18:27.387695+0000","last_undegraded":"2023-12-19T18:18:27.387695+0000","last_fullsized":"2023-12-19T18:18:27.387695+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:18.216283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386434+0000","last_change":"2023-12-19T18:18:26.187332+0000","last_active":"2023-12-19T18:18:27.386434+0000","last_peered":"2023-12-19T18:18:27.386434+0000","last_clean":"2023-12-19T18:18:27.386434+0000","last_became_active":"2023-12-19T18:18:26.184793+0000","last_became_peered":"2023-12-19T18:18:26.184793+0000","last_unstale":"2023-12-19T18:18:27.386434+0000","last_undegraded":"2023-12-19T18:18:27.386434+0000","last_fullsized":"2023-12-19T18:18:27.386434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:35.953639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385513+0000","last_change":"2023-12-19T18:18:26.174826+0000","last_active":"2023-12-19T18:18:27.385513+0000","last_peered":"2023-12-19T18:18:27.385513+0000","last_clean":"2023-12-19T18:18:27.385513+0000","last_became_active":"2023-12-19T18:18:26.174132+0000","last_became_peered":"2023-12-19T18:18:26.174132+0000","last_unstale":"2023-12-19T18:18:27.385513+0000","last_undegraded":"2023-12-19T18:18:27.385513+0000","last_fullsized":"2023-12-19T18:18:27.385513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:35:21.296096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159700+0000","last_change":"2023-12-19T18:18:26.191936+0000","last_active":"2023-12-19T18:18:27.159700+0000","last_peered":"2023-12-19T18:18:27.159700+0000","last_clean":"2023-12-19T18:18:27.159700+0000","last_became_active":"2023-12-19T18:18:26.190735+0000","last_became_peered":"2023-12-19T18:18:26.190735+0000","last_unstale":"2023-12-19T18:18:27.159700+0000","last_undegraded":"2023-12-19T18:18:27.159700+0000","last_fullsized":"2023-12-19T18:18:27.159700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:17:16.473679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405766+0000","last_change":"2023-12-19T18:18:26.184020+0000","last_active":"2023-12-19T18:18:27.405766+0000","last_peered":"2023-12-19T18:18:27.405766+0000","last_clean":"2023-12-19T18:18:27.405766+0000","last_became_active":"2023-12-19T18:18:26.183782+0000","last_became_peered":"2023-12-19T18:18:26.183782+0000","last_unstale":"2023-12-19T18:18:27.405766+0000","last_undegraded":"2023-12-19T18:18:27.405766+0000","last_fullsized":"2023-12-19T18:18:27.405766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:45:09.091173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404685+0000","last_change":"2023-12-19T18:18:26.168328+0000","last_active":"2023-12-19T18:18:27.404685+0000","last_peered":"2023-12-19T18:18:27.404685+0000","last_clean":"2023-12-19T18:18:27.404685+0000","last_became_active":"2023-12-19T18:18:26.167646+0000","last_became_peered":"2023-12-19T18:18:26.167646+0000","last_unstale":"2023-12-19T18:18:27.404685+0000","last_undegraded":"2023-12-19T18:18:27.404685+0000","last_fullsized":"2023-12-19T18:18:27.404685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:14:33.249521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405399+0000","last_change":"2023-12-19T18:18:26.188763+0000","last_active":"2023-12-19T18:18:27.405399+0000","last_peered":"2023-12-19T18:18:27.405399+0000","last_clean":"2023-12-19T18:18:27.405399+0000","last_became_active":"2023-12-19T18:18:26.187324+0000","last_became_peered":"2023-12-19T18:18:26.187324+0000","last_unstale":"2023-12-19T18:18:27.405399+0000","last_undegraded":"2023-12-19T18:18:27.405399+0000","last_fullsized":"2023-12-19T18:18:27.405399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:30:32.066088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159359+0000","last_change":"2023-12-19T18:18:26.177951+0000","last_active":"2023-12-19T18:18:27.159359+0000","last_peered":"2023-12-19T18:18:27.159359+0000","last_clean":"2023-12-19T18:18:27.159359+0000","last_became_active":"2023-12-19T18:18:26.177730+0000","last_became_peered":"2023-12-19T18:18:26.177730+0000","last_unstale":"2023-12-19T18:18:27.159359+0000","last_undegraded":"2023-12-19T18:18:27.159359+0000","last_fullsized":"2023-12-19T18:18:27.159359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:24:32.250568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405608+0000","last_change":"2023-12-19T18:18:26.180944+0000","last_active":"2023-12-19T18:18:27.405608+0000","last_peered":"2023-12-19T18:18:27.405608+0000","last_clean":"2023-12-19T18:18:27.405608+0000","last_became_active":"2023-12-19T18:18:26.180501+0000","last_became_peered":"2023-12-19T18:18:26.180501+0000","last_unstale":"2023-12-19T18:18:27.405608+0000","last_undegraded":"2023-12-19T18:18:27.405608+0000","last_fullsized":"2023-12-19T18:18:27.405608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:30.829184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160206+0000","last_change":"2023-12-19T18:18:26.185900+0000","last_active":"2023-12-19T18:18:27.160206+0000","last_peered":"2023-12-19T18:18:27.160206+0000","last_clean":"2023-12-19T18:18:27.160206+0000","last_became_active":"2023-12-19T18:18:26.185652+0000","last_became_peered":"2023-12-19T18:18:26.185652+0000","last_unstale":"2023-12-19T18:18:27.160206+0000","last_undegraded":"2023-12-19T18:18:27.160206+0000","last_fullsized":"2023-12-19T18:18:27.160206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:03.870971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404947+0000","last_change":"2023-12-19T18:18:26.168479+0000","last_active":"2023-12-19T18:18:27.404947+0000","last_peered":"2023-12-19T18:18:27.404947+0000","last_clean":"2023-12-19T18:18:27.404947+0000","last_became_active":"2023-12-19T18:18:26.168126+0000","last_became_peered":"2023-12-19T18:18:26.168126+0000","last_unstale":"2023-12-19T18:18:27.404947+0000","last_undegraded":"2023-12-19T18:18:27.404947+0000","last_fullsized":"2023-12-19T18:18:27.404947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:24.891680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160537+0000","last_change":"2023-12-19T18:18:26.183701+0000","last_active":"2023-12-19T18:18:27.160537+0000","last_peered":"2023-12-19T18:18:27.160537+0000","last_clean":"2023-12-19T18:18:27.160537+0000","last_became_active":"2023-12-19T18:18:26.183257+0000","last_became_peered":"2023-12-19T18:18:26.183257+0000","last_unstale":"2023-12-19T18:18:27.160537+0000","last_undegraded":"2023-12-19T18:18:27.160537+0000","last_fullsized":"2023-12-19T18:18:27.160537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:35.953639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157029+0000","last_change":"2023-12-19T18:18:26.175166+0000","last_active":"2023-12-19T18:18:27.157029+0000","last_peered":"2023-12-19T18:18:27.157029+0000","last_clean":"2023-12-19T18:18:27.157029+0000","last_became_active":"2023-12-19T18:18:26.173675+0000","last_became_peered":"2023-12-19T18:18:26.173675+0000","last_unstale":"2023-12-19T18:18:27.157029+0000","last_undegraded":"2023-12-19T18:18:27.157029+0000","last_fullsized":"2023-12-19T18:18:27.157029+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:43.950711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157169+0000","last_change":"2023-12-19T18:18:26.182814+0000","last_active":"2023-12-19T18:18:27.157169+0000","last_peered":"2023-12-19T18:18:27.157169+0000","last_clean":"2023-12-19T18:18:27.157169+0000","last_became_active":"2023-12-19T18:18:26.180169+0000","last_became_peered":"2023-12-19T18:18:26.180169+0000","last_unstale":"2023-12-19T18:18:27.157169+0000","last_undegraded":"2023-12-19T18:18:27.157169+0000","last_fullsized":"2023-12-19T18:18:27.157169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:42:46.005407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159819+0000","last_change":"2023-12-19T18:18:26.172842+0000","last_active":"2023-12-19T18:18:27.159819+0000","last_peered":"2023-12-19T18:18:27.159819+0000","last_clean":"2023-12-19T18:18:27.159819+0000","last_became_active":"2023-12-19T18:18:26.172608+0000","last_became_peered":"2023-12-19T18:18:26.172608+0000","last_unstale":"2023-12-19T18:18:27.159819+0000","last_undegraded":"2023-12-19T18:18:27.159819+0000","last_fullsized":"2023-12-19T18:18:27.159819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:26.234202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160952+0000","last_change":"2023-12-19T18:18:26.177844+0000","last_active":"2023-12-19T18:18:27.160952+0000","last_peered":"2023-12-19T18:18:27.160952+0000","last_clean":"2023-12-19T18:18:27.160952+0000","last_became_active":"2023-12-19T18:18:26.177469+0000","last_became_peered":"2023-12-19T18:18:26.177469+0000","last_unstale":"2023-12-19T18:18:27.160952+0000","last_undegraded":"2023-12-19T18:18:27.160952+0000","last_fullsized":"2023-12-19T18:18:27.160952+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:48.749078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405633+0000","last_change":"2023-12-19T18:18:26.187922+0000","last_active":"2023-12-19T18:18:27.405633+0000","last_peered":"2023-12-19T18:18:27.405633+0000","last_clean":"2023-12-19T18:18:27.405633+0000","last_became_active":"2023-12-19T18:18:26.187012+0000","last_became_peered":"2023-12-19T18:18:26.187012+0000","last_unstale":"2023-12-19T18:18:27.405633+0000","last_undegraded":"2023-12-19T18:18:27.405633+0000","last_fullsized":"2023-12-19T18:18:27.405633+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:48.749078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386569+0000","last_change":"2023-12-19T18:18:26.178758+0000","last_active":"2023-12-19T18:18:27.386569+0000","last_peered":"2023-12-19T18:18:27.386569+0000","last_clean":"2023-12-19T18:18:27.386569+0000","last_became_active":"2023-12-19T18:18:26.178233+0000","last_became_peered":"2023-12-19T18:18:26.178233+0000","last_unstale":"2023-12-19T18:18:27.386569+0000","last_undegraded":"2023-12-19T18:18:27.386569+0000","last_fullsized":"2023-12-19T18:18:27.386569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:48.749078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"74'4","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405641+0000","last_change":"2023-12-19T18:18:25.642415+0000","last_active":"2023-12-19T18:18:27.405641+0000","last_peered":"2023-12-19T18:18:27.405641+0000","last_clean":"2023-12-19T18:18:27.405641+0000","last_became_active":"2023-12-19T18:18:25.642159+0000","last_became_peered":"2023-12-19T18:18:25.642159+0000","last_unstale":"2023-12-19T18:18:27.405641+0000","last_undegraded":"2023-12-19T18:18:27.405641+0000","last_fullsized":"2023-12-19T18:18:27.405641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:17.696507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386687+0000","last_change":"2023-12-19T18:18:26.185519+0000","last_active":"2023-12-19T18:18:27.386687+0000","last_peered":"2023-12-19T18:18:27.386687+0000","last_clean":"2023-12-19T18:18:27.386687+0000","last_became_active":"2023-12-19T18:18:26.185280+0000","last_became_peered":"2023-12-19T18:18:26.185280+0000","last_unstale":"2023-12-19T18:18:27.386687+0000","last_undegraded":"2023-12-19T18:18:27.386687+0000","last_fullsized":"2023-12-19T18:18:27.386687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:03.870971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"74'1","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159944+0000","last_change":"2023-12-19T18:18:25.647684+0000","last_active":"2023-12-19T18:18:27.159944+0000","last_peered":"2023-12-19T18:18:27.159944+0000","last_clean":"2023-12-19T18:18:27.159944+0000","last_became_active":"2023-12-19T18:18:25.646407+0000","last_became_peered":"2023-12-19T18:18:25.646407+0000","last_unstale":"2023-12-19T18:18:27.159944+0000","last_undegraded":"2023-12-19T18:18:27.159944+0000","last_fullsized":"2023-12-19T18:18:27.159944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:35:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157644+0000","last_change":"2023-12-19T18:18:26.190067+0000","last_active":"2023-12-19T18:18:27.157644+0000","last_peered":"2023-12-19T18:18:27.157644+0000","last_clean":"2023-12-19T18:18:27.157644+0000","last_became_active":"2023-12-19T18:18:26.188964+0000","last_became_peered":"2023-12-19T18:18:26.188964+0000","last_unstale":"2023-12-19T18:18:27.157644+0000","last_undegraded":"2023-12-19T18:18:27.157644+0000","last_fullsized":"2023-12-19T18:18:27.157644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:55.059947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"74'2","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387408+0000","last_change":"2023-12-19T18:18:25.643598+0000","last_active":"2023-12-19T18:18:27.387408+0000","last_peered":"2023-12-19T18:18:27.387408+0000","last_clean":"2023-12-19T18:18:27.387408+0000","last_became_active":"2023-12-19T18:18:25.642024+0000","last_became_peered":"2023-12-19T18:18:25.642024+0000","last_unstale":"2023-12-19T18:18:27.387408+0000","last_undegraded":"2023-12-19T18:18:27.387408+0000","last_fullsized":"2023-12-19T18:18:27.387408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:54.700911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387232+0000","last_change":"2023-12-19T18:18:26.187024+0000","last_active":"2023-12-19T18:18:27.387232+0000","last_peered":"2023-12-19T18:18:27.387232+0000","last_clean":"2023-12-19T18:18:27.387232+0000","last_became_active":"2023-12-19T18:18:26.186060+0000","last_became_peered":"2023-12-19T18:18:26.186060+0000","last_unstale":"2023-12-19T18:18:27.387232+0000","last_undegraded":"2023-12-19T18:18:27.387232+0000","last_fullsized":"2023-12-19T18:18:27.387232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:04.518339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"74'2","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159745+0000","last_change":"2023-12-19T18:18:25.652572+0000","last_active":"2023-12-19T18:18:27.159745+0000","last_peered":"2023-12-19T18:18:27.159745+0000","last_clean":"2023-12-19T18:18:27.159745+0000","last_became_active":"2023-12-19T18:18:25.652276+0000","last_became_peered":"2023-12-19T18:18:25.652276+0000","last_unstale":"2023-12-19T18:18:27.159745+0000","last_undegraded":"2023-12-19T18:18:27.159745+0000","last_fullsized":"2023-12-19T18:18:27.159745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:26:38.805836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157280+0000","last_change":"2023-12-19T18:18:26.179955+0000","last_active":"2023-12-19T18:18:27.157280+0000","last_peered":"2023-12-19T18:18:27.157280+0000","last_clean":"2023-12-19T18:18:27.157280+0000","last_became_active":"2023-12-19T18:18:26.179463+0000","last_became_peered":"2023-12-19T18:18:26.179463+0000","last_unstale":"2023-12-19T18:18:27.157280+0000","last_undegraded":"2023-12-19T18:18:27.157280+0000","last_fullsized":"2023-12-19T18:18:27.157280+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:24:01.815702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"74'1","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386093+0000","last_change":"2023-12-19T18:18:25.646823+0000","last_active":"2023-12-19T18:18:27.386093+0000","last_peered":"2023-12-19T18:18:27.386093+0000","last_clean":"2023-12-19T18:18:27.386093+0000","last_became_active":"2023-12-19T18:18:25.645877+0000","last_became_peered":"2023-12-19T18:18:25.645877+0000","last_unstale":"2023-12-19T18:18:27.386093+0000","last_undegraded":"2023-12-19T18:18:27.386093+0000","last_fullsized":"2023-12-19T18:18:27.386093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.746287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.406027+0000","last_change":"2023-12-19T18:18:26.187545+0000","last_active":"2023-12-19T18:18:27.406027+0000","last_peered":"2023-12-19T18:18:27.406027+0000","last_clean":"2023-12-19T18:18:27.406027+0000","last_became_active":"2023-12-19T18:18:26.184505+0000","last_became_peered":"2023-12-19T18:18:26.184505+0000","last_unstale":"2023-12-19T18:18:27.406027+0000","last_undegraded":"2023-12-19T18:18:27.406027+0000","last_fullsized":"2023-12-19T18:18:27.406027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:35.953639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"74'1","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386153+0000","last_change":"2023-12-19T18:18:25.636434+0000","last_active":"2023-12-19T18:18:27.386153+0000","last_peered":"2023-12-19T18:18:27.386153+0000","last_clean":"2023-12-19T18:18:27.386153+0000","last_became_active":"2023-12-19T18:18:25.634947+0000","last_became_peered":"2023-12-19T18:18:25.634947+0000","last_unstale":"2023-12-19T18:18:27.386153+0000","last_undegraded":"2023-12-19T18:18:27.386153+0000","last_fullsized":"2023-12-19T18:18:27.386153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:11.980864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385596+0000","last_change":"2023-12-19T18:18:26.173110+0000","last_active":"2023-12-19T18:18:27.385596+0000","last_peered":"2023-12-19T18:18:27.385596+0000","last_clean":"2023-12-19T18:18:27.385596+0000","last_became_active":"2023-12-19T18:18:26.172854+0000","last_became_peered":"2023-12-19T18:18:26.172854+0000","last_unstale":"2023-12-19T18:18:27.385596+0000","last_undegraded":"2023-12-19T18:18:27.385596+0000","last_fullsized":"2023-12-19T18:18:27.385596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:52:02.166717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.406283+0000","last_change":"2023-12-19T18:18:25.654398+0000","last_active":"2023-12-19T18:18:27.406283+0000","last_peered":"2023-12-19T18:18:27.406283+0000","last_clean":"2023-12-19T18:18:27.406283+0000","last_became_active":"2023-12-19T18:18:25.652138+0000","last_became_peered":"2023-12-19T18:18:25.652138+0000","last_unstale":"2023-12-19T18:18:27.406283+0000","last_undegraded":"2023-12-19T18:18:27.406283+0000","last_fullsized":"2023-12-19T18:18:27.406283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:34:38.298206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387311+0000","last_change":"2023-12-19T18:18:26.189853+0000","last_active":"2023-12-19T18:18:27.387311+0000","last_peered":"2023-12-19T18:18:27.387311+0000","last_clean":"2023-12-19T18:18:27.387311+0000","last_became_active":"2023-12-19T18:18:26.188328+0000","last_became_peered":"2023-12-19T18:18:26.188328+0000","last_unstale":"2023-12-19T18:18:27.387311+0000","last_undegraded":"2023-12-19T18:18:27.387311+0000","last_fullsized":"2023-12-19T18:18:27.387311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:18.711127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404903+0000","last_change":"2023-12-19T18:18:25.649471+0000","last_active":"2023-12-19T18:18:27.404903+0000","last_peered":"2023-12-19T18:18:27.404903+0000","last_clean":"2023-12-19T18:18:27.404903+0000","last_became_active":"2023-12-19T18:18:25.648142+0000","last_became_peered":"2023-12-19T18:18:25.648142+0000","last_unstale":"2023-12-19T18:18:27.404903+0000","last_undegraded":"2023-12-19T18:18:27.404903+0000","last_fullsized":"2023-12-19T18:18:27.404903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.746287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405003+0000","last_change":"2023-12-19T18:18:26.171909+0000","last_active":"2023-12-19T18:18:27.405003+0000","last_peered":"2023-12-19T18:18:27.405003+0000","last_clean":"2023-12-19T18:18:27.405003+0000","last_became_active":"2023-12-19T18:18:26.171659+0000","last_became_peered":"2023-12-19T18:18:26.171659+0000","last_unstale":"2023-12-19T18:18:27.405003+0000","last_undegraded":"2023-12-19T18:18:27.405003+0000","last_fullsized":"2023-12-19T18:18:27.405003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:42:30.659361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.406192+0000","last_change":"2023-12-19T18:18:25.644004+0000","last_active":"2023-12-19T18:18:27.406192+0000","last_peered":"2023-12-19T18:18:27.406192+0000","last_clean":"2023-12-19T18:18:27.406192+0000","last_became_active":"2023-12-19T18:18:25.643360+0000","last_became_peered":"2023-12-19T18:18:25.643360+0000","last_unstale":"2023-12-19T18:18:27.406192+0000","last_undegraded":"2023-12-19T18:18:27.406192+0000","last_fullsized":"2023-12-19T18:18:27.406192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:14:52.815745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385270+0000","last_change":"2023-12-19T18:18:26.171008+0000","last_active":"2023-12-19T18:18:27.385270+0000","last_peered":"2023-12-19T18:18:27.385270+0000","last_clean":"2023-12-19T18:18:27.385270+0000","last_became_active":"2023-12-19T18:18:26.168177+0000","last_became_peered":"2023-12-19T18:18:26.168177+0000","last_unstale":"2023-12-19T18:18:27.385270+0000","last_undegraded":"2023-12-19T18:18:27.385270+0000","last_fullsized":"2023-12-19T18:18:27.385270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:35.953639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157056+0000","last_change":"2023-12-19T18:18:25.648257+0000","last_active":"2023-12-19T18:18:27.157056+0000","last_peered":"2023-12-19T18:18:27.157056+0000","last_clean":"2023-12-19T18:18:27.157056+0000","last_became_active":"2023-12-19T18:18:25.647998+0000","last_became_peered":"2023-12-19T18:18:25.647998+0000","last_unstale":"2023-12-19T18:18:27.157056+0000","last_undegraded":"2023-12-19T18:18:27.157056+0000","last_fullsized":"2023-12-19T18:18:27.157056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:35:20.281476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157455+0000","last_change":"2023-12-19T18:18:26.182984+0000","last_active":"2023-12-19T18:18:27.157455+0000","last_peered":"2023-12-19T18:18:27.157455+0000","last_clean":"2023-12-19T18:18:27.157455+0000","last_became_active":"2023-12-19T18:18:26.181788+0000","last_became_peered":"2023-12-19T18:18:26.181788+0000","last_unstale":"2023-12-19T18:18:27.157455+0000","last_undegraded":"2023-12-19T18:18:27.157455+0000","last_fullsized":"2023-12-19T18:18:27.157455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:27:54.181063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"74'3","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159847+0000","last_change":"2023-12-19T18:18:25.652531+0000","last_active":"2023-12-19T18:18:27.159847+0000","last_peered":"2023-12-19T18:18:27.159847+0000","last_clean":"2023-12-19T18:18:27.159847+0000","last_became_active":"2023-12-19T18:18:25.652187+0000","last_became_peered":"2023-12-19T18:18:25.652187+0000","last_unstale":"2023-12-19T18:18:27.159847+0000","last_undegraded":"2023-12-19T18:18:27.159847+0000","last_fullsized":"2023-12-19T18:18:27.159847+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:12:14.402191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159667+0000","last_change":"2023-12-19T18:18:26.178973+0000","last_active":"2023-12-19T18:18:27.159667+0000","last_peered":"2023-12-19T18:18:27.159667+0000","last_clean":"2023-12-19T18:18:27.159667+0000","last_became_active":"2023-12-19T18:18:26.178443+0000","last_became_peered":"2023-12-19T18:18:26.178443+0000","last_unstale":"2023-12-19T18:18:27.159667+0000","last_undegraded":"2023-12-19T18:18:27.159667+0000","last_fullsized":"2023-12-19T18:18:27.159667+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:32.413260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"74'6","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404463+0000","last_change":"2023-12-19T18:18:25.651835+0000","last_active":"2023-12-19T18:18:27.404463+0000","last_peered":"2023-12-19T18:18:27.404463+0000","last_clean":"2023-12-19T18:18:27.404463+0000","last_became_active":"2023-12-19T18:18:25.650986+0000","last_became_peered":"2023-12-19T18:18:25.650986+0000","last_unstale":"2023-12-19T18:18:27.404463+0000","last_undegraded":"2023-12-19T18:18:27.404463+0000","last_fullsized":"2023-12-19T18:18:27.404463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:35:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405938+0000","last_change":"2023-12-19T18:18:26.174755+0000","last_active":"2023-12-19T18:18:27.405938+0000","last_peered":"2023-12-19T18:18:27.405938+0000","last_clean":"2023-12-19T18:18:27.405938+0000","last_became_active":"2023-12-19T18:18:26.174465+0000","last_became_peered":"2023-12-19T18:18:26.174465+0000","last_unstale":"2023-12-19T18:18:27.405938+0000","last_undegraded":"2023-12-19T18:18:27.405938+0000","last_fullsized":"2023-12-19T18:18:27.405938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:27:58.752396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"74'2","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157597+0000","last_change":"2023-12-19T18:18:25.648921+0000","last_active":"2023-12-19T18:18:27.157597+0000","last_peered":"2023-12-19T18:18:27.157597+0000","last_clean":"2023-12-19T18:18:27.157597+0000","last_became_active":"2023-12-19T18:18:25.648160+0000","last_became_peered":"2023-12-19T18:18:25.648160+0000","last_unstale":"2023-12-19T18:18:27.157597+0000","last_undegraded":"2023-12-19T18:18:27.157597+0000","last_fullsized":"2023-12-19T18:18:27.157597+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:06.329734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385850+0000","last_change":"2023-12-19T18:18:26.165872+0000","last_active":"2023-12-19T18:18:27.385850+0000","last_peered":"2023-12-19T18:18:27.385850+0000","last_clean":"2023-12-19T18:18:27.385850+0000","last_became_active":"2023-12-19T18:18:26.165595+0000","last_became_peered":"2023-12-19T18:18:26.165595+0000","last_unstale":"2023-12-19T18:18:27.385850+0000","last_undegraded":"2023-12-19T18:18:27.385850+0000","last_fullsized":"2023-12-19T18:18:27.385850+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:49.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159820+0000","last_change":"2023-12-19T18:18:25.645337+0000","last_active":"2023-12-19T18:18:27.159820+0000","last_peered":"2023-12-19T18:18:27.159820+0000","last_clean":"2023-12-19T18:18:27.159820+0000","last_became_active":"2023-12-19T18:18:25.644793+0000","last_became_peered":"2023-12-19T18:18:25.644793+0000","last_unstale":"2023-12-19T18:18:27.159820+0000","last_undegraded":"2023-12-19T18:18:27.159820+0000","last_fullsized":"2023-12-19T18:18:27.159820+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:27.907166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405137+0000","last_change":"2023-12-19T18:18:26.173174+0000","last_active":"2023-12-19T18:18:27.405137+0000","last_peered":"2023-12-19T18:18:27.405137+0000","last_clean":"2023-12-19T18:18:27.405137+0000","last_became_active":"2023-12-19T18:18:26.171601+0000","last_became_peered":"2023-12-19T18:18:26.171601+0000","last_unstale":"2023-12-19T18:18:27.405137+0000","last_undegraded":"2023-12-19T18:18:27.405137+0000","last_fullsized":"2023-12-19T18:18:27.405137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:55.715531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159705+0000","last_change":"2023-12-19T18:18:25.641384+0000","last_active":"2023-12-19T18:18:27.159705+0000","last_peered":"2023-12-19T18:18:27.159705+0000","last_clean":"2023-12-19T18:18:27.159705+0000","last_became_active":"2023-12-19T18:18:25.641132+0000","last_became_peered":"2023-12-19T18:18:25.641132+0000","last_unstale":"2023-12-19T18:18:27.159705+0000","last_undegraded":"2023-12-19T18:18:27.159705+0000","last_fullsized":"2023-12-19T18:18:27.159705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:37.407060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159587+0000","last_change":"2023-12-19T18:18:26.184503+0000","last_active":"2023-12-19T18:18:27.159587+0000","last_peered":"2023-12-19T18:18:27.159587+0000","last_clean":"2023-12-19T18:18:27.159587+0000","last_became_active":"2023-12-19T18:18:26.184290+0000","last_became_peered":"2023-12-19T18:18:26.184290+0000","last_unstale":"2023-12-19T18:18:27.159587+0000","last_undegraded":"2023-12-19T18:18:27.159587+0000","last_fullsized":"2023-12-19T18:18:27.159587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:45:09.232836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"74'8","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160007+0000","last_change":"2023-12-19T18:18:25.648211+0000","last_active":"2023-12-19T18:18:27.160007+0000","last_peered":"2023-12-19T18:18:27.160007+0000","last_clean":"2023-12-19T18:18:27.160007+0000","last_became_active":"2023-12-19T18:18:25.647758+0000","last_became_peered":"2023-12-19T18:18:25.647758+0000","last_unstale":"2023-12-19T18:18:27.160007+0000","last_undegraded":"2023-12-19T18:18:27.160007+0000","last_fullsized":"2023-12-19T18:18:27.160007+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:29:10.083405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386124+0000","last_change":"2023-12-19T18:18:26.166348+0000","last_active":"2023-12-19T18:18:27.386124+0000","last_peered":"2023-12-19T18:18:27.386124+0000","last_clean":"2023-12-19T18:18:27.386124+0000","last_became_active":"2023-12-19T18:18:26.166130+0000","last_became_peered":"2023-12-19T18:18:26.166130+0000","last_unstale":"2023-12-19T18:18:27.386124+0000","last_undegraded":"2023-12-19T18:18:27.386124+0000","last_fullsized":"2023-12-19T18:18:27.386124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:45:09.091173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"74'1","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160211+0000","last_change":"2023-12-19T18:18:25.648971+0000","last_active":"2023-12-19T18:18:27.160211+0000","last_peered":"2023-12-19T18:18:27.160211+0000","last_clean":"2023-12-19T18:18:27.160211+0000","last_became_active":"2023-12-19T18:18:25.647379+0000","last_became_peered":"2023-12-19T18:18:25.647379+0000","last_unstale":"2023-12-19T18:18:27.160211+0000","last_undegraded":"2023-12-19T18:18:27.160211+0000","last_fullsized":"2023-12-19T18:18:27.160211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:13:58.558747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385602+0000","last_change":"2023-12-19T18:18:26.188646+0000","last_active":"2023-12-19T18:18:27.385602+0000","last_peered":"2023-12-19T18:18:27.385602+0000","last_clean":"2023-12-19T18:18:27.385602+0000","last_became_active":"2023-12-19T18:18:26.188433+0000","last_became_peered":"2023-12-19T18:18:26.188433+0000","last_unstale":"2023-12-19T18:18:27.385602+0000","last_undegraded":"2023-12-19T18:18:27.385602+0000","last_fullsized":"2023-12-19T18:18:27.385602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:30:32.066088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"74'2","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157856+0000","last_change":"2023-12-19T18:18:25.649082+0000","last_active":"2023-12-19T18:18:27.157856+0000","last_peered":"2023-12-19T18:18:27.157856+0000","last_clean":"2023-12-19T18:18:27.157856+0000","last_became_active":"2023-12-19T18:18:25.648691+0000","last_became_peered":"2023-12-19T18:18:25.648691+0000","last_unstale":"2023-12-19T18:18:27.157856+0000","last_undegraded":"2023-12-19T18:18:27.157856+0000","last_fullsized":"2023-12-19T18:18:27.157856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:04:23.201506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160753+0000","last_change":"2023-12-19T18:18:26.187565+0000","last_active":"2023-12-19T18:18:27.160753+0000","last_peered":"2023-12-19T18:18:27.160753+0000","last_clean":"2023-12-19T18:18:27.160753+0000","last_became_active":"2023-12-19T18:18:26.186600+0000","last_became_peered":"2023-12-19T18:18:26.186600+0000","last_unstale":"2023-12-19T18:18:27.160753+0000","last_undegraded":"2023-12-19T18:18:27.160753+0000","last_fullsized":"2023-12-19T18:18:27.160753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:16:36.724138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"74'1","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385371+0000","last_change":"2023-12-19T18:18:25.651716+0000","last_active":"2023-12-19T18:18:27.385371+0000","last_peered":"2023-12-19T18:18:27.385371+0000","last_clean":"2023-12-19T18:18:27.385371+0000","last_became_active":"2023-12-19T18:18:25.651377+0000","last_became_peered":"2023-12-19T18:18:25.651377+0000","last_unstale":"2023-12-19T18:18:27.385371+0000","last_undegraded":"2023-12-19T18:18:27.385371+0000","last_fullsized":"2023-12-19T18:18:27.385371+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:25:01.093605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159622+0000","last_change":"2023-12-19T18:18:26.186728+0000","last_active":"2023-12-19T18:18:27.159622+0000","last_peered":"2023-12-19T18:18:27.159622+0000","last_clean":"2023-12-19T18:18:27.159622+0000","last_became_active":"2023-12-19T18:18:26.186517+0000","last_became_peered":"2023-12-19T18:18:26.186517+0000","last_unstale":"2023-12-19T18:18:27.159622+0000","last_undegraded":"2023-12-19T18:18:27.159622+0000","last_fullsized":"2023-12-19T18:18:27.159622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:01:42.880186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157564+0000","last_change":"2023-12-19T18:18:25.637316+0000","last_active":"2023-12-19T18:18:27.157564+0000","last_peered":"2023-12-19T18:18:27.157564+0000","last_clean":"2023-12-19T18:18:27.157564+0000","last_became_active":"2023-12-19T18:18:25.636678+0000","last_became_peered":"2023-12-19T18:18:25.636678+0000","last_unstale":"2023-12-19T18:18:27.157564+0000","last_undegraded":"2023-12-19T18:18:27.157564+0000","last_fullsized":"2023-12-19T18:18:27.157564+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:52:01.152097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.406064+0000","last_change":"2023-12-19T18:18:26.184123+0000","last_active":"2023-12-19T18:18:27.406064+0000","last_peered":"2023-12-19T18:18:27.406064+0000","last_clean":"2023-12-19T18:18:27.406064+0000","last_became_active":"2023-12-19T18:18:26.183826+0000","last_became_peered":"2023-12-19T18:18:26.183826+0000","last_unstale":"2023-12-19T18:18:27.406064+0000","last_undegraded":"2023-12-19T18:18:27.406064+0000","last_fullsized":"2023-12-19T18:18:27.406064+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:40.739835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"74'2","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405779+0000","last_change":"2023-12-19T18:18:25.647443+0000","last_active":"2023-12-19T18:18:27.405779+0000","last_peered":"2023-12-19T18:18:27.405779+0000","last_clean":"2023-12-19T18:18:27.405779+0000","last_became_active":"2023-12-19T18:18:25.646993+0000","last_became_peered":"2023-12-19T18:18:25.646993+0000","last_unstale":"2023-12-19T18:18:27.405779+0000","last_undegraded":"2023-12-19T18:18:27.405779+0000","last_fullsized":"2023-12-19T18:18:27.405779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:03.503719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157750+0000","last_change":"2023-12-19T18:18:26.175408+0000","last_active":"2023-12-19T18:18:27.157750+0000","last_peered":"2023-12-19T18:18:27.157750+0000","last_clean":"2023-12-19T18:18:27.157750+0000","last_became_active":"2023-12-19T18:18:26.174670+0000","last_became_peered":"2023-12-19T18:18:26.174670+0000","last_unstale":"2023-12-19T18:18:27.157750+0000","last_undegraded":"2023-12-19T18:18:27.157750+0000","last_fullsized":"2023-12-19T18:18:27.157750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:24.949899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.158127+0000","last_change":"2023-12-19T18:18:25.637162+0000","last_active":"2023-12-19T18:18:27.158127+0000","last_peered":"2023-12-19T18:18:27.158127+0000","last_clean":"2023-12-19T18:18:27.158127+0000","last_became_active":"2023-12-19T18:18:25.636558+0000","last_became_peered":"2023-12-19T18:18:25.636558+0000","last_unstale":"2023-12-19T18:18:27.158127+0000","last_undegraded":"2023-12-19T18:18:27.158127+0000","last_fullsized":"2023-12-19T18:18:27.158127+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:48.532680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157038+0000","last_change":"2023-12-19T18:18:26.174479+0000","last_active":"2023-12-19T18:18:27.157038+0000","last_peered":"2023-12-19T18:18:27.157038+0000","last_clean":"2023-12-19T18:18:27.157038+0000","last_became_active":"2023-12-19T18:18:26.174034+0000","last_became_peered":"2023-12-19T18:18:26.174034+0000","last_unstale":"2023-12-19T18:18:27.157038+0000","last_undegraded":"2023-12-19T18:18:27.157038+0000","last_fullsized":"2023-12-19T18:18:27.157038+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:07.749338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.406398+0000","last_change":"2023-12-19T18:18:25.653722+0000","last_active":"2023-12-19T18:18:27.406398+0000","last_peered":"2023-12-19T18:18:27.406398+0000","last_clean":"2023-12-19T18:18:27.406398+0000","last_became_active":"2023-12-19T18:18:25.653235+0000","last_became_peered":"2023-12-19T18:18:25.653235+0000","last_unstale":"2023-12-19T18:18:27.406398+0000","last_undegraded":"2023-12-19T18:18:27.406398+0000","last_fullsized":"2023-12-19T18:18:27.406398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:17:10.736690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386104+0000","last_change":"2023-12-19T18:18:26.176510+0000","last_active":"2023-12-19T18:18:27.386104+0000","last_peered":"2023-12-19T18:18:27.386104+0000","last_clean":"2023-12-19T18:18:27.386104+0000","last_became_active":"2023-12-19T18:18:26.174585+0000","last_became_peered":"2023-12-19T18:18:26.174585+0000","last_unstale":"2023-12-19T18:18:27.386104+0000","last_undegraded":"2023-12-19T18:18:27.386104+0000","last_fullsized":"2023-12-19T18:18:27.386104+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:48.749078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"74'2","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.161081+0000","last_change":"2023-12-19T18:18:25.650117+0000","last_active":"2023-12-19T18:18:27.161081+0000","last_peered":"2023-12-19T18:18:27.161081+0000","last_clean":"2023-12-19T18:18:27.161081+0000","last_became_active":"2023-12-19T18:18:25.649880+0000","last_became_peered":"2023-12-19T18:18:25.649880+0000","last_unstale":"2023-12-19T18:18:27.161081+0000","last_undegraded":"2023-12-19T18:18:27.161081+0000","last_fullsized":"2023-12-19T18:18:27.161081+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:42:00.156430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385834+0000","last_change":"2023-12-19T18:18:26.174807+0000","last_active":"2023-12-19T18:18:27.385834+0000","last_peered":"2023-12-19T18:18:27.385834+0000","last_clean":"2023-12-19T18:18:27.385834+0000","last_became_active":"2023-12-19T18:18:26.174523+0000","last_became_peered":"2023-12-19T18:18:26.174523+0000","last_unstale":"2023-12-19T18:18:27.385834+0000","last_undegraded":"2023-12-19T18:18:27.385834+0000","last_fullsized":"2023-12-19T18:18:27.385834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:04:24.216126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386435+0000","last_change":"2023-12-19T18:18:25.636867+0000","last_active":"2023-12-19T18:18:27.386435+0000","last_peered":"2023-12-19T18:18:27.386435+0000","last_clean":"2023-12-19T18:18:27.386435+0000","last_became_active":"2023-12-19T18:18:25.635811+0000","last_became_peered":"2023-12-19T18:18:25.635811+0000","last_unstale":"2023-12-19T18:18:27.386435+0000","last_undegraded":"2023-12-19T18:18:27.386435+0000","last_fullsized":"2023-12-19T18:18:27.386435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:29.814564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160468+0000","last_change":"2023-12-19T18:18:26.180020+0000","last_active":"2023-12-19T18:18:27.160468+0000","last_peered":"2023-12-19T18:18:27.160468+0000","last_clean":"2023-12-19T18:18:27.160468+0000","last_became_active":"2023-12-19T18:18:26.179727+0000","last_became_peered":"2023-12-19T18:18:26.179727+0000","last_unstale":"2023-12-19T18:18:27.160468+0000","last_undegraded":"2023-12-19T18:18:27.160468+0000","last_fullsized":"2023-12-19T18:18:27.160468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:42:30.659361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.406478+0000","last_change":"2023-12-19T18:18:25.655472+0000","last_active":"2023-12-19T18:18:27.406478+0000","last_peered":"2023-12-19T18:18:27.406478+0000","last_clean":"2023-12-19T18:18:27.406478+0000","last_became_active":"2023-12-19T18:18:25.654890+0000","last_became_peered":"2023-12-19T18:18:25.654890+0000","last_unstale":"2023-12-19T18:18:27.406478+0000","last_undegraded":"2023-12-19T18:18:27.406478+0000","last_fullsized":"2023-12-19T18:18:27.406478+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:33:56.536294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157687+0000","last_change":"2023-12-19T18:18:26.187171+0000","last_active":"2023-12-19T18:18:27.157687+0000","last_peered":"2023-12-19T18:18:27.157687+0000","last_clean":"2023-12-19T18:18:27.157687+0000","last_became_active":"2023-12-19T18:18:26.182517+0000","last_became_peered":"2023-12-19T18:18:26.182517+0000","last_unstale":"2023-12-19T18:18:27.157687+0000","last_undegraded":"2023-12-19T18:18:27.157687+0000","last_fullsized":"2023-12-19T18:18:27.157687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:55:45.918724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386345+0000","last_change":"2023-12-19T18:18:25.646228+0000","last_active":"2023-12-19T18:18:27.386345+0000","last_peered":"2023-12-19T18:18:27.386345+0000","last_clean":"2023-12-19T18:18:27.386345+0000","last_became_active":"2023-12-19T18:18:25.645749+0000","last_became_peered":"2023-12-19T18:18:25.645749+0000","last_unstale":"2023-12-19T18:18:27.386345+0000","last_undegraded":"2023-12-19T18:18:27.386345+0000","last_fullsized":"2023-12-19T18:18:27.386345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:25:37.100754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386706+0000","last_change":"2023-12-19T18:18:26.180001+0000","last_active":"2023-12-19T18:18:27.386706+0000","last_peered":"2023-12-19T18:18:27.386706+0000","last_clean":"2023-12-19T18:18:27.386706+0000","last_became_active":"2023-12-19T18:18:26.179696+0000","last_became_peered":"2023-12-19T18:18:26.179696+0000","last_unstale":"2023-12-19T18:18:27.386706+0000","last_undegraded":"2023-12-19T18:18:27.386706+0000","last_fullsized":"2023-12-19T18:18:27.386706+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:42:30.659361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.161456+0000","last_change":"2023-12-19T18:18:25.651305+0000","last_active":"2023-12-19T18:18:27.161456+0000","last_peered":"2023-12-19T18:18:27.161456+0000","last_clean":"2023-12-19T18:18:27.161456+0000","last_became_active":"2023-12-19T18:18:25.651021+0000","last_became_peered":"2023-12-19T18:18:25.651021+0000","last_unstale":"2023-12-19T18:18:27.161456+0000","last_undegraded":"2023-12-19T18:18:27.161456+0000","last_fullsized":"2023-12-19T18:18:27.161456+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:11.564449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405248+0000","last_change":"2023-12-19T18:18:26.169730+0000","last_active":"2023-12-19T18:18:27.405248+0000","last_peered":"2023-12-19T18:18:27.405248+0000","last_clean":"2023-12-19T18:18:27.405248+0000","last_became_active":"2023-12-19T18:18:26.169485+0000","last_became_peered":"2023-12-19T18:18:26.169485+0000","last_unstale":"2023-12-19T18:18:27.405248+0000","last_undegraded":"2023-12-19T18:18:27.405248+0000","last_fullsized":"2023-12-19T18:18:27.405248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:18.216283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386269+0000","last_change":"2023-12-19T18:18:25.650233+0000","last_active":"2023-12-19T18:18:27.386269+0000","last_peered":"2023-12-19T18:18:27.386269+0000","last_clean":"2023-12-19T18:18:27.386269+0000","last_became_active":"2023-12-19T18:18:25.649697+0000","last_became_peered":"2023-12-19T18:18:25.649697+0000","last_unstale":"2023-12-19T18:18:27.386269+0000","last_undegraded":"2023-12-19T18:18:27.386269+0000","last_fullsized":"2023-12-19T18:18:27.386269+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:11.572301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157916+0000","last_change":"2023-12-19T18:18:26.183143+0000","last_active":"2023-12-19T18:18:27.157916+0000","last_peered":"2023-12-19T18:18:27.157916+0000","last_clean":"2023-12-19T18:18:27.157916+0000","last_became_active":"2023-12-19T18:18:26.182120+0000","last_became_peered":"2023-12-19T18:18:26.182120+0000","last_unstale":"2023-12-19T18:18:27.157916+0000","last_undegraded":"2023-12-19T18:18:27.157916+0000","last_fullsized":"2023-12-19T18:18:27.157916+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:59:56.344185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386909+0000","last_change":"2023-12-19T18:18:25.644154+0000","last_active":"2023-12-19T18:18:27.386909+0000","last_peered":"2023-12-19T18:18:27.386909+0000","last_clean":"2023-12-19T18:18:27.386909+0000","last_became_active":"2023-12-19T18:18:25.638110+0000","last_became_peered":"2023-12-19T18:18:25.638110+0000","last_unstale":"2023-12-19T18:18:27.386909+0000","last_undegraded":"2023-12-19T18:18:27.386909+0000","last_fullsized":"2023-12-19T18:18:27.386909+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:39.725215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160473+0000","last_change":"2023-12-19T18:18:26.178921+0000","last_active":"2023-12-19T18:18:27.160473+0000","last_peered":"2023-12-19T18:18:27.160473+0000","last_clean":"2023-12-19T18:18:27.160473+0000","last_became_active":"2023-12-19T18:18:26.178595+0000","last_became_peered":"2023-12-19T18:18:26.178595+0000","last_unstale":"2023-12-19T18:18:27.160473+0000","last_undegraded":"2023-12-19T18:18:27.160473+0000","last_fullsized":"2023-12-19T18:18:27.160473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:18.711127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.158338+0000","last_change":"2023-12-19T18:18:25.645696+0000","last_active":"2023-12-19T18:18:27.158338+0000","last_peered":"2023-12-19T18:18:27.158338+0000","last_clean":"2023-12-19T18:18:27.158338+0000","last_became_active":"2023-12-19T18:18:25.642985+0000","last_became_peered":"2023-12-19T18:18:25.642985+0000","last_unstale":"2023-12-19T18:18:27.158338+0000","last_undegraded":"2023-12-19T18:18:27.158338+0000","last_fullsized":"2023-12-19T18:18:27.158338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:21:15.067174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157897+0000","last_change":"2023-12-19T18:18:26.181049+0000","last_active":"2023-12-19T18:18:27.157897+0000","last_peered":"2023-12-19T18:18:27.157897+0000","last_clean":"2023-12-19T18:18:27.157897+0000","last_became_active":"2023-12-19T18:18:26.179752+0000","last_became_peered":"2023-12-19T18:18:26.179752+0000","last_unstale":"2023-12-19T18:18:27.157897+0000","last_undegraded":"2023-12-19T18:18:27.157897+0000","last_fullsized":"2023-12-19T18:18:27.157897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:46:15.147936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404985+0000","last_change":"2023-12-19T18:18:25.649687+0000","last_active":"2023-12-19T18:18:27.404985+0000","last_peered":"2023-12-19T18:18:27.404985+0000","last_clean":"2023-12-19T18:18:27.404985+0000","last_became_active":"2023-12-19T18:18:25.648972+0000","last_became_peered":"2023-12-19T18:18:25.648972+0000","last_unstale":"2023-12-19T18:18:27.404985+0000","last_undegraded":"2023-12-19T18:18:27.404985+0000","last_fullsized":"2023-12-19T18:18:27.404985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:42:00.156430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":212,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-12-19T18:18:26.161329+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:18:26.161329+0000","last_peered":"2023-12-19T18:18:26.161329+0000","last_clean":"2023-12-19T18:18:26.161329+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:18:26.161329+0000","last_undegraded":"2023-12-19T18:18:26.161329+0000","last_fullsized":"2023-12-19T18:18:26.161329+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":386,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194926,"statfs":{"total":200000000,"available":199604642,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.123,"15min":1.1120000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1619999999999999,"15min":1.1479999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.73499999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":1.2749999999999999,"15min":1.2030000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":10.926,"5min":10.926,"15min":10.926},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.706,"5min":1.2789999999999999,"15min":1.22},"min":{"1min":0.79700000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":10.904999999999999,"5min":10.904999999999999,"15min":10.904999999999999},"last":1.077}]},{"osd":2,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.075,"15min":1.079},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6970000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.224,"5min":1.175,"15min":1.159},"min":{"1min":0.77100000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.581,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.95799999999999996}]},{"osd":3,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1299999999999999,"15min":1.1060000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.014},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.276,"15min":1.236},"min":{"1min":0.749,"5min":0.748,"15min":0.748},"max":{"1min":1.7529999999999999,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":1.1140000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.0940000000000001,"15min":1.0589999999999999},"min":{"1min":0.65400000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":1.179},{"interface":"front","average":{"1min":1.218,"5min":1.2170000000000001,"15min":1.2090000000000001},"min":{"1min":0.72999999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8180000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.2609999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3260000000000001,"15min":1.2989999999999999},"min":{"1min":0.749,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7789999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":1.23},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.298,"15min":1.288},"min":{"1min":0.71599999999999997,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.8460000000000001,"5min":2.754,"15min":2.754},"last":1.3089999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.444,"15min":1.421},"min":{"1min":0.78600000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.909,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.359,"15min":1.329},"min":{"1min":0.92300000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9390000000000001,"5min":2.149,"15min":2.149},"last":1.53}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":386,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194926,"statfs":{"total":200000000,"available":199604712,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.014,"15min":0.96799999999999997},"min":{"1min":0.66700000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.141,"5min":1.0549999999999999,"15min":1.024},"min":{"1min":0.71199999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.498,"5min":1.573,"15min":1.573},"last":0.73699999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1339999999999999,"15min":1.1240000000000001},"min":{"1min":0.74299999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":4.5540000000000003,"5min":4.5540000000000003,"15min":4.5540000000000003},"last":1.294},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.151,"15min":1.1479999999999999},"min":{"1min":0.86399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.6950000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.149,"15min":1.109},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.208,"15min":1.234},"min":{"1min":0.82599999999999996,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5189999999999999,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.24}]},{"osd":3,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.1819999999999999,"15min":1.161},"min":{"1min":0.93899999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.5620000000000001,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.137},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.1870000000000001,"15min":1.159},"min":{"1min":0.78300000000000003,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":1.524}]},{"osd":4,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.262,"15min":1.236},"min":{"1min":0.89300000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.768},{"interface":"front","average":{"1min":1.464,"5min":1.3520000000000001,"15min":1.3120000000000001},"min":{"1min":1.0589999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.859,"5min":2.859,"15min":2.859},"last":1.413}]},{"osd":5,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.2,"15min":1.1539999999999999},"min":{"1min":0.88900000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":4.6680000000000001,"5min":4.6680000000000001,"15min":4.6680000000000001},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.361,"15min":1.3220000000000001},"min":{"1min":1.0169999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":3.3580000000000001,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.8169999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.339,"15min":1.3080000000000001},"min":{"1min":1.012,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.633},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.3819999999999999,"15min":1.329},"min":{"1min":1.05,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.6639999999999997,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":1.6539999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":962,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194349,"statfs":{"total":200000000,"available":199014332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.149,"15min":1.137},"min":{"1min":0.67200000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.837,"5min":1.863,"15min":1.863},"last":1.167},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.179,"15min":1.147},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.117}]},{"osd":1,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.0309999999999999,"15min":1},"min":{"1min":0.70899999999999996,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.02},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.054,"15min":1.0229999999999999},"min":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5680000000000001,"5min":1.843,"15min":1.843},"last":1.3300000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0860000000000001,"15min":1.0529999999999999},"min":{"1min":0.78200000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.1359999999999999,"15min":1.107},"min":{"1min":0.95199999999999996,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":1.4199999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.087,"15min":1.052},"min":{"1min":0.85699999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.077},{"interface":"front","average":{"1min":1.351,"5min":1.232,"15min":1.1859999999999999},"min":{"1min":0.75700000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.274}]},{"osd":4,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.194,"15min":1.127},"min":{"1min":0.65200000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1220000000000001,"15min":1.1000000000000001},"min":{"1min":0.73999999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.637,"5min":1.901,"15min":1.901},"last":1.1499999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.3540000000000001,"15min":1.296},"min":{"1min":0.79500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.482},{"interface":"front","average":{"1min":1.29,"5min":1.238,"15min":1.214},"min":{"1min":0.91400000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.7949999999999999,"5min":1.976,"15min":1.976},"last":1.2270000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2270000000000001,"15min":1.2},"min":{"1min":0.81899999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.4950000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.407,"15min":1.389},"min":{"1min":0.84999999999999998,"5min":0.623,"15min":0.623},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.304}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":386,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194925,"statfs":{"total":200000000,"available":199604118,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0049999999999999,"15min":1},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.2270000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.0289999999999999,"15min":1.0249999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.3360000000000001,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":1.296}]},{"osd":1,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1479999999999999,"15min":1.145},"min":{"1min":0.76900000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6950000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.175,"15min":1.151},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.6479999999999999,"5min":1.696,"15min":1.696},"last":1.353}]},{"osd":2,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1200000000000001,"15min":1.121},"min":{"1min":0.85799999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.099,"15min":1.083},"min":{"1min":0.85199999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.99299999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.2649999999999999,"15min":1.2470000000000001},"min":{"1min":0.94899999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.627,"5min":1.645,"15min":1.645},"last":1.526},{"interface":"front","average":{"1min":1.248,"5min":1.2869999999999999,"15min":1.2849999999999999},"min":{"1min":0.82399999999999995,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5640000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.155}]},{"osd":5,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.2,"15min":1.167},"min":{"1min":1.0369999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.76,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.093},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.3540000000000001,"15min":1.3240000000000001},"min":{"1min":0.85999999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.5860000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.292,"15min":1.294},"min":{"1min":0.78100000000000003,"5min":0.432,"15min":0.432},"max":{"1min":1.7809999999999999,"5min":1.853,"15min":1.853},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.3839999999999999,"15min":1.3660000000000001},"min":{"1min":1.2410000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.4099999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.353,"15min":1.327},"min":{"1min":0.98099999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.7889999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.3959999999999999,"15min":1.3939999999999999},"min":{"1min":0.98899999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.835,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.5469999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":386,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194926,"statfs":{"total":200000000,"available":199604644,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.95499999999999996,"15min":0.97999999999999998},"min":{"1min":0.51800000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.5449999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.93200000000000005,"15min":0.92100000000000004},"min":{"1min":0.437,"5min":0.434,"15min":0.434},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.042,"15min":1.0720000000000001},"min":{"1min":0.67800000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.238,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.0780000000000001,"15min":1.0960000000000001},"min":{"1min":0.68999999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.53400000000000003}]},{"osd":2,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.9330000000000001,"5min":1.302,"15min":1.2430000000000001},"min":{"1min":0.58299999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":13.833,"5min":13.833,"15min":13.833},"last":0.60799999999999998},{"interface":"front","average":{"1min":2.1280000000000001,"5min":1.3680000000000001,"15min":1.2569999999999999},"min":{"1min":0.74099999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":16.748999999999999,"5min":16.748999999999999,"15min":16.748999999999999},"last":0.61699999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.212,"15min":1.1839999999999999},"min":{"1min":0.80100000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.3959999999999999,"15min":1.4099999999999999},"min":{"1min":0.96899999999999997,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.5710000000000002,"5min":4.0209999999999999,"15min":4.0209999999999999},"last":0.80800000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.4279999999999999,"15min":1.446},"min":{"1min":0.876,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.1539999999999999,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.2609999999999999,"15min":1.242},"min":{"1min":0.874,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.8300000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.95899999999999996}]},{"osd":6,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.462,"15min":1.488},"min":{"1min":0.82299999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.4790000000000001,"5min":3.9399999999999999,"15min":3.9399999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.556,"5min":1.4870000000000001,"15min":1.506},"min":{"1min":1.0169999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.0649999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.1419999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.367,"15min":1.351},"min":{"1min":0.89100000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.966,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.54,"15min":1.55},"min":{"1min":1.0489999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.657,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":0.94199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":387,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194924,"statfs":{"total":200000000,"available":199603010,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.115,"15min":1.0349999999999999},"min":{"1min":0.65000000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":9.2550000000000008,"5min":9.2550000000000008,"15min":9.2550000000000008},"last":0.752},{"interface":"front","average":{"1min":1.087,"5min":1.03,"15min":1.046},"min":{"1min":0.68300000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.0800000000000001,"5min":4.7850000000000001,"15min":4.7850000000000001},"last":0.82099999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.155,"15min":1.256},"min":{"1min":0.61099999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3480000000000001,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.115,"5min":1.1499999999999999,"15min":1.1910000000000001},"min":{"1min":0.623,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.528,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":0.98899999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.3480000000000001,"15min":1.45},"min":{"1min":0.54200000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6779999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"last":1.139},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3200000000000001,"15min":1.4059999999999999},"min":{"1min":0.69299999999999995,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.0059999999999998,"5min":8.0690000000000008,"15min":8.0690000000000008},"last":1.0369999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.4490000000000001,"15min":1.4870000000000001},"min":{"1min":0.98199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.976,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.5089999999999999,"15min":1.5860000000000001},"min":{"1min":0.96099999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.016,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":1.7969999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.544,"15min":1.6200000000000001},"min":{"1min":0.80000000000000004,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1629999999999998,"5min":8.1839999999999993,"15min":8.1839999999999993},"last":1.222},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4059999999999999,"15min":1.4179999999999999},"min":{"1min":1.0089999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.3159999999999998,"5min":5.165,"15min":5.165},"last":1.6439999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.4419999999999999,"15min":1.464},"min":{"1min":0.95999999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.9399999999999999,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":1.915},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5569999999999999,"15min":1.609},"min":{"1min":0.89200000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.2850000000000001,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":1.2789999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.5369999999999999,"15min":1.5680000000000001},"min":{"1min":0.91300000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0640000000000001,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":2.1040000000000001},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.6559999999999999,"15min":1.714},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":2.3719999999999999,"5min":8.2300000000000004,"15min":8.2300000000000004},"last":1.3280000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":963,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194349,"statfs":{"total":200000000,"available":199013784,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.877,"5min":0.93799999999999994,"15min":0.96099999999999997},"min":{"1min":0.56399999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3280000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.82599999999999996}]},{"osd":2,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.038,"15min":1.056},"min":{"1min":0.47399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4970000000000001,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.89500000000000002,"15min":0.89000000000000001},"min":{"1min":0.51400000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.264,"5min":1.597,"15min":1.597},"last":0.90900000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.123,"15min":1.105},"min":{"1min":0.65500000000000003,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.575,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.171,"15min":1.181},"min":{"1min":0.63500000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8300000000000001,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.1759999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7589999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.2789999999999999,"15min":1.2669999999999999},"min":{"1min":0.93899999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.796,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.339}]},{"osd":5,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.286,"15min":1.2609999999999999},"min":{"1min":1.0009999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":1.073},{"interface":"front","average":{"1min":1.45,"5min":1.3979999999999999,"15min":1.3859999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.0790000000000002,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.4850000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.365,"15min":1.361},"min":{"1min":1.0289999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.7729999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.302},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4330000000000001,"15min":1.4219999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.004,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.4019999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.4339999999999999,"15min":1.345},"min":{"1min":0.879,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":9.4830000000000005,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":1.536},{"interface":"front","average":{"1min":1.992,"5min":1.544,"15min":1.47},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":9.4109999999999996,"5min":9.4109999999999996,"15min":9.4109999999999996},"last":1.256}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":387,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194924,"statfs":{"total":200000000,"available":199603170,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.88600000000000001,"15min":0.86399999999999999},"min":{"1min":0.46700000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.617,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0029999999999999,"15min":0.996},"min":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.6180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.92800000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":0.93899999999999995,"15min":0.92800000000000005},"min":{"1min":0.69499999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.024,"5min":0.92600000000000005,"15min":0.90900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.7810000000000001,"5min":2.7810000000000001,"15min":2.7810000000000001},"last":0.65200000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96999999999999997,"15min":0.98599999999999999},"min":{"1min":0.54800000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.583,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.177,"15min":1.1890000000000001},"min":{"1min":0.63500000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.73,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.85599999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1890000000000001,"15min":1.157},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2529999999999999,"15min":1.2709999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9930000000000001,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.5449999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":2.089,"5min":1.4670000000000001,"15min":1.389},"min":{"1min":0.71099999999999997,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":16.298999999999999,"5min":16.298999999999999,"15min":16.298999999999999},"last":1.423},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.262,"15min":1.2450000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.079}]},{"osd":6,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.3660000000000001,"15min":1.3640000000000001},"min":{"1min":0.85499999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.948,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.353,"15min":1.325},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.1870000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.383,"15min":1.3500000000000001},"min":{"1min":0.89200000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3999999999999999,"15min":1.4179999999999999},"min":{"1min":0.873,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.1360000000000001,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":1.2350000000000001}]}]}],"pool_statfs":[]}} 2023-12-19T18:18:31.155 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:18:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.429+0000 7fd4b4573640 1 Processor -- start 2023-12-19T18:18:31.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.429+0000 7fd4b4573640 1 -- start start 2023-12-19T18:18:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.430+0000 7fd4b4573640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0b4f80 0x7fd4ac0d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.430+0000 7fd4b4573640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd4ac0d2370 0x7fd4ac0d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.431+0000 7fd4b4573640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4ac0d2ba0 con 0x7fd4ac160780 2023-12-19T18:18:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.431+0000 7fd4b4573640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4ac0d0830 con 0x7fd4ac0d2370 2023-12-19T18:18:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.431+0000 7fd4b4573640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0d21a0 con 0x7fd4ac0b4f80 2023-12-19T18:18:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.431+0000 7fd4b230f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd4ac160780 0x7fd4ac0b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34714/0 (socket says 172.21.15.27:34714) 2023-12-19T18:18:31.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.431+0000 7fd4b230f640 1 -- 172.21.15.27:0/1248667896 learned_addr learned my addr 172.21.15.27:0/1248667896 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:31.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.431+0000 7fd4b2b10640 1 --2- 172.21.15.27:0/1248667896 >> v2:172.21.15.27:3300/0 conn(0x7fd4ac0d2370 0x7fd4ac0d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:31.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.431+0000 7fd4b1b0e640 1 --2- 172.21.15.27:0/1248667896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0b4f80 0x7fd4ac0d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:31.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.431+0000 7fd4b130d640 1 -- 172.21.15.27:0/1248667896 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2800727862 0 0) 0x7fd4ac0d2ba0 con 0x7fd4ac160780 2023-12-19T18:18:31.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.432+0000 7fd4b130d640 1 -- 172.21.15.27:0/1248667896 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd49c003520 con 0x7fd4ac160780 2023-12-19T18:18:31.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.432+0000 7fd4b2b10640 1 -- 172.21.15.27:0/1248667896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0b4f80 msgr2=0x7fd4ac0d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:31.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.432+0000 7fd4b2b10640 1 --2- 172.21.15.27:0/1248667896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0b4f80 0x7fd4ac0d09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.432+0000 7fd4b2b10640 1 -- 172.21.15.27:0/1248667896 >> v1:172.21.15.27:6789/0 conn(0x7fd4ac160780 legacy=0x7fd4ac0b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:31.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.432+0000 7fd4b2b10640 1 -- 172.21.15.27:0/1248667896 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4ac0b6ef0 con 0x7fd4ac0d2370 2023-12-19T18:18:31.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.433+0000 7fd4b2b10640 1 --2- 172.21.15.27:0/1248667896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac0d2370 0x7fd4ac0d0f00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40097a0 tx=0x7fd4a402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1c6b18a704ba305 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:31.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.433+0000 7fd4b130d640 1 -- 172.21.15.27:0/1248667896 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a402d040 con 0x7fd4ac0d2370 2023-12-19T18:18:31.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.433+0000 7fd4b130d640 1 -- 172.21.15.27:0/1248667896 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4a4031810 con 0x7fd4ac0d2370 2023-12-19T18:18:31.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.433+0000 7fd4b130d640 1 -- 172.21.15.27:0/1248667896 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4a4031ba0 con 0x7fd4ac0d2370 2023-12-19T18:18:31.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.434+0000 7fd4b4573640 1 -- 172.21.15.27:0/1248667896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac0d2370 msgr2=0x7fd4ac0d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:31.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.434+0000 7fd4b4573640 1 --2- 172.21.15.27:0/1248667896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac0d2370 0x7fd4ac0d0f00 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fd4a40097a0 tx=0x7fd4a402b8b0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.434+0000 7fd4b4573640 1 -- 172.21.15.27:0/1248667896 shutdown_connections 2023-12-19T18:18:31.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.434+0000 7fd4b4573640 1 --2- 172.21.15.27:0/1248667896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac0d2370 0x7fd4ac0d0f00 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.434+0000 7fd4b4573640 1 --2- 172.21.15.27:0/1248667896 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0b4f80 0x7fd4ac0d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.434+0000 7fd4b4573640 1 -- 172.21.15.27:0/1248667896 >> 172.21.15.27:0/1248667896 conn(0x7fd4ac0cf870 msgr2=0x7fd4ac0cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:31.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.434+0000 7fd4b4573640 1 -- 172.21.15.27:0/1248667896 shutdown_connections 2023-12-19T18:18:31.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.434+0000 7fd4b4573640 1 -- 172.21.15.27:0/1248667896 wait complete. 2023-12-19T18:18:31.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.436+0000 7fd4b4573640 1 Processor -- start 2023-12-19T18:18:31.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.436+0000 7fd4b4573640 1 -- start start 2023-12-19T18:18:31.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.437+0000 7fd4b4573640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0b4f80 0x7fd4ac15e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:31.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.437+0000 7fd4b4573640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac0d2370 0x7fd4ac0c8350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:31.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.437+0000 7fd4b4573640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4ac160780 0x7fd4ac1f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:31.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.437+0000 7fd4b4573640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac1f6920 con 0x7fd4ac0d2370 2023-12-19T18:18:31.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.437+0000 7fd4b4573640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0c6ff0 con 0x7fd4ac160780 2023-12-19T18:18:31.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.437+0000 7fd4b4573640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4ac0c72b0 con 0x7fd4ac0b4f80 2023-12-19T18:18:31.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.437+0000 7fd4b1b0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac0d2370 0x7fd4ac0c8350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:31.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.437+0000 7fd4b1b0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac0d2370 0x7fd4ac0c8350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34228/0 (socket says 172.21.15.27:34228) 2023-12-19T18:18:31.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.437+0000 7fd4b1b0e640 1 -- 172.21.15.27:0/2825274640 learned_addr learned my addr 172.21.15.27:0/2825274640 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:31.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.438+0000 7fd4b230f640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0b4f80 0x7fd4ac15e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:31.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.438+0000 7fd4b2b10640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4ac160780 0x7fd4ac1f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:31.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.438+0000 7fd4b1b0e640 1 -- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0b4f80 msgr2=0x7fd4ac15e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:31.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.438+0000 7fd4b1b0e640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0b4f80 0x7fd4ac15e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.438+0000 7fd4b1b0e640 1 -- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4ac160780 msgr2=0x7fd4ac1f63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:31.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.438+0000 7fd4b1b0e640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4ac160780 0x7fd4ac1f63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.438+0000 7fd4b1b0e640 1 -- 172.21.15.27:0/2825274640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a4009400 con 0x7fd4ac0d2370 2023-12-19T18:18:31.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.438+0000 7fd4b230f640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0b4f80 0x7fd4ac15e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:31.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.438+0000 7fd4b1b0e640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac0d2370 0x7fd4ac0c8350 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e980 tx=0x7fd49800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:31.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.439+0000 7fd4aaffd640 1 -- 172.21.15.27:0/2825274640 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd498009670 con 0x7fd4ac0d2370 2023-12-19T18:18:31.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.439+0000 7fd4aaffd640 1 -- 172.21.15.27:0/2825274640 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd49800ca00 con 0x7fd4ac0d2370 2023-12-19T18:18:31.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.439+0000 7fd4aaffd640 1 -- 172.21.15.27:0/2825274640 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd49800cd90 con 0x7fd4ac0d2370 2023-12-19T18:18:31.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.439+0000 7fd4b4573640 1 -- 172.21.15.27:0/2825274640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4ac0c7590 con 0x7fd4ac0d2370 2023-12-19T18:18:31.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.439+0000 7fd4b4573640 1 -- 172.21.15.27:0/2825274640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4ac0c7b50 con 0x7fd4ac0d2370 2023-12-19T18:18:31.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.440+0000 7fd4b4573640 1 -- 172.21.15.27:0/2825274640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4ac068e90 con 0x7fd4ac0d2370 2023-12-19T18:18:31.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.441+0000 7fd4aaffd640 1 -- 172.21.15.27:0/2825274640 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd498013470 con 0x7fd4ac0d2370 2023-12-19T18:18:31.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.442+0000 7fd4aaffd640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd49006a430 0x7fd49006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:31.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.442+0000 7fd4aaffd640 1 -- 172.21.15.27:0/2825274640 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd498088b20 con 0x7fd4ac0d2370 2023-12-19T18:18:31.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.443+0000 7fd4b230f640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd49006a430 0x7fd49006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:31.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.444+0000 7fd4b230f640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd49006a430 0x7fd49006c8f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0001560 tx=0x7fd4a0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:31.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.446+0000 7fd4aaffd640 1 -- 172.21.15.27:0/2825274640 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd4980541a0 con 0x7fd4ac0d2370 2023-12-19T18:18:31.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.639+0000 7fd4b4573640 1 -- 172.21.15.27:0/2825274640 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd4ac0d28c0 con 0x7fd49006a430 2023-12-19T18:18:31.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.662+0000 7fd4aaffd640 1 -- 172.21.15.27:0/2825274640 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282493 (secure 0 0 0) 0x7fd4a0001d90 con 0x7fd49006a430 2023-12-19T18:18:31.665 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:31.668 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:18:31.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.669+0000 7fd4b4573640 1 -- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd49006a430 msgr2=0x7fd49006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:31.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.669+0000 7fd4b4573640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd49006a430 0x7fd49006c8f0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fd4a0001560 tx=0x7fd4a0009040 comp rx=0 tx=0).stop 2023-12-19T18:18:31.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.669+0000 7fd4b4573640 1 -- 172.21.15.27:0/2825274640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac0d2370 msgr2=0x7fd4ac0c8350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:31.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.669+0000 7fd4b4573640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac0d2370 0x7fd4ac0c8350 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fd49800e980 tx=0x7fd49800ee40 comp rx=0 tx=0).stop 2023-12-19T18:18:31.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.669+0000 7fd4b4573640 1 -- 172.21.15.27:0/2825274640 shutdown_connections 2023-12-19T18:18:31.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.669+0000 7fd4b4573640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd49006a430 0x7fd49006c8f0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.669+0000 7fd4b4573640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4ac160780 0x7fd4ac1f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.669+0000 7fd4b4573640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4ac0d2370 0x7fd4ac0c8350 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.669+0000 7fd4b4573640 1 --2- 172.21.15.27:0/2825274640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4ac0b4f80 0x7fd4ac15e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.669+0000 7fd4b4573640 1 -- 172.21.15.27:0/2825274640 >> 172.21.15.27:0/2825274640 conn(0x7fd4ac0cf870 msgr2=0x7fd4ac0b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.669+0000 7fd4b4573640 1 -- 172.21.15.27:0/2825274640 shutdown_connections 2023-12-19T18:18:31.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.669+0000 7fd4b4573640 1 -- 172.21.15.27:0/2825274640 wait complete. 2023-12-19T18:18:31.692 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":180,"stamp":"2023-12-19T18:18:29.781159+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":150,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":4245,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1558254,"statfs":{"total":1600000000,"available":1595652412,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.162180"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387197+0000","last_change":"2023-12-19T18:18:26.189381+0000","last_active":"2023-12-19T18:18:27.387197+0000","last_peered":"2023-12-19T18:18:27.387197+0000","last_clean":"2023-12-19T18:18:27.387197+0000","last_became_active":"2023-12-19T18:18:26.189131+0000","last_became_peered":"2023-12-19T18:18:26.189131+0000","last_unstale":"2023-12-19T18:18:27.387197+0000","last_undegraded":"2023-12-19T18:18:27.387197+0000","last_fullsized":"2023-12-19T18:18:27.387197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:24.891680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160097+0000","last_change":"2023-12-19T18:18:26.186884+0000","last_active":"2023-12-19T18:18:27.160097+0000","last_peered":"2023-12-19T18:18:27.160097+0000","last_clean":"2023-12-19T18:18:27.160097+0000","last_became_active":"2023-12-19T18:18:26.186128+0000","last_became_peered":"2023-12-19T18:18:26.186128+0000","last_unstale":"2023-12-19T18:18:27.160097+0000","last_undegraded":"2023-12-19T18:18:27.160097+0000","last_fullsized":"2023-12-19T18:18:27.160097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:14:33.249521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385253+0000","last_change":"2023-12-19T18:18:26.170465+0000","last_active":"2023-12-19T18:18:27.385253+0000","last_peered":"2023-12-19T18:18:27.385253+0000","last_clean":"2023-12-19T18:18:27.385253+0000","last_became_active":"2023-12-19T18:18:26.170240+0000","last_became_peered":"2023-12-19T18:18:26.170240+0000","last_unstale":"2023-12-19T18:18:27.385253+0000","last_undegraded":"2023-12-19T18:18:27.385253+0000","last_fullsized":"2023-12-19T18:18:27.385253+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:07.344354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159435+0000","last_change":"2023-12-19T18:18:26.184974+0000","last_active":"2023-12-19T18:18:27.159435+0000","last_peered":"2023-12-19T18:18:27.159435+0000","last_clean":"2023-12-19T18:18:27.159435+0000","last_became_active":"2023-12-19T18:18:26.184801+0000","last_became_peered":"2023-12-19T18:18:26.184801+0000","last_unstale":"2023-12-19T18:18:27.159435+0000","last_undegraded":"2023-12-19T18:18:27.159435+0000","last_fullsized":"2023-12-19T18:18:27.159435+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:13:02.427340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386795+0000","last_change":"2023-12-19T18:18:26.186303+0000","last_active":"2023-12-19T18:18:27.386795+0000","last_peered":"2023-12-19T18:18:27.386795+0000","last_clean":"2023-12-19T18:18:27.386795+0000","last_became_active":"2023-12-19T18:18:26.185558+0000","last_became_peered":"2023-12-19T18:18:26.185558+0000","last_unstale":"2023-12-19T18:18:27.386795+0000","last_undegraded":"2023-12-19T18:18:27.386795+0000","last_fullsized":"2023-12-19T18:18:27.386795+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:14:33.249521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387475+0000","last_change":"2023-12-19T18:18:26.185934+0000","last_active":"2023-12-19T18:18:27.387475+0000","last_peered":"2023-12-19T18:18:27.387475+0000","last_clean":"2023-12-19T18:18:27.387475+0000","last_became_active":"2023-12-19T18:18:26.185209+0000","last_became_peered":"2023-12-19T18:18:26.185209+0000","last_unstale":"2023-12-19T18:18:27.387475+0000","last_undegraded":"2023-12-19T18:18:27.387475+0000","last_fullsized":"2023-12-19T18:18:27.387475+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:16:36.724138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159456+0000","last_change":"2023-12-19T18:18:26.175772+0000","last_active":"2023-12-19T18:18:27.159456+0000","last_peered":"2023-12-19T18:18:27.159456+0000","last_clean":"2023-12-19T18:18:27.159456+0000","last_became_active":"2023-12-19T18:18:26.175429+0000","last_became_peered":"2023-12-19T18:18:26.175429+0000","last_unstale":"2023-12-19T18:18:27.159456+0000","last_undegraded":"2023-12-19T18:18:27.159456+0000","last_fullsized":"2023-12-19T18:18:27.159456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:01:22.524186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.156740+0000","last_change":"2023-12-19T18:18:26.169375+0000","last_active":"2023-12-19T18:18:27.156740+0000","last_peered":"2023-12-19T18:18:27.156740+0000","last_clean":"2023-12-19T18:18:27.156740+0000","last_became_active":"2023-12-19T18:18:26.168092+0000","last_became_peered":"2023-12-19T18:18:26.168092+0000","last_unstale":"2023-12-19T18:18:27.156740+0000","last_undegraded":"2023-12-19T18:18:27.156740+0000","last_fullsized":"2023-12-19T18:18:27.156740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:55:54.904265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405359+0000","last_change":"2023-12-19T18:18:26.180745+0000","last_active":"2023-12-19T18:18:27.405359+0000","last_peered":"2023-12-19T18:18:27.405359+0000","last_clean":"2023-12-19T18:18:27.405359+0000","last_became_active":"2023-12-19T18:18:26.180147+0000","last_became_peered":"2023-12-19T18:18:26.180147+0000","last_unstale":"2023-12-19T18:18:27.405359+0000","last_undegraded":"2023-12-19T18:18:27.405359+0000","last_fullsized":"2023-12-19T18:18:27.405359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:12.995484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387000+0000","last_change":"2023-12-19T18:18:26.188142+0000","last_active":"2023-12-19T18:18:27.387000+0000","last_peered":"2023-12-19T18:18:27.387000+0000","last_clean":"2023-12-19T18:18:27.387000+0000","last_became_active":"2023-12-19T18:18:26.186633+0000","last_became_peered":"2023-12-19T18:18:26.186633+0000","last_unstale":"2023-12-19T18:18:27.387000+0000","last_undegraded":"2023-12-19T18:18:27.387000+0000","last_fullsized":"2023-12-19T18:18:27.387000+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:17:16.473679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405395+0000","last_change":"2023-12-19T18:18:26.191246+0000","last_active":"2023-12-19T18:18:27.405395+0000","last_peered":"2023-12-19T18:18:27.405395+0000","last_clean":"2023-12-19T18:18:27.405395+0000","last_became_active":"2023-12-19T18:18:26.190893+0000","last_became_peered":"2023-12-19T18:18:26.190893+0000","last_unstale":"2023-12-19T18:18:27.405395+0000","last_undegraded":"2023-12-19T18:18:27.405395+0000","last_fullsized":"2023-12-19T18:18:27.405395+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:21:16.081794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405464+0000","last_change":"2023-12-19T18:18:26.173892+0000","last_active":"2023-12-19T18:18:27.405464+0000","last_peered":"2023-12-19T18:18:27.405464+0000","last_clean":"2023-12-19T18:18:27.405464+0000","last_became_active":"2023-12-19T18:18:26.173667+0000","last_became_peered":"2023-12-19T18:18:26.173667+0000","last_unstale":"2023-12-19T18:18:27.405464+0000","last_undegraded":"2023-12-19T18:18:27.405464+0000","last_fullsized":"2023-12-19T18:18:27.405464+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:31:41.207311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404652+0000","last_change":"2023-12-19T18:18:26.178392+0000","last_active":"2023-12-19T18:18:27.404652+0000","last_peered":"2023-12-19T18:18:27.404652+0000","last_clean":"2023-12-19T18:18:27.404652+0000","last_became_active":"2023-12-19T18:18:26.178135+0000","last_became_peered":"2023-12-19T18:18:26.178135+0000","last_unstale":"2023-12-19T18:18:27.404652+0000","last_undegraded":"2023-12-19T18:18:27.404652+0000","last_fullsized":"2023-12-19T18:18:27.404652+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:03.870971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385473+0000","last_change":"2023-12-19T18:18:26.183937+0000","last_active":"2023-12-19T18:18:27.385473+0000","last_peered":"2023-12-19T18:18:27.385473+0000","last_clean":"2023-12-19T18:18:27.385473+0000","last_became_active":"2023-12-19T18:18:26.183679+0000","last_became_peered":"2023-12-19T18:18:26.183679+0000","last_unstale":"2023-12-19T18:18:27.385473+0000","last_undegraded":"2023-12-19T18:18:27.385473+0000","last_fullsized":"2023-12-19T18:18:27.385473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:21:16.081794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404824+0000","last_change":"2023-12-19T18:18:26.185740+0000","last_active":"2023-12-19T18:18:27.404824+0000","last_peered":"2023-12-19T18:18:27.404824+0000","last_clean":"2023-12-19T18:18:27.404824+0000","last_became_active":"2023-12-19T18:18:26.185525+0000","last_became_peered":"2023-12-19T18:18:26.185525+0000","last_unstale":"2023-12-19T18:18:27.404824+0000","last_undegraded":"2023-12-19T18:18:27.404824+0000","last_fullsized":"2023-12-19T18:18:27.404824+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:16:36.724138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387695+0000","last_change":"2023-12-19T18:18:26.186077+0000","last_active":"2023-12-19T18:18:27.387695+0000","last_peered":"2023-12-19T18:18:27.387695+0000","last_clean":"2023-12-19T18:18:27.387695+0000","last_became_active":"2023-12-19T18:18:26.185678+0000","last_became_peered":"2023-12-19T18:18:26.185678+0000","last_unstale":"2023-12-19T18:18:27.387695+0000","last_undegraded":"2023-12-19T18:18:27.387695+0000","last_fullsized":"2023-12-19T18:18:27.387695+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:18.216283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386434+0000","last_change":"2023-12-19T18:18:26.187332+0000","last_active":"2023-12-19T18:18:27.386434+0000","last_peered":"2023-12-19T18:18:27.386434+0000","last_clean":"2023-12-19T18:18:27.386434+0000","last_became_active":"2023-12-19T18:18:26.184793+0000","last_became_peered":"2023-12-19T18:18:26.184793+0000","last_unstale":"2023-12-19T18:18:27.386434+0000","last_undegraded":"2023-12-19T18:18:27.386434+0000","last_fullsized":"2023-12-19T18:18:27.386434+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:35.953639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385513+0000","last_change":"2023-12-19T18:18:26.174826+0000","last_active":"2023-12-19T18:18:27.385513+0000","last_peered":"2023-12-19T18:18:27.385513+0000","last_clean":"2023-12-19T18:18:27.385513+0000","last_became_active":"2023-12-19T18:18:26.174132+0000","last_became_peered":"2023-12-19T18:18:26.174132+0000","last_unstale":"2023-12-19T18:18:27.385513+0000","last_undegraded":"2023-12-19T18:18:27.385513+0000","last_fullsized":"2023-12-19T18:18:27.385513+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:35:21.296096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159700+0000","last_change":"2023-12-19T18:18:26.191936+0000","last_active":"2023-12-19T18:18:27.159700+0000","last_peered":"2023-12-19T18:18:27.159700+0000","last_clean":"2023-12-19T18:18:27.159700+0000","last_became_active":"2023-12-19T18:18:26.190735+0000","last_became_peered":"2023-12-19T18:18:26.190735+0000","last_unstale":"2023-12-19T18:18:27.159700+0000","last_undegraded":"2023-12-19T18:18:27.159700+0000","last_fullsized":"2023-12-19T18:18:27.159700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:17:16.473679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405766+0000","last_change":"2023-12-19T18:18:26.184020+0000","last_active":"2023-12-19T18:18:27.405766+0000","last_peered":"2023-12-19T18:18:27.405766+0000","last_clean":"2023-12-19T18:18:27.405766+0000","last_became_active":"2023-12-19T18:18:26.183782+0000","last_became_peered":"2023-12-19T18:18:26.183782+0000","last_unstale":"2023-12-19T18:18:27.405766+0000","last_undegraded":"2023-12-19T18:18:27.405766+0000","last_fullsized":"2023-12-19T18:18:27.405766+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:45:09.091173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404685+0000","last_change":"2023-12-19T18:18:26.168328+0000","last_active":"2023-12-19T18:18:27.404685+0000","last_peered":"2023-12-19T18:18:27.404685+0000","last_clean":"2023-12-19T18:18:27.404685+0000","last_became_active":"2023-12-19T18:18:26.167646+0000","last_became_peered":"2023-12-19T18:18:26.167646+0000","last_unstale":"2023-12-19T18:18:27.404685+0000","last_undegraded":"2023-12-19T18:18:27.404685+0000","last_fullsized":"2023-12-19T18:18:27.404685+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:14:33.249521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405399+0000","last_change":"2023-12-19T18:18:26.188763+0000","last_active":"2023-12-19T18:18:27.405399+0000","last_peered":"2023-12-19T18:18:27.405399+0000","last_clean":"2023-12-19T18:18:27.405399+0000","last_became_active":"2023-12-19T18:18:26.187324+0000","last_became_peered":"2023-12-19T18:18:26.187324+0000","last_unstale":"2023-12-19T18:18:27.405399+0000","last_undegraded":"2023-12-19T18:18:27.405399+0000","last_fullsized":"2023-12-19T18:18:27.405399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:30:32.066088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159359+0000","last_change":"2023-12-19T18:18:26.177951+0000","last_active":"2023-12-19T18:18:27.159359+0000","last_peered":"2023-12-19T18:18:27.159359+0000","last_clean":"2023-12-19T18:18:27.159359+0000","last_became_active":"2023-12-19T18:18:26.177730+0000","last_became_peered":"2023-12-19T18:18:26.177730+0000","last_unstale":"2023-12-19T18:18:27.159359+0000","last_undegraded":"2023-12-19T18:18:27.159359+0000","last_fullsized":"2023-12-19T18:18:27.159359+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:24:32.250568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405608+0000","last_change":"2023-12-19T18:18:26.180944+0000","last_active":"2023-12-19T18:18:27.405608+0000","last_peered":"2023-12-19T18:18:27.405608+0000","last_clean":"2023-12-19T18:18:27.405608+0000","last_became_active":"2023-12-19T18:18:26.180501+0000","last_became_peered":"2023-12-19T18:18:26.180501+0000","last_unstale":"2023-12-19T18:18:27.405608+0000","last_undegraded":"2023-12-19T18:18:27.405608+0000","last_fullsized":"2023-12-19T18:18:27.405608+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:30.829184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160206+0000","last_change":"2023-12-19T18:18:26.185900+0000","last_active":"2023-12-19T18:18:27.160206+0000","last_peered":"2023-12-19T18:18:27.160206+0000","last_clean":"2023-12-19T18:18:27.160206+0000","last_became_active":"2023-12-19T18:18:26.185652+0000","last_became_peered":"2023-12-19T18:18:26.185652+0000","last_unstale":"2023-12-19T18:18:27.160206+0000","last_undegraded":"2023-12-19T18:18:27.160206+0000","last_fullsized":"2023-12-19T18:18:27.160206+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:03.870971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404947+0000","last_change":"2023-12-19T18:18:26.168479+0000","last_active":"2023-12-19T18:18:27.404947+0000","last_peered":"2023-12-19T18:18:27.404947+0000","last_clean":"2023-12-19T18:18:27.404947+0000","last_became_active":"2023-12-19T18:18:26.168126+0000","last_became_peered":"2023-12-19T18:18:26.168126+0000","last_unstale":"2023-12-19T18:18:27.404947+0000","last_undegraded":"2023-12-19T18:18:27.404947+0000","last_fullsized":"2023-12-19T18:18:27.404947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:24.891680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160537+0000","last_change":"2023-12-19T18:18:26.183701+0000","last_active":"2023-12-19T18:18:27.160537+0000","last_peered":"2023-12-19T18:18:27.160537+0000","last_clean":"2023-12-19T18:18:27.160537+0000","last_became_active":"2023-12-19T18:18:26.183257+0000","last_became_peered":"2023-12-19T18:18:26.183257+0000","last_unstale":"2023-12-19T18:18:27.160537+0000","last_undegraded":"2023-12-19T18:18:27.160537+0000","last_fullsized":"2023-12-19T18:18:27.160537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:35.953639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157029+0000","last_change":"2023-12-19T18:18:26.175166+0000","last_active":"2023-12-19T18:18:27.157029+0000","last_peered":"2023-12-19T18:18:27.157029+0000","last_clean":"2023-12-19T18:18:27.157029+0000","last_became_active":"2023-12-19T18:18:26.173675+0000","last_became_peered":"2023-12-19T18:18:26.173675+0000","last_unstale":"2023-12-19T18:18:27.157029+0000","last_undegraded":"2023-12-19T18:18:27.157029+0000","last_fullsized":"2023-12-19T18:18:27.157029+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:43.950711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157169+0000","last_change":"2023-12-19T18:18:26.182814+0000","last_active":"2023-12-19T18:18:27.157169+0000","last_peered":"2023-12-19T18:18:27.157169+0000","last_clean":"2023-12-19T18:18:27.157169+0000","last_became_active":"2023-12-19T18:18:26.180169+0000","last_became_peered":"2023-12-19T18:18:26.180169+0000","last_unstale":"2023-12-19T18:18:27.157169+0000","last_undegraded":"2023-12-19T18:18:27.157169+0000","last_fullsized":"2023-12-19T18:18:27.157169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:42:46.005407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159819+0000","last_change":"2023-12-19T18:18:26.172842+0000","last_active":"2023-12-19T18:18:27.159819+0000","last_peered":"2023-12-19T18:18:27.159819+0000","last_clean":"2023-12-19T18:18:27.159819+0000","last_became_active":"2023-12-19T18:18:26.172608+0000","last_became_peered":"2023-12-19T18:18:26.172608+0000","last_unstale":"2023-12-19T18:18:27.159819+0000","last_undegraded":"2023-12-19T18:18:27.159819+0000","last_fullsized":"2023-12-19T18:18:27.159819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:26.234202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160952+0000","last_change":"2023-12-19T18:18:26.177844+0000","last_active":"2023-12-19T18:18:27.160952+0000","last_peered":"2023-12-19T18:18:27.160952+0000","last_clean":"2023-12-19T18:18:27.160952+0000","last_became_active":"2023-12-19T18:18:26.177469+0000","last_became_peered":"2023-12-19T18:18:26.177469+0000","last_unstale":"2023-12-19T18:18:27.160952+0000","last_undegraded":"2023-12-19T18:18:27.160952+0000","last_fullsized":"2023-12-19T18:18:27.160952+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:48.749078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405633+0000","last_change":"2023-12-19T18:18:26.187922+0000","last_active":"2023-12-19T18:18:27.405633+0000","last_peered":"2023-12-19T18:18:27.405633+0000","last_clean":"2023-12-19T18:18:27.405633+0000","last_became_active":"2023-12-19T18:18:26.187012+0000","last_became_peered":"2023-12-19T18:18:26.187012+0000","last_unstale":"2023-12-19T18:18:27.405633+0000","last_undegraded":"2023-12-19T18:18:27.405633+0000","last_fullsized":"2023-12-19T18:18:27.405633+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:48.749078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386569+0000","last_change":"2023-12-19T18:18:26.178758+0000","last_active":"2023-12-19T18:18:27.386569+0000","last_peered":"2023-12-19T18:18:27.386569+0000","last_clean":"2023-12-19T18:18:27.386569+0000","last_became_active":"2023-12-19T18:18:26.178233+0000","last_became_peered":"2023-12-19T18:18:26.178233+0000","last_unstale":"2023-12-19T18:18:27.386569+0000","last_undegraded":"2023-12-19T18:18:27.386569+0000","last_fullsized":"2023-12-19T18:18:27.386569+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:48.749078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"74'4","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405641+0000","last_change":"2023-12-19T18:18:25.642415+0000","last_active":"2023-12-19T18:18:27.405641+0000","last_peered":"2023-12-19T18:18:27.405641+0000","last_clean":"2023-12-19T18:18:27.405641+0000","last_became_active":"2023-12-19T18:18:25.642159+0000","last_became_peered":"2023-12-19T18:18:25.642159+0000","last_unstale":"2023-12-19T18:18:27.405641+0000","last_undegraded":"2023-12-19T18:18:27.405641+0000","last_fullsized":"2023-12-19T18:18:27.405641+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:17.696507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386687+0000","last_change":"2023-12-19T18:18:26.185519+0000","last_active":"2023-12-19T18:18:27.386687+0000","last_peered":"2023-12-19T18:18:27.386687+0000","last_clean":"2023-12-19T18:18:27.386687+0000","last_became_active":"2023-12-19T18:18:26.185280+0000","last_became_peered":"2023-12-19T18:18:26.185280+0000","last_unstale":"2023-12-19T18:18:27.386687+0000","last_undegraded":"2023-12-19T18:18:27.386687+0000","last_fullsized":"2023-12-19T18:18:27.386687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:03.870971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"74'1","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159944+0000","last_change":"2023-12-19T18:18:25.647684+0000","last_active":"2023-12-19T18:18:27.159944+0000","last_peered":"2023-12-19T18:18:27.159944+0000","last_clean":"2023-12-19T18:18:27.159944+0000","last_became_active":"2023-12-19T18:18:25.646407+0000","last_became_peered":"2023-12-19T18:18:25.646407+0000","last_unstale":"2023-12-19T18:18:27.159944+0000","last_undegraded":"2023-12-19T18:18:27.159944+0000","last_fullsized":"2023-12-19T18:18:27.159944+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:35:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157644+0000","last_change":"2023-12-19T18:18:26.190067+0000","last_active":"2023-12-19T18:18:27.157644+0000","last_peered":"2023-12-19T18:18:27.157644+0000","last_clean":"2023-12-19T18:18:27.157644+0000","last_became_active":"2023-12-19T18:18:26.188964+0000","last_became_peered":"2023-12-19T18:18:26.188964+0000","last_unstale":"2023-12-19T18:18:27.157644+0000","last_undegraded":"2023-12-19T18:18:27.157644+0000","last_fullsized":"2023-12-19T18:18:27.157644+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:55.059947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"74'2","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387408+0000","last_change":"2023-12-19T18:18:25.643598+0000","last_active":"2023-12-19T18:18:27.387408+0000","last_peered":"2023-12-19T18:18:27.387408+0000","last_clean":"2023-12-19T18:18:27.387408+0000","last_became_active":"2023-12-19T18:18:25.642024+0000","last_became_peered":"2023-12-19T18:18:25.642024+0000","last_unstale":"2023-12-19T18:18:27.387408+0000","last_undegraded":"2023-12-19T18:18:27.387408+0000","last_fullsized":"2023-12-19T18:18:27.387408+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:54.700911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387232+0000","last_change":"2023-12-19T18:18:26.187024+0000","last_active":"2023-12-19T18:18:27.387232+0000","last_peered":"2023-12-19T18:18:27.387232+0000","last_clean":"2023-12-19T18:18:27.387232+0000","last_became_active":"2023-12-19T18:18:26.186060+0000","last_became_peered":"2023-12-19T18:18:26.186060+0000","last_unstale":"2023-12-19T18:18:27.387232+0000","last_undegraded":"2023-12-19T18:18:27.387232+0000","last_fullsized":"2023-12-19T18:18:27.387232+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:04.518339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"74'2","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159745+0000","last_change":"2023-12-19T18:18:25.652572+0000","last_active":"2023-12-19T18:18:27.159745+0000","last_peered":"2023-12-19T18:18:27.159745+0000","last_clean":"2023-12-19T18:18:27.159745+0000","last_became_active":"2023-12-19T18:18:25.652276+0000","last_became_peered":"2023-12-19T18:18:25.652276+0000","last_unstale":"2023-12-19T18:18:27.159745+0000","last_undegraded":"2023-12-19T18:18:27.159745+0000","last_fullsized":"2023-12-19T18:18:27.159745+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:26:38.805836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157280+0000","last_change":"2023-12-19T18:18:26.179955+0000","last_active":"2023-12-19T18:18:27.157280+0000","last_peered":"2023-12-19T18:18:27.157280+0000","last_clean":"2023-12-19T18:18:27.157280+0000","last_became_active":"2023-12-19T18:18:26.179463+0000","last_became_peered":"2023-12-19T18:18:26.179463+0000","last_unstale":"2023-12-19T18:18:27.157280+0000","last_undegraded":"2023-12-19T18:18:27.157280+0000","last_fullsized":"2023-12-19T18:18:27.157280+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:24:01.815702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"74'1","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386093+0000","last_change":"2023-12-19T18:18:25.646823+0000","last_active":"2023-12-19T18:18:27.386093+0000","last_peered":"2023-12-19T18:18:27.386093+0000","last_clean":"2023-12-19T18:18:27.386093+0000","last_became_active":"2023-12-19T18:18:25.645877+0000","last_became_peered":"2023-12-19T18:18:25.645877+0000","last_unstale":"2023-12-19T18:18:27.386093+0000","last_undegraded":"2023-12-19T18:18:27.386093+0000","last_fullsized":"2023-12-19T18:18:27.386093+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.746287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.406027+0000","last_change":"2023-12-19T18:18:26.187545+0000","last_active":"2023-12-19T18:18:27.406027+0000","last_peered":"2023-12-19T18:18:27.406027+0000","last_clean":"2023-12-19T18:18:27.406027+0000","last_became_active":"2023-12-19T18:18:26.184505+0000","last_became_peered":"2023-12-19T18:18:26.184505+0000","last_unstale":"2023-12-19T18:18:27.406027+0000","last_undegraded":"2023-12-19T18:18:27.406027+0000","last_fullsized":"2023-12-19T18:18:27.406027+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:35.953639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"74'1","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386153+0000","last_change":"2023-12-19T18:18:25.636434+0000","last_active":"2023-12-19T18:18:27.386153+0000","last_peered":"2023-12-19T18:18:27.386153+0000","last_clean":"2023-12-19T18:18:27.386153+0000","last_became_active":"2023-12-19T18:18:25.634947+0000","last_became_peered":"2023-12-19T18:18:25.634947+0000","last_unstale":"2023-12-19T18:18:27.386153+0000","last_undegraded":"2023-12-19T18:18:27.386153+0000","last_fullsized":"2023-12-19T18:18:27.386153+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:11.980864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385596+0000","last_change":"2023-12-19T18:18:26.173110+0000","last_active":"2023-12-19T18:18:27.385596+0000","last_peered":"2023-12-19T18:18:27.385596+0000","last_clean":"2023-12-19T18:18:27.385596+0000","last_became_active":"2023-12-19T18:18:26.172854+0000","last_became_peered":"2023-12-19T18:18:26.172854+0000","last_unstale":"2023-12-19T18:18:27.385596+0000","last_undegraded":"2023-12-19T18:18:27.385596+0000","last_fullsized":"2023-12-19T18:18:27.385596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:52:02.166717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.406283+0000","last_change":"2023-12-19T18:18:25.654398+0000","last_active":"2023-12-19T18:18:27.406283+0000","last_peered":"2023-12-19T18:18:27.406283+0000","last_clean":"2023-12-19T18:18:27.406283+0000","last_became_active":"2023-12-19T18:18:25.652138+0000","last_became_peered":"2023-12-19T18:18:25.652138+0000","last_unstale":"2023-12-19T18:18:27.406283+0000","last_undegraded":"2023-12-19T18:18:27.406283+0000","last_fullsized":"2023-12-19T18:18:27.406283+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:34:38.298206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.387311+0000","last_change":"2023-12-19T18:18:26.189853+0000","last_active":"2023-12-19T18:18:27.387311+0000","last_peered":"2023-12-19T18:18:27.387311+0000","last_clean":"2023-12-19T18:18:27.387311+0000","last_became_active":"2023-12-19T18:18:26.188328+0000","last_became_peered":"2023-12-19T18:18:26.188328+0000","last_unstale":"2023-12-19T18:18:27.387311+0000","last_undegraded":"2023-12-19T18:18:27.387311+0000","last_fullsized":"2023-12-19T18:18:27.387311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:18.711127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404903+0000","last_change":"2023-12-19T18:18:25.649471+0000","last_active":"2023-12-19T18:18:27.404903+0000","last_peered":"2023-12-19T18:18:27.404903+0000","last_clean":"2023-12-19T18:18:27.404903+0000","last_became_active":"2023-12-19T18:18:25.648142+0000","last_became_peered":"2023-12-19T18:18:25.648142+0000","last_unstale":"2023-12-19T18:18:27.404903+0000","last_undegraded":"2023-12-19T18:18:27.404903+0000","last_fullsized":"2023-12-19T18:18:27.404903+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:14:41.746287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405003+0000","last_change":"2023-12-19T18:18:26.171909+0000","last_active":"2023-12-19T18:18:27.405003+0000","last_peered":"2023-12-19T18:18:27.405003+0000","last_clean":"2023-12-19T18:18:27.405003+0000","last_became_active":"2023-12-19T18:18:26.171659+0000","last_became_peered":"2023-12-19T18:18:26.171659+0000","last_unstale":"2023-12-19T18:18:27.405003+0000","last_undegraded":"2023-12-19T18:18:27.405003+0000","last_fullsized":"2023-12-19T18:18:27.405003+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:42:30.659361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.406192+0000","last_change":"2023-12-19T18:18:25.644004+0000","last_active":"2023-12-19T18:18:27.406192+0000","last_peered":"2023-12-19T18:18:27.406192+0000","last_clean":"2023-12-19T18:18:27.406192+0000","last_became_active":"2023-12-19T18:18:25.643360+0000","last_became_peered":"2023-12-19T18:18:25.643360+0000","last_unstale":"2023-12-19T18:18:27.406192+0000","last_undegraded":"2023-12-19T18:18:27.406192+0000","last_fullsized":"2023-12-19T18:18:27.406192+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:14:52.815745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385270+0000","last_change":"2023-12-19T18:18:26.171008+0000","last_active":"2023-12-19T18:18:27.385270+0000","last_peered":"2023-12-19T18:18:27.385270+0000","last_clean":"2023-12-19T18:18:27.385270+0000","last_became_active":"2023-12-19T18:18:26.168177+0000","last_became_peered":"2023-12-19T18:18:26.168177+0000","last_unstale":"2023-12-19T18:18:27.385270+0000","last_undegraded":"2023-12-19T18:18:27.385270+0000","last_fullsized":"2023-12-19T18:18:27.385270+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:35.953639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157056+0000","last_change":"2023-12-19T18:18:25.648257+0000","last_active":"2023-12-19T18:18:27.157056+0000","last_peered":"2023-12-19T18:18:27.157056+0000","last_clean":"2023-12-19T18:18:27.157056+0000","last_became_active":"2023-12-19T18:18:25.647998+0000","last_became_peered":"2023-12-19T18:18:25.647998+0000","last_unstale":"2023-12-19T18:18:27.157056+0000","last_undegraded":"2023-12-19T18:18:27.157056+0000","last_fullsized":"2023-12-19T18:18:27.157056+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:35:20.281476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157455+0000","last_change":"2023-12-19T18:18:26.182984+0000","last_active":"2023-12-19T18:18:27.157455+0000","last_peered":"2023-12-19T18:18:27.157455+0000","last_clean":"2023-12-19T18:18:27.157455+0000","last_became_active":"2023-12-19T18:18:26.181788+0000","last_became_peered":"2023-12-19T18:18:26.181788+0000","last_unstale":"2023-12-19T18:18:27.157455+0000","last_undegraded":"2023-12-19T18:18:27.157455+0000","last_fullsized":"2023-12-19T18:18:27.157455+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:27:54.181063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"74'3","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159847+0000","last_change":"2023-12-19T18:18:25.652531+0000","last_active":"2023-12-19T18:18:27.159847+0000","last_peered":"2023-12-19T18:18:27.159847+0000","last_clean":"2023-12-19T18:18:27.159847+0000","last_became_active":"2023-12-19T18:18:25.652187+0000","last_became_peered":"2023-12-19T18:18:25.652187+0000","last_unstale":"2023-12-19T18:18:27.159847+0000","last_undegraded":"2023-12-19T18:18:27.159847+0000","last_fullsized":"2023-12-19T18:18:27.159847+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:12:14.402191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159667+0000","last_change":"2023-12-19T18:18:26.178973+0000","last_active":"2023-12-19T18:18:27.159667+0000","last_peered":"2023-12-19T18:18:27.159667+0000","last_clean":"2023-12-19T18:18:27.159667+0000","last_became_active":"2023-12-19T18:18:26.178443+0000","last_became_peered":"2023-12-19T18:18:26.178443+0000","last_unstale":"2023-12-19T18:18:27.159667+0000","last_undegraded":"2023-12-19T18:18:27.159667+0000","last_fullsized":"2023-12-19T18:18:27.159667+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:32.413260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"74'6","reported_seq":17,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404463+0000","last_change":"2023-12-19T18:18:25.651835+0000","last_active":"2023-12-19T18:18:27.404463+0000","last_peered":"2023-12-19T18:18:27.404463+0000","last_clean":"2023-12-19T18:18:27.404463+0000","last_became_active":"2023-12-19T18:18:25.650986+0000","last_became_peered":"2023-12-19T18:18:25.650986+0000","last_unstale":"2023-12-19T18:18:27.404463+0000","last_undegraded":"2023-12-19T18:18:27.404463+0000","last_fullsized":"2023-12-19T18:18:27.404463+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:35:02.123188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405938+0000","last_change":"2023-12-19T18:18:26.174755+0000","last_active":"2023-12-19T18:18:27.405938+0000","last_peered":"2023-12-19T18:18:27.405938+0000","last_clean":"2023-12-19T18:18:27.405938+0000","last_became_active":"2023-12-19T18:18:26.174465+0000","last_became_peered":"2023-12-19T18:18:26.174465+0000","last_unstale":"2023-12-19T18:18:27.405938+0000","last_undegraded":"2023-12-19T18:18:27.405938+0000","last_fullsized":"2023-12-19T18:18:27.405938+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:27:58.752396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"74'2","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157597+0000","last_change":"2023-12-19T18:18:25.648921+0000","last_active":"2023-12-19T18:18:27.157597+0000","last_peered":"2023-12-19T18:18:27.157597+0000","last_clean":"2023-12-19T18:18:27.157597+0000","last_became_active":"2023-12-19T18:18:25.648160+0000","last_became_peered":"2023-12-19T18:18:25.648160+0000","last_unstale":"2023-12-19T18:18:27.157597+0000","last_undegraded":"2023-12-19T18:18:27.157597+0000","last_fullsized":"2023-12-19T18:18:27.157597+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:06.329734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385850+0000","last_change":"2023-12-19T18:18:26.165872+0000","last_active":"2023-12-19T18:18:27.385850+0000","last_peered":"2023-12-19T18:18:27.385850+0000","last_clean":"2023-12-19T18:18:27.385850+0000","last_became_active":"2023-12-19T18:18:26.165595+0000","last_became_peered":"2023-12-19T18:18:26.165595+0000","last_unstale":"2023-12-19T18:18:27.385850+0000","last_undegraded":"2023-12-19T18:18:27.385850+0000","last_fullsized":"2023-12-19T18:18:27.385850+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:49.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159820+0000","last_change":"2023-12-19T18:18:25.645337+0000","last_active":"2023-12-19T18:18:27.159820+0000","last_peered":"2023-12-19T18:18:27.159820+0000","last_clean":"2023-12-19T18:18:27.159820+0000","last_became_active":"2023-12-19T18:18:25.644793+0000","last_became_peered":"2023-12-19T18:18:25.644793+0000","last_unstale":"2023-12-19T18:18:27.159820+0000","last_undegraded":"2023-12-19T18:18:27.159820+0000","last_fullsized":"2023-12-19T18:18:27.159820+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:27.907166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405137+0000","last_change":"2023-12-19T18:18:26.173174+0000","last_active":"2023-12-19T18:18:27.405137+0000","last_peered":"2023-12-19T18:18:27.405137+0000","last_clean":"2023-12-19T18:18:27.405137+0000","last_became_active":"2023-12-19T18:18:26.171601+0000","last_became_peered":"2023-12-19T18:18:26.171601+0000","last_unstale":"2023-12-19T18:18:27.405137+0000","last_undegraded":"2023-12-19T18:18:27.405137+0000","last_fullsized":"2023-12-19T18:18:27.405137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:39:55.715531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159705+0000","last_change":"2023-12-19T18:18:25.641384+0000","last_active":"2023-12-19T18:18:27.159705+0000","last_peered":"2023-12-19T18:18:27.159705+0000","last_clean":"2023-12-19T18:18:27.159705+0000","last_became_active":"2023-12-19T18:18:25.641132+0000","last_became_peered":"2023-12-19T18:18:25.641132+0000","last_unstale":"2023-12-19T18:18:27.159705+0000","last_undegraded":"2023-12-19T18:18:27.159705+0000","last_fullsized":"2023-12-19T18:18:27.159705+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:12:37.407060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159587+0000","last_change":"2023-12-19T18:18:26.184503+0000","last_active":"2023-12-19T18:18:27.159587+0000","last_peered":"2023-12-19T18:18:27.159587+0000","last_clean":"2023-12-19T18:18:27.159587+0000","last_became_active":"2023-12-19T18:18:26.184290+0000","last_became_peered":"2023-12-19T18:18:26.184290+0000","last_unstale":"2023-12-19T18:18:27.159587+0000","last_undegraded":"2023-12-19T18:18:27.159587+0000","last_fullsized":"2023-12-19T18:18:27.159587+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:45:09.232836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"74'8","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160007+0000","last_change":"2023-12-19T18:18:25.648211+0000","last_active":"2023-12-19T18:18:27.160007+0000","last_peered":"2023-12-19T18:18:27.160007+0000","last_clean":"2023-12-19T18:18:27.160007+0000","last_became_active":"2023-12-19T18:18:25.647758+0000","last_became_peered":"2023-12-19T18:18:25.647758+0000","last_unstale":"2023-12-19T18:18:27.160007+0000","last_undegraded":"2023-12-19T18:18:27.160007+0000","last_fullsized":"2023-12-19T18:18:27.160007+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:29:10.083405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386124+0000","last_change":"2023-12-19T18:18:26.166348+0000","last_active":"2023-12-19T18:18:27.386124+0000","last_peered":"2023-12-19T18:18:27.386124+0000","last_clean":"2023-12-19T18:18:27.386124+0000","last_became_active":"2023-12-19T18:18:26.166130+0000","last_became_peered":"2023-12-19T18:18:26.166130+0000","last_unstale":"2023-12-19T18:18:27.386124+0000","last_undegraded":"2023-12-19T18:18:27.386124+0000","last_fullsized":"2023-12-19T18:18:27.386124+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:45:09.091173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"74'1","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160211+0000","last_change":"2023-12-19T18:18:25.648971+0000","last_active":"2023-12-19T18:18:27.160211+0000","last_peered":"2023-12-19T18:18:27.160211+0000","last_clean":"2023-12-19T18:18:27.160211+0000","last_became_active":"2023-12-19T18:18:25.647379+0000","last_became_peered":"2023-12-19T18:18:25.647379+0000","last_unstale":"2023-12-19T18:18:27.160211+0000","last_undegraded":"2023-12-19T18:18:27.160211+0000","last_fullsized":"2023-12-19T18:18:27.160211+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:13:58.558747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385602+0000","last_change":"2023-12-19T18:18:26.188646+0000","last_active":"2023-12-19T18:18:27.385602+0000","last_peered":"2023-12-19T18:18:27.385602+0000","last_clean":"2023-12-19T18:18:27.385602+0000","last_became_active":"2023-12-19T18:18:26.188433+0000","last_became_peered":"2023-12-19T18:18:26.188433+0000","last_unstale":"2023-12-19T18:18:27.385602+0000","last_undegraded":"2023-12-19T18:18:27.385602+0000","last_fullsized":"2023-12-19T18:18:27.385602+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:30:32.066088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"74'2","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157856+0000","last_change":"2023-12-19T18:18:25.649082+0000","last_active":"2023-12-19T18:18:27.157856+0000","last_peered":"2023-12-19T18:18:27.157856+0000","last_clean":"2023-12-19T18:18:27.157856+0000","last_became_active":"2023-12-19T18:18:25.648691+0000","last_became_peered":"2023-12-19T18:18:25.648691+0000","last_unstale":"2023-12-19T18:18:27.157856+0000","last_undegraded":"2023-12-19T18:18:27.157856+0000","last_fullsized":"2023-12-19T18:18:27.157856+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:04:23.201506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160753+0000","last_change":"2023-12-19T18:18:26.187565+0000","last_active":"2023-12-19T18:18:27.160753+0000","last_peered":"2023-12-19T18:18:27.160753+0000","last_clean":"2023-12-19T18:18:27.160753+0000","last_became_active":"2023-12-19T18:18:26.186600+0000","last_became_peered":"2023-12-19T18:18:26.186600+0000","last_unstale":"2023-12-19T18:18:27.160753+0000","last_undegraded":"2023-12-19T18:18:27.160753+0000","last_fullsized":"2023-12-19T18:18:27.160753+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:16:36.724138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"74'1","reported_seq":12,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385371+0000","last_change":"2023-12-19T18:18:25.651716+0000","last_active":"2023-12-19T18:18:27.385371+0000","last_peered":"2023-12-19T18:18:27.385371+0000","last_clean":"2023-12-19T18:18:27.385371+0000","last_became_active":"2023-12-19T18:18:25.651377+0000","last_became_peered":"2023-12-19T18:18:25.651377+0000","last_unstale":"2023-12-19T18:18:27.385371+0000","last_undegraded":"2023-12-19T18:18:27.385371+0000","last_fullsized":"2023-12-19T18:18:27.385371+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:25:01.093605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.159622+0000","last_change":"2023-12-19T18:18:26.186728+0000","last_active":"2023-12-19T18:18:27.159622+0000","last_peered":"2023-12-19T18:18:27.159622+0000","last_clean":"2023-12-19T18:18:27.159622+0000","last_became_active":"2023-12-19T18:18:26.186517+0000","last_became_peered":"2023-12-19T18:18:26.186517+0000","last_unstale":"2023-12-19T18:18:27.159622+0000","last_undegraded":"2023-12-19T18:18:27.159622+0000","last_fullsized":"2023-12-19T18:18:27.159622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:01:42.880186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157564+0000","last_change":"2023-12-19T18:18:25.637316+0000","last_active":"2023-12-19T18:18:27.157564+0000","last_peered":"2023-12-19T18:18:27.157564+0000","last_clean":"2023-12-19T18:18:27.157564+0000","last_became_active":"2023-12-19T18:18:25.636678+0000","last_became_peered":"2023-12-19T18:18:25.636678+0000","last_unstale":"2023-12-19T18:18:27.157564+0000","last_undegraded":"2023-12-19T18:18:27.157564+0000","last_fullsized":"2023-12-19T18:18:27.157564+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:52:01.152097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.406064+0000","last_change":"2023-12-19T18:18:26.184123+0000","last_active":"2023-12-19T18:18:27.406064+0000","last_peered":"2023-12-19T18:18:27.406064+0000","last_clean":"2023-12-19T18:18:27.406064+0000","last_became_active":"2023-12-19T18:18:26.183826+0000","last_became_peered":"2023-12-19T18:18:26.183826+0000","last_unstale":"2023-12-19T18:18:27.406064+0000","last_undegraded":"2023-12-19T18:18:27.406064+0000","last_fullsized":"2023-12-19T18:18:27.406064+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:40.739835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"74'2","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405779+0000","last_change":"2023-12-19T18:18:25.647443+0000","last_active":"2023-12-19T18:18:27.405779+0000","last_peered":"2023-12-19T18:18:27.405779+0000","last_clean":"2023-12-19T18:18:27.405779+0000","last_became_active":"2023-12-19T18:18:25.646993+0000","last_became_peered":"2023-12-19T18:18:25.646993+0000","last_unstale":"2023-12-19T18:18:27.405779+0000","last_undegraded":"2023-12-19T18:18:27.405779+0000","last_fullsized":"2023-12-19T18:18:27.405779+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:03.503719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157750+0000","last_change":"2023-12-19T18:18:26.175408+0000","last_active":"2023-12-19T18:18:27.157750+0000","last_peered":"2023-12-19T18:18:27.157750+0000","last_clean":"2023-12-19T18:18:27.157750+0000","last_became_active":"2023-12-19T18:18:26.174670+0000","last_became_peered":"2023-12-19T18:18:26.174670+0000","last_unstale":"2023-12-19T18:18:27.157750+0000","last_undegraded":"2023-12-19T18:18:27.157750+0000","last_fullsized":"2023-12-19T18:18:27.157750+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:24.949899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.158127+0000","last_change":"2023-12-19T18:18:25.637162+0000","last_active":"2023-12-19T18:18:27.158127+0000","last_peered":"2023-12-19T18:18:27.158127+0000","last_clean":"2023-12-19T18:18:27.158127+0000","last_became_active":"2023-12-19T18:18:25.636558+0000","last_became_peered":"2023-12-19T18:18:25.636558+0000","last_unstale":"2023-12-19T18:18:27.158127+0000","last_undegraded":"2023-12-19T18:18:27.158127+0000","last_fullsized":"2023-12-19T18:18:27.158127+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:24:48.532680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157038+0000","last_change":"2023-12-19T18:18:26.174479+0000","last_active":"2023-12-19T18:18:27.157038+0000","last_peered":"2023-12-19T18:18:27.157038+0000","last_clean":"2023-12-19T18:18:27.157038+0000","last_became_active":"2023-12-19T18:18:26.174034+0000","last_became_peered":"2023-12-19T18:18:26.174034+0000","last_unstale":"2023-12-19T18:18:27.157038+0000","last_undegraded":"2023-12-19T18:18:27.157038+0000","last_fullsized":"2023-12-19T18:18:27.157038+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:07.749338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.406398+0000","last_change":"2023-12-19T18:18:25.653722+0000","last_active":"2023-12-19T18:18:27.406398+0000","last_peered":"2023-12-19T18:18:27.406398+0000","last_clean":"2023-12-19T18:18:27.406398+0000","last_became_active":"2023-12-19T18:18:25.653235+0000","last_became_peered":"2023-12-19T18:18:25.653235+0000","last_unstale":"2023-12-19T18:18:27.406398+0000","last_undegraded":"2023-12-19T18:18:27.406398+0000","last_fullsized":"2023-12-19T18:18:27.406398+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:17:10.736690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386104+0000","last_change":"2023-12-19T18:18:26.176510+0000","last_active":"2023-12-19T18:18:27.386104+0000","last_peered":"2023-12-19T18:18:27.386104+0000","last_clean":"2023-12-19T18:18:27.386104+0000","last_became_active":"2023-12-19T18:18:26.174585+0000","last_became_peered":"2023-12-19T18:18:26.174585+0000","last_unstale":"2023-12-19T18:18:27.386104+0000","last_undegraded":"2023-12-19T18:18:27.386104+0000","last_fullsized":"2023-12-19T18:18:27.386104+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:03:48.749078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"74'2","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.161081+0000","last_change":"2023-12-19T18:18:25.650117+0000","last_active":"2023-12-19T18:18:27.161081+0000","last_peered":"2023-12-19T18:18:27.161081+0000","last_clean":"2023-12-19T18:18:27.161081+0000","last_became_active":"2023-12-19T18:18:25.649880+0000","last_became_peered":"2023-12-19T18:18:25.649880+0000","last_unstale":"2023-12-19T18:18:27.161081+0000","last_undegraded":"2023-12-19T18:18:27.161081+0000","last_fullsized":"2023-12-19T18:18:27.161081+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:42:00.156430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.385834+0000","last_change":"2023-12-19T18:18:26.174807+0000","last_active":"2023-12-19T18:18:27.385834+0000","last_peered":"2023-12-19T18:18:27.385834+0000","last_clean":"2023-12-19T18:18:27.385834+0000","last_became_active":"2023-12-19T18:18:26.174523+0000","last_became_peered":"2023-12-19T18:18:26.174523+0000","last_unstale":"2023-12-19T18:18:27.385834+0000","last_undegraded":"2023-12-19T18:18:27.385834+0000","last_fullsized":"2023-12-19T18:18:27.385834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:04:24.216126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386435+0000","last_change":"2023-12-19T18:18:25.636867+0000","last_active":"2023-12-19T18:18:27.386435+0000","last_peered":"2023-12-19T18:18:27.386435+0000","last_clean":"2023-12-19T18:18:27.386435+0000","last_became_active":"2023-12-19T18:18:25.635811+0000","last_became_peered":"2023-12-19T18:18:25.635811+0000","last_unstale":"2023-12-19T18:18:27.386435+0000","last_undegraded":"2023-12-19T18:18:27.386435+0000","last_fullsized":"2023-12-19T18:18:27.386435+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:29.814564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160468+0000","last_change":"2023-12-19T18:18:26.180020+0000","last_active":"2023-12-19T18:18:27.160468+0000","last_peered":"2023-12-19T18:18:27.160468+0000","last_clean":"2023-12-19T18:18:27.160468+0000","last_became_active":"2023-12-19T18:18:26.179727+0000","last_became_peered":"2023-12-19T18:18:26.179727+0000","last_unstale":"2023-12-19T18:18:27.160468+0000","last_undegraded":"2023-12-19T18:18:27.160468+0000","last_fullsized":"2023-12-19T18:18:27.160468+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:42:30.659361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.406478+0000","last_change":"2023-12-19T18:18:25.655472+0000","last_active":"2023-12-19T18:18:27.406478+0000","last_peered":"2023-12-19T18:18:27.406478+0000","last_clean":"2023-12-19T18:18:27.406478+0000","last_became_active":"2023-12-19T18:18:25.654890+0000","last_became_peered":"2023-12-19T18:18:25.654890+0000","last_unstale":"2023-12-19T18:18:27.406478+0000","last_undegraded":"2023-12-19T18:18:27.406478+0000","last_fullsized":"2023-12-19T18:18:27.406478+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:33:56.536294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157687+0000","last_change":"2023-12-19T18:18:26.187171+0000","last_active":"2023-12-19T18:18:27.157687+0000","last_peered":"2023-12-19T18:18:27.157687+0000","last_clean":"2023-12-19T18:18:27.157687+0000","last_became_active":"2023-12-19T18:18:26.182517+0000","last_became_peered":"2023-12-19T18:18:26.182517+0000","last_unstale":"2023-12-19T18:18:27.157687+0000","last_undegraded":"2023-12-19T18:18:27.157687+0000","last_fullsized":"2023-12-19T18:18:27.157687+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:55:45.918724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386345+0000","last_change":"2023-12-19T18:18:25.646228+0000","last_active":"2023-12-19T18:18:27.386345+0000","last_peered":"2023-12-19T18:18:27.386345+0000","last_clean":"2023-12-19T18:18:27.386345+0000","last_became_active":"2023-12-19T18:18:25.645749+0000","last_became_peered":"2023-12-19T18:18:25.645749+0000","last_unstale":"2023-12-19T18:18:27.386345+0000","last_undegraded":"2023-12-19T18:18:27.386345+0000","last_fullsized":"2023-12-19T18:18:27.386345+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:25:37.100754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386706+0000","last_change":"2023-12-19T18:18:26.180001+0000","last_active":"2023-12-19T18:18:27.386706+0000","last_peered":"2023-12-19T18:18:27.386706+0000","last_clean":"2023-12-19T18:18:27.386706+0000","last_became_active":"2023-12-19T18:18:26.179696+0000","last_became_peered":"2023-12-19T18:18:26.179696+0000","last_unstale":"2023-12-19T18:18:27.386706+0000","last_undegraded":"2023-12-19T18:18:27.386706+0000","last_fullsized":"2023-12-19T18:18:27.386706+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:42:30.659361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.161456+0000","last_change":"2023-12-19T18:18:25.651305+0000","last_active":"2023-12-19T18:18:27.161456+0000","last_peered":"2023-12-19T18:18:27.161456+0000","last_clean":"2023-12-19T18:18:27.161456+0000","last_became_active":"2023-12-19T18:18:25.651021+0000","last_became_peered":"2023-12-19T18:18:25.651021+0000","last_unstale":"2023-12-19T18:18:27.161456+0000","last_undegraded":"2023-12-19T18:18:27.161456+0000","last_fullsized":"2023-12-19T18:18:27.161456+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:02:11.564449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.405248+0000","last_change":"2023-12-19T18:18:26.169730+0000","last_active":"2023-12-19T18:18:27.405248+0000","last_peered":"2023-12-19T18:18:27.405248+0000","last_clean":"2023-12-19T18:18:27.405248+0000","last_became_active":"2023-12-19T18:18:26.169485+0000","last_became_peered":"2023-12-19T18:18:26.169485+0000","last_unstale":"2023-12-19T18:18:27.405248+0000","last_undegraded":"2023-12-19T18:18:27.405248+0000","last_fullsized":"2023-12-19T18:18:27.405248+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:18.216283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386269+0000","last_change":"2023-12-19T18:18:25.650233+0000","last_active":"2023-12-19T18:18:27.386269+0000","last_peered":"2023-12-19T18:18:27.386269+0000","last_clean":"2023-12-19T18:18:27.386269+0000","last_became_active":"2023-12-19T18:18:25.649697+0000","last_became_peered":"2023-12-19T18:18:25.649697+0000","last_unstale":"2023-12-19T18:18:27.386269+0000","last_undegraded":"2023-12-19T18:18:27.386269+0000","last_fullsized":"2023-12-19T18:18:27.386269+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:11.572301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157916+0000","last_change":"2023-12-19T18:18:26.183143+0000","last_active":"2023-12-19T18:18:27.157916+0000","last_peered":"2023-12-19T18:18:27.157916+0000","last_clean":"2023-12-19T18:18:27.157916+0000","last_became_active":"2023-12-19T18:18:26.182120+0000","last_became_peered":"2023-12-19T18:18:26.182120+0000","last_unstale":"2023-12-19T18:18:27.157916+0000","last_undegraded":"2023-12-19T18:18:27.157916+0000","last_fullsized":"2023-12-19T18:18:27.157916+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:59:56.344185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.386909+0000","last_change":"2023-12-19T18:18:25.644154+0000","last_active":"2023-12-19T18:18:27.386909+0000","last_peered":"2023-12-19T18:18:27.386909+0000","last_clean":"2023-12-19T18:18:27.386909+0000","last_became_active":"2023-12-19T18:18:25.638110+0000","last_became_peered":"2023-12-19T18:18:25.638110+0000","last_unstale":"2023-12-19T18:18:27.386909+0000","last_undegraded":"2023-12-19T18:18:27.386909+0000","last_fullsized":"2023-12-19T18:18:27.386909+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:39.725215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.160473+0000","last_change":"2023-12-19T18:18:26.178921+0000","last_active":"2023-12-19T18:18:27.160473+0000","last_peered":"2023-12-19T18:18:27.160473+0000","last_clean":"2023-12-19T18:18:27.160473+0000","last_became_active":"2023-12-19T18:18:26.178595+0000","last_became_peered":"2023-12-19T18:18:26.178595+0000","last_unstale":"2023-12-19T18:18:27.160473+0000","last_undegraded":"2023-12-19T18:18:27.160473+0000","last_fullsized":"2023-12-19T18:18:27.160473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:29:18.711127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.158338+0000","last_change":"2023-12-19T18:18:25.645696+0000","last_active":"2023-12-19T18:18:27.158338+0000","last_peered":"2023-12-19T18:18:27.158338+0000","last_clean":"2023-12-19T18:18:27.158338+0000","last_became_active":"2023-12-19T18:18:25.642985+0000","last_became_peered":"2023-12-19T18:18:25.642985+0000","last_unstale":"2023-12-19T18:18:27.158338+0000","last_undegraded":"2023-12-19T18:18:27.158338+0000","last_fullsized":"2023-12-19T18:18:27.158338+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:21:15.067174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.157897+0000","last_change":"2023-12-19T18:18:26.181049+0000","last_active":"2023-12-19T18:18:27.157897+0000","last_peered":"2023-12-19T18:18:27.157897+0000","last_clean":"2023-12-19T18:18:27.157897+0000","last_became_active":"2023-12-19T18:18:26.179752+0000","last_became_peered":"2023-12-19T18:18:26.179752+0000","last_unstale":"2023-12-19T18:18:27.157897+0000","last_undegraded":"2023-12-19T18:18:27.157897+0000","last_fullsized":"2023-12-19T18:18:27.157897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:25.614882+0000","last_clean_scrub_stamp":"2023-12-19T18:18:25.614882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:46:15.147936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":75,"state":"active+clean","last_fresh":"2023-12-19T18:18:27.404985+0000","last_change":"2023-12-19T18:18:25.649687+0000","last_active":"2023-12-19T18:18:27.404985+0000","last_peered":"2023-12-19T18:18:27.404985+0000","last_clean":"2023-12-19T18:18:27.404985+0000","last_became_active":"2023-12-19T18:18:25.648972+0000","last_became_peered":"2023-12-19T18:18:25.648972+0000","last_unstale":"2023-12-19T18:18:27.404985+0000","last_undegraded":"2023-12-19T18:18:27.404985+0000","last_fullsized":"2023-12-19T18:18:27.404985+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":72,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:18:24.600262+0000","last_clean_scrub_stamp":"2023-12-19T18:18:24.600262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:42:00.156430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":212,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-12-19T18:18:26.161329+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:18:26.161329+0000","last_peered":"2023-12-19T18:18:26.161329+0000","last_clean":"2023-12-19T18:18:26.161329+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:18:26.161329+0000","last_undegraded":"2023-12-19T18:18:26.161329+0000","last_fullsized":"2023-12-19T18:18:26.161329+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":386,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194926,"statfs":{"total":200000000,"available":199604642,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.135,"5min":1.123,"15min":1.1120000000000001},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.645,"5min":1.645,"15min":1.645},"last":0.70199999999999996},{"interface":"front","average":{"1min":1.2090000000000001,"5min":1.1619999999999999,"15min":1.1479999999999999},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.79,"5min":1.79,"15min":1.79},"last":0.73499999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.6659999999999999,"5min":1.2749999999999999,"15min":1.2030000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":10.926,"5min":10.926,"15min":10.926},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.706,"5min":1.2789999999999999,"15min":1.22},"min":{"1min":0.79700000000000004,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":10.904999999999999,"5min":10.904999999999999,"15min":10.904999999999999},"last":1.077}]},{"osd":2,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.075,"15min":1.079},"min":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.6970000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":1.224,"5min":1.175,"15min":1.159},"min":{"1min":0.77100000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.581,"5min":1.7629999999999999,"15min":1.7629999999999999},"last":0.95799999999999996}]},{"osd":3,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1299999999999999,"15min":1.1060000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":1.014},{"interface":"front","average":{"1min":1.3520000000000001,"5min":1.276,"15min":1.236},"min":{"1min":0.749,"5min":0.748,"15min":0.748},"max":{"1min":1.7529999999999999,"5min":3.0110000000000001,"15min":3.0110000000000001},"last":1.1140000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1599999999999999,"5min":1.0940000000000001,"15min":1.0589999999999999},"min":{"1min":0.65400000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.714,"5min":1.714,"15min":1.714},"last":1.179},{"interface":"front","average":{"1min":1.218,"5min":1.2170000000000001,"15min":1.2090000000000001},"min":{"1min":0.72999999999999998,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.8180000000000001,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.2609999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3500000000000001,"5min":1.3260000000000001,"15min":1.2989999999999999},"min":{"1min":0.749,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.7789999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":1.23},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.298,"15min":1.288},"min":{"1min":0.71599999999999997,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.8460000000000001,"5min":2.754,"15min":2.754},"last":1.3089999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.444,"15min":1.421},"min":{"1min":0.78600000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.909,"5min":3.0720000000000001,"15min":3.0720000000000001},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.4339999999999999,"5min":1.359,"15min":1.329},"min":{"1min":0.92300000000000004,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.9390000000000001,"5min":2.149,"15min":2.149},"last":1.53}]}]},{"osd":6,"up_from":13,"seq":55834574895,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":386,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194926,"statfs":{"total":200000000,"available":199604712,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.014,"15min":0.96799999999999997},"min":{"1min":0.66700000000000004,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":1.141,"5min":1.0549999999999999,"15min":1.024},"min":{"1min":0.71199999999999997,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.498,"5min":1.573,"15min":1.573},"last":0.73699999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.1339999999999999,"15min":1.1240000000000001},"min":{"1min":0.74299999999999999,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":4.5540000000000003,"5min":4.5540000000000003,"15min":4.5540000000000003},"last":1.294},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.151,"15min":1.1479999999999999},"min":{"1min":0.86399999999999999,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.776,"5min":1.776,"15min":1.776},"last":1.6950000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2929999999999999,"5min":1.149,"15min":1.109},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.821,"5min":1.821,"15min":1.821},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.208,"15min":1.234},"min":{"1min":0.82599999999999996,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.5189999999999999,"5min":2.8340000000000001,"15min":2.8340000000000001},"last":1.24}]},{"osd":3,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.1819999999999999,"15min":1.161},"min":{"1min":0.93899999999999995,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":1.5620000000000001,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":1.137},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.1870000000000001,"15min":1.159},"min":{"1min":0.78300000000000003,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":1.524}]},{"osd":4,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.262,"15min":1.236},"min":{"1min":0.89300000000000002,"5min":0.74199999999999999,"15min":0.74199999999999999},"max":{"1min":3.0680000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.768},{"interface":"front","average":{"1min":1.464,"5min":1.3520000000000001,"15min":1.3120000000000001},"min":{"1min":1.0589999999999999,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":2.859,"5min":2.859,"15min":2.859},"last":1.413}]},{"osd":5,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.2,"15min":1.1539999999999999},"min":{"1min":0.88900000000000001,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":4.6680000000000001,"5min":4.6680000000000001,"15min":4.6680000000000001},"last":1.5980000000000001},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.361,"15min":1.3220000000000001},"min":{"1min":1.0169999999999999,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":3.3580000000000001,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.8169999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:17:51 2023","interfaces":[{"interface":"back","average":{"1min":1.4370000000000001,"5min":1.339,"15min":1.3080000000000001},"min":{"1min":1.012,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.0699999999999998,"5min":2.0699999999999998,"15min":2.0699999999999998},"last":1.633},{"interface":"front","average":{"1min":1.6140000000000001,"5min":1.3819999999999999,"15min":1.329},"min":{"1min":1.05,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":5.6639999999999997,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":1.6539999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":962,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194349,"statfs":{"total":200000000,"available":199014332,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.149,"15min":1.137},"min":{"1min":0.67200000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.837,"5min":1.863,"15min":1.863},"last":1.167},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.179,"15min":1.147},"min":{"1min":0.75,"5min":0.75,"15min":0.75},"max":{"1min":1.694,"5min":1.694,"15min":1.694},"last":1.117}]},{"osd":1,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.0309999999999999,"15min":1},"min":{"1min":0.70899999999999996,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6519999999999999,"5min":1.6519999999999999,"15min":1.6519999999999999},"last":1.02},{"interface":"front","average":{"1min":1.1240000000000001,"5min":1.054,"15min":1.0229999999999999},"min":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.5680000000000001,"5min":1.843,"15min":1.843},"last":1.3300000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.0860000000000001,"15min":1.0529999999999999},"min":{"1min":0.78200000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.1359999999999999,"15min":1.107},"min":{"1min":0.95199999999999996,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":1.4199999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.087,"15min":1.052},"min":{"1min":0.85699999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.077},{"interface":"front","average":{"1min":1.351,"5min":1.232,"15min":1.1859999999999999},"min":{"1min":0.75700000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.274}]},{"osd":4,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.194,"15min":1.127},"min":{"1min":0.65200000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.1779999999999999,"5min":1.1220000000000001,"15min":1.1000000000000001},"min":{"1min":0.73999999999999999,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.637,"5min":1.901,"15min":1.901},"last":1.1499999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.4770000000000001,"5min":1.3540000000000001,"15min":1.296},"min":{"1min":0.79500000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":2.1589999999999998,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.482},{"interface":"front","average":{"1min":1.29,"5min":1.238,"15min":1.214},"min":{"1min":0.91400000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.7949999999999999,"5min":1.976,"15min":1.976},"last":1.2270000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:17:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.2270000000000001,"15min":1.2},"min":{"1min":0.81899999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.4950000000000001,"5min":2.0030000000000001,"15min":2.0030000000000001},"last":1.3899999999999999},{"interface":"front","average":{"1min":1.421,"5min":1.407,"15min":1.389},"min":{"1min":0.84999999999999998,"5min":0.623,"15min":0.623},"max":{"1min":1.9850000000000001,"5min":1.9850000000000001,"15min":1.9850000000000001},"last":1.304}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":386,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194925,"statfs":{"total":200000000,"available":199604118,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.0049999999999999,"15min":1},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.579,"5min":1.579,"15min":1.579},"last":1.2270000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":1.0289999999999999,"15min":1.0249999999999999},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.3360000000000001,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":1.296}]},{"osd":1,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.1479999999999999,"15min":1.145},"min":{"1min":0.76900000000000002,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.6950000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.208,"5min":1.175,"15min":1.151},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.6479999999999999,"5min":1.696,"15min":1.696},"last":1.353}]},{"osd":2,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1719999999999999,"5min":1.1200000000000001,"15min":1.121},"min":{"1min":0.85799999999999998,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.194,"5min":1.099,"15min":1.083},"min":{"1min":0.85199999999999998,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.99299999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.2649999999999999,"15min":1.2470000000000001},"min":{"1min":0.94899999999999995,"5min":0.83499999999999996,"15min":0.83499999999999996},"max":{"1min":1.627,"5min":1.645,"15min":1.645},"last":1.526},{"interface":"front","average":{"1min":1.248,"5min":1.2869999999999999,"15min":1.2849999999999999},"min":{"1min":0.82399999999999995,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.5640000000000001,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.155}]},{"osd":5,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.2,"15min":1.167},"min":{"1min":1.0369999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.76,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.093},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.3540000000000001,"15min":1.3240000000000001},"min":{"1min":0.85999999999999999,"5min":0.504,"15min":0.504},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.5860000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":1.26,"5min":1.292,"15min":1.294},"min":{"1min":0.78100000000000003,"5min":0.432,"15min":0.432},"max":{"1min":1.7809999999999999,"5min":1.853,"15min":1.853},"last":1.4490000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.3839999999999999,"15min":1.3660000000000001},"min":{"1min":1.2410000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.4099999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:17:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.353,"15min":1.327},"min":{"1min":0.98099999999999998,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.7889999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.3959999999999999,"15min":1.3939999999999999},"min":{"1min":0.98899999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.835,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.5469999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574896,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":386,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194926,"statfs":{"total":200000000,"available":199604644,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.95499999999999996,"15min":0.97999999999999998},"min":{"1min":0.51800000000000002,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.5449999999999999,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.93200000000000005,"15min":0.92100000000000004},"min":{"1min":0.437,"5min":0.434,"15min":0.434},"max":{"1min":1.8140000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.115,"5min":1.042,"15min":1.0720000000000001},"min":{"1min":0.67800000000000005,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.238,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.0780000000000001,"15min":1.0960000000000001},"min":{"1min":0.68999999999999995,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.901,"5min":1.901,"15min":1.901},"last":0.53400000000000003}]},{"osd":2,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.9330000000000001,"5min":1.302,"15min":1.2430000000000001},"min":{"1min":0.58299999999999996,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":13.833,"5min":13.833,"15min":13.833},"last":0.60799999999999998},{"interface":"front","average":{"1min":2.1280000000000001,"5min":1.3680000000000001,"15min":1.2569999999999999},"min":{"1min":0.74099999999999999,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":16.748999999999999,"5min":16.748999999999999,"15min":16.748999999999999},"last":0.61699999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.3919999999999999,"5min":1.212,"15min":1.1839999999999999},"min":{"1min":0.80100000000000005,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":2.1190000000000002,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.3959999999999999,"15min":1.4099999999999999},"min":{"1min":0.96899999999999997,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.5710000000000002,"5min":4.0209999999999999,"15min":4.0209999999999999},"last":0.80800000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.4279999999999999,"15min":1.446},"min":{"1min":0.876,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":2.1539999999999999,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":1.1739999999999999},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.2609999999999999,"15min":1.242},"min":{"1min":0.874,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.8300000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.95899999999999996}]},{"osd":6,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.4990000000000001,"5min":1.462,"15min":1.488},"min":{"1min":0.82299999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":2.4790000000000001,"5min":3.9399999999999999,"15min":3.9399999999999999},"last":0.91200000000000003},{"interface":"front","average":{"1min":1.556,"5min":1.4870000000000001,"15min":1.506},"min":{"1min":1.0169999999999999,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.0649999999999999,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.1419999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:17:46 2023","interfaces":[{"interface":"back","average":{"1min":1.504,"5min":1.367,"15min":1.351},"min":{"1min":0.89100000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.966,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.6479999999999999,"5min":1.54,"15min":1.55},"min":{"1min":1.0489999999999999,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.657,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":0.94199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574895,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":387,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194924,"statfs":{"total":200000000,"available":199603010,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.115,"15min":1.0349999999999999},"min":{"1min":0.65000000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":9.2550000000000008,"5min":9.2550000000000008,"15min":9.2550000000000008},"last":0.752},{"interface":"front","average":{"1min":1.087,"5min":1.03,"15min":1.046},"min":{"1min":0.68300000000000005,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.0800000000000001,"5min":4.7850000000000001,"15min":4.7850000000000001},"last":0.82099999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":0.94399999999999995,"5min":1.155,"15min":1.256},"min":{"1min":0.61099999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.3480000000000001,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":0.68500000000000005},{"interface":"front","average":{"1min":1.115,"5min":1.1499999999999999,"15min":1.1910000000000001},"min":{"1min":0.623,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.528,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":0.98899999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.3480000000000001,"15min":1.45},"min":{"1min":0.54200000000000004,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6779999999999999,"5min":8.3109999999999999,"15min":8.3109999999999999},"last":1.139},{"interface":"front","average":{"1min":1.2010000000000001,"5min":1.3200000000000001,"15min":1.4059999999999999},"min":{"1min":0.69299999999999995,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":2.0059999999999998,"5min":8.0690000000000008,"15min":8.0690000000000008},"last":1.0369999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.454,"5min":1.4490000000000001,"15min":1.4870000000000001},"min":{"1min":0.98199999999999998,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.976,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":1.1699999999999999},{"interface":"front","average":{"1min":1.401,"5min":1.5089999999999999,"15min":1.5860000000000001},"min":{"1min":0.96099999999999997,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":2.016,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":1.7969999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4590000000000001,"5min":1.544,"15min":1.6200000000000001},"min":{"1min":0.80000000000000004,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":2.1629999999999998,"5min":8.1839999999999993,"15min":8.1839999999999993},"last":1.222},{"interface":"front","average":{"1min":1.4330000000000001,"5min":1.4059999999999999,"15min":1.4179999999999999},"min":{"1min":1.0089999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.3159999999999998,"5min":5.165,"15min":5.165},"last":1.6439999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.4419999999999999,"15min":1.464},"min":{"1min":0.95999999999999996,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.9399999999999999,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":1.915},{"interface":"front","average":{"1min":1.5249999999999999,"5min":1.5569999999999999,"15min":1.609},"min":{"1min":0.89200000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.2850000000000001,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":1.2789999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:17:47 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.5369999999999999,"15min":1.5680000000000001},"min":{"1min":0.91300000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":2.0640000000000001,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":2.1040000000000001},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.6559999999999999,"15min":1.714},"min":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":2.3719999999999999,"5min":8.2300000000000004,"15min":8.2300000000000004},"last":1.3280000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574896,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":963,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194349,"statfs":{"total":200000000,"available":199013784,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83599999999999997,"15min":0.84399999999999997},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.359,"5min":1.359,"15min":1.359},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.877,"5min":0.93799999999999994,"15min":0.96099999999999997},"min":{"1min":0.56399999999999995,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.3280000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":0.82599999999999996}]},{"osd":2,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":1.038,"15min":1.056},"min":{"1min":0.47399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.4970000000000001,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.97899999999999998},{"interface":"front","average":{"1min":0.88400000000000001,"5min":0.89500000000000002,"15min":0.89000000000000001},"min":{"1min":0.51400000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.264,"5min":1.597,"15min":1.597},"last":0.90900000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":1.1359999999999999,"5min":1.123,"15min":1.105},"min":{"1min":0.65500000000000003,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.575,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.151,"5min":1.171,"15min":1.181},"min":{"1min":0.63500000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.8300000000000001,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.1759999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.7589999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.2789999999999999,"15min":1.2669999999999999},"min":{"1min":0.93899999999999995,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.796,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.339}]},{"osd":5,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":1.403,"5min":1.286,"15min":1.2609999999999999},"min":{"1min":1.0009999999999999,"5min":0.88400000000000001,"15min":0.88400000000000001},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":1.073},{"interface":"front","average":{"1min":1.45,"5min":1.3979999999999999,"15min":1.3859999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":2.0790000000000002,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.4850000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.365,"15min":1.361},"min":{"1min":1.0289999999999999,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":1.7729999999999999,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.302},{"interface":"front","average":{"1min":1.4590000000000001,"5min":1.4330000000000001,"15min":1.4219999999999999},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":2.004,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.4019999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:17:48 2023","interfaces":[{"interface":"back","average":{"1min":1.8740000000000001,"5min":1.4339999999999999,"15min":1.345},"min":{"1min":0.879,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":9.4830000000000005,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":1.536},{"interface":"front","average":{"1min":1.992,"5min":1.544,"15min":1.47},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":9.4109999999999996,"5min":9.4109999999999996,"15min":9.4109999999999996},"last":1.256}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":387,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194924,"statfs":{"total":200000000,"available":199603170,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.92600000000000005,"5min":0.88600000000000001,"15min":0.86399999999999999},"min":{"1min":0.46700000000000003,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.617,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.79100000000000004},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0029999999999999,"15min":0.996},"min":{"1min":0.52400000000000002,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.6180000000000001,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":0.92800000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":0.93899999999999995,"15min":0.92800000000000005},"min":{"1min":0.69499999999999995,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.6279999999999999,"5min":1.6279999999999999,"15min":1.6279999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.024,"5min":0.92600000000000005,"15min":0.90900000000000003},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.7810000000000001,"5min":2.7810000000000001,"15min":2.7810000000000001},"last":0.65200000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.96999999999999997,"15min":0.98599999999999999},"min":{"1min":0.54800000000000004,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.583,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.264,"5min":1.177,"15min":1.1890000000000001},"min":{"1min":0.63500000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.73,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.85599999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.1890000000000001,"15min":1.157},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.903,"5min":1.903,"15min":1.903},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2529999999999999,"15min":1.2709999999999999},"min":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.9930000000000001,"5min":2.9300000000000002,"15min":2.9300000000000002},"last":1.5449999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":2.089,"5min":1.4670000000000001,"15min":1.389},"min":{"1min":0.71099999999999997,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":16.298999999999999,"5min":16.298999999999999,"15min":16.298999999999999},"last":1.423},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.262,"15min":1.2450000000000001},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.079}]},{"osd":6,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":1.385,"5min":1.3660000000000001,"15min":1.3640000000000001},"min":{"1min":0.85499999999999998,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.948,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.353,"15min":1.325},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.46,"5min":2.46,"15min":2.46},"last":1.1870000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:17:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4610000000000001,"5min":1.383,"15min":1.3500000000000001},"min":{"1min":0.89200000000000002,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.1560000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.3999999999999999,"15min":1.4179999999999999},"min":{"1min":0.873,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.1360000000000001,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":1.2350000000000001}]}]}],"pool_statfs":[]}} 2023-12-19T18:18:31.696 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:18:31.697 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:18:31.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.982+0000 7f05acf49640 1 Processor -- start 2023-12-19T18:18:31.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.983+0000 7f05acf49640 1 -- start start 2023-12-19T18:18:31.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.984+0000 7f05acf49640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f05a80d0d60 0x7f05a80cf770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:31.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.984+0000 7f05acf49640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05a80cfcb0 0x7f05a80b17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:31.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.984+0000 7f05acf49640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a80d2bc0 con 0x7f05a815ce50 2023-12-19T18:18:31.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.984+0000 7f05acf49640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05a80d0100 con 0x7f05a80d0d60 2023-12-19T18:18:31.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.984+0000 7f05acf49640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80d1ee0 con 0x7f05a80cfcb0 2023-12-19T18:18:31.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.985+0000 7f059ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f05a80d0d60 0x7f05a80cf770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:31.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.985+0000 7f059ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f05a80d0d60 0x7f05a80cf770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34242/0 (socket says 172.21.15.27:34242) 2023-12-19T18:18:31.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.985+0000 7f059ffff640 1 -- 172.21.15.27:0/786160776 learned_addr learned my addr 172.21.15.27:0/786160776 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:31.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.985+0000 7f05a7fff640 1 --2- 172.21.15.27:0/786160776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05a80cfcb0 0x7f05a80b17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:31.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.985+0000 7f059ffff640 1 -- 172.21.15.27:0/786160776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05a80cfcb0 msgr2=0x7f05a80b17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:31.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.985+0000 7f059ffff640 1 --2- 172.21.15.27:0/786160776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05a80cfcb0 0x7f05a80b17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.985+0000 7f059ffff640 1 -- 172.21.15.27:0/786160776 >> v1:172.21.15.192:6789/0 conn(0x7f05a815ce50 legacy=0x7f05a80d24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:31.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.985+0000 7f059ffff640 1 -- 172.21.15.27:0/786160776 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80d12d0 con 0x7f05a80d0d60 2023-12-19T18:18:31.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.985+0000 7f05a7fff640 1 --2- 172.21.15.27:0/786160776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05a80cfcb0 0x7f05a80b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.986+0000 7f059ffff640 1 --2- 172.21.15.27:0/786160776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a80d0d60 0x7f05a80cf770 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0590009870 tx=0x7f059002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a008ecda53cab65c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.987+0000 7f05a6ffd640 1 -- 172.21.15.27:0/786160776 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590031640 con 0x7f05a80d0d60 2023-12-19T18:18:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.987+0000 7f05a6ffd640 1 -- 172.21.15.27:0/786160776 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05900317a0 con 0x7f05a80d0d60 2023-12-19T18:18:31.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.987+0000 7f05a6ffd640 1 -- 172.21.15.27:0/786160776 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590031b30 con 0x7f05a80d0d60 2023-12-19T18:18:31.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.987+0000 7f05acf49640 1 -- 172.21.15.27:0/786160776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a80d0d60 msgr2=0x7f05a80cf770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:31.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.988+0000 7f05acf49640 1 --2- 172.21.15.27:0/786160776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a80d0d60 0x7f05a80cf770 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f0590009870 tx=0x7f059002b8f0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.988+0000 7f05acf49640 1 -- 172.21.15.27:0/786160776 shutdown_connections 2023-12-19T18:18:31.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.988+0000 7f05acf49640 1 --2- 172.21.15.27:0/786160776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05a80cfcb0 0x7f05a80b17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.988+0000 7f05acf49640 1 --2- 172.21.15.27:0/786160776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a80d0d60 0x7f05a80cf770 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.988+0000 7f05acf49640 1 -- 172.21.15.27:0/786160776 >> 172.21.15.27:0/786160776 conn(0x7f05a806a2d0 msgr2=0x7f05a806a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:31.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.988+0000 7f05acf49640 1 -- 172.21.15.27:0/786160776 shutdown_connections 2023-12-19T18:18:31.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.988+0000 7f05acf49640 1 -- 172.21.15.27:0/786160776 wait complete. 2023-12-19T18:18:31.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.990+0000 7f05acf49640 1 Processor -- start 2023-12-19T18:18:31.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.990+0000 7f05acf49640 1 -- start start 2023-12-19T18:18:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.991+0000 7f05acf49640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05a80cfcb0 0x7f05a81f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.991+0000 7f05acf49640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a80d0d60 0x7f05a815dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.991+0000 7f05acf49640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a815ce50 0x7f05a81f1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.991+0000 7f05acf49640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a81f6980 con 0x7f05a80d0d60 2023-12-19T18:18:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.991+0000 7f05acf49640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8156df0 con 0x7f05a815ce50 2023-12-19T18:18:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.991+0000 7f05acf49640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8157090 con 0x7f05a80cfcb0 2023-12-19T18:18:31.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.991+0000 7f059ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a80d0d60 0x7f05a815dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:31.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.991+0000 7f059ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a80d0d60 0x7f05a815dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42464/0 (socket says 172.21.15.27:42464) 2023-12-19T18:18:31.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.991+0000 7f059ffff640 1 -- 172.21.15.27:0/4011720300 learned_addr learned my addr 172.21.15.27:0/4011720300 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:31.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.991+0000 7f05a77fe640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05a80cfcb0 0x7f05a81f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:31.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.992+0000 7f05a7fff640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a815ce50 0x7f05a81f1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:31.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.992+0000 7f059ffff640 1 -- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05a80cfcb0 msgr2=0x7f05a81f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:31.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.992+0000 7f059ffff640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05a80cfcb0 0x7f05a81f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.992+0000 7f059ffff640 1 -- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a815ce50 msgr2=0x7f05a81f1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:31.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.992+0000 7f059ffff640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a815ce50 0x7f05a81f1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:31.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.992+0000 7f059ffff640 1 -- 172.21.15.27:0/4011720300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05900094d0 con 0x7f05a80d0d60 2023-12-19T18:18:31.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.992+0000 7f05a7fff640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a815ce50 0x7f05a81f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:31.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.992+0000 7f059ffff640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a80d0d60 0x7f05a815dcd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0590009870 tx=0x7f05900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:31.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.992+0000 7f05a4ff9640 1 -- 172.21.15.27:0/4011720300 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059003ac60 con 0x7f05a80d0d60 2023-12-19T18:18:31.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.992+0000 7f05a4ff9640 1 -- 172.21.15.27:0/4011720300 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059003ade0 con 0x7f05a80d0d60 2023-12-19T18:18:31.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.992+0000 7f05a4ff9640 1 -- 172.21.15.27:0/4011720300 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0590044630 con 0x7f05a80d0d60 2023-12-19T18:18:31.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.992+0000 7f05acf49640 1 -- 172.21.15.27:0/4011720300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a8157370 con 0x7f05a80d0d60 2023-12-19T18:18:31.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.993+0000 7f05acf49640 1 -- 172.21.15.27:0/4011720300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a8157930 con 0x7f05a80d0d60 2023-12-19T18:18:31.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.994+0000 7f05acf49640 1 -- 172.21.15.27:0/4011720300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a80b8d70 con 0x7f05a80d0d60 2023-12-19T18:18:31.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.995+0000 7f05a4ff9640 1 -- 172.21.15.27:0/4011720300 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f059002f050 con 0x7f05a80d0d60 2023-12-19T18:18:31.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.996+0000 7f05a4ff9640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f058806a430 0x7f058806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:31.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.996+0000 7f05a4ff9640 1 -- 172.21.15.27:0/4011720300 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05900aa960 con 0x7f05a80d0d60 2023-12-19T18:18:31.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.996+0000 7f05a77fe640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f058806a430 0x7f058806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:31.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:31.998+0000 7f05a77fe640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f058806a430 0x7f058806c8f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0598006fd0 tx=0x7f0598008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:32.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.000+0000 7f05a4ff9640 1 -- 172.21.15.27:0/4011720300 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0590076000 con 0x7f05a80d0d60 2023-12-19T18:18:32.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.265+0000 7f05acf49640 1 -- 172.21.15.27:0/4011720300 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f05a80d2bc0 con 0x7f05a80d0d60 2023-12-19T18:18:32.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.274+0000 7f05a4ff9640 1 -- 172.21.15.27:0/4011720300 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f05a80d2bc0 con 0x7f05a80d0d60 2023-12-19T18:18:32.274 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2023-12-19T18:18:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.277+0000 7f05acf49640 1 -- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f058806a430 msgr2=0x7f058806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.277+0000 7f05acf49640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f058806a430 0x7f058806c8f0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f0598006fd0 tx=0x7f0598008040 comp rx=0 tx=0).stop 2023-12-19T18:18:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.277+0000 7f05acf49640 1 -- 172.21.15.27:0/4011720300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a80d0d60 msgr2=0x7f05a815dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.277+0000 7f05acf49640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a80d0d60 0x7f05a815dcd0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f0590009870 tx=0x7f05900046c0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.277+0000 7f05acf49640 1 -- 172.21.15.27:0/4011720300 shutdown_connections 2023-12-19T18:18:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.277+0000 7f05acf49640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f058806a430 0x7f058806c8f0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.277+0000 7f05acf49640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05a815ce50 0x7f05a81f1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.277+0000 7f05acf49640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05a80d0d60 0x7f05a815dcd0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.277+0000 7f05acf49640 1 --2- 172.21.15.27:0/4011720300 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05a80cfcb0 0x7f05a81f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.277+0000 7f05acf49640 1 -- 172.21.15.27:0/4011720300 >> 172.21.15.27:0/4011720300 conn(0x7f05a806a2d0 msgr2=0x7f05a815fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.277+0000 7f05acf49640 1 -- 172.21.15.27:0/4011720300 shutdown_connections 2023-12-19T18:18:32.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.277+0000 7f05acf49640 1 -- 172.21.15.27:0/4011720300 wait complete. 2023-12-19T18:18:32.295 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:32.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.521+0000 7f356d7ef640 1 Processor -- start 2023-12-19T18:18:32.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.522+0000 7f356d7ef640 1 -- start start 2023-12-19T18:18:32.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.523+0000 7f356d7ef640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3568069c70 0x7f3568070450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:32.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.523+0000 7f356d7ef640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f356815d220 0x7f356806a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:32.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.523+0000 7f356d7ef640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35680ca280 con 0x7f35680c60d0 2023-12-19T18:18:32.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.524+0000 7f356d7ef640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35680714d0 con 0x7f3568069c70 2023-12-19T18:18:32.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.524+0000 7f356d7ef640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3568070990 con 0x7f356815d220 2023-12-19T18:18:32.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.524+0000 7f355f7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f356815d220 0x7f356806a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:32.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.524+0000 7f355f7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f356815d220 0x7f356806a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:55452/0 (socket says 172.21.15.27:55452) 2023-12-19T18:18:32.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.524+0000 7f355f7fe640 1 -- 172.21.15.27:0/3197625796 learned_addr learned my addr 172.21.15.27:0/3197625796 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:32.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.524+0000 7f3567fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3568069c70 0x7f3568070450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:32.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.525+0000 7f355f7fe640 1 -- 172.21.15.27:0/3197625796 >> v2:172.21.15.192:3300/0 conn(0x7f3568069c70 msgr2=0x7f3568070450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:32.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.525+0000 7f355f7fe640 1 --2- 172.21.15.27:0/3197625796 >> v2:172.21.15.192:3300/0 conn(0x7f3568069c70 0x7f3568070450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.525+0000 7f355f7fe640 1 -- 172.21.15.27:0/3197625796 >> v1:172.21.15.192:6789/0 conn(0x7f35680c60d0 legacy=0x7f35680c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:32.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.525+0000 7f355f7fe640 1 -- 172.21.15.27:0/3197625796 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356806a590 con 0x7f356815d220 2023-12-19T18:18:32.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.526+0000 7f355f7fe640 1 --2- 172.21.15.27:0/3197625796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f356815d220 0x7f356806a050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3550009870 tx=0x7f355002b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70836b5b45ee969b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:32.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.527+0000 7f35677fe640 1 -- 172.21.15.27:0/3197625796 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550031640 con 0x7f356815d220 2023-12-19T18:18:32.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.527+0000 7f35677fe640 1 -- 172.21.15.27:0/3197625796 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35500317a0 con 0x7f356815d220 2023-12-19T18:18:32.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.527+0000 7f35677fe640 1 -- 172.21.15.27:0/3197625796 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3550031b30 con 0x7f356815d220 2023-12-19T18:18:32.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.528+0000 7f356d7ef640 1 -- 172.21.15.27:0/3197625796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f356815d220 msgr2=0x7f356806a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:32.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.528+0000 7f356d7ef640 1 --2- 172.21.15.27:0/3197625796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f356815d220 0x7f356806a050 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f3550009870 tx=0x7f355002b8f0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.528+0000 7f356d7ef640 1 -- 172.21.15.27:0/3197625796 shutdown_connections 2023-12-19T18:18:32.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.528+0000 7f356d7ef640 1 --2- 172.21.15.27:0/3197625796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f356815d220 0x7f356806a050 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.528+0000 7f356d7ef640 1 --2- 172.21.15.27:0/3197625796 >> v2:172.21.15.192:3300/0 conn(0x7f3568069c70 0x7f3568070450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.528+0000 7f356d7ef640 1 -- 172.21.15.27:0/3197625796 >> 172.21.15.27:0/3197625796 conn(0x7f35680cfa70 msgr2=0x7f35680cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:32.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.528+0000 7f356d7ef640 1 -- 172.21.15.27:0/3197625796 shutdown_connections 2023-12-19T18:18:32.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.528+0000 7f356d7ef640 1 -- 172.21.15.27:0/3197625796 wait complete. 2023-12-19T18:18:32.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.530+0000 7f356d7ef640 1 Processor -- start 2023-12-19T18:18:32.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.530+0000 7f356d7ef640 1 -- start start 2023-12-19T18:18:32.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.531+0000 7f356d7ef640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3568069c70 0x7f35680ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:32.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.531+0000 7f356d7ef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680c60d0 0x7f35681f3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:32.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.531+0000 7f356d7ef640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f356815d220 0x7f356815a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:32.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.531+0000 7f356d7ef640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356815f2c0 con 0x7f35680c60d0 2023-12-19T18:18:32.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.531+0000 7f356d7ef640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f356814ce10 con 0x7f356815d220 2023-12-19T18:18:32.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.531+0000 7f356d7ef640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35680aea00 con 0x7f3568069c70 2023-12-19T18:18:32.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.531+0000 7f355f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680c60d0 0x7f35681f3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:32.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.531+0000 7f355f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680c60d0 0x7f35681f3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42470/0 (socket says 172.21.15.27:42470) 2023-12-19T18:18:32.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.531+0000 7f355f7fe640 1 -- 172.21.15.27:0/718373968 learned_addr learned my addr 172.21.15.27:0/718373968 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:32.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.532+0000 7f3567fff640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3568069c70 0x7f35680ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:32.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.532+0000 7f356cfee640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f356815d220 0x7f356815a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:32.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.532+0000 7f355f7fe640 1 -- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3568069c70 msgr2=0x7f35680ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:32.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.532+0000 7f355f7fe640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3568069c70 0x7f35680ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.532+0000 7f355f7fe640 1 -- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f356815d220 msgr2=0x7f356815a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:32.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.532+0000 7f355f7fe640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f356815d220 0x7f356815a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.532+0000 7f355f7fe640 1 -- 172.21.15.27:0/718373968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35500094d0 con 0x7f35680c60d0 2023-12-19T18:18:32.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.532+0000 7f356cfee640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f356815d220 0x7f356815a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:32.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.532+0000 7f355f7fe640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680c60d0 0x7f35681f3830 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f35500090b0 tx=0x7f35500046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:32.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.533+0000 7f35657fa640 1 -- 172.21.15.27:0/718373968 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f355003a9f0 con 0x7f35680c60d0 2023-12-19T18:18:32.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.533+0000 7f35657fa640 1 -- 172.21.15.27:0/718373968 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f355003ab90 con 0x7f35680c60d0 2023-12-19T18:18:32.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.533+0000 7f356d7ef640 1 -- 172.21.15.27:0/718373968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35680ae160 con 0x7f35680c60d0 2023-12-19T18:18:32.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.533+0000 7f35657fa640 1 -- 172.21.15.27:0/718373968 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35500443f0 con 0x7f35680c60d0 2023-12-19T18:18:32.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.533+0000 7f356d7ef640 1 -- 172.21.15.27:0/718373968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35680ae720 con 0x7f35680c60d0 2023-12-19T18:18:32.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.534+0000 7f356d7ef640 1 -- 172.21.15.27:0/718373968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f356804e840 con 0x7f35680c60d0 2023-12-19T18:18:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.535+0000 7f35657fa640 1 -- 172.21.15.27:0/718373968 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f355002f050 con 0x7f35680c60d0 2023-12-19T18:18:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.536+0000 7f35657fa640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f354806a430 0x7f354806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:32.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.536+0000 7f35657fa640 1 -- 172.21.15.27:0/718373968 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f35500aa750 con 0x7f35680c60d0 2023-12-19T18:18:32.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.536+0000 7f3567fff640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f354806a430 0x7f354806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:32.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.537+0000 7f3567fff640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f354806a430 0x7f354806c8f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3558009ad0 tx=0x7f3558008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:32.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.540+0000 7f35657fa640 1 -- 172.21.15.27:0/718373968 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3550075ef0 con 0x7f35680c60d0 2023-12-19T18:18:32.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.767+0000 7f356d7ef640 1 -- 172.21.15.27:0/718373968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35680c5860 con 0x7f35680c60d0 2023-12-19T18:18:32.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.769+0000 7f35657fa640 1 -- 172.21.15.27:0/718373968 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f3550079510 con 0x7f35680c60d0 2023-12-19T18:18:32.769 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:32.769 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1624358030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1624358030},{"type":"v1","addr":"172.21.15.192:6836","nonce":1624358030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5030,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3025339386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3025339386},{"type":"v1","addr":"172.21.15.192:6837","nonce":3025339386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/1685117850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1685117850},{"type":"v1","addr":"172.21.15.27:6833","nonce":1685117850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:18:26.147336+0000","modified":"2023-12-19T18:18:29.663526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2772294131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2772294131},{"type":"v1","addr":"172.21.15.27:6835","nonce":2772294131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-19T18:18:32.769 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2023-12-19T18:18:32.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.771+0000 7f356d7ef640 1 -- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f354806a430 msgr2=0x7f354806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:32.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.771+0000 7f356d7ef640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f354806a430 0x7f354806c8f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3558009ad0 tx=0x7f3558008040 comp rx=0 tx=0).stop 2023-12-19T18:18:32.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.771+0000 7f356d7ef640 1 -- 172.21.15.27:0/718373968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680c60d0 msgr2=0x7f35681f3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:32.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.771+0000 7f356d7ef640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680c60d0 0x7f35681f3830 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f35500090b0 tx=0x7f35500046c0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.771+0000 7f356d7ef640 1 -- 172.21.15.27:0/718373968 shutdown_connections 2023-12-19T18:18:32.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.771+0000 7f356d7ef640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f354806a430 0x7f354806c8f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.771+0000 7f356d7ef640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f356815d220 0x7f356815a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.771+0000 7f356d7ef640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35680c60d0 0x7f35681f3830 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.771+0000 7f356d7ef640 1 --2- 172.21.15.27:0/718373968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3568069c70 0x7f35680ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:32.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.771+0000 7f356d7ef640 1 -- 172.21.15.27:0/718373968 >> 172.21.15.27:0/718373968 conn(0x7f35680cfa70 msgr2=0x7f35680cd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:32.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.771+0000 7f356d7ef640 1 -- 172.21.15.27:0/718373968 shutdown_connections 2023-12-19T18:18:32.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:32.772+0000 7f356d7ef640 1 -- 172.21.15.27:0/718373968 wait complete. 2023-12-19T18:18:33.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:18:26.147336+0000', 'modified': '2023-12-19T18:18:29.663526+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6123}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6123': {'gid': 6123, 'name': 'c', 'rank': 0, 'incarnation': 27, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/2772294131', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6834', 'nonce': 2772294131}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 2772294131}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:18:33.193 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:18:33.193 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-19T18:18:33.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.437+0000 7fdf09def640 1 Processor -- start 2023-12-19T18:18:33.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.437+0000 7fdf09def640 1 -- start start 2023-12-19T18:18:33.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.439+0000 7fdf09def640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdf0406f8e0 0x7fdf0406fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.439+0000 7fdf09def640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdf04161a40 0x7fdf04161e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.439+0000 7fdf09def640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf0406b310 con 0x7fdf0415b9e0 2023-12-19T18:18:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.439+0000 7fdf09def640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf040702a0 con 0x7fdf04161a40 2023-12-19T18:18:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.439+0000 7fdf09def640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf04162790 con 0x7fdf0406f8e0 2023-12-19T18:18:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.439+0000 7fdf095ee640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdf04161a40 0x7fdf04161e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.439+0000 7fdf095ee640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdf04161a40 0x7fdf04161e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42484/0 (socket says 172.21.15.27:42484) 2023-12-19T18:18:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.439+0000 7fdf095ee640 1 -- 172.21.15.27:0/762335717 learned_addr learned my addr 172.21.15.27:0/762335717 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.440+0000 7fdf03fff640 1 --2- 172.21.15.27:0/762335717 >> v2:172.21.15.192:3300/0 conn(0x7fdf0406f8e0 0x7fdf0406fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.440+0000 7fdf095ee640 1 -- 172.21.15.27:0/762335717 >> v2:172.21.15.192:3300/0 conn(0x7fdf0406f8e0 msgr2=0x7fdf0406fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.440+0000 7fdf095ee640 1 --2- 172.21.15.27:0/762335717 >> v2:172.21.15.192:3300/0 conn(0x7fdf0406f8e0 0x7fdf0406fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.440+0000 7fdf095ee640 1 -- 172.21.15.27:0/762335717 >> v1:172.21.15.192:6789/0 conn(0x7fdf0415b9e0 legacy=0x7fdf040c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.440+0000 7fdf095ee640 1 -- 172.21.15.27:0/762335717 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0406b560 con 0x7fdf04161a40 2023-12-19T18:18:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.441+0000 7fdf095ee640 1 --2- 172.21.15.27:0/762335717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04161a40 0x7fdf04161e60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdf0406d020 tx=0x7fdefc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9dc4ee4d610356 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.441+0000 7fdf037fe640 1 -- 172.21.15.27:0/762335717 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc02ca80 con 0x7fdf04161a40 2023-12-19T18:18:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.441+0000 7fdf037fe640 1 -- 172.21.15.27:0/762335717 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc02cbe0 con 0x7fdf04161a40 2023-12-19T18:18:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.441+0000 7fdf037fe640 1 -- 172.21.15.27:0/762335717 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc0343f0 con 0x7fdf04161a40 2023-12-19T18:18:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.442+0000 7fdf09def640 1 -- 172.21.15.27:0/762335717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04161a40 msgr2=0x7fdf04161e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.442+0000 7fdf09def640 1 --2- 172.21.15.27:0/762335717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04161a40 0x7fdf04161e60 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fdf0406d020 tx=0x7fdefc02bee0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.442+0000 7fdf09def640 1 -- 172.21.15.27:0/762335717 shutdown_connections 2023-12-19T18:18:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.442+0000 7fdf09def640 1 --2- 172.21.15.27:0/762335717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf04161a40 0x7fdf04161e60 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.442+0000 7fdf09def640 1 --2- 172.21.15.27:0/762335717 >> v2:172.21.15.192:3300/0 conn(0x7fdf0406f8e0 0x7fdf0406fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.442+0000 7fdf09def640 1 -- 172.21.15.27:0/762335717 >> 172.21.15.27:0/762335717 conn(0x7fdf0406a7d0 msgr2=0x7fdf040ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.442+0000 7fdf09def640 1 -- 172.21.15.27:0/762335717 shutdown_connections 2023-12-19T18:18:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.442+0000 7fdf09def640 1 -- 172.21.15.27:0/762335717 wait complete. 2023-12-19T18:18:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.444+0000 7fdf09def640 1 Processor -- start 2023-12-19T18:18:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.444+0000 7fdf09def640 1 -- start start 2023-12-19T18:18:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.445+0000 7fdf09def640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0406f8e0 0x7fdf040c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.445+0000 7fdf09def640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf0415b9e0 0x7fdf0415a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.445+0000 7fdf09def640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf04161a40 0x7fdf040ac9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.445+0000 7fdf09def640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040c4b50 con 0x7fdf0406f8e0 2023-12-19T18:18:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.445+0000 7fdf09def640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040c3850 con 0x7fdf04161a40 2023-12-19T18:18:33.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.445+0000 7fdf09def640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040c3b30 con 0x7fdf0415b9e0 2023-12-19T18:18:33.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.446+0000 7fdf08ded640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0406f8e0 0x7fdf040c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.446+0000 7fdf08ded640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0406f8e0 0x7fdf040c4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42490/0 (socket says 172.21.15.27:42490) 2023-12-19T18:18:33.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.446+0000 7fdf08ded640 1 -- 172.21.15.27:0/2430543711 learned_addr learned my addr 172.21.15.27:0/2430543711 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:33.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.446+0000 7fdf095ee640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf04161a40 0x7fdf040ac9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.446+0000 7fdf03fff640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf0415b9e0 0x7fdf0415a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.446+0000 7fdf08ded640 1 -- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf0415b9e0 msgr2=0x7fdf0415a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.446+0000 7fdf08ded640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf0415b9e0 0x7fdf0415a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.446+0000 7fdf08ded640 1 -- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf04161a40 msgr2=0x7fdf040ac9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.446+0000 7fdf08ded640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf04161a40 0x7fdf040ac9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.446+0000 7fdf08ded640 1 -- 172.21.15.27:0/2430543711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdefc009b30 con 0x7fdf0406f8e0 2023-12-19T18:18:33.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.447+0000 7fdf08ded640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0406f8e0 0x7fdf040c4610 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdef400c430 tx=0x7fdef400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.447+0000 7fdf017fa640 1 -- 172.21.15.27:0/2430543711 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4011790 con 0x7fdf0406f8e0 2023-12-19T18:18:33.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.447+0000 7fdf017fa640 1 -- 172.21.15.27:0/2430543711 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef40118f0 con 0x7fdf0406f8e0 2023-12-19T18:18:33.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.447+0000 7fdf017fa640 1 -- 172.21.15.27:0/2430543711 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef4011ce0 con 0x7fdf0406f8e0 2023-12-19T18:18:33.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.447+0000 7fdf09def640 1 -- 172.21.15.27:0/2430543711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf040c3e10 con 0x7fdf0406f8e0 2023-12-19T18:18:33.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.447+0000 7fdf09def640 1 -- 172.21.15.27:0/2430543711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf040c85c0 con 0x7fdf0406f8e0 2023-12-19T18:18:33.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.449+0000 7fdf017fa640 1 -- 172.21.15.27:0/2430543711 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdef401c460 con 0x7fdf0406f8e0 2023-12-19T18:18:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.450+0000 7fdf017fa640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdef006a430 0x7fdef006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.451+0000 7fdf017fa640 1 -- 172.21.15.27:0/2430543711 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdef4088860 con 0x7fdf0406f8e0 2023-12-19T18:18:33.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.451+0000 7fdf0dbc6740 1 Processor -- start 2023-12-19T18:18:33.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.451+0000 7fdf0dbc6740 1 -- start start 2023-12-19T18:18:33.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.451+0000 7fdf03fff640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdef006a430 0x7fdef006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.452+0000 7fdf0dbc6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f01e110 0x55e09f01e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.452+0000 7fdf0dbc6740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09efee090 0x55e09f01b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.452+0000 7fdf0dbc6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09eff1c90 0x55e09f01b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.452+0000 7fdf0dbc6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09efc9650 con 0x55e09eff1c90 2023-12-19T18:18:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.452+0000 7fdf0dbc6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09ef5e000 con 0x55e09f01e110 2023-12-19T18:18:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.452+0000 7fdf0dbc6740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e09ef5b790 con 0x55e09efee090 2023-12-19T18:18:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.452+0000 7fdf095ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09eff1c90 0x55e09f01b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.452+0000 7fdf095ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09eff1c90 0x55e09f01b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42494/0 (socket says 172.21.15.27:42494) 2023-12-19T18:18:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.452+0000 7fdf095ee640 1 -- 172.21.15.27:0/1681125367 learned_addr learned my addr 172.21.15.27:0/1681125367 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.452+0000 7fdf03fff640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdef006a430 0x7fdef006c8f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdf041faf80 tx=0x7fdef8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.452+0000 7fdf03fff640 1 --2- 172.21.15.27:0/1681125367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09efee090 0x55e09f01b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.452+0000 7fdf08ded640 1 --2- 172.21.15.27:0/1681125367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f01e110 0x55e09f01e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.453+0000 7fdf095ee640 1 -- 172.21.15.27:0/1681125367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09efee090 msgr2=0x55e09f01b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.453+0000 7fdf095ee640 1 --2- 172.21.15.27:0/1681125367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09efee090 0x55e09f01b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.453+0000 7fdf095ee640 1 -- 172.21.15.27:0/1681125367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f01e110 msgr2=0x55e09f01e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.453+0000 7fdf095ee640 1 --2- 172.21.15.27:0/1681125367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f01e110 0x55e09f01e4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.453+0000 7fdf095ee640 1 -- 172.21.15.27:0/1681125367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e09ef1e710 con 0x55e09eff1c90 2023-12-19T18:18:33.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.453+0000 7fdf095ee640 1 --2- 172.21.15.27:0/1681125367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09eff1c90 0x55e09f01b8c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009f40 tx=0x7fdefc02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4034682960ae53e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.453+0000 7fdeea7fc640 1 -- 172.21.15.27:0/1681125367 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc03e430 con 0x55e09eff1c90 2023-12-19T18:18:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.453+0000 7fdeea7fc640 1 -- 172.21.15.27:0/1681125367 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc03e590 con 0x55e09eff1c90 2023-12-19T18:18:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.453+0000 7fdeea7fc640 1 -- 172.21.15.27:0/1681125367 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc03e920 con 0x55e09eff1c90 2023-12-19T18:18:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.454+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1681125367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09eff1c90 msgr2=0x55e09f01b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.454+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1681125367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09eff1c90 0x55e09f01b8c0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fdefc009f40 tx=0x7fdefc02e000 comp rx=0 tx=0).stop 2023-12-19T18:18:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.454+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1681125367 shutdown_connections 2023-12-19T18:18:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.454+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1681125367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09eff1c90 0x55e09f01b8c0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.454+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1681125367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09efee090 0x55e09f01b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.454+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1681125367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f01e110 0x55e09f01e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.454+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1681125367 >> 172.21.15.27:0/1681125367 conn(0x55e09eff03a0 msgr2=0x55e09f038f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.454+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1681125367 shutdown_connections 2023-12-19T18:18:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.454+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1681125367 wait complete. 2023-12-19T18:18:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.455+0000 7fdf0dbc6740 1 -- start start 2023-12-19T18:18:33.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.455+0000 7fdf0dbc6740 1 Processor -- start 2023-12-19T18:18:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.456+0000 7fdf0dbc6740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09eff1c90 0x55e09f01e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.456+0000 7fdf0dbc6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f01e110 0x55e09f0178e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.456+0000 7fdf0dbc6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f013910 0x55e09f0212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.456+0000 7fdf0dbc6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09ef57080 con 0x55e09f013910 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.456+0000 7fdf0dbc6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09eef8850 con 0x55e09f01e110 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.456+0000 7fdf0dbc6740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e09eef84f0 con 0x55e09eff1c90 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.456+0000 7fdf0dbc6740 10 client.0 _ll_register_callbacks cb 0x55e09ef509c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.456+0000 7fdf095ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f013910 0x55e09f0212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.456+0000 7fdf095ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f013910 0x55e09f0212c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42510/0 (socket says 172.21.15.27:42510) 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.456+0000 7fdf095ee640 1 -- 172.21.15.27:0/3423378900 learned_addr learned my addr 172.21.15.27:0/3423378900 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.457+0000 7fdf095ee640 1 -- 172.21.15.27:0/3423378900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09eff1c90 msgr2=0x55e09f01e4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.457+0000 7fdf095ee640 1 --2- 172.21.15.27:0/3423378900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09eff1c90 0x55e09f01e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.457+0000 7fdf03fff640 1 --2- 172.21.15.27:0/3423378900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f01e110 0x55e09f0178e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.457+0000 7fdf095ee640 1 -- 172.21.15.27:0/3423378900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f01e110 msgr2=0x55e09f0178e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.457+0000 7fdf095ee640 1 --2- 172.21.15.27:0/3423378900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f01e110 0x55e09f0178e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.457+0000 7fdf095ee640 1 -- 172.21.15.27:0/3423378900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0406b560 con 0x55e09f013910 2023-12-19T18:18:33.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.457+0000 7fdf03fff640 1 --2- 172.21.15.27:0/3423378900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f01e110 0x55e09f0178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.457+0000 7fdf095ee640 1 --2- 172.21.15.27:0/3423378900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f013910 0x55e09f0212c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdefc02bf20 tx=0x7fdefc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b2cf5f99cdfcedc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.457+0000 7fdee37fe640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.457+0000 7fdee37fe640 1 -- 172.21.15.27:0/3423378900 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc04e070 con 0x55e09f013910 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.457+0000 7fdee37fe640 1 -- 172.21.15.27:0/3423378900 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdefc003990 con 0x55e09f013910 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.457+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/3423378900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e09ef815d0 con 0x55e09f013910 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.458+0000 7fdee37fe640 1 -- 172.21.15.27:0/3423378900 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdefc03e430 con 0x55e09f013910 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.458+0000 7fdf0dbc6740 10 client.6261 fetch_fsmap learned FSMap version 30 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.458+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/3423378900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55e09ef55700 con 0x55e09f013910 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.458+0000 7fdee37fe640 1 -- 172.21.15.27:0/3423378900 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55e09ef815d0 con 0x55e09f013910 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.458+0000 7fdee37fe640 1 -- 172.21.15.27:0/3423378900 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdefc051020 con 0x55e09f013910 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.458+0000 7fdf0dbc6740 10 client.6261 fetch_fsmap finished waiting for FSMap version 30 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.458+0000 7fdf0dbc6740 10 client.6261 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.458+0000 7fdf0dbc6740 20 client.6261 populate_metadata read hostname 'smithi027' 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.458+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/3423378900 >> [v2:172.21.15.27:6834/2772294131,v1:172.21.15.27:6835/2772294131] conn(0x55e09f015810 0x55e09f0562d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.458+0000 7fdf0dbc6740 4 client.6261 mds_command: new command op to 6123 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:18:33.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.458+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/3423378900 --> [v2:172.21.15.27:6834/2772294131,v1:172.21.15.27:6835/2772294131] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e09ef498d0 con 0x55e09f015810 2023-12-19T18:18:33.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.458+0000 7fdf08ded640 1 --2- 172.21.15.27:0/3423378900 >> [v2:172.21.15.27:6834/2772294131,v1:172.21.15.27:6835/2772294131] conn(0x55e09f015810 0x55e09f0562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.459+0000 7fdf08ded640 1 --2- 172.21.15.27:0/3423378900 >> [v2:172.21.15.27:6834/2772294131,v1:172.21.15.27:6835/2772294131] conn(0x55e09f015810 0x55e09f0562d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb665b522c54bcd4 server_cookie=83b75dab67f1e714 in_seq=0 out_seq=0 2023-12-19T18:18:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.459+0000 7fdee37fe640 10 client.6261 ms_handle_connect on v2:172.21.15.27:6834/2772294131 2023-12-19T18:18:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdee37fe640 1 -- 172.21.15.27:0/3423378900 <== mds.0 v2:172.21.15.27:6834/2772294131 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55e09ef498d0 con 0x55e09f015810 2023-12-19T18:18:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdee37fe640 10 client.6261 handle_command_reply: tid=0 2023-12-19T18:18:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdf0dbc6740 1 client.6261 shutdown 2023-12-19T18:18:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/3423378900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f013910 msgr2=0x55e09f0212c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/3423378900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f013910 0x55e09f0212c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fdefc02bf20 tx=0x7fdefc004ab0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/3423378900 shutdown_connections 2023-12-19T18:18:33.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/3423378900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f013910 0x55e09f0212c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/3423378900 >> [v2:172.21.15.27:6834/2772294131,v1:172.21.15.27:6835/2772294131] conn(0x55e09f015810 0x55e09f0562d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/3423378900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f01e110 0x55e09f0178e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/3423378900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09eff1c90 0x55e09f01e4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdee37fe640 0 client.6261 ms_handle_reset on v2:172.21.15.27:6834/2772294131 2023-12-19T18:18:33.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/3423378900 >> 172.21.15.27:0/3423378900 conn(0x55e09efee090 msgr2=0x55e09f021ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:33.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/3423378900 shutdown_connections 2023-12-19T18:18:33.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/3423378900 wait complete. 2023-12-19T18:18:33.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.460+0000 7fdf0dbc6740 20 client.6261 trim_cache size 0 max 16384 2023-12-19T18:18:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.477+0000 7fdf0dbc6740 1 Processor -- start 2023-12-19T18:18:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.477+0000 7fdf0dbc6740 1 -- start start 2023-12-19T18:18:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf0dbc6740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09eff1c90 0x55e09f015d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf0dbc6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f013910 0x55e09f0162a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf0dbc6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 0x55e09f01af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf0dbc6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09ecbef20 con 0x55e09f01e110 2023-12-19T18:18:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf0dbc6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09eff4ce0 con 0x55e09f013910 2023-12-19T18:18:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf0dbc6740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e09eff2330 con 0x55e09eff1c90 2023-12-19T18:18:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf095ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 0x55e09f01af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf095ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 0x55e09f01af40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42514/0 (socket says 172.21.15.27:42514) 2023-12-19T18:18:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf095ee640 1 -- 172.21.15.27:0/1178980874 learned_addr learned my addr 172.21.15.27:0/1178980874 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf095ee640 1 -- 172.21.15.27:0/1178980874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09eff1c90 msgr2=0x55e09f015d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf095ee640 1 --2- 172.21.15.27:0/1178980874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09eff1c90 0x55e09f015d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf03fff640 1 --2- 172.21.15.27:0/1178980874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f013910 0x55e09f0162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf095ee640 1 -- 172.21.15.27:0/1178980874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f013910 msgr2=0x55e09f0162a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf095ee640 1 --2- 172.21.15.27:0/1178980874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f013910 0x55e09f0162a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.478+0000 7fdf095ee640 1 -- 172.21.15.27:0/1178980874 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e09ef8b280 con 0x55e09f01e110 2023-12-19T18:18:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.479+0000 7fdf095ee640 1 --2- 172.21.15.27:0/1178980874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 0x55e09f01af40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdefc02c4e0 tx=0x7fdefc042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=817ca0b3b4449ec0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.479+0000 7fdeeaffd640 1 -- 172.21.15.27:0/1178980874 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc04e070 con 0x55e09f01e110 2023-12-19T18:18:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.479+0000 7fdeeaffd640 1 -- 172.21.15.27:0/1178980874 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc04f040 con 0x55e09f01e110 2023-12-19T18:18:33.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.479+0000 7fdeeaffd640 1 -- 172.21.15.27:0/1178980874 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc03edc0 con 0x55e09f01e110 2023-12-19T18:18:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.480+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1178980874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 msgr2=0x55e09f01af40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.480+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1178980874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 0x55e09f01af40 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fdefc02c4e0 tx=0x7fdefc042000 comp rx=0 tx=0).stop 2023-12-19T18:18:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.480+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1178980874 shutdown_connections 2023-12-19T18:18:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.480+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1178980874 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 0x55e09f01af40 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.480+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1178980874 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09f013910 0x55e09f0162a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.480+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1178980874 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09eff1c90 0x55e09f015d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.480+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1178980874 >> 172.21.15.27:0/1178980874 conn(0x55e09efee090 msgr2=0x55e09f012730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.480+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1178980874 shutdown_connections 2023-12-19T18:18:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.480+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1178980874 wait complete. 2023-12-19T18:18:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.480+0000 7fdf0dbc6740 1 -- start start 2023-12-19T18:18:33.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.481+0000 7fdf0dbc6740 1 Processor -- start 2023-12-19T18:18:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf0dbc6740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09eff1c90 0x55e09f0a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf0dbc6740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09f013910 0x55e09f015d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf0dbc6740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 0x55e09f0162a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf0dbc6740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09f007fb0 con 0x55e09f01e110 2023-12-19T18:18:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf0dbc6740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e09eff3d80 con 0x55e09eff1c90 2023-12-19T18:18:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf0dbc6740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e09f0185f0 con 0x55e09f013910 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf0dbc6740 10 client.0 _ll_register_callbacks cb 0x7fdefc009510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf095ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 0x55e09f0162a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf095ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 0x55e09f0162a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42530/0 (socket says 172.21.15.27:42530) 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf095ee640 1 -- 172.21.15.27:0/1298074532 learned_addr learned my addr 172.21.15.27:0/1298074532 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf08ded640 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09eff1c90 0x55e09f0a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf03fff640 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09f013910 0x55e09f015d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf095ee640 1 -- 172.21.15.27:0/1298074532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09f013910 msgr2=0x55e09f015d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.482+0000 7fdf095ee640 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09f013910 0x55e09f015d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.483+0000 7fdf095ee640 1 -- 172.21.15.27:0/1298074532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09eff1c90 msgr2=0x55e09f0a1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.483+0000 7fdf095ee640 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09eff1c90 0x55e09f0a1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.483+0000 7fdf095ee640 1 -- 172.21.15.27:0/1298074532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e09ef55700 con 0x55e09f01e110 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.483+0000 7fdf03fff640 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09f013910 0x55e09f015d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.483+0000 7fdf095ee640 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 0x55e09f0162a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdefc02c6d0 tx=0x7fdefc030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e8539d85f38016 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.483+0000 7fdee37fe640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:18:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.483+0000 7fdee37fe640 1 -- 172.21.15.27:0/1298074532 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc05f070 con 0x55e09f01e110 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.483+0000 7fdee37fe640 1 -- 172.21.15.27:0/1298074532 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdefc003990 con 0x55e09f01e110 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.483+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1298074532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdefc03e590 con 0x55e09f01e110 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.483+0000 7fdee37fe640 1 -- 172.21.15.27:0/1298074532 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdefc04e040 con 0x55e09f01e110 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.483+0000 7fdf0dbc6740 10 client.6267 fetch_fsmap learned FSMap version 30 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.483+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1298074532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=2+}) v3 -- 0x55e09eef7a60 con 0x55e09f01e110 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.484+0000 7fdee37fe640 1 -- 172.21.15.27:0/1298074532 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc03e590 con 0x55e09f01e110 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.484+0000 7fdee37fe640 1 -- 172.21.15.27:0/1298074532 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 30) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fdefc051050 con 0x55e09f01e110 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.484+0000 7fdf0dbc6740 10 client.6267 fetch_fsmap finished waiting for FSMap version 30 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.484+0000 7fdf0dbc6740 10 client.6267 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.484+0000 7fdf0dbc6740 20 client.6267 populate_metadata read hostname 'smithi027' 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.484+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.27:6834/2772294131,v1:172.21.15.27:6835/2772294131] conn(0x55e09f016fb0 0x55e09eff2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.484+0000 7fdf0dbc6740 4 client.6267 mds_command: new command op to 6123 tid=0[{"prefix": "status"}] 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.484+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1298074532 --> [v2:172.21.15.27:6834/2772294131,v1:172.21.15.27:6835/2772294131] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55e09ef50100 con 0x55e09f016fb0 2023-12-19T18:18:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.484+0000 7fdf08ded640 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.27:6834/2772294131,v1:172.21.15.27:6835/2772294131] conn(0x55e09f016fb0 0x55e09eff2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdf08ded640 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.27:6834/2772294131,v1:172.21.15.27:6835/2772294131] conn(0x55e09f016fb0 0x55e09eff2ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7403843ed5b23675 server_cookie=64a3af30d9f824fe in_seq=0 out_seq=0 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdee37fe640 10 client.6267 ms_handle_connect on v2:172.21.15.27:6834/2772294131 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdee37fe640 1 -- 172.21.15.27:0/1298074532 <== mds.0 v2:172.21.15.27:6834/2772294131 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x55e09ef50100 con 0x55e09f016fb0 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdee37fe640 10 client.6267 handle_command_reply: tid=0 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdf0dbc6740 1 client.6267 shutdown 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1298074532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 msgr2=0x55e09f0162a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 0x55e09f0162a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fdefc02c6d0 tx=0x7fdefc030000 comp rx=0 tx=0).stop 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1298074532 shutdown_connections 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55e09f01e110 0x55e09f0162a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55e09f013910 0x55e09f015d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.27:6834/2772294131,v1:172.21.15.27:6835/2772294131] conn(0x55e09f016fb0 0x55e09eff2ed0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdf0dbc6740 1 --2- 172.21.15.27:0/1298074532 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55e09eff1c90 0x55e09f0a1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1298074532 >> 172.21.15.27:0/1298074532 conn(0x55e09efee090 msgr2=0x55e09f01eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdee37fe640 0 client.6267 ms_handle_reset on v2:172.21.15.27:6834/2772294131 2023-12-19T18:18:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.485+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1298074532 shutdown_connections 2023-12-19T18:18:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.486+0000 7fdf0dbc6740 1 -- 172.21.15.27:0/1298074532 wait complete. 2023-12-19T18:18:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.486+0000 7fdf0dbc6740 20 client.6267 trim_cache size 0 max 16384 2023-12-19T18:18:33.489 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:18:33.489 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:18:33.489 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-12-19T18:18:33.489 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 6123, 2023-12-19T18:18:33.489 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-19T18:18:33.489 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-19T18:18:33.489 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-19T18:18:33.489 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 7.333256413, 2023-12-19T18:18:33.490 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 30, 2023-12-19T18:18:33.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 75, 2023-12-19T18:18:33.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 75, 2023-12-19T18:18:33.490 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 12.634913434 2023-12-19T18:18:33.490 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.486+0000 7fdf09def640 1 -- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdef006a430 msgr2=0x7fdef006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.486+0000 7fdf09def640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdef006a430 0x7fdef006c8f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fdf041faf80 tx=0x7fdef8008040 comp rx=0 tx=0).stop 2023-12-19T18:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.486+0000 7fdf09def640 1 -- 172.21.15.27:0/2430543711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0406f8e0 msgr2=0x7fdf040c4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.486+0000 7fdf09def640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0406f8e0 0x7fdf040c4610 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fdef400c430 tx=0x7fdef400c8f0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.486+0000 7fdf09def640 1 -- 172.21.15.27:0/2430543711 shutdown_connections 2023-12-19T18:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.486+0000 7fdf09def640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdef006a430 0x7fdef006c8f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.486+0000 7fdf09def640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdf04161a40 0x7fdf040ac9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.486+0000 7fdf09def640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdf0415b9e0 0x7fdf0415a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.486+0000 7fdf09def640 1 --2- 172.21.15.27:0/2430543711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf0406f8e0 0x7fdf040c4610 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.486+0000 7fdf09def640 1 -- 172.21.15.27:0/2430543711 >> 172.21.15.27:0/2430543711 conn(0x7fdf0406a7d0 msgr2=0x7fdf040cd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.487+0000 7fdf09def640 1 -- 172.21.15.27:0/2430543711 shutdown_connections 2023-12-19T18:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.487+0000 7fdf09def640 1 -- 172.21.15.27:0/2430543711 wait complete. 2023-12-19T18:18:33.504 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:18:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.722+0000 7fdc22a62640 1 Processor -- start 2023-12-19T18:18:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.722+0000 7fdc22a62640 1 -- start start 2023-12-19T18:18:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.724+0000 7fdc22a62640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c0af630 0x7fdc1c0afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.724+0000 7fdc22a62640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdc1c0c4550 0x7fdc1c0c4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.724+0000 7fdc22a62640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c0affd0 con 0x7fdc1c154030 2023-12-19T18:18:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.724+0000 7fdc22a62640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0cdc00 con 0x7fdc1c0c4550 2023-12-19T18:18:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.724+0000 7fdc22a62640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c0cc190 con 0x7fdc1c0af630 2023-12-19T18:18:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.724+0000 7fdc22261640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdc1c0c4550 0x7fdc1c0c4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.724+0000 7fdc21a60640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdc1c154030 0x7fdc1c0cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50256/0 (socket says 172.21.15.27:50256) 2023-12-19T18:18:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.724+0000 7fdc21a60640 1 -- 172.21.15.27:0/3159898431 learned_addr learned my addr 172.21.15.27:0/3159898431 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.724+0000 7fdc2125f640 1 --2- 172.21.15.27:0/3159898431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c0af630 0x7fdc1c0afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.725+0000 7fdc20a5e640 1 -- 172.21.15.27:0/3159898431 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619900512 0 0) 0x7fdc1c0affd0 con 0x7fdc1c154030 2023-12-19T18:18:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.725+0000 7fdc22261640 1 -- 172.21.15.27:0/3159898431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c0af630 msgr2=0x7fdc1c0afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.725+0000 7fdc22261640 1 --2- 172.21.15.27:0/3159898431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c0af630 0x7fdc1c0afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.725+0000 7fdc22261640 1 -- 172.21.15.27:0/3159898431 >> v1:172.21.15.27:6789/0 conn(0x7fdc1c154030 legacy=0x7fdc1c0cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.725+0000 7fdc22261640 1 -- 172.21.15.27:0/3159898431 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0cb2a0 con 0x7fdc1c0c4550 2023-12-19T18:18:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.726+0000 7fdc22261640 1 --2- 172.21.15.27:0/3159898431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc1c0c4550 0x7fdc1c0c4990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009d90 tx=0x7fdc1402bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5980ccbac8392f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.726+0000 7fdc20a5e640 1 -- 172.21.15.27:0/3159898431 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1402cc50 con 0x7fdc1c0c4550 2023-12-19T18:18:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.726+0000 7fdc20a5e640 1 -- 172.21.15.27:0/3159898431 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1402cdb0 con 0x7fdc1c0c4550 2023-12-19T18:18:33.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.726+0000 7fdc20a5e640 1 -- 172.21.15.27:0/3159898431 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14034660 con 0x7fdc1c0c4550 2023-12-19T18:18:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.727+0000 7fdc22a62640 1 -- 172.21.15.27:0/3159898431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc1c0c4550 msgr2=0x7fdc1c0c4990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.727+0000 7fdc22a62640 1 --2- 172.21.15.27:0/3159898431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc1c0c4550 0x7fdc1c0c4990 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009d90 tx=0x7fdc1402bea0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.727+0000 7fdc22a62640 1 -- 172.21.15.27:0/3159898431 shutdown_connections 2023-12-19T18:18:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.727+0000 7fdc22a62640 1 --2- 172.21.15.27:0/3159898431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc1c0c4550 0x7fdc1c0c4990 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.727+0000 7fdc22a62640 1 --2- 172.21.15.27:0/3159898431 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c0af630 0x7fdc1c0afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.727+0000 7fdc22a62640 1 -- 172.21.15.27:0/3159898431 >> 172.21.15.27:0/3159898431 conn(0x7fdc1c069c70 msgr2=0x7fdc1c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:33.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.727+0000 7fdc22a62640 1 -- 172.21.15.27:0/3159898431 shutdown_connections 2023-12-19T18:18:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.727+0000 7fdc22a62640 1 -- 172.21.15.27:0/3159898431 wait complete. 2023-12-19T18:18:33.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.729+0000 7fdc22a62640 1 Processor -- start 2023-12-19T18:18:33.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.730+0000 7fdc22a62640 1 -- start start 2023-12-19T18:18:33.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.731+0000 7fdc22a62640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc1c0af630 0x7fdc1c142fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.731+0000 7fdc22a62640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c0c4550 0x7fdc1c143c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.731+0000 7fdc22a62640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc1c154030 0x7fdc1c06cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.731+0000 7fdc22a62640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c06d290 con 0x7fdc1c154030 2023-12-19T18:18:33.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.731+0000 7fdc22a62640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c145570 con 0x7fdc1c0af630 2023-12-19T18:18:33.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.731+0000 7fdc22a62640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c1457f0 con 0x7fdc1c0c4550 2023-12-19T18:18:33.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.731+0000 7fdc22261640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc1c154030 0x7fdc1c06cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.731+0000 7fdc22261640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc1c154030 0x7fdc1c06cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42562/0 (socket says 172.21.15.27:42562) 2023-12-19T18:18:33.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.731+0000 7fdc22261640 1 -- 172.21.15.27:0/3753106556 learned_addr learned my addr 172.21.15.27:0/3753106556 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.732+0000 7fdc21a60640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc1c0af630 0x7fdc1c142fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.732+0000 7fdc2125f640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c0c4550 0x7fdc1c143c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.732+0000 7fdc22261640 1 -- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c0c4550 msgr2=0x7fdc1c143c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.732+0000 7fdc22261640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c0c4550 0x7fdc1c143c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.732+0000 7fdc22261640 1 -- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc1c0af630 msgr2=0x7fdc1c142fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.732+0000 7fdc22261640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc1c0af630 0x7fdc1c142fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.732+0000 7fdc22261640 1 -- 172.21.15.27:0/3753106556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc14009a70 con 0x7fdc1c154030 2023-12-19T18:18:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.732+0000 7fdc22261640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc1c154030 0x7fdc1c06cd50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009d90 tx=0x7fdc14004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.733+0000 7fdc127fc640 1 -- 172.21.15.27:0/3753106556 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14040650 con 0x7fdc1c154030 2023-12-19T18:18:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.733+0000 7fdc127fc640 1 -- 172.21.15.27:0/3753106556 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc140407d0 con 0x7fdc1c154030 2023-12-19T18:18:33.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.733+0000 7fdc127fc640 1 -- 172.21.15.27:0/3753106556 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14040b80 con 0x7fdc1c154030 2023-12-19T18:18:33.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.733+0000 7fdc22a62640 1 -- 172.21.15.27:0/3753106556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c145ad0 con 0x7fdc1c154030 2023-12-19T18:18:33.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.733+0000 7fdc22a62640 1 -- 172.21.15.27:0/3753106556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c06c550 con 0x7fdc1c154030 2023-12-19T18:18:33.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.734+0000 7fdc22a62640 1 -- 172.21.15.27:0/3753106556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc1c04e840 con 0x7fdc1c154030 2023-12-19T18:18:33.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.735+0000 7fdc127fc640 1 -- 172.21.15.27:0/3753106556 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdc14040ce0 con 0x7fdc1c154030 2023-12-19T18:18:33.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.736+0000 7fdc127fc640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc0006a430 0x7fdc0006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:33.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.736+0000 7fdc127fc640 1 -- 172.21.15.27:0/3753106556 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc140ab700 con 0x7fdc1c154030 2023-12-19T18:18:33.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.737+0000 7fdc21a60640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc0006a430 0x7fdc0006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:33.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.738+0000 7fdc21a60640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc0006a430 0x7fdc0006c8f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdc18000f60 tx=0x7fdc18007b50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:33.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.740+0000 7fdc127fc640 1 -- 172.21.15.27:0/3753106556 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdc14076d80 con 0x7fdc1c154030 2023-12-19T18:18:33.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.976+0000 7fdc22a62640 1 -- 172.21.15.27:0/3753106556 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdc1c0c3c50 con 0x7fdc1c154030 2023-12-19T18:18:33.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.977+0000 7fdc127fc640 1 -- 172.21.15.27:0/3753106556 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdc1407a3a0 con 0x7fdc1c154030 2023-12-19T18:18:33.977 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:18:33.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.980+0000 7fdc22a62640 1 -- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc0006a430 msgr2=0x7fdc0006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.980+0000 7fdc22a62640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc0006a430 0x7fdc0006c8f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fdc18000f60 tx=0x7fdc18007b50 comp rx=0 tx=0).stop 2023-12-19T18:18:33.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.980+0000 7fdc22a62640 1 -- 172.21.15.27:0/3753106556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc1c154030 msgr2=0x7fdc1c06cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:33.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.980+0000 7fdc22a62640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc1c154030 0x7fdc1c06cd50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fdc14009d90 tx=0x7fdc14004a40 comp rx=0 tx=0).stop 2023-12-19T18:18:33.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.980+0000 7fdc22a62640 1 -- 172.21.15.27:0/3753106556 shutdown_connections 2023-12-19T18:18:33.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.980+0000 7fdc22a62640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc0006a430 0x7fdc0006c8f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.980+0000 7fdc22a62640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc1c154030 0x7fdc1c06cd50 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.980+0000 7fdc22a62640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc1c0c4550 0x7fdc1c143c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.980+0000 7fdc22a62640 1 --2- 172.21.15.27:0/3753106556 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc1c0af630 0x7fdc1c142fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:33.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.980+0000 7fdc22a62640 1 -- 172.21.15.27:0/3753106556 >> 172.21.15.27:0/3753106556 conn(0x7fdc1c069c70 msgr2=0x7fdc1c1468d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:33.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.980+0000 7fdc22a62640 1 -- 172.21.15.27:0/3753106556 shutdown_connections 2023-12-19T18:18:33.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:33.980+0000 7fdc22a62640 1 -- 172.21.15.27:0/3753106556 wait complete. 2023-12-19T18:18:33.998 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:18:33.999 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:18:33.999 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-12-19T18:18:33.999 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:18:33.999 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:18:33.999 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:18:33.999 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:18:33.999 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:18:33.999 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:18:33.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:34.000 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-12-19T18:18:34.052 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-12-19T18:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-12-19T18:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42126sec preferred_lft 42126sec 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::8053:76ff:fe69:c66c/64 scope link 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:18:34.055 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:18:34.055 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::781d:d6ff:feb4:7a2d/64 scope link 2023-12-19T18:18:34.055 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:34.055 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:18:34.055 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether f2:29:63:cc:14:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:18:34.055 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::f029:63ff:fecc:140e/64 scope link 2023-12-19T18:18:34.055 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:34.056 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:34.056 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:18:34.161 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-12-19T18:18:34.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:34.166 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-12-19T18:18:34.228 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-12-19T18:18:34.228 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:18:34.228 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-19T18:18:34.228 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-19T18:18:34.229 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:18:34.229 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:34.229 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:18:34.229 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:18:34.229 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:18:34.229 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:18:34.229 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:18:34.229 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:18:34.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:34.438 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:18:34.493 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:18:34.494 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:18:34.494 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:18:34.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:34.494 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-12-19T18:18:34.591 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-19T18:18:34.591 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:18:34.592 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:18:34.654 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:18:34.654 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:18:34.655 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:18:34.717 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:18:34.828 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:18:34.828 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:18:34.940 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:18:34.943 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:18:34.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:34.944 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:18:35.004 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:18:35.004 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-19T18:18:35.010 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:18:35.099 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:18:35.099 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:18:35.176 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:18:35.178 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:18:35.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:35.179 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:18:35.224 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-12-19T18:18:35.224+0000 7faa7546f480 -1 init, newargv = 0x56482dc8a6d0 newargc=15 2023-12-19T18:18:35.224 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[100298]: starting ceph client 2023-12-19T18:18:35.232 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[100298]: starting fuse 2023-12-19T18:18:36.323 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:18:36.443 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:18:36.443 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:18:36.551 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:18:36.557 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:18:36.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:36.557 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:18:36.615 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:18:36.616 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-19T18:18:36.616 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:18:36.616 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:18:36.616 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:18:36.616 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:18:36.616 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:18:36.616 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:18:36.616 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:18:36.617 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:18:36.618 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:18:36.618 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:18:36.618 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:18:36.618 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:18:36.618 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:18:36.618 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:18:36.799 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.100298.asok 2023-12-19T18:18:36.976 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.100298.asok 2023-12-19T18:18:36.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:36.977 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.100298.asok status 2023-12-19T18:18:37.237 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:18:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:18:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:18:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:18:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:18:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:18:37.237 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "100298", 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 6282, 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 6282 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:18:37.238 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1295122371 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 1295122371 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.6282 192.168.0.1:0/1295122371", 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/1295122371", 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 30, 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 75, 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:18:37.239 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:18:37.240 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:18:37.240 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:18:37.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:37.253 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-12-19T18:18:37.254 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-19T18:18:37.312 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-12-19T18:18:37.312 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-19T18:18:37.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:37.312 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:18:37.429 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:18:37.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.665+0000 7f10a2973640 1 Processor -- start 2023-12-19T18:18:37.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.665+0000 7f10a2973640 1 -- start start 2023-12-19T18:18:37.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.667+0000 7f10a2973640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f109c0b8920 0x7f109c160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.667+0000 7f10a2973640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f109c0b5060 0x7f109c0d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.667+0000 7f10a2973640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f109c0d0d20 con 0x7f109c0b4c80 2023-12-19T18:18:37.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.667+0000 7f10a2973640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f109c0b6130 con 0x7f109c0b5060 2023-12-19T18:18:37.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.667+0000 7f10a2973640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c161bf0 con 0x7f109c0b8920 2023-12-19T18:18:37.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.667+0000 7f10a0f10640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f109c0b5060 0x7f109c0d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.667+0000 7f10a0f10640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f109c0b5060 0x7f109c0d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42572/0 (socket says 172.21.15.27:42572) 2023-12-19T18:18:37.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.667+0000 7f10a0f10640 1 -- 172.21.15.27:0/4033492524 learned_addr learned my addr 172.21.15.27:0/4033492524 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:37.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.667+0000 7f109b7fe640 1 --2- 172.21.15.27:0/4033492524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f109c0b8920 0x7f109c160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.668+0000 7f10a0f10640 1 -- 172.21.15.27:0/4033492524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f109c0b8920 msgr2=0x7f109c160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:37.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.668+0000 7f10a0f10640 1 --2- 172.21.15.27:0/4033492524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f109c0b8920 0x7f109c160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.668+0000 7f10a0f10640 1 -- 172.21.15.27:0/4033492524 >> v1:172.21.15.192:6789/0 conn(0x7f109c0b4c80 legacy=0x7f109c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:37.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.668+0000 7f10a0f10640 1 -- 172.21.15.27:0/4033492524 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c161350 con 0x7f109c0b5060 2023-12-19T18:18:37.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.669+0000 7f10a0f10640 1 --2- 172.21.15.27:0/4033492524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f109c0b5060 0x7f109c0d1a20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1094009e50 tx=0x7f109402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f867909accf2eebe server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:37.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.669+0000 7f109affd640 1 -- 172.21.15.27:0/4033492524 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109402c9e0 con 0x7f109c0b5060 2023-12-19T18:18:37.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.669+0000 7f109affd640 1 -- 172.21.15.27:0/4033492524 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109402cb40 con 0x7f109c0b5060 2023-12-19T18:18:37.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.669+0000 7f109affd640 1 -- 172.21.15.27:0/4033492524 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10940343f0 con 0x7f109c0b5060 2023-12-19T18:18:37.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.670+0000 7f10a2973640 1 -- 172.21.15.27:0/4033492524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f109c0b5060 msgr2=0x7f109c0d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.670+0000 7f10a2973640 1 --2- 172.21.15.27:0/4033492524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f109c0b5060 0x7f109c0d1a20 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f1094009e50 tx=0x7f109402be40 comp rx=0 tx=0).stop 2023-12-19T18:18:37.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.670+0000 7f10a2973640 1 -- 172.21.15.27:0/4033492524 shutdown_connections 2023-12-19T18:18:37.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.670+0000 7f10a2973640 1 --2- 172.21.15.27:0/4033492524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f109c0b5060 0x7f109c0d1a20 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.670+0000 7f10a2973640 1 --2- 172.21.15.27:0/4033492524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f109c0b8920 0x7f109c160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.670+0000 7f10a2973640 1 -- 172.21.15.27:0/4033492524 >> 172.21.15.27:0/4033492524 conn(0x7f109c0d2f90 msgr2=0x7f109c0d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:37.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.670+0000 7f10a2973640 1 -- 172.21.15.27:0/4033492524 shutdown_connections 2023-12-19T18:18:37.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.670+0000 7f10a2973640 1 -- 172.21.15.27:0/4033492524 wait complete. 2023-12-19T18:18:37.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.672+0000 7f10a2973640 1 Processor -- start 2023-12-19T18:18:37.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.672+0000 7f10a2973640 1 -- start start 2023-12-19T18:18:37.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.673+0000 7f10a2973640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f109c0b4c80 0x7f109c1546d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.673+0000 7f10a2973640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f109c0b5060 0x7f109c0c8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.673+0000 7f10a2973640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f109c0b8920 0x7f109c0cb590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.673+0000 7f10a2973640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0ca4f0 con 0x7f109c0b8920 2023-12-19T18:18:37.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.673+0000 7f10a2973640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0ca7d0 con 0x7f109c0b5060 2023-12-19T18:18:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.673+0000 7f10a2973640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f109c0caab0 con 0x7f109c0b4c80 2023-12-19T18:18:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.673+0000 7f10a0f10640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f109c0b8920 0x7f109c0cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.674+0000 7f10a0f10640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f109c0b8920 0x7f109c0cb590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42576/0 (socket says 172.21.15.27:42576) 2023-12-19T18:18:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.674+0000 7f10a0f10640 1 -- 172.21.15.27:0/3987059484 learned_addr learned my addr 172.21.15.27:0/3987059484 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.674+0000 7f109bfff640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f109c0b4c80 0x7f109c1546d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.674+0000 7f109b7fe640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f109c0b5060 0x7f109c0c8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.674+0000 7f10a0f10640 1 -- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f109c0b4c80 msgr2=0x7f109c1546d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.674+0000 7f10a0f10640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f109c0b4c80 0x7f109c1546d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.674+0000 7f10a0f10640 1 -- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f109c0b5060 msgr2=0x7f109c0c8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.674+0000 7f10a0f10640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f109c0b5060 0x7f109c0c8e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.674+0000 7f10a0f10640 1 -- 172.21.15.27:0/3987059484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1094009b30 con 0x7f109c0b8920 2023-12-19T18:18:37.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.674+0000 7f10a0f10640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f109c0b8920 0x7f109c0cb590 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1094036040 tx=0x7f1094004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:37.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.675+0000 7f1098ff9640 1 -- 172.21.15.27:0/3987059484 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1094009510 con 0x7f109c0b8920 2023-12-19T18:18:37.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.675+0000 7f1098ff9640 1 -- 172.21.15.27:0/3987059484 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109403e450 con 0x7f109c0b8920 2023-12-19T18:18:37.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.675+0000 7f1098ff9640 1 -- 172.21.15.27:0/3987059484 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109403e7e0 con 0x7f109c0b8920 2023-12-19T18:18:37.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.675+0000 7f10a2973640 1 -- 172.21.15.27:0/3987059484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109c0cad90 con 0x7f109c0b8920 2023-12-19T18:18:37.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.675+0000 7f10a2973640 1 -- 172.21.15.27:0/3987059484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109c154c10 con 0x7f109c0b8920 2023-12-19T18:18:37.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.676+0000 7f10a2973640 1 -- 172.21.15.27:0/3987059484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f109c0cb2b0 con 0x7f109c0b8920 2023-12-19T18:18:37.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.677+0000 7f1098ff9640 1 -- 172.21.15.27:0/3987059484 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f109403e940 con 0x7f109c0b8920 2023-12-19T18:18:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.678+0000 7f1098ff9640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f107c06a430 0x7f107c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.678+0000 7f1098ff9640 1 -- 172.21.15.27:0/3987059484 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10940ab770 con 0x7f109c0b8920 2023-12-19T18:18:37.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.679+0000 7f109bfff640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f107c06a430 0x7f107c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:37.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.680+0000 7f109bfff640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f107c06a430 0x7f107c06c8f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f109c0c99a0 tx=0x7f108c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:37.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.682+0000 7f1098ff9640 1 -- 172.21.15.27:0/3987059484 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f109c0cb2b0 con 0x7f109c0b8920 2023-12-19T18:18:37.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.921+0000 7f10a2973640 1 -- 172.21.15.27:0/3987059484 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f109c0cb2b0 con 0x7f109c0b8920 2023-12-19T18:18:37.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.922+0000 7f1098ff9640 1 -- 172.21.15.27:0/3987059484 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v30) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1094043060 con 0x7f109c0b8920 2023-12-19T18:18:37.922 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:18:37.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.925+0000 7f10a2973640 1 -- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f107c06a430 msgr2=0x7f107c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.925+0000 7f10a2973640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f107c06a430 0x7f107c06c8f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f109c0c99a0 tx=0x7f108c009040 comp rx=0 tx=0).stop 2023-12-19T18:18:37.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.925+0000 7f10a2973640 1 -- 172.21.15.27:0/3987059484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f109c0b8920 msgr2=0x7f109c0cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:37.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.925+0000 7f10a2973640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f109c0b8920 0x7f109c0cb590 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f1094036040 tx=0x7f1094004a40 comp rx=0 tx=0).stop 2023-12-19T18:18:37.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.925+0000 7f10a2973640 1 -- 172.21.15.27:0/3987059484 shutdown_connections 2023-12-19T18:18:37.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.925+0000 7f10a2973640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f107c06a430 0x7f107c06c8f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.925+0000 7f10a2973640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f109c0b8920 0x7f109c0cb590 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.925+0000 7f10a2973640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f109c0b5060 0x7f109c0c8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.925+0000 7f10a2973640 1 --2- 172.21.15.27:0/3987059484 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f109c0b4c80 0x7f109c1546d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:37.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.925+0000 7f10a2973640 1 -- 172.21.15.27:0/3987059484 >> 172.21.15.27:0/3987059484 conn(0x7f109c0d2f90 msgr2=0x7f109c0b62a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:37.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.925+0000 7f10a2973640 1 -- 172.21.15.27:0/3987059484 shutdown_connections 2023-12-19T18:18:37.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:37.925+0000 7f10a2973640 1 -- 172.21.15.27:0/3987059484 wait complete. 2023-12-19T18:18:37.943 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:18:37.943 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:18:37.943 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-19T18:18:37.943 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-19T18:18:37.943 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T18:18:37.943 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:18:37.943 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:18:37.943 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:18:37.943 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:18:37.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:37.944 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-19T18:18:38.005 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:18:38.006 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:18:38.006 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:18:38.006 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:38.006 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:18:38.006 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:38.006 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:18:38.006 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:38.006 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-19T18:18:38.006 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:18:38.006 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:38.006 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-19T18:18:38.006 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 42085sec preferred_lft 42085sec 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::b892:1cff:fedc:4bed/64 scope link 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:38.007 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:18:38.008 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether a6:e1:fc:e6:56:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:18:38.008 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::a4e1:fcff:fee6:56fa/64 scope link 2023-12-19T18:18:38.008 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:38.008 INFO:teuthology.orchestra.run.smithi192.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:18:38.008 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T18:18:38.008 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::404f:6cff:fe73:9aa2/64 scope link 2023-12-19T18:18:38.008 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:18:38.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:38.008 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:18:38.100 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-19T18:18:38.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:38.104 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-19T18:18:38.160 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-19T18:18:38.160 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:18:38.160 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T18:18:38.160 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T18:18:38.160 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:18:38.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:38.161 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:18:38.161 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:18:38.161 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T18:18:38.161 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T18:18:38.161 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T18:18:38.161 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:18:38.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:38.348 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:18:38.398 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T18:18:38.398 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-19T18:18:38.399 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:18:38.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:38.399 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-19T18:18:38.494 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-19T18:18:38.494 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:18:38.494 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-19T18:18:38.552 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:18:38.552 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:18:38.552 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:18:38.606 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:18:38.701 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:18:38.701 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:18:38.795 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:18:38.799 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:18:38.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:38.799 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:18:38.852 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:18:38.852 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-19T18:18:38.857 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:18:38.937 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:18:38.937 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:18:39.008 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:18:39.011 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:18:39.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:39.011 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:18:39.042 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-19T18:18:39.042+0000 7fdcb163a480 -1 init, newargv = 0x5578955c4870 newargc=15 2023-12-19T18:18:39.042 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[87832]: starting ceph client 2023-12-19T18:18:39.053 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[87832]: starting fuse 2023-12-19T18:18:40.152 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:18:40.255 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:18:40.256 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:18:40.351 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:18:40.355 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:18:40.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:40.355 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:18:40.407 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:18:40.408 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-12-19T18:18:40.408 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:18:40.408 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-19T18:18:40.408 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-19T18:18:40.408 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-19T18:18:40.408 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-19T18:18:40.408 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:18:40.408 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-19T18:18:40.409 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-19T18:18:40.410 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:18:40.410 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:18:40.410 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-19T18:18:40.410 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-19T18:18:40.585 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.87832.asok 2023-12-19T18:18:40.767 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.87832.asok 2023-12-19T18:18:40.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:40.767 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.87832.asok status 2023-12-19T18:18:41.006 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-19T18:18:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-19T18:18:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:18:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:18:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-19T18:18:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-19T18:18:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-19T18:18:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "87832", 2023-12-19T18:18:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-19T18:18:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:18:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-19T18:18:41.007 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5099, 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5099 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3748623284 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:18:41.008 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:18:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3748623284 2023-12-19T18:18:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:18:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5099 192.168.0.1:0/3748623284", 2023-12-19T18:18:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/3748623284", 2023-12-19T18:18:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-19T18:18:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 30, 2023-12-19T18:18:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 75, 2023-12-19T18:18:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:18:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-19T18:18:41.009 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-19T18:18:41.009 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-19T18:18:41.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:41.022 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-19T18:18:41.022 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-19T18:18:41.075 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-19T18:18:41.075 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-19T18:18:41.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:18:41.075 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:18:41.169 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:18:41.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.411+0000 7ff60c915640 1 Processor -- start 2023-12-19T18:18:41.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.411+0000 7ff60c915640 1 -- start start 2023-12-19T18:18:41.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.413+0000 7ff60c915640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff6081553a0 0x7ff6080c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.413+0000 7ff60c915640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6080b0360 con 0x7ff6080afeb0 2023-12-19T18:18:41.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.413+0000 7ff60c915640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6080cb350 con 0x7ff60815ee30 2023-12-19T18:18:41.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.413+0000 7ff60c915640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6080cbcf0 con 0x7ff6081553a0 2023-12-19T18:18:41.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.413+0000 7ff60659c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff6080afeb0 0x7ff60815e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:50270/0 (socket says 172.21.15.27:50270) 2023-12-19T18:18:41.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.414+0000 7ff60659c640 1 -- 172.21.15.27:0/820749294 learned_addr learned my addr 172.21.15.27:0/820749294 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.414+0000 7ff606d9d640 1 --2- 172.21.15.27:0/820749294 >> v2:172.21.15.192:3300/0 conn(0x7ff6081553a0 0x7ff6080c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.414+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637090478 0 0) 0x7ff6080b0360 con 0x7ff6080afeb0 2023-12-19T18:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.414+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5f0003520 con 0x7ff6080afeb0 2023-12-19T18:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.414+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 843110557 0 0) 0x7ff5f0003520 con 0x7ff6080afeb0 2023-12-19T18:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 >> v2:172.21.15.192:3300/0 conn(0x7ff6081553a0 msgr2=0x7ff6080c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff605d9b640 1 --2- 172.21.15.27:0/820749294 >> v2:172.21.15.192:3300/0 conn(0x7ff6081553a0 0x7ff6080c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 >> v1:172.21.15.192:6789/0 conn(0x7ff60815ee30 legacy=0x7ff6080c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6080cb630 con 0x7ff6080afeb0 2023-12-19T18:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7ff5fc001670 con 0x7ff6080afeb0 2023-12-19T18:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 >> v1:172.21.15.27:6789/0 conn(0x7ff6080afeb0 legacy=0x7ff60815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff605d9b640 1 --2- 172.21.15.27:0/820749294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5f0003970 0x7ff5f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff605d9b640 1 --2- 172.21.15.27:0/820749294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5f00047c0 0x7ff5f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff605d9b640 1 --2- 172.21.15.27:0/820749294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5f0005180 0x7ff5f000fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6080cbcf0 con 0x7ff5f00047c0 2023-12-19T18:18:41.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f0010530 con 0x7ff5f0003970 2023-12-19T18:18:41.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5f00106f0 con 0x7ff5f0005180 2023-12-19T18:18:41.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff60659c640 1 --2- 172.21.15.27:0/820749294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5f00047c0 0x7ff5f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff606d9d640 1 --2- 172.21.15.27:0/820749294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5f0003970 0x7ff5f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff60659c640 1 -- 172.21.15.27:0/820749294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5f0005180 msgr2=0x7ff5f000fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:41.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff60759e640 1 --2- 172.21.15.27:0/820749294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5f0005180 0x7ff5f000fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff60659c640 1 --2- 172.21.15.27:0/820749294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5f0005180 0x7ff5f000fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff60659c640 1 -- 172.21.15.27:0/820749294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5f0003970 msgr2=0x7ff5f0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:18:41.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.415+0000 7ff60659c640 1 --2- 172.21.15.27:0/820749294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5f0003970 0x7ff5f0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.416+0000 7ff60659c640 1 -- 172.21.15.27:0/820749294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5f0003520 con 0x7ff5f00047c0 2023-12-19T18:18:41.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.416+0000 7ff606d9d640 1 --2- 172.21.15.27:0/820749294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5f0003970 0x7ff5f0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:18:41.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.416+0000 7ff60759e640 1 -- 172.21.15.27:0/820749294 reap_dead start 2023-12-19T18:18:41.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.417+0000 7ff60659c640 1 --2- 172.21.15.27:0/820749294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5f00047c0 0x7ff5f0004c40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00c240 tx=0x7ff5f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2076e1c8381d1aa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:41.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.417+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc005ad0 con 0x7ff5f00047c0 2023-12-19T18:18:41.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.417+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc005c30 con 0x7ff5f00047c0 2023-12-19T18:18:41.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.417+0000 7ff605d9b640 1 -- 172.21.15.27:0/820749294 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc02caf0 con 0x7ff5f00047c0 2023-12-19T18:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.417+0000 7ff60c915640 1 -- 172.21.15.27:0/820749294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5f00047c0 msgr2=0x7ff5f0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.417+0000 7ff60c915640 1 --2- 172.21.15.27:0/820749294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5f00047c0 0x7ff5f0004c40 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc00c240 tx=0x7ff5f00057f0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.418+0000 7ff60c915640 1 -- 172.21.15.27:0/820749294 shutdown_connections 2023-12-19T18:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.418+0000 7ff60c915640 1 --2- 172.21.15.27:0/820749294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5f00047c0 0x7ff5f0004c40 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.418+0000 7ff60c915640 1 -- 172.21.15.27:0/820749294 >> 172.21.15.27:0/820749294 conn(0x7ff608069c70 msgr2=0x7ff60806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.418+0000 7ff60c915640 1 -- 172.21.15.27:0/820749294 shutdown_connections 2023-12-19T18:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.418+0000 7ff60c915640 1 -- 172.21.15.27:0/820749294 wait complete. 2023-12-19T18:18:41.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.420+0000 7ff60c915640 1 Processor -- start 2023-12-19T18:18:41.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.420+0000 7ff60c915640 1 -- start start 2023-12-19T18:18:41.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60c915640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5f00047c0 0x7ff6080b3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60c915640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6080ccc40 0x7ff6081ee200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60c915640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6080cd0c0 0x7ff60814f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60c915640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6080cc1f0 con 0x7ff6080ccc40 2023-12-19T18:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60c915640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6081eecb0 con 0x7ff6080cd0c0 2023-12-19T18:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60c915640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6081eef90 con 0x7ff5f00047c0 2023-12-19T18:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6080ccc40 0x7ff6081ee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6080ccc40 0x7ff6081ee200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42582/0 (socket says 172.21.15.27:42582) 2023-12-19T18:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60659c640 1 -- 172.21.15.27:0/2578234776 learned_addr learned my addr 172.21.15.27:0/2578234776 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60759e640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6080cd0c0 0x7ff60814f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff606d9d640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5f00047c0 0x7ff6080b3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60659c640 1 -- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5f00047c0 msgr2=0x7ff6080b3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60659c640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5f00047c0 0x7ff6080b3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60659c640 1 -- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6080cd0c0 msgr2=0x7ff60814f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60659c640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6080cd0c0 0x7ff60814f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.421+0000 7ff60659c640 1 -- 172.21.15.27:0/2578234776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6080cc440 con 0x7ff6080ccc40 2023-12-19T18:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.422+0000 7ff60659c640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6080ccc40 0x7ff6081ee200 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc001930 tx=0x7ff5fc02e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.422+0000 7ff5ef7fe640 1 -- 172.21.15.27:0/2578234776 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc03c040 con 0x7ff6080ccc40 2023-12-19T18:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.422+0000 7ff5ef7fe640 1 -- 172.21.15.27:0/2578234776 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5fc02cd80 con 0x7ff6080ccc40 2023-12-19T18:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.422+0000 7ff60c915640 1 -- 172.21.15.27:0/2578234776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6080cc720 con 0x7ff6080ccc40 2023-12-19T18:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.422+0000 7ff5ef7fe640 1 -- 172.21.15.27:0/2578234776 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5fc007290 con 0x7ff6080ccc40 2023-12-19T18:18:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.422+0000 7ff60c915640 1 -- 172.21.15.27:0/2578234776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6080b4050 con 0x7ff6080ccc40 2023-12-19T18:18:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.424+0000 7ff60c915640 1 -- 172.21.15.27:0/2578234776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6081ef150 con 0x7ff6080ccc40 2023-12-19T18:18:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.424+0000 7ff5ef7fe640 1 -- 172.21.15.27:0/2578234776 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff5fc032070 con 0x7ff6080ccc40 2023-12-19T18:18:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.425+0000 7ff5ef7fe640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5f406a430 0x7ff5f406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:18:41.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.425+0000 7ff5ef7fe640 1 -- 172.21.15.27:0/2578234776 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5fc0a9ee0 con 0x7ff6080ccc40 2023-12-19T18:18:41.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.426+0000 7ff606d9d640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5f406a430 0x7ff5f406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:18:41.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.427+0000 7ff606d9d640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5f406a430 0x7ff5f406c8f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff6080cddb0 tx=0x7ff5f8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:18:41.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.430+0000 7ff5ef7fe640 1 -- 172.21.15.27:0/2578234776 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff5fc075430 con 0x7ff6080ccc40 2023-12-19T18:18:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.673+0000 7ff60c915640 1 -- 172.21.15.27:0/2578234776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6080c5070 con 0x7ff6080ccc40 2023-12-19T18:18:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.674+0000 7ff5ef7fe640 1 -- 172.21.15.27:0/2578234776 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4715 (secure 0 0 0) 0x7ff5fc078a50 con 0x7ff6080ccc40 2023-12-19T18:18:41.674 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:18:41.674 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1624358030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1624358030},{"type":"v1","addr":"172.21.15.192:6836","nonce":1624358030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5030,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3025339386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3025339386},{"type":"v1","addr":"172.21.15.192:6837","nonce":3025339386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/1685117850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1685117850},{"type":"v1","addr":"172.21.15.27:6833","nonce":1685117850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:18:26.147336+0000","modified":"2023-12-19T18:18:29.663526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2772294131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2772294131},{"type":"v1","addr":"172.21.15.27:6835","nonce":2772294131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-19T18:18:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2023-12-19T18:18:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.676+0000 7ff60c915640 1 -- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5f406a430 msgr2=0x7ff5f406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.676+0000 7ff60c915640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5f406a430 0x7ff5f406c8f0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7ff6080cddb0 tx=0x7ff5f8008040 comp rx=0 tx=0).stop 2023-12-19T18:18:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.676+0000 7ff60c915640 1 -- 172.21.15.27:0/2578234776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6080ccc40 msgr2=0x7ff6081ee200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:18:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.676+0000 7ff60c915640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6080ccc40 0x7ff6081ee200 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc001930 tx=0x7ff5fc02e070 comp rx=0 tx=0).stop 2023-12-19T18:18:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.676+0000 7ff60c915640 1 -- 172.21.15.27:0/2578234776 shutdown_connections 2023-12-19T18:18:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.677+0000 7ff60c915640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5f406a430 0x7ff5f406c8f0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.677+0000 7ff60c915640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff6080cd0c0 0x7ff60814f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.677+0000 7ff60c915640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6080ccc40 0x7ff6081ee200 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.677+0000 7ff60c915640 1 --2- 172.21.15.27:0/2578234776 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5f00047c0 0x7ff6080b3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:18:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.677+0000 7ff60c915640 1 -- 172.21.15.27:0/2578234776 >> 172.21.15.27:0/2578234776 conn(0x7ff608069c70 msgr2=0x7ff6080c3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:18:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.677+0000 7ff60c915640 1 -- 172.21.15.27:0/2578234776 shutdown_connections 2023-12-19T18:18:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:18:41.677+0000 7ff60c915640 1 -- 172.21.15.27:0/2578234776 wait complete. 2023-12-19T18:18:41.697 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-12-19T18:18:41.942 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":6123,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":15.772596059,"mdsmap_epoch":30,"osdmap_epoch":75,"osdmap_epoch_barrier":75,"uptime":21.07426663} 2023-12-19T18:18:41.942 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 6123, "mdsmap_epoch": 30, "osdmap_epoch": 75, "osdmap_epoch_barrier": 75, "rank_uptime": 15.772596059, "state": "up:active", "uptime": 21.07426663, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:18:41.943 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-12-19T18:18:42.182 INFO:teuthology.orchestra.run.smithi027.stdout:{"osd_mon_report_interval":"5"} 2023-12-19T18:18:42.183 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-12-19T18:18:42.183 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-12-19T18:18:42.415 INFO:teuthology.orchestra.run.smithi027.stdout:{"mon_tick_interval":"5"} 2023-12-19T18:18:42.415 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-12-19T18:18:42.415 INFO:tasks.cephfs.test_full:pool capacity 557028928, 531MB should be enough to fill it 2023-12-19T18:18:42.415 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:18:42.416 DEBUG:teuthology.orchestra.run.smithi027:> import time 2023-12-19T18:18:42.416 DEBUG:teuthology.orchestra.run.smithi027:> import datetime 2023-12-19T18:18:42.416 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:18:42.416 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:18:42.416 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:18:42.416 DEBUG:teuthology.orchestra.run.smithi027:> # Write some buffered data through before going full, all should be well 2023-12-19T18:18:42.416 DEBUG:teuthology.orchestra.run.smithi027:> print("writing some data through which we expect to succeed") 2023-12-19T18:18:42.416 DEBUG:teuthology.orchestra.run.smithi027:> bytes = 0 2023-12-19T18:18:42.416 DEBUG:teuthology.orchestra.run.smithi027:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-12-19T18:18:42.416 DEBUG:teuthology.orchestra.run.smithi027:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-12-19T18:18:42.416 DEBUG:teuthology.orchestra.run.smithi027:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-12-19T18:18:42.416 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(f) 2023-12-19T18:18:42.416 DEBUG:teuthology.orchestra.run.smithi027:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> # buffered IO, the error should always only appear from fsync and not 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> # from write 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> full = False 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> for n in range(0, int(531 * 0.9)): 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> print("done writing {0} bytes".format(bytes)) 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> # OK, now we should sneak in under the full condition 2023-12-19T18:18:42.417 DEBUG:teuthology.orchestra.run.smithi027:> # due to the time it takes the OSDs to report to the 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> # mons, and get a successful fsync on our full-making data 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(f) 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> # buffered write, add more dirty data to the buffer 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> print("starting buffered write") 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> try: 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> for n in range(0, int(531 * 0.2)): 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> time.sleep(15.0) 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> except OSError: 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> pass; 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-12-19T18:18:42.418 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> # Wait long enough for a background flush that should fail 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> time.sleep(30) 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> if True: 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> # ...and check that the failed background flush is reflected in fclose 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> try: 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> os.close(f) 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> except OSError: 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> print("close() returned an error as expected") 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> else: 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("close() failed to raise error") 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> else: 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> # The kernel cephfs client does not raise errors on fclose 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> os.close(f) 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-12-19T18:18:42.419 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:18:45.449 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:18:45.459 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:18:47.641 INFO:teuthology.orchestra.run.smithi027.stdout:writing some data through which we expect to succeed 2023-12-19T18:18:47.642 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-12-19T18:18:47.642 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-12-19T18:18:47.642 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-12-19T18:18:47.642 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-12-19T18:18:47.642 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-12-19T18:18:47.642 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-12-19T18:18:47.642 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-12-19T18:18:47.642 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-12-19T18:18:47.642 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-12-19T18:18:47.642 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-12-19T18:18:47.642 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-12-19T18:18:47.642 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-12-19T18:18:47.643 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 30932992 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 31981568 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 33030144 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 34078720 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 35127296 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 36175872 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 37224448 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 38273024 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 39321600 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 40370176 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 41418752 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 42467328 bytes via buffered write, may repeat 2023-12-19T18:18:47.644 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 43515904 bytes via buffered write, may repeat 2023-12-19T18:18:47.645 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 44564480 bytes via buffered write, may repeat 2023-12-19T18:18:47.645 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 45613056 bytes via buffered write, may repeat 2023-12-19T18:18:47.645 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 46661632 bytes via buffered write, may repeat 2023-12-19T18:18:47.645 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 47710208 bytes via buffered write, may repeat 2023-12-19T18:18:47.645 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 48758784 bytes via buffered write, may repeat 2023-12-19T18:18:47.645 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 49807360 bytes via buffered write, may repeat 2023-12-19T18:18:47.645 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 50855936 bytes via buffered write, may repeat 2023-12-19T18:18:47.645 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 51904512 bytes via buffered write, may repeat 2023-12-19T18:18:47.645 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 52953088 bytes via buffered write, may repeat 2023-12-19T18:18:47.645 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 54001664 bytes via buffered write, may repeat 2023-12-19T18:18:47.645 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 55050240 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 56098816 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 57147392 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 58195968 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 59244544 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 60293120 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 61341696 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 62390272 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 63438848 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 64487424 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 65536000 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 66584576 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 67633152 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 68681728 bytes via buffered write, may repeat 2023-12-19T18:18:47.646 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 69730304 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 70778880 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 71827456 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 72876032 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 73924608 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 74973184 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 76021760 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 77070336 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 78118912 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 79167488 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 80216064 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 81264640 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 82313216 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 83361792 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 84410368 bytes via buffered write, may repeat 2023-12-19T18:18:47.647 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 85458944 bytes via buffered write, may repeat 2023-12-19T18:18:47.648 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 86507520 bytes via buffered write, may repeat 2023-12-19T18:18:47.648 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 87556096 bytes via buffered write, may repeat 2023-12-19T18:18:47.648 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 88604672 bytes via buffered write, may repeat 2023-12-19T18:18:47.648 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 89653248 bytes via buffered write, may repeat 2023-12-19T18:18:47.648 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 90701824 bytes via buffered write, may repeat 2023-12-19T18:18:47.648 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 91750400 bytes via buffered write, may repeat 2023-12-19T18:18:47.648 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 92798976 bytes via buffered write, may repeat 2023-12-19T18:18:47.648 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 93847552 bytes via buffered write, may repeat 2023-12-19T18:18:47.648 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 94896128 bytes via buffered write, may repeat 2023-12-19T18:18:47.648 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 95944704 bytes via buffered write, may repeat 2023-12-19T18:18:47.648 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 96993280 bytes via buffered write, may repeat 2023-12-19T18:18:47.648 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 98041856 bytes via buffered write, may repeat 2023-12-19T18:18:47.648 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 99090432 bytes via buffered write, may repeat 2023-12-19T18:18:47.649 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 100139008 bytes via buffered write, may repeat 2023-12-19T18:18:47.649 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 101187584 bytes via buffered write, may repeat 2023-12-19T18:18:47.649 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 102236160 bytes via buffered write, may repeat 2023-12-19T18:18:47.649 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 103284736 bytes via buffered write, may repeat 2023-12-19T18:18:47.649 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 104333312 bytes via buffered write, may repeat 2023-12-19T18:18:47.649 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 105381888 bytes via buffered write, may repeat 2023-12-19T18:18:47.649 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 106430464 bytes via buffered write, may repeat 2023-12-19T18:18:47.650 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 107479040 bytes via buffered write, may repeat 2023-12-19T18:18:47.650 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 108527616 bytes via buffered write, may repeat 2023-12-19T18:18:47.650 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 109576192 bytes via buffered write, may repeat 2023-12-19T18:18:47.650 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 110624768 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 111673344 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 112721920 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 113770496 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 114819072 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 115867648 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 116916224 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 117964800 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 119013376 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 120061952 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 121110528 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 122159104 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 123207680 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 124256256 bytes via buffered write, may repeat 2023-12-19T18:18:47.651 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 125304832 bytes via buffered write, may repeat 2023-12-19T18:18:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 126353408 bytes via buffered write, may repeat 2023-12-19T18:18:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 127401984 bytes via buffered write, may repeat 2023-12-19T18:18:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 128450560 bytes via buffered write, may repeat 2023-12-19T18:18:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 129499136 bytes via buffered write, may repeat 2023-12-19T18:18:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 130547712 bytes via buffered write, may repeat 2023-12-19T18:18:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 131596288 bytes via buffered write, may repeat 2023-12-19T18:18:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 132644864 bytes via buffered write, may repeat 2023-12-19T18:18:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 133693440 bytes via buffered write, may repeat 2023-12-19T18:18:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 134742016 bytes via buffered write, may repeat 2023-12-19T18:18:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 135790592 bytes via buffered write, may repeat 2023-12-19T18:18:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 136839168 bytes via buffered write, may repeat 2023-12-19T18:18:47.652 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 137887744 bytes via buffered write, may repeat 2023-12-19T18:18:47.653 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 138936320 bytes via buffered write, may repeat 2023-12-19T18:18:47.653 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 139984896 bytes via buffered write, may repeat 2023-12-19T18:18:47.653 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 141033472 bytes via buffered write, may repeat 2023-12-19T18:18:47.653 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 142082048 bytes via buffered write, may repeat 2023-12-19T18:18:47.653 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 143130624 bytes via buffered write, may repeat 2023-12-19T18:18:47.653 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 144179200 bytes via buffered write, may repeat 2023-12-19T18:18:47.653 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 145227776 bytes via buffered write, may repeat 2023-12-19T18:18:47.653 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 146276352 bytes via buffered write, may repeat 2023-12-19T18:18:47.653 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 147324928 bytes via buffered write, may repeat 2023-12-19T18:18:47.653 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 148373504 bytes via buffered write, may repeat 2023-12-19T18:18:47.653 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 149422080 bytes via buffered write, may repeat 2023-12-19T18:18:47.653 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 150470656 bytes via buffered write, may repeat 2023-12-19T18:18:47.653 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 151519232 bytes via buffered write, may repeat 2023-12-19T18:18:47.654 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 152567808 bytes via buffered write, may repeat 2023-12-19T18:18:47.654 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 153616384 bytes via buffered write, may repeat 2023-12-19T18:18:47.654 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 154664960 bytes via buffered write, may repeat 2023-12-19T18:18:47.654 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 155713536 bytes via buffered write, may repeat 2023-12-19T18:18:47.654 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 156762112 bytes via buffered write, may repeat 2023-12-19T18:18:47.654 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 157810688 bytes via buffered write, may repeat 2023-12-19T18:18:47.654 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 158859264 bytes via buffered write, may repeat 2023-12-19T18:18:47.654 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 159907840 bytes via buffered write, may repeat 2023-12-19T18:18:47.654 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 160956416 bytes via buffered write, may repeat 2023-12-19T18:18:47.654 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 162004992 bytes via buffered write, may repeat 2023-12-19T18:18:49.030 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 163053568 bytes via buffered write, may repeat 2023-12-19T18:18:49.030 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 164102144 bytes via buffered write, may repeat 2023-12-19T18:18:49.030 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 165150720 bytes via buffered write, may repeat 2023-12-19T18:18:49.030 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 166199296 bytes via buffered write, may repeat 2023-12-19T18:18:49.030 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 167247872 bytes via buffered write, may repeat 2023-12-19T18:18:49.030 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 168296448 bytes via buffered write, may repeat 2023-12-19T18:18:49.030 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 169345024 bytes via buffered write, may repeat 2023-12-19T18:18:49.030 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 170393600 bytes via buffered write, may repeat 2023-12-19T18:18:49.030 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 171442176 bytes via buffered write, may repeat 2023-12-19T18:18:49.030 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 172490752 bytes via buffered write, may repeat 2023-12-19T18:18:49.030 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 173539328 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 174587904 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 175636480 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 176685056 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 177733632 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 178782208 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 179830784 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 180879360 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 181927936 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 182976512 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 184025088 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 185073664 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 186122240 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 187170816 bytes via buffered write, may repeat 2023-12-19T18:18:49.031 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 188219392 bytes via buffered write, may repeat 2023-12-19T18:18:49.032 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 189267968 bytes via buffered write, may repeat 2023-12-19T18:18:49.032 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 190316544 bytes via buffered write, may repeat 2023-12-19T18:18:49.032 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 191365120 bytes via buffered write, may repeat 2023-12-19T18:18:49.032 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 192413696 bytes via buffered write, may repeat 2023-12-19T18:18:49.032 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 193462272 bytes via buffered write, may repeat 2023-12-19T18:18:49.032 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 194510848 bytes via buffered write, may repeat 2023-12-19T18:18:49.032 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 195559424 bytes via buffered write, may repeat 2023-12-19T18:18:49.033 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 196608000 bytes via buffered write, may repeat 2023-12-19T18:18:49.033 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 197656576 bytes via buffered write, may repeat 2023-12-19T18:18:49.033 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 198705152 bytes via buffered write, may repeat 2023-12-19T18:18:49.033 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 199753728 bytes via buffered write, may repeat 2023-12-19T18:18:49.033 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 200802304 bytes via buffered write, may repeat 2023-12-19T18:18:49.034 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 201850880 bytes via buffered write, may repeat 2023-12-19T18:18:49.034 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 202899456 bytes via buffered write, may repeat 2023-12-19T18:18:49.034 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 203948032 bytes via buffered write, may repeat 2023-12-19T18:18:49.034 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 204996608 bytes via buffered write, may repeat 2023-12-19T18:18:49.034 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 206045184 bytes via buffered write, may repeat 2023-12-19T18:18:49.034 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 207093760 bytes via buffered write, may repeat 2023-12-19T18:18:49.034 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 208142336 bytes via buffered write, may repeat 2023-12-19T18:18:49.035 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 209190912 bytes via buffered write, may repeat 2023-12-19T18:18:49.035 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 210239488 bytes via buffered write, may repeat 2023-12-19T18:18:49.035 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 211288064 bytes via buffered write, may repeat 2023-12-19T18:18:49.035 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 212336640 bytes via buffered write, may repeat 2023-12-19T18:18:49.035 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 213385216 bytes via buffered write, may repeat 2023-12-19T18:18:49.036 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 214433792 bytes via buffered write, may repeat 2023-12-19T18:18:49.036 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 215482368 bytes via buffered write, may repeat 2023-12-19T18:18:49.036 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 216530944 bytes via buffered write, may repeat 2023-12-19T18:18:49.036 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 217579520 bytes via buffered write, may repeat 2023-12-19T18:18:49.036 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 218628096 bytes via buffered write, may repeat 2023-12-19T18:18:49.036 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 219676672 bytes via buffered write, may repeat 2023-12-19T18:18:49.037 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 220725248 bytes via buffered write, may repeat 2023-12-19T18:18:49.037 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 221773824 bytes via buffered write, may repeat 2023-12-19T18:18:49.037 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 222822400 bytes via buffered write, may repeat 2023-12-19T18:18:49.037 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 223870976 bytes via buffered write, may repeat 2023-12-19T18:18:49.037 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 224919552 bytes via buffered write, may repeat 2023-12-19T18:18:49.037 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 225968128 bytes via buffered write, may repeat 2023-12-19T18:18:49.038 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 227016704 bytes via buffered write, may repeat 2023-12-19T18:18:49.038 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 228065280 bytes via buffered write, may repeat 2023-12-19T18:18:49.038 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 229113856 bytes via buffered write, may repeat 2023-12-19T18:18:49.038 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 230162432 bytes via buffered write, may repeat 2023-12-19T18:18:49.038 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 231211008 bytes via buffered write, may repeat 2023-12-19T18:18:49.038 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 232259584 bytes via buffered write, may repeat 2023-12-19T18:18:49.038 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 233308160 bytes via buffered write, may repeat 2023-12-19T18:18:49.038 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 234356736 bytes via buffered write, may repeat 2023-12-19T18:18:49.038 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 235405312 bytes via buffered write, may repeat 2023-12-19T18:18:49.038 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 236453888 bytes via buffered write, may repeat 2023-12-19T18:18:49.038 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 237502464 bytes via buffered write, may repeat 2023-12-19T18:18:49.038 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 238551040 bytes via buffered write, may repeat 2023-12-19T18:18:49.038 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 239599616 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 240648192 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 241696768 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 242745344 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 243793920 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 244842496 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 245891072 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 246939648 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 247988224 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 249036800 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 250085376 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 251133952 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 252182528 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 253231104 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 254279680 bytes via buffered write, may repeat 2023-12-19T18:18:49.039 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 255328256 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 256376832 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 257425408 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 258473984 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 259522560 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 260571136 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 261619712 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 262668288 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 263716864 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 264765440 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 265814016 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 266862592 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 267911168 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 268959744 bytes via buffered write, may repeat 2023-12-19T18:18:49.040 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 270008320 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 271056896 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 272105472 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 273154048 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 274202624 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 275251200 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 276299776 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 277348352 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 278396928 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 279445504 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 280494080 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 281542656 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 282591232 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 283639808 bytes via buffered write, may repeat 2023-12-19T18:18:49.041 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 284688384 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 285736960 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 286785536 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 287834112 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 288882688 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 289931264 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 290979840 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 292028416 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 293076992 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 294125568 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 295174144 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 296222720 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 297271296 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 298319872 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 299368448 bytes via buffered write, may repeat 2023-12-19T18:18:49.042 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 300417024 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 301465600 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 302514176 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 303562752 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 304611328 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 305659904 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 306708480 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 307757056 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 308805632 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 309854208 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 310902784 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 311951360 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 312999936 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 314048512 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 315097088 bytes via buffered write, may repeat 2023-12-19T18:18:49.043 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 316145664 bytes via buffered write, may repeat 2023-12-19T18:18:49.044 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 317194240 bytes via buffered write, may repeat 2023-12-19T18:18:49.044 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 318242816 bytes via buffered write, may repeat 2023-12-19T18:18:49.044 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 319291392 bytes via buffered write, may repeat 2023-12-19T18:18:49.044 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 320339968 bytes via buffered write, may repeat 2023-12-19T18:18:49.044 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 321388544 bytes via buffered write, may repeat 2023-12-19T18:18:49.044 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 322437120 bytes via buffered write, may repeat 2023-12-19T18:18:49.044 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 323485696 bytes via buffered write, may repeat 2023-12-19T18:18:50.217 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 324534272 bytes via buffered write, may repeat 2023-12-19T18:18:50.217 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 325582848 bytes via buffered write, may repeat 2023-12-19T18:18:50.217 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 326631424 bytes via buffered write, may repeat 2023-12-19T18:18:50.217 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 327680000 bytes via buffered write, may repeat 2023-12-19T18:18:50.217 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 328728576 bytes via buffered write, may repeat 2023-12-19T18:18:50.217 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 329777152 bytes via buffered write, may repeat 2023-12-19T18:18:50.217 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 330825728 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 331874304 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 332922880 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 333971456 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 335020032 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 336068608 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 337117184 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 338165760 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 339214336 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 340262912 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 341311488 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 342360064 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 343408640 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 344457216 bytes via buffered write, may repeat 2023-12-19T18:18:50.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 345505792 bytes via buffered write, may repeat 2023-12-19T18:18:50.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 346554368 bytes via buffered write, may repeat 2023-12-19T18:18:50.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 347602944 bytes via buffered write, may repeat 2023-12-19T18:18:50.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 348651520 bytes via buffered write, may repeat 2023-12-19T18:18:50.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 349700096 bytes via buffered write, may repeat 2023-12-19T18:18:50.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 350748672 bytes via buffered write, may repeat 2023-12-19T18:18:50.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 351797248 bytes via buffered write, may repeat 2023-12-19T18:18:50.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 352845824 bytes via buffered write, may repeat 2023-12-19T18:18:50.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 353894400 bytes via buffered write, may repeat 2023-12-19T18:18:50.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 354942976 bytes via buffered write, may repeat 2023-12-19T18:18:50.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 355991552 bytes via buffered write, may repeat 2023-12-19T18:18:50.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 357040128 bytes via buffered write, may repeat 2023-12-19T18:18:50.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 358088704 bytes via buffered write, may repeat 2023-12-19T18:18:50.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 359137280 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 360185856 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 361234432 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 362283008 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 363331584 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 364380160 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 365428736 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 366477312 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 367525888 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 368574464 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 369623040 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 370671616 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 371720192 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 372768768 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 373817344 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 374865920 bytes via buffered write, may repeat 2023-12-19T18:18:50.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 375914496 bytes via buffered write, may repeat 2023-12-19T18:18:50.221 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 376963072 bytes via buffered write, may repeat 2023-12-19T18:18:50.221 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 378011648 bytes via buffered write, may repeat 2023-12-19T18:18:50.221 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 379060224 bytes via buffered write, may repeat 2023-12-19T18:18:50.221 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 380108800 bytes via buffered write, may repeat 2023-12-19T18:18:50.221 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 381157376 bytes via buffered write, may repeat 2023-12-19T18:18:50.221 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 382205952 bytes via buffered write, may repeat 2023-12-19T18:18:50.221 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 383254528 bytes via buffered write, may repeat 2023-12-19T18:18:50.221 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 384303104 bytes via buffered write, may repeat 2023-12-19T18:18:50.221 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 385351680 bytes via buffered write, may repeat 2023-12-19T18:18:50.221 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 386400256 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 387448832 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 388497408 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 389545984 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 390594560 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 391643136 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 392691712 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 393740288 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 394788864 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 395837440 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 396886016 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 397934592 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 398983168 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 400031744 bytes via buffered write, may repeat 2023-12-19T18:18:50.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 401080320 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 402128896 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 403177472 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 404226048 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 405274624 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 406323200 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 407371776 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 408420352 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 409468928 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 410517504 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 411566080 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 412614656 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 413663232 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 414711808 bytes via buffered write, may repeat 2023-12-19T18:18:50.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 415760384 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 416808960 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 417857536 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 418906112 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 419954688 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 421003264 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 422051840 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 423100416 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 424148992 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 425197568 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 426246144 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 427294720 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 428343296 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 429391872 bytes via buffered write, may repeat 2023-12-19T18:18:50.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 430440448 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 431489024 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 432537600 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 433586176 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 434634752 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 435683328 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 436731904 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 437780480 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 438829056 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 439877632 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 440926208 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 441974784 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 443023360 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 444071936 bytes via buffered write, may repeat 2023-12-19T18:18:50.225 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 445120512 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 446169088 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 447217664 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 448266240 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 449314816 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 450363392 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 451411968 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 452460544 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 453509120 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 454557696 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 455606272 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 456654848 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 457703424 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 458752000 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 459800576 bytes via buffered write, may repeat 2023-12-19T18:18:50.226 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 460849152 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 461897728 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 462946304 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 463994880 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 465043456 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 466092032 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 467140608 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 468189184 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 469237760 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 470286336 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 471334912 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 472383488 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 473432064 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 474480640 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 475529216 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 476577792 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 477626368 bytes via buffered write, may repeat 2023-12-19T18:18:50.227 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 478674944 bytes via buffered write, may repeat 2023-12-19T18:18:50.228 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 479723520 bytes via buffered write, may repeat 2023-12-19T18:18:50.228 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 480772096 bytes via buffered write, may repeat 2023-12-19T18:18:50.228 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 481820672 bytes via buffered write, may repeat 2023-12-19T18:18:50.228 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 482869248 bytes via buffered write, may repeat 2023-12-19T18:18:50.228 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 483917824 bytes via buffered write, may repeat 2023-12-19T18:18:50.228 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 484966400 bytes via buffered write, may repeat 2023-12-19T18:18:51.606 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:18:51.606+0000 7f8f3ae61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-12-19T18:19:01.595 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:19:01.594+0000 7f8f3ae61640 -1 log_channel(cluster) log [ERR] : Health check update: 3 full osd(s) (OSD_FULL) 2023-12-19T18:19:15.613 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:19:15.624 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:19:35.519 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 486014976 bytes via buffered write, may repeat 2023-12-19T18:19:35.519 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 487063552 bytes via buffered write, may repeat 2023-12-19T18:19:35.519 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 488112128 bytes via buffered write, may repeat 2023-12-19T18:19:35.519 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 489160704 bytes via buffered write, may repeat 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 490209280 bytes via buffered write, may repeat 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 491257856 bytes via buffered write, may repeat 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 492306432 bytes via buffered write, may repeat 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 493355008 bytes via buffered write, may repeat 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 494403584 bytes via buffered write, may repeat 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 495452160 bytes via buffered write, may repeat 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 496500736 bytes via buffered write, may repeat 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 497549312 bytes via buffered write, may repeat 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 498597888 bytes via buffered write, may repeat 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 499646464 bytes via buffered write, may repeat 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 500695040 bytes via buffered write, may repeat 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:done writing 500695040 bytes 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:successfully fsync'ed prior to getting full state reported 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:starting buffered write 2023-12-19T18:19:35.520 INFO:teuthology.orchestra.run.smithi027.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-12-19T18:19:35.521 INFO:teuthology.orchestra.run.smithi027.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-12-19T18:19:35.521 INFO:teuthology.orchestra.run.smithi027.stdout:close() returned an error as expected 2023-12-19T18:19:35.650 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:19:35.650 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:19:35.786 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:19:35.909 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:19:35.909 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:19:35.909 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:19:35.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:19:36.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.205+0000 7f4f892b6640 1 Processor -- start 2023-12-19T18:19:36.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.205+0000 7f4f892b6640 1 -- start start 2023-12-19T18:19:36.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.207+0000 7f4f892b6640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4f8415ee30 0x7f4f840c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.207+0000 7f4f892b6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f840afeb0 0x7f4f840c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.207+0000 7f4f892b6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f840cb350 con 0x7f4f841553a0 2023-12-19T18:19:36.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.207+0000 7f4f892b6640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f840cbcf0 con 0x7f4f8415ee30 2023-12-19T18:19:36.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.207+0000 7f4f892b6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f840cb630 con 0x7f4f840afeb0 2023-12-19T18:19:36.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.207+0000 7f4f83fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4f841553a0 0x7f4f8415e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45382/0 (socket says 172.21.15.27:45382) 2023-12-19T18:19:36.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.207+0000 7f4f83fff640 1 -- 172.21.15.27:0/1621484520 learned_addr learned my addr 172.21.15.27:0/1621484520 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:36.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.207+0000 7f4f837fe640 1 --2- 172.21.15.27:0/1621484520 >> v2:172.21.15.192:3300/0 conn(0x7f4f8415ee30 0x7f4f840c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f88ab5640 1 --2- 172.21.15.27:0/1621484520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f840afeb0 0x7f4f840c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1970809374 0 0) 0x7f4f840cb350 con 0x7f4f841553a0 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f70003520 con 0x7f4f841553a0 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2815814824 0 0) 0x7f4f70003520 con 0x7f4f841553a0 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f840afeb0 msgr2=0x7f4f840c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 --2- 172.21.15.27:0/1621484520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f840afeb0 0x7f4f840c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 >> v2:172.21.15.192:3300/0 conn(0x7f4f8415ee30 msgr2=0x7f4f840c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 --2- 172.21.15.27:0/1621484520 >> v2:172.21.15.192:3300/0 conn(0x7f4f8415ee30 0x7f4f840c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f840cb910 con 0x7f4f841553a0 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f4f6c001670 con 0x7f4f841553a0 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 >> v1:172.21.15.27:6789/0 conn(0x7f4f841553a0 legacy=0x7f4f8415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 --2- 172.21.15.27:0/1621484520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f70003970 0x7f4f70003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 --2- 172.21.15.27:0/1621484520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f840afeb0 0x7f4f700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 --2- 172.21.15.27:0/1621484520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f70004d00 0x7f4f70005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f840cbcf0 con 0x7f4f70003970 2023-12-19T18:19:36.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f840cb630 con 0x7f4f70004d00 2023-12-19T18:19:36.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.208+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7000ba00 con 0x7f4f840afeb0 2023-12-19T18:19:36.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.209+0000 7f4f83fff640 1 --2- 172.21.15.27:0/1621484520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f70003970 0x7f4f70003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.209+0000 7f4f837fe640 1 --2- 172.21.15.27:0/1621484520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f840afeb0 0x7f4f700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.209+0000 7f4f88ab5640 1 --2- 172.21.15.27:0/1621484520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f70004d00 0x7f4f70005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.209+0000 7f4f83fff640 1 -- 172.21.15.27:0/1621484520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f840afeb0 msgr2=0x7f4f700047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:36.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.209+0000 7f4f83fff640 1 --2- 172.21.15.27:0/1621484520 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f840afeb0 0x7f4f700047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.209+0000 7f4f83fff640 1 -- 172.21.15.27:0/1621484520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f70004d00 msgr2=0x7f4f70005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:36.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.209+0000 7f4f83fff640 1 --2- 172.21.15.27:0/1621484520 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f70004d00 0x7f4f70005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.209+0000 7f4f83fff640 1 -- 172.21.15.27:0/1621484520 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f70003520 con 0x7f4f70003970 2023-12-19T18:19:36.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.210+0000 7f4f83fff640 1 --2- 172.21.15.27:0/1621484520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f70003970 0x7f4f70003d90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00c110 tx=0x7f4f700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d1d0008c1690a4e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:36.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.211+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c005ad0 con 0x7f4f70003970 2023-12-19T18:19:36.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.211+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f6c005c50 con 0x7f4f70003970 2023-12-19T18:19:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.211+0000 7f4f82ffd640 1 -- 172.21.15.27:0/1621484520 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c02c830 con 0x7f4f70003970 2023-12-19T18:19:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.211+0000 7f4f892b6640 1 -- 172.21.15.27:0/1621484520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f70003970 msgr2=0x7f4f70003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.211+0000 7f4f892b6640 1 --2- 172.21.15.27:0/1621484520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f70003970 0x7f4f70003d90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00c110 tx=0x7f4f700057f0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.211+0000 7f4f88ab5640 1 -- 172.21.15.27:0/1621484520 reap_dead start 2023-12-19T18:19:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.212+0000 7f4f892b6640 1 -- 172.21.15.27:0/1621484520 shutdown_connections 2023-12-19T18:19:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.212+0000 7f4f892b6640 1 -- 172.21.15.27:0/1621484520 >> 172.21.15.27:0/1621484520 conn(0x7f4f84069c70 msgr2=0x7f4f8406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.212+0000 7f4f892b6640 1 -- 172.21.15.27:0/1621484520 shutdown_connections 2023-12-19T18:19:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.212+0000 7f4f892b6640 1 -- 172.21.15.27:0/1621484520 wait complete. 2023-12-19T18:19:36.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.214+0000 7f4f892b6640 1 Processor -- start 2023-12-19T18:19:36.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.214+0000 7f4f892b6640 1 -- start start 2023-12-19T18:19:36.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f892b6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f70003970 0x7f4f84153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f892b6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f840b4400 0x7f4f84153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f892b6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f840c7df0 0x7f4f840c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f892b6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f840c6a20 con 0x7f4f840c7df0 2023-12-19T18:19:36.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f892b6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f840c6380 con 0x7f4f70003970 2023-12-19T18:19:36.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f892b6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f840c5a10 con 0x7f4f840b4400 2023-12-19T18:19:36.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f88ab5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f840c7df0 0x7f4f840c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f88ab5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f840c7df0 0x7f4f840c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38336/0 (socket says 172.21.15.27:38336) 2023-12-19T18:19:36.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f88ab5640 1 -- 172.21.15.27:0/743007032 learned_addr learned my addr 172.21.15.27:0/743007032 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:36.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f837fe640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f840b4400 0x7f4f84153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f83fff640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f70003970 0x7f4f84153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f88ab5640 1 -- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f840b4400 msgr2=0x7f4f84153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:36.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f88ab5640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f840b4400 0x7f4f84153590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f88ab5640 1 -- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f70003970 msgr2=0x7f4f84153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:36.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f88ab5640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f70003970 0x7f4f84153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f88ab5640 1 -- 172.21.15.27:0/743007032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f840c5cf0 con 0x7f4f840c7df0 2023-12-19T18:19:36.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.215+0000 7f4f83fff640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f70003970 0x7f4f84153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:36.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.216+0000 7f4f88ab5640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f840c7df0 0x7f4f840c8270 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4f84158d00 tx=0x7f4f7c011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:36.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.216+0000 7f4f80ff9640 1 -- 172.21.15.27:0/743007032 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c004440 con 0x7f4f840c7df0 2023-12-19T18:19:36.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.216+0000 7f4f80ff9640 1 -- 172.21.15.27:0/743007032 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f7c003610 con 0x7f4f840c7df0 2023-12-19T18:19:36.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.216+0000 7f4f80ff9640 1 -- 172.21.15.27:0/743007032 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7c003a00 con 0x7f4f840c7df0 2023-12-19T18:19:36.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.216+0000 7f4f892b6640 1 -- 172.21.15.27:0/743007032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f840c6640 con 0x7f4f840c7df0 2023-12-19T18:19:36.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.216+0000 7f4f892b6640 1 -- 172.21.15.27:0/743007032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f840c87b0 con 0x7f4f840c7df0 2023-12-19T18:19:36.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.218+0000 7f4f892b6640 1 -- 172.21.15.27:0/743007032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f8404e840 con 0x7f4f840c7df0 2023-12-19T18:19:36.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.219+0000 7f4f80ff9640 1 -- 172.21.15.27:0/743007032 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4f7c003b60 con 0x7f4f840c7df0 2023-12-19T18:19:36.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.220+0000 7f4f80ff9640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4f6406a430 0x7f4f6406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.220+0000 7f4f80ff9640 1 -- 172.21.15.27:0/743007032 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4f7c088cc0 con 0x7f4f840c7df0 2023-12-19T18:19:36.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.220+0000 7f4f80ff9640 1 -- 172.21.15.27:0/743007032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=82}) v3 -- 0x7f4f64072020 con 0x7f4f840c7df0 2023-12-19T18:19:36.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.220+0000 7f4f83fff640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4f6406a430 0x7f4f6406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.222+0000 7f4f83fff640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4f6406a430 0x7f4f6406c8f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c034000 tx=0x7f4f6c007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:36.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.224+0000 7f4f80ff9640 1 -- 172.21.15.27:0/743007032 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4f7c054300 con 0x7f4f840c7df0 2023-12-19T18:19:36.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.420+0000 7f4f892b6640 1 -- 172.21.15.27:0/743007032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f840af6a0 con 0x7f4f840c7df0 2023-12-19T18:19:36.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.423+0000 7f4f80ff9640 1 -- 172.21.15.27:0/743007032 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16244 (secure 0 0 0) 0x7f4f7c019030 con 0x7f4f840c7df0 2023-12-19T18:19:36.424 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:19:36.424 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":81,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:18:55.644223+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:24.350728+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:25.147723+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","full","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.14","mappings":[{"from":6,"to":4}]},{"pgid":"9.2c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:19:36.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.426+0000 7f4f892b6640 1 -- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4f6406a430 msgr2=0x7f4f6406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:36.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.426+0000 7f4f892b6640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4f6406a430 0x7f4f6406c8f0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c034000 tx=0x7f4f6c007000 comp rx=0 tx=0).stop 2023-12-19T18:19:36.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.426+0000 7f4f892b6640 1 -- 172.21.15.27:0/743007032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f840c7df0 msgr2=0x7f4f840c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:36.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.426+0000 7f4f892b6640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f840c7df0 0x7f4f840c8270 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f4f84158d00 tx=0x7f4f7c011920 comp rx=0 tx=0).stop 2023-12-19T18:19:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.426+0000 7f4f892b6640 1 -- 172.21.15.27:0/743007032 shutdown_connections 2023-12-19T18:19:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.426+0000 7f4f892b6640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4f6406a430 0x7f4f6406c8f0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.426+0000 7f4f892b6640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f840c7df0 0x7f4f840c8270 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.426+0000 7f4f892b6640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4f840b4400 0x7f4f84153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.426+0000 7f4f892b6640 1 --2- 172.21.15.27:0/743007032 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4f70003970 0x7f4f84153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.426+0000 7f4f892b6640 1 -- 172.21.15.27:0/743007032 >> 172.21.15.27:0/743007032 conn(0x7f4f84069c70 msgr2=0x7f4f840ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.426+0000 7f4f892b6640 1 -- 172.21.15.27:0/743007032 shutdown_connections 2023-12-19T18:19:36.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.426+0000 7f4f892b6640 1 -- 172.21.15.27:0/743007032 wait complete. 2023-12-19T18:19:36.447 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:18:24.350728+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:18:25.147723+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:19:36.447 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:19:36.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.688+0000 7f471759e640 1 Processor -- start 2023-12-19T18:19:36.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.689+0000 7f471759e640 1 -- start start 2023-12-19T18:19:36.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.690+0000 7f471759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f471815e9d0 0x7f47180cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.690+0000 7f471759e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4718152070 0x7f47180cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.690+0000 7f471759e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471815b3e0 con 0x7f4718151c90 2023-12-19T18:19:36.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.690+0000 7f471759e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f471815eed0 con 0x7f4718152070 2023-12-19T18:19:36.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.690+0000 7f471759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471815f1b0 con 0x7f471815e9d0 2023-12-19T18:19:36.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.691+0000 7f471659c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4718151c90 0x7f471806b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45394/0 (socket says 172.21.15.27:45394) 2023-12-19T18:19:36.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.691+0000 7f471659c640 1 -- 172.21.15.27:0/2246708004 learned_addr learned my addr 172.21.15.27:0/2246708004 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:36.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.691+0000 7f4715d9b640 1 --2- 172.21.15.27:0/2246708004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f471815e9d0 0x7f47180cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.691+0000 7f4716d9d640 1 --2- 172.21.15.27:0/2246708004 >> v2:172.21.15.192:3300/0 conn(0x7f4718152070 0x7f47180cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.691+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3233849105 0 0) 0x7f471815b3e0 con 0x7f4718151c90 2023-12-19T18:19:36.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.691+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4704003520 con 0x7f4718151c90 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 770149013 0 0) 0x7f4704003520 con 0x7f4718151c90 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f471815e9d0 msgr2=0x7f47180cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 --2- 172.21.15.27:0/2246708004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f471815e9d0 0x7f47180cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 >> v2:172.21.15.192:3300/0 conn(0x7f4718152070 msgr2=0x7f47180cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 --2- 172.21.15.27:0/2246708004 >> v2:172.21.15.192:3300/0 conn(0x7f4718152070 0x7f47180cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471815f490 con 0x7f4718151c90 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f47080019f0 con 0x7f4718151c90 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 >> v1:172.21.15.27:6789/0 conn(0x7f4718151c90 legacy=0x7f471806b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 --2- 172.21.15.27:0/2246708004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4704003970 0x7f4704003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 --2- 172.21.15.27:0/2246708004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47040047c0 0x7f4704004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 --2- 172.21.15.27:0/2246708004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f471815e9d0 0x7f4704005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471815eed0 con 0x7f4704003970 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f471815f1b0 con 0x7f47040047c0 2023-12-19T18:19:36.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f470400b9d0 con 0x7f471815e9d0 2023-12-19T18:19:36.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.692+0000 7f471659c640 1 --2- 172.21.15.27:0/2246708004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4704003970 0x7f4704003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.693+0000 7f4716d9d640 1 --2- 172.21.15.27:0/2246708004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f471815e9d0 0x7f4704005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.693+0000 7f4715d9b640 1 --2- 172.21.15.27:0/2246708004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47040047c0 0x7f4704004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.693+0000 7f471659c640 1 -- 172.21.15.27:0/2246708004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f471815e9d0 msgr2=0x7f4704005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:36.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.693+0000 7f471659c640 1 --2- 172.21.15.27:0/2246708004 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f471815e9d0 0x7f4704005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.693+0000 7f471659c640 1 -- 172.21.15.27:0/2246708004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47040047c0 msgr2=0x7f4704004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:36.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.693+0000 7f471659c640 1 --2- 172.21.15.27:0/2246708004 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47040047c0 0x7f4704004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.693+0000 7f471659c640 1 -- 172.21.15.27:0/2246708004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4704003520 con 0x7f4704003970 2023-12-19T18:19:36.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.694+0000 7f471659c640 1 --2- 172.21.15.27:0/2246708004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4704003970 0x7f4704003d90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f470800a7b0 tx=0x7f47040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e67c657049352a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:36.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.694+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708034630 con 0x7f4704003970 2023-12-19T18:19:36.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.694+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470802f070 con 0x7f4704003970 2023-12-19T18:19:36.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.694+0000 7f471559a640 1 -- 172.21.15.27:0/2246708004 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47080349d0 con 0x7f4704003970 2023-12-19T18:19:36.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.695+0000 7f471759e640 1 -- 172.21.15.27:0/2246708004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4704003970 msgr2=0x7f4704003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:36.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.695+0000 7f471759e640 1 --2- 172.21.15.27:0/2246708004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4704003970 0x7f4704003d90 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f470800a7b0 tx=0x7f47040057f0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.695+0000 7f4716d9d640 1 -- 172.21.15.27:0/2246708004 reap_dead start 2023-12-19T18:19:36.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.695+0000 7f471759e640 1 -- 172.21.15.27:0/2246708004 shutdown_connections 2023-12-19T18:19:36.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.695+0000 7f471759e640 1 -- 172.21.15.27:0/2246708004 >> 172.21.15.27:0/2246708004 conn(0x7f471806a060 msgr2=0x7f471806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:36.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.695+0000 7f471759e640 1 -- 172.21.15.27:0/2246708004 shutdown_connections 2023-12-19T18:19:36.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.695+0000 7f471759e640 1 -- 172.21.15.27:0/2246708004 wait complete. 2023-12-19T18:19:36.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.697+0000 7f471759e640 1 Processor -- start 2023-12-19T18:19:36.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.697+0000 7f471759e640 1 -- start start 2023-12-19T18:19:36.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.698+0000 7f471759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4704003970 0x7f47180b0310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.698+0000 7f471759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47180aed70 0x7f471815e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.698+0000 7f471759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f471815ba10 0x7f4718144c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.698+0000 7f471759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47180b0850 con 0x7f4704003970 2023-12-19T18:19:36.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.698+0000 7f471759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47180afe80 con 0x7f47180aed70 2023-12-19T18:19:36.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.698+0000 7f471759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47180af270 con 0x7f471815ba10 2023-12-19T18:19:36.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.699+0000 7f471659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4704003970 0x7f47180b0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.699+0000 7f471659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4704003970 0x7f47180b0310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38366/0 (socket says 172.21.15.27:38366) 2023-12-19T18:19:36.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.699+0000 7f471659c640 1 -- 172.21.15.27:0/3569344371 learned_addr learned my addr 172.21.15.27:0/3569344371 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:36.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.699+0000 7f4715d9b640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47180aed70 0x7f471815e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.699+0000 7f4716d9d640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f471815ba10 0x7f4718144c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.699+0000 7f471659c640 1 -- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f471815ba10 msgr2=0x7f4718144c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:36.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.699+0000 7f471659c640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f471815ba10 0x7f4718144c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.699+0000 7f471659c640 1 -- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47180aed70 msgr2=0x7f471815e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:36.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.699+0000 7f471659c640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47180aed70 0x7f471815e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.699+0000 7f471659c640 1 -- 172.21.15.27:0/3569344371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47180af550 con 0x7f4704003970 2023-12-19T18:19:36.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.699+0000 7f4715d9b640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47180aed70 0x7f471815e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:36.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.700+0000 7f471659c640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4704003970 0x7f47180b0310 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f470800a7b0 tx=0x7f4708001360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:36.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.700+0000 7f470effd640 1 -- 172.21.15.27:0/3569344371 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4708001a10 con 0x7f4704003970 2023-12-19T18:19:36.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.700+0000 7f470effd640 1 -- 172.21.15.27:0/3569344371 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4708034640 con 0x7f4704003970 2023-12-19T18:19:36.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.700+0000 7f470effd640 1 -- 172.21.15.27:0/3569344371 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47080349f0 con 0x7f4704003970 2023-12-19T18:19:36.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.700+0000 7f471759e640 1 -- 172.21.15.27:0/3569344371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47180af830 con 0x7f4704003970 2023-12-19T18:19:36.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.700+0000 7f471759e640 1 -- 172.21.15.27:0/3569344371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f471815e590 con 0x7f4704003970 2023-12-19T18:19:36.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.701+0000 7f471759e640 1 -- 172.21.15.27:0/3569344371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471804e840 con 0x7f4704003970 2023-12-19T18:19:36.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.703+0000 7f470effd640 1 -- 172.21.15.27:0/3569344371 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4708031030 con 0x7f4704003970 2023-12-19T18:19:36.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.704+0000 7f470effd640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f46f806a430 0x7f46f806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:36.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.704+0000 7f470effd640 1 -- 172.21.15.27:0/3569344371 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f47080aaa10 con 0x7f4704003970 2023-12-19T18:19:36.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.704+0000 7f470effd640 1 -- 172.21.15.27:0/3569344371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=82}) v3 -- 0x7f46f8072020 con 0x7f4704003970 2023-12-19T18:19:36.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.704+0000 7f4715d9b640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f46f806a430 0x7f46f806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:36.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.706+0000 7f4715d9b640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f46f806a430 0x7f46f806c8f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f471815add0 tx=0x7f4700008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:36.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.709+0000 7f470effd640 1 -- 172.21.15.27:0/3569344371 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4708075f20 con 0x7f4704003970 2023-12-19T18:19:36.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.907+0000 7f471759e640 1 -- 172.21.15.27:0/3569344371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f47180b2130 con 0x7f4704003970 2023-12-19T18:19:36.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.908+0000 7f470effd640 1 -- 172.21.15.27:0/3569344371 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4708043070 con 0x7f4704003970 2023-12-19T18:19:36.908 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:19:36.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.911+0000 7f471759e640 1 -- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f46f806a430 msgr2=0x7f46f806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:36.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.911+0000 7f471759e640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f46f806a430 0x7f46f806c8f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f471815add0 tx=0x7f4700008040 comp rx=0 tx=0).stop 2023-12-19T18:19:36.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.911+0000 7f471759e640 1 -- 172.21.15.27:0/3569344371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4704003970 msgr2=0x7f47180b0310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:36.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.911+0000 7f471759e640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4704003970 0x7f47180b0310 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f470800a7b0 tx=0x7f4708001360 comp rx=0 tx=0).stop 2023-12-19T18:19:36.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.911+0000 7f471759e640 1 -- 172.21.15.27:0/3569344371 shutdown_connections 2023-12-19T18:19:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.911+0000 7f471759e640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f46f806a430 0x7f46f806c8f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.911+0000 7f471759e640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f471815ba10 0x7f4718144c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.911+0000 7f471759e640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f47180aed70 0x7f471815e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.911+0000 7f471759e640 1 --2- 172.21.15.27:0/3569344371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4704003970 0x7f47180b0310 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.911+0000 7f471759e640 1 -- 172.21.15.27:0/3569344371 >> 172.21.15.27:0/3569344371 conn(0x7f471806a060 msgr2=0x7f47180cf020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.912+0000 7f471759e640 1 -- 172.21.15.27:0/3569344371 shutdown_connections 2023-12-19T18:19:36.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:36.912+0000 7f471759e640 1 -- 172.21.15.27:0/3569344371 wait complete. 2023-12-19T18:19:36.932 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:19:37.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.145+0000 7fed8d929640 1 Processor -- start 2023-12-19T18:19:37.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.145+0000 7fed8d929640 1 -- start start 2023-12-19T18:19:37.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.146+0000 7fed8d929640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fed88151c90 0x7fed8806b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:37.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.146+0000 7fed8d929640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fed8815e9d0 0x7fed880cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:37.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.147+0000 7fed8d929640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed8815b3e0 con 0x7fed88152070 2023-12-19T18:19:37.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.147+0000 7fed8d929640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8815eed0 con 0x7fed8815e9d0 2023-12-19T18:19:37.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.147+0000 7fed8d929640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed8815f1b0 con 0x7fed88151c90 2023-12-19T18:19:37.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.147+0000 7fed87fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fed8815e9d0 0x7fed880cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:37.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.147+0000 7fed87fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fed8815e9d0 0x7fed880cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38376/0 (socket says 172.21.15.27:38376) 2023-12-19T18:19:37.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.147+0000 7fed87fff640 1 -- 172.21.15.27:0/3797624472 learned_addr learned my addr 172.21.15.27:0/3797624472 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:37.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.147+0000 7fed8c927640 1 --2- 172.21.15.27:0/3797624472 >> v2:172.21.15.192:3300/0 conn(0x7fed88151c90 0x7fed8806b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:37.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.148+0000 7fed87fff640 1 -- 172.21.15.27:0/3797624472 >> v2:172.21.15.192:3300/0 conn(0x7fed88151c90 msgr2=0x7fed8806b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:37.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.148+0000 7fed87fff640 1 --2- 172.21.15.27:0/3797624472 >> v2:172.21.15.192:3300/0 conn(0x7fed88151c90 0x7fed8806b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.148+0000 7fed87fff640 1 -- 172.21.15.27:0/3797624472 >> v1:172.21.15.27:6789/0 conn(0x7fed88152070 legacy=0x7fed8815b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:37.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.148+0000 7fed87fff640 1 -- 172.21.15.27:0/3797624472 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed8815f490 con 0x7fed8815e9d0 2023-12-19T18:19:37.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.149+0000 7fed87fff640 1 --2- 172.21.15.27:0/3797624472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed8815e9d0 0x7fed880cee40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009870 tx=0x7fed7c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2613bfdf1d167a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:37.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.149+0000 7fed877fe640 1 -- 172.21.15.27:0/3797624472 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c031870 con 0x7fed8815e9d0 2023-12-19T18:19:37.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.149+0000 7fed877fe640 1 -- 172.21.15.27:0/3797624472 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c0319d0 con 0x7fed8815e9d0 2023-12-19T18:19:37.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.149+0000 7fed877fe640 1 -- 172.21.15.27:0/3797624472 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c031d60 con 0x7fed8815e9d0 2023-12-19T18:19:37.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.150+0000 7fed8d929640 1 -- 172.21.15.27:0/3797624472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed8815e9d0 msgr2=0x7fed880cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:37.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.150+0000 7fed8d929640 1 --2- 172.21.15.27:0/3797624472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed8815e9d0 0x7fed880cee40 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fed7c009870 tx=0x7fed7c02b910 comp rx=0 tx=0).stop 2023-12-19T18:19:37.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.150+0000 7fed8d929640 1 -- 172.21.15.27:0/3797624472 shutdown_connections 2023-12-19T18:19:37.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.150+0000 7fed8d929640 1 --2- 172.21.15.27:0/3797624472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed8815e9d0 0x7fed880cee40 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.150+0000 7fed8d929640 1 --2- 172.21.15.27:0/3797624472 >> v2:172.21.15.192:3300/0 conn(0x7fed88151c90 0x7fed8806b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.150+0000 7fed8d929640 1 -- 172.21.15.27:0/3797624472 >> 172.21.15.27:0/3797624472 conn(0x7fed8806a060 msgr2=0x7fed8806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:37.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.150+0000 7fed8d929640 1 -- 172.21.15.27:0/3797624472 shutdown_connections 2023-12-19T18:19:37.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.150+0000 7fed8d929640 1 -- 172.21.15.27:0/3797624472 wait complete. 2023-12-19T18:19:37.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.152+0000 7fed8d929640 1 Processor -- start 2023-12-19T18:19:37.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.152+0000 7fed8d929640 1 -- start start 2023-12-19T18:19:37.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.153+0000 7fed8d929640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed88151c90 0x7fed88154db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:37.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.153+0000 7fed8d929640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88152070 0x7fed8815d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:37.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.153+0000 7fed8d929640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed8815e9d0 0x7fed8815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:37.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.153+0000 7fed8d929640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed881552f0 con 0x7fed88152070 2023-12-19T18:19:37.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.153+0000 7fed8d929640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed88153620 con 0x7fed8815e9d0 2023-12-19T18:19:37.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.153+0000 7fed8d929640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed88153900 con 0x7fed88151c90 2023-12-19T18:19:37.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.153+0000 7fed87fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88152070 0x7fed8815d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:37.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.153+0000 7fed87fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88152070 0x7fed8815d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38384/0 (socket says 172.21.15.27:38384) 2023-12-19T18:19:37.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.153+0000 7fed87fff640 1 -- 172.21.15.27:0/3021988626 learned_addr learned my addr 172.21.15.27:0/3021988626 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:37.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.154+0000 7fed8d128640 1 --2- 172.21.15.27:0/3021988626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed8815e9d0 0x7fed8815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:37.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.154+0000 7fed87fff640 1 -- 172.21.15.27:0/3021988626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed88151c90 msgr2=0x7fed88154db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:37.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.154+0000 7fed87fff640 1 --2- 172.21.15.27:0/3021988626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed88151c90 0x7fed88154db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.154+0000 7fed87fff640 1 -- 172.21.15.27:0/3021988626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed8815e9d0 msgr2=0x7fed8815d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:37.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.154+0000 7fed87fff640 1 --2- 172.21.15.27:0/3021988626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed8815e9d0 0x7fed8815d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.154+0000 7fed87fff640 1 -- 172.21.15.27:0/3021988626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed7c0094d0 con 0x7fed88152070 2023-12-19T18:19:37.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.154+0000 7fed87fff640 1 --2- 172.21.15.27:0/3021988626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88152070 0x7fed8815d380 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fed7c000c00 tx=0x7fed7c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:37.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.154+0000 7fed857fa640 1 -- 172.21.15.27:0/3021988626 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c03abe0 con 0x7fed88152070 2023-12-19T18:19:37.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.155+0000 7fed857fa640 1 -- 172.21.15.27:0/3021988626 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed7c03ad40 con 0x7fed88152070 2023-12-19T18:19:37.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.155+0000 7fed857fa640 1 -- 172.21.15.27:0/3021988626 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed7c044630 con 0x7fed88152070 2023-12-19T18:19:37.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.155+0000 7fed8d929640 1 -- 172.21.15.27:0/3021988626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed88153be0 con 0x7fed88152070 2023-12-19T18:19:37.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.155+0000 7fed8d929640 1 -- 172.21.15.27:0/3021988626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed881541a0 con 0x7fed88152070 2023-12-19T18:19:37.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.156+0000 7fed8d929640 1 -- 172.21.15.27:0/3021988626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed8806b340 con 0x7fed88152070 2023-12-19T18:19:37.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.157+0000 7fed857fa640 1 -- 172.21.15.27:0/3021988626 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fed7c04d450 con 0x7fed88152070 2023-12-19T18:19:37.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.158+0000 7fed857fa640 1 --2- 172.21.15.27:0/3021988626 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed7406a430 0x7fed7406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:37.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.158+0000 7fed857fa640 1 -- 172.21.15.27:0/3021988626 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fed7c02d040 con 0x7fed88152070 2023-12-19T18:19:37.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.159+0000 7fed857fa640 1 -- 172.21.15.27:0/3021988626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=82}) v3 -- 0x7fed74072020 con 0x7fed88152070 2023-12-19T18:19:37.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.159+0000 7fed8c927640 1 --2- 172.21.15.27:0/3021988626 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed7406a430 0x7fed7406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:37.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.160+0000 7fed8c927640 1 --2- 172.21.15.27:0/3021988626 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed7406a430 0x7fed7406c8f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fed78006a90 tx=0x7fed78006a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:37.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.163+0000 7fed857fa640 1 -- 172.21.15.27:0/3021988626 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fed7c035060 con 0x7fed88152070 2023-12-19T18:19:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.416+0000 7fed8d929640 1 -- 172.21.15.27:0/3021988626 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed88153a40 con 0x7fed88152070 2023-12-19T18:19:37.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.417+0000 7fed857fa640 1 -- 172.21.15.27:0/3021988626 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v81) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed88153a40 con 0x7fed88152070 2023-12-19T18:19:37.417 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:19:37.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.419+0000 7fed8d929640 1 -- 172.21.15.27:0/3021988626 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed7406a430 msgr2=0x7fed7406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.419+0000 7fed8d929640 1 --2- 172.21.15.27:0/3021988626 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed7406a430 0x7fed7406c8f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fed78006a90 tx=0x7fed78006a20 comp rx=0 tx=0).stop 2023-12-19T18:19:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.419+0000 7fed8d929640 1 -- 172.21.15.27:0/3021988626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88152070 msgr2=0x7fed8815d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.419+0000 7fed8d929640 1 --2- 172.21.15.27:0/3021988626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88152070 0x7fed8815d380 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fed7c000c00 tx=0x7fed7c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.419+0000 7fed8d929640 1 -- 172.21.15.27:0/3021988626 shutdown_connections 2023-12-19T18:19:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.419+0000 7fed8d929640 1 --2- 172.21.15.27:0/3021988626 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fed7406a430 0x7fed7406c8f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.419+0000 7fed8d929640 1 --2- 172.21.15.27:0/3021988626 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fed8815e9d0 0x7fed8815d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.419+0000 7fed8d929640 1 --2- 172.21.15.27:0/3021988626 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fed88152070 0x7fed8815d380 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.419+0000 7fed8d929640 1 --2- 172.21.15.27:0/3021988626 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fed88151c90 0x7fed88154db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.419+0000 7fed8d929640 1 -- 172.21.15.27:0/3021988626 >> 172.21.15.27:0/3021988626 conn(0x7fed8806a060 msgr2=0x7fed8815b6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.419+0000 7fed8d929640 1 -- 172.21.15.27:0/3021988626 shutdown_connections 2023-12-19T18:19:37.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.419+0000 7fed8d929640 1 -- 172.21.15.27:0/3021988626 wait complete. 2023-12-19T18:19:37.437 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:19:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.644+0000 7f426dbb0640 1 Processor -- start 2023-12-19T18:19:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.644+0000 7f426dbb0640 1 -- start start 2023-12-19T18:19:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.646+0000 7f426dbb0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f426815e9d0 0x7f42680cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.646+0000 7f426dbb0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4268152070 0x7f42680cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.646+0000 7f426dbb0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f426815b3e0 con 0x7f4268151c90 2023-12-19T18:19:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.646+0000 7f426dbb0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426815eed0 con 0x7f4268152070 2023-12-19T18:19:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.646+0000 7f426dbb0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f426815f1b0 con 0x7f426815e9d0 2023-12-19T18:19:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.646+0000 7f426cbae640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4268151c90 0x7f426806b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45420/0 (socket says 172.21.15.27:45420) 2023-12-19T18:19:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.646+0000 7f426cbae640 1 -- 172.21.15.27:0/2406189015 learned_addr learned my addr 172.21.15.27:0/2406189015 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.646+0000 7f426d3af640 1 --2- 172.21.15.27:0/2406189015 >> v2:172.21.15.27:3300/0 conn(0x7f4268152070 0x7f42680cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.647+0000 7f4267fff640 1 --2- 172.21.15.27:0/2406189015 >> v2:172.21.15.192:3300/0 conn(0x7f426815e9d0 0x7f42680cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.647+0000 7f42677fe640 1 -- 172.21.15.27:0/2406189015 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3251926680 0 0) 0x7f426815b3e0 con 0x7f4268151c90 2023-12-19T18:19:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.647+0000 7f42677fe640 1 -- 172.21.15.27:0/2406189015 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4258003520 con 0x7f4268151c90 2023-12-19T18:19:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.647+0000 7f426d3af640 1 -- 172.21.15.27:0/2406189015 >> v2:172.21.15.192:3300/0 conn(0x7f426815e9d0 msgr2=0x7f42680cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.647+0000 7f426d3af640 1 --2- 172.21.15.27:0/2406189015 >> v2:172.21.15.192:3300/0 conn(0x7f426815e9d0 0x7f42680cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.647+0000 7f426d3af640 1 -- 172.21.15.27:0/2406189015 >> v1:172.21.15.27:6789/0 conn(0x7f4268151c90 legacy=0x7f426806b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.647+0000 7f426d3af640 1 -- 172.21.15.27:0/2406189015 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426815f490 con 0x7f4268152070 2023-12-19T18:19:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.648+0000 7f426d3af640 1 --2- 172.21.15.27:0/2406189015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4268152070 0x7f42680cf380 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4260009d90 tx=0x7f426002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edc9317cb0934d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.649+0000 7f42677fe640 1 -- 172.21.15.27:0/2406189015 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426002cbd0 con 0x7f4268152070 2023-12-19T18:19:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.649+0000 7f42677fe640 1 -- 172.21.15.27:0/2406189015 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f426002cd30 con 0x7f4268152070 2023-12-19T18:19:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.649+0000 7f42677fe640 1 -- 172.21.15.27:0/2406189015 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426002cea0 con 0x7f4268152070 2023-12-19T18:19:37.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.649+0000 7f426dbb0640 1 -- 172.21.15.27:0/2406189015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4268152070 msgr2=0x7f42680cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:37.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.649+0000 7f426dbb0640 1 --2- 172.21.15.27:0/2406189015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4268152070 0x7f42680cf380 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4260009d90 tx=0x7f426002be20 comp rx=0 tx=0).stop 2023-12-19T18:19:37.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.650+0000 7f426dbb0640 1 -- 172.21.15.27:0/2406189015 shutdown_connections 2023-12-19T18:19:37.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.650+0000 7f426dbb0640 1 --2- 172.21.15.27:0/2406189015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4268152070 0x7f42680cf380 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.650+0000 7f426dbb0640 1 --2- 172.21.15.27:0/2406189015 >> v2:172.21.15.192:3300/0 conn(0x7f426815e9d0 0x7f42680cee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.650+0000 7f426dbb0640 1 -- 172.21.15.27:0/2406189015 >> 172.21.15.27:0/2406189015 conn(0x7f426806a060 msgr2=0x7f426806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:37.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.650+0000 7f426dbb0640 1 -- 172.21.15.27:0/2406189015 shutdown_connections 2023-12-19T18:19:37.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.650+0000 7f426dbb0640 1 -- 172.21.15.27:0/2406189015 wait complete. 2023-12-19T18:19:37.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.652+0000 7f426dbb0640 1 Processor -- start 2023-12-19T18:19:37.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.652+0000 7f426dbb0640 1 -- start start 2023-12-19T18:19:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.653+0000 7f426dbb0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4268151c90 0x7f426815c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:37.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.653+0000 7f426dbb0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4268152070 0x7f426814a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:37.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.653+0000 7f426dbb0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f426815e9d0 0x7f4268154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:37.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.653+0000 7f426dbb0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4268159140 con 0x7f426815e9d0 2023-12-19T18:19:37.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.653+0000 7f426dbb0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4268159420 con 0x7f4268152070 2023-12-19T18:19:37.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.653+0000 7f426dbb0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4268159700 con 0x7f4268151c90 2023-12-19T18:19:37.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.654+0000 7f426d3af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f426815e9d0 0x7f4268154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:37.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.654+0000 7f426d3af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f426815e9d0 0x7f4268154460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38392/0 (socket says 172.21.15.27:38392) 2023-12-19T18:19:37.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.654+0000 7f426d3af640 1 -- 172.21.15.27:0/1351982900 learned_addr learned my addr 172.21.15.27:0/1351982900 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:37.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.654+0000 7f426cbae640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4268151c90 0x7f426815c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:37.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.654+0000 7f4267fff640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4268152070 0x7f426814a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:37.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.655+0000 7f426d3af640 1 -- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4268151c90 msgr2=0x7f426815c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:37.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.655+0000 7f426d3af640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4268151c90 0x7f426815c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.655+0000 7f426d3af640 1 -- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4268152070 msgr2=0x7f426814a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:37.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.655+0000 7f426d3af640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4268152070 0x7f426814a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.655+0000 7f426d3af640 1 -- 172.21.15.27:0/1351982900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4260009a70 con 0x7f426815e9d0 2023-12-19T18:19:37.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.655+0000 7f426d3af640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f426815e9d0 0x7f4268154460 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4260009d90 tx=0x7f4260004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:37.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.655+0000 7f42657fa640 1 -- 172.21.15.27:0/1351982900 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426002cea0 con 0x7f426815e9d0 2023-12-19T18:19:37.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.655+0000 7f42657fa640 1 -- 172.21.15.27:0/1351982900 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4260040650 con 0x7f426815e9d0 2023-12-19T18:19:37.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.655+0000 7f42657fa640 1 -- 172.21.15.27:0/1351982900 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4260040a00 con 0x7f426815e9d0 2023-12-19T18:19:37.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.655+0000 7f426dbb0640 1 -- 172.21.15.27:0/1351982900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42681599e0 con 0x7f426815e9d0 2023-12-19T18:19:37.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.656+0000 7f426dbb0640 1 -- 172.21.15.27:0/1351982900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f426815c7b0 con 0x7f426815e9d0 2023-12-19T18:19:37.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.657+0000 7f426dbb0640 1 -- 172.21.15.27:0/1351982900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4268159f00 con 0x7f426815e9d0 2023-12-19T18:19:37.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.658+0000 7f42657fa640 1 -- 172.21.15.27:0/1351982900 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4260040b60 con 0x7f426815e9d0 2023-12-19T18:19:37.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.659+0000 7f42657fa640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f424c06a460 0x7f424c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:37.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.659+0000 7f42657fa640 1 -- 172.21.15.27:0/1351982900 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f42600ab130 con 0x7f426815e9d0 2023-12-19T18:19:37.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.659+0000 7f42657fa640 1 -- 172.21.15.27:0/1351982900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=82}) v3 -- 0x7f424c072050 con 0x7f426815e9d0 2023-12-19T18:19:37.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.659+0000 7f426cbae640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f424c06a460 0x7f424c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:37.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.660+0000 7f426cbae640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f424c06a460 0x7f424c06c920 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f425c0011e0 tx=0x7f425c007d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:37.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.663+0000 7f42657fa640 1 -- 172.21.15.27:0/1351982900 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4268159f00 con 0x7f426815e9d0 2023-12-19T18:19:37.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.859+0000 7f426dbb0640 1 -- 172.21.15.27:0/1351982900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4268159f00 con 0x7f426815e9d0 2023-12-19T18:19:37.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.860+0000 7f42657fa640 1 -- 172.21.15.27:0/1351982900 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v81) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4268159f00 con 0x7f426815e9d0 2023-12-19T18:19:37.860 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:19:37.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.862+0000 7f426dbb0640 1 -- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f424c06a460 msgr2=0x7f424c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:37.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.862+0000 7f426dbb0640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f424c06a460 0x7f424c06c920 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f425c0011e0 tx=0x7f425c007d30 comp rx=0 tx=0).stop 2023-12-19T18:19:37.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.862+0000 7f426dbb0640 1 -- 172.21.15.27:0/1351982900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f426815e9d0 msgr2=0x7f4268154460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:37.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.862+0000 7f426dbb0640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f426815e9d0 0x7f4268154460 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4260009d90 tx=0x7f4260004a40 comp rx=0 tx=0).stop 2023-12-19T18:19:37.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.862+0000 7f426dbb0640 1 -- 172.21.15.27:0/1351982900 shutdown_connections 2023-12-19T18:19:37.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.862+0000 7f426dbb0640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f424c06a460 0x7f424c06c920 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.862+0000 7f426dbb0640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f426815e9d0 0x7f4268154460 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.862+0000 7f426dbb0640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4268152070 0x7f426814a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.862+0000 7f426dbb0640 1 --2- 172.21.15.27:0/1351982900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4268151c90 0x7f426815c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:37.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.862+0000 7f426dbb0640 1 -- 172.21.15.27:0/1351982900 >> 172.21.15.27:0/1351982900 conn(0x7f426806a060 msgr2=0x7f42680cdba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:37.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.862+0000 7f426dbb0640 1 -- 172.21.15.27:0/1351982900 shutdown_connections 2023-12-19T18:19:37.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:37.863+0000 7f426dbb0640 1 -- 172.21.15.27:0/1351982900 wait complete. 2023-12-19T18:19:37.883 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:19:38.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.115+0000 7f168165b640 1 Processor -- start 2023-12-19T18:19:38.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.115+0000 7f168165b640 1 -- start start 2023-12-19T18:19:38.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.117+0000 7f168165b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f167c0cddf0 0x7f167c06fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.117+0000 7f168165b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f167c06c670 con 0x7f167c0c4150 2023-12-19T18:19:38.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.117+0000 7f168165b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f167c0d0ae0 con 0x7f167c0cf660 2023-12-19T18:19:38.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.117+0000 7f168165b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167c06cdc0 con 0x7f167c0cddf0 2023-12-19T18:19:38.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.117+0000 7f167affd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f167c0c4150 0x7f167c0cd6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45428/0 (socket says 172.21.15.27:45428) 2023-12-19T18:19:38.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.117+0000 7f167affd640 1 -- 172.21.15.27:0/2887523194 learned_addr learned my addr 172.21.15.27:0/2887523194 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:38.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.118+0000 7f167b7fe640 1 --2- 172.21.15.27:0/2887523194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f167c0cddf0 0x7f167c06fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.118+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3133230925 0 0) 0x7f167c06c670 con 0x7f167c0c4150 2023-12-19T18:19:38.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.118+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1668003520 con 0x7f167c0c4150 2023-12-19T18:19:38.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.118+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4033288827 0 0) 0x7f1668003520 con 0x7f167c0c4150 2023-12-19T18:19:38.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.118+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f167c0cddf0 msgr2=0x7f167c06fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:38.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.118+0000 7f1679ffb640 1 --2- 172.21.15.27:0/2887523194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f167c0cddf0 0x7f167c06fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.118+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 >> v1:172.21.15.192:6789/0 conn(0x7f167c0cf660 legacy=0x7f167c165280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:38.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.118+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167c0cfae0 con 0x7f167c0c4150 2023-12-19T18:19:38.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f166c0019f0 con 0x7f167c0c4150 2023-12-19T18:19:38.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 >> v1:172.21.15.27:6789/0 conn(0x7f167c0c4150 legacy=0x7f167c0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:38.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f1679ffb640 1 --2- 172.21.15.27:0/2887523194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16680039a0 0x7f1668003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f1679ffb640 1 --2- 172.21.15.27:0/2887523194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f167c0cddf0 0x7f16680047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f1679ffb640 1 --2- 172.21.15.27:0/2887523194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1668004d00 0x7f1668005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c06cdc0 con 0x7f1668004d00 2023-12-19T18:19:38.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f166800bae0 con 0x7f16680039a0 2023-12-19T18:19:38.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f166800bca0 con 0x7f167c0cddf0 2023-12-19T18:19:38.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f167b7fe640 1 --2- 172.21.15.27:0/2887523194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1668004d00 0x7f1668005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f167affd640 1 --2- 172.21.15.27:0/2887523194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16680039a0 0x7f1668003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f167a7fc640 1 --2- 172.21.15.27:0/2887523194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f167c0cddf0 0x7f16680047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f167b7fe640 1 -- 172.21.15.27:0/2887523194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f167c0cddf0 msgr2=0x7f16680047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:38.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f167b7fe640 1 --2- 172.21.15.27:0/2887523194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f167c0cddf0 0x7f16680047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f167b7fe640 1 -- 172.21.15.27:0/2887523194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16680039a0 msgr2=0x7f1668003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:38.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f167b7fe640 1 --2- 172.21.15.27:0/2887523194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16680039a0 0x7f1668003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.119+0000 7f167b7fe640 1 -- 172.21.15.27:0/2887523194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1668003520 con 0x7f1668004d00 2023-12-19T18:19:38.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.121+0000 7f167b7fe640 1 --2- 172.21.15.27:0/2887523194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1668004d00 0x7f1668005180 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1674009ac0 tx=0x7f16680057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51f070cc393a9b43 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:38.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.121+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167402c6b0 con 0x7f1668004d00 2023-12-19T18:19:38.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.121+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f167402c810 con 0x7f1668004d00 2023-12-19T18:19:38.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.121+0000 7f1679ffb640 1 -- 172.21.15.27:0/2887523194 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167402cba0 con 0x7f1668004d00 2023-12-19T18:19:38.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.122+0000 7f168165b640 1 -- 172.21.15.27:0/2887523194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1668004d00 msgr2=0x7f1668005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:38.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.122+0000 7f168165b640 1 --2- 172.21.15.27:0/2887523194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1668004d00 0x7f1668005180 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f1674009ac0 tx=0x7f16680057f0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.122+0000 7f167b7fe640 1 -- 172.21.15.27:0/2887523194 reap_dead start 2023-12-19T18:19:38.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.122+0000 7f168165b640 1 -- 172.21.15.27:0/2887523194 shutdown_connections 2023-12-19T18:19:38.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.122+0000 7f168165b640 1 -- 172.21.15.27:0/2887523194 >> 172.21.15.27:0/2887523194 conn(0x7f167c06f1d0 msgr2=0x7f167c06f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:38.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.122+0000 7f168165b640 1 -- 172.21.15.27:0/2887523194 shutdown_connections 2023-12-19T18:19:38.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.122+0000 7f168165b640 1 -- 172.21.15.27:0/2887523194 wait complete. 2023-12-19T18:19:38.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.124+0000 7f168165b640 1 Processor -- start 2023-12-19T18:19:38.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.124+0000 7f168165b640 1 -- start start 2023-12-19T18:19:38.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.125+0000 7f168165b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1668004d00 0x7f167c150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.125+0000 7f168165b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f167c15d2d0 0x7f167c0b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.125+0000 7f168165b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f167c0b59d0 0x7f167c0c0830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.125+0000 7f168165b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c1507c0 con 0x7f167c0b59d0 2023-12-19T18:19:38.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.125+0000 7f168165b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f167c0b5e50 con 0x7f1668004d00 2023-12-19T18:19:38.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.125+0000 7f168165b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f167c0c6b50 con 0x7f167c15d2d0 2023-12-19T18:19:38.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.125+0000 7f167b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f167c0b59d0 0x7f167c0c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.125+0000 7f167b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f167c0b59d0 0x7f167c0c0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38420/0 (socket says 172.21.15.27:38420) 2023-12-19T18:19:38.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.125+0000 7f167b7fe640 1 -- 172.21.15.27:0/1748845195 learned_addr learned my addr 172.21.15.27:0/1748845195 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:38.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.126+0000 7f167affd640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1668004d00 0x7f167c150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.126+0000 7f167a7fc640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f167c15d2d0 0x7f167c0b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.126+0000 7f167b7fe640 1 -- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f167c15d2d0 msgr2=0x7f167c0b5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:38.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.126+0000 7f167b7fe640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f167c15d2d0 0x7f167c0b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.126+0000 7f167b7fe640 1 -- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1668004d00 msgr2=0x7f167c150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:38.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.126+0000 7f167b7fe640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1668004d00 0x7f167c150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.126+0000 7f167b7fe640 1 -- 172.21.15.27:0/1748845195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f167c0bfa70 con 0x7f167c0b59d0 2023-12-19T18:19:38.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.126+0000 7f167affd640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1668004d00 0x7f167c150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:38.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.126+0000 7f167b7fe640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f167c0b59d0 0x7f167c0c0830 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1674037040 tx=0x7f1674004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:38.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.126+0000 7f16737fe640 1 -- 172.21.15.27:0/1748845195 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167402c6b0 con 0x7f167c0b59d0 2023-12-19T18:19:38.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.126+0000 7f16737fe640 1 -- 172.21.15.27:0/1748845195 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1674003830 con 0x7f167c0b59d0 2023-12-19T18:19:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.126+0000 7f168165b640 1 -- 172.21.15.27:0/1748845195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f167c0bfd50 con 0x7f167c0b59d0 2023-12-19T18:19:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.126+0000 7f16737fe640 1 -- 172.21.15.27:0/1748845195 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1674003be0 con 0x7f167c0b59d0 2023-12-19T18:19:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.127+0000 7f168165b640 1 -- 172.21.15.27:0/1748845195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f167c0c0310 con 0x7f167c0b59d0 2023-12-19T18:19:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.128+0000 7f168165b640 1 -- 172.21.15.27:0/1748845195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f167c067110 con 0x7f167c0b59d0 2023-12-19T18:19:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.129+0000 7f16737fe640 1 -- 172.21.15.27:0/1748845195 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1674003d40 con 0x7f167c0b59d0 2023-12-19T18:19:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.130+0000 7f16737fe640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f165c06a430 0x7f165c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.130+0000 7f16737fe640 1 -- 172.21.15.27:0/1748845195 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f16740aa450 con 0x7f167c0b59d0 2023-12-19T18:19:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.131+0000 7f16737fe640 1 -- 172.21.15.27:0/1748845195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=82}) v3 -- 0x7f165c072020 con 0x7f167c0b59d0 2023-12-19T18:19:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.131+0000 7f167affd640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f165c06a430 0x7f165c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.132+0000 7f167affd640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f165c06a430 0x7f165c06c8f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f166c002010 tx=0x7f166c001880 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:38.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.135+0000 7f16737fe640 1 -- 172.21.15.27:0/1748845195 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1674075b10 con 0x7f167c0b59d0 2023-12-19T18:19:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.346+0000 7f168165b640 1 -- 172.21.15.27:0/1748845195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f167c0b6d00 con 0x7f167c0b59d0 2023-12-19T18:19:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.347+0000 7f16737fe640 1 -- 172.21.15.27:0/1748845195 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v81) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1674039080 con 0x7f167c0b59d0 2023-12-19T18:19:38.347 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:19:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.349+0000 7f168165b640 1 -- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f165c06a430 msgr2=0x7f165c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.349+0000 7f168165b640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f165c06a430 0x7f165c06c8f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f166c002010 tx=0x7f166c001880 comp rx=0 tx=0).stop 2023-12-19T18:19:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.349+0000 7f168165b640 1 -- 172.21.15.27:0/1748845195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f167c0b59d0 msgr2=0x7f167c0c0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.349+0000 7f168165b640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f167c0b59d0 0x7f167c0c0830 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f1674037040 tx=0x7f1674004730 comp rx=0 tx=0).stop 2023-12-19T18:19:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.349+0000 7f168165b640 1 -- 172.21.15.27:0/1748845195 shutdown_connections 2023-12-19T18:19:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.349+0000 7f168165b640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f165c06a430 0x7f165c06c8f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.349+0000 7f168165b640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f167c0b59d0 0x7f167c0c0830 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.349+0000 7f168165b640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f167c15d2d0 0x7f167c0b5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.349+0000 7f168165b640 1 --2- 172.21.15.27:0/1748845195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1668004d00 0x7f167c150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.349+0000 7f168165b640 1 -- 172.21.15.27:0/1748845195 >> 172.21.15.27:0/1748845195 conn(0x7f167c06f1d0 msgr2=0x7f167c0cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:38.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.350+0000 7f168165b640 1 -- 172.21.15.27:0/1748845195 shutdown_connections 2023-12-19T18:19:38.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.350+0000 7f168165b640 1 -- 172.21.15.27:0/1748845195 wait complete. 2023-12-19T18:19:38.367 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-12-19T18:19:38.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:19:38.368 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-19T18:19:38.481 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:19:38.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:19:38.482 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:19:38.490 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[100298]: fuse finished with error 0 and tester_r 0 2023-12-19T18:19:38.679 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:19:38.679 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:19:38.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:19:38.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.976+0000 7f8665b37640 1 Processor -- start 2023-12-19T18:19:38.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.976+0000 7f8665b37640 1 -- start start 2023-12-19T18:19:38.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.978+0000 7f8665b37640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f866006bbf0 0x7f86600cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.978+0000 7f8665b37640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f866006c880 0x7f86600cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.978+0000 7f8665b37640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866015b180 con 0x7f8660159c40 2023-12-19T18:19:38.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.978+0000 7f8665b37640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86600ca850 con 0x7f866006bbf0 2023-12-19T18:19:38.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.978+0000 7f8665b37640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866015a570 con 0x7f866006c880 2023-12-19T18:19:38.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.978+0000 7f865effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f866006bbf0 0x7f86600cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.979+0000 7f865effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f866006bbf0 0x7f86600cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38432/0 (socket says 172.21.15.27:38432) 2023-12-19T18:19:38.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.979+0000 7f865effd640 1 -- 172.21.15.27:0/1438898278 learned_addr learned my addr 172.21.15.27:0/1438898278 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:38.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.979+0000 7f865ffff640 1 --2- 172.21.15.27:0/1438898278 >> v2:172.21.15.192:3300/0 conn(0x7f866006c880 0x7f86600cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.979+0000 7f865effd640 1 -- 172.21.15.27:0/1438898278 >> v2:172.21.15.192:3300/0 conn(0x7f866006c880 msgr2=0x7f86600cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:38.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.979+0000 7f865effd640 1 --2- 172.21.15.27:0/1438898278 >> v2:172.21.15.192:3300/0 conn(0x7f866006c880 0x7f86600cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.979+0000 7f865effd640 1 -- 172.21.15.27:0/1438898278 >> v1:172.21.15.192:6789/0 conn(0x7f8660159c40 legacy=0x7f866006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:38.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.979+0000 7f865effd640 1 -- 172.21.15.27:0/1438898278 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866015a850 con 0x7f866006bbf0 2023-12-19T18:19:38.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.980+0000 7f865effd640 1 --2- 172.21.15.27:0/1438898278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f866006bbf0 0x7f86600cd800 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f866006f470 tx=0x7f865402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66326a06d6d5aa73 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:38.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.981+0000 7f865e7fc640 1 -- 172.21.15.27:0/1438898278 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654031870 con 0x7f866006bbf0 2023-12-19T18:19:38.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.981+0000 7f865e7fc640 1 -- 172.21.15.27:0/1438898278 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86540319d0 con 0x7f866006bbf0 2023-12-19T18:19:38.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.981+0000 7f865e7fc640 1 -- 172.21.15.27:0/1438898278 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8654031d60 con 0x7f866006bbf0 2023-12-19T18:19:38.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.981+0000 7f8665b37640 1 -- 172.21.15.27:0/1438898278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f866006bbf0 msgr2=0x7f86600cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:38.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.981+0000 7f8665b37640 1 --2- 172.21.15.27:0/1438898278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f866006bbf0 0x7f86600cd800 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f866006f470 tx=0x7f865402b910 comp rx=0 tx=0).stop 2023-12-19T18:19:38.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.982+0000 7f8665b37640 1 -- 172.21.15.27:0/1438898278 shutdown_connections 2023-12-19T18:19:38.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.982+0000 7f8665b37640 1 --2- 172.21.15.27:0/1438898278 >> v2:172.21.15.192:3300/0 conn(0x7f866006c880 0x7f86600cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.982+0000 7f8665b37640 1 --2- 172.21.15.27:0/1438898278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f866006bbf0 0x7f86600cd800 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.982+0000 7f8665b37640 1 -- 172.21.15.27:0/1438898278 >> 172.21.15.27:0/1438898278 conn(0x7f866006a330 msgr2=0x7f86600b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:38.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.982+0000 7f8665b37640 1 -- 172.21.15.27:0/1438898278 shutdown_connections 2023-12-19T18:19:38.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.982+0000 7f8665b37640 1 -- 172.21.15.27:0/1438898278 wait complete. 2023-12-19T18:19:38.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.983+0000 7f8665b37640 1 Processor -- start 2023-12-19T18:19:38.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.984+0000 7f8665b37640 1 -- start start 2023-12-19T18:19:38.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f8665b37640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f866006bbf0 0x7f86601561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f8665b37640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f866006c880 0x7f86600c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f8665b37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8660159c40 0x7f86600ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f8665b37640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86600b4530 con 0x7f866006c880 2023-12-19T18:19:38.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f8665b37640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86600b3f70 con 0x7f8660159c40 2023-12-19T18:19:38.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f8665b37640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86600b4250 con 0x7f866006bbf0 2023-12-19T18:19:38.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f865effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f866006c880 0x7f86600c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f865effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f866006c880 0x7f86600c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38434/0 (socket says 172.21.15.27:38434) 2023-12-19T18:19:38.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f865effd640 1 -- 172.21.15.27:0/3527048517 learned_addr learned my addr 172.21.15.27:0/3527048517 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:38.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f865ffff640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8660159c40 0x7f86600ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f865f7fe640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f866006bbf0 0x7f86601561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f865effd640 1 -- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f866006bbf0 msgr2=0x7f86601561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f865effd640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f866006bbf0 0x7f86601561e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f865effd640 1 -- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8660159c40 msgr2=0x7f86600ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f865effd640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8660159c40 0x7f86600ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.985+0000 7f865effd640 1 -- 172.21.15.27:0/3527048517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86540094d0 con 0x7f866006c880 2023-12-19T18:19:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.986+0000 7f865effd640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f866006c880 0x7f86600c46b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f865402b950 tx=0x7f86540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.986+0000 7f8647fff640 1 -- 172.21.15.27:0/3527048517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865403ad30 con 0x7f866006c880 2023-12-19T18:19:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.986+0000 7f8647fff640 1 -- 172.21.15.27:0/3527048517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86540443f0 con 0x7f866006c880 2023-12-19T18:19:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.986+0000 7f8647fff640 1 -- 172.21.15.27:0/3527048517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86540447a0 con 0x7f866006c880 2023-12-19T18:19:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.986+0000 7f8665b37640 1 -- 172.21.15.27:0/3527048517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86600ae670 con 0x7f866006c880 2023-12-19T18:19:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.986+0000 7f8665b37640 1 -- 172.21.15.27:0/3527048517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8660156720 con 0x7f866006c880 2023-12-19T18:19:38.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.987+0000 7f8665b37640 1 -- 172.21.15.27:0/3527048517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86600aeb90 con 0x7f866006c880 2023-12-19T18:19:38.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.989+0000 7f8647fff640 1 -- 172.21.15.27:0/3527048517 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f865404d450 con 0x7f866006c880 2023-12-19T18:19:38.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.990+0000 7f8647fff640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f864c06a430 0x7f864c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:38.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.990+0000 7f8647fff640 1 -- 172.21.15.27:0/3527048517 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f865402d040 con 0x7f866006c880 2023-12-19T18:19:38.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.991+0000 7f8647fff640 1 -- 172.21.15.27:0/3527048517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=82}) v3 -- 0x7f864c072020 con 0x7f866006c880 2023-12-19T18:19:38.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.991+0000 7f865f7fe640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f864c06a430 0x7f864c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:38.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.992+0000 7f865f7fe640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f864c06a430 0x7f864c06c8f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f865000a750 tx=0x7f8650008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:38.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:38.995+0000 7f8647fff640 1 -- 172.21.15.27:0/3527048517 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8654035020 con 0x7f866006c880 2023-12-19T18:19:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.193+0000 7f8665b37640 1 -- 172.21.15.27:0/3527048517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8660068e90 con 0x7f866006c880 2023-12-19T18:19:39.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.196+0000 7f8647fff640 1 -- 172.21.15.27:0/3527048517 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16244 (secure 0 0 0) 0x7f86600aeb90 con 0x7f866006c880 2023-12-19T18:19:39.196 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:19:39.196 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":81,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:18:55.644223+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:24.350728+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:25.147723+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["backfillfull","exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","full","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","full","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","full","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.14","mappings":[{"from":6,"to":4}]},{"pgid":"9.2c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:19:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.198+0000 7f8665b37640 1 -- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f864c06a430 msgr2=0x7f864c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.198+0000 7f8665b37640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f864c06a430 0x7f864c06c8f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f865000a750 tx=0x7f8650008040 comp rx=0 tx=0).stop 2023-12-19T18:19:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.198+0000 7f8665b37640 1 -- 172.21.15.27:0/3527048517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f866006c880 msgr2=0x7f86600c46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.198+0000 7f8665b37640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f866006c880 0x7f86600c46b0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f865402b950 tx=0x7f86540046c0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.199+0000 7f8665b37640 1 -- 172.21.15.27:0/3527048517 shutdown_connections 2023-12-19T18:19:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.199+0000 7f8665b37640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f864c06a430 0x7f864c06c8f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.199+0000 7f8665b37640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8660159c40 0x7f86600ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.199+0000 7f8665b37640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f866006c880 0x7f86600c46b0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.199+0000 7f8665b37640 1 --2- 172.21.15.27:0/3527048517 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f866006bbf0 0x7f86601561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.199+0000 7f8665b37640 1 -- 172.21.15.27:0/3527048517 >> 172.21.15.27:0/3527048517 conn(0x7f866006a330 msgr2=0x7f866015c530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.199+0000 7f8665b37640 1 -- 172.21.15.27:0/3527048517 shutdown_connections 2023-12-19T18:19:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.199+0000 7f8665b37640 1 -- 172.21.15.27:0/3527048517 wait complete. 2023-12-19T18:19:39.217 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:18:24.350728+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:18:25.147723+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:19:39.218 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:19:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.446+0000 7f6da7cec640 1 Processor -- start 2023-12-19T18:19:39.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.446+0000 7f6da7cec640 1 -- start start 2023-12-19T18:19:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.448+0000 7f6da7cec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6da015b1a0 0x7f6da006f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.448+0000 7f6da7cec640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6da00c2720 0x7f6da006fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.448+0000 7f6da7cec640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da006bc80 con 0x7f6da0071420 2023-12-19T18:19:39.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.448+0000 7f6da7cec640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da015fd90 con 0x7f6da015b1a0 2023-12-19T18:19:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.448+0000 7f6da7cec640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da006d690 con 0x7f6da00c2720 2023-12-19T18:19:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.448+0000 7f6da5a88640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6da015b1a0 0x7f6da006f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.448+0000 7f6da5a88640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6da015b1a0 0x7f6da006f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38436/0 (socket says 172.21.15.27:38436) 2023-12-19T18:19:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.448+0000 7f6da5a88640 1 -- 172.21.15.27:0/4235271804 learned_addr learned my addr 172.21.15.27:0/4235271804 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.448+0000 7f6da6289640 1 --1- 172.21.15.27:0/4235271804 >> v1:172.21.15.27:6789/0 conn(0x7f6da0071420 0x7f6da00c9d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45440/0 (socket says 172.21.15.27:45440) 2023-12-19T18:19:39.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.449+0000 7f6da5287640 1 --2- 172.21.15.27:0/4235271804 >> v2:172.21.15.192:3300/0 conn(0x7f6da00c2720 0x7f6da006fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.449+0000 7f6da5a88640 1 -- 172.21.15.27:0/4235271804 >> v2:172.21.15.192:3300/0 conn(0x7f6da00c2720 msgr2=0x7f6da006fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:39.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.449+0000 7f6da5a88640 1 --2- 172.21.15.27:0/4235271804 >> v2:172.21.15.192:3300/0 conn(0x7f6da00c2720 0x7f6da006fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.449+0000 7f6da5a88640 1 -- 172.21.15.27:0/4235271804 >> v1:172.21.15.27:6789/0 conn(0x7f6da0071420 legacy=0x7f6da00c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:39.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.449+0000 7f6da5a88640 1 -- 172.21.15.27:0/4235271804 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da006d0a0 con 0x7f6da015b1a0 2023-12-19T18:19:39.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.450+0000 7f6da5a88640 1 --2- 172.21.15.27:0/4235271804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da015b1a0 0x7f6da006f520 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6da006acf0 tx=0x7f6d9c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5f763327148f010 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:39.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.450+0000 7f6da4a86640 1 -- 172.21.15.27:0/4235271804 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c031870 con 0x7f6da015b1a0 2023-12-19T18:19:39.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.450+0000 7f6da4a86640 1 -- 172.21.15.27:0/4235271804 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9c0319d0 con 0x7f6da015b1a0 2023-12-19T18:19:39.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.450+0000 7f6da4a86640 1 -- 172.21.15.27:0/4235271804 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9c031d60 con 0x7f6da015b1a0 2023-12-19T18:19:39.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.451+0000 7f6da7cec640 1 -- 172.21.15.27:0/4235271804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da015b1a0 msgr2=0x7f6da006f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:39.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.451+0000 7f6da7cec640 1 --2- 172.21.15.27:0/4235271804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da015b1a0 0x7f6da006f520 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f6da006acf0 tx=0x7f6d9c02b910 comp rx=0 tx=0).stop 2023-12-19T18:19:39.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.451+0000 7f6da7cec640 1 -- 172.21.15.27:0/4235271804 shutdown_connections 2023-12-19T18:19:39.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.451+0000 7f6da7cec640 1 --2- 172.21.15.27:0/4235271804 >> v2:172.21.15.192:3300/0 conn(0x7f6da00c2720 0x7f6da006fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.451+0000 7f6da7cec640 1 --2- 172.21.15.27:0/4235271804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da015b1a0 0x7f6da006f520 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.451+0000 7f6da7cec640 1 -- 172.21.15.27:0/4235271804 >> 172.21.15.27:0/4235271804 conn(0x7f6da00cebd0 msgr2=0x7f6da00cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:39.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.451+0000 7f6da7cec640 1 -- 172.21.15.27:0/4235271804 shutdown_connections 2023-12-19T18:19:39.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.451+0000 7f6da7cec640 1 -- 172.21.15.27:0/4235271804 wait complete. 2023-12-19T18:19:39.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.453+0000 7f6da7cec640 1 Processor -- start 2023-12-19T18:19:39.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.453+0000 7f6da7cec640 1 -- start start 2023-12-19T18:19:39.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.454+0000 7f6da7cec640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da0071420 0x7f6da01f58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.454+0000 7f6da7cec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da00c2720 0x7f6da00c7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.454+0000 7f6da7cec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da015b1a0 0x7f6da00c8520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.454+0000 7f6da7cec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da01f5e30 con 0x7f6da015b1a0 2023-12-19T18:19:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.454+0000 7f6da7cec640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00c6920 con 0x7f6da0071420 2023-12-19T18:19:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.454+0000 7f6da7cec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00c6c00 con 0x7f6da00c2720 2023-12-19T18:19:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.454+0000 7f6da6289640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da015b1a0 0x7f6da00c8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.455+0000 7f6da6289640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da015b1a0 0x7f6da00c8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38446/0 (socket says 172.21.15.27:38446) 2023-12-19T18:19:39.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.455+0000 7f6da6289640 1 -- 172.21.15.27:0/2195884350 learned_addr learned my addr 172.21.15.27:0/2195884350 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.455+0000 7f6da5a88640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da0071420 0x7f6da01f58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.455+0000 7f6da5287640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da00c2720 0x7f6da00c7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.455+0000 7f6da6289640 1 -- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da00c2720 msgr2=0x7f6da00c7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.455+0000 7f6da6289640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da00c2720 0x7f6da00c7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.455+0000 7f6da6289640 1 -- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da0071420 msgr2=0x7f6da01f58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.455+0000 7f6da6289640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da0071420 0x7f6da01f58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.455+0000 7f6da6289640 1 -- 172.21.15.27:0/2195884350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d9c0094d0 con 0x7f6da015b1a0 2023-12-19T18:19:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.455+0000 7f6da5a88640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da0071420 0x7f6da01f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.455+0000 7f6da6289640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da015b1a0 0x7f6da00c8520 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c7a0 tx=0x7f6d9800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:39.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.456+0000 7f6d967fc640 1 -- 172.21.15.27:0/2195884350 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9800a640 con 0x7f6da015b1a0 2023-12-19T18:19:39.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.456+0000 7f6d967fc640 1 -- 172.21.15.27:0/2195884350 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9800a7a0 con 0x7f6da015b1a0 2023-12-19T18:19:39.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.456+0000 7f6d967fc640 1 -- 172.21.15.27:0/2195884350 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9800ab10 con 0x7f6da015b1a0 2023-12-19T18:19:39.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.456+0000 7f6da7cec640 1 -- 172.21.15.27:0/2195884350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da00c6ee0 con 0x7f6da015b1a0 2023-12-19T18:19:39.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.456+0000 7f6da7cec640 1 -- 172.21.15.27:0/2195884350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da00c74a0 con 0x7f6da015b1a0 2023-12-19T18:19:39.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.457+0000 7f6da7cec640 1 -- 172.21.15.27:0/2195884350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da00c7690 con 0x7f6da015b1a0 2023-12-19T18:19:39.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.458+0000 7f6d967fc640 1 -- 172.21.15.27:0/2195884350 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6d9800ac70 con 0x7f6da015b1a0 2023-12-19T18:19:39.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.459+0000 7f6d967fc640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6d8406a430 0x7f6d8406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.460+0000 7f6d967fc640 1 -- 172.21.15.27:0/2195884350 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6d98087fb0 con 0x7f6da015b1a0 2023-12-19T18:19:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.460+0000 7f6d967fc640 1 -- 172.21.15.27:0/2195884350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=82}) v3 -- 0x7f6d84072020 con 0x7f6da015b1a0 2023-12-19T18:19:39.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.460+0000 7f6da5a88640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6d8406a430 0x7f6d8406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.461+0000 7f6da5a88640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6d8406a430 0x7f6d8406c8f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c036000 tx=0x7f6d9c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:39.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.464+0000 7f6d967fc640 1 -- 172.21.15.27:0/2195884350 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6d98017060 con 0x7f6da015b1a0 2023-12-19T18:19:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.655+0000 7f6da7cec640 1 -- 172.21.15.27:0/2195884350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6da006bc80 con 0x7f6da015b1a0 2023-12-19T18:19:39.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.656+0000 7f6d967fc640 1 -- 172.21.15.27:0/2195884350 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6da006bc80 con 0x7f6da015b1a0 2023-12-19T18:19:39.656 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:19:39.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.658+0000 7f6da7cec640 1 -- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6d8406a430 msgr2=0x7f6d8406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:39.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.658+0000 7f6da7cec640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6d8406a430 0x7f6d8406c8f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c036000 tx=0x7f6d9c03c040 comp rx=0 tx=0).stop 2023-12-19T18:19:39.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.658+0000 7f6da7cec640 1 -- 172.21.15.27:0/2195884350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da015b1a0 msgr2=0x7f6da00c8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:39.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.658+0000 7f6da7cec640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da015b1a0 0x7f6da00c8520 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6d9800c7a0 tx=0x7f6d9800cc60 comp rx=0 tx=0).stop 2023-12-19T18:19:39.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.658+0000 7f6da7cec640 1 -- 172.21.15.27:0/2195884350 shutdown_connections 2023-12-19T18:19:39.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.658+0000 7f6da7cec640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6d8406a430 0x7f6d8406c8f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.658+0000 7f6da7cec640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6da015b1a0 0x7f6da00c8520 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.658+0000 7f6da7cec640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6da00c2720 0x7f6da00c7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.658+0000 7f6da7cec640 1 --2- 172.21.15.27:0/2195884350 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6da0071420 0x7f6da01f58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.658+0000 7f6da7cec640 1 -- 172.21.15.27:0/2195884350 >> 172.21.15.27:0/2195884350 conn(0x7f6da00cebd0 msgr2=0x7f6da00cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:39.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.659+0000 7f6da7cec640 1 -- 172.21.15.27:0/2195884350 shutdown_connections 2023-12-19T18:19:39.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.659+0000 7f6da7cec640 1 -- 172.21.15.27:0/2195884350 wait complete. 2023-12-19T18:19:39.667 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:19:39.667+0000 7f8f3ae61640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2023-12-19T18:19:39.682 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:19:39.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.902+0000 7f021a7d6640 1 Processor -- start 2023-12-19T18:19:39.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.902+0000 7f021a7d6640 1 -- start start 2023-12-19T18:19:39.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.903+0000 7f021a7d6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f021415aed0 0x7f02140cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.903+0000 7f021a7d6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02140b3fd0 con 0x7f02140d1450 2023-12-19T18:19:39.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.903+0000 7f021a7d6640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02140cf6f0 con 0x7f02140cdaa0 2023-12-19T18:19:39.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.903+0000 7f021a7d6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02140cf9d0 con 0x7f021415aed0 2023-12-19T18:19:39.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.904+0000 7f0218fd3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f02140d1450 0x7f02140cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45446/0 (socket says 172.21.15.27:45446) 2023-12-19T18:19:39.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.904+0000 7f0218fd3640 1 -- 172.21.15.27:0/3922985903 learned_addr learned my addr 172.21.15.27:0/3922985903 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:39.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.904+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960625342 0 0) 0x7f02140b3fd0 con 0x7f02140d1450 2023-12-19T18:19:39.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.904+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01fc003520 con 0x7f02140d1450 2023-12-19T18:19:39.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.904+0000 7f02197d4640 1 --2- 172.21.15.27:0/3922985903 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f021415aed0 0x7f02140cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1340255759 0 0) 0x7f01fc003520 con 0x7f02140d1450 2023-12-19T18:19:39.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f021415aed0 msgr2=0x7f02140cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:39.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0213fff640 1 --2- 172.21.15.27:0/3922985903 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f021415aed0 0x7f02140cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 >> v1:172.21.15.192:6789/0 conn(0x7f02140cdaa0 legacy=0x7f02140b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:39.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02140b3980 con 0x7f02140d1450 2023-12-19T18:19:39.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f0208001670 con 0x7f02140d1450 2023-12-19T18:19:39.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 >> v1:172.21.15.27:6789/0 conn(0x7f02140d1450 legacy=0x7f02140cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:39.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0213fff640 1 --2- 172.21.15.27:0/3922985903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01fc0039a0 0x7f01fc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0213fff640 1 --2- 172.21.15.27:0/3922985903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01fc0047c0 0x7f01fc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0213fff640 1 --2- 172.21.15.27:0/3922985903 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01fc005250 0x7f01fc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02140cf9d0 con 0x7f01fc0047c0 2023-12-19T18:19:39.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc00bfa0 con 0x7f01fc0039a0 2023-12-19T18:19:39.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01fc00c160 con 0x7f01fc005250 2023-12-19T18:19:39.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0218fd3640 1 --2- 172.21.15.27:0/3922985903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01fc0047c0 0x7f01fc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f02197d4640 1 --2- 172.21.15.27:0/3922985903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01fc0039a0 0x7f01fc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.905+0000 7f0219fd5640 1 --2- 172.21.15.27:0/3922985903 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01fc005250 0x7f01fc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.906+0000 7f0218fd3640 1 -- 172.21.15.27:0/3922985903 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01fc005250 msgr2=0x7f01fc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:39.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.906+0000 7f0218fd3640 1 --2- 172.21.15.27:0/3922985903 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01fc005250 0x7f01fc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.906+0000 7f0218fd3640 1 -- 172.21.15.27:0/3922985903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01fc0039a0 msgr2=0x7f01fc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:39.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.906+0000 7f0218fd3640 1 --2- 172.21.15.27:0/3922985903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01fc0039a0 0x7f01fc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.906+0000 7f0218fd3640 1 -- 172.21.15.27:0/3922985903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01fc003520 con 0x7f01fc0047c0 2023-12-19T18:19:39.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.907+0000 7f0218fd3640 1 --2- 172.21.15.27:0/3922985903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01fc0047c0 0x7f01fc004c40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f020800c0e0 tx=0x7f01fc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d8c0502911d03d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:39.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.907+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0208005ad0 con 0x7f01fc0047c0 2023-12-19T18:19:39.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.907+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0208005c30 con 0x7f01fc0047c0 2023-12-19T18:19:39.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.907+0000 7f0213fff640 1 -- 172.21.15.27:0/3922985903 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020802c810 con 0x7f01fc0047c0 2023-12-19T18:19:39.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.908+0000 7f021a7d6640 1 -- 172.21.15.27:0/3922985903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01fc0047c0 msgr2=0x7f01fc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:39.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.908+0000 7f021a7d6640 1 --2- 172.21.15.27:0/3922985903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01fc0047c0 0x7f01fc004c40 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f020800c0e0 tx=0x7f01fc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.908+0000 7f0219fd5640 1 -- 172.21.15.27:0/3922985903 reap_dead start 2023-12-19T18:19:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.908+0000 7f021a7d6640 1 -- 172.21.15.27:0/3922985903 shutdown_connections 2023-12-19T18:19:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.908+0000 7f021a7d6640 1 -- 172.21.15.27:0/3922985903 >> 172.21.15.27:0/3922985903 conn(0x7f0214099ee0 msgr2=0x7f02140d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.908+0000 7f021a7d6640 1 -- 172.21.15.27:0/3922985903 shutdown_connections 2023-12-19T18:19:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.908+0000 7f021a7d6640 1 -- 172.21.15.27:0/3922985903 wait complete. 2023-12-19T18:19:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.910+0000 7f021a7d6640 1 Processor -- start 2023-12-19T18:19:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.910+0000 7f021a7d6640 1 -- start start 2023-12-19T18:19:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f021a7d6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01fc0047c0 0x7f02140c6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f021a7d6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0214158160 0x7f02140ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f021a7d6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02140c4640 0x7f02140c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f021a7d6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0214156b90 con 0x7f01fc0047c0 2023-12-19T18:19:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f021a7d6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0214157150 con 0x7f02140c4640 2023-12-19T18:19:39.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f021a7d6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02140c6a80 con 0x7f0214158160 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f02197d4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01fc0047c0 0x7f02140c6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f02197d4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01fc0047c0 0x7f02140c6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38462/0 (socket says 172.21.15.27:38462) 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f02197d4640 1 -- 172.21.15.27:0/2438735046 learned_addr learned my addr 172.21.15.27:0/2438735046 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f0219fd5640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02140c4640 0x7f02140c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f0218fd3640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0214158160 0x7f02140ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f02197d4640 1 -- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0214158160 msgr2=0x7f02140ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f02197d4640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0214158160 0x7f02140ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f02197d4640 1 -- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02140c4640 msgr2=0x7f02140c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f02197d4640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02140c4640 0x7f02140c4ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.911+0000 7f02197d4640 1 -- 172.21.15.27:0/2438735046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02140c6d60 con 0x7f01fc0047c0 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.912+0000 7f02197d4640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01fc0047c0 0x7f02140c6180 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0204009cb0 tx=0x7f0204007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.912+0000 7f0211ffb640 1 -- 172.21.15.27:0/2438735046 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204014610 con 0x7f01fc0047c0 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.912+0000 7f0211ffb640 1 -- 172.21.15.27:0/2438735046 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020400f050 con 0x7f01fc0047c0 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.912+0000 7f0211ffb640 1 -- 172.21.15.27:0/2438735046 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0204014990 con 0x7f01fc0047c0 2023-12-19T18:19:39.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.912+0000 7f021a7d6640 1 -- 172.21.15.27:0/2438735046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0214156de0 con 0x7f01fc0047c0 2023-12-19T18:19:39.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.912+0000 7f021a7d6640 1 -- 172.21.15.27:0/2438735046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02140ca7c0 con 0x7f01fc0047c0 2023-12-19T18:19:39.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.913+0000 7f021a7d6640 1 -- 172.21.15.27:0/2438735046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02140ca9b0 con 0x7f01fc0047c0 2023-12-19T18:19:39.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.914+0000 7f0211ffb640 1 -- 172.21.15.27:0/2438735046 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0204014af0 con 0x7f01fc0047c0 2023-12-19T18:19:39.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.916+0000 7f0211ffb640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020006a460 0x7f020006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:39.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.916+0000 7f0211ffb640 1 -- 172.21.15.27:0/2438735046 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0204089220 con 0x7f01fc0047c0 2023-12-19T18:19:39.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.916+0000 7f0211ffb640 1 -- 172.21.15.27:0/2438735046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=83}) v3 -- 0x7f0200072050 con 0x7f01fc0047c0 2023-12-19T18:19:39.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.916+0000 7f0218fd3640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020006a460 0x7f020006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:39.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.917+0000 7f0218fd3640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020006a460 0x7f020006c920 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0208034000 tx=0x7f0208009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:39.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:39.920+0000 7f0211ffb640 1 -- 172.21.15.27:0/2438735046 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0204054890 con 0x7f01fc0047c0 2023-12-19T18:19:40.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.113+0000 7f021a7d6640 1 -- 172.21.15.27:0/2438735046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f02140c6fa0 con 0x7f01fc0047c0 2023-12-19T18:19:40.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.114+0000 7f0211ffb640 1 -- 172.21.15.27:0/2438735046 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v82) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0204057eb0 con 0x7f01fc0047c0 2023-12-19T18:19:40.118 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:19:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.116+0000 7f021a7d6640 1 -- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020006a460 msgr2=0x7f020006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.116+0000 7f021a7d6640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020006a460 0x7f020006c920 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f0208034000 tx=0x7f0208009030 comp rx=0 tx=0).stop 2023-12-19T18:19:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.116+0000 7f021a7d6640 1 -- 172.21.15.27:0/2438735046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01fc0047c0 msgr2=0x7f02140c6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.116+0000 7f021a7d6640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01fc0047c0 0x7f02140c6180 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0204009cb0 tx=0x7f0204007700 comp rx=0 tx=0).stop 2023-12-19T18:19:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.116+0000 7f021a7d6640 1 -- 172.21.15.27:0/2438735046 shutdown_connections 2023-12-19T18:19:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.116+0000 7f021a7d6640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020006a460 0x7f020006c920 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.116+0000 7f021a7d6640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02140c4640 0x7f02140c4ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.116+0000 7f021a7d6640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0214158160 0x7f02140ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.116+0000 7f021a7d6640 1 --2- 172.21.15.27:0/2438735046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01fc0047c0 0x7f02140c6180 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.116+0000 7f021a7d6640 1 -- 172.21.15.27:0/2438735046 >> 172.21.15.27:0/2438735046 conn(0x7f0214099ee0 msgr2=0x7f021415a080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:40.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.116+0000 7f021a7d6640 1 -- 172.21.15.27:0/2438735046 shutdown_connections 2023-12-19T18:19:40.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.117+0000 7f021a7d6640 1 -- 172.21.15.27:0/2438735046 wait complete. 2023-12-19T18:19:40.135 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:19:40.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.351+0000 7fd786557640 1 Processor -- start 2023-12-19T18:19:40.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.351+0000 7fd786557640 1 -- start start 2023-12-19T18:19:40.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.353+0000 7fd786557640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7800cf540 0x7fd78015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:40.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.353+0000 7fd786557640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd78015aaf0 0x7fd78006b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:40.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.353+0000 7fd786557640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd78006b4c0 con 0x7fd7800cf160 2023-12-19T18:19:40.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.353+0000 7fd786557640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd780161bd0 con 0x7fd78015aaf0 2023-12-19T18:19:40.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.353+0000 7fd786557640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd780070080 con 0x7fd7800cf540 2023-12-19T18:19:40.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.354+0000 7fd784d54640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7800cf540 0x7fd78015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:40.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.354+0000 7fd785d56640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd78015aaf0 0x7fd78006b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:40.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.354+0000 7fd785555640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fd7800cf160 0x7fd78015e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:44478/0 (socket says 172.21.15.27:44478) 2023-12-19T18:19:40.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.354+0000 7fd785d56640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd78015aaf0 0x7fd78006b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:58716/0 (socket says 172.21.15.27:58716) 2023-12-19T18:19:40.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.354+0000 7fd784d54640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7800cf540 0x7fd78015a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:46892/0 (socket says 172.21.15.27:46892) 2023-12-19T18:19:40.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.354+0000 7fd785555640 1 -- 172.21.15.27:0/878286091 learned_addr learned my addr 172.21.15.27:0/878286091 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:40.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.355+0000 7fd785d56640 1 -- 172.21.15.27:0/878286091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7800cf540 msgr2=0x7fd78015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:40.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.355+0000 7fd785d56640 1 --2- 172.21.15.27:0/878286091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7800cf540 0x7fd78015a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.355+0000 7fd785d56640 1 -- 172.21.15.27:0/878286091 >> v1:172.21.15.192:6789/0 conn(0x7fd7800cf160 legacy=0x7fd78015e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:40.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.355+0000 7fd785d56640 1 -- 172.21.15.27:0/878286091 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78006d140 con 0x7fd78015aaf0 2023-12-19T18:19:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.357+0000 7fd785d56640 1 --2- 172.21.15.27:0/878286091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd78015aaf0 0x7fd78006b930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd7780097a0 tx=0x7fd77802b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b99ffd7686fead1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:40.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.358+0000 7fd777fff640 1 -- 172.21.15.27:0/878286091 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77802d040 con 0x7fd78015aaf0 2023-12-19T18:19:40.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.358+0000 7fd777fff640 1 -- 172.21.15.27:0/878286091 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd778031810 con 0x7fd78015aaf0 2023-12-19T18:19:40.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.358+0000 7fd777fff640 1 -- 172.21.15.27:0/878286091 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd778031ba0 con 0x7fd78015aaf0 2023-12-19T18:19:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.358+0000 7fd786557640 1 -- 172.21.15.27:0/878286091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd78015aaf0 msgr2=0x7fd78006b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.358+0000 7fd786557640 1 --2- 172.21.15.27:0/878286091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd78015aaf0 0x7fd78006b930 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd7780097a0 tx=0x7fd77802b8b0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.359+0000 7fd786557640 1 -- 172.21.15.27:0/878286091 shutdown_connections 2023-12-19T18:19:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.359+0000 7fd786557640 1 --2- 172.21.15.27:0/878286091 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd78015aaf0 0x7fd78006b930 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.359+0000 7fd786557640 1 --2- 172.21.15.27:0/878286091 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7800cf540 0x7fd78015a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.359+0000 7fd786557640 1 -- 172.21.15.27:0/878286091 >> 172.21.15.27:0/878286091 conn(0x7fd78006a060 msgr2=0x7fd78006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.359+0000 7fd786557640 1 -- 172.21.15.27:0/878286091 shutdown_connections 2023-12-19T18:19:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.359+0000 7fd786557640 1 -- 172.21.15.27:0/878286091 wait complete. 2023-12-19T18:19:40.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.360+0000 7fd786557640 1 Processor -- start 2023-12-19T18:19:40.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.361+0000 7fd786557640 1 -- start start 2023-12-19T18:19:40.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.362+0000 7fd786557640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7800cf160 0x7fd7801f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:40.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.362+0000 7fd786557640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7800cf540 0x7fd7800cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:40.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.362+0000 7fd786557640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd78015aaf0 0x7fd78006c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:40.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.362+0000 7fd786557640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7800ccf70 con 0x7fd7800cf160 2023-12-19T18:19:40.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.362+0000 7fd786557640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7800cc360 con 0x7fd7800cf540 2023-12-19T18:19:40.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.362+0000 7fd786557640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7800cc640 con 0x7fd78015aaf0 2023-12-19T18:19:40.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.362+0000 7fd785555640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7800cf160 0x7fd7801f5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:40.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.362+0000 7fd785555640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7800cf160 0x7fd7801f5230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38478/0 (socket says 172.21.15.27:38478) 2023-12-19T18:19:40.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.362+0000 7fd785555640 1 -- 172.21.15.27:0/350039055 learned_addr learned my addr 172.21.15.27:0/350039055 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:40.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.362+0000 7fd785d56640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd78015aaf0 0x7fd78006c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:40.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.362+0000 7fd784d54640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7800cf540 0x7fd7800cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:40.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.363+0000 7fd785555640 1 -- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd78015aaf0 msgr2=0x7fd78006c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:40.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.363+0000 7fd785555640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd78015aaf0 0x7fd78006c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.363+0000 7fd785555640 1 -- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7800cf540 msgr2=0x7fd7800cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:40.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.363+0000 7fd785555640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7800cf540 0x7fd7800cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.363+0000 7fd785555640 1 -- 172.21.15.27:0/350039055 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd778009400 con 0x7fd7800cf160 2023-12-19T18:19:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.363+0000 7fd785555640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7800cf160 0x7fd7801f5230 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00cba0 tx=0x7fd77c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.363+0000 7fd775ffb640 1 -- 172.21.15.27:0/350039055 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c00ad40 con 0x7fd7800cf160 2023-12-19T18:19:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.363+0000 7fd775ffb640 1 -- 172.21.15.27:0/350039055 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77c012400 con 0x7fd7800cf160 2023-12-19T18:19:40.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.363+0000 7fd775ffb640 1 -- 172.21.15.27:0/350039055 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77c00aea0 con 0x7fd7800cf160 2023-12-19T18:19:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.363+0000 7fd786557640 1 -- 172.21.15.27:0/350039055 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7800cc920 con 0x7fd7800cf160 2023-12-19T18:19:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.363+0000 7fd786557640 1 -- 172.21.15.27:0/350039055 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7800cbfc0 con 0x7fd7800cf160 2023-12-19T18:19:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.365+0000 7fd786557640 1 -- 172.21.15.27:0/350039055 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78004e840 con 0x7fd7800cf160 2023-12-19T18:19:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.366+0000 7fd775ffb640 1 -- 172.21.15.27:0/350039055 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd77c007070 con 0x7fd7800cf160 2023-12-19T18:19:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.367+0000 7fd775ffb640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd76406a430 0x7fd76406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.367+0000 7fd775ffb640 1 -- 172.21.15.27:0/350039055 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd77c0887e0 con 0x7fd7800cf160 2023-12-19T18:19:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.367+0000 7fd775ffb640 1 -- 172.21.15.27:0/350039055 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=83}) v3 -- 0x7fd764072020 con 0x7fd7800cf160 2023-12-19T18:19:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.367+0000 7fd784d54640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd76406a430 0x7fd76406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:40.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.368+0000 7fd784d54640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd76406a430 0x7fd76406c8f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd7800706f0 tx=0x7fd770008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:40.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.371+0000 7fd775ffb640 1 -- 172.21.15.27:0/350039055 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd77c017020 con 0x7fd7800cf160 2023-12-19T18:19:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.570+0000 7fd786557640 1 -- 172.21.15.27:0/350039055 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd7801f2380 con 0x7fd7800cf160 2023-12-19T18:19:40.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.570+0000 7fd775ffb640 1 -- 172.21.15.27:0/350039055 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v82) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd77c0578d0 con 0x7fd7800cf160 2023-12-19T18:19:40.570 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:19:40.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.573+0000 7fd786557640 1 -- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd76406a430 msgr2=0x7fd76406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:40.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.573+0000 7fd786557640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd76406a430 0x7fd76406c8f0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fd7800706f0 tx=0x7fd770008040 comp rx=0 tx=0).stop 2023-12-19T18:19:40.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.573+0000 7fd786557640 1 -- 172.21.15.27:0/350039055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7800cf160 msgr2=0x7fd7801f5230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:40.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.573+0000 7fd786557640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7800cf160 0x7fd7801f5230 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fd77c00cba0 tx=0x7fd77c00a430 comp rx=0 tx=0).stop 2023-12-19T18:19:40.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.573+0000 7fd786557640 1 -- 172.21.15.27:0/350039055 shutdown_connections 2023-12-19T18:19:40.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.573+0000 7fd786557640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd76406a430 0x7fd76406c8f0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.573+0000 7fd786557640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd78015aaf0 0x7fd78006c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.573+0000 7fd786557640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7800cf540 0x7fd7800cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.573+0000 7fd786557640 1 --2- 172.21.15.27:0/350039055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7800cf160 0x7fd7801f5230 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.573+0000 7fd786557640 1 -- 172.21.15.27:0/350039055 >> 172.21.15.27:0/350039055 conn(0x7fd78006a060 msgr2=0x7fd7800b2180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:40.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.574+0000 7fd786557640 1 -- 172.21.15.27:0/350039055 shutdown_connections 2023-12-19T18:19:40.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.574+0000 7fd786557640 1 -- 172.21.15.27:0/350039055 wait complete. 2023-12-19T18:19:40.592 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:19:40.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.814+0000 7f8dfca37640 1 Processor -- start 2023-12-19T18:19:40.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.814+0000 7f8dfca37640 1 -- start start 2023-12-19T18:19:40.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.815+0000 7f8dfca37640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8df80b49d0 0x7f8df8160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:40.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.815+0000 7f8dfca37640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df80b5ca0 con 0x7f8df815cf90 2023-12-19T18:19:40.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.815+0000 7f8dfca37640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8df8161800 con 0x7f8df80b89b0 2023-12-19T18:19:40.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.815+0000 7f8dfca37640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8df8161240 con 0x7f8df80b49d0 2023-12-19T18:19:40.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.815+0000 7f8df6d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8df80b49d0 0x7f8df8160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:40.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.815+0000 7f8df759e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8df815cf90 0x7f8df80d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45452/0 (socket says 172.21.15.27:45452) 2023-12-19T18:19:40.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.815+0000 7f8df6d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8df80b49d0 0x7f8df8160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38484/0 (socket says 172.21.15.27:38484) 2023-12-19T18:19:40.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.815+0000 7f8df759e640 1 -- 172.21.15.27:0/2051941543 learned_addr learned my addr 172.21.15.27:0/2051941543 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:40.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.816+0000 7f8df6d9d640 1 -- 172.21.15.27:0/2051941543 >> v1:172.21.15.192:6789/0 conn(0x7f8df80b89b0 legacy=0x7f8df815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:40.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.816+0000 7f8df6d9d640 1 -- 172.21.15.27:0/2051941543 >> v1:172.21.15.27:6789/0 conn(0x7f8df815cf90 legacy=0x7f8df80d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:40.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.816+0000 7f8df6d9d640 1 -- 172.21.15.27:0/2051941543 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df8161520 con 0x7f8df80b49d0 2023-12-19T18:19:40.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.816+0000 7f8df5d9b640 1 -- 172.21.15.27:0/2051941543 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3925248342 0 0) 0x7f8df80b5ca0 con 0x7f8df815cf90 2023-12-19T18:19:40.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.817+0000 7f8df6d9d640 1 --2- 172.21.15.27:0/2051941543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df80b49d0 0x7f8df8160940 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009790 tx=0x7f8de002b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=489bf8484d23ede3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:40.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.817+0000 7f8df5d9b640 1 -- 172.21.15.27:0/2051941543 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0030800 con 0x7f8df80b49d0 2023-12-19T18:19:40.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.818+0000 7f8df5d9b640 1 -- 172.21.15.27:0/2051941543 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de0030960 con 0x7f8df80b49d0 2023-12-19T18:19:40.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.818+0000 7f8df5d9b640 1 -- 172.21.15.27:0/2051941543 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de0030cf0 con 0x7f8df80b49d0 2023-12-19T18:19:40.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.818+0000 7f8dfca37640 1 -- 172.21.15.27:0/2051941543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df80b49d0 msgr2=0x7f8df8160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:40.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.818+0000 7f8dfca37640 1 --2- 172.21.15.27:0/2051941543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df80b49d0 0x7f8df8160940 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f8de0009790 tx=0x7f8de002b940 comp rx=0 tx=0).stop 2023-12-19T18:19:40.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.818+0000 7f8dfca37640 1 -- 172.21.15.27:0/2051941543 shutdown_connections 2023-12-19T18:19:40.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.818+0000 7f8dfca37640 1 --2- 172.21.15.27:0/2051941543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df80b49d0 0x7f8df8160940 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.818+0000 7f8dfca37640 1 -- 172.21.15.27:0/2051941543 >> 172.21.15.27:0/2051941543 conn(0x7f8df80cf760 msgr2=0x7f8df80cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:40.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.818+0000 7f8dfca37640 1 -- 172.21.15.27:0/2051941543 shutdown_connections 2023-12-19T18:19:40.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.818+0000 7f8dfca37640 1 -- 172.21.15.27:0/2051941543 wait complete. 2023-12-19T18:19:40.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.820+0000 7f8dfca37640 1 Processor -- start 2023-12-19T18:19:40.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.820+0000 7f8dfca37640 1 -- start start 2023-12-19T18:19:40.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.821+0000 7f8dfca37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8df80b49d0 0x7f8df80c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:40.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.821+0000 7f8dfca37640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8df80b89b0 0x7f8df815e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:40.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.821+0000 7f8dfca37640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df815cf90 0x7f8df815e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.821+0000 7f8dfca37640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80c9db0 con 0x7f8df815cf90 2023-12-19T18:19:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.821+0000 7f8dfca37640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80ca8f0 con 0x7f8df80b49d0 2023-12-19T18:19:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.821+0000 7f8dfca37640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8df80c9fc0 con 0x7f8df80b89b0 2023-12-19T18:19:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.822+0000 7f8df759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df815cf90 0x7f8df815e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.822+0000 7f8df759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df815cf90 0x7f8df815e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38498/0 (socket says 172.21.15.27:38498) 2023-12-19T18:19:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.822+0000 7f8df759e640 1 -- 172.21.15.27:0/981556074 learned_addr learned my addr 172.21.15.27:0/981556074 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.822+0000 7f8df6d9d640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8df80b49d0 0x7f8df80c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.822+0000 7f8df659c640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8df80b89b0 0x7f8df815e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.822+0000 7f8df759e640 1 -- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8df80b89b0 msgr2=0x7f8df815e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.822+0000 7f8df759e640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8df80b89b0 0x7f8df815e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.822+0000 7f8df759e640 1 -- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8df80b49d0 msgr2=0x7f8df80c8f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.822+0000 7f8df759e640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8df80b49d0 0x7f8df80c8f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.822+0000 7f8df759e640 1 -- 172.21.15.27:0/981556074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de00093a0 con 0x7f8df815cf90 2023-12-19T18:19:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.822+0000 7f8df659c640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8df80b89b0 0x7f8df815e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.822+0000 7f8df6d9d640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8df80b49d0 0x7f8df80c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.822+0000 7f8df759e640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df815cf90 0x7f8df815e990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8df000c410 tx=0x7f8df000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.823+0000 7f8def7fe640 1 -- 172.21.15.27:0/981556074 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df00074e0 con 0x7f8df815cf90 2023-12-19T18:19:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.823+0000 7f8def7fe640 1 -- 172.21.15.27:0/981556074 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df0007640 con 0x7f8df815cf90 2023-12-19T18:19:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.823+0000 7f8def7fe640 1 -- 172.21.15.27:0/981556074 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0004a20 con 0x7f8df815cf90 2023-12-19T18:19:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.823+0000 7f8dfca37640 1 -- 172.21.15.27:0/981556074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8df80ca2a0 con 0x7f8df815cf90 2023-12-19T18:19:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.823+0000 7f8dfca37640 1 -- 172.21.15.27:0/981556074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8df80c9480 con 0x7f8df815cf90 2023-12-19T18:19:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.824+0000 7f8dfca37640 1 -- 172.21.15.27:0/981556074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8df80caa80 con 0x7f8df815cf90 2023-12-19T18:19:40.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.825+0000 7f8def7fe640 1 -- 172.21.15.27:0/981556074 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8df0004b80 con 0x7f8df815cf90 2023-12-19T18:19:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.826+0000 7f8def7fe640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8dd806a430 0x7f8dd806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.826+0000 7f8def7fe640 1 -- 172.21.15.27:0/981556074 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8df0088710 con 0x7f8df815cf90 2023-12-19T18:19:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.827+0000 7f8df6d9d640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8dd806a430 0x7f8dd806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:40.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.828+0000 7f8df6d9d640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8dd806a430 0x7f8dd806c8f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8de0036000 tx=0x7f8de003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:40.830+0000 7f8def7fe640 1 -- 172.21.15.27:0/981556074 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8df0053cc0 con 0x7f8df815cf90 2023-12-19T18:19:41.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.035+0000 7f8dfca37640 1 -- 172.21.15.27:0/981556074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8df804e840 con 0x7f8df815cf90 2023-12-19T18:19:41.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.036+0000 7f8def7fe640 1 -- 172.21.15.27:0/981556074 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v83) v1 ==== 80+0+0 (secure 0 0 0) 0x7f8df0001060 con 0x7f8df815cf90 2023-12-19T18:19:41.036 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:19:41.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.039+0000 7f8dfca37640 1 -- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8dd806a430 msgr2=0x7f8dd806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.039+0000 7f8dfca37640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8dd806a430 0x7f8dd806c8f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f8de0036000 tx=0x7f8de003b040 comp rx=0 tx=0).stop 2023-12-19T18:19:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.039+0000 7f8dfca37640 1 -- 172.21.15.27:0/981556074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df815cf90 msgr2=0x7f8df815e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.039+0000 7f8dfca37640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df815cf90 0x7f8df815e990 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f8df000c410 tx=0x7f8df000c8d0 comp rx=0 tx=0).stop 2023-12-19T18:19:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.039+0000 7f8dfca37640 1 -- 172.21.15.27:0/981556074 shutdown_connections 2023-12-19T18:19:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.039+0000 7f8dfca37640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8dd806a430 0x7f8dd806c8f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.039+0000 7f8dfca37640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8df815cf90 0x7f8df815e990 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.039+0000 7f8dfca37640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8df80b89b0 0x7f8df815e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.039+0000 7f8dfca37640 1 --2- 172.21.15.27:0/981556074 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8df80b49d0 0x7f8df80c8f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.039+0000 7f8dfca37640 1 -- 172.21.15.27:0/981556074 >> 172.21.15.27:0/981556074 conn(0x7f8df80cf760 msgr2=0x7f8df8157150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.039+0000 7f8dfca37640 1 -- 172.21.15.27:0/981556074 shutdown_connections 2023-12-19T18:19:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.039+0000 7f8dfca37640 1 -- 172.21.15.27:0/981556074 wait complete. 2023-12-19T18:19:41.056 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-19T18:19:41.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:19:41.057 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-19T18:19:41.154 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[87832]: fuse finished with error 0 and tester_r 0 2023-12-19T18:19:41.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:19:41.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:19:41.160 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-19T18:19:41.334 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:19:41.334 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:19:41.400 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:19:41.400 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:19:41.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:19:41.401 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-19T18:19:41.459 INFO:teuthology.orchestra.run.smithi027.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-19T18:19:41.460 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:19:41.460 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:19:41.460 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:19:41.520 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:19:41.520 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:19:41.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:19:41.520 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-19T18:19:41.571 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-19T18:19:41.572 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:19:41.572 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:19:41.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.817+0000 7fe671e2d640 1 Processor -- start 2023-12-19T18:19:41.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.817+0000 7fe671e2d640 1 -- start start 2023-12-19T18:19:41.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.818+0000 7fe671e2d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe66c0b4ef0 0x7fe66c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.819+0000 7fe671e2d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe66c0d2350 0x7fe66c0b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.819+0000 7fe671e2d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe66c0b5e90 con 0x7fe66c1604d0 2023-12-19T18:19:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.819+0000 7fe671e2d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66c0d2b30 con 0x7fe66c0b4ef0 2023-12-19T18:19:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.819+0000 7fe671e2d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe66c0d2850 con 0x7fe66c0d2350 2023-12-19T18:19:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.819+0000 7fe66bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe66c0b4ef0 0x7fe66c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.819+0000 7fe670e2b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe66c1604d0 0x7fe66c0b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:45464/0 (socket says 172.21.15.27:45464) 2023-12-19T18:19:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.819+0000 7fe66bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe66c0b4ef0 0x7fe66c0b6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38500/0 (socket says 172.21.15.27:38500) 2023-12-19T18:19:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.819+0000 7fe670e2b640 1 -- 172.21.15.27:0/31168260 learned_addr learned my addr 172.21.15.27:0/31168260 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.819+0000 7fe67162c640 1 --2- 172.21.15.27:0/31168260 >> v2:172.21.15.192:3300/0 conn(0x7fe66c0d2350 0x7fe66c0b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.819+0000 7fe66bfff640 1 -- 172.21.15.27:0/31168260 >> v2:172.21.15.192:3300/0 conn(0x7fe66c0d2350 msgr2=0x7fe66c0b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.820+0000 7fe66bfff640 1 --2- 172.21.15.27:0/31168260 >> v2:172.21.15.192:3300/0 conn(0x7fe66c0d2350 0x7fe66c0b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.820+0000 7fe66bfff640 1 -- 172.21.15.27:0/31168260 >> v1:172.21.15.27:6789/0 conn(0x7fe66c1604d0 legacy=0x7fe66c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.820+0000 7fe66bfff640 1 -- 172.21.15.27:0/31168260 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe66c0cded0 con 0x7fe66c0b4ef0 2023-12-19T18:19:41.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.820+0000 7fe66b7fe640 1 -- 172.21.15.27:0/31168260 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 687956028 0 0) 0x7fe66c0b5e90 con 0x7fe66c1604d0 2023-12-19T18:19:41.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.821+0000 7fe66bfff640 1 --2- 172.21.15.27:0/31168260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe66c0b4ef0 0x7fe66c0b6300 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe66c071e80 tx=0x7fe65402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d1edeeb370058b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:41.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.821+0000 7fe66b7fe640 1 -- 172.21.15.27:0/31168260 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654031830 con 0x7fe66c0b4ef0 2023-12-19T18:19:41.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.821+0000 7fe66b7fe640 1 -- 172.21.15.27:0/31168260 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe654031990 con 0x7fe66c0b4ef0 2023-12-19T18:19:41.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.821+0000 7fe66b7fe640 1 -- 172.21.15.27:0/31168260 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654031d20 con 0x7fe66c0b4ef0 2023-12-19T18:19:41.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.822+0000 7fe671e2d640 1 -- 172.21.15.27:0/31168260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe66c0b4ef0 msgr2=0x7fe66c0b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:41.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.822+0000 7fe671e2d640 1 --2- 172.21.15.27:0/31168260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe66c0b4ef0 0x7fe66c0b6300 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe66c071e80 tx=0x7fe65402b8d0 comp rx=0 tx=0).stop 2023-12-19T18:19:41.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.822+0000 7fe671e2d640 1 -- 172.21.15.27:0/31168260 shutdown_connections 2023-12-19T18:19:41.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.822+0000 7fe671e2d640 1 --2- 172.21.15.27:0/31168260 >> v2:172.21.15.192:3300/0 conn(0x7fe66c0d2350 0x7fe66c0b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:41.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.822+0000 7fe671e2d640 1 --2- 172.21.15.27:0/31168260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe66c0b4ef0 0x7fe66c0b6300 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:41.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.822+0000 7fe671e2d640 1 -- 172.21.15.27:0/31168260 >> 172.21.15.27:0/31168260 conn(0x7fe66c0cf7e0 msgr2=0x7fe66c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:41.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.822+0000 7fe671e2d640 1 -- 172.21.15.27:0/31168260 shutdown_connections 2023-12-19T18:19:41.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.822+0000 7fe671e2d640 1 -- 172.21.15.27:0/31168260 wait complete. 2023-12-19T18:19:41.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.824+0000 7fe671e2d640 1 Processor -- start 2023-12-19T18:19:41.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.824+0000 7fe671e2d640 1 -- start start 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe671e2d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe66c0b4ef0 0x7fe66c0ca380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe671e2d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe66c0d2350 0x7fe66c1e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe671e2d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe66c1604d0 0x7fe66c150140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe671e2d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c150a10 con 0x7fe66c0d2350 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe671e2d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c0ca8c0 con 0x7fe66c0b4ef0 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe671e2d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe66c150c60 con 0x7fe66c1604d0 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe66bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe66c0d2350 0x7fe66c1e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe66bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe66c0d2350 0x7fe66c1e6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38512/0 (socket says 172.21.15.27:38512) 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe66bfff640 1 -- 172.21.15.27:0/3496850546 learned_addr learned my addr 172.21.15.27:0/3496850546 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe670e2b640 1 --2- 172.21.15.27:0/3496850546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe66c0b4ef0 0x7fe66c0ca380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe66bfff640 1 -- 172.21.15.27:0/3496850546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe66c1604d0 msgr2=0x7fe66c150140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe66bfff640 1 --2- 172.21.15.27:0/3496850546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe66c1604d0 0x7fe66c150140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe66bfff640 1 -- 172.21.15.27:0/3496850546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe66c0b4ef0 msgr2=0x7fe66c0ca380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe66bfff640 1 --2- 172.21.15.27:0/3496850546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe66c0b4ef0 0x7fe66c0ca380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:41.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.825+0000 7fe66bfff640 1 -- 172.21.15.27:0/3496850546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe654009400 con 0x7fe66c0d2350 2023-12-19T18:19:41.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.826+0000 7fe66bfff640 1 --2- 172.21.15.27:0/3496850546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe66c0d2350 0x7fe66c1e6fd0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe654000c00 tx=0x7fe6540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:41.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.826+0000 7fe6697fa640 1 -- 172.21.15.27:0/3496850546 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654004230 con 0x7fe66c0d2350 2023-12-19T18:19:41.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.826+0000 7fe6697fa640 1 -- 172.21.15.27:0/3496850546 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6540043b0 con 0x7fe66c0d2350 2023-12-19T18:19:41.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.826+0000 7fe6697fa640 1 -- 172.21.15.27:0/3496850546 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe654004520 con 0x7fe66c0d2350 2023-12-19T18:19:41.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.826+0000 7fe671e2d640 1 -- 172.21.15.27:0/3496850546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe66c150f40 con 0x7fe66c0d2350 2023-12-19T18:19:41.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.826+0000 7fe671e2d640 1 -- 172.21.15.27:0/3496850546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe66c151500 con 0x7fe66c0d2350 2023-12-19T18:19:41.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.827+0000 7fe671e2d640 1 -- 172.21.15.27:0/3496850546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe66c066f80 con 0x7fe66c0d2350 2023-12-19T18:19:41.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.828+0000 7fe6697fa640 1 -- 172.21.15.27:0/3496850546 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe65403a8a0 con 0x7fe66c0d2350 2023-12-19T18:19:41.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.830+0000 7fe6697fa640 1 --2- 172.21.15.27:0/3496850546 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe64c06a430 0x7fe64c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:41.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.830+0000 7fe6697fa640 1 -- 172.21.15.27:0/3496850546 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe6540ab1b0 con 0x7fe66c0d2350 2023-12-19T18:19:41.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.830+0000 7fe670e2b640 1 --2- 172.21.15.27:0/3496850546 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe64c06a430 0x7fe64c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:41.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.832+0000 7fe670e2b640 1 --2- 172.21.15.27:0/3496850546 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe64c06a430 0x7fe64c06c8f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe65c000f60 tx=0x7fe65c007c00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:41.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:41.835+0000 7fe6697fa640 1 -- 172.21.15.27:0/3496850546 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe6540767e0 con 0x7fe66c0d2350 2023-12-19T18:19:42.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.066+0000 7fe671e2d640 1 -- 172.21.15.27:0/3496850546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe66c04e840 con 0x7fe66c0d2350 2023-12-19T18:19:42.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.067+0000 7fe6697fa640 1 -- 172.21.15.27:0/3496850546 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4715 (secure 0 0 0) 0x7fe654079e00 con 0x7fe66c0d2350 2023-12-19T18:19:42.068 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:19:42.068 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1624358030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1624358030},{"type":"v1","addr":"172.21.15.192:6836","nonce":1624358030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5030,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3025339386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3025339386},{"type":"v1","addr":"172.21.15.192:6837","nonce":3025339386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/1685117850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1685117850},{"type":"v1","addr":"172.21.15.27:6833","nonce":1685117850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:18:26.147336+0000","modified":"2023-12-19T18:18:29.663526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2772294131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2772294131},{"type":"v1","addr":"172.21.15.27:6835","nonce":2772294131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-19T18:19:42.068 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2023-12-19T18:19:42.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.070+0000 7fe671e2d640 1 -- 172.21.15.27:0/3496850546 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe64c06a430 msgr2=0x7fe64c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:42.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.070+0000 7fe671e2d640 1 --2- 172.21.15.27:0/3496850546 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe64c06a430 0x7fe64c06c8f0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fe65c000f60 tx=0x7fe65c007c00 comp rx=0 tx=0).stop 2023-12-19T18:19:42.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.070+0000 7fe671e2d640 1 -- 172.21.15.27:0/3496850546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe66c0d2350 msgr2=0x7fe66c1e6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:42.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.070+0000 7fe671e2d640 1 --2- 172.21.15.27:0/3496850546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe66c0d2350 0x7fe66c1e6fd0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe654000c00 tx=0x7fe6540046c0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.070+0000 7fe671e2d640 1 -- 172.21.15.27:0/3496850546 shutdown_connections 2023-12-19T18:19:42.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.070+0000 7fe671e2d640 1 --2- 172.21.15.27:0/3496850546 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe64c06a430 0x7fe64c06c8f0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.070+0000 7fe671e2d640 1 --2- 172.21.15.27:0/3496850546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe66c1604d0 0x7fe66c150140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.070+0000 7fe671e2d640 1 --2- 172.21.15.27:0/3496850546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe66c0d2350 0x7fe66c1e6fd0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.070+0000 7fe671e2d640 1 --2- 172.21.15.27:0/3496850546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe66c0b4ef0 0x7fe66c0ca380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.070+0000 7fe671e2d640 1 -- 172.21.15.27:0/3496850546 >> 172.21.15.27:0/3496850546 conn(0x7fe66c0cf7e0 msgr2=0x7fe66c15ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:42.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.070+0000 7fe671e2d640 1 -- 172.21.15.27:0/3496850546 shutdown_connections 2023-12-19T18:19:42.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.070+0000 7fe671e2d640 1 -- 172.21.15.27:0/3496850546 wait complete. 2023-12-19T18:19:42.092 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:19:42.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.315+0000 7fa0d8f3d640 1 Processor -- start 2023-12-19T18:19:42.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.315+0000 7fa0d8f3d640 1 -- start start 2023-12-19T18:19:42.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.317+0000 7fa0d8f3d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa0d41553a0 0x7fa0d40c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:42.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.317+0000 7fa0d8f3d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0d40afeb0 0x7fa0d415e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:42.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.317+0000 7fa0d8f3d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa0d415ec60 0x7fa0d415b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:42.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.317+0000 7fa0d8f3d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d40cbcf0 con 0x7fa0d41553a0 2023-12-19T18:19:42.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.317+0000 7fa0d8f3d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0d415f0d0 con 0x7fa0d415ec60 2023-12-19T18:19:42.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.317+0000 7fa0d8f3d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40b0330 con 0x7fa0d40afeb0 2023-12-19T18:19:42.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.317+0000 7fa0d37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa0d41553a0 0x7fa0d40c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:42.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.317+0000 7fa0d37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa0d41553a0 0x7fa0d40c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44554/0 (socket says 172.21.15.27:44554) 2023-12-19T18:19:42.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.317+0000 7fa0d37fe640 1 -- 172.21.15.27:0/1981207351 learned_addr learned my addr 172.21.15.27:0/1981207351 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:42.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.318+0000 7fa0d3fff640 1 --2- 172.21.15.27:0/1981207351 >> v2:172.21.15.192:3300/0 conn(0x7fa0d415ec60 0x7fa0d415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:42.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.318+0000 7fa0cbfff640 1 --2- 172.21.15.27:0/1981207351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0d40afeb0 0x7fa0d415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:42.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.318+0000 7fa0d37fe640 1 -- 172.21.15.27:0/1981207351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0d40afeb0 msgr2=0x7fa0d415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:42.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.318+0000 7fa0d37fe640 1 --2- 172.21.15.27:0/1981207351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0d40afeb0 0x7fa0d415e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.318+0000 7fa0d37fe640 1 -- 172.21.15.27:0/1981207351 >> v2:172.21.15.192:3300/0 conn(0x7fa0d415ec60 msgr2=0x7fa0d415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:42.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.318+0000 7fa0d37fe640 1 --2- 172.21.15.27:0/1981207351 >> v2:172.21.15.192:3300/0 conn(0x7fa0d415ec60 0x7fa0d415b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.318+0000 7fa0d37fe640 1 -- 172.21.15.27:0/1981207351 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d40cb350 con 0x7fa0d41553a0 2023-12-19T18:19:42.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.319+0000 7fa0d37fe640 1 --2- 172.21.15.27:0/1981207351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0d41553a0 0x7fa0d40c50a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa0d406f2d0 tx=0x7fa0c402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4634cf2ad147f065 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:42.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.319+0000 7fa0d2ffd640 1 -- 172.21.15.27:0/1981207351 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c40317c0 con 0x7fa0d41553a0 2023-12-19T18:19:42.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.319+0000 7fa0d2ffd640 1 -- 172.21.15.27:0/1981207351 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c4031920 con 0x7fa0d41553a0 2023-12-19T18:19:42.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.319+0000 7fa0d2ffd640 1 -- 172.21.15.27:0/1981207351 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4031cb0 con 0x7fa0d41553a0 2023-12-19T18:19:42.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.320+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/1981207351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0d41553a0 msgr2=0x7fa0d40c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:42.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.320+0000 7fa0d8f3d640 1 --2- 172.21.15.27:0/1981207351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0d41553a0 0x7fa0d40c50a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fa0d406f2d0 tx=0x7fa0c402b860 comp rx=0 tx=0).stop 2023-12-19T18:19:42.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.320+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/1981207351 shutdown_connections 2023-12-19T18:19:42.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.320+0000 7fa0d8f3d640 1 --2- 172.21.15.27:0/1981207351 >> v2:172.21.15.192:3300/0 conn(0x7fa0d415ec60 0x7fa0d415b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.320+0000 7fa0d8f3d640 1 --2- 172.21.15.27:0/1981207351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0d40afeb0 0x7fa0d415e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.320+0000 7fa0d8f3d640 1 --2- 172.21.15.27:0/1981207351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0d41553a0 0x7fa0d40c50a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.320+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/1981207351 >> 172.21.15.27:0/1981207351 conn(0x7fa0d4069c70 msgr2=0x7fa0d406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:42.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.320+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/1981207351 shutdown_connections 2023-12-19T18:19:42.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.320+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/1981207351 wait complete. 2023-12-19T18:19:42.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.322+0000 7fa0d8f3d640 1 Processor -- start 2023-12-19T18:19:42.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.322+0000 7fa0d8f3d640 1 -- start start 2023-12-19T18:19:42.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.323+0000 7fa0d8f3d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0d40afeb0 0x7fa0d40c84c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:42.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.323+0000 7fa0d8f3d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0d41553a0 0x7fa0d41545a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:42.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.323+0000 7fa0d8f3d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0d415ec60 0x7fa0d40c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:42.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.323+0000 7fa0d8f3d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d4154ae0 con 0x7fa0d40afeb0 2023-12-19T18:19:42.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.323+0000 7fa0d8f3d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40c68d0 con 0x7fa0d41553a0 2023-12-19T18:19:42.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.323+0000 7fa0d8f3d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d40ade00 con 0x7fa0d415ec60 2023-12-19T18:19:42.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.323+0000 7fa0d37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0d40afeb0 0x7fa0d40c84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:42.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.323+0000 7fa0d37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0d40afeb0 0x7fa0d40c84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44562/0 (socket says 172.21.15.27:44562) 2023-12-19T18:19:42.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.324+0000 7fa0d37fe640 1 -- 172.21.15.27:0/973762404 learned_addr learned my addr 172.21.15.27:0/973762404 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:42.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.324+0000 7fa0d3fff640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0d415ec60 0x7fa0d40c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:42.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.324+0000 7fa0cbfff640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0d41553a0 0x7fa0d41545a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:42.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.324+0000 7fa0d37fe640 1 -- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0d415ec60 msgr2=0x7fa0d40c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:42.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.324+0000 7fa0d37fe640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0d415ec60 0x7fa0d40c6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.324+0000 7fa0d37fe640 1 -- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0d41553a0 msgr2=0x7fa0d41545a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:42.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.324+0000 7fa0d37fe640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0d41553a0 0x7fa0d41545a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.324+0000 7fa0d37fe640 1 -- 172.21.15.27:0/973762404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c40094d0 con 0x7fa0d40afeb0 2023-12-19T18:19:42.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.324+0000 7fa0d3fff640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0d415ec60 0x7fa0d40c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:42.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.324+0000 7fa0cbfff640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0d41553a0 0x7fa0d41545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:42.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.324+0000 7fa0d37fe640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0d40afeb0 0x7fa0d40c84c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa0c402b8a0 tx=0x7fa0c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:42.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.325+0000 7fa0d0ff9640 1 -- 172.21.15.27:0/973762404 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4032860 con 0x7fa0d40afeb0 2023-12-19T18:19:42.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.325+0000 7fa0d0ff9640 1 -- 172.21.15.27:0/973762404 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0c40329e0 con 0x7fa0d40afeb0 2023-12-19T18:19:42.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.325+0000 7fa0d0ff9640 1 -- 172.21.15.27:0/973762404 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0c4032d90 con 0x7fa0d40afeb0 2023-12-19T18:19:42.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.325+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/973762404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d40ae0e0 con 0x7fa0d40afeb0 2023-12-19T18:19:42.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.325+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/973762404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d40ae6a0 con 0x7fa0d40afeb0 2023-12-19T18:19:42.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.326+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/973762404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d404e840 con 0x7fa0d40afeb0 2023-12-19T18:19:42.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.327+0000 7fa0d0ff9640 1 -- 172.21.15.27:0/973762404 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa0c404e450 con 0x7fa0d40afeb0 2023-12-19T18:19:42.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.328+0000 7fa0d0ff9640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0b406a430 0x7fa0b406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:42.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.328+0000 7fa0d0ff9640 1 -- 172.21.15.27:0/973762404 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa0c40aab10 con 0x7fa0d40afeb0 2023-12-19T18:19:42.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.328+0000 7fa0cbfff640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0b406a430 0x7fa0b406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:42.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.330+0000 7fa0cbfff640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0b406a430 0x7fa0b406c8f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc006fd0 tx=0x7fa0bc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:42.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.332+0000 7fa0d0ff9640 1 -- 172.21.15.27:0/973762404 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa0c40760c0 con 0x7fa0d40afeb0 2023-12-19T18:19:42.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.529+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/973762404 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0d404e0d0 con 0x7fa0d40afeb0 2023-12-19T18:19:42.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.532+0000 7fa0d0ff9640 1 -- 172.21.15.27:0/973762404 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16193 (secure 0 0 0) 0x7fa0c40796e0 con 0x7fa0d40afeb0 2023-12-19T18:19:42.532 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:19:42.532 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":84,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:19:41.696772+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:24.350728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:25.147723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.14","mappings":[{"from":6,"to":4}]},{"pgid":"9.2c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:19:42.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.534+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0b406a430 msgr2=0x7fa0b406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:42.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.534+0000 7fa0d8f3d640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0b406a430 0x7fa0b406c8f0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa0bc006fd0 tx=0x7fa0bc008040 comp rx=0 tx=0).stop 2023-12-19T18:19:42.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.535+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/973762404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0d40afeb0 msgr2=0x7fa0d40c84c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:42.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.535+0000 7fa0d8f3d640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0d40afeb0 0x7fa0d40c84c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa0c402b8a0 tx=0x7fa0c40046c0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.535+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/973762404 shutdown_connections 2023-12-19T18:19:42.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.535+0000 7fa0d8f3d640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0b406a430 0x7fa0b406c8f0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.535+0000 7fa0d8f3d640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0d415ec60 0x7fa0d40c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.535+0000 7fa0d8f3d640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0d41553a0 0x7fa0d41545a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.535+0000 7fa0d8f3d640 1 --2- 172.21.15.27:0/973762404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0d40afeb0 0x7fa0d40c84c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.535+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/973762404 >> 172.21.15.27:0/973762404 conn(0x7fa0d4069c70 msgr2=0x7fa0d4159770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:42.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.535+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/973762404 shutdown_connections 2023-12-19T18:19:42.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.535+0000 7fa0d8f3d640 1 -- 172.21.15.27:0/973762404 wait complete. 2023-12-19T18:19:42.555 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:18:24.350728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:18:25.147723+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:19:42.556 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:19:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.794+0000 7ffa4ec9e640 1 Processor -- start 2023-12-19T18:19:42.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.794+0000 7ffa4ec9e640 1 -- start start 2023-12-19T18:19:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.795+0000 7ffa4ec9e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa480d2100 0x7ffa480cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.796+0000 7ffa4ec9e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffa480cdf80 0x7ffa480b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.796+0000 7ffa4ec9e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa480d0e10 con 0x7ffa4815cf80 2023-12-19T18:19:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.796+0000 7ffa4ec9e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa480b8cf0 con 0x7ffa480cdf80 2023-12-19T18:19:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.796+0000 7ffa4ec9e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480ce410 con 0x7ffa480d2100 2023-12-19T18:19:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.796+0000 7ffa4e49d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffa480cdf80 0x7ffa480b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.796+0000 7ffa4e49d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffa480cdf80 0x7ffa480b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44574/0 (socket says 172.21.15.27:44574) 2023-12-19T18:19:42.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.796+0000 7ffa4e49d640 1 -- 172.21.15.27:0/2049921435 learned_addr learned my addr 172.21.15.27:0/2049921435 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.796+0000 7ffa4d49b640 1 --2- 172.21.15.27:0/2049921435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa480d2100 0x7ffa480cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.796+0000 7ffa4e49d640 1 -- 172.21.15.27:0/2049921435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa480d2100 msgr2=0x7ffa480cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.796+0000 7ffa4e49d640 1 --2- 172.21.15.27:0/2049921435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa480d2100 0x7ffa480cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.796+0000 7ffa4e49d640 1 -- 172.21.15.27:0/2049921435 >> v1:172.21.15.192:6789/0 conn(0x7ffa4815cf80 legacy=0x7ffa480b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.796+0000 7ffa4e49d640 1 -- 172.21.15.27:0/2049921435 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa480d1c70 con 0x7ffa480cdf80 2023-12-19T18:19:42.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.797+0000 7ffa4d49b640 1 --2- 172.21.15.27:0/2049921435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa480d2100 0x7ffa480cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:42.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.798+0000 7ffa4e49d640 1 --2- 172.21.15.27:0/2049921435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa480cdf80 0x7ffa480b5d60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009870 tx=0x7ffa4002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fe5f5ec655e925 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:42.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.798+0000 7ffa4cc9a640 1 -- 172.21.15.27:0/2049921435 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa40030840 con 0x7ffa480cdf80 2023-12-19T18:19:42.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.798+0000 7ffa4cc9a640 1 -- 172.21.15.27:0/2049921435 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa400309a0 con 0x7ffa480cdf80 2023-12-19T18:19:42.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.798+0000 7ffa4cc9a640 1 -- 172.21.15.27:0/2049921435 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa40030d30 con 0x7ffa480cdf80 2023-12-19T18:19:42.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.799+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/2049921435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa480cdf80 msgr2=0x7ffa480b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:42.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.799+0000 7ffa4ec9e640 1 --2- 172.21.15.27:0/2049921435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa480cdf80 0x7ffa480b5d60 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009870 tx=0x7ffa4002b980 comp rx=0 tx=0).stop 2023-12-19T18:19:42.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.799+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/2049921435 shutdown_connections 2023-12-19T18:19:42.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.799+0000 7ffa4ec9e640 1 --2- 172.21.15.27:0/2049921435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa480cdf80 0x7ffa480b5d60 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.799+0000 7ffa4ec9e640 1 --2- 172.21.15.27:0/2049921435 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa480d2100 0x7ffa480cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.799+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/2049921435 >> 172.21.15.27:0/2049921435 conn(0x7ffa48072e20 msgr2=0x7ffa48073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:42.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.799+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/2049921435 shutdown_connections 2023-12-19T18:19:42.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.799+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/2049921435 wait complete. 2023-12-19T18:19:42.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.801+0000 7ffa4ec9e640 1 Processor -- start 2023-12-19T18:19:42.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.801+0000 7ffa4ec9e640 1 -- start start 2023-12-19T18:19:42.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.802+0000 7ffa4ec9e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa480cdf80 0x7ffa4815f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:42.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.802+0000 7ffa4ec9e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa480d2100 0x7ffa481f6720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:42.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.802+0000 7ffa4ec9e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa4815cf80 0x7ffa4815d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:42.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.802+0000 7ffa4ec9e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa481f6c60 con 0x7ffa480cdf80 2023-12-19T18:19:42.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.802+0000 7ffa4ec9e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa480c71a0 con 0x7ffa4815cf80 2023-12-19T18:19:42.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.802+0000 7ffa4ec9e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa481607d0 con 0x7ffa480d2100 2023-12-19T18:19:42.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.802+0000 7ffa4dc9c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa480cdf80 0x7ffa4815f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:42.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.802+0000 7ffa4dc9c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa480cdf80 0x7ffa4815f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44590/0 (socket says 172.21.15.27:44590) 2023-12-19T18:19:42.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.802+0000 7ffa4dc9c640 1 -- 172.21.15.27:0/3986737299 learned_addr learned my addr 172.21.15.27:0/3986737299 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:42.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa4d49b640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa480d2100 0x7ffa481f6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa4dc9c640 1 -- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa480d2100 msgr2=0x7ffa481f6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa4dc9c640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa480d2100 0x7ffa481f6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa4dc9c640 1 -- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa4815cf80 msgr2=0x7ffa4815d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa4e49d640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa4815cf80 0x7ffa4815d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa4dc9c640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa4815cf80 0x7ffa4815d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa4dc9c640 1 -- 172.21.15.27:0/3986737299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400094d0 con 0x7ffa480cdf80 2023-12-19T18:19:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa4d49b640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa480d2100 0x7ffa481f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa4e49d640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa4815cf80 0x7ffa4815d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa4dc9c640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa480cdf80 0x7ffa4815f640 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800ba50 tx=0x7ffa3800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa467fc640 1 -- 172.21.15.27:0/3986737299 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa3800cbe0 con 0x7ffa480cdf80 2023-12-19T18:19:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa467fc640 1 -- 172.21.15.27:0/3986737299 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa3800cd80 con 0x7ffa480cdf80 2023-12-19T18:19:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa467fc640 1 -- 172.21.15.27:0/3986737299 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa38014680 con 0x7ffa480cdf80 2023-12-19T18:19:42.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.803+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/3986737299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4815fc50 con 0x7ffa480cdf80 2023-12-19T18:19:42.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.804+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/3986737299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa48160210 con 0x7ffa480cdf80 2023-12-19T18:19:42.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.805+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/3986737299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa480cd0c0 con 0x7ffa480cdf80 2023-12-19T18:19:42.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.806+0000 7ffa467fc640 1 -- 172.21.15.27:0/3986737299 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ffa380147e0 con 0x7ffa480cdf80 2023-12-19T18:19:42.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.807+0000 7ffa467fc640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffa3406a430 0x7ffa3406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:42.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.807+0000 7ffa467fc640 1 -- 172.21.15.27:0/3986737299 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ffa38088800 con 0x7ffa480cdf80 2023-12-19T18:19:42.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.807+0000 7ffa4d49b640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffa3406a430 0x7ffa3406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:42.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.809+0000 7ffa4d49b640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffa3406a430 0x7ffa3406c8f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c006fd0 tx=0x7ffa3c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:42.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:42.811+0000 7ffa467fc640 1 -- 172.21.15.27:0/3986737299 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ffa38053e50 con 0x7ffa480cdf80 2023-12-19T18:19:43.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.002+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/3986737299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffa4804e840 con 0x7ffa480cdf80 2023-12-19T18:19:43.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.003+0000 7ffa467fc640 1 -- 172.21.15.27:0/3986737299 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffa380190a0 con 0x7ffa480cdf80 2023-12-19T18:19:43.003 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:19:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.005+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffa3406a430 msgr2=0x7ffa3406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.005+0000 7ffa4ec9e640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffa3406a430 0x7ffa3406c8f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ffa3c006fd0 tx=0x7ffa3c008040 comp rx=0 tx=0).stop 2023-12-19T18:19:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.005+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/3986737299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa480cdf80 msgr2=0x7ffa4815f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.005+0000 7ffa4ec9e640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa480cdf80 0x7ffa4815f640 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800ba50 tx=0x7ffa3800bf10 comp rx=0 tx=0).stop 2023-12-19T18:19:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.005+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/3986737299 shutdown_connections 2023-12-19T18:19:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.005+0000 7ffa4ec9e640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffa3406a430 0x7ffa3406c8f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.005+0000 7ffa4ec9e640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa4815cf80 0x7ffa4815d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.005+0000 7ffa4ec9e640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa480d2100 0x7ffa481f6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.005+0000 7ffa4ec9e640 1 --2- 172.21.15.27:0/3986737299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa480cdf80 0x7ffa4815f640 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.005+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/3986737299 >> 172.21.15.27:0/3986737299 conn(0x7ffa48072e20 msgr2=0x7ffa480d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:43.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.006+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/3986737299 shutdown_connections 2023-12-19T18:19:43.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.006+0000 7ffa4ec9e640 1 -- 172.21.15.27:0/3986737299 wait complete. 2023-12-19T18:19:43.027 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:19:43.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.266+0000 7fc0cce7d640 1 Processor -- start 2023-12-19T18:19:43.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.266+0000 7fc0cce7d640 1 -- start start 2023-12-19T18:19:43.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.268+0000 7fc0cce7d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0c80c2720 0x7fc0c806fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:43.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.268+0000 7fc0cce7d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c806bc80 con 0x7fc0c8071420 2023-12-19T18:19:43.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.268+0000 7fc0cce7d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0c806d690 con 0x7fc0c815b1a0 2023-12-19T18:19:43.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.268+0000 7fc0cce7d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c806d0a0 con 0x7fc0c80c2720 2023-12-19T18:19:43.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.268+0000 7fc0c6d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc0c8071420 0x7fc0c815ed80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49190/0 (socket says 172.21.15.27:49190) 2023-12-19T18:19:43.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.268+0000 7fc0c6d9d640 1 -- 172.21.15.27:0/1106605371 learned_addr learned my addr 172.21.15.27:0/1106605371 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:43.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.268+0000 7fc0bffff640 1 --2- 172.21.15.27:0/1106605371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0c80c2720 0x7fc0c806fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:43.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.269+0000 7fc0c5d9b640 1 -- 172.21.15.27:0/1106605371 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3592325694 0 0) 0x7fc0c0001490 con 0x7fc0c8071420 2023-12-19T18:19:43.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.269+0000 7fc0c5d9b640 1 -- 172.21.15.27:0/1106605371 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0b4003520 con 0x7fc0c8071420 2023-12-19T18:19:43.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.269+0000 7fc0c5d9b640 1 -- 172.21.15.27:0/1106605371 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1077227812 0 0) 0x7fc0b4003520 con 0x7fc0c8071420 2023-12-19T18:19:43.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.269+0000 7fc0bffff640 1 -- 172.21.15.27:0/1106605371 >> v1:172.21.15.192:6789/0 conn(0x7fc0c815b1a0 legacy=0x7fc0c806f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.269+0000 7fc0bffff640 1 -- 172.21.15.27:0/1106605371 >> v1:172.21.15.27:6789/0 conn(0x7fc0c8071420 legacy=0x7fc0c815ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.269+0000 7fc0bffff640 1 -- 172.21.15.27:0/1106605371 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0c806b6c0 con 0x7fc0c80c2720 2023-12-19T18:19:43.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.271+0000 7fc0bffff640 1 --2- 172.21.15.27:0/1106605371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0c80c2720 0x7fc0c806fc30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00097d0 tx=0x7fc0b002b770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=765e8baa98ce6072 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:43.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.271+0000 7fc0c5d9b640 1 -- 172.21.15.27:0/1106605371 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0001b80 con 0x7fc0c80c2720 2023-12-19T18:19:43.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.271+0000 7fc0c5d9b640 1 -- 172.21.15.27:0/1106605371 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b002e050 con 0x7fc0c80c2720 2023-12-19T18:19:43.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.271+0000 7fc0c5d9b640 1 -- 172.21.15.27:0/1106605371 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b0030710 con 0x7fc0c80c2720 2023-12-19T18:19:43.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.272+0000 7fc0cce7d640 1 -- 172.21.15.27:0/1106605371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0c80c2720 msgr2=0x7fc0c806fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.272+0000 7fc0cce7d640 1 --2- 172.21.15.27:0/1106605371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0c80c2720 0x7fc0c806fc30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fc0b00097d0 tx=0x7fc0b002b770 comp rx=0 tx=0).stop 2023-12-19T18:19:43.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.272+0000 7fc0cce7d640 1 -- 172.21.15.27:0/1106605371 shutdown_connections 2023-12-19T18:19:43.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.272+0000 7fc0cce7d640 1 --2- 172.21.15.27:0/1106605371 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0c80c2720 0x7fc0c806fc30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.272+0000 7fc0cce7d640 1 -- 172.21.15.27:0/1106605371 >> 172.21.15.27:0/1106605371 conn(0x7fc0c80cebd0 msgr2=0x7fc0c80cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:43.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.272+0000 7fc0cce7d640 1 -- 172.21.15.27:0/1106605371 shutdown_connections 2023-12-19T18:19:43.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.272+0000 7fc0cce7d640 1 -- 172.21.15.27:0/1106605371 wait complete. 2023-12-19T18:19:43.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.274+0000 7fc0cce7d640 1 Processor -- start 2023-12-19T18:19:43.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.274+0000 7fc0cce7d640 1 -- start start 2023-12-19T18:19:43.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.275+0000 7fc0cce7d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0c8071420 0x7fc0c814ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:43.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.275+0000 7fc0cce7d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0c80c2720 0x7fc0c80ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:43.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.275+0000 7fc0cce7d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815b1a0 0x7fc0c80ad750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:43.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.275+0000 7fc0cce7d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c80ab7e0 con 0x7fc0c815b1a0 2023-12-19T18:19:43.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.275+0000 7fc0cce7d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c80abac0 con 0x7fc0c8071420 2023-12-19T18:19:43.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.275+0000 7fc0cce7d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c80abda0 con 0x7fc0c80c2720 2023-12-19T18:19:43.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.275+0000 7fc0c6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815b1a0 0x7fc0c80ad750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:43.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.275+0000 7fc0c6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815b1a0 0x7fc0c80ad750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44600/0 (socket says 172.21.15.27:44600) 2023-12-19T18:19:43.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.275+0000 7fc0c6d9d640 1 -- 172.21.15.27:0/2928260058 learned_addr learned my addr 172.21.15.27:0/2928260058 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:43.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.276+0000 7fc0bffff640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0c80c2720 0x7fc0c80ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:43.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.276+0000 7fc0c659c640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0c8071420 0x7fc0c814ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:43.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.276+0000 7fc0c6d9d640 1 -- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0c80c2720 msgr2=0x7fc0c80ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.276+0000 7fc0c6d9d640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0c80c2720 0x7fc0c80ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.276+0000 7fc0c6d9d640 1 -- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0c8071420 msgr2=0x7fc0c814ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.276+0000 7fc0c6d9d640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0c8071420 0x7fc0c814ae70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.276+0000 7fc0c6d9d640 1 -- 172.21.15.27:0/2928260058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b0009400 con 0x7fc0c815b1a0 2023-12-19T18:19:43.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.276+0000 7fc0c6d9d640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815b1a0 0x7fc0c80ad750 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0001f70 tx=0x7fc0c000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:43.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.276+0000 7fc0beffd640 1 -- 172.21.15.27:0/2928260058 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0012400 con 0x7fc0c815b1a0 2023-12-19T18:19:43.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.276+0000 7fc0beffd640 1 -- 172.21.15.27:0/2928260058 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c0012580 con 0x7fc0c815b1a0 2023-12-19T18:19:43.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.277+0000 7fc0beffd640 1 -- 172.21.15.27:0/2928260058 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c0012930 con 0x7fc0c815b1a0 2023-12-19T18:19:43.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.277+0000 7fc0cce7d640 1 -- 172.21.15.27:0/2928260058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c80ac080 con 0x7fc0c815b1a0 2023-12-19T18:19:43.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.277+0000 7fc0cce7d640 1 -- 172.21.15.27:0/2928260058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c814b3b0 con 0x7fc0c815b1a0 2023-12-19T18:19:43.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.278+0000 7fc0cce7d640 1 -- 172.21.15.27:0/2928260058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c80ac5a0 con 0x7fc0c815b1a0 2023-12-19T18:19:43.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.279+0000 7fc0beffd640 1 -- 172.21.15.27:0/2928260058 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc0c0012a90 con 0x7fc0c815b1a0 2023-12-19T18:19:43.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.280+0000 7fc0beffd640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0a806a430 0x7fc0a806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:43.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.280+0000 7fc0beffd640 1 -- 172.21.15.27:0/2928260058 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc0c0088fe0 con 0x7fc0c815b1a0 2023-12-19T18:19:43.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.281+0000 7fc0c659c640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0a806a430 0x7fc0a806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:43.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.282+0000 7fc0c659c640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0a806a430 0x7fc0a806c8f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8000f60 tx=0x7fc0b8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:43.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.284+0000 7fc0beffd640 1 -- 172.21.15.27:0/2928260058 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc0c80ac5a0 con 0x7fc0c815b1a0 2023-12-19T18:19:43.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.479+0000 7fc0cce7d640 1 -- 172.21.15.27:0/2928260058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc0c806bc80 con 0x7fc0c815b1a0 2023-12-19T18:19:43.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.479+0000 7fc0beffd640 1 -- 172.21.15.27:0/2928260058 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc0c806bc80 con 0x7fc0c815b1a0 2023-12-19T18:19:43.480 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:19:43.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.482+0000 7fc0cce7d640 1 -- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0a806a430 msgr2=0x7fc0a806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.482+0000 7fc0cce7d640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0a806a430 0x7fc0a806c8f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8000f60 tx=0x7fc0b8009040 comp rx=0 tx=0).stop 2023-12-19T18:19:43.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.482+0000 7fc0cce7d640 1 -- 172.21.15.27:0/2928260058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815b1a0 msgr2=0x7fc0c80ad750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.482+0000 7fc0cce7d640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815b1a0 0x7fc0c80ad750 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc0c0001f70 tx=0x7fc0c000a430 comp rx=0 tx=0).stop 2023-12-19T18:19:43.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.482+0000 7fc0cce7d640 1 -- 172.21.15.27:0/2928260058 shutdown_connections 2023-12-19T18:19:43.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.482+0000 7fc0cce7d640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0a806a430 0x7fc0a806c8f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.482+0000 7fc0cce7d640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c815b1a0 0x7fc0c80ad750 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.482+0000 7fc0cce7d640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0c80c2720 0x7fc0c80ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.482+0000 7fc0cce7d640 1 --2- 172.21.15.27:0/2928260058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0c8071420 0x7fc0c814ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.482+0000 7fc0cce7d640 1 -- 172.21.15.27:0/2928260058 >> 172.21.15.27:0/2928260058 conn(0x7fc0c80cebd0 msgr2=0x7fc0c80c3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:43.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.482+0000 7fc0cce7d640 1 -- 172.21.15.27:0/2928260058 shutdown_connections 2023-12-19T18:19:43.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.482+0000 7fc0cce7d640 1 -- 172.21.15.27:0/2928260058 wait complete. 2023-12-19T18:19:43.504 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:19:43.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.744+0000 7f9ff0718640 1 Processor -- start 2023-12-19T18:19:43.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.745+0000 7f9ff0718640 1 -- start start 2023-12-19T18:19:43.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.745+0000 7f9ff0718640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80ad9e0 0x7f9fe81683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:43.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.745+0000 7f9ff0718640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe80ce0b0 con 0x7f9fe80c9750 2023-12-19T18:19:43.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.746+0000 7f9ff0718640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fe80ce920 con 0x7f9fe80ccfa0 2023-12-19T18:19:43.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.746+0000 7f9ff0718640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe80cd4a0 con 0x7f9fe80ad9e0 2023-12-19T18:19:43.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.746+0000 7f9fee4b4640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9fe80c9750 0x7f9fe815e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49192/0 (socket says 172.21.15.27:49192) 2023-12-19T18:19:43.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.746+0000 7f9fee4b4640 1 -- 172.21.15.27:0/1727281733 learned_addr learned my addr 172.21.15.27:0/1727281733 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:43.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.746+0000 7f9feecb5640 1 --2- 172.21.15.27:0/1727281733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80ad9e0 0x7f9fe81683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:43.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.746+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3402867341 0 0) 0x7f9fe80ce0b0 con 0x7f9fe80c9750 2023-12-19T18:19:43.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fd4003520 con 0x7f9fe80c9750 2023-12-19T18:19:43.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 335100216 0 0) 0x7f9fd4003520 con 0x7f9fe80c9750 2023-12-19T18:19:43.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80ad9e0 msgr2=0x7f9fe81683a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:43.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 --2- 172.21.15.27:0/1727281733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80ad9e0 0x7f9fe81683a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 >> v1:172.21.15.192:6789/0 conn(0x7f9fe80ccfa0 legacy=0x7f9fe80c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe80cd780 con 0x7f9fe80c9750 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f9fe4001670 con 0x7f9fe80c9750 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 >> v1:172.21.15.27:6789/0 conn(0x7f9fe80c9750 legacy=0x7f9fe815e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 --2- 172.21.15.27:0/1727281733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fd40039a0 0x7f9fd4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 --2- 172.21.15.27:0/1727281733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fd40047c0 0x7f9fd4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 --2- 172.21.15.27:0/1727281733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80ad9e0 0x7f9fd4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe80cd4a0 con 0x7f9fd40047c0 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd400ba90 con 0x7f9fd40039a0 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd400bc50 con 0x7f9fe80ad9e0 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.747+0000 7f9fedcb3640 1 --2- 172.21.15.27:0/1727281733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fd40047c0 0x7f9fd4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.748+0000 7f9fee4b4640 1 --2- 172.21.15.27:0/1727281733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fd40039a0 0x7f9fd4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.748+0000 7f9feecb5640 1 --2- 172.21.15.27:0/1727281733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80ad9e0 0x7f9fd4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.748+0000 7f9fedcb3640 1 -- 172.21.15.27:0/1727281733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80ad9e0 msgr2=0x7f9fd4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:43.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.748+0000 7f9fedcb3640 1 --2- 172.21.15.27:0/1727281733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe80ad9e0 0x7f9fd4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.748+0000 7f9fedcb3640 1 -- 172.21.15.27:0/1727281733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fd40039a0 msgr2=0x7f9fd4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:43.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.748+0000 7f9fedcb3640 1 --2- 172.21.15.27:0/1727281733 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fd40039a0 0x7f9fd4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.748+0000 7f9fedcb3640 1 -- 172.21.15.27:0/1727281733 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd4003520 con 0x7f9fd40047c0 2023-12-19T18:19:43.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.749+0000 7f9fedcb3640 1 --2- 172.21.15.27:0/1727281733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fd40047c0 0x7f9fd4004c40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80cd3d0 tx=0x7f9fd40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e60a97a06aaab3ff server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:43.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.749+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd80052d0 con 0x7f9fd40047c0 2023-12-19T18:19:43.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.749+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd802e050 con 0x7f9fd40047c0 2023-12-19T18:19:43.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.749+0000 7f9fed4b2640 1 -- 172.21.15.27:0/1727281733 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd80035f0 con 0x7f9fd40047c0 2023-12-19T18:19:43.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.750+0000 7f9ff0718640 1 -- 172.21.15.27:0/1727281733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fd40047c0 msgr2=0x7f9fd4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.750+0000 7f9ff0718640 1 --2- 172.21.15.27:0/1727281733 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fd40047c0 0x7f9fd4004c40 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80cd3d0 tx=0x7f9fd40057f0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.750+0000 7f9feecb5640 1 -- 172.21.15.27:0/1727281733 reap_dead start 2023-12-19T18:19:43.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.750+0000 7f9ff0718640 1 -- 172.21.15.27:0/1727281733 shutdown_connections 2023-12-19T18:19:43.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.750+0000 7f9ff0718640 1 -- 172.21.15.27:0/1727281733 >> 172.21.15.27:0/1727281733 conn(0x7f9fe80cebd0 msgr2=0x7f9fe80cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:43.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.750+0000 7f9ff0718640 1 -- 172.21.15.27:0/1727281733 shutdown_connections 2023-12-19T18:19:43.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.750+0000 7f9ff0718640 1 -- 172.21.15.27:0/1727281733 wait complete. 2023-12-19T18:19:43.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.752+0000 7f9ff0718640 1 Processor -- start 2023-12-19T18:19:43.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.752+0000 7f9ff0718640 1 -- start start 2023-12-19T18:19:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9ff0718640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fd40047c0 0x7f9fe81f6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9ff0718640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe8155e20 0x7f9fe815c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9ff0718640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe806c490 0x7f9fe806c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9ff0718640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe81f7200 con 0x7f9fe806c490 2023-12-19T18:19:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9ff0718640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe806ce50 con 0x7f9fd40047c0 2023-12-19T18:19:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9ff0718640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fe815c630 con 0x7f9fe8155e20 2023-12-19T18:19:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9feecb5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe806c490 0x7f9fe806c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9feecb5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe806c490 0x7f9fe806c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44626/0 (socket says 172.21.15.27:44626) 2023-12-19T18:19:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9feecb5640 1 -- 172.21.15.27:0/3927762923 learned_addr learned my addr 172.21.15.27:0/3927762923 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9fee4b4640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fd40047c0 0x7f9fe81f6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9fedcb3640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe8155e20 0x7f9fe815c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:43.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9feecb5640 1 -- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe8155e20 msgr2=0x7f9fe815c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9feecb5640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe8155e20 0x7f9fe815c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9feecb5640 1 -- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fd40047c0 msgr2=0x7f9fe81f6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9feecb5640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fd40047c0 0x7f9fe81f6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9feecb5640 1 -- 172.21.15.27:0/3927762923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fe815d2d0 con 0x7f9fe806c490 2023-12-19T18:19:43.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9feecb5640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe806c490 0x7f9fe806c910 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8066790 tx=0x7f9fe00115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9fdeffd640 1 -- 172.21.15.27:0/3927762923 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe000e070 con 0x7f9fe806c490 2023-12-19T18:19:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9fdeffd640 1 -- 172.21.15.27:0/3927762923 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe0011ed0 con 0x7f9fe806c490 2023-12-19T18:19:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9fdeffd640 1 -- 172.21.15.27:0/3927762923 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe0003990 con 0x7f9fe806c490 2023-12-19T18:19:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.754+0000 7f9ff0718640 1 -- 172.21.15.27:0/3927762923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe815d5b0 con 0x7f9fe806c490 2023-12-19T18:19:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.755+0000 7f9ff0718640 1 -- 172.21.15.27:0/3927762923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe815b8f0 con 0x7f9fe806c490 2023-12-19T18:19:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.756+0000 7f9ff0718640 1 -- 172.21.15.27:0/3927762923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fe8068e90 con 0x7f9fe806c490 2023-12-19T18:19:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.757+0000 7f9fdeffd640 1 -- 172.21.15.27:0/3927762923 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9fe0003af0 con 0x7f9fe806c490 2023-12-19T18:19:43.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.758+0000 7f9fdeffd640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fcc06a430 0x7f9fcc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:43.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.758+0000 7f9fdeffd640 1 -- 172.21.15.27:0/3927762923 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9fe00890d0 con 0x7f9fe806c490 2023-12-19T18:19:43.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.758+0000 7f9fee4b4640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fcc06a430 0x7f9fcc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:43.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.759+0000 7f9fee4b4640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fcc06a430 0x7f9fcc06c8f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4001050 tx=0x7f9fe4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:43.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.762+0000 7f9fdeffd640 1 -- 172.21.15.27:0/3927762923 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9fe0054670 con 0x7f9fe806c490 2023-12-19T18:19:43.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.953+0000 7f9ff0718640 1 -- 172.21.15.27:0/3927762923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9fe80cc870 con 0x7f9fe806c490 2023-12-19T18:19:43.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.954+0000 7f9fdeffd640 1 -- 172.21.15.27:0/3927762923 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9fe0057c90 con 0x7f9fe806c490 2023-12-19T18:19:43.954 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:19:43.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.956+0000 7f9ff0718640 1 -- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fcc06a430 msgr2=0x7f9fcc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.956+0000 7f9ff0718640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fcc06a430 0x7f9fcc06c8f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9fe4001050 tx=0x7f9fe4009040 comp rx=0 tx=0).stop 2023-12-19T18:19:43.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.956+0000 7f9ff0718640 1 -- 172.21.15.27:0/3927762923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe806c490 msgr2=0x7f9fe806c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:43.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.956+0000 7f9ff0718640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe806c490 0x7f9fe806c910 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8066790 tx=0x7f9fe00115a0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.957+0000 7f9ff0718640 1 -- 172.21.15.27:0/3927762923 shutdown_connections 2023-12-19T18:19:43.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.957+0000 7f9ff0718640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fcc06a430 0x7f9fcc06c8f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.957+0000 7f9ff0718640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fe806c490 0x7f9fe806c910 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.957+0000 7f9ff0718640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fe8155e20 0x7f9fe815c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.957+0000 7f9ff0718640 1 --2- 172.21.15.27:0/3927762923 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fd40047c0 0x7f9fe81f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:43.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.957+0000 7f9ff0718640 1 -- 172.21.15.27:0/3927762923 >> 172.21.15.27:0/3927762923 conn(0x7f9fe80cebd0 msgr2=0x7f9fe80cb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:43.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.957+0000 7f9ff0718640 1 -- 172.21.15.27:0/3927762923 shutdown_connections 2023-12-19T18:19:43.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:43.957+0000 7f9ff0718640 1 -- 172.21.15.27:0/3927762923 wait complete. 2023-12-19T18:19:43.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:19:44.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.208+0000 7f8e33fff640 1 Processor -- start 2023-12-19T18:19:44.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.208+0000 7f8e33fff640 1 -- start start 2023-12-19T18:19:44.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.210+0000 7f8e33fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8e3415ee30 0x7f8e340c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.210+0000 7f8e33fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e340afeb0 0x7f8e340c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.210+0000 7f8e33fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e340cb350 con 0x7f8e341553a0 2023-12-19T18:19:44.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.210+0000 7f8e33fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e340cbcf0 con 0x7f8e3415ee30 2023-12-19T18:19:44.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.210+0000 7f8e33fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e340cb630 con 0x7f8e340afeb0 2023-12-19T18:19:44.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.211+0000 7f8e327fc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8e3415ee30 0x7f8e340c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.211+0000 7f8e337fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e340afeb0 0x7f8e340c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.211+0000 7f8e327fc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8e3415ee30 0x7f8e340c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:44130/0 (socket says 172.21.15.27:44130) 2023-12-19T18:19:44.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.211+0000 7f8e327fc640 1 -- 172.21.15.27:0/2117246180 learned_addr learned my addr 172.21.15.27:0/2117246180 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:44.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.212+0000 7f8e337fe640 1 -- 172.21.15.27:0/2117246180 >> v2:172.21.15.192:3300/0 conn(0x7f8e3415ee30 msgr2=0x7f8e340c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:44.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.212+0000 7f8e337fe640 1 --2- 172.21.15.27:0/2117246180 >> v2:172.21.15.192:3300/0 conn(0x7f8e3415ee30 0x7f8e340c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.212+0000 7f8e337fe640 1 -- 172.21.15.27:0/2117246180 >> v1:172.21.15.192:6789/0 conn(0x7f8e341553a0 legacy=0x7f8e3415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:44.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.212+0000 7f8e337fe640 1 -- 172.21.15.27:0/2117246180 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e340cb910 con 0x7f8e340afeb0 2023-12-19T18:19:44.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.214+0000 7f8e337fe640 1 --2- 172.21.15.27:0/2117246180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e340afeb0 0x7f8e340c39d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009d90 tx=0x7f8e2c02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=736be98495b20c70 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:44.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.214+0000 7f8e31ffb640 1 -- 172.21.15.27:0/2117246180 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c02cbc0 con 0x7f8e340afeb0 2023-12-19T18:19:44.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.214+0000 7f8e31ffb640 1 -- 172.21.15.27:0/2117246180 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e2c02cd20 con 0x7f8e340afeb0 2023-12-19T18:19:44.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.214+0000 7f8e31ffb640 1 -- 172.21.15.27:0/2117246180 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c034660 con 0x7f8e340afeb0 2023-12-19T18:19:44.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.215+0000 7f8e33fff640 1 -- 172.21.15.27:0/2117246180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e340afeb0 msgr2=0x7f8e340c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:44.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.215+0000 7f8e33fff640 1 --2- 172.21.15.27:0/2117246180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e340afeb0 0x7f8e340c39d0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c009d90 tx=0x7f8e2c02be10 comp rx=0 tx=0).stop 2023-12-19T18:19:44.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.215+0000 7f8e33fff640 1 -- 172.21.15.27:0/2117246180 shutdown_connections 2023-12-19T18:19:44.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.215+0000 7f8e33fff640 1 --2- 172.21.15.27:0/2117246180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e340afeb0 0x7f8e340c39d0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.215+0000 7f8e33fff640 1 --2- 172.21.15.27:0/2117246180 >> v2:172.21.15.192:3300/0 conn(0x7f8e3415ee30 0x7f8e340c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.215+0000 7f8e33fff640 1 -- 172.21.15.27:0/2117246180 >> 172.21.15.27:0/2117246180 conn(0x7f8e34069c70 msgr2=0x7f8e3406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:44.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.215+0000 7f8e33fff640 1 -- 172.21.15.27:0/2117246180 shutdown_connections 2023-12-19T18:19:44.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.216+0000 7f8e33fff640 1 -- 172.21.15.27:0/2117246180 wait complete. 2023-12-19T18:19:44.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.218+0000 7f8e33fff640 1 Processor -- start 2023-12-19T18:19:44.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.218+0000 7f8e33fff640 1 -- start start 2023-12-19T18:19:44.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.219+0000 7f8e33fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e340afeb0 0x7f8e34148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.220+0000 7f8e33fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e341553a0 0x7f8e340cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.220+0000 7f8e33fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e3415ee30 0x7f8e340ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.220+0000 7f8e33fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e340cd730 con 0x7f8e340afeb0 2023-12-19T18:19:44.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.220+0000 7f8e33fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e34148940 con 0x7f8e341553a0 2023-12-19T18:19:44.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.220+0000 7f8e33fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e340ccab0 con 0x7f8e3415ee30 2023-12-19T18:19:44.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.220+0000 7f8e32ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e340afeb0 0x7f8e34148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.220+0000 7f8e32ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e340afeb0 0x7f8e34148400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44632/0 (socket says 172.21.15.27:44632) 2023-12-19T18:19:44.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.220+0000 7f8e32ffd640 1 -- 172.21.15.27:0/3684794307 learned_addr learned my addr 172.21.15.27:0/3684794307 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:44.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.220+0000 7f8e337fe640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e3415ee30 0x7f8e340ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.220+0000 7f8e327fc640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e341553a0 0x7f8e340cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.220+0000 7f8e32ffd640 1 -- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e3415ee30 msgr2=0x7f8e340ad910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:44.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.221+0000 7f8e32ffd640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e3415ee30 0x7f8e340ad910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.221+0000 7f8e32ffd640 1 -- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e341553a0 msgr2=0x7f8e340cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:44.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.221+0000 7f8e32ffd640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e341553a0 0x7f8e340cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.221+0000 7f8e32ffd640 1 -- 172.21.15.27:0/3684794307 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e2c009a70 con 0x7f8e340afeb0 2023-12-19T18:19:44.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.221+0000 7f8e32ffd640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e340afeb0 0x7f8e34148400 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400c450 tx=0x7f8e2400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:44.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.221+0000 7f8e1b7fe640 1 -- 172.21.15.27:0/3684794307 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e240052d0 con 0x7f8e340afeb0 2023-12-19T18:19:44.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.221+0000 7f8e1b7fe640 1 -- 172.21.15.27:0/3684794307 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e24005430 con 0x7f8e340afeb0 2023-12-19T18:19:44.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.221+0000 7f8e1b7fe640 1 -- 172.21.15.27:0/3684794307 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e24012870 con 0x7f8e340afeb0 2023-12-19T18:19:44.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.221+0000 7f8e33fff640 1 -- 172.21.15.27:0/3684794307 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e340ccd90 con 0x7f8e340afeb0 2023-12-19T18:19:44.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.221+0000 7f8e33fff640 1 -- 172.21.15.27:0/3684794307 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e340cd350 con 0x7f8e340afeb0 2023-12-19T18:19:44.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.223+0000 7f8e33fff640 1 -- 172.21.15.27:0/3684794307 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3404e840 con 0x7f8e340afeb0 2023-12-19T18:19:44.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.223+0000 7f8e1b7fe640 1 -- 172.21.15.27:0/3684794307 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8e240129d0 con 0x7f8e340afeb0 2023-12-19T18:19:44.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.224+0000 7f8e1b7fe640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8e2006a430 0x7f8e2006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.224+0000 7f8e1b7fe640 1 -- 172.21.15.27:0/3684794307 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f8e24088460 con 0x7f8e340afeb0 2023-12-19T18:19:44.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.225+0000 7f8e327fc640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8e2006a430 0x7f8e2006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.226+0000 7f8e327fc640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8e2006a430 0x7f8e2006c8f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8e34066120 tx=0x7f8e28008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:44.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.229+0000 7f8e1b7fe640 1 -- 172.21.15.27:0/3684794307 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8e24053a00 con 0x7f8e340afeb0 2023-12-19T18:19:44.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.461+0000 7f8e33fff640 1 -- 172.21.15.27:0/3684794307 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8e340af6a0 con 0x7f8e340afeb0 2023-12-19T18:19:44.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.462+0000 7f8e1b7fe640 1 -- 172.21.15.27:0/3684794307 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f8e24057020 con 0x7f8e340afeb0 2023-12-19T18:19:44.462 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:19:44.462 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1624358030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1624358030},{"type":"v1","addr":"172.21.15.192:6836","nonce":1624358030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5030,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3025339386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3025339386},{"type":"v1","addr":"172.21.15.192:6837","nonce":3025339386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/1685117850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1685117850},{"type":"v1","addr":"172.21.15.27:6833","nonce":1685117850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:18:26.147336+0000","modified":"2023-12-19T18:18:29.663526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2772294131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2772294131},{"type":"v1","addr":"172.21.15.27:6835","nonce":2772294131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-19T18:19:44.462 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2023-12-19T18:19:44.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.465+0000 7f8e33fff640 1 -- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8e2006a430 msgr2=0x7f8e2006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:44.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.465+0000 7f8e33fff640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8e2006a430 0x7f8e2006c8f0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8e34066120 tx=0x7f8e28008040 comp rx=0 tx=0).stop 2023-12-19T18:19:44.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.465+0000 7f8e33fff640 1 -- 172.21.15.27:0/3684794307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e340afeb0 msgr2=0x7f8e34148400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:44.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.465+0000 7f8e33fff640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e340afeb0 0x7f8e34148400 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f8e2400c450 tx=0x7f8e2400c910 comp rx=0 tx=0).stop 2023-12-19T18:19:44.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.465+0000 7f8e33fff640 1 -- 172.21.15.27:0/3684794307 shutdown_connections 2023-12-19T18:19:44.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.465+0000 7f8e33fff640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8e2006a430 0x7f8e2006c8f0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.465+0000 7f8e33fff640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8e3415ee30 0x7f8e340ad910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.465+0000 7f8e33fff640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8e341553a0 0x7f8e340cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.465+0000 7f8e33fff640 1 --2- 172.21.15.27:0/3684794307 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e340afeb0 0x7f8e34148400 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.465+0000 7f8e33fff640 1 -- 172.21.15.27:0/3684794307 >> 172.21.15.27:0/3684794307 conn(0x7f8e34069c70 msgr2=0x7f8e34159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:44.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.465+0000 7f8e33fff640 1 -- 172.21.15.27:0/3684794307 shutdown_connections 2023-12-19T18:19:44.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.466+0000 7f8e33fff640 1 -- 172.21.15.27:0/3684794307 wait complete. 2023-12-19T18:19:44.485 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:19:44.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.721+0000 7f739359e640 1 Processor -- start 2023-12-19T18:19:44.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.721+0000 7f739359e640 1 -- start start 2023-12-19T18:19:44.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.724+0000 7f739359e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f739415ee30 0x7f73940c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.724+0000 7f739359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73940cb350 con 0x7f73941553a0 2023-12-19T18:19:44.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.724+0000 7f739359e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73940cb630 con 0x7f73940afeb0 2023-12-19T18:19:44.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.724+0000 7f739359e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73940cbcf0 con 0x7f739415ee30 2023-12-19T18:19:44.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.724+0000 7f739259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f73941553a0 0x7f739415e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49196/0 (socket says 172.21.15.27:49196) 2023-12-19T18:19:44.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.724+0000 7f739259c640 1 -- 172.21.15.27:0/2812137519 learned_addr learned my addr 172.21.15.27:0/2812137519 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:44.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.724+0000 7f7391d9b640 1 --2- 172.21.15.27:0/2812137519 >> v2:172.21.15.192:3300/0 conn(0x7f739415ee30 0x7f73940c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.724+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2793015272 0 0) 0x7f73940cb350 con 0x7f73941553a0 2023-12-19T18:19:44.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.724+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f737c003520 con 0x7f73941553a0 2023-12-19T18:19:44.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2739588843 0 0) 0x7f737c003520 con 0x7f73941553a0 2023-12-19T18:19:44.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 >> v2:172.21.15.192:3300/0 conn(0x7f739415ee30 msgr2=0x7f73940c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:44.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f739159a640 1 --2- 172.21.15.27:0/2812137519 >> v2:172.21.15.192:3300/0 conn(0x7f739415ee30 0x7f73940c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 >> v1:172.21.15.192:6789/0 conn(0x7f73940afeb0 legacy=0x7f73940c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:44.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73940cb910 con 0x7f73941553a0 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f7384001670 con 0x7f73941553a0 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 >> v1:172.21.15.27:6789/0 conn(0x7f73941553a0 legacy=0x7f739415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f739159a640 1 --2- 172.21.15.27:0/2812137519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f737c003970 0x7f737c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f739159a640 1 --2- 172.21.15.27:0/2812137519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f737c0047c0 0x7f737c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f739159a640 1 --2- 172.21.15.27:0/2812137519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f737c005180 0x7f737c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73940cbcf0 con 0x7f737c0047c0 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737c00c120 con 0x7f737c005180 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f737c00c2e0 con 0x7f737c003970 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.725+0000 7f7391d9b640 1 --2- 172.21.15.27:0/2812137519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f737c0047c0 0x7f737c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.726+0000 7f739259c640 1 --2- 172.21.15.27:0/2812137519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f737c003970 0x7f737c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.726+0000 7f7392d9d640 1 --2- 172.21.15.27:0/2812137519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f737c005180 0x7f737c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.726+0000 7f7391d9b640 1 -- 172.21.15.27:0/2812137519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f737c003970 msgr2=0x7f737c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.726+0000 7f7391d9b640 1 --2- 172.21.15.27:0/2812137519 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f737c003970 0x7f737c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.726+0000 7f7391d9b640 1 -- 172.21.15.27:0/2812137519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f737c005180 msgr2=0x7f737c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:44.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.726+0000 7f7391d9b640 1 --2- 172.21.15.27:0/2812137519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f737c005180 0x7f737c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.726+0000 7f7391d9b640 1 -- 172.21.15.27:0/2812137519 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c003520 con 0x7f737c0047c0 2023-12-19T18:19:44.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.726+0000 7f7392d9d640 1 --2- 172.21.15.27:0/2812137519 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f737c005180 0x7f737c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:44.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.726+0000 7f7392d9d640 1 -- 172.21.15.27:0/2812137519 reap_dead start 2023-12-19T18:19:44.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.727+0000 7f7391d9b640 1 --2- 172.21.15.27:0/2812137519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f737c0047c0 0x7f737c004c40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7388009ac0 tx=0x7f737c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8301d886d057bc4d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:44.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.727+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738802c6b0 con 0x7f737c0047c0 2023-12-19T18:19:44.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.727+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738802c810 con 0x7f737c0047c0 2023-12-19T18:19:44.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.727+0000 7f739159a640 1 -- 172.21.15.27:0/2812137519 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738802cba0 con 0x7f737c0047c0 2023-12-19T18:19:44.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.728+0000 7f739359e640 1 -- 172.21.15.27:0/2812137519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f737c0047c0 msgr2=0x7f737c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:44.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.728+0000 7f739359e640 1 --2- 172.21.15.27:0/2812137519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f737c0047c0 0x7f737c004c40 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f7388009ac0 tx=0x7f737c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.728+0000 7f739359e640 1 -- 172.21.15.27:0/2812137519 shutdown_connections 2023-12-19T18:19:44.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.728+0000 7f739359e640 1 --2- 172.21.15.27:0/2812137519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f737c0047c0 0x7f737c004c40 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.728+0000 7f739359e640 1 -- 172.21.15.27:0/2812137519 >> 172.21.15.27:0/2812137519 conn(0x7f7394069c70 msgr2=0x7f739406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:44.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.728+0000 7f739359e640 1 -- 172.21.15.27:0/2812137519 shutdown_connections 2023-12-19T18:19:44.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.728+0000 7f739359e640 1 -- 172.21.15.27:0/2812137519 wait complete. 2023-12-19T18:19:44.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.730+0000 7f739359e640 1 Processor -- start 2023-12-19T18:19:44.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.730+0000 7f739359e640 1 -- start start 2023-12-19T18:19:44.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.731+0000 7f739359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f737c0047c0 0x7f7394152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.731+0000 7f739359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73940b3e10 0x7f73940b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.731+0000 7f739359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7394153530 0x7f73941539b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.731+0000 7f739359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73940cc8f0 con 0x7f73940b3e10 2023-12-19T18:19:44.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.731+0000 7f739359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73940b47d0 con 0x7f737c0047c0 2023-12-19T18:19:44.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.731+0000 7f739359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73941542f0 con 0x7f7394153530 2023-12-19T18:19:44.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.732+0000 7f7391d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73940b3e10 0x7f73940b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.732+0000 7f7391d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73940b3e10 0x7f73940b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44638/0 (socket says 172.21.15.27:44638) 2023-12-19T18:19:44.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.732+0000 7f7391d9b640 1 -- 172.21.15.27:0/3269032468 learned_addr learned my addr 172.21.15.27:0/3269032468 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.732+0000 7f7392d9d640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7394153530 0x7f73941539b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.732+0000 7f739259c640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f737c0047c0 0x7f7394152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.732+0000 7f7391d9b640 1 -- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7394153530 msgr2=0x7f73941539b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.732+0000 7f7391d9b640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7394153530 0x7f73941539b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.732+0000 7f7391d9b640 1 -- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f737c0047c0 msgr2=0x7f7394152c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.732+0000 7f7391d9b640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f737c0047c0 0x7f7394152c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.732+0000 7f7391d9b640 1 -- 172.21.15.27:0/3269032468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73941545d0 con 0x7f73940b3e10 2023-12-19T18:19:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.733+0000 7f7391d9b640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73940b3e10 0x7f73940b4290 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f738802c250 tx=0x7f7388004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.733+0000 7f7382ffd640 1 -- 172.21.15.27:0/3269032468 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388003810 con 0x7f73940b3e10 2023-12-19T18:19:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.733+0000 7f7382ffd640 1 -- 172.21.15.27:0/3269032468 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7388003970 con 0x7f73940b3e10 2023-12-19T18:19:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.733+0000 7f7382ffd640 1 -- 172.21.15.27:0/3269032468 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388003d00 con 0x7f73940b3e10 2023-12-19T18:19:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.733+0000 7f739359e640 1 -- 172.21.15.27:0/3269032468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73941548b0 con 0x7f73940b3e10 2023-12-19T18:19:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.733+0000 7f739359e640 1 -- 172.21.15.27:0/3269032468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7394153150 con 0x7f73940b3e10 2023-12-19T18:19:44.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.734+0000 7f739359e640 1 -- 172.21.15.27:0/3269032468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f739404e840 con 0x7f73940b3e10 2023-12-19T18:19:44.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.735+0000 7f7382ffd640 1 -- 172.21.15.27:0/3269032468 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7388034440 con 0x7f73940b3e10 2023-12-19T18:19:44.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.736+0000 7f7382ffd640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f737406a430 0x7f737406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:44.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.736+0000 7f7382ffd640 1 -- 172.21.15.27:0/3269032468 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f73880aacd0 con 0x7f73940b3e10 2023-12-19T18:19:44.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.737+0000 7f739259c640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f737406a430 0x7f737406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:44.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.738+0000 7f739259c640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f737406a430 0x7f737406c8f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7384000f60 tx=0x7f7384000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:44.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.741+0000 7f7382ffd640 1 -- 172.21.15.27:0/3269032468 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7388076320 con 0x7f73940b3e10 2023-12-19T18:19:44.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.937+0000 7f739359e640 1 -- 172.21.15.27:0/3269032468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73940af6a0 con 0x7f73940b3e10 2023-12-19T18:19:44.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.941+0000 7f7382ffd640 1 -- 172.21.15.27:0/3269032468 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16193 (secure 0 0 0) 0x7f738804c090 con 0x7f73940b3e10 2023-12-19T18:19:44.941 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:19:44.941 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":84,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:19:41.696772+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:24.350728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:25.147723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.14","mappings":[{"from":6,"to":4}]},{"pgid":"9.2c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:19:44.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.944+0000 7f739359e640 1 -- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f737406a430 msgr2=0x7f737406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:44.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.944+0000 7f739359e640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f737406a430 0x7f737406c8f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7384000f60 tx=0x7f7384000ed0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.944+0000 7f739359e640 1 -- 172.21.15.27:0/3269032468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73940b3e10 msgr2=0x7f73940b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:44.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.944+0000 7f739359e640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73940b3e10 0x7f73940b4290 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f738802c250 tx=0x7f7388004730 comp rx=0 tx=0).stop 2023-12-19T18:19:44.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.944+0000 7f739359e640 1 -- 172.21.15.27:0/3269032468 shutdown_connections 2023-12-19T18:19:44.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.944+0000 7f739359e640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f737406a430 0x7f737406c8f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.944+0000 7f739359e640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7394153530 0x7f73941539b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.944+0000 7f739359e640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73940b3e10 0x7f73940b4290 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.944+0000 7f739359e640 1 --2- 172.21.15.27:0/3269032468 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f737c0047c0 0x7f7394152c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.944+0000 7f739359e640 1 -- 172.21.15.27:0/3269032468 >> 172.21.15.27:0/3269032468 conn(0x7f7394069c70 msgr2=0x7f7394158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.944+0000 7f739359e640 1 -- 172.21.15.27:0/3269032468 shutdown_connections 2023-12-19T18:19:44.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:44.944+0000 7f739359e640 1 -- 172.21.15.27:0/3269032468 wait complete. 2023-12-19T18:19:44.966 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:19:44.966 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:19:45.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.198+0000 7f853ee41640 1 Processor -- start 2023-12-19T18:19:45.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.199+0000 7f853ee41640 1 -- start start 2023-12-19T18:19:45.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.200+0000 7f853ee41640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f853815af70 0x7f85380ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.200+0000 7f853ee41640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380adc20 0x7f853806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.200+0000 7f853ee41640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85380cb850 con 0x7f85380cbce0 2023-12-19T18:19:45.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.200+0000 7f853ee41640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853815b350 con 0x7f853815af70 2023-12-19T18:19:45.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.200+0000 7f853ee41640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85380cac40 con 0x7f85380adc20 2023-12-19T18:19:45.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.201+0000 7f853cbdd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f853815af70 0x7f85380ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.201+0000 7f853d3de640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380adc20 0x7f853806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.201+0000 7f853cbdd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f853815af70 0x7f85380ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:44160/0 (socket says 172.21.15.27:44160) 2023-12-19T18:19:45.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.201+0000 7f853cbdd640 1 -- 172.21.15.27:0/2329357104 learned_addr learned my addr 172.21.15.27:0/2329357104 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:45.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.202+0000 7f853cbdd640 1 -- 172.21.15.27:0/2329357104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380adc20 msgr2=0x7f853806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:45.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.202+0000 7f853cbdd640 1 --2- 172.21.15.27:0/2329357104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380adc20 0x7f853806b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.202+0000 7f853cbdd640 1 -- 172.21.15.27:0/2329357104 >> v1:172.21.15.192:6789/0 conn(0x7f85380cbce0 legacy=0x7f85380ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:45.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.202+0000 7f853cbdd640 1 -- 172.21.15.27:0/2329357104 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85380caf20 con 0x7f853815af70 2023-12-19T18:19:45.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.202+0000 7f853d3de640 1 --2- 172.21.15.27:0/2329357104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380adc20 0x7f853806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:19:45.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.204+0000 7f853cbdd640 1 --2- 172.21.15.27:0/2329357104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f853815af70 0x7f85380ca540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f852c009870 tx=0x7f852c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4b608b923189b27 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:45.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.204+0000 7f85377fe640 1 -- 172.21.15.27:0/2329357104 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c030840 con 0x7f853815af70 2023-12-19T18:19:45.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.204+0000 7f85377fe640 1 -- 172.21.15.27:0/2329357104 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852c0309a0 con 0x7f853815af70 2023-12-19T18:19:45.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.204+0000 7f85377fe640 1 -- 172.21.15.27:0/2329357104 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c030d30 con 0x7f853815af70 2023-12-19T18:19:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.205+0000 7f853ee41640 1 -- 172.21.15.27:0/2329357104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f853815af70 msgr2=0x7f85380ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.205+0000 7f853ee41640 1 --2- 172.21.15.27:0/2329357104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f853815af70 0x7f85380ca540 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f852c009870 tx=0x7f852c02b980 comp rx=0 tx=0).stop 2023-12-19T18:19:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.205+0000 7f853ee41640 1 -- 172.21.15.27:0/2329357104 shutdown_connections 2023-12-19T18:19:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.205+0000 7f853ee41640 1 --2- 172.21.15.27:0/2329357104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380adc20 0x7f853806b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.205+0000 7f853ee41640 1 --2- 172.21.15.27:0/2329357104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f853815af70 0x7f85380ca540 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.205+0000 7f853ee41640 1 -- 172.21.15.27:0/2329357104 >> 172.21.15.27:0/2329357104 conn(0x7f853806a330 msgr2=0x7f85380b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.205+0000 7f853ee41640 1 -- 172.21.15.27:0/2329357104 shutdown_connections 2023-12-19T18:19:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.205+0000 7f853ee41640 1 -- 172.21.15.27:0/2329357104 wait complete. 2023-12-19T18:19:45.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.208+0000 7f853ee41640 1 Processor -- start 2023-12-19T18:19:45.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.208+0000 7f853ee41640 1 -- start start 2023-12-19T18:19:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.209+0000 7f853ee41640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85380adc20 0x7f85380ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.209+0000 7f853ee41640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380cbce0 0x7f85380c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.209+0000 7f853ee41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f853815af70 0x7f8538070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.209+0000 7f853ee41640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8538152af0 con 0x7f853815af70 2023-12-19T18:19:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.209+0000 7f853ee41640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f853815cbb0 con 0x7f85380adc20 2023-12-19T18:19:45.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.209+0000 7f853ee41640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85380c8b30 con 0x7f85380cbce0 2023-12-19T18:19:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.210+0000 7f853d3de640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f853815af70 0x7f8538070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.210+0000 7f853d3de640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f853815af70 0x7f8538070dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44650/0 (socket says 172.21.15.27:44650) 2023-12-19T18:19:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.210+0000 7f853d3de640 1 -- 172.21.15.27:0/102837683 learned_addr learned my addr 172.21.15.27:0/102837683 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.210+0000 7f853cbdd640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85380adc20 0x7f85380ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.210+0000 7f8537fff640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380cbce0 0x7f85380c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.210+0000 7f853d3de640 1 -- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380cbce0 msgr2=0x7f85380c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.210+0000 7f853d3de640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380cbce0 0x7f85380c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.210+0000 7f853d3de640 1 -- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85380adc20 msgr2=0x7f85380ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.210+0000 7f853d3de640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85380adc20 0x7f85380ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.210+0000 7f853d3de640 1 -- 172.21.15.27:0/102837683 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8530009a70 con 0x7f853815af70 2023-12-19T18:19:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.210+0000 7f8537fff640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380cbce0 0x7f85380c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:19:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.211+0000 7f853d3de640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f853815af70 0x7f8538070dd0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f853000efc0 tx=0x7f853000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.211+0000 7f85357fa640 1 -- 172.21.15.27:0/102837683 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530009cd0 con 0x7f853815af70 2023-12-19T18:19:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.211+0000 7f85357fa640 1 -- 172.21.15.27:0/102837683 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f853000f040 con 0x7f853815af70 2023-12-19T18:19:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.211+0000 7f85357fa640 1 -- 172.21.15.27:0/102837683 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530003990 con 0x7f853815af70 2023-12-19T18:19:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.211+0000 7f853ee41640 1 -- 172.21.15.27:0/102837683 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f852c0094d0 con 0x7f853815af70 2023-12-19T18:19:45.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.211+0000 7f853ee41640 1 -- 172.21.15.27:0/102837683 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f853815c720 con 0x7f853815af70 2023-12-19T18:19:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.213+0000 7f853ee41640 1 -- 172.21.15.27:0/102837683 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8538068e90 con 0x7f853815af70 2023-12-19T18:19:45.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.214+0000 7f85357fa640 1 -- 172.21.15.27:0/102837683 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8530014480 con 0x7f853815af70 2023-12-19T18:19:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.215+0000 7f85357fa640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f851c06a430 0x7f851c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.215+0000 7f85357fa640 1 -- 172.21.15.27:0/102837683 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f85300894d0 con 0x7f853815af70 2023-12-19T18:19:45.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.215+0000 7f853cbdd640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f851c06a430 0x7f851c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.216+0000 7f853cbdd640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f851c06a430 0x7f851c06c8f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f852c036000 tx=0x7f852c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:45.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.219+0000 7f85357fa640 1 -- 172.21.15.27:0/102837683 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f85300549f0 con 0x7f853815af70 2023-12-19T18:19:45.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.449+0000 7f853ee41640 1 -- 172.21.15.27:0/102837683 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f853815c8b0 con 0x7f853815af70 2023-12-19T18:19:45.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.450+0000 7f85357fa640 1 -- 172.21.15.27:0/102837683 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4715 (secure 0 0 0) 0x7f8530058010 con 0x7f853815af70 2023-12-19T18:19:45.451 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:19:45.451 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1624358030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1624358030},{"type":"v1","addr":"172.21.15.192:6836","nonce":1624358030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5030,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3025339386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3025339386},{"type":"v1","addr":"172.21.15.192:6837","nonce":3025339386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/1685117850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1685117850},{"type":"v1","addr":"172.21.15.27:6833","nonce":1685117850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:18:26.147336+0000","modified":"2023-12-19T18:18:29.663526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2772294131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2772294131},{"type":"v1","addr":"172.21.15.27:6835","nonce":2772294131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-19T18:19:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2023-12-19T18:19:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.453+0000 7f853ee41640 1 -- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f851c06a430 msgr2=0x7f851c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.453+0000 7f853ee41640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f851c06a430 0x7f851c06c8f0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f852c036000 tx=0x7f852c008030 comp rx=0 tx=0).stop 2023-12-19T18:19:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.453+0000 7f853ee41640 1 -- 172.21.15.27:0/102837683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f853815af70 msgr2=0x7f8538070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.453+0000 7f853ee41640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f853815af70 0x7f8538070dd0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f853000efc0 tx=0x7f853000c490 comp rx=0 tx=0).stop 2023-12-19T18:19:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.453+0000 7f853ee41640 1 -- 172.21.15.27:0/102837683 shutdown_connections 2023-12-19T18:19:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.453+0000 7f853ee41640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f851c06a430 0x7f851c06c8f0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.453+0000 7f853ee41640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f853815af70 0x7f8538070dd0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.453+0000 7f853ee41640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f85380cbce0 0x7f85380c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.453+0000 7f853ee41640 1 --2- 172.21.15.27:0/102837683 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f85380adc20 0x7f85380ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.453+0000 7f853ee41640 1 -- 172.21.15.27:0/102837683 >> 172.21.15.27:0/102837683 conn(0x7f853806a330 msgr2=0x7f85380cd840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.454+0000 7f853ee41640 1 -- 172.21.15.27:0/102837683 shutdown_connections 2023-12-19T18:19:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.454+0000 7f853ee41640 1 -- 172.21.15.27:0/102837683 wait complete. 2023-12-19T18:19:45.475 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:19:45.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.693+0000 7ff00d74c640 1 Processor -- start 2023-12-19T18:19:45.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.693+0000 7ff00d74c640 1 -- start start 2023-12-19T18:19:45.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.694+0000 7ff00d74c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff008154030 0x7ff0080cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.695+0000 7ff00d74c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0080c5560 0x7ff0080afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.695+0000 7ff00d74c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0080cc190 con 0x7ff0080af630 2023-12-19T18:19:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.695+0000 7ff00d74c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0080b0000 con 0x7ff008154030 2023-12-19T18:19:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.695+0000 7ff00d74c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080cb2a0 con 0x7ff0080c5560 2023-12-19T18:19:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.695+0000 7ff0077fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff0080af630 0x7ff00815d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49206/0 (socket says 172.21.15.27:49206) 2023-12-19T18:19:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.695+0000 7ff0077fe640 1 -- 172.21.15.27:0/3341762497 learned_addr learned my addr 172.21.15.27:0/3341762497 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.695+0000 7ff007fff640 1 --2- 172.21.15.27:0/3341762497 >> v2:172.21.15.192:3300/0 conn(0x7ff008154030 0x7ff0080cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.695+0000 7ff00cf4b640 1 --2- 172.21.15.27:0/3341762497 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0080c5560 0x7ff0080afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.695+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3167835066 0 0) 0x7ff0080cc190 con 0x7ff0080af630 2023-12-19T18:19:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feff4005730 con 0x7ff0080af630 2023-12-19T18:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2457306825 0 0) 0x7feff4005730 con 0x7ff0080af630 2023-12-19T18:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0080c5560 msgr2=0x7ff0080afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 --2- 172.21.15.27:0/3341762497 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0080c5560 0x7ff0080afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 >> v2:172.21.15.192:3300/0 conn(0x7ff008154030 msgr2=0x7ff0080cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 --2- 172.21.15.27:0/3341762497 >> v2:172.21.15.192:3300/0 conn(0x7ff008154030 0x7ff0080cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0080cb580 con 0x7ff0080af630 2023-12-19T18:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7feff0001670 con 0x7ff0080af630 2023-12-19T18:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 >> v1:172.21.15.27:6789/0 conn(0x7ff0080af630 legacy=0x7ff00815d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 --2- 172.21.15.27:0/3341762497 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0080c5560 0x7feff4005b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 --2- 172.21.15.27:0/3341762497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff4006150 0x7feff40069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 --2- 172.21.15.27:0/3341762497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feff4006f10 0x7feff4007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080b0000 con 0x7feff4006150 2023-12-19T18:19:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080cb2a0 con 0x7feff4006f10 2023-12-19T18:19:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feff400db90 con 0x7ff0080c5560 2023-12-19T18:19:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.696+0000 7ff0077fe640 1 --2- 172.21.15.27:0/3341762497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff4006150 0x7feff40069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.697+0000 7ff00cf4b640 1 --2- 172.21.15.27:0/3341762497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feff4006f10 0x7feff4007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.697+0000 7ff0077fe640 1 -- 172.21.15.27:0/3341762497 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0080c5560 msgr2=0x7feff4005b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.697+0000 7ff0077fe640 1 --2- 172.21.15.27:0/3341762497 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff0080c5560 0x7feff4005b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.697+0000 7ff0077fe640 1 -- 172.21.15.27:0/3341762497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feff4006f10 msgr2=0x7feff4007390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.697+0000 7ff0077fe640 1 --2- 172.21.15.27:0/3341762497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feff4006f10 0x7feff4007390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.697+0000 7ff0077fe640 1 -- 172.21.15.27:0/3341762497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feff4005730 con 0x7feff4006150 2023-12-19T18:19:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.697+0000 7ff00cf4b640 1 --2- 172.21.15.27:0/3341762497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feff4006f10 0x7feff4007390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.698+0000 7ff0077fe640 1 --2- 172.21.15.27:0/3341762497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff4006150 0x7feff40069d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7feff000c140 tx=0x7feff4007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fba9ade6130d0c69 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:45.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.698+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff0005ad0 con 0x7feff4006150 2023-12-19T18:19:45.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.698+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feff0005c30 con 0x7feff4006150 2023-12-19T18:19:45.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.698+0000 7ff006ffd640 1 -- 172.21.15.27:0/3341762497 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feff002ca70 con 0x7feff4006150 2023-12-19T18:19:45.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.699+0000 7ff00d74c640 1 -- 172.21.15.27:0/3341762497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff4006150 msgr2=0x7feff40069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:45.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.699+0000 7ff00d74c640 1 --2- 172.21.15.27:0/3341762497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff4006150 0x7feff40069d0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7feff000c140 tx=0x7feff4007a00 comp rx=0 tx=0).stop 2023-12-19T18:19:45.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.699+0000 7ff00cf4b640 1 -- 172.21.15.27:0/3341762497 reap_dead start 2023-12-19T18:19:45.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.699+0000 7ff00d74c640 1 -- 172.21.15.27:0/3341762497 shutdown_connections 2023-12-19T18:19:45.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.699+0000 7ff00d74c640 1 -- 172.21.15.27:0/3341762497 >> 172.21.15.27:0/3341762497 conn(0x7ff008069c70 msgr2=0x7ff00806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:45.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.699+0000 7ff00d74c640 1 -- 172.21.15.27:0/3341762497 shutdown_connections 2023-12-19T18:19:45.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.699+0000 7ff00d74c640 1 -- 172.21.15.27:0/3341762497 wait complete. 2023-12-19T18:19:45.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.701+0000 7ff00d74c640 1 Processor -- start 2023-12-19T18:19:45.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.701+0000 7ff00d74c640 1 -- start start 2023-12-19T18:19:45.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.702+0000 7ff00d74c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feff4006150 0x7ff0080c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.702+0000 7ff00d74c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff008143880 0x7ff0081f1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.702+0000 7ff00d74c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0081f2020 0x7ff00806a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.702+0000 7ff00d74c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080c62d0 con 0x7ff0081f2020 2023-12-19T18:19:45.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.702+0000 7ff00d74c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080c5ff0 con 0x7feff4006150 2023-12-19T18:19:45.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.702+0000 7ff00d74c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0080c83a0 con 0x7ff008143880 2023-12-19T18:19:45.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.702+0000 7ff00cf4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0081f2020 0x7ff00806a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.702+0000 7ff00cf4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0081f2020 0x7ff00806a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44670/0 (socket says 172.21.15.27:44670) 2023-12-19T18:19:45.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.702+0000 7ff00cf4b640 1 -- 172.21.15.27:0/1527573465 learned_addr learned my addr 172.21.15.27:0/1527573465 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:45.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.703+0000 7ff0077fe640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff008143880 0x7ff0081f1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.703+0000 7ff00cf4b640 1 -- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff008143880 msgr2=0x7ff0081f1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:45.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.703+0000 7ff00cf4b640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff008143880 0x7ff0081f1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.703+0000 7ff007fff640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feff4006150 0x7ff0080c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.703+0000 7ff00cf4b640 1 -- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feff4006150 msgr2=0x7ff0080c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:45.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.703+0000 7ff00cf4b640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feff4006150 0x7ff0080c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.703+0000 7ff00cf4b640 1 -- 172.21.15.27:0/1527573465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0080c8680 con 0x7ff0081f2020 2023-12-19T18:19:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.703+0000 7ff0077fe640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff008143880 0x7ff0081f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.703+0000 7ff00cf4b640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0081f2020 0x7ff00806a050 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff0080b0250 tx=0x7ff00000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.703+0000 7ff004ff9640 1 -- 172.21.15.27:0/1527573465 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff00000fe90 con 0x7ff0081f2020 2023-12-19T18:19:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.703+0000 7ff004ff9640 1 -- 172.21.15.27:0/1527573465 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff000011040 con 0x7ff0081f2020 2023-12-19T18:19:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.703+0000 7ff00d74c640 1 -- 172.21.15.27:0/1527573465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0080c8960 con 0x7ff0081f2020 2023-12-19T18:19:45.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.704+0000 7ff004ff9640 1 -- 172.21.15.27:0/1527573465 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff000017620 con 0x7ff0081f2020 2023-12-19T18:19:45.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.704+0000 7ff00d74c640 1 -- 172.21.15.27:0/1527573465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff008143510 con 0x7ff0081f2020 2023-12-19T18:19:45.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.705+0000 7ff00d74c640 1 -- 172.21.15.27:0/1527573465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0080c8200 con 0x7ff0081f2020 2023-12-19T18:19:45.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.706+0000 7ff004ff9640 1 -- 172.21.15.27:0/1527573465 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff0000203b0 con 0x7ff0081f2020 2023-12-19T18:19:45.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.707+0000 7ff004ff9640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fefe806a430 0x7fefe806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:45.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.707+0000 7ff004ff9640 1 -- 172.21.15.27:0/1527573465 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7ff000088a70 con 0x7ff0081f2020 2023-12-19T18:19:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.708+0000 7ff007fff640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fefe806a430 0x7fefe806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.709+0000 7ff007fff640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fefe806a430 0x7fefe806c8f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7feff8004770 tx=0x7feff8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.711+0000 7ff004ff9640 1 -- 172.21.15.27:0/1527573465 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff0080c8200 con 0x7ff0081f2020 2023-12-19T18:19:45.780 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:19:45.789 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:19:45.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.949+0000 7ff00d74c640 1 -- 172.21.15.27:0/1527573465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff008141220 con 0x7ff0081f2020 2023-12-19T18:19:45.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.950+0000 7ff004ff9640 1 -- 172.21.15.27:0/1527573465 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+4715 (secure 0 0 0) 0x7ff008141220 con 0x7ff0081f2020 2023-12-19T18:19:45.951 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:19:45.951 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":30,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1624358030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1624358030},{"type":"v1","addr":"172.21.15.192:6836","nonce":1624358030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5030,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3025339386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3025339386},{"type":"v1","addr":"172.21.15.192:6837","nonce":3025339386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/1685117850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1685117850},{"type":"v1","addr":"172.21.15.27:6833","nonce":1685117850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[{"mdsmap":{"epoch":30,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:18:26.147336+0000","modified":"2023-12-19T18:18:29.663526+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6123},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6123":{"gid":6123,"name":"c","rank":0,"incarnation":27,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2772294131","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2772294131},{"type":"v1","addr":"172.21.15.27:6835","nonce":2772294131}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-19T18:19:45.951 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2023-12-19T18:19:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.953+0000 7ff00d74c640 1 -- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fefe806a430 msgr2=0x7fefe806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.953+0000 7ff00d74c640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fefe806a430 0x7fefe806c8f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7feff8004770 tx=0x7feff8008040 comp rx=0 tx=0).stop 2023-12-19T18:19:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.953+0000 7ff00d74c640 1 -- 172.21.15.27:0/1527573465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0081f2020 msgr2=0x7ff00806a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.953+0000 7ff00d74c640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0081f2020 0x7ff00806a050 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7ff0080b0250 tx=0x7ff00000f4a0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.953+0000 7ff00d74c640 1 -- 172.21.15.27:0/1527573465 shutdown_connections 2023-12-19T18:19:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.953+0000 7ff00d74c640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fefe806a430 0x7fefe806c8f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.953+0000 7ff00d74c640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0081f2020 0x7ff00806a050 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.953+0000 7ff00d74c640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff008143880 0x7ff0081f1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.953+0000 7ff00d74c640 1 --2- 172.21.15.27:0/1527573465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feff4006150 0x7ff0080c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:45.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.953+0000 7ff00d74c640 1 -- 172.21.15.27:0/1527573465 >> 172.21.15.27:0/1527573465 conn(0x7ff008069c70 msgr2=0x7ff0080c96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:45.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.953+0000 7ff00d74c640 1 -- 172.21.15.27:0/1527573465 shutdown_connections 2023-12-19T18:19:45.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:45.953+0000 7ff00d74c640 1 -- 172.21.15.27:0/1527573465 wait complete. 2023-12-19T18:19:45.973 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:19:46.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.211+0000 7f5be335c640 1 Processor -- start 2023-12-19T18:19:46.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.211+0000 7f5be335c640 1 -- start start 2023-12-19T18:19:46.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.213+0000 7f5be335c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5bdc0ca540 0x7f5bdc0b1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.213+0000 7f5be335c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc0cb850 con 0x7f5bdc0cbce0 2023-12-19T18:19:46.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.213+0000 7f5be335c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bdc0ae330 con 0x7f5bdc15af70 2023-12-19T18:19:46.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.213+0000 7f5be335c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bdc15b350 con 0x7f5bdc0ca540 2023-12-19T18:19:46.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.213+0000 7f5be08f7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5bdc0cbce0 0x7f5bdc0adc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49214/0 (socket says 172.21.15.27:49214) 2023-12-19T18:19:46.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.213+0000 7f5be08f7640 1 -- 172.21.15.27:0/2055970073 learned_addr learned my addr 172.21.15.27:0/2055970073 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:46.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5be18f9640 1 --2- 172.21.15.27:0/2055970073 >> v2:172.21.15.192:3300/0 conn(0x7f5bdc0ca540 0x7f5bdc0b1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4259442944 0 0) 0x7f5bdc0cb850 con 0x7f5bdc0cbce0 2023-12-19T18:19:46.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bc8003520 con 0x7f5bdc0cbce0 2023-12-19T18:19:46.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3038384647 0 0) 0x7f5bc8003520 con 0x7f5bdc0cbce0 2023-12-19T18:19:46.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 >> v2:172.21.15.192:3300/0 conn(0x7f5bdc0ca540 msgr2=0x7f5bdc0b1990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 --2- 172.21.15.27:0/2055970073 >> v2:172.21.15.192:3300/0 conn(0x7f5bdc0ca540 0x7f5bdc0b1990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 >> v1:172.21.15.192:6789/0 conn(0x7f5bdc15af70 legacy=0x7f5bdc0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc0ca990 con 0x7f5bdc0cbce0 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f5bcc001670 con 0x7f5bdc0cbce0 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 >> v1:172.21.15.27:6789/0 conn(0x7f5bdc0cbce0 legacy=0x7f5bdc0adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 --2- 172.21.15.27:0/2055970073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bc8003970 0x7f5bc8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 --2- 172.21.15.27:0/2055970073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc80047c0 0x7f5bc8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 --2- 172.21.15.27:0/2055970073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bc8005180 0x7f5bc800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc15b350 con 0x7f5bc80047c0 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc800c020 con 0x7f5bc8003970 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.214+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc800c1e0 con 0x7f5bc8005180 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.215+0000 7f5be08f7640 1 --2- 172.21.15.27:0/2055970073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc80047c0 0x7f5bc8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.215+0000 7f5be10f8640 1 --2- 172.21.15.27:0/2055970073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bc8003970 0x7f5bc8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.215+0000 7f5be08f7640 1 -- 172.21.15.27:0/2055970073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bc8005180 msgr2=0x7f5bc800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:46.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.215+0000 7f5be08f7640 1 --2- 172.21.15.27:0/2055970073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bc8005180 0x7f5bc800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.215+0000 7f5be08f7640 1 -- 172.21.15.27:0/2055970073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bc8003970 msgr2=0x7f5bc8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:46.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.215+0000 7f5be08f7640 1 --2- 172.21.15.27:0/2055970073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bc8003970 0x7f5bc8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.215+0000 7f5be08f7640 1 -- 172.21.15.27:0/2055970073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc8003520 con 0x7f5bc80047c0 2023-12-19T18:19:46.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.215+0000 7f5be18f9640 1 -- 172.21.15.27:0/2055970073 reap_dead start 2023-12-19T18:19:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.216+0000 7f5be08f7640 1 --2- 172.21.15.27:0/2055970073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc80047c0 0x7f5bc8004c40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc007fd0 tx=0x7f5bc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1a4f962a556589 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.217+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc005ad0 con 0x7f5bc80047c0 2023-12-19T18:19:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.217+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc005c50 con 0x7f5bc80047c0 2023-12-19T18:19:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.217+0000 7f5bd3fff640 1 -- 172.21.15.27:0/2055970073 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc02c850 con 0x7f5bc80047c0 2023-12-19T18:19:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.217+0000 7f5be335c640 1 -- 172.21.15.27:0/2055970073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc80047c0 msgr2=0x7f5bc8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.217+0000 7f5be335c640 1 --2- 172.21.15.27:0/2055970073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc80047c0 0x7f5bc8004c40 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc007fd0 tx=0x7f5bc80057f0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.217+0000 7f5be335c640 1 -- 172.21.15.27:0/2055970073 shutdown_connections 2023-12-19T18:19:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.217+0000 7f5be335c640 1 --2- 172.21.15.27:0/2055970073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bc80047c0 0x7f5bc8004c40 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.217+0000 7f5be335c640 1 -- 172.21.15.27:0/2055970073 >> 172.21.15.27:0/2055970073 conn(0x7f5bdc06a330 msgr2=0x7f5bdc0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:46.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.217+0000 7f5be335c640 1 -- 172.21.15.27:0/2055970073 shutdown_connections 2023-12-19T18:19:46.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.217+0000 7f5be335c640 1 -- 172.21.15.27:0/2055970073 wait complete. 2023-12-19T18:19:46.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.219+0000 7f5be335c640 1 Processor -- start 2023-12-19T18:19:46.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.219+0000 7f5be335c640 1 -- start start 2023-12-19T18:19:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.220+0000 7f5be335c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bc80047c0 0x7f5bdc1520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.220+0000 7f5be335c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bdc1f2260 0x7f5bdc0c6620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.220+0000 7f5be335c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bdc0c6b60 0x7f5bdc0b2ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.220+0000 7f5be335c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc0c7a70 con 0x7f5bdc0c6b60 2023-12-19T18:19:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.220+0000 7f5be335c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc15b630 con 0x7f5bdc1f2260 2023-12-19T18:19:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.220+0000 7f5be335c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc15b910 con 0x7f5bc80047c0 2023-12-19T18:19:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.221+0000 7f5be18f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bdc0c6b60 0x7f5bdc0b2ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.221+0000 7f5be18f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bdc0c6b60 0x7f5bdc0b2ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44680/0 (socket says 172.21.15.27:44680) 2023-12-19T18:19:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.221+0000 7f5be18f9640 1 -- 172.21.15.27:0/2455161644 learned_addr learned my addr 172.21.15.27:0/2455161644 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.221+0000 7f5be10f8640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bc80047c0 0x7f5bdc1520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.221+0000 7f5be08f7640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bdc1f2260 0x7f5bdc0c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.221+0000 7f5be18f9640 1 -- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bc80047c0 msgr2=0x7f5bdc1520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.221+0000 7f5be18f9640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bc80047c0 0x7f5bdc1520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.221+0000 7f5be18f9640 1 -- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bdc1f2260 msgr2=0x7f5bdc0c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.221+0000 7f5be18f9640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bdc1f2260 0x7f5bdc0c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.221+0000 7f5be18f9640 1 -- 172.21.15.27:0/2455161644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bdc15bbf0 con 0x7f5bdc0c6b60 2023-12-19T18:19:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.222+0000 7f5be18f9640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bdc0c6b60 0x7f5bdc0b2ab0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0b4830 tx=0x7f5bd400ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.222+0000 7f5bd1ffb640 1 -- 172.21.15.27:0/2455161644 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4010040 con 0x7f5bdc0c6b60 2023-12-19T18:19:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.222+0000 7f5bd1ffb640 1 -- 172.21.15.27:0/2455161644 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bd400bb70 con 0x7f5bdc0c6b60 2023-12-19T18:19:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.222+0000 7f5bd1ffb640 1 -- 172.21.15.27:0/2455161644 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4014400 con 0x7f5bdc0c6b60 2023-12-19T18:19:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.222+0000 7f5be335c640 1 -- 172.21.15.27:0/2455161644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc15bed0 con 0x7f5bdc0c6b60 2023-12-19T18:19:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.222+0000 7f5be335c640 1 -- 172.21.15.27:0/2455161644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc1525e0 con 0x7f5bdc0c6b60 2023-12-19T18:19:46.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.223+0000 7f5be335c640 1 -- 172.21.15.27:0/2455161644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bdc15c3f0 con 0x7f5bdc0c6b60 2023-12-19T18:19:46.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.224+0000 7f5bd1ffb640 1 -- 172.21.15.27:0/2455161644 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5bd4014560 con 0x7f5bdc0c6b60 2023-12-19T18:19:46.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.225+0000 7f5bd1ffb640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5bc006a460 0x7f5bc006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.225+0000 7f5bd1ffb640 1 -- 172.21.15.27:0/2455161644 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5bd4088cb0 con 0x7f5bdc0c6b60 2023-12-19T18:19:46.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.226+0000 7f5be10f8640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5bc006a460 0x7f5bc006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.227+0000 7f5be10f8640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5bc006a460 0x7f5bc006c920 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8000c00 tx=0x7f5bd8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:46.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.230+0000 7f5bd1ffb640 1 -- 172.21.15.27:0/2455161644 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5bdc15c3f0 con 0x7f5bdc0c6b60 2023-12-19T18:19:46.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.421+0000 7f5be335c640 1 -- 172.21.15.27:0/2455161644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5bdc15c3f0 con 0x7f5bdc0c6b60 2023-12-19T18:19:46.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.423+0000 7f5bd1ffb640 1 -- 172.21.15.27:0/2455161644 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+16193 (secure 0 0 0) 0x7f5bdc15c3f0 con 0x7f5bdc0c6b60 2023-12-19T18:19:46.424 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:19:46.424 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":84,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:19:41.696772+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:18:24.350728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-19T18:18:25.147723+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.14","mappings":[{"from":6,"to":4}]},{"pgid":"9.2c","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:19:46.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.426+0000 7f5be335c640 1 -- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5bc006a460 msgr2=0x7f5bc006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:46.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.426+0000 7f5be335c640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5bc006a460 0x7f5bc006c920 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8000c00 tx=0x7f5bd8009040 comp rx=0 tx=0).stop 2023-12-19T18:19:46.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.426+0000 7f5be335c640 1 -- 172.21.15.27:0/2455161644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bdc0c6b60 msgr2=0x7f5bdc0b2ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:46.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.426+0000 7f5be335c640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bdc0c6b60 0x7f5bdc0b2ab0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc0b4830 tx=0x7f5bd400ef30 comp rx=0 tx=0).stop 2023-12-19T18:19:46.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.426+0000 7f5be335c640 1 -- 172.21.15.27:0/2455161644 shutdown_connections 2023-12-19T18:19:46.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.426+0000 7f5be335c640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5bc006a460 0x7f5bc006c920 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.426+0000 7f5be335c640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bdc0c6b60 0x7f5bdc0b2ab0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.426+0000 7f5be335c640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bdc1f2260 0x7f5bdc0c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.426+0000 7f5be335c640 1 --2- 172.21.15.27:0/2455161644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bc80047c0 0x7f5bdc1520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.426+0000 7f5be335c640 1 -- 172.21.15.27:0/2455161644 >> 172.21.15.27:0/2455161644 conn(0x7f5bdc06a330 msgr2=0x7f5bdc0af600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:46.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.426+0000 7f5be335c640 1 -- 172.21.15.27:0/2455161644 shutdown_connections 2023-12-19T18:19:46.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.426+0000 7f5be335c640 1 -- 172.21.15.27:0/2455161644 wait complete. 2023-12-19T18:19:46.448 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:19:46.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.684+0000 7fa0fabe8640 1 Processor -- start 2023-12-19T18:19:46.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.685+0000 7fa0fabe8640 1 -- start start 2023-12-19T18:19:46.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.686+0000 7fa0fabe8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f40b89b0 0x7fa0f415c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.686+0000 7fa0fabe8640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa0f40b4db0 0x7fa0f415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.686+0000 7fa0fabe8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0f40b5ca0 con 0x7fa0f40b49d0 2023-12-19T18:19:46.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.686+0000 7fa0fabe8640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0f415d300 con 0x7fa0f40b4db0 2023-12-19T18:19:46.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.686+0000 7fa0fabe8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f4161050 con 0x7fa0f40b89b0 2023-12-19T18:19:46.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.686+0000 7fa0f9be6640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa0f40b49d0 0x7fa0f4160940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49224/0 (socket says 172.21.15.27:49224) 2023-12-19T18:19:46.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.686+0000 7fa0f9be6640 1 -- 172.21.15.27:0/447858751 learned_addr learned my addr 172.21.15.27:0/447858751 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:46.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.687+0000 7fa0f93e5640 1 --2- 172.21.15.27:0/447858751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f40b89b0 0x7fa0f415c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.687+0000 7fa0fa3e7640 1 --2- 172.21.15.27:0/447858751 >> v2:172.21.15.192:3300/0 conn(0x7fa0f40b4db0 0x7fa0f415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.687+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3822266822 0 0) 0x7fa0f40b5ca0 con 0x7fa0f40b49d0 2023-12-19T18:19:46.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.687+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0e0003520 con 0x7fa0f40b49d0 2023-12-19T18:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.687+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3399644030 0 0) 0x7fa0e0003520 con 0x7fa0f40b49d0 2023-12-19T18:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.687+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f40b89b0 msgr2=0x7fa0f415c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.687+0000 7fa0f8be4640 1 --2- 172.21.15.27:0/447858751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f40b89b0 0x7fa0f415c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.687+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 >> v2:172.21.15.192:3300/0 conn(0x7fa0f40b4db0 msgr2=0x7fa0f415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.687+0000 7fa0f8be4640 1 --2- 172.21.15.27:0/447858751 >> v2:172.21.15.192:3300/0 conn(0x7fa0f40b4db0 0x7fa0f415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.687+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f4161800 con 0x7fa0f40b49d0 2023-12-19T18:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.687+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fa0f00017f0 con 0x7fa0f40b49d0 2023-12-19T18:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.688+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 >> v1:172.21.15.27:6789/0 conn(0x7fa0f40b49d0 legacy=0x7fa0f4160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.688+0000 7fa0f8be4640 1 --2- 172.21.15.27:0/447858751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e0003970 0x7fa0e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.688+0000 7fa0f8be4640 1 --2- 172.21.15.27:0/447858751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0e00047c0 0x7fa0e0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.688+0000 7fa0f8be4640 1 --2- 172.21.15.27:0/447858751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f40b89b0 0x7fa0e0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.688+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f415d300 con 0x7fa0e0003970 2023-12-19T18:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.688+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e000b980 con 0x7fa0e00047c0 2023-12-19T18:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.688+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e000baf0 con 0x7fa0f40b89b0 2023-12-19T18:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.688+0000 7fa0f9be6640 1 --2- 172.21.15.27:0/447858751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e0003970 0x7fa0e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.688+0000 7fa0fa3e7640 1 --2- 172.21.15.27:0/447858751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f40b89b0 0x7fa0e0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.688+0000 7fa0f93e5640 1 --2- 172.21.15.27:0/447858751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0e00047c0 0x7fa0e0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.688+0000 7fa0f9be6640 1 -- 172.21.15.27:0/447858751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f40b89b0 msgr2=0x7fa0e0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.689+0000 7fa0f9be6640 1 --2- 172.21.15.27:0/447858751 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f40b89b0 0x7fa0e0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.689+0000 7fa0f9be6640 1 -- 172.21.15.27:0/447858751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0e00047c0 msgr2=0x7fa0e0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.689+0000 7fa0f9be6640 1 --2- 172.21.15.27:0/447858751 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0e00047c0 0x7fa0e0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.689+0000 7fa0f9be6640 1 -- 172.21.15.27:0/447858751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e0003520 con 0x7fa0e0003970 2023-12-19T18:19:46.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.690+0000 7fa0f9be6640 1 --2- 172.21.15.27:0/447858751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e0003970 0x7fa0e0003d90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000c210 tx=0x7fa0e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40e9ee4a789215d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:46.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.690+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f002cad0 con 0x7fa0e0003970 2023-12-19T18:19:46.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.690+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f002cc30 con 0x7fa0e0003970 2023-12-19T18:19:46.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.690+0000 7fa0f8be4640 1 -- 172.21.15.27:0/447858751 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f0007070 con 0x7fa0e0003970 2023-12-19T18:19:46.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.691+0000 7fa0fabe8640 1 -- 172.21.15.27:0/447858751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e0003970 msgr2=0x7fa0e0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:46.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.691+0000 7fa0fabe8640 1 --2- 172.21.15.27:0/447858751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e0003970 0x7fa0e0003d90 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fa0f000c210 tx=0x7fa0e00057f0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.691+0000 7fa0fa3e7640 1 -- 172.21.15.27:0/447858751 reap_dead start 2023-12-19T18:19:46.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.691+0000 7fa0fabe8640 1 -- 172.21.15.27:0/447858751 shutdown_connections 2023-12-19T18:19:46.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.691+0000 7fa0fabe8640 1 -- 172.21.15.27:0/447858751 >> 172.21.15.27:0/447858751 conn(0x7fa0f40cf760 msgr2=0x7fa0f40cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:46.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.691+0000 7fa0fabe8640 1 -- 172.21.15.27:0/447858751 shutdown_connections 2023-12-19T18:19:46.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.691+0000 7fa0fabe8640 1 -- 172.21.15.27:0/447858751 wait complete. 2023-12-19T18:19:46.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.693+0000 7fa0fabe8640 1 Processor -- start 2023-12-19T18:19:46.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.693+0000 7fa0fabe8640 1 -- start start 2023-12-19T18:19:46.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.694+0000 7fa0fabe8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e0003970 0x7fa0f414dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.694+0000 7fa0fabe8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f414e240 0x7fa0f414f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.694+0000 7fa0fabe8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f414f970 0x7fa0f415f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.694+0000 7fa0fabe8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f414e6e0 con 0x7fa0e0003970 2023-12-19T18:19:46.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.694+0000 7fa0fabe8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f414e9c0 con 0x7fa0f414e240 2023-12-19T18:19:46.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.694+0000 7fa0fabe8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f414eca0 con 0x7fa0f414f970 2023-12-19T18:19:46.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.694+0000 7fa0f9be6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e0003970 0x7fa0f414dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.694+0000 7fa0f9be6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e0003970 0x7fa0f414dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44700/0 (socket says 172.21.15.27:44700) 2023-12-19T18:19:46.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.694+0000 7fa0f9be6640 1 -- 172.21.15.27:0/673425598 learned_addr learned my addr 172.21.15.27:0/673425598 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:46.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.695+0000 7fa0f9be6640 1 -- 172.21.15.27:0/673425598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f414f970 msgr2=0x7fa0f415f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:46.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.695+0000 7fa0f9be6640 1 --2- 172.21.15.27:0/673425598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f414f970 0x7fa0f415f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.695+0000 7fa0f9be6640 1 -- 172.21.15.27:0/673425598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f414e240 msgr2=0x7fa0f414f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:46.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.695+0000 7fa0f9be6640 1 --2- 172.21.15.27:0/673425598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f414e240 0x7fa0f414f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.695+0000 7fa0f9be6640 1 -- 172.21.15.27:0/673425598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f4151b20 con 0x7fa0e0003970 2023-12-19T18:19:46.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.695+0000 7fa0f9be6640 1 --2- 172.21.15.27:0/673425598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e0003970 0x7fa0f414dd00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa0f002c470 tx=0x7fa0f0001160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:46.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.695+0000 7fa0ea7fc640 1 -- 172.21.15.27:0/673425598 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f003c040 con 0x7fa0e0003970 2023-12-19T18:19:46.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.696+0000 7fa0ea7fc640 1 -- 172.21.15.27:0/673425598 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0f002de80 con 0x7fa0e0003970 2023-12-19T18:19:46.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.696+0000 7fa0fabe8640 1 -- 172.21.15.27:0/673425598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f4151e00 con 0x7fa0e0003970 2023-12-19T18:19:46.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.696+0000 7fa0ea7fc640 1 -- 172.21.15.27:0/673425598 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0f00072b0 con 0x7fa0e0003970 2023-12-19T18:19:46.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.696+0000 7fa0fabe8640 1 -- 172.21.15.27:0/673425598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f41f2780 con 0x7fa0e0003970 2023-12-19T18:19:46.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.697+0000 7fa0fabe8640 1 -- 172.21.15.27:0/673425598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f40d0d80 con 0x7fa0e0003970 2023-12-19T18:19:46.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.698+0000 7fa0ea7fc640 1 -- 172.21.15.27:0/673425598 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa0f0032070 con 0x7fa0e0003970 2023-12-19T18:19:46.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.699+0000 7fa0ea7fc640 1 --2- 172.21.15.27:0/673425598 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0d806a430 0x7fa0d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:46.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.699+0000 7fa0ea7fc640 1 -- 172.21.15.27:0/673425598 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fa0f00aa080 con 0x7fa0e0003970 2023-12-19T18:19:46.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.700+0000 7fa0f93e5640 1 --2- 172.21.15.27:0/673425598 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0d806a430 0x7fa0d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:46.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.701+0000 7fa0f93e5640 1 --2- 172.21.15.27:0/673425598 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0d806a430 0x7fa0d806c8f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4158220 tx=0x7fa0e4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:46.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.704+0000 7fa0ea7fc640 1 -- 172.21.15.27:0/673425598 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa0f00dcca0 con 0x7fa0e0003970 2023-12-19T18:19:46.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.941+0000 7fa0fabe8640 1 -- 172.21.15.27:0/673425598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa0f404e840 con 0x7fa0e0003970 2023-12-19T18:19:46.943 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:19:46.943+0000 7f8f3865c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:19:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.956+0000 7fa0ea7fc640 1 -- 172.21.15.27:0/673425598 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v31) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa0f40d0d80 con 0x7fa0e0003970 2023-12-19T18:19:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:19:46.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.960+0000 7fa0fabe8640 1 -- 172.21.15.27:0/673425598 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0d806a430 msgr2=0x7fa0d806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:46.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.960+0000 7fa0fabe8640 1 --2- 172.21.15.27:0/673425598 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0d806a430 0x7fa0d806c8f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fa0f4158220 tx=0x7fa0e4008040 comp rx=0 tx=0).stop 2023-12-19T18:19:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.960+0000 7fa0fabe8640 1 -- 172.21.15.27:0/673425598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e0003970 msgr2=0x7fa0f414dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.960+0000 7fa0fabe8640 1 --2- 172.21.15.27:0/673425598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e0003970 0x7fa0f414dd00 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fa0f002c470 tx=0x7fa0f0001160 comp rx=0 tx=0).stop 2023-12-19T18:19:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.960+0000 7fa0fabe8640 1 -- 172.21.15.27:0/673425598 shutdown_connections 2023-12-19T18:19:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.960+0000 7fa0fabe8640 1 --2- 172.21.15.27:0/673425598 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0d806a430 0x7fa0d806c8f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.960+0000 7fa0fabe8640 1 --2- 172.21.15.27:0/673425598 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0f414f970 0x7fa0f415f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.960+0000 7fa0fabe8640 1 --2- 172.21.15.27:0/673425598 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0f414e240 0x7fa0f414f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.960+0000 7fa0fabe8640 1 --2- 172.21.15.27:0/673425598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0e0003970 0x7fa0f414dd00 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.960+0000 7fa0fabe8640 1 -- 172.21.15.27:0/673425598 >> 172.21.15.27:0/673425598 conn(0x7fa0f40cf760 msgr2=0x7fa0f40b62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.961+0000 7fa0fabe8640 1 -- 172.21.15.27:0/673425598 shutdown_connections 2023-12-19T18:19:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:46.961+0000 7fa0fabe8640 1 -- 172.21.15.27:0/673425598 wait complete. 2023-12-19T18:19:46.971 INFO:tasks.ceph.mds.c.smithi027.stderr: -8024> 2023-12-19T18:18:26.591+0000 7fe717f61640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:19:46.980 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:19:47.027 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:19:47.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.183+0000 7fd1803f0640 1 Processor -- start 2023-12-19T18:19:47.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.183+0000 7fd1803f0640 1 -- start start 2023-12-19T18:19:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.184+0000 7fd1803f0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd17815ac90 0x7fd17815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.184+0000 7fd1803f0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd178071410 0x7fd17806ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.184+0000 7fd1803f0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd17806cf60 con 0x7fd17815b070 2023-12-19T18:19:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.184+0000 7fd1803f0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17815eb40 con 0x7fd17815ac90 2023-12-19T18:19:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.184+0000 7fd1803f0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd17806f770 con 0x7fd178071410 2023-12-19T18:19:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.185+0000 7fd17e18c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd17815ac90 0x7fd17815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.185+0000 7fd17e18c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd17815ac90 0x7fd17815f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44734/0 (socket says 172.21.15.27:44734) 2023-12-19T18:19:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.185+0000 7fd17e18c640 1 -- 172.21.15.27:0/3953465731 learned_addr learned my addr 172.21.15.27:0/3953465731 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:47.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.185+0000 7fd17e18c640 1 -- 172.21.15.27:0/3953465731 >> v2:172.21.15.192:3300/0 conn(0x7fd178071410 msgr2=0x7fd17806ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:47.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.185+0000 7fd17e98d640 1 --2- 172.21.15.27:0/3953465731 >> v2:172.21.15.192:3300/0 conn(0x7fd178071410 0x7fd17806ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:47.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.185+0000 7fd17e18c640 1 --2- 172.21.15.27:0/3953465731 >> v2:172.21.15.192:3300/0 conn(0x7fd178071410 0x7fd17806ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:47.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.185+0000 7fd17e18c640 1 -- 172.21.15.27:0/3953465731 >> v1:172.21.15.192:6789/0 conn(0x7fd17815b070 legacy=0x7fd178070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:47.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.185+0000 7fd17e18c640 1 -- 172.21.15.27:0/3953465731 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1780cb050 con 0x7fd17815ac90 2023-12-19T18:19:47.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.186+0000 7fd17e18c640 1 --2- 172.21.15.27:0/3953465731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd17815ac90 0x7fd17815f5e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd1780c2630 tx=0x7fd17402b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb87e59a9fca31e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:47.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.187+0000 7fd17d18a640 1 -- 172.21.15.27:0/3953465731 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174030860 con 0x7fd17815ac90 2023-12-19T18:19:47.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.187+0000 7fd17d18a640 1 -- 172.21.15.27:0/3953465731 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1740309c0 con 0x7fd17815ac90 2023-12-19T18:19:47.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.187+0000 7fd17d18a640 1 -- 172.21.15.27:0/3953465731 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174030d50 con 0x7fd17815ac90 2023-12-19T18:19:47.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.187+0000 7fd1803f0640 1 -- 172.21.15.27:0/3953465731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd17815ac90 msgr2=0x7fd17815f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:47.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.187+0000 7fd1803f0640 1 --2- 172.21.15.27:0/3953465731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd17815ac90 0x7fd17815f5e0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fd1780c2630 tx=0x7fd17402b9a0 comp rx=0 tx=0).stop 2023-12-19T18:19:47.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.187+0000 7fd1803f0640 1 -- 172.21.15.27:0/3953465731 shutdown_connections 2023-12-19T18:19:47.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.187+0000 7fd1803f0640 1 --2- 172.21.15.27:0/3953465731 >> v2:172.21.15.192:3300/0 conn(0x7fd178071410 0x7fd17806ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:47.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.187+0000 7fd1803f0640 1 --2- 172.21.15.27:0/3953465731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd17815ac90 0x7fd17815f5e0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:47.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.187+0000 7fd1803f0640 1 -- 172.21.15.27:0/3953465731 >> 172.21.15.27:0/3953465731 conn(0x7fd1780cb5b0 msgr2=0x7fd1780cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:47.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.188+0000 7fd1803f0640 1 -- 172.21.15.27:0/3953465731 shutdown_connections 2023-12-19T18:19:47.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.188+0000 7fd1803f0640 1 -- 172.21.15.27:0/3953465731 wait complete. 2023-12-19T18:19:47.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.189+0000 7fd1803f0640 1 Processor -- start 2023-12-19T18:19:47.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.189+0000 7fd1803f0640 1 -- start start 2023-12-19T18:19:47.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.190+0000 7fd1803f0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd178071410 0x7fd1780b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:47.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.190+0000 7fd1803f0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd17815ac90 0x7fd1781f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:47.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.190+0000 7fd1803f0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd17815b070 0x7fd178150200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:47.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.190+0000 7fd1803f0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1780b1f80 con 0x7fd178071410 2023-12-19T18:19:47.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.190+0000 7fd1803f0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1781583d0 con 0x7fd17815ac90 2023-12-19T18:19:47.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.190+0000 7fd1803f0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1781586b0 con 0x7fd17815b070 2023-12-19T18:19:47.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.191+0000 7fd17e18c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd178071410 0x7fd1780b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:47.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.191+0000 7fd17e18c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd178071410 0x7fd1780b1750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44748/0 (socket says 172.21.15.27:44748) 2023-12-19T18:19:47.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.191+0000 7fd17e18c640 1 -- 172.21.15.27:0/2989326658 learned_addr learned my addr 172.21.15.27:0/2989326658 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:47.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.191+0000 7fd17e18c640 1 -- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd17815b070 msgr2=0x7fd178150200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:47.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.191+0000 7fd17e98d640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd17815b070 0x7fd178150200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:47.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.191+0000 7fd17d98b640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd17815ac90 0x7fd1781f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:47.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.191+0000 7fd17e18c640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd17815b070 0x7fd178150200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:47.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.191+0000 7fd17e18c640 1 -- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd17815ac90 msgr2=0x7fd1781f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:47.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.191+0000 7fd17e18c640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd17815ac90 0x7fd1781f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:47.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.191+0000 7fd17e18c640 1 -- 172.21.15.27:0/2989326658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1740094d0 con 0x7fd178071410 2023-12-19T18:19:47.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.191+0000 7fd17d98b640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd17815ac90 0x7fd1781f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:47.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.192+0000 7fd17e18c640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd178071410 0x7fd1780b1750 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd174000c00 tx=0x7fd174004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:47.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.192+0000 7fd16effd640 1 -- 172.21.15.27:0/2989326658 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd174030830 con 0x7fd178071410 2023-12-19T18:19:47.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.192+0000 7fd16effd640 1 -- 172.21.15.27:0/2989326658 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd174043400 con 0x7fd178071410 2023-12-19T18:19:47.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.192+0000 7fd16effd640 1 -- 172.21.15.27:0/2989326658 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1740437b0 con 0x7fd178071410 2023-12-19T18:19:47.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.192+0000 7fd1803f0640 1 -- 172.21.15.27:0/2989326658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd178158990 con 0x7fd178071410 2023-12-19T18:19:47.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.192+0000 7fd1803f0640 1 -- 172.21.15.27:0/2989326658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1780b1c90 con 0x7fd178071410 2023-12-19T18:19:47.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.193+0000 7fd1803f0640 1 -- 172.21.15.27:0/2989326658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd178068e90 con 0x7fd178071410 2023-12-19T18:19:47.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.194+0000 7fd16effd640 1 -- 172.21.15.27:0/2989326658 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd174043910 con 0x7fd178071410 2023-12-19T18:19:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.195+0000 7fd16effd640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd15c06a430 0x7fd15c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:47.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.195+0000 7fd16effd640 1 -- 172.21.15.27:0/2989326658 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fd17403e070 con 0x7fd178071410 2023-12-19T18:19:47.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.196+0000 7fd17d98b640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd15c06a430 0x7fd15c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:47.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.197+0000 7fd17d98b640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd15c06a430 0x7fd15c06c8f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd1781513f0 tx=0x7fd168008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:47.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.199+0000 7fd16effd640 1 -- 172.21.15.27:0/2989326658 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd174077120 con 0x7fd178071410 2023-12-19T18:19:47.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.432+0000 7fd1803f0640 1 -- 172.21.15.27:0/2989326658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fd17806cf60 con 0x7fd178071410 2023-12-19T18:19:47.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.961+0000 7fd16effd640 1 -- 172.21.15.27:0/2989326658 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v32) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd17806cf60 con 0x7fd178071410 2023-12-19T18:19:47.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.965+0000 7fd1803f0640 1 -- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd15c06a430 msgr2=0x7fd15c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.965+0000 7fd1803f0640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd15c06a430 0x7fd15c06c8f0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd1781513f0 tx=0x7fd168008040 comp rx=0 tx=0).stop 2023-12-19T18:19:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.965+0000 7fd1803f0640 1 -- 172.21.15.27:0/2989326658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd178071410 msgr2=0x7fd1780b1750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.965+0000 7fd1803f0640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd178071410 0x7fd1780b1750 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fd174000c00 tx=0x7fd174004730 comp rx=0 tx=0).stop 2023-12-19T18:19:47.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.965+0000 7fd1803f0640 1 -- 172.21.15.27:0/2989326658 shutdown_connections 2023-12-19T18:19:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.965+0000 7fd1803f0640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd15c06a430 0x7fd15c06c8f0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.965+0000 7fd1803f0640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd17815b070 0x7fd178150200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.965+0000 7fd1803f0640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd17815ac90 0x7fd1781f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.965+0000 7fd1803f0640 1 --2- 172.21.15.27:0/2989326658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd178071410 0x7fd1780b1750 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.965+0000 7fd1803f0640 1 -- 172.21.15.27:0/2989326658 >> 172.21.15.27:0/2989326658 conn(0x7fd1780cb5b0 msgr2=0x7fd1780c7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.965+0000 7fd1803f0640 1 -- 172.21.15.27:0/2989326658 shutdown_connections 2023-12-19T18:19:47.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:47.965+0000 7fd1803f0640 1 -- 172.21.15.27:0/2989326658 wait complete. 2023-12-19T18:19:47.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:19:48.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.272+0000 7feedd537640 1 Processor -- start 2023-12-19T18:19:48.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.272+0000 7feedd537640 1 -- start start 2023-12-19T18:19:48.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.274+0000 7feedd537640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7feed815cfc0 0x7feed8160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:48.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.274+0000 7feedd537640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feed80d27b0 0x7feed80cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:48.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.274+0000 7feedd537640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feed8161a50 con 0x7feed80cdb30 2023-12-19T18:19:48.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.274+0000 7feedd537640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed80ce500 con 0x7feed80d27b0 2023-12-19T18:19:48.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.274+0000 7feedd537640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed8161490 con 0x7feed815cfc0 2023-12-19T18:19:48.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.274+0000 7feedcd36640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feed80d27b0 0x7feed80cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:48.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.274+0000 7feedcd36640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feed80d27b0 0x7feed80cdfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44750/0 (socket says 172.21.15.27:44750) 2023-12-19T18:19:48.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.274+0000 7feedcd36640 1 -- 172.21.15.27:0/1881064892 learned_addr learned my addr 172.21.15.27:0/1881064892 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:48.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.274+0000 7feed7fff640 1 --2- 172.21.15.27:0/1881064892 >> v2:172.21.15.192:3300/0 conn(0x7feed815cfc0 0x7feed8160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:48.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.275+0000 7feedcd36640 1 -- 172.21.15.27:0/1881064892 >> v2:172.21.15.192:3300/0 conn(0x7feed815cfc0 msgr2=0x7feed8160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:48.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.275+0000 7feedcd36640 1 --2- 172.21.15.27:0/1881064892 >> v2:172.21.15.192:3300/0 conn(0x7feed815cfc0 0x7feed8160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:48.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.275+0000 7feedcd36640 1 -- 172.21.15.27:0/1881064892 >> v1:172.21.15.192:6789/0 conn(0x7feed80cdb30 legacy=0x7feed80d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:48.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.275+0000 7feedcd36640 1 -- 172.21.15.27:0/1881064892 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed8161770 con 0x7feed80d27b0 2023-12-19T18:19:48.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.276+0000 7feedcd36640 1 --2- 172.21.15.27:0/1881064892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d27b0 0x7feed80cdfc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7feed80b4fc0 tx=0x7feed002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64dee22b365d95f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:48.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.276+0000 7feed6ffd640 1 -- 172.21.15.27:0/1881064892 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed00317e0 con 0x7feed80d27b0 2023-12-19T18:19:48.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.276+0000 7feed6ffd640 1 -- 172.21.15.27:0/1881064892 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed0031940 con 0x7feed80d27b0 2023-12-19T18:19:48.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.276+0000 7feed6ffd640 1 -- 172.21.15.27:0/1881064892 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0031cd0 con 0x7feed80d27b0 2023-12-19T18:19:48.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.277+0000 7feedd537640 1 -- 172.21.15.27:0/1881064892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d27b0 msgr2=0x7feed80cdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:48.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.277+0000 7feedd537640 1 --2- 172.21.15.27:0/1881064892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d27b0 0x7feed80cdfc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7feed80b4fc0 tx=0x7feed002b880 comp rx=0 tx=0).stop 2023-12-19T18:19:48.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.277+0000 7feedd537640 1 -- 172.21.15.27:0/1881064892 shutdown_connections 2023-12-19T18:19:48.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.277+0000 7feedd537640 1 --2- 172.21.15.27:0/1881064892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d27b0 0x7feed80cdfc0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:48.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.277+0000 7feedd537640 1 --2- 172.21.15.27:0/1881064892 >> v2:172.21.15.192:3300/0 conn(0x7feed815cfc0 0x7feed8160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:48.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.277+0000 7feedd537640 1 -- 172.21.15.27:0/1881064892 >> 172.21.15.27:0/1881064892 conn(0x7feed8072e20 msgr2=0x7feed8073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:48.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.277+0000 7feedd537640 1 -- 172.21.15.27:0/1881064892 shutdown_connections 2023-12-19T18:19:48.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.277+0000 7feedd537640 1 -- 172.21.15.27:0/1881064892 wait complete. 2023-12-19T18:19:48.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.279+0000 7feedd537640 1 Processor -- start 2023-12-19T18:19:48.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.279+0000 7feedd537640 1 -- start start 2023-12-19T18:19:48.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.280+0000 7feedd537640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80cdb30 0x7feed81564c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:48.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.280+0000 7feedd537640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed80d27b0 0x7feed80d2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:48.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.280+0000 7feedd537640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed815cfc0 0x7feed80d3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:48.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.280+0000 7feedd537640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed80c93c0 con 0x7feed815cfc0 2023-12-19T18:19:48.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.280+0000 7feedd537640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed81f25c0 con 0x7feed80d27b0 2023-12-19T18:19:48.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.280+0000 7feedd537640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed81f28a0 con 0x7feed80cdb30 2023-12-19T18:19:48.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.280+0000 7feedcd36640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed815cfc0 0x7feed80d3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:48.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.280+0000 7feedcd36640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed815cfc0 0x7feed80d3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44764/0 (socket says 172.21.15.27:44764) 2023-12-19T18:19:48.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.280+0000 7feedcd36640 1 -- 172.21.15.27:0/3173480122 learned_addr learned my addr 172.21.15.27:0/3173480122 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:48.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.281+0000 7feed7fff640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80cdb30 0x7feed81564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:48.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.281+0000 7feedcd36640 1 -- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80cdb30 msgr2=0x7feed81564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.281+0000 7feedcd36640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80cdb30 0x7feed81564c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.281+0000 7feedcd36640 1 -- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed80d27b0 msgr2=0x7feed80d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.281+0000 7feed77fe640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed80d27b0 0x7feed80d2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.281+0000 7feedcd36640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed80d27b0 0x7feed80d2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.281+0000 7feedcd36640 1 -- 172.21.15.27:0/3173480122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed00094d0 con 0x7feed815cfc0 2023-12-19T18:19:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.281+0000 7feed77fe640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed80d27b0 0x7feed80d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.281+0000 7feedcd36640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed815cfc0 0x7feed80d3230 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7feed00090b0 tx=0x7feed00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.282+0000 7feed4ff9640 1 -- 172.21.15.27:0/3173480122 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0032880 con 0x7feed815cfc0 2023-12-19T18:19:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.282+0000 7feed4ff9640 1 -- 172.21.15.27:0/3173480122 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feed0032a00 con 0x7feed815cfc0 2023-12-19T18:19:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.282+0000 7feedd537640 1 -- 172.21.15.27:0/3173480122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed81f2b80 con 0x7feed815cfc0 2023-12-19T18:19:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.282+0000 7feed4ff9640 1 -- 172.21.15.27:0/3173480122 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feed0032db0 con 0x7feed815cfc0 2023-12-19T18:19:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.282+0000 7feedd537640 1 -- 172.21.15.27:0/3173480122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed81f3140 con 0x7feed815cfc0 2023-12-19T18:19:48.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.283+0000 7feedd537640 1 -- 172.21.15.27:0/3173480122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed81f3330 con 0x7feed815cfc0 2023-12-19T18:19:48.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.284+0000 7feed4ff9640 1 -- 172.21.15.27:0/3173480122 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7feed004e450 con 0x7feed815cfc0 2023-12-19T18:19:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.285+0000 7feed4ff9640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feeb806a430 0x7feeb806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.285+0000 7feed4ff9640 1 -- 172.21.15.27:0/3173480122 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 5853+0+0 (secure 0 0 0) 0x7feed00aab20 con 0x7feed815cfc0 2023-12-19T18:19:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.286+0000 7feed7fff640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feeb806a430 0x7feeb806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:48.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.287+0000 7feed7fff640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feeb806a430 0x7feeb806c8f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feec8006fd0 tx=0x7feec8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.289+0000 7feed4ff9640 1 -- 172.21.15.27:0/3173480122 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7feed0076090 con 0x7feed815cfc0 2023-12-19T18:19:48.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.484+0000 7feedd537640 1 -- 172.21.15.27:0/3173480122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7feed804e840 con 0x7feed815cfc0 2023-12-19T18:19:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.965+0000 7feed4ff9640 1 -- 172.21.15.27:0/3173480122 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7feed00796b0 con 0x7feed815cfc0 2023-12-19T18:19:48.966 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:19:48.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.970+0000 7feedd537640 1 -- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feeb806a430 msgr2=0x7feeb806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:48.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.970+0000 7feedd537640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feeb806a430 0x7feeb806c8f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feec8006fd0 tx=0x7feec8008040 comp rx=0 tx=0).stop 2023-12-19T18:19:48.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.970+0000 7feedd537640 1 -- 172.21.15.27:0/3173480122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed815cfc0 msgr2=0x7feed80d3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:48.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.970+0000 7feedd537640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed815cfc0 0x7feed80d3230 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7feed00090b0 tx=0x7feed00046c0 comp rx=0 tx=0).stop 2023-12-19T18:19:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.971+0000 7feedd537640 1 -- 172.21.15.27:0/3173480122 shutdown_connections 2023-12-19T18:19:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.971+0000 7feedd537640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feeb806a430 0x7feeb806c8f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.971+0000 7feedd537640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed815cfc0 0x7feed80d3230 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.971+0000 7feedd537640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed80d27b0 0x7feed80d2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.971+0000 7feedd537640 1 --2- 172.21.15.27:0/3173480122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80cdb30 0x7feed81564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.971+0000 7feedd537640 1 -- 172.21.15.27:0/3173480122 >> 172.21.15.27:0/3173480122 conn(0x7feed8072e20 msgr2=0x7feed8160bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.971+0000 7feedd537640 1 -- 172.21.15.27:0/3173480122 shutdown_connections 2023-12-19T18:19:48.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:48.971+0000 7feedd537640 1 -- 172.21.15.27:0/3173480122 wait complete. 2023-12-19T18:19:48.996 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:19:49.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.239+0000 7f4177fff640 1 Processor -- start 2023-12-19T18:19:49.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.239+0000 7f4177fff640 1 -- start start 2023-12-19T18:19:49.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.241+0000 7f4177fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f41780ade20 0x7f417815b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:49.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.241+0000 7f4177fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f417815c030 con 0x7f4178158790 2023-12-19T18:19:49.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.241+0000 7f4177fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41780c9030 con 0x7f417815b810 2023-12-19T18:19:49.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.241+0000 7f4177fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41780c99d0 con 0x7f41780ade20 2023-12-19T18:19:49.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.242+0000 7f4176ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4178158790 0x7f41780ad710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49244/0 (socket says 172.21.15.27:49244) 2023-12-19T18:19:49.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.242+0000 7f4176ffd640 1 -- 172.21.15.27:0/2441821829 learned_addr learned my addr 172.21.15.27:0/2441821829 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:49.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.242+0000 7f416e7fc640 1 --2- 172.21.15.27:0/2441821829 >> v2:172.21.15.27:3300/0 conn(0x7f41780ade20 0x7f417815b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:49.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.242+0000 7f41767fc640 1 -- 172.21.15.27:0/2441821829 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 870641770 0 0) 0x7f417815c030 con 0x7f4178158790 2023-12-19T18:19:49.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.242+0000 7f41767fc640 1 -- 172.21.15.27:0/2441821829 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4164003520 con 0x7f4178158790 2023-12-19T18:19:49.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.242+0000 7f416e7fc640 1 -- 172.21.15.27:0/2441821829 >> v1:172.21.15.192:6789/0 conn(0x7f417815b810 legacy=0x7f41780c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:49.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.242+0000 7f416e7fc640 1 -- 172.21.15.27:0/2441821829 >> v1:172.21.15.27:6789/0 conn(0x7f4178158790 legacy=0x7f41780ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:49.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.242+0000 7f416e7fc640 1 -- 172.21.15.27:0/2441821829 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41780c9310 con 0x7f41780ade20 2023-12-19T18:19:49.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.244+0000 7f416e7fc640 1 --2- 172.21.15.27:0/2441821829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41780ade20 0x7f417815b2d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f41600097a0 tx=0x7f416002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d190a42097cf7a58 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:49.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.244+0000 7f41767fc640 1 -- 172.21.15.27:0/2441821829 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160031830 con 0x7f41780ade20 2023-12-19T18:19:49.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.244+0000 7f41767fc640 1 -- 172.21.15.27:0/2441821829 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4160031990 con 0x7f41780ade20 2023-12-19T18:19:49.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.244+0000 7f41767fc640 1 -- 172.21.15.27:0/2441821829 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160031d20 con 0x7f41780ade20 2023-12-19T18:19:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.245+0000 7f4177fff640 1 -- 172.21.15.27:0/2441821829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41780ade20 msgr2=0x7f417815b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.245+0000 7f4177fff640 1 --2- 172.21.15.27:0/2441821829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41780ade20 0x7f417815b2d0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f41600097a0 tx=0x7f416002b8d0 comp rx=0 tx=0).stop 2023-12-19T18:19:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.245+0000 7f4177fff640 1 -- 172.21.15.27:0/2441821829 shutdown_connections 2023-12-19T18:19:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.245+0000 7f4177fff640 1 --2- 172.21.15.27:0/2441821829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f41780ade20 0x7f417815b2d0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.245+0000 7f4177fff640 1 -- 172.21.15.27:0/2441821829 >> 172.21.15.27:0/2441821829 conn(0x7f41780b05b0 msgr2=0x7f41780b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.245+0000 7f4177fff640 1 -- 172.21.15.27:0/2441821829 shutdown_connections 2023-12-19T18:19:49.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.245+0000 7f4177fff640 1 -- 172.21.15.27:0/2441821829 wait complete. 2023-12-19T18:19:49.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.247+0000 7f4177fff640 1 Processor -- start 2023-12-19T18:19:49.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.247+0000 7f4177fff640 1 -- start start 2023-12-19T18:19:49.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.248+0000 7f4177fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41780ade20 0x7f417814e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:49.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.248+0000 7f4177fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178158790 0x7f41780c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:49.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.248+0000 7f4177fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f417815b810 0x7f417815a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:49.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.248+0000 7f4177fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4178159c60 con 0x7f4178158790 2023-12-19T18:19:49.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.248+0000 7f4177fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f417814ea80 con 0x7f41780ade20 2023-12-19T18:19:49.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.248+0000 7f4177fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4178159050 con 0x7f417815b810 2023-12-19T18:19:49.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.248+0000 7f416e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178158790 0x7f41780c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:49.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.248+0000 7f416e7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178158790 0x7f41780c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44778/0 (socket says 172.21.15.27:44778) 2023-12-19T18:19:49.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.248+0000 7f416e7fc640 1 -- 172.21.15.27:0/2814875185 learned_addr learned my addr 172.21.15.27:0/2814875185 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:49.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.249+0000 7f4176ffd640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41780ade20 0x7f417814e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:49.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.249+0000 7f416e7fc640 1 -- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f417815b810 msgr2=0x7f417815a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:49.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.249+0000 7f41777fe640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f417815b810 0x7f417815a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:49.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.249+0000 7f416e7fc640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f417815b810 0x7f417815a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:49.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.249+0000 7f416e7fc640 1 -- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41780ade20 msgr2=0x7f417814e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:49.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.249+0000 7f416e7fc640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41780ade20 0x7f417814e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:49.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.249+0000 7f416e7fc640 1 -- 172.21.15.27:0/2814875185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4160009400 con 0x7f4178158790 2023-12-19T18:19:49.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.249+0000 7f4176ffd640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41780ade20 0x7f417814e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:49.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.249+0000 7f41777fe640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f417815b810 0x7f417815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:49.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.249+0000 7f416e7fc640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178158790 0x7f41780c5a10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f41600098c0 tx=0x7f41600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:49.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.249+0000 7f4174ff9640 1 -- 172.21.15.27:0/2814875185 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416003a6f0 con 0x7f4178158790 2023-12-19T18:19:49.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.249+0000 7f4174ff9640 1 -- 172.21.15.27:0/2814875185 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416003a870 con 0x7f4178158790 2023-12-19T18:19:49.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.249+0000 7f4177fff640 1 -- 172.21.15.27:0/2814875185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4178159330 con 0x7f4178158790 2023-12-19T18:19:49.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.250+0000 7f4174ff9640 1 -- 172.21.15.27:0/2814875185 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416003ac20 con 0x7f4178158790 2023-12-19T18:19:49.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.250+0000 7f4177fff640 1 -- 172.21.15.27:0/2814875185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41781598f0 con 0x7f4178158790 2023-12-19T18:19:49.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.251+0000 7f4177fff640 1 -- 172.21.15.27:0/2814875185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4178159e10 con 0x7f4178158790 2023-12-19T18:19:49.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.252+0000 7f4174ff9640 1 -- 172.21.15.27:0/2814875185 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f416004c3b0 con 0x7f4178158790 2023-12-19T18:19:49.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.253+0000 7f4174ff9640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f415806a430 0x7f415806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:49.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.253+0000 7f4174ff9640 1 -- 172.21.15.27:0/2814875185 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 5415+0+0 (secure 0 0 0) 0x7f41600aa4c0 con 0x7f4178158790 2023-12-19T18:19:49.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.253+0000 7f4176ffd640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f415806a430 0x7f415806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:49.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.254+0000 7f4176ffd640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f415806a430 0x7f415806c8f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4168001560 tx=0x7f4168009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:49.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.257+0000 7f4174ff9640 1 -- 172.21.15.27:0/2814875185 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4178159e10 con 0x7f4178158790 2023-12-19T18:19:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.452+0000 7f4177fff640 1 -- 172.21.15.27:0/2814875185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f417804e840 con 0x7f4178158790 2023-12-19T18:19:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.970+0000 7f4174ff9640 1 -- 172.21.15.27:0/2814875185 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7f41600795c0 con 0x7f4178158790 2023-12-19T18:19:49.971 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-12-19T18:19:49.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.975+0000 7f4177fff640 1 -- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f415806a430 msgr2=0x7f415806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:49.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.975+0000 7f4177fff640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f415806a430 0x7f415806c8f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f4168001560 tx=0x7f4168009040 comp rx=0 tx=0).stop 2023-12-19T18:19:49.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.975+0000 7f4177fff640 1 -- 172.21.15.27:0/2814875185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178158790 msgr2=0x7f41780c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:49.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.975+0000 7f4177fff640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178158790 0x7f41780c5a10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f41600098c0 tx=0x7f41600046c0 comp rx=0 tx=0).stop 2023-12-19T18:19:49.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.975+0000 7f4177fff640 1 -- 172.21.15.27:0/2814875185 shutdown_connections 2023-12-19T18:19:49.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.975+0000 7f4177fff640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f415806a430 0x7f415806c8f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:49.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.975+0000 7f4177fff640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f417815b810 0x7f417815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:49.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.975+0000 7f4177fff640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4178158790 0x7f41780c5a10 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:49.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.975+0000 7f4177fff640 1 --2- 172.21.15.27:0/2814875185 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41780ade20 0x7f417814e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:49.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.975+0000 7f4177fff640 1 -- 172.21.15.27:0/2814875185 >> 172.21.15.27:0/2814875185 conn(0x7f41780b05b0 msgr2=0x7f41780ad5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:49.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.976+0000 7f4177fff640 1 -- 172.21.15.27:0/2814875185 shutdown_connections 2023-12-19T18:19:49.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:49.976+0000 7f4177fff640 1 -- 172.21.15.27:0/2814875185 wait complete. 2023-12-19T18:19:49.999 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:19:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.234+0000 7f7a7b3d6640 1 Processor -- start 2023-12-19T18:19:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.234+0000 7f7a7b3d6640 1 -- start start 2023-12-19T18:19:50.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.235+0000 7f7a7b3d6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a7415cfc0 0x7f7a74160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:50.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.235+0000 7f7a7b3d6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7a740d27b0 0x7f7a740cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.235+0000 7f7a7b3d6640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a740ce4d0 con 0x7f7a740cdb30 2023-12-19T18:19:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.236+0000 7f7a7b3d6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a74161a50 con 0x7f7a740d27b0 2023-12-19T18:19:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.236+0000 7f7a7b3d6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a74161490 con 0x7f7a7415cfc0 2023-12-19T18:19:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.236+0000 7f7a7abd5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7a740d27b0 0x7f7a740cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.236+0000 7f7a7abd5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7a740d27b0 0x7f7a740cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44790/0 (socket says 172.21.15.27:44790) 2023-12-19T18:19:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.236+0000 7f7a7abd5640 1 -- 172.21.15.27:0/768927402 learned_addr learned my addr 172.21.15.27:0/768927402 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.236+0000 7f7a7a3d4640 1 --2- 172.21.15.27:0/768927402 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a7415cfc0 0x7f7a74160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.236+0000 7f7a7abd5640 1 -- 172.21.15.27:0/768927402 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a7415cfc0 msgr2=0x7f7a74160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.236+0000 7f7a7abd5640 1 --2- 172.21.15.27:0/768927402 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a7415cfc0 0x7f7a74160b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.236+0000 7f7a7abd5640 1 -- 172.21.15.27:0/768927402 >> v1:172.21.15.192:6789/0 conn(0x7f7a740cdb30 legacy=0x7f7a740d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.237+0000 7f7a7abd5640 1 -- 172.21.15.27:0/768927402 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a74161770 con 0x7f7a740d27b0 2023-12-19T18:19:50.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.238+0000 7f7a7abd5640 1 --2- 172.21.15.27:0/768927402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a740d27b0 0x7f7a740cdf90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009870 tx=0x7f7a6c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2130c7940e3b1220 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:50.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.238+0000 7f7a793d2640 1 -- 172.21.15.27:0/768927402 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c0317c0 con 0x7f7a740d27b0 2023-12-19T18:19:50.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.238+0000 7f7a793d2640 1 -- 172.21.15.27:0/768927402 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6c031920 con 0x7f7a740d27b0 2023-12-19T18:19:50.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.238+0000 7f7a793d2640 1 -- 172.21.15.27:0/768927402 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a6c031cb0 con 0x7f7a740d27b0 2023-12-19T18:19:50.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.239+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/768927402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a740d27b0 msgr2=0x7f7a740cdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:50.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.239+0000 7f7a7b3d6640 1 --2- 172.21.15.27:0/768927402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a740d27b0 0x7f7a740cdf90 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c009870 tx=0x7f7a6c02b860 comp rx=0 tx=0).stop 2023-12-19T18:19:50.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.239+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/768927402 shutdown_connections 2023-12-19T18:19:50.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.239+0000 7f7a7b3d6640 1 --2- 172.21.15.27:0/768927402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a740d27b0 0x7f7a740cdf90 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.239+0000 7f7a7b3d6640 1 --2- 172.21.15.27:0/768927402 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a7415cfc0 0x7f7a74160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.239+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/768927402 >> 172.21.15.27:0/768927402 conn(0x7f7a74072e20 msgr2=0x7f7a74073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:50.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.239+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/768927402 shutdown_connections 2023-12-19T18:19:50.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.239+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/768927402 wait complete. 2023-12-19T18:19:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.241+0000 7f7a7b3d6640 1 Processor -- start 2023-12-19T18:19:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.241+0000 7f7a7b3d6640 1 -- start start 2023-12-19T18:19:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.242+0000 7f7a7b3d6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a740cdb30 0x7f7a74151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.242+0000 7f7a7b3d6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a740d27b0 0x7f7a741598e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.242+0000 7f7a7b3d6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a7415cfc0 0x7f7a74155c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.242+0000 7f7a7b3d6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a74151790 con 0x7f7a740d27b0 2023-12-19T18:19:50.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.242+0000 7f7a7b3d6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7415e580 con 0x7f7a740cdb30 2023-12-19T18:19:50.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.242+0000 7f7a7b3d6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a7415d970 con 0x7f7a7415cfc0 2023-12-19T18:19:50.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.242+0000 7f7a79bd3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a740d27b0 0x7f7a741598e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:50.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.242+0000 7f7a79bd3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a740d27b0 0x7f7a741598e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44800/0 (socket says 172.21.15.27:44800) 2023-12-19T18:19:50.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.242+0000 7f7a79bd3640 1 -- 172.21.15.27:0/1668050655 learned_addr learned my addr 172.21.15.27:0/1668050655 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:50.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.242+0000 7f7a7a3d4640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a740cdb30 0x7f7a74151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:50.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.243+0000 7f7a7abd5640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a7415cfc0 0x7f7a74155c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:50.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.243+0000 7f7a79bd3640 1 -- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a7415cfc0 msgr2=0x7f7a74155c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:50.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.243+0000 7f7a79bd3640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a7415cfc0 0x7f7a74155c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.243+0000 7f7a79bd3640 1 -- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a740cdb30 msgr2=0x7f7a74151250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:50.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.243+0000 7f7a79bd3640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a740cdb30 0x7f7a74151250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.243+0000 7f7a79bd3640 1 -- 172.21.15.27:0/1668050655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c0094d0 con 0x7f7a740d27b0 2023-12-19T18:19:50.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.243+0000 7f7a79bd3640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a740d27b0 0x7f7a741598e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000b3f0 tx=0x7f7a6000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:50.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.243+0000 7f7a72ffd640 1 -- 172.21.15.27:0/1668050655 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60011830 con 0x7f7a740d27b0 2023-12-19T18:19:50.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.243+0000 7f7a72ffd640 1 -- 172.21.15.27:0/1668050655 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a60011990 con 0x7f7a740d27b0 2023-12-19T18:19:50.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.243+0000 7f7a72ffd640 1 -- 172.21.15.27:0/1668050655 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60011d40 con 0x7f7a740d27b0 2023-12-19T18:19:50.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.243+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/1668050655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a7415dc50 con 0x7f7a740d27b0 2023-12-19T18:19:50.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.244+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/1668050655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a7415e210 con 0x7f7a740d27b0 2023-12-19T18:19:50.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.245+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/1668050655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a7415e710 con 0x7f7a740d27b0 2023-12-19T18:19:50.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.246+0000 7f7a72ffd640 1 -- 172.21.15.27:0/1668050655 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7a6001a400 con 0x7f7a740d27b0 2023-12-19T18:19:50.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.247+0000 7f7a72ffd640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a5806a460 0x7f7a5806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:50.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.247+0000 7f7a72ffd640 1 -- 172.21.15.27:0/1668050655 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7a600884c0 con 0x7f7a740d27b0 2023-12-19T18:19:50.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.248+0000 7f7a7a3d4640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a5806a460 0x7f7a5806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:50.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.249+0000 7f7a7a3d4640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a5806a460 0x7f7a5806c920 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7a680099c0 tx=0x7f7a68003990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:50.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.252+0000 7f7a72ffd640 1 -- 172.21.15.27:0/1668050655 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7a60053da0 con 0x7f7a740d27b0 2023-12-19T18:19:50.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.446+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/1668050655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7a7404e840 con 0x7f7a740d27b0 2023-12-19T18:19:50.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.454+0000 7f7a72ffd640 1 -- 172.21.15.27:0/1668050655 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7a600573c0 con 0x7f7a740d27b0 2023-12-19T18:19:50.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.456+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a5806a460 msgr2=0x7f7a5806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.456+0000 7f7a7b3d6640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a5806a460 0x7f7a5806c920 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f7a680099c0 tx=0x7f7a68003990 comp rx=0 tx=0).stop 2023-12-19T18:19:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.456+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/1668050655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a740d27b0 msgr2=0x7f7a741598e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.456+0000 7f7a7b3d6640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a740d27b0 0x7f7a741598e0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000b3f0 tx=0x7f7a6000b8b0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.456+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/1668050655 shutdown_connections 2023-12-19T18:19:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.456+0000 7f7a7b3d6640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a5806a460 0x7f7a5806c920 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.456+0000 7f7a7b3d6640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a7415cfc0 0x7f7a74155c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.456+0000 7f7a7b3d6640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a740d27b0 0x7f7a741598e0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.456+0000 7f7a7b3d6640 1 --2- 172.21.15.27:0/1668050655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a740cdb30 0x7f7a74151250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.456+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/1668050655 >> 172.21.15.27:0/1668050655 conn(0x7f7a74072e20 msgr2=0x7f7a741524c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.457+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/1668050655 shutdown_connections 2023-12-19T18:19:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.457+0000 7f7a7b3d6640 1 -- 172.21.15.27:0/1668050655 wait complete. 2023-12-19T18:19:50.477 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose' 2023-12-19T18:19:50.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.688+0000 7f60c9b98640 1 Processor -- start 2023-12-19T18:19:50.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.688+0000 7f60c9b98640 1 -- start start 2023-12-19T18:19:50.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.690+0000 7f60c9b98640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f60c40b4ef0 0x7f60c40b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:50.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.690+0000 7f60c9b98640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60c40d2350 0x7f60c40b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:50.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.690+0000 7f60c9b98640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60c40b5e90 con 0x7f60c41604d0 2023-12-19T18:19:50.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.690+0000 7f60c9b98640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60c40d2b30 con 0x7f60c40b4ef0 2023-12-19T18:19:50.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.690+0000 7f60c9b98640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c40d2850 con 0x7f60c40d2350 2023-12-19T18:19:50.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.690+0000 7f60c3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f60c40b4ef0 0x7f60c40b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:50.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.691+0000 7f60c3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f60c40b4ef0 0x7f60c40b6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44814/0 (socket says 172.21.15.27:44814) 2023-12-19T18:19:50.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.691+0000 7f60c3fff640 1 -- 172.21.15.27:0/3421563076 learned_addr learned my addr 172.21.15.27:0/3421563076 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:50.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.691+0000 7f60c9397640 1 --2- 172.21.15.27:0/3421563076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60c40d2350 0x7f60c40b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:50.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.691+0000 7f60c3fff640 1 -- 172.21.15.27:0/3421563076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60c40d2350 msgr2=0x7f60c40b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:50.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.691+0000 7f60c3fff640 1 --2- 172.21.15.27:0/3421563076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60c40d2350 0x7f60c40b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.691+0000 7f60c3fff640 1 -- 172.21.15.27:0/3421563076 >> v1:172.21.15.192:6789/0 conn(0x7f60c41604d0 legacy=0x7f60c40b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:50.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.691+0000 7f60c3fff640 1 -- 172.21.15.27:0/3421563076 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c40cded0 con 0x7f60c40b4ef0 2023-12-19T18:19:50.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.691+0000 7f60c9397640 1 --2- 172.21.15.27:0/3421563076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60c40d2350 0x7f60c40b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:50.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.692+0000 7f60c3fff640 1 --2- 172.21.15.27:0/3421563076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60c40b4ef0 0x7f60c40b6300 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009870 tx=0x7f60b002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a89fc10bfdf9aba0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:50.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.692+0000 7f60c37fe640 1 -- 172.21.15.27:0/3421563076 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0031640 con 0x7f60c40b4ef0 2023-12-19T18:19:50.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.693+0000 7f60c37fe640 1 -- 172.21.15.27:0/3421563076 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b00317a0 con 0x7f60c40b4ef0 2023-12-19T18:19:50.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.693+0000 7f60c37fe640 1 -- 172.21.15.27:0/3421563076 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0031b30 con 0x7f60c40b4ef0 2023-12-19T18:19:50.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.693+0000 7f60c9b98640 1 -- 172.21.15.27:0/3421563076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60c40b4ef0 msgr2=0x7f60c40b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:50.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.693+0000 7f60c9b98640 1 --2- 172.21.15.27:0/3421563076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60c40b4ef0 0x7f60c40b6300 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009870 tx=0x7f60b002b8f0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.693+0000 7f60c9b98640 1 -- 172.21.15.27:0/3421563076 shutdown_connections 2023-12-19T18:19:50.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.693+0000 7f60c9b98640 1 --2- 172.21.15.27:0/3421563076 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60c40d2350 0x7f60c40b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.693+0000 7f60c9b98640 1 --2- 172.21.15.27:0/3421563076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60c40b4ef0 0x7f60c40b6300 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.693+0000 7f60c9b98640 1 -- 172.21.15.27:0/3421563076 >> 172.21.15.27:0/3421563076 conn(0x7f60c40cf7e0 msgr2=0x7f60c40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:50.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.693+0000 7f60c9b98640 1 -- 172.21.15.27:0/3421563076 shutdown_connections 2023-12-19T18:19:50.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.693+0000 7f60c9b98640 1 -- 172.21.15.27:0/3421563076 wait complete. 2023-12-19T18:19:50.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.695+0000 7f60c9b98640 1 Processor -- start 2023-12-19T18:19:50.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.695+0000 7f60c9b98640 1 -- start start 2023-12-19T18:19:50.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.696+0000 7f60c9b98640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60c40b4ef0 0x7f60c41482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:50.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.696+0000 7f60c9b98640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60c40d2350 0x7f60c4147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:50.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.696+0000 7f60c9b98640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60c41604d0 0x7f60c4158a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:50.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.696+0000 7f60c9b98640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c40cb5e0 con 0x7f60c40d2350 2023-12-19T18:19:50.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.696+0000 7f60c9b98640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41f32b0 con 0x7f60c40b4ef0 2023-12-19T18:19:50.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.696+0000 7f60c9b98640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60c41f2fd0 con 0x7f60c41604d0 2023-12-19T18:19:50.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.696+0000 7f60c3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60c40d2350 0x7f60c4147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:50.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.696+0000 7f60c3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60c40d2350 0x7f60c4147860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44816/0 (socket says 172.21.15.27:44816) 2023-12-19T18:19:50.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.696+0000 7f60c3fff640 1 -- 172.21.15.27:0/348690583 learned_addr learned my addr 172.21.15.27:0/348690583 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:50.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.697+0000 7f60c8b96640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60c40b4ef0 0x7f60c41482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:50.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.697+0000 7f60c9397640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60c41604d0 0x7f60c4158a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:50.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.697+0000 7f60c3fff640 1 -- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60c41604d0 msgr2=0x7f60c4158a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:50.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.697+0000 7f60c3fff640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60c41604d0 0x7f60c4158a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.697+0000 7f60c3fff640 1 -- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60c40b4ef0 msgr2=0x7f60c41482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:50.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.697+0000 7f60c3fff640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60c40b4ef0 0x7f60c41482c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:50.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.697+0000 7f60c3fff640 1 -- 172.21.15.27:0/348690583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b00094d0 con 0x7f60c40d2350 2023-12-19T18:19:50.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.697+0000 7f60c3fff640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60c40d2350 0x7f60c4147860 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009870 tx=0x7f60b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:50.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.697+0000 7f60c17fa640 1 -- 172.21.15.27:0/348690583 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b003ac60 con 0x7f60c40d2350 2023-12-19T18:19:50.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.698+0000 7f60c17fa640 1 -- 172.21.15.27:0/348690583 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60b003ade0 con 0x7f60c40d2350 2023-12-19T18:19:50.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.698+0000 7f60c9b98640 1 -- 172.21.15.27:0/348690583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60c40caf90 con 0x7f60c40d2350 2023-12-19T18:19:50.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.698+0000 7f60c17fa640 1 -- 172.21.15.27:0/348690583 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60b0044630 con 0x7f60c40d2350 2023-12-19T18:19:50.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.698+0000 7f60c9b98640 1 -- 172.21.15.27:0/348690583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60c4158600 con 0x7f60c40d2350 2023-12-19T18:19:50.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.699+0000 7f60c9b98640 1 -- 172.21.15.27:0/348690583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60c4066f80 con 0x7f60c40d2350 2023-12-19T18:19:50.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.700+0000 7f60c17fa640 1 -- 172.21.15.27:0/348690583 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f60b002f050 con 0x7f60c40d2350 2023-12-19T18:19:50.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.700+0000 7f60c17fa640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f60a806a460 0x7f60a806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:50.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.701+0000 7f60c17fa640 1 -- 172.21.15.27:0/348690583 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f60b00aa790 con 0x7f60c40d2350 2023-12-19T18:19:50.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.701+0000 7f60c8b96640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f60a806a460 0x7f60a806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:50.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.702+0000 7f60c8b96640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f60a806a460 0x7f60a806c920 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f60c41efed0 tx=0x7f60b8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:50.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.705+0000 7f60c17fa640 1 -- 172.21.15.27:0/348690583 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f60b0076100 con 0x7f60c40d2350 2023-12-19T18:19:50.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:50.971+0000 7f60c9b98640 1 -- 172.21.15.27:0/348690583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]} v 0) v1 -- 0x7f60c404e840 con 0x7f60c40d2350 2023-12-19T18:19:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:51.978+0000 7f60c17fa640 1 -- 172.21.15.27:0/348690583 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fclose"]}]=0 v260) v1 ==== 134+0+0 (secure 0 0 0) 0x7f60b004e020 con 0x7f60c40d2350 2023-12-19T18:19:51.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:51.981+0000 7f60c9b98640 1 -- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f60a806a460 msgr2=0x7f60a806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:51.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:51.981+0000 7f60c9b98640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f60a806a460 0x7f60a806c920 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f60c41efed0 tx=0x7f60b8009040 comp rx=0 tx=0).stop 2023-12-19T18:19:51.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:51.981+0000 7f60c9b98640 1 -- 172.21.15.27:0/348690583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60c40d2350 msgr2=0x7f60c4147860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:51.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:51.981+0000 7f60c9b98640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60c40d2350 0x7f60c4147860 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f60b0009870 tx=0x7f60b00046c0 comp rx=0 tx=0).stop 2023-12-19T18:19:51.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:51.981+0000 7f60c9b98640 1 -- 172.21.15.27:0/348690583 shutdown_connections 2023-12-19T18:19:51.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:51.981+0000 7f60c9b98640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f60a806a460 0x7f60a806c920 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:51.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:51.981+0000 7f60c9b98640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60c41604d0 0x7f60c4158a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:51.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:51.981+0000 7f60c9b98640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60c40d2350 0x7f60c4147860 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:51.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:51.981+0000 7f60c9b98640 1 --2- 172.21.15.27:0/348690583 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60c40b4ef0 0x7f60c41482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:51.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:51.981+0000 7f60c9b98640 1 -- 172.21.15.27:0/348690583 >> 172.21.15.27:0/348690583 conn(0x7f60c40cf7e0 msgr2=0x7f60c415ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:51.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:51.981+0000 7f60c9b98640 1 -- 172.21.15.27:0/348690583 shutdown_connections 2023-12-19T18:19:51.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:51.981+0000 7f60c9b98640 1 -- 172.21.15.27:0/348690583 wait complete. 2023-12-19T18:19:52.003 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-12-19T18:19:52.003 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestClusterFull) That when the full flag is encountered during asynchronous 2023-12-19T18:19:52.003 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestClusterFull) 2023-12-19T18:19:52.004 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-12-19T18:19:52.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.232+0000 7f102d6dc640 1 Processor -- start 2023-12-19T18:19:52.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.233+0000 7f102d6dc640 1 -- start start 2023-12-19T18:19:52.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.234+0000 7f102d6dc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f10281604d0 0x7f10280b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:52.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.234+0000 7f102d6dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10280b7310 0x7f10280b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:52.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.234+0000 7f102d6dc640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10280b5e90 con 0x7f10280d2350 2023-12-19T18:19:52.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.234+0000 7f102d6dc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10280d2b30 con 0x7f10281604d0 2023-12-19T18:19:52.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.234+0000 7f102d6dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10280d2850 con 0x7f10280b7310 2023-12-19T18:19:52.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.234+0000 7f101ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f10280d2350 0x7f10280d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40676/0 (socket says 172.21.15.27:40676) 2023-12-19T18:19:52.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.234+0000 7f101ffff640 1 -- 172.21.15.27:0/1158884188 learned_addr learned my addr 172.21.15.27:0/1158884188 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:52.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f1027fff640 1 --2- 172.21.15.27:0/1158884188 >> v2:172.21.15.192:3300/0 conn(0x7f10281604d0 0x7f10280b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:52.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f102cedb640 1 --2- 172.21.15.27:0/1158884188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10280b7310 0x7f10280b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:52.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 600006777 0 0) 0x7f10280b5e90 con 0x7f10280d2350 2023-12-19T18:19:52.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1014003520 con 0x7f10280d2350 2023-12-19T18:19:52.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1614155274 0 0) 0x7f1014003520 con 0x7f10280d2350 2023-12-19T18:19:52.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10280b7310 msgr2=0x7f10280b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:52.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f10277fe640 1 --2- 172.21.15.27:0/1158884188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10280b7310 0x7f10280b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:52.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 >> v2:172.21.15.192:3300/0 conn(0x7f10281604d0 msgr2=0x7f10280b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:52.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f10277fe640 1 --2- 172.21.15.27:0/1158884188 >> v2:172.21.15.192:3300/0 conn(0x7f10281604d0 0x7f10280b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:52.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280cded0 con 0x7f10280d2350 2023-12-19T18:19:52.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f1010001670 con 0x7f10280d2350 2023-12-19T18:19:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 >> v1:172.21.15.27:6789/0 conn(0x7f10280d2350 legacy=0x7f10280d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f10277fe640 1 --2- 172.21.15.27:0/1158884188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1014003970 0x7f1014003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.235+0000 7f10277fe640 1 --2- 172.21.15.27:0/1158884188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10140047c0 0x7f1014004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.236+0000 7f10277fe640 1 --2- 172.21.15.27:0/1158884188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10280b7310 0x7f1014005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.236+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280d2b30 con 0x7f1014003970 2023-12-19T18:19:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.236+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10280d2850 con 0x7f10140047c0 2023-12-19T18:19:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.236+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101400b980 con 0x7f10280b7310 2023-12-19T18:19:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.236+0000 7f1027fff640 1 --2- 172.21.15.27:0/1158884188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1014003970 0x7f1014003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.236+0000 7f102cedb640 1 --2- 172.21.15.27:0/1158884188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10280b7310 0x7f1014005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.236+0000 7f1027fff640 1 -- 172.21.15.27:0/1158884188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10280b7310 msgr2=0x7f1014005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:52.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.236+0000 7f1027fff640 1 --2- 172.21.15.27:0/1158884188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10280b7310 0x7f1014005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:52.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.236+0000 7f1027fff640 1 -- 172.21.15.27:0/1158884188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10140047c0 msgr2=0x7f1014004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:52.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.236+0000 7f1027fff640 1 --2- 172.21.15.27:0/1158884188 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10140047c0 0x7f1014004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:52.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.236+0000 7f1027fff640 1 -- 172.21.15.27:0/1158884188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1014003520 con 0x7f1014003970 2023-12-19T18:19:52.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.236+0000 7f102cedb640 1 --2- 172.21.15.27:0/1158884188 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f10280b7310 0x7f1014005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.237+0000 7f1027fff640 1 --2- 172.21.15.27:0/1158884188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1014003970 0x7f1014003d90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1018009ac0 tx=0x7f10140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f69fea911f51dd8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.238+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101802c710 con 0x7f1014003970 2023-12-19T18:19:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.238+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f101802c870 con 0x7f1014003970 2023-12-19T18:19:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.238+0000 7f10277fe640 1 -- 172.21.15.27:0/1158884188 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f101802cc00 con 0x7f1014003970 2023-12-19T18:19:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.238+0000 7f102d6dc640 1 -- 172.21.15.27:0/1158884188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1014003970 msgr2=0x7f1014003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.238+0000 7f102d6dc640 1 --2- 172.21.15.27:0/1158884188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1014003970 0x7f1014003d90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f1018009ac0 tx=0x7f10140057f0 comp rx=0 tx=0).stop 2023-12-19T18:19:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.238+0000 7f102cedb640 1 -- 172.21.15.27:0/1158884188 reap_dead start 2023-12-19T18:19:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.238+0000 7f102d6dc640 1 -- 172.21.15.27:0/1158884188 shutdown_connections 2023-12-19T18:19:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.238+0000 7f102d6dc640 1 -- 172.21.15.27:0/1158884188 >> 172.21.15.27:0/1158884188 conn(0x7f10280cf7e0 msgr2=0x7f10280cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.238+0000 7f102d6dc640 1 -- 172.21.15.27:0/1158884188 shutdown_connections 2023-12-19T18:19:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.238+0000 7f102d6dc640 1 -- 172.21.15.27:0/1158884188 wait complete. 2023-12-19T18:19:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.240+0000 7f102d6dc640 1 Processor -- start 2023-12-19T18:19:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.240+0000 7f102d6dc640 1 -- start start 2023-12-19T18:19:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.241+0000 7f102d6dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014003970 0x7f10281f21b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.241+0000 7f102d6dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10280c6870 0x7f10280b3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.241+0000 7f102d6dc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10280c6cf0 0x7f10280b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.241+0000 7f102d6dc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1028149c40 con 0x7f10280c6cf0 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.241+0000 7f102d6dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10281f26f0 con 0x7f10280c6870 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.241+0000 7f102d6dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10280c9f60 con 0x7f1014003970 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.242+0000 7f102cedb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10280c6cf0 0x7f10280b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.242+0000 7f102cedb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10280c6cf0 0x7f10280b3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58212/0 (socket says 172.21.15.27:58212) 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.242+0000 7f102cedb640 1 -- 172.21.15.27:0/1512722360 learned_addr learned my addr 172.21.15.27:0/1512722360 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.242+0000 7f1027fff640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014003970 0x7f10281f21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.242+0000 7f101ffff640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10280c6870 0x7f10280b3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.242+0000 7f102cedb640 1 -- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014003970 msgr2=0x7f10281f21b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.242+0000 7f102cedb640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014003970 0x7f10281f21b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.242+0000 7f102cedb640 1 -- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10280c6870 msgr2=0x7f10280b3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.242+0000 7f102cedb640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10280c6870 0x7f10280b3720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.242+0000 7f102cedb640 1 -- 172.21.15.27:0/1512722360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10280ca240 con 0x7f10280c6cf0 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.242+0000 7f1027fff640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014003970 0x7f10281f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.242+0000 7f101ffff640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10280c6870 0x7f10280b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.243+0000 7f102cedb640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10280c6cf0 0x7f10280b3c60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f10280cf170 tx=0x7f102000f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:52.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.243+0000 7f10257fa640 1 -- 172.21.15.27:0/1512722360 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020017400 con 0x7f10280c6cf0 2023-12-19T18:19:52.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.243+0000 7f10257fa640 1 -- 172.21.15.27:0/1512722360 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1020011040 con 0x7f10280c6cf0 2023-12-19T18:19:52.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.243+0000 7f10257fa640 1 -- 172.21.15.27:0/1512722360 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1020017780 con 0x7f10280c6cf0 2023-12-19T18:19:52.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.243+0000 7f102d6dc640 1 -- 172.21.15.27:0/1512722360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10280ca520 con 0x7f10280c6cf0 2023-12-19T18:19:52.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.243+0000 7f102d6dc640 1 -- 172.21.15.27:0/1512722360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10280caae0 con 0x7f10280c6cf0 2023-12-19T18:19:52.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.244+0000 7f102d6dc640 1 -- 172.21.15.27:0/1512722360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1028066f80 con 0x7f10280c6cf0 2023-12-19T18:19:52.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.245+0000 7f10257fa640 1 -- 172.21.15.27:0/1512722360 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f10200203b0 con 0x7f10280c6cf0 2023-12-19T18:19:52.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.246+0000 7f10257fa640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f100806a480 0x7f100806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:52.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.246+0000 7f10257fa640 1 -- 172.21.15.27:0/1512722360 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1020088990 con 0x7f10280c6cf0 2023-12-19T18:19:52.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.247+0000 7f1027fff640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f100806a480 0x7f100806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:52.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.248+0000 7f1027fff640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f100806a480 0x7f100806c940 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1018036000 tx=0x7f1018030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:52.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.250+0000 7f10257fa640 1 -- 172.21.15.27:0/1512722360 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f10200542f0 con 0x7f10280c6cf0 2023-12-19T18:19:52.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.515+0000 7f102d6dc640 1 -- 172.21.15.27:0/1512722360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f10280cad00 con 0x7f10280c6cf0 2023-12-19T18:19:52.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.980+0000 7f10257fa640 1 -- 172.21.15.27:0/1512722360 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v261) v1 ==== 136+0+0 (secure 0 0 0) 0x7f1020057910 con 0x7f10280c6cf0 2023-12-19T18:19:52.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.983+0000 7f102d6dc640 1 -- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f100806a480 msgr2=0x7f100806c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:52.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.983+0000 7f102d6dc640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f100806a480 0x7f100806c940 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f1018036000 tx=0x7f1018030000 comp rx=0 tx=0).stop 2023-12-19T18:19:52.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.983+0000 7f102d6dc640 1 -- 172.21.15.27:0/1512722360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10280c6cf0 msgr2=0x7f10280b3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:52.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.983+0000 7f102d6dc640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10280c6cf0 0x7f10280b3c60 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f10280cf170 tx=0x7f102000f4a0 comp rx=0 tx=0).stop 2023-12-19T18:19:52.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.984+0000 7f102d6dc640 1 -- 172.21.15.27:0/1512722360 shutdown_connections 2023-12-19T18:19:52.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.984+0000 7f102d6dc640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f100806a480 0x7f100806c940 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:52.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.984+0000 7f102d6dc640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10280c6cf0 0x7f10280b3c60 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:52.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.984+0000 7f102d6dc640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f10280c6870 0x7f10280b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:52.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.984+0000 7f102d6dc640 1 --2- 172.21.15.27:0/1512722360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1014003970 0x7f10281f21b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.984+0000 7f102d6dc640 1 -- 172.21.15.27:0/1512722360 >> 172.21.15.27:0/1512722360 conn(0x7f10280cf7e0 msgr2=0x7f10280d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.984+0000 7f102d6dc640 1 -- 172.21.15.27:0/1512722360 shutdown_connections 2023-12-19T18:19:52.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:52.984+0000 7f102d6dc640 1 -- 172.21.15.27:0/1512722360 wait complete. 2023-12-19T18:19:53.005 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-12-19T18:19:53.255 INFO:teuthology.orchestra.run.smithi027.stdout:{"osd_objectstore":"memstore"} 2023-12-19T18:19:53.255 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-12-19T18:19:53.255 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:19:53.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.490+0000 7f0623c41640 1 Processor -- start 2023-12-19T18:19:53.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.491+0000 7f0623c41640 1 -- start start 2023-12-19T18:19:53.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.492+0000 7f0623c41640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061c0b8c30 0x7f061c0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:53.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.492+0000 7f0623c41640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f061c0ce130 0x7f061c0cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:53.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.492+0000 7f0623c41640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f061c0d0350 con 0x7f061c0cf9b0 2023-12-19T18:19:53.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.492+0000 7f0623c41640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f061c0cea80 con 0x7f061c0ce130 2023-12-19T18:19:53.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.492+0000 7f0623c41640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c0b5f10 con 0x7f061c0b8c30 2023-12-19T18:19:53.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.492+0000 7f06221de640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f061c0ce130 0x7f061c0cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:53.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.492+0000 7f06211dc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f061c0cf9b0 0x7f061c0cda20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40680/0 (socket says 172.21.15.27:40680) 2023-12-19T18:19:53.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.492+0000 7f06211dc640 1 -- 172.21.15.27:0/1138485969 learned_addr learned my addr 172.21.15.27:0/1138485969 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:53.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.493+0000 7f06219dd640 1 --2- 172.21.15.27:0/1138485969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061c0b8c30 0x7f061c0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:53.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.493+0000 7f06221de640 1 -- 172.21.15.27:0/1138485969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061c0b8c30 msgr2=0x7f061c0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:53.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.493+0000 7f06221de640 1 --2- 172.21.15.27:0/1138485969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061c0b8c30 0x7f061c0d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:53.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.493+0000 7f06221de640 1 -- 172.21.15.27:0/1138485969 >> v1:172.21.15.27:6789/0 conn(0x7f061c0cf9b0 legacy=0x7f061c0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:53.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.493+0000 7f06221de640 1 -- 172.21.15.27:0/1138485969 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c0ce7a0 con 0x7f061c0ce130 2023-12-19T18:19:53.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.493+0000 7f06209db640 1 -- 172.21.15.27:0/1138485969 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2261346857 0 0) 0x7f061c0d0350 con 0x7f061c0cf9b0 2023-12-19T18:19:53.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.494+0000 7f06221de640 1 --2- 172.21.15.27:0/1138485969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0ce130 0x7f061c0cfe10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0614009d90 tx=0x7f061402be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a99f36e3812a8da server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:53.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.494+0000 7f06209db640 1 -- 172.21.15.27:0/1138485969 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061402cbd0 con 0x7f061c0ce130 2023-12-19T18:19:53.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.494+0000 7f06209db640 1 -- 172.21.15.27:0/1138485969 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f061402cd30 con 0x7f061c0ce130 2023-12-19T18:19:53.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.494+0000 7f06209db640 1 -- 172.21.15.27:0/1138485969 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061402cea0 con 0x7f061c0ce130 2023-12-19T18:19:53.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.495+0000 7f0623c41640 1 -- 172.21.15.27:0/1138485969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0ce130 msgr2=0x7f061c0cfe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:53.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.495+0000 7f0623c41640 1 --2- 172.21.15.27:0/1138485969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0ce130 0x7f061c0cfe10 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f0614009d90 tx=0x7f061402be20 comp rx=0 tx=0).stop 2023-12-19T18:19:53.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.495+0000 7f0623c41640 1 -- 172.21.15.27:0/1138485969 shutdown_connections 2023-12-19T18:19:53.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.495+0000 7f0623c41640 1 --2- 172.21.15.27:0/1138485969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0ce130 0x7f061c0cfe10 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:53.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.495+0000 7f0623c41640 1 --2- 172.21.15.27:0/1138485969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061c0b8c30 0x7f061c0d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:53.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.495+0000 7f0623c41640 1 -- 172.21.15.27:0/1138485969 >> 172.21.15.27:0/1138485969 conn(0x7f061c0d2fa0 msgr2=0x7f061c0d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:53.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.495+0000 7f0623c41640 1 -- 172.21.15.27:0/1138485969 shutdown_connections 2023-12-19T18:19:53.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.495+0000 7f0623c41640 1 -- 172.21.15.27:0/1138485969 wait complete. 2023-12-19T18:19:53.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.497+0000 7f0623c41640 1 Processor -- start 2023-12-19T18:19:53.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.497+0000 7f0623c41640 1 -- start start 2023-12-19T18:19:53.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.498+0000 7f0623c41640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f061c0b8c30 0x7f061c154de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:53.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.498+0000 7f0623c41640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061c0ce130 0x7f061c0cb110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:53.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.498+0000 7f0623c41640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0cf9b0 0x7f061c0cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:53.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.498+0000 7f0623c41640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c1f2c00 con 0x7f061c0cf9b0 2023-12-19T18:19:53.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.498+0000 7f0623c41640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c159d60 con 0x7f061c0b8c30 2023-12-19T18:19:53.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.498+0000 7f0623c41640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f061c1591e0 con 0x7f061c0ce130 2023-12-19T18:19:53.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.498+0000 7f06221de640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0cf9b0 0x7f061c0cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:53.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.498+0000 7f06221de640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0cf9b0 0x7f061c0cb650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58236/0 (socket says 172.21.15.27:58236) 2023-12-19T18:19:53.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.498+0000 7f06221de640 1 -- 172.21.15.27:0/1428429434 learned_addr learned my addr 172.21.15.27:0/1428429434 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:53.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.499+0000 7f06211dc640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061c0ce130 0x7f061c0cb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:53.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.499+0000 7f06219dd640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f061c0b8c30 0x7f061c154de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:53.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.499+0000 7f06221de640 1 -- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061c0ce130 msgr2=0x7f061c0cb110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:53.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.499+0000 7f06221de640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061c0ce130 0x7f061c0cb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:53.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.499+0000 7f06221de640 1 -- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f061c0b8c30 msgr2=0x7f061c154de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:53.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.499+0000 7f06221de640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f061c0b8c30 0x7f061c154de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:53.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.499+0000 7f06221de640 1 -- 172.21.15.27:0/1428429434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0614009a70 con 0x7f061c0cf9b0 2023-12-19T18:19:53.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.499+0000 7f06219dd640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f061c0b8c30 0x7f061c154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:53.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.499+0000 7f06221de640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0cf9b0 0x7f061c0cb650 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0614009d90 tx=0x7f0614004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:53.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.499+0000 7f06127fc640 1 -- 172.21.15.27:0/1428429434 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f061402cea0 con 0x7f061c0cf9b0 2023-12-19T18:19:53.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.499+0000 7f06127fc640 1 -- 172.21.15.27:0/1428429434 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0614003bb0 con 0x7f061c0cf9b0 2023-12-19T18:19:53.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.499+0000 7f06127fc640 1 -- 172.21.15.27:0/1428429434 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0614009450 con 0x7f061c0cf9b0 2023-12-19T18:19:53.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.500+0000 7f0623c41640 1 -- 172.21.15.27:0/1428429434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c1594c0 con 0x7f061c0cf9b0 2023-12-19T18:19:53.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.500+0000 7f0623c41640 1 -- 172.21.15.27:0/1428429434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c159a80 con 0x7f061c0cf9b0 2023-12-19T18:19:53.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.501+0000 7f0623c41640 1 -- 172.21.15.27:0/1428429434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c0d14c0 con 0x7f061c0cf9b0 2023-12-19T18:19:53.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.502+0000 7f06127fc640 1 -- 172.21.15.27:0/1428429434 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f061403b050 con 0x7f061c0cf9b0 2023-12-19T18:19:53.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.503+0000 7f06127fc640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f060006a430 0x7f060006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:53.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.503+0000 7f06127fc640 1 -- 172.21.15.27:0/1428429434 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f06140aa990 con 0x7f061c0cf9b0 2023-12-19T18:19:53.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.503+0000 7f06219dd640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f060006a430 0x7f060006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:53.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.505+0000 7f06219dd640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f060006a430 0x7f060006c8f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0618006fd0 tx=0x7f0618008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:53.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.508+0000 7f06127fc640 1 -- 172.21.15.27:0/1428429434 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f061c0d14c0 con 0x7f061c0cf9b0 2023-12-19T18:19:53.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.703+0000 7f0623c41640 1 -- 172.21.15.27:0/1428429434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f061c0b3ac0 con 0x7f061c0cf9b0 2023-12-19T18:19:53.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.712+0000 7f06127fc640 1 -- 172.21.15.27:0/1428429434 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v8)=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f061c0b3ac0 con 0x7f061c0cf9b0 2023-12-19T18:19:53.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.714+0000 7f0623c41640 1 -- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f060006a430 msgr2=0x7f060006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:53.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.714+0000 7f0623c41640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f060006a430 0x7f060006c8f0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f0618006fd0 tx=0x7f0618008040 comp rx=0 tx=0).stop 2023-12-19T18:19:53.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.714+0000 7f0623c41640 1 -- 172.21.15.27:0/1428429434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0cf9b0 msgr2=0x7f061c0cb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:53.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.714+0000 7f0623c41640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0cf9b0 0x7f061c0cb650 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f0614009d90 tx=0x7f0614004a40 comp rx=0 tx=0).stop 2023-12-19T18:19:53.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.714+0000 7f0623c41640 1 -- 172.21.15.27:0/1428429434 shutdown_connections 2023-12-19T18:19:53.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.714+0000 7f0623c41640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f060006a430 0x7f060006c8f0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:53.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.714+0000 7f0623c41640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0cf9b0 0x7f061c0cb650 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:53.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.714+0000 7f0623c41640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f061c0ce130 0x7f061c0cb110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:53.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.714+0000 7f0623c41640 1 --2- 172.21.15.27:0/1428429434 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f061c0b8c30 0x7f061c154de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:53.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.715+0000 7f0623c41640 1 -- 172.21.15.27:0/1428429434 >> 172.21.15.27:0/1428429434 conn(0x7f061c0d2fa0 msgr2=0x7f061c0d1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:53.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.715+0000 7f0623c41640 1 -- 172.21.15.27:0/1428429434 shutdown_connections 2023-12-19T18:19:53.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:53.715+0000 7f0623c41640 1 -- 172.21.15.27:0/1428429434 wait complete. 2023-12-19T18:19:53.736 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:19:53.736 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:19:53.872 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:19:54.000 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:19:54.000 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:19:54.000 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:19:54.066 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:19:54.066 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:19:54.066 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:19:54.125 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:19:54.126 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:19:54.126 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:19:54.191 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:19:54.191 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:19:54.192 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:19:54.251 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:19:54.252 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:19:54.252 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:19:54.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.495+0000 7f0740612640 1 Processor -- start 2023-12-19T18:19:54.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.495+0000 7f0740612640 1 -- start start 2023-12-19T18:19:54.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.497+0000 7f0740612640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f073806bbf0 0x7f07380cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:54.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.497+0000 7f0740612640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f073806c880 0x7f07380cdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:54.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.497+0000 7f0740612640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f073815b180 con 0x7f0738159c40 2023-12-19T18:19:54.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.497+0000 7f0740612640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07380ca850 con 0x7f073806bbf0 2023-12-19T18:19:54.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.497+0000 7f0740612640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073815a570 con 0x7f073806c880 2023-12-19T18:19:54.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.497+0000 7f073e3ae640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0738159c40 0x7f073806b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40692/0 (socket says 172.21.15.27:40692) 2023-12-19T18:19:54.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.497+0000 7f073e3ae640 1 -- 172.21.15.27:0/1160462624 learned_addr learned my addr 172.21.15.27:0/1160462624 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:54.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.498+0000 7f073dbad640 1 --2- 172.21.15.27:0/1160462624 >> v2:172.21.15.27:3300/0 conn(0x7f073806bbf0 0x7f07380cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.498+0000 7f073ebaf640 1 --2- 172.21.15.27:0/1160462624 >> v2:172.21.15.192:3300/0 conn(0x7f073806c880 0x7f07380cdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.498+0000 7f073dbad640 1 -- 172.21.15.27:0/1160462624 >> v2:172.21.15.192:3300/0 conn(0x7f073806c880 msgr2=0x7f07380cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.498+0000 7f073dbad640 1 --2- 172.21.15.27:0/1160462624 >> v2:172.21.15.192:3300/0 conn(0x7f073806c880 0x7f07380cdd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.498+0000 7f073dbad640 1 -- 172.21.15.27:0/1160462624 >> v1:172.21.15.27:6789/0 conn(0x7f0738159c40 legacy=0x7f073806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.498+0000 7f073d3ac640 1 -- 172.21.15.27:0/1160462624 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 55089265 0 0) 0x7f073815b180 con 0x7f0738159c40 2023-12-19T18:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.498+0000 7f073dbad640 1 -- 172.21.15.27:0/1160462624 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073815a850 con 0x7f073806bbf0 2023-12-19T18:19:54.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.498+0000 7f073ebaf640 1 --2- 172.21.15.27:0/1160462624 >> v2:172.21.15.192:3300/0 conn(0x7f073806c880 0x7f07380cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.499+0000 7f073dbad640 1 --2- 172.21.15.27:0/1160462624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806bbf0 0x7f07380cd800 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f073806f470 tx=0x7f072802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8a82e94109d086 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.500+0000 7f073d3ac640 1 -- 172.21.15.27:0/1160462624 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728031830 con 0x7f073806bbf0 2023-12-19T18:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.500+0000 7f073d3ac640 1 -- 172.21.15.27:0/1160462624 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0728031990 con 0x7f073806bbf0 2023-12-19T18:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.500+0000 7f073d3ac640 1 -- 172.21.15.27:0/1160462624 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728031d20 con 0x7f073806bbf0 2023-12-19T18:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.500+0000 7f0740612640 1 -- 172.21.15.27:0/1160462624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806bbf0 msgr2=0x7f07380cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.500+0000 7f0740612640 1 --2- 172.21.15.27:0/1160462624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806bbf0 0x7f07380cd800 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f073806f470 tx=0x7f072802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.500+0000 7f0740612640 1 -- 172.21.15.27:0/1160462624 shutdown_connections 2023-12-19T18:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.500+0000 7f0740612640 1 --2- 172.21.15.27:0/1160462624 >> v2:172.21.15.192:3300/0 conn(0x7f073806c880 0x7f07380cdd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.500+0000 7f0740612640 1 --2- 172.21.15.27:0/1160462624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806bbf0 0x7f07380cd800 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.500+0000 7f0740612640 1 -- 172.21.15.27:0/1160462624 >> 172.21.15.27:0/1160462624 conn(0x7f073806a330 msgr2=0x7f07380b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.501+0000 7f0740612640 1 -- 172.21.15.27:0/1160462624 shutdown_connections 2023-12-19T18:19:54.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.501+0000 7f0740612640 1 -- 172.21.15.27:0/1160462624 wait complete. 2023-12-19T18:19:54.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.503+0000 7f0740612640 1 Processor -- start 2023-12-19T18:19:54.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.503+0000 7f0740612640 1 -- start start 2023-12-19T18:19:54.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.504+0000 7f0740612640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f073806bbf0 0x7f07380b2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:54.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.504+0000 7f0740612640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f073806c880 0x7f0738155400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.504+0000 7f0740612640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0738159c40 0x7f07380ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.504+0000 7f0740612640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380c3a00 con 0x7f0738159c40 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.504+0000 7f0740612640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380b20d0 con 0x7f073806bbf0 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.504+0000 7f0740612640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07380b2370 con 0x7f073806c880 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.504+0000 7f073ebaf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0738159c40 0x7f07380ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.504+0000 7f073ebaf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0738159c40 0x7f07380ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58254/0 (socket says 172.21.15.27:58254) 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.504+0000 7f073ebaf640 1 -- 172.21.15.27:0/4253917050 learned_addr learned my addr 172.21.15.27:0/4253917050 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.505+0000 7f073e3ae640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f073806bbf0 0x7f07380b2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.505+0000 7f073dbad640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f073806c880 0x7f0738155400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.505+0000 7f073ebaf640 1 -- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f073806c880 msgr2=0x7f0738155400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.505+0000 7f073ebaf640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f073806c880 0x7f0738155400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.505+0000 7f073ebaf640 1 -- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f073806bbf0 msgr2=0x7f07380b2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.505+0000 7f073ebaf640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f073806bbf0 0x7f07380b2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.505+0000 7f073ebaf640 1 -- 172.21.15.27:0/4253917050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0728009400 con 0x7f0738159c40 2023-12-19T18:19:54.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.505+0000 7f073e3ae640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f073806bbf0 0x7f07380b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:54.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.505+0000 7f073ebaf640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0738159c40 0x7f07380ccee0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f073000cef0 tx=0x7f073000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:54.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.505+0000 7f072effd640 1 -- 172.21.15.27:0/4253917050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073000d840 con 0x7f0738159c40 2023-12-19T18:19:54.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.505+0000 7f072effd640 1 -- 172.21.15.27:0/4253917050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073000d9c0 con 0x7f0738159c40 2023-12-19T18:19:54.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.505+0000 7f0740612640 1 -- 172.21.15.27:0/4253917050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07380b2650 con 0x7f0738159c40 2023-12-19T18:19:54.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.506+0000 7f072effd640 1 -- 172.21.15.27:0/4253917050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073000dd70 con 0x7f0738159c40 2023-12-19T18:19:54.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.506+0000 7f0740612640 1 -- 172.21.15.27:0/4253917050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0738155940 con 0x7f0738159c40 2023-12-19T18:19:54.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.507+0000 7f0740612640 1 -- 172.21.15.27:0/4253917050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07380c3860 con 0x7f0738159c40 2023-12-19T18:19:54.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.508+0000 7f072effd640 1 -- 172.21.15.27:0/4253917050 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f073001e3b0 con 0x7f0738159c40 2023-12-19T18:19:54.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.509+0000 7f072effd640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f071c06a430 0x7f071c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:54.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.509+0000 7f072effd640 1 -- 172.21.15.27:0/4253917050 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f07300890e0 con 0x7f0738159c40 2023-12-19T18:19:54.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.510+0000 7f073e3ae640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f071c06a430 0x7f071c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:54.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.511+0000 7f073e3ae640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f071c06a430 0x7f071c06c8f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0734000f60 tx=0x7f0734009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:54.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.516+0000 7f072effd640 1 -- 172.21.15.27:0/4253917050 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f07380c3860 con 0x7f0738159c40 2023-12-19T18:19:54.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.749+0000 7f0740612640 1 -- 172.21.15.27:0/4253917050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f07380c3eb0 con 0x7f0738159c40 2023-12-19T18:19:54.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.751+0000 7f072effd640 1 -- 172.21.15.27:0/4253917050 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 32 v32) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f073001ade0 con 0x7f0738159c40 2023-12-19T18:19:54.751 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:19:54.751 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":32,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5024,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1624358030","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1624358030},{"type":"v1","addr":"172.21.15.192:6836","nonce":1624358030}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":5030,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3025339386","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3025339386},{"type":"v1","addr":"172.21.15.192:6837","nonce":3025339386}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6117,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/1685117850","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":1685117850},{"type":"v1","addr":"172.21.15.27:6833","nonce":1685117850}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26},{"gid":6435,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2731153231","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2731153231},{"type":"v1","addr":"172.21.15.27:6835","nonce":2731153231}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":32}],"filesystems":[]} 2023-12-19T18:19:54.751 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 32 2023-12-19T18:19:54.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.753+0000 7f0740612640 1 -- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f071c06a430 msgr2=0x7f071c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:54.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.753+0000 7f0740612640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f071c06a430 0x7f071c06c8f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0734000f60 tx=0x7f0734009040 comp rx=0 tx=0).stop 2023-12-19T18:19:54.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.753+0000 7f0740612640 1 -- 172.21.15.27:0/4253917050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0738159c40 msgr2=0x7f07380ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:54.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.753+0000 7f0740612640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0738159c40 0x7f07380ccee0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f073000cef0 tx=0x7f073000a710 comp rx=0 tx=0).stop 2023-12-19T18:19:54.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.754+0000 7f0740612640 1 -- 172.21.15.27:0/4253917050 shutdown_connections 2023-12-19T18:19:54.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.754+0000 7f0740612640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f071c06a430 0x7f071c06c8f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:54.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.754+0000 7f0740612640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0738159c40 0x7f07380ccee0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:54.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.754+0000 7f0740612640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f073806c880 0x7f0738155400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:54.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.754+0000 7f0740612640 1 --2- 172.21.15.27:0/4253917050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f073806bbf0 0x7f07380b2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:54.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.754+0000 7f0740612640 1 -- 172.21.15.27:0/4253917050 >> 172.21.15.27:0/4253917050 conn(0x7f073806a330 msgr2=0x7f073815be60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:54.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.754+0000 7f0740612640 1 -- 172.21.15.27:0/4253917050 shutdown_connections 2023-12-19T18:19:54.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:54.754+0000 7f0740612640 1 -- 172.21.15.27:0/4253917050 wait complete. 2023-12-19T18:19:54.774 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:19:54.774 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:19:54.775 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:19:54.775 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:54.775 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:19:54.775 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:19:54.775 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:19:54.775 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:54.775 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:19:54.776 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:19:54.776 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:19:54.776 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:54.776 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:19:54.776 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:19:54.776 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:19:54.776 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:19:54.791 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:19:54.791 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:19:54.796 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:19:54.797 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:19:54.800 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:19:54.800 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:19:54.802 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:19:54.802 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:19:54.849 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:19:54.850 DEBUG:teuthology.parallel:result is None 2023-12-19T18:19:54.850 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:19:54.851 DEBUG:teuthology.parallel:result is None 2023-12-19T18:19:54.856 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:19:54.857 DEBUG:teuthology.parallel:result is None 2023-12-19T18:19:54.858 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:19:54.858 DEBUG:teuthology.parallel:result is None 2023-12-19T18:19:54.858 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:19:55.014 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-19T18:19:55.015 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-19T18:19:55.018 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:19:55.023 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:19:55.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.066+0000 7f038f59e640 1 Processor -- start 2023-12-19T18:19:55.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.066+0000 7f038f59e640 1 -- start start 2023-12-19T18:19:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.068+0000 7f038f59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f03900b49d0 0x7f0390160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.068+0000 7f038f59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03900b89b0 0x7f039015c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.068+0000 7f038f59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03900b5ca0 con 0x7f03900b4db0 2023-12-19T18:19:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.068+0000 7f038f59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0390161800 con 0x7f03900b89b0 2023-12-19T18:19:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.068+0000 7f038f59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0390161240 con 0x7f03900b49d0 2023-12-19T18:19:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.068+0000 7f0387fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f03900b89b0 0x7f039015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.068+0000 7f038ed9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f03900b4db0 0x7f039006bc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40722/0 (socket says 172.21.15.27:40722) 2023-12-19T18:19:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.068+0000 7f038ed9d640 1 -- 172.21.15.27:0/899048993 learned_addr learned my addr 172.21.15.27:0/899048993 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:55.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.068+0000 7f038e59c640 1 --2- 172.21.15.27:0/899048993 >> v2:172.21.15.192:3300/0 conn(0x7f03900b49d0 0x7f0390160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.068+0000 7f0387fff640 1 -- 172.21.15.27:0/899048993 >> v2:172.21.15.192:3300/0 conn(0x7f03900b49d0 msgr2=0x7f0390160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.068+0000 7f0387fff640 1 --2- 172.21.15.27:0/899048993 >> v2:172.21.15.192:3300/0 conn(0x7f03900b49d0 0x7f0390160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.068+0000 7f0387fff640 1 -- 172.21.15.27:0/899048993 >> v1:172.21.15.27:6789/0 conn(0x7f03900b4db0 legacy=0x7f039006bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.068+0000 7f038dd9b640 1 -- 172.21.15.27:0/899048993 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 110028877 0 0) 0x7f03900b5ca0 con 0x7f03900b4db0 2023-12-19T18:19:55.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.069+0000 7f0387fff640 1 -- 172.21.15.27:0/899048993 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390161520 con 0x7f03900b89b0 2023-12-19T18:19:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.070+0000 7f0387fff640 1 --2- 172.21.15.27:0/899048993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03900b89b0 0x7f039015c880 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f039006cd00 tx=0x7f037802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494653962b1b3d1e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.070+0000 7f038dd9b640 1 -- 172.21.15.27:0/899048993 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378030610 con 0x7f03900b89b0 2023-12-19T18:19:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.070+0000 7f038dd9b640 1 -- 172.21.15.27:0/899048993 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0378030770 con 0x7f03900b89b0 2023-12-19T18:19:55.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.070+0000 7f038dd9b640 1 -- 172.21.15.27:0/899048993 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378030b00 con 0x7f03900b89b0 2023-12-19T18:19:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.071+0000 7f038f59e640 1 -- 172.21.15.27:0/899048993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03900b89b0 msgr2=0x7f039015c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.071+0000 7f038f59e640 1 --2- 172.21.15.27:0/899048993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03900b89b0 0x7f039015c880 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f039006cd00 tx=0x7f037802b960 comp rx=0 tx=0).stop 2023-12-19T18:19:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.071+0000 7f038f59e640 1 -- 172.21.15.27:0/899048993 shutdown_connections 2023-12-19T18:19:55.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.071+0000 7f038f59e640 1 --2- 172.21.15.27:0/899048993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03900b89b0 0x7f039015c880 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.071+0000 7f038f59e640 1 --2- 172.21.15.27:0/899048993 >> v2:172.21.15.192:3300/0 conn(0x7f03900b49d0 0x7f0390160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.071+0000 7f038f59e640 1 -- 172.21.15.27:0/899048993 >> 172.21.15.27:0/899048993 conn(0x7f03900cf760 msgr2=0x7f03900cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.071+0000 7f038f59e640 1 -- 172.21.15.27:0/899048993 shutdown_connections 2023-12-19T18:19:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.071+0000 7f038f59e640 1 -- 172.21.15.27:0/899048993 wait complete. 2023-12-19T18:19:55.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.073+0000 7f038f59e640 1 Processor -- start 2023-12-19T18:19:55.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.073+0000 7f038f59e640 1 -- start start 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f038f59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03900b49d0 0x7f03900caa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f038f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03900b4db0 0x7f0390149150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f038f59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03900b89b0 0x7f03900c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f038f59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0390149690 con 0x7f03900b4db0 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f038f59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03900ccd50 con 0x7f03900b49d0 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f038f59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03900cd030 con 0x7f03900b89b0 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f0387fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03900b4db0 0x7f0390149150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f0387fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03900b4db0 0x7f0390149150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58298/0 (socket says 172.21.15.27:58298) 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f0387fff640 1 -- 172.21.15.27:0/1082111706 learned_addr learned my addr 172.21.15.27:0/1082111706 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f038e59c640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03900b49d0 0x7f03900caa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f038ed9d640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03900b89b0 0x7f03900c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f0387fff640 1 -- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03900b89b0 msgr2=0x7f03900c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f0387fff640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03900b89b0 0x7f03900c9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:55.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f0387fff640 1 -- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03900b49d0 msgr2=0x7f03900caa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f0387fff640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03900b49d0 0x7f03900caa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.074+0000 7f0387fff640 1 -- 172.21.15.27:0/1082111706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0378009400 con 0x7f03900b4db0 2023-12-19T18:19:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.075+0000 7f0387fff640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03900b4db0 0x7f0390149150 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0378009890 tx=0x7f03780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.075+0000 7f0386ffd640 1 -- 172.21.15.27:0/1082111706 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037803e070 con 0x7f03900b4db0 2023-12-19T18:19:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.075+0000 7f0386ffd640 1 -- 172.21.15.27:0/1082111706 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0378030e10 con 0x7f03900b4db0 2023-12-19T18:19:55.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.075+0000 7f0386ffd640 1 -- 172.21.15.27:0/1082111706 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0378039c30 con 0x7f03900b4db0 2023-12-19T18:19:55.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.076+0000 7f038f59e640 1 -- 172.21.15.27:0/1082111706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03900cd310 con 0x7f03900b4db0 2023-12-19T18:19:55.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.076+0000 7f038f59e640 1 -- 172.21.15.27:0/1082111706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039015d310 con 0x7f03900b4db0 2023-12-19T18:19:55.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.077+0000 7f038f59e640 1 -- 172.21.15.27:0/1082111706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039004e840 con 0x7f03900b4db0 2023-12-19T18:19:55.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.078+0000 7f0386ffd640 1 -- 172.21.15.27:0/1082111706 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f037804b400 con 0x7f03900b4db0 2023-12-19T18:19:55.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.079+0000 7f0386ffd640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f037006a430 0x7f037006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:55.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.079+0000 7f0386ffd640 1 -- 172.21.15.27:0/1082111706 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f03780aa680 con 0x7f03900b4db0 2023-12-19T18:19:55.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.079+0000 7f038e59c640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f037006a430 0x7f037006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:55.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.080+0000 7f038e59c640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f037006a430 0x7f037006c8f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0380009b30 tx=0x7f0380009110 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:55.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.083+0000 7f0386ffd640 1 -- 172.21.15.27:0/1082111706 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0378076070 con 0x7f03900b4db0 2023-12-19T18:19:55.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:55.294+0000 7f038f59e640 1 -- 172.21.15.27:0/1082111706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f03900c92a0 con 0x7f03900b4db0 2023-12-19T18:19:56.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.069+0000 7f0386ffd640 1 -- 172.21.15.27:0/1082111706 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v92) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0378079690 con 0x7f03900b4db0 2023-12-19T18:19:56.070 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2023-12-19T18:19:56.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.073+0000 7f038f59e640 1 -- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f037006a430 msgr2=0x7f037006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:56.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.073+0000 7f038f59e640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f037006a430 0x7f037006c8f0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f0380009b30 tx=0x7f0380009110 comp rx=0 tx=0).stop 2023-12-19T18:19:56.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.073+0000 7f038f59e640 1 -- 172.21.15.27:0/1082111706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03900b4db0 msgr2=0x7f0390149150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:56.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.073+0000 7f038f59e640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03900b4db0 0x7f0390149150 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0378009890 tx=0x7f03780046c0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.073+0000 7f038f59e640 1 -- 172.21.15.27:0/1082111706 shutdown_connections 2023-12-19T18:19:56.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.073+0000 7f038f59e640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f037006a430 0x7f037006c8f0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.073+0000 7f038f59e640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f03900b89b0 0x7f03900c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.073+0000 7f038f59e640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03900b4db0 0x7f0390149150 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.073+0000 7f038f59e640 1 --2- 172.21.15.27:0/1082111706 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03900b49d0 0x7f03900caa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.073+0000 7f038f59e640 1 -- 172.21.15.27:0/1082111706 >> 172.21.15.27:0/1082111706 conn(0x7f03900cf760 msgr2=0x7f03900b5f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:56.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.073+0000 7f038f59e640 1 -- 172.21.15.27:0/1082111706 shutdown_connections 2023-12-19T18:19:56.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.074+0000 7f038f59e640 1 -- 172.21.15.27:0/1082111706 wait complete. 2023-12-19T18:19:56.097 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:19:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.334+0000 7f343dcf3640 1 Processor -- start 2023-12-19T18:19:56.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.334+0000 7f343dcf3640 1 -- start start 2023-12-19T18:19:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.336+0000 7f343dcf3640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f343815ac50 0x7f3438070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.336+0000 7f343dcf3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f34380aef20 0x7f343806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:56.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.336+0000 7f343dcf3640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34380adf10 con 0x7f34380af360 2023-12-19T18:19:56.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.336+0000 7f343dcf3640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34380adac0 con 0x7f34380aef20 2023-12-19T18:19:56.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.336+0000 7f343dcf3640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34380ae160 con 0x7f343815ac50 2023-12-19T18:19:56.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.336+0000 7f342ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f34380aef20 0x7f343806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:56.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.336+0000 7f342ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f34380aef20 0x7f343806b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58308/0 (socket says 172.21.15.27:58308) 2023-12-19T18:19:56.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.336+0000 7f342ffff640 1 -- 172.21.15.27:0/2857384971 learned_addr learned my addr 172.21.15.27:0/2857384971 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:56.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.337+0000 7f343ccf1640 1 --2- 172.21.15.27:0/2857384971 >> v2:172.21.15.192:3300/0 conn(0x7f343815ac50 0x7f3438070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:56.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.337+0000 7f342ffff640 1 -- 172.21.15.27:0/2857384971 >> v2:172.21.15.192:3300/0 conn(0x7f343815ac50 msgr2=0x7f3438070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:56.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.337+0000 7f342ffff640 1 --2- 172.21.15.27:0/2857384971 >> v2:172.21.15.192:3300/0 conn(0x7f343815ac50 0x7f3438070fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.337+0000 7f342ffff640 1 -- 172.21.15.27:0/2857384971 >> v1:172.21.15.192:6789/0 conn(0x7f34380af360 legacy=0x7f343806b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:56.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.337+0000 7f342ffff640 1 -- 172.21.15.27:0/2857384971 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34380ae440 con 0x7f34380aef20 2023-12-19T18:19:56.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.338+0000 7f342ffff640 1 --2- 172.21.15.27:0/2857384971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34380aef20 0x7f343806b310 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f343806cfb0 tx=0x7f342802b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf38c26cd1e8cbd8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:56.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.339+0000 7f342f7fe640 1 -- 172.21.15.27:0/2857384971 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428031750 con 0x7f34380aef20 2023-12-19T18:19:56.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.339+0000 7f342f7fe640 1 -- 172.21.15.27:0/2857384971 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34280318b0 con 0x7f34380aef20 2023-12-19T18:19:56.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.339+0000 7f342f7fe640 1 -- 172.21.15.27:0/2857384971 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3428031c40 con 0x7f34380aef20 2023-12-19T18:19:56.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.339+0000 7f343dcf3640 1 -- 172.21.15.27:0/2857384971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34380aef20 msgr2=0x7f343806b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.339+0000 7f343dcf3640 1 --2- 172.21.15.27:0/2857384971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34380aef20 0x7f343806b310 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f343806cfb0 tx=0x7f342802b7f0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.339+0000 7f343dcf3640 1 -- 172.21.15.27:0/2857384971 shutdown_connections 2023-12-19T18:19:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.339+0000 7f343dcf3640 1 --2- 172.21.15.27:0/2857384971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34380aef20 0x7f343806b310 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.339+0000 7f343dcf3640 1 --2- 172.21.15.27:0/2857384971 >> v2:172.21.15.192:3300/0 conn(0x7f343815ac50 0x7f3438070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.339+0000 7f343dcf3640 1 -- 172.21.15.27:0/2857384971 >> 172.21.15.27:0/2857384971 conn(0x7f343806a7d0 msgr2=0x7f34380ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.340+0000 7f343dcf3640 1 -- 172.21.15.27:0/2857384971 shutdown_connections 2023-12-19T18:19:56.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.340+0000 7f343dcf3640 1 -- 172.21.15.27:0/2857384971 wait complete. 2023-12-19T18:19:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.341+0000 7f343dcf3640 1 Processor -- start 2023-12-19T18:19:56.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.342+0000 7f343dcf3640 1 -- start start 2023-12-19T18:19:56.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.342+0000 7f343dcf3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34380aef20 0x7f343806f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:56.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.342+0000 7f343dcf3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34380af360 0x7f343814a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.342+0000 7f343dcf3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f343815ac50 0x7f343814a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.342+0000 7f343dcf3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34380b0a90 con 0x7f34380aef20 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.342+0000 7f343dcf3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34380b1330 con 0x7f343815ac50 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.342+0000 7f343dcf3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343815b4d0 con 0x7f34380af360 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.343+0000 7f343ccf1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34380aef20 0x7f343806f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.343+0000 7f343ccf1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34380aef20 0x7f343806f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58322/0 (socket says 172.21.15.27:58322) 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.343+0000 7f343ccf1640 1 -- 172.21.15.27:0/2965427342 learned_addr learned my addr 172.21.15.27:0/2965427342 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.343+0000 7f343ccf1640 1 -- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34380af360 msgr2=0x7f343814a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.343+0000 7f342ffff640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34380af360 0x7f343814a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.343+0000 7f343d4f2640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f343815ac50 0x7f343814a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.343+0000 7f343ccf1640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34380af360 0x7f343814a1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.343+0000 7f343ccf1640 1 -- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f343815ac50 msgr2=0x7f343814a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.343+0000 7f343ccf1640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f343815ac50 0x7f343814a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.343+0000 7f343ccf1640 1 -- 172.21.15.27:0/2965427342 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34280094d0 con 0x7f34380aef20 2023-12-19T18:19:56.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.343+0000 7f343d4f2640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f343815ac50 0x7f343814a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:56.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.344+0000 7f343ccf1640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34380aef20 0x7f343806f8c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3434007f20 tx=0x7f343400e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:56.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.344+0000 7f342d7fa640 1 -- 172.21.15.27:0/2965427342 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434014620 con 0x7f34380aef20 2023-12-19T18:19:56.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.344+0000 7f342d7fa640 1 -- 172.21.15.27:0/2965427342 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3434014780 con 0x7f34380aef20 2023-12-19T18:19:56.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.344+0000 7f342d7fa640 1 -- 172.21.15.27:0/2965427342 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3434014b30 con 0x7f34380aef20 2023-12-19T18:19:56.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.344+0000 7f343dcf3640 1 -- 172.21.15.27:0/2965427342 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34380b0ce0 con 0x7f34380aef20 2023-12-19T18:19:56.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.344+0000 7f343dcf3640 1 -- 172.21.15.27:0/2965427342 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34381f24b0 con 0x7f34380aef20 2023-12-19T18:19:56.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.345+0000 7f343dcf3640 1 -- 172.21.15.27:0/2965427342 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343804e840 con 0x7f34380aef20 2023-12-19T18:19:56.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.346+0000 7f342d7fa640 1 -- 172.21.15.27:0/2965427342 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3434014c90 con 0x7f34380aef20 2023-12-19T18:19:56.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.347+0000 7f342d7fa640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f342406a430 0x7f342406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:56.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.347+0000 7f342d7fa640 1 -- 172.21.15.27:0/2965427342 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3434088ff0 con 0x7f34380aef20 2023-12-19T18:19:56.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.348+0000 7f342ffff640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f342406a430 0x7f342406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:56.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.349+0000 7f342ffff640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f342406a430 0x7f342406c8f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3428035000 tx=0x7f3428008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:56.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.352+0000 7f342d7fa640 1 -- 172.21.15.27:0/2965427342 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f34340bbca0 con 0x7f34380aef20 2023-12-19T18:19:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.615+0000 7f343dcf3640 1 -- 172.21.15.27:0/2965427342 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f343806f430 con 0x7f34380aef20 2023-12-19T18:19:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.616+0000 7f342d7fa640 1 -- 172.21.15.27:0/2965427342 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f343405e020 con 0x7f34380aef20 2023-12-19T18:19:56.616 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFl+4ruHBAAXwbpmT43SmlvNbuZj5e+bQ==", 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.617 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmZyUBhAAG3kZkXvJIo7N9f1j+DPCSQ==", 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlChNOMBAAX4GV3iIGkSQdAsYu9oCPDw==", 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:19:56.618 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlTmFXFhAABA0NMUxKBV1ExAhYc5RFbQ==", 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.619 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFluleFIBAAs1goFVMOX4CbjwFhg0KoBg==", 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFlInW3MxAAI4hY1GBxXYXDswKKrEyDTg==", 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2023-12-19T18:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFlAWZ5CRAA82/2r9/sT0L9VlGHbXksQw==", 2023-12-19T18:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2023-12-19T18:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFl1IvaGRAAqVacDDPLx+NqBOGp6MeX2w==", 2023-12-19T18:19:56.621 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlr0BBHRAA+7gfs/RG6DtABnYPg059Tw==", 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlXZbELRAAKD0qOkhk7skfc4xcAC8Lhg==", 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlF07MARAA2TJJ+5C1HIU+SVhqmKaVHA==", 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:19:56.623 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlISPGEhAAdFIRRkXlLyOx/oMZ4MgL9Q==", 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2023-12-19T18:19:56.624 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmNM9JRAARgS20O8zpm6H70WDv+lgGw==", 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlWoZ8AhAADWvKVzeDhDGHtYNUFIOWsA==", 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:19:56.625 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFllWFVMhAAD9ebDOppHTODVQjg6ppP3g==", 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlwbIQDRAAt/h55zBNCEId0vkYSUQ3TQ==", 2023-12-19T18:19:56.626 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB53YFloCb6CxAAmZQyz5XOEqDOOiROJHLzxQ==", 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFltBhxDxAAiAS9CYPY86IMmJ7NGgA2lw==", 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlUnAhBBAAi++W9XK3o3Y/185hB3LuIg==", 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:19:56.628 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:19:56.629 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:19:56.629 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.629 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:56.629 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:19:56.629 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:19:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.619+0000 7f343dcf3640 1 -- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f342406a430 msgr2=0x7f342406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.619+0000 7f343dcf3640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f342406a430 0x7f342406c8f0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f3428035000 tx=0x7f3428008030 comp rx=0 tx=0).stop 2023-12-19T18:19:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.619+0000 7f343dcf3640 1 -- 172.21.15.27:0/2965427342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34380aef20 msgr2=0x7f343806f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.619+0000 7f343dcf3640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34380aef20 0x7f343806f8c0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f3434007f20 tx=0x7f343400e570 comp rx=0 tx=0).stop 2023-12-19T18:19:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.619+0000 7f343dcf3640 1 -- 172.21.15.27:0/2965427342 shutdown_connections 2023-12-19T18:19:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.619+0000 7f343dcf3640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f342406a430 0x7f342406c8f0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.619+0000 7f343dcf3640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f343815ac50 0x7f343814a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.619+0000 7f343dcf3640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34380af360 0x7f343814a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.619+0000 7f343dcf3640 1 --2- 172.21.15.27:0/2965427342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34380aef20 0x7f343806f8c0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.619+0000 7f343dcf3640 1 -- 172.21.15.27:0/2965427342 >> 172.21.15.27:0/2965427342 conn(0x7f343806a7d0 msgr2=0x7f34380b3b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:56.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.619+0000 7f343dcf3640 1 -- 172.21.15.27:0/2965427342 shutdown_connections 2023-12-19T18:19:56.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.619+0000 7f343dcf3640 1 -- 172.21.15.27:0/2965427342 wait complete. 2023-12-19T18:19:56.640 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:19:56.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.875+0000 7faf29f2e640 1 Processor -- start 2023-12-19T18:19:56.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.875+0000 7faf29f2e640 1 -- start start 2023-12-19T18:19:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.877+0000 7faf29f2e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faf24159c40 0x7faf2406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.877+0000 7faf29f2e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf2415a020 0x7faf2406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.877+0000 7faf29f2e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2415b180 con 0x7faf2406c880 2023-12-19T18:19:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.877+0000 7faf29f2e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf240ca850 con 0x7faf24159c40 2023-12-19T18:19:56.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.877+0000 7faf29f2e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2415a570 con 0x7faf2415a020 2023-12-19T18:19:56.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.877+0000 7faf237fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faf24159c40 0x7faf2406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:56.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.877+0000 7faf237fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faf24159c40 0x7faf2406b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58336/0 (socket says 172.21.15.27:58336) 2023-12-19T18:19:56.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.877+0000 7faf1bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faf2406c880 0x7faf240cd800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40726/0 (socket says 172.21.15.27:40726) 2023-12-19T18:19:56.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.877+0000 7faf237fe640 1 -- 172.21.15.27:0/1823922078 learned_addr learned my addr 172.21.15.27:0/1823922078 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:56.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.878+0000 7faf237fe640 1 -- 172.21.15.27:0/1823922078 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf2415a020 msgr2=0x7faf2406ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:56.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.878+0000 7faf23fff640 1 --2- 172.21.15.27:0/1823922078 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf2415a020 0x7faf2406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:56.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.878+0000 7faf22ffd640 1 -- 172.21.15.27:0/1823922078 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2116676313 0 0) 0x7faf2415b180 con 0x7faf2406c880 2023-12-19T18:19:56.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.878+0000 7faf237fe640 1 --2- 172.21.15.27:0/1823922078 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf2415a020 0x7faf2406ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.878+0000 7faf237fe640 1 -- 172.21.15.27:0/1823922078 >> v1:172.21.15.27:6789/0 conn(0x7faf2406c880 legacy=0x7faf240cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:56.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.878+0000 7faf237fe640 1 -- 172.21.15.27:0/1823922078 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2415a850 con 0x7faf24159c40 2023-12-19T18:19:56.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.879+0000 7faf237fe640 1 --2- 172.21.15.27:0/1823922078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf24159c40 0x7faf2406b4e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7faf14009870 tx=0x7faf1402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=808743d3ded12504 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:56.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.879+0000 7faf22ffd640 1 -- 172.21.15.27:0/1823922078 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf14030840 con 0x7faf24159c40 2023-12-19T18:19:56.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.879+0000 7faf22ffd640 1 -- 172.21.15.27:0/1823922078 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf140309a0 con 0x7faf24159c40 2023-12-19T18:19:56.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.879+0000 7faf22ffd640 1 -- 172.21.15.27:0/1823922078 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf14030d30 con 0x7faf24159c40 2023-12-19T18:19:56.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.880+0000 7faf29f2e640 1 -- 172.21.15.27:0/1823922078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf24159c40 msgr2=0x7faf2406b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:56.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.880+0000 7faf29f2e640 1 --2- 172.21.15.27:0/1823922078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf24159c40 0x7faf2406b4e0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7faf14009870 tx=0x7faf1402b980 comp rx=0 tx=0).stop 2023-12-19T18:19:56.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.880+0000 7faf29f2e640 1 -- 172.21.15.27:0/1823922078 shutdown_connections 2023-12-19T18:19:56.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.880+0000 7faf29f2e640 1 --2- 172.21.15.27:0/1823922078 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf2415a020 0x7faf2406ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.880+0000 7faf29f2e640 1 --2- 172.21.15.27:0/1823922078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf24159c40 0x7faf2406b4e0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.880+0000 7faf29f2e640 1 -- 172.21.15.27:0/1823922078 >> 172.21.15.27:0/1823922078 conn(0x7faf2406a330 msgr2=0x7faf240b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:56.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.881+0000 7faf29f2e640 1 -- 172.21.15.27:0/1823922078 shutdown_connections 2023-12-19T18:19:56.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.881+0000 7faf29f2e640 1 -- 172.21.15.27:0/1823922078 wait complete. 2023-12-19T18:19:56.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.883+0000 7faf29f2e640 1 Processor -- start 2023-12-19T18:19:56.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.883+0000 7faf29f2e640 1 -- start start 2023-12-19T18:19:56.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.884+0000 7faf29f2e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf2406c880 0x7faf240cd1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.884+0000 7faf29f2e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf24159c40 0x7faf240cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.884+0000 7faf29f2e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf2415a020 0x7faf240cbb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.885+0000 7faf29f2e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf240ccdb0 con 0x7faf24159c40 2023-12-19T18:19:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.885+0000 7faf29f2e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf24159120 con 0x7faf2406c880 2023-12-19T18:19:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.885+0000 7faf29f2e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf240cd710 con 0x7faf2415a020 2023-12-19T18:19:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.885+0000 7faf1bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf24159c40 0x7faf240cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.885+0000 7faf1bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf24159c40 0x7faf240cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58344/0 (socket says 172.21.15.27:58344) 2023-12-19T18:19:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.885+0000 7faf1bfff640 1 -- 172.21.15.27:0/3924204284 learned_addr learned my addr 172.21.15.27:0/3924204284 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:56.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.885+0000 7faf237fe640 1 --2- 172.21.15.27:0/3924204284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf2406c880 0x7faf240cd1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:56.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.885+0000 7faf1bfff640 1 -- 172.21.15.27:0/3924204284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf2415a020 msgr2=0x7faf240cbb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:56.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.885+0000 7faf1bfff640 1 --2- 172.21.15.27:0/3924204284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf2415a020 0x7faf240cbb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.885+0000 7faf1bfff640 1 -- 172.21.15.27:0/3924204284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf2406c880 msgr2=0x7faf240cd1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:56.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.885+0000 7faf1bfff640 1 --2- 172.21.15.27:0/3924204284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf2406c880 0x7faf240cd1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:56.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.885+0000 7faf1bfff640 1 -- 172.21.15.27:0/3924204284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf140094d0 con 0x7faf24159c40 2023-12-19T18:19:56.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.886+0000 7faf1bfff640 1 --2- 172.21.15.27:0/3924204284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf24159c40 0x7faf240cb5c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00c4b0 tx=0x7faf0c00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:56.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.886+0000 7faf20ff9640 1 -- 172.21.15.27:0/3924204284 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c0074e0 con 0x7faf24159c40 2023-12-19T18:19:56.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.886+0000 7faf20ff9640 1 -- 172.21.15.27:0/3924204284 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf0c007660 con 0x7faf24159c40 2023-12-19T18:19:56.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.886+0000 7faf20ff9640 1 -- 172.21.15.27:0/3924204284 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c004a20 con 0x7faf24159c40 2023-12-19T18:19:56.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.886+0000 7faf29f2e640 1 -- 172.21.15.27:0/3924204284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf240cc130 con 0x7faf24159c40 2023-12-19T18:19:56.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.886+0000 7faf29f2e640 1 -- 172.21.15.27:0/3924204284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf240cc6f0 con 0x7faf24159c40 2023-12-19T18:19:56.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.887+0000 7faf29f2e640 1 -- 172.21.15.27:0/3924204284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf240ccef0 con 0x7faf24159c40 2023-12-19T18:19:56.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.888+0000 7faf20ff9640 1 -- 172.21.15.27:0/3924204284 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faf0c004b80 con 0x7faf24159c40 2023-12-19T18:19:56.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.889+0000 7faf20ff9640 1 --2- 172.21.15.27:0/3924204284 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf0406a430 0x7faf0406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:56.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.889+0000 7faf20ff9640 1 -- 172.21.15.27:0/3924204284 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faf0c0877e0 con 0x7faf24159c40 2023-12-19T18:19:56.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.890+0000 7faf237fe640 1 --2- 172.21.15.27:0/3924204284 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf0406a430 0x7faf0406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:56.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.891+0000 7faf237fe640 1 --2- 172.21.15.27:0/3924204284 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf0406a430 0x7faf0406c8f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faf14036000 tx=0x7faf14008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:56.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:56.894+0000 7faf20ff9640 1 -- 172.21.15.27:0/3924204284 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7faf240ccef0 con 0x7faf24159c40 2023-12-19T18:19:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.090+0000 7faf29f2e640 1 -- 172.21.15.27:0/3924204284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf240ccef0 con 0x7faf24159c40 2023-12-19T18:19:57.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.091+0000 7faf20ff9640 1 -- 172.21.15.27:0/3924204284 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+12389 (secure 0 0 0) 0x7faf240ccef0 con 0x7faf24159c40 2023-12-19T18:19:57.092 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:19:57.092 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":92,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:19:56.059896+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:19:57.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.094+0000 7faf29f2e640 1 -- 172.21.15.27:0/3924204284 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf0406a430 msgr2=0x7faf0406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:57.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.094+0000 7faf29f2e640 1 --2- 172.21.15.27:0/3924204284 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf0406a430 0x7faf0406c8f0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faf14036000 tx=0x7faf14008030 comp rx=0 tx=0).stop 2023-12-19T18:19:57.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.094+0000 7faf29f2e640 1 -- 172.21.15.27:0/3924204284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf24159c40 msgr2=0x7faf240cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:57.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.094+0000 7faf29f2e640 1 --2- 172.21.15.27:0/3924204284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf24159c40 0x7faf240cb5c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00c4b0 tx=0x7faf0c00c970 comp rx=0 tx=0).stop 2023-12-19T18:19:57.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.094+0000 7faf29f2e640 1 -- 172.21.15.27:0/3924204284 shutdown_connections 2023-12-19T18:19:57.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.094+0000 7faf29f2e640 1 --2- 172.21.15.27:0/3924204284 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf0406a430 0x7faf0406c8f0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.094+0000 7faf29f2e640 1 --2- 172.21.15.27:0/3924204284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf2415a020 0x7faf240cbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.094+0000 7faf29f2e640 1 --2- 172.21.15.27:0/3924204284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf24159c40 0x7faf240cb5c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.094+0000 7faf29f2e640 1 --2- 172.21.15.27:0/3924204284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf2406c880 0x7faf240cd1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.094+0000 7faf29f2e640 1 -- 172.21.15.27:0/3924204284 >> 172.21.15.27:0/3924204284 conn(0x7faf2406a330 msgr2=0x7faf240b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:57.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.094+0000 7faf29f2e640 1 -- 172.21.15.27:0/3924204284 shutdown_connections 2023-12-19T18:19:57.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.094+0000 7faf29f2e640 1 -- 172.21.15.27:0/3924204284 wait complete. 2023-12-19T18:19:57.115 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:19:57.115 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:19:57.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.352+0000 7f6242ab4640 1 Processor -- start 2023-12-19T18:19:57.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.353+0000 7f6242ab4640 1 -- start start 2023-12-19T18:19:57.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.354+0000 7f6242ab4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f623c151c90 0x7f623c06b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.354+0000 7f6242ab4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f623c0cf550 0x7f623c15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.354+0000 7f6242ab4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f623c15b3e0 con 0x7f623c15e9d0 2023-12-19T18:19:57.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.354+0000 7f6242ab4640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f623c15eed0 con 0x7f623c151c90 2023-12-19T18:19:57.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.354+0000 7f6242ab4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c15f1b0 con 0x7f623c0cf550 2023-12-19T18:19:57.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.355+0000 7f62422b3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f623c0cf550 0x7f623c15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:57.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.355+0000 7f6241ab2640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f623c151c90 0x7f623c06b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:57.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.355+0000 7f6241ab2640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f623c151c90 0x7f623c06b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:54902/0 (socket says 172.21.15.27:54902) 2023-12-19T18:19:57.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.355+0000 7f6241ab2640 1 -- 172.21.15.27:0/312482819 learned_addr learned my addr 172.21.15.27:0/312482819 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:57.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.355+0000 7f62422b3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f623c0cf550 0x7f623c15b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:36216/0 (socket says 172.21.15.27:36216) 2023-12-19T18:19:57.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.356+0000 7f6241ab2640 1 -- 172.21.15.27:0/312482819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f623c0cf550 msgr2=0x7f623c15b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:57.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.356+0000 7f6241ab2640 1 --2- 172.21.15.27:0/312482819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f623c0cf550 0x7f623c15b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.356+0000 7f6241ab2640 1 -- 172.21.15.27:0/312482819 >> v1:172.21.15.192:6789/0 conn(0x7f623c15e9d0 legacy=0x7f623c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:57.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.356+0000 7f6241ab2640 1 -- 172.21.15.27:0/312482819 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c15f490 con 0x7f623c151c90 2023-12-19T18:19:57.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.357+0000 7f6241ab2640 1 --2- 172.21.15.27:0/312482819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f623c151c90 0x7f623c06b4d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f622c0098a0 tx=0x7f622c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5b8e3acdbf35ce0f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:57.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.357+0000 7f6240ab0640 1 -- 172.21.15.27:0/312482819 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c031640 con 0x7f623c151c90 2023-12-19T18:19:57.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.358+0000 7f6240ab0640 1 -- 172.21.15.27:0/312482819 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f622c0317a0 con 0x7f623c151c90 2023-12-19T18:19:57.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.358+0000 7f6240ab0640 1 -- 172.21.15.27:0/312482819 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f622c031b30 con 0x7f623c151c90 2023-12-19T18:19:57.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.358+0000 7f6242ab4640 1 -- 172.21.15.27:0/312482819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f623c151c90 msgr2=0x7f623c06b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:57.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.358+0000 7f6242ab4640 1 --2- 172.21.15.27:0/312482819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f623c151c90 0x7f623c06b4d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f622c0098a0 tx=0x7f622c02b8f0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.358+0000 7f6242ab4640 1 -- 172.21.15.27:0/312482819 shutdown_connections 2023-12-19T18:19:57.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.358+0000 7f6242ab4640 1 --2- 172.21.15.27:0/312482819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f623c0cf550 0x7f623c15b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.358+0000 7f6242ab4640 1 --2- 172.21.15.27:0/312482819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f623c151c90 0x7f623c06b4d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.358+0000 7f6242ab4640 1 -- 172.21.15.27:0/312482819 >> 172.21.15.27:0/312482819 conn(0x7f623c06a060 msgr2=0x7f623c06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:57.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.358+0000 7f6242ab4640 1 -- 172.21.15.27:0/312482819 shutdown_connections 2023-12-19T18:19:57.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.358+0000 7f6242ab4640 1 -- 172.21.15.27:0/312482819 wait complete. 2023-12-19T18:19:57.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.360+0000 7f6242ab4640 1 Processor -- start 2023-12-19T18:19:57.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.360+0000 7f6242ab4640 1 -- start start 2023-12-19T18:19:57.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.361+0000 7f6242ab4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f623c0cf550 0x7f623c0ace30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.361+0000 7f6242ab4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f623c151c90 0x7f623c154e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.361+0000 7f6242ab4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f623c15e9d0 0x7f623c1542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.361+0000 7f6242ab4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c0ad370 con 0x7f623c151c90 2023-12-19T18:19:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.361+0000 7f6242ab4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c153240 con 0x7f623c0cf550 2023-12-19T18:19:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.361+0000 7f6242ab4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c153520 con 0x7f623c15e9d0 2023-12-19T18:19:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.362+0000 7f62412b1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f623c151c90 0x7f623c154e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.362+0000 7f62412b1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f623c151c90 0x7f623c154e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58352/0 (socket says 172.21.15.27:58352) 2023-12-19T18:19:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.362+0000 7f62412b1640 1 -- 172.21.15.27:0/3478271981 learned_addr learned my addr 172.21.15.27:0/3478271981 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.362+0000 7f6241ab2640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f623c0cf550 0x7f623c0ace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.362+0000 7f62422b3640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f623c15e9d0 0x7f623c1542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.362+0000 7f62412b1640 1 -- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f623c15e9d0 msgr2=0x7f623c1542e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.362+0000 7f62412b1640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f623c15e9d0 0x7f623c1542e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.362+0000 7f62412b1640 1 -- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f623c0cf550 msgr2=0x7f623c0ace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:57.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.362+0000 7f62412b1640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f623c0cf550 0x7f623c0ace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.362+0000 7f62412b1640 1 -- 172.21.15.27:0/3478271981 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f622c0094d0 con 0x7f623c151c90 2023-12-19T18:19:57.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.363+0000 7f62412b1640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f623c151c90 0x7f623c154e80 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f623000c3c0 tx=0x7f623000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:57.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.363+0000 7f623a7fc640 1 -- 172.21.15.27:0/3478271981 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62300052d0 con 0x7f623c151c90 2023-12-19T18:19:57.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.363+0000 7f623a7fc640 1 -- 172.21.15.27:0/3478271981 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6230005430 con 0x7f623c151c90 2023-12-19T18:19:57.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.363+0000 7f623a7fc640 1 -- 172.21.15.27:0/3478271981 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6230003830 con 0x7f623c151c90 2023-12-19T18:19:57.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.363+0000 7f6242ab4640 1 -- 172.21.15.27:0/3478271981 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f623c153800 con 0x7f623c151c90 2023-12-19T18:19:57.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.363+0000 7f6242ab4640 1 -- 172.21.15.27:0/3478271981 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f623c153dc0 con 0x7f623c151c90 2023-12-19T18:19:57.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.364+0000 7f6242ab4640 1 -- 172.21.15.27:0/3478271981 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623c04e840 con 0x7f623c151c90 2023-12-19T18:19:57.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.365+0000 7f623a7fc640 1 -- 172.21.15.27:0/3478271981 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6230003990 con 0x7f623c151c90 2023-12-19T18:19:57.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.366+0000 7f623a7fc640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f622806a430 0x7f622806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.366+0000 7f623a7fc640 1 -- 172.21.15.27:0/3478271981 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6230087b50 con 0x7f623c151c90 2023-12-19T18:19:57.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.367+0000 7f6241ab2640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f622806a430 0x7f622806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:57.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.368+0000 7f6241ab2640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f622806a430 0x7f622806c8f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f622c036000 tx=0x7f622c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:57.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.371+0000 7f623a7fc640 1 -- 172.21.15.27:0/3478271981 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f62300533f0 con 0x7f623c151c90 2023-12-19T18:19:57.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.565+0000 7f6242ab4640 1 -- 172.21.15.27:0/3478271981 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f623c0cbfc0 con 0x7f623c151c90 2023-12-19T18:19:57.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.566+0000 7f623a7fc640 1 -- 172.21.15.27:0/3478271981 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6230056a10 con 0x7f623c151c90 2023-12-19T18:19:57.566 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:19:57.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.569+0000 7f6242ab4640 1 -- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f622806a430 msgr2=0x7f622806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:57.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.569+0000 7f6242ab4640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f622806a430 0x7f622806c8f0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f622c036000 tx=0x7f622c03c040 comp rx=0 tx=0).stop 2023-12-19T18:19:57.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.569+0000 7f6242ab4640 1 -- 172.21.15.27:0/3478271981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f623c151c90 msgr2=0x7f623c154e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:57.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.569+0000 7f6242ab4640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f623c151c90 0x7f623c154e80 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f623000c3c0 tx=0x7f623000c880 comp rx=0 tx=0).stop 2023-12-19T18:19:57.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.569+0000 7f6242ab4640 1 -- 172.21.15.27:0/3478271981 shutdown_connections 2023-12-19T18:19:57.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.569+0000 7f6242ab4640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f622806a430 0x7f622806c8f0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.569+0000 7f6242ab4640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f623c15e9d0 0x7f623c1542e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.569+0000 7f6242ab4640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f623c151c90 0x7f623c154e80 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.569+0000 7f6242ab4640 1 --2- 172.21.15.27:0/3478271981 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f623c0cf550 0x7f623c0ace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.569+0000 7f6242ab4640 1 -- 172.21.15.27:0/3478271981 >> 172.21.15.27:0/3478271981 conn(0x7f623c06a060 msgr2=0x7f623c151360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:57.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.569+0000 7f6242ab4640 1 -- 172.21.15.27:0/3478271981 shutdown_connections 2023-12-19T18:19:57.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.569+0000 7f6242ab4640 1 -- 172.21.15.27:0/3478271981 wait complete. 2023-12-19T18:19:57.589 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:19:57.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.826+0000 7fa38f300640 1 Processor -- start 2023-12-19T18:19:57.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.826+0000 7fa38f300640 1 -- start start 2023-12-19T18:19:57.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.828+0000 7fa38f300640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa388165c60 0x7fa388168050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.828+0000 7fa38f300640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3880ad8e0 con 0x7fa38815d1d0 2023-12-19T18:19:57.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.828+0000 7fa38f300640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3880cdb30 con 0x7fa388071140 2023-12-19T18:19:57.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.828+0000 7fa38f300640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa388070150 con 0x7fa388165c60 2023-12-19T18:19:57.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.828+0000 7fa38d09c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa38815d1d0 0x7fa3880ccd10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40734/0 (socket says 172.21.15.27:40734) 2023-12-19T18:19:57.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.828+0000 7fa38d09c640 1 -- 172.21.15.27:0/364084669 learned_addr learned my addr 172.21.15.27:0/364084669 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:57.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa38d89d640 1 --2- 172.21.15.27:0/364084669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa388165c60 0x7fa388168050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:57.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3430737179 0 0) 0x7fa3880ad8e0 con 0x7fa38815d1d0 2023-12-19T18:19:57.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa374003520 con 0x7fa38815d1d0 2023-12-19T18:19:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1347557485 0 0) 0x7fa374003520 con 0x7fa38815d1d0 2023-12-19T18:19:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa388165c60 msgr2=0x7fa388168050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 --2- 172.21.15.27:0/364084669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa388165c60 0x7fa388168050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 >> v1:172.21.15.192:6789/0 conn(0x7fa388071140 legacy=0x7fa3880cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3880ce7c0 con 0x7fa38815d1d0 2023-12-19T18:19:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fa384001690 con 0x7fa38815d1d0 2023-12-19T18:19:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 >> v1:172.21.15.27:6789/0 conn(0x7fa38815d1d0 legacy=0x7fa3880ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 --2- 172.21.15.27:0/364084669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa388165c60 0x7fa3740039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 --2- 172.21.15.27:0/364084669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa374003f70 0x7fa3740047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 --2- 172.21.15.27:0/364084669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa374004d00 0x7fa374005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa388070150 con 0x7fa374003f70 2023-12-19T18:19:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37400bb70 con 0x7fa374004d00 2023-12-19T18:19:57.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.829+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37400bd30 con 0x7fa388165c60 2023-12-19T18:19:57.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.830+0000 7fa38c89b640 1 --2- 172.21.15.27:0/364084669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa374003f70 0x7fa3740047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:57.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.830+0000 7fa38d89d640 1 --2- 172.21.15.27:0/364084669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa374004d00 0x7fa374005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:57.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.830+0000 7fa38c89b640 1 -- 172.21.15.27:0/364084669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa388165c60 msgr2=0x7fa3740039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:57.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.830+0000 7fa38c89b640 1 --2- 172.21.15.27:0/364084669 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa388165c60 0x7fa3740039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.830+0000 7fa38c89b640 1 -- 172.21.15.27:0/364084669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa374004d00 msgr2=0x7fa374005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:57.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.830+0000 7fa38c89b640 1 --2- 172.21.15.27:0/364084669 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa374004d00 0x7fa374005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.830+0000 7fa38c89b640 1 -- 172.21.15.27:0/364084669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa374003520 con 0x7fa374003f70 2023-12-19T18:19:57.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.831+0000 7fa38c89b640 1 --2- 172.21.15.27:0/364084669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa374003f70 0x7fa3740047c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa378001750 tx=0x7fa3740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7fa4a20227e122 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:57.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.831+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3780052d0 con 0x7fa374003f70 2023-12-19T18:19:57.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.832+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa37802e050 con 0x7fa374003f70 2023-12-19T18:19:57.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.832+0000 7fa37ffff640 1 -- 172.21.15.27:0/364084669 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa378003830 con 0x7fa374003f70 2023-12-19T18:19:57.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.832+0000 7fa38f300640 1 -- 172.21.15.27:0/364084669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa374003f70 msgr2=0x7fa3740047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:57.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.832+0000 7fa38f300640 1 --2- 172.21.15.27:0/364084669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa374003f70 0x7fa3740047c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fa378001750 tx=0x7fa3740057f0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.832+0000 7fa38d89d640 1 -- 172.21.15.27:0/364084669 reap_dead start 2023-12-19T18:19:57.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.832+0000 7fa38f300640 1 -- 172.21.15.27:0/364084669 shutdown_connections 2023-12-19T18:19:57.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.832+0000 7fa38f300640 1 -- 172.21.15.27:0/364084669 >> 172.21.15.27:0/364084669 conn(0x7fa3880ceb50 msgr2=0x7fa3880cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:57.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.832+0000 7fa38f300640 1 -- 172.21.15.27:0/364084669 shutdown_connections 2023-12-19T18:19:57.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.832+0000 7fa38f300640 1 -- 172.21.15.27:0/364084669 wait complete. 2023-12-19T18:19:57.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.834+0000 7fa38f300640 1 Processor -- start 2023-12-19T18:19:57.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.834+0000 7fa38f300640 1 -- start start 2023-12-19T18:19:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.835+0000 7fa38f300640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa374003f70 0x7fa38815c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.835+0000 7fa38f300640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa388155870 0x7fa3880c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.835+0000 7fa38f300640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa388154c90 0x7fa388155110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.835+0000 7fa38f300640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa388155650 con 0x7fa374003f70 2023-12-19T18:19:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.835+0000 7fa38f300640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa388158770 con 0x7fa388154c90 2023-12-19T18:19:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.835+0000 7fa38f300640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3881595d0 con 0x7fa388155870 2023-12-19T18:19:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.836+0000 7fa38d09c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa374003f70 0x7fa38815c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.836+0000 7fa38d09c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa374003f70 0x7fa38815c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58368/0 (socket says 172.21.15.27:58368) 2023-12-19T18:19:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.836+0000 7fa38d09c640 1 -- 172.21.15.27:0/2345879138 learned_addr learned my addr 172.21.15.27:0/2345879138 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.836+0000 7fa38d09c640 1 -- 172.21.15.27:0/2345879138 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa388155870 msgr2=0x7fa3880c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.836+0000 7fa38d09c640 1 --2- 172.21.15.27:0/2345879138 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa388155870 0x7fa3880c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.836+0000 7fa38d09c640 1 -- 172.21.15.27:0/2345879138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa388154c90 msgr2=0x7fa388155110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:57.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.836+0000 7fa38d09c640 1 --2- 172.21.15.27:0/2345879138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa388154c90 0x7fa388155110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:57.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.836+0000 7fa38d09c640 1 -- 172.21.15.27:0/2345879138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa388159870 con 0x7fa374003f70 2023-12-19T18:19:57.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.836+0000 7fa38d09c640 1 --2- 172.21.15.27:0/2345879138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa374003f70 0x7fa38815c5d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa38400c650 tx=0x7fa38400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:57.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.837+0000 7fa37dffb640 1 -- 172.21.15.27:0/2345879138 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3840072d0 con 0x7fa374003f70 2023-12-19T18:19:57.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.837+0000 7fa37dffb640 1 -- 172.21.15.27:0/2345879138 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa384007450 con 0x7fa374003f70 2023-12-19T18:19:57.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.837+0000 7fa38f300640 1 -- 172.21.15.27:0/2345879138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3880c2800 con 0x7fa374003f70 2023-12-19T18:19:57.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.837+0000 7fa37dffb640 1 -- 172.21.15.27:0/2345879138 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3840047e0 con 0x7fa374003f70 2023-12-19T18:19:57.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.837+0000 7fa38f300640 1 -- 172.21.15.27:0/2345879138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3880c2ae0 con 0x7fa374003f70 2023-12-19T18:19:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.838+0000 7fa38f300640 1 -- 172.21.15.27:0/2345879138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3881601e0 con 0x7fa374003f70 2023-12-19T18:19:57.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.839+0000 7fa37dffb640 1 -- 172.21.15.27:0/2345879138 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa384004960 con 0x7fa374003f70 2023-12-19T18:19:57.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.840+0000 7fa37dffb640 1 --2- 172.21.15.27:0/2345879138 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa36c06a460 0x7fa36c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:57.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.840+0000 7fa37dffb640 1 -- 172.21.15.27:0/2345879138 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa384087c40 con 0x7fa374003f70 2023-12-19T18:19:57.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.841+0000 7fa38c89b640 1 --2- 172.21.15.27:0/2345879138 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa36c06a460 0x7fa36c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:57.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.842+0000 7fa38c89b640 1 --2- 172.21.15.27:0/2345879138 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa36c06a460 0x7fa36c06c920 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa378033000 tx=0x7fa37802f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:57.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:57.845+0000 7fa37dffb640 1 -- 172.21.15.27:0/2345879138 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa3881601e0 con 0x7fa374003f70 2023-12-19T18:19:58.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.046+0000 7fa38f300640 1 -- 172.21.15.27:0/2345879138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa3881601e0 con 0x7fa374003f70 2023-12-19T18:19:58.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.047+0000 7fa37dffb640 1 -- 172.21.15.27:0/2345879138 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) v1 ==== 84+0+65 (secure 0 0 0) 0x7fa3881601e0 con 0x7fa374003f70 2023-12-19T18:19:58.047 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:19:58.047 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-12-19T18:19:58.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:19:58.047 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-12-19T18:19:58.047 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-12-19T18:19:58.047 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:19:58.047 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-12-19T18:19:58.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.049+0000 7fa38f300640 1 -- 172.21.15.27:0/2345879138 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa36c06a460 msgr2=0x7fa36c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:58.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.049+0000 7fa38f300640 1 --2- 172.21.15.27:0/2345879138 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa36c06a460 0x7fa36c06c920 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa378033000 tx=0x7fa37802f000 comp rx=0 tx=0).stop 2023-12-19T18:19:58.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.049+0000 7fa38f300640 1 -- 172.21.15.27:0/2345879138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa374003f70 msgr2=0x7fa38815c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:58.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.049+0000 7fa38f300640 1 --2- 172.21.15.27:0/2345879138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa374003f70 0x7fa38815c5d0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fa38400c650 tx=0x7fa38400cb10 comp rx=0 tx=0).stop 2023-12-19T18:19:58.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.049+0000 7fa38f300640 1 -- 172.21.15.27:0/2345879138 shutdown_connections 2023-12-19T18:19:58.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.050+0000 7fa38f300640 1 --2- 172.21.15.27:0/2345879138 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa36c06a460 0x7fa36c06c920 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.050+0000 7fa38f300640 1 --2- 172.21.15.27:0/2345879138 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa388154c90 0x7fa388155110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.050+0000 7fa38f300640 1 --2- 172.21.15.27:0/2345879138 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa388155870 0x7fa3880c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.050+0000 7fa38f300640 1 --2- 172.21.15.27:0/2345879138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa374003f70 0x7fa38815c5d0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.050+0000 7fa38f300640 1 -- 172.21.15.27:0/2345879138 >> 172.21.15.27:0/2345879138 conn(0x7fa3880ceb50 msgr2=0x7fa3880ccd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:58.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.050+0000 7fa38f300640 1 -- 172.21.15.27:0/2345879138 shutdown_connections 2023-12-19T18:19:58.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.050+0000 7fa38f300640 1 -- 172.21.15.27:0/2345879138 wait complete. 2023-12-19T18:19:58.070 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:19:58.070 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:19:58.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.306+0000 7faea7377640 1 Processor -- start 2023-12-19T18:19:58.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.306+0000 7faea7377640 1 -- start start 2023-12-19T18:19:58.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.308+0000 7faea7377640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faea0161030 0x7faea01657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:58.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.308+0000 7faea7377640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faea00cba60 0x7faea00cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:58.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.308+0000 7faea7377640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faea0165d00 con 0x7faea00c55b0 2023-12-19T18:19:58.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.308+0000 7faea7377640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faea00c95f0 con 0x7faea00cba60 2023-12-19T18:19:58.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.308+0000 7faea7377640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea00cb100 con 0x7faea0161030 2023-12-19T18:19:58.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.308+0000 7faea6b76640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faea00cba60 0x7faea00cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:58.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.308+0000 7faea5b74640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faea00c55b0 0x7faea00cb350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40746/0 (socket says 172.21.15.27:40746) 2023-12-19T18:19:58.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.308+0000 7faea5b74640 1 -- 172.21.15.27:0/1173918625 learned_addr learned my addr 172.21.15.27:0/1173918625 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:58.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.308+0000 7faea6375640 1 --2- 172.21.15.27:0/1173918625 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faea0161030 0x7faea01657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:58.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.309+0000 7faea6b76640 1 -- 172.21.15.27:0/1173918625 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faea0161030 msgr2=0x7faea01657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:58.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.309+0000 7faea6b76640 1 --2- 172.21.15.27:0/1173918625 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faea0161030 0x7faea01657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.309+0000 7faea6b76640 1 -- 172.21.15.27:0/1173918625 >> v1:172.21.15.27:6789/0 conn(0x7faea00c55b0 legacy=0x7faea00cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:58.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.309+0000 7faea6b76640 1 -- 172.21.15.27:0/1173918625 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea00cae20 con 0x7faea00cba60 2023-12-19T18:19:58.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.311+0000 7faea6b76640 1 --2- 172.21.15.27:0/1173918625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea00cba60 0x7faea00cc510 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fae98009d90 tx=0x7fae9802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bda55bda6ff1c46 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:58.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.311+0000 7faea5373640 1 -- 172.21.15.27:0/1173918625 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9802cc60 con 0x7faea00cba60 2023-12-19T18:19:58.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.311+0000 7faea5373640 1 -- 172.21.15.27:0/1173918625 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae9802cdc0 con 0x7faea00cba60 2023-12-19T18:19:58.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.311+0000 7faea5373640 1 -- 172.21.15.27:0/1173918625 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98034660 con 0x7faea00cba60 2023-12-19T18:19:58.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.312+0000 7faea7377640 1 -- 172.21.15.27:0/1173918625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea00cba60 msgr2=0x7faea00cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:58.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.312+0000 7faea7377640 1 --2- 172.21.15.27:0/1173918625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea00cba60 0x7faea00cc510 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7fae98009d90 tx=0x7fae9802beb0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.312+0000 7faea7377640 1 -- 172.21.15.27:0/1173918625 shutdown_connections 2023-12-19T18:19:58.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.312+0000 7faea7377640 1 --2- 172.21.15.27:0/1173918625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea00cba60 0x7faea00cc510 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.312+0000 7faea7377640 1 --2- 172.21.15.27:0/1173918625 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faea0161030 0x7faea01657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.312+0000 7faea7377640 1 -- 172.21.15.27:0/1173918625 >> 172.21.15.27:0/1173918625 conn(0x7faea00b05b0 msgr2=0x7faea00b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:58.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.312+0000 7faea7377640 1 -- 172.21.15.27:0/1173918625 shutdown_connections 2023-12-19T18:19:58.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.312+0000 7faea7377640 1 -- 172.21.15.27:0/1173918625 wait complete. 2023-12-19T18:19:58.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.314+0000 7faea7377640 1 Processor -- start 2023-12-19T18:19:58.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.314+0000 7faea7377640 1 -- start start 2023-12-19T18:19:58.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.315+0000 7faea7377640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faea00c55b0 0x7faea00c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:58.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.315+0000 7faea7377640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faea00cba60 0x7faea00c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:58.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.315+0000 7faea7377640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea0161030 0x7faea015f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:58.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.315+0000 7faea7377640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea00c6a20 con 0x7faea0161030 2023-12-19T18:19:58.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.315+0000 7faea7377640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea00c5a10 con 0x7faea00c55b0 2023-12-19T18:19:58.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.315+0000 7faea7377640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faea00c5c60 con 0x7faea00cba60 2023-12-19T18:19:58.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.315+0000 7faea6b76640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea0161030 0x7faea015f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:58.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.315+0000 7faea6b76640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea0161030 0x7faea015f670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58390/0 (socket says 172.21.15.27:58390) 2023-12-19T18:19:58.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.315+0000 7faea6b76640 1 -- 172.21.15.27:0/706953618 learned_addr learned my addr 172.21.15.27:0/706953618 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:58.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.316+0000 7faea6375640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faea00c55b0 0x7faea00c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:58.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.316+0000 7faea5b74640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faea00cba60 0x7faea00c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:58.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.316+0000 7faea6b76640 1 -- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faea00cba60 msgr2=0x7faea00c7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:58.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.316+0000 7faea6b76640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faea00cba60 0x7faea00c7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.316+0000 7faea6b76640 1 -- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faea00c55b0 msgr2=0x7faea00c6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:58.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.316+0000 7faea6b76640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faea00c55b0 0x7faea00c6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.316+0000 7faea6b76640 1 -- 172.21.15.27:0/706953618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae98009a70 con 0x7faea0161030 2023-12-19T18:19:58.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.316+0000 7faea5b74640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faea00cba60 0x7faea00c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:19:58.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.316+0000 7faea6b76640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea0161030 0x7faea015f670 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fae98009d90 tx=0x7fae98004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:58.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.317+0000 7fae96ffd640 1 -- 172.21.15.27:0/706953618 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98040650 con 0x7faea0161030 2023-12-19T18:19:58.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.317+0000 7fae96ffd640 1 -- 172.21.15.27:0/706953618 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae980407b0 con 0x7faea0161030 2023-12-19T18:19:58.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.317+0000 7fae96ffd640 1 -- 172.21.15.27:0/706953618 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98040b60 con 0x7faea0161030 2023-12-19T18:19:58.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.317+0000 7faea7377640 1 -- 172.21.15.27:0/706953618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea00c5f40 con 0x7faea0161030 2023-12-19T18:19:58.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.317+0000 7faea7377640 1 -- 172.21.15.27:0/706953618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea00c6500 con 0x7faea0161030 2023-12-19T18:19:58.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.318+0000 7faea7377640 1 -- 172.21.15.27:0/706953618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faea004e840 con 0x7faea0161030 2023-12-19T18:19:58.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.319+0000 7fae96ffd640 1 -- 172.21.15.27:0/706953618 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fae98040cc0 con 0x7faea0161030 2023-12-19T18:19:58.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.320+0000 7fae96ffd640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae8406a430 0x7fae8406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:58.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.320+0000 7fae96ffd640 1 -- 172.21.15.27:0/706953618 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fae980ab330 con 0x7faea0161030 2023-12-19T18:19:58.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.321+0000 7faea6375640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae8406a430 0x7fae8406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:58.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.322+0000 7faea6375640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae8406a430 0x7fae8406c8f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009a30 tx=0x7fae9c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:58.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.325+0000 7fae96ffd640 1 -- 172.21.15.27:0/706953618 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fae98076c60 con 0x7faea0161030 2023-12-19T18:19:58.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.524+0000 7faea7377640 1 -- 172.21.15.27:0/706953618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7faea00c6690 con 0x7faea0161030 2023-12-19T18:19:58.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.759+0000 7fae96ffd640 1 -- 172.21.15.27:0/706953618 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v93) v1 ==== 138+0+0 (secure 0 0 0) 0x7fae98009450 con 0x7faea0161030 2023-12-19T18:19:58.761 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-12-19T18:19:58.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.767+0000 7faea7377640 1 -- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae8406a430 msgr2=0x7fae8406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:58.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.767+0000 7faea7377640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae8406a430 0x7fae8406c8f0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fae9c009a30 tx=0x7fae9c008040 comp rx=0 tx=0).stop 2023-12-19T18:19:58.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.767+0000 7faea7377640 1 -- 172.21.15.27:0/706953618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea0161030 msgr2=0x7faea015f670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:58.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.767+0000 7faea7377640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea0161030 0x7faea015f670 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7fae98009d90 tx=0x7fae98004a40 comp rx=0 tx=0).stop 2023-12-19T18:19:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.768+0000 7faea7377640 1 -- 172.21.15.27:0/706953618 shutdown_connections 2023-12-19T18:19:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.768+0000 7faea7377640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae8406a430 0x7fae8406c8f0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.768+0000 7faea7377640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faea0161030 0x7faea015f670 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.768+0000 7faea7377640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faea00cba60 0x7faea00c7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.768+0000 7faea7377640 1 --2- 172.21.15.27:0/706953618 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faea00c55b0 0x7faea00c6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.768+0000 7faea7377640 1 -- 172.21.15.27:0/706953618 >> 172.21.15.27:0/706953618 conn(0x7faea00b05b0 msgr2=0x7faea00cb350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.768+0000 7faea7377640 1 -- 172.21.15.27:0/706953618 shutdown_connections 2023-12-19T18:19:58.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:58.768+0000 7faea7377640 1 -- 172.21.15.27:0/706953618 wait complete. 2023-12-19T18:19:58.792 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:19:59.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.027+0000 7f33d0428640 1 Processor -- start 2023-12-19T18:19:59.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.027+0000 7f33d0428640 1 -- start start 2023-12-19T18:19:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.028+0000 7f33d0428640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c80b8c30 0x7f33c80d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.028+0000 7f33d0428640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f33c80cf9b0 0x7f33c80cfe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.028+0000 7f33d0428640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33c80d0350 con 0x7f33c80cda20 2023-12-19T18:19:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.028+0000 7f33d0428640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33c80cea80 con 0x7f33c80cf9b0 2023-12-19T18:19:59.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.028+0000 7f33d0428640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80b5f10 con 0x7f33c80b8c30 2023-12-19T18:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.029+0000 7f33ce9c5640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f33c80cda20 0x7f33c80cde70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40748/0 (socket says 172.21.15.27:40748) 2023-12-19T18:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.029+0000 7f33ce9c5640 1 -- 172.21.15.27:0/940132687 learned_addr learned my addr 172.21.15.27:0/940132687 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.029+0000 7f33cd9c3640 1 --2- 172.21.15.27:0/940132687 >> v2:172.21.15.192:3300/0 conn(0x7f33c80cf9b0 0x7f33c80cfe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.029+0000 7f33ce1c4640 1 --2- 172.21.15.27:0/940132687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c80b8c30 0x7f33c80d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.029+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2260173568 0 0) 0x7f33c80d0350 con 0x7f33c80cda20 2023-12-19T18:19:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.029+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33b4003520 con 0x7f33c80cda20 2023-12-19T18:19:59.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1726269577 0 0) 0x7f33b4003520 con 0x7f33c80cda20 2023-12-19T18:19:59.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c80b8c30 msgr2=0x7f33c80d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:59.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 --2- 172.21.15.27:0/940132687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c80b8c30 0x7f33c80d1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:59.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 >> v2:172.21.15.192:3300/0 conn(0x7f33c80cf9b0 msgr2=0x7f33c80cfe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:59.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 --2- 172.21.15.27:0/940132687 >> v2:172.21.15.192:3300/0 conn(0x7f33c80cf9b0 0x7f33c80cfe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:59.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c80ce7a0 con 0x7f33c80cda20 2023-12-19T18:19:59.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f33c0001670 con 0x7f33c80cda20 2023-12-19T18:19:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 >> v1:172.21.15.27:6789/0 conn(0x7f33c80cda20 legacy=0x7f33c80cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 --2- 172.21.15.27:0/940132687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c80b8c30 0x7f33b4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 --2- 172.21.15.27:0/940132687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33b4003f40 0x7f33b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 --2- 172.21.15.27:0/940132687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b4004d00 0x7f33b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80cea80 con 0x7f33b4004d00 2023-12-19T18:19:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80b5f10 con 0x7f33b4003f40 2023-12-19T18:19:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b400ba30 con 0x7f33c80b8c30 2023-12-19T18:19:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33ce9c5640 1 --2- 172.21.15.27:0/940132687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b4004d00 0x7f33b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.030+0000 7f33cd9c3640 1 --2- 172.21.15.27:0/940132687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33b4003f40 0x7f33b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.031+0000 7f33ce1c4640 1 --2- 172.21.15.27:0/940132687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c80b8c30 0x7f33b4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.031+0000 7f33ce9c5640 1 -- 172.21.15.27:0/940132687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c80b8c30 msgr2=0x7f33b4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.031+0000 7f33ce9c5640 1 --2- 172.21.15.27:0/940132687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c80b8c30 0x7f33b4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.031+0000 7f33ce9c5640 1 -- 172.21.15.27:0/940132687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33b4003f40 msgr2=0x7f33b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:19:59.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.031+0000 7f33ce9c5640 1 --2- 172.21.15.27:0/940132687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33b4003f40 0x7f33b40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:59.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.031+0000 7f33ce9c5640 1 -- 172.21.15.27:0/940132687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b4003520 con 0x7f33b4004d00 2023-12-19T18:19:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.032+0000 7f33ce9c5640 1 --2- 172.21.15.27:0/940132687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b4004d00 0x7f33b4005180 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f33c000c110 tx=0x7f33b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10963e45ad4be341 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.032+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c0005ad0 con 0x7f33b4004d00 2023-12-19T18:19:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.032+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c0005c50 con 0x7f33b4004d00 2023-12-19T18:19:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.032+0000 7f33cd1c2640 1 -- 172.21.15.27:0/940132687 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c002c830 con 0x7f33b4004d00 2023-12-19T18:19:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.033+0000 7f33d0428640 1 -- 172.21.15.27:0/940132687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b4004d00 msgr2=0x7f33b4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.033+0000 7f33d0428640 1 --2- 172.21.15.27:0/940132687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33b4004d00 0x7f33b4005180 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f33c000c110 tx=0x7f33b40057f0 comp rx=0 tx=0).stop 2023-12-19T18:19:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.033+0000 7f33ce9c5640 1 -- 172.21.15.27:0/940132687 reap_dead start 2023-12-19T18:19:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.033+0000 7f33d0428640 1 -- 172.21.15.27:0/940132687 shutdown_connections 2023-12-19T18:19:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.033+0000 7f33d0428640 1 -- 172.21.15.27:0/940132687 >> 172.21.15.27:0/940132687 conn(0x7f33c80d2fa0 msgr2=0x7f33c80d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.033+0000 7f33d0428640 1 -- 172.21.15.27:0/940132687 shutdown_connections 2023-12-19T18:19:59.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.033+0000 7f33d0428640 1 -- 172.21.15.27:0/940132687 wait complete. 2023-12-19T18:19:59.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.036+0000 7f33d0428640 1 Processor -- start 2023-12-19T18:19:59.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.036+0000 7f33d0428640 1 -- start start 2023-12-19T18:19:59.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.037+0000 7f33d0428640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33b4004d00 0x7f33c814bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:59.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.037+0000 7f33d0428640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c814c4d0 0x7f33c815af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:59.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.037+0000 7f33d0428640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c815b450 0x7f33c80b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:59.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.037+0000 7f33d0428640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c814bd50 con 0x7f33c815b450 2023-12-19T18:19:59.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.037+0000 7f33d0428640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80b2900 con 0x7f33b4004d00 2023-12-19T18:19:59.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.037+0000 7f33d0428640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c80b2be0 con 0x7f33c814c4d0 2023-12-19T18:19:59.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.038+0000 7f33ce9c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c815b450 0x7f33c80b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:59.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.038+0000 7f33ce9c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c815b450 0x7f33c80b39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58408/0 (socket says 172.21.15.27:58408) 2023-12-19T18:19:59.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.038+0000 7f33ce9c5640 1 -- 172.21.15.27:0/2910928439 learned_addr learned my addr 172.21.15.27:0/2910928439 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:19:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.038+0000 7f33ce1c4640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33b4004d00 0x7f33c814bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.038+0000 7f33cd9c3640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c814c4d0 0x7f33c815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.038+0000 7f33ce9c5640 1 -- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c814c4d0 msgr2=0x7f33c815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.038+0000 7f33ce9c5640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c814c4d0 0x7f33c815af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.038+0000 7f33ce9c5640 1 -- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33b4004d00 msgr2=0x7f33c814bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.038+0000 7f33ce9c5640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33b4004d00 0x7f33c814bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.038+0000 7f33ce9c5640 1 -- 172.21.15.27:0/2910928439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c80b2ec0 con 0x7f33c815b450 2023-12-19T18:19:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.038+0000 7f33ce1c4640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33b4004d00 0x7f33c814bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:19:59.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.038+0000 7f33cd9c3640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c814c4d0 0x7f33c815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:19:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.039+0000 7f33ce9c5640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c815b450 0x7f33c80b39a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f33c002c3d0 tx=0x7f33c00019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.039+0000 7f33beffd640 1 -- 172.21.15.27:0/2910928439 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c003c040 con 0x7f33c815b450 2023-12-19T18:19:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.039+0000 7f33beffd640 1 -- 172.21.15.27:0/2910928439 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c002cc40 con 0x7f33c815b450 2023-12-19T18:19:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.039+0000 7f33beffd640 1 -- 172.21.15.27:0/2910928439 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c0005aa0 con 0x7f33c815b450 2023-12-19T18:19:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.039+0000 7f33d0428640 1 -- 172.21.15.27:0/2910928439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c80b31a0 con 0x7f33c815b450 2023-12-19T18:19:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.039+0000 7f33d0428640 1 -- 172.21.15.27:0/2910928439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c8157550 con 0x7f33c815b450 2023-12-19T18:19:59.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.040+0000 7f33d0428640 1 -- 172.21.15.27:0/2910928439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c80b36c0 con 0x7f33c815b450 2023-12-19T18:19:59.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.042+0000 7f33beffd640 1 -- 172.21.15.27:0/2910928439 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f33c0032070 con 0x7f33c815b450 2023-12-19T18:19:59.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.043+0000 7f33beffd640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f33ac06a460 0x7f33ac06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:19:59.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.043+0000 7f33beffd640 1 -- 172.21.15.27:0/2910928439 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f33c00a9b00 con 0x7f33c815b450 2023-12-19T18:19:59.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.044+0000 7f33ce1c4640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f33ac06a460 0x7f33ac06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:19:59.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.045+0000 7f33ce1c4640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f33ac06a460 0x7f33ac06c920 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f33c40074d0 tx=0x7f33c400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:19:59.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.047+0000 7f33beffd640 1 -- 172.21.15.27:0/2910928439 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f33c00753b0 con 0x7f33c815b450 2023-12-19T18:19:59.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.246+0000 7f33d0428640 1 -- 172.21.15.27:0/2910928439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f33c80b36c0 con 0x7f33c815b450 2023-12-19T18:19:59.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.771+0000 7f33beffd640 1 -- 172.21.15.27:0/2910928439 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v94) v1 ==== 170+0+0 (secure 0 0 0) 0x7f33c80b36c0 con 0x7f33c815b450 2023-12-19T18:19:59.777 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-12-19T18:19:59.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.787+0000 7f33d0428640 1 -- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f33ac06a460 msgr2=0x7f33ac06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:59.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.787+0000 7f33d0428640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f33ac06a460 0x7f33ac06c920 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f33c40074d0 tx=0x7f33c400b040 comp rx=0 tx=0).stop 2023-12-19T18:19:59.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.787+0000 7f33d0428640 1 -- 172.21.15.27:0/2910928439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c815b450 msgr2=0x7f33c80b39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:19:59.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.787+0000 7f33d0428640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c815b450 0x7f33c80b39a0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f33c002c3d0 tx=0x7f33c00019a0 comp rx=0 tx=0).stop 2023-12-19T18:19:59.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.787+0000 7f33d0428640 1 -- 172.21.15.27:0/2910928439 shutdown_connections 2023-12-19T18:19:59.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.787+0000 7f33d0428640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f33ac06a460 0x7f33ac06c920 secure :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f33c40074d0 tx=0x7f33c400b040 comp rx=0 tx=0).stop 2023-12-19T18:19:59.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.787+0000 7f33d0428640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c815b450 0x7f33c80b39a0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.787+0000 7f33d0428640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33c814c4d0 0x7f33c815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.787+0000 7f33d0428640 1 --2- 172.21.15.27:0/2910928439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33b4004d00 0x7f33c814bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:19:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.787+0000 7f33d0428640 1 -- 172.21.15.27:0/2910928439 >> 172.21.15.27:0/2910928439 conn(0x7f33c80d2fa0 msgr2=0x7f33c80d1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:19:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.788+0000 7f33d0428640 1 -- 172.21.15.27:0/2910928439 shutdown_connections 2023-12-19T18:19:59.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:19:59.788+0000 7f33d0428640 1 -- 172.21.15.27:0/2910928439 wait complete. 2023-12-19T18:19:59.810 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:20:00.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.016+0000 7f35320e5640 1 Processor -- start 2023-12-19T18:20:00.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.016+0000 7f35320e5640 1 -- start start 2023-12-19T18:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.017+0000 7f35320e5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f352c0d1a20 0x7f352c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.017+0000 7f35320e5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f352c0b3980 0x7f352c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.017+0000 7f35320e5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352c0d3850 con 0x7f352c15d120 2023-12-19T18:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.017+0000 7f35320e5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352c0d0b10 con 0x7f352c0b3980 2023-12-19T18:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.017+0000 7f35320e5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352c15d500 con 0x7f352c0d1a20 2023-12-19T18:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.018+0000 7f35318e4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f352c0b3980 0x7f352c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.018+0000 7f35318e4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f352c0b3980 0x7f352c0cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58416/0 (socket says 172.21.15.27:58416) 2023-12-19T18:20:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.018+0000 7f35318e4640 1 -- 172.21.15.27:0/2963675515 learned_addr learned my addr 172.21.15.27:0/2963675515 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:00.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.018+0000 7f35318e4640 1 -- 172.21.15.27:0/2963675515 >> v2:172.21.15.192:3300/0 conn(0x7f352c0d1a20 msgr2=0x7f352c0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:00.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.018+0000 7f35318e4640 1 --2- 172.21.15.27:0/2963675515 >> v2:172.21.15.192:3300/0 conn(0x7f352c0d1a20 0x7f352c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.018+0000 7f35318e4640 1 -- 172.21.15.27:0/2963675515 >> v1:172.21.15.192:6789/0 conn(0x7f352c15d120 legacy=0x7f352c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:00.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.018+0000 7f35318e4640 1 -- 172.21.15.27:0/2963675515 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c0d1040 con 0x7f352c0b3980 2023-12-19T18:20:00.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.020+0000 7f35318e4640 1 --2- 172.21.15.27:0/2963675515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0b3980 0x7f352c0cd940 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f352c08d040 tx=0x7f352402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b45b8184f5c80ae server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:00.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.020+0000 7f3523fff640 1 -- 172.21.15.27:0/2963675515 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352402ca80 con 0x7f352c0b3980 2023-12-19T18:20:00.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.020+0000 7f3523fff640 1 -- 172.21.15.27:0/2963675515 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f352402cbe0 con 0x7f352c0b3980 2023-12-19T18:20:00.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.020+0000 7f3523fff640 1 -- 172.21.15.27:0/2963675515 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35240343f0 con 0x7f352c0b3980 2023-12-19T18:20:00.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.021+0000 7f35320e5640 1 -- 172.21.15.27:0/2963675515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0b3980 msgr2=0x7f352c0cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.021+0000 7f35320e5640 1 --2- 172.21.15.27:0/2963675515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0b3980 0x7f352c0cd940 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f352c08d040 tx=0x7f352402bee0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.021+0000 7f35320e5640 1 -- 172.21.15.27:0/2963675515 shutdown_connections 2023-12-19T18:20:00.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.021+0000 7f35320e5640 1 --2- 172.21.15.27:0/2963675515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0b3980 0x7f352c0cd940 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.021+0000 7f35320e5640 1 --2- 172.21.15.27:0/2963675515 >> v2:172.21.15.192:3300/0 conn(0x7f352c0d1a20 0x7f352c0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.021+0000 7f35320e5640 1 -- 172.21.15.27:0/2963675515 >> 172.21.15.27:0/2963675515 conn(0x7f352c0d2d30 msgr2=0x7f352c0d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.021+0000 7f35320e5640 1 -- 172.21.15.27:0/2963675515 shutdown_connections 2023-12-19T18:20:00.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.021+0000 7f35320e5640 1 -- 172.21.15.27:0/2963675515 wait complete. 2023-12-19T18:20:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.023+0000 7f35320e5640 1 Processor -- start 2023-12-19T18:20:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.023+0000 7f35320e5640 1 -- start start 2023-12-19T18:20:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.024+0000 7f35320e5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0b3980 0x7f352c0ce790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.024+0000 7f35320e5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c0d1a20 0x7f352c0c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.024+0000 7f35320e5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c15d120 0x7f352c1596f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.024+0000 7f35320e5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c159c30 con 0x7f352c0b3980 2023-12-19T18:20:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.024+0000 7f35320e5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0c8350 con 0x7f352c15d120 2023-12-19T18:20:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.024+0000 7f35320e5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0c8630 con 0x7f352c0d1a20 2023-12-19T18:20:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.024+0000 7f35310e3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0b3980 0x7f352c0ce790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.024+0000 7f35310e3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0b3980 0x7f352c0ce790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58428/0 (socket says 172.21.15.27:58428) 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.024+0000 7f35310e3640 1 -- 172.21.15.27:0/3896891523 learned_addr learned my addr 172.21.15.27:0/3896891523 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.025+0000 7f35318e4640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c15d120 0x7f352c1596f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.025+0000 7f35308e2640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c0d1a20 0x7f352c0c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.025+0000 7f35310e3640 1 -- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c0d1a20 msgr2=0x7f352c0c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.025+0000 7f35310e3640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c0d1a20 0x7f352c0c7ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.025+0000 7f35310e3640 1 -- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c15d120 msgr2=0x7f352c1596f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.025+0000 7f35310e3640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c15d120 0x7f352c1596f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.025+0000 7f35310e3640 1 -- 172.21.15.27:0/3896891523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3524009b30 con 0x7f352c0b3980 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.025+0000 7f35318e4640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c15d120 0x7f352c1596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.025+0000 7f35310e3640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0b3980 0x7f352c0ce790 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f352800b3f0 tx=0x7f352800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.026+0000 7f3521ffb640 1 -- 172.21.15.27:0/3896891523 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528011830 con 0x7f352c0b3980 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.026+0000 7f3521ffb640 1 -- 172.21.15.27:0/3896891523 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3528011990 con 0x7f352c0b3980 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.026+0000 7f3521ffb640 1 -- 172.21.15.27:0/3896891523 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3528011d40 con 0x7f352c0b3980 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.026+0000 7f35320e5640 1 -- 172.21.15.27:0/3896891523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c0c8910 con 0x7f352c0b3980 2023-12-19T18:20:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.026+0000 7f35320e5640 1 -- 172.21.15.27:0/3896891523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c0c8ed0 con 0x7f352c0b3980 2023-12-19T18:20:00.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.027+0000 7f35320e5640 1 -- 172.21.15.27:0/3896891523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352c0c90f0 con 0x7f352c0b3980 2023-12-19T18:20:00.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.028+0000 7f3521ffb640 1 -- 172.21.15.27:0/3896891523 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f352801a400 con 0x7f352c0b3980 2023-12-19T18:20:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.029+0000 7f3521ffb640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f351006a430 0x7f351006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.029+0000 7f3521ffb640 1 -- 172.21.15.27:0/3896891523 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f3528088670 con 0x7f352c0b3980 2023-12-19T18:20:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.030+0000 7f35308e2640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f351006a430 0x7f351006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.031+0000 7f35308e2640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f351006a430 0x7f351006c8f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f352c15fa50 tx=0x7f351c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:00.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.035+0000 7f3521ffb640 1 -- 172.21.15.27:0/3896891523 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3528053d50 con 0x7f352c0b3980 2023-12-19T18:20:00.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.261+0000 7f35320e5640 1 -- 172.21.15.27:0/3896891523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f352c04e840 con 0x7f352c0b3980 2023-12-19T18:20:00.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.278+0000 7f3521ffb640 1 -- 172.21.15.27:0/3896891523 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v34) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3528057370 con 0x7f352c0b3980 2023-12-19T18:20:00.279 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 10 and data pool 11 2023-12-19T18:20:00.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.283+0000 7f35320e5640 1 -- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f351006a430 msgr2=0x7f351006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.283+0000 7f35320e5640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f351006a430 0x7f351006c8f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f352c15fa50 tx=0x7f351c008040 comp rx=0 tx=0).stop 2023-12-19T18:20:00.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.283+0000 7f35320e5640 1 -- 172.21.15.27:0/3896891523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0b3980 msgr2=0x7f352c0ce790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.283+0000 7f35320e5640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0b3980 0x7f352c0ce790 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f352800b3f0 tx=0x7f352800b8b0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.283+0000 7f35320e5640 1 -- 172.21.15.27:0/3896891523 shutdown_connections 2023-12-19T18:20:00.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.283+0000 7f35320e5640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f351006a430 0x7f351006c8f0 secure :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f352c15fa50 tx=0x7f351c008040 comp rx=0 tx=0).stop 2023-12-19T18:20:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.283+0000 7f35320e5640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c15d120 0x7f352c1596f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.283+0000 7f35320e5640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c0d1a20 0x7f352c0c7ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.283+0000 7f35320e5640 1 --2- 172.21.15.27:0/3896891523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0b3980 0x7f352c0ce790 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.283+0000 7f35320e5640 1 -- 172.21.15.27:0/3896891523 >> 172.21.15.27:0/3896891523 conn(0x7f352c0d2d30 msgr2=0x7f352c0d1310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.283+0000 7f35320e5640 1 -- 172.21.15.27:0/3896891523 shutdown_connections 2023-12-19T18:20:00.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.284+0000 7f35320e5640 1 -- 172.21.15.27:0/3896891523 wait complete. 2023-12-19T18:20:00.310 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:20:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.494+0000 7f9c2b59e640 1 Processor -- start 2023-12-19T18:20:00.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.495+0000 7f9c2b59e640 1 -- start start 2023-12-19T18:20:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.496+0000 7f9c2b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c2c15ce70 0x7f9c2c06b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.496+0000 7f9c2b59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c2c06bdb0 con 0x7f9c2c06b950 2023-12-19T18:20:00.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.496+0000 7f9c2b59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c2c06d940 con 0x7f9c2c070c00 2023-12-19T18:20:00.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.496+0000 7f9c2b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c0aeeb0 con 0x7f9c2c15ce70 2023-12-19T18:20:00.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.496+0000 7f9c29d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9c2c06b950 0x7f9c2c0704f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40756/0 (socket says 172.21.15.27:40756) 2023-12-19T18:20:00.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.497+0000 7f9c29d9b640 1 -- 172.21.15.27:0/4210496785 learned_addr learned my addr 172.21.15.27:0/4210496785 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.497+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1243570822 0 0) 0x7f9c2c06bdb0 con 0x7f9c2c06b950 2023-12-19T18:20:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.497+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c18003520 con 0x7f9c2c06b950 2023-12-19T18:20:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.497+0000 7f9c2a59c640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c2c15ce70 0x7f9c2c06b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.497+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2168471234 0 0) 0x7f9c18003520 con 0x7f9c2c06b950 2023-12-19T18:20:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.497+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c2c15ce70 msgr2=0x7f9c2c06b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.497+0000 7f9c2959a640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c2c15ce70 0x7f9c2c06b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.497+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 >> v1:172.21.15.192:6789/0 conn(0x7f9c2c070c00 legacy=0x7f9c2c069de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.497+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c06dc20 con 0x7f9c2c06b950 2023-12-19T18:20:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.497+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f9c14001670 con 0x7f9c2c06b950 2023-12-19T18:20:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.497+0000 7f9c2a59c640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c2c15ce70 0x7f9c2c06b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:20:00.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.497+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 >> v1:172.21.15.27:6789/0 conn(0x7f9c2c06b950 legacy=0x7f9c2c0704f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c2959a640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c180039a0 0x7f9c18003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c2959a640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c180047c0 0x7f9c18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c2959a640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c2c15ce70 0x7f9c180051d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c0aeeb0 con 0x7f9c180047c0 2023-12-19T18:20:00.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1800bb10 con 0x7f9c180039a0 2023-12-19T18:20:00.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1800bc80 con 0x7f9c2c15ce70 2023-12-19T18:20:00.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c29d9b640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c180047c0 0x7f9c18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c2ad9d640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c2c15ce70 0x7f9c180051d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c2a59c640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c180039a0 0x7f9c18003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c29d9b640 1 -- 172.21.15.27:0/4210496785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c2c15ce70 msgr2=0x7f9c180051d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:00.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c29d9b640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c2c15ce70 0x7f9c180051d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c29d9b640 1 -- 172.21.15.27:0/4210496785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c180039a0 msgr2=0x7f9c18003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:00.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c29d9b640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c180039a0 0x7f9c18003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c29d9b640 1 -- 172.21.15.27:0/4210496785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c18003520 con 0x7f9c180047c0 2023-12-19T18:20:00.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.498+0000 7f9c2a59c640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c180039a0 0x7f9c18003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:00.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.499+0000 7f9c29d9b640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c180047c0 0x7f9c18004c40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400c0e0 tx=0x7f9c180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a3326b45f983c43 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:00.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.500+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14005ad0 con 0x7f9c180047c0 2023-12-19T18:20:00.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.500+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c14005c30 con 0x7f9c180047c0 2023-12-19T18:20:00.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.500+0000 7f9c2959a640 1 -- 172.21.15.27:0/4210496785 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1402c810 con 0x7f9c180047c0 2023-12-19T18:20:00.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.500+0000 7f9c2b59e640 1 -- 172.21.15.27:0/4210496785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c180047c0 msgr2=0x7f9c18004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.500+0000 7f9c2b59e640 1 --2- 172.21.15.27:0/4210496785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c180047c0 0x7f9c18004c40 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f9c1400c0e0 tx=0x7f9c180057f0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.500+0000 7f9c2ad9d640 1 -- 172.21.15.27:0/4210496785 reap_dead start 2023-12-19T18:20:00.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.500+0000 7f9c2b59e640 1 -- 172.21.15.27:0/4210496785 shutdown_connections 2023-12-19T18:20:00.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.500+0000 7f9c2b59e640 1 -- 172.21.15.27:0/4210496785 >> 172.21.15.27:0/4210496785 conn(0x7f9c2c0cfaa0 msgr2=0x7f9c2c0cfe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:00.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.500+0000 7f9c2b59e640 1 -- 172.21.15.27:0/4210496785 shutdown_connections 2023-12-19T18:20:00.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.501+0000 7f9c2b59e640 1 -- 172.21.15.27:0/4210496785 wait complete. 2023-12-19T18:20:00.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.502+0000 7f9c2b59e640 1 Processor -- start 2023-12-19T18:20:00.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.502+0000 7f9c2b59e640 1 -- start start 2023-12-19T18:20:00.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.503+0000 7f9c2b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c180047c0 0x7f9c2c0b45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.503+0000 7f9c2b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c2c0b4b20 0x7f9c2c0b2610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.503+0000 7f9c2b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c2c0b2b50 0x7f9c2c1f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.503+0000 7f9c2b59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c0b2fe0 con 0x7f9c2c0b4b20 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.503+0000 7f9c2b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c0b0550 con 0x7f9c2c0b2b50 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.503+0000 7f9c2b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c2c0b4fc0 con 0x7f9c180047c0 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.504+0000 7f9c29d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c2c0b4b20 0x7f9c2c0b2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.504+0000 7f9c29d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c2c0b4b20 0x7f9c2c0b2610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58434/0 (socket says 172.21.15.27:58434) 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.504+0000 7f9c29d9b640 1 -- 172.21.15.27:0/3262925316 learned_addr learned my addr 172.21.15.27:0/3262925316 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.504+0000 7f9c2a59c640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c180047c0 0x7f9c2c0b45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.504+0000 7f9c29d9b640 1 -- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c180047c0 msgr2=0x7f9c2c0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.504+0000 7f9c2ad9d640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c2c0b2b50 0x7f9c2c1f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.504+0000 7f9c29d9b640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c180047c0 0x7f9c2c0b45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.504+0000 7f9c29d9b640 1 -- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c2c0b2b50 msgr2=0x7f9c2c1f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.504+0000 7f9c29d9b640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c2c0b2b50 0x7f9c2c1f6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.504+0000 7f9c29d9b640 1 -- 172.21.15.27:0/3262925316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c2c0b52a0 con 0x7f9c2c0b4b20 2023-12-19T18:20:00.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.505+0000 7f9c29d9b640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c2c0b4b20 0x7f9c2c0b2610 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9c1402c3b0 tx=0x7f9c14001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:00.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.505+0000 7f9c22ffd640 1 -- 172.21.15.27:0/3262925316 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c14039070 con 0x7f9c2c0b4b20 2023-12-19T18:20:00.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.505+0000 7f9c22ffd640 1 -- 172.21.15.27:0/3262925316 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c1402cc20 con 0x7f9c2c0b4b20 2023-12-19T18:20:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.505+0000 7f9c22ffd640 1 -- 172.21.15.27:0/3262925316 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1403c040 con 0x7f9c2c0b4b20 2023-12-19T18:20:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.505+0000 7f9c2b59e640 1 -- 172.21.15.27:0/3262925316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c2c0b5580 con 0x7f9c2c0b4b20 2023-12-19T18:20:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.505+0000 7f9c2b59e640 1 -- 172.21.15.27:0/3262925316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c2c1f6ab0 con 0x7f9c2c0b4b20 2023-12-19T18:20:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.506+0000 7f9c2b59e640 1 -- 172.21.15.27:0/3262925316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c2c04e840 con 0x7f9c2c0b4b20 2023-12-19T18:20:00.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.507+0000 7f9c22ffd640 1 -- 172.21.15.27:0/3262925316 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9c14032070 con 0x7f9c2c0b4b20 2023-12-19T18:20:00.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.508+0000 7f9c22ffd640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c0c06a460 0x7f9c0c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.508+0000 7f9c22ffd640 1 -- 172.21.15.27:0/3262925316 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9c140a9ce0 con 0x7f9c2c0b4b20 2023-12-19T18:20:00.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.509+0000 7f9c2a59c640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c0c06a460 0x7f9c0c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.510+0000 7f9c2a59c640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c0c06a460 0x7f9c0c06c920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009a40 tx=0x7f9c1c00b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:00.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.512+0000 7f9c22ffd640 1 -- 172.21.15.27:0/3262925316 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9c14075230 con 0x7f9c2c0b4b20 2023-12-19T18:20:00.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.697+0000 7f9c2b59e640 1 -- 172.21.15.27:0/3262925316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9c2c15e500 con 0x7f9c2c0b4b20 2023-12-19T18:20:00.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.699+0000 7f9c22ffd640 1 -- 172.21.15.27:0/3262925316 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16053 (secure 0 0 0) 0x7f9c14078850 con 0x7f9c2c0b4b20 2023-12-19T18:20:00.699 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:00.699 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":95,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:20:00.263800+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:19:58.591127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T18:19:59.313961+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:20:00.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.701+0000 7f9c2b59e640 1 -- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c0c06a460 msgr2=0x7f9c0c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.701+0000 7f9c2b59e640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c0c06a460 0x7f9c0c06c920 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c009a40 tx=0x7f9c1c00b040 comp rx=0 tx=0).stop 2023-12-19T18:20:00.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.701+0000 7f9c2b59e640 1 -- 172.21.15.27:0/3262925316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c2c0b4b20 msgr2=0x7f9c2c0b2610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.701+0000 7f9c2b59e640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c2c0b4b20 0x7f9c2c0b2610 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f9c1402c3b0 tx=0x7f9c14001930 comp rx=0 tx=0).stop 2023-12-19T18:20:00.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.702+0000 7f9c2b59e640 1 -- 172.21.15.27:0/3262925316 shutdown_connections 2023-12-19T18:20:00.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.702+0000 7f9c2b59e640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c0c06a460 0x7f9c0c06c920 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.702+0000 7f9c2b59e640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c2c0b2b50 0x7f9c2c1f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.702+0000 7f9c2b59e640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c2c0b4b20 0x7f9c2c0b2610 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.702+0000 7f9c2b59e640 1 --2- 172.21.15.27:0/3262925316 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c180047c0 0x7f9c2c0b45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.702+0000 7f9c2b59e640 1 -- 172.21.15.27:0/3262925316 >> 172.21.15.27:0/3262925316 conn(0x7f9c2c0cfaa0 msgr2=0x7f9c2c069c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:00.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.702+0000 7f9c2b59e640 1 -- 172.21.15.27:0/3262925316 shutdown_connections 2023-12-19T18:20:00.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.702+0000 7f9c2b59e640 1 -- 172.21.15.27:0/3262925316 wait complete. 2023-12-19T18:20:00.721 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:20:00.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.933+0000 7f581b59e640 1 Processor -- start 2023-12-19T18:20:00.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.933+0000 7f581b59e640 1 -- start start 2023-12-19T18:20:00.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.935+0000 7f581b59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f581c1553a0 0x7f581c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.935+0000 7f581b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f581c15ee30 0x7f581c0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.935+0000 7f581b59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f581c0b0360 con 0x7f581c0afeb0 2023-12-19T18:20:00.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.935+0000 7f581b59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f581c0cbcf0 con 0x7f581c1553a0 2023-12-19T18:20:00.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.935+0000 7f581b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0cb350 con 0x7f581c15ee30 2023-12-19T18:20:00.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.936+0000 7f581a59c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f581c1553a0 0x7f581c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.936+0000 7f5819d9b640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f581c0afeb0 0x7f581c15e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:35032/0 (socket says 172.21.15.27:35032) 2023-12-19T18:20:00.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.936+0000 7f581a59c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f581c1553a0 0x7f581c0c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:55024/0 (socket says 172.21.15.27:55024) 2023-12-19T18:20:00.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.936+0000 7f5819d9b640 1 -- 172.21.15.27:0/1496206148 learned_addr learned my addr 172.21.15.27:0/1496206148 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:00.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.936+0000 7f581ad9d640 1 --2- 172.21.15.27:0/1496206148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f581c15ee30 0x7f581c0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.937+0000 7f581ad9d640 1 -- 172.21.15.27:0/1496206148 >> v2:172.21.15.192:3300/0 conn(0x7f581c1553a0 msgr2=0x7f581c0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:00.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.937+0000 7f581ad9d640 1 --2- 172.21.15.27:0/1496206148 >> v2:172.21.15.192:3300/0 conn(0x7f581c1553a0 0x7f581c0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.937+0000 7f581ad9d640 1 -- 172.21.15.27:0/1496206148 >> v1:172.21.15.192:6789/0 conn(0x7f581c0afeb0 legacy=0x7f581c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.937+0000 7f581ad9d640 1 -- 172.21.15.27:0/1496206148 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0cb630 con 0x7f581c15ee30 2023-12-19T18:20:00.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.937+0000 7f581a59c640 1 --2- 172.21.15.27:0/1496206148 >> v2:172.21.15.192:3300/0 conn(0x7f581c1553a0 0x7f581c0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:20:00.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.937+0000 7f581959a640 1 -- 172.21.15.27:0/1496206148 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436865404 0 0) 0x7f581c0b0360 con 0x7f581c0afeb0 2023-12-19T18:20:00.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.938+0000 7f581ad9d640 1 --2- 172.21.15.27:0/1496206148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f581c15ee30 0x7f581c0c39d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5814009d90 tx=0x7f581402be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4ae52322e7704bf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:00.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.939+0000 7f581959a640 1 -- 172.21.15.27:0/1496206148 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581402cbd0 con 0x7f581c15ee30 2023-12-19T18:20:00.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.939+0000 7f581959a640 1 -- 172.21.15.27:0/1496206148 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581402cd30 con 0x7f581c15ee30 2023-12-19T18:20:00.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.939+0000 7f581959a640 1 -- 172.21.15.27:0/1496206148 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581402cea0 con 0x7f581c15ee30 2023-12-19T18:20:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.940+0000 7f581b59e640 1 -- 172.21.15.27:0/1496206148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f581c15ee30 msgr2=0x7f581c0c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.940+0000 7f581b59e640 1 --2- 172.21.15.27:0/1496206148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f581c15ee30 0x7f581c0c39d0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5814009d90 tx=0x7f581402be20 comp rx=0 tx=0).stop 2023-12-19T18:20:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.940+0000 7f581b59e640 1 -- 172.21.15.27:0/1496206148 shutdown_connections 2023-12-19T18:20:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.940+0000 7f581b59e640 1 --2- 172.21.15.27:0/1496206148 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f581c15ee30 0x7f581c0c39d0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.940+0000 7f581b59e640 1 --2- 172.21.15.27:0/1496206148 >> v2:172.21.15.192:3300/0 conn(0x7f581c1553a0 0x7f581c0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.940+0000 7f581b59e640 1 -- 172.21.15.27:0/1496206148 >> 172.21.15.27:0/1496206148 conn(0x7f581c069c70 msgr2=0x7f581c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.940+0000 7f581b59e640 1 -- 172.21.15.27:0/1496206148 shutdown_connections 2023-12-19T18:20:00.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.940+0000 7f581b59e640 1 -- 172.21.15.27:0/1496206148 wait complete. 2023-12-19T18:20:00.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.942+0000 7f581b59e640 1 Processor -- start 2023-12-19T18:20:00.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.942+0000 7f581b59e640 1 -- start start 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.944+0000 7f581b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f581c0afeb0 0x7f581c0c6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.944+0000 7f581b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f581c1553a0 0x7f581c06bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.944+0000 7f581b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f581c15ee30 0x7f581c06c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.944+0000 7f581b59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0c68d0 con 0x7f581c1553a0 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.944+0000 7f581b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0b22b0 con 0x7f581c15ee30 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.944+0000 7f581b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f581c0b2590 con 0x7f581c0afeb0 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.944+0000 7f5819d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f581c1553a0 0x7f581c06bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.944+0000 7f5819d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f581c1553a0 0x7f581c06bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58448/0 (socket says 172.21.15.27:58448) 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.944+0000 7f5819d9b640 1 -- 172.21.15.27:0/448015749 learned_addr learned my addr 172.21.15.27:0/448015749 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.944+0000 7f581a59c640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f581c0afeb0 0x7f581c0c6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.944+0000 7f581ad9d640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f581c15ee30 0x7f581c06c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.945+0000 7f5819d9b640 1 -- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f581c0afeb0 msgr2=0x7f581c0c6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.945+0000 7f5819d9b640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f581c0afeb0 0x7f581c0c6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.945+0000 7f5819d9b640 1 -- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f581c15ee30 msgr2=0x7f581c06c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:00.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.945+0000 7f5819d9b640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f581c15ee30 0x7f581c06c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:00.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.945+0000 7f5819d9b640 1 -- 172.21.15.27:0/448015749 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580c009400 con 0x7f581c1553a0 2023-12-19T18:20:00.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.945+0000 7f5819d9b640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f581c1553a0 0x7f581c06bcc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f581000c4d0 tx=0x7f581000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:00.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.945+0000 7f580affd640 1 -- 172.21.15.27:0/448015749 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58100074e0 con 0x7f581c1553a0 2023-12-19T18:20:00.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.945+0000 7f580affd640 1 -- 172.21.15.27:0/448015749 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5810007660 con 0x7f581c1553a0 2023-12-19T18:20:00.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.945+0000 7f581b59e640 1 -- 172.21.15.27:0/448015749 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5814009a70 con 0x7f581c1553a0 2023-12-19T18:20:00.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.945+0000 7f580affd640 1 -- 172.21.15.27:0/448015749 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5810004a20 con 0x7f581c1553a0 2023-12-19T18:20:00.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.945+0000 7f581b59e640 1 -- 172.21.15.27:0/448015749 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f581c0b2b50 con 0x7f581c1553a0 2023-12-19T18:20:00.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.946+0000 7f581b59e640 1 -- 172.21.15.27:0/448015749 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f581c04e840 con 0x7f581c1553a0 2023-12-19T18:20:00.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.948+0000 7f580affd640 1 -- 172.21.15.27:0/448015749 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5810004b80 con 0x7f581c1553a0 2023-12-19T18:20:00.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.949+0000 7f580affd640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f57fc06a460 0x7f57fc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:00.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.949+0000 7f580affd640 1 -- 172.21.15.27:0/448015749 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5810087d40 con 0x7f581c1553a0 2023-12-19T18:20:00.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.949+0000 7f581a59c640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f57fc06a460 0x7f57fc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:00.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.950+0000 7f581a59c640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f57fc06a460 0x7f57fc06c920 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f581c0c9a10 tx=0x7f580c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:00.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:00.952+0000 7f580affd640 1 -- 172.21.15.27:0/448015749 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f58100533d0 con 0x7f581c1553a0 2023-12-19T18:20:01.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.138+0000 7f581b59e640 1 -- 172.21.15.27:0/448015749 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f581c0af6a0 con 0x7f581c1553a0 2023-12-19T18:20:01.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.140+0000 7f580affd640 1 -- 172.21.15.27:0/448015749 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16053 (secure 0 0 0) 0x7f5810018090 con 0x7f581c1553a0 2023-12-19T18:20:01.140 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:01.140 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":95,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:20:00.263800+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:19:58.591127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T18:19:59.313961+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:20:01.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.142+0000 7f581b59e640 1 -- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f57fc06a460 msgr2=0x7f57fc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:01.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.142+0000 7f581b59e640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f57fc06a460 0x7f57fc06c920 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f581c0c9a10 tx=0x7f580c009190 comp rx=0 tx=0).stop 2023-12-19T18:20:01.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.142+0000 7f581b59e640 1 -- 172.21.15.27:0/448015749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f581c1553a0 msgr2=0x7f581c06bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.142+0000 7f581b59e640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f581c1553a0 0x7f581c06bcc0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f581000c4d0 tx=0x7f581000c990 comp rx=0 tx=0).stop 2023-12-19T18:20:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.142+0000 7f581b59e640 1 -- 172.21.15.27:0/448015749 shutdown_connections 2023-12-19T18:20:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.142+0000 7f581b59e640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f57fc06a460 0x7f57fc06c920 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.142+0000 7f581b59e640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f581c15ee30 0x7f581c06c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.142+0000 7f581b59e640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f581c1553a0 0x7f581c06bcc0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.142+0000 7f581b59e640 1 --2- 172.21.15.27:0/448015749 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f581c0afeb0 0x7f581c0c6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.142+0000 7f581b59e640 1 -- 172.21.15.27:0/448015749 >> 172.21.15.27:0/448015749 conn(0x7f581c069c70 msgr2=0x7f581c0cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.142+0000 7f581b59e640 1 -- 172.21.15.27:0/448015749 shutdown_connections 2023-12-19T18:20:01.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.142+0000 7f581b59e640 1 -- 172.21.15.27:0/448015749 wait complete. 2023-12-19T18:20:01.161 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:20:01.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.392+0000 7f44b8f73640 1 Processor -- start 2023-12-19T18:20:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.392+0000 7f44b8f73640 1 -- start start 2023-12-19T18:20:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.393+0000 7f44b8f73640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f44b415b1a0 0x7f44b406f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.393+0000 7f44b8f73640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b406bc80 con 0x7f44b40c2720 2023-12-19T18:20:01.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.393+0000 7f44b8f73640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44b406d690 con 0x7f44b4071420 2023-12-19T18:20:01.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.394+0000 7f44b8f73640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44b406d0a0 con 0x7f44b415b1a0 2023-12-19T18:20:01.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.394+0000 7f44abfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f44b40c2720 0x7f44b406fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40766/0 (socket says 172.21.15.27:40766) 2023-12-19T18:20:01.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.394+0000 7f44abfff640 1 -- 172.21.15.27:0/1380040427 learned_addr learned my addr 172.21.15.27:0/1380040427 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:01.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.394+0000 7f44b259c640 1 --2- 172.21.15.27:0/1380040427 >> v2:172.21.15.27:3300/0 conn(0x7f44b415b1a0 0x7f44b406f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:01.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.394+0000 7f44b1d9b640 1 -- 172.21.15.27:0/1380040427 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895230047 0 0) 0x7f449c001110 con 0x7f44b40c2720 2023-12-19T18:20:01.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.394+0000 7f44b1d9b640 1 -- 172.21.15.27:0/1380040427 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44a0003520 con 0x7f44b40c2720 2023-12-19T18:20:01.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.394+0000 7f44b259c640 1 -- 172.21.15.27:0/1380040427 >> v1:172.21.15.192:6789/0 conn(0x7f44b4071420 legacy=0x7f44b415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:01.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.394+0000 7f44b259c640 1 -- 172.21.15.27:0/1380040427 >> v1:172.21.15.27:6789/0 conn(0x7f44b40c2720 legacy=0x7f44b406fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:01.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.394+0000 7f44b259c640 1 -- 172.21.15.27:0/1380040427 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b406b6c0 con 0x7f44b415b1a0 2023-12-19T18:20:01.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.396+0000 7f44b259c640 1 --2- 172.21.15.27:0/1380040427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44b415b1a0 0x7f44b406f520 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f44a40097a0 tx=0x7f44a402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4a808e9752b609 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:01.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.396+0000 7f44b1d9b640 1 -- 172.21.15.27:0/1380040427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a402e070 con 0x7f44b415b1a0 2023-12-19T18:20:01.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.396+0000 7f44b1d9b640 1 -- 172.21.15.27:0/1380040427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44a40315a0 con 0x7f44b415b1a0 2023-12-19T18:20:01.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.396+0000 7f44b1d9b640 1 -- 172.21.15.27:0/1380040427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44a4031910 con 0x7f44b415b1a0 2023-12-19T18:20:01.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.396+0000 7f44b8f73640 1 -- 172.21.15.27:0/1380040427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44b415b1a0 msgr2=0x7f44b406f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:01.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.396+0000 7f44b8f73640 1 --2- 172.21.15.27:0/1380040427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44b415b1a0 0x7f44b406f520 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f44a40097a0 tx=0x7f44a402b840 comp rx=0 tx=0).stop 2023-12-19T18:20:01.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.397+0000 7f44b8f73640 1 -- 172.21.15.27:0/1380040427 shutdown_connections 2023-12-19T18:20:01.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.397+0000 7f44b8f73640 1 --2- 172.21.15.27:0/1380040427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44b415b1a0 0x7f44b406f520 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:01.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.397+0000 7f44b8f73640 1 -- 172.21.15.27:0/1380040427 >> 172.21.15.27:0/1380040427 conn(0x7f44b40cebd0 msgr2=0x7f44b40cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:01.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.397+0000 7f44b8f73640 1 -- 172.21.15.27:0/1380040427 shutdown_connections 2023-12-19T18:20:01.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.397+0000 7f44b8f73640 1 -- 172.21.15.27:0/1380040427 wait complete. 2023-12-19T18:20:01.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.398+0000 7f44b8f73640 1 Processor -- start 2023-12-19T18:20:01.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.398+0000 7f44b8f73640 1 -- start start 2023-12-19T18:20:01.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.399+0000 7f44b8f73640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b4071420 0x7f44b40c9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:01.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.399+0000 7f44b8f73640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44b40c2720 0x7f44b40c7780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:01.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.399+0000 7f44b8f73640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44b415b1a0 0x7f44b40cde00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:01.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.399+0000 7f44b8f73640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40c7360 con 0x7f44b40c2720 2023-12-19T18:20:01.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.399+0000 7f44b8f73640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40c66e0 con 0x7f44b4071420 2023-12-19T18:20:01.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.399+0000 7f44b8f73640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44b40c69c0 con 0x7f44b415b1a0 2023-12-19T18:20:01.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.400+0000 7f44abfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44b40c2720 0x7f44b40c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:01.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.400+0000 7f44abfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44b40c2720 0x7f44b40c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58464/0 (socket says 172.21.15.27:58464) 2023-12-19T18:20:01.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.400+0000 7f44abfff640 1 -- 172.21.15.27:0/4095319443 learned_addr learned my addr 172.21.15.27:0/4095319443 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:01.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.400+0000 7f44b259c640 1 --2- 172.21.15.27:0/4095319443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b4071420 0x7f44b40c9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:01.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.400+0000 7f44abfff640 1 -- 172.21.15.27:0/4095319443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44b415b1a0 msgr2=0x7f44b40cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:01.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.400+0000 7f44abfff640 1 --2- 172.21.15.27:0/4095319443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44b415b1a0 0x7f44b40cde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:01.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.400+0000 7f44abfff640 1 -- 172.21.15.27:0/4095319443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b4071420 msgr2=0x7f44b40c9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:01.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.400+0000 7f44abfff640 1 --2- 172.21.15.27:0/4095319443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b4071420 0x7f44b40c9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:01.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.400+0000 7f44abfff640 1 -- 172.21.15.27:0/4095319443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a4009400 con 0x7f44b40c2720 2023-12-19T18:20:01.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.400+0000 7f44abfff640 1 --2- 172.21.15.27:0/4095319443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44b40c2720 0x7f44b40c7780 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f449c00c700 tx=0x7f449c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:01.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.401+0000 7f44aaffd640 1 -- 172.21.15.27:0/4095319443 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c005ce0 con 0x7f44b40c2720 2023-12-19T18:20:01.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.401+0000 7f44aaffd640 1 -- 172.21.15.27:0/4095319443 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c005e40 con 0x7f44b40c2720 2023-12-19T18:20:01.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.401+0000 7f44b8f73640 1 -- 172.21.15.27:0/4095319443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b40c6ca0 con 0x7f44b40c2720 2023-12-19T18:20:01.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.401+0000 7f44aaffd640 1 -- 172.21.15.27:0/4095319443 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c007290 con 0x7f44b40c2720 2023-12-19T18:20:01.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.401+0000 7f44b8f73640 1 -- 172.21.15.27:0/4095319443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b40c7cc0 con 0x7f44b40c2720 2023-12-19T18:20:01.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.402+0000 7f44b8f73640 1 -- 172.21.15.27:0/4095319443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b40c74a0 con 0x7f44b40c2720 2023-12-19T18:20:01.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.403+0000 7f44aaffd640 1 -- 172.21.15.27:0/4095319443 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f449c0074b0 con 0x7f44b40c2720 2023-12-19T18:20:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.404+0000 7f44aaffd640 1 --2- 172.21.15.27:0/4095319443 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f449406a430 0x7f449406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:01.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.404+0000 7f44aaffd640 1 -- 172.21.15.27:0/4095319443 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f449c087cd0 con 0x7f44b40c2720 2023-12-19T18:20:01.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.405+0000 7f44b259c640 1 --2- 172.21.15.27:0/4095319443 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f449406a430 0x7f449406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:01.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.406+0000 7f44b259c640 1 --2- 172.21.15.27:0/4095319443 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f449406a430 0x7f449406c8f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f44a4036000 tx=0x7f44a403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:01.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.409+0000 7f44aaffd640 1 -- 172.21.15.27:0/4095319443 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f44b40c74a0 con 0x7f44b40c2720 2023-12-19T18:20:01.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:01.631+0000 7f44b8f73640 1 -- 172.21.15.27:0/4095319443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f44b406bc80 con 0x7f44b40c2720 2023-12-19T18:20:02.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.627+0000 7f44aaffd640 1 -- 172.21.15.27:0/4095319443 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v37) v1 ==== 118+0+0 (secure 0 0 0) 0x7f44b406bc80 con 0x7f44b40c2720 2023-12-19T18:20:02.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.631+0000 7f44b8f73640 1 -- 172.21.15.27:0/4095319443 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f449406a430 msgr2=0x7f449406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:02.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.631+0000 7f44b8f73640 1 --2- 172.21.15.27:0/4095319443 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f449406a430 0x7f449406c8f0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f44a4036000 tx=0x7f44a403d040 comp rx=0 tx=0).stop 2023-12-19T18:20:02.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.631+0000 7f44b8f73640 1 -- 172.21.15.27:0/4095319443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44b40c2720 msgr2=0x7f44b40c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:02.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.631+0000 7f44b8f73640 1 --2- 172.21.15.27:0/4095319443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44b40c2720 0x7f44b40c7780 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f449c00c700 tx=0x7f449c00cb00 comp rx=0 tx=0).stop 2023-12-19T18:20:02.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.631+0000 7f44b8f73640 1 -- 172.21.15.27:0/4095319443 shutdown_connections 2023-12-19T18:20:02.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.631+0000 7f44b8f73640 1 --2- 172.21.15.27:0/4095319443 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f449406a430 0x7f449406c8f0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:02.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.631+0000 7f44b8f73640 1 --2- 172.21.15.27:0/4095319443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f44b415b1a0 0x7f44b40cde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:02.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.631+0000 7f44b8f73640 1 --2- 172.21.15.27:0/4095319443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44b40c2720 0x7f44b40c7780 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:02.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.631+0000 7f44b8f73640 1 --2- 172.21.15.27:0/4095319443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f44b4071420 0x7f44b40c9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:02.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.631+0000 7f44b8f73640 1 -- 172.21.15.27:0/4095319443 >> 172.21.15.27:0/4095319443 conn(0x7f44b40cebd0 msgr2=0x7f44b40aef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:02.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.631+0000 7f44b8f73640 1 -- 172.21.15.27:0/4095319443 shutdown_connections 2023-12-19T18:20:02.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.631+0000 7f44b8f73640 1 -- 172.21.15.27:0/4095319443 wait complete. 2023-12-19T18:20:02.653 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:20:02.653 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:20:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.898+0000 7f630c42c640 1 Processor -- start 2023-12-19T18:20:02.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.898+0000 7f630c42c640 1 -- start start 2023-12-19T18:20:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.900+0000 7f630c42c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63040cf9b0 0x7f63040cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.900+0000 7f630c42c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63040ce540 con 0x7f63040cda20 2023-12-19T18:20:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.900+0000 7f630c42c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63040d26d0 con 0x7f63040b8c30 2023-12-19T18:20:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.900+0000 7f630c42c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63040cea80 con 0x7f63040cf9b0 2023-12-19T18:20:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.900+0000 7f63099c7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63040cf9b0 0x7f63040cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.900+0000 7f630a9c9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f63040cda20 0x7f63040cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36366/0 (socket says 172.21.15.27:36366) 2023-12-19T18:20:02.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.900+0000 7f630a9c9640 1 -- 172.21.15.27:0/3988070201 learned_addr learned my addr 172.21.15.27:0/3988070201 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:02.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.901+0000 7f63099c7640 1 -- 172.21.15.27:0/3988070201 >> v1:172.21.15.192:6789/0 conn(0x7f63040b8c30 legacy=0x7f63040d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:02.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.901+0000 7f63099c7640 1 -- 172.21.15.27:0/3988070201 >> v1:172.21.15.27:6789/0 conn(0x7f63040cda20 legacy=0x7f63040cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:02.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.901+0000 7f63099c7640 1 -- 172.21.15.27:0/3988070201 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63040ce7a0 con 0x7f63040cf9b0 2023-12-19T18:20:02.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.902+0000 7f63099c7640 1 --2- 172.21.15.27:0/3988070201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63040cf9b0 0x7f63040cfe40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f62f40097a0 tx=0x7f62f402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=791b34366d96bfb9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:02.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.902+0000 7f63091c6640 1 -- 172.21.15.27:0/3988070201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f402e070 con 0x7f63040cf9b0 2023-12-19T18:20:02.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.902+0000 7f63091c6640 1 -- 172.21.15.27:0/3988070201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f40315a0 con 0x7f63040cf9b0 2023-12-19T18:20:02.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.902+0000 7f63091c6640 1 -- 172.21.15.27:0/3988070201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4031930 con 0x7f63040cf9b0 2023-12-19T18:20:02.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.903+0000 7f630c42c640 1 -- 172.21.15.27:0/3988070201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63040cf9b0 msgr2=0x7f63040cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:02.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.903+0000 7f630c42c640 1 --2- 172.21.15.27:0/3988070201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63040cf9b0 0x7f63040cfe40 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f62f40097a0 tx=0x7f62f402b840 comp rx=0 tx=0).stop 2023-12-19T18:20:02.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.903+0000 7f630c42c640 1 -- 172.21.15.27:0/3988070201 shutdown_connections 2023-12-19T18:20:02.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.903+0000 7f630c42c640 1 --2- 172.21.15.27:0/3988070201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63040cf9b0 0x7f63040cfe40 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:02.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.903+0000 7f630c42c640 1 -- 172.21.15.27:0/3988070201 >> 172.21.15.27:0/3988070201 conn(0x7f63040d2fa0 msgr2=0x7f63040d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:02.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.903+0000 7f630c42c640 1 -- 172.21.15.27:0/3988070201 shutdown_connections 2023-12-19T18:20:02.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.903+0000 7f630c42c640 1 -- 172.21.15.27:0/3988070201 wait complete. 2023-12-19T18:20:02.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.905+0000 7f630c42c640 1 Processor -- start 2023-12-19T18:20:02.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.905+0000 7f630c42c640 1 -- start start 2023-12-19T18:20:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.906+0000 7f630c42c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f63040b8c30 0x7f6304145cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.906+0000 7f630c42c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63040cda20 0x7f630415b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.906+0000 7f630c42c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63040cf9b0 0x7f63041465b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.906+0000 7f630c42c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630415b210 con 0x7f63040cda20 2023-12-19T18:20:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.906+0000 7f630c42c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630415aee0 con 0x7f63040cf9b0 2023-12-19T18:20:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.906+0000 7f630c42c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6304156fc0 con 0x7f63040b8c30 2023-12-19T18:20:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.906+0000 7f63099c7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63040cda20 0x7f630415b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.907+0000 7f63099c7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63040cda20 0x7f630415b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50980/0 (socket says 172.21.15.27:50980) 2023-12-19T18:20:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.907+0000 7f63099c7640 1 -- 172.21.15.27:0/2575600793 learned_addr learned my addr 172.21.15.27:0/2575600793 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:02.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.907+0000 7f630a1c8640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f63040b8c30 0x7f6304145cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.907+0000 7f630a9c9640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63040cf9b0 0x7f63041465b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.907+0000 7f63099c7640 1 -- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f63040b8c30 msgr2=0x7f6304145cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.907+0000 7f63099c7640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f63040b8c30 0x7f6304145cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.907+0000 7f63099c7640 1 -- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63040cf9b0 msgr2=0x7f63041465b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.907+0000 7f63099c7640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63040cf9b0 0x7f63041465b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.907+0000 7f63099c7640 1 -- 172.21.15.27:0/2575600793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f4009400 con 0x7f63040cda20 2023-12-19T18:20:02.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.907+0000 7f630a9c9640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63040cf9b0 0x7f63041465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.907+0000 7f63099c7640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63040cda20 0x7f630415b6a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f62f4034040 tx=0x7f62f40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.908+0000 7f62faffd640 1 -- 172.21.15.27:0/2575600793 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4032900 con 0x7f63040cda20 2023-12-19T18:20:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.908+0000 7f62faffd640 1 -- 172.21.15.27:0/2575600793 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62f4032a80 con 0x7f63040cda20 2023-12-19T18:20:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.908+0000 7f62faffd640 1 -- 172.21.15.27:0/2575600793 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4032e30 con 0x7f63040cda20 2023-12-19T18:20:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.908+0000 7f630c42c640 1 -- 172.21.15.27:0/2575600793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63041572a0 con 0x7f63040cda20 2023-12-19T18:20:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.908+0000 7f630c42c640 1 -- 172.21.15.27:0/2575600793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6304157580 con 0x7f63040cda20 2023-12-19T18:20:02.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.909+0000 7f630c42c640 1 -- 172.21.15.27:0/2575600793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63040d1a80 con 0x7f63040cda20 2023-12-19T18:20:02.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.910+0000 7f62faffd640 1 -- 172.21.15.27:0/2575600793 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f62f403b3f0 con 0x7f63040cda20 2023-12-19T18:20:02.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.911+0000 7f62faffd640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62e806a430 0x7f62e806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:02.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.911+0000 7f62faffd640 1 -- 172.21.15.27:0/2575600793 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62f40aa9c0 con 0x7f63040cda20 2023-12-19T18:20:02.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.912+0000 7f630a1c8640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62e806a430 0x7f62e806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:02.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.913+0000 7f630a1c8640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62e806a430 0x7f62e806c8f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6300005520 tx=0x7f6300009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:02.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:02.916+0000 7f62faffd640 1 -- 172.21.15.27:0/2575600793 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f63040d1a80 con 0x7f63040cda20 2023-12-19T18:20:03.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.147+0000 7f630c42c640 1 -- 172.21.15.27:0/2575600793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f63040d1a80 con 0x7f63040cda20 2023-12-19T18:20:03.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.802+0000 7f62faffd640 1 -- 172.21.15.27:0/2575600793 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v38) v1 ==== 122+0+0 (secure 0 0 0) 0x7f63040d1a80 con 0x7f63040cda20 2023-12-19T18:20:03.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.805+0000 7f630c42c640 1 -- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62e806a430 msgr2=0x7f62e806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:03.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.805+0000 7f630c42c640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62e806a430 0x7f62e806c8f0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6300005520 tx=0x7f6300009040 comp rx=0 tx=0).stop 2023-12-19T18:20:03.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.806+0000 7f630c42c640 1 -- 172.21.15.27:0/2575600793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63040cda20 msgr2=0x7f630415b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:03.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.806+0000 7f630c42c640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63040cda20 0x7f630415b6a0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f62f4034040 tx=0x7f62f40046c0 comp rx=0 tx=0).stop 2023-12-19T18:20:03.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.806+0000 7f630c42c640 1 -- 172.21.15.27:0/2575600793 shutdown_connections 2023-12-19T18:20:03.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.806+0000 7f630c42c640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62e806a430 0x7f62e806c8f0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:03.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.806+0000 7f630c42c640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63040cf9b0 0x7f63041465b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:03.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.806+0000 7f630c42c640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63040cda20 0x7f630415b6a0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:03.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.806+0000 7f630c42c640 1 --2- 172.21.15.27:0/2575600793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f63040b8c30 0x7f6304145cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:03.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.806+0000 7f630c42c640 1 -- 172.21.15.27:0/2575600793 >> 172.21.15.27:0/2575600793 conn(0x7f63040d2fa0 msgr2=0x7f63040d1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:03.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.806+0000 7f630c42c640 1 -- 172.21.15.27:0/2575600793 shutdown_connections 2023-12-19T18:20:03.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:03.806+0000 7f630c42c640 1 -- 172.21.15.27:0/2575600793 wait complete. 2023-12-19T18:20:03.829 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:20:03.830 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:20:04.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.063+0000 7f22f08df640 1 Processor -- start 2023-12-19T18:20:04.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.063+0000 7f22f08df640 1 -- start start 2023-12-19T18:20:04.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.064+0000 7f22f08df640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f22ec15ceb0 0x7f22ec0b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:04.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.065+0000 7f22f08df640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ec0b5170 con 0x7f22ec0b4d10 2023-12-19T18:20:04.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.065+0000 7f22f08df640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22ec0b6540 con 0x7f22ec0d2130 2023-12-19T18:20:04.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.065+0000 7f22f08df640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22ec0cdfb0 con 0x7f22ec15ceb0 2023-12-19T18:20:04.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.065+0000 7f22e3fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f22ec0b4d10 0x7f22ec0d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36382/0 (socket says 172.21.15.27:36382) 2023-12-19T18:20:04.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.065+0000 7f22e3fff640 1 -- 172.21.15.27:0/3485235668 learned_addr learned my addr 172.21.15.27:0/3485235668 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:04.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.065+0000 7f22ead9d640 1 --2- 172.21.15.27:0/3485235668 >> v2:172.21.15.192:3300/0 conn(0x7f22ec15ceb0 0x7f22ec0b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:04.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.065+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287366127 0 0) 0x7f22ec0b5170 con 0x7f22ec0b4d10 2023-12-19T18:20:04.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.065+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f22d8003520 con 0x7f22ec0b4d10 2023-12-19T18:20:04.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4010433193 0 0) 0x7f22d8003520 con 0x7f22ec0b4d10 2023-12-19T18:20:04.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 >> v2:172.21.15.192:3300/0 conn(0x7f22ec15ceb0 msgr2=0x7f22ec0b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:04.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22ea59c640 1 --2- 172.21.15.27:0/3485235668 >> v2:172.21.15.192:3300/0 conn(0x7f22ec15ceb0 0x7f22ec0b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 >> v1:172.21.15.192:6789/0 conn(0x7f22ec0d2130 legacy=0x7f22ec0b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:04.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec0d2600 con 0x7f22ec0b4d10 2023-12-19T18:20:04.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f22d4001670 con 0x7f22ec0b4d10 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 >> v1:172.21.15.27:6789/0 conn(0x7f22ec0b4d10 legacy=0x7f22ec0d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22ea59c640 1 --2- 172.21.15.27:0/3485235668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d80039c0 0x7f22d8003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22ea59c640 1 --2- 172.21.15.27:0/3485235668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d80047c0 0x7f22d8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22ea59c640 1 --2- 172.21.15.27:0/3485235668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d8005180 0x7f22d800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0cdfb0 con 0x7f22d8005180 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d800c080 con 0x7f22d80047c0 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d800c240 con 0x7f22d80039c0 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.066+0000 7f22eb59e640 1 --2- 172.21.15.27:0/3485235668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d8005180 0x7f22d800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.067+0000 7f22ead9d640 1 --2- 172.21.15.27:0/3485235668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d80039c0 0x7f22d8003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.067+0000 7f22e3fff640 1 --2- 172.21.15.27:0/3485235668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d80047c0 0x7f22d8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.067+0000 7f22eb59e640 1 -- 172.21.15.27:0/3485235668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d80039c0 msgr2=0x7f22d8003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.067+0000 7f22eb59e640 1 --2- 172.21.15.27:0/3485235668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d80039c0 0x7f22d8003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.067+0000 7f22eb59e640 1 -- 172.21.15.27:0/3485235668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d80047c0 msgr2=0x7f22d8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:04.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.067+0000 7f22eb59e640 1 --2- 172.21.15.27:0/3485235668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22d80047c0 0x7f22d8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.067+0000 7f22eb59e640 1 -- 172.21.15.27:0/3485235668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d8003520 con 0x7f22d8005180 2023-12-19T18:20:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.068+0000 7f22eb59e640 1 -- 172.21.15.27:0/3485235668 reap_dead start 2023-12-19T18:20:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.068+0000 7f22eb59e640 1 --2- 172.21.15.27:0/3485235668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d8005180 0x7f22d800b980 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f22e4001e50 tx=0x7f22d80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5ca5d7a5aa96c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.068+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4003b70 con 0x7f22d8005180 2023-12-19T18:20:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.068+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22e4003d10 con 0x7f22d8005180 2023-12-19T18:20:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.068+0000 7f22ea59c640 1 -- 172.21.15.27:0/3485235668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22e4005310 con 0x7f22d8005180 2023-12-19T18:20:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.069+0000 7f22f08df640 1 -- 172.21.15.27:0/3485235668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d8005180 msgr2=0x7f22d800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.069+0000 7f22f08df640 1 --2- 172.21.15.27:0/3485235668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d8005180 0x7f22d800b980 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f22e4001e50 tx=0x7f22d80057f0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.069+0000 7f22f08df640 1 -- 172.21.15.27:0/3485235668 shutdown_connections 2023-12-19T18:20:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.069+0000 7f22f08df640 1 --2- 172.21.15.27:0/3485235668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22d8005180 0x7f22d800b980 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.069+0000 7f22f08df640 1 -- 172.21.15.27:0/3485235668 >> 172.21.15.27:0/3485235668 conn(0x7f22ec0cf7d0 msgr2=0x7f22ec0cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.069+0000 7f22f08df640 1 -- 172.21.15.27:0/3485235668 shutdown_connections 2023-12-19T18:20:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.069+0000 7f22f08df640 1 -- 172.21.15.27:0/3485235668 wait complete. 2023-12-19T18:20:04.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.071+0000 7f22f08df640 1 Processor -- start 2023-12-19T18:20:04.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.071+0000 7f22f08df640 1 -- start start 2023-12-19T18:20:04.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.072+0000 7f22f08df640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d8005180 0x7f22ec0aa0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:04.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.072+0000 7f22f08df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22ec0cbd30 0x7f22ec0d1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:04.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.072+0000 7f22f08df640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22ec14ebf0 0x7f22ec14f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:04.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.072+0000 7f22f08df640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0ab530 con 0x7f22ec0cbd30 2023-12-19T18:20:04.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.072+0000 7f22f08df640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0aa920 con 0x7f22ec14ebf0 2023-12-19T18:20:04.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.072+0000 7f22f08df640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22ec0aac00 con 0x7f22d8005180 2023-12-19T18:20:04.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.072+0000 7f22e3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22ec0cbd30 0x7f22ec0d1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:04.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.072+0000 7f22e3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22ec0cbd30 0x7f22ec0d1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51004/0 (socket says 172.21.15.27:51004) 2023-12-19T18:20:04.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.072+0000 7f22e3fff640 1 -- 172.21.15.27:0/3913446578 learned_addr learned my addr 172.21.15.27:0/3913446578 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:04.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.073+0000 7f22ead9d640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d8005180 0x7f22ec0aa0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:04.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.073+0000 7f22e3fff640 1 -- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d8005180 msgr2=0x7f22ec0aa0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:04.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.073+0000 7f22e3fff640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d8005180 0x7f22ec0aa0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.073+0000 7f22eb59e640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22ec14ebf0 0x7f22ec14f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:04.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.073+0000 7f22e3fff640 1 -- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22ec14ebf0 msgr2=0x7f22ec14f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:04.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.073+0000 7f22e3fff640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22ec14ebf0 0x7f22ec14f070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.073+0000 7f22e3fff640 1 -- 172.21.15.27:0/3913446578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22ec0aaee0 con 0x7f22ec0cbd30 2023-12-19T18:20:04.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.073+0000 7f22e3fff640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22ec0cbd30 0x7f22ec0d1280 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f22d4001050 tx=0x7f22d400eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:04.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.073+0000 7f22e37fe640 1 -- 172.21.15.27:0/3913446578 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d400c660 con 0x7f22ec0cbd30 2023-12-19T18:20:04.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.073+0000 7f22e37fe640 1 -- 172.21.15.27:0/3913446578 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d400c7e0 con 0x7f22ec0cbd30 2023-12-19T18:20:04.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.073+0000 7f22e37fe640 1 -- 172.21.15.27:0/3913446578 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d400cb90 con 0x7f22ec0cbd30 2023-12-19T18:20:04.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.074+0000 7f22f08df640 1 -- 172.21.15.27:0/3913446578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ec0ab1c0 con 0x7f22ec0cbd30 2023-12-19T18:20:04.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.074+0000 7f22f08df640 1 -- 172.21.15.27:0/3913446578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ec0aa610 con 0x7f22ec0cbd30 2023-12-19T18:20:04.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.075+0000 7f22f08df640 1 -- 172.21.15.27:0/3913446578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ec0ab6e0 con 0x7f22ec0cbd30 2023-12-19T18:20:04.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.076+0000 7f22e37fe640 1 -- 172.21.15.27:0/3913446578 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f22d400ccf0 con 0x7f22ec0cbd30 2023-12-19T18:20:04.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.077+0000 7f22e37fe640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22cc06a430 0x7f22cc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:04.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.077+0000 7f22e37fe640 1 -- 172.21.15.27:0/3913446578 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f22d4088810 con 0x7f22ec0cbd30 2023-12-19T18:20:04.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.077+0000 7f22ead9d640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22cc06a430 0x7f22cc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:04.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.078+0000 7f22ead9d640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22cc06a430 0x7f22cc06c8f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0098d0 tx=0x7f22dc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:04.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.081+0000 7f22e37fe640 1 -- 172.21.15.27:0/3913446578 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f22ec0ab6e0 con 0x7f22ec0cbd30 2023-12-19T18:20:04.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.305+0000 7f22f08df640 1 -- 172.21.15.27:0/3913446578 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22ec0ab6e0 con 0x7f22ec0cbd30 2023-12-19T18:20:04.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.306+0000 7f22e37fe640 1 -- 172.21.15.27:0/3913446578 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f22ec0ab6e0 con 0x7f22ec0cbd30 2023-12-19T18:20:04.306 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:04.306 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2860412135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2860412135},{"type":"v1","addr":"172.21.15.192:6837","nonce":2860412135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/797098667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":797098667},{"type":"v1","addr":"172.21.15.192:6836","nonce":797098667}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":6507,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/4009181263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4009181263},{"type":"v1","addr":"172.21.15.27:6833","nonce":4009181263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:20:00.264651+0000","modified":"2023-12-19T18:20:03.796309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6510},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6510":{"gid":6510,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/202939736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":202939736},{"type":"v1","addr":"172.21.15.27:6835","nonce":202939736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-19T18:20:04.306 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2023-12-19T18:20:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.308+0000 7f22f08df640 1 -- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22cc06a430 msgr2=0x7f22cc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.308+0000 7f22f08df640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22cc06a430 0x7f22cc06c8f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f22dc0098d0 tx=0x7f22dc008040 comp rx=0 tx=0).stop 2023-12-19T18:20:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.308+0000 7f22f08df640 1 -- 172.21.15.27:0/3913446578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22ec0cbd30 msgr2=0x7f22ec0d1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.308+0000 7f22f08df640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22ec0cbd30 0x7f22ec0d1280 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f22d4001050 tx=0x7f22d400eba0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.309+0000 7f22f08df640 1 -- 172.21.15.27:0/3913446578 shutdown_connections 2023-12-19T18:20:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.309+0000 7f22f08df640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22cc06a430 0x7f22cc06c8f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.309+0000 7f22f08df640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22ec14ebf0 0x7f22ec14f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.309+0000 7f22f08df640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22ec0cbd30 0x7f22ec0d1280 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.309+0000 7f22f08df640 1 --2- 172.21.15.27:0/3913446578 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22d8005180 0x7f22ec0aa0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.309+0000 7f22f08df640 1 -- 172.21.15.27:0/3913446578 >> 172.21.15.27:0/3913446578 conn(0x7f22ec0cf7d0 msgr2=0x7f22ec15c0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:04.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.309+0000 7f22f08df640 1 -- 172.21.15.27:0/3913446578 shutdown_connections 2023-12-19T18:20:04.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.309+0000 7f22f08df640 1 -- 172.21.15.27:0/3913446578 wait complete. 2023-12-19T18:20:04.330 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:20:04.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.555+0000 7fdb3feaf640 1 Processor -- start 2023-12-19T18:20:04.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.555+0000 7fdb3feaf640 1 -- start start 2023-12-19T18:20:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.557+0000 7fdb3feaf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdb380cbce0 0x7fdb380ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.557+0000 7fdb3feaf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb380cb850 con 0x7fdb380adc20 2023-12-19T18:20:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.557+0000 7fdb3feaf640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb3815b350 con 0x7fdb3815af70 2023-12-19T18:20:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.557+0000 7fdb3feaf640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb380cac40 con 0x7fdb380cbce0 2023-12-19T18:20:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.557+0000 7fdb3d44a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdb380cbce0 0x7fdb380ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.557+0000 7fdb3d44a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdb380cbce0 0x7fdb380ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51008/0 (socket says 172.21.15.27:51008) 2023-12-19T18:20:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.557+0000 7fdb3e44c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdb380adc20 0x7fdb3806b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36392/0 (socket says 172.21.15.27:36392) 2023-12-19T18:20:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.557+0000 7fdb3d44a640 1 -- 172.21.15.27:0/2504572768 learned_addr learned my addr 172.21.15.27:0/2504572768 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.558+0000 7fdb3d44a640 1 -- 172.21.15.27:0/2504572768 >> v1:172.21.15.192:6789/0 conn(0x7fdb3815af70 legacy=0x7fdb380ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.558+0000 7fdb3d44a640 1 -- 172.21.15.27:0/2504572768 >> v1:172.21.15.27:6789/0 conn(0x7fdb380adc20 legacy=0x7fdb3806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.558+0000 7fdb3cc49640 1 -- 172.21.15.27:0/2504572768 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716491294 0 0) 0x7fdb380cb850 con 0x7fdb380adc20 2023-12-19T18:20:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.558+0000 7fdb3d44a640 1 -- 172.21.15.27:0/2504572768 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb380caf20 con 0x7fdb380cbce0 2023-12-19T18:20:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.559+0000 7fdb3d44a640 1 --2- 172.21.15.27:0/2504572768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb380cbce0 0x7fdb380ca540 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009870 tx=0x7fdb2802b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7028779520e5cd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.559+0000 7fdb3cc49640 1 -- 172.21.15.27:0/2504572768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28030860 con 0x7fdb380cbce0 2023-12-19T18:20:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.559+0000 7fdb3cc49640 1 -- 172.21.15.27:0/2504572768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb280309c0 con 0x7fdb380cbce0 2023-12-19T18:20:04.559 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.559+0000 7fdb3cc49640 1 -- 172.21.15.27:0/2504572768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb28030d50 con 0x7fdb380cbce0 2023-12-19T18:20:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.560+0000 7fdb3feaf640 1 -- 172.21.15.27:0/2504572768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb380cbce0 msgr2=0x7fdb380ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.560+0000 7fdb3feaf640 1 --2- 172.21.15.27:0/2504572768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb380cbce0 0x7fdb380ca540 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fdb28009870 tx=0x7fdb2802b9a0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.560+0000 7fdb3feaf640 1 -- 172.21.15.27:0/2504572768 shutdown_connections 2023-12-19T18:20:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.560+0000 7fdb3feaf640 1 --2- 172.21.15.27:0/2504572768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb380cbce0 0x7fdb380ca540 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.560+0000 7fdb3feaf640 1 -- 172.21.15.27:0/2504572768 >> 172.21.15.27:0/2504572768 conn(0x7fdb3806a330 msgr2=0x7fdb380b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.560+0000 7fdb3feaf640 1 -- 172.21.15.27:0/2504572768 shutdown_connections 2023-12-19T18:20:04.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.560+0000 7fdb3feaf640 1 -- 172.21.15.27:0/2504572768 wait complete. 2023-12-19T18:20:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.562+0000 7fdb3feaf640 1 Processor -- start 2023-12-19T18:20:04.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.562+0000 7fdb3feaf640 1 -- start start 2023-12-19T18:20:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.563+0000 7fdb3feaf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb380adc20 0x7fdb380ce110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.563+0000 7fdb3feaf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb380cbce0 0x7fdb380ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:04.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.563+0000 7fdb3feaf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb3815af70 0x7fdb380ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.563+0000 7fdb3feaf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb380c4020 con 0x7fdb380adc20 2023-12-19T18:20:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.563+0000 7fdb3feaf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb380c3410 con 0x7fdb380cbce0 2023-12-19T18:20:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.563+0000 7fdb3feaf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb380c36f0 con 0x7fdb3815af70 2023-12-19T18:20:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.563+0000 7fdb3dc4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb380adc20 0x7fdb380ce110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.563+0000 7fdb3dc4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb380adc20 0x7fdb380ce110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51022/0 (socket says 172.21.15.27:51022) 2023-12-19T18:20:04.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.563+0000 7fdb3dc4b640 1 -- 172.21.15.27:0/126122233 learned_addr learned my addr 172.21.15.27:0/126122233 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:04.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.564+0000 7fdb3dc4b640 1 -- 172.21.15.27:0/126122233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb3815af70 msgr2=0x7fdb380ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:04.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.564+0000 7fdb3dc4b640 1 --2- 172.21.15.27:0/126122233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb3815af70 0x7fdb380ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.564+0000 7fdb3dc4b640 1 -- 172.21.15.27:0/126122233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb380cbce0 msgr2=0x7fdb380ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:04.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.564+0000 7fdb3d44a640 1 --2- 172.21.15.27:0/126122233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb380cbce0 0x7fdb380ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:04.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.564+0000 7fdb3dc4b640 1 --2- 172.21.15.27:0/126122233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb380cbce0 0x7fdb380ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.564+0000 7fdb3dc4b640 1 -- 172.21.15.27:0/126122233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb280094d0 con 0x7fdb380adc20 2023-12-19T18:20:04.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.564+0000 7fdb3dc4b640 1 --2- 172.21.15.27:0/126122233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb380adc20 0x7fdb380ce110 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400b3f0 tx=0x7fdb3400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:04.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.564+0000 7fdb2e7fc640 1 -- 172.21.15.27:0/126122233 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34011830 con 0x7fdb380adc20 2023-12-19T18:20:04.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.564+0000 7fdb2e7fc640 1 -- 172.21.15.27:0/126122233 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb34011990 con 0x7fdb380adc20 2023-12-19T18:20:04.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.565+0000 7fdb3feaf640 1 -- 172.21.15.27:0/126122233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb380c39d0 con 0x7fdb380adc20 2023-12-19T18:20:04.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.565+0000 7fdb2e7fc640 1 -- 172.21.15.27:0/126122233 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb34011d40 con 0x7fdb380adc20 2023-12-19T18:20:04.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.565+0000 7fdb3feaf640 1 -- 172.21.15.27:0/126122233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb380ce650 con 0x7fdb380adc20 2023-12-19T18:20:04.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.566+0000 7fdb3feaf640 1 -- 172.21.15.27:0/126122233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb38068e90 con 0x7fdb380adc20 2023-12-19T18:20:04.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.567+0000 7fdb2e7fc640 1 -- 172.21.15.27:0/126122233 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdb3401a400 con 0x7fdb380adc20 2023-12-19T18:20:04.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.568+0000 7fdb2e7fc640 1 --2- 172.21.15.27:0/126122233 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdb1c06a460 0x7fdb1c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:04.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.568+0000 7fdb2e7fc640 1 -- 172.21.15.27:0/126122233 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdb34088810 con 0x7fdb380adc20 2023-12-19T18:20:04.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.568+0000 7fdb3d44a640 1 --2- 172.21.15.27:0/126122233 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdb1c06a460 0x7fdb1c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:04.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.570+0000 7fdb3d44a640 1 --2- 172.21.15.27:0/126122233 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdb1c06a460 0x7fdb1c06c920 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdb28036000 tx=0x7fdb2803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:04.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.572+0000 7fdb2e7fc640 1 -- 172.21.15.27:0/126122233 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdb34053d60 con 0x7fdb380adc20 2023-12-19T18:20:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.768+0000 7fdb3feaf640 1 -- 172.21.15.27:0/126122233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb380c8f50 con 0x7fdb380adc20 2023-12-19T18:20:04.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.771+0000 7fdb2e7fc640 1 -- 172.21.15.27:0/126122233 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16040 (secure 0 0 0) 0x7fdb34057380 con 0x7fdb380adc20 2023-12-19T18:20:04.772 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:04.772 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":96,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:20:01.268935+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:19:58.591127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T18:19:59.313961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:20:04.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.774+0000 7fdb3feaf640 1 -- 172.21.15.27:0/126122233 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdb1c06a460 msgr2=0x7fdb1c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:04.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.774+0000 7fdb3feaf640 1 --2- 172.21.15.27:0/126122233 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdb1c06a460 0x7fdb1c06c920 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fdb28036000 tx=0x7fdb2803b040 comp rx=0 tx=0).stop 2023-12-19T18:20:04.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.774+0000 7fdb3feaf640 1 -- 172.21.15.27:0/126122233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb380adc20 msgr2=0x7fdb380ce110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:04.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.774+0000 7fdb3feaf640 1 --2- 172.21.15.27:0/126122233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb380adc20 0x7fdb380ce110 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fdb3400b3f0 tx=0x7fdb3400b8b0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.774+0000 7fdb3feaf640 1 -- 172.21.15.27:0/126122233 shutdown_connections 2023-12-19T18:20:04.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.774+0000 7fdb3feaf640 1 --2- 172.21.15.27:0/126122233 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdb1c06a460 0x7fdb1c06c920 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.774+0000 7fdb3feaf640 1 --2- 172.21.15.27:0/126122233 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdb3815af70 0x7fdb380ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.774+0000 7fdb3feaf640 1 --2- 172.21.15.27:0/126122233 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdb380cbce0 0x7fdb380ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.774+0000 7fdb3feaf640 1 --2- 172.21.15.27:0/126122233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdb380adc20 0x7fdb380ce110 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:04.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.774+0000 7fdb3feaf640 1 -- 172.21.15.27:0/126122233 >> 172.21.15.27:0/126122233 conn(0x7fdb3806a330 msgr2=0x7fdb380b4090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:04.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.774+0000 7fdb3feaf640 1 -- 172.21.15.27:0/126122233 shutdown_connections 2023-12-19T18:20:04.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:04.774+0000 7fdb3feaf640 1 -- 172.21.15.27:0/126122233 wait complete. 2023-12-19T18:20:04.796 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:20:04.796 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:20:05.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.018+0000 7fe3c159b640 1 Processor -- start 2023-12-19T18:20:05.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.019+0000 7fe3c159b640 1 -- start start 2023-12-19T18:20:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.021+0000 7fe3c159b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe3bc15ee30 0x7fe3bc0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.021+0000 7fe3c159b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe3bc0afeb0 0x7fe3bc0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.021+0000 7fe3c159b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3bc0cb350 con 0x7fe3bc1553a0 2023-12-19T18:20:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.021+0000 7fe3c159b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3bc0cbcf0 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.021+0000 7fe3c159b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3bc0cb630 con 0x7fe3bc15ee30 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.021+0000 7fe3c0d9a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe3bc0afeb0 0x7fe3bc0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.021+0000 7fe3c0d9a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe3bc0afeb0 0x7fe3bc0c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51034/0 (socket says 172.21.15.27:51034) 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.021+0000 7fe3c0d9a640 1 -- 172.21.15.27:0/1732150067 learned_addr learned my addr 172.21.15.27:0/1732150067 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.021+0000 7fe3b3fff640 1 --2- 172.21.15.27:0/1732150067 >> v2:172.21.15.192:3300/0 conn(0x7fe3bc15ee30 0x7fe3bc0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.021+0000 7fe3c0d9a640 1 -- 172.21.15.27:0/1732150067 >> v2:172.21.15.192:3300/0 conn(0x7fe3bc15ee30 msgr2=0x7fe3bc0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.021+0000 7fe3c0d9a640 1 --2- 172.21.15.27:0/1732150067 >> v2:172.21.15.192:3300/0 conn(0x7fe3bc15ee30 0x7fe3bc0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.021+0000 7fe3c0d9a640 1 -- 172.21.15.27:0/1732150067 >> v1:172.21.15.27:6789/0 conn(0x7fe3bc1553a0 legacy=0x7fe3bc15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.022+0000 7fe3c0d9a640 1 -- 172.21.15.27:0/1732150067 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc0cb910 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.023+0000 7fe3c0d9a640 1 --2- 172.21.15.27:0/1732150067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3bc0afeb0 0x7fe3bc0c39d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc06d4a0 tx=0x7fe3b402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d93b4c5d233b5ad8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.023+0000 7fe3bb7fe640 1 -- 172.21.15.27:0/1732150067 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b402ca80 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.023+0000 7fe3bb7fe640 1 -- 172.21.15.27:0/1732150067 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3b402cbe0 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.023+0000 7fe3bb7fe640 1 -- 172.21.15.27:0/1732150067 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3b40343f0 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.024+0000 7fe3c159b640 1 -- 172.21.15.27:0/1732150067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3bc0afeb0 msgr2=0x7fe3bc0c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.024+0000 7fe3c159b640 1 --2- 172.21.15.27:0/1732150067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3bc0afeb0 0x7fe3bc0c39d0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fe3bc06d4a0 tx=0x7fe3b402bee0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.024+0000 7fe3c159b640 1 -- 172.21.15.27:0/1732150067 shutdown_connections 2023-12-19T18:20:05.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.024+0000 7fe3c159b640 1 --2- 172.21.15.27:0/1732150067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3bc0afeb0 0x7fe3bc0c39d0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.024+0000 7fe3c159b640 1 --2- 172.21.15.27:0/1732150067 >> v2:172.21.15.192:3300/0 conn(0x7fe3bc15ee30 0x7fe3bc0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.024+0000 7fe3c159b640 1 -- 172.21.15.27:0/1732150067 >> 172.21.15.27:0/1732150067 conn(0x7fe3bc069c70 msgr2=0x7fe3bc06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:05.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.024+0000 7fe3c159b640 1 -- 172.21.15.27:0/1732150067 shutdown_connections 2023-12-19T18:20:05.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.024+0000 7fe3c159b640 1 -- 172.21.15.27:0/1732150067 wait complete. 2023-12-19T18:20:05.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.026+0000 7fe3c159b640 1 Processor -- start 2023-12-19T18:20:05.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.026+0000 7fe3c159b640 1 -- start start 2023-12-19T18:20:05.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3c159b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3bc0afeb0 0x7fe3bc0b4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:05.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3c159b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3bc1553a0 0x7fe3bc154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:05.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3c159b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3bc15ee30 0x7fe3bc15e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:05.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3c159b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0b3570 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3c159b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc0b20f0 con 0x7fe3bc15ee30 2023-12-19T18:20:05.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3c159b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3bc154a50 con 0x7fe3bc1553a0 2023-12-19T18:20:05.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3bc0afeb0 0x7fe3bc0b4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:05.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3bc0afeb0 0x7fe3bc0b4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51048/0 (socket says 172.21.15.27:51048) 2023-12-19T18:20:05.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3bbfff640 1 -- 172.21.15.27:0/881513073 learned_addr learned my addr 172.21.15.27:0/881513073 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:05.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3c0d9a640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3bc15ee30 0x7fe3bc15e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:05.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3b3fff640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3bc1553a0 0x7fe3bc154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:05.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3bbfff640 1 -- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3bc1553a0 msgr2=0x7fe3bc154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:05.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3bbfff640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3bc1553a0 0x7fe3bc154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3bbfff640 1 -- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3bc15ee30 msgr2=0x7fe3bc15e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:05.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3bbfff640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3bc15ee30 0x7fe3bc15e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.027+0000 7fe3bbfff640 1 -- 172.21.15.27:0/881513073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3b4009b30 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.028+0000 7fe3bbfff640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3bc0afeb0 0x7fe3bc0b4010 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c450 tx=0x7fe3ac00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:05.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.028+0000 7fe3b97fa640 1 -- 172.21.15.27:0/881513073 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac0052d0 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.028+0000 7fe3b97fa640 1 -- 172.21.15.27:0/881513073 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3ac005430 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.028+0000 7fe3b97fa640 1 -- 172.21.15.27:0/881513073 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3ac012870 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.028+0000 7fe3c159b640 1 -- 172.21.15.27:0/881513073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3bc0b1b30 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.028+0000 7fe3c159b640 1 -- 172.21.15.27:0/881513073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3bc0b3810 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.030+0000 7fe3c159b640 1 -- 172.21.15.27:0/881513073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3bc04e840 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.030+0000 7fe3b97fa640 1 -- 172.21.15.27:0/881513073 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe3ac0129d0 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.031+0000 7fe3b97fa640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe39c06a460 0x7fe39c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:05.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.031+0000 7fe3b97fa640 1 -- 172.21.15.27:0/881513073 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3ac0883e0 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.032+0000 7fe3b3fff640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe39c06a460 0x7fe39c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:05.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.033+0000 7fe3b3fff640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe39c06a460 0x7fe39c06c920 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4006fd0 tx=0x7fe3a4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:05.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.036+0000 7fe3b97fa640 1 -- 172.21.15.27:0/881513073 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe3ac0539d0 con 0x7fe3bc0afeb0 2023-12-19T18:20:05.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.228+0000 7fe3c159b640 1 -- 172.21.15.27:0/881513073 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3bc0c5730 con 0x7fe39c06a460 2023-12-19T18:20:05.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.254+0000 7fe3b97fa640 1 -- 172.21.15.27:0/881513073 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282751 (secure 0 0 0) 0x7fe3a400e4c0 con 0x7fe39c06a460 2023-12-19T18:20:05.260 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:05.263 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:20:05.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.261+0000 7fe3c159b640 1 -- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe39c06a460 msgr2=0x7fe39c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:05.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.261+0000 7fe3c159b640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe39c06a460 0x7fe39c06c920 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4006fd0 tx=0x7fe3a4008040 comp rx=0 tx=0).stop 2023-12-19T18:20:05.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.261+0000 7fe3c159b640 1 -- 172.21.15.27:0/881513073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3bc0afeb0 msgr2=0x7fe3bc0b4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:05.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.261+0000 7fe3c159b640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3bc0afeb0 0x7fe3bc0b4010 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7fe3ac00c450 tx=0x7fe3ac00c910 comp rx=0 tx=0).stop 2023-12-19T18:20:05.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.261+0000 7fe3c159b640 1 -- 172.21.15.27:0/881513073 shutdown_connections 2023-12-19T18:20:05.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.261+0000 7fe3c159b640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe39c06a460 0x7fe39c06c920 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.261+0000 7fe3c159b640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3bc15ee30 0x7fe3bc15e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.261+0000 7fe3c159b640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3bc1553a0 0x7fe3bc154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.261+0000 7fe3c159b640 1 --2- 172.21.15.27:0/881513073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3bc0afeb0 0x7fe3bc0b4010 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.261+0000 7fe3c159b640 1 -- 172.21.15.27:0/881513073 >> 172.21.15.27:0/881513073 conn(0x7fe3bc069c70 msgr2=0x7fe3bc159d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:05.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.261+0000 7fe3c159b640 1 -- 172.21.15.27:0/881513073 shutdown_connections 2023-12-19T18:20:05.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.261+0000 7fe3c159b640 1 -- 172.21.15.27:0/881513073 wait complete. 2023-12-19T18:20:05.284 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2023-12-19T18:20:03.815122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5217,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557282,"statfs":{"total":1600000000,"available":1594656836,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.060261"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306302+0000","last_change":"2023-12-19T18:20:00.304894+0000","last_active":"2023-12-19T18:20:03.306302+0000","last_peered":"2023-12-19T18:20:03.306302+0000","last_clean":"2023-12-19T18:20:03.306302+0000","last_became_active":"2023-12-19T18:20:00.301978+0000","last_became_peered":"2023-12-19T18:20:00.301978+0000","last_unstale":"2023-12-19T18:20:03.306302+0000","last_undegraded":"2023-12-19T18:20:03.306302+0000","last_fullsized":"2023-12-19T18:20:03.306302+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:08:41.917246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.778749+0000","last_change":"2023-12-19T18:20:00.288505+0000","last_active":"2023-12-19T18:20:01.778749+0000","last_peered":"2023-12-19T18:20:01.778749+0000","last_clean":"2023-12-19T18:20:01.778749+0000","last_became_active":"2023-12-19T18:20:00.287919+0000","last_became_peered":"2023-12-19T18:20:00.287919+0000","last_unstale":"2023-12-19T18:20:01.778749+0000","last_undegraded":"2023-12-19T18:20:01.778749+0000","last_fullsized":"2023-12-19T18:20:01.778749+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:31.358973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.284989+0000","last_change":"2023-12-19T18:20:00.303723+0000","last_active":"2023-12-19T18:20:01.284989+0000","last_peered":"2023-12-19T18:20:01.284989+0000","last_clean":"2023-12-19T18:20:01.284989+0000","last_became_active":"2023-12-19T18:20:00.303185+0000","last_became_peered":"2023-12-19T18:20:00.303185+0000","last_unstale":"2023-12-19T18:20:01.284989+0000","last_undegraded":"2023-12-19T18:20:01.284989+0000","last_fullsized":"2023-12-19T18:20:01.284989+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:08:04.208807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.284940+0000","last_change":"2023-12-19T18:20:00.302506+0000","last_active":"2023-12-19T18:20:01.284940+0000","last_peered":"2023-12-19T18:20:01.284940+0000","last_clean":"2023-12-19T18:20:01.284940+0000","last_became_active":"2023-12-19T18:20:00.302198+0000","last_became_peered":"2023-12-19T18:20:00.302198+0000","last_unstale":"2023-12-19T18:20:01.284940+0000","last_undegraded":"2023-12-19T18:20:01.284940+0000","last_fullsized":"2023-12-19T18:20:01.284940+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:37:06.953401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275748+0000","last_change":"2023-12-19T18:20:00.287459+0000","last_active":"2023-12-19T18:20:01.275748+0000","last_peered":"2023-12-19T18:20:01.275748+0000","last_clean":"2023-12-19T18:20:01.275748+0000","last_became_active":"2023-12-19T18:20:00.284333+0000","last_became_peered":"2023-12-19T18:20:00.284333+0000","last_unstale":"2023-12-19T18:20:01.275748+0000","last_undegraded":"2023-12-19T18:20:01.275748+0000","last_fullsized":"2023-12-19T18:20:01.275748+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:21:08.215981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275684+0000","last_change":"2023-12-19T18:20:00.301624+0000","last_active":"2023-12-19T18:20:01.275684+0000","last_peered":"2023-12-19T18:20:01.275684+0000","last_clean":"2023-12-19T18:20:01.275684+0000","last_became_active":"2023-12-19T18:20:00.301331+0000","last_became_peered":"2023-12-19T18:20:00.301331+0000","last_unstale":"2023-12-19T18:20:01.275684+0000","last_undegraded":"2023-12-19T18:20:01.275684+0000","last_fullsized":"2023-12-19T18:20:01.275684+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:46:53.516319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.304638+0000","last_change":"2023-12-19T18:20:00.290116+0000","last_active":"2023-12-19T18:20:03.304638+0000","last_peered":"2023-12-19T18:20:03.304638+0000","last_clean":"2023-12-19T18:20:03.304638+0000","last_became_active":"2023-12-19T18:20:00.289720+0000","last_became_peered":"2023-12-19T18:20:00.289720+0000","last_unstale":"2023-12-19T18:20:03.304638+0000","last_undegraded":"2023-12-19T18:20:03.304638+0000","last_fullsized":"2023-12-19T18:20:03.304638+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:46:13.690282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306402+0000","last_change":"2023-12-19T18:20:00.299960+0000","last_active":"2023-12-19T18:20:03.306402+0000","last_peered":"2023-12-19T18:20:03.306402+0000","last_clean":"2023-12-19T18:20:03.306402+0000","last_became_active":"2023-12-19T18:20:00.296942+0000","last_became_peered":"2023-12-19T18:20:00.296942+0000","last_unstale":"2023-12-19T18:20:03.306402+0000","last_undegraded":"2023-12-19T18:20:03.306402+0000","last_fullsized":"2023-12-19T18:20:03.306402+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:25:37.993826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285133+0000","last_change":"2023-12-19T18:20:00.301936+0000","last_active":"2023-12-19T18:20:01.285133+0000","last_peered":"2023-12-19T18:20:01.285133+0000","last_clean":"2023-12-19T18:20:01.285133+0000","last_became_active":"2023-12-19T18:20:00.301656+0000","last_became_peered":"2023-12-19T18:20:00.301656+0000","last_unstale":"2023-12-19T18:20:01.285133+0000","last_undegraded":"2023-12-19T18:20:01.285133+0000","last_fullsized":"2023-12-19T18:20:01.285133+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:51.525570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305215+0000","last_change":"2023-12-19T18:20:00.291042+0000","last_active":"2023-12-19T18:20:03.305215+0000","last_peered":"2023-12-19T18:20:03.305215+0000","last_clean":"2023-12-19T18:20:03.305215+0000","last_became_active":"2023-12-19T18:20:00.290468+0000","last_became_peered":"2023-12-19T18:20:00.290468+0000","last_unstale":"2023-12-19T18:20:03.305215+0000","last_undegraded":"2023-12-19T18:20:03.305215+0000","last_fullsized":"2023-12-19T18:20:03.305215+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:22.842549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.307068+0000","last_change":"2023-12-19T18:20:00.302200+0000","last_active":"2023-12-19T18:20:03.307068+0000","last_peered":"2023-12-19T18:20:03.307068+0000","last_clean":"2023-12-19T18:20:03.307068+0000","last_became_active":"2023-12-19T18:20:00.297369+0000","last_became_peered":"2023-12-19T18:20:00.297369+0000","last_unstale":"2023-12-19T18:20:03.307068+0000","last_undegraded":"2023-12-19T18:20:03.307068+0000","last_fullsized":"2023-12-19T18:20:03.307068+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:22:50.155455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.277713+0000","last_change":"2023-12-19T18:20:00.299152+0000","last_active":"2023-12-19T18:20:01.277713+0000","last_peered":"2023-12-19T18:20:01.277713+0000","last_clean":"2023-12-19T18:20:01.277713+0000","last_became_active":"2023-12-19T18:20:00.298501+0000","last_became_peered":"2023-12-19T18:20:00.298501+0000","last_unstale":"2023-12-19T18:20:01.277713+0000","last_undegraded":"2023-12-19T18:20:01.277713+0000","last_fullsized":"2023-12-19T18:20:01.277713+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:46:53.516319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285269+0000","last_change":"2023-12-19T18:20:00.304617+0000","last_active":"2023-12-19T18:20:01.285269+0000","last_peered":"2023-12-19T18:20:01.285269+0000","last_clean":"2023-12-19T18:20:01.285269+0000","last_became_active":"2023-12-19T18:20:00.303490+0000","last_became_peered":"2023-12-19T18:20:00.303490+0000","last_unstale":"2023-12-19T18:20:01.285269+0000","last_undegraded":"2023-12-19T18:20:01.285269+0000","last_fullsized":"2023-12-19T18:20:01.285269+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:06.820663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.276901+0000","last_change":"2023-12-19T18:20:00.286672+0000","last_active":"2023-12-19T18:20:01.276901+0000","last_peered":"2023-12-19T18:20:01.276901+0000","last_clean":"2023-12-19T18:20:01.276901+0000","last_became_active":"2023-12-19T18:20:00.284223+0000","last_became_peered":"2023-12-19T18:20:00.284223+0000","last_unstale":"2023-12-19T18:20:01.276901+0000","last_undegraded":"2023-12-19T18:20:01.276901+0000","last_fullsized":"2023-12-19T18:20:01.276901+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:07.454418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.278087+0000","last_change":"2023-12-19T18:20:00.290245+0000","last_active":"2023-12-19T18:20:01.278087+0000","last_peered":"2023-12-19T18:20:01.278087+0000","last_clean":"2023-12-19T18:20:01.278087+0000","last_became_active":"2023-12-19T18:20:00.289155+0000","last_became_peered":"2023-12-19T18:20:00.289155+0000","last_unstale":"2023-12-19T18:20:01.278087+0000","last_undegraded":"2023-12-19T18:20:01.278087+0000","last_fullsized":"2023-12-19T18:20:01.278087+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:22.340036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.307905+0000","last_change":"2023-12-19T18:20:00.302332+0000","last_active":"2023-12-19T18:20:03.307905+0000","last_peered":"2023-12-19T18:20:03.307905+0000","last_clean":"2023-12-19T18:20:03.307905+0000","last_became_active":"2023-12-19T18:20:00.297724+0000","last_became_peered":"2023-12-19T18:20:00.297724+0000","last_unstale":"2023-12-19T18:20:03.307905+0000","last_undegraded":"2023-12-19T18:20:03.307905+0000","last_fullsized":"2023-12-19T18:20:03.307905+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:14:22.920400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.274768+0000","last_change":"2023-12-19T18:20:00.301404+0000","last_active":"2023-12-19T18:20:01.274768+0000","last_peered":"2023-12-19T18:20:01.274768+0000","last_clean":"2023-12-19T18:20:01.274768+0000","last_became_active":"2023-12-19T18:20:00.300818+0000","last_became_peered":"2023-12-19T18:20:00.300818+0000","last_unstale":"2023-12-19T18:20:01.274768+0000","last_undegraded":"2023-12-19T18:20:01.274768+0000","last_fullsized":"2023-12-19T18:20:01.274768+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:01:38.275644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.274761+0000","last_change":"2023-12-19T18:20:00.287659+0000","last_active":"2023-12-19T18:20:01.274761+0000","last_peered":"2023-12-19T18:20:01.274761+0000","last_clean":"2023-12-19T18:20:01.274761+0000","last_became_active":"2023-12-19T18:20:00.286002+0000","last_became_peered":"2023-12-19T18:20:00.286002+0000","last_unstale":"2023-12-19T18:20:01.274761+0000","last_undegraded":"2023-12-19T18:20:01.274761+0000","last_fullsized":"2023-12-19T18:20:01.274761+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:15.918235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306298+0000","last_change":"2023-12-19T18:20:00.304463+0000","last_active":"2023-12-19T18:20:03.306298+0000","last_peered":"2023-12-19T18:20:03.306298+0000","last_clean":"2023-12-19T18:20:03.306298+0000","last_became_active":"2023-12-19T18:20:00.304240+0000","last_became_peered":"2023-12-19T18:20:00.304240+0000","last_unstale":"2023-12-19T18:20:03.306298+0000","last_undegraded":"2023-12-19T18:20:03.306298+0000","last_fullsized":"2023-12-19T18:20:03.306298+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:20:28.327402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275993+0000","last_change":"2023-12-19T18:20:00.287872+0000","last_active":"2023-12-19T18:20:01.275993+0000","last_peered":"2023-12-19T18:20:01.275993+0000","last_clean":"2023-12-19T18:20:01.275993+0000","last_became_active":"2023-12-19T18:20:00.284927+0000","last_became_peered":"2023-12-19T18:20:00.284927+0000","last_unstale":"2023-12-19T18:20:01.275993+0000","last_undegraded":"2023-12-19T18:20:01.275993+0000","last_fullsized":"2023-12-19T18:20:01.275993+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:38:59.579237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.778977+0000","last_change":"2023-12-19T18:20:00.297701+0000","last_active":"2023-12-19T18:20:01.778977+0000","last_peered":"2023-12-19T18:20:01.778977+0000","last_clean":"2023-12-19T18:20:01.778977+0000","last_became_active":"2023-12-19T18:20:00.297484+0000","last_became_peered":"2023-12-19T18:20:00.297484+0000","last_unstale":"2023-12-19T18:20:01.778977+0000","last_undegraded":"2023-12-19T18:20:01.778977+0000","last_fullsized":"2023-12-19T18:20:01.778977+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:46:53.516319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275807+0000","last_change":"2023-12-19T18:20:00.289902+0000","last_active":"2023-12-19T18:20:01.275807+0000","last_peered":"2023-12-19T18:20:01.275807+0000","last_clean":"2023-12-19T18:20:01.275807+0000","last_became_active":"2023-12-19T18:20:00.288872+0000","last_became_peered":"2023-12-19T18:20:00.288872+0000","last_unstale":"2023-12-19T18:20:01.275807+0000","last_undegraded":"2023-12-19T18:20:01.275807+0000","last_fullsized":"2023-12-19T18:20:01.275807+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:01:06.119977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305323+0000","last_change":"2023-12-19T18:20:00.286335+0000","last_active":"2023-12-19T18:20:03.305323+0000","last_peered":"2023-12-19T18:20:03.305323+0000","last_clean":"2023-12-19T18:20:03.305323+0000","last_became_active":"2023-12-19T18:20:00.284662+0000","last_became_peered":"2023-12-19T18:20:00.284662+0000","last_unstale":"2023-12-19T18:20:03.305323+0000","last_undegraded":"2023-12-19T18:20:03.305323+0000","last_fullsized":"2023-12-19T18:20:03.305323+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:52:00.930870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306584+0000","last_change":"2023-12-19T18:20:00.291608+0000","last_active":"2023-12-19T18:20:03.306584+0000","last_peered":"2023-12-19T18:20:03.306584+0000","last_clean":"2023-12-19T18:20:03.306584+0000","last_became_active":"2023-12-19T18:20:00.290726+0000","last_became_peered":"2023-12-19T18:20:00.290726+0000","last_unstale":"2023-12-19T18:20:03.306584+0000","last_undegraded":"2023-12-19T18:20:03.306584+0000","last_fullsized":"2023-12-19T18:20:03.306584+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:44.483918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.304997+0000","last_change":"2023-12-19T18:20:00.291181+0000","last_active":"2023-12-19T18:20:03.304997+0000","last_peered":"2023-12-19T18:20:03.304997+0000","last_clean":"2023-12-19T18:20:03.304997+0000","last_became_active":"2023-12-19T18:20:00.290802+0000","last_became_peered":"2023-12-19T18:20:00.290802+0000","last_unstale":"2023-12-19T18:20:03.304997+0000","last_undegraded":"2023-12-19T18:20:03.304997+0000","last_fullsized":"2023-12-19T18:20:03.304997+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:21:12.632362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.307472+0000","last_change":"2023-12-19T18:20:00.303648+0000","last_active":"2023-12-19T18:20:03.307472+0000","last_peered":"2023-12-19T18:20:03.307472+0000","last_clean":"2023-12-19T18:20:03.307472+0000","last_became_active":"2023-12-19T18:20:00.301502+0000","last_became_peered":"2023-12-19T18:20:00.301502+0000","last_unstale":"2023-12-19T18:20:03.307472+0000","last_undegraded":"2023-12-19T18:20:03.307472+0000","last_fullsized":"2023-12-19T18:20:03.307472+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:20:14.411996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.276804+0000","last_change":"2023-12-19T18:20:00.301064+0000","last_active":"2023-12-19T18:20:01.276804+0000","last_peered":"2023-12-19T18:20:01.276804+0000","last_clean":"2023-12-19T18:20:01.276804+0000","last_became_active":"2023-12-19T18:20:00.298390+0000","last_became_peered":"2023-12-19T18:20:00.298390+0000","last_unstale":"2023-12-19T18:20:01.276804+0000","last_undegraded":"2023-12-19T18:20:01.276804+0000","last_fullsized":"2023-12-19T18:20:01.276804+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:37:43.579760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.778661+0000","last_change":"2023-12-19T18:20:00.297059+0000","last_active":"2023-12-19T18:20:01.778661+0000","last_peered":"2023-12-19T18:20:01.778661+0000","last_clean":"2023-12-19T18:20:01.778661+0000","last_became_active":"2023-12-19T18:20:00.296828+0000","last_became_peered":"2023-12-19T18:20:00.296828+0000","last_unstale":"2023-12-19T18:20:01.778661+0000","last_undegraded":"2023-12-19T18:20:01.778661+0000","last_fullsized":"2023-12-19T18:20:01.778661+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:01:06.119977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285385+0000","last_change":"2023-12-19T18:20:00.299968+0000","last_active":"2023-12-19T18:20:01.285385+0000","last_peered":"2023-12-19T18:20:01.285385+0000","last_clean":"2023-12-19T18:20:01.285385+0000","last_became_active":"2023-12-19T18:20:00.299573+0000","last_became_peered":"2023-12-19T18:20:00.299573+0000","last_unstale":"2023-12-19T18:20:01.285385+0000","last_undegraded":"2023-12-19T18:20:01.285385+0000","last_fullsized":"2023-12-19T18:20:01.285385+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:43:02.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306721+0000","last_change":"2023-12-19T18:20:00.298663+0000","last_active":"2023-12-19T18:20:03.306721+0000","last_peered":"2023-12-19T18:20:03.306721+0000","last_clean":"2023-12-19T18:20:03.306721+0000","last_became_active":"2023-12-19T18:20:00.298324+0000","last_became_peered":"2023-12-19T18:20:00.298324+0000","last_unstale":"2023-12-19T18:20:03.306721+0000","last_undegraded":"2023-12-19T18:20:03.306721+0000","last_fullsized":"2023-12-19T18:20:03.306721+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:31:11.720275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779248+0000","last_change":"2023-12-19T18:20:00.296832+0000","last_active":"2023-12-19T18:20:01.779248+0000","last_peered":"2023-12-19T18:20:01.779248+0000","last_clean":"2023-12-19T18:20:01.779248+0000","last_became_active":"2023-12-19T18:20:00.296605+0000","last_became_peered":"2023-12-19T18:20:00.296605+0000","last_unstale":"2023-12-19T18:20:01.779248+0000","last_undegraded":"2023-12-19T18:20:01.779248+0000","last_fullsized":"2023-12-19T18:20:01.779248+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:34:18.478814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779047+0000","last_change":"2023-12-19T18:20:00.299455+0000","last_active":"2023-12-19T18:20:01.779047+0000","last_peered":"2023-12-19T18:20:01.779047+0000","last_clean":"2023-12-19T18:20:01.779047+0000","last_became_active":"2023-12-19T18:20:00.299198+0000","last_became_peered":"2023-12-19T18:20:00.299198+0000","last_unstale":"2023-12-19T18:20:01.779047+0000","last_undegraded":"2023-12-19T18:20:01.779047+0000","last_fullsized":"2023-12-19T18:20:01.779047+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:19:28.119124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306777+0000","last_change":"2023-12-19T18:20:00.298188+0000","last_active":"2023-12-19T18:20:03.306777+0000","last_peered":"2023-12-19T18:20:03.306777+0000","last_clean":"2023-12-19T18:20:03.306777+0000","last_became_active":"2023-12-19T18:20:00.296149+0000","last_became_peered":"2023-12-19T18:20:00.296149+0000","last_unstale":"2023-12-19T18:20:03.306777+0000","last_undegraded":"2023-12-19T18:20:03.306777+0000","last_fullsized":"2023-12-19T18:20:03.306777+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:54:01.938427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"95'4","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.301034+0000","last_change":"2023-12-19T18:19:59.770522+0000","last_active":"2023-12-19T18:20:00.301034+0000","last_peered":"2023-12-19T18:20:00.301034+0000","last_clean":"2023-12-19T18:20:00.301034+0000","last_became_active":"2023-12-19T18:19:59.769869+0000","last_became_peered":"2023-12-19T18:19:59.769869+0000","last_unstale":"2023-12-19T18:20:00.301034+0000","last_undegraded":"2023-12-19T18:20:00.301034+0000","last_fullsized":"2023-12-19T18:20:00.301034+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:05.807764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285542+0000","last_change":"2023-12-19T18:20:00.305850+0000","last_active":"2023-12-19T18:20:01.285542+0000","last_peered":"2023-12-19T18:20:01.285542+0000","last_clean":"2023-12-19T18:20:01.285542+0000","last_became_active":"2023-12-19T18:20:00.305406+0000","last_became_peered":"2023-12-19T18:20:00.305406+0000","last_unstale":"2023-12-19T18:20:01.285542+0000","last_undegraded":"2023-12-19T18:20:01.285542+0000","last_fullsized":"2023-12-19T18:20:01.285542+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:21.279347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.274186+0000","last_change":"2023-12-19T18:19:59.784415+0000","last_active":"2023-12-19T18:20:01.274186+0000","last_peered":"2023-12-19T18:20:01.274186+0000","last_clean":"2023-12-19T18:20:01.274186+0000","last_became_active":"2023-12-19T18:19:59.783902+0000","last_became_peered":"2023-12-19T18:19:59.783902+0000","last_unstale":"2023-12-19T18:20:01.274186+0000","last_undegraded":"2023-12-19T18:20:01.274186+0000","last_fullsized":"2023-12-19T18:20:01.274186+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:07.322258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275832+0000","last_change":"2023-12-19T18:20:00.295082+0000","last_active":"2023-12-19T18:20:01.275832+0000","last_peered":"2023-12-19T18:20:01.275832+0000","last_clean":"2023-12-19T18:20:01.275832+0000","last_became_active":"2023-12-19T18:20:00.294842+0000","last_became_peered":"2023-12-19T18:20:00.294842+0000","last_unstale":"2023-12-19T18:20:01.275832+0000","last_undegraded":"2023-12-19T18:20:01.275832+0000","last_fullsized":"2023-12-19T18:20:01.275832+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:34:18.478814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"95'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.298772+0000","last_change":"2023-12-19T18:19:59.791361+0000","last_active":"2023-12-19T18:20:00.298772+0000","last_peered":"2023-12-19T18:20:00.298772+0000","last_clean":"2023-12-19T18:20:00.298772+0000","last_became_active":"2023-12-19T18:19:59.791131+0000","last_became_peered":"2023-12-19T18:19:59.791131+0000","last_unstale":"2023-12-19T18:20:00.298772+0000","last_undegraded":"2023-12-19T18:20:00.298772+0000","last_fullsized":"2023-12-19T18:20:00.298772+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:17.492321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305459+0000","last_change":"2023-12-19T18:20:00.285747+0000","last_active":"2023-12-19T18:20:03.305459+0000","last_peered":"2023-12-19T18:20:03.305459+0000","last_clean":"2023-12-19T18:20:03.305459+0000","last_became_active":"2023-12-19T18:20:00.285530+0000","last_became_peered":"2023-12-19T18:20:00.285530+0000","last_unstale":"2023-12-19T18:20:03.305459+0000","last_undegraded":"2023-12-19T18:20:03.305459+0000","last_fullsized":"2023-12-19T18:20:03.305459+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:57:12.333501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"95'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.296687+0000","last_change":"2023-12-19T18:19:59.785574+0000","last_active":"2023-12-19T18:20:00.296687+0000","last_peered":"2023-12-19T18:20:00.296687+0000","last_clean":"2023-12-19T18:20:00.296687+0000","last_became_active":"2023-12-19T18:19:59.784958+0000","last_became_peered":"2023-12-19T18:19:59.784958+0000","last_unstale":"2023-12-19T18:20:00.296687+0000","last_undegraded":"2023-12-19T18:20:00.296687+0000","last_fullsized":"2023-12-19T18:20:00.296687+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:43:01.628238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306126+0000","last_change":"2023-12-19T18:20:00.300634+0000","last_active":"2023-12-19T18:20:03.306126+0000","last_peered":"2023-12-19T18:20:03.306126+0000","last_clean":"2023-12-19T18:20:03.306126+0000","last_became_active":"2023-12-19T18:20:00.299718+0000","last_became_peered":"2023-12-19T18:20:00.299718+0000","last_unstale":"2023-12-19T18:20:03.306126+0000","last_undegraded":"2023-12-19T18:20:03.306126+0000","last_fullsized":"2023-12-19T18:20:03.306126+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:01:01.654519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"95'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.298167+0000","last_change":"2023-12-19T18:19:59.775462+0000","last_active":"2023-12-19T18:20:00.298167+0000","last_peered":"2023-12-19T18:20:00.298167+0000","last_clean":"2023-12-19T18:20:00.298167+0000","last_became_active":"2023-12-19T18:19:59.774457+0000","last_became_peered":"2023-12-19T18:19:59.774457+0000","last_unstale":"2023-12-19T18:20:00.298167+0000","last_undegraded":"2023-12-19T18:20:00.298167+0000","last_fullsized":"2023-12-19T18:20:00.298167+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:43:01.628238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275907+0000","last_change":"2023-12-19T18:20:00.290984+0000","last_active":"2023-12-19T18:20:01.275907+0000","last_peered":"2023-12-19T18:20:01.275907+0000","last_clean":"2023-12-19T18:20:01.275907+0000","last_became_active":"2023-12-19T18:20:00.290765+0000","last_became_peered":"2023-12-19T18:20:00.290765+0000","last_unstale":"2023-12-19T18:20:01.275907+0000","last_undegraded":"2023-12-19T18:20:01.275907+0000","last_fullsized":"2023-12-19T18:20:01.275907+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:35.301273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305595+0000","last_change":"2023-12-19T18:19:59.792674+0000","last_active":"2023-12-19T18:20:03.305595+0000","last_peered":"2023-12-19T18:20:03.305595+0000","last_clean":"2023-12-19T18:20:03.305595+0000","last_became_active":"2023-12-19T18:19:59.791269+0000","last_became_peered":"2023-12-19T18:19:59.791269+0000","last_unstale":"2023-12-19T18:20:03.305595+0000","last_undegraded":"2023-12-19T18:20:03.305595+0000","last_fullsized":"2023-12-19T18:20:03.305595+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:01:01.346973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779788+0000","last_change":"2023-12-19T18:20:00.302341+0000","last_active":"2023-12-19T18:20:01.779788+0000","last_peered":"2023-12-19T18:20:01.779788+0000","last_clean":"2023-12-19T18:20:01.779788+0000","last_became_active":"2023-12-19T18:20:00.302101+0000","last_became_peered":"2023-12-19T18:20:00.302101+0000","last_unstale":"2023-12-19T18:20:01.779788+0000","last_undegraded":"2023-12-19T18:20:01.779788+0000","last_fullsized":"2023-12-19T18:20:01.779788+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:08:04.208807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779974+0000","last_change":"2023-12-19T18:19:59.790921+0000","last_active":"2023-12-19T18:20:01.779974+0000","last_peered":"2023-12-19T18:20:01.779974+0000","last_clean":"2023-12-19T18:20:01.779974+0000","last_became_active":"2023-12-19T18:19:59.790726+0000","last_became_peered":"2023-12-19T18:19:59.790726+0000","last_unstale":"2023-12-19T18:20:01.779974+0000","last_undegraded":"2023-12-19T18:20:01.779974+0000","last_fullsized":"2023-12-19T18:20:01.779974+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:36.681763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.277143+0000","last_change":"2023-12-19T18:20:00.284839+0000","last_active":"2023-12-19T18:20:01.277143+0000","last_peered":"2023-12-19T18:20:01.277143+0000","last_clean":"2023-12-19T18:20:01.277143+0000","last_became_active":"2023-12-19T18:20:00.283239+0000","last_became_peered":"2023-12-19T18:20:00.283239+0000","last_unstale":"2023-12-19T18:20:01.277143+0000","last_undegraded":"2023-12-19T18:20:01.277143+0000","last_fullsized":"2023-12-19T18:20:01.277143+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:31.358973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.281383+0000","last_change":"2023-12-19T18:19:59.781903+0000","last_active":"2023-12-19T18:20:00.281383+0000","last_peered":"2023-12-19T18:20:00.281383+0000","last_clean":"2023-12-19T18:20:00.281383+0000","last_became_active":"2023-12-19T18:19:59.781313+0000","last_became_peered":"2023-12-19T18:19:59.781313+0000","last_unstale":"2023-12-19T18:20:00.281383+0000","last_undegraded":"2023-12-19T18:20:00.281383+0000","last_fullsized":"2023-12-19T18:20:00.281383+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:53:15.711817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.304930+0000","last_change":"2023-12-19T18:20:00.282702+0000","last_active":"2023-12-19T18:20:03.304930+0000","last_peered":"2023-12-19T18:20:03.304930+0000","last_clean":"2023-12-19T18:20:03.304930+0000","last_became_active":"2023-12-19T18:20:00.282401+0000","last_became_peered":"2023-12-19T18:20:00.282401+0000","last_unstale":"2023-12-19T18:20:03.304930+0000","last_undegraded":"2023-12-19T18:20:03.304930+0000","last_fullsized":"2023-12-19T18:20:03.304930+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:15.972242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.292373+0000","last_change":"2023-12-19T18:19:59.786867+0000","last_active":"2023-12-19T18:20:00.292373+0000","last_peered":"2023-12-19T18:20:00.292373+0000","last_clean":"2023-12-19T18:20:00.292373+0000","last_became_active":"2023-12-19T18:19:59.784118+0000","last_became_peered":"2023-12-19T18:19:59.784118+0000","last_unstale":"2023-12-19T18:20:00.292373+0000","last_undegraded":"2023-12-19T18:20:00.292373+0000","last_fullsized":"2023-12-19T18:20:00.292373+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:37:42.566861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779419+0000","last_change":"2023-12-19T18:20:00.289832+0000","last_active":"2023-12-19T18:20:01.779419+0000","last_peered":"2023-12-19T18:20:01.779419+0000","last_clean":"2023-12-19T18:20:01.779419+0000","last_became_active":"2023-12-19T18:20:00.287480+0000","last_became_peered":"2023-12-19T18:20:00.287480+0000","last_unstale":"2023-12-19T18:20:01.779419+0000","last_undegraded":"2023-12-19T18:20:01.779419+0000","last_fullsized":"2023-12-19T18:20:01.779419+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:43.029632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779299+0000","last_change":"2023-12-19T18:19:59.776511+0000","last_active":"2023-12-19T18:20:01.779299+0000","last_peered":"2023-12-19T18:20:01.779299+0000","last_clean":"2023-12-19T18:20:01.779299+0000","last_became_active":"2023-12-19T18:19:59.776115+0000","last_became_peered":"2023-12-19T18:19:59.776115+0000","last_unstale":"2023-12-19T18:20:01.779299+0000","last_undegraded":"2023-12-19T18:20:01.779299+0000","last_fullsized":"2023-12-19T18:20:01.779299+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:43:05.535962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306900+0000","last_change":"2023-12-19T18:20:00.291458+0000","last_active":"2023-12-19T18:20:03.306900+0000","last_peered":"2023-12-19T18:20:03.306900+0000","last_clean":"2023-12-19T18:20:03.306900+0000","last_became_active":"2023-12-19T18:20:00.289773+0000","last_became_peered":"2023-12-19T18:20:00.289773+0000","last_unstale":"2023-12-19T18:20:03.306900+0000","last_undegraded":"2023-12-19T18:20:03.306900+0000","last_fullsized":"2023-12-19T18:20:03.306900+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:24.817998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"95'3","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.303839+0000","last_change":"2023-12-19T18:19:59.786986+0000","last_active":"2023-12-19T18:20:00.303839+0000","last_peered":"2023-12-19T18:20:00.303839+0000","last_clean":"2023-12-19T18:20:00.303839+0000","last_became_active":"2023-12-19T18:19:59.786497+0000","last_became_peered":"2023-12-19T18:19:59.786497+0000","last_unstale":"2023-12-19T18:20:00.303839+0000","last_undegraded":"2023-12-19T18:20:00.303839+0000","last_fullsized":"2023-12-19T18:20:00.303839+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:50.512671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275004+0000","last_change":"2023-12-19T18:20:00.298187+0000","last_active":"2023-12-19T18:20:01.275004+0000","last_peered":"2023-12-19T18:20:01.275004+0000","last_clean":"2023-12-19T18:20:01.275004+0000","last_became_active":"2023-12-19T18:20:00.297199+0000","last_became_peered":"2023-12-19T18:20:00.297199+0000","last_unstale":"2023-12-19T18:20:01.275004+0000","last_undegraded":"2023-12-19T18:20:01.275004+0000","last_fullsized":"2023-12-19T18:20:01.275004+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:18.505220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"95'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.780198+0000","last_change":"2023-12-19T18:19:59.789502+0000","last_active":"2023-12-19T18:20:01.780198+0000","last_peered":"2023-12-19T18:20:01.780198+0000","last_clean":"2023-12-19T18:20:01.780198+0000","last_became_active":"2023-12-19T18:19:59.789090+0000","last_became_peered":"2023-12-19T18:19:59.789090+0000","last_unstale":"2023-12-19T18:20:01.780198+0000","last_undegraded":"2023-12-19T18:20:01.780198+0000","last_fullsized":"2023-12-19T18:20:01.780198+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:45.860622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275997+0000","last_change":"2023-12-19T18:20:00.280622+0000","last_active":"2023-12-19T18:20:01.275997+0000","last_peered":"2023-12-19T18:20:01.275997+0000","last_clean":"2023-12-19T18:20:01.275997+0000","last_became_active":"2023-12-19T18:20:00.280402+0000","last_became_peered":"2023-12-19T18:20:00.280402+0000","last_unstale":"2023-12-19T18:20:01.275997+0000","last_undegraded":"2023-12-19T18:20:01.275997+0000","last_fullsized":"2023-12-19T18:20:01.275997+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:05.897496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.780018+0000","last_change":"2023-12-19T18:19:59.783119+0000","last_active":"2023-12-19T18:20:01.780018+0000","last_peered":"2023-12-19T18:20:01.780018+0000","last_clean":"2023-12-19T18:20:01.780018+0000","last_became_active":"2023-12-19T18:19:59.782829+0000","last_became_peered":"2023-12-19T18:19:59.782829+0000","last_unstale":"2023-12-19T18:20:01.780018+0000","last_undegraded":"2023-12-19T18:20:01.780018+0000","last_fullsized":"2023-12-19T18:20:01.780018+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:21:44.158670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305711+0000","last_change":"2023-12-19T18:20:00.297847+0000","last_active":"2023-12-19T18:20:03.305711+0000","last_peered":"2023-12-19T18:20:03.305711+0000","last_clean":"2023-12-19T18:20:03.305711+0000","last_became_active":"2023-12-19T18:20:00.295980+0000","last_became_peered":"2023-12-19T18:20:00.295980+0000","last_unstale":"2023-12-19T18:20:03.305711+0000","last_undegraded":"2023-12-19T18:20:03.305711+0000","last_fullsized":"2023-12-19T18:20:03.305711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:44:18.425235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305106+0000","last_change":"2023-12-19T18:19:59.788329+0000","last_active":"2023-12-19T18:20:03.305106+0000","last_peered":"2023-12-19T18:20:03.305106+0000","last_clean":"2023-12-19T18:20:03.305106+0000","last_became_active":"2023-12-19T18:19:59.788017+0000","last_became_peered":"2023-12-19T18:19:59.788017+0000","last_unstale":"2023-12-19T18:20:03.305106+0000","last_undegraded":"2023-12-19T18:20:03.305106+0000","last_fullsized":"2023-12-19T18:20:03.305106+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:08:06.261382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275224+0000","last_change":"2023-12-19T18:20:00.287261+0000","last_active":"2023-12-19T18:20:01.275224+0000","last_peered":"2023-12-19T18:20:01.275224+0000","last_clean":"2023-12-19T18:20:01.275224+0000","last_became_active":"2023-12-19T18:20:00.285021+0000","last_became_peered":"2023-12-19T18:20:00.285021+0000","last_unstale":"2023-12-19T18:20:01.275224+0000","last_undegraded":"2023-12-19T18:20:01.275224+0000","last_fullsized":"2023-12-19T18:20:01.275224+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:23:26.019352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305612+0000","last_change":"2023-12-19T18:19:59.789130+0000","last_active":"2023-12-19T18:20:03.305612+0000","last_peered":"2023-12-19T18:20:03.305612+0000","last_clean":"2023-12-19T18:20:03.305612+0000","last_became_active":"2023-12-19T18:19:59.787049+0000","last_became_peered":"2023-12-19T18:19:59.787049+0000","last_unstale":"2023-12-19T18:20:03.305612+0000","last_undegraded":"2023-12-19T18:20:03.305612+0000","last_fullsized":"2023-12-19T18:20:03.305612+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:24:05.605488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285590+0000","last_change":"2023-12-19T18:20:00.305082+0000","last_active":"2023-12-19T18:20:01.285590+0000","last_peered":"2023-12-19T18:20:01.285590+0000","last_clean":"2023-12-19T18:20:01.285590+0000","last_became_active":"2023-12-19T18:20:00.304847+0000","last_became_peered":"2023-12-19T18:20:00.304847+0000","last_unstale":"2023-12-19T18:20:01.285590+0000","last_undegraded":"2023-12-19T18:20:01.285590+0000","last_fullsized":"2023-12-19T18:20:01.285590+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:08:04.212994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"95'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305837+0000","last_change":"2023-12-19T18:19:59.792885+0000","last_active":"2023-12-19T18:20:03.305837+0000","last_peered":"2023-12-19T18:20:03.305837+0000","last_clean":"2023-12-19T18:20:03.305837+0000","last_became_active":"2023-12-19T18:19:59.792428+0000","last_became_peered":"2023-12-19T18:19:59.792428+0000","last_unstale":"2023-12-19T18:20:03.305837+0000","last_undegraded":"2023-12-19T18:20:03.305837+0000","last_fullsized":"2023-12-19T18:20:03.305837+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:08:06.603565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779348+0000","last_change":"2023-12-19T18:20:00.292415+0000","last_active":"2023-12-19T18:20:01.779348+0000","last_peered":"2023-12-19T18:20:01.779348+0000","last_clean":"2023-12-19T18:20:01.779348+0000","last_became_active":"2023-12-19T18:20:00.291562+0000","last_became_peered":"2023-12-19T18:20:00.291562+0000","last_unstale":"2023-12-19T18:20:01.779348+0000","last_undegraded":"2023-12-19T18:20:01.779348+0000","last_fullsized":"2023-12-19T18:20:01.779348+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:05.897496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"95'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.302780+0000","last_change":"2023-12-19T18:19:59.787044+0000","last_active":"2023-12-19T18:20:00.302780+0000","last_peered":"2023-12-19T18:20:00.302780+0000","last_clean":"2023-12-19T18:20:00.302780+0000","last_became_active":"2023-12-19T18:19:59.786807+0000","last_became_peered":"2023-12-19T18:19:59.786807+0000","last_unstale":"2023-12-19T18:20:00.302780+0000","last_undegraded":"2023-12-19T18:20:00.302780+0000","last_fullsized":"2023-12-19T18:20:00.302780+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:41:21.837669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.778601+0000","last_change":"2023-12-19T18:20:00.285691+0000","last_active":"2023-12-19T18:20:01.778601+0000","last_peered":"2023-12-19T18:20:01.778601+0000","last_clean":"2023-12-19T18:20:01.778601+0000","last_became_active":"2023-12-19T18:20:00.285469+0000","last_became_peered":"2023-12-19T18:20:00.285469+0000","last_unstale":"2023-12-19T18:20:01.778601+0000","last_undegraded":"2023-12-19T18:20:01.778601+0000","last_fullsized":"2023-12-19T18:20:01.778601+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:35.301273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779632+0000","last_change":"2023-12-19T18:19:59.784766+0000","last_active":"2023-12-19T18:20:01.779632+0000","last_peered":"2023-12-19T18:20:01.779632+0000","last_clean":"2023-12-19T18:20:01.779632+0000","last_became_active":"2023-12-19T18:19:59.784413+0000","last_became_peered":"2023-12-19T18:19:59.784413+0000","last_unstale":"2023-12-19T18:20:01.779632+0000","last_undegraded":"2023-12-19T18:20:01.779632+0000","last_fullsized":"2023-12-19T18:20:01.779632+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:01:20.510530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285826+0000","last_change":"2023-12-19T18:20:00.302920+0000","last_active":"2023-12-19T18:20:01.285826+0000","last_peered":"2023-12-19T18:20:01.285826+0000","last_clean":"2023-12-19T18:20:01.285826+0000","last_became_active":"2023-12-19T18:20:00.301404+0000","last_became_peered":"2023-12-19T18:20:00.301404+0000","last_unstale":"2023-12-19T18:20:01.285826+0000","last_undegraded":"2023-12-19T18:20:01.285826+0000","last_fullsized":"2023-12-19T18:20:01.285826+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:47:13.415066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779703+0000","last_change":"2023-12-19T18:19:59.788293+0000","last_active":"2023-12-19T18:20:01.779703+0000","last_peered":"2023-12-19T18:20:01.779703+0000","last_clean":"2023-12-19T18:20:01.779703+0000","last_became_active":"2023-12-19T18:19:59.786626+0000","last_became_peered":"2023-12-19T18:19:59.786626+0000","last_unstale":"2023-12-19T18:20:01.779703+0000","last_undegraded":"2023-12-19T18:20:01.779703+0000","last_fullsized":"2023-12-19T18:20:01.779703+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:36.681763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285517+0000","last_change":"2023-12-19T18:20:00.303424+0000","last_active":"2023-12-19T18:20:01.285517+0000","last_peered":"2023-12-19T18:20:01.285517+0000","last_clean":"2023-12-19T18:20:01.285517+0000","last_became_active":"2023-12-19T18:20:00.303161+0000","last_became_peered":"2023-12-19T18:20:00.303161+0000","last_unstale":"2023-12-19T18:20:01.285517+0000","last_undegraded":"2023-12-19T18:20:01.285517+0000","last_fullsized":"2023-12-19T18:20:01.285517+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:53:16.724716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.274123+0000","last_change":"2023-12-19T18:19:59.787423+0000","last_active":"2023-12-19T18:20:00.274123+0000","last_peered":"2023-12-19T18:20:00.274123+0000","last_clean":"2023-12-19T18:20:00.274123+0000","last_became_active":"2023-12-19T18:19:59.787094+0000","last_became_peered":"2023-12-19T18:19:59.787094+0000","last_unstale":"2023-12-19T18:20:00.274123+0000","last_undegraded":"2023-12-19T18:20:00.274123+0000","last_fullsized":"2023-12-19T18:20:00.274123+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:37:05.940502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.780210+0000","last_change":"2023-12-19T18:20:00.303532+0000","last_active":"2023-12-19T18:20:01.780210+0000","last_peered":"2023-12-19T18:20:01.780210+0000","last_clean":"2023-12-19T18:20:01.780210+0000","last_became_active":"2023-12-19T18:20:00.302923+0000","last_became_peered":"2023-12-19T18:20:00.302923+0000","last_unstale":"2023-12-19T18:20:01.780210+0000","last_undegraded":"2023-12-19T18:20:01.780210+0000","last_fullsized":"2023-12-19T18:20:01.780210+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:06.820663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.273330+0000","last_change":"2023-12-19T18:19:59.774115+0000","last_active":"2023-12-19T18:20:00.273330+0000","last_peered":"2023-12-19T18:20:00.273330+0000","last_clean":"2023-12-19T18:20:00.273330+0000","last_became_active":"2023-12-19T18:19:59.773598+0000","last_became_peered":"2023-12-19T18:19:59.773598+0000","last_unstale":"2023-12-19T18:20:00.273330+0000","last_undegraded":"2023-12-19T18:20:00.273330+0000","last_fullsized":"2023-12-19T18:20:00.273330+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:20.266447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779656+0000","last_change":"2023-12-19T18:20:00.296833+0000","last_active":"2023-12-19T18:20:01.779656+0000","last_peered":"2023-12-19T18:20:01.779656+0000","last_clean":"2023-12-19T18:20:01.779656+0000","last_became_active":"2023-12-19T18:20:00.295532+0000","last_became_peered":"2023-12-19T18:20:00.295532+0000","last_unstale":"2023-12-19T18:20:01.779656+0000","last_undegraded":"2023-12-19T18:20:01.779656+0000","last_fullsized":"2023-12-19T18:20:01.779656+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:37:06.953401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779745+0000","last_change":"2023-12-19T18:19:59.772374+0000","last_active":"2023-12-19T18:20:01.779745+0000","last_peered":"2023-12-19T18:20:01.779745+0000","last_clean":"2023-12-19T18:20:01.779745+0000","last_became_active":"2023-12-19T18:19:59.771722+0000","last_became_peered":"2023-12-19T18:19:59.771722+0000","last_unstale":"2023-12-19T18:20:01.779745+0000","last_undegraded":"2023-12-19T18:20:01.779745+0000","last_fullsized":"2023-12-19T18:20:01.779745+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:53.382720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.278208+0000","last_change":"2023-12-19T18:20:00.300239+0000","last_active":"2023-12-19T18:20:01.278208+0000","last_peered":"2023-12-19T18:20:01.278208+0000","last_clean":"2023-12-19T18:20:01.278208+0000","last_became_active":"2023-12-19T18:20:00.299701+0000","last_became_peered":"2023-12-19T18:20:00.299701+0000","last_unstale":"2023-12-19T18:20:01.278208+0000","last_undegraded":"2023-12-19T18:20:01.278208+0000","last_fullsized":"2023-12-19T18:20:01.278208+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:41.013088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275385+0000","last_change":"2023-12-19T18:19:59.782636+0000","last_active":"2023-12-19T18:20:01.275385+0000","last_peered":"2023-12-19T18:20:01.275385+0000","last_clean":"2023-12-19T18:20:01.275385+0000","last_became_active":"2023-12-19T18:19:59.782260+0000","last_became_peered":"2023-12-19T18:19:59.782260+0000","last_unstale":"2023-12-19T18:20:01.275385+0000","last_undegraded":"2023-12-19T18:20:01.275385+0000","last_fullsized":"2023-12-19T18:20:01.275385+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:43:48.093779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.276419+0000","last_change":"2023-12-19T18:20:00.298382+0000","last_active":"2023-12-19T18:20:01.276419+0000","last_peered":"2023-12-19T18:20:01.276419+0000","last_clean":"2023-12-19T18:20:01.276419+0000","last_became_active":"2023-12-19T18:20:00.297934+0000","last_became_peered":"2023-12-19T18:20:00.297934+0000","last_unstale":"2023-12-19T18:20:01.276419+0000","last_undegraded":"2023-12-19T18:20:01.276419+0000","last_fullsized":"2023-12-19T18:20:01.276419+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:24:02.318855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"95'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.303304+0000","last_change":"2023-12-19T18:19:59.784490+0000","last_active":"2023-12-19T18:20:00.303304+0000","last_peered":"2023-12-19T18:20:00.303304+0000","last_clean":"2023-12-19T18:20:00.303304+0000","last_became_active":"2023-12-19T18:19:59.784048+0000","last_became_peered":"2023-12-19T18:19:59.784048+0000","last_unstale":"2023-12-19T18:20:00.303304+0000","last_undegraded":"2023-12-19T18:20:00.303304+0000","last_fullsized":"2023-12-19T18:20:00.303304+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.905336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305241+0000","last_change":"2023-12-19T18:20:00.298044+0000","last_active":"2023-12-19T18:20:03.305241+0000","last_peered":"2023-12-19T18:20:03.305241+0000","last_clean":"2023-12-19T18:20:03.305241+0000","last_became_active":"2023-12-19T18:20:00.297834+0000","last_became_peered":"2023-12-19T18:20:00.297834+0000","last_unstale":"2023-12-19T18:20:03.305241+0000","last_undegraded":"2023-12-19T18:20:03.305241+0000","last_fullsized":"2023-12-19T18:20:03.305241+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:23.800025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.287591+0000","last_change":"2023-12-19T18:19:59.789915+0000","last_active":"2023-12-19T18:20:00.287591+0000","last_peered":"2023-12-19T18:20:00.287591+0000","last_clean":"2023-12-19T18:20:00.287591+0000","last_became_active":"2023-12-19T18:19:59.789682+0000","last_became_peered":"2023-12-19T18:19:59.789682+0000","last_unstale":"2023-12-19T18:20:00.287591+0000","last_undegraded":"2023-12-19T18:20:00.287591+0000","last_fullsized":"2023-12-19T18:20:00.287591+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:01:37.262744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779503+0000","last_change":"2023-12-19T18:20:00.287734+0000","last_active":"2023-12-19T18:20:01.779503+0000","last_peered":"2023-12-19T18:20:01.779503+0000","last_clean":"2023-12-19T18:20:01.779503+0000","last_became_active":"2023-12-19T18:20:00.287429+0000","last_became_peered":"2023-12-19T18:20:00.287429+0000","last_unstale":"2023-12-19T18:20:01.779503+0000","last_undegraded":"2023-12-19T18:20:01.779503+0000","last_fullsized":"2023-12-19T18:20:01.779503+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:07.454418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.279544+0000","last_change":"2023-12-19T18:19:59.784892+0000","last_active":"2023-12-19T18:20:00.279544+0000","last_peered":"2023-12-19T18:20:00.279544+0000","last_clean":"2023-12-19T18:20:00.279544+0000","last_became_active":"2023-12-19T18:19:59.784406+0000","last_became_peered":"2023-12-19T18:19:59.784406+0000","last_unstale":"2023-12-19T18:20:00.279544+0000","last_undegraded":"2023-12-19T18:20:00.279544+0000","last_fullsized":"2023-12-19T18:20:00.279544+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:02:17.934136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305201+0000","last_change":"2023-12-19T18:20:00.281819+0000","last_active":"2023-12-19T18:20:03.305201+0000","last_peered":"2023-12-19T18:20:03.305201+0000","last_clean":"2023-12-19T18:20:03.305201+0000","last_became_active":"2023-12-19T18:20:00.281499+0000","last_became_peered":"2023-12-19T18:20:00.281499+0000","last_unstale":"2023-12-19T18:20:03.305201+0000","last_undegraded":"2023-12-19T18:20:03.305201+0000","last_fullsized":"2023-12-19T18:20:03.305201+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:14:14.407377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.281603+0000","last_change":"2023-12-19T18:19:59.785508+0000","last_active":"2023-12-19T18:20:00.281603+0000","last_peered":"2023-12-19T18:20:00.281603+0000","last_clean":"2023-12-19T18:20:00.281603+0000","last_became_active":"2023-12-19T18:19:59.785024+0000","last_became_peered":"2023-12-19T18:19:59.785024+0000","last_unstale":"2023-12-19T18:20:00.281603+0000","last_undegraded":"2023-12-19T18:20:00.281603+0000","last_fullsized":"2023-12-19T18:20:00.281603+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:08:03.195907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275012+0000","last_change":"2023-12-19T18:20:00.301214+0000","last_active":"2023-12-19T18:20:01.275012+0000","last_peered":"2023-12-19T18:20:01.275012+0000","last_clean":"2023-12-19T18:20:01.275012+0000","last_became_active":"2023-12-19T18:20:00.300460+0000","last_became_peered":"2023-12-19T18:20:00.300460+0000","last_unstale":"2023-12-19T18:20:01.275012+0000","last_undegraded":"2023-12-19T18:20:01.275012+0000","last_fullsized":"2023-12-19T18:20:01.275012+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:41:22.850569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.278166+0000","last_change":"2023-12-19T18:19:59.774874+0000","last_active":"2023-12-19T18:20:00.278166+0000","last_peered":"2023-12-19T18:20:00.278166+0000","last_clean":"2023-12-19T18:20:00.278166+0000","last_became_active":"2023-12-19T18:19:59.774277+0000","last_became_peered":"2023-12-19T18:19:59.774277+0000","last_unstale":"2023-12-19T18:20:00.278166+0000","last_undegraded":"2023-12-19T18:20:00.278166+0000","last_fullsized":"2023-12-19T18:20:00.278166+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:16:23.123225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.307689+0000","last_change":"2023-12-19T18:20:00.302991+0000","last_active":"2023-12-19T18:20:03.307689+0000","last_peered":"2023-12-19T18:20:03.307689+0000","last_clean":"2023-12-19T18:20:03.307689+0000","last_became_active":"2023-12-19T18:20:00.297999+0000","last_became_peered":"2023-12-19T18:20:00.297999+0000","last_unstale":"2023-12-19T18:20:03.307689+0000","last_undegraded":"2023-12-19T18:20:03.307689+0000","last_fullsized":"2023-12-19T18:20:03.307689+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:16.354181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.277003+0000","last_change":"2023-12-19T18:19:59.790846+0000","last_active":"2023-12-19T18:20:01.277003+0000","last_peered":"2023-12-19T18:20:01.277003+0000","last_clean":"2023-12-19T18:20:01.277003+0000","last_became_active":"2023-12-19T18:19:59.790553+0000","last_became_peered":"2023-12-19T18:19:59.790553+0000","last_unstale":"2023-12-19T18:20:01.277003+0000","last_undegraded":"2023-12-19T18:20:01.277003+0000","last_fullsized":"2023-12-19T18:20:01.277003+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:55:21.799985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779542+0000","last_change":"2023-12-19T18:20:00.289838+0000","last_active":"2023-12-19T18:20:01.779542+0000","last_peered":"2023-12-19T18:20:01.779542+0000","last_clean":"2023-12-19T18:20:01.779542+0000","last_became_active":"2023-12-19T18:20:00.289236+0000","last_became_peered":"2023-12-19T18:20:00.289236+0000","last_unstale":"2023-12-19T18:20:01.779542+0000","last_undegraded":"2023-12-19T18:20:01.779542+0000","last_fullsized":"2023-12-19T18:20:01.779542+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:21.279347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779236+0000","last_change":"2023-12-19T18:19:59.774095+0000","last_active":"2023-12-19T18:20:01.779236+0000","last_peered":"2023-12-19T18:20:01.779236+0000","last_clean":"2023-12-19T18:20:01.779236+0000","last_became_active":"2023-12-19T18:19:59.773691+0000","last_became_peered":"2023-12-19T18:19:59.773691+0000","last_unstale":"2023-12-19T18:20:01.779236+0000","last_undegraded":"2023-12-19T18:20:01.779236+0000","last_fullsized":"2023-12-19T18:20:01.779236+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:45.860622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.778967+0000","last_change":"2023-12-19T18:20:00.290178+0000","last_active":"2023-12-19T18:20:01.778967+0000","last_peered":"2023-12-19T18:20:01.778967+0000","last_clean":"2023-12-19T18:20:01.778967+0000","last_became_active":"2023-12-19T18:20:00.289413+0000","last_became_peered":"2023-12-19T18:20:00.289413+0000","last_unstale":"2023-12-19T18:20:01.778967+0000","last_undegraded":"2023-12-19T18:20:01.778967+0000","last_fullsized":"2023-12-19T18:20:01.778967+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:22.340036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.273430+0000","last_change":"2023-12-19T18:19:59.789939+0000","last_active":"2023-12-19T18:20:00.273430+0000","last_peered":"2023-12-19T18:20:00.273430+0000","last_clean":"2023-12-19T18:20:00.273430+0000","last_became_active":"2023-12-19T18:19:59.783654+0000","last_became_peered":"2023-12-19T18:19:59.783654+0000","last_unstale":"2023-12-19T18:20:00.273430+0000","last_undegraded":"2023-12-19T18:20:00.273430+0000","last_fullsized":"2023-12-19T18:20:00.273430+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:08:03.195907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.274655+0000","last_change":"2023-12-19T18:20:00.295026+0000","last_active":"2023-12-19T18:20:01.274655+0000","last_peered":"2023-12-19T18:20:01.274655+0000","last_clean":"2023-12-19T18:20:01.274655+0000","last_became_active":"2023-12-19T18:20:00.294785+0000","last_became_peered":"2023-12-19T18:20:00.294785+0000","last_unstale":"2023-12-19T18:20:01.274655+0000","last_undegraded":"2023-12-19T18:20:01.274655+0000","last_fullsized":"2023-12-19T18:20:01.274655+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:38.902775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306096+0000","last_change":"2023-12-19T18:19:59.786747+0000","last_active":"2023-12-19T18:20:03.306096+0000","last_peered":"2023-12-19T18:20:03.306096+0000","last_clean":"2023-12-19T18:20:03.306096+0000","last_became_active":"2023-12-19T18:19:59.786012+0000","last_became_peered":"2023-12-19T18:19:59.786012+0000","last_unstale":"2023-12-19T18:20:03.306096+0000","last_undegraded":"2023-12-19T18:20:03.306096+0000","last_fullsized":"2023-12-19T18:20:03.306096+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:55:44.169276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.278467+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:20:00.278467+0000","last_peered":"2023-12-19T18:20:00.278467+0000","last_clean":"2023-12-19T18:20:00.278467+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:20:00.278467+0000","last_undegraded":"2023-12-19T18:20:00.278467+0000","last_fullsized":"2023-12-19T18:20:00.278467+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477148,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.2250000000000001,"15min":1.1459999999999999},"min":{"1min":1.02,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":5.6879999999999997,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":1.419},{"interface":"front","average":{"1min":1.373,"5min":1.236,"15min":1.1719999999999999},"min":{"1min":0.91000000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.7330000000000001,"5min":1.79,"15min":1.79},"last":1.3879999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":1.361,"15min":1.232},"min":{"1min":0.86299999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":5.9029999999999996,"5min":10.926,"15min":10.926},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.3169999999999999,"15min":1.2330000000000001},"min":{"1min":0.93600000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6180000000000001,"5min":10.904999999999999,"15min":10.904999999999999},"last":1.3}]},{"osd":2,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.141,"15min":1.101},"min":{"1min":0.89400000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":3.1960000000000002,"5min":3.1960000000000002,"15min":3.1960000000000002},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.254,"15min":1.1859999999999999},"min":{"1min":1.0009999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":6.008,"5min":6.008,"15min":6.008},"last":1.2629999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.222,"15min":1.1359999999999999},"min":{"1min":0.95499999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":3.0379999999999998,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.391,"15min":1.274},"min":{"1min":1.1240000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":5.8120000000000003,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":1.373}]},{"osd":4,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.198,"15min":1.0940000000000001},"min":{"1min":0.82099999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.0110000000000001,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.2849999999999999,"15min":1.232},"min":{"1min":0.88600000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.2290000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.4339999999999999,"15min":1.335},"min":{"1min":0.81100000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":7.6429999999999998,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":1.119},{"interface":"front","average":{"1min":1.71,"5min":1.407,"15min":1.325},"min":{"1min":0.996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":6.0839999999999996,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":1.1930000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.6120000000000001,"5min":1.5209999999999999,"15min":1.4470000000000001},"min":{"1min":0.85899999999999999,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.1309999999999998,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.625,"5min":1.4510000000000001,"15min":1.3600000000000001},"min":{"1min":1.046,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":4.1479999999999997,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":1.1759999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477114,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.0529999999999999,"15min":0.98099999999999998},"min":{"1min":0.68899999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.546,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.26},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1319999999999999,"15min":1.05},"min":{"1min":0.64300000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.619,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":1.0089999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1339999999999999,"15min":1.1240000000000001},"min":{"1min":0.71399999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.488,"5min":4.5540000000000003,"15min":4.5540000000000003},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.175,"15min":1.1559999999999999},"min":{"1min":0.76300000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.746}]},{"osd":2,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2,"15min":1.1259999999999999},"min":{"1min":0.82199999999999995,"5min":0.626,"15min":0.626},"max":{"1min":1.7330000000000001,"5min":1.821,"15min":1.821},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.214,"15min":1.236},"min":{"1min":0.89200000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.494,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":1.3819999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2330000000000001,"15min":1.1779999999999999},"min":{"1min":0.98299999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.52,"5min":3.8820000000000001,"15min":3.8820000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.1879999999999999,"15min":1.1599999999999999},"min":{"1min":0.83799999999999997,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.5649999999999999,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":1.218}]},{"osd":4,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.27,"15min":1.238},"min":{"1min":0.81699999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9670000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.3939999999999999,"15min":1.3260000000000001},"min":{"1min":0.70799999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7490000000000001,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.7949999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.238,"15min":1.1659999999999999},"min":{"1min":0.66600000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5309999999999999,"5min":4.6680000000000001,"15min":4.6680000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.371,"15min":1.325},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.0489999999999999,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":2.0459999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.397,"15min":1.327},"min":{"1min":0.94399999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.702,"5min":3.9670000000000001,"15min":3.9670000000000001},"last":1.877},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.407,"15min":1.337},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.9339999999999999,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":1.5720000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198888254,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2050000000000001,"15min":1.1559999999999999},"min":{"1min":0.752,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.774,"5min":1.863,"15min":1.863},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.2470000000000001,"15min":1.1699999999999999},"min":{"1min":1.0549999999999999,"5min":0.75,"15min":0.75},"max":{"1min":1.6850000000000001,"5min":1.879,"15min":1.879},"last":1.3380000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.1419999999999999,"15min":1.0369999999999999},"min":{"1min":0.94999999999999996,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.246},{"interface":"front","average":{"1min":1.218,"5min":1.1240000000000001,"15min":1.046},"min":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6359999999999999,"5min":1.843,"15min":1.843},"last":1.5229999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.145,"15min":1.073},"min":{"1min":0.78400000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4970000000000001,"5min":1.794,"15min":1.794},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2070000000000001,"15min":1.131},"min":{"1min":0.86199999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.732,"5min":2.036,"15min":2.036},"last":0.84899999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.1499999999999999,"15min":1.073},"min":{"1min":0.86399999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.522,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.45},{"interface":"front","average":{"1min":1.417,"5min":1.3100000000000001,"15min":1.212},"min":{"1min":1.1020000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.597}]},{"osd":4,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.286,"15min":1.1579999999999999},"min":{"1min":0.98399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.802,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.1539999999999999,"15min":1.111},"min":{"1min":0.84199999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.49,"5min":1.901,"15min":1.901},"last":1.387}]},{"osd":6,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4259999999999999,"15min":1.3200000000000001},"min":{"1min":1.1850000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.71,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.544},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.28,"15min":1.228},"min":{"1min":1.0129999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.653,"5min":1.976,"15min":1.976},"last":1.2889999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2789999999999999,"15min":1.2170000000000001},"min":{"1min":0.88600000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.881,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.423},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4390000000000001,"15min":1.3999999999999999},"min":{"1min":1.177,"5min":0.623,"15min":0.623},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.498}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":509,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194802,"statfs":{"total":200000000,"available":199478018,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.996,"15min":0.997},"min":{"1min":0.69899999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.389,"5min":1.579,"15min":1.579},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.056,"15min":1.034},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4390000000000001,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.1639999999999999,"15min":1.1499999999999999},"min":{"1min":0.77900000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5589999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.383,"5min":1.2350000000000001,"15min":1.171},"min":{"1min":1.008,"5min":0.624,"15min":0.624},"max":{"1min":1.6220000000000001,"5min":1.696,"15min":1.696},"last":1.0980000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1180000000000001,"15min":1.1200000000000001},"min":{"1min":0.874,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.627,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.018},{"interface":"front","average":{"1min":1.149,"5min":1.113,"15min":1.0880000000000001},"min":{"1min":0.86099999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.4870000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2809999999999999,"15min":1.252},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.258},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.3120000000000001,"15min":1.294},"min":{"1min":0.91000000000000003,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.718,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.4239999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.2529999999999999,"15min":1.1850000000000001},"min":{"1min":1.0549999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6779999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.585},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.391,"15min":1.337},"min":{"1min":0.89600000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.7709999999999999,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.4670000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3149999999999999,"15min":1.302},"min":{"1min":1.0940000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.6619999999999999,"5min":1.853,"15min":1.853},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4219999999999999,"15min":1.3779999999999999},"min":{"1min":1.218,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.7310000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.659}]},{"osd":7,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3779999999999999,"15min":1.335},"min":{"1min":1.1220000000000001,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8640000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.389},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.415,"15min":1.401},"min":{"1min":1.1910000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.645,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.621}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":503,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194809,"statfs":{"total":200000000,"available":199484886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.90000000000000002,"15min":0.96199999999999997},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.26,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.95399999999999996,"15min":0.92800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3919999999999999,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.39500000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.99099999999999999,"15min":1.0549999999999999},"min":{"1min":0.44400000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.623,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":1.014,"5min":1.0569999999999999,"15min":1.089},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4570000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.57399999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2909999999999999,"15min":1.2390000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.8560000000000001,"5min":13.833,"15min":13.833},"last":0.48199999999999998},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.371,"15min":1.258},"min":{"1min":0.65900000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4950000000000001,"5min":16.748999999999999,"15min":16.748999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.256,"15min":1.1990000000000001},"min":{"1min":0.77400000000000002,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.641,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.375,"15min":1.403},"min":{"1min":0.89100000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.8100000000000001,"5min":4.0209999999999999,"15min":4.0209999999999999},"last":1.024}]},{"osd":5,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.381,"15min":1.431},"min":{"1min":0.82399999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5580000000000001,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.29,"15min":1.252},"min":{"1min":0.80600000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7010000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.84699999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.4159999999999999,"15min":1.4730000000000001},"min":{"1min":0.81799999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.9319999999999999,"5min":3.9399999999999999,"15min":3.9399999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4590000000000001,"15min":1.4970000000000001},"min":{"1min":0.92300000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.8240000000000001,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.115}]},{"osd":7,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.385,"15min":1.357},"min":{"1min":0.89500000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.9370000000000001,"5min":2.48,"15min":2.48},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.5249999999999999,"15min":1.5449999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.879,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.1539999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":509,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194802,"statfs":{"total":200000000,"available":199478130,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.0629999999999999,"15min":1.0169999999999999},"min":{"1min":0.35299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.5369999999999999,"5min":9.2550000000000008,"15min":9.2550000000000008},"last":1.105},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.004,"15min":1.0369999999999999},"min":{"1min":0.41999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3360000000000001,"5min":4.7850000000000001,"15min":4.7850000000000001},"last":1.8799999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.99299999999999999,"15min":1.202},"min":{"1min":0.55300000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5369999999999999,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.081,"15min":1.1679999999999999},"min":{"1min":0.504,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.6220000000000001,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":1.028}]},{"osd":3,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1819999999999999,"15min":1.3939999999999999},"min":{"1min":0.626,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6240000000000001,"5min":8.3109999999999999,"15min":8.3109999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.1879999999999999,"15min":1.3620000000000001},"min":{"1min":0.65100000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.4730000000000001,"5min":8.0690000000000008,"15min":8.0690000000000008},"last":1.3260000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.405,"15min":1.472},"min":{"1min":0.83399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.829,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4259999999999999,"15min":1.5580000000000001},"min":{"1min":1.036,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.75,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":2.3460000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.397,"15min":1.571},"min":{"1min":0.96599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.5529999999999999,"5min":8.1839999999999993,"15min":8.1839999999999993},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.3899999999999999,"15min":1.4119999999999999},"min":{"1min":1.1339999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.754,"5min":5.165,"15min":5.165},"last":1.5109999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3959999999999999,"15min":1.4490000000000001},"min":{"1min":1.004,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.6419999999999999,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":2.2429999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.456,"15min":1.575},"min":{"1min":0.93899999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.758,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":1.4550000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.484,"15min":1.55},"min":{"1min":0.91100000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.758,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":2.4430000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.5169999999999999,"15min":1.6679999999999999},"min":{"1min":0.88500000000000001,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.8759999999999999,"5min":8.2300000000000004,"15min":8.2300000000000004},"last":1.3979999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1079,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194233,"statfs":{"total":200000000,"available":198894644,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82299999999999995,"15min":0.83999999999999997},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.298,"5min":1.359,"15min":1.359},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.91200000000000003,"15min":0.95199999999999996},"min":{"1min":0.48899999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5760000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.74}]},{"osd":2,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0169999999999999,"15min":1.0489999999999999},"min":{"1min":0.42299999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.514,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.95099999999999996,"15min":0.90900000000000003},"min":{"1min":0.46700000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.456,"5min":1.845,"15min":1.845},"last":0.90400000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.2,"15min":1.1299999999999999},"min":{"1min":0.60799999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.8300000000000001,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.637},{"interface":"front","average":{"1min":1.242,"5min":1.167,"15min":1.179},"min":{"1min":0.59199999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7889999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.8049999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2789999999999999,"15min":1.2769999999999999},"min":{"1min":0.72599999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.6419999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.3109999999999999,"15min":1.278},"min":{"1min":0.79600000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.6479999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.409}]},{"osd":5,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.323,"15min":1.2729999999999999},"min":{"1min":0.94099999999999995,"5min":0.88,"15min":0.88},"max":{"1min":1.7689999999999999,"5min":2.04,"15min":2.04},"last":1.288},{"interface":"front","average":{"1min":1.383,"5min":1.4079999999999999,"15min":1.3899999999999999},"min":{"1min":0.95399999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.661,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.462}]},{"osd":6,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.373,"15min":1.3640000000000001},"min":{"1min":0.875,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.109,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.456,"15min":1.4299999999999999},"min":{"1min":0.92200000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.9039999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.601}]},{"osd":7,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.484,"15min":1.3620000000000001},"min":{"1min":0.96199999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.831,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":1.556},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.583,"15min":1.4830000000000001},"min":{"1min":1.012,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.2360000000000002,"5min":9.4109999999999996,"15min":9.4109999999999996},"last":1.5009999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":509,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194803,"statfs":{"total":200000000,"available":199478642,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.93799999999999994,"15min":0.88200000000000001},"min":{"1min":0.69599999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5720000000000001,"5min":1.887,"15min":1.887},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.083,"15min":1.0229999999999999},"min":{"1min":0.86399999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.639,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.169}]},{"osd":2,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":0.98899999999999999,"15min":0.94499999999999995},"min":{"1min":0.753,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.417,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":0.97499999999999998,"15min":0.92500000000000004},"min":{"1min":0.624,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.746,"5min":2.7810000000000001,"15min":2.7810000000000001},"last":1.3879999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0289999999999999,"15min":1.0049999999999999},"min":{"1min":0.65300000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.6950000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.2669999999999999,"15min":1.2190000000000001},"min":{"1min":0.80900000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.7450000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.696}]},{"osd":4,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.2669999999999999,"15min":1.1830000000000001},"min":{"1min":1.0169999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6970000000000001,"5min":1.903,"15min":1.903},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.27,"15min":1.2769999999999999},"min":{"1min":1.048,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.4570000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.5660000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.48,"15min":1.393},"min":{"1min":1.014,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.73,"5min":16.298999999999999,"15min":16.298999999999999},"last":1.7809999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.323,"15min":1.2649999999999999},"min":{"1min":1.2330000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.7010000000000001,"5min":1.95,"15min":1.95},"last":1.5029999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.423,"15min":1.383},"min":{"1min":1.1639999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8200000000000001,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.417},{"interface":"front","average":{"1min":1.464,"5min":1.4259999999999999,"15min":1.349},"min":{"1min":1.1120000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8220000000000001,"5min":2.46,"15min":2.46},"last":1.9179999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.466,"15min":1.3779999999999999},"min":{"1min":1.115,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.9370000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":2.024},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.3979999999999999,"15min":1.417},"min":{"1min":1.2150000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6990000000000001,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":1.877}]}]}],"pool_statfs":[]}} 2023-12-19T18:20:05.292 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:20:05.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.563+0000 7f7bb2b8a640 1 Processor -- start 2023-12-19T18:20:05.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.564+0000 7f7bb2b8a640 1 -- start start 2023-12-19T18:20:05.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.565+0000 7f7bb2b8a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7bac1604d0 0x7f7bac0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:05.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.565+0000 7f7bb2b8a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7bac0b7310 0x7f7bac0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:05.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.565+0000 7f7bb2b8a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bac0b5e90 con 0x7f7bac0d2350 2023-12-19T18:20:05.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.565+0000 7f7bb2b8a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bac0d2b30 con 0x7f7bac1604d0 2023-12-19T18:20:05.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.565+0000 7f7bb2b8a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bac0d2850 con 0x7f7bac0b7310 2023-12-19T18:20:05.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.566+0000 7f7bb1b88640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7bac1604d0 0x7f7bac0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:05.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.566+0000 7f7bb1b88640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7bac1604d0 0x7f7bac0b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51056/0 (socket says 172.21.15.27:51056) 2023-12-19T18:20:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.566+0000 7f7bb1b88640 1 -- 172.21.15.27:0/1152495365 learned_addr learned my addr 172.21.15.27:0/1152495365 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.566+0000 7f7bb2389640 1 --2- 172.21.15.27:0/1152495365 >> v2:172.21.15.192:3300/0 conn(0x7f7bac0b7310 0x7f7bac0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.566+0000 7f7bb1b88640 1 -- 172.21.15.27:0/1152495365 >> v2:172.21.15.192:3300/0 conn(0x7f7bac0b7310 msgr2=0x7f7bac0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.566+0000 7f7bb1b88640 1 --2- 172.21.15.27:0/1152495365 >> v2:172.21.15.192:3300/0 conn(0x7f7bac0b7310 0x7f7bac0b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.566+0000 7f7bb1b88640 1 -- 172.21.15.27:0/1152495365 >> v1:172.21.15.27:6789/0 conn(0x7f7bac0d2350 legacy=0x7f7bac0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.566+0000 7f7bb1b88640 1 -- 172.21.15.27:0/1152495365 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac0cded0 con 0x7f7bac1604d0 2023-12-19T18:20:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.566+0000 7f7bb2389640 1 --2- 172.21.15.27:0/1152495365 >> v2:172.21.15.192:3300/0 conn(0x7f7bac0b7310 0x7f7bac0b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.567+0000 7f7bb1b88640 1 --2- 172.21.15.27:0/1152495365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bac1604d0 0x7f7bac0b47e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0cd540 tx=0x7f7ba802b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23d11f54dd7bf3f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.568+0000 7f7bb0b86640 1 -- 172.21.15.27:0/1152495365 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8031880 con 0x7f7bac1604d0 2023-12-19T18:20:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.568+0000 7f7bb0b86640 1 -- 172.21.15.27:0/1152495365 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ba80319e0 con 0x7f7bac1604d0 2023-12-19T18:20:05.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.568+0000 7f7bb0b86640 1 -- 172.21.15.27:0/1152495365 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ba8031d70 con 0x7f7bac1604d0 2023-12-19T18:20:05.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.568+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/1152495365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bac1604d0 msgr2=0x7f7bac0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:05.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.568+0000 7f7bb2b8a640 1 --2- 172.21.15.27:0/1152495365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bac1604d0 0x7f7bac0b47e0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0cd540 tx=0x7f7ba802b920 comp rx=0 tx=0).stop 2023-12-19T18:20:05.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.568+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/1152495365 shutdown_connections 2023-12-19T18:20:05.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.568+0000 7f7bb2b8a640 1 --2- 172.21.15.27:0/1152495365 >> v2:172.21.15.192:3300/0 conn(0x7f7bac0b7310 0x7f7bac0b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.568+0000 7f7bb2b8a640 1 --2- 172.21.15.27:0/1152495365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bac1604d0 0x7f7bac0b47e0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.568+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/1152495365 >> 172.21.15.27:0/1152495365 conn(0x7f7bac0cf7e0 msgr2=0x7f7bac0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:05.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.568+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/1152495365 shutdown_connections 2023-12-19T18:20:05.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.568+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/1152495365 wait complete. 2023-12-19T18:20:05.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.570+0000 7f7bb2b8a640 1 Processor -- start 2023-12-19T18:20:05.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.570+0000 7f7bb2b8a640 1 -- start start 2023-12-19T18:20:05.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.571+0000 7f7bb2b8a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bac0b7310 0x7f7bac159170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:05.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.571+0000 7f7bb2b8a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bac0d2350 0x7f7bac0b7a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:05.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.571+0000 7f7bb2b8a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bac1604d0 0x7f7bac1562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:05.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.571+0000 7f7bb2b8a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac158d50 con 0x7f7bac0d2350 2023-12-19T18:20:05.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.571+0000 7f7bb2b8a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac1580d0 con 0x7f7bac0b7310 2023-12-19T18:20:05.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.571+0000 7f7bb2b8a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bac1583b0 con 0x7f7bac1604d0 2023-12-19T18:20:05.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.571+0000 7f7bb1387640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bac0d2350 0x7f7bac0b7a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:05.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.572+0000 7f7bb1387640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bac0d2350 0x7f7bac0b7a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51060/0 (socket says 172.21.15.27:51060) 2023-12-19T18:20:05.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.572+0000 7f7bb1387640 1 -- 172.21.15.27:0/2191792498 learned_addr learned my addr 172.21.15.27:0/2191792498 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:05.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.572+0000 7f7bb1b88640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bac0b7310 0x7f7bac159170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:05.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.572+0000 7f7bb2389640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bac1604d0 0x7f7bac1562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:05.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.572+0000 7f7bb1387640 1 -- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bac1604d0 msgr2=0x7f7bac1562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:05.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.572+0000 7f7bb1387640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bac1604d0 0x7f7bac1562c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.572+0000 7f7bb1387640 1 -- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bac0b7310 msgr2=0x7f7bac159170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:05.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.572+0000 7f7bb1387640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bac0b7310 0x7f7bac159170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.572+0000 7f7bb1387640 1 -- 172.21.15.27:0/2191792498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ba80094d0 con 0x7f7bac0d2350 2023-12-19T18:20:05.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.572+0000 7f7bb1b88640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bac0b7310 0x7f7bac159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:05.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.572+0000 7f7bb1387640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bac0d2350 0x7f7bac0b7a40 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00c400 tx=0x7f7b9c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:05.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.573+0000 7f7ba27fc640 1 -- 172.21.15.27:0/2191792498 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c0052d0 con 0x7f7bac0d2350 2023-12-19T18:20:05.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.573+0000 7f7ba27fc640 1 -- 172.21.15.27:0/2191792498 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b9c005430 con 0x7f7bac0d2350 2023-12-19T18:20:05.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.573+0000 7f7ba27fc640 1 -- 172.21.15.27:0/2191792498 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9c003830 con 0x7f7bac0d2350 2023-12-19T18:20:05.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.573+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/2191792498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bac158690 con 0x7f7bac0d2350 2023-12-19T18:20:05.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.573+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/2191792498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bac0c92d0 con 0x7f7bac0d2350 2023-12-19T18:20:05.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.574+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/2191792498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bac0c9490 con 0x7f7bac0d2350 2023-12-19T18:20:05.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.575+0000 7f7ba27fc640 1 -- 172.21.15.27:0/2191792498 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7b9c003990 con 0x7f7bac0d2350 2023-12-19T18:20:05.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.576+0000 7f7ba27fc640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b9006a430 0x7f7b9006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:05.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.576+0000 7f7ba27fc640 1 -- 172.21.15.27:0/2191792498 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7b9c087f20 con 0x7f7bac0d2350 2023-12-19T18:20:05.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.577+0000 7f7bb1b88640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b9006a430 0x7f7b9006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:05.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.578+0000 7f7bb1b88640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b9006a430 0x7f7b9006c8f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8036000 tx=0x7f7ba803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:05.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.581+0000 7f7ba27fc640 1 -- 172.21.15.27:0/2191792498 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7b9c053500 con 0x7f7bac0d2350 2023-12-19T18:20:05.735 INFO:tasks.ceph.mds.a.smithi027.stderr:2023-12-19T18:20:05.735+0000 7f3132af5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:20:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.776+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/2191792498 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7bac0ce490 con 0x7f7b9006a430 2023-12-19T18:20:05.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.798+0000 7f7ba27fc640 1 -- 172.21.15.27:0/2191792498 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282751 (secure 0 0 0) 0x7f7ba803a420 con 0x7f7b9006a430 2023-12-19T18:20:05.801 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:05.803 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:20:05.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.804+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b9006a430 msgr2=0x7f7b9006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.804+0000 7f7bb2b8a640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b9006a430 0x7f7b9006c8f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f7ba8036000 tx=0x7f7ba803c040 comp rx=0 tx=0).stop 2023-12-19T18:20:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.804+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/2191792498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bac0d2350 msgr2=0x7f7bac0b7a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.804+0000 7f7bb2b8a640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bac0d2350 0x7f7bac0b7a40 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7b9c00c400 tx=0x7f7b9c00c8c0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.804+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/2191792498 shutdown_connections 2023-12-19T18:20:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.805+0000 7f7bb2b8a640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7b9006a430 0x7f7b9006c8f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.805+0000 7f7bb2b8a640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bac1604d0 0x7f7bac1562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.805+0000 7f7bb2b8a640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bac0d2350 0x7f7bac0b7a40 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.805+0000 7f7bb2b8a640 1 --2- 172.21.15.27:0/2191792498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bac0b7310 0x7f7bac159170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.805+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/2191792498 >> 172.21.15.27:0/2191792498 conn(0x7f7bac0cf7e0 msgr2=0x7f7bac0b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.805+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/2191792498 shutdown_connections 2023-12-19T18:20:05.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:05.805+0000 7f7bb2b8a640 1 -- 172.21.15.27:0/2191792498 wait complete. 2023-12-19T18:20:05.828 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":248,"stamp":"2023-12-19T18:20:03.815122+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":151,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":5217,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1557282,"statfs":{"total":1600000000,"available":1594656836,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.060261"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306302+0000","last_change":"2023-12-19T18:20:00.304894+0000","last_active":"2023-12-19T18:20:03.306302+0000","last_peered":"2023-12-19T18:20:03.306302+0000","last_clean":"2023-12-19T18:20:03.306302+0000","last_became_active":"2023-12-19T18:20:00.301978+0000","last_became_peered":"2023-12-19T18:20:00.301978+0000","last_unstale":"2023-12-19T18:20:03.306302+0000","last_undegraded":"2023-12-19T18:20:03.306302+0000","last_fullsized":"2023-12-19T18:20:03.306302+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:08:41.917246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.778749+0000","last_change":"2023-12-19T18:20:00.288505+0000","last_active":"2023-12-19T18:20:01.778749+0000","last_peered":"2023-12-19T18:20:01.778749+0000","last_clean":"2023-12-19T18:20:01.778749+0000","last_became_active":"2023-12-19T18:20:00.287919+0000","last_became_peered":"2023-12-19T18:20:00.287919+0000","last_unstale":"2023-12-19T18:20:01.778749+0000","last_undegraded":"2023-12-19T18:20:01.778749+0000","last_fullsized":"2023-12-19T18:20:01.778749+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:31.358973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.284989+0000","last_change":"2023-12-19T18:20:00.303723+0000","last_active":"2023-12-19T18:20:01.284989+0000","last_peered":"2023-12-19T18:20:01.284989+0000","last_clean":"2023-12-19T18:20:01.284989+0000","last_became_active":"2023-12-19T18:20:00.303185+0000","last_became_peered":"2023-12-19T18:20:00.303185+0000","last_unstale":"2023-12-19T18:20:01.284989+0000","last_undegraded":"2023-12-19T18:20:01.284989+0000","last_fullsized":"2023-12-19T18:20:01.284989+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:08:04.208807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.284940+0000","last_change":"2023-12-19T18:20:00.302506+0000","last_active":"2023-12-19T18:20:01.284940+0000","last_peered":"2023-12-19T18:20:01.284940+0000","last_clean":"2023-12-19T18:20:01.284940+0000","last_became_active":"2023-12-19T18:20:00.302198+0000","last_became_peered":"2023-12-19T18:20:00.302198+0000","last_unstale":"2023-12-19T18:20:01.284940+0000","last_undegraded":"2023-12-19T18:20:01.284940+0000","last_fullsized":"2023-12-19T18:20:01.284940+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:37:06.953401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275748+0000","last_change":"2023-12-19T18:20:00.287459+0000","last_active":"2023-12-19T18:20:01.275748+0000","last_peered":"2023-12-19T18:20:01.275748+0000","last_clean":"2023-12-19T18:20:01.275748+0000","last_became_active":"2023-12-19T18:20:00.284333+0000","last_became_peered":"2023-12-19T18:20:00.284333+0000","last_unstale":"2023-12-19T18:20:01.275748+0000","last_undegraded":"2023-12-19T18:20:01.275748+0000","last_fullsized":"2023-12-19T18:20:01.275748+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:21:08.215981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275684+0000","last_change":"2023-12-19T18:20:00.301624+0000","last_active":"2023-12-19T18:20:01.275684+0000","last_peered":"2023-12-19T18:20:01.275684+0000","last_clean":"2023-12-19T18:20:01.275684+0000","last_became_active":"2023-12-19T18:20:00.301331+0000","last_became_peered":"2023-12-19T18:20:00.301331+0000","last_unstale":"2023-12-19T18:20:01.275684+0000","last_undegraded":"2023-12-19T18:20:01.275684+0000","last_fullsized":"2023-12-19T18:20:01.275684+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:46:53.516319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.304638+0000","last_change":"2023-12-19T18:20:00.290116+0000","last_active":"2023-12-19T18:20:03.304638+0000","last_peered":"2023-12-19T18:20:03.304638+0000","last_clean":"2023-12-19T18:20:03.304638+0000","last_became_active":"2023-12-19T18:20:00.289720+0000","last_became_peered":"2023-12-19T18:20:00.289720+0000","last_unstale":"2023-12-19T18:20:03.304638+0000","last_undegraded":"2023-12-19T18:20:03.304638+0000","last_fullsized":"2023-12-19T18:20:03.304638+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:46:13.690282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306402+0000","last_change":"2023-12-19T18:20:00.299960+0000","last_active":"2023-12-19T18:20:03.306402+0000","last_peered":"2023-12-19T18:20:03.306402+0000","last_clean":"2023-12-19T18:20:03.306402+0000","last_became_active":"2023-12-19T18:20:00.296942+0000","last_became_peered":"2023-12-19T18:20:00.296942+0000","last_unstale":"2023-12-19T18:20:03.306402+0000","last_undegraded":"2023-12-19T18:20:03.306402+0000","last_fullsized":"2023-12-19T18:20:03.306402+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:25:37.993826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285133+0000","last_change":"2023-12-19T18:20:00.301936+0000","last_active":"2023-12-19T18:20:01.285133+0000","last_peered":"2023-12-19T18:20:01.285133+0000","last_clean":"2023-12-19T18:20:01.285133+0000","last_became_active":"2023-12-19T18:20:00.301656+0000","last_became_peered":"2023-12-19T18:20:00.301656+0000","last_unstale":"2023-12-19T18:20:01.285133+0000","last_undegraded":"2023-12-19T18:20:01.285133+0000","last_fullsized":"2023-12-19T18:20:01.285133+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:51.525570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305215+0000","last_change":"2023-12-19T18:20:00.291042+0000","last_active":"2023-12-19T18:20:03.305215+0000","last_peered":"2023-12-19T18:20:03.305215+0000","last_clean":"2023-12-19T18:20:03.305215+0000","last_became_active":"2023-12-19T18:20:00.290468+0000","last_became_peered":"2023-12-19T18:20:00.290468+0000","last_unstale":"2023-12-19T18:20:03.305215+0000","last_undegraded":"2023-12-19T18:20:03.305215+0000","last_fullsized":"2023-12-19T18:20:03.305215+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:22:22.842549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.307068+0000","last_change":"2023-12-19T18:20:00.302200+0000","last_active":"2023-12-19T18:20:03.307068+0000","last_peered":"2023-12-19T18:20:03.307068+0000","last_clean":"2023-12-19T18:20:03.307068+0000","last_became_active":"2023-12-19T18:20:00.297369+0000","last_became_peered":"2023-12-19T18:20:00.297369+0000","last_unstale":"2023-12-19T18:20:03.307068+0000","last_undegraded":"2023-12-19T18:20:03.307068+0000","last_fullsized":"2023-12-19T18:20:03.307068+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:22:50.155455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.277713+0000","last_change":"2023-12-19T18:20:00.299152+0000","last_active":"2023-12-19T18:20:01.277713+0000","last_peered":"2023-12-19T18:20:01.277713+0000","last_clean":"2023-12-19T18:20:01.277713+0000","last_became_active":"2023-12-19T18:20:00.298501+0000","last_became_peered":"2023-12-19T18:20:00.298501+0000","last_unstale":"2023-12-19T18:20:01.277713+0000","last_undegraded":"2023-12-19T18:20:01.277713+0000","last_fullsized":"2023-12-19T18:20:01.277713+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:46:53.516319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285269+0000","last_change":"2023-12-19T18:20:00.304617+0000","last_active":"2023-12-19T18:20:01.285269+0000","last_peered":"2023-12-19T18:20:01.285269+0000","last_clean":"2023-12-19T18:20:01.285269+0000","last_became_active":"2023-12-19T18:20:00.303490+0000","last_became_peered":"2023-12-19T18:20:00.303490+0000","last_unstale":"2023-12-19T18:20:01.285269+0000","last_undegraded":"2023-12-19T18:20:01.285269+0000","last_fullsized":"2023-12-19T18:20:01.285269+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:06.820663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.276901+0000","last_change":"2023-12-19T18:20:00.286672+0000","last_active":"2023-12-19T18:20:01.276901+0000","last_peered":"2023-12-19T18:20:01.276901+0000","last_clean":"2023-12-19T18:20:01.276901+0000","last_became_active":"2023-12-19T18:20:00.284223+0000","last_became_peered":"2023-12-19T18:20:00.284223+0000","last_unstale":"2023-12-19T18:20:01.276901+0000","last_undegraded":"2023-12-19T18:20:01.276901+0000","last_fullsized":"2023-12-19T18:20:01.276901+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:07.454418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.278087+0000","last_change":"2023-12-19T18:20:00.290245+0000","last_active":"2023-12-19T18:20:01.278087+0000","last_peered":"2023-12-19T18:20:01.278087+0000","last_clean":"2023-12-19T18:20:01.278087+0000","last_became_active":"2023-12-19T18:20:00.289155+0000","last_became_peered":"2023-12-19T18:20:00.289155+0000","last_unstale":"2023-12-19T18:20:01.278087+0000","last_undegraded":"2023-12-19T18:20:01.278087+0000","last_fullsized":"2023-12-19T18:20:01.278087+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:22.340036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.307905+0000","last_change":"2023-12-19T18:20:00.302332+0000","last_active":"2023-12-19T18:20:03.307905+0000","last_peered":"2023-12-19T18:20:03.307905+0000","last_clean":"2023-12-19T18:20:03.307905+0000","last_became_active":"2023-12-19T18:20:00.297724+0000","last_became_peered":"2023-12-19T18:20:00.297724+0000","last_unstale":"2023-12-19T18:20:03.307905+0000","last_undegraded":"2023-12-19T18:20:03.307905+0000","last_fullsized":"2023-12-19T18:20:03.307905+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:14:22.920400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.274768+0000","last_change":"2023-12-19T18:20:00.301404+0000","last_active":"2023-12-19T18:20:01.274768+0000","last_peered":"2023-12-19T18:20:01.274768+0000","last_clean":"2023-12-19T18:20:01.274768+0000","last_became_active":"2023-12-19T18:20:00.300818+0000","last_became_peered":"2023-12-19T18:20:00.300818+0000","last_unstale":"2023-12-19T18:20:01.274768+0000","last_undegraded":"2023-12-19T18:20:01.274768+0000","last_fullsized":"2023-12-19T18:20:01.274768+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:01:38.275644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.274761+0000","last_change":"2023-12-19T18:20:00.287659+0000","last_active":"2023-12-19T18:20:01.274761+0000","last_peered":"2023-12-19T18:20:01.274761+0000","last_clean":"2023-12-19T18:20:01.274761+0000","last_became_active":"2023-12-19T18:20:00.286002+0000","last_became_peered":"2023-12-19T18:20:00.286002+0000","last_unstale":"2023-12-19T18:20:01.274761+0000","last_undegraded":"2023-12-19T18:20:01.274761+0000","last_fullsized":"2023-12-19T18:20:01.274761+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:15.918235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306298+0000","last_change":"2023-12-19T18:20:00.304463+0000","last_active":"2023-12-19T18:20:03.306298+0000","last_peered":"2023-12-19T18:20:03.306298+0000","last_clean":"2023-12-19T18:20:03.306298+0000","last_became_active":"2023-12-19T18:20:00.304240+0000","last_became_peered":"2023-12-19T18:20:00.304240+0000","last_unstale":"2023-12-19T18:20:03.306298+0000","last_undegraded":"2023-12-19T18:20:03.306298+0000","last_fullsized":"2023-12-19T18:20:03.306298+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:20:28.327402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275993+0000","last_change":"2023-12-19T18:20:00.287872+0000","last_active":"2023-12-19T18:20:01.275993+0000","last_peered":"2023-12-19T18:20:01.275993+0000","last_clean":"2023-12-19T18:20:01.275993+0000","last_became_active":"2023-12-19T18:20:00.284927+0000","last_became_peered":"2023-12-19T18:20:00.284927+0000","last_unstale":"2023-12-19T18:20:01.275993+0000","last_undegraded":"2023-12-19T18:20:01.275993+0000","last_fullsized":"2023-12-19T18:20:01.275993+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:38:59.579237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.778977+0000","last_change":"2023-12-19T18:20:00.297701+0000","last_active":"2023-12-19T18:20:01.778977+0000","last_peered":"2023-12-19T18:20:01.778977+0000","last_clean":"2023-12-19T18:20:01.778977+0000","last_became_active":"2023-12-19T18:20:00.297484+0000","last_became_peered":"2023-12-19T18:20:00.297484+0000","last_unstale":"2023-12-19T18:20:01.778977+0000","last_undegraded":"2023-12-19T18:20:01.778977+0000","last_fullsized":"2023-12-19T18:20:01.778977+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:46:53.516319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275807+0000","last_change":"2023-12-19T18:20:00.289902+0000","last_active":"2023-12-19T18:20:01.275807+0000","last_peered":"2023-12-19T18:20:01.275807+0000","last_clean":"2023-12-19T18:20:01.275807+0000","last_became_active":"2023-12-19T18:20:00.288872+0000","last_became_peered":"2023-12-19T18:20:00.288872+0000","last_unstale":"2023-12-19T18:20:01.275807+0000","last_undegraded":"2023-12-19T18:20:01.275807+0000","last_fullsized":"2023-12-19T18:20:01.275807+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:01:06.119977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305323+0000","last_change":"2023-12-19T18:20:00.286335+0000","last_active":"2023-12-19T18:20:03.305323+0000","last_peered":"2023-12-19T18:20:03.305323+0000","last_clean":"2023-12-19T18:20:03.305323+0000","last_became_active":"2023-12-19T18:20:00.284662+0000","last_became_peered":"2023-12-19T18:20:00.284662+0000","last_unstale":"2023-12-19T18:20:03.305323+0000","last_undegraded":"2023-12-19T18:20:03.305323+0000","last_fullsized":"2023-12-19T18:20:03.305323+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:52:00.930870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306584+0000","last_change":"2023-12-19T18:20:00.291608+0000","last_active":"2023-12-19T18:20:03.306584+0000","last_peered":"2023-12-19T18:20:03.306584+0000","last_clean":"2023-12-19T18:20:03.306584+0000","last_became_active":"2023-12-19T18:20:00.290726+0000","last_became_peered":"2023-12-19T18:20:00.290726+0000","last_unstale":"2023-12-19T18:20:03.306584+0000","last_undegraded":"2023-12-19T18:20:03.306584+0000","last_fullsized":"2023-12-19T18:20:03.306584+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:44.483918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.304997+0000","last_change":"2023-12-19T18:20:00.291181+0000","last_active":"2023-12-19T18:20:03.304997+0000","last_peered":"2023-12-19T18:20:03.304997+0000","last_clean":"2023-12-19T18:20:03.304997+0000","last_became_active":"2023-12-19T18:20:00.290802+0000","last_became_peered":"2023-12-19T18:20:00.290802+0000","last_unstale":"2023-12-19T18:20:03.304997+0000","last_undegraded":"2023-12-19T18:20:03.304997+0000","last_fullsized":"2023-12-19T18:20:03.304997+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:21:12.632362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.307472+0000","last_change":"2023-12-19T18:20:00.303648+0000","last_active":"2023-12-19T18:20:03.307472+0000","last_peered":"2023-12-19T18:20:03.307472+0000","last_clean":"2023-12-19T18:20:03.307472+0000","last_became_active":"2023-12-19T18:20:00.301502+0000","last_became_peered":"2023-12-19T18:20:00.301502+0000","last_unstale":"2023-12-19T18:20:03.307472+0000","last_undegraded":"2023-12-19T18:20:03.307472+0000","last_fullsized":"2023-12-19T18:20:03.307472+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:20:14.411996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.276804+0000","last_change":"2023-12-19T18:20:00.301064+0000","last_active":"2023-12-19T18:20:01.276804+0000","last_peered":"2023-12-19T18:20:01.276804+0000","last_clean":"2023-12-19T18:20:01.276804+0000","last_became_active":"2023-12-19T18:20:00.298390+0000","last_became_peered":"2023-12-19T18:20:00.298390+0000","last_unstale":"2023-12-19T18:20:01.276804+0000","last_undegraded":"2023-12-19T18:20:01.276804+0000","last_fullsized":"2023-12-19T18:20:01.276804+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:37:43.579760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.778661+0000","last_change":"2023-12-19T18:20:00.297059+0000","last_active":"2023-12-19T18:20:01.778661+0000","last_peered":"2023-12-19T18:20:01.778661+0000","last_clean":"2023-12-19T18:20:01.778661+0000","last_became_active":"2023-12-19T18:20:00.296828+0000","last_became_peered":"2023-12-19T18:20:00.296828+0000","last_unstale":"2023-12-19T18:20:01.778661+0000","last_undegraded":"2023-12-19T18:20:01.778661+0000","last_fullsized":"2023-12-19T18:20:01.778661+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:01:06.119977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285385+0000","last_change":"2023-12-19T18:20:00.299968+0000","last_active":"2023-12-19T18:20:01.285385+0000","last_peered":"2023-12-19T18:20:01.285385+0000","last_clean":"2023-12-19T18:20:01.285385+0000","last_became_active":"2023-12-19T18:20:00.299573+0000","last_became_peered":"2023-12-19T18:20:00.299573+0000","last_unstale":"2023-12-19T18:20:01.285385+0000","last_undegraded":"2023-12-19T18:20:01.285385+0000","last_fullsized":"2023-12-19T18:20:01.285385+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:43:02.641137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306721+0000","last_change":"2023-12-19T18:20:00.298663+0000","last_active":"2023-12-19T18:20:03.306721+0000","last_peered":"2023-12-19T18:20:03.306721+0000","last_clean":"2023-12-19T18:20:03.306721+0000","last_became_active":"2023-12-19T18:20:00.298324+0000","last_became_peered":"2023-12-19T18:20:00.298324+0000","last_unstale":"2023-12-19T18:20:03.306721+0000","last_undegraded":"2023-12-19T18:20:03.306721+0000","last_fullsized":"2023-12-19T18:20:03.306721+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:31:11.720275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779248+0000","last_change":"2023-12-19T18:20:00.296832+0000","last_active":"2023-12-19T18:20:01.779248+0000","last_peered":"2023-12-19T18:20:01.779248+0000","last_clean":"2023-12-19T18:20:01.779248+0000","last_became_active":"2023-12-19T18:20:00.296605+0000","last_became_peered":"2023-12-19T18:20:00.296605+0000","last_unstale":"2023-12-19T18:20:01.779248+0000","last_undegraded":"2023-12-19T18:20:01.779248+0000","last_fullsized":"2023-12-19T18:20:01.779248+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:34:18.478814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779047+0000","last_change":"2023-12-19T18:20:00.299455+0000","last_active":"2023-12-19T18:20:01.779047+0000","last_peered":"2023-12-19T18:20:01.779047+0000","last_clean":"2023-12-19T18:20:01.779047+0000","last_became_active":"2023-12-19T18:20:00.299198+0000","last_became_peered":"2023-12-19T18:20:00.299198+0000","last_unstale":"2023-12-19T18:20:01.779047+0000","last_undegraded":"2023-12-19T18:20:01.779047+0000","last_fullsized":"2023-12-19T18:20:01.779047+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:19:28.119124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306777+0000","last_change":"2023-12-19T18:20:00.298188+0000","last_active":"2023-12-19T18:20:03.306777+0000","last_peered":"2023-12-19T18:20:03.306777+0000","last_clean":"2023-12-19T18:20:03.306777+0000","last_became_active":"2023-12-19T18:20:00.296149+0000","last_became_peered":"2023-12-19T18:20:00.296149+0000","last_unstale":"2023-12-19T18:20:03.306777+0000","last_undegraded":"2023-12-19T18:20:03.306777+0000","last_fullsized":"2023-12-19T18:20:03.306777+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:54:01.938427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"95'4","reported_seq":15,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.301034+0000","last_change":"2023-12-19T18:19:59.770522+0000","last_active":"2023-12-19T18:20:00.301034+0000","last_peered":"2023-12-19T18:20:00.301034+0000","last_clean":"2023-12-19T18:20:00.301034+0000","last_became_active":"2023-12-19T18:19:59.769869+0000","last_became_peered":"2023-12-19T18:19:59.769869+0000","last_unstale":"2023-12-19T18:20:00.301034+0000","last_undegraded":"2023-12-19T18:20:00.301034+0000","last_fullsized":"2023-12-19T18:20:00.301034+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:05.807764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285542+0000","last_change":"2023-12-19T18:20:00.305850+0000","last_active":"2023-12-19T18:20:01.285542+0000","last_peered":"2023-12-19T18:20:01.285542+0000","last_clean":"2023-12-19T18:20:01.285542+0000","last_became_active":"2023-12-19T18:20:00.305406+0000","last_became_peered":"2023-12-19T18:20:00.305406+0000","last_unstale":"2023-12-19T18:20:01.285542+0000","last_undegraded":"2023-12-19T18:20:01.285542+0000","last_fullsized":"2023-12-19T18:20:01.285542+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:21.279347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.274186+0000","last_change":"2023-12-19T18:19:59.784415+0000","last_active":"2023-12-19T18:20:01.274186+0000","last_peered":"2023-12-19T18:20:01.274186+0000","last_clean":"2023-12-19T18:20:01.274186+0000","last_became_active":"2023-12-19T18:19:59.783902+0000","last_became_peered":"2023-12-19T18:19:59.783902+0000","last_unstale":"2023-12-19T18:20:01.274186+0000","last_undegraded":"2023-12-19T18:20:01.274186+0000","last_fullsized":"2023-12-19T18:20:01.274186+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:25:07.322258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275832+0000","last_change":"2023-12-19T18:20:00.295082+0000","last_active":"2023-12-19T18:20:01.275832+0000","last_peered":"2023-12-19T18:20:01.275832+0000","last_clean":"2023-12-19T18:20:01.275832+0000","last_became_active":"2023-12-19T18:20:00.294842+0000","last_became_peered":"2023-12-19T18:20:00.294842+0000","last_unstale":"2023-12-19T18:20:01.275832+0000","last_undegraded":"2023-12-19T18:20:01.275832+0000","last_fullsized":"2023-12-19T18:20:01.275832+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:34:18.478814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"95'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.298772+0000","last_change":"2023-12-19T18:19:59.791361+0000","last_active":"2023-12-19T18:20:00.298772+0000","last_peered":"2023-12-19T18:20:00.298772+0000","last_clean":"2023-12-19T18:20:00.298772+0000","last_became_active":"2023-12-19T18:19:59.791131+0000","last_became_peered":"2023-12-19T18:19:59.791131+0000","last_unstale":"2023-12-19T18:20:00.298772+0000","last_undegraded":"2023-12-19T18:20:00.298772+0000","last_fullsized":"2023-12-19T18:20:00.298772+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:17.492321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305459+0000","last_change":"2023-12-19T18:20:00.285747+0000","last_active":"2023-12-19T18:20:03.305459+0000","last_peered":"2023-12-19T18:20:03.305459+0000","last_clean":"2023-12-19T18:20:03.305459+0000","last_became_active":"2023-12-19T18:20:00.285530+0000","last_became_peered":"2023-12-19T18:20:00.285530+0000","last_unstale":"2023-12-19T18:20:03.305459+0000","last_undegraded":"2023-12-19T18:20:03.305459+0000","last_fullsized":"2023-12-19T18:20:03.305459+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:57:12.333501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"95'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.296687+0000","last_change":"2023-12-19T18:19:59.785574+0000","last_active":"2023-12-19T18:20:00.296687+0000","last_peered":"2023-12-19T18:20:00.296687+0000","last_clean":"2023-12-19T18:20:00.296687+0000","last_became_active":"2023-12-19T18:19:59.784958+0000","last_became_peered":"2023-12-19T18:19:59.784958+0000","last_unstale":"2023-12-19T18:20:00.296687+0000","last_undegraded":"2023-12-19T18:20:00.296687+0000","last_fullsized":"2023-12-19T18:20:00.296687+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:43:01.628238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306126+0000","last_change":"2023-12-19T18:20:00.300634+0000","last_active":"2023-12-19T18:20:03.306126+0000","last_peered":"2023-12-19T18:20:03.306126+0000","last_clean":"2023-12-19T18:20:03.306126+0000","last_became_active":"2023-12-19T18:20:00.299718+0000","last_became_peered":"2023-12-19T18:20:00.299718+0000","last_unstale":"2023-12-19T18:20:03.306126+0000","last_undegraded":"2023-12-19T18:20:03.306126+0000","last_fullsized":"2023-12-19T18:20:03.306126+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:01:01.654519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"95'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.298167+0000","last_change":"2023-12-19T18:19:59.775462+0000","last_active":"2023-12-19T18:20:00.298167+0000","last_peered":"2023-12-19T18:20:00.298167+0000","last_clean":"2023-12-19T18:20:00.298167+0000","last_became_active":"2023-12-19T18:19:59.774457+0000","last_became_peered":"2023-12-19T18:19:59.774457+0000","last_unstale":"2023-12-19T18:20:00.298167+0000","last_undegraded":"2023-12-19T18:20:00.298167+0000","last_fullsized":"2023-12-19T18:20:00.298167+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:43:01.628238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275907+0000","last_change":"2023-12-19T18:20:00.290984+0000","last_active":"2023-12-19T18:20:01.275907+0000","last_peered":"2023-12-19T18:20:01.275907+0000","last_clean":"2023-12-19T18:20:01.275907+0000","last_became_active":"2023-12-19T18:20:00.290765+0000","last_became_peered":"2023-12-19T18:20:00.290765+0000","last_unstale":"2023-12-19T18:20:01.275907+0000","last_undegraded":"2023-12-19T18:20:01.275907+0000","last_fullsized":"2023-12-19T18:20:01.275907+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:35.301273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305595+0000","last_change":"2023-12-19T18:19:59.792674+0000","last_active":"2023-12-19T18:20:03.305595+0000","last_peered":"2023-12-19T18:20:03.305595+0000","last_clean":"2023-12-19T18:20:03.305595+0000","last_became_active":"2023-12-19T18:19:59.791269+0000","last_became_peered":"2023-12-19T18:19:59.791269+0000","last_unstale":"2023-12-19T18:20:03.305595+0000","last_undegraded":"2023-12-19T18:20:03.305595+0000","last_fullsized":"2023-12-19T18:20:03.305595+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:01:01.346973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779788+0000","last_change":"2023-12-19T18:20:00.302341+0000","last_active":"2023-12-19T18:20:01.779788+0000","last_peered":"2023-12-19T18:20:01.779788+0000","last_clean":"2023-12-19T18:20:01.779788+0000","last_became_active":"2023-12-19T18:20:00.302101+0000","last_became_peered":"2023-12-19T18:20:00.302101+0000","last_unstale":"2023-12-19T18:20:01.779788+0000","last_undegraded":"2023-12-19T18:20:01.779788+0000","last_fullsized":"2023-12-19T18:20:01.779788+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:08:04.208807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779974+0000","last_change":"2023-12-19T18:19:59.790921+0000","last_active":"2023-12-19T18:20:01.779974+0000","last_peered":"2023-12-19T18:20:01.779974+0000","last_clean":"2023-12-19T18:20:01.779974+0000","last_became_active":"2023-12-19T18:19:59.790726+0000","last_became_peered":"2023-12-19T18:19:59.790726+0000","last_unstale":"2023-12-19T18:20:01.779974+0000","last_undegraded":"2023-12-19T18:20:01.779974+0000","last_fullsized":"2023-12-19T18:20:01.779974+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:36.681763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.277143+0000","last_change":"2023-12-19T18:20:00.284839+0000","last_active":"2023-12-19T18:20:01.277143+0000","last_peered":"2023-12-19T18:20:01.277143+0000","last_clean":"2023-12-19T18:20:01.277143+0000","last_became_active":"2023-12-19T18:20:00.283239+0000","last_became_peered":"2023-12-19T18:20:00.283239+0000","last_unstale":"2023-12-19T18:20:01.277143+0000","last_undegraded":"2023-12-19T18:20:01.277143+0000","last_fullsized":"2023-12-19T18:20:01.277143+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:31.358973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.281383+0000","last_change":"2023-12-19T18:19:59.781903+0000","last_active":"2023-12-19T18:20:00.281383+0000","last_peered":"2023-12-19T18:20:00.281383+0000","last_clean":"2023-12-19T18:20:00.281383+0000","last_became_active":"2023-12-19T18:19:59.781313+0000","last_became_peered":"2023-12-19T18:19:59.781313+0000","last_unstale":"2023-12-19T18:20:00.281383+0000","last_undegraded":"2023-12-19T18:20:00.281383+0000","last_fullsized":"2023-12-19T18:20:00.281383+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:53:15.711817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.304930+0000","last_change":"2023-12-19T18:20:00.282702+0000","last_active":"2023-12-19T18:20:03.304930+0000","last_peered":"2023-12-19T18:20:03.304930+0000","last_clean":"2023-12-19T18:20:03.304930+0000","last_became_active":"2023-12-19T18:20:00.282401+0000","last_became_peered":"2023-12-19T18:20:00.282401+0000","last_unstale":"2023-12-19T18:20:03.304930+0000","last_undegraded":"2023-12-19T18:20:03.304930+0000","last_fullsized":"2023-12-19T18:20:03.304930+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:15.972242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.292373+0000","last_change":"2023-12-19T18:19:59.786867+0000","last_active":"2023-12-19T18:20:00.292373+0000","last_peered":"2023-12-19T18:20:00.292373+0000","last_clean":"2023-12-19T18:20:00.292373+0000","last_became_active":"2023-12-19T18:19:59.784118+0000","last_became_peered":"2023-12-19T18:19:59.784118+0000","last_unstale":"2023-12-19T18:20:00.292373+0000","last_undegraded":"2023-12-19T18:20:00.292373+0000","last_fullsized":"2023-12-19T18:20:00.292373+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:37:42.566861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779419+0000","last_change":"2023-12-19T18:20:00.289832+0000","last_active":"2023-12-19T18:20:01.779419+0000","last_peered":"2023-12-19T18:20:01.779419+0000","last_clean":"2023-12-19T18:20:01.779419+0000","last_became_active":"2023-12-19T18:20:00.287480+0000","last_became_peered":"2023-12-19T18:20:00.287480+0000","last_unstale":"2023-12-19T18:20:01.779419+0000","last_undegraded":"2023-12-19T18:20:01.779419+0000","last_fullsized":"2023-12-19T18:20:01.779419+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:43.029632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779299+0000","last_change":"2023-12-19T18:19:59.776511+0000","last_active":"2023-12-19T18:20:01.779299+0000","last_peered":"2023-12-19T18:20:01.779299+0000","last_clean":"2023-12-19T18:20:01.779299+0000","last_became_active":"2023-12-19T18:19:59.776115+0000","last_became_peered":"2023-12-19T18:19:59.776115+0000","last_unstale":"2023-12-19T18:20:01.779299+0000","last_undegraded":"2023-12-19T18:20:01.779299+0000","last_fullsized":"2023-12-19T18:20:01.779299+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:43:05.535962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306900+0000","last_change":"2023-12-19T18:20:00.291458+0000","last_active":"2023-12-19T18:20:03.306900+0000","last_peered":"2023-12-19T18:20:03.306900+0000","last_clean":"2023-12-19T18:20:03.306900+0000","last_became_active":"2023-12-19T18:20:00.289773+0000","last_became_peered":"2023-12-19T18:20:00.289773+0000","last_unstale":"2023-12-19T18:20:03.306900+0000","last_undegraded":"2023-12-19T18:20:03.306900+0000","last_fullsized":"2023-12-19T18:20:03.306900+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:24.817998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"95'3","reported_seq":14,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.303839+0000","last_change":"2023-12-19T18:19:59.786986+0000","last_active":"2023-12-19T18:20:00.303839+0000","last_peered":"2023-12-19T18:20:00.303839+0000","last_clean":"2023-12-19T18:20:00.303839+0000","last_became_active":"2023-12-19T18:19:59.786497+0000","last_became_peered":"2023-12-19T18:19:59.786497+0000","last_unstale":"2023-12-19T18:20:00.303839+0000","last_undegraded":"2023-12-19T18:20:00.303839+0000","last_fullsized":"2023-12-19T18:20:00.303839+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:50.512671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275004+0000","last_change":"2023-12-19T18:20:00.298187+0000","last_active":"2023-12-19T18:20:01.275004+0000","last_peered":"2023-12-19T18:20:01.275004+0000","last_clean":"2023-12-19T18:20:01.275004+0000","last_became_active":"2023-12-19T18:20:00.297199+0000","last_became_peered":"2023-12-19T18:20:00.297199+0000","last_unstale":"2023-12-19T18:20:01.275004+0000","last_undegraded":"2023-12-19T18:20:01.275004+0000","last_fullsized":"2023-12-19T18:20:01.275004+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:18.505220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"95'6","reported_seq":17,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.780198+0000","last_change":"2023-12-19T18:19:59.789502+0000","last_active":"2023-12-19T18:20:01.780198+0000","last_peered":"2023-12-19T18:20:01.780198+0000","last_clean":"2023-12-19T18:20:01.780198+0000","last_became_active":"2023-12-19T18:19:59.789090+0000","last_became_peered":"2023-12-19T18:19:59.789090+0000","last_unstale":"2023-12-19T18:20:01.780198+0000","last_undegraded":"2023-12-19T18:20:01.780198+0000","last_fullsized":"2023-12-19T18:20:01.780198+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:45.860622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275997+0000","last_change":"2023-12-19T18:20:00.280622+0000","last_active":"2023-12-19T18:20:01.275997+0000","last_peered":"2023-12-19T18:20:01.275997+0000","last_clean":"2023-12-19T18:20:01.275997+0000","last_became_active":"2023-12-19T18:20:00.280402+0000","last_became_peered":"2023-12-19T18:20:00.280402+0000","last_unstale":"2023-12-19T18:20:01.275997+0000","last_undegraded":"2023-12-19T18:20:01.275997+0000","last_fullsized":"2023-12-19T18:20:01.275997+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:05.897496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.780018+0000","last_change":"2023-12-19T18:19:59.783119+0000","last_active":"2023-12-19T18:20:01.780018+0000","last_peered":"2023-12-19T18:20:01.780018+0000","last_clean":"2023-12-19T18:20:01.780018+0000","last_became_active":"2023-12-19T18:19:59.782829+0000","last_became_peered":"2023-12-19T18:19:59.782829+0000","last_unstale":"2023-12-19T18:20:01.780018+0000","last_undegraded":"2023-12-19T18:20:01.780018+0000","last_fullsized":"2023-12-19T18:20:01.780018+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:21:44.158670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305711+0000","last_change":"2023-12-19T18:20:00.297847+0000","last_active":"2023-12-19T18:20:03.305711+0000","last_peered":"2023-12-19T18:20:03.305711+0000","last_clean":"2023-12-19T18:20:03.305711+0000","last_became_active":"2023-12-19T18:20:00.295980+0000","last_became_peered":"2023-12-19T18:20:00.295980+0000","last_unstale":"2023-12-19T18:20:03.305711+0000","last_undegraded":"2023-12-19T18:20:03.305711+0000","last_fullsized":"2023-12-19T18:20:03.305711+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:44:18.425235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305106+0000","last_change":"2023-12-19T18:19:59.788329+0000","last_active":"2023-12-19T18:20:03.305106+0000","last_peered":"2023-12-19T18:20:03.305106+0000","last_clean":"2023-12-19T18:20:03.305106+0000","last_became_active":"2023-12-19T18:19:59.788017+0000","last_became_peered":"2023-12-19T18:19:59.788017+0000","last_unstale":"2023-12-19T18:20:03.305106+0000","last_undegraded":"2023-12-19T18:20:03.305106+0000","last_fullsized":"2023-12-19T18:20:03.305106+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:08:06.261382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275224+0000","last_change":"2023-12-19T18:20:00.287261+0000","last_active":"2023-12-19T18:20:01.275224+0000","last_peered":"2023-12-19T18:20:01.275224+0000","last_clean":"2023-12-19T18:20:01.275224+0000","last_became_active":"2023-12-19T18:20:00.285021+0000","last_became_peered":"2023-12-19T18:20:00.285021+0000","last_unstale":"2023-12-19T18:20:01.275224+0000","last_undegraded":"2023-12-19T18:20:01.275224+0000","last_fullsized":"2023-12-19T18:20:01.275224+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:23:26.019352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305612+0000","last_change":"2023-12-19T18:19:59.789130+0000","last_active":"2023-12-19T18:20:03.305612+0000","last_peered":"2023-12-19T18:20:03.305612+0000","last_clean":"2023-12-19T18:20:03.305612+0000","last_became_active":"2023-12-19T18:19:59.787049+0000","last_became_peered":"2023-12-19T18:19:59.787049+0000","last_unstale":"2023-12-19T18:20:03.305612+0000","last_undegraded":"2023-12-19T18:20:03.305612+0000","last_fullsized":"2023-12-19T18:20:03.305612+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:24:05.605488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285590+0000","last_change":"2023-12-19T18:20:00.305082+0000","last_active":"2023-12-19T18:20:01.285590+0000","last_peered":"2023-12-19T18:20:01.285590+0000","last_clean":"2023-12-19T18:20:01.285590+0000","last_became_active":"2023-12-19T18:20:00.304847+0000","last_became_peered":"2023-12-19T18:20:00.304847+0000","last_unstale":"2023-12-19T18:20:01.285590+0000","last_undegraded":"2023-12-19T18:20:01.285590+0000","last_fullsized":"2023-12-19T18:20:01.285590+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:08:04.212994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"95'8","reported_seq":19,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305837+0000","last_change":"2023-12-19T18:19:59.792885+0000","last_active":"2023-12-19T18:20:03.305837+0000","last_peered":"2023-12-19T18:20:03.305837+0000","last_clean":"2023-12-19T18:20:03.305837+0000","last_became_active":"2023-12-19T18:19:59.792428+0000","last_became_peered":"2023-12-19T18:19:59.792428+0000","last_unstale":"2023-12-19T18:20:03.305837+0000","last_undegraded":"2023-12-19T18:20:03.305837+0000","last_fullsized":"2023-12-19T18:20:03.305837+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:08:06.603565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779348+0000","last_change":"2023-12-19T18:20:00.292415+0000","last_active":"2023-12-19T18:20:01.779348+0000","last_peered":"2023-12-19T18:20:01.779348+0000","last_clean":"2023-12-19T18:20:01.779348+0000","last_became_active":"2023-12-19T18:20:00.291562+0000","last_became_peered":"2023-12-19T18:20:00.291562+0000","last_unstale":"2023-12-19T18:20:01.779348+0000","last_undegraded":"2023-12-19T18:20:01.779348+0000","last_fullsized":"2023-12-19T18:20:01.779348+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:05.897496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"95'1","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.302780+0000","last_change":"2023-12-19T18:19:59.787044+0000","last_active":"2023-12-19T18:20:00.302780+0000","last_peered":"2023-12-19T18:20:00.302780+0000","last_clean":"2023-12-19T18:20:00.302780+0000","last_became_active":"2023-12-19T18:19:59.786807+0000","last_became_peered":"2023-12-19T18:19:59.786807+0000","last_unstale":"2023-12-19T18:20:00.302780+0000","last_undegraded":"2023-12-19T18:20:00.302780+0000","last_fullsized":"2023-12-19T18:20:00.302780+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:41:21.837669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.778601+0000","last_change":"2023-12-19T18:20:00.285691+0000","last_active":"2023-12-19T18:20:01.778601+0000","last_peered":"2023-12-19T18:20:01.778601+0000","last_clean":"2023-12-19T18:20:01.778601+0000","last_became_active":"2023-12-19T18:20:00.285469+0000","last_became_peered":"2023-12-19T18:20:00.285469+0000","last_unstale":"2023-12-19T18:20:01.778601+0000","last_undegraded":"2023-12-19T18:20:01.778601+0000","last_fullsized":"2023-12-19T18:20:01.778601+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:35.301273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779632+0000","last_change":"2023-12-19T18:19:59.784766+0000","last_active":"2023-12-19T18:20:01.779632+0000","last_peered":"2023-12-19T18:20:01.779632+0000","last_clean":"2023-12-19T18:20:01.779632+0000","last_became_active":"2023-12-19T18:19:59.784413+0000","last_became_peered":"2023-12-19T18:19:59.784413+0000","last_unstale":"2023-12-19T18:20:01.779632+0000","last_undegraded":"2023-12-19T18:20:01.779632+0000","last_fullsized":"2023-12-19T18:20:01.779632+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:01:20.510530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285826+0000","last_change":"2023-12-19T18:20:00.302920+0000","last_active":"2023-12-19T18:20:01.285826+0000","last_peered":"2023-12-19T18:20:01.285826+0000","last_clean":"2023-12-19T18:20:01.285826+0000","last_became_active":"2023-12-19T18:20:00.301404+0000","last_became_peered":"2023-12-19T18:20:00.301404+0000","last_unstale":"2023-12-19T18:20:01.285826+0000","last_undegraded":"2023-12-19T18:20:01.285826+0000","last_fullsized":"2023-12-19T18:20:01.285826+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:47:13.415066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"95'1","reported_seq":12,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779703+0000","last_change":"2023-12-19T18:19:59.788293+0000","last_active":"2023-12-19T18:20:01.779703+0000","last_peered":"2023-12-19T18:20:01.779703+0000","last_clean":"2023-12-19T18:20:01.779703+0000","last_became_active":"2023-12-19T18:19:59.786626+0000","last_became_peered":"2023-12-19T18:19:59.786626+0000","last_unstale":"2023-12-19T18:20:01.779703+0000","last_undegraded":"2023-12-19T18:20:01.779703+0000","last_fullsized":"2023-12-19T18:20:01.779703+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:36.681763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.285517+0000","last_change":"2023-12-19T18:20:00.303424+0000","last_active":"2023-12-19T18:20:01.285517+0000","last_peered":"2023-12-19T18:20:01.285517+0000","last_clean":"2023-12-19T18:20:01.285517+0000","last_became_active":"2023-12-19T18:20:00.303161+0000","last_became_peered":"2023-12-19T18:20:00.303161+0000","last_unstale":"2023-12-19T18:20:01.285517+0000","last_undegraded":"2023-12-19T18:20:01.285517+0000","last_fullsized":"2023-12-19T18:20:01.285517+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:53:16.724716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.274123+0000","last_change":"2023-12-19T18:19:59.787423+0000","last_active":"2023-12-19T18:20:00.274123+0000","last_peered":"2023-12-19T18:20:00.274123+0000","last_clean":"2023-12-19T18:20:00.274123+0000","last_became_active":"2023-12-19T18:19:59.787094+0000","last_became_peered":"2023-12-19T18:19:59.787094+0000","last_unstale":"2023-12-19T18:20:00.274123+0000","last_undegraded":"2023-12-19T18:20:00.274123+0000","last_fullsized":"2023-12-19T18:20:00.274123+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:37:05.940502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.780210+0000","last_change":"2023-12-19T18:20:00.303532+0000","last_active":"2023-12-19T18:20:01.780210+0000","last_peered":"2023-12-19T18:20:01.780210+0000","last_clean":"2023-12-19T18:20:01.780210+0000","last_became_active":"2023-12-19T18:20:00.302923+0000","last_became_peered":"2023-12-19T18:20:00.302923+0000","last_unstale":"2023-12-19T18:20:01.780210+0000","last_undegraded":"2023-12-19T18:20:01.780210+0000","last_fullsized":"2023-12-19T18:20:01.780210+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:06.820663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.273330+0000","last_change":"2023-12-19T18:19:59.774115+0000","last_active":"2023-12-19T18:20:00.273330+0000","last_peered":"2023-12-19T18:20:00.273330+0000","last_clean":"2023-12-19T18:20:00.273330+0000","last_became_active":"2023-12-19T18:19:59.773598+0000","last_became_peered":"2023-12-19T18:19:59.773598+0000","last_unstale":"2023-12-19T18:20:00.273330+0000","last_undegraded":"2023-12-19T18:20:00.273330+0000","last_fullsized":"2023-12-19T18:20:00.273330+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:20.266447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779656+0000","last_change":"2023-12-19T18:20:00.296833+0000","last_active":"2023-12-19T18:20:01.779656+0000","last_peered":"2023-12-19T18:20:01.779656+0000","last_clean":"2023-12-19T18:20:01.779656+0000","last_became_active":"2023-12-19T18:20:00.295532+0000","last_became_peered":"2023-12-19T18:20:00.295532+0000","last_unstale":"2023-12-19T18:20:01.779656+0000","last_undegraded":"2023-12-19T18:20:01.779656+0000","last_fullsized":"2023-12-19T18:20:01.779656+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:37:06.953401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779745+0000","last_change":"2023-12-19T18:19:59.772374+0000","last_active":"2023-12-19T18:20:01.779745+0000","last_peered":"2023-12-19T18:20:01.779745+0000","last_clean":"2023-12-19T18:20:01.779745+0000","last_became_active":"2023-12-19T18:19:59.771722+0000","last_became_peered":"2023-12-19T18:19:59.771722+0000","last_unstale":"2023-12-19T18:20:01.779745+0000","last_undegraded":"2023-12-19T18:20:01.779745+0000","last_fullsized":"2023-12-19T18:20:01.779745+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:58:53.382720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.278208+0000","last_change":"2023-12-19T18:20:00.300239+0000","last_active":"2023-12-19T18:20:01.278208+0000","last_peered":"2023-12-19T18:20:01.278208+0000","last_clean":"2023-12-19T18:20:01.278208+0000","last_became_active":"2023-12-19T18:20:00.299701+0000","last_became_peered":"2023-12-19T18:20:00.299701+0000","last_unstale":"2023-12-19T18:20:01.278208+0000","last_undegraded":"2023-12-19T18:20:01.278208+0000","last_fullsized":"2023-12-19T18:20:01.278208+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:33:41.013088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275385+0000","last_change":"2023-12-19T18:19:59.782636+0000","last_active":"2023-12-19T18:20:01.275385+0000","last_peered":"2023-12-19T18:20:01.275385+0000","last_clean":"2023-12-19T18:20:01.275385+0000","last_became_active":"2023-12-19T18:19:59.782260+0000","last_became_peered":"2023-12-19T18:19:59.782260+0000","last_unstale":"2023-12-19T18:20:01.275385+0000","last_undegraded":"2023-12-19T18:20:01.275385+0000","last_fullsized":"2023-12-19T18:20:01.275385+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:43:48.093779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.276419+0000","last_change":"2023-12-19T18:20:00.298382+0000","last_active":"2023-12-19T18:20:01.276419+0000","last_peered":"2023-12-19T18:20:01.276419+0000","last_clean":"2023-12-19T18:20:01.276419+0000","last_became_active":"2023-12-19T18:20:00.297934+0000","last_became_peered":"2023-12-19T18:20:00.297934+0000","last_unstale":"2023-12-19T18:20:01.276419+0000","last_undegraded":"2023-12-19T18:20:01.276419+0000","last_fullsized":"2023-12-19T18:20:01.276419+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:24:02.318855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"95'2","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.303304+0000","last_change":"2023-12-19T18:19:59.784490+0000","last_active":"2023-12-19T18:20:00.303304+0000","last_peered":"2023-12-19T18:20:00.303304+0000","last_clean":"2023-12-19T18:20:00.303304+0000","last_became_active":"2023-12-19T18:19:59.784048+0000","last_became_peered":"2023-12-19T18:19:59.784048+0000","last_unstale":"2023-12-19T18:20:00.303304+0000","last_undegraded":"2023-12-19T18:20:00.303304+0000","last_fullsized":"2023-12-19T18:20:00.303304+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:14:14.905336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305241+0000","last_change":"2023-12-19T18:20:00.298044+0000","last_active":"2023-12-19T18:20:03.305241+0000","last_peered":"2023-12-19T18:20:03.305241+0000","last_clean":"2023-12-19T18:20:03.305241+0000","last_became_active":"2023-12-19T18:20:00.297834+0000","last_became_peered":"2023-12-19T18:20:00.297834+0000","last_unstale":"2023-12-19T18:20:03.305241+0000","last_undegraded":"2023-12-19T18:20:03.305241+0000","last_fullsized":"2023-12-19T18:20:03.305241+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:32:23.800025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.287591+0000","last_change":"2023-12-19T18:19:59.789915+0000","last_active":"2023-12-19T18:20:00.287591+0000","last_peered":"2023-12-19T18:20:00.287591+0000","last_clean":"2023-12-19T18:20:00.287591+0000","last_became_active":"2023-12-19T18:19:59.789682+0000","last_became_peered":"2023-12-19T18:19:59.789682+0000","last_unstale":"2023-12-19T18:20:00.287591+0000","last_undegraded":"2023-12-19T18:20:00.287591+0000","last_fullsized":"2023-12-19T18:20:00.287591+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:01:37.262744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779503+0000","last_change":"2023-12-19T18:20:00.287734+0000","last_active":"2023-12-19T18:20:01.779503+0000","last_peered":"2023-12-19T18:20:01.779503+0000","last_clean":"2023-12-19T18:20:01.779503+0000","last_became_active":"2023-12-19T18:20:00.287429+0000","last_became_peered":"2023-12-19T18:20:00.287429+0000","last_unstale":"2023-12-19T18:20:01.779503+0000","last_undegraded":"2023-12-19T18:20:01.779503+0000","last_fullsized":"2023-12-19T18:20:01.779503+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:13:07.454418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.279544+0000","last_change":"2023-12-19T18:19:59.784892+0000","last_active":"2023-12-19T18:20:00.279544+0000","last_peered":"2023-12-19T18:20:00.279544+0000","last_clean":"2023-12-19T18:20:00.279544+0000","last_became_active":"2023-12-19T18:19:59.784406+0000","last_became_peered":"2023-12-19T18:19:59.784406+0000","last_unstale":"2023-12-19T18:20:00.279544+0000","last_undegraded":"2023-12-19T18:20:00.279544+0000","last_fullsized":"2023-12-19T18:20:00.279544+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:02:17.934136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.305201+0000","last_change":"2023-12-19T18:20:00.281819+0000","last_active":"2023-12-19T18:20:03.305201+0000","last_peered":"2023-12-19T18:20:03.305201+0000","last_clean":"2023-12-19T18:20:03.305201+0000","last_became_active":"2023-12-19T18:20:00.281499+0000","last_became_peered":"2023-12-19T18:20:00.281499+0000","last_unstale":"2023-12-19T18:20:03.305201+0000","last_undegraded":"2023-12-19T18:20:03.305201+0000","last_fullsized":"2023-12-19T18:20:03.305201+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:14:14.407377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.281603+0000","last_change":"2023-12-19T18:19:59.785508+0000","last_active":"2023-12-19T18:20:00.281603+0000","last_peered":"2023-12-19T18:20:00.281603+0000","last_clean":"2023-12-19T18:20:00.281603+0000","last_became_active":"2023-12-19T18:19:59.785024+0000","last_became_peered":"2023-12-19T18:19:59.785024+0000","last_unstale":"2023-12-19T18:20:00.281603+0000","last_undegraded":"2023-12-19T18:20:00.281603+0000","last_fullsized":"2023-12-19T18:20:00.281603+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:08:03.195907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.275012+0000","last_change":"2023-12-19T18:20:00.301214+0000","last_active":"2023-12-19T18:20:01.275012+0000","last_peered":"2023-12-19T18:20:01.275012+0000","last_clean":"2023-12-19T18:20:01.275012+0000","last_became_active":"2023-12-19T18:20:00.300460+0000","last_became_peered":"2023-12-19T18:20:00.300460+0000","last_unstale":"2023-12-19T18:20:01.275012+0000","last_undegraded":"2023-12-19T18:20:01.275012+0000","last_fullsized":"2023-12-19T18:20:01.275012+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:41:22.850569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.278166+0000","last_change":"2023-12-19T18:19:59.774874+0000","last_active":"2023-12-19T18:20:00.278166+0000","last_peered":"2023-12-19T18:20:00.278166+0000","last_clean":"2023-12-19T18:20:00.278166+0000","last_became_active":"2023-12-19T18:19:59.774277+0000","last_became_peered":"2023-12-19T18:19:59.774277+0000","last_unstale":"2023-12-19T18:20:00.278166+0000","last_undegraded":"2023-12-19T18:20:00.278166+0000","last_fullsized":"2023-12-19T18:20:00.278166+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:16:23.123225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.307689+0000","last_change":"2023-12-19T18:20:00.302991+0000","last_active":"2023-12-19T18:20:03.307689+0000","last_peered":"2023-12-19T18:20:03.307689+0000","last_clean":"2023-12-19T18:20:03.307689+0000","last_became_active":"2023-12-19T18:20:00.297999+0000","last_became_peered":"2023-12-19T18:20:00.297999+0000","last_unstale":"2023-12-19T18:20:03.307689+0000","last_undegraded":"2023-12-19T18:20:03.307689+0000","last_fullsized":"2023-12-19T18:20:03.307689+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:16.354181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.277003+0000","last_change":"2023-12-19T18:19:59.790846+0000","last_active":"2023-12-19T18:20:01.277003+0000","last_peered":"2023-12-19T18:20:01.277003+0000","last_clean":"2023-12-19T18:20:01.277003+0000","last_became_active":"2023-12-19T18:19:59.790553+0000","last_became_peered":"2023-12-19T18:19:59.790553+0000","last_unstale":"2023-12-19T18:20:01.277003+0000","last_undegraded":"2023-12-19T18:20:01.277003+0000","last_fullsized":"2023-12-19T18:20:01.277003+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:55:21.799985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779542+0000","last_change":"2023-12-19T18:20:00.289838+0000","last_active":"2023-12-19T18:20:01.779542+0000","last_peered":"2023-12-19T18:20:01.779542+0000","last_clean":"2023-12-19T18:20:01.779542+0000","last_became_active":"2023-12-19T18:20:00.289236+0000","last_became_peered":"2023-12-19T18:20:00.289236+0000","last_unstale":"2023-12-19T18:20:01.779542+0000","last_undegraded":"2023-12-19T18:20:01.779542+0000","last_fullsized":"2023-12-19T18:20:01.779542+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:35:21.279347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.779236+0000","last_change":"2023-12-19T18:19:59.774095+0000","last_active":"2023-12-19T18:20:01.779236+0000","last_peered":"2023-12-19T18:20:01.779236+0000","last_clean":"2023-12-19T18:20:01.779236+0000","last_became_active":"2023-12-19T18:19:59.773691+0000","last_became_peered":"2023-12-19T18:19:59.773691+0000","last_unstale":"2023-12-19T18:20:01.779236+0000","last_undegraded":"2023-12-19T18:20:01.779236+0000","last_fullsized":"2023-12-19T18:20:01.779236+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:10:45.860622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.778967+0000","last_change":"2023-12-19T18:20:00.290178+0000","last_active":"2023-12-19T18:20:01.778967+0000","last_peered":"2023-12-19T18:20:01.778967+0000","last_clean":"2023-12-19T18:20:01.778967+0000","last_became_active":"2023-12-19T18:20:00.289413+0000","last_became_peered":"2023-12-19T18:20:00.289413+0000","last_unstale":"2023-12-19T18:20:01.778967+0000","last_undegraded":"2023-12-19T18:20:01.778967+0000","last_fullsized":"2023-12-19T18:20:01.778967+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:09:22.340036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.273430+0000","last_change":"2023-12-19T18:19:59.789939+0000","last_active":"2023-12-19T18:20:00.273430+0000","last_peered":"2023-12-19T18:20:00.273430+0000","last_clean":"2023-12-19T18:20:00.273430+0000","last_became_active":"2023-12-19T18:19:59.783654+0000","last_became_peered":"2023-12-19T18:19:59.783654+0000","last_unstale":"2023-12-19T18:20:00.273430+0000","last_undegraded":"2023-12-19T18:20:00.273430+0000","last_fullsized":"2023-12-19T18:20:00.273430+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:08:03.195907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:01.274655+0000","last_change":"2023-12-19T18:20:00.295026+0000","last_active":"2023-12-19T18:20:01.274655+0000","last_peered":"2023-12-19T18:20:01.274655+0000","last_clean":"2023-12-19T18:20:01.274655+0000","last_became_active":"2023-12-19T18:20:00.294785+0000","last_became_peered":"2023-12-19T18:20:00.294785+0000","last_unstale":"2023-12-19T18:20:01.274655+0000","last_undegraded":"2023-12-19T18:20:01.274655+0000","last_fullsized":"2023-12-19T18:20:01.274655+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:59.752183+0000","last_clean_scrub_stamp":"2023-12-19T18:19:59.752183+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:04:38.902775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"95'2","reported_seq":13,"reported_epoch":96,"state":"active+clean","last_fresh":"2023-12-19T18:20:03.306096+0000","last_change":"2023-12-19T18:19:59.786747+0000","last_active":"2023-12-19T18:20:03.306096+0000","last_peered":"2023-12-19T18:20:03.306096+0000","last_clean":"2023-12-19T18:20:03.306096+0000","last_became_active":"2023-12-19T18:19:59.786012+0000","last_became_peered":"2023-12-19T18:19:59.786012+0000","last_unstale":"2023-12-19T18:20:03.306096+0000","last_undegraded":"2023-12-19T18:20:03.306096+0000","last_fullsized":"2023-12-19T18:20:03.306096+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:19:58.739284+0000","last_clean_scrub_stamp":"2023-12-19T18:19:58.739284+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:55:44.169276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":232,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-12-19T18:20:00.278467+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:20:00.278467+0000","last_peered":"2023-12-19T18:20:00.278467+0000","last_clean":"2023-12-19T18:20:00.278467+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:20:00.278467+0000","last_undegraded":"2023-12-19T18:20:00.278467+0000","last_fullsized":"2023-12-19T18:20:00.278467+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":18,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477148,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5820000000000001,"5min":1.2250000000000001,"15min":1.1459999999999999},"min":{"1min":1.02,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":5.6879999999999997,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":1.419},{"interface":"front","average":{"1min":1.373,"5min":1.236,"15min":1.1719999999999999},"min":{"1min":0.91000000000000003,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.7330000000000001,"5min":1.79,"15min":1.79},"last":1.3879999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.5549999999999999,"5min":1.361,"15min":1.232},"min":{"1min":0.86299999999999999,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":5.9029999999999996,"5min":10.926,"15min":10.926},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2749999999999999,"5min":1.3169999999999999,"15min":1.2330000000000001},"min":{"1min":0.93600000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.6180000000000001,"5min":10.904999999999999,"15min":10.904999999999999},"last":1.3}]},{"osd":2,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3580000000000001,"5min":1.141,"15min":1.101},"min":{"1min":0.89400000000000002,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":3.1960000000000002,"5min":3.1960000000000002,"15min":3.1960000000000002},"last":0.84999999999999998},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.254,"15min":1.1859999999999999},"min":{"1min":1.0009999999999999,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":6.008,"5min":6.008,"15min":6.008},"last":1.2629999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.222,"15min":1.1359999999999999},"min":{"1min":0.95499999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":3.0379999999999998,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":1.0880000000000001},{"interface":"front","average":{"1min":1.7150000000000001,"5min":1.391,"15min":1.274},"min":{"1min":1.1240000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":5.8120000000000003,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":1.373}]},{"osd":4,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.198,"15min":1.0940000000000001},"min":{"1min":0.82099999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":4.0110000000000001,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":1.0309999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.2849999999999999,"15min":1.232},"min":{"1min":0.88600000000000001,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":3.3180000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.2290000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.75,"5min":1.4339999999999999,"15min":1.335},"min":{"1min":0.81100000000000005,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":7.6429999999999998,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":1.119},{"interface":"front","average":{"1min":1.71,"5min":1.407,"15min":1.325},"min":{"1min":0.996,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":6.0839999999999996,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":1.1930000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:19:55 2023","interfaces":[{"interface":"back","average":{"1min":1.6120000000000001,"5min":1.5209999999999999,"15min":1.4470000000000001},"min":{"1min":0.85899999999999999,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":3.1309999999999998,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.5049999999999999},{"interface":"front","average":{"1min":1.625,"5min":1.4510000000000001,"15min":1.3600000000000001},"min":{"1min":1.046,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":4.1479999999999997,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":1.1759999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574914,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":510,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194801,"statfs":{"total":200000000,"available":199477114,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.046,"5min":1.0529999999999999,"15min":0.98099999999999998},"min":{"1min":0.68899999999999995,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.546,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.26},{"interface":"front","average":{"1min":1.0980000000000001,"5min":1.1319999999999999,"15min":1.05},"min":{"1min":0.64300000000000002,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.619,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":1.0089999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1339999999999999,"15min":1.1240000000000001},"min":{"1min":0.71399999999999997,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.488,"5min":4.5540000000000003,"15min":4.5540000000000003},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.175,"15min":1.1559999999999999},"min":{"1min":0.76300000000000001,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.885,"5min":1.885,"15min":1.885},"last":1.746}]},{"osd":2,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2,"15min":1.1259999999999999},"min":{"1min":0.82199999999999995,"5min":0.626,"15min":0.626},"max":{"1min":1.7330000000000001,"5min":1.821,"15min":1.821},"last":1.6240000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.214,"15min":1.236},"min":{"1min":0.89200000000000002,"5min":0.79900000000000004,"15min":0.79900000000000004},"max":{"1min":1.494,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":1.3819999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.2330000000000001,"15min":1.1779999999999999},"min":{"1min":0.98299999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.52,"5min":3.8820000000000001,"15min":3.8820000000000001},"last":1.5109999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.1879999999999999,"15min":1.1599999999999999},"min":{"1min":0.83799999999999997,"5min":0.77500000000000002,"15min":0.77500000000000002},"max":{"1min":1.5649999999999999,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":1.218}]},{"osd":4,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.27,"15min":1.238},"min":{"1min":0.81699999999999995,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.9670000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.8520000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.3939999999999999,"15min":1.3260000000000001},"min":{"1min":0.70799999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.7490000000000001,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.7949999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.238,"15min":1.1659999999999999},"min":{"1min":0.66600000000000004,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.5309999999999999,"5min":4.6680000000000001,"15min":4.6680000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.371,"15min":1.325},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":2.0489999999999999,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":2.0459999999999998}]},{"osd":7,"last update":"Tue Dec 19 18:19:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.397,"15min":1.327},"min":{"1min":0.94399999999999995,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.702,"5min":3.9670000000000001,"15min":3.9670000000000001},"last":1.877},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.407,"15min":1.337},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.9339999999999999,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":1.5720000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1085,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194226,"statfs":{"total":200000000,"available":198888254,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2050000000000001,"15min":1.1559999999999999},"min":{"1min":0.752,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.774,"5min":1.863,"15min":1.863},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.2470000000000001,"15min":1.1699999999999999},"min":{"1min":1.0549999999999999,"5min":0.75,"15min":0.75},"max":{"1min":1.6850000000000001,"5min":1.879,"15min":1.879},"last":1.3380000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.1419999999999999,"15min":1.0369999999999999},"min":{"1min":0.94999999999999996,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.246},{"interface":"front","average":{"1min":1.218,"5min":1.1240000000000001,"15min":1.046},"min":{"1min":0.71299999999999997,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.6359999999999999,"5min":1.843,"15min":1.843},"last":1.5229999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.204,"5min":1.145,"15min":1.073},"min":{"1min":0.78400000000000003,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.4970000000000001,"5min":1.794,"15min":1.794},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.2070000000000001,"15min":1.131},"min":{"1min":0.86199999999999999,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.732,"5min":2.036,"15min":2.036},"last":0.84899999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.1499999999999999,"15min":1.073},"min":{"1min":0.86399999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.522,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.45},{"interface":"front","average":{"1min":1.417,"5min":1.3100000000000001,"15min":1.212},"min":{"1min":1.1020000000000001,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":2.0590000000000002,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.597}]},{"osd":4,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.286,"15min":1.1579999999999999},"min":{"1min":0.98399999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.802,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.149,"5min":1.1539999999999999,"15min":1.111},"min":{"1min":0.84199999999999997,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.49,"5min":1.901,"15min":1.901},"last":1.387}]},{"osd":6,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4259999999999999,"15min":1.3200000000000001},"min":{"1min":1.1850000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.71,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.544},{"interface":"front","average":{"1min":1.3400000000000001,"5min":1.28,"15min":1.228},"min":{"1min":1.0129999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.653,"5min":1.976,"15min":1.976},"last":1.2889999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3680000000000001,"5min":1.2789999999999999,"15min":1.2170000000000001},"min":{"1min":0.88600000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.881,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.423},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4390000000000001,"15min":1.3999999999999999},"min":{"1min":1.177,"5min":0.623,"15min":0.623},"max":{"1min":2.117,"5min":2.117,"15min":2.117},"last":1.498}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":509,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194802,"statfs":{"total":200000000,"available":199478018,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":0.996,"15min":0.997},"min":{"1min":0.69899999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.389,"5min":1.579,"15min":1.579},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.0680000000000001,"5min":1.056,"15min":1.034},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4390000000000001,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.76900000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.1639999999999999,"15min":1.1499999999999999},"min":{"1min":0.77900000000000003,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.5589999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.383,"5min":1.2350000000000001,"15min":1.171},"min":{"1min":1.008,"5min":0.624,"15min":0.624},"max":{"1min":1.6220000000000001,"5min":1.696,"15min":1.696},"last":1.0980000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.1180000000000001,"15min":1.1200000000000001},"min":{"1min":0.874,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.627,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.018},{"interface":"front","average":{"1min":1.149,"5min":1.113,"15min":1.0880000000000001},"min":{"1min":0.86099999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.4870000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3009999999999999,"5min":1.2809999999999999,"15min":1.252},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.6639999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.258},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.3120000000000001,"15min":1.294},"min":{"1min":0.91000000000000003,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.718,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":1.4239999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.3460000000000001,"5min":1.2529999999999999,"15min":1.1850000000000001},"min":{"1min":1.0549999999999999,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6779999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.585},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.391,"15min":1.337},"min":{"1min":0.89600000000000002,"5min":0.504,"15min":0.504},"max":{"1min":1.7709999999999999,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.4670000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3149999999999999,"15min":1.302},"min":{"1min":1.0940000000000001,"5min":0.432,"15min":0.432},"max":{"1min":1.6619999999999999,"5min":1.853,"15min":1.853},"last":1.5409999999999999},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4219999999999999,"15min":1.3779999999999999},"min":{"1min":1.218,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.7310000000000001,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.659}]},{"osd":7,"last update":"Tue Dec 19 18:19:54 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.3779999999999999,"15min":1.335},"min":{"1min":1.1220000000000001,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.8640000000000001,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.389},{"interface":"front","average":{"1min":1.4570000000000001,"5min":1.415,"15min":1.401},"min":{"1min":1.1910000000000001,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.645,"5min":1.9330000000000001,"15min":1.9330000000000001},"last":1.621}]}]},{"osd":3,"up_from":13,"seq":55834574915,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":503,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194809,"statfs":{"total":200000000,"available":199484886,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80500000000000005,"5min":0.90000000000000002,"15min":0.96199999999999997},"min":{"1min":0.42099999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.26,"5min":3.4350000000000001,"15min":3.4350000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.95399999999999996,"15min":0.92800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.3919999999999999,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.39500000000000002}]},{"osd":1,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":0.93600000000000005,"5min":0.99099999999999999,"15min":1.0549999999999999},"min":{"1min":0.44400000000000001,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.623,"5min":4.5129999999999999,"15min":4.5129999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":1.014,"5min":1.0569999999999999,"15min":1.089},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.4570000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.57399999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.2909999999999999,"15min":1.2390000000000001},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.8560000000000001,"5min":13.833,"15min":13.833},"last":0.48199999999999998},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.371,"15min":1.258},"min":{"1min":0.65900000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4950000000000001,"5min":16.748999999999999,"15min":16.748999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.256,"15min":1.1990000000000001},"min":{"1min":0.77400000000000002,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.641,"5min":2.1190000000000002,"15min":2.1190000000000002},"last":0.88100000000000001},{"interface":"front","average":{"1min":1.3420000000000001,"5min":1.375,"15min":1.403},"min":{"1min":0.89100000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.8100000000000001,"5min":4.0209999999999999,"15min":4.0209999999999999},"last":1.024}]},{"osd":5,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.381,"15min":1.431},"min":{"1min":0.82399999999999995,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.5580000000000001,"5min":3.3370000000000002,"15min":3.3370000000000002},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.2689999999999999,"5min":1.29,"15min":1.252},"min":{"1min":0.80600000000000005,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.7010000000000001,"5min":2.4009999999999998,"15min":2.4009999999999998},"last":0.84699999999999998}]},{"osd":6,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3560000000000001,"5min":1.4159999999999999,"15min":1.4730000000000001},"min":{"1min":0.81799999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.9319999999999999,"5min":3.9399999999999999,"15min":3.9399999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.3999999999999999,"5min":1.4590000000000001,"15min":1.4970000000000001},"min":{"1min":0.92300000000000004,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":1.8240000000000001,"5min":3.2229999999999999,"15min":3.2229999999999999},"last":1.115}]},{"osd":7,"last update":"Tue Dec 19 18:19:49 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.385,"15min":1.357},"min":{"1min":0.89500000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":1.9370000000000001,"5min":2.48,"15min":2.48},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.5249999999999999,"15min":1.5449999999999999},"min":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":1.879,"5min":3.9849999999999999,"15min":3.9849999999999999},"last":1.1539999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":509,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194802,"statfs":{"total":200000000,"available":199478130,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":1.0629999999999999,"15min":1.0169999999999999},"min":{"1min":0.35299999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.5369999999999999,"5min":9.2550000000000008,"15min":9.2550000000000008},"last":1.105},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.004,"15min":1.0369999999999999},"min":{"1min":0.41999999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.3360000000000001,"5min":4.7850000000000001,"15min":4.7850000000000001},"last":1.8799999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":0.95699999999999996,"5min":0.99299999999999999,"15min":1.202},"min":{"1min":0.55300000000000005,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5369999999999999,"5min":8.2720000000000002,"15min":8.2720000000000002},"last":0.92300000000000004},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.081,"15min":1.1679999999999999},"min":{"1min":0.504,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.6220000000000001,"5min":5.0620000000000003,"15min":5.0620000000000003},"last":1.028}]},{"osd":3,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.1819999999999999,"15min":1.3939999999999999},"min":{"1min":0.626,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":1.6240000000000001,"5min":8.3109999999999999,"15min":8.3109999999999999},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.157,"5min":1.1879999999999999,"15min":1.3620000000000001},"min":{"1min":0.65100000000000002,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.4730000000000001,"5min":8.0690000000000008,"15min":8.0690000000000008},"last":1.3260000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.405,"15min":1.472},"min":{"1min":0.83399999999999996,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.829,"5min":5.3929999999999998,"15min":5.3929999999999998},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4259999999999999,"15min":1.5580000000000001},"min":{"1min":1.036,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":1.75,"5min":7.3570000000000002,"15min":7.3570000000000002},"last":2.3460000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.397,"15min":1.571},"min":{"1min":0.96599999999999997,"5min":0.78700000000000003,"15min":0.78700000000000003},"max":{"1min":1.5529999999999999,"5min":8.1839999999999993,"15min":8.1839999999999993},"last":1.5580000000000001},{"interface":"front","average":{"1min":1.425,"5min":1.3899999999999999,"15min":1.4119999999999999},"min":{"1min":1.1339999999999999,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.754,"5min":5.165,"15min":5.165},"last":1.5109999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.3959999999999999,"15min":1.4490000000000001},"min":{"1min":1.004,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.6419999999999999,"5min":5.1059999999999999,"15min":5.1059999999999999},"last":2.2429999999999999},{"interface":"front","average":{"1min":1.448,"5min":1.456,"15min":1.575},"min":{"1min":0.93899999999999995,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.758,"5min":8.1370000000000005,"15min":8.1370000000000005},"last":1.4550000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:19:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4450000000000001,"5min":1.484,"15min":1.55},"min":{"1min":0.91100000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.758,"5min":5.1449999999999996,"15min":5.1449999999999996},"last":2.4430000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.5169999999999999,"15min":1.6679999999999999},"min":{"1min":0.88500000000000001,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":1.8759999999999999,"5min":8.2300000000000004,"15min":8.2300000000000004},"last":1.3979999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1079,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194233,"statfs":{"total":200000000,"available":198894644,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.82299999999999995,"15min":0.83999999999999997},"min":{"1min":0.38,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.298,"5min":1.359,"15min":1.359},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.93200000000000005,"5min":0.91200000000000003,"15min":0.95199999999999996},"min":{"1min":0.48899999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.5760000000000001,"5min":1.6719999999999999,"15min":1.6719999999999999},"last":1.74}]},{"osd":2,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":1.0169999999999999,"15min":1.0489999999999999},"min":{"1min":0.42299999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.514,"5min":2.5739999999999998,"15min":2.5739999999999998},"last":0.99299999999999999},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.95099999999999996,"15min":0.90900000000000003},"min":{"1min":0.46700000000000003,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.456,"5min":1.845,"15min":1.845},"last":0.90400000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3300000000000001,"5min":1.2,"15min":1.1299999999999999},"min":{"1min":0.60799999999999998,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.8300000000000001,"5min":2.1019999999999999,"15min":2.1019999999999999},"last":1.637},{"interface":"front","average":{"1min":1.242,"5min":1.167,"15min":1.179},"min":{"1min":0.59199999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.7889999999999999,"5min":2.3959999999999999,"15min":2.3959999999999999},"last":1.8049999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.2789999999999999,"15min":1.2769999999999999},"min":{"1min":0.72599999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.6419999999999999,"5min":2.0419999999999998,"15min":2.0419999999999998},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.359,"5min":1.3109999999999999,"15min":1.278},"min":{"1min":0.79600000000000004,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":1.6479999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.409}]},{"osd":5,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.387,"5min":1.323,"15min":1.2729999999999999},"min":{"1min":0.94099999999999995,"5min":0.88,"15min":0.88},"max":{"1min":1.7689999999999999,"5min":2.04,"15min":2.04},"last":1.288},{"interface":"front","average":{"1min":1.383,"5min":1.4079999999999999,"15min":1.3899999999999999},"min":{"1min":0.95399999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.661,"5min":2.2349999999999999,"15min":2.2349999999999999},"last":1.462}]},{"osd":6,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.397,"5min":1.373,"15min":1.3640000000000001},"min":{"1min":0.875,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.109,"5min":2.1699999999999999,"15min":2.1699999999999999},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.456,"15min":1.4299999999999999},"min":{"1min":0.92200000000000004,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.9039999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":1.601}]},{"osd":7,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.484,"15min":1.3620000000000001},"min":{"1min":0.96199999999999997,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.831,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":1.556},{"interface":"front","average":{"1min":1.5649999999999999,"5min":1.583,"15min":1.4830000000000001},"min":{"1min":1.012,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.2360000000000002,"5min":9.4109999999999996,"15min":9.4109999999999996},"last":1.5009999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":509,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194803,"statfs":{"total":200000000,"available":199478642,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.93799999999999994,"15min":0.88200000000000001},"min":{"1min":0.69599999999999995,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.5720000000000001,"5min":1.887,"15min":1.887},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.083,"15min":1.0229999999999999},"min":{"1min":0.86399999999999999,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.639,"5min":2.4180000000000001,"15min":2.4180000000000001},"last":1.169}]},{"osd":2,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":0.98899999999999999,"15min":0.94499999999999995},"min":{"1min":0.753,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.417,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1120000000000001,"5min":0.97499999999999998,"15min":0.92500000000000004},"min":{"1min":0.624,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.746,"5min":2.7810000000000001,"15min":2.7810000000000001},"last":1.3879999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1180000000000001,"5min":1.0289999999999999,"15min":1.0049999999999999},"min":{"1min":0.65300000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.6950000000000001,"5min":1.8959999999999999,"15min":1.8959999999999999},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.4219999999999999,"5min":1.2669999999999999,"15min":1.2190000000000001},"min":{"1min":0.80900000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.7450000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.696}]},{"osd":4,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.2669999999999999,"15min":1.1830000000000001},"min":{"1min":1.0169999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6970000000000001,"5min":1.903,"15min":1.903},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.2370000000000001,"5min":1.27,"15min":1.2769999999999999},"min":{"1min":1.048,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.4570000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.5660000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.48,"15min":1.393},"min":{"1min":1.014,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.73,"5min":16.298999999999999,"15min":16.298999999999999},"last":1.7809999999999999},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.323,"15min":1.2649999999999999},"min":{"1min":1.2330000000000001,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.7010000000000001,"5min":1.95,"15min":1.95},"last":1.5029999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5449999999999999,"5min":1.423,"15min":1.383},"min":{"1min":1.1639999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":1.8200000000000001,"5min":2.5910000000000002,"15min":2.5910000000000002},"last":1.417},{"interface":"front","average":{"1min":1.464,"5min":1.4259999999999999,"15min":1.349},"min":{"1min":1.1120000000000001,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.8220000000000001,"5min":2.46,"15min":2.46},"last":1.9179999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:19:53 2023","interfaces":[{"interface":"back","average":{"1min":1.5209999999999999,"5min":1.466,"15min":1.3779999999999999},"min":{"1min":1.115,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.9370000000000001,"5min":2.1560000000000001,"15min":2.1560000000000001},"last":2.024},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.3979999999999999,"15min":1.417},"min":{"1min":1.2150000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":1.6990000000000001,"5min":3.1429999999999998,"15min":3.1429999999999998},"last":1.877}]}]}],"pool_statfs":[]}} 2023-12-19T18:20:05.833 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:20:05.833 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:20:06.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.119+0000 7f6636370640 1 Processor -- start 2023-12-19T18:20:06.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.120+0000 7f6636370640 1 -- start start 2023-12-19T18:20:06.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.121+0000 7f6636370640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66300b0660 0x7f663006f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:06.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.121+0000 7f6636370640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f66300ce450 0x7f66300ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:06.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.121+0000 7f6636370640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6630161970 con 0x7f66300c8fc0 2023-12-19T18:20:06.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.121+0000 7f6636370640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6630070c50 con 0x7f66300ce450 2023-12-19T18:20:06.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.121+0000 7f6636370640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f663006fbb0 con 0x7f66300b0660 2023-12-19T18:20:06.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.122+0000 7f6635b6f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f66300ce450 0x7f66300ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:06.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.122+0000 7f6635b6f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f66300ce450 0x7f66300ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51070/0 (socket says 172.21.15.27:51070) 2023-12-19T18:20:06.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.122+0000 7f6635b6f640 1 -- 172.21.15.27:0/3532819146 learned_addr learned my addr 172.21.15.27:0/3532819146 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:06.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.122+0000 7f6634b6d640 1 --1- 172.21.15.27:0/3532819146 >> v1:172.21.15.27:6789/0 conn(0x7f66300c8fc0 0x7f66300cdd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36416/0 (socket says 172.21.15.27:36416) 2023-12-19T18:20:06.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.122+0000 7f663536e640 1 --2- 172.21.15.27:0/3532819146 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66300b0660 0x7f663006f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:06.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.122+0000 7f6635b6f640 1 -- 172.21.15.27:0/3532819146 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66300b0660 msgr2=0x7f663006f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:06.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.122+0000 7f6635b6f640 1 --2- 172.21.15.27:0/3532819146 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66300b0660 0x7f663006f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.122+0000 7f6635b6f640 1 -- 172.21.15.27:0/3532819146 >> v1:172.21.15.27:6789/0 conn(0x7f66300c8fc0 legacy=0x7f66300cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:06.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.122+0000 7f6635b6f640 1 -- 172.21.15.27:0/3532819146 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f663006fe90 con 0x7f66300ce450 2023-12-19T18:20:06.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.124+0000 7f6635b6f640 1 --2- 172.21.15.27:0/3532819146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66300ce450 0x7f66300ad510 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6628009e50 tx=0x7f662802bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f123dd750eb8851 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:06.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.124+0000 7f6627fff640 1 -- 172.21.15.27:0/3532819146 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f662802cb00 con 0x7f66300ce450 2023-12-19T18:20:06.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.124+0000 7f6627fff640 1 -- 172.21.15.27:0/3532819146 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f662802cc60 con 0x7f66300ce450 2023-12-19T18:20:06.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.124+0000 7f6627fff640 1 -- 172.21.15.27:0/3532819146 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66280343f0 con 0x7f66300ce450 2023-12-19T18:20:06.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.124+0000 7f6636370640 1 -- 172.21.15.27:0/3532819146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66300ce450 msgr2=0x7f66300ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:06.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.124+0000 7f6636370640 1 --2- 172.21.15.27:0/3532819146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66300ce450 0x7f66300ad510 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f6628009e50 tx=0x7f662802bf60 comp rx=0 tx=0).stop 2023-12-19T18:20:06.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.125+0000 7f6636370640 1 -- 172.21.15.27:0/3532819146 shutdown_connections 2023-12-19T18:20:06.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.125+0000 7f6636370640 1 --2- 172.21.15.27:0/3532819146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66300ce450 0x7f66300ad510 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.125+0000 7f6636370640 1 --2- 172.21.15.27:0/3532819146 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66300b0660 0x7f663006f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.125+0000 7f6636370640 1 -- 172.21.15.27:0/3532819146 >> 172.21.15.27:0/3532819146 conn(0x7f66300cb3f0 msgr2=0x7f66300cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:06.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.125+0000 7f6636370640 1 -- 172.21.15.27:0/3532819146 shutdown_connections 2023-12-19T18:20:06.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.125+0000 7f6636370640 1 -- 172.21.15.27:0/3532819146 wait complete. 2023-12-19T18:20:06.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.127+0000 7f6636370640 1 Processor -- start 2023-12-19T18:20:06.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.127+0000 7f6636370640 1 -- start start 2023-12-19T18:20:06.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.128+0000 7f6636370640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66300b0660 0x7f66301fab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:06.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.128+0000 7f6636370640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66300c8fc0 0x7f6630152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.128+0000 7f6636370640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66300ce450 0x7f66300cd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.128+0000 7f6636370640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66301f6b40 con 0x7f66300c8fc0 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.128+0000 7f6636370640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66301fb0b0 con 0x7f66300b0660 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.128+0000 7f6636370640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66300c36c0 con 0x7f66300ce450 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.128+0000 7f6634b6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66300c8fc0 0x7f6630152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.128+0000 7f6634b6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66300c8fc0 0x7f6630152640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51084/0 (socket says 172.21.15.27:51084) 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.128+0000 7f6634b6d640 1 -- 172.21.15.27:0/3872549034 learned_addr learned my addr 172.21.15.27:0/3872549034 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.128+0000 7f663536e640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66300b0660 0x7f66301fab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.128+0000 7f6635b6f640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66300ce450 0x7f66300cd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.129+0000 7f6634b6d640 1 -- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66300ce450 msgr2=0x7f66300cd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.129+0000 7f6634b6d640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66300ce450 0x7f66300cd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.129+0000 7f6634b6d640 1 -- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66300b0660 msgr2=0x7f66301fab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.129+0000 7f6634b6d640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66300b0660 0x7f66301fab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.129+0000 7f6634b6d640 1 -- 172.21.15.27:0/3872549034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6628009b30 con 0x7f66300c8fc0 2023-12-19T18:20:06.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.129+0000 7f6634b6d640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66300c8fc0 0x7f6630152640 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f662000c500 tx=0x7f662000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:06.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.129+0000 7f6625ffb640 1 -- 172.21.15.27:0/3872549034 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6620007500 con 0x7f66300c8fc0 2023-12-19T18:20:06.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.129+0000 7f6625ffb640 1 -- 172.21.15.27:0/3872549034 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6620007660 con 0x7f66300c8fc0 2023-12-19T18:20:06.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.129+0000 7f6625ffb640 1 -- 172.21.15.27:0/3872549034 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66200049e0 con 0x7f66300c8fc0 2023-12-19T18:20:06.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.129+0000 7f6636370640 1 -- 172.21.15.27:0/3872549034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66300c39a0 con 0x7f66300c8fc0 2023-12-19T18:20:06.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.129+0000 7f6636370640 1 -- 172.21.15.27:0/3872549034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66300c6f60 con 0x7f66300c8fc0 2023-12-19T18:20:06.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.131+0000 7f6636370640 1 -- 172.21.15.27:0/3872549034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f663004e840 con 0x7f66300c8fc0 2023-12-19T18:20:06.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.132+0000 7f6625ffb640 1 -- 172.21.15.27:0/3872549034 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6620004b40 con 0x7f66300c8fc0 2023-12-19T18:20:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.133+0000 7f6625ffb640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f661406a430 0x7f661406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.133+0000 7f6625ffb640 1 -- 172.21.15.27:0/3872549034 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6620087a40 con 0x7f66300c8fc0 2023-12-19T18:20:06.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.133+0000 7f663536e640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f661406a430 0x7f661406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:06.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.134+0000 7f663536e640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f661406a430 0x7f661406c8f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f662c006fd0 tx=0x7f662c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:06.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.137+0000 7f6625ffb640 1 -- 172.21.15.27:0/3872549034 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f66200530d0 con 0x7f66300c8fc0 2023-12-19T18:20:06.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.409+0000 7f6636370640 1 -- 172.21.15.27:0/3872549034 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f66300712a0 con 0x7f66300c8fc0 2023-12-19T18:20:06.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.416+0000 7f6625ffb640 1 -- 172.21.15.27:0/3872549034 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6620018020 con 0x7f66300c8fc0 2023-12-19T18:20:06.417 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2023-12-19T18:20:06.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.419+0000 7f6636370640 1 -- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f661406a430 msgr2=0x7f661406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:06.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.419+0000 7f6636370640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f661406a430 0x7f661406c8f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f662c006fd0 tx=0x7f662c008040 comp rx=0 tx=0).stop 2023-12-19T18:20:06.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.419+0000 7f6636370640 1 -- 172.21.15.27:0/3872549034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66300c8fc0 msgr2=0x7f6630152640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:06.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.419+0000 7f6636370640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66300c8fc0 0x7f6630152640 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f662000c500 tx=0x7f662000c9c0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.419+0000 7f6636370640 1 -- 172.21.15.27:0/3872549034 shutdown_connections 2023-12-19T18:20:06.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.419+0000 7f6636370640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f661406a430 0x7f661406c8f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.419+0000 7f6636370640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66300ce450 0x7f66300cd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.419+0000 7f6636370640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66300c8fc0 0x7f6630152640 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.419+0000 7f6636370640 1 --2- 172.21.15.27:0/3872549034 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66300b0660 0x7f66301fab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.419+0000 7f6636370640 1 -- 172.21.15.27:0/3872549034 >> 172.21.15.27:0/3872549034 conn(0x7f66300cb3f0 msgr2=0x7f66300cb7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:06.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.419+0000 7f6636370640 1 -- 172.21.15.27:0/3872549034 shutdown_connections 2023-12-19T18:20:06.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.419+0000 7f6636370640 1 -- 172.21.15.27:0/3872549034 wait complete. 2023-12-19T18:20:06.441 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:20:06.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.664+0000 7f6f6f59e640 1 Processor -- start 2023-12-19T18:20:06.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.664+0000 7f6f6f59e640 1 -- start start 2023-12-19T18:20:06.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.665+0000 7f6f6f59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6f701553a0 0x7f6f700c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.665+0000 7f6f6f59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f700afeb0 0x7f6f7015e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.665+0000 7f6f6f59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f700b0330 con 0x7f6f7015ec60 2023-12-19T18:20:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.665+0000 7f6f6f59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f700cbcf0 con 0x7f6f701553a0 2023-12-19T18:20:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.665+0000 7f6f6f59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7015f0d0 con 0x7f6f700afeb0 2023-12-19T18:20:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.666+0000 7f6f6e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6f701553a0 0x7f6f700c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.666+0000 7f6f6e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6f701553a0 0x7f6f700c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51098/0 (socket says 172.21.15.27:51098) 2023-12-19T18:20:06.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.666+0000 7f6f6e59c640 1 -- 172.21.15.27:0/3842949142 learned_addr learned my addr 172.21.15.27:0/3842949142 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:06.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.666+0000 7f6f6dd9b640 1 --2- 172.21.15.27:0/3842949142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f700afeb0 0x7f6f7015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:06.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.666+0000 7f6f6e59c640 1 -- 172.21.15.27:0/3842949142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f700afeb0 msgr2=0x7f6f7015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:06.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.666+0000 7f6f6e59c640 1 --2- 172.21.15.27:0/3842949142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f700afeb0 0x7f6f7015e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.666+0000 7f6f6e59c640 1 -- 172.21.15.27:0/3842949142 >> v1:172.21.15.192:6789/0 conn(0x7f6f7015ec60 legacy=0x7f6f7015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:06.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.666+0000 7f6f6e59c640 1 -- 172.21.15.27:0/3842949142 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f700cb350 con 0x7f6f701553a0 2023-12-19T18:20:06.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.666+0000 7f6f6dd9b640 1 --2- 172.21.15.27:0/3842949142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f700afeb0 0x7f6f7015e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.667+0000 7f6f6e59c640 1 --2- 172.21.15.27:0/3842949142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f701553a0 0x7f6f700c50a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6f600098a0 tx=0x7f6f6002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b10840ee65b8f67 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.668+0000 7f6f6d59a640 1 -- 172.21.15.27:0/3842949142 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f600317c0 con 0x7f6f701553a0 2023-12-19T18:20:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.668+0000 7f6f6d59a640 1 -- 172.21.15.27:0/3842949142 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f60031920 con 0x7f6f701553a0 2023-12-19T18:20:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.668+0000 7f6f6d59a640 1 -- 172.21.15.27:0/3842949142 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f60031cb0 con 0x7f6f701553a0 2023-12-19T18:20:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.668+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3842949142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f701553a0 msgr2=0x7f6f700c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.668+0000 7f6f6f59e640 1 --2- 172.21.15.27:0/3842949142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f701553a0 0x7f6f700c50a0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f6f600098a0 tx=0x7f6f6002b860 comp rx=0 tx=0).stop 2023-12-19T18:20:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.668+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3842949142 shutdown_connections 2023-12-19T18:20:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.668+0000 7f6f6f59e640 1 --2- 172.21.15.27:0/3842949142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f700afeb0 0x7f6f7015e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.668+0000 7f6f6f59e640 1 --2- 172.21.15.27:0/3842949142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f701553a0 0x7f6f700c50a0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.668+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3842949142 >> 172.21.15.27:0/3842949142 conn(0x7f6f70069c70 msgr2=0x7f6f7006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:06.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.669+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3842949142 shutdown_connections 2023-12-19T18:20:06.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.669+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3842949142 wait complete. 2023-12-19T18:20:06.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.670+0000 7f6f6f59e640 1 Processor -- start 2023-12-19T18:20:06.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.671+0000 7f6f6f59e640 1 -- start start 2023-12-19T18:20:06.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f700afeb0 0x7f6f70146b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:06.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6f59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f701553a0 0x7f6f700b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:06.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6f59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f7015ec60 0x7f6f70145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6f59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f700cdaf0 con 0x7f6f700afeb0 2023-12-19T18:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6f59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f700ccf20 con 0x7f6f7015ec60 2023-12-19T18:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6f59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f700cd1c0 con 0x7f6f701553a0 2023-12-19T18:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f700afeb0 0x7f6f70146b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f700afeb0 0x7f6f70146b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51106/0 (socket says 172.21.15.27:51106) 2023-12-19T18:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6e59c640 1 -- 172.21.15.27:0/3286404496 learned_addr learned my addr 172.21.15.27:0/3286404496 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6e59c640 1 -- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f701553a0 msgr2=0x7f6f700b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6e59c640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f701553a0 0x7f6f700b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6e59c640 1 -- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f7015ec60 msgr2=0x7f6f70145fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6ed9d640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f7015ec60 0x7f6f70145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6e59c640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f7015ec60 0x7f6f70145fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6e59c640 1 -- 172.21.15.27:0/3286404496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f600094d0 con 0x7f6f700afeb0 2023-12-19T18:20:06.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.672+0000 7f6f6ed9d640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f7015ec60 0x7f6f70145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.673+0000 7f6f6e59c640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f700afeb0 0x7f6f70146b50 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6f600098a0 tx=0x7f6f600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.673+0000 7f6f66ffd640 1 -- 172.21.15.27:0/3286404496 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f600329a0 con 0x7f6f700afeb0 2023-12-19T18:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.673+0000 7f6f66ffd640 1 -- 172.21.15.27:0/3286404496 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f60032b20 con 0x7f6f700afeb0 2023-12-19T18:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.673+0000 7f6f66ffd640 1 -- 172.21.15.27:0/3286404496 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f600453f0 con 0x7f6f700afeb0 2023-12-19T18:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.673+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3286404496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f700cd4a0 con 0x7f6f700afeb0 2023-12-19T18:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.673+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3286404496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f700b4220 con 0x7f6f700afeb0 2023-12-19T18:20:06.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.675+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3286404496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f700b4410 con 0x7f6f700afeb0 2023-12-19T18:20:06.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.675+0000 7f6f66ffd640 1 -- 172.21.15.27:0/3286404496 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6f6004d3f0 con 0x7f6f700afeb0 2023-12-19T18:20:06.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.676+0000 7f6f66ffd640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5006a430 0x7f6f5006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:06.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.676+0000 7f6f66ffd640 1 -- 172.21.15.27:0/3286404496 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f600aa7d0 con 0x7f6f700afeb0 2023-12-19T18:20:06.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.677+0000 7f6f6dd9b640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5006a430 0x7f6f5006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:06.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.678+0000 7f6f6dd9b640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5006a430 0x7f6f5006c8f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6f7015aaf0 tx=0x7f6f58008330 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.681+0000 7f6f66ffd640 1 -- 172.21.15.27:0/3286404496 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6f60075e50 con 0x7f6f700afeb0 2023-12-19T18:20:06.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.913+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3286404496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6f7004e840 con 0x7f6f700afeb0 2023-12-19T18:20:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.915+0000 7f6f66ffd640 1 -- 172.21.15.27:0/3286404496 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f6f60079470 con 0x7f6f700afeb0 2023-12-19T18:20:06.915 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:06.915 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2860412135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2860412135},{"type":"v1","addr":"172.21.15.192:6837","nonce":2860412135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/797098667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":797098667},{"type":"v1","addr":"172.21.15.192:6836","nonce":797098667}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":6507,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/4009181263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4009181263},{"type":"v1","addr":"172.21.15.27:6833","nonce":4009181263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:20:00.264651+0000","modified":"2023-12-19T18:20:03.796309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6510},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6510":{"gid":6510,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/202939736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":202939736},{"type":"v1","addr":"172.21.15.27:6835","nonce":202939736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-19T18:20:06.915 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2023-12-19T18:20:06.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.917+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5006a430 msgr2=0x7f6f5006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:06.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.917+0000 7f6f6f59e640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5006a430 0x7f6f5006c8f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f6f7015aaf0 tx=0x7f6f58008330 comp rx=0 tx=0).stop 2023-12-19T18:20:06.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.917+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3286404496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f700afeb0 msgr2=0x7f6f70146b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:06.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.917+0000 7f6f6f59e640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f700afeb0 0x7f6f70146b50 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f6f600098a0 tx=0x7f6f600046c0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.917+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3286404496 shutdown_connections 2023-12-19T18:20:06.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.917+0000 7f6f6f59e640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5006a430 0x7f6f5006c8f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.917+0000 7f6f6f59e640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f7015ec60 0x7f6f70145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.917+0000 7f6f6f59e640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f701553a0 0x7f6f700b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.917+0000 7f6f6f59e640 1 --2- 172.21.15.27:0/3286404496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f700afeb0 0x7f6f70146b50 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.917+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3286404496 >> 172.21.15.27:0/3286404496 conn(0x7f6f70069c70 msgr2=0x7f6f700c4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.918+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3286404496 shutdown_connections 2023-12-19T18:20:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:06.918+0000 7f6f6f59e640 1 -- 172.21.15.27:0/3286404496 wait complete. 2023-12-19T18:20:07.339 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 38, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:20:00.264651+0000', 'modified': '2023-12-19T18:20:03.796309+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6510}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6510': {'gid': 6510, 'name': 'a', 'rank': 0, 'incarnation': 34, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/202939736', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6834', 'nonce': 202939736}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 202939736}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:20:07.339 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:20:07.339 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-19T18:20:07.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.582+0000 7f0afaee5640 1 Processor -- start 2023-12-19T18:20:07.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.583+0000 7f0afaee5640 1 -- start start 2023-12-19T18:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.584+0000 7f0afaee5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0af415b3a0 0x7f0af406c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.584+0000 7f0afaee5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af40cad90 0x7f0af406b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.584+0000 7f0afaee5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0af40cabc0 con 0x7f0af415afc0 2023-12-19T18:20:07.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.584+0000 7f0afaee5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0af40c2ba0 con 0x7f0af415b3a0 2023-12-19T18:20:07.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.584+0000 7f0afaee5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af40ca040 con 0x7f0af40cad90 2023-12-19T18:20:07.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.585+0000 7f0af9ee3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0af415afc0 0x7f0af40c2490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36418/0 (socket says 172.21.15.27:36418) 2023-12-19T18:20:07.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.585+0000 7f0af9ee3640 1 -- 172.21.15.27:0/2334068398 learned_addr learned my addr 172.21.15.27:0/2334068398 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:07.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.585+0000 7f0af96e2640 1 --2- 172.21.15.27:0/2334068398 >> v2:172.21.15.192:3300/0 conn(0x7f0af415b3a0 0x7f0af406c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.585+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142846338 0 0) 0x7f0af40cabc0 con 0x7f0af415afc0 2023-12-19T18:20:07.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.585+0000 7f0afa6e4640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af40cad90 0x7f0af406b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.585+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ae0003520 con 0x7f0af415afc0 2023-12-19T18:20:07.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3394641026 0 0) 0x7f0ae0003520 con 0x7f0af415afc0 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af40cad90 msgr2=0x7f0af406b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af40cad90 0x7f0af406b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 >> v2:172.21.15.192:3300/0 conn(0x7f0af415b3a0 msgr2=0x7f0af406c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 --2- 172.21.15.27:0/2334068398 >> v2:172.21.15.192:3300/0 conn(0x7f0af415b3a0 0x7f0af406c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af40ca320 con 0x7f0af415afc0 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f0af0001670 con 0x7f0af415afc0 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 >> v1:172.21.15.27:6789/0 conn(0x7f0af415afc0 legacy=0x7f0af40c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae0003970 0x7f0ae0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae00047c0 0x7f0ae0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ae0005180 0x7f0ae000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af40c2ba0 con 0x7f0ae0005180 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af40ca040 con 0x7f0ae0003970 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ae000bff0 con 0x7f0ae00047c0 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0afa6e4640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af40cad90 0x7f0af406b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:20:07.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.586+0000 7f0afa6e4640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ae0005180 0x7f0ae000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.587+0000 7f0af9ee3640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae0003970 0x7f0ae0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.587+0000 7f0af96e2640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae00047c0 0x7f0ae0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.587+0000 7f0afa6e4640 1 -- 172.21.15.27:0/2334068398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae00047c0 msgr2=0x7f0ae0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:07.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.587+0000 7f0afa6e4640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae00047c0 0x7f0ae0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.587+0000 7f0afa6e4640 1 -- 172.21.15.27:0/2334068398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae0003970 msgr2=0x7f0ae0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:07.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.587+0000 7f0afa6e4640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae0003970 0x7f0ae0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.587+0000 7f0afa6e4640 1 -- 172.21.15.27:0/2334068398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ae0003520 con 0x7f0ae0005180 2023-12-19T18:20:07.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.587+0000 7f0af96e2640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0ae00047c0 0x7f0ae0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:07.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.588+0000 7f0afa6e4640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ae0005180 0x7f0ae000b980 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009f00 tx=0x7f0ae00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6deae816fae126a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:07.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.588+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec02cc70 con 0x7f0ae0005180 2023-12-19T18:20:07.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.588+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec02cdd0 con 0x7f0ae0005180 2023-12-19T18:20:07.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.588+0000 7f0af8ee1640 1 -- 172.21.15.27:0/2334068398 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec003bb0 con 0x7f0ae0005180 2023-12-19T18:20:07.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.589+0000 7f0afaee5640 1 -- 172.21.15.27:0/2334068398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ae0005180 msgr2=0x7f0ae000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.589+0000 7f0afaee5640 1 --2- 172.21.15.27:0/2334068398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ae0005180 0x7f0ae000b980 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f0aec009f00 tx=0x7f0ae00057f0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.589+0000 7f0afa6e4640 1 -- 172.21.15.27:0/2334068398 reap_dead start 2023-12-19T18:20:07.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.589+0000 7f0afaee5640 1 -- 172.21.15.27:0/2334068398 shutdown_connections 2023-12-19T18:20:07.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.589+0000 7f0afaee5640 1 -- 172.21.15.27:0/2334068398 >> 172.21.15.27:0/2334068398 conn(0x7f0af40cb450 msgr2=0x7f0af40cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:07.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.589+0000 7f0afaee5640 1 -- 172.21.15.27:0/2334068398 shutdown_connections 2023-12-19T18:20:07.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.589+0000 7f0afaee5640 1 -- 172.21.15.27:0/2334068398 wait complete. 2023-12-19T18:20:07.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.591+0000 7f0afaee5640 1 Processor -- start 2023-12-19T18:20:07.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.591+0000 7f0afaee5640 1 -- start start 2023-12-19T18:20:07.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.592+0000 7f0afaee5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae0005180 0x7f0af414ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.592+0000 7f0afaee5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af414f7e0 0x7f0af4070140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.592+0000 7f0afaee5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af40c44a0 0x7f0af40c4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.592+0000 7f0afaee5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af40c4e60 con 0x7f0af40c44a0 2023-12-19T18:20:07.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.592+0000 7f0afaee5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0af40b15c0 con 0x7f0ae0005180 2023-12-19T18:20:07.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.592+0000 7f0afaee5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0af40b2140 con 0x7f0af414f7e0 2023-12-19T18:20:07.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.592+0000 7f0afa6e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af40c44a0 0x7f0af40c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.592+0000 7f0afa6e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af40c44a0 0x7f0af40c4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51126/0 (socket says 172.21.15.27:51126) 2023-12-19T18:20:07.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.592+0000 7f0afa6e4640 1 -- 172.21.15.27:0/2829439712 learned_addr learned my addr 172.21.15.27:0/2829439712 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:07.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.593+0000 7f0af9ee3640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae0005180 0x7f0af414ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.593+0000 7f0af96e2640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af414f7e0 0x7f0af4070140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.593+0000 7f0afa6e4640 1 -- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af414f7e0 msgr2=0x7f0af4070140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.593+0000 7f0afa6e4640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af414f7e0 0x7f0af4070140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.593+0000 7f0afa6e4640 1 -- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae0005180 msgr2=0x7f0af414ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.593+0000 7f0afa6e4640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae0005180 0x7f0af414ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.593+0000 7f0afa6e4640 1 -- 172.21.15.27:0/2829439712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0af40b1810 con 0x7f0af40c44a0 2023-12-19T18:20:07.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.593+0000 7f0afa6e4640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af40c44a0 0x7f0af40c4920 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0aec007330 tx=0x7f0aec004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:07.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.594+0000 7f0aea7fc640 1 -- 172.21.15.27:0/2829439712 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec03c070 con 0x7f0af40c44a0 2023-12-19T18:20:07.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.594+0000 7f0aea7fc640 1 -- 172.21.15.27:0/2829439712 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec003e30 con 0x7f0af40c44a0 2023-12-19T18:20:07.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.594+0000 7f0aea7fc640 1 -- 172.21.15.27:0/2829439712 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec004760 con 0x7f0af40c44a0 2023-12-19T18:20:07.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.594+0000 7f0afaee5640 1 -- 172.21.15.27:0/2829439712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0af40b1af0 con 0x7f0af40c44a0 2023-12-19T18:20:07.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.594+0000 7f0afaee5640 1 -- 172.21.15.27:0/2829439712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af414f440 con 0x7f0af40c44a0 2023-12-19T18:20:07.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.596+0000 7f0aea7fc640 1 -- 172.21.15.27:0/2829439712 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0aec041460 con 0x7f0af40c44a0 2023-12-19T18:20:07.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.597+0000 7f0aea7fc640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0ad806a460 0x7f0ad806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.597+0000 7f0aea7fc640 1 -- 172.21.15.27:0/2829439712 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0aec0a9d80 con 0x7f0af40c44a0 2023-12-19T18:20:07.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.598+0000 7f0af9ee3640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0ad806a460 0x7f0ad806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.598+0000 7f0afed96740 1 Processor -- start 2023-12-19T18:20:07.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.598+0000 7f0afed96740 1 -- start start 2023-12-19T18:20:07.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.599+0000 7f0af9ee3640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0ad806a460 0x7f0ad806c920 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0af0001050 tx=0x7f0af0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:07.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.599+0000 7f0afed96740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb28a0d0 0x5565eb28a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.599+0000 7f0afed96740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb25a050 0x5565eb287340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.599+0000 7f0afed96740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb287880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.599+0000 7f0afed96740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565eb235650 con 0x5565eb25dc50 2023-12-19T18:20:07.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.599+0000 7f0afed96740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565eb1ca000 con 0x5565eb25a050 2023-12-19T18:20:07.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.599+0000 7f0afed96740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565eb1c7790 con 0x5565eb28a0d0 2023-12-19T18:20:07.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.599+0000 7f0afa6e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb287880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.599+0000 7f0afa6e4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb287880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51130/0 (socket says 172.21.15.27:51130) 2023-12-19T18:20:07.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.599+0000 7f0afa6e4640 1 -- 172.21.15.27:0/278948494 learned_addr learned my addr 172.21.15.27:0/278948494 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:07.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.599+0000 7f0af9ee3640 1 --2- 172.21.15.27:0/278948494 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb28a0d0 0x5565eb28a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.599+0000 7f0af96e2640 1 --2- 172.21.15.27:0/278948494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb25a050 0x5565eb287340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.600+0000 7f0afa6e4640 1 -- 172.21.15.27:0/278948494 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb28a0d0 msgr2=0x5565eb28a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:07.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.600+0000 7f0afa6e4640 1 --2- 172.21.15.27:0/278948494 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb28a0d0 0x5565eb28a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.600+0000 7f0afa6e4640 1 -- 172.21.15.27:0/278948494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb25a050 msgr2=0x5565eb287340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:07.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.600+0000 7f0afa6e4640 1 --2- 172.21.15.27:0/278948494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb25a050 0x5565eb287340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.600+0000 7f0afa6e4640 1 -- 172.21.15.27:0/278948494 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565eb18a710 con 0x5565eb25dc50 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.600+0000 7f0af96e2640 1 --2- 172.21.15.27:0/278948494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb25a050 0x5565eb287340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.600+0000 7f0afa6e4640 1 --2- 172.21.15.27:0/278948494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb287880 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0755b0 tx=0x7f0aec078f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562ad603d3e74824 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.600+0000 7f0ad77fe640 1 -- 172.21.15.27:0/278948494 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec079050 con 0x5565eb25dc50 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.600+0000 7f0ad77fe640 1 -- 172.21.15.27:0/278948494 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec080f70 con 0x5565eb25dc50 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.600+0000 7f0ad77fe640 1 -- 172.21.15.27:0/278948494 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aec009b50 con 0x5565eb25dc50 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.601+0000 7f0afed96740 1 -- 172.21.15.27:0/278948494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 msgr2=0x5565eb287880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.601+0000 7f0afed96740 1 --2- 172.21.15.27:0/278948494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb287880 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f0aec0755b0 tx=0x7f0aec078f30 comp rx=0 tx=0).stop 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.601+0000 7f0afed96740 1 -- 172.21.15.27:0/278948494 shutdown_connections 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.601+0000 7f0afed96740 1 --2- 172.21.15.27:0/278948494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb287880 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.601+0000 7f0afed96740 1 --2- 172.21.15.27:0/278948494 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb25a050 0x5565eb287340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.601+0000 7f0afed96740 1 --2- 172.21.15.27:0/278948494 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb28a0d0 0x5565eb28a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.601+0000 7f0afed96740 1 -- 172.21.15.27:0/278948494 >> 172.21.15.27:0/278948494 conn(0x5565eb25c360 msgr2=0x5565eb2a4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:07.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.601+0000 7f0afed96740 1 -- 172.21.15.27:0/278948494 shutdown_connections 2023-12-19T18:20:07.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.601+0000 7f0afed96740 1 -- 172.21.15.27:0/278948494 wait complete. 2023-12-19T18:20:07.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.602+0000 7f0afed96740 1 -- start start 2023-12-19T18:20:07.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.602+0000 7f0afed96740 1 Processor -- start 2023-12-19T18:20:07.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.603+0000 7f0afed96740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb28bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.603+0000 7f0afed96740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb28a0d0 0x5565eb28c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.603+0000 7f0afed96740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb2741a0 0x5565eb2a7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.603+0000 7f0afed96740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565eb1c3080 con 0x5565eb25dc50 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.603+0000 7f0afed96740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565eb164850 con 0x5565eb28a0d0 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.603+0000 7f0afed96740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565eb1644f0 con 0x5565eb2741a0 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.603+0000 7f0afed96740 10 client.0 _ll_register_callbacks cb 0x5565eb1bc9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.603+0000 7f0af9ee3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb28bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.603+0000 7f0af9ee3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb28bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51134/0 (socket says 172.21.15.27:51134) 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.603+0000 7f0af9ee3640 1 -- 172.21.15.27:0/228094521 learned_addr learned my addr 172.21.15.27:0/228094521 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.604+0000 7f0af96e2640 1 --2- 172.21.15.27:0/228094521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb28a0d0 0x5565eb28c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.604+0000 7f0af9ee3640 1 -- 172.21.15.27:0/228094521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb2741a0 msgr2=0x5565eb2a7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.604+0000 7f0af9ee3640 1 --2- 172.21.15.27:0/228094521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb2741a0 0x5565eb2a7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.604+0000 7f0af9ee3640 1 -- 172.21.15.27:0/228094521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb28a0d0 msgr2=0x5565eb28c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.604+0000 7f0af9ee3640 1 --2- 172.21.15.27:0/228094521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb28a0d0 0x5565eb28c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.604+0000 7f0af9ee3640 1 -- 172.21.15.27:0/228094521 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0af414f440 con 0x5565eb25dc50 2023-12-19T18:20:07.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.604+0000 7f0af96e2640 1 --2- 172.21.15.27:0/228094521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb28a0d0 0x5565eb28c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.604+0000 7f0af9ee3640 1 --2- 172.21.15.27:0/228094521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb28bda0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0af0014430 tx=0x7f0af00019a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=285e5eee75dc2926 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.604+0000 7f0ad4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.604+0000 7f0ad4ff9640 1 -- 172.21.15.27:0/228094521 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af0013770 con 0x5565eb25dc50 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.604+0000 7f0ad4ff9640 1 -- 172.21.15.27:0/228094521 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0af001c040 con 0x5565eb25dc50 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.604+0000 7f0afed96740 1 -- 172.21.15.27:0/228094521 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565eb1ed5d0 con 0x5565eb25dc50 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.605+0000 7f0ad4ff9640 1 -- 172.21.15.27:0/228094521 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0af0014e70 con 0x5565eb25dc50 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.605+0000 7f0afed96740 10 client.6651 fetch_fsmap learned FSMap version 38 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.605+0000 7f0afed96740 1 -- 172.21.15.27:0/228094521 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x5565eb1c1700 con 0x5565eb25dc50 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.605+0000 7f0ad4ff9640 1 -- 172.21.15.27:0/228094521 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5565eb1ed5d0 con 0x5565eb25dc50 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.605+0000 7f0ad4ff9640 1 -- 172.21.15.27:0/228094521 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 38) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0af002b050 con 0x5565eb25dc50 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.605+0000 7f0afed96740 10 client.6651 fetch_fsmap finished waiting for FSMap version 38 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.605+0000 7f0afed96740 10 client.6651 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.605+0000 7f0afed96740 20 client.6651 populate_metadata read hostname 'smithi027' 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.605+0000 7f0afed96740 1 --2- 172.21.15.27:0/228094521 >> [v2:172.21.15.27:6834/202939736,v1:172.21.15.27:6835/202939736] conn(0x5565eb2b84f0 0x5565eb275670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.605+0000 7f0afed96740 4 client.6651 mds_command: new command op to 6510 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:20:07.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.605+0000 7f0afed96740 1 -- 172.21.15.27:0/228094521 --> [v2:172.21.15.27:6834/202939736,v1:172.21.15.27:6835/202939736] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565eb1b58d0 con 0x5565eb2b84f0 2023-12-19T18:20:07.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.605+0000 7f0af96e2640 1 --2- 172.21.15.27:0/228094521 >> [v2:172.21.15.27:6834/202939736,v1:172.21.15.27:6835/202939736] conn(0x5565eb2b84f0 0x5565eb275670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.606+0000 7f0af96e2640 1 --2- 172.21.15.27:0/228094521 >> [v2:172.21.15.27:6834/202939736,v1:172.21.15.27:6835/202939736] conn(0x5565eb2b84f0 0x5565eb275670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b1827d3f15dbf32 server_cookie=b99870b127be3360 in_seq=0 out_seq=0 2023-12-19T18:20:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.606+0000 7f0ad4ff9640 10 client.6651 ms_handle_connect on v2:172.21.15.27:6834/202939736 2023-12-19T18:20:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0ad4ff9640 1 -- 172.21.15.27:0/228094521 <== mds.0 v2:172.21.15.27:6834/202939736 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5565eb1b58d0 con 0x5565eb2b84f0 2023-12-19T18:20:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0ad4ff9640 10 client.6651 handle_command_reply: tid=0 2023-12-19T18:20:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0afed96740 1 client.6651 shutdown 2023-12-19T18:20:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0afed96740 1 -- 172.21.15.27:0/228094521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 msgr2=0x5565eb28bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0afed96740 1 --2- 172.21.15.27:0/228094521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb28bda0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f0af0014430 tx=0x7f0af00019a0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0afed96740 1 -- 172.21.15.27:0/228094521 shutdown_connections 2023-12-19T18:20:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0afed96740 1 --2- 172.21.15.27:0/228094521 >> [v2:172.21.15.27:6834/202939736,v1:172.21.15.27:6835/202939736] conn(0x5565eb2b84f0 0x5565eb275670 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0afed96740 1 --2- 172.21.15.27:0/228094521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb2741a0 0x5565eb2a7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0afed96740 1 --2- 172.21.15.27:0/228094521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb28a0d0 0x5565eb28c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0afed96740 1 --2- 172.21.15.27:0/228094521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb28bda0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0afed96740 1 -- 172.21.15.27:0/228094521 >> 172.21.15.27:0/228094521 conn(0x5565eb25a050 msgr2=0x5565eb28dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:07.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0ad4ff9640 0 client.6651 ms_handle_reset on v2:172.21.15.27:6834/202939736 2023-12-19T18:20:07.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0afed96740 1 -- 172.21.15.27:0/228094521 shutdown_connections 2023-12-19T18:20:07.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.607+0000 7f0afed96740 1 -- 172.21.15.27:0/228094521 wait complete. 2023-12-19T18:20:07.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.608+0000 7f0afed96740 20 client.6651 trim_cache size 0 max 16384 2023-12-19T18:20:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.624+0000 7f0afed96740 1 Processor -- start 2023-12-19T18:20:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.625+0000 7f0afed96740 1 -- start start 2023-12-19T18:20:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.625+0000 7f0afed96740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb25dc50 0x5565eb28b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.625+0000 7f0afed96740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb2741a0 0x5565eb291c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.625+0000 7f0afed96740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb28a0d0 0x5565eb305740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.625+0000 7f0afed96740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565eb28d7a0 con 0x5565eb2741a0 2023-12-19T18:20:07.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.625+0000 7f0afed96740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565eaf2af20 con 0x5565eb28a0d0 2023-12-19T18:20:07.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.625+0000 7f0afed96740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565eb292bb0 con 0x5565eb25dc50 2023-12-19T18:20:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.626+0000 7f0af96e2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb2741a0 0x5565eb291c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.626+0000 7f0af96e2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb2741a0 0x5565eb291c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51148/0 (socket says 172.21.15.27:51148) 2023-12-19T18:20:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.626+0000 7f0af96e2640 1 -- 172.21.15.27:0/2594713417 learned_addr learned my addr 172.21.15.27:0/2594713417 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.626+0000 7f0af96e2640 1 -- 172.21.15.27:0/2594713417 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb25dc50 msgr2=0x5565eb28b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.626+0000 7f0af96e2640 1 --2- 172.21.15.27:0/2594713417 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb25dc50 0x5565eb28b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.626+0000 7f0af96e2640 1 -- 172.21.15.27:0/2594713417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb28a0d0 msgr2=0x5565eb305740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.626+0000 7f0af96e2640 1 --2- 172.21.15.27:0/2594713417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb28a0d0 0x5565eb305740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.626+0000 7f0af96e2640 1 -- 172.21.15.27:0/2594713417 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565eb1f7280 con 0x5565eb2741a0 2023-12-19T18:20:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.626+0000 7f0af96e2640 1 --2- 172.21.15.27:0/2594713417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb2741a0 0x5565eb291c70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0af41f5a10 tx=0x7f0ae4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1304faed0bd824ab server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.627+0000 7f0ad7fff640 1 -- 172.21.15.27:0/2594713417 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae401b070 con 0x5565eb2741a0 2023-12-19T18:20:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.627+0000 7f0ad7fff640 1 -- 172.21.15.27:0/2594713417 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ae401f070 con 0x5565eb2741a0 2023-12-19T18:20:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.627+0000 7f0ad7fff640 1 -- 172.21.15.27:0/2594713417 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ae4004260 con 0x5565eb2741a0 2023-12-19T18:20:07.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.627+0000 7f0afed96740 1 -- 172.21.15.27:0/2594713417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb2741a0 msgr2=0x5565eb291c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.627+0000 7f0afed96740 1 --2- 172.21.15.27:0/2594713417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb2741a0 0x5565eb291c70 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f0af41f5a10 tx=0x7f0ae4008040 comp rx=0 tx=0).stop 2023-12-19T18:20:07.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.627+0000 7f0afed96740 1 -- 172.21.15.27:0/2594713417 shutdown_connections 2023-12-19T18:20:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.627+0000 7f0afed96740 1 --2- 172.21.15.27:0/2594713417 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb28a0d0 0x5565eb305740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.627+0000 7f0afed96740 1 --2- 172.21.15.27:0/2594713417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb2741a0 0x5565eb291c70 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.627+0000 7f0afed96740 1 --2- 172.21.15.27:0/2594713417 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb25dc50 0x5565eb28b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.627+0000 7f0afed96740 1 -- 172.21.15.27:0/2594713417 >> 172.21.15.27:0/2594713417 conn(0x5565eb25a050 msgr2=0x5565eb279fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.627+0000 7f0afed96740 1 -- 172.21.15.27:0/2594713417 shutdown_connections 2023-12-19T18:20:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.627+0000 7f0afed96740 1 -- 172.21.15.27:0/2594713417 wait complete. 2023-12-19T18:20:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.628+0000 7f0afed96740 1 -- start start 2023-12-19T18:20:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.628+0000 7f0afed96740 1 Processor -- start 2023-12-19T18:20:07.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.629+0000 7f0afed96740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb28bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.629+0000 7f0afed96740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb2741a0 0x5565eb30b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.629+0000 7f0afed96740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb28a0d0 0x5565eb2596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.629+0000 7f0afed96740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565eb302ae0 con 0x5565eb25dc50 2023-12-19T18:20:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.629+0000 7f0afed96740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565eb30c1d0 con 0x5565eb2741a0 2023-12-19T18:20:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.629+0000 7f0afed96740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5565eb2be870 con 0x5565eb28a0d0 2023-12-19T18:20:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.629+0000 7f0afed96740 10 client.0 _ll_register_callbacks cb 0x7f0af0001650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:20:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.630+0000 7f0af9ee3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb28bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.630+0000 7f0af9ee3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb28bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51152/0 (socket says 172.21.15.27:51152) 2023-12-19T18:20:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.630+0000 7f0af9ee3640 1 -- 172.21.15.27:0/3420165164 learned_addr learned my addr 172.21.15.27:0/3420165164 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.630+0000 7f0af9ee3640 1 -- 172.21.15.27:0/3420165164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb28a0d0 msgr2=0x5565eb2596b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.630+0000 7f0af9ee3640 1 --2- 172.21.15.27:0/3420165164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb28a0d0 0x5565eb2596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.630+0000 7f0af9ee3640 1 -- 172.21.15.27:0/3420165164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb2741a0 msgr2=0x5565eb30b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.630+0000 7f0af9ee3640 1 --2- 172.21.15.27:0/3420165164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb2741a0 0x5565eb30b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.630+0000 7f0af9ee3640 1 -- 172.21.15.27:0/3420165164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ae4009850 con 0x5565eb25dc50 2023-12-19T18:20:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.630+0000 7f0af9ee3640 1 --2- 172.21.15.27:0/3420165164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb28bed0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0af0013820 tx=0x7f0af0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b9dc0e1b54fd27f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:07.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.630+0000 7f0ad4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:20:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.631+0000 7f0ad4ff9640 1 -- 172.21.15.27:0/3420165164 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0af002d500 con 0x5565eb25dc50 2023-12-19T18:20:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.631+0000 7f0ad4ff9640 1 -- 172.21.15.27:0/3420165164 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0af0032020 con 0x5565eb25dc50 2023-12-19T18:20:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.631+0000 7f0afed96740 1 -- 172.21.15.27:0/3420165164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0aec080f70 con 0x5565eb25dc50 2023-12-19T18:20:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.631+0000 7f0ad4ff9640 1 -- 172.21.15.27:0/3420165164 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0af002d660 con 0x5565eb25dc50 2023-12-19T18:20:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.631+0000 7f0afed96740 10 client.6657 fetch_fsmap learned FSMap version 38 2023-12-19T18:20:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.631+0000 7f0afed96740 1 -- 172.21.15.27:0/3420165164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=2+}) v3 -- 0x5565eb163a60 con 0x5565eb25dc50 2023-12-19T18:20:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.631+0000 7f0ad4ff9640 1 -- 172.21.15.27:0/3420165164 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aec080f70 con 0x5565eb25dc50 2023-12-19T18:20:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.631+0000 7f0ad4ff9640 1 -- 172.21.15.27:0/3420165164 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 38) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f0af002b070 con 0x5565eb25dc50 2023-12-19T18:20:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.631+0000 7f0afed96740 10 client.6657 fetch_fsmap finished waiting for FSMap version 38 2023-12-19T18:20:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.631+0000 7f0afed96740 10 client.6657 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T18:20:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.631+0000 7f0afed96740 20 client.6657 populate_metadata read hostname 'smithi027' 2023-12-19T18:20:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.632+0000 7f0afed96740 1 --2- 172.21.15.27:0/3420165164 >> [v2:172.21.15.27:6834/202939736,v1:172.21.15.27:6835/202939736] conn(0x5565eb290820 0x5565eb290c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.632+0000 7f0afed96740 4 client.6657 mds_command: new command op to 6510 tid=0[{"prefix": "status"}] 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.632+0000 7f0afed96740 1 -- 172.21.15.27:0/3420165164 --> [v2:172.21.15.27:6834/202939736,v1:172.21.15.27:6835/202939736] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5565eb1bc100 con 0x5565eb290820 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.632+0000 7f0af96e2640 1 --2- 172.21.15.27:0/3420165164 >> [v2:172.21.15.27:6834/202939736,v1:172.21.15.27:6835/202939736] conn(0x5565eb290820 0x5565eb290c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.632+0000 7f0af96e2640 1 --2- 172.21.15.27:0/3420165164 >> [v2:172.21.15.27:6834/202939736,v1:172.21.15.27:6835/202939736] conn(0x5565eb290820 0x5565eb290c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5f5cc4be09ab9ce server_cookie=ebeca2ec49c9a05a in_seq=0 out_seq=0 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.632+0000 7f0ad4ff9640 10 client.6657 ms_handle_connect on v2:172.21.15.27:6834/202939736 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0ad4ff9640 1 -- 172.21.15.27:0/3420165164 <== mds.0 v2:172.21.15.27:6834/202939736 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x5565eb1bc100 con 0x5565eb290820 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0ad4ff9640 10 client.6657 handle_command_reply: tid=0 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0afed96740 1 client.6657 shutdown 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0afed96740 1 -- 172.21.15.27:0/3420165164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 msgr2=0x5565eb28bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0afed96740 1 --2- 172.21.15.27:0/3420165164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb28bed0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f0af0013820 tx=0x7f0af0001930 comp rx=0 tx=0).stop 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0afed96740 1 -- 172.21.15.27:0/3420165164 shutdown_connections 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0afed96740 1 --2- 172.21.15.27:0/3420165164 >> [v2:172.21.15.27:6834/202939736,v1:172.21.15.27:6835/202939736] conn(0x5565eb290820 0x5565eb290c00 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0afed96740 1 --2- 172.21.15.27:0/3420165164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5565eb28a0d0 0x5565eb2596b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0afed96740 1 --2- 172.21.15.27:0/3420165164 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5565eb2741a0 0x5565eb30b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0afed96740 1 --2- 172.21.15.27:0/3420165164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5565eb25dc50 0x5565eb28bed0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0ad4ff9640 0 client.6657 ms_handle_reset on v2:172.21.15.27:6834/202939736 2023-12-19T18:20:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0afed96740 1 -- 172.21.15.27:0/3420165164 >> 172.21.15.27:0/3420165164 conn(0x5565eb25a050 msgr2=0x5565eb2a56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0afed96740 1 -- 172.21.15.27:0/3420165164 shutdown_connections 2023-12-19T18:20:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0afed96740 1 -- 172.21.15.27:0/3420165164 wait complete. 2023-12-19T18:20:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.633+0000 7f0afed96740 20 client.6657 trim_cache size 0 max 16384 2023-12-19T18:20:07.636 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:20:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:20:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-12-19T18:20:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 6510, 2023-12-19T18:20:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-19T18:20:07.636 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-19T18:20:07.637 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-19T18:20:07.637 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 7.3634586610000001, 2023-12-19T18:20:07.637 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 38, 2023-12-19T18:20:07.637 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 96, 2023-12-19T18:20:07.637 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 96, 2023-12-19T18:20:07.637 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 12.603190233999999 2023-12-19T18:20:07.637 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.634+0000 7f0afaee5640 1 -- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0ad806a460 msgr2=0x7f0ad806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.634+0000 7f0afaee5640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0ad806a460 0x7f0ad806c920 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f0af0001050 tx=0x7f0af0009040 comp rx=0 tx=0).stop 2023-12-19T18:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.634+0000 7f0afaee5640 1 -- 172.21.15.27:0/2829439712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af40c44a0 msgr2=0x7f0af40c4920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.634+0000 7f0afaee5640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af40c44a0 0x7f0af40c4920 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f0aec007330 tx=0x7f0aec004ab0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.634+0000 7f0afaee5640 1 -- 172.21.15.27:0/2829439712 shutdown_connections 2023-12-19T18:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.634+0000 7f0afaee5640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0ad806a460 0x7f0ad806c920 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.634+0000 7f0afaee5640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0af40c44a0 0x7f0af40c4920 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.634+0000 7f0afaee5640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0af414f7e0 0x7f0af4070140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.634+0000 7f0afaee5640 1 --2- 172.21.15.27:0/2829439712 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0ae0005180 0x7f0af414ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.634+0000 7f0afaee5640 1 -- 172.21.15.27:0/2829439712 >> 172.21.15.27:0/2829439712 conn(0x7f0af40cb450 msgr2=0x7f0af40c3800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.634+0000 7f0afaee5640 1 -- 172.21.15.27:0/2829439712 shutdown_connections 2023-12-19T18:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.634+0000 7f0afaee5640 1 -- 172.21.15.27:0/2829439712 wait complete. 2023-12-19T18:20:07.651 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:20:07.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.866+0000 7fe250a49640 1 Processor -- start 2023-12-19T18:20:07.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.867+0000 7fe250a49640 1 -- start start 2023-12-19T18:20:07.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.868+0000 7fe250a49640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe24c0c8f90 0x7fe24c160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.868+0000 7fe250a49640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe24c160e40 0x7fe24c0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.868+0000 7fe250a49640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24c1612d0 con 0x7fe24c0cb300 2023-12-19T18:20:07.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.868+0000 7fe250a49640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24c0c4480 con 0x7fe24c0c8f90 2023-12-19T18:20:07.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.868+0000 7fe250a49640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c1468c0 con 0x7fe24c160e40 2023-12-19T18:20:07.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.869+0000 7fe243fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe24c0c8f90 0x7fe24c160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.869+0000 7fe243fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe24c0c8f90 0x7fe24c160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51160/0 (socket says 172.21.15.27:51160) 2023-12-19T18:20:07.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.869+0000 7fe243fff640 1 -- 172.21.15.27:0/1013847900 learned_addr learned my addr 172.21.15.27:0/1013847900 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:07.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.869+0000 7fe24b59e640 1 --2- 172.21.15.27:0/1013847900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe24c160e40 0x7fe24c0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.869+0000 7fe243fff640 1 -- 172.21.15.27:0/1013847900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe24c160e40 msgr2=0x7fe24c0cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:07.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.869+0000 7fe243fff640 1 --2- 172.21.15.27:0/1013847900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe24c160e40 0x7fe24c0cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.869+0000 7fe243fff640 1 -- 172.21.15.27:0/1013847900 >> v1:172.21.15.192:6789/0 conn(0x7fe24c0cb300 legacy=0x7fe24c0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.869+0000 7fe243fff640 1 -- 172.21.15.27:0/1013847900 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24c145d60 con 0x7fe24c0c8f90 2023-12-19T18:20:07.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.871+0000 7fe243fff640 1 --2- 172.21.15.27:0/1013847900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe24c0c8f90 0x7fe24c160900 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe2340098a0 tx=0x7fe23402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6fede53f18fcba server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:07.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.871+0000 7fe24a59c640 1 -- 172.21.15.27:0/1013847900 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234030840 con 0x7fe24c0c8f90 2023-12-19T18:20:07.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.871+0000 7fe24a59c640 1 -- 172.21.15.27:0/1013847900 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2340309a0 con 0x7fe24c0c8f90 2023-12-19T18:20:07.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.871+0000 7fe24a59c640 1 -- 172.21.15.27:0/1013847900 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234030d30 con 0x7fe24c0c8f90 2023-12-19T18:20:07.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.872+0000 7fe250a49640 1 -- 172.21.15.27:0/1013847900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe24c0c8f90 msgr2=0x7fe24c160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.872+0000 7fe250a49640 1 --2- 172.21.15.27:0/1013847900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe24c0c8f90 0x7fe24c160900 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe2340098a0 tx=0x7fe23402b980 comp rx=0 tx=0).stop 2023-12-19T18:20:07.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.872+0000 7fe250a49640 1 -- 172.21.15.27:0/1013847900 shutdown_connections 2023-12-19T18:20:07.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.872+0000 7fe250a49640 1 --2- 172.21.15.27:0/1013847900 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe24c160e40 0x7fe24c0cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.872+0000 7fe250a49640 1 --2- 172.21.15.27:0/1013847900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe24c0c8f90 0x7fe24c160900 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.872+0000 7fe250a49640 1 -- 172.21.15.27:0/1013847900 >> 172.21.15.27:0/1013847900 conn(0x7fe24c069c70 msgr2=0x7fe24c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:07.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.872+0000 7fe250a49640 1 -- 172.21.15.27:0/1013847900 shutdown_connections 2023-12-19T18:20:07.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.872+0000 7fe250a49640 1 -- 172.21.15.27:0/1013847900 wait complete. 2023-12-19T18:20:07.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.874+0000 7fe250a49640 1 Processor -- start 2023-12-19T18:20:07.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.874+0000 7fe250a49640 1 -- start start 2023-12-19T18:20:07.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.875+0000 7fe250a49640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe24c0c8f90 0x7fe24c0c9470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.875+0000 7fe250a49640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe24c0cb300 0x7fe24c0c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.875+0000 7fe250a49640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe24c160e40 0x7fe24c0c7ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.875+0000 7fe250a49640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c15ccf0 con 0x7fe24c0c8f90 2023-12-19T18:20:07.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.875+0000 7fe250a49640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c15f150 con 0x7fe24c160e40 2023-12-19T18:20:07.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.875+0000 7fe250a49640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24c0c5a10 con 0x7fe24c0cb300 2023-12-19T18:20:07.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.875+0000 7fe24ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe24c0c8f90 0x7fe24c0c9470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.875+0000 7fe24ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe24c0c8f90 0x7fe24c0c9470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51174/0 (socket says 172.21.15.27:51174) 2023-12-19T18:20:07.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.875+0000 7fe24ad9d640 1 -- 172.21.15.27:0/3918297139 learned_addr learned my addr 172.21.15.27:0/3918297139 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:07.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.875+0000 7fe24b59e640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe24c160e40 0x7fe24c0c7ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.876+0000 7fe243fff640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe24c0cb300 0x7fe24c0c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.876+0000 7fe24ad9d640 1 -- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe24c0cb300 msgr2=0x7fe24c0c99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.876+0000 7fe24ad9d640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe24c0cb300 0x7fe24c0c99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.876+0000 7fe24ad9d640 1 -- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe24c160e40 msgr2=0x7fe24c0c7ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:07.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.876+0000 7fe24ad9d640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe24c160e40 0x7fe24c0c7ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:07.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.876+0000 7fe24ad9d640 1 -- 172.21.15.27:0/3918297139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2340094d0 con 0x7fe24c0c8f90 2023-12-19T18:20:07.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.876+0000 7fe243fff640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe24c0cb300 0x7fe24c0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:07.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.876+0000 7fe24b59e640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe24c160e40 0x7fe24c0c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:07.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.876+0000 7fe24ad9d640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe24c0c8f90 0x7fe24c0c9470 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00c450 tx=0x7fe23c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:07.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.876+0000 7fe2437fe640 1 -- 172.21.15.27:0/3918297139 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0052d0 con 0x7fe24c0c8f90 2023-12-19T18:20:07.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.876+0000 7fe2437fe640 1 -- 172.21.15.27:0/3918297139 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c005430 con 0x7fe24c0c8f90 2023-12-19T18:20:07.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.876+0000 7fe2437fe640 1 -- 172.21.15.27:0/3918297139 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c012870 con 0x7fe24c0c8f90 2023-12-19T18:20:07.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.876+0000 7fe250a49640 1 -- 172.21.15.27:0/3918297139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24c15e540 con 0x7fe24c0c8f90 2023-12-19T18:20:07.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.877+0000 7fe250a49640 1 -- 172.21.15.27:0/3918297139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24c15eb00 con 0x7fe24c0c8f90 2023-12-19T18:20:07.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.878+0000 7fe250a49640 1 -- 172.21.15.27:0/3918297139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24c04e840 con 0x7fe24c0c8f90 2023-12-19T18:20:07.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.879+0000 7fe2437fe640 1 -- 172.21.15.27:0/3918297139 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe23c0129d0 con 0x7fe24c0c8f90 2023-12-19T18:20:07.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.880+0000 7fe2437fe640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe22c06a460 0x7fe22c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:07.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.880+0000 7fe2437fe640 1 -- 172.21.15.27:0/3918297139 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe23c088330 con 0x7fe24c0c8f90 2023-12-19T18:20:07.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.881+0000 7fe243fff640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe22c06a460 0x7fe22c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:07.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.882+0000 7fe243fff640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe22c06a460 0x7fe22c06c920 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe234036000 tx=0x7fe23403b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:07.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:07.884+0000 7fe2437fe640 1 -- 172.21.15.27:0/3918297139 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe23c053880 con 0x7fe24c0c8f90 2023-12-19T18:20:08.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.123+0000 7fe250a49640 1 -- 172.21.15.27:0/3918297139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe24c1612d0 con 0x7fe24c0c8f90 2023-12-19T18:20:08.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.124+0000 7fe2437fe640 1 -- 172.21.15.27:0/3918297139 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe24c1612d0 con 0x7fe24c0c8f90 2023-12-19T18:20:08.125 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:20:08.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.127+0000 7fe250a49640 1 -- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe22c06a460 msgr2=0x7fe22c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.127+0000 7fe250a49640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe22c06a460 0x7fe22c06c920 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fe234036000 tx=0x7fe23403b040 comp rx=0 tx=0).stop 2023-12-19T18:20:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.127+0000 7fe250a49640 1 -- 172.21.15.27:0/3918297139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe24c0c8f90 msgr2=0x7fe24c0c9470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.127+0000 7fe250a49640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe24c0c8f90 0x7fe24c0c9470 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe23c00c450 tx=0x7fe23c00c910 comp rx=0 tx=0).stop 2023-12-19T18:20:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.127+0000 7fe250a49640 1 -- 172.21.15.27:0/3918297139 shutdown_connections 2023-12-19T18:20:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.127+0000 7fe250a49640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe22c06a460 0x7fe22c06c920 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.127+0000 7fe250a49640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe24c160e40 0x7fe24c0c7ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.127+0000 7fe250a49640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe24c0cb300 0x7fe24c0c99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.127+0000 7fe250a49640 1 --2- 172.21.15.27:0/3918297139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe24c0c8f90 0x7fe24c0c9470 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.127+0000 7fe250a49640 1 -- 172.21.15.27:0/3918297139 >> 172.21.15.27:0/3918297139 conn(0x7fe24c069c70 msgr2=0x7fe24c0cb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.128+0000 7fe250a49640 1 -- 172.21.15.27:0/3918297139 shutdown_connections 2023-12-19T18:20:08.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:08.128+0000 7fe250a49640 1 -- 172.21.15.27:0/3918297139 wait complete. 2023-12-19T18:20:08.148 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:20:08.148 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:20:08.149 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-12-19T18:20:08.149 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:20:08.149 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:20:08.149 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:20:08.149 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:20:08.149 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:20:08.149 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:20:08.150 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:08.150 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-12-19T18:20:08.210 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:20:08.210 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-12-19T18:20:08.211 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42032sec preferred_lft 42032sec 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::8053:76ff:fe69:c66c/64 scope link 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::781d:d6ff:feb4:7a2d/64 scope link 2023-12-19T18:20:08.212 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:08.213 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:20:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether f2:29:63:cc:14:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:20:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::f029:63ff:fecc:140e/64 scope link 2023-12-19T18:20:08.213 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:08.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:08.214 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:20:08.318 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-12-19T18:20:08.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:08.323 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-12-19T18:20:08.385 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-12-19T18:20:08.385 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:20:08.385 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-19T18:20:08.385 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-19T18:20:08.386 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:20:08.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:08.386 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:20:08.386 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:20:08.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:20:08.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:20:08.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:20:08.386 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:20:08.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:08.604 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:20:08.658 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:20:08.659 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:20:08.660 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:20:08.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:08.660 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-12-19T18:20:08.767 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-19T18:20:08.767 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:20:08.767 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:20:08.826 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:20:08.826 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:20:08.826 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:20:08.884 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:20:08.994 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:20:08.995 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:20:09.101 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:20:09.105 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:20:09.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:09.106 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:20:09.166 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:20:09.166 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-19T18:20:09.175 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:20:09.261 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:20:09.262 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:20:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:20:09.342 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:20:09.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:09.343 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:20:09.378 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-12-19T18:20:09.378+0000 7ff46906d480 -1 init, newargv = 0x55bd73abc270 newargc=15 2023-12-19T18:20:09.378 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[103318]: starting ceph client 2023-12-19T18:20:09.386 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[103318]: starting fuse 2023-12-19T18:20:09.392 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:20:09.393 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-19T18:20:09.393 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:20:09.393 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:20:09.393 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:20:09.393 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:20:09.393 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:20:09.393 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:20:09.393 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:20:09.393 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:20:09.393 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:20:09.393 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:20:09.394 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:20:09.395 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:20:09.395 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:20:09.604 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.103318.asok 2023-12-19T18:20:09.783 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.103318.asok 2023-12-19T18:20:09.783 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:09.784 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.103318.asok status 2023-12-19T18:20:10.045 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:20:10.045 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:20:10.045 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "103318", 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 6669, 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:20:10.046 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 6669 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3031572523 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3031572523 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.6669 192.168.0.1:0/3031572523", 2023-12-19T18:20:10.047 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3031572523", 2023-12-19T18:20:10.048 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-12-19T18:20:10.048 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 38, 2023-12-19T18:20:10.048 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 96, 2023-12-19T18:20:10.048 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:20:10.048 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:20:10.048 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:20:10.048 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:20:10.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:10.063 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-12-19T18:20:10.063 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-19T18:20:10.125 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-12-19T18:20:10.126 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-19T18:20:10.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:10.126 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:20:10.235 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:20:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.477+0000 7f8d4e634640 1 Processor -- start 2023-12-19T18:20:10.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.477+0000 7f8d4e634640 1 -- start start 2023-12-19T18:20:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.479+0000 7f8d4e634640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b2ce0 0x7f8d48165b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.479+0000 7f8d4e634640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d480ce050 con 0x7f8d4815ad60 2023-12-19T18:20:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.479+0000 7f8d4e634640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4815b850 con 0x7f8d480b3170 2023-12-19T18:20:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.479+0000 7f8d4e634640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4815bb20 con 0x7f8d480b2ce0 2023-12-19T18:20:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.479+0000 7f8d4d632640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8d4815ad60 0x7f8d4815b140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:36420/0 (socket says 172.21.15.27:36420) 2023-12-19T18:20:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.479+0000 7f8d4d632640 1 -- 172.21.15.27:0/3024361861 learned_addr learned my addr 172.21.15.27:0/3024361861 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.479+0000 7f8d4de33640 1 --2- 172.21.15.27:0/3024361861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b2ce0 0x7f8d48165b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.479+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1032871912 0 0) 0x7f8d480ce050 con 0x7f8d4815ad60 2023-12-19T18:20:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d30003520 con 0x7f8d4815ad60 2023-12-19T18:20:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3745642965 0 0) 0x7f8d30003520 con 0x7f8d4815ad60 2023-12-19T18:20:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b2ce0 msgr2=0x7f8d48165b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 --2- 172.21.15.27:0/3024361861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b2ce0 0x7f8d48165b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 >> v1:172.21.15.192:6789/0 conn(0x7f8d480b3170 legacy=0x7f8d480b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4815be00 con 0x7f8d4815ad60 2023-12-19T18:20:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f8d380019f0 con 0x7f8d4815ad60 2023-12-19T18:20:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 >> v1:172.21.15.27:6789/0 conn(0x7f8d4815ad60 legacy=0x7f8d4815b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 --2- 172.21.15.27:0/3024361861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b2ce0 0x7f8d300039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 --2- 172.21.15.27:0/3024361861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d30003f20 0x7f8d300047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 --2- 172.21.15.27:0/3024361861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d30004d00 0x7f8d30005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4815bb20 con 0x7f8d30003f20 2023-12-19T18:20:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3000ba60 con 0x7f8d30004d00 2023-12-19T18:20:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d3000bc20 con 0x7f8d480b2ce0 2023-12-19T18:20:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.480+0000 7f8d4ce31640 1 --2- 172.21.15.27:0/3024361861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d30003f20 0x7f8d300047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.481+0000 7f8d4d632640 1 --2- 172.21.15.27:0/3024361861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b2ce0 0x7f8d300039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.481+0000 7f8d4de33640 1 --2- 172.21.15.27:0/3024361861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d30004d00 0x7f8d30005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.481+0000 7f8d4ce31640 1 -- 172.21.15.27:0/3024361861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b2ce0 msgr2=0x7f8d300039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.481+0000 7f8d4ce31640 1 --2- 172.21.15.27:0/3024361861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b2ce0 0x7f8d300039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.481+0000 7f8d4ce31640 1 -- 172.21.15.27:0/3024361861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d30004d00 msgr2=0x7f8d30005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.481+0000 7f8d4ce31640 1 --2- 172.21.15.27:0/3024361861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d30004d00 0x7f8d30005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.481+0000 7f8d4ce31640 1 -- 172.21.15.27:0/3024361861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d30003520 con 0x7f8d30003f20 2023-12-19T18:20:10.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.481+0000 7f8d4d632640 1 --2- 172.21.15.27:0/3024361861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b2ce0 0x7f8d300039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:20:10.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.482+0000 7f8d4ce31640 1 --2- 172.21.15.27:0/3024361861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d30003f20 0x7f8d300047c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c001750 tx=0x7f8d300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb95c381fc596ae5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:10.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.483+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c0052d0 con 0x7f8d30003f20 2023-12-19T18:20:10.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.483+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3c02e050 con 0x7f8d30003f20 2023-12-19T18:20:10.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.483+0000 7f8d47fff640 1 -- 172.21.15.27:0/3024361861 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3c003830 con 0x7f8d30003f20 2023-12-19T18:20:10.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.483+0000 7f8d4e634640 1 -- 172.21.15.27:0/3024361861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d30003f20 msgr2=0x7f8d300047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:10.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.483+0000 7f8d4e634640 1 --2- 172.21.15.27:0/3024361861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d30003f20 0x7f8d300047c0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f8d3c001750 tx=0x7f8d300057f0 comp rx=0 tx=0).stop 2023-12-19T18:20:10.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.483+0000 7f8d4de33640 1 -- 172.21.15.27:0/3024361861 reap_dead start 2023-12-19T18:20:10.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.484+0000 7f8d4e634640 1 -- 172.21.15.27:0/3024361861 shutdown_connections 2023-12-19T18:20:10.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.484+0000 7f8d4e634640 1 -- 172.21.15.27:0/3024361861 >> 172.21.15.27:0/3024361861 conn(0x7f8d4806d5f0 msgr2=0x7f8d4806d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:10.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.484+0000 7f8d4e634640 1 -- 172.21.15.27:0/3024361861 shutdown_connections 2023-12-19T18:20:10.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.484+0000 7f8d4e634640 1 -- 172.21.15.27:0/3024361861 wait complete. 2023-12-19T18:20:10.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.486+0000 7f8d4e634640 1 Processor -- start 2023-12-19T18:20:10.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.486+0000 7f8d4e634640 1 -- start start 2023-12-19T18:20:10.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.487+0000 7f8d4e634640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d30003f20 0x7f8d48152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.487+0000 7f8d4e634640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b53f0 0x7f8d48152070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.487+0000 7f8d4e634640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d480c8eb0 0x7f8d480b4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.487+0000 7f8d4e634640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4806bb30 con 0x7f8d480c8eb0 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.487+0000 7f8d4e634640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4806af20 con 0x7f8d30003f20 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.487+0000 7f8d4e634640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4806b200 con 0x7f8d480b53f0 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.487+0000 7f8d4de33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d480c8eb0 0x7f8d480b4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.487+0000 7f8d4de33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d480c8eb0 0x7f8d480b4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:51180/0 (socket says 172.21.15.27:51180) 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.487+0000 7f8d4de33640 1 -- 172.21.15.27:0/2418148289 learned_addr learned my addr 172.21.15.27:0/2418148289 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.487+0000 7f8d4d632640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d30003f20 0x7f8d48152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.487+0000 7f8d4ce31640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b53f0 0x7f8d48152070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.487+0000 7f8d4de33640 1 -- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b53f0 msgr2=0x7f8d48152070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.487+0000 7f8d4de33640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b53f0 0x7f8d48152070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.488+0000 7f8d4de33640 1 -- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d30003f20 msgr2=0x7f8d48152d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:10.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.488+0000 7f8d4de33640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d30003f20 0x7f8d48152d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.488+0000 7f8d4de33640 1 -- 172.21.15.27:0/2418148289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4806b4e0 con 0x7f8d480c8eb0 2023-12-19T18:20:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.488+0000 7f8d4d632640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d30003f20 0x7f8d48152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.488+0000 7f8d4de33640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d480c8eb0 0x7f8d480b4750 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8d48071420 tx=0x7f8d40011920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.488+0000 7f8d45ffb640 1 -- 172.21.15.27:0/2418148289 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d40004500 con 0x7f8d480c8eb0 2023-12-19T18:20:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.488+0000 7f8d45ffb640 1 -- 172.21.15.27:0/2418148289 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d40003610 con 0x7f8d480c8eb0 2023-12-19T18:20:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.488+0000 7f8d45ffb640 1 -- 172.21.15.27:0/2418148289 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d400039a0 con 0x7f8d480c8eb0 2023-12-19T18:20:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.488+0000 7f8d4e634640 1 -- 172.21.15.27:0/2418148289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4806b7c0 con 0x7f8d480c8eb0 2023-12-19T18:20:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.488+0000 7f8d4e634640 1 -- 172.21.15.27:0/2418148289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d480c9330 con 0x7f8d480c8eb0 2023-12-19T18:20:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.489+0000 7f8d4e634640 1 -- 172.21.15.27:0/2418148289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d480ccb00 con 0x7f8d480c8eb0 2023-12-19T18:20:10.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.490+0000 7f8d45ffb640 1 -- 172.21.15.27:0/2418148289 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8d40003b00 con 0x7f8d480c8eb0 2023-12-19T18:20:10.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.491+0000 7f8d45ffb640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8d3406a460 0x7f8d3406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:10.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.491+0000 7f8d45ffb640 1 -- 172.21.15.27:0/2418148289 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8d40088760 con 0x7f8d480c8eb0 2023-12-19T18:20:10.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.492+0000 7f8d4d632640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8d3406a460 0x7f8d3406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:10.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.493+0000 7f8d4d632640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8d3406a460 0x7f8d3406c920 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8d38002230 tx=0x7f8d38002180 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:10.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.496+0000 7f8d45ffb640 1 -- 172.21.15.27:0/2418148289 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8d480ccb00 con 0x7f8d480c8eb0 2023-12-19T18:20:10.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.730+0000 7f8d4e634640 1 -- 172.21.15.27:0/2418148289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f8d480ccb00 con 0x7f8d480c8eb0 2023-12-19T18:20:10.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.731+0000 7f8d45ffb640 1 -- 172.21.15.27:0/2418148289 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v38) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8d480ccb00 con 0x7f8d480c8eb0 2023-12-19T18:20:10.731 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:20:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.733+0000 7f8d4e634640 1 -- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8d3406a460 msgr2=0x7f8d3406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.733+0000 7f8d4e634640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8d3406a460 0x7f8d3406c920 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f8d38002230 tx=0x7f8d38002180 comp rx=0 tx=0).stop 2023-12-19T18:20:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.734+0000 7f8d4e634640 1 -- 172.21.15.27:0/2418148289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d480c8eb0 msgr2=0x7f8d480b4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.734+0000 7f8d4e634640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d480c8eb0 0x7f8d480b4750 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f8d48071420 tx=0x7f8d40011920 comp rx=0 tx=0).stop 2023-12-19T18:20:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.734+0000 7f8d4e634640 1 -- 172.21.15.27:0/2418148289 shutdown_connections 2023-12-19T18:20:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.734+0000 7f8d4e634640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8d3406a460 0x7f8d3406c920 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.734+0000 7f8d4e634640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d480c8eb0 0x7f8d480b4750 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:10.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.734+0000 7f8d4e634640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d480b53f0 0x7f8d48152070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:10.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.734+0000 7f8d4e634640 1 --2- 172.21.15.27:0/2418148289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d30003f20 0x7f8d48152d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:10.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.734+0000 7f8d4e634640 1 -- 172.21.15.27:0/2418148289 >> 172.21.15.27:0/2418148289 conn(0x7f8d4806d5f0 msgr2=0x7f8d480b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:10.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.734+0000 7f8d4e634640 1 -- 172.21.15.27:0/2418148289 shutdown_connections 2023-12-19T18:20:10.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:10.734+0000 7f8d4e634640 1 -- 172.21.15.27:0/2418148289 wait complete. 2023-12-19T18:20:10.755 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:20:10.755 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:20:10.755 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-19T18:20:10.755 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-19T18:20:10.755 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T18:20:10.755 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:20:10.755 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:20:10.755 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:20:10.755 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:20:10.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:10.755 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-19T18:20:10.816 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:20:10.816 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:20:10.816 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:20:10.816 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:10.816 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:20:10.816 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:10.816 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:20:10.816 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-19T18:20:10.816 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-19T18:20:10.816 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:20:10.817 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-19T18:20:10.817 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-19T18:20:10.817 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:20:10.817 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:20:10.817 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:20:10.817 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:20:10.817 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-19T18:20:10.817 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 41992sec preferred_lft 41992sec 2023-12-19T18:20:10.817 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-19T18:20:10.817 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:10.817 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:20:10.817 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:20:10.818 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:20:10.818 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:10.818 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::b892:1cff:fedc:4bed/64 scope link 2023-12-19T18:20:10.818 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:10.818 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:20:10.818 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether a6:e1:fc:e6:56:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:20:10.818 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::a4e1:fcff:fee6:56fa/64 scope link 2023-12-19T18:20:10.818 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:10.818 INFO:teuthology.orchestra.run.smithi192.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:20:10.818 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T18:20:10.818 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::404f:6cff:fe73:9aa2/64 scope link 2023-12-19T18:20:10.818 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:20:10.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:10.819 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:20:10.913 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-19T18:20:10.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:10.917 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-19T18:20:10.967 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-19T18:20:10.967 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:20:10.967 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T18:20:10.967 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T18:20:10.967 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:20:10.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:10.968 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:20:10.968 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:20:10.968 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T18:20:10.968 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T18:20:10.968 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T18:20:10.968 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:20:11.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:11.125 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:20:11.166 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T18:20:11.166 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-19T18:20:11.167 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:20:11.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:11.167 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-19T18:20:11.279 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-19T18:20:11.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:20:11.279 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-19T18:20:11.318 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:20:11.318 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:20:11.318 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:20:11.405 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:20:11.540 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:20:11.540 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:20:11.626 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:20:11.630 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:20:11.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:11.630 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:20:11.684 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:20:11.685 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-19T18:20:11.690 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:20:11.765 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:20:11.765 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:20:11.836 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:20:11.838 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:20:11.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:11.839 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:20:11.877 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-19T18:20:11.877+0000 7fdf15e45480 -1 init, newargv = 0x5564a2122850 newargc=15 2023-12-19T18:20:11.877 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[88552]: starting ceph client 2023-12-19T18:20:11.886 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[88552]: starting fuse 2023-12-19T18:20:12.979 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:20:13.083 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:20:13.084 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:20:13.174 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:20:13.178 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:20:13.178 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:13.178 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:20:13.233 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:20:13.233 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-12-19T18:20:13.233 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:20:13.233 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-19T18:20:13.233 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-19T18:20:13.233 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-19T18:20:13.233 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:20:13.234 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-19T18:20:13.235 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-19T18:20:13.235 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-19T18:20:13.235 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:20:13.235 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:20:13.235 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-19T18:20:13.235 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-19T18:20:13.416 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.88552.asok 2023-12-19T18:20:13.597 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.88552.asok 2023-12-19T18:20:13.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:13.598 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.88552.asok status 2023-12-19T18:20:13.838 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-19T18:20:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-19T18:20:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:20:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:20:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-19T18:20:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-19T18:20:13.839 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-19T18:20:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "88552", 2023-12-19T18:20:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-19T18:20:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:20:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-19T18:20:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-19T18:20:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5258, 2023-12-19T18:20:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-19T18:20:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-19T18:20:13.840 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-19T18:20:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5258 2023-12-19T18:20:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:20:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:20:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:20:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:20:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2535164405 2023-12-19T18:20:13.841 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-19T18:20:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:20:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:20:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:20:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:20:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2535164405 2023-12-19T18:20:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:20:13.842 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5258 192.168.0.1:0/2535164405", 2023-12-19T18:20:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/2535164405", 2023-12-19T18:20:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-19T18:20:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 38, 2023-12-19T18:20:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 96, 2023-12-19T18:20:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:20:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-19T18:20:13.843 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-19T18:20:13.843 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-19T18:20:13.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:13.854 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-19T18:20:13.854 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-19T18:20:13.908 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-19T18:20:13.908 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-19T18:20:13.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:13.909 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:20:13.998 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:20:14.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.244+0000 7f8512a7d640 1 Processor -- start 2023-12-19T18:20:14.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.244+0000 7f8512a7d640 1 -- start start 2023-12-19T18:20:14.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.246+0000 7f8512a7d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f850c0cb300 0x7f850c0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.246+0000 7f8512a7d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f850c0c8f90 0x7f850c160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.246+0000 7f8512a7d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f850c161290 con 0x7f850c160e40 2023-12-19T18:20:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.246+0000 7f8512a7d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850c0c4480 con 0x7f850c0cb300 2023-12-19T18:20:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.246+0000 7f8512a7d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f850c1468c0 con 0x7f850c0c8f90 2023-12-19T18:20:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.246+0000 7f8511a7b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f850c0cb300 0x7f850c0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.246+0000 7f8511a7b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f850c0cb300 0x7f850c0cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57522/0 (socket says 172.21.15.27:57522) 2023-12-19T18:20:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.246+0000 7f8511a7b640 1 -- 172.21.15.27:0/3519055283 learned_addr learned my addr 172.21.15.27:0/3519055283 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.246+0000 7f851127a640 1 --2- 172.21.15.27:0/3519055283 >> v2:172.21.15.192:3300/0 conn(0x7f850c0c8f90 0x7f850c160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.247+0000 7f8511a7b640 1 -- 172.21.15.27:0/3519055283 >> v2:172.21.15.192:3300/0 conn(0x7f850c0c8f90 msgr2=0x7f850c160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.247+0000 7f8511a7b640 1 --2- 172.21.15.27:0/3519055283 >> v2:172.21.15.192:3300/0 conn(0x7f850c0c8f90 0x7f850c160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.247+0000 7f8511a7b640 1 -- 172.21.15.27:0/3519055283 >> v1:172.21.15.192:6789/0 conn(0x7f850c160e40 legacy=0x7f850c0ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.247+0000 7f8511a7b640 1 -- 172.21.15.27:0/3519055283 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f850c0cbc20 con 0x7f850c0cb300 2023-12-19T18:20:14.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.248+0000 7f8511a7b640 1 --2- 172.21.15.27:0/3519055283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850c0cb300 0x7f850c0cb6e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8508009870 tx=0x7f850802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1af6bf030f121d2d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:14.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.248+0000 7f8510a79640 1 -- 172.21.15.27:0/3519055283 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508031870 con 0x7f850c0cb300 2023-12-19T18:20:14.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.248+0000 7f8510a79640 1 -- 172.21.15.27:0/3519055283 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85080319d0 con 0x7f850c0cb300 2023-12-19T18:20:14.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.248+0000 7f8510a79640 1 -- 172.21.15.27:0/3519055283 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508031d60 con 0x7f850c0cb300 2023-12-19T18:20:14.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.249+0000 7f8512a7d640 1 -- 172.21.15.27:0/3519055283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850c0cb300 msgr2=0x7f850c0cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:14.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.249+0000 7f8512a7d640 1 --2- 172.21.15.27:0/3519055283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850c0cb300 0x7f850c0cb6e0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8508009870 tx=0x7f850802b910 comp rx=0 tx=0).stop 2023-12-19T18:20:14.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.249+0000 7f8512a7d640 1 -- 172.21.15.27:0/3519055283 shutdown_connections 2023-12-19T18:20:14.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.249+0000 7f8512a7d640 1 --2- 172.21.15.27:0/3519055283 >> v2:172.21.15.192:3300/0 conn(0x7f850c0c8f90 0x7f850c160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:14.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.249+0000 7f8512a7d640 1 --2- 172.21.15.27:0/3519055283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850c0cb300 0x7f850c0cb6e0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:14.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.249+0000 7f8512a7d640 1 -- 172.21.15.27:0/3519055283 >> 172.21.15.27:0/3519055283 conn(0x7f850c069c70 msgr2=0x7f850c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:14.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.249+0000 7f8512a7d640 1 -- 172.21.15.27:0/3519055283 shutdown_connections 2023-12-19T18:20:14.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.249+0000 7f8512a7d640 1 -- 172.21.15.27:0/3519055283 wait complete. 2023-12-19T18:20:14.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.251+0000 7f8512a7d640 1 Processor -- start 2023-12-19T18:20:14.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.252+0000 7f8512a7d640 1 -- start start 2023-12-19T18:20:14.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.253+0000 7f8512a7d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850c0c8f90 0x7f850c1420f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:14.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.253+0000 7f8512a7d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f850c0cb300 0x7f850c15c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:14.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.253+0000 7f8512a7d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850c160e40 0x7f850c144170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:14.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.253+0000 7f8512a7d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c15ca80 con 0x7f850c0c8f90 2023-12-19T18:20:14.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.253+0000 7f8512a7d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f850c143ce0 con 0x7f850c160e40 2023-12-19T18:20:14.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.253+0000 7f8512a7d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f850c142980 con 0x7f850c0cb300 2023-12-19T18:20:14.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.253+0000 7f8511a7b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850c0c8f90 0x7f850c1420f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:14.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.253+0000 7f8511a7b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850c0c8f90 0x7f850c1420f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57534/0 (socket says 172.21.15.27:57534) 2023-12-19T18:20:14.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.253+0000 7f8511a7b640 1 -- 172.21.15.27:0/1859903309 learned_addr learned my addr 172.21.15.27:0/1859903309 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:14.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.253+0000 7f851127a640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f850c0cb300 0x7f850c15c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:14.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.254+0000 7f851227c640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850c160e40 0x7f850c144170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:14.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.254+0000 7f8511a7b640 1 -- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f850c0cb300 msgr2=0x7f850c15c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:14.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.254+0000 7f8511a7b640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f850c0cb300 0x7f850c15c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:14.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.254+0000 7f8511a7b640 1 -- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850c160e40 msgr2=0x7f850c144170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:14.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.254+0000 7f8511a7b640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850c160e40 0x7f850c144170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:14.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.254+0000 7f8511a7b640 1 -- 172.21.15.27:0/1859903309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85080094d0 con 0x7f850c0c8f90 2023-12-19T18:20:14.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.254+0000 7f851227c640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850c160e40 0x7f850c144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:14.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.254+0000 7f8511a7b640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850c0c8f90 0x7f850c1420f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8508009870 tx=0x7f85080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.254+0000 7f85027fc640 1 -- 172.21.15.27:0/1859903309 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f850803ac60 con 0x7f850c0c8f90 2023-12-19T18:20:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.254+0000 7f85027fc640 1 -- 172.21.15.27:0/1859903309 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f850803adc0 con 0x7f850c0c8f90 2023-12-19T18:20:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.254+0000 7f85027fc640 1 -- 172.21.15.27:0/1859903309 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8508044610 con 0x7f850c0c8f90 2023-12-19T18:20:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.255+0000 7f8512a7d640 1 -- 172.21.15.27:0/1859903309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f850c1430d0 con 0x7f850c0c8f90 2023-12-19T18:20:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.255+0000 7f8512a7d640 1 -- 172.21.15.27:0/1859903309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f850c143690 con 0x7f850c0c8f90 2023-12-19T18:20:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.256+0000 7f8512a7d640 1 -- 172.21.15.27:0/1859903309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f850c04e840 con 0x7f850c0c8f90 2023-12-19T18:20:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.257+0000 7f85027fc640 1 -- 172.21.15.27:0/1859903309 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f850804d450 con 0x7f850c0c8f90 2023-12-19T18:20:14.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.258+0000 7f85027fc640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f84f006a430 0x7f84f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:14.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.258+0000 7f85027fc640 1 -- 172.21.15.27:0/1859903309 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f850802d040 con 0x7f850c0c8f90 2023-12-19T18:20:14.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.259+0000 7f851127a640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f84f006a430 0x7f84f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.260+0000 7f851127a640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f84f006a430 0x7f84f006c8f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f84fc006fd0 tx=0x7f84fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.264+0000 7f85027fc640 1 -- 172.21.15.27:0/1859903309 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8508076510 con 0x7f850c0c8f90 2023-12-19T18:20:14.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.497+0000 7f8512a7d640 1 -- 172.21.15.27:0/1859903309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f850c15cbf0 con 0x7f850c0c8f90 2023-12-19T18:20:14.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.499+0000 7f85027fc640 1 -- 172.21.15.27:0/1859903309 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f8508035020 con 0x7f850c0c8f90 2023-12-19T18:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:14.499 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2860412135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2860412135},{"type":"v1","addr":"172.21.15.192:6837","nonce":2860412135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/797098667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":797098667},{"type":"v1","addr":"172.21.15.192:6836","nonce":797098667}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":6507,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/4009181263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4009181263},{"type":"v1","addr":"172.21.15.27:6833","nonce":4009181263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:20:00.264651+0000","modified":"2023-12-19T18:20:03.796309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6510},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6510":{"gid":6510,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/202939736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":202939736},{"type":"v1","addr":"172.21.15.27:6835","nonce":202939736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-19T18:20:14.499 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2023-12-19T18:20:14.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.501+0000 7f8512a7d640 1 -- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f84f006a430 msgr2=0x7f84f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:14.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.501+0000 7f8512a7d640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f84f006a430 0x7f84f006c8f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f84fc006fd0 tx=0x7f84fc008040 comp rx=0 tx=0).stop 2023-12-19T18:20:14.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.502+0000 7f8512a7d640 1 -- 172.21.15.27:0/1859903309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850c0c8f90 msgr2=0x7f850c1420f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:14.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.502+0000 7f8512a7d640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850c0c8f90 0x7f850c1420f0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8508009870 tx=0x7f85080046c0 comp rx=0 tx=0).stop 2023-12-19T18:20:14.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.502+0000 7f8512a7d640 1 -- 172.21.15.27:0/1859903309 shutdown_connections 2023-12-19T18:20:14.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.502+0000 7f8512a7d640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f84f006a430 0x7f84f006c8f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:14.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.502+0000 7f8512a7d640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f850c160e40 0x7f850c144170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.502+0000 7f8512a7d640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f850c0cb300 0x7f850c15c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.502+0000 7f8512a7d640 1 --2- 172.21.15.27:0/1859903309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f850c0c8f90 0x7f850c1420f0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.502+0000 7f8512a7d640 1 -- 172.21.15.27:0/1859903309 >> 172.21.15.27:0/1859903309 conn(0x7f850c069c70 msgr2=0x7f850c0ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.502+0000 7f8512a7d640 1 -- 172.21.15.27:0/1859903309 shutdown_connections 2023-12-19T18:20:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:14.502+0000 7f8512a7d640 1 -- 172.21.15.27:0/1859903309 wait complete. 2023-12-19T18:20:14.522 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-12-19T18:20:14.780 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":6510,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.491767678,"mdsmap_epoch":38,"osdmap_epoch":96,"osdmap_epoch_barrier":96,"uptime":19.731501250000001} 2023-12-19T18:20:14.781 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 6510, "mdsmap_epoch": 38, "osdmap_epoch": 96, "osdmap_epoch_barrier": 96, "rank_uptime": 14.491767678, "state": "up:active", "uptime": 19.73150125, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:20:14.781 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-12-19T18:20:15.032 INFO:teuthology.orchestra.run.smithi027.stdout:{"osd_mon_report_interval":"5"} 2023-12-19T18:20:15.032 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-12-19T18:20:15.032 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-12-19T18:20:15.275 INFO:teuthology.orchestra.run.smithi027.stdout:{"mon_tick_interval":"5"} 2023-12-19T18:20:15.276 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-12-19T18:20:15.276 INFO:tasks.cephfs.test_full:pool capacity 557028928, 531MB should be enough to fill it 2023-12-19T18:20:15.276 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:20:15.276 DEBUG:teuthology.orchestra.run.smithi027:> import time 2023-12-19T18:20:15.276 DEBUG:teuthology.orchestra.run.smithi027:> import datetime 2023-12-19T18:20:15.276 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:20:15.277 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:20:15.277 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:20:15.277 DEBUG:teuthology.orchestra.run.smithi027:> # Write some buffered data through before going full, all should be well 2023-12-19T18:20:15.277 DEBUG:teuthology.orchestra.run.smithi027:> print("writing some data through which we expect to succeed") 2023-12-19T18:20:15.277 DEBUG:teuthology.orchestra.run.smithi027:> bytes = 0 2023-12-19T18:20:15.277 DEBUG:teuthology.orchestra.run.smithi027:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-12-19T18:20:15.277 DEBUG:teuthology.orchestra.run.smithi027:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-12-19T18:20:15.277 DEBUG:teuthology.orchestra.run.smithi027:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-12-19T18:20:15.278 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(f) 2023-12-19T18:20:15.278 DEBUG:teuthology.orchestra.run.smithi027:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-12-19T18:20:15.278 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:20:15.278 DEBUG:teuthology.orchestra.run.smithi027:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-12-19T18:20:15.278 DEBUG:teuthology.orchestra.run.smithi027:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-12-19T18:20:15.278 DEBUG:teuthology.orchestra.run.smithi027:> # buffered IO, the error should always only appear from fsync and not 2023-12-19T18:20:15.278 DEBUG:teuthology.orchestra.run.smithi027:> # from write 2023-12-19T18:20:15.278 DEBUG:teuthology.orchestra.run.smithi027:> full = False 2023-12-19T18:20:15.279 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:20:15.279 DEBUG:teuthology.orchestra.run.smithi027:> for n in range(0, int(531 * 1.1)): 2023-12-19T18:20:15.279 DEBUG:teuthology.orchestra.run.smithi027:> try: 2023-12-19T18:20:15.279 DEBUG:teuthology.orchestra.run.smithi027:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-12-19T18:20:15.279 DEBUG:teuthology.orchestra.run.smithi027:> print("wrote bytes via buffered write, moving on to fsync") 2023-12-19T18:20:15.279 DEBUG:teuthology.orchestra.run.smithi027:> except OSError as e: 2023-12-19T18:20:15.279 DEBUG:teuthology.orchestra.run.smithi027:> if True: 2023-12-19T18:20:15.279 DEBUG:teuthology.orchestra.run.smithi027:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-12-19T18:20:15.279 DEBUG:teuthology.orchestra.run.smithi027:> raise 2023-12-19T18:20:15.279 DEBUG:teuthology.orchestra.run.smithi027:> else: 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> full = True 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> break 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> try: 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(f) 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> print("fsync'"'"'ed successfully") 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> except OSError as e: 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> full = True 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> break 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> else: 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:20:15.280 DEBUG:teuthology.orchestra.run.smithi027:> if n > 531 * 0.9: 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> # Be cautious in the last region where we expect to hit 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> time.sleep(15.0) 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> if not full: 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> # close() should not raise an error because we already caught it in 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> # since then because all IOs got cancelled on the full flag. 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> print("calling close") 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> os.close(f) 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> print("close() did not raise error") 2023-12-19T18:20:15.281 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:20:15.282 DEBUG:teuthology.orchestra.run.smithi027:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-12-19T18:20:15.282 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:20:15.953 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:20:15.963 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:20:21.503 INFO:teuthology.orchestra.run.smithi027.stdout:writing some data through which we expect to succeed 2023-12-19T18:20:21.503 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-12-19T18:20:21.503 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.504 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.504 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 1.00 MB 2023-12-19T18:20:21.504 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.504 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.504 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 2.00 MB 2023-12-19T18:20:21.504 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.504 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.504 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 3.00 MB 2023-12-19T18:20:21.504 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.504 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.504 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 4.00 MB 2023-12-19T18:20:21.504 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.505 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.505 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 5.00 MB 2023-12-19T18:20:21.505 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.505 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.505 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 6.00 MB 2023-12-19T18:20:21.505 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.505 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.505 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 7.00 MB 2023-12-19T18:20:21.505 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.505 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.505 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 8.00 MB 2023-12-19T18:20:21.506 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.506 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.506 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 9.00 MB 2023-12-19T18:20:21.506 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.506 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.506 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 10.00 MB 2023-12-19T18:20:21.506 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.506 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.506 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 11.00 MB 2023-12-19T18:20:21.506 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.506 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.506 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 12.00 MB 2023-12-19T18:20:21.506 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.507 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.507 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 13.00 MB 2023-12-19T18:20:21.507 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.507 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.507 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 14.00 MB 2023-12-19T18:20:21.507 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.507 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.507 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 15.00 MB 2023-12-19T18:20:21.507 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.507 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.507 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 16.00 MB 2023-12-19T18:20:21.507 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.507 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.508 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 17.00 MB 2023-12-19T18:20:21.508 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.508 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.508 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 18.00 MB 2023-12-19T18:20:21.508 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.508 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.508 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 19.00 MB 2023-12-19T18:20:21.508 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.508 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.508 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 20.00 MB 2023-12-19T18:20:21.508 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.508 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.508 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 21.00 MB 2023-12-19T18:20:21.509 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.509 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.509 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 22.00 MB 2023-12-19T18:20:21.509 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.509 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.509 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 23.00 MB 2023-12-19T18:20:21.509 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.509 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.509 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 24.00 MB 2023-12-19T18:20:21.509 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.509 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.509 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 25.00 MB 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 26.00 MB 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 27.00 MB 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 28.00 MB 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 29.00 MB 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.510 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.511 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 30.00 MB 2023-12-19T18:20:21.511 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.511 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.511 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 31.00 MB 2023-12-19T18:20:21.511 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.511 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.511 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 32.00 MB 2023-12-19T18:20:21.511 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.511 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.511 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 33.00 MB 2023-12-19T18:20:21.511 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.511 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.511 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 34.00 MB 2023-12-19T18:20:21.512 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.512 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.512 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 35.00 MB 2023-12-19T18:20:21.512 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.512 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.512 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 36.00 MB 2023-12-19T18:20:21.512 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.512 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.512 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 37.00 MB 2023-12-19T18:20:21.512 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.512 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.512 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 38.00 MB 2023-12-19T18:20:21.512 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.513 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.513 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 39.00 MB 2023-12-19T18:20:21.513 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.513 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.513 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 40.00 MB 2023-12-19T18:20:21.513 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.513 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.513 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 41.00 MB 2023-12-19T18:20:21.513 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.513 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.513 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 42.00 MB 2023-12-19T18:20:21.513 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.513 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.514 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 43.00 MB 2023-12-19T18:20:21.514 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.514 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.514 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 44.00 MB 2023-12-19T18:20:21.514 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.514 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.514 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 45.00 MB 2023-12-19T18:20:21.514 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.514 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.514 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 46.00 MB 2023-12-19T18:20:21.514 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.514 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 47.00 MB 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 48.00 MB 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 49.00 MB 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 50.00 MB 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 51.00 MB 2023-12-19T18:20:21.515 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 52.00 MB 2023-12-19T18:20:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 53.00 MB 2023-12-19T18:20:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 54.00 MB 2023-12-19T18:20:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 55.00 MB 2023-12-19T18:20:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.516 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.517 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 56.00 MB 2023-12-19T18:20:21.517 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.517 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.517 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 57.00 MB 2023-12-19T18:20:21.517 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.517 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.517 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 58.00 MB 2023-12-19T18:20:21.517 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.517 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.517 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 59.00 MB 2023-12-19T18:20:21.517 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.518 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.518 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 60.00 MB 2023-12-19T18:20:21.518 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.518 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.518 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 61.00 MB 2023-12-19T18:20:21.518 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.518 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.518 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 62.00 MB 2023-12-19T18:20:21.518 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.518 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.518 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 63.00 MB 2023-12-19T18:20:21.518 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.518 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.519 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 64.00 MB 2023-12-19T18:20:21.519 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.519 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.519 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 65.00 MB 2023-12-19T18:20:21.519 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.519 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.519 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 66.00 MB 2023-12-19T18:20:21.519 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.519 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.519 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 67.00 MB 2023-12-19T18:20:21.519 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.519 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.520 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 68.00 MB 2023-12-19T18:20:21.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.520 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.520 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 69.00 MB 2023-12-19T18:20:21.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.520 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.520 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 70.00 MB 2023-12-19T18:20:21.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.520 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.520 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 71.00 MB 2023-12-19T18:20:21.520 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.520 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.520 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 72.00 MB 2023-12-19T18:20:21.521 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.521 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.521 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 73.00 MB 2023-12-19T18:20:21.521 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.521 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.521 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 74.00 MB 2023-12-19T18:20:21.521 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.521 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.521 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 75.00 MB 2023-12-19T18:20:21.521 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.521 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.521 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 76.00 MB 2023-12-19T18:20:21.522 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.522 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.522 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 77.00 MB 2023-12-19T18:20:21.522 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.522 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.522 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 78.00 MB 2023-12-19T18:20:21.522 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.522 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.522 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 79.00 MB 2023-12-19T18:20:21.522 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:21.522 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:21.522 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 80.00 MB 2023-12-19T18:20:22.933 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 81.00 MB 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 82.00 MB 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 83.00 MB 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 84.00 MB 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.934 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 85.00 MB 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 86.00 MB 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 87.00 MB 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 88.00 MB 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 89.00 MB 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.935 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 90.00 MB 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 91.00 MB 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 92.00 MB 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 93.00 MB 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 94.00 MB 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.936 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 95.00 MB 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 96.00 MB 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 97.00 MB 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 98.00 MB 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 99.00 MB 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.937 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 100.00 MB 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 101.00 MB 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 102.00 MB 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 103.00 MB 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 104.00 MB 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.938 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 105.00 MB 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 106.00 MB 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 107.00 MB 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 108.00 MB 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 109.00 MB 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.939 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 110.00 MB 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 111.00 MB 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 112.00 MB 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 113.00 MB 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 114.00 MB 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.940 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 115.00 MB 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 116.00 MB 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 117.00 MB 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 118.00 MB 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 119.00 MB 2023-12-19T18:20:22.941 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 120.00 MB 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 121.00 MB 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 122.00 MB 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 123.00 MB 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.942 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 124.00 MB 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 125.00 MB 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 126.00 MB 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 127.00 MB 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 128.00 MB 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 129.00 MB 2023-12-19T18:20:22.943 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 130.00 MB 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 131.00 MB 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 132.00 MB 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 133.00 MB 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.944 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 134.00 MB 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 135.00 MB 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 136.00 MB 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 137.00 MB 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 138.00 MB 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.945 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 139.00 MB 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 140.00 MB 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 141.00 MB 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 142.00 MB 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 143.00 MB 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.946 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 144.00 MB 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 145.00 MB 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 146.00 MB 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 147.00 MB 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 148.00 MB 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 149.00 MB 2023-12-19T18:20:22.947 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 150.00 MB 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 151.00 MB 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 152.00 MB 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 153.00 MB 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 154.00 MB 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.948 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 155.00 MB 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 156.00 MB 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 157.00 MB 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 158.00 MB 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 159.00 MB 2023-12-19T18:20:22.949 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:22.950 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:22.950 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 160.00 MB 2023-12-19T18:20:22.950 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.207 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.207 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 161.00 MB 2023-12-19T18:20:24.207 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.207 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.207 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 162.00 MB 2023-12-19T18:20:24.207 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.207 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.207 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 163.00 MB 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 164.00 MB 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 165.00 MB 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 166.00 MB 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 167.00 MB 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.208 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 168.00 MB 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 169.00 MB 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 170.00 MB 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 171.00 MB 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 172.00 MB 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.209 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.210 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 173.00 MB 2023-12-19T18:20:24.210 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.210 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.210 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 174.00 MB 2023-12-19T18:20:24.210 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.210 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.210 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 175.00 MB 2023-12-19T18:20:24.210 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.210 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.210 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 176.00 MB 2023-12-19T18:20:24.210 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.210 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 177.00 MB 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 178.00 MB 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 179.00 MB 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 180.00 MB 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 181.00 MB 2023-12-19T18:20:24.211 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 182.00 MB 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 183.00 MB 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 184.00 MB 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 185.00 MB 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 186.00 MB 2023-12-19T18:20:24.212 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 187.00 MB 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 188.00 MB 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 189.00 MB 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 190.00 MB 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.213 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 191.00 MB 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 192.00 MB 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 193.00 MB 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 194.00 MB 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 195.00 MB 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.214 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 196.00 MB 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 197.00 MB 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 198.00 MB 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 199.00 MB 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 200.00 MB 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.215 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 201.00 MB 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 202.00 MB 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 203.00 MB 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 204.00 MB 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 205.00 MB 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.216 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 206.00 MB 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 207.00 MB 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 208.00 MB 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 209.00 MB 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 210.00 MB 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.217 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 211.00 MB 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 212.00 MB 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 213.00 MB 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 214.00 MB 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 215.00 MB 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 216.00 MB 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 217.00 MB 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 218.00 MB 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 219.00 MB 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 220.00 MB 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 221.00 MB 2023-12-19T18:20:24.219 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.220 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.220 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 222.00 MB 2023-12-19T18:20:24.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.220 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.220 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 223.00 MB 2023-12-19T18:20:24.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.220 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.220 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 224.00 MB 2023-12-19T18:20:24.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.220 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.220 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 225.00 MB 2023-12-19T18:20:24.220 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.220 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.221 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 226.00 MB 2023-12-19T18:20:24.221 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.221 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.221 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 227.00 MB 2023-12-19T18:20:24.221 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.221 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.221 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 228.00 MB 2023-12-19T18:20:24.221 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.221 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.221 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 229.00 MB 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 230.00 MB 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 231.00 MB 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 232.00 MB 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 233.00 MB 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.222 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 234.00 MB 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 235.00 MB 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 236.00 MB 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 237.00 MB 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 238.00 MB 2023-12-19T18:20:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.224 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.224 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 239.00 MB 2023-12-19T18:20:24.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.224 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:24.224 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 240.00 MB 2023-12-19T18:20:24.224 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:24.224 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.332 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 241.00 MB 2023-12-19T18:20:25.332 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.332 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.332 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 242.00 MB 2023-12-19T18:20:25.332 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.332 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.332 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 243.00 MB 2023-12-19T18:20:25.332 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.332 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 244.00 MB 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 245.00 MB 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 246.00 MB 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 247.00 MB 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 248.00 MB 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.333 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 249.00 MB 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 250.00 MB 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 251.00 MB 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 252.00 MB 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 253.00 MB 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.334 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 254.00 MB 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 255.00 MB 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 256.00 MB 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 257.00 MB 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 258.00 MB 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.335 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 259.00 MB 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 260.00 MB 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 261.00 MB 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 262.00 MB 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 263.00 MB 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.336 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 264.00 MB 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 265.00 MB 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 266.00 MB 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 267.00 MB 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 268.00 MB 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.337 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 269.00 MB 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 270.00 MB 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 271.00 MB 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 272.00 MB 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 273.00 MB 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.338 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 274.00 MB 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 275.00 MB 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 276.00 MB 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 277.00 MB 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 278.00 MB 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.339 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 279.00 MB 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 280.00 MB 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 281.00 MB 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 282.00 MB 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 283.00 MB 2023-12-19T18:20:25.340 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 284.00 MB 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 285.00 MB 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 286.00 MB 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 287.00 MB 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.341 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 288.00 MB 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 289.00 MB 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 290.00 MB 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 291.00 MB 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 292.00 MB 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.342 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 293.00 MB 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 294.00 MB 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 295.00 MB 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 296.00 MB 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 297.00 MB 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.343 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 298.00 MB 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 299.00 MB 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 300.00 MB 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 301.00 MB 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 302.00 MB 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.344 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 303.00 MB 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 304.00 MB 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 305.00 MB 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 306.00 MB 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 307.00 MB 2023-12-19T18:20:25.345 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 308.00 MB 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 309.00 MB 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 310.00 MB 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 311.00 MB 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 312.00 MB 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.346 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 313.00 MB 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 314.00 MB 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 315.00 MB 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 316.00 MB 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 317.00 MB 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.347 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.348 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 318.00 MB 2023-12-19T18:20:25.348 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.348 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.348 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 319.00 MB 2023-12-19T18:20:25.348 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.348 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.348 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 320.00 MB 2023-12-19T18:20:25.348 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:25.348 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:25.348 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 321.00 MB 2023-12-19T18:20:26.450 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.450 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.450 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 322.00 MB 2023-12-19T18:20:26.450 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.450 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.450 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 323.00 MB 2023-12-19T18:20:26.450 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.450 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.450 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 324.00 MB 2023-12-19T18:20:26.450 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.450 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 325.00 MB 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 326.00 MB 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 327.00 MB 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 328.00 MB 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 329.00 MB 2023-12-19T18:20:26.451 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 330.00 MB 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 331.00 MB 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 332.00 MB 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 333.00 MB 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 334.00 MB 2023-12-19T18:20:26.452 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 335.00 MB 2023-12-19T18:20:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 336.00 MB 2023-12-19T18:20:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 337.00 MB 2023-12-19T18:20:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 338.00 MB 2023-12-19T18:20:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.453 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 339.00 MB 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 340.00 MB 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 341.00 MB 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 342.00 MB 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 343.00 MB 2023-12-19T18:20:26.454 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 344.00 MB 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 345.00 MB 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 346.00 MB 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 347.00 MB 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 348.00 MB 2023-12-19T18:20:26.455 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 349.00 MB 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 350.00 MB 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 351.00 MB 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 352.00 MB 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 353.00 MB 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.456 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.457 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 354.00 MB 2023-12-19T18:20:26.457 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.457 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.457 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 355.00 MB 2023-12-19T18:20:26.457 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.457 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.457 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 356.00 MB 2023-12-19T18:20:26.457 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.457 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.457 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 357.00 MB 2023-12-19T18:20:26.457 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 358.00 MB 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 359.00 MB 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 360.00 MB 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 361.00 MB 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 362.00 MB 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.458 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 363.00 MB 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 364.00 MB 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 365.00 MB 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 366.00 MB 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 367.00 MB 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.459 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 368.00 MB 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 369.00 MB 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 370.00 MB 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 371.00 MB 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 372.00 MB 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.460 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 373.00 MB 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 374.00 MB 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 375.00 MB 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 376.00 MB 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 377.00 MB 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.461 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 378.00 MB 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 379.00 MB 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 380.00 MB 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 381.00 MB 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 382.00 MB 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.462 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 383.00 MB 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 384.00 MB 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 385.00 MB 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 386.00 MB 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 387.00 MB 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.463 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 388.00 MB 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 389.00 MB 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 390.00 MB 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 391.00 MB 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 392.00 MB 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.464 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 393.00 MB 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 394.00 MB 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 395.00 MB 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 396.00 MB 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 397.00 MB 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.465 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 398.00 MB 2023-12-19T18:20:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 399.00 MB 2023-12-19T18:20:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 400.00 MB 2023-12-19T18:20:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:26.466 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 401.00 MB 2023-12-19T18:20:28.684 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:20:28.684+0000 7f8f3ae61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-12-19T18:20:36.631 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:20:36.630+0000 7f8f3ae61640 -1 log_channel(cluster) log [ERR] : Health check update: 2 full osd(s) (OSD_FULL) 2023-12-19T18:20:42.879 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.879 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.879 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 402.00 MB 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 403.00 MB 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 404.00 MB 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 405.00 MB 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 406.00 MB 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.880 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 407.00 MB 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 408.00 MB 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 409.00 MB 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 410.00 MB 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 411.00 MB 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 412.00 MB 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 413.00 MB 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 414.00 MB 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 415.00 MB 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 416.00 MB 2023-12-19T18:20:42.882 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 417.00 MB 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 418.00 MB 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 419.00 MB 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 420.00 MB 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.883 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 421.00 MB 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 422.00 MB 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 423.00 MB 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 424.00 MB 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 425.00 MB 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.884 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 426.00 MB 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 427.00 MB 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 428.00 MB 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 429.00 MB 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 430.00 MB 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 431.00 MB 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 432.00 MB 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 433.00 MB 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 434.00 MB 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 435.00 MB 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.886 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 436.00 MB 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 437.00 MB 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 438.00 MB 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 439.00 MB 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 440.00 MB 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.887 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 441.00 MB 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 442.00 MB 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 443.00 MB 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 444.00 MB 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 445.00 MB 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.888 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 446.00 MB 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 447.00 MB 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 448.00 MB 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 449.00 MB 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 450.00 MB 2023-12-19T18:20:42.889 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 451.00 MB 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 452.00 MB 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 453.00 MB 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 454.00 MB 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 455.00 MB 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.890 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 456.00 MB 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 457.00 MB 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 458.00 MB 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 459.00 MB 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 460.00 MB 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.891 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 461.00 MB 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 462.00 MB 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 463.00 MB 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 464.00 MB 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 465.00 MB 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.892 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.893 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 466.00 MB 2023-12-19T18:20:42.893 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.893 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.893 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 467.00 MB 2023-12-19T18:20:42.893 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.893 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.893 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 468.00 MB 2023-12-19T18:20:42.893 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.893 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.893 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 469.00 MB 2023-12-19T18:20:42.893 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.893 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.893 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 470.00 MB 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 471.00 MB 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 472.00 MB 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 473.00 MB 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 474.00 MB 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.894 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.895 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 475.00 MB 2023-12-19T18:20:42.895 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.895 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.895 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 476.00 MB 2023-12-19T18:20:42.895 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.895 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.895 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 477.00 MB 2023-12-19T18:20:42.895 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.895 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.895 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 478.00 MB 2023-12-19T18:20:42.895 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.895 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:20:42.895 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 479.00 MB 2023-12-19T18:20:42.896 INFO:teuthology.orchestra.run.smithi027.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-12-19T18:20:42.896 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:20:42.896 INFO:teuthology.orchestra.run.smithi027.stdout:Reached fullness after 480.00 MB 2023-12-19T18:20:42.896 INFO:teuthology.orchestra.run.smithi027.stdout:calling close 2023-12-19T18:20:42.896 INFO:teuthology.orchestra.run.smithi027.stdout:close() did not raise error 2023-12-19T18:20:43.066 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:20:43.066 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:20:43.209 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:20:43.335 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:20:43.335 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:20:43.335 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:20:43.400 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:20:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.629+0000 7fbd5bfff640 1 Processor -- start 2023-12-19T18:20:43.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.630+0000 7fbd5bfff640 1 -- start start 2023-12-19T18:20:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.632+0000 7fbd5bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbd5c0c8f90 0x7fbd5c160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.632+0000 7fbd5bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd5c160e40 0x7fbd5c0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.632+0000 7fbd5bfff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd5c1612d0 con 0x7fbd5c0cb300 2023-12-19T18:20:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.632+0000 7fbd5bfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd5c0c4480 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.632+0000 7fbd5bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c1468c0 con 0x7fbd5c160e40 2023-12-19T18:20:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.633+0000 7fbd53fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbd5c0c8f90 0x7fbd5c160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.633+0000 7fbd5b7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd5c160e40 0x7fbd5c0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.633+0000 7fbd5affd640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fbd5c0cb300 0x7fbd5c0cb6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:41238/0 (socket says 172.21.15.27:41238) 2023-12-19T18:20:43.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.633+0000 7fbd5affd640 1 -- 172.21.15.27:0/1984481340 learned_addr learned my addr 172.21.15.27:0/1984481340 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:43.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.633+0000 7fbd5a7fc640 1 -- 172.21.15.27:0/1984481340 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1953795054 0 0) 0x7fbd4c001110 con 0x7fbd5c0cb300 2023-12-19T18:20:43.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.634+0000 7fbd5a7fc640 1 -- 172.21.15.27:0/1984481340 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbd48003520 con 0x7fbd5c0cb300 2023-12-19T18:20:43.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.634+0000 7fbd5b7fe640 1 -- 172.21.15.27:0/1984481340 >> v2:172.21.15.192:3300/0 conn(0x7fbd5c0c8f90 msgr2=0x7fbd5c160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:43.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.634+0000 7fbd5b7fe640 1 --2- 172.21.15.27:0/1984481340 >> v2:172.21.15.192:3300/0 conn(0x7fbd5c0c8f90 0x7fbd5c160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:43.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.634+0000 7fbd5b7fe640 1 -- 172.21.15.27:0/1984481340 >> v1:172.21.15.192:6789/0 conn(0x7fbd5c0cb300 legacy=0x7fbd5c0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:43.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.634+0000 7fbd5b7fe640 1 -- 172.21.15.27:0/1984481340 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd5c145d60 con 0x7fbd5c160e40 2023-12-19T18:20:43.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.634+0000 7fbd53fff640 1 --2- 172.21.15.27:0/1984481340 >> v2:172.21.15.192:3300/0 conn(0x7fbd5c0c8f90 0x7fbd5c160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:20:43.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.635+0000 7fbd5b7fe640 1 --2- 172.21.15.27:0/1984481340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd5c160e40 0x7fbd5c0cd090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009dc0 tx=0x7fbd5402be80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bc7bbac3cc5e383 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:43.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.636+0000 7fbd5a7fc640 1 -- 172.21.15.27:0/1984481340 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd5402cc30 con 0x7fbd5c160e40 2023-12-19T18:20:43.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.636+0000 7fbd5a7fc640 1 -- 172.21.15.27:0/1984481340 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd5402cd90 con 0x7fbd5c160e40 2023-12-19T18:20:43.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.636+0000 7fbd5a7fc640 1 -- 172.21.15.27:0/1984481340 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd54034640 con 0x7fbd5c160e40 2023-12-19T18:20:43.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.636+0000 7fbd5bfff640 1 -- 172.21.15.27:0/1984481340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd5c160e40 msgr2=0x7fbd5c0cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:43.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.636+0000 7fbd5bfff640 1 --2- 172.21.15.27:0/1984481340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd5c160e40 0x7fbd5c0cd090 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009dc0 tx=0x7fbd5402be80 comp rx=0 tx=0).stop 2023-12-19T18:20:43.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.637+0000 7fbd5bfff640 1 -- 172.21.15.27:0/1984481340 shutdown_connections 2023-12-19T18:20:43.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.637+0000 7fbd5bfff640 1 --2- 172.21.15.27:0/1984481340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd5c160e40 0x7fbd5c0cd090 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:43.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.637+0000 7fbd5bfff640 1 --2- 172.21.15.27:0/1984481340 >> v2:172.21.15.192:3300/0 conn(0x7fbd5c0c8f90 0x7fbd5c160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:43.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.637+0000 7fbd5bfff640 1 -- 172.21.15.27:0/1984481340 >> 172.21.15.27:0/1984481340 conn(0x7fbd5c069c70 msgr2=0x7fbd5c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:43.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.637+0000 7fbd5bfff640 1 -- 172.21.15.27:0/1984481340 shutdown_connections 2023-12-19T18:20:43.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.637+0000 7fbd5bfff640 1 -- 172.21.15.27:0/1984481340 wait complete. 2023-12-19T18:20:43.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.639+0000 7fbd5bfff640 1 Processor -- start 2023-12-19T18:20:43.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.640+0000 7fbd5bfff640 1 -- start start 2023-12-19T18:20:43.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.641+0000 7fbd5bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd5c0c8f90 0x7fbd5c1425c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:43.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.641+0000 7fbd5bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd5c0cb300 0x7fbd5c0ce940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:43.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.641+0000 7fbd5bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd5c160e40 0x7fbd5c0cee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:43.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.641+0000 7fbd5bfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c0afce0 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.641+0000 7fbd5bfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c142b00 con 0x7fbd5c0cb300 2023-12-19T18:20:43.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.641+0000 7fbd5bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd5c0af060 con 0x7fbd5c160e40 2023-12-19T18:20:43.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.641+0000 7fbd5affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd5c0c8f90 0x7fbd5c1425c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:43.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.641+0000 7fbd5affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd5c0c8f90 0x7fbd5c1425c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49284/0 (socket says 172.21.15.27:49284) 2023-12-19T18:20:43.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.641+0000 7fbd5affd640 1 -- 172.21.15.27:0/4271501189 learned_addr learned my addr 172.21.15.27:0/4271501189 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:43.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.642+0000 7fbd53fff640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd5c0cb300 0x7fbd5c0ce940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:43.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.642+0000 7fbd5b7fe640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd5c160e40 0x7fbd5c0cee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:43.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.642+0000 7fbd5affd640 1 -- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd5c160e40 msgr2=0x7fbd5c0cee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:43.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.642+0000 7fbd5affd640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd5c160e40 0x7fbd5c0cee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:43.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.642+0000 7fbd5affd640 1 -- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd5c0cb300 msgr2=0x7fbd5c0ce940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:43.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.642+0000 7fbd5affd640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd5c0cb300 0x7fbd5c0ce940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:43.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.642+0000 7fbd5affd640 1 -- 172.21.15.27:0/4271501189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd44009400 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.642+0000 7fbd5affd640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd5c0c8f90 0x7fbd5c1425c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c700 tx=0x7fbd4c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:43.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.647+0000 7fbd537fe640 1 -- 172.21.15.27:0/4271501189 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c005ce0 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.647+0000 7fbd537fe640 1 -- 172.21.15.27:0/4271501189 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c005e40 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.647+0000 7fbd537fe640 1 -- 172.21.15.27:0/4271501189 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c007290 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.647+0000 7fbd5bfff640 1 -- 172.21.15.27:0/4271501189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd54009a70 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.647+0000 7fbd5bfff640 1 -- 172.21.15.27:0/4271501189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd5c0af620 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.648+0000 7fbd5bfff640 1 -- 172.21.15.27:0/4271501189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd5c0afe20 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.650+0000 7fbd537fe640 1 -- 172.21.15.27:0/4271501189 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbd4c0073f0 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.650+0000 7fbd537fe640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbd3c06a4b0 0x7fbd3c06c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:43.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.651+0000 7fbd537fe640 1 -- 172.21.15.27:0/4271501189 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbd4c087d20 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.651+0000 7fbd537fe640 1 -- 172.21.15.27:0/4271501189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=101}) v3 -- 0x7fbd3c0720a0 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.652+0000 7fbd53fff640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbd3c06a4b0 0x7fbd3c06c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:43.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.654+0000 7fbd53fff640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbd3c06a4b0 0x7fbd3c06c970 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0c8ef0 tx=0x7fbd44008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:43.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.655+0000 7fbd537fe640 1 -- 172.21.15.27:0/4271501189 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbd5c0afe20 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.851+0000 7fbd5bfff640 1 -- 172.21.15.27:0/4271501189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd5c1612d0 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.853+0000 7fbd537fe640 1 -- 172.21.15.27:0/4271501189 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) v1 ==== 74+0+16228 (secure 0 0 0) 0x7fbd5c1612d0 con 0x7fbd5c0c8f90 2023-12-19T18:20:43.854 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:43.854 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":100,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:20:43.640322+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:19:58.591127+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T18:19:59.313961+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["backfillfull","exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["backfillfull","exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.19","mappings":[{"from":2,"to":7}]},{"pgid":"11.29","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:20:43.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.856+0000 7fbd5bfff640 1 -- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbd3c06a4b0 msgr2=0x7fbd3c06c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:43.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.856+0000 7fbd5bfff640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbd3c06a4b0 0x7fbd3c06c970 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fbd5c0c8ef0 tx=0x7fbd44008040 comp rx=0 tx=0).stop 2023-12-19T18:20:43.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.856+0000 7fbd5bfff640 1 -- 172.21.15.27:0/4271501189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd5c0c8f90 msgr2=0x7fbd5c1425c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:43.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.856+0000 7fbd5bfff640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd5c0c8f90 0x7fbd5c1425c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c700 tx=0x7fbd4c00cb00 comp rx=0 tx=0).stop 2023-12-19T18:20:43.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.856+0000 7fbd5bfff640 1 -- 172.21.15.27:0/4271501189 shutdown_connections 2023-12-19T18:20:43.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.856+0000 7fbd5bfff640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbd3c06a4b0 0x7fbd3c06c970 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:43.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.856+0000 7fbd5bfff640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd5c160e40 0x7fbd5c0cee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:43.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.856+0000 7fbd5bfff640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd5c0cb300 0x7fbd5c0ce940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:43.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.856+0000 7fbd5bfff640 1 --2- 172.21.15.27:0/4271501189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd5c0c8f90 0x7fbd5c1425c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:43.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.856+0000 7fbd5bfff640 1 -- 172.21.15.27:0/4271501189 >> 172.21.15.27:0/4271501189 conn(0x7fbd5c069c70 msgr2=0x7fbd5c1606a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:43.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.856+0000 7fbd5bfff640 1 -- 172.21.15.27:0/4271501189 shutdown_connections 2023-12-19T18:20:43.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:43.856+0000 7fbd5bfff640 1 -- 172.21.15.27:0/4271501189 wait complete. 2023-12-19T18:20:43.877 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:19:58.591127+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:19:59.313961+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:20:43.877 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:20:44.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.106+0000 7f32e4ce7640 1 Processor -- start 2023-12-19T18:20:44.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.106+0000 7f32e4ce7640 1 -- start start 2023-12-19T18:20:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.108+0000 7f32e4ce7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32e00ce210 0x7f32e00cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.108+0000 7f32e4ce7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e00c9880 con 0x7f32e015a0b0 2023-12-19T18:20:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.108+0000 7f32e4ce7640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e00c9b20 con 0x7f32e006cee0 2023-12-19T18:20:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.108+0000 7f32e4ce7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e00c9de0 con 0x7f32e00ce210 2023-12-19T18:20:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.108+0000 7f32de59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f32e015a0b0 0x7f32e00cd400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54356/0 (socket says 172.21.15.27:54356) 2023-12-19T18:20:44.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.108+0000 7f32de59c640 1 -- 172.21.15.27:0/2898631753 learned_addr learned my addr 172.21.15.27:0/2898631753 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32d7fff640 1 --2- 172.21.15.27:0/2898631753 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32e00ce210 0x7f32e00cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 383880766 0 0) 0x7f32e00c9880 con 0x7f32e015a0b0 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32cc003520 con 0x7f32e015a0b0 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3974382791 0 0) 0x7f32cc003520 con 0x7f32e015a0b0 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32e00ce210 msgr2=0x7f32e00cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 --2- 172.21.15.27:0/2898631753 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32e00ce210 0x7f32e00cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 >> v1:172.21.15.192:6789/0 conn(0x7f32e006cee0 legacy=0x7f32e0070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e00cc920 con 0x7f32e015a0b0 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f32d0001670 con 0x7f32e015a0b0 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 >> v1:172.21.15.27:6789/0 conn(0x7f32e015a0b0 legacy=0x7f32e00cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 --2- 172.21.15.27:0/2898631753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32cc0039a0 0x7f32cc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 --2- 172.21.15.27:0/2898631753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32cc0047c0 0x7f32cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 --2- 172.21.15.27:0/2898631753 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32e00ce210 0x7f32cc005210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e00c9de0 con 0x7f32cc0047c0 2023-12-19T18:20:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc00bb10 con 0x7f32cc0039a0 2023-12-19T18:20:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.109+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32cc00dee0 con 0x7f32e00ce210 2023-12-19T18:20:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.110+0000 7f32d7fff640 1 --2- 172.21.15.27:0/2898631753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32cc0047c0 0x7f32cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.110+0000 7f32de59c640 1 --2- 172.21.15.27:0/2898631753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32cc0039a0 0x7f32cc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.110+0000 7f32ded9d640 1 --2- 172.21.15.27:0/2898631753 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32e00ce210 0x7f32cc005210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.110+0000 7f32d7fff640 1 -- 172.21.15.27:0/2898631753 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32e00ce210 msgr2=0x7f32cc005210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.110+0000 7f32d7fff640 1 --2- 172.21.15.27:0/2898631753 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32e00ce210 0x7f32cc005210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.110+0000 7f32d7fff640 1 -- 172.21.15.27:0/2898631753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32cc0039a0 msgr2=0x7f32cc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.110+0000 7f32d7fff640 1 --2- 172.21.15.27:0/2898631753 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32cc0039a0 0x7f32cc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.110+0000 7f32d7fff640 1 -- 172.21.15.27:0/2898631753 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32cc003520 con 0x7f32cc0047c0 2023-12-19T18:20:44.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.110+0000 7f32ded9d640 1 --2- 172.21.15.27:0/2898631753 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32e00ce210 0x7f32cc005210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.112+0000 7f32d7fff640 1 --2- 172.21.15.27:0/2898631753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32cc0047c0 0x7f32cc004c40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f32c800ba10 tx=0x7f32cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439d13626c540c47 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.112+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8032630 con 0x7f32cc0047c0 2023-12-19T18:20:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.112+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32c8032790 con 0x7f32cc0047c0 2023-12-19T18:20:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.112+0000 7f32ddd9b640 1 -- 172.21.15.27:0/2898631753 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32c8032b20 con 0x7f32cc0047c0 2023-12-19T18:20:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.112+0000 7f32e4ce7640 1 -- 172.21.15.27:0/2898631753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32cc0047c0 msgr2=0x7f32cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.112+0000 7f32e4ce7640 1 --2- 172.21.15.27:0/2898631753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32cc0047c0 0x7f32cc004c40 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f32c800ba10 tx=0x7f32cc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.112+0000 7f32ded9d640 1 -- 172.21.15.27:0/2898631753 reap_dead start 2023-12-19T18:20:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.113+0000 7f32e4ce7640 1 -- 172.21.15.27:0/2898631753 shutdown_connections 2023-12-19T18:20:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.113+0000 7f32e4ce7640 1 -- 172.21.15.27:0/2898631753 >> 172.21.15.27:0/2898631753 conn(0x7f32e00cebd0 msgr2=0x7f32e00cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:44.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.113+0000 7f32e4ce7640 1 -- 172.21.15.27:0/2898631753 shutdown_connections 2023-12-19T18:20:44.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.113+0000 7f32e4ce7640 1 -- 172.21.15.27:0/2898631753 wait complete. 2023-12-19T18:20:44.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.115+0000 7f32e4ce7640 1 Processor -- start 2023-12-19T18:20:44.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.115+0000 7f32e4ce7640 1 -- start start 2023-12-19T18:20:44.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32e4ce7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32cc0047c0 0x7f32e00c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32e4ce7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32e00c6cf0 0x7f32e01581e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32e4ce7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32e006bdd0 0x7f32e006c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32e4ce7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e00ad1e0 con 0x7f32cc0047c0 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32e4ce7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e006c790 con 0x7f32e006bdd0 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32e4ce7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e00c7150 con 0x7f32e00c6cf0 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32de59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32cc0047c0 0x7f32e00c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32de59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32cc0047c0 0x7f32e00c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49294/0 (socket says 172.21.15.27:49294) 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32de59c640 1 -- 172.21.15.27:0/823566445 learned_addr learned my addr 172.21.15.27:0/823566445 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32de59c640 1 -- 172.21.15.27:0/823566445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32e00c6cf0 msgr2=0x7f32e01581e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32de59c640 1 --2- 172.21.15.27:0/823566445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32e00c6cf0 0x7f32e01581e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32de59c640 1 -- 172.21.15.27:0/823566445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32e006bdd0 msgr2=0x7f32e006c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32de59c640 1 --2- 172.21.15.27:0/823566445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32e006bdd0 0x7f32e006c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.116+0000 7f32de59c640 1 -- 172.21.15.27:0/823566445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e00c7420 con 0x7f32cc0047c0 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.117+0000 7f32de59c640 1 --2- 172.21.15.27:0/823566445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32cc0047c0 0x7f32e00c8c40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f32d000e800 tx=0x7f32d000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:44.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.117+0000 7f32d6ffd640 1 -- 172.21.15.27:0/823566445 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d000c870 con 0x7f32cc0047c0 2023-12-19T18:20:44.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.117+0000 7f32d6ffd640 1 -- 172.21.15.27:0/823566445 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32d000c9d0 con 0x7f32cc0047c0 2023-12-19T18:20:44.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.117+0000 7f32d6ffd640 1 -- 172.21.15.27:0/823566445 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32d000cd80 con 0x7f32cc0047c0 2023-12-19T18:20:44.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.117+0000 7f32e4ce7640 1 -- 172.21.15.27:0/823566445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e00c7700 con 0x7f32cc0047c0 2023-12-19T18:20:44.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.117+0000 7f32e4ce7640 1 -- 172.21.15.27:0/823566445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e00c7cc0 con 0x7f32cc0047c0 2023-12-19T18:20:44.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.118+0000 7f32e4ce7640 1 -- 172.21.15.27:0/823566445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e0068e90 con 0x7f32cc0047c0 2023-12-19T18:20:44.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.119+0000 7f32d6ffd640 1 -- 172.21.15.27:0/823566445 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f32d001d460 con 0x7f32cc0047c0 2023-12-19T18:20:44.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.120+0000 7f32d6ffd640 1 --2- 172.21.15.27:0/823566445 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f32c006a430 0x7f32c006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.120+0000 7f32d6ffd640 1 -- 172.21.15.27:0/823566445 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f32d0088ac0 con 0x7f32cc0047c0 2023-12-19T18:20:44.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.121+0000 7f32d6ffd640 1 -- 172.21.15.27:0/823566445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=101}) v3 -- 0x7f32c0072020 con 0x7f32cc0047c0 2023-12-19T18:20:44.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.121+0000 7f32d7fff640 1 --2- 172.21.15.27:0/823566445 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f32c006a430 0x7f32c006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:44.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.122+0000 7f32d7fff640 1 --2- 172.21.15.27:0/823566445 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f32c006a430 0x7f32c006c8f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f32c8037000 tx=0x7f32c8032000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:44.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.124+0000 7f32d6ffd640 1 -- 172.21.15.27:0/823566445 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f32d0053f40 con 0x7f32cc0047c0 2023-12-19T18:20:44.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.316+0000 7f32e4ce7640 1 -- 172.21.15.27:0/823566445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f32e00acf40 con 0x7f32cc0047c0 2023-12-19T18:20:44.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.317+0000 7f32d6ffd640 1 -- 172.21.15.27:0/823566445 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) v1 ==== 93+0+10 (secure 0 0 0) 0x7f32d0001560 con 0x7f32cc0047c0 2023-12-19T18:20:44.318 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:20:44.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.320+0000 7f32e4ce7640 1 -- 172.21.15.27:0/823566445 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f32c006a430 msgr2=0x7f32c006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:44.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.320+0000 7f32e4ce7640 1 --2- 172.21.15.27:0/823566445 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f32c006a430 0x7f32c006c8f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f32c8037000 tx=0x7f32c8032000 comp rx=0 tx=0).stop 2023-12-19T18:20:44.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.320+0000 7f32e4ce7640 1 -- 172.21.15.27:0/823566445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32cc0047c0 msgr2=0x7f32e00c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:44.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.320+0000 7f32e4ce7640 1 --2- 172.21.15.27:0/823566445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32cc0047c0 0x7f32e00c8c40 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f32d000e800 tx=0x7f32d000ecc0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.320+0000 7f32e4ce7640 1 -- 172.21.15.27:0/823566445 shutdown_connections 2023-12-19T18:20:44.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.320+0000 7f32e4ce7640 1 --2- 172.21.15.27:0/823566445 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f32c006a430 0x7f32c006c8f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.320+0000 7f32e4ce7640 1 --2- 172.21.15.27:0/823566445 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f32e006bdd0 0x7f32e006c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.320+0000 7f32e4ce7640 1 --2- 172.21.15.27:0/823566445 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f32e00c6cf0 0x7f32e01581e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.320+0000 7f32e4ce7640 1 --2- 172.21.15.27:0/823566445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32cc0047c0 0x7f32e00c8c40 secure :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f32d000e800 tx=0x7f32d000ecc0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.320+0000 7f32e4ce7640 1 -- 172.21.15.27:0/823566445 >> 172.21.15.27:0/823566445 conn(0x7f32e00cebd0 msgr2=0x7f32e015b2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:44.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.320+0000 7f32e4ce7640 1 -- 172.21.15.27:0/823566445 shutdown_connections 2023-12-19T18:20:44.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.320+0000 7f32e4ce7640 1 -- 172.21.15.27:0/823566445 wait complete. 2023-12-19T18:20:44.341 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:20:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.582+0000 7fe547fff640 1 Processor -- start 2023-12-19T18:20:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.582+0000 7fe547fff640 1 -- start start 2023-12-19T18:20:44.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.583+0000 7fe547fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe54815e9d0 0x7fe5480cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.583+0000 7fe547fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe548152070 0x7fe5480cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.583+0000 7fe547fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe54815b3e0 con 0x7fe548151c90 2023-12-19T18:20:44.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.583+0000 7fe547fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe54815eed0 con 0x7fe54815e9d0 2023-12-19T18:20:44.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.583+0000 7fe547fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe54815f1b0 con 0x7fe548152070 2023-12-19T18:20:44.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.584+0000 7fe5477fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe548152070 0x7fe5480cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:44.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.584+0000 7fe5467fc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe54815e9d0 0x7fe5480cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:44.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.584+0000 7fe5467fc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe54815e9d0 0x7fe5480cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:45958/0 (socket says 172.21.15.27:45958) 2023-12-19T18:20:44.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.584+0000 7fe5467fc640 1 -- 172.21.15.27:0/1050715557 learned_addr learned my addr 172.21.15.27:0/1050715557 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:44.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.585+0000 7fe5467fc640 1 -- 172.21.15.27:0/1050715557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe548152070 msgr2=0x7fe5480cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:44.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.585+0000 7fe5467fc640 1 --2- 172.21.15.27:0/1050715557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe548152070 0x7fe5480cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.585+0000 7fe5467fc640 1 -- 172.21.15.27:0/1050715557 >> v1:172.21.15.192:6789/0 conn(0x7fe548151c90 legacy=0x7fe54806b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:44.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.585+0000 7fe5467fc640 1 -- 172.21.15.27:0/1050715557 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe54815f490 con 0x7fe54815e9d0 2023-12-19T18:20:44.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.585+0000 7fe5477fe640 1 --2- 172.21.15.27:0/1050715557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe548152070 0x7fe5480cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:20:44.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.586+0000 7fe5467fc640 1 --2- 172.21.15.27:0/1050715557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe54815e9d0 0x7fe5480cee40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe5300098a0 tx=0x7fe53002b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd3cd7cad60610fb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.587+0000 7fe545ffb640 1 -- 172.21.15.27:0/1050715557 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530031640 con 0x7fe54815e9d0 2023-12-19T18:20:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.587+0000 7fe545ffb640 1 -- 172.21.15.27:0/1050715557 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5300317a0 con 0x7fe54815e9d0 2023-12-19T18:20:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.587+0000 7fe545ffb640 1 -- 172.21.15.27:0/1050715557 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe530031b30 con 0x7fe54815e9d0 2023-12-19T18:20:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.587+0000 7fe547fff640 1 -- 172.21.15.27:0/1050715557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe54815e9d0 msgr2=0x7fe5480cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.587+0000 7fe547fff640 1 --2- 172.21.15.27:0/1050715557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe54815e9d0 0x7fe5480cee40 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe5300098a0 tx=0x7fe53002b8f0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.588+0000 7fe547fff640 1 -- 172.21.15.27:0/1050715557 shutdown_connections 2023-12-19T18:20:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.588+0000 7fe547fff640 1 --2- 172.21.15.27:0/1050715557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe548152070 0x7fe5480cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.588+0000 7fe547fff640 1 --2- 172.21.15.27:0/1050715557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe54815e9d0 0x7fe5480cee40 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.588+0000 7fe547fff640 1 -- 172.21.15.27:0/1050715557 >> 172.21.15.27:0/1050715557 conn(0x7fe54806a060 msgr2=0x7fe54806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.588+0000 7fe547fff640 1 -- 172.21.15.27:0/1050715557 shutdown_connections 2023-12-19T18:20:44.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.588+0000 7fe547fff640 1 -- 172.21.15.27:0/1050715557 wait complete. 2023-12-19T18:20:44.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.590+0000 7fe547fff640 1 Processor -- start 2023-12-19T18:20:44.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.590+0000 7fe547fff640 1 -- start start 2023-12-19T18:20:44.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.591+0000 7fe547fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe548151c90 0x7fe548148ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.591+0000 7fe547fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe548152070 0x7fe548152e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.591+0000 7fe547fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe54815e9d0 0x7fe548154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.591+0000 7fe547fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe548149220 con 0x7fe54815e9d0 2023-12-19T18:20:44.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.591+0000 7fe547fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5481525a0 con 0x7fe548152070 2023-12-19T18:20:44.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.591+0000 7fe547fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe548152880 con 0x7fe548151c90 2023-12-19T18:20:44.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.591+0000 7fe5477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe54815e9d0 0x7fe548154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:44.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.591+0000 7fe5477fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe54815e9d0 0x7fe548154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49310/0 (socket says 172.21.15.27:49310) 2023-12-19T18:20:44.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.591+0000 7fe5477fe640 1 -- 172.21.15.27:0/3014732670 learned_addr learned my addr 172.21.15.27:0/3014732670 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:44.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.592+0000 7fe5467fc640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe548152070 0x7fe548152e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:44.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.592+0000 7fe546ffd640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe548151c90 0x7fe548148ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.592+0000 7fe5477fe640 1 -- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe548151c90 msgr2=0x7fe548148ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.592+0000 7fe5477fe640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe548151c90 0x7fe548148ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.592+0000 7fe5477fe640 1 -- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe548152070 msgr2=0x7fe548152e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.592+0000 7fe5477fe640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe548152070 0x7fe548152e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.592+0000 7fe5477fe640 1 -- 172.21.15.27:0/3014732670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe540009400 con 0x7fe54815e9d0 2023-12-19T18:20:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.592+0000 7fe5467fc640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe548152070 0x7fe548152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.592+0000 7fe5477fe640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe54815e9d0 0x7fe548154b50 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe54000e880 tx=0x7fe54000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:44.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.592+0000 7fe53ffff640 1 -- 172.21.15.27:0/3014732670 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe540009650 con 0x7fe54815e9d0 2023-12-19T18:20:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.592+0000 7fe53ffff640 1 -- 172.21.15.27:0/3014732670 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe540004020 con 0x7fe54815e9d0 2023-12-19T18:20:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.592+0000 7fe53ffff640 1 -- 172.21.15.27:0/3014732670 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5400043b0 con 0x7fe54815e9d0 2023-12-19T18:20:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.592+0000 7fe547fff640 1 -- 172.21.15.27:0/3014732670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5300094d0 con 0x7fe54815e9d0 2023-12-19T18:20:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.593+0000 7fe547fff640 1 -- 172.21.15.27:0/3014732670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe548149e10 con 0x7fe54815e9d0 2023-12-19T18:20:44.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.594+0000 7fe547fff640 1 -- 172.21.15.27:0/3014732670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54804e840 con 0x7fe54815e9d0 2023-12-19T18:20:44.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.595+0000 7fe53ffff640 1 -- 172.21.15.27:0/3014732670 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe54000c940 con 0x7fe54815e9d0 2023-12-19T18:20:44.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.596+0000 7fe53ffff640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe52806a430 0x7fe52806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:44.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.596+0000 7fe53ffff640 1 -- 172.21.15.27:0/3014732670 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fe540088770 con 0x7fe54815e9d0 2023-12-19T18:20:44.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.596+0000 7fe53ffff640 1 -- 172.21.15.27:0/3014732670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=101}) v3 -- 0x7fe528072020 con 0x7fe54815e9d0 2023-12-19T18:20:44.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.596+0000 7fe546ffd640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe52806a430 0x7fe52806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:44.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.598+0000 7fe546ffd640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe52806a430 0x7fe52806c8f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe5480b2bd0 tx=0x7fe538009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:44.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.600+0000 7fe53ffff640 1 -- 172.21.15.27:0/3014732670 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe540017060 con 0x7fe54815e9d0 2023-12-19T18:20:44.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.672+0000 7fe53ffff640 1 -- 172.21.15.27:0/3014732670 <== mon.0 v2:172.21.15.27:3300/0 7 ==== osd_map(101..101 src has 1..101) v4 ==== 304+0+0 (secure 0 0 0) 0x7fe540058170 con 0x7fe54815e9d0 2023-12-19T18:20:44.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.672+0000 7fe53ffff640 1 -- 172.21.15.27:0/3014732670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=102}) v3 -- 0x7fe528072860 con 0x7fe54815e9d0 2023-12-19T18:20:44.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.810+0000 7fe547fff640 1 -- 172.21.15.27:0/3014732670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe54806cda0 con 0x7fe54815e9d0 2023-12-19T18:20:44.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.811+0000 7fe53ffff640 1 -- 172.21.15.27:0/3014732670 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe540020020 con 0x7fe54815e9d0 2023-12-19T18:20:44.811 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:20:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.814+0000 7fe547fff640 1 -- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe52806a430 msgr2=0x7fe52806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.814+0000 7fe547fff640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe52806a430 0x7fe52806c8f0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe5480b2bd0 tx=0x7fe538009040 comp rx=0 tx=0).stop 2023-12-19T18:20:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.814+0000 7fe547fff640 1 -- 172.21.15.27:0/3014732670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe54815e9d0 msgr2=0x7fe548154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.814+0000 7fe547fff640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe54815e9d0 0x7fe548154b50 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe54000e880 tx=0x7fe54000ed40 comp rx=0 tx=0).stop 2023-12-19T18:20:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.814+0000 7fe547fff640 1 -- 172.21.15.27:0/3014732670 shutdown_connections 2023-12-19T18:20:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.814+0000 7fe547fff640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe52806a430 0x7fe52806c8f0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.814+0000 7fe547fff640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe54815e9d0 0x7fe548154b50 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.814+0000 7fe547fff640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe548152070 0x7fe548152e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.814+0000 7fe547fff640 1 --2- 172.21.15.27:0/3014732670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe548151c90 0x7fe548148ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.814+0000 7fe547fff640 1 -- 172.21.15.27:0/3014732670 >> 172.21.15.27:0/3014732670 conn(0x7fe54806a060 msgr2=0x7fe54806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.814+0000 7fe547fff640 1 -- 172.21.15.27:0/3014732670 shutdown_connections 2023-12-19T18:20:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:44.814+0000 7fe547fff640 1 -- 172.21.15.27:0/3014732670 wait complete. 2023-12-19T18:20:44.835 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:20:45.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.066+0000 7f800a012640 1 Processor -- start 2023-12-19T18:20:45.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.067+0000 7f800a012640 1 -- start start 2023-12-19T18:20:45.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.068+0000 7f800a012640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f800415b1a0 0x7f800406f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:45.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.068+0000 7f800a012640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8004071420 0x7f800415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:45.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.068+0000 7f800a012640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800406bc80 con 0x7f80040c2720 2023-12-19T18:20:45.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.068+0000 7f800a012640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800406d690 con 0x7f800415b1a0 2023-12-19T18:20:45.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.068+0000 7f800a012640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800406d0a0 con 0x7f8004071420 2023-12-19T18:20:45.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.069+0000 7f8003fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8004071420 0x7f800415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:45.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.069+0000 7f80037fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f800415b1a0 0x7f800406f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:45.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.069+0000 7f7ffbfff640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f80040c2720 0x7f800406fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:41256/0 (socket says 172.21.15.27:41256) 2023-12-19T18:20:45.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.069+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3294600276 learned_addr learned my addr 172.21.15.27:0/3294600276 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:45.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.070+0000 7f80037fe640 1 -- 172.21.15.27:0/3294600276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8004071420 msgr2=0x7f800415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:45.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.070+0000 7f80037fe640 1 --2- 172.21.15.27:0/3294600276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8004071420 0x7f800415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.070+0000 7f80037fe640 1 -- 172.21.15.27:0/3294600276 >> v1:172.21.15.192:6789/0 conn(0x7f80040c2720 legacy=0x7f800406fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:45.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.070+0000 7f80037fe640 1 -- 172.21.15.27:0/3294600276 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800406b6c0 con 0x7f800415b1a0 2023-12-19T18:20:45.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.070+0000 7f8002ffd640 1 -- 172.21.15.27:0/3294600276 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2548179509 0 0) 0x7f7fec001110 con 0x7f80040c2720 2023-12-19T18:20:45.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.071+0000 7f80037fe640 1 --2- 172.21.15.27:0/3294600276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800415b1a0 0x7f800406f520 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40097a0 tx=0x7f7ff402b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=21642629e67aea0b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:45.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.071+0000 7f8002ffd640 1 -- 172.21.15.27:0/3294600276 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4031780 con 0x7f800415b1a0 2023-12-19T18:20:45.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.071+0000 7f8002ffd640 1 -- 172.21.15.27:0/3294600276 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff40318e0 con 0x7f800415b1a0 2023-12-19T18:20:45.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.071+0000 7f8002ffd640 1 -- 172.21.15.27:0/3294600276 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4031c70 con 0x7f800415b1a0 2023-12-19T18:20:45.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.072+0000 7f800a012640 1 -- 172.21.15.27:0/3294600276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800415b1a0 msgr2=0x7f800406f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:45.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.072+0000 7f800a012640 1 --2- 172.21.15.27:0/3294600276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800415b1a0 0x7f800406f520 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f7ff40097a0 tx=0x7f7ff402b820 comp rx=0 tx=0).stop 2023-12-19T18:20:45.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.072+0000 7f800a012640 1 -- 172.21.15.27:0/3294600276 shutdown_connections 2023-12-19T18:20:45.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.072+0000 7f800a012640 1 --2- 172.21.15.27:0/3294600276 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8004071420 0x7f800415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.072+0000 7f800a012640 1 --2- 172.21.15.27:0/3294600276 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800415b1a0 0x7f800406f520 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.072+0000 7f800a012640 1 -- 172.21.15.27:0/3294600276 >> 172.21.15.27:0/3294600276 conn(0x7f80040cebd0 msgr2=0x7f80040cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:45.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.072+0000 7f800a012640 1 -- 172.21.15.27:0/3294600276 shutdown_connections 2023-12-19T18:20:45.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.072+0000 7f800a012640 1 -- 172.21.15.27:0/3294600276 wait complete. 2023-12-19T18:20:45.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.074+0000 7f800a012640 1 Processor -- start 2023-12-19T18:20:45.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.074+0000 7f800a012640 1 -- start start 2023-12-19T18:20:45.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.075+0000 7f800a012640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8004071420 0x7f80040c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:45.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.075+0000 7f800a012640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80040c2720 0x7f800414aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:45.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.075+0000 7f800a012640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800415b1a0 0x7f800414a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.075+0000 7f800a012640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800414afb0 con 0x7f8004071420 2023-12-19T18:20:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.075+0000 7f800a012640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800414a560 con 0x7f80040c2720 2023-12-19T18:20:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.075+0000 7f800a012640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80040c8a30 con 0x7f800415b1a0 2023-12-19T18:20:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.076+0000 7f80037fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8004071420 0x7f80040c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.076+0000 7f80037fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8004071420 0x7f80040c8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49314/0 (socket says 172.21.15.27:49314) 2023-12-19T18:20:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.076+0000 7f80037fe640 1 -- 172.21.15.27:0/1635498085 learned_addr learned my addr 172.21.15.27:0/1635498085 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.076+0000 7f8003fff640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800415b1a0 0x7f800414a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.076+0000 7f80037fe640 1 -- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800415b1a0 msgr2=0x7f800414a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.076+0000 7f80037fe640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800415b1a0 0x7f800414a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.076+0000 7f7ffbfff640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80040c2720 0x7f800414aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:45.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.076+0000 7f80037fe640 1 -- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80040c2720 msgr2=0x7f800414aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:45.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.076+0000 7f80037fe640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80040c2720 0x7f800414aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.076+0000 7f80037fe640 1 -- 172.21.15.27:0/1635498085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff4009400 con 0x7f8004071420 2023-12-19T18:20:45.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.076+0000 7f7ffbfff640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80040c2720 0x7f800414aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:45.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.077+0000 7f80037fe640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8004071420 0x7f80040c8330 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4034040 tx=0x7f7ff40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.077+0000 7f8000ff9640 1 -- 172.21.15.27:0/1635498085 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4032820 con 0x7f8004071420 2023-12-19T18:20:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.077+0000 7f8000ff9640 1 -- 172.21.15.27:0/1635498085 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff4032980 con 0x7f8004071420 2023-12-19T18:20:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.077+0000 7f8000ff9640 1 -- 172.21.15.27:0/1635498085 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4032d30 con 0x7f8004071420 2023-12-19T18:20:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.077+0000 7f800a012640 1 -- 172.21.15.27:0/1635498085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80040c66b0 con 0x7f8004071420 2023-12-19T18:20:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.077+0000 7f800a012640 1 -- 172.21.15.27:0/1635498085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80040c6c70 con 0x7f8004071420 2023-12-19T18:20:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.078+0000 7f800a012640 1 -- 172.21.15.27:0/1635498085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8004068e90 con 0x7f8004071420 2023-12-19T18:20:45.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.079+0000 7f8000ff9640 1 -- 172.21.15.27:0/1635498085 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7ff404e440 con 0x7f8004071420 2023-12-19T18:20:45.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.080+0000 7f8000ff9640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fe406a430 0x7f7fe406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:45.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.080+0000 7f8000ff9640 1 -- 172.21.15.27:0/1635498085 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7ff40aab30 con 0x7f8004071420 2023-12-19T18:20:45.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.080+0000 7f8000ff9640 1 -- 172.21.15.27:0/1635498085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=102}) v3 -- 0x7f7fe4072020 con 0x7f8004071420 2023-12-19T18:20:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.081+0000 7f7ffbfff640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fe406a430 0x7f7fe406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.082+0000 7f7ffbfff640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fe406a430 0x7f7fe406c8f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80040a98a0 tx=0x7f7fec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.085+0000 7f8000ff9640 1 -- 172.21.15.27:0/1635498085 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7ff4076040 con 0x7f8004071420 2023-12-19T18:20:45.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.278+0000 7f800a012640 1 -- 172.21.15.27:0/1635498085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f80040cad60 con 0x7f8004071420 2023-12-19T18:20:45.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.279+0000 7f8000ff9640 1 -- 172.21.15.27:0/1635498085 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7ff4079660 con 0x7f8004071420 2023-12-19T18:20:45.279 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:20:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.282+0000 7f800a012640 1 -- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fe406a430 msgr2=0x7f7fe406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.282+0000 7f800a012640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fe406a430 0x7f7fe406c8f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f80040a98a0 tx=0x7f7fec009040 comp rx=0 tx=0).stop 2023-12-19T18:20:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.282+0000 7f800a012640 1 -- 172.21.15.27:0/1635498085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8004071420 msgr2=0x7f80040c8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.282+0000 7f800a012640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8004071420 0x7f80040c8330 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4034040 tx=0x7f7ff40046c0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.282+0000 7f800a012640 1 -- 172.21.15.27:0/1635498085 shutdown_connections 2023-12-19T18:20:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.282+0000 7f800a012640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fe406a430 0x7f7fe406c8f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.282+0000 7f800a012640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800415b1a0 0x7f800414a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.282+0000 7f800a012640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80040c2720 0x7f800414aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.282+0000 7f800a012640 1 --2- 172.21.15.27:0/1635498085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8004071420 0x7f80040c8330 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.282+0000 7f800a012640 1 -- 172.21.15.27:0/1635498085 >> 172.21.15.27:0/1635498085 conn(0x7f80040cebd0 msgr2=0x7f80040c3e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.282+0000 7f800a012640 1 -- 172.21.15.27:0/1635498085 shutdown_connections 2023-12-19T18:20:45.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.282+0000 7f800a012640 1 -- 172.21.15.27:0/1635498085 wait complete. 2023-12-19T18:20:45.301 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:20:45.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.535+0000 7f0d69500640 1 Processor -- start 2023-12-19T18:20:45.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.535+0000 7f0d69500640 1 -- start start 2023-12-19T18:20:45.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.537+0000 7f0d69500640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0d64071140 0x7f0d640cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:45.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.537+0000 7f0d69500640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d640ac8d0 0x7f0d640acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:45.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.537+0000 7f0d69500640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d640ce7c0 con 0x7f0d6415d1d0 2023-12-19T18:20:45.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.537+0000 7f0d69500640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d64070150 con 0x7f0d64071140 2023-12-19T18:20:45.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.537+0000 7f0d69500640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6415d5b0 con 0x7f0d640ac8d0 2023-12-19T18:20:45.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.538+0000 7f0d5bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0d64071140 0x7f0d640cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:45.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.538+0000 7f0d637fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d640ac8d0 0x7f0d640acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:45.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.538+0000 7f0d5bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0d64071140 0x7f0d640cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:46002/0 (socket says 172.21.15.27:46002) 2023-12-19T18:20:45.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.538+0000 7f0d5bfff640 1 -- 172.21.15.27:0/2312106514 learned_addr learned my addr 172.21.15.27:0/2312106514 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:45.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.539+0000 7f0d637fe640 1 -- 172.21.15.27:0/2312106514 >> v2:172.21.15.192:3300/0 conn(0x7f0d64071140 msgr2=0x7f0d640cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:45.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.539+0000 7f0d637fe640 1 --2- 172.21.15.27:0/2312106514 >> v2:172.21.15.192:3300/0 conn(0x7f0d64071140 0x7f0d640cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.539+0000 7f0d637fe640 1 -- 172.21.15.27:0/2312106514 >> v1:172.21.15.192:6789/0 conn(0x7f0d6415d1d0 legacy=0x7f0d640ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:45.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.539+0000 7f0d637fe640 1 -- 172.21.15.27:0/2312106514 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d64070360 con 0x7f0d640ac8d0 2023-12-19T18:20:45.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.540+0000 7f0d637fe640 1 --2- 172.21.15.27:0/2312106514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d640ac8d0 0x7f0d640acd40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009d90 tx=0x7f0d5c02be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce679cae2f23b881 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.541+0000 7f0d627fc640 1 -- 172.21.15.27:0/2312106514 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c02cbc0 con 0x7f0d640ac8d0 2023-12-19T18:20:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.541+0000 7f0d627fc640 1 -- 172.21.15.27:0/2312106514 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5c02cd20 con 0x7f0d640ac8d0 2023-12-19T18:20:45.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.541+0000 7f0d627fc640 1 -- 172.21.15.27:0/2312106514 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c034660 con 0x7f0d640ac8d0 2023-12-19T18:20:45.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.541+0000 7f0d69500640 1 -- 172.21.15.27:0/2312106514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d640ac8d0 msgr2=0x7f0d640acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:45.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.541+0000 7f0d69500640 1 --2- 172.21.15.27:0/2312106514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d640ac8d0 0x7f0d640acd40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c009d90 tx=0x7f0d5c02be10 comp rx=0 tx=0).stop 2023-12-19T18:20:45.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.542+0000 7f0d69500640 1 -- 172.21.15.27:0/2312106514 shutdown_connections 2023-12-19T18:20:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.542+0000 7f0d69500640 1 --2- 172.21.15.27:0/2312106514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d640ac8d0 0x7f0d640acd40 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.542+0000 7f0d69500640 1 --2- 172.21.15.27:0/2312106514 >> v2:172.21.15.192:3300/0 conn(0x7f0d64071140 0x7f0d640cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.542+0000 7f0d69500640 1 -- 172.21.15.27:0/2312106514 >> 172.21.15.27:0/2312106514 conn(0x7f0d640ceb50 msgr2=0x7f0d640cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.542+0000 7f0d69500640 1 -- 172.21.15.27:0/2312106514 shutdown_connections 2023-12-19T18:20:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.542+0000 7f0d69500640 1 -- 172.21.15.27:0/2312106514 wait complete. 2023-12-19T18:20:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.544+0000 7f0d69500640 1 Processor -- start 2023-12-19T18:20:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.544+0000 7f0d69500640 1 -- start start 2023-12-19T18:20:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.545+0000 7f0d69500640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d64071140 0x7f0d641f30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.545+0000 7f0d69500640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d640ac8d0 0x7f0d640c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.545+0000 7f0d69500640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d6415d1d0 0x7f0d640c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.545+0000 7f0d69500640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d641f4040 con 0x7f0d640ac8d0 2023-12-19T18:20:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.545+0000 7f0d69500640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d640c4af0 con 0x7f0d6415d1d0 2023-12-19T18:20:45.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.545+0000 7f0d69500640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d641f36e0 con 0x7f0d64071140 2023-12-19T18:20:45.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.545+0000 7f0d5bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d640ac8d0 0x7f0d640c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:45.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.545+0000 7f0d5bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d640ac8d0 0x7f0d640c9590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49318/0 (socket says 172.21.15.27:49318) 2023-12-19T18:20:45.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.545+0000 7f0d5bfff640 1 -- 172.21.15.27:0/2871379560 learned_addr learned my addr 172.21.15.27:0/2871379560 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:45.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.545+0000 7f0d62ffd640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d64071140 0x7f0d641f30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:45.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.545+0000 7f0d637fe640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d6415d1d0 0x7f0d640c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:45.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.546+0000 7f0d5bfff640 1 -- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d64071140 msgr2=0x7f0d641f30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:45.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.546+0000 7f0d5bfff640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d64071140 0x7f0d641f30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.546+0000 7f0d5bfff640 1 -- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d6415d1d0 msgr2=0x7f0d640c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:45.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.546+0000 7f0d5bfff640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d6415d1d0 0x7f0d640c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.546+0000 7f0d5bfff640 1 -- 172.21.15.27:0/2871379560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d5c009a70 con 0x7f0d640ac8d0 2023-12-19T18:20:45.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.546+0000 7f0d5bfff640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d640ac8d0 0x7f0d640c9590 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009fd0 tx=0x7f0d4c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:45.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.546+0000 7f0d5b7fe640 1 -- 172.21.15.27:0/2871379560 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c00c940 con 0x7f0d640ac8d0 2023-12-19T18:20:45.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.546+0000 7f0d5b7fe640 1 -- 172.21.15.27:0/2871379560 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d4c00caa0 con 0x7f0d640ac8d0 2023-12-19T18:20:45.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.546+0000 7f0d5b7fe640 1 -- 172.21.15.27:0/2871379560 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d4c00ce30 con 0x7f0d640ac8d0 2023-12-19T18:20:45.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.546+0000 7f0d69500640 1 -- 172.21.15.27:0/2871379560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d641f3980 con 0x7f0d640ac8d0 2023-12-19T18:20:45.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.546+0000 7f0d69500640 1 -- 172.21.15.27:0/2871379560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d640c9ad0 con 0x7f0d640ac8d0 2023-12-19T18:20:45.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.548+0000 7f0d69500640 1 -- 172.21.15.27:0/2871379560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d641f42b0 con 0x7f0d640ac8d0 2023-12-19T18:20:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.549+0000 7f0d5b7fe640 1 -- 172.21.15.27:0/2871379560 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0d4c013470 con 0x7f0d640ac8d0 2023-12-19T18:20:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.550+0000 7f0d5b7fe640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d4406a430 0x7f0d4406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.550+0000 7f0d5b7fe640 1 -- 172.21.15.27:0/2871379560 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0d4c088ad0 con 0x7f0d640ac8d0 2023-12-19T18:20:45.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.550+0000 7f0d5b7fe640 1 -- 172.21.15.27:0/2871379560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=102}) v3 -- 0x7f0d44072020 con 0x7f0d640ac8d0 2023-12-19T18:20:45.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.550+0000 7f0d62ffd640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d4406a430 0x7f0d4406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:45.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.551+0000 7f0d62ffd640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d4406a430 0x7f0d4406c8f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0d54000f60 tx=0x7f0d54009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:45.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.554+0000 7f0d5b7fe640 1 -- 172.21.15.27:0/2871379560 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0d4c017060 con 0x7f0d640ac8d0 2023-12-19T18:20:45.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.760+0000 7f0d69500640 1 -- 172.21.15.27:0/2871379560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f0d64068e90 con 0x7f0d640ac8d0 2023-12-19T18:20:45.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.761+0000 7f0d5b7fe640 1 -- 172.21.15.27:0/2871379560 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0d4c021020 con 0x7f0d640ac8d0 2023-12-19T18:20:45.761 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:20:45.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.763+0000 7f0d69500640 1 -- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d4406a430 msgr2=0x7f0d4406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:45.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.763+0000 7f0d69500640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d4406a430 0x7f0d4406c8f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0d54000f60 tx=0x7f0d54009040 comp rx=0 tx=0).stop 2023-12-19T18:20:45.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.763+0000 7f0d69500640 1 -- 172.21.15.27:0/2871379560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d640ac8d0 msgr2=0x7f0d640c9590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:45.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.763+0000 7f0d69500640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d640ac8d0 0x7f0d640c9590 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c009fd0 tx=0x7f0d4c00ed40 comp rx=0 tx=0).stop 2023-12-19T18:20:45.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.763+0000 7f0d69500640 1 -- 172.21.15.27:0/2871379560 shutdown_connections 2023-12-19T18:20:45.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.763+0000 7f0d69500640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d4406a430 0x7f0d4406c8f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.763+0000 7f0d69500640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d6415d1d0 0x7f0d640c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.763+0000 7f0d69500640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d640ac8d0 0x7f0d640c9590 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.763+0000 7f0d69500640 1 --2- 172.21.15.27:0/2871379560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d64071140 0x7f0d641f30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:45.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.763+0000 7f0d69500640 1 -- 172.21.15.27:0/2871379560 >> 172.21.15.27:0/2871379560 conn(0x7f0d640ceb50 msgr2=0x7f0d640cde10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:45.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.764+0000 7f0d69500640 1 -- 172.21.15.27:0/2871379560 shutdown_connections 2023-12-19T18:20:45.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:45.764+0000 7f0d69500640 1 -- 172.21.15.27:0/2871379560 wait complete. 2023-12-19T18:20:45.781 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-12-19T18:20:45.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:45.781 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-19T18:20:45.875 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[103318]: fuse finished with error 0 and tester_r 0 2023-12-19T18:20:45.881 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:20:45.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:45.881 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:20:46.029 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:20:46.029 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:20:46.091 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:20:46.105 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:20:46.147 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:20:46.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.328+0000 7f40896b4640 1 Processor -- start 2023-12-19T18:20:46.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.328+0000 7f40896b4640 1 -- start start 2023-12-19T18:20:46.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.330+0000 7f40896b4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f40840c50a0 0x7f40840afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:46.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.330+0000 7f40896b4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40840c54f0 con 0x7f40841553a0 2023-12-19T18:20:46.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.330+0000 7f40896b4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40840cb350 con 0x7f408415ee30 2023-12-19T18:20:46.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.330+0000 7f40896b4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40840cbcf0 con 0x7f40840c50a0 2023-12-19T18:20:46.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.330+0000 7f4088eb3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f40840c50a0 0x7f40840afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:46.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.330+0000 7f4083fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f40841553a0 0x7f408415e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54364/0 (socket says 172.21.15.27:54364) 2023-12-19T18:20:46.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.330+0000 7f4083fff640 1 -- 172.21.15.27:0/2734595327 learned_addr learned my addr 172.21.15.27:0/2734595327 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:46.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.331+0000 7f4088eb3640 1 -- 172.21.15.27:0/2734595327 >> v1:172.21.15.192:6789/0 conn(0x7f408415ee30 legacy=0x7f40840c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:46.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.331+0000 7f4088eb3640 1 -- 172.21.15.27:0/2734595327 >> v1:172.21.15.27:6789/0 conn(0x7f40841553a0 legacy=0x7f408415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:46.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.331+0000 7f4088eb3640 1 -- 172.21.15.27:0/2734595327 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40840cb630 con 0x7f40840c50a0 2023-12-19T18:20:46.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.331+0000 7f40837fe640 1 -- 172.21.15.27:0/2734595327 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3202366566 0 0) 0x7f4074001110 con 0x7f40841553a0 2023-12-19T18:20:46.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.332+0000 7f4088eb3640 1 --2- 172.21.15.27:0/2734595327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40840c50a0 0x7f40840afeb0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f407c009d90 tx=0x7f407c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6dfb9aaf8b188ea server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:46.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.332+0000 7f40837fe640 1 -- 172.21.15.27:0/2734595327 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c02cbd0 con 0x7f40840c50a0 2023-12-19T18:20:46.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.332+0000 7f40837fe640 1 -- 172.21.15.27:0/2734595327 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c02cd30 con 0x7f40840c50a0 2023-12-19T18:20:46.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.332+0000 7f40837fe640 1 -- 172.21.15.27:0/2734595327 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c02cea0 con 0x7f40840c50a0 2023-12-19T18:20:46.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.333+0000 7f40896b4640 1 -- 172.21.15.27:0/2734595327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40840c50a0 msgr2=0x7f40840afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:46.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.333+0000 7f40896b4640 1 --2- 172.21.15.27:0/2734595327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40840c50a0 0x7f40840afeb0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f407c009d90 tx=0x7f407c02be20 comp rx=0 tx=0).stop 2023-12-19T18:20:46.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.333+0000 7f40896b4640 1 -- 172.21.15.27:0/2734595327 shutdown_connections 2023-12-19T18:20:46.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.333+0000 7f40896b4640 1 --2- 172.21.15.27:0/2734595327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40840c50a0 0x7f40840afeb0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:46.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.333+0000 7f40896b4640 1 -- 172.21.15.27:0/2734595327 >> 172.21.15.27:0/2734595327 conn(0x7f4084069c70 msgr2=0x7f408406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:46.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.333+0000 7f40896b4640 1 -- 172.21.15.27:0/2734595327 shutdown_connections 2023-12-19T18:20:46.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.333+0000 7f40896b4640 1 -- 172.21.15.27:0/2734595327 wait complete. 2023-12-19T18:20:46.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.335+0000 7f40896b4640 1 Processor -- start 2023-12-19T18:20:46.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.335+0000 7f40896b4640 1 -- start start 2023-12-19T18:20:46.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.336+0000 7f40896b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40840c50a0 0x7f40840b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:46.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.336+0000 7f40896b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40841553a0 0x7f408415c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:46.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.336+0000 7f40896b4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f408415ee30 0x7f408414c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:46.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.336+0000 7f40896b4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408415daa0 con 0x7f40840c50a0 2023-12-19T18:20:46.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.336+0000 7f40896b4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408415cbc0 con 0x7f408415ee30 2023-12-19T18:20:46.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.336+0000 7f40896b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408415ce90 con 0x7f40841553a0 2023-12-19T18:20:46.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.336+0000 7f4083fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40840c50a0 0x7f40840b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:46.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.336+0000 7f4083fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40840c50a0 0x7f40840b4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49340/0 (socket says 172.21.15.27:49340) 2023-12-19T18:20:46.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.336+0000 7f4083fff640 1 -- 172.21.15.27:0/2454898972 learned_addr learned my addr 172.21.15.27:0/2454898972 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:46.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.336+0000 7f4088eb3640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f408415ee30 0x7f408414c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:46.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.337+0000 7f407bfff640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40841553a0 0x7f408415c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:46.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.337+0000 7f4083fff640 1 -- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40841553a0 msgr2=0x7f408415c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:46.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.337+0000 7f4083fff640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40841553a0 0x7f408415c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:46.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.337+0000 7f4083fff640 1 -- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f408415ee30 msgr2=0x7f408414c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:46.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.337+0000 7f4083fff640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f408415ee30 0x7f408414c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:46.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.337+0000 7f4083fff640 1 -- 172.21.15.27:0/2454898972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c009a70 con 0x7f40840c50a0 2023-12-19T18:20:46.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.337+0000 7f4083fff640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40840c50a0 0x7f40840b4120 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f407400c780 tx=0x7f407400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:46.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.337+0000 7f40817fa640 1 -- 172.21.15.27:0/2454898972 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40740072d0 con 0x7f40840c50a0 2023-12-19T18:20:46.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.337+0000 7f40817fa640 1 -- 172.21.15.27:0/2454898972 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4074007450 con 0x7f40840c50a0 2023-12-19T18:20:46.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.337+0000 7f40817fa640 1 -- 172.21.15.27:0/2454898972 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40740047e0 con 0x7f40840c50a0 2023-12-19T18:20:46.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.337+0000 7f40896b4640 1 -- 172.21.15.27:0/2454898972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408415d170 con 0x7f40840c50a0 2023-12-19T18:20:46.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.338+0000 7f40896b4640 1 -- 172.21.15.27:0/2454898972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408415d730 con 0x7f40840c50a0 2023-12-19T18:20:46.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.339+0000 7f40896b4640 1 -- 172.21.15.27:0/2454898972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408404e840 con 0x7f40840c50a0 2023-12-19T18:20:46.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.339+0000 7f40817fa640 1 -- 172.21.15.27:0/2454898972 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4074004960 con 0x7f40840c50a0 2023-12-19T18:20:46.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.340+0000 7f40817fa640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f406406a430 0x7f406406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:46.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.340+0000 7f40817fa640 1 -- 172.21.15.27:0/2454898972 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f4074087ae0 con 0x7f40840c50a0 2023-12-19T18:20:46.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.341+0000 7f40817fa640 1 -- 172.21.15.27:0/2454898972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=102}) v3 -- 0x7f4064072020 con 0x7f40840c50a0 2023-12-19T18:20:46.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.341+0000 7f407bfff640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f406406a430 0x7f406406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:46.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.342+0000 7f407bfff640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f406406a430 0x7f406406c8f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f406c006fd0 tx=0x7f406c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:46.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.345+0000 7f40817fa640 1 -- 172.21.15.27:0/2454898972 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4074018070 con 0x7f40840c50a0 2023-12-19T18:20:46.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.593+0000 7f40896b4640 1 -- 172.21.15.27:0/2454898972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40840c54f0 con 0x7f40840c50a0 2023-12-19T18:20:46.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.595+0000 7f40817fa640 1 -- 172.21.15.27:0/2454898972 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v101) v1 ==== 74+0+16230 (secure 0 0 0) 0x7f40840c54f0 con 0x7f40840c50a0 2023-12-19T18:20:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:46.596 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":101,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:20:44.657708+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:19:58.591127+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T18:19:59.313961+0000","flags":3,"flags_names":"hashpspool,full","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"97","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","full","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","full","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["backfillfull","exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["backfillfull","exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.19","mappings":[{"from":2,"to":7}]},{"pgid":"11.29","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:20:46.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.598+0000 7f40896b4640 1 -- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f406406a430 msgr2=0x7f406406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:46.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.598+0000 7f40896b4640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f406406a430 0x7f406406c8f0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f406c006fd0 tx=0x7f406c008040 comp rx=0 tx=0).stop 2023-12-19T18:20:46.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.598+0000 7f40896b4640 1 -- 172.21.15.27:0/2454898972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40840c50a0 msgr2=0x7f40840b4120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:46.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.598+0000 7f40896b4640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40840c50a0 0x7f40840b4120 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7f407400c780 tx=0x7f407400cc40 comp rx=0 tx=0).stop 2023-12-19T18:20:46.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.598+0000 7f40896b4640 1 -- 172.21.15.27:0/2454898972 shutdown_connections 2023-12-19T18:20:46.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.598+0000 7f40896b4640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f406406a430 0x7f406406c8f0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:46.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.598+0000 7f40896b4640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f408415ee30 0x7f408414c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:46.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.598+0000 7f40896b4640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40841553a0 0x7f408415c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:46.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.598+0000 7f40896b4640 1 --2- 172.21.15.27:0/2454898972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40840c50a0 0x7f40840b4120 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:46.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.598+0000 7f40896b4640 1 -- 172.21.15.27:0/2454898972 >> 172.21.15.27:0/2454898972 conn(0x7f4084069c70 msgr2=0x7f408406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:46.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.598+0000 7f40896b4640 1 -- 172.21.15.27:0/2454898972 shutdown_connections 2023-12-19T18:20:46.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.598+0000 7f40896b4640 1 -- 172.21.15.27:0/2454898972 wait complete. 2023-12-19T18:20:46.618 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:19:58.591127+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:19:59.313961+0000', 'flags': 3, 'flags_names': 'hashpspool,full', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '97', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:20:46.618 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:20:46.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.854+0000 7fdc0b59e640 1 Processor -- start 2023-12-19T18:20:46.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.854+0000 7fdc0b59e640 1 -- start start 2023-12-19T18:20:46.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.856+0000 7fdc0b59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdc0c0d2100 0x7fdc0c0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:46.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.856+0000 7fdc0b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc0c0cdf80 0x7fdc0c0b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:46.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.856+0000 7fdc0b59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0c0d0e10 con 0x7fdc0c15cf80 2023-12-19T18:20:46.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.856+0000 7fdc0b59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc0c0b8cf0 con 0x7fdc0c0d2100 2023-12-19T18:20:46.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.856+0000 7fdc0b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c0ce410 con 0x7fdc0c0cdf80 2023-12-19T18:20:46.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.856+0000 7fdc09d9b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdc0c0d2100 0x7fdc0c0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:46.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.856+0000 7fdc0ad9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc0c0cdf80 0x7fdc0c0b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:46.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.857+0000 7fdc09d9b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdc0c0d2100 0x7fdc0c0cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:46028/0 (socket says 172.21.15.27:46028) 2023-12-19T18:20:46.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.857+0000 7fdc09d9b640 1 -- 172.21.15.27:0/4191793232 learned_addr learned my addr 172.21.15.27:0/4191793232 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:46.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.857+0000 7fdc09d9b640 1 -- 172.21.15.27:0/4191793232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc0c0cdf80 msgr2=0x7fdc0c0b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:46.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.857+0000 7fdc09d9b640 1 --2- 172.21.15.27:0/4191793232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc0c0cdf80 0x7fdc0c0b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:46.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.857+0000 7fdc09d9b640 1 -- 172.21.15.27:0/4191793232 >> v1:172.21.15.192:6789/0 conn(0x7fdc0c15cf80 legacy=0x7fdc0c0b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:46.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.857+0000 7fdc09d9b640 1 -- 172.21.15.27:0/4191793232 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c0d1c70 con 0x7fdc0c0d2100 2023-12-19T18:20:46.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.859+0000 7fdc09d9b640 1 --2- 172.21.15.27:0/4191793232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc0c0d2100 0x7fdc0c0cda40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009870 tx=0x7fdc0002ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc503c1a9df65139 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:46.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.860+0000 7fdc0959a640 1 -- 172.21.15.27:0/4191793232 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0002f830 con 0x7fdc0c0d2100 2023-12-19T18:20:46.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.860+0000 7fdc0959a640 1 -- 172.21.15.27:0/4191793232 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0002f990 con 0x7fdc0c0d2100 2023-12-19T18:20:46.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.860+0000 7fdc0959a640 1 -- 172.21.15.27:0/4191793232 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0002fd20 con 0x7fdc0c0d2100 2023-12-19T18:20:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.860+0000 7fdc0b59e640 1 -- 172.21.15.27:0/4191793232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc0c0d2100 msgr2=0x7fdc0c0cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.860+0000 7fdc0b59e640 1 --2- 172.21.15.27:0/4191793232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc0c0d2100 0x7fdc0c0cda40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdc00009870 tx=0x7fdc0002ba10 comp rx=0 tx=0).stop 2023-12-19T18:20:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.861+0000 7fdc0b59e640 1 -- 172.21.15.27:0/4191793232 shutdown_connections 2023-12-19T18:20:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.861+0000 7fdc0b59e640 1 --2- 172.21.15.27:0/4191793232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc0c0cdf80 0x7fdc0c0b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.861+0000 7fdc0b59e640 1 --2- 172.21.15.27:0/4191793232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc0c0d2100 0x7fdc0c0cda40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.861+0000 7fdc0b59e640 1 -- 172.21.15.27:0/4191793232 >> 172.21.15.27:0/4191793232 conn(0x7fdc0c072e20 msgr2=0x7fdc0c073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.861+0000 7fdc0b59e640 1 -- 172.21.15.27:0/4191793232 shutdown_connections 2023-12-19T18:20:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.861+0000 7fdc0b59e640 1 -- 172.21.15.27:0/4191793232 wait complete. 2023-12-19T18:20:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.863+0000 7fdc0b59e640 1 Processor -- start 2023-12-19T18:20:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.863+0000 7fdc0b59e640 1 -- start start 2023-12-19T18:20:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.864+0000 7fdc0b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc0c0cdf80 0x7fdc0c0a9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.864+0000 7fdc0b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc0c0d2100 0x7fdc0c0ca870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.864+0000 7fdc0b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc0c15cf80 0x7fdc0c0b6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.864+0000 7fdc0b59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c0cb810 con 0x7fdc0c0d2100 2023-12-19T18:20:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.864+0000 7fdc0b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c1f2370 con 0x7fdc0c15cf80 2023-12-19T18:20:46.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.864+0000 7fdc0b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0c0aa2e0 con 0x7fdc0c0cdf80 2023-12-19T18:20:46.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.864+0000 7fdc09d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc0c0d2100 0x7fdc0c0ca870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:46.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.864+0000 7fdc09d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc0c0d2100 0x7fdc0c0ca870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49356/0 (socket says 172.21.15.27:49356) 2023-12-19T18:20:46.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.864+0000 7fdc09d9b640 1 -- 172.21.15.27:0/1336725436 learned_addr learned my addr 172.21.15.27:0/1336725436 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:46.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.864+0000 7fdc0a59c640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc0c0cdf80 0x7fdc0c0a9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:46.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.864+0000 7fdc0ad9d640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc0c15cf80 0x7fdc0c0b6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:46.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.865+0000 7fdc09d9b640 1 -- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc0c0cdf80 msgr2=0x7fdc0c0a9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:46.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.865+0000 7fdc09d9b640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc0c0cdf80 0x7fdc0c0a9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:46.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.865+0000 7fdc09d9b640 1 -- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc0c15cf80 msgr2=0x7fdc0c0b6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:46.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.865+0000 7fdc09d9b640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc0c15cf80 0x7fdc0c0b6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:46.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.865+0000 7fdc09d9b640 1 -- 172.21.15.27:0/1336725436 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc000094d0 con 0x7fdc0c0d2100 2023-12-19T18:20:46.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.865+0000 7fdc09d9b640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc0c0d2100 0x7fdc0c0ca870 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fdc00000c00 tx=0x7fdc00004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:46.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.865+0000 7fdbfaffd640 1 -- 172.21.15.27:0/1336725436 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00037c40 con 0x7fdc0c0d2100 2023-12-19T18:20:46.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.865+0000 7fdbfaffd640 1 -- 172.21.15.27:0/1336725436 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc00037da0 con 0x7fdc0c0d2100 2023-12-19T18:20:46.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.865+0000 7fdc0b59e640 1 -- 172.21.15.27:0/1336725436 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0c0cb150 con 0x7fdc0c0d2100 2023-12-19T18:20:46.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.865+0000 7fdbfaffd640 1 -- 172.21.15.27:0/1336725436 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc00041640 con 0x7fdc0c0d2100 2023-12-19T18:20:46.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.866+0000 7fdc0b59e640 1 -- 172.21.15.27:0/1336725436 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0c0cadb0 con 0x7fdc0c0d2100 2023-12-19T18:20:46.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.867+0000 7fdc0b59e640 1 -- 172.21.15.27:0/1336725436 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0c04e840 con 0x7fdc0c0d2100 2023-12-19T18:20:46.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.868+0000 7fdbfaffd640 1 -- 172.21.15.27:0/1336725436 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdc000417a0 con 0x7fdc0c0d2100 2023-12-19T18:20:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.869+0000 7fdbfaffd640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdbec06a430 0x7fdbec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:46.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.869+0000 7fdbfaffd640 1 -- 172.21.15.27:0/1336725436 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fdc0003c070 con 0x7fdc0c0d2100 2023-12-19T18:20:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.870+0000 7fdbfaffd640 1 -- 172.21.15.27:0/1336725436 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=102}) v3 -- 0x7fdbec072020 con 0x7fdc0c0d2100 2023-12-19T18:20:46.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.870+0000 7fdc0a59c640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdbec06a430 0x7fdbec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.871+0000 7fdc0a59c640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdbec06a430 0x7fdbec06c8f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc000f60 tx=0x7fdbfc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:46.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:46.874+0000 7fdbfaffd640 1 -- 172.21.15.27:0/1336725436 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdc00076bc0 con 0x7fdc0c0d2100 2023-12-19T18:20:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.069+0000 7fdc0b59e640 1 -- 172.21.15.27:0/1336725436 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc0c15c580 con 0x7fdc0c0d2100 2023-12-19T18:20:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.070+0000 7fdbfaffd640 1 -- 172.21.15.27:0/1336725436 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v101) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc0007a1e0 con 0x7fdc0c0d2100 2023-12-19T18:20:47.070 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:20:47.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.072+0000 7fdc0b59e640 1 -- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdbec06a430 msgr2=0x7fdbec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:47.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.072+0000 7fdc0b59e640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdbec06a430 0x7fdbec06c8f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc000f60 tx=0x7fdbfc009040 comp rx=0 tx=0).stop 2023-12-19T18:20:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.072+0000 7fdc0b59e640 1 -- 172.21.15.27:0/1336725436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc0c0d2100 msgr2=0x7fdc0c0ca870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.072+0000 7fdc0b59e640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc0c0d2100 0x7fdc0c0ca870 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fdc00000c00 tx=0x7fdc00004730 comp rx=0 tx=0).stop 2023-12-19T18:20:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.072+0000 7fdc0b59e640 1 -- 172.21.15.27:0/1336725436 shutdown_connections 2023-12-19T18:20:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.072+0000 7fdc0b59e640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdbec06a430 0x7fdbec06c8f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.072+0000 7fdc0b59e640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc0c15cf80 0x7fdc0c0b6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.072+0000 7fdc0b59e640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc0c0d2100 0x7fdc0c0ca870 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.072+0000 7fdc0b59e640 1 --2- 172.21.15.27:0/1336725436 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc0c0cdf80 0x7fdc0c0a9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.072+0000 7fdc0b59e640 1 -- 172.21.15.27:0/1336725436 >> 172.21.15.27:0/1336725436 conn(0x7fdc0c072e20 msgr2=0x7fdc0c0b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:47.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.073+0000 7fdc0b59e640 1 -- 172.21.15.27:0/1336725436 shutdown_connections 2023-12-19T18:20:47.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.073+0000 7fdc0b59e640 1 -- 172.21.15.27:0/1336725436 wait complete. 2023-12-19T18:20:47.094 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:20:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.336+0000 7f96d6b37640 1 Processor -- start 2023-12-19T18:20:47.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.336+0000 7f96d6b37640 1 -- start start 2023-12-19T18:20:47.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.337+0000 7f96d6b37640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f96d00c2720 0x7f96d006fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:47.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.337+0000 7f96d6b37640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f96d0071420 0x7f96d015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:47.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.337+0000 7f96d6b37640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d006bc80 con 0x7f96d015b1a0 2023-12-19T18:20:47.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.337+0000 7f96d6b37640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d006d690 con 0x7f96d00c2720 2023-12-19T18:20:47.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.337+0000 7f96d6b37640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d006d0a0 con 0x7f96d0071420 2023-12-19T18:20:47.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.338+0000 7f96d48d3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f96d015b1a0 0x7f96d006f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54370/0 (socket says 172.21.15.27:54370) 2023-12-19T18:20:47.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.338+0000 7f96d48d3640 1 -- 172.21.15.27:0/1778243623 learned_addr learned my addr 172.21.15.27:0/1778243623 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:47.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.338+0000 7f96cffff640 1 --2- 172.21.15.27:0/1778243623 >> v2:172.21.15.27:3300/0 conn(0x7f96d00c2720 0x7f96d006fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:47.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.338+0000 7f96cffff640 1 -- 172.21.15.27:0/1778243623 >> v2:172.21.15.192:3300/0 conn(0x7f96d0071420 msgr2=0x7f96d015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:47.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.338+0000 7f96d50d4640 1 --2- 172.21.15.27:0/1778243623 >> v2:172.21.15.192:3300/0 conn(0x7f96d0071420 0x7f96d015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:47.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.338+0000 7f96cf7fe640 1 -- 172.21.15.27:0/1778243623 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2544317090 0 0) 0x7f96c4001110 con 0x7f96d015b1a0 2023-12-19T18:20:47.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.338+0000 7f96cffff640 1 --2- 172.21.15.27:0/1778243623 >> v2:172.21.15.192:3300/0 conn(0x7f96d0071420 0x7f96d015ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.338+0000 7f96cffff640 1 -- 172.21.15.27:0/1778243623 >> v1:172.21.15.27:6789/0 conn(0x7f96d015b1a0 legacy=0x7f96d006f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:47.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.338+0000 7f96cffff640 1 -- 172.21.15.27:0/1778243623 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d006b6c0 con 0x7f96d00c2720 2023-12-19T18:20:47.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.338+0000 7f96d50d4640 1 --2- 172.21.15.27:0/1778243623 >> v2:172.21.15.192:3300/0 conn(0x7f96d0071420 0x7f96d015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:47.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.339+0000 7f96cffff640 1 --2- 172.21.15.27:0/1778243623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d00c2720 0x7f96d006fc30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f96d006d600 tx=0x7f96bc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba20e79ba40a9335 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:47.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.340+0000 7f96cf7fe640 1 -- 172.21.15.27:0/1778243623 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc02e070 con 0x7f96d00c2720 2023-12-19T18:20:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.340+0000 7f96cf7fe640 1 -- 172.21.15.27:0/1778243623 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96bc0315a0 con 0x7f96d00c2720 2023-12-19T18:20:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.340+0000 7f96cf7fe640 1 -- 172.21.15.27:0/1778243623 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc031930 con 0x7f96d00c2720 2023-12-19T18:20:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.340+0000 7f96d6b37640 1 -- 172.21.15.27:0/1778243623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d00c2720 msgr2=0x7f96d006fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.340+0000 7f96d6b37640 1 --2- 172.21.15.27:0/1778243623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d00c2720 0x7f96d006fc30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f96d006d600 tx=0x7f96bc02b840 comp rx=0 tx=0).stop 2023-12-19T18:20:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.340+0000 7f96d6b37640 1 -- 172.21.15.27:0/1778243623 shutdown_connections 2023-12-19T18:20:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.340+0000 7f96d6b37640 1 --2- 172.21.15.27:0/1778243623 >> v2:172.21.15.192:3300/0 conn(0x7f96d0071420 0x7f96d015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.340+0000 7f96d6b37640 1 --2- 172.21.15.27:0/1778243623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d00c2720 0x7f96d006fc30 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.340+0000 7f96d6b37640 1 -- 172.21.15.27:0/1778243623 >> 172.21.15.27:0/1778243623 conn(0x7f96d00cebd0 msgr2=0x7f96d00cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.341+0000 7f96d6b37640 1 -- 172.21.15.27:0/1778243623 shutdown_connections 2023-12-19T18:20:47.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.341+0000 7f96d6b37640 1 -- 172.21.15.27:0/1778243623 wait complete. 2023-12-19T18:20:47.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.342+0000 7f96d6b37640 1 Processor -- start 2023-12-19T18:20:47.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.342+0000 7f96d6b37640 1 -- start start 2023-12-19T18:20:47.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.343+0000 7f96d6b37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d0071420 0x7f96d015a190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:47.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.343+0000 7f96d6b37640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d00c2720 0x7f96d01e2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.343+0000 7f96d6b37640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96d015b1a0 0x7f96d00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.343+0000 7f96d6b37640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d00c89f0 con 0x7f96d00c2720 2023-12-19T18:20:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.343+0000 7f96d6b37640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d01e36e0 con 0x7f96d0071420 2023-12-19T18:20:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.343+0000 7f96d6b37640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d01e39c0 con 0x7f96d015b1a0 2023-12-19T18:20:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.344+0000 7f96cffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d00c2720 0x7f96d01e2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.344+0000 7f96cffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d00c2720 0x7f96d01e2180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49384/0 (socket says 172.21.15.27:49384) 2023-12-19T18:20:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.344+0000 7f96cffff640 1 -- 172.21.15.27:0/3073826997 learned_addr learned my addr 172.21.15.27:0/3073826997 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.344+0000 7f96d48d3640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d0071420 0x7f96d015a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:47.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.344+0000 7f96d50d4640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96d015b1a0 0x7f96d00ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.344+0000 7f96cffff640 1 -- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96d015b1a0 msgr2=0x7f96d00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.344+0000 7f96cffff640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96d015b1a0 0x7f96d00ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.344+0000 7f96cffff640 1 -- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d0071420 msgr2=0x7f96d015a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.344+0000 7f96cffff640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d0071420 0x7f96d015a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.344+0000 7f96cffff640 1 -- 172.21.15.27:0/3073826997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc009400 con 0x7f96d00c2720 2023-12-19T18:20:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.344+0000 7f96d48d3640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d0071420 0x7f96d015a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.345+0000 7f96cffff640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d00c2720 0x7f96d01e2180 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009890 tx=0x7f96bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.345+0000 7f96cd7fa640 1 -- 172.21.15.27:0/3073826997 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc032900 con 0x7f96d00c2720 2023-12-19T18:20:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.345+0000 7f96cd7fa640 1 -- 172.21.15.27:0/3073826997 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96bc032a60 con 0x7f96d00c2720 2023-12-19T18:20:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.345+0000 7f96cd7fa640 1 -- 172.21.15.27:0/3073826997 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96bc032e10 con 0x7f96d00c2720 2023-12-19T18:20:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.345+0000 7f96d6b37640 1 -- 172.21.15.27:0/3073826997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96d01e3ca0 con 0x7f96d00c2720 2023-12-19T18:20:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.345+0000 7f96d6b37640 1 -- 172.21.15.27:0/3073826997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d01f2df0 con 0x7f96d00c2720 2023-12-19T18:20:47.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.346+0000 7f96d6b37640 1 -- 172.21.15.27:0/3073826997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d01e41c0 con 0x7f96d00c2720 2023-12-19T18:20:47.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.347+0000 7f96cd7fa640 1 -- 172.21.15.27:0/3073826997 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f96bc0453f0 con 0x7f96d00c2720 2023-12-19T18:20:47.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.348+0000 7f96cd7fa640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f96b406a430 0x7f96b406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:47.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.348+0000 7f96cd7fa640 1 -- 172.21.15.27:0/3073826997 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f96bc0aaad0 con 0x7f96d00c2720 2023-12-19T18:20:47.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.348+0000 7f96cd7fa640 1 -- 172.21.15.27:0/3073826997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=102}) v3 -- 0x7f96b4072020 con 0x7f96d00c2720 2023-12-19T18:20:47.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.349+0000 7f96d48d3640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f96b406a430 0x7f96b406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:47.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.350+0000 7f96d48d3640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f96b406a430 0x7f96b406c8f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f96c4000f60 tx=0x7f96c4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:47.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.352+0000 7f96cd7fa640 1 -- 172.21.15.27:0/3073826997 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f96d01e41c0 con 0x7f96d00c2720 2023-12-19T18:20:47.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.553+0000 7f96d6b37640 1 -- 172.21.15.27:0/3073826997 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f96d01e41c0 con 0x7f96d00c2720 2023-12-19T18:20:47.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.554+0000 7f96cd7fa640 1 -- 172.21.15.27:0/3073826997 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v101) v1 ==== 104+0+11 (secure 0 0 0) 0x7f96d01e41c0 con 0x7f96d00c2720 2023-12-19T18:20:47.554 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:20:47.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.557+0000 7f96d6b37640 1 -- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f96b406a430 msgr2=0x7f96b406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:47.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.557+0000 7f96d6b37640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f96b406a430 0x7f96b406c8f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f96c4000f60 tx=0x7f96c4009040 comp rx=0 tx=0).stop 2023-12-19T18:20:47.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.557+0000 7f96d6b37640 1 -- 172.21.15.27:0/3073826997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d00c2720 msgr2=0x7f96d01e2180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:47.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.557+0000 7f96d6b37640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d00c2720 0x7f96d01e2180 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f96bc009890 tx=0x7f96bc0046c0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.557+0000 7f96d6b37640 1 -- 172.21.15.27:0/3073826997 shutdown_connections 2023-12-19T18:20:47.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.557+0000 7f96d6b37640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f96b406a430 0x7f96b406c8f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.557+0000 7f96d6b37640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96d015b1a0 0x7f96d00ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.557+0000 7f96d6b37640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96d00c2720 0x7f96d01e2180 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.557+0000 7f96d6b37640 1 --2- 172.21.15.27:0/3073826997 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96d0071420 0x7f96d015a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.557+0000 7f96d6b37640 1 -- 172.21.15.27:0/3073826997 >> 172.21.15.27:0/3073826997 conn(0x7f96d00cebd0 msgr2=0x7f96d00ce280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:47.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.557+0000 7f96d6b37640 1 -- 172.21.15.27:0/3073826997 shutdown_connections 2023-12-19T18:20:47.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.557+0000 7f96d6b37640 1 -- 172.21.15.27:0/3073826997 wait complete. 2023-12-19T18:20:47.578 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:20:47.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.817+0000 7f7c29470640 1 Processor -- start 2023-12-19T18:20:47.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.817+0000 7f7c29470640 1 -- start start 2023-12-19T18:20:47.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.818+0000 7f7c29470640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7c2415ae80 0x7f7c240cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:47.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.818+0000 7f7c29470640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7c240b3380 0x7f7c240b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:47.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.818+0000 7f7c29470640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c240cd6b0 con 0x7f7c240b2810 2023-12-19T18:20:47.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.818+0000 7f7c29470640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c240cdc70 con 0x7f7c240b3380 2023-12-19T18:20:47.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.818+0000 7f7c29470640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c240d0450 con 0x7f7c2415ae80 2023-12-19T18:20:47.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.819+0000 7f7c237fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7c240b3380 0x7f7c240b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:47.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.819+0000 7f7c237fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7c240b3380 0x7f7c240b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49400/0 (socket says 172.21.15.27:49400) 2023-12-19T18:20:47.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.819+0000 7f7c237fe640 1 -- 172.21.15.27:0/3366341251 learned_addr learned my addr 172.21.15.27:0/3366341251 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:47.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.819+0000 7f7c22ffd640 1 --2- 172.21.15.27:0/3366341251 >> v2:172.21.15.192:3300/0 conn(0x7f7c2415ae80 0x7f7c240cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:47.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.819+0000 7f7c237fe640 1 -- 172.21.15.27:0/3366341251 >> v2:172.21.15.192:3300/0 conn(0x7f7c2415ae80 msgr2=0x7f7c240cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:47.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.819+0000 7f7c237fe640 1 --2- 172.21.15.27:0/3366341251 >> v2:172.21.15.192:3300/0 conn(0x7f7c2415ae80 0x7f7c240cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.819+0000 7f7c237fe640 1 -- 172.21.15.27:0/3366341251 >> v1:172.21.15.192:6789/0 conn(0x7f7c240b2810 legacy=0x7f7c240b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:47.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.819+0000 7f7c237fe640 1 -- 172.21.15.27:0/3366341251 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c240d0730 con 0x7f7c240b3380 2023-12-19T18:20:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.821+0000 7f7c237fe640 1 --2- 172.21.15.27:0/3366341251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c240b3380 0x7f7c240b3be0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7c2406d4c0 tx=0x7f7c1c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa5173a109baa270 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.821+0000 7f7c21ffb640 1 -- 172.21.15.27:0/3366341251 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c031870 con 0x7f7c240b3380 2023-12-19T18:20:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.821+0000 7f7c21ffb640 1 -- 172.21.15.27:0/3366341251 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c1c0319d0 con 0x7f7c240b3380 2023-12-19T18:20:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.821+0000 7f7c21ffb640 1 -- 172.21.15.27:0/3366341251 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c1c031d60 con 0x7f7c240b3380 2023-12-19T18:20:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.822+0000 7f7c29470640 1 -- 172.21.15.27:0/3366341251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c240b3380 msgr2=0x7f7c240b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.822+0000 7f7c29470640 1 --2- 172.21.15.27:0/3366341251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c240b3380 0x7f7c240b3be0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f7c2406d4c0 tx=0x7f7c1c02b910 comp rx=0 tx=0).stop 2023-12-19T18:20:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.822+0000 7f7c29470640 1 -- 172.21.15.27:0/3366341251 shutdown_connections 2023-12-19T18:20:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.822+0000 7f7c29470640 1 --2- 172.21.15.27:0/3366341251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c240b3380 0x7f7c240b3be0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.822+0000 7f7c29470640 1 --2- 172.21.15.27:0/3366341251 >> v2:172.21.15.192:3300/0 conn(0x7f7c2415ae80 0x7f7c240cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.822+0000 7f7c29470640 1 -- 172.21.15.27:0/3366341251 >> 172.21.15.27:0/3366341251 conn(0x7f7c2406f1f0 msgr2=0x7f7c2406f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:47.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.822+0000 7f7c29470640 1 -- 172.21.15.27:0/3366341251 shutdown_connections 2023-12-19T18:20:47.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.822+0000 7f7c29470640 1 -- 172.21.15.27:0/3366341251 wait complete. 2023-12-19T18:20:47.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.824+0000 7f7c29470640 1 Processor -- start 2023-12-19T18:20:47.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.824+0000 7f7c29470640 1 -- start start 2023-12-19T18:20:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.825+0000 7f7c29470640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c240b2810 0x7f7c24159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.825+0000 7f7c29470640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c240b3380 0x7f7c241e4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.825+0000 7f7c29470640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c2415ae80 0x7f7c241e4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.825+0000 7f7c29470640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c240ca1a0 con 0x7f7c240b3380 2023-12-19T18:20:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.825+0000 7f7c29470640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c241e5400 con 0x7f7c240b2810 2023-12-19T18:20:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.825+0000 7f7c29470640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c240c9900 con 0x7f7c2415ae80 2023-12-19T18:20:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.825+0000 7f7c227fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c240b3380 0x7f7c241e4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.825+0000 7f7c227fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c240b3380 0x7f7c241e4980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49402/0 (socket says 172.21.15.27:49402) 2023-12-19T18:20:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.825+0000 7f7c227fc640 1 -- 172.21.15.27:0/625022906 learned_addr learned my addr 172.21.15.27:0/625022906 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.826+0000 7f7c227fc640 1 -- 172.21.15.27:0/625022906 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c2415ae80 msgr2=0x7f7c241e4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.826+0000 7f7c227fc640 1 --2- 172.21.15.27:0/625022906 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c2415ae80 0x7f7c241e4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.826+0000 7f7c22ffd640 1 --2- 172.21.15.27:0/625022906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c240b2810 0x7f7c24159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.826+0000 7f7c227fc640 1 -- 172.21.15.27:0/625022906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c240b2810 msgr2=0x7f7c24159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.826+0000 7f7c227fc640 1 --2- 172.21.15.27:0/625022906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c240b2810 0x7f7c24159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.826+0000 7f7c227fc640 1 -- 172.21.15.27:0/625022906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c1c0094d0 con 0x7f7c240b3380 2023-12-19T18:20:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.826+0000 7f7c227fc640 1 --2- 172.21.15.27:0/625022906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c240b3380 0x7f7c241e4980 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00c430 tx=0x7f7c0c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.826+0000 7f7c1b7fe640 1 -- 172.21.15.27:0/625022906 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c011790 con 0x7f7c240b3380 2023-12-19T18:20:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.826+0000 7f7c1b7fe640 1 -- 172.21.15.27:0/625022906 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c0c0118f0 con 0x7f7c240b3380 2023-12-19T18:20:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.826+0000 7f7c1b7fe640 1 -- 172.21.15.27:0/625022906 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c0c011ce0 con 0x7f7c240b3380 2023-12-19T18:20:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.826+0000 7f7c29470640 1 -- 172.21.15.27:0/625022906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c240c9be0 con 0x7f7c240b3380 2023-12-19T18:20:47.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.827+0000 7f7c29470640 1 -- 172.21.15.27:0/625022906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c240c4c90 con 0x7f7c240b3380 2023-12-19T18:20:47.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.828+0000 7f7c29470640 1 -- 172.21.15.27:0/625022906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c240c4e80 con 0x7f7c240b3380 2023-12-19T18:20:47.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.829+0000 7f7c1b7fe640 1 -- 172.21.15.27:0/625022906 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7c0c01c460 con 0x7f7c240b3380 2023-12-19T18:20:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.830+0000 7f7c1b7fe640 1 --2- 172.21.15.27:0/625022906 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c0406a430 0x7f7c0406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.830+0000 7f7c1b7fe640 1 -- 172.21.15.27:0/625022906 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7c0c0888d0 con 0x7f7c240b3380 2023-12-19T18:20:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.830+0000 7f7c1b7fe640 1 -- 172.21.15.27:0/625022906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=102}) v3 -- 0x7f7c04072020 con 0x7f7c240b3380 2023-12-19T18:20:47.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.830+0000 7f7c22ffd640 1 --2- 172.21.15.27:0/625022906 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c0406a430 0x7f7c0406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:47.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.832+0000 7f7c22ffd640 1 --2- 172.21.15.27:0/625022906 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c0406a430 0x7f7c0406c8f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009e90 tx=0x7f7c1400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:47.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:47.834+0000 7f7c1b7fe640 1 -- 172.21.15.27:0/625022906 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7c0c053e70 con 0x7f7c240b3380 2023-12-19T18:20:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.028+0000 7f7c29470640 1 -- 172.21.15.27:0/625022906 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7c24067110 con 0x7f7c240b3380 2023-12-19T18:20:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.029+0000 7f7c1b7fe640 1 -- 172.21.15.27:0/625022906 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v101) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7c0c057490 con 0x7f7c240b3380 2023-12-19T18:20:48.029 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:20:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.031+0000 7f7c29470640 1 -- 172.21.15.27:0/625022906 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c0406a430 msgr2=0x7f7c0406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.031+0000 7f7c29470640 1 --2- 172.21.15.27:0/625022906 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c0406a430 0x7f7c0406c8f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f7c14009e90 tx=0x7f7c1400b040 comp rx=0 tx=0).stop 2023-12-19T18:20:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.031+0000 7f7c29470640 1 -- 172.21.15.27:0/625022906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c240b3380 msgr2=0x7f7c241e4980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.031+0000 7f7c29470640 1 --2- 172.21.15.27:0/625022906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c240b3380 0x7f7c241e4980 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f7c0c00c430 tx=0x7f7c0c00c8f0 comp rx=0 tx=0).stop 2023-12-19T18:20:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.032+0000 7f7c29470640 1 -- 172.21.15.27:0/625022906 shutdown_connections 2023-12-19T18:20:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.032+0000 7f7c29470640 1 --2- 172.21.15.27:0/625022906 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c0406a430 0x7f7c0406c8f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.032+0000 7f7c29470640 1 --2- 172.21.15.27:0/625022906 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c2415ae80 0x7f7c241e4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.032+0000 7f7c29470640 1 --2- 172.21.15.27:0/625022906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c240b3380 0x7f7c241e4980 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.032+0000 7f7c29470640 1 --2- 172.21.15.27:0/625022906 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c240b2810 0x7f7c24159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.032+0000 7f7c29470640 1 -- 172.21.15.27:0/625022906 >> 172.21.15.27:0/625022906 conn(0x7f7c2406f1f0 msgr2=0x7f7c240c3dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.032+0000 7f7c29470640 1 -- 172.21.15.27:0/625022906 shutdown_connections 2023-12-19T18:20:48.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.032+0000 7f7c29470640 1 -- 172.21.15.27:0/625022906 wait complete. 2023-12-19T18:20:48.051 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:20:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.283+0000 7f320d6f3640 1 Processor -- start 2023-12-19T18:20:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.283+0000 7f320d6f3640 1 -- start start 2023-12-19T18:20:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.285+0000 7f320d6f3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f320806ba20 0x7f3208165c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.285+0000 7f320d6f3640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32080adf10 con 0x7f320815ac50 2023-12-19T18:20:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.285+0000 7f320d6f3640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32080adac0 con 0x7f3208070fb0 2023-12-19T18:20:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.285+0000 7f320d6f3640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32080af630 con 0x7f320806ba20 2023-12-19T18:20:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.285+0000 7f3207fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f320815ac50 0x7f32080aef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54380/0 (socket says 172.21.15.27:54380) 2023-12-19T18:20:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.285+0000 7f3207fff640 1 -- 172.21.15.27:0/1243190992 learned_addr learned my addr 172.21.15.27:0/1243190992 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:48.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.285+0000 7f320cef2640 1 --2- 172.21.15.27:0/1243190992 >> v2:172.21.15.27:3300/0 conn(0x7f320806ba20 0x7f3208165c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:48.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.286+0000 7f3206ffd640 1 -- 172.21.15.27:0/1243190992 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1892766260 0 0) 0x7f32080adf10 con 0x7f320815ac50 2023-12-19T18:20:48.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.286+0000 7f320cef2640 1 -- 172.21.15.27:0/1243190992 >> v1:172.21.15.192:6789/0 conn(0x7f3208070fb0 legacy=0x7f320806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:48.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.286+0000 7f320cef2640 1 -- 172.21.15.27:0/1243190992 >> v1:172.21.15.27:6789/0 conn(0x7f320815ac50 legacy=0x7f32080aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:48.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.286+0000 7f320cef2640 1 -- 172.21.15.27:0/1243190992 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32080713f0 con 0x7f320806ba20 2023-12-19T18:20:48.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.287+0000 7f320cef2640 1 --2- 172.21.15.27:0/1243190992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320806ba20 0x7f3208165c90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3200009d90 tx=0x7f320002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dcc538c2bd37985 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:48.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.288+0000 7f3206ffd640 1 -- 172.21.15.27:0/1243190992 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320002cbd0 con 0x7f320806ba20 2023-12-19T18:20:48.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.288+0000 7f3206ffd640 1 -- 172.21.15.27:0/1243190992 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320002cd30 con 0x7f320806ba20 2023-12-19T18:20:48.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.288+0000 7f3206ffd640 1 -- 172.21.15.27:0/1243190992 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320002cea0 con 0x7f320806ba20 2023-12-19T18:20:48.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.288+0000 7f320d6f3640 1 -- 172.21.15.27:0/1243190992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320806ba20 msgr2=0x7f3208165c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:48.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.288+0000 7f320d6f3640 1 --2- 172.21.15.27:0/1243190992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320806ba20 0x7f3208165c90 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f3200009d90 tx=0x7f320002be20 comp rx=0 tx=0).stop 2023-12-19T18:20:48.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.289+0000 7f320d6f3640 1 -- 172.21.15.27:0/1243190992 shutdown_connections 2023-12-19T18:20:48.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.289+0000 7f320d6f3640 1 --2- 172.21.15.27:0/1243190992 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320806ba20 0x7f3208165c90 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:48.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.289+0000 7f320d6f3640 1 -- 172.21.15.27:0/1243190992 >> 172.21.15.27:0/1243190992 conn(0x7f320806a7d0 msgr2=0x7f32080ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:48.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.289+0000 7f320d6f3640 1 -- 172.21.15.27:0/1243190992 shutdown_connections 2023-12-19T18:20:48.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.289+0000 7f320d6f3640 1 -- 172.21.15.27:0/1243190992 wait complete. 2023-12-19T18:20:48.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.291+0000 7f320d6f3640 1 Processor -- start 2023-12-19T18:20:48.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.291+0000 7f320d6f3640 1 -- start start 2023-12-19T18:20:48.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.292+0000 7f320d6f3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f320806ba20 0x7f32080b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.292+0000 7f320d6f3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3208070fb0 0x7f32081533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.292+0000 7f320d6f3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320815ac50 0x7f32081f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.292+0000 7f320d6f3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320814d8e0 con 0x7f320815ac50 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.292+0000 7f320d6f3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32080b0b20 con 0x7f3208070fb0 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.292+0000 7f320d6f3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32080ca280 con 0x7f320806ba20 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.293+0000 7f320cef2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320815ac50 0x7f32081f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.293+0000 7f320cef2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320815ac50 0x7f32081f60b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49416/0 (socket says 172.21.15.27:49416) 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.293+0000 7f320cef2640 1 -- 172.21.15.27:0/2705131632 learned_addr learned my addr 172.21.15.27:0/2705131632 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.293+0000 7f3207fff640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f320806ba20 0x7f32080b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.293+0000 7f32077fe640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3208070fb0 0x7f32081533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.293+0000 7f320cef2640 1 -- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f320806ba20 msgr2=0x7f32080b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.293+0000 7f320cef2640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f320806ba20 0x7f32080b32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.293+0000 7f320cef2640 1 -- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3208070fb0 msgr2=0x7f32081533a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.293+0000 7f320cef2640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3208070fb0 0x7f32081533a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:48.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.293+0000 7f320cef2640 1 -- 172.21.15.27:0/2705131632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3200009a70 con 0x7f320815ac50 2023-12-19T18:20:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.294+0000 7f320cef2640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320815ac50 0x7f32081f60b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3200037040 tx=0x7f3200004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.294+0000 7f3204ff9640 1 -- 172.21.15.27:0/2705131632 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320002cea0 con 0x7f320815ac50 2023-12-19T18:20:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.294+0000 7f3204ff9640 1 -- 172.21.15.27:0/2705131632 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3200040450 con 0x7f320815ac50 2023-12-19T18:20:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.294+0000 7f3204ff9640 1 -- 172.21.15.27:0/2705131632 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32000407e0 con 0x7f320815ac50 2023-12-19T18:20:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.294+0000 7f320d6f3640 1 -- 172.21.15.27:0/2705131632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32080ca560 con 0x7f320815ac50 2023-12-19T18:20:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.294+0000 7f320d6f3640 1 -- 172.21.15.27:0/2705131632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32080b0d70 con 0x7f320815ac50 2023-12-19T18:20:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.295+0000 7f320d6f3640 1 -- 172.21.15.27:0/2705131632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320804e840 con 0x7f320815ac50 2023-12-19T18:20:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.296+0000 7f3204ff9640 1 -- 172.21.15.27:0/2705131632 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3200040940 con 0x7f320815ac50 2023-12-19T18:20:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.297+0000 7f3204ff9640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f31f406a430 0x7f31f406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.298+0000 7f3204ff9640 1 -- 172.21.15.27:0/2705131632 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f32000ab3b0 con 0x7f320815ac50 2023-12-19T18:20:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.298+0000 7f3204ff9640 1 -- 172.21.15.27:0/2705131632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=102}) v3 -- 0x7f31f4072020 con 0x7f320815ac50 2023-12-19T18:20:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.299+0000 7f3207fff640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f31f406a430 0x7f31f406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:48.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.300+0000 7f3207fff640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f31f406a430 0x7f31f406c8f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f31f8000f60 tx=0x7f31f8007b90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:48.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.303+0000 7f3204ff9640 1 -- 172.21.15.27:0/2705131632 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3200076a10 con 0x7f320815ac50 2023-12-19T18:20:48.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.513+0000 7f320d6f3640 1 -- 172.21.15.27:0/2705131632 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f32080ccfc0 con 0x7f320815ac50 2023-12-19T18:20:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.514+0000 7f3204ff9640 1 -- 172.21.15.27:0/2705131632 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v101) v1 ==== 80+0+0 (secure 0 0 0) 0x7f320007a030 con 0x7f320815ac50 2023-12-19T18:20:48.514 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.517+0000 7f320d6f3640 1 -- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f31f406a430 msgr2=0x7f31f406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.517+0000 7f320d6f3640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f31f406a430 0x7f31f406c8f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f31f8000f60 tx=0x7f31f8007b90 comp rx=0 tx=0).stop 2023-12-19T18:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.517+0000 7f320d6f3640 1 -- 172.21.15.27:0/2705131632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320815ac50 msgr2=0x7f32081f60b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.517+0000 7f320d6f3640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320815ac50 0x7f32081f60b0 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f3200037040 tx=0x7f3200004a40 comp rx=0 tx=0).stop 2023-12-19T18:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.517+0000 7f320d6f3640 1 -- 172.21.15.27:0/2705131632 shutdown_connections 2023-12-19T18:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.517+0000 7f320d6f3640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f31f406a430 0x7f31f406c8f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.517+0000 7f320d6f3640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320815ac50 0x7f32081f60b0 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.517+0000 7f320d6f3640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3208070fb0 0x7f32081533a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.517+0000 7f320d6f3640 1 --2- 172.21.15.27:0/2705131632 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f320806ba20 0x7f32080b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.517+0000 7f320d6f3640 1 -- 172.21.15.27:0/2705131632 >> 172.21.15.27:0/2705131632 conn(0x7f320806a7d0 msgr2=0x7f32080ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:48.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.518+0000 7f320d6f3640 1 -- 172.21.15.27:0/2705131632 shutdown_connections 2023-12-19T18:20:48.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:48.518+0000 7f320d6f3640 1 -- 172.21.15.27:0/2705131632 wait complete. 2023-12-19T18:20:48.537 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-19T18:20:48.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:48.537 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-19T18:20:48.636 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[88552]: fuse finished with error 0 and tester_r 0 2023-12-19T18:20:48.641 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:20:48.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:48.642 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-19T18:20:48.717 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:20:48.717 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:20:48.777 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:20:48.777 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:20:48.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:48.777 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-19T18:20:48.828 INFO:teuthology.orchestra.run.smithi027.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-19T18:20:48.829 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:20:48.830 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:20:48.830 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:20:48.890 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:20:48.891 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:20:48.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:20:48.891 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-19T18:20:48.943 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-19T18:20:48.944 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:20:48.945 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:20:49.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.187+0000 7f75db59e640 1 Processor -- start 2023-12-19T18:20:49.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.187+0000 7f75db59e640 1 -- start start 2023-12-19T18:20:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.189+0000 7f75db59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f75dc15ee30 0x7f75dc0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.189+0000 7f75db59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75dc0cb350 con 0x7f75dc0afeb0 2023-12-19T18:20:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.189+0000 7f75db59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75dc0cb630 con 0x7f75dc1553a0 2023-12-19T18:20:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.189+0000 7f75db59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75dc0cbcf0 con 0x7f75dc15ee30 2023-12-19T18:20:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.190+0000 7f75dad9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f75dc0afeb0 0x7f75dc0c39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54392/0 (socket says 172.21.15.27:54392) 2023-12-19T18:20:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.190+0000 7f75dad9d640 1 -- 172.21.15.27:0/2525021507 learned_addr learned my addr 172.21.15.27:0/2525021507 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.190+0000 7f75d3fff640 1 --2- 172.21.15.27:0/2525021507 >> v2:172.21.15.27:3300/0 conn(0x7f75dc15ee30 0x7f75dc0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.190+0000 7f75d3fff640 1 -- 172.21.15.27:0/2525021507 >> v1:172.21.15.192:6789/0 conn(0x7f75dc1553a0 legacy=0x7f75dc15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.190+0000 7f75d3fff640 1 -- 172.21.15.27:0/2525021507 >> v1:172.21.15.27:6789/0 conn(0x7f75dc0afeb0 legacy=0x7f75dc0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.190+0000 7f75d3fff640 1 -- 172.21.15.27:0/2525021507 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75dc0cb910 con 0x7f75dc15ee30 2023-12-19T18:20:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.191+0000 7f75d3fff640 1 --2- 172.21.15.27:0/2525021507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75dc15ee30 0x7f75dc0c50a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f75c40097a0 tx=0x7f75c402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403b1dfbd16fa5ae server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.192+0000 7f75d9d9b640 1 -- 172.21.15.27:0/2525021507 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c402e070 con 0x7f75dc15ee30 2023-12-19T18:20:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.192+0000 7f75d9d9b640 1 -- 172.21.15.27:0/2525021507 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75c40315a0 con 0x7f75dc15ee30 2023-12-19T18:20:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.192+0000 7f75d9d9b640 1 -- 172.21.15.27:0/2525021507 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4031930 con 0x7f75dc15ee30 2023-12-19T18:20:49.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.192+0000 7f75db59e640 1 -- 172.21.15.27:0/2525021507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75dc15ee30 msgr2=0x7f75dc0c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:49.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.192+0000 7f75db59e640 1 --2- 172.21.15.27:0/2525021507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75dc15ee30 0x7f75dc0c50a0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f75c40097a0 tx=0x7f75c402b840 comp rx=0 tx=0).stop 2023-12-19T18:20:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.192+0000 7f75db59e640 1 -- 172.21.15.27:0/2525021507 shutdown_connections 2023-12-19T18:20:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.192+0000 7f75db59e640 1 --2- 172.21.15.27:0/2525021507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75dc15ee30 0x7f75dc0c50a0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.192+0000 7f75db59e640 1 -- 172.21.15.27:0/2525021507 >> 172.21.15.27:0/2525021507 conn(0x7f75dc069c70 msgr2=0x7f75dc06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.193+0000 7f75db59e640 1 -- 172.21.15.27:0/2525021507 shutdown_connections 2023-12-19T18:20:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.193+0000 7f75db59e640 1 -- 172.21.15.27:0/2525021507 wait complete. 2023-12-19T18:20:49.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.194+0000 7f75db59e640 1 Processor -- start 2023-12-19T18:20:49.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.195+0000 7f75db59e640 1 -- start start 2023-12-19T18:20:49.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.195+0000 7f75db59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75dc0afeb0 0x7f75dc150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:49.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.195+0000 7f75db59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75dc1553a0 0x7f75dc0cde10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:49.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75db59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75dc15ee30 0x7f75dc157f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:49.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75db59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc15e3a0 con 0x7f75dc0afeb0 2023-12-19T18:20:49.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75db59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc157b00 con 0x7f75dc1553a0 2023-12-19T18:20:49.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75db59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75dc157800 con 0x7f75dc15ee30 2023-12-19T18:20:49.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75da59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75dc0afeb0 0x7f75dc150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:49.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75da59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75dc0afeb0 0x7f75dc150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49438/0 (socket says 172.21.15.27:49438) 2023-12-19T18:20:49.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75da59c640 1 -- 172.21.15.27:0/1696662680 learned_addr learned my addr 172.21.15.27:0/1696662680 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75d3fff640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75dc1553a0 0x7f75dc0cde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75dad9d640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75dc15ee30 0x7f75dc157f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75da59c640 1 -- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75dc15ee30 msgr2=0x7f75dc157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75da59c640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75dc15ee30 0x7f75dc157f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75da59c640 1 -- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75dc1553a0 msgr2=0x7f75dc0cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75da59c640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75dc1553a0 0x7f75dc0cde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75da59c640 1 -- 172.21.15.27:0/1696662680 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c4009400 con 0x7f75dc0afeb0 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75d3fff640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75dc1553a0 0x7f75dc0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.196+0000 7f75dad9d640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75dc15ee30 0x7f75dc157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.197+0000 7f75da59c640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75dc0afeb0 0x7f75dc150030 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c460 tx=0x7f75cc00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.197+0000 7f75d2ffd640 1 -- 172.21.15.27:0/1696662680 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc012830 con 0x7f75dc0afeb0 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.197+0000 7f75d2ffd640 1 -- 172.21.15.27:0/1696662680 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75cc0129d0 con 0x7f75dc0afeb0 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.197+0000 7f75d2ffd640 1 -- 172.21.15.27:0/1696662680 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75cc012d80 con 0x7f75dc0afeb0 2023-12-19T18:20:49.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.197+0000 7f75db59e640 1 -- 172.21.15.27:0/1696662680 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75dc1557f0 con 0x7f75dc0afeb0 2023-12-19T18:20:49.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.197+0000 7f75db59e640 1 -- 172.21.15.27:0/1696662680 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75dc155db0 con 0x7f75dc0afeb0 2023-12-19T18:20:49.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.199+0000 7f75db59e640 1 -- 172.21.15.27:0/1696662680 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75dc157660 con 0x7f75dc0afeb0 2023-12-19T18:20:49.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.200+0000 7f75d2ffd640 1 -- 172.21.15.27:0/1696662680 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f75cc01a3b0 con 0x7f75dc0afeb0 2023-12-19T18:20:49.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.201+0000 7f75d2ffd640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f75bc06a430 0x7f75bc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:49.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.201+0000 7f75d2ffd640 1 -- 172.21.15.27:0/1696662680 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f75cc088850 con 0x7f75dc0afeb0 2023-12-19T18:20:49.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.201+0000 7f75d3fff640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f75bc06a430 0x7f75bc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:49.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.202+0000 7f75d3fff640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f75bc06a430 0x7f75bc06c8f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f75c4036000 tx=0x7f75c403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:49.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.205+0000 7f75d2ffd640 1 -- 172.21.15.27:0/1696662680 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f75dc157660 con 0x7f75dc0afeb0 2023-12-19T18:20:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.437+0000 7f75db59e640 1 -- 172.21.15.27:0/1696662680 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f75dc157660 con 0x7f75dc0afeb0 2023-12-19T18:20:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.439+0000 7f75d2ffd640 1 -- 172.21.15.27:0/1696662680 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f75dc157660 con 0x7f75dc0afeb0 2023-12-19T18:20:49.439 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:49.439 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2860412135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2860412135},{"type":"v1","addr":"172.21.15.192:6837","nonce":2860412135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/797098667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":797098667},{"type":"v1","addr":"172.21.15.192:6836","nonce":797098667}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":6507,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/4009181263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4009181263},{"type":"v1","addr":"172.21.15.27:6833","nonce":4009181263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:20:00.264651+0000","modified":"2023-12-19T18:20:03.796309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6510},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6510":{"gid":6510,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/202939736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":202939736},{"type":"v1","addr":"172.21.15.27:6835","nonce":202939736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-19T18:20:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2023-12-19T18:20:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.442+0000 7f75db59e640 1 -- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f75bc06a430 msgr2=0x7f75bc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.442+0000 7f75db59e640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f75bc06a430 0x7f75bc06c8f0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f75c4036000 tx=0x7f75c403d040 comp rx=0 tx=0).stop 2023-12-19T18:20:49.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.442+0000 7f75db59e640 1 -- 172.21.15.27:0/1696662680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75dc0afeb0 msgr2=0x7f75dc150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.442+0000 7f75db59e640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75dc0afeb0 0x7f75dc150030 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f75cc00c460 tx=0x7f75cc00c920 comp rx=0 tx=0).stop 2023-12-19T18:20:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.442+0000 7f75db59e640 1 -- 172.21.15.27:0/1696662680 shutdown_connections 2023-12-19T18:20:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.442+0000 7f75db59e640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f75bc06a430 0x7f75bc06c8f0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.442+0000 7f75db59e640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75dc15ee30 0x7f75dc157f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.442+0000 7f75db59e640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75dc1553a0 0x7f75dc0cde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.442+0000 7f75db59e640 1 --2- 172.21.15.27:0/1696662680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75dc0afeb0 0x7f75dc150030 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.442+0000 7f75db59e640 1 -- 172.21.15.27:0/1696662680 >> 172.21.15.27:0/1696662680 conn(0x7f75dc069c70 msgr2=0x7f75dc159f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.442+0000 7f75db59e640 1 -- 172.21.15.27:0/1696662680 shutdown_connections 2023-12-19T18:20:49.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.442+0000 7f75db59e640 1 -- 172.21.15.27:0/1696662680 wait complete. 2023-12-19T18:20:49.461 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:20:49.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.697+0000 7f5efd5d4640 1 Processor -- start 2023-12-19T18:20:49.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.697+0000 7f5efd5d4640 1 -- start start 2023-12-19T18:20:49.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.700+0000 7f5efd5d4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ef815b870 0x7f5ef80c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:49.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.700+0000 7f5efd5d4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef80c4a20 con 0x7f5ef8165460 2023-12-19T18:20:49.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.700+0000 7f5efd5d4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef815bc50 con 0x7f5ef80cdf90 2023-12-19T18:20:49.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.700+0000 7f5efd5d4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef80ce750 con 0x7f5ef815b870 2023-12-19T18:20:49.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.701+0000 7f5efcdd3640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5ef8165460 0x7f5ef806f340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54406/0 (socket says 172.21.15.27:54406) 2023-12-19T18:20:49.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.701+0000 7f5efcdd3640 1 -- 172.21.15.27:0/1895044121 learned_addr learned my addr 172.21.15.27:0/1895044121 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999628411 0 0) 0x7f5ef80c4a20 con 0x7f5ef8165460 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ee0003520 con 0x7f5ef8165460 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3761506971 0 0) 0x7f5ee0003520 con 0x7f5ef8165460 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ef815b870 msgr2=0x7f5ef80c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 --2- 172.21.15.27:0/1895044121 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ef815b870 0x7f5ef80c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 >> v1:172.21.15.192:6789/0 conn(0x7f5ef80cdf90 legacy=0x7f5ef8164d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef80c4eb0 con 0x7f5ef8165460 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f5ef00019f0 con 0x7f5ef8165460 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 >> v1:172.21.15.27:6789/0 conn(0x7f5ef8165460 legacy=0x7f5ef806f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 --2- 172.21.15.27:0/1895044121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ee00039a0 0x7f5ee0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 --2- 172.21.15.27:0/1895044121 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ef815b870 0x7f5ee00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 --2- 172.21.15.27:0/1895044121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee0004d70 0x7f5ee00051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef80ce750 con 0x7f5ee0004d70 2023-12-19T18:20:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee000bb90 con 0x7f5ee00039a0 2023-12-19T18:20:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ee000bd50 con 0x7f5ef815b870 2023-12-19T18:20:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.702+0000 7f5efcdd3640 1 --2- 172.21.15.27:0/1895044121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee0004d70 0x7f5ee00051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.703+0000 7f5efcdd3640 1 -- 172.21.15.27:0/1895044121 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ef815b870 msgr2=0x7f5ee00047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.703+0000 7f5efcdd3640 1 --2- 172.21.15.27:0/1895044121 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ef815b870 0x7f5ee00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.703+0000 7f5efcdd3640 1 -- 172.21.15.27:0/1895044121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ee00039a0 msgr2=0x7f5ee0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.703+0000 7f5efcdd3640 1 --2- 172.21.15.27:0/1895044121 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ee00039a0 0x7f5ee0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.703+0000 7f5efcdd3640 1 -- 172.21.15.27:0/1895044121 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee0003520 con 0x7f5ee0004d70 2023-12-19T18:20:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.704+0000 7f5efcdd3640 1 --2- 172.21.15.27:0/1895044121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee0004d70 0x7f5ee00051f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0002230 tx=0x7f5ee00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5249615559f0aa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.704+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0034630 con 0x7f5ee0004d70 2023-12-19T18:20:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.704+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ef002f070 con 0x7f5ee0004d70 2023-12-19T18:20:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.704+0000 7f5ef6ffd640 1 -- 172.21.15.27:0/1895044121 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef00349f0 con 0x7f5ee0004d70 2023-12-19T18:20:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.705+0000 7f5efd5d4640 1 -- 172.21.15.27:0/1895044121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee0004d70 msgr2=0x7f5ee00051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.705+0000 7f5efd5d4640 1 --2- 172.21.15.27:0/1895044121 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee0004d70 0x7f5ee00051f0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0002230 tx=0x7f5ee00057f0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.705+0000 7f5efcdd3640 1 -- 172.21.15.27:0/1895044121 reap_dead start 2023-12-19T18:20:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.705+0000 7f5efd5d4640 1 -- 172.21.15.27:0/1895044121 shutdown_connections 2023-12-19T18:20:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.705+0000 7f5efd5d4640 1 -- 172.21.15.27:0/1895044121 >> 172.21.15.27:0/1895044121 conn(0x7f5ef8069c70 msgr2=0x7f5ef806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.705+0000 7f5efd5d4640 1 -- 172.21.15.27:0/1895044121 shutdown_connections 2023-12-19T18:20:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.705+0000 7f5efd5d4640 1 -- 172.21.15.27:0/1895044121 wait complete. 2023-12-19T18:20:49.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.707+0000 7f5efd5d4640 1 Processor -- start 2023-12-19T18:20:49.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.707+0000 7f5efd5d4640 1 -- start start 2023-12-19T18:20:49.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.708+0000 7f5efd5d4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee0004d70 0x7f5ef80b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:49.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.708+0000 7f5efd5d4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ef8070890 0x7f5ef814b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:49.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.708+0000 7f5efd5d4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ef814bc60 0x7f5ef814c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.708+0000 7f5efd5d4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef814c620 con 0x7f5ee0004d70 2023-12-19T18:20:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.708+0000 7f5efd5d4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef80b47e0 con 0x7f5ef814bc60 2023-12-19T18:20:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.708+0000 7f5efd5d4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef806fdf0 con 0x7f5ef8070890 2023-12-19T18:20:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.709+0000 7f5ef7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee0004d70 0x7f5ef80b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.709+0000 7f5ef7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee0004d70 0x7f5ef80b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49444/0 (socket says 172.21.15.27:49444) 2023-12-19T18:20:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.709+0000 7f5ef7fff640 1 -- 172.21.15.27:0/141637964 learned_addr learned my addr 172.21.15.27:0/141637964 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.709+0000 7f5efcdd3640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ef814bc60 0x7f5ef814c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.709+0000 7f5ef77fe640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ef8070890 0x7f5ef814b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.709+0000 7f5ef7fff640 1 -- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ef8070890 msgr2=0x7f5ef814b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.709+0000 7f5ef7fff640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ef8070890 0x7f5ef814b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.709+0000 7f5ef7fff640 1 -- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ef814bc60 msgr2=0x7f5ef814c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.709+0000 7f5ef7fff640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ef814bc60 0x7f5ef814c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.709+0000 7f5ef7fff640 1 -- 172.21.15.27:0/141637964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef80b4500 con 0x7f5ee0004d70 2023-12-19T18:20:49.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.709+0000 7f5efcdd3640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ef814bc60 0x7f5ef814c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:49.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.709+0000 7f5ef77fe640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ef8070890 0x7f5ef814b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:49.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.710+0000 7f5ef7fff640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee0004d70 0x7f5ef80b1600 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b480 tx=0x7f5ee800b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.710+0000 7f5ef4ff9640 1 -- 172.21.15.27:0/141637964 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8010870 con 0x7f5ee0004d70 2023-12-19T18:20:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.710+0000 7f5ef4ff9640 1 -- 172.21.15.27:0/141637964 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee80109d0 con 0x7f5ee0004d70 2023-12-19T18:20:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.710+0000 7f5ef4ff9640 1 -- 172.21.15.27:0/141637964 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8010d80 con 0x7f5ee0004d70 2023-12-19T18:20:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.710+0000 7f5efd5d4640 1 -- 172.21.15.27:0/141637964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef8070090 con 0x7f5ee0004d70 2023-12-19T18:20:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.710+0000 7f5efd5d4640 1 -- 172.21.15.27:0/141637964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef80b1b40 con 0x7f5ee0004d70 2023-12-19T18:20:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.711+0000 7f5efd5d4640 1 -- 172.21.15.27:0/141637964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef806fc50 con 0x7f5ee0004d70 2023-12-19T18:20:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.713+0000 7f5ef4ff9640 1 -- 172.21.15.27:0/141637964 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5ee800f3c0 con 0x7f5ee0004d70 2023-12-19T18:20:49.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.714+0000 7f5ef4ff9640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ee406a460 0x7f5ee406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:49.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.714+0000 7f5ef4ff9640 1 -- 172.21.15.27:0/141637964 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5ee8088820 con 0x7f5ee0004d70 2023-12-19T18:20:49.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.715+0000 7f5ef77fe640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ee406a460 0x7f5ee406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:49.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.716+0000 7f5ef77fe640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ee406a460 0x7f5ee406c920 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80c5b30 tx=0x7f5eec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.719+0000 7f5ef4ff9640 1 -- 172.21.15.27:0/141637964 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5ef806fc50 con 0x7f5ee0004d70 2023-12-19T18:20:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.916+0000 7f5efd5d4640 1 -- 172.21.15.27:0/141637964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ef80705b0 con 0x7f5ee0004d70 2023-12-19T18:20:49.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.919+0000 7f5ef4ff9640 1 -- 172.21.15.27:0/141637964 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+16222 (secure 0 0 0) 0x7f5ef80705b0 con 0x7f5ee0004d70 2023-12-19T18:20:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:49.919 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":103,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:20:49.695182+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:19:58.591127+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T18:19:59.313961+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["backfillfull","exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.19","mappings":[{"from":2,"to":7}]},{"pgid":"11.29","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:20:49.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.922+0000 7f5efd5d4640 1 -- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ee406a460 msgr2=0x7f5ee406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:49.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.922+0000 7f5efd5d4640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ee406a460 0x7f5ee406c920 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80c5b30 tx=0x7f5eec009040 comp rx=0 tx=0).stop 2023-12-19T18:20:49.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.922+0000 7f5efd5d4640 1 -- 172.21.15.27:0/141637964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee0004d70 msgr2=0x7f5ef80b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:49.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.922+0000 7f5efd5d4640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee0004d70 0x7f5ef80b1600 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b480 tx=0x7f5ee800b940 comp rx=0 tx=0).stop 2023-12-19T18:20:49.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.922+0000 7f5efd5d4640 1 -- 172.21.15.27:0/141637964 shutdown_connections 2023-12-19T18:20:49.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.922+0000 7f5efd5d4640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ee406a460 0x7f5ee406c920 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.922+0000 7f5efd5d4640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ef814bc60 0x7f5ef814c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.922+0000 7f5efd5d4640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ef8070890 0x7f5ef814b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.922+0000 7f5efd5d4640 1 --2- 172.21.15.27:0/141637964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ee0004d70 0x7f5ef80b1600 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:49.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.922+0000 7f5efd5d4640 1 -- 172.21.15.27:0/141637964 >> 172.21.15.27:0/141637964 conn(0x7f5ef8069c70 msgr2=0x7f5ef806b4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:49.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.922+0000 7f5efd5d4640 1 -- 172.21.15.27:0/141637964 shutdown_connections 2023-12-19T18:20:49.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:49.922+0000 7f5efd5d4640 1 -- 172.21.15.27:0/141637964 wait complete. 2023-12-19T18:20:49.943 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 4097, 'flags_names': 'hashpspool,backfillfull', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:19:58.591127+0000', 'flags': 4097, 'flags_names': 'hashpspool,backfillfull', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:19:59.313961+0000', 'flags': 4097, 'flags_names': 'hashpspool,backfillfull', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '102', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:20:49.943 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:20:50.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.166+0000 7effb5a4a640 1 Processor -- start 2023-12-19T18:20:50.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.166+0000 7effb5a4a640 1 -- start start 2023-12-19T18:20:50.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.167+0000 7effb5a4a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7effb0163b00 0x7effb0165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.167+0000 7effb5a4a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb00cd6b0 con 0x7effb00cb3e0 2023-12-19T18:20:50.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.167+0000 7effb5a4a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb00cdc70 con 0x7effb015ae80 2023-12-19T18:20:50.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.167+0000 7effb5a4a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb00b3630 con 0x7effb0163b00 2023-12-19T18:20:50.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.167+0000 7effae7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7effb00cb3e0 0x7effb00b2f20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54416/0 (socket says 172.21.15.27:54416) 2023-12-19T18:20:50.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.167+0000 7effae7fc640 1 -- 172.21.15.27:0/1069630606 learned_addr learned my addr 172.21.15.27:0/1069630606 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:50.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.168+0000 7effaf7fe640 1 --2- 172.21.15.27:0/1069630606 >> v2:172.21.15.192:3300/0 conn(0x7effb0163b00 0x7effb0165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.168+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3302601017 0 0) 0x7effb00cd6b0 con 0x7effb00cb3e0 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.168+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff9c003520 con 0x7effb00cb3e0 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.168+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1875581028 0 0) 0x7eff9c003520 con 0x7effb00cb3e0 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.168+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 >> v2:172.21.15.192:3300/0 conn(0x7effb0163b00 msgr2=0x7effb0165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.168+0000 7effadffb640 1 --2- 172.21.15.27:0/1069630606 >> v2:172.21.15.192:3300/0 conn(0x7effb0163b00 0x7effb0165ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.168+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 >> v1:172.21.15.192:6789/0 conn(0x7effb015ae80 legacy=0x7effb00b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.168+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb00d0450 con 0x7effb00cb3e0 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.168+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7eff98001670 con 0x7effb00cb3e0 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.168+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 >> v1:172.21.15.27:6789/0 conn(0x7effb00cb3e0 legacy=0x7effb00b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.168+0000 7effadffb640 1 --2- 172.21.15.27:0/1069630606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff9c0039c0 0x7eff9c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effadffb640 1 --2- 172.21.15.27:0/1069630606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff9c0047c0 0x7eff9c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effadffb640 1 --2- 172.21.15.27:0/1069630606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9c005180 0x7eff9c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb00b3630 con 0x7eff9c005180 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c00c080 con 0x7eff9c0047c0 2023-12-19T18:20:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c00c240 con 0x7eff9c0039c0 2023-12-19T18:20:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effaf7fe640 1 --2- 172.21.15.27:0/1069630606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9c005180 0x7eff9c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effae7fc640 1 --2- 172.21.15.27:0/1069630606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff9c0047c0 0x7eff9c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effaf7fe640 1 -- 172.21.15.27:0/1069630606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff9c0039c0 msgr2=0x7eff9c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effaeffd640 1 --2- 172.21.15.27:0/1069630606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff9c0039c0 0x7eff9c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effaf7fe640 1 --2- 172.21.15.27:0/1069630606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff9c0039c0 0x7eff9c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effaf7fe640 1 -- 172.21.15.27:0/1069630606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff9c0047c0 msgr2=0x7eff9c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effaf7fe640 1 --2- 172.21.15.27:0/1069630606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff9c0047c0 0x7eff9c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effaf7fe640 1 -- 172.21.15.27:0/1069630606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c003520 con 0x7eff9c005180 2023-12-19T18:20:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effae7fc640 1 --2- 172.21.15.27:0/1069630606 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff9c0047c0 0x7eff9c004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.169+0000 7effaeffd640 1 --2- 172.21.15.27:0/1069630606 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff9c0039c0 0x7eff9c003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.170+0000 7effaf7fe640 1 -- 172.21.15.27:0/1069630606 reap_dead start 2023-12-19T18:20:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.170+0000 7effaf7fe640 1 --2- 172.21.15.27:0/1069630606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9c005180 0x7eff9c00b980 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7effa8009ac0 tx=0x7eff9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a4a26228b2c7b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.171+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8004020 con 0x7eff9c005180 2023-12-19T18:20:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.171+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa8004180 con 0x7eff9c005180 2023-12-19T18:20:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.171+0000 7effadffb640 1 -- 172.21.15.27:0/1069630606 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa80044f0 con 0x7eff9c005180 2023-12-19T18:20:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.171+0000 7effb5a4a640 1 -- 172.21.15.27:0/1069630606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9c005180 msgr2=0x7eff9c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.171+0000 7effb5a4a640 1 --2- 172.21.15.27:0/1069630606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9c005180 0x7eff9c00b980 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7effa8009ac0 tx=0x7eff9c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.171+0000 7effb5a4a640 1 -- 172.21.15.27:0/1069630606 shutdown_connections 2023-12-19T18:20:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.171+0000 7effb5a4a640 1 --2- 172.21.15.27:0/1069630606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9c005180 0x7eff9c00b980 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.171+0000 7effb5a4a640 1 -- 172.21.15.27:0/1069630606 >> 172.21.15.27:0/1069630606 conn(0x7effb006f1f0 msgr2=0x7effb006f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.171+0000 7effb5a4a640 1 -- 172.21.15.27:0/1069630606 shutdown_connections 2023-12-19T18:20:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.172+0000 7effb5a4a640 1 -- 172.21.15.27:0/1069630606 wait complete. 2023-12-19T18:20:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.173+0000 7effb5a4a640 1 Processor -- start 2023-12-19T18:20:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.173+0000 7effb5a4a640 1 -- start start 2023-12-19T18:20:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.174+0000 7effb5a4a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9c005180 0x7effb014c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.174+0000 7effb5a4a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effb00c4880 0x7effb00c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.174+0000 7effb5a4a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c70c0 0x7effb00c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.174+0000 7effb5a4a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb00c8250 con 0x7eff9c005180 2023-12-19T18:20:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.174+0000 7effb5a4a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb014c7b0 con 0x7effb00c4880 2023-12-19T18:20:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.174+0000 7effb5a4a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb00c7f00 con 0x7effb00c70c0 2023-12-19T18:20:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.175+0000 7effaeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9c005180 0x7effb014c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.175+0000 7effaeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9c005180 0x7effb014c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49464/0 (socket says 172.21.15.27:49464) 2023-12-19T18:20:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.175+0000 7effaeffd640 1 -- 172.21.15.27:0/783058659 learned_addr learned my addr 172.21.15.27:0/783058659 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.175+0000 7effae7fc640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effb00c4880 0x7effb00c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.175+0000 7effaf7fe640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c70c0 0x7effb00c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.175+0000 7effaeffd640 1 -- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c70c0 msgr2=0x7effb00c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.175+0000 7effaeffd640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c70c0 0x7effb00c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.175+0000 7effaeffd640 1 -- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effb00c4880 msgr2=0x7effb00c6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.175+0000 7effaeffd640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effb00c4880 0x7effb00c6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.175+0000 7effaeffd640 1 -- 172.21.15.27:0/783058659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb00c5040 con 0x7eff9c005180 2023-12-19T18:20:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.175+0000 7effaf7fe640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c70c0 0x7effb00c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.176+0000 7effaeffd640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9c005180 0x7effb014c270 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7effa000db10 tx=0x7effa00020c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:50.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.176+0000 7effa7fff640 1 -- 172.21.15.27:0/783058659 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0003da0 con 0x7eff9c005180 2023-12-19T18:20:50.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.176+0000 7effa7fff640 1 -- 172.21.15.27:0/783058659 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa0006b50 con 0x7eff9c005180 2023-12-19T18:20:50.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.176+0000 7effa7fff640 1 -- 172.21.15.27:0/783058659 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa000b740 con 0x7eff9c005180 2023-12-19T18:20:50.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.176+0000 7effb5a4a640 1 -- 172.21.15.27:0/783058659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb00c5320 con 0x7eff9c005180 2023-12-19T18:20:50.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.176+0000 7effb5a4a640 1 -- 172.21.15.27:0/783058659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb00c4d00 con 0x7eff9c005180 2023-12-19T18:20:50.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.177+0000 7effb5a4a640 1 -- 172.21.15.27:0/783058659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb0067110 con 0x7eff9c005180 2023-12-19T18:20:50.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.178+0000 7effa7fff640 1 -- 172.21.15.27:0/783058659 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7effa000b8a0 con 0x7eff9c005180 2023-12-19T18:20:50.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.179+0000 7effa7fff640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff9006a430 0x7eff9006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.179+0000 7effa7fff640 1 -- 172.21.15.27:0/783058659 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5767+0+0 (secure 0 0 0) 0x7effa0087fc0 con 0x7eff9c005180 2023-12-19T18:20:50.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.180+0000 7effae7fc640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff9006a430 0x7eff9006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.181+0000 7effae7fc640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff9006a430 0x7eff9006c8f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7effb00ca0c0 tx=0x7eff98001510 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:50.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.183+0000 7effa7fff640 1 -- 172.21.15.27:0/783058659 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7effa0053580 con 0x7eff9c005180 2023-12-19T18:20:50.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.370+0000 7effb5a4a640 1 -- 172.21.15.27:0/783058659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7effb01564b0 con 0x7eff9c005180 2023-12-19T18:20:50.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.371+0000 7effa7fff640 1 -- 172.21.15.27:0/783058659 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7effa0018020 con 0x7eff9c005180 2023-12-19T18:20:50.371 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:20:50.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.373+0000 7effb5a4a640 1 -- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff9006a430 msgr2=0x7eff9006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:50.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.373+0000 7effb5a4a640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff9006a430 0x7eff9006c8f0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7effb00ca0c0 tx=0x7eff98001510 comp rx=0 tx=0).stop 2023-12-19T18:20:50.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.373+0000 7effb5a4a640 1 -- 172.21.15.27:0/783058659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9c005180 msgr2=0x7effb014c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:50.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.373+0000 7effb5a4a640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9c005180 0x7effb014c270 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7effa000db10 tx=0x7effa00020c0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.373+0000 7effb5a4a640 1 -- 172.21.15.27:0/783058659 shutdown_connections 2023-12-19T18:20:50.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.373+0000 7effb5a4a640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff9006a430 0x7eff9006c8f0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.373+0000 7effb5a4a640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c70c0 0x7effb00c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.373+0000 7effb5a4a640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effb00c4880 0x7effb00c6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.373+0000 7effb5a4a640 1 --2- 172.21.15.27:0/783058659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff9c005180 0x7effb014c270 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.373+0000 7effb5a4a640 1 -- 172.21.15.27:0/783058659 >> 172.21.15.27:0/783058659 conn(0x7effb006f1f0 msgr2=0x7effb00b4560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:50.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.374+0000 7effb5a4a640 1 -- 172.21.15.27:0/783058659 shutdown_connections 2023-12-19T18:20:50.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.374+0000 7effb5a4a640 1 -- 172.21.15.27:0/783058659 wait complete. 2023-12-19T18:20:50.393 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:20:50.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.617+0000 7fc66b59e640 1 Processor -- start 2023-12-19T18:20:50.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.617+0000 7fc66b59e640 1 -- start start 2023-12-19T18:20:50.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.618+0000 7fc66b59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc66c15abf0 0x7fc66c0cb690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.618+0000 7fc66b59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc66c070bd0 0x7fc66c0cd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.618+0000 7fc66b59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc66c06b1e0 con 0x7fc66c0cbbd0 2023-12-19T18:20:50.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.618+0000 7fc66b59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66c0cc020 con 0x7fc66c15abf0 2023-12-19T18:20:50.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.618+0000 7fc66b59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc66c06b450 con 0x7fc66c070bd0 2023-12-19T18:20:50.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.619+0000 7fc66a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc66c15abf0 0x7fc66c0cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.619+0000 7fc669d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc66c0cbbd0 0x7fc66c0704c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54422/0 (socket says 172.21.15.27:54422) 2023-12-19T18:20:50.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.619+0000 7fc66a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc66c15abf0 0x7fc66c0cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49480/0 (socket says 172.21.15.27:49480) 2023-12-19T18:20:50.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.619+0000 7fc669d9b640 1 -- 172.21.15.27:0/3541532669 learned_addr learned my addr 172.21.15.27:0/3541532669 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:50.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.619+0000 7fc66ad9d640 1 --2- 172.21.15.27:0/3541532669 >> v2:172.21.15.192:3300/0 conn(0x7fc66c070bd0 0x7fc66c0cd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.619+0000 7fc66a59c640 1 -- 172.21.15.27:0/3541532669 >> v2:172.21.15.192:3300/0 conn(0x7fc66c070bd0 msgr2=0x7fc66c0cd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:50.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.619+0000 7fc66a59c640 1 --2- 172.21.15.27:0/3541532669 >> v2:172.21.15.192:3300/0 conn(0x7fc66c070bd0 0x7fc66c0cd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.619+0000 7fc66a59c640 1 -- 172.21.15.27:0/3541532669 >> v1:172.21.15.27:6789/0 conn(0x7fc66c0cbbd0 legacy=0x7fc66c0704c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:50.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.619+0000 7fc66a59c640 1 -- 172.21.15.27:0/3541532669 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc66c06b730 con 0x7fc66c15abf0 2023-12-19T18:20:50.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.620+0000 7fc66a59c640 1 --2- 172.21.15.27:0/3541532669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c15abf0 0x7fc66c0cb690 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0c6ab0 tx=0x7fc65c02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6a3d68be24964e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:50.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.621+0000 7fc66959a640 1 -- 172.21.15.27:0/3541532669 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c02cd10 con 0x7fc66c15abf0 2023-12-19T18:20:50.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.621+0000 7fc66959a640 1 -- 172.21.15.27:0/3541532669 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c02ce70 con 0x7fc66c15abf0 2023-12-19T18:20:50.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.621+0000 7fc66959a640 1 -- 172.21.15.27:0/3541532669 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c034640 con 0x7fc66c15abf0 2023-12-19T18:20:50.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.621+0000 7fc66b59e640 1 -- 172.21.15.27:0/3541532669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c15abf0 msgr2=0x7fc66c0cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:50.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.621+0000 7fc66b59e640 1 --2- 172.21.15.27:0/3541532669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c15abf0 0x7fc66c0cb690 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0c6ab0 tx=0x7fc65c02bf60 comp rx=0 tx=0).stop 2023-12-19T18:20:50.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.622+0000 7fc66b59e640 1 -- 172.21.15.27:0/3541532669 shutdown_connections 2023-12-19T18:20:50.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.622+0000 7fc66b59e640 1 --2- 172.21.15.27:0/3541532669 >> v2:172.21.15.192:3300/0 conn(0x7fc66c070bd0 0x7fc66c0cd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.622+0000 7fc66b59e640 1 --2- 172.21.15.27:0/3541532669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c15abf0 0x7fc66c0cb690 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.622+0000 7fc66b59e640 1 -- 172.21.15.27:0/3541532669 >> 172.21.15.27:0/3541532669 conn(0x7fc66c0cfa60 msgr2=0x7fc66c0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:50.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.622+0000 7fc66b59e640 1 -- 172.21.15.27:0/3541532669 shutdown_connections 2023-12-19T18:20:50.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.622+0000 7fc66b59e640 1 -- 172.21.15.27:0/3541532669 wait complete. 2023-12-19T18:20:50.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.623+0000 7fc66b59e640 1 Processor -- start 2023-12-19T18:20:50.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.624+0000 7fc66b59e640 1 -- start start 2023-12-19T18:20:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.624+0000 7fc66b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c070bd0 0x7fc66c0c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.624+0000 7fc66b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc66c0cbbd0 0x7fc66c150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.624+0000 7fc66b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc66c15abf0 0x7fc66c1f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.624+0000 7fc66b59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c1f6a20 con 0x7fc66c070bd0 2023-12-19T18:20:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.625+0000 7fc66b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c150c90 con 0x7fc66c0cbbd0 2023-12-19T18:20:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.625+0000 7fc66b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc66c150f70 con 0x7fc66c15abf0 2023-12-19T18:20:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.625+0000 7fc66a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c070bd0 0x7fc66c0c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.625+0000 7fc66a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c070bd0 0x7fc66c0c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49494/0 (socket says 172.21.15.27:49494) 2023-12-19T18:20:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.625+0000 7fc66a59c640 1 -- 172.21.15.27:0/3885517313 learned_addr learned my addr 172.21.15.27:0/3885517313 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.625+0000 7fc669d9b640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc66c0cbbd0 0x7fc66c150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.625+0000 7fc66ad9d640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc66c15abf0 0x7fc66c1f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.625+0000 7fc66a59c640 1 -- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc66c15abf0 msgr2=0x7fc66c1f64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:50.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.625+0000 7fc66a59c640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc66c15abf0 0x7fc66c1f64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.625+0000 7fc66a59c640 1 -- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc66c0cbbd0 msgr2=0x7fc66c150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:50.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.625+0000 7fc66a59c640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc66c0cbbd0 0x7fc66c150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.625+0000 7fc66a59c640 1 -- 172.21.15.27:0/3885517313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c009a70 con 0x7fc66c070bd0 2023-12-19T18:20:50.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.625+0000 7fc66ad9d640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc66c15abf0 0x7fc66c1f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:50.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.626+0000 7fc66a59c640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c070bd0 0x7fc66c0c6cf0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009e80 tx=0x7fc65c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:50.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.626+0000 7fc662ffd640 1 -- 172.21.15.27:0/3885517313 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c03d600 con 0x7fc66c070bd0 2023-12-19T18:20:50.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.626+0000 7fc662ffd640 1 -- 172.21.15.27:0/3885517313 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc65c03d780 con 0x7fc66c070bd0 2023-12-19T18:20:50.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.626+0000 7fc662ffd640 1 -- 172.21.15.27:0/3885517313 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc65c03db30 con 0x7fc66c070bd0 2023-12-19T18:20:50.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.626+0000 7fc66b59e640 1 -- 172.21.15.27:0/3885517313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc66c0c8850 con 0x7fc66c070bd0 2023-12-19T18:20:50.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.626+0000 7fc66b59e640 1 -- 172.21.15.27:0/3885517313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc66c0c8540 con 0x7fc66c070bd0 2023-12-19T18:20:50.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.627+0000 7fc66b59e640 1 -- 172.21.15.27:0/3885517313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc66c0c8d70 con 0x7fc66c070bd0 2023-12-19T18:20:50.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.628+0000 7fc662ffd640 1 -- 172.21.15.27:0/3885517313 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc65c03b040 con 0x7fc66c070bd0 2023-12-19T18:20:50.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.629+0000 7fc662ffd640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc64c06a430 0x7fc64c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:50.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.629+0000 7fc662ffd640 1 -- 172.21.15.27:0/3885517313 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc65c0aaf80 con 0x7fc66c070bd0 2023-12-19T18:20:50.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.630+0000 7fc669d9b640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc64c06a430 0x7fc64c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:50.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.631+0000 7fc669d9b640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc64c06a430 0x7fc64c06c8f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0b2b90 tx=0x7fc654009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:50.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.634+0000 7fc662ffd640 1 -- 172.21.15.27:0/3885517313 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc65c0ddca0 con 0x7fc66c070bd0 2023-12-19T18:20:50.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.824+0000 7fc66b59e640 1 -- 172.21.15.27:0/3885517313 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc66c151130 con 0x7fc66c070bd0 2023-12-19T18:20:50.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.825+0000 7fc662ffd640 1 -- 172.21.15.27:0/3885517313 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v104) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc66c0c8d70 con 0x7fc66c070bd0 2023-12-19T18:20:50.825 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:20:50.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.827+0000 7fc66b59e640 1 -- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc64c06a430 msgr2=0x7fc64c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.827+0000 7fc66b59e640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc64c06a430 0x7fc64c06c8f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc66c0b2b90 tx=0x7fc654009040 comp rx=0 tx=0).stop 2023-12-19T18:20:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.827+0000 7fc66b59e640 1 -- 172.21.15.27:0/3885517313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c070bd0 msgr2=0x7fc66c0c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.827+0000 7fc66b59e640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c070bd0 0x7fc66c0c6cf0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7fc65c009e80 tx=0x7fc65c004a40 comp rx=0 tx=0).stop 2023-12-19T18:20:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.827+0000 7fc66b59e640 1 -- 172.21.15.27:0/3885517313 shutdown_connections 2023-12-19T18:20:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.827+0000 7fc66b59e640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc64c06a430 0x7fc64c06c8f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.827+0000 7fc66b59e640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc66c15abf0 0x7fc66c1f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.827+0000 7fc66b59e640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc66c0cbbd0 0x7fc66c150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.827+0000 7fc66b59e640 1 --2- 172.21.15.27:0/3885517313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc66c070bd0 0x7fc66c0c6cf0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.827+0000 7fc66b59e640 1 -- 172.21.15.27:0/3885517313 >> 172.21.15.27:0/3885517313 conn(0x7fc66c0cfa60 msgr2=0x7fc66c0b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.828+0000 7fc66b59e640 1 -- 172.21.15.27:0/3885517313 shutdown_connections 2023-12-19T18:20:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:50.828+0000 7fc66b59e640 1 -- 172.21.15.27:0/3885517313 wait complete. 2023-12-19T18:20:50.844 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:20:51.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.065+0000 7f7293428640 1 Processor -- start 2023-12-19T18:20:51.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.065+0000 7f7293428640 1 -- start start 2023-12-19T18:20:51.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.066+0000 7f7293428640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f728c15b1a0 0x7f728c06f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:51.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.066+0000 7f7293428640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f728c0c2720 0x7f728c06fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:51.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.066+0000 7f7293428640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f728c06bc80 con 0x7f728c071420 2023-12-19T18:20:51.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.066+0000 7f7293428640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728c15fd90 con 0x7f728c0c2720 2023-12-19T18:20:51.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.066+0000 7f7293428640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f728c06d690 con 0x7f728c15b1a0 2023-12-19T18:20:51.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.067+0000 7f72909c3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f728c0c2720 0x7f728c06fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:51.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.067+0000 7f72909c3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f728c0c2720 0x7f728c06fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49502/0 (socket says 172.21.15.27:49502) 2023-12-19T18:20:51.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.067+0000 7f72909c3640 1 -- 172.21.15.27:0/2751273653 learned_addr learned my addr 172.21.15.27:0/2751273653 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:51.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.067+0000 7f72911c4640 1 --2- 172.21.15.27:0/2751273653 >> v2:172.21.15.192:3300/0 conn(0x7f728c15b1a0 0x7f728c06f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:51.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.067+0000 7f72909c3640 1 -- 172.21.15.27:0/2751273653 >> v2:172.21.15.192:3300/0 conn(0x7f728c15b1a0 msgr2=0x7f728c06f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:51.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.067+0000 7f72909c3640 1 --2- 172.21.15.27:0/2751273653 >> v2:172.21.15.192:3300/0 conn(0x7f728c15b1a0 0x7f728c06f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.067+0000 7f72909c3640 1 -- 172.21.15.27:0/2751273653 >> v1:172.21.15.27:6789/0 conn(0x7f728c071420 legacy=0x7f728c0c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:51.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.067+0000 7f72909c3640 1 -- 172.21.15.27:0/2751273653 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f728c06d0a0 con 0x7f728c0c2720 2023-12-19T18:20:51.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.067+0000 7f728bfff640 1 -- 172.21.15.27:0/2751273653 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235433777 0 0) 0x7f7284001110 con 0x7f728c071420 2023-12-19T18:20:51.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.069+0000 7f72909c3640 1 --2- 172.21.15.27:0/2751273653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f728c0c2720 0x7f728c06fa60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f728c0cea40 tx=0x7f728002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41fa0ecd07dc7888 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:51.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.069+0000 7f728bfff640 1 -- 172.21.15.27:0/2751273653 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280031870 con 0x7f728c0c2720 2023-12-19T18:20:51.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.069+0000 7f728bfff640 1 -- 172.21.15.27:0/2751273653 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72800319d0 con 0x7f728c0c2720 2023-12-19T18:20:51.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.069+0000 7f728bfff640 1 -- 172.21.15.27:0/2751273653 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7280031d60 con 0x7f728c0c2720 2023-12-19T18:20:51.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.069+0000 7f7293428640 1 -- 172.21.15.27:0/2751273653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f728c0c2720 msgr2=0x7f728c06fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:51.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.070+0000 7f7293428640 1 --2- 172.21.15.27:0/2751273653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f728c0c2720 0x7f728c06fa60 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f728c0cea40 tx=0x7f728002b910 comp rx=0 tx=0).stop 2023-12-19T18:20:51.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.070+0000 7f7293428640 1 -- 172.21.15.27:0/2751273653 shutdown_connections 2023-12-19T18:20:51.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.070+0000 7f7293428640 1 --2- 172.21.15.27:0/2751273653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f728c0c2720 0x7f728c06fa60 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.070+0000 7f7293428640 1 --2- 172.21.15.27:0/2751273653 >> v2:172.21.15.192:3300/0 conn(0x7f728c15b1a0 0x7f728c06f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.070+0000 7f7293428640 1 -- 172.21.15.27:0/2751273653 >> 172.21.15.27:0/2751273653 conn(0x7f728c0cebd0 msgr2=0x7f728c0cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:51.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.070+0000 7f7293428640 1 -- 172.21.15.27:0/2751273653 shutdown_connections 2023-12-19T18:20:51.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.070+0000 7f7293428640 1 -- 172.21.15.27:0/2751273653 wait complete. 2023-12-19T18:20:51.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.072+0000 7f7293428640 1 Processor -- start 2023-12-19T18:20:51.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.072+0000 7f7293428640 1 -- start start 2023-12-19T18:20:51.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.073+0000 7f7293428640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f728c071420 0x7f728c1f5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.073+0000 7f7293428640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f728c0c2720 0x7f728c14fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.073+0000 7f7293428640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f728c15b1a0 0x7f728c0cd920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.073+0000 7f7293428640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c158430 con 0x7f728c15b1a0 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.073+0000 7f7293428640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f728c153410 con 0x7f728c071420 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.073+0000 7f7293428640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f728c1577d0 con 0x7f728c0c2720 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.073+0000 7f72919c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f728c15b1a0 0x7f728c0cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.073+0000 7f72919c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f728c15b1a0 0x7f728c0cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49512/0 (socket says 172.21.15.27:49512) 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.073+0000 7f72919c5640 1 -- 172.21.15.27:0/3624289287 learned_addr learned my addr 172.21.15.27:0/3624289287 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.073+0000 7f72911c4640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f728c071420 0x7f728c1f5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.073+0000 7f72909c3640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f728c0c2720 0x7f728c14fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.074+0000 7f72919c5640 1 -- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f728c0c2720 msgr2=0x7f728c14fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.074+0000 7f72919c5640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f728c0c2720 0x7f728c14fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.074+0000 7f72919c5640 1 -- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f728c071420 msgr2=0x7f728c1f5a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:51.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.074+0000 7f72919c5640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f728c071420 0x7f728c1f5a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.074+0000 7f72919c5640 1 -- 172.21.15.27:0/3624289287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72800094d0 con 0x7f728c15b1a0 2023-12-19T18:20:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.074+0000 7f72919c5640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f728c15b1a0 0x7f728c0cd920 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f728400c780 tx=0x7f728400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.074+0000 7f7289ffb640 1 -- 172.21.15.27:0/3624289287 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7284005070 con 0x7f728c15b1a0 2023-12-19T18:20:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.074+0000 7f7289ffb640 1 -- 172.21.15.27:0/3624289287 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72840051d0 con 0x7f728c15b1a0 2023-12-19T18:20:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.074+0000 7f7289ffb640 1 -- 172.21.15.27:0/3624289287 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f728400a640 con 0x7f728c15b1a0 2023-12-19T18:20:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.074+0000 7f7293428640 1 -- 172.21.15.27:0/3624289287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728c157a90 con 0x7f728c15b1a0 2023-12-19T18:20:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.074+0000 7f7293428640 1 -- 172.21.15.27:0/3624289287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728c158050 con 0x7f728c15b1a0 2023-12-19T18:20:51.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.076+0000 7f7293428640 1 -- 172.21.15.27:0/3624289287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728c158570 con 0x7f728c15b1a0 2023-12-19T18:20:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.076+0000 7f7289ffb640 1 -- 172.21.15.27:0/3624289287 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f728400a7a0 con 0x7f728c15b1a0 2023-12-19T18:20:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.077+0000 7f7289ffb640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f727806a430 0x7f727806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.077+0000 7f7289ffb640 1 -- 172.21.15.27:0/3624289287 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7284087c30 con 0x7f728c15b1a0 2023-12-19T18:20:51.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.078+0000 7f72911c4640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f727806a430 0x7f727806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:51.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.079+0000 7f72911c4640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f727806a430 0x7f727806c8f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f727c0099c0 tx=0x7f727c003990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:51.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.082+0000 7f7289ffb640 1 -- 172.21.15.27:0/3624289287 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f728c158570 con 0x7f728c15b1a0 2023-12-19T18:20:51.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.277+0000 7f7293428640 1 -- 172.21.15.27:0/3624289287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f728c06bc80 con 0x7f728c15b1a0 2023-12-19T18:20:51.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.278+0000 7f7289ffb640 1 -- 172.21.15.27:0/3624289287 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v104) v1 ==== 100+0+11 (secure 0 0 0) 0x7f728c06bc80 con 0x7f728c15b1a0 2023-12-19T18:20:51.278 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:20:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.280+0000 7f7293428640 1 -- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f727806a430 msgr2=0x7f727806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.280+0000 7f7293428640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f727806a430 0x7f727806c8f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f727c0099c0 tx=0x7f727c003990 comp rx=0 tx=0).stop 2023-12-19T18:20:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.280+0000 7f7293428640 1 -- 172.21.15.27:0/3624289287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f728c15b1a0 msgr2=0x7f728c0cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.280+0000 7f7293428640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f728c15b1a0 0x7f728c0cd920 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f728400c780 tx=0x7f728400cc40 comp rx=0 tx=0).stop 2023-12-19T18:20:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.280+0000 7f7293428640 1 -- 172.21.15.27:0/3624289287 shutdown_connections 2023-12-19T18:20:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.280+0000 7f7293428640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f727806a430 0x7f727806c8f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.280+0000 7f7293428640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f728c15b1a0 0x7f728c0cd920 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.280+0000 7f7293428640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f728c0c2720 0x7f728c14fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.280+0000 7f7293428640 1 --2- 172.21.15.27:0/3624289287 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f728c071420 0x7f728c1f5a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.280+0000 7f7293428640 1 -- 172.21.15.27:0/3624289287 >> 172.21.15.27:0/3624289287 conn(0x7f728c0cebd0 msgr2=0x7f728c0cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.281+0000 7f7293428640 1 -- 172.21.15.27:0/3624289287 shutdown_connections 2023-12-19T18:20:51.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.281+0000 7f7293428640 1 -- 172.21.15.27:0/3624289287 wait complete. 2023-12-19T18:20:51.299 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:20:51.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.526+0000 7f27b587e640 1 Processor -- start 2023-12-19T18:20:51.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.526+0000 7f27b587e640 1 -- start start 2023-12-19T18:20:51.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.528+0000 7f27b587e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f27b015cec0 0x7f27b0072cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:51.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.528+0000 7f27b587e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f27b015d2a0 0x7f27b00caed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:51.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.528+0000 7f27b587e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27b006b490 con 0x7f27b0160b80 2023-12-19T18:20:51.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.528+0000 7f27b587e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b01610d0 con 0x7f27b015d2a0 2023-12-19T18:20:51.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.528+0000 7f27b587e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27b01613b0 con 0x7f27b015cec0 2023-12-19T18:20:51.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.528+0000 7f27affff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f27b015d2a0 0x7f27b00caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:51.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.528+0000 7f27b507d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f27b0160b80 0x7f27b00cce00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54448/0 (socket says 172.21.15.27:54448) 2023-12-19T18:20:51.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.528+0000 7f27b507d640 1 -- 172.21.15.27:0/588618879 learned_addr learned my addr 172.21.15.27:0/588618879 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:51.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.528+0000 7f27b487c640 1 --2- 172.21.15.27:0/588618879 >> v2:172.21.15.192:3300/0 conn(0x7f27b015cec0 0x7f27b0072cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:51.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.529+0000 7f27affff640 1 -- 172.21.15.27:0/588618879 >> v2:172.21.15.192:3300/0 conn(0x7f27b015cec0 msgr2=0x7f27b0072cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:51.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.529+0000 7f27affff640 1 --2- 172.21.15.27:0/588618879 >> v2:172.21.15.192:3300/0 conn(0x7f27b015cec0 0x7f27b0072cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.529+0000 7f27affff640 1 -- 172.21.15.27:0/588618879 >> v1:172.21.15.27:6789/0 conn(0x7f27b0160b80 legacy=0x7f27b00cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.529+0000 7f27affff640 1 -- 172.21.15.27:0/588618879 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27b0161690 con 0x7f27b015d2a0 2023-12-19T18:20:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.529+0000 7f27af7fe640 1 -- 172.21.15.27:0/588618879 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2898907780 0 0) 0x7f27b006b490 con 0x7f27b0160b80 2023-12-19T18:20:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.530+0000 7f27affff640 1 --2- 172.21.15.27:0/588618879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b015d2a0 0x7f27b00caed0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f27b006ac30 tx=0x7f279c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5498b0c496d0262 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.530+0000 7f27af7fe640 1 -- 172.21.15.27:0/588618879 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c02e070 con 0x7f27b015d2a0 2023-12-19T18:20:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.530+0000 7f27af7fe640 1 -- 172.21.15.27:0/588618879 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c0315a0 con 0x7f27b015d2a0 2023-12-19T18:20:51.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.530+0000 7f27af7fe640 1 -- 172.21.15.27:0/588618879 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c031930 con 0x7f27b015d2a0 2023-12-19T18:20:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.531+0000 7f27b587e640 1 -- 172.21.15.27:0/588618879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b015d2a0 msgr2=0x7f27b00caed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.531+0000 7f27b587e640 1 --2- 172.21.15.27:0/588618879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b015d2a0 0x7f27b00caed0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f27b006ac30 tx=0x7f279c02b840 comp rx=0 tx=0).stop 2023-12-19T18:20:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.531+0000 7f27b587e640 1 -- 172.21.15.27:0/588618879 shutdown_connections 2023-12-19T18:20:51.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.531+0000 7f27b587e640 1 --2- 172.21.15.27:0/588618879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b015d2a0 0x7f27b00caed0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.531+0000 7f27b587e640 1 --2- 172.21.15.27:0/588618879 >> v2:172.21.15.192:3300/0 conn(0x7f27b015cec0 0x7f27b0072cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.531+0000 7f27b587e640 1 -- 172.21.15.27:0/588618879 >> 172.21.15.27:0/588618879 conn(0x7f27b00cf230 msgr2=0x7f27b00cf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:51.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.531+0000 7f27b587e640 1 -- 172.21.15.27:0/588618879 shutdown_connections 2023-12-19T18:20:51.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.531+0000 7f27b587e640 1 -- 172.21.15.27:0/588618879 wait complete. 2023-12-19T18:20:51.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.534+0000 7f27b587e640 1 Processor -- start 2023-12-19T18:20:51.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.534+0000 7f27b587e640 1 -- start start 2023-12-19T18:20:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.535+0000 7f27b587e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27b015cec0 0x7f27b00b3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.535+0000 7f27b587e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b015d2a0 0x7f27b01f6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.535+0000 7f27b587e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b0160b80 0x7f27b00b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.535+0000 7f27b587e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b009aaa0 con 0x7f27b015d2a0 2023-12-19T18:20:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.535+0000 7f27b587e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27b009ad80 con 0x7f27b0160b80 2023-12-19T18:20:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.535+0000 7f27b587e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27b00b3210 con 0x7f27b015cec0 2023-12-19T18:20:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.535+0000 7f27affff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b015d2a0 0x7f27b01f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.535+0000 7f27affff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b015d2a0 0x7f27b01f6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49518/0 (socket says 172.21.15.27:49518) 2023-12-19T18:20:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.535+0000 7f27affff640 1 -- 172.21.15.27:0/4163643672 learned_addr learned my addr 172.21.15.27:0/4163643672 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.536+0000 7f27b507d640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b0160b80 0x7f27b00b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:51.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.536+0000 7f27b487c640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27b015cec0 0x7f27b00b3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:51.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.536+0000 7f27affff640 1 -- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27b015cec0 msgr2=0x7f27b00b3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:51.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.536+0000 7f27affff640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27b015cec0 0x7f27b00b3dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.536+0000 7f27affff640 1 -- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b0160b80 msgr2=0x7f27b00b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:51.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.536+0000 7f27affff640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b0160b80 0x7f27b00b1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.536+0000 7f27affff640 1 -- 172.21.15.27:0/4163643672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279c009400 con 0x7f27b015d2a0 2023-12-19T18:20:51.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.536+0000 7f27b487c640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27b015cec0 0x7f27b00b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:51.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.536+0000 7f27affff640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b015d2a0 0x7f27b01f6300 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f279c02be50 tx=0x7f279c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:51.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.536+0000 7f27ad7fa640 1 -- 172.21.15.27:0/4163643672 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c032840 con 0x7f27b015d2a0 2023-12-19T18:20:51.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.536+0000 7f27ad7fa640 1 -- 172.21.15.27:0/4163643672 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279c0329a0 con 0x7f27b015d2a0 2023-12-19T18:20:51.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.536+0000 7f27ad7fa640 1 -- 172.21.15.27:0/4163643672 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279c032d50 con 0x7f27b015d2a0 2023-12-19T18:20:51.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.537+0000 7f27b587e640 1 -- 172.21.15.27:0/4163643672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27b00b4300 con 0x7f27b015d2a0 2023-12-19T18:20:51.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.537+0000 7f27b587e640 1 -- 172.21.15.27:0/4163643672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27b00b45a0 con 0x7f27b015d2a0 2023-12-19T18:20:51.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.538+0000 7f27b587e640 1 -- 172.21.15.27:0/4163643672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27b009af10 con 0x7f27b015d2a0 2023-12-19T18:20:51.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.539+0000 7f27ad7fa640 1 -- 172.21.15.27:0/4163643672 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f279c0453f0 con 0x7f27b015d2a0 2023-12-19T18:20:51.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.540+0000 7f27ad7fa640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f279406a430 0x7f279406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:51.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.540+0000 7f27ad7fa640 1 -- 172.21.15.27:0/4163643672 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f279c0aa940 con 0x7f27b015d2a0 2023-12-19T18:20:51.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.540+0000 7f27b487c640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f279406a430 0x7f279406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:51.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.541+0000 7f27b487c640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f279406a430 0x7f279406c8f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f27a4006fd0 tx=0x7f27a4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:51.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.544+0000 7f27ad7fa640 1 -- 172.21.15.27:0/4163643672 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f279c075fa0 con 0x7f27b015d2a0 2023-12-19T18:20:51.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.772+0000 7f27b587e640 1 -- 172.21.15.27:0/4163643672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27b004e840 con 0x7f27b015d2a0 2023-12-19T18:20:51.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.773+0000 7f27ad7fa640 1 -- 172.21.15.27:0/4163643672 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f279c0795c0 con 0x7f27b015d2a0 2023-12-19T18:20:51.773 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:51.773 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2860412135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2860412135},{"type":"v1","addr":"172.21.15.192:6837","nonce":2860412135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/797098667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":797098667},{"type":"v1","addr":"172.21.15.192:6836","nonce":797098667}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":6507,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/4009181263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4009181263},{"type":"v1","addr":"172.21.15.27:6833","nonce":4009181263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:20:00.264651+0000","modified":"2023-12-19T18:20:03.796309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6510},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6510":{"gid":6510,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/202939736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":202939736},{"type":"v1","addr":"172.21.15.27:6835","nonce":202939736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-19T18:20:51.774 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2023-12-19T18:20:51.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.776+0000 7f27b587e640 1 -- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f279406a430 msgr2=0x7f279406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:51.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.776+0000 7f27b587e640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f279406a430 0x7f279406c8f0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f27a4006fd0 tx=0x7f27a4008040 comp rx=0 tx=0).stop 2023-12-19T18:20:51.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.776+0000 7f27b587e640 1 -- 172.21.15.27:0/4163643672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b015d2a0 msgr2=0x7f27b01f6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:51.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.776+0000 7f27b587e640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b015d2a0 0x7f27b01f6300 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f279c02be50 tx=0x7f279c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.777+0000 7f27b587e640 1 -- 172.21.15.27:0/4163643672 shutdown_connections 2023-12-19T18:20:51.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.777+0000 7f27b587e640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f279406a430 0x7f279406c8f0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.777+0000 7f27b587e640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f27b0160b80 0x7f27b00b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.777+0000 7f27b587e640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27b015d2a0 0x7f27b01f6300 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.777+0000 7f27b587e640 1 --2- 172.21.15.27:0/4163643672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f27b015cec0 0x7f27b00b3dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:51.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.777+0000 7f27b587e640 1 -- 172.21.15.27:0/4163643672 >> 172.21.15.27:0/4163643672 conn(0x7f27b00cf230 msgr2=0x7f27b00adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:51.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.777+0000 7f27b587e640 1 -- 172.21.15.27:0/4163643672 shutdown_connections 2023-12-19T18:20:51.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:51.777+0000 7f27b587e640 1 -- 172.21.15.27:0/4163643672 wait complete. 2023-12-19T18:20:51.794 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:20:52.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.019+0000 7f078adb0640 1 Processor -- start 2023-12-19T18:20:52.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.020+0000 7f078adb0640 1 -- start start 2023-12-19T18:20:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.021+0000 7f078adb0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f07840cb300 0x7f07840cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.022+0000 7f078adb0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07840cbc20 con 0x7f0784161010 2023-12-19T18:20:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.022+0000 7f078adb0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0784145d60 con 0x7f07840c8f90 2023-12-19T18:20:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.022+0000 7f078adb0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07840c4480 con 0x7f07840cb300 2023-12-19T18:20:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.022+0000 7f078a5af640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0784161010 0x7f07840cd090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54434/0 (socket says 172.21.15.27:54434) 2023-12-19T18:20:52.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.022+0000 7f078a5af640 1 -- 172.21.15.27:0/2530098713 learned_addr learned my addr 172.21.15.27:0/2530098713 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:52.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.022+0000 7f0789dae640 1 --2- 172.21.15.27:0/2530098713 >> v2:172.21.15.192:3300/0 conn(0x7f07840cb300 0x7f07840cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:52.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.022+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 298823409 0 0) 0x7f07840cbc20 con 0x7f0784161010 2023-12-19T18:20:52.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.022+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0770003520 con 0x7f0784161010 2023-12-19T18:20:52.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1827233361 0 0) 0x7f0770003520 con 0x7f0784161010 2023-12-19T18:20:52.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 >> v2:172.21.15.192:3300/0 conn(0x7f07840cb300 msgr2=0x7f07840cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:52.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0788dac640 1 --2- 172.21.15.27:0/2530098713 >> v2:172.21.15.192:3300/0 conn(0x7f07840cb300 0x7f07840cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 >> v1:172.21.15.192:6789/0 conn(0x7f07840c8f90 legacy=0x7f0784160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0784161510 con 0x7f0784161010 2023-12-19T18:20:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f077c0019f0 con 0x7f0784161010 2023-12-19T18:20:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 >> v1:172.21.15.27:6789/0 conn(0x7f0784161010 legacy=0x7f07840cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0788dac640 1 --2- 172.21.15.27:0/2530098713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700039c0 0x7f0770003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0788dac640 1 --2- 172.21.15.27:0/2530098713 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07700047c0 0x7f0770004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0788dac640 1 --2- 172.21.15.27:0/2530098713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0770005180 0x7f077000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077000c050 con 0x7f07700039c0 2023-12-19T18:20:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077000c1c0 con 0x7f0770005180 2023-12-19T18:20:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077000c380 con 0x7f07700047c0 2023-12-19T18:20:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.023+0000 7f0789dae640 1 --2- 172.21.15.27:0/2530098713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700039c0 0x7f0770003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.024+0000 7f0789dae640 1 -- 172.21.15.27:0/2530098713 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07700047c0 msgr2=0x7f0770004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:52.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.024+0000 7f0789dae640 1 --2- 172.21.15.27:0/2530098713 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07700047c0 0x7f0770004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.024+0000 7f078a5af640 1 --2- 172.21.15.27:0/2530098713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0770005180 0x7f077000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.024+0000 7f0789dae640 1 -- 172.21.15.27:0/2530098713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0770005180 msgr2=0x7f077000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.024+0000 7f0789dae640 1 --2- 172.21.15.27:0/2530098713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0770005180 0x7f077000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.024+0000 7f0789dae640 1 -- 172.21.15.27:0/2530098713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0770003520 con 0x7f07700039c0 2023-12-19T18:20:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.024+0000 7f078a5af640 1 --2- 172.21.15.27:0/2530098713 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0770005180 0x7f077000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:52.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.024+0000 7f078a5af640 1 -- 172.21.15.27:0/2530098713 reap_dead start 2023-12-19T18:20:52.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.025+0000 7f0789dae640 1 --2- 172.21.15.27:0/2530098713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700039c0 0x7f0770003de0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0780009ac0 tx=0x7f07700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a1113b7a607ec1d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:52.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.025+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078002c6b0 con 0x7f07700039c0 2023-12-19T18:20:52.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.025+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f078002c810 con 0x7f07700039c0 2023-12-19T18:20:52.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.025+0000 7f0788dac640 1 -- 172.21.15.27:0/2530098713 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f078002cba0 con 0x7f07700039c0 2023-12-19T18:20:52.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.026+0000 7f078adb0640 1 -- 172.21.15.27:0/2530098713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700039c0 msgr2=0x7f0770003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:52.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.026+0000 7f078adb0640 1 --2- 172.21.15.27:0/2530098713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700039c0 0x7f0770003de0 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f0780009ac0 tx=0x7f07700057f0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.026+0000 7f078adb0640 1 -- 172.21.15.27:0/2530098713 shutdown_connections 2023-12-19T18:20:52.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.026+0000 7f078adb0640 1 --2- 172.21.15.27:0/2530098713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700039c0 0x7f0770003de0 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.026+0000 7f078adb0640 1 -- 172.21.15.27:0/2530098713 >> 172.21.15.27:0/2530098713 conn(0x7f0784069c70 msgr2=0x7f078406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:52.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.026+0000 7f078adb0640 1 -- 172.21.15.27:0/2530098713 shutdown_connections 2023-12-19T18:20:52.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.026+0000 7f078adb0640 1 -- 172.21.15.27:0/2530098713 wait complete. 2023-12-19T18:20:52.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.028+0000 7f078adb0640 1 Processor -- start 2023-12-19T18:20:52.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.028+0000 7f078adb0640 1 -- start start 2023-12-19T18:20:52.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.029+0000 7f078adb0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700039c0 0x7f078415c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.029+0000 7f078adb0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0784140ed0 0x7f078415da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.029+0000 7f078adb0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0784141dc0 0x7f078415df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.029+0000 7f078adb0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f078415c9b0 con 0x7f07700039c0 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.029+0000 7f078adb0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07840ae5b0 con 0x7f0784141dc0 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.029+0000 7f078adb0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07840aeb70 con 0x7f0784140ed0 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.029+0000 7f0789dae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700039c0 0x7f078415c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.029+0000 7f0789dae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700039c0 0x7f078415c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39392/0 (socket says 172.21.15.27:39392) 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.029+0000 7f0789dae640 1 -- 172.21.15.27:0/2095351631 learned_addr learned my addr 172.21.15.27:0/2095351631 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.030+0000 7f0789dae640 1 -- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0784140ed0 msgr2=0x7f078415da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.030+0000 7f07895ad640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0784140ed0 0x7f078415da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.030+0000 7f078a5af640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0784141dc0 0x7f078415df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.030+0000 7f0789dae640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0784140ed0 0x7f078415da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.030+0000 7f0789dae640 1 -- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0784141dc0 msgr2=0x7f078415df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.030+0000 7f0789dae640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0784141dc0 0x7f078415df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.030+0000 7f0789dae640 1 -- 172.21.15.27:0/2095351631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0784142370 con 0x7f07700039c0 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.030+0000 7f078a5af640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0784141dc0 0x7f078415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.030+0000 7f0789dae640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700039c0 0x7f078415c470 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0780037040 tx=0x7f0780004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:52.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.030+0000 7f077a7fc640 1 -- 172.21.15.27:0/2095351631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780003810 con 0x7f07700039c0 2023-12-19T18:20:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.030+0000 7f077a7fc640 1 -- 172.21.15.27:0/2095351631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0780003970 con 0x7f07700039c0 2023-12-19T18:20:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.031+0000 7f077a7fc640 1 -- 172.21.15.27:0/2095351631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0780003d00 con 0x7f07700039c0 2023-12-19T18:20:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.031+0000 7f078adb0640 1 -- 172.21.15.27:0/2095351631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0784142650 con 0x7f07700039c0 2023-12-19T18:20:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.031+0000 7f078adb0640 1 -- 172.21.15.27:0/2095351631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0784160090 con 0x7f07700039c0 2023-12-19T18:20:52.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.032+0000 7f078adb0640 1 -- 172.21.15.27:0/2095351631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0784142840 con 0x7f07700039c0 2023-12-19T18:20:52.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.033+0000 7f077a7fc640 1 -- 172.21.15.27:0/2095351631 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0780040450 con 0x7f07700039c0 2023-12-19T18:20:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.034+0000 7f077a7fc640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f076806a430 0x7f076806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.034+0000 7f077a7fc640 1 -- 172.21.15.27:0/2095351631 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f07800aab80 con 0x7f07700039c0 2023-12-19T18:20:52.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.035+0000 7f07895ad640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f076806a430 0x7f076806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:52.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.036+0000 7f07895ad640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f076806a430 0x7f076806c8f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f07841f5c80 tx=0x7f0774009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:52.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.040+0000 7f077a7fc640 1 -- 172.21.15.27:0/2095351631 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0780076120 con 0x7f07700039c0 2023-12-19T18:20:52.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.237+0000 7f078adb0640 1 -- 172.21.15.27:0/2095351631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f078404e840 con 0x7f07700039c0 2023-12-19T18:20:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.240+0000 7f077a7fc640 1 -- 172.21.15.27:0/2095351631 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) v1 ==== 74+0+16207 (secure 0 0 0) 0x7f0780003e60 con 0x7f07700039c0 2023-12-19T18:20:52.241 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:52.241 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":104,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:20:50.713291+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:19:58.591127+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T18:19:59.313961+0000","flags":4097,"flags_names":"hashpspool,backfillfull","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"102","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["backfillfull","exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.19","mappings":[{"from":2,"to":7}]},{"pgid":"11.29","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:20:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.243+0000 7f078adb0640 1 -- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f076806a430 msgr2=0x7f076806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.243+0000 7f078adb0640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f076806a430 0x7f076806c8f0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f07841f5c80 tx=0x7f0774009040 comp rx=0 tx=0).stop 2023-12-19T18:20:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.243+0000 7f078adb0640 1 -- 172.21.15.27:0/2095351631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700039c0 msgr2=0x7f078415c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.243+0000 7f078adb0640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700039c0 0x7f078415c470 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f0780037040 tx=0x7f0780004730 comp rx=0 tx=0).stop 2023-12-19T18:20:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.243+0000 7f078adb0640 1 -- 172.21.15.27:0/2095351631 shutdown_connections 2023-12-19T18:20:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.243+0000 7f078adb0640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f076806a430 0x7f076806c8f0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.243+0000 7f078adb0640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0784141dc0 0x7f078415df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.243+0000 7f078adb0640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0784140ed0 0x7f078415da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.243+0000 7f078adb0640 1 --2- 172.21.15.27:0/2095351631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700039c0 0x7f078415c470 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.243+0000 7f078adb0640 1 -- 172.21.15.27:0/2095351631 >> 172.21.15.27:0/2095351631 conn(0x7f0784069c70 msgr2=0x7f078406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.243+0000 7f078adb0640 1 -- 172.21.15.27:0/2095351631 shutdown_connections 2023-12-19T18:20:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.244+0000 7f078adb0640 1 -- 172.21.15.27:0/2095351631 wait complete. 2023-12-19T18:20:52.262 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:20:52.262 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:20:52.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.486+0000 7f2d70a62640 1 Processor -- start 2023-12-19T18:20:52.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.486+0000 7f2d70a62640 1 -- start start 2023-12-19T18:20:52.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.488+0000 7f2d70a62640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2d6c0cb300 0x7f2d6c0cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.488+0000 7f2d70a62640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2d6c161010 0x7f2d6c0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.488+0000 7f2d70a62640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d6c0cbc20 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.488+0000 7f2d70a62640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c0c4480 con 0x7f2d6c161010 2023-12-19T18:20:52.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.488+0000 7f2d70a62640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d6c1468c0 con 0x7f2d6c0cb300 2023-12-19T18:20:52.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.488+0000 7f2d6bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2d6c161010 0x7f2d6c0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:52.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.488+0000 7f2d6bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2d6c161010 0x7f2d6c0cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39406/0 (socket says 172.21.15.27:39406) 2023-12-19T18:20:52.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.488+0000 7f2d6bfff640 1 -- 172.21.15.27:0/2930651342 learned_addr learned my addr 172.21.15.27:0/2930651342 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:52.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.488+0000 7f2d6b7fe640 1 --2- 172.21.15.27:0/2930651342 >> v2:172.21.15.192:3300/0 conn(0x7f2d6c0cb300 0x7f2d6c0cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:52.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.488+0000 7f2d6bfff640 1 -- 172.21.15.27:0/2930651342 >> v2:172.21.15.192:3300/0 conn(0x7f2d6c0cb300 msgr2=0x7f2d6c0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:52.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.488+0000 7f2d6bfff640 1 --2- 172.21.15.27:0/2930651342 >> v2:172.21.15.192:3300/0 conn(0x7f2d6c0cb300 0x7f2d6c0cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.488+0000 7f2d6bfff640 1 -- 172.21.15.27:0/2930651342 >> v1:172.21.15.192:6789/0 conn(0x7f2d6c0c8f90 legacy=0x7f2d6c160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:52.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.489+0000 7f2d6bfff640 1 -- 172.21.15.27:0/2930651342 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d6c145d60 con 0x7f2d6c161010 2023-12-19T18:20:52.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.489+0000 7f2d6b7fe640 1 --2- 172.21.15.27:0/2930651342 >> v2:172.21.15.192:3300/0 conn(0x7f2d6c0cb300 0x7f2d6c0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:52.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.490+0000 7f2d6bfff640 1 --2- 172.21.15.27:0/2930651342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d6c161010 0x7f2d6c0cd090 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009870 tx=0x7f2d6402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc1e4e6262ee2d4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:52.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.490+0000 7f2d6a7fc640 1 -- 172.21.15.27:0/2930651342 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64031860 con 0x7f2d6c161010 2023-12-19T18:20:52.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.490+0000 7f2d6a7fc640 1 -- 172.21.15.27:0/2930651342 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d640319c0 con 0x7f2d6c161010 2023-12-19T18:20:52.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.490+0000 7f2d6a7fc640 1 -- 172.21.15.27:0/2930651342 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64031d50 con 0x7f2d6c161010 2023-12-19T18:20:52.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.491+0000 7f2d70a62640 1 -- 172.21.15.27:0/2930651342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d6c161010 msgr2=0x7f2d6c0cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:52.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.491+0000 7f2d70a62640 1 --2- 172.21.15.27:0/2930651342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d6c161010 0x7f2d6c0cd090 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009870 tx=0x7f2d6402b900 comp rx=0 tx=0).stop 2023-12-19T18:20:52.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.491+0000 7f2d70a62640 1 -- 172.21.15.27:0/2930651342 shutdown_connections 2023-12-19T18:20:52.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.491+0000 7f2d70a62640 1 --2- 172.21.15.27:0/2930651342 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d6c161010 0x7f2d6c0cd090 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.491+0000 7f2d70a62640 1 --2- 172.21.15.27:0/2930651342 >> v2:172.21.15.192:3300/0 conn(0x7f2d6c0cb300 0x7f2d6c0cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.491+0000 7f2d70a62640 1 -- 172.21.15.27:0/2930651342 >> 172.21.15.27:0/2930651342 conn(0x7f2d6c069c70 msgr2=0x7f2d6c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:52.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.491+0000 7f2d70a62640 1 -- 172.21.15.27:0/2930651342 shutdown_connections 2023-12-19T18:20:52.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.491+0000 7f2d70a62640 1 -- 172.21.15.27:0/2930651342 wait complete. 2023-12-19T18:20:52.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.493+0000 7f2d70a62640 1 Processor -- start 2023-12-19T18:20:52.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.493+0000 7f2d70a62640 1 -- start start 2023-12-19T18:20:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.494+0000 7f2d70a62640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d6c0c8f90 0x7f2d6c134190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.494+0000 7f2d70a62640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d6c0cb300 0x7f2d6c0b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.494+0000 7f2d70a62640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d6c161010 0x7f2d6c0b3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.494+0000 7f2d70a62640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0af770 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.494+0000 7f2d70a62640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0aeb60 con 0x7f2d6c161010 2023-12-19T18:20:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.494+0000 7f2d70a62640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6c0aee40 con 0x7f2d6c0cb300 2023-12-19T18:20:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.494+0000 7f2d6b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d6c0c8f90 0x7f2d6c134190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.494+0000 7f2d6b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d6c0c8f90 0x7f2d6c134190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39410/0 (socket says 172.21.15.27:39410) 2023-12-19T18:20:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.494+0000 7f2d6b7fe640 1 -- 172.21.15.27:0/384693093 learned_addr learned my addr 172.21.15.27:0/384693093 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:52.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.495+0000 7f2d6bfff640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d6c161010 0x7f2d6c0b3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.495+0000 7f2d6affd640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d6c0cb300 0x7f2d6c0b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.495+0000 7f2d6b7fe640 1 -- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d6c0cb300 msgr2=0x7f2d6c0b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.495+0000 7f2d6b7fe640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d6c0cb300 0x7f2d6c0b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.495+0000 7f2d6b7fe640 1 -- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d6c161010 msgr2=0x7f2d6c0b3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.495+0000 7f2d6b7fe640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d6c161010 0x7f2d6c0b3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.495+0000 7f2d6b7fe640 1 -- 172.21.15.27:0/384693093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d640094d0 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.495+0000 7f2d6affd640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d6c0cb300 0x7f2d6c0b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:52.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.495+0000 7f2d6b7fe640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d6c0c8f90 0x7f2d6c134190 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00c7f0 tx=0x7f2d5c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:52.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.495+0000 7f2d53fff640 1 -- 172.21.15.27:0/384693093 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c007820 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.496+0000 7f2d53fff640 1 -- 172.21.15.27:0/384693093 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d5c007980 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.496+0000 7f2d70a62640 1 -- 172.21.15.27:0/384693093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c0af120 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.496+0000 7f2d53fff640 1 -- 172.21.15.27:0/384693093 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d5c007cf0 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.496+0000 7f2d70a62640 1 -- 172.21.15.27:0/384693093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d6c1346d0 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.497+0000 7f2d70a62640 1 -- 172.21.15.27:0/384693093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d6c04e840 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.498+0000 7f2d53fff640 1 -- 172.21.15.27:0/384693093 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2d5c0113f0 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.499+0000 7f2d53fff640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2d5806a430 0x7f2d5806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.499+0000 7f2d53fff640 1 -- 172.21.15.27:0/384693093 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2d5c088cc0 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.499+0000 7f2d6affd640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2d5806a430 0x7f2d5806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:52.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.500+0000 7f2d6affd640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2d5806a430 0x7f2d5806c8f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c1385a0 tx=0x7f2d600073c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:52.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.505+0000 7f2d53fff640 1 -- 172.21.15.27:0/384693093 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2d5c08f080 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.741+0000 7f2d70a62640 1 -- 172.21.15.27:0/384693093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d6c0c4f10 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.742+0000 7f2d53fff640 1 -- 172.21.15.27:0/384693093 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f2d5c01f090 con 0x7f2d6c0c8f90 2023-12-19T18:20:52.743 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:52.743 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2860412135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2860412135},{"type":"v1","addr":"172.21.15.192:6837","nonce":2860412135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/797098667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":797098667},{"type":"v1","addr":"172.21.15.192:6836","nonce":797098667}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":6507,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/4009181263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4009181263},{"type":"v1","addr":"172.21.15.27:6833","nonce":4009181263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:20:00.264651+0000","modified":"2023-12-19T18:20:03.796309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6510},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6510":{"gid":6510,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/202939736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":202939736},{"type":"v1","addr":"172.21.15.27:6835","nonce":202939736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-19T18:20:52.743 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2023-12-19T18:20:52.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.745+0000 7f2d70a62640 1 -- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2d5806a430 msgr2=0x7f2d5806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:52.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.745+0000 7f2d70a62640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2d5806a430 0x7f2d5806c8f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c1385a0 tx=0x7f2d600073c0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.745+0000 7f2d70a62640 1 -- 172.21.15.27:0/384693093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d6c0c8f90 msgr2=0x7f2d6c134190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.745+0000 7f2d70a62640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d6c0c8f90 0x7f2d6c134190 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c00c7f0 tx=0x7f2d5c00ccb0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.745+0000 7f2d70a62640 1 -- 172.21.15.27:0/384693093 shutdown_connections 2023-12-19T18:20:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.745+0000 7f2d70a62640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2d5806a430 0x7f2d5806c8f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.745+0000 7f2d70a62640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d6c161010 0x7f2d6c0b3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.745+0000 7f2d70a62640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d6c0cb300 0x7f2d6c0b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.745+0000 7f2d70a62640 1 --2- 172.21.15.27:0/384693093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d6c0c8f90 0x7f2d6c134190 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.745+0000 7f2d70a62640 1 -- 172.21.15.27:0/384693093 >> 172.21.15.27:0/384693093 conn(0x7f2d6c069c70 msgr2=0x7f2d6c06ca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.745+0000 7f2d70a62640 1 -- 172.21.15.27:0/384693093 shutdown_connections 2023-12-19T18:20:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.745+0000 7f2d70a62640 1 -- 172.21.15.27:0/384693093 wait complete. 2023-12-19T18:20:52.766 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:20:52.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.997+0000 7fd1717bf640 1 Processor -- start 2023-12-19T18:20:52.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.997+0000 7fd1717bf640 1 -- start start 2023-12-19T18:20:52.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.998+0000 7fd1717bf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd16c0c50a0 0x7fd16c0afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:52.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.998+0000 7fd1717bf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16c0c54f0 con 0x7fd16c15ee30 2023-12-19T18:20:52.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.998+0000 7fd1717bf640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd16c0cb350 con 0x7fd16c1553a0 2023-12-19T18:20:52.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.998+0000 7fd1717bf640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd16c0cbcf0 con 0x7fd16c0c50a0 2023-12-19T18:20:52.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.999+0000 7fd16b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd16c15ee30 0x7fd16c0c39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54444/0 (socket says 172.21.15.27:54444) 2023-12-19T18:20:52.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.999+0000 7fd16b7fe640 1 -- 172.21.15.27:0/4002331227 learned_addr learned my addr 172.21.15.27:0/4002331227 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:53.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.999+0000 7fd170fbe640 1 --2- 172.21.15.27:0/4002331227 >> v2:172.21.15.192:3300/0 conn(0x7fd16c0c50a0 0x7fd16c0afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.999+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 414958513 0 0) 0x7fd160001110 con 0x7fd16c15ee30 2023-12-19T18:20:53.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:52.999+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd154003520 con 0x7fd16c15ee30 2023-12-19T18:20:53.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2373744916 0 0) 0x7fd154003520 con 0x7fd16c15ee30 2023-12-19T18:20:53.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 >> v2:172.21.15.192:3300/0 conn(0x7fd16c0c50a0 msgr2=0x7fd16c0afeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:53.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16affd640 1 --2- 172.21.15.27:0/4002331227 >> v2:172.21.15.192:3300/0 conn(0x7fd16c0c50a0 0x7fd16c0afeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 >> v1:172.21.15.192:6789/0 conn(0x7fd16c1553a0 legacy=0x7fd16c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c0cb630 con 0x7fd16c15ee30 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fd160001800 con 0x7fd16c15ee30 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 >> v1:172.21.15.27:6789/0 conn(0x7fd16c15ee30 legacy=0x7fd16c0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16affd640 1 --2- 172.21.15.27:0/4002331227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1540039c0 0x7fd154003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16affd640 1 --2- 172.21.15.27:0/4002331227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1540047c0 0x7fd154004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16affd640 1 --2- 172.21.15.27:0/4002331227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1540051d0 0x7fd15400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c0cbcf0 con 0x7fd1540039c0 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15400c120 con 0x7fd1540051d0 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd15400c2e0 con 0x7fd1540047c0 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.000+0000 7fd16bfff640 1 --2- 172.21.15.27:0/4002331227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1540039c0 0x7fd154003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.001+0000 7fd170fbe640 1 --2- 172.21.15.27:0/4002331227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1540051d0 0x7fd15400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.001+0000 7fd16bfff640 1 -- 172.21.15.27:0/4002331227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1540047c0 msgr2=0x7fd154004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.001+0000 7fd16bfff640 1 --2- 172.21.15.27:0/4002331227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd1540047c0 0x7fd154004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.001+0000 7fd16bfff640 1 -- 172.21.15.27:0/4002331227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1540051d0 msgr2=0x7fd15400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:53.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.001+0000 7fd16bfff640 1 --2- 172.21.15.27:0/4002331227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1540051d0 0x7fd15400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.001+0000 7fd16bfff640 1 -- 172.21.15.27:0/4002331227 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd154003520 con 0x7fd1540039c0 2023-12-19T18:20:53.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.001+0000 7fd170fbe640 1 --2- 172.21.15.27:0/4002331227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1540051d0 0x7fd15400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:53.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.001+0000 7fd170fbe640 1 -- 172.21.15.27:0/4002331227 reap_dead start 2023-12-19T18:20:53.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.002+0000 7fd16bfff640 1 --2- 172.21.15.27:0/4002331227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1540039c0 0x7fd154003de0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd15c001720 tx=0x7fd1540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a54bfc05ada369 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:53.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.002+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c0052d0 con 0x7fd1540039c0 2023-12-19T18:20:53.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.002+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd15c02e050 con 0x7fd1540039c0 2023-12-19T18:20:53.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.002+0000 7fd16affd640 1 -- 172.21.15.27:0/4002331227 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd15c003830 con 0x7fd1540039c0 2023-12-19T18:20:53.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.002+0000 7fd1717bf640 1 -- 172.21.15.27:0/4002331227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1540039c0 msgr2=0x7fd154003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.002+0000 7fd1717bf640 1 --2- 172.21.15.27:0/4002331227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1540039c0 0x7fd154003de0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fd15c001720 tx=0x7fd1540057f0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.003+0000 7fd1717bf640 1 -- 172.21.15.27:0/4002331227 shutdown_connections 2023-12-19T18:20:53.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.003+0000 7fd1717bf640 1 --2- 172.21.15.27:0/4002331227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1540039c0 0x7fd154003de0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.003+0000 7fd1717bf640 1 -- 172.21.15.27:0/4002331227 >> 172.21.15.27:0/4002331227 conn(0x7fd16c069c70 msgr2=0x7fd16c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:53.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.003+0000 7fd1717bf640 1 -- 172.21.15.27:0/4002331227 shutdown_connections 2023-12-19T18:20:53.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.003+0000 7fd1717bf640 1 -- 172.21.15.27:0/4002331227 wait complete. 2023-12-19T18:20:53.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.004+0000 7fd1717bf640 1 Processor -- start 2023-12-19T18:20:53.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.005+0000 7fd1717bf640 1 -- start start 2023-12-19T18:20:53.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd1717bf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1540039c0 0x7fd16c14e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd1717bf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd16c14fe40 0x7fd16c0cde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd1717bf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd16c0b15c0 0x7fd16c0b1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd1717bf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c14eeb0 con 0x7fd16c0b15c0 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd1717bf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c14eaa0 con 0x7fd1540039c0 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd1717bf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd16c150320 con 0x7fd16c14fe40 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd170fbe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd16c0b15c0 0x7fd16c0b1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd170fbe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd16c0b15c0 0x7fd16c0b1a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39436/0 (socket says 172.21.15.27:39436) 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd170fbe640 1 -- 172.21.15.27:0/2092718179 learned_addr learned my addr 172.21.15.27:0/2092718179 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd16bfff640 1 --2- 172.21.15.27:0/2092718179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1540039c0 0x7fd16c14e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd170fbe640 1 -- 172.21.15.27:0/2092718179 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd16c14fe40 msgr2=0x7fd16c0cde90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd170fbe640 1 --2- 172.21.15.27:0/2092718179 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd16c14fe40 0x7fd16c0cde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd170fbe640 1 -- 172.21.15.27:0/2092718179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1540039c0 msgr2=0x7fd16c14e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd170fbe640 1 --2- 172.21.15.27:0/2092718179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1540039c0 0x7fd16c14e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.006+0000 7fd170fbe640 1 -- 172.21.15.27:0/2092718179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd16c150600 con 0x7fd16c0b15c0 2023-12-19T18:20:53.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.007+0000 7fd170fbe640 1 --2- 172.21.15.27:0/2092718179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd16c0b15c0 0x7fd16c0b1a70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd16c071090 tx=0x7fd16400f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:53.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.007+0000 7fd168ff9640 1 -- 172.21.15.27:0/2092718179 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd164016420 con 0x7fd16c0b15c0 2023-12-19T18:20:53.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.007+0000 7fd168ff9640 1 -- 172.21.15.27:0/2092718179 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd164016580 con 0x7fd16c0b15c0 2023-12-19T18:20:53.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.007+0000 7fd1717bf640 1 -- 172.21.15.27:0/2092718179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd16c1508e0 con 0x7fd16c0b15c0 2023-12-19T18:20:53.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.007+0000 7fd168ff9640 1 -- 172.21.15.27:0/2092718179 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1640168f0 con 0x7fd16c0b15c0 2023-12-19T18:20:53.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.007+0000 7fd1717bf640 1 -- 172.21.15.27:0/2092718179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd16c0ce5b0 con 0x7fd16c0b15c0 2023-12-19T18:20:53.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.008+0000 7fd1717bf640 1 -- 172.21.15.27:0/2092718179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd16c150b20 con 0x7fd16c0b15c0 2023-12-19T18:20:53.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.010+0000 7fd168ff9640 1 -- 172.21.15.27:0/2092718179 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd164016a50 con 0x7fd16c0b15c0 2023-12-19T18:20:53.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.011+0000 7fd168ff9640 1 --2- 172.21.15.27:0/2092718179 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd15806a460 0x7fd15806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.011+0000 7fd168ff9640 1 -- 172.21.15.27:0/2092718179 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd164089780 con 0x7fd16c0b15c0 2023-12-19T18:20:53.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.012+0000 7fd16bfff640 1 --2- 172.21.15.27:0/2092718179 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd15806a460 0x7fd15806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.013+0000 7fd16bfff640 1 --2- 172.21.15.27:0/2092718179 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd15806a460 0x7fd15806c920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd15c033000 tx=0x7fd15c009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:53.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.016+0000 7fd168ff9640 1 -- 172.21.15.27:0/2092718179 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd16c150b20 con 0x7fd16c0b15c0 2023-12-19T18:20:53.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.253+0000 7fd1717bf640 1 -- 172.21.15.27:0/2092718179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd16c150b20 con 0x7fd16c0b15c0 2023-12-19T18:20:53.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.254+0000 7fd168ff9640 1 -- 172.21.15.27:0/2092718179 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) v1 ==== 94+0+4711 (secure 0 0 0) 0x7fd16c150b20 con 0x7fd16c0b15c0 2023-12-19T18:20:53.254 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:53.254 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":38,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2860412135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2860412135},{"type":"v1","addr":"172.21.15.192:6837","nonce":2860412135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/797098667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":797098667},{"type":"v1","addr":"172.21.15.192:6836","nonce":797098667}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":6507,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/4009181263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4009181263},{"type":"v1","addr":"172.21.15.27:6833","nonce":4009181263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:20:00.264651+0000","modified":"2023-12-19T18:20:03.796309+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6510},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6510":{"gid":6510,"name":"a","rank":0,"incarnation":34,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/202939736","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":202939736},{"type":"v1","addr":"172.21.15.27:6835","nonce":202939736}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-19T18:20:53.255 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 38 2023-12-19T18:20:53.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.257+0000 7fd1717bf640 1 -- 172.21.15.27:0/2092718179 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd15806a460 msgr2=0x7fd15806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.257+0000 7fd1717bf640 1 --2- 172.21.15.27:0/2092718179 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd15806a460 0x7fd15806c920 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fd15c033000 tx=0x7fd15c009030 comp rx=0 tx=0).stop 2023-12-19T18:20:53.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.257+0000 7fd1717bf640 1 -- 172.21.15.27:0/2092718179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd16c0b15c0 msgr2=0x7fd16c0b1a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.257+0000 7fd1717bf640 1 --2- 172.21.15.27:0/2092718179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd16c0b15c0 0x7fd16c0b1a70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7fd16c071090 tx=0x7fd16400f520 comp rx=0 tx=0).stop 2023-12-19T18:20:53.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.257+0000 7fd1717bf640 1 -- 172.21.15.27:0/2092718179 shutdown_connections 2023-12-19T18:20:53.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.257+0000 7fd1717bf640 1 --2- 172.21.15.27:0/2092718179 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd15806a460 0x7fd15806c920 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.257+0000 7fd1717bf640 1 --2- 172.21.15.27:0/2092718179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd16c0b15c0 0x7fd16c0b1a70 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.257+0000 7fd1717bf640 1 --2- 172.21.15.27:0/2092718179 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd16c14fe40 0x7fd16c0cde90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.257+0000 7fd1717bf640 1 --2- 172.21.15.27:0/2092718179 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd1540039c0 0x7fd16c14e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.257+0000 7fd1717bf640 1 -- 172.21.15.27:0/2092718179 >> 172.21.15.27:0/2092718179 conn(0x7fd16c069c70 msgr2=0x7fd16c070180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:53.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.258+0000 7fd1717bf640 1 -- 172.21.15.27:0/2092718179 shutdown_connections 2023-12-19T18:20:53.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.258+0000 7fd1717bf640 1 -- 172.21.15.27:0/2092718179 wait complete. 2023-12-19T18:20:53.278 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:20:53.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.510+0000 7f61f4d1c640 1 Processor -- start 2023-12-19T18:20:53.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.510+0000 7f61f4d1c640 1 -- start start 2023-12-19T18:20:53.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.511+0000 7f61f4d1c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61f00b4ef0 0x7f61f00b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.511+0000 7f61f4d1c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f00b5e90 con 0x7f61f01604d0 2023-12-19T18:20:53.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.511+0000 7f61f4d1c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f00d2850 con 0x7f61f00d2350 2023-12-19T18:20:53.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.511+0000 7f61f4d1c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f00d2b30 con 0x7f61f00b4ef0 2023-12-19T18:20:53.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.512+0000 7f61ef7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f61f01604d0 0x7f61f00b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54454/0 (socket says 172.21.15.27:54454) 2023-12-19T18:20:53.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.512+0000 7f61ef7fe640 1 -- 172.21.15.27:0/403926050 learned_addr learned my addr 172.21.15.27:0/403926050 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:53.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.512+0000 7f61eeffd640 1 --2- 172.21.15.27:0/403926050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61f00b4ef0 0x7f61f00b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.512+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360555513 0 0) 0x7f61f00b5e90 con 0x7f61f01604d0 2023-12-19T18:20:53.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.512+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61dc003520 con 0x7f61f01604d0 2023-12-19T18:20:53.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2175639270 0 0) 0x7f61dc003520 con 0x7f61f01604d0 2023-12-19T18:20:53.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61f00b4ef0 msgr2=0x7f61f00b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:53.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61ee7fc640 1 --2- 172.21.15.27:0/403926050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61f00b4ef0 0x7f61f00b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 >> v1:172.21.15.192:6789/0 conn(0x7f61f00d2350 legacy=0x7f61f00b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f00cded0 con 0x7f61f01604d0 2023-12-19T18:20:53.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f61e00019f0 con 0x7f61f01604d0 2023-12-19T18:20:53.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 >> v1:172.21.15.27:6789/0 conn(0x7f61f01604d0 legacy=0x7f61f00b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61ee7fc640 1 --2- 172.21.15.27:0/403926050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61dc0039a0 0x7f61dc003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61ee7fc640 1 --2- 172.21.15.27:0/403926050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61f00b4ef0 0x7f61dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61ee7fc640 1 --2- 172.21.15.27:0/403926050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc004d00 0x7f61dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f00d2b30 con 0x7f61dc004d00 2023-12-19T18:20:53.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc00ba60 con 0x7f61dc0039a0 2023-12-19T18:20:53.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61dc00bc20 con 0x7f61f00b4ef0 2023-12-19T18:20:53.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.513+0000 7f61effff640 1 --2- 172.21.15.27:0/403926050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc004d00 0x7f61dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.514+0000 7f61ef7fe640 1 --2- 172.21.15.27:0/403926050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61dc0039a0 0x7f61dc003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.514+0000 7f61eeffd640 1 --2- 172.21.15.27:0/403926050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61f00b4ef0 0x7f61dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.514+0000 7f61effff640 1 -- 172.21.15.27:0/403926050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61f00b4ef0 msgr2=0x7f61dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:53.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.514+0000 7f61effff640 1 --2- 172.21.15.27:0/403926050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61f00b4ef0 0x7f61dc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.514+0000 7f61effff640 1 -- 172.21.15.27:0/403926050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61dc0039a0 msgr2=0x7f61dc003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:53.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.514+0000 7f61effff640 1 --2- 172.21.15.27:0/403926050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61dc0039a0 0x7f61dc003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.514+0000 7f61effff640 1 -- 172.21.15.27:0/403926050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61dc003520 con 0x7f61dc004d00 2023-12-19T18:20:53.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.515+0000 7f61effff640 1 --2- 172.21.15.27:0/403926050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc004d00 0x7f61dc005180 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f61f00d2780 tx=0x7f61dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cac8646929791c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:53.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.515+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80052d0 con 0x7f61dc004d00 2023-12-19T18:20:53.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.515+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e802e050 con 0x7f61dc004d00 2023-12-19T18:20:53.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.515+0000 7f61ee7fc640 1 -- 172.21.15.27:0/403926050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e80035f0 con 0x7f61dc004d00 2023-12-19T18:20:53.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.516+0000 7f61f4d1c640 1 -- 172.21.15.27:0/403926050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc004d00 msgr2=0x7f61dc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.516+0000 7f61f4d1c640 1 --2- 172.21.15.27:0/403926050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc004d00 0x7f61dc005180 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f61f00d2780 tx=0x7f61dc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.516+0000 7f61effff640 1 -- 172.21.15.27:0/403926050 reap_dead start 2023-12-19T18:20:53.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.516+0000 7f61f4d1c640 1 -- 172.21.15.27:0/403926050 shutdown_connections 2023-12-19T18:20:53.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.516+0000 7f61f4d1c640 1 -- 172.21.15.27:0/403926050 >> 172.21.15.27:0/403926050 conn(0x7f61f00cf7e0 msgr2=0x7f61f00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:53.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.516+0000 7f61f4d1c640 1 -- 172.21.15.27:0/403926050 shutdown_connections 2023-12-19T18:20:53.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.516+0000 7f61f4d1c640 1 -- 172.21.15.27:0/403926050 wait complete. 2023-12-19T18:20:53.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.518+0000 7f61f4d1c640 1 Processor -- start 2023-12-19T18:20:53.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.518+0000 7f61f4d1c640 1 -- start start 2023-12-19T18:20:53.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61f4d1c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc004d00 0x7f61f0159ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61f4d1c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61f0155f00 0x7f61f0156350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61f4d1c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61f00c92f0 0x7f61f015ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61f4d1c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f015a020 con 0x7f61dc004d00 2023-12-19T18:20:53.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61f4d1c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f015a250 con 0x7f61f0155f00 2023-12-19T18:20:53.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61f4d1c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f015bbc0 con 0x7f61f00c92f0 2023-12-19T18:20:53.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc004d00 0x7f61f0159ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc004d00 0x7f61f0159ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39460/0 (socket says 172.21.15.27:39460) 2023-12-19T18:20:53.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61ef7fe640 1 -- 172.21.15.27:0/2905049745 learned_addr learned my addr 172.21.15.27:0/2905049745 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:53.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61eeffd640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61f0155f00 0x7f61f0156350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61effff640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61f00c92f0 0x7f61f015ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61ef7fe640 1 -- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61f00c92f0 msgr2=0x7f61f015ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61ef7fe640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61f00c92f0 0x7f61f015ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.519+0000 7f61ef7fe640 1 -- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61f0155f00 msgr2=0x7f61f0156350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.520+0000 7f61ef7fe640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61f0155f00 0x7f61f0156350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.520+0000 7f61ef7fe640 1 -- 172.21.15.27:0/2905049745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f00c6910 con 0x7f61dc004d00 2023-12-19T18:20:53.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.520+0000 7f61ef7fe640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc004d00 0x7f61f0159ae0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f61e0005fd0 tx=0x7f61e000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:53.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.520+0000 7f61e7fff640 1 -- 172.21.15.27:0/2905049745 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e000edf0 con 0x7f61dc004d00 2023-12-19T18:20:53.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.520+0000 7f61e7fff640 1 -- 172.21.15.27:0/2905049745 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61e0013420 con 0x7f61dc004d00 2023-12-19T18:20:53.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.520+0000 7f61e7fff640 1 -- 172.21.15.27:0/2905049745 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61e00137d0 con 0x7f61dc004d00 2023-12-19T18:20:53.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.520+0000 7f61f4d1c640 1 -- 172.21.15.27:0/2905049745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f00c6bf0 con 0x7f61dc004d00 2023-12-19T18:20:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.520+0000 7f61f4d1c640 1 -- 172.21.15.27:0/2905049745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f0156890 con 0x7f61dc004d00 2023-12-19T18:20:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.521+0000 7f61f4d1c640 1 -- 172.21.15.27:0/2905049745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f0066f80 con 0x7f61dc004d00 2023-12-19T18:20:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.522+0000 7f61e7fff640 1 -- 172.21.15.27:0/2905049745 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f61e0013930 con 0x7f61dc004d00 2023-12-19T18:20:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.523+0000 7f61e7fff640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f61d006a460 0x7f61d006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.523+0000 7f61e7fff640 1 -- 172.21.15.27:0/2905049745 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f61e00891c0 con 0x7f61dc004d00 2023-12-19T18:20:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.524+0000 7f61eeffd640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f61d006a460 0x7f61d006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.525+0000 7f61eeffd640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f61d006a460 0x7f61d006c920 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f61f0156300 tx=0x7f61d8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:53.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.528+0000 7f61e7fff640 1 -- 172.21.15.27:0/2905049745 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f61e00581f0 con 0x7f61dc004d00 2023-12-19T18:20:53.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.721+0000 7f61f4d1c640 1 -- 172.21.15.27:0/2905049745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61f004e840 con 0x7f61dc004d00 2023-12-19T18:20:53.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.723+0000 7f61e7fff640 1 -- 172.21.15.27:0/2905049745 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) v1 ==== 74+0+16144 (secure 0 0 0) 0x7f61e00018e0 con 0x7f61dc004d00 2023-12-19T18:20:53.724 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:20:53.724 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":105,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:20:52.726201+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:19:58.591127+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-19T18:19:59.313961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"10.19","mappings":[{"from":2,"to":7}]},{"pgid":"11.29","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:20:53.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.726+0000 7f61f4d1c640 1 -- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f61d006a460 msgr2=0x7f61d006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.726+0000 7f61f4d1c640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f61d006a460 0x7f61d006c920 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f61f0156300 tx=0x7f61d8008040 comp rx=0 tx=0).stop 2023-12-19T18:20:53.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.726+0000 7f61f4d1c640 1 -- 172.21.15.27:0/2905049745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc004d00 msgr2=0x7f61f0159ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.726+0000 7f61f4d1c640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc004d00 0x7f61f0159ae0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f61e0005fd0 tx=0x7f61e000e530 comp rx=0 tx=0).stop 2023-12-19T18:20:53.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.727+0000 7f61f4d1c640 1 -- 172.21.15.27:0/2905049745 shutdown_connections 2023-12-19T18:20:53.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.727+0000 7f61f4d1c640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f61d006a460 0x7f61d006c920 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.727+0000 7f61f4d1c640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61f00c92f0 0x7f61f015ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.727+0000 7f61f4d1c640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61f0155f00 0x7f61f0156350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.727+0000 7f61f4d1c640 1 --2- 172.21.15.27:0/2905049745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61dc004d00 0x7f61f0159ae0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.727+0000 7f61f4d1c640 1 -- 172.21.15.27:0/2905049745 >> 172.21.15.27:0/2905049745 conn(0x7f61f00cf7e0 msgr2=0x7f61f00b23f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:53.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.727+0000 7f61f4d1c640 1 -- 172.21.15.27:0/2905049745 shutdown_connections 2023-12-19T18:20:53.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.727+0000 7f61f4d1c640 1 -- 172.21.15.27:0/2905049745 wait complete. 2023-12-19T18:20:53.746 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:20:53.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.960+0000 7f2cc3fff640 1 Processor -- start 2023-12-19T18:20:53.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.961+0000 7f2cc3fff640 1 -- start start 2023-12-19T18:20:53.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.962+0000 7f2cc3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2cc415d180 0x7f2cc40d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.962+0000 7f2cc3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cc40d2250 0x7f2cc40b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.962+0000 7f2cc3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2cc40b4750 0x7f2cc40b4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.962+0000 7f2cc3fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc415d560 con 0x7f2cc40b4750 2023-12-19T18:20:53.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.962+0000 7f2cc3fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cc40b5120 con 0x7f2cc415d180 2023-12-19T18:20:53.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.962+0000 7f2cc3fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc40d1dc0 con 0x7f2cc40d2250 2023-12-19T18:20:53.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.963+0000 7f2cc37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2cc40b4750 0x7f2cc40b4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.963+0000 7f2cc37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2cc40b4750 0x7f2cc40b4be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39474/0 (socket says 172.21.15.27:39474) 2023-12-19T18:20:53.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.963+0000 7f2cc37fe640 1 -- 172.21.15.27:0/2713795928 learned_addr learned my addr 172.21.15.27:0/2713795928 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:53.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.963+0000 7f2cc27fc640 1 --2- 172.21.15.27:0/2713795928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cc40d2250 0x7f2cc40b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.963+0000 7f2cc2ffd640 1 --2- 172.21.15.27:0/2713795928 >> v2:172.21.15.192:3300/0 conn(0x7f2cc415d180 0x7f2cc40d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.963+0000 7f2cc37fe640 1 -- 172.21.15.27:0/2713795928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cc40d2250 msgr2=0x7f2cc40b20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:53.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.963+0000 7f2cc37fe640 1 --2- 172.21.15.27:0/2713795928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cc40d2250 0x7f2cc40b20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.963+0000 7f2cc37fe640 1 -- 172.21.15.27:0/2713795928 >> v2:172.21.15.192:3300/0 conn(0x7f2cc415d180 msgr2=0x7f2cc40d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:53.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.963+0000 7f2cc37fe640 1 --2- 172.21.15.27:0/2713795928 >> v2:172.21.15.192:3300/0 conn(0x7f2cc415d180 0x7f2cc40d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.963+0000 7f2cc37fe640 1 -- 172.21.15.27:0/2713795928 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cc40d2690 con 0x7f2cc40b4750 2023-12-19T18:20:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.964+0000 7f2cc37fe640 1 --2- 172.21.15.27:0/2713795928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cc40b4750 0x7f2cc40b4be0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4068520 tx=0x7f2cbc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c6d7a339d2f0261 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.965+0000 7f2cc1ffb640 1 -- 172.21.15.27:0/2713795928 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc030860 con 0x7f2cc40b4750 2023-12-19T18:20:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.965+0000 7f2cc1ffb640 1 -- 172.21.15.27:0/2713795928 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cbc0309c0 con 0x7f2cc40b4750 2023-12-19T18:20:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.965+0000 7f2cc1ffb640 1 -- 172.21.15.27:0/2713795928 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cbc030d50 con 0x7f2cc40b4750 2023-12-19T18:20:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.965+0000 7f2cc3fff640 1 -- 172.21.15.27:0/2713795928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cc40b4750 msgr2=0x7f2cc40b4be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.965+0000 7f2cc3fff640 1 --2- 172.21.15.27:0/2713795928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cc40b4750 0x7f2cc40b4be0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f2cc4068520 tx=0x7f2cbc02b9a0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.965+0000 7f2cc3fff640 1 -- 172.21.15.27:0/2713795928 shutdown_connections 2023-12-19T18:20:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.965+0000 7f2cc3fff640 1 --2- 172.21.15.27:0/2713795928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cc40b4750 0x7f2cc40b4be0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.965+0000 7f2cc3fff640 1 --2- 172.21.15.27:0/2713795928 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cc40d2250 0x7f2cc40b20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.965+0000 7f2cc3fff640 1 --2- 172.21.15.27:0/2713795928 >> v2:172.21.15.192:3300/0 conn(0x7f2cc415d180 0x7f2cc40d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.965+0000 7f2cc3fff640 1 -- 172.21.15.27:0/2713795928 >> 172.21.15.27:0/2713795928 conn(0x7f2cc40cf750 msgr2=0x7f2cc40cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:53.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.965+0000 7f2cc3fff640 1 -- 172.21.15.27:0/2713795928 shutdown_connections 2023-12-19T18:20:53.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.965+0000 7f2cc3fff640 1 -- 172.21.15.27:0/2713795928 wait complete. 2023-12-19T18:20:53.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.967+0000 7f2cc3fff640 1 Processor -- start 2023-12-19T18:20:53.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.967+0000 7f2cc3fff640 1 -- start start 2023-12-19T18:20:53.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.968+0000 7f2cc3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cc40b4750 0x7f2cc41520f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.968+0000 7f2cc3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cc40d2250 0x7f2cc415e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.968+0000 7f2cc3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cc415d180 0x7f2cc40b6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.968+0000 7f2cc3fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc4154060 con 0x7f2cc40d2250 2023-12-19T18:20:53.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.968+0000 7f2cc3fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc4153450 con 0x7f2cc40b4750 2023-12-19T18:20:53.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.968+0000 7f2cc3fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cc4153730 con 0x7f2cc415d180 2023-12-19T18:20:53.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.968+0000 7f2cc27fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cc40d2250 0x7f2cc415e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.968+0000 7f2cc27fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cc40d2250 0x7f2cc415e0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39476/0 (socket says 172.21.15.27:39476) 2023-12-19T18:20:53.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.968+0000 7f2cc27fc640 1 -- 172.21.15.27:0/1571063459 learned_addr learned my addr 172.21.15.27:0/1571063459 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:53.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.969+0000 7f2cc37fe640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cc415d180 0x7f2cc40b6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.969+0000 7f2cc27fc640 1 -- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cc415d180 msgr2=0x7f2cc40b6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.969+0000 7f2cc27fc640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cc415d180 0x7f2cc40b6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.969+0000 7f2cc27fc640 1 -- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cc40b4750 msgr2=0x7f2cc41520f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:53.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.969+0000 7f2cc2ffd640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cc40b4750 0x7f2cc41520f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.969+0000 7f2cc27fc640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cc40b4750 0x7f2cc41520f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:53.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.969+0000 7f2cc27fc640 1 -- 172.21.15.27:0/1571063459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc0094d0 con 0x7f2cc40d2250 2023-12-19T18:20:53.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.969+0000 7f2cc27fc640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cc40d2250 0x7f2cc415e0e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00d800 tx=0x7f2cac00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:53.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.969+0000 7f2cbb7fe640 1 -- 172.21.15.27:0/1571063459 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac0096a0 con 0x7f2cc40d2250 2023-12-19T18:20:53.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.969+0000 7f2cbb7fe640 1 -- 172.21.15.27:0/1571063459 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cac009800 con 0x7f2cc40d2250 2023-12-19T18:20:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.969+0000 7f2cbb7fe640 1 -- 172.21.15.27:0/1571063459 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cac009b90 con 0x7f2cc40d2250 2023-12-19T18:20:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.969+0000 7f2cc3fff640 1 -- 172.21.15.27:0/1571063459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cc4153a10 con 0x7f2cc40d2250 2023-12-19T18:20:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.970+0000 7f2cc3fff640 1 -- 172.21.15.27:0/1571063459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cc4152630 con 0x7f2cc40d2250 2023-12-19T18:20:53.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.970+0000 7f2cc3fff640 1 -- 172.21.15.27:0/1571063459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cc404e840 con 0x7f2cc40d2250 2023-12-19T18:20:53.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.971+0000 7f2cbb7fe640 1 -- 172.21.15.27:0/1571063459 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2cac009cf0 con 0x7f2cc40d2250 2023-12-19T18:20:53.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.972+0000 7f2cbb7fe640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ca406a430 0x7f2ca406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:53.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.972+0000 7f2cbb7fe640 1 -- 172.21.15.27:0/1571063459 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f2cac0890e0 con 0x7f2cc40d2250 2023-12-19T18:20:53.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.973+0000 7f2cc2ffd640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ca406a430 0x7f2ca406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:53.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.974+0000 7f2cc2ffd640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ca406a430 0x7f2ca406c8f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009e90 tx=0x7f2cb400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:53.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:53.977+0000 7f2cbb7fe640 1 -- 172.21.15.27:0/1571063459 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2cac054690 con 0x7f2cc40d2250 2023-12-19T18:20:54.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.211+0000 7f2cc3fff640 1 -- 172.21.15.27:0/1571063459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2cc40659a0 con 0x7f2cc40d2250 2023-12-19T18:20:54.214 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:20:54.214+0000 7f8f3865c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:20:54.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.226+0000 7f2cbb7fe640 1 -- 172.21.15.27:0/1571063459 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v39) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2cac00b440 con 0x7f2cc40d2250 2023-12-19T18:20:54.226 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:20:54.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.229+0000 7f2cc3fff640 1 -- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ca406a430 msgr2=0x7f2ca406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:54.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.229+0000 7f2cc3fff640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ca406a430 0x7f2ca406c8f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009e90 tx=0x7f2cb400b040 comp rx=0 tx=0).stop 2023-12-19T18:20:54.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.229+0000 7f2cc3fff640 1 -- 172.21.15.27:0/1571063459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cc40d2250 msgr2=0x7f2cc415e0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:54.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.229+0000 7f2cc3fff640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cc40d2250 0x7f2cc415e0e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f2cac00d800 tx=0x7f2cac00dcc0 comp rx=0 tx=0).stop 2023-12-19T18:20:54.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.229+0000 7f2cc3fff640 1 -- 172.21.15.27:0/1571063459 shutdown_connections 2023-12-19T18:20:54.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.229+0000 7f2cc3fff640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ca406a430 0x7f2ca406c8f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:54.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.229+0000 7f2cc3fff640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cc415d180 0x7f2cc40b6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:54.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.229+0000 7f2cc3fff640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cc40d2250 0x7f2cc415e0e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:54.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.229+0000 7f2cc3fff640 1 --2- 172.21.15.27:0/1571063459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cc40b4750 0x7f2cc41520f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:54.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.229+0000 7f2cc3fff640 1 -- 172.21.15.27:0/1571063459 >> 172.21.15.27:0/1571063459 conn(0x7f2cc40cf750 msgr2=0x7f2cc40b2020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:54.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.230+0000 7f2cc3fff640 1 -- 172.21.15.27:0/1571063459 shutdown_connections 2023-12-19T18:20:54.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.230+0000 7f2cc3fff640 1 -- 172.21.15.27:0/1571063459 wait complete. 2023-12-19T18:20:54.250 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:20:54.355 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:20:54.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.472+0000 7f363000b640 1 Processor -- start 2023-12-19T18:20:54.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.472+0000 7f363000b640 1 -- start start 2023-12-19T18:20:54.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.473+0000 7f363000b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3628071410 0x7f362815f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:54.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.473+0000 7f363000b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f362815b070 0x7f362806ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:54.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.473+0000 7f363000b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f362806cf60 con 0x7f362815ac90 2023-12-19T18:20:54.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.473+0000 7f363000b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f362815eb40 con 0x7f362815b070 2023-12-19T18:20:54.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.473+0000 7f363000b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362806f770 con 0x7f3628071410 2023-12-19T18:20:54.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.474+0000 7f362dda7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f362815ac90 0x7f3628070d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54480/0 (socket says 172.21.15.27:54480) 2023-12-19T18:20:54.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.474+0000 7f362dda7640 1 -- 172.21.15.27:0/139315397 learned_addr learned my addr 172.21.15.27:0/139315397 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:54.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.474+0000 7f362e5a8640 1 --2- 172.21.15.27:0/139315397 >> v2:172.21.15.27:3300/0 conn(0x7f362815b070 0x7f362806ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:54.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.474+0000 7f362d5a6640 1 --2- 172.21.15.27:0/139315397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3628071410 0x7f362815f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:54.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.474+0000 7f362cda5640 1 -- 172.21.15.27:0/139315397 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032135377 0 0) 0x7f3624001110 con 0x7f362815ac90 2023-12-19T18:20:54.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.474+0000 7f362cda5640 1 -- 172.21.15.27:0/139315397 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3614003520 con 0x7f362815ac90 2023-12-19T18:20:54.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.474+0000 7f362e5a8640 1 -- 172.21.15.27:0/139315397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3628071410 msgr2=0x7f362815f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:54.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.474+0000 7f362e5a8640 1 --2- 172.21.15.27:0/139315397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3628071410 0x7f362815f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:54.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.474+0000 7f362e5a8640 1 -- 172.21.15.27:0/139315397 >> v1:172.21.15.27:6789/0 conn(0x7f362815ac90 legacy=0x7f3628070d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:54.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.474+0000 7f362e5a8640 1 -- 172.21.15.27:0/139315397 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36280cb050 con 0x7f362815b070 2023-12-19T18:20:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.476+0000 7f362e5a8640 1 --2- 172.21.15.27:0/139315397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f362815b070 0x7f362806ca20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3620009d90 tx=0x7f362002bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=babf3e175810c60a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.476+0000 7f362cda5640 1 -- 172.21.15.27:0/139315397 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362002cce0 con 0x7f362815b070 2023-12-19T18:20:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.476+0000 7f362cda5640 1 -- 172.21.15.27:0/139315397 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362002ce40 con 0x7f362815b070 2023-12-19T18:20:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.476+0000 7f362cda5640 1 -- 172.21.15.27:0/139315397 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3620034660 con 0x7f362815b070 2023-12-19T18:20:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.476+0000 7f363000b640 1 -- 172.21.15.27:0/139315397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f362815b070 msgr2=0x7f362806ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.476+0000 7f363000b640 1 --2- 172.21.15.27:0/139315397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f362815b070 0x7f362806ca20 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f3620009d90 tx=0x7f362002bf30 comp rx=0 tx=0).stop 2023-12-19T18:20:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.477+0000 7f363000b640 1 -- 172.21.15.27:0/139315397 shutdown_connections 2023-12-19T18:20:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.477+0000 7f363000b640 1 --2- 172.21.15.27:0/139315397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f362815b070 0x7f362806ca20 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.477+0000 7f363000b640 1 --2- 172.21.15.27:0/139315397 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3628071410 0x7f362815f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.477+0000 7f363000b640 1 -- 172.21.15.27:0/139315397 >> 172.21.15.27:0/139315397 conn(0x7f36280cb5b0 msgr2=0x7f36280cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.477+0000 7f363000b640 1 -- 172.21.15.27:0/139315397 shutdown_connections 2023-12-19T18:20:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.477+0000 7f363000b640 1 -- 172.21.15.27:0/139315397 wait complete. 2023-12-19T18:20:54.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.479+0000 7f363000b640 1 Processor -- start 2023-12-19T18:20:54.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.479+0000 7f363000b640 1 -- start start 2023-12-19T18:20:54.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.480+0000 7f363000b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3628071410 0x7f362814cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:54.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.480+0000 7f363000b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f362815ac90 0x7f362815dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:54.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.480+0000 7f363000b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f362815b070 0x7f362815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.480+0000 7f363000b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362814d480 con 0x7f362815b070 2023-12-19T18:20:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.480+0000 7f363000b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f362814eed0 con 0x7f362815ac90 2023-12-19T18:20:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.480+0000 7f363000b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f362814f1b0 con 0x7f3628071410 2023-12-19T18:20:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.480+0000 7f362e5a8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f362815b070 0x7f362815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.480+0000 7f362e5a8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f362815b070 0x7f362815b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39516/0 (socket says 172.21.15.27:39516) 2023-12-19T18:20:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.480+0000 7f362e5a8640 1 -- 172.21.15.27:0/1414949821 learned_addr learned my addr 172.21.15.27:0/1414949821 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.480+0000 7f362dda7640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3628071410 0x7f362814cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.480+0000 7f362d5a6640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f362815ac90 0x7f362815dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.481+0000 7f362e5a8640 1 -- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3628071410 msgr2=0x7f362814cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.481+0000 7f362e5a8640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3628071410 0x7f362814cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.481+0000 7f362e5a8640 1 -- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f362815ac90 msgr2=0x7f362815dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.481+0000 7f362e5a8640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f362815ac90 0x7f362815dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.481+0000 7f362e5a8640 1 -- 172.21.15.27:0/1414949821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3620009a70 con 0x7f362815b070 2023-12-19T18:20:54.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.481+0000 7f362e5a8640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f362815b070 0x7f362815b7f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3620009d90 tx=0x7f3620004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:54.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.481+0000 7f361a7fc640 1 -- 172.21.15.27:0/1414949821 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362003d600 con 0x7f362815b070 2023-12-19T18:20:54.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.481+0000 7f361a7fc640 1 -- 172.21.15.27:0/1414949821 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362003d780 con 0x7f362815b070 2023-12-19T18:20:54.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.481+0000 7f363000b640 1 -- 172.21.15.27:0/1414949821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f362814f490 con 0x7f362815b070 2023-12-19T18:20:54.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.481+0000 7f361a7fc640 1 -- 172.21.15.27:0/1414949821 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362003db30 con 0x7f362815b070 2023-12-19T18:20:54.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.482+0000 7f363000b640 1 -- 172.21.15.27:0/1414949821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f362815e430 con 0x7f362815b070 2023-12-19T18:20:54.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.483+0000 7f363000b640 1 -- 172.21.15.27:0/1414949821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3628067170 con 0x7f362815b070 2023-12-19T18:20:54.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.484+0000 7f361a7fc640 1 -- 172.21.15.27:0/1414949821 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f362003b040 con 0x7f362815b070 2023-12-19T18:20:54.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.485+0000 7f361a7fc640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f360c06a430 0x7f360c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:54.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.485+0000 7f361a7fc640 1 -- 172.21.15.27:0/1414949821 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f36200aafa0 con 0x7f362815b070 2023-12-19T18:20:54.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.486+0000 7f362dda7640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f360c06a430 0x7f360c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.487+0000 7f362dda7640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f360c06a430 0x7f360c06c8f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3624007fd0 tx=0x7f3624009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:54.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.489+0000 7f361a7fc640 1 -- 172.21.15.27:0/1414949821 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3620076610 con 0x7f362815b070 2023-12-19T18:20:54.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:54.716+0000 7f363000b640 1 -- 172.21.15.27:0/1414949821 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f3628068e90 con 0x7f362815b070 2023-12-19T18:20:55.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.232+0000 7f361a7fc640 1 -- 172.21.15.27:0/1414949821 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v40) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3620079c30 con 0x7f362815b070 2023-12-19T18:20:55.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.235+0000 7f363000b640 1 -- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f360c06a430 msgr2=0x7f360c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:55.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.235+0000 7f363000b640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f360c06a430 0x7f360c06c8f0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3624007fd0 tx=0x7f3624009040 comp rx=0 tx=0).stop 2023-12-19T18:20:55.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.235+0000 7f363000b640 1 -- 172.21.15.27:0/1414949821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f362815b070 msgr2=0x7f362815b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:55.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.235+0000 7f363000b640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f362815b070 0x7f362815b7f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f3620009d90 tx=0x7f3620004a40 comp rx=0 tx=0).stop 2023-12-19T18:20:55.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.235+0000 7f363000b640 1 -- 172.21.15.27:0/1414949821 shutdown_connections 2023-12-19T18:20:55.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.235+0000 7f363000b640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f360c06a430 0x7f360c06c8f0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:55.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.235+0000 7f363000b640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f362815b070 0x7f362815b7f0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:55.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.235+0000 7f363000b640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f362815ac90 0x7f362815dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:55.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.235+0000 7f363000b640 1 --2- 172.21.15.27:0/1414949821 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3628071410 0x7f362814cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:55.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.235+0000 7f363000b640 1 -- 172.21.15.27:0/1414949821 >> 172.21.15.27:0/1414949821 conn(0x7f36280cb5b0 msgr2=0x7f36280ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:55.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.235+0000 7f363000b640 1 -- 172.21.15.27:0/1414949821 shutdown_connections 2023-12-19T18:20:55.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.235+0000 7f363000b640 1 -- 172.21.15.27:0/1414949821 wait complete. 2023-12-19T18:20:55.256 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:20:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.492+0000 7fe9cb59e640 1 Processor -- start 2023-12-19T18:20:55.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.492+0000 7fe9cb59e640 1 -- start start 2023-12-19T18:20:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.493+0000 7fe9cb59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9cc15cfc0 0x7fe9cc160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.494+0000 7fe9cb59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe9cc0d27b0 0x7fe9cc0cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.494+0000 7fe9cb59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9cc0ce4d0 con 0x7fe9cc0cdb30 2023-12-19T18:20:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.494+0000 7fe9cb59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9cc161a50 con 0x7fe9cc0d27b0 2023-12-19T18:20:55.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.494+0000 7fe9cb59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc161490 con 0x7fe9cc15cfc0 2023-12-19T18:20:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.494+0000 7fe9cad9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe9cc0d27b0 0x7fe9cc0cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.494+0000 7fe9ca59c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9cc15cfc0 0x7fe9cc160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.494+0000 7fe9cad9d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe9cc0d27b0 0x7fe9cc0cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:54196/0 (socket says 172.21.15.27:54196) 2023-12-19T18:20:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.495+0000 7fe9cad9d640 1 -- 172.21.15.27:0/3378774605 learned_addr learned my addr 172.21.15.27:0/3378774605 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:55.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.495+0000 7fe9ca59c640 1 -- 172.21.15.27:0/3378774605 >> v2:172.21.15.192:3300/0 conn(0x7fe9cc0d27b0 msgr2=0x7fe9cc0cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.495+0000 7fe9ca59c640 1 --2- 172.21.15.27:0/3378774605 >> v2:172.21.15.192:3300/0 conn(0x7fe9cc0d27b0 0x7fe9cc0cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.495+0000 7fe9ca59c640 1 -- 172.21.15.27:0/3378774605 >> v1:172.21.15.192:6789/0 conn(0x7fe9cc0cdb30 legacy=0x7fe9cc0d2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.495+0000 7fe9ca59c640 1 -- 172.21.15.27:0/3378774605 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9cc161770 con 0x7fe9cc15cfc0 2023-12-19T18:20:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.495+0000 7fe9cad9d640 1 --2- 172.21.15.27:0/3378774605 >> v2:172.21.15.192:3300/0 conn(0x7fe9cc0d27b0 0x7fe9cc0cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:20:55.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.497+0000 7fe9ca59c640 1 --2- 172.21.15.27:0/3378774605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9cc15cfc0 0x7fe9cc160b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009d90 tx=0x7fe9bc02be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb60c264cd74ffa7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:55.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.497+0000 7fe9c959a640 1 -- 172.21.15.27:0/3378774605 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc02cbd0 con 0x7fe9cc15cfc0 2023-12-19T18:20:55.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.497+0000 7fe9c959a640 1 -- 172.21.15.27:0/3378774605 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9bc02cd30 con 0x7fe9cc15cfc0 2023-12-19T18:20:55.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.497+0000 7fe9c959a640 1 -- 172.21.15.27:0/3378774605 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc02cea0 con 0x7fe9cc15cfc0 2023-12-19T18:20:55.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.498+0000 7fe9cb59e640 1 -- 172.21.15.27:0/3378774605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9cc15cfc0 msgr2=0x7fe9cc160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:55.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.498+0000 7fe9cb59e640 1 --2- 172.21.15.27:0/3378774605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9cc15cfc0 0x7fe9cc160b40 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009d90 tx=0x7fe9bc02be20 comp rx=0 tx=0).stop 2023-12-19T18:20:55.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.498+0000 7fe9cb59e640 1 -- 172.21.15.27:0/3378774605 shutdown_connections 2023-12-19T18:20:55.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.498+0000 7fe9cb59e640 1 --2- 172.21.15.27:0/3378774605 >> v2:172.21.15.192:3300/0 conn(0x7fe9cc0d27b0 0x7fe9cc0cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:55.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.498+0000 7fe9cb59e640 1 --2- 172.21.15.27:0/3378774605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9cc15cfc0 0x7fe9cc160b40 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:55.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.498+0000 7fe9cb59e640 1 -- 172.21.15.27:0/3378774605 >> 172.21.15.27:0/3378774605 conn(0x7fe9cc072e20 msgr2=0x7fe9cc073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:55.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.498+0000 7fe9cb59e640 1 -- 172.21.15.27:0/3378774605 shutdown_connections 2023-12-19T18:20:55.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.498+0000 7fe9cb59e640 1 -- 172.21.15.27:0/3378774605 wait complete. 2023-12-19T18:20:55.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.501+0000 7fe9cb59e640 1 Processor -- start 2023-12-19T18:20:55.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.501+0000 7fe9cb59e640 1 -- start start 2023-12-19T18:20:55.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.502+0000 7fe9cb59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9cc0cdb30 0x7fe9cc154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:55.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.502+0000 7fe9cb59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9cc0d27b0 0x7fe9cc15dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:55.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.502+0000 7fe9cb59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9cc15cfc0 0x7fe9cc0cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:55.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.502+0000 7fe9cb59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc15ed60 con 0x7fe9cc0d27b0 2023-12-19T18:20:55.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.502+0000 7fe9cb59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc15f040 con 0x7fe9cc0cdb30 2023-12-19T18:20:55.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.502+0000 7fe9cb59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9cc15f320 con 0x7fe9cc15cfc0 2023-12-19T18:20:55.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.503+0000 7fe9c9d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9cc0d27b0 0x7fe9cc15dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:55.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.503+0000 7fe9c9d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9cc0d27b0 0x7fe9cc15dfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39526/0 (socket says 172.21.15.27:39526) 2023-12-19T18:20:55.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.503+0000 7fe9c9d9b640 1 -- 172.21.15.27:0/4226585543 learned_addr learned my addr 172.21.15.27:0/4226585543 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:55.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.503+0000 7fe9ca59c640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9cc0cdb30 0x7fe9cc154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:55.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.503+0000 7fe9cad9d640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9cc15cfc0 0x7fe9cc0cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:55.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.503+0000 7fe9c9d9b640 1 -- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9cc15cfc0 msgr2=0x7fe9cc0cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:55.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.503+0000 7fe9c9d9b640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9cc15cfc0 0x7fe9cc0cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:55.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.503+0000 7fe9c9d9b640 1 -- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9cc0cdb30 msgr2=0x7fe9cc154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:55.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.503+0000 7fe9c9d9b640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9cc0cdb30 0x7fe9cc154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:55.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.503+0000 7fe9c9d9b640 1 -- 172.21.15.27:0/4226585543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c40094d0 con 0x7fe9cc0d27b0 2023-12-19T18:20:55.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.503+0000 7fe9ca59c640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9cc0cdb30 0x7fe9cc154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:20:55.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.504+0000 7fe9c9d9b640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9cc0d27b0 0x7fe9cc15dfe0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400c400 tx=0x7fe9b400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:55.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.504+0000 7fe9c2ffd640 1 -- 172.21.15.27:0/4226585543 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b40052d0 con 0x7fe9cc0d27b0 2023-12-19T18:20:55.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.504+0000 7fe9c2ffd640 1 -- 172.21.15.27:0/4226585543 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b4005430 con 0x7fe9cc0d27b0 2023-12-19T18:20:55.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.504+0000 7fe9c2ffd640 1 -- 172.21.15.27:0/4226585543 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4003830 con 0x7fe9cc0d27b0 2023-12-19T18:20:55.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.504+0000 7fe9cb59e640 1 -- 172.21.15.27:0/4226585543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9bc009a70 con 0x7fe9cc0d27b0 2023-12-19T18:20:55.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.504+0000 7fe9cb59e640 1 -- 172.21.15.27:0/4226585543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9cc15e520 con 0x7fe9cc0d27b0 2023-12-19T18:20:55.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.505+0000 7fe9cb59e640 1 -- 172.21.15.27:0/4226585543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9cc15f840 con 0x7fe9cc0d27b0 2023-12-19T18:20:55.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.506+0000 7fe9c2ffd640 1 -- 172.21.15.27:0/4226585543 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe9b4003990 con 0x7fe9cc0d27b0 2023-12-19T18:20:55.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.507+0000 7fe9c2ffd640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe9ac06a430 0x7fe9ac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:55.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.507+0000 7fe9c2ffd640 1 -- 172.21.15.27:0/4226585543 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe9b4087e00 con 0x7fe9cc0d27b0 2023-12-19T18:20:55.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.508+0000 7fe9ca59c640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe9ac06a430 0x7fe9ac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:55.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.509+0000 7fe9ca59c640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe9ac06a430 0x7fe9ac06c8f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc036000 tx=0x7fe9bc030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:55.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.512+0000 7fe9c2ffd640 1 -- 172.21.15.27:0/4226585543 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe9cc15f840 con 0x7fe9cc0d27b0 2023-12-19T18:20:55.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:55.705+0000 7fe9cb59e640 1 -- 172.21.15.27:0/4226585543 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe9cc15f840 con 0x7fe9cc0d27b0 2023-12-19T18:20:56.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.237+0000 7fe9c2ffd640 1 -- 172.21.15.27:0/4226585543 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v107) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe9cc15f840 con 0x7fe9cc0d27b0 2023-12-19T18:20:56.237 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:20:56.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.240+0000 7fe9cb59e640 1 -- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe9ac06a430 msgr2=0x7fe9ac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:56.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.240+0000 7fe9cb59e640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe9ac06a430 0x7fe9ac06c8f0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc036000 tx=0x7fe9bc030000 comp rx=0 tx=0).stop 2023-12-19T18:20:56.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.240+0000 7fe9cb59e640 1 -- 172.21.15.27:0/4226585543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9cc0d27b0 msgr2=0x7fe9cc15dfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:56.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.240+0000 7fe9cb59e640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9cc0d27b0 0x7fe9cc15dfe0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400c400 tx=0x7fe9b400c8c0 comp rx=0 tx=0).stop 2023-12-19T18:20:56.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.241+0000 7fe9cb59e640 1 -- 172.21.15.27:0/4226585543 shutdown_connections 2023-12-19T18:20:56.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.241+0000 7fe9cb59e640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe9ac06a430 0x7fe9ac06c8f0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:56.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.241+0000 7fe9cb59e640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe9cc15cfc0 0x7fe9cc0cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:56.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.241+0000 7fe9cb59e640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9cc0d27b0 0x7fe9cc15dfe0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:56.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.241+0000 7fe9cb59e640 1 --2- 172.21.15.27:0/4226585543 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe9cc0cdb30 0x7fe9cc154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:56.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.241+0000 7fe9cb59e640 1 -- 172.21.15.27:0/4226585543 >> 172.21.15.27:0/4226585543 conn(0x7fe9cc072e20 msgr2=0x7fe9cc0b3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:56.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.241+0000 7fe9cb59e640 1 -- 172.21.15.27:0/4226585543 shutdown_connections 2023-12-19T18:20:56.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.241+0000 7fe9cb59e640 1 -- 172.21.15.27:0/4226585543 wait complete. 2023-12-19T18:20:56.267 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:20:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.509+0000 7fbaf1d27640 1 Processor -- start 2023-12-19T18:20:56.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.510+0000 7fbaf1d27640 1 -- start start 2023-12-19T18:20:56.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.511+0000 7fbaf1d27640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbaec15cfc0 0x7fbaec160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:56.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.511+0000 7fbaf1d27640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbaec0cdb30 0x7fbaec0cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:56.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.511+0000 7fbaf1d27640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbaec0ce4d0 con 0x7fbaec0d2cf0 2023-12-19T18:20:56.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.511+0000 7fbaf1d27640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbaec161a50 con 0x7fbaec0cdb30 2023-12-19T18:20:56.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.511+0000 7fbaf1d27640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec161490 con 0x7fbaec15cfc0 2023-12-19T18:20:56.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.512+0000 7fbaebfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbaec0cdb30 0x7fbaec0cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:56.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.512+0000 7fbaebfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbaec0cdb30 0x7fbaec0cdf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39534/0 (socket says 172.21.15.27:39534) 2023-12-19T18:20:56.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.512+0000 7fbaebfff640 1 -- 172.21.15.27:0/3762811321 learned_addr learned my addr 172.21.15.27:0/3762811321 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:56.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.512+0000 7fbaebfff640 1 -- 172.21.15.27:0/3762811321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbaec15cfc0 msgr2=0x7fbaec160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:56.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.512+0000 7fbaebfff640 1 --2- 172.21.15.27:0/3762811321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbaec15cfc0 0x7fbaec160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:56.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.512+0000 7fbaebfff640 1 -- 172.21.15.27:0/3762811321 >> v1:172.21.15.192:6789/0 conn(0x7fbaec0d2cf0 legacy=0x7fbaec0d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:56.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.512+0000 7fbaebfff640 1 -- 172.21.15.27:0/3762811321 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaec161770 con 0x7fbaec0cdb30 2023-12-19T18:20:56.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.513+0000 7fbaebfff640 1 --2- 172.21.15.27:0/3762811321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec0cdb30 0x7fbaec0cdf90 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009870 tx=0x7fbae002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89317463f6dd6ae server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:56.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.514+0000 7fbaeb7fe640 1 -- 172.21.15.27:0/3762811321 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae00317c0 con 0x7fbaec0cdb30 2023-12-19T18:20:56.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.514+0000 7fbaeb7fe640 1 -- 172.21.15.27:0/3762811321 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbae0031920 con 0x7fbaec0cdb30 2023-12-19T18:20:56.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.514+0000 7fbaeb7fe640 1 -- 172.21.15.27:0/3762811321 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbae0031cb0 con 0x7fbaec0cdb30 2023-12-19T18:20:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.514+0000 7fbaf1d27640 1 -- 172.21.15.27:0/3762811321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec0cdb30 msgr2=0x7fbaec0cdf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.514+0000 7fbaf1d27640 1 --2- 172.21.15.27:0/3762811321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec0cdb30 0x7fbaec0cdf90 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fbae0009870 tx=0x7fbae002b860 comp rx=0 tx=0).stop 2023-12-19T18:20:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.515+0000 7fbaf1d27640 1 -- 172.21.15.27:0/3762811321 shutdown_connections 2023-12-19T18:20:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.515+0000 7fbaf1d27640 1 --2- 172.21.15.27:0/3762811321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec0cdb30 0x7fbaec0cdf90 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.515+0000 7fbaf1d27640 1 --2- 172.21.15.27:0/3762811321 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbaec15cfc0 0x7fbaec160b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.515+0000 7fbaf1d27640 1 -- 172.21.15.27:0/3762811321 >> 172.21.15.27:0/3762811321 conn(0x7fbaec072e20 msgr2=0x7fbaec073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.515+0000 7fbaf1d27640 1 -- 172.21.15.27:0/3762811321 shutdown_connections 2023-12-19T18:20:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.515+0000 7fbaf1d27640 1 -- 172.21.15.27:0/3762811321 wait complete. 2023-12-19T18:20:56.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.517+0000 7fbaf1d27640 1 Processor -- start 2023-12-19T18:20:56.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.517+0000 7fbaf1d27640 1 -- start start 2023-12-19T18:20:56.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf1d27640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec0cdb30 0x7fbaec150720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:56.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf1d27640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaec0d2cf0 0x7fbaec154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:56.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf1d27640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbaec15cfc0 0x7fbaec0b6460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:56.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf1d27640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec150c60 con 0x7fbaec0cdb30 2023-12-19T18:20:56.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf1d27640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec155100 con 0x7fbaec0d2cf0 2023-12-19T18:20:56.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf1d27640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbaec1514a0 con 0x7fbaec15cfc0 2023-12-19T18:20:56.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf0d25640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec0cdb30 0x7fbaec150720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:56.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf0d25640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec0cdb30 0x7fbaec150720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39550/0 (socket says 172.21.15.27:39550) 2023-12-19T18:20:56.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf0d25640 1 -- 172.21.15.27:0/588595947 learned_addr learned my addr 172.21.15.27:0/588595947 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:56.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf0d25640 1 -- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbaec15cfc0 msgr2=0x7fbaec0b6460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:56.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf0d25640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbaec15cfc0 0x7fbaec0b6460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:56.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf0d25640 1 -- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaec0d2cf0 msgr2=0x7fbaec154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:56.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaebfff640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaec0d2cf0 0x7fbaec154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:56.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf0d25640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaec0d2cf0 0x7fbaec154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:56.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaf0d25640 1 -- 172.21.15.27:0/588595947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbae00094d0 con 0x7fbaec0cdb30 2023-12-19T18:20:56.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.518+0000 7fbaebfff640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaec0d2cf0 0x7fbaec154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:20:56.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.519+0000 7fbaf0d25640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec0cdb30 0x7fbaec150720 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00ba50 tx=0x7fbadc00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:56.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.519+0000 7fbae97fa640 1 -- 172.21.15.27:0/588595947 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc00cbe0 con 0x7fbaec0cdb30 2023-12-19T18:20:56.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.519+0000 7fbae97fa640 1 -- 172.21.15.27:0/588595947 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbadc00cd60 con 0x7fbaec0cdb30 2023-12-19T18:20:56.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.519+0000 7fbaf1d27640 1 -- 172.21.15.27:0/588595947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaec151780 con 0x7fbaec0cdb30 2023-12-19T18:20:56.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.519+0000 7fbae97fa640 1 -- 172.21.15.27:0/588595947 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbadc014680 con 0x7fbaec0cdb30 2023-12-19T18:20:56.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.519+0000 7fbaf1d27640 1 -- 172.21.15.27:0/588595947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaec151d40 con 0x7fbaec0cdb30 2023-12-19T18:20:56.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.520+0000 7fbaf1d27640 1 -- 172.21.15.27:0/588595947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaec04e840 con 0x7fbaec0cdb30 2023-12-19T18:20:56.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.521+0000 7fbae97fa640 1 -- 172.21.15.27:0/588595947 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbadc01d3b0 con 0x7fbaec0cdb30 2023-12-19T18:20:56.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.522+0000 7fbae97fa640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbad806a430 0x7fbad806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:56.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.523+0000 7fbae97fa640 1 -- 172.21.15.27:0/588595947 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5386+0+0 (secure 0 0 0) 0x7fbadc088870 con 0x7fbaec0cdb30 2023-12-19T18:20:56.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.523+0000 7fbaebfff640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbad806a430 0x7fbad806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:56.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.524+0000 7fbaebfff640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbad806a430 0x7fbad806c8f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbae0036000 tx=0x7fbae003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:56.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.527+0000 7fbae97fa640 1 -- 172.21.15.27:0/588595947 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbadc053f00 con 0x7fbaec0cdb30 2023-12-19T18:20:56.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:56.720+0000 7fbaf1d27640 1 -- 172.21.15.27:0/588595947 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbaec151f30 con 0x7fbaec0cdb30 2023-12-19T18:20:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.651+0000 7fbae97fa640 1 -- 172.21.15.27:0/588595947 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v108) v1 ==== 169+0+0 (secure 0 0 0) 0x7fbadc0190d0 con 0x7fbaec0cdb30 2023-12-19T18:20:57.652 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-12-19T18:20:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.656+0000 7fbaf1d27640 1 -- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbad806a430 msgr2=0x7fbad806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.656+0000 7fbaf1d27640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbad806a430 0x7fbad806c8f0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbae0036000 tx=0x7fbae003d040 comp rx=0 tx=0).stop 2023-12-19T18:20:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.656+0000 7fbaf1d27640 1 -- 172.21.15.27:0/588595947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec0cdb30 msgr2=0x7fbaec150720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.656+0000 7fbaf1d27640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec0cdb30 0x7fbaec150720 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fbadc00ba50 tx=0x7fbadc00bf10 comp rx=0 tx=0).stop 2023-12-19T18:20:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.657+0000 7fbaf1d27640 1 -- 172.21.15.27:0/588595947 shutdown_connections 2023-12-19T18:20:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.657+0000 7fbaf1d27640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbad806a430 0x7fbad806c8f0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.657+0000 7fbaf1d27640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbaec15cfc0 0x7fbaec0b6460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.657+0000 7fbaf1d27640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbaec0d2cf0 0x7fbaec154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.657+0000 7fbaf1d27640 1 --2- 172.21.15.27:0/588595947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaec0cdb30 0x7fbaec150720 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.657+0000 7fbaf1d27640 1 -- 172.21.15.27:0/588595947 >> 172.21.15.27:0/588595947 conn(0x7fbaec072e20 msgr2=0x7fbaec160790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.657+0000 7fbaf1d27640 1 -- 172.21.15.27:0/588595947 shutdown_connections 2023-12-19T18:20:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.657+0000 7fbaf1d27640 1 -- 172.21.15.27:0/588595947 wait complete. 2023-12-19T18:20:57.679 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:20:57.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.911+0000 7fb6eca2d640 1 Processor -- start 2023-12-19T18:20:57.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.911+0000 7fb6eca2d640 1 -- start start 2023-12-19T18:20:57.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.913+0000 7fb6eca2d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb6e80ade20 0x7fb6e815b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:57.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.913+0000 7fb6eca2d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6e815b810 0x7fb6e80c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:57.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.913+0000 7fb6eca2d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6e815c030 con 0x7fb6e8158790 2023-12-19T18:20:57.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.913+0000 7fb6eca2d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6e80c99d0 con 0x7fb6e80ade20 2023-12-19T18:20:57.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.913+0000 7fb6eca2d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e8158b70 con 0x7fb6e815b810 2023-12-19T18:20:57.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.913+0000 7fb6e77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb6e8158790 0x7fb6e80ad710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54486/0 (socket says 172.21.15.27:54486) 2023-12-19T18:20:57.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.913+0000 7fb6e77fe640 1 -- 172.21.15.27:0/1241007970 learned_addr learned my addr 172.21.15.27:0/1241007970 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e6ffd640 1 --2- 172.21.15.27:0/1241007970 >> v2:172.21.15.192:3300/0 conn(0x7fb6e80ade20 0x7fb6e815b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3537913411 0 0) 0x7fb6e815c030 con 0x7fb6e8158790 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6d0003520 con 0x7fb6e8158790 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e7fff640 1 --2- 172.21.15.27:0/1241007970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6e815b810 0x7fb6e80c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 554978926 0 0) 0x7fb6d0003520 con 0x7fb6e8158790 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6e815b810 msgr2=0x7fb6e80c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 --2- 172.21.15.27:0/1241007970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6e815b810 0x7fb6e80c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 >> v2:172.21.15.192:3300/0 conn(0x7fb6e80ade20 msgr2=0x7fb6e815b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 --2- 172.21.15.27:0/1241007970 >> v2:172.21.15.192:3300/0 conn(0x7fb6e80ade20 0x7fb6e815b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e80c9030 con 0x7fb6e8158790 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fb6d8001690 con 0x7fb6e8158790 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e6ffd640 1 --2- 172.21.15.27:0/1241007970 >> v2:172.21.15.192:3300/0 conn(0x7fb6e80ade20 0x7fb6e815b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 >> v1:172.21.15.27:6789/0 conn(0x7fb6e8158790 legacy=0x7fb6e80ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 --2- 172.21.15.27:0/1241007970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6e815b810 0x7fb6d0003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 --2- 172.21.15.27:0/1241007970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6d0003ff0 0x7fb6d00047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 --2- 172.21.15.27:0/1241007970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d0004d00 0x7fb6d0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80c99d0 con 0x7fb6d0003ff0 2023-12-19T18:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e8158b70 con 0x7fb6d0004d00 2023-12-19T18:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.914+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d000bae0 con 0x7fb6e815b810 2023-12-19T18:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.915+0000 7fb6e6ffd640 1 --2- 172.21.15.27:0/1241007970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6d0003ff0 0x7fb6d00047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.915+0000 7fb6e7fff640 1 --2- 172.21.15.27:0/1241007970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d0004d00 0x7fb6d0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.915+0000 7fb6e6ffd640 1 -- 172.21.15.27:0/1241007970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6e815b810 msgr2=0x7fb6d0003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.915+0000 7fb6e6ffd640 1 --2- 172.21.15.27:0/1241007970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6e815b810 0x7fb6d0003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.915+0000 7fb6e6ffd640 1 -- 172.21.15.27:0/1241007970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d0004d00 msgr2=0x7fb6d0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.915+0000 7fb6e6ffd640 1 --2- 172.21.15.27:0/1241007970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6d0004d00 0x7fb6d0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.915+0000 7fb6e6ffd640 1 -- 172.21.15.27:0/1241007970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d0003520 con 0x7fb6d0003ff0 2023-12-19T18:20:57.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.916+0000 7fb6e6ffd640 1 --2- 172.21.15.27:0/1241007970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6d0003ff0 0x7fb6d00047c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0098a0 tx=0x7fb6d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9411313e192d5958 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:57.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.916+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc004230 con 0x7fb6d0003ff0 2023-12-19T18:20:57.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.916+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6dc004390 con 0x7fb6d0003ff0 2023-12-19T18:20:57.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.916+0000 7fb6e67fc640 1 -- 172.21.15.27:0/1241007970 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6dc003830 con 0x7fb6d0003ff0 2023-12-19T18:20:57.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.917+0000 7fb6eca2d640 1 -- 172.21.15.27:0/1241007970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6d0003ff0 msgr2=0x7fb6d00047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:57.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.917+0000 7fb6eca2d640 1 --2- 172.21.15.27:0/1241007970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6d0003ff0 0x7fb6d00047c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc0098a0 tx=0x7fb6d00057f0 comp rx=0 tx=0).stop 2023-12-19T18:20:57.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.917+0000 7fb6e7fff640 1 -- 172.21.15.27:0/1241007970 reap_dead start 2023-12-19T18:20:57.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.917+0000 7fb6eca2d640 1 -- 172.21.15.27:0/1241007970 shutdown_connections 2023-12-19T18:20:57.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.917+0000 7fb6eca2d640 1 -- 172.21.15.27:0/1241007970 >> 172.21.15.27:0/1241007970 conn(0x7fb6e80b05b0 msgr2=0x7fb6e80b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:57.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.917+0000 7fb6eca2d640 1 -- 172.21.15.27:0/1241007970 shutdown_connections 2023-12-19T18:20:57.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.917+0000 7fb6eca2d640 1 -- 172.21.15.27:0/1241007970 wait complete. 2023-12-19T18:20:57.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.919+0000 7fb6eca2d640 1 Processor -- start 2023-12-19T18:20:57.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.919+0000 7fb6eca2d640 1 -- start start 2023-12-19T18:20:57.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.920+0000 7fb6eca2d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6d0003ff0 0x7fb6e81f1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:57.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.920+0000 7fb6eca2d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6e815a2a0 0x7fb6e815a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:57.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.920+0000 7fb6eca2d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6e80ae200 0x7fb6e80ae660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:57.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.920+0000 7fb6eca2d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e815ac60 con 0x7fb6d0003ff0 2023-12-19T18:20:57.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.920+0000 7fb6eca2d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80c8830 con 0x7fb6e815a2a0 2023-12-19T18:20:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.920+0000 7fb6eca2d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6e80aeba0 con 0x7fb6e80ae200 2023-12-19T18:20:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.920+0000 7fb6e77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6d0003ff0 0x7fb6e81f1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.921+0000 7fb6e77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6d0003ff0 0x7fb6e81f1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39566/0 (socket says 172.21.15.27:39566) 2023-12-19T18:20:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.921+0000 7fb6e77fe640 1 -- 172.21.15.27:0/3761614325 learned_addr learned my addr 172.21.15.27:0/3761614325 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.921+0000 7fb6e77fe640 1 -- 172.21.15.27:0/3761614325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6e80ae200 msgr2=0x7fb6e80ae660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.921+0000 7fb6e77fe640 1 --2- 172.21.15.27:0/3761614325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6e80ae200 0x7fb6e80ae660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.921+0000 7fb6e77fe640 1 -- 172.21.15.27:0/3761614325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6e815a2a0 msgr2=0x7fb6e815a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.921+0000 7fb6e77fe640 1 --2- 172.21.15.27:0/3761614325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6e815a2a0 0x7fb6e815a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.921+0000 7fb6e77fe640 1 -- 172.21.15.27:0/3761614325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6e80aede0 con 0x7fb6d0003ff0 2023-12-19T18:20:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.921+0000 7fb6e77fe640 1 --2- 172.21.15.27:0/3761614325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6d0003ff0 0x7fb6e81f1bc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800c650 tx=0x7fb6d800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:57.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.922+0000 7fb6cffff640 1 -- 172.21.15.27:0/3761614325 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d80072d0 con 0x7fb6d0003ff0 2023-12-19T18:20:57.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.922+0000 7fb6cffff640 1 -- 172.21.15.27:0/3761614325 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6d8007430 con 0x7fb6d0003ff0 2023-12-19T18:20:57.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.922+0000 7fb6cffff640 1 -- 172.21.15.27:0/3761614325 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6d80047e0 con 0x7fb6d0003ff0 2023-12-19T18:20:57.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.922+0000 7fb6eca2d640 1 -- 172.21.15.27:0/3761614325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6e80af0c0 con 0x7fb6d0003ff0 2023-12-19T18:20:57.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.922+0000 7fb6eca2d640 1 -- 172.21.15.27:0/3761614325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6e80af680 con 0x7fb6d0003ff0 2023-12-19T18:20:57.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.923+0000 7fb6eca2d640 1 -- 172.21.15.27:0/3761614325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6e804e840 con 0x7fb6d0003ff0 2023-12-19T18:20:57.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.924+0000 7fb6cffff640 1 -- 172.21.15.27:0/3761614325 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb6d8004940 con 0x7fb6d0003ff0 2023-12-19T18:20:57.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.925+0000 7fb6cffff640 1 --2- 172.21.15.27:0/3761614325 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb6d406a460 0x7fb6d406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:57.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.926+0000 7fb6cffff640 1 -- 172.21.15.27:0/3761614325 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb6d80879b0 con 0x7fb6d0003ff0 2023-12-19T18:20:57.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.926+0000 7fb6e6ffd640 1 --2- 172.21.15.27:0/3761614325 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb6d406a460 0x7fb6d406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:57.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.927+0000 7fb6e6ffd640 1 --2- 172.21.15.27:0/3761614325 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb6d406a460 0x7fb6d406c920 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc035000 tx=0x7fb6dc030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:57.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:57.930+0000 7fb6cffff640 1 -- 172.21.15.27:0/3761614325 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb6d80532a0 con 0x7fb6d0003ff0 2023-12-19T18:20:58.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.124+0000 7fb6eca2d640 1 -- 172.21.15.27:0/3761614325 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb6e80c3ad0 con 0x7fb6d0003ff0 2023-12-19T18:20:58.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.131+0000 7fb6cffff640 1 -- 172.21.15.27:0/3761614325 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb6d8018020 con 0x7fb6d0003ff0 2023-12-19T18:20:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.134+0000 7fb6eca2d640 1 -- 172.21.15.27:0/3761614325 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb6d406a460 msgr2=0x7fb6d406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.134+0000 7fb6eca2d640 1 --2- 172.21.15.27:0/3761614325 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb6d406a460 0x7fb6d406c920 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb6dc035000 tx=0x7fb6dc030000 comp rx=0 tx=0).stop 2023-12-19T18:20:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.134+0000 7fb6eca2d640 1 -- 172.21.15.27:0/3761614325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6d0003ff0 msgr2=0x7fb6e81f1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.134+0000 7fb6eca2d640 1 --2- 172.21.15.27:0/3761614325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6d0003ff0 0x7fb6e81f1bc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fb6d800c650 tx=0x7fb6d800cb10 comp rx=0 tx=0).stop 2023-12-19T18:20:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.134+0000 7fb6eca2d640 1 -- 172.21.15.27:0/3761614325 shutdown_connections 2023-12-19T18:20:58.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.134+0000 7fb6eca2d640 1 --2- 172.21.15.27:0/3761614325 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb6d406a460 0x7fb6d406c920 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:58.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.134+0000 7fb6eca2d640 1 --2- 172.21.15.27:0/3761614325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb6e80ae200 0x7fb6e80ae660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:58.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.134+0000 7fb6eca2d640 1 --2- 172.21.15.27:0/3761614325 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb6e815a2a0 0x7fb6e815a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:58.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.134+0000 7fb6eca2d640 1 --2- 172.21.15.27:0/3761614325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6d0003ff0 0x7fb6e81f1bc0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:58.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.134+0000 7fb6eca2d640 1 -- 172.21.15.27:0/3761614325 >> 172.21.15.27:0/3761614325 conn(0x7fb6e80b05b0 msgr2=0x7fb6e80ca720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:58.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.135+0000 7fb6eca2d640 1 -- 172.21.15.27:0/3761614325 shutdown_connections 2023-12-19T18:20:58.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.135+0000 7fb6eca2d640 1 -- 172.21.15.27:0/3761614325 wait complete. 2023-12-19T18:20:58.154 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync' 2023-12-19T18:20:58.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.389+0000 7f56d4d99640 1 Processor -- start 2023-12-19T18:20:58.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.389+0000 7f56d4d99640 1 -- start start 2023-12-19T18:20:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.391+0000 7f56d4d99640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f56d00b4ef0 0x7f56d00b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.391+0000 7f56d4d99640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f56d00d2350 0x7f56d00b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.391+0000 7f56d4d99640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d00b5e90 con 0x7f56d01604d0 2023-12-19T18:20:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.391+0000 7f56d4d99640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d00d2b30 con 0x7f56d00d2350 2023-12-19T18:20:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.391+0000 7f56d4d99640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d00d2850 con 0x7f56d00b4ef0 2023-12-19T18:20:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.392+0000 7f56cffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f56d00d2350 0x7f56d00b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:58.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.392+0000 7f56cffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f56d00d2350 0x7f56d00b6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39570/0 (socket says 172.21.15.27:39570) 2023-12-19T18:20:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.392+0000 7f56cffff640 1 -- 172.21.15.27:0/1556647846 learned_addr learned my addr 172.21.15.27:0/1556647846 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.392+0000 7f56c7fff640 1 --2- 172.21.15.27:0/1556647846 >> v2:172.21.15.192:3300/0 conn(0x7f56d00b4ef0 0x7f56d00b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.392+0000 7f56cffff640 1 -- 172.21.15.27:0/1556647846 >> v2:172.21.15.192:3300/0 conn(0x7f56d00b4ef0 msgr2=0x7f56d00b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.392+0000 7f56cffff640 1 --2- 172.21.15.27:0/1556647846 >> v2:172.21.15.192:3300/0 conn(0x7f56d00b4ef0 0x7f56d00b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.392+0000 7f56cffff640 1 -- 172.21.15.27:0/1556647846 >> v1:172.21.15.192:6789/0 conn(0x7f56d01604d0 legacy=0x7f56d00b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.392+0000 7f56cffff640 1 -- 172.21.15.27:0/1556647846 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d00cded0 con 0x7f56d00d2350 2023-12-19T18:20:58.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.393+0000 7f56cffff640 1 --2- 172.21.15.27:0/1556647846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56d00d2350 0x7f56d00b6840 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f56d0072120 tx=0x7f56c802bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262dab0467b76743 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.394+0000 7f56ceffd640 1 -- 172.21.15.27:0/1556647846 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c802ca80 con 0x7f56d00d2350 2023-12-19T18:20:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.394+0000 7f56ceffd640 1 -- 172.21.15.27:0/1556647846 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c802cbe0 con 0x7f56d00d2350 2023-12-19T18:20:58.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.394+0000 7f56ceffd640 1 -- 172.21.15.27:0/1556647846 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c80343f0 con 0x7f56d00d2350 2023-12-19T18:20:58.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.394+0000 7f56d4d99640 1 -- 172.21.15.27:0/1556647846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56d00d2350 msgr2=0x7f56d00b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:58.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.394+0000 7f56d4d99640 1 --2- 172.21.15.27:0/1556647846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56d00d2350 0x7f56d00b6840 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f56d0072120 tx=0x7f56c802bee0 comp rx=0 tx=0).stop 2023-12-19T18:20:58.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.394+0000 7f56d4d99640 1 -- 172.21.15.27:0/1556647846 shutdown_connections 2023-12-19T18:20:58.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.394+0000 7f56d4d99640 1 --2- 172.21.15.27:0/1556647846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56d00d2350 0x7f56d00b6840 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:58.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.394+0000 7f56d4d99640 1 --2- 172.21.15.27:0/1556647846 >> v2:172.21.15.192:3300/0 conn(0x7f56d00b4ef0 0x7f56d00b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:58.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.394+0000 7f56d4d99640 1 -- 172.21.15.27:0/1556647846 >> 172.21.15.27:0/1556647846 conn(0x7f56d00cf7e0 msgr2=0x7f56d00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:58.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.395+0000 7f56d4d99640 1 -- 172.21.15.27:0/1556647846 shutdown_connections 2023-12-19T18:20:58.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.395+0000 7f56d4d99640 1 -- 172.21.15.27:0/1556647846 wait complete. 2023-12-19T18:20:58.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.397+0000 7f56d4d99640 1 Processor -- start 2023-12-19T18:20:58.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.397+0000 7f56d4d99640 1 -- start start 2023-12-19T18:20:58.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.399+0000 7f56d4d99640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56d00b4ef0 0x7f56d00c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:58.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.399+0000 7f56d4d99640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56d00d2350 0x7f56d01f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:58.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.399+0000 7f56d4d99640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56d01604d0 0x7f56d014bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:58.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.399+0000 7f56d4d99640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d00c7210 con 0x7f56d00d2350 2023-12-19T18:20:58.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.399+0000 7f56d4d99640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d015cf90 con 0x7f56d00b4ef0 2023-12-19T18:20:58.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.399+0000 7f56d4d99640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d00cc330 con 0x7f56d01604d0 2023-12-19T18:20:58.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.399+0000 7f56c7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56d00d2350 0x7f56d01f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:58.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.399+0000 7f56c7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56d00d2350 0x7f56d01f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39574/0 (socket says 172.21.15.27:39574) 2023-12-19T18:20:58.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.399+0000 7f56c7fff640 1 -- 172.21.15.27:0/3527370655 learned_addr learned my addr 172.21.15.27:0/3527370655 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:58.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.399+0000 7f56cf7fe640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56d00b4ef0 0x7f56d00c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:58.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.399+0000 7f56cffff640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56d01604d0 0x7f56d014bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:58.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.400+0000 7f56c7fff640 1 -- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56d01604d0 msgr2=0x7f56d014bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.400+0000 7f56c7fff640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56d01604d0 0x7f56d014bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.400+0000 7f56c7fff640 1 -- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56d00b4ef0 msgr2=0x7f56d00c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.400+0000 7f56c7fff640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56d00b4ef0 0x7f56d00c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.400+0000 7f56c7fff640 1 -- 172.21.15.27:0/3527370655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c8009b30 con 0x7f56d00d2350 2023-12-19T18:20:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.400+0000 7f56cf7fe640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56d00b4ef0 0x7f56d00c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:20:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.400+0000 7f56c7fff640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56d00d2350 0x7f56d01f58e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f56b800d5f0 tx=0x7f56b800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.400+0000 7f56ccff9640 1 -- 172.21.15.27:0/3527370655 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80125f0 con 0x7f56d00d2350 2023-12-19T18:20:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.400+0000 7f56ccff9640 1 -- 172.21.15.27:0/3527370655 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b8012750 con 0x7f56d00d2350 2023-12-19T18:20:58.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.400+0000 7f56d4d99640 1 -- 172.21.15.27:0/3527370655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d00cc610 con 0x7f56d00d2350 2023-12-19T18:20:58.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.400+0000 7f56ccff9640 1 -- 172.21.15.27:0/3527370655 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b8012b00 con 0x7f56d00d2350 2023-12-19T18:20:58.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.401+0000 7f56d4d99640 1 -- 172.21.15.27:0/3527370655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d00c78a0 con 0x7f56d00d2350 2023-12-19T18:20:58.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.402+0000 7f56d4d99640 1 -- 172.21.15.27:0/3527370655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d00cc850 con 0x7f56d00d2350 2023-12-19T18:20:58.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.403+0000 7f56ccff9640 1 -- 172.21.15.27:0/3527370655 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f56b8012c60 con 0x7f56d00d2350 2023-12-19T18:20:58.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.404+0000 7f56ccff9640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f56b006a430 0x7f56b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:58.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.404+0000 7f56ccff9640 1 -- 172.21.15.27:0/3527370655 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f56b8088ad0 con 0x7f56d00d2350 2023-12-19T18:20:58.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.404+0000 7f56cf7fe640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f56b006a430 0x7f56b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:58.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.405+0000 7f56cf7fe640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f56b006a430 0x7f56b006c8f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f56c0003eb0 tx=0x7f56c0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:58.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.408+0000 7f56ccff9640 1 -- 172.21.15.27:0/3527370655 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f56b80543b0 con 0x7f56d00d2350 2023-12-19T18:20:58.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:58.681+0000 7f56d4d99640 1 -- 172.21.15.27:0/3527370655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]} v 0) v1 -- 0x7f56d00c7400 con 0x7f56d00d2350 2023-12-19T18:20:59.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.662+0000 7f56ccff9640 1 -- 172.21.15.27:0/3527370655 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_fsync"]}]=0 v310) v1 ==== 133+0+0 (secure 0 0 0) 0x7f56b80579d0 con 0x7f56d00d2350 2023-12-19T18:20:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.666+0000 7f56d4d99640 1 -- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f56b006a430 msgr2=0x7f56b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.666+0000 7f56d4d99640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f56b006a430 0x7f56b006c8f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f56c0003eb0 tx=0x7f56c0009040 comp rx=0 tx=0).stop 2023-12-19T18:20:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.666+0000 7f56d4d99640 1 -- 172.21.15.27:0/3527370655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56d00d2350 msgr2=0x7f56d01f58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.666+0000 7f56d4d99640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56d00d2350 0x7f56d01f58e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f56b800d5f0 tx=0x7f56b800dab0 comp rx=0 tx=0).stop 2023-12-19T18:20:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.666+0000 7f56d4d99640 1 -- 172.21.15.27:0/3527370655 shutdown_connections 2023-12-19T18:20:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.666+0000 7f56d4d99640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f56b006a430 0x7f56b006c8f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.666+0000 7f56d4d99640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f56d01604d0 0x7f56d014bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.666+0000 7f56d4d99640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56d00d2350 0x7f56d01f58e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.666+0000 7f56d4d99640 1 --2- 172.21.15.27:0/3527370655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f56d00b4ef0 0x7f56d00c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.666+0000 7f56d4d99640 1 -- 172.21.15.27:0/3527370655 >> 172.21.15.27:0/3527370655 conn(0x7f56d00cf7e0 msgr2=0x7f56d00d1670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.667+0000 7f56d4d99640 1 -- 172.21.15.27:0/3527370655 shutdown_connections 2023-12-19T18:20:59.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.667+0000 7f56d4d99640 1 -- 172.21.15.27:0/3527370655 wait complete. 2023-12-19T18:20:59.694 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2023-12-19T18:20:59.694 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestClusterFull) 2023-12-19T18:20:59.695 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-12-19T18:20:59.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.931+0000 7f53cc788640 1 Processor -- start 2023-12-19T18:20:59.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.931+0000 7f53cc788640 1 -- start start 2023-12-19T18:20:59.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.932+0000 7f53cc788640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f53c415af70 0x7f53c40ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:59.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.932+0000 7f53cc788640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c40cbce0 0x7f53c40ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:59.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.932+0000 7f53cc788640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53c40cb850 con 0x7f53c40ada50 2023-12-19T18:20:59.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.933+0000 7f53cc788640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53c415b350 con 0x7f53c415af70 2023-12-19T18:20:59.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.933+0000 7f53cc788640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40cac40 con 0x7f53c40cbce0 2023-12-19T18:20:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.933+0000 7f53ca524640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f53c415af70 0x7f53c40ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.933+0000 7f53ca524640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f53c415af70 0x7f53c40ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:54274/0 (socket says 172.21.15.27:54274) 2023-12-19T18:20:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.933+0000 7f53ca524640 1 -- 172.21.15.27:0/2738791367 learned_addr learned my addr 172.21.15.27:0/2738791367 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.933+0000 7f53c9d23640 1 --2- 172.21.15.27:0/2738791367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c40cbce0 0x7f53c40ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:59.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.934+0000 7f53ca524640 1 -- 172.21.15.27:0/2738791367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c40cbce0 msgr2=0x7f53c40ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:20:59.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.934+0000 7f53ca524640 1 --2- 172.21.15.27:0/2738791367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c40cbce0 0x7f53c40ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:59.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.934+0000 7f53ca524640 1 -- 172.21.15.27:0/2738791367 >> v1:172.21.15.192:6789/0 conn(0x7f53c40ada50 legacy=0x7f53c40c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:59.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.934+0000 7f53ca524640 1 -- 172.21.15.27:0/2738791367 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c40caf20 con 0x7f53c415af70 2023-12-19T18:20:59.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.935+0000 7f53ca524640 1 --2- 172.21.15.27:0/2738791367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c415af70 0x7f53c40ca540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009870 tx=0x7f53c002b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8ac4f728e7a5fcf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:59.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.936+0000 7f53c9522640 1 -- 172.21.15.27:0/2738791367 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0030840 con 0x7f53c415af70 2023-12-19T18:20:59.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.936+0000 7f53c9522640 1 -- 172.21.15.27:0/2738791367 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53c00309a0 con 0x7f53c415af70 2023-12-19T18:20:59.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.936+0000 7f53c9522640 1 -- 172.21.15.27:0/2738791367 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53c0030d30 con 0x7f53c415af70 2023-12-19T18:20:59.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.937+0000 7f53cc788640 1 -- 172.21.15.27:0/2738791367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c415af70 msgr2=0x7f53c40ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:59.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.937+0000 7f53cc788640 1 --2- 172.21.15.27:0/2738791367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c415af70 0x7f53c40ca540 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009870 tx=0x7f53c002b980 comp rx=0 tx=0).stop 2023-12-19T18:20:59.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.937+0000 7f53cc788640 1 -- 172.21.15.27:0/2738791367 shutdown_connections 2023-12-19T18:20:59.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.937+0000 7f53cc788640 1 --2- 172.21.15.27:0/2738791367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c40cbce0 0x7f53c40ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:59.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.937+0000 7f53cc788640 1 --2- 172.21.15.27:0/2738791367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c415af70 0x7f53c40ca540 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:59.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.937+0000 7f53cc788640 1 -- 172.21.15.27:0/2738791367 >> 172.21.15.27:0/2738791367 conn(0x7f53c406a330 msgr2=0x7f53c40b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:20:59.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.937+0000 7f53cc788640 1 -- 172.21.15.27:0/2738791367 shutdown_connections 2023-12-19T18:20:59.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.937+0000 7f53cc788640 1 -- 172.21.15.27:0/2738791367 wait complete. 2023-12-19T18:20:59.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.939+0000 7f53cc788640 1 Processor -- start 2023-12-19T18:20:59.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.939+0000 7f53cc788640 1 -- start start 2023-12-19T18:20:59.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.940+0000 7f53cc788640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c40ada50 0x7f53c415d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:59.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.940+0000 7f53cc788640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c40cbce0 0x7f53c4158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:59.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.940+0000 7f53cc788640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c415af70 0x7f53c4159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:59.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.940+0000 7f53cc788640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c415cbc0 con 0x7f53c415af70 2023-12-19T18:20:59.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.940+0000 7f53cc788640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c415c040 con 0x7f53c40cbce0 2023-12-19T18:20:59.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.940+0000 7f53cc788640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53c415c320 con 0x7f53c40ada50 2023-12-19T18:20:59.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.940+0000 7f53cad25640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c415af70 0x7f53c4159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:59.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.941+0000 7f53cad25640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c415af70 0x7f53c4159090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39580/0 (socket says 172.21.15.27:39580) 2023-12-19T18:20:59.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.941+0000 7f53cad25640 1 -- 172.21.15.27:0/1666017498 learned_addr learned my addr 172.21.15.27:0/1666017498 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:20:59.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.941+0000 7f53c9d23640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c40cbce0 0x7f53c4158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:59.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.941+0000 7f53ca524640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c40ada50 0x7f53c415d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:59.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.941+0000 7f53cad25640 1 -- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c40ada50 msgr2=0x7f53c415d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:59.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.941+0000 7f53cad25640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c40ada50 0x7f53c415d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:59.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.941+0000 7f53cad25640 1 -- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c40cbce0 msgr2=0x7f53c4158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:20:59.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.941+0000 7f53cad25640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c40cbce0 0x7f53c4158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:20:59.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.941+0000 7f53cad25640 1 -- 172.21.15.27:0/1666017498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c00094d0 con 0x7f53c415af70 2023-12-19T18:20:59.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.941+0000 7f53c9d23640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c40cbce0 0x7f53c4158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:20:59.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.941+0000 7f53cad25640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c415af70 0x7f53c4159090 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f53bc001eb0 tx=0x7f53bc00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:59.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.942+0000 7f53b6ffd640 1 -- 172.21.15.27:0/1666017498 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc0052d0 con 0x7f53c415af70 2023-12-19T18:20:59.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.942+0000 7f53b6ffd640 1 -- 172.21.15.27:0/1666017498 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53bc005430 con 0x7f53c415af70 2023-12-19T18:20:59.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.942+0000 7f53b6ffd640 1 -- 172.21.15.27:0/1666017498 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53bc003810 con 0x7f53c415af70 2023-12-19T18:20:59.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.942+0000 7f53cc788640 1 -- 172.21.15.27:0/1666017498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c415c600 con 0x7f53c415af70 2023-12-19T18:20:59.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.942+0000 7f53cc788640 1 -- 172.21.15.27:0/1666017498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c415d730 con 0x7f53c415af70 2023-12-19T18:20:59.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.943+0000 7f53cc788640 1 -- 172.21.15.27:0/1666017498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c415d8c0 con 0x7f53c415af70 2023-12-19T18:20:59.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.944+0000 7f53b6ffd640 1 -- 172.21.15.27:0/1666017498 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f53bc003970 con 0x7f53c415af70 2023-12-19T18:20:59.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.945+0000 7f53b6ffd640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f53a806a430 0x7f53a806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:20:59.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.945+0000 7f53b6ffd640 1 -- 172.21.15.27:0/1666017498 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f53bc087bd0 con 0x7f53c415af70 2023-12-19T18:20:59.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.946+0000 7f53ca524640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f53a806a430 0x7f53a806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:20:59.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.947+0000 7f53ca524640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f53a806a430 0x7f53a806c8f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f53c0036000 tx=0x7f53c003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:20:59.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:20:59.949+0000 7f53b6ffd640 1 -- 172.21.15.27:0/1666017498 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f53bc0534b0 con 0x7f53c415af70 2023-12-19T18:21:00.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.216+0000 7f53cc788640 1 -- 172.21.15.27:0/1666017498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f53c4068e90 con 0x7f53c415af70 2023-12-19T18:21:00.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.671+0000 7f53b6ffd640 1 -- 172.21.15.27:0/1666017498 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v311) v1 ==== 140+0+0 (secure 0 0 0) 0x7f53bc056ad0 con 0x7f53c415af70 2023-12-19T18:21:00.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.676+0000 7f53cc788640 1 -- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f53a806a430 msgr2=0x7f53a806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:00.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.676+0000 7f53cc788640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f53a806a430 0x7f53a806c8f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f53c0036000 tx=0x7f53c003b040 comp rx=0 tx=0).stop 2023-12-19T18:21:00.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.676+0000 7f53cc788640 1 -- 172.21.15.27:0/1666017498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c415af70 msgr2=0x7f53c4159090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:00.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.676+0000 7f53cc788640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c415af70 0x7f53c4159090 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f53bc001eb0 tx=0x7f53bc00c890 comp rx=0 tx=0).stop 2023-12-19T18:21:00.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.676+0000 7f53cc788640 1 -- 172.21.15.27:0/1666017498 shutdown_connections 2023-12-19T18:21:00.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.676+0000 7f53cc788640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f53a806a430 0x7f53a806c8f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:00.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.676+0000 7f53cc788640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c415af70 0x7f53c4159090 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:00.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.676+0000 7f53cc788640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f53c40cbce0 0x7f53c4158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:00.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.676+0000 7f53cc788640 1 --2- 172.21.15.27:0/1666017498 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f53c40ada50 0x7f53c415d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:00.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.676+0000 7f53cc788640 1 -- 172.21.15.27:0/1666017498 >> 172.21.15.27:0/1666017498 conn(0x7f53c406a330 msgr2=0x7f53c415a240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:00.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.676+0000 7f53cc788640 1 -- 172.21.15.27:0/1666017498 shutdown_connections 2023-12-19T18:21:00.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:00.676+0000 7f53cc788640 1 -- 172.21.15.27:0/1666017498 wait complete. 2023-12-19T18:21:00.700 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_objectstore 2023-12-19T18:21:00.952 INFO:teuthology.orchestra.run.smithi027.stdout:{"osd_objectstore":"memstore"} 2023-12-19T18:21:00.953 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_objectstore": "memstore" } 2023-12-19T18:21:00.953 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:21:01.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.197+0000 7f87a3fff640 1 Processor -- start 2023-12-19T18:21:01.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.198+0000 7f87a3fff640 1 -- start start 2023-12-19T18:21:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.200+0000 7f87a3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f87a415e9d0 0x7f87a40cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.200+0000 7f87a3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87a4152070 0x7f87a40cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.200+0000 7f87a3fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a415b3e0 con 0x7f87a4151c90 2023-12-19T18:21:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.200+0000 7f87a3fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a415eed0 con 0x7f87a415e9d0 2023-12-19T18:21:01.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.200+0000 7f87a3fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a415f1b0 con 0x7f87a4152070 2023-12-19T18:21:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.201+0000 7f879bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f87a415e9d0 0x7f87a40cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.201+0000 7f879bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f87a415e9d0 0x7f87a40cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:54300/0 (socket says 172.21.15.27:54300) 2023-12-19T18:21:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.201+0000 7f879bfff640 1 -- 172.21.15.27:0/1252208529 learned_addr learned my addr 172.21.15.27:0/1252208529 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.201+0000 7f87a37fe640 1 --2- 172.21.15.27:0/1252208529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87a4152070 0x7f87a40cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.201+0000 7f879bfff640 1 -- 172.21.15.27:0/1252208529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87a4152070 msgr2=0x7f87a40cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.201+0000 7f879bfff640 1 --2- 172.21.15.27:0/1252208529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87a4152070 0x7f87a40cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.201+0000 7f879bfff640 1 -- 172.21.15.27:0/1252208529 >> v1:172.21.15.192:6789/0 conn(0x7f87a4151c90 legacy=0x7f87a406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.201+0000 7f879bfff640 1 -- 172.21.15.27:0/1252208529 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a415f490 con 0x7f87a415e9d0 2023-12-19T18:21:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.203+0000 7f879bfff640 1 --2- 172.21.15.27:0/1252208529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a415e9d0 0x7f87a40cee40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f878c0098a0 tx=0x7f878c02b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7106bd911f2a4d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.204+0000 7f87a27fc640 1 -- 172.21.15.27:0/1252208529 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c031640 con 0x7f87a415e9d0 2023-12-19T18:21:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.204+0000 7f87a27fc640 1 -- 172.21.15.27:0/1252208529 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c0317a0 con 0x7f87a415e9d0 2023-12-19T18:21:01.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.204+0000 7f87a27fc640 1 -- 172.21.15.27:0/1252208529 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c031b30 con 0x7f87a415e9d0 2023-12-19T18:21:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.205+0000 7f87a3fff640 1 -- 172.21.15.27:0/1252208529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a415e9d0 msgr2=0x7f87a40cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.205+0000 7f87a3fff640 1 --2- 172.21.15.27:0/1252208529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a415e9d0 0x7f87a40cee40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f878c0098a0 tx=0x7f878c02b8f0 comp rx=0 tx=0).stop 2023-12-19T18:21:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.205+0000 7f87a3fff640 1 -- 172.21.15.27:0/1252208529 shutdown_connections 2023-12-19T18:21:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.205+0000 7f87a3fff640 1 --2- 172.21.15.27:0/1252208529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87a4152070 0x7f87a40cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:01.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.205+0000 7f87a3fff640 1 --2- 172.21.15.27:0/1252208529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a415e9d0 0x7f87a40cee40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:01.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.205+0000 7f87a3fff640 1 -- 172.21.15.27:0/1252208529 >> 172.21.15.27:0/1252208529 conn(0x7f87a406a060 msgr2=0x7f87a406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:01.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.205+0000 7f87a3fff640 1 -- 172.21.15.27:0/1252208529 shutdown_connections 2023-12-19T18:21:01.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.205+0000 7f87a3fff640 1 -- 172.21.15.27:0/1252208529 wait complete. 2023-12-19T18:21:01.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.207+0000 7f87a3fff640 1 Processor -- start 2023-12-19T18:21:01.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.207+0000 7f87a3fff640 1 -- start start 2023-12-19T18:21:01.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.208+0000 7f87a3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a4151c90 0x7f87a40afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:01.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.208+0000 7f87a3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a4152070 0x7f87a414bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:01.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.208+0000 7f87a3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87a415e9d0 0x7f87a414c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:01.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.208+0000 7f87a3fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40b0400 con 0x7f87a4152070 2023-12-19T18:21:01.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.208+0000 7f87a3fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40b2340 con 0x7f87a4151c90 2023-12-19T18:21:01.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.208+0000 7f87a3fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a40b2620 con 0x7f87a415e9d0 2023-12-19T18:21:01.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.208+0000 7f879bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a4152070 0x7f87a414bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.208+0000 7f879bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a4152070 0x7f87a414bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39588/0 (socket says 172.21.15.27:39588) 2023-12-19T18:21:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.208+0000 7f879bfff640 1 -- 172.21.15.27:0/1023880853 learned_addr learned my addr 172.21.15.27:0/1023880853 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.208+0000 7f87a2ffd640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a4151c90 0x7f87a40afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:01.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.208+0000 7f87a37fe640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87a415e9d0 0x7f87a414c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:01.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.209+0000 7f879bfff640 1 -- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87a415e9d0 msgr2=0x7f87a414c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:01.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.209+0000 7f879bfff640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87a415e9d0 0x7f87a414c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:01.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.209+0000 7f879bfff640 1 -- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a4151c90 msgr2=0x7f87a40afec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:01.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.209+0000 7f879bfff640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a4151c90 0x7f87a40afec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:01.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.209+0000 7f879bfff640 1 -- 172.21.15.27:0/1023880853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c0094d0 con 0x7f87a4152070 2023-12-19T18:21:01.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.209+0000 7f879bfff640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a4152070 0x7f87a414bee0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f878c000c00 tx=0x7f878c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:01.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.209+0000 7f87a0ff9640 1 -- 172.21.15.27:0/1023880853 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c03a9f0 con 0x7f87a4152070 2023-12-19T18:21:01.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.209+0000 7f87a0ff9640 1 -- 172.21.15.27:0/1023880853 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c03ab70 con 0x7f87a4152070 2023-12-19T18:21:01.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.209+0000 7f87a3fff640 1 -- 172.21.15.27:0/1023880853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a40b2900 con 0x7f87a4152070 2023-12-19T18:21:01.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.209+0000 7f87a0ff9640 1 -- 172.21.15.27:0/1023880853 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0443f0 con 0x7f87a4152070 2023-12-19T18:21:01.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.210+0000 7f87a3fff640 1 -- 172.21.15.27:0/1023880853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a40b2ec0 con 0x7f87a4152070 2023-12-19T18:21:01.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.211+0000 7f87a3fff640 1 -- 172.21.15.27:0/1023880853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a4066750 con 0x7f87a4152070 2023-12-19T18:21:01.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.212+0000 7f87a0ff9640 1 -- 172.21.15.27:0/1023880853 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f878c02f050 con 0x7f87a4152070 2023-12-19T18:21:01.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.213+0000 7f87a0ff9640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f878406a460 0x7f878406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:01.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.213+0000 7f87a0ff9640 1 -- 172.21.15.27:0/1023880853 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f878c0aa520 con 0x7f87a4152070 2023-12-19T18:21:01.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.214+0000 7f87a2ffd640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f878406a460 0x7f878406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:01.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.215+0000 7f87a2ffd640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f878406a460 0x7f878406c920 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8794000f60 tx=0x7f8794009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:01.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.218+0000 7f87a0ff9640 1 -- 172.21.15.27:0/1023880853 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f878c075d80 con 0x7f87a4152070 2023-12-19T18:21:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.412+0000 7f87a3fff640 1 -- 172.21.15.27:0/1023880853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f87a40b3050 con 0x7f87a4152070 2023-12-19T18:21:01.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.418+0000 7f87a0ff9640 1 -- 172.21.15.27:0/1023880853 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f878c0aa070 con 0x7f87a4152070 2023-12-19T18:21:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.421+0000 7f87a3fff640 1 -- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f878406a460 msgr2=0x7f878406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:01.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.421+0000 7f87a3fff640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f878406a460 0x7f878406c920 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8794000f60 tx=0x7f8794009040 comp rx=0 tx=0).stop 2023-12-19T18:21:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.421+0000 7f87a3fff640 1 -- 172.21.15.27:0/1023880853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a4152070 msgr2=0x7f87a414bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.421+0000 7f87a3fff640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a4152070 0x7f87a414bee0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f878c000c00 tx=0x7f878c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:21:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.421+0000 7f87a3fff640 1 -- 172.21.15.27:0/1023880853 shutdown_connections 2023-12-19T18:21:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.421+0000 7f87a3fff640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f878406a460 0x7f878406c920 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.421+0000 7f87a3fff640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f87a415e9d0 0x7f87a414c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.421+0000 7f87a3fff640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87a4152070 0x7f87a414bee0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.421+0000 7f87a3fff640 1 --2- 172.21.15.27:0/1023880853 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f87a4151c90 0x7f87a40afec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.421+0000 7f87a3fff640 1 -- 172.21.15.27:0/1023880853 >> 172.21.15.27:0/1023880853 conn(0x7f87a406a060 msgr2=0x7f87a406a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.421+0000 7f87a3fff640 1 -- 172.21.15.27:0/1023880853 shutdown_connections 2023-12-19T18:21:01.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:01.422+0000 7f87a3fff640 1 -- 172.21.15.27:0/1023880853 wait complete. 2023-12-19T18:21:01.441 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:21:01.441 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:21:01.564 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:21:01.696 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:21:01.697 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:21:01.697 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:21:01.759 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:21:01.759 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:21:01.759 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:21:01.818 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:21:01.818 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:21:01.818 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:21:01.883 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:21:01.884 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:21:01.884 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:21:01.942 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:21:01.943 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:21:01.943 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:21:02.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.184+0000 7f8cbbfff640 1 Processor -- start 2023-12-19T18:21:02.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.184+0000 7f8cbbfff640 1 -- start start 2023-12-19T18:21:02.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.186+0000 7f8cbbfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc1553a0 0x7f8cbc0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:02.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.186+0000 7f8cbbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8cbc0afeb0 0x7f8cbc15e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.186+0000 7f8cbbfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8cbc15ec60 0x7f8cbc15b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.186+0000 7f8cbbfff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cbc0cbcf0 con 0x7f8cbc0afeb0 2023-12-19T18:21:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.186+0000 7f8cbbfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cbc15f0d0 con 0x7f8cbc15ec60 2023-12-19T18:21:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.186+0000 7f8cbbfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0b0330 con 0x7f8cbc1553a0 2023-12-19T18:21:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.186+0000 7f8cba7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8cbc0afeb0 0x7f8cbc15e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.187+0000 7f8cba7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8cbc0afeb0 0x7f8cbc15e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33142/0 (socket says 172.21.15.27:33142) 2023-12-19T18:21:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.187+0000 7f8cba7fc640 1 -- 172.21.15.27:0/256571912 learned_addr learned my addr 172.21.15.27:0/256571912 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.187+0000 7f8cba7fc640 1 -- 172.21.15.27:0/256571912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc1553a0 msgr2=0x7f8cbc0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.187+0000 7f8cba7fc640 1 --2- 172.21.15.27:0/256571912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc1553a0 0x7f8cbc0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.187+0000 7f8cba7fc640 1 -- 172.21.15.27:0/256571912 >> v2:172.21.15.192:3300/0 conn(0x7f8cbc15ec60 msgr2=0x7f8cbc15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.187+0000 7f8cba7fc640 1 --2- 172.21.15.27:0/256571912 >> v2:172.21.15.192:3300/0 conn(0x7f8cbc15ec60 0x7f8cbc15b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.187+0000 7f8cba7fc640 1 -- 172.21.15.27:0/256571912 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cbc0cb350 con 0x7f8cbc0afeb0 2023-12-19T18:21:02.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.188+0000 7f8cba7fc640 1 --2- 172.21.15.27:0/256571912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cbc0afeb0 0x7f8cbc15e720 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc06f2d0 tx=0x7f8cb002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5b748469102da0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:02.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.189+0000 7f8cb9ffb640 1 -- 172.21.15.27:0/256571912 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00317c0 con 0x7f8cbc0afeb0 2023-12-19T18:21:02.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.189+0000 7f8cb9ffb640 1 -- 172.21.15.27:0/256571912 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb0031920 con 0x7f8cbc0afeb0 2023-12-19T18:21:02.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.189+0000 7f8cb9ffb640 1 -- 172.21.15.27:0/256571912 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0031cb0 con 0x7f8cbc0afeb0 2023-12-19T18:21:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.189+0000 7f8cbbfff640 1 -- 172.21.15.27:0/256571912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cbc0afeb0 msgr2=0x7f8cbc15e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.189+0000 7f8cbbfff640 1 --2- 172.21.15.27:0/256571912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cbc0afeb0 0x7f8cbc15e720 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc06f2d0 tx=0x7f8cb002b860 comp rx=0 tx=0).stop 2023-12-19T18:21:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.190+0000 7f8cbbfff640 1 -- 172.21.15.27:0/256571912 shutdown_connections 2023-12-19T18:21:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.190+0000 7f8cbbfff640 1 --2- 172.21.15.27:0/256571912 >> v2:172.21.15.192:3300/0 conn(0x7f8cbc15ec60 0x7f8cbc15b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.190+0000 7f8cbbfff640 1 --2- 172.21.15.27:0/256571912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cbc0afeb0 0x7f8cbc15e720 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.190+0000 7f8cbbfff640 1 --2- 172.21.15.27:0/256571912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc1553a0 0x7f8cbc0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.190+0000 7f8cbbfff640 1 -- 172.21.15.27:0/256571912 >> 172.21.15.27:0/256571912 conn(0x7f8cbc069c70 msgr2=0x7f8cbc06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.190+0000 7f8cbbfff640 1 -- 172.21.15.27:0/256571912 shutdown_connections 2023-12-19T18:21:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.190+0000 7f8cbbfff640 1 -- 172.21.15.27:0/256571912 wait complete. 2023-12-19T18:21:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.191+0000 7f8cbbfff640 1 Processor -- start 2023-12-19T18:21:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.192+0000 7f8cbbfff640 1 -- start start 2023-12-19T18:21:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cbc0afeb0 0x7f8cbc1500c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbbfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc1553a0 0x7f8cbc0cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cbc15ec60 0x7f8cbc1518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbbfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc150600 con 0x7f8cbc15ec60 2023-12-19T18:21:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbbfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0ccf60 con 0x7f8cbc0afeb0 2023-12-19T18:21:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbbfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cbc0cd240 con 0x7f8cbc1553a0 2023-12-19T18:21:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cbc15ec60 0x7f8cbc1518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cbc15ec60 0x7f8cbc1518e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33148/0 (socket says 172.21.15.27:33148) 2023-12-19T18:21:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbb7fe640 1 -- 172.21.15.27:0/3697419393 learned_addr learned my addr 172.21.15.27:0/3697419393 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbaffd640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cbc0afeb0 0x7f8cbc1500c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cba7fc640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc1553a0 0x7f8cbc0cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbb7fe640 1 -- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc1553a0 msgr2=0x7f8cbc0cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbb7fe640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc1553a0 0x7f8cbc0cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbb7fe640 1 -- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cbc0afeb0 msgr2=0x7f8cbc1500c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbb7fe640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cbc0afeb0 0x7f8cbc1500c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cbb7fe640 1 -- 172.21.15.27:0/3697419393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb00094d0 con 0x7f8cbc15ec60 2023-12-19T18:21:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.193+0000 7f8cba7fc640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc1553a0 0x7f8cbc0cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.194+0000 7f8cbb7fe640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cbc15ec60 0x7f8cbc1518e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400b970 tx=0x7f8cb400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.194+0000 7f8ca37fe640 1 -- 172.21.15.27:0/3697419393 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb400cb40 con 0x7f8cbc15ec60 2023-12-19T18:21:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.194+0000 7f8ca37fe640 1 -- 172.21.15.27:0/3697419393 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb400ccc0 con 0x7f8cbc15ec60 2023-12-19T18:21:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.194+0000 7f8ca37fe640 1 -- 172.21.15.27:0/3697419393 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4014680 con 0x7f8cbc15ec60 2023-12-19T18:21:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.194+0000 7f8cbbfff640 1 -- 172.21.15.27:0/3697419393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cbc0cd520 con 0x7f8cbc15ec60 2023-12-19T18:21:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.194+0000 7f8cbbfff640 1 -- 172.21.15.27:0/3697419393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cbc0cdae0 con 0x7f8cbc15ec60 2023-12-19T18:21:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.195+0000 7f8cbbfff640 1 -- 172.21.15.27:0/3697419393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cbc0671b0 con 0x7f8cbc15ec60 2023-12-19T18:21:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.196+0000 7f8ca37fe640 1 -- 172.21.15.27:0/3697419393 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8cb401d3b0 con 0x7f8cbc15ec60 2023-12-19T18:21:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.197+0000 7f8ca37fe640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ca806a480 0x7f8ca806c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.197+0000 7f8ca37fe640 1 -- 172.21.15.27:0/3697419393 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8cb40885e0 con 0x7f8cbc15ec60 2023-12-19T18:21:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.198+0000 7f8cbaffd640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ca806a480 0x7f8ca806c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:02.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.199+0000 7f8cbaffd640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ca806a480 0x7f8ca806c940 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc151070 tx=0x7f8cac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:02.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.202+0000 7f8ca37fe640 1 -- 172.21.15.27:0/3697419393 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8cbc0671b0 con 0x7f8cbc15ec60 2023-12-19T18:21:02.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.441+0000 7f8cbbfff640 1 -- 172.21.15.27:0/3697419393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8cbc0cdd00 con 0x7f8cbc15ec60 2023-12-19T18:21:02.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.442+0000 7f8ca37fe640 1 -- 172.21.15.27:0/3697419393 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f8cb4019de0 con 0x7f8cbc15ec60 2023-12-19T18:21:02.442 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:02.442 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":40,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5189,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2860412135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2860412135},{"type":"v1","addr":"172.21.15.192:6837","nonce":2860412135}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":5644,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/797098667","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":797098667},{"type":"v1","addr":"172.21.15.192:6836","nonce":797098667}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":6507,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/4009181263","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":4009181263},{"type":"v1","addr":"172.21.15.27:6833","nonce":4009181263}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33},{"gid":6822,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1133028681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1133028681},{"type":"v1","addr":"172.21.15.27:6835","nonce":1133028681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":40}],"filesystems":[]} 2023-12-19T18:21:02.443 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 40 2023-12-19T18:21:02.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.445+0000 7f8cbbfff640 1 -- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ca806a480 msgr2=0x7f8ca806c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.445+0000 7f8cbbfff640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ca806a480 0x7f8ca806c940 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f8cbc151070 tx=0x7f8cac008040 comp rx=0 tx=0).stop 2023-12-19T18:21:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.445+0000 7f8cbbfff640 1 -- 172.21.15.27:0/3697419393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cbc15ec60 msgr2=0x7f8cbc1518e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.445+0000 7f8cbbfff640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cbc15ec60 0x7f8cbc1518e0 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400b970 tx=0x7f8cb400be30 comp rx=0 tx=0).stop 2023-12-19T18:21:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.445+0000 7f8cbbfff640 1 -- 172.21.15.27:0/3697419393 shutdown_connections 2023-12-19T18:21:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.445+0000 7f8cbbfff640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8ca806a480 0x7f8ca806c940 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.445+0000 7f8cbbfff640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8cbc15ec60 0x7f8cbc1518e0 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.445+0000 7f8cbbfff640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8cbc1553a0 0x7f8cbc0cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.445+0000 7f8cbbfff640 1 --2- 172.21.15.27:0/3697419393 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8cbc0afeb0 0x7f8cbc1500c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.445+0000 7f8cbbfff640 1 -- 172.21.15.27:0/3697419393 >> 172.21.15.27:0/3697419393 conn(0x7f8cbc069c70 msgr2=0x7f8cbc0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.445+0000 7f8cbbfff640 1 -- 172.21.15.27:0/3697419393 shutdown_connections 2023-12-19T18:21:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.445+0000 7f8cbbfff640 1 -- 172.21.15.27:0/3697419393 wait complete. 2023-12-19T18:21:02.464 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:21:02.464 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:21:02.464 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:21:02.464 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:21:02.465 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:21:02.465 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:21:02.465 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:21:02.465 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:21:02.465 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:21:02.465 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:21:02.465 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:21:02.465 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:21:02.466 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:21:02.466 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:21:02.466 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:21:02.466 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:21:02.483 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:21:02.483 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:21:02.484 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:21:02.484 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:21:02.485 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:21:02.486 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:21:02.486 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:21:02.486 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:21:02.533 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:21:02.533 DEBUG:teuthology.parallel:result is None 2023-12-19T18:21:02.534 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:21:02.535 DEBUG:teuthology.parallel:result is None 2023-12-19T18:21:02.540 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:21:02.540 DEBUG:teuthology.parallel:result is None 2023-12-19T18:21:02.541 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:21:02.542 DEBUG:teuthology.parallel:result is None 2023-12-19T18:21:02.542 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:21:02.697 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-19T18:21:02.698 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-19T18:21:02.705 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:21:02.706 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:21:02.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.746+0000 7fccc8de7640 1 Processor -- start 2023-12-19T18:21:02.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.746+0000 7fccc8de7640 1 -- start start 2023-12-19T18:21:02.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.747+0000 7fccc8de7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccc415d3e0 0x7fccc4161890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:02.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.747+0000 7fccc8de7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc40d1530 con 0x7fccc406c360 2023-12-19T18:21:02.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.747+0000 7fccc8de7640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc4161dd0 con 0x7fccc406c7e0 2023-12-19T18:21:02.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.747+0000 7fccc8de7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc406bba0 con 0x7fccc415d3e0 2023-12-19T18:21:02.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.747+0000 7fccc1d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fccc406c360 0x7fccc406ad80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40882/0 (socket says 172.21.15.27:40882) 2023-12-19T18:21:02.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.748+0000 7fccc1d9b640 1 -- 172.21.15.27:0/3857622852 learned_addr learned my addr 172.21.15.27:0/3857622852 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:02.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.748+0000 7fccc259c640 1 --2- 172.21.15.27:0/3857622852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccc415d3e0 0x7fccc4161890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:02.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.748+0000 7fccc159a640 1 -- 172.21.15.27:0/3857622852 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1834155353 0 0) 0x7fccc40d1530 con 0x7fccc406c360 2023-12-19T18:21:02.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.748+0000 7fccc159a640 1 -- 172.21.15.27:0/3857622852 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccac003520 con 0x7fccc406c360 2023-12-19T18:21:02.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.748+0000 7fccc159a640 1 -- 172.21.15.27:0/3857622852 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 765935772 0 0) 0x7fccc4161dd0 con 0x7fccc406c7e0 2023-12-19T18:21:02.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.748+0000 7fccc159a640 1 -- 172.21.15.27:0/3857622852 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccc40d1530 con 0x7fccc406c7e0 2023-12-19T18:21:02.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.748+0000 7fccc259c640 1 -- 172.21.15.27:0/3857622852 >> v1:172.21.15.192:6789/0 conn(0x7fccc406c7e0 legacy=0x7fccc406b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:02.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.748+0000 7fccc259c640 1 -- 172.21.15.27:0/3857622852 >> v1:172.21.15.27:6789/0 conn(0x7fccc406c360 legacy=0x7fccc406ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:02.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.748+0000 7fccc259c640 1 -- 172.21.15.27:0/3857622852 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc415d7c0 con 0x7fccc415d3e0 2023-12-19T18:21:02.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.750+0000 7fccc259c640 1 --2- 172.21.15.27:0/3857622852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccc415d3e0 0x7fccc4161890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fccb40097d0 tx=0x7fccb402b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=997891a48f63b004 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:02.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.750+0000 7fccc159a640 1 -- 172.21.15.27:0/3857622852 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4031890 con 0x7fccc415d3e0 2023-12-19T18:21:02.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.750+0000 7fccc159a640 1 -- 172.21.15.27:0/3857622852 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb40319f0 con 0x7fccc415d3e0 2023-12-19T18:21:02.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.750+0000 7fccc159a640 1 -- 172.21.15.27:0/3857622852 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4031d60 con 0x7fccc415d3e0 2023-12-19T18:21:02.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.751+0000 7fccc8de7640 1 -- 172.21.15.27:0/3857622852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccc415d3e0 msgr2=0x7fccc4161890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:02.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.751+0000 7fccc8de7640 1 --2- 172.21.15.27:0/3857622852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccc415d3e0 0x7fccc4161890 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fccb40097d0 tx=0x7fccb402b930 comp rx=0 tx=0).stop 2023-12-19T18:21:02.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.751+0000 7fccc8de7640 1 -- 172.21.15.27:0/3857622852 shutdown_connections 2023-12-19T18:21:02.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.751+0000 7fccc8de7640 1 --2- 172.21.15.27:0/3857622852 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccc415d3e0 0x7fccc4161890 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.751+0000 7fccc8de7640 1 -- 172.21.15.27:0/3857622852 >> 172.21.15.27:0/3857622852 conn(0x7fccc4072c40 msgr2=0x7fccc4073020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:02.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.751+0000 7fccc8de7640 1 -- 172.21.15.27:0/3857622852 shutdown_connections 2023-12-19T18:21:02.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.751+0000 7fccc8de7640 1 -- 172.21.15.27:0/3857622852 wait complete. 2023-12-19T18:21:02.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.753+0000 7fccc8de7640 1 Processor -- start 2023-12-19T18:21:02.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.753+0000 7fccc8de7640 1 -- start start 2023-12-19T18:21:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.754+0000 7fccc8de7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc406c360 0x7fccc4157f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.754+0000 7fccc8de7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccc406c7e0 0x7fccc40c6f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.754+0000 7fccc8de7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccc415d3e0 0x7fccc40b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.754+0000 7fccc8de7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40c5fe0 con 0x7fccc406c360 2023-12-19T18:21:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.754+0000 7fccc8de7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40c74c0 con 0x7fccc415d3e0 2023-12-19T18:21:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.754+0000 7fccc8de7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc40c61c0 con 0x7fccc406c7e0 2023-12-19T18:21:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.754+0000 7fccc259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc406c360 0x7fccc4157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.754+0000 7fccc259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc406c360 0x7fccc4157f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33186/0 (socket says 172.21.15.27:33186) 2023-12-19T18:21:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.754+0000 7fccc259c640 1 -- 172.21.15.27:0/514948579 learned_addr learned my addr 172.21.15.27:0/514948579 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:02.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.754+0000 7fccc1d9b640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccc406c7e0 0x7fccc40c6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:02.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.754+0000 7fccc2d9d640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccc415d3e0 0x7fccc40b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:02.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.755+0000 7fccc259c640 1 -- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccc406c7e0 msgr2=0x7fccc40c6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:02.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.755+0000 7fccc259c640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccc406c7e0 0x7fccc40c6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.755+0000 7fccc259c640 1 -- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccc415d3e0 msgr2=0x7fccc40b1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:02.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.755+0000 7fccc259c640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccc415d3e0 0x7fccc40b1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:02.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.755+0000 7fccc259c640 1 -- 172.21.15.27:0/514948579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb4009400 con 0x7fccc406c360 2023-12-19T18:21:02.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.755+0000 7fccc259c640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc406c360 0x7fccc4157f60 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fccb402b970 tx=0x7fccb40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:02.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.756+0000 7fccb2ffd640 1 -- 172.21.15.27:0/514948579 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb403abe0 con 0x7fccc406c360 2023-12-19T18:21:02.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.756+0000 7fccb2ffd640 1 -- 172.21.15.27:0/514948579 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb403ad40 con 0x7fccc406c360 2023-12-19T18:21:02.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.756+0000 7fccc8de7640 1 -- 172.21.15.27:0/514948579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc40c64a0 con 0x7fccc406c360 2023-12-19T18:21:02.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.756+0000 7fccb2ffd640 1 -- 172.21.15.27:0/514948579 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb4044610 con 0x7fccc406c360 2023-12-19T18:21:02.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.756+0000 7fccc8de7640 1 -- 172.21.15.27:0/514948579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc40c6a60 con 0x7fccc406c360 2023-12-19T18:21:02.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.757+0000 7fccc8de7640 1 -- 172.21.15.27:0/514948579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc4068e90 con 0x7fccc406c360 2023-12-19T18:21:02.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.758+0000 7fccb2ffd640 1 -- 172.21.15.27:0/514948579 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fccb404d450 con 0x7fccc406c360 2023-12-19T18:21:02.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.759+0000 7fccb2ffd640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcca406a430 0x7fcca406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:02.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.759+0000 7fccb2ffd640 1 -- 172.21.15.27:0/514948579 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fccb40aaaf0 con 0x7fccc406c360 2023-12-19T18:21:02.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.759+0000 7fccc1d9b640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcca406a430 0x7fcca406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:02.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.760+0000 7fccc1d9b640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcca406a430 0x7fcca406c8f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fccc40cb610 tx=0x7fccb8007d70 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:02.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.763+0000 7fccb2ffd640 1 -- 172.21.15.27:0/514948579 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fccb40763f0 con 0x7fccc406c360 2023-12-19T18:21:02.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:02.970+0000 7fccc8de7640 1 -- 172.21.15.27:0/514948579 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fccc40b2e30 con 0x7fccc406c360 2023-12-19T18:21:03.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:03.744+0000 7fccb2ffd640 1 -- 172.21.15.27:0/514948579 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v113) v1 ==== 97+0+0 (secure 0 0 0) 0x7fccb4079a10 con 0x7fccc406c360 2023-12-19T18:21:03.744 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2023-12-19T18:21:03.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:03.747+0000 7fccc8de7640 1 -- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcca406a430 msgr2=0x7fcca406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:03.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:03.747+0000 7fccc8de7640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcca406a430 0x7fcca406c8f0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fccc40cb610 tx=0x7fccb8007d70 comp rx=0 tx=0).stop 2023-12-19T18:21:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:03.747+0000 7fccc8de7640 1 -- 172.21.15.27:0/514948579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc406c360 msgr2=0x7fccc4157f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:03.747+0000 7fccc8de7640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc406c360 0x7fccc4157f60 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fccb402b970 tx=0x7fccb40046c0 comp rx=0 tx=0).stop 2023-12-19T18:21:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:03.747+0000 7fccc8de7640 1 -- 172.21.15.27:0/514948579 shutdown_connections 2023-12-19T18:21:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:03.747+0000 7fccc8de7640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcca406a430 0x7fcca406c8f0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:03.747+0000 7fccc8de7640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccc415d3e0 0x7fccc40b1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:03.747+0000 7fccc8de7640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccc406c7e0 0x7fccc40c6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:03.747+0000 7fccc8de7640 1 --2- 172.21.15.27:0/514948579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc406c360 0x7fccc4157f60 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:03.747+0000 7fccc8de7640 1 -- 172.21.15.27:0/514948579 >> 172.21.15.27:0/514948579 conn(0x7fccc4072c40 msgr2=0x7fccc406ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:03.748+0000 7fccc8de7640 1 -- 172.21.15.27:0/514948579 shutdown_connections 2023-12-19T18:21:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:03.748+0000 7fccc8de7640 1 -- 172.21.15.27:0/514948579 wait complete. 2023-12-19T18:21:03.771 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:21:04.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.005+0000 7ff4af27e640 1 Processor -- start 2023-12-19T18:21:04.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.006+0000 7ff4af27e640 1 -- start start 2023-12-19T18:21:04.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.007+0000 7ff4af27e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff4a815ac50 0x7ff4a8070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:04.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.007+0000 7ff4af27e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff4a80af360 0x7ff4a806ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:04.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.007+0000 7ff4af27e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4a80adf10 con 0x7ff4a80aef20 2023-12-19T18:21:04.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.007+0000 7ff4af27e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a80adac0 con 0x7ff4a80af360 2023-12-19T18:21:04.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.008+0000 7ff4af27e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4a80ae160 con 0x7ff4a815ac50 2023-12-19T18:21:04.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.008+0000 7ff4aea7d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff4a80af360 0x7ff4a806ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:04.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.008+0000 7ff4aea7d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff4a80af360 0x7ff4a806ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33200/0 (socket says 172.21.15.27:33200) 2023-12-19T18:21:04.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.008+0000 7ff4aea7d640 1 -- 172.21.15.27:0/1924678817 learned_addr learned my addr 172.21.15.27:0/1924678817 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:04.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.008+0000 7ff4ae27c640 1 --2- 172.21.15.27:0/1924678817 >> v2:172.21.15.192:3300/0 conn(0x7ff4a815ac50 0x7ff4a8070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:04.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.008+0000 7ff4aea7d640 1 -- 172.21.15.27:0/1924678817 >> v2:172.21.15.192:3300/0 conn(0x7ff4a815ac50 msgr2=0x7ff4a8070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:04.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.008+0000 7ff4aea7d640 1 --2- 172.21.15.27:0/1924678817 >> v2:172.21.15.192:3300/0 conn(0x7ff4a815ac50 0x7ff4a8070fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.008+0000 7ff4aea7d640 1 -- 172.21.15.27:0/1924678817 >> v1:172.21.15.27:6789/0 conn(0x7ff4a80aef20 legacy=0x7ff4a806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:04.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.008+0000 7ff4ad27a640 1 -- 172.21.15.27:0/1924678817 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3632548612 0 0) 0x7ff4a80adf10 con 0x7ff4a80aef20 2023-12-19T18:21:04.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.009+0000 7ff4aea7d640 1 -- 172.21.15.27:0/1924678817 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a80ae440 con 0x7ff4a80af360 2023-12-19T18:21:04.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.009+0000 7ff4ae27c640 1 --2- 172.21.15.27:0/1924678817 >> v2:172.21.15.192:3300/0 conn(0x7ff4a815ac50 0x7ff4a8070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:21:04.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.010+0000 7ff4aea7d640 1 --2- 172.21.15.27:0/1924678817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a80af360 0x7ff4a806ba20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff4a806cfb0 tx=0x7ff4a002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a84bae7d60484fb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:04.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.010+0000 7ff4ad27a640 1 -- 172.21.15.27:0/1924678817 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a00317e0 con 0x7ff4a80af360 2023-12-19T18:21:04.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.010+0000 7ff4ad27a640 1 -- 172.21.15.27:0/1924678817 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a0031940 con 0x7ff4a80af360 2023-12-19T18:21:04.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.010+0000 7ff4ad27a640 1 -- 172.21.15.27:0/1924678817 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0031cd0 con 0x7ff4a80af360 2023-12-19T18:21:04.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.011+0000 7ff4af27e640 1 -- 172.21.15.27:0/1924678817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a80af360 msgr2=0x7ff4a806ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:04.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.011+0000 7ff4af27e640 1 --2- 172.21.15.27:0/1924678817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a80af360 0x7ff4a806ba20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7ff4a806cfb0 tx=0x7ff4a002b880 comp rx=0 tx=0).stop 2023-12-19T18:21:04.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.011+0000 7ff4af27e640 1 -- 172.21.15.27:0/1924678817 shutdown_connections 2023-12-19T18:21:04.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.011+0000 7ff4af27e640 1 --2- 172.21.15.27:0/1924678817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a80af360 0x7ff4a806ba20 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.011+0000 7ff4af27e640 1 --2- 172.21.15.27:0/1924678817 >> v2:172.21.15.192:3300/0 conn(0x7ff4a815ac50 0x7ff4a8070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.011+0000 7ff4af27e640 1 -- 172.21.15.27:0/1924678817 >> 172.21.15.27:0/1924678817 conn(0x7ff4a806a7d0 msgr2=0x7ff4a80ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:04.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.011+0000 7ff4af27e640 1 -- 172.21.15.27:0/1924678817 shutdown_connections 2023-12-19T18:21:04.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.011+0000 7ff4af27e640 1 -- 172.21.15.27:0/1924678817 wait complete. 2023-12-19T18:21:04.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.013+0000 7ff4af27e640 1 Processor -- start 2023-12-19T18:21:04.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.013+0000 7ff4af27e640 1 -- start start 2023-12-19T18:21:04.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.014+0000 7ff4af27e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4a80aef20 0x7ff4a80acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:04.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.014+0000 7ff4af27e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4a80af360 0x7ff4a80b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:04.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.014+0000 7ff4af27e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a815ac50 0x7ff4a813f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:04.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.014+0000 7ff4af27e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80b39e0 con 0x7ff4a815ac50 2023-12-19T18:21:04.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.014+0000 7ff4af27e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80b2dd0 con 0x7ff4a80af360 2023-12-19T18:21:04.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.014+0000 7ff4af27e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a80b30b0 con 0x7ff4a80aef20 2023-12-19T18:21:04.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.014+0000 7ff4aea7d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a815ac50 0x7ff4a813f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:04.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.014+0000 7ff4aea7d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a815ac50 0x7ff4a813f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33202/0 (socket says 172.21.15.27:33202) 2023-12-19T18:21:04.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.014+0000 7ff4aea7d640 1 -- 172.21.15.27:0/231087414 learned_addr learned my addr 172.21.15.27:0/231087414 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:04.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.014+0000 7ff4ada7b640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4a80af360 0x7ff4a80b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:04.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.015+0000 7ff4ae27c640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4a80aef20 0x7ff4a80acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:04.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.015+0000 7ff4aea7d640 1 -- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4a80aef20 msgr2=0x7ff4a80acb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:04.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.015+0000 7ff4aea7d640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4a80aef20 0x7ff4a80acb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.015+0000 7ff4aea7d640 1 -- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4a80af360 msgr2=0x7ff4a80b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:04.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.015+0000 7ff4aea7d640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4a80af360 0x7ff4a80b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.015+0000 7ff4aea7d640 1 -- 172.21.15.27:0/231087414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a00094d0 con 0x7ff4a815ac50 2023-12-19T18:21:04.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.015+0000 7ff4aea7d640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a815ac50 0x7ff4a813f0e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00090b0 tx=0x7ff4a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:04.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.015+0000 7ff4a6ffd640 1 -- 172.21.15.27:0/231087414 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0032880 con 0x7ff4a815ac50 2023-12-19T18:21:04.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.015+0000 7ff4a6ffd640 1 -- 172.21.15.27:0/231087414 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4a0032a20 con 0x7ff4a815ac50 2023-12-19T18:21:04.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.015+0000 7ff4a6ffd640 1 -- 172.21.15.27:0/231087414 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4a0032dd0 con 0x7ff4a815ac50 2023-12-19T18:21:04.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.015+0000 7ff4af27e640 1 -- 172.21.15.27:0/231087414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4a80b3390 con 0x7ff4a815ac50 2023-12-19T18:21:04.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.016+0000 7ff4af27e640 1 -- 172.21.15.27:0/231087414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4a80ad070 con 0x7ff4a815ac50 2023-12-19T18:21:04.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.017+0000 7ff4af27e640 1 -- 172.21.15.27:0/231087414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4a80b3b50 con 0x7ff4a815ac50 2023-12-19T18:21:04.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.018+0000 7ff4a6ffd640 1 -- 172.21.15.27:0/231087414 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff4a004e450 con 0x7ff4a815ac50 2023-12-19T18:21:04.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.019+0000 7ff4a6ffd640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff49406a460 0x7ff49406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:04.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.019+0000 7ff4a6ffd640 1 -- 172.21.15.27:0/231087414 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff4a00aa720 con 0x7ff4a815ac50 2023-12-19T18:21:04.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.019+0000 7ff4ae27c640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff49406a460 0x7ff49406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:04.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.020+0000 7ff4ae27c640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff49406a460 0x7ff49406c920 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff498003fd0 tx=0x7ff498003ce0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:04.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.023+0000 7ff4a6ffd640 1 -- 172.21.15.27:0/231087414 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff4a0075fc0 con 0x7ff4a815ac50 2023-12-19T18:21:04.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.284+0000 7ff4af27e640 1 -- 172.21.15.27:0/231087414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff4a80ad260 con 0x7ff4a815ac50 2023-12-19T18:21:04.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.285+0000 7ff4a6ffd640 1 -- 172.21.15.27:0/231087414 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff4a00795e0 con 0x7ff4a815ac50 2023-12-19T18:21:04.285 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:04.285 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFl+4ruHBAAXwbpmT43SmlvNbuZj5e+bQ==", 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2023-12-19T18:21:04.286 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmZyUBhAAG3kZkXvJIo7N9f1j+DPCSQ==", 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlChNOMBAAX4GV3iIGkSQdAsYu9oCPDw==", 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.287 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2023-12-19T18:21:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlTmFXFhAABA0NMUxKBV1ExAhYc5RFbQ==", 2023-12-19T18:21:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:21:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:21:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:21:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.288 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFluleFIBAAs1goFVMOX4CbjwFhg0KoBg==", 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFlInW3MxAAI4hY1GBxXYXDswKKrEyDTg==", 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:21:04.289 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFlAWZ5CRAA82/2r9/sT0L9VlGHbXksQw==", 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2023-12-19T18:21:04.290 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFl1IvaGRAAqVacDDPLx+NqBOGp6MeX2w==", 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlr0BBHRAA+7gfs/RG6DtABnYPg059Tw==", 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2023-12-19T18:21:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlXZbELRAAKD0qOkhk7skfc4xcAC8Lhg==", 2023-12-19T18:21:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:21:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:21:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2023-12-19T18:21:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlF07MARAA2TJJ+5C1HIU+SVhqmKaVHA==", 2023-12-19T18:21:04.292 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlISPGEhAAdFIRRkXlLyOx/oMZ4MgL9Q==", 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmNM9JRAARgS20O8zpm6H70WDv+lgGw==", 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlWoZ8AhAADWvKVzeDhDGHtYNUFIOWsA==", 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.294 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFllWFVMhAAD9ebDOppHTODVQjg6ppP3g==", 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlwbIQDRAAt/h55zBNCEId0vkYSUQ3TQ==", 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB53YFloCb6CxAAmZQyz5XOEqDOOiROJHLzxQ==", 2023-12-19T18:21:04.296 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFltBhxDxAAiAS9CYPY86IMmJ7NGgA2lw==", 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2023-12-19T18:21:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlUnAhBBAAi++W9XK3o3Y/185hB3LuIg==", 2023-12-19T18:21:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:21:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:21:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:21:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:21:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:04.298 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:04.298 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:21:04.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.288+0000 7ff4af27e640 1 -- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff49406a460 msgr2=0x7ff49406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.288+0000 7ff4af27e640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff49406a460 0x7ff49406c920 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ff498003fd0 tx=0x7ff498003ce0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.288+0000 7ff4af27e640 1 -- 172.21.15.27:0/231087414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a815ac50 msgr2=0x7ff4a813f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.288+0000 7ff4af27e640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a815ac50 0x7ff4a813f0e0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7ff4a00090b0 tx=0x7ff4a00046c0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.288+0000 7ff4af27e640 1 -- 172.21.15.27:0/231087414 shutdown_connections 2023-12-19T18:21:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.288+0000 7ff4af27e640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff49406a460 0x7ff49406c920 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.288+0000 7ff4af27e640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4a815ac50 0x7ff4a813f0e0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.288+0000 7ff4af27e640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4a80af360 0x7ff4a80b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.288+0000 7ff4af27e640 1 --2- 172.21.15.27:0/231087414 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4a80aef20 0x7ff4a80acb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.288+0000 7ff4af27e640 1 -- 172.21.15.27:0/231087414 >> 172.21.15.27:0/231087414 conn(0x7ff4a806a7d0 msgr2=0x7ff4a8159c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.288+0000 7ff4af27e640 1 -- 172.21.15.27:0/231087414 shutdown_connections 2023-12-19T18:21:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.288+0000 7ff4af27e640 1 -- 172.21.15.27:0/231087414 wait complete. 2023-12-19T18:21:04.308 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:21:04.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.541+0000 7f0a27fff640 1 Processor -- start 2023-12-19T18:21:04.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.541+0000 7f0a27fff640 1 -- start start 2023-12-19T18:21:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.543+0000 7f0a27fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a280cb300 0x7f0a280cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.543+0000 7f0a27fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0a28161010 0x7f0a280cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.543+0000 7f0a27fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a280cbc20 con 0x7f0a280c8f90 2023-12-19T18:21:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.543+0000 7f0a27fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a280c4480 con 0x7f0a28161010 2023-12-19T18:21:04.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.543+0000 7f0a27fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a281468c0 con 0x7f0a280cb300 2023-12-19T18:21:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.544+0000 7f0a277fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0a28161010 0x7f0a280cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.544+0000 7f0a277fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0a28161010 0x7f0a280cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:40850/0 (socket says 172.21.15.27:40850) 2023-12-19T18:21:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.544+0000 7f0a26ffd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a280cb300 0x7f0a280cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:04.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.544+0000 7f0a277fe640 1 -- 172.21.15.27:0/2632121011 learned_addr learned my addr 172.21.15.27:0/2632121011 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:04.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.545+0000 7f0a277fe640 1 -- 172.21.15.27:0/2632121011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a280cb300 msgr2=0x7f0a280cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:04.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.545+0000 7f0a277fe640 1 --2- 172.21.15.27:0/2632121011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a280cb300 0x7f0a280cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.545+0000 7f0a277fe640 1 -- 172.21.15.27:0/2632121011 >> v1:172.21.15.192:6789/0 conn(0x7f0a280c8f90 legacy=0x7f0a28160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:04.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.545+0000 7f0a277fe640 1 -- 172.21.15.27:0/2632121011 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a28145d60 con 0x7f0a28161010 2023-12-19T18:21:04.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.545+0000 7f0a26ffd640 1 --2- 172.21.15.27:0/2632121011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a280cb300 0x7f0a280cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:21:04.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.546+0000 7f0a277fe640 1 --2- 172.21.15.27:0/2632121011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a28161010 0x7f0a280cd090 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009e50 tx=0x7f0a2002bf60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c98a12abe98aaf3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:04.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.546+0000 7f0a267fc640 1 -- 172.21.15.27:0/2632121011 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2002cb00 con 0x7f0a28161010 2023-12-19T18:21:04.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.547+0000 7f0a267fc640 1 -- 172.21.15.27:0/2632121011 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a2002cc60 con 0x7f0a28161010 2023-12-19T18:21:04.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.547+0000 7f0a267fc640 1 -- 172.21.15.27:0/2632121011 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a200343f0 con 0x7f0a28161010 2023-12-19T18:21:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.547+0000 7f0a27fff640 1 -- 172.21.15.27:0/2632121011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a28161010 msgr2=0x7f0a280cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.547+0000 7f0a27fff640 1 --2- 172.21.15.27:0/2632121011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a28161010 0x7f0a280cd090 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f0a20009e50 tx=0x7f0a2002bf60 comp rx=0 tx=0).stop 2023-12-19T18:21:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.547+0000 7f0a27fff640 1 -- 172.21.15.27:0/2632121011 shutdown_connections 2023-12-19T18:21:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.547+0000 7f0a27fff640 1 --2- 172.21.15.27:0/2632121011 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a28161010 0x7f0a280cd090 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.547+0000 7f0a27fff640 1 --2- 172.21.15.27:0/2632121011 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a280cb300 0x7f0a280cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.547+0000 7f0a27fff640 1 -- 172.21.15.27:0/2632121011 >> 172.21.15.27:0/2632121011 conn(0x7f0a28069c70 msgr2=0x7f0a2806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.548+0000 7f0a27fff640 1 -- 172.21.15.27:0/2632121011 shutdown_connections 2023-12-19T18:21:04.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.548+0000 7f0a27fff640 1 -- 172.21.15.27:0/2632121011 wait complete. 2023-12-19T18:21:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.549+0000 7f0a27fff640 1 Processor -- start 2023-12-19T18:21:04.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.550+0000 7f0a27fff640 1 -- start start 2023-12-19T18:21:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.551+0000 7f0a27fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a280c8f90 0x7f0a28136480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.551+0000 7f0a27fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a280cb300 0x7f0a2815c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.551+0000 7f0a27fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a28161010 0x7f0a280b3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.551+0000 7f0a27fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a28141b30 con 0x7f0a280cb300 2023-12-19T18:21:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.551+0000 7f0a27fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a28140f80 con 0x7f0a280c8f90 2023-12-19T18:21:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.551+0000 7f0a27fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a28141190 con 0x7f0a28161010 2023-12-19T18:21:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.551+0000 7f0a1ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a280cb300 0x7f0a2815c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.551+0000 7f0a1ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a280cb300 0x7f0a2815c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33218/0 (socket says 172.21.15.27:33218) 2023-12-19T18:21:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.551+0000 7f0a1ffff640 1 -- 172.21.15.27:0/3233482736 learned_addr learned my addr 172.21.15.27:0/3233482736 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.551+0000 7f0a26ffd640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a280c8f90 0x7f0a28136480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.551+0000 7f0a277fe640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a28161010 0x7f0a280b3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:04.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.552+0000 7f0a1ffff640 1 -- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a28161010 msgr2=0x7f0a280b3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.552+0000 7f0a1ffff640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a28161010 0x7f0a280b3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.552+0000 7f0a1ffff640 1 -- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a280c8f90 msgr2=0x7f0a28136480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.552+0000 7f0a1ffff640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a280c8f90 0x7f0a28136480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.552+0000 7f0a1ffff640 1 -- 172.21.15.27:0/3233482736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a18009400 con 0x7f0a280cb300 2023-12-19T18:21:04.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.552+0000 7f0a277fe640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a28161010 0x7f0a280b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:21:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.552+0000 7f0a1ffff640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a280cb300 0x7f0a2815c6d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000c400 tx=0x7f0a1000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.552+0000 7f0a24ff9640 1 -- 172.21.15.27:0/3233482736 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a100052d0 con 0x7f0a280cb300 2023-12-19T18:21:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.552+0000 7f0a24ff9640 1 -- 172.21.15.27:0/3233482736 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a10005430 con 0x7f0a280cb300 2023-12-19T18:21:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.552+0000 7f0a24ff9640 1 -- 172.21.15.27:0/3233482736 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10003830 con 0x7f0a280cb300 2023-12-19T18:21:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.553+0000 7f0a27fff640 1 -- 172.21.15.27:0/3233482736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a20009b30 con 0x7f0a280cb300 2023-12-19T18:21:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.553+0000 7f0a27fff640 1 -- 172.21.15.27:0/3233482736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a28141750 con 0x7f0a280cb300 2023-12-19T18:21:04.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.554+0000 7f0a27fff640 1 -- 172.21.15.27:0/3233482736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a28141c70 con 0x7f0a280cb300 2023-12-19T18:21:04.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.555+0000 7f0a24ff9640 1 -- 172.21.15.27:0/3233482736 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0a10003990 con 0x7f0a280cb300 2023-12-19T18:21:04.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.556+0000 7f0a24ff9640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a0806a430 0x7f0a0806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:04.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.556+0000 7f0a24ff9640 1 -- 172.21.15.27:0/3233482736 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f0a10087ad0 con 0x7f0a280cb300 2023-12-19T18:21:04.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.557+0000 7f0a26ffd640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a0806a430 0x7f0a0806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:04.558 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.558+0000 7f0a26ffd640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a0806a430 0x7f0a0806c8f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009c30 tx=0x7f0a18009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:04.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.560+0000 7f0a24ff9640 1 -- 172.21.15.27:0/3233482736 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0a28141c70 con 0x7f0a280cb300 2023-12-19T18:21:04.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.761+0000 7f0a27fff640 1 -- 172.21.15.27:0/3233482736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0a28141c70 con 0x7f0a280cb300 2023-12-19T18:21:04.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.763+0000 7f0a24ff9640 1 -- 172.21.15.27:0/3233482736 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+12394 (secure 0 0 0) 0x7f0a28141c70 con 0x7f0a280cb300 2023-12-19T18:21:04.763 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:04.763 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":113,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:21:03.736657+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":100,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:21:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.765+0000 7f0a27fff640 1 -- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a0806a430 msgr2=0x7f0a0806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.765+0000 7f0a27fff640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a0806a430 0x7f0a0806c8f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f0a18009c30 tx=0x7f0a18009190 comp rx=0 tx=0).stop 2023-12-19T18:21:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.765+0000 7f0a27fff640 1 -- 172.21.15.27:0/3233482736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a280cb300 msgr2=0x7f0a2815c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.765+0000 7f0a27fff640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a280cb300 0x7f0a2815c6d0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000c400 tx=0x7f0a1000c8c0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.766+0000 7f0a27fff640 1 -- 172.21.15.27:0/3233482736 shutdown_connections 2023-12-19T18:21:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.766+0000 7f0a27fff640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a0806a430 0x7f0a0806c8f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.766+0000 7f0a27fff640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a28161010 0x7f0a280b3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.766+0000 7f0a27fff640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a280cb300 0x7f0a2815c6d0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.766+0000 7f0a27fff640 1 --2- 172.21.15.27:0/3233482736 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a280c8f90 0x7f0a28136480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.766+0000 7f0a27fff640 1 -- 172.21.15.27:0/3233482736 >> 172.21.15.27:0/3233482736 conn(0x7f0a28069c70 msgr2=0x7f0a280ad450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.766+0000 7f0a27fff640 1 -- 172.21.15.27:0/3233482736 shutdown_connections 2023-12-19T18:21:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:04.766+0000 7f0a27fff640 1 -- 172.21.15.27:0/3233482736 wait complete. 2023-12-19T18:21:04.787 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '105', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:21:04.787 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:21:05.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.022+0000 7faca1fca640 1 Processor -- start 2023-12-19T18:21:05.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.022+0000 7faca1fca640 1 -- start start 2023-12-19T18:21:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.024+0000 7faca1fca640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fac9c14ab60 0x7fac9c06d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.024+0000 7faca1fca640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c0b2020 con 0x7fac9c0cf580 2023-12-19T18:21:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.024+0000 7faca1fca640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac9c1654c0 con 0x7fac9c06baa0 2023-12-19T18:21:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.024+0000 7faca1fca640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac9c1657a0 con 0x7fac9c14ab60 2023-12-19T18:21:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.024+0000 7faca17c9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fac9c0cf580 0x7fac9c0b2500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40898/0 (socket says 172.21.15.27:40898) 2023-12-19T18:21:05.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.024+0000 7faca17c9640 1 -- 172.21.15.27:0/2874509984 learned_addr learned my addr 172.21.15.27:0/2874509984 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:05.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.025+0000 7faca0fc8640 1 --2- 172.21.15.27:0/2874509984 >> v2:172.21.15.192:3300/0 conn(0x7fac9c14ab60 0x7fac9c06d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.025+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971696510 0 0) 0x7fac9c0b2020 con 0x7fac9c0cf580 2023-12-19T18:21:05.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.025+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac8c003520 con 0x7fac9c0cf580 2023-12-19T18:21:05.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.025+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1696699875 0 0) 0x7fac8c003520 con 0x7fac9c0cf580 2023-12-19T18:21:05.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.025+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 >> v2:172.21.15.192:3300/0 conn(0x7fac9c14ab60 msgr2=0x7fac9c06d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:05.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.025+0000 7fac9b7fe640 1 --2- 172.21.15.27:0/2874509984 >> v2:172.21.15.192:3300/0 conn(0x7fac9c14ab60 0x7fac9c06d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.025+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 >> v1:172.21.15.192:6789/0 conn(0x7fac9c06baa0 legacy=0x7fac9c0cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.025+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c165a80 con 0x7fac9c0cf580 2023-12-19T18:21:05.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.025+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fac940019f0 con 0x7fac9c0cf580 2023-12-19T18:21:05.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.026+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 >> v1:172.21.15.27:6789/0 conn(0x7fac9c0cf580 legacy=0x7fac9c0b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.026+0000 7fac9b7fe640 1 --2- 172.21.15.27:0/2874509984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac8c003970 0x7fac8c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.026+0000 7fac9b7fe640 1 --2- 172.21.15.27:0/2874509984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8c0047c0 0x7fac8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.026+0000 7fac9b7fe640 1 --2- 172.21.15.27:0/2874509984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac8c005180 0x7fac8c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.026+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1657a0 con 0x7fac8c003970 2023-12-19T18:21:05.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.026+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c00c120 con 0x7fac8c005180 2023-12-19T18:21:05.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.026+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac8c00c2e0 con 0x7fac8c0047c0 2023-12-19T18:21:05.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.026+0000 7faca0fc8640 1 --2- 172.21.15.27:0/2874509984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac8c003970 0x7fac8c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.026+0000 7fac9bfff640 1 --2- 172.21.15.27:0/2874509984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8c0047c0 0x7fac8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.026+0000 7faca17c9640 1 --2- 172.21.15.27:0/2874509984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac8c005180 0x7fac8c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.026+0000 7faca0fc8640 1 -- 172.21.15.27:0/2874509984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8c0047c0 msgr2=0x7fac8c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:05.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.026+0000 7faca0fc8640 1 --2- 172.21.15.27:0/2874509984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8c0047c0 0x7fac8c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.027+0000 7faca0fc8640 1 -- 172.21.15.27:0/2874509984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac8c005180 msgr2=0x7fac8c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:05.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.027+0000 7faca0fc8640 1 --2- 172.21.15.27:0/2874509984 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac8c005180 0x7fac8c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.027+0000 7faca17c9640 1 -- 172.21.15.27:0/2874509984 reap_dead start 2023-12-19T18:21:05.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.027+0000 7faca0fc8640 1 -- 172.21.15.27:0/2874509984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac8c003520 con 0x7fac8c003970 2023-12-19T18:21:05.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.028+0000 7faca0fc8640 1 --2- 172.21.15.27:0/2874509984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac8c003970 0x7fac8c003d90 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fac90009ac0 tx=0x7fac8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17731bc32e7b6ab8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:05.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.028+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9002c6b0 con 0x7fac8c003970 2023-12-19T18:21:05.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.028+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9002c810 con 0x7fac8c003970 2023-12-19T18:21:05.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.028+0000 7fac9b7fe640 1 -- 172.21.15.27:0/2874509984 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9002cba0 con 0x7fac8c003970 2023-12-19T18:21:05.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.029+0000 7faca1fca640 1 -- 172.21.15.27:0/2874509984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac8c003970 msgr2=0x7fac8c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.029+0000 7faca1fca640 1 --2- 172.21.15.27:0/2874509984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac8c003970 0x7fac8c003d90 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7fac90009ac0 tx=0x7fac8c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.029+0000 7faca1fca640 1 -- 172.21.15.27:0/2874509984 shutdown_connections 2023-12-19T18:21:05.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.029+0000 7faca1fca640 1 --2- 172.21.15.27:0/2874509984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac8c003970 0x7fac8c003d90 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.029+0000 7faca1fca640 1 -- 172.21.15.27:0/2874509984 >> 172.21.15.27:0/2874509984 conn(0x7fac9c06a060 msgr2=0x7fac9c06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:05.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.029+0000 7faca1fca640 1 -- 172.21.15.27:0/2874509984 shutdown_connections 2023-12-19T18:21:05.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.029+0000 7faca1fca640 1 -- 172.21.15.27:0/2874509984 wait complete. 2023-12-19T18:21:05.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.031+0000 7faca1fca640 1 Processor -- start 2023-12-19T18:21:05.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.031+0000 7faca1fca640 1 -- start start 2023-12-19T18:21:05.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.032+0000 7faca1fca640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8c003970 0x7fac9c160830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.032+0000 7faca1fca640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac9c143ff0 0x7fac9c163060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.032+0000 7faca1fca640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac9c144470 0x7fac9c1635a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.032+0000 7faca1fca640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1458c0 con 0x7fac9c144470 2023-12-19T18:21:05.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.032+0000 7faca1fca640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c1452c0 con 0x7fac9c143ff0 2023-12-19T18:21:05.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.032+0000 7faca1fca640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac9c144a20 con 0x7fac8c003970 2023-12-19T18:21:05.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.033+0000 7faca17c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac9c144470 0x7fac9c1635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.033+0000 7faca17c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac9c144470 0x7fac9c1635a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33228/0 (socket says 172.21.15.27:33228) 2023-12-19T18:21:05.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.033+0000 7faca17c9640 1 -- 172.21.15.27:0/3202063565 learned_addr learned my addr 172.21.15.27:0/3202063565 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:05.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.033+0000 7faca0fc8640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8c003970 0x7fac9c160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.033+0000 7fac9bfff640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac9c143ff0 0x7fac9c163060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.033+0000 7faca17c9640 1 -- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8c003970 msgr2=0x7fac9c160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.033+0000 7faca17c9640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8c003970 0x7fac9c160830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.033+0000 7faca17c9640 1 -- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac9c143ff0 msgr2=0x7fac9c163060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.033+0000 7faca17c9640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac9c143ff0 0x7fac9c163060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.033+0000 7faca17c9640 1 -- 172.21.15.27:0/3202063565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac9c144d00 con 0x7fac9c144470 2023-12-19T18:21:05.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.034+0000 7faca17c9640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac9c144470 0x7fac9c1635a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fac9c070470 tx=0x7fac94007a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:05.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.034+0000 7fac997fa640 1 -- 172.21.15.27:0/3202063565 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9400e900 con 0x7fac9c144470 2023-12-19T18:21:05.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.034+0000 7fac997fa640 1 -- 172.21.15.27:0/3202063565 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac9400ea60 con 0x7fac9c144470 2023-12-19T18:21:05.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.034+0000 7fac997fa640 1 -- 172.21.15.27:0/3202063565 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac9400ee10 con 0x7fac9c144470 2023-12-19T18:21:05.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.034+0000 7faca1fca640 1 -- 172.21.15.27:0/3202063565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac9c144fe0 con 0x7fac9c144470 2023-12-19T18:21:05.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.034+0000 7faca1fca640 1 -- 172.21.15.27:0/3202063565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac9c143850 con 0x7fac9c144470 2023-12-19T18:21:05.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.035+0000 7faca1fca640 1 -- 172.21.15.27:0/3202063565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac9c145a70 con 0x7fac9c144470 2023-12-19T18:21:05.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.036+0000 7fac997fa640 1 -- 172.21.15.27:0/3202063565 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fac9401d460 con 0x7fac9c144470 2023-12-19T18:21:05.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.037+0000 7fac997fa640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fac8006a460 0x7fac8006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.037+0000 7fac997fa640 1 -- 172.21.15.27:0/3202063565 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fac94088750 con 0x7fac9c144470 2023-12-19T18:21:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.038+0000 7faca0fc8640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fac8006a460 0x7fac8006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.039+0000 7faca0fc8640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fac8006a460 0x7fac8006c920 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fac90036000 tx=0x7fac90030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:05.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.042+0000 7fac997fa640 1 -- 172.21.15.27:0/3202063565 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fac9c145a70 con 0x7fac9c144470 2023-12-19T18:21:05.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.232+0000 7faca1fca640 1 -- 172.21.15.27:0/3202063565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac9c04e840 con 0x7fac9c144470 2023-12-19T18:21:05.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.233+0000 7fac997fa640 1 -- 172.21.15.27:0/3202063565 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v113) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac94054120 con 0x7fac9c144470 2023-12-19T18:21:05.233 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:21:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.236+0000 7faca1fca640 1 -- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fac8006a460 msgr2=0x7fac8006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.236+0000 7faca1fca640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fac8006a460 0x7fac8006c920 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fac90036000 tx=0x7fac90030000 comp rx=0 tx=0).stop 2023-12-19T18:21:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.236+0000 7faca1fca640 1 -- 172.21.15.27:0/3202063565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac9c144470 msgr2=0x7fac9c1635a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.236+0000 7faca1fca640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac9c144470 0x7fac9c1635a0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7fac9c070470 tx=0x7fac94007a80 comp rx=0 tx=0).stop 2023-12-19T18:21:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.236+0000 7faca1fca640 1 -- 172.21.15.27:0/3202063565 shutdown_connections 2023-12-19T18:21:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.236+0000 7faca1fca640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fac8006a460 0x7fac8006c920 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.236+0000 7faca1fca640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac9c144470 0x7fac9c1635a0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.236+0000 7faca1fca640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fac9c143ff0 0x7fac9c163060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.236+0000 7faca1fca640 1 --2- 172.21.15.27:0/3202063565 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fac8c003970 0x7fac9c160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.236+0000 7faca1fca640 1 -- 172.21.15.27:0/3202063565 >> 172.21.15.27:0/3202063565 conn(0x7fac9c06a060 msgr2=0x7fac9c0ada50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.236+0000 7faca1fca640 1 -- 172.21.15.27:0/3202063565 shutdown_connections 2023-12-19T18:21:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.236+0000 7faca1fca640 1 -- 172.21.15.27:0/3202063565 wait complete. 2023-12-19T18:21:05.255 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:21:05.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.494+0000 7f9ee7311640 1 Processor -- start 2023-12-19T18:21:05.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.494+0000 7f9ee7311640 1 -- start start 2023-12-19T18:21:05.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.496+0000 7f9ee7311640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9ee0154170 0x7f9ee0070170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.496+0000 7f9ee7311640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9ee015aab0 0x7f9ee0070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.496+0000 7f9ee7311640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ee015b110 con 0x7f9ee0070aa0 2023-12-19T18:21:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.496+0000 7f9ee7311640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee015af20 con 0x7f9ee015aab0 2023-12-19T18:21:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.496+0000 7f9ee7311640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ee0071470 con 0x7f9ee0154170 2023-12-19T18:21:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.496+0000 7f9ee6b10640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9ee015aab0 0x7f9ee0070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.497+0000 7f9ee6b10640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9ee015aab0 0x7f9ee0070f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33240/0 (socket says 172.21.15.27:33240) 2023-12-19T18:21:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.497+0000 7f9ee6b10640 1 -- 172.21.15.27:0/941210489 learned_addr learned my addr 172.21.15.27:0/941210489 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:05.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.497+0000 7f9ee630f640 1 --2- 172.21.15.27:0/941210489 >> v2:172.21.15.192:3300/0 conn(0x7f9ee0154170 0x7f9ee0070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.497+0000 7f9ee6b10640 1 -- 172.21.15.27:0/941210489 >> v2:172.21.15.192:3300/0 conn(0x7f9ee0154170 msgr2=0x7f9ee0070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:05.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.497+0000 7f9ee6b10640 1 --2- 172.21.15.27:0/941210489 >> v2:172.21.15.192:3300/0 conn(0x7f9ee0154170 0x7f9ee0070170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.497+0000 7f9ee6b10640 1 -- 172.21.15.27:0/941210489 >> v1:172.21.15.192:6789/0 conn(0x7f9ee0070aa0 legacy=0x7f9ee015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:05.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.497+0000 7f9ee6b10640 1 -- 172.21.15.27:0/941210489 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ee006fce0 con 0x7f9ee015aab0 2023-12-19T18:21:05.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.498+0000 7f9ee6b10640 1 --2- 172.21.15.27:0/941210489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee015aab0 0x7f9ee0070f30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9ee006d770 tx=0x7f9ed802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a77153b36730ab server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:05.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.499+0000 7f9ee530d640 1 -- 172.21.15.27:0/941210489 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed802c9f0 con 0x7f9ee015aab0 2023-12-19T18:21:05.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.499+0000 7f9ee530d640 1 -- 172.21.15.27:0/941210489 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ed802cb50 con 0x7f9ee015aab0 2023-12-19T18:21:05.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.499+0000 7f9ee530d640 1 -- 172.21.15.27:0/941210489 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ed80343f0 con 0x7f9ee015aab0 2023-12-19T18:21:05.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.499+0000 7f9ee7311640 1 -- 172.21.15.27:0/941210489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee015aab0 msgr2=0x7f9ee0070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.499+0000 7f9ee7311640 1 --2- 172.21.15.27:0/941210489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee015aab0 0x7f9ee0070f30 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f9ee006d770 tx=0x7f9ed802be50 comp rx=0 tx=0).stop 2023-12-19T18:21:05.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.500+0000 7f9ee7311640 1 -- 172.21.15.27:0/941210489 shutdown_connections 2023-12-19T18:21:05.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.500+0000 7f9ee7311640 1 --2- 172.21.15.27:0/941210489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee015aab0 0x7f9ee0070f30 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.500+0000 7f9ee7311640 1 --2- 172.21.15.27:0/941210489 >> v2:172.21.15.192:3300/0 conn(0x7f9ee0154170 0x7f9ee0070170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.500+0000 7f9ee7311640 1 -- 172.21.15.27:0/941210489 >> 172.21.15.27:0/941210489 conn(0x7f9ee006a7d0 msgr2=0x7f9ee00ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:05.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.500+0000 7f9ee7311640 1 -- 172.21.15.27:0/941210489 shutdown_connections 2023-12-19T18:21:05.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.500+0000 7f9ee7311640 1 -- 172.21.15.27:0/941210489 wait complete. 2023-12-19T18:21:05.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.502+0000 7f9ee7311640 1 Processor -- start 2023-12-19T18:21:05.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.502+0000 7f9ee7311640 1 -- start start 2023-12-19T18:21:05.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.503+0000 7f9ee7311640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee0070aa0 0x7f9ee00cd480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.503+0000 7f9ee7311640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ee0154170 0x7f9ee00cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.503+0000 7f9ee7311640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ee015aab0 0x7f9ee01f27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.503+0000 7f9ee7311640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee01f3510 con 0x7f9ee0070aa0 2023-12-19T18:21:05.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.503+0000 7f9ee7311640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee00b28c0 con 0x7f9ee0154170 2023-12-19T18:21:05.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.503+0000 7f9ee7311640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ee00b2ba0 con 0x7f9ee015aab0 2023-12-19T18:21:05.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.503+0000 7f9ee630f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee0070aa0 0x7f9ee00cd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.503+0000 7f9ee630f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee0070aa0 0x7f9ee00cd480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33252/0 (socket says 172.21.15.27:33252) 2023-12-19T18:21:05.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.503+0000 7f9ee630f640 1 -- 172.21.15.27:0/82859496 learned_addr learned my addr 172.21.15.27:0/82859496 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:05.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.503+0000 7f9ee5b0e640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ee0154170 0x7f9ee00cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.503+0000 7f9ee6b10640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ee015aab0 0x7f9ee01f27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.504+0000 7f9ee630f640 1 -- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ee015aab0 msgr2=0x7f9ee01f27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.504+0000 7f9ee630f640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ee015aab0 0x7f9ee01f27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.504+0000 7f9ee630f640 1 -- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ee0154170 msgr2=0x7f9ee00cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.504+0000 7f9ee630f640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ee0154170 0x7f9ee00cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.504+0000 7f9ee630f640 1 -- 172.21.15.27:0/82859496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ed8009b30 con 0x7f9ee0070aa0 2023-12-19T18:21:05.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.504+0000 7f9ee6b10640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ee015aab0 0x7f9ee01f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:05.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.504+0000 7f9ee630f640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee0070aa0 0x7f9ee00cd480 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00d800 tx=0x7f9edc00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:05.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.504+0000 7f9ed6ffd640 1 -- 172.21.15.27:0/82859496 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00b6c0 con 0x7f9ee0070aa0 2023-12-19T18:21:05.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.504+0000 7f9ed6ffd640 1 -- 172.21.15.27:0/82859496 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9edc00b820 con 0x7f9ee0070aa0 2023-12-19T18:21:05.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.504+0000 7f9ee7311640 1 -- 172.21.15.27:0/82859496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ee00b2e80 con 0x7f9ee0070aa0 2023-12-19T18:21:05.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.504+0000 7f9ed6ffd640 1 -- 172.21.15.27:0/82859496 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9edc00bbb0 con 0x7f9ee0070aa0 2023-12-19T18:21:05.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.505+0000 7f9ee7311640 1 -- 172.21.15.27:0/82859496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ee01f36c0 con 0x7f9ee0070aa0 2023-12-19T18:21:05.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.506+0000 7f9ee7311640 1 -- 172.21.15.27:0/82859496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ee004e840 con 0x7f9ee0070aa0 2023-12-19T18:21:05.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.507+0000 7f9ed6ffd640 1 -- 172.21.15.27:0/82859496 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9edc00bd10 con 0x7f9ee0070aa0 2023-12-19T18:21:05.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.508+0000 7f9ed6ffd640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ec406a430 0x7f9ec406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.508+0000 7f9ed6ffd640 1 -- 172.21.15.27:0/82859496 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9edc088bb0 con 0x7f9ee0070aa0 2023-12-19T18:21:05.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.508+0000 7f9ee5b0e640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ec406a430 0x7f9ec406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.509+0000 7f9ee5b0e640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ec406a430 0x7f9ec406c8f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0006fd0 tx=0x7f9ed0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:05.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.512+0000 7f9ed6ffd640 1 -- 172.21.15.27:0/82859496 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9edc054450 con 0x7f9ee0070aa0 2023-12-19T18:21:05.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.708+0000 7f9ee7311640 1 -- 172.21.15.27:0/82859496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f9ee0152660 con 0x7f9ee0070aa0 2023-12-19T18:21:05.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.709+0000 7f9ed6ffd640 1 -- 172.21.15.27:0/82859496 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v113) v1 ==== 84+0+65 (secure 0 0 0) 0x7f9edc057a70 con 0x7f9ee0070aa0 2023-12-19T18:21:05.709 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:05.709 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-12-19T18:21:05.710 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:05.710 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-12-19T18:21:05.710 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-12-19T18:21:05.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:05.710 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-12-19T18:21:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.711+0000 7f9ee7311640 1 -- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ec406a430 msgr2=0x7f9ec406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.711+0000 7f9ee7311640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ec406a430 0x7f9ec406c8f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9ed0006fd0 tx=0x7f9ed0008040 comp rx=0 tx=0).stop 2023-12-19T18:21:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.711+0000 7f9ee7311640 1 -- 172.21.15.27:0/82859496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee0070aa0 msgr2=0x7f9ee00cd480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.711+0000 7f9ee7311640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee0070aa0 0x7f9ee00cd480 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f9edc00d800 tx=0x7f9edc00dcc0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.712+0000 7f9ee7311640 1 -- 172.21.15.27:0/82859496 shutdown_connections 2023-12-19T18:21:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.712+0000 7f9ee7311640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ec406a430 0x7f9ec406c8f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.712+0000 7f9ee7311640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9ee015aab0 0x7f9ee01f27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.712+0000 7f9ee7311640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9ee0154170 0x7f9ee00cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.712+0000 7f9ee7311640 1 --2- 172.21.15.27:0/82859496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ee0070aa0 0x7f9ee00cd480 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.712+0000 7f9ee7311640 1 -- 172.21.15.27:0/82859496 >> 172.21.15.27:0/82859496 conn(0x7f9ee006a7d0 msgr2=0x7f9ee006bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.712+0000 7f9ee7311640 1 -- 172.21.15.27:0/82859496 shutdown_connections 2023-12-19T18:21:05.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.712+0000 7f9ee7311640 1 -- 172.21.15.27:0/82859496 wait complete. 2023-12-19T18:21:05.734 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:21:05.734 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:21:05.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.974+0000 7f3ccddd6640 1 Processor -- start 2023-12-19T18:21:05.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.974+0000 7f3ccddd6640 1 -- start start 2023-12-19T18:21:05.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.976+0000 7f3ccddd6640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3cc80cdb30 0x7f3cc80cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.976+0000 7f3ccddd6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cc80d17a0 0x7f3cc80d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.976+0000 7f3ccddd6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cc8161250 con 0x7f3cc815cfc0 2023-12-19T18:21:05.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.976+0000 7f3ccddd6640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc80ce500 con 0x7f3cc80cdb30 2023-12-19T18:21:05.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.976+0000 7f3ccddd6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc8161a50 con 0x7f3cc80d17a0 2023-12-19T18:21:05.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.976+0000 7f3cccdd4640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3cc815cfc0 0x7f3cc8160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40906/0 (socket says 172.21.15.27:40906) 2023-12-19T18:21:05.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.976+0000 7f3cccdd4640 1 -- 172.21.15.27:0/184599893 learned_addr learned my addr 172.21.15.27:0/184599893 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:05.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.976+0000 7f3ccd5d5640 1 --2- 172.21.15.27:0/184599893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cc80d17a0 0x7f3cc80d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.976+0000 7f3cc7fff640 1 --2- 172.21.15.27:0/184599893 >> v2:172.21.15.192:3300/0 conn(0x7f3cc80cdb30 0x7f3cc80cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.976+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469088163 0 0) 0x7f3cbc001110 con 0x7f3cc815cfc0 2023-12-19T18:21:05.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cb8003520 con 0x7f3cc815cfc0 2023-12-19T18:21:05.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1635054963 0 0) 0x7f3cb8003520 con 0x7f3cc815cfc0 2023-12-19T18:21:05.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cc80d17a0 msgr2=0x7f3cc80d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:05.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 --2- 172.21.15.27:0/184599893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cc80d17a0 0x7f3cc80d1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 >> v2:172.21.15.192:3300/0 conn(0x7f3cc80cdb30 msgr2=0x7f3cc80cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:05.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 --2- 172.21.15.27:0/184599893 >> v2:172.21.15.192:3300/0 conn(0x7f3cc80cdb30 0x7f3cc80cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc8161490 con 0x7f3cc815cfc0 2023-12-19T18:21:05.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f3cbc001800 con 0x7f3cc815cfc0 2023-12-19T18:21:05.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc7fff640 1 --2- 172.21.15.27:0/184599893 >> v2:172.21.15.192:3300/0 conn(0x7f3cc80cdb30 0x7f3cc80cdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:21:05.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 >> v1:172.21.15.27:6789/0 conn(0x7f3cc815cfc0 legacy=0x7f3cc8160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 --2- 172.21.15.27:0/184599893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb8003970 0x7f3cb8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 --2- 172.21.15.27:0/184599893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cc80d17a0 0x7f3cb80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 --2- 172.21.15.27:0/184599893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb8004d00 0x7f3cb8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80ce500 con 0x7f3cb8004d00 2023-12-19T18:21:05.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc8161a50 con 0x7f3cb8003970 2023-12-19T18:21:05.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb800ba60 con 0x7f3cc80d17a0 2023-12-19T18:21:05.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.977+0000 7f3ccd5d5640 1 --2- 172.21.15.27:0/184599893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb8004d00 0x7f3cb8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.978+0000 7f3cccdd4640 1 --2- 172.21.15.27:0/184599893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb8003970 0x7f3cb8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.978+0000 7f3cc7fff640 1 --2- 172.21.15.27:0/184599893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cc80d17a0 0x7f3cb80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.978+0000 7f3ccd5d5640 1 -- 172.21.15.27:0/184599893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cc80d17a0 msgr2=0x7f3cb80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:05.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.978+0000 7f3ccd5d5640 1 --2- 172.21.15.27:0/184599893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cc80d17a0 0x7f3cb80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.978+0000 7f3ccd5d5640 1 -- 172.21.15.27:0/184599893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb8003970 msgr2=0x7f3cb8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:05.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.978+0000 7f3ccd5d5640 1 --2- 172.21.15.27:0/184599893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cb8003970 0x7f3cb8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.978+0000 7f3ccd5d5640 1 -- 172.21.15.27:0/184599893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb8003520 con 0x7f3cb8004d00 2023-12-19T18:21:05.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.978+0000 7f3cc7fff640 1 --2- 172.21.15.27:0/184599893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cc80d17a0 0x7f3cb80047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:05.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.979+0000 7f3ccd5d5640 1 --2- 172.21.15.27:0/184599893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb8004d00 0x7f3cb8005180 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009fc0 tx=0x7f3cb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb636d3b8ba5d31a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:05.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.979+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc002cbf0 con 0x7f3cb8004d00 2023-12-19T18:21:05.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.979+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc000f040 con 0x7f3cb8004d00 2023-12-19T18:21:05.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.979+0000 7f3cc77fe640 1 -- 172.21.15.27:0/184599893 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0034440 con 0x7f3cb8004d00 2023-12-19T18:21:05.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.980+0000 7f3ccddd6640 1 -- 172.21.15.27:0/184599893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb8004d00 msgr2=0x7f3cb8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.980+0000 7f3ccddd6640 1 --2- 172.21.15.27:0/184599893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb8004d00 0x7f3cb8005180 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f3cc0009fc0 tx=0x7f3cb80057f0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.980+0000 7f3ccd5d5640 1 -- 172.21.15.27:0/184599893 reap_dead start 2023-12-19T18:21:05.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.980+0000 7f3ccddd6640 1 -- 172.21.15.27:0/184599893 shutdown_connections 2023-12-19T18:21:05.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.980+0000 7f3ccddd6640 1 -- 172.21.15.27:0/184599893 >> 172.21.15.27:0/184599893 conn(0x7f3cc8072e20 msgr2=0x7f3cc8073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:05.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.980+0000 7f3ccddd6640 1 -- 172.21.15.27:0/184599893 shutdown_connections 2023-12-19T18:21:05.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.980+0000 7f3ccddd6640 1 -- 172.21.15.27:0/184599893 wait complete. 2023-12-19T18:21:05.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.982+0000 7f3ccddd6640 1 Processor -- start 2023-12-19T18:21:05.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.982+0000 7f3ccddd6640 1 -- start start 2023-12-19T18:21:05.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.983+0000 7f3ccddd6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb8004d00 0x7f3cc815b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.983+0000 7f3ccddd6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc8153ab0 0x7f3cc8153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.983+0000 7f3ccddd6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cc80cbee0 0x7f3cc80cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.983+0000 7f3ccddd6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc8154470 con 0x7f3cb8004d00 2023-12-19T18:21:05.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.983+0000 7f3ccddd6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc8143be0 con 0x7f3cc8153ab0 2023-12-19T18:21:05.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.983+0000 7f3ccddd6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80cc8a0 con 0x7f3cc80cbee0 2023-12-19T18:21:05.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.983+0000 7f3cccdd4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb8004d00 0x7f3cc815b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.983+0000 7f3cccdd4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb8004d00 0x7f3cc815b5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33276/0 (socket says 172.21.15.27:33276) 2023-12-19T18:21:05.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.983+0000 7f3cccdd4640 1 -- 172.21.15.27:0/1824087752 learned_addr learned my addr 172.21.15.27:0/1824087752 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:05.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.984+0000 7f3cccdd4640 1 -- 172.21.15.27:0/1824087752 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cc80cbee0 msgr2=0x7f3cc80cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.984+0000 7f3cccdd4640 1 --2- 172.21.15.27:0/1824087752 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cc80cbee0 0x7f3cc80cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.984+0000 7f3cccdd4640 1 -- 172.21.15.27:0/1824087752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc8153ab0 msgr2=0x7f3cc8153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:05.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.984+0000 7f3cccdd4640 1 --2- 172.21.15.27:0/1824087752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc8153ab0 0x7f3cc8153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:05.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.984+0000 7f3cccdd4640 1 -- 172.21.15.27:0/1824087752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc815dfe0 con 0x7f3cb8004d00 2023-12-19T18:21:05.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.984+0000 7f3cccdd4640 1 --2- 172.21.15.27:0/1824087752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb8004d00 0x7f3cc815b5a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc001c70 tx=0x7f3cbc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:05.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.984+0000 7f3cc57fa640 1 -- 172.21.15.27:0/1824087752 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc00ee70 con 0x7f3cb8004d00 2023-12-19T18:21:05.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.984+0000 7f3cc57fa640 1 -- 172.21.15.27:0/1824087752 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc00c870 con 0x7f3cb8004d00 2023-12-19T18:21:05.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.984+0000 7f3cc57fa640 1 -- 172.21.15.27:0/1824087752 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc00cc40 con 0x7f3cb8004d00 2023-12-19T18:21:05.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.985+0000 7f3ccddd6640 1 -- 172.21.15.27:0/1824087752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc815e2c0 con 0x7f3cb8004d00 2023-12-19T18:21:05.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.985+0000 7f3ccddd6640 1 -- 172.21.15.27:0/1824087752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc815e880 con 0x7f3cb8004d00 2023-12-19T18:21:05.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.986+0000 7f3ccddd6640 1 -- 172.21.15.27:0/1824087752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc804e840 con 0x7f3cb8004d00 2023-12-19T18:21:05.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.987+0000 7f3cc57fa640 1 -- 172.21.15.27:0/1824087752 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3cbc01d460 con 0x7f3cb8004d00 2023-12-19T18:21:05.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.988+0000 7f3cc57fa640 1 --2- 172.21.15.27:0/1824087752 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3cac06a460 0x7f3cac06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:05.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.988+0000 7f3cc57fa640 1 -- 172.21.15.27:0/1824087752 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3cbc088400 con 0x7f3cb8004d00 2023-12-19T18:21:05.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.988+0000 7f3cc7fff640 1 --2- 172.21.15.27:0/1824087752 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3cac06a460 0x7f3cac06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:05.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.990+0000 7f3cc7fff640 1 --2- 172.21.15.27:0/1824087752 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3cac06a460 0x7f3cac06c920 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4006f00 tx=0x7f3cb4006df0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:05.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:05.993+0000 7f3cc57fa640 1 -- 172.21.15.27:0/1824087752 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3cbc053d20 con 0x7f3cb8004d00 2023-12-19T18:21:06.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.192+0000 7f3ccddd6640 1 -- 172.21.15.27:0/1824087752 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3cc81611d0 con 0x7f3cb8004d00 2023-12-19T18:21:06.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.760+0000 7f3cc57fa640 1 -- 172.21.15.27:0/1824087752 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v114) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3cbc057340 con 0x7f3cb8004d00 2023-12-19T18:21:06.761 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-12-19T18:21:06.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.766+0000 7f3ccddd6640 1 -- 172.21.15.27:0/1824087752 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3cac06a460 msgr2=0x7f3cac06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:06.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.766+0000 7f3ccddd6640 1 --2- 172.21.15.27:0/1824087752 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3cac06a460 0x7f3cac06c920 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4006f00 tx=0x7f3cb4006df0 comp rx=0 tx=0).stop 2023-12-19T18:21:06.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.766+0000 7f3ccddd6640 1 -- 172.21.15.27:0/1824087752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb8004d00 msgr2=0x7f3cc815b5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:06.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.766+0000 7f3ccddd6640 1 --2- 172.21.15.27:0/1824087752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb8004d00 0x7f3cc815b5a0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc001c70 tx=0x7f3cbc00ec80 comp rx=0 tx=0).stop 2023-12-19T18:21:06.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.766+0000 7f3ccddd6640 1 -- 172.21.15.27:0/1824087752 shutdown_connections 2023-12-19T18:21:06.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.766+0000 7f3ccddd6640 1 --2- 172.21.15.27:0/1824087752 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3cac06a460 0x7f3cac06c920 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.766+0000 7f3ccddd6640 1 --2- 172.21.15.27:0/1824087752 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3cc80cbee0 0x7f3cc80cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.766+0000 7f3ccddd6640 1 --2- 172.21.15.27:0/1824087752 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3cc8153ab0 0x7f3cc8153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.766+0000 7f3ccddd6640 1 --2- 172.21.15.27:0/1824087752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cb8004d00 0x7f3cc815b5a0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.766+0000 7f3ccddd6640 1 -- 172.21.15.27:0/1824087752 >> 172.21.15.27:0/1824087752 conn(0x7f3cc8072e20 msgr2=0x7f3cc80d3460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.767+0000 7f3ccddd6640 1 -- 172.21.15.27:0/1824087752 shutdown_connections 2023-12-19T18:21:06.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:06.767+0000 7f3ccddd6640 1 -- 172.21.15.27:0/1824087752 wait complete. 2023-12-19T18:21:06.789 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:21:07.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.026+0000 7effb7789640 1 Processor -- start 2023-12-19T18:21:07.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.026+0000 7effb7789640 1 -- start start 2023-12-19T18:21:07.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.028+0000 7effb7789640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c9750 0x7effb015e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:07.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.028+0000 7effb7789640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7effb00ccfa0 0x7effb00c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:07.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.028+0000 7effb7789640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7effb0161010 0x7effb01614a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:07.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.028+0000 7effb7789640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb00ce0b0 con 0x7effb00ccfa0 2023-12-19T18:21:07.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.028+0000 7effb7789640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb00ce920 con 0x7effb0161010 2023-12-19T18:21:07.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.028+0000 7effb7789640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb015e9e0 con 0x7effb00c9750 2023-12-19T18:21:07.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.028+0000 7effb4d24640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7effb00ccfa0 0x7effb00c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:07.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.028+0000 7effb4d24640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7effb00ccfa0 0x7effb00c9b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33280/0 (socket says 172.21.15.27:33280) 2023-12-19T18:21:07.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.028+0000 7effb4d24640 1 -- 172.21.15.27:0/414432150 learned_addr learned my addr 172.21.15.27:0/414432150 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:07.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.028+0000 7effb4d24640 1 -- 172.21.15.27:0/414432150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c9750 msgr2=0x7effb015e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:07.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.028+0000 7effb4d24640 1 --2- 172.21.15.27:0/414432150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c9750 0x7effb015e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:07.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.028+0000 7effb5d26640 1 --2- 172.21.15.27:0/414432150 >> v2:172.21.15.192:3300/0 conn(0x7effb0161010 0x7effb01614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:07.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.029+0000 7effb4d24640 1 -- 172.21.15.27:0/414432150 >> v2:172.21.15.192:3300/0 conn(0x7effb0161010 msgr2=0x7effb01614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.029+0000 7effb4d24640 1 --2- 172.21.15.27:0/414432150 >> v2:172.21.15.192:3300/0 conn(0x7effb0161010 0x7effb01614a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.029+0000 7effb4d24640 1 -- 172.21.15.27:0/414432150 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb00ca070 con 0x7effb00ccfa0 2023-12-19T18:21:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.030+0000 7effb4d24640 1 --2- 172.21.15.27:0/414432150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effb00ccfa0 0x7effb00c9b30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7effb00ce020 tx=0x7effa002b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de0ac3adb1420e8c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.030+0000 7effa7fff640 1 -- 172.21.15.27:0/414432150 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0031880 con 0x7effb00ccfa0 2023-12-19T18:21:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.030+0000 7effa7fff640 1 -- 172.21.15.27:0/414432150 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa00319e0 con 0x7effb00ccfa0 2023-12-19T18:21:07.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.030+0000 7effa7fff640 1 -- 172.21.15.27:0/414432150 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa0031d70 con 0x7effb00ccfa0 2023-12-19T18:21:07.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.031+0000 7effb7789640 1 -- 172.21.15.27:0/414432150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effb00ccfa0 msgr2=0x7effb00c9b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:07.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.031+0000 7effb7789640 1 --2- 172.21.15.27:0/414432150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effb00ccfa0 0x7effb00c9b30 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7effb00ce020 tx=0x7effa002b920 comp rx=0 tx=0).stop 2023-12-19T18:21:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.031+0000 7effb7789640 1 -- 172.21.15.27:0/414432150 shutdown_connections 2023-12-19T18:21:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.031+0000 7effb7789640 1 --2- 172.21.15.27:0/414432150 >> v2:172.21.15.192:3300/0 conn(0x7effb0161010 0x7effb01614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.031+0000 7effb7789640 1 --2- 172.21.15.27:0/414432150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effb00ccfa0 0x7effb00c9b30 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.031+0000 7effb7789640 1 --2- 172.21.15.27:0/414432150 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c9750 0x7effb015e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.031+0000 7effb7789640 1 -- 172.21.15.27:0/414432150 >> 172.21.15.27:0/414432150 conn(0x7effb00cebd0 msgr2=0x7effb00cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.031+0000 7effb7789640 1 -- 172.21.15.27:0/414432150 shutdown_connections 2023-12-19T18:21:07.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.031+0000 7effb7789640 1 -- 172.21.15.27:0/414432150 wait complete. 2023-12-19T18:21:07.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.033+0000 7effb7789640 1 Processor -- start 2023-12-19T18:21:07.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.033+0000 7effb7789640 1 -- start start 2023-12-19T18:21:07.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.034+0000 7effb7789640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c9750 0x7effb00c5ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:07.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.034+0000 7effb7789640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effb00ccfa0 0x7effb00b0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:07.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.034+0000 7effb7789640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effb0161010 0x7effb014fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:07.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.034+0000 7effb7789640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb00c8b80 con 0x7effb0161010 2023-12-19T18:21:07.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.034+0000 7effb7789640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb01504b0 con 0x7effb00ccfa0 2023-12-19T18:21:07.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.034+0000 7effb7789640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb014ff40 con 0x7effb00c9750 2023-12-19T18:21:07.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.034+0000 7effb5d26640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effb0161010 0x7effb014fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:07.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.034+0000 7effb5d26640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effb0161010 0x7effb014fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33282/0 (socket says 172.21.15.27:33282) 2023-12-19T18:21:07.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.034+0000 7effb5d26640 1 -- 172.21.15.27:0/3115362538 learned_addr learned my addr 172.21.15.27:0/3115362538 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:07.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.035+0000 7effb4d24640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effb00ccfa0 0x7effb00b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:07.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.035+0000 7effb5525640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c9750 0x7effb00c5ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:07.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.035+0000 7effb5d26640 1 -- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c9750 msgr2=0x7effb00c5ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:07.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.035+0000 7effb5d26640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c9750 0x7effb00c5ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:07.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.035+0000 7effb5d26640 1 -- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effb00ccfa0 msgr2=0x7effb00b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:07.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.035+0000 7effb5d26640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effb00ccfa0 0x7effb00b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:07.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.035+0000 7effb5d26640 1 -- 172.21.15.27:0/3115362538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa00094d0 con 0x7effb0161010 2023-12-19T18:21:07.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.035+0000 7effb5525640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c9750 0x7effb00c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:07.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.035+0000 7effb4d24640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effb00ccfa0 0x7effb00b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:07.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.035+0000 7effb5d26640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effb0161010 0x7effb014fa00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7effa800cc60 tx=0x7effa800a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:07.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.036+0000 7effa5ffb640 1 -- 172.21.15.27:0/3115362538 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa800ae80 con 0x7effb0161010 2023-12-19T18:21:07.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.036+0000 7effa5ffb640 1 -- 172.21.15.27:0/3115362538 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa800ce70 con 0x7effb0161010 2023-12-19T18:21:07.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.036+0000 7effb7789640 1 -- 172.21.15.27:0/3115362538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb00c62f0 con 0x7effb0161010 2023-12-19T18:21:07.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.036+0000 7effa5ffb640 1 -- 172.21.15.27:0/3115362538 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8003b90 con 0x7effb0161010 2023-12-19T18:21:07.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.036+0000 7effb7789640 1 -- 172.21.15.27:0/3115362538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb00c68b0 con 0x7effb0161010 2023-12-19T18:21:07.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.037+0000 7effb7789640 1 -- 172.21.15.27:0/3115362538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb0068e90 con 0x7effb0161010 2023-12-19T18:21:07.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.038+0000 7effa5ffb640 1 -- 172.21.15.27:0/3115362538 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7effa800f3f0 con 0x7effb0161010 2023-12-19T18:21:07.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.039+0000 7effa5ffb640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff9406a460 0x7eff9406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:07.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.039+0000 7effa5ffb640 1 -- 172.21.15.27:0/3115362538 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5227+0+0 (secure 0 0 0) 0x7effa8088a10 con 0x7effb0161010 2023-12-19T18:21:07.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.040+0000 7effb5525640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff9406a460 0x7eff9406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:07.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.041+0000 7effb5525640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff9406a460 0x7eff9406c920 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7effac006fd0 tx=0x7effac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:07.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.045+0000 7effa5ffb640 1 -- 172.21.15.27:0/3115362538 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7effa80541e0 con 0x7effb0161010 2023-12-19T18:21:07.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.244+0000 7effb7789640 1 -- 172.21.15.27:0/3115362538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7effb00cc870 con 0x7effb0161010 2023-12-19T18:21:07.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.780+0000 7effa5ffb640 1 -- 172.21.15.27:0/3115362538 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v115) v1 ==== 170+0+0 (secure 0 0 0) 0x7effa801d070 con 0x7effb0161010 2023-12-19T18:21:07.784 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-12-19T18:21:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.789+0000 7effb7789640 1 -- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff9406a460 msgr2=0x7eff9406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.789+0000 7effb7789640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff9406a460 0x7eff9406c920 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7effac006fd0 tx=0x7effac008040 comp rx=0 tx=0).stop 2023-12-19T18:21:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.789+0000 7effb7789640 1 -- 172.21.15.27:0/3115362538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effb0161010 msgr2=0x7effb014fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:07.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.789+0000 7effb7789640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effb0161010 0x7effb014fa00 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7effa800cc60 tx=0x7effa800a5c0 comp rx=0 tx=0).stop 2023-12-19T18:21:07.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.793+0000 7effb7789640 1 -- 172.21.15.27:0/3115362538 shutdown_connections 2023-12-19T18:21:07.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.793+0000 7effb7789640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7eff9406a460 0x7eff9406c920 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:07.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.793+0000 7effb7789640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7effb0161010 0x7effb014fa00 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:07.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.793+0000 7effb7789640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7effb00ccfa0 0x7effb00b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:07.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.793+0000 7effb7789640 1 --2- 172.21.15.27:0/3115362538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7effb00c9750 0x7effb00c5ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:07.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.793+0000 7effb7789640 1 -- 172.21.15.27:0/3115362538 >> 172.21.15.27:0/3115362538 conn(0x7effb00cebd0 msgr2=0x7effb0070160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:07.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.795+0000 7effb7789640 1 -- 172.21.15.27:0/3115362538 shutdown_connections 2023-12-19T18:21:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:07.796+0000 7effb7789640 1 -- 172.21.15.27:0/3115362538 wait complete. 2023-12-19T18:21:07.823 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:21:08.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.026+0000 7f71817b9640 1 Processor -- start 2023-12-19T18:21:08.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.026+0000 7f71817b9640 1 -- start start 2023-12-19T18:21:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.028+0000 7f71817b9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f717c15a0b0 0x7f717c0cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.028+0000 7f71817b9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f717c0ce210 0x7f717c0cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.028+0000 7f71817b9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f717c0c9880 con 0x7f717c06cee0 2023-12-19T18:21:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.028+0000 7f71817b9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f717c0c9de0 con 0x7f717c0ce210 2023-12-19T18:21:08.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.028+0000 7f71817b9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0c9b20 con 0x7f717c15a0b0 2023-12-19T18:21:08.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.029+0000 7f7173fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f717c0ce210 0x7f717c0cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:08.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.029+0000 7f7173fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f717c0ce210 0x7f717c0cd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:40970/0 (socket says 172.21.15.27:40970) 2023-12-19T18:21:08.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.029+0000 7f717affd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f717c15a0b0 0x7f717c0cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:08.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.029+0000 7f7173fff640 1 -- 172.21.15.27:0/1405229169 learned_addr learned my addr 172.21.15.27:0/1405229169 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.030+0000 7f717affd640 1 -- 172.21.15.27:0/1405229169 >> v2:172.21.15.192:3300/0 conn(0x7f717c0ce210 msgr2=0x7f717c0cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.030+0000 7f717affd640 1 --2- 172.21.15.27:0/1405229169 >> v2:172.21.15.192:3300/0 conn(0x7f717c0ce210 0x7f717c0cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.030+0000 7f717affd640 1 -- 172.21.15.27:0/1405229169 >> v1:172.21.15.192:6789/0 conn(0x7f717c06cee0 legacy=0x7f717c070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.030+0000 7f717affd640 1 -- 172.21.15.27:0/1405229169 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f717c0cc920 con 0x7f717c15a0b0 2023-12-19T18:21:08.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.030+0000 7f7173fff640 1 --2- 172.21.15.27:0/1405229169 >> v2:172.21.15.192:3300/0 conn(0x7f717c0ce210 0x7f717c0cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:21:08.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.031+0000 7f717affd640 1 --2- 172.21.15.27:0/1405229169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f717c15a0b0 0x7f717c0cdc80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f716c0097a0 tx=0x7f716c02dac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91c53fbe92a549a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:08.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.032+0000 7f717a7fc640 1 -- 172.21.15.27:0/1405229169 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c02f880 con 0x7f717c15a0b0 2023-12-19T18:21:08.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.032+0000 7f717a7fc640 1 -- 172.21.15.27:0/1405229169 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c02f9e0 con 0x7f717c15a0b0 2023-12-19T18:21:08.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.032+0000 7f717a7fc640 1 -- 172.21.15.27:0/1405229169 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c02fd70 con 0x7f717c15a0b0 2023-12-19T18:21:08.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.033+0000 7f71817b9640 1 -- 172.21.15.27:0/1405229169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f717c15a0b0 msgr2=0x7f717c0cdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:08.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.033+0000 7f71817b9640 1 --2- 172.21.15.27:0/1405229169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f717c15a0b0 0x7f717c0cdc80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f716c0097a0 tx=0x7f716c02dac0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.033+0000 7f71817b9640 1 -- 172.21.15.27:0/1405229169 shutdown_connections 2023-12-19T18:21:08.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.033+0000 7f71817b9640 1 --2- 172.21.15.27:0/1405229169 >> v2:172.21.15.192:3300/0 conn(0x7f717c0ce210 0x7f717c0cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.033+0000 7f71817b9640 1 --2- 172.21.15.27:0/1405229169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f717c15a0b0 0x7f717c0cdc80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.033+0000 7f71817b9640 1 -- 172.21.15.27:0/1405229169 >> 172.21.15.27:0/1405229169 conn(0x7f717c0cebd0 msgr2=0x7f717c0cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:08.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.033+0000 7f71817b9640 1 -- 172.21.15.27:0/1405229169 shutdown_connections 2023-12-19T18:21:08.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.033+0000 7f71817b9640 1 -- 172.21.15.27:0/1405229169 wait complete. 2023-12-19T18:21:08.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.035+0000 7f71817b9640 1 Processor -- start 2023-12-19T18:21:08.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.035+0000 7f71817b9640 1 -- start start 2023-12-19T18:21:08.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.036+0000 7f71817b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717c06cee0 0x7f717c1583b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:08.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.036+0000 7f71817b9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f717c0ce210 0x7f717c0b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:08.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.036+0000 7f71817b9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f717c15a0b0 0x7f717c0b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:08.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.036+0000 7f71817b9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c1f2b90 con 0x7f717c06cee0 2023-12-19T18:21:08.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.036+0000 7f71817b9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0ac8d0 con 0x7f717c15a0b0 2023-12-19T18:21:08.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.036+0000 7f71817b9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f717c0b2400 con 0x7f717c0ce210 2023-12-19T18:21:08.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.036+0000 7f717affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717c06cee0 0x7f717c1583b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:08.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.036+0000 7f717affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717c06cee0 0x7f717c1583b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33296/0 (socket says 172.21.15.27:33296) 2023-12-19T18:21:08.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.036+0000 7f717affd640 1 -- 172.21.15.27:0/1223545139 learned_addr learned my addr 172.21.15.27:0/1223545139 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:08.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.037+0000 7f7173fff640 1 --2- 172.21.15.27:0/1223545139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f717c0ce210 0x7f717c0b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:08.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.037+0000 7f717affd640 1 -- 172.21.15.27:0/1223545139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f717c0ce210 msgr2=0x7f717c0b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:08.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.037+0000 7f717affd640 1 --2- 172.21.15.27:0/1223545139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f717c0ce210 0x7f717c0b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.037+0000 7f717affd640 1 -- 172.21.15.27:0/1223545139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f717c15a0b0 msgr2=0x7f717c0b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:08.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.037+0000 7f717affd640 1 --2- 172.21.15.27:0/1223545139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f717c15a0b0 0x7f717c0b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.037+0000 7f717affd640 1 -- 172.21.15.27:0/1223545139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71640094d0 con 0x7f717c06cee0 2023-12-19T18:21:08.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.037+0000 7f717affd640 1 --2- 172.21.15.27:0/1223545139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717c06cee0 0x7f717c1583b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f716c0098c0 tx=0x7f716c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:08.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.038+0000 7f71737fe640 1 -- 172.21.15.27:0/1223545139 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c03d070 con 0x7f717c06cee0 2023-12-19T18:21:08.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.038+0000 7f71737fe640 1 -- 172.21.15.27:0/1223545139 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716c0308c0 con 0x7f717c06cee0 2023-12-19T18:21:08.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.038+0000 7f71817b9640 1 -- 172.21.15.27:0/1223545139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f716c009400 con 0x7f717c06cee0 2023-12-19T18:21:08.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.038+0000 7f71737fe640 1 -- 172.21.15.27:0/1223545139 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716c030c70 con 0x7f717c06cee0 2023-12-19T18:21:08.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.038+0000 7f71817b9640 1 -- 172.21.15.27:0/1223545139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f717c0b29c0 con 0x7f717c06cee0 2023-12-19T18:21:08.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.039+0000 7f71817b9640 1 -- 172.21.15.27:0/1223545139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717c068e90 con 0x7f717c06cee0 2023-12-19T18:21:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.040+0000 7f71737fe640 1 -- 172.21.15.27:0/1223545139 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f716c043400 con 0x7f717c06cee0 2023-12-19T18:21:08.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.041+0000 7f71737fe640 1 --2- 172.21.15.27:0/1223545139 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f715c06a430 0x7f715c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:08.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.041+0000 7f71737fe640 1 -- 172.21.15.27:0/1223545139 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f716c0aa950 con 0x7f717c06cee0 2023-12-19T18:21:08.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.042+0000 7f7173fff640 1 --2- 172.21.15.27:0/1223545139 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f715c06a430 0x7f715c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:08.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.043+0000 7f7173fff640 1 --2- 172.21.15.27:0/1223545139 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f715c06a430 0x7f715c06c8f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f717c071910 tx=0x7f7164008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:08.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.047+0000 7f71737fe640 1 -- 172.21.15.27:0/1223545139 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f716c076020 con 0x7f717c06cee0 2023-12-19T18:21:08.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.279+0000 7f71817b9640 1 -- 172.21.15.27:0/1223545139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f717c06b6f0 con 0x7f717c06cee0 2023-12-19T18:21:08.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.293+0000 7f71737fe640 1 -- 172.21.15.27:0/1223545139 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v42) v1 ==== 174+0+0 (secure 0 0 0) 0x7f716c079640 con 0x7f717c06cee0 2023-12-19T18:21:08.295 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 12 and data pool 13 2023-12-19T18:21:08.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.300+0000 7f71817b9640 1 -- 172.21.15.27:0/1223545139 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f715c06a430 msgr2=0x7f715c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:08.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.300+0000 7f71817b9640 1 --2- 172.21.15.27:0/1223545139 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f715c06a430 0x7f715c06c8f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f717c071910 tx=0x7f7164008040 comp rx=0 tx=0).stop 2023-12-19T18:21:08.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.300+0000 7f71817b9640 1 -- 172.21.15.27:0/1223545139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717c06cee0 msgr2=0x7f717c1583b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:08.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.300+0000 7f71817b9640 1 --2- 172.21.15.27:0/1223545139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717c06cee0 0x7f717c1583b0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f716c0098c0 tx=0x7f716c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.300+0000 7f71817b9640 1 -- 172.21.15.27:0/1223545139 shutdown_connections 2023-12-19T18:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.300+0000 7f71817b9640 1 --2- 172.21.15.27:0/1223545139 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f715c06a430 0x7f715c06c8f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.300+0000 7f71817b9640 1 --2- 172.21.15.27:0/1223545139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f717c15a0b0 0x7f717c0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.300+0000 7f71817b9640 1 --2- 172.21.15.27:0/1223545139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f717c0ce210 0x7f717c0b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.300+0000 7f71817b9640 1 --2- 172.21.15.27:0/1223545139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f717c06cee0 0x7f717c1583b0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.301+0000 7f71817b9640 1 -- 172.21.15.27:0/1223545139 >> 172.21.15.27:0/1223545139 conn(0x7f717c0cebd0 msgr2=0x7f717c0705d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.301+0000 7f71817b9640 1 -- 172.21.15.27:0/1223545139 shutdown_connections 2023-12-19T18:21:08.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.301+0000 7f71817b9640 1 -- 172.21.15.27:0/1223545139 wait complete. 2023-12-19T18:21:08.331 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:21:08.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.525+0000 7f0535615640 1 Processor -- start 2023-12-19T18:21:08.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.525+0000 7f0535615640 1 -- start start 2023-12-19T18:21:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.526+0000 7f0535615640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0530154030 0x7f05300cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.527+0000 7f0535615640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f05300c5560 0x7f05300afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.527+0000 7f0535615640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05300cc190 con 0x7f05300af630 2023-12-19T18:21:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.527+0000 7f0535615640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05300b0000 con 0x7f0530154030 2023-12-19T18:21:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.527+0000 7f0535615640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05300cb2a0 con 0x7f05300c5560 2023-12-19T18:21:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.527+0000 7f052ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0530154030 0x7f05300cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:08.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.527+0000 7f0527fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f05300af630 0x7f053015d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40918/0 (socket says 172.21.15.27:40918) 2023-12-19T18:21:08.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.527+0000 7f0527fff640 1 -- 172.21.15.27:0/712275894 learned_addr learned my addr 172.21.15.27:0/712275894 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:08.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.527+0000 7f0534e14640 1 --2- 172.21.15.27:0/712275894 >> v2:172.21.15.192:3300/0 conn(0x7f05300c5560 0x7f05300afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:08.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.528+0000 7f052ffff640 1 -- 172.21.15.27:0/712275894 >> v2:172.21.15.192:3300/0 conn(0x7f05300c5560 msgr2=0x7f05300afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:08.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.528+0000 7f052ffff640 1 --2- 172.21.15.27:0/712275894 >> v2:172.21.15.192:3300/0 conn(0x7f05300c5560 0x7f05300afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.528+0000 7f052ffff640 1 -- 172.21.15.27:0/712275894 >> v1:172.21.15.27:6789/0 conn(0x7f05300af630 legacy=0x7f053015d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:08.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.528+0000 7f052ffff640 1 -- 172.21.15.27:0/712275894 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05300cb580 con 0x7f0530154030 2023-12-19T18:21:08.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.528+0000 7f052f7fe640 1 -- 172.21.15.27:0/712275894 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630964701 0 0) 0x7f05300cc190 con 0x7f05300af630 2023-12-19T18:21:08.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.529+0000 7f052ffff640 1 --2- 172.21.15.27:0/712275894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0530154030 0x7f05300cd4f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f05200097a0 tx=0x7f052002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f16239e30a17d64 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:08.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.529+0000 7f052f7fe640 1 -- 172.21.15.27:0/712275894 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520031830 con 0x7f0530154030 2023-12-19T18:21:08.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.530+0000 7f052f7fe640 1 -- 172.21.15.27:0/712275894 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0520031990 con 0x7f0530154030 2023-12-19T18:21:08.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.530+0000 7f052f7fe640 1 -- 172.21.15.27:0/712275894 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520031d20 con 0x7f0530154030 2023-12-19T18:21:08.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.530+0000 7f0535615640 1 -- 172.21.15.27:0/712275894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0530154030 msgr2=0x7f05300cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:08.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.530+0000 7f0535615640 1 --2- 172.21.15.27:0/712275894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0530154030 0x7f05300cd4f0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f05200097a0 tx=0x7f052002b8d0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.530+0000 7f0535615640 1 -- 172.21.15.27:0/712275894 shutdown_connections 2023-12-19T18:21:08.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.530+0000 7f0535615640 1 --2- 172.21.15.27:0/712275894 >> v2:172.21.15.192:3300/0 conn(0x7f05300c5560 0x7f05300afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.530+0000 7f0535615640 1 --2- 172.21.15.27:0/712275894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0530154030 0x7f05300cd4f0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.530+0000 7f0535615640 1 -- 172.21.15.27:0/712275894 >> 172.21.15.27:0/712275894 conn(0x7f0530069c70 msgr2=0x7f053006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:08.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.530+0000 7f0535615640 1 -- 172.21.15.27:0/712275894 shutdown_connections 2023-12-19T18:21:08.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.530+0000 7f0535615640 1 -- 172.21.15.27:0/712275894 wait complete. 2023-12-19T18:21:08.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.532+0000 7f0535615640 1 Processor -- start 2023-12-19T18:21:08.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.533+0000 7f0535615640 1 -- start start 2023-12-19T18:21:08.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.534+0000 7f0535615640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05300af630 0x7f0530152e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:08.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.534+0000 7f0535615640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05300c5560 0x7f05301401a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:08.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.534+0000 7f0535615640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0530154030 0x7f05301495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:08.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.534+0000 7f0535615640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0530143300 con 0x7f05300af630 2023-12-19T18:21:08.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.534+0000 7f0535615640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0530148c90 con 0x7f05300c5560 2023-12-19T18:21:08.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.534+0000 7f0535615640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05301533a0 con 0x7f0530154030 2023-12-19T18:21:08.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.534+0000 7f052ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05300af630 0x7f0530152e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:08.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.534+0000 7f052ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05300af630 0x7f0530152e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33306/0 (socket says 172.21.15.27:33306) 2023-12-19T18:21:08.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.534+0000 7f052ffff640 1 -- 172.21.15.27:0/1812462041 learned_addr learned my addr 172.21.15.27:0/1812462041 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:08.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.535+0000 7f0527fff640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05300c5560 0x7f05301401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:08.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.535+0000 7f052ffff640 1 -- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0530154030 msgr2=0x7f05301495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:08.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.535+0000 7f0534e14640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0530154030 0x7f05301495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:08.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.535+0000 7f052ffff640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0530154030 0x7f05301495c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.535+0000 7f052ffff640 1 -- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05300c5560 msgr2=0x7f05301401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:08.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.535+0000 7f052ffff640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05300c5560 0x7f05301401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:08.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.535+0000 7f052ffff640 1 -- 172.21.15.27:0/1812462041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0520009400 con 0x7f05300af630 2023-12-19T18:21:08.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.535+0000 7f0527fff640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05300c5560 0x7f05301401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:08.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.535+0000 7f052ffff640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05300af630 0x7f0530152e60 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f05200097a0 tx=0x7f05200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:08.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.535+0000 7f052d7fa640 1 -- 172.21.15.27:0/1812462041 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520004230 con 0x7f05300af630 2023-12-19T18:21:08.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.535+0000 7f052d7fa640 1 -- 172.21.15.27:0/1812462041 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0520004390 con 0x7f05300af630 2023-12-19T18:21:08.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.535+0000 7f0535615640 1 -- 172.21.15.27:0/1812462041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05301592b0 con 0x7f05300af630 2023-12-19T18:21:08.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.536+0000 7f052d7fa640 1 -- 172.21.15.27:0/1812462041 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052003a870 con 0x7f05300af630 2023-12-19T18:21:08.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.536+0000 7f0535615640 1 -- 172.21.15.27:0/1812462041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0530148920 con 0x7f05300af630 2023-12-19T18:21:08.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.537+0000 7f0535615640 1 -- 172.21.15.27:0/1812462041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f053004e840 con 0x7f05300af630 2023-12-19T18:21:08.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.538+0000 7f052d7fa640 1 -- 172.21.15.27:0/1812462041 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f052003a9d0 con 0x7f05300af630 2023-12-19T18:21:08.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.539+0000 7f052d7fa640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f051006a430 0x7f051006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:08.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.539+0000 7f052d7fa640 1 -- 172.21.15.27:0/1812462041 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05200aab70 con 0x7f05300af630 2023-12-19T18:21:08.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.540+0000 7f0527fff640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f051006a430 0x7f051006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:08.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.541+0000 7f0527fff640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f051006a430 0x7f051006c8f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f05301480e0 tx=0x7f0518009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:08.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.544+0000 7f052d7fa640 1 -- 172.21.15.27:0/1812462041 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0520076160 con 0x7f05300af630 2023-12-19T18:21:08.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.739+0000 7f0535615640 1 -- 172.21.15.27:0/1812462041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f05300c3c50 con 0x7f05300af630 2023-12-19T18:21:09.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.741+0000 7f052d7fa640 1 -- 172.21.15.27:0/1812462041 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v116) v1 ==== 74+0+16129 (secure 0 0 0) 0x7f0520079780 con 0x7f05300af630 2023-12-19T18:21:09.159 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:09.159 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":116,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:21:08.281141+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:21:06.258281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T18:21:07.308887+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:21:09.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.743+0000 7f0535615640 1 -- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f051006a430 msgr2=0x7f051006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:09.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.744+0000 7f0535615640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f051006a430 0x7f051006c8f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f05301480e0 tx=0x7f0518009040 comp rx=0 tx=0).stop 2023-12-19T18:21:09.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.744+0000 7f0535615640 1 -- 172.21.15.27:0/1812462041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05300af630 msgr2=0x7f0530152e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:09.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.744+0000 7f0535615640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05300af630 0x7f0530152e60 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f05200097a0 tx=0x7f05200046c0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.744+0000 7f0535615640 1 -- 172.21.15.27:0/1812462041 shutdown_connections 2023-12-19T18:21:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.744+0000 7f0535615640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f051006a430 0x7f051006c8f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.744+0000 7f0535615640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0530154030 0x7f05301495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.744+0000 7f0535615640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05300c5560 0x7f05301401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.744+0000 7f0535615640 1 --2- 172.21.15.27:0/1812462041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05300af630 0x7f0530152e60 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.744+0000 7f0535615640 1 -- 172.21.15.27:0/1812462041 >> 172.21.15.27:0/1812462041 conn(0x7f0530069c70 msgr2=0x7f053015d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.744+0000 7f0535615640 1 -- 172.21.15.27:0/1812462041 shutdown_connections 2023-12-19T18:21:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:08.744+0000 7f0535615640 1 -- 172.21.15.27:0/1812462041 wait complete. 2023-12-19T18:21:09.162 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:21:09.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.404+0000 7fe32b59e640 1 Processor -- start 2023-12-19T18:21:09.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.404+0000 7fe32b59e640 1 -- start start 2023-12-19T18:21:09.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.406+0000 7fe32b59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe32c1553a0 0x7fe32c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:09.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.406+0000 7fe32b59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe32c15ee30 0x7fe32c0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:09.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.406+0000 7fe32b59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe32c0b0360 con 0x7fe32c0afeb0 2023-12-19T18:21:09.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.406+0000 7fe32b59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32c0cbcf0 con 0x7fe32c1553a0 2023-12-19T18:21:09.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.406+0000 7fe32b59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe32c0cb350 con 0x7fe32c15ee30 2023-12-19T18:21:09.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.406+0000 7fe32a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe32c1553a0 0x7fe32c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.406+0000 7fe32a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe32c1553a0 0x7fe32c0c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33308/0 (socket says 172.21.15.27:33308) 2023-12-19T18:21:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.406+0000 7fe32a59c640 1 -- 172.21.15.27:0/3283062174 learned_addr learned my addr 172.21.15.27:0/3283062174 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.407+0000 7fe32ad9d640 1 --2- 172.21.15.27:0/3283062174 >> v2:172.21.15.192:3300/0 conn(0x7fe32c15ee30 0x7fe32c0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.407+0000 7fe32a59c640 1 -- 172.21.15.27:0/3283062174 >> v2:172.21.15.192:3300/0 conn(0x7fe32c15ee30 msgr2=0x7fe32c0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.407+0000 7fe32a59c640 1 --2- 172.21.15.27:0/3283062174 >> v2:172.21.15.192:3300/0 conn(0x7fe32c15ee30 0x7fe32c0c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.407+0000 7fe32a59c640 1 -- 172.21.15.27:0/3283062174 >> v1:172.21.15.192:6789/0 conn(0x7fe32c0afeb0 legacy=0x7fe32c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.407+0000 7fe32a59c640 1 -- 172.21.15.27:0/3283062174 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe32c0cb630 con 0x7fe32c1553a0 2023-12-19T18:21:09.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.407+0000 7fe32ad9d640 1 --2- 172.21.15.27:0/3283062174 >> v2:172.21.15.192:3300/0 conn(0x7fe32c15ee30 0x7fe32c0c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:09.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.408+0000 7fe32a59c640 1 --2- 172.21.15.27:0/3283062174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe32c1553a0 0x7fe32c0c50a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe32c158d00 tx=0x7fe31c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d11399a8f74f4ccd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:09.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.408+0000 7fe329d9b640 1 -- 172.21.15.27:0/3283062174 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c0317e0 con 0x7fe32c1553a0 2023-12-19T18:21:09.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.408+0000 7fe329d9b640 1 -- 172.21.15.27:0/3283062174 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31c031940 con 0x7fe32c1553a0 2023-12-19T18:21:09.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.408+0000 7fe329d9b640 1 -- 172.21.15.27:0/3283062174 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c031cd0 con 0x7fe32c1553a0 2023-12-19T18:21:09.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.409+0000 7fe32b59e640 1 -- 172.21.15.27:0/3283062174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe32c1553a0 msgr2=0x7fe32c0c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:09.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.409+0000 7fe32b59e640 1 --2- 172.21.15.27:0/3283062174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe32c1553a0 0x7fe32c0c50a0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fe32c158d00 tx=0x7fe31c02b880 comp rx=0 tx=0).stop 2023-12-19T18:21:09.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.409+0000 7fe32b59e640 1 -- 172.21.15.27:0/3283062174 shutdown_connections 2023-12-19T18:21:09.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.409+0000 7fe32b59e640 1 --2- 172.21.15.27:0/3283062174 >> v2:172.21.15.192:3300/0 conn(0x7fe32c15ee30 0x7fe32c0c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.409+0000 7fe32b59e640 1 --2- 172.21.15.27:0/3283062174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe32c1553a0 0x7fe32c0c50a0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.409+0000 7fe32b59e640 1 -- 172.21.15.27:0/3283062174 >> 172.21.15.27:0/3283062174 conn(0x7fe32c069c70 msgr2=0x7fe32c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:09.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.409+0000 7fe32b59e640 1 -- 172.21.15.27:0/3283062174 shutdown_connections 2023-12-19T18:21:09.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.409+0000 7fe32b59e640 1 -- 172.21.15.27:0/3283062174 wait complete. 2023-12-19T18:21:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.411+0000 7fe32b59e640 1 Processor -- start 2023-12-19T18:21:09.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.411+0000 7fe32b59e640 1 -- start start 2023-12-19T18:21:09.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.412+0000 7fe32b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe32c0afeb0 0x7fe32c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:09.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.412+0000 7fe32b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe32c1553a0 0x7fe32c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:09.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.412+0000 7fe32b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe32c15ee30 0x7fe32c157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:09.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.412+0000 7fe32b59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c1559f0 con 0x7fe32c0afeb0 2023-12-19T18:21:09.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.412+0000 7fe32b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c1588d0 con 0x7fe32c15ee30 2023-12-19T18:21:09.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.412+0000 7fe32b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe32c155cc0 con 0x7fe32c1553a0 2023-12-19T18:21:09.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.413+0000 7fe32a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe32c0afeb0 0x7fe32c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:09.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.413+0000 7fe32a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe32c0afeb0 0x7fe32c1544a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33318/0 (socket says 172.21.15.27:33318) 2023-12-19T18:21:09.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.413+0000 7fe32a59c640 1 -- 172.21.15.27:0/3135192340 learned_addr learned my addr 172.21.15.27:0/3135192340 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:09.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.413+0000 7fe32ad9d640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe32c15ee30 0x7fe32c157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:09.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.413+0000 7fe323fff640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe32c1553a0 0x7fe32c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:09.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.413+0000 7fe32a59c640 1 -- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe32c1553a0 msgr2=0x7fe32c0c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:09.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.413+0000 7fe32a59c640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe32c1553a0 0x7fe32c0c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.413+0000 7fe32a59c640 1 -- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe32c15ee30 msgr2=0x7fe32c157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:09.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.413+0000 7fe32a59c640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe32c15ee30 0x7fe32c157df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.413+0000 7fe32a59c640 1 -- 172.21.15.27:0/3135192340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c0094d0 con 0x7fe32c0afeb0 2023-12-19T18:21:09.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.413+0000 7fe32ad9d640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe32c15ee30 0x7fe32c157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:09.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.413+0000 7fe323fff640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe32c1553a0 0x7fe32c0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:09.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.414+0000 7fe32a59c640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe32c0afeb0 0x7fe32c1544a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe31c02b8c0 tx=0x7fe31c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:09.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.414+0000 7fe322ffd640 1 -- 172.21.15.27:0/3135192340 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c0329c0 con 0x7fe32c0afeb0 2023-12-19T18:21:09.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.414+0000 7fe322ffd640 1 -- 172.21.15.27:0/3135192340 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe31c032b40 con 0x7fe32c0afeb0 2023-12-19T18:21:09.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.414+0000 7fe32b59e640 1 -- 172.21.15.27:0/3135192340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32c155fa0 con 0x7fe32c0afeb0 2023-12-19T18:21:09.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.414+0000 7fe322ffd640 1 -- 172.21.15.27:0/3135192340 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31c0453f0 con 0x7fe32c0afeb0 2023-12-19T18:21:09.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.414+0000 7fe32b59e640 1 -- 172.21.15.27:0/3135192340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe32c1549e0 con 0x7fe32c0afeb0 2023-12-19T18:21:09.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.415+0000 7fe32b59e640 1 -- 172.21.15.27:0/3135192340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe32c04e840 con 0x7fe32c0afeb0 2023-12-19T18:21:09.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.416+0000 7fe322ffd640 1 -- 172.21.15.27:0/3135192340 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe31c04e450 con 0x7fe32c0afeb0 2023-12-19T18:21:09.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.418+0000 7fe322ffd640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe30c06a460 0x7fe30c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:09.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.418+0000 7fe322ffd640 1 -- 172.21.15.27:0/3135192340 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe31c0aac20 con 0x7fe32c0afeb0 2023-12-19T18:21:09.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.418+0000 7fe323fff640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe30c06a460 0x7fe30c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:09.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.420+0000 7fe323fff640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe30c06a460 0x7fe30c06c920 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe314007fd0 tx=0x7fe314009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:09.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.423+0000 7fe322ffd640 1 -- 172.21.15.27:0/3135192340 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe31c0762b0 con 0x7fe32c0afeb0 2023-12-19T18:21:09.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.617+0000 7fe32b59e640 1 -- 172.21.15.27:0/3135192340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe32c155b30 con 0x7fe32c0afeb0 2023-12-19T18:21:09.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.619+0000 7fe322ffd640 1 -- 172.21.15.27:0/3135192340 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16116 (secure 0 0 0) 0x7fe31c0798d0 con 0x7fe32c0afeb0 2023-12-19T18:21:09.619 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:09.619 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":117,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:21:09.285480+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:21:06.258281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T18:21:07.308887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:21:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.622+0000 7fe32b59e640 1 -- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe30c06a460 msgr2=0x7fe30c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.622+0000 7fe32b59e640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe30c06a460 0x7fe30c06c920 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe314007fd0 tx=0x7fe314009040 comp rx=0 tx=0).stop 2023-12-19T18:21:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.622+0000 7fe32b59e640 1 -- 172.21.15.27:0/3135192340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe32c0afeb0 msgr2=0x7fe32c1544a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.622+0000 7fe32b59e640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe32c0afeb0 0x7fe32c1544a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fe31c02b8c0 tx=0x7fe31c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.622+0000 7fe32b59e640 1 -- 172.21.15.27:0/3135192340 shutdown_connections 2023-12-19T18:21:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.622+0000 7fe32b59e640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe30c06a460 0x7fe30c06c920 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.622+0000 7fe32b59e640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe32c15ee30 0x7fe32c157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.622+0000 7fe32b59e640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe32c1553a0 0x7fe32c0c6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.622+0000 7fe32b59e640 1 --2- 172.21.15.27:0/3135192340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe32c0afeb0 0x7fe32c1544a0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.622+0000 7fe32b59e640 1 -- 172.21.15.27:0/3135192340 >> 172.21.15.27:0/3135192340 conn(0x7fe32c069c70 msgr2=0x7fe32c0cf0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.622+0000 7fe32b59e640 1 -- 172.21.15.27:0/3135192340 shutdown_connections 2023-12-19T18:21:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.622+0000 7fe32b59e640 1 -- 172.21.15.27:0/3135192340 wait complete. 2023-12-19T18:21:09.642 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:21:09.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.884+0000 7fd6f6bbc640 1 Processor -- start 2023-12-19T18:21:09.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.884+0000 7fd6f6bbc640 1 -- start start 2023-12-19T18:21:09.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.886+0000 7fd6f6bbc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6f00c5040 0x7fd6f00ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:09.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.886+0000 7fd6f6bbc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd6f00adc50 0x7fd6f0159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:09.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.886+0000 7fd6f6bbc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f015d780 con 0x7fd6f00cbe20 2023-12-19T18:21:09.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.886+0000 7fd6f6bbc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f00cc200 con 0x7fd6f00c5040 2023-12-19T18:21:09.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.886+0000 7fd6f6bbc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6f015d9d0 con 0x7fd6f00adc50 2023-12-19T18:21:09.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.887+0000 7fd6f53b9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6f00c5040 0x7fd6f00ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:09.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.887+0000 7fd6f53b9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd6f00c5040 0x7fd6f00ad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33328/0 (socket says 172.21.15.27:33328) 2023-12-19T18:21:09.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.887+0000 7fd6f53b9640 1 -- 172.21.15.27:0/873983573 learned_addr learned my addr 172.21.15.27:0/873983573 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:09.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.887+0000 7fd6f63bb640 1 --2- 172.21.15.27:0/873983573 >> v2:172.21.15.192:3300/0 conn(0x7fd6f00adc50 0x7fd6f0159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:09.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.887+0000 7fd6f53b9640 1 -- 172.21.15.27:0/873983573 >> v2:172.21.15.192:3300/0 conn(0x7fd6f00adc50 msgr2=0x7fd6f0159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:09.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.888+0000 7fd6f53b9640 1 --2- 172.21.15.27:0/873983573 >> v2:172.21.15.192:3300/0 conn(0x7fd6f00adc50 0x7fd6f0159ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.888+0000 7fd6f53b9640 1 -- 172.21.15.27:0/873983573 >> v1:172.21.15.192:6789/0 conn(0x7fd6f00cbe20 legacy=0x7fd6f00c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:09.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.888+0000 7fd6f53b9640 1 -- 172.21.15.27:0/873983573 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f015dcb0 con 0x7fd6f00c5040 2023-12-19T18:21:09.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.890+0000 7fd6f53b9640 1 --2- 172.21.15.27:0/873983573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f00c5040 0x7fd6f00ad710 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd6f006f6c0 tx=0x7fd6e002b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1816393755b44ae7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.891+0000 7fd6f4bb8640 1 -- 172.21.15.27:0/873983573 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e00317e0 con 0x7fd6f00c5040 2023-12-19T18:21:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.891+0000 7fd6f4bb8640 1 -- 172.21.15.27:0/873983573 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e0031940 con 0x7fd6f00c5040 2023-12-19T18:21:09.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.891+0000 7fd6f4bb8640 1 -- 172.21.15.27:0/873983573 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e0031cd0 con 0x7fd6f00c5040 2023-12-19T18:21:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.892+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/873983573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f00c5040 msgr2=0x7fd6f00ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.892+0000 7fd6f6bbc640 1 --2- 172.21.15.27:0/873983573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f00c5040 0x7fd6f00ad710 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fd6f006f6c0 tx=0x7fd6e002b880 comp rx=0 tx=0).stop 2023-12-19T18:21:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.892+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/873983573 shutdown_connections 2023-12-19T18:21:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.892+0000 7fd6f6bbc640 1 --2- 172.21.15.27:0/873983573 >> v2:172.21.15.192:3300/0 conn(0x7fd6f00adc50 0x7fd6f0159ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.892+0000 7fd6f6bbc640 1 --2- 172.21.15.27:0/873983573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f00c5040 0x7fd6f00ad710 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.892+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/873983573 >> 172.21.15.27:0/873983573 conn(0x7fd6f00b05b0 msgr2=0x7fd6f00b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:09.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.892+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/873983573 shutdown_connections 2023-12-19T18:21:09.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.892+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/873983573 wait complete. 2023-12-19T18:21:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.894+0000 7fd6f6bbc640 1 Processor -- start 2023-12-19T18:21:09.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.894+0000 7fd6f6bbc640 1 -- start start 2023-12-19T18:21:09.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.895+0000 7fd6f6bbc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f00adc50 0x7fd6f00c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:09.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.895+0000 7fd6f6bbc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6f00c5040 0x7fd6f00c4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:09.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.896+0000 7fd6f6bbc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6f00cbe20 0x7fd6f0154710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:09.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.896+0000 7fd6f6bbc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00c5f90 con 0x7fd6f00adc50 2023-12-19T18:21:09.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.896+0000 7fd6f6bbc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00c61e0 con 0x7fd6f00cbe20 2023-12-19T18:21:09.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.896+0000 7fd6f6bbc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f00c64c0 con 0x7fd6f00c5040 2023-12-19T18:21:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.896+0000 7fd6f5bba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f00adc50 0x7fd6f00c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.896+0000 7fd6f5bba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f00adc50 0x7fd6f00c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33336/0 (socket says 172.21.15.27:33336) 2023-12-19T18:21:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.896+0000 7fd6f5bba640 1 -- 172.21.15.27:0/3680310501 learned_addr learned my addr 172.21.15.27:0/3680310501 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.896+0000 7fd6f63bb640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6f00cbe20 0x7fd6f0154710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.896+0000 7fd6f53b9640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6f00c5040 0x7fd6f00c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.896+0000 7fd6f5bba640 1 -- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6f00c5040 msgr2=0x7fd6f00c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.896+0000 7fd6f5bba640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6f00c5040 0x7fd6f00c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.896+0000 7fd6f5bba640 1 -- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6f00cbe20 msgr2=0x7fd6f0154710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.896+0000 7fd6f5bba640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6f00cbe20 0x7fd6f0154710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.897+0000 7fd6f5bba640 1 -- 172.21.15.27:0/3680310501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e00094d0 con 0x7fd6f00adc50 2023-12-19T18:21:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.897+0000 7fd6f5bba640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f00adc50 0x7fd6f00c6fa0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00c400 tx=0x7fd6ec00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.897+0000 7fd6e67fc640 1 -- 172.21.15.27:0/3680310501 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec0052d0 con 0x7fd6f00adc50 2023-12-19T18:21:09.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.897+0000 7fd6e67fc640 1 -- 172.21.15.27:0/3680310501 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ec005430 con 0x7fd6f00adc50 2023-12-19T18:21:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.897+0000 7fd6e67fc640 1 -- 172.21.15.27:0/3680310501 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec003830 con 0x7fd6f00adc50 2023-12-19T18:21:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.897+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/3680310501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f00c67a0 con 0x7fd6f00adc50 2023-12-19T18:21:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.897+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/3680310501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f00c4860 con 0x7fd6f00adc50 2023-12-19T18:21:09.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.899+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/3680310501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f00c6cc0 con 0x7fd6f00adc50 2023-12-19T18:21:09.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.900+0000 7fd6e67fc640 1 -- 172.21.15.27:0/3680310501 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd6ec003990 con 0x7fd6f00adc50 2023-12-19T18:21:09.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.901+0000 7fd6e67fc640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6d406a460 0x7fd6d406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:09.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.901+0000 7fd6e67fc640 1 -- 172.21.15.27:0/3680310501 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6ec087e20 con 0x7fd6f00adc50 2023-12-19T18:21:09.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.902+0000 7fd6f53b9640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6d406a460 0x7fd6d406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:09.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.903+0000 7fd6f53b9640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6d406a460 0x7fd6d406c920 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0036000 tx=0x7fd6e003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:09.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:09.906+0000 7fd6e67fc640 1 -- 172.21.15.27:0/3680310501 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd6f00c6cc0 con 0x7fd6f00adc50 2023-12-19T18:21:10.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.137+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/3680310501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fd6f00c6cc0 con 0x7fd6f00adc50 2023-12-19T18:21:10.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.803+0000 7fd6e67fc640 1 -- 172.21.15.27:0/3680310501 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7fd6f00c6cc0 con 0x7fd6f00adc50 2023-12-19T18:21:10.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.806+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6d406a460 msgr2=0x7fd6d406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:10.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.806+0000 7fd6f6bbc640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6d406a460 0x7fd6d406c920 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd6e0036000 tx=0x7fd6e003d040 comp rx=0 tx=0).stop 2023-12-19T18:21:10.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.806+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/3680310501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f00adc50 msgr2=0x7fd6f00c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:10.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.806+0000 7fd6f6bbc640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f00adc50 0x7fd6f00c6fa0 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec00c400 tx=0x7fd6ec00c8c0 comp rx=0 tx=0).stop 2023-12-19T18:21:10.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.807+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/3680310501 shutdown_connections 2023-12-19T18:21:10.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.807+0000 7fd6f6bbc640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6d406a460 0x7fd6d406c920 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:10.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.807+0000 7fd6f6bbc640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6f00cbe20 0x7fd6f0154710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:10.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.807+0000 7fd6f6bbc640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6f00c5040 0x7fd6f00c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:10.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.807+0000 7fd6f6bbc640 1 --2- 172.21.15.27:0/3680310501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6f00adc50 0x7fd6f00c6fa0 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:10.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.807+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/3680310501 >> 172.21.15.27:0/3680310501 conn(0x7fd6f00b05b0 msgr2=0x7fd6f00ceec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:10.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.807+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/3680310501 shutdown_connections 2023-12-19T18:21:10.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:10.807+0000 7fd6f6bbc640 1 -- 172.21.15.27:0/3680310501 wait complete. 2023-12-19T18:21:10.829 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:21:10.829 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:21:11.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.072+0000 7f75bb4bd640 1 Processor -- start 2023-12-19T18:21:11.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.072+0000 7f75bb4bd640 1 -- start start 2023-12-19T18:21:11.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.074+0000 7f75bb4bd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75b415a0b0 0x7f75b40cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:11.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.074+0000 7f75bb4bd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f75b40ce210 0x7f75b40cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:11.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.074+0000 7f75bb4bd640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b40c9880 con 0x7f75b406cee0 2023-12-19T18:21:11.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.074+0000 7f75bb4bd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b40c9de0 con 0x7f75b40ce210 2023-12-19T18:21:11.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.074+0000 7f75bb4bd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40c9b20 con 0x7f75b415a0b0 2023-12-19T18:21:11.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.074+0000 7f75b8a58640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f75b40ce210 0x7f75b40cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:11.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.074+0000 7f75b8a58640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f75b40ce210 0x7f75b40cd400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33340/0 (socket says 172.21.15.27:33340) 2023-12-19T18:21:11.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.074+0000 7f75b8a58640 1 -- 172.21.15.27:0/2963794138 learned_addr learned my addr 172.21.15.27:0/2963794138 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:11.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.074+0000 7f75b9259640 1 --2- 172.21.15.27:0/2963794138 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75b415a0b0 0x7f75b40cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:11.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.075+0000 7f75b8a58640 1 -- 172.21.15.27:0/2963794138 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75b415a0b0 msgr2=0x7f75b40cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:11.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.075+0000 7f75b8a58640 1 --2- 172.21.15.27:0/2963794138 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75b415a0b0 0x7f75b40cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:11.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.075+0000 7f75b8a58640 1 -- 172.21.15.27:0/2963794138 >> v1:172.21.15.192:6789/0 conn(0x7f75b406cee0 legacy=0x7f75b4070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:11.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.075+0000 7f75b8a58640 1 -- 172.21.15.27:0/2963794138 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b40cc920 con 0x7f75b40ce210 2023-12-19T18:21:11.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.076+0000 7f75b8a58640 1 --2- 172.21.15.27:0/2963794138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b40ce210 0x7f75b40cd400 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009870 tx=0x7f75a402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=858e058b47e60fb7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:11.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.076+0000 7f75abfff640 1 -- 172.21.15.27:0/2963794138 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4031640 con 0x7f75b40ce210 2023-12-19T18:21:11.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.076+0000 7f75abfff640 1 -- 172.21.15.27:0/2963794138 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75a40317a0 con 0x7f75b40ce210 2023-12-19T18:21:11.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.076+0000 7f75abfff640 1 -- 172.21.15.27:0/2963794138 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75a4031b30 con 0x7f75b40ce210 2023-12-19T18:21:11.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.077+0000 7f75bb4bd640 1 -- 172.21.15.27:0/2963794138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b40ce210 msgr2=0x7f75b40cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:11.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.077+0000 7f75bb4bd640 1 --2- 172.21.15.27:0/2963794138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b40ce210 0x7f75b40cd400 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f75a4009870 tx=0x7f75a402b8f0 comp rx=0 tx=0).stop 2023-12-19T18:21:11.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.077+0000 7f75bb4bd640 1 -- 172.21.15.27:0/2963794138 shutdown_connections 2023-12-19T18:21:11.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.077+0000 7f75bb4bd640 1 --2- 172.21.15.27:0/2963794138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b40ce210 0x7f75b40cd400 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:11.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.077+0000 7f75bb4bd640 1 --2- 172.21.15.27:0/2963794138 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75b415a0b0 0x7f75b40cdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:11.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.077+0000 7f75bb4bd640 1 -- 172.21.15.27:0/2963794138 >> 172.21.15.27:0/2963794138 conn(0x7f75b40cebd0 msgr2=0x7f75b40cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:11.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.077+0000 7f75bb4bd640 1 -- 172.21.15.27:0/2963794138 shutdown_connections 2023-12-19T18:21:11.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.077+0000 7f75bb4bd640 1 -- 172.21.15.27:0/2963794138 wait complete. 2023-12-19T18:21:11.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.079+0000 7f75bb4bd640 1 Processor -- start 2023-12-19T18:21:11.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.079+0000 7f75bb4bd640 1 -- start start 2023-12-19T18:21:11.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.080+0000 7f75bb4bd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75b406cee0 0x7f75b4158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:11.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.080+0000 7f75bb4bd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75b40ce210 0x7f75b41f28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:11.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.080+0000 7f75bb4bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b415a0b0 0x7f75b415b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:11.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.080+0000 7f75bb4bd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b4158740 con 0x7f75b415a0b0 2023-12-19T18:21:11.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.080+0000 7f75bb4bd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b41f3920 con 0x7f75b406cee0 2023-12-19T18:21:11.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.080+0000 7f75bb4bd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b41f3360 con 0x7f75b40ce210 2023-12-19T18:21:11.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.080+0000 7f75b9a5a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b415a0b0 0x7f75b415b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:11.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.080+0000 7f75b9a5a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b415a0b0 0x7f75b415b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33348/0 (socket says 172.21.15.27:33348) 2023-12-19T18:21:11.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.080+0000 7f75b9a5a640 1 -- 172.21.15.27:0/3355098993 learned_addr learned my addr 172.21.15.27:0/3355098993 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:11.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.081+0000 7f75b8a58640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75b40ce210 0x7f75b41f28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:11.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.081+0000 7f75b9259640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75b406cee0 0x7f75b4158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:11.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.081+0000 7f75b9a5a640 1 -- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75b40ce210 msgr2=0x7f75b41f28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:11.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.081+0000 7f75b9a5a640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75b40ce210 0x7f75b41f28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:11.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.081+0000 7f75b9a5a640 1 -- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75b406cee0 msgr2=0x7f75b4158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:11.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.081+0000 7f75b9a5a640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75b406cee0 0x7f75b4158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.081+0000 7f75b9a5a640 1 -- 172.21.15.27:0/3355098993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a40094d0 con 0x7f75b415a0b0 2023-12-19T18:21:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.081+0000 7f75b9259640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75b406cee0 0x7f75b4158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.081+0000 7f75b9a5a640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b415a0b0 0x7f75b415b000 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00b970 tx=0x7f75ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.082+0000 7f75a9ffb640 1 -- 172.21.15.27:0/3355098993 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac00cb40 con 0x7f75b415a0b0 2023-12-19T18:21:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.082+0000 7f75a9ffb640 1 -- 172.21.15.27:0/3355098993 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ac00cca0 con 0x7f75b415a0b0 2023-12-19T18:21:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.082+0000 7f75a9ffb640 1 -- 172.21.15.27:0/3355098993 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac014680 con 0x7f75b415a0b0 2023-12-19T18:21:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.082+0000 7f75bb4bd640 1 -- 172.21.15.27:0/3355098993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b41f3640 con 0x7f75b415a0b0 2023-12-19T18:21:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.082+0000 7f75bb4bd640 1 -- 172.21.15.27:0/3355098993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b415aae0 con 0x7f75b415a0b0 2023-12-19T18:21:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.083+0000 7f75bb4bd640 1 -- 172.21.15.27:0/3355098993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b415acd0 con 0x7f75b415a0b0 2023-12-19T18:21:11.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.084+0000 7f75a9ffb640 1 -- 172.21.15.27:0/3355098993 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f75ac01d3b0 con 0x7f75b415a0b0 2023-12-19T18:21:11.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.085+0000 7f75a9ffb640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f759806a430 0x7f759806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:11.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.085+0000 7f75a9ffb640 1 -- 172.21.15.27:0/3355098993 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f75ac088750 con 0x7f75b415a0b0 2023-12-19T18:21:11.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.086+0000 7f75b9259640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f759806a430 0x7f759806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:11.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.087+0000 7f75b9259640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f759806a430 0x7f759806c8f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f75b0006fd0 tx=0x7f75b0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:11.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.089+0000 7f75a9ffb640 1 -- 172.21.15.27:0/3355098993 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f75ac053d30 con 0x7f75b415a0b0 2023-12-19T18:21:11.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.320+0000 7f75bb4bd640 1 -- 172.21.15.27:0/3355098993 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f75b4068e90 con 0x7f75b415a0b0 2023-12-19T18:21:11.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.809+0000 7f75a9ffb640 1 -- 172.21.15.27:0/3355098993 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f75ac057350 con 0x7f75b415a0b0 2023-12-19T18:21:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.814+0000 7f75bb4bd640 1 -- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f759806a430 msgr2=0x7f759806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.814+0000 7f75bb4bd640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f759806a430 0x7f759806c8f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f75b0006fd0 tx=0x7f75b0008040 comp rx=0 tx=0).stop 2023-12-19T18:21:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.814+0000 7f75bb4bd640 1 -- 172.21.15.27:0/3355098993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b415a0b0 msgr2=0x7f75b415b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.814+0000 7f75bb4bd640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b415a0b0 0x7f75b415b000 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00b970 tx=0x7f75ac00be30 comp rx=0 tx=0).stop 2023-12-19T18:21:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.814+0000 7f75bb4bd640 1 -- 172.21.15.27:0/3355098993 shutdown_connections 2023-12-19T18:21:11.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.814+0000 7f75bb4bd640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f759806a430 0x7f759806c8f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.814+0000 7f75bb4bd640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b415a0b0 0x7f75b415b000 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.814+0000 7f75bb4bd640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f75b40ce210 0x7f75b41f28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.814+0000 7f75bb4bd640 1 --2- 172.21.15.27:0/3355098993 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f75b406cee0 0x7f75b4158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.814+0000 7f75bb4bd640 1 -- 172.21.15.27:0/3355098993 >> 172.21.15.27:0/3355098993 conn(0x7f75b40cebd0 msgr2=0x7f75b4070d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.814+0000 7f75bb4bd640 1 -- 172.21.15.27:0/3355098993 shutdown_connections 2023-12-19T18:21:11.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:11.814+0000 7f75bb4bd640 1 -- 172.21.15.27:0/3355098993 wait complete. 2023-12-19T18:21:11.840 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:21:11.840 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:21:12.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.080+0000 7ff17359e640 1 Processor -- start 2023-12-19T18:21:12.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.080+0000 7ff17359e640 1 -- start start 2023-12-19T18:21:12.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.082+0000 7ff17359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff17415ee30 0x7ff1740c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:12.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.082+0000 7ff17359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1740cb350 con 0x7ff1740afeb0 2023-12-19T18:21:12.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.082+0000 7ff17359e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1740cb630 con 0x7ff1741553a0 2023-12-19T18:21:12.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.082+0000 7ff17359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1740cbcf0 con 0x7ff17415ee30 2023-12-19T18:21:12.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.083+0000 7ff16bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff17415ee30 0x7ff1740c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:12.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.083+0000 7ff172d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff1740afeb0 0x7ff1740c39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56512/0 (socket says 172.21.15.27:56512) 2023-12-19T18:21:12.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.083+0000 7ff172d9d640 1 -- 172.21.15.27:0/3924891216 learned_addr learned my addr 172.21.15.27:0/3924891216 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:12.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.083+0000 7ff171d9b640 1 -- 172.21.15.27:0/3924891216 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1280525651 0 0) 0x7ff1740cb350 con 0x7ff1740afeb0 2023-12-19T18:21:12.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.083+0000 7ff171d9b640 1 -- 172.21.15.27:0/3924891216 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff160003520 con 0x7ff1740afeb0 2023-12-19T18:21:12.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.083+0000 7ff16bfff640 1 -- 172.21.15.27:0/3924891216 >> v1:172.21.15.192:6789/0 conn(0x7ff1741553a0 legacy=0x7ff17415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.083+0000 7ff16bfff640 1 -- 172.21.15.27:0/3924891216 >> v1:172.21.15.27:6789/0 conn(0x7ff1740afeb0 legacy=0x7ff1740c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.083+0000 7ff16bfff640 1 -- 172.21.15.27:0/3924891216 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1740cb910 con 0x7ff17415ee30 2023-12-19T18:21:12.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.085+0000 7ff16bfff640 1 --2- 172.21.15.27:0/3924891216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17415ee30 0x7ff1740c50a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0097a0 tx=0x7ff15c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e3c77ee4eeb146 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:12.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.085+0000 7ff171d9b640 1 -- 172.21.15.27:0/3924891216 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c031830 con 0x7ff17415ee30 2023-12-19T18:21:12.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.085+0000 7ff171d9b640 1 -- 172.21.15.27:0/3924891216 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15c031990 con 0x7ff17415ee30 2023-12-19T18:21:12.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.085+0000 7ff171d9b640 1 -- 172.21.15.27:0/3924891216 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15c031d00 con 0x7ff17415ee30 2023-12-19T18:21:12.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.086+0000 7ff17359e640 1 -- 172.21.15.27:0/3924891216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17415ee30 msgr2=0x7ff1740c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.086+0000 7ff17359e640 1 --2- 172.21.15.27:0/3924891216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17415ee30 0x7ff1740c50a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7ff15c0097a0 tx=0x7ff15c02b8d0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.086+0000 7ff17359e640 1 -- 172.21.15.27:0/3924891216 shutdown_connections 2023-12-19T18:21:12.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.086+0000 7ff17359e640 1 --2- 172.21.15.27:0/3924891216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17415ee30 0x7ff1740c50a0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.086+0000 7ff17359e640 1 -- 172.21.15.27:0/3924891216 >> 172.21.15.27:0/3924891216 conn(0x7ff174069c70 msgr2=0x7ff17406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:12.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.086+0000 7ff17359e640 1 -- 172.21.15.27:0/3924891216 shutdown_connections 2023-12-19T18:21:12.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.086+0000 7ff17359e640 1 -- 172.21.15.27:0/3924891216 wait complete. 2023-12-19T18:21:12.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.088+0000 7ff17359e640 1 Processor -- start 2023-12-19T18:21:12.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.088+0000 7ff17359e640 1 -- start start 2023-12-19T18:21:12.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.089+0000 7ff17359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1740afeb0 0x7ff174158550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:12.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.089+0000 7ff17359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1741553a0 0x7ff17415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:12.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.089+0000 7ff17359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17415ee30 0x7ff17414fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:12.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.089+0000 7ff17359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17415cda0 con 0x7ff17415ee30 2023-12-19T18:21:12.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.089+0000 7ff17359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff17415c470 con 0x7ff1741553a0 2023-12-19T18:21:12.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.089+0000 7ff17359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17415c750 con 0x7ff1740afeb0 2023-12-19T18:21:12.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.089+0000 7ff172d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17415ee30 0x7ff17414fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:12.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.089+0000 7ff172d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17415ee30 0x7ff17414fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35166/0 (socket says 172.21.15.27:35166) 2023-12-19T18:21:12.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.089+0000 7ff172d9d640 1 -- 172.21.15.27:0/1720763991 learned_addr learned my addr 172.21.15.27:0/1720763991 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:12.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.090+0000 7ff17259c640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1740afeb0 0x7ff174158550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:12.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.090+0000 7ff16bfff640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1741553a0 0x7ff17415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:12.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.090+0000 7ff172d9d640 1 -- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1740afeb0 msgr2=0x7ff174158550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.090+0000 7ff172d9d640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1740afeb0 0x7ff174158550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.090+0000 7ff172d9d640 1 -- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1741553a0 msgr2=0x7ff17415b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.090+0000 7ff172d9d640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1741553a0 0x7ff17415b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.090+0000 7ff172d9d640 1 -- 172.21.15.27:0/1720763991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff15c009400 con 0x7ff17415ee30 2023-12-19T18:21:12.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.090+0000 7ff172d9d640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17415ee30 0x7ff17414fe00 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00cb40 tx=0x7ff16c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:12.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.090+0000 7ff16affd640 1 -- 172.21.15.27:0/1720763991 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c00aca0 con 0x7ff17415ee30 2023-12-19T18:21:12.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.090+0000 7ff16affd640 1 -- 172.21.15.27:0/1720763991 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16c00ae00 con 0x7ff17415ee30 2023-12-19T18:21:12.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.090+0000 7ff16affd640 1 -- 172.21.15.27:0/1720763991 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16c012620 con 0x7ff17415ee30 2023-12-19T18:21:12.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.090+0000 7ff17359e640 1 -- 172.21.15.27:0/1720763991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17415ca30 con 0x7ff17415ee30 2023-12-19T18:21:12.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.091+0000 7ff17359e640 1 -- 172.21.15.27:0/1720763991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1741558a0 con 0x7ff17415ee30 2023-12-19T18:21:12.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.092+0000 7ff17359e640 1 -- 172.21.15.27:0/1720763991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17415cf10 con 0x7ff17415ee30 2023-12-19T18:21:12.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.093+0000 7ff16affd640 1 -- 172.21.15.27:0/1720763991 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff16c007070 con 0x7ff17415ee30 2023-12-19T18:21:12.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.094+0000 7ff16affd640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff15406a430 0x7ff15406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.094+0000 7ff16affd640 1 -- 172.21.15.27:0/1720763991 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff16c088ab0 con 0x7ff17415ee30 2023-12-19T18:21:12.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.094+0000 7ff17259c640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff15406a430 0x7ff15406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:12.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.095+0000 7ff17259c640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff15406a430 0x7ff15406c8f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff164001640 tx=0x7ff164009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:12.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.098+0000 7ff16affd640 1 -- 172.21.15.27:0/1720763991 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff16c053f70 con 0x7ff17415ee30 2023-12-19T18:21:12.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.330+0000 7ff17359e640 1 -- 172.21.15.27:0/1720763991 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff17404e840 con 0x7ff17415ee30 2023-12-19T18:21:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.332+0000 7ff16affd640 1 -- 172.21.15.27:0/1720763991 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4714 (secure 0 0 0) 0x7ff16c017090 con 0x7ff17415ee30 2023-12-19T18:21:12.332 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:12.332 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2779436922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2779436922},{"type":"v1","addr":"172.21.15.192:6836","nonce":2779436922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3121062779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3121062779},{"type":"v1","addr":"172.21.15.192:6837","nonce":3121062779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5357,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2032553869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2032553869},{"type":"v1","addr":"172.21.15.27:6835","nonce":2032553869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:21:08.281615+0000","modified":"2023-12-19T18:21:11.800655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/252970177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":252970177},{"type":"v1","addr":"172.21.15.27:6834","nonce":252970177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T18:21:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 46 2023-12-19T18:21:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.334+0000 7ff17359e640 1 -- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff15406a430 msgr2=0x7ff15406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.334+0000 7ff17359e640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff15406a430 0x7ff15406c8f0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7ff164001640 tx=0x7ff164009040 comp rx=0 tx=0).stop 2023-12-19T18:21:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.335+0000 7ff17359e640 1 -- 172.21.15.27:0/1720763991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17415ee30 msgr2=0x7ff17414fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.335+0000 7ff17359e640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17415ee30 0x7ff17414fe00 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7ff16c00cb40 tx=0x7ff16c00a060 comp rx=0 tx=0).stop 2023-12-19T18:21:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.335+0000 7ff17359e640 1 -- 172.21.15.27:0/1720763991 shutdown_connections 2023-12-19T18:21:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.335+0000 7ff17359e640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff15406a430 0x7ff15406c8f0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.335+0000 7ff17359e640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff17415ee30 0x7ff17414fe00 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.335+0000 7ff17359e640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff1741553a0 0x7ff17415b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.335+0000 7ff17359e640 1 --2- 172.21.15.27:0/1720763991 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff1740afeb0 0x7ff174158550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.335+0000 7ff17359e640 1 -- 172.21.15.27:0/1720763991 >> 172.21.15.27:0/1720763991 conn(0x7ff174069c70 msgr2=0x7ff1740c4d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.335+0000 7ff17359e640 1 -- 172.21.15.27:0/1720763991 shutdown_connections 2023-12-19T18:21:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.335+0000 7ff17359e640 1 -- 172.21.15.27:0/1720763991 wait complete. 2023-12-19T18:21:12.354 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:21:12.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.589+0000 7fd070abe640 1 Processor -- start 2023-12-19T18:21:12.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.589+0000 7fd070abe640 1 -- start start 2023-12-19T18:21:12.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.591+0000 7fd070abe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd06c0b4ef0 0x7fd06c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:12.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.591+0000 7fd070abe640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06c0b5e90 con 0x7fd06c1604d0 2023-12-19T18:21:12.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.591+0000 7fd070abe640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd06c0d2850 con 0x7fd06c0d2350 2023-12-19T18:21:12.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.591+0000 7fd070abe640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd06c0d2b30 con 0x7fd06c0b4ef0 2023-12-19T18:21:12.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.591+0000 7fd06b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd06c1604d0 0x7fd06c0b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56516/0 (socket says 172.21.15.27:56516) 2023-12-19T18:21:12.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.591+0000 7fd06b7fe640 1 -- 172.21.15.27:0/3726817905 learned_addr learned my addr 172.21.15.27:0/3726817905 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:12.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.591+0000 7fd06affd640 1 --2- 172.21.15.27:0/3726817905 >> v2:172.21.15.192:3300/0 conn(0x7fd06c0b4ef0 0x7fd06c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:12.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.592+0000 7fd06a7fc640 1 -- 172.21.15.27:0/3726817905 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 276578144 0 0) 0x7fd06c0b5e90 con 0x7fd06c1604d0 2023-12-19T18:21:12.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.592+0000 7fd06a7fc640 1 -- 172.21.15.27:0/3726817905 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd054003520 con 0x7fd06c1604d0 2023-12-19T18:21:12.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.592+0000 7fd06affd640 1 -- 172.21.15.27:0/3726817905 >> v1:172.21.15.192:6789/0 conn(0x7fd06c0d2350 legacy=0x7fd06c0b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.592+0000 7fd06affd640 1 -- 172.21.15.27:0/3726817905 >> v1:172.21.15.27:6789/0 conn(0x7fd06c1604d0 legacy=0x7fd06c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.592+0000 7fd06a7fc640 1 -- 172.21.15.27:0/3726817905 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1483604285 0 0) 0x7fd054003520 con 0x7fd06c1604d0 2023-12-19T18:21:12.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.592+0000 7fd06affd640 1 -- 172.21.15.27:0/3726817905 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c0cded0 con 0x7fd06c0b4ef0 2023-12-19T18:21:12.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.594+0000 7fd06affd640 1 --2- 172.21.15.27:0/3726817905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06c0b4ef0 0x7fd06c0b6300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd0600097a0 tx=0x7fd06002b960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=49457da63d6395bd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:12.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.594+0000 7fd06a7fc640 1 -- 172.21.15.27:0/3726817905 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060030610 con 0x7fd06c0b4ef0 2023-12-19T18:21:12.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.594+0000 7fd06a7fc640 1 -- 172.21.15.27:0/3726817905 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd060030770 con 0x7fd06c0b4ef0 2023-12-19T18:21:12.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.594+0000 7fd06a7fc640 1 -- 172.21.15.27:0/3726817905 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060030b00 con 0x7fd06c0b4ef0 2023-12-19T18:21:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.595+0000 7fd070abe640 1 -- 172.21.15.27:0/3726817905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06c0b4ef0 msgr2=0x7fd06c0b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.595+0000 7fd070abe640 1 --2- 172.21.15.27:0/3726817905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06c0b4ef0 0x7fd06c0b6300 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fd0600097a0 tx=0x7fd06002b960 comp rx=0 tx=0).stop 2023-12-19T18:21:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.595+0000 7fd070abe640 1 -- 172.21.15.27:0/3726817905 shutdown_connections 2023-12-19T18:21:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.595+0000 7fd070abe640 1 --2- 172.21.15.27:0/3726817905 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06c0b4ef0 0x7fd06c0b6300 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.595+0000 7fd070abe640 1 -- 172.21.15.27:0/3726817905 >> 172.21.15.27:0/3726817905 conn(0x7fd06c0cf7e0 msgr2=0x7fd06c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.595+0000 7fd070abe640 1 -- 172.21.15.27:0/3726817905 shutdown_connections 2023-12-19T18:21:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.595+0000 7fd070abe640 1 -- 172.21.15.27:0/3726817905 wait complete. 2023-12-19T18:21:12.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.597+0000 7fd070abe640 1 Processor -- start 2023-12-19T18:21:12.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.597+0000 7fd070abe640 1 -- start start 2023-12-19T18:21:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.598+0000 7fd070abe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd06c0b4ef0 0x7fd06c0c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.598+0000 7fd070abe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd06c0d2350 0x7fd06c0ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.598+0000 7fd070abe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06c1604d0 0x7fd06c154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.598+0000 7fd070abe640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0cae80 con 0x7fd06c0d2350 2023-12-19T18:21:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.598+0000 7fd070abe640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0c6ee0 con 0x7fd06c1604d0 2023-12-19T18:21:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.598+0000 7fd070abe640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd06c0c71c0 con 0x7fd06c0b4ef0 2023-12-19T18:21:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.598+0000 7fd06affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd06c0d2350 0x7fd06c0ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.598+0000 7fd06affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd06c0d2350 0x7fd06c0ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35176/0 (socket says 172.21.15.27:35176) 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.598+0000 7fd06affd640 1 -- 172.21.15.27:0/1294777529 learned_addr learned my addr 172.21.15.27:0/1294777529 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd06b7fe640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd06c0b4ef0 0x7fd06c0c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd06bfff640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06c1604d0 0x7fd06c154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd06affd640 1 -- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd06c0b4ef0 msgr2=0x7fd06c0c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd06affd640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd06c0b4ef0 0x7fd06c0c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd06affd640 1 -- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06c1604d0 msgr2=0x7fd06c154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd06affd640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06c1604d0 0x7fd06c154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd06affd640 1 -- 172.21.15.27:0/1294777529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd060009400 con 0x7fd06c0d2350 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd06b7fe640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd06c0b4ef0 0x7fd06c0c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd06bfff640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06c1604d0 0x7fd06c154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd06affd640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd06c0d2350 0x7fd06c0ca940 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd0600098c0 tx=0x7fd060004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd053fff640 1 -- 172.21.15.27:0/1294777529 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd06003e040 con 0x7fd06c0d2350 2023-12-19T18:21:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd053fff640 1 -- 172.21.15.27:0/1294777529 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd060039c10 con 0x7fd06c0d2350 2023-12-19T18:21:12.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd053fff640 1 -- 172.21.15.27:0/1294777529 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd060030e30 con 0x7fd06c0d2350 2023-12-19T18:21:12.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.599+0000 7fd070abe640 1 -- 172.21.15.27:0/1294777529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd06c0c74a0 con 0x7fd06c0d2350 2023-12-19T18:21:12.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.600+0000 7fd070abe640 1 -- 172.21.15.27:0/1294777529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd06c0c7a60 con 0x7fd06c0d2350 2023-12-19T18:21:12.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.601+0000 7fd070abe640 1 -- 172.21.15.27:0/1294777529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd06c066f80 con 0x7fd06c0d2350 2023-12-19T18:21:12.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.602+0000 7fd053fff640 1 -- 172.21.15.27:0/1294777529 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd060043390 con 0x7fd06c0d2350 2023-12-19T18:21:12.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.603+0000 7fd053fff640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd05806a460 0x7fd05806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:12.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.603+0000 7fd053fff640 1 -- 172.21.15.27:0/1294777529 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0600aaaf0 con 0x7fd06c0d2350 2023-12-19T18:21:12.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.604+0000 7fd06b7fe640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd05806a460 0x7fd05806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:12.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.605+0000 7fd06b7fe640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd05806a460 0x7fd05806c920 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd05c005c00 tx=0x7fd05c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:12.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.608+0000 7fd053fff640 1 -- 172.21.15.27:0/1294777529 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd0600761f0 con 0x7fd06c0d2350 2023-12-19T18:21:12.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.803+0000 7fd070abe640 1 -- 172.21.15.27:0/1294777529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd06c04e840 con 0x7fd06c0d2350 2023-12-19T18:21:12.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.806+0000 7fd053fff640 1 -- 172.21.15.27:0/1294777529 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v117) v1 ==== 74+0+16116 (secure 0 0 0) 0x7fd060079810 con 0x7fd06c0d2350 2023-12-19T18:21:12.807 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:12.807 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":117,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:21:09.285480+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:21:06.258281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"116","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T18:21:07.308887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:21:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.809+0000 7fd070abe640 1 -- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd05806a460 msgr2=0x7fd05806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.809+0000 7fd070abe640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd05806a460 0x7fd05806c920 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fd05c005c00 tx=0x7fd05c009040 comp rx=0 tx=0).stop 2023-12-19T18:21:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.809+0000 7fd070abe640 1 -- 172.21.15.27:0/1294777529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd06c0d2350 msgr2=0x7fd06c0ca940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.809+0000 7fd070abe640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd06c0d2350 0x7fd06c0ca940 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fd0600098c0 tx=0x7fd060004730 comp rx=0 tx=0).stop 2023-12-19T18:21:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.810+0000 7fd070abe640 1 -- 172.21.15.27:0/1294777529 shutdown_connections 2023-12-19T18:21:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.810+0000 7fd070abe640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd05806a460 0x7fd05806c920 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.810+0000 7fd070abe640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd06c1604d0 0x7fd06c154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.810+0000 7fd070abe640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd06c0d2350 0x7fd06c0ca940 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.810+0000 7fd070abe640 1 --2- 172.21.15.27:0/1294777529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd06c0b4ef0 0x7fd06c0c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:12.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.810+0000 7fd070abe640 1 -- 172.21.15.27:0/1294777529 >> 172.21.15.27:0/1294777529 conn(0x7fd06c0cf7e0 msgr2=0x7fd06c15d530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:12.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.810+0000 7fd070abe640 1 -- 172.21.15.27:0/1294777529 shutdown_connections 2023-12-19T18:21:12.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:12.810+0000 7fd070abe640 1 -- 172.21.15.27:0/1294777529 wait complete. 2023-12-19T18:21:12.829 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:21:12.829 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:21:13.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.060+0000 7fd6782af640 1 Processor -- start 2023-12-19T18:21:13.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.060+0000 7fd6782af640 1 -- start start 2023-12-19T18:21:13.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.062+0000 7fd6782af640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd670159c40 0x7fd67006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:13.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.062+0000 7fd6782af640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd67006c880 0x7fd67006ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:13.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.062+0000 7fd6782af640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd67015a020 0x7fd6700cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:13.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.062+0000 7fd6782af640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd67015b180 con 0x7fd67015a020 2023-12-19T18:21:13.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.062+0000 7fd6782af640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6700ca850 con 0x7fd670159c40 2023-12-19T18:21:13.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.062+0000 7fd6782af640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd67015a570 con 0x7fd67006c880 2023-12-19T18:21:13.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.062+0000 7fd67684c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd67015a020 0x7fd6700cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:13.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.062+0000 7fd67684c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd67015a020 0x7fd6700cd800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35180/0 (socket says 172.21.15.27:35180) 2023-12-19T18:21:13.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.062+0000 7fd67684c640 1 -- 172.21.15.27:0/130032081 learned_addr learned my addr 172.21.15.27:0/130032081 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:13.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.063+0000 7fd67584a640 1 --2- 172.21.15.27:0/130032081 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd67006c880 0x7fd67006ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:13.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.063+0000 7fd67604b640 1 --2- 172.21.15.27:0/130032081 >> v2:172.21.15.192:3300/0 conn(0x7fd670159c40 0x7fd67006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:13.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.063+0000 7fd67684c640 1 -- 172.21.15.27:0/130032081 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd67006c880 msgr2=0x7fd67006ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:13.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.063+0000 7fd67684c640 1 --2- 172.21.15.27:0/130032081 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd67006c880 0x7fd67006ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.063+0000 7fd67684c640 1 -- 172.21.15.27:0/130032081 >> v2:172.21.15.192:3300/0 conn(0x7fd670159c40 msgr2=0x7fd67006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:13.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.063+0000 7fd67684c640 1 --2- 172.21.15.27:0/130032081 >> v2:172.21.15.192:3300/0 conn(0x7fd670159c40 0x7fd67006b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.063+0000 7fd67684c640 1 -- 172.21.15.27:0/130032081 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd67015a850 con 0x7fd67015a020 2023-12-19T18:21:13.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.063+0000 7fd67604b640 1 --2- 172.21.15.27:0/130032081 >> v2:172.21.15.192:3300/0 conn(0x7fd670159c40 0x7fd67006b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:13.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.064+0000 7fd67684c640 1 --2- 172.21.15.27:0/130032081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd67015a020 0x7fd6700cd800 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd67006d6e0 tx=0x7fd66802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5772316f364fb907 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:13.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.065+0000 7fd675049640 1 -- 172.21.15.27:0/130032081 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66802ca70 con 0x7fd67015a020 2023-12-19T18:21:13.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.065+0000 7fd675049640 1 -- 172.21.15.27:0/130032081 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd66802cbd0 con 0x7fd67015a020 2023-12-19T18:21:13.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.065+0000 7fd675049640 1 -- 172.21.15.27:0/130032081 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6680343f0 con 0x7fd67015a020 2023-12-19T18:21:13.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.065+0000 7fd6782af640 1 -- 172.21.15.27:0/130032081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd67015a020 msgr2=0x7fd6700cd800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:13.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.065+0000 7fd6782af640 1 --2- 172.21.15.27:0/130032081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd67015a020 0x7fd6700cd800 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7fd67006d6e0 tx=0x7fd66802bed0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.066+0000 7fd6782af640 1 -- 172.21.15.27:0/130032081 shutdown_connections 2023-12-19T18:21:13.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.066+0000 7fd6782af640 1 --2- 172.21.15.27:0/130032081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd67015a020 0x7fd6700cd800 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.066+0000 7fd6782af640 1 --2- 172.21.15.27:0/130032081 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd67006c880 0x7fd67006ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.066+0000 7fd6782af640 1 --2- 172.21.15.27:0/130032081 >> v2:172.21.15.192:3300/0 conn(0x7fd670159c40 0x7fd67006b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.066+0000 7fd6782af640 1 -- 172.21.15.27:0/130032081 >> 172.21.15.27:0/130032081 conn(0x7fd67006a330 msgr2=0x7fd6700b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:13.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.066+0000 7fd6782af640 1 -- 172.21.15.27:0/130032081 shutdown_connections 2023-12-19T18:21:13.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.066+0000 7fd6782af640 1 -- 172.21.15.27:0/130032081 wait complete. 2023-12-19T18:21:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.068+0000 7fd6782af640 1 Processor -- start 2023-12-19T18:21:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.068+0000 7fd6782af640 1 -- start start 2023-12-19T18:21:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.069+0000 7fd6782af640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd67006c880 0x7fd670157b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.069+0000 7fd6782af640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd670159c40 0x7fd6700aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.069+0000 7fd6782af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd67015a020 0x7fd6700b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:13.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.069+0000 7fd6782af640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700b45d0 con 0x7fd67015a020 2023-12-19T18:21:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.069+0000 7fd6782af640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700b2290 con 0x7fd67006c880 2023-12-19T18:21:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.069+0000 7fd6782af640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6700afef0 con 0x7fd670159c40 2023-12-19T18:21:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.069+0000 7fd67684c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd67015a020 0x7fd6700b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.069+0000 7fd67684c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd67015a020 0x7fd6700b4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35182/0 (socket says 172.21.15.27:35182) 2023-12-19T18:21:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.069+0000 7fd67684c640 1 -- 172.21.15.27:0/1463873135 learned_addr learned my addr 172.21.15.27:0/1463873135 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.069+0000 7fd67604b640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd67006c880 0x7fd670157b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.069+0000 7fd67584a640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd670159c40 0x7fd6700aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.070+0000 7fd67684c640 1 -- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd670159c40 msgr2=0x7fd6700aef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.070+0000 7fd67684c640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd670159c40 0x7fd6700aef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.070+0000 7fd67684c640 1 -- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd67006c880 msgr2=0x7fd670157b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.070+0000 7fd67684c640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd67006c880 0x7fd670157b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.070+0000 7fd67684c640 1 -- 172.21.15.27:0/1463873135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd668009b30 con 0x7fd67015a020 2023-12-19T18:21:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.070+0000 7fd67684c640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd67015a020 0x7fd6700b4090 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd668036040 tx=0x7fd668004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.070+0000 7fd666ffd640 1 -- 172.21.15.27:0/1463873135 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668040670 con 0x7fd67015a020 2023-12-19T18:21:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.070+0000 7fd666ffd640 1 -- 172.21.15.27:0/1463873135 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6680407f0 con 0x7fd67015a020 2023-12-19T18:21:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.070+0000 7fd666ffd640 1 -- 172.21.15.27:0/1463873135 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd668040ba0 con 0x7fd67015a020 2023-12-19T18:21:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.070+0000 7fd6782af640 1 -- 172.21.15.27:0/1463873135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6700af650 con 0x7fd67015a020 2023-12-19T18:21:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.071+0000 7fd6782af640 1 -- 172.21.15.27:0/1463873135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6700afc10 con 0x7fd67015a020 2023-12-19T18:21:13.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.072+0000 7fd6782af640 1 -- 172.21.15.27:0/1463873135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6700b2400 con 0x7fd67015a020 2023-12-19T18:21:13.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.073+0000 7fd666ffd640 1 -- 172.21.15.27:0/1463873135 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd668040d00 con 0x7fd67015a020 2023-12-19T18:21:13.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.074+0000 7fd666ffd640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd65406a460 0x7fd65406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.074+0000 7fd666ffd640 1 -- 172.21.15.27:0/1463873135 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6680ab750 con 0x7fd67015a020 2023-12-19T18:21:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.074+0000 7fd67604b640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd65406a460 0x7fd65406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.075+0000 7fd67604b640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd65406a460 0x7fd65406c920 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00a910 tx=0x7fd66c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:13.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.078+0000 7fd666ffd640 1 -- 172.21.15.27:0/1463873135 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd668076d30 con 0x7fd67015a020 2023-12-19T18:21:13.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.267+0000 7fd6782af640 1 -- 172.21.15.27:0/1463873135 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd67015ae10 con 0x7fd65406a460 2023-12-19T18:21:13.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.293+0000 7fd666ffd640 1 -- 172.21.15.27:0/1463873135 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282902 (secure 0 0 0) 0x7fd66c00a680 con 0x7fd65406a460 2023-12-19T18:21:13.298 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:13.299 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:21:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.300+0000 7fd6782af640 1 -- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd65406a460 msgr2=0x7fd65406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.300+0000 7fd6782af640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd65406a460 0x7fd65406c920 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00a910 tx=0x7fd66c008040 comp rx=0 tx=0).stop 2023-12-19T18:21:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.300+0000 7fd6782af640 1 -- 172.21.15.27:0/1463873135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd67015a020 msgr2=0x7fd6700b4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.300+0000 7fd6782af640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd67015a020 0x7fd6700b4090 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fd668036040 tx=0x7fd668004a40 comp rx=0 tx=0).stop 2023-12-19T18:21:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.300+0000 7fd6782af640 1 -- 172.21.15.27:0/1463873135 shutdown_connections 2023-12-19T18:21:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.300+0000 7fd6782af640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd65406a460 0x7fd65406c920 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.300+0000 7fd6782af640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd67015a020 0x7fd6700b4090 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.300+0000 7fd6782af640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd670159c40 0x7fd6700aef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.300+0000 7fd6782af640 1 --2- 172.21.15.27:0/1463873135 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd67006c880 0x7fd670157b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.300+0000 7fd6782af640 1 -- 172.21.15.27:0/1463873135 >> 172.21.15.27:0/1463873135 conn(0x7fd67006a330 msgr2=0x7fd67015b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.300+0000 7fd6782af640 1 -- 172.21.15.27:0/1463873135 shutdown_connections 2023-12-19T18:21:13.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.300+0000 7fd6782af640 1 -- 172.21.15.27:0/1463873135 wait complete. 2023-12-19T18:21:13.323 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2023-12-19T18:21:11.839801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6149,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556350,"statfs":{"total":1600000000,"available":1593703010,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.078714"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.316093+0000","last_change":"2023-12-19T18:21:08.315962+0000","last_active":"2023-12-19T18:21:08.316093+0000","last_peered":"2023-12-19T18:21:08.316093+0000","last_clean":"2023-12-19T18:21:08.316093+0000","last_became_active":"2023-12-19T18:21:08.315687+0000","last_became_peered":"2023-12-19T18:21:08.315687+0000","last_unstale":"2023-12-19T18:21:08.316093+0000","last_undegraded":"2023-12-19T18:21:08.316093+0000","last_fullsized":"2023-12-19T18:21:08.316093+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:12.815660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.309059+0000","last_change":"2023-12-19T18:21:08.307792+0000","last_active":"2023-12-19T18:21:08.309059+0000","last_peered":"2023-12-19T18:21:08.309059+0000","last_clean":"2023-12-19T18:21:08.309059+0000","last_became_active":"2023-12-19T18:21:08.307119+0000","last_became_peered":"2023-12-19T18:21:08.307119+0000","last_unstale":"2023-12-19T18:21:08.309059+0000","last_undegraded":"2023-12-19T18:21:08.309059+0000","last_fullsized":"2023-12-19T18:21:08.309059+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:34.078946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.308268+0000","last_change":"2023-12-19T18:21:08.307769+0000","last_active":"2023-12-19T18:21:08.308268+0000","last_peered":"2023-12-19T18:21:08.308268+0000","last_clean":"2023-12-19T18:21:08.308268+0000","last_became_active":"2023-12-19T18:21:08.307451+0000","last_became_peered":"2023-12-19T18:21:08.307451+0000","last_unstale":"2023-12-19T18:21:08.308268+0000","last_undegraded":"2023-12-19T18:21:08.308268+0000","last_fullsized":"2023-12-19T18:21:08.308268+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:09.791993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.307335+0000","last_change":"2023-12-19T18:21:08.304846+0000","last_active":"2023-12-19T18:21:08.307335+0000","last_peered":"2023-12-19T18:21:08.307335+0000","last_clean":"2023-12-19T18:21:08.307335+0000","last_became_active":"2023-12-19T18:21:08.304222+0000","last_became_peered":"2023-12-19T18:21:08.304222+0000","last_unstale":"2023-12-19T18:21:08.307335+0000","last_undegraded":"2023-12-19T18:21:08.307335+0000","last_fullsized":"2023-12-19T18:21:08.307335+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:20.930922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.318137+0000","last_change":"2023-12-19T18:21:08.317815+0000","last_active":"2023-12-19T18:21:08.318137+0000","last_peered":"2023-12-19T18:21:08.318137+0000","last_clean":"2023-12-19T18:21:08.318137+0000","last_became_active":"2023-12-19T18:21:08.317572+0000","last_became_peered":"2023-12-19T18:21:08.317572+0000","last_unstale":"2023-12-19T18:21:08.318137+0000","last_undegraded":"2023-12-19T18:21:08.318137+0000","last_fullsized":"2023-12-19T18:21:08.318137+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:08:36.344800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.320955+0000","last_change":"2023-12-19T18:21:08.320559+0000","last_active":"2023-12-19T18:21:08.320955+0000","last_peered":"2023-12-19T18:21:08.320955+0000","last_clean":"2023-12-19T18:21:08.320955+0000","last_became_active":"2023-12-19T18:21:08.320189+0000","last_became_peered":"2023-12-19T18:21:08.320189+0000","last_unstale":"2023-12-19T18:21:08.320955+0000","last_undegraded":"2023-12-19T18:21:08.320955+0000","last_fullsized":"2023-12-19T18:21:08.320955+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:35:32.513844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.306160+0000","last_change":"2023-12-19T18:21:08.305650+0000","last_active":"2023-12-19T18:21:08.306160+0000","last_peered":"2023-12-19T18:21:08.306160+0000","last_clean":"2023-12-19T18:21:08.306160+0000","last_became_active":"2023-12-19T18:21:08.304970+0000","last_became_peered":"2023-12-19T18:21:08.304970+0000","last_unstale":"2023-12-19T18:21:08.306160+0000","last_undegraded":"2023-12-19T18:21:08.306160+0000","last_fullsized":"2023-12-19T18:21:08.306160+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:03:03.110026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.319497+0000","last_change":"2023-12-19T18:21:08.318243+0000","last_active":"2023-12-19T18:21:08.319497+0000","last_peered":"2023-12-19T18:21:08.319497+0000","last_clean":"2023-12-19T18:21:08.319497+0000","last_became_active":"2023-12-19T18:21:08.317599+0000","last_became_peered":"2023-12-19T18:21:08.317599+0000","last_unstale":"2023-12-19T18:21:08.319497+0000","last_undegraded":"2023-12-19T18:21:08.319497+0000","last_fullsized":"2023-12-19T18:21:08.319497+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:39:56.350341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.307160+0000","last_change":"2023-12-19T18:21:08.306932+0000","last_active":"2023-12-19T18:21:08.307160+0000","last_peered":"2023-12-19T18:21:08.307160+0000","last_clean":"2023-12-19T18:21:08.307160+0000","last_became_active":"2023-12-19T18:21:08.306706+0000","last_became_peered":"2023-12-19T18:21:08.306706+0000","last_unstale":"2023-12-19T18:21:08.307160+0000","last_undegraded":"2023-12-19T18:21:08.307160+0000","last_fullsized":"2023-12-19T18:21:08.307160+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:00.134574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.315905+0000","last_change":"2023-12-19T18:21:08.315830+0000","last_active":"2023-12-19T18:21:08.315905+0000","last_peered":"2023-12-19T18:21:08.315905+0000","last_clean":"2023-12-19T18:21:08.315905+0000","last_became_active":"2023-12-19T18:21:08.315594+0000","last_became_peered":"2023-12-19T18:21:08.315594+0000","last_unstale":"2023-12-19T18:21:08.315905+0000","last_undegraded":"2023-12-19T18:21:08.315905+0000","last_fullsized":"2023-12-19T18:21:08.315905+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:01.407622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.322218+0000","last_change":"2023-12-19T18:21:08.321948+0000","last_active":"2023-12-19T18:21:08.322218+0000","last_peered":"2023-12-19T18:21:08.322218+0000","last_clean":"2023-12-19T18:21:08.322218+0000","last_became_active":"2023-12-19T18:21:08.318810+0000","last_became_peered":"2023-12-19T18:21:08.318810+0000","last_unstale":"2023-12-19T18:21:08.322218+0000","last_undegraded":"2023-12-19T18:21:08.322218+0000","last_fullsized":"2023-12-19T18:21:08.322218+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:47.329029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.317957+0000","last_change":"2023-12-19T18:21:08.317788+0000","last_active":"2023-12-19T18:21:08.317957+0000","last_peered":"2023-12-19T18:21:08.317957+0000","last_clean":"2023-12-19T18:21:08.317957+0000","last_became_active":"2023-12-19T18:21:08.317551+0000","last_became_peered":"2023-12-19T18:21:08.317551+0000","last_unstale":"2023-12-19T18:21:08.317957+0000","last_undegraded":"2023-12-19T18:21:08.317957+0000","last_fullsized":"2023-12-19T18:21:08.317957+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:01.378947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.308305+0000","last_change":"2023-12-19T18:21:08.308162+0000","last_active":"2023-12-19T18:21:08.308305+0000","last_peered":"2023-12-19T18:21:08.308305+0000","last_clean":"2023-12-19T18:21:08.308305+0000","last_became_active":"2023-12-19T18:21:08.304144+0000","last_became_peered":"2023-12-19T18:21:08.304144+0000","last_unstale":"2023-12-19T18:21:08.308305+0000","last_undegraded":"2023-12-19T18:21:08.308305+0000","last_fullsized":"2023-12-19T18:21:08.308305+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:32:20.010608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.321004+0000","last_change":"2023-12-19T18:21:08.320714+0000","last_active":"2023-12-19T18:21:08.321004+0000","last_peered":"2023-12-19T18:21:08.321004+0000","last_clean":"2023-12-19T18:21:08.321004+0000","last_became_active":"2023-12-19T18:21:08.317148+0000","last_became_peered":"2023-12-19T18:21:08.317148+0000","last_unstale":"2023-12-19T18:21:08.321004+0000","last_undegraded":"2023-12-19T18:21:08.321004+0000","last_fullsized":"2023-12-19T18:21:08.321004+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:04.632373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.303406+0000","last_change":"2023-12-19T18:21:08.302966+0000","last_active":"2023-12-19T18:21:08.303406+0000","last_peered":"2023-12-19T18:21:08.303406+0000","last_clean":"2023-12-19T18:21:08.303406+0000","last_became_active":"2023-12-19T18:21:08.302378+0000","last_became_peered":"2023-12-19T18:21:08.302378+0000","last_unstale":"2023-12-19T18:21:08.303406+0000","last_undegraded":"2023-12-19T18:21:08.303406+0000","last_fullsized":"2023-12-19T18:21:08.303406+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:38:31.365048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.314650+0000","last_change":"2023-12-19T18:21:08.314536+0000","last_active":"2023-12-19T18:21:08.314650+0000","last_peered":"2023-12-19T18:21:08.314650+0000","last_clean":"2023-12-19T18:21:08.314650+0000","last_became_active":"2023-12-19T18:21:08.314264+0000","last_became_peered":"2023-12-19T18:21:08.314264+0000","last_unstale":"2023-12-19T18:21:08.314650+0000","last_undegraded":"2023-12-19T18:21:08.314650+0000","last_fullsized":"2023-12-19T18:21:08.314650+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:37.658754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.314344+0000","last_change":"2023-12-19T18:21:08.314264+0000","last_active":"2023-12-19T18:21:08.314344+0000","last_peered":"2023-12-19T18:21:08.314344+0000","last_clean":"2023-12-19T18:21:08.314344+0000","last_became_active":"2023-12-19T18:21:08.314020+0000","last_became_peered":"2023-12-19T18:21:08.314020+0000","last_unstale":"2023-12-19T18:21:08.314344+0000","last_undegraded":"2023-12-19T18:21:08.314344+0000","last_fullsized":"2023-12-19T18:21:08.314344+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:14:53.195396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.321029+0000","last_change":"2023-12-19T18:21:08.314404+0000","last_active":"2023-12-19T18:21:08.321029+0000","last_peered":"2023-12-19T18:21:08.321029+0000","last_clean":"2023-12-19T18:21:08.321029+0000","last_became_active":"2023-12-19T18:21:08.313869+0000","last_became_peered":"2023-12-19T18:21:08.313869+0000","last_unstale":"2023-12-19T18:21:08.321029+0000","last_undegraded":"2023-12-19T18:21:08.321029+0000","last_fullsized":"2023-12-19T18:21:08.321029+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:04.632373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.324202+0000","last_change":"2023-12-19T18:21:08.321406+0000","last_active":"2023-12-19T18:21:08.324202+0000","last_peered":"2023-12-19T18:21:08.324202+0000","last_clean":"2023-12-19T18:21:08.324202+0000","last_became_active":"2023-12-19T18:21:08.320496+0000","last_became_peered":"2023-12-19T18:21:08.320496+0000","last_unstale":"2023-12-19T18:21:08.324202+0000","last_undegraded":"2023-12-19T18:21:08.324202+0000","last_fullsized":"2023-12-19T18:21:08.324202+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:38:31.365048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.325068+0000","last_change":"2023-12-19T18:21:08.324976+0000","last_active":"2023-12-19T18:21:08.325068+0000","last_peered":"2023-12-19T18:21:08.325068+0000","last_clean":"2023-12-19T18:21:08.325068+0000","last_became_active":"2023-12-19T18:21:08.322809+0000","last_became_peered":"2023-12-19T18:21:08.322809+0000","last_unstale":"2023-12-19T18:21:08.325068+0000","last_undegraded":"2023-12-19T18:21:08.325068+0000","last_fullsized":"2023-12-19T18:21:08.325068+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:14:03.228122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.319788+0000","last_change":"2023-12-19T18:21:08.318225+0000","last_active":"2023-12-19T18:21:08.319788+0000","last_peered":"2023-12-19T18:21:08.319788+0000","last_clean":"2023-12-19T18:21:08.319788+0000","last_became_active":"2023-12-19T18:21:08.313899+0000","last_became_peered":"2023-12-19T18:21:08.313899+0000","last_unstale":"2023-12-19T18:21:08.319788+0000","last_undegraded":"2023-12-19T18:21:08.319788+0000","last_fullsized":"2023-12-19T18:21:08.319788+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:17:45.304080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.318688+0000","last_change":"2023-12-19T18:21:08.317509+0000","last_active":"2023-12-19T18:21:08.318688+0000","last_peered":"2023-12-19T18:21:08.318688+0000","last_clean":"2023-12-19T18:21:08.318688+0000","last_became_active":"2023-12-19T18:21:08.314334+0000","last_became_peered":"2023-12-19T18:21:08.314334+0000","last_unstale":"2023-12-19T18:21:08.318688+0000","last_undegraded":"2023-12-19T18:21:08.318688+0000","last_fullsized":"2023-12-19T18:21:08.318688+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:28:44.357084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.319515+0000","last_change":"2023-12-19T18:21:08.318390+0000","last_active":"2023-12-19T18:21:08.319515+0000","last_peered":"2023-12-19T18:21:08.319515+0000","last_clean":"2023-12-19T18:21:08.319515+0000","last_became_active":"2023-12-19T18:21:08.318011+0000","last_became_peered":"2023-12-19T18:21:08.318011+0000","last_unstale":"2023-12-19T18:21:08.319515+0000","last_undegraded":"2023-12-19T18:21:08.319515+0000","last_fullsized":"2023-12-19T18:21:08.319515+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:21:20.647321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.321270+0000","last_change":"2023-12-19T18:21:08.317824+0000","last_active":"2023-12-19T18:21:08.321270+0000","last_peered":"2023-12-19T18:21:08.321270+0000","last_clean":"2023-12-19T18:21:08.321270+0000","last_became_active":"2023-12-19T18:21:08.316205+0000","last_became_peered":"2023-12-19T18:21:08.316205+0000","last_unstale":"2023-12-19T18:21:08.321270+0000","last_undegraded":"2023-12-19T18:21:08.321270+0000","last_fullsized":"2023-12-19T18:21:08.321270+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:28:44.357084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.321297+0000","last_change":"2023-12-19T18:21:08.318883+0000","last_active":"2023-12-19T18:21:08.321297+0000","last_peered":"2023-12-19T18:21:08.321297+0000","last_clean":"2023-12-19T18:21:08.321297+0000","last_became_active":"2023-12-19T18:21:08.318645+0000","last_became_peered":"2023-12-19T18:21:08.318645+0000","last_unstale":"2023-12-19T18:21:08.321297+0000","last_undegraded":"2023-12-19T18:21:08.321297+0000","last_fullsized":"2023-12-19T18:21:08.321297+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:34:03.699275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.309909+0000","last_change":"2023-12-19T18:21:08.309412+0000","last_active":"2023-12-19T18:21:08.309909+0000","last_peered":"2023-12-19T18:21:08.309909+0000","last_clean":"2023-12-19T18:21:08.309909+0000","last_became_active":"2023-12-19T18:21:08.309195+0000","last_became_peered":"2023-12-19T18:21:08.309195+0000","last_unstale":"2023-12-19T18:21:08.309909+0000","last_undegraded":"2023-12-19T18:21:08.309909+0000","last_fullsized":"2023-12-19T18:21:08.309909+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:29:36.782413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.295770+0000","last_change":"2023-12-19T18:21:08.295667+0000","last_active":"2023-12-19T18:21:08.295770+0000","last_peered":"2023-12-19T18:21:08.295770+0000","last_clean":"2023-12-19T18:21:08.295770+0000","last_became_active":"2023-12-19T18:21:08.294934+0000","last_became_peered":"2023-12-19T18:21:08.294934+0000","last_unstale":"2023-12-19T18:21:08.295770+0000","last_undegraded":"2023-12-19T18:21:08.295770+0000","last_fullsized":"2023-12-19T18:21:08.295770+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:00.470255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.299903+0000","last_change":"2023-12-19T18:21:08.299764+0000","last_active":"2023-12-19T18:21:08.299903+0000","last_peered":"2023-12-19T18:21:08.299903+0000","last_clean":"2023-12-19T18:21:08.299903+0000","last_became_active":"2023-12-19T18:21:08.297643+0000","last_became_peered":"2023-12-19T18:21:08.297643+0000","last_unstale":"2023-12-19T18:21:08.299903+0000","last_undegraded":"2023-12-19T18:21:08.299903+0000","last_fullsized":"2023-12-19T18:21:08.299903+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:22:02.320818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.300140+0000","last_change":"2023-12-19T18:21:08.299950+0000","last_active":"2023-12-19T18:21:08.300140+0000","last_peered":"2023-12-19T18:21:08.300140+0000","last_clean":"2023-12-19T18:21:08.300140+0000","last_became_active":"2023-12-19T18:21:08.298315+0000","last_became_peered":"2023-12-19T18:21:08.298315+0000","last_unstale":"2023-12-19T18:21:08.300140+0000","last_undegraded":"2023-12-19T18:21:08.300140+0000","last_fullsized":"2023-12-19T18:21:08.300140+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:00.301926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.324812+0000","last_change":"2023-12-19T18:21:08.323087+0000","last_active":"2023-12-19T18:21:08.324812+0000","last_peered":"2023-12-19T18:21:08.324812+0000","last_clean":"2023-12-19T18:21:08.324812+0000","last_became_active":"2023-12-19T18:21:08.322460+0000","last_became_peered":"2023-12-19T18:21:08.322460+0000","last_unstale":"2023-12-19T18:21:08.324812+0000","last_undegraded":"2023-12-19T18:21:08.324812+0000","last_fullsized":"2023-12-19T18:21:08.324812+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:34.078946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.320937+0000","last_change":"2023-12-19T18:21:08.320405+0000","last_active":"2023-12-19T18:21:08.320937+0000","last_peered":"2023-12-19T18:21:08.320937+0000","last_clean":"2023-12-19T18:21:08.320937+0000","last_became_active":"2023-12-19T18:21:08.319880+0000","last_became_peered":"2023-12-19T18:21:08.319880+0000","last_unstale":"2023-12-19T18:21:08.320937+0000","last_undegraded":"2023-12-19T18:21:08.320937+0000","last_fullsized":"2023-12-19T18:21:08.320937+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:06.368856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.313009+0000","last_change":"2023-12-19T18:21:08.311544+0000","last_active":"2023-12-19T18:21:08.313009+0000","last_peered":"2023-12-19T18:21:08.313009+0000","last_clean":"2023-12-19T18:21:08.313009+0000","last_became_active":"2023-12-19T18:21:08.311201+0000","last_became_peered":"2023-12-19T18:21:08.311201+0000","last_unstale":"2023-12-19T18:21:08.313009+0000","last_undegraded":"2023-12-19T18:21:08.313009+0000","last_fullsized":"2023-12-19T18:21:08.313009+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:23.066899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.303824+0000","last_change":"2023-12-19T18:21:08.303092+0000","last_active":"2023-12-19T18:21:08.303824+0000","last_peered":"2023-12-19T18:21:08.303824+0000","last_clean":"2023-12-19T18:21:08.303824+0000","last_became_active":"2023-12-19T18:21:08.302857+0000","last_became_peered":"2023-12-19T18:21:08.302857+0000","last_unstale":"2023-12-19T18:21:08.303824+0000","last_undegraded":"2023-12-19T18:21:08.303824+0000","last_fullsized":"2023-12-19T18:21:08.303824+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:50.191686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"116'4","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.326650+0000","last_change":"2023-12-19T18:21:07.806330+0000","last_active":"2023-12-19T18:21:08.326650+0000","last_peered":"2023-12-19T18:21:08.326650+0000","last_clean":"2023-12-19T18:21:08.326650+0000","last_became_active":"2023-12-19T18:21:07.806113+0000","last_became_peered":"2023-12-19T18:21:07.806113+0000","last_unstale":"2023-12-19T18:21:08.326650+0000","last_undegraded":"2023-12-19T18:21:08.326650+0000","last_fullsized":"2023-12-19T18:21:08.326650+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:48:25.943021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.306443+0000","last_change":"2023-12-19T18:21:08.306367+0000","last_active":"2023-12-19T18:21:08.306443+0000","last_peered":"2023-12-19T18:21:08.306443+0000","last_clean":"2023-12-19T18:21:08.306443+0000","last_became_active":"2023-12-19T18:21:08.303918+0000","last_became_peered":"2023-12-19T18:21:08.303918+0000","last_unstale":"2023-12-19T18:21:08.306443+0000","last_undegraded":"2023-12-19T18:21:08.306443+0000","last_fullsized":"2023-12-19T18:21:08.306443+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:38.893277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.313224+0000","last_change":"2023-12-19T18:21:07.796271+0000","last_active":"2023-12-19T18:21:08.313224+0000","last_peered":"2023-12-19T18:21:08.313224+0000","last_clean":"2023-12-19T18:21:08.313224+0000","last_became_active":"2023-12-19T18:21:07.795997+0000","last_became_peered":"2023-12-19T18:21:07.795997+0000","last_unstale":"2023-12-19T18:21:08.313224+0000","last_undegraded":"2023-12-19T18:21:08.313224+0000","last_fullsized":"2023-12-19T18:21:08.313224+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:33.067212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.305957+0000","last_change":"2023-12-19T18:21:08.305888+0000","last_active":"2023-12-19T18:21:08.305957+0000","last_peered":"2023-12-19T18:21:08.305957+0000","last_clean":"2023-12-19T18:21:08.305957+0000","last_became_active":"2023-12-19T18:21:08.305418+0000","last_became_peered":"2023-12-19T18:21:08.305418+0000","last_unstale":"2023-12-19T18:21:08.305957+0000","last_undegraded":"2023-12-19T18:21:08.305957+0000","last_fullsized":"2023-12-19T18:21:08.305957+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:39:13.992641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.323742+0000","last_change":"2023-12-19T18:21:07.806193+0000","last_active":"2023-12-19T18:21:08.323742+0000","last_peered":"2023-12-19T18:21:08.323742+0000","last_clean":"2023-12-19T18:21:08.323742+0000","last_became_active":"2023-12-19T18:21:07.805886+0000","last_became_peered":"2023-12-19T18:21:07.805886+0000","last_unstale":"2023-12-19T18:21:08.323742+0000","last_undegraded":"2023-12-19T18:21:08.323742+0000","last_fullsized":"2023-12-19T18:21:08.323742+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:54:01.049760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.323427+0000","last_change":"2023-12-19T18:21:08.323372+0000","last_active":"2023-12-19T18:21:08.323427+0000","last_peered":"2023-12-19T18:21:08.323427+0000","last_clean":"2023-12-19T18:21:08.323427+0000","last_became_active":"2023-12-19T18:21:08.322260+0000","last_became_peered":"2023-12-19T18:21:08.322260+0000","last_unstale":"2023-12-19T18:21:08.323427+0000","last_undegraded":"2023-12-19T18:21:08.323427+0000","last_fullsized":"2023-12-19T18:21:08.323427+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:54:16.721327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.325642+0000","last_change":"2023-12-19T18:21:07.801223+0000","last_active":"2023-12-19T18:21:08.325642+0000","last_peered":"2023-12-19T18:21:08.325642+0000","last_clean":"2023-12-19T18:21:08.325642+0000","last_became_active":"2023-12-19T18:21:07.800988+0000","last_became_peered":"2023-12-19T18:21:07.800988+0000","last_unstale":"2023-12-19T18:21:08.325642+0000","last_undegraded":"2023-12-19T18:21:08.325642+0000","last_fullsized":"2023-12-19T18:21:08.325642+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:08.780259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.320712+0000","last_change":"2023-12-19T18:21:08.320228+0000","last_active":"2023-12-19T18:21:08.320712+0000","last_peered":"2023-12-19T18:21:08.320712+0000","last_clean":"2023-12-19T18:21:08.320712+0000","last_became_active":"2023-12-19T18:21:08.320004+0000","last_became_peered":"2023-12-19T18:21:08.320004+0000","last_unstale":"2023-12-19T18:21:08.320712+0000","last_undegraded":"2023-12-19T18:21:08.320712+0000","last_fullsized":"2023-12-19T18:21:08.320712+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:35:08.910728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.311670+0000","last_change":"2023-12-19T18:21:07.800300+0000","last_active":"2023-12-19T18:21:08.311670+0000","last_peered":"2023-12-19T18:21:08.311670+0000","last_clean":"2023-12-19T18:21:08.311670+0000","last_became_active":"2023-12-19T18:21:07.794856+0000","last_became_peered":"2023-12-19T18:21:07.794856+0000","last_unstale":"2023-12-19T18:21:08.311670+0000","last_undegraded":"2023-12-19T18:21:08.311670+0000","last_fullsized":"2023-12-19T18:21:08.311670+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:34.703586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.319370+0000","last_change":"2023-12-19T18:21:08.318474+0000","last_active":"2023-12-19T18:21:08.319370+0000","last_peered":"2023-12-19T18:21:08.319370+0000","last_clean":"2023-12-19T18:21:08.319370+0000","last_became_active":"2023-12-19T18:21:08.318213+0000","last_became_peered":"2023-12-19T18:21:08.318213+0000","last_unstale":"2023-12-19T18:21:08.319370+0000","last_undegraded":"2023-12-19T18:21:08.319370+0000","last_fullsized":"2023-12-19T18:21:08.319370+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:08:26.816978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.315261+0000","last_change":"2023-12-19T18:21:07.791887+0000","last_active":"2023-12-19T18:21:08.315261+0000","last_peered":"2023-12-19T18:21:08.315261+0000","last_clean":"2023-12-19T18:21:08.315261+0000","last_became_active":"2023-12-19T18:21:07.791173+0000","last_became_peered":"2023-12-19T18:21:07.791173+0000","last_unstale":"2023-12-19T18:21:08.315261+0000","last_undegraded":"2023-12-19T18:21:08.315261+0000","last_fullsized":"2023-12-19T18:21:08.315261+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:01:54.458894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.323506+0000","last_change":"2023-12-19T18:21:08.323464+0000","last_active":"2023-12-19T18:21:08.323506+0000","last_peered":"2023-12-19T18:21:08.323506+0000","last_clean":"2023-12-19T18:21:08.323506+0000","last_became_active":"2023-12-19T18:21:08.322597+0000","last_became_peered":"2023-12-19T18:21:08.322597+0000","last_unstale":"2023-12-19T18:21:08.323506+0000","last_undegraded":"2023-12-19T18:21:08.323506+0000","last_fullsized":"2023-12-19T18:21:08.323506+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:44.485880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.310980+0000","last_change":"2023-12-19T18:21:07.802335+0000","last_active":"2023-12-19T18:21:08.310980+0000","last_peered":"2023-12-19T18:21:08.310980+0000","last_clean":"2023-12-19T18:21:08.310980+0000","last_became_active":"2023-12-19T18:21:07.802044+0000","last_became_peered":"2023-12-19T18:21:07.802044+0000","last_unstale":"2023-12-19T18:21:08.310980+0000","last_undegraded":"2023-12-19T18:21:08.310980+0000","last_fullsized":"2023-12-19T18:21:08.310980+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:47:59.122840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.316864+0000","last_change":"2023-12-19T18:21:08.315294+0000","last_active":"2023-12-19T18:21:08.316864+0000","last_peered":"2023-12-19T18:21:08.316864+0000","last_clean":"2023-12-19T18:21:08.316864+0000","last_became_active":"2023-12-19T18:21:08.315003+0000","last_became_peered":"2023-12-19T18:21:08.315003+0000","last_unstale":"2023-12-19T18:21:08.316864+0000","last_undegraded":"2023-12-19T18:21:08.316864+0000","last_fullsized":"2023-12-19T18:21:08.316864+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:03:03.110026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.312570+0000","last_change":"2023-12-19T18:21:07.804454+0000","last_active":"2023-12-19T18:21:08.312570+0000","last_peered":"2023-12-19T18:21:08.312570+0000","last_clean":"2023-12-19T18:21:08.312570+0000","last_became_active":"2023-12-19T18:21:07.803328+0000","last_became_peered":"2023-12-19T18:21:07.803328+0000","last_unstale":"2023-12-19T18:21:08.312570+0000","last_undegraded":"2023-12-19T18:21:08.312570+0000","last_fullsized":"2023-12-19T18:21:08.312570+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:28.405924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.318670+0000","last_change":"2023-12-19T18:21:08.317326+0000","last_active":"2023-12-19T18:21:08.318670+0000","last_peered":"2023-12-19T18:21:08.318670+0000","last_clean":"2023-12-19T18:21:08.318670+0000","last_became_active":"2023-12-19T18:21:08.314030+0000","last_became_peered":"2023-12-19T18:21:08.314030+0000","last_unstale":"2023-12-19T18:21:08.318670+0000","last_undegraded":"2023-12-19T18:21:08.318670+0000","last_fullsized":"2023-12-19T18:21:08.318670+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:47.329029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.294355+0000","last_change":"2023-12-19T18:21:07.798583+0000","last_active":"2023-12-19T18:21:08.294355+0000","last_peered":"2023-12-19T18:21:08.294355+0000","last_clean":"2023-12-19T18:21:08.294355+0000","last_became_active":"2023-12-19T18:21:07.798192+0000","last_became_peered":"2023-12-19T18:21:07.798192+0000","last_unstale":"2023-12-19T18:21:08.294355+0000","last_undegraded":"2023-12-19T18:21:08.294355+0000","last_fullsized":"2023-12-19T18:21:08.294355+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:32:03.257881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.322957+0000","last_change":"2023-12-19T18:21:08.322737+0000","last_active":"2023-12-19T18:21:08.322957+0000","last_peered":"2023-12-19T18:21:08.322957+0000","last_clean":"2023-12-19T18:21:08.322957+0000","last_became_active":"2023-12-19T18:21:08.321951+0000","last_became_peered":"2023-12-19T18:21:08.321951+0000","last_unstale":"2023-12-19T18:21:08.322957+0000","last_undegraded":"2023-12-19T18:21:08.322957+0000","last_fullsized":"2023-12-19T18:21:08.322957+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:54:02.061494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.300998+0000","last_change":"2023-12-19T18:21:07.795155+0000","last_active":"2023-12-19T18:21:08.300998+0000","last_peered":"2023-12-19T18:21:08.300998+0000","last_clean":"2023-12-19T18:21:08.300998+0000","last_became_active":"2023-12-19T18:21:07.794472+0000","last_became_peered":"2023-12-19T18:21:07.794472+0000","last_unstale":"2023-12-19T18:21:08.300998+0000","last_undegraded":"2023-12-19T18:21:08.300998+0000","last_fullsized":"2023-12-19T18:21:08.300998+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:14:02.216388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.321247+0000","last_change":"2023-12-19T18:21:08.317693+0000","last_active":"2023-12-19T18:21:08.321247+0000","last_peered":"2023-12-19T18:21:08.321247+0000","last_clean":"2023-12-19T18:21:08.321247+0000","last_became_active":"2023-12-19T18:21:08.315906+0000","last_became_peered":"2023-12-19T18:21:08.315906+0000","last_unstale":"2023-12-19T18:21:08.321247+0000","last_undegraded":"2023-12-19T18:21:08.321247+0000","last_fullsized":"2023-12-19T18:21:08.321247+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:34.078946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"116'3","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.322257+0000","last_change":"2023-12-19T18:21:07.805933+0000","last_active":"2023-12-19T18:21:08.322257+0000","last_peered":"2023-12-19T18:21:08.322257+0000","last_clean":"2023-12-19T18:21:08.322257+0000","last_became_active":"2023-12-19T18:21:07.805312+0000","last_became_peered":"2023-12-19T18:21:07.805312+0000","last_unstale":"2023-12-19T18:21:08.322257+0000","last_undegraded":"2023-12-19T18:21:08.322257+0000","last_fullsized":"2023-12-19T18:21:08.322257+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:01.573086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.318590+0000","last_change":"2023-12-19T18:21:08.318537+0000","last_active":"2023-12-19T18:21:08.318590+0000","last_peered":"2023-12-19T18:21:08.318590+0000","last_clean":"2023-12-19T18:21:08.318590+0000","last_became_active":"2023-12-19T18:21:08.317766+0000","last_became_peered":"2023-12-19T18:21:08.317766+0000","last_unstale":"2023-12-19T18:21:08.318590+0000","last_undegraded":"2023-12-19T18:21:08.318590+0000","last_fullsized":"2023-12-19T18:21:08.318590+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:42:30.571517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"116'6","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.322359+0000","last_change":"2023-12-19T18:21:07.799098+0000","last_active":"2023-12-19T18:21:08.322359+0000","last_peered":"2023-12-19T18:21:08.322359+0000","last_clean":"2023-12-19T18:21:08.322359+0000","last_became_active":"2023-12-19T18:21:07.794686+0000","last_became_peered":"2023-12-19T18:21:07.794686+0000","last_unstale":"2023-12-19T18:21:08.322359+0000","last_undegraded":"2023-12-19T18:21:08.322359+0000","last_fullsized":"2023-12-19T18:21:08.322359+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:03:28.205860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.321857+0000","last_change":"2023-12-19T18:21:08.321784+0000","last_active":"2023-12-19T18:21:08.321857+0000","last_peered":"2023-12-19T18:21:08.321857+0000","last_clean":"2023-12-19T18:21:08.321857+0000","last_became_active":"2023-12-19T18:21:08.318398+0000","last_became_peered":"2023-12-19T18:21:08.318398+0000","last_unstale":"2023-12-19T18:21:08.321857+0000","last_undegraded":"2023-12-19T18:21:08.321857+0000","last_fullsized":"2023-12-19T18:21:08.321857+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:12.792194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.325676+0000","last_change":"2023-12-19T18:21:07.793726+0000","last_active":"2023-12-19T18:21:08.325676+0000","last_peered":"2023-12-19T18:21:08.325676+0000","last_clean":"2023-12-19T18:21:08.325676+0000","last_became_active":"2023-12-19T18:21:07.792928+0000","last_became_peered":"2023-12-19T18:21:07.792928+0000","last_unstale":"2023-12-19T18:21:08.325676+0000","last_undegraded":"2023-12-19T18:21:08.325676+0000","last_fullsized":"2023-12-19T18:21:08.325676+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:41:34.768171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.322976+0000","last_change":"2023-12-19T18:21:08.322816+0000","last_active":"2023-12-19T18:21:08.322976+0000","last_peered":"2023-12-19T18:21:08.322976+0000","last_clean":"2023-12-19T18:21:08.322976+0000","last_became_active":"2023-12-19T18:21:08.321553+0000","last_became_peered":"2023-12-19T18:21:08.321553+0000","last_unstale":"2023-12-19T18:21:08.322976+0000","last_undegraded":"2023-12-19T18:21:08.322976+0000","last_fullsized":"2023-12-19T18:21:08.322976+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:29.417658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.293835+0000","last_change":"2023-12-19T18:21:07.800708+0000","last_active":"2023-12-19T18:21:08.293835+0000","last_peered":"2023-12-19T18:21:08.293835+0000","last_clean":"2023-12-19T18:21:08.293835+0000","last_became_active":"2023-12-19T18:21:07.800418+0000","last_became_peered":"2023-12-19T18:21:07.800418+0000","last_unstale":"2023-12-19T18:21:08.293835+0000","last_undegraded":"2023-12-19T18:21:08.293835+0000","last_fullsized":"2023-12-19T18:21:08.293835+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:13:56.765152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.317003+0000","last_change":"2023-12-19T18:21:08.316931+0000","last_active":"2023-12-19T18:21:08.317003+0000","last_peered":"2023-12-19T18:21:08.317003+0000","last_clean":"2023-12-19T18:21:08.317003+0000","last_became_active":"2023-12-19T18:21:08.316689+0000","last_became_peered":"2023-12-19T18:21:08.316689+0000","last_unstale":"2023-12-19T18:21:08.317003+0000","last_undegraded":"2023-12-19T18:21:08.317003+0000","last_fullsized":"2023-12-19T18:21:08.317003+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:34:03.699275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.301119+0000","last_change":"2023-12-19T18:21:07.799650+0000","last_active":"2023-12-19T18:21:08.301119+0000","last_peered":"2023-12-19T18:21:08.301119+0000","last_clean":"2023-12-19T18:21:08.301119+0000","last_became_active":"2023-12-19T18:21:07.799430+0000","last_became_peered":"2023-12-19T18:21:07.799430+0000","last_unstale":"2023-12-19T18:21:08.301119+0000","last_undegraded":"2023-12-19T18:21:08.301119+0000","last_fullsized":"2023-12-19T18:21:08.301119+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:46.317296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.320868+0000","last_change":"2023-12-19T18:21:08.314318+0000","last_active":"2023-12-19T18:21:08.320868+0000","last_peered":"2023-12-19T18:21:08.320868+0000","last_clean":"2023-12-19T18:21:08.320868+0000","last_became_active":"2023-12-19T18:21:08.313932+0000","last_became_peered":"2023-12-19T18:21:08.313932+0000","last_unstale":"2023-12-19T18:21:08.320868+0000","last_undegraded":"2023-12-19T18:21:08.320868+0000","last_fullsized":"2023-12-19T18:21:08.320868+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:03:03.110026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"116'8","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.327848+0000","last_change":"2023-12-19T18:21:07.807077+0000","last_active":"2023-12-19T18:21:08.327848+0000","last_peered":"2023-12-19T18:21:08.327848+0000","last_clean":"2023-12-19T18:21:08.327848+0000","last_became_active":"2023-12-19T18:21:07.805873+0000","last_became_peered":"2023-12-19T18:21:07.805873+0000","last_unstale":"2023-12-19T18:21:08.327848+0000","last_undegraded":"2023-12-19T18:21:08.327848+0000","last_fullsized":"2023-12-19T18:21:08.327848+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:22:01.309084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.301091+0000","last_change":"2023-12-19T18:21:08.301007+0000","last_active":"2023-12-19T18:21:08.301091+0000","last_peered":"2023-12-19T18:21:08.301091+0000","last_clean":"2023-12-19T18:21:08.301091+0000","last_became_active":"2023-12-19T18:21:08.300783+0000","last_became_peered":"2023-12-19T18:21:08.300783+0000","last_unstale":"2023-12-19T18:21:08.301091+0000","last_undegraded":"2023-12-19T18:21:08.301091+0000","last_fullsized":"2023-12-19T18:21:08.301091+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:03:03.110026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.315131+0000","last_change":"2023-12-19T18:21:07.803614+0000","last_active":"2023-12-19T18:21:08.315131+0000","last_peered":"2023-12-19T18:21:08.315131+0000","last_clean":"2023-12-19T18:21:08.315131+0000","last_became_active":"2023-12-19T18:21:07.803317+0000","last_became_peered":"2023-12-19T18:21:07.803317+0000","last_unstale":"2023-12-19T18:21:08.315131+0000","last_undegraded":"2023-12-19T18:21:08.315131+0000","last_fullsized":"2023-12-19T18:21:08.315131+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:22:15.124299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.313057+0000","last_change":"2023-12-19T18:21:08.311926+0000","last_active":"2023-12-19T18:21:08.313057+0000","last_peered":"2023-12-19T18:21:08.313057+0000","last_clean":"2023-12-19T18:21:08.313057+0000","last_became_active":"2023-12-19T18:21:08.311703+0000","last_became_peered":"2023-12-19T18:21:08.311703+0000","last_unstale":"2023-12-19T18:21:08.313057+0000","last_undegraded":"2023-12-19T18:21:08.313057+0000","last_fullsized":"2023-12-19T18:21:08.313057+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:41:35.779905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.322167+0000","last_change":"2023-12-19T18:21:07.804857+0000","last_active":"2023-12-19T18:21:08.322167+0000","last_peered":"2023-12-19T18:21:08.322167+0000","last_clean":"2023-12-19T18:21:08.322167+0000","last_became_active":"2023-12-19T18:21:07.804173+0000","last_became_peered":"2023-12-19T18:21:07.804173+0000","last_unstale":"2023-12-19T18:21:08.322167+0000","last_undegraded":"2023-12-19T18:21:08.322167+0000","last_fullsized":"2023-12-19T18:21:08.322167+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:22:01.309084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.301108+0000","last_change":"2023-12-19T18:21:08.301050+0000","last_active":"2023-12-19T18:21:08.301108+0000","last_peered":"2023-12-19T18:21:08.301108+0000","last_clean":"2023-12-19T18:21:08.301108+0000","last_became_active":"2023-12-19T18:21:08.300813+0000","last_became_peered":"2023-12-19T18:21:08.300813+0000","last_unstale":"2023-12-19T18:21:08.301108+0000","last_undegraded":"2023-12-19T18:21:08.301108+0000","last_fullsized":"2023-12-19T18:21:08.301108+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:28:44.357084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.308380+0000","last_change":"2023-12-19T18:21:07.799014+0000","last_active":"2023-12-19T18:21:08.308380+0000","last_peered":"2023-12-19T18:21:08.308380+0000","last_clean":"2023-12-19T18:21:08.308380+0000","last_became_active":"2023-12-19T18:21:07.798604+0000","last_became_peered":"2023-12-19T18:21:07.798604+0000","last_unstale":"2023-12-19T18:21:08.308380+0000","last_undegraded":"2023-12-19T18:21:08.308380+0000","last_fullsized":"2023-12-19T18:21:08.308380+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:37.881543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.323230+0000","last_change":"2023-12-19T18:21:08.323145+0000","last_active":"2023-12-19T18:21:08.323230+0000","last_peered":"2023-12-19T18:21:08.323230+0000","last_clean":"2023-12-19T18:21:08.323230+0000","last_became_active":"2023-12-19T18:21:08.322239+0000","last_became_peered":"2023-12-19T18:21:08.322239+0000","last_unstale":"2023-12-19T18:21:08.323230+0000","last_undegraded":"2023-12-19T18:21:08.323230+0000","last_fullsized":"2023-12-19T18:21:08.323230+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:28:44.357084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.323582+0000","last_change":"2023-12-19T18:21:07.803456+0000","last_active":"2023-12-19T18:21:08.323582+0000","last_peered":"2023-12-19T18:21:08.323582+0000","last_clean":"2023-12-19T18:21:08.323582+0000","last_became_active":"2023-12-19T18:21:07.803181+0000","last_became_peered":"2023-12-19T18:21:07.803181+0000","last_unstale":"2023-12-19T18:21:08.323582+0000","last_undegraded":"2023-12-19T18:21:08.323582+0000","last_fullsized":"2023-12-19T18:21:08.323582+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:54:01.049760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.318566+0000","last_change":"2023-12-19T18:21:08.318329+0000","last_active":"2023-12-19T18:21:08.318566+0000","last_peered":"2023-12-19T18:21:08.318566+0000","last_clean":"2023-12-19T18:21:08.318566+0000","last_became_active":"2023-12-19T18:21:08.317440+0000","last_became_peered":"2023-12-19T18:21:08.317440+0000","last_unstale":"2023-12-19T18:21:08.318566+0000","last_undegraded":"2023-12-19T18:21:08.318566+0000","last_fullsized":"2023-12-19T18:21:08.318566+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:10.958477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.294584+0000","last_change":"2023-12-19T18:21:07.796837+0000","last_active":"2023-12-19T18:21:08.294584+0000","last_peered":"2023-12-19T18:21:08.294584+0000","last_clean":"2023-12-19T18:21:08.294584+0000","last_became_active":"2023-12-19T18:21:07.796313+0000","last_became_peered":"2023-12-19T18:21:07.796313+0000","last_unstale":"2023-12-19T18:21:08.294584+0000","last_undegraded":"2023-12-19T18:21:08.294584+0000","last_fullsized":"2023-12-19T18:21:08.294584+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:34.703586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.307299+0000","last_change":"2023-12-19T18:21:08.303871+0000","last_active":"2023-12-19T18:21:08.307299+0000","last_peered":"2023-12-19T18:21:08.307299+0000","last_clean":"2023-12-19T18:21:08.307299+0000","last_became_active":"2023-12-19T18:21:08.302965+0000","last_became_peered":"2023-12-19T18:21:08.302965+0000","last_unstale":"2023-12-19T18:21:08.307299+0000","last_undegraded":"2023-12-19T18:21:08.307299+0000","last_fullsized":"2023-12-19T18:21:08.307299+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:38:33.981360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.312515+0000","last_change":"2023-12-19T18:21:07.805332+0000","last_active":"2023-12-19T18:21:08.312515+0000","last_peered":"2023-12-19T18:21:08.312515+0000","last_clean":"2023-12-19T18:21:08.312515+0000","last_became_active":"2023-12-19T18:21:07.804079+0000","last_became_peered":"2023-12-19T18:21:07.804079+0000","last_unstale":"2023-12-19T18:21:08.312515+0000","last_undegraded":"2023-12-19T18:21:08.312515+0000","last_fullsized":"2023-12-19T18:21:08.312515+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:11.803926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.302468+0000","last_change":"2023-12-19T18:21:08.302376+0000","last_active":"2023-12-19T18:21:08.302468+0000","last_peered":"2023-12-19T18:21:08.302468+0000","last_clean":"2023-12-19T18:21:08.302468+0000","last_became_active":"2023-12-19T18:21:08.302047+0000","last_became_peered":"2023-12-19T18:21:08.302047+0000","last_unstale":"2023-12-19T18:21:08.302468+0000","last_undegraded":"2023-12-19T18:21:08.302468+0000","last_fullsized":"2023-12-19T18:21:08.302468+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:04.632373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.310638+0000","last_change":"2023-12-19T18:21:07.805766+0000","last_active":"2023-12-19T18:21:08.310638+0000","last_peered":"2023-12-19T18:21:08.310638+0000","last_clean":"2023-12-19T18:21:08.310638+0000","last_became_active":"2023-12-19T18:21:07.805278+0000","last_became_peered":"2023-12-19T18:21:07.805278+0000","last_unstale":"2023-12-19T18:21:08.310638+0000","last_undegraded":"2023-12-19T18:21:08.310638+0000","last_fullsized":"2023-12-19T18:21:08.310638+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:43.474146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.297365+0000","last_change":"2023-12-19T18:21:08.296307+0000","last_active":"2023-12-19T18:21:08.297365+0000","last_peered":"2023-12-19T18:21:08.297365+0000","last_clean":"2023-12-19T18:21:08.297365+0000","last_became_active":"2023-12-19T18:21:08.296082+0000","last_became_peered":"2023-12-19T18:21:08.296082+0000","last_unstale":"2023-12-19T18:21:08.297365+0000","last_undegraded":"2023-12-19T18:21:08.297365+0000","last_fullsized":"2023-12-19T18:21:08.297365+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:49:40.809863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.326093+0000","last_change":"2023-12-19T18:21:07.797860+0000","last_active":"2023-12-19T18:21:08.326093+0000","last_peered":"2023-12-19T18:21:08.326093+0000","last_clean":"2023-12-19T18:21:08.326093+0000","last_became_active":"2023-12-19T18:21:07.797444+0000","last_became_peered":"2023-12-19T18:21:07.797444+0000","last_unstale":"2023-12-19T18:21:08.326093+0000","last_undegraded":"2023-12-19T18:21:08.326093+0000","last_fullsized":"2023-12-19T18:21:08.326093+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:11.803926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.303295+0000","last_change":"2023-12-19T18:21:08.302442+0000","last_active":"2023-12-19T18:21:08.303295+0000","last_peered":"2023-12-19T18:21:08.303295+0000","last_clean":"2023-12-19T18:21:08.303295+0000","last_became_active":"2023-12-19T18:21:08.302226+0000","last_became_peered":"2023-12-19T18:21:08.302226+0000","last_unstale":"2023-12-19T18:21:08.303295+0000","last_undegraded":"2023-12-19T18:21:08.303295+0000","last_fullsized":"2023-12-19T18:21:08.303295+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:30:58.163868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.310079+0000","last_change":"2023-12-19T18:21:07.806540+0000","last_active":"2023-12-19T18:21:08.310079+0000","last_peered":"2023-12-19T18:21:08.310079+0000","last_clean":"2023-12-19T18:21:08.310079+0000","last_became_active":"2023-12-19T18:21:07.805458+0000","last_became_peered":"2023-12-19T18:21:07.805458+0000","last_unstale":"2023-12-19T18:21:08.310079+0000","last_undegraded":"2023-12-19T18:21:08.310079+0000","last_fullsized":"2023-12-19T18:21:08.310079+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:28.405924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.303275+0000","last_change":"2023-12-19T18:21:08.299482+0000","last_active":"2023-12-19T18:21:08.303275+0000","last_peered":"2023-12-19T18:21:08.303275+0000","last_clean":"2023-12-19T18:21:08.303275+0000","last_became_active":"2023-12-19T18:21:08.299265+0000","last_became_peered":"2023-12-19T18:21:08.299265+0000","last_unstale":"2023-12-19T18:21:08.303275+0000","last_undegraded":"2023-12-19T18:21:08.303275+0000","last_fullsized":"2023-12-19T18:21:08.303275+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:03:47.164205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.296511+0000","last_change":"2023-12-19T18:21:07.805012+0000","last_active":"2023-12-19T18:21:08.296511+0000","last_peered":"2023-12-19T18:21:08.296511+0000","last_clean":"2023-12-19T18:21:08.296511+0000","last_became_active":"2023-12-19T18:21:07.802455+0000","last_became_peered":"2023-12-19T18:21:07.802455+0000","last_unstale":"2023-12-19T18:21:08.296511+0000","last_undegraded":"2023-12-19T18:21:08.296511+0000","last_fullsized":"2023-12-19T18:21:08.296511+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:53.467791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.317018+0000","last_change":"2023-12-19T18:21:08.316953+0000","last_active":"2023-12-19T18:21:08.317018+0000","last_peered":"2023-12-19T18:21:08.317018+0000","last_clean":"2023-12-19T18:21:08.317018+0000","last_became_active":"2023-12-19T18:21:08.316739+0000","last_became_peered":"2023-12-19T18:21:08.316739+0000","last_unstale":"2023-12-19T18:21:08.317018+0000","last_undegraded":"2023-12-19T18:21:08.317018+0000","last_fullsized":"2023-12-19T18:21:08.317018+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:07:00.931274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.300021+0000","last_change":"2023-12-19T18:21:07.802844+0000","last_active":"2023-12-19T18:21:08.300021+0000","last_peered":"2023-12-19T18:21:08.300021+0000","last_clean":"2023-12-19T18:21:08.300021+0000","last_became_active":"2023-12-19T18:21:07.801866+0000","last_became_peered":"2023-12-19T18:21:07.801866+0000","last_unstale":"2023-12-19T18:21:08.300021+0000","last_undegraded":"2023-12-19T18:21:08.300021+0000","last_fullsized":"2023-12-19T18:21:08.300021+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:38:30.353314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.303175+0000","last_change":"2023-12-19T18:21:08.303040+0000","last_active":"2023-12-19T18:21:08.303175+0000","last_peered":"2023-12-19T18:21:08.303175+0000","last_clean":"2023-12-19T18:21:08.303175+0000","last_became_active":"2023-12-19T18:21:08.302793+0000","last_became_peered":"2023-12-19T18:21:08.302793+0000","last_unstale":"2023-12-19T18:21:08.303175+0000","last_undegraded":"2023-12-19T18:21:08.303175+0000","last_fullsized":"2023-12-19T18:21:08.303175+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:31.559355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.294056+0000","last_change":"2023-12-19T18:21:07.793133+0000","last_active":"2023-12-19T18:21:08.294056+0000","last_peered":"2023-12-19T18:21:08.294056+0000","last_clean":"2023-12-19T18:21:08.294056+0000","last_became_active":"2023-12-19T18:21:07.792316+0000","last_became_peered":"2023-12-19T18:21:07.792316+0000","last_unstale":"2023-12-19T18:21:08.294056+0000","last_undegraded":"2023-12-19T18:21:08.294056+0000","last_fullsized":"2023-12-19T18:21:08.294056+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:53.467791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.323083+0000","last_change":"2023-12-19T18:21:08.322148+0000","last_active":"2023-12-19T18:21:08.323083+0000","last_peered":"2023-12-19T18:21:08.323083+0000","last_clean":"2023-12-19T18:21:08.323083+0000","last_became_active":"2023-12-19T18:21:08.319104+0000","last_became_peered":"2023-12-19T18:21:08.319104+0000","last_unstale":"2023-12-19T18:21:08.323083+0000","last_undegraded":"2023-12-19T18:21:08.323083+0000","last_fullsized":"2023-12-19T18:21:08.323083+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:14:03.228122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.301582+0000","last_change":"2023-12-19T18:21:07.795313+0000","last_active":"2023-12-19T18:21:08.301582+0000","last_peered":"2023-12-19T18:21:08.301582+0000","last_clean":"2023-12-19T18:21:08.301582+0000","last_became_active":"2023-12-19T18:21:07.794826+0000","last_became_peered":"2023-12-19T18:21:07.794826+0000","last_unstale":"2023-12-19T18:21:08.301582+0000","last_undegraded":"2023-12-19T18:21:08.301582+0000","last_fullsized":"2023-12-19T18:21:08.301582+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:27:51.591572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.303547+0000","last_change":"2023-12-19T18:21:08.303447+0000","last_active":"2023-12-19T18:21:08.303547+0000","last_peered":"2023-12-19T18:21:08.303547+0000","last_clean":"2023-12-19T18:21:08.303547+0000","last_became_active":"2023-12-19T18:21:08.303203+0000","last_became_peered":"2023-12-19T18:21:08.303203+0000","last_unstale":"2023-12-19T18:21:08.303547+0000","last_undegraded":"2023-12-19T18:21:08.303547+0000","last_fullsized":"2023-12-19T18:21:08.303547+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:48.490025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.295122+0000","last_change":"2023-12-19T18:21:07.806245+0000","last_active":"2023-12-19T18:21:08.295122+0000","last_peered":"2023-12-19T18:21:08.295122+0000","last_clean":"2023-12-19T18:21:08.295122+0000","last_became_active":"2023-12-19T18:21:07.805665+0000","last_became_peered":"2023-12-19T18:21:07.805665+0000","last_unstale":"2023-12-19T18:21:08.295122+0000","last_undegraded":"2023-12-19T18:21:08.295122+0000","last_fullsized":"2023-12-19T18:21:08.295122+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:13:26.501953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.307140+0000","last_change":"2023-12-19T18:21:08.306860+0000","last_active":"2023-12-19T18:21:08.307140+0000","last_peered":"2023-12-19T18:21:08.307140+0000","last_clean":"2023-12-19T18:21:08.307140+0000","last_became_active":"2023-12-19T18:21:08.306625+0000","last_became_peered":"2023-12-19T18:21:08.306625+0000","last_unstale":"2023-12-19T18:21:08.307140+0000","last_undegraded":"2023-12-19T18:21:08.307140+0000","last_fullsized":"2023-12-19T18:21:08.307140+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:46:10.913345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.296626+0000","last_change":"2023-12-19T18:21:07.802817+0000","last_active":"2023-12-19T18:21:08.296626+0000","last_peered":"2023-12-19T18:21:08.296626+0000","last_clean":"2023-12-19T18:21:08.296626+0000","last_became_active":"2023-12-19T18:21:07.801981+0000","last_became_peered":"2023-12-19T18:21:07.801981+0000","last_unstale":"2023-12-19T18:21:08.296626+0000","last_undegraded":"2023-12-19T18:21:08.296626+0000","last_fullsized":"2023-12-19T18:21:08.296626+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:58:53.471739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.320795+0000","last_change":"2023-12-19T18:21:08.314162+0000","last_active":"2023-12-19T18:21:08.320795+0000","last_peered":"2023-12-19T18:21:08.320795+0000","last_clean":"2023-12-19T18:21:08.320795+0000","last_became_active":"2023-12-19T18:21:08.313589+0000","last_became_peered":"2023-12-19T18:21:08.313589+0000","last_unstale":"2023-12-19T18:21:08.320795+0000","last_undegraded":"2023-12-19T18:21:08.320795+0000","last_fullsized":"2023-12-19T18:21:08.320795+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:38:31.365048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.319299+0000","last_change":"2023-12-19T18:21:07.804936+0000","last_active":"2023-12-19T18:21:08.319299+0000","last_peered":"2023-12-19T18:21:08.319299+0000","last_clean":"2023-12-19T18:21:08.319299+0000","last_became_active":"2023-12-19T18:21:07.804679+0000","last_became_peered":"2023-12-19T18:21:07.804679+0000","last_unstale":"2023-12-19T18:21:08.319299+0000","last_undegraded":"2023-12-19T18:21:08.319299+0000","last_fullsized":"2023-12-19T18:21:08.319299+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:00:29.556804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":250,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-12-19T18:21:07.784836+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:21:07.784836+0000","last_peered":"2023-12-19T18:21:07.784836+0000","last_clean":"2023-12-19T18:21:07.784836+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:21:07.784836+0000","last_undegraded":"2023-12-19T18:21:07.784836+0000","last_fullsized":"2023-12-19T18:21:07.784836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359091,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.22,"15min":1.1439999999999999},"min":{"1min":0.69999999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.6830000000000001,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":1.24},{"interface":"front","average":{"1min":1.264,"5min":1.26,"15min":1.1799999999999999},"min":{"1min":0.76100000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.665,"5min":1.79,"15min":1.79},"last":1.05}]},{"osd":1,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.361,"15min":1.232},"min":{"1min":0.63600000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5229999999999999,"5min":10.926,"15min":10.926},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3240000000000001,"15min":1.2350000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.67900000000000005},"max":{"1min":1.589,"5min":10.904999999999999,"15min":10.904999999999999},"last":1.621}]},{"osd":2,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.157,"15min":1.107},"min":{"1min":0.72699999999999998,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.5409999999999999,"5min":3.1960000000000002,"15min":3.1960000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.2569999999999999,"15min":1.1870000000000001},"min":{"1min":0.81999999999999995,"5min":0.72199999999999998,"15min":0.69699999999999995},"max":{"1min":1.4430000000000001,"5min":6.008,"15min":6.008},"last":1.4490000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.266,"15min":1.151},"min":{"1min":0.995,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.696,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.409,"15min":1.28},"min":{"1min":0.77300000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.7829999999999999,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":1.5820000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2350000000000001,"15min":1.1060000000000001},"min":{"1min":0.92200000000000004,"5min":0.65400000000000003,"15min":0.47899999999999998},"max":{"1min":1.7529999999999999,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":1.8009999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3089999999999999,"15min":1.24},"min":{"1min":0.98099999999999998,"5min":0.72999999999999998,"15min":0.57599999999999996},"max":{"1min":1.6040000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.54}]},{"osd":5,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.4279999999999999,"15min":1.333},"min":{"1min":0.81399999999999995,"5min":0.749,"15min":0.55000000000000004},"max":{"1min":1.873,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":1.748},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4239999999999999,"15min":1.3300000000000001},"min":{"1min":1.0449999999999999,"5min":0.71599999999999997,"15min":0.53500000000000003},"max":{"1min":1.9830000000000001,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":2.1179999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5529999999999999,"15min":1.4570000000000001},"min":{"1min":1.119,"5min":0.78600000000000003,"15min":0.59899999999999998},"max":{"1min":2.1320000000000001,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":2.423},{"interface":"front","average":{"1min":1.484,"5min":1.4850000000000001,"15min":1.371},"min":{"1min":1.125,"5min":0.92300000000000004,"15min":0.63200000000000001},"max":{"1min":1.9710000000000001,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":2.0179999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199358489,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.0840000000000001,"15min":0.99199999999999999},"min":{"1min":0.74099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6240000000000001,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.1759999999999999,"15min":1.0640000000000001},"min":{"1min":0.79900000000000004,"5min":0.64300000000000002,"15min":0.55600000000000005},"max":{"1min":1.748,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.94899999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1359999999999999,"15min":1.1240000000000001},"min":{"1min":0.84899999999999998,"5min":0.71399999999999997,"15min":0.68200000000000005},"max":{"1min":1.474,"5min":4.5540000000000003,"15min":4.5540000000000003},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.2030000000000001,"15min":1.165},"min":{"1min":0.93000000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.746,"5min":1.885,"15min":1.885},"last":1.2529999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.25,"15min":1.143},"min":{"1min":0.83499999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.7090000000000001,"5min":1.821,"15min":1.821},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.2150000000000001,"15min":1.2370000000000001},"min":{"1min":0.88200000000000001,"5min":0.81499999999999995,"15min":0.79900000000000004},"max":{"1min":1.944,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":1.0620000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.2769999999999999,"15min":1.1930000000000001},"min":{"1min":0.85699999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.7110000000000001,"5min":3.8820000000000001,"15min":3.8820000000000001},"last":1.173},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2070000000000001,"15min":1.1659999999999999},"min":{"1min":0.878,"5min":0.78300000000000003,"15min":0.77500000000000002},"max":{"1min":1.5640000000000001,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":1.458}]},{"osd":4,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2789999999999999,"15min":1.2410000000000001},"min":{"1min":0.85199999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.8520000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.534},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.4359999999999999,"15min":1.3400000000000001},"min":{"1min":1.208,"5min":0.70799999999999996,"15min":0.57999999999999996},"max":{"1min":2.0430000000000001,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.4990000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.284,"15min":1.1819999999999999},"min":{"1min":0.90500000000000003,"5min":0.66600000000000004,"15min":0.66400000000000003},"max":{"1min":2.2400000000000002,"5min":4.6680000000000001,"15min":4.6680000000000001},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.4059999999999999,"15min":1.337},"min":{"1min":1.0169999999999999,"5min":0.626,"15min":0.626},"max":{"1min":2.0459999999999998,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.6990000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.4370000000000001,"15min":1.341},"min":{"1min":1.034,"5min":0.77200000000000002,"15min":0.61899999999999999},"max":{"1min":2.133,"5min":3.9670000000000001,"15min":3.9670000000000001},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.4470000000000001,"15min":1.351},"min":{"1min":1.075,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.3519999999999999,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":1.619}]}]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1195,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194116,"statfs":{"total":200000000,"available":198775568,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.242,"15min":1.1679999999999999},"min":{"1min":0.79600000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5600000000000001,"5min":1.837,"15min":1.863},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.274,"15min":1.179},"min":{"1min":1.004,"5min":0.75,"15min":0.75},"max":{"1min":1.724,"5min":1.879,"15min":1.879},"last":1.3200000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1819999999999999,"15min":1.05},"min":{"1min":0.86199999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.639,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.135,"5min":1.149,"15min":1.0549999999999999},"min":{"1min":0.754,"5min":0.68700000000000006,"15min":0.61299999999999999},"max":{"1min":1.617,"5min":1.843,"15min":1.843},"last":0.92100000000000004}]},{"osd":2,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1699999999999999,"15min":1.081},"min":{"1min":0.93500000000000005,"5min":0.74299999999999999,"15min":0.59499999999999997},"max":{"1min":1.431,"5min":1.794,"15min":1.794},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.228,"15min":1.1379999999999999},"min":{"1min":0.84899999999999998,"5min":0.76800000000000002,"15min":0.65200000000000002},"max":{"1min":1.768,"5min":2.036,"15min":2.036},"last":1.004}]},{"osd":3,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1850000000000001,"15min":1.085},"min":{"1min":0.71999999999999997,"5min":0.69399999999999995,"15min":0.68899999999999995},"max":{"1min":1.45,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.085},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.339,"15min":1.222},"min":{"1min":0.875,"5min":0.75700000000000001,"15min":0.70399999999999996},"max":{"1min":1.8380000000000001,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.25}]},{"osd":4,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.302,"15min":1.163},"min":{"1min":0.83299999999999996,"5min":0.60899999999999999,"15min":0.46700000000000003},"max":{"1min":1.556,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.577},{"interface":"front","average":{"1min":1.23,"5min":1.1819999999999999,"15min":1.1200000000000001},"min":{"1min":0.872,"5min":0.73999999999999999,"15min":0.57099999999999995},"max":{"1min":1.708,"5min":1.804,"15min":1.901},"last":1.5229999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4430000000000001,"15min":1.325},"min":{"1min":0.92500000000000004,"5min":0.66600000000000004,"15min":0.54400000000000004},"max":{"1min":1.544,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.387},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.3049999999999999,"15min":1.236},"min":{"1min":0.81999999999999995,"5min":0.81200000000000006,"15min":0.49299999999999999},"max":{"1min":1.643,"5min":1.7949999999999999,"15min":1.976},"last":1.22}]},{"osd":7,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2849999999999999,"15min":1.2190000000000001},"min":{"1min":0.88500000000000001,"5min":0.68300000000000005,"15min":0.50900000000000001},"max":{"1min":1.752,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4590000000000001,"15min":1.407},"min":{"1min":0.95099999999999996,"5min":0.84999999999999998,"15min":0.623},"max":{"1min":1.929,"5min":2.117,"15min":2.117},"last":1.617}]}]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194693,"statfs":{"total":200000000,"available":199365936,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.997,"15min":0.997},"min":{"1min":0.69299999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.532,"5min":1.579,"15min":1.579},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0469999999999999,"15min":1.0309999999999999},"min":{"1min":0.76900000000000002,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.575,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.80000000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1679999999999999,"15min":1.1519999999999999},"min":{"1min":0.81699999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6679999999999999,"5min":1.7130000000000001,"15min":1.8160000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.248,"15min":1.175},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4470000000000001,"5min":1.696,"15min":1.696},"last":1.149}]},{"osd":2,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1279999999999999,"15min":1.1240000000000001},"min":{"1min":0.89700000000000002,"5min":0.747,"15min":0.69799999999999995},"max":{"1min":1.4079999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.1479999999999999,"15min":1.1000000000000001},"min":{"1min":0.78000000000000003,"5min":0.72199999999999998,"15min":0.70499999999999996},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.91500000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2869999999999999,"15min":1.254},"min":{"1min":0.95299999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.5249999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.254},{"interface":"front","average":{"1min":1.359,"5min":1.327,"15min":1.2989999999999999},"min":{"1min":1.1699999999999999,"5min":0.82399999999999995,"15min":0.74399999999999999},"max":{"1min":1.5740000000000001,"5min":1.718,"15min":1.7529999999999999},"last":1.2310000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.2929999999999999,"15min":1.198},"min":{"1min":0.89200000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6659999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.4179999999999999,"15min":1.3460000000000001},"min":{"1min":0.86699999999999999,"5min":0.625,"15min":0.504},"max":{"1min":1.6950000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.353}]},{"osd":6,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.319,"15min":1.3029999999999999},"min":{"1min":0.76700000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.627,"5min":1.853,"15min":1.853},"last":1.306},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.454,"15min":1.389},"min":{"1min":1.2170000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.7529999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.452}]},{"osd":7,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.387,"15min":1.3380000000000001},"min":{"1min":1.1279999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.6739999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.55},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.427,"15min":1.405},"min":{"1min":0.98699999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.7070000000000001,"5min":1.905,"15min":1.9330000000000001},"last":1.5249999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199358487,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":0.91800000000000004,"15min":0.96799999999999997},"min":{"1min":0.38100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.3140000000000001,"5min":3.3140000000000001,"15min":3.4350000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.018,"5min":0.97499999999999998,"15min":0.93500000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7170000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.82599999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":0.98799999999999999,"15min":1.054},"min":{"1min":0.439,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":4.5129999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.218,"5min":1.0800000000000001,"15min":1.0960000000000001},"min":{"1min":0.57099999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.123}]},{"osd":2,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3240000000000001,"15min":1.25},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.5369999999999999,"5min":13.833,"15min":13.833},"last":1.375},{"interface":"front","average":{"1min":1.587,"5min":1.448,"15min":1.284},"min":{"1min":0.66300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":3.9710000000000001,"5min":16.748999999999999,"15min":16.748999999999999},"last":1.4239999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.302,"15min":1.214},"min":{"1min":0.88100000000000001,"5min":0.77400000000000002,"15min":0.65600000000000003},"max":{"1min":3.0910000000000002,"5min":3.0910000000000002,"15min":3.0910000000000002},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.359,"15min":1.397},"min":{"1min":0.84599999999999997,"5min":0.80800000000000005,"15min":0.73199999999999998},"max":{"1min":2.0640000000000001,"5min":2.5710000000000002,"15min":4.0209999999999999},"last":1.708}]},{"osd":5,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.4119999999999999,"15min":1.4410000000000001},"min":{"1min":1.0509999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":1.581},{"interface":"front","average":{"1min":1.415,"5min":1.327,"15min":1.264},"min":{"1min":0.84699999999999998,"5min":0.76600000000000001,"15min":0.69999999999999996},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.1679999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.4119999999999999,"15min":1.472},"min":{"1min":0.877,"5min":0.81799999999999995,"15min":0.81599999999999995},"max":{"1min":2.1739999999999999,"5min":2.4790000000000001,"15min":3.9399999999999999},"last":1.526},{"interface":"front","average":{"1min":1.627,"5min":1.4810000000000001,"15min":1.504},"min":{"1min":0.83199999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.2120000000000002,"5min":2.2400000000000002,"15min":3.2229999999999999},"last":1.327}]},{"osd":7,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.4350000000000001,"15min":1.3740000000000001},"min":{"1min":0.90600000000000003,"5min":0.75900000000000001,"15min":0.73199999999999998},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.026},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.52,"15min":1.544},"min":{"1min":1.1160000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.456,"5min":2.657,"15min":3.9849999999999999},"last":1.635}]}]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359161,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.0600000000000001,"15min":1.016},"min":{"1min":0.38200000000000001,"5min":0.35299999999999998,"15min":0.34399999999999997},"max":{"1min":2.879,"5min":9.2550000000000008,"15min":9.2550000000000008},"last":1.286},{"interface":"front","average":{"1min":1.165,"5min":1.026,"15min":1.044},"min":{"1min":0.57899999999999996,"5min":0.41999999999999998,"15min":0.38900000000000001},"max":{"1min":2.387,"5min":2.387,"15min":4.7850000000000001},"last":0.84099999999999997}]},{"osd":1,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":0.94399999999999995,"15min":1.1859999999999999},"min":{"1min":0.50600000000000001,"5min":0.49199999999999999,"15min":0.46000000000000002},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":8.2720000000000002},"last":1.542},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.0860000000000001,"15min":1.1699999999999999},"min":{"1min":0.61599999999999999,"5min":0.504,"15min":0.47199999999999998},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":5.0620000000000003},"last":0.88900000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.149,"15min":1.383},"min":{"1min":0.56000000000000005,"5min":0.54200000000000004,"15min":0.51900000000000002},"max":{"1min":2.238,"5min":2.238,"15min":8.3109999999999999},"last":1.103},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.149,"15min":1.349},"min":{"1min":0.55500000000000005,"5min":0.46300000000000002,"15min":0.41799999999999998},"max":{"1min":2.323,"5min":2.323,"15min":8.0690000000000008},"last":1.1910000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.405,"15min":1.472},"min":{"1min":0.86399999999999999,"5min":0.82599999999999996,"15min":0.79300000000000004},"max":{"1min":2.6139999999999999,"5min":2.6139999999999999,"15min":5.3929999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.421,"15min":1.5569999999999999},"min":{"1min":0.89500000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":3.1070000000000002,"5min":3.1070000000000002,"15min":7.3570000000000002},"last":0.96799999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.3680000000000001,"15min":1.5620000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":8.1839999999999993},"last":1.474},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.409,"15min":1.419},"min":{"1min":0.98299999999999998,"5min":0.81000000000000005,"15min":0.72199999999999998},"max":{"1min":2.4940000000000002,"5min":2.4940000000000002,"15min":5.165},"last":1.3919999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.4119999999999999,"15min":1.454},"min":{"1min":0.92200000000000004,"5min":0.752,"15min":0.72999999999999998},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":5.1059999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.4470000000000001,"15min":1.5720000000000001},"min":{"1min":0.93300000000000005,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.9580000000000002,"5min":2.9580000000000002,"15min":8.1370000000000005},"last":1.6060000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.708,"5min":1.5089999999999999,"15min":1.5589999999999999},"min":{"1min":0.98999999999999999,"5min":0.79200000000000004,"15min":0.78200000000000003},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":5.1449999999999996},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.5009999999999999,"15min":1.663},"min":{"1min":0.88100000000000001,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":8.2300000000000004},"last":1.7569999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194110,"statfs":{"total":200000000,"available":198768703,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.48399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":0.94299999999999995,"15min":0.96299999999999997},"min":{"1min":0.68000000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.79100000000000004}]},{"osd":2,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.048,"15min":1.0589999999999999},"min":{"1min":0.45500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":2.5739999999999998},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.0089999999999999,"15min":0.92800000000000005},"min":{"1min":0.76700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.716,"5min":1.845,"15min":1.845},"last":1.268}]},{"osd":3,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.2869999999999999,"15min":1.159},"min":{"1min":0.68799999999999994,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.22,"15min":1.1970000000000001},"min":{"1min":0.746,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":2.3959999999999999},"last":1.3740000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.329,"15min":1.294},"min":{"1min":0.72199999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.377,"15min":1.3},"min":{"1min":0.93300000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.623}]},{"osd":5,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.6679999999999999,"5min":1.407,"15min":1.3009999999999999},"min":{"1min":1.1180000000000001,"5min":0.88,"15min":0.88},"max":{"1min":2.5459999999999998,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.484,"15min":1.415},"min":{"1min":1.024,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.766}]},{"osd":6,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.736,"5min":1.4490000000000001,"15min":1.389},"min":{"1min":1.196,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.7000000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":1.034},{"interface":"front","average":{"1min":1.792,"5min":1.5309999999999999,"15min":1.4550000000000001},"min":{"1min":0.91400000000000003,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.895,"5min":2.895,"15min":2.895},"last":1.6759999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.7230000000000001,"5min":1.569,"15min":1.3899999999999999},"min":{"1min":0.88800000000000001,"5min":0.82399999999999995,"15min":0.69999999999999996},"max":{"1min":2.5670000000000002,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.7729999999999999,"5min":1.651,"15min":1.506},"min":{"1min":1.157,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.456,"5min":9.4109999999999996,"15min":9.4109999999999996},"last":1.472}]}]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":627,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199357575,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":0.98699999999999999,"15min":0.89800000000000002},"min":{"1min":0.51100000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.127,"15min":1.0369999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.956,"5min":2.1040000000000001,"15min":2.4180000000000001},"last":1.3580000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.03,"15min":0.95799999999999996},"min":{"1min":0.70599999999999996,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":1.536,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.028,"15min":0.94299999999999995},"min":{"1min":0.55700000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.0950000000000002,"5min":2.7810000000000001,"15min":2.7810000000000001},"last":0.98399999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.046,"15min":1.0109999999999999},"min":{"1min":0.52400000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3049999999999999,"15min":1.2310000000000001},"min":{"1min":0.63100000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.8420000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.681}]},{"osd":4,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.331,"15min":1.2050000000000001},"min":{"1min":0.93100000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.5529999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.444},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.306,"15min":1.2889999999999999},"min":{"1min":0.89800000000000002,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.2450000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.2609999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5129999999999999,"15min":1.4039999999999999},"min":{"1min":0.751,"5min":0.71099999999999997,"15min":0.64500000000000002},"max":{"1min":2.3940000000000001,"5min":16.298999999999999,"15min":16.298999999999999},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.3700000000000001,"15min":1.2809999999999999},"min":{"1min":0.78400000000000003,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.4830000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.478,"15min":1.4019999999999999},"min":{"1min":1.0369999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.5910000000000002},"last":1.575},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.4710000000000001,"15min":1.3640000000000001},"min":{"1min":0.87,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.4359999999999999,"5min":2.46,"15min":2.46},"last":1.3160000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.5269999999999999,"15min":1.3979999999999999},"min":{"1min":0.90900000000000003,"5min":0.89200000000000002,"15min":0.70899999999999996},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.4390000000000001,"15min":1.431},"min":{"1min":1.0600000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.4580000000000002,"5min":2.4580000000000002,"15min":3.1429999999999998},"last":1.6499999999999999}]}]}],"pool_statfs":[]}} 2023-12-19T18:21:13.329 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:21:13.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.602+0000 7f088602c640 1 Processor -- start 2023-12-19T18:21:13.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.602+0000 7f088602c640 1 -- start start 2023-12-19T18:21:13.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.604+0000 7f088602c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f08801604d0 0x7f08800b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:13.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.604+0000 7f088602c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08800b7310 0x7f08800b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:13.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.604+0000 7f088602c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08800b5e90 con 0x7f08800d2350 2023-12-19T18:21:13.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.604+0000 7f088602c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08800d2b30 con 0x7f08801604d0 2023-12-19T18:21:13.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.604+0000 7f088602c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08800d2850 con 0x7f08800b7310 2023-12-19T18:21:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.605+0000 7f088502a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f08801604d0 0x7f08800b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.605+0000 7f088502a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f08801604d0 0x7f08800b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35188/0 (socket says 172.21.15.27:35188) 2023-12-19T18:21:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.605+0000 7f088502a640 1 -- 172.21.15.27:0/216419963 learned_addr learned my addr 172.21.15.27:0/216419963 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.605+0000 7f088582b640 1 --2- 172.21.15.27:0/216419963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08800b7310 0x7f08800b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.605+0000 7f088502a640 1 -- 172.21.15.27:0/216419963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08800b7310 msgr2=0x7f08800b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.605+0000 7f088502a640 1 --2- 172.21.15.27:0/216419963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08800b7310 0x7f08800b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.605+0000 7f088502a640 1 -- 172.21.15.27:0/216419963 >> v1:172.21.15.192:6789/0 conn(0x7f08800d2350 legacy=0x7f08800d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:13.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.605+0000 7f088502a640 1 -- 172.21.15.27:0/216419963 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08800cded0 con 0x7f08801604d0 2023-12-19T18:21:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.607+0000 7f088502a640 1 --2- 172.21.15.27:0/216419963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08801604d0 0x7f08800b47e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f0870009e50 tx=0x7f087002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e234fbaf001f07c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.607+0000 7f087ffff640 1 -- 172.21.15.27:0/216419963 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087002cb00 con 0x7f08801604d0 2023-12-19T18:21:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.607+0000 7f087ffff640 1 -- 172.21.15.27:0/216419963 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087002cc60 con 0x7f08801604d0 2023-12-19T18:21:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.607+0000 7f087ffff640 1 -- 172.21.15.27:0/216419963 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08700343f0 con 0x7f08801604d0 2023-12-19T18:21:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.608+0000 7f088602c640 1 -- 172.21.15.27:0/216419963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08801604d0 msgr2=0x7f08800b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.608+0000 7f088602c640 1 --2- 172.21.15.27:0/216419963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08801604d0 0x7f08800b47e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f0870009e50 tx=0x7f087002bf60 comp rx=0 tx=0).stop 2023-12-19T18:21:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.608+0000 7f088602c640 1 -- 172.21.15.27:0/216419963 shutdown_connections 2023-12-19T18:21:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.608+0000 7f088602c640 1 --2- 172.21.15.27:0/216419963 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08800b7310 0x7f08800b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.608+0000 7f088602c640 1 --2- 172.21.15.27:0/216419963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08801604d0 0x7f08800b47e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.608+0000 7f088602c640 1 -- 172.21.15.27:0/216419963 >> 172.21.15.27:0/216419963 conn(0x7f08800cf7e0 msgr2=0x7f08800cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.608+0000 7f088602c640 1 -- 172.21.15.27:0/216419963 shutdown_connections 2023-12-19T18:21:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.608+0000 7f088602c640 1 -- 172.21.15.27:0/216419963 wait complete. 2023-12-19T18:21:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.610+0000 7f088602c640 1 Processor -- start 2023-12-19T18:21:13.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.610+0000 7f088602c640 1 -- start start 2023-12-19T18:21:13.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.611+0000 7f088602c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800b7310 0x7f08800cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:13.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.611+0000 7f088602c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08800d2350 0x7f08801f20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:13.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.611+0000 7f088602c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08801604d0 0x7f0880156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:13.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.611+0000 7f088602c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08801f2600 con 0x7f08800b7310 2023-12-19T18:21:13.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.611+0000 7f088602c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0880156790 con 0x7f08800d2350 2023-12-19T18:21:13.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.611+0000 7f088602c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08800c9690 con 0x7f08801604d0 2023-12-19T18:21:13.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.611+0000 7f088502a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800b7310 0x7f08800cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:13.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.611+0000 7f088502a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800b7310 0x7f08800cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35196/0 (socket says 172.21.15.27:35196) 2023-12-19T18:21:13.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.611+0000 7f088502a640 1 -- 172.21.15.27:0/201918355 learned_addr learned my addr 172.21.15.27:0/201918355 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:13.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.611+0000 7f0884829640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08800d2350 0x7f08801f20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:13.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.612+0000 7f088582b640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08801604d0 0x7f0880156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:13.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.612+0000 7f088502a640 1 -- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08801604d0 msgr2=0x7f0880156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.612+0000 7f088502a640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08801604d0 0x7f0880156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.612+0000 7f088502a640 1 -- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08800d2350 msgr2=0x7f08801f20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.612+0000 7f088502a640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08800d2350 0x7f08801f20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.612+0000 7f088502a640 1 -- 172.21.15.27:0/201918355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0870009b30 con 0x7f08800b7310 2023-12-19T18:21:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.612+0000 7f088502a640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800b7310 0x7f08800cae80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f0870000f80 tx=0x7f0870004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.612+0000 7f087dffb640 1 -- 172.21.15.27:0/201918355 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870009510 con 0x7f08800b7310 2023-12-19T18:21:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.612+0000 7f087dffb640 1 -- 172.21.15.27:0/201918355 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087003e450 con 0x7f08800b7310 2023-12-19T18:21:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.612+0000 7f087dffb640 1 -- 172.21.15.27:0/201918355 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f087003e800 con 0x7f08800b7310 2023-12-19T18:21:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.612+0000 7f088602c640 1 -- 172.21.15.27:0/201918355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08800c6910 con 0x7f08800b7310 2023-12-19T18:21:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.613+0000 7f088602c640 1 -- 172.21.15.27:0/201918355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08800c6ed0 con 0x7f08800b7310 2023-12-19T18:21:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.614+0000 7f088602c640 1 -- 172.21.15.27:0/201918355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0880066f80 con 0x7f08800b7310 2023-12-19T18:21:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.615+0000 7f087dffb640 1 -- 172.21.15.27:0/201918355 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f087003e960 con 0x7f08800b7310 2023-12-19T18:21:13.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.616+0000 7f087dffb640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f086c06a430 0x7f086c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:13.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.616+0000 7f087dffb640 1 -- 172.21.15.27:0/201918355 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f08700ab670 con 0x7f08800b7310 2023-12-19T18:21:13.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.617+0000 7f0884829640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f086c06a430 0x7f086c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:13.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.618+0000 7f0884829640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f086c06a430 0x7f086c06c8f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f08801561e0 tx=0x7f0874009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:13.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.621+0000 7f087dffb640 1 -- 172.21.15.27:0/201918355 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0870076cf0 con 0x7f08800b7310 2023-12-19T18:21:13.704 INFO:tasks.ceph.mds.c.smithi027.stderr:2023-12-19T18:21:13.704+0000 7f2f32e9a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:21:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.816+0000 7f088602c640 1 -- 172.21.15.27:0/201918355 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f08800ce1b0 con 0x7f086c06a430 2023-12-19T18:21:13.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.844+0000 7f087dffb640 1 -- 172.21.15.27:0/201918355 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282902 (secure 0 0 0) 0x7f08740050d0 con 0x7f086c06a430 2023-12-19T18:21:13.846 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:21:13.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.851+0000 7f088602c640 1 -- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f086c06a430 msgr2=0x7f086c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:13.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.851+0000 7f088602c640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f086c06a430 0x7f086c06c8f0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f08801561e0 tx=0x7f0874009040 comp rx=0 tx=0).stop 2023-12-19T18:21:13.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.851+0000 7f088602c640 1 -- 172.21.15.27:0/201918355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800b7310 msgr2=0x7f08800cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:13.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.851+0000 7f088602c640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800b7310 0x7f08800cae80 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f0870000f80 tx=0x7f0870004a40 comp rx=0 tx=0).stop 2023-12-19T18:21:13.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.852+0000 7f088602c640 1 -- 172.21.15.27:0/201918355 shutdown_connections 2023-12-19T18:21:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.852+0000 7f088602c640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f086c06a430 0x7f086c06c8f0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.852+0000 7f088602c640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08801604d0 0x7f0880156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.852+0000 7f088602c640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08800d2350 0x7f08801f20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.852+0000 7f088602c640 1 --2- 172.21.15.27:0/201918355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800b7310 0x7f08800cae80 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.852+0000 7f088602c640 1 -- 172.21.15.27:0/201918355 >> 172.21.15.27:0/201918355 conn(0x7f08800cf7e0 msgr2=0x7f08800b77d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.852+0000 7f088602c640 1 -- 172.21.15.27:0/201918355 shutdown_connections 2023-12-19T18:21:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:13.852+0000 7f088602c640 1 -- 172.21.15.27:0/201918355 wait complete. 2023-12-19T18:21:13.876 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":303,"stamp":"2023-12-19T18:21:11.839801+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6149,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1556350,"statfs":{"total":1600000000,"available":1593703010,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.078714"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.316093+0000","last_change":"2023-12-19T18:21:08.315962+0000","last_active":"2023-12-19T18:21:08.316093+0000","last_peered":"2023-12-19T18:21:08.316093+0000","last_clean":"2023-12-19T18:21:08.316093+0000","last_became_active":"2023-12-19T18:21:08.315687+0000","last_became_peered":"2023-12-19T18:21:08.315687+0000","last_unstale":"2023-12-19T18:21:08.316093+0000","last_undegraded":"2023-12-19T18:21:08.316093+0000","last_fullsized":"2023-12-19T18:21:08.316093+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:12.815660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.309059+0000","last_change":"2023-12-19T18:21:08.307792+0000","last_active":"2023-12-19T18:21:08.309059+0000","last_peered":"2023-12-19T18:21:08.309059+0000","last_clean":"2023-12-19T18:21:08.309059+0000","last_became_active":"2023-12-19T18:21:08.307119+0000","last_became_peered":"2023-12-19T18:21:08.307119+0000","last_unstale":"2023-12-19T18:21:08.309059+0000","last_undegraded":"2023-12-19T18:21:08.309059+0000","last_fullsized":"2023-12-19T18:21:08.309059+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:34.078946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.308268+0000","last_change":"2023-12-19T18:21:08.307769+0000","last_active":"2023-12-19T18:21:08.308268+0000","last_peered":"2023-12-19T18:21:08.308268+0000","last_clean":"2023-12-19T18:21:08.308268+0000","last_became_active":"2023-12-19T18:21:08.307451+0000","last_became_peered":"2023-12-19T18:21:08.307451+0000","last_unstale":"2023-12-19T18:21:08.308268+0000","last_undegraded":"2023-12-19T18:21:08.308268+0000","last_fullsized":"2023-12-19T18:21:08.308268+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:09.791993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.307335+0000","last_change":"2023-12-19T18:21:08.304846+0000","last_active":"2023-12-19T18:21:08.307335+0000","last_peered":"2023-12-19T18:21:08.307335+0000","last_clean":"2023-12-19T18:21:08.307335+0000","last_became_active":"2023-12-19T18:21:08.304222+0000","last_became_peered":"2023-12-19T18:21:08.304222+0000","last_unstale":"2023-12-19T18:21:08.307335+0000","last_undegraded":"2023-12-19T18:21:08.307335+0000","last_fullsized":"2023-12-19T18:21:08.307335+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:20.930922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.318137+0000","last_change":"2023-12-19T18:21:08.317815+0000","last_active":"2023-12-19T18:21:08.318137+0000","last_peered":"2023-12-19T18:21:08.318137+0000","last_clean":"2023-12-19T18:21:08.318137+0000","last_became_active":"2023-12-19T18:21:08.317572+0000","last_became_peered":"2023-12-19T18:21:08.317572+0000","last_unstale":"2023-12-19T18:21:08.318137+0000","last_undegraded":"2023-12-19T18:21:08.318137+0000","last_fullsized":"2023-12-19T18:21:08.318137+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:08:36.344800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.320955+0000","last_change":"2023-12-19T18:21:08.320559+0000","last_active":"2023-12-19T18:21:08.320955+0000","last_peered":"2023-12-19T18:21:08.320955+0000","last_clean":"2023-12-19T18:21:08.320955+0000","last_became_active":"2023-12-19T18:21:08.320189+0000","last_became_peered":"2023-12-19T18:21:08.320189+0000","last_unstale":"2023-12-19T18:21:08.320955+0000","last_undegraded":"2023-12-19T18:21:08.320955+0000","last_fullsized":"2023-12-19T18:21:08.320955+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:35:32.513844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.306160+0000","last_change":"2023-12-19T18:21:08.305650+0000","last_active":"2023-12-19T18:21:08.306160+0000","last_peered":"2023-12-19T18:21:08.306160+0000","last_clean":"2023-12-19T18:21:08.306160+0000","last_became_active":"2023-12-19T18:21:08.304970+0000","last_became_peered":"2023-12-19T18:21:08.304970+0000","last_unstale":"2023-12-19T18:21:08.306160+0000","last_undegraded":"2023-12-19T18:21:08.306160+0000","last_fullsized":"2023-12-19T18:21:08.306160+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:03:03.110026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.319497+0000","last_change":"2023-12-19T18:21:08.318243+0000","last_active":"2023-12-19T18:21:08.319497+0000","last_peered":"2023-12-19T18:21:08.319497+0000","last_clean":"2023-12-19T18:21:08.319497+0000","last_became_active":"2023-12-19T18:21:08.317599+0000","last_became_peered":"2023-12-19T18:21:08.317599+0000","last_unstale":"2023-12-19T18:21:08.319497+0000","last_undegraded":"2023-12-19T18:21:08.319497+0000","last_fullsized":"2023-12-19T18:21:08.319497+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:39:56.350341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.307160+0000","last_change":"2023-12-19T18:21:08.306932+0000","last_active":"2023-12-19T18:21:08.307160+0000","last_peered":"2023-12-19T18:21:08.307160+0000","last_clean":"2023-12-19T18:21:08.307160+0000","last_became_active":"2023-12-19T18:21:08.306706+0000","last_became_peered":"2023-12-19T18:21:08.306706+0000","last_unstale":"2023-12-19T18:21:08.307160+0000","last_undegraded":"2023-12-19T18:21:08.307160+0000","last_fullsized":"2023-12-19T18:21:08.307160+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:00.134574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.315905+0000","last_change":"2023-12-19T18:21:08.315830+0000","last_active":"2023-12-19T18:21:08.315905+0000","last_peered":"2023-12-19T18:21:08.315905+0000","last_clean":"2023-12-19T18:21:08.315905+0000","last_became_active":"2023-12-19T18:21:08.315594+0000","last_became_peered":"2023-12-19T18:21:08.315594+0000","last_unstale":"2023-12-19T18:21:08.315905+0000","last_undegraded":"2023-12-19T18:21:08.315905+0000","last_fullsized":"2023-12-19T18:21:08.315905+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:46:01.407622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.322218+0000","last_change":"2023-12-19T18:21:08.321948+0000","last_active":"2023-12-19T18:21:08.322218+0000","last_peered":"2023-12-19T18:21:08.322218+0000","last_clean":"2023-12-19T18:21:08.322218+0000","last_became_active":"2023-12-19T18:21:08.318810+0000","last_became_peered":"2023-12-19T18:21:08.318810+0000","last_unstale":"2023-12-19T18:21:08.322218+0000","last_undegraded":"2023-12-19T18:21:08.322218+0000","last_fullsized":"2023-12-19T18:21:08.322218+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:47.329029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.317957+0000","last_change":"2023-12-19T18:21:08.317788+0000","last_active":"2023-12-19T18:21:08.317957+0000","last_peered":"2023-12-19T18:21:08.317957+0000","last_clean":"2023-12-19T18:21:08.317957+0000","last_became_active":"2023-12-19T18:21:08.317551+0000","last_became_peered":"2023-12-19T18:21:08.317551+0000","last_unstale":"2023-12-19T18:21:08.317957+0000","last_undegraded":"2023-12-19T18:21:08.317957+0000","last_fullsized":"2023-12-19T18:21:08.317957+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:01.378947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.308305+0000","last_change":"2023-12-19T18:21:08.308162+0000","last_active":"2023-12-19T18:21:08.308305+0000","last_peered":"2023-12-19T18:21:08.308305+0000","last_clean":"2023-12-19T18:21:08.308305+0000","last_became_active":"2023-12-19T18:21:08.304144+0000","last_became_peered":"2023-12-19T18:21:08.304144+0000","last_unstale":"2023-12-19T18:21:08.308305+0000","last_undegraded":"2023-12-19T18:21:08.308305+0000","last_fullsized":"2023-12-19T18:21:08.308305+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:32:20.010608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.321004+0000","last_change":"2023-12-19T18:21:08.320714+0000","last_active":"2023-12-19T18:21:08.321004+0000","last_peered":"2023-12-19T18:21:08.321004+0000","last_clean":"2023-12-19T18:21:08.321004+0000","last_became_active":"2023-12-19T18:21:08.317148+0000","last_became_peered":"2023-12-19T18:21:08.317148+0000","last_unstale":"2023-12-19T18:21:08.321004+0000","last_undegraded":"2023-12-19T18:21:08.321004+0000","last_fullsized":"2023-12-19T18:21:08.321004+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:04.632373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.303406+0000","last_change":"2023-12-19T18:21:08.302966+0000","last_active":"2023-12-19T18:21:08.303406+0000","last_peered":"2023-12-19T18:21:08.303406+0000","last_clean":"2023-12-19T18:21:08.303406+0000","last_became_active":"2023-12-19T18:21:08.302378+0000","last_became_peered":"2023-12-19T18:21:08.302378+0000","last_unstale":"2023-12-19T18:21:08.303406+0000","last_undegraded":"2023-12-19T18:21:08.303406+0000","last_fullsized":"2023-12-19T18:21:08.303406+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:38:31.365048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.314650+0000","last_change":"2023-12-19T18:21:08.314536+0000","last_active":"2023-12-19T18:21:08.314650+0000","last_peered":"2023-12-19T18:21:08.314650+0000","last_clean":"2023-12-19T18:21:08.314650+0000","last_became_active":"2023-12-19T18:21:08.314264+0000","last_became_peered":"2023-12-19T18:21:08.314264+0000","last_unstale":"2023-12-19T18:21:08.314650+0000","last_undegraded":"2023-12-19T18:21:08.314650+0000","last_fullsized":"2023-12-19T18:21:08.314650+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:37.658754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.314344+0000","last_change":"2023-12-19T18:21:08.314264+0000","last_active":"2023-12-19T18:21:08.314344+0000","last_peered":"2023-12-19T18:21:08.314344+0000","last_clean":"2023-12-19T18:21:08.314344+0000","last_became_active":"2023-12-19T18:21:08.314020+0000","last_became_peered":"2023-12-19T18:21:08.314020+0000","last_unstale":"2023-12-19T18:21:08.314344+0000","last_undegraded":"2023-12-19T18:21:08.314344+0000","last_fullsized":"2023-12-19T18:21:08.314344+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:14:53.195396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.321029+0000","last_change":"2023-12-19T18:21:08.314404+0000","last_active":"2023-12-19T18:21:08.321029+0000","last_peered":"2023-12-19T18:21:08.321029+0000","last_clean":"2023-12-19T18:21:08.321029+0000","last_became_active":"2023-12-19T18:21:08.313869+0000","last_became_peered":"2023-12-19T18:21:08.313869+0000","last_unstale":"2023-12-19T18:21:08.321029+0000","last_undegraded":"2023-12-19T18:21:08.321029+0000","last_fullsized":"2023-12-19T18:21:08.321029+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:04.632373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.324202+0000","last_change":"2023-12-19T18:21:08.321406+0000","last_active":"2023-12-19T18:21:08.324202+0000","last_peered":"2023-12-19T18:21:08.324202+0000","last_clean":"2023-12-19T18:21:08.324202+0000","last_became_active":"2023-12-19T18:21:08.320496+0000","last_became_peered":"2023-12-19T18:21:08.320496+0000","last_unstale":"2023-12-19T18:21:08.324202+0000","last_undegraded":"2023-12-19T18:21:08.324202+0000","last_fullsized":"2023-12-19T18:21:08.324202+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:38:31.365048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.325068+0000","last_change":"2023-12-19T18:21:08.324976+0000","last_active":"2023-12-19T18:21:08.325068+0000","last_peered":"2023-12-19T18:21:08.325068+0000","last_clean":"2023-12-19T18:21:08.325068+0000","last_became_active":"2023-12-19T18:21:08.322809+0000","last_became_peered":"2023-12-19T18:21:08.322809+0000","last_unstale":"2023-12-19T18:21:08.325068+0000","last_undegraded":"2023-12-19T18:21:08.325068+0000","last_fullsized":"2023-12-19T18:21:08.325068+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:14:03.228122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.319788+0000","last_change":"2023-12-19T18:21:08.318225+0000","last_active":"2023-12-19T18:21:08.319788+0000","last_peered":"2023-12-19T18:21:08.319788+0000","last_clean":"2023-12-19T18:21:08.319788+0000","last_became_active":"2023-12-19T18:21:08.313899+0000","last_became_peered":"2023-12-19T18:21:08.313899+0000","last_unstale":"2023-12-19T18:21:08.319788+0000","last_undegraded":"2023-12-19T18:21:08.319788+0000","last_fullsized":"2023-12-19T18:21:08.319788+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:17:45.304080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.318688+0000","last_change":"2023-12-19T18:21:08.317509+0000","last_active":"2023-12-19T18:21:08.318688+0000","last_peered":"2023-12-19T18:21:08.318688+0000","last_clean":"2023-12-19T18:21:08.318688+0000","last_became_active":"2023-12-19T18:21:08.314334+0000","last_became_peered":"2023-12-19T18:21:08.314334+0000","last_unstale":"2023-12-19T18:21:08.318688+0000","last_undegraded":"2023-12-19T18:21:08.318688+0000","last_fullsized":"2023-12-19T18:21:08.318688+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:28:44.357084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.319515+0000","last_change":"2023-12-19T18:21:08.318390+0000","last_active":"2023-12-19T18:21:08.319515+0000","last_peered":"2023-12-19T18:21:08.319515+0000","last_clean":"2023-12-19T18:21:08.319515+0000","last_became_active":"2023-12-19T18:21:08.318011+0000","last_became_peered":"2023-12-19T18:21:08.318011+0000","last_unstale":"2023-12-19T18:21:08.319515+0000","last_undegraded":"2023-12-19T18:21:08.319515+0000","last_fullsized":"2023-12-19T18:21:08.319515+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:21:20.647321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.321270+0000","last_change":"2023-12-19T18:21:08.317824+0000","last_active":"2023-12-19T18:21:08.321270+0000","last_peered":"2023-12-19T18:21:08.321270+0000","last_clean":"2023-12-19T18:21:08.321270+0000","last_became_active":"2023-12-19T18:21:08.316205+0000","last_became_peered":"2023-12-19T18:21:08.316205+0000","last_unstale":"2023-12-19T18:21:08.321270+0000","last_undegraded":"2023-12-19T18:21:08.321270+0000","last_fullsized":"2023-12-19T18:21:08.321270+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:28:44.357084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.321297+0000","last_change":"2023-12-19T18:21:08.318883+0000","last_active":"2023-12-19T18:21:08.321297+0000","last_peered":"2023-12-19T18:21:08.321297+0000","last_clean":"2023-12-19T18:21:08.321297+0000","last_became_active":"2023-12-19T18:21:08.318645+0000","last_became_peered":"2023-12-19T18:21:08.318645+0000","last_unstale":"2023-12-19T18:21:08.321297+0000","last_undegraded":"2023-12-19T18:21:08.321297+0000","last_fullsized":"2023-12-19T18:21:08.321297+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:34:03.699275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.309909+0000","last_change":"2023-12-19T18:21:08.309412+0000","last_active":"2023-12-19T18:21:08.309909+0000","last_peered":"2023-12-19T18:21:08.309909+0000","last_clean":"2023-12-19T18:21:08.309909+0000","last_became_active":"2023-12-19T18:21:08.309195+0000","last_became_peered":"2023-12-19T18:21:08.309195+0000","last_unstale":"2023-12-19T18:21:08.309909+0000","last_undegraded":"2023-12-19T18:21:08.309909+0000","last_fullsized":"2023-12-19T18:21:08.309909+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:29:36.782413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.295770+0000","last_change":"2023-12-19T18:21:08.295667+0000","last_active":"2023-12-19T18:21:08.295770+0000","last_peered":"2023-12-19T18:21:08.295770+0000","last_clean":"2023-12-19T18:21:08.295770+0000","last_became_active":"2023-12-19T18:21:08.294934+0000","last_became_peered":"2023-12-19T18:21:08.294934+0000","last_unstale":"2023-12-19T18:21:08.295770+0000","last_undegraded":"2023-12-19T18:21:08.295770+0000","last_fullsized":"2023-12-19T18:21:08.295770+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:00.470255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.299903+0000","last_change":"2023-12-19T18:21:08.299764+0000","last_active":"2023-12-19T18:21:08.299903+0000","last_peered":"2023-12-19T18:21:08.299903+0000","last_clean":"2023-12-19T18:21:08.299903+0000","last_became_active":"2023-12-19T18:21:08.297643+0000","last_became_peered":"2023-12-19T18:21:08.297643+0000","last_unstale":"2023-12-19T18:21:08.299903+0000","last_undegraded":"2023-12-19T18:21:08.299903+0000","last_fullsized":"2023-12-19T18:21:08.299903+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:22:02.320818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.300140+0000","last_change":"2023-12-19T18:21:08.299950+0000","last_active":"2023-12-19T18:21:08.300140+0000","last_peered":"2023-12-19T18:21:08.300140+0000","last_clean":"2023-12-19T18:21:08.300140+0000","last_became_active":"2023-12-19T18:21:08.298315+0000","last_became_peered":"2023-12-19T18:21:08.298315+0000","last_unstale":"2023-12-19T18:21:08.300140+0000","last_undegraded":"2023-12-19T18:21:08.300140+0000","last_fullsized":"2023-12-19T18:21:08.300140+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:00.301926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.324812+0000","last_change":"2023-12-19T18:21:08.323087+0000","last_active":"2023-12-19T18:21:08.324812+0000","last_peered":"2023-12-19T18:21:08.324812+0000","last_clean":"2023-12-19T18:21:08.324812+0000","last_became_active":"2023-12-19T18:21:08.322460+0000","last_became_peered":"2023-12-19T18:21:08.322460+0000","last_unstale":"2023-12-19T18:21:08.324812+0000","last_undegraded":"2023-12-19T18:21:08.324812+0000","last_fullsized":"2023-12-19T18:21:08.324812+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:34.078946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.320937+0000","last_change":"2023-12-19T18:21:08.320405+0000","last_active":"2023-12-19T18:21:08.320937+0000","last_peered":"2023-12-19T18:21:08.320937+0000","last_clean":"2023-12-19T18:21:08.320937+0000","last_became_active":"2023-12-19T18:21:08.319880+0000","last_became_peered":"2023-12-19T18:21:08.319880+0000","last_unstale":"2023-12-19T18:21:08.320937+0000","last_undegraded":"2023-12-19T18:21:08.320937+0000","last_fullsized":"2023-12-19T18:21:08.320937+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:06.368856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.313009+0000","last_change":"2023-12-19T18:21:08.311544+0000","last_active":"2023-12-19T18:21:08.313009+0000","last_peered":"2023-12-19T18:21:08.313009+0000","last_clean":"2023-12-19T18:21:08.313009+0000","last_became_active":"2023-12-19T18:21:08.311201+0000","last_became_peered":"2023-12-19T18:21:08.311201+0000","last_unstale":"2023-12-19T18:21:08.313009+0000","last_undegraded":"2023-12-19T18:21:08.313009+0000","last_fullsized":"2023-12-19T18:21:08.313009+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:23.066899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.303824+0000","last_change":"2023-12-19T18:21:08.303092+0000","last_active":"2023-12-19T18:21:08.303824+0000","last_peered":"2023-12-19T18:21:08.303824+0000","last_clean":"2023-12-19T18:21:08.303824+0000","last_became_active":"2023-12-19T18:21:08.302857+0000","last_became_peered":"2023-12-19T18:21:08.302857+0000","last_unstale":"2023-12-19T18:21:08.303824+0000","last_undegraded":"2023-12-19T18:21:08.303824+0000","last_fullsized":"2023-12-19T18:21:08.303824+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:50.191686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"116'4","reported_seq":14,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.326650+0000","last_change":"2023-12-19T18:21:07.806330+0000","last_active":"2023-12-19T18:21:08.326650+0000","last_peered":"2023-12-19T18:21:08.326650+0000","last_clean":"2023-12-19T18:21:08.326650+0000","last_became_active":"2023-12-19T18:21:07.806113+0000","last_became_peered":"2023-12-19T18:21:07.806113+0000","last_unstale":"2023-12-19T18:21:08.326650+0000","last_undegraded":"2023-12-19T18:21:08.326650+0000","last_fullsized":"2023-12-19T18:21:08.326650+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:48:25.943021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.306443+0000","last_change":"2023-12-19T18:21:08.306367+0000","last_active":"2023-12-19T18:21:08.306443+0000","last_peered":"2023-12-19T18:21:08.306443+0000","last_clean":"2023-12-19T18:21:08.306443+0000","last_became_active":"2023-12-19T18:21:08.303918+0000","last_became_peered":"2023-12-19T18:21:08.303918+0000","last_unstale":"2023-12-19T18:21:08.306443+0000","last_undegraded":"2023-12-19T18:21:08.306443+0000","last_fullsized":"2023-12-19T18:21:08.306443+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:38.893277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.313224+0000","last_change":"2023-12-19T18:21:07.796271+0000","last_active":"2023-12-19T18:21:08.313224+0000","last_peered":"2023-12-19T18:21:08.313224+0000","last_clean":"2023-12-19T18:21:08.313224+0000","last_became_active":"2023-12-19T18:21:07.795997+0000","last_became_peered":"2023-12-19T18:21:07.795997+0000","last_unstale":"2023-12-19T18:21:08.313224+0000","last_undegraded":"2023-12-19T18:21:08.313224+0000","last_fullsized":"2023-12-19T18:21:08.313224+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:33.067212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.305957+0000","last_change":"2023-12-19T18:21:08.305888+0000","last_active":"2023-12-19T18:21:08.305957+0000","last_peered":"2023-12-19T18:21:08.305957+0000","last_clean":"2023-12-19T18:21:08.305957+0000","last_became_active":"2023-12-19T18:21:08.305418+0000","last_became_peered":"2023-12-19T18:21:08.305418+0000","last_unstale":"2023-12-19T18:21:08.305957+0000","last_undegraded":"2023-12-19T18:21:08.305957+0000","last_fullsized":"2023-12-19T18:21:08.305957+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:39:13.992641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.323742+0000","last_change":"2023-12-19T18:21:07.806193+0000","last_active":"2023-12-19T18:21:08.323742+0000","last_peered":"2023-12-19T18:21:08.323742+0000","last_clean":"2023-12-19T18:21:08.323742+0000","last_became_active":"2023-12-19T18:21:07.805886+0000","last_became_peered":"2023-12-19T18:21:07.805886+0000","last_unstale":"2023-12-19T18:21:08.323742+0000","last_undegraded":"2023-12-19T18:21:08.323742+0000","last_fullsized":"2023-12-19T18:21:08.323742+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:54:01.049760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.323427+0000","last_change":"2023-12-19T18:21:08.323372+0000","last_active":"2023-12-19T18:21:08.323427+0000","last_peered":"2023-12-19T18:21:08.323427+0000","last_clean":"2023-12-19T18:21:08.323427+0000","last_became_active":"2023-12-19T18:21:08.322260+0000","last_became_peered":"2023-12-19T18:21:08.322260+0000","last_unstale":"2023-12-19T18:21:08.323427+0000","last_undegraded":"2023-12-19T18:21:08.323427+0000","last_fullsized":"2023-12-19T18:21:08.323427+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:54:16.721327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.325642+0000","last_change":"2023-12-19T18:21:07.801223+0000","last_active":"2023-12-19T18:21:08.325642+0000","last_peered":"2023-12-19T18:21:08.325642+0000","last_clean":"2023-12-19T18:21:08.325642+0000","last_became_active":"2023-12-19T18:21:07.800988+0000","last_became_peered":"2023-12-19T18:21:07.800988+0000","last_unstale":"2023-12-19T18:21:08.325642+0000","last_undegraded":"2023-12-19T18:21:08.325642+0000","last_fullsized":"2023-12-19T18:21:08.325642+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:45:08.780259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.320712+0000","last_change":"2023-12-19T18:21:08.320228+0000","last_active":"2023-12-19T18:21:08.320712+0000","last_peered":"2023-12-19T18:21:08.320712+0000","last_clean":"2023-12-19T18:21:08.320712+0000","last_became_active":"2023-12-19T18:21:08.320004+0000","last_became_peered":"2023-12-19T18:21:08.320004+0000","last_unstale":"2023-12-19T18:21:08.320712+0000","last_undegraded":"2023-12-19T18:21:08.320712+0000","last_fullsized":"2023-12-19T18:21:08.320712+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:35:08.910728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.311670+0000","last_change":"2023-12-19T18:21:07.800300+0000","last_active":"2023-12-19T18:21:08.311670+0000","last_peered":"2023-12-19T18:21:08.311670+0000","last_clean":"2023-12-19T18:21:08.311670+0000","last_became_active":"2023-12-19T18:21:07.794856+0000","last_became_peered":"2023-12-19T18:21:07.794856+0000","last_unstale":"2023-12-19T18:21:08.311670+0000","last_undegraded":"2023-12-19T18:21:08.311670+0000","last_fullsized":"2023-12-19T18:21:08.311670+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:34.703586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.319370+0000","last_change":"2023-12-19T18:21:08.318474+0000","last_active":"2023-12-19T18:21:08.319370+0000","last_peered":"2023-12-19T18:21:08.319370+0000","last_clean":"2023-12-19T18:21:08.319370+0000","last_became_active":"2023-12-19T18:21:08.318213+0000","last_became_peered":"2023-12-19T18:21:08.318213+0000","last_unstale":"2023-12-19T18:21:08.319370+0000","last_undegraded":"2023-12-19T18:21:08.319370+0000","last_fullsized":"2023-12-19T18:21:08.319370+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:08:26.816978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.315261+0000","last_change":"2023-12-19T18:21:07.791887+0000","last_active":"2023-12-19T18:21:08.315261+0000","last_peered":"2023-12-19T18:21:08.315261+0000","last_clean":"2023-12-19T18:21:08.315261+0000","last_became_active":"2023-12-19T18:21:07.791173+0000","last_became_peered":"2023-12-19T18:21:07.791173+0000","last_unstale":"2023-12-19T18:21:08.315261+0000","last_undegraded":"2023-12-19T18:21:08.315261+0000","last_fullsized":"2023-12-19T18:21:08.315261+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:01:54.458894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.323506+0000","last_change":"2023-12-19T18:21:08.323464+0000","last_active":"2023-12-19T18:21:08.323506+0000","last_peered":"2023-12-19T18:21:08.323506+0000","last_clean":"2023-12-19T18:21:08.323506+0000","last_became_active":"2023-12-19T18:21:08.322597+0000","last_became_peered":"2023-12-19T18:21:08.322597+0000","last_unstale":"2023-12-19T18:21:08.323506+0000","last_undegraded":"2023-12-19T18:21:08.323506+0000","last_fullsized":"2023-12-19T18:21:08.323506+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:44.485880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.310980+0000","last_change":"2023-12-19T18:21:07.802335+0000","last_active":"2023-12-19T18:21:08.310980+0000","last_peered":"2023-12-19T18:21:08.310980+0000","last_clean":"2023-12-19T18:21:08.310980+0000","last_became_active":"2023-12-19T18:21:07.802044+0000","last_became_peered":"2023-12-19T18:21:07.802044+0000","last_unstale":"2023-12-19T18:21:08.310980+0000","last_undegraded":"2023-12-19T18:21:08.310980+0000","last_fullsized":"2023-12-19T18:21:08.310980+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:47:59.122840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.316864+0000","last_change":"2023-12-19T18:21:08.315294+0000","last_active":"2023-12-19T18:21:08.316864+0000","last_peered":"2023-12-19T18:21:08.316864+0000","last_clean":"2023-12-19T18:21:08.316864+0000","last_became_active":"2023-12-19T18:21:08.315003+0000","last_became_peered":"2023-12-19T18:21:08.315003+0000","last_unstale":"2023-12-19T18:21:08.316864+0000","last_undegraded":"2023-12-19T18:21:08.316864+0000","last_fullsized":"2023-12-19T18:21:08.316864+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:03:03.110026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.312570+0000","last_change":"2023-12-19T18:21:07.804454+0000","last_active":"2023-12-19T18:21:08.312570+0000","last_peered":"2023-12-19T18:21:08.312570+0000","last_clean":"2023-12-19T18:21:08.312570+0000","last_became_active":"2023-12-19T18:21:07.803328+0000","last_became_peered":"2023-12-19T18:21:07.803328+0000","last_unstale":"2023-12-19T18:21:08.312570+0000","last_undegraded":"2023-12-19T18:21:08.312570+0000","last_fullsized":"2023-12-19T18:21:08.312570+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:28.405924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.318670+0000","last_change":"2023-12-19T18:21:08.317326+0000","last_active":"2023-12-19T18:21:08.318670+0000","last_peered":"2023-12-19T18:21:08.318670+0000","last_clean":"2023-12-19T18:21:08.318670+0000","last_became_active":"2023-12-19T18:21:08.314030+0000","last_became_peered":"2023-12-19T18:21:08.314030+0000","last_unstale":"2023-12-19T18:21:08.318670+0000","last_undegraded":"2023-12-19T18:21:08.318670+0000","last_fullsized":"2023-12-19T18:21:08.318670+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:47.329029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.294355+0000","last_change":"2023-12-19T18:21:07.798583+0000","last_active":"2023-12-19T18:21:08.294355+0000","last_peered":"2023-12-19T18:21:08.294355+0000","last_clean":"2023-12-19T18:21:08.294355+0000","last_became_active":"2023-12-19T18:21:07.798192+0000","last_became_peered":"2023-12-19T18:21:07.798192+0000","last_unstale":"2023-12-19T18:21:08.294355+0000","last_undegraded":"2023-12-19T18:21:08.294355+0000","last_fullsized":"2023-12-19T18:21:08.294355+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:32:03.257881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.322957+0000","last_change":"2023-12-19T18:21:08.322737+0000","last_active":"2023-12-19T18:21:08.322957+0000","last_peered":"2023-12-19T18:21:08.322957+0000","last_clean":"2023-12-19T18:21:08.322957+0000","last_became_active":"2023-12-19T18:21:08.321951+0000","last_became_peered":"2023-12-19T18:21:08.321951+0000","last_unstale":"2023-12-19T18:21:08.322957+0000","last_undegraded":"2023-12-19T18:21:08.322957+0000","last_fullsized":"2023-12-19T18:21:08.322957+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:54:02.061494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.300998+0000","last_change":"2023-12-19T18:21:07.795155+0000","last_active":"2023-12-19T18:21:08.300998+0000","last_peered":"2023-12-19T18:21:08.300998+0000","last_clean":"2023-12-19T18:21:08.300998+0000","last_became_active":"2023-12-19T18:21:07.794472+0000","last_became_peered":"2023-12-19T18:21:07.794472+0000","last_unstale":"2023-12-19T18:21:08.300998+0000","last_undegraded":"2023-12-19T18:21:08.300998+0000","last_fullsized":"2023-12-19T18:21:08.300998+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:14:02.216388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.321247+0000","last_change":"2023-12-19T18:21:08.317693+0000","last_active":"2023-12-19T18:21:08.321247+0000","last_peered":"2023-12-19T18:21:08.321247+0000","last_clean":"2023-12-19T18:21:08.321247+0000","last_became_active":"2023-12-19T18:21:08.315906+0000","last_became_peered":"2023-12-19T18:21:08.315906+0000","last_unstale":"2023-12-19T18:21:08.321247+0000","last_undegraded":"2023-12-19T18:21:08.321247+0000","last_fullsized":"2023-12-19T18:21:08.321247+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:10:34.078946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"116'3","reported_seq":13,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.322257+0000","last_change":"2023-12-19T18:21:07.805933+0000","last_active":"2023-12-19T18:21:08.322257+0000","last_peered":"2023-12-19T18:21:08.322257+0000","last_clean":"2023-12-19T18:21:08.322257+0000","last_became_active":"2023-12-19T18:21:07.805312+0000","last_became_peered":"2023-12-19T18:21:07.805312+0000","last_unstale":"2023-12-19T18:21:08.322257+0000","last_undegraded":"2023-12-19T18:21:08.322257+0000","last_fullsized":"2023-12-19T18:21:08.322257+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:01.573086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.318590+0000","last_change":"2023-12-19T18:21:08.318537+0000","last_active":"2023-12-19T18:21:08.318590+0000","last_peered":"2023-12-19T18:21:08.318590+0000","last_clean":"2023-12-19T18:21:08.318590+0000","last_became_active":"2023-12-19T18:21:08.317766+0000","last_became_peered":"2023-12-19T18:21:08.317766+0000","last_unstale":"2023-12-19T18:21:08.318590+0000","last_undegraded":"2023-12-19T18:21:08.318590+0000","last_fullsized":"2023-12-19T18:21:08.318590+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:42:30.571517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"116'6","reported_seq":16,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.322359+0000","last_change":"2023-12-19T18:21:07.799098+0000","last_active":"2023-12-19T18:21:08.322359+0000","last_peered":"2023-12-19T18:21:08.322359+0000","last_clean":"2023-12-19T18:21:08.322359+0000","last_became_active":"2023-12-19T18:21:07.794686+0000","last_became_peered":"2023-12-19T18:21:07.794686+0000","last_unstale":"2023-12-19T18:21:08.322359+0000","last_undegraded":"2023-12-19T18:21:08.322359+0000","last_fullsized":"2023-12-19T18:21:08.322359+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:03:28.205860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.321857+0000","last_change":"2023-12-19T18:21:08.321784+0000","last_active":"2023-12-19T18:21:08.321857+0000","last_peered":"2023-12-19T18:21:08.321857+0000","last_clean":"2023-12-19T18:21:08.321857+0000","last_became_active":"2023-12-19T18:21:08.318398+0000","last_became_peered":"2023-12-19T18:21:08.318398+0000","last_unstale":"2023-12-19T18:21:08.321857+0000","last_undegraded":"2023-12-19T18:21:08.321857+0000","last_fullsized":"2023-12-19T18:21:08.321857+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:12.792194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.325676+0000","last_change":"2023-12-19T18:21:07.793726+0000","last_active":"2023-12-19T18:21:08.325676+0000","last_peered":"2023-12-19T18:21:08.325676+0000","last_clean":"2023-12-19T18:21:08.325676+0000","last_became_active":"2023-12-19T18:21:07.792928+0000","last_became_peered":"2023-12-19T18:21:07.792928+0000","last_unstale":"2023-12-19T18:21:08.325676+0000","last_undegraded":"2023-12-19T18:21:08.325676+0000","last_fullsized":"2023-12-19T18:21:08.325676+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:41:34.768171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.322976+0000","last_change":"2023-12-19T18:21:08.322816+0000","last_active":"2023-12-19T18:21:08.322976+0000","last_peered":"2023-12-19T18:21:08.322976+0000","last_clean":"2023-12-19T18:21:08.322976+0000","last_became_active":"2023-12-19T18:21:08.321553+0000","last_became_peered":"2023-12-19T18:21:08.321553+0000","last_unstale":"2023-12-19T18:21:08.322976+0000","last_undegraded":"2023-12-19T18:21:08.322976+0000","last_fullsized":"2023-12-19T18:21:08.322976+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:29.417658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.293835+0000","last_change":"2023-12-19T18:21:07.800708+0000","last_active":"2023-12-19T18:21:08.293835+0000","last_peered":"2023-12-19T18:21:08.293835+0000","last_clean":"2023-12-19T18:21:08.293835+0000","last_became_active":"2023-12-19T18:21:07.800418+0000","last_became_peered":"2023-12-19T18:21:07.800418+0000","last_unstale":"2023-12-19T18:21:08.293835+0000","last_undegraded":"2023-12-19T18:21:08.293835+0000","last_fullsized":"2023-12-19T18:21:08.293835+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:13:56.765152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.317003+0000","last_change":"2023-12-19T18:21:08.316931+0000","last_active":"2023-12-19T18:21:08.317003+0000","last_peered":"2023-12-19T18:21:08.317003+0000","last_clean":"2023-12-19T18:21:08.317003+0000","last_became_active":"2023-12-19T18:21:08.316689+0000","last_became_peered":"2023-12-19T18:21:08.316689+0000","last_unstale":"2023-12-19T18:21:08.317003+0000","last_undegraded":"2023-12-19T18:21:08.317003+0000","last_fullsized":"2023-12-19T18:21:08.317003+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:34:03.699275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.301119+0000","last_change":"2023-12-19T18:21:07.799650+0000","last_active":"2023-12-19T18:21:08.301119+0000","last_peered":"2023-12-19T18:21:08.301119+0000","last_clean":"2023-12-19T18:21:08.301119+0000","last_became_active":"2023-12-19T18:21:07.799430+0000","last_became_peered":"2023-12-19T18:21:07.799430+0000","last_unstale":"2023-12-19T18:21:08.301119+0000","last_undegraded":"2023-12-19T18:21:08.301119+0000","last_fullsized":"2023-12-19T18:21:08.301119+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:56:46.317296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.320868+0000","last_change":"2023-12-19T18:21:08.314318+0000","last_active":"2023-12-19T18:21:08.320868+0000","last_peered":"2023-12-19T18:21:08.320868+0000","last_clean":"2023-12-19T18:21:08.320868+0000","last_became_active":"2023-12-19T18:21:08.313932+0000","last_became_peered":"2023-12-19T18:21:08.313932+0000","last_unstale":"2023-12-19T18:21:08.320868+0000","last_undegraded":"2023-12-19T18:21:08.320868+0000","last_fullsized":"2023-12-19T18:21:08.320868+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:03:03.110026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"116'8","reported_seq":18,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.327848+0000","last_change":"2023-12-19T18:21:07.807077+0000","last_active":"2023-12-19T18:21:08.327848+0000","last_peered":"2023-12-19T18:21:08.327848+0000","last_clean":"2023-12-19T18:21:08.327848+0000","last_became_active":"2023-12-19T18:21:07.805873+0000","last_became_peered":"2023-12-19T18:21:07.805873+0000","last_unstale":"2023-12-19T18:21:08.327848+0000","last_undegraded":"2023-12-19T18:21:08.327848+0000","last_fullsized":"2023-12-19T18:21:08.327848+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:22:01.309084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.301091+0000","last_change":"2023-12-19T18:21:08.301007+0000","last_active":"2023-12-19T18:21:08.301091+0000","last_peered":"2023-12-19T18:21:08.301091+0000","last_clean":"2023-12-19T18:21:08.301091+0000","last_became_active":"2023-12-19T18:21:08.300783+0000","last_became_peered":"2023-12-19T18:21:08.300783+0000","last_unstale":"2023-12-19T18:21:08.301091+0000","last_undegraded":"2023-12-19T18:21:08.301091+0000","last_fullsized":"2023-12-19T18:21:08.301091+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:03:03.110026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.315131+0000","last_change":"2023-12-19T18:21:07.803614+0000","last_active":"2023-12-19T18:21:08.315131+0000","last_peered":"2023-12-19T18:21:08.315131+0000","last_clean":"2023-12-19T18:21:08.315131+0000","last_became_active":"2023-12-19T18:21:07.803317+0000","last_became_peered":"2023-12-19T18:21:07.803317+0000","last_unstale":"2023-12-19T18:21:08.315131+0000","last_undegraded":"2023-12-19T18:21:08.315131+0000","last_fullsized":"2023-12-19T18:21:08.315131+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:22:15.124299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.313057+0000","last_change":"2023-12-19T18:21:08.311926+0000","last_active":"2023-12-19T18:21:08.313057+0000","last_peered":"2023-12-19T18:21:08.313057+0000","last_clean":"2023-12-19T18:21:08.313057+0000","last_became_active":"2023-12-19T18:21:08.311703+0000","last_became_peered":"2023-12-19T18:21:08.311703+0000","last_unstale":"2023-12-19T18:21:08.313057+0000","last_undegraded":"2023-12-19T18:21:08.313057+0000","last_fullsized":"2023-12-19T18:21:08.313057+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:41:35.779905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.322167+0000","last_change":"2023-12-19T18:21:07.804857+0000","last_active":"2023-12-19T18:21:08.322167+0000","last_peered":"2023-12-19T18:21:08.322167+0000","last_clean":"2023-12-19T18:21:08.322167+0000","last_became_active":"2023-12-19T18:21:07.804173+0000","last_became_peered":"2023-12-19T18:21:07.804173+0000","last_unstale":"2023-12-19T18:21:08.322167+0000","last_undegraded":"2023-12-19T18:21:08.322167+0000","last_fullsized":"2023-12-19T18:21:08.322167+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:22:01.309084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.301108+0000","last_change":"2023-12-19T18:21:08.301050+0000","last_active":"2023-12-19T18:21:08.301108+0000","last_peered":"2023-12-19T18:21:08.301108+0000","last_clean":"2023-12-19T18:21:08.301108+0000","last_became_active":"2023-12-19T18:21:08.300813+0000","last_became_peered":"2023-12-19T18:21:08.300813+0000","last_unstale":"2023-12-19T18:21:08.301108+0000","last_undegraded":"2023-12-19T18:21:08.301108+0000","last_fullsized":"2023-12-19T18:21:08.301108+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:28:44.357084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.308380+0000","last_change":"2023-12-19T18:21:07.799014+0000","last_active":"2023-12-19T18:21:08.308380+0000","last_peered":"2023-12-19T18:21:08.308380+0000","last_clean":"2023-12-19T18:21:08.308380+0000","last_became_active":"2023-12-19T18:21:07.798604+0000","last_became_peered":"2023-12-19T18:21:07.798604+0000","last_unstale":"2023-12-19T18:21:08.308380+0000","last_undegraded":"2023-12-19T18:21:08.308380+0000","last_fullsized":"2023-12-19T18:21:08.308380+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:37.881543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.323230+0000","last_change":"2023-12-19T18:21:08.323145+0000","last_active":"2023-12-19T18:21:08.323230+0000","last_peered":"2023-12-19T18:21:08.323230+0000","last_clean":"2023-12-19T18:21:08.323230+0000","last_became_active":"2023-12-19T18:21:08.322239+0000","last_became_peered":"2023-12-19T18:21:08.322239+0000","last_unstale":"2023-12-19T18:21:08.323230+0000","last_undegraded":"2023-12-19T18:21:08.323230+0000","last_fullsized":"2023-12-19T18:21:08.323230+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:28:44.357084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.323582+0000","last_change":"2023-12-19T18:21:07.803456+0000","last_active":"2023-12-19T18:21:08.323582+0000","last_peered":"2023-12-19T18:21:08.323582+0000","last_clean":"2023-12-19T18:21:08.323582+0000","last_became_active":"2023-12-19T18:21:07.803181+0000","last_became_peered":"2023-12-19T18:21:07.803181+0000","last_unstale":"2023-12-19T18:21:08.323582+0000","last_undegraded":"2023-12-19T18:21:08.323582+0000","last_fullsized":"2023-12-19T18:21:08.323582+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:54:01.049760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.318566+0000","last_change":"2023-12-19T18:21:08.318329+0000","last_active":"2023-12-19T18:21:08.318566+0000","last_peered":"2023-12-19T18:21:08.318566+0000","last_clean":"2023-12-19T18:21:08.318566+0000","last_became_active":"2023-12-19T18:21:08.317440+0000","last_became_peered":"2023-12-19T18:21:08.317440+0000","last_unstale":"2023-12-19T18:21:08.318566+0000","last_undegraded":"2023-12-19T18:21:08.318566+0000","last_fullsized":"2023-12-19T18:21:08.318566+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:10.958477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.294584+0000","last_change":"2023-12-19T18:21:07.796837+0000","last_active":"2023-12-19T18:21:08.294584+0000","last_peered":"2023-12-19T18:21:08.294584+0000","last_clean":"2023-12-19T18:21:08.294584+0000","last_became_active":"2023-12-19T18:21:07.796313+0000","last_became_peered":"2023-12-19T18:21:07.796313+0000","last_unstale":"2023-12-19T18:21:08.294584+0000","last_undegraded":"2023-12-19T18:21:08.294584+0000","last_fullsized":"2023-12-19T18:21:08.294584+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:34.703586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.307299+0000","last_change":"2023-12-19T18:21:08.303871+0000","last_active":"2023-12-19T18:21:08.307299+0000","last_peered":"2023-12-19T18:21:08.307299+0000","last_clean":"2023-12-19T18:21:08.307299+0000","last_became_active":"2023-12-19T18:21:08.302965+0000","last_became_peered":"2023-12-19T18:21:08.302965+0000","last_unstale":"2023-12-19T18:21:08.307299+0000","last_undegraded":"2023-12-19T18:21:08.307299+0000","last_fullsized":"2023-12-19T18:21:08.307299+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:38:33.981360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.312515+0000","last_change":"2023-12-19T18:21:07.805332+0000","last_active":"2023-12-19T18:21:08.312515+0000","last_peered":"2023-12-19T18:21:08.312515+0000","last_clean":"2023-12-19T18:21:08.312515+0000","last_became_active":"2023-12-19T18:21:07.804079+0000","last_became_peered":"2023-12-19T18:21:07.804079+0000","last_unstale":"2023-12-19T18:21:08.312515+0000","last_undegraded":"2023-12-19T18:21:08.312515+0000","last_fullsized":"2023-12-19T18:21:08.312515+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:11.803926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.302468+0000","last_change":"2023-12-19T18:21:08.302376+0000","last_active":"2023-12-19T18:21:08.302468+0000","last_peered":"2023-12-19T18:21:08.302468+0000","last_clean":"2023-12-19T18:21:08.302468+0000","last_became_active":"2023-12-19T18:21:08.302047+0000","last_became_peered":"2023-12-19T18:21:08.302047+0000","last_unstale":"2023-12-19T18:21:08.302468+0000","last_undegraded":"2023-12-19T18:21:08.302468+0000","last_fullsized":"2023-12-19T18:21:08.302468+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:52:04.632373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.310638+0000","last_change":"2023-12-19T18:21:07.805766+0000","last_active":"2023-12-19T18:21:08.310638+0000","last_peered":"2023-12-19T18:21:08.310638+0000","last_clean":"2023-12-19T18:21:08.310638+0000","last_became_active":"2023-12-19T18:21:07.805278+0000","last_became_peered":"2023-12-19T18:21:07.805278+0000","last_unstale":"2023-12-19T18:21:08.310638+0000","last_undegraded":"2023-12-19T18:21:08.310638+0000","last_fullsized":"2023-12-19T18:21:08.310638+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:43.474146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.297365+0000","last_change":"2023-12-19T18:21:08.296307+0000","last_active":"2023-12-19T18:21:08.297365+0000","last_peered":"2023-12-19T18:21:08.297365+0000","last_clean":"2023-12-19T18:21:08.297365+0000","last_became_active":"2023-12-19T18:21:08.296082+0000","last_became_peered":"2023-12-19T18:21:08.296082+0000","last_unstale":"2023-12-19T18:21:08.297365+0000","last_undegraded":"2023-12-19T18:21:08.297365+0000","last_fullsized":"2023-12-19T18:21:08.297365+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:49:40.809863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"116'2","reported_seq":12,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.326093+0000","last_change":"2023-12-19T18:21:07.797860+0000","last_active":"2023-12-19T18:21:08.326093+0000","last_peered":"2023-12-19T18:21:08.326093+0000","last_clean":"2023-12-19T18:21:08.326093+0000","last_became_active":"2023-12-19T18:21:07.797444+0000","last_became_peered":"2023-12-19T18:21:07.797444+0000","last_unstale":"2023-12-19T18:21:08.326093+0000","last_undegraded":"2023-12-19T18:21:08.326093+0000","last_fullsized":"2023-12-19T18:21:08.326093+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:27:11.803926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.303295+0000","last_change":"2023-12-19T18:21:08.302442+0000","last_active":"2023-12-19T18:21:08.303295+0000","last_peered":"2023-12-19T18:21:08.303295+0000","last_clean":"2023-12-19T18:21:08.303295+0000","last_became_active":"2023-12-19T18:21:08.302226+0000","last_became_peered":"2023-12-19T18:21:08.302226+0000","last_unstale":"2023-12-19T18:21:08.303295+0000","last_undegraded":"2023-12-19T18:21:08.303295+0000","last_fullsized":"2023-12-19T18:21:08.303295+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:30:58.163868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.310079+0000","last_change":"2023-12-19T18:21:07.806540+0000","last_active":"2023-12-19T18:21:08.310079+0000","last_peered":"2023-12-19T18:21:08.310079+0000","last_clean":"2023-12-19T18:21:08.310079+0000","last_became_active":"2023-12-19T18:21:07.805458+0000","last_became_peered":"2023-12-19T18:21:07.805458+0000","last_unstale":"2023-12-19T18:21:08.310079+0000","last_undegraded":"2023-12-19T18:21:08.310079+0000","last_fullsized":"2023-12-19T18:21:08.310079+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:42:28.405924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.303275+0000","last_change":"2023-12-19T18:21:08.299482+0000","last_active":"2023-12-19T18:21:08.303275+0000","last_peered":"2023-12-19T18:21:08.303275+0000","last_clean":"2023-12-19T18:21:08.303275+0000","last_became_active":"2023-12-19T18:21:08.299265+0000","last_became_peered":"2023-12-19T18:21:08.299265+0000","last_unstale":"2023-12-19T18:21:08.303275+0000","last_undegraded":"2023-12-19T18:21:08.303275+0000","last_fullsized":"2023-12-19T18:21:08.303275+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:03:47.164205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.296511+0000","last_change":"2023-12-19T18:21:07.805012+0000","last_active":"2023-12-19T18:21:08.296511+0000","last_peered":"2023-12-19T18:21:08.296511+0000","last_clean":"2023-12-19T18:21:08.296511+0000","last_became_active":"2023-12-19T18:21:07.802455+0000","last_became_peered":"2023-12-19T18:21:07.802455+0000","last_unstale":"2023-12-19T18:21:08.296511+0000","last_undegraded":"2023-12-19T18:21:08.296511+0000","last_fullsized":"2023-12-19T18:21:08.296511+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:53.467791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.317018+0000","last_change":"2023-12-19T18:21:08.316953+0000","last_active":"2023-12-19T18:21:08.317018+0000","last_peered":"2023-12-19T18:21:08.317018+0000","last_clean":"2023-12-19T18:21:08.317018+0000","last_became_active":"2023-12-19T18:21:08.316739+0000","last_became_peered":"2023-12-19T18:21:08.316739+0000","last_unstale":"2023-12-19T18:21:08.317018+0000","last_undegraded":"2023-12-19T18:21:08.317018+0000","last_fullsized":"2023-12-19T18:21:08.317018+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:07:00.931274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.300021+0000","last_change":"2023-12-19T18:21:07.802844+0000","last_active":"2023-12-19T18:21:08.300021+0000","last_peered":"2023-12-19T18:21:08.300021+0000","last_clean":"2023-12-19T18:21:08.300021+0000","last_became_active":"2023-12-19T18:21:07.801866+0000","last_became_peered":"2023-12-19T18:21:07.801866+0000","last_unstale":"2023-12-19T18:21:08.300021+0000","last_undegraded":"2023-12-19T18:21:08.300021+0000","last_fullsized":"2023-12-19T18:21:08.300021+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:38:30.353314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.303175+0000","last_change":"2023-12-19T18:21:08.303040+0000","last_active":"2023-12-19T18:21:08.303175+0000","last_peered":"2023-12-19T18:21:08.303175+0000","last_clean":"2023-12-19T18:21:08.303175+0000","last_became_active":"2023-12-19T18:21:08.302793+0000","last_became_peered":"2023-12-19T18:21:08.302793+0000","last_unstale":"2023-12-19T18:21:08.303175+0000","last_undegraded":"2023-12-19T18:21:08.303175+0000","last_fullsized":"2023-12-19T18:21:08.303175+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:31.559355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.294056+0000","last_change":"2023-12-19T18:21:07.793133+0000","last_active":"2023-12-19T18:21:08.294056+0000","last_peered":"2023-12-19T18:21:08.294056+0000","last_clean":"2023-12-19T18:21:08.294056+0000","last_became_active":"2023-12-19T18:21:07.792316+0000","last_became_peered":"2023-12-19T18:21:07.792316+0000","last_unstale":"2023-12-19T18:21:08.294056+0000","last_undegraded":"2023-12-19T18:21:08.294056+0000","last_fullsized":"2023-12-19T18:21:08.294056+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:15:53.467791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.323083+0000","last_change":"2023-12-19T18:21:08.322148+0000","last_active":"2023-12-19T18:21:08.323083+0000","last_peered":"2023-12-19T18:21:08.323083+0000","last_clean":"2023-12-19T18:21:08.323083+0000","last_became_active":"2023-12-19T18:21:08.319104+0000","last_became_peered":"2023-12-19T18:21:08.319104+0000","last_unstale":"2023-12-19T18:21:08.323083+0000","last_undegraded":"2023-12-19T18:21:08.323083+0000","last_fullsized":"2023-12-19T18:21:08.323083+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:14:03.228122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.301582+0000","last_change":"2023-12-19T18:21:07.795313+0000","last_active":"2023-12-19T18:21:08.301582+0000","last_peered":"2023-12-19T18:21:08.301582+0000","last_clean":"2023-12-19T18:21:08.301582+0000","last_became_active":"2023-12-19T18:21:07.794826+0000","last_became_peered":"2023-12-19T18:21:07.794826+0000","last_unstale":"2023-12-19T18:21:08.301582+0000","last_undegraded":"2023-12-19T18:21:08.301582+0000","last_fullsized":"2023-12-19T18:21:08.301582+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:27:51.591572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.303547+0000","last_change":"2023-12-19T18:21:08.303447+0000","last_active":"2023-12-19T18:21:08.303547+0000","last_peered":"2023-12-19T18:21:08.303547+0000","last_clean":"2023-12-19T18:21:08.303547+0000","last_became_active":"2023-12-19T18:21:08.303203+0000","last_became_peered":"2023-12-19T18:21:08.303203+0000","last_unstale":"2023-12-19T18:21:08.303547+0000","last_undegraded":"2023-12-19T18:21:08.303547+0000","last_fullsized":"2023-12-19T18:21:08.303547+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:56:48.490025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.295122+0000","last_change":"2023-12-19T18:21:07.806245+0000","last_active":"2023-12-19T18:21:08.295122+0000","last_peered":"2023-12-19T18:21:08.295122+0000","last_clean":"2023-12-19T18:21:08.295122+0000","last_became_active":"2023-12-19T18:21:07.805665+0000","last_became_peered":"2023-12-19T18:21:07.805665+0000","last_unstale":"2023-12-19T18:21:08.295122+0000","last_undegraded":"2023-12-19T18:21:08.295122+0000","last_fullsized":"2023-12-19T18:21:08.295122+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:13:26.501953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.307140+0000","last_change":"2023-12-19T18:21:08.306860+0000","last_active":"2023-12-19T18:21:08.307140+0000","last_peered":"2023-12-19T18:21:08.307140+0000","last_clean":"2023-12-19T18:21:08.307140+0000","last_became_active":"2023-12-19T18:21:08.306625+0000","last_became_peered":"2023-12-19T18:21:08.306625+0000","last_unstale":"2023-12-19T18:21:08.307140+0000","last_undegraded":"2023-12-19T18:21:08.307140+0000","last_fullsized":"2023-12-19T18:21:08.307140+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:46:10.913345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":10,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.296626+0000","last_change":"2023-12-19T18:21:07.802817+0000","last_active":"2023-12-19T18:21:08.296626+0000","last_peered":"2023-12-19T18:21:08.296626+0000","last_clean":"2023-12-19T18:21:08.296626+0000","last_became_active":"2023-12-19T18:21:07.801981+0000","last_became_peered":"2023-12-19T18:21:07.801981+0000","last_unstale":"2023-12-19T18:21:08.296626+0000","last_undegraded":"2023-12-19T18:21:08.296626+0000","last_fullsized":"2023-12-19T18:21:08.296626+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:58:53.471739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.320795+0000","last_change":"2023-12-19T18:21:08.314162+0000","last_active":"2023-12-19T18:21:08.320795+0000","last_peered":"2023-12-19T18:21:08.320795+0000","last_clean":"2023-12-19T18:21:08.320795+0000","last_became_active":"2023-12-19T18:21:08.313589+0000","last_became_peered":"2023-12-19T18:21:08.313589+0000","last_unstale":"2023-12-19T18:21:08.320795+0000","last_undegraded":"2023-12-19T18:21:08.320795+0000","last_fullsized":"2023-12-19T18:21:08.320795+0000","mapping_epoch":115,"log_start":"0'0","ondisk_log_start":"0'0","created":115,"last_epoch_clean":116,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:07.758380+0000","last_clean_scrub_stamp":"2023-12-19T18:21:07.758380+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:38:31.365048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"116'1","reported_seq":11,"reported_epoch":116,"state":"active+clean","last_fresh":"2023-12-19T18:21:08.319299+0000","last_change":"2023-12-19T18:21:07.804936+0000","last_active":"2023-12-19T18:21:08.319299+0000","last_peered":"2023-12-19T18:21:08.319299+0000","last_clean":"2023-12-19T18:21:08.319299+0000","last_became_active":"2023-12-19T18:21:07.804679+0000","last_became_peered":"2023-12-19T18:21:07.804679+0000","last_unstale":"2023-12-19T18:21:08.319299+0000","last_undegraded":"2023-12-19T18:21:08.319299+0000","last_fullsized":"2023-12-19T18:21:08.319299+0000","mapping_epoch":114,"log_start":"0'0","ondisk_log_start":"0'0","created":114,"last_epoch_clean":115,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:21:06.746646+0000","last_clean_scrub_stamp":"2023-12-19T18:21:06.746646+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:00:29.556804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":250,"reported_epoch":115,"state":"active+clean","last_fresh":"2023-12-19T18:21:07.784836+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:21:07.784836+0000","last_peered":"2023-12-19T18:21:07.784836+0000","last_clean":"2023-12-19T18:21:07.784836+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:21:07.784836+0000","last_undegraded":"2023-12-19T18:21:07.784836+0000","last_fullsized":"2023-12-19T18:21:07.784836+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359091,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.0840000000000001,"5min":1.22,"15min":1.1439999999999999},"min":{"1min":0.69999999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.6830000000000001,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":1.24},{"interface":"front","average":{"1min":1.264,"5min":1.26,"15min":1.1799999999999999},"min":{"1min":0.76100000000000001,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.665,"5min":1.79,"15min":1.79},"last":1.05}]},{"osd":1,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1679999999999999,"5min":1.361,"15min":1.232},"min":{"1min":0.63600000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.5229999999999999,"5min":10.926,"15min":10.926},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.2250000000000001,"5min":1.3240000000000001,"15min":1.2350000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.67900000000000005},"max":{"1min":1.589,"5min":10.904999999999999,"15min":10.904999999999999},"last":1.621}]},{"osd":2,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.157,"15min":1.107},"min":{"1min":0.72699999999999998,"5min":0.66800000000000004,"15min":0.66800000000000004},"max":{"1min":1.5409999999999999,"5min":3.1960000000000002,"15min":3.1960000000000002},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.2569999999999999,"15min":1.1870000000000001},"min":{"1min":0.81999999999999995,"5min":0.72199999999999998,"15min":0.69699999999999995},"max":{"1min":1.4430000000000001,"5min":6.008,"15min":6.008},"last":1.4490000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3109999999999999,"5min":1.266,"15min":1.151},"min":{"1min":0.995,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.696,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.306,"5min":1.409,"15min":1.28},"min":{"1min":0.77300000000000002,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.7829999999999999,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":1.5820000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2230000000000001,"5min":1.2350000000000001,"15min":1.1060000000000001},"min":{"1min":0.92200000000000004,"5min":0.65400000000000003,"15min":0.47899999999999998},"max":{"1min":1.7529999999999999,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":1.8009999999999999},{"interface":"front","average":{"1min":1.3260000000000001,"5min":1.3089999999999999,"15min":1.24},"min":{"1min":0.98099999999999998,"5min":0.72999999999999998,"15min":0.57599999999999996},"max":{"1min":1.6040000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.54}]},{"osd":5,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2549999999999999,"5min":1.4279999999999999,"15min":1.333},"min":{"1min":0.81399999999999995,"5min":0.749,"15min":0.55000000000000004},"max":{"1min":1.873,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":1.748},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.4239999999999999,"15min":1.3300000000000001},"min":{"1min":1.0449999999999999,"5min":0.71599999999999997,"15min":0.53500000000000003},"max":{"1min":1.9830000000000001,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":2.1179999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.5720000000000001,"5min":1.5529999999999999,"15min":1.4570000000000001},"min":{"1min":1.119,"5min":0.78600000000000003,"15min":0.59899999999999998},"max":{"1min":2.1320000000000001,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":2.423},{"interface":"front","average":{"1min":1.484,"5min":1.4850000000000001,"15min":1.371},"min":{"1min":1.125,"5min":0.92300000000000004,"15min":0.63200000000000001},"max":{"1min":1.9710000000000001,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":2.0179999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199358489,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.0840000000000001,"15min":0.99199999999999999},"min":{"1min":0.74099999999999999,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.6240000000000001,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.2310000000000001,"5min":1.1759999999999999,"15min":1.0640000000000001},"min":{"1min":0.79900000000000004,"5min":0.64300000000000002,"15min":0.55600000000000005},"max":{"1min":1.748,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":0.94899999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.1359999999999999,"15min":1.1240000000000001},"min":{"1min":0.84899999999999998,"5min":0.71399999999999997,"15min":0.68200000000000005},"max":{"1min":1.474,"5min":4.5540000000000003,"15min":4.5540000000000003},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.286,"5min":1.2030000000000001,"15min":1.165},"min":{"1min":0.93000000000000005,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.746,"5min":1.885,"15min":1.885},"last":1.2529999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.25,"15min":1.143},"min":{"1min":0.83499999999999996,"5min":0.626,"15min":0.626},"max":{"1min":1.7090000000000001,"5min":1.821,"15min":1.821},"last":1.3759999999999999},{"interface":"front","average":{"1min":1.252,"5min":1.2150000000000001,"15min":1.2370000000000001},"min":{"1min":0.88200000000000001,"5min":0.81499999999999995,"15min":0.79900000000000004},"max":{"1min":1.944,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":1.0620000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.2769999999999999,"15min":1.1930000000000001},"min":{"1min":0.85699999999999998,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.7110000000000001,"5min":3.8820000000000001,"15min":3.8820000000000001},"last":1.173},{"interface":"front","average":{"1min":1.2430000000000001,"5min":1.2070000000000001,"15min":1.1659999999999999},"min":{"1min":0.878,"5min":0.78300000000000003,"15min":0.77500000000000002},"max":{"1min":1.5640000000000001,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":1.458}]},{"osd":4,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.2789999999999999,"15min":1.2410000000000001},"min":{"1min":0.85199999999999998,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.8520000000000001,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.534},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.4359999999999999,"15min":1.3400000000000001},"min":{"1min":1.208,"5min":0.70799999999999996,"15min":0.57999999999999996},"max":{"1min":2.0430000000000001,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.4990000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.284,"15min":1.1819999999999999},"min":{"1min":0.90500000000000003,"5min":0.66600000000000004,"15min":0.66400000000000003},"max":{"1min":2.2400000000000002,"5min":4.6680000000000001,"15min":4.6680000000000001},"last":1.7330000000000001},{"interface":"front","average":{"1min":1.474,"5min":1.4059999999999999,"15min":1.337},"min":{"1min":1.0169999999999999,"5min":0.626,"15min":0.626},"max":{"1min":2.0459999999999998,"5min":3.3580000000000001,"15min":3.3580000000000001},"last":1.6990000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4930000000000001,"5min":1.4370000000000001,"15min":1.341},"min":{"1min":1.034,"5min":0.77200000000000002,"15min":0.61899999999999999},"max":{"1min":2.133,"5min":3.9670000000000001,"15min":3.9670000000000001},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.5029999999999999,"5min":1.4470000000000001,"15min":1.351},"min":{"1min":1.075,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.3519999999999999,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":1.619}]}]},{"osd":5,"up_from":13,"seq":55834574927,"num_pgs":14,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1195,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194116,"statfs":{"total":200000000,"available":198775568,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.242,"15min":1.1679999999999999},"min":{"1min":0.79600000000000004,"5min":0.65200000000000002,"15min":0.65200000000000002},"max":{"1min":1.5600000000000001,"5min":1.837,"15min":1.863},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.274,"15min":1.179},"min":{"1min":1.004,"5min":0.75,"15min":0.75},"max":{"1min":1.724,"5min":1.879,"15min":1.879},"last":1.3200000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1819999999999999,"15min":1.05},"min":{"1min":0.86199999999999999,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.639,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.84599999999999997},{"interface":"front","average":{"1min":1.135,"5min":1.149,"15min":1.0549999999999999},"min":{"1min":0.754,"5min":0.68700000000000006,"15min":0.61299999999999999},"max":{"1min":1.617,"5min":1.843,"15min":1.843},"last":0.92100000000000004}]},{"osd":2,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.165,"5min":1.1699999999999999,"15min":1.081},"min":{"1min":0.93500000000000005,"5min":0.74299999999999999,"15min":0.59499999999999997},"max":{"1min":1.431,"5min":1.794,"15min":1.794},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.228,"15min":1.1379999999999999},"min":{"1min":0.84899999999999998,"5min":0.76800000000000002,"15min":0.65200000000000002},"max":{"1min":1.768,"5min":2.036,"15min":2.036},"last":1.004}]},{"osd":3,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.1850000000000001,"15min":1.085},"min":{"1min":0.71999999999999997,"5min":0.69399999999999995,"15min":0.68899999999999995},"max":{"1min":1.45,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.085},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.339,"15min":1.222},"min":{"1min":0.875,"5min":0.75700000000000001,"15min":0.70399999999999996},"max":{"1min":1.8380000000000001,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.25}]},{"osd":4,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.302,"15min":1.163},"min":{"1min":0.83299999999999996,"5min":0.60899999999999999,"15min":0.46700000000000003},"max":{"1min":1.556,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.577},{"interface":"front","average":{"1min":1.23,"5min":1.1819999999999999,"15min":1.1200000000000001},"min":{"1min":0.872,"5min":0.73999999999999999,"15min":0.57099999999999995},"max":{"1min":1.708,"5min":1.804,"15min":1.901},"last":1.5229999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.4430000000000001,"15min":1.325},"min":{"1min":0.92500000000000004,"5min":0.66600000000000004,"15min":0.54400000000000004},"max":{"1min":1.544,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.387},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.3049999999999999,"15min":1.236},"min":{"1min":0.81999999999999995,"5min":0.81200000000000006,"15min":0.49299999999999999},"max":{"1min":1.643,"5min":1.7949999999999999,"15min":1.976},"last":1.22}]},{"osd":7,"last update":"Tue Dec 19 18:20:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2849999999999999,"15min":1.2190000000000001},"min":{"1min":0.88500000000000001,"5min":0.68300000000000005,"15min":0.50900000000000001},"max":{"1min":1.752,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.4359999999999999},{"interface":"front","average":{"1min":1.4810000000000001,"5min":1.4590000000000001,"15min":1.407},"min":{"1min":0.95099999999999996,"5min":0.84999999999999998,"15min":0.623},"max":{"1min":1.929,"5min":2.117,"15min":2.117},"last":1.617}]}]},{"osd":4,"up_from":13,"seq":55834574927,"num_pgs":10,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":619,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194693,"statfs":{"total":200000000,"available":199365936,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.006,"5min":0.997,"15min":0.997},"min":{"1min":0.69299999999999995,"5min":0.62,"15min":0.62},"max":{"1min":1.532,"5min":1.579,"15min":1.579},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.0469999999999999,"15min":1.0309999999999999},"min":{"1min":0.76900000000000002,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.575,"5min":1.7909999999999999,"15min":1.7909999999999999},"last":0.80000000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1639999999999999,"5min":1.1679999999999999,"15min":1.1519999999999999},"min":{"1min":0.81699999999999995,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.6679999999999999,"5min":1.7130000000000001,"15min":1.8160000000000001},"last":1.0780000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.248,"15min":1.175},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.4470000000000001,"5min":1.696,"15min":1.696},"last":1.149}]},{"osd":2,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1279999999999999,"15min":1.1240000000000001},"min":{"1min":0.89700000000000002,"5min":0.747,"15min":0.69799999999999995},"max":{"1min":1.4079999999999999,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.25,"5min":1.1479999999999999,"15min":1.1000000000000001},"min":{"1min":0.78000000000000003,"5min":0.72199999999999998,"15min":0.70499999999999996},"max":{"1min":1.5920000000000001,"5min":1.5920000000000001,"15min":1.5920000000000001},"last":0.91500000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2689999999999999,"5min":1.2869999999999999,"15min":1.254},"min":{"1min":0.95299999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":1.5249999999999999,"5min":1.6639999999999999,"15min":1.6639999999999999},"last":1.254},{"interface":"front","average":{"1min":1.359,"5min":1.327,"15min":1.2989999999999999},"min":{"1min":1.1699999999999999,"5min":0.82399999999999995,"15min":0.74399999999999999},"max":{"1min":1.5740000000000001,"5min":1.718,"15min":1.7529999999999999},"last":1.2310000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.347,"5min":1.2929999999999999,"15min":1.198},"min":{"1min":0.89200000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.6659999999999999,"5min":1.8740000000000001,"15min":1.8740000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.4430000000000001,"5min":1.4179999999999999,"15min":1.3460000000000001},"min":{"1min":0.86699999999999999,"5min":0.625,"15min":0.504},"max":{"1min":1.6950000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.353}]},{"osd":6,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3149999999999999,"5min":1.319,"15min":1.3029999999999999},"min":{"1min":0.76700000000000002,"5min":0.432,"15min":0.432},"max":{"1min":1.627,"5min":1.853,"15min":1.853},"last":1.306},{"interface":"front","average":{"1min":1.5129999999999999,"5min":1.454,"15min":1.389},"min":{"1min":1.2170000000000001,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.7529999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.452}]},{"osd":7,"last update":"Tue Dec 19 18:20:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.387,"15min":1.3380000000000001},"min":{"1min":1.1279999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.6739999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.55},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.427,"15min":1.405},"min":{"1min":0.98699999999999999,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.7070000000000001,"5min":1.905,"15min":1.9330000000000001},"last":1.5249999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574928,"num_pgs":17,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":626,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199358487,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":0.91800000000000004,"15min":0.96799999999999997},"min":{"1min":0.38100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":3.3140000000000001,"5min":3.3140000000000001,"15min":3.4350000000000001},"last":0.96299999999999997},{"interface":"front","average":{"1min":1.018,"5min":0.97499999999999998,"15min":0.93500000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7170000000000001,"5min":1.8140000000000001,"15min":1.8140000000000001},"last":0.82599999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0740000000000001,"5min":0.98799999999999999,"15min":1.054},"min":{"1min":0.439,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":4.5129999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":1.218,"5min":1.0800000000000001,"15min":1.0960000000000001},"min":{"1min":0.57099999999999995,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":1.123}]},{"osd":2,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3799999999999999,"5min":1.3240000000000001,"15min":1.25},"min":{"1min":0.48199999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.5369999999999999,"5min":13.833,"15min":13.833},"last":1.375},{"interface":"front","average":{"1min":1.587,"5min":1.448,"15min":1.284},"min":{"1min":0.66300000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":3.9710000000000001,"5min":16.748999999999999,"15min":16.748999999999999},"last":1.4239999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.4019999999999999,"5min":1.302,"15min":1.214},"min":{"1min":0.88100000000000001,"5min":0.77400000000000002,"15min":0.65600000000000003},"max":{"1min":3.0910000000000002,"5min":3.0910000000000002,"15min":3.0910000000000002},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.359,"15min":1.397},"min":{"1min":0.84599999999999997,"5min":0.80800000000000005,"15min":0.73199999999999998},"max":{"1min":2.0640000000000001,"5min":2.5710000000000002,"15min":4.0209999999999999},"last":1.708}]},{"osd":5,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.613,"5min":1.4119999999999999,"15min":1.4410000000000001},"min":{"1min":1.0509999999999999,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":3.472,"5min":3.472,"15min":3.472},"last":1.581},{"interface":"front","average":{"1min":1.415,"5min":1.327,"15min":1.264},"min":{"1min":0.84699999999999998,"5min":0.76600000000000001,"15min":0.69999999999999996},"max":{"1min":2.9660000000000002,"5min":2.9660000000000002,"15min":2.9660000000000002},"last":1.1679999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.4119999999999999,"15min":1.472},"min":{"1min":0.877,"5min":0.81799999999999995,"15min":0.81599999999999995},"max":{"1min":2.1739999999999999,"5min":2.4790000000000001,"15min":3.9399999999999999},"last":1.526},{"interface":"front","average":{"1min":1.627,"5min":1.4810000000000001,"15min":1.504},"min":{"1min":0.83199999999999996,"5min":0.80500000000000005,"15min":0.80500000000000005},"max":{"1min":2.2120000000000002,"5min":2.2400000000000002,"15min":3.2229999999999999},"last":1.327}]},{"osd":7,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.4350000000000001,"15min":1.3740000000000001},"min":{"1min":0.90600000000000003,"5min":0.75900000000000001,"15min":0.73199999999999998},"max":{"1min":3.1920000000000002,"5min":3.1920000000000002,"15min":3.1920000000000002},"last":1.026},{"interface":"front","average":{"1min":1.5329999999999999,"5min":1.52,"15min":1.544},"min":{"1min":1.1160000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.456,"5min":2.657,"15min":3.9849999999999999},"last":1.635}]}]},{"osd":2,"up_from":13,"seq":55834574927,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":625,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194686,"statfs":{"total":200000000,"available":199359161,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":1.0600000000000001,"15min":1.016},"min":{"1min":0.38200000000000001,"5min":0.35299999999999998,"15min":0.34399999999999997},"max":{"1min":2.879,"5min":9.2550000000000008,"15min":9.2550000000000008},"last":1.286},{"interface":"front","average":{"1min":1.165,"5min":1.026,"15min":1.044},"min":{"1min":0.57899999999999996,"5min":0.41999999999999998,"15min":0.38900000000000001},"max":{"1min":2.387,"5min":2.387,"15min":4.7850000000000001},"last":0.84099999999999997}]},{"osd":1,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":0.94399999999999995,"15min":1.1859999999999999},"min":{"1min":0.50600000000000001,"5min":0.49199999999999999,"15min":0.46000000000000002},"max":{"1min":2.4390000000000001,"5min":2.4390000000000001,"15min":8.2720000000000002},"last":1.542},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.0860000000000001,"15min":1.1699999999999999},"min":{"1min":0.61599999999999999,"5min":0.504,"15min":0.47199999999999998},"max":{"1min":2.7200000000000002,"5min":2.7200000000000002,"15min":5.0620000000000003},"last":0.88900000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.149,"15min":1.383},"min":{"1min":0.56000000000000005,"5min":0.54200000000000004,"15min":0.51900000000000002},"max":{"1min":2.238,"5min":2.238,"15min":8.3109999999999999},"last":1.103},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.149,"15min":1.349},"min":{"1min":0.55500000000000005,"5min":0.46300000000000002,"15min":0.41799999999999998},"max":{"1min":2.323,"5min":2.323,"15min":8.0690000000000008},"last":1.1910000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.405,"15min":1.472},"min":{"1min":0.86399999999999999,"5min":0.82599999999999996,"15min":0.79300000000000004},"max":{"1min":2.6139999999999999,"5min":2.6139999999999999,"15min":5.3929999999999998},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.601,"5min":1.421,"15min":1.5569999999999999},"min":{"1min":0.89500000000000002,"5min":0.78300000000000003,"15min":0.78300000000000003},"max":{"1min":3.1070000000000002,"5min":3.1070000000000002,"15min":7.3570000000000002},"last":0.96799999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.3680000000000001,"15min":1.5620000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.5579999999999998,"5min":2.5579999999999998,"15min":8.1839999999999993},"last":1.474},{"interface":"front","average":{"1min":1.5209999999999999,"5min":1.409,"15min":1.419},"min":{"1min":0.98299999999999998,"5min":0.81000000000000005,"15min":0.72199999999999998},"max":{"1min":2.4940000000000002,"5min":2.4940000000000002,"15min":5.165},"last":1.3919999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.5569999999999999,"5min":1.4119999999999999,"15min":1.454},"min":{"1min":0.92200000000000004,"5min":0.752,"15min":0.72999999999999998},"max":{"1min":3.0179999999999998,"5min":3.0179999999999998,"15min":5.1059999999999999},"last":1.2649999999999999},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.4470000000000001,"15min":1.5720000000000001},"min":{"1min":0.93300000000000005,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.9580000000000002,"5min":2.9580000000000002,"15min":8.1370000000000005},"last":1.6060000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:20:50 2023","interfaces":[{"interface":"back","average":{"1min":1.708,"5min":1.5089999999999999,"15min":1.5589999999999999},"min":{"1min":0.98999999999999999,"5min":0.79200000000000004,"15min":0.78200000000000003},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":5.1449999999999996},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.6639999999999999,"5min":1.5009999999999999,"15min":1.663},"min":{"1min":0.88100000000000001,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":2.6859999999999999,"5min":2.6859999999999999,"15min":8.2300000000000004},"last":1.7569999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574928,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1202,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194110,"statfs":{"total":200000000,"available":198768703,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":0.97199999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.48399999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.52,"5min":1.52,"15min":1.52},"last":1.3440000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":0.94299999999999995,"15min":0.96299999999999997},"min":{"1min":0.68000000000000005,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.79100000000000004}]},{"osd":2,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.048,"15min":1.0589999999999999},"min":{"1min":0.45500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":2.5739999999999998},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.0089999999999999,"15min":0.92800000000000005},"min":{"1min":0.76700000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.716,"5min":1.845,"15min":1.845},"last":1.268}]},{"osd":3,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.5309999999999999,"5min":1.2869999999999999,"15min":1.159},"min":{"1min":0.68799999999999994,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.137,"5min":2.137,"15min":2.137},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.22,"15min":1.1970000000000001},"min":{"1min":0.746,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":2.3959999999999999},"last":1.3740000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.526,"5min":1.329,"15min":1.294},"min":{"1min":0.72199999999999998,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.5920000000000001,"5min":1.377,"15min":1.3},"min":{"1min":0.93300000000000005,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.623}]},{"osd":5,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.6679999999999999,"5min":1.407,"15min":1.3009999999999999},"min":{"1min":1.1180000000000001,"5min":0.88,"15min":0.88},"max":{"1min":2.5459999999999998,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.3149999999999999},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.484,"15min":1.415},"min":{"1min":1.024,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.6179999999999999,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.766}]},{"osd":6,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.736,"5min":1.4490000000000001,"15min":1.389},"min":{"1min":1.196,"5min":0.76900000000000002,"15min":0.76900000000000002},"max":{"1min":2.7000000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":1.034},{"interface":"front","average":{"1min":1.792,"5min":1.5309999999999999,"15min":1.4550000000000001},"min":{"1min":0.91400000000000003,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.895,"5min":2.895,"15min":2.895},"last":1.6759999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:20:54 2023","interfaces":[{"interface":"back","average":{"1min":1.7230000000000001,"5min":1.569,"15min":1.3899999999999999},"min":{"1min":0.88800000000000001,"5min":0.82399999999999995,"15min":0.69999999999999996},"max":{"1min":2.5670000000000002,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":1.8129999999999999},{"interface":"front","average":{"1min":1.7729999999999999,"5min":1.651,"15min":1.506},"min":{"1min":1.157,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.456,"5min":9.4109999999999996,"15min":9.4109999999999996},"last":1.472}]}]},{"osd":0,"up_from":13,"seq":55834574927,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":627,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194685,"statfs":{"total":200000000,"available":199357575,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.099,"5min":0.98699999999999999,"15min":0.89800000000000002},"min":{"1min":0.51100000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.3199999999999998,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.86399999999999999},{"interface":"front","average":{"1min":1.2130000000000001,"5min":1.127,"15min":1.0369999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.956,"5min":2.1040000000000001,"15min":2.4180000000000001},"last":1.3580000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.03,"15min":0.95799999999999996},"min":{"1min":0.70599999999999996,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":1.536,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.92100000000000004},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.028,"15min":0.94299999999999995},"min":{"1min":0.55700000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":2.0950000000000002,"5min":2.7810000000000001,"15min":2.7810000000000001},"last":0.98399999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.083,"5min":1.046,"15min":1.0109999999999999},"min":{"1min":0.52400000000000002,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":2.5190000000000001,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.3859999999999999,"5min":1.3049999999999999,"15min":1.2310000000000001},"min":{"1min":0.63100000000000001,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.8420000000000001,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.681}]},{"osd":4,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.464,"5min":1.331,"15min":1.2050000000000001},"min":{"1min":0.93100000000000005,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.5529999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.444},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.306,"15min":1.2889999999999999},"min":{"1min":0.89800000000000002,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":2.2450000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.2609999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.5169999999999999,"5min":1.5129999999999999,"15min":1.4039999999999999},"min":{"1min":0.751,"5min":0.71099999999999997,"15min":0.64500000000000002},"max":{"1min":2.3940000000000001,"5min":16.298999999999999,"15min":16.298999999999999},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.474,"5min":1.3700000000000001,"15min":1.2809999999999999},"min":{"1min":0.78400000000000003,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.4830000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.478,"15min":1.4019999999999999},"min":{"1min":1.0369999999999999,"5min":0.75900000000000001,"15min":0.75900000000000001},"max":{"1min":2.2559999999999998,"5min":2.2559999999999998,"15min":2.5910000000000002},"last":1.575},{"interface":"front","average":{"1min":1.5349999999999999,"5min":1.4710000000000001,"15min":1.3640000000000001},"min":{"1min":0.87,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":2.4359999999999999,"5min":2.46,"15min":2.46},"last":1.3160000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:20:57 2023","interfaces":[{"interface":"back","average":{"1min":1.6399999999999999,"5min":1.5269999999999999,"15min":1.3979999999999999},"min":{"1min":0.90900000000000003,"5min":0.89200000000000002,"15min":0.70899999999999996},"max":{"1min":2.4969999999999999,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.6319999999999999,"5min":1.4390000000000001,"15min":1.431},"min":{"1min":1.0600000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"max":{"1min":2.4580000000000002,"5min":2.4580000000000002,"15min":3.1429999999999998},"last":1.6499999999999999}]}]}],"pool_statfs":[]}} 2023-12-19T18:21:13.881 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:21:13.881 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:21:14.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.161+0000 7f1fef311640 1 Processor -- start 2023-12-19T18:21:14.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.161+0000 7f1fef311640 1 -- start start 2023-12-19T18:21:14.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.163+0000 7f1fef311640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe806c030 0x7f1fe806c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.163+0000 7f1fef311640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1fe8070650 0x7f1fe815a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.163+0000 7f1fef311640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fe80ccba0 con 0x7f1fe8070a90 2023-12-19T18:21:14.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.163+0000 7f1fef311640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fe806f960 con 0x7f1fe8070650 2023-12-19T18:21:14.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.163+0000 7f1fef311640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe815b070 con 0x7f1fe806c030 2023-12-19T18:21:14.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.163+0000 7f1fedb0e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1fe8070650 0x7f1fe815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.163+0000 7f1feeb10640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1fe8070a90 0x7f1fe80cd030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56528/0 (socket says 172.21.15.27:56528) 2023-12-19T18:21:14.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.163+0000 7f1feeb10640 1 -- 172.21.15.27:0/1753971502 learned_addr learned my addr 172.21.15.27:0/1753971502 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:14.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.164+0000 7f1fee30f640 1 --2- 172.21.15.27:0/1753971502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe806c030 0x7f1fe806c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.164+0000 7f1fedb0e640 1 -- 172.21.15.27:0/1753971502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe806c030 msgr2=0x7f1fe806c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:14.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.164+0000 7f1fedb0e640 1 --2- 172.21.15.27:0/1753971502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe806c030 0x7f1fe806c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.164+0000 7f1fedb0e640 1 -- 172.21.15.27:0/1753971502 >> v1:172.21.15.27:6789/0 conn(0x7f1fe8070a90 legacy=0x7f1fe80cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:14.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.164+0000 7f1fedb0e640 1 -- 172.21.15.27:0/1753971502 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fe80701c0 con 0x7f1fe8070650 2023-12-19T18:21:14.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.164+0000 7f1fed30d640 1 -- 172.21.15.27:0/1753971502 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4255751605 0 0) 0x7f1fe80ccba0 con 0x7f1fe8070a90 2023-12-19T18:21:14.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.165+0000 7f1fedb0e640 1 --2- 172.21.15.27:0/1753971502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fe8070650 0x7f1fe815a140 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80097a0 tx=0x7f1fd802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50d5716696f1b08 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:14.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.166+0000 7f1fed30d640 1 -- 172.21.15.27:0/1753971502 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd802e070 con 0x7f1fe8070650 2023-12-19T18:21:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.166+0000 7f1fed30d640 1 -- 172.21.15.27:0/1753971502 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd80315a0 con 0x7f1fe8070650 2023-12-19T18:21:14.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.166+0000 7f1fed30d640 1 -- 172.21.15.27:0/1753971502 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8031930 con 0x7f1fe8070650 2023-12-19T18:21:14.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.166+0000 7f1fef311640 1 -- 172.21.15.27:0/1753971502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fe8070650 msgr2=0x7f1fe815a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:14.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.166+0000 7f1fef311640 1 --2- 172.21.15.27:0/1753971502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fe8070650 0x7f1fe815a140 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80097a0 tx=0x7f1fd802b840 comp rx=0 tx=0).stop 2023-12-19T18:21:14.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.166+0000 7f1fef311640 1 -- 172.21.15.27:0/1753971502 shutdown_connections 2023-12-19T18:21:14.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.166+0000 7f1fef311640 1 --2- 172.21.15.27:0/1753971502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fe8070650 0x7f1fe815a140 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.166+0000 7f1fef311640 1 --2- 172.21.15.27:0/1753971502 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe806c030 0x7f1fe806c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.166+0000 7f1fef311640 1 -- 172.21.15.27:0/1753971502 >> 172.21.15.27:0/1753971502 conn(0x7f1fe80cb3f0 msgr2=0x7f1fe80cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:14.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.166+0000 7f1fef311640 1 -- 172.21.15.27:0/1753971502 shutdown_connections 2023-12-19T18:21:14.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.166+0000 7f1fef311640 1 -- 172.21.15.27:0/1753971502 wait complete. 2023-12-19T18:21:14.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.168+0000 7f1fef311640 1 Processor -- start 2023-12-19T18:21:14.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.168+0000 7f1fef311640 1 -- start start 2023-12-19T18:21:14.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.169+0000 7f1fef311640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fe806c030 0x7f1fe81efe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.169+0000 7f1fef311640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fe8070650 0x7f1fe80c50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.169+0000 7f1fef311640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe8070a90 0x7f1fe815b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.169+0000 7f1fef311640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80c4c20 con 0x7f1fe8070650 2023-12-19T18:21:14.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.169+0000 7f1fef311640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80c2540 con 0x7f1fe806c030 2023-12-19T18:21:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.169+0000 7f1fef311640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fe80c27b0 con 0x7f1fe8070a90 2023-12-19T18:21:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.170+0000 7f1fedb0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fe8070650 0x7f1fe80c50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.170+0000 7f1fedb0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fe8070650 0x7f1fe80c50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35218/0 (socket says 172.21.15.27:35218) 2023-12-19T18:21:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.170+0000 7f1fedb0e640 1 -- 172.21.15.27:0/764470763 learned_addr learned my addr 172.21.15.27:0/764470763 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.170+0000 7f1fee30f640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fe806c030 0x7f1fe81efe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.170+0000 7f1feeb10640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe8070a90 0x7f1fe815b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.170+0000 7f1fedb0e640 1 -- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe8070a90 msgr2=0x7f1fe815b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.170+0000 7f1fedb0e640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe8070a90 0x7f1fe815b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.170+0000 7f1fedb0e640 1 -- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fe806c030 msgr2=0x7f1fe81efe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.170+0000 7f1fedb0e640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fe806c030 0x7f1fe81efe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.170+0000 7f1fedb0e640 1 -- 172.21.15.27:0/764470763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fd8009400 con 0x7f1fe8070650 2023-12-19T18:21:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.170+0000 7f1feeb10640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe8070a90 0x7f1fe815b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:14.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.171+0000 7f1fedb0e640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fe8070650 0x7f1fe80c50b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80098c0 tx=0x7f1fd80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:14.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.171+0000 7f1fdeffd640 1 -- 172.21.15.27:0/764470763 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8032840 con 0x7f1fe8070650 2023-12-19T18:21:14.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.171+0000 7f1fdeffd640 1 -- 172.21.15.27:0/764470763 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fd80329a0 con 0x7f1fe8070650 2023-12-19T18:21:14.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.171+0000 7f1fdeffd640 1 -- 172.21.15.27:0/764470763 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8032d50 con 0x7f1fe8070650 2023-12-19T18:21:14.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.171+0000 7f1fef311640 1 -- 172.21.15.27:0/764470763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fe81f03b0 con 0x7f1fe8070650 2023-12-19T18:21:14.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.171+0000 7f1fef311640 1 -- 172.21.15.27:0/764470763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fe80c2a90 con 0x7f1fe8070650 2023-12-19T18:21:14.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.172+0000 7f1fef311640 1 -- 172.21.15.27:0/764470763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe804e840 con 0x7f1fe8070650 2023-12-19T18:21:14.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.173+0000 7f1fdeffd640 1 -- 172.21.15.27:0/764470763 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1fd804e430 con 0x7f1fe8070650 2023-12-19T18:21:14.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.174+0000 7f1fdeffd640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fcc06a430 0x7f1fcc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.174+0000 7f1fdeffd640 1 -- 172.21.15.27:0/764470763 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fd80aa930 con 0x7f1fe8070650 2023-12-19T18:21:14.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.175+0000 7f1fee30f640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fcc06a430 0x7f1fcc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.176+0000 7f1fee30f640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fcc06a430 0x7f1fcc06c8f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1fe815d5d0 tx=0x7f1fe400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:14.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.178+0000 7f1fdeffd640 1 -- 172.21.15.27:0/764470763 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1fd8075f10 con 0x7f1fe8070650 2023-12-19T18:21:14.443 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.443+0000 7f1fef311640 1 -- 172.21.15.27:0/764470763 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f1fe80b48f0 con 0x7f1fe8070650 2023-12-19T18:21:14.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.451+0000 7f1fdeffd640 1 -- 172.21.15.27:0/764470763 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f1fd8079530 con 0x7f1fe8070650 2023-12-19T18:21:14.451 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2023-12-19T18:21:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.454+0000 7f1fef311640 1 -- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fcc06a430 msgr2=0x7f1fcc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.454+0000 7f1fef311640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fcc06a430 0x7f1fcc06c8f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f1fe815d5d0 tx=0x7f1fe400b040 comp rx=0 tx=0).stop 2023-12-19T18:21:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.454+0000 7f1fef311640 1 -- 172.21.15.27:0/764470763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fe8070650 msgr2=0x7f1fe80c50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.454+0000 7f1fef311640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fe8070650 0x7f1fe80c50b0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80098c0 tx=0x7f1fd80046c0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.454+0000 7f1fef311640 1 -- 172.21.15.27:0/764470763 shutdown_connections 2023-12-19T18:21:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.454+0000 7f1fef311640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fcc06a430 0x7f1fcc06c8f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.454+0000 7f1fef311640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fe8070a90 0x7f1fe815b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.454+0000 7f1fef311640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fe8070650 0x7f1fe80c50b0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.454+0000 7f1fef311640 1 --2- 172.21.15.27:0/764470763 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fe806c030 0x7f1fe81efe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.454+0000 7f1fef311640 1 -- 172.21.15.27:0/764470763 >> 172.21.15.27:0/764470763 conn(0x7f1fe80cb3f0 msgr2=0x7f1fe80cf230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:14.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.454+0000 7f1fef311640 1 -- 172.21.15.27:0/764470763 shutdown_connections 2023-12-19T18:21:14.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.454+0000 7f1fef311640 1 -- 172.21.15.27:0/764470763 wait complete. 2023-12-19T18:21:14.475 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:21:14.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.707+0000 7fa32b032640 1 Processor -- start 2023-12-19T18:21:14.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.707+0000 7fa32b032640 1 -- start start 2023-12-19T18:21:14.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.709+0000 7fa32b032640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa324159c40 0x7fa32406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.709+0000 7fa32b032640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa32415a020 0x7fa32406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.709+0000 7fa32b032640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa32415b180 con 0x7fa32406c880 2023-12-19T18:21:14.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.709+0000 7fa32b032640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3240ca850 con 0x7fa324159c40 2023-12-19T18:21:14.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.709+0000 7fa32b032640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32415a570 con 0x7fa32415a020 2023-12-19T18:21:14.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.709+0000 7fa323fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa32406c880 0x7fa3240cd800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56542/0 (socket says 172.21.15.27:56542) 2023-12-19T18:21:14.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.709+0000 7fa323fff640 1 -- 172.21.15.27:0/1451847875 learned_addr learned my addr 172.21.15.27:0/1451847875 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:14.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.709+0000 7fa328dce640 1 --2- 172.21.15.27:0/1451847875 >> v2:172.21.15.192:3300/0 conn(0x7fa324159c40 0x7fa32406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.709+0000 7fa3295cf640 1 --2- 172.21.15.27:0/1451847875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa32415a020 0x7fa32406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743052907 0 0) 0x7fa32415b180 con 0x7fa32406c880 2023-12-19T18:21:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa314003520 con 0x7fa32406c880 2023-12-19T18:21:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2836810041 0 0) 0x7fa314003520 con 0x7fa32406c880 2023-12-19T18:21:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa32415a020 msgr2=0x7fa32406ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 --2- 172.21.15.27:0/1451847875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa32415a020 0x7fa32406ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 >> v2:172.21.15.192:3300/0 conn(0x7fa324159c40 msgr2=0x7fa32406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 --2- 172.21.15.27:0/1451847875 >> v2:172.21.15.192:3300/0 conn(0x7fa324159c40 0x7fa32406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa32415a850 con 0x7fa32406c880 2023-12-19T18:21:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fa310001670 con 0x7fa32406c880 2023-12-19T18:21:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 >> v1:172.21.15.27:6789/0 conn(0x7fa32406c880 legacy=0x7fa3240cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 --2- 172.21.15.27:0/1451847875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa32415a020 0x7fa314003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 --2- 172.21.15.27:0/1451847875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa314003f40 0x7fa3140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 --2- 172.21.15.27:0/1451847875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa314004d00 0x7fa314005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240ca850 con 0x7fa314004d00 2023-12-19T18:21:14.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa32415a570 con 0x7fa314003f40 2023-12-19T18:21:14.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.710+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa31400b980 con 0x7fa32415a020 2023-12-19T18:21:14.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.711+0000 7fa3295cf640 1 --2- 172.21.15.27:0/1451847875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa314004d00 0x7fa314005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.711+0000 7fa328dce640 1 --2- 172.21.15.27:0/1451847875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa32415a020 0x7fa314003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.711+0000 7fa323fff640 1 --2- 172.21.15.27:0/1451847875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa314003f40 0x7fa3140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.711+0000 7fa3295cf640 1 -- 172.21.15.27:0/1451847875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa32415a020 msgr2=0x7fa314003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:14.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.711+0000 7fa3295cf640 1 --2- 172.21.15.27:0/1451847875 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa32415a020 0x7fa314003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.711+0000 7fa3295cf640 1 -- 172.21.15.27:0/1451847875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa314003f40 msgr2=0x7fa3140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:14.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.711+0000 7fa3295cf640 1 --2- 172.21.15.27:0/1451847875 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa314003f40 0x7fa3140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.711+0000 7fa3295cf640 1 -- 172.21.15.27:0/1451847875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa314003520 con 0x7fa314004d00 2023-12-19T18:21:14.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.712+0000 7fa3295cf640 1 --2- 172.21.15.27:0/1451847875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa314004d00 0x7fa314005180 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00e170 tx=0x7fa3140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0d7ca716afb09d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:14.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.712+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c02cbf0 con 0x7fa314004d00 2023-12-19T18:21:14.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.713+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31c00f040 con 0x7fa314004d00 2023-12-19T18:21:14.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.713+0000 7fa3237fe640 1 -- 172.21.15.27:0/1451847875 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c034440 con 0x7fa314004d00 2023-12-19T18:21:14.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.713+0000 7fa32b032640 1 -- 172.21.15.27:0/1451847875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa314004d00 msgr2=0x7fa314005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:14.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.713+0000 7fa32b032640 1 --2- 172.21.15.27:0/1451847875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa314004d00 0x7fa314005180 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00e170 tx=0x7fa3140057f0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.713+0000 7fa3295cf640 1 -- 172.21.15.27:0/1451847875 reap_dead start 2023-12-19T18:21:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.713+0000 7fa32b032640 1 -- 172.21.15.27:0/1451847875 shutdown_connections 2023-12-19T18:21:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.713+0000 7fa32b032640 1 -- 172.21.15.27:0/1451847875 >> 172.21.15.27:0/1451847875 conn(0x7fa32406a330 msgr2=0x7fa3240b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.713+0000 7fa32b032640 1 -- 172.21.15.27:0/1451847875 shutdown_connections 2023-12-19T18:21:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.713+0000 7fa32b032640 1 -- 172.21.15.27:0/1451847875 wait complete. 2023-12-19T18:21:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.715+0000 7fa32b032640 1 Processor -- start 2023-12-19T18:21:14.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.715+0000 7fa32b032640 1 -- start start 2023-12-19T18:21:14.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.716+0000 7fa32b032640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa314004d00 0x7fa3240c1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.716+0000 7fa32b032640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa324140d40 0x7fa32406ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.716+0000 7fa32b032640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3241411c0 0x7fa324070480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.716+0000 7fa32b032640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240b1c20 con 0x7fa3241411c0 2023-12-19T18:21:14.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.716+0000 7fa32b032640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240b3010 con 0x7fa324140d40 2023-12-19T18:21:14.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.716+0000 7fa32b032640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240b35b0 con 0x7fa314004d00 2023-12-19T18:21:14.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.716+0000 7fa3295cf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3241411c0 0x7fa324070480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.716+0000 7fa3295cf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3241411c0 0x7fa324070480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35234/0 (socket says 172.21.15.27:35234) 2023-12-19T18:21:14.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.716+0000 7fa3295cf640 1 -- 172.21.15.27:0/1838973317 learned_addr learned my addr 172.21.15.27:0/1838973317 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:14.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.717+0000 7fa328dce640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa314004d00 0x7fa3240c1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.717+0000 7fa3295cf640 1 -- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa314004d00 msgr2=0x7fa3240c1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:14.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.717+0000 7fa3295cf640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa314004d00 0x7fa3240c1c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.717+0000 7fa3295cf640 1 -- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa324140d40 msgr2=0x7fa32406ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:14.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.717+0000 7fa323fff640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa324140d40 0x7fa32406ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.717+0000 7fa3295cf640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa324140d40 0x7fa32406ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.717+0000 7fa3295cf640 1 -- 172.21.15.27:0/1838973317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa324159750 con 0x7fa3241411c0 2023-12-19T18:21:14.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.717+0000 7fa3295cf640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3241411c0 0x7fa324070480 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009f90 tx=0x7fa31c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:14.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.717+0000 7fa3217fa640 1 -- 172.21.15.27:0/1838973317 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c043070 con 0x7fa3241411c0 2023-12-19T18:21:14.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.718+0000 7fa3217fa640 1 -- 172.21.15.27:0/1838973317 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31c02ce90 con 0x7fa3241411c0 2023-12-19T18:21:14.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.718+0000 7fa3217fa640 1 -- 172.21.15.27:0/1838973317 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c034d90 con 0x7fa3241411c0 2023-12-19T18:21:14.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.718+0000 7fa32b032640 1 -- 172.21.15.27:0/1838973317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa324159190 con 0x7fa3241411c0 2023-12-19T18:21:14.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.718+0000 7fa32b032640 1 -- 172.21.15.27:0/1838973317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3240b3240 con 0x7fa3241411c0 2023-12-19T18:21:14.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.719+0000 7fa32b032640 1 -- 172.21.15.27:0/1838973317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa324068e90 con 0x7fa3241411c0 2023-12-19T18:21:14.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.720+0000 7fa3217fa640 1 -- 172.21.15.27:0/1838973317 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa31c03e440 con 0x7fa3241411c0 2023-12-19T18:21:14.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.721+0000 7fa3217fa640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa30806a4b0 0x7fa30806c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:14.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.721+0000 7fa3217fa640 1 -- 172.21.15.27:0/1838973317 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa31c0a9fd0 con 0x7fa3241411c0 2023-12-19T18:21:14.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.722+0000 7fa328dce640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa30806a4b0 0x7fa30806c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:14.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.723+0000 7fa328dce640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa30806a4b0 0x7fa30806c970 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa3180099d0 tx=0x7fa318008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:14.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.726+0000 7fa3217fa640 1 -- 172.21.15.27:0/1838973317 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa31c075750 con 0x7fa3241411c0 2023-12-19T18:21:14.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.960+0000 7fa32b032640 1 -- 172.21.15.27:0/1838973317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa32415ade0 con 0x7fa3241411c0 2023-12-19T18:21:14.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.961+0000 7fa3217fa640 1 -- 172.21.15.27:0/1838973317 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4714 (secure 0 0 0) 0x7fa31c078d70 con 0x7fa3241411c0 2023-12-19T18:21:14.961 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:14.961 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2779436922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2779436922},{"type":"v1","addr":"172.21.15.192:6836","nonce":2779436922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3121062779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3121062779},{"type":"v1","addr":"172.21.15.192:6837","nonce":3121062779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5357,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2032553869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2032553869},{"type":"v1","addr":"172.21.15.27:6835","nonce":2032553869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:21:08.281615+0000","modified":"2023-12-19T18:21:11.800655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/252970177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":252970177},{"type":"v1","addr":"172.21.15.27:6834","nonce":252970177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T18:21:14.961 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 46 2023-12-19T18:21:14.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.963+0000 7fa32b032640 1 -- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa30806a4b0 msgr2=0x7fa30806c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:14.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.963+0000 7fa32b032640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa30806a4b0 0x7fa30806c970 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa3180099d0 tx=0x7fa318008040 comp rx=0 tx=0).stop 2023-12-19T18:21:14.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.963+0000 7fa32b032640 1 -- 172.21.15.27:0/1838973317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3241411c0 msgr2=0x7fa324070480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:14.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.963+0000 7fa32b032640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3241411c0 0x7fa324070480 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7fa31c009f90 tx=0x7fa31c009750 comp rx=0 tx=0).stop 2023-12-19T18:21:14.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.963+0000 7fa32b032640 1 -- 172.21.15.27:0/1838973317 shutdown_connections 2023-12-19T18:21:14.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.963+0000 7fa32b032640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa30806a4b0 0x7fa30806c970 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.963+0000 7fa32b032640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3241411c0 0x7fa324070480 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.963+0000 7fa32b032640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa324140d40 0x7fa32406ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.963+0000 7fa32b032640 1 --2- 172.21.15.27:0/1838973317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa314004d00 0x7fa3240c1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:14.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.964+0000 7fa32b032640 1 -- 172.21.15.27:0/1838973317 >> 172.21.15.27:0/1838973317 conn(0x7fa32406a330 msgr2=0x7fa3240af300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:14.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.964+0000 7fa32b032640 1 -- 172.21.15.27:0/1838973317 shutdown_connections 2023-12-19T18:21:14.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:14.964+0000 7fa32b032640 1 -- 172.21.15.27:0/1838973317 wait complete. 2023-12-19T18:21:15.387 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:21:08.281615+0000', 'modified': '2023-12-19T18:21:11.800655+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6885}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6885': {'gid': 6885, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6834/252970177', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6832', 'nonce': 252970177}, {'type': 'v1', 'addr': '172.21.15.27:6834', 'nonce': 252970177}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:21:15.388 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:21:15.388 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-19T18:21:15.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.635+0000 7fbd5dfc6640 1 Processor -- start 2023-12-19T18:21:15.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.635+0000 7fbd5dfc6640 1 -- start start 2023-12-19T18:21:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.636+0000 7fbd5dfc6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbd580b49d0 0x7fbd58160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.636+0000 7fbd5dfc6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd580b89b0 0x7fbd5815c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.637+0000 7fbd5dfc6640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd580b5ca0 con 0x7fbd580b4db0 2023-12-19T18:21:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.637+0000 7fbd5dfc6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd58161800 con 0x7fbd580b49d0 2023-12-19T18:21:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.637+0000 7fbd5dfc6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58161240 con 0x7fbd580b89b0 2023-12-19T18:21:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.637+0000 7fbd5cfc4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbd580b49d0 0x7fbd58160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.637+0000 7fbd5cfc4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbd580b49d0 0x7fbd58160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35240/0 (socket says 172.21.15.27:35240) 2023-12-19T18:21:15.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.637+0000 7fbd5cfc4640 1 -- 172.21.15.27:0/3538061511 learned_addr learned my addr 172.21.15.27:0/3538061511 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:15.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.637+0000 7fbd5cfc4640 1 -- 172.21.15.27:0/3538061511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd580b89b0 msgr2=0x7fbd5815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.637+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/3538061511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd580b89b0 0x7fbd5815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.637+0000 7fbd5cfc4640 1 -- 172.21.15.27:0/3538061511 >> v1:172.21.15.192:6789/0 conn(0x7fbd580b4db0 legacy=0x7fbd5806bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.637+0000 7fbd5cfc4640 1 -- 172.21.15.27:0/3538061511 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd58161520 con 0x7fbd580b49d0 2023-12-19T18:21:15.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.638+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/3538061511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd580b49d0 0x7fbd58160940 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009ea0 tx=0x7fbd4802bfa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f46f6a2eab8e49ce server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:15.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.639+0000 7fbd577fe640 1 -- 172.21.15.27:0/3538061511 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4802cb40 con 0x7fbd580b49d0 2023-12-19T18:21:15.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.639+0000 7fbd577fe640 1 -- 172.21.15.27:0/3538061511 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4802cca0 con 0x7fbd580b49d0 2023-12-19T18:21:15.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.639+0000 7fbd577fe640 1 -- 172.21.15.27:0/3538061511 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48034610 con 0x7fbd580b49d0 2023-12-19T18:21:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.639+0000 7fbd5dfc6640 1 -- 172.21.15.27:0/3538061511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd580b49d0 msgr2=0x7fbd58160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.639+0000 7fbd5dfc6640 1 --2- 172.21.15.27:0/3538061511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd580b49d0 0x7fbd58160940 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7fbd48009ea0 tx=0x7fbd4802bfa0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.640+0000 7fbd5dfc6640 1 -- 172.21.15.27:0/3538061511 shutdown_connections 2023-12-19T18:21:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.640+0000 7fbd5dfc6640 1 --2- 172.21.15.27:0/3538061511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd580b89b0 0x7fbd5815c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.640+0000 7fbd5dfc6640 1 --2- 172.21.15.27:0/3538061511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd580b49d0 0x7fbd58160940 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.640+0000 7fbd5dfc6640 1 -- 172.21.15.27:0/3538061511 >> 172.21.15.27:0/3538061511 conn(0x7fbd580cf760 msgr2=0x7fbd580cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.640+0000 7fbd5dfc6640 1 -- 172.21.15.27:0/3538061511 shutdown_connections 2023-12-19T18:21:15.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.640+0000 7fbd5dfc6640 1 -- 172.21.15.27:0/3538061511 wait complete. 2023-12-19T18:21:15.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.641+0000 7fbd5dfc6640 1 Processor -- start 2023-12-19T18:21:15.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.642+0000 7fbd5dfc6640 1 -- start start 2023-12-19T18:21:15.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.642+0000 7fbd5dfc6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd580b49d0 0x7fbd58145630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.642+0000 7fbd5dfc6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd580b4db0 0x7fbd581f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.642+0000 7fbd5dfc6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd580b89b0 0x7fbd581435a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.642+0000 7fbd5dfc6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd580cbff0 con 0x7fbd580b4db0 2023-12-19T18:21:15.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.642+0000 7fbd5dfc6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58145c60 con 0x7fbd580b89b0 2023-12-19T18:21:15.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.642+0000 7fbd5dfc6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd58145f40 con 0x7fbd580b49d0 2023-12-19T18:21:15.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.643+0000 7fbd57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd580b4db0 0x7fbd581f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.643+0000 7fbd57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd580b4db0 0x7fbd581f2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35248/0 (socket says 172.21.15.27:35248) 2023-12-19T18:21:15.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.643+0000 7fbd57fff640 1 -- 172.21.15.27:0/3598675459 learned_addr learned my addr 172.21.15.27:0/3598675459 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:15.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.643+0000 7fbd5d7c5640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd580b89b0 0x7fbd581435a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.643+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd580b49d0 0x7fbd58145630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.643+0000 7fbd57fff640 1 -- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd580b49d0 msgr2=0x7fbd58145630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:15.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.643+0000 7fbd57fff640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd580b49d0 0x7fbd58145630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.643+0000 7fbd57fff640 1 -- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd580b89b0 msgr2=0x7fbd581435a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:15.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.643+0000 7fbd57fff640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd580b89b0 0x7fbd581435a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.643+0000 7fbd57fff640 1 -- 172.21.15.27:0/3598675459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd48009b30 con 0x7fbd580b4db0 2023-12-19T18:21:15.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.644+0000 7fbd57fff640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd580b4db0 0x7fbd581f2c10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00b410 tx=0x7fbd4c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:15.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.644+0000 7fbd557fa640 1 -- 172.21.15.27:0/3598675459 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c011830 con 0x7fbd580b4db0 2023-12-19T18:21:15.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.644+0000 7fbd557fa640 1 -- 172.21.15.27:0/3598675459 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c0119b0 con 0x7fbd580b4db0 2023-12-19T18:21:15.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.644+0000 7fbd557fa640 1 -- 172.21.15.27:0/3598675459 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c011d60 con 0x7fbd580b4db0 2023-12-19T18:21:15.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.644+0000 7fbd5dfc6640 1 -- 172.21.15.27:0/3598675459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd58146220 con 0x7fbd580b4db0 2023-12-19T18:21:15.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.644+0000 7fbd5dfc6640 1 -- 172.21.15.27:0/3598675459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd581f3150 con 0x7fbd580b4db0 2023-12-19T18:21:15.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.648+0000 7fbd557fa640 1 -- 172.21.15.27:0/3598675459 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbd4c01a400 con 0x7fbd580b4db0 2023-12-19T18:21:15.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.649+0000 7fbd557fa640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbd4406a460 0x7fbd4406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.649+0000 7fbd557fa640 1 -- 172.21.15.27:0/3598675459 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd4c0889b0 con 0x7fbd580b4db0 2023-12-19T18:21:15.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.649+0000 7fbd61d99740 1 Processor -- start 2023-12-19T18:21:15.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.649+0000 7fbd61d99740 1 -- start start 2023-12-19T18:21:15.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.649+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbd4406a460 0x7fbd4406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.650+0000 7fbd61d99740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf360110 0x558bdf3604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.650+0000 7fbd61d99740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf330090 0x558bdf35d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.650+0000 7fbd61d99740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf333c90 0x558bdf35d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.650+0000 7fbd61d99740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bdf30b650 con 0x558bdf360110 2023-12-19T18:21:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.650+0000 7fbd61d99740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bdf2a0000 con 0x558bdf330090 2023-12-19T18:21:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.650+0000 7fbd61d99740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bdf29d790 con 0x558bdf333c90 2023-12-19T18:21:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.650+0000 7fbd5cfc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf360110 0x558bdf3604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.650+0000 7fbd5cfc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf360110 0x558bdf3604f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35264/0 (socket says 172.21.15.27:35264) 2023-12-19T18:21:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.650+0000 7fbd5cfc4640 1 -- 172.21.15.27:0/3919255200 learned_addr learned my addr 172.21.15.27:0/3919255200 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.651+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbd4406a460 0x7fbd4406c920 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbd48035000 tx=0x7fbd48030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.651+0000 7fbd57fff640 1 --2- 172.21.15.27:0/3919255200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf330090 0x558bdf35d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.651+0000 7fbd5d7c5640 1 --2- 172.21.15.27:0/3919255200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf333c90 0x558bdf35d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.651+0000 7fbd5cfc4640 1 -- 172.21.15.27:0/3919255200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf333c90 msgr2=0x558bdf35d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.651+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/3919255200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf333c90 0x558bdf35d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.651+0000 7fbd5cfc4640 1 -- 172.21.15.27:0/3919255200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf330090 msgr2=0x558bdf35d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.651+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/3919255200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf330090 0x558bdf35d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.651+0000 7fbd5cfc4640 1 -- 172.21.15.27:0/3919255200 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bdf260710 con 0x558bdf360110 2023-12-19T18:21:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.651+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/3919255200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf360110 0x558bdf3604f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fbd48036040 tx=0x7fbd48009000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa38b6d8d9f9b887 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.652+0000 7fbd3e7fc640 1 -- 172.21.15.27:0/3919255200 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4804f070 con 0x558bdf360110 2023-12-19T18:21:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.652+0000 7fbd3e7fc640 1 -- 172.21.15.27:0/3919255200 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd48047040 con 0x558bdf360110 2023-12-19T18:21:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.652+0000 7fbd3e7fc640 1 -- 172.21.15.27:0/3919255200 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48034e90 con 0x558bdf360110 2023-12-19T18:21:15.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.652+0000 7fbd61d99740 1 -- 172.21.15.27:0/3919255200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf360110 msgr2=0x558bdf3604f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.652+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3919255200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf360110 0x558bdf3604f0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fbd48036040 tx=0x7fbd48009000 comp rx=0 tx=0).stop 2023-12-19T18:21:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.652+0000 7fbd61d99740 1 -- 172.21.15.27:0/3919255200 shutdown_connections 2023-12-19T18:21:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.652+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3919255200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf333c90 0x558bdf35d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.652+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3919255200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf330090 0x558bdf35d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.652+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3919255200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf360110 0x558bdf3604f0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.652+0000 7fbd61d99740 1 -- 172.21.15.27:0/3919255200 >> 172.21.15.27:0/3919255200 conn(0x558bdf3323a0 msgr2=0x558bdf37af20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.652+0000 7fbd61d99740 1 -- 172.21.15.27:0/3919255200 shutdown_connections 2023-12-19T18:21:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.653+0000 7fbd61d99740 1 -- 172.21.15.27:0/3919255200 wait complete. 2023-12-19T18:21:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.653+0000 7fbd61d99740 1 -- start start 2023-12-19T18:21:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.654+0000 7fbd61d99740 1 Processor -- start 2023-12-19T18:21:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.654+0000 7fbd61d99740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf333c90 0x558bdf359280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd61d99740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf360110 0x558bdf3597c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd61d99740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 0x558bdf38a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd61d99740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bdf299080 con 0x558bdf355810 2023-12-19T18:21:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd61d99740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bdf23a850 con 0x558bdf360110 2023-12-19T18:21:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd61d99740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bdf23a4f0 con 0x558bdf333c90 2023-12-19T18:21:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd61d99740 10 client.0 _ll_register_callbacks cb 0x558bdf2929c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:21:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd5d7c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 0x558bdf38a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd5d7c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 0x558bdf38a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35276/0 (socket says 172.21.15.27:35276) 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd5d7c5640 1 -- 172.21.15.27:0/2605194638 learned_addr learned my addr 172.21.15.27:0/2605194638 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd57fff640 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf360110 0x558bdf3597c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf333c90 0x558bdf359280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd5d7c5640 1 -- 172.21.15.27:0/2605194638 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf333c90 msgr2=0x558bdf359280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd5d7c5640 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf333c90 0x558bdf359280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd5d7c5640 1 -- 172.21.15.27:0/2605194638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf360110 msgr2=0x558bdf3597c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd5d7c5640 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf360110 0x558bdf3597c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd5d7c5640 1 -- 172.21.15.27:0/2605194638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd58161520 con 0x558bdf355810 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.655+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf333c90 0x558bdf359280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.656+0000 7fbd5d7c5640 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 0x558bdf38a5e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fbd50003bc0 tx=0x7fbd500014f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2657f5d39838f444 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.656+0000 7fbd377fe640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.656+0000 7fbd377fe640 1 -- 172.21.15.27:0/2605194638 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd50001d20 con 0x558bdf355810 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.656+0000 7fbd377fe640 1 -- 172.21.15.27:0/2605194638 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd50016b10 con 0x558bdf355810 2023-12-19T18:21:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.656+0000 7fbd61d99740 1 -- 172.21.15.27:0/2605194638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558bdf2c35d0 con 0x558bdf355810 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.656+0000 7fbd377fe640 1 -- 172.21.15.27:0/2605194638 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd50016cd0 con 0x558bdf355810 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.656+0000 7fbd61d99740 10 client.7014 fetch_fsmap learned FSMap version 46 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.656+0000 7fbd61d99740 1 -- 172.21.15.27:0/2605194638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x558bdf297700 con 0x558bdf355810 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.657+0000 7fbd377fe640 1 -- 172.21.15.27:0/2605194638 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558bdf2c35d0 con 0x558bdf355810 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.657+0000 7fbd377fe640 1 -- 172.21.15.27:0/2605194638 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fbd500124d0 con 0x558bdf355810 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.657+0000 7fbd61d99740 10 client.7014 fetch_fsmap finished waiting for FSMap version 46 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.657+0000 7fbd61d99740 10 client.7014 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.657+0000 7fbd61d99740 20 client.7014 populate_metadata read hostname 'smithi027' 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.657+0000 7fbd61d99740 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.27:6832/252970177,v1:172.21.15.27:6834/252970177] conn(0x558bdf3569a0 0x558bdf356d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.657+0000 7fbd61d99740 4 client.7014 mds_command: new command op to 6885 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.657+0000 7fbd61d99740 1 -- 172.21.15.27:0/2605194638 --> [v2:172.21.15.27:6832/252970177,v1:172.21.15.27:6834/252970177] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558bdf28b8d0 con 0x558bdf3569a0 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.657+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.27:6832/252970177,v1:172.21.15.27:6834/252970177] conn(0x558bdf3569a0 0x558bdf356d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.658+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.27:6832/252970177,v1:172.21.15.27:6834/252970177] conn(0x558bdf3569a0 0x558bdf356d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0f98b9cdaab7c5 server_cookie=262239ca0977bcce in_seq=0 out_seq=0 2023-12-19T18:21:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.658+0000 7fbd377fe640 10 client.7014 ms_handle_connect on v2:172.21.15.27:6832/252970177 2023-12-19T18:21:15.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd377fe640 1 -- 172.21.15.27:0/2605194638 <== mds.0 v2:172.21.15.27:6832/252970177 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558bdf28b8d0 con 0x558bdf3569a0 2023-12-19T18:21:15.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd377fe640 10 client.7014 handle_command_reply: tid=0 2023-12-19T18:21:15.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd61d99740 1 client.7014 shutdown 2023-12-19T18:21:15.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd61d99740 1 -- 172.21.15.27:0/2605194638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 msgr2=0x558bdf38a5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd61d99740 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 0x558bdf38a5e0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7fbd50003bc0 tx=0x7fbd500014f0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd61d99740 1 -- 172.21.15.27:0/2605194638 shutdown_connections 2023-12-19T18:21:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd61d99740 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.27:6832/252970177,v1:172.21.15.27:6834/252970177] conn(0x558bdf3569a0 0x558bdf356d80 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd61d99740 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 0x558bdf38a5e0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd61d99740 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf360110 0x558bdf3597c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd61d99740 1 --2- 172.21.15.27:0/2605194638 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf333c90 0x558bdf359280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd61d99740 1 -- 172.21.15.27:0/2605194638 >> 172.21.15.27:0/2605194638 conn(0x558bdf330090 msgr2=0x558bdf3345b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd377fe640 0 client.7014 ms_handle_reset on v2:172.21.15.27:6832/252970177 2023-12-19T18:21:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd61d99740 1 -- 172.21.15.27:0/2605194638 shutdown_connections 2023-12-19T18:21:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd61d99740 1 -- 172.21.15.27:0/2605194638 wait complete. 2023-12-19T18:21:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.659+0000 7fbd61d99740 20 client.7014 trim_cache size 0 max 16384 2023-12-19T18:21:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.676+0000 7fbd61d99740 1 Processor -- start 2023-12-19T18:21:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.676+0000 7fbd61d99740 1 -- start start 2023-12-19T18:21:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.677+0000 7fbd61d99740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf333c90 0x558bdf356d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.677+0000 7fbd61d99740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 0x558bdf38a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.677+0000 7fbd61d99740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf360110 0x558bdf37e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.677+0000 7fbd61d99740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bdf000f20 con 0x558bdf355810 2023-12-19T18:21:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.677+0000 7fbd61d99740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bdf354d20 con 0x558bdf333c90 2023-12-19T18:21:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.677+0000 7fbd61d99740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bdf336c70 con 0x558bdf360110 2023-12-19T18:21:15.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.677+0000 7fbd57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 0x558bdf38a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.677+0000 7fbd57fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 0x558bdf38a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35292/0 (socket says 172.21.15.27:35292) 2023-12-19T18:21:15.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.678+0000 7fbd57fff640 1 -- 172.21.15.27:0/3301811298 learned_addr learned my addr 172.21.15.27:0/3301811298 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:15.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.678+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/3301811298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf333c90 0x558bdf356d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.678+0000 7fbd5d7c5640 1 --2- 172.21.15.27:0/3301811298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf360110 0x558bdf37e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.678+0000 7fbd57fff640 1 -- 172.21.15.27:0/3301811298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf360110 msgr2=0x558bdf37e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.678+0000 7fbd57fff640 1 --2- 172.21.15.27:0/3301811298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf360110 0x558bdf37e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.678+0000 7fbd57fff640 1 -- 172.21.15.27:0/3301811298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf333c90 msgr2=0x558bdf356d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.678+0000 7fbd57fff640 1 --2- 172.21.15.27:0/3301811298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf333c90 0x558bdf356d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.678+0000 7fbd57fff640 1 -- 172.21.15.27:0/3301811298 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558bdf2cd280 con 0x558bdf355810 2023-12-19T18:21:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.678+0000 7fbd5d7c5640 1 --2- 172.21.15.27:0/3301811298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf360110 0x558bdf37e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:21:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.678+0000 7fbd57fff640 1 --2- 172.21.15.27:0/3301811298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 0x558bdf38a310 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0579f0 tx=0x7fbd4c0583f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c0632b445bf715 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:15.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.679+0000 7fbd3effd640 1 -- 172.21.15.27:0/3301811298 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c05fcd0 con 0x558bdf355810 2023-12-19T18:21:15.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.679+0000 7fbd3effd640 1 -- 172.21.15.27:0/3301811298 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c055e30 con 0x558bdf355810 2023-12-19T18:21:15.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.679+0000 7fbd3effd640 1 -- 172.21.15.27:0/3301811298 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c06f210 con 0x558bdf355810 2023-12-19T18:21:15.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.679+0000 7fbd61d99740 1 -- 172.21.15.27:0/3301811298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 msgr2=0x558bdf38a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:15.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.679+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3301811298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 0x558bdf38a310 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c0579f0 tx=0x7fbd4c0583f0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.679+0000 7fbd61d99740 1 -- 172.21.15.27:0/3301811298 shutdown_connections 2023-12-19T18:21:15.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.679+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3301811298 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf360110 0x558bdf37e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.679+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3301811298 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf355810 0x558bdf38a310 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.679+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3301811298 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf333c90 0x558bdf356d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.679+0000 7fbd61d99740 1 -- 172.21.15.27:0/3301811298 >> 172.21.15.27:0/3301811298 conn(0x558bdf330090 msgr2=0x558bdf35ae60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:15.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.679+0000 7fbd61d99740 1 -- 172.21.15.27:0/3301811298 shutdown_connections 2023-12-19T18:21:15.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.680+0000 7fbd61d99740 1 -- 172.21.15.27:0/3301811298 wait complete. 2023-12-19T18:21:15.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.680+0000 7fbd61d99740 1 -- start start 2023-12-19T18:21:15.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.680+0000 7fbd61d99740 1 Processor -- start 2023-12-19T18:21:15.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.681+0000 7fbd61d99740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf333c90 0x558bdf3de790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.681+0000 7fbd61d99740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf355810 0x558bdf32f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.681+0000 7fbd61d99740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf360110 0x558bdf32f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.681+0000 7fbd61d99740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bdf383eb0 con 0x558bdf333c90 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.681+0000 7fbd61d99740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x558bdf383820 con 0x558bdf360110 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.681+0000 7fbd61d99740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x558bdf3e1b70 con 0x558bdf355810 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.682+0000 7fbd61d99740 10 client.0 _ll_register_callbacks cb 0x7fbd50007e70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.682+0000 7fbd5cfc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf333c90 0x558bdf3de790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.682+0000 7fbd5cfc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf333c90 0x558bdf3de790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35300/0 (socket says 172.21.15.27:35300) 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.682+0000 7fbd5cfc4640 1 -- 172.21.15.27:0/3500547778 learned_addr learned my addr 172.21.15.27:0/3500547778 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.682+0000 7fbd57fff640 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf355810 0x558bdf32f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.682+0000 7fbd5d7c5640 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf360110 0x558bdf32f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.682+0000 7fbd5cfc4640 1 -- 172.21.15.27:0/3500547778 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf355810 msgr2=0x558bdf32f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.682+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf355810 0x558bdf32f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.682+0000 7fbd5cfc4640 1 -- 172.21.15.27:0/3500547778 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf360110 msgr2=0x558bdf32f570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.682+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf360110 0x558bdf32f570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.682+0000 7fbd5cfc4640 1 -- 172.21.15.27:0/3500547778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd581f3150 con 0x558bdf333c90 2023-12-19T18:21:15.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.682+0000 7fbd57fff640 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf355810 0x558bdf32f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.682+0000 7fbd5d7c5640 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf360110 0x558bdf32f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.683+0000 7fbd5cfc4640 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf333c90 0x558bdf3de790 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fbd48034610 tx=0x7fbd4802e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53bdfd2a6d7b15e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.683+0000 7fbd377fe640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.683+0000 7fbd377fe640 1 -- 172.21.15.27:0/3500547778 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4804f040 con 0x558bdf333c90 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.683+0000 7fbd377fe640 1 -- 172.21.15.27:0/3500547778 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd48057910 con 0x558bdf333c90 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.683+0000 7fbd61d99740 1 -- 172.21.15.27:0/3500547778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd48047040 con 0x558bdf333c90 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.683+0000 7fbd377fe640 1 -- 172.21.15.27:0/3500547778 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd48057ad0 con 0x558bdf333c90 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.683+0000 7fbd61d99740 10 client.7020 fetch_fsmap learned FSMap version 46 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.683+0000 7fbd61d99740 1 -- 172.21.15.27:0/3500547778 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x558bdf239a60 con 0x558bdf333c90 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.684+0000 7fbd377fe640 1 -- 172.21.15.27:0/3500547778 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd48047040 con 0x558bdf333c90 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.684+0000 7fbd377fe640 1 -- 172.21.15.27:0/3500547778 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fbd4804a070 con 0x558bdf333c90 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.684+0000 7fbd61d99740 10 client.7020 fetch_fsmap finished waiting for FSMap version 46 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.684+0000 7fbd61d99740 10 client.7020 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.684+0000 7fbd61d99740 20 client.7020 populate_metadata read hostname 'smithi027' 2023-12-19T18:21:15.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.684+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.27:6832/252970177,v1:172.21.15.27:6834/252970177] conn(0x558bdf37af90 0x558bdf37b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.684+0000 7fbd61d99740 4 client.7020 mds_command: new command op to 6885 tid=0[{"prefix": "status"}] 2023-12-19T18:21:15.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.684+0000 7fbd61d99740 1 -- 172.21.15.27:0/3500547778 --> [v2:172.21.15.27:6832/252970177,v1:172.21.15.27:6834/252970177] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558bdf292100 con 0x558bdf37af90 2023-12-19T18:21:15.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.684+0000 7fbd57fff640 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.27:6832/252970177,v1:172.21.15.27:6834/252970177] conn(0x558bdf37af90 0x558bdf37b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.684+0000 7fbd57fff640 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.27:6832/252970177,v1:172.21.15.27:6834/252970177] conn(0x558bdf37af90 0x558bdf37b370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=822c1274ae31443e server_cookie=dec4c4d1959bcfb3 in_seq=0 out_seq=0 2023-12-19T18:21:15.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.684+0000 7fbd377fe640 10 client.7020 ms_handle_connect on v2:172.21.15.27:6832/252970177 2023-12-19T18:21:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd377fe640 1 -- 172.21.15.27:0/3500547778 <== mds.0 v2:172.21.15.27:6832/252970177 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558bdf292100 con 0x558bdf37af90 2023-12-19T18:21:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd377fe640 10 client.7020 handle_command_reply: tid=0 2023-12-19T18:21:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd61d99740 1 client.7020 shutdown 2023-12-19T18:21:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd61d99740 1 -- 172.21.15.27:0/3500547778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf333c90 msgr2=0x558bdf3de790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf333c90 0x558bdf3de790 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7fbd48034610 tx=0x7fbd4802e070 comp rx=0 tx=0).stop 2023-12-19T18:21:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd61d99740 1 -- 172.21.15.27:0/3500547778 shutdown_connections 2023-12-19T18:21:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x558bdf360110 0x558bdf32f570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x558bdf355810 0x558bdf32f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.27:6832/252970177,v1:172.21.15.27:6834/252970177] conn(0x558bdf37af90 0x558bdf37b370 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd61d99740 1 --2- 172.21.15.27:0/3500547778 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x558bdf333c90 0x558bdf3de790 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd377fe640 0 client.7020 ms_handle_reset on v2:172.21.15.27:6832/252970177 2023-12-19T18:21:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd61d99740 1 -- 172.21.15.27:0/3500547778 >> 172.21.15.27:0/3500547778 conn(0x558bdf330090 msgr2=0x558bdf37eae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd61d99740 1 -- 172.21.15.27:0/3500547778 shutdown_connections 2023-12-19T18:21:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd61d99740 1 -- 172.21.15.27:0/3500547778 wait complete. 2023-12-19T18:21:15.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.685+0000 7fbd61d99740 20 client.7020 trim_cache size 0 max 16384 2023-12-19T18:21:15.689 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:21:15.689 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:21:15.689 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-12-19T18:21:15.689 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 6885, 2023-12-19T18:21:15.689 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-19T18:21:15.689 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-19T18:21:15.690 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-19T18:21:15.690 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 7.3987906460000001, 2023-12-19T18:21:15.690 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 46, 2023-12-19T18:21:15.690 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 117, 2023-12-19T18:21:15.690 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 117, 2023-12-19T18:21:15.690 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 12.974342209 2023-12-19T18:21:15.690 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:21:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.686+0000 7fbd5dfc6640 1 -- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbd4406a460 msgr2=0x7fbd4406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.686+0000 7fbd5dfc6640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbd4406a460 0x7fbd4406c920 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fbd48035000 tx=0x7fbd48030000 comp rx=0 tx=0).stop 2023-12-19T18:21:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.686+0000 7fbd5dfc6640 1 -- 172.21.15.27:0/3598675459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd580b4db0 msgr2=0x7fbd581f2c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.686+0000 7fbd5dfc6640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd580b4db0 0x7fbd581f2c10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00b410 tx=0x7fbd4c00b8d0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.686+0000 7fbd5dfc6640 1 -- 172.21.15.27:0/3598675459 shutdown_connections 2023-12-19T18:21:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.686+0000 7fbd5dfc6640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbd4406a460 0x7fbd4406c920 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.686+0000 7fbd5dfc6640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbd580b89b0 0x7fbd581435a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.686+0000 7fbd5dfc6640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd580b4db0 0x7fbd581f2c10 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.686+0000 7fbd5dfc6640 1 --2- 172.21.15.27:0/3598675459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbd580b49d0 0x7fbd58145630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.686+0000 7fbd5dfc6640 1 -- 172.21.15.27:0/3598675459 >> 172.21.15.27:0/3598675459 conn(0x7fbd580cf760 msgr2=0x7fbd580d2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.686+0000 7fbd5dfc6640 1 -- 172.21.15.27:0/3598675459 shutdown_connections 2023-12-19T18:21:15.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.686+0000 7fbd5dfc6640 1 -- 172.21.15.27:0/3598675459 wait complete. 2023-12-19T18:21:15.703 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:21:15.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.923+0000 7f4ebffff640 1 Processor -- start 2023-12-19T18:21:15.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.923+0000 7f4ebffff640 1 -- start start 2023-12-19T18:21:15.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.925+0000 7f4ebffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec00c8f90 0x7f4ec0160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.925+0000 7f4ebffff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4ec0160e40 0x7f4ec00cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.925+0000 7f4ebffff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec01612d0 con 0x7f4ec00cb300 2023-12-19T18:21:15.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.925+0000 7f4ebffff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec00c4480 con 0x7f4ec0160e40 2023-12-19T18:21:15.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.925+0000 7f4ebffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01468c0 con 0x7f4ec00c8f90 2023-12-19T18:21:15.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.925+0000 7f4ebeffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4ec00cb300 0x7f4ec00cb6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56544/0 (socket says 172.21.15.27:56544) 2023-12-19T18:21:15.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.925+0000 7f4ebeffd640 1 -- 172.21.15.27:0/775415044 learned_addr learned my addr 172.21.15.27:0/775415044 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:15.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebe7fc640 1 --2- 172.21.15.27:0/775415044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec00c8f90 0x7f4ec0160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640653292 0 0) 0x7f4eb0001110 con 0x7f4ec00cb300 2023-12-19T18:21:15.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebf7fe640 1 --2- 172.21.15.27:0/775415044 >> v2:172.21.15.192:3300/0 conn(0x7f4ec0160e40 0x7f4ec00cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ea8003520 con 0x7f4ec00cb300 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 853765217 0 0) 0x7f4ea8003520 con 0x7f4ec00cb300 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec00c8f90 msgr2=0x7f4ec0160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 --2- 172.21.15.27:0/775415044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec00c8f90 0x7f4ec0160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 >> v2:172.21.15.192:3300/0 conn(0x7f4ec0160e40 msgr2=0x7f4ec00cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 --2- 172.21.15.27:0/775415044 >> v2:172.21.15.192:3300/0 conn(0x7f4ec0160e40 0x7f4ec00cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec0145d60 con 0x7f4ec00cb300 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f4eb0001800 con 0x7f4ec00cb300 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 >> v1:172.21.15.27:6789/0 conn(0x7f4ec00cb300 legacy=0x7f4ec00cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 --2- 172.21.15.27:0/775415044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ea8003970 0x7f4ea8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 --2- 172.21.15.27:0/775415044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ea8004360 0x7f4ea8004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 --2- 172.21.15.27:0/775415044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea8004f60 0x7f4ea800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01468c0 con 0x7f4ea8004f60 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea800bec0 con 0x7f4ea8004360 2023-12-19T18:21:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.926+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ea800c030 con 0x7f4ea8003970 2023-12-19T18:21:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.927+0000 7f4ebe7fc640 1 --2- 172.21.15.27:0/775415044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec00c8f90 0x7f4ec0160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:21:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.927+0000 7f4ebf7fe640 1 --2- 172.21.15.27:0/775415044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea8004f60 0x7f4ea800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.927+0000 7f4ebf7fe640 1 -- 172.21.15.27:0/775415044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ea8003970 msgr2=0x7f4ea8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.927+0000 7f4ebf7fe640 1 --2- 172.21.15.27:0/775415044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ea8003970 0x7f4ea8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.927+0000 7f4ebf7fe640 1 -- 172.21.15.27:0/775415044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ea8004360 msgr2=0x7f4ea8004840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.927+0000 7f4ebf7fe640 1 --2- 172.21.15.27:0/775415044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ea8004360 0x7f4ea8004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.927+0000 7f4ebf7fe640 1 -- 172.21.15.27:0/775415044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ea8003520 con 0x7f4ea8004f60 2023-12-19T18:21:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.928+0000 7f4ebf7fe640 1 --2- 172.21.15.27:0/775415044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea8004f60 0x7f4ea800b980 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009fc0 tx=0x7f4ea80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9c3d71f89b4ad0e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.929+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb802cc30 con 0x7f4ea8004f60 2023-12-19T18:21:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.929+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb800f040 con 0x7f4ea8004f60 2023-12-19T18:21:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.929+0000 7f4ebdffb640 1 -- 172.21.15.27:0/775415044 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8034440 con 0x7f4ea8004f60 2023-12-19T18:21:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.929+0000 7f4ebffff640 1 -- 172.21.15.27:0/775415044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea8004f60 msgr2=0x7f4ea800b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.929+0000 7f4ebffff640 1 --2- 172.21.15.27:0/775415044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ea8004f60 0x7f4ea800b980 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8009fc0 tx=0x7f4ea80057f0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.929+0000 7f4ebf7fe640 1 -- 172.21.15.27:0/775415044 reap_dead start 2023-12-19T18:21:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.929+0000 7f4ebffff640 1 -- 172.21.15.27:0/775415044 shutdown_connections 2023-12-19T18:21:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.929+0000 7f4ebffff640 1 -- 172.21.15.27:0/775415044 >> 172.21.15.27:0/775415044 conn(0x7f4ec0069c70 msgr2=0x7f4ec006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.929+0000 7f4ebffff640 1 -- 172.21.15.27:0/775415044 shutdown_connections 2023-12-19T18:21:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.930+0000 7f4ebffff640 1 -- 172.21.15.27:0/775415044 wait complete. 2023-12-19T18:21:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.931+0000 7f4ebffff640 1 Processor -- start 2023-12-19T18:21:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.932+0000 7f4ebffff640 1 -- start start 2023-12-19T18:21:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.932+0000 7f4ebffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ea8004f60 0x7f4ec0160510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec00b4010 0x7f4ec01f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec00b2ef0 0x7f4ec00c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebffff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0160a50 con 0x7f4ec00b2ef0 2023-12-19T18:21:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebffff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec0144ea0 con 0x7f4ea8004f60 2023-12-19T18:21:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec01f2d00 con 0x7f4ec00b4010 2023-12-19T18:21:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec00b2ef0 0x7f4ec00c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebf7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec00b2ef0 0x7f4ec00c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35322/0 (socket says 172.21.15.27:35322) 2023-12-19T18:21:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebf7fe640 1 -- 172.21.15.27:0/3485073765 learned_addr learned my addr 172.21.15.27:0/3485073765 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebe7fc640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec00b4010 0x7f4ec01f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebeffd640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ea8004f60 0x7f4ec0160510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebf7fe640 1 -- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec00b4010 msgr2=0x7f4ec01f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebf7fe640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec00b4010 0x7f4ec01f2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebf7fe640 1 -- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ea8004f60 msgr2=0x7f4ec0160510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebf7fe640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ea8004f60 0x7f4ec0160510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebf7fe640 1 -- 172.21.15.27:0/3485073765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec01f2fe0 con 0x7f4ec00b2ef0 2023-12-19T18:21:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.933+0000 7f4ebe7fc640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec00b4010 0x7f4ec01f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.934+0000 7f4ebf7fe640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec00b2ef0 0x7f4ec00c5a10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8037040 tx=0x7f4eb8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.934+0000 7f4ea7fff640 1 -- 172.21.15.27:0/3485073765 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8043070 con 0x7f4ec00b2ef0 2023-12-19T18:21:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.934+0000 7f4ea7fff640 1 -- 172.21.15.27:0/3485073765 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eb8034ad0 con 0x7f4ec00b2ef0 2023-12-19T18:21:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.934+0000 7f4ea7fff640 1 -- 172.21.15.27:0/3485073765 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eb8034e60 con 0x7f4ec00b2ef0 2023-12-19T18:21:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.934+0000 7f4ebffff640 1 -- 172.21.15.27:0/3485073765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec01f32c0 con 0x7f4ec00b2ef0 2023-12-19T18:21:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.934+0000 7f4ebffff640 1 -- 172.21.15.27:0/3485073765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec00b29d0 con 0x7f4ec00b2ef0 2023-12-19T18:21:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.935+0000 7f4ebffff640 1 -- 172.21.15.27:0/3485073765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec004e840 con 0x7f4ec00b2ef0 2023-12-19T18:21:15.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.936+0000 7f4ea7fff640 1 -- 172.21.15.27:0/3485073765 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4eb8003990 con 0x7f4ec00b2ef0 2023-12-19T18:21:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.938+0000 7f4ea7fff640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4eac06a460 0x7f4eac06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.938+0000 7f4ea7fff640 1 -- 172.21.15.27:0/3485073765 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4eb80aa3c0 con 0x7f4ec00b2ef0 2023-12-19T18:21:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.938+0000 7f4ebeffd640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4eac06a460 0x7f4eac06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:15.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.940+0000 7f4ebeffd640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4eac06a460 0x7f4eac06c920 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0001eb0 tx=0x7f4eb0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:15.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:15.942+0000 7f4ea7fff640 1 -- 172.21.15.27:0/3485073765 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4eb8075910 con 0x7f4ec00b2ef0 2023-12-19T18:21:16.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.173+0000 7f4ebffff640 1 -- 172.21.15.27:0/3485073765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4ec00adc50 con 0x7f4ec00b2ef0 2023-12-19T18:21:16.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.174+0000 7f4ea7fff640 1 -- 172.21.15.27:0/3485073765 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4eb8039060 con 0x7f4ec00b2ef0 2023-12-19T18:21:16.174 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:21:16.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.176+0000 7f4ebffff640 1 -- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4eac06a460 msgr2=0x7f4eac06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:16.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.176+0000 7f4ebffff640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4eac06a460 0x7f4eac06c920 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4eb0001eb0 tx=0x7f4eb0009040 comp rx=0 tx=0).stop 2023-12-19T18:21:16.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.176+0000 7f4ebffff640 1 -- 172.21.15.27:0/3485073765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec00b2ef0 msgr2=0x7f4ec00c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:16.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.176+0000 7f4ebffff640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec00b2ef0 0x7f4ec00c5a10 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f4eb8037040 tx=0x7f4eb8009750 comp rx=0 tx=0).stop 2023-12-19T18:21:16.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.176+0000 7f4ebffff640 1 -- 172.21.15.27:0/3485073765 shutdown_connections 2023-12-19T18:21:16.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.176+0000 7f4ebffff640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4eac06a460 0x7f4eac06c920 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:16.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.176+0000 7f4ebffff640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec00b2ef0 0x7f4ec00c5a10 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:16.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.176+0000 7f4ebffff640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec00b4010 0x7f4ec01f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:16.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.176+0000 7f4ebffff640 1 --2- 172.21.15.27:0/3485073765 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ea8004f60 0x7f4ec0160510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:16.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.176+0000 7f4ebffff640 1 -- 172.21.15.27:0/3485073765 >> 172.21.15.27:0/3485073765 conn(0x7f4ec0069c70 msgr2=0x7f4ec00ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:16.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.177+0000 7f4ebffff640 1 -- 172.21.15.27:0/3485073765 shutdown_connections 2023-12-19T18:21:16.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:16.177+0000 7f4ebffff640 1 -- 172.21.15.27:0/3485073765 wait complete. 2023-12-19T18:21:16.196 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:21:16.196 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:21:16.196 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-12-19T18:21:16.196 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:21:16.196 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:21:16.196 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:21:16.197 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:21:16.197 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:21:16.197 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:21:16.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:16.197 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-12-19T18:21:16.251 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:21:16.251 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:21:16.251 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:21:16.251 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:16.251 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:21:16.251 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:16.251 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:21:16.251 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:21:16.251 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-12-19T18:21:16.251 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:21:16.251 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:21:16.251 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41964sec preferred_lft 41964sec 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:16.252 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::8053:76ff:fe69:c66c/64 scope link 2023-12-19T18:21:16.253 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:16.253 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:21:16.253 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:21:16.253 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::781d:d6ff:feb4:7a2d/64 scope link 2023-12-19T18:21:16.253 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:16.253 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:21:16.253 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether f2:29:63:cc:14:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:21:16.253 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::f029:63ff:fecc:140e/64 scope link 2023-12-19T18:21:16.253 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:16.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:16.254 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:21:16.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:21:16.308 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:21:16.343 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-12-19T18:21:16.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:16.346 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-12-19T18:21:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-12-19T18:21:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:21:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-19T18:21:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-19T18:21:16.444 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:21:16.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:16.445 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:21:16.445 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:21:16.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:21:16.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:21:16.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:21:16.445 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:21:16.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:16.704 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:21:16.763 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:21:16.763 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:21:16.764 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:21:16.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:16.764 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-12-19T18:21:16.872 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-19T18:21:16.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:21:16.872 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:21:16.933 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:21:16.933 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:21:16.933 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:21:16.990 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:21:17.101 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:21:17.105 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:21:17.213 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:21:17.218 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:21:17.219 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:17.219 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:21:17.279 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:21:17.279 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-19T18:21:17.288 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:21:17.383 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:21:17.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:21:17.462 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:21:17.465 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:21:17.465 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:17.465 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:21:17.496 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-12-19T18:21:17.496+0000 7f1b33646480 -1 init, newargv = 0x55e4185b67d0 newargc=15 2023-12-19T18:21:17.496 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[106284]: starting ceph client 2023-12-19T18:21:17.503 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[106284]: starting fuse 2023-12-19T18:21:17.510 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:21:17.511 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-19T18:21:17.511 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:21:17.511 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:21:17.511 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:21:17.511 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:21:17.511 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:21:17.511 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:21:17.511 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:21:17.511 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:21:17.511 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:21:17.511 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:21:17.512 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:21:17.513 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:21:17.513 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:21:17.717 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.106284.asok 2023-12-19T18:21:17.895 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.106284.asok 2023-12-19T18:21:17.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:17.896 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.106284.asok status 2023-12-19T18:21:18.165 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:21:18.165 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:21:18.165 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:21:18.165 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:21:18.165 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:21:18.165 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "106284", 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 7029, 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 7029 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:18.166 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3499361445 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3499361445 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.7029 192.168.0.1:0/3499361445", 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3499361445", 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 46, 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 117, 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:21:18.167 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:21:18.168 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:21:18.168 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:21:18.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:18.182 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-12-19T18:21:18.182 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-19T18:21:18.240 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-12-19T18:21:18.241 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-19T18:21:18.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:18.241 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:21:18.348 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:21:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.575+0000 7f2334a62640 1 Processor -- start 2023-12-19T18:21:18.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.575+0000 7f2334a62640 1 -- start start 2023-12-19T18:21:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.576+0000 7f2334a62640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23300b4ef0 0x7f23300b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.576+0000 7f2334a62640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f23300d2350 0x7f23300b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:18.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.576+0000 7f2334a62640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23300b5e90 con 0x7f23301604d0 2023-12-19T18:21:18.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.576+0000 7f2334a62640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23300d2b30 con 0x7f23300b4ef0 2023-12-19T18:21:18.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.576+0000 7f2334a62640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23300d2850 con 0x7f23300d2350 2023-12-19T18:21:18.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.577+0000 7f232effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23300b4ef0 0x7f23300b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:18.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.577+0000 7f232effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23300b4ef0 0x7f23300b6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35330/0 (socket says 172.21.15.27:35330) 2023-12-19T18:21:18.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.577+0000 7f232effd640 1 -- 172.21.15.27:0/2289244890 learned_addr learned my addr 172.21.15.27:0/2289244890 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:18.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.577+0000 7f232effd640 1 -- 172.21.15.27:0/2289244890 >> v2:172.21.15.192:3300/0 conn(0x7f23300d2350 msgr2=0x7f23300b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:18.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.577+0000 7f232ffff640 1 --2- 172.21.15.27:0/2289244890 >> v2:172.21.15.192:3300/0 conn(0x7f23300d2350 0x7f23300b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:18.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.577+0000 7f232effd640 1 --2- 172.21.15.27:0/2289244890 >> v2:172.21.15.192:3300/0 conn(0x7f23300d2350 0x7f23300b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:18.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.577+0000 7f232effd640 1 -- 172.21.15.27:0/2289244890 >> v1:172.21.15.192:6789/0 conn(0x7f23301604d0 legacy=0x7f23300b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:18.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.577+0000 7f232effd640 1 -- 172.21.15.27:0/2289244890 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23300cded0 con 0x7f23300b4ef0 2023-12-19T18:21:18.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.577+0000 7f232ffff640 1 --2- 172.21.15.27:0/2289244890 >> v2:172.21.15.192:3300/0 conn(0x7f23300d2350 0x7f23300b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:18.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.578+0000 7f232effd640 1 --2- 172.21.15.27:0/2289244890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23300b4ef0 0x7f23300b6300 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2330071e80 tx=0x7f231802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb9678a03974fd2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:18.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.579+0000 7f232e7fc640 1 -- 172.21.15.27:0/2289244890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318031870 con 0x7f23300b4ef0 2023-12-19T18:21:18.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.579+0000 7f232e7fc640 1 -- 172.21.15.27:0/2289244890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23180319d0 con 0x7f23300b4ef0 2023-12-19T18:21:18.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.579+0000 7f232e7fc640 1 -- 172.21.15.27:0/2289244890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318031d60 con 0x7f23300b4ef0 2023-12-19T18:21:18.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.579+0000 7f2334a62640 1 -- 172.21.15.27:0/2289244890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23300b4ef0 msgr2=0x7f23300b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:18.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.579+0000 7f2334a62640 1 --2- 172.21.15.27:0/2289244890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23300b4ef0 0x7f23300b6300 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f2330071e80 tx=0x7f231802b910 comp rx=0 tx=0).stop 2023-12-19T18:21:18.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.579+0000 7f2334a62640 1 -- 172.21.15.27:0/2289244890 shutdown_connections 2023-12-19T18:21:18.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.579+0000 7f2334a62640 1 --2- 172.21.15.27:0/2289244890 >> v2:172.21.15.192:3300/0 conn(0x7f23300d2350 0x7f23300b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:18.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.579+0000 7f2334a62640 1 --2- 172.21.15.27:0/2289244890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23300b4ef0 0x7f23300b6300 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:18.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.579+0000 7f2334a62640 1 -- 172.21.15.27:0/2289244890 >> 172.21.15.27:0/2289244890 conn(0x7f23300cf7e0 msgr2=0x7f23300cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:18.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.579+0000 7f2334a62640 1 -- 172.21.15.27:0/2289244890 shutdown_connections 2023-12-19T18:21:18.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.579+0000 7f2334a62640 1 -- 172.21.15.27:0/2289244890 wait complete. 2023-12-19T18:21:18.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.581+0000 7f2334a62640 1 Processor -- start 2023-12-19T18:21:18.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.581+0000 7f2334a62640 1 -- start start 2023-12-19T18:21:18.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.582+0000 7f2334a62640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23300b4ef0 0x7f233015a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:18.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.582+0000 7f2334a62640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23300d2350 0x7f23301f2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:18.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.582+0000 7f2334a62640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23301604d0 0x7f23300c8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:18.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.582+0000 7f2334a62640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23300ca480 con 0x7f23300d2350 2023-12-19T18:21:18.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.582+0000 7f2334a62640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23300c7b00 con 0x7f23300b4ef0 2023-12-19T18:21:18.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.582+0000 7f2334a62640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23300c7de0 con 0x7f23301604d0 2023-12-19T18:21:18.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.583+0000 7f232effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23300d2350 0x7f23301f2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:18.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.583+0000 7f232effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23300d2350 0x7f23301f2040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35334/0 (socket says 172.21.15.27:35334) 2023-12-19T18:21:18.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.583+0000 7f232effd640 1 -- 172.21.15.27:0/339627688 learned_addr learned my addr 172.21.15.27:0/339627688 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.583+0000 7f232f7fe640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23300b4ef0 0x7f233015a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.583+0000 7f232ffff640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23301604d0 0x7f23300c8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.583+0000 7f232effd640 1 -- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23301604d0 msgr2=0x7f23300c8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.583+0000 7f232effd640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23301604d0 0x7f23300c8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.583+0000 7f232effd640 1 -- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23300b4ef0 msgr2=0x7f233015a200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.583+0000 7f232effd640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23300b4ef0 0x7f233015a200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.583+0000 7f232effd640 1 -- 172.21.15.27:0/339627688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23180094d0 con 0x7f23300d2350 2023-12-19T18:21:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.583+0000 7f232ffff640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23301604d0 0x7f23300c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.584+0000 7f232effd640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23300d2350 0x7f23301f2040 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2318000c00 tx=0x7f23180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.584+0000 7f2327fff640 1 -- 172.21.15.27:0/339627688 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2318031840 con 0x7f23300d2350 2023-12-19T18:21:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.584+0000 7f2327fff640 1 -- 172.21.15.27:0/339627688 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23180443f0 con 0x7f23300d2350 2023-12-19T18:21:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.584+0000 7f2327fff640 1 -- 172.21.15.27:0/339627688 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23180447a0 con 0x7f23300d2350 2023-12-19T18:21:18.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.584+0000 7f2334a62640 1 -- 172.21.15.27:0/339627688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23300c9e30 con 0x7f23300d2350 2023-12-19T18:21:18.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.584+0000 7f2334a62640 1 -- 172.21.15.27:0/339627688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23301f2580 con 0x7f23300d2350 2023-12-19T18:21:18.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.585+0000 7f2334a62640 1 -- 172.21.15.27:0/339627688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2330066f80 con 0x7f23300d2350 2023-12-19T18:21:18.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.586+0000 7f2327fff640 1 -- 172.21.15.27:0/339627688 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f231804d450 con 0x7f23300d2350 2023-12-19T18:21:18.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.587+0000 7f2327fff640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f231006a430 0x7f231006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:18.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.587+0000 7f2327fff640 1 -- 172.21.15.27:0/339627688 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f231802d040 con 0x7f23300d2350 2023-12-19T18:21:18.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.588+0000 7f232f7fe640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f231006a430 0x7f231006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:18.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.589+0000 7f232f7fe640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f231006a430 0x7f231006c8f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2320009f60 tx=0x7f23200094f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:18.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.592+0000 7f2327fff640 1 -- 172.21.15.27:0/339627688 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f23180766f0 con 0x7f23300d2350 2023-12-19T18:21:18.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.825+0000 7f2334a62640 1 -- 172.21.15.27:0/339627688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f233004e840 con 0x7f23300d2350 2023-12-19T18:21:18.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.826+0000 7f2327fff640 1 -- 172.21.15.27:0/339627688 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2318079d10 con 0x7f23300d2350 2023-12-19T18:21:18.826 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:21:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.829+0000 7f2334a62640 1 -- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f231006a430 msgr2=0x7f231006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.829+0000 7f2334a62640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f231006a430 0x7f231006c8f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2320009f60 tx=0x7f23200094f0 comp rx=0 tx=0).stop 2023-12-19T18:21:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.829+0000 7f2334a62640 1 -- 172.21.15.27:0/339627688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23300d2350 msgr2=0x7f23301f2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.829+0000 7f2334a62640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23300d2350 0x7f23301f2040 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f2318000c00 tx=0x7f23180046c0 comp rx=0 tx=0).stop 2023-12-19T18:21:18.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.829+0000 7f2334a62640 1 -- 172.21.15.27:0/339627688 shutdown_connections 2023-12-19T18:21:18.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.829+0000 7f2334a62640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f231006a430 0x7f231006c8f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:18.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.829+0000 7f2334a62640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23301604d0 0x7f23300c8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:18.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.829+0000 7f2334a62640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23300d2350 0x7f23301f2040 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:18.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.829+0000 7f2334a62640 1 --2- 172.21.15.27:0/339627688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23300b4ef0 0x7f233015a200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:18.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.829+0000 7f2334a62640 1 -- 172.21.15.27:0/339627688 >> 172.21.15.27:0/339627688 conn(0x7f23300cf7e0 msgr2=0x7f23300b8310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:18.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.829+0000 7f2334a62640 1 -- 172.21.15.27:0/339627688 shutdown_connections 2023-12-19T18:21:18.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:18.829+0000 7f2334a62640 1 -- 172.21.15.27:0/339627688 wait complete. 2023-12-19T18:21:18.847 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:21:18.847 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:21:18.847 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-19T18:21:18.848 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-19T18:21:18.848 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T18:21:18.848 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:21:18.848 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:21:18.848 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:21:18.848 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:21:18.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:18.848 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-19T18:21:18.909 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:21:18.910 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 41924sec preferred_lft 41924sec 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::b892:1cff:fedc:4bed/64 scope link 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether a6:e1:fc:e6:56:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::a4e1:fcff:fee6:56fa/64 scope link 2023-12-19T18:21:18.911 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:18.912 INFO:teuthology.orchestra.run.smithi192.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:21:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T18:21:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::404f:6cff:fe73:9aa2/64 scope link 2023-12-19T18:21:18.912 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:21:18.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:18.913 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:21:19.004 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-19T18:21:19.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:19.008 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-19T18:21:19.063 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-19T18:21:19.063 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:21:19.063 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T18:21:19.063 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T18:21:19.064 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:21:19.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:19.064 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:21:19.064 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:21:19.064 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T18:21:19.064 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T18:21:19.064 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T18:21:19.064 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:21:19.243 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:19.243 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:21:19.294 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T18:21:19.294 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-19T18:21:19.295 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:21:19.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:19.295 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-19T18:21:19.385 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-19T18:21:19.385 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:21:19.385 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-19T18:21:19.438 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:21:19.439 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:21:19.439 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:21:19.491 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:21:19.589 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:21:19.590 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:21:19.684 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:21:19.687 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:21:19.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:19.688 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:21:19.744 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:21:19.744 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-19T18:21:19.750 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:21:19.828 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:21:19.829 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:21:19.898 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:21:19.901 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:21:19.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:19.902 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:21:19.941 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-19T18:21:19.941+0000 7f567ce3b480 -1 init, newargv = 0x556ffceb8660 newargc=15 2023-12-19T18:21:19.942 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[89274]: starting ceph client 2023-12-19T18:21:19.951 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[89274]: starting fuse 2023-12-19T18:21:21.047 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:21:21.151 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:21:21.152 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:21:21.245 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:21:21.248 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:21:21.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:21.249 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:21:21.303 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:21:21.303 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-12-19T18:21:21.303 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:21:21.303 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-19T18:21:21.303 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-19T18:21:21.303 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-19T18:21:21.303 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-19T18:21:21.303 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:21:21.304 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-19T18:21:21.305 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-19T18:21:21.305 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-19T18:21:21.305 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:21:21.305 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:21:21.305 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-19T18:21:21.305 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-19T18:21:21.479 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.89274.asok 2023-12-19T18:21:21.662 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.89274.asok 2023-12-19T18:21:21.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:21.662 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.89274.asok status 2023-12-19T18:21:21.902 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-19T18:21:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-19T18:21:21.902 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:21:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:21:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-19T18:21:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-19T18:21:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-19T18:21:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "89274", 2023-12-19T18:21:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-19T18:21:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:21:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-19T18:21:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-19T18:21:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5432, 2023-12-19T18:21:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-19T18:21:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-19T18:21:21.903 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-19T18:21:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5432 2023-12-19T18:21:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:21:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:21:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:21:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:21:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2034382093 2023-12-19T18:21:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-19T18:21:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:21:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:21:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:21:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:21:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2034382093 2023-12-19T18:21:21.904 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:21:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5432 192.168.0.1:0/2034382093", 2023-12-19T18:21:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/2034382093", 2023-12-19T18:21:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-19T18:21:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 46, 2023-12-19T18:21:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 117, 2023-12-19T18:21:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:21:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-19T18:21:21.905 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-19T18:21:21.905 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-19T18:21:21.917 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:21.917 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-19T18:21:21.917 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-19T18:21:21.970 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-19T18:21:21.970 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-19T18:21:21.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:21.970 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:21:22.063 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:21:22.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.307+0000 7f4d1b59e640 1 Processor -- start 2023-12-19T18:21:22.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.307+0000 7f4d1b59e640 1 -- start start 2023-12-19T18:21:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.308+0000 7f4d1b59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4d1c1553a0 0x7f4d1c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:22.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.309+0000 7f4d1b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15ee30 0x7f4d1c0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.309+0000 7f4d1b59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c0b0360 con 0x7f4d1c0afeb0 2023-12-19T18:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.309+0000 7f4d1b59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c0cbcf0 con 0x7f4d1c1553a0 2023-12-19T18:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.309+0000 7f4d1b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0cb350 con 0x7f4d1c15ee30 2023-12-19T18:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.309+0000 7f4d19d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4d1c0afeb0 0x7f4d1c15e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44734/0 (socket says 172.21.15.27:44734) 2023-12-19T18:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.309+0000 7f4d19d9b640 1 -- 172.21.15.27:0/2649906000 learned_addr learned my addr 172.21.15.27:0/2649906000 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:22.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.309+0000 7f4d1a59c640 1 --2- 172.21.15.27:0/2649906000 >> v2:172.21.15.192:3300/0 conn(0x7f4d1c1553a0 0x7f4d1c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:22.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.309+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1944964490 0 0) 0x7f4d1c0b0360 con 0x7f4d1c0afeb0 2023-12-19T18:21:22.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.309+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d04003520 con 0x7f4d1c0afeb0 2023-12-19T18:21:22.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.309+0000 7f4d1ad9d640 1 --2- 172.21.15.27:0/2649906000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15ee30 0x7f4d1c0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:22.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2502448099 0 0) 0x7f4d04003520 con 0x7f4d1c0afeb0 2023-12-19T18:21:22.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15ee30 msgr2=0x7f4d1c0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:22.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 --2- 172.21.15.27:0/2649906000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15ee30 0x7f4d1c0c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 >> v2:172.21.15.192:3300/0 conn(0x7f4d1c1553a0 msgr2=0x7f4d1c0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 --2- 172.21.15.27:0/2649906000 >> v2:172.21.15.192:3300/0 conn(0x7f4d1c1553a0 0x7f4d1c0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c0cb630 con 0x7f4d1c0afeb0 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f4d10001670 con 0x7f4d1c0afeb0 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 >> v1:172.21.15.27:6789/0 conn(0x7f4d1c0afeb0 legacy=0x7f4d1c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 --2- 172.21.15.27:0/2649906000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15ee30 0x7f4d04003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 --2- 172.21.15.27:0/2649906000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d04003f40 0x7f4d040047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 --2- 172.21.15.27:0/2649906000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04004d00 0x7f4d04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0cbcf0 con 0x7f4d04003f40 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0cb350 con 0x7f4d04004d00 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0400bae0 con 0x7f4d1c15ee30 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.310+0000 7f4d19d9b640 1 --2- 172.21.15.27:0/2649906000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d04003f40 0x7f4d040047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.311+0000 7f4d1ad9d640 1 --2- 172.21.15.27:0/2649906000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04004d00 0x7f4d04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.311+0000 7f4d1a59c640 1 --2- 172.21.15.27:0/2649906000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15ee30 0x7f4d04003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.311+0000 7f4d19d9b640 1 -- 172.21.15.27:0/2649906000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15ee30 msgr2=0x7f4d04003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.311+0000 7f4d19d9b640 1 --2- 172.21.15.27:0/2649906000 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15ee30 0x7f4d04003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.311+0000 7f4d19d9b640 1 -- 172.21.15.27:0/2649906000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04004d00 msgr2=0x7f4d04005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.311+0000 7f4d19d9b640 1 --2- 172.21.15.27:0/2649906000 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04004d00 0x7f4d04005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.311+0000 7f4d19d9b640 1 -- 172.21.15.27:0/2649906000 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d04003520 con 0x7f4d04003f40 2023-12-19T18:21:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.312+0000 7f4d19d9b640 1 --2- 172.21.15.27:0/2649906000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d04003f40 0x7f4d040047c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000c110 tx=0x7f4d040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7218a74498ba2494 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.312+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d10005ad0 con 0x7f4d04003f40 2023-12-19T18:21:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.312+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d10005c50 con 0x7f4d04003f40 2023-12-19T18:21:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.312+0000 7f4d1959a640 1 -- 172.21.15.27:0/2649906000 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1002c830 con 0x7f4d04003f40 2023-12-19T18:21:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.313+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2649906000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d04003f40 msgr2=0x7f4d040047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.313+0000 7f4d1b59e640 1 --2- 172.21.15.27:0/2649906000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d04003f40 0x7f4d040047c0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f4d1000c110 tx=0x7f4d040057f0 comp rx=0 tx=0).stop 2023-12-19T18:21:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.313+0000 7f4d1ad9d640 1 -- 172.21.15.27:0/2649906000 reap_dead start 2023-12-19T18:21:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.313+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2649906000 shutdown_connections 2023-12-19T18:21:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.313+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2649906000 >> 172.21.15.27:0/2649906000 conn(0x7f4d1c069c70 msgr2=0x7f4d1c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:22.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.313+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2649906000 shutdown_connections 2023-12-19T18:21:22.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.313+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2649906000 wait complete. 2023-12-19T18:21:22.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.315+0000 7f4d1b59e640 1 Processor -- start 2023-12-19T18:21:22.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.315+0000 7f4d1b59e640 1 -- start start 2023-12-19T18:21:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.316+0000 7f4d1b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04003f40 0x7f4d1c1f0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:22.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.316+0000 7f4d1b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d1c154700 0x7f4d1c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.316+0000 7f4d1b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15db40 0x7f4d1c155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.316+0000 7f4d1b59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0b3580 con 0x7f4d1c154700 2023-12-19T18:21:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.316+0000 7f4d1b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0702a0 con 0x7f4d04003f40 2023-12-19T18:21:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.316+0000 7f4d1b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c070540 con 0x7f4d1c15db40 2023-12-19T18:21:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.316+0000 7f4d19d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d1c154700 0x7f4d1c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.317+0000 7f4d19d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d1c154700 0x7f4d1c06fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44796/0 (socket says 172.21.15.27:44796) 2023-12-19T18:21:22.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.317+0000 7f4d19d9b640 1 -- 172.21.15.27:0/2212816823 learned_addr learned my addr 172.21.15.27:0/2212816823 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:22.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.317+0000 7f4d1a59c640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04003f40 0x7f4d1c1f0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:22.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.317+0000 7f4d1ad9d640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15db40 0x7f4d1c155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:22.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.317+0000 7f4d19d9b640 1 -- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15db40 msgr2=0x7f4d1c155a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:22.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.317+0000 7f4d19d9b640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15db40 0x7f4d1c155a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.317+0000 7f4d19d9b640 1 -- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04003f40 msgr2=0x7f4d1c1f0000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.317+0000 7f4d19d9b640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04003f40 0x7f4d1c1f0000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.317+0000 7f4d19d9b640 1 -- 172.21.15.27:0/2212816823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c15cd80 con 0x7f4d1c154700 2023-12-19T18:21:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.317+0000 7f4d1ad9d640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15db40 0x7f4d1c155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.317+0000 7f4d1a59c640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04003f40 0x7f4d1c1f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.317+0000 7f4d19d9b640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d1c154700 0x7f4d1c06fd60 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4d1002c3d0 tx=0x7f4d10001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.318+0000 7f4d0affd640 1 -- 172.21.15.27:0/2212816823 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1002cc40 con 0x7f4d1c154700 2023-12-19T18:21:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.318+0000 7f4d0affd640 1 -- 172.21.15.27:0/2212816823 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1002cda0 con 0x7f4d1c154700 2023-12-19T18:21:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.318+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2212816823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c15d060 con 0x7f4d1c154700 2023-12-19T18:21:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.318+0000 7f4d0affd640 1 -- 172.21.15.27:0/2212816823 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d100072b0 con 0x7f4d1c154700 2023-12-19T18:21:22.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.318+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2212816823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c15d620 con 0x7f4d1c154700 2023-12-19T18:21:22.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.319+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2212816823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c04e840 con 0x7f4d1c154700 2023-12-19T18:21:22.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.320+0000 7f4d0affd640 1 -- 172.21.15.27:0/2212816823 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4d10032070 con 0x7f4d1c154700 2023-12-19T18:21:22.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.321+0000 7f4d0affd640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4cfc06a430 0x7f4cfc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.322+0000 7f4d0affd640 1 -- 172.21.15.27:0/2212816823 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4d1003c040 con 0x7f4d1c154700 2023-12-19T18:21:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.322+0000 7f4d1a59c640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4cfc06a430 0x7f4cfc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:22.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.323+0000 7f4d1a59c640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4cfc06a430 0x7f4cfc06c8f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009a80 tx=0x7f4d0c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:22.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.326+0000 7f4d0affd640 1 -- 172.21.15.27:0/2212816823 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4d10075360 con 0x7f4d1c154700 2023-12-19T18:21:22.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.556+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2212816823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d1c0af6a0 con 0x7f4d1c154700 2023-12-19T18:21:22.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.557+0000 7f4d0affd640 1 -- 172.21.15.27:0/2212816823 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f4d10078980 con 0x7f4d1c154700 2023-12-19T18:21:22.558 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:22.558 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2779436922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2779436922},{"type":"v1","addr":"172.21.15.192:6836","nonce":2779436922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3121062779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3121062779},{"type":"v1","addr":"172.21.15.192:6837","nonce":3121062779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5357,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2032553869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2032553869},{"type":"v1","addr":"172.21.15.27:6835","nonce":2032553869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:21:08.281615+0000","modified":"2023-12-19T18:21:11.800655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/252970177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":252970177},{"type":"v1","addr":"172.21.15.27:6834","nonce":252970177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T18:21:22.558 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 46 2023-12-19T18:21:22.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.560+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4cfc06a430 msgr2=0x7f4cfc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:22.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.560+0000 7f4d1b59e640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4cfc06a430 0x7f4cfc06c8f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c009a80 tx=0x7f4d0c008040 comp rx=0 tx=0).stop 2023-12-19T18:21:22.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.560+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2212816823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d1c154700 msgr2=0x7f4d1c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:22.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.560+0000 7f4d1b59e640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d1c154700 0x7f4d1c06fd60 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4d1002c3d0 tx=0x7f4d10001930 comp rx=0 tx=0).stop 2023-12-19T18:21:22.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.561+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2212816823 shutdown_connections 2023-12-19T18:21:22.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.561+0000 7f4d1b59e640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4cfc06a430 0x7f4cfc06c8f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:22.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.561+0000 7f4d1b59e640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d1c15db40 0x7f4d1c155a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:22.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.561+0000 7f4d1b59e640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d1c154700 0x7f4d1c06fd60 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:22.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.561+0000 7f4d1b59e640 1 --2- 172.21.15.27:0/2212816823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d04003f40 0x7f4d1c1f0000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:22.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.561+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2212816823 >> 172.21.15.27:0/2212816823 conn(0x7f4d1c069c70 msgr2=0x7f4d1c0ca760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:22.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.561+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2212816823 shutdown_connections 2023-12-19T18:21:22.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:22.561+0000 7f4d1b59e640 1 -- 172.21.15.27:0/2212816823 wait complete. 2023-12-19T18:21:22.582 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-12-19T18:21:22.832 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":6885,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.529386348999999,"mdsmap_epoch":46,"osdmap_epoch":117,"osdmap_epoch_barrier":117,"uptime":20.104936949999999} 2023-12-19T18:21:22.832 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 6885, "mdsmap_epoch": 46, "osdmap_epoch": 117, "osdmap_epoch_barrier": 117, "rank_uptime": 14.529386349, "state": "up:active", "uptime": 20.10493695, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:21:22.833 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-12-19T18:21:23.083 INFO:teuthology.orchestra.run.smithi027.stdout:{"osd_mon_report_interval":"5"} 2023-12-19T18:21:23.084 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-12-19T18:21:23.084 INFO:tasks.cephfs.test_full:Writing 531MB should fill this cluster 2023-12-19T18:21:23.084 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:21:23.084 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:21:23.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:23.139 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=265 seek=0) 2023-12-19T18:21:29.045 INFO:teuthology.orchestra.run.smithi027.stderr:265+0 records in 2023-12-19T18:21:29.045 INFO:teuthology.orchestra.run.smithi027.stderr:265+0 records out 2023-12-19T18:21:29.046 INFO:teuthology.orchestra.run.smithi027.stderr:277872640 bytes (278 MB, 265 MiB) copied, 5.84758 s, 47.5 MB/s 2023-12-19T18:21:29.047 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:21:29.047 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:21:29.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:29.108 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=292 seek=0) 2023-12-19T18:21:32.700 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:21:32.700+0000 7f8f3ae61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 full osd(s) (OSD_FULL) 2023-12-19T18:21:32.749 INFO:teuthology.orchestra.run.smithi027.stderr:dd: error writing 'large_file_b': No space left on device 2023-12-19T18:21:32.749 INFO:teuthology.orchestra.run.smithi027.stderr:dd: closing output file 'large_file_b': No space left on device 2023-12-19T18:21:32.752 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:21:32.752 INFO:tasks.cephfs.test_full:Writing file B failed (full status happened already) 2023-12-19T18:21:32.753 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:21:32.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.986+0000 7fdd7cd3d640 1 Processor -- start 2023-12-19T18:21:32.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.986+0000 7fdd7cd3d640 1 -- start start 2023-12-19T18:21:32.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.988+0000 7fdd7cd3d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd7815d290 0x7fdd780cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:32.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.988+0000 7fdd7cd3d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdd780cd6d0 0x7fdd780ce2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:32.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.988+0000 7fdd7cd3d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7815d670 con 0x7fdd780d02c0 2023-12-19T18:21:32.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.988+0000 7fdd7cd3d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd780cdc40 con 0x7fdd780cd6d0 2023-12-19T18:21:32.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.988+0000 7fdd7cd3d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780d2400 con 0x7fdd7815d290 2023-12-19T18:21:32.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.988+0000 7fdd76ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdd780cd6d0 0x7fdd780ce2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:32.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.988+0000 7fdd76ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdd780cd6d0 0x7fdd780ce2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55774/0 (socket says 172.21.15.27:55774) 2023-12-19T18:21:32.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.988+0000 7fdd76ffd640 1 -- 172.21.15.27:0/1228854802 learned_addr learned my addr 172.21.15.27:0/1228854802 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:32.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.988+0000 7fdd777fe640 1 --2- 172.21.15.27:0/1228854802 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd7815d290 0x7fdd780cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:32.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.989+0000 7fdd76ffd640 1 -- 172.21.15.27:0/1228854802 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd7815d290 msgr2=0x7fdd780cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:32.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.989+0000 7fdd76ffd640 1 --2- 172.21.15.27:0/1228854802 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd7815d290 0x7fdd780cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:32.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.989+0000 7fdd76ffd640 1 -- 172.21.15.27:0/1228854802 >> v1:172.21.15.27:6789/0 conn(0x7fdd780d02c0 legacy=0x7fdd780d0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:32.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.989+0000 7fdd76ffd640 1 -- 172.21.15.27:0/1228854802 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd780d1ad0 con 0x7fdd780cd6d0 2023-12-19T18:21:32.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.990+0000 7fdd76ffd640 1 --2- 172.21.15.27:0/1228854802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd780cd6d0 0x7fdd780ce2f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009870 tx=0x7fdd6002b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13e7440118e13ae server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:32.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.990+0000 7fdd767fc640 1 -- 172.21.15.27:0/1228854802 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60030840 con 0x7fdd780cd6d0 2023-12-19T18:21:32.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.990+0000 7fdd767fc640 1 -- 172.21.15.27:0/1228854802 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd600309a0 con 0x7fdd780cd6d0 2023-12-19T18:21:32.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.990+0000 7fdd767fc640 1 -- 172.21.15.27:0/1228854802 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60030d30 con 0x7fdd780cd6d0 2023-12-19T18:21:32.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.991+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/1228854802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd780cd6d0 msgr2=0x7fdd780ce2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:32.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.991+0000 7fdd7cd3d640 1 --2- 172.21.15.27:0/1228854802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd780cd6d0 0x7fdd780ce2f0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7fdd60009870 tx=0x7fdd6002b980 comp rx=0 tx=0).stop 2023-12-19T18:21:32.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.991+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/1228854802 shutdown_connections 2023-12-19T18:21:32.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.991+0000 7fdd7cd3d640 1 --2- 172.21.15.27:0/1228854802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd780cd6d0 0x7fdd780ce2f0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:32.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.991+0000 7fdd7cd3d640 1 --2- 172.21.15.27:0/1228854802 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd7815d290 0x7fdd780cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:32.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.991+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/1228854802 >> 172.21.15.27:0/1228854802 conn(0x7fdd780cf7d0 msgr2=0x7fdd780cfbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:32.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.991+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/1228854802 shutdown_connections 2023-12-19T18:21:32.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.991+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/1228854802 wait complete. 2023-12-19T18:21:32.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.993+0000 7fdd7cd3d640 1 Processor -- start 2023-12-19T18:21:32.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.993+0000 7fdd7cd3d640 1 -- start start 2023-12-19T18:21:32.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.994+0000 7fdd7cd3d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd780cd6d0 0x7fdd781f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:32.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.994+0000 7fdd7cd3d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd780d02c0 0x7fdd780c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:32.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.994+0000 7fdd7cd3d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd7815d290 0x7fdd780c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:32.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.995+0000 7fdd7cd3d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780c8310 con 0x7fdd7815d290 2023-12-19T18:21:32.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.995+0000 7fdd7cd3d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd780c8030 con 0x7fdd780cd6d0 2023-12-19T18:21:32.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.995+0000 7fdd7cd3d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7815d9b0 con 0x7fdd780d02c0 2023-12-19T18:21:32.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.995+0000 7fdd77fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd7815d290 0x7fdd780c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:32.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.995+0000 7fdd77fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd7815d290 0x7fdd780c8c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55782/0 (socket says 172.21.15.27:55782) 2023-12-19T18:21:32.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.995+0000 7fdd77fff640 1 -- 172.21.15.27:0/3963427557 learned_addr learned my addr 172.21.15.27:0/3963427557 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:32.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.995+0000 7fdd76ffd640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd780d02c0 0x7fdd780c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:32.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.995+0000 7fdd777fe640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd780cd6d0 0x7fdd781f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:32.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.995+0000 7fdd77fff640 1 -- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd780d02c0 msgr2=0x7fdd780c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:32.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.995+0000 7fdd77fff640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd780d02c0 0x7fdd780c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:32.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.995+0000 7fdd77fff640 1 -- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd780cd6d0 msgr2=0x7fdd781f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:32.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.995+0000 7fdd77fff640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd780cd6d0 0x7fdd781f23c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:32.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.995+0000 7fdd77fff640 1 -- 172.21.15.27:0/3963427557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd600094d0 con 0x7fdd7815d290 2023-12-19T18:21:32.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.996+0000 7fdd77fff640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd7815d290 0x7fdd780c8c10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c390 tx=0x7fdd7000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:32.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.996+0000 7fdd6ffff640 1 -- 172.21.15.27:0/3963427557 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd700052d0 con 0x7fdd7815d290 2023-12-19T18:21:32.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.996+0000 7fdd6ffff640 1 -- 172.21.15.27:0/3963427557 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70005430 con 0x7fdd7815d290 2023-12-19T18:21:32.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.996+0000 7fdd6ffff640 1 -- 172.21.15.27:0/3963427557 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70003810 con 0x7fdd7815d290 2023-12-19T18:21:32.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.996+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/3963427557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7815dc90 con 0x7fdd7815d290 2023-12-19T18:21:32.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.996+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/3963427557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd780c7300 con 0x7fdd7815d290 2023-12-19T18:21:32.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.998+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/3963427557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7815d810 con 0x7fdd7815d290 2023-12-19T18:21:32.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.999+0000 7fdd6ffff640 1 -- 172.21.15.27:0/3963427557 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdd70003970 con 0x7fdd7815d290 2023-12-19T18:21:32.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:32.999+0000 7fdd6ffff640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd5806a430 0x7fdd5806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:32.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.000+0000 7fdd6ffff640 1 -- 172.21.15.27:0/3963427557 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd70087ca0 con 0x7fdd7815d290 2023-12-19T18:21:33.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.000+0000 7fdd6ffff640 1 -- 172.21.15.27:0/3963427557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=119}) v3 -- 0x7fdd58071f60 con 0x7fdd7815d290 2023-12-19T18:21:33.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.000+0000 7fdd777fe640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd5806a430 0x7fdd5806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:33.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.001+0000 7fdd777fe640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd5806a430 0x7fdd5806c8f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdd680099c0 tx=0x7fdd68003990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:33.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.004+0000 7fdd6ffff640 1 -- 172.21.15.27:0/3963427557 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdd70017060 con 0x7fdd7815d290 2023-12-19T18:21:33.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.197+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/3963427557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fdd7815e1b0 con 0x7fdd7815d290 2023-12-19T18:21:33.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.200+0000 7fdd6ffff640 1 -- 172.21.15.27:0/3963427557 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v118) v1 ==== 81+0+33622 (secure 0 0 0) 0x7fdd7815e1b0 con 0x7fdd7815d290 2023-12-19T18:21:33.200 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:33.200 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:21:33.200 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 118, 2023-12-19T18:21:33.200 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:21:32.700894+0000", 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:21:33.201 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 13, 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:21:33.202 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:21:33.203 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "118", 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:33.204 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:33.205 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:33.206 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:33.206 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:33.207 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:33.207 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:33.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:33.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.207 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:33.207 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:33.207 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:33.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:33.207 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:33.207 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:21:33.208 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 12, 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:21:06.258281+0000", 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:33.209 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:33.210 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "118", 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:33.211 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:33.212 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:33.213 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.75, 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.75, 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.75, 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.75, 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:33.214 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 13, 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:21:07.308887+0000", 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:33.215 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:33.216 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "118", 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:33.217 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:33.218 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.219 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:33.219 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:33.219 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:33.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:33.219 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:33.219 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:33.219 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:33.219 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:33.219 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:33.219 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:33.219 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:33.219 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:21:33.220 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.221 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.222 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.224 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:21:33.225 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.226 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.227 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.228 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.229 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:21:33.230 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.231 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.232 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.233 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.234 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:21:33.235 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.236 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:21:33.237 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.238 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.240 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.241 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.242 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.243 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.244 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:21:33.245 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.246 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.248 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.249 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:21:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:21:33.250 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.251 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.252 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.253 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.254 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.255 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:21:33.256 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.257 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.258 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.259 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.260 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:21:33.261 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull", 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.262 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.265 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:21:33.267 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:21:33.268 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:21:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:21:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:21:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:21:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:21:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:21:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:21:33.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.269 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:21:33.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.202+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd5806a430 msgr2=0x7fdd5806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:33.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.202+0000 7fdd7cd3d640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd5806a430 0x7fdd5806c8f0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fdd680099c0 tx=0x7fdd68003990 comp rx=0 tx=0).stop 2023-12-19T18:21:33.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.202+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/3963427557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd7815d290 msgr2=0x7fdd780c8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:33.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.202+0000 7fdd7cd3d640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd7815d290 0x7fdd780c8c10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fdd7000c390 tx=0x7fdd7000c850 comp rx=0 tx=0).stop 2023-12-19T18:21:33.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.202+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/3963427557 shutdown_connections 2023-12-19T18:21:33.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.202+0000 7fdd7cd3d640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdd5806a430 0x7fdd5806c8f0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:33.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.202+0000 7fdd7cd3d640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd7815d290 0x7fdd780c8c10 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:33.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.202+0000 7fdd7cd3d640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdd780d02c0 0x7fdd780c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:33.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.202+0000 7fdd7cd3d640 1 --2- 172.21.15.27:0/3963427557 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdd780cd6d0 0x7fdd781f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:33.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.202+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/3963427557 >> 172.21.15.27:0/3963427557 conn(0x7fdd780cf7d0 msgr2=0x7fdd780b81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:33.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.203+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/3963427557 shutdown_connections 2023-12-19T18:21:33.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.203+0000 7fdd7cd3d640 1 -- 172.21.15.27:0/3963427557 wait complete. 2023-12-19T18:21:33.274 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:21:33.274 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:21:33.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:33.339 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=265) 2023-12-19T18:21:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:dd: error writing 'large_file_b': No space left on device 2023-12-19T18:21:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2023-12-19T18:21:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:0+0 records out 2023-12-19T18:21:33.474 INFO:teuthology.orchestra.run.smithi027.stderr:0 bytes copied, 0.00648366 s, 0.0 kB/s 2023-12-19T18:21:33.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:21:33.475 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:21:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.709+0000 7f6c6f4d7640 1 Processor -- start 2023-12-19T18:21:33.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.709+0000 7f6c6f4d7640 1 -- start start 2023-12-19T18:21:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.711+0000 7f6c6f4d7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6c68070fb0 0x7f6c6806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.711+0000 7f6c6f4d7640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6c6806b850 0x7f6c6806f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.711+0000 7f6c6f4d7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c680adf10 con 0x7f6c6815ac50 2023-12-19T18:21:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.711+0000 7f6c6f4d7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c680adac0 con 0x7f6c68070fb0 2023-12-19T18:21:33.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.711+0000 7f6c6f4d7640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c680af630 con 0x7f6c6806b850 2023-12-19T18:21:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.711+0000 7f6c6e4d5640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6c6815ac50 0x7f6c680aef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40992/0 (socket says 172.21.15.27:40992) 2023-12-19T18:21:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.711+0000 7f6c6e4d5640 1 -- 172.21.15.27:0/1308632635 learned_addr learned my addr 172.21.15.27:0/1308632635 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.711+0000 7f6c6dcd4640 1 --2- 172.21.15.27:0/1308632635 >> v2:172.21.15.27:3300/0 conn(0x7f6c68070fb0 0x7f6c6806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.712+0000 7f6c6ecd6640 1 --2- 172.21.15.27:0/1308632635 >> v2:172.21.15.192:3300/0 conn(0x7f6c6806b850 0x7f6c6806f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:33.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.712+0000 7f6c6dcd4640 1 -- 172.21.15.27:0/1308632635 >> v2:172.21.15.192:3300/0 conn(0x7f6c6806b850 msgr2=0x7f6c6806f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.712+0000 7f6c6dcd4640 1 --2- 172.21.15.27:0/1308632635 >> v2:172.21.15.192:3300/0 conn(0x7f6c6806b850 0x7f6c6806f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.712+0000 7f6c6dcd4640 1 -- 172.21.15.27:0/1308632635 >> v1:172.21.15.27:6789/0 conn(0x7f6c6815ac50 legacy=0x7f6c680aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.712+0000 7f6c6dcd4640 1 -- 172.21.15.27:0/1308632635 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c680713f0 con 0x7f6c68070fb0 2023-12-19T18:21:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.712+0000 7f6c6ecd6640 1 --2- 172.21.15.27:0/1308632635 >> v2:172.21.15.192:3300/0 conn(0x7f6c6806b850 0x7f6c6806f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:21:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.713+0000 7f6c6dcd4640 1 --2- 172.21.15.27:0/1308632635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c68070fb0 0x7f6c6806b310 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f6c680cf840 tx=0x7f6c5802b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b474c31ee99c0a41 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:33.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.713+0000 7f6c6d4d3640 1 -- 172.21.15.27:0/1308632635 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58031520 con 0x7f6c68070fb0 2023-12-19T18:21:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.713+0000 7f6c6d4d3640 1 -- 172.21.15.27:0/1308632635 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c58031680 con 0x7f6c68070fb0 2023-12-19T18:21:33.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.713+0000 7f6c6d4d3640 1 -- 172.21.15.27:0/1308632635 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c58031a10 con 0x7f6c68070fb0 2023-12-19T18:21:33.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.714+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/1308632635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c68070fb0 msgr2=0x7f6c6806b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:33.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.714+0000 7f6c6f4d7640 1 --2- 172.21.15.27:0/1308632635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c68070fb0 0x7f6c6806b310 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f6c680cf840 tx=0x7f6c5802b7c0 comp rx=0 tx=0).stop 2023-12-19T18:21:33.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.714+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/1308632635 shutdown_connections 2023-12-19T18:21:33.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.714+0000 7f6c6f4d7640 1 --2- 172.21.15.27:0/1308632635 >> v2:172.21.15.192:3300/0 conn(0x7f6c6806b850 0x7f6c6806f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:33.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.714+0000 7f6c6f4d7640 1 --2- 172.21.15.27:0/1308632635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c68070fb0 0x7f6c6806b310 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:33.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.714+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/1308632635 >> 172.21.15.27:0/1308632635 conn(0x7f6c6806a7d0 msgr2=0x7f6c680ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:33.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.714+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/1308632635 shutdown_connections 2023-12-19T18:21:33.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.714+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/1308632635 wait complete. 2023-12-19T18:21:33.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.716+0000 7f6c6f4d7640 1 Processor -- start 2023-12-19T18:21:33.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.717+0000 7f6c6f4d7640 1 -- start start 2023-12-19T18:21:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.718+0000 7f6c6f4d7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c6806b850 0x7f6c681471a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.718+0000 7f6c6f4d7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c68070fb0 0x7f6c680c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.718+0000 7f6c6f4d7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6815ac50 0x7f6c680c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.718+0000 7f6c6f4d7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680c9fb0 con 0x7f6c6815ac50 2023-12-19T18:21:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.718+0000 7f6c6f4d7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680ca290 con 0x7f6c6806b850 2023-12-19T18:21:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.718+0000 7f6c6f4d7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c680ca570 con 0x7f6c68070fb0 2023-12-19T18:21:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.718+0000 7f6c6ecd6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6815ac50 0x7f6c680c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:33.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.718+0000 7f6c6ecd6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6815ac50 0x7f6c680c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55798/0 (socket says 172.21.15.27:55798) 2023-12-19T18:21:33.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.718+0000 7f6c6ecd6640 1 -- 172.21.15.27:0/56014340 learned_addr learned my addr 172.21.15.27:0/56014340 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:33.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.718+0000 7f6c6e4d5640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c6806b850 0x7f6c681471a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:33.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.718+0000 7f6c6dcd4640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c68070fb0 0x7f6c680c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.719+0000 7f6c6ecd6640 1 -- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c68070fb0 msgr2=0x7f6c680c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.719+0000 7f6c6ecd6640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c68070fb0 0x7f6c680c7bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.719+0000 7f6c6ecd6640 1 -- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c6806b850 msgr2=0x7f6c681471a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.719+0000 7f6c6ecd6640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c6806b850 0x7f6c681471a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.719+0000 7f6c6ecd6640 1 -- 172.21.15.27:0/56014340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c58009400 con 0x7f6c6815ac50 2023-12-19T18:21:33.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.719+0000 7f6c6ecd6640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6815ac50 0x7f6c680c8100 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6c60007e70 tx=0x7f6c6000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.719+0000 7f6c5effd640 1 -- 172.21.15.27:0/56014340 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c60003b90 con 0x7f6c6815ac50 2023-12-19T18:21:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.719+0000 7f6c5effd640 1 -- 172.21.15.27:0/56014340 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6000f040 con 0x7f6c6815ac50 2023-12-19T18:21:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.719+0000 7f6c5effd640 1 -- 172.21.15.27:0/56014340 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6000de80 con 0x7f6c6815ac50 2023-12-19T18:21:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.719+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/56014340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c680ca850 con 0x7f6c6815ac50 2023-12-19T18:21:33.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.719+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/56014340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c681476e0 con 0x7f6c6815ac50 2023-12-19T18:21:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.721+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/56014340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c680cad70 con 0x7f6c6815ac50 2023-12-19T18:21:33.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.722+0000 7f6c5effd640 1 -- 172.21.15.27:0/56014340 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6c60004030 con 0x7f6c6815ac50 2023-12-19T18:21:33.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.723+0000 7f6c5effd640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c4c06a430 0x7f6c4c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:33.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.723+0000 7f6c5effd640 1 -- 172.21.15.27:0/56014340 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c60088c00 con 0x7f6c6815ac50 2023-12-19T18:21:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.724+0000 7f6c5effd640 1 -- 172.21.15.27:0/56014340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=119}) v3 -- 0x7f6c4c071f60 con 0x7f6c6815ac50 2023-12-19T18:21:33.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.724+0000 7f6c6e4d5640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c4c06a430 0x7f6c4c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:33.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.725+0000 7f6c6e4d5640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c4c06a430 0x7f6c4c06c8f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6c64005880 tx=0x7f6c64009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.728+0000 7f6c5effd640 1 -- 172.21.15.27:0/56014340 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6c6001a060 con 0x7f6c6815ac50 2023-12-19T18:21:33.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.925+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/56014340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f6c68147870 con 0x7f6c6815ac50 2023-12-19T18:21:33.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.928+0000 7f6c5effd640 1 -- 172.21.15.27:0/56014340 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v118) v1 ==== 81+0+33622 (secure 0 0 0) 0x7f6c680cad70 con 0x7f6c6815ac50 2023-12-19T18:21:33.929 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:33.929 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:21:33.929 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 118, 2023-12-19T18:21:33.929 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:21:33.929 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:21:33.929 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:21:32.700894+0000", 2023-12-19T18:21:33.929 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:21:33.930 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 13, 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:33.931 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:21:33.932 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:33.933 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:33.933 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:33.933 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:33.933 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:33.933 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:33.933 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:33.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.933 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "118", 2023-12-19T18:21:33.933 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:33.933 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:33.933 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:33.933 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:33.933 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:33.934 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:33.935 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:21:33.936 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 12, 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:21:06.258281+0000", 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:21:33.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:21:33.938 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "118", 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:33.939 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:33.940 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:33.941 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.75, 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.75, 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:21:33.942 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.75, 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.75, 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 13, 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:21:07.308887+0000", 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:33.943 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:33.944 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "118", 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:33.945 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:33.946 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:33.947 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:21:33.948 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.949 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:21:33.951 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:21:33.952 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:33.953 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.954 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:21:33.955 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.959 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.960 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.962 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:21:33.962 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.962 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:21:33.962 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.962 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.962 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.962 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.962 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:21:33.964 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.965 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:21:33.966 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.967 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.969 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.969 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.969 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:21:33.969 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.969 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:21:33.969 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:21:33.970 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.971 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.976 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.977 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.978 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:21:33.979 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.980 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.981 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.982 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.983 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.984 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:21:33.985 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:33.986 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:21:33.987 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.988 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.989 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.990 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull", 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.991 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.992 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.993 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.994 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:21:33.995 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:33.996 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:21:33.997 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:21:33.998 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:21:33.999 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:33.999 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:21:34.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.931+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c4c06a430 msgr2=0x7f6c4c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:34.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.931+0000 7f6c6f4d7640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c4c06a430 0x7f6c4c06c8f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f6c64005880 tx=0x7f6c64009040 comp rx=0 tx=0).stop 2023-12-19T18:21:34.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.931+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/56014340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6815ac50 msgr2=0x7f6c680c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:34.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.931+0000 7f6c6f4d7640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6815ac50 0x7f6c680c8100 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6c60007e70 tx=0x7f6c6000d4a0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.931+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/56014340 shutdown_connections 2023-12-19T18:21:34.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.931+0000 7f6c6f4d7640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c4c06a430 0x7f6c4c06c8f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.931+0000 7f6c6f4d7640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6815ac50 0x7f6c680c8100 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.931+0000 7f6c6f4d7640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c68070fb0 0x7f6c680c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.931+0000 7f6c6f4d7640 1 --2- 172.21.15.27:0/56014340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c6806b850 0x7f6c681471a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.931+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/56014340 >> 172.21.15.27:0/56014340 conn(0x7f6c6806a7d0 msgr2=0x7f6c680cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:34.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.932+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/56014340 shutdown_connections 2023-12-19T18:21:34.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:33.932+0000 7f6c6f4d7640 1 -- 172.21.15.27:0/56014340 wait complete. 2023-12-19T18:21:34.002 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:21:34.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.242+0000 7f143eb3c640 1 Processor -- start 2023-12-19T18:21:34.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.243+0000 7f143eb3c640 1 -- start start 2023-12-19T18:21:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.244+0000 7f143eb3c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1438164a80 0x7f14380ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.244+0000 7f143eb3c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14380cbc60 0x7f14380cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.244+0000 7f143eb3c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14380caba0 con 0x7f14380c9190 2023-12-19T18:21:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.244+0000 7f143eb3c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1438164e60 con 0x7f1438164a80 2023-12-19T18:21:34.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.244+0000 7f143eb3c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143806c2d0 con 0x7f14380cbc60 2023-12-19T18:21:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.245+0000 7f142ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f14380c9190 0x7f14380cb550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41000/0 (socket says 172.21.15.27:41000) 2023-12-19T18:21:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.245+0000 7f142ffff640 1 -- 172.21.15.27:0/481010962 learned_addr learned my addr 172.21.15.27:0/481010962 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.245+0000 7f143c8d8640 1 --2- 172.21.15.27:0/481010962 >> v2:172.21.15.192:3300/0 conn(0x7f1438164a80 0x7f14380ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.245+0000 7f143d0d9640 1 --2- 172.21.15.27:0/481010962 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14380cbc60 0x7f14380cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.245+0000 7f142f7fe640 1 -- 172.21.15.27:0/481010962 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354238337 0 0) 0x7f14380caba0 con 0x7f14380c9190 2023-12-19T18:21:34.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.245+0000 7f142f7fe640 1 -- 172.21.15.27:0/481010962 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1420003520 con 0x7f14380c9190 2023-12-19T18:21:34.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.246+0000 7f143c8d8640 1 -- 172.21.15.27:0/481010962 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14380cbc60 msgr2=0x7f14380cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:34.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.246+0000 7f142f7fe640 1 -- 172.21.15.27:0/481010962 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2452973533 0 0) 0x7f1420003520 con 0x7f14380c9190 2023-12-19T18:21:34.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.246+0000 7f143c8d8640 1 --2- 172.21.15.27:0/481010962 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14380cbc60 0x7f14380cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.246+0000 7f143c8d8640 1 -- 172.21.15.27:0/481010962 >> v1:172.21.15.27:6789/0 conn(0x7f14380c9190 legacy=0x7f14380cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:34.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.246+0000 7f143c8d8640 1 -- 172.21.15.27:0/481010962 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14380cae40 con 0x7f1438164a80 2023-12-19T18:21:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.247+0000 7f143c8d8640 1 --2- 172.21.15.27:0/481010962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1438164a80 0x7f14380ce520 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f14340097a0 tx=0x7f143402b710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e443663278cb1e60 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.248+0000 7f142f7fe640 1 -- 172.21.15.27:0/481010962 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428001670 con 0x7f1438164a80 2023-12-19T18:21:34.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.248+0000 7f142f7fe640 1 -- 172.21.15.27:0/481010962 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143402e050 con 0x7f1438164a80 2023-12-19T18:21:34.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.248+0000 7f142f7fe640 1 -- 172.21.15.27:0/481010962 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1434030710 con 0x7f1438164a80 2023-12-19T18:21:34.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.248+0000 7f143eb3c640 1 -- 172.21.15.27:0/481010962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1438164a80 msgr2=0x7f14380ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:34.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.248+0000 7f143eb3c640 1 --2- 172.21.15.27:0/481010962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1438164a80 0x7f14380ce520 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f14340097a0 tx=0x7f143402b710 comp rx=0 tx=0).stop 2023-12-19T18:21:34.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.248+0000 7f143eb3c640 1 -- 172.21.15.27:0/481010962 shutdown_connections 2023-12-19T18:21:34.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.248+0000 7f143eb3c640 1 --2- 172.21.15.27:0/481010962 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14380cbc60 0x7f14380cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.248+0000 7f143eb3c640 1 --2- 172.21.15.27:0/481010962 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1438164a80 0x7f14380ce520 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.248+0000 7f143eb3c640 1 -- 172.21.15.27:0/481010962 >> 172.21.15.27:0/481010962 conn(0x7f143806a330 msgr2=0x7f14380b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:34.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.248+0000 7f143eb3c640 1 -- 172.21.15.27:0/481010962 shutdown_connections 2023-12-19T18:21:34.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.248+0000 7f143eb3c640 1 -- 172.21.15.27:0/481010962 wait complete. 2023-12-19T18:21:34.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.250+0000 7f143eb3c640 1 Processor -- start 2023-12-19T18:21:34.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.250+0000 7f143eb3c640 1 -- start start 2023-12-19T18:21:34.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.251+0000 7f143eb3c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14380c9190 0x7f14381f6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:34.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.251+0000 7f143eb3c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14380cbc60 0x7f14380c2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:34.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.251+0000 7f143eb3c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1438164a80 0x7f14380c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:34.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.251+0000 7f143eb3c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14380c4e90 con 0x7f14380cbc60 2023-12-19T18:21:34.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.251+0000 7f143eb3c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14380c4bb0 con 0x7f14380c9190 2023-12-19T18:21:34.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.251+0000 7f143eb3c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14380c5d90 con 0x7f1438164a80 2023-12-19T18:21:34.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.252+0000 7f142ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14380cbc60 0x7f14380c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:34.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.252+0000 7f142ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14380cbc60 0x7f14380c2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55808/0 (socket says 172.21.15.27:55808) 2023-12-19T18:21:34.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.252+0000 7f142ffff640 1 -- 172.21.15.27:0/736325709 learned_addr learned my addr 172.21.15.27:0/736325709 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:34.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.252+0000 7f143c8d8640 1 --2- 172.21.15.27:0/736325709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14380c9190 0x7f14381f6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:34.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.252+0000 7f142ffff640 1 -- 172.21.15.27:0/736325709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1438164a80 msgr2=0x7f14380c2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:34.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.252+0000 7f142ffff640 1 --2- 172.21.15.27:0/736325709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1438164a80 0x7f14380c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.252+0000 7f142ffff640 1 -- 172.21.15.27:0/736325709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14380c9190 msgr2=0x7f14381f6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:34.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.252+0000 7f142ffff640 1 --2- 172.21.15.27:0/736325709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14380c9190 0x7f14381f6380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.252+0000 7f142ffff640 1 -- 172.21.15.27:0/736325709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1434009400 con 0x7f14380cbc60 2023-12-19T18:21:34.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.253+0000 7f142ffff640 1 --2- 172.21.15.27:0/736325709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14380cbc60 0x7f14380c2670 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f142800c750 tx=0x7f142800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:34.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.253+0000 7f142d7fa640 1 -- 172.21.15.27:0/736325709 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428005ad0 con 0x7f14380cbc60 2023-12-19T18:21:34.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.253+0000 7f142d7fa640 1 -- 172.21.15.27:0/736325709 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1428005c30 con 0x7f14380cbc60 2023-12-19T18:21:34.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.253+0000 7f142d7fa640 1 -- 172.21.15.27:0/736325709 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428007070 con 0x7f14380cbc60 2023-12-19T18:21:34.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.253+0000 7f143eb3c640 1 -- 172.21.15.27:0/736325709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14380c6070 con 0x7f14380cbc60 2023-12-19T18:21:34.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.253+0000 7f143eb3c640 1 -- 172.21.15.27:0/736325709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14380c47f0 con 0x7f14380cbc60 2023-12-19T18:21:34.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.254+0000 7f143eb3c640 1 -- 172.21.15.27:0/736325709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14380c5bf0 con 0x7f14380cbc60 2023-12-19T18:21:34.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.255+0000 7f142d7fa640 1 -- 172.21.15.27:0/736325709 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f14280071d0 con 0x7f14380cbc60 2023-12-19T18:21:34.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.256+0000 7f142d7fa640 1 --2- 172.21.15.27:0/736325709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f142406a430 0x7f142406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:34.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.256+0000 7f142d7fa640 1 -- 172.21.15.27:0/736325709 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1428087ff0 con 0x7f14380cbc60 2023-12-19T18:21:34.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.256+0000 7f142d7fa640 1 -- 172.21.15.27:0/736325709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=119}) v3 -- 0x7f1424071f60 con 0x7f14380cbc60 2023-12-19T18:21:34.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.257+0000 7f143c8d8640 1 --2- 172.21.15.27:0/736325709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f142406a430 0x7f142406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:34.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.258+0000 7f143c8d8640 1 --2- 172.21.15.27:0/736325709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f142406a430 0x7f142406c8f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1434034000 tx=0x7f1434008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:34.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.261+0000 7f142d7fa640 1 -- 172.21.15.27:0/736325709 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f14280baca0 con 0x7f14380cbc60 2023-12-19T18:21:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.497+0000 7f143eb3c640 1 -- 172.21.15.27:0/736325709 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14380c6590 con 0x7f14380cbc60 2023-12-19T18:21:34.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.499+0000 7f142d7fa640 1 -- 172.21.15.27:0/736325709 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f14380c6590 con 0x7f14380cbc60 2023-12-19T18:21:34.501 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:34.501 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2779436922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2779436922},{"type":"v1","addr":"172.21.15.192:6836","nonce":2779436922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3121062779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3121062779},{"type":"v1","addr":"172.21.15.192:6837","nonce":3121062779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5357,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2032553869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2032553869},{"type":"v1","addr":"172.21.15.27:6835","nonce":2032553869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:21:08.281615+0000","modified":"2023-12-19T18:21:11.800655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/252970177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":252970177},{"type":"v1","addr":"172.21.15.27:6834","nonce":252970177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T18:21:34.501 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 46 2023-12-19T18:21:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.502+0000 7f143eb3c640 1 -- 172.21.15.27:0/736325709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f142406a430 msgr2=0x7f142406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.502+0000 7f143eb3c640 1 --2- 172.21.15.27:0/736325709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f142406a430 0x7f142406c8f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1434034000 tx=0x7f1434008030 comp rx=0 tx=0).stop 2023-12-19T18:21:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.502+0000 7f143eb3c640 1 -- 172.21.15.27:0/736325709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14380cbc60 msgr2=0x7f14380c2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.502+0000 7f143eb3c640 1 --2- 172.21.15.27:0/736325709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14380cbc60 0x7f14380c2670 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f142800c750 tx=0x7f142800cc10 comp rx=0 tx=0).stop 2023-12-19T18:21:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.502+0000 7f143eb3c640 1 -- 172.21.15.27:0/736325709 shutdown_connections 2023-12-19T18:21:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.502+0000 7f143eb3c640 1 --2- 172.21.15.27:0/736325709 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f142406a430 0x7f142406c8f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.502+0000 7f143eb3c640 1 --2- 172.21.15.27:0/736325709 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1438164a80 0x7f14380c2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.502+0000 7f143eb3c640 1 --2- 172.21.15.27:0/736325709 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14380cbc60 0x7f14380c2670 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.502+0000 7f143eb3c640 1 --2- 172.21.15.27:0/736325709 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14380c9190 0x7f14381f6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.502+0000 7f143eb3c640 1 -- 172.21.15.27:0/736325709 >> 172.21.15.27:0/736325709 conn(0x7f143806a330 msgr2=0x7f14380b3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.502+0000 7f143eb3c640 1 -- 172.21.15.27:0/736325709 shutdown_connections 2023-12-19T18:21:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:34.502+0000 7f143eb3c640 1 -- 172.21.15.27:0/736325709 wait complete. 2023-12-19T18:21:34.522 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-12-19T18:21:34.775 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":6885,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":26.469308795,"mdsmap_epoch":46,"osdmap_epoch":118,"osdmap_epoch_barrier":118,"uptime":32.044869536999997} 2023-12-19T18:21:34.776 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 6885, "mdsmap_epoch": 46, "osdmap_epoch": 118, "osdmap_epoch_barrier": 118, "rank_uptime": 26.469308795, "state": "up:active", "uptime": 32.044869537, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:21:34.776 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-12-19T18:21:34.776 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:21:34.776 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:21:34.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:34.827 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=small_file_1 bs=1M conv=fdatasync count=0 seek=0) 2023-12-19T18:21:34.877 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-12-19T18:21:34.877+0000 7f1b13fff640 -1 client.7029 handle_client_reply no pending request on tid 12 safe is:1 2023-12-19T18:21:34.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:21:34.879 INFO:teuthology.orchestra.run.smithi027.stderr:dd: failed to open 'small_file_1': Permission denied 2023-12-19T18:21:34.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:34.879 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-12-19T18:21:34.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:34.937 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-12-19T18:21:35.051 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:21:35.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.285+0000 7fbcd3173640 1 Processor -- start 2023-12-19T18:21:35.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.285+0000 7fbcd3173640 1 -- start start 2023-12-19T18:21:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.286+0000 7fbcd3173640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbccc06ab00 0x7fbccc0d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.286+0000 7fbcd3173640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbccc0d1a50 con 0x7fbccc0d2af0 2023-12-19T18:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.287+0000 7fbcd3173640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbccc0d1d30 con 0x7fbccc15d260 2023-12-19T18:21:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.287+0000 7fbcd3173640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0cdaf0 con 0x7fbccc06ab00 2023-12-19T18:21:35.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.287+0000 7fbcd1710640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbccc0d2af0 0x7fbccc0b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41014/0 (socket says 172.21.15.27:41014) 2023-12-19T18:21:35.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.287+0000 7fbcd1710640 1 -- 172.21.15.27:0/1742911096 learned_addr learned my addr 172.21.15.27:0/1742911096 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:35.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.287+0000 7fbccbfff640 1 --2- 172.21.15.27:0/1742911096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbccc06ab00 0x7fbccc0d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 834311825 0 0) 0x7fbccc0d1a50 con 0x7fbccc0d2af0 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcb8003520 con 0x7fbccc0d2af0 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3875806887 0 0) 0x7fbcb8003520 con 0x7fbccc0d2af0 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbccc06ab00 msgr2=0x7fbccc0d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 --2- 172.21.15.27:0/1742911096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbccc06ab00 0x7fbccc0d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 >> v1:172.21.15.192:6789/0 conn(0x7fbccc15d260 legacy=0x7fbccc06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc0d2010 con 0x7fbccc0d2af0 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fbcc4001670 con 0x7fbccc0d2af0 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 >> v1:172.21.15.27:6789/0 conn(0x7fbccc0d2af0 legacy=0x7fbccc0b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 --2- 172.21.15.27:0/1742911096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbccc06ab00 0x7fbcb80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 --2- 172.21.15.27:0/1742911096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb8003f70 0x7fbcb80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 --2- 172.21.15.27:0/1742911096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb8004d00 0x7fbcb8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0cdaf0 con 0x7fbcb8004d00 2023-12-19T18:21:35.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb800bac0 con 0x7fbcb8003f70 2023-12-19T18:21:35.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb800bc80 con 0x7fbccc06ab00 2023-12-19T18:21:35.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.288+0000 7fbcd1710640 1 --2- 172.21.15.27:0/1742911096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb8004d00 0x7fbcb8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:35.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.289+0000 7fbccbfff640 1 --2- 172.21.15.27:0/1742911096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb8003f70 0x7fbcb80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:35.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.289+0000 7fbcd1710640 1 -- 172.21.15.27:0/1742911096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbccc06ab00 msgr2=0x7fbcb80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:35.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.289+0000 7fbcd1710640 1 --2- 172.21.15.27:0/1742911096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbccc06ab00 0x7fbcb80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:35.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.289+0000 7fbcd1710640 1 -- 172.21.15.27:0/1742911096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb8003f70 msgr2=0x7fbcb80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:35.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.289+0000 7fbcd1710640 1 --2- 172.21.15.27:0/1742911096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb8003f70 0x7fbcb80047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:35.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.289+0000 7fbcd1710640 1 -- 172.21.15.27:0/1742911096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb8003520 con 0x7fbcb8004d00 2023-12-19T18:21:35.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.290+0000 7fbcd1710640 1 --2- 172.21.15.27:0/1742911096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb8004d00 0x7fbcb8005180 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0d2f40 tx=0x7fbcb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=728f891a90a7eeea server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:35.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.290+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4005ad0 con 0x7fbcb8004d00 2023-12-19T18:21:35.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.290+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc4005c30 con 0x7fbcb8004d00 2023-12-19T18:21:35.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.290+0000 7fbccb7fe640 1 -- 172.21.15.27:0/1742911096 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc402ca00 con 0x7fbcb8004d00 2023-12-19T18:21:35.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.291+0000 7fbcd3173640 1 -- 172.21.15.27:0/1742911096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb8004d00 msgr2=0x7fbcb8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:35.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.291+0000 7fbcd3173640 1 --2- 172.21.15.27:0/1742911096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb8004d00 0x7fbcb8005180 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0d2f40 tx=0x7fbcb80057f0 comp rx=0 tx=0).stop 2023-12-19T18:21:35.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.291+0000 7fbcd1710640 1 -- 172.21.15.27:0/1742911096 reap_dead start 2023-12-19T18:21:35.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.291+0000 7fbcd3173640 1 -- 172.21.15.27:0/1742911096 shutdown_connections 2023-12-19T18:21:35.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.291+0000 7fbcd3173640 1 -- 172.21.15.27:0/1742911096 >> 172.21.15.27:0/1742911096 conn(0x7fbccc0d2f70 msgr2=0x7fbccc0d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:35.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.291+0000 7fbcd3173640 1 -- 172.21.15.27:0/1742911096 shutdown_connections 2023-12-19T18:21:35.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.291+0000 7fbcd3173640 1 -- 172.21.15.27:0/1742911096 wait complete. 2023-12-19T18:21:35.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.293+0000 7fbcd3173640 1 Processor -- start 2023-12-19T18:21:35.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.293+0000 7fbcd3173640 1 -- start start 2023-12-19T18:21:35.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.294+0000 7fbcd3173640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb8004d00 0x7fbccc0b54b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:35.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.294+0000 7fbcd3173640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbccc1f21e0 0x7fbccc14d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.294+0000 7fbcd3173640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbccc15fda0 0x7fbccc160220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.294+0000 7fbcd3173640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc1f2660 con 0x7fbccc1f21e0 2023-12-19T18:21:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.294+0000 7fbcd3173640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc160760 con 0x7fbcb8004d00 2023-12-19T18:21:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.294+0000 7fbcd3173640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0b3830 con 0x7fbccc15fda0 2023-12-19T18:21:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.295+0000 7fbccbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbccc1f21e0 0x7fbccc14d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.295+0000 7fbccbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbccc1f21e0 0x7fbccc14d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55822/0 (socket says 172.21.15.27:55822) 2023-12-19T18:21:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.295+0000 7fbccbfff640 1 -- 172.21.15.27:0/3638535817 learned_addr learned my addr 172.21.15.27:0/3638535817 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:35.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.295+0000 7fbcd0f0f640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb8004d00 0x7fbccc0b54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.295+0000 7fbcd1710640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbccc15fda0 0x7fbccc160220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.295+0000 7fbccbfff640 1 -- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbccc15fda0 msgr2=0x7fbccc160220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.295+0000 7fbccbfff640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbccc15fda0 0x7fbccc160220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.295+0000 7fbccbfff640 1 -- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb8004d00 msgr2=0x7fbccc0b54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.295+0000 7fbccbfff640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb8004d00 0x7fbccc0b54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.295+0000 7fbccbfff640 1 -- 172.21.15.27:0/3638535817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc0b3ad0 con 0x7fbccc1f21e0 2023-12-19T18:21:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.295+0000 7fbcd0f0f640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb8004d00 0x7fbccc0b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.295+0000 7fbccbfff640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbccc1f21e0 0x7fbccc14d3f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4010890 tx=0x7fbcb4010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.296+0000 7fbcc97fa640 1 -- 172.21.15.27:0/3638535817 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb400eb60 con 0x7fbccc1f21e0 2023-12-19T18:21:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.296+0000 7fbcc97fa640 1 -- 172.21.15.27:0/3638535817 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb400ecc0 con 0x7fbccc1f21e0 2023-12-19T18:21:35.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.296+0000 7fbcc97fa640 1 -- 172.21.15.27:0/3638535817 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb401c400 con 0x7fbccc1f21e0 2023-12-19T18:21:35.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.296+0000 7fbcd3173640 1 -- 172.21.15.27:0/3638535817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbccc14d930 con 0x7fbccc1f21e0 2023-12-19T18:21:35.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.296+0000 7fbcd3173640 1 -- 172.21.15.27:0/3638535817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc0b4050 con 0x7fbccc1f21e0 2023-12-19T18:21:35.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.297+0000 7fbcd3173640 1 -- 172.21.15.27:0/3638535817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbccc066810 con 0x7fbccc1f21e0 2023-12-19T18:21:35.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.298+0000 7fbcc97fa640 1 -- 172.21.15.27:0/3638535817 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbcb401c560 con 0x7fbccc1f21e0 2023-12-19T18:21:35.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.299+0000 7fbcc97fa640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbcac06a430 0x7fbcac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:35.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.299+0000 7fbcc97fa640 1 -- 172.21.15.27:0/3638535817 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbcb4088e20 con 0x7fbccc1f21e0 2023-12-19T18:21:35.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.299+0000 7fbcc97fa640 1 -- 172.21.15.27:0/3638535817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=120}) v3 -- 0x7fbcac071f60 con 0x7fbccc1f21e0 2023-12-19T18:21:35.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.300+0000 7fbcd0f0f640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbcac06a430 0x7fbcac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:35.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.301+0000 7fbcd0f0f640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbcac06a430 0x7fbcac06c8f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc007fd0 tx=0x7fbcbc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:35.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.303+0000 7fbcc97fa640 1 -- 172.21.15.27:0/3638535817 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbcb40544c0 con 0x7fbccc1f21e0 2023-12-19T18:21:35.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.500+0000 7fbcd3173640 1 -- 172.21.15.27:0/3638535817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fbccc068e90 con 0x7fbccc1f21e0 2023-12-19T18:21:35.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.502+0000 7fbcc97fa640 1 -- 172.21.15.27:0/3638535817 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v119) v1 ==== 81+0+33646 (secure 0 0 0) 0x7fbcb4021020 con 0x7fbccc1f21e0 2023-12-19T18:21:35.503 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:35.503 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:21:35.503 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 119, 2023-12-19T18:21:35.503 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:21:35.503 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:21:35.503 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:21:34.758151+0000", 2023-12-19T18:21:35.503 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:21:35.503 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 13, 2023-12-19T18:21:35.504 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:21:35.505 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:21:35.505 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:21:35.505 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:21:35.505 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:21:35.505 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:21:35.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.505 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:21:35.505 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:21:35.505 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:21:35.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:21:35.505 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:21:35.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:35.505 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:35.506 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "118", 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:35.507 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:35.508 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:35.509 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:35.510 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 12, 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:21:06.258281+0000", 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:35.511 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.512 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "118", 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:35.513 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:35.514 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:35.515 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.75, 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.75, 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.75, 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.75, 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.516 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 13, 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:21:07.308887+0000", 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:35.517 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "118", 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:35.518 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:35.519 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:35.520 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.521 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.522 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:21:35.523 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:35.523 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:35.524 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:35.524 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:35.524 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:35.524 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:35.524 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:35.524 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:35.525 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.527 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:21:35.528 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:35.529 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:21:35.530 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:35.531 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.532 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.532 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:35.532 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.532 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:21:35.532 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:35.532 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.532 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.532 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.532 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:21:35.533 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:35.533 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.533 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.533 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:35.533 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.533 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:21:35.533 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:35.533 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.533 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.533 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.535 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.535 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:21:35.535 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:35.535 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:35.535 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:35.535 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:35.535 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:35.535 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:35.536 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:35.536 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:35.536 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:35.536 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:35.536 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:35.536 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:35.536 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.537 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:21:35.537 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:35.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.537 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.537 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.537 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:21:35.537 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:35.537 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.537 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.538 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:35.538 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.538 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:21:35.538 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:35.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.538 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.538 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.539 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:35.540 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:35.541 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:35.542 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:35.542 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:35.542 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:35.542 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:35.542 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:35.542 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:35.542 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:35.542 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:35.542 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:35.542 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:35.543 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.543 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:21:35.543 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:35.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.543 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.543 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:21:35.544 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:35.545 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.546 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:35.547 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.548 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.549 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:21:35.550 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.551 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:35.552 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:35.553 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:35.553 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:35.553 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:35.553 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:35.553 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:35.553 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:35.553 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:35.553 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:35.553 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.554 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.554 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:21:35.554 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:35.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.554 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.554 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:21:35.554 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:35.554 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.554 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.554 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:35.554 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:21:35.557 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:35.558 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.559 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:35.563 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:21:35.564 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull", 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:21:35.574 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:21:35.575 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:21:35.576 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:35.576 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:21:35.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.505+0000 7fbcd3173640 1 -- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbcac06a430 msgr2=0x7fbcac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:35.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.505+0000 7fbcd3173640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbcac06a430 0x7fbcac06c8f0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc007fd0 tx=0x7fbcbc009040 comp rx=0 tx=0).stop 2023-12-19T18:21:35.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.505+0000 7fbcd3173640 1 -- 172.21.15.27:0/3638535817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbccc1f21e0 msgr2=0x7fbccc14d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:35.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.505+0000 7fbcd3173640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbccc1f21e0 0x7fbccc14d3f0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4010890 tx=0x7fbcb4010d50 comp rx=0 tx=0).stop 2023-12-19T18:21:35.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.505+0000 7fbcd3173640 1 -- 172.21.15.27:0/3638535817 shutdown_connections 2023-12-19T18:21:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.505+0000 7fbcd3173640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbcac06a430 0x7fbcac06c8f0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.505+0000 7fbcd3173640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbccc15fda0 0x7fbccc160220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.505+0000 7fbcd3173640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbccc1f21e0 0x7fbccc14d3f0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:35.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.505+0000 7fbcd3173640 1 --2- 172.21.15.27:0/3638535817 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcb8004d00 0x7fbccc0b54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:35.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.505+0000 7fbcd3173640 1 -- 172.21.15.27:0/3638535817 >> 172.21.15.27:0/3638535817 conn(0x7fbccc0d2f70 msgr2=0x7fbccc154540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:35.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.505+0000 7fbcd3173640 1 -- 172.21.15.27:0/3638535817 shutdown_connections 2023-12-19T18:21:35.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:35.505+0000 7fbcd3173640 1 -- 172.21.15.27:0/3638535817 wait complete. 2023-12-19T18:21:35.579 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-12-19T18:21:39.806 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:21:39.807+0000 7f8f3ae61640 -1 log_channel(cluster) log [ERR] : Health check update: 1 full osd(s) (OSD_FULL) 2023-12-19T18:21:40.580 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:21:40.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.819+0000 7f58b359e640 1 Processor -- start 2023-12-19T18:21:40.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.819+0000 7f58b359e640 1 -- start start 2023-12-19T18:21:40.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.820+0000 7f58b359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58b415ac50 0x7f58b4070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:40.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b359e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f58b40af360 0x7f58b406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:40.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58b40adf10 con 0x7f58b40aef20 2023-12-19T18:21:40.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b40adac0 con 0x7f58b415ac50 2023-12-19T18:21:40.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b359e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58b40ae160 con 0x7f58b40af360 2023-12-19T18:21:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58b415ac50 0x7f58b4070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58b415ac50 0x7f58b4070fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55836/0 (socket says 172.21.15.27:55836) 2023-12-19T18:21:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b1d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f58b40aef20 0x7f58b406b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41018/0 (socket says 172.21.15.27:41018) 2023-12-19T18:21:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b259c640 1 -- 172.21.15.27:0/3086149221 learned_addr learned my addr 172.21.15.27:0/3086149221 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b259c640 1 -- 172.21.15.27:0/3086149221 >> v2:172.21.15.192:3300/0 conn(0x7f58b40af360 msgr2=0x7f58b406ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b2d9d640 1 --2- 172.21.15.27:0/3086149221 >> v2:172.21.15.192:3300/0 conn(0x7f58b40af360 0x7f58b406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b259c640 1 --2- 172.21.15.27:0/3086149221 >> v2:172.21.15.192:3300/0 conn(0x7f58b40af360 0x7f58b406ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b159a640 1 -- 172.21.15.27:0/3086149221 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087297727 0 0) 0x7f58b40adf10 con 0x7f58b40aef20 2023-12-19T18:21:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b259c640 1 -- 172.21.15.27:0/3086149221 >> v1:172.21.15.27:6789/0 conn(0x7f58b40aef20 legacy=0x7f58b406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b259c640 1 -- 172.21.15.27:0/3086149221 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58b40ae440 con 0x7f58b415ac50 2023-12-19T18:21:40.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.821+0000 7f58b2d9d640 1 --2- 172.21.15.27:0/3086149221 >> v2:172.21.15.192:3300/0 conn(0x7f58b40af360 0x7f58b406ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.823+0000 7f58b259c640 1 --2- 172.21.15.27:0/3086149221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b415ac50 0x7f58b4070fb0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f58b40cf840 tx=0x7f58a402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edb6a4332cea9e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.823+0000 7f58b159a640 1 -- 172.21.15.27:0/3086149221 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a40317e0 con 0x7f58b415ac50 2023-12-19T18:21:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.823+0000 7f58b159a640 1 -- 172.21.15.27:0/3086149221 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58a4031940 con 0x7f58b415ac50 2023-12-19T18:21:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.823+0000 7f58b159a640 1 -- 172.21.15.27:0/3086149221 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4031cd0 con 0x7f58b415ac50 2023-12-19T18:21:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.824+0000 7f58b359e640 1 -- 172.21.15.27:0/3086149221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b415ac50 msgr2=0x7f58b4070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.824+0000 7f58b359e640 1 --2- 172.21.15.27:0/3086149221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b415ac50 0x7f58b4070fb0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f58b40cf840 tx=0x7f58a402b880 comp rx=0 tx=0).stop 2023-12-19T18:21:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.824+0000 7f58b359e640 1 -- 172.21.15.27:0/3086149221 shutdown_connections 2023-12-19T18:21:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.824+0000 7f58b359e640 1 --2- 172.21.15.27:0/3086149221 >> v2:172.21.15.192:3300/0 conn(0x7f58b40af360 0x7f58b406ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.824+0000 7f58b359e640 1 --2- 172.21.15.27:0/3086149221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b415ac50 0x7f58b4070fb0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.824+0000 7f58b359e640 1 -- 172.21.15.27:0/3086149221 >> 172.21.15.27:0/3086149221 conn(0x7f58b406a7d0 msgr2=0x7f58b40ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:40.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.824+0000 7f58b359e640 1 -- 172.21.15.27:0/3086149221 shutdown_connections 2023-12-19T18:21:40.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.824+0000 7f58b359e640 1 -- 172.21.15.27:0/3086149221 wait complete. 2023-12-19T18:21:40.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.826+0000 7f58b359e640 1 Processor -- start 2023-12-19T18:21:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.826+0000 7f58b359e640 1 -- start start 2023-12-19T18:21:40.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.827+0000 7f58b359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58b40aef20 0x7f58b40b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:40.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.827+0000 7f58b359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58b40af360 0x7f58b414c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:40.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.827+0000 7f58b359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b415ac50 0x7f58b41f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:40.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.827+0000 7f58b359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b4148ee0 con 0x7f58b415ac50 2023-12-19T18:21:40.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.827+0000 7f58b359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40b0d80 con 0x7f58b40aef20 2023-12-19T18:21:40.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.827+0000 7f58b359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58b40b3b00 con 0x7f58b40af360 2023-12-19T18:21:40.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.827+0000 7f58b2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b415ac50 0x7f58b41f39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:40.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.827+0000 7f58b2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b415ac50 0x7f58b41f39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55840/0 (socket says 172.21.15.27:55840) 2023-12-19T18:21:40.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.827+0000 7f58b2d9d640 1 -- 172.21.15.27:0/304032088 learned_addr learned my addr 172.21.15.27:0/304032088 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:40.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.828+0000 7f58b1d9b640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58b40af360 0x7f58b414c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:40.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.828+0000 7f58b259c640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58b40aef20 0x7f58b40b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:40.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.828+0000 7f58b2d9d640 1 -- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58b40af360 msgr2=0x7f58b414c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.828+0000 7f58b2d9d640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58b40af360 0x7f58b414c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.828+0000 7f58b2d9d640 1 -- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58b40aef20 msgr2=0x7f58b40b35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.828+0000 7f58b2d9d640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58b40aef20 0x7f58b40b35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.828+0000 7f58b2d9d640 1 -- 172.21.15.27:0/304032088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58a40094d0 con 0x7f58b415ac50 2023-12-19T18:21:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.828+0000 7f58b2d9d640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b415ac50 0x7f58b41f39a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00cd80 tx=0x7f58ac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.828+0000 7f58aaffd640 1 -- 172.21.15.27:0/304032088 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac003b90 con 0x7f58b415ac50 2023-12-19T18:21:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.829+0000 7f58aaffd640 1 -- 172.21.15.27:0/304032088 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac003d10 con 0x7f58b415ac50 2023-12-19T18:21:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.829+0000 7f58b359e640 1 -- 172.21.15.27:0/304032088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b40b0fd0 con 0x7f58b415ac50 2023-12-19T18:21:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.829+0000 7f58aaffd640 1 -- 172.21.15.27:0/304032088 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac004260 con 0x7f58b415ac50 2023-12-19T18:21:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.829+0000 7f58b359e640 1 -- 172.21.15.27:0/304032088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b40b1590 con 0x7f58b415ac50 2023-12-19T18:21:40.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.830+0000 7f58b359e640 1 -- 172.21.15.27:0/304032088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58b404e840 con 0x7f58b415ac50 2023-12-19T18:21:40.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.831+0000 7f58aaffd640 1 -- 172.21.15.27:0/304032088 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f58ac00b040 con 0x7f58b415ac50 2023-12-19T18:21:40.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.832+0000 7f58aaffd640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f589406a430 0x7f589406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:40.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.832+0000 7f58aaffd640 1 -- 172.21.15.27:0/304032088 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f58ac089020 con 0x7f58b415ac50 2023-12-19T18:21:40.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.832+0000 7f58aaffd640 1 -- 172.21.15.27:0/304032088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=123}) v3 -- 0x7f5894071f60 con 0x7f58b415ac50 2023-12-19T18:21:40.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.832+0000 7f58b259c640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f589406a430 0x7f589406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:40.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.834+0000 7f58b259c640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f589406a430 0x7f589406c8f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f58a4036000 tx=0x7f58a403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:40.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:40.837+0000 7f58aaffd640 1 -- 172.21.15.27:0/304032088 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f58ac01d060 con 0x7f58b415ac50 2023-12-19T18:21:41.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.031+0000 7f58b359e640 1 -- 172.21.15.27:0/304032088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f58b406f430 con 0x7f58b415ac50 2023-12-19T18:21:41.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.035+0000 7f58aaffd640 1 -- 172.21.15.27:0/304032088 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v122) v1 ==== 81+0+33590 (secure 0 0 0) 0x7f58ac054610 con 0x7f58b415ac50 2023-12-19T18:21:41.035 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:41.035 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 122, 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:21:39.807258+0000", 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:21:41.036 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:41.037 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:21:41.037 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:21:41.037 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:21:41.037 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:21:41.037 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:21:41.037 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 13, 2023-12-19T18:21:41.037 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:21:41.037 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:21:41.037 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:21:41.037 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:21:41.037 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:21:41.037 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:21:41.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.038 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:21:41.038 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:21:41.038 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:21:41.038 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:21:41.038 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:21:41.038 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:41.038 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:41.038 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:41.038 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:41.038 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:41.039 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:41.040 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:41.040 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:41.040 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:41.040 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.040 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "118", 2023-12-19T18:21:41.040 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:41.040 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:41.040 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:41.040 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:41.041 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:41.042 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:21:41.043 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 12, 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:21:06.258281+0000", 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:41.044 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:41.045 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "118", 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:41.046 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:41.047 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:41.047 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:41.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:41.047 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:41.047 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:41.047 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:41.047 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:41.047 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:41.047 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:41.047 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:41.048 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:41.048 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:41.048 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:41.048 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:41.048 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:41.048 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:41.049 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:21:41.050 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:21:41.050 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:21:41.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.050 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:41.050 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:21:41.050 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:21:41.050 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.75, 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.75, 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.75, 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.75, 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 13, 2023-12-19T18:21:41.051 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:21:07.308887+0000", 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 3, 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full", 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:21:41.052 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "118", 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:41.053 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:41.054 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:41.055 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.056 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.057 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:21:41.058 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:41.058 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:41.058 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:41.058 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:41.058 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:41.058 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:41.058 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:41.058 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:41.058 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:41.058 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.059 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.060 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:41.060 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.060 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.060 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:21:41.060 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:41.060 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.060 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.060 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.060 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:21:41.060 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:41.060 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.060 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.061 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:41.061 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.061 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.061 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:21:41.061 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:41.061 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.061 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.062 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.063 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.064 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.065 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.065 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:21:41.065 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:41.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.065 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.065 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.065 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:21:41.065 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:41.065 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.065 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.065 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:41.066 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.066 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.066 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:21:41.066 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:41.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.066 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.066 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.066 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:21:41.066 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:41.066 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.067 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.068 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:41.069 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:41.070 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.071 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.072 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.074 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.076 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.077 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.078 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:41.079 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:41.080 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.081 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.082 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.083 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:21:41.084 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:41.085 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:41.087 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:21:41.089 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:41.090 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:41.091 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.092 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: "full", 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:41.095 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:41.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.100 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:41.101 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:21:41.102 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:41.103 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:41.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:41.105 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:41.106 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:41.106 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:41.106 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:21:41.106 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:21:41.107 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:41.108 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.037+0000 7f58b359e640 1 -- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f589406a430 msgr2=0x7f589406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.037+0000 7f58b359e640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f589406a430 0x7f589406c8f0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f58a4036000 tx=0x7f58a403d040 comp rx=0 tx=0).stop 2023-12-19T18:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.037+0000 7f58b359e640 1 -- 172.21.15.27:0/304032088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b415ac50 msgr2=0x7f58b41f39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.037+0000 7f58b359e640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b415ac50 0x7f58b41f39a0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f58ac00cd80 tx=0x7f58ac00a650 comp rx=0 tx=0).stop 2023-12-19T18:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.038+0000 7f58b359e640 1 -- 172.21.15.27:0/304032088 shutdown_connections 2023-12-19T18:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.038+0000 7f58b359e640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f589406a430 0x7f589406c8f0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.038+0000 7f58b359e640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58b415ac50 0x7f58b41f39a0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.038+0000 7f58b359e640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58b40af360 0x7f58b414c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.038+0000 7f58b359e640 1 --2- 172.21.15.27:0/304032088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58b40aef20 0x7f58b40b35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.038+0000 7f58b359e640 1 -- 172.21.15.27:0/304032088 >> 172.21.15.27:0/304032088 conn(0x7f58b406a7d0 msgr2=0x7f58b4150f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.038+0000 7f58b359e640 1 -- 172.21.15.27:0/304032088 shutdown_connections 2023-12-19T18:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:41.038+0000 7f58b359e640 1 -- 172.21.15.27:0/304032088 wait complete. 2023-12-19T18:21:41.111 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-12-19T18:21:46.112 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:21:46.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.357+0000 7fe6c1c69640 1 Processor -- start 2023-12-19T18:21:46.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.358+0000 7fe6c1c69640 1 -- start start 2023-12-19T18:21:46.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.359+0000 7fe6c1c69640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe6bc15d1d0 0x7fe6bc0ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.359+0000 7fe6c1c69640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6bc071140 0x7fe6bc0cd250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.359+0000 7fe6c1c69640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6bc0ad8e0 con 0x7fe6bc0cd790 2023-12-19T18:21:46.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.359+0000 7fe6c1c69640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6bc070150 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.359+0000 7fe6c1c69640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc15d5b0 con 0x7fe6bc071140 2023-12-19T18:21:46.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.359+0000 7fe6bb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe6bc15d1d0 0x7fe6bc0ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.359+0000 7fe6bb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe6bc15d1d0 0x7fe6bc0ccd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40758/0 (socket says 172.21.15.27:40758) 2023-12-19T18:21:46.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.359+0000 7fe6bb7fe640 1 -- 172.21.15.27:0/212592264 learned_addr learned my addr 172.21.15.27:0/212592264 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:46.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.360+0000 7fe6b3fff640 1 --2- 172.21.15.27:0/212592264 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6bc071140 0x7fe6bc0cd250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.360+0000 7fe6bb7fe640 1 -- 172.21.15.27:0/212592264 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6bc071140 msgr2=0x7fe6bc0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:46.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.360+0000 7fe6bb7fe640 1 --2- 172.21.15.27:0/212592264 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6bc071140 0x7fe6bc0cd250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.360+0000 7fe6bb7fe640 1 -- 172.21.15.27:0/212592264 >> v1:172.21.15.192:6789/0 conn(0x7fe6bc0cd790 legacy=0x7fe6bc0c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:46.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.360+0000 7fe6bb7fe640 1 -- 172.21.15.27:0/212592264 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6bc0ce7c0 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.362+0000 7fe6bb7fe640 1 --2- 172.21.15.27:0/212592264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6bc15d1d0 0x7fe6bc0ccd10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009870 tx=0x7fe6ac02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d99c2d85d27ce28 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:46.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.362+0000 7fe6baffd640 1 -- 172.21.15.27:0/212592264 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac031640 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.362+0000 7fe6baffd640 1 -- 172.21.15.27:0/212592264 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6ac0317a0 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.362+0000 7fe6baffd640 1 -- 172.21.15.27:0/212592264 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac031b30 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.362+0000 7fe6c1c69640 1 -- 172.21.15.27:0/212592264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6bc15d1d0 msgr2=0x7fe6bc0ccd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:46.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.362+0000 7fe6c1c69640 1 --2- 172.21.15.27:0/212592264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6bc15d1d0 0x7fe6bc0ccd10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac009870 tx=0x7fe6ac02b8f0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.363+0000 7fe6c1c69640 1 -- 172.21.15.27:0/212592264 shutdown_connections 2023-12-19T18:21:46.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.363+0000 7fe6c1c69640 1 --2- 172.21.15.27:0/212592264 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6bc071140 0x7fe6bc0cd250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.363+0000 7fe6c1c69640 1 --2- 172.21.15.27:0/212592264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6bc15d1d0 0x7fe6bc0ccd10 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.363+0000 7fe6c1c69640 1 -- 172.21.15.27:0/212592264 >> 172.21.15.27:0/212592264 conn(0x7fe6bc0ceb50 msgr2=0x7fe6bc0cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:46.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.363+0000 7fe6c1c69640 1 -- 172.21.15.27:0/212592264 shutdown_connections 2023-12-19T18:21:46.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.363+0000 7fe6c1c69640 1 -- 172.21.15.27:0/212592264 wait complete. 2023-12-19T18:21:46.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.365+0000 7fe6c1c69640 1 Processor -- start 2023-12-19T18:21:46.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.365+0000 7fe6c1c69640 1 -- start start 2023-12-19T18:21:46.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.366+0000 7fe6c1c69640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6bc071140 0x7fe6bc159c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.366+0000 7fe6c1c69640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6bc0cd790 0x7fe6bc06cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.366+0000 7fe6c1c69640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6bc15d1d0 0x7fe6bc06d140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.366+0000 7fe6c1c69640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc1f2630 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.366+0000 7fe6c1c69640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc0c2cd0 con 0x7fe6bc071140 2023-12-19T18:21:46.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.366+0000 7fe6c1c69640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6bc15a190 con 0x7fe6bc0cd790 2023-12-19T18:21:46.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.366+0000 7fe6bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6bc15d1d0 0x7fe6bc06d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.366+0000 7fe6bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6bc15d1d0 0x7fe6bc06d140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40766/0 (socket says 172.21.15.27:40766) 2023-12-19T18:21:46.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.366+0000 7fe6bbfff640 1 -- 172.21.15.27:0/1197044020 learned_addr learned my addr 172.21.15.27:0/1197044020 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:46.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.366+0000 7fe6bb7fe640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6bc071140 0x7fe6bc159c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.366+0000 7fe6b3fff640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6bc0cd790 0x7fe6bc06cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.367+0000 7fe6bbfff640 1 -- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6bc0cd790 msgr2=0x7fe6bc06cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:46.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.367+0000 7fe6bbfff640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6bc0cd790 0x7fe6bc06cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.367+0000 7fe6bbfff640 1 -- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6bc071140 msgr2=0x7fe6bc159c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:46.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.367+0000 7fe6bbfff640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6bc071140 0x7fe6bc159c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.367+0000 7fe6bbfff640 1 -- 172.21.15.27:0/1197044020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6ac0094d0 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.367+0000 7fe6bb7fe640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6bc071140 0x7fe6bc159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:21:46.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.367+0000 7fe6bbfff640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6bc15d1d0 0x7fe6bc06d140 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400c7a0 tx=0x7fe6b400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:46.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.367+0000 7fe6b8ff9640 1 -- 172.21.15.27:0/1197044020 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4003b70 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.367+0000 7fe6b8ff9640 1 -- 172.21.15.27:0/1197044020 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6b4003cf0 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.367+0000 7fe6b8ff9640 1 -- 172.21.15.27:0/1197044020 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6b4003ea0 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.367+0000 7fe6c1c69640 1 -- 172.21.15.27:0/1197044020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6bc0b2450 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.368+0000 7fe6c1c69640 1 -- 172.21.15.27:0/1197044020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6bc0c2680 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.369+0000 7fe6c1c69640 1 -- 172.21.15.27:0/1197044020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6bc0c2e80 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.370+0000 7fe6b8ff9640 1 -- 172.21.15.27:0/1197044020 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe6b400aa90 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.371+0000 7fe6b8ff9640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe69c06a430 0x7fe69c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.371+0000 7fe6b8ff9640 1 -- 172.21.15.27:0/1197044020 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6b4088750 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.371+0000 7fe6bb7fe640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe69c06a430 0x7fe69c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.372+0000 7fe6bb7fe640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe69c06a430 0x7fe69c06c8f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac036000 tx=0x7fe6ac03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:46.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.375+0000 7fe6b8ff9640 1 -- 172.21.15.27:0/1197044020 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe6bc0c2e80 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.447 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:21:46.456 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:21:46.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.571+0000 7fe6c1c69640 1 -- 172.21.15.27:0/1197044020 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe6bc0c2e80 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.574+0000 7fe6b8ff9640 1 -- 172.21.15.27:0/1197044020 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v123) v1 ==== 81+0+33551 (secure 0 0 0) 0x7fe6bc0c2e80 con 0x7fe6bc15d1d0 2023-12-19T18:21:46.574 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:46.575 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:21:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 123, 2023-12-19T18:21:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:21:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:21:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:21:41.838074+0000", 2023-12-19T18:21:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:21:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:21:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:21:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:21:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:21:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:21:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:21:46.575 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:21:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:21:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:21:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:21:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:21:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:21:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:21:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 13, 2023-12-19T18:21:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:21:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:21:46.576 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:21:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:21:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:21:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:21:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:21:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:21:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:21:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:21:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:21:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:46.577 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:21:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:21:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:21:46.578 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:21:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:21:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:21:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:46.579 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:46.580 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:46.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:46.582 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:46.583 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 12, 2023-12-19T18:21:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:21:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:21:06.258281+0000", 2023-12-19T18:21:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:21:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:21:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:46.584 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:21:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:21:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:21:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:21:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:21:46.585 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:21:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:46.586 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:46.587 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:46.588 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:21:46.589 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:21:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:21:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:21:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:21:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.75, 2023-12-19T18:21:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.75, 2023-12-19T18:21:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:21:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.75, 2023-12-19T18:21:46.590 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.75, 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 13, 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:21:07.308887+0000", 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:46.591 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:21:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:21:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:21:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:21:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:21:46.592 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:21:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:46.593 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:46.594 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.595 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:46.596 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:21:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:21:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:21:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:21:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:21:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:21:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:21:46.597 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:46.598 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:21:46.599 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:21:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.600 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:21:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:21:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:46.601 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:21:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:21:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.602 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:21:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.603 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:21:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:21:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:21:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:21:46.605 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:21:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:21:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:46.606 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:21:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.607 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:21:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:21:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:46.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:21:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.609 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:21:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:21:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:21:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:21:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:21:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:21:46.611 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:21:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:21:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:21:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:46.612 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.613 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:21:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:21:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.614 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:21:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:21:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.615 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:21:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:21:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.616 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:21:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:21:46.617 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:21:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:21:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:21:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:21:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:21:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:46.619 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:21:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:21:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.620 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:21:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:21:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:46.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:21:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.622 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:21:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:21:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.623 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:21:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:21:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:21:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:21:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:21:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:46.624 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:21:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:46.625 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:21:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.626 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:21:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:21:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:21:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:21:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:46.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:21:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:21:46.629 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:21:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:21:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:21:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:21:46.630 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:21:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:21:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:46.631 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:21:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:46.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:21:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.633 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:21:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:21:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.634 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:21:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:21:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:21:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:21:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:21:46.636 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:21:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:21:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:21:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:21:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:46.637 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:21:46.638 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:21:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.639 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:21:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:21:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.640 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:21:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:21:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:21:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:21:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:21:46.642 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:21:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:21:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:21:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:21:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:46.643 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:21:46.644 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:21:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.646 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:21:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:21:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:46.647 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:21:46.648 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:21:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:21:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:21:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:21:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:21:46.649 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:46.650 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:21:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:21:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:21:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:46.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:21:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:21:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.652 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:21:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:21:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:21:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:46.653 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:21:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:21:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:46.654 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:21:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:21:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:21:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.655 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:21:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:21:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:21:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:21:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:21:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:21:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:21:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:21:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:21:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:21:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:21:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:21:46.656 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:21:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:21:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:21:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:21:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:21:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:21:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:21:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:21:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:21:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:21:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:21:46.657 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:21:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:21:46.658 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:46.658 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:21:46.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.576+0000 7fe6c1c69640 1 -- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe69c06a430 msgr2=0x7fe69c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:46.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.576+0000 7fe6c1c69640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe69c06a430 0x7fe69c06c8f0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac036000 tx=0x7fe6ac03c040 comp rx=0 tx=0).stop 2023-12-19T18:21:46.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.577+0000 7fe6c1c69640 1 -- 172.21.15.27:0/1197044020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6bc15d1d0 msgr2=0x7fe6bc06d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:46.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.577+0000 7fe6c1c69640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6bc15d1d0 0x7fe6bc06d140 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fe6b400c7a0 tx=0x7fe6b400cc60 comp rx=0 tx=0).stop 2023-12-19T18:21:46.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.577+0000 7fe6c1c69640 1 -- 172.21.15.27:0/1197044020 shutdown_connections 2023-12-19T18:21:46.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.577+0000 7fe6c1c69640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe69c06a430 0x7fe69c06c8f0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.577+0000 7fe6c1c69640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6bc15d1d0 0x7fe6bc06d140 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.577+0000 7fe6c1c69640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6bc0cd790 0x7fe6bc06cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.577+0000 7fe6c1c69640 1 --2- 172.21.15.27:0/1197044020 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6bc071140 0x7fe6bc159c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.577+0000 7fe6c1c69640 1 -- 172.21.15.27:0/1197044020 >> 172.21.15.27:0/1197044020 conn(0x7fe6bc0ceb50 msgr2=0x7fe6bc0afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:46.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.577+0000 7fe6c1c69640 1 -- 172.21.15.27:0/1197044020 shutdown_connections 2023-12-19T18:21:46.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.577+0000 7fe6c1c69640 1 -- 172.21.15.27:0/1197044020 wait complete. 2023-12-19T18:21:46.661 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-12-19T18:21:46.662 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:21:46.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.943+0000 7f57af59e640 1 Processor -- start 2023-12-19T18:21:46.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.943+0000 7f57af59e640 1 -- start start 2023-12-19T18:21:46.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.944+0000 7f57af59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57b015ad80 0x7f57b015f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.944+0000 7f57af59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f57b00b30d0 0x7f57b00cc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.944+0000 7f57af59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57b00cffc0 con 0x7f57b00b2560 2023-12-19T18:21:46.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.944+0000 7f57af59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57b015f8a0 con 0x7f57b00b30d0 2023-12-19T18:21:46.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.944+0000 7f57af59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b006e470 con 0x7f57b015ad80 2023-12-19T18:21:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.945+0000 7f57add9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f57b00b2560 0x7f57b00b29c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39616/0 (socket says 172.21.15.27:39616) 2023-12-19T18:21:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.945+0000 7f57add9b640 1 -- 172.21.15.27:0/3227600826 learned_addr learned my addr 172.21.15.27:0/3227600826 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.945+0000 7f57ae59c640 1 --2- 172.21.15.27:0/3227600826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57b015ad80 0x7f57b015f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.945+0000 7f57aed9d640 1 --2- 172.21.15.27:0/3227600826 >> v2:172.21.15.192:3300/0 conn(0x7f57b00b30d0 0x7f57b00cc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.945+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768876144 0 0) 0x7f57b00cffc0 con 0x7f57b00b2560 2023-12-19T18:21:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.945+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f579c003520 con 0x7f57b00b2560 2023-12-19T18:21:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4023595461 0 0) 0x7f579c003520 con 0x7f57b00b2560 2023-12-19T18:21:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57b015ad80 msgr2=0x7f57b015f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 --2- 172.21.15.27:0/3227600826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57b015ad80 0x7f57b015f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 >> v2:172.21.15.192:3300/0 conn(0x7f57b00b30d0 msgr2=0x7f57b00cc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 --2- 172.21.15.27:0/3227600826 >> v2:172.21.15.192:3300/0 conn(0x7f57b00b30d0 0x7f57b00cc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b00cfa00 con 0x7f57b00b2560 2023-12-19T18:21:46.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f57980019f0 con 0x7f57b00b2560 2023-12-19T18:21:46.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 >> v1:172.21.15.27:6789/0 conn(0x7f57b00b2560 legacy=0x7f57b00b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:46.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 --2- 172.21.15.27:0/3227600826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57b015ad80 0x7f579c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 --2- 172.21.15.27:0/3227600826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f579c003f40 0x7f579c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 --2- 172.21.15.27:0/3227600826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c004d00 0x7f579c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b015f8a0 con 0x7f579c004d00 2023-12-19T18:21:46.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b006e470 con 0x7f579c003f40 2023-12-19T18:21:46.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f579c00ba30 con 0x7f57b015ad80 2023-12-19T18:21:46.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57aed9d640 1 --2- 172.21.15.27:0/3227600826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c004d00 0x7f579c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.946+0000 7f57add9b640 1 --2- 172.21.15.27:0/3227600826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f579c003f40 0x7f579c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.947+0000 7f57ae59c640 1 --2- 172.21.15.27:0/3227600826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57b015ad80 0x7f579c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.947+0000 7f57aed9d640 1 -- 172.21.15.27:0/3227600826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57b015ad80 msgr2=0x7f579c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:46.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.947+0000 7f57aed9d640 1 --2- 172.21.15.27:0/3227600826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57b015ad80 0x7f579c003970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.947+0000 7f57aed9d640 1 -- 172.21.15.27:0/3227600826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f579c003f40 msgr2=0x7f579c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:46.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.947+0000 7f57aed9d640 1 --2- 172.21.15.27:0/3227600826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f579c003f40 0x7f579c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.947+0000 7f57aed9d640 1 -- 172.21.15.27:0/3227600826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f579c003520 con 0x7f579c004d00 2023-12-19T18:21:46.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.948+0000 7f57aed9d640 1 --2- 172.21.15.27:0/3227600826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c004d00 0x7f579c005180 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009a90 tx=0x7f579c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8caa574453e7a322 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:46.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.949+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a802c6b0 con 0x7f579c004d00 2023-12-19T18:21:46.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.949+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57a802c810 con 0x7f579c004d00 2023-12-19T18:21:46.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.949+0000 7f57ad59a640 1 -- 172.21.15.27:0/3227600826 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57a802cba0 con 0x7f579c004d00 2023-12-19T18:21:46.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.949+0000 7f57af59e640 1 -- 172.21.15.27:0/3227600826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c004d00 msgr2=0x7f579c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:46.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.949+0000 7f57af59e640 1 --2- 172.21.15.27:0/3227600826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c004d00 0x7f579c005180 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f57a8009a90 tx=0x7f579c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.949+0000 7f57aed9d640 1 -- 172.21.15.27:0/3227600826 reap_dead start 2023-12-19T18:21:46.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.949+0000 7f57af59e640 1 -- 172.21.15.27:0/3227600826 shutdown_connections 2023-12-19T18:21:46.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.949+0000 7f57af59e640 1 -- 172.21.15.27:0/3227600826 >> 172.21.15.27:0/3227600826 conn(0x7f57b006d4e0 msgr2=0x7f57b006d8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:46.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.949+0000 7f57af59e640 1 -- 172.21.15.27:0/3227600826 shutdown_connections 2023-12-19T18:21:46.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.950+0000 7f57af59e640 1 -- 172.21.15.27:0/3227600826 wait complete. 2023-12-19T18:21:46.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.951+0000 7f57af59e640 1 Processor -- start 2023-12-19T18:21:46.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.952+0000 7f57af59e640 1 -- start start 2023-12-19T18:21:46.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.952+0000 7f57af59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f579c004d00 0x7f57b00c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57af59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57b00b5800 0x7f57b00c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57af59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57b00b5c80 0x7f57b00c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57af59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b00c7020 con 0x7f57b00b5800 2023-12-19T18:21:46.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57af59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57b014b410 con 0x7f579c004d00 2023-12-19T18:21:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57af59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57b00c5090 con 0x7f57b00b5c80 2023-12-19T18:21:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57add9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57b00b5800 0x7f57b00c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57add9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57b00b5800 0x7f57b00c4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40778/0 (socket says 172.21.15.27:40778) 2023-12-19T18:21:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57add9b640 1 -- 172.21.15.27:0/1578458111 learned_addr learned my addr 172.21.15.27:0/1578458111 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57ae59c640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f579c004d00 0x7f57b00c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57add9b640 1 -- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57b00b5c80 msgr2=0x7f57b00c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57add9b640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57b00b5c80 0x7f57b00c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57add9b640 1 -- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f579c004d00 msgr2=0x7f57b00c7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57add9b640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f579c004d00 0x7f57b00c7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57add9b640 1 -- 172.21.15.27:0/1578458111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b00c7980 con 0x7f57b00b5800 2023-12-19T18:21:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.953+0000 7f57ae59c640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f579c004d00 0x7f57b00c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.954+0000 7f57add9b640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57b00b5800 0x7f57b00c4610 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5798007a80 tx=0x7f579800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.954+0000 7f57a6ffd640 1 -- 172.21.15.27:0/1578458111 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f579800ee90 con 0x7f57b00b5800 2023-12-19T18:21:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.954+0000 7f57a6ffd640 1 -- 172.21.15.27:0/1578458111 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5798013420 con 0x7f57b00b5800 2023-12-19T18:21:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.954+0000 7f57af59e640 1 -- 172.21.15.27:0/1578458111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57b0148460 con 0x7f57b00b5800 2023-12-19T18:21:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.954+0000 7f57a6ffd640 1 -- 172.21.15.27:0/1578458111 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57980137d0 con 0x7f57b00b5800 2023-12-19T18:21:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.954+0000 7f57af59e640 1 -- 172.21.15.27:0/1578458111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57b00c6960 con 0x7f57b00b5800 2023-12-19T18:21:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.955+0000 7f57af59e640 1 -- 172.21.15.27:0/1578458111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57b004e840 con 0x7f57b00b5800 2023-12-19T18:21:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.956+0000 7f57a6ffd640 1 -- 172.21.15.27:0/1578458111 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5798013930 con 0x7f57b00b5800 2023-12-19T18:21:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.957+0000 7f57a6ffd640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f579006a430 0x7f579006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.957+0000 7f57a6ffd640 1 -- 172.21.15.27:0/1578458111 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5798089200 con 0x7f57b00b5800 2023-12-19T18:21:46.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.958+0000 7f57ae59c640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f579006a430 0x7f579006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:46.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.959+0000 7f57ae59c640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f579006a430 0x7f579006c8f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f57a00099e0 tx=0x7f57a0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:46.962+0000 7f57a6ffd640 1 -- 172.21.15.27:0/1578458111 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5798054750 con 0x7f57b00b5800 2023-12-19T18:21:47.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.155+0000 7f57af59e640 1 -- 172.21.15.27:0/1578458111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f57b00ce850 con 0x7f57b00b5800 2023-12-19T18:21:47.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.158+0000 7f57a6ffd640 1 -- 172.21.15.27:0/1578458111 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v123) v1 ==== 81+0+33551 (secure 0 0 0) 0x7f5798057d70 con 0x7f57b00b5800 2023-12-19T18:21:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:47.158 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:21:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 123, 2023-12-19T18:21:47.158 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:21:41.838074+0000", 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:21:47.159 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 13, 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:21:47.160 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:21:47.161 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:47.162 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:47.163 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:47.164 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:21:47.165 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:21:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:21:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:21:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 12, 2023-12-19T18:21:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:21:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:21:06.258281+0000", 2023-12-19T18:21:47.166 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:21:47.167 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:47.168 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:47.169 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:47.170 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.171 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.75, 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.75, 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.75, 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.75, 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 13, 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:21:07.308887+0000", 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:21:47.172 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:21:47.173 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:21:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:21:47.175 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:21:47.176 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:21:47.177 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:47.178 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:47.179 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.182 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:21:47.183 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.184 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:21:47.185 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:47.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.187 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.188 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:47.189 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.190 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.191 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.192 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:47.193 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:47.194 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.195 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.196 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:47.197 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:21:47.198 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:47.199 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:47.200 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:47.201 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.202 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.203 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:21:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:21:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:21:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:21:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.204 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.205 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:21:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:21:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:47.206 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.207 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:21:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:21:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.208 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.209 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:47.210 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.211 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.212 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.213 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.214 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.215 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 115, 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.216 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.217 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.220 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.221 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:21:47.222 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:21:47.223 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.224 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:47.225 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:47.226 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:47.227 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:47.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:21:47.229 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:21:47.230 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:21:47.231 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:21:47.231 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:21:47.231 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:21:47.231 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:21:47.231 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:21:47.231 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:21:47.231 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:21:47.231 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:21:47.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:21:47.231 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:21:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.161+0000 7f57af59e640 1 -- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f579006a430 msgr2=0x7f579006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.161+0000 7f57af59e640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f579006a430 0x7f579006c8f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f57a00099e0 tx=0x7f57a0008040 comp rx=0 tx=0).stop 2023-12-19T18:21:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.161+0000 7f57af59e640 1 -- 172.21.15.27:0/1578458111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57b00b5800 msgr2=0x7f57b00c4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.161+0000 7f57af59e640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57b00b5800 0x7f57b00c4610 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5798007a80 tx=0x7f579800e530 comp rx=0 tx=0).stop 2023-12-19T18:21:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.161+0000 7f57af59e640 1 -- 172.21.15.27:0/1578458111 shutdown_connections 2023-12-19T18:21:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.161+0000 7f57af59e640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f579006a430 0x7f579006c8f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.161+0000 7f57af59e640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f57b00b5c80 0x7f57b00c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.161+0000 7f57af59e640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57b00b5800 0x7f57b00c4610 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.161+0000 7f57af59e640 1 --2- 172.21.15.27:0/1578458111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f579c004d00 0x7f57b00c7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.161+0000 7f57af59e640 1 -- 172.21.15.27:0/1578458111 >> 172.21.15.27:0/1578458111 conn(0x7f57b006d4e0 msgr2=0x7f57b0159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.161+0000 7f57af59e640 1 -- 172.21.15.27:0/1578458111 shutdown_connections 2023-12-19T18:21:47.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.161+0000 7f57af59e640 1 -- 172.21.15.27:0/1578458111 wait complete. 2023-12-19T18:21:47.234 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:21:47.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.476+0000 7f497359e640 1 Processor -- start 2023-12-19T18:21:47.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.476+0000 7f497359e640 1 -- start start 2023-12-19T18:21:47.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.478+0000 7f497359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f49741553a0 0x7f49740c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:47.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.478+0000 7f497359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49740b0360 con 0x7f49740afeb0 2023-12-19T18:21:47.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.478+0000 7f497359e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49740cb350 con 0x7f497415ee30 2023-12-19T18:21:47.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.478+0000 7f497359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49740cbcf0 con 0x7f49741553a0 2023-12-19T18:21:47.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.478+0000 7f497259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f49741553a0 0x7f49740c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:47.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.478+0000 7f4971d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f49740afeb0 0x7f497415e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39630/0 (socket says 172.21.15.27:39630) 2023-12-19T18:21:47.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.478+0000 7f4971d9b640 1 -- 172.21.15.27:0/3599519220 learned_addr learned my addr 172.21.15.27:0/3599519220 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:47.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.479+0000 7f497159a640 1 -- 172.21.15.27:0/3599519220 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 617858093 0 0) 0x7f49740b0360 con 0x7f49740afeb0 2023-12-19T18:21:47.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.479+0000 7f497259c640 1 -- 172.21.15.27:0/3599519220 >> v1:172.21.15.192:6789/0 conn(0x7f497415ee30 legacy=0x7f49740c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:47.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.479+0000 7f497259c640 1 -- 172.21.15.27:0/3599519220 >> v1:172.21.15.27:6789/0 conn(0x7f49740afeb0 legacy=0x7f497415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:47.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.479+0000 7f497259c640 1 -- 172.21.15.27:0/3599519220 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49740cb630 con 0x7f49741553a0 2023-12-19T18:21:47.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.480+0000 7f497259c640 1 --2- 172.21.15.27:0/3599519220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49741553a0 0x7f49740c50a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f49640097a0 tx=0x7f496402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d88c3fcadd32c04b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:47.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.481+0000 7f497159a640 1 -- 172.21.15.27:0/3599519220 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964031830 con 0x7f49741553a0 2023-12-19T18:21:47.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.481+0000 7f497159a640 1 -- 172.21.15.27:0/3599519220 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4964031990 con 0x7f49741553a0 2023-12-19T18:21:47.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.481+0000 7f497159a640 1 -- 172.21.15.27:0/3599519220 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964031d20 con 0x7f49741553a0 2023-12-19T18:21:47.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.481+0000 7f497359e640 1 -- 172.21.15.27:0/3599519220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49741553a0 msgr2=0x7f49740c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:47.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.481+0000 7f497359e640 1 --2- 172.21.15.27:0/3599519220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49741553a0 0x7f49740c50a0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f49640097a0 tx=0x7f496402b8d0 comp rx=0 tx=0).stop 2023-12-19T18:21:47.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.482+0000 7f497359e640 1 -- 172.21.15.27:0/3599519220 shutdown_connections 2023-12-19T18:21:47.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.482+0000 7f497359e640 1 --2- 172.21.15.27:0/3599519220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49741553a0 0x7f49740c50a0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:47.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.482+0000 7f497359e640 1 -- 172.21.15.27:0/3599519220 >> 172.21.15.27:0/3599519220 conn(0x7f4974069c70 msgr2=0x7f497406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:47.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.482+0000 7f497359e640 1 -- 172.21.15.27:0/3599519220 shutdown_connections 2023-12-19T18:21:47.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.482+0000 7f497359e640 1 -- 172.21.15.27:0/3599519220 wait complete. 2023-12-19T18:21:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.483+0000 7f497359e640 1 Processor -- start 2023-12-19T18:21:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.484+0000 7f497359e640 1 -- start start 2023-12-19T18:21:47.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.485+0000 7f497359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49740afeb0 0x7f49741f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:47.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.485+0000 7f497359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49741553a0 0x7f49740c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:47.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.485+0000 7f497359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f497415ee30 0x7f49740cd560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:47.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.485+0000 7f497359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49741f2fc0 con 0x7f49741553a0 2023-12-19T18:21:47.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.485+0000 7f497359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49741f2360 con 0x7f49740afeb0 2023-12-19T18:21:47.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.485+0000 7f497359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49741f2620 con 0x7f497415ee30 2023-12-19T18:21:47.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.485+0000 7f4971d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49741553a0 0x7f49740c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:47.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.485+0000 7f4971d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49741553a0 0x7f49740c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40790/0 (socket says 172.21.15.27:40790) 2023-12-19T18:21:47.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.485+0000 7f4971d9b640 1 -- 172.21.15.27:0/1473340082 learned_addr learned my addr 172.21.15.27:0/1473340082 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:47.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.485+0000 7f497259c640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49740afeb0 0x7f49741f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:47.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.486+0000 7f4972d9d640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f497415ee30 0x7f49740cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:47.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.486+0000 7f4971d9b640 1 -- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f497415ee30 msgr2=0x7f49740cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:47.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.486+0000 7f4971d9b640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f497415ee30 0x7f49740cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:47.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.486+0000 7f4971d9b640 1 -- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49740afeb0 msgr2=0x7f49741f52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:47.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.486+0000 7f4971d9b640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49740afeb0 0x7f49741f52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:47.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.486+0000 7f4971d9b640 1 -- 172.21.15.27:0/1473340082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4964009400 con 0x7f49741553a0 2023-12-19T18:21:47.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.486+0000 7f4972d9d640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f497415ee30 0x7f49740cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:47.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.486+0000 7f4971d9b640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49741553a0 0x7f49740c5a10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f496800c4a0 tx=0x7f496800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:47.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.486+0000 7f4962ffd640 1 -- 172.21.15.27:0/1473340082 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49680074e0 con 0x7f49741553a0 2023-12-19T18:21:47.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.486+0000 7f4962ffd640 1 -- 172.21.15.27:0/1473340082 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4968007640 con 0x7f49741553a0 2023-12-19T18:21:47.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.486+0000 7f497359e640 1 -- 172.21.15.27:0/1473340082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49741f2900 con 0x7f49741553a0 2023-12-19T18:21:47.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.486+0000 7f4962ffd640 1 -- 172.21.15.27:0/1473340082 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4968004a00 con 0x7f49741553a0 2023-12-19T18:21:47.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.487+0000 7f497359e640 1 -- 172.21.15.27:0/1473340082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49740c5f50 con 0x7f49741553a0 2023-12-19T18:21:47.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.488+0000 7f497359e640 1 -- 172.21.15.27:0/1473340082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49741f3100 con 0x7f49741553a0 2023-12-19T18:21:47.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.489+0000 7f4962ffd640 1 -- 172.21.15.27:0/1473340082 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4968004b60 con 0x7f49741553a0 2023-12-19T18:21:47.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.490+0000 7f4962ffd640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f495406a430 0x7f495406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:47.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.490+0000 7f4962ffd640 1 -- 172.21.15.27:0/1473340082 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4968087e00 con 0x7f49741553a0 2023-12-19T18:21:47.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.490+0000 7f497259c640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f495406a430 0x7f495406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:47.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.492+0000 7f497259c640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f495406a430 0x7f495406c8f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4964036000 tx=0x7f4964008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:47.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.495+0000 7f4962ffd640 1 -- 172.21.15.27:0/1473340082 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f49741f3100 con 0x7f49741553a0 2023-12-19T18:21:47.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.733+0000 7f497359e640 1 -- 172.21.15.27:0/1473340082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49741f3100 con 0x7f49741553a0 2023-12-19T18:21:47.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.734+0000 7f4962ffd640 1 -- 172.21.15.27:0/1473340082 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f49741f3100 con 0x7f49741553a0 2023-12-19T18:21:47.734 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:47.734 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2779436922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2779436922},{"type":"v1","addr":"172.21.15.192:6836","nonce":2779436922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3121062779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3121062779},{"type":"v1","addr":"172.21.15.192:6837","nonce":3121062779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5357,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2032553869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2032553869},{"type":"v1","addr":"172.21.15.27:6835","nonce":2032553869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:21:08.281615+0000","modified":"2023-12-19T18:21:11.800655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/252970177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":252970177},{"type":"v1","addr":"172.21.15.27:6834","nonce":252970177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T18:21:47.735 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 46 2023-12-19T18:21:47.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.737+0000 7f497359e640 1 -- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f495406a430 msgr2=0x7f495406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:47.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.737+0000 7f497359e640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f495406a430 0x7f495406c8f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f4964036000 tx=0x7f4964008030 comp rx=0 tx=0).stop 2023-12-19T18:21:47.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.737+0000 7f497359e640 1 -- 172.21.15.27:0/1473340082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49741553a0 msgr2=0x7f49740c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:47.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.737+0000 7f497359e640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49741553a0 0x7f49740c5a10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f496800c4a0 tx=0x7f496800c960 comp rx=0 tx=0).stop 2023-12-19T18:21:47.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.737+0000 7f497359e640 1 -- 172.21.15.27:0/1473340082 shutdown_connections 2023-12-19T18:21:47.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.737+0000 7f497359e640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f495406a430 0x7f495406c8f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:47.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.737+0000 7f497359e640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f497415ee30 0x7f49740cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:47.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.737+0000 7f497359e640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49741553a0 0x7f49740c5a10 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:47.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.737+0000 7f497359e640 1 --2- 172.21.15.27:0/1473340082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f49740afeb0 0x7f49741f52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:47.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.737+0000 7f497359e640 1 -- 172.21.15.27:0/1473340082 >> 172.21.15.27:0/1473340082 conn(0x7f4974069c70 msgr2=0x7f49740ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:47.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.737+0000 7f497359e640 1 -- 172.21.15.27:0/1473340082 shutdown_connections 2023-12-19T18:21:47.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:47.737+0000 7f497359e640 1 -- 172.21.15.27:0/1473340082 wait complete. 2023-12-19T18:21:47.758 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-12-19T18:21:48.000 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":6885,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":39.697031537999997,"mdsmap_epoch":46,"osdmap_epoch":123,"osdmap_epoch_barrier":118,"uptime":45.272594816999998} 2023-12-19T18:21:48.000 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 6885, "mdsmap_epoch": 46, "osdmap_epoch": 123, "osdmap_epoch_barrier": 118, "rank_uptime": 39.697031538, "state": "up:active", "uptime": 45.272594817, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:21:48.001 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-12-19T18:21:48.001 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:21:48.001 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:21:48.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:48.057 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-12-19T18:21:49.266 INFO:teuthology.orchestra.run.smithi027.stderr:50+0 records in 2023-12-19T18:21:49.267 INFO:teuthology.orchestra.run.smithi027.stderr:50+0 records out 2023-12-19T18:21:49.267 INFO:teuthology.orchestra.run.smithi027.stderr:52428800 bytes (52 MB, 50 MiB) copied, 1.14614 s, 45.7 MB/s 2023-12-19T18:21:49.269 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:21:49.269 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:21:49.396 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:21:49.515 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:21:49.516 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:21:49.516 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:21:49.579 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:21:49.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.811+0000 7fc2fd93c640 1 Processor -- start 2023-12-19T18:21:49.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.811+0000 7fc2fd93c640 1 -- start start 2023-12-19T18:21:49.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.813+0000 7fc2fd93c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc2f815d270 0x7fc2f80cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:49.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.813+0000 7fc2fd93c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc2f80cce20 0x7fc2f80cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:49.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.813+0000 7fc2fd93c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f806b4e0 con 0x7fc2f815ce90 2023-12-19T18:21:49.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.813+0000 7fc2fd93c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f806ce90 con 0x7fc2f80cce20 2023-12-19T18:21:49.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.813+0000 7fc2fd93c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f80704f0 con 0x7fc2f815d270 2023-12-19T18:21:49.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.813+0000 7fc2f77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc2f80cce20 0x7fc2f80cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:49.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.813+0000 7fc2f77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc2f80cce20 0x7fc2f80cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40804/0 (socket says 172.21.15.27:40804) 2023-12-19T18:21:49.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.813+0000 7fc2f77fe640 1 -- 172.21.15.27:0/365297913 learned_addr learned my addr 172.21.15.27:0/365297913 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:49.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.813+0000 7fc2effff640 1 --2- 172.21.15.27:0/365297913 >> v2:172.21.15.192:3300/0 conn(0x7fc2f815d270 0x7fc2f80cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:49.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.814+0000 7fc2f77fe640 1 -- 172.21.15.27:0/365297913 >> v2:172.21.15.192:3300/0 conn(0x7fc2f815d270 msgr2=0x7fc2f80cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:49.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.814+0000 7fc2f77fe640 1 --2- 172.21.15.27:0/365297913 >> v2:172.21.15.192:3300/0 conn(0x7fc2f815d270 0x7fc2f80cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:49.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.814+0000 7fc2f77fe640 1 -- 172.21.15.27:0/365297913 >> v1:172.21.15.192:6789/0 conn(0x7fc2f815ce90 legacy=0x7fc2f80cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:49.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.814+0000 7fc2f77fe640 1 -- 172.21.15.27:0/365297913 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f806b7c0 con 0x7fc2f80cce20 2023-12-19T18:21:49.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.815+0000 7fc2f77fe640 1 --2- 172.21.15.27:0/365297913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2f80cce20 0x7fc2f80cb7f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80cea20 tx=0x7fc2f002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7cb2721f5fe30 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:49.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.815+0000 7fc2f67fc640 1 -- 172.21.15.27:0/365297913 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f002ca80 con 0x7fc2f80cce20 2023-12-19T18:21:49.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.815+0000 7fc2f67fc640 1 -- 172.21.15.27:0/365297913 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f002cbe0 con 0x7fc2f80cce20 2023-12-19T18:21:49.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.815+0000 7fc2f67fc640 1 -- 172.21.15.27:0/365297913 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f00343f0 con 0x7fc2f80cce20 2023-12-19T18:21:49.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.816+0000 7fc2fd93c640 1 -- 172.21.15.27:0/365297913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2f80cce20 msgr2=0x7fc2f80cb7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:49.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.816+0000 7fc2fd93c640 1 --2- 172.21.15.27:0/365297913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2f80cce20 0x7fc2f80cb7f0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fc2f80cea20 tx=0x7fc2f002bee0 comp rx=0 tx=0).stop 2023-12-19T18:21:49.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.816+0000 7fc2fd93c640 1 -- 172.21.15.27:0/365297913 shutdown_connections 2023-12-19T18:21:49.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.816+0000 7fc2fd93c640 1 --2- 172.21.15.27:0/365297913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2f80cce20 0x7fc2f80cb7f0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:49.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.816+0000 7fc2fd93c640 1 --2- 172.21.15.27:0/365297913 >> v2:172.21.15.192:3300/0 conn(0x7fc2f815d270 0x7fc2f80cb2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:49.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.816+0000 7fc2fd93c640 1 -- 172.21.15.27:0/365297913 >> 172.21.15.27:0/365297913 conn(0x7fc2f806a330 msgr2=0x7fc2f80b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:49.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.816+0000 7fc2fd93c640 1 -- 172.21.15.27:0/365297913 shutdown_connections 2023-12-19T18:21:49.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.816+0000 7fc2fd93c640 1 -- 172.21.15.27:0/365297913 wait complete. 2023-12-19T18:21:49.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.818+0000 7fc2fd93c640 1 Processor -- start 2023-12-19T18:21:49.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.818+0000 7fc2fd93c640 1 -- start start 2023-12-19T18:21:49.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.819+0000 7fc2fd93c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2f80cce20 0x7fc2f81f1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:49.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.819+0000 7fc2fd93c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2f815ce90 0x7fc2f80b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:49.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.819+0000 7fc2fd93c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2f80b3f10 0x7fc2f80c3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:49.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.819+0000 7fc2fd93c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f815a210 con 0x7fc2f80b3f10 2023-12-19T18:21:49.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.819+0000 7fc2fd93c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f81f2500 con 0x7fc2f80cce20 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.819+0000 7fc2fd93c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f80b4390 con 0x7fc2f815ce90 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.820+0000 7fc2f77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2f80b3f10 0x7fc2f80c3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.820+0000 7fc2f77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2f80b3f10 0x7fc2f80c3c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40806/0 (socket says 172.21.15.27:40806) 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.820+0000 7fc2f77fe640 1 -- 172.21.15.27:0/4128828 learned_addr learned my addr 172.21.15.27:0/4128828 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.820+0000 7fc2effff640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2f815ce90 0x7fc2f80b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.820+0000 7fc2f77fe640 1 -- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2f815ce90 msgr2=0x7fc2f80b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.820+0000 7fc2f6ffd640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2f80cce20 0x7fc2f81f1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.820+0000 7fc2f77fe640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2f815ce90 0x7fc2f80b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.820+0000 7fc2f77fe640 1 -- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2f80cce20 msgr2=0x7fc2f81f1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.820+0000 7fc2f77fe640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2f80cce20 0x7fc2f81f1fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.820+0000 7fc2f77fe640 1 -- 172.21.15.27:0/4128828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0009b30 con 0x7fc2f80b3f10 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.820+0000 7fc2effff640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2f815ce90 0x7fc2f80b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.820+0000 7fc2f6ffd640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2f80cce20 0x7fc2f81f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:49.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.821+0000 7fc2f77fe640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2f80b3f10 0x7fc2f80c3c60 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc2f002c4f0 tx=0x7fc2f0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.821+0000 7fc2f4ff9640 1 -- 172.21.15.27:0/4128828 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f0009510 con 0x7fc2f80b3f10 2023-12-19T18:21:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.821+0000 7fc2f4ff9640 1 -- 172.21.15.27:0/4128828 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2f003e450 con 0x7fc2f80b3f10 2023-12-19T18:21:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.821+0000 7fc2f4ff9640 1 -- 172.21.15.27:0/4128828 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2f003e7e0 con 0x7fc2f80b3f10 2023-12-19T18:21:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.821+0000 7fc2fd93c640 1 -- 172.21.15.27:0/4128828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f80b4670 con 0x7fc2f80b3f10 2023-12-19T18:21:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.821+0000 7fc2fd93c640 1 -- 172.21.15.27:0/4128828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f80b1370 con 0x7fc2f80b3f10 2023-12-19T18:21:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.823+0000 7fc2fd93c640 1 -- 172.21.15.27:0/4128828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f80b48b0 con 0x7fc2f80b3f10 2023-12-19T18:21:49.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.824+0000 7fc2f4ff9640 1 -- 172.21.15.27:0/4128828 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc2f002cd50 con 0x7fc2f80b3f10 2023-12-19T18:21:49.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.825+0000 7fc2f4ff9640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc2d806a430 0x7fc2d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:49.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.825+0000 7fc2f4ff9640 1 -- 172.21.15.27:0/4128828 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2f00ab3f0 con 0x7fc2f80b3f10 2023-12-19T18:21:49.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.825+0000 7fc2f6ffd640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc2d806a430 0x7fc2d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.826+0000 7fc2f6ffd640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc2d806a430 0x7fc2d806c8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8003eb0 tx=0x7fc2e8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:49.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:49.829+0000 7fc2f4ff9640 1 -- 172.21.15.27:0/4128828 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc2f80b48b0 con 0x7fc2f80b3f10 2023-12-19T18:21:50.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.032+0000 7fc2fd93c640 1 -- 172.21.15.27:0/4128828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2f815a350 con 0x7fc2f80b3f10 2023-12-19T18:21:50.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.035+0000 7fc2f4ff9640 1 -- 172.21.15.27:0/4128828 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16116 (secure 0 0 0) 0x7fc2f815a350 con 0x7fc2f80b3f10 2023-12-19T18:21:50.035 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:50.035 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":123,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:21:41.838074+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:21:06.258281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T18:21:07.308887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:21:50.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.038+0000 7fc2fd93c640 1 -- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc2d806a430 msgr2=0x7fc2d806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.038+0000 7fc2fd93c640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc2d806a430 0x7fc2d806c8f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8003eb0 tx=0x7fc2e8009040 comp rx=0 tx=0).stop 2023-12-19T18:21:50.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.038+0000 7fc2fd93c640 1 -- 172.21.15.27:0/4128828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2f80b3f10 msgr2=0x7fc2f80c3c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.038+0000 7fc2fd93c640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2f80b3f10 0x7fc2f80c3c60 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc2f002c4f0 tx=0x7fc2f0004a40 comp rx=0 tx=0).stop 2023-12-19T18:21:50.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.038+0000 7fc2fd93c640 1 -- 172.21.15.27:0/4128828 shutdown_connections 2023-12-19T18:21:50.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.038+0000 7fc2fd93c640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc2d806a430 0x7fc2d806c8f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.038+0000 7fc2fd93c640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2f80b3f10 0x7fc2f80c3c60 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.038+0000 7fc2fd93c640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2f815ce90 0x7fc2f80b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.038+0000 7fc2fd93c640 1 --2- 172.21.15.27:0/4128828 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2f80cce20 0x7fc2f81f1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.038+0000 7fc2fd93c640 1 -- 172.21.15.27:0/4128828 >> 172.21.15.27:0/4128828 conn(0x7fc2f806a330 msgr2=0x7fc2f815f770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:50.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.038+0000 7fc2fd93c640 1 -- 172.21.15.27:0/4128828 shutdown_connections 2023-12-19T18:21:50.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.038+0000 7fc2fd93c640 1 -- 172.21.15.27:0/4128828 wait complete. 2023-12-19T18:21:50.058 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:21:06.258281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:21:07.308887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:21:50.058 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:21:50.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.300+0000 7fd49cd7e640 1 Processor -- start 2023-12-19T18:21:50.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.301+0000 7fd49cd7e640 1 -- start start 2023-12-19T18:21:50.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.302+0000 7fd49cd7e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49815d370 0x7fd4980b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.302+0000 7fd49cd7e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd49815d750 con 0x7fd4980b5160 2023-12-19T18:21:50.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.303+0000 7fd49cd7e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4980d1750 con 0x7fd4980b55f0 2023-12-19T18:21:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.303+0000 7fd49cd7e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4980b2880 con 0x7fd49815d370 2023-12-19T18:21:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.303+0000 7fd496d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd4980b5160 0x7fd4980b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39636/0 (socket says 172.21.15.27:39636) 2023-12-19T18:21:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.303+0000 7fd496d9d640 1 -- 172.21.15.27:0/1618409184 learned_addr learned my addr 172.21.15.27:0/1618409184 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.303+0000 7fd49659c640 1 --2- 172.21.15.27:0/1618409184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49815d370 0x7fd4980b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.303+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230956125 0 0) 0x7fd49815d750 con 0x7fd4980b5160 2023-12-19T18:21:50.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.303+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd484003520 con 0x7fd4980b5160 2023-12-19T18:21:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3237537851 0 0) 0x7fd484003520 con 0x7fd4980b5160 2023-12-19T18:21:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49815d370 msgr2=0x7fd4980b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49559a640 1 --2- 172.21.15.27:0/1618409184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49815d370 0x7fd4980b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 >> v1:172.21.15.192:6789/0 conn(0x7fd4980b55f0 legacy=0x7fd4980b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4980d1a30 con 0x7fd4980b5160 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fd490001670 con 0x7fd4980b5160 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 >> v1:172.21.15.27:6789/0 conn(0x7fd4980b5160 legacy=0x7fd4980b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49559a640 1 --2- 172.21.15.27:0/1618409184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4840039a0 0x7fd484003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49559a640 1 --2- 172.21.15.27:0/1618409184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4840047c0 0x7fd484004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49559a640 1 --2- 172.21.15.27:0/1618409184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49815d370 0x7fd484005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48400ba30 con 0x7fd4840039a0 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd48400bb70 con 0x7fd4840047c0 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd48400bce0 con 0x7fd49815d370 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.304+0000 7fd49659c640 1 --2- 172.21.15.27:0/1618409184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4840039a0 0x7fd484003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.305+0000 7fd495d9b640 1 --2- 172.21.15.27:0/1618409184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4840047c0 0x7fd484004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.305+0000 7fd496d9d640 1 --2- 172.21.15.27:0/1618409184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49815d370 0x7fd484005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.305+0000 7fd49659c640 1 -- 172.21.15.27:0/1618409184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49815d370 msgr2=0x7fd484005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.305+0000 7fd49659c640 1 --2- 172.21.15.27:0/1618409184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49815d370 0x7fd484005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.305+0000 7fd49659c640 1 -- 172.21.15.27:0/1618409184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4840047c0 msgr2=0x7fd484004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.305+0000 7fd49659c640 1 --2- 172.21.15.27:0/1618409184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd4840047c0 0x7fd484004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.305+0000 7fd49659c640 1 -- 172.21.15.27:0/1618409184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd484003520 con 0x7fd4840039a0 2023-12-19T18:21:50.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.305+0000 7fd496d9d640 1 --2- 172.21.15.27:0/1618409184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd49815d370 0x7fd484005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.306+0000 7fd49659c640 1 --2- 172.21.15.27:0/1618409184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4840039a0 0x7fd484003d80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd488009f90 tx=0x7fd4840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a421a484a90ca2c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.306+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48802cbc0 con 0x7fd4840039a0 2023-12-19T18:21:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.306+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48800f040 con 0x7fd4840039a0 2023-12-19T18:21:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.306+0000 7fd49559a640 1 -- 172.21.15.27:0/1618409184 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488034440 con 0x7fd4840039a0 2023-12-19T18:21:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.307+0000 7fd49cd7e640 1 -- 172.21.15.27:0/1618409184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4840039a0 msgr2=0x7fd484003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.307+0000 7fd49cd7e640 1 --2- 172.21.15.27:0/1618409184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4840039a0 0x7fd484003d80 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fd488009f90 tx=0x7fd4840057f0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.307+0000 7fd496d9d640 1 -- 172.21.15.27:0/1618409184 reap_dead start 2023-12-19T18:21:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.307+0000 7fd49cd7e640 1 -- 172.21.15.27:0/1618409184 shutdown_connections 2023-12-19T18:21:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.307+0000 7fd49cd7e640 1 -- 172.21.15.27:0/1618409184 >> 172.21.15.27:0/1618409184 conn(0x7fd49806a480 msgr2=0x7fd49806a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.307+0000 7fd49cd7e640 1 -- 172.21.15.27:0/1618409184 shutdown_connections 2023-12-19T18:21:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.307+0000 7fd49cd7e640 1 -- 172.21.15.27:0/1618409184 wait complete. 2023-12-19T18:21:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.309+0000 7fd49cd7e640 1 Processor -- start 2023-12-19T18:21:50.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.309+0000 7fd49cd7e640 1 -- start start 2023-12-19T18:21:50.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.310+0000 7fd49cd7e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4840039a0 0x7fd498148290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.310+0000 7fd49cd7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd498155350 0x7fd4980ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.310+0000 7fd49cd7e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980cbb90 0x7fd4980cc010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.310+0000 7fd49cd7e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd498156f70 con 0x7fd4840039a0 2023-12-19T18:21:50.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.310+0000 7fd49cd7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4981487d0 con 0x7fd498155350 2023-12-19T18:21:50.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.310+0000 7fd49cd7e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd498157210 con 0x7fd4980cbb90 2023-12-19T18:21:50.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.311+0000 7fd49659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4840039a0 0x7fd498148290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.311+0000 7fd49659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4840039a0 0x7fd498148290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40832/0 (socket says 172.21.15.27:40832) 2023-12-19T18:21:50.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.311+0000 7fd49659c640 1 -- 172.21.15.27:0/4063543825 learned_addr learned my addr 172.21.15.27:0/4063543825 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.311+0000 7fd496d9d640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980cbb90 0x7fd4980cc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.311+0000 7fd495d9b640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd498155350 0x7fd4980ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.311+0000 7fd49659c640 1 -- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980cbb90 msgr2=0x7fd4980cc010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.311+0000 7fd49659c640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980cbb90 0x7fd4980cc010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.311+0000 7fd49659c640 1 -- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd498155350 msgr2=0x7fd4980ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.311+0000 7fd49659c640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd498155350 0x7fd4980ca0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.311+0000 7fd49659c640 1 -- 172.21.15.27:0/4063543825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4981574f0 con 0x7fd4840039a0 2023-12-19T18:21:50.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.311+0000 7fd496d9d640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980cbb90 0x7fd4980cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:50.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.311+0000 7fd49659c640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4840039a0 0x7fd498148290 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd48802c5c0 tx=0x7fd488009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:50.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.312+0000 7fd48effd640 1 -- 172.21.15.27:0/4063543825 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488034d40 con 0x7fd4840039a0 2023-12-19T18:21:50.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.312+0000 7fd48effd640 1 -- 172.21.15.27:0/4063543825 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd488040450 con 0x7fd4840039a0 2023-12-19T18:21:50.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.312+0000 7fd48effd640 1 -- 172.21.15.27:0/4063543825 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd488040800 con 0x7fd4840039a0 2023-12-19T18:21:50.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.312+0000 7fd49cd7e640 1 -- 172.21.15.27:0/4063543825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4981577d0 con 0x7fd4840039a0 2023-12-19T18:21:50.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.312+0000 7fd49cd7e640 1 -- 172.21.15.27:0/4063543825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4980ca5e0 con 0x7fd4840039a0 2023-12-19T18:21:50.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.313+0000 7fd49cd7e640 1 -- 172.21.15.27:0/4063543825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4980ca770 con 0x7fd4840039a0 2023-12-19T18:21:50.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.314+0000 7fd48effd640 1 -- 172.21.15.27:0/4063543825 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd488040960 con 0x7fd4840039a0 2023-12-19T18:21:50.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.315+0000 7fd48effd640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd47806a430 0x7fd47806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.315+0000 7fd48effd640 1 -- 172.21.15.27:0/4063543825 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd4880ab120 con 0x7fd4840039a0 2023-12-19T18:21:50.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.316+0000 7fd495d9b640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd47806a430 0x7fd47806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.317+0000 7fd495d9b640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd47806a430 0x7fd47806c8f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd4981585a0 tx=0x7fd480007b70 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:50.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.319+0000 7fd48effd640 1 -- 172.21.15.27:0/4063543825 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd4980ca770 con 0x7fd4840039a0 2023-12-19T18:21:50.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.510+0000 7fd49cd7e640 1 -- 172.21.15.27:0/4063543825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd4980ca770 con 0x7fd4840039a0 2023-12-19T18:21:50.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.512+0000 7fd48effd640 1 -- 172.21.15.27:0/4063543825 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd4980ca770 con 0x7fd4840039a0 2023-12-19T18:21:50.512 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:21:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.514+0000 7fd49cd7e640 1 -- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd47806a430 msgr2=0x7fd47806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.514+0000 7fd49cd7e640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd47806a430 0x7fd47806c8f0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd4981585a0 tx=0x7fd480007b70 comp rx=0 tx=0).stop 2023-12-19T18:21:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.514+0000 7fd49cd7e640 1 -- 172.21.15.27:0/4063543825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4840039a0 msgr2=0x7fd498148290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.514+0000 7fd49cd7e640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4840039a0 0x7fd498148290 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fd48802c5c0 tx=0x7fd488009750 comp rx=0 tx=0).stop 2023-12-19T18:21:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.514+0000 7fd49cd7e640 1 -- 172.21.15.27:0/4063543825 shutdown_connections 2023-12-19T18:21:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.514+0000 7fd49cd7e640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd47806a430 0x7fd47806c8f0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.514+0000 7fd49cd7e640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd4980cbb90 0x7fd4980cc010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.514+0000 7fd49cd7e640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd498155350 0x7fd4980ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.514+0000 7fd49cd7e640 1 --2- 172.21.15.27:0/4063543825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4840039a0 0x7fd498148290 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.514+0000 7fd49cd7e640 1 -- 172.21.15.27:0/4063543825 >> 172.21.15.27:0/4063543825 conn(0x7fd49806a480 msgr2=0x7fd49815fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.514+0000 7fd49cd7e640 1 -- 172.21.15.27:0/4063543825 shutdown_connections 2023-12-19T18:21:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.514+0000 7fd49cd7e640 1 -- 172.21.15.27:0/4063543825 wait complete. 2023-12-19T18:21:50.535 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:21:50.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.773+0000 7f8c46151640 1 Processor -- start 2023-12-19T18:21:50.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.773+0000 7f8c46151640 1 -- start start 2023-12-19T18:21:50.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.774+0000 7f8c46151640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8c4015b1a0 0x7f8c4006f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.775+0000 7f8c46151640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40071420 0x7f8c4015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.775+0000 7f8c46151640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c4006bc80 con 0x7f8c400c2720 2023-12-19T18:21:50.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.775+0000 7f8c46151640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c4006d690 con 0x7f8c4015b1a0 2023-12-19T18:21:50.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.775+0000 7f8c46151640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4006d0a0 con 0x7f8c40071420 2023-12-19T18:21:50.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.775+0000 7f8c37fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8c400c2720 0x7f8c4006fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39648/0 (socket says 172.21.15.27:39648) 2023-12-19T18:21:50.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.775+0000 7f8c37fff640 1 -- 172.21.15.27:0/405221462 learned_addr learned my addr 172.21.15.27:0/405221462 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:50.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.775+0000 7f8c3f7fe640 1 --2- 172.21.15.27:0/405221462 >> v2:172.21.15.192:3300/0 conn(0x7f8c4015b1a0 0x7f8c4006f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.775+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1538202035 0 0) 0x7f8c28001110 con 0x7f8c400c2720 2023-12-19T18:21:50.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.775+0000 7f8c3ffff640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40071420 0x7f8c4015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.775+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c20003520 con 0x7f8c400c2720 2023-12-19T18:21:50.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3518575139 0 0) 0x7f8c20003520 con 0x7f8c400c2720 2023-12-19T18:21:50.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40071420 msgr2=0x7f8c4015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:50.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40071420 0x7f8c4015ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 >> v2:172.21.15.192:3300/0 conn(0x7f8c4015b1a0 msgr2=0x7f8c4006f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 --2- 172.21.15.27:0/405221462 >> v2:172.21.15.192:3300/0 conn(0x7f8c4015b1a0 0x7f8c4006f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c4006b6c0 con 0x7f8c400c2720 2023-12-19T18:21:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f8c28001800 con 0x7f8c400c2720 2023-12-19T18:21:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 >> v1:172.21.15.27:6789/0 conn(0x7f8c400c2720 legacy=0x7f8c4006fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40071420 0x7f8c20003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c20003f40 0x7f8c200047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c20004d00 0x7f8c20005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4006d690 con 0x7f8c20003f40 2023-12-19T18:21:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c4006d0a0 con 0x7f8c20004d00 2023-12-19T18:21:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2000ba30 con 0x7f8c40071420 2023-12-19T18:21:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.776+0000 7f8c37fff640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c20003f40 0x7f8c200047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.777+0000 7f8c3ffff640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c20004d00 0x7f8c20005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.777+0000 7f8c3f7fe640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40071420 0x7f8c20003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.777+0000 7f8c37fff640 1 -- 172.21.15.27:0/405221462 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40071420 msgr2=0x7f8c20003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.777+0000 7f8c37fff640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40071420 0x7f8c20003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.777+0000 7f8c37fff640 1 -- 172.21.15.27:0/405221462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c20004d00 msgr2=0x7f8c20005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.777+0000 7f8c37fff640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c20004d00 0x7f8c20005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.777+0000 7f8c37fff640 1 -- 172.21.15.27:0/405221462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c20003520 con 0x7f8c20003f40 2023-12-19T18:21:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.777+0000 7f8c3ffff640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c20004d00 0x7f8c20005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.777+0000 7f8c3f7fe640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40071420 0x7f8c20003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.778+0000 7f8c37fff640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c20003f40 0x7f8c200047c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800c0e0 tx=0x7f8c200057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86fa211e83428450 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:50.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.778+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28005ad0 con 0x7f8c20003f40 2023-12-19T18:21:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.778+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c28005c30 con 0x7f8c20003f40 2023-12-19T18:21:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.778+0000 7f8c3effd640 1 -- 172.21.15.27:0/405221462 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2802c7f0 con 0x7f8c20003f40 2023-12-19T18:21:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.779+0000 7f8c46151640 1 -- 172.21.15.27:0/405221462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c20003f40 msgr2=0x7f8c200047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.779+0000 7f8c46151640 1 --2- 172.21.15.27:0/405221462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c20003f40 0x7f8c200047c0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f8c2800c0e0 tx=0x7f8c200057f0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.779+0000 7f8c3ffff640 1 -- 172.21.15.27:0/405221462 reap_dead start 2023-12-19T18:21:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.779+0000 7f8c46151640 1 -- 172.21.15.27:0/405221462 shutdown_connections 2023-12-19T18:21:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.779+0000 7f8c46151640 1 -- 172.21.15.27:0/405221462 >> 172.21.15.27:0/405221462 conn(0x7f8c400cebd0 msgr2=0x7f8c400cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.779+0000 7f8c46151640 1 -- 172.21.15.27:0/405221462 shutdown_connections 2023-12-19T18:21:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.779+0000 7f8c46151640 1 -- 172.21.15.27:0/405221462 wait complete. 2023-12-19T18:21:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.781+0000 7f8c46151640 1 Processor -- start 2023-12-19T18:21:50.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.781+0000 7f8c46151640 1 -- start start 2023-12-19T18:21:50.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.782+0000 7f8c46151640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c20003f40 0x7f8c401f6a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.782+0000 7f8c46151640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c4014f9a0 0x7f8c4014fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.782+0000 7f8c46151640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40153af0 0x7f8c40153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.782+0000 7f8c46151640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c401544e0 con 0x7f8c4014f9a0 2023-12-19T18:21:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.782+0000 7f8c46151640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c401f6f50 con 0x7f8c20003f40 2023-12-19T18:21:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.782+0000 7f8c46151640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c401ecea0 con 0x7f8c40153af0 2023-12-19T18:21:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.783+0000 7f8c37fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c4014f9a0 0x7f8c4014fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.783+0000 7f8c37fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c4014f9a0 0x7f8c4014fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40844/0 (socket says 172.21.15.27:40844) 2023-12-19T18:21:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.783+0000 7f8c37fff640 1 -- 172.21.15.27:0/3959190558 learned_addr learned my addr 172.21.15.27:0/3959190558 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.783+0000 7f8c3f7fe640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c20003f40 0x7f8c401f6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.783+0000 7f8c3ffff640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40153af0 0x7f8c40153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.783+0000 7f8c37fff640 1 -- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40153af0 msgr2=0x7f8c40153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.783+0000 7f8c37fff640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40153af0 0x7f8c40153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.783+0000 7f8c37fff640 1 -- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c20003f40 msgr2=0x7f8c401f6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.783+0000 7f8c37fff640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c20003f40 0x7f8c401f6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.783+0000 7f8c37fff640 1 -- 172.21.15.27:0/3959190558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c401f2db0 con 0x7f8c4014f9a0 2023-12-19T18:21:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.783+0000 7f8c3ffff640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40153af0 0x7f8c40153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.784+0000 7f8c37fff640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c4014f9a0 0x7f8c4014fe20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f8c2802c390 tx=0x7f8c28000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.784+0000 7f8c3cff9640 1 -- 172.21.15.27:0/3959190558 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28039070 con 0x7f8c4014f9a0 2023-12-19T18:21:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.784+0000 7f8c3cff9640 1 -- 172.21.15.27:0/3959190558 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c2802cc20 con 0x7f8c4014f9a0 2023-12-19T18:21:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.784+0000 7f8c3cff9640 1 -- 172.21.15.27:0/3959190558 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28005e60 con 0x7f8c4014f9a0 2023-12-19T18:21:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.784+0000 7f8c46151640 1 -- 172.21.15.27:0/3959190558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c401f3090 con 0x7f8c4014f9a0 2023-12-19T18:21:50.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.784+0000 7f8c46151640 1 -- 172.21.15.27:0/3959190558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c401f3650 con 0x7f8c4014f9a0 2023-12-19T18:21:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.786+0000 7f8c46151640 1 -- 172.21.15.27:0/3959190558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c40068e90 con 0x7f8c4014f9a0 2023-12-19T18:21:50.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.786+0000 7f8c3cff9640 1 -- 172.21.15.27:0/3959190558 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8c28032070 con 0x7f8c4014f9a0 2023-12-19T18:21:50.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.788+0000 7f8c3cff9640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8c2406a460 0x7f8c2406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:50.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.788+0000 7f8c3cff9640 1 -- 172.21.15.27:0/3959190558 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c2803c040 con 0x7f8c4014f9a0 2023-12-19T18:21:50.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.789+0000 7f8c3f7fe640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8c2406a460 0x7f8c2406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:50.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.790+0000 7f8c3f7fe640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8c2406a460 0x7f8c2406c920 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8c400cbe10 tx=0x7f8c30008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:50.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.793+0000 7f8c3cff9640 1 -- 172.21.15.27:0/3959190558 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8c280750a0 con 0x7f8c4014f9a0 2023-12-19T18:21:50.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.987+0000 7f8c46151640 1 -- 172.21.15.27:0/3959190558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c4006bc80 con 0x7f8c4014f9a0 2023-12-19T18:21:50.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.988+0000 7f8c3cff9640 1 -- 172.21.15.27:0/3959190558 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8c4006bc80 con 0x7f8c4014f9a0 2023-12-19T18:21:50.988 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:21:50.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.990+0000 7f8c46151640 1 -- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8c2406a460 msgr2=0x7f8c2406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.990+0000 7f8c46151640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8c2406a460 0x7f8c2406c920 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f8c400cbe10 tx=0x7f8c30008040 comp rx=0 tx=0).stop 2023-12-19T18:21:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.990+0000 7f8c46151640 1 -- 172.21.15.27:0/3959190558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c4014f9a0 msgr2=0x7f8c4014fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.990+0000 7f8c46151640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c4014f9a0 0x7f8c4014fe20 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f8c2802c390 tx=0x7f8c28000fe0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.990+0000 7f8c46151640 1 -- 172.21.15.27:0/3959190558 shutdown_connections 2023-12-19T18:21:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.990+0000 7f8c46151640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8c2406a460 0x7f8c2406c920 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.990+0000 7f8c46151640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8c40153af0 0x7f8c40153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.990+0000 7f8c46151640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c4014f9a0 0x7f8c4014fe20 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.990+0000 7f8c46151640 1 --2- 172.21.15.27:0/3959190558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8c20003f40 0x7f8c401f6a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.991+0000 7f8c46151640 1 -- 172.21.15.27:0/3959190558 >> 172.21.15.27:0/3959190558 conn(0x7f8c400cebd0 msgr2=0x7f8c400c2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:50.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.991+0000 7f8c46151640 1 -- 172.21.15.27:0/3959190558 shutdown_connections 2023-12-19T18:21:50.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:50.991+0000 7f8c46151640 1 -- 172.21.15.27:0/3959190558 wait complete. 2023-12-19T18:21:51.011 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:21:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.249+0000 7f62dea70640 1 Processor -- start 2023-12-19T18:21:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.249+0000 7f62dea70640 1 -- start start 2023-12-19T18:21:51.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.250+0000 7f62dea70640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f62d80b8c30 0x7f62d80d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:51.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.250+0000 7f62dea70640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62d80ce130 0x7f62d80cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:51.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.250+0000 7f62dea70640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62d80ce560 con 0x7f62d80cf9b0 2023-12-19T18:21:51.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.250+0000 7f62dea70640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d80cea80 con 0x7f62d80b8c30 2023-12-19T18:21:51.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.250+0000 7f62dea70640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80b5f10 con 0x7f62d80ce130 2023-12-19T18:21:51.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.251+0000 7f62dc80c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f62d80b8c30 0x7f62d80d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:51.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.251+0000 7f62dc80c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f62d80b8c30 0x7f62d80d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40856/0 (socket says 172.21.15.27:40856) 2023-12-19T18:21:51.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.251+0000 7f62dc80c640 1 -- 172.21.15.27:0/4280448971 learned_addr learned my addr 172.21.15.27:0/4280448971 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:51.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.251+0000 7f62dd00d640 1 --2- 172.21.15.27:0/4280448971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62d80ce130 0x7f62d80cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:51.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.251+0000 7f62dc80c640 1 -- 172.21.15.27:0/4280448971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62d80ce130 msgr2=0x7f62d80cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:51.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.251+0000 7f62dc80c640 1 --2- 172.21.15.27:0/4280448971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62d80ce130 0x7f62d80cfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.251+0000 7f62dc80c640 1 -- 172.21.15.27:0/4280448971 >> v1:172.21.15.192:6789/0 conn(0x7f62d80cf9b0 legacy=0x7f62d80cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:51.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.251+0000 7f62dc80c640 1 -- 172.21.15.27:0/4280448971 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80ce7a0 con 0x7f62d80b8c30 2023-12-19T18:21:51.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.252+0000 7f62dc80c640 1 --2- 172.21.15.27:0/4280448971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d80b8c30 0x7f62d80d1fc0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009e50 tx=0x7f62c802be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed64c9233249f53 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.253+0000 7f62d77fe640 1 -- 172.21.15.27:0/4280448971 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c802c9f0 con 0x7f62d80b8c30 2023-12-19T18:21:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.253+0000 7f62d77fe640 1 -- 172.21.15.27:0/4280448971 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62c802cb50 con 0x7f62d80b8c30 2023-12-19T18:21:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.253+0000 7f62d77fe640 1 -- 172.21.15.27:0/4280448971 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62c80343f0 con 0x7f62d80b8c30 2023-12-19T18:21:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.253+0000 7f62dea70640 1 -- 172.21.15.27:0/4280448971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d80b8c30 msgr2=0x7f62d80d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.253+0000 7f62dea70640 1 --2- 172.21.15.27:0/4280448971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d80b8c30 0x7f62d80d1fc0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f62c8009e50 tx=0x7f62c802be50 comp rx=0 tx=0).stop 2023-12-19T18:21:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.253+0000 7f62dea70640 1 -- 172.21.15.27:0/4280448971 shutdown_connections 2023-12-19T18:21:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.253+0000 7f62dea70640 1 --2- 172.21.15.27:0/4280448971 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62d80ce130 0x7f62d80cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.253+0000 7f62dea70640 1 --2- 172.21.15.27:0/4280448971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d80b8c30 0x7f62d80d1fc0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.253+0000 7f62dea70640 1 -- 172.21.15.27:0/4280448971 >> 172.21.15.27:0/4280448971 conn(0x7f62d80d2fa0 msgr2=0x7f62d80d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:51.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.253+0000 7f62dea70640 1 -- 172.21.15.27:0/4280448971 shutdown_connections 2023-12-19T18:21:51.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.254+0000 7f62dea70640 1 -- 172.21.15.27:0/4280448971 wait complete. 2023-12-19T18:21:51.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.255+0000 7f62dea70640 1 Processor -- start 2023-12-19T18:21:51.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.255+0000 7f62dea70640 1 -- start start 2023-12-19T18:21:51.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.256+0000 7f62dea70640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62d80b8c30 0x7f62d814f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.256+0000 7f62dea70640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d80ce130 0x7f62d814e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.256+0000 7f62dea70640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d80cf9b0 0x7f62d814e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.256+0000 7f62dea70640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80ca070 con 0x7f62d80cf9b0 2023-12-19T18:21:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.256+0000 7f62dea70640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80ca350 con 0x7f62d80ce130 2023-12-19T18:21:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.256+0000 7f62dea70640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d80cc810 con 0x7f62d80b8c30 2023-12-19T18:21:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.257+0000 7f62dd00d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d80cf9b0 0x7f62d814e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.257+0000 7f62dd00d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d80cf9b0 0x7f62d814e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40864/0 (socket says 172.21.15.27:40864) 2023-12-19T18:21:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.257+0000 7f62dd00d640 1 -- 172.21.15.27:0/1154860033 learned_addr learned my addr 172.21.15.27:0/1154860033 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:51.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.257+0000 7f62dd00d640 1 -- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62d80b8c30 msgr2=0x7f62d814f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:51.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.257+0000 7f62dd00d640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62d80b8c30 0x7f62d814f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.257+0000 7f62dd00d640 1 -- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d80ce130 msgr2=0x7f62d814e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:51.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.257+0000 7f62d7fff640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d80ce130 0x7f62d814e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:51.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.257+0000 7f62dd00d640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d80ce130 0x7f62d814e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.257+0000 7f62dd00d640 1 -- 172.21.15.27:0/1154860033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62c8009b30 con 0x7f62d80cf9b0 2023-12-19T18:21:51.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.257+0000 7f62d7fff640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d80ce130 0x7f62d814e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:51.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.257+0000 7f62dd00d640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d80cf9b0 0x7f62d814e970 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c570 tx=0x7f62d000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:51.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.258+0000 7f62d57fa640 1 -- 172.21.15.27:0/1154860033 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0004220 con 0x7f62d80cf9b0 2023-12-19T18:21:51.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.258+0000 7f62d57fa640 1 -- 172.21.15.27:0/1154860033 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d0004380 con 0x7f62d80cf9b0 2023-12-19T18:21:51.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.258+0000 7f62d57fa640 1 -- 172.21.15.27:0/1154860033 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0003830 con 0x7f62d80cf9b0 2023-12-19T18:21:51.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.258+0000 7f62dea70640 1 -- 172.21.15.27:0/1154860033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d80ccad0 con 0x7f62d80cf9b0 2023-12-19T18:21:51.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.258+0000 7f62dea70640 1 -- 172.21.15.27:0/1154860033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d81f6710 con 0x7f62d80cf9b0 2023-12-19T18:21:51.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.259+0000 7f62dea70640 1 -- 172.21.15.27:0/1154860033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d8068e90 con 0x7f62d80cf9b0 2023-12-19T18:21:51.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.260+0000 7f62d57fa640 1 -- 172.21.15.27:0/1154860033 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f62d0003990 con 0x7f62d80cf9b0 2023-12-19T18:21:51.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.261+0000 7f62d57fa640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62b806a430 0x7f62b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:51.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.262+0000 7f62d57fa640 1 -- 172.21.15.27:0/1154860033 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62d0088570 con 0x7f62d80cf9b0 2023-12-19T18:21:51.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.262+0000 7f62dc80c640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62b806a430 0x7f62b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:51.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.263+0000 7f62dc80c640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62b806a430 0x7f62b806c8f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f62c8035000 tx=0x7f62c802e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:51.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.266+0000 7f62d57fa640 1 -- 172.21.15.27:0/1154860033 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f62d0053b60 con 0x7f62d80cf9b0 2023-12-19T18:21:51.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.459+0000 7f62dea70640 1 -- 172.21.15.27:0/1154860033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f62d80ca490 con 0x7f62d80cf9b0 2023-12-19T18:21:51.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.460+0000 7f62d57fa640 1 -- 172.21.15.27:0/1154860033 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7f62d0057180 con 0x7f62d80cf9b0 2023-12-19T18:21:51.461 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:21:51.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.463+0000 7f62dea70640 1 -- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62b806a430 msgr2=0x7f62b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:51.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.463+0000 7f62dea70640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62b806a430 0x7f62b806c8f0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f62c8035000 tx=0x7f62c802e000 comp rx=0 tx=0).stop 2023-12-19T18:21:51.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.463+0000 7f62dea70640 1 -- 172.21.15.27:0/1154860033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d80cf9b0 msgr2=0x7f62d814e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:51.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.463+0000 7f62dea70640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d80cf9b0 0x7f62d814e970 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c570 tx=0x7f62d000ca30 comp rx=0 tx=0).stop 2023-12-19T18:21:51.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.463+0000 7f62dea70640 1 -- 172.21.15.27:0/1154860033 shutdown_connections 2023-12-19T18:21:51.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.463+0000 7f62dea70640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f62b806a430 0x7f62b806c8f0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.463+0000 7f62dea70640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62d80cf9b0 0x7f62d814e970 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.463+0000 7f62dea70640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f62d80ce130 0x7f62d814e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.463+0000 7f62dea70640 1 --2- 172.21.15.27:0/1154860033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62d80b8c30 0x7f62d814f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.463+0000 7f62dea70640 1 -- 172.21.15.27:0/1154860033 >> 172.21.15.27:0/1154860033 conn(0x7f62d80d2fa0 msgr2=0x7f62d80b7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:51.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.463+0000 7f62dea70640 1 -- 172.21.15.27:0/1154860033 shutdown_connections 2023-12-19T18:21:51.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.463+0000 7f62dea70640 1 -- 172.21.15.27:0/1154860033 wait complete. 2023-12-19T18:21:51.484 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:21:51.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.719+0000 7f4efae38640 1 Processor -- start 2023-12-19T18:21:51.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.719+0000 7f4efae38640 1 -- start start 2023-12-19T18:21:51.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.720+0000 7f4efae38640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ef415adf0 0x7f4ef406d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:51.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.720+0000 7f4efae38640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4ef406fc10 0x7f4ef40712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:51.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.720+0000 7f4efae38640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ef4071820 con 0x7f4ef4070c00 2023-12-19T18:21:51.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.720+0000 7f4efae38640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef40703c0 con 0x7f4ef415adf0 2023-12-19T18:21:51.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.720+0000 7f4efae38640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ef406d8c0 con 0x7f4ef406fc10 2023-12-19T18:21:51.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.721+0000 7f4ef8bd4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ef415adf0 0x7f4ef406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:51.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.721+0000 7f4ef8bd4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ef415adf0 0x7f4ef406d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40866/0 (socket says 172.21.15.27:40866) 2023-12-19T18:21:51.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.721+0000 7f4ef8bd4640 1 -- 172.21.15.27:0/1193606540 learned_addr learned my addr 172.21.15.27:0/1193606540 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:51.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.721+0000 7f4ef93d5640 1 --2- 172.21.15.27:0/1193606540 >> v2:172.21.15.192:3300/0 conn(0x7f4ef406fc10 0x7f4ef40712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:51.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.721+0000 7f4ef8bd4640 1 -- 172.21.15.27:0/1193606540 >> v2:172.21.15.192:3300/0 conn(0x7f4ef406fc10 msgr2=0x7f4ef40712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:51.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.721+0000 7f4ef8bd4640 1 --2- 172.21.15.27:0/1193606540 >> v2:172.21.15.192:3300/0 conn(0x7f4ef406fc10 0x7f4ef40712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.721+0000 7f4ef8bd4640 1 -- 172.21.15.27:0/1193606540 >> v1:172.21.15.27:6789/0 conn(0x7f4ef4070c00 legacy=0x7f4ef406f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:51.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.721+0000 7f4ef8bd4640 1 -- 172.21.15.27:0/1193606540 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ef406b520 con 0x7f4ef415adf0 2023-12-19T18:21:51.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.723+0000 7f4ef8bd4640 1 --2- 172.21.15.27:0/1193606540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ef415adf0 0x7f4ef406d380 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4ef406aff0 tx=0x7f4ee802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c2a27c1d6897b46 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:51.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.723+0000 7f4ef37fe640 1 -- 172.21.15.27:0/1193606540 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8031870 con 0x7f4ef415adf0 2023-12-19T18:21:51.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.723+0000 7f4ef37fe640 1 -- 172.21.15.27:0/1193606540 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee80319d0 con 0x7f4ef415adf0 2023-12-19T18:21:51.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.723+0000 7f4ef37fe640 1 -- 172.21.15.27:0/1193606540 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8031d60 con 0x7f4ef415adf0 2023-12-19T18:21:51.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.723+0000 7f4efae38640 1 -- 172.21.15.27:0/1193606540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ef415adf0 msgr2=0x7f4ef406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:51.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.723+0000 7f4efae38640 1 --2- 172.21.15.27:0/1193606540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ef415adf0 0x7f4ef406d380 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4ef406aff0 tx=0x7f4ee802b910 comp rx=0 tx=0).stop 2023-12-19T18:21:51.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.724+0000 7f4efae38640 1 -- 172.21.15.27:0/1193606540 shutdown_connections 2023-12-19T18:21:51.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.724+0000 7f4efae38640 1 --2- 172.21.15.27:0/1193606540 >> v2:172.21.15.192:3300/0 conn(0x7f4ef406fc10 0x7f4ef40712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.724+0000 7f4efae38640 1 --2- 172.21.15.27:0/1193606540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ef415adf0 0x7f4ef406d380 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.724+0000 7f4efae38640 1 -- 172.21.15.27:0/1193606540 >> 172.21.15.27:0/1193606540 conn(0x7f4ef40ceb50 msgr2=0x7f4ef40cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:51.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.724+0000 7f4efae38640 1 -- 172.21.15.27:0/1193606540 shutdown_connections 2023-12-19T18:21:51.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.724+0000 7f4efae38640 1 -- 172.21.15.27:0/1193606540 wait complete. 2023-12-19T18:21:51.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.726+0000 7f4efae38640 1 Processor -- start 2023-12-19T18:21:51.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.726+0000 7f4efae38640 1 -- start start 2023-12-19T18:21:51.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.727+0000 7f4efae38640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ef406fc10 0x7f4ef414df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.727+0000 7f4efae38640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ef4070c00 0x7f4ef40ada00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.727+0000 7f4efae38640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ef415adf0 0x7f4ef40c3b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.727+0000 7f4efae38640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40af020 con 0x7f4ef406fc10 2023-12-19T18:21:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.727+0000 7f4efae38640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40ae410 con 0x7f4ef4070c00 2023-12-19T18:21:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.727+0000 7f4efae38640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ef40ae6f0 con 0x7f4ef415adf0 2023-12-19T18:21:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.727+0000 7f4ef8bd4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ef406fc10 0x7f4ef414df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.727+0000 7f4ef8bd4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ef406fc10 0x7f4ef414df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40874/0 (socket says 172.21.15.27:40874) 2023-12-19T18:21:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.727+0000 7f4ef8bd4640 1 -- 172.21.15.27:0/3167953910 learned_addr learned my addr 172.21.15.27:0/3167953910 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.728+0000 7f4ef93d5640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ef415adf0 0x7f4ef40c3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.728+0000 7f4ef8bd4640 1 -- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ef415adf0 msgr2=0x7f4ef40c3b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.728+0000 7f4ef8bd4640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ef415adf0 0x7f4ef40c3b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.728+0000 7f4ef8bd4640 1 -- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ef4070c00 msgr2=0x7f4ef40ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:51.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.728+0000 7f4ef8bd4640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ef4070c00 0x7f4ef40ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.728+0000 7f4ef8bd4640 1 -- 172.21.15.27:0/3167953910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee80094d0 con 0x7f4ef406fc10 2023-12-19T18:21:51.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.728+0000 7f4ef93d5640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ef415adf0 0x7f4ef40c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:51.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.728+0000 7f4ef8bd4640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ef406fc10 0x7f4ef414df70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8000c00 tx=0x7f4ee80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:51.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.728+0000 7f4ef17fa640 1 -- 172.21.15.27:0/3167953910 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee803abe0 con 0x7f4ef406fc10 2023-12-19T18:21:51.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.728+0000 7f4ef17fa640 1 -- 172.21.15.27:0/3167953910 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee803ad40 con 0x7f4ef406fc10 2023-12-19T18:21:51.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.728+0000 7f4ef17fa640 1 -- 172.21.15.27:0/3167953910 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee8044630 con 0x7f4ef406fc10 2023-12-19T18:21:51.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.728+0000 7f4efae38640 1 -- 172.21.15.27:0/3167953910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ef40ae9d0 con 0x7f4ef406fc10 2023-12-19T18:21:51.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.729+0000 7f4efae38640 1 -- 172.21.15.27:0/3167953910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ef414e4b0 con 0x7f4ef406fc10 2023-12-19T18:21:51.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.730+0000 7f4efae38640 1 -- 172.21.15.27:0/3167953910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ef4068e90 con 0x7f4ef406fc10 2023-12-19T18:21:51.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.731+0000 7f4ef17fa640 1 -- 172.21.15.27:0/3167953910 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4ee804d450 con 0x7f4ef406fc10 2023-12-19T18:21:51.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.731+0000 7f4ef17fa640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ed806a430 0x7f4ed806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:51.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.732+0000 7f4ef17fa640 1 -- 172.21.15.27:0/3167953910 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ee802d040 con 0x7f4ef406fc10 2023-12-19T18:21:51.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.732+0000 7f4ef3fff640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ed806a430 0x7f4ed806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:51.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.733+0000 7f4ef3fff640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ed806a430 0x7f4ed806c8f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0000f60 tx=0x7f4ee00079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:51.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.736+0000 7f4ef17fa640 1 -- 172.21.15.27:0/3167953910 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4ee80af070 con 0x7f4ef406fc10 2023-12-19T18:21:51.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.945+0000 7f4efae38640 1 -- 172.21.15.27:0/3167953910 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4ef4071ca0 con 0x7f4ef406fc10 2023-12-19T18:21:51.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.946+0000 7f4ef17fa640 1 -- 172.21.15.27:0/3167953910 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v123) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4ee80764a0 con 0x7f4ef406fc10 2023-12-19T18:21:51.946 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:21:51.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.948+0000 7f4efae38640 1 -- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ed806a430 msgr2=0x7f4ed806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:51.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.948+0000 7f4efae38640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ed806a430 0x7f4ed806c8f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f4ee0000f60 tx=0x7f4ee00079b0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.948+0000 7f4efae38640 1 -- 172.21.15.27:0/3167953910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ef406fc10 msgr2=0x7f4ef414df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:51.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.948+0000 7f4efae38640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ef406fc10 0x7f4ef414df70 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8000c00 tx=0x7f4ee80046c0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.949+0000 7f4efae38640 1 -- 172.21.15.27:0/3167953910 shutdown_connections 2023-12-19T18:21:51.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.949+0000 7f4efae38640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ed806a430 0x7f4ed806c8f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.949+0000 7f4efae38640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ef415adf0 0x7f4ef40c3b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.949+0000 7f4efae38640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ef4070c00 0x7f4ef40ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.949+0000 7f4efae38640 1 --2- 172.21.15.27:0/3167953910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ef406fc10 0x7f4ef414df70 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:51.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.949+0000 7f4efae38640 1 -- 172.21.15.27:0/3167953910 >> 172.21.15.27:0/3167953910 conn(0x7f4ef40ceb50 msgr2=0x7f4ef40cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:51.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.949+0000 7f4efae38640 1 -- 172.21.15.27:0/3167953910 shutdown_connections 2023-12-19T18:21:51.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:51.949+0000 7f4efae38640 1 -- 172.21.15.27:0/3167953910 wait complete. 2023-12-19T18:21:51.968 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-12-19T18:21:51.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:51.968 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-19T18:21:52.070 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[106284]: fuse finished with error 0 and tester_r 0 2023-12-19T18:21:52.099 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:21:52.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:52.099 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:21:52.287 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:21:52.287 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:21:52.348 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:21:52.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.590+0000 7fc7ce6a3640 1 Processor -- start 2023-12-19T18:21:52.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.591+0000 7fc7ce6a3640 1 -- start start 2023-12-19T18:21:52.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.593+0000 7fc7ce6a3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc7c815ee30 0x7fc7c80c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:52.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.593+0000 7fc7ce6a3640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80cb350 con 0x7fc7c81553a0 2023-12-19T18:21:52.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.593+0000 7fc7ce6a3640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7c80cb630 con 0x7fc7c80afeb0 2023-12-19T18:21:52.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.593+0000 7fc7ce6a3640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7c80cbcf0 con 0x7fc7c815ee30 2023-12-19T18:21:52.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.593+0000 7fc7ccea0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc7c815ee30 0x7fc7c80c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:52.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.593+0000 7fc7cd6a1640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc7c81553a0 0x7fc7c815e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55724/0 (socket says 172.21.15.27:55724) 2023-12-19T18:21:52.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.593+0000 7fc7cd6a1640 1 -- 172.21.15.27:0/2452105559 learned_addr learned my addr 172.21.15.27:0/2452105559 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:52.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.594+0000 7fc7ccea0640 1 -- 172.21.15.27:0/2452105559 >> v1:172.21.15.192:6789/0 conn(0x7fc7c80afeb0 legacy=0x7fc7c80c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:52.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.594+0000 7fc7ccea0640 1 -- 172.21.15.27:0/2452105559 >> v1:172.21.15.27:6789/0 conn(0x7fc7c81553a0 legacy=0x7fc7c815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:52.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.594+0000 7fc7ccea0640 1 -- 172.21.15.27:0/2452105559 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c80cb910 con 0x7fc7c815ee30 2023-12-19T18:21:52.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.595+0000 7fc7ccea0640 1 --2- 172.21.15.27:0/2452105559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7c815ee30 0x7fc7c80c50a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80097a0 tx=0x7fc7b802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9796ab06eb146c0f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:52.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.595+0000 7fc7bffff640 1 -- 172.21.15.27:0/2452105559 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b802e070 con 0x7fc7c815ee30 2023-12-19T18:21:52.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.595+0000 7fc7bffff640 1 -- 172.21.15.27:0/2452105559 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b80315a0 con 0x7fc7c815ee30 2023-12-19T18:21:52.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.595+0000 7fc7bffff640 1 -- 172.21.15.27:0/2452105559 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8031930 con 0x7fc7c815ee30 2023-12-19T18:21:52.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.596+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/2452105559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7c815ee30 msgr2=0x7fc7c80c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:52.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.596+0000 7fc7ce6a3640 1 --2- 172.21.15.27:0/2452105559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7c815ee30 0x7fc7c80c50a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc7b80097a0 tx=0x7fc7b802b840 comp rx=0 tx=0).stop 2023-12-19T18:21:52.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.596+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/2452105559 shutdown_connections 2023-12-19T18:21:52.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.596+0000 7fc7ce6a3640 1 --2- 172.21.15.27:0/2452105559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7c815ee30 0x7fc7c80c50a0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:52.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.596+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/2452105559 >> 172.21.15.27:0/2452105559 conn(0x7fc7c8069c70 msgr2=0x7fc7c806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:52.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.596+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/2452105559 shutdown_connections 2023-12-19T18:21:52.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.596+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/2452105559 wait complete. 2023-12-19T18:21:52.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.598+0000 7fc7ce6a3640 1 Processor -- start 2023-12-19T18:21:52.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.598+0000 7fc7ce6a3640 1 -- start start 2023-12-19T18:21:52.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.599+0000 7fc7ce6a3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7c80afeb0 0x7fc7c8150410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:52.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.599+0000 7fc7ce6a3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7c81553a0 0x7fc7c80b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:52.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.599+0000 7fc7ce6a3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7c815ee30 0x7fc7c81557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:52.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.599+0000 7fc7ce6a3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80b1b00 con 0x7fc7c81553a0 2023-12-19T18:21:52.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.599+0000 7fc7ce6a3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c80cca00 con 0x7fc7c815ee30 2023-12-19T18:21:52.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.599+0000 7fc7ce6a3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c81566b0 con 0x7fc7c80afeb0 2023-12-19T18:21:52.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.600+0000 7fc7ccea0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7c81553a0 0x7fc7c80b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:52.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.600+0000 7fc7ccea0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7c81553a0 0x7fc7c80b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36908/0 (socket says 172.21.15.27:36908) 2023-12-19T18:21:52.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.600+0000 7fc7ccea0640 1 -- 172.21.15.27:0/3024459793 learned_addr learned my addr 172.21.15.27:0/3024459793 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:52.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.600+0000 7fc7cdea2640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7c815ee30 0x7fc7c81557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.600+0000 7fc7cd6a1640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7c80afeb0 0x7fc7c8150410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.600+0000 7fc7ccea0640 1 -- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7c80afeb0 msgr2=0x7fc7c8150410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.600+0000 7fc7ccea0640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7c80afeb0 0x7fc7c8150410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.600+0000 7fc7ccea0640 1 -- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7c815ee30 msgr2=0x7fc7c81557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.600+0000 7fc7ccea0640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7c815ee30 0x7fc7c81557f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.600+0000 7fc7ccea0640 1 -- 172.21.15.27:0/3024459793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7b8009400 con 0x7fc7c81553a0 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.600+0000 7fc7cdea2640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7c815ee30 0x7fc7c81557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.600+0000 7fc7cd6a1640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7c80afeb0 0x7fc7c8150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.601+0000 7fc7ccea0640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7c81553a0 0x7fc7c80b15c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8034040 tx=0x7fc7b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.601+0000 7fc7bdffb640 1 -- 172.21.15.27:0/3024459793 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8032900 con 0x7fc7c81553a0 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.601+0000 7fc7bdffb640 1 -- 172.21.15.27:0/3024459793 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b8032a80 con 0x7fc7c81553a0 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.601+0000 7fc7bdffb640 1 -- 172.21.15.27:0/3024459793 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b8032e30 con 0x7fc7c81553a0 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.601+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/3024459793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7c81560f0 con 0x7fc7c81553a0 2023-12-19T18:21:52.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.601+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/3024459793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7c80cc690 con 0x7fc7c81553a0 2023-12-19T18:21:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.602+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/3024459793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7c8150950 con 0x7fc7c81553a0 2023-12-19T18:21:52.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.603+0000 7fc7bdffb640 1 -- 172.21.15.27:0/3024459793 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc7b803b3f0 con 0x7fc7c81553a0 2023-12-19T18:21:52.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.604+0000 7fc7bdffb640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc7ac06a430 0x7fc7ac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:52.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.604+0000 7fc7bdffb640 1 -- 172.21.15.27:0/3024459793 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7b80aaa50 con 0x7fc7c81553a0 2023-12-19T18:21:52.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.605+0000 7fc7cd6a1640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc7ac06a430 0x7fc7ac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:52.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.606+0000 7fc7cd6a1640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc7ac06a430 0x7fc7ac06c8f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4007fd0 tx=0x7fc7c4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.608+0000 7fc7bdffb640 1 -- 172.21.15.27:0/3024459793 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc7c8150950 con 0x7fc7c81553a0 2023-12-19T18:21:52.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.808+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/3024459793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7c8150950 con 0x7fc7c81553a0 2023-12-19T18:21:52.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.811+0000 7fc7bdffb640 1 -- 172.21.15.27:0/3024459793 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16116 (secure 0 0 0) 0x7fc7c8150950 con 0x7fc7c81553a0 2023-12-19T18:21:52.811 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:52.811 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":123,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:21:41.838074+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:21:06.258281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T18:21:07.308887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:21:52.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.814+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc7ac06a430 msgr2=0x7fc7ac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:52.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.814+0000 7fc7ce6a3640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc7ac06a430 0x7fc7ac06c8f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fc7c4007fd0 tx=0x7fc7c4009040 comp rx=0 tx=0).stop 2023-12-19T18:21:52.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.814+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/3024459793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7c81553a0 msgr2=0x7fc7c80b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:52.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.814+0000 7fc7ce6a3640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7c81553a0 0x7fc7c80b15c0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8034040 tx=0x7fc7b80046c0 comp rx=0 tx=0).stop 2023-12-19T18:21:52.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.814+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/3024459793 shutdown_connections 2023-12-19T18:21:52.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.814+0000 7fc7ce6a3640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc7ac06a430 0x7fc7ac06c8f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:52.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.814+0000 7fc7ce6a3640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7c815ee30 0x7fc7c81557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:52.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.814+0000 7fc7ce6a3640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7c81553a0 0x7fc7c80b15c0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:52.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.814+0000 7fc7ce6a3640 1 --2- 172.21.15.27:0/3024459793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7c80afeb0 0x7fc7c8150410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:52.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.814+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/3024459793 >> 172.21.15.27:0/3024459793 conn(0x7fc7c8069c70 msgr2=0x7fc7c8159240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:52.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.814+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/3024459793 shutdown_connections 2023-12-19T18:21:52.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:52.814+0000 7fc7ce6a3640 1 -- 172.21.15.27:0/3024459793 wait complete. 2023-12-19T18:21:52.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:21:06.258281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:21:07.308887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:21:52.835 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:21:53.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.056+0000 7f3e91b0b640 1 Processor -- start 2023-12-19T18:21:53.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.056+0000 7f3e91b0b640 1 -- start start 2023-12-19T18:21:53.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.058+0000 7f3e91b0b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8c0cf540 0x7f3e8c15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.058+0000 7f3e91b0b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8c06b4c0 con 0x7f3e8c0cf160 2023-12-19T18:21:53.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.058+0000 7f3e91b0b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e8c161bd0 con 0x7f3e8c15aaf0 2023-12-19T18:21:53.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.058+0000 7f3e91b0b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c070080 con 0x7f3e8c0cf540 2023-12-19T18:21:53.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.059+0000 7f3e90b09640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3e8c0cf160 0x7f3e8c15e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55736/0 (socket says 172.21.15.27:55736) 2023-12-19T18:21:53.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.059+0000 7f3e90b09640 1 -- 172.21.15.27:0/2562743225 learned_addr learned my addr 172.21.15.27:0/2562743225 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:53.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.059+0000 7f3e8bfff640 1 --2- 172.21.15.27:0/2562743225 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8c0cf540 0x7f3e8c15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.059+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3137889417 0 0) 0x7f3e8c06b4c0 con 0x7f3e8c0cf160 2023-12-19T18:21:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.059+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e7c003520 con 0x7f3e8c0cf160 2023-12-19T18:21:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.059+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1911096926 0 0) 0x7f3e7c003520 con 0x7f3e8c0cf160 2023-12-19T18:21:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.059+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8c0cf540 msgr2=0x7f3e8c15a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.059+0000 7f3e8b7fe640 1 --2- 172.21.15.27:0/2562743225 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8c0cf540 0x7f3e8c15a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.059+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 >> v1:172.21.15.192:6789/0 conn(0x7f3e8c15aaf0 legacy=0x7f3e8c06b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.059+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c06d140 con 0x7f3e8c0cf160 2023-12-19T18:21:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.059+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f3e80001670 con 0x7f3e8c0cf160 2023-12-19T18:21:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 >> v1:172.21.15.27:6789/0 conn(0x7f3e8c0cf160 legacy=0x7f3e8c15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e8b7fe640 1 --2- 172.21.15.27:0/2562743225 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8c0cf540 0x7f3e7c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e8b7fe640 1 --2- 172.21.15.27:0/2562743225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c003f20 0x7f3e7c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e8b7fe640 1 --2- 172.21.15.27:0/2562743225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e7c004d00 0x7f3e7c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c070080 con 0x7f3e7c004d00 2023-12-19T18:21:53.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c00ba60 con 0x7f3e7c003f20 2023-12-19T18:21:53.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e7c00bc20 con 0x7f3e8c0cf540 2023-12-19T18:21:53.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e9130a640 1 --2- 172.21.15.27:0/2562743225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e7c004d00 0x7f3e7c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:53.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e8bfff640 1 --2- 172.21.15.27:0/2562743225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c003f20 0x7f3e7c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:53.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e9130a640 1 -- 172.21.15.27:0/2562743225 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8c0cf540 msgr2=0x7f3e7c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:53.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e9130a640 1 --2- 172.21.15.27:0/2562743225 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8c0cf540 0x7f3e7c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e9130a640 1 -- 172.21.15.27:0/2562743225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c003f20 msgr2=0x7f3e7c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:53.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e9130a640 1 --2- 172.21.15.27:0/2562743225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c003f20 0x7f3e7c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e9130a640 1 -- 172.21.15.27:0/2562743225 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e7c003520 con 0x7f3e7c004d00 2023-12-19T18:21:53.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.060+0000 7f3e8bfff640 1 --2- 172.21.15.27:0/2562743225 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c003f20 0x7f3e7c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:53.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.061+0000 7f3e9130a640 1 --2- 172.21.15.27:0/2562743225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e7c004d00 0x7f3e7c005180 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3e84001e50 tx=0x7f3e7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f18ba009c5452acb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:53.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.062+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84003b70 con 0x7f3e7c004d00 2023-12-19T18:21:53.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.062+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e84003cd0 con 0x7f3e7c004d00 2023-12-19T18:21:53.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.062+0000 7f3e8b7fe640 1 -- 172.21.15.27:0/2562743225 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e84005310 con 0x7f3e7c004d00 2023-12-19T18:21:53.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.062+0000 7f3e91b0b640 1 -- 172.21.15.27:0/2562743225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e7c004d00 msgr2=0x7f3e7c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:53.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.062+0000 7f3e91b0b640 1 --2- 172.21.15.27:0/2562743225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e7c004d00 0x7f3e7c005180 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3e84001e50 tx=0x7f3e7c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.062+0000 7f3e9130a640 1 -- 172.21.15.27:0/2562743225 reap_dead start 2023-12-19T18:21:53.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.062+0000 7f3e91b0b640 1 -- 172.21.15.27:0/2562743225 shutdown_connections 2023-12-19T18:21:53.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.062+0000 7f3e91b0b640 1 -- 172.21.15.27:0/2562743225 >> 172.21.15.27:0/2562743225 conn(0x7f3e8c06a060 msgr2=0x7f3e8c06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:53.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.062+0000 7f3e91b0b640 1 -- 172.21.15.27:0/2562743225 shutdown_connections 2023-12-19T18:21:53.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.062+0000 7f3e91b0b640 1 -- 172.21.15.27:0/2562743225 wait complete. 2023-12-19T18:21:53.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.064+0000 7f3e91b0b640 1 Processor -- start 2023-12-19T18:21:53.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.064+0000 7f3e91b0b640 1 -- start start 2023-12-19T18:21:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.065+0000 7f3e91b0b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c004d00 0x7f3e8c144350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.065+0000 7f3e91b0b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e8c0cb070 0x7f3e8c0cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.065+0000 7f3e91b0b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8c0b0670 0x7f3e8c0b0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.065+0000 7f3e91b0b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0cba30 con 0x7f3e8c0cb070 2023-12-19T18:21:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.065+0000 7f3e91b0b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c0b1030 con 0x7f3e7c004d00 2023-12-19T18:21:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.065+0000 7f3e91b0b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8c147b60 con 0x7f3e8c0b0670 2023-12-19T18:21:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.066+0000 7f3e8bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e8c0cb070 0x7f3e8c0cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.066+0000 7f3e8bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e8c0cb070 0x7f3e8c0cb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36934/0 (socket says 172.21.15.27:36934) 2023-12-19T18:21:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.066+0000 7f3e8bfff640 1 -- 172.21.15.27:0/3689490491 learned_addr learned my addr 172.21.15.27:0/3689490491 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.066+0000 7f3e90b09640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c004d00 0x7f3e8c144350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.066+0000 7f3e8bfff640 1 -- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8c0b0670 msgr2=0x7f3e8c0b0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.066+0000 7f3e9130a640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8c0b0670 0x7f3e8c0b0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.066+0000 7f3e8bfff640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8c0b0670 0x7f3e8c0b0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.066+0000 7f3e8bfff640 1 -- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c004d00 msgr2=0x7f3e8c144350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.066+0000 7f3e8bfff640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c004d00 0x7f3e8c144350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.066+0000 7f3e8bfff640 1 -- 172.21.15.27:0/3689490491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e8c144890 con 0x7f3e8c0cb070 2023-12-19T18:21:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.066+0000 7f3e9130a640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8c0b0670 0x7f3e8c0b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.066+0000 7f3e8bfff640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e8c0cb070 0x7f3e8c0cb4f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3e78010810 tx=0x7f3e78010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.067+0000 7f3e897fa640 1 -- 172.21.15.27:0/3689490491 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7800eb60 con 0x7f3e8c0cb070 2023-12-19T18:21:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.067+0000 7f3e897fa640 1 -- 172.21.15.27:0/3689490491 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7800ecc0 con 0x7f3e8c0cb070 2023-12-19T18:21:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.067+0000 7f3e897fa640 1 -- 172.21.15.27:0/3689490491 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7801c400 con 0x7f3e8c0cb070 2023-12-19T18:21:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.067+0000 7f3e91b0b640 1 -- 172.21.15.27:0/3689490491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8c1f5f30 con 0x7f3e8c0cb070 2023-12-19T18:21:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.067+0000 7f3e91b0b640 1 -- 172.21.15.27:0/3689490491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8c1f64f0 con 0x7f3e8c0cb070 2023-12-19T18:21:53.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.068+0000 7f3e91b0b640 1 -- 172.21.15.27:0/3689490491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8c04e840 con 0x7f3e8c0cb070 2023-12-19T18:21:53.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.069+0000 7f3e897fa640 1 -- 172.21.15.27:0/3689490491 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e7801c560 con 0x7f3e8c0cb070 2023-12-19T18:21:53.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.070+0000 7f3e897fa640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e7006a460 0x7f3e7006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.070+0000 7f3e897fa640 1 -- 172.21.15.27:0/3689490491 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e78088d80 con 0x7f3e8c0cb070 2023-12-19T18:21:53.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.071+0000 7f3e90b09640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e7006a460 0x7f3e7006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:53.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.072+0000 7f3e90b09640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e7006a460 0x7f3e7006c920 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3e80007a10 tx=0x7f3e80009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:53.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.074+0000 7f3e897fa640 1 -- 172.21.15.27:0/3689490491 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3e780542d0 con 0x7f3e8c0cb070 2023-12-19T18:21:53.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.267+0000 7f3e91b0b640 1 -- 172.21.15.27:0/3689490491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3e8c144890 con 0x7f3e8c0cb070 2023-12-19T18:21:53.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.268+0000 7f3e897fa640 1 -- 172.21.15.27:0/3689490491 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3e8c144890 con 0x7f3e8c0cb070 2023-12-19T18:21:53.269 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:21:53.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.271+0000 7f3e91b0b640 1 -- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e7006a460 msgr2=0x7f3e7006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:53.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.271+0000 7f3e91b0b640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e7006a460 0x7f3e7006c920 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f3e80007a10 tx=0x7f3e80009040 comp rx=0 tx=0).stop 2023-12-19T18:21:53.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.271+0000 7f3e91b0b640 1 -- 172.21.15.27:0/3689490491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e8c0cb070 msgr2=0x7f3e8c0cb4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:53.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.271+0000 7f3e91b0b640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e8c0cb070 0x7f3e8c0cb4f0 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f3e78010810 tx=0x7f3e78010cd0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.271+0000 7f3e91b0b640 1 -- 172.21.15.27:0/3689490491 shutdown_connections 2023-12-19T18:21:53.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.271+0000 7f3e91b0b640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e7006a460 0x7f3e7006c920 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.271+0000 7f3e91b0b640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8c0b0670 0x7f3e8c0b0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.271+0000 7f3e91b0b640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e8c0cb070 0x7f3e8c0cb4f0 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.271+0000 7f3e91b0b640 1 --2- 172.21.15.27:0/3689490491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e7c004d00 0x7f3e8c144350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.271+0000 7f3e91b0b640 1 -- 172.21.15.27:0/3689490491 >> 172.21.15.27:0/3689490491 conn(0x7f3e8c06a060 msgr2=0x7f3e8c06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:53.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.271+0000 7f3e91b0b640 1 -- 172.21.15.27:0/3689490491 shutdown_connections 2023-12-19T18:21:53.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.272+0000 7f3e91b0b640 1 -- 172.21.15.27:0/3689490491 wait complete. 2023-12-19T18:21:53.291 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:21:53.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.528+0000 7f4ec8acc640 1 Processor -- start 2023-12-19T18:21:53.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.529+0000 7f4ec8acc640 1 -- start start 2023-12-19T18:21:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.530+0000 7f4ec8acc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4ec4151c90 0x7f4ec406b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.530+0000 7f4ec8acc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ec415e9d0 0x7f4ec40cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.530+0000 7f4ec8acc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ec415b3e0 con 0x7f4ec4152070 2023-12-19T18:21:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.530+0000 7f4ec8acc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec415eed0 con 0x7f4ec415e9d0 2023-12-19T18:21:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.530+0000 7f4ec8acc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ec415f1b0 con 0x7f4ec4151c90 2023-12-19T18:21:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.531+0000 7f4ebbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ec415e9d0 0x7f4ec40cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.531+0000 7f4ebbfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4ec415e9d0 0x7f4ec40cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36946/0 (socket says 172.21.15.27:36946) 2023-12-19T18:21:53.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.531+0000 7f4ebbfff640 1 -- 172.21.15.27:0/2630523240 learned_addr learned my addr 172.21.15.27:0/2630523240 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:53.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.531+0000 7f4ec37fe640 1 --2- 172.21.15.27:0/2630523240 >> v2:172.21.15.192:3300/0 conn(0x7f4ec4151c90 0x7f4ec406b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:53.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.531+0000 7f4ebbfff640 1 -- 172.21.15.27:0/2630523240 >> v2:172.21.15.192:3300/0 conn(0x7f4ec4151c90 msgr2=0x7f4ec406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:53.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.531+0000 7f4ebbfff640 1 --2- 172.21.15.27:0/2630523240 >> v2:172.21.15.192:3300/0 conn(0x7f4ec4151c90 0x7f4ec406b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.531+0000 7f4ebbfff640 1 -- 172.21.15.27:0/2630523240 >> v1:172.21.15.192:6789/0 conn(0x7f4ec4152070 legacy=0x7f4ec415b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:53.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.531+0000 7f4ebbfff640 1 -- 172.21.15.27:0/2630523240 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec415f490 con 0x7f4ec415e9d0 2023-12-19T18:21:53.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.533+0000 7f4ebbfff640 1 --2- 172.21.15.27:0/2630523240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec415e9d0 0x7f4ec40cee40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009870 tx=0x7f4eac02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504d94ade9835d44 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:53.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.533+0000 7f4ec2ffd640 1 -- 172.21.15.27:0/2630523240 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac0317e0 con 0x7f4ec415e9d0 2023-12-19T18:21:53.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.533+0000 7f4ec2ffd640 1 -- 172.21.15.27:0/2630523240 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4eac031940 con 0x7f4ec415e9d0 2023-12-19T18:21:53.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.533+0000 7f4ec2ffd640 1 -- 172.21.15.27:0/2630523240 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4eac031cd0 con 0x7f4ec415e9d0 2023-12-19T18:21:53.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.534+0000 7f4ec8acc640 1 -- 172.21.15.27:0/2630523240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec415e9d0 msgr2=0x7f4ec40cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:53.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.534+0000 7f4ec8acc640 1 --2- 172.21.15.27:0/2630523240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec415e9d0 0x7f4ec40cee40 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f4eac009870 tx=0x7f4eac02b880 comp rx=0 tx=0).stop 2023-12-19T18:21:53.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.534+0000 7f4ec8acc640 1 -- 172.21.15.27:0/2630523240 shutdown_connections 2023-12-19T18:21:53.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.534+0000 7f4ec8acc640 1 --2- 172.21.15.27:0/2630523240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec415e9d0 0x7f4ec40cee40 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.534+0000 7f4ec8acc640 1 --2- 172.21.15.27:0/2630523240 >> v2:172.21.15.192:3300/0 conn(0x7f4ec4151c90 0x7f4ec406b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.534+0000 7f4ec8acc640 1 -- 172.21.15.27:0/2630523240 >> 172.21.15.27:0/2630523240 conn(0x7f4ec406a060 msgr2=0x7f4ec406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:53.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.534+0000 7f4ec8acc640 1 -- 172.21.15.27:0/2630523240 shutdown_connections 2023-12-19T18:21:53.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.534+0000 7f4ec8acc640 1 -- 172.21.15.27:0/2630523240 wait complete. 2023-12-19T18:21:53.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.536+0000 7f4ec8acc640 1 Processor -- start 2023-12-19T18:21:53.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.536+0000 7f4ec8acc640 1 -- start start 2023-12-19T18:21:53.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.537+0000 7f4ec8acc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec4151c90 0x7f4ec41504d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.537+0000 7f4ec8acc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec4152070 0x7f4ec4154690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.537+0000 7f4ec8acc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec415e9d0 0x7f4ec4154bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.537+0000 7f4ec8acc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40b27c0 con 0x7f4ec415e9d0 2023-12-19T18:21:53.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.537+0000 7f4ec8acc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40b2aa0 con 0x7f4ec4151c90 2023-12-19T18:21:53.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.537+0000 7f4ec8acc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec40b2d80 con 0x7f4ec4152070 2023-12-19T18:21:53.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.537+0000 7f4ec3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec415e9d0 0x7f4ec4154bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:53.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.538+0000 7f4ec3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec415e9d0 0x7f4ec4154bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36954/0 (socket says 172.21.15.27:36954) 2023-12-19T18:21:53.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.538+0000 7f4ec3fff640 1 -- 172.21.15.27:0/3796946046 learned_addr learned my addr 172.21.15.27:0/3796946046 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:53.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.538+0000 7f4ebbfff640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec4152070 0x7f4ec4154690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:53.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.538+0000 7f4ec37fe640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec4151c90 0x7f4ec41504d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:53.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.538+0000 7f4ec3fff640 1 -- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec4152070 msgr2=0x7f4ec4154690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:53.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.538+0000 7f4ec3fff640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec4152070 0x7f4ec4154690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.538+0000 7f4ec3fff640 1 -- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec4151c90 msgr2=0x7f4ec41504d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:53.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.538+0000 7f4ec3fff640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec4151c90 0x7f4ec41504d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.538+0000 7f4ec3fff640 1 -- 172.21.15.27:0/3796946046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eac0094d0 con 0x7f4ec415e9d0 2023-12-19T18:21:53.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.538+0000 7f4ec37fe640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec4151c90 0x7f4ec41504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:53.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.538+0000 7f4ec3fff640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec415e9d0 0x7f4ec4154bd0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00c780 tx=0x7f4ebc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:53.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.539+0000 7f4ec0ff9640 1 -- 172.21.15.27:0/3796946046 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc004b10 con 0x7f4ec415e9d0 2023-12-19T18:21:53.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.539+0000 7f4ec0ff9640 1 -- 172.21.15.27:0/3796946046 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ebc004c70 con 0x7f4ec415e9d0 2023-12-19T18:21:53.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.539+0000 7f4ec0ff9640 1 -- 172.21.15.27:0/3796946046 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ebc00aa90 con 0x7f4ec415e9d0 2023-12-19T18:21:53.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.539+0000 7f4ec8acc640 1 -- 172.21.15.27:0/3796946046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ec40b3060 con 0x7f4ec415e9d0 2023-12-19T18:21:53.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.539+0000 7f4ec8acc640 1 -- 172.21.15.27:0/3796946046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ec4150a10 con 0x7f4ec415e9d0 2023-12-19T18:21:53.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.540+0000 7f4ec8acc640 1 -- 172.21.15.27:0/3796946046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ec404e840 con 0x7f4ec415e9d0 2023-12-19T18:21:53.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.541+0000 7f4ec0ff9640 1 -- 172.21.15.27:0/3796946046 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4ebc00abf0 con 0x7f4ec415e9d0 2023-12-19T18:21:53.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.542+0000 7f4ec0ff9640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ea406a400 0x7f4ea406c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:53.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.543+0000 7f4ec0ff9640 1 -- 172.21.15.27:0/3796946046 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4ebc088a60 con 0x7f4ec415e9d0 2023-12-19T18:21:53.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.543+0000 7f4ec37fe640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ea406a400 0x7f4ea406c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:53.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.544+0000 7f4ec37fe640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ea406a400 0x7f4ea406c8c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4006fd0 tx=0x7f4eb4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:53.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.547+0000 7f4ec0ff9640 1 -- 172.21.15.27:0/3796946046 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4ebc054050 con 0x7f4ec415e9d0 2023-12-19T18:21:53.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.743+0000 7f4ec8acc640 1 -- 172.21.15.27:0/3796946046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4ec40ccb70 con 0x7f4ec415e9d0 2023-12-19T18:21:53.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.744+0000 7f4ec0ff9640 1 -- 172.21.15.27:0/3796946046 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4ebc057670 con 0x7f4ec415e9d0 2023-12-19T18:21:53.745 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:21:53.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.747+0000 7f4ec8acc640 1 -- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ea406a400 msgr2=0x7f4ea406c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:53.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.747+0000 7f4ec8acc640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ea406a400 0x7f4ea406c8c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f4eb4006fd0 tx=0x7f4eb4008040 comp rx=0 tx=0).stop 2023-12-19T18:21:53.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.747+0000 7f4ec8acc640 1 -- 172.21.15.27:0/3796946046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec415e9d0 msgr2=0x7f4ec4154bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:53.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.747+0000 7f4ec8acc640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec415e9d0 0x7f4ec4154bd0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f4ebc00c780 tx=0x7f4ebc00cc40 comp rx=0 tx=0).stop 2023-12-19T18:21:53.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.747+0000 7f4ec8acc640 1 -- 172.21.15.27:0/3796946046 shutdown_connections 2023-12-19T18:21:53.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.747+0000 7f4ec8acc640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ea406a400 0x7f4ea406c8c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.747+0000 7f4ec8acc640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ec415e9d0 0x7f4ec4154bd0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.747+0000 7f4ec8acc640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4ec4152070 0x7f4ec4154690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.747+0000 7f4ec8acc640 1 --2- 172.21.15.27:0/3796946046 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4ec4151c90 0x7f4ec41504d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:53.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.747+0000 7f4ec8acc640 1 -- 172.21.15.27:0/3796946046 >> 172.21.15.27:0/3796946046 conn(0x7f4ec406a060 msgr2=0x7f4ec406a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:53.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.747+0000 7f4ec8acc640 1 -- 172.21.15.27:0/3796946046 shutdown_connections 2023-12-19T18:21:53.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:53.747+0000 7f4ec8acc640 1 -- 172.21.15.27:0/3796946046 wait complete. 2023-12-19T18:21:53.767 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:21:54.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.003+0000 7f7a3c910640 1 Processor -- start 2023-12-19T18:21:54.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.003+0000 7f7a3c910640 1 -- start start 2023-12-19T18:21:54.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.005+0000 7f7a3c910640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7a3806c850 0x7f7a38164860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.005+0000 7f7a3c910640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a380b27d0 con 0x7f7a38143450 2023-12-19T18:21:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.005+0000 7f7a3c910640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a380cce00 con 0x7f7a380cf600 2023-12-19T18:21:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.005+0000 7f7a3c910640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a380b33e0 con 0x7f7a3806c850 2023-12-19T18:21:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.005+0000 7f7a36d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7a38143450 0x7f7a380ceef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55738/0 (socket says 172.21.15.27:55738) 2023-12-19T18:21:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.005+0000 7f7a36d9d640 1 -- 172.21.15.27:0/554522891 learned_addr learned my addr 172.21.15.27:0/554522891 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.005+0000 7f7a3759e640 1 --2- 172.21.15.27:0/554522891 >> v2:172.21.15.192:3300/0 conn(0x7f7a3806c850 0x7f7a38164860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.005+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142140050 0 0) 0x7f7a380b27d0 con 0x7f7a38143450 2023-12-19T18:21:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.005+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a24005730 con 0x7f7a38143450 2023-12-19T18:21:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3738920441 0 0) 0x7f7a24005730 con 0x7f7a38143450 2023-12-19T18:21:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 >> v2:172.21.15.192:3300/0 conn(0x7f7a3806c850 msgr2=0x7f7a38164860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a3659c640 1 --2- 172.21.15.27:0/554522891 >> v2:172.21.15.192:3300/0 conn(0x7f7a3806c850 0x7f7a38164860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 >> v1:172.21.15.192:6789/0 conn(0x7f7a380cf600 legacy=0x7f7a3806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a380b2a40 con 0x7f7a38143450 2023-12-19T18:21:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f7a280019f0 con 0x7f7a38143450 2023-12-19T18:21:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 >> v1:172.21.15.27:6789/0 conn(0x7f7a38143450 legacy=0x7f7a380ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a3659c640 1 --2- 172.21.15.27:0/554522891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a24005bd0 0x7f7a24005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a3659c640 1 --2- 172.21.15.27:0/554522891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a24006530 0x7f7a240069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a3659c640 1 --2- 172.21.15.27:0/554522891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a24007270 0x7f7a2400ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380b33e0 con 0x7f7a24006530 2023-12-19T18:21:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a24010580 con 0x7f7a24005bd0 2023-12-19T18:21:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a240106f0 con 0x7f7a24007270 2023-12-19T18:21:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.006+0000 7f7a2e59c640 1 --2- 172.21.15.27:0/554522891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a24006530 0x7f7a240069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.007+0000 7f7a36d9d640 1 --2- 172.21.15.27:0/554522891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a24005bd0 0x7f7a24005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.007+0000 7f7a3759e640 1 --2- 172.21.15.27:0/554522891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a24007270 0x7f7a2400ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.007+0000 7f7a2e59c640 1 -- 172.21.15.27:0/554522891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a24007270 msgr2=0x7f7a2400ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.007+0000 7f7a2e59c640 1 --2- 172.21.15.27:0/554522891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a24007270 0x7f7a2400ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.007+0000 7f7a2e59c640 1 -- 172.21.15.27:0/554522891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a24005bd0 msgr2=0x7f7a24005ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:54.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.007+0000 7f7a2e59c640 1 --2- 172.21.15.27:0/554522891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a24005bd0 0x7f7a24005ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.007+0000 7f7a2e59c640 1 -- 172.21.15.27:0/554522891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a24005730 con 0x7f7a24006530 2023-12-19T18:21:54.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.007+0000 7f7a3759e640 1 -- 172.21.15.27:0/554522891 reap_dead start 2023-12-19T18:21:54.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.008+0000 7f7a2e59c640 1 --2- 172.21.15.27:0/554522891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a24006530 0x7f7a240069d0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7a20001750 tx=0x7f7a24007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec8925528dc7edb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:54.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.008+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a200052d0 con 0x7f7a24006530 2023-12-19T18:21:54.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.008+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2002e050 con 0x7f7a24006530 2023-12-19T18:21:54.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.008+0000 7f7a3659c640 1 -- 172.21.15.27:0/554522891 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20003830 con 0x7f7a24006530 2023-12-19T18:21:54.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.009+0000 7f7a3c910640 1 -- 172.21.15.27:0/554522891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a24006530 msgr2=0x7f7a240069d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:54.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.009+0000 7f7a3c910640 1 --2- 172.21.15.27:0/554522891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a24006530 0x7f7a240069d0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f7a20001750 tx=0x7f7a24007a00 comp rx=0 tx=0).stop 2023-12-19T18:21:54.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.009+0000 7f7a3c910640 1 -- 172.21.15.27:0/554522891 shutdown_connections 2023-12-19T18:21:54.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.009+0000 7f7a3c910640 1 --2- 172.21.15.27:0/554522891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a24006530 0x7f7a240069d0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.009+0000 7f7a3c910640 1 -- 172.21.15.27:0/554522891 >> 172.21.15.27:0/554522891 conn(0x7f7a3806a060 msgr2=0x7f7a3806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:54.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.009+0000 7f7a3c910640 1 -- 172.21.15.27:0/554522891 shutdown_connections 2023-12-19T18:21:54.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.009+0000 7f7a3c910640 1 -- 172.21.15.27:0/554522891 wait complete. 2023-12-19T18:21:54.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.011+0000 7f7a3c910640 1 Processor -- start 2023-12-19T18:21:54.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.011+0000 7f7a3c910640 1 -- start start 2023-12-19T18:21:54.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.012+0000 7f7a3c910640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a24006530 0x7f7a3813a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.012+0000 7f7a3c910640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a38142210 0x7f7a38142690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.012+0000 7f7a3c910640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a38159950 0x7f7a38159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.012+0000 7f7a3c910640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a38142bd0 con 0x7f7a38142210 2023-12-19T18:21:54.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.012+0000 7f7a3c910640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3815a6b0 con 0x7f7a24006530 2023-12-19T18:21:54.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.012+0000 7f7a3c910640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3815a990 con 0x7f7a38159950 2023-12-19T18:21:54.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.012+0000 7f7a2e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a38142210 0x7f7a38142690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.012+0000 7f7a2e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a38142210 0x7f7a38142690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36980/0 (socket says 172.21.15.27:36980) 2023-12-19T18:21:54.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.012+0000 7f7a2e59c640 1 -- 172.21.15.27:0/825699861 learned_addr learned my addr 172.21.15.27:0/825699861 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:54.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a36d9d640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a24006530 0x7f7a3813a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a3759e640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a38159950 0x7f7a38159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a2e59c640 1 -- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a38159950 msgr2=0x7f7a38159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:54.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a2e59c640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a38159950 0x7f7a38159de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a2e59c640 1 -- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a24006530 msgr2=0x7f7a3813a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:54.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a2e59c640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a24006530 0x7f7a3813a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a2e59c640 1 -- 172.21.15.27:0/825699861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3815ac70 con 0x7f7a38142210 2023-12-19T18:21:54.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a3759e640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a38159950 0x7f7a38159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:54.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a36d9d640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a24006530 0x7f7a3813a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:54.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a2e59c640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a38142210 0x7f7a38142690 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7a20000c00 tx=0x7f7a20001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:54.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a2ffff640 1 -- 172.21.15.27:0/825699861 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a200076c0 con 0x7f7a38142210 2023-12-19T18:21:54.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a2ffff640 1 -- 172.21.15.27:0/825699861 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a20007a30 con 0x7f7a38142210 2023-12-19T18:21:54.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a3c910640 1 -- 172.21.15.27:0/825699861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a380a9b50 con 0x7f7a38142210 2023-12-19T18:21:54.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.013+0000 7f7a2ffff640 1 -- 172.21.15.27:0/825699861 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a20006be0 con 0x7f7a38142210 2023-12-19T18:21:54.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.014+0000 7f7a3c910640 1 -- 172.21.15.27:0/825699861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a3806f320 con 0x7f7a38142210 2023-12-19T18:21:54.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.015+0000 7f7a3c910640 1 -- 172.21.15.27:0/825699861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a380a99b0 con 0x7f7a38142210 2023-12-19T18:21:54.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.016+0000 7f7a2ffff640 1 -- 172.21.15.27:0/825699861 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7a20006d40 con 0x7f7a38142210 2023-12-19T18:21:54.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.016+0000 7f7a2ffff640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a1806a460 0x7f7a1806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.017+0000 7f7a2ffff640 1 -- 172.21.15.27:0/825699861 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a200aa5f0 con 0x7f7a38142210 2023-12-19T18:21:54.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.017+0000 7f7a36d9d640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a1806a460 0x7f7a1806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.018+0000 7f7a36d9d640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a1806a460 0x7f7a1806c920 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7a28002230 tx=0x7f7a28001250 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:54.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.021+0000 7f7a2ffff640 1 -- 172.21.15.27:0/825699861 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7a380a99b0 con 0x7f7a38142210 2023-12-19T18:21:54.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.214+0000 7f7a3c910640 1 -- 172.21.15.27:0/825699861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7a380a99b0 con 0x7f7a38142210 2023-12-19T18:21:54.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.214+0000 7f7a2ffff640 1 -- 172.21.15.27:0/825699861 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7a380a99b0 con 0x7f7a38142210 2023-12-19T18:21:54.215 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:21:54.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.217+0000 7f7a3c910640 1 -- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a1806a460 msgr2=0x7f7a1806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:54.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.217+0000 7f7a3c910640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a1806a460 0x7f7a1806c920 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7a28002230 tx=0x7f7a28001250 comp rx=0 tx=0).stop 2023-12-19T18:21:54.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.217+0000 7f7a3c910640 1 -- 172.21.15.27:0/825699861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a38142210 msgr2=0x7f7a38142690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:54.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.217+0000 7f7a3c910640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a38142210 0x7f7a38142690 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f7a20000c00 tx=0x7f7a20001120 comp rx=0 tx=0).stop 2023-12-19T18:21:54.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.217+0000 7f7a3c910640 1 -- 172.21.15.27:0/825699861 shutdown_connections 2023-12-19T18:21:54.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.217+0000 7f7a3c910640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a1806a460 0x7f7a1806c920 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.217+0000 7f7a3c910640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a38159950 0x7f7a38159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.217+0000 7f7a3c910640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a38142210 0x7f7a38142690 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.217+0000 7f7a3c910640 1 --2- 172.21.15.27:0/825699861 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a24006530 0x7f7a3813a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.217+0000 7f7a3c910640 1 -- 172.21.15.27:0/825699861 >> 172.21.15.27:0/825699861 conn(0x7f7a3806a060 msgr2=0x7f7a381580d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:54.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.218+0000 7f7a3c910640 1 -- 172.21.15.27:0/825699861 shutdown_connections 2023-12-19T18:21:54.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.218+0000 7f7a3c910640 1 -- 172.21.15.27:0/825699861 wait complete. 2023-12-19T18:21:54.236 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:21:54.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.473+0000 7ff8db3c9640 1 Processor -- start 2023-12-19T18:21:54.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.474+0000 7ff8db3c9640 1 -- start start 2023-12-19T18:21:54.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.475+0000 7ff8db3c9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff8d40c9770 0x7ff8d406ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.475+0000 7ff8db3c9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff8d4070470 0x7ff8d4070900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.475+0000 7ff8db3c9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d40cc720 con 0x7ff8d406cec0 2023-12-19T18:21:54.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.475+0000 7ff8db3c9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d415a330 con 0x7ff8d4070470 2023-12-19T18:21:54.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.475+0000 7ff8db3c9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d40b1a00 con 0x7ff8d40c9770 2023-12-19T18:21:54.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.476+0000 7ff8d9165640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff8d406cec0 0x7ff8d406d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55740/0 (socket says 172.21.15.27:55740) 2023-12-19T18:21:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.476+0000 7ff8d9165640 1 -- 172.21.15.27:0/3867140088 learned_addr learned my addr 172.21.15.27:0/3867140088 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.476+0000 7ff8d9966640 1 --2- 172.21.15.27:0/3867140088 >> v2:172.21.15.27:3300/0 conn(0x7ff8d4070470 0x7ff8d4070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.476+0000 7ff8d8964640 1 --2- 172.21.15.27:0/3867140088 >> v2:172.21.15.192:3300/0 conn(0x7ff8d40c9770 0x7ff8d406ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.476+0000 7ff8c7fff640 1 -- 172.21.15.27:0/3867140088 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 92763342 0 0) 0x7ff8d40cc720 con 0x7ff8d406cec0 2023-12-19T18:21:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.476+0000 7ff8c7fff640 1 -- 172.21.15.27:0/3867140088 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c0003520 con 0x7ff8d406cec0 2023-12-19T18:21:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.476+0000 7ff8d9966640 1 -- 172.21.15.27:0/3867140088 >> v2:172.21.15.192:3300/0 conn(0x7ff8d40c9770 msgr2=0x7ff8d406ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.476+0000 7ff8d9966640 1 --2- 172.21.15.27:0/3867140088 >> v2:172.21.15.192:3300/0 conn(0x7ff8d40c9770 0x7ff8d406ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.476+0000 7ff8d9966640 1 -- 172.21.15.27:0/3867140088 >> v1:172.21.15.27:6789/0 conn(0x7ff8d406cec0 legacy=0x7ff8d406d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:54.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.476+0000 7ff8d9966640 1 -- 172.21.15.27:0/3867140088 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d4071640 con 0x7ff8d4070470 2023-12-19T18:21:54.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.478+0000 7ff8d9966640 1 --2- 172.21.15.27:0/3867140088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8d4070470 0x7ff8d4070900 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40cb530 tx=0x7ff8cc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43b1a1a37710afcc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:54.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.478+0000 7ff8c7fff640 1 -- 172.21.15.27:0/3867140088 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc02cc80 con 0x7ff8d4070470 2023-12-19T18:21:54.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.478+0000 7ff8c7fff640 1 -- 172.21.15.27:0/3867140088 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc02cde0 con 0x7ff8d4070470 2023-12-19T18:21:54.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.478+0000 7ff8c7fff640 1 -- 172.21.15.27:0/3867140088 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc034640 con 0x7ff8d4070470 2023-12-19T18:21:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.479+0000 7ff8db3c9640 1 -- 172.21.15.27:0/3867140088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8d4070470 msgr2=0x7ff8d4070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.479+0000 7ff8db3c9640 1 --2- 172.21.15.27:0/3867140088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8d4070470 0x7ff8d4070900 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7ff8d40cb530 tx=0x7ff8cc02bed0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.479+0000 7ff8db3c9640 1 -- 172.21.15.27:0/3867140088 shutdown_connections 2023-12-19T18:21:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.479+0000 7ff8db3c9640 1 --2- 172.21.15.27:0/3867140088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8d4070470 0x7ff8d4070900 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.479+0000 7ff8db3c9640 1 --2- 172.21.15.27:0/3867140088 >> v2:172.21.15.192:3300/0 conn(0x7ff8d40c9770 0x7ff8d406ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.479+0000 7ff8db3c9640 1 -- 172.21.15.27:0/3867140088 >> 172.21.15.27:0/3867140088 conn(0x7ff8d40ceb50 msgr2=0x7ff8d40cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.479+0000 7ff8db3c9640 1 -- 172.21.15.27:0/3867140088 shutdown_connections 2023-12-19T18:21:54.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.479+0000 7ff8db3c9640 1 -- 172.21.15.27:0/3867140088 wait complete. 2023-12-19T18:21:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.481+0000 7ff8db3c9640 1 Processor -- start 2023-12-19T18:21:54.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.481+0000 7ff8db3c9640 1 -- start start 2023-12-19T18:21:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.482+0000 7ff8db3c9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8d406cec0 0x7ff8d4158970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.482+0000 7ff8db3c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8d4070470 0x7ff8d4150000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.482+0000 7ff8db3c9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8d40c9770 0x7ff8d40c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.482+0000 7ff8db3c9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d4158eb0 con 0x7ff8d4070470 2023-12-19T18:21:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.482+0000 7ff8db3c9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d41578d0 con 0x7ff8d406cec0 2023-12-19T18:21:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.482+0000 7ff8db3c9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d4157bb0 con 0x7ff8d40c9770 2023-12-19T18:21:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.482+0000 7ff8d8964640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8d4070470 0x7ff8d4150000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.482+0000 7ff8d8964640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8d4070470 0x7ff8d4150000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37002/0 (socket says 172.21.15.27:37002) 2023-12-19T18:21:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.482+0000 7ff8d8964640 1 -- 172.21.15.27:0/1835906550 learned_addr learned my addr 172.21.15.27:0/1835906550 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.483+0000 7ff8d8964640 1 -- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8d40c9770 msgr2=0x7ff8d40c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:54.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.483+0000 7ff8d9966640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8d40c9770 0x7ff8d40c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.483+0000 7ff8d8964640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8d40c9770 0x7ff8d40c66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.483+0000 7ff8d9165640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8d406cec0 0x7ff8d4158970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.483+0000 7ff8d8964640 1 -- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8d406cec0 msgr2=0x7ff8d4158970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:54.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.483+0000 7ff8d8964640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8d406cec0 0x7ff8d4158970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.483+0000 7ff8d8964640 1 -- 172.21.15.27:0/1835906550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc009a70 con 0x7ff8d4070470 2023-12-19T18:21:54.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.483+0000 7ff8d8964640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8d4070470 0x7ff8d4150000 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000d5f0 tx=0x7ff8d000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:54.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.483+0000 7ff8c5ffb640 1 -- 172.21.15.27:0/1835906550 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d00125f0 con 0x7ff8d4070470 2023-12-19T18:21:54.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.484+0000 7ff8c5ffb640 1 -- 172.21.15.27:0/1835906550 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d0012750 con 0x7ff8d4070470 2023-12-19T18:21:54.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.484+0000 7ff8db3c9640 1 -- 172.21.15.27:0/1835906550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d4157e90 con 0x7ff8d4070470 2023-12-19T18:21:54.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.484+0000 7ff8c5ffb640 1 -- 172.21.15.27:0/1835906550 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d0012b00 con 0x7ff8d4070470 2023-12-19T18:21:54.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.484+0000 7ff8db3c9640 1 -- 172.21.15.27:0/1835906550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d4158450 con 0x7ff8d4070470 2023-12-19T18:21:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.485+0000 7ff8db3c9640 1 -- 172.21.15.27:0/1835906550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d41585e0 con 0x7ff8d4070470 2023-12-19T18:21:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.486+0000 7ff8c5ffb640 1 -- 172.21.15.27:0/1835906550 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff8d0012c60 con 0x7ff8d4070470 2023-12-19T18:21:54.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.487+0000 7ff8c5ffb640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff8b806a460 0x7ff8b806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:54.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.487+0000 7ff8c5ffb640 1 -- 172.21.15.27:0/1835906550 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff8d0088d90 con 0x7ff8d4070470 2023-12-19T18:21:54.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.488+0000 7ff8d9165640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff8b806a460 0x7ff8b806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:54.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.489+0000 7ff8d9165640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff8b806a460 0x7ff8b806c920 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8000f60 tx=0x7ff8c8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:54.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.492+0000 7ff8c5ffb640 1 -- 172.21.15.27:0/1835906550 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff8d0054380 con 0x7ff8d4070470 2023-12-19T18:21:54.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.699+0000 7ff8db3c9640 1 -- 172.21.15.27:0/1835906550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff8d40ca980 con 0x7ff8d4070470 2023-12-19T18:21:54.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.700+0000 7ff8c5ffb640 1 -- 172.21.15.27:0/1835906550 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v123) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff8d00579a0 con 0x7ff8d4070470 2023-12-19T18:21:54.700 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:21:54.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.703+0000 7ff8db3c9640 1 -- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff8b806a460 msgr2=0x7ff8b806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.703+0000 7ff8db3c9640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff8b806a460 0x7ff8b806c920 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ff8c8000f60 tx=0x7ff8c8009040 comp rx=0 tx=0).stop 2023-12-19T18:21:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.703+0000 7ff8db3c9640 1 -- 172.21.15.27:0/1835906550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8d4070470 msgr2=0x7ff8d4150000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.703+0000 7ff8db3c9640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8d4070470 0x7ff8d4150000 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7ff8d000d5f0 tx=0x7ff8d000dab0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.703+0000 7ff8db3c9640 1 -- 172.21.15.27:0/1835906550 shutdown_connections 2023-12-19T18:21:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.703+0000 7ff8db3c9640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff8b806a460 0x7ff8b806c920 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.703+0000 7ff8db3c9640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8d40c9770 0x7ff8d40c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.703+0000 7ff8db3c9640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8d4070470 0x7ff8d4150000 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.703+0000 7ff8db3c9640 1 --2- 172.21.15.27:0/1835906550 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8d406cec0 0x7ff8d4158970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.703+0000 7ff8db3c9640 1 -- 172.21.15.27:0/1835906550 >> 172.21.15.27:0/1835906550 conn(0x7ff8d40ceb50 msgr2=0x7ff8d4070850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.703+0000 7ff8db3c9640 1 -- 172.21.15.27:0/1835906550 shutdown_connections 2023-12-19T18:21:54.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:54.703+0000 7ff8db3c9640 1 -- 172.21.15.27:0/1835906550 wait complete. 2023-12-19T18:21:54.721 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-19T18:21:54.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:54.721 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-19T18:21:54.822 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[89274]: fuse finished with error 0 and tester_r 0 2023-12-19T18:21:54.828 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:21:54.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:54.829 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-19T18:21:54.987 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:21:54.987 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:21:55.053 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:21:55.053 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:21:55.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:55.054 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-19T18:21:55.112 INFO:teuthology.orchestra.run.smithi027.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-19T18:21:55.113 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:21:55.114 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:21:55.114 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:21:55.173 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:21:55.174 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:21:55.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:21:55.174 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-19T18:21:55.228 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-19T18:21:55.229 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:21:55.229 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:21:55.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.472+0000 7f31c2e15640 1 Processor -- start 2023-12-19T18:21:55.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.472+0000 7f31c2e15640 1 -- start start 2023-12-19T18:21:55.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.474+0000 7f31c2e15640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f31bc15ee30 0x7f31bc0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:55.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.474+0000 7f31c2e15640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31bc0afeb0 0x7f31bc0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:55.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.474+0000 7f31c2e15640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31bc0cb350 con 0x7f31bc1553a0 2023-12-19T18:21:55.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.474+0000 7f31c2e15640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31bc0cbcf0 con 0x7f31bc15ee30 2023-12-19T18:21:55.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.474+0000 7f31c2e15640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0cb630 con 0x7f31bc0afeb0 2023-12-19T18:21:55.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.474+0000 7f31c1e13640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f31bc1553a0 0x7f31bc15e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55748/0 (socket says 172.21.15.27:55748) 2023-12-19T18:21:55.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.474+0000 7f31c1e13640 1 -- 172.21.15.27:0/1594381942 learned_addr learned my addr 172.21.15.27:0/1594381942 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:55.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.474+0000 7f31c1612640 1 --2- 172.21.15.27:0/1594381942 >> v2:172.21.15.27:3300/0 conn(0x7f31bc15ee30 0x7f31bc0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:55.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.475+0000 7f31c0e11640 1 -- 172.21.15.27:0/1594381942 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1277460972 0 0) 0x7f31bc0cb350 con 0x7f31bc1553a0 2023-12-19T18:21:55.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.475+0000 7f31c0e11640 1 -- 172.21.15.27:0/1594381942 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31a8003520 con 0x7f31bc1553a0 2023-12-19T18:21:55.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.475+0000 7f31c2614640 1 --2- 172.21.15.27:0/1594381942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31bc0afeb0 0x7f31bc0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:55.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.475+0000 7f31c1612640 1 -- 172.21.15.27:0/1594381942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31bc0afeb0 msgr2=0x7f31bc0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:55.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.475+0000 7f31c1612640 1 --2- 172.21.15.27:0/1594381942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31bc0afeb0 0x7f31bc0c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.475+0000 7f31c1612640 1 -- 172.21.15.27:0/1594381942 >> v1:172.21.15.27:6789/0 conn(0x7f31bc1553a0 legacy=0x7f31bc15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:55.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.475+0000 7f31c1612640 1 -- 172.21.15.27:0/1594381942 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc0cb910 con 0x7f31bc15ee30 2023-12-19T18:21:55.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.475+0000 7f31c2614640 1 --2- 172.21.15.27:0/1594381942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31bc0afeb0 0x7f31bc0c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:55.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.476+0000 7f31c1612640 1 --2- 172.21.15.27:0/1594381942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc15ee30 0x7f31bc0c50a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0097a0 tx=0x7f31ac02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86a4ce2ec127ba98 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:55.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.476+0000 7f31c0e11640 1 -- 172.21.15.27:0/1594381942 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac031780 con 0x7f31bc15ee30 2023-12-19T18:21:55.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.476+0000 7f31c0e11640 1 -- 172.21.15.27:0/1594381942 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31ac0318e0 con 0x7f31bc15ee30 2023-12-19T18:21:55.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.476+0000 7f31c0e11640 1 -- 172.21.15.27:0/1594381942 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31ac031c70 con 0x7f31bc15ee30 2023-12-19T18:21:55.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.477+0000 7f31c2e15640 1 -- 172.21.15.27:0/1594381942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc15ee30 msgr2=0x7f31bc0c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:55.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.477+0000 7f31c2e15640 1 --2- 172.21.15.27:0/1594381942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc15ee30 0x7f31bc0c50a0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f31ac0097a0 tx=0x7f31ac02b820 comp rx=0 tx=0).stop 2023-12-19T18:21:55.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.477+0000 7f31c2e15640 1 -- 172.21.15.27:0/1594381942 shutdown_connections 2023-12-19T18:21:55.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.477+0000 7f31c2e15640 1 --2- 172.21.15.27:0/1594381942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31bc0afeb0 0x7f31bc0c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.477+0000 7f31c2e15640 1 --2- 172.21.15.27:0/1594381942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc15ee30 0x7f31bc0c50a0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.477+0000 7f31c2e15640 1 -- 172.21.15.27:0/1594381942 >> 172.21.15.27:0/1594381942 conn(0x7f31bc069c70 msgr2=0x7f31bc06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:55.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.477+0000 7f31c2e15640 1 -- 172.21.15.27:0/1594381942 shutdown_connections 2023-12-19T18:21:55.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.477+0000 7f31c2e15640 1 -- 172.21.15.27:0/1594381942 wait complete. 2023-12-19T18:21:55.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.479+0000 7f31c2e15640 1 Processor -- start 2023-12-19T18:21:55.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.479+0000 7f31c2e15640 1 -- start start 2023-12-19T18:21:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.480+0000 7f31c2e15640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc0afeb0 0x7f31bc06fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.480+0000 7f31c2e15640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc1553a0 0x7f31bc159d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.480+0000 7f31c2e15640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31bc15ee30 0x7f31bc15a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.480+0000 7f31c2e15640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0cdf60 con 0x7f31bc0afeb0 2023-12-19T18:21:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.480+0000 7f31c2e15640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc1f6350 con 0x7f31bc1553a0 2023-12-19T18:21:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.480+0000 7f31c2e15640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc0ac8c0 con 0x7f31bc15ee30 2023-12-19T18:21:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.480+0000 7f31c1e13640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc0afeb0 0x7f31bc06fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.481+0000 7f31c1e13640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc0afeb0 0x7f31bc06fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37016/0 (socket says 172.21.15.27:37016) 2023-12-19T18:21:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.481+0000 7f31c1e13640 1 -- 172.21.15.27:0/925135016 learned_addr learned my addr 172.21.15.27:0/925135016 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:55.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.481+0000 7f31c1e13640 1 -- 172.21.15.27:0/925135016 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31bc15ee30 msgr2=0x7f31bc15a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:55.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.481+0000 7f31c1612640 1 --2- 172.21.15.27:0/925135016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc1553a0 0x7f31bc159d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:55.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.481+0000 7f31c1e13640 1 --2- 172.21.15.27:0/925135016 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31bc15ee30 0x7f31bc15a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.481+0000 7f31c1e13640 1 -- 172.21.15.27:0/925135016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc1553a0 msgr2=0x7f31bc159d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:55.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.481+0000 7f31c1e13640 1 --2- 172.21.15.27:0/925135016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc1553a0 0x7f31bc159d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.481+0000 7f31c1e13640 1 -- 172.21.15.27:0/925135016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ac009400 con 0x7f31bc0afeb0 2023-12-19T18:21:55.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.481+0000 7f31c1e13640 1 --2- 172.21.15.27:0/925135016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc0afeb0 0x7f31bc06fae0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f31b800c500 tx=0x7f31b800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:55.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.482+0000 7f31b27fc640 1 -- 172.21.15.27:0/925135016 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b8005ce0 con 0x7f31bc0afeb0 2023-12-19T18:21:55.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.482+0000 7f31b27fc640 1 -- 172.21.15.27:0/925135016 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31b8005e40 con 0x7f31bc0afeb0 2023-12-19T18:21:55.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.482+0000 7f31b27fc640 1 -- 172.21.15.27:0/925135016 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31b8007330 con 0x7f31bc0afeb0 2023-12-19T18:21:55.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.482+0000 7f31c2e15640 1 -- 172.21.15.27:0/925135016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc0acba0 con 0x7f31bc0afeb0 2023-12-19T18:21:55.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.482+0000 7f31c2e15640 1 -- 172.21.15.27:0/925135016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc1f5f70 con 0x7f31bc0afeb0 2023-12-19T18:21:55.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.483+0000 7f31c2e15640 1 -- 172.21.15.27:0/925135016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31bc04e840 con 0x7f31bc0afeb0 2023-12-19T18:21:55.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.484+0000 7f31b27fc640 1 -- 172.21.15.27:0/925135016 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f31b8007550 con 0x7f31bc0afeb0 2023-12-19T18:21:55.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.485+0000 7f31b27fc640 1 --2- 172.21.15.27:0/925135016 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f31a006a430 0x7f31a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:55.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.485+0000 7f31b27fc640 1 -- 172.21.15.27:0/925135016 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f31b8087fe0 con 0x7f31bc0afeb0 2023-12-19T18:21:55.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.486+0000 7f31c1612640 1 --2- 172.21.15.27:0/925135016 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f31a006a430 0x7f31a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:55.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.487+0000 7f31c1612640 1 --2- 172.21.15.27:0/925135016 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f31a006a430 0x7f31a006c8f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f31ac036000 tx=0x7f31ac03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:55.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.491+0000 7f31b27fc640 1 -- 172.21.15.27:0/925135016 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f31b8053680 con 0x7f31bc0afeb0 2023-12-19T18:21:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.726+0000 7f31c2e15640 1 -- 172.21.15.27:0/925135016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f31bc0cdb80 con 0x7f31bc0afeb0 2023-12-19T18:21:55.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.727+0000 7f31b27fc640 1 -- 172.21.15.27:0/925135016 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f31b8017090 con 0x7f31bc0afeb0 2023-12-19T18:21:55.727 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:55.728 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2779436922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2779436922},{"type":"v1","addr":"172.21.15.192:6836","nonce":2779436922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3121062779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3121062779},{"type":"v1","addr":"172.21.15.192:6837","nonce":3121062779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5357,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2032553869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2032553869},{"type":"v1","addr":"172.21.15.27:6835","nonce":2032553869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:21:08.281615+0000","modified":"2023-12-19T18:21:11.800655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/252970177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":252970177},{"type":"v1","addr":"172.21.15.27:6834","nonce":252970177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T18:21:55.728 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 46 2023-12-19T18:21:55.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.731+0000 7f31c2e15640 1 -- 172.21.15.27:0/925135016 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f31a006a430 msgr2=0x7f31a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:55.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.731+0000 7f31c2e15640 1 --2- 172.21.15.27:0/925135016 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f31a006a430 0x7f31a006c8f0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f31ac036000 tx=0x7f31ac03d040 comp rx=0 tx=0).stop 2023-12-19T18:21:55.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.731+0000 7f31c2e15640 1 -- 172.21.15.27:0/925135016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc0afeb0 msgr2=0x7f31bc06fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:55.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.731+0000 7f31c2e15640 1 --2- 172.21.15.27:0/925135016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc0afeb0 0x7f31bc06fae0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7f31b800c500 tx=0x7f31b800c9c0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.731+0000 7f31c2e15640 1 -- 172.21.15.27:0/925135016 shutdown_connections 2023-12-19T18:21:55.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.731+0000 7f31c2e15640 1 --2- 172.21.15.27:0/925135016 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f31a006a430 0x7f31a006c8f0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.731+0000 7f31c2e15640 1 --2- 172.21.15.27:0/925135016 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31bc15ee30 0x7f31bc15a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.731+0000 7f31c2e15640 1 --2- 172.21.15.27:0/925135016 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31bc1553a0 0x7f31bc159d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.731+0000 7f31c2e15640 1 --2- 172.21.15.27:0/925135016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31bc0afeb0 0x7f31bc06fae0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.731+0000 7f31c2e15640 1 -- 172.21.15.27:0/925135016 >> 172.21.15.27:0/925135016 conn(0x7f31bc069c70 msgr2=0x7f31bc0c4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:55.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.731+0000 7f31c2e15640 1 -- 172.21.15.27:0/925135016 shutdown_connections 2023-12-19T18:21:55.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.731+0000 7f31c2e15640 1 -- 172.21.15.27:0/925135016 wait complete. 2023-12-19T18:21:55.753 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:21:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.990+0000 7fd697fff640 1 Processor -- start 2023-12-19T18:21:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.990+0000 7fd697fff640 1 -- start start 2023-12-19T18:21:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.992+0000 7fd697fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd69815c7f0 0x7fd6980c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.992+0000 7fd697fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd6980cab50 0x7fd6980c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.992+0000 7fd697fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6980ca310 con 0x7fd6980cc510 2023-12-19T18:21:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.992+0000 7fd697fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd69815cbd0 con 0x7fd69815c7f0 2023-12-19T18:21:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.992+0000 7fd697fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6980ae490 con 0x7fd6980cab50 2023-12-19T18:21:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.992+0000 7fd696ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd69815c7f0 0x7fd6980c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.992+0000 7fd6977fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd6980cc510 0x7fd6980af250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55760/0 (socket says 172.21.15.27:55760) 2023-12-19T18:21:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.992+0000 7fd6977fe640 1 -- 172.21.15.27:0/2074031867 learned_addr learned my addr 172.21.15.27:0/2074031867 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.993+0000 7fd68ffff640 1 --2- 172.21.15.27:0/2074031867 >> v2:172.21.15.192:3300/0 conn(0x7fd6980cab50 0x7fd6980c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.993+0000 7fd696ffd640 1 -- 172.21.15.27:0/2074031867 >> v2:172.21.15.192:3300/0 conn(0x7fd6980cab50 msgr2=0x7fd6980c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.993+0000 7fd696ffd640 1 --2- 172.21.15.27:0/2074031867 >> v2:172.21.15.192:3300/0 conn(0x7fd6980cab50 0x7fd6980c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.993+0000 7fd696ffd640 1 -- 172.21.15.27:0/2074031867 >> v1:172.21.15.27:6789/0 conn(0x7fd6980cc510 legacy=0x7fd6980af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:55.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.993+0000 7fd696ffd640 1 -- 172.21.15.27:0/2074031867 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6980ae770 con 0x7fd69815c7f0 2023-12-19T18:21:55.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.994+0000 7fd696ffd640 1 --2- 172.21.15.27:0/2074031867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd69815c7f0 0x7fd6980c3820 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd6980b4b70 tx=0x7fd68802b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a9c8135c78e5a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:55.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.994+0000 7fd6967fc640 1 -- 172.21.15.27:0/2074031867 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688031520 con 0x7fd69815c7f0 2023-12-19T18:21:55.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.994+0000 7fd6967fc640 1 -- 172.21.15.27:0/2074031867 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd688031680 con 0x7fd69815c7f0 2023-12-19T18:21:55.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.994+0000 7fd6967fc640 1 -- 172.21.15.27:0/2074031867 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688031a10 con 0x7fd69815c7f0 2023-12-19T18:21:55.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.995+0000 7fd697fff640 1 -- 172.21.15.27:0/2074031867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd69815c7f0 msgr2=0x7fd6980c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:55.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.995+0000 7fd697fff640 1 --2- 172.21.15.27:0/2074031867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd69815c7f0 0x7fd6980c3820 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fd6980b4b70 tx=0x7fd68802b7c0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.995+0000 7fd697fff640 1 -- 172.21.15.27:0/2074031867 shutdown_connections 2023-12-19T18:21:55.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.995+0000 7fd697fff640 1 --2- 172.21.15.27:0/2074031867 >> v2:172.21.15.192:3300/0 conn(0x7fd6980cab50 0x7fd6980c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.995+0000 7fd697fff640 1 --2- 172.21.15.27:0/2074031867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd69815c7f0 0x7fd6980c3820 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.995+0000 7fd697fff640 1 -- 172.21.15.27:0/2074031867 >> 172.21.15.27:0/2074031867 conn(0x7fd698069c70 msgr2=0x7fd69806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.995+0000 7fd697fff640 1 -- 172.21.15.27:0/2074031867 shutdown_connections 2023-12-19T18:21:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.995+0000 7fd697fff640 1 -- 172.21.15.27:0/2074031867 wait complete. 2023-12-19T18:21:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.997+0000 7fd697fff640 1 Processor -- start 2023-12-19T18:21:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.997+0000 7fd697fff640 1 -- start start 2023-12-19T18:21:56.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.998+0000 7fd697fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980cab50 0x7fd69814a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.998+0000 7fd697fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6980cc510 0x7fd69815a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.998+0000 7fd697fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd69815c7f0 0x7fd698150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.998+0000 7fd697fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd698152dc0 con 0x7fd6980cab50 2023-12-19T18:21:56.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.999+0000 7fd697fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69806d070 con 0x7fd69815c7f0 2023-12-19T18:21:56.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.999+0000 7fd697fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd69806c440 con 0x7fd6980cc510 2023-12-19T18:21:56.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.999+0000 7fd696ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980cab50 0x7fd69814a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.999+0000 7fd696ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980cab50 0x7fd69814a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37038/0 (socket says 172.21.15.27:37038) 2023-12-19T18:21:56.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.999+0000 7fd696ffd640 1 -- 172.21.15.27:0/1708629823 learned_addr learned my addr 172.21.15.27:0/1708629823 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:56.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.999+0000 7fd6977fe640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd69815c7f0 0x7fd698150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.999+0000 7fd68ffff640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6980cc510 0x7fd69815a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.999+0000 7fd696ffd640 1 -- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6980cc510 msgr2=0x7fd69815a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.999+0000 7fd696ffd640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6980cc510 0x7fd69815a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.999+0000 7fd696ffd640 1 -- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd69815c7f0 msgr2=0x7fd698150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.999+0000 7fd696ffd640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd69815c7f0 0x7fd698150ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:55.999+0000 7fd696ffd640 1 -- 172.21.15.27:0/1708629823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd688009400 con 0x7fd6980cab50 2023-12-19T18:21:56.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.000+0000 7fd696ffd640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980cab50 0x7fd69814a3b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd688000c00 tx=0x7fd6880046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:56.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.000+0000 7fd694ff9640 1 -- 172.21.15.27:0/1708629823 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688031520 con 0x7fd6980cab50 2023-12-19T18:21:56.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.000+0000 7fd694ff9640 1 -- 172.21.15.27:0/1708629823 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd688033c70 con 0x7fd6980cab50 2023-12-19T18:21:56.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.000+0000 7fd694ff9640 1 -- 172.21.15.27:0/1708629823 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd68803c400 con 0x7fd6980cab50 2023-12-19T18:21:56.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.000+0000 7fd697fff640 1 -- 172.21.15.27:0/1708629823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd69806c720 con 0x7fd6980cab50 2023-12-19T18:21:56.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.000+0000 7fd697fff640 1 -- 172.21.15.27:0/1708629823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd698152a50 con 0x7fd6980cab50 2023-12-19T18:21:56.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.001+0000 7fd697fff640 1 -- 172.21.15.27:0/1708629823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69804e840 con 0x7fd6980cab50 2023-12-19T18:21:56.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.002+0000 7fd694ff9640 1 -- 172.21.15.27:0/1708629823 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd68803a040 con 0x7fd6980cab50 2023-12-19T18:21:56.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.003+0000 7fd694ff9640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd67c06a430 0x7fd67c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.003+0000 7fd694ff9640 1 -- 172.21.15.27:0/1708629823 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6880aa3e0 con 0x7fd6980cab50 2023-12-19T18:21:56.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.004+0000 7fd68ffff640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd67c06a430 0x7fd67c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.005+0000 7fd68ffff640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd67c06a430 0x7fd67c06c8f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd680006fd0 tx=0x7fd680008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:56.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.009+0000 7fd694ff9640 1 -- 172.21.15.27:0/1708629823 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd688075ae0 con 0x7fd6980cab50 2023-12-19T18:21:56.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.208+0000 7fd697fff640 1 -- 172.21.15.27:0/1708629823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6980c98e0 con 0x7fd6980cab50 2023-12-19T18:21:56.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.209+0000 7fd694ff9640 1 -- 172.21.15.27:0/1708629823 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16116 (secure 0 0 0) 0x7fd688079100 con 0x7fd6980cab50 2023-12-19T18:21:56.210 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:56.210 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":123,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:21:41.838074+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:21:06.258281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T18:21:07.308887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:21:56.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.213+0000 7fd697fff640 1 -- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd67c06a430 msgr2=0x7fd67c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.213+0000 7fd697fff640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd67c06a430 0x7fd67c06c8f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fd680006fd0 tx=0x7fd680008040 comp rx=0 tx=0).stop 2023-12-19T18:21:56.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.213+0000 7fd697fff640 1 -- 172.21.15.27:0/1708629823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980cab50 msgr2=0x7fd69814a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.213+0000 7fd697fff640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980cab50 0x7fd69814a3b0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fd688000c00 tx=0x7fd6880046c0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.213+0000 7fd697fff640 1 -- 172.21.15.27:0/1708629823 shutdown_connections 2023-12-19T18:21:56.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.213+0000 7fd697fff640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd67c06a430 0x7fd67c06c8f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.213+0000 7fd697fff640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd69815c7f0 0x7fd698150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.213+0000 7fd697fff640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6980cc510 0x7fd69815a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.213+0000 7fd697fff640 1 --2- 172.21.15.27:0/1708629823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6980cab50 0x7fd69814a3b0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.213+0000 7fd697fff640 1 -- 172.21.15.27:0/1708629823 >> 172.21.15.27:0/1708629823 conn(0x7fd698069c70 msgr2=0x7fd6981544e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:56.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.213+0000 7fd697fff640 1 -- 172.21.15.27:0/1708629823 shutdown_connections 2023-12-19T18:21:56.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.213+0000 7fd697fff640 1 -- 172.21.15.27:0/1708629823 wait complete. 2023-12-19T18:21:56.234 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:21:06.258281+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:21:07.308887+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:21:56.235 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:21:56.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.486+0000 7feaabfff640 1 Processor -- start 2023-12-19T18:21:56.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.486+0000 7feaabfff640 1 -- start start 2023-12-19T18:21:56.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.488+0000 7feaabfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7feaac0cf540 0x7feaac15a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.488+0000 7feaabfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaac15aaf0 0x7feaac06b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.488+0000 7feaabfff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaac06b4c0 con 0x7feaac0cf160 2023-12-19T18:21:56.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.488+0000 7feaabfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaac161bd0 con 0x7feaac0cf540 2023-12-19T18:21:56.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.488+0000 7feaabfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaac070080 con 0x7feaac15aaf0 2023-12-19T18:21:56.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.489+0000 7feaaaffd640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7feaac0cf160 0x7feaac15e0d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:39656/0 (socket says 172.21.15.27:39656) 2023-12-19T18:21:56.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.489+0000 7feaaaffd640 1 -- 172.21.15.27:0/18240601 learned_addr learned my addr 172.21.15.27:0/18240601 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:56.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.489+0000 7feaaa7fc640 1 --2- 172.21.15.27:0/18240601 >> v2:172.21.15.192:3300/0 conn(0x7feaac0cf540 0x7feaac15a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.489+0000 7feaab7fe640 1 --2- 172.21.15.27:0/18240601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaac15aaf0 0x7feaac06b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.490+0000 7feaa9ffb640 1 -- 172.21.15.27:0/18240601 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464327447 0 0) 0x7feaac06b4c0 con 0x7feaac0cf160 2023-12-19T18:21:56.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.490+0000 7feaa9ffb640 1 -- 172.21.15.27:0/18240601 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea94003520 con 0x7feaac0cf160 2023-12-19T18:21:56.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.490+0000 7feaaa7fc640 1 -- 172.21.15.27:0/18240601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaac15aaf0 msgr2=0x7feaac06b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:56.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.490+0000 7feaaa7fc640 1 --2- 172.21.15.27:0/18240601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaac15aaf0 0x7feaac06b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.490+0000 7feaaa7fc640 1 -- 172.21.15.27:0/18240601 >> v1:172.21.15.192:6789/0 conn(0x7feaac0cf160 legacy=0x7feaac15e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.490+0000 7feaaa7fc640 1 -- 172.21.15.27:0/18240601 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaac06d140 con 0x7feaac0cf540 2023-12-19T18:21:56.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.491+0000 7feaaa7fc640 1 --2- 172.21.15.27:0/18240601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaac0cf540 0x7feaac15a5b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7feaa00097a0 tx=0x7feaa002b820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e94e68068feb341 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:56.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.492+0000 7feaa9ffb640 1 -- 172.21.15.27:0/18240601 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0031780 con 0x7feaac0cf540 2023-12-19T18:21:56.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.492+0000 7feaa9ffb640 1 -- 172.21.15.27:0/18240601 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa00318e0 con 0x7feaac0cf540 2023-12-19T18:21:56.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.492+0000 7feaa9ffb640 1 -- 172.21.15.27:0/18240601 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa0031c70 con 0x7feaac0cf540 2023-12-19T18:21:56.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.492+0000 7feaabfff640 1 -- 172.21.15.27:0/18240601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaac0cf540 msgr2=0x7feaac15a5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.492+0000 7feaabfff640 1 --2- 172.21.15.27:0/18240601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaac0cf540 0x7feaac15a5b0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7feaa00097a0 tx=0x7feaa002b820 comp rx=0 tx=0).stop 2023-12-19T18:21:56.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.493+0000 7feaabfff640 1 -- 172.21.15.27:0/18240601 shutdown_connections 2023-12-19T18:21:56.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.493+0000 7feaabfff640 1 --2- 172.21.15.27:0/18240601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaac15aaf0 0x7feaac06b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.493+0000 7feaabfff640 1 --2- 172.21.15.27:0/18240601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaac0cf540 0x7feaac15a5b0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.493+0000 7feaabfff640 1 -- 172.21.15.27:0/18240601 >> 172.21.15.27:0/18240601 conn(0x7feaac06a060 msgr2=0x7feaac06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:56.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.493+0000 7feaabfff640 1 -- 172.21.15.27:0/18240601 shutdown_connections 2023-12-19T18:21:56.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.493+0000 7feaabfff640 1 -- 172.21.15.27:0/18240601 wait complete. 2023-12-19T18:21:56.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.494+0000 7feaabfff640 1 Processor -- start 2023-12-19T18:21:56.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.495+0000 7feaabfff640 1 -- start start 2023-12-19T18:21:56.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.496+0000 7feaabfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaac0cf160 0x7feaac065bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.496+0000 7feaabfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaac0cf540 0x7feaac06bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.496+0000 7feaabfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaac15aaf0 0x7feaac1463c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.496+0000 7feaabfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac145fa0 con 0x7feaac15aaf0 2023-12-19T18:21:56.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.496+0000 7feaabfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaac1499b0 con 0x7feaac0cf160 2023-12-19T18:21:56.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.496+0000 7feaabfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaac06cd90 con 0x7feaac0cf540 2023-12-19T18:21:56.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.496+0000 7feaab7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaac15aaf0 0x7feaac1463c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.496+0000 7feaab7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaac15aaf0 0x7feaac1463c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37044/0 (socket says 172.21.15.27:37044) 2023-12-19T18:21:56.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.496+0000 7feaab7fe640 1 -- 172.21.15.27:0/2996143737 learned_addr learned my addr 172.21.15.27:0/2996143737 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:56.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.496+0000 7feaaa7fc640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaac0cf540 0x7feaac06bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.496+0000 7feaaaffd640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaac0cf160 0x7feaac065bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.496+0000 7feaab7fe640 1 -- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaac0cf540 msgr2=0x7feaac06bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.497+0000 7feaab7fe640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaac0cf540 0x7feaac06bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.497+0000 7feaab7fe640 1 -- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaac0cf160 msgr2=0x7feaac065bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.497+0000 7feaab7fe640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaac0cf160 0x7feaac065bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.497+0000 7feaab7fe640 1 -- 172.21.15.27:0/2996143737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa0009400 con 0x7feaac15aaf0 2023-12-19T18:21:56.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.497+0000 7feaaa7fc640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaac0cf540 0x7feaac06bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:56.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.497+0000 7feaab7fe640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaac15aaf0 0x7feaac1463c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009d20 tx=0x7feaa400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:56.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.497+0000 7fea937fe640 1 -- 172.21.15.27:0/2996143737 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa4009b10 con 0x7feaac15aaf0 2023-12-19T18:21:56.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.497+0000 7fea937fe640 1 -- 172.21.15.27:0/2996143737 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa400f040 con 0x7feaac15aaf0 2023-12-19T18:21:56.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.497+0000 7fea937fe640 1 -- 172.21.15.27:0/2996143737 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa40146c0 con 0x7feaac15aaf0 2023-12-19T18:21:56.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.497+0000 7feaabfff640 1 -- 172.21.15.27:0/2996143737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaac06c7d0 con 0x7feaac15aaf0 2023-12-19T18:21:56.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.497+0000 7feaabfff640 1 -- 172.21.15.27:0/2996143737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaac145bc0 con 0x7feaac15aaf0 2023-12-19T18:21:56.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.499+0000 7feaabfff640 1 -- 172.21.15.27:0/2996143737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaac1460e0 con 0x7feaac15aaf0 2023-12-19T18:21:56.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.500+0000 7fea937fe640 1 -- 172.21.15.27:0/2996143737 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7feaa4014820 con 0x7feaac15aaf0 2023-12-19T18:21:56.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.501+0000 7fea937fe640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea9806a430 0x7fea9806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.501+0000 7fea937fe640 1 -- 172.21.15.27:0/2996143737 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feaa40895f0 con 0x7feaac15aaf0 2023-12-19T18:21:56.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.501+0000 7feaaaffd640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea9806a430 0x7fea9806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.502+0000 7feaaaffd640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea9806a430 0x7fea9806c8f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0011e0 tx=0x7fea9c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:56.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.505+0000 7fea937fe640 1 -- 172.21.15.27:0/2996143737 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7feaac1460e0 con 0x7feaac15aaf0 2023-12-19T18:21:56.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.699+0000 7feaabfff640 1 -- 172.21.15.27:0/2996143737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7feaac1460e0 con 0x7feaac15aaf0 2023-12-19T18:21:56.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.700+0000 7fea937fe640 1 -- 172.21.15.27:0/2996143737 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v123) v1 ==== 93+0+10 (secure 0 0 0) 0x7feaa4018020 con 0x7feaac15aaf0 2023-12-19T18:21:56.700 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:21:56.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.703+0000 7feaabfff640 1 -- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea9806a430 msgr2=0x7fea9806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.703+0000 7feaabfff640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea9806a430 0x7fea9806c8f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fea9c0011e0 tx=0x7fea9c009040 comp rx=0 tx=0).stop 2023-12-19T18:21:56.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.703+0000 7feaabfff640 1 -- 172.21.15.27:0/2996143737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaac15aaf0 msgr2=0x7feaac1463c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.703+0000 7feaabfff640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaac15aaf0 0x7feaac1463c0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009d20 tx=0x7feaa400c490 comp rx=0 tx=0).stop 2023-12-19T18:21:56.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.703+0000 7feaabfff640 1 -- 172.21.15.27:0/2996143737 shutdown_connections 2023-12-19T18:21:56.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.703+0000 7feaabfff640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea9806a430 0x7fea9806c8f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.703+0000 7feaabfff640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaac15aaf0 0x7feaac1463c0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.703+0000 7feaabfff640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaac0cf540 0x7feaac06bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.703+0000 7feaabfff640 1 --2- 172.21.15.27:0/2996143737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaac0cf160 0x7feaac065bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.703+0000 7feaabfff640 1 -- 172.21.15.27:0/2996143737 >> 172.21.15.27:0/2996143737 conn(0x7feaac06a060 msgr2=0x7feaac0b2440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.703+0000 7feaabfff640 1 -- 172.21.15.27:0/2996143737 shutdown_connections 2023-12-19T18:21:56.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.703+0000 7feaabfff640 1 -- 172.21.15.27:0/2996143737 wait complete. 2023-12-19T18:21:56.721 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:21:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.959+0000 7fa17e896640 1 Processor -- start 2023-12-19T18:21:56.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.959+0000 7fa17e896640 1 -- start start 2023-12-19T18:21:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.960+0000 7fa17e896640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa1780b8c30 0x7fa1780d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.960+0000 7fa17e896640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1780ce130 0x7fa1780cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.960+0000 7fa17e896640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1780ce560 con 0x7fa1780cf9b0 2023-12-19T18:21:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.960+0000 7fa17e896640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1780cea80 con 0x7fa1780b8c30 2023-12-19T18:21:56.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.960+0000 7fa17e896640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1780b5f10 con 0x7fa1780ce130 2023-12-19T18:21:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.961+0000 7fa177fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa1780b8c30 0x7fa1780d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.961+0000 7fa17ce33640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1780ce130 0x7fa1780cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.961+0000 7fa177fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa1780b8c30 0x7fa1780d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:54962/0 (socket says 172.21.15.27:54962) 2023-12-19T18:21:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.961+0000 7fa177fff640 1 -- 172.21.15.27:0/3206077964 learned_addr learned my addr 172.21.15.27:0/3206077964 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.962+0000 7fa177fff640 1 -- 172.21.15.27:0/3206077964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1780ce130 msgr2=0x7fa1780cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.962+0000 7fa177fff640 1 --2- 172.21.15.27:0/3206077964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1780ce130 0x7fa1780cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.962+0000 7fa177fff640 1 -- 172.21.15.27:0/3206077964 >> v1:172.21.15.192:6789/0 conn(0x7fa1780cf9b0 legacy=0x7fa1780cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.962+0000 7fa177fff640 1 -- 172.21.15.27:0/3206077964 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1780ce7a0 con 0x7fa1780b8c30 2023-12-19T18:21:56.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.962+0000 7fa176ffd640 1 -- 172.21.15.27:0/3206077964 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703588801 0 0) 0x7fa16c001110 con 0x7fa1780cf9b0 2023-12-19T18:21:56.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.963+0000 7fa177fff640 1 --2- 172.21.15.27:0/3206077964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1780b8c30 0x7fa1780d1fc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa168009e50 tx=0x7fa16802bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=892d9c047e07bf0e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:56.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.964+0000 7fa176ffd640 1 -- 172.21.15.27:0/3206077964 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa16802ca70 con 0x7fa1780b8c30 2023-12-19T18:21:56.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.964+0000 7fa176ffd640 1 -- 172.21.15.27:0/3206077964 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa16802cbd0 con 0x7fa1780b8c30 2023-12-19T18:21:56.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.964+0000 7fa176ffd640 1 -- 172.21.15.27:0/3206077964 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1680343f0 con 0x7fa1780b8c30 2023-12-19T18:21:56.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.965+0000 7fa17e896640 1 -- 172.21.15.27:0/3206077964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1780b8c30 msgr2=0x7fa1780d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.965+0000 7fa17e896640 1 --2- 172.21.15.27:0/3206077964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1780b8c30 0x7fa1780d1fc0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fa168009e50 tx=0x7fa16802bed0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.965+0000 7fa17e896640 1 -- 172.21.15.27:0/3206077964 shutdown_connections 2023-12-19T18:21:56.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.965+0000 7fa17e896640 1 --2- 172.21.15.27:0/3206077964 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1780ce130 0x7fa1780cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.965+0000 7fa17e896640 1 --2- 172.21.15.27:0/3206077964 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1780b8c30 0x7fa1780d1fc0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.965+0000 7fa17e896640 1 -- 172.21.15.27:0/3206077964 >> 172.21.15.27:0/3206077964 conn(0x7fa1780d2fa0 msgr2=0x7fa1780d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:56.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.965+0000 7fa17e896640 1 -- 172.21.15.27:0/3206077964 shutdown_connections 2023-12-19T18:21:56.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.965+0000 7fa17e896640 1 -- 172.21.15.27:0/3206077964 wait complete. 2023-12-19T18:21:56.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.967+0000 7fa17e896640 1 Processor -- start 2023-12-19T18:21:56.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.967+0000 7fa17e896640 1 -- start start 2023-12-19T18:21:56.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.968+0000 7fa17e896640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1780b8c30 0x7fa178158420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.968+0000 7fa17e896640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1780ce130 0x7fa1780b6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.968+0000 7fa17e896640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1780cf9b0 0x7fa1780b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.968+0000 7fa17e896640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1780cb0c0 con 0x7fa1780cf9b0 2023-12-19T18:21:56.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.968+0000 7fa17e896640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1781f2cd0 con 0x7fa1780ce130 2023-12-19T18:21:56.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.968+0000 7fa17e896640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1780cc090 con 0x7fa1780b8c30 2023-12-19T18:21:56.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.968+0000 7fa17ce33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1780cf9b0 0x7fa1780b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.968+0000 7fa17ce33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1780cf9b0 0x7fa1780b6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37060/0 (socket says 172.21.15.27:37060) 2023-12-19T18:21:56.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.968+0000 7fa17ce33640 1 -- 172.21.15.27:0/3134049438 learned_addr learned my addr 172.21.15.27:0/3134049438 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:56.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.969+0000 7fa177fff640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1780b8c30 0x7fa178158420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.969+0000 7fa1777fe640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1780ce130 0x7fa1780b6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.969+0000 7fa17ce33640 1 -- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1780b8c30 msgr2=0x7fa178158420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.969+0000 7fa17ce33640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1780b8c30 0x7fa178158420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.969+0000 7fa17ce33640 1 -- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1780ce130 msgr2=0x7fa1780b6940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:56.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.969+0000 7fa17ce33640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1780ce130 0x7fa1780b6940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:56.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.969+0000 7fa17ce33640 1 -- 172.21.15.27:0/3134049438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa168009b30 con 0x7fa1780cf9b0 2023-12-19T18:21:56.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.969+0000 7fa177fff640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1780b8c30 0x7fa178158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:21:56.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.969+0000 7fa17ce33640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1780cf9b0 0x7fa1780b6e80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa17000e860 tx=0x7fa17000ed20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:56.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.970+0000 7fa174ff9640 1 -- 172.21.15.27:0/3134049438 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17000c6d0 con 0x7fa1780cf9b0 2023-12-19T18:21:56.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.970+0000 7fa174ff9640 1 -- 172.21.15.27:0/3134049438 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa17000c830 con 0x7fa1780cf9b0 2023-12-19T18:21:56.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.970+0000 7fa17e896640 1 -- 172.21.15.27:0/3134049438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1780cb2d0 con 0x7fa1780cf9b0 2023-12-19T18:21:56.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.970+0000 7fa174ff9640 1 -- 172.21.15.27:0/3134049438 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa17000cbe0 con 0x7fa1780cf9b0 2023-12-19T18:21:56.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.970+0000 7fa17e896640 1 -- 172.21.15.27:0/3134049438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1780cb890 con 0x7fa1780cf9b0 2023-12-19T18:21:56.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.971+0000 7fa17e896640 1 -- 172.21.15.27:0/3134049438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa178067110 con 0x7fa1780cf9b0 2023-12-19T18:21:56.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.972+0000 7fa174ff9640 1 -- 172.21.15.27:0/3134049438 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa170013470 con 0x7fa1780cf9b0 2023-12-19T18:21:56.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.973+0000 7fa174ff9640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa16406a430 0x7fa16406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:56.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.973+0000 7fa174ff9640 1 -- 172.21.15.27:0/3134049438 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa170088de0 con 0x7fa1780cf9b0 2023-12-19T18:21:56.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.974+0000 7fa177fff640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa16406a430 0x7fa16406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:56.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.975+0000 7fa177fff640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa16406a430 0x7fa16406c8f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa168035000 tx=0x7fa168030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:56.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:56.985+0000 7fa174ff9640 1 -- 172.21.15.27:0/3134049438 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa170054350 con 0x7fa1780cf9b0 2023-12-19T18:21:57.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.186+0000 7fa17e896640 1 -- 172.21.15.27:0/3134049438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa1780d0ef0 con 0x7fa1780cf9b0 2023-12-19T18:21:57.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.188+0000 7fa174ff9640 1 -- 172.21.15.27:0/3134049438 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v123) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa170057970 con 0x7fa1780cf9b0 2023-12-19T18:21:57.188 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:21:57.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.190+0000 7fa17e896640 1 -- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa16406a430 msgr2=0x7fa16406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:57.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.190+0000 7fa17e896640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa16406a430 0x7fa16406c8f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fa168035000 tx=0x7fa168030000 comp rx=0 tx=0).stop 2023-12-19T18:21:57.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.190+0000 7fa17e896640 1 -- 172.21.15.27:0/3134049438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1780cf9b0 msgr2=0x7fa1780b6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:57.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.190+0000 7fa17e896640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1780cf9b0 0x7fa1780b6e80 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fa17000e860 tx=0x7fa17000ed20 comp rx=0 tx=0).stop 2023-12-19T18:21:57.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.190+0000 7fa17e896640 1 -- 172.21.15.27:0/3134049438 shutdown_connections 2023-12-19T18:21:57.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.190+0000 7fa17e896640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa16406a430 0x7fa16406c8f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.190+0000 7fa17e896640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1780cf9b0 0x7fa1780b6e80 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.190+0000 7fa17e896640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1780ce130 0x7fa1780b6940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.190+0000 7fa17e896640 1 --2- 172.21.15.27:0/3134049438 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1780b8c30 0x7fa178158420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.190+0000 7fa17e896640 1 -- 172.21.15.27:0/3134049438 >> 172.21.15.27:0/3134049438 conn(0x7fa1780d2fa0 msgr2=0x7fa1780b2f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:57.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.191+0000 7fa17e896640 1 -- 172.21.15.27:0/3134049438 shutdown_connections 2023-12-19T18:21:57.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.191+0000 7fa17e896640 1 -- 172.21.15.27:0/3134049438 wait complete. 2023-12-19T18:21:57.211 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:21:57.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.449+0000 7f8ff7149640 1 Processor -- start 2023-12-19T18:21:57.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.450+0000 7f8ff7149640 1 -- start start 2023-12-19T18:21:57.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.451+0000 7f8ff7149640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8ff015e9d0 0x7f8ff00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:57.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.451+0000 7f8ff7149640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff0152070 0x7f8ff00cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:57.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.451+0000 7f8ff7149640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ff015b3e0 con 0x7f8ff0151c90 2023-12-19T18:21:57.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.451+0000 7f8ff7149640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ff015eed0 con 0x7f8ff015e9d0 2023-12-19T18:21:57.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.451+0000 7f8ff7149640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff015f1b0 con 0x7f8ff0152070 2023-12-19T18:21:57.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.452+0000 7f8ff5946640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8ff015e9d0 0x7f8ff00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:57.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.452+0000 7f8ff5946640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8ff015e9d0 0x7f8ff00cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37068/0 (socket says 172.21.15.27:37068) 2023-12-19T18:21:57.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.452+0000 7f8ff5946640 1 -- 172.21.15.27:0/1708726482 learned_addr learned my addr 172.21.15.27:0/1708726482 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:57.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.452+0000 7f8ff6948640 1 --2- 172.21.15.27:0/1708726482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff0152070 0x7f8ff00cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:57.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.452+0000 7f8ff5946640 1 -- 172.21.15.27:0/1708726482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff0152070 msgr2=0x7f8ff00cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:57.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.452+0000 7f8ff5946640 1 --2- 172.21.15.27:0/1708726482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff0152070 0x7f8ff00cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.452+0000 7f8ff5946640 1 -- 172.21.15.27:0/1708726482 >> v1:172.21.15.192:6789/0 conn(0x7f8ff0151c90 legacy=0x7f8ff006b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:57.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.453+0000 7f8ff5946640 1 -- 172.21.15.27:0/1708726482 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff015f490 con 0x7f8ff015e9d0 2023-12-19T18:21:57.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.454+0000 7f8ff5946640 1 --2- 172.21.15.27:0/1708726482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff015e9d0 0x7f8ff00cee40 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00098a0 tx=0x7f8fe002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9c89e865d2078bb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:57.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.454+0000 7f8ff5145640 1 -- 172.21.15.27:0/1708726482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0031640 con 0x7f8ff015e9d0 2023-12-19T18:21:57.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.454+0000 7f8ff5145640 1 -- 172.21.15.27:0/1708726482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe00317a0 con 0x7f8ff015e9d0 2023-12-19T18:21:57.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.454+0000 7f8ff5145640 1 -- 172.21.15.27:0/1708726482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0031b30 con 0x7f8ff015e9d0 2023-12-19T18:21:57.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.455+0000 7f8ff7149640 1 -- 172.21.15.27:0/1708726482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff015e9d0 msgr2=0x7f8ff00cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:57.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.455+0000 7f8ff7149640 1 --2- 172.21.15.27:0/1708726482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff015e9d0 0x7f8ff00cee40 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00098a0 tx=0x7f8fe002b8f0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.455+0000 7f8ff7149640 1 -- 172.21.15.27:0/1708726482 shutdown_connections 2023-12-19T18:21:57.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.455+0000 7f8ff7149640 1 --2- 172.21.15.27:0/1708726482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff0152070 0x7f8ff00cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.455+0000 7f8ff7149640 1 --2- 172.21.15.27:0/1708726482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff015e9d0 0x7f8ff00cee40 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.455+0000 7f8ff7149640 1 -- 172.21.15.27:0/1708726482 >> 172.21.15.27:0/1708726482 conn(0x7f8ff006a060 msgr2=0x7f8ff006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:57.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.455+0000 7f8ff7149640 1 -- 172.21.15.27:0/1708726482 shutdown_connections 2023-12-19T18:21:57.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.455+0000 7f8ff7149640 1 -- 172.21.15.27:0/1708726482 wait complete. 2023-12-19T18:21:57.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.457+0000 7f8ff7149640 1 Processor -- start 2023-12-19T18:21:57.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.457+0000 7f8ff7149640 1 -- start start 2023-12-19T18:21:57.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.458+0000 7f8ff7149640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff0151c90 0x7f8ff0152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:57.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.458+0000 7f8ff7149640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff0152070 0x7f8ff00ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:57.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.458+0000 7f8ff7149640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff015e9d0 0x7f8ff00ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:57.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.458+0000 7f8ff7149640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01f30f0 con 0x7f8ff0152070 2023-12-19T18:21:57.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.458+0000 7f8ff7149640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01f2280 con 0x7f8ff0151c90 2023-12-19T18:21:57.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.458+0000 7f8ff7149640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff01f2470 con 0x7f8ff015e9d0 2023-12-19T18:21:57.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.458+0000 7f8ff5946640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff0152070 0x7f8ff00ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:57.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.458+0000 7f8ff5946640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff0152070 0x7f8ff00ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37080/0 (socket says 172.21.15.27:37080) 2023-12-19T18:21:57.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.458+0000 7f8ff5946640 1 -- 172.21.15.27:0/851093155 learned_addr learned my addr 172.21.15.27:0/851093155 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:57.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.459+0000 7f8ff6948640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff015e9d0 0x7f8ff00ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:57.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.459+0000 7f8ff6147640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff0151c90 0x7f8ff0152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:57.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.459+0000 7f8ff5946640 1 -- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff015e9d0 msgr2=0x7f8ff00ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:57.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.459+0000 7f8ff5946640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff015e9d0 0x7f8ff00ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.459+0000 7f8ff5946640 1 -- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff0151c90 msgr2=0x7f8ff0152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:57.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.459+0000 7f8ff5946640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff0151c90 0x7f8ff0152ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.459+0000 7f8ff5946640 1 -- 172.21.15.27:0/851093155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe00094d0 con 0x7f8ff0152070 2023-12-19T18:21:57.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.459+0000 7f8ff5946640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff0152070 0x7f8ff00ac8c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0000c00 tx=0x7f8fe00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:57.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.459+0000 7f8fe6ffd640 1 -- 172.21.15.27:0/851093155 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe003ac60 con 0x7f8ff0152070 2023-12-19T18:21:57.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.459+0000 7f8fe6ffd640 1 -- 172.21.15.27:0/851093155 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fe003adc0 con 0x7f8ff0152070 2023-12-19T18:21:57.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.459+0000 7f8fe6ffd640 1 -- 172.21.15.27:0/851093155 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0044630 con 0x7f8ff0152070 2023-12-19T18:21:57.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.459+0000 7f8ff7149640 1 -- 172.21.15.27:0/851093155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff01f2750 con 0x7f8ff0152070 2023-12-19T18:21:57.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.460+0000 7f8ff7149640 1 -- 172.21.15.27:0/851093155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff01f2d10 con 0x7f8ff0152070 2023-12-19T18:21:57.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.461+0000 7f8ff7149640 1 -- 172.21.15.27:0/851093155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff004e840 con 0x7f8ff0152070 2023-12-19T18:21:57.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.462+0000 7f8fe6ffd640 1 -- 172.21.15.27:0/851093155 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8fe002f050 con 0x7f8ff0152070 2023-12-19T18:21:57.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.463+0000 7f8fe6ffd640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8fd406a4b0 0x7f8fd406c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:57.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.463+0000 7f8fe6ffd640 1 -- 172.21.15.27:0/851093155 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fe00aa610 con 0x7f8ff0152070 2023-12-19T18:21:57.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.463+0000 7f8ff6147640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8fd406a4b0 0x7f8fd406c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:57.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.464+0000 7f8ff6147640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8fd406a4b0 0x7f8fd406c970 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8fec007fd0 tx=0x7f8fec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:57.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.467+0000 7f8fe6ffd640 1 -- 172.21.15.27:0/851093155 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8fe0075ce0 con 0x7f8ff0152070 2023-12-19T18:21:57.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.662+0000 7f8ff7149640 1 -- 172.21.15.27:0/851093155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ff00b1670 con 0x7f8ff0152070 2023-12-19T18:21:57.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.662+0000 7f8fe6ffd640 1 -- 172.21.15.27:0/851093155 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v123) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8fe0031e60 con 0x7f8ff0152070 2023-12-19T18:21:57.662 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:21:57.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.665+0000 7f8ff7149640 1 -- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8fd406a4b0 msgr2=0x7f8fd406c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:57.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.665+0000 7f8ff7149640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8fd406a4b0 0x7f8fd406c970 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f8fec007fd0 tx=0x7f8fec009040 comp rx=0 tx=0).stop 2023-12-19T18:21:57.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.665+0000 7f8ff7149640 1 -- 172.21.15.27:0/851093155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff0152070 msgr2=0x7f8ff00ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:57.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.665+0000 7f8ff7149640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff0152070 0x7f8ff00ac8c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0000c00 tx=0x7f8fe00046c0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.666+0000 7f8ff7149640 1 -- 172.21.15.27:0/851093155 shutdown_connections 2023-12-19T18:21:57.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.666+0000 7f8ff7149640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8fd406a4b0 0x7f8fd406c970 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.666+0000 7f8ff7149640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff015e9d0 0x7f8ff00ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.666+0000 7f8ff7149640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff0152070 0x7f8ff00ac8c0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.666+0000 7f8ff7149640 1 --2- 172.21.15.27:0/851093155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff0151c90 0x7f8ff0152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.666+0000 7f8ff7149640 1 -- 172.21.15.27:0/851093155 >> 172.21.15.27:0/851093155 conn(0x7f8ff006a060 msgr2=0x7f8ff0151160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:57.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.666+0000 7f8ff7149640 1 -- 172.21.15.27:0/851093155 shutdown_connections 2023-12-19T18:21:57.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.666+0000 7f8ff7149640 1 -- 172.21.15.27:0/851093155 wait complete. 2023-12-19T18:21:57.685 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:21:57.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.920+0000 7f3aebfff640 1 Processor -- start 2023-12-19T18:21:57.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.920+0000 7f3aebfff640 1 -- start start 2023-12-19T18:21:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.922+0000 7f3aebfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3aec0d2350 0x7f3aec0b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.922+0000 7f3aebfff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aec0b5e90 con 0x7f3aec0b4ef0 2023-12-19T18:21:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.922+0000 7f3aebfff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3aec0d2850 con 0x7f3aec1604d0 2023-12-19T18:21:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.922+0000 7f3aebfff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3aec0d2b30 con 0x7f3aec0d2350 2023-12-19T18:21:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.922+0000 7f3aeb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3aec0d2350 0x7f3aec0b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:57.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.922+0000 7f3aea7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3aec0b4ef0 0x7f3aec0b6300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55776/0 (socket says 172.21.15.27:55776) 2023-12-19T18:21:57.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.922+0000 7f3aea7fc640 1 -- 172.21.15.27:0/2120132105 learned_addr learned my addr 172.21.15.27:0/2120132105 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:57.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.923+0000 7f3aeb7fe640 1 -- 172.21.15.27:0/2120132105 >> v1:172.21.15.192:6789/0 conn(0x7f3aec1604d0 legacy=0x7f3aec0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:57.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.923+0000 7f3aeb7fe640 1 -- 172.21.15.27:0/2120132105 >> v1:172.21.15.27:6789/0 conn(0x7f3aec0b4ef0 legacy=0x7f3aec0b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:57.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.923+0000 7f3aeb7fe640 1 -- 172.21.15.27:0/2120132105 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aec0cded0 con 0x7f3aec0d2350 2023-12-19T18:21:57.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.924+0000 7f3aeb7fe640 1 --2- 172.21.15.27:0/2120132105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aec0d2350 0x7f3aec0b6a10 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40097a0 tx=0x7f3ae402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae787098109b227 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:57.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.924+0000 7f3ae9ffb640 1 -- 172.21.15.27:0/2120132105 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4031830 con 0x7f3aec0d2350 2023-12-19T18:21:57.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.924+0000 7f3ae9ffb640 1 -- 172.21.15.27:0/2120132105 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ae4031990 con 0x7f3aec0d2350 2023-12-19T18:21:57.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.924+0000 7f3ae9ffb640 1 -- 172.21.15.27:0/2120132105 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ae4031d20 con 0x7f3aec0d2350 2023-12-19T18:21:57.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.925+0000 7f3aebfff640 1 -- 172.21.15.27:0/2120132105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aec0d2350 msgr2=0x7f3aec0b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:57.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.925+0000 7f3aebfff640 1 --2- 172.21.15.27:0/2120132105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aec0d2350 0x7f3aec0b6a10 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f3ae40097a0 tx=0x7f3ae402b8d0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.925+0000 7f3aebfff640 1 -- 172.21.15.27:0/2120132105 shutdown_connections 2023-12-19T18:21:57.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.925+0000 7f3aebfff640 1 --2- 172.21.15.27:0/2120132105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aec0d2350 0x7f3aec0b6a10 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.925+0000 7f3aebfff640 1 -- 172.21.15.27:0/2120132105 >> 172.21.15.27:0/2120132105 conn(0x7f3aec0cf7e0 msgr2=0x7f3aec0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:57.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.925+0000 7f3aebfff640 1 -- 172.21.15.27:0/2120132105 shutdown_connections 2023-12-19T18:21:57.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.925+0000 7f3aebfff640 1 -- 172.21.15.27:0/2120132105 wait complete. 2023-12-19T18:21:57.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.927+0000 7f3aebfff640 1 Processor -- start 2023-12-19T18:21:57.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.927+0000 7f3aebfff640 1 -- start start 2023-12-19T18:21:57.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.928+0000 7f3aebfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3aec0b4ef0 0x7f3aec154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:57.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.928+0000 7f3aebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aec0d2350 0x7f3aec0cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:57.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.928+0000 7f3aebfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3aec1604d0 0x7f3aec0c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:57.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.928+0000 7f3aebfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec154850 con 0x7f3aec0d2350 2023-12-19T18:21:57.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.928+0000 7f3aebfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec0c9f60 con 0x7f3aec0b4ef0 2023-12-19T18:21:57.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.928+0000 7f3aebfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3aec0ca240 con 0x7f3aec1604d0 2023-12-19T18:21:57.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.928+0000 7f3aea7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aec0d2350 0x7f3aec0cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:57.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.928+0000 7f3aea7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aec0d2350 0x7f3aec0cb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37092/0 (socket says 172.21.15.27:37092) 2023-12-19T18:21:57.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.928+0000 7f3aea7fc640 1 -- 172.21.15.27:0/3144265263 learned_addr learned my addr 172.21.15.27:0/3144265263 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:57.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.929+0000 7f3aea7fc640 1 -- 172.21.15.27:0/3144265263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3aec1604d0 msgr2=0x7f3aec0c8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:57.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.929+0000 7f3aea7fc640 1 --2- 172.21.15.27:0/3144265263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3aec1604d0 0x7f3aec0c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.929+0000 7f3aeaffd640 1 --2- 172.21.15.27:0/3144265263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3aec0b4ef0 0x7f3aec154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:57.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.929+0000 7f3aea7fc640 1 -- 172.21.15.27:0/3144265263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3aec0b4ef0 msgr2=0x7f3aec154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:57.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.929+0000 7f3aea7fc640 1 --2- 172.21.15.27:0/3144265263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3aec0b4ef0 0x7f3aec154310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:57.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.929+0000 7f3aea7fc640 1 -- 172.21.15.27:0/3144265263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae4009400 con 0x7f3aec0d2350 2023-12-19T18:21:57.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.929+0000 7f3aea7fc640 1 --2- 172.21.15.27:0/3144265263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aec0d2350 0x7f3aec0cb000 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c4a0 tx=0x7f3ad400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:57.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.930+0000 7f3ae37fe640 1 -- 172.21.15.27:0/3144265263 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4005ce0 con 0x7f3aec0d2350 2023-12-19T18:21:57.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.930+0000 7f3ae37fe640 1 -- 172.21.15.27:0/3144265263 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad4005e40 con 0x7f3aec0d2350 2023-12-19T18:21:57.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.930+0000 7f3aebfff640 1 -- 172.21.15.27:0/3144265263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3aec0ca520 con 0x7f3aec0d2350 2023-12-19T18:21:57.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.930+0000 7f3ae37fe640 1 -- 172.21.15.27:0/3144265263 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad4007270 con 0x7f3aec0d2350 2023-12-19T18:21:57.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.930+0000 7f3aebfff640 1 -- 172.21.15.27:0/3144265263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3aec0caae0 con 0x7f3aec0d2350 2023-12-19T18:21:57.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.931+0000 7f3aebfff640 1 -- 172.21.15.27:0/3144265263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3aec066f80 con 0x7f3aec0d2350 2023-12-19T18:21:57.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.932+0000 7f3ae37fe640 1 -- 172.21.15.27:0/3144265263 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3ad4007490 con 0x7f3aec0d2350 2023-12-19T18:21:57.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.933+0000 7f3ae37fe640 1 --2- 172.21.15.27:0/3144265263 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3acc06a430 0x7f3acc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:57.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.934+0000 7f3ae37fe640 1 -- 172.21.15.27:0/3144265263 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ad4087de0 con 0x7f3aec0d2350 2023-12-19T18:21:57.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.934+0000 7f3aeaffd640 1 --2- 172.21.15.27:0/3144265263 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3acc06a430 0x7f3acc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:57.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.935+0000 7f3aeaffd640 1 --2- 172.21.15.27:0/3144265263 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3acc06a430 0x7f3acc06c8f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3aec15dc00 tx=0x7f3adc0072f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:57.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:57.939+0000 7f3ae37fe640 1 -- 172.21.15.27:0/3144265263 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3ad4053470 con 0x7f3aec0d2350 2023-12-19T18:21:58.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.167+0000 7f3aebfff640 1 -- 172.21.15.27:0/3144265263 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3aec0cacd0 con 0x7f3aec0d2350 2023-12-19T18:21:58.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.168+0000 7f3ae37fe640 1 -- 172.21.15.27:0/3144265263 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f3ad4017090 con 0x7f3aec0d2350 2023-12-19T18:21:58.168 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:58.169 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2779436922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2779436922},{"type":"v1","addr":"172.21.15.192:6836","nonce":2779436922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3121062779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3121062779},{"type":"v1","addr":"172.21.15.192:6837","nonce":3121062779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5357,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2032553869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2032553869},{"type":"v1","addr":"172.21.15.27:6835","nonce":2032553869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:21:08.281615+0000","modified":"2023-12-19T18:21:11.800655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/252970177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":252970177},{"type":"v1","addr":"172.21.15.27:6834","nonce":252970177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T18:21:58.169 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 46 2023-12-19T18:21:58.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.170+0000 7f3aebfff640 1 -- 172.21.15.27:0/3144265263 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3acc06a430 msgr2=0x7f3acc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:58.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.170+0000 7f3aebfff640 1 --2- 172.21.15.27:0/3144265263 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3acc06a430 0x7f3acc06c8f0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f3aec15dc00 tx=0x7f3adc0072f0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.170+0000 7f3aebfff640 1 -- 172.21.15.27:0/3144265263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aec0d2350 msgr2=0x7f3aec0cb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:58.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.170+0000 7f3aebfff640 1 --2- 172.21.15.27:0/3144265263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aec0d2350 0x7f3aec0cb000 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f3ad400c4a0 tx=0x7f3ad400c960 comp rx=0 tx=0).stop 2023-12-19T18:21:58.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.170+0000 7f3aebfff640 1 -- 172.21.15.27:0/3144265263 shutdown_connections 2023-12-19T18:21:58.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.170+0000 7f3aebfff640 1 --2- 172.21.15.27:0/3144265263 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3acc06a430 0x7f3acc06c8f0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.170+0000 7f3aebfff640 1 --2- 172.21.15.27:0/3144265263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3aec1604d0 0x7f3aec0c8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.170+0000 7f3aebfff640 1 --2- 172.21.15.27:0/3144265263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3aec0d2350 0x7f3aec0cb000 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.170+0000 7f3aebfff640 1 --2- 172.21.15.27:0/3144265263 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3aec0b4ef0 0x7f3aec154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.170+0000 7f3aebfff640 1 -- 172.21.15.27:0/3144265263 >> 172.21.15.27:0/3144265263 conn(0x7f3aec0cf7e0 msgr2=0x7f3aec0d1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:58.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.171+0000 7f3aebfff640 1 -- 172.21.15.27:0/3144265263 shutdown_connections 2023-12-19T18:21:58.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.171+0000 7f3aebfff640 1 -- 172.21.15.27:0/3144265263 wait complete. 2023-12-19T18:21:58.190 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:21:58.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.421+0000 7f0749412640 1 Processor -- start 2023-12-19T18:21:58.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.421+0000 7f0749412640 1 -- start start 2023-12-19T18:21:58.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.422+0000 7f0749412640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07440cdf90 0x7f07440ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.422+0000 7f0749412640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f07440d1e00 0x7f07440d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.422+0000 7f0749412640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07440ce930 con 0x7f074415d240 2023-12-19T18:21:58.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.422+0000 7f0749412640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07440d1970 con 0x7f07440d1e00 2023-12-19T18:21:58.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.422+0000 7f0749412640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f074406a5e0 con 0x7f07440cdf90 2023-12-19T18:21:58.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.423+0000 7f07437fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f07440d1e00 0x7f07440d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:58.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.423+0000 7f0742ffd640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f074415d240 0x7f07441618d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:39694/0 (socket says 172.21.15.27:39694) 2023-12-19T18:21:58.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.423+0000 7f07437fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f07440d1e00 0x7f07440d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:54988/0 (socket says 172.21.15.27:54988) 2023-12-19T18:21:58.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.423+0000 7f0742ffd640 1 -- 172.21.15.27:0/904311248 learned_addr learned my addr 172.21.15.27:0/904311248 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:58.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.423+0000 7f07427fc640 1 --2- 172.21.15.27:0/904311248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07440cdf90 0x7f07440ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:58.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.424+0000 7f0741ffb640 1 -- 172.21.15.27:0/904311248 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221214259 0 0) 0x7f07440ce930 con 0x7f074415d240 2023-12-19T18:21:58.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.424+0000 7f0741ffb640 1 -- 172.21.15.27:0/904311248 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0730003520 con 0x7f074415d240 2023-12-19T18:21:58.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.424+0000 7f07437fe640 1 -- 172.21.15.27:0/904311248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07440cdf90 msgr2=0x7f07440ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:58.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.424+0000 7f07437fe640 1 --2- 172.21.15.27:0/904311248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07440cdf90 0x7f07440ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.424+0000 7f07437fe640 1 -- 172.21.15.27:0/904311248 >> v1:172.21.15.192:6789/0 conn(0x7f074415d240 legacy=0x7f07441618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:58.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.425+0000 7f07437fe640 1 -- 172.21.15.27:0/904311248 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f074406a880 con 0x7f07440d1e00 2023-12-19T18:21:58.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.425+0000 7f07427fc640 1 --2- 172.21.15.27:0/904311248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07440cdf90 0x7f07440ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:21:58.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.426+0000 7f07437fe640 1 --2- 172.21.15.27:0/904311248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07440d1e00 0x7f07440d2290 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f073c0097a0 tx=0x7f073c02b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72f7eaf398e6430f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:58.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.427+0000 7f0741ffb640 1 -- 172.21.15.27:0/904311248 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c030800 con 0x7f07440d1e00 2023-12-19T18:21:58.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.427+0000 7f0741ffb640 1 -- 172.21.15.27:0/904311248 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073c030960 con 0x7f07440d1e00 2023-12-19T18:21:58.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.427+0000 7f0741ffb640 1 -- 172.21.15.27:0/904311248 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c030cf0 con 0x7f07440d1e00 2023-12-19T18:21:58.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.427+0000 7f0749412640 1 -- 172.21.15.27:0/904311248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07440d1e00 msgr2=0x7f07440d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:58.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.427+0000 7f0749412640 1 --2- 172.21.15.27:0/904311248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07440d1e00 0x7f07440d2290 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f073c0097a0 tx=0x7f073c02b940 comp rx=0 tx=0).stop 2023-12-19T18:21:58.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.427+0000 7f0749412640 1 -- 172.21.15.27:0/904311248 shutdown_connections 2023-12-19T18:21:58.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.427+0000 7f0749412640 1 --2- 172.21.15.27:0/904311248 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07440d1e00 0x7f07440d2290 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.427+0000 7f0749412640 1 --2- 172.21.15.27:0/904311248 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07440cdf90 0x7f07440ce3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.427+0000 7f0749412640 1 -- 172.21.15.27:0/904311248 >> 172.21.15.27:0/904311248 conn(0x7f07440d31e0 msgr2=0x7f07440d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:58.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.428+0000 7f0749412640 1 -- 172.21.15.27:0/904311248 shutdown_connections 2023-12-19T18:21:58.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.428+0000 7f0749412640 1 -- 172.21.15.27:0/904311248 wait complete. 2023-12-19T18:21:58.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.429+0000 7f0749412640 1 Processor -- start 2023-12-19T18:21:58.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.430+0000 7f0749412640 1 -- start start 2023-12-19T18:21:58.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.431+0000 7f0749412640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07440cdf90 0x7f07440c9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.431+0000 7f0749412640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07440d1e00 0x7f07440cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.431+0000 7f0749412640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f074415d240 0x7f07440cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.431+0000 7f0749412640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f074414db30 con 0x7f074415d240 2023-12-19T18:21:58.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.431+0000 7f0749412640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0744153be0 con 0x7f07440cdf90 2023-12-19T18:21:58.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.431+0000 7f0749412640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0744153620 con 0x7f07440d1e00 2023-12-19T18:21:58.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.431+0000 7f07437fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f074415d240 0x7f07440cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:58.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.431+0000 7f07437fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f074415d240 0x7f07440cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37108/0 (socket says 172.21.15.27:37108) 2023-12-19T18:21:58.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.431+0000 7f07437fe640 1 -- 172.21.15.27:0/807455282 learned_addr learned my addr 172.21.15.27:0/807455282 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:58.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.431+0000 7f0742ffd640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07440cdf90 0x7f07440c9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:58.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.431+0000 7f07427fc640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07440d1e00 0x7f07440cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:58.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.432+0000 7f07437fe640 1 -- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07440d1e00 msgr2=0x7f07440cbd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:58.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.432+0000 7f07437fe640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07440d1e00 0x7f07440cbd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.432+0000 7f07437fe640 1 -- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07440cdf90 msgr2=0x7f07440c9450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:58.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.432+0000 7f07437fe640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07440cdf90 0x7f07440c9450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.432+0000 7f07437fe640 1 -- 172.21.15.27:0/807455282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c009400 con 0x7f074415d240 2023-12-19T18:21:58.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.432+0000 7f07437fe640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f074415d240 0x7f07440cc2c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f073c0098c0 tx=0x7f073c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:58.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.432+0000 7f073b7fe640 1 -- 172.21.15.27:0/807455282 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c03e040 con 0x7f074415d240 2023-12-19T18:21:58.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.432+0000 7f073b7fe640 1 -- 172.21.15.27:0/807455282 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f073c039c10 con 0x7f074415d240 2023-12-19T18:21:58.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.432+0000 7f073b7fe640 1 -- 172.21.15.27:0/807455282 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f073c006e10 con 0x7f074415d240 2023-12-19T18:21:58.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.432+0000 7f0749412640 1 -- 172.21.15.27:0/807455282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c009400 con 0x7f074415d240 2023-12-19T18:21:58.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.433+0000 7f0749412640 1 -- 172.21.15.27:0/807455282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07440ccf30 con 0x7f074415d240 2023-12-19T18:21:58.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.434+0000 7f0749412640 1 -- 172.21.15.27:0/807455282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07440665d0 con 0x7f074415d240 2023-12-19T18:21:58.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.435+0000 7f073b7fe640 1 -- 172.21.15.27:0/807455282 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f073c04d3b0 con 0x7f074415d240 2023-12-19T18:21:58.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.435+0000 7f073b7fe640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f072406a430 0x7f072406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.436+0000 7f073b7fe640 1 -- 172.21.15.27:0/807455282 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f073c0aafb0 con 0x7f074415d240 2023-12-19T18:21:58.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.436+0000 7f0742ffd640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f072406a430 0x7f072406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:58.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.437+0000 7f0742ffd640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f072406a430 0x7f072406c8f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0734001560 tx=0x7f0734009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:58.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.440+0000 7f073b7fe640 1 -- 172.21.15.27:0/807455282 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f073c076500 con 0x7f074415d240 2023-12-19T18:21:58.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.637+0000 7f0749412640 1 -- 172.21.15.27:0/807455282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07440cd120 con 0x7f074415d240 2023-12-19T18:21:58.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.640+0000 7f073b7fe640 1 -- 172.21.15.27:0/807455282 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16116 (secure 0 0 0) 0x7f073c079b20 con 0x7f074415d240 2023-12-19T18:21:58.640 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:58.640 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":123,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:21:41.838074+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:21:06.258281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T18:21:07.308887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:21:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.643+0000 7f0749412640 1 -- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f072406a430 msgr2=0x7f072406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.643+0000 7f0749412640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f072406a430 0x7f072406c8f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f0734001560 tx=0x7f0734009040 comp rx=0 tx=0).stop 2023-12-19T18:21:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.643+0000 7f0749412640 1 -- 172.21.15.27:0/807455282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f074415d240 msgr2=0x7f07440cc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:58.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.643+0000 7f0749412640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f074415d240 0x7f07440cc2c0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f073c0098c0 tx=0x7f073c004730 comp rx=0 tx=0).stop 2023-12-19T18:21:58.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.643+0000 7f0749412640 1 -- 172.21.15.27:0/807455282 shutdown_connections 2023-12-19T18:21:58.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.643+0000 7f0749412640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f072406a430 0x7f072406c8f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.643+0000 7f0749412640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f074415d240 0x7f07440cc2c0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.643+0000 7f0749412640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07440d1e00 0x7f07440cbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.643+0000 7f0749412640 1 --2- 172.21.15.27:0/807455282 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07440cdf90 0x7f07440c9450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.643+0000 7f0749412640 1 -- 172.21.15.27:0/807455282 >> 172.21.15.27:0/807455282 conn(0x7f07440d31e0 msgr2=0x7f0744160220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:58.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.643+0000 7f0749412640 1 -- 172.21.15.27:0/807455282 shutdown_connections 2023-12-19T18:21:58.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.643+0000 7f0749412640 1 -- 172.21.15.27:0/807455282 wait complete. 2023-12-19T18:21:58.664 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:21:58.665 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:21:58.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.903+0000 7f9a3d234640 1 Processor -- start 2023-12-19T18:21:58.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.903+0000 7f9a3d234640 1 -- start start 2023-12-19T18:21:58.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.905+0000 7f9a3d234640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9a380cb300 0x7f9a380cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.905+0000 7f9a3d234640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a380cbc20 con 0x7f9a38161010 2023-12-19T18:21:58.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.905+0000 7f9a3d234640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a38145d60 con 0x7f9a380c8f90 2023-12-19T18:21:58.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.905+0000 7f9a3d234640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a380c4480 con 0x7f9a380cb300 2023-12-19T18:21:58.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.905+0000 7f9a3ca33640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9a38161010 0x7f9a380cd090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55788/0 (socket says 172.21.15.27:55788) 2023-12-19T18:21:58.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.905+0000 7f9a3ca33640 1 -- 172.21.15.27:0/4278338596 learned_addr learned my addr 172.21.15.27:0/4278338596 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:58.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a37fff640 1 --2- 172.21.15.27:0/4278338596 >> v2:172.21.15.192:3300/0 conn(0x7f9a380cb300 0x7f9a380cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:58.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884784091 0 0) 0x7f9a380cbc20 con 0x7f9a38161010 2023-12-19T18:21:58.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a24003520 con 0x7f9a38161010 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3326545494 0 0) 0x7f9a24003520 con 0x7f9a38161010 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 >> v2:172.21.15.192:3300/0 conn(0x7f9a380cb300 msgr2=0x7f9a380cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 --2- 172.21.15.27:0/4278338596 >> v2:172.21.15.192:3300/0 conn(0x7f9a380cb300 0x7f9a380cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 >> v1:172.21.15.192:6789/0 conn(0x7f9a380c8f90 legacy=0x7f9a38160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a38161510 con 0x7f9a38161010 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f9a300019f0 con 0x7f9a38161010 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a37fff640 1 --2- 172.21.15.27:0/4278338596 >> v2:172.21.15.192:3300/0 conn(0x7f9a380cb300 0x7f9a380cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 >> v1:172.21.15.27:6789/0 conn(0x7f9a38161010 legacy=0x7f9a380cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 --2- 172.21.15.27:0/4278338596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a24003970 0x7f9a24003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 --2- 172.21.15.27:0/4278338596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a240047c0 0x7f9a24004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 --2- 172.21.15.27:0/4278338596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a24005180 0x7f9a2400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2400c0a0 con 0x7f9a24003970 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2400c210 con 0x7f9a24005180 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.906+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2400c3d0 con 0x7f9a240047c0 2023-12-19T18:21:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.907+0000 7f9a37fff640 1 --2- 172.21.15.27:0/4278338596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a24003970 0x7f9a24003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.907+0000 7f9a2ffff640 1 --2- 172.21.15.27:0/4278338596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a240047c0 0x7f9a24004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.907+0000 7f9a37fff640 1 -- 172.21.15.27:0/4278338596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a240047c0 msgr2=0x7f9a24004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.907+0000 7f9a37fff640 1 --2- 172.21.15.27:0/4278338596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a240047c0 0x7f9a24004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.907+0000 7f9a3ca33640 1 --2- 172.21.15.27:0/4278338596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a24005180 0x7f9a2400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.907+0000 7f9a37fff640 1 -- 172.21.15.27:0/4278338596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a24005180 msgr2=0x7f9a2400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.907+0000 7f9a37fff640 1 --2- 172.21.15.27:0/4278338596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a24005180 0x7f9a2400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.907+0000 7f9a37fff640 1 -- 172.21.15.27:0/4278338596 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a24003520 con 0x7f9a24003970 2023-12-19T18:21:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.907+0000 7f9a3ca33640 1 -- 172.21.15.27:0/4278338596 reap_dead start 2023-12-19T18:21:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.907+0000 7f9a2ffff640 1 --2- 172.21.15.27:0/4278338596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a240047c0 0x7f9a24004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.908+0000 7f9a37fff640 1 --2- 172.21.15.27:0/4278338596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a24003970 0x7f9a24003d90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f9a280098a0 tx=0x7f9a240057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c9d42095e8103 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.909+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28004230 con 0x7f9a24003970 2023-12-19T18:21:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.909+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a28004390 con 0x7f9a24003970 2023-12-19T18:21:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.909+0000 7f9a377fe640 1 -- 172.21.15.27:0/4278338596 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a28003830 con 0x7f9a24003970 2023-12-19T18:21:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.909+0000 7f9a3d234640 1 -- 172.21.15.27:0/4278338596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a24003970 msgr2=0x7f9a24003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.909+0000 7f9a3d234640 1 --2- 172.21.15.27:0/4278338596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a24003970 0x7f9a24003d90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f9a280098a0 tx=0x7f9a240057f0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.909+0000 7f9a3d234640 1 -- 172.21.15.27:0/4278338596 shutdown_connections 2023-12-19T18:21:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.909+0000 7f9a3d234640 1 --2- 172.21.15.27:0/4278338596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a24003970 0x7f9a24003d90 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.909+0000 7f9a3d234640 1 -- 172.21.15.27:0/4278338596 >> 172.21.15.27:0/4278338596 conn(0x7f9a38069c70 msgr2=0x7f9a3806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.909+0000 7f9a3d234640 1 -- 172.21.15.27:0/4278338596 shutdown_connections 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.909+0000 7f9a3d234640 1 -- 172.21.15.27:0/4278338596 wait complete. 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.911+0000 7f9a3d234640 1 Processor -- start 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.911+0000 7f9a3d234640 1 -- start start 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.912+0000 7f9a3d234640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a24003970 0x7f9a380cebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.912+0000 7f9a3d234640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38140430 0x7f9a381408b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.912+0000 7f9a3d234640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380b3850 0x7f9a380b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.912+0000 7f9a3d234640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a381f6740 con 0x7f9a38140430 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.912+0000 7f9a3d234640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a38140df0 con 0x7f9a24003970 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.912+0000 7f9a3d234640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a380b2400 con 0x7f9a380b3850 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.913+0000 7f9a2ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38140430 0x7f9a381408b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.913+0000 7f9a2ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38140430 0x7f9a381408b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37120/0 (socket says 172.21.15.27:37120) 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.913+0000 7f9a2ffff640 1 -- 172.21.15.27:0/526527596 learned_addr learned my addr 172.21.15.27:0/526527596 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.913+0000 7f9a2ffff640 1 -- 172.21.15.27:0/526527596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380b3850 msgr2=0x7f9a380b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:58.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.913+0000 7f9a2ffff640 1 --2- 172.21.15.27:0/526527596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380b3850 0x7f9a380b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.913+0000 7f9a2ffff640 1 -- 172.21.15.27:0/526527596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a24003970 msgr2=0x7f9a380cebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:58.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.913+0000 7f9a2ffff640 1 --2- 172.21.15.27:0/526527596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a24003970 0x7f9a380cebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:58.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.913+0000 7f9a2ffff640 1 -- 172.21.15.27:0/526527596 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a381f5b30 con 0x7f9a38140430 2023-12-19T18:21:58.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.913+0000 7f9a2ffff640 1 --2- 172.21.15.27:0/526527596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38140430 0x7f9a381408b0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9a2000d480 tx=0x7f9a2000d940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:58.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.914+0000 7f9a357fa640 1 -- 172.21.15.27:0/526527596 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a20005480 con 0x7f9a38140430 2023-12-19T18:21:58.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.914+0000 7f9a357fa640 1 -- 172.21.15.27:0/526527596 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a20003610 con 0x7f9a38140430 2023-12-19T18:21:58.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.914+0000 7f9a357fa640 1 -- 172.21.15.27:0/526527596 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a20003a00 con 0x7f9a38140430 2023-12-19T18:21:58.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.914+0000 7f9a3d234640 1 -- 172.21.15.27:0/526527596 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a381f5e10 con 0x7f9a38140430 2023-12-19T18:21:58.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.914+0000 7f9a3d234640 1 -- 172.21.15.27:0/526527596 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a381f63d0 con 0x7f9a38140430 2023-12-19T18:21:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.915+0000 7f9a3d234640 1 -- 172.21.15.27:0/526527596 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a381f6590 con 0x7f9a38140430 2023-12-19T18:21:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.916+0000 7f9a357fa640 1 -- 172.21.15.27:0/526527596 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9a20003b60 con 0x7f9a38140430 2023-12-19T18:21:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.917+0000 7f9a357fa640 1 --2- 172.21.15.27:0/526527596 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a1806a460 0x7f9a1806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.917+0000 7f9a357fa640 1 -- 172.21.15.27:0/526527596 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a20088760 con 0x7f9a38140430 2023-12-19T18:21:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.918+0000 7f9a37fff640 1 --2- 172.21.15.27:0/526527596 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a1806a460 0x7f9a1806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:58.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.919+0000 7f9a37fff640 1 --2- 172.21.15.27:0/526527596 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a1806a460 0x7f9a1806c920 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9a28035000 tx=0x7f9a28030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:58.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:58.921+0000 7f9a357fa640 1 -- 172.21.15.27:0/526527596 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9a20053ea0 con 0x7f9a38140430 2023-12-19T18:21:59.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.147+0000 7f9a3d234640 1 -- 172.21.15.27:0/526527596 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a3804e840 con 0x7f9a38140430 2023-12-19T18:21:59.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.149+0000 7f9a357fa640 1 -- 172.21.15.27:0/526527596 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f9a200574c0 con 0x7f9a38140430 2023-12-19T18:21:59.149 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:59.149 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2779436922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2779436922},{"type":"v1","addr":"172.21.15.192:6836","nonce":2779436922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3121062779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3121062779},{"type":"v1","addr":"172.21.15.192:6837","nonce":3121062779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5357,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2032553869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2032553869},{"type":"v1","addr":"172.21.15.27:6835","nonce":2032553869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:21:08.281615+0000","modified":"2023-12-19T18:21:11.800655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/252970177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":252970177},{"type":"v1","addr":"172.21.15.27:6834","nonce":252970177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T18:21:59.149 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 46 2023-12-19T18:21:59.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.152+0000 7f9a3d234640 1 -- 172.21.15.27:0/526527596 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a1806a460 msgr2=0x7f9a1806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:59.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.152+0000 7f9a3d234640 1 --2- 172.21.15.27:0/526527596 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a1806a460 0x7f9a1806c920 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f9a28035000 tx=0x7f9a28030000 comp rx=0 tx=0).stop 2023-12-19T18:21:59.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.152+0000 7f9a3d234640 1 -- 172.21.15.27:0/526527596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38140430 msgr2=0x7f9a381408b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:59.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.152+0000 7f9a3d234640 1 --2- 172.21.15.27:0/526527596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38140430 0x7f9a381408b0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f9a2000d480 tx=0x7f9a2000d940 comp rx=0 tx=0).stop 2023-12-19T18:21:59.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.152+0000 7f9a3d234640 1 -- 172.21.15.27:0/526527596 shutdown_connections 2023-12-19T18:21:59.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.152+0000 7f9a3d234640 1 --2- 172.21.15.27:0/526527596 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a1806a460 0x7f9a1806c920 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.152+0000 7f9a3d234640 1 --2- 172.21.15.27:0/526527596 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380b3850 0x7f9a380b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.152+0000 7f9a3d234640 1 --2- 172.21.15.27:0/526527596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a38140430 0x7f9a381408b0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.152+0000 7f9a3d234640 1 --2- 172.21.15.27:0/526527596 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a24003970 0x7f9a380cebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.152+0000 7f9a3d234640 1 -- 172.21.15.27:0/526527596 >> 172.21.15.27:0/526527596 conn(0x7f9a38069c70 msgr2=0x7f9a3806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:59.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.152+0000 7f9a3d234640 1 -- 172.21.15.27:0/526527596 shutdown_connections 2023-12-19T18:21:59.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.152+0000 7f9a3d234640 1 -- 172.21.15.27:0/526527596 wait complete. 2023-12-19T18:21:59.172 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:21:59.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.391+0000 7fb191915640 1 Processor -- start 2023-12-19T18:21:59.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.392+0000 7fb191915640 1 -- start start 2023-12-19T18:21:59.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.393+0000 7fb191915640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb18c15ee30 0x7fb18c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.393+0000 7fb191915640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb18c0afeb0 0x7fb18c0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.393+0000 7fb191915640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb18c0cb350 con 0x7fb18c1553a0 2023-12-19T18:21:59.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.393+0000 7fb191915640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c0cbcf0 con 0x7fb18c0afeb0 2023-12-19T18:21:59.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.393+0000 7fb191915640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb18c0cb630 con 0x7fb18c15ee30 2023-12-19T18:21:59.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.393+0000 7fb191114640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb18c0afeb0 0x7fb18c0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.393+0000 7fb191114640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb18c0afeb0 0x7fb18c0c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37122/0 (socket says 172.21.15.27:37122) 2023-12-19T18:21:59.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.393+0000 7fb191114640 1 -- 172.21.15.27:0/724281410 learned_addr learned my addr 172.21.15.27:0/724281410 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:59.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.394+0000 7fb18bfff640 1 --2- 172.21.15.27:0/724281410 >> v2:172.21.15.192:3300/0 conn(0x7fb18c15ee30 0x7fb18c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.394+0000 7fb191114640 1 -- 172.21.15.27:0/724281410 >> v2:172.21.15.192:3300/0 conn(0x7fb18c15ee30 msgr2=0x7fb18c0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:59.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.394+0000 7fb191114640 1 --2- 172.21.15.27:0/724281410 >> v2:172.21.15.192:3300/0 conn(0x7fb18c15ee30 0x7fb18c0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.394+0000 7fb191114640 1 -- 172.21.15.27:0/724281410 >> v1:172.21.15.192:6789/0 conn(0x7fb18c1553a0 legacy=0x7fb18c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:59.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.394+0000 7fb191114640 1 -- 172.21.15.27:0/724281410 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb18c0cb910 con 0x7fb18c0afeb0 2023-12-19T18:21:59.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.394+0000 7fb18bfff640 1 --2- 172.21.15.27:0/724281410 >> v2:172.21.15.192:3300/0 conn(0x7fb18c15ee30 0x7fb18c0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.395+0000 7fb191114640 1 --2- 172.21.15.27:0/724281410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18c0afeb0 0x7fb18c0c39d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fb18c06d4a0 tx=0x7fb18402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=183266adc91d12c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.395+0000 7fb18b7fe640 1 -- 172.21.15.27:0/724281410 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb18402c9f0 con 0x7fb18c0afeb0 2023-12-19T18:21:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.395+0000 7fb18b7fe640 1 -- 172.21.15.27:0/724281410 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb18402cb50 con 0x7fb18c0afeb0 2023-12-19T18:21:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.395+0000 7fb18b7fe640 1 -- 172.21.15.27:0/724281410 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1840343f0 con 0x7fb18c0afeb0 2023-12-19T18:21:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.396+0000 7fb191915640 1 -- 172.21.15.27:0/724281410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18c0afeb0 msgr2=0x7fb18c0c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.396+0000 7fb191915640 1 --2- 172.21.15.27:0/724281410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18c0afeb0 0x7fb18c0c39d0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fb18c06d4a0 tx=0x7fb18402be50 comp rx=0 tx=0).stop 2023-12-19T18:21:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.396+0000 7fb191915640 1 -- 172.21.15.27:0/724281410 shutdown_connections 2023-12-19T18:21:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.396+0000 7fb191915640 1 --2- 172.21.15.27:0/724281410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18c0afeb0 0x7fb18c0c39d0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.396+0000 7fb191915640 1 --2- 172.21.15.27:0/724281410 >> v2:172.21.15.192:3300/0 conn(0x7fb18c15ee30 0x7fb18c0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.396+0000 7fb191915640 1 -- 172.21.15.27:0/724281410 >> 172.21.15.27:0/724281410 conn(0x7fb18c069c70 msgr2=0x7fb18c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:59.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.396+0000 7fb191915640 1 -- 172.21.15.27:0/724281410 shutdown_connections 2023-12-19T18:21:59.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.396+0000 7fb191915640 1 -- 172.21.15.27:0/724281410 wait complete. 2023-12-19T18:21:59.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.398+0000 7fb191915640 1 Processor -- start 2023-12-19T18:21:59.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.398+0000 7fb191915640 1 -- start start 2023-12-19T18:21:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb191915640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18c0afeb0 0x7fb18c0ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb191915640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18c1553a0 0x7fb18c15e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb191915640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c15ee30 0x7fb18c14c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb191915640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c0ce540 con 0x7fb18c1553a0 2023-12-19T18:21:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb191915640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c15e1d0 con 0x7fb18c0afeb0 2023-12-19T18:21:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb191915640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb18c15d5c0 con 0x7fb18c15ee30 2023-12-19T18:21:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb18bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18c1553a0 0x7fb18c15e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb18bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18c1553a0 0x7fb18c15e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37138/0 (socket says 172.21.15.27:37138) 2023-12-19T18:21:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb18bfff640 1 -- 172.21.15.27:0/3450726503 learned_addr learned my addr 172.21.15.27:0/3450726503 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:59.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb18bfff640 1 -- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c15ee30 msgr2=0x7fb18c14c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:59.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb190913640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18c0afeb0 0x7fb18c0ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb191114640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c15ee30 0x7fb18c14c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb18bfff640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c15ee30 0x7fb18c14c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb18bfff640 1 -- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18c0afeb0 msgr2=0x7fb18c0ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:59.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb18bfff640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18c0afeb0 0x7fb18c0ce000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.399+0000 7fb18bfff640 1 -- 172.21.15.27:0/3450726503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb184009b30 con 0x7fb18c1553a0 2023-12-19T18:21:59.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.400+0000 7fb18bfff640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18c1553a0 0x7fb18c15e660 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb17800c7f0 tx=0x7fb17800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:59.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.400+0000 7fb1897fa640 1 -- 172.21.15.27:0/3450726503 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb178007820 con 0x7fb18c1553a0 2023-12-19T18:21:59.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.400+0000 7fb1897fa640 1 -- 172.21.15.27:0/3450726503 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb178007980 con 0x7fb18c1553a0 2023-12-19T18:21:59.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.400+0000 7fb1897fa640 1 -- 172.21.15.27:0/3450726503 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb178007cf0 con 0x7fb18c1553a0 2023-12-19T18:21:59.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.400+0000 7fb191915640 1 -- 172.21.15.27:0/3450726503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18c15d8a0 con 0x7fb18c1553a0 2023-12-19T18:21:59.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.400+0000 7fb191915640 1 -- 172.21.15.27:0/3450726503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18c15de60 con 0x7fb18c1553a0 2023-12-19T18:21:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.401+0000 7fb191915640 1 -- 172.21.15.27:0/3450726503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb18c15dff0 con 0x7fb18c1553a0 2023-12-19T18:21:59.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.402+0000 7fb1897fa640 1 -- 172.21.15.27:0/3450726503 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb1780113f0 con 0x7fb18c1553a0 2023-12-19T18:21:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.403+0000 7fb1897fa640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb17006a430 0x7fb17006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.403+0000 7fb1897fa640 1 -- 172.21.15.27:0/3450726503 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb178088ce0 con 0x7fb18c1553a0 2023-12-19T18:21:59.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.404+0000 7fb190913640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb17006a430 0x7fb17006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.405+0000 7fb190913640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb17006a430 0x7fb17006c8f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb180003d90 tx=0x7fb180009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:59.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.407+0000 7fb1897fa640 1 -- 172.21.15.27:0/3450726503 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb178054230 con 0x7fb18c1553a0 2023-12-19T18:21:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.627+0000 7fb191915640 1 -- 172.21.15.27:0/3450726503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb18c04e840 con 0x7fb18c1553a0 2023-12-19T18:21:59.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.628+0000 7fb1897fa640 1 -- 172.21.15.27:0/3450726503 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4714 (secure 0 0 0) 0x7fb17801f090 con 0x7fb18c1553a0 2023-12-19T18:21:59.628 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:21:59.628 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2779436922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2779436922},{"type":"v1","addr":"172.21.15.192:6836","nonce":2779436922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3121062779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3121062779},{"type":"v1","addr":"172.21.15.192:6837","nonce":3121062779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5357,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2032553869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2032553869},{"type":"v1","addr":"172.21.15.27:6835","nonce":2032553869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:21:08.281615+0000","modified":"2023-12-19T18:21:11.800655+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6885},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6885":{"gid":6885,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/252970177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":252970177},{"type":"v1","addr":"172.21.15.27:6834","nonce":252970177}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-19T18:21:59.629 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 46 2023-12-19T18:21:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.630+0000 7fb191915640 1 -- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb17006a430 msgr2=0x7fb17006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.630+0000 7fb191915640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb17006a430 0x7fb17006c8f0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fb180003d90 tx=0x7fb180009040 comp rx=0 tx=0).stop 2023-12-19T18:21:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.631+0000 7fb191915640 1 -- 172.21.15.27:0/3450726503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18c1553a0 msgr2=0x7fb18c15e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.631+0000 7fb191915640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18c1553a0 0x7fb18c15e660 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb17800c7f0 tx=0x7fb17800ccb0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.631+0000 7fb191915640 1 -- 172.21.15.27:0/3450726503 shutdown_connections 2023-12-19T18:21:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.631+0000 7fb191915640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb17006a430 0x7fb17006c8f0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.631+0000 7fb191915640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb18c15ee30 0x7fb18c14c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.631+0000 7fb191915640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb18c1553a0 0x7fb18c15e660 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.631+0000 7fb191915640 1 --2- 172.21.15.27:0/3450726503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb18c0afeb0 0x7fb18c0ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.631+0000 7fb191915640 1 -- 172.21.15.27:0/3450726503 >> 172.21.15.27:0/3450726503 conn(0x7fb18c069c70 msgr2=0x7fb18c0c4720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:59.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.631+0000 7fb191915640 1 -- 172.21.15.27:0/3450726503 shutdown_connections 2023-12-19T18:21:59.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.631+0000 7fb191915640 1 -- 172.21.15.27:0/3450726503 wait complete. 2023-12-19T18:21:59.650 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:21:59.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.875+0000 7f7363100640 1 Processor -- start 2023-12-19T18:21:59.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.875+0000 7f7363100640 1 -- start start 2023-12-19T18:21:59.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.877+0000 7f7363100640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c0ce420 0x7f735c0d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.877+0000 7f7363100640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735c0d1970 con 0x7f735c0cdf90 2023-12-19T18:21:59.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.877+0000 7f7363100640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f735c06a5e0 con 0x7f735c15d240 2023-12-19T18:21:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.877+0000 7f7363100640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0ce8b0 con 0x7f735c0ce420 2023-12-19T18:21:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.877+0000 7f735bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f735c0cdf90 0x7f735c0d1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55790/0 (socket says 172.21.15.27:55790) 2023-12-19T18:21:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.877+0000 7f735bfff640 1 -- 172.21.15.27:0/2058179249 learned_addr learned my addr 172.21.15.27:0/2058179249 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:21:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.877+0000 7f736169d640 1 --2- 172.21.15.27:0/2058179249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c0ce420 0x7f735c0d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.877+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2999518594 0 0) 0x7f735c0d1970 con 0x7f735c0cdf90 2023-12-19T18:21:59.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.877+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7344003520 con 0x7f735c0cdf90 2023-12-19T18:21:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3057325267 0 0) 0x7f7344003520 con 0x7f735c0cdf90 2023-12-19T18:21:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c0ce420 msgr2=0x7f735c0d2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735b7fe640 1 --2- 172.21.15.27:0/2058179249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c0ce420 0x7f735c0d2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 >> v1:172.21.15.192:6789/0 conn(0x7f735c15d240 legacy=0x7f735c1618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c06a880 con 0x7f735c0cdf90 2023-12-19T18:21:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f73500019f0 con 0x7f735c0cdf90 2023-12-19T18:21:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 >> v1:172.21.15.27:6789/0 conn(0x7f735c0cdf90 legacy=0x7f735c0d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:59.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735b7fe640 1 --2- 172.21.15.27:0/2058179249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73440039a0 0x7f7344003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735b7fe640 1 --2- 172.21.15.27:0/2058179249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73440047c0 0x7f7344004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735b7fe640 1 --2- 172.21.15.27:0/2058179249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c0ce420 0x7f7344005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c0ce8b0 con 0x7f73440047c0 2023-12-19T18:21:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734400bac0 con 0x7f73440039a0 2023-12-19T18:21:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734400bc80 con 0x7f735c0ce420 2023-12-19T18:21:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.878+0000 7f735bfff640 1 --2- 172.21.15.27:0/2058179249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73440047c0 0x7f7344004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.879+0000 7f7360e9c640 1 --2- 172.21.15.27:0/2058179249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73440039a0 0x7f7344003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.879+0000 7f736169d640 1 --2- 172.21.15.27:0/2058179249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c0ce420 0x7f7344005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.879+0000 7f735bfff640 1 -- 172.21.15.27:0/2058179249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c0ce420 msgr2=0x7f7344005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.879+0000 7f735bfff640 1 --2- 172.21.15.27:0/2058179249 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c0ce420 0x7f7344005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.879+0000 7f735bfff640 1 -- 172.21.15.27:0/2058179249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73440039a0 msgr2=0x7f7344003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:21:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.879+0000 7f735bfff640 1 --2- 172.21.15.27:0/2058179249 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f73440039a0 0x7f7344003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.879+0000 7f735bfff640 1 -- 172.21.15.27:0/2058179249 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c06a880 con 0x7f73440047c0 2023-12-19T18:21:59.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.880+0000 7f735bfff640 1 --2- 172.21.15.27:0/2058179249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73440047c0 0x7f7344004c40 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f73500013d0 tx=0x7f73440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3714e2c19c48618 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:59.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.880+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7350034630 con 0x7f73440047c0 2023-12-19T18:21:59.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.880+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735002f070 con 0x7f73440047c0 2023-12-19T18:21:59.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.880+0000 7f735b7fe640 1 -- 172.21.15.27:0/2058179249 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73500349f0 con 0x7f73440047c0 2023-12-19T18:21:59.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.881+0000 7f7363100640 1 -- 172.21.15.27:0/2058179249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73440047c0 msgr2=0x7f7344004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.881+0000 7f7363100640 1 --2- 172.21.15.27:0/2058179249 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73440047c0 0x7f7344004c40 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f73500013d0 tx=0x7f73440057f0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.881+0000 7f736169d640 1 -- 172.21.15.27:0/2058179249 reap_dead start 2023-12-19T18:21:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.881+0000 7f7363100640 1 -- 172.21.15.27:0/2058179249 shutdown_connections 2023-12-19T18:21:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.881+0000 7f7363100640 1 -- 172.21.15.27:0/2058179249 >> 172.21.15.27:0/2058179249 conn(0x7f735c0d31e0 msgr2=0x7f735c0d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:21:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.881+0000 7f7363100640 1 -- 172.21.15.27:0/2058179249 shutdown_connections 2023-12-19T18:21:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.881+0000 7f7363100640 1 -- 172.21.15.27:0/2058179249 wait complete. 2023-12-19T18:21:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.883+0000 7f7363100640 1 Processor -- start 2023-12-19T18:21:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.883+0000 7f7363100640 1 -- start start 2023-12-19T18:21:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.884+0000 7f7363100640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73440047c0 0x7f735c1f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.884+0000 7f7363100640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f735c1f3d60 0x7f735c151590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.884+0000 7f7363100640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c154720 0x7f735c15edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.884+0000 7f7363100640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c152ec0 con 0x7f73440047c0 2023-12-19T18:21:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.884+0000 7f7363100640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c1f31f0 con 0x7f735c1f3d60 2023-12-19T18:21:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.884+0000 7f7363100640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f735c1f34d0 con 0x7f735c154720 2023-12-19T18:21:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.884+0000 7f7360e9c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73440047c0 0x7f735c1f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.884+0000 7f7360e9c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73440047c0 0x7f735c1f2ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37152/0 (socket says 172.21.15.27:37152) 2023-12-19T18:21:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.884+0000 7f7360e9c640 1 -- 172.21.15.27:0/615338394 learned_addr learned my addr 172.21.15.27:0/615338394 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:21:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.885+0000 7f7360e9c640 1 -- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c154720 msgr2=0x7f735c15edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.885+0000 7f736169d640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c154720 0x7f735c15edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.885+0000 7f735bfff640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f735c1f3d60 0x7f735c151590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.885+0000 7f7360e9c640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c154720 0x7f735c15edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.885+0000 7f7360e9c640 1 -- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f735c1f3d60 msgr2=0x7f735c151590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:21:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.885+0000 7f7360e9c640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f735c1f3d60 0x7f735c151590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:21:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.885+0000 7f7360e9c640 1 -- 172.21.15.27:0/615338394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c1f37b0 con 0x7f73440047c0 2023-12-19T18:21:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.885+0000 7f736169d640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c154720 0x7f735c15edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:21:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.885+0000 7f7360e9c640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73440047c0 0x7f735c1f2ba0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f734c00e550 tx=0x7f734c00ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.886+0000 7f73597fa640 1 -- 172.21.15.27:0/615338394 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c0052e0 con 0x7f73440047c0 2023-12-19T18:21:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.886+0000 7f73597fa640 1 -- 172.21.15.27:0/615338394 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734c005440 con 0x7f73440047c0 2023-12-19T18:21:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.886+0000 7f73597fa640 1 -- 172.21.15.27:0/615338394 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734c003810 con 0x7f73440047c0 2023-12-19T18:21:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.886+0000 7f7363100640 1 -- 172.21.15.27:0/615338394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735c151ad0 con 0x7f73440047c0 2023-12-19T18:21:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.886+0000 7f7363100640 1 -- 172.21.15.27:0/615338394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735c1f3a90 con 0x7f73440047c0 2023-12-19T18:21:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.887+0000 7f7363100640 1 -- 172.21.15.27:0/615338394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735c0d1580 con 0x7f73440047c0 2023-12-19T18:21:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.888+0000 7f73597fa640 1 -- 172.21.15.27:0/615338394 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f734c003970 con 0x7f73440047c0 2023-12-19T18:21:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.889+0000 7f73597fa640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f734806a460 0x7f734806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:21:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.889+0000 7f73597fa640 1 -- 172.21.15.27:0/615338394 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f734c0885b0 con 0x7f73440047c0 2023-12-19T18:21:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.890+0000 7f735bfff640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f734806a460 0x7f734806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:21:59.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.891+0000 7f735bfff640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f734806a460 0x7f734806c920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7350034000 tx=0x7f735002f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:21:59.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:21:59.894+0000 7f73597fa640 1 -- 172.21.15.27:0/615338394 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f734c053c30 con 0x7f73440047c0 2023-12-19T18:22:00.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.095+0000 7f7363100640 1 -- 172.21.15.27:0/615338394 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f735c068e90 con 0x7f73440047c0 2023-12-19T18:22:00.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.098+0000 7f73597fa640 1 -- 172.21.15.27:0/615338394 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16116 (secure 0 0 0) 0x7f734c057250 con 0x7f73440047c0 2023-12-19T18:22:00.098 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":123,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:21:41.838074+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:21:06.258281+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-19T18:21:07.308887+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:22:00.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.101+0000 7f7363100640 1 -- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f734806a460 msgr2=0x7f734806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:00.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.101+0000 7f7363100640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f734806a460 0x7f734806c920 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7350034000 tx=0x7f735002f000 comp rx=0 tx=0).stop 2023-12-19T18:22:00.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.101+0000 7f7363100640 1 -- 172.21.15.27:0/615338394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73440047c0 msgr2=0x7f735c1f2ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:00.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.101+0000 7f7363100640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73440047c0 0x7f735c1f2ba0 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f734c00e550 tx=0x7f734c00ea10 comp rx=0 tx=0).stop 2023-12-19T18:22:00.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.101+0000 7f7363100640 1 -- 172.21.15.27:0/615338394 shutdown_connections 2023-12-19T18:22:00.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.101+0000 7f7363100640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f734806a460 0x7f734806c920 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.101+0000 7f7363100640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f735c154720 0x7f735c15edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.101+0000 7f7363100640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f735c1f3d60 0x7f735c151590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.101+0000 7f7363100640 1 --2- 172.21.15.27:0/615338394 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73440047c0 0x7f735c1f2ba0 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.101+0000 7f7363100640 1 -- 172.21.15.27:0/615338394 >> 172.21.15.27:0/615338394 conn(0x7f735c0d31e0 msgr2=0x7f735c0b7d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:00.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.101+0000 7f7363100640 1 -- 172.21.15.27:0/615338394 shutdown_connections 2023-12-19T18:22:00.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.101+0000 7f7363100640 1 -- 172.21.15.27:0/615338394 wait complete. 2023-12-19T18:22:00.124 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:22:00.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.351+0000 7f7c4052e640 1 Processor -- start 2023-12-19T18:22:00.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.351+0000 7f7c4052e640 1 -- start start 2023-12-19T18:22:00.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.353+0000 7f7c4052e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7c3806cec0 0x7f7c3806d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:00.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.353+0000 7f7c4052e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7c380c9770 0x7f7c380cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.353+0000 7f7c4052e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c380cc720 con 0x7f7c380cd990 2023-12-19T18:22:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.353+0000 7f7c4052e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c3815a330 con 0x7f7c380c9770 2023-12-19T18:22:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.353+0000 7f7c4052e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c380b1a00 con 0x7f7c3806cec0 2023-12-19T18:22:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.353+0000 7f7c3dac9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7c380c9770 0x7f7c380cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.353+0000 7f7c3eacb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7c380cd990 0x7f7c380cde00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55792/0 (socket says 172.21.15.27:55792) 2023-12-19T18:22:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.353+0000 7f7c3eacb640 1 -- 172.21.15.27:0/1445069006 learned_addr learned my addr 172.21.15.27:0/1445069006 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.353+0000 7f7c3e2ca640 1 --2- 172.21.15.27:0/1445069006 >> v2:172.21.15.192:3300/0 conn(0x7f7c3806cec0 0x7f7c3806d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.354+0000 7f7c3dac9640 1 -- 172.21.15.27:0/1445069006 >> v2:172.21.15.192:3300/0 conn(0x7f7c3806cec0 msgr2=0x7f7c3806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.354+0000 7f7c3dac9640 1 --2- 172.21.15.27:0/1445069006 >> v2:172.21.15.192:3300/0 conn(0x7f7c3806cec0 0x7f7c3806d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.354+0000 7f7c3dac9640 1 -- 172.21.15.27:0/1445069006 >> v1:172.21.15.27:6789/0 conn(0x7f7c380cd990 legacy=0x7f7c380cde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:00.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.354+0000 7f7c3d2c8640 1 -- 172.21.15.27:0/1445069006 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2269001516 0 0) 0x7f7c380cc720 con 0x7f7c380cd990 2023-12-19T18:22:00.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.354+0000 7f7c3dac9640 1 -- 172.21.15.27:0/1445069006 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c3806d7e0 con 0x7f7c380c9770 2023-12-19T18:22:00.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.355+0000 7f7c3dac9640 1 --2- 172.21.15.27:0/1445069006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c380c9770 0x7f7c380cd450 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f7c38066860 tx=0x7f7c2802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4120eaa92b999941 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:00.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.355+0000 7f7c3d2c8640 1 -- 172.21.15.27:0/1445069006 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28031830 con 0x7f7c380c9770 2023-12-19T18:22:00.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.355+0000 7f7c3d2c8640 1 -- 172.21.15.27:0/1445069006 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c28031990 con 0x7f7c380c9770 2023-12-19T18:22:00.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.355+0000 7f7c3d2c8640 1 -- 172.21.15.27:0/1445069006 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c28031d20 con 0x7f7c380c9770 2023-12-19T18:22:00.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.356+0000 7f7c4052e640 1 -- 172.21.15.27:0/1445069006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c380c9770 msgr2=0x7f7c380cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:00.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.356+0000 7f7c4052e640 1 --2- 172.21.15.27:0/1445069006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c380c9770 0x7f7c380cd450 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f7c38066860 tx=0x7f7c2802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.356+0000 7f7c4052e640 1 -- 172.21.15.27:0/1445069006 shutdown_connections 2023-12-19T18:22:00.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.356+0000 7f7c4052e640 1 --2- 172.21.15.27:0/1445069006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c380c9770 0x7f7c380cd450 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.356+0000 7f7c4052e640 1 --2- 172.21.15.27:0/1445069006 >> v2:172.21.15.192:3300/0 conn(0x7f7c3806cec0 0x7f7c3806d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.356+0000 7f7c4052e640 1 -- 172.21.15.27:0/1445069006 >> 172.21.15.27:0/1445069006 conn(0x7f7c380ceb50 msgr2=0x7f7c380cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:00.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.356+0000 7f7c4052e640 1 -- 172.21.15.27:0/1445069006 shutdown_connections 2023-12-19T18:22:00.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.356+0000 7f7c4052e640 1 -- 172.21.15.27:0/1445069006 wait complete. 2023-12-19T18:22:00.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.358+0000 7f7c4052e640 1 Processor -- start 2023-12-19T18:22:00.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.358+0000 7f7c4052e640 1 -- start start 2023-12-19T18:22:00.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c4052e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3806cec0 0x7f7c3814d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:00.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c4052e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c380c9770 0x7f7c3814c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:00.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c4052e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c380cd990 0x7f7c380c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c4052e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380c54f0 con 0x7f7c3806cec0 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c4052e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3814ce70 con 0x7f7c380cd990 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c4052e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c380c4c50 con 0x7f7c380c9770 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c3e2ca640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3806cec0 0x7f7c3814d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c3e2ca640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3806cec0 0x7f7c3814d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37168/0 (socket says 172.21.15.27:37168) 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c3e2ca640 1 -- 172.21.15.27:0/2302614664 learned_addr learned my addr 172.21.15.27:0/2302614664 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c3e2ca640 1 -- 172.21.15.27:0/2302614664 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c380c9770 msgr2=0x7f7c3814c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c3e2ca640 1 --2- 172.21.15.27:0/2302614664 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c380c9770 0x7f7c3814c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c3e2ca640 1 -- 172.21.15.27:0/2302614664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c380cd990 msgr2=0x7f7c380c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c3e2ca640 1 --2- 172.21.15.27:0/2302614664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c380cd990 0x7f7c380c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.359+0000 7f7c3e2ca640 1 -- 172.21.15.27:0/2302614664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c28009400 con 0x7f7c3806cec0 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.360+0000 7f7c3e2ca640 1 --2- 172.21.15.27:0/2302614664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3806cec0 0x7f7c3814d060 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400d830 tx=0x7f7c3400dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.360+0000 7f7c2effd640 1 -- 172.21.15.27:0/2302614664 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3400b6c0 con 0x7f7c3806cec0 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.360+0000 7f7c2effd640 1 -- 172.21.15.27:0/2302614664 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c3400b820 con 0x7f7c3806cec0 2023-12-19T18:22:00.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.360+0000 7f7c2effd640 1 -- 172.21.15.27:0/2302614664 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c3400bbb0 con 0x7f7c3806cec0 2023-12-19T18:22:00.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.360+0000 7f7c4052e640 1 -- 172.21.15.27:0/2302614664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c380c4f30 con 0x7f7c3806cec0 2023-12-19T18:22:00.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.360+0000 7f7c4052e640 1 -- 172.21.15.27:0/2302614664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c380c2790 con 0x7f7c3806cec0 2023-12-19T18:22:00.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.361+0000 7f7c4052e640 1 -- 172.21.15.27:0/2302614664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c380c2920 con 0x7f7c3806cec0 2023-12-19T18:22:00.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.362+0000 7f7c2effd640 1 -- 172.21.15.27:0/2302614664 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7c3401e460 con 0x7f7c3806cec0 2023-12-19T18:22:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.363+0000 7f7c2effd640 1 --2- 172.21.15.27:0/2302614664 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c1c06a430 0x7f7c1c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.363+0000 7f7c2effd640 1 -- 172.21.15.27:0/2302614664 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c34089000 con 0x7f7c3806cec0 2023-12-19T18:22:00.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.364+0000 7f7c3dac9640 1 --2- 172.21.15.27:0/2302614664 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c1c06a430 0x7f7c1c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:00.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.365+0000 7f7c3dac9640 1 --2- 172.21.15.27:0/2302614664 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c1c06a430 0x7f7c1c06c8f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7c28036000 tx=0x7f7c28030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:00.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.367+0000 7f7c2effd640 1 -- 172.21.15.27:0/2302614664 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7c34054680 con 0x7f7c3806cec0 2023-12-19T18:22:00.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.591+0000 7f7c4052e640 1 -- 172.21.15.27:0/2302614664 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7c38068e90 con 0x7f7c3806cec0 2023-12-19T18:22:00.592 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:22:00.592+0000 7f8f3865c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:22:00.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.611+0000 7f7c2effd640 1 -- 172.21.15.27:0/2302614664 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7c34057ca0 con 0x7f7c3806cec0 2023-12-19T18:22:00.611 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:22:00.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.615+0000 7f7c4052e640 1 -- 172.21.15.27:0/2302614664 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c1c06a430 msgr2=0x7f7c1c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:00.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.615+0000 7f7c4052e640 1 --2- 172.21.15.27:0/2302614664 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c1c06a430 0x7f7c1c06c8f0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7c28036000 tx=0x7f7c28030000 comp rx=0 tx=0).stop 2023-12-19T18:22:00.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.615+0000 7f7c4052e640 1 -- 172.21.15.27:0/2302614664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3806cec0 msgr2=0x7f7c3814d060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:00.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.615+0000 7f7c4052e640 1 --2- 172.21.15.27:0/2302614664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3806cec0 0x7f7c3814d060 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f7c3400d830 tx=0x7f7c3400dcf0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.615+0000 7f7c4052e640 1 -- 172.21.15.27:0/2302614664 shutdown_connections 2023-12-19T18:22:00.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.615+0000 7f7c4052e640 1 --2- 172.21.15.27:0/2302614664 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c1c06a430 0x7f7c1c06c8f0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.615+0000 7f7c4052e640 1 --2- 172.21.15.27:0/2302614664 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c380cd990 0x7f7c380c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.615+0000 7f7c4052e640 1 --2- 172.21.15.27:0/2302614664 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c380c9770 0x7f7c3814c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.615+0000 7f7c4052e640 1 --2- 172.21.15.27:0/2302614664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3806cec0 0x7f7c3814d060 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.615+0000 7f7c4052e640 1 -- 172.21.15.27:0/2302614664 >> 172.21.15.27:0/2302614664 conn(0x7f7c380ceb50 msgr2=0x7f7c381590d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:00.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.615+0000 7f7c4052e640 1 -- 172.21.15.27:0/2302614664 shutdown_connections 2023-12-19T18:22:00.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.615+0000 7f7c4052e640 1 -- 172.21.15.27:0/2302614664 wait complete. 2023-12-19T18:22:00.629 INFO:tasks.ceph.mds.c.smithi027.stderr: -9366> 2023-12-19T18:21:13.704+0000 7f2f32e9a640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:22:00.640 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:22:00.688 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:22:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.848+0000 7fb933fff640 1 Processor -- start 2023-12-19T18:22:00.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.848+0000 7fb933fff640 1 -- start start 2023-12-19T18:22:00.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.850+0000 7fb933fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9340cdb30 0x7fb9340cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:00.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.850+0000 7fb933fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb9340d17a0 0x7fb9340d1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:00.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.850+0000 7fb933fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9340ce4d0 con 0x7fb93415cfc0 2023-12-19T18:22:00.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.850+0000 7fb933fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb934161250 con 0x7fb9340d17a0 2023-12-19T18:22:00.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.850+0000 7fb933fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb934161a50 con 0x7fb9340cdb30 2023-12-19T18:22:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.850+0000 7fb9337fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb9340d17a0 0x7fb9340d1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.850+0000 7fb9337fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb9340d17a0 0x7fb9340d1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:55098/0 (socket says 172.21.15.27:55098) 2023-12-19T18:22:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.850+0000 7fb9337fe640 1 -- 172.21.15.27:0/140852387 learned_addr learned my addr 172.21.15.27:0/140852387 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.851+0000 7fb92a7fc640 1 --2- 172.21.15.27:0/140852387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9340cdb30 0x7fb9340cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.851+0000 7fb9337fe640 1 -- 172.21.15.27:0/140852387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9340cdb30 msgr2=0x7fb9340cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.851+0000 7fb9337fe640 1 --2- 172.21.15.27:0/140852387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9340cdb30 0x7fb9340cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.851+0000 7fb9337fe640 1 -- 172.21.15.27:0/140852387 >> v1:172.21.15.192:6789/0 conn(0x7fb93415cfc0 legacy=0x7fb934160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.851+0000 7fb9337fe640 1 -- 172.21.15.27:0/140852387 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb934161490 con 0x7fb9340d17a0 2023-12-19T18:22:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.853+0000 7fb9337fe640 1 --2- 172.21.15.27:0/140852387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9340d17a0 0x7fb9340d1bf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009e50 tx=0x7fb92c02bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=596e0f1cab8d97b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.853+0000 7fb9327fc640 1 -- 172.21.15.27:0/140852387 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c02ca70 con 0x7fb9340d17a0 2023-12-19T18:22:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.853+0000 7fb9327fc640 1 -- 172.21.15.27:0/140852387 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92c02cbd0 con 0x7fb9340d17a0 2023-12-19T18:22:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.853+0000 7fb9327fc640 1 -- 172.21.15.27:0/140852387 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0343f0 con 0x7fb9340d17a0 2023-12-19T18:22:00.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.854+0000 7fb933fff640 1 -- 172.21.15.27:0/140852387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9340d17a0 msgr2=0x7fb9340d1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:00.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.854+0000 7fb933fff640 1 --2- 172.21.15.27:0/140852387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9340d17a0 0x7fb9340d1bf0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb92c009e50 tx=0x7fb92c02bed0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.854+0000 7fb933fff640 1 -- 172.21.15.27:0/140852387 shutdown_connections 2023-12-19T18:22:00.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.854+0000 7fb933fff640 1 --2- 172.21.15.27:0/140852387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9340d17a0 0x7fb9340d1bf0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.854+0000 7fb933fff640 1 --2- 172.21.15.27:0/140852387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9340cdb30 0x7fb9340cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.854+0000 7fb933fff640 1 -- 172.21.15.27:0/140852387 >> 172.21.15.27:0/140852387 conn(0x7fb934072e20 msgr2=0x7fb934073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.854+0000 7fb933fff640 1 -- 172.21.15.27:0/140852387 shutdown_connections 2023-12-19T18:22:00.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.854+0000 7fb933fff640 1 -- 172.21.15.27:0/140852387 wait complete. 2023-12-19T18:22:00.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.856+0000 7fb933fff640 1 Processor -- start 2023-12-19T18:22:00.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.856+0000 7fb933fff640 1 -- start start 2023-12-19T18:22:00.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.857+0000 7fb933fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9340cdb30 0x7fb934156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:00.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.857+0000 7fb933fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9340d17a0 0x7fb9341f59b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:00.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.857+0000 7fb933fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb93415cfc0 0x7fb934154f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:00.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.857+0000 7fb933fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9341f6950 con 0x7fb93415cfc0 2023-12-19T18:22:00.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.857+0000 7fb933fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9340ca060 con 0x7fb9340d17a0 2023-12-19T18:22:00.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.857+0000 7fb933fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb934155490 con 0x7fb9340cdb30 2023-12-19T18:22:00.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.857+0000 7fb9337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb93415cfc0 0x7fb934154f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:00.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.857+0000 7fb9337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb93415cfc0 0x7fb934154f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37202/0 (socket says 172.21.15.27:37202) 2023-12-19T18:22:00.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.857+0000 7fb9337fe640 1 -- 172.21.15.27:0/3661505336 learned_addr learned my addr 172.21.15.27:0/3661505336 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:00.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.858+0000 7fb92a7fc640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9340d17a0 0x7fb9341f59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:00.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.858+0000 7fb932ffd640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9340cdb30 0x7fb934156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:00.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.858+0000 7fb9337fe640 1 -- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9340cdb30 msgr2=0x7fb934156450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:00.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.858+0000 7fb9337fe640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9340cdb30 0x7fb934156450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.858+0000 7fb9337fe640 1 -- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9340d17a0 msgr2=0x7fb9341f59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:00.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.858+0000 7fb9337fe640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9340d17a0 0x7fb9341f59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:00.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.858+0000 7fb9337fe640 1 -- 172.21.15.27:0/3661505336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c009b30 con 0x7fb93415cfc0 2023-12-19T18:22:00.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.858+0000 7fb9337fe640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb93415cfc0 0x7fb934154f50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb92c036040 tx=0x7fb92c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:00.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.859+0000 7fb930ff9640 1 -- 172.21.15.27:0/3661505336 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c009510 con 0x7fb93415cfc0 2023-12-19T18:22:00.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.859+0000 7fb930ff9640 1 -- 172.21.15.27:0/3661505336 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb92c03e450 con 0x7fb93415cfc0 2023-12-19T18:22:00.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.859+0000 7fb933fff640 1 -- 172.21.15.27:0/3661505336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9341f60b0 con 0x7fb93415cfc0 2023-12-19T18:22:00.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.859+0000 7fb930ff9640 1 -- 172.21.15.27:0/3661505336 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb92c03e800 con 0x7fb93415cfc0 2023-12-19T18:22:00.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.859+0000 7fb933fff640 1 -- 172.21.15.27:0/3661505336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9341f6670 con 0x7fb93415cfc0 2023-12-19T18:22:00.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.860+0000 7fb933fff640 1 -- 172.21.15.27:0/3661505336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9340c9e30 con 0x7fb93415cfc0 2023-12-19T18:22:00.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.861+0000 7fb930ff9640 1 -- 172.21.15.27:0/3661505336 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb92c03e960 con 0x7fb93415cfc0 2023-12-19T18:22:00.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.862+0000 7fb930ff9640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb91406a430 0x7fb91406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:00.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.862+0000 7fb930ff9640 1 -- 172.21.15.27:0/3661505336 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5795+0+0 (secure 0 0 0) 0x7fb92c0ab710 con 0x7fb93415cfc0 2023-12-19T18:22:00.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.862+0000 7fb932ffd640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb91406a430 0x7fb91406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:00.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.864+0000 7fb932ffd640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb91406a430 0x7fb91406c8f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb93415c760 tx=0x7fb924009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:00.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:00.866+0000 7fb930ff9640 1 -- 172.21.15.27:0/3661505336 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb9340c9e30 con 0x7fb93415cfc0 2023-12-19T18:22:01.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.101+0000 7fb933fff640 1 -- 172.21.15.27:0/3661505336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb9340c9e30 con 0x7fb93415cfc0 2023-12-19T18:22:01.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.617+0000 7fb930ff9640 1 -- 172.21.15.27:0/3661505336 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb9340c9e30 con 0x7fb93415cfc0 2023-12-19T18:22:01.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.620+0000 7fb933fff640 1 -- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb91406a430 msgr2=0x7fb91406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:01.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.620+0000 7fb933fff640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb91406a430 0x7fb91406c8f0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fb93415c760 tx=0x7fb924009040 comp rx=0 tx=0).stop 2023-12-19T18:22:01.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.620+0000 7fb933fff640 1 -- 172.21.15.27:0/3661505336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb93415cfc0 msgr2=0x7fb934154f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:01.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.620+0000 7fb933fff640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb93415cfc0 0x7fb934154f50 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fb92c036040 tx=0x7fb92c004a40 comp rx=0 tx=0).stop 2023-12-19T18:22:01.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.621+0000 7fb933fff640 1 -- 172.21.15.27:0/3661505336 shutdown_connections 2023-12-19T18:22:01.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.621+0000 7fb933fff640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb91406a430 0x7fb91406c8f0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:01.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.621+0000 7fb933fff640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb93415cfc0 0x7fb934154f50 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:01.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.621+0000 7fb933fff640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb9340d17a0 0x7fb9341f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:01.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.621+0000 7fb933fff640 1 --2- 172.21.15.27:0/3661505336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb9340cdb30 0x7fb934156450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:01.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.621+0000 7fb933fff640 1 -- 172.21.15.27:0/3661505336 >> 172.21.15.27:0/3661505336 conn(0x7fb934072e20 msgr2=0x7fb934152fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:01.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.621+0000 7fb933fff640 1 -- 172.21.15.27:0/3661505336 shutdown_connections 2023-12-19T18:22:01.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.622+0000 7fb933fff640 1 -- 172.21.15.27:0/3661505336 wait complete. 2023-12-19T18:22:01.643 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:22:01.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.873+0000 7f3dcfe84640 1 Processor -- start 2023-12-19T18:22:01.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.873+0000 7f3dcfe84640 1 -- start start 2023-12-19T18:22:01.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.875+0000 7f3dcfe84640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3dc815b4b0 0x7f3dc80cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:01.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.875+0000 7f3dcfe84640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc8070fb0 con 0x7f3dc815fa70 2023-12-19T18:22:01.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.875+0000 7f3dcfe84640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc806fab0 con 0x7f3dc815b0d0 2023-12-19T18:22:01.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.875+0000 7f3dcfe84640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc806f4f0 con 0x7f3dc815b4b0 2023-12-19T18:22:01.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.875+0000 7f3dcd41f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3dc815b4b0 0x7f3dc80cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:01.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.875+0000 7f3dce421640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3dc815fa70 0x7f3dc80c91c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55796/0 (socket says 172.21.15.27:55796) 2023-12-19T18:22:01.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.875+0000 7f3dce421640 1 -- 172.21.15.27:0/4075639677 learned_addr learned my addr 172.21.15.27:0/4075639677 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:01.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.876+0000 7f3dcd41f640 1 -- 172.21.15.27:0/4075639677 >> v1:172.21.15.192:6789/0 conn(0x7f3dc815b0d0 legacy=0x7f3dc80cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:01.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.876+0000 7f3dcd41f640 1 -- 172.21.15.27:0/4075639677 >> v1:172.21.15.27:6789/0 conn(0x7f3dc815fa70 legacy=0x7f3dc80c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:01.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.876+0000 7f3dcd41f640 1 -- 172.21.15.27:0/4075639677 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc806f7d0 con 0x7f3dc815b4b0 2023-12-19T18:22:01.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.876+0000 7f3dccc1e640 1 -- 172.21.15.27:0/4075639677 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439379661 0 0) 0x7f3dc8070fb0 con 0x7f3dc815fa70 2023-12-19T18:22:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.877+0000 7f3dcd41f640 1 --2- 172.21.15.27:0/4075639677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc815b4b0 0x7f3dc80cb450 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f3db80097d0 tx=0x7f3db802b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71770211b2774a30 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.877+0000 7f3dccc1e640 1 -- 172.21.15.27:0/4075639677 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8030890 con 0x7f3dc815b4b0 2023-12-19T18:22:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.877+0000 7f3dccc1e640 1 -- 172.21.15.27:0/4075639677 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db80309f0 con 0x7f3dc815b4b0 2023-12-19T18:22:01.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.877+0000 7f3dccc1e640 1 -- 172.21.15.27:0/4075639677 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8030d80 con 0x7f3dc815b4b0 2023-12-19T18:22:01.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.878+0000 7f3dcfe84640 1 -- 172.21.15.27:0/4075639677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc815b4b0 msgr2=0x7f3dc80cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:01.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.878+0000 7f3dcfe84640 1 --2- 172.21.15.27:0/4075639677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc815b4b0 0x7f3dc80cb450 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f3db80097d0 tx=0x7f3db802b9d0 comp rx=0 tx=0).stop 2023-12-19T18:22:01.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.878+0000 7f3dcfe84640 1 -- 172.21.15.27:0/4075639677 shutdown_connections 2023-12-19T18:22:01.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.878+0000 7f3dcfe84640 1 --2- 172.21.15.27:0/4075639677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc815b4b0 0x7f3dc80cb450 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:01.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.878+0000 7f3dcfe84640 1 -- 172.21.15.27:0/4075639677 >> 172.21.15.27:0/4075639677 conn(0x7f3dc80b0620 msgr2=0x7f3dc80b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:01.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.878+0000 7f3dcfe84640 1 -- 172.21.15.27:0/4075639677 shutdown_connections 2023-12-19T18:22:01.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.878+0000 7f3dcfe84640 1 -- 172.21.15.27:0/4075639677 wait complete. 2023-12-19T18:22:01.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.880+0000 7f3dcfe84640 1 Processor -- start 2023-12-19T18:22:01.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.880+0000 7f3dcfe84640 1 -- start start 2023-12-19T18:22:01.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.881+0000 7f3dcfe84640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc815b0d0 0x7f3dc81561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:01.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.881+0000 7f3dcfe84640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3dc815b4b0 0x7f3dc80c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:01.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.881+0000 7f3dcfe84640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dc815fa70 0x7f3dc80c85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:01.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.881+0000 7f3dcfe84640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc80ad580 con 0x7f3dc815b0d0 2023-12-19T18:22:01.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.881+0000 7f3dcfe84640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc80ac970 con 0x7f3dc815fa70 2023-12-19T18:22:01.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.881+0000 7f3dcfe84640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc80acc50 con 0x7f3dc815b4b0 2023-12-19T18:22:01.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.881+0000 7f3dcdc20640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc815b0d0 0x7f3dc81561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:01.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.881+0000 7f3dcdc20640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc815b0d0 0x7f3dc81561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37222/0 (socket says 172.21.15.27:37222) 2023-12-19T18:22:01.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.882+0000 7f3dcdc20640 1 -- 172.21.15.27:0/2198229088 learned_addr learned my addr 172.21.15.27:0/2198229088 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:01.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.882+0000 7f3dcdc20640 1 -- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3dc815b4b0 msgr2=0x7f3dc80c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:01.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.882+0000 7f3dcd41f640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3dc815b4b0 0x7f3dc80c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:01.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.882+0000 7f3dce421640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dc815fa70 0x7f3dc80c85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:01.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.882+0000 7f3dcdc20640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3dc815b4b0 0x7f3dc80c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:01.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.882+0000 7f3dcdc20640 1 -- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dc815fa70 msgr2=0x7f3dc80c85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.882+0000 7f3dcdc20640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dc815fa70 0x7f3dc80c85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.882+0000 7f3dcdc20640 1 -- 172.21.15.27:0/2198229088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db8009400 con 0x7f3dc815b0d0 2023-12-19T18:22:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.882+0000 7f3dce421640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dc815fa70 0x7f3dc80c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.882+0000 7f3dcdc20640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc815b0d0 0x7f3dc81561a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400c450 tx=0x7f3dc400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.883+0000 7f3dbe7fc640 1 -- 172.21.15.27:0/2198229088 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc40052d0 con 0x7f3dc815b0d0 2023-12-19T18:22:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.883+0000 7f3dbe7fc640 1 -- 172.21.15.27:0/2198229088 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc4005430 con 0x7f3dc815b0d0 2023-12-19T18:22:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.883+0000 7f3dbe7fc640 1 -- 172.21.15.27:0/2198229088 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc4012870 con 0x7f3dc815b0d0 2023-12-19T18:22:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.883+0000 7f3dcfe84640 1 -- 172.21.15.27:0/2198229088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dc80acf30 con 0x7f3dc815b0d0 2023-12-19T18:22:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.883+0000 7f3dcfe84640 1 -- 172.21.15.27:0/2198229088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc81f72d0 con 0x7f3dc815b0d0 2023-12-19T18:22:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.884+0000 7f3dcfe84640 1 -- 172.21.15.27:0/2198229088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc8068e90 con 0x7f3dc815b0d0 2023-12-19T18:22:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.885+0000 7f3dbe7fc640 1 -- 172.21.15.27:0/2198229088 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3dc40129d0 con 0x7f3dc815b0d0 2023-12-19T18:22:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.886+0000 7f3dbe7fc640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3dac06a430 0x7f3dac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.886+0000 7f3dbe7fc640 1 -- 172.21.15.27:0/2198229088 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f3dc4088590 con 0x7f3dc815b0d0 2023-12-19T18:22:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.887+0000 7f3dcd41f640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3dac06a430 0x7f3dac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.888+0000 7f3dcd41f640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3dac06a430 0x7f3dac06c8f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3db8036000 tx=0x7f3db803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:01.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:01.891+0000 7f3dbe7fc640 1 -- 172.21.15.27:0/2198229088 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3dc4053a80 con 0x7f3dc815b0d0 2023-12-19T18:22:02.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.089+0000 7f3dcfe84640 1 -- 172.21.15.27:0/2198229088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3dc806c530 con 0x7f3dc815b0d0 2023-12-19T18:22:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.975+0000 7f3dbe7fc640 1 -- 172.21.15.27:0/2198229088 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v125) v1 ==== 181+0+0 (secure 0 0 0) 0x7f3dc40570a0 con 0x7f3dc815b0d0 2023-12-19T18:22:02.975 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:22:02.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.978+0000 7f3dcfe84640 1 -- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3dac06a430 msgr2=0x7f3dac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:02.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.978+0000 7f3dcfe84640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3dac06a430 0x7f3dac06c8f0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f3db8036000 tx=0x7f3db803b040 comp rx=0 tx=0).stop 2023-12-19T18:22:02.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.978+0000 7f3dcfe84640 1 -- 172.21.15.27:0/2198229088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc815b0d0 msgr2=0x7f3dc81561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:02.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.978+0000 7f3dcfe84640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc815b0d0 0x7f3dc81561a0 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f3dc400c450 tx=0x7f3dc400c910 comp rx=0 tx=0).stop 2023-12-19T18:22:02.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.979+0000 7f3dcfe84640 1 -- 172.21.15.27:0/2198229088 shutdown_connections 2023-12-19T18:22:02.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.979+0000 7f3dcfe84640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3dac06a430 0x7f3dac06c8f0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:02.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.979+0000 7f3dcfe84640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3dc815fa70 0x7f3dc80c85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:02.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.979+0000 7f3dcfe84640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3dc815b4b0 0x7f3dc80c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:02.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.979+0000 7f3dcfe84640 1 --2- 172.21.15.27:0/2198229088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3dc815b0d0 0x7f3dc81561a0 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:02.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.979+0000 7f3dcfe84640 1 -- 172.21.15.27:0/2198229088 >> 172.21.15.27:0/2198229088 conn(0x7f3dc80b0620 msgr2=0x7f3dc80ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:02.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.979+0000 7f3dcfe84640 1 -- 172.21.15.27:0/2198229088 shutdown_connections 2023-12-19T18:22:02.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:02.979+0000 7f3dcfe84640 1 -- 172.21.15.27:0/2198229088 wait complete. 2023-12-19T18:22:03.005 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:22:03.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.222+0000 7fd30a422640 1 Processor -- start 2023-12-19T18:22:03.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.222+0000 7fd30a422640 1 -- start start 2023-12-19T18:22:03.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.224+0000 7fd30a422640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd3040c55b0 0x7fd3040cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:03.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.224+0000 7fd30a422640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3040ce520 0x7fd3040cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:03.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.224+0000 7fd30a422640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3040cb100 con 0x7fd304161030 2023-12-19T18:22:03.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.224+0000 7fd30a422640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3040c95f0 con 0x7fd3040c55b0 2023-12-19T18:22:03.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.224+0000 7fd30a422640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040cae20 con 0x7fd3040ce520 2023-12-19T18:22:03.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.224+0000 7fd309420640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd304161030 0x7fd3041657c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34792/0 (socket says 172.21.15.27:34792) 2023-12-19T18:22:03.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.224+0000 7fd309420640 1 -- 172.21.15.27:0/3775801050 learned_addr learned my addr 172.21.15.27:0/3775801050 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:03.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.225+0000 7fd308c1f640 1 --2- 172.21.15.27:0/3775801050 >> v2:172.21.15.192:3300/0 conn(0x7fd3040c55b0 0x7fd3040cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:03.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.225+0000 7fd309c21640 1 --2- 172.21.15.27:0/3775801050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3040ce520 0x7fd3040cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:03.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.225+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3126755484 0 0) 0x7fd3040cb100 con 0x7fd304161030 2023-12-19T18:22:03.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.225+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd2f0003520 con 0x7fd304161030 2023-12-19T18:22:03.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.225+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2117545985 0 0) 0x7fd2f0003520 con 0x7fd304161030 2023-12-19T18:22:03.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3040ce520 msgr2=0x7fd3040cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:03.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd2fbfff640 1 --2- 172.21.15.27:0/3775801050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3040ce520 0x7fd3040cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:03.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 >> v2:172.21.15.192:3300/0 conn(0x7fd3040c55b0 msgr2=0x7fd3040cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:03.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd2fbfff640 1 --2- 172.21.15.27:0/3775801050 >> v2:172.21.15.192:3300/0 conn(0x7fd3040c55b0 0x7fd3040cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:03.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3040c9030 con 0x7fd304161030 2023-12-19T18:22:03.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fd300001670 con 0x7fd304161030 2023-12-19T18:22:03.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 >> v1:172.21.15.27:6789/0 conn(0x7fd304161030 legacy=0x7fd3041657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:03.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd2fbfff640 1 --2- 172.21.15.27:0/3775801050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f0003970 0x7fd2f0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:03.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd2fbfff640 1 --2- 172.21.15.27:0/3775801050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2f00047c0 0x7fd2f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:03.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd2fbfff640 1 --2- 172.21.15.27:0/3775801050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3040ce520 0x7fd2f0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:03.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040c95f0 con 0x7fd2f0003970 2023-12-19T18:22:03.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3040cae20 con 0x7fd2f00047c0 2023-12-19T18:22:03.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f000b980 con 0x7fd3040ce520 2023-12-19T18:22:03.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd309420640 1 --2- 172.21.15.27:0/3775801050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f0003970 0x7fd2f0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:03.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.226+0000 7fd308c1f640 1 --2- 172.21.15.27:0/3775801050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2f00047c0 0x7fd2f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:03.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.227+0000 7fd309c21640 1 --2- 172.21.15.27:0/3775801050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3040ce520 0x7fd2f0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:03.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.227+0000 7fd309420640 1 -- 172.21.15.27:0/3775801050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3040ce520 msgr2=0x7fd2f0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:03.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.227+0000 7fd309420640 1 --2- 172.21.15.27:0/3775801050 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3040ce520 0x7fd2f0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:03.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.227+0000 7fd309420640 1 -- 172.21.15.27:0/3775801050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2f00047c0 msgr2=0x7fd2f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:03.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.227+0000 7fd309420640 1 --2- 172.21.15.27:0/3775801050 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2f00047c0 0x7fd2f0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:03.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.227+0000 7fd309420640 1 -- 172.21.15.27:0/3775801050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f0003520 con 0x7fd2f0003970 2023-12-19T18:22:03.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.228+0000 7fd309420640 1 --2- 172.21.15.27:0/3775801050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f0003970 0x7fd2f0003d90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd30000c160 tx=0x7fd2f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89dc890033dcf2f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:03.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.229+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300005ad0 con 0x7fd2f0003970 2023-12-19T18:22:03.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.229+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd300005c30 con 0x7fd2f0003970 2023-12-19T18:22:03.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.229+0000 7fd2fbfff640 1 -- 172.21.15.27:0/3775801050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30002c810 con 0x7fd2f0003970 2023-12-19T18:22:03.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.229+0000 7fd30a422640 1 -- 172.21.15.27:0/3775801050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f0003970 msgr2=0x7fd2f0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:03.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.229+0000 7fd30a422640 1 --2- 172.21.15.27:0/3775801050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f0003970 0x7fd2f0003d90 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fd30000c160 tx=0x7fd2f00057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:03.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.229+0000 7fd309c21640 1 -- 172.21.15.27:0/3775801050 reap_dead start 2023-12-19T18:22:03.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.229+0000 7fd30a422640 1 -- 172.21.15.27:0/3775801050 shutdown_connections 2023-12-19T18:22:03.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.229+0000 7fd30a422640 1 -- 172.21.15.27:0/3775801050 >> 172.21.15.27:0/3775801050 conn(0x7fd3040b05b0 msgr2=0x7fd3040b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:03.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.229+0000 7fd30a422640 1 -- 172.21.15.27:0/3775801050 shutdown_connections 2023-12-19T18:22:03.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.230+0000 7fd30a422640 1 -- 172.21.15.27:0/3775801050 wait complete. 2023-12-19T18:22:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.231+0000 7fd30a422640 1 Processor -- start 2023-12-19T18:22:03.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.231+0000 7fd30a422640 1 -- start start 2023-12-19T18:22:03.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.232+0000 7fd30a422640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f0003970 0x7fd3041f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:03.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.232+0000 7fd30a422640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3040c88f0 0x7fd3040c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:03.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.232+0000 7fd30a422640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd304141330 0x7fd3041417b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:03.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.232+0000 7fd30a422640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd304161640 con 0x7fd2f0003970 2023-12-19T18:22:03.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.232+0000 7fd30a422640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd304161920 con 0x7fd3040c88f0 2023-12-19T18:22:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.232+0000 7fd30a422640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd304161c00 con 0x7fd304141330 2023-12-19T18:22:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.233+0000 7fd309420640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f0003970 0x7fd3041f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.233+0000 7fd309420640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f0003970 0x7fd3041f67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37648/0 (socket says 172.21.15.27:37648) 2023-12-19T18:22:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.233+0000 7fd309420640 1 -- 172.21.15.27:0/3426342235 learned_addr learned my addr 172.21.15.27:0/3426342235 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.233+0000 7fd309c21640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd304141330 0x7fd3041417b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.233+0000 7fd308c1f640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3040c88f0 0x7fd3040c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.233+0000 7fd309420640 1 -- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd304141330 msgr2=0x7fd3041417b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.233+0000 7fd309420640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd304141330 0x7fd3041417b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.233+0000 7fd309420640 1 -- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3040c88f0 msgr2=0x7fd3040c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.233+0000 7fd309420640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3040c88f0 0x7fd3040c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.233+0000 7fd309420640 1 -- 172.21.15.27:0/3426342235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd30406bd90 con 0x7fd2f0003970 2023-12-19T18:22:03.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.233+0000 7fd309c21640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd304141330 0x7fd3041417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:03.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.234+0000 7fd309420640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f0003970 0x7fd3041f67c0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd300000fb0 tx=0x7fd300000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:03.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.234+0000 7fd2f9ffb640 1 -- 172.21.15.27:0/3426342235 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300039070 con 0x7fd2f0003970 2023-12-19T18:22:03.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.234+0000 7fd2f9ffb640 1 -- 172.21.15.27:0/3426342235 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd30002dcb0 con 0x7fd2f0003970 2023-12-19T18:22:03.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.234+0000 7fd30a422640 1 -- 172.21.15.27:0/3426342235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30406c070 con 0x7fd2f0003970 2023-12-19T18:22:03.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.234+0000 7fd2f9ffb640 1 -- 172.21.15.27:0/3426342235 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000072b0 con 0x7fd2f0003970 2023-12-19T18:22:03.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.234+0000 7fd30a422640 1 -- 172.21.15.27:0/3426342235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3040c5f50 con 0x7fd2f0003970 2023-12-19T18:22:03.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.235+0000 7fd30a422640 1 -- 172.21.15.27:0/3426342235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd304161dc0 con 0x7fd2f0003970 2023-12-19T18:22:03.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.236+0000 7fd2f9ffb640 1 -- 172.21.15.27:0/3426342235 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd300032070 con 0x7fd2f0003970 2023-12-19T18:22:03.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.237+0000 7fd2f9ffb640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd2e806a460 0x7fd2e806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:03.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.237+0000 7fd2f9ffb640 1 -- 172.21.15.27:0/3426342235 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fd30003c040 con 0x7fd2f0003970 2023-12-19T18:22:03.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.238+0000 7fd308c1f640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd2e806a460 0x7fd2e806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:03.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.239+0000 7fd308c1f640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd2e806a460 0x7fd2e806c920 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd3040aef50 tx=0x7fd2f4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:03.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.242+0000 7fd2f9ffb640 1 -- 172.21.15.27:0/3426342235 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd300074ed0 con 0x7fd2f0003970 2023-12-19T18:22:03.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.435+0000 7fd30a422640 1 -- 172.21.15.27:0/3426342235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd304160870 con 0x7fd2f0003970 2023-12-19T18:22:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.988+0000 7fd2f9ffb640 1 -- 172.21.15.27:0/3426342235 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v126) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd3000450d0 con 0x7fd2f0003970 2023-12-19T18:22:03.989 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-12-19T18:22:03.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.992+0000 7fd30a422640 1 -- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd2e806a460 msgr2=0x7fd2e806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:03.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.992+0000 7fd30a422640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd2e806a460 0x7fd2e806c920 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd3040aef50 tx=0x7fd2f4008040 comp rx=0 tx=0).stop 2023-12-19T18:22:03.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.992+0000 7fd30a422640 1 -- 172.21.15.27:0/3426342235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f0003970 msgr2=0x7fd3041f67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:03.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.992+0000 7fd30a422640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f0003970 0x7fd3041f67c0 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7fd300000fb0 tx=0x7fd300000fe0 comp rx=0 tx=0).stop 2023-12-19T18:22:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.992+0000 7fd30a422640 1 -- 172.21.15.27:0/3426342235 shutdown_connections 2023-12-19T18:22:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.992+0000 7fd30a422640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd2e806a460 0x7fd2e806c920 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.992+0000 7fd30a422640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd304141330 0x7fd3041417b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.992+0000 7fd30a422640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3040c88f0 0x7fd3040c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:03.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.992+0000 7fd30a422640 1 --2- 172.21.15.27:0/3426342235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f0003970 0x7fd3041f67c0 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:03.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.992+0000 7fd30a422640 1 -- 172.21.15.27:0/3426342235 >> 172.21.15.27:0/3426342235 conn(0x7fd3040b05b0 msgr2=0x7fd304070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:03.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.993+0000 7fd30a422640 1 -- 172.21.15.27:0/3426342235 shutdown_connections 2023-12-19T18:22:03.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:03.993+0000 7fd30a422640 1 -- 172.21.15.27:0/3426342235 wait complete. 2023-12-19T18:22:04.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:22:04.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.247+0000 7f057f44d640 1 Processor -- start 2023-12-19T18:22:04.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.248+0000 7f057f44d640 1 -- start start 2023-12-19T18:22:04.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.250+0000 7f057f44d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0578161030 0x7f05781657c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:04.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.250+0000 7f057f44d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f05780c55b0 0x7f05780cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:04.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.250+0000 7f057f44d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0578165d00 con 0x7f05780ce520 2023-12-19T18:22:04.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.250+0000 7f057f44d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05780c95f0 con 0x7f05780c55b0 2023-12-19T18:22:04.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.250+0000 7f057f44d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05780cb100 con 0x7f0578161030 2023-12-19T18:22:04.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.250+0000 7f057ec4c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f05780ce520 0x7f05780c3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34808/0 (socket says 172.21.15.27:34808) 2023-12-19T18:22:04.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.250+0000 7f057ec4c640 1 -- 172.21.15.27:0/2840233827 learned_addr learned my addr 172.21.15.27:0/2840233827 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:04.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.250+0000 7f057dc4a640 1 --2- 172.21.15.27:0/2840233827 >> v2:172.21.15.27:3300/0 conn(0x7f05780c55b0 0x7f05780cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:04.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.250+0000 7f057e44b640 1 --2- 172.21.15.27:0/2840233827 >> v2:172.21.15.192:3300/0 conn(0x7f0578161030 0x7f05781657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:04.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.251+0000 7f057d449640 1 -- 172.21.15.27:0/2840233827 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 551833934 0 0) 0x7f0578165d00 con 0x7f05780ce520 2023-12-19T18:22:04.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.251+0000 7f057dc4a640 1 -- 172.21.15.27:0/2840233827 >> v2:172.21.15.192:3300/0 conn(0x7f0578161030 msgr2=0x7f05781657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:04.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.251+0000 7f057dc4a640 1 --2- 172.21.15.27:0/2840233827 >> v2:172.21.15.192:3300/0 conn(0x7f0578161030 0x7f05781657c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.251+0000 7f057dc4a640 1 -- 172.21.15.27:0/2840233827 >> v1:172.21.15.27:6789/0 conn(0x7f05780ce520 legacy=0x7f05780c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:04.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.251+0000 7f057dc4a640 1 -- 172.21.15.27:0/2840233827 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05780cae20 con 0x7f05780c55b0 2023-12-19T18:22:04.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.252+0000 7f057dc4a640 1 --2- 172.21.15.27:0/2840233827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05780c55b0 0x7f05780cb350 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f05780cb320 tx=0x7f056802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e414edb2a3c853b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:04.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.252+0000 7f057d449640 1 -- 172.21.15.27:0/2840233827 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568031830 con 0x7f05780c55b0 2023-12-19T18:22:04.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.253+0000 7f057d449640 1 -- 172.21.15.27:0/2840233827 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0568031990 con 0x7f05780c55b0 2023-12-19T18:22:04.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.253+0000 7f057d449640 1 -- 172.21.15.27:0/2840233827 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0568031d20 con 0x7f05780c55b0 2023-12-19T18:22:04.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.253+0000 7f057f44d640 1 -- 172.21.15.27:0/2840233827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05780c55b0 msgr2=0x7f05780cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:04.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.253+0000 7f057f44d640 1 --2- 172.21.15.27:0/2840233827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05780c55b0 0x7f05780cb350 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f05780cb320 tx=0x7f056802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.253+0000 7f057f44d640 1 -- 172.21.15.27:0/2840233827 shutdown_connections 2023-12-19T18:22:04.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.253+0000 7f057f44d640 1 --2- 172.21.15.27:0/2840233827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05780c55b0 0x7f05780cb350 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.253+0000 7f057f44d640 1 --2- 172.21.15.27:0/2840233827 >> v2:172.21.15.192:3300/0 conn(0x7f0578161030 0x7f05781657c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.253+0000 7f057f44d640 1 -- 172.21.15.27:0/2840233827 >> 172.21.15.27:0/2840233827 conn(0x7f05780b05b0 msgr2=0x7f05780b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:04.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.253+0000 7f057f44d640 1 -- 172.21.15.27:0/2840233827 shutdown_connections 2023-12-19T18:22:04.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.254+0000 7f057f44d640 1 -- 172.21.15.27:0/2840233827 wait complete. 2023-12-19T18:22:04.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.255+0000 7f057f44d640 1 Processor -- start 2023-12-19T18:22:04.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.255+0000 7f057f44d640 1 -- start start 2023-12-19T18:22:04.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.256+0000 7f057f44d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05780c55b0 0x7f057814dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:04.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.256+0000 7f057f44d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05780ce520 0x7f057815fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.256+0000 7f057f44d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0578161030 0x7f057815ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.256+0000 7f057f44d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057806c2f0 con 0x7f0578161030 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.256+0000 7f057f44d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f057815e620 con 0x7f05780c55b0 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.256+0000 7f057f44d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057814e220 con 0x7f05780ce520 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.257+0000 7f057ec4c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0578161030 0x7f057815ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.257+0000 7f057ec4c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0578161030 0x7f057815ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37660/0 (socket says 172.21.15.27:37660) 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.257+0000 7f057ec4c640 1 -- 172.21.15.27:0/962838516 learned_addr learned my addr 172.21.15.27:0/962838516 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.257+0000 7f057e44b640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05780c55b0 0x7f057814dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.257+0000 7f057dc4a640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05780ce520 0x7f057815fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.257+0000 7f057ec4c640 1 -- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05780ce520 msgr2=0x7f057815fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.257+0000 7f057ec4c640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05780ce520 0x7f057815fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.257+0000 7f057ec4c640 1 -- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05780c55b0 msgr2=0x7f057814dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.257+0000 7f057ec4c640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05780c55b0 0x7f057814dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.257+0000 7f057ec4c640 1 -- 172.21.15.27:0/962838516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0568009400 con 0x7f0578161030 2023-12-19T18:22:04.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.258+0000 7f057ec4c640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0578161030 0x7f057815ff40 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f057000c810 tx=0x7f057000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:04.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.258+0000 7f056effd640 1 -- 172.21.15.27:0/962838516 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057000a640 con 0x7f0578161030 2023-12-19T18:22:04.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.258+0000 7f056effd640 1 -- 172.21.15.27:0/962838516 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f057000a7c0 con 0x7f0578161030 2023-12-19T18:22:04.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.258+0000 7f056effd640 1 -- 172.21.15.27:0/962838516 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f057000ab70 con 0x7f0578161030 2023-12-19T18:22:04.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.258+0000 7f057f44d640 1 -- 172.21.15.27:0/962838516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f057815f630 con 0x7f0578161030 2023-12-19T18:22:04.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.258+0000 7f057f44d640 1 -- 172.21.15.27:0/962838516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f057815eb50 con 0x7f0578161030 2023-12-19T18:22:04.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.259+0000 7f057f44d640 1 -- 172.21.15.27:0/962838516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f057804e840 con 0x7f0578161030 2023-12-19T18:22:04.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.260+0000 7f056effd640 1 -- 172.21.15.27:0/962838516 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f057000acd0 con 0x7f0578161030 2023-12-19T18:22:04.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.261+0000 7f056effd640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f055c06a430 0x7f055c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:04.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.261+0000 7f056effd640 1 -- 172.21.15.27:0/962838516 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0570087ba0 con 0x7f0578161030 2023-12-19T18:22:04.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.262+0000 7f057e44b640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f055c06a430 0x7f055c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:04.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.263+0000 7f057e44b640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f055c06a430 0x7f055c06c8f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0574006fd0 tx=0x7f0574008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:04.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.265+0000 7f056effd640 1 -- 172.21.15.27:0/962838516 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f05700533f0 con 0x7f0578161030 2023-12-19T18:22:04.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.461+0000 7f057f44d640 1 -- 172.21.15.27:0/962838516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f05780c4fd0 con 0x7f0578161030 2023-12-19T18:22:04.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.467+0000 7f056effd640 1 -- 172.21.15.27:0/962838516 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0570056a10 con 0x7f0578161030 2023-12-19T18:22:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.470+0000 7f057f44d640 1 -- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f055c06a430 msgr2=0x7f055c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.470+0000 7f057f44d640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f055c06a430 0x7f055c06c8f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0574006fd0 tx=0x7f0574008040 comp rx=0 tx=0).stop 2023-12-19T18:22:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.470+0000 7f057f44d640 1 -- 172.21.15.27:0/962838516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0578161030 msgr2=0x7f057815ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.470+0000 7f057f44d640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0578161030 0x7f057815ff40 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f057000c810 tx=0x7f057000ccd0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.470+0000 7f057f44d640 1 -- 172.21.15.27:0/962838516 shutdown_connections 2023-12-19T18:22:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.470+0000 7f057f44d640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f055c06a430 0x7f055c06c8f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.470+0000 7f057f44d640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0578161030 0x7f057815ff40 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.470+0000 7f057f44d640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05780ce520 0x7f057815fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.470+0000 7f057f44d640 1 --2- 172.21.15.27:0/962838516 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05780c55b0 0x7f057814dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.470+0000 7f057f44d640 1 -- 172.21.15.27:0/962838516 >> 172.21.15.27:0/962838516 conn(0x7f05780b05b0 msgr2=0x7f05780afa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:04.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.470+0000 7f057f44d640 1 -- 172.21.15.27:0/962838516 shutdown_connections 2023-12-19T18:22:04.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.470+0000 7f057f44d640 1 -- 172.21.15.27:0/962838516 wait complete. 2023-12-19T18:22:04.491 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file' 2023-12-19T18:22:04.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.722+0000 7f2474e7d640 1 Processor -- start 2023-12-19T18:22:04.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.723+0000 7f2474e7d640 1 -- start start 2023-12-19T18:22:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.724+0000 7f2474e7d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f24701604d0 0x7f24700b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.724+0000 7f2474e7d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f24700d2350 0x7f24700b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.724+0000 7f2474e7d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24700b5e90 con 0x7f24700d0300 2023-12-19T18:22:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.724+0000 7f2474e7d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24700d2b30 con 0x7f24701604d0 2023-12-19T18:22:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.724+0000 7f2474e7d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24700d2850 con 0x7f24700d2350 2023-12-19T18:22:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.725+0000 7f246f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f24701604d0 0x7f24700b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.725+0000 7f246f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f24701604d0 0x7f24700b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37670/0 (socket says 172.21.15.27:37670) 2023-12-19T18:22:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.725+0000 7f246f7fe640 1 -- 172.21.15.27:0/4179242676 learned_addr learned my addr 172.21.15.27:0/4179242676 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:04.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.725+0000 7f246effd640 1 --2- 172.21.15.27:0/4179242676 >> v2:172.21.15.192:3300/0 conn(0x7f24700d2350 0x7f24700b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:04.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.725+0000 7f246f7fe640 1 -- 172.21.15.27:0/4179242676 >> v2:172.21.15.192:3300/0 conn(0x7f24700d2350 msgr2=0x7f24700b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:04.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.725+0000 7f246f7fe640 1 --2- 172.21.15.27:0/4179242676 >> v2:172.21.15.192:3300/0 conn(0x7f24700d2350 0x7f24700b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.725+0000 7f246f7fe640 1 -- 172.21.15.27:0/4179242676 >> v1:172.21.15.192:6789/0 conn(0x7f24700d0300 legacy=0x7f24700d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:04.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.725+0000 7f246f7fe640 1 -- 172.21.15.27:0/4179242676 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24700cded0 con 0x7f24701604d0 2023-12-19T18:22:04.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.727+0000 7f246f7fe640 1 --2- 172.21.15.27:0/4179242676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24701604d0 0x7f24700b47e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2470072120 tx=0x7f246002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba68536f44dcf921 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:04.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.727+0000 7f246e7fc640 1 -- 172.21.15.27:0/4179242676 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460031870 con 0x7f24701604d0 2023-12-19T18:22:04.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.727+0000 7f246e7fc640 1 -- 172.21.15.27:0/4179242676 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24600319d0 con 0x7f24701604d0 2023-12-19T18:22:04.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.727+0000 7f246e7fc640 1 -- 172.21.15.27:0/4179242676 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2460031d60 con 0x7f24701604d0 2023-12-19T18:22:04.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.728+0000 7f2474e7d640 1 -- 172.21.15.27:0/4179242676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24701604d0 msgr2=0x7f24700b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:04.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.728+0000 7f2474e7d640 1 --2- 172.21.15.27:0/4179242676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24701604d0 0x7f24700b47e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2470072120 tx=0x7f246002b910 comp rx=0 tx=0).stop 2023-12-19T18:22:04.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.728+0000 7f2474e7d640 1 -- 172.21.15.27:0/4179242676 shutdown_connections 2023-12-19T18:22:04.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.728+0000 7f2474e7d640 1 --2- 172.21.15.27:0/4179242676 >> v2:172.21.15.192:3300/0 conn(0x7f24700d2350 0x7f24700b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.728+0000 7f2474e7d640 1 --2- 172.21.15.27:0/4179242676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24701604d0 0x7f24700b47e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.728+0000 7f2474e7d640 1 -- 172.21.15.27:0/4179242676 >> 172.21.15.27:0/4179242676 conn(0x7f24700cf7e0 msgr2=0x7f24700cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:04.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.728+0000 7f2474e7d640 1 -- 172.21.15.27:0/4179242676 shutdown_connections 2023-12-19T18:22:04.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.728+0000 7f2474e7d640 1 -- 172.21.15.27:0/4179242676 wait complete. 2023-12-19T18:22:04.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.730+0000 7f2474e7d640 1 Processor -- start 2023-12-19T18:22:04.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.730+0000 7f2474e7d640 1 -- start start 2023-12-19T18:22:04.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.731+0000 7f2474e7d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24700d0300 0x7f2470157f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.731+0000 7f2474e7d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f24700d2350 0x7f2470148840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.731+0000 7f2474e7d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24701604d0 0x7f2470148d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.731+0000 7f2474e7d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2470158920 con 0x7f24701604d0 2023-12-19T18:22:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.731+0000 7f2474e7d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2470158c00 con 0x7f24700d0300 2023-12-19T18:22:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.731+0000 7f2474e7d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2470158ee0 con 0x7f24700d2350 2023-12-19T18:22:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.731+0000 7f246ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24701604d0 0x7f2470148d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.731+0000 7f246ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24701604d0 0x7f2470148d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37686/0 (socket says 172.21.15.27:37686) 2023-12-19T18:22:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.731+0000 7f246ffff640 1 -- 172.21.15.27:0/898838559 learned_addr learned my addr 172.21.15.27:0/898838559 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.732+0000 7f246f7fe640 1 --2- 172.21.15.27:0/898838559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24700d0300 0x7f2470157f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.732+0000 7f246ffff640 1 -- 172.21.15.27:0/898838559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f24700d2350 msgr2=0x7f2470148840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.732+0000 7f246ffff640 1 --2- 172.21.15.27:0/898838559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f24700d2350 0x7f2470148840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.732+0000 7f246ffff640 1 -- 172.21.15.27:0/898838559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24700d0300 msgr2=0x7f2470157f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:04.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.732+0000 7f246ffff640 1 --2- 172.21.15.27:0/898838559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24700d0300 0x7f2470157f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:04.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.732+0000 7f246ffff640 1 -- 172.21.15.27:0/898838559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24600094d0 con 0x7f24701604d0 2023-12-19T18:22:04.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.732+0000 7f246ffff640 1 --2- 172.21.15.27:0/898838559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24701604d0 0x7f2470148d80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f246800c780 tx=0x7f246800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:04.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.732+0000 7f2457fff640 1 -- 172.21.15.27:0/898838559 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2468004b60 con 0x7f24701604d0 2023-12-19T18:22:04.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.732+0000 7f2457fff640 1 -- 172.21.15.27:0/898838559 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f246800ce90 con 0x7f24701604d0 2023-12-19T18:22:04.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.732+0000 7f2474e7d640 1 -- 172.21.15.27:0/898838559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24701591c0 con 0x7f24701604d0 2023-12-19T18:22:04.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.733+0000 7f2457fff640 1 -- 172.21.15.27:0/898838559 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f246800a850 con 0x7f24701604d0 2023-12-19T18:22:04.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.733+0000 7f2474e7d640 1 -- 172.21.15.27:0/898838559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2470158640 con 0x7f24701604d0 2023-12-19T18:22:04.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.734+0000 7f2474e7d640 1 -- 172.21.15.27:0/898838559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2470066f80 con 0x7f24701604d0 2023-12-19T18:22:04.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.735+0000 7f2457fff640 1 -- 172.21.15.27:0/898838559 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f246800a9b0 con 0x7f24701604d0 2023-12-19T18:22:04.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.736+0000 7f2457fff640 1 --2- 172.21.15.27:0/898838559 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f245c06a4b0 0x7f245c06c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.736+0000 7f2457fff640 1 -- 172.21.15.27:0/898838559 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f24680886a0 con 0x7f24701604d0 2023-12-19T18:22:04.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.736+0000 7f246f7fe640 1 --2- 172.21.15.27:0/898838559 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f245c06a4b0 0x7f245c06c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:04.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.737+0000 7f246f7fe640 1 --2- 172.21.15.27:0/898838559 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f245c06a4b0 0x7f245c06c970 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2460036000 tx=0x7f246003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:04.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:04.740+0000 7f2457fff640 1 -- 172.21.15.27:0/898838559 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2468054070 con 0x7f24701604d0 2023-12-19T18:22:05.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:05.004+0000 7f2474e7d640 1 -- 172.21.15.27:0/898838559 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]} v 0) v1 -- 0x7f247004e840 con 0x7f24701604d0 2023-12-19T18:22:05.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:05.997+0000 7f2457fff640 1 -- 172.21.15.27:0/898838559 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestClusterFull.test_full_same_file"]}]=0 v365) v1 ==== 137+0+0 (secure 0 0 0) 0x7f2468057690 con 0x7f24701604d0 2023-12-19T18:22:06.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.001+0000 7f2474e7d640 1 -- 172.21.15.27:0/898838559 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f245c06a4b0 msgr2=0x7f245c06c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:06.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.001+0000 7f2474e7d640 1 --2- 172.21.15.27:0/898838559 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f245c06a4b0 0x7f245c06c970 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f2460036000 tx=0x7f246003c040 comp rx=0 tx=0).stop 2023-12-19T18:22:06.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.001+0000 7f2474e7d640 1 -- 172.21.15.27:0/898838559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24701604d0 msgr2=0x7f2470148d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:06.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.001+0000 7f2474e7d640 1 --2- 172.21.15.27:0/898838559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24701604d0 0x7f2470148d80 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f246800c780 tx=0x7f246800cc40 comp rx=0 tx=0).stop 2023-12-19T18:22:06.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.001+0000 7f2474e7d640 1 -- 172.21.15.27:0/898838559 shutdown_connections 2023-12-19T18:22:06.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.001+0000 7f2474e7d640 1 --2- 172.21.15.27:0/898838559 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f245c06a4b0 0x7f245c06c970 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:06.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.001+0000 7f2474e7d640 1 --2- 172.21.15.27:0/898838559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24701604d0 0x7f2470148d80 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:06.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.001+0000 7f2474e7d640 1 --2- 172.21.15.27:0/898838559 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f24700d2350 0x7f2470148840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:06.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.001+0000 7f2474e7d640 1 --2- 172.21.15.27:0/898838559 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f24700d0300 0x7f2470157f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:06.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.001+0000 7f2474e7d640 1 -- 172.21.15.27:0/898838559 >> 172.21.15.27:0/898838559 conn(0x7f24700cf7e0 msgr2=0x7f24700b25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:06.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.001+0000 7f2474e7d640 1 -- 172.21.15.27:0/898838559 shutdown_connections 2023-12-19T18:22:06.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.001+0000 7f2474e7d640 1 -- 172.21.15.27:0/898838559 wait complete. 2023-12-19T18:22:06.024 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestClusterFull) ... ok 2023-12-19T18:22:06.025 INFO:tasks.cephfs_test_runner:Starting test: test_barrier (tasks.cephfs.test_full.TestQuotaFull) That when an OSD epoch barrier is set on an MDS, subsequently 2023-12-19T18:22:06.025 INFO:tasks.cephfs_test_runner:test_barrier (tasks.cephfs.test_full.TestQuotaFull) 2023-12-19T18:22:06.025 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-12-19T18:22:06.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.259+0000 7f237d89e640 1 Processor -- start 2023-12-19T18:22:06.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.259+0000 7f237d89e640 1 -- start start 2023-12-19T18:22:06.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.261+0000 7f237d89e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2378154030 0x7f23780cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:06.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.261+0000 7f237d89e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23780c5560 0x7f23780afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:06.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.261+0000 7f237d89e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23780cc190 con 0x7f23780af630 2023-12-19T18:22:06.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.261+0000 7f237d89e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23780b0000 con 0x7f23780c5560 2023-12-19T18:22:06.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.261+0000 7f237d89e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23780cb2a0 con 0x7f2378154030 2023-12-19T18:22:06.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.261+0000 7f237d09d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f23780c5560 0x7f23780afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:06.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.261+0000 7f2377fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f23780af630 0x7f237815d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34816/0 (socket says 172.21.15.27:34816) 2023-12-19T18:22:06.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.261+0000 7f2377fff640 1 -- 172.21.15.27:0/1837003224 learned_addr learned my addr 172.21.15.27:0/1837003224 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:06.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.261+0000 7f237c89c640 1 --2- 172.21.15.27:0/1837003224 >> v2:172.21.15.192:3300/0 conn(0x7f2378154030 0x7f23780cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:06.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.262+0000 7f237d09d640 1 -- 172.21.15.27:0/1837003224 >> v2:172.21.15.192:3300/0 conn(0x7f2378154030 msgr2=0x7f23780cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:06.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.262+0000 7f237d09d640 1 --2- 172.21.15.27:0/1837003224 >> v2:172.21.15.192:3300/0 conn(0x7f2378154030 0x7f23780cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:06.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.262+0000 7f237d09d640 1 -- 172.21.15.27:0/1837003224 >> v1:172.21.15.27:6789/0 conn(0x7f23780af630 legacy=0x7f237815d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:06.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.262+0000 7f23777fe640 1 -- 172.21.15.27:0/1837003224 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 525307488 0 0) 0x7f23780cc190 con 0x7f23780af630 2023-12-19T18:22:06.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.262+0000 7f237d09d640 1 -- 172.21.15.27:0/1837003224 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23780cb580 con 0x7f23780c5560 2023-12-19T18:22:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.263+0000 7f237d09d640 1 --2- 172.21.15.27:0/1837003224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23780c5560 0x7f23780afac0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f23700097a0 tx=0x7f237002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac17491ea9287c07 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.263+0000 7f23777fe640 1 -- 172.21.15.27:0/1837003224 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2370031830 con 0x7f23780c5560 2023-12-19T18:22:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.263+0000 7f23777fe640 1 -- 172.21.15.27:0/1837003224 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2370031990 con 0x7f23780c5560 2023-12-19T18:22:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.263+0000 7f23777fe640 1 -- 172.21.15.27:0/1837003224 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2370031d20 con 0x7f23780c5560 2023-12-19T18:22:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.264+0000 7f237d89e640 1 -- 172.21.15.27:0/1837003224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23780c5560 msgr2=0x7f23780afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.264+0000 7f237d89e640 1 --2- 172.21.15.27:0/1837003224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23780c5560 0x7f23780afac0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f23700097a0 tx=0x7f237002b8d0 comp rx=0 tx=0).stop 2023-12-19T18:22:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.264+0000 7f237d89e640 1 -- 172.21.15.27:0/1837003224 shutdown_connections 2023-12-19T18:22:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.264+0000 7f237d89e640 1 --2- 172.21.15.27:0/1837003224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23780c5560 0x7f23780afac0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:06.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.264+0000 7f237d89e640 1 --2- 172.21.15.27:0/1837003224 >> v2:172.21.15.192:3300/0 conn(0x7f2378154030 0x7f23780cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:06.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.264+0000 7f237d89e640 1 -- 172.21.15.27:0/1837003224 >> 172.21.15.27:0/1837003224 conn(0x7f2378069c70 msgr2=0x7f237806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:06.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.264+0000 7f237d89e640 1 -- 172.21.15.27:0/1837003224 shutdown_connections 2023-12-19T18:22:06.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.264+0000 7f237d89e640 1 -- 172.21.15.27:0/1837003224 wait complete. 2023-12-19T18:22:06.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.266+0000 7f237d89e640 1 Processor -- start 2023-12-19T18:22:06.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.266+0000 7f237d89e640 1 -- start start 2023-12-19T18:22:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.267+0000 7f237d89e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23780af630 0x7f23780c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.267+0000 7f237d89e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23780c5560 0x7f23780b2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.267+0000 7f237d89e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2378154030 0x7f23780c4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.267+0000 7f237d89e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23780b38e0 con 0x7f2378154030 2023-12-19T18:22:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.267+0000 7f237d89e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23780b3bc0 con 0x7f23780c5560 2023-12-19T18:22:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.267+0000 7f237d89e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23780b3ea0 con 0x7f23780af630 2023-12-19T18:22:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.268+0000 7f237d09d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2378154030 0x7f23780c4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.268+0000 7f237d09d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2378154030 0x7f23780c4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37702/0 (socket says 172.21.15.27:37702) 2023-12-19T18:22:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.268+0000 7f237d09d640 1 -- 172.21.15.27:0/2811028086 learned_addr learned my addr 172.21.15.27:0/2811028086 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.268+0000 7f237c89c640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23780af630 0x7f23780c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.268+0000 7f2377fff640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23780c5560 0x7f23780b2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.268+0000 7f237d09d640 1 -- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23780af630 msgr2=0x7f23780c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.268+0000 7f237d09d640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23780af630 0x7f23780c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.268+0000 7f237d09d640 1 -- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23780c5560 msgr2=0x7f23780b2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.268+0000 7f237d09d640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23780c5560 0x7f23780b2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.268+0000 7f237d09d640 1 -- 172.21.15.27:0/2811028086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2370009400 con 0x7f2378154030 2023-12-19T18:22:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.268+0000 7f2377fff640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23780c5560 0x7f23780b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.268+0000 7f237d09d640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2378154030 0x7f23780c4550 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f23700098c0 tx=0x7f23700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.269+0000 7f23757fa640 1 -- 172.21.15.27:0/2811028086 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23700317d0 con 0x7f2378154030 2023-12-19T18:22:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.269+0000 7f23757fa640 1 -- 172.21.15.27:0/2811028086 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237003b9f0 con 0x7f2378154030 2023-12-19T18:22:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.269+0000 7f23757fa640 1 -- 172.21.15.27:0/2811028086 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237003bda0 con 0x7f2378154030 2023-12-19T18:22:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.269+0000 7f237d89e640 1 -- 172.21.15.27:0/2811028086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23780b4180 con 0x7f2378154030 2023-12-19T18:22:06.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.269+0000 7f237d89e640 1 -- 172.21.15.27:0/2811028086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23781f3220 con 0x7f2378154030 2023-12-19T18:22:06.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.270+0000 7f237d89e640 1 -- 172.21.15.27:0/2811028086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237804e840 con 0x7f2378154030 2023-12-19T18:22:06.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.271+0000 7f23757fa640 1 -- 172.21.15.27:0/2811028086 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f23700453f0 con 0x7f2378154030 2023-12-19T18:22:06.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.273+0000 7f23757fa640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f236406a430 0x7f236406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.273+0000 7f23757fa640 1 -- 172.21.15.27:0/2811028086 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f23700aa910 con 0x7f2378154030 2023-12-19T18:22:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.273+0000 7f237c89c640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f236406a430 0x7f236406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:06.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.274+0000 7f237c89c640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f236406a430 0x7f236406c8f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f237815e1d0 tx=0x7f23680039b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.277+0000 7f23757fa640 1 -- 172.21.15.27:0/2811028086 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2370076320 con 0x7f2378154030 2023-12-19T18:22:06.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:06.547+0000 7f237d89e640 1 -- 172.21.15.27:0/2811028086 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7f23780c3c50 con 0x7f2378154030 2023-12-19T18:22:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.001+0000 7f23757fa640 1 -- 172.21.15.27:0/2811028086 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v366) v1 ==== 131+0+0 (secure 0 0 0) 0x7f23700350c0 con 0x7f2378154030 2023-12-19T18:22:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.004+0000 7f237d89e640 1 -- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f236406a430 msgr2=0x7f236406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.004+0000 7f237d89e640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f236406a430 0x7f236406c8f0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f237815e1d0 tx=0x7f23680039b0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.004+0000 7f237d89e640 1 -- 172.21.15.27:0/2811028086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2378154030 msgr2=0x7f23780c4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.004+0000 7f237d89e640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2378154030 0x7f23780c4550 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f23700098c0 tx=0x7f23700046c0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.004+0000 7f237d89e640 1 -- 172.21.15.27:0/2811028086 shutdown_connections 2023-12-19T18:22:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.004+0000 7f237d89e640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f236406a430 0x7f236406c8f0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.004+0000 7f237d89e640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2378154030 0x7f23780c4550 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.004+0000 7f237d89e640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23780c5560 0x7f23780b2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.004+0000 7f237d89e640 1 --2- 172.21.15.27:0/2811028086 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23780af630 0x7f23780c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.004+0000 7f237d89e640 1 -- 172.21.15.27:0/2811028086 >> 172.21.15.27:0/2811028086 conn(0x7f2378069c70 msgr2=0x7f23780ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.005+0000 7f237d89e640 1 -- 172.21.15.27:0/2811028086 shutdown_connections 2023-12-19T18:22:07.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.005+0000 7f237d89e640 1 -- 172.21.15.27:0/2811028086 wait complete. 2023-12-19T18:22:07.026 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:22:07.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.267+0000 7f88359b9640 1 Processor -- start 2023-12-19T18:22:07.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.267+0000 7f88359b9640 1 -- start start 2023-12-19T18:22:07.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.269+0000 7f88359b9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f883015b1a0 0x7f883006f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:07.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.269+0000 7f88359b9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8830071420 0x7f883015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:07.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.270+0000 7f88359b9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f883006bc80 con 0x7f88300c2720 2023-12-19T18:22:07.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.270+0000 7f88359b9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f883006d690 con 0x7f883015b1a0 2023-12-19T18:22:07.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.270+0000 7f88359b9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f883006d0a0 con 0x7f8830071420 2023-12-19T18:22:07.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.270+0000 7f882effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f883015b1a0 0x7f883006f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:07.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.270+0000 7f882effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f883015b1a0 0x7f883006f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37704/0 (socket says 172.21.15.27:37704) 2023-12-19T18:22:07.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.270+0000 7f882effd640 1 -- 172.21.15.27:0/3871194808 learned_addr learned my addr 172.21.15.27:0/3871194808 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:07.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.270+0000 7f882f7fe640 1 --2- 172.21.15.27:0/3871194808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8830071420 0x7f883015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:07.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.270+0000 7f882effd640 1 -- 172.21.15.27:0/3871194808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8830071420 msgr2=0x7f883015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:07.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.270+0000 7f882effd640 1 --2- 172.21.15.27:0/3871194808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8830071420 0x7f883015ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.270+0000 7f882effd640 1 -- 172.21.15.27:0/3871194808 >> v1:172.21.15.192:6789/0 conn(0x7f88300c2720 legacy=0x7f883006fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:07.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.270+0000 7f882effd640 1 -- 172.21.15.27:0/3871194808 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f883006b6c0 con 0x7f883015b1a0 2023-12-19T18:22:07.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.271+0000 7f882effd640 1 --2- 172.21.15.27:0/3871194808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883015b1a0 0x7f883006f520 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f8820009870 tx=0x7f882002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2707e1e0d0ddd7ba server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:07.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.272+0000 7f882e7fc640 1 -- 172.21.15.27:0/3871194808 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88200317c0 con 0x7f883015b1a0 2023-12-19T18:22:07.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.272+0000 7f882e7fc640 1 -- 172.21.15.27:0/3871194808 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8820031920 con 0x7f883015b1a0 2023-12-19T18:22:07.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.272+0000 7f882e7fc640 1 -- 172.21.15.27:0/3871194808 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8820031cb0 con 0x7f883015b1a0 2023-12-19T18:22:07.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.273+0000 7f88359b9640 1 -- 172.21.15.27:0/3871194808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883015b1a0 msgr2=0x7f883006f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:07.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.273+0000 7f88359b9640 1 --2- 172.21.15.27:0/3871194808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883015b1a0 0x7f883006f520 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f8820009870 tx=0x7f882002b860 comp rx=0 tx=0).stop 2023-12-19T18:22:07.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.273+0000 7f88359b9640 1 -- 172.21.15.27:0/3871194808 shutdown_connections 2023-12-19T18:22:07.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.273+0000 7f88359b9640 1 --2- 172.21.15.27:0/3871194808 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8830071420 0x7f883015ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.273+0000 7f88359b9640 1 --2- 172.21.15.27:0/3871194808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f883015b1a0 0x7f883006f520 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.273+0000 7f88359b9640 1 -- 172.21.15.27:0/3871194808 >> 172.21.15.27:0/3871194808 conn(0x7f88300cebd0 msgr2=0x7f88300cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:07.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.273+0000 7f88359b9640 1 -- 172.21.15.27:0/3871194808 shutdown_connections 2023-12-19T18:22:07.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.273+0000 7f88359b9640 1 -- 172.21.15.27:0/3871194808 wait complete. 2023-12-19T18:22:07.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.275+0000 7f88359b9640 1 Processor -- start 2023-12-19T18:22:07.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.275+0000 7f88359b9640 1 -- start start 2023-12-19T18:22:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.276+0000 7f88359b9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8830071420 0x7f88300c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.276+0000 7f88359b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88300c2720 0x7f8830156660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.276+0000 7f88359b9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f883015b1a0 0x7f8830156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.276+0000 7f88359b9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88301498e0 con 0x7f88300c2720 2023-12-19T18:22:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.276+0000 7f88359b9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88300acd40 con 0x7f883015b1a0 2023-12-19T18:22:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.276+0000 7f88359b9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88300ad020 con 0x7f8830071420 2023-12-19T18:22:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.276+0000 7f88267fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88300c2720 0x7f8830156660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.277+0000 7f88267fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88300c2720 0x7f8830156660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37718/0 (socket says 172.21.15.27:37718) 2023-12-19T18:22:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.277+0000 7f88267fc640 1 -- 172.21.15.27:0/3484349284 learned_addr learned my addr 172.21.15.27:0/3484349284 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.277+0000 7f882effd640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8830071420 0x7f88300c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:07.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.277+0000 7f882f7fe640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f883015b1a0 0x7f8830156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.277+0000 7f88267fc640 1 -- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8830071420 msgr2=0x7f88300c5d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.277+0000 7f88267fc640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8830071420 0x7f88300c5d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.277+0000 7f88267fc640 1 -- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f883015b1a0 msgr2=0x7f8830156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.277+0000 7f88267fc640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f883015b1a0 0x7f8830156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.277+0000 7f88267fc640 1 -- 172.21.15.27:0/3484349284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88200094d0 con 0x7f88300c2720 2023-12-19T18:22:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.277+0000 7f882f7fe640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f883015b1a0 0x7f8830156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.277+0000 7f88267fc640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88300c2720 0x7f8830156660 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f881800c7c0 tx=0x7f881800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.278+0000 7f882cff9640 1 -- 172.21.15.27:0/3484349284 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f881800aa50 con 0x7f88300c2720 2023-12-19T18:22:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.278+0000 7f882cff9640 1 -- 172.21.15.27:0/3484349284 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f881800abb0 con 0x7f88300c2720 2023-12-19T18:22:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.278+0000 7f882cff9640 1 -- 172.21.15.27:0/3484349284 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8818012420 con 0x7f88300c2720 2023-12-19T18:22:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.278+0000 7f88359b9640 1 -- 172.21.15.27:0/3484349284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88300ad300 con 0x7f88300c2720 2023-12-19T18:22:07.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.278+0000 7f88359b9640 1 -- 172.21.15.27:0/3484349284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88300ae280 con 0x7f88300c2720 2023-12-19T18:22:07.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.279+0000 7f88359b9640 1 -- 172.21.15.27:0/3484349284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88300cfa30 con 0x7f88300c2720 2023-12-19T18:22:07.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.280+0000 7f882cff9640 1 -- 172.21.15.27:0/3484349284 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8818012580 con 0x7f88300c2720 2023-12-19T18:22:07.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.281+0000 7f882cff9640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f881006a460 0x7f881006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:07.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.281+0000 7f882cff9640 1 -- 172.21.15.27:0/3484349284 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f88180885c0 con 0x7f88300c2720 2023-12-19T18:22:07.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.281+0000 7f882effd640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f881006a460 0x7f881006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:07.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.283+0000 7f882effd640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f881006a460 0x7f881006c920 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8820036000 tx=0x7f882003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:07.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.286+0000 7f882cff9640 1 -- 172.21.15.27:0/3484349284 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8818053e10 con 0x7f88300c2720 2023-12-19T18:22:07.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.483+0000 7f88359b9640 1 -- 172.21.15.27:0/3484349284 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f8830068e90 con 0x7f88300c2720 2023-12-19T18:22:07.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.491+0000 7f882cff9640 1 -- 172.21.15.27:0/3484349284 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7f88180160c0 con 0x7f88300c2720 2023-12-19T18:22:07.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.494+0000 7f88359b9640 1 -- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f881006a460 msgr2=0x7f881006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:07.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.494+0000 7f88359b9640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f881006a460 0x7f881006c920 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8820036000 tx=0x7f882003d040 comp rx=0 tx=0).stop 2023-12-19T18:22:07.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.494+0000 7f88359b9640 1 -- 172.21.15.27:0/3484349284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88300c2720 msgr2=0x7f8830156660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:07.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.494+0000 7f88359b9640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88300c2720 0x7f8830156660 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f881800c7c0 tx=0x7f881800cc80 comp rx=0 tx=0).stop 2023-12-19T18:22:07.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.495+0000 7f88359b9640 1 -- 172.21.15.27:0/3484349284 shutdown_connections 2023-12-19T18:22:07.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.495+0000 7f88359b9640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f881006a460 0x7f881006c920 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.495+0000 7f88359b9640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f883015b1a0 0x7f8830156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.495+0000 7f88359b9640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88300c2720 0x7f8830156660 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.495+0000 7f88359b9640 1 --2- 172.21.15.27:0/3484349284 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8830071420 0x7f88300c5d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:07.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.495+0000 7f88359b9640 1 -- 172.21.15.27:0/3484349284 >> 172.21.15.27:0/3484349284 conn(0x7f88300cebd0 msgr2=0x7f88300cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:07.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.495+0000 7f88359b9640 1 -- 172.21.15.27:0/3484349284 shutdown_connections 2023-12-19T18:22:07.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:07.495+0000 7f88359b9640 1 -- 172.21.15.27:0/3484349284 wait complete. 2023-12-19T18:22:07.514 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:22:07.514 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:22:07.637 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:22:07.768 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:22:07.769 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:22:07.769 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:22:07.835 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:22:07.835 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:22:07.835 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:22:07.895 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:22:07.895 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:22:07.895 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:22:07.960 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:22:07.960 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:22:07.960 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:22:08.020 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:22:08.020 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:22:08.021 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:22:08.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.258+0000 7f0f8e3ca640 1 Processor -- start 2023-12-19T18:22:08.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.258+0000 7f0f8e3ca640 1 -- start start 2023-12-19T18:22:08.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.260+0000 7f0f8e3ca640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0f880cb300 0x7f0f880cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:08.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.260+0000 7f0f8e3ca640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0f88161010 0x7f0f880cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:08.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.260+0000 7f0f8e3ca640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f880cbc20 con 0x7f0f880c8f90 2023-12-19T18:22:08.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.260+0000 7f0f8e3ca640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f880c4480 con 0x7f0f880cb300 2023-12-19T18:22:08.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.260+0000 7f0f8e3ca640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f881468c0 con 0x7f0f88161010 2023-12-19T18:22:08.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.260+0000 7f0f8cbc7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0f880c8f90 0x7f0f88160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34830/0 (socket says 172.21.15.27:34830) 2023-12-19T18:22:08.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.260+0000 7f0f8cbc7640 1 -- 172.21.15.27:0/649594025 learned_addr learned my addr 172.21.15.27:0/649594025 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:08.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.260+0000 7f0f8d3c8640 1 --2- 172.21.15.27:0/649594025 >> v2:172.21.15.27:3300/0 conn(0x7f0f880cb300 0x7f0f880cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:08.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.261+0000 7f0f8dbc9640 1 --2- 172.21.15.27:0/649594025 >> v2:172.21.15.192:3300/0 conn(0x7f0f88161010 0x7f0f880cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:08.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.261+0000 7f0f7ffff640 1 -- 172.21.15.27:0/649594025 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2580841467 0 0) 0x7f0f880cbc20 con 0x7f0f880c8f90 2023-12-19T18:22:08.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.261+0000 7f0f7ffff640 1 -- 172.21.15.27:0/649594025 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f74003520 con 0x7f0f880c8f90 2023-12-19T18:22:08.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.261+0000 7f0f8d3c8640 1 -- 172.21.15.27:0/649594025 >> v2:172.21.15.192:3300/0 conn(0x7f0f88161010 msgr2=0x7f0f880cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:08.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.261+0000 7f0f8d3c8640 1 --2- 172.21.15.27:0/649594025 >> v2:172.21.15.192:3300/0 conn(0x7f0f88161010 0x7f0f880cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.261+0000 7f0f8d3c8640 1 -- 172.21.15.27:0/649594025 >> v1:172.21.15.27:6789/0 conn(0x7f0f880c8f90 legacy=0x7f0f88160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:08.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.261+0000 7f0f8d3c8640 1 -- 172.21.15.27:0/649594025 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f88145d60 con 0x7f0f880cb300 2023-12-19T18:22:08.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.263+0000 7f0f8d3c8640 1 --2- 172.21.15.27:0/649594025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f880cb300 0x7f0f880cb6e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0f840097a0 tx=0x7f0f8402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838c3d1dc9d21581 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:08.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.263+0000 7f0f7ffff640 1 -- 172.21.15.27:0/649594025 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84031830 con 0x7f0f880cb300 2023-12-19T18:22:08.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.263+0000 7f0f7ffff640 1 -- 172.21.15.27:0/649594025 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f84031990 con 0x7f0f880cb300 2023-12-19T18:22:08.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.263+0000 7f0f7ffff640 1 -- 172.21.15.27:0/649594025 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f84031d20 con 0x7f0f880cb300 2023-12-19T18:22:08.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.263+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/649594025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f880cb300 msgr2=0x7f0f880cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:08.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.263+0000 7f0f8e3ca640 1 --2- 172.21.15.27:0/649594025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f880cb300 0x7f0f880cb6e0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f0f840097a0 tx=0x7f0f8402b8d0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.264+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/649594025 shutdown_connections 2023-12-19T18:22:08.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.264+0000 7f0f8e3ca640 1 --2- 172.21.15.27:0/649594025 >> v2:172.21.15.192:3300/0 conn(0x7f0f88161010 0x7f0f880cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.264+0000 7f0f8e3ca640 1 --2- 172.21.15.27:0/649594025 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f880cb300 0x7f0f880cb6e0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.264+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/649594025 >> 172.21.15.27:0/649594025 conn(0x7f0f88069c70 msgr2=0x7f0f8806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:08.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.264+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/649594025 shutdown_connections 2023-12-19T18:22:08.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.264+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/649594025 wait complete. 2023-12-19T18:22:08.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.265+0000 7f0f8e3ca640 1 Processor -- start 2023-12-19T18:22:08.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.266+0000 7f0f8e3ca640 1 -- start start 2023-12-19T18:22:08.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8e3ca640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f880c8f90 0x7f0f881f2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:08.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8e3ca640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f880cb300 0x7f0f881f6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:08.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8e3ca640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f88161010 0x7f0f8806c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:08.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8e3ca640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f88142af0 con 0x7f0f880cb300 2023-12-19T18:22:08.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8e3ca640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f881f6a70 con 0x7f0f88161010 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8e3ca640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f8815f4e0 con 0x7f0f880c8f90 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8cbc7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f880cb300 0x7f0f881f6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8cbc7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f880cb300 0x7f0f881f6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37722/0 (socket says 172.21.15.27:37722) 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8cbc7640 1 -- 172.21.15.27:0/2720553010 learned_addr learned my addr 172.21.15.27:0/2720553010 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8d3c8640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f880c8f90 0x7f0f881f2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8cbc7640 1 -- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f880c8f90 msgr2=0x7f0f881f2020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8cbc7640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f880c8f90 0x7f0f881f2020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8dbc9640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f88161010 0x7f0f8806c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8cbc7640 1 -- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f88161010 msgr2=0x7f0f8806c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8cbc7640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f88161010 0x7f0f8806c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8cbc7640 1 -- 172.21.15.27:0/2720553010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f84009400 con 0x7f0f880cb300 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.267+0000 7f0f8dbc9640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f88161010 0x7f0f8806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.268+0000 7f0f8cbc7640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f880cb300 0x7f0f881f6530 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800c410 tx=0x7f0f7800c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:08.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.268+0000 7f0f7dffb640 1 -- 172.21.15.27:0/2720553010 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f78005ce0 con 0x7f0f880cb300 2023-12-19T18:22:08.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.268+0000 7f0f7dffb640 1 -- 172.21.15.27:0/2720553010 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f78005e40 con 0x7f0f880cb300 2023-12-19T18:22:08.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.268+0000 7f0f7dffb640 1 -- 172.21.15.27:0/2720553010 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f780072f0 con 0x7f0f880cb300 2023-12-19T18:22:08.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.268+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/2720553010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8815f7c0 con 0x7f0f880cb300 2023-12-19T18:22:08.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.268+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/2720553010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8815fd80 con 0x7f0f880cb300 2023-12-19T18:22:08.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.269+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/2720553010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f8804e840 con 0x7f0f880cb300 2023-12-19T18:22:08.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.270+0000 7f0f7dffb640 1 -- 172.21.15.27:0/2720553010 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0f78007510 con 0x7f0f880cb300 2023-12-19T18:22:08.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.271+0000 7f0f7dffb640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0f6c06a430 0x7f0f6c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:08.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.272+0000 7f0f7dffb640 1 -- 172.21.15.27:0/2720553010 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0f78087c80 con 0x7f0f880cb300 2023-12-19T18:22:08.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.272+0000 7f0f8d3c8640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0f6c06a430 0x7f0f6c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:08.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.273+0000 7f0f8d3c8640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0f6c06a430 0x7f0f6c06c8f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0f84036000 tx=0x7f0f8403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:08.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.276+0000 7f0f7dffb640 1 -- 172.21.15.27:0/2720553010 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0f78053620 con 0x7f0f880cb300 2023-12-19T18:22:08.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.514+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/2720553010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0f880adc50 con 0x7f0f880cb300 2023-12-19T18:22:08.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.515+0000 7f0f7dffb640 1 -- 172.21.15.27:0/2720553010 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f0f78017de0 con 0x7f0f880cb300 2023-12-19T18:22:08.515 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:08.515 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":48,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5348,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/2779436922","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2779436922},{"type":"v1","addr":"172.21.15.192:6836","nonce":2779436922}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5351,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3121062779","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3121062779},{"type":"v1","addr":"172.21.15.192:6837","nonce":3121062779}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":5357,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/2032553869","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":2032553869},{"type":"v1","addr":"172.21.15.27:6835","nonce":2032553869}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41},{"gid":7242,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/2172057174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2172057174},{"type":"v1","addr":"172.21.15.27:6834","nonce":2172057174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[]} 2023-12-19T18:22:08.515 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 48 2023-12-19T18:22:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.517+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0f6c06a430 msgr2=0x7f0f6c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.518+0000 7f0f8e3ca640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0f6c06a430 0x7f0f6c06c8f0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0f84036000 tx=0x7f0f8403c040 comp rx=0 tx=0).stop 2023-12-19T18:22:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.518+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/2720553010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f880cb300 msgr2=0x7f0f881f6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.518+0000 7f0f8e3ca640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f880cb300 0x7f0f881f6530 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f0f7800c410 tx=0x7f0f7800c8d0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.518+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/2720553010 shutdown_connections 2023-12-19T18:22:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.518+0000 7f0f8e3ca640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0f6c06a430 0x7f0f6c06c8f0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.518+0000 7f0f8e3ca640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0f88161010 0x7f0f8806c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.518+0000 7f0f8e3ca640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f880cb300 0x7f0f881f6530 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.518+0000 7f0f8e3ca640 1 --2- 172.21.15.27:0/2720553010 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0f880c8f90 0x7f0f881f2020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.518+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/2720553010 >> 172.21.15.27:0/2720553010 conn(0x7f0f88069c70 msgr2=0x7f0f880ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.518+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/2720553010 shutdown_connections 2023-12-19T18:22:08.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.518+0000 7f0f8e3ca640 1 -- 172.21.15.27:0/2720553010 wait complete. 2023-12-19T18:22:08.538 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:22:08.539 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:22:08.539 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:22:08.539 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:22:08.539 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:22:08.539 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:22:08.539 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:22:08.540 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:22:08.540 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:22:08.540 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:22:08.540 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:22:08.540 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:22:08.540 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:22:08.540 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:22:08.540 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:22:08.541 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:22:08.562 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:22:08.562 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:22:08.563 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:22:08.563 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:22:08.565 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:22:08.565 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:22:08.567 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:22:08.567 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:22:08.611 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:22:08.611 DEBUG:teuthology.parallel:result is None 2023-12-19T18:22:08.614 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:22:08.615 DEBUG:teuthology.parallel:result is None 2023-12-19T18:22:08.618 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:22:08.619 DEBUG:teuthology.parallel:result is None 2023-12-19T18:22:08.621 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:22:08.622 DEBUG:teuthology.parallel:result is None 2023-12-19T18:22:08.622 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:22:08.768 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-12-19T18:22:08.768 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-12-19T18:22:08.773 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-19T18:22:08.773 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-19T18:22:08.789 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:22:08.789 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:22:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.827+0000 7faec1028640 1 Processor -- start 2023-12-19T18:22:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.827+0000 7faec1028640 1 -- start start 2023-12-19T18:22:08.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.828+0000 7faec1028640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7faebc0cee80 0x7faebc0cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:08.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.828+0000 7faec1028640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faebc15a100 con 0x7faebc0ce9f0 2023-12-19T18:22:08.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.828+0000 7faec1028640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faebc0b1270 con 0x7faebc0ca2a0 2023-12-19T18:22:08.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.828+0000 7faec1028640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faebc15d050 con 0x7faebc0cee80 2023-12-19T18:22:08.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.828+0000 7faeb3fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faebc0ce9f0 0x7faebc1599f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34846/0 (socket says 172.21.15.27:34846) 2023-12-19T18:22:08.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.828+0000 7faeb3fff640 1 -- 172.21.15.27:0/1102124616 learned_addr learned my addr 172.21.15.27:0/1102124616 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:08.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.829+0000 7faebb59e640 1 --2- 172.21.15.27:0/1102124616 >> v2:172.21.15.192:3300/0 conn(0x7faebc0cee80 0x7faebc0cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:08.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.829+0000 7faeba59c640 1 -- 172.21.15.27:0/1102124616 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2086038351 0 0) 0x7faebc15a100 con 0x7faebc0ce9f0 2023-12-19T18:22:08.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.829+0000 7faeba59c640 1 -- 172.21.15.27:0/1102124616 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faea8005730 con 0x7faebc0ce9f0 2023-12-19T18:22:08.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.829+0000 7faeba59c640 1 -- 172.21.15.27:0/1102124616 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583031665 0 0) 0x7faebc0b1270 con 0x7faebc0ca2a0 2023-12-19T18:22:08.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.829+0000 7faeba59c640 1 -- 172.21.15.27:0/1102124616 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faebc15a100 con 0x7faebc0ca2a0 2023-12-19T18:22:08.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.829+0000 7faeba59c640 1 -- 172.21.15.27:0/1102124616 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1712977656 0 0) 0x7faea8005730 con 0x7faebc0ce9f0 2023-12-19T18:22:08.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.829+0000 7faebb59e640 1 -- 172.21.15.27:0/1102124616 >> v1:172.21.15.192:6789/0 conn(0x7faebc0ca2a0 legacy=0x7faebc072c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:08.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.829+0000 7faebb59e640 1 -- 172.21.15.27:0/1102124616 >> v1:172.21.15.27:6789/0 conn(0x7faebc0ce9f0 legacy=0x7faebc1599f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:08.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.829+0000 7faebb59e640 1 -- 172.21.15.27:0/1102124616 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faebc0b1550 con 0x7faebc0cee80 2023-12-19T18:22:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.831+0000 7faebb59e640 1 --2- 172.21.15.27:0/1102124616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc0cee80 0x7faebc0cc2e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009d90 tx=0x7faeb402bd80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9fe8e477663da09d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.831+0000 7faeba59c640 1 -- 172.21.15.27:0/1102124616 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faea4001670 con 0x7faebc0cee80 2023-12-19T18:22:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.831+0000 7faeba59c640 1 -- 172.21.15.27:0/1102124616 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb402c8b0 con 0x7faebc0cee80 2023-12-19T18:22:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.831+0000 7faeba59c640 1 -- 172.21.15.27:0/1102124616 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb402cc40 con 0x7faebc0cee80 2023-12-19T18:22:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.831+0000 7faec1028640 1 -- 172.21.15.27:0/1102124616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc0cee80 msgr2=0x7faebc0cc2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.831+0000 7faec1028640 1 --2- 172.21.15.27:0/1102124616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc0cee80 0x7faebc0cc2e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009d90 tx=0x7faeb402bd80 comp rx=0 tx=0).stop 2023-12-19T18:22:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.832+0000 7faec1028640 1 -- 172.21.15.27:0/1102124616 shutdown_connections 2023-12-19T18:22:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.832+0000 7faec1028640 1 --2- 172.21.15.27:0/1102124616 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc0cee80 0x7faebc0cc2e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.832+0000 7faec1028640 1 -- 172.21.15.27:0/1102124616 >> 172.21.15.27:0/1102124616 conn(0x7faebc06a060 msgr2=0x7faebc06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:08.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.832+0000 7faec1028640 1 -- 172.21.15.27:0/1102124616 shutdown_connections 2023-12-19T18:22:08.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.832+0000 7faec1028640 1 -- 172.21.15.27:0/1102124616 wait complete. 2023-12-19T18:22:08.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.833+0000 7faec1028640 1 Processor -- start 2023-12-19T18:22:08.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.834+0000 7faec1028640 1 -- start start 2023-12-19T18:22:08.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.834+0000 7faec1028640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faebc0ca2a0 0x7faebc158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:08.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.834+0000 7faec1028640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc0ce9f0 0x7faebc0c4160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:08.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.834+0000 7faec1028640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faebc0cee80 0x7faebc15bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.834+0000 7faec1028640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc0c5cc0 con 0x7faebc0ca2a0 2023-12-19T18:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.834+0000 7faec1028640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faebc0c4c20 con 0x7faebc0ce9f0 2023-12-19T18:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.834+0000 7faec1028640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faebc0c4f00 con 0x7faebc0cee80 2023-12-19T18:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.835+0000 7faebad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faebc0ca2a0 0x7faebc158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.835+0000 7faeb3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc0ce9f0 0x7faebc0c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.835+0000 7faebad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faebc0ca2a0 0x7faebc158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37760/0 (socket says 172.21.15.27:37760) 2023-12-19T18:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.835+0000 7faebad9d640 1 -- 172.21.15.27:0/2229357196 learned_addr learned my addr 172.21.15.27:0/2229357196 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.835+0000 7faebb59e640 1 --2- 172.21.15.27:0/2229357196 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faebc0cee80 0x7faebc15bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.835+0000 7faebad9d640 1 -- 172.21.15.27:0/2229357196 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faebc0cee80 msgr2=0x7faebc15bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.835+0000 7faebad9d640 1 --2- 172.21.15.27:0/2229357196 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faebc0cee80 0x7faebc15bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.835+0000 7faebad9d640 1 -- 172.21.15.27:0/2229357196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc0ce9f0 msgr2=0x7faebc0c4160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.835+0000 7faebad9d640 1 --2- 172.21.15.27:0/2229357196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc0ce9f0 0x7faebc0c4160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:08.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.835+0000 7faebad9d640 1 -- 172.21.15.27:0/2229357196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb4009a70 con 0x7faebc0ca2a0 2023-12-19T18:22:08.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.836+0000 7faebad9d640 1 --2- 172.21.15.27:0/2229357196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faebc0ca2a0 0x7faebc158760 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7faeac00c430 tx=0x7faeac00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.836+0000 7faeb37fe640 1 -- 172.21.15.27:0/2229357196 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac0052d0 con 0x7faebc0ca2a0 2023-12-19T18:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.836+0000 7faeb37fe640 1 -- 172.21.15.27:0/2229357196 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeac005450 con 0x7faebc0ca2a0 2023-12-19T18:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.836+0000 7faeb37fe640 1 -- 172.21.15.27:0/2229357196 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeac003830 con 0x7faebc0ca2a0 2023-12-19T18:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.836+0000 7faec1028640 1 -- 172.21.15.27:0/2229357196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faebc0c51e0 con 0x7faebc0ca2a0 2023-12-19T18:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.836+0000 7faec1028640 1 -- 172.21.15.27:0/2229357196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faebc0c57a0 con 0x7faebc0ca2a0 2023-12-19T18:22:08.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.837+0000 7faec1028640 1 -- 172.21.15.27:0/2229357196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faebc068e90 con 0x7faebc0ca2a0 2023-12-19T18:22:08.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.839+0000 7faeb37fe640 1 -- 172.21.15.27:0/2229357196 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faeac003990 con 0x7faebc0ca2a0 2023-12-19T18:22:08.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.839+0000 7faeb37fe640 1 --2- 172.21.15.27:0/2229357196 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae9c06a480 0x7fae9c06c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:08.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.840+0000 7faeb37fe640 1 -- 172.21.15.27:0/2229357196 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 4943+0+0 (secure 0 0 0) 0x7faeac087b90 con 0x7faebc0ca2a0 2023-12-19T18:22:08.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.840+0000 7faeb3fff640 1 --2- 172.21.15.27:0/2229357196 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae9c06a480 0x7fae9c06c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:08.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.841+0000 7faeb3fff640 1 --2- 172.21.15.27:0/2229357196 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae9c06a480 0x7fae9c06c940 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7faea4001b50 tx=0x7faea4001ae0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:08.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:08.844+0000 7faeb37fe640 1 -- 172.21.15.27:0/2229357196 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7faeac0535a0 con 0x7faebc0ca2a0 2023-12-19T18:22:09.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.052+0000 7faec1028640 1 -- 172.21.15.27:0/2229357196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7faebc06ce10 con 0x7faebc0ca2a0 2023-12-19T18:22:09.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.825+0000 7faeb37fe640 1 -- 172.21.15.27:0/2229357196 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v131) v1 ==== 97+0+0 (secure 0 0 0) 0x7faeac056bc0 con 0x7faebc0ca2a0 2023-12-19T18:22:09.826 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2023-12-19T18:22:09.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.829+0000 7faec1028640 1 -- 172.21.15.27:0/2229357196 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae9c06a480 msgr2=0x7fae9c06c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:09.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.829+0000 7faec1028640 1 --2- 172.21.15.27:0/2229357196 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae9c06a480 0x7fae9c06c940 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7faea4001b50 tx=0x7faea4001ae0 comp rx=0 tx=0).stop 2023-12-19T18:22:09.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.829+0000 7faec1028640 1 -- 172.21.15.27:0/2229357196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faebc0ca2a0 msgr2=0x7faebc158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:09.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.829+0000 7faec1028640 1 --2- 172.21.15.27:0/2229357196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faebc0ca2a0 0x7faebc158760 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7faeac00c430 tx=0x7faeac00c8f0 comp rx=0 tx=0).stop 2023-12-19T18:22:09.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.829+0000 7faec1028640 1 -- 172.21.15.27:0/2229357196 shutdown_connections 2023-12-19T18:22:09.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.829+0000 7faec1028640 1 --2- 172.21.15.27:0/2229357196 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae9c06a480 0x7fae9c06c940 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:09.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.829+0000 7faec1028640 1 --2- 172.21.15.27:0/2229357196 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faebc0cee80 0x7faebc15bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:09.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.829+0000 7faec1028640 1 --2- 172.21.15.27:0/2229357196 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faebc0ce9f0 0x7faebc0c4160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:09.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.829+0000 7faec1028640 1 --2- 172.21.15.27:0/2229357196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faebc0ca2a0 0x7faebc158760 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:09.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.829+0000 7faec1028640 1 -- 172.21.15.27:0/2229357196 >> 172.21.15.27:0/2229357196 conn(0x7faebc06a060 msgr2=0x7faebc06a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:09.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.830+0000 7faec1028640 1 -- 172.21.15.27:0/2229357196 shutdown_connections 2023-12-19T18:22:09.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:09.830+0000 7faec1028640 1 -- 172.21.15.27:0/2229357196 wait complete. 2023-12-19T18:22:09.851 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:22:10.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.086+0000 7f2df7fff640 1 Processor -- start 2023-12-19T18:22:10.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.087+0000 7f2df7fff640 1 -- start start 2023-12-19T18:22:10.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.089+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df8070fb0 0x7f2df806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:10.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.089+0000 7f2df7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2df806b850 0x7f2df806f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:10.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.089+0000 7f2df7fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2df80adf10 con 0x7f2df815ac50 2023-12-19T18:22:10.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.089+0000 7f2df7fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2df80adac0 con 0x7f2df806b850 2023-12-19T18:22:10.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.089+0000 7f2df7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df80af630 con 0x7f2df8070fb0 2023-12-19T18:22:10.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.089+0000 7f2df77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2df806b850 0x7f2df806f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:10.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.090+0000 7f2df77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2df806b850 0x7f2df806f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37772/0 (socket says 172.21.15.27:37772) 2023-12-19T18:22:10.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.090+0000 7f2df77fe640 1 -- 172.21.15.27:0/4192266064 learned_addr learned my addr 172.21.15.27:0/4192266064 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:10.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.090+0000 7f2deffff640 1 --2- 172.21.15.27:0/4192266064 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df8070fb0 0x7f2df806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:10.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.090+0000 7f2df77fe640 1 -- 172.21.15.27:0/4192266064 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df8070fb0 msgr2=0x7f2df806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:10.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.090+0000 7f2df77fe640 1 --2- 172.21.15.27:0/4192266064 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df8070fb0 0x7f2df806b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.090+0000 7f2df77fe640 1 -- 172.21.15.27:0/4192266064 >> v1:172.21.15.192:6789/0 conn(0x7f2df815ac50 legacy=0x7f2df80aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:10.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.090+0000 7f2df77fe640 1 -- 172.21.15.27:0/4192266064 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df80713f0 con 0x7f2df806b850 2023-12-19T18:22:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.092+0000 7f2df77fe640 1 --2- 172.21.15.27:0/4192266064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df806b850 0x7f2df806f650 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009e50 tx=0x7f2df002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adcda14d76fe085f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:10.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.092+0000 7f2df67fc640 1 -- 172.21.15.27:0/4192266064 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df002c9f0 con 0x7f2df806b850 2023-12-19T18:22:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.092+0000 7f2df67fc640 1 -- 172.21.15.27:0/4192266064 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df002cb50 con 0x7f2df806b850 2023-12-19T18:22:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.092+0000 7f2df67fc640 1 -- 172.21.15.27:0/4192266064 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df00343f0 con 0x7f2df806b850 2023-12-19T18:22:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.093+0000 7f2df7fff640 1 -- 172.21.15.27:0/4192266064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df806b850 msgr2=0x7f2df806f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.093+0000 7f2df7fff640 1 --2- 172.21.15.27:0/4192266064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df806b850 0x7f2df806f650 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009e50 tx=0x7f2df002be50 comp rx=0 tx=0).stop 2023-12-19T18:22:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.093+0000 7f2df7fff640 1 -- 172.21.15.27:0/4192266064 shutdown_connections 2023-12-19T18:22:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.093+0000 7f2df7fff640 1 --2- 172.21.15.27:0/4192266064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df806b850 0x7f2df806f650 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.093+0000 7f2df7fff640 1 --2- 172.21.15.27:0/4192266064 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df8070fb0 0x7f2df806b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.093+0000 7f2df7fff640 1 -- 172.21.15.27:0/4192266064 >> 172.21.15.27:0/4192266064 conn(0x7f2df806a7d0 msgr2=0x7f2df80ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.093+0000 7f2df7fff640 1 -- 172.21.15.27:0/4192266064 shutdown_connections 2023-12-19T18:22:10.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.093+0000 7f2df7fff640 1 -- 172.21.15.27:0/4192266064 wait complete. 2023-12-19T18:22:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.095+0000 7f2df7fff640 1 Processor -- start 2023-12-19T18:22:10.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.096+0000 7f2df7fff640 1 -- start start 2023-12-19T18:22:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.097+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df806b850 0x7f2df8154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.097+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df8070fb0 0x7f2df8152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.097+0000 7f2df7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df815ac50 0x7f2df8148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.097+0000 7f2df7fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df8157d40 con 0x7f2df815ac50 2023-12-19T18:22:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.097+0000 7f2df7fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df8152190 con 0x7f2df806b850 2023-12-19T18:22:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.097+0000 7f2df7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2df8151bd0 con 0x7f2df8070fb0 2023-12-19T18:22:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.097+0000 7f2df77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df815ac50 0x7f2df8148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.097+0000 7f2df77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df815ac50 0x7f2df8148de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37774/0 (socket says 172.21.15.27:37774) 2023-12-19T18:22:10.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.097+0000 7f2df77fe640 1 -- 172.21.15.27:0/1512612856 learned_addr learned my addr 172.21.15.27:0/1512612856 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.097+0000 7f2deffff640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df8070fb0 0x7f2df8152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.097+0000 7f2df6ffd640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df806b850 0x7f2df8154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.098+0000 7f2df77fe640 1 -- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df8070fb0 msgr2=0x7f2df8152730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.098+0000 7f2df77fe640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df8070fb0 0x7f2df8152730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.098+0000 7f2df77fe640 1 -- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df806b850 msgr2=0x7f2df8154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.098+0000 7f2df77fe640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df806b850 0x7f2df8154cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.098+0000 7f2df77fe640 1 -- 172.21.15.27:0/1512612856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2df0009b30 con 0x7f2df815ac50 2023-12-19T18:22:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.098+0000 7f2df77fe640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df815ac50 0x7f2df8148de0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009e50 tx=0x7f2df0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:10.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.098+0000 7f2def7fe640 1 -- 172.21.15.27:0/1512612856 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df0009510 con 0x7f2df815ac50 2023-12-19T18:22:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.098+0000 7f2def7fe640 1 -- 172.21.15.27:0/1512612856 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2df003e450 con 0x7f2df815ac50 2023-12-19T18:22:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.098+0000 7f2df7fff640 1 -- 172.21.15.27:0/1512612856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df8151eb0 con 0x7f2df815ac50 2023-12-19T18:22:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.098+0000 7f2def7fe640 1 -- 172.21.15.27:0/1512612856 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2df003e800 con 0x7f2df815ac50 2023-12-19T18:22:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.098+0000 7f2df7fff640 1 -- 172.21.15.27:0/1512612856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df8155200 con 0x7f2df815ac50 2023-12-19T18:22:10.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.100+0000 7f2df7fff640 1 -- 172.21.15.27:0/1512612856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df8157e80 con 0x7f2df815ac50 2023-12-19T18:22:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.101+0000 7f2def7fe640 1 -- 172.21.15.27:0/1512612856 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2df003e960 con 0x7f2df815ac50 2023-12-19T18:22:10.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.102+0000 7f2def7fe640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2dd806a430 0x7f2dd806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.102+0000 7f2def7fe640 1 -- 172.21.15.27:0/1512612856 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2df00ab320 con 0x7f2df815ac50 2023-12-19T18:22:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.102+0000 7f2df6ffd640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2dd806a430 0x7f2dd806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:10.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.104+0000 7f2df6ffd640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2dd806a430 0x7f2dd806c8f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2df8150be0 tx=0x7f2de8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:10.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.106+0000 7f2def7fe640 1 -- 172.21.15.27:0/1512612856 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2df8157e80 con 0x7f2df815ac50 2023-12-19T18:22:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.373+0000 7f2df7fff640 1 -- 172.21.15.27:0/1512612856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f2df8157e80 con 0x7f2df815ac50 2023-12-19T18:22:10.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.373+0000 7f2def7fe640 1 -- 172.21.15.27:0/1512612856 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f2df8157e80 con 0x7f2df815ac50 2023-12-19T18:22:10.374 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:10.374 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:22:10.374 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2023-12-19T18:22:10.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.374 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2023-12-19T18:22:10.374 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFl+4ruHBAAXwbpmT43SmlvNbuZj5e+bQ==", 2023-12-19T18:22:10.374 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.374 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:22:10.374 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmZyUBhAAG3kZkXvJIo7N9f1j+DPCSQ==", 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.376 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2023-12-19T18:22:10.376 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlChNOMBAAX4GV3iIGkSQdAsYu9oCPDw==", 2023-12-19T18:22:10.376 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.376 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:22:10.376 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:22:10.376 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:22:10.376 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.376 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.376 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2023-12-19T18:22:10.376 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlTmFXFhAABA0NMUxKBV1ExAhYc5RFbQ==", 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFluleFIBAAs1goFVMOX4CbjwFhg0KoBg==", 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:22:10.377 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFlInW3MxAAI4hY1GBxXYXDswKKrEyDTg==", 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFlAWZ5CRAA82/2r9/sT0L9VlGHbXksQw==", 2023-12-19T18:22:10.378 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFl1IvaGRAAqVacDDPLx+NqBOGp6MeX2w==", 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.379 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlr0BBHRAA+7gfs/RG6DtABnYPg059Tw==", 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlXZbELRAAKD0qOkhk7skfc4xcAC8Lhg==", 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:22:10.380 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.381 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.381 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2023-12-19T18:22:10.381 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlF07MARAA2TJJ+5C1HIU+SVhqmKaVHA==", 2023-12-19T18:22:10.381 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.381 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:22:10.381 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:22:10.381 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.381 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlISPGEhAAdFIRRkXlLyOx/oMZ4MgL9Q==", 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmNM9JRAARgS20O8zpm6H70WDv+lgGw==", 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:22:10.382 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlWoZ8AhAADWvKVzeDhDGHtYNUFIOWsA==", 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.383 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2023-12-19T18:22:10.384 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFllWFVMhAAD9ebDOppHTODVQjg6ppP3g==", 2023-12-19T18:22:10.384 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.384 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:22:10.384 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:22:10.384 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:22:10.384 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:22:10.384 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.384 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2023-12-19T18:22:10.384 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlwbIQDRAAt/h55zBNCEId0vkYSUQ3TQ==", 2023-12-19T18:22:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:22:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:22:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:22:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:22:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2023-12-19T18:22:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB53YFloCb6CxAAmZQyz5XOEqDOOiROJHLzxQ==", 2023-12-19T18:22:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.385 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFltBhxDxAAiAS9CYPY86IMmJ7NGgA2lw==", 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:10.386 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2023-12-19T18:22:10.387 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlUnAhBBAAi++W9XK3o3Y/185hB3LuIg==", 2023-12-19T18:22:10.387 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:22:10.387 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:22:10.387 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:22:10.387 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:22:10.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:10.387 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:22:10.387 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:22:10.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.376+0000 7f2df7fff640 1 -- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2dd806a430 msgr2=0x7f2dd806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:10.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.376+0000 7f2df7fff640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2dd806a430 0x7f2dd806c8f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f2df8150be0 tx=0x7f2de8009040 comp rx=0 tx=0).stop 2023-12-19T18:22:10.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.376+0000 7f2df7fff640 1 -- 172.21.15.27:0/1512612856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df815ac50 msgr2=0x7f2df8148de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:10.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.376+0000 7f2df7fff640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df815ac50 0x7f2df8148de0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009e50 tx=0x7f2df0004a40 comp rx=0 tx=0).stop 2023-12-19T18:22:10.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.376+0000 7f2df7fff640 1 -- 172.21.15.27:0/1512612856 shutdown_connections 2023-12-19T18:22:10.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.376+0000 7f2df7fff640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2dd806a430 0x7f2dd806c8f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.376+0000 7f2df7fff640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2df815ac50 0x7f2df8148de0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.376+0000 7f2df7fff640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2df8070fb0 0x7f2df8152730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.376+0000 7f2df7fff640 1 --2- 172.21.15.27:0/1512612856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2df806b850 0x7f2df8154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.376+0000 7f2df7fff640 1 -- 172.21.15.27:0/1512612856 >> 172.21.15.27:0/1512612856 conn(0x7f2df806a7d0 msgr2=0x7f2df80cdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:10.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.376+0000 7f2df7fff640 1 -- 172.21.15.27:0/1512612856 shutdown_connections 2023-12-19T18:22:10.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.377+0000 7f2df7fff640 1 -- 172.21.15.27:0/1512612856 wait complete. 2023-12-19T18:22:10.398 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:22:10.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.630+0000 7f66af36e640 1 Processor -- start 2023-12-19T18:22:10.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.630+0000 7f66af36e640 1 -- start start 2023-12-19T18:22:10.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.632+0000 7f66af36e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f66a81553a0 0x7f66a80c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:10.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.632+0000 7f66af36e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f66a80afeb0 0x7f66a815e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:10.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.632+0000 7f66af36e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66a815f090 con 0x7f66a815ec60 2023-12-19T18:22:10.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.632+0000 7f66af36e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a80cbcf0 con 0x7f66a80afeb0 2023-12-19T18:22:10.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.632+0000 7f66af36e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66a80b0360 con 0x7f66a81553a0 2023-12-19T18:22:10.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.632+0000 7f66adb6b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f66a80afeb0 0x7f66a815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:10.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.632+0000 7f66adb6b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f66a80afeb0 0x7f66a815e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37780/0 (socket says 172.21.15.27:37780) 2023-12-19T18:22:10.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.632+0000 7f66adb6b640 1 -- 172.21.15.27:0/3035044191 learned_addr learned my addr 172.21.15.27:0/3035044191 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:10.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.633+0000 7f66adb6b640 1 -- 172.21.15.27:0/3035044191 >> v2:172.21.15.192:3300/0 conn(0x7f66a81553a0 msgr2=0x7f66a80c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:10.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.633+0000 7f66adb6b640 1 --2- 172.21.15.27:0/3035044191 >> v2:172.21.15.192:3300/0 conn(0x7f66a81553a0 0x7f66a80c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.633+0000 7f66adb6b640 1 -- 172.21.15.27:0/3035044191 >> v1:172.21.15.27:6789/0 conn(0x7f66a815ec60 legacy=0x7f66a815b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:10.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.633+0000 7f66adb6b640 1 -- 172.21.15.27:0/3035044191 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a80cb350 con 0x7f66a80afeb0 2023-12-19T18:22:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.634+0000 7f66adb6b640 1 --2- 172.21.15.27:0/3035044191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66a80afeb0 0x7f66a815e720 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f66a80cc1c0 tx=0x7f669802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=897ef34ab533cb9d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.634+0000 7f66ad36a640 1 -- 172.21.15.27:0/3035044191 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698031870 con 0x7f66a80afeb0 2023-12-19T18:22:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.634+0000 7f66ad36a640 1 -- 172.21.15.27:0/3035044191 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66980319d0 con 0x7f66a80afeb0 2023-12-19T18:22:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.634+0000 7f66ad36a640 1 -- 172.21.15.27:0/3035044191 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698031d60 con 0x7f66a80afeb0 2023-12-19T18:22:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.635+0000 7f66af36e640 1 -- 172.21.15.27:0/3035044191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66a80afeb0 msgr2=0x7f66a815e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.635+0000 7f66af36e640 1 --2- 172.21.15.27:0/3035044191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66a80afeb0 0x7f66a815e720 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f66a80cc1c0 tx=0x7f669802b910 comp rx=0 tx=0).stop 2023-12-19T18:22:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.635+0000 7f66af36e640 1 -- 172.21.15.27:0/3035044191 shutdown_connections 2023-12-19T18:22:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.635+0000 7f66af36e640 1 --2- 172.21.15.27:0/3035044191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66a80afeb0 0x7f66a815e720 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.635+0000 7f66af36e640 1 --2- 172.21.15.27:0/3035044191 >> v2:172.21.15.192:3300/0 conn(0x7f66a81553a0 0x7f66a80c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.635+0000 7f66af36e640 1 -- 172.21.15.27:0/3035044191 >> 172.21.15.27:0/3035044191 conn(0x7f66a8069c70 msgr2=0x7f66a806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.635+0000 7f66af36e640 1 -- 172.21.15.27:0/3035044191 shutdown_connections 2023-12-19T18:22:10.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.635+0000 7f66af36e640 1 -- 172.21.15.27:0/3035044191 wait complete. 2023-12-19T18:22:10.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.637+0000 7f66af36e640 1 Processor -- start 2023-12-19T18:22:10.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.637+0000 7f66af36e640 1 -- start start 2023-12-19T18:22:10.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66af36e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66a80afeb0 0x7f66a8144290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:10.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66af36e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66a81553a0 0x7f66a81504a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:10.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66af36e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66a815ec60 0x7f66a81509e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:10.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66af36e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a815dc60 con 0x7f66a815ec60 2023-12-19T18:22:10.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66af36e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a80cddc0 con 0x7f66a81553a0 2023-12-19T18:22:10.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66af36e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a81447d0 con 0x7f66a80afeb0 2023-12-19T18:22:10.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66aeb6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66a815ec60 0x7f66a81509e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:10.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66aeb6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66a815ec60 0x7f66a81509e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37790/0 (socket says 172.21.15.27:37790) 2023-12-19T18:22:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66aeb6d640 1 -- 172.21.15.27:0/3708776869 learned_addr learned my addr 172.21.15.27:0/3708776869 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66adb6b640 1 --2- 172.21.15.27:0/3708776869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66a81553a0 0x7f66a81504a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66aeb6d640 1 -- 172.21.15.27:0/3708776869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66a80afeb0 msgr2=0x7f66a8144290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66aeb6d640 1 --2- 172.21.15.27:0/3708776869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66a80afeb0 0x7f66a8144290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66aeb6d640 1 -- 172.21.15.27:0/3708776869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66a81553a0 msgr2=0x7f66a81504a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66aeb6d640 1 --2- 172.21.15.27:0/3708776869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66a81553a0 0x7f66a81504a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.638+0000 7f66aeb6d640 1 -- 172.21.15.27:0/3708776869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66980094d0 con 0x7f66a815ec60 2023-12-19T18:22:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.639+0000 7f66aeb6d640 1 --2- 172.21.15.27:0/3708776869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66a815ec60 0x7f66a81509e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f66a000c730 tx=0x7f66a000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.639+0000 7f669effd640 1 -- 172.21.15.27:0/3708776869 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0003b90 con 0x7f66a815ec60 2023-12-19T18:22:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.639+0000 7f669effd640 1 -- 172.21.15.27:0/3708776869 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a0003cf0 con 0x7f66a815ec60 2023-12-19T18:22:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.639+0000 7f669effd640 1 -- 172.21.15.27:0/3708776869 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a0005310 con 0x7f66a815ec60 2023-12-19T18:22:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.639+0000 7f66af36e640 1 -- 172.21.15.27:0/3708776869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66a80cd1b0 con 0x7f66a815ec60 2023-12-19T18:22:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.639+0000 7f66af36e640 1 -- 172.21.15.27:0/3708776869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66a80cd770 con 0x7f66a815ec60 2023-12-19T18:22:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.640+0000 7f66af36e640 1 -- 172.21.15.27:0/3708776869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66a804e840 con 0x7f66a815ec60 2023-12-19T18:22:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.641+0000 7f669effd640 1 -- 172.21.15.27:0/3708776869 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f66a0005680 con 0x7f66a815ec60 2023-12-19T18:22:10.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.642+0000 7f669effd640 1 --2- 172.21.15.27:0/3708776869 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f668c06a430 0x7f668c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:10.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.642+0000 7f669effd640 1 -- 172.21.15.27:0/3708776869 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f66a0088670 con 0x7f66a815ec60 2023-12-19T18:22:10.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.643+0000 7f66ae36c640 1 --2- 172.21.15.27:0/3708776869 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f668c06a430 0x7f668c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:10.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.644+0000 7f66ae36c640 1 --2- 172.21.15.27:0/3708776869 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f668c06a430 0x7f668c06c8f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009710 tx=0x7f66a4006c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.646+0000 7f669effd640 1 -- 172.21.15.27:0/3708776869 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f66a0053f10 con 0x7f66a815ec60 2023-12-19T18:22:10.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.846+0000 7f66af36e640 1 -- 172.21.15.27:0/3708776869 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66a80af6a0 con 0x7f66a815ec60 2023-12-19T18:22:10.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.848+0000 7f669effd640 1 -- 172.21.15.27:0/3708776869 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) v1 ==== 74+0+12400 (secure 0 0 0) 0x7f66a0003e60 con 0x7f66a815ec60 2023-12-19T18:22:10.849 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:10.849 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":131,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:22:09.818262+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":115,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:22:10.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.851+0000 7f66af36e640 1 -- 172.21.15.27:0/3708776869 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f668c06a430 msgr2=0x7f668c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:10.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.851+0000 7f66af36e640 1 --2- 172.21.15.27:0/3708776869 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f668c06a430 0x7f668c06c8f0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f66a4009710 tx=0x7f66a4006c90 comp rx=0 tx=0).stop 2023-12-19T18:22:10.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.851+0000 7f66af36e640 1 -- 172.21.15.27:0/3708776869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66a815ec60 msgr2=0x7f66a81509e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:10.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.851+0000 7f66af36e640 1 --2- 172.21.15.27:0/3708776869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66a815ec60 0x7f66a81509e0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f66a000c730 tx=0x7f66a000cbf0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.851+0000 7f66af36e640 1 -- 172.21.15.27:0/3708776869 shutdown_connections 2023-12-19T18:22:10.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.851+0000 7f66af36e640 1 --2- 172.21.15.27:0/3708776869 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f668c06a430 0x7f668c06c8f0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.851+0000 7f66af36e640 1 --2- 172.21.15.27:0/3708776869 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66a815ec60 0x7f66a81509e0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.851+0000 7f66af36e640 1 --2- 172.21.15.27:0/3708776869 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66a81553a0 0x7f66a81504a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.851+0000 7f66af36e640 1 --2- 172.21.15.27:0/3708776869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66a80afeb0 0x7f66a8144290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:10.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.851+0000 7f66af36e640 1 -- 172.21.15.27:0/3708776869 >> 172.21.15.27:0/3708776869 conn(0x7f66a8069c70 msgr2=0x7f66a815afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:10.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.852+0000 7f66af36e640 1 -- 172.21.15.27:0/3708776869 shutdown_connections 2023-12-19T18:22:10.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:10.852+0000 7f66af36e640 1 -- 172.21.15.27:0/3708776869 wait complete. 2023-12-19T18:22:10.871 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:22:10.871 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:22:11.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.106+0000 7f34f0e22640 1 Processor -- start 2023-12-19T18:22:11.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.106+0000 7f34f0e22640 1 -- start start 2023-12-19T18:22:11.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.108+0000 7f34f0e22640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34ec06c850 0x7f34ec164860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.108+0000 7f34f0e22640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ec0b27d0 con 0x7f34ec143450 2023-12-19T18:22:11.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.108+0000 7f34f0e22640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34ec0cce00 con 0x7f34ec0cf600 2023-12-19T18:22:11.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.108+0000 7f34f0e22640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0b33e0 con 0x7f34ec06c850 2023-12-19T18:22:11.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.108+0000 7f34eb7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f34ec143450 0x7f34ec0ceef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34872/0 (socket says 172.21.15.27:34872) 2023-12-19T18:22:11.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.108+0000 7f34eb7fe640 1 -- 172.21.15.27:0/4269174005 learned_addr learned my addr 172.21.15.27:0/4269174005 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.108+0000 7f34ebfff640 1 --2- 172.21.15.27:0/4269174005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34ec06c850 0x7f34ec164860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.108+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3561117839 0 0) 0x7f34ec0b27d0 con 0x7f34ec143450 2023-12-19T18:22:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34d4003520 con 0x7f34ec143450 2023-12-19T18:22:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2229187517 0 0) 0x7f34d4003520 con 0x7f34ec143450 2023-12-19T18:22:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34ec06c850 msgr2=0x7f34ec164860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 --2- 172.21.15.27:0/4269174005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34ec06c850 0x7f34ec164860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 >> v1:172.21.15.192:6789/0 conn(0x7f34ec0cf600 legacy=0x7f34ec06c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0b2a40 con 0x7f34ec143450 2023-12-19T18:22:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f34dc001670 con 0x7f34ec143450 2023-12-19T18:22:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 >> v1:172.21.15.27:6789/0 conn(0x7f34ec143450 legacy=0x7f34ec0ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 --2- 172.21.15.27:0/4269174005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34d40039a0 0x7f34d4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 --2- 172.21.15.27:0/4269174005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d40047c0 0x7f34d4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 --2- 172.21.15.27:0/4269174005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34ec06c850 0x7f34d400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0b33e0 con 0x7f34d40047c0 2023-12-19T18:22:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40054b0 con 0x7f34d40039a0 2023-12-19T18:22:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.109+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d4005670 con 0x7f34ec06c850 2023-12-19T18:22:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.110+0000 7f34eaffd640 1 --2- 172.21.15.27:0/4269174005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d40047c0 0x7f34d4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.110+0000 7f34eb7fe640 1 --2- 172.21.15.27:0/4269174005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34d40039a0 0x7f34d4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.110+0000 7f34ebfff640 1 --2- 172.21.15.27:0/4269174005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34ec06c850 0x7f34d400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.110+0000 7f34eaffd640 1 -- 172.21.15.27:0/4269174005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34ec06c850 msgr2=0x7f34d400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.110+0000 7f34eaffd640 1 --2- 172.21.15.27:0/4269174005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34ec06c850 0x7f34d400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.110+0000 7f34eaffd640 1 -- 172.21.15.27:0/4269174005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34d40039a0 msgr2=0x7f34d4003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.110+0000 7f34eaffd640 1 --2- 172.21.15.27:0/4269174005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34d40039a0 0x7f34d4003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.110+0000 7f34eaffd640 1 -- 172.21.15.27:0/4269174005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d4003520 con 0x7f34d40047c0 2023-12-19T18:22:11.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.111+0000 7f34eaffd640 1 --2- 172.21.15.27:0/4269174005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d40047c0 0x7f34d4004c40 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f34e0001e50 tx=0x7f34d40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62235f1bf7e67f01 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:11.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.111+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e0003b70 con 0x7f34d40047c0 2023-12-19T18:22:11.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.112+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34e0003cd0 con 0x7f34d40047c0 2023-12-19T18:22:11.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.112+0000 7f34ea7fc640 1 -- 172.21.15.27:0/4269174005 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34e0005310 con 0x7f34d40047c0 2023-12-19T18:22:11.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.112+0000 7f34f0e22640 1 -- 172.21.15.27:0/4269174005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d40047c0 msgr2=0x7f34d4004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:11.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.112+0000 7f34f0e22640 1 --2- 172.21.15.27:0/4269174005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d40047c0 0x7f34d4004c40 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f34e0001e50 tx=0x7f34d40057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.112+0000 7f34ebfff640 1 -- 172.21.15.27:0/4269174005 reap_dead start 2023-12-19T18:22:11.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.112+0000 7f34f0e22640 1 -- 172.21.15.27:0/4269174005 shutdown_connections 2023-12-19T18:22:11.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.112+0000 7f34f0e22640 1 -- 172.21.15.27:0/4269174005 >> 172.21.15.27:0/4269174005 conn(0x7f34ec06a060 msgr2=0x7f34ec06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:11.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.112+0000 7f34f0e22640 1 -- 172.21.15.27:0/4269174005 shutdown_connections 2023-12-19T18:22:11.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.112+0000 7f34f0e22640 1 -- 172.21.15.27:0/4269174005 wait complete. 2023-12-19T18:22:11.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.114+0000 7f34f0e22640 1 Processor -- start 2023-12-19T18:22:11.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.115+0000 7f34f0e22640 1 -- start start 2023-12-19T18:22:11.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.116+0000 7f34f0e22640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d40047c0 0x7f34ec13a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.116+0000 7f34f0e22640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34ec0ab4a0 0x7f34ec0a9840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.116+0000 7f34f0e22640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34ec0a9d80 0x7f34ec0afb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.116+0000 7f34f0e22640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0aa200 con 0x7f34d40047c0 2023-12-19T18:22:11.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.116+0000 7f34f0e22640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0ac2a0 con 0x7f34ec0a9d80 2023-12-19T18:22:11.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.116+0000 7f34f0e22640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34ec0aa400 con 0x7f34ec0ab4a0 2023-12-19T18:22:11.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.116+0000 7f34eb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d40047c0 0x7f34ec13a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:11.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.116+0000 7f34eb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d40047c0 0x7f34ec13a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37808/0 (socket says 172.21.15.27:37808) 2023-12-19T18:22:11.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.116+0000 7f34eb7fe640 1 -- 172.21.15.27:0/3194213409 learned_addr learned my addr 172.21.15.27:0/3194213409 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:11.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.116+0000 7f34eaffd640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34ec0ab4a0 0x7f34ec0a9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:11.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.116+0000 7f34ebfff640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34ec0a9d80 0x7f34ec0afb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:11.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.117+0000 7f34eb7fe640 1 -- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34ec0ab4a0 msgr2=0x7f34ec0a9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:11.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.117+0000 7f34eb7fe640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34ec0ab4a0 0x7f34ec0a9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.117+0000 7f34eb7fe640 1 -- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34ec0a9d80 msgr2=0x7f34ec0afb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:11.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.117+0000 7f34eb7fe640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34ec0a9d80 0x7f34ec0afb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.117+0000 7f34eb7fe640 1 -- 172.21.15.27:0/3194213409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ec0aa6e0 con 0x7f34d40047c0 2023-12-19T18:22:11.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.117+0000 7f34ebfff640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34ec0a9d80 0x7f34ec0afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:11.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.117+0000 7f34eb7fe640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d40047c0 0x7f34ec13a590 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f34dc001050 tx=0x7f34dc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:11.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.117+0000 7f34d3fff640 1 -- 172.21.15.27:0/3194213409 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00eed0 con 0x7f34d40047c0 2023-12-19T18:22:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.117+0000 7f34d3fff640 1 -- 172.21.15.27:0/3194213409 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34dc00c870 con 0x7f34d40047c0 2023-12-19T18:22:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.117+0000 7f34d3fff640 1 -- 172.21.15.27:0/3194213409 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34dc00cc20 con 0x7f34d40047c0 2023-12-19T18:22:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.117+0000 7f34f0e22640 1 -- 172.21.15.27:0/3194213409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34ec0aa9c0 con 0x7f34d40047c0 2023-12-19T18:22:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.118+0000 7f34f0e22640 1 -- 172.21.15.27:0/3194213409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34ec0aaf80 con 0x7f34d40047c0 2023-12-19T18:22:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.119+0000 7f34f0e22640 1 -- 172.21.15.27:0/3194213409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34ec04e840 con 0x7f34d40047c0 2023-12-19T18:22:11.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.120+0000 7f34d3fff640 1 -- 172.21.15.27:0/3194213409 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f34dc01d460 con 0x7f34d40047c0 2023-12-19T18:22:11.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.121+0000 7f34d3fff640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f34d806a430 0x7f34d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.121+0000 7f34d3fff640 1 -- 172.21.15.27:0/3194213409 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f34dc088660 con 0x7f34d40047c0 2023-12-19T18:22:11.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.122+0000 7f34eaffd640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f34d806a430 0x7f34d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:11.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.123+0000 7f34eaffd640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f34d806a430 0x7f34d806c8f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f34e0034000 tx=0x7f34e0009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:11.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.126+0000 7f34d3fff640 1 -- 172.21.15.27:0/3194213409 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f34dc053f00 con 0x7f34d40047c0 2023-12-19T18:22:11.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.321+0000 7f34f0e22640 1 -- 172.21.15.27:0/3194213409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f34ec0b2cf0 con 0x7f34d40047c0 2023-12-19T18:22:11.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.322+0000 7f34d3fff640 1 -- 172.21.15.27:0/3194213409 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v131) v1 ==== 93+0+10 (secure 0 0 0) 0x7f34dc057520 con 0x7f34d40047c0 2023-12-19T18:22:11.322 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:22:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.324+0000 7f34f0e22640 1 -- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f34d806a430 msgr2=0x7f34d806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.324+0000 7f34f0e22640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f34d806a430 0x7f34d806c8f0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f34e0034000 tx=0x7f34e0009030 comp rx=0 tx=0).stop 2023-12-19T18:22:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.325+0000 7f34f0e22640 1 -- 172.21.15.27:0/3194213409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d40047c0 msgr2=0x7f34ec13a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.325+0000 7f34f0e22640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d40047c0 0x7f34ec13a590 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f34dc001050 tx=0x7f34dc00ec80 comp rx=0 tx=0).stop 2023-12-19T18:22:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.325+0000 7f34f0e22640 1 -- 172.21.15.27:0/3194213409 shutdown_connections 2023-12-19T18:22:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.325+0000 7f34f0e22640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f34d806a430 0x7f34d806c8f0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.325+0000 7f34f0e22640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34ec0a9d80 0x7f34ec0afb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.325+0000 7f34f0e22640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34ec0ab4a0 0x7f34ec0a9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.325+0000 7f34f0e22640 1 --2- 172.21.15.27:0/3194213409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d40047c0 0x7f34ec13a590 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.325+0000 7f34f0e22640 1 -- 172.21.15.27:0/3194213409 >> 172.21.15.27:0/3194213409 conn(0x7f34ec06a060 msgr2=0x7f34ec06c140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.325+0000 7f34f0e22640 1 -- 172.21.15.27:0/3194213409 shutdown_connections 2023-12-19T18:22:11.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.325+0000 7f34f0e22640 1 -- 172.21.15.27:0/3194213409 wait complete. 2023-12-19T18:22:11.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:22:11.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.573+0000 7f614359e640 1 Processor -- start 2023-12-19T18:22:11.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.574+0000 7f614359e640 1 -- start start 2023-12-19T18:22:11.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.575+0000 7f614359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f61440b89b0 0x7f614415c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.575+0000 7f614359e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f61440b4db0 0x7f614415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.575+0000 7f614359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61440b5ca0 con 0x7f61440b49d0 2023-12-19T18:22:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.575+0000 7f614359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614415d300 con 0x7f61440b89b0 2023-12-19T18:22:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.575+0000 7f614359e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6144161050 con 0x7f61440b4db0 2023-12-19T18:22:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.576+0000 7f614259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f61440b49d0 0x7f6144160940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34882/0 (socket says 172.21.15.27:34882) 2023-12-19T18:22:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.576+0000 7f614259c640 1 -- 172.21.15.27:0/1485721290 learned_addr learned my addr 172.21.15.27:0/1485721290 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.576+0000 7f6141d9b640 1 --2- 172.21.15.27:0/1485721290 >> v2:172.21.15.27:3300/0 conn(0x7f61440b89b0 0x7f614415c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.576+0000 7f6142d9d640 1 --2- 172.21.15.27:0/1485721290 >> v2:172.21.15.192:3300/0 conn(0x7f61440b4db0 0x7f614415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.576+0000 7f6141d9b640 1 -- 172.21.15.27:0/1485721290 >> v2:172.21.15.192:3300/0 conn(0x7f61440b4db0 msgr2=0x7f614415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.576+0000 7f6141d9b640 1 --2- 172.21.15.27:0/1485721290 >> v2:172.21.15.192:3300/0 conn(0x7f61440b4db0 0x7f614415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.576+0000 7f6141d9b640 1 -- 172.21.15.27:0/1485721290 >> v1:172.21.15.27:6789/0 conn(0x7f61440b49d0 legacy=0x7f6144160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.576+0000 7f6141d9b640 1 -- 172.21.15.27:0/1485721290 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6144161800 con 0x7f61440b89b0 2023-12-19T18:22:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.577+0000 7f6141d9b640 1 --2- 172.21.15.27:0/1485721290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61440b89b0 0x7f614415c880 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f61440d03f0 tx=0x7f613802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=becd3228a7fe4bc2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:11.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.578+0000 7f614159a640 1 -- 172.21.15.27:0/1485721290 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138030610 con 0x7f61440b89b0 2023-12-19T18:22:11.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.578+0000 7f614159a640 1 -- 172.21.15.27:0/1485721290 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6138030770 con 0x7f61440b89b0 2023-12-19T18:22:11.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.578+0000 7f614159a640 1 -- 172.21.15.27:0/1485721290 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6138030b00 con 0x7f61440b89b0 2023-12-19T18:22:11.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.578+0000 7f614359e640 1 -- 172.21.15.27:0/1485721290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61440b89b0 msgr2=0x7f614415c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:11.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.578+0000 7f614359e640 1 --2- 172.21.15.27:0/1485721290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61440b89b0 0x7f614415c880 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f61440d03f0 tx=0x7f613802b960 comp rx=0 tx=0).stop 2023-12-19T18:22:11.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.578+0000 7f614359e640 1 -- 172.21.15.27:0/1485721290 shutdown_connections 2023-12-19T18:22:11.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.579+0000 7f614359e640 1 --2- 172.21.15.27:0/1485721290 >> v2:172.21.15.192:3300/0 conn(0x7f61440b4db0 0x7f614415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.579+0000 7f614359e640 1 --2- 172.21.15.27:0/1485721290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61440b89b0 0x7f614415c880 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.579+0000 7f614359e640 1 -- 172.21.15.27:0/1485721290 >> 172.21.15.27:0/1485721290 conn(0x7f61440cf760 msgr2=0x7f61440cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:11.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.579+0000 7f614359e640 1 -- 172.21.15.27:0/1485721290 shutdown_connections 2023-12-19T18:22:11.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.579+0000 7f614359e640 1 -- 172.21.15.27:0/1485721290 wait complete. 2023-12-19T18:22:11.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.580+0000 7f614359e640 1 Processor -- start 2023-12-19T18:22:11.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.581+0000 7f614359e640 1 -- start start 2023-12-19T18:22:11.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.581+0000 7f614359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61440b49d0 0x7f614414b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.581+0000 7f614359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61440b4db0 0x7f61440c9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.581+0000 7f614359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61440b89b0 0x7f61440ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.581+0000 7f614359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61440cbbe0 con 0x7f61440b49d0 2023-12-19T18:22:11.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.582+0000 7f614359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614415bd10 con 0x7f61440b4db0 2023-12-19T18:22:11.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.582+0000 7f614359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614415bfb0 con 0x7f61440b89b0 2023-12-19T18:22:11.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.582+0000 7f614259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61440b49d0 0x7f614414b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:11.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.582+0000 7f614259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61440b49d0 0x7f614414b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37830/0 (socket says 172.21.15.27:37830) 2023-12-19T18:22:11.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.582+0000 7f614259c640 1 -- 172.21.15.27:0/1526456838 learned_addr learned my addr 172.21.15.27:0/1526456838 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:11.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.582+0000 7f614259c640 1 -- 172.21.15.27:0/1526456838 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61440b89b0 msgr2=0x7f61440ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:11.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.582+0000 7f614259c640 1 --2- 172.21.15.27:0/1526456838 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61440b89b0 0x7f61440ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.582+0000 7f614259c640 1 -- 172.21.15.27:0/1526456838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61440b4db0 msgr2=0x7f61440c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:11.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.582+0000 7f614259c640 1 --2- 172.21.15.27:0/1526456838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61440b4db0 0x7f61440c9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.582+0000 7f614259c640 1 -- 172.21.15.27:0/1526456838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6138009400 con 0x7f61440b49d0 2023-12-19T18:22:11.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.582+0000 7f614259c640 1 --2- 172.21.15.27:0/1526456838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61440b49d0 0x7f614414b420 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f613400c480 tx=0x7f613400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:11.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.583+0000 7f6132ffd640 1 -- 172.21.15.27:0/1526456838 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6134007480 con 0x7f61440b49d0 2023-12-19T18:22:11.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.583+0000 7f6132ffd640 1 -- 172.21.15.27:0/1526456838 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61340075e0 con 0x7f61440b49d0 2023-12-19T18:22:11.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.583+0000 7f6132ffd640 1 -- 172.21.15.27:0/1526456838 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61340059a0 con 0x7f61440b49d0 2023-12-19T18:22:11.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.583+0000 7f614359e640 1 -- 172.21.15.27:0/1526456838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61440cb870 con 0x7f61440b49d0 2023-12-19T18:22:11.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.583+0000 7f614359e640 1 -- 172.21.15.27:0/1526456838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614414b960 con 0x7f61440b49d0 2023-12-19T18:22:11.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.584+0000 7f614359e640 1 -- 172.21.15.27:0/1526456838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f614404e840 con 0x7f61440b49d0 2023-12-19T18:22:11.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.585+0000 7f6132ffd640 1 -- 172.21.15.27:0/1526456838 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6134005b00 con 0x7f61440b49d0 2023-12-19T18:22:11.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.587+0000 7f6132ffd640 1 --2- 172.21.15.27:0/1526456838 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f612406a430 0x7f612406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:11.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.587+0000 7f6132ffd640 1 -- 172.21.15.27:0/1526456838 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6134087b90 con 0x7f61440b49d0 2023-12-19T18:22:11.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.587+0000 7f6141d9b640 1 --2- 172.21.15.27:0/1526456838 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f612406a430 0x7f612406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:11.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.589+0000 7f6141d9b640 1 --2- 172.21.15.27:0/1526456838 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f612406a430 0x7f612406c8f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6138036000 tx=0x7f6138030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:11.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.592+0000 7f6132ffd640 1 -- 172.21.15.27:0/1526456838 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6134053560 con 0x7f61440b49d0 2023-12-19T18:22:11.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.790+0000 7f614359e640 1 -- 172.21.15.27:0/1526456838 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f61440d08b0 con 0x7f61440b49d0 2023-12-19T18:22:11.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.790+0000 7f6132ffd640 1 -- 172.21.15.27:0/1526456838 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v131) v1 ==== 84+0+65 (secure 0 0 0) 0x7f6134017020 con 0x7f61440b49d0 2023-12-19T18:22:11.790 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:11.790 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-12-19T18:22:11.791 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:22:11.791 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-12-19T18:22:11.791 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-12-19T18:22:11.791 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:11.791 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-12-19T18:22:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.793+0000 7f614359e640 1 -- 172.21.15.27:0/1526456838 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f612406a430 msgr2=0x7f612406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.793+0000 7f614359e640 1 --2- 172.21.15.27:0/1526456838 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f612406a430 0x7f612406c8f0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6138036000 tx=0x7f6138030000 comp rx=0 tx=0).stop 2023-12-19T18:22:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.793+0000 7f614359e640 1 -- 172.21.15.27:0/1526456838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61440b49d0 msgr2=0x7f614414b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.793+0000 7f614359e640 1 --2- 172.21.15.27:0/1526456838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61440b49d0 0x7f614414b420 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f613400c480 tx=0x7f613400c940 comp rx=0 tx=0).stop 2023-12-19T18:22:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.793+0000 7f614359e640 1 -- 172.21.15.27:0/1526456838 shutdown_connections 2023-12-19T18:22:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.793+0000 7f614359e640 1 --2- 172.21.15.27:0/1526456838 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f612406a430 0x7f612406c8f0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.793+0000 7f614359e640 1 --2- 172.21.15.27:0/1526456838 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f61440b89b0 0x7f61440ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.793+0000 7f614359e640 1 --2- 172.21.15.27:0/1526456838 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f61440b4db0 0x7f61440c9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.793+0000 7f614359e640 1 --2- 172.21.15.27:0/1526456838 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f61440b49d0 0x7f614414b420 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.793+0000 7f614359e640 1 -- 172.21.15.27:0/1526456838 >> 172.21.15.27:0/1526456838 conn(0x7f61440cf760 msgr2=0x7f61440d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.793+0000 7f614359e640 1 -- 172.21.15.27:0/1526456838 shutdown_connections 2023-12-19T18:22:11.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:11.793+0000 7f614359e640 1 -- 172.21.15.27:0/1526456838 wait complete. 2023-12-19T18:22:11.811 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:22:11.812 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:22:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.050+0000 7f3ea1a99640 1 Processor -- start 2023-12-19T18:22:12.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.050+0000 7f3ea1a99640 1 -- start start 2023-12-19T18:22:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.052+0000 7f3ea1a99640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e9c0ade20 0x7f3e9c15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.052+0000 7f3ea1a99640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3e9c15b810 0x7f3e9c0c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.052+0000 7f3ea1a99640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e9c15c030 con 0x7f3e9c158790 2023-12-19T18:22:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.052+0000 7f3ea1a99640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9c0c99d0 con 0x7f3e9c15b810 2023-12-19T18:22:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.052+0000 7f3ea1a99640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c158b70 con 0x7f3e9c0ade20 2023-12-19T18:22:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.052+0000 7f3ea1298640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3e9c15b810 0x7f3e9c0c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.052+0000 7f3ea1298640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3e9c15b810 0x7f3e9c0c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42780/0 (socket says 172.21.15.27:42780) 2023-12-19T18:22:12.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.052+0000 7f3ea1298640 1 -- 172.21.15.27:0/667528313 learned_addr learned my addr 172.21.15.27:0/667528313 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.052+0000 7f3e9bfff640 1 --2- 172.21.15.27:0/667528313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e9c0ade20 0x7f3e9c15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.053+0000 7f3ea1298640 1 -- 172.21.15.27:0/667528313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e9c0ade20 msgr2=0x7f3e9c15b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.053+0000 7f3ea1298640 1 --2- 172.21.15.27:0/667528313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e9c0ade20 0x7f3e9c15b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.053+0000 7f3ea1298640 1 -- 172.21.15.27:0/667528313 >> v1:172.21.15.192:6789/0 conn(0x7f3e9c158790 legacy=0x7f3e9c0ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:12.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.053+0000 7f3ea1298640 1 -- 172.21.15.27:0/667528313 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c0c9030 con 0x7f3e9c15b810 2023-12-19T18:22:12.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.054+0000 7f3ea1298640 1 --2- 172.21.15.27:0/667528313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9c15b810 0x7f3e9c0c3820 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009e50 tx=0x7f3e9402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29ccfb83fa1e72d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:12.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.054+0000 7f3e9b7fe640 1 -- 172.21.15.27:0/667528313 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9402cb00 con 0x7f3e9c15b810 2023-12-19T18:22:12.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.054+0000 7f3e9b7fe640 1 -- 172.21.15.27:0/667528313 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9402cc60 con 0x7f3e9c15b810 2023-12-19T18:22:12.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.054+0000 7f3e9b7fe640 1 -- 172.21.15.27:0/667528313 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e940343f0 con 0x7f3e9c15b810 2023-12-19T18:22:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.055+0000 7f3ea1a99640 1 -- 172.21.15.27:0/667528313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9c15b810 msgr2=0x7f3e9c0c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.055+0000 7f3ea1a99640 1 --2- 172.21.15.27:0/667528313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9c15b810 0x7f3e9c0c3820 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f3e94009e50 tx=0x7f3e9402bf60 comp rx=0 tx=0).stop 2023-12-19T18:22:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.055+0000 7f3ea1a99640 1 -- 172.21.15.27:0/667528313 shutdown_connections 2023-12-19T18:22:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.055+0000 7f3ea1a99640 1 --2- 172.21.15.27:0/667528313 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9c15b810 0x7f3e9c0c3820 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.055+0000 7f3ea1a99640 1 --2- 172.21.15.27:0/667528313 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e9c0ade20 0x7f3e9c15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.055+0000 7f3ea1a99640 1 -- 172.21.15.27:0/667528313 >> 172.21.15.27:0/667528313 conn(0x7f3e9c0b05b0 msgr2=0x7f3e9c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.055+0000 7f3ea1a99640 1 -- 172.21.15.27:0/667528313 shutdown_connections 2023-12-19T18:22:12.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.055+0000 7f3ea1a99640 1 -- 172.21.15.27:0/667528313 wait complete. 2023-12-19T18:22:12.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.057+0000 7f3ea1a99640 1 Processor -- start 2023-12-19T18:22:12.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.057+0000 7f3ea1a99640 1 -- start start 2023-12-19T18:22:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.058+0000 7f3ea1a99640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e9c0ade20 0x7f3e9c0cd750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.058+0000 7f3ea1a99640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9c158790 0x7f3e9c155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.058+0000 7f3ea1a99640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e9c15b810 0x7f3e9c1596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.058+0000 7f3ea1a99640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c156b40 con 0x7f3e9c158790 2023-12-19T18:22:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.058+0000 7f3ea1a99640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c0cc6b0 con 0x7f3e9c15b810 2023-12-19T18:22:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.058+0000 7f3ea1a99640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9c0cc990 con 0x7f3e9c0ade20 2023-12-19T18:22:12.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.058+0000 7f3e9bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9c158790 0x7f3e9c155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.059+0000 7f3e9bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9c158790 0x7f3e9c155c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42782/0 (socket says 172.21.15.27:42782) 2023-12-19T18:22:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.059+0000 7f3e9bfff640 1 -- 172.21.15.27:0/1041615593 learned_addr learned my addr 172.21.15.27:0/1041615593 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.059+0000 7f3ea0a97640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e9c0ade20 0x7f3e9c0cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.059+0000 7f3ea1298640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e9c15b810 0x7f3e9c1596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.059+0000 7f3e9bfff640 1 -- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e9c0ade20 msgr2=0x7f3e9c0cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.059+0000 7f3e9bfff640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e9c0ade20 0x7f3e9c0cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.059+0000 7f3e9bfff640 1 -- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e9c15b810 msgr2=0x7f3e9c1596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.059+0000 7f3e9bfff640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e9c15b810 0x7f3e9c1596c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.059+0000 7f3e9bfff640 1 -- 172.21.15.27:0/1041615593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e94009b30 con 0x7f3e9c158790 2023-12-19T18:22:12.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.059+0000 7f3e9bfff640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9c158790 0x7f3e9c155c90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800d800 tx=0x7f3e8800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:12.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.060+0000 7f3e997fa640 1 -- 172.21.15.27:0/1041615593 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8800b6c0 con 0x7f3e9c158790 2023-12-19T18:22:12.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.060+0000 7f3e997fa640 1 -- 172.21.15.27:0/1041615593 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8800b820 con 0x7f3e9c158790 2023-12-19T18:22:12.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.060+0000 7f3e997fa640 1 -- 172.21.15.27:0/1041615593 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8800bbb0 con 0x7f3e9c158790 2023-12-19T18:22:12.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.060+0000 7f3ea1a99640 1 -- 172.21.15.27:0/1041615593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e9c0ccc70 con 0x7f3e9c158790 2023-12-19T18:22:12.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.060+0000 7f3ea1a99640 1 -- 172.21.15.27:0/1041615593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e9c0cd230 con 0x7f3e9c158790 2023-12-19T18:22:12.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.061+0000 7f3ea1a99640 1 -- 172.21.15.27:0/1041615593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e9c0cd3c0 con 0x7f3e9c158790 2023-12-19T18:22:12.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.062+0000 7f3e997fa640 1 -- 172.21.15.27:0/1041615593 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e8800bd10 con 0x7f3e9c158790 2023-12-19T18:22:12.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.063+0000 7f3e997fa640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e8006a430 0x7f3e8006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:12.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.063+0000 7f3e997fa640 1 -- 172.21.15.27:0/1041615593 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3e88088a60 con 0x7f3e9c158790 2023-12-19T18:22:12.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.064+0000 7f3ea0a97640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e8006a430 0x7f3e8006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:12.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.065+0000 7f3ea0a97640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e8006a430 0x7f3e8006c8f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3e90000f60 tx=0x7f3e90009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:12.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.067+0000 7f3e997fa640 1 -- 172.21.15.27:0/1041615593 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3e88054430 con 0x7f3e9c158790 2023-12-19T18:22:12.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.260+0000 7f3ea1a99640 1 -- 172.21.15.27:0/1041615593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3e9c04e840 con 0x7f3e9c158790 2023-12-19T18:22:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.698+0000 7f3e997fa640 1 -- 172.21.15.27:0/1041615593 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v132) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3e88057a50 con 0x7f3e9c158790 2023-12-19T18:22:12.698 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-12-19T18:22:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.703+0000 7f3ea1a99640 1 -- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e8006a430 msgr2=0x7f3e8006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.703+0000 7f3ea1a99640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e8006a430 0x7f3e8006c8f0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f3e90000f60 tx=0x7f3e90009040 comp rx=0 tx=0).stop 2023-12-19T18:22:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.703+0000 7f3ea1a99640 1 -- 172.21.15.27:0/1041615593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9c158790 msgr2=0x7f3e9c155c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:12.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.703+0000 7f3ea1a99640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9c158790 0x7f3e9c155c90 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f3e8800d800 tx=0x7f3e8800dcc0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.703+0000 7f3ea1a99640 1 -- 172.21.15.27:0/1041615593 shutdown_connections 2023-12-19T18:22:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.703+0000 7f3ea1a99640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e8006a430 0x7f3e8006c8f0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.703+0000 7f3ea1a99640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e9c15b810 0x7f3e9c1596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.703+0000 7f3ea1a99640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9c158790 0x7f3e9c155c90 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.703+0000 7f3ea1a99640 1 --2- 172.21.15.27:0/1041615593 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e9c0ade20 0x7f3e9c0cd750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.704+0000 7f3ea1a99640 1 -- 172.21.15.27:0/1041615593 >> 172.21.15.27:0/1041615593 conn(0x7f3e9c0b05b0 msgr2=0x7f3e9c0c9df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:12.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.704+0000 7f3ea1a99640 1 -- 172.21.15.27:0/1041615593 shutdown_connections 2023-12-19T18:22:12.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.705+0000 7f3ea1a99640 1 -- 172.21.15.27:0/1041615593 wait complete. 2023-12-19T18:22:12.729 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:22:12.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.966+0000 7f0775e65640 1 Processor -- start 2023-12-19T18:22:12.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.966+0000 7f0775e65640 1 -- start start 2023-12-19T18:22:12.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.968+0000 7f0775e65640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f077006f2e0 0x7f07700cab80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:12.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.968+0000 7f0775e65640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f07700cb0c0 0x7f07700ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:12.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.968+0000 7f0775e65640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f077015afe0 con 0x7f077015ac00 2023-12-19T18:22:12.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.968+0000 7f0775e65640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07700c8f90 con 0x7f07700cb0c0 2023-12-19T18:22:12.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.968+0000 7f0775e65640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07700c4880 con 0x7f077006f2e0 2023-12-19T18:22:12.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.969+0000 7f0775664640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f07700cb0c0 0x7f07700ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:12.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.969+0000 7f0774e63640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f077015ac00 0x7f07700c3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41210/0 (socket says 172.21.15.27:41210) 2023-12-19T18:22:12.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.969+0000 7f0774e63640 1 -- 172.21.15.27:0/1423856500 learned_addr learned my addr 172.21.15.27:0/1423856500 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:12.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.969+0000 7f076ffff640 1 --2- 172.21.15.27:0/1423856500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f077006f2e0 0x7f07700cab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:12.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.969+0000 7f076f7fe640 1 -- 172.21.15.27:0/1423856500 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292167789 0 0) 0x7f077015afe0 con 0x7f077015ac00 2023-12-19T18:22:12.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.970+0000 7f076f7fe640 1 -- 172.21.15.27:0/1423856500 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0758003520 con 0x7f077015ac00 2023-12-19T18:22:12.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.970+0000 7f0775664640 1 -- 172.21.15.27:0/1423856500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f077006f2e0 msgr2=0x7f07700cab80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:12.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.970+0000 7f0775664640 1 --2- 172.21.15.27:0/1423856500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f077006f2e0 0x7f07700cab80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.970+0000 7f0775664640 1 -- 172.21.15.27:0/1423856500 >> v1:172.21.15.27:6789/0 conn(0x7f077015ac00 legacy=0x7f07700c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:12.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.970+0000 7f0775664640 1 -- 172.21.15.27:0/1423856500 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07700c9200 con 0x7f07700cb0c0 2023-12-19T18:22:12.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.971+0000 7f0775664640 1 --2- 172.21.15.27:0/1423856500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700cb0c0 0x7f07700ad710 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f0768009d90 tx=0x7f076802bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a1be443b1e0df38 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:12.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.971+0000 7f076f7fe640 1 -- 172.21.15.27:0/1423856500 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076802cc50 con 0x7f07700cb0c0 2023-12-19T18:22:12.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.971+0000 7f076f7fe640 1 -- 172.21.15.27:0/1423856500 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076802cdb0 con 0x7f07700cb0c0 2023-12-19T18:22:12.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.971+0000 7f076f7fe640 1 -- 172.21.15.27:0/1423856500 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768034640 con 0x7f07700cb0c0 2023-12-19T18:22:12.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.972+0000 7f0775e65640 1 -- 172.21.15.27:0/1423856500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700cb0c0 msgr2=0x7f07700ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:12.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.972+0000 7f0775e65640 1 --2- 172.21.15.27:0/1423856500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700cb0c0 0x7f07700ad710 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f0768009d90 tx=0x7f076802bea0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.972+0000 7f0775e65640 1 -- 172.21.15.27:0/1423856500 shutdown_connections 2023-12-19T18:22:12.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.972+0000 7f0775e65640 1 --2- 172.21.15.27:0/1423856500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700cb0c0 0x7f07700ad710 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.972+0000 7f0775e65640 1 --2- 172.21.15.27:0/1423856500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f077006f2e0 0x7f07700cab80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.972+0000 7f0775e65640 1 -- 172.21.15.27:0/1423856500 >> 172.21.15.27:0/1423856500 conn(0x7f07700b05b0 msgr2=0x7f07700b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:12.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.972+0000 7f0775e65640 1 -- 172.21.15.27:0/1423856500 shutdown_connections 2023-12-19T18:22:12.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.972+0000 7f0775e65640 1 -- 172.21.15.27:0/1423856500 wait complete. 2023-12-19T18:22:12.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.974+0000 7f0775e65640 1 Processor -- start 2023-12-19T18:22:12.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.974+0000 7f0775e65640 1 -- start start 2023-12-19T18:22:12.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.975+0000 7f0775e65640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077006f2e0 0x7f07700cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:12.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.975+0000 7f0775e65640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700cb0c0 0x7f07700cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:12.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.975+0000 7f0775e65640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f077015ac00 0x7f07700ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:12.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.975+0000 7f0775e65640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07700ccce0 con 0x7f07700cb0c0 2023-12-19T18:22:12.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.975+0000 7f0775e65640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07701f6320 con 0x7f077006f2e0 2023-12-19T18:22:12.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.975+0000 7f0775e65640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07700cc690 con 0x7f077015ac00 2023-12-19T18:22:12.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.976+0000 7f076ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700cb0c0 0x7f07700cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:12.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.976+0000 7f076ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700cb0c0 0x7f07700cdf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42792/0 (socket says 172.21.15.27:42792) 2023-12-19T18:22:12.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.976+0000 7f076ffff640 1 -- 172.21.15.27:0/1425855534 learned_addr learned my addr 172.21.15.27:0/1425855534 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:12.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.976+0000 7f0775664640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f077015ac00 0x7f07700ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:12.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.976+0000 7f0774e63640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077006f2e0 0x7f07700cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:12.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.976+0000 7f076ffff640 1 -- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f077015ac00 msgr2=0x7f07700ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:12.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.976+0000 7f076ffff640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f077015ac00 0x7f07700ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.976+0000 7f076ffff640 1 -- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077006f2e0 msgr2=0x7f07700cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:12.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.976+0000 7f076ffff640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077006f2e0 0x7f07700cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:12.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.976+0000 7f076ffff640 1 -- 172.21.15.27:0/1425855534 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0768009a70 con 0x7f07700cb0c0 2023-12-19T18:22:12.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.977+0000 7f076ffff640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700cb0c0 0x7f07700cdf60 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f076000e9b0 tx=0x7f076000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:12.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.977+0000 7f076d7fa640 1 -- 172.21.15.27:0/1425855534 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0760009670 con 0x7f07700cb0c0 2023-12-19T18:22:12.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.977+0000 7f076d7fa640 1 -- 172.21.15.27:0/1425855534 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f076000c9a0 con 0x7f07700cb0c0 2023-12-19T18:22:12.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.977+0000 7f076d7fa640 1 -- 172.21.15.27:0/1425855534 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f076000cd50 con 0x7f07700cb0c0 2023-12-19T18:22:12.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.977+0000 7f0775e65640 1 -- 172.21.15.27:0/1425855534 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07700cc970 con 0x7f07700cb0c0 2023-12-19T18:22:12.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.977+0000 7f0775e65640 1 -- 172.21.15.27:0/1425855534 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07700ace00 con 0x7f07700cb0c0 2023-12-19T18:22:12.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.978+0000 7f0775e65640 1 -- 172.21.15.27:0/1425855534 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f077004e840 con 0x7f07700cb0c0 2023-12-19T18:22:12.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.980+0000 7f076d7fa640 1 -- 172.21.15.27:0/1425855534 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0760013470 con 0x7f07700cb0c0 2023-12-19T18:22:12.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.981+0000 7f076d7fa640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f075c06a460 0x7f075c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:12.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.981+0000 7f076d7fa640 1 -- 172.21.15.27:0/1425855534 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(132..132 src has 1..132) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f0760089090 con 0x7f07700cb0c0 2023-12-19T18:22:12.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.981+0000 7f0774e63640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f075c06a460 0x7f075c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:12.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.982+0000 7f0774e63640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f075c06a460 0x7f075c06c920 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f07640011e0 tx=0x7f0764009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:12.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:12.985+0000 7f076d7fa640 1 -- 172.21.15.27:0/1425855534 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f07600548d0 con 0x7f07700cb0c0 2023-12-19T18:22:13.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.181+0000 7f0775e65640 1 -- 172.21.15.27:0/1425855534 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f07700c4e10 con 0x7f07700cb0c0 2023-12-19T18:22:13.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.710+0000 7f076d7fa640 1 -- 172.21.15.27:0/1425855534 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v133) v1 ==== 170+0+0 (secure 0 0 0) 0x7f07600170d0 con 0x7f07700cb0c0 2023-12-19T18:22:13.713 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-12-19T18:22:13.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.718+0000 7f0775e65640 1 -- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f075c06a460 msgr2=0x7f075c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:13.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.718+0000 7f0775e65640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f075c06a460 0x7f075c06c920 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f07640011e0 tx=0x7f0764009040 comp rx=0 tx=0).stop 2023-12-19T18:22:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.719+0000 7f0775e65640 1 -- 172.21.15.27:0/1425855534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700cb0c0 msgr2=0x7f07700cdf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.719+0000 7f0775e65640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700cb0c0 0x7f07700cdf60 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f076000e9b0 tx=0x7f076000ee70 comp rx=0 tx=0).stop 2023-12-19T18:22:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.720+0000 7f0775e65640 1 -- 172.21.15.27:0/1425855534 shutdown_connections 2023-12-19T18:22:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.720+0000 7f0775e65640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f075c06a460 0x7f075c06c920 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.720+0000 7f0775e65640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f077015ac00 0x7f07700ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.720+0000 7f0775e65640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07700cb0c0 0x7f07700cdf60 secure :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f076000e9b0 tx=0x7f076000ee70 comp rx=0 tx=0).stop 2023-12-19T18:22:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.720+0000 7f0775e65640 1 --2- 172.21.15.27:0/1425855534 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077006f2e0 0x7f07700cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.720+0000 7f0775e65640 1 -- 172.21.15.27:0/1425855534 >> 172.21.15.27:0/1425855534 conn(0x7f07700b05b0 msgr2=0x7f07700ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.721+0000 7f0775e65640 1 -- 172.21.15.27:0/1425855534 shutdown_connections 2023-12-19T18:22:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.721+0000 7f0775e65640 1 -- 172.21.15.27:0/1425855534 wait complete. 2023-12-19T18:22:13.747 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:22:13.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.960+0000 7faf79a04640 1 Processor -- start 2023-12-19T18:22:13.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.960+0000 7faf79a04640 1 -- start start 2023-12-19T18:22:13.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.962+0000 7faf79a04640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faf740c2720 0x7faf7406fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:13.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.962+0000 7faf79a04640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf74071420 0x7faf7415ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:13.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.962+0000 7faf79a04640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf7406bc80 con 0x7faf7415b1a0 2023-12-19T18:22:13.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.962+0000 7faf79a04640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf7406d690 con 0x7faf740c2720 2023-12-19T18:22:13.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.962+0000 7faf79a04640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf7406d0a0 con 0x7faf74071420 2023-12-19T18:22:13.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.963+0000 7faf6bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7faf740c2720 0x7faf7406fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:13.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.963+0000 7faf72ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faf7415b1a0 0x7faf7406f520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41212/0 (socket says 172.21.15.27:41212) 2023-12-19T18:22:13.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.963+0000 7faf72ffd640 1 -- 172.21.15.27:0/4221044988 learned_addr learned my addr 172.21.15.27:0/4221044988 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:13.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.963+0000 7faf737fe640 1 --2- 172.21.15.27:0/4221044988 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf74071420 0x7faf7415ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:13.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.963+0000 7faf6bfff640 1 -- 172.21.15.27:0/4221044988 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf74071420 msgr2=0x7faf7415ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:13.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.963+0000 7faf6bfff640 1 --2- 172.21.15.27:0/4221044988 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf74071420 0x7faf7415ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:13.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.963+0000 7faf6bfff640 1 -- 172.21.15.27:0/4221044988 >> v1:172.21.15.27:6789/0 conn(0x7faf7415b1a0 legacy=0x7faf7406f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:13.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.963+0000 7faf6bfff640 1 -- 172.21.15.27:0/4221044988 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7406b6c0 con 0x7faf740c2720 2023-12-19T18:22:13.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.965+0000 7faf6bfff640 1 --2- 172.21.15.27:0/4221044988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf740c2720 0x7faf7406fc30 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0097a0 tx=0x7faf5c02b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acb9d13503c6d37 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:13.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.965+0000 7faf727fc640 1 -- 172.21.15.27:0/4221044988 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c031780 con 0x7faf740c2720 2023-12-19T18:22:13.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.965+0000 7faf727fc640 1 -- 172.21.15.27:0/4221044988 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf5c0318e0 con 0x7faf740c2720 2023-12-19T18:22:13.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.965+0000 7faf727fc640 1 -- 172.21.15.27:0/4221044988 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf5c031c70 con 0x7faf740c2720 2023-12-19T18:22:13.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.966+0000 7faf79a04640 1 -- 172.21.15.27:0/4221044988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf740c2720 msgr2=0x7faf7406fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:13.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.966+0000 7faf79a04640 1 --2- 172.21.15.27:0/4221044988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf740c2720 0x7faf7406fc30 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7faf5c0097a0 tx=0x7faf5c02b820 comp rx=0 tx=0).stop 2023-12-19T18:22:13.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.966+0000 7faf79a04640 1 -- 172.21.15.27:0/4221044988 shutdown_connections 2023-12-19T18:22:13.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.966+0000 7faf79a04640 1 --2- 172.21.15.27:0/4221044988 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf74071420 0x7faf7415ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:13.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.966+0000 7faf79a04640 1 --2- 172.21.15.27:0/4221044988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf740c2720 0x7faf7406fc30 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:13.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.966+0000 7faf79a04640 1 -- 172.21.15.27:0/4221044988 >> 172.21.15.27:0/4221044988 conn(0x7faf740cebd0 msgr2=0x7faf740cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:13.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.966+0000 7faf79a04640 1 -- 172.21.15.27:0/4221044988 shutdown_connections 2023-12-19T18:22:13.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.966+0000 7faf79a04640 1 -- 172.21.15.27:0/4221044988 wait complete. 2023-12-19T18:22:13.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.968+0000 7faf79a04640 1 Processor -- start 2023-12-19T18:22:13.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.968+0000 7faf79a04640 1 -- start start 2023-12-19T18:22:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.969+0000 7faf79a04640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf74071420 0x7faf740c1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.969+0000 7faf79a04640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf740c2720 0x7faf741e0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.969+0000 7faf79a04640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf7415b1a0 0x7faf740c3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.969+0000 7faf79a04640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf741f1c70 con 0x7faf74071420 2023-12-19T18:22:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.969+0000 7faf79a04640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf741f1f50 con 0x7faf740c2720 2023-12-19T18:22:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.969+0000 7faf79a04640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf741f2230 con 0x7faf7415b1a0 2023-12-19T18:22:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.969+0000 7faf72ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf74071420 0x7faf740c1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.969+0000 7faf72ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf74071420 0x7faf740c1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42824/0 (socket says 172.21.15.27:42824) 2023-12-19T18:22:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.969+0000 7faf72ffd640 1 -- 172.21.15.27:0/3915700922 learned_addr learned my addr 172.21.15.27:0/3915700922 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.970+0000 7faf737fe640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf7415b1a0 0x7faf740c3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.970+0000 7faf6bfff640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf740c2720 0x7faf741e0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:13.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.970+0000 7faf72ffd640 1 -- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf7415b1a0 msgr2=0x7faf740c3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:13.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.970+0000 7faf72ffd640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf7415b1a0 0x7faf740c3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:13.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.970+0000 7faf72ffd640 1 -- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf740c2720 msgr2=0x7faf741e0de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:13.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.970+0000 7faf72ffd640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf740c2720 0x7faf741e0de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:13.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.970+0000 7faf72ffd640 1 -- 172.21.15.27:0/3915700922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf5c009400 con 0x7faf74071420 2023-12-19T18:22:13.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.970+0000 7faf72ffd640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf74071420 0x7faf740c1160 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7faf6400c480 tx=0x7faf6400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:13.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.970+0000 7faf70ff9640 1 -- 172.21.15.27:0/3915700922 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf64005ce0 con 0x7faf74071420 2023-12-19T18:22:13.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.970+0000 7faf70ff9640 1 -- 172.21.15.27:0/3915700922 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf64005e40 con 0x7faf74071420 2023-12-19T18:22:13.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.970+0000 7faf70ff9640 1 -- 172.21.15.27:0/3915700922 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf640072f0 con 0x7faf74071420 2023-12-19T18:22:13.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.970+0000 7faf79a04640 1 -- 172.21.15.27:0/3915700922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf741f2510 con 0x7faf74071420 2023-12-19T18:22:13.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.971+0000 7faf79a04640 1 -- 172.21.15.27:0/3915700922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf740c16a0 con 0x7faf74071420 2023-12-19T18:22:13.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.972+0000 7faf79a04640 1 -- 172.21.15.27:0/3915700922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf74068e90 con 0x7faf74071420 2023-12-19T18:22:13.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.973+0000 7faf70ff9640 1 -- 172.21.15.27:0/3915700922 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faf64007510 con 0x7faf74071420 2023-12-19T18:22:13.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.974+0000 7faf70ff9640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf5406a430 0x7faf5406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:13.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.974+0000 7faf70ff9640 1 -- 172.21.15.27:0/3915700922 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5609+0+0 (secure 0 0 0) 0x7faf64087c40 con 0x7faf74071420 2023-12-19T18:22:13.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.975+0000 7faf6bfff640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf5406a430 0x7faf5406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:13.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.976+0000 7faf6bfff640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf5406a430 0x7faf5406c8f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faf5c036000 tx=0x7faf5c03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:13.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:13.979+0000 7faf70ff9640 1 -- 172.21.15.27:0/3915700922 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7faf64053330 con 0x7faf74071420 2023-12-19T18:22:14.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.215+0000 7faf79a04640 1 -- 172.21.15.27:0/3915700922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7faf7406bc80 con 0x7faf74071420 2023-12-19T18:22:14.230 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 14 and data pool 15 2023-12-19T18:22:14.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.230+0000 7faf70ff9640 1 -- 172.21.15.27:0/3915700922 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v50) v1 ==== 174+0+0 (secure 0 0 0) 0x7faf7406bc80 con 0x7faf74071420 2023-12-19T18:22:14.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.237+0000 7faf79a04640 1 -- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf5406a430 msgr2=0x7faf5406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:14.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.237+0000 7faf79a04640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf5406a430 0x7faf5406c8f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7faf5c036000 tx=0x7faf5c03d040 comp rx=0 tx=0).stop 2023-12-19T18:22:14.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.237+0000 7faf79a04640 1 -- 172.21.15.27:0/3915700922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf74071420 msgr2=0x7faf740c1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:14.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.237+0000 7faf79a04640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf74071420 0x7faf740c1160 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7faf6400c480 tx=0x7faf6400c940 comp rx=0 tx=0).stop 2023-12-19T18:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.237+0000 7faf79a04640 1 -- 172.21.15.27:0/3915700922 shutdown_connections 2023-12-19T18:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.237+0000 7faf79a04640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faf5406a430 0x7faf5406c8f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.237+0000 7faf79a04640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faf7415b1a0 0x7faf740c3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.237+0000 7faf79a04640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faf740c2720 0x7faf741e0de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.237+0000 7faf79a04640 1 --2- 172.21.15.27:0/3915700922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf74071420 0x7faf740c1160 secure :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7faf6400c480 tx=0x7faf6400c940 comp rx=0 tx=0).stop 2023-12-19T18:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.237+0000 7faf79a04640 1 -- 172.21.15.27:0/3915700922 >> 172.21.15.27:0/3915700922 conn(0x7faf740cebd0 msgr2=0x7faf740c62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.238+0000 7faf79a04640 1 -- 172.21.15.27:0/3915700922 shutdown_connections 2023-12-19T18:22:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.238+0000 7faf79a04640 1 -- 172.21.15.27:0/3915700922 wait complete. 2023-12-19T18:22:14.264 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:22:14.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.457+0000 7feaa7fff640 1 Processor -- start 2023-12-19T18:22:14.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.457+0000 7feaa7fff640 1 -- start start 2023-12-19T18:22:14.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.459+0000 7feaa7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa81553a0 0x7feaa80c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.459+0000 7feaa7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feaa815ee30 0x7feaa80c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.459+0000 7feaa7fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa80b0330 con 0x7feaa80afeb0 2023-12-19T18:22:14.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.459+0000 7feaa7fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa80cbcf0 con 0x7feaa815ee30 2023-12-19T18:22:14.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.459+0000 7feaa7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa80cb350 con 0x7feaa81553a0 2023-12-19T18:22:14.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.459+0000 7fea9ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7feaa80afeb0 0x7feaa815e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41222/0 (socket says 172.21.15.27:41222) 2023-12-19T18:22:14.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.459+0000 7fea9ffff640 1 -- 172.21.15.27:0/1389033616 learned_addr learned my addr 172.21.15.27:0/1389033616 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:14.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.460+0000 7feaa77fe640 1 --2- 172.21.15.27:0/1389033616 >> v2:172.21.15.27:3300/0 conn(0x7feaa815ee30 0x7feaa80c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.460+0000 7feaa6ffd640 1 --2- 172.21.15.27:0/1389033616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa81553a0 0x7feaa80c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.460+0000 7feaa77fe640 1 -- 172.21.15.27:0/1389033616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa81553a0 msgr2=0x7feaa80c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:14.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.460+0000 7feaa77fe640 1 --2- 172.21.15.27:0/1389033616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa81553a0 0x7feaa80c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.460+0000 7feaa77fe640 1 -- 172.21.15.27:0/1389033616 >> v1:172.21.15.27:6789/0 conn(0x7feaa80afeb0 legacy=0x7feaa815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:14.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.460+0000 7feaa77fe640 1 -- 172.21.15.27:0/1389033616 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa80cb630 con 0x7feaa815ee30 2023-12-19T18:22:14.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.460+0000 7feaa67fc640 1 -- 172.21.15.27:0/1389033616 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3895950696 0 0) 0x7feaa80b0330 con 0x7feaa80afeb0 2023-12-19T18:22:14.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.462+0000 7feaa77fe640 1 --2- 172.21.15.27:0/1389033616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa815ee30 0x7feaa80c39d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009d90 tx=0x7feaa002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677a744792e7e58e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:14.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.462+0000 7feaa67fc640 1 -- 172.21.15.27:0/1389033616 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa002cbd0 con 0x7feaa815ee30 2023-12-19T18:22:14.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.462+0000 7feaa67fc640 1 -- 172.21.15.27:0/1389033616 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feaa002cd30 con 0x7feaa815ee30 2023-12-19T18:22:14.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.462+0000 7feaa67fc640 1 -- 172.21.15.27:0/1389033616 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feaa002cea0 con 0x7feaa815ee30 2023-12-19T18:22:14.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.463+0000 7feaa7fff640 1 -- 172.21.15.27:0/1389033616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa815ee30 msgr2=0x7feaa80c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:14.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.463+0000 7feaa7fff640 1 --2- 172.21.15.27:0/1389033616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa815ee30 0x7feaa80c39d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7feaa0009d90 tx=0x7feaa002be20 comp rx=0 tx=0).stop 2023-12-19T18:22:14.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.463+0000 7feaa7fff640 1 -- 172.21.15.27:0/1389033616 shutdown_connections 2023-12-19T18:22:14.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.463+0000 7feaa7fff640 1 --2- 172.21.15.27:0/1389033616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa815ee30 0x7feaa80c39d0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.463+0000 7feaa7fff640 1 --2- 172.21.15.27:0/1389033616 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa81553a0 0x7feaa80c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.463+0000 7feaa7fff640 1 -- 172.21.15.27:0/1389033616 >> 172.21.15.27:0/1389033616 conn(0x7feaa8069c70 msgr2=0x7feaa806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:14.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.463+0000 7feaa7fff640 1 -- 172.21.15.27:0/1389033616 shutdown_connections 2023-12-19T18:22:14.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.463+0000 7feaa7fff640 1 -- 172.21.15.27:0/1389033616 wait complete. 2023-12-19T18:22:14.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.465+0000 7feaa7fff640 1 Processor -- start 2023-12-19T18:22:14.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.465+0000 7feaa7fff640 1 -- start start 2023-12-19T18:22:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.466+0000 7feaa7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa80afeb0 0x7feaa8149480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.466+0000 7feaa7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa81553a0 0x7feaa806f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.466+0000 7feaa7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaa815ee30 0x7feaa806fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.466+0000 7feaa7fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa80cddb0 con 0x7feaa80afeb0 2023-12-19T18:22:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.466+0000 7feaa7fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa80b42d0 con 0x7feaa815ee30 2023-12-19T18:22:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.466+0000 7feaa7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa81499c0 con 0x7feaa81553a0 2023-12-19T18:22:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.466+0000 7feaa6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa80afeb0 0x7feaa8149480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.466+0000 7feaa6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa80afeb0 0x7feaa8149480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42832/0 (socket says 172.21.15.27:42832) 2023-12-19T18:22:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.466+0000 7feaa6ffd640 1 -- 172.21.15.27:0/3057544555 learned_addr learned my addr 172.21.15.27:0/3057544555 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.467+0000 7fea9ffff640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa81553a0 0x7feaa806f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.467+0000 7feaa77fe640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaa815ee30 0x7feaa806fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.467+0000 7feaa6ffd640 1 -- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa81553a0 msgr2=0x7feaa806f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:14.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.467+0000 7feaa6ffd640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa81553a0 0x7feaa806f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.467+0000 7feaa6ffd640 1 -- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaa815ee30 msgr2=0x7feaa806fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:14.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.467+0000 7feaa6ffd640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaa815ee30 0x7feaa806fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.467+0000 7feaa6ffd640 1 -- 172.21.15.27:0/3057544555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa0009a70 con 0x7feaa80afeb0 2023-12-19T18:22:14.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.467+0000 7fea9ffff640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa81553a0 0x7feaa806f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:14.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.467+0000 7feaa77fe640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaa815ee30 0x7feaa806fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:14.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.467+0000 7feaa6ffd640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa80afeb0 0x7feaa8149480 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fea9800d830 tx=0x7fea9800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:14.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.468+0000 7fea9f7fe640 1 -- 172.21.15.27:0/3057544555 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9800b6c0 con 0x7feaa80afeb0 2023-12-19T18:22:14.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.468+0000 7fea9f7fe640 1 -- 172.21.15.27:0/3057544555 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea9800b820 con 0x7feaa80afeb0 2023-12-19T18:22:14.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.468+0000 7fea9f7fe640 1 -- 172.21.15.27:0/3057544555 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea9800bbb0 con 0x7feaa80afeb0 2023-12-19T18:22:14.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.468+0000 7feaa7fff640 1 -- 172.21.15.27:0/3057544555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa80cd1a0 con 0x7feaa80afeb0 2023-12-19T18:22:14.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.468+0000 7feaa7fff640 1 -- 172.21.15.27:0/3057544555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa80cd760 con 0x7feaa80afeb0 2023-12-19T18:22:14.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.469+0000 7feaa7fff640 1 -- 172.21.15.27:0/3057544555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa804e840 con 0x7feaa80afeb0 2023-12-19T18:22:14.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.470+0000 7fea9f7fe640 1 -- 172.21.15.27:0/3057544555 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fea9801e460 con 0x7feaa80afeb0 2023-12-19T18:22:14.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.471+0000 7fea9f7fe640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea8806a430 0x7fea8806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.471+0000 7fea9f7fe640 1 -- 172.21.15.27:0/3057544555 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea980891c0 con 0x7feaa80afeb0 2023-12-19T18:22:14.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.472+0000 7fea9ffff640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea8806a430 0x7fea8806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.473+0000 7fea9ffff640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea8806a430 0x7fea8806c8f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7feaa80b45c0 tx=0x7fea90009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:14.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.476+0000 7fea9f7fe640 1 -- 172.21.15.27:0/3057544555 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fea98054680 con 0x7feaa80afeb0 2023-12-19T18:22:14.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.670+0000 7feaa7fff640 1 -- 172.21.15.27:0/3057544555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feaa80af6a0 con 0x7feaa80afeb0 2023-12-19T18:22:14.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.673+0000 7fea9f7fe640 1 -- 172.21.15.27:0/3057544555 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+16065 (secure 0 0 0) 0x7fea98057ca0 con 0x7feaa80afeb0 2023-12-19T18:22:14.673 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:14.673 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":134,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:22:14.216731+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:22:12.328518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T18:22:13.251708+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:22:14.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.676+0000 7feaa7fff640 1 -- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea8806a430 msgr2=0x7fea8806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:14.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.676+0000 7feaa7fff640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea8806a430 0x7fea8806c8f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7feaa80b45c0 tx=0x7fea90009040 comp rx=0 tx=0).stop 2023-12-19T18:22:14.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.676+0000 7feaa7fff640 1 -- 172.21.15.27:0/3057544555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa80afeb0 msgr2=0x7feaa8149480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:14.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.676+0000 7feaa7fff640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa80afeb0 0x7feaa8149480 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fea9800d830 tx=0x7fea9800dcf0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.676+0000 7feaa7fff640 1 -- 172.21.15.27:0/3057544555 shutdown_connections 2023-12-19T18:22:14.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.676+0000 7feaa7fff640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea8806a430 0x7fea8806c8f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.676+0000 7feaa7fff640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feaa815ee30 0x7feaa806fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.676+0000 7feaa7fff640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feaa81553a0 0x7feaa806f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.676+0000 7feaa7fff640 1 --2- 172.21.15.27:0/3057544555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feaa80afeb0 0x7feaa8149480 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.676+0000 7feaa7fff640 1 -- 172.21.15.27:0/3057544555 >> 172.21.15.27:0/3057544555 conn(0x7feaa8069c70 msgr2=0x7feaa8152f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:14.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.676+0000 7feaa7fff640 1 -- 172.21.15.27:0/3057544555 shutdown_connections 2023-12-19T18:22:14.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.676+0000 7feaa7fff640 1 -- 172.21.15.27:0/3057544555 wait complete. 2023-12-19T18:22:14.695 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:22:14.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.925+0000 7f8fab0f3640 1 Processor -- start 2023-12-19T18:22:14.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.925+0000 7f8fab0f3640 1 -- start start 2023-12-19T18:22:14.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.927+0000 7f8fab0f3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8fa415ee30 0x7f8fa40c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.927+0000 7f8fab0f3640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa40b0320 con 0x7f8fa41553a0 2023-12-19T18:22:14.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.927+0000 7f8fab0f3640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fa40cb350 con 0x7f8fa40afeb0 2023-12-19T18:22:14.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.927+0000 7f8fab0f3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40cbcf0 con 0x7f8fa415ee30 2023-12-19T18:22:14.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.927+0000 7f8faa0f1640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8fa41553a0 0x7f8fa415e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41230/0 (socket says 172.21.15.27:41230) 2023-12-19T18:22:14.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.927+0000 7f8faa0f1640 1 -- 172.21.15.27:0/2112293987 learned_addr learned my addr 172.21.15.27:0/2112293987 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:14.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.927+0000 7f8fa98f0640 1 --2- 172.21.15.27:0/2112293987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8fa415ee30 0x7f8fa40c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3920600509 0 0) 0x7f8fa40b0320 con 0x7f8fa41553a0 2023-12-19T18:22:14.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f90003520 con 0x7f8fa41553a0 2023-12-19T18:22:14.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 251821417 0 0) 0x7f8f90003520 con 0x7f8fa41553a0 2023-12-19T18:22:14.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8fa415ee30 msgr2=0x7f8fa40c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:14.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 --2- 172.21.15.27:0/2112293987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8fa415ee30 0x7f8fa40c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 >> v1:172.21.15.192:6789/0 conn(0x7f8fa40afeb0 legacy=0x7f8fa40c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:14.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa40cb630 con 0x7f8fa41553a0 2023-12-19T18:22:14.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f8fa0001670 con 0x7f8fa41553a0 2023-12-19T18:22:14.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 >> v1:172.21.15.27:6789/0 conn(0x7f8fa41553a0 legacy=0x7f8fa415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:14.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 --2- 172.21.15.27:0/2112293987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f900039a0 0x7f8f90003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 --2- 172.21.15.27:0/2112293987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f900047c0 0x7f8f90004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 --2- 172.21.15.27:0/2112293987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8fa415ee30 0x7f8f90005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40cbcf0 con 0x7f8f900047c0 2023-12-19T18:22:14.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9000bb10 con 0x7f8f900039a0 2023-12-19T18:22:14.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.928+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f9000bcd0 con 0x7f8fa415ee30 2023-12-19T18:22:14.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.929+0000 7f8fa98f0640 1 --2- 172.21.15.27:0/2112293987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f900047c0 0x7f8f90004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.929+0000 7f8faa0f1640 1 --2- 172.21.15.27:0/2112293987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f900039a0 0x7f8f90003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.929+0000 7f8faa8f2640 1 --2- 172.21.15.27:0/2112293987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8fa415ee30 0x7f8f90005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.929+0000 7f8fa98f0640 1 -- 172.21.15.27:0/2112293987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8fa415ee30 msgr2=0x7f8f90005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:14.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.929+0000 7f8fa98f0640 1 --2- 172.21.15.27:0/2112293987 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8fa415ee30 0x7f8f90005230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.929+0000 7f8fa98f0640 1 -- 172.21.15.27:0/2112293987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f900039a0 msgr2=0x7f8f90003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:14.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.929+0000 7f8fa98f0640 1 --2- 172.21.15.27:0/2112293987 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8f900039a0 0x7f8f90003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.929+0000 7f8fa98f0640 1 -- 172.21.15.27:0/2112293987 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f90003520 con 0x7f8f900047c0 2023-12-19T18:22:14.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.930+0000 7f8fa98f0640 1 --2- 172.21.15.27:0/2112293987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f900047c0 0x7f8f90004c40 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009ac0 tx=0x7f8f900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd047c576ef08d76 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:14.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.931+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9402c6b0 con 0x7f8f900047c0 2023-12-19T18:22:14.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.931+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f9402c810 con 0x7f8f900047c0 2023-12-19T18:22:14.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.931+0000 7f8fa90ef640 1 -- 172.21.15.27:0/2112293987 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f9402cba0 con 0x7f8f900047c0 2023-12-19T18:22:14.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.931+0000 7f8fab0f3640 1 -- 172.21.15.27:0/2112293987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f900047c0 msgr2=0x7f8f90004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:14.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.931+0000 7f8fab0f3640 1 --2- 172.21.15.27:0/2112293987 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f900047c0 0x7f8f90004c40 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009ac0 tx=0x7f8f900057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.931+0000 7f8faa8f2640 1 -- 172.21.15.27:0/2112293987 reap_dead start 2023-12-19T18:22:14.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.932+0000 7f8fab0f3640 1 -- 172.21.15.27:0/2112293987 shutdown_connections 2023-12-19T18:22:14.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.932+0000 7f8fab0f3640 1 -- 172.21.15.27:0/2112293987 >> 172.21.15.27:0/2112293987 conn(0x7f8fa4069c70 msgr2=0x7f8fa406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:14.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.932+0000 7f8fab0f3640 1 -- 172.21.15.27:0/2112293987 shutdown_connections 2023-12-19T18:22:14.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.932+0000 7f8fab0f3640 1 -- 172.21.15.27:0/2112293987 wait complete. 2023-12-19T18:22:14.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.933+0000 7f8fab0f3640 1 Processor -- start 2023-12-19T18:22:14.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.934+0000 7f8fab0f3640 1 -- start start 2023-12-19T18:22:14.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.934+0000 7f8fab0f3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f900047c0 0x7f8fa40c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.934+0000 7f8fab0f3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8fa4151530 0x7f8fa406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8fab0f3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fa4144f80 0x7f8fa4145400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8fab0f3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa415cb90 con 0x7f8f900047c0 2023-12-19T18:22:14.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8fab0f3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa415c860 con 0x7f8fa4144f80 2023-12-19T18:22:14.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8fab0f3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa40c6240 con 0x7f8fa4151530 2023-12-19T18:22:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8faa0f1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f900047c0 0x7f8fa40c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8faa0f1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f900047c0 0x7f8fa40c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42860/0 (socket says 172.21.15.27:42860) 2023-12-19T18:22:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8faa0f1640 1 -- 172.21.15.27:0/705510815 learned_addr learned my addr 172.21.15.27:0/705510815 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8faa8f2640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fa4144f80 0x7f8fa4145400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8fa98f0640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8fa4151530 0x7f8fa406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8faa0f1640 1 -- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8fa4151530 msgr2=0x7f8fa406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8faa0f1640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8fa4151530 0x7f8fa406c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8faa0f1640 1 -- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fa4144f80 msgr2=0x7f8fa4145400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:14.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8faa0f1640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fa4144f80 0x7f8fa4145400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:14.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8faa0f1640 1 -- 172.21.15.27:0/705510815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa40c6520 con 0x7f8f900047c0 2023-12-19T18:22:14.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.935+0000 7f8faa8f2640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fa4144f80 0x7f8fa4145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:14.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.936+0000 7f8faa0f1640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f900047c0 0x7f8fa40c5a10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e800 tx=0x7f8fa000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:14.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.936+0000 7f8f9affd640 1 -- 172.21.15.27:0/705510815 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000c870 con 0x7f8f900047c0 2023-12-19T18:22:14.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.936+0000 7f8f9affd640 1 -- 172.21.15.27:0/705510815 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa000c9d0 con 0x7f8f900047c0 2023-12-19T18:22:14.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.936+0000 7f8f9affd640 1 -- 172.21.15.27:0/705510815 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000cd80 con 0x7f8f900047c0 2023-12-19T18:22:14.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.936+0000 7f8fab0f3640 1 -- 172.21.15.27:0/705510815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa4145940 con 0x7f8f900047c0 2023-12-19T18:22:14.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.936+0000 7f8fab0f3640 1 -- 172.21.15.27:0/705510815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa40c6800 con 0x7f8f900047c0 2023-12-19T18:22:14.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.937+0000 7f8fab0f3640 1 -- 172.21.15.27:0/705510815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa404e840 con 0x7f8f900047c0 2023-12-19T18:22:14.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.938+0000 7f8f9affd640 1 -- 172.21.15.27:0/705510815 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8fa001d460 con 0x7f8f900047c0 2023-12-19T18:22:14.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.939+0000 7f8f9affd640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8f8806a460 0x7f8f8806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:14.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.939+0000 7f8f9affd640 1 -- 172.21.15.27:0/705510815 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8fa0088910 con 0x7f8f900047c0 2023-12-19T18:22:14.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.940+0000 7f8fa98f0640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8f8806a460 0x7f8f8806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:14.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.941+0000 7f8fa98f0640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8f8806a460 0x7f8f8806c920 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009ac0 tx=0x7f8f94030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:14.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:14.944+0000 7f8f9affd640 1 -- 172.21.15.27:0/705510815 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8fa0053ef0 con 0x7f8f900047c0 2023-12-19T18:22:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.141+0000 7f8fab0f3640 1 -- 172.21.15.27:0/705510815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8fa404e0d0 con 0x7f8f900047c0 2023-12-19T18:22:15.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.144+0000 7f8f9affd640 1 -- 172.21.15.27:0/705510815 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f8fa0001560 con 0x7f8f900047c0 2023-12-19T18:22:15.145 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:15.145 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":134,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:22:14.216731+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:22:12.328518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T18:22:13.251708+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:22:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.147+0000 7f8fab0f3640 1 -- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8f8806a460 msgr2=0x7f8f8806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.147+0000 7f8fab0f3640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8f8806a460 0x7f8f8806c920 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009ac0 tx=0x7f8f94030000 comp rx=0 tx=0).stop 2023-12-19T18:22:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.147+0000 7f8fab0f3640 1 -- 172.21.15.27:0/705510815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f900047c0 msgr2=0x7f8fa40c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.147+0000 7f8fab0f3640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f900047c0 0x7f8fa40c5a10 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000e800 tx=0x7f8fa000ecc0 comp rx=0 tx=0).stop 2023-12-19T18:22:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.147+0000 7f8fab0f3640 1 -- 172.21.15.27:0/705510815 shutdown_connections 2023-12-19T18:22:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.147+0000 7f8fab0f3640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8f8806a460 0x7f8f8806c920 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:15.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.147+0000 7f8fab0f3640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8fa4144f80 0x7f8fa4145400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:15.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.147+0000 7f8fab0f3640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8fa4151530 0x7f8fa406c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:15.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.147+0000 7f8fab0f3640 1 --2- 172.21.15.27:0/705510815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f900047c0 0x7f8fa40c5a10 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:15.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.147+0000 7f8fab0f3640 1 -- 172.21.15.27:0/705510815 >> 172.21.15.27:0/705510815 conn(0x7f8fa4069c70 msgr2=0x7f8fa40ca760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:15.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.147+0000 7f8fab0f3640 1 -- 172.21.15.27:0/705510815 shutdown_connections 2023-12-19T18:22:15.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.148+0000 7f8fab0f3640 1 -- 172.21.15.27:0/705510815 wait complete. 2023-12-19T18:22:15.166 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:22:15.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.405+0000 7eff13fff640 1 Processor -- start 2023-12-19T18:22:15.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.405+0000 7eff13fff640 1 -- start start 2023-12-19T18:22:15.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.406+0000 7eff13fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7eff140cdb30 0x7eff140cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:15.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.407+0000 7eff13fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff14161250 con 0x7eff1415cfc0 2023-12-19T18:22:15.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.407+0000 7eff13fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff14161a50 con 0x7eff140d17a0 2023-12-19T18:22:15.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.407+0000 7eff13fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff140ce500 con 0x7eff140cdb30 2023-12-19T18:22:15.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.407+0000 7eff12ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7eff1415cfc0 0x7eff14160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41246/0 (socket says 172.21.15.27:41246) 2023-12-19T18:22:15.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.407+0000 7eff12ffd640 1 -- 172.21.15.27:0/405599159 learned_addr learned my addr 172.21.15.27:0/405599159 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.407+0000 7eff127fc640 1 --2- 172.21.15.27:0/405599159 >> v2:172.21.15.192:3300/0 conn(0x7eff140cdb30 0x7eff140cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.407+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 953406506 0 0) 0x7eff04001490 con 0x7eff1415cfc0 2023-12-19T18:22:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.407+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff00003520 con 0x7eff1415cfc0 2023-12-19T18:22:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2256224244 0 0) 0x7eff00003520 con 0x7eff1415cfc0 2023-12-19T18:22:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 >> v2:172.21.15.192:3300/0 conn(0x7eff140cdb30 msgr2=0x7eff140cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff11ffb640 1 --2- 172.21.15.27:0/405599159 >> v2:172.21.15.192:3300/0 conn(0x7eff140cdb30 0x7eff140cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 >> v1:172.21.15.192:6789/0 conn(0x7eff140d17a0 legacy=0x7eff140d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff14161490 con 0x7eff1415cfc0 2023-12-19T18:22:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7eff04001b80 con 0x7eff1415cfc0 2023-12-19T18:22:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 >> v1:172.21.15.27:6789/0 conn(0x7eff1415cfc0 legacy=0x7eff14160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff11ffb640 1 --2- 172.21.15.27:0/405599159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff000039c0 0x7eff00003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff11ffb640 1 --2- 172.21.15.27:0/405599159 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff000047c0 0x7eff00004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff11ffb640 1 --2- 172.21.15.27:0/405599159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff000051d0 0x7eff0000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:15.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff140ce500 con 0x7eff000051d0 2023-12-19T18:22:15.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff0000c120 con 0x7eff000039c0 2023-12-19T18:22:15.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff0000c2e0 con 0x7eff000047c0 2023-12-19T18:22:15.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.408+0000 7eff137fe640 1 --2- 172.21.15.27:0/405599159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff000051d0 0x7eff0000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:15.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.409+0000 7eff12ffd640 1 --2- 172.21.15.27:0/405599159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff000039c0 0x7eff00003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:15.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.409+0000 7eff137fe640 1 -- 172.21.15.27:0/405599159 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff000047c0 msgr2=0x7eff00004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:15.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.409+0000 7eff137fe640 1 --2- 172.21.15.27:0/405599159 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff000047c0 0x7eff00004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:15.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.409+0000 7eff137fe640 1 -- 172.21.15.27:0/405599159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff000039c0 msgr2=0x7eff00003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:15.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.409+0000 7eff137fe640 1 --2- 172.21.15.27:0/405599159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff000039c0 0x7eff00003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:15.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.409+0000 7eff137fe640 1 -- 172.21.15.27:0/405599159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff00003520 con 0x7eff000051d0 2023-12-19T18:22:15.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.409+0000 7eff12ffd640 1 --2- 172.21.15.27:0/405599159 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff000039c0 0x7eff00003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:15.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.410+0000 7eff137fe640 1 -- 172.21.15.27:0/405599159 reap_dead start 2023-12-19T18:22:15.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.410+0000 7eff137fe640 1 --2- 172.21.15.27:0/405599159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff000051d0 0x7eff0000b980 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7eff0c001720 tx=0x7eff000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72f73597bc19196 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:15.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.410+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c0052d0 con 0x7eff000051d0 2023-12-19T18:22:15.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.410+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0c02e050 con 0x7eff000051d0 2023-12-19T18:22:15.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.410+0000 7eff11ffb640 1 -- 172.21.15.27:0/405599159 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c0035f0 con 0x7eff000051d0 2023-12-19T18:22:15.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.411+0000 7eff13fff640 1 -- 172.21.15.27:0/405599159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff000051d0 msgr2=0x7eff0000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:15.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.411+0000 7eff13fff640 1 --2- 172.21.15.27:0/405599159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff000051d0 0x7eff0000b980 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7eff0c001720 tx=0x7eff000057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:15.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.411+0000 7eff13fff640 1 -- 172.21.15.27:0/405599159 shutdown_connections 2023-12-19T18:22:15.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.411+0000 7eff13fff640 1 --2- 172.21.15.27:0/405599159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff000051d0 0x7eff0000b980 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:15.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.411+0000 7eff13fff640 1 -- 172.21.15.27:0/405599159 >> 172.21.15.27:0/405599159 conn(0x7eff14072e20 msgr2=0x7eff14073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:15.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.411+0000 7eff13fff640 1 -- 172.21.15.27:0/405599159 shutdown_connections 2023-12-19T18:22:15.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.411+0000 7eff13fff640 1 -- 172.21.15.27:0/405599159 wait complete. 2023-12-19T18:22:15.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.413+0000 7eff13fff640 1 Processor -- start 2023-12-19T18:22:15.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.413+0000 7eff13fff640 1 -- start start 2023-12-19T18:22:15.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.414+0000 7eff13fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff000051d0 0x7eff14149a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:15.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.414+0000 7eff13fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff14148b20 0x7eff1414a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:15.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.414+0000 7eff13fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff1414a900 0x7eff1415b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:15.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.414+0000 7eff13fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff14148fa0 con 0x7eff1414a900 2023-12-19T18:22:15.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.414+0000 7eff13fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff141f2720 con 0x7eff14148b20 2023-12-19T18:22:15.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.414+0000 7eff13fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff141f2a00 con 0x7eff000051d0 2023-12-19T18:22:15.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.414+0000 7eff137fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff1414a900 0x7eff1415b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:15.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.414+0000 7eff137fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff1414a900 0x7eff1415b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42878/0 (socket says 172.21.15.27:42878) 2023-12-19T18:22:15.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.414+0000 7eff137fe640 1 -- 172.21.15.27:0/1946846340 learned_addr learned my addr 172.21.15.27:0/1946846340 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.415+0000 7eff137fe640 1 -- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff000051d0 msgr2=0x7eff14149a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.415+0000 7eff137fe640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff000051d0 0x7eff14149a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.415+0000 7eff137fe640 1 -- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff14148b20 msgr2=0x7eff1414a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.415+0000 7eff127fc640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff14148b20 0x7eff1414a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.415+0000 7eff137fe640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff14148b20 0x7eff1414a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.415+0000 7eff137fe640 1 -- 172.21.15.27:0/1946846340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff141f2ce0 con 0x7eff1414a900 2023-12-19T18:22:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.415+0000 7eff127fc640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff14148b20 0x7eff1414a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.415+0000 7eff137fe640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff1414a900 0x7eff1415b730 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7eff0c001840 tx=0x7eff0c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.415+0000 7eff0b7fe640 1 -- 172.21.15.27:0/1946846340 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c007a30 con 0x7eff1414a900 2023-12-19T18:22:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.415+0000 7eff0b7fe640 1 -- 172.21.15.27:0/1946846340 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff0c038040 con 0x7eff1414a900 2023-12-19T18:22:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.415+0000 7eff0b7fe640 1 -- 172.21.15.27:0/1946846340 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff0c007250 con 0x7eff1414a900 2023-12-19T18:22:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.415+0000 7eff13fff640 1 -- 172.21.15.27:0/1946846340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff141f2fc0 con 0x7eff1414a900 2023-12-19T18:22:15.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.416+0000 7eff13fff640 1 -- 172.21.15.27:0/1946846340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff141f2320 con 0x7eff1414a900 2023-12-19T18:22:15.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.417+0000 7eff13fff640 1 -- 172.21.15.27:0/1946846340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff1404e840 con 0x7eff1414a900 2023-12-19T18:22:15.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.418+0000 7eff0b7fe640 1 -- 172.21.15.27:0/1946846340 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7eff0c0073b0 con 0x7eff1414a900 2023-12-19T18:22:15.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.419+0000 7eff0b7fe640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efef406a430 0x7efef406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:15.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.419+0000 7eff0b7fe640 1 -- 172.21.15.27:0/1946846340 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff0c0a9040 con 0x7eff1414a900 2023-12-19T18:22:15.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.419+0000 7eff12ffd640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efef406a430 0x7efef406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.420+0000 7eff12ffd640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efef406a430 0x7efef406c8f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7eff04007c40 tx=0x7eff04009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.423+0000 7eff0b7fe640 1 -- 172.21.15.27:0/1946846340 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7eff0c074590 con 0x7eff1414a900 2023-12-19T18:22:15.480 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-12-19T18:22:15.480 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-12-19T18:22:15.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:15.662+0000 7eff13fff640 1 -- 172.21.15.27:0/1946846340 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7eff140c9e30 con 0x7eff1414a900 2023-12-19T18:22:16.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.248+0000 7eff0b7fe640 1 -- 172.21.15.27:0/1946846340 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) v1 ==== 118+0+0 (secure 0 0 0) 0x7eff0c0430b0 con 0x7eff1414a900 2023-12-19T18:22:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.252+0000 7eff13fff640 1 -- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efef406a430 msgr2=0x7efef406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.252+0000 7eff13fff640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efef406a430 0x7efef406c8f0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7eff04007c40 tx=0x7eff04009040 comp rx=0 tx=0).stop 2023-12-19T18:22:16.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.252+0000 7eff13fff640 1 -- 172.21.15.27:0/1946846340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff1414a900 msgr2=0x7eff1415b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.252+0000 7eff13fff640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff1414a900 0x7eff1415b730 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7eff0c001840 tx=0x7eff0c001120 comp rx=0 tx=0).stop 2023-12-19T18:22:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.252+0000 7eff13fff640 1 -- 172.21.15.27:0/1946846340 shutdown_connections 2023-12-19T18:22:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.252+0000 7eff13fff640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efef406a430 0x7efef406c8f0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.252+0000 7eff13fff640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff1414a900 0x7eff1415b730 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.252+0000 7eff13fff640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7eff14148b20 0x7eff1414a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.252+0000 7eff13fff640 1 --2- 172.21.15.27:0/1946846340 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7eff000051d0 0x7eff14149a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.252+0000 7eff13fff640 1 -- 172.21.15.27:0/1946846340 >> 172.21.15.27:0/1946846340 conn(0x7eff14072e20 msgr2=0x7eff140d3620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.252+0000 7eff13fff640 1 -- 172.21.15.27:0/1946846340 shutdown_connections 2023-12-19T18:22:16.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.252+0000 7eff13fff640 1 -- 172.21.15.27:0/1946846340 wait complete. 2023-12-19T18:22:16.274 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:22:16.274 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:22:16.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.506+0000 7f2cc1434640 1 Processor -- start 2023-12-19T18:22:16.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.506+0000 7f2cc1434640 1 -- start start 2023-12-19T18:22:16.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.508+0000 7f2cc1434640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cbc0ade20 0x7f2cbc15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.508+0000 7f2cc1434640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2cbc15b810 0x7f2cbc0c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.508+0000 7f2cc1434640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cbc15c030 con 0x7f2cbc158790 2023-12-19T18:22:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.508+0000 7f2cc1434640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cbc0c99d0 con 0x7f2cbc15b810 2023-12-19T18:22:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.508+0000 7f2cc1434640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc158b70 con 0x7f2cbc0ade20 2023-12-19T18:22:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.508+0000 7f2cc0c33640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2cbc15b810 0x7f2cbc0c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.508+0000 7f2cc0c33640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f2cbc15b810 0x7f2cbc0c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42892/0 (socket says 172.21.15.27:42892) 2023-12-19T18:22:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.508+0000 7f2cc0c33640 1 -- 172.21.15.27:0/1202830448 learned_addr learned my addr 172.21.15.27:0/1202830448 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.508+0000 7f2cbb7fe640 1 --2- 172.21.15.27:0/1202830448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cbc0ade20 0x7f2cbc15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.508+0000 7f2cc0c33640 1 -- 172.21.15.27:0/1202830448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cbc0ade20 msgr2=0x7f2cbc15b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:16.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.508+0000 7f2cc0c33640 1 --2- 172.21.15.27:0/1202830448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cbc0ade20 0x7f2cbc15b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:16.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.509+0000 7f2cc0c33640 1 -- 172.21.15.27:0/1202830448 >> v1:172.21.15.192:6789/0 conn(0x7f2cbc158790 legacy=0x7f2cbc0ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:16.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.509+0000 7f2cc0c33640 1 -- 172.21.15.27:0/1202830448 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cbc0c9030 con 0x7f2cbc15b810 2023-12-19T18:22:16.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.509+0000 7f2cbb7fe640 1 --2- 172.21.15.27:0/1202830448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cbc0ade20 0x7f2cbc15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.510+0000 7f2cc0c33640 1 --2- 172.21.15.27:0/1202830448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cbc15b810 0x7f2cbc0c3820 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009870 tx=0x7f2cb402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9934130beecf0e29 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.510+0000 7f2cbaffd640 1 -- 172.21.15.27:0/1202830448 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4031640 con 0x7f2cbc15b810 2023-12-19T18:22:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.510+0000 7f2cbaffd640 1 -- 172.21.15.27:0/1202830448 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb40317a0 con 0x7f2cbc15b810 2023-12-19T18:22:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.510+0000 7f2cbaffd640 1 -- 172.21.15.27:0/1202830448 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb4031b30 con 0x7f2cbc15b810 2023-12-19T18:22:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.511+0000 7f2cc1434640 1 -- 172.21.15.27:0/1202830448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cbc15b810 msgr2=0x7f2cbc0c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.511+0000 7f2cc1434640 1 --2- 172.21.15.27:0/1202830448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cbc15b810 0x7f2cbc0c3820 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2cb4009870 tx=0x7f2cb402b8f0 comp rx=0 tx=0).stop 2023-12-19T18:22:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.511+0000 7f2cc1434640 1 -- 172.21.15.27:0/1202830448 shutdown_connections 2023-12-19T18:22:16.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.511+0000 7f2cc1434640 1 --2- 172.21.15.27:0/1202830448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cbc15b810 0x7f2cbc0c3820 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:16.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.511+0000 7f2cc1434640 1 --2- 172.21.15.27:0/1202830448 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cbc0ade20 0x7f2cbc15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:16.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.511+0000 7f2cc1434640 1 -- 172.21.15.27:0/1202830448 >> 172.21.15.27:0/1202830448 conn(0x7f2cbc0b05b0 msgr2=0x7f2cbc0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:16.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.511+0000 7f2cc1434640 1 -- 172.21.15.27:0/1202830448 shutdown_connections 2023-12-19T18:22:16.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.511+0000 7f2cc1434640 1 -- 172.21.15.27:0/1202830448 wait complete. 2023-12-19T18:22:16.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.513+0000 7f2cc1434640 1 Processor -- start 2023-12-19T18:22:16.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.513+0000 7f2cc1434640 1 -- start start 2023-12-19T18:22:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.514+0000 7f2cc1434640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cbc0ade20 0x7f2cbc0cc580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.514+0000 7f2cc1434640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cbc158790 0x7f2cbc0cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.514+0000 7f2cc1434640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cbc15b810 0x7f2cbc14e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.514+0000 7f2cc1434640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0ccac0 con 0x7f2cbc158790 2023-12-19T18:22:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.514+0000 7f2cc1434640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc0ce4e0 con 0x7f2cbc0ade20 2023-12-19T18:22:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.514+0000 7f2cc1434640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cbc15d000 con 0x7f2cbc15b810 2023-12-19T18:22:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.514+0000 7f2cbb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cbc158790 0x7f2cbc0cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.514+0000 7f2cbb7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cbc158790 0x7f2cbc0cdfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42898/0 (socket says 172.21.15.27:42898) 2023-12-19T18:22:16.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.514+0000 7f2cbb7fe640 1 -- 172.21.15.27:0/921291005 learned_addr learned my addr 172.21.15.27:0/921291005 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.515+0000 7f2cc0c33640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cbc15b810 0x7f2cbc14e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.515+0000 7f2cbbfff640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cbc0ade20 0x7f2cbc0cc580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.515+0000 7f2cbb7fe640 1 -- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cbc15b810 msgr2=0x7f2cbc14e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.515+0000 7f2cbb7fe640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cbc15b810 0x7f2cbc14e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.515+0000 7f2cbb7fe640 1 -- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cbc0ade20 msgr2=0x7f2cbc0cc580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.515+0000 7f2cbb7fe640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cbc0ade20 0x7f2cbc0cc580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.515+0000 7f2cbb7fe640 1 -- 172.21.15.27:0/921291005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb40094d0 con 0x7f2cbc158790 2023-12-19T18:22:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.515+0000 7f2cbb7fe640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cbc158790 0x7f2cbc0cdfa0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000cf60 tx=0x7f2cb000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.515+0000 7f2cb8ff9640 1 -- 172.21.15.27:0/921291005 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb000d840 con 0x7f2cbc158790 2023-12-19T18:22:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.516+0000 7f2cb8ff9640 1 -- 172.21.15.27:0/921291005 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cb000d9a0 con 0x7f2cbc158790 2023-12-19T18:22:16.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.516+0000 7f2cb8ff9640 1 -- 172.21.15.27:0/921291005 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cb000dd30 con 0x7f2cbc158790 2023-12-19T18:22:16.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.516+0000 7f2cc1434640 1 -- 172.21.15.27:0/921291005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cbc15d2e0 con 0x7f2cbc158790 2023-12-19T18:22:16.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.516+0000 7f2cc1434640 1 -- 172.21.15.27:0/921291005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cbc15d8a0 con 0x7f2cbc158790 2023-12-19T18:22:16.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.517+0000 7f2cc1434640 1 -- 172.21.15.27:0/921291005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cbc04e840 con 0x7f2cbc158790 2023-12-19T18:22:16.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.518+0000 7f2cb8ff9640 1 -- 172.21.15.27:0/921291005 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2cb0015440 con 0x7f2cbc158790 2023-12-19T18:22:16.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.519+0000 7f2cb8ff9640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ca806a430 0x7f2ca806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:16.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.519+0000 7f2cb8ff9640 1 -- 172.21.15.27:0/921291005 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2cb0089240 con 0x7f2cbc158790 2023-12-19T18:22:16.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.520+0000 7f2cbbfff640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ca806a430 0x7f2ca806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.521+0000 7f2cbbfff640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ca806a430 0x7f2ca806c8f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc1528c0 tx=0x7f2cac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:16.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.523+0000 7f2cb8ff9640 1 -- 172.21.15.27:0/921291005 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2cb0054970 con 0x7f2cbc158790 2023-12-19T18:22:16.663 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:22:16.671 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:22:16.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:16.767+0000 7f2cc1434640 1 -- 172.21.15.27:0/921291005 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2cbc06cbe0 con 0x7f2cbc158790 2023-12-19T18:22:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:17.750+0000 7f2cb8ff9640 1 -- 172.21.15.27:0/921291005 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v54) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2cb0057f90 con 0x7f2cbc158790 2023-12-19T18:22:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:17.753+0000 7f2cc1434640 1 -- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ca806a430 msgr2=0x7f2ca806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:17.753+0000 7f2cc1434640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ca806a430 0x7f2ca806c8f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f2cbc1528c0 tx=0x7f2cac009040 comp rx=0 tx=0).stop 2023-12-19T18:22:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:17.753+0000 7f2cc1434640 1 -- 172.21.15.27:0/921291005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cbc158790 msgr2=0x7f2cbc0cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:17.753+0000 7f2cc1434640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cbc158790 0x7f2cbc0cdfa0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f2cb000cf60 tx=0x7f2cb000cf90 comp rx=0 tx=0).stop 2023-12-19T18:22:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:17.753+0000 7f2cc1434640 1 -- 172.21.15.27:0/921291005 shutdown_connections 2023-12-19T18:22:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:17.753+0000 7f2cc1434640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ca806a430 0x7f2ca806c8f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:17.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:17.753+0000 7f2cc1434640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2cbc15b810 0x7f2cbc14e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:17.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:17.753+0000 7f2cc1434640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cbc158790 0x7f2cbc0cdfa0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:17.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:17.753+0000 7f2cc1434640 1 --2- 172.21.15.27:0/921291005 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2cbc0ade20 0x7f2cbc0cc580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:17.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:17.753+0000 7f2cc1434640 1 -- 172.21.15.27:0/921291005 >> 172.21.15.27:0/921291005 conn(0x7f2cbc0b05b0 msgr2=0x7f2cbc0b0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:17.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:17.753+0000 7f2cc1434640 1 -- 172.21.15.27:0/921291005 shutdown_connections 2023-12-19T18:22:17.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:17.753+0000 7f2cc1434640 1 -- 172.21.15.27:0/921291005 wait complete. 2023-12-19T18:22:17.773 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:22:17.773 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:22:18.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.011+0000 7fc9935e7640 1 Processor -- start 2023-12-19T18:22:18.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.011+0000 7fc9935e7640 1 -- start start 2023-12-19T18:22:18.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.012+0000 7fc9935e7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc98c15ae80 0x7fc98c0d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.012+0000 7fc9935e7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98c15e4c0 con 0x7fc98c06b390 2023-12-19T18:22:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.012+0000 7fc9935e7640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc98c06c940 con 0x7fc98c0cea40 2023-12-19T18:22:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.012+0000 7fc9935e7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c15e7a0 con 0x7fc98c15ae80 2023-12-19T18:22:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.013+0000 7fc991b84640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc98c06b390 0x7fc98c0cc3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41252/0 (socket says 172.21.15.27:41252) 2023-12-19T18:22:18.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.013+0000 7fc991b84640 1 -- 172.21.15.27:0/2590270864 learned_addr learned my addr 172.21.15.27:0/2590270864 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:18.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.013+0000 7fc991383640 1 --2- 172.21.15.27:0/2590270864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc98c15ae80 0x7fc98c0d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.013+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3410724316 0 0) 0x7fc98c15e4c0 con 0x7fc98c06b390 2023-12-19T18:22:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.013+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc978003520 con 0x7fc98c06b390 2023-12-19T18:22:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 630428560 0 0) 0x7fc978003520 con 0x7fc98c06b390 2023-12-19T18:22:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc98c15ae80 msgr2=0x7fc98c0d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc97ffff640 1 --2- 172.21.15.27:0/2590270864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc98c15ae80 0x7fc98c0d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 >> v1:172.21.15.192:6789/0 conn(0x7fc98c0cea40 legacy=0x7fc98c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c06cc20 con 0x7fc98c06b390 2023-12-19T18:22:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fc9840019f0 con 0x7fc98c06b390 2023-12-19T18:22:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 >> v1:172.21.15.27:6789/0 conn(0x7fc98c06b390 legacy=0x7fc98c0cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc97ffff640 1 --2- 172.21.15.27:0/2590270864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780039a0 0x7fc978003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc97ffff640 1 --2- 172.21.15.27:0/2590270864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc98c15ae80 0x7fc9780047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc97ffff640 1 --2- 172.21.15.27:0/2590270864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc978004d00 0x7fc978005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c15e7a0 con 0x7fc9780039a0 2023-12-19T18:22:18.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc97800bb10 con 0x7fc978004d00 2023-12-19T18:22:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc97800bcd0 con 0x7fc98c15ae80 2023-12-19T18:22:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.014+0000 7fc991383640 1 --2- 172.21.15.27:0/2590270864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780039a0 0x7fc978003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.015+0000 7fc990b82640 1 --2- 172.21.15.27:0/2590270864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc98c15ae80 0x7fc9780047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.015+0000 7fc991b84640 1 --2- 172.21.15.27:0/2590270864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc978004d00 0x7fc978005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.015+0000 7fc991383640 1 -- 172.21.15.27:0/2590270864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc98c15ae80 msgr2=0x7fc9780047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.015+0000 7fc991383640 1 --2- 172.21.15.27:0/2590270864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc98c15ae80 0x7fc9780047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.015+0000 7fc991383640 1 -- 172.21.15.27:0/2590270864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc978004d00 msgr2=0x7fc978005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.015+0000 7fc991383640 1 --2- 172.21.15.27:0/2590270864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc978004d00 0x7fc978005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.015+0000 7fc991383640 1 -- 172.21.15.27:0/2590270864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc978003520 con 0x7fc9780039a0 2023-12-19T18:22:18.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.016+0000 7fc991383640 1 --2- 172.21.15.27:0/2590270864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780039a0 0x7fc978003dc0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc988009ac0 tx=0x7fc9780057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc9e6610d407a6b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.016+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98802c6b0 con 0x7fc9780039a0 2023-12-19T18:22:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.016+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc98802c810 con 0x7fc9780039a0 2023-12-19T18:22:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.016+0000 7fc97ffff640 1 -- 172.21.15.27:0/2590270864 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc98802cba0 con 0x7fc9780039a0 2023-12-19T18:22:18.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.017+0000 7fc9935e7640 1 -- 172.21.15.27:0/2590270864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780039a0 msgr2=0x7fc978003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.017+0000 7fc9935e7640 1 --2- 172.21.15.27:0/2590270864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780039a0 0x7fc978003dc0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7fc988009ac0 tx=0x7fc9780057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.017+0000 7fc991b84640 1 -- 172.21.15.27:0/2590270864 reap_dead start 2023-12-19T18:22:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.017+0000 7fc9935e7640 1 -- 172.21.15.27:0/2590270864 shutdown_connections 2023-12-19T18:22:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.017+0000 7fc9935e7640 1 -- 172.21.15.27:0/2590270864 >> 172.21.15.27:0/2590270864 conn(0x7fc98c0cd810 msgr2=0x7fc98c0cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.017+0000 7fc9935e7640 1 -- 172.21.15.27:0/2590270864 shutdown_connections 2023-12-19T18:22:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.017+0000 7fc9935e7640 1 -- 172.21.15.27:0/2590270864 wait complete. 2023-12-19T18:22:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.019+0000 7fc9935e7640 1 Processor -- start 2023-12-19T18:22:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.019+0000 7fc9935e7640 1 -- start start 2023-12-19T18:22:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.020+0000 7fc9935e7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780039a0 0x7fc98c15fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.020+0000 7fc9935e7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc98c146da0 0x7fc98c0c6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.020+0000 7fc9935e7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc98c0b5c60 0x7fc98c0b60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc9935e7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c0b57d0 con 0x7fc9780039a0 2023-12-19T18:22:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc9935e7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c14d160 con 0x7fc98c146da0 2023-12-19T18:22:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc9935e7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc98c0b4c20 con 0x7fc98c0b5c60 2023-12-19T18:22:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc991383640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780039a0 0x7fc98c15fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc991383640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780039a0 0x7fc98c15fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42918/0 (socket says 172.21.15.27:42918) 2023-12-19T18:22:18.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc991383640 1 -- 172.21.15.27:0/925666958 learned_addr learned my addr 172.21.15.27:0/925666958 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc991383640 1 -- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc98c0b5c60 msgr2=0x7fc98c0b60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc991b84640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc98c0b5c60 0x7fc98c0b60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc990b82640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc98c146da0 0x7fc98c0c6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc991383640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc98c0b5c60 0x7fc98c0b60e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc991383640 1 -- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc98c146da0 msgr2=0x7fc98c0c6e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc991383640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc98c146da0 0x7fc98c0c6e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc991383640 1 -- 172.21.15.27:0/925666958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc98c0b4ea0 con 0x7fc9780039a0 2023-12-19T18:22:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.021+0000 7fc990b82640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc98c146da0 0x7fc98c0c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.022+0000 7fc991383640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780039a0 0x7fc98c15fff0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc988009ac0 tx=0x7fc988004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.022+0000 7fc97dffb640 1 -- 172.21.15.27:0/925666958 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988003810 con 0x7fc9780039a0 2023-12-19T18:22:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.022+0000 7fc97dffb640 1 -- 172.21.15.27:0/925666958 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc988003970 con 0x7fc9780039a0 2023-12-19T18:22:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.022+0000 7fc97dffb640 1 -- 172.21.15.27:0/925666958 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc988003d00 con 0x7fc9780039a0 2023-12-19T18:22:18.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.022+0000 7fc9935e7640 1 -- 172.21.15.27:0/925666958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc98c0b5180 con 0x7fc9780039a0 2023-12-19T18:22:18.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.022+0000 7fc9935e7640 1 -- 172.21.15.27:0/925666958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc98c0c7350 con 0x7fc9780039a0 2023-12-19T18:22:18.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.023+0000 7fc9935e7640 1 -- 172.21.15.27:0/925666958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc98c0c74e0 con 0x7fc9780039a0 2023-12-19T18:22:18.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.024+0000 7fc97dffb640 1 -- 172.21.15.27:0/925666958 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc988040450 con 0x7fc9780039a0 2023-12-19T18:22:18.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.025+0000 7fc97dffb640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc97006a460 0x7fc97006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.025+0000 7fc97dffb640 1 -- 172.21.15.27:0/925666958 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc9880aa820 con 0x7fc9780039a0 2023-12-19T18:22:18.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.026+0000 7fc990b82640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc97006a460 0x7fc97006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.027+0000 7fc990b82640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc97006a460 0x7fc97006c920 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc980000c00 tx=0x7fc980009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:18.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.030+0000 7fc97dffb640 1 -- 172.21.15.27:0/925666958 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc988075d70 con 0x7fc9780039a0 2023-12-19T18:22:18.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.262+0000 7fc9935e7640 1 -- 172.21.15.27:0/925666958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc98c068e90 con 0x7fc9780039a0 2023-12-19T18:22:18.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.263+0000 7fc97dffb640 1 -- 172.21.15.27:0/925666958 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4711 (secure 0 0 0) 0x7fc9880044e0 con 0x7fc9780039a0 2023-12-19T18:22:18.262 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:18.263 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2713867269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2713867269},{"type":"v1","addr":"172.21.15.192:6837","nonce":2713867269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":5534,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/919878747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":919878747},{"type":"v1","addr":"172.21.15.27:6835","nonce":919878747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6160,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/861111763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":861111763},{"type":"v1","addr":"172.21.15.192:6836","nonce":861111763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:22:14.217224+0000","modified":"2023-12-19T18:22:17.743185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7320":{"gid":7320,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3670863819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3670863819},{"type":"v1","addr":"172.21.15.27:6834","nonce":3670863819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T18:22:18.263 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 54 2023-12-19T18:22:18.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.266+0000 7fc9935e7640 1 -- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc97006a460 msgr2=0x7fc97006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.266+0000 7fc9935e7640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc97006a460 0x7fc97006c920 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fc980000c00 tx=0x7fc980009040 comp rx=0 tx=0).stop 2023-12-19T18:22:18.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.266+0000 7fc9935e7640 1 -- 172.21.15.27:0/925666958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780039a0 msgr2=0x7fc98c15fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.266+0000 7fc9935e7640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780039a0 0x7fc98c15fff0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7fc988009ac0 tx=0x7fc988004730 comp rx=0 tx=0).stop 2023-12-19T18:22:18.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.266+0000 7fc9935e7640 1 -- 172.21.15.27:0/925666958 shutdown_connections 2023-12-19T18:22:18.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.266+0000 7fc9935e7640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc97006a460 0x7fc97006c920 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.266+0000 7fc9935e7640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc98c0b5c60 0x7fc98c0b60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.266+0000 7fc9935e7640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc98c146da0 0x7fc98c0c6e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.266+0000 7fc9935e7640 1 --2- 172.21.15.27:0/925666958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780039a0 0x7fc98c15fff0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.266+0000 7fc9935e7640 1 -- 172.21.15.27:0/925666958 >> 172.21.15.27:0/925666958 conn(0x7fc98c0cd810 msgr2=0x7fc98c15d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:18.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.266+0000 7fc9935e7640 1 -- 172.21.15.27:0/925666958 shutdown_connections 2023-12-19T18:22:18.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.266+0000 7fc9935e7640 1 -- 172.21.15.27:0/925666958 wait complete. 2023-12-19T18:22:18.286 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:22:18.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.520+0000 7f5125127640 1 Processor -- start 2023-12-19T18:22:18.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.520+0000 7f5125127640 1 -- start start 2023-12-19T18:22:18.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.522+0000 7f5125127640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f512015ee30 0x7f51200c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.522+0000 7f5125127640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51200cb350 con 0x7f51200afeb0 2023-12-19T18:22:18.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.522+0000 7f5125127640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51200cb630 con 0x7f51201553a0 2023-12-19T18:22:18.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.522+0000 7f5125127640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51200cbcf0 con 0x7f512015ee30 2023-12-19T18:22:18.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.522+0000 7f5117fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f512015ee30 0x7f51200c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.522+0000 7f5124926640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f51200afeb0 0x7f51200c39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41264/0 (socket says 172.21.15.27:41264) 2023-12-19T18:22:18.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.522+0000 7f5124926640 1 -- 172.21.15.27:0/3268741871 learned_addr learned my addr 172.21.15.27:0/3268741871 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:18.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.523+0000 7f5117fff640 1 -- 172.21.15.27:0/3268741871 >> v1:172.21.15.192:6789/0 conn(0x7f51201553a0 legacy=0x7f512015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:18.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.523+0000 7f5117fff640 1 -- 172.21.15.27:0/3268741871 >> v1:172.21.15.27:6789/0 conn(0x7f51200afeb0 legacy=0x7f51200c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.523+0000 7f5117fff640 1 -- 172.21.15.27:0/3268741871 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51200cb910 con 0x7f512015ee30 2023-12-19T18:22:18.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.523+0000 7f511f7fe640 1 -- 172.21.15.27:0/3268741871 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1919008503 0 0) 0x7f51200cb350 con 0x7f51200afeb0 2023-12-19T18:22:18.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.524+0000 7f5117fff640 1 --2- 172.21.15.27:0/3268741871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512015ee30 0x7f51200c50a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f51080097a0 tx=0x7f510802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a256d3cbdb7d0f08 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:18.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.524+0000 7f511f7fe640 1 -- 172.21.15.27:0/3268741871 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108031830 con 0x7f512015ee30 2023-12-19T18:22:18.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.524+0000 7f511f7fe640 1 -- 172.21.15.27:0/3268741871 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5108031990 con 0x7f512015ee30 2023-12-19T18:22:18.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.524+0000 7f511f7fe640 1 -- 172.21.15.27:0/3268741871 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5108031d20 con 0x7f512015ee30 2023-12-19T18:22:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.525+0000 7f5125127640 1 -- 172.21.15.27:0/3268741871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512015ee30 msgr2=0x7f51200c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.525+0000 7f5125127640 1 --2- 172.21.15.27:0/3268741871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512015ee30 0x7f51200c50a0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f51080097a0 tx=0x7f510802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.525+0000 7f5125127640 1 -- 172.21.15.27:0/3268741871 shutdown_connections 2023-12-19T18:22:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.525+0000 7f5125127640 1 --2- 172.21.15.27:0/3268741871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512015ee30 0x7f51200c50a0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.525+0000 7f5125127640 1 -- 172.21.15.27:0/3268741871 >> 172.21.15.27:0/3268741871 conn(0x7f5120069c70 msgr2=0x7f512006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.525+0000 7f5125127640 1 -- 172.21.15.27:0/3268741871 shutdown_connections 2023-12-19T18:22:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.525+0000 7f5125127640 1 -- 172.21.15.27:0/3268741871 wait complete. 2023-12-19T18:22:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.527+0000 7f5125127640 1 Processor -- start 2023-12-19T18:22:18.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.527+0000 7f5125127640 1 -- start start 2023-12-19T18:22:18.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.528+0000 7f5125127640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51200afeb0 0x7f5120157ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.528+0000 7f5125127640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51201553a0 0x7f51200b1750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.528+0000 7f5125127640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512015ee30 0x7f51200c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.528+0000 7f5125127640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51200cd1c0 con 0x7f512015ee30 2023-12-19T18:22:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.528+0000 7f5125127640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51200cc5b0 con 0x7f51201553a0 2023-12-19T18:22:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.528+0000 7f5125127640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51200cc890 con 0x7f51200afeb0 2023-12-19T18:22:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.529+0000 7f5124926640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512015ee30 0x7f51200c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.529+0000 7f5124926640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512015ee30 0x7f51200c8f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42936/0 (socket says 172.21.15.27:42936) 2023-12-19T18:22:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.529+0000 7f5124926640 1 -- 172.21.15.27:0/3407905052 learned_addr learned my addr 172.21.15.27:0/3407905052 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.529+0000 7f5117fff640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51201553a0 0x7f51200b1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.529+0000 7f511ffff640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51200afeb0 0x7f5120157ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.529+0000 7f5124926640 1 -- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51200afeb0 msgr2=0x7f5120157ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.529+0000 7f5124926640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51200afeb0 0x7f5120157ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.529+0000 7f5124926640 1 -- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51201553a0 msgr2=0x7f51200b1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.529+0000 7f5124926640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51201553a0 0x7f51200b1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.529+0000 7f5124926640 1 -- 172.21.15.27:0/3407905052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5108009400 con 0x7f512015ee30 2023-12-19T18:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.530+0000 7f5124926640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512015ee30 0x7f51200c8f00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f511800c7e0 tx=0x7f511800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:18.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.530+0000 7f511d7fa640 1 -- 172.21.15.27:0/3407905052 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511800a640 con 0x7f512015ee30 2023-12-19T18:22:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.530+0000 7f511d7fa640 1 -- 172.21.15.27:0/3407905052 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511800a7c0 con 0x7f512015ee30 2023-12-19T18:22:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.530+0000 7f511d7fa640 1 -- 172.21.15.27:0/3407905052 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511800ab70 con 0x7f512015ee30 2023-12-19T18:22:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.530+0000 7f5125127640 1 -- 172.21.15.27:0/3407905052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51200ccb70 con 0x7f512015ee30 2023-12-19T18:22:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.530+0000 7f5125127640 1 -- 172.21.15.27:0/3407905052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5120158410 con 0x7f512015ee30 2023-12-19T18:22:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.531+0000 7f5125127640 1 -- 172.21.15.27:0/3407905052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51200cd330 con 0x7f512015ee30 2023-12-19T18:22:18.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.532+0000 7f511d7fa640 1 -- 172.21.15.27:0/3407905052 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f511800acd0 con 0x7f512015ee30 2023-12-19T18:22:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.533+0000 7f511d7fa640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f510006a460 0x7f510006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.534+0000 7f511d7fa640 1 -- 172.21.15.27:0/3407905052 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5118087e80 con 0x7f512015ee30 2023-12-19T18:22:18.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.534+0000 7f511ffff640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f510006a460 0x7f510006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.535+0000 7f511ffff640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f510006a460 0x7f510006c920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5110000f60 tx=0x7f5110009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:18.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.538+0000 7f511d7fa640 1 -- 172.21.15.27:0/3407905052 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f51180533d0 con 0x7f512015ee30 2023-12-19T18:22:18.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.732+0000 7f5125127640 1 -- 172.21.15.27:0/3407905052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f512004e840 con 0x7f512015ee30 2023-12-19T18:22:18.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.735+0000 7f511d7fa640 1 -- 172.21.15.27:0/3407905052 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v135) v1 ==== 74+0+16052 (secure 0 0 0) 0x7f5118018090 con 0x7f512015ee30 2023-12-19T18:22:18.734 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:18.735 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":135,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:22:15.220628+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:22:12.328518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T18:22:13.251708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"135","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:22:18.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.737+0000 7f5125127640 1 -- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f510006a460 msgr2=0x7f510006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.737+0000 7f5125127640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f510006a460 0x7f510006c920 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f5110000f60 tx=0x7f5110009040 comp rx=0 tx=0).stop 2023-12-19T18:22:18.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.737+0000 7f5125127640 1 -- 172.21.15.27:0/3407905052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512015ee30 msgr2=0x7f51200c8f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.737+0000 7f5125127640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512015ee30 0x7f51200c8f00 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f511800c7e0 tx=0x7f511800cca0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.738+0000 7f5125127640 1 -- 172.21.15.27:0/3407905052 shutdown_connections 2023-12-19T18:22:18.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.738+0000 7f5125127640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f510006a460 0x7f510006c920 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.738+0000 7f5125127640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f512015ee30 0x7f51200c8f00 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.738+0000 7f5125127640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51201553a0 0x7f51200b1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.738+0000 7f5125127640 1 --2- 172.21.15.27:0/3407905052 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51200afeb0 0x7f5120157ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.738+0000 7f5125127640 1 -- 172.21.15.27:0/3407905052 >> 172.21.15.27:0/3407905052 conn(0x7f5120069c70 msgr2=0x7f51200c4ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.738+0000 7f5125127640 1 -- 172.21.15.27:0/3407905052 shutdown_connections 2023-12-19T18:22:18.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.738+0000 7f5125127640 1 -- 172.21.15.27:0/3407905052 wait complete. 2023-12-19T18:22:18.758 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:22:18.759 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:22:18.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.988+0000 7fe3e628c640 1 Processor -- start 2023-12-19T18:22:18.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.988+0000 7fe3e628c640 1 -- start start 2023-12-19T18:22:18.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.990+0000 7fe3e628c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe3e015ee30 0x7fe3e00c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.990+0000 7fe3e628c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3e00afeb0 0x7fe3e00c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.990+0000 7fe3e628c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3e00cb350 con 0x7fe3e01553a0 2023-12-19T18:22:18.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.990+0000 7fe3e628c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3e00cbcf0 con 0x7fe3e015ee30 2023-12-19T18:22:18.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.990+0000 7fe3e628c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00cb630 con 0x7fe3e00afeb0 2023-12-19T18:22:18.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.991+0000 7fe3e528a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe3e01553a0 0x7fe3e015e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41266/0 (socket says 172.21.15.27:41266) 2023-12-19T18:22:18.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.991+0000 7fe3e528a640 1 -- 172.21.15.27:0/173126273 learned_addr learned my addr 172.21.15.27:0/173126273 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:18.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.991+0000 7fe3e5a8b640 1 --2- 172.21.15.27:0/173126273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3e00afeb0 0x7fe3e00c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.991+0000 7fe3e4a89640 1 --2- 172.21.15.27:0/173126273 >> v2:172.21.15.192:3300/0 conn(0x7fe3e015ee30 0x7fe3e00c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.991+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3812485509 0 0) 0x7fe3e00cb350 con 0x7fe3e01553a0 2023-12-19T18:22:18.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.991+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3cc003520 con 0x7fe3e01553a0 2023-12-19T18:22:18.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 313854165 0 0) 0x7fe3cc003520 con 0x7fe3e01553a0 2023-12-19T18:22:18.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3e00afeb0 msgr2=0x7fe3e00c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:18.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 --2- 172.21.15.27:0/173126273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3e00afeb0 0x7fe3e00c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 >> v2:172.21.15.192:3300/0 conn(0x7fe3e015ee30 msgr2=0x7fe3e00c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:18.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 --2- 172.21.15.27:0/173126273 >> v2:172.21.15.192:3300/0 conn(0x7fe3e015ee30 0x7fe3e00c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e00cb910 con 0x7fe3e01553a0 2023-12-19T18:22:18.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe3dc001670 con 0x7fe3e01553a0 2023-12-19T18:22:18.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 >> v1:172.21.15.27:6789/0 conn(0x7fe3e01553a0 legacy=0x7fe3e015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 --2- 172.21.15.27:0/173126273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc003970 0x7fe3cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 --2- 172.21.15.27:0/173126273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc0047c0 0x7fe3cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 --2- 172.21.15.27:0/173126273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3e00afeb0 0x7fe3cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00cbcf0 con 0x7fe3cc003970 2023-12-19T18:22:18.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00cb630 con 0x7fe3cc0047c0 2023-12-19T18:22:18.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc00fdc0 con 0x7fe3e00afeb0 2023-12-19T18:22:18.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3e528a640 1 --2- 172.21.15.27:0/173126273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc003970 0x7fe3cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.992+0000 7fe3e4a89640 1 --2- 172.21.15.27:0/173126273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc0047c0 0x7fe3cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.993+0000 7fe3e528a640 1 -- 172.21.15.27:0/173126273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3e00afeb0 msgr2=0x7fe3cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:18.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.993+0000 7fe3e528a640 1 --2- 172.21.15.27:0/173126273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3e00afeb0 0x7fe3cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.993+0000 7fe3e528a640 1 -- 172.21.15.27:0/173126273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc0047c0 msgr2=0x7fe3cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:18.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.993+0000 7fe3e528a640 1 --2- 172.21.15.27:0/173126273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc0047c0 0x7fe3cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.993+0000 7fe3e528a640 1 -- 172.21.15.27:0/173126273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc003520 con 0x7fe3cc003970 2023-12-19T18:22:18.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.994+0000 7fe3e528a640 1 --2- 172.21.15.27:0/173126273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc003970 0x7fe3cc003d90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00c240 tx=0x7fe3cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad0d4afbd33f872 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:18.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.994+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc02cae0 con 0x7fe3cc003970 2023-12-19T18:22:18.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.994+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3dc02cc40 con 0x7fe3cc003970 2023-12-19T18:22:18.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.994+0000 7fe3d7fff640 1 -- 172.21.15.27:0/173126273 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3dc0058d0 con 0x7fe3cc003970 2023-12-19T18:22:18.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.994+0000 7fe3e628c640 1 -- 172.21.15.27:0/173126273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc003970 msgr2=0x7fe3cc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:18.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.994+0000 7fe3e628c640 1 --2- 172.21.15.27:0/173126273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc003970 0x7fe3cc003d90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc00c240 tx=0x7fe3cc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:18.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.994+0000 7fe3e5a8b640 1 -- 172.21.15.27:0/173126273 reap_dead start 2023-12-19T18:22:18.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.995+0000 7fe3e628c640 1 -- 172.21.15.27:0/173126273 shutdown_connections 2023-12-19T18:22:18.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.995+0000 7fe3e628c640 1 -- 172.21.15.27:0/173126273 >> 172.21.15.27:0/173126273 conn(0x7fe3e0069c70 msgr2=0x7fe3e006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:18.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.995+0000 7fe3e628c640 1 -- 172.21.15.27:0/173126273 shutdown_connections 2023-12-19T18:22:18.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.995+0000 7fe3e628c640 1 -- 172.21.15.27:0/173126273 wait complete. 2023-12-19T18:22:18.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.997+0000 7fe3e628c640 1 Processor -- start 2023-12-19T18:22:18.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.997+0000 7fe3e628c640 1 -- start start 2023-12-19T18:22:18.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.998+0000 7fe3e628c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc003970 0x7fe3e015a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.998+0000 7fe3e628c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3e01500a0 0x7fe3e0158d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.998+0000 7fe3e628c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3e01592c0 0x7fe3e015c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:18.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.998+0000 7fe3e628c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e0159740 con 0x7fe3e01592c0 2023-12-19T18:22:18.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.998+0000 7fe3e628c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e015e460 con 0x7fe3cc003970 2023-12-19T18:22:18.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.998+0000 7fe3e628c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3e00cdc50 con 0x7fe3e01500a0 2023-12-19T18:22:18.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.998+0000 7fe3e5a8b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3e01592c0 0x7fe3e015c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.998+0000 7fe3e5a8b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3e01592c0 0x7fe3e015c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42966/0 (socket says 172.21.15.27:42966) 2023-12-19T18:22:18.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.998+0000 7fe3e5a8b640 1 -- 172.21.15.27:0/3724889978 learned_addr learned my addr 172.21.15.27:0/3724889978 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:18.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.998+0000 7fe3e528a640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc003970 0x7fe3e015a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:18.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.998+0000 7fe3e4a89640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3e01500a0 0x7fe3e0158d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:19.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.999+0000 7fe3e5a8b640 1 -- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3e01500a0 msgr2=0x7fe3e0158d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:19.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.999+0000 7fe3e5a8b640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3e01500a0 0x7fe3e0158d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.999+0000 7fe3e5a8b640 1 -- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc003970 msgr2=0x7fe3e015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:19.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.999+0000 7fe3e5a8b640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc003970 0x7fe3e015a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.999+0000 7fe3e5a8b640 1 -- 172.21.15.27:0/3724889978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e015e6b0 con 0x7fe3e01592c0 2023-12-19T18:22:19.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.999+0000 7fe3e4a89640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3e01500a0 0x7fe3e0158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:22:19.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.999+0000 7fe3e5a8b640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3e01592c0 0x7fe3e015c7a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0158d00 tx=0x7fe3d800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:19.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.999+0000 7fe3d5ffb640 1 -- 172.21.15.27:0/3724889978 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8010040 con 0x7fe3e01592c0 2023-12-19T18:22:19.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.999+0000 7fe3d5ffb640 1 -- 172.21.15.27:0/3724889978 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3d800bce0 con 0x7fe3e01592c0 2023-12-19T18:22:19.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.999+0000 7fe3d5ffb640 1 -- 172.21.15.27:0/3724889978 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3d8014640 con 0x7fe3e01592c0 2023-12-19T18:22:19.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:18.999+0000 7fe3e628c640 1 -- 172.21.15.27:0/3724889978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3e015e990 con 0x7fe3e01592c0 2023-12-19T18:22:19.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.000+0000 7fe3e628c640 1 -- 172.21.15.27:0/3724889978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3e014fde0 con 0x7fe3e01592c0 2023-12-19T18:22:19.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.001+0000 7fe3e628c640 1 -- 172.21.15.27:0/3724889978 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3e015ab90 con 0x7fe3e01592c0 2023-12-19T18:22:19.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.002+0000 7fe3d5ffb640 1 -- 172.21.15.27:0/3724889978 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe3d80147a0 con 0x7fe3e01592c0 2023-12-19T18:22:19.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.003+0000 7fe3d5ffb640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe3c406a460 0x7fe3c406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:19.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.003+0000 7fe3d5ffb640 1 -- 172.21.15.27:0/3724889978 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3d80890f0 con 0x7fe3e01592c0 2023-12-19T18:22:19.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.004+0000 7fe3e528a640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe3c406a460 0x7fe3c406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:19.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.005+0000 7fe3e528a640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe3c406a460 0x7fe3c406c920 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc034000 tx=0x7fe3dc007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:19.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.007+0000 7fe3d5ffb640 1 -- 172.21.15.27:0/3724889978 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe3d8054830 con 0x7fe3e01592c0 2023-12-19T18:22:19.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.197+0000 7fe3e628c640 1 -- 172.21.15.27:0/3724889978 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe3e0150480 con 0x7fe3c406a460 2023-12-19T18:22:19.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.220+0000 7fe3d5ffb640 1 -- 172.21.15.27:0/3724889978 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+280208 (secure 0 0 0) 0x7fe3e0150480 con 0x7fe3c406a460 2023-12-19T18:22:19.223 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:19.226 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:22:19.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.227+0000 7fe3e628c640 1 -- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe3c406a460 msgr2=0x7fe3c406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:19.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.227+0000 7fe3e628c640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe3c406a460 0x7fe3c406c920 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe3dc034000 tx=0x7fe3dc007000 comp rx=0 tx=0).stop 2023-12-19T18:22:19.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.227+0000 7fe3e628c640 1 -- 172.21.15.27:0/3724889978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3e01592c0 msgr2=0x7fe3e015c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:19.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.227+0000 7fe3e628c640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3e01592c0 0x7fe3e015c7a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fe3e0158d00 tx=0x7fe3d800b480 comp rx=0 tx=0).stop 2023-12-19T18:22:19.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.227+0000 7fe3e628c640 1 -- 172.21.15.27:0/3724889978 shutdown_connections 2023-12-19T18:22:19.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.227+0000 7fe3e628c640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe3c406a460 0x7fe3c406c920 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.227+0000 7fe3e628c640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3e01592c0 0x7fe3e015c7a0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.227+0000 7fe3e628c640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3e01500a0 0x7fe3e0158d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.227+0000 7fe3e628c640 1 --2- 172.21.15.27:0/3724889978 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc003970 0x7fe3e015a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.227+0000 7fe3e628c640 1 -- 172.21.15.27:0/3724889978 >> 172.21.15.27:0/3724889978 conn(0x7fe3e0069c70 msgr2=0x7fe3e0151490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:19.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.227+0000 7fe3e628c640 1 -- 172.21.15.27:0/3724889978 shutdown_connections 2023-12-19T18:22:19.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.228+0000 7fe3e628c640 1 -- 172.21.15.27:0/3724889978 wait complete. 2023-12-19T18:22:19.246 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2023-12-19T18:22:17.865945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6888,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555611,"statfs":{"total":1600000000,"available":1592945936,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.171043"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.726129+0000","last_change":"2023-12-19T18:22:13.726129+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.726129+0000","last_undegraded":"2023-12-19T18:22:13.726129+0000","last_fullsized":"2023-12-19T18:22:13.726129+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:40.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.726390+0000","last_change":"2023-12-19T18:22:13.726390+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.726390+0000","last_undegraded":"2023-12-19T18:22:13.726390+0000","last_fullsized":"2023-12-19T18:22:13.726390+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:15.726948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.709168+0000","last_change":"2023-12-19T18:22:13.709168+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.709168+0000","last_undegraded":"2023-12-19T18:22:13.709168+0000","last_fullsized":"2023-12-19T18:22:13.709168+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:16.333492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.718039+0000","last_change":"2023-12-19T18:22:13.718039+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.718039+0000","last_undegraded":"2023-12-19T18:22:13.718039+0000","last_fullsized":"2023-12-19T18:22:13.718039+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:09.942336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.723318+0000","last_change":"2023-12-19T18:22:13.723318+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.723318+0000","last_undegraded":"2023-12-19T18:22:13.723318+0000","last_fullsized":"2023-12-19T18:22:13.723318+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:26.506333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.725569+0000","last_change":"2023-12-19T18:22:13.725569+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.725569+0000","last_undegraded":"2023-12-19T18:22:13.725569+0000","last_fullsized":"2023-12-19T18:22:13.725569+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:02:55.424114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.722057+0000","last_change":"2023-12-19T18:22:13.722057+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.722057+0000","last_undegraded":"2023-12-19T18:22:13.722057+0000","last_fullsized":"2023-12-19T18:22:13.722057+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:37:37.391952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.708366+0000","last_change":"2023-12-19T18:22:13.708366+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.708366+0000","last_undegraded":"2023-12-19T18:22:13.708366+0000","last_fullsized":"2023-12-19T18:22:13.708366+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:23.915911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.723598+0000","last_change":"2023-12-19T18:22:13.723598+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.723598+0000","last_undegraded":"2023-12-19T18:22:13.723598+0000","last_fullsized":"2023-12-19T18:22:13.723598+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:29.637020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.721388+0000","last_change":"2023-12-19T18:22:13.721388+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.721388+0000","last_undegraded":"2023-12-19T18:22:13.721388+0000","last_fullsized":"2023-12-19T18:22:13.721388+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:11:57.860995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.723865+0000","last_change":"2023-12-19T18:22:13.723865+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.723865+0000","last_undegraded":"2023-12-19T18:22:13.723865+0000","last_fullsized":"2023-12-19T18:22:13.723865+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:23.903727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716942+0000","last_change":"2023-12-19T18:22:13.716942+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716942+0000","last_undegraded":"2023-12-19T18:22:13.716942+0000","last_fullsized":"2023-12-19T18:22:13.716942+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:31.105594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.719484+0000","last_change":"2023-12-19T18:22:13.719484+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.719484+0000","last_undegraded":"2023-12-19T18:22:13.719484+0000","last_fullsized":"2023-12-19T18:22:13.719484+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:33.150825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716881+0000","last_change":"2023-12-19T18:22:13.716881+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716881+0000","last_undegraded":"2023-12-19T18:22:13.716881+0000","last_fullsized":"2023-12-19T18:22:13.716881+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:16.333492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.728029+0000","last_change":"2023-12-19T18:22:13.728029+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.728029+0000","last_undegraded":"2023-12-19T18:22:13.728029+0000","last_fullsized":"2023-12-19T18:22:13.728029+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:34.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.722342+0000","last_change":"2023-12-19T18:22:13.722342+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.722342+0000","last_undegraded":"2023-12-19T18:22:13.722342+0000","last_fullsized":"2023-12-19T18:22:13.722342+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:02.299174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.722109+0000","last_change":"2023-12-19T18:22:13.722109+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.722109+0000","last_undegraded":"2023-12-19T18:22:13.722109+0000","last_fullsized":"2023-12-19T18:22:13.722109+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:41:21.616217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.718271+0000","last_change":"2023-12-19T18:22:13.718271+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.718271+0000","last_undegraded":"2023-12-19T18:22:13.718271+0000","last_fullsized":"2023-12-19T18:22:13.718271+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:44.388621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.718310+0000","last_change":"2023-12-19T18:22:13.718310+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.718310+0000","last_undegraded":"2023-12-19T18:22:13.718310+0000","last_fullsized":"2023-12-19T18:22:13.718310+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:41:30.585976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.717068+0000","last_change":"2023-12-19T18:22:13.717068+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.717068+0000","last_undegraded":"2023-12-19T18:22:13.717068+0000","last_fullsized":"2023-12-19T18:22:13.717068+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:40.700106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.711531+0000","last_change":"2023-12-19T18:22:13.711531+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.711531+0000","last_undegraded":"2023-12-19T18:22:13.711531+0000","last_fullsized":"2023-12-19T18:22:13.711531+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:11.523949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.708779+0000","last_change":"2023-12-19T18:22:13.708779+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.708779+0000","last_undegraded":"2023-12-19T18:22:13.708779+0000","last_fullsized":"2023-12-19T18:22:13.708779+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:13:15.250452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.717448+0000","last_change":"2023-12-19T18:22:13.717448+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.717448+0000","last_undegraded":"2023-12-19T18:22:13.717448+0000","last_fullsized":"2023-12-19T18:22:13.717448+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:08.465323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.708091+0000","last_change":"2023-12-19T18:22:13.708091+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.708091+0000","last_undegraded":"2023-12-19T18:22:13.708091+0000","last_fullsized":"2023-12-19T18:22:13.708091+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:02:16.250351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.720837+0000","last_change":"2023-12-19T18:22:13.720837+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.720837+0000","last_undegraded":"2023-12-19T18:22:13.720837+0000","last_fullsized":"2023-12-19T18:22:13.720837+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:32.485695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.715635+0000","last_change":"2023-12-19T18:22:13.715635+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.715635+0000","last_undegraded":"2023-12-19T18:22:13.715635+0000","last_fullsized":"2023-12-19T18:22:13.715635+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:08.366785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.707470+0000","last_change":"2023-12-19T18:22:13.707470+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.707470+0000","last_undegraded":"2023-12-19T18:22:13.707470+0000","last_fullsized":"2023-12-19T18:22:13.707470+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:57:01.585693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.707503+0000","last_change":"2023-12-19T18:22:13.707503+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.707503+0000","last_undegraded":"2023-12-19T18:22:13.707503+0000","last_fullsized":"2023-12-19T18:22:13.707503+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:01:59.548259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.714779+0000","last_change":"2023-12-19T18:22:13.714779+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.714779+0000","last_undegraded":"2023-12-19T18:22:13.714779+0000","last_fullsized":"2023-12-19T18:22:13.714779+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:32.485695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.721121+0000","last_change":"2023-12-19T18:22:13.721121+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.721121+0000","last_undegraded":"2023-12-19T18:22:13.721121+0000","last_fullsized":"2023-12-19T18:22:13.721121+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:47:09.595830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.710659+0000","last_change":"2023-12-19T18:22:13.710659+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.710659+0000","last_undegraded":"2023-12-19T18:22:13.710659+0000","last_fullsized":"2023-12-19T18:22:13.710659+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:01:36.203567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.724266+0000","last_change":"2023-12-19T18:22:13.724266+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.724266+0000","last_undegraded":"2023-12-19T18:22:13.724266+0000","last_fullsized":"2023-12-19T18:22:13.724266+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:16.254938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.717402+0000","last_change":"2023-12-19T18:22:13.717402+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.717402+0000","last_undegraded":"2023-12-19T18:22:13.717402+0000","last_fullsized":"2023-12-19T18:22:13.717402+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:33.697121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.735336+0000","last_change":"2023-12-19T18:22:13.732827+0000","last_active":"2023-12-19T18:22:13.735336+0000","last_peered":"2023-12-19T18:22:13.735336+0000","last_clean":"2023-12-19T18:22:13.735336+0000","last_became_active":"2023-12-19T18:22:13.732603+0000","last_became_peered":"2023-12-19T18:22:13.732603+0000","last_unstale":"2023-12-19T18:22:13.735336+0000","last_undegraded":"2023-12-19T18:22:13.735336+0000","last_fullsized":"2023-12-19T18:22:13.735336+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:41:12.315002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.718562+0000","last_change":"2023-12-19T18:22:13.718562+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.718562+0000","last_undegraded":"2023-12-19T18:22:13.718562+0000","last_fullsized":"2023-12-19T18:22:13.718562+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:39.384923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.732498+0000","last_change":"2023-12-19T18:22:13.731624+0000","last_active":"2023-12-19T18:22:13.732498+0000","last_peered":"2023-12-19T18:22:13.732498+0000","last_clean":"2023-12-19T18:22:13.732498+0000","last_became_active":"2023-12-19T18:22:13.731335+0000","last_became_peered":"2023-12-19T18:22:13.731335+0000","last_unstale":"2023-12-19T18:22:13.732498+0000","last_undegraded":"2023-12-19T18:22:13.732498+0000","last_fullsized":"2023-12-19T18:22:13.732498+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:53:16.279722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.725678+0000","last_change":"2023-12-19T18:22:13.725678+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.725678+0000","last_undegraded":"2023-12-19T18:22:13.725678+0000","last_fullsized":"2023-12-19T18:22:13.725678+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:16.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.733264+0000","last_change":"2023-12-19T18:22:13.733176+0000","last_active":"2023-12-19T18:22:13.733264+0000","last_peered":"2023-12-19T18:22:13.733264+0000","last_clean":"2023-12-19T18:22:13.733264+0000","last_became_active":"2023-12-19T18:22:13.732940+0000","last_became_peered":"2023-12-19T18:22:13.732940+0000","last_unstale":"2023-12-19T18:22:13.733264+0000","last_undegraded":"2023-12-19T18:22:13.733264+0000","last_fullsized":"2023-12-19T18:22:13.733264+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:30.091059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716091+0000","last_change":"2023-12-19T18:22:13.716091+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716091+0000","last_undegraded":"2023-12-19T18:22:13.716091+0000","last_fullsized":"2023-12-19T18:22:13.716091+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:39.384923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.733455+0000","last_change":"2023-12-19T18:22:13.733119+0000","last_active":"2023-12-19T18:22:13.733455+0000","last_peered":"2023-12-19T18:22:13.733455+0000","last_clean":"2023-12-19T18:22:13.733455+0000","last_became_active":"2023-12-19T18:22:13.732868+0000","last_became_peered":"2023-12-19T18:22:13.732868+0000","last_unstale":"2023-12-19T18:22:13.733455+0000","last_undegraded":"2023-12-19T18:22:13.733455+0000","last_fullsized":"2023-12-19T18:22:13.733455+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:07.352249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.717476+0000","last_change":"2023-12-19T18:22:13.717476+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.717476+0000","last_undegraded":"2023-12-19T18:22:13.717476+0000","last_fullsized":"2023-12-19T18:22:13.717476+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:53.745021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.731132+0000","last_change":"2023-12-19T18:22:13.730941+0000","last_active":"2023-12-19T18:22:13.731132+0000","last_peered":"2023-12-19T18:22:13.731132+0000","last_clean":"2023-12-19T18:22:13.731132+0000","last_became_active":"2023-12-19T18:22:13.730684+0000","last_became_peered":"2023-12-19T18:22:13.730684+0000","last_unstale":"2023-12-19T18:22:13.731132+0000","last_undegraded":"2023-12-19T18:22:13.731132+0000","last_fullsized":"2023-12-19T18:22:13.731132+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:43:06.634088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.722437+0000","last_change":"2023-12-19T18:22:13.722437+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.722437+0000","last_undegraded":"2023-12-19T18:22:13.722437+0000","last_fullsized":"2023-12-19T18:22:13.722437+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:41.584580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.732419+0000","last_change":"2023-12-19T18:22:13.731736+0000","last_active":"2023-12-19T18:22:13.732419+0000","last_peered":"2023-12-19T18:22:13.732419+0000","last_clean":"2023-12-19T18:22:13.732419+0000","last_became_active":"2023-12-19T18:22:13.731153+0000","last_became_peered":"2023-12-19T18:22:13.731153+0000","last_unstale":"2023-12-19T18:22:13.732419+0000","last_undegraded":"2023-12-19T18:22:13.732419+0000","last_fullsized":"2023-12-19T18:22:13.732419+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:52.730485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.720362+0000","last_change":"2023-12-19T18:22:13.720362+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.720362+0000","last_undegraded":"2023-12-19T18:22:13.720362+0000","last_fullsized":"2023-12-19T18:22:13.720362+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:11:57.860995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.730782+0000","last_change":"2023-12-19T18:22:13.730291+0000","last_active":"2023-12-19T18:22:13.730782+0000","last_peered":"2023-12-19T18:22:13.730782+0000","last_clean":"2023-12-19T18:22:13.730782+0000","last_became_active":"2023-12-19T18:22:13.730015+0000","last_became_peered":"2023-12-19T18:22:13.730015+0000","last_unstale":"2023-12-19T18:22:13.730782+0000","last_undegraded":"2023-12-19T18:22:13.730782+0000","last_fullsized":"2023-12-19T18:22:13.730782+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:02:54.040583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716117+0000","last_change":"2023-12-19T18:22:13.716117+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716117+0000","last_undegraded":"2023-12-19T18:22:13.716117+0000","last_fullsized":"2023-12-19T18:22:13.716117+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:45.553627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.732144+0000","last_change":"2023-12-19T18:22:13.730656+0000","last_active":"2023-12-19T18:22:13.732144+0000","last_peered":"2023-12-19T18:22:13.732144+0000","last_clean":"2023-12-19T18:22:13.732144+0000","last_became_active":"2023-12-19T18:22:13.730408+0000","last_became_peered":"2023-12-19T18:22:13.730408+0000","last_unstale":"2023-12-19T18:22:13.732144+0000","last_undegraded":"2023-12-19T18:22:13.732144+0000","last_fullsized":"2023-12-19T18:22:13.732144+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:33.562949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.725020+0000","last_change":"2023-12-19T18:22:13.725020+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.725020+0000","last_undegraded":"2023-12-19T18:22:13.725020+0000","last_fullsized":"2023-12-19T18:22:13.725020+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:13:15.250452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.724646+0000","last_change":"2023-12-19T18:22:13.724536+0000","last_active":"2023-12-19T18:22:13.724646+0000","last_peered":"2023-12-19T18:22:13.724646+0000","last_clean":"2023-12-19T18:22:13.724646+0000","last_became_active":"2023-12-19T18:22:13.724110+0000","last_became_peered":"2023-12-19T18:22:13.724110+0000","last_unstale":"2023-12-19T18:22:13.724646+0000","last_undegraded":"2023-12-19T18:22:13.724646+0000","last_fullsized":"2023-12-19T18:22:13.724646+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:44.539092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.706194+0000","last_change":"2023-12-19T18:22:13.706194+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.706194+0000","last_undegraded":"2023-12-19T18:22:13.706194+0000","last_fullsized":"2023-12-19T18:22:13.706194+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:15.726948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.733474+0000","last_change":"2023-12-19T18:22:13.733261+0000","last_active":"2023-12-19T18:22:13.733474+0000","last_peered":"2023-12-19T18:22:13.733474+0000","last_clean":"2023-12-19T18:22:13.733474+0000","last_became_active":"2023-12-19T18:22:13.733015+0000","last_became_peered":"2023-12-19T18:22:13.733015+0000","last_unstale":"2023-12-19T18:22:13.733474+0000","last_undegraded":"2023-12-19T18:22:13.733474+0000","last_fullsized":"2023-12-19T18:22:13.733474+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:52.730485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716780+0000","last_change":"2023-12-19T18:22:13.716780+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716780+0000","last_undegraded":"2023-12-19T18:22:13.716780+0000","last_fullsized":"2023-12-19T18:22:13.716780+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:32:56.103686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.728083+0000","last_change":"2023-12-19T18:22:13.727917+0000","last_active":"2023-12-19T18:22:13.728083+0000","last_peered":"2023-12-19T18:22:13.728083+0000","last_clean":"2023-12-19T18:22:13.728083+0000","last_became_active":"2023-12-19T18:22:13.727633+0000","last_became_peered":"2023-12-19T18:22:13.727633+0000","last_unstale":"2023-12-19T18:22:13.728083+0000","last_undegraded":"2023-12-19T18:22:13.728083+0000","last_fullsized":"2023-12-19T18:22:13.728083+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:36:19.046380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.706628+0000","last_change":"2023-12-19T18:22:13.706628+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.706628+0000","last_undegraded":"2023-12-19T18:22:13.706628+0000","last_fullsized":"2023-12-19T18:22:13.706628+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:40.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.725596+0000","last_change":"2023-12-19T18:22:13.725517+0000","last_active":"2023-12-19T18:22:13.725596+0000","last_peered":"2023-12-19T18:22:13.725596+0000","last_clean":"2023-12-19T18:22:13.725596+0000","last_became_active":"2023-12-19T18:22:13.724094+0000","last_became_peered":"2023-12-19T18:22:13.724094+0000","last_unstale":"2023-12-19T18:22:13.725596+0000","last_undegraded":"2023-12-19T18:22:13.725596+0000","last_fullsized":"2023-12-19T18:22:13.725596+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:29:20.466956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.724994+0000","last_change":"2023-12-19T18:22:13.724994+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.724994+0000","last_undegraded":"2023-12-19T18:22:13.724994+0000","last_fullsized":"2023-12-19T18:22:13.724994+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:37.118051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.732124+0000","last_change":"2023-12-19T18:22:13.730064+0000","last_active":"2023-12-19T18:22:13.732124+0000","last_peered":"2023-12-19T18:22:13.732124+0000","last_clean":"2023-12-19T18:22:13.732124+0000","last_became_active":"2023-12-19T18:22:13.729780+0000","last_became_peered":"2023-12-19T18:22:13.729780+0000","last_unstale":"2023-12-19T18:22:13.732124+0000","last_undegraded":"2023-12-19T18:22:13.732124+0000","last_fullsized":"2023-12-19T18:22:13.732124+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:14.712413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.710003+0000","last_change":"2023-12-19T18:22:13.710003+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.710003+0000","last_undegraded":"2023-12-19T18:22:13.710003+0000","last_fullsized":"2023-12-19T18:22:13.710003+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:16.254938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.733437+0000","last_change":"2023-12-19T18:22:13.732632+0000","last_active":"2023-12-19T18:22:13.733437+0000","last_peered":"2023-12-19T18:22:13.733437+0000","last_clean":"2023-12-19T18:22:13.733437+0000","last_became_active":"2023-12-19T18:22:13.732365+0000","last_became_peered":"2023-12-19T18:22:13.732365+0000","last_unstale":"2023-12-19T18:22:13.733437+0000","last_undegraded":"2023-12-19T18:22:13.733437+0000","last_fullsized":"2023-12-19T18:22:13.733437+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.720885+0000","last_change":"2023-12-19T18:22:13.720885+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.720885+0000","last_undegraded":"2023-12-19T18:22:13.720885+0000","last_fullsized":"2023-12-19T18:22:13.720885+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:06:47.768050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.731449+0000","last_change":"2023-12-19T18:22:13.729287+0000","last_active":"2023-12-19T18:22:13.731449+0000","last_peered":"2023-12-19T18:22:13.731449+0000","last_clean":"2023-12-19T18:22:13.731449+0000","last_became_active":"2023-12-19T18:22:13.725118+0000","last_became_peered":"2023-12-19T18:22:13.725118+0000","last_unstale":"2023-12-19T18:22:13.731449+0000","last_undegraded":"2023-12-19T18:22:13.731449+0000","last_fullsized":"2023-12-19T18:22:13.731449+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:13:44.939138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.707867+0000","last_change":"2023-12-19T18:22:13.707867+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.707867+0000","last_undegraded":"2023-12-19T18:22:13.707867+0000","last_fullsized":"2023-12-19T18:22:13.707867+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:08.465323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.735138+0000","last_change":"2023-12-19T18:22:13.734908+0000","last_active":"2023-12-19T18:22:13.735138+0000","last_peered":"2023-12-19T18:22:13.735138+0000","last_clean":"2023-12-19T18:22:13.735138+0000","last_became_active":"2023-12-19T18:22:13.734666+0000","last_became_peered":"2023-12-19T18:22:13.734666+0000","last_unstale":"2023-12-19T18:22:13.735138+0000","last_undegraded":"2023-12-19T18:22:13.735138+0000","last_fullsized":"2023-12-19T18:22:13.735138+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:44.539092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.707148+0000","last_change":"2023-12-19T18:22:13.707148+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.707148+0000","last_undegraded":"2023-12-19T18:22:13.707148+0000","last_fullsized":"2023-12-19T18:22:13.707148+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:12.207051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.724869+0000","last_change":"2023-12-19T18:22:13.724787+0000","last_active":"2023-12-19T18:22:13.724869+0000","last_peered":"2023-12-19T18:22:13.724869+0000","last_clean":"2023-12-19T18:22:13.724869+0000","last_became_active":"2023-12-19T18:22:13.724547+0000","last_became_peered":"2023-12-19T18:22:13.724547+0000","last_unstale":"2023-12-19T18:22:13.724869+0000","last_undegraded":"2023-12-19T18:22:13.724869+0000","last_fullsized":"2023-12-19T18:22:13.724869+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:11.095769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.706495+0000","last_change":"2023-12-19T18:22:13.706495+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.706495+0000","last_undegraded":"2023-12-19T18:22:13.706495+0000","last_fullsized":"2023-12-19T18:22:13.706495+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:34.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.730755+0000","last_change":"2023-12-19T18:22:13.728901+0000","last_active":"2023-12-19T18:22:13.730755+0000","last_peered":"2023-12-19T18:22:13.730755+0000","last_clean":"2023-12-19T18:22:13.730755+0000","last_became_active":"2023-12-19T18:22:13.728001+0000","last_became_peered":"2023-12-19T18:22:13.728001+0000","last_unstale":"2023-12-19T18:22:13.730755+0000","last_undegraded":"2023-12-19T18:22:13.730755+0000","last_fullsized":"2023-12-19T18:22:13.730755+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:59.426392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716530+0000","last_change":"2023-12-19T18:22:13.716530+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716530+0000","last_undegraded":"2023-12-19T18:22:13.716530+0000","last_fullsized":"2023-12-19T18:22:13.716530+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:01.033125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.729108+0000","last_change":"2023-12-19T18:22:13.729021+0000","last_active":"2023-12-19T18:22:13.729108+0000","last_peered":"2023-12-19T18:22:13.729108+0000","last_clean":"2023-12-19T18:22:13.729108+0000","last_became_active":"2023-12-19T18:22:13.728792+0000","last_became_peered":"2023-12-19T18:22:13.728792+0000","last_unstale":"2023-12-19T18:22:13.729108+0000","last_undegraded":"2023-12-19T18:22:13.729108+0000","last_fullsized":"2023-12-19T18:22:13.729108+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:39.606223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716582+0000","last_change":"2023-12-19T18:22:13.716582+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716582+0000","last_undegraded":"2023-12-19T18:22:13.716582+0000","last_fullsized":"2023-12-19T18:22:13.716582+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:50:35.183653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.734645+0000","last_change":"2023-12-19T18:22:13.734402+0000","last_active":"2023-12-19T18:22:13.734645+0000","last_peered":"2023-12-19T18:22:13.734645+0000","last_clean":"2023-12-19T18:22:13.734645+0000","last_became_active":"2023-12-19T18:22:13.733935+0000","last_became_peered":"2023-12-19T18:22:13.733935+0000","last_unstale":"2023-12-19T18:22:13.734645+0000","last_undegraded":"2023-12-19T18:22:13.734645+0000","last_fullsized":"2023-12-19T18:22:13.734645+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:30.091059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.719693+0000","last_change":"2023-12-19T18:22:13.719693+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.719693+0000","last_undegraded":"2023-12-19T18:22:13.719693+0000","last_fullsized":"2023-12-19T18:22:13.719693+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:29.507646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.730782+0000","last_change":"2023-12-19T18:22:13.730665+0000","last_active":"2023-12-19T18:22:13.730782+0000","last_peered":"2023-12-19T18:22:13.730782+0000","last_clean":"2023-12-19T18:22:13.730782+0000","last_became_active":"2023-12-19T18:22:13.729630+0000","last_became_peered":"2023-12-19T18:22:13.729630+0000","last_unstale":"2023-12-19T18:22:13.730782+0000","last_undegraded":"2023-12-19T18:22:13.730782+0000","last_fullsized":"2023-12-19T18:22:13.730782+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:01:35.189032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.724524+0000","last_change":"2023-12-19T18:22:13.724524+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.724524+0000","last_undegraded":"2023-12-19T18:22:13.724524+0000","last_fullsized":"2023-12-19T18:22:13.724524+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:12.207051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.732244+0000","last_change":"2023-12-19T18:22:13.731257+0000","last_active":"2023-12-19T18:22:13.732244+0000","last_peered":"2023-12-19T18:22:13.732244+0000","last_clean":"2023-12-19T18:22:13.732244+0000","last_became_active":"2023-12-19T18:22:13.730910+0000","last_became_peered":"2023-12-19T18:22:13.730910+0000","last_unstale":"2023-12-19T18:22:13.732244+0000","last_undegraded":"2023-12-19T18:22:13.732244+0000","last_fullsized":"2023-12-19T18:22:13.732244+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.708423+0000","last_change":"2023-12-19T18:22:13.708423+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.708423+0000","last_undegraded":"2023-12-19T18:22:13.708423+0000","last_fullsized":"2023-12-19T18:22:13.708423+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:44.388621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.716654+0000","last_change":"2023-12-19T18:22:13.714716+0000","last_active":"2023-12-19T18:22:13.716654+0000","last_peered":"2023-12-19T18:22:13.716654+0000","last_clean":"2023-12-19T18:22:13.716654+0000","last_became_active":"2023-12-19T18:22:13.713977+0000","last_became_peered":"2023-12-19T18:22:13.713977+0000","last_unstale":"2023-12-19T18:22:13.716654+0000","last_undegraded":"2023-12-19T18:22:13.716654+0000","last_fullsized":"2023-12-19T18:22:13.716654+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.718774+0000","last_change":"2023-12-19T18:22:13.718774+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.718774+0000","last_undegraded":"2023-12-19T18:22:13.718774+0000","last_fullsized":"2023-12-19T18:22:13.718774+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:45:23.784185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.731485+0000","last_change":"2023-12-19T18:22:13.729500+0000","last_active":"2023-12-19T18:22:13.731485+0000","last_peered":"2023-12-19T18:22:13.731485+0000","last_clean":"2023-12-19T18:22:13.731485+0000","last_became_active":"2023-12-19T18:22:13.725715+0000","last_became_peered":"2023-12-19T18:22:13.725715+0000","last_unstale":"2023-12-19T18:22:13.731485+0000","last_undegraded":"2023-12-19T18:22:13.731485+0000","last_fullsized":"2023-12-19T18:22:13.731485+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:30:23.979686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.721429+0000","last_change":"2023-12-19T18:22:13.721429+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.721429+0000","last_undegraded":"2023-12-19T18:22:13.721429+0000","last_fullsized":"2023-12-19T18:22:13.721429+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:43:39.731660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.726875+0000","last_change":"2023-12-19T18:22:13.726768+0000","last_active":"2023-12-19T18:22:13.726875+0000","last_peered":"2023-12-19T18:22:13.726875+0000","last_clean":"2023-12-19T18:22:13.726875+0000","last_became_active":"2023-12-19T18:22:13.722319+0000","last_became_peered":"2023-12-19T18:22:13.722319+0000","last_unstale":"2023-12-19T18:22:13.726875+0000","last_undegraded":"2023-12-19T18:22:13.726875+0000","last_fullsized":"2023-12-19T18:22:13.726875+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:11.192516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.707706+0000","last_change":"2023-12-19T18:22:13.707706+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.707706+0000","last_undegraded":"2023-12-19T18:22:13.707706+0000","last_fullsized":"2023-12-19T18:22:13.707706+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:09.942336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.713603+0000","last_change":"2023-12-19T18:22:13.713351+0000","last_active":"2023-12-19T18:22:13.713603+0000","last_peered":"2023-12-19T18:22:13.713603+0000","last_clean":"2023-12-19T18:22:13.713603+0000","last_became_active":"2023-12-19T18:22:13.712581+0000","last_became_peered":"2023-12-19T18:22:13.712581+0000","last_unstale":"2023-12-19T18:22:13.713603+0000","last_undegraded":"2023-12-19T18:22:13.713603+0000","last_fullsized":"2023-12-19T18:22:13.713603+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:36.103516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.714719+0000","last_change":"2023-12-19T18:22:13.714719+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.714719+0000","last_undegraded":"2023-12-19T18:22:13.714719+0000","last_fullsized":"2023-12-19T18:22:13.714719+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:08.710594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.720004+0000","last_change":"2023-12-19T18:22:13.718488+0000","last_active":"2023-12-19T18:22:13.720004+0000","last_peered":"2023-12-19T18:22:13.720004+0000","last_clean":"2023-12-19T18:22:13.720004+0000","last_became_active":"2023-12-19T18:22:13.717351+0000","last_became_peered":"2023-12-19T18:22:13.717351+0000","last_unstale":"2023-12-19T18:22:13.720004+0000","last_undegraded":"2023-12-19T18:22:13.720004+0000","last_fullsized":"2023-12-19T18:22:13.720004+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:49:11.599052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.717816+0000","last_change":"2023-12-19T18:22:13.717816+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.717816+0000","last_undegraded":"2023-12-19T18:22:13.717816+0000","last_fullsized":"2023-12-19T18:22:13.717816+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:24.011514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.733864+0000","last_change":"2023-12-19T18:22:13.733542+0000","last_active":"2023-12-19T18:22:13.733864+0000","last_peered":"2023-12-19T18:22:13.733864+0000","last_clean":"2023-12-19T18:22:13.733864+0000","last_became_active":"2023-12-19T18:22:13.733273+0000","last_became_peered":"2023-12-19T18:22:13.733273+0000","last_unstale":"2023-12-19T18:22:13.733864+0000","last_undegraded":"2023-12-19T18:22:13.733864+0000","last_fullsized":"2023-12-19T18:22:13.733864+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:07.352249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.721501+0000","last_change":"2023-12-19T18:22:13.721501+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.721501+0000","last_undegraded":"2023-12-19T18:22:13.721501+0000","last_fullsized":"2023-12-19T18:22:13.721501+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:08.470720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.723182+0000","last_change":"2023-12-19T18:22:13.723012+0000","last_active":"2023-12-19T18:22:13.723182+0000","last_peered":"2023-12-19T18:22:13.723182+0000","last_clean":"2023-12-19T18:22:13.723182+0000","last_became_active":"2023-12-19T18:22:13.717308+0000","last_became_peered":"2023-12-19T18:22:13.717308+0000","last_unstale":"2023-12-19T18:22:13.723182+0000","last_undegraded":"2023-12-19T18:22:13.723182+0000","last_fullsized":"2023-12-19T18:22:13.723182+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:01.277914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.715269+0000","last_change":"2023-12-19T18:22:13.715269+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.715269+0000","last_undegraded":"2023-12-19T18:22:13.715269+0000","last_fullsized":"2023-12-19T18:22:13.715269+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:30:01.499864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.736548+0000","last_change":"2023-12-19T18:22:13.736355+0000","last_active":"2023-12-19T18:22:13.736548+0000","last_peered":"2023-12-19T18:22:13.736548+0000","last_clean":"2023-12-19T18:22:13.736548+0000","last_became_active":"2023-12-19T18:22:13.736056+0000","last_became_peered":"2023-12-19T18:22:13.736056+0000","last_unstale":"2023-12-19T18:22:13.736548+0000","last_undegraded":"2023-12-19T18:22:13.736548+0000","last_fullsized":"2023-12-19T18:22:13.736548+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:47.766382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716628+0000","last_change":"2023-12-19T18:22:13.716628+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716628+0000","last_undegraded":"2023-12-19T18:22:13.716628+0000","last_fullsized":"2023-12-19T18:22:13.716628+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:08.366785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.716737+0000","last_change":"2023-12-19T18:22:13.715051+0000","last_active":"2023-12-19T18:22:13.716737+0000","last_peered":"2023-12-19T18:22:13.716737+0000","last_clean":"2023-12-19T18:22:13.716737+0000","last_became_active":"2023-12-19T18:22:13.714392+0000","last_became_peered":"2023-12-19T18:22:13.714392+0000","last_unstale":"2023-12-19T18:22:13.716737+0000","last_undegraded":"2023-12-19T18:22:13.716737+0000","last_fullsized":"2023-12-19T18:22:13.716737+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:00.018590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.708590+0000","last_change":"2023-12-19T18:22:13.708590+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.708590+0000","last_undegraded":"2023-12-19T18:22:13.708590+0000","last_fullsized":"2023-12-19T18:22:13.708590+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:16.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.726584+0000","last_change":"2023-12-19T18:22:13.725717+0000","last_active":"2023-12-19T18:22:13.726584+0000","last_peered":"2023-12-19T18:22:13.726584+0000","last_clean":"2023-12-19T18:22:13.726584+0000","last_became_active":"2023-12-19T18:22:13.724435+0000","last_became_peered":"2023-12-19T18:22:13.724435+0000","last_unstale":"2023-12-19T18:22:13.726584+0000","last_undegraded":"2023-12-19T18:22:13.726584+0000","last_fullsized":"2023-12-19T18:22:13.726584+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:00:05.094962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":265,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-12-19T18:22:12.698399+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:22:12.698399+0000","last_peered":"2023-12-19T18:22:12.698399+0000","last_clean":"2023-12-19T18:22:12.698399+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:22:12.698399+0000","last_undegraded":"2023-12-19T18:22:12.698399+0000","last_fullsized":"2023-12-19T18:22:12.698399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2310000000000001,"15min":1.1519999999999999},"min":{"1min":0.57899999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9630000000000001,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.272,"5min":1.278,"15min":1.1890000000000001},"min":{"1min":0.76900000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6990000000000001,"5min":1.79,"15min":1.79},"last":1.4219999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.3480000000000001,"15min":1.23},"min":{"1min":0.752,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.7010000000000001,"5min":10.926,"15min":10.926},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.365,"15min":1.244},"min":{"1min":0.80900000000000005,"5min":0.71899999999999997,"15min":0.67900000000000005},"max":{"1min":1.6240000000000001,"5min":10.904999999999999,"15min":10.904999999999999},"last":1.6579999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.161,"15min":1.1040000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.427,"5min":3.1960000000000002,"15min":3.1960000000000002},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.266,"15min":1.1930000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0670000000000002,"5min":6.008,"15min":6.008},"last":1.226}]},{"osd":3,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.28,"15min":1.1619999999999999},"min":{"1min":0.66900000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.595,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4179999999999999,"15min":1.294},"min":{"1min":0.85699999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.157,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":0.99199999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.258,"15min":1.123},"min":{"1min":0.88600000000000001,"5min":0.65400000000000003,"15min":0.47899999999999998},"max":{"1min":1.8009999999999999,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.304,"15min":1.2410000000000001},"min":{"1min":0.94599999999999995,"5min":0.72999999999999998,"15min":0.57599999999999996},"max":{"1min":1.6759999999999999,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.369}]},{"osd":5,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.399,"15min":1.333},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.55000000000000004},"max":{"1min":1.8620000000000001,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":1.141},{"interface":"front","average":{"1min":1.385,"5min":1.4339999999999999,"15min":1.337},"min":{"1min":0.80700000000000005,"5min":0.71599999999999997,"15min":0.53500000000000003},"max":{"1min":2.1179999999999999,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":1.8280000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.5669999999999999,"15min":1.4690000000000001},"min":{"1min":1.036,"5min":0.78600000000000003,"15min":0.59899999999999998},"max":{"1min":2.423,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.742},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.49,"15min":1.3799999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.63200000000000001},"max":{"1min":2.0179999999999998,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":1.3280000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.121,"15min":1.012},"min":{"1min":0.79800000000000004,"5min":0.66700000000000004,"15min":0.65800000000000003},"max":{"1min":1.6259999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.751},{"interface":"front","average":{"1min":1.163,"5min":1.1870000000000001,"15min":1.075},"min":{"1min":0.78400000000000003,"5min":0.64300000000000002,"15min":0.55600000000000005},"max":{"1min":1.663,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":1.1100000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.1699999999999999,"15min":1.1319999999999999},"min":{"1min":0.86599999999999999,"5min":0.71399999999999997,"15min":0.68200000000000005},"max":{"1min":1.5609999999999999,"5min":4.5540000000000003,"15min":4.5540000000000003},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.226,"15min":1.1699999999999999},"min":{"1min":0.86599999999999999,"5min":0.625,"15min":0.60299999999999998},"max":{"1min":1.627,"5min":1.885,"15min":1.885},"last":1.3220000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.254,"15min":1.151},"min":{"1min":0.84799999999999998,"5min":0.626,"15min":0.626},"max":{"1min":1.583,"5min":1.821,"15min":1.821},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.218,"15min":1.2310000000000001},"min":{"1min":0.83599999999999997,"5min":0.81999999999999995,"15min":0.79900000000000004},"max":{"1min":1.821,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":0.91500000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.294,"15min":1.202},"min":{"1min":1.0129999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.722,"5min":3.8820000000000001,"15min":3.8820000000000001},"last":1.024},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2170000000000001,"15min":1.1699999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6559999999999999,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.94999999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.296,"15min":1.2450000000000001},"min":{"1min":0.74099999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6299999999999999,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.458},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.431,"15min":1.347},"min":{"1min":0.76700000000000002,"5min":0.70799999999999996,"15min":0.57999999999999996},"max":{"1min":1.774,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.367}]},{"osd":5,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2829999999999999,"15min":1.1879999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7330000000000001,"5min":4.6680000000000001,"15min":4.6680000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.4450000000000001,"15min":1.3560000000000001},"min":{"1min":0.84799999999999998,"5min":0.626,"15min":0.626},"max":{"1min":4.1159999999999997,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.405}]},{"osd":7,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4430000000000001,"15min":1.349},"min":{"1min":0.92700000000000005,"5min":0.82199999999999995,"15min":0.61899999999999999},"max":{"1min":1.7010000000000001,"5min":3.9670000000000001,"15min":3.9670000000000001},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.4470000000000001,"15min":1.357},"min":{"1min":0.81899999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8169999999999999,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":1.4370000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1293,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194019,"statfs":{"total":200000000,"available":198675466,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.23,"15min":1.1679999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.65200000000000002},"max":{"1min":1.6830000000000001,"5min":1.837,"15min":1.863},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2729999999999999,"15min":1.1870000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.571,"5min":1.879,"15min":1.879},"last":1.4410000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2090000000000001,"15min":1.0629999999999999},"min":{"1min":0.76200000000000001,"5min":0.70899999999999996,"15min":0.65700000000000003},"max":{"1min":1.782,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.167},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1599999999999999,"15min":1.0660000000000001},"min":{"1min":0.81899999999999995,"5min":0.71299999999999997,"15min":0.61299999999999999},"max":{"1min":1.5640000000000001,"5min":1.6359999999999999,"15min":1.843},"last":1.089}]},{"osd":2,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.155,"15min":1.0860000000000001},"min":{"1min":0.81799999999999995,"5min":0.74299999999999999,"15min":0.59499999999999997},"max":{"1min":1.3069999999999999,"5min":1.794,"15min":1.794},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.258,"15min":1.147},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.65200000000000002},"max":{"1min":1.5920000000000001,"5min":2.036,"15min":2.036},"last":1.2589999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2010000000000001,"15min":1.097},"min":{"1min":0.83699999999999997,"5min":0.71999999999999997,"15min":0.68899999999999995},"max":{"1min":1.72,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3320000000000001,"15min":1.23},"min":{"1min":0.91300000000000003,"5min":0.75700000000000001,"15min":0.70399999999999996},"max":{"1min":1.595,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.3160000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3069999999999999,"15min":1.179},"min":{"1min":0.95299999999999996,"5min":0.60899999999999999,"15min":0.46700000000000003},"max":{"1min":1.6479999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.395},{"interface":"front","average":{"1min":1.25,"5min":1.1990000000000001,"15min":1.131},"min":{"1min":0.90500000000000003,"5min":0.73999999999999999,"15min":0.57099999999999995},"max":{"1min":1.7689999999999999,"5min":1.804,"15min":1.901},"last":1.0529999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4299999999999999,"15min":1.3360000000000001},"min":{"1min":1.054,"5min":0.66600000000000004,"15min":0.54400000000000004},"max":{"1min":1.839,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.518},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.319,"15min":1.2470000000000001},"min":{"1min":1.163,"5min":0.81200000000000006,"15min":0.49299999999999999},"max":{"1min":1.702,"5min":1.7949999999999999,"15min":1.976},"last":1.1399999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.274,"15min":1.2290000000000001},"min":{"1min":1.0189999999999999,"5min":0.68300000000000005,"15min":0.50900000000000001},"max":{"1min":1.6779999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.681},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4570000000000001,"15min":1.4119999999999999},"min":{"1min":1.2310000000000001,"5min":0.84999999999999998,"15min":0.623},"max":{"1min":1.8120000000000001,"5min":2.117,"15min":2.117},"last":1.2230000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98899999999999999,"15min":0.998},"min":{"1min":0.68500000000000005,"5min":0.62,"15min":0.62},"max":{"1min":1.3899999999999999,"5min":1.579,"15min":1.579},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0409999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4259999999999999,"5min":1.7470000000000001,"15min":1.7909999999999999},"last":1.5009999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1679999999999999,"15min":1.149},"min":{"1min":0.67200000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.589,"5min":1.7130000000000001,"15min":1.8160000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.29,"5min":1.2549999999999999,"15min":1.1850000000000001},"min":{"1min":0.77400000000000002,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.696},"last":0.81200000000000006}]},{"osd":2,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1499999999999999,"15min":1.127},"min":{"1min":0.82399999999999995,"5min":0.749,"15min":0.69799999999999995},"max":{"1min":1.407,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.1060000000000001},"min":{"1min":0.872,"5min":0.72199999999999998,"15min":0.70499999999999996},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.2010000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.294,"15min":1.26},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.3340000000000001,"15min":1.304},"min":{"1min":0.93100000000000005,"5min":0.82399999999999995,"15min":0.74399999999999999},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.369}]},{"osd":5,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.294,"15min":1.2030000000000001},"min":{"1min":0.90000000000000002,"5min":0.68600000000000005,"15min":0.44500000000000001},"max":{"1min":1.46,"5min":1.76,"15min":1.8740000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.413,"15min":1.353},"min":{"1min":0.76500000000000001,"5min":0.625,"15min":0.504},"max":{"1min":1.827,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.1160000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3160000000000001,"15min":1.304},"min":{"1min":0.98399999999999999,"5min":0.71299999999999997,"15min":0.432},"max":{"1min":1.5509999999999999,"5min":1.7809999999999999,"15min":1.853},"last":1.258},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4630000000000001,"15min":1.3959999999999999},"min":{"1min":1.2170000000000001,"5min":0.73599999999999999,"15min":0.64500000000000002},"max":{"1min":1.7829999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.3029999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.3999999999999999,"15min":1.3480000000000001},"min":{"1min":1.0249999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.7589999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4379999999999999,"15min":1.4079999999999999},"min":{"1min":0.88500000000000001,"5min":0.60399999999999998,"15min":0.57399999999999995},"max":{"1min":1.8,"5min":1.905,"15min":1.9330000000000001},"last":1.5880000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.52900000000000003,"5min":0.38100000000000001,"15min":0.35299999999999998},"max":{"1min":2.516,"5min":3.3140000000000001,"15min":3.4350000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1,"15min":0.94499999999999995},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.3490000000000002,"5min":4.3490000000000002,"15min":4.3490000000000002},"last":1.5860000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.0629999999999999,"15min":1.0620000000000001},"min":{"1min":0.45300000000000001,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":5.2510000000000003,"5min":5.2510000000000003,"15min":5.2510000000000003},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.403,"5min":1.1719999999999999,"15min":1.1160000000000001},"min":{"1min":0.63800000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":5.4119999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.1299999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.377,"15min":1.25},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.335,"5min":13.833,"15min":13.833},"last":1.232},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5349999999999999,"15min":1.306},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.53200000000000003},"max":{"1min":5.4480000000000004,"5min":16.748999999999999,"15min":16.748999999999999},"last":1.5369999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.3620000000000001,"15min":1.2330000000000001},"min":{"1min":0.81899999999999995,"5min":0.77400000000000002,"15min":0.65600000000000003},"max":{"1min":5.1790000000000003,"5min":5.1790000000000003,"15min":5.1790000000000003},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.591,"5min":1.4239999999999999,"15min":1.409},"min":{"1min":0.873,"5min":0.80800000000000005,"15min":0.73199999999999998},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":1.427}]},{"osd":5,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.4750000000000001,"15min":1.45},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.76000000000000001},"max":{"1min":5.3289999999999997,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":1.371},{"interface":"front","average":{"1min":1.46,"5min":1.367,"15min":1.2789999999999999},"min":{"1min":0.83799999999999997,"5min":0.80600000000000005,"15min":0.69999999999999996},"max":{"1min":4.6369999999999996,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.323}]},{"osd":6,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.4630000000000001,"15min":1.476},"min":{"1min":0.81999999999999995,"5min":0.81799999999999995,"15min":0.81599999999999995},"max":{"1min":4.5039999999999996,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":1.502},{"interface":"front","average":{"1min":1.667,"5min":1.548,"15min":1.514},"min":{"1min":0.83999999999999997,"5min":0.83199999999999996,"15min":0.80500000000000005},"max":{"1min":5.3739999999999997,"5min":5.3739999999999997,"15min":5.3739999999999997},"last":1.6759999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.476,"15min":1.385},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.73199999999999998},"max":{"1min":5.1349999999999998,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.712,"5min":1.585,"15min":1.554},"min":{"1min":0.92200000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":4.4619999999999997,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":1.6279999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.075,"15min":1.0109999999999999},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.34399999999999997},"max":{"1min":1.506,"5min":9.2550000000000008,"15min":9.2550000000000008},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0409999999999999,"15min":1.0389999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.7649999999999999,"5min":2.387,"15min":4.7850000000000001},"last":0.69499999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95999999999999996,"15min":1.165},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.46000000000000002},"max":{"1min":1.675,"5min":2.4390000000000001,"15min":8.2720000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.107,"15min":1.1619999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6519999999999999,"5min":2.7200000000000002,"15min":5.0620000000000003},"last":0.76500000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1830000000000001,"15min":1.3620000000000001},"min":{"1min":0.63300000000000001,"5min":0.54200000000000004,"15min":0.51900000000000002},"max":{"1min":1.708,"5min":2.238,"15min":8.3109999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.163,"15min":1.327},"min":{"1min":0.69599999999999995,"5min":0.46300000000000002,"15min":0.41799999999999998},"max":{"1min":1.73,"5min":2.323,"15min":8.0690000000000008},"last":0.64000000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.417,"15min":1.4610000000000001},"min":{"1min":0.93000000000000005,"5min":0.83399999999999996,"15min":0.79300000000000004},"max":{"1min":1.8009999999999999,"5min":2.6139999999999999,"15min":5.3929999999999998},"last":1.081},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.468,"15min":1.5489999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.78300000000000003},"max":{"1min":2.1949999999999998,"5min":3.1070000000000002,"15min":7.3570000000000002},"last":1.1100000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3939999999999999,"15min":1.546},"min":{"1min":0.93200000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9199999999999999,"5min":2.5579999999999998,"15min":8.1839999999999993},"last":1.052},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.417},"min":{"1min":0.99299999999999999,"5min":0.81000000000000005,"15min":0.72199999999999998},"max":{"1min":1.845,"5min":2.4940000000000002,"15min":5.165},"last":0.86499999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.4350000000000001,"15min":1.454},"min":{"1min":0.82399999999999995,"5min":0.752,"15min":0.72999999999999998},"max":{"1min":2.5139999999999998,"5min":3.0179999999999998,"15min":5.1059999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.446,"5min":1.4650000000000001,"15min":1.5589999999999999},"min":{"1min":1.0049999999999999,"5min":0.81200000000000006,"15min":0.77000000000000002},"max":{"1min":1.879,"5min":2.9580000000000002,"15min":8.1370000000000005},"last":1.153}]},{"osd":7,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.5149999999999999,"15min":1.552},"min":{"1min":0.98499999999999999,"5min":0.79200000000000004,"15min":0.78200000000000003},"max":{"1min":2.3700000000000001,"5min":3.2189999999999999,"15min":5.1449999999999996},"last":1.016},{"interface":"front","average":{"1min":1.696,"5min":1.548,"15min":1.659},"min":{"1min":1.054,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":4.8289999999999997,"5min":4.8289999999999997,"15min":8.2300000000000004},"last":0.90000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1293,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194019,"statfs":{"total":200000000,"available":198675466,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.84599999999999997},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3660000000000001,"5min":1.52,"15min":1.52},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.40000000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0720000000000001,"15min":1.0609999999999999},"min":{"1min":0.52500000000000002,"5min":0.42299999999999999,"15min":0.41399999999999998},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.5739999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.026,"15min":0.93600000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":1.6839999999999999,"5min":1.845,"15min":1.845},"last":0.79900000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3049999999999999,"15min":1.1719999999999999},"min":{"1min":0.64200000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.1040000000000001,"5min":2.137,"15min":2.137},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.2170000000000001,"15min":1.1930000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.44900000000000001},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.3959999999999999},"last":0.63600000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3520000000000001,"15min":1.2989999999999999},"min":{"1min":0.83999999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1440000000000001,"5min":2.444,"15min":2.444},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4039999999999999,"15min":1.3100000000000001},"min":{"1min":0.95299999999999996,"5min":0.76900000000000002,"15min":0.71099999999999997},"max":{"1min":1.7569999999999999,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.034}]},{"osd":5,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.425,"15min":1.3100000000000001},"min":{"1min":0.92800000000000005,"5min":0.88,"15min":0.88},"max":{"1min":1.915,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5049999999999999,"15min":1.423},"min":{"1min":1.0009999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.371,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.0089999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4530000000000001,"15min":1.3899999999999999},"min":{"1min":0.88400000000000001,"5min":0.875,"15min":0.76900000000000002},"max":{"1min":2.3610000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.538,"15min":1.46},"min":{"1min":1.032,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.2719999999999998,"5min":2.895,"15min":2.895},"last":1.0529999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.571,"15min":1.397},"min":{"1min":1.018,"5min":0.879,"15min":0.69999999999999996},"max":{"1min":1.8129999999999999,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.659,"15min":1.508},"min":{"1min":0.97399999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.2360000000000002,"5min":9.4109999999999996,"15min":9.4109999999999996},"last":0.874}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.99399999999999999,"15min":0.90600000000000003},"min":{"1min":0.64000000000000001,"5min":0.46700000000000003,"15min":0.39900000000000002},"max":{"1min":1.3220000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.143,"15min":1.0429999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.0369999999999999,"5min":2.1040000000000001,"15min":2.4180000000000001},"last":1.3460000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0660000000000001,"15min":0.96299999999999997},"min":{"1min":0.73099999999999998,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":1.4810000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0389999999999999,"15min":0.94699999999999995},"min":{"1min":0.55100000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.335,"5min":2.7810000000000001,"15min":2.7810000000000001},"last":0.70299999999999996}]},{"osd":3,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0880000000000001,"15min":1.0209999999999999},"min":{"1min":0.61299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.732,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.999},{"interface":"front","average":{"1min":1.306,"5min":1.3580000000000001,"15min":1.2390000000000001},"min":{"1min":0.746,"5min":0.63100000000000001,"15min":0.47499999999999998},"max":{"1min":1.982,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.411}]},{"osd":4,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.335,"15min":1.216},"min":{"1min":0.86299999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6739999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.282},{"interface":"front","average":{"1min":1.343,"5min":1.3360000000000001,"15min":1.2929999999999999},"min":{"1min":1.0409999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.663,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.119}]},{"osd":5,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.5489999999999999,"15min":1.4059999999999999},"min":{"1min":1.0169999999999999,"5min":0.71099999999999997,"15min":0.64500000000000002},"max":{"1min":1.877,"5min":16.298999999999999,"15min":16.298999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.385,"15min":1.2909999999999999},"min":{"1min":0.94899999999999995,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8109999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.696}]},{"osd":6,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.5029999999999999,"15min":1.409},"min":{"1min":1.171,"5min":0.85499999999999998,"15min":0.75900000000000001},"max":{"1min":1.9119999999999999,"5min":2.2559999999999998,"15min":2.5910000000000002},"last":1.776},{"interface":"front","average":{"1min":1.391,"5min":1.462,"15min":1.369},"min":{"1min":1.0349999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.776,"5min":2.46,"15min":2.46},"last":1.3819999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5369999999999999,"15min":1.409},"min":{"1min":1.1279999999999999,"5min":0.89200000000000002,"15min":0.70899999999999996},"max":{"1min":1.8740000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.466,"15min":1.4330000000000001},"min":{"1min":0.88400000000000001,"5min":0.873,"15min":0.81999999999999995},"max":{"1min":2.113,"5min":2.4580000000000002,"15min":3.1429999999999998},"last":1.3100000000000001}]}]}],"pool_statfs":[]}} 2023-12-19T18:22:19.252 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:22:19.511 INFO:tasks.ceph.mds.c.smithi027.stderr:2023-12-19T18:22:19.511+0000 7fc78cfa1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:22:19.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.524+0000 7f302bbb1640 1 Processor -- start 2023-12-19T18:22:19.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.524+0000 7f302bbb1640 1 -- start start 2023-12-19T18:22:19.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.526+0000 7f302bbb1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f302415ce90 0x7f30240cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:19.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.526+0000 7f302bbb1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30240ade10 0x7f30240ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:19.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.526+0000 7f302bbb1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f302406b4e0 con 0x7f302415d270 2023-12-19T18:22:19.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.526+0000 7f302bbb1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302406ce90 con 0x7f302415ce90 2023-12-19T18:22:19.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.526+0000 7f302bbb1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30240704f0 con 0x7f30240ade10 2023-12-19T18:22:19.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.526+0000 7f302994d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f302415ce90 0x7f30240cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:19.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.526+0000 7f302914c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f302415d270 0x7f30240cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41268/0 (socket says 172.21.15.27:41268) 2023-12-19T18:22:19.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.526+0000 7f302914c640 1 -- 172.21.15.27:0/3288859142 learned_addr learned my addr 172.21.15.27:0/3288859142 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:19.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.527+0000 7f302a14e640 1 --2- 172.21.15.27:0/3288859142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30240ade10 0x7f30240ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:19.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.527+0000 7f302994d640 1 -- 172.21.15.27:0/3288859142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30240ade10 msgr2=0x7f30240ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:19.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.527+0000 7f302994d640 1 --2- 172.21.15.27:0/3288859142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30240ade10 0x7f30240ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.527+0000 7f302994d640 1 -- 172.21.15.27:0/3288859142 >> v1:172.21.15.27:6789/0 conn(0x7f302415d270 legacy=0x7f30240cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:19.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.527+0000 7f302994d640 1 -- 172.21.15.27:0/3288859142 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302406b7c0 con 0x7f302415ce90 2023-12-19T18:22:19.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.527+0000 7f302894b640 1 -- 172.21.15.27:0/3288859142 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3781964757 0 0) 0x7f302406b4e0 con 0x7f302415d270 2023-12-19T18:22:19.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.528+0000 7f302994d640 1 --2- 172.21.15.27:0/3288859142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302415ce90 0x7f30240cc710 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f30200097a0 tx=0x7f302002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9999e832293a38 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:19.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.529+0000 7f302894b640 1 -- 172.21.15.27:0/3288859142 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020031830 con 0x7f302415ce90 2023-12-19T18:22:19.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.529+0000 7f302894b640 1 -- 172.21.15.27:0/3288859142 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3020031990 con 0x7f302415ce90 2023-12-19T18:22:19.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.529+0000 7f302894b640 1 -- 172.21.15.27:0/3288859142 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020031d20 con 0x7f302415ce90 2023-12-19T18:22:19.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.529+0000 7f302bbb1640 1 -- 172.21.15.27:0/3288859142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302415ce90 msgr2=0x7f30240cc710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:19.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.529+0000 7f302bbb1640 1 --2- 172.21.15.27:0/3288859142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302415ce90 0x7f30240cc710 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f30200097a0 tx=0x7f302002b8d0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.529+0000 7f302bbb1640 1 -- 172.21.15.27:0/3288859142 shutdown_connections 2023-12-19T18:22:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.529+0000 7f302bbb1640 1 --2- 172.21.15.27:0/3288859142 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30240ade10 0x7f30240ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.529+0000 7f302bbb1640 1 --2- 172.21.15.27:0/3288859142 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302415ce90 0x7f30240cc710 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.529+0000 7f302bbb1640 1 -- 172.21.15.27:0/3288859142 >> 172.21.15.27:0/3288859142 conn(0x7f302406a330 msgr2=0x7f30240b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.529+0000 7f302bbb1640 1 -- 172.21.15.27:0/3288859142 shutdown_connections 2023-12-19T18:22:19.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.529+0000 7f302bbb1640 1 -- 172.21.15.27:0/3288859142 wait complete. 2023-12-19T18:22:19.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.531+0000 7f302bbb1640 1 Processor -- start 2023-12-19T18:22:19.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.531+0000 7f302bbb1640 1 -- start start 2023-12-19T18:22:19.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.532+0000 7f302bbb1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30240ade10 0x7f30241515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:19.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.532+0000 7f302bbb1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f302415ce90 0x7f30240c2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:19.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.532+0000 7f302bbb1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3024151e60 0x7f30240b31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:19.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.532+0000 7f302bbb1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30240c2f70 con 0x7f30240ade10 2023-12-19T18:22:19.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.532+0000 7f302bbb1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30240c3250 con 0x7f302415ce90 2023-12-19T18:22:19.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.532+0000 7f302bbb1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30240c3530 con 0x7f3024151e60 2023-12-19T18:22:19.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.533+0000 7f302994d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30240ade10 0x7f30241515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:19.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.533+0000 7f302994d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30240ade10 0x7f30241515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42986/0 (socket says 172.21.15.27:42986) 2023-12-19T18:22:19.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.533+0000 7f302994d640 1 -- 172.21.15.27:0/3339992879 learned_addr learned my addr 172.21.15.27:0/3339992879 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:19.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.533+0000 7f302a14e640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3024151e60 0x7f30240b31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.533+0000 7f302914c640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f302415ce90 0x7f30240c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.533+0000 7f302994d640 1 -- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3024151e60 msgr2=0x7f30240b31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.533+0000 7f302994d640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3024151e60 0x7f30240b31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.533+0000 7f302994d640 1 -- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f302415ce90 msgr2=0x7f30240c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.533+0000 7f302994d640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f302415ce90 0x7f30240c2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.533+0000 7f302994d640 1 -- 172.21.15.27:0/3339992879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3020009400 con 0x7f30240ade10 2023-12-19T18:22:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.533+0000 7f302914c640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f302415ce90 0x7f30240c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.534+0000 7f302994d640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30240ade10 0x7f30241515c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f3020000c00 tx=0x7f30200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.534+0000 7f301a7fc640 1 -- 172.21.15.27:0/3339992879 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020004230 con 0x7f30240ade10 2023-12-19T18:22:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.534+0000 7f301a7fc640 1 -- 172.21.15.27:0/3339992879 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3020004390 con 0x7f30240ade10 2023-12-19T18:22:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.534+0000 7f301a7fc640 1 -- 172.21.15.27:0/3339992879 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302003a8a0 con 0x7f30240ade10 2023-12-19T18:22:19.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.534+0000 7f302bbb1640 1 -- 172.21.15.27:0/3339992879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30240c3810 con 0x7f30240ade10 2023-12-19T18:22:19.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.534+0000 7f302bbb1640 1 -- 172.21.15.27:0/3339992879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30240c4010 con 0x7f30240ade10 2023-12-19T18:22:19.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.535+0000 7f302bbb1640 1 -- 172.21.15.27:0/3339992879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30240c3d30 con 0x7f30240ade10 2023-12-19T18:22:19.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.536+0000 7f301a7fc640 1 -- 172.21.15.27:0/3339992879 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f302003aa00 con 0x7f30240ade10 2023-12-19T18:22:19.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.537+0000 7f301a7fc640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f300806a430 0x7f300806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:19.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.537+0000 7f301a7fc640 1 -- 172.21.15.27:0/3339992879 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f30200ab2b0 con 0x7f30240ade10 2023-12-19T18:22:19.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.538+0000 7f302914c640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f300806a430 0x7f300806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:19.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.539+0000 7f302914c640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f300806a430 0x7f300806c8f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f30240c4c10 tx=0x7f3014009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:19.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.541+0000 7f301a7fc640 1 -- 172.21.15.27:0/3339992879 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f30240c3d30 con 0x7f30240ade10 2023-12-19T18:22:19.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.732+0000 7f302bbb1640 1 -- 172.21.15.27:0/3339992879 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f302406bd80 con 0x7f300806a430 2023-12-19T18:22:19.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.756+0000 7f301a7fc640 1 -- 172.21.15.27:0/3339992879 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+280208 (secure 0 0 0) 0x7f30140013d0 con 0x7f300806a430 2023-12-19T18:22:19.758 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:19.762 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:22:19.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.763+0000 7f302bbb1640 1 -- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f300806a430 msgr2=0x7f300806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:19.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.763+0000 7f302bbb1640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f300806a430 0x7f300806c8f0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f30240c4c10 tx=0x7f3014009040 comp rx=0 tx=0).stop 2023-12-19T18:22:19.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.763+0000 7f302bbb1640 1 -- 172.21.15.27:0/3339992879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30240ade10 msgr2=0x7f30241515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:19.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.763+0000 7f302bbb1640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30240ade10 0x7f30241515c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f3020000c00 tx=0x7f30200046c0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.764+0000 7f302bbb1640 1 -- 172.21.15.27:0/3339992879 shutdown_connections 2023-12-19T18:22:19.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.764+0000 7f302bbb1640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f300806a430 0x7f300806c8f0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.764+0000 7f302bbb1640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3024151e60 0x7f30240b31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.764+0000 7f302bbb1640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f302415ce90 0x7f30240c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.764+0000 7f302bbb1640 1 --2- 172.21.15.27:0/3339992879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30240ade10 0x7f30241515c0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:19.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.764+0000 7f302bbb1640 1 -- 172.21.15.27:0/3339992879 >> 172.21.15.27:0/3339992879 conn(0x7f302406a330 msgr2=0x7f30240cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:19.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.764+0000 7f302bbb1640 1 -- 172.21.15.27:0/3339992879 shutdown_connections 2023-12-19T18:22:19.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:19.764+0000 7f302bbb1640 1 -- 172.21.15.27:0/3339992879 wait complete. 2023-12-19T18:22:19.786 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2023-12-19T18:22:17.865945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":6888,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555611,"statfs":{"total":1600000000,"available":1592945936,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.171043"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.726129+0000","last_change":"2023-12-19T18:22:13.726129+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.726129+0000","last_undegraded":"2023-12-19T18:22:13.726129+0000","last_fullsized":"2023-12-19T18:22:13.726129+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:40.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.726390+0000","last_change":"2023-12-19T18:22:13.726390+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.726390+0000","last_undegraded":"2023-12-19T18:22:13.726390+0000","last_fullsized":"2023-12-19T18:22:13.726390+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:15.726948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.709168+0000","last_change":"2023-12-19T18:22:13.709168+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.709168+0000","last_undegraded":"2023-12-19T18:22:13.709168+0000","last_fullsized":"2023-12-19T18:22:13.709168+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:16.333492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.718039+0000","last_change":"2023-12-19T18:22:13.718039+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.718039+0000","last_undegraded":"2023-12-19T18:22:13.718039+0000","last_fullsized":"2023-12-19T18:22:13.718039+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:09.942336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.723318+0000","last_change":"2023-12-19T18:22:13.723318+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.723318+0000","last_undegraded":"2023-12-19T18:22:13.723318+0000","last_fullsized":"2023-12-19T18:22:13.723318+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:26.506333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.725569+0000","last_change":"2023-12-19T18:22:13.725569+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.725569+0000","last_undegraded":"2023-12-19T18:22:13.725569+0000","last_fullsized":"2023-12-19T18:22:13.725569+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:02:55.424114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.722057+0000","last_change":"2023-12-19T18:22:13.722057+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.722057+0000","last_undegraded":"2023-12-19T18:22:13.722057+0000","last_fullsized":"2023-12-19T18:22:13.722057+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:37:37.391952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.708366+0000","last_change":"2023-12-19T18:22:13.708366+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.708366+0000","last_undegraded":"2023-12-19T18:22:13.708366+0000","last_fullsized":"2023-12-19T18:22:13.708366+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:23.915911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.723598+0000","last_change":"2023-12-19T18:22:13.723598+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.723598+0000","last_undegraded":"2023-12-19T18:22:13.723598+0000","last_fullsized":"2023-12-19T18:22:13.723598+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:29.637020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.721388+0000","last_change":"2023-12-19T18:22:13.721388+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.721388+0000","last_undegraded":"2023-12-19T18:22:13.721388+0000","last_fullsized":"2023-12-19T18:22:13.721388+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:11:57.860995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.723865+0000","last_change":"2023-12-19T18:22:13.723865+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.723865+0000","last_undegraded":"2023-12-19T18:22:13.723865+0000","last_fullsized":"2023-12-19T18:22:13.723865+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:23.903727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716942+0000","last_change":"2023-12-19T18:22:13.716942+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716942+0000","last_undegraded":"2023-12-19T18:22:13.716942+0000","last_fullsized":"2023-12-19T18:22:13.716942+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:31.105594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.719484+0000","last_change":"2023-12-19T18:22:13.719484+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.719484+0000","last_undegraded":"2023-12-19T18:22:13.719484+0000","last_fullsized":"2023-12-19T18:22:13.719484+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:33.150825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716881+0000","last_change":"2023-12-19T18:22:13.716881+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716881+0000","last_undegraded":"2023-12-19T18:22:13.716881+0000","last_fullsized":"2023-12-19T18:22:13.716881+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:16.333492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.728029+0000","last_change":"2023-12-19T18:22:13.728029+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.728029+0000","last_undegraded":"2023-12-19T18:22:13.728029+0000","last_fullsized":"2023-12-19T18:22:13.728029+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:34.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.722342+0000","last_change":"2023-12-19T18:22:13.722342+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.722342+0000","last_undegraded":"2023-12-19T18:22:13.722342+0000","last_fullsized":"2023-12-19T18:22:13.722342+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:02.299174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.722109+0000","last_change":"2023-12-19T18:22:13.722109+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.722109+0000","last_undegraded":"2023-12-19T18:22:13.722109+0000","last_fullsized":"2023-12-19T18:22:13.722109+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:41:21.616217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.718271+0000","last_change":"2023-12-19T18:22:13.718271+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.718271+0000","last_undegraded":"2023-12-19T18:22:13.718271+0000","last_fullsized":"2023-12-19T18:22:13.718271+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:44.388621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.718310+0000","last_change":"2023-12-19T18:22:13.718310+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.718310+0000","last_undegraded":"2023-12-19T18:22:13.718310+0000","last_fullsized":"2023-12-19T18:22:13.718310+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:41:30.585976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.717068+0000","last_change":"2023-12-19T18:22:13.717068+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.717068+0000","last_undegraded":"2023-12-19T18:22:13.717068+0000","last_fullsized":"2023-12-19T18:22:13.717068+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:40.700106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.711531+0000","last_change":"2023-12-19T18:22:13.711531+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.711531+0000","last_undegraded":"2023-12-19T18:22:13.711531+0000","last_fullsized":"2023-12-19T18:22:13.711531+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:11.523949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.708779+0000","last_change":"2023-12-19T18:22:13.708779+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.708779+0000","last_undegraded":"2023-12-19T18:22:13.708779+0000","last_fullsized":"2023-12-19T18:22:13.708779+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:13:15.250452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.717448+0000","last_change":"2023-12-19T18:22:13.717448+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.717448+0000","last_undegraded":"2023-12-19T18:22:13.717448+0000","last_fullsized":"2023-12-19T18:22:13.717448+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:08.465323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.708091+0000","last_change":"2023-12-19T18:22:13.708091+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.708091+0000","last_undegraded":"2023-12-19T18:22:13.708091+0000","last_fullsized":"2023-12-19T18:22:13.708091+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:02:16.250351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.720837+0000","last_change":"2023-12-19T18:22:13.720837+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.720837+0000","last_undegraded":"2023-12-19T18:22:13.720837+0000","last_fullsized":"2023-12-19T18:22:13.720837+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:32.485695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.715635+0000","last_change":"2023-12-19T18:22:13.715635+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.715635+0000","last_undegraded":"2023-12-19T18:22:13.715635+0000","last_fullsized":"2023-12-19T18:22:13.715635+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:08.366785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.707470+0000","last_change":"2023-12-19T18:22:13.707470+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.707470+0000","last_undegraded":"2023-12-19T18:22:13.707470+0000","last_fullsized":"2023-12-19T18:22:13.707470+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:57:01.585693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.707503+0000","last_change":"2023-12-19T18:22:13.707503+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.707503+0000","last_undegraded":"2023-12-19T18:22:13.707503+0000","last_fullsized":"2023-12-19T18:22:13.707503+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:01:59.548259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.714779+0000","last_change":"2023-12-19T18:22:13.714779+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.714779+0000","last_undegraded":"2023-12-19T18:22:13.714779+0000","last_fullsized":"2023-12-19T18:22:13.714779+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:32.485695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.721121+0000","last_change":"2023-12-19T18:22:13.721121+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.721121+0000","last_undegraded":"2023-12-19T18:22:13.721121+0000","last_fullsized":"2023-12-19T18:22:13.721121+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:47:09.595830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.710659+0000","last_change":"2023-12-19T18:22:13.710659+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.710659+0000","last_undegraded":"2023-12-19T18:22:13.710659+0000","last_fullsized":"2023-12-19T18:22:13.710659+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:01:36.203567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.724266+0000","last_change":"2023-12-19T18:22:13.724266+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.724266+0000","last_undegraded":"2023-12-19T18:22:13.724266+0000","last_fullsized":"2023-12-19T18:22:13.724266+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:16.254938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.717402+0000","last_change":"2023-12-19T18:22:13.717402+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.717402+0000","last_undegraded":"2023-12-19T18:22:13.717402+0000","last_fullsized":"2023-12-19T18:22:13.717402+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:33.697121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.735336+0000","last_change":"2023-12-19T18:22:13.732827+0000","last_active":"2023-12-19T18:22:13.735336+0000","last_peered":"2023-12-19T18:22:13.735336+0000","last_clean":"2023-12-19T18:22:13.735336+0000","last_became_active":"2023-12-19T18:22:13.732603+0000","last_became_peered":"2023-12-19T18:22:13.732603+0000","last_unstale":"2023-12-19T18:22:13.735336+0000","last_undegraded":"2023-12-19T18:22:13.735336+0000","last_fullsized":"2023-12-19T18:22:13.735336+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:41:12.315002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.718562+0000","last_change":"2023-12-19T18:22:13.718562+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.718562+0000","last_undegraded":"2023-12-19T18:22:13.718562+0000","last_fullsized":"2023-12-19T18:22:13.718562+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:39.384923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.732498+0000","last_change":"2023-12-19T18:22:13.731624+0000","last_active":"2023-12-19T18:22:13.732498+0000","last_peered":"2023-12-19T18:22:13.732498+0000","last_clean":"2023-12-19T18:22:13.732498+0000","last_became_active":"2023-12-19T18:22:13.731335+0000","last_became_peered":"2023-12-19T18:22:13.731335+0000","last_unstale":"2023-12-19T18:22:13.732498+0000","last_undegraded":"2023-12-19T18:22:13.732498+0000","last_fullsized":"2023-12-19T18:22:13.732498+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:53:16.279722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.725678+0000","last_change":"2023-12-19T18:22:13.725678+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.725678+0000","last_undegraded":"2023-12-19T18:22:13.725678+0000","last_fullsized":"2023-12-19T18:22:13.725678+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:16.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.733264+0000","last_change":"2023-12-19T18:22:13.733176+0000","last_active":"2023-12-19T18:22:13.733264+0000","last_peered":"2023-12-19T18:22:13.733264+0000","last_clean":"2023-12-19T18:22:13.733264+0000","last_became_active":"2023-12-19T18:22:13.732940+0000","last_became_peered":"2023-12-19T18:22:13.732940+0000","last_unstale":"2023-12-19T18:22:13.733264+0000","last_undegraded":"2023-12-19T18:22:13.733264+0000","last_fullsized":"2023-12-19T18:22:13.733264+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:30.091059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716091+0000","last_change":"2023-12-19T18:22:13.716091+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716091+0000","last_undegraded":"2023-12-19T18:22:13.716091+0000","last_fullsized":"2023-12-19T18:22:13.716091+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:39.384923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.733455+0000","last_change":"2023-12-19T18:22:13.733119+0000","last_active":"2023-12-19T18:22:13.733455+0000","last_peered":"2023-12-19T18:22:13.733455+0000","last_clean":"2023-12-19T18:22:13.733455+0000","last_became_active":"2023-12-19T18:22:13.732868+0000","last_became_peered":"2023-12-19T18:22:13.732868+0000","last_unstale":"2023-12-19T18:22:13.733455+0000","last_undegraded":"2023-12-19T18:22:13.733455+0000","last_fullsized":"2023-12-19T18:22:13.733455+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:07.352249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.717476+0000","last_change":"2023-12-19T18:22:13.717476+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.717476+0000","last_undegraded":"2023-12-19T18:22:13.717476+0000","last_fullsized":"2023-12-19T18:22:13.717476+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:53.745021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.731132+0000","last_change":"2023-12-19T18:22:13.730941+0000","last_active":"2023-12-19T18:22:13.731132+0000","last_peered":"2023-12-19T18:22:13.731132+0000","last_clean":"2023-12-19T18:22:13.731132+0000","last_became_active":"2023-12-19T18:22:13.730684+0000","last_became_peered":"2023-12-19T18:22:13.730684+0000","last_unstale":"2023-12-19T18:22:13.731132+0000","last_undegraded":"2023-12-19T18:22:13.731132+0000","last_fullsized":"2023-12-19T18:22:13.731132+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:43:06.634088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.722437+0000","last_change":"2023-12-19T18:22:13.722437+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.722437+0000","last_undegraded":"2023-12-19T18:22:13.722437+0000","last_fullsized":"2023-12-19T18:22:13.722437+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:41.584580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.732419+0000","last_change":"2023-12-19T18:22:13.731736+0000","last_active":"2023-12-19T18:22:13.732419+0000","last_peered":"2023-12-19T18:22:13.732419+0000","last_clean":"2023-12-19T18:22:13.732419+0000","last_became_active":"2023-12-19T18:22:13.731153+0000","last_became_peered":"2023-12-19T18:22:13.731153+0000","last_unstale":"2023-12-19T18:22:13.732419+0000","last_undegraded":"2023-12-19T18:22:13.732419+0000","last_fullsized":"2023-12-19T18:22:13.732419+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:52.730485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.720362+0000","last_change":"2023-12-19T18:22:13.720362+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.720362+0000","last_undegraded":"2023-12-19T18:22:13.720362+0000","last_fullsized":"2023-12-19T18:22:13.720362+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:11:57.860995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.730782+0000","last_change":"2023-12-19T18:22:13.730291+0000","last_active":"2023-12-19T18:22:13.730782+0000","last_peered":"2023-12-19T18:22:13.730782+0000","last_clean":"2023-12-19T18:22:13.730782+0000","last_became_active":"2023-12-19T18:22:13.730015+0000","last_became_peered":"2023-12-19T18:22:13.730015+0000","last_unstale":"2023-12-19T18:22:13.730782+0000","last_undegraded":"2023-12-19T18:22:13.730782+0000","last_fullsized":"2023-12-19T18:22:13.730782+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:02:54.040583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716117+0000","last_change":"2023-12-19T18:22:13.716117+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716117+0000","last_undegraded":"2023-12-19T18:22:13.716117+0000","last_fullsized":"2023-12-19T18:22:13.716117+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:45.553627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.732144+0000","last_change":"2023-12-19T18:22:13.730656+0000","last_active":"2023-12-19T18:22:13.732144+0000","last_peered":"2023-12-19T18:22:13.732144+0000","last_clean":"2023-12-19T18:22:13.732144+0000","last_became_active":"2023-12-19T18:22:13.730408+0000","last_became_peered":"2023-12-19T18:22:13.730408+0000","last_unstale":"2023-12-19T18:22:13.732144+0000","last_undegraded":"2023-12-19T18:22:13.732144+0000","last_fullsized":"2023-12-19T18:22:13.732144+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:33.562949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.725020+0000","last_change":"2023-12-19T18:22:13.725020+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.725020+0000","last_undegraded":"2023-12-19T18:22:13.725020+0000","last_fullsized":"2023-12-19T18:22:13.725020+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:13:15.250452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.724646+0000","last_change":"2023-12-19T18:22:13.724536+0000","last_active":"2023-12-19T18:22:13.724646+0000","last_peered":"2023-12-19T18:22:13.724646+0000","last_clean":"2023-12-19T18:22:13.724646+0000","last_became_active":"2023-12-19T18:22:13.724110+0000","last_became_peered":"2023-12-19T18:22:13.724110+0000","last_unstale":"2023-12-19T18:22:13.724646+0000","last_undegraded":"2023-12-19T18:22:13.724646+0000","last_fullsized":"2023-12-19T18:22:13.724646+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:44.539092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.706194+0000","last_change":"2023-12-19T18:22:13.706194+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.706194+0000","last_undegraded":"2023-12-19T18:22:13.706194+0000","last_fullsized":"2023-12-19T18:22:13.706194+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:15.726948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.733474+0000","last_change":"2023-12-19T18:22:13.733261+0000","last_active":"2023-12-19T18:22:13.733474+0000","last_peered":"2023-12-19T18:22:13.733474+0000","last_clean":"2023-12-19T18:22:13.733474+0000","last_became_active":"2023-12-19T18:22:13.733015+0000","last_became_peered":"2023-12-19T18:22:13.733015+0000","last_unstale":"2023-12-19T18:22:13.733474+0000","last_undegraded":"2023-12-19T18:22:13.733474+0000","last_fullsized":"2023-12-19T18:22:13.733474+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:52.730485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716780+0000","last_change":"2023-12-19T18:22:13.716780+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716780+0000","last_undegraded":"2023-12-19T18:22:13.716780+0000","last_fullsized":"2023-12-19T18:22:13.716780+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:32:56.103686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.728083+0000","last_change":"2023-12-19T18:22:13.727917+0000","last_active":"2023-12-19T18:22:13.728083+0000","last_peered":"2023-12-19T18:22:13.728083+0000","last_clean":"2023-12-19T18:22:13.728083+0000","last_became_active":"2023-12-19T18:22:13.727633+0000","last_became_peered":"2023-12-19T18:22:13.727633+0000","last_unstale":"2023-12-19T18:22:13.728083+0000","last_undegraded":"2023-12-19T18:22:13.728083+0000","last_fullsized":"2023-12-19T18:22:13.728083+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:36:19.046380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.706628+0000","last_change":"2023-12-19T18:22:13.706628+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.706628+0000","last_undegraded":"2023-12-19T18:22:13.706628+0000","last_fullsized":"2023-12-19T18:22:13.706628+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:40.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.725596+0000","last_change":"2023-12-19T18:22:13.725517+0000","last_active":"2023-12-19T18:22:13.725596+0000","last_peered":"2023-12-19T18:22:13.725596+0000","last_clean":"2023-12-19T18:22:13.725596+0000","last_became_active":"2023-12-19T18:22:13.724094+0000","last_became_peered":"2023-12-19T18:22:13.724094+0000","last_unstale":"2023-12-19T18:22:13.725596+0000","last_undegraded":"2023-12-19T18:22:13.725596+0000","last_fullsized":"2023-12-19T18:22:13.725596+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:29:20.466956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.724994+0000","last_change":"2023-12-19T18:22:13.724994+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.724994+0000","last_undegraded":"2023-12-19T18:22:13.724994+0000","last_fullsized":"2023-12-19T18:22:13.724994+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:37.118051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.732124+0000","last_change":"2023-12-19T18:22:13.730064+0000","last_active":"2023-12-19T18:22:13.732124+0000","last_peered":"2023-12-19T18:22:13.732124+0000","last_clean":"2023-12-19T18:22:13.732124+0000","last_became_active":"2023-12-19T18:22:13.729780+0000","last_became_peered":"2023-12-19T18:22:13.729780+0000","last_unstale":"2023-12-19T18:22:13.732124+0000","last_undegraded":"2023-12-19T18:22:13.732124+0000","last_fullsized":"2023-12-19T18:22:13.732124+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:14.712413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.710003+0000","last_change":"2023-12-19T18:22:13.710003+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.710003+0000","last_undegraded":"2023-12-19T18:22:13.710003+0000","last_fullsized":"2023-12-19T18:22:13.710003+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:16.254938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.733437+0000","last_change":"2023-12-19T18:22:13.732632+0000","last_active":"2023-12-19T18:22:13.733437+0000","last_peered":"2023-12-19T18:22:13.733437+0000","last_clean":"2023-12-19T18:22:13.733437+0000","last_became_active":"2023-12-19T18:22:13.732365+0000","last_became_peered":"2023-12-19T18:22:13.732365+0000","last_unstale":"2023-12-19T18:22:13.733437+0000","last_undegraded":"2023-12-19T18:22:13.733437+0000","last_fullsized":"2023-12-19T18:22:13.733437+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.720885+0000","last_change":"2023-12-19T18:22:13.720885+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.720885+0000","last_undegraded":"2023-12-19T18:22:13.720885+0000","last_fullsized":"2023-12-19T18:22:13.720885+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:06:47.768050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.731449+0000","last_change":"2023-12-19T18:22:13.729287+0000","last_active":"2023-12-19T18:22:13.731449+0000","last_peered":"2023-12-19T18:22:13.731449+0000","last_clean":"2023-12-19T18:22:13.731449+0000","last_became_active":"2023-12-19T18:22:13.725118+0000","last_became_peered":"2023-12-19T18:22:13.725118+0000","last_unstale":"2023-12-19T18:22:13.731449+0000","last_undegraded":"2023-12-19T18:22:13.731449+0000","last_fullsized":"2023-12-19T18:22:13.731449+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:13:44.939138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.707867+0000","last_change":"2023-12-19T18:22:13.707867+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.707867+0000","last_undegraded":"2023-12-19T18:22:13.707867+0000","last_fullsized":"2023-12-19T18:22:13.707867+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:08.465323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.735138+0000","last_change":"2023-12-19T18:22:13.734908+0000","last_active":"2023-12-19T18:22:13.735138+0000","last_peered":"2023-12-19T18:22:13.735138+0000","last_clean":"2023-12-19T18:22:13.735138+0000","last_became_active":"2023-12-19T18:22:13.734666+0000","last_became_peered":"2023-12-19T18:22:13.734666+0000","last_unstale":"2023-12-19T18:22:13.735138+0000","last_undegraded":"2023-12-19T18:22:13.735138+0000","last_fullsized":"2023-12-19T18:22:13.735138+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:44.539092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.707148+0000","last_change":"2023-12-19T18:22:13.707148+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.707148+0000","last_undegraded":"2023-12-19T18:22:13.707148+0000","last_fullsized":"2023-12-19T18:22:13.707148+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:12.207051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.724869+0000","last_change":"2023-12-19T18:22:13.724787+0000","last_active":"2023-12-19T18:22:13.724869+0000","last_peered":"2023-12-19T18:22:13.724869+0000","last_clean":"2023-12-19T18:22:13.724869+0000","last_became_active":"2023-12-19T18:22:13.724547+0000","last_became_peered":"2023-12-19T18:22:13.724547+0000","last_unstale":"2023-12-19T18:22:13.724869+0000","last_undegraded":"2023-12-19T18:22:13.724869+0000","last_fullsized":"2023-12-19T18:22:13.724869+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:11.095769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.706495+0000","last_change":"2023-12-19T18:22:13.706495+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.706495+0000","last_undegraded":"2023-12-19T18:22:13.706495+0000","last_fullsized":"2023-12-19T18:22:13.706495+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:34.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.730755+0000","last_change":"2023-12-19T18:22:13.728901+0000","last_active":"2023-12-19T18:22:13.730755+0000","last_peered":"2023-12-19T18:22:13.730755+0000","last_clean":"2023-12-19T18:22:13.730755+0000","last_became_active":"2023-12-19T18:22:13.728001+0000","last_became_peered":"2023-12-19T18:22:13.728001+0000","last_unstale":"2023-12-19T18:22:13.730755+0000","last_undegraded":"2023-12-19T18:22:13.730755+0000","last_fullsized":"2023-12-19T18:22:13.730755+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:59.426392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716530+0000","last_change":"2023-12-19T18:22:13.716530+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716530+0000","last_undegraded":"2023-12-19T18:22:13.716530+0000","last_fullsized":"2023-12-19T18:22:13.716530+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:01.033125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.729108+0000","last_change":"2023-12-19T18:22:13.729021+0000","last_active":"2023-12-19T18:22:13.729108+0000","last_peered":"2023-12-19T18:22:13.729108+0000","last_clean":"2023-12-19T18:22:13.729108+0000","last_became_active":"2023-12-19T18:22:13.728792+0000","last_became_peered":"2023-12-19T18:22:13.728792+0000","last_unstale":"2023-12-19T18:22:13.729108+0000","last_undegraded":"2023-12-19T18:22:13.729108+0000","last_fullsized":"2023-12-19T18:22:13.729108+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:39.606223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716582+0000","last_change":"2023-12-19T18:22:13.716582+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716582+0000","last_undegraded":"2023-12-19T18:22:13.716582+0000","last_fullsized":"2023-12-19T18:22:13.716582+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:50:35.183653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.734645+0000","last_change":"2023-12-19T18:22:13.734402+0000","last_active":"2023-12-19T18:22:13.734645+0000","last_peered":"2023-12-19T18:22:13.734645+0000","last_clean":"2023-12-19T18:22:13.734645+0000","last_became_active":"2023-12-19T18:22:13.733935+0000","last_became_peered":"2023-12-19T18:22:13.733935+0000","last_unstale":"2023-12-19T18:22:13.734645+0000","last_undegraded":"2023-12-19T18:22:13.734645+0000","last_fullsized":"2023-12-19T18:22:13.734645+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:30.091059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.719693+0000","last_change":"2023-12-19T18:22:13.719693+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.719693+0000","last_undegraded":"2023-12-19T18:22:13.719693+0000","last_fullsized":"2023-12-19T18:22:13.719693+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:29.507646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.730782+0000","last_change":"2023-12-19T18:22:13.730665+0000","last_active":"2023-12-19T18:22:13.730782+0000","last_peered":"2023-12-19T18:22:13.730782+0000","last_clean":"2023-12-19T18:22:13.730782+0000","last_became_active":"2023-12-19T18:22:13.729630+0000","last_became_peered":"2023-12-19T18:22:13.729630+0000","last_unstale":"2023-12-19T18:22:13.730782+0000","last_undegraded":"2023-12-19T18:22:13.730782+0000","last_fullsized":"2023-12-19T18:22:13.730782+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:01:35.189032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.724524+0000","last_change":"2023-12-19T18:22:13.724524+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.724524+0000","last_undegraded":"2023-12-19T18:22:13.724524+0000","last_fullsized":"2023-12-19T18:22:13.724524+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:12.207051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.732244+0000","last_change":"2023-12-19T18:22:13.731257+0000","last_active":"2023-12-19T18:22:13.732244+0000","last_peered":"2023-12-19T18:22:13.732244+0000","last_clean":"2023-12-19T18:22:13.732244+0000","last_became_active":"2023-12-19T18:22:13.730910+0000","last_became_peered":"2023-12-19T18:22:13.730910+0000","last_unstale":"2023-12-19T18:22:13.732244+0000","last_undegraded":"2023-12-19T18:22:13.732244+0000","last_fullsized":"2023-12-19T18:22:13.732244+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.708423+0000","last_change":"2023-12-19T18:22:13.708423+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.708423+0000","last_undegraded":"2023-12-19T18:22:13.708423+0000","last_fullsized":"2023-12-19T18:22:13.708423+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:44.388621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.716654+0000","last_change":"2023-12-19T18:22:13.714716+0000","last_active":"2023-12-19T18:22:13.716654+0000","last_peered":"2023-12-19T18:22:13.716654+0000","last_clean":"2023-12-19T18:22:13.716654+0000","last_became_active":"2023-12-19T18:22:13.713977+0000","last_became_peered":"2023-12-19T18:22:13.713977+0000","last_unstale":"2023-12-19T18:22:13.716654+0000","last_undegraded":"2023-12-19T18:22:13.716654+0000","last_fullsized":"2023-12-19T18:22:13.716654+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.718774+0000","last_change":"2023-12-19T18:22:13.718774+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.718774+0000","last_undegraded":"2023-12-19T18:22:13.718774+0000","last_fullsized":"2023-12-19T18:22:13.718774+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:45:23.784185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.731485+0000","last_change":"2023-12-19T18:22:13.729500+0000","last_active":"2023-12-19T18:22:13.731485+0000","last_peered":"2023-12-19T18:22:13.731485+0000","last_clean":"2023-12-19T18:22:13.731485+0000","last_became_active":"2023-12-19T18:22:13.725715+0000","last_became_peered":"2023-12-19T18:22:13.725715+0000","last_unstale":"2023-12-19T18:22:13.731485+0000","last_undegraded":"2023-12-19T18:22:13.731485+0000","last_fullsized":"2023-12-19T18:22:13.731485+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:30:23.979686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.721429+0000","last_change":"2023-12-19T18:22:13.721429+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.721429+0000","last_undegraded":"2023-12-19T18:22:13.721429+0000","last_fullsized":"2023-12-19T18:22:13.721429+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:43:39.731660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.726875+0000","last_change":"2023-12-19T18:22:13.726768+0000","last_active":"2023-12-19T18:22:13.726875+0000","last_peered":"2023-12-19T18:22:13.726875+0000","last_clean":"2023-12-19T18:22:13.726875+0000","last_became_active":"2023-12-19T18:22:13.722319+0000","last_became_peered":"2023-12-19T18:22:13.722319+0000","last_unstale":"2023-12-19T18:22:13.726875+0000","last_undegraded":"2023-12-19T18:22:13.726875+0000","last_fullsized":"2023-12-19T18:22:13.726875+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:11.192516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.707706+0000","last_change":"2023-12-19T18:22:13.707706+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.707706+0000","last_undegraded":"2023-12-19T18:22:13.707706+0000","last_fullsized":"2023-12-19T18:22:13.707706+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:09.942336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.713603+0000","last_change":"2023-12-19T18:22:13.713351+0000","last_active":"2023-12-19T18:22:13.713603+0000","last_peered":"2023-12-19T18:22:13.713603+0000","last_clean":"2023-12-19T18:22:13.713603+0000","last_became_active":"2023-12-19T18:22:13.712581+0000","last_became_peered":"2023-12-19T18:22:13.712581+0000","last_unstale":"2023-12-19T18:22:13.713603+0000","last_undegraded":"2023-12-19T18:22:13.713603+0000","last_fullsized":"2023-12-19T18:22:13.713603+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:36.103516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.714719+0000","last_change":"2023-12-19T18:22:13.714719+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.714719+0000","last_undegraded":"2023-12-19T18:22:13.714719+0000","last_fullsized":"2023-12-19T18:22:13.714719+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:08.710594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.720004+0000","last_change":"2023-12-19T18:22:13.718488+0000","last_active":"2023-12-19T18:22:13.720004+0000","last_peered":"2023-12-19T18:22:13.720004+0000","last_clean":"2023-12-19T18:22:13.720004+0000","last_became_active":"2023-12-19T18:22:13.717351+0000","last_became_peered":"2023-12-19T18:22:13.717351+0000","last_unstale":"2023-12-19T18:22:13.720004+0000","last_undegraded":"2023-12-19T18:22:13.720004+0000","last_fullsized":"2023-12-19T18:22:13.720004+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:49:11.599052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.717816+0000","last_change":"2023-12-19T18:22:13.717816+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.717816+0000","last_undegraded":"2023-12-19T18:22:13.717816+0000","last_fullsized":"2023-12-19T18:22:13.717816+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:24.011514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.733864+0000","last_change":"2023-12-19T18:22:13.733542+0000","last_active":"2023-12-19T18:22:13.733864+0000","last_peered":"2023-12-19T18:22:13.733864+0000","last_clean":"2023-12-19T18:22:13.733864+0000","last_became_active":"2023-12-19T18:22:13.733273+0000","last_became_peered":"2023-12-19T18:22:13.733273+0000","last_unstale":"2023-12-19T18:22:13.733864+0000","last_undegraded":"2023-12-19T18:22:13.733864+0000","last_fullsized":"2023-12-19T18:22:13.733864+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:07.352249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.721501+0000","last_change":"2023-12-19T18:22:13.721501+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.721501+0000","last_undegraded":"2023-12-19T18:22:13.721501+0000","last_fullsized":"2023-12-19T18:22:13.721501+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:08.470720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.723182+0000","last_change":"2023-12-19T18:22:13.723012+0000","last_active":"2023-12-19T18:22:13.723182+0000","last_peered":"2023-12-19T18:22:13.723182+0000","last_clean":"2023-12-19T18:22:13.723182+0000","last_became_active":"2023-12-19T18:22:13.717308+0000","last_became_peered":"2023-12-19T18:22:13.717308+0000","last_unstale":"2023-12-19T18:22:13.723182+0000","last_undegraded":"2023-12-19T18:22:13.723182+0000","last_fullsized":"2023-12-19T18:22:13.723182+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:01.277914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.715269+0000","last_change":"2023-12-19T18:22:13.715269+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.715269+0000","last_undegraded":"2023-12-19T18:22:13.715269+0000","last_fullsized":"2023-12-19T18:22:13.715269+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:30:01.499864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.736548+0000","last_change":"2023-12-19T18:22:13.736355+0000","last_active":"2023-12-19T18:22:13.736548+0000","last_peered":"2023-12-19T18:22:13.736548+0000","last_clean":"2023-12-19T18:22:13.736548+0000","last_became_active":"2023-12-19T18:22:13.736056+0000","last_became_peered":"2023-12-19T18:22:13.736056+0000","last_unstale":"2023-12-19T18:22:13.736548+0000","last_undegraded":"2023-12-19T18:22:13.736548+0000","last_fullsized":"2023-12-19T18:22:13.736548+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:47.766382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.716628+0000","last_change":"2023-12-19T18:22:13.716628+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.716628+0000","last_undegraded":"2023-12-19T18:22:13.716628+0000","last_fullsized":"2023-12-19T18:22:13.716628+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:08.366785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.716737+0000","last_change":"2023-12-19T18:22:13.715051+0000","last_active":"2023-12-19T18:22:13.716737+0000","last_peered":"2023-12-19T18:22:13.716737+0000","last_clean":"2023-12-19T18:22:13.716737+0000","last_became_active":"2023-12-19T18:22:13.714392+0000","last_became_peered":"2023-12-19T18:22:13.714392+0000","last_unstale":"2023-12-19T18:22:13.716737+0000","last_undegraded":"2023-12-19T18:22:13.716737+0000","last_fullsized":"2023-12-19T18:22:13.716737+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:00.018590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":1,"reported_epoch":133,"state":"creating+peering","last_fresh":"2023-12-19T18:22:13.708590+0000","last_change":"2023-12-19T18:22:13.708590+0000","last_active":"2023-12-19T18:22:13.691897+0000","last_peered":"2023-12-19T18:22:13.691897+0000","last_clean":"2023-12-19T18:22:13.691897+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-19T18:22:13.708590+0000","last_undegraded":"2023-12-19T18:22:13.708590+0000","last_fullsized":"2023-12-19T18:22:13.708590+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:16.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":9,"reported_epoch":133,"state":"active+clean","last_fresh":"2023-12-19T18:22:13.726584+0000","last_change":"2023-12-19T18:22:13.725717+0000","last_active":"2023-12-19T18:22:13.726584+0000","last_peered":"2023-12-19T18:22:13.726584+0000","last_clean":"2023-12-19T18:22:13.726584+0000","last_became_active":"2023-12-19T18:22:13.724435+0000","last_became_peered":"2023-12-19T18:22:13.724435+0000","last_unstale":"2023-12-19T18:22:13.726584+0000","last_undegraded":"2023-12-19T18:22:13.726584+0000","last_fullsized":"2023-12-19T18:22:13.726584+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:00:05.094962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":265,"reported_epoch":132,"state":"active+clean","last_fresh":"2023-12-19T18:22:12.698399+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:22:12.698399+0000","last_peered":"2023-12-19T18:22:12.698399+0000","last_clean":"2023-12-19T18:22:12.698399+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:22:12.698399+0000","last_undegraded":"2023-12-19T18:22:12.698399+0000","last_fullsized":"2023-12-19T18:22:12.698399+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2310000000000001,"15min":1.1519999999999999},"min":{"1min":0.57899999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9630000000000001,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":1.272,"5min":1.278,"15min":1.1890000000000001},"min":{"1min":0.76900000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6990000000000001,"5min":1.79,"15min":1.79},"last":1.4219999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.3480000000000001,"15min":1.23},"min":{"1min":0.752,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.7010000000000001,"5min":10.926,"15min":10.926},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.365,"15min":1.244},"min":{"1min":0.80900000000000005,"5min":0.71899999999999997,"15min":0.67900000000000005},"max":{"1min":1.6240000000000001,"5min":10.904999999999999,"15min":10.904999999999999},"last":1.6579999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.161,"15min":1.1040000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.427,"5min":3.1960000000000002,"15min":3.1960000000000002},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.266,"15min":1.1930000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0670000000000002,"5min":6.008,"15min":6.008},"last":1.226}]},{"osd":3,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.28,"15min":1.1619999999999999},"min":{"1min":0.66900000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.595,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4179999999999999,"15min":1.294},"min":{"1min":0.85699999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.157,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":0.99199999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.258,"15min":1.123},"min":{"1min":0.88600000000000001,"5min":0.65400000000000003,"15min":0.47899999999999998},"max":{"1min":1.8009999999999999,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.222,"5min":1.304,"15min":1.2410000000000001},"min":{"1min":0.94599999999999995,"5min":0.72999999999999998,"15min":0.57599999999999996},"max":{"1min":1.6759999999999999,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.369}]},{"osd":5,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.399,"15min":1.333},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.55000000000000004},"max":{"1min":1.8620000000000001,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":1.141},{"interface":"front","average":{"1min":1.385,"5min":1.4339999999999999,"15min":1.337},"min":{"1min":0.80700000000000005,"5min":0.71599999999999997,"15min":0.53500000000000003},"max":{"1min":2.1179999999999999,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":1.8280000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.5669999999999999,"15min":1.4690000000000001},"min":{"1min":1.036,"5min":0.78600000000000003,"15min":0.59899999999999998},"max":{"1min":2.423,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.742},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.49,"15min":1.3799999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.63200000000000001},"max":{"1min":2.0179999999999998,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":1.3280000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.121,"15min":1.012},"min":{"1min":0.79800000000000004,"5min":0.66700000000000004,"15min":0.65800000000000003},"max":{"1min":1.6259999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.751},{"interface":"front","average":{"1min":1.163,"5min":1.1870000000000001,"15min":1.075},"min":{"1min":0.78400000000000003,"5min":0.64300000000000002,"15min":0.55600000000000005},"max":{"1min":1.663,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":1.1100000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.1699999999999999,"15min":1.1319999999999999},"min":{"1min":0.86599999999999999,"5min":0.71399999999999997,"15min":0.68200000000000005},"max":{"1min":1.5609999999999999,"5min":4.5540000000000003,"15min":4.5540000000000003},"last":1.0589999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.226,"15min":1.1699999999999999},"min":{"1min":0.86599999999999999,"5min":0.625,"15min":0.60299999999999998},"max":{"1min":1.627,"5min":1.885,"15min":1.885},"last":1.3220000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.254,"15min":1.151},"min":{"1min":0.84799999999999998,"5min":0.626,"15min":0.626},"max":{"1min":1.583,"5min":1.821,"15min":1.821},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.218,"15min":1.2310000000000001},"min":{"1min":0.83599999999999997,"5min":0.81999999999999995,"15min":0.79900000000000004},"max":{"1min":1.821,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":0.91500000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.294,"15min":1.202},"min":{"1min":1.0129999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.722,"5min":3.8820000000000001,"15min":3.8820000000000001},"last":1.024},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2170000000000001,"15min":1.1699999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6559999999999999,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.94999999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.296,"15min":1.2450000000000001},"min":{"1min":0.74099999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6299999999999999,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.458},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.431,"15min":1.347},"min":{"1min":0.76700000000000002,"5min":0.70799999999999996,"15min":0.57999999999999996},"max":{"1min":1.774,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.367}]},{"osd":5,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2829999999999999,"15min":1.1879999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7330000000000001,"5min":4.6680000000000001,"15min":4.6680000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.4450000000000001,"15min":1.3560000000000001},"min":{"1min":0.84799999999999998,"5min":0.626,"15min":0.626},"max":{"1min":4.1159999999999997,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.405}]},{"osd":7,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4430000000000001,"15min":1.349},"min":{"1min":0.92700000000000005,"5min":0.82199999999999995,"15min":0.61899999999999999},"max":{"1min":1.7010000000000001,"5min":3.9670000000000001,"15min":3.9670000000000001},"last":1.2410000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.4470000000000001,"15min":1.357},"min":{"1min":0.81899999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8169999999999999,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":1.4370000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574940,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1293,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194019,"statfs":{"total":200000000,"available":198675466,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.23,"15min":1.1679999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.65200000000000002},"max":{"1min":1.6830000000000001,"5min":1.837,"15min":1.863},"last":0.76300000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2729999999999999,"15min":1.1870000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.571,"5min":1.879,"15min":1.879},"last":1.4410000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2090000000000001,"15min":1.0629999999999999},"min":{"1min":0.76200000000000001,"5min":0.70899999999999996,"15min":0.65700000000000003},"max":{"1min":1.782,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.167},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1599999999999999,"15min":1.0660000000000001},"min":{"1min":0.81899999999999995,"5min":0.71299999999999997,"15min":0.61299999999999999},"max":{"1min":1.5640000000000001,"5min":1.6359999999999999,"15min":1.843},"last":1.089}]},{"osd":2,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.155,"15min":1.0860000000000001},"min":{"1min":0.81799999999999995,"5min":0.74299999999999999,"15min":0.59499999999999997},"max":{"1min":1.3069999999999999,"5min":1.794,"15min":1.794},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.258,"15min":1.147},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.65200000000000002},"max":{"1min":1.5920000000000001,"5min":2.036,"15min":2.036},"last":1.2589999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2010000000000001,"15min":1.097},"min":{"1min":0.83699999999999997,"5min":0.71999999999999997,"15min":0.68899999999999995},"max":{"1min":1.72,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3320000000000001,"15min":1.23},"min":{"1min":0.91300000000000003,"5min":0.75700000000000001,"15min":0.70399999999999996},"max":{"1min":1.595,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.3160000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3069999999999999,"15min":1.179},"min":{"1min":0.95299999999999996,"5min":0.60899999999999999,"15min":0.46700000000000003},"max":{"1min":1.6479999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.395},{"interface":"front","average":{"1min":1.25,"5min":1.1990000000000001,"15min":1.131},"min":{"1min":0.90500000000000003,"5min":0.73999999999999999,"15min":0.57099999999999995},"max":{"1min":1.7689999999999999,"5min":1.804,"15min":1.901},"last":1.0529999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4299999999999999,"15min":1.3360000000000001},"min":{"1min":1.054,"5min":0.66600000000000004,"15min":0.54400000000000004},"max":{"1min":1.839,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.518},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.319,"15min":1.2470000000000001},"min":{"1min":1.163,"5min":0.81200000000000006,"15min":0.49299999999999999},"max":{"1min":1.702,"5min":1.7949999999999999,"15min":1.976},"last":1.1399999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.274,"15min":1.2290000000000001},"min":{"1min":1.0189999999999999,"5min":0.68300000000000005,"15min":0.50900000000000001},"max":{"1min":1.6779999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.681},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4570000000000001,"15min":1.4119999999999999},"min":{"1min":1.2310000000000001,"5min":0.84999999999999998,"15min":0.623},"max":{"1min":1.8120000000000001,"5min":2.117,"15min":2.117},"last":1.2230000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98899999999999999,"15min":0.998},"min":{"1min":0.68500000000000005,"5min":0.62,"15min":0.62},"max":{"1min":1.3899999999999999,"5min":1.579,"15min":1.579},"last":0.69099999999999995},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0409999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4259999999999999,"5min":1.7470000000000001,"15min":1.7909999999999999},"last":1.5009999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1679999999999999,"15min":1.149},"min":{"1min":0.67200000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.589,"5min":1.7130000000000001,"15min":1.8160000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.29,"5min":1.2549999999999999,"15min":1.1850000000000001},"min":{"1min":0.77400000000000002,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.696},"last":0.81200000000000006}]},{"osd":2,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1499999999999999,"15min":1.127},"min":{"1min":0.82399999999999995,"5min":0.749,"15min":0.69799999999999995},"max":{"1min":1.407,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.1060000000000001},"min":{"1min":0.872,"5min":0.72199999999999998,"15min":0.70499999999999996},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.2010000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.294,"15min":1.26},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.369,"5min":1.3340000000000001,"15min":1.304},"min":{"1min":0.93100000000000005,"5min":0.82399999999999995,"15min":0.74399999999999999},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.369}]},{"osd":5,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.294,"15min":1.2030000000000001},"min":{"1min":0.90000000000000002,"5min":0.68600000000000005,"15min":0.44500000000000001},"max":{"1min":1.46,"5min":1.76,"15min":1.8740000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.413,"15min":1.353},"min":{"1min":0.76500000000000001,"5min":0.625,"15min":0.504},"max":{"1min":1.827,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.1160000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3160000000000001,"15min":1.304},"min":{"1min":0.98399999999999999,"5min":0.71299999999999997,"15min":0.432},"max":{"1min":1.5509999999999999,"5min":1.7809999999999999,"15min":1.853},"last":1.258},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4630000000000001,"15min":1.3959999999999999},"min":{"1min":1.2170000000000001,"5min":0.73599999999999999,"15min":0.64500000000000002},"max":{"1min":1.7829999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.3029999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.3999999999999999,"15min":1.3480000000000001},"min":{"1min":1.0249999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.7589999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4379999999999999,"15min":1.4079999999999999},"min":{"1min":0.88500000000000001,"5min":0.60399999999999998,"15min":0.57399999999999995},"max":{"1min":1.8,"5min":1.905,"15min":1.9330000000000001},"last":1.5880000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574941,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.52900000000000003,"5min":0.38100000000000001,"15min":0.35299999999999998},"max":{"1min":2.516,"5min":3.3140000000000001,"15min":3.4350000000000001},"last":0.80300000000000005},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1,"15min":0.94499999999999995},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.3490000000000002,"5min":4.3490000000000002,"15min":4.3490000000000002},"last":1.5860000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.0629999999999999,"15min":1.0620000000000001},"min":{"1min":0.45300000000000001,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":5.2510000000000003,"5min":5.2510000000000003,"15min":5.2510000000000003},"last":0.92200000000000004},{"interface":"front","average":{"1min":1.403,"5min":1.1719999999999999,"15min":1.1160000000000001},"min":{"1min":0.63800000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":5.4119999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.1299999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.377,"15min":1.25},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.335,"5min":13.833,"15min":13.833},"last":1.232},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5349999999999999,"15min":1.306},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.53200000000000003},"max":{"1min":5.4480000000000004,"5min":16.748999999999999,"15min":16.748999999999999},"last":1.5369999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.3620000000000001,"15min":1.2330000000000001},"min":{"1min":0.81899999999999995,"5min":0.77400000000000002,"15min":0.65600000000000003},"max":{"1min":5.1790000000000003,"5min":5.1790000000000003,"15min":5.1790000000000003},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.591,"5min":1.4239999999999999,"15min":1.409},"min":{"1min":0.873,"5min":0.80800000000000005,"15min":0.73199999999999998},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":1.427}]},{"osd":5,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.4750000000000001,"15min":1.45},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.76000000000000001},"max":{"1min":5.3289999999999997,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":1.371},{"interface":"front","average":{"1min":1.46,"5min":1.367,"15min":1.2789999999999999},"min":{"1min":0.83799999999999997,"5min":0.80600000000000005,"15min":0.69999999999999996},"max":{"1min":4.6369999999999996,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.323}]},{"osd":6,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.4630000000000001,"15min":1.476},"min":{"1min":0.81999999999999995,"5min":0.81799999999999995,"15min":0.81599999999999995},"max":{"1min":4.5039999999999996,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":1.502},{"interface":"front","average":{"1min":1.667,"5min":1.548,"15min":1.514},"min":{"1min":0.83999999999999997,"5min":0.83199999999999996,"15min":0.80500000000000005},"max":{"1min":5.3739999999999997,"5min":5.3739999999999997,"15min":5.3739999999999997},"last":1.6759999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.476,"15min":1.385},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.73199999999999998},"max":{"1min":5.1349999999999998,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.4770000000000001},{"interface":"front","average":{"1min":1.712,"5min":1.585,"15min":1.554},"min":{"1min":0.92200000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":4.4619999999999997,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":1.6279999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574940,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.075,"15min":1.0109999999999999},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.34399999999999997},"max":{"1min":1.506,"5min":9.2550000000000008,"15min":9.2550000000000008},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0409999999999999,"15min":1.0389999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.7649999999999999,"5min":2.387,"15min":4.7850000000000001},"last":0.69499999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95999999999999996,"15min":1.165},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.46000000000000002},"max":{"1min":1.675,"5min":2.4390000000000001,"15min":8.2720000000000002},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.107,"15min":1.1619999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6519999999999999,"5min":2.7200000000000002,"15min":5.0620000000000003},"last":0.76500000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1830000000000001,"15min":1.3620000000000001},"min":{"1min":0.63300000000000001,"5min":0.54200000000000004,"15min":0.51900000000000002},"max":{"1min":1.708,"5min":2.238,"15min":8.3109999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.163,"15min":1.327},"min":{"1min":0.69599999999999995,"5min":0.46300000000000002,"15min":0.41799999999999998},"max":{"1min":1.73,"5min":2.323,"15min":8.0690000000000008},"last":0.64000000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.417,"15min":1.4610000000000001},"min":{"1min":0.93000000000000005,"5min":0.83399999999999996,"15min":0.79300000000000004},"max":{"1min":1.8009999999999999,"5min":2.6139999999999999,"15min":5.3929999999999998},"last":1.081},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.468,"15min":1.5489999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.78300000000000003},"max":{"1min":2.1949999999999998,"5min":3.1070000000000002,"15min":7.3570000000000002},"last":1.1100000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3939999999999999,"15min":1.546},"min":{"1min":0.93200000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9199999999999999,"5min":2.5579999999999998,"15min":8.1839999999999993},"last":1.052},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.417},"min":{"1min":0.99299999999999999,"5min":0.81000000000000005,"15min":0.72199999999999998},"max":{"1min":1.845,"5min":2.4940000000000002,"15min":5.165},"last":0.86499999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.4350000000000001,"15min":1.454},"min":{"1min":0.82399999999999995,"5min":0.752,"15min":0.72999999999999998},"max":{"1min":2.5139999999999998,"5min":3.0179999999999998,"15min":5.1059999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":1.446,"5min":1.4650000000000001,"15min":1.5589999999999999},"min":{"1min":1.0049999999999999,"5min":0.81200000000000006,"15min":0.77000000000000002},"max":{"1min":1.879,"5min":2.9580000000000002,"15min":8.1370000000000005},"last":1.153}]},{"osd":7,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.5149999999999999,"15min":1.552},"min":{"1min":0.98499999999999999,"5min":0.79200000000000004,"15min":0.78200000000000003},"max":{"1min":2.3700000000000001,"5min":3.2189999999999999,"15min":5.1449999999999996},"last":1.016},{"interface":"front","average":{"1min":1.696,"5min":1.548,"15min":1.659},"min":{"1min":1.054,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":4.8289999999999997,"5min":4.8289999999999997,"15min":8.2300000000000004},"last":0.90000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1293,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194019,"statfs":{"total":200000000,"available":198675466,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.84599999999999997},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3660000000000001,"5min":1.52,"15min":1.52},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.40000000000000002}]},{"osd":2,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0720000000000001,"15min":1.0609999999999999},"min":{"1min":0.52500000000000002,"5min":0.42299999999999999,"15min":0.41399999999999998},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.5739999999999998},"last":0.72599999999999998},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.026,"15min":0.93600000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":1.6839999999999999,"5min":1.845,"15min":1.845},"last":0.79900000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3049999999999999,"15min":1.1719999999999999},"min":{"1min":0.64200000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.1040000000000001,"5min":2.137,"15min":2.137},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.2170000000000001,"15min":1.1930000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.44900000000000001},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.3959999999999999},"last":0.63600000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3520000000000001,"15min":1.2989999999999999},"min":{"1min":0.83999999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1440000000000001,"5min":2.444,"15min":2.444},"last":0.76400000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4039999999999999,"15min":1.3100000000000001},"min":{"1min":0.95299999999999996,"5min":0.76900000000000002,"15min":0.71099999999999997},"max":{"1min":1.7569999999999999,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.034}]},{"osd":5,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.425,"15min":1.3100000000000001},"min":{"1min":0.92800000000000005,"5min":0.88,"15min":0.88},"max":{"1min":1.915,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5049999999999999,"15min":1.423},"min":{"1min":1.0009999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.371,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.0089999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4530000000000001,"15min":1.3899999999999999},"min":{"1min":0.88400000000000001,"5min":0.875,"15min":0.76900000000000002},"max":{"1min":2.3610000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.82899999999999996},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.538,"15min":1.46},"min":{"1min":1.032,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.2719999999999998,"5min":2.895,"15min":2.895},"last":1.0529999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.571,"15min":1.397},"min":{"1min":1.018,"5min":0.879,"15min":0.69999999999999996},"max":{"1min":1.8129999999999999,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":1.1200000000000001},{"interface":"front","average":{"1min":1.468,"5min":1.659,"15min":1.508},"min":{"1min":0.97399999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.2360000000000002,"5min":9.4109999999999996,"15min":9.4109999999999996},"last":0.874}]}]},{"osd":0,"up_from":13,"seq":55834574940,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":716,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194595,"statfs":{"total":200000000,"available":199265834,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.99399999999999999,"15min":0.90600000000000003},"min":{"1min":0.64000000000000001,"5min":0.46700000000000003,"15min":0.39900000000000002},"max":{"1min":1.3220000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.143,"15min":1.0429999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.0369999999999999,"5min":2.1040000000000001,"15min":2.4180000000000001},"last":1.3460000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0660000000000001,"15min":0.96299999999999997},"min":{"1min":0.73099999999999998,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":1.4810000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0389999999999999,"15min":0.94699999999999995},"min":{"1min":0.55100000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.335,"5min":2.7810000000000001,"15min":2.7810000000000001},"last":0.70299999999999996}]},{"osd":3,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0880000000000001,"15min":1.0209999999999999},"min":{"1min":0.61299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.732,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.999},{"interface":"front","average":{"1min":1.306,"5min":1.3580000000000001,"15min":1.2390000000000001},"min":{"1min":0.746,"5min":0.63100000000000001,"15min":0.47499999999999998},"max":{"1min":1.982,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.411}]},{"osd":4,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.335,"15min":1.216},"min":{"1min":0.86299999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6739999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.282},{"interface":"front","average":{"1min":1.343,"5min":1.3360000000000001,"15min":1.2929999999999999},"min":{"1min":1.0409999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.663,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.119}]},{"osd":5,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.5489999999999999,"15min":1.4059999999999999},"min":{"1min":1.0169999999999999,"5min":0.71099999999999997,"15min":0.64500000000000002},"max":{"1min":1.877,"5min":16.298999999999999,"15min":16.298999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.385,"15min":1.2909999999999999},"min":{"1min":0.94899999999999995,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8109999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.696}]},{"osd":6,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.5029999999999999,"15min":1.409},"min":{"1min":1.171,"5min":0.85499999999999998,"15min":0.75900000000000001},"max":{"1min":1.9119999999999999,"5min":2.2559999999999998,"15min":2.5910000000000002},"last":1.776},{"interface":"front","average":{"1min":1.391,"5min":1.462,"15min":1.369},"min":{"1min":1.0349999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.776,"5min":2.46,"15min":2.46},"last":1.3819999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5369999999999999,"15min":1.409},"min":{"1min":1.1279999999999999,"5min":0.89200000000000002,"15min":0.70899999999999996},"max":{"1min":1.8740000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.466,"15min":1.4330000000000001},"min":{"1min":0.88400000000000001,"5min":0.873,"15min":0.81999999999999995},"max":{"1min":2.113,"5min":2.4580000000000002,"15min":3.1429999999999998},"last":1.3100000000000001}]}]}],"pool_statfs":[]}} 2023-12-19T18:22:19.791 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:22:20.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.068+0000 7fe4b021e640 1 Processor -- start 2023-12-19T18:22:20.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.068+0000 7fe4b021e640 1 -- start start 2023-12-19T18:22:20.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.069+0000 7fe4b021e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe4a80cdf90 0x7fe4a80ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:20.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.069+0000 7fe4b021e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4a80d1e00 0x7fe4a80d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:20.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.069+0000 7fe4b021e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4a80d1970 con 0x7fe4a815d240 2023-12-19T18:22:20.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.069+0000 7fe4b021e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a80ce960 con 0x7fe4a80d1e00 2023-12-19T18:22:20.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.069+0000 7fe4b021e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4a806a5e0 con 0x7fe4a80cdf90 2023-12-19T18:22:20.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.069+0000 7fe4adfba640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe4a815d240 0x7fe4a81618d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41276/0 (socket says 172.21.15.27:41276) 2023-12-19T18:22:20.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.069+0000 7fe4adfba640 1 -- 172.21.15.27:0/3654711442 learned_addr learned my addr 172.21.15.27:0/3654711442 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:20.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.069+0000 7fe4ae7bb640 1 --2- 172.21.15.27:0/3654711442 >> v2:172.21.15.27:3300/0 conn(0x7fe4a80d1e00 0x7fe4a80d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:20.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.070+0000 7fe4acfb8640 1 -- 172.21.15.27:0/3654711442 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251021827 0 0) 0x7fe4a80d1970 con 0x7fe4a815d240 2023-12-19T18:22:20.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.070+0000 7fe4ad7b9640 1 --2- 172.21.15.27:0/3654711442 >> v2:172.21.15.192:3300/0 conn(0x7fe4a80cdf90 0x7fe4a80ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:20.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.070+0000 7fe4acfb8640 1 -- 172.21.15.27:0/3654711442 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe494003520 con 0x7fe4a815d240 2023-12-19T18:22:20.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.070+0000 7fe4ae7bb640 1 -- 172.21.15.27:0/3654711442 >> v2:172.21.15.192:3300/0 conn(0x7fe4a80cdf90 msgr2=0x7fe4a80ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:20.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.070+0000 7fe4ae7bb640 1 --2- 172.21.15.27:0/3654711442 >> v2:172.21.15.192:3300/0 conn(0x7fe4a80cdf90 0x7fe4a80ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.070+0000 7fe4ae7bb640 1 -- 172.21.15.27:0/3654711442 >> v1:172.21.15.27:6789/0 conn(0x7fe4a815d240 legacy=0x7fe4a81618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:20.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.070+0000 7fe4ae7bb640 1 -- 172.21.15.27:0/3654711442 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a806a880 con 0x7fe4a80d1e00 2023-12-19T18:22:20.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.071+0000 7fe4ae7bb640 1 --2- 172.21.15.27:0/3654711442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4a80d1e00 0x7fe4a80d2270 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80d1550 tx=0x7fe4a002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0960d5dd43ac017 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:20.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.072+0000 7fe4acfb8640 1 -- 172.21.15.27:0/3654711442 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a002cd00 con 0x7fe4a80d1e00 2023-12-19T18:22:20.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.072+0000 7fe4acfb8640 1 -- 172.21.15.27:0/3654711442 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4a002ce60 con 0x7fe4a80d1e00 2023-12-19T18:22:20.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.072+0000 7fe4acfb8640 1 -- 172.21.15.27:0/3654711442 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0034660 con 0x7fe4a80d1e00 2023-12-19T18:22:20.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.072+0000 7fe4b021e640 1 -- 172.21.15.27:0/3654711442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4a80d1e00 msgr2=0x7fe4a80d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:20.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.072+0000 7fe4b021e640 1 --2- 172.21.15.27:0/3654711442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4a80d1e00 0x7fe4a80d2270 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fe4a80d1550 tx=0x7fe4a002bf50 comp rx=0 tx=0).stop 2023-12-19T18:22:20.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.072+0000 7fe4b021e640 1 -- 172.21.15.27:0/3654711442 shutdown_connections 2023-12-19T18:22:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.072+0000 7fe4b021e640 1 --2- 172.21.15.27:0/3654711442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4a80d1e00 0x7fe4a80d2270 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.072+0000 7fe4b021e640 1 --2- 172.21.15.27:0/3654711442 >> v2:172.21.15.192:3300/0 conn(0x7fe4a80cdf90 0x7fe4a80ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.072+0000 7fe4b021e640 1 -- 172.21.15.27:0/3654711442 >> 172.21.15.27:0/3654711442 conn(0x7fe4a80d31e0 msgr2=0x7fe4a80d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.072+0000 7fe4b021e640 1 -- 172.21.15.27:0/3654711442 shutdown_connections 2023-12-19T18:22:20.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.072+0000 7fe4b021e640 1 -- 172.21.15.27:0/3654711442 wait complete. 2023-12-19T18:22:20.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.074+0000 7fe4b021e640 1 Processor -- start 2023-12-19T18:22:20.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.074+0000 7fe4b021e640 1 -- start start 2023-12-19T18:22:20.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.075+0000 7fe4b021e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4a80cdf90 0x7fe4a81f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:20.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.075+0000 7fe4b021e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4a80d1e00 0x7fe4a81f39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:20.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.075+0000 7fe4b021e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4a815d240 0x7fe4a81f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:20.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.075+0000 7fe4b021e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a814f9f0 con 0x7fe4a80d1e00 2023-12-19T18:22:20.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.075+0000 7fe4b021e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a8150570 con 0x7fe4a815d240 2023-12-19T18:22:20.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.075+0000 7fe4b021e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4a81f2680 con 0x7fe4a80cdf90 2023-12-19T18:22:20.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.075+0000 7fe4ad7b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4a80d1e00 0x7fe4a81f39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:20.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.076+0000 7fe4ad7b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4a80d1e00 0x7fe4a81f39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43010/0 (socket says 172.21.15.27:43010) 2023-12-19T18:22:20.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.076+0000 7fe4ad7b9640 1 -- 172.21.15.27:0/2996745851 learned_addr learned my addr 172.21.15.27:0/2996745851 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:20.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.076+0000 7fe4ad7b9640 1 -- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4a80cdf90 msgr2=0x7fe4a81f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:20.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.076+0000 7fe4adfba640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4a80cdf90 0x7fe4a81f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:20.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.076+0000 7fe4ae7bb640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4a815d240 0x7fe4a81f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.076+0000 7fe4ad7b9640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4a80cdf90 0x7fe4a81f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.076+0000 7fe4ad7b9640 1 -- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4a815d240 msgr2=0x7fe4a81f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.076+0000 7fe4ad7b9640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4a815d240 0x7fe4a81f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.076+0000 7fe4ad7b9640 1 -- 172.21.15.27:0/2996745851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4a0009a70 con 0x7fe4a80d1e00 2023-12-19T18:22:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.076+0000 7fe4adfba640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4a80cdf90 0x7fe4a81f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.076+0000 7fe4ae7bb640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4a815d240 0x7fe4a81f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.076+0000 7fe4ad7b9640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4a80d1e00 0x7fe4a81f39c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe49800d800 tx=0x7fe49800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.077+0000 7fe49e7fc640 1 -- 172.21.15.27:0/2996745851 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49800b6c0 con 0x7fe4a80d1e00 2023-12-19T18:22:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.077+0000 7fe49e7fc640 1 -- 172.21.15.27:0/2996745851 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49800b820 con 0x7fe4a80d1e00 2023-12-19T18:22:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.077+0000 7fe49e7fc640 1 -- 172.21.15.27:0/2996745851 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49800bbb0 con 0x7fe4a80d1e00 2023-12-19T18:22:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.077+0000 7fe4b021e640 1 -- 172.21.15.27:0/2996745851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4a814fc40 con 0x7fe4a80d1e00 2023-12-19T18:22:20.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.077+0000 7fe4b021e640 1 -- 172.21.15.27:0/2996745851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4a8150200 con 0x7fe4a80d1e00 2023-12-19T18:22:20.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.078+0000 7fe4b021e640 1 -- 172.21.15.27:0/2996745851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4a80d1580 con 0x7fe4a80d1e00 2023-12-19T18:22:20.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.079+0000 7fe49e7fc640 1 -- 172.21.15.27:0/2996745851 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe49800bd10 con 0x7fe4a80d1e00 2023-12-19T18:22:20.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.080+0000 7fe49e7fc640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe48c06a460 0x7fe48c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:20.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.080+0000 7fe49e7fc640 1 -- 172.21.15.27:0/2996745851 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe498088ec0 con 0x7fe4a80d1e00 2023-12-19T18:22:20.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.080+0000 7fe4adfba640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe48c06a460 0x7fe48c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:20.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.081+0000 7fe4adfba640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe48c06a460 0x7fe48c06c920 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40011e0 tx=0x7fe4a4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:20.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.084+0000 7fe49e7fc640 1 -- 172.21.15.27:0/2996745851 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe498054410 con 0x7fe4a80d1e00 2023-12-19T18:22:20.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.276+0000 7fe4b021e640 1 -- 172.21.15.27:0/2996745851 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe4a806ae40 con 0x7fe48c06a460 2023-12-19T18:22:20.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.298+0000 7fe49e7fc640 1 -- 172.21.15.27:0/2996745851 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283057 (secure 0 0 0) 0x7fe4a4001c80 con 0x7fe48c06a460 2023-12-19T18:22:20.303 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:20.304 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:22:20.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.305+0000 7fe4b021e640 1 -- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe48c06a460 msgr2=0x7fe48c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:20.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.305+0000 7fe4b021e640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe48c06a460 0x7fe48c06c920 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fe4a40011e0 tx=0x7fe4a4009040 comp rx=0 tx=0).stop 2023-12-19T18:22:20.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.305+0000 7fe4b021e640 1 -- 172.21.15.27:0/2996745851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4a80d1e00 msgr2=0x7fe4a81f39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.305+0000 7fe4b021e640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4a80d1e00 0x7fe4a81f39c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fe49800d800 tx=0x7fe49800dcc0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.305+0000 7fe4b021e640 1 -- 172.21.15.27:0/2996745851 shutdown_connections 2023-12-19T18:22:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.305+0000 7fe4b021e640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe48c06a460 0x7fe48c06c920 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.305+0000 7fe4b021e640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4a815d240 0x7fe4a81f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.305+0000 7fe4b021e640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4a80d1e00 0x7fe4a81f39c0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.305+0000 7fe4b021e640 1 --2- 172.21.15.27:0/2996745851 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4a80cdf90 0x7fe4a81f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.305+0000 7fe4b021e640 1 -- 172.21.15.27:0/2996745851 >> 172.21.15.27:0/2996745851 conn(0x7fe4a80d31e0 msgr2=0x7fe4a8160170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.305+0000 7fe4b021e640 1 -- 172.21.15.27:0/2996745851 shutdown_connections 2023-12-19T18:22:20.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.305+0000 7fe4b021e640 1 -- 172.21.15.27:0/2996745851 wait complete. 2023-12-19T18:22:20.326 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2023-12-19T18:22:19.866577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7044,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555455,"statfs":{"total":1600000000,"available":1592786412,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.162920"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258377+0000","last_change":"2023-12-19T18:22:14.257437+0000","last_active":"2023-12-19T18:22:14.258377+0000","last_peered":"2023-12-19T18:22:14.258377+0000","last_clean":"2023-12-19T18:22:14.258377+0000","last_became_active":"2023-12-19T18:22:14.256527+0000","last_became_peered":"2023-12-19T18:22:14.256527+0000","last_unstale":"2023-12-19T18:22:14.258377+0000","last_undegraded":"2023-12-19T18:22:14.258377+0000","last_fullsized":"2023-12-19T18:22:14.258377+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:40.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258346+0000","last_change":"2023-12-19T18:22:14.255774+0000","last_active":"2023-12-19T18:22:14.258346+0000","last_peered":"2023-12-19T18:22:14.258346+0000","last_clean":"2023-12-19T18:22:14.258346+0000","last_became_active":"2023-12-19T18:22:14.255220+0000","last_became_peered":"2023-12-19T18:22:14.255220+0000","last_unstale":"2023-12-19T18:22:14.258346+0000","last_undegraded":"2023-12-19T18:22:14.258346+0000","last_fullsized":"2023-12-19T18:22:14.258346+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:15.726948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.256564+0000","last_change":"2023-12-19T18:22:14.256359+0000","last_active":"2023-12-19T18:22:14.256564+0000","last_peered":"2023-12-19T18:22:14.256564+0000","last_clean":"2023-12-19T18:22:14.256564+0000","last_became_active":"2023-12-19T18:22:14.255780+0000","last_became_peered":"2023-12-19T18:22:14.255780+0000","last_unstale":"2023-12-19T18:22:14.256564+0000","last_undegraded":"2023-12-19T18:22:14.256564+0000","last_fullsized":"2023-12-19T18:22:14.256564+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:16.333492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258589+0000","last_change":"2023-12-19T18:22:14.258495+0000","last_active":"2023-12-19T18:22:14.258589+0000","last_peered":"2023-12-19T18:22:14.258589+0000","last_clean":"2023-12-19T18:22:14.258589+0000","last_became_active":"2023-12-19T18:22:14.258217+0000","last_became_peered":"2023-12-19T18:22:14.258217+0000","last_unstale":"2023-12-19T18:22:14.258589+0000","last_undegraded":"2023-12-19T18:22:14.258589+0000","last_fullsized":"2023-12-19T18:22:14.258589+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:09.942336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.244066+0000","last_change":"2023-12-19T18:22:14.243278+0000","last_active":"2023-12-19T18:22:14.244066+0000","last_peered":"2023-12-19T18:22:14.244066+0000","last_clean":"2023-12-19T18:22:14.244066+0000","last_became_active":"2023-12-19T18:22:14.242452+0000","last_became_peered":"2023-12-19T18:22:14.242452+0000","last_unstale":"2023-12-19T18:22:14.244066+0000","last_undegraded":"2023-12-19T18:22:14.244066+0000","last_fullsized":"2023-12-19T18:22:14.244066+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:26.506333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.234713+0000","last_change":"2023-12-19T18:22:14.234640+0000","last_active":"2023-12-19T18:22:14.234713+0000","last_peered":"2023-12-19T18:22:14.234713+0000","last_clean":"2023-12-19T18:22:14.234713+0000","last_became_active":"2023-12-19T18:22:14.234044+0000","last_became_peered":"2023-12-19T18:22:14.234044+0000","last_unstale":"2023-12-19T18:22:14.234713+0000","last_undegraded":"2023-12-19T18:22:14.234713+0000","last_fullsized":"2023-12-19T18:22:14.234713+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:02:55.424114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239891+0000","last_change":"2023-12-19T18:22:14.239762+0000","last_active":"2023-12-19T18:22:14.239891+0000","last_peered":"2023-12-19T18:22:14.239891+0000","last_clean":"2023-12-19T18:22:14.239891+0000","last_became_active":"2023-12-19T18:22:14.239355+0000","last_became_peered":"2023-12-19T18:22:14.239355+0000","last_unstale":"2023-12-19T18:22:14.239891+0000","last_undegraded":"2023-12-19T18:22:14.239891+0000","last_fullsized":"2023-12-19T18:22:14.239891+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:37:37.391952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.256258+0000","last_change":"2023-12-19T18:22:14.256157+0000","last_active":"2023-12-19T18:22:14.256258+0000","last_peered":"2023-12-19T18:22:14.256258+0000","last_clean":"2023-12-19T18:22:14.256258+0000","last_became_active":"2023-12-19T18:22:14.255443+0000","last_became_peered":"2023-12-19T18:22:14.255443+0000","last_unstale":"2023-12-19T18:22:14.256258+0000","last_undegraded":"2023-12-19T18:22:14.256258+0000","last_fullsized":"2023-12-19T18:22:14.256258+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:23.915911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252425+0000","last_change":"2023-12-19T18:22:14.252334+0000","last_active":"2023-12-19T18:22:14.252425+0000","last_peered":"2023-12-19T18:22:14.252425+0000","last_clean":"2023-12-19T18:22:14.252425+0000","last_became_active":"2023-12-19T18:22:14.252176+0000","last_became_peered":"2023-12-19T18:22:14.252176+0000","last_unstale":"2023-12-19T18:22:14.252425+0000","last_undegraded":"2023-12-19T18:22:14.252425+0000","last_fullsized":"2023-12-19T18:22:14.252425+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:29.637020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259112+0000","last_change":"2023-12-19T18:22:14.258799+0000","last_active":"2023-12-19T18:22:14.259112+0000","last_peered":"2023-12-19T18:22:14.259112+0000","last_clean":"2023-12-19T18:22:14.259112+0000","last_became_active":"2023-12-19T18:22:14.258575+0000","last_became_peered":"2023-12-19T18:22:14.258575+0000","last_unstale":"2023-12-19T18:22:14.259112+0000","last_undegraded":"2023-12-19T18:22:14.259112+0000","last_fullsized":"2023-12-19T18:22:14.259112+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:11:57.860995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.234856+0000","last_change":"2023-12-19T18:22:14.234773+0000","last_active":"2023-12-19T18:22:14.234856+0000","last_peered":"2023-12-19T18:22:14.234856+0000","last_clean":"2023-12-19T18:22:14.234856+0000","last_became_active":"2023-12-19T18:22:14.234418+0000","last_became_peered":"2023-12-19T18:22:14.234418+0000","last_unstale":"2023-12-19T18:22:14.234856+0000","last_undegraded":"2023-12-19T18:22:14.234856+0000","last_fullsized":"2023-12-19T18:22:14.234856+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:23.903727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.256370+0000","last_change":"2023-12-19T18:22:14.256302+0000","last_active":"2023-12-19T18:22:14.256370+0000","last_peered":"2023-12-19T18:22:14.256370+0000","last_clean":"2023-12-19T18:22:14.256370+0000","last_became_active":"2023-12-19T18:22:14.256075+0000","last_became_peered":"2023-12-19T18:22:14.256075+0000","last_unstale":"2023-12-19T18:22:14.256370+0000","last_undegraded":"2023-12-19T18:22:14.256370+0000","last_fullsized":"2023-12-19T18:22:14.256370+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:31.105594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245932+0000","last_change":"2023-12-19T18:22:14.244697+0000","last_active":"2023-12-19T18:22:14.245932+0000","last_peered":"2023-12-19T18:22:14.245932+0000","last_clean":"2023-12-19T18:22:14.245932+0000","last_became_active":"2023-12-19T18:22:14.244184+0000","last_became_peered":"2023-12-19T18:22:14.244184+0000","last_unstale":"2023-12-19T18:22:14.245932+0000","last_undegraded":"2023-12-19T18:22:14.245932+0000","last_fullsized":"2023-12-19T18:22:14.245932+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:33.150825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.256363+0000","last_change":"2023-12-19T18:22:14.254104+0000","last_active":"2023-12-19T18:22:14.256363+0000","last_peered":"2023-12-19T18:22:14.256363+0000","last_clean":"2023-12-19T18:22:14.256363+0000","last_became_active":"2023-12-19T18:22:14.253879+0000","last_became_peered":"2023-12-19T18:22:14.253879+0000","last_unstale":"2023-12-19T18:22:14.256363+0000","last_undegraded":"2023-12-19T18:22:14.256363+0000","last_fullsized":"2023-12-19T18:22:14.256363+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:16.333492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259380+0000","last_change":"2023-12-19T18:22:14.258835+0000","last_active":"2023-12-19T18:22:14.259380+0000","last_peered":"2023-12-19T18:22:14.259380+0000","last_clean":"2023-12-19T18:22:14.259380+0000","last_became_active":"2023-12-19T18:22:14.257437+0000","last_became_peered":"2023-12-19T18:22:14.257437+0000","last_unstale":"2023-12-19T18:22:14.259380+0000","last_undegraded":"2023-12-19T18:22:14.259380+0000","last_fullsized":"2023-12-19T18:22:14.259380+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:34.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.240074+0000","last_change":"2023-12-19T18:22:14.239965+0000","last_active":"2023-12-19T18:22:14.240074+0000","last_peered":"2023-12-19T18:22:14.240074+0000","last_clean":"2023-12-19T18:22:14.240074+0000","last_became_active":"2023-12-19T18:22:14.239741+0000","last_became_peered":"2023-12-19T18:22:14.239741+0000","last_unstale":"2023-12-19T18:22:14.240074+0000","last_undegraded":"2023-12-19T18:22:14.240074+0000","last_fullsized":"2023-12-19T18:22:14.240074+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:02.299174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.237521+0000","last_change":"2023-12-19T18:22:14.237215+0000","last_active":"2023-12-19T18:22:14.237521+0000","last_peered":"2023-12-19T18:22:14.237521+0000","last_clean":"2023-12-19T18:22:14.237521+0000","last_became_active":"2023-12-19T18:22:14.236055+0000","last_became_peered":"2023-12-19T18:22:14.236055+0000","last_unstale":"2023-12-19T18:22:14.237521+0000","last_undegraded":"2023-12-19T18:22:14.237521+0000","last_fullsized":"2023-12-19T18:22:14.237521+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:41:21.616217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.261591+0000","last_change":"2023-12-19T18:22:14.261506+0000","last_active":"2023-12-19T18:22:14.261591+0000","last_peered":"2023-12-19T18:22:14.261591+0000","last_clean":"2023-12-19T18:22:14.261591+0000","last_became_active":"2023-12-19T18:22:14.259331+0000","last_became_peered":"2023-12-19T18:22:14.259331+0000","last_unstale":"2023-12-19T18:22:14.261591+0000","last_undegraded":"2023-12-19T18:22:14.261591+0000","last_fullsized":"2023-12-19T18:22:14.261591+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:44.388621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.260554+0000","last_change":"2023-12-19T18:22:14.260473+0000","last_active":"2023-12-19T18:22:14.260554+0000","last_peered":"2023-12-19T18:22:14.260554+0000","last_clean":"2023-12-19T18:22:14.260554+0000","last_became_active":"2023-12-19T18:22:14.259952+0000","last_became_peered":"2023-12-19T18:22:14.259952+0000","last_unstale":"2023-12-19T18:22:14.260554+0000","last_undegraded":"2023-12-19T18:22:14.260554+0000","last_fullsized":"2023-12-19T18:22:14.260554+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:41:30.585976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246133+0000","last_change":"2023-12-19T18:22:14.246068+0000","last_active":"2023-12-19T18:22:14.246133+0000","last_peered":"2023-12-19T18:22:14.246133+0000","last_clean":"2023-12-19T18:22:14.246133+0000","last_became_active":"2023-12-19T18:22:14.245845+0000","last_became_peered":"2023-12-19T18:22:14.245845+0000","last_unstale":"2023-12-19T18:22:14.246133+0000","last_undegraded":"2023-12-19T18:22:14.246133+0000","last_fullsized":"2023-12-19T18:22:14.246133+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:40.700106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.254074+0000","last_change":"2023-12-19T18:22:14.253786+0000","last_active":"2023-12-19T18:22:14.254074+0000","last_peered":"2023-12-19T18:22:14.254074+0000","last_clean":"2023-12-19T18:22:14.254074+0000","last_became_active":"2023-12-19T18:22:14.253409+0000","last_became_peered":"2023-12-19T18:22:14.253409+0000","last_unstale":"2023-12-19T18:22:14.254074+0000","last_undegraded":"2023-12-19T18:22:14.254074+0000","last_fullsized":"2023-12-19T18:22:14.254074+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:11.523949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.257498+0000","last_change":"2023-12-19T18:22:14.255178+0000","last_active":"2023-12-19T18:22:14.257498+0000","last_peered":"2023-12-19T18:22:14.257498+0000","last_clean":"2023-12-19T18:22:14.257498+0000","last_became_active":"2023-12-19T18:22:14.254755+0000","last_became_peered":"2023-12-19T18:22:14.254755+0000","last_unstale":"2023-12-19T18:22:14.257498+0000","last_undegraded":"2023-12-19T18:22:14.257498+0000","last_fullsized":"2023-12-19T18:22:14.257498+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:13:15.250452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252608+0000","last_change":"2023-12-19T18:22:14.250276+0000","last_active":"2023-12-19T18:22:14.252608+0000","last_peered":"2023-12-19T18:22:14.252608+0000","last_clean":"2023-12-19T18:22:14.252608+0000","last_became_active":"2023-12-19T18:22:14.250040+0000","last_became_peered":"2023-12-19T18:22:14.250040+0000","last_unstale":"2023-12-19T18:22:14.252608+0000","last_undegraded":"2023-12-19T18:22:14.252608+0000","last_fullsized":"2023-12-19T18:22:14.252608+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:08.465323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259701+0000","last_change":"2023-12-19T18:22:14.259592+0000","last_active":"2023-12-19T18:22:14.259701+0000","last_peered":"2023-12-19T18:22:14.259701+0000","last_clean":"2023-12-19T18:22:14.259701+0000","last_became_active":"2023-12-19T18:22:14.259307+0000","last_became_peered":"2023-12-19T18:22:14.259307+0000","last_unstale":"2023-12-19T18:22:14.259701+0000","last_undegraded":"2023-12-19T18:22:14.259701+0000","last_fullsized":"2023-12-19T18:22:14.259701+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:02:16.250351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239849+0000","last_change":"2023-12-19T18:22:14.236493+0000","last_active":"2023-12-19T18:22:14.239849+0000","last_peered":"2023-12-19T18:22:14.239849+0000","last_clean":"2023-12-19T18:22:14.239849+0000","last_became_active":"2023-12-19T18:22:14.236218+0000","last_became_peered":"2023-12-19T18:22:14.236218+0000","last_unstale":"2023-12-19T18:22:14.239849+0000","last_undegraded":"2023-12-19T18:22:14.239849+0000","last_fullsized":"2023-12-19T18:22:14.239849+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:32.485695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245899+0000","last_change":"2023-12-19T18:22:14.242241+0000","last_active":"2023-12-19T18:22:14.245899+0000","last_peered":"2023-12-19T18:22:14.245899+0000","last_clean":"2023-12-19T18:22:14.245899+0000","last_became_active":"2023-12-19T18:22:14.241678+0000","last_became_peered":"2023-12-19T18:22:14.241678+0000","last_unstale":"2023-12-19T18:22:14.245899+0000","last_undegraded":"2023-12-19T18:22:14.245899+0000","last_fullsized":"2023-12-19T18:22:14.245899+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:08.366785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248454+0000","last_change":"2023-12-19T18:22:14.247740+0000","last_active":"2023-12-19T18:22:14.248454+0000","last_peered":"2023-12-19T18:22:14.248454+0000","last_clean":"2023-12-19T18:22:14.248454+0000","last_became_active":"2023-12-19T18:22:14.247511+0000","last_became_peered":"2023-12-19T18:22:14.247511+0000","last_unstale":"2023-12-19T18:22:14.248454+0000","last_undegraded":"2023-12-19T18:22:14.248454+0000","last_fullsized":"2023-12-19T18:22:14.248454+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:57:01.585693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.257231+0000","last_change":"2023-12-19T18:22:14.257151+0000","last_active":"2023-12-19T18:22:14.257231+0000","last_peered":"2023-12-19T18:22:14.257231+0000","last_clean":"2023-12-19T18:22:14.257231+0000","last_became_active":"2023-12-19T18:22:14.256854+0000","last_became_peered":"2023-12-19T18:22:14.256854+0000","last_unstale":"2023-12-19T18:22:14.257231+0000","last_undegraded":"2023-12-19T18:22:14.257231+0000","last_fullsized":"2023-12-19T18:22:14.257231+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:01:59.548259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.238540+0000","last_change":"2023-12-19T18:22:14.236261+0000","last_active":"2023-12-19T18:22:14.238540+0000","last_peered":"2023-12-19T18:22:14.238540+0000","last_clean":"2023-12-19T18:22:14.238540+0000","last_became_active":"2023-12-19T18:22:14.234425+0000","last_became_peered":"2023-12-19T18:22:14.234425+0000","last_unstale":"2023-12-19T18:22:14.238540+0000","last_undegraded":"2023-12-19T18:22:14.238540+0000","last_fullsized":"2023-12-19T18:22:14.238540+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:32.485695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245871+0000","last_change":"2023-12-19T18:22:14.241276+0000","last_active":"2023-12-19T18:22:14.245871+0000","last_peered":"2023-12-19T18:22:14.245871+0000","last_clean":"2023-12-19T18:22:14.245871+0000","last_became_active":"2023-12-19T18:22:14.240498+0000","last_became_peered":"2023-12-19T18:22:14.240498+0000","last_unstale":"2023-12-19T18:22:14.245871+0000","last_undegraded":"2023-12-19T18:22:14.245871+0000","last_fullsized":"2023-12-19T18:22:14.245871+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:47:09.595830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252788+0000","last_change":"2023-12-19T18:22:14.252269+0000","last_active":"2023-12-19T18:22:14.252788+0000","last_peered":"2023-12-19T18:22:14.252788+0000","last_clean":"2023-12-19T18:22:14.252788+0000","last_became_active":"2023-12-19T18:22:14.251959+0000","last_became_peered":"2023-12-19T18:22:14.251959+0000","last_unstale":"2023-12-19T18:22:14.252788+0000","last_undegraded":"2023-12-19T18:22:14.252788+0000","last_fullsized":"2023-12-19T18:22:14.252788+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:01:36.203567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258305+0000","last_change":"2023-12-19T18:22:14.255516+0000","last_active":"2023-12-19T18:22:14.258305+0000","last_peered":"2023-12-19T18:22:14.258305+0000","last_clean":"2023-12-19T18:22:14.258305+0000","last_became_active":"2023-12-19T18:22:14.254491+0000","last_became_peered":"2023-12-19T18:22:14.254491+0000","last_unstale":"2023-12-19T18:22:14.258305+0000","last_undegraded":"2023-12-19T18:22:14.258305+0000","last_fullsized":"2023-12-19T18:22:14.258305+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:16.254938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.234589+0000","last_change":"2023-12-19T18:22:14.232017+0000","last_active":"2023-12-19T18:22:14.234589+0000","last_peered":"2023-12-19T18:22:14.234589+0000","last_clean":"2023-12-19T18:22:14.234589+0000","last_became_active":"2023-12-19T18:22:14.231247+0000","last_became_peered":"2023-12-19T18:22:14.231247+0000","last_unstale":"2023-12-19T18:22:14.234589+0000","last_undegraded":"2023-12-19T18:22:14.234589+0000","last_fullsized":"2023-12-19T18:22:14.234589+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:33.697121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"134'4","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.257068+0000","last_change":"2023-12-19T18:22:13.732827+0000","last_active":"2023-12-19T18:22:14.257068+0000","last_peered":"2023-12-19T18:22:14.257068+0000","last_clean":"2023-12-19T18:22:14.257068+0000","last_became_active":"2023-12-19T18:22:13.732603+0000","last_became_peered":"2023-12-19T18:22:13.732603+0000","last_unstale":"2023-12-19T18:22:14.257068+0000","last_undegraded":"2023-12-19T18:22:14.257068+0000","last_fullsized":"2023-12-19T18:22:14.257068+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:41:12.315002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245917+0000","last_change":"2023-12-19T18:22:14.243673+0000","last_active":"2023-12-19T18:22:14.245917+0000","last_peered":"2023-12-19T18:22:14.245917+0000","last_clean":"2023-12-19T18:22:14.245917+0000","last_became_active":"2023-12-19T18:22:14.243147+0000","last_became_peered":"2023-12-19T18:22:14.243147+0000","last_unstale":"2023-12-19T18:22:14.245917+0000","last_undegraded":"2023-12-19T18:22:14.245917+0000","last_fullsized":"2023-12-19T18:22:14.245917+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:39.384923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248064+0000","last_change":"2023-12-19T18:22:13.731624+0000","last_active":"2023-12-19T18:22:14.248064+0000","last_peered":"2023-12-19T18:22:14.248064+0000","last_clean":"2023-12-19T18:22:14.248064+0000","last_became_active":"2023-12-19T18:22:13.731335+0000","last_became_peered":"2023-12-19T18:22:13.731335+0000","last_unstale":"2023-12-19T18:22:14.248064+0000","last_undegraded":"2023-12-19T18:22:14.248064+0000","last_fullsized":"2023-12-19T18:22:14.248064+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:53:16.279722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258361+0000","last_change":"2023-12-19T18:22:14.257304+0000","last_active":"2023-12-19T18:22:14.258361+0000","last_peered":"2023-12-19T18:22:14.258361+0000","last_clean":"2023-12-19T18:22:14.258361+0000","last_became_active":"2023-12-19T18:22:14.256258+0000","last_became_peered":"2023-12-19T18:22:14.256258+0000","last_unstale":"2023-12-19T18:22:14.258361+0000","last_undegraded":"2023-12-19T18:22:14.258361+0000","last_fullsized":"2023-12-19T18:22:14.258361+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:16.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.262268+0000","last_change":"2023-12-19T18:22:13.733176+0000","last_active":"2023-12-19T18:22:14.262268+0000","last_peered":"2023-12-19T18:22:14.262268+0000","last_clean":"2023-12-19T18:22:14.262268+0000","last_became_active":"2023-12-19T18:22:13.732940+0000","last_became_peered":"2023-12-19T18:22:13.732940+0000","last_unstale":"2023-12-19T18:22:14.262268+0000","last_undegraded":"2023-12-19T18:22:14.262268+0000","last_fullsized":"2023-12-19T18:22:14.262268+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:30.091059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.244211+0000","last_change":"2023-12-19T18:22:14.243888+0000","last_active":"2023-12-19T18:22:14.244211+0000","last_peered":"2023-12-19T18:22:14.244211+0000","last_clean":"2023-12-19T18:22:14.244211+0000","last_became_active":"2023-12-19T18:22:14.243667+0000","last_became_peered":"2023-12-19T18:22:14.243667+0000","last_unstale":"2023-12-19T18:22:14.244211+0000","last_undegraded":"2023-12-19T18:22:14.244211+0000","last_fullsized":"2023-12-19T18:22:14.244211+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:39.384923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.257151+0000","last_change":"2023-12-19T18:22:13.733119+0000","last_active":"2023-12-19T18:22:14.257151+0000","last_peered":"2023-12-19T18:22:14.257151+0000","last_clean":"2023-12-19T18:22:14.257151+0000","last_became_active":"2023-12-19T18:22:13.732868+0000","last_became_peered":"2023-12-19T18:22:13.732868+0000","last_unstale":"2023-12-19T18:22:14.257151+0000","last_undegraded":"2023-12-19T18:22:14.257151+0000","last_fullsized":"2023-12-19T18:22:14.257151+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:07.352249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.235616+0000","last_change":"2023-12-19T18:22:14.235017+0000","last_active":"2023-12-19T18:22:14.235616+0000","last_peered":"2023-12-19T18:22:14.235616+0000","last_clean":"2023-12-19T18:22:14.235616+0000","last_became_active":"2023-12-19T18:22:14.234807+0000","last_became_peered":"2023-12-19T18:22:14.234807+0000","last_unstale":"2023-12-19T18:22:14.235616+0000","last_undegraded":"2023-12-19T18:22:14.235616+0000","last_fullsized":"2023-12-19T18:22:14.235616+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:53.745021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252491+0000","last_change":"2023-12-19T18:22:13.730941+0000","last_active":"2023-12-19T18:22:14.252491+0000","last_peered":"2023-12-19T18:22:14.252491+0000","last_clean":"2023-12-19T18:22:14.252491+0000","last_became_active":"2023-12-19T18:22:13.730684+0000","last_became_peered":"2023-12-19T18:22:13.730684+0000","last_unstale":"2023-12-19T18:22:14.252491+0000","last_undegraded":"2023-12-19T18:22:14.252491+0000","last_fullsized":"2023-12-19T18:22:14.252491+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:43:06.634088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239743+0000","last_change":"2023-12-19T18:22:14.239588+0000","last_active":"2023-12-19T18:22:14.239743+0000","last_peered":"2023-12-19T18:22:14.239743+0000","last_clean":"2023-12-19T18:22:14.239743+0000","last_became_active":"2023-12-19T18:22:14.238776+0000","last_became_peered":"2023-12-19T18:22:14.238776+0000","last_unstale":"2023-12-19T18:22:14.239743+0000","last_undegraded":"2023-12-19T18:22:14.239743+0000","last_fullsized":"2023-12-19T18:22:14.239743+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:41.584580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259067+0000","last_change":"2023-12-19T18:22:13.731736+0000","last_active":"2023-12-19T18:22:14.259067+0000","last_peered":"2023-12-19T18:22:14.259067+0000","last_clean":"2023-12-19T18:22:14.259067+0000","last_became_active":"2023-12-19T18:22:13.731153+0000","last_became_peered":"2023-12-19T18:22:13.731153+0000","last_unstale":"2023-12-19T18:22:14.259067+0000","last_undegraded":"2023-12-19T18:22:14.259067+0000","last_fullsized":"2023-12-19T18:22:14.259067+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:52.730485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.254538+0000","last_change":"2023-12-19T18:22:14.254248+0000","last_active":"2023-12-19T18:22:14.254538+0000","last_peered":"2023-12-19T18:22:14.254538+0000","last_clean":"2023-12-19T18:22:14.254538+0000","last_became_active":"2023-12-19T18:22:14.254005+0000","last_became_peered":"2023-12-19T18:22:14.254005+0000","last_unstale":"2023-12-19T18:22:14.254538+0000","last_undegraded":"2023-12-19T18:22:14.254538+0000","last_fullsized":"2023-12-19T18:22:14.254538+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:11:57.860995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.235224+0000","last_change":"2023-12-19T18:22:13.730291+0000","last_active":"2023-12-19T18:22:14.235224+0000","last_peered":"2023-12-19T18:22:14.235224+0000","last_clean":"2023-12-19T18:22:14.235224+0000","last_became_active":"2023-12-19T18:22:13.730015+0000","last_became_peered":"2023-12-19T18:22:13.730015+0000","last_unstale":"2023-12-19T18:22:14.235224+0000","last_undegraded":"2023-12-19T18:22:14.235224+0000","last_fullsized":"2023-12-19T18:22:14.235224+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:02:54.040583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.241518+0000","last_change":"2023-12-19T18:22:14.241428+0000","last_active":"2023-12-19T18:22:14.241518+0000","last_peered":"2023-12-19T18:22:14.241518+0000","last_clean":"2023-12-19T18:22:14.241518+0000","last_became_active":"2023-12-19T18:22:14.241208+0000","last_became_peered":"2023-12-19T18:22:14.241208+0000","last_unstale":"2023-12-19T18:22:14.241518+0000","last_undegraded":"2023-12-19T18:22:14.241518+0000","last_fullsized":"2023-12-19T18:22:14.241518+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:45.553627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.224445+0000","last_change":"2023-12-19T18:22:13.730656+0000","last_active":"2023-12-19T18:22:14.224445+0000","last_peered":"2023-12-19T18:22:14.224445+0000","last_clean":"2023-12-19T18:22:14.224445+0000","last_became_active":"2023-12-19T18:22:13.730408+0000","last_became_peered":"2023-12-19T18:22:13.730408+0000","last_unstale":"2023-12-19T18:22:14.224445+0000","last_undegraded":"2023-12-19T18:22:14.224445+0000","last_fullsized":"2023-12-19T18:22:14.224445+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:33.562949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258327+0000","last_change":"2023-12-19T18:22:14.255646+0000","last_active":"2023-12-19T18:22:14.258327+0000","last_peered":"2023-12-19T18:22:14.258327+0000","last_clean":"2023-12-19T18:22:14.258327+0000","last_became_active":"2023-12-19T18:22:14.254964+0000","last_became_peered":"2023-12-19T18:22:14.254964+0000","last_unstale":"2023-12-19T18:22:14.258327+0000","last_undegraded":"2023-12-19T18:22:14.258327+0000","last_fullsized":"2023-12-19T18:22:14.258327+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:13:15.250452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245923+0000","last_change":"2023-12-19T18:22:13.724536+0000","last_active":"2023-12-19T18:22:14.245923+0000","last_peered":"2023-12-19T18:22:14.245923+0000","last_clean":"2023-12-19T18:22:14.245923+0000","last_became_active":"2023-12-19T18:22:13.724110+0000","last_became_peered":"2023-12-19T18:22:13.724110+0000","last_unstale":"2023-12-19T18:22:14.245923+0000","last_undegraded":"2023-12-19T18:22:14.245923+0000","last_fullsized":"2023-12-19T18:22:14.245923+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:44.539092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245061+0000","last_change":"2023-12-19T18:22:14.244657+0000","last_active":"2023-12-19T18:22:14.245061+0000","last_peered":"2023-12-19T18:22:14.245061+0000","last_clean":"2023-12-19T18:22:14.245061+0000","last_became_active":"2023-12-19T18:22:14.244264+0000","last_became_peered":"2023-12-19T18:22:14.244264+0000","last_unstale":"2023-12-19T18:22:14.245061+0000","last_undegraded":"2023-12-19T18:22:14.245061+0000","last_fullsized":"2023-12-19T18:22:14.245061+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:15.726948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248854+0000","last_change":"2023-12-19T18:22:13.733261+0000","last_active":"2023-12-19T18:22:14.248854+0000","last_peered":"2023-12-19T18:22:14.248854+0000","last_clean":"2023-12-19T18:22:14.248854+0000","last_became_active":"2023-12-19T18:22:13.733015+0000","last_became_peered":"2023-12-19T18:22:13.733015+0000","last_unstale":"2023-12-19T18:22:14.248854+0000","last_undegraded":"2023-12-19T18:22:14.248854+0000","last_fullsized":"2023-12-19T18:22:14.248854+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:52.730485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259593+0000","last_change":"2023-12-19T18:22:14.259137+0000","last_active":"2023-12-19T18:22:14.259593+0000","last_peered":"2023-12-19T18:22:14.259593+0000","last_clean":"2023-12-19T18:22:14.259593+0000","last_became_active":"2023-12-19T18:22:14.258878+0000","last_became_peered":"2023-12-19T18:22:14.258878+0000","last_unstale":"2023-12-19T18:22:14.259593+0000","last_undegraded":"2023-12-19T18:22:14.259593+0000","last_fullsized":"2023-12-19T18:22:14.259593+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:32:56.103686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"134'3","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.247294+0000","last_change":"2023-12-19T18:22:13.727917+0000","last_active":"2023-12-19T18:22:14.247294+0000","last_peered":"2023-12-19T18:22:14.247294+0000","last_clean":"2023-12-19T18:22:14.247294+0000","last_became_active":"2023-12-19T18:22:13.727633+0000","last_became_peered":"2023-12-19T18:22:13.727633+0000","last_unstale":"2023-12-19T18:22:14.247294+0000","last_undegraded":"2023-12-19T18:22:14.247294+0000","last_fullsized":"2023-12-19T18:22:14.247294+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:36:19.046380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.247094+0000","last_change":"2023-12-19T18:22:14.246791+0000","last_active":"2023-12-19T18:22:14.247094+0000","last_peered":"2023-12-19T18:22:14.247094+0000","last_clean":"2023-12-19T18:22:14.247094+0000","last_became_active":"2023-12-19T18:22:14.246271+0000","last_became_peered":"2023-12-19T18:22:14.246271+0000","last_unstale":"2023-12-19T18:22:14.247094+0000","last_undegraded":"2023-12-19T18:22:14.247094+0000","last_fullsized":"2023-12-19T18:22:14.247094+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:40.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"134'6","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259355+0000","last_change":"2023-12-19T18:22:13.725517+0000","last_active":"2023-12-19T18:22:14.259355+0000","last_peered":"2023-12-19T18:22:14.259355+0000","last_clean":"2023-12-19T18:22:14.259355+0000","last_became_active":"2023-12-19T18:22:13.724094+0000","last_became_peered":"2023-12-19T18:22:13.724094+0000","last_unstale":"2023-12-19T18:22:14.259355+0000","last_undegraded":"2023-12-19T18:22:14.259355+0000","last_fullsized":"2023-12-19T18:22:14.259355+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:29:20.466956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252592+0000","last_change":"2023-12-19T18:22:14.250414+0000","last_active":"2023-12-19T18:22:14.252592+0000","last_peered":"2023-12-19T18:22:14.252592+0000","last_clean":"2023-12-19T18:22:14.252592+0000","last_became_active":"2023-12-19T18:22:14.249822+0000","last_became_peered":"2023-12-19T18:22:14.249822+0000","last_unstale":"2023-12-19T18:22:14.252592+0000","last_undegraded":"2023-12-19T18:22:14.252592+0000","last_fullsized":"2023-12-19T18:22:14.252592+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:37.118051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248641+0000","last_change":"2023-12-19T18:22:13.730064+0000","last_active":"2023-12-19T18:22:14.248641+0000","last_peered":"2023-12-19T18:22:14.248641+0000","last_clean":"2023-12-19T18:22:14.248641+0000","last_became_active":"2023-12-19T18:22:13.729780+0000","last_became_peered":"2023-12-19T18:22:13.729780+0000","last_unstale":"2023-12-19T18:22:14.248641+0000","last_undegraded":"2023-12-19T18:22:14.248641+0000","last_fullsized":"2023-12-19T18:22:14.248641+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:14.712413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.254201+0000","last_change":"2023-12-19T18:22:14.253925+0000","last_active":"2023-12-19T18:22:14.254201+0000","last_peered":"2023-12-19T18:22:14.254201+0000","last_clean":"2023-12-19T18:22:14.254201+0000","last_became_active":"2023-12-19T18:22:14.253159+0000","last_became_peered":"2023-12-19T18:22:14.253159+0000","last_unstale":"2023-12-19T18:22:14.254201+0000","last_undegraded":"2023-12-19T18:22:14.254201+0000","last_fullsized":"2023-12-19T18:22:14.254201+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:16.254938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246679+0000","last_change":"2023-12-19T18:22:13.732632+0000","last_active":"2023-12-19T18:22:14.246679+0000","last_peered":"2023-12-19T18:22:14.246679+0000","last_clean":"2023-12-19T18:22:14.246679+0000","last_became_active":"2023-12-19T18:22:13.732365+0000","last_became_peered":"2023-12-19T18:22:13.732365+0000","last_unstale":"2023-12-19T18:22:14.246679+0000","last_undegraded":"2023-12-19T18:22:14.246679+0000","last_fullsized":"2023-12-19T18:22:14.246679+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.235724+0000","last_change":"2023-12-19T18:22:14.235442+0000","last_active":"2023-12-19T18:22:14.235724+0000","last_peered":"2023-12-19T18:22:14.235724+0000","last_clean":"2023-12-19T18:22:14.235724+0000","last_became_active":"2023-12-19T18:22:14.235055+0000","last_became_peered":"2023-12-19T18:22:14.235055+0000","last_unstale":"2023-12-19T18:22:14.235724+0000","last_undegraded":"2023-12-19T18:22:14.235724+0000","last_fullsized":"2023-12-19T18:22:14.235724+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:06:47.768050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.229590+0000","last_change":"2023-12-19T18:22:13.729287+0000","last_active":"2023-12-19T18:22:14.229590+0000","last_peered":"2023-12-19T18:22:14.229590+0000","last_clean":"2023-12-19T18:22:14.229590+0000","last_became_active":"2023-12-19T18:22:13.725118+0000","last_became_peered":"2023-12-19T18:22:13.725118+0000","last_unstale":"2023-12-19T18:22:14.229590+0000","last_undegraded":"2023-12-19T18:22:14.229590+0000","last_fullsized":"2023-12-19T18:22:14.229590+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:13:44.939138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.255529+0000","last_change":"2023-12-19T18:22:14.255009+0000","last_active":"2023-12-19T18:22:14.255529+0000","last_peered":"2023-12-19T18:22:14.255529+0000","last_clean":"2023-12-19T18:22:14.255529+0000","last_became_active":"2023-12-19T18:22:14.254396+0000","last_became_peered":"2023-12-19T18:22:14.254396+0000","last_unstale":"2023-12-19T18:22:14.255529+0000","last_undegraded":"2023-12-19T18:22:14.255529+0000","last_fullsized":"2023-12-19T18:22:14.255529+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:08.465323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"134'8","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.263107+0000","last_change":"2023-12-19T18:22:13.734908+0000","last_active":"2023-12-19T18:22:14.263107+0000","last_peered":"2023-12-19T18:22:14.263107+0000","last_clean":"2023-12-19T18:22:14.263107+0000","last_became_active":"2023-12-19T18:22:13.734666+0000","last_became_peered":"2023-12-19T18:22:13.734666+0000","last_unstale":"2023-12-19T18:22:14.263107+0000","last_undegraded":"2023-12-19T18:22:14.263107+0000","last_fullsized":"2023-12-19T18:22:14.263107+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:44.539092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.243196+0000","last_change":"2023-12-19T18:22:14.243113+0000","last_active":"2023-12-19T18:22:14.243196+0000","last_peered":"2023-12-19T18:22:14.243196+0000","last_clean":"2023-12-19T18:22:14.243196+0000","last_became_active":"2023-12-19T18:22:14.242834+0000","last_became_peered":"2023-12-19T18:22:14.242834+0000","last_unstale":"2023-12-19T18:22:14.243196+0000","last_undegraded":"2023-12-19T18:22:14.243196+0000","last_fullsized":"2023-12-19T18:22:14.243196+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:12.207051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.243309+0000","last_change":"2023-12-19T18:22:13.724787+0000","last_active":"2023-12-19T18:22:14.243309+0000","last_peered":"2023-12-19T18:22:14.243309+0000","last_clean":"2023-12-19T18:22:14.243309+0000","last_became_active":"2023-12-19T18:22:13.724547+0000","last_became_peered":"2023-12-19T18:22:13.724547+0000","last_unstale":"2023-12-19T18:22:14.243309+0000","last_undegraded":"2023-12-19T18:22:14.243309+0000","last_fullsized":"2023-12-19T18:22:14.243309+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:11.095769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.243792+0000","last_change":"2023-12-19T18:22:14.243719+0000","last_active":"2023-12-19T18:22:14.243792+0000","last_peered":"2023-12-19T18:22:14.243792+0000","last_clean":"2023-12-19T18:22:14.243792+0000","last_became_active":"2023-12-19T18:22:14.242848+0000","last_became_peered":"2023-12-19T18:22:14.242848+0000","last_unstale":"2023-12-19T18:22:14.243792+0000","last_undegraded":"2023-12-19T18:22:14.243792+0000","last_fullsized":"2023-12-19T18:22:14.243792+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:34.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246567+0000","last_change":"2023-12-19T18:22:13.728901+0000","last_active":"2023-12-19T18:22:14.246567+0000","last_peered":"2023-12-19T18:22:14.246567+0000","last_clean":"2023-12-19T18:22:14.246567+0000","last_became_active":"2023-12-19T18:22:13.728001+0000","last_became_peered":"2023-12-19T18:22:13.728001+0000","last_unstale":"2023-12-19T18:22:14.246567+0000","last_undegraded":"2023-12-19T18:22:14.246567+0000","last_fullsized":"2023-12-19T18:22:14.246567+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:59.426392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246154+0000","last_change":"2023-12-19T18:22:14.246103+0000","last_active":"2023-12-19T18:22:14.246154+0000","last_peered":"2023-12-19T18:22:14.246154+0000","last_clean":"2023-12-19T18:22:14.246154+0000","last_became_active":"2023-12-19T18:22:14.245847+0000","last_became_peered":"2023-12-19T18:22:14.245847+0000","last_unstale":"2023-12-19T18:22:14.246154+0000","last_undegraded":"2023-12-19T18:22:14.246154+0000","last_fullsized":"2023-12-19T18:22:14.246154+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:01.033125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.226401+0000","last_change":"2023-12-19T18:22:13.729021+0000","last_active":"2023-12-19T18:22:14.226401+0000","last_peered":"2023-12-19T18:22:14.226401+0000","last_clean":"2023-12-19T18:22:14.226401+0000","last_became_active":"2023-12-19T18:22:13.728792+0000","last_became_peered":"2023-12-19T18:22:13.728792+0000","last_unstale":"2023-12-19T18:22:14.226401+0000","last_undegraded":"2023-12-19T18:22:14.226401+0000","last_fullsized":"2023-12-19T18:22:14.226401+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:39.606223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252679+0000","last_change":"2023-12-19T18:22:14.252307+0000","last_active":"2023-12-19T18:22:14.252679+0000","last_peered":"2023-12-19T18:22:14.252679+0000","last_clean":"2023-12-19T18:22:14.252679+0000","last_became_active":"2023-12-19T18:22:14.251969+0000","last_became_peered":"2023-12-19T18:22:14.251969+0000","last_unstale":"2023-12-19T18:22:14.252679+0000","last_undegraded":"2023-12-19T18:22:14.252679+0000","last_fullsized":"2023-12-19T18:22:14.252679+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:50:35.183653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.263780+0000","last_change":"2023-12-19T18:22:13.734402+0000","last_active":"2023-12-19T18:22:14.263780+0000","last_peered":"2023-12-19T18:22:14.263780+0000","last_clean":"2023-12-19T18:22:14.263780+0000","last_became_active":"2023-12-19T18:22:13.733935+0000","last_became_peered":"2023-12-19T18:22:13.733935+0000","last_unstale":"2023-12-19T18:22:14.263780+0000","last_undegraded":"2023-12-19T18:22:14.263780+0000","last_fullsized":"2023-12-19T18:22:14.263780+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:30.091059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258421+0000","last_change":"2023-12-19T18:22:14.257490+0000","last_active":"2023-12-19T18:22:14.258421+0000","last_peered":"2023-12-19T18:22:14.258421+0000","last_clean":"2023-12-19T18:22:14.258421+0000","last_became_active":"2023-12-19T18:22:14.256714+0000","last_became_peered":"2023-12-19T18:22:14.256714+0000","last_unstale":"2023-12-19T18:22:14.258421+0000","last_undegraded":"2023-12-19T18:22:14.258421+0000","last_fullsized":"2023-12-19T18:22:14.258421+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:29.507646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.238218+0000","last_change":"2023-12-19T18:22:13.730665+0000","last_active":"2023-12-19T18:22:14.238218+0000","last_peered":"2023-12-19T18:22:14.238218+0000","last_clean":"2023-12-19T18:22:14.238218+0000","last_became_active":"2023-12-19T18:22:13.729630+0000","last_became_peered":"2023-12-19T18:22:13.729630+0000","last_unstale":"2023-12-19T18:22:14.238218+0000","last_undegraded":"2023-12-19T18:22:14.238218+0000","last_fullsized":"2023-12-19T18:22:14.238218+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:01:35.189032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258658+0000","last_change":"2023-12-19T18:22:14.257612+0000","last_active":"2023-12-19T18:22:14.258658+0000","last_peered":"2023-12-19T18:22:14.258658+0000","last_clean":"2023-12-19T18:22:14.258658+0000","last_became_active":"2023-12-19T18:22:14.256820+0000","last_became_peered":"2023-12-19T18:22:14.256820+0000","last_unstale":"2023-12-19T18:22:14.258658+0000","last_undegraded":"2023-12-19T18:22:14.258658+0000","last_fullsized":"2023-12-19T18:22:14.258658+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:12.207051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.231579+0000","last_change":"2023-12-19T18:22:13.731257+0000","last_active":"2023-12-19T18:22:14.231579+0000","last_peered":"2023-12-19T18:22:14.231579+0000","last_clean":"2023-12-19T18:22:14.231579+0000","last_became_active":"2023-12-19T18:22:13.730910+0000","last_became_peered":"2023-12-19T18:22:13.730910+0000","last_unstale":"2023-12-19T18:22:14.231579+0000","last_undegraded":"2023-12-19T18:22:14.231579+0000","last_fullsized":"2023-12-19T18:22:14.231579+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.254075+0000","last_change":"2023-12-19T18:22:14.253991+0000","last_active":"2023-12-19T18:22:14.254075+0000","last_peered":"2023-12-19T18:22:14.254075+0000","last_clean":"2023-12-19T18:22:14.254075+0000","last_became_active":"2023-12-19T18:22:14.253757+0000","last_became_peered":"2023-12-19T18:22:14.253757+0000","last_unstale":"2023-12-19T18:22:14.254075+0000","last_undegraded":"2023-12-19T18:22:14.254075+0000","last_fullsized":"2023-12-19T18:22:14.254075+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:44.388621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246101+0000","last_change":"2023-12-19T18:22:13.714716+0000","last_active":"2023-12-19T18:22:14.246101+0000","last_peered":"2023-12-19T18:22:14.246101+0000","last_clean":"2023-12-19T18:22:14.246101+0000","last_became_active":"2023-12-19T18:22:13.713977+0000","last_became_peered":"2023-12-19T18:22:13.713977+0000","last_unstale":"2023-12-19T18:22:14.246101+0000","last_undegraded":"2023-12-19T18:22:14.246101+0000","last_fullsized":"2023-12-19T18:22:14.246101+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246372+0000","last_change":"2023-12-19T18:22:14.245059+0000","last_active":"2023-12-19T18:22:14.246372+0000","last_peered":"2023-12-19T18:22:14.246372+0000","last_clean":"2023-12-19T18:22:14.246372+0000","last_became_active":"2023-12-19T18:22:14.244526+0000","last_became_peered":"2023-12-19T18:22:14.244526+0000","last_unstale":"2023-12-19T18:22:14.246372+0000","last_undegraded":"2023-12-19T18:22:14.246372+0000","last_fullsized":"2023-12-19T18:22:14.246372+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:45:23.784185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.241662+0000","last_change":"2023-12-19T18:22:13.729500+0000","last_active":"2023-12-19T18:22:14.241662+0000","last_peered":"2023-12-19T18:22:14.241662+0000","last_clean":"2023-12-19T18:22:14.241662+0000","last_became_active":"2023-12-19T18:22:13.725715+0000","last_became_peered":"2023-12-19T18:22:13.725715+0000","last_unstale":"2023-12-19T18:22:14.241662+0000","last_undegraded":"2023-12-19T18:22:14.241662+0000","last_fullsized":"2023-12-19T18:22:14.241662+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:30:23.979686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259484+0000","last_change":"2023-12-19T18:22:14.258603+0000","last_active":"2023-12-19T18:22:14.259484+0000","last_peered":"2023-12-19T18:22:14.259484+0000","last_clean":"2023-12-19T18:22:14.259484+0000","last_became_active":"2023-12-19T18:22:14.258065+0000","last_became_peered":"2023-12-19T18:22:14.258065+0000","last_unstale":"2023-12-19T18:22:14.259484+0000","last_undegraded":"2023-12-19T18:22:14.259484+0000","last_fullsized":"2023-12-19T18:22:14.259484+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:43:39.731660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.225364+0000","last_change":"2023-12-19T18:22:13.726768+0000","last_active":"2023-12-19T18:22:14.225364+0000","last_peered":"2023-12-19T18:22:14.225364+0000","last_clean":"2023-12-19T18:22:14.225364+0000","last_became_active":"2023-12-19T18:22:13.722319+0000","last_became_peered":"2023-12-19T18:22:13.722319+0000","last_unstale":"2023-12-19T18:22:14.225364+0000","last_undegraded":"2023-12-19T18:22:14.225364+0000","last_fullsized":"2023-12-19T18:22:14.225364+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:11.192516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.255546+0000","last_change":"2023-12-19T18:22:14.255042+0000","last_active":"2023-12-19T18:22:14.255546+0000","last_peered":"2023-12-19T18:22:14.255546+0000","last_clean":"2023-12-19T18:22:14.255546+0000","last_became_active":"2023-12-19T18:22:14.253623+0000","last_became_peered":"2023-12-19T18:22:14.253623+0000","last_unstale":"2023-12-19T18:22:14.255546+0000","last_undegraded":"2023-12-19T18:22:14.255546+0000","last_fullsized":"2023-12-19T18:22:14.255546+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:09.942336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.227165+0000","last_change":"2023-12-19T18:22:13.713351+0000","last_active":"2023-12-19T18:22:14.227165+0000","last_peered":"2023-12-19T18:22:14.227165+0000","last_clean":"2023-12-19T18:22:14.227165+0000","last_became_active":"2023-12-19T18:22:13.712581+0000","last_became_peered":"2023-12-19T18:22:13.712581+0000","last_unstale":"2023-12-19T18:22:14.227165+0000","last_undegraded":"2023-12-19T18:22:14.227165+0000","last_fullsized":"2023-12-19T18:22:14.227165+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:36.103516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.242422+0000","last_change":"2023-12-19T18:22:14.241466+0000","last_active":"2023-12-19T18:22:14.242422+0000","last_peered":"2023-12-19T18:22:14.242422+0000","last_clean":"2023-12-19T18:22:14.242422+0000","last_became_active":"2023-12-19T18:22:14.241205+0000","last_became_peered":"2023-12-19T18:22:14.241205+0000","last_unstale":"2023-12-19T18:22:14.242422+0000","last_undegraded":"2023-12-19T18:22:14.242422+0000","last_fullsized":"2023-12-19T18:22:14.242422+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:08.710594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239506+0000","last_change":"2023-12-19T18:22:13.718488+0000","last_active":"2023-12-19T18:22:14.239506+0000","last_peered":"2023-12-19T18:22:14.239506+0000","last_clean":"2023-12-19T18:22:14.239506+0000","last_became_active":"2023-12-19T18:22:13.717351+0000","last_became_peered":"2023-12-19T18:22:13.717351+0000","last_unstale":"2023-12-19T18:22:14.239506+0000","last_undegraded":"2023-12-19T18:22:14.239506+0000","last_fullsized":"2023-12-19T18:22:14.239506+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:49:11.599052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248088+0000","last_change":"2023-12-19T18:22:14.247933+0000","last_active":"2023-12-19T18:22:14.248088+0000","last_peered":"2023-12-19T18:22:14.248088+0000","last_clean":"2023-12-19T18:22:14.248088+0000","last_became_active":"2023-12-19T18:22:14.247461+0000","last_became_peered":"2023-12-19T18:22:14.247461+0000","last_unstale":"2023-12-19T18:22:14.248088+0000","last_undegraded":"2023-12-19T18:22:14.248088+0000","last_fullsized":"2023-12-19T18:22:14.248088+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:24.011514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246268+0000","last_change":"2023-12-19T18:22:13.733542+0000","last_active":"2023-12-19T18:22:14.246268+0000","last_peered":"2023-12-19T18:22:14.246268+0000","last_clean":"2023-12-19T18:22:14.246268+0000","last_became_active":"2023-12-19T18:22:13.733273+0000","last_became_peered":"2023-12-19T18:22:13.733273+0000","last_unstale":"2023-12-19T18:22:14.246268+0000","last_undegraded":"2023-12-19T18:22:14.246268+0000","last_fullsized":"2023-12-19T18:22:14.246268+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:07.352249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252415+0000","last_change":"2023-12-19T18:22:14.252315+0000","last_active":"2023-12-19T18:22:14.252415+0000","last_peered":"2023-12-19T18:22:14.252415+0000","last_clean":"2023-12-19T18:22:14.252415+0000","last_became_active":"2023-12-19T18:22:14.252108+0000","last_became_peered":"2023-12-19T18:22:14.252108+0000","last_unstale":"2023-12-19T18:22:14.252415+0000","last_undegraded":"2023-12-19T18:22:14.252415+0000","last_fullsized":"2023-12-19T18:22:14.252415+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:08.470720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239128+0000","last_change":"2023-12-19T18:22:13.723012+0000","last_active":"2023-12-19T18:22:14.239128+0000","last_peered":"2023-12-19T18:22:14.239128+0000","last_clean":"2023-12-19T18:22:14.239128+0000","last_became_active":"2023-12-19T18:22:13.717308+0000","last_became_peered":"2023-12-19T18:22:13.717308+0000","last_unstale":"2023-12-19T18:22:14.239128+0000","last_undegraded":"2023-12-19T18:22:14.239128+0000","last_fullsized":"2023-12-19T18:22:14.239128+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:01.277914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246337+0000","last_change":"2023-12-19T18:22:14.244959+0000","last_active":"2023-12-19T18:22:14.246337+0000","last_peered":"2023-12-19T18:22:14.246337+0000","last_clean":"2023-12-19T18:22:14.246337+0000","last_became_active":"2023-12-19T18:22:14.238592+0000","last_became_peered":"2023-12-19T18:22:14.238592+0000","last_unstale":"2023-12-19T18:22:14.246337+0000","last_undegraded":"2023-12-19T18:22:14.246337+0000","last_fullsized":"2023-12-19T18:22:14.246337+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:30:01.499864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.232648+0000","last_change":"2023-12-19T18:22:13.736355+0000","last_active":"2023-12-19T18:22:14.232648+0000","last_peered":"2023-12-19T18:22:14.232648+0000","last_clean":"2023-12-19T18:22:14.232648+0000","last_became_active":"2023-12-19T18:22:13.736056+0000","last_became_peered":"2023-12-19T18:22:13.736056+0000","last_unstale":"2023-12-19T18:22:14.232648+0000","last_undegraded":"2023-12-19T18:22:14.232648+0000","last_fullsized":"2023-12-19T18:22:14.232648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:47.766382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.255816+0000","last_change":"2023-12-19T18:22:14.255696+0000","last_active":"2023-12-19T18:22:14.255816+0000","last_peered":"2023-12-19T18:22:14.255816+0000","last_clean":"2023-12-19T18:22:14.255816+0000","last_became_active":"2023-12-19T18:22:14.255205+0000","last_became_peered":"2023-12-19T18:22:14.255205+0000","last_unstale":"2023-12-19T18:22:14.255816+0000","last_undegraded":"2023-12-19T18:22:14.255816+0000","last_fullsized":"2023-12-19T18:22:14.255816+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:08.366785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248750+0000","last_change":"2023-12-19T18:22:13.715051+0000","last_active":"2023-12-19T18:22:14.248750+0000","last_peered":"2023-12-19T18:22:14.248750+0000","last_clean":"2023-12-19T18:22:14.248750+0000","last_became_active":"2023-12-19T18:22:13.714392+0000","last_became_peered":"2023-12-19T18:22:13.714392+0000","last_unstale":"2023-12-19T18:22:14.248750+0000","last_undegraded":"2023-12-19T18:22:14.248750+0000","last_fullsized":"2023-12-19T18:22:14.248750+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:00.018590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259379+0000","last_change":"2023-12-19T18:22:14.259186+0000","last_active":"2023-12-19T18:22:14.259379+0000","last_peered":"2023-12-19T18:22:14.259379+0000","last_clean":"2023-12-19T18:22:14.259379+0000","last_became_active":"2023-12-19T18:22:14.258862+0000","last_became_peered":"2023-12-19T18:22:14.258862+0000","last_unstale":"2023-12-19T18:22:14.259379+0000","last_undegraded":"2023-12-19T18:22:14.259379+0000","last_fullsized":"2023-12-19T18:22:14.259379+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:16.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.247514+0000","last_change":"2023-12-19T18:22:13.725717+0000","last_active":"2023-12-19T18:22:14.247514+0000","last_peered":"2023-12-19T18:22:14.247514+0000","last_clean":"2023-12-19T18:22:14.247514+0000","last_became_active":"2023-12-19T18:22:13.724435+0000","last_became_peered":"2023-12-19T18:22:13.724435+0000","last_unstale":"2023-12-19T18:22:14.247514+0000","last_undegraded":"2023-12-19T18:22:14.247514+0000","last_fullsized":"2023-12-19T18:22:14.247514+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:00:05.094962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":267,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.229283+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:22:14.229283+0000","last_peered":"2023-12-19T18:22:14.229283+0000","last_clean":"2023-12-19T18:22:14.229283+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:22:14.229283+0000","last_undegraded":"2023-12-19T18:22:14.229283+0000","last_fullsized":"2023-12-19T18:22:14.229283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":736,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199245902,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2310000000000001,"15min":1.1519999999999999},"min":{"1min":0.57899999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9630000000000001,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.278,"15min":1.1890000000000001},"min":{"1min":0.76900000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6990000000000001,"5min":1.79,"15min":1.79},"last":1.2410000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.3480000000000001,"15min":1.23},"min":{"1min":0.752,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.7010000000000001,"5min":10.926,"15min":10.926},"last":1.081},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.365,"15min":1.244},"min":{"1min":0.80900000000000005,"5min":0.71899999999999997,"15min":0.67900000000000005},"max":{"1min":1.6240000000000001,"5min":10.904999999999999,"15min":10.904999999999999},"last":0.98999999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.161,"15min":1.1040000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.427,"5min":3.1960000000000002,"15min":3.1960000000000002},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.266,"15min":1.1930000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0670000000000002,"5min":6.008,"15min":6.008},"last":0.877}]},{"osd":3,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.28,"15min":1.1619999999999999},"min":{"1min":0.66900000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.595,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":1.659},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4179999999999999,"15min":1.294},"min":{"1min":0.85699999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.157,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":1.3759999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.258,"15min":1.123},"min":{"1min":0.88600000000000001,"5min":0.65400000000000003,"15min":0.47899999999999998},"max":{"1min":1.8009999999999999,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":1.331},{"interface":"front","average":{"1min":1.222,"5min":1.304,"15min":1.2410000000000001},"min":{"1min":0.94599999999999995,"5min":0.72999999999999998,"15min":0.57599999999999996},"max":{"1min":1.6759999999999999,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.736}]},{"osd":5,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.399,"15min":1.333},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.55000000000000004},"max":{"1min":1.8620000000000001,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.4339999999999999,"15min":1.337},"min":{"1min":0.80700000000000005,"5min":0.71599999999999997,"15min":0.53500000000000003},"max":{"1min":2.1179999999999999,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":1.1439999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.5669999999999999,"15min":1.4690000000000001},"min":{"1min":1.036,"5min":0.78600000000000003,"15min":0.59899999999999998},"max":{"1min":2.423,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.407},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.49,"15min":1.3799999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.63200000000000001},"max":{"1min":2.0179999999999998,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":1.8169999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":735,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199246484,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.121,"15min":1.012},"min":{"1min":0.79800000000000004,"5min":0.66700000000000004,"15min":0.65800000000000003},"max":{"1min":1.6259999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.163,"5min":1.1870000000000001,"15min":1.075},"min":{"1min":0.78400000000000003,"5min":0.64300000000000002,"15min":0.55600000000000005},"max":{"1min":1.663,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":1}]},{"osd":1,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.1699999999999999,"15min":1.1319999999999999},"min":{"1min":0.86599999999999999,"5min":0.71399999999999997,"15min":0.68200000000000005},"max":{"1min":1.5609999999999999,"5min":4.5540000000000003,"15min":4.5540000000000003},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.226,"15min":1.1699999999999999},"min":{"1min":0.86599999999999999,"5min":0.625,"15min":0.60299999999999998},"max":{"1min":1.627,"5min":1.885,"15min":1.885},"last":1.331}]},{"osd":2,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.254,"15min":1.151},"min":{"1min":0.84799999999999998,"5min":0.626,"15min":0.626},"max":{"1min":1.583,"5min":1.821,"15min":1.821},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.218,"15min":1.2310000000000001},"min":{"1min":0.83599999999999997,"5min":0.81999999999999995,"15min":0.79900000000000004},"max":{"1min":1.821,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":1.1930000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.294,"15min":1.202},"min":{"1min":1.0129999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.722,"5min":3.8820000000000001,"15min":3.8820000000000001},"last":1.359},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2170000000000001,"15min":1.1699999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6559999999999999,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":1.133}]},{"osd":4,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.296,"15min":1.2450000000000001},"min":{"1min":0.74099999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6299999999999999,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.621},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.431,"15min":1.347},"min":{"1min":0.76700000000000002,"5min":0.70799999999999996,"15min":0.57999999999999996},"max":{"1min":1.774,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.448}]},{"osd":5,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2829999999999999,"15min":1.1879999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7330000000000001,"5min":4.6680000000000001,"15min":4.6680000000000001},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.4450000000000001,"15min":1.3560000000000001},"min":{"1min":0.84799999999999998,"5min":0.626,"15min":0.626},"max":{"1min":4.1159999999999997,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.4790000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4430000000000001,"15min":1.349},"min":{"1min":0.92700000000000005,"5min":0.82199999999999995,"15min":0.61899999999999999},"max":{"1min":1.7010000000000001,"5min":3.9670000000000001,"15min":3.9670000000000001},"last":1.274},{"interface":"front","average":{"1min":1.393,"5min":1.4470000000000001,"15min":1.357},"min":{"1min":0.81899999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8169999999999999,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":1.522}]}]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1313,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193998,"statfs":{"total":200000000,"available":198654496,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.23,"15min":1.1679999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.65200000000000002},"max":{"1min":1.6830000000000001,"5min":1.837,"15min":1.863},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2729999999999999,"15min":1.1870000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.571,"5min":1.879,"15min":1.879},"last":0.76600000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2090000000000001,"15min":1.0629999999999999},"min":{"1min":0.76200000000000001,"5min":0.70899999999999996,"15min":0.65700000000000003},"max":{"1min":1.782,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1599999999999999,"15min":1.0660000000000001},"min":{"1min":0.81899999999999995,"5min":0.71299999999999997,"15min":0.61299999999999999},"max":{"1min":1.5640000000000001,"5min":1.6359999999999999,"15min":1.843},"last":1.2509999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.155,"15min":1.0860000000000001},"min":{"1min":0.81799999999999995,"5min":0.74299999999999999,"15min":0.59499999999999997},"max":{"1min":1.3069999999999999,"5min":1.794,"15min":1.794},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.258,"15min":1.147},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.65200000000000002},"max":{"1min":1.5920000000000001,"5min":2.036,"15min":2.036},"last":0.93400000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2010000000000001,"15min":1.097},"min":{"1min":0.83699999999999997,"5min":0.71999999999999997,"15min":0.68899999999999995},"max":{"1min":1.72,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.101},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3320000000000001,"15min":1.23},"min":{"1min":0.91300000000000003,"5min":0.75700000000000001,"15min":0.70399999999999996},"max":{"1min":1.595,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.226}]},{"osd":4,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3069999999999999,"15min":1.179},"min":{"1min":0.95299999999999996,"5min":0.60899999999999999,"15min":0.46700000000000003},"max":{"1min":1.6479999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.1990000000000001,"15min":1.131},"min":{"1min":0.90500000000000003,"5min":0.73999999999999999,"15min":0.57099999999999995},"max":{"1min":1.7689999999999999,"5min":1.804,"15min":1.901},"last":1.526}]},{"osd":6,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4299999999999999,"15min":1.3360000000000001},"min":{"1min":1.054,"5min":0.66600000000000004,"15min":0.54400000000000004},"max":{"1min":1.839,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.319,"15min":1.2470000000000001},"min":{"1min":1.163,"5min":0.81200000000000006,"15min":0.49299999999999999},"max":{"1min":1.702,"5min":1.7949999999999999,"15min":1.976},"last":1.3999999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.274,"15min":1.2290000000000001},"min":{"1min":1.0189999999999999,"5min":0.68300000000000005,"15min":0.50900000000000001},"max":{"1min":1.6779999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4570000000000001,"15min":1.4119999999999999},"min":{"1min":1.2310000000000001,"5min":0.84999999999999998,"15min":0.623},"max":{"1min":1.8120000000000001,"5min":2.117,"15min":2.117},"last":1.6120000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":736,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199245902,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98899999999999999,"15min":0.998},"min":{"1min":0.68500000000000005,"5min":0.62,"15min":0.62},"max":{"1min":1.3899999999999999,"5min":1.579,"15min":1.579},"last":1.45},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0409999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4259999999999999,"5min":1.7470000000000001,"15min":1.7909999999999999},"last":1.502}]},{"osd":1,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1679999999999999,"15min":1.149},"min":{"1min":0.67200000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.589,"5min":1.7130000000000001,"15min":1.8160000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.29,"5min":1.2549999999999999,"15min":1.1850000000000001},"min":{"1min":0.77400000000000002,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.696},"last":1.2050000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1499999999999999,"15min":1.127},"min":{"1min":0.82399999999999995,"5min":0.749,"15min":0.69799999999999995},"max":{"1min":1.407,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.159},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.1060000000000001},"min":{"1min":0.872,"5min":0.72199999999999998,"15min":0.70499999999999996},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.2430000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.294,"15min":1.26},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.357},{"interface":"front","average":{"1min":1.369,"5min":1.3340000000000001,"15min":1.304},"min":{"1min":0.93100000000000005,"5min":0.82399999999999995,"15min":0.74399999999999999},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.5780000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.294,"15min":1.2030000000000001},"min":{"1min":0.90000000000000002,"5min":0.68600000000000005,"15min":0.44500000000000001},"max":{"1min":1.46,"5min":1.76,"15min":1.8740000000000001},"last":1.331},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.413,"15min":1.353},"min":{"1min":0.76500000000000001,"5min":0.625,"15min":0.504},"max":{"1min":1.827,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.93200000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3160000000000001,"15min":1.304},"min":{"1min":0.98399999999999999,"5min":0.71299999999999997,"15min":0.432},"max":{"1min":1.5509999999999999,"5min":1.7809999999999999,"15min":1.853},"last":1.296},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4630000000000001,"15min":1.3959999999999999},"min":{"1min":1.2170000000000001,"5min":0.73599999999999999,"15min":0.64500000000000002},"max":{"1min":1.7829999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.4650000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.3999999999999999,"15min":1.3480000000000001},"min":{"1min":1.0249999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.7589999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4379999999999999,"15min":1.4079999999999999},"min":{"1min":0.88500000000000001,"5min":0.60399999999999998,"15min":0.57399999999999995},"max":{"1min":1.8,"5min":1.905,"15min":1.9330000000000001},"last":1.407}]}]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":736,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199246290,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.52900000000000003,"5min":0.38100000000000001,"15min":0.35299999999999998},"max":{"1min":2.516,"5min":3.3140000000000001,"15min":3.4350000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1,"15min":0.94499999999999995},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.3490000000000002,"5min":4.3490000000000002,"15min":4.3490000000000002},"last":1.0209999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.0629999999999999,"15min":1.0620000000000001},"min":{"1min":0.45300000000000001,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":5.2510000000000003,"5min":5.2510000000000003,"15min":5.2510000000000003},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.403,"5min":1.1719999999999999,"15min":1.1160000000000001},"min":{"1min":0.63800000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":5.4119999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.2250000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.377,"15min":1.25},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.335,"5min":13.833,"15min":13.833},"last":1.417},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5349999999999999,"15min":1.306},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.53200000000000003},"max":{"1min":5.4480000000000004,"5min":16.748999999999999,"15min":16.748999999999999},"last":1.556}]},{"osd":4,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.3620000000000001,"15min":1.2330000000000001},"min":{"1min":0.81899999999999995,"5min":0.77400000000000002,"15min":0.65600000000000003},"max":{"1min":5.1790000000000003,"5min":5.1790000000000003,"15min":5.1790000000000003},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.591,"5min":1.4239999999999999,"15min":1.409},"min":{"1min":0.873,"5min":0.80800000000000005,"15min":0.73199999999999998},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":1.1719999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.4750000000000001,"15min":1.45},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.76000000000000001},"max":{"1min":5.3289999999999997,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.367,"15min":1.2789999999999999},"min":{"1min":0.83799999999999997,"5min":0.80600000000000005,"15min":0.69999999999999996},"max":{"1min":4.6369999999999996,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.7470000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.4630000000000001,"15min":1.476},"min":{"1min":0.81999999999999995,"5min":0.81799999999999995,"15min":0.81599999999999995},"max":{"1min":4.5039999999999996,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":1.52},{"interface":"front","average":{"1min":1.667,"5min":1.548,"15min":1.514},"min":{"1min":0.83999999999999997,"5min":0.83199999999999996,"15min":0.80500000000000005},"max":{"1min":5.3739999999999997,"5min":5.3739999999999997,"15min":5.3739999999999997},"last":1.679}]},{"osd":7,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.476,"15min":1.385},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.73199999999999998},"max":{"1min":5.1349999999999998,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.843},{"interface":"front","average":{"1min":1.712,"5min":1.585,"15min":1.554},"min":{"1min":0.92200000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":4.4619999999999997,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":1.621}]}]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":735,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199246414,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.075,"15min":1.0109999999999999},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.34399999999999997},"max":{"1min":1.506,"5min":9.2550000000000008,"15min":9.2550000000000008},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0409999999999999,"15min":1.0389999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.7649999999999999,"5min":2.387,"15min":4.7850000000000001},"last":1.1679999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95999999999999996,"15min":1.165},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.46000000000000002},"max":{"1min":1.675,"5min":2.4390000000000001,"15min":8.2720000000000002},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.107,"15min":1.1619999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6519999999999999,"5min":2.7200000000000002,"15min":5.0620000000000003},"last":1.802}]},{"osd":3,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1830000000000001,"15min":1.3620000000000001},"min":{"1min":0.63300000000000001,"5min":0.54200000000000004,"15min":0.51900000000000002},"max":{"1min":1.708,"5min":2.238,"15min":8.3109999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.163,"15min":1.327},"min":{"1min":0.69599999999999995,"5min":0.46300000000000002,"15min":0.41799999999999998},"max":{"1min":1.73,"5min":2.323,"15min":8.0690000000000008},"last":1.3340000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.417,"15min":1.4610000000000001},"min":{"1min":0.93000000000000005,"5min":0.83399999999999996,"15min":0.79300000000000004},"max":{"1min":1.8009999999999999,"5min":2.6139999999999999,"15min":5.3929999999999998},"last":1.623},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.468,"15min":1.5489999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.78300000000000003},"max":{"1min":2.1949999999999998,"5min":3.1070000000000002,"15min":7.3570000000000002},"last":2.1179999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3939999999999999,"15min":1.546},"min":{"1min":0.93200000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9199999999999999,"5min":2.5579999999999998,"15min":8.1839999999999993},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.417},"min":{"1min":0.99299999999999999,"5min":0.81000000000000005,"15min":0.72199999999999998},"max":{"1min":1.845,"5min":2.4940000000000002,"15min":5.165},"last":1.8240000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.4350000000000001,"15min":1.454},"min":{"1min":0.82399999999999995,"5min":0.752,"15min":0.72999999999999998},"max":{"1min":2.5139999999999998,"5min":3.0179999999999998,"15min":5.1059999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.4650000000000001,"15min":1.5589999999999999},"min":{"1min":1.0049999999999999,"5min":0.81200000000000006,"15min":0.77000000000000002},"max":{"1min":1.879,"5min":2.9580000000000002,"15min":8.1370000000000005},"last":1.915}]},{"osd":7,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.5149999999999999,"15min":1.552},"min":{"1min":0.98499999999999999,"5min":0.79200000000000004,"15min":0.78200000000000003},"max":{"1min":2.3700000000000001,"5min":3.2189999999999999,"15min":5.1449999999999996},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.696,"5min":1.548,"15min":1.659},"min":{"1min":1.054,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":4.8289999999999997,"5min":4.8289999999999997,"15min":8.2300000000000004},"last":1.7629999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1314,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193998,"statfs":{"total":200000000,"available":198654440,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.84599999999999997},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3660000000000001,"5min":1.52,"15min":1.52},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.46899999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0720000000000001,"15min":1.0609999999999999},"min":{"1min":0.52500000000000002,"5min":0.42299999999999999,"15min":0.41399999999999998},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.5739999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.026,"15min":0.93600000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":1.6839999999999999,"5min":1.845,"15min":1.845},"last":0.52200000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3049999999999999,"15min":1.1719999999999999},"min":{"1min":0.64200000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.1040000000000001,"5min":2.137,"15min":2.137},"last":0.59599999999999997},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.2170000000000001,"15min":1.1930000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.44900000000000001},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.3959999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3520000000000001,"15min":1.2989999999999999},"min":{"1min":0.83999999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1440000000000001,"5min":2.444,"15min":2.444},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4039999999999999,"15min":1.3100000000000001},"min":{"1min":0.95299999999999996,"5min":0.76900000000000002,"15min":0.71099999999999997},"max":{"1min":1.7569999999999999,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":0.96499999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.425,"15min":1.3100000000000001},"min":{"1min":0.92800000000000005,"5min":0.88,"15min":0.88},"max":{"1min":1.915,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5049999999999999,"15min":1.423},"min":{"1min":1.0009999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.371,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.107}]},{"osd":6,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4530000000000001,"15min":1.3899999999999999},"min":{"1min":0.88400000000000001,"5min":0.875,"15min":0.76900000000000002},"max":{"1min":2.3610000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.538,"15min":1.46},"min":{"1min":1.032,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.2719999999999998,"5min":2.895,"15min":2.895},"last":1.014}]},{"osd":7,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.571,"15min":1.397},"min":{"1min":1.018,"5min":0.879,"15min":0.69999999999999996},"max":{"1min":1.8129999999999999,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":1.002},{"interface":"front","average":{"1min":1.468,"5min":1.659,"15min":1.508},"min":{"1min":0.97399999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.2360000000000002,"5min":9.4109999999999996,"15min":9.4109999999999996},"last":0.98099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":735,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199246484,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.99399999999999999,"15min":0.90600000000000003},"min":{"1min":0.64000000000000001,"5min":0.46700000000000003,"15min":0.39900000000000002},"max":{"1min":1.3220000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.143,"15min":1.0429999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.0369999999999999,"5min":2.1040000000000001,"15min":2.4180000000000001},"last":1.3460000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0660000000000001,"15min":0.96299999999999997},"min":{"1min":0.73099999999999998,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":1.4810000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0389999999999999,"15min":0.94699999999999995},"min":{"1min":0.55100000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.335,"5min":2.7810000000000001,"15min":2.7810000000000001},"last":0.70299999999999996}]},{"osd":3,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0880000000000001,"15min":1.0209999999999999},"min":{"1min":0.61299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.732,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.999},{"interface":"front","average":{"1min":1.306,"5min":1.3580000000000001,"15min":1.2390000000000001},"min":{"1min":0.746,"5min":0.63100000000000001,"15min":0.47499999999999998},"max":{"1min":1.982,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.411}]},{"osd":4,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.335,"15min":1.216},"min":{"1min":0.86299999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6739999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.282},{"interface":"front","average":{"1min":1.343,"5min":1.3360000000000001,"15min":1.2929999999999999},"min":{"1min":1.0409999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.663,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.119}]},{"osd":5,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.5489999999999999,"15min":1.4059999999999999},"min":{"1min":1.0169999999999999,"5min":0.71099999999999997,"15min":0.64500000000000002},"max":{"1min":1.877,"5min":16.298999999999999,"15min":16.298999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.385,"15min":1.2909999999999999},"min":{"1min":0.94899999999999995,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8109999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.696}]},{"osd":6,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.5029999999999999,"15min":1.409},"min":{"1min":1.171,"5min":0.85499999999999998,"15min":0.75900000000000001},"max":{"1min":1.9119999999999999,"5min":2.2559999999999998,"15min":2.5910000000000002},"last":1.776},{"interface":"front","average":{"1min":1.391,"5min":1.462,"15min":1.369},"min":{"1min":1.0349999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.776,"5min":2.46,"15min":2.46},"last":1.3819999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5369999999999999,"15min":1.409},"min":{"1min":1.1279999999999999,"5min":0.89200000000000002,"15min":0.70899999999999996},"max":{"1min":1.8740000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.466,"15min":1.4330000000000001},"min":{"1min":0.88400000000000001,"5min":0.873,"15min":0.81999999999999995},"max":{"1min":2.113,"5min":2.4580000000000002,"15min":3.1429999999999998},"last":1.3100000000000001}]}]}],"pool_statfs":[]}} 2023-12-19T18:22:20.331 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-19T18:22:20.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.605+0000 7f9bcc9cb640 1 Processor -- start 2023-12-19T18:22:20.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.605+0000 7f9bcc9cb640 1 -- start start 2023-12-19T18:22:20.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.607+0000 7f9bcc9cb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bc80c9770 0x7f9bc806ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:20.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.607+0000 7f9bcc9cb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9bc8070470 0x7f9bc8070900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:20.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.607+0000 7f9bcc9cb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc80cc720 con 0x7f9bc806cec0 2023-12-19T18:22:20.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.607+0000 7f9bcc9cb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc815a330 con 0x7f9bc8070470 2023-12-19T18:22:20.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.607+0000 7f9bcc9cb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80b1a00 con 0x7f9bc80c9770 2023-12-19T18:22:20.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.607+0000 7f9bc6d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9bc8070470 0x7f9bc8070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:20.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.607+0000 7f9bc659c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9bc806cec0 0x7f9bc806d2a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41290/0 (socket says 172.21.15.27:41290) 2023-12-19T18:22:20.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.607+0000 7f9bc659c640 1 -- 172.21.15.27:0/166524773 learned_addr learned my addr 172.21.15.27:0/166524773 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:20.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.607+0000 7f9bc5d9b640 1 --2- 172.21.15.27:0/166524773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bc80c9770 0x7f9bc806ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:20.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.608+0000 7f9bc6d9d640 1 -- 172.21.15.27:0/166524773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bc80c9770 msgr2=0x7f9bc806ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:20.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.608+0000 7f9bc6d9d640 1 --2- 172.21.15.27:0/166524773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bc80c9770 0x7f9bc806ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.608+0000 7f9bc6d9d640 1 -- 172.21.15.27:0/166524773 >> v1:172.21.15.27:6789/0 conn(0x7f9bc806cec0 legacy=0x7f9bc806d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:20.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.608+0000 7f9bc6d9d640 1 -- 172.21.15.27:0/166524773 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc8071640 con 0x7f9bc8070470 2023-12-19T18:22:20.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.609+0000 7f9bc6d9d640 1 --2- 172.21.15.27:0/166524773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc8070470 0x7f9bc8070900 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009d90 tx=0x7f9bc002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee186a92519b8c36 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:20.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.609+0000 7f9bc559a640 1 -- 172.21.15.27:0/166524773 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc002cc60 con 0x7f9bc8070470 2023-12-19T18:22:20.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.609+0000 7f9bc559a640 1 -- 172.21.15.27:0/166524773 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc002cdc0 con 0x7f9bc8070470 2023-12-19T18:22:20.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.609+0000 7f9bc559a640 1 -- 172.21.15.27:0/166524773 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc0034660 con 0x7f9bc8070470 2023-12-19T18:22:20.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.610+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/166524773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc8070470 msgr2=0x7f9bc8070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:20.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.610+0000 7f9bcc9cb640 1 --2- 172.21.15.27:0/166524773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc8070470 0x7f9bc8070900 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009d90 tx=0x7f9bc002beb0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.610+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/166524773 shutdown_connections 2023-12-19T18:22:20.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.610+0000 7f9bcc9cb640 1 --2- 172.21.15.27:0/166524773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc8070470 0x7f9bc8070900 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.610+0000 7f9bcc9cb640 1 --2- 172.21.15.27:0/166524773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bc80c9770 0x7f9bc806ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.610+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/166524773 >> 172.21.15.27:0/166524773 conn(0x7f9bc80ceb50 msgr2=0x7f9bc80cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:20.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.610+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/166524773 shutdown_connections 2023-12-19T18:22:20.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.610+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/166524773 wait complete. 2023-12-19T18:22:20.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.612+0000 7f9bcc9cb640 1 Processor -- start 2023-12-19T18:22:20.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.612+0000 7f9bcc9cb640 1 -- start start 2023-12-19T18:22:20.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.613+0000 7f9bcc9cb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bc806cec0 0x7f9bc80b2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:20.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.613+0000 7f9bcc9cb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bc8070470 0x7f9bc8147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:20.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.613+0000 7f9bcc9cb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc80c9770 0x7f9bc80c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:20.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.613+0000 7f9bcc9cb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80b2620 con 0x7f9bc80c9770 2023-12-19T18:22:20.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.613+0000 7f9bcc9cb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80c94d0 con 0x7f9bc8070470 2023-12-19T18:22:20.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.613+0000 7f9bcc9cb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc80b2160 con 0x7f9bc806cec0 2023-12-19T18:22:20.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.613+0000 7f9bc6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc80c9770 0x7f9bc80c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:20.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.613+0000 7f9bc6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc80c9770 0x7f9bc80c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43032/0 (socket says 172.21.15.27:43032) 2023-12-19T18:22:20.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.613+0000 7f9bc6d9d640 1 -- 172.21.15.27:0/2528973942 learned_addr learned my addr 172.21.15.27:0/2528973942 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:20.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.614+0000 7f9bc659c640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bc806cec0 0x7f9bc80b2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:20.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.614+0000 7f9bc5d9b640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bc8070470 0x7f9bc8147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:20.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.614+0000 7f9bc6d9d640 1 -- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bc806cec0 msgr2=0x7f9bc80b2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:20.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.614+0000 7f9bc6d9d640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bc806cec0 0x7f9bc80b2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.614+0000 7f9bc6d9d640 1 -- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bc8070470 msgr2=0x7f9bc8147ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:20.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.614+0000 7f9bc6d9d640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bc8070470 0x7f9bc8147ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.614+0000 7f9bc6d9d640 1 -- 172.21.15.27:0/2528973942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc0009a70 con 0x7f9bc80c9770 2023-12-19T18:22:20.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.614+0000 7f9bc6d9d640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc80c9770 0x7f9bc80c8f90 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009d90 tx=0x7f9bc0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:20.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.615+0000 7f9bb6ffd640 1 -- 172.21.15.27:0/2528973942 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc003e610 con 0x7f9bc80c9770 2023-12-19T18:22:20.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.615+0000 7f9bb6ffd640 1 -- 172.21.15.27:0/2528973942 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc003e790 con 0x7f9bc80c9770 2023-12-19T18:22:20.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.615+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/2528973942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc80b43e0 con 0x7f9bc80c9770 2023-12-19T18:22:20.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.615+0000 7f9bb6ffd640 1 -- 172.21.15.27:0/2528973942 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc003eb40 con 0x7f9bc80c9770 2023-12-19T18:22:20.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.615+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/2528973942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc80b49a0 con 0x7f9bc80c9770 2023-12-19T18:22:20.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.616+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/2528973942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc80b4240 con 0x7f9bc80c9770 2023-12-19T18:22:20.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.617+0000 7f9bb6ffd640 1 -- 172.21.15.27:0/2528973942 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9bc003b050 con 0x7f9bc80c9770 2023-12-19T18:22:20.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.618+0000 7f9bb6ffd640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ba806a430 0x7f9ba806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:20.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.618+0000 7f9bb6ffd640 1 -- 172.21.15.27:0/2528973942 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9bc00ab2d0 con 0x7f9bc80c9770 2023-12-19T18:22:20.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.619+0000 7f9bc659c640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ba806a430 0x7f9ba806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:20.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.620+0000 7f9bc659c640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ba806a430 0x7f9ba806c8f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8005880 tx=0x7f9bb8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:20.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.622+0000 7f9bb6ffd640 1 -- 172.21.15.27:0/2528973942 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9bc80b4240 con 0x7f9bc80c9770 2023-12-19T18:22:20.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.889+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/2528973942 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f9bc80b2760 con 0x7f9bc80c9770 2023-12-19T18:22:20.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.890+0000 7f9bb6ffd640 1 -- 172.21.15.27:0/2528973942 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1169 (secure 0 0 0) 0x7f9bc80b2760 con 0x7f9bc80c9770 2023-12-19T18:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:20.890 INFO:teuthology.orchestra.run.smithi027.stdout:{"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":464,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":135,"num_osds":8,"num_up_osds":8,"osd_up_since":1703009683,"num_in_osds":8,"osd_in_since":1703009680,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":23,"data_bytes":592730,"bytes_used":7213588,"bytes_avail":1592786412,"bytes_total":1600000000,"write_bytes_sec":2326,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":54,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7320}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":13,"modified":"2023-12-19T18:22:03.861875+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-19T18:22:20.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.892+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ba806a430 msgr2=0x7f9ba806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.892+0000 7f9bcc9cb640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ba806a430 0x7f9ba806c8f0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9bb8005880 tx=0x7f9bb8009040 comp rx=0 tx=0).stop 2023-12-19T18:22:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.892+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/2528973942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc80c9770 msgr2=0x7f9bc80c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.892+0000 7f9bcc9cb640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc80c9770 0x7f9bc80c8f90 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009d90 tx=0x7f9bc0004a40 comp rx=0 tx=0).stop 2023-12-19T18:22:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.892+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/2528973942 shutdown_connections 2023-12-19T18:22:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.892+0000 7f9bcc9cb640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ba806a430 0x7f9ba806c8f0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.892+0000 7f9bcc9cb640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9bc80c9770 0x7f9bc80c8f90 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.892+0000 7f9bcc9cb640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9bc8070470 0x7f9bc8147ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.892+0000 7f9bcc9cb640 1 --2- 172.21.15.27:0/2528973942 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9bc806cec0 0x7f9bc80b2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.892+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/2528973942 >> 172.21.15.27:0/2528973942 conn(0x7f9bc80ceb50 msgr2=0x7f9bc80ce220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.893+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/2528973942 shutdown_connections 2023-12-19T18:22:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:20.893+0000 7f9bcc9cb640 1 -- 172.21.15.27:0/2528973942 wait complete. 2023-12-19T18:22:20.909 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-19T18:22:20.910 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:22:21.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.125+0000 7f36c482b640 1 Processor -- start 2023-12-19T18:22:21.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.125+0000 7f36c482b640 1 -- start start 2023-12-19T18:22:21.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.127+0000 7f36c482b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f36c01553a0 0x7f36c00c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:21.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.127+0000 7f36c482b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c00afeb0 0x7f36c015e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.127+0000 7f36c482b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36c00b0330 con 0x7f36c015ec60 2023-12-19T18:22:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.127+0000 7f36c482b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36c00cbcf0 con 0x7f36c01553a0 2023-12-19T18:22:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.127+0000 7f36c482b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c015f0d0 con 0x7f36c00afeb0 2023-12-19T18:22:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.127+0000 7f36bed9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f36c01553a0 0x7f36c00c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.127+0000 7f36bf59e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f36c015ec60 0x7f36c015b130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:41304/0 (socket says 172.21.15.27:41304) 2023-12-19T18:22:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.127+0000 7f36bf59e640 1 -- 172.21.15.27:0/2381504968 learned_addr learned my addr 172.21.15.27:0/2381504968 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:21.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.128+0000 7f36b7fff640 1 --2- 172.21.15.27:0/2381504968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c00afeb0 0x7f36c015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.128+0000 7f36bed9d640 1 -- 172.21.15.27:0/2381504968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c00afeb0 msgr2=0x7f36c015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.128+0000 7f36bed9d640 1 --2- 172.21.15.27:0/2381504968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c00afeb0 0x7f36c015e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.128+0000 7f36bed9d640 1 -- 172.21.15.27:0/2381504968 >> v1:172.21.15.27:6789/0 conn(0x7f36c015ec60 legacy=0x7f36c015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.128+0000 7f36bed9d640 1 -- 172.21.15.27:0/2381504968 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c00cb350 con 0x7f36c01553a0 2023-12-19T18:22:21.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.128+0000 7f36be59c640 1 -- 172.21.15.27:0/2381504968 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 76490087 0 0) 0x7f36c00b0330 con 0x7f36c015ec60 2023-12-19T18:22:21.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.129+0000 7f36bed9d640 1 --2- 172.21.15.27:0/2381504968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36c01553a0 0x7f36c00c50a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f36b00097d0 tx=0x7f36b002b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ccea26d511707d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:21.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.130+0000 7f36be59c640 1 -- 172.21.15.27:0/2381504968 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0031770 con 0x7f36c01553a0 2023-12-19T18:22:21.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.130+0000 7f36be59c640 1 -- 172.21.15.27:0/2381504968 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b00318d0 con 0x7f36c01553a0 2023-12-19T18:22:21.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.130+0000 7f36be59c640 1 -- 172.21.15.27:0/2381504968 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0031c60 con 0x7f36c01553a0 2023-12-19T18:22:21.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.130+0000 7f36c482b640 1 -- 172.21.15.27:0/2381504968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36c01553a0 msgr2=0x7f36c00c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:21.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.130+0000 7f36c482b640 1 --2- 172.21.15.27:0/2381504968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36c01553a0 0x7f36c00c50a0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f36b00097d0 tx=0x7f36b002b810 comp rx=0 tx=0).stop 2023-12-19T18:22:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.130+0000 7f36c482b640 1 -- 172.21.15.27:0/2381504968 shutdown_connections 2023-12-19T18:22:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.130+0000 7f36c482b640 1 --2- 172.21.15.27:0/2381504968 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c00afeb0 0x7f36c015e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.130+0000 7f36c482b640 1 --2- 172.21.15.27:0/2381504968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36c01553a0 0x7f36c00c50a0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.130+0000 7f36c482b640 1 -- 172.21.15.27:0/2381504968 >> 172.21.15.27:0/2381504968 conn(0x7f36c0069c70 msgr2=0x7f36c006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.130+0000 7f36c482b640 1 -- 172.21.15.27:0/2381504968 shutdown_connections 2023-12-19T18:22:21.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.130+0000 7f36c482b640 1 -- 172.21.15.27:0/2381504968 wait complete. 2023-12-19T18:22:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.132+0000 7f36c482b640 1 Processor -- start 2023-12-19T18:22:21.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.132+0000 7f36c482b640 1 -- start start 2023-12-19T18:22:21.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.133+0000 7f36c482b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36c00afeb0 0x7f36c01f27b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:21.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.133+0000 7f36c482b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c01553a0 0x7f36c0156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:21.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.133+0000 7f36c482b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c015ec60 0x7f36c0159c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:21.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.133+0000 7f36c482b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c015c2d0 con 0x7f36c00afeb0 2023-12-19T18:22:21.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.133+0000 7f36c482b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36c0155b70 con 0x7f36c01553a0 2023-12-19T18:22:21.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.133+0000 7f36c482b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36c01f3030 con 0x7f36c015ec60 2023-12-19T18:22:21.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.133+0000 7f36bed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36c00afeb0 0x7f36c01f27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.134+0000 7f36bed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36c00afeb0 0x7f36c01f27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43048/0 (socket says 172.21.15.27:43048) 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.134+0000 7f36bed9d640 1 -- 172.21.15.27:0/2836036624 learned_addr learned my addr 172.21.15.27:0/2836036624 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.134+0000 7f36b7fff640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c01553a0 0x7f36c0156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.134+0000 7f36bf59e640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c015ec60 0x7f36c0159c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.134+0000 7f36bed9d640 1 -- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c015ec60 msgr2=0x7f36c0159c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.134+0000 7f36bed9d640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c015ec60 0x7f36c0159c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.134+0000 7f36bed9d640 1 -- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c01553a0 msgr2=0x7f36c0156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.134+0000 7f36bed9d640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c01553a0 0x7f36c0156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.134+0000 7f36bed9d640 1 -- 172.21.15.27:0/2836036624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b0009400 con 0x7f36c00afeb0 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.134+0000 7f36bf59e640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c015ec60 0x7f36c0159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.134+0000 7f36b7fff640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c01553a0 0x7f36c0156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.134+0000 7f36bed9d640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36c00afeb0 0x7f36c01f27b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f36b0034040 tx=0x7f36b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.135+0000 7f36b77fe640 1 -- 172.21.15.27:0/2836036624 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0004230 con 0x7f36c00afeb0 2023-12-19T18:22:21.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.135+0000 7f36b77fe640 1 -- 172.21.15.27:0/2836036624 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36b00043b0 con 0x7f36c00afeb0 2023-12-19T18:22:21.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.135+0000 7f36b77fe640 1 -- 172.21.15.27:0/2836036624 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36b0004520 con 0x7f36c00afeb0 2023-12-19T18:22:21.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.135+0000 7f36c482b640 1 -- 172.21.15.27:0/2836036624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36c01f3310 con 0x7f36c00afeb0 2023-12-19T18:22:21.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.135+0000 7f36c482b640 1 -- 172.21.15.27:0/2836036624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36c015c4e0 con 0x7f36c00afeb0 2023-12-19T18:22:21.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.136+0000 7f36c482b640 1 -- 172.21.15.27:0/2836036624 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36c004e840 con 0x7f36c00afeb0 2023-12-19T18:22:21.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.137+0000 7f36b77fe640 1 -- 172.21.15.27:0/2836036624 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f36b0003850 con 0x7f36c00afeb0 2023-12-19T18:22:21.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.138+0000 7f36b77fe640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f36a006a430 0x7f36a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:21.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.138+0000 7f36b77fe640 1 -- 172.21.15.27:0/2836036624 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36b00aa9e0 con 0x7f36c00afeb0 2023-12-19T18:22:21.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.139+0000 7f36b7fff640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f36a006a430 0x7f36a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:21.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.140+0000 7f36b7fff640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f36a006a430 0x7f36a006c8f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f36a8006fd0 tx=0x7f36a8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:21.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.142+0000 7f36b77fe640 1 -- 172.21.15.27:0/2836036624 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f36b0075f30 con 0x7f36c00afeb0 2023-12-19T18:22:21.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.326+0000 7f36c482b640 1 -- 172.21.15.27:0/2836036624 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36c00cb910 con 0x7f36a006a430 2023-12-19T18:22:21.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.349+0000 7f36b77fe640 1 -- 172.21.15.27:0/2836036624 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283057 (secure 0 0 0) 0x7f36a8009530 con 0x7f36a006a430 2023-12-19T18:22:21.353 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:21.355 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:22:21.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.354+0000 7f36c482b640 1 -- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f36a006a430 msgr2=0x7f36a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:21.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.354+0000 7f36c482b640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f36a006a430 0x7f36a006c8f0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f36a8006fd0 tx=0x7f36a8008040 comp rx=0 tx=0).stop 2023-12-19T18:22:21.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.354+0000 7f36c482b640 1 -- 172.21.15.27:0/2836036624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36c00afeb0 msgr2=0x7f36c01f27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:21.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.354+0000 7f36c482b640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36c00afeb0 0x7f36c01f27b0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f36b0034040 tx=0x7f36b00046c0 comp rx=0 tx=0).stop 2023-12-19T18:22:21.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.355+0000 7f36c482b640 1 -- 172.21.15.27:0/2836036624 shutdown_connections 2023-12-19T18:22:21.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.355+0000 7f36c482b640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f36a006a430 0x7f36a006c8f0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:21.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.355+0000 7f36c482b640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f36c015ec60 0x7f36c0159c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:21.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.355+0000 7f36c482b640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f36c01553a0 0x7f36c0156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:21.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.355+0000 7f36c482b640 1 --2- 172.21.15.27:0/2836036624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36c00afeb0 0x7f36c01f27b0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:21.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.355+0000 7f36c482b640 1 -- 172.21.15.27:0/2836036624 >> 172.21.15.27:0/2836036624 conn(0x7f36c0069c70 msgr2=0x7f36c015b8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:21.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.355+0000 7f36c482b640 1 -- 172.21.15.27:0/2836036624 shutdown_connections 2023-12-19T18:22:21.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:21.355+0000 7f36c482b640 1 -- 172.21.15.27:0/2836036624 wait complete. 2023-12-19T18:22:21.374 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":355,"stamp":"2023-12-19T18:22:19.866577+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":183,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7044,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555455,"statfs":{"total":1600000000,"available":1592786412,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.162920"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258377+0000","last_change":"2023-12-19T18:22:14.257437+0000","last_active":"2023-12-19T18:22:14.258377+0000","last_peered":"2023-12-19T18:22:14.258377+0000","last_clean":"2023-12-19T18:22:14.258377+0000","last_became_active":"2023-12-19T18:22:14.256527+0000","last_became_peered":"2023-12-19T18:22:14.256527+0000","last_unstale":"2023-12-19T18:22:14.258377+0000","last_undegraded":"2023-12-19T18:22:14.258377+0000","last_fullsized":"2023-12-19T18:22:14.258377+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:40.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258346+0000","last_change":"2023-12-19T18:22:14.255774+0000","last_active":"2023-12-19T18:22:14.258346+0000","last_peered":"2023-12-19T18:22:14.258346+0000","last_clean":"2023-12-19T18:22:14.258346+0000","last_became_active":"2023-12-19T18:22:14.255220+0000","last_became_peered":"2023-12-19T18:22:14.255220+0000","last_unstale":"2023-12-19T18:22:14.258346+0000","last_undegraded":"2023-12-19T18:22:14.258346+0000","last_fullsized":"2023-12-19T18:22:14.258346+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:15.726948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.256564+0000","last_change":"2023-12-19T18:22:14.256359+0000","last_active":"2023-12-19T18:22:14.256564+0000","last_peered":"2023-12-19T18:22:14.256564+0000","last_clean":"2023-12-19T18:22:14.256564+0000","last_became_active":"2023-12-19T18:22:14.255780+0000","last_became_peered":"2023-12-19T18:22:14.255780+0000","last_unstale":"2023-12-19T18:22:14.256564+0000","last_undegraded":"2023-12-19T18:22:14.256564+0000","last_fullsized":"2023-12-19T18:22:14.256564+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:16.333492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258589+0000","last_change":"2023-12-19T18:22:14.258495+0000","last_active":"2023-12-19T18:22:14.258589+0000","last_peered":"2023-12-19T18:22:14.258589+0000","last_clean":"2023-12-19T18:22:14.258589+0000","last_became_active":"2023-12-19T18:22:14.258217+0000","last_became_peered":"2023-12-19T18:22:14.258217+0000","last_unstale":"2023-12-19T18:22:14.258589+0000","last_undegraded":"2023-12-19T18:22:14.258589+0000","last_fullsized":"2023-12-19T18:22:14.258589+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:09.942336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.244066+0000","last_change":"2023-12-19T18:22:14.243278+0000","last_active":"2023-12-19T18:22:14.244066+0000","last_peered":"2023-12-19T18:22:14.244066+0000","last_clean":"2023-12-19T18:22:14.244066+0000","last_became_active":"2023-12-19T18:22:14.242452+0000","last_became_peered":"2023-12-19T18:22:14.242452+0000","last_unstale":"2023-12-19T18:22:14.244066+0000","last_undegraded":"2023-12-19T18:22:14.244066+0000","last_fullsized":"2023-12-19T18:22:14.244066+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:26.506333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.234713+0000","last_change":"2023-12-19T18:22:14.234640+0000","last_active":"2023-12-19T18:22:14.234713+0000","last_peered":"2023-12-19T18:22:14.234713+0000","last_clean":"2023-12-19T18:22:14.234713+0000","last_became_active":"2023-12-19T18:22:14.234044+0000","last_became_peered":"2023-12-19T18:22:14.234044+0000","last_unstale":"2023-12-19T18:22:14.234713+0000","last_undegraded":"2023-12-19T18:22:14.234713+0000","last_fullsized":"2023-12-19T18:22:14.234713+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:02:55.424114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239891+0000","last_change":"2023-12-19T18:22:14.239762+0000","last_active":"2023-12-19T18:22:14.239891+0000","last_peered":"2023-12-19T18:22:14.239891+0000","last_clean":"2023-12-19T18:22:14.239891+0000","last_became_active":"2023-12-19T18:22:14.239355+0000","last_became_peered":"2023-12-19T18:22:14.239355+0000","last_unstale":"2023-12-19T18:22:14.239891+0000","last_undegraded":"2023-12-19T18:22:14.239891+0000","last_fullsized":"2023-12-19T18:22:14.239891+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:37:37.391952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.256258+0000","last_change":"2023-12-19T18:22:14.256157+0000","last_active":"2023-12-19T18:22:14.256258+0000","last_peered":"2023-12-19T18:22:14.256258+0000","last_clean":"2023-12-19T18:22:14.256258+0000","last_became_active":"2023-12-19T18:22:14.255443+0000","last_became_peered":"2023-12-19T18:22:14.255443+0000","last_unstale":"2023-12-19T18:22:14.256258+0000","last_undegraded":"2023-12-19T18:22:14.256258+0000","last_fullsized":"2023-12-19T18:22:14.256258+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:23.915911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252425+0000","last_change":"2023-12-19T18:22:14.252334+0000","last_active":"2023-12-19T18:22:14.252425+0000","last_peered":"2023-12-19T18:22:14.252425+0000","last_clean":"2023-12-19T18:22:14.252425+0000","last_became_active":"2023-12-19T18:22:14.252176+0000","last_became_peered":"2023-12-19T18:22:14.252176+0000","last_unstale":"2023-12-19T18:22:14.252425+0000","last_undegraded":"2023-12-19T18:22:14.252425+0000","last_fullsized":"2023-12-19T18:22:14.252425+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:29.637020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259112+0000","last_change":"2023-12-19T18:22:14.258799+0000","last_active":"2023-12-19T18:22:14.259112+0000","last_peered":"2023-12-19T18:22:14.259112+0000","last_clean":"2023-12-19T18:22:14.259112+0000","last_became_active":"2023-12-19T18:22:14.258575+0000","last_became_peered":"2023-12-19T18:22:14.258575+0000","last_unstale":"2023-12-19T18:22:14.259112+0000","last_undegraded":"2023-12-19T18:22:14.259112+0000","last_fullsized":"2023-12-19T18:22:14.259112+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:11:57.860995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.234856+0000","last_change":"2023-12-19T18:22:14.234773+0000","last_active":"2023-12-19T18:22:14.234856+0000","last_peered":"2023-12-19T18:22:14.234856+0000","last_clean":"2023-12-19T18:22:14.234856+0000","last_became_active":"2023-12-19T18:22:14.234418+0000","last_became_peered":"2023-12-19T18:22:14.234418+0000","last_unstale":"2023-12-19T18:22:14.234856+0000","last_undegraded":"2023-12-19T18:22:14.234856+0000","last_fullsized":"2023-12-19T18:22:14.234856+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:23.903727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.256370+0000","last_change":"2023-12-19T18:22:14.256302+0000","last_active":"2023-12-19T18:22:14.256370+0000","last_peered":"2023-12-19T18:22:14.256370+0000","last_clean":"2023-12-19T18:22:14.256370+0000","last_became_active":"2023-12-19T18:22:14.256075+0000","last_became_peered":"2023-12-19T18:22:14.256075+0000","last_unstale":"2023-12-19T18:22:14.256370+0000","last_undegraded":"2023-12-19T18:22:14.256370+0000","last_fullsized":"2023-12-19T18:22:14.256370+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:31.105594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245932+0000","last_change":"2023-12-19T18:22:14.244697+0000","last_active":"2023-12-19T18:22:14.245932+0000","last_peered":"2023-12-19T18:22:14.245932+0000","last_clean":"2023-12-19T18:22:14.245932+0000","last_became_active":"2023-12-19T18:22:14.244184+0000","last_became_peered":"2023-12-19T18:22:14.244184+0000","last_unstale":"2023-12-19T18:22:14.245932+0000","last_undegraded":"2023-12-19T18:22:14.245932+0000","last_fullsized":"2023-12-19T18:22:14.245932+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:33.150825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.256363+0000","last_change":"2023-12-19T18:22:14.254104+0000","last_active":"2023-12-19T18:22:14.256363+0000","last_peered":"2023-12-19T18:22:14.256363+0000","last_clean":"2023-12-19T18:22:14.256363+0000","last_became_active":"2023-12-19T18:22:14.253879+0000","last_became_peered":"2023-12-19T18:22:14.253879+0000","last_unstale":"2023-12-19T18:22:14.256363+0000","last_undegraded":"2023-12-19T18:22:14.256363+0000","last_fullsized":"2023-12-19T18:22:14.256363+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:16.333492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259380+0000","last_change":"2023-12-19T18:22:14.258835+0000","last_active":"2023-12-19T18:22:14.259380+0000","last_peered":"2023-12-19T18:22:14.259380+0000","last_clean":"2023-12-19T18:22:14.259380+0000","last_became_active":"2023-12-19T18:22:14.257437+0000","last_became_peered":"2023-12-19T18:22:14.257437+0000","last_unstale":"2023-12-19T18:22:14.259380+0000","last_undegraded":"2023-12-19T18:22:14.259380+0000","last_fullsized":"2023-12-19T18:22:14.259380+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:34.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.240074+0000","last_change":"2023-12-19T18:22:14.239965+0000","last_active":"2023-12-19T18:22:14.240074+0000","last_peered":"2023-12-19T18:22:14.240074+0000","last_clean":"2023-12-19T18:22:14.240074+0000","last_became_active":"2023-12-19T18:22:14.239741+0000","last_became_peered":"2023-12-19T18:22:14.239741+0000","last_unstale":"2023-12-19T18:22:14.240074+0000","last_undegraded":"2023-12-19T18:22:14.240074+0000","last_fullsized":"2023-12-19T18:22:14.240074+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:02.299174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.237521+0000","last_change":"2023-12-19T18:22:14.237215+0000","last_active":"2023-12-19T18:22:14.237521+0000","last_peered":"2023-12-19T18:22:14.237521+0000","last_clean":"2023-12-19T18:22:14.237521+0000","last_became_active":"2023-12-19T18:22:14.236055+0000","last_became_peered":"2023-12-19T18:22:14.236055+0000","last_unstale":"2023-12-19T18:22:14.237521+0000","last_undegraded":"2023-12-19T18:22:14.237521+0000","last_fullsized":"2023-12-19T18:22:14.237521+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:41:21.616217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.261591+0000","last_change":"2023-12-19T18:22:14.261506+0000","last_active":"2023-12-19T18:22:14.261591+0000","last_peered":"2023-12-19T18:22:14.261591+0000","last_clean":"2023-12-19T18:22:14.261591+0000","last_became_active":"2023-12-19T18:22:14.259331+0000","last_became_peered":"2023-12-19T18:22:14.259331+0000","last_unstale":"2023-12-19T18:22:14.261591+0000","last_undegraded":"2023-12-19T18:22:14.261591+0000","last_fullsized":"2023-12-19T18:22:14.261591+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:44.388621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.260554+0000","last_change":"2023-12-19T18:22:14.260473+0000","last_active":"2023-12-19T18:22:14.260554+0000","last_peered":"2023-12-19T18:22:14.260554+0000","last_clean":"2023-12-19T18:22:14.260554+0000","last_became_active":"2023-12-19T18:22:14.259952+0000","last_became_peered":"2023-12-19T18:22:14.259952+0000","last_unstale":"2023-12-19T18:22:14.260554+0000","last_undegraded":"2023-12-19T18:22:14.260554+0000","last_fullsized":"2023-12-19T18:22:14.260554+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:41:30.585976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246133+0000","last_change":"2023-12-19T18:22:14.246068+0000","last_active":"2023-12-19T18:22:14.246133+0000","last_peered":"2023-12-19T18:22:14.246133+0000","last_clean":"2023-12-19T18:22:14.246133+0000","last_became_active":"2023-12-19T18:22:14.245845+0000","last_became_peered":"2023-12-19T18:22:14.245845+0000","last_unstale":"2023-12-19T18:22:14.246133+0000","last_undegraded":"2023-12-19T18:22:14.246133+0000","last_fullsized":"2023-12-19T18:22:14.246133+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:40.700106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.254074+0000","last_change":"2023-12-19T18:22:14.253786+0000","last_active":"2023-12-19T18:22:14.254074+0000","last_peered":"2023-12-19T18:22:14.254074+0000","last_clean":"2023-12-19T18:22:14.254074+0000","last_became_active":"2023-12-19T18:22:14.253409+0000","last_became_peered":"2023-12-19T18:22:14.253409+0000","last_unstale":"2023-12-19T18:22:14.254074+0000","last_undegraded":"2023-12-19T18:22:14.254074+0000","last_fullsized":"2023-12-19T18:22:14.254074+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:11.523949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.257498+0000","last_change":"2023-12-19T18:22:14.255178+0000","last_active":"2023-12-19T18:22:14.257498+0000","last_peered":"2023-12-19T18:22:14.257498+0000","last_clean":"2023-12-19T18:22:14.257498+0000","last_became_active":"2023-12-19T18:22:14.254755+0000","last_became_peered":"2023-12-19T18:22:14.254755+0000","last_unstale":"2023-12-19T18:22:14.257498+0000","last_undegraded":"2023-12-19T18:22:14.257498+0000","last_fullsized":"2023-12-19T18:22:14.257498+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:13:15.250452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252608+0000","last_change":"2023-12-19T18:22:14.250276+0000","last_active":"2023-12-19T18:22:14.252608+0000","last_peered":"2023-12-19T18:22:14.252608+0000","last_clean":"2023-12-19T18:22:14.252608+0000","last_became_active":"2023-12-19T18:22:14.250040+0000","last_became_peered":"2023-12-19T18:22:14.250040+0000","last_unstale":"2023-12-19T18:22:14.252608+0000","last_undegraded":"2023-12-19T18:22:14.252608+0000","last_fullsized":"2023-12-19T18:22:14.252608+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:08.465323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259701+0000","last_change":"2023-12-19T18:22:14.259592+0000","last_active":"2023-12-19T18:22:14.259701+0000","last_peered":"2023-12-19T18:22:14.259701+0000","last_clean":"2023-12-19T18:22:14.259701+0000","last_became_active":"2023-12-19T18:22:14.259307+0000","last_became_peered":"2023-12-19T18:22:14.259307+0000","last_unstale":"2023-12-19T18:22:14.259701+0000","last_undegraded":"2023-12-19T18:22:14.259701+0000","last_fullsized":"2023-12-19T18:22:14.259701+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:02:16.250351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239849+0000","last_change":"2023-12-19T18:22:14.236493+0000","last_active":"2023-12-19T18:22:14.239849+0000","last_peered":"2023-12-19T18:22:14.239849+0000","last_clean":"2023-12-19T18:22:14.239849+0000","last_became_active":"2023-12-19T18:22:14.236218+0000","last_became_peered":"2023-12-19T18:22:14.236218+0000","last_unstale":"2023-12-19T18:22:14.239849+0000","last_undegraded":"2023-12-19T18:22:14.239849+0000","last_fullsized":"2023-12-19T18:22:14.239849+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:32.485695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245899+0000","last_change":"2023-12-19T18:22:14.242241+0000","last_active":"2023-12-19T18:22:14.245899+0000","last_peered":"2023-12-19T18:22:14.245899+0000","last_clean":"2023-12-19T18:22:14.245899+0000","last_became_active":"2023-12-19T18:22:14.241678+0000","last_became_peered":"2023-12-19T18:22:14.241678+0000","last_unstale":"2023-12-19T18:22:14.245899+0000","last_undegraded":"2023-12-19T18:22:14.245899+0000","last_fullsized":"2023-12-19T18:22:14.245899+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:08.366785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248454+0000","last_change":"2023-12-19T18:22:14.247740+0000","last_active":"2023-12-19T18:22:14.248454+0000","last_peered":"2023-12-19T18:22:14.248454+0000","last_clean":"2023-12-19T18:22:14.248454+0000","last_became_active":"2023-12-19T18:22:14.247511+0000","last_became_peered":"2023-12-19T18:22:14.247511+0000","last_unstale":"2023-12-19T18:22:14.248454+0000","last_undegraded":"2023-12-19T18:22:14.248454+0000","last_fullsized":"2023-12-19T18:22:14.248454+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:57:01.585693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.257231+0000","last_change":"2023-12-19T18:22:14.257151+0000","last_active":"2023-12-19T18:22:14.257231+0000","last_peered":"2023-12-19T18:22:14.257231+0000","last_clean":"2023-12-19T18:22:14.257231+0000","last_became_active":"2023-12-19T18:22:14.256854+0000","last_became_peered":"2023-12-19T18:22:14.256854+0000","last_unstale":"2023-12-19T18:22:14.257231+0000","last_undegraded":"2023-12-19T18:22:14.257231+0000","last_fullsized":"2023-12-19T18:22:14.257231+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:01:59.548259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.238540+0000","last_change":"2023-12-19T18:22:14.236261+0000","last_active":"2023-12-19T18:22:14.238540+0000","last_peered":"2023-12-19T18:22:14.238540+0000","last_clean":"2023-12-19T18:22:14.238540+0000","last_became_active":"2023-12-19T18:22:14.234425+0000","last_became_peered":"2023-12-19T18:22:14.234425+0000","last_unstale":"2023-12-19T18:22:14.238540+0000","last_undegraded":"2023-12-19T18:22:14.238540+0000","last_fullsized":"2023-12-19T18:22:14.238540+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:32.485695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245871+0000","last_change":"2023-12-19T18:22:14.241276+0000","last_active":"2023-12-19T18:22:14.245871+0000","last_peered":"2023-12-19T18:22:14.245871+0000","last_clean":"2023-12-19T18:22:14.245871+0000","last_became_active":"2023-12-19T18:22:14.240498+0000","last_became_peered":"2023-12-19T18:22:14.240498+0000","last_unstale":"2023-12-19T18:22:14.245871+0000","last_undegraded":"2023-12-19T18:22:14.245871+0000","last_fullsized":"2023-12-19T18:22:14.245871+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:47:09.595830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252788+0000","last_change":"2023-12-19T18:22:14.252269+0000","last_active":"2023-12-19T18:22:14.252788+0000","last_peered":"2023-12-19T18:22:14.252788+0000","last_clean":"2023-12-19T18:22:14.252788+0000","last_became_active":"2023-12-19T18:22:14.251959+0000","last_became_peered":"2023-12-19T18:22:14.251959+0000","last_unstale":"2023-12-19T18:22:14.252788+0000","last_undegraded":"2023-12-19T18:22:14.252788+0000","last_fullsized":"2023-12-19T18:22:14.252788+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:01:36.203567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258305+0000","last_change":"2023-12-19T18:22:14.255516+0000","last_active":"2023-12-19T18:22:14.258305+0000","last_peered":"2023-12-19T18:22:14.258305+0000","last_clean":"2023-12-19T18:22:14.258305+0000","last_became_active":"2023-12-19T18:22:14.254491+0000","last_became_peered":"2023-12-19T18:22:14.254491+0000","last_unstale":"2023-12-19T18:22:14.258305+0000","last_undegraded":"2023-12-19T18:22:14.258305+0000","last_fullsized":"2023-12-19T18:22:14.258305+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:16.254938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.234589+0000","last_change":"2023-12-19T18:22:14.232017+0000","last_active":"2023-12-19T18:22:14.234589+0000","last_peered":"2023-12-19T18:22:14.234589+0000","last_clean":"2023-12-19T18:22:14.234589+0000","last_became_active":"2023-12-19T18:22:14.231247+0000","last_became_peered":"2023-12-19T18:22:14.231247+0000","last_unstale":"2023-12-19T18:22:14.234589+0000","last_undegraded":"2023-12-19T18:22:14.234589+0000","last_fullsized":"2023-12-19T18:22:14.234589+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:33.697121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"134'4","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.257068+0000","last_change":"2023-12-19T18:22:13.732827+0000","last_active":"2023-12-19T18:22:14.257068+0000","last_peered":"2023-12-19T18:22:14.257068+0000","last_clean":"2023-12-19T18:22:14.257068+0000","last_became_active":"2023-12-19T18:22:13.732603+0000","last_became_peered":"2023-12-19T18:22:13.732603+0000","last_unstale":"2023-12-19T18:22:14.257068+0000","last_undegraded":"2023-12-19T18:22:14.257068+0000","last_fullsized":"2023-12-19T18:22:14.257068+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:41:12.315002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245917+0000","last_change":"2023-12-19T18:22:14.243673+0000","last_active":"2023-12-19T18:22:14.245917+0000","last_peered":"2023-12-19T18:22:14.245917+0000","last_clean":"2023-12-19T18:22:14.245917+0000","last_became_active":"2023-12-19T18:22:14.243147+0000","last_became_peered":"2023-12-19T18:22:14.243147+0000","last_unstale":"2023-12-19T18:22:14.245917+0000","last_undegraded":"2023-12-19T18:22:14.245917+0000","last_fullsized":"2023-12-19T18:22:14.245917+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:39.384923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248064+0000","last_change":"2023-12-19T18:22:13.731624+0000","last_active":"2023-12-19T18:22:14.248064+0000","last_peered":"2023-12-19T18:22:14.248064+0000","last_clean":"2023-12-19T18:22:14.248064+0000","last_became_active":"2023-12-19T18:22:13.731335+0000","last_became_peered":"2023-12-19T18:22:13.731335+0000","last_unstale":"2023-12-19T18:22:14.248064+0000","last_undegraded":"2023-12-19T18:22:14.248064+0000","last_fullsized":"2023-12-19T18:22:14.248064+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:53:16.279722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258361+0000","last_change":"2023-12-19T18:22:14.257304+0000","last_active":"2023-12-19T18:22:14.258361+0000","last_peered":"2023-12-19T18:22:14.258361+0000","last_clean":"2023-12-19T18:22:14.258361+0000","last_became_active":"2023-12-19T18:22:14.256258+0000","last_became_peered":"2023-12-19T18:22:14.256258+0000","last_unstale":"2023-12-19T18:22:14.258361+0000","last_undegraded":"2023-12-19T18:22:14.258361+0000","last_fullsized":"2023-12-19T18:22:14.258361+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:16.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.262268+0000","last_change":"2023-12-19T18:22:13.733176+0000","last_active":"2023-12-19T18:22:14.262268+0000","last_peered":"2023-12-19T18:22:14.262268+0000","last_clean":"2023-12-19T18:22:14.262268+0000","last_became_active":"2023-12-19T18:22:13.732940+0000","last_became_peered":"2023-12-19T18:22:13.732940+0000","last_unstale":"2023-12-19T18:22:14.262268+0000","last_undegraded":"2023-12-19T18:22:14.262268+0000","last_fullsized":"2023-12-19T18:22:14.262268+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:30.091059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.244211+0000","last_change":"2023-12-19T18:22:14.243888+0000","last_active":"2023-12-19T18:22:14.244211+0000","last_peered":"2023-12-19T18:22:14.244211+0000","last_clean":"2023-12-19T18:22:14.244211+0000","last_became_active":"2023-12-19T18:22:14.243667+0000","last_became_peered":"2023-12-19T18:22:14.243667+0000","last_unstale":"2023-12-19T18:22:14.244211+0000","last_undegraded":"2023-12-19T18:22:14.244211+0000","last_fullsized":"2023-12-19T18:22:14.244211+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:39.384923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.257151+0000","last_change":"2023-12-19T18:22:13.733119+0000","last_active":"2023-12-19T18:22:14.257151+0000","last_peered":"2023-12-19T18:22:14.257151+0000","last_clean":"2023-12-19T18:22:14.257151+0000","last_became_active":"2023-12-19T18:22:13.732868+0000","last_became_peered":"2023-12-19T18:22:13.732868+0000","last_unstale":"2023-12-19T18:22:14.257151+0000","last_undegraded":"2023-12-19T18:22:14.257151+0000","last_fullsized":"2023-12-19T18:22:14.257151+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:07.352249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.235616+0000","last_change":"2023-12-19T18:22:14.235017+0000","last_active":"2023-12-19T18:22:14.235616+0000","last_peered":"2023-12-19T18:22:14.235616+0000","last_clean":"2023-12-19T18:22:14.235616+0000","last_became_active":"2023-12-19T18:22:14.234807+0000","last_became_peered":"2023-12-19T18:22:14.234807+0000","last_unstale":"2023-12-19T18:22:14.235616+0000","last_undegraded":"2023-12-19T18:22:14.235616+0000","last_fullsized":"2023-12-19T18:22:14.235616+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:53.745021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252491+0000","last_change":"2023-12-19T18:22:13.730941+0000","last_active":"2023-12-19T18:22:14.252491+0000","last_peered":"2023-12-19T18:22:14.252491+0000","last_clean":"2023-12-19T18:22:14.252491+0000","last_became_active":"2023-12-19T18:22:13.730684+0000","last_became_peered":"2023-12-19T18:22:13.730684+0000","last_unstale":"2023-12-19T18:22:14.252491+0000","last_undegraded":"2023-12-19T18:22:14.252491+0000","last_fullsized":"2023-12-19T18:22:14.252491+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:43:06.634088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239743+0000","last_change":"2023-12-19T18:22:14.239588+0000","last_active":"2023-12-19T18:22:14.239743+0000","last_peered":"2023-12-19T18:22:14.239743+0000","last_clean":"2023-12-19T18:22:14.239743+0000","last_became_active":"2023-12-19T18:22:14.238776+0000","last_became_peered":"2023-12-19T18:22:14.238776+0000","last_unstale":"2023-12-19T18:22:14.239743+0000","last_undegraded":"2023-12-19T18:22:14.239743+0000","last_fullsized":"2023-12-19T18:22:14.239743+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:41.584580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259067+0000","last_change":"2023-12-19T18:22:13.731736+0000","last_active":"2023-12-19T18:22:14.259067+0000","last_peered":"2023-12-19T18:22:14.259067+0000","last_clean":"2023-12-19T18:22:14.259067+0000","last_became_active":"2023-12-19T18:22:13.731153+0000","last_became_peered":"2023-12-19T18:22:13.731153+0000","last_unstale":"2023-12-19T18:22:14.259067+0000","last_undegraded":"2023-12-19T18:22:14.259067+0000","last_fullsized":"2023-12-19T18:22:14.259067+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:52.730485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.254538+0000","last_change":"2023-12-19T18:22:14.254248+0000","last_active":"2023-12-19T18:22:14.254538+0000","last_peered":"2023-12-19T18:22:14.254538+0000","last_clean":"2023-12-19T18:22:14.254538+0000","last_became_active":"2023-12-19T18:22:14.254005+0000","last_became_peered":"2023-12-19T18:22:14.254005+0000","last_unstale":"2023-12-19T18:22:14.254538+0000","last_undegraded":"2023-12-19T18:22:14.254538+0000","last_fullsized":"2023-12-19T18:22:14.254538+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:11:57.860995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.235224+0000","last_change":"2023-12-19T18:22:13.730291+0000","last_active":"2023-12-19T18:22:14.235224+0000","last_peered":"2023-12-19T18:22:14.235224+0000","last_clean":"2023-12-19T18:22:14.235224+0000","last_became_active":"2023-12-19T18:22:13.730015+0000","last_became_peered":"2023-12-19T18:22:13.730015+0000","last_unstale":"2023-12-19T18:22:14.235224+0000","last_undegraded":"2023-12-19T18:22:14.235224+0000","last_fullsized":"2023-12-19T18:22:14.235224+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:02:54.040583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.241518+0000","last_change":"2023-12-19T18:22:14.241428+0000","last_active":"2023-12-19T18:22:14.241518+0000","last_peered":"2023-12-19T18:22:14.241518+0000","last_clean":"2023-12-19T18:22:14.241518+0000","last_became_active":"2023-12-19T18:22:14.241208+0000","last_became_peered":"2023-12-19T18:22:14.241208+0000","last_unstale":"2023-12-19T18:22:14.241518+0000","last_undegraded":"2023-12-19T18:22:14.241518+0000","last_fullsized":"2023-12-19T18:22:14.241518+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:45.553627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.224445+0000","last_change":"2023-12-19T18:22:13.730656+0000","last_active":"2023-12-19T18:22:14.224445+0000","last_peered":"2023-12-19T18:22:14.224445+0000","last_clean":"2023-12-19T18:22:14.224445+0000","last_became_active":"2023-12-19T18:22:13.730408+0000","last_became_peered":"2023-12-19T18:22:13.730408+0000","last_unstale":"2023-12-19T18:22:14.224445+0000","last_undegraded":"2023-12-19T18:22:14.224445+0000","last_fullsized":"2023-12-19T18:22:14.224445+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:33.562949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258327+0000","last_change":"2023-12-19T18:22:14.255646+0000","last_active":"2023-12-19T18:22:14.258327+0000","last_peered":"2023-12-19T18:22:14.258327+0000","last_clean":"2023-12-19T18:22:14.258327+0000","last_became_active":"2023-12-19T18:22:14.254964+0000","last_became_peered":"2023-12-19T18:22:14.254964+0000","last_unstale":"2023-12-19T18:22:14.258327+0000","last_undegraded":"2023-12-19T18:22:14.258327+0000","last_fullsized":"2023-12-19T18:22:14.258327+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:13:15.250452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245923+0000","last_change":"2023-12-19T18:22:13.724536+0000","last_active":"2023-12-19T18:22:14.245923+0000","last_peered":"2023-12-19T18:22:14.245923+0000","last_clean":"2023-12-19T18:22:14.245923+0000","last_became_active":"2023-12-19T18:22:13.724110+0000","last_became_peered":"2023-12-19T18:22:13.724110+0000","last_unstale":"2023-12-19T18:22:14.245923+0000","last_undegraded":"2023-12-19T18:22:14.245923+0000","last_fullsized":"2023-12-19T18:22:14.245923+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:44.539092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245061+0000","last_change":"2023-12-19T18:22:14.244657+0000","last_active":"2023-12-19T18:22:14.245061+0000","last_peered":"2023-12-19T18:22:14.245061+0000","last_clean":"2023-12-19T18:22:14.245061+0000","last_became_active":"2023-12-19T18:22:14.244264+0000","last_became_peered":"2023-12-19T18:22:14.244264+0000","last_unstale":"2023-12-19T18:22:14.245061+0000","last_undegraded":"2023-12-19T18:22:14.245061+0000","last_fullsized":"2023-12-19T18:22:14.245061+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:15.726948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248854+0000","last_change":"2023-12-19T18:22:13.733261+0000","last_active":"2023-12-19T18:22:14.248854+0000","last_peered":"2023-12-19T18:22:14.248854+0000","last_clean":"2023-12-19T18:22:14.248854+0000","last_became_active":"2023-12-19T18:22:13.733015+0000","last_became_peered":"2023-12-19T18:22:13.733015+0000","last_unstale":"2023-12-19T18:22:14.248854+0000","last_undegraded":"2023-12-19T18:22:14.248854+0000","last_fullsized":"2023-12-19T18:22:14.248854+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:52.730485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259593+0000","last_change":"2023-12-19T18:22:14.259137+0000","last_active":"2023-12-19T18:22:14.259593+0000","last_peered":"2023-12-19T18:22:14.259593+0000","last_clean":"2023-12-19T18:22:14.259593+0000","last_became_active":"2023-12-19T18:22:14.258878+0000","last_became_peered":"2023-12-19T18:22:14.258878+0000","last_unstale":"2023-12-19T18:22:14.259593+0000","last_undegraded":"2023-12-19T18:22:14.259593+0000","last_fullsized":"2023-12-19T18:22:14.259593+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:32:56.103686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"134'3","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.247294+0000","last_change":"2023-12-19T18:22:13.727917+0000","last_active":"2023-12-19T18:22:14.247294+0000","last_peered":"2023-12-19T18:22:14.247294+0000","last_clean":"2023-12-19T18:22:14.247294+0000","last_became_active":"2023-12-19T18:22:13.727633+0000","last_became_peered":"2023-12-19T18:22:13.727633+0000","last_unstale":"2023-12-19T18:22:14.247294+0000","last_undegraded":"2023-12-19T18:22:14.247294+0000","last_fullsized":"2023-12-19T18:22:14.247294+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:36:19.046380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.247094+0000","last_change":"2023-12-19T18:22:14.246791+0000","last_active":"2023-12-19T18:22:14.247094+0000","last_peered":"2023-12-19T18:22:14.247094+0000","last_clean":"2023-12-19T18:22:14.247094+0000","last_became_active":"2023-12-19T18:22:14.246271+0000","last_became_peered":"2023-12-19T18:22:14.246271+0000","last_unstale":"2023-12-19T18:22:14.247094+0000","last_undegraded":"2023-12-19T18:22:14.247094+0000","last_fullsized":"2023-12-19T18:22:14.247094+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:40.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"134'6","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259355+0000","last_change":"2023-12-19T18:22:13.725517+0000","last_active":"2023-12-19T18:22:14.259355+0000","last_peered":"2023-12-19T18:22:14.259355+0000","last_clean":"2023-12-19T18:22:14.259355+0000","last_became_active":"2023-12-19T18:22:13.724094+0000","last_became_peered":"2023-12-19T18:22:13.724094+0000","last_unstale":"2023-12-19T18:22:14.259355+0000","last_undegraded":"2023-12-19T18:22:14.259355+0000","last_fullsized":"2023-12-19T18:22:14.259355+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:29:20.466956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252592+0000","last_change":"2023-12-19T18:22:14.250414+0000","last_active":"2023-12-19T18:22:14.252592+0000","last_peered":"2023-12-19T18:22:14.252592+0000","last_clean":"2023-12-19T18:22:14.252592+0000","last_became_active":"2023-12-19T18:22:14.249822+0000","last_became_peered":"2023-12-19T18:22:14.249822+0000","last_unstale":"2023-12-19T18:22:14.252592+0000","last_undegraded":"2023-12-19T18:22:14.252592+0000","last_fullsized":"2023-12-19T18:22:14.252592+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:37.118051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248641+0000","last_change":"2023-12-19T18:22:13.730064+0000","last_active":"2023-12-19T18:22:14.248641+0000","last_peered":"2023-12-19T18:22:14.248641+0000","last_clean":"2023-12-19T18:22:14.248641+0000","last_became_active":"2023-12-19T18:22:13.729780+0000","last_became_peered":"2023-12-19T18:22:13.729780+0000","last_unstale":"2023-12-19T18:22:14.248641+0000","last_undegraded":"2023-12-19T18:22:14.248641+0000","last_fullsized":"2023-12-19T18:22:14.248641+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:14.712413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.254201+0000","last_change":"2023-12-19T18:22:14.253925+0000","last_active":"2023-12-19T18:22:14.254201+0000","last_peered":"2023-12-19T18:22:14.254201+0000","last_clean":"2023-12-19T18:22:14.254201+0000","last_became_active":"2023-12-19T18:22:14.253159+0000","last_became_peered":"2023-12-19T18:22:14.253159+0000","last_unstale":"2023-12-19T18:22:14.254201+0000","last_undegraded":"2023-12-19T18:22:14.254201+0000","last_fullsized":"2023-12-19T18:22:14.254201+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:16.254938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246679+0000","last_change":"2023-12-19T18:22:13.732632+0000","last_active":"2023-12-19T18:22:14.246679+0000","last_peered":"2023-12-19T18:22:14.246679+0000","last_clean":"2023-12-19T18:22:14.246679+0000","last_became_active":"2023-12-19T18:22:13.732365+0000","last_became_peered":"2023-12-19T18:22:13.732365+0000","last_unstale":"2023-12-19T18:22:14.246679+0000","last_undegraded":"2023-12-19T18:22:14.246679+0000","last_fullsized":"2023-12-19T18:22:14.246679+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.235724+0000","last_change":"2023-12-19T18:22:14.235442+0000","last_active":"2023-12-19T18:22:14.235724+0000","last_peered":"2023-12-19T18:22:14.235724+0000","last_clean":"2023-12-19T18:22:14.235724+0000","last_became_active":"2023-12-19T18:22:14.235055+0000","last_became_peered":"2023-12-19T18:22:14.235055+0000","last_unstale":"2023-12-19T18:22:14.235724+0000","last_undegraded":"2023-12-19T18:22:14.235724+0000","last_fullsized":"2023-12-19T18:22:14.235724+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:06:47.768050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.229590+0000","last_change":"2023-12-19T18:22:13.729287+0000","last_active":"2023-12-19T18:22:14.229590+0000","last_peered":"2023-12-19T18:22:14.229590+0000","last_clean":"2023-12-19T18:22:14.229590+0000","last_became_active":"2023-12-19T18:22:13.725118+0000","last_became_peered":"2023-12-19T18:22:13.725118+0000","last_unstale":"2023-12-19T18:22:14.229590+0000","last_undegraded":"2023-12-19T18:22:14.229590+0000","last_fullsized":"2023-12-19T18:22:14.229590+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:13:44.939138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.255529+0000","last_change":"2023-12-19T18:22:14.255009+0000","last_active":"2023-12-19T18:22:14.255529+0000","last_peered":"2023-12-19T18:22:14.255529+0000","last_clean":"2023-12-19T18:22:14.255529+0000","last_became_active":"2023-12-19T18:22:14.254396+0000","last_became_peered":"2023-12-19T18:22:14.254396+0000","last_unstale":"2023-12-19T18:22:14.255529+0000","last_undegraded":"2023-12-19T18:22:14.255529+0000","last_fullsized":"2023-12-19T18:22:14.255529+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:08.465323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"134'8","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.263107+0000","last_change":"2023-12-19T18:22:13.734908+0000","last_active":"2023-12-19T18:22:14.263107+0000","last_peered":"2023-12-19T18:22:14.263107+0000","last_clean":"2023-12-19T18:22:14.263107+0000","last_became_active":"2023-12-19T18:22:13.734666+0000","last_became_peered":"2023-12-19T18:22:13.734666+0000","last_unstale":"2023-12-19T18:22:14.263107+0000","last_undegraded":"2023-12-19T18:22:14.263107+0000","last_fullsized":"2023-12-19T18:22:14.263107+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:44.539092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.243196+0000","last_change":"2023-12-19T18:22:14.243113+0000","last_active":"2023-12-19T18:22:14.243196+0000","last_peered":"2023-12-19T18:22:14.243196+0000","last_clean":"2023-12-19T18:22:14.243196+0000","last_became_active":"2023-12-19T18:22:14.242834+0000","last_became_peered":"2023-12-19T18:22:14.242834+0000","last_unstale":"2023-12-19T18:22:14.243196+0000","last_undegraded":"2023-12-19T18:22:14.243196+0000","last_fullsized":"2023-12-19T18:22:14.243196+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:12.207051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.243309+0000","last_change":"2023-12-19T18:22:13.724787+0000","last_active":"2023-12-19T18:22:14.243309+0000","last_peered":"2023-12-19T18:22:14.243309+0000","last_clean":"2023-12-19T18:22:14.243309+0000","last_became_active":"2023-12-19T18:22:13.724547+0000","last_became_peered":"2023-12-19T18:22:13.724547+0000","last_unstale":"2023-12-19T18:22:14.243309+0000","last_undegraded":"2023-12-19T18:22:14.243309+0000","last_fullsized":"2023-12-19T18:22:14.243309+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:11.095769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.243792+0000","last_change":"2023-12-19T18:22:14.243719+0000","last_active":"2023-12-19T18:22:14.243792+0000","last_peered":"2023-12-19T18:22:14.243792+0000","last_clean":"2023-12-19T18:22:14.243792+0000","last_became_active":"2023-12-19T18:22:14.242848+0000","last_became_peered":"2023-12-19T18:22:14.242848+0000","last_unstale":"2023-12-19T18:22:14.243792+0000","last_undegraded":"2023-12-19T18:22:14.243792+0000","last_fullsized":"2023-12-19T18:22:14.243792+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:34.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246567+0000","last_change":"2023-12-19T18:22:13.728901+0000","last_active":"2023-12-19T18:22:14.246567+0000","last_peered":"2023-12-19T18:22:14.246567+0000","last_clean":"2023-12-19T18:22:14.246567+0000","last_became_active":"2023-12-19T18:22:13.728001+0000","last_became_peered":"2023-12-19T18:22:13.728001+0000","last_unstale":"2023-12-19T18:22:14.246567+0000","last_undegraded":"2023-12-19T18:22:14.246567+0000","last_fullsized":"2023-12-19T18:22:14.246567+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:59.426392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246154+0000","last_change":"2023-12-19T18:22:14.246103+0000","last_active":"2023-12-19T18:22:14.246154+0000","last_peered":"2023-12-19T18:22:14.246154+0000","last_clean":"2023-12-19T18:22:14.246154+0000","last_became_active":"2023-12-19T18:22:14.245847+0000","last_became_peered":"2023-12-19T18:22:14.245847+0000","last_unstale":"2023-12-19T18:22:14.246154+0000","last_undegraded":"2023-12-19T18:22:14.246154+0000","last_fullsized":"2023-12-19T18:22:14.246154+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:01.033125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.226401+0000","last_change":"2023-12-19T18:22:13.729021+0000","last_active":"2023-12-19T18:22:14.226401+0000","last_peered":"2023-12-19T18:22:14.226401+0000","last_clean":"2023-12-19T18:22:14.226401+0000","last_became_active":"2023-12-19T18:22:13.728792+0000","last_became_peered":"2023-12-19T18:22:13.728792+0000","last_unstale":"2023-12-19T18:22:14.226401+0000","last_undegraded":"2023-12-19T18:22:14.226401+0000","last_fullsized":"2023-12-19T18:22:14.226401+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:39.606223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252679+0000","last_change":"2023-12-19T18:22:14.252307+0000","last_active":"2023-12-19T18:22:14.252679+0000","last_peered":"2023-12-19T18:22:14.252679+0000","last_clean":"2023-12-19T18:22:14.252679+0000","last_became_active":"2023-12-19T18:22:14.251969+0000","last_became_peered":"2023-12-19T18:22:14.251969+0000","last_unstale":"2023-12-19T18:22:14.252679+0000","last_undegraded":"2023-12-19T18:22:14.252679+0000","last_fullsized":"2023-12-19T18:22:14.252679+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:50:35.183653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.263780+0000","last_change":"2023-12-19T18:22:13.734402+0000","last_active":"2023-12-19T18:22:14.263780+0000","last_peered":"2023-12-19T18:22:14.263780+0000","last_clean":"2023-12-19T18:22:14.263780+0000","last_became_active":"2023-12-19T18:22:13.733935+0000","last_became_peered":"2023-12-19T18:22:13.733935+0000","last_unstale":"2023-12-19T18:22:14.263780+0000","last_undegraded":"2023-12-19T18:22:14.263780+0000","last_fullsized":"2023-12-19T18:22:14.263780+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:30.091059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258421+0000","last_change":"2023-12-19T18:22:14.257490+0000","last_active":"2023-12-19T18:22:14.258421+0000","last_peered":"2023-12-19T18:22:14.258421+0000","last_clean":"2023-12-19T18:22:14.258421+0000","last_became_active":"2023-12-19T18:22:14.256714+0000","last_became_peered":"2023-12-19T18:22:14.256714+0000","last_unstale":"2023-12-19T18:22:14.258421+0000","last_undegraded":"2023-12-19T18:22:14.258421+0000","last_fullsized":"2023-12-19T18:22:14.258421+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:29.507646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.238218+0000","last_change":"2023-12-19T18:22:13.730665+0000","last_active":"2023-12-19T18:22:14.238218+0000","last_peered":"2023-12-19T18:22:14.238218+0000","last_clean":"2023-12-19T18:22:14.238218+0000","last_became_active":"2023-12-19T18:22:13.729630+0000","last_became_peered":"2023-12-19T18:22:13.729630+0000","last_unstale":"2023-12-19T18:22:14.238218+0000","last_undegraded":"2023-12-19T18:22:14.238218+0000","last_fullsized":"2023-12-19T18:22:14.238218+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:01:35.189032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258658+0000","last_change":"2023-12-19T18:22:14.257612+0000","last_active":"2023-12-19T18:22:14.258658+0000","last_peered":"2023-12-19T18:22:14.258658+0000","last_clean":"2023-12-19T18:22:14.258658+0000","last_became_active":"2023-12-19T18:22:14.256820+0000","last_became_peered":"2023-12-19T18:22:14.256820+0000","last_unstale":"2023-12-19T18:22:14.258658+0000","last_undegraded":"2023-12-19T18:22:14.258658+0000","last_fullsized":"2023-12-19T18:22:14.258658+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:12.207051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.231579+0000","last_change":"2023-12-19T18:22:13.731257+0000","last_active":"2023-12-19T18:22:14.231579+0000","last_peered":"2023-12-19T18:22:14.231579+0000","last_clean":"2023-12-19T18:22:14.231579+0000","last_became_active":"2023-12-19T18:22:13.730910+0000","last_became_peered":"2023-12-19T18:22:13.730910+0000","last_unstale":"2023-12-19T18:22:14.231579+0000","last_undegraded":"2023-12-19T18:22:14.231579+0000","last_fullsized":"2023-12-19T18:22:14.231579+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.254075+0000","last_change":"2023-12-19T18:22:14.253991+0000","last_active":"2023-12-19T18:22:14.254075+0000","last_peered":"2023-12-19T18:22:14.254075+0000","last_clean":"2023-12-19T18:22:14.254075+0000","last_became_active":"2023-12-19T18:22:14.253757+0000","last_became_peered":"2023-12-19T18:22:14.253757+0000","last_unstale":"2023-12-19T18:22:14.254075+0000","last_undegraded":"2023-12-19T18:22:14.254075+0000","last_fullsized":"2023-12-19T18:22:14.254075+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:44.388621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246101+0000","last_change":"2023-12-19T18:22:13.714716+0000","last_active":"2023-12-19T18:22:14.246101+0000","last_peered":"2023-12-19T18:22:14.246101+0000","last_clean":"2023-12-19T18:22:14.246101+0000","last_became_active":"2023-12-19T18:22:13.713977+0000","last_became_peered":"2023-12-19T18:22:13.713977+0000","last_unstale":"2023-12-19T18:22:14.246101+0000","last_undegraded":"2023-12-19T18:22:14.246101+0000","last_fullsized":"2023-12-19T18:22:14.246101+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246372+0000","last_change":"2023-12-19T18:22:14.245059+0000","last_active":"2023-12-19T18:22:14.246372+0000","last_peered":"2023-12-19T18:22:14.246372+0000","last_clean":"2023-12-19T18:22:14.246372+0000","last_became_active":"2023-12-19T18:22:14.244526+0000","last_became_peered":"2023-12-19T18:22:14.244526+0000","last_unstale":"2023-12-19T18:22:14.246372+0000","last_undegraded":"2023-12-19T18:22:14.246372+0000","last_fullsized":"2023-12-19T18:22:14.246372+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:45:23.784185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.241662+0000","last_change":"2023-12-19T18:22:13.729500+0000","last_active":"2023-12-19T18:22:14.241662+0000","last_peered":"2023-12-19T18:22:14.241662+0000","last_clean":"2023-12-19T18:22:14.241662+0000","last_became_active":"2023-12-19T18:22:13.725715+0000","last_became_peered":"2023-12-19T18:22:13.725715+0000","last_unstale":"2023-12-19T18:22:14.241662+0000","last_undegraded":"2023-12-19T18:22:14.241662+0000","last_fullsized":"2023-12-19T18:22:14.241662+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:30:23.979686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259484+0000","last_change":"2023-12-19T18:22:14.258603+0000","last_active":"2023-12-19T18:22:14.259484+0000","last_peered":"2023-12-19T18:22:14.259484+0000","last_clean":"2023-12-19T18:22:14.259484+0000","last_became_active":"2023-12-19T18:22:14.258065+0000","last_became_peered":"2023-12-19T18:22:14.258065+0000","last_unstale":"2023-12-19T18:22:14.259484+0000","last_undegraded":"2023-12-19T18:22:14.259484+0000","last_fullsized":"2023-12-19T18:22:14.259484+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:43:39.731660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.225364+0000","last_change":"2023-12-19T18:22:13.726768+0000","last_active":"2023-12-19T18:22:14.225364+0000","last_peered":"2023-12-19T18:22:14.225364+0000","last_clean":"2023-12-19T18:22:14.225364+0000","last_became_active":"2023-12-19T18:22:13.722319+0000","last_became_peered":"2023-12-19T18:22:13.722319+0000","last_unstale":"2023-12-19T18:22:14.225364+0000","last_undegraded":"2023-12-19T18:22:14.225364+0000","last_fullsized":"2023-12-19T18:22:14.225364+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:11.192516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.255546+0000","last_change":"2023-12-19T18:22:14.255042+0000","last_active":"2023-12-19T18:22:14.255546+0000","last_peered":"2023-12-19T18:22:14.255546+0000","last_clean":"2023-12-19T18:22:14.255546+0000","last_became_active":"2023-12-19T18:22:14.253623+0000","last_became_peered":"2023-12-19T18:22:14.253623+0000","last_unstale":"2023-12-19T18:22:14.255546+0000","last_undegraded":"2023-12-19T18:22:14.255546+0000","last_fullsized":"2023-12-19T18:22:14.255546+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:09.942336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.227165+0000","last_change":"2023-12-19T18:22:13.713351+0000","last_active":"2023-12-19T18:22:14.227165+0000","last_peered":"2023-12-19T18:22:14.227165+0000","last_clean":"2023-12-19T18:22:14.227165+0000","last_became_active":"2023-12-19T18:22:13.712581+0000","last_became_peered":"2023-12-19T18:22:13.712581+0000","last_unstale":"2023-12-19T18:22:14.227165+0000","last_undegraded":"2023-12-19T18:22:14.227165+0000","last_fullsized":"2023-12-19T18:22:14.227165+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:36.103516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.242422+0000","last_change":"2023-12-19T18:22:14.241466+0000","last_active":"2023-12-19T18:22:14.242422+0000","last_peered":"2023-12-19T18:22:14.242422+0000","last_clean":"2023-12-19T18:22:14.242422+0000","last_became_active":"2023-12-19T18:22:14.241205+0000","last_became_peered":"2023-12-19T18:22:14.241205+0000","last_unstale":"2023-12-19T18:22:14.242422+0000","last_undegraded":"2023-12-19T18:22:14.242422+0000","last_fullsized":"2023-12-19T18:22:14.242422+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:08.710594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239506+0000","last_change":"2023-12-19T18:22:13.718488+0000","last_active":"2023-12-19T18:22:14.239506+0000","last_peered":"2023-12-19T18:22:14.239506+0000","last_clean":"2023-12-19T18:22:14.239506+0000","last_became_active":"2023-12-19T18:22:13.717351+0000","last_became_peered":"2023-12-19T18:22:13.717351+0000","last_unstale":"2023-12-19T18:22:14.239506+0000","last_undegraded":"2023-12-19T18:22:14.239506+0000","last_fullsized":"2023-12-19T18:22:14.239506+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:49:11.599052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248088+0000","last_change":"2023-12-19T18:22:14.247933+0000","last_active":"2023-12-19T18:22:14.248088+0000","last_peered":"2023-12-19T18:22:14.248088+0000","last_clean":"2023-12-19T18:22:14.248088+0000","last_became_active":"2023-12-19T18:22:14.247461+0000","last_became_peered":"2023-12-19T18:22:14.247461+0000","last_unstale":"2023-12-19T18:22:14.248088+0000","last_undegraded":"2023-12-19T18:22:14.248088+0000","last_fullsized":"2023-12-19T18:22:14.248088+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:24.011514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246268+0000","last_change":"2023-12-19T18:22:13.733542+0000","last_active":"2023-12-19T18:22:14.246268+0000","last_peered":"2023-12-19T18:22:14.246268+0000","last_clean":"2023-12-19T18:22:14.246268+0000","last_became_active":"2023-12-19T18:22:13.733273+0000","last_became_peered":"2023-12-19T18:22:13.733273+0000","last_unstale":"2023-12-19T18:22:14.246268+0000","last_undegraded":"2023-12-19T18:22:14.246268+0000","last_fullsized":"2023-12-19T18:22:14.246268+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:07.352249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252415+0000","last_change":"2023-12-19T18:22:14.252315+0000","last_active":"2023-12-19T18:22:14.252415+0000","last_peered":"2023-12-19T18:22:14.252415+0000","last_clean":"2023-12-19T18:22:14.252415+0000","last_became_active":"2023-12-19T18:22:14.252108+0000","last_became_peered":"2023-12-19T18:22:14.252108+0000","last_unstale":"2023-12-19T18:22:14.252415+0000","last_undegraded":"2023-12-19T18:22:14.252415+0000","last_fullsized":"2023-12-19T18:22:14.252415+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:08.470720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239128+0000","last_change":"2023-12-19T18:22:13.723012+0000","last_active":"2023-12-19T18:22:14.239128+0000","last_peered":"2023-12-19T18:22:14.239128+0000","last_clean":"2023-12-19T18:22:14.239128+0000","last_became_active":"2023-12-19T18:22:13.717308+0000","last_became_peered":"2023-12-19T18:22:13.717308+0000","last_unstale":"2023-12-19T18:22:14.239128+0000","last_undegraded":"2023-12-19T18:22:14.239128+0000","last_fullsized":"2023-12-19T18:22:14.239128+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:01.277914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246337+0000","last_change":"2023-12-19T18:22:14.244959+0000","last_active":"2023-12-19T18:22:14.246337+0000","last_peered":"2023-12-19T18:22:14.246337+0000","last_clean":"2023-12-19T18:22:14.246337+0000","last_became_active":"2023-12-19T18:22:14.238592+0000","last_became_peered":"2023-12-19T18:22:14.238592+0000","last_unstale":"2023-12-19T18:22:14.246337+0000","last_undegraded":"2023-12-19T18:22:14.246337+0000","last_fullsized":"2023-12-19T18:22:14.246337+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:30:01.499864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.232648+0000","last_change":"2023-12-19T18:22:13.736355+0000","last_active":"2023-12-19T18:22:14.232648+0000","last_peered":"2023-12-19T18:22:14.232648+0000","last_clean":"2023-12-19T18:22:14.232648+0000","last_became_active":"2023-12-19T18:22:13.736056+0000","last_became_peered":"2023-12-19T18:22:13.736056+0000","last_unstale":"2023-12-19T18:22:14.232648+0000","last_undegraded":"2023-12-19T18:22:14.232648+0000","last_fullsized":"2023-12-19T18:22:14.232648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:47.766382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.255816+0000","last_change":"2023-12-19T18:22:14.255696+0000","last_active":"2023-12-19T18:22:14.255816+0000","last_peered":"2023-12-19T18:22:14.255816+0000","last_clean":"2023-12-19T18:22:14.255816+0000","last_became_active":"2023-12-19T18:22:14.255205+0000","last_became_peered":"2023-12-19T18:22:14.255205+0000","last_unstale":"2023-12-19T18:22:14.255816+0000","last_undegraded":"2023-12-19T18:22:14.255816+0000","last_fullsized":"2023-12-19T18:22:14.255816+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:08.366785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248750+0000","last_change":"2023-12-19T18:22:13.715051+0000","last_active":"2023-12-19T18:22:14.248750+0000","last_peered":"2023-12-19T18:22:14.248750+0000","last_clean":"2023-12-19T18:22:14.248750+0000","last_became_active":"2023-12-19T18:22:13.714392+0000","last_became_peered":"2023-12-19T18:22:13.714392+0000","last_unstale":"2023-12-19T18:22:14.248750+0000","last_undegraded":"2023-12-19T18:22:14.248750+0000","last_fullsized":"2023-12-19T18:22:14.248750+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:00.018590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259379+0000","last_change":"2023-12-19T18:22:14.259186+0000","last_active":"2023-12-19T18:22:14.259379+0000","last_peered":"2023-12-19T18:22:14.259379+0000","last_clean":"2023-12-19T18:22:14.259379+0000","last_became_active":"2023-12-19T18:22:14.258862+0000","last_became_peered":"2023-12-19T18:22:14.258862+0000","last_unstale":"2023-12-19T18:22:14.259379+0000","last_undegraded":"2023-12-19T18:22:14.259379+0000","last_fullsized":"2023-12-19T18:22:14.259379+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:16.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.247514+0000","last_change":"2023-12-19T18:22:13.725717+0000","last_active":"2023-12-19T18:22:14.247514+0000","last_peered":"2023-12-19T18:22:14.247514+0000","last_clean":"2023-12-19T18:22:14.247514+0000","last_became_active":"2023-12-19T18:22:13.724435+0000","last_became_peered":"2023-12-19T18:22:13.724435+0000","last_unstale":"2023-12-19T18:22:14.247514+0000","last_undegraded":"2023-12-19T18:22:14.247514+0000","last_fullsized":"2023-12-19T18:22:14.247514+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:00:05.094962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":267,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.229283+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:22:14.229283+0000","last_peered":"2023-12-19T18:22:14.229283+0000","last_clean":"2023-12-19T18:22:14.229283+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:22:14.229283+0000","last_undegraded":"2023-12-19T18:22:14.229283+0000","last_fullsized":"2023-12-19T18:22:14.229283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574941,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":736,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199245902,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2310000000000001,"15min":1.1519999999999999},"min":{"1min":0.57899999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9630000000000001,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.272,"5min":1.278,"15min":1.1890000000000001},"min":{"1min":0.76900000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6990000000000001,"5min":1.79,"15min":1.79},"last":1.2410000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.3480000000000001,"15min":1.23},"min":{"1min":0.752,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.7010000000000001,"5min":10.926,"15min":10.926},"last":1.081},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.365,"15min":1.244},"min":{"1min":0.80900000000000005,"5min":0.71899999999999997,"15min":0.67900000000000005},"max":{"1min":1.6240000000000001,"5min":10.904999999999999,"15min":10.904999999999999},"last":0.98999999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.161,"15min":1.1040000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.427,"5min":3.1960000000000002,"15min":3.1960000000000002},"last":1.5620000000000001},{"interface":"front","average":{"1min":1.24,"5min":1.266,"15min":1.1930000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0670000000000002,"5min":6.008,"15min":6.008},"last":0.877}]},{"osd":3,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.28,"15min":1.1619999999999999},"min":{"1min":0.66900000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.595,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":1.659},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4179999999999999,"15min":1.294},"min":{"1min":0.85699999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.157,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":1.3759999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.258,"15min":1.123},"min":{"1min":0.88600000000000001,"5min":0.65400000000000003,"15min":0.47899999999999998},"max":{"1min":1.8009999999999999,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":1.331},{"interface":"front","average":{"1min":1.222,"5min":1.304,"15min":1.2410000000000001},"min":{"1min":0.94599999999999995,"5min":0.72999999999999998,"15min":0.57599999999999996},"max":{"1min":1.6759999999999999,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.736}]},{"osd":5,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.399,"15min":1.333},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.55000000000000004},"max":{"1min":1.8620000000000001,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":1.1890000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.4339999999999999,"15min":1.337},"min":{"1min":0.80700000000000005,"5min":0.71599999999999997,"15min":0.53500000000000003},"max":{"1min":2.1179999999999999,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":1.1439999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.5669999999999999,"15min":1.4690000000000001},"min":{"1min":1.036,"5min":0.78600000000000003,"15min":0.59899999999999998},"max":{"1min":2.423,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.407},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.49,"15min":1.3799999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.63200000000000001},"max":{"1min":2.0179999999999998,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":1.8169999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574941,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":735,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199246484,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.121,"15min":1.012},"min":{"1min":0.79800000000000004,"5min":0.66700000000000004,"15min":0.65800000000000003},"max":{"1min":1.6259999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":1.163,"5min":1.1870000000000001,"15min":1.075},"min":{"1min":0.78400000000000003,"5min":0.64300000000000002,"15min":0.55600000000000005},"max":{"1min":1.663,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":1}]},{"osd":1,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.1699999999999999,"15min":1.1319999999999999},"min":{"1min":0.86599999999999999,"5min":0.71399999999999997,"15min":0.68200000000000005},"max":{"1min":1.5609999999999999,"5min":4.5540000000000003,"15min":4.5540000000000003},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.214,"5min":1.226,"15min":1.1699999999999999},"min":{"1min":0.86599999999999999,"5min":0.625,"15min":0.60299999999999998},"max":{"1min":1.627,"5min":1.885,"15min":1.885},"last":1.331}]},{"osd":2,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.254,"15min":1.151},"min":{"1min":0.84799999999999998,"5min":0.626,"15min":0.626},"max":{"1min":1.583,"5min":1.821,"15min":1.821},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.218,"15min":1.2310000000000001},"min":{"1min":0.83599999999999997,"5min":0.81999999999999995,"15min":0.79900000000000004},"max":{"1min":1.821,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":1.1930000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.294,"15min":1.202},"min":{"1min":1.0129999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.722,"5min":3.8820000000000001,"15min":3.8820000000000001},"last":1.359},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2170000000000001,"15min":1.1699999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6559999999999999,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":1.133}]},{"osd":4,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.296,"15min":1.2450000000000001},"min":{"1min":0.74099999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6299999999999999,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.621},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.431,"15min":1.347},"min":{"1min":0.76700000000000002,"5min":0.70799999999999996,"15min":0.57999999999999996},"max":{"1min":1.774,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.448}]},{"osd":5,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2829999999999999,"15min":1.1879999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7330000000000001,"5min":4.6680000000000001,"15min":4.6680000000000001},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.4450000000000001,"15min":1.3560000000000001},"min":{"1min":0.84799999999999998,"5min":0.626,"15min":0.626},"max":{"1min":4.1159999999999997,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.4790000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4430000000000001,"15min":1.349},"min":{"1min":0.92700000000000005,"5min":0.82199999999999995,"15min":0.61899999999999999},"max":{"1min":1.7010000000000001,"5min":3.9670000000000001,"15min":3.9670000000000001},"last":1.274},{"interface":"front","average":{"1min":1.393,"5min":1.4470000000000001,"15min":1.357},"min":{"1min":0.81899999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8169999999999999,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":1.522}]}]},{"osd":5,"up_from":13,"seq":55834574941,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1313,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193998,"statfs":{"total":200000000,"available":198654496,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.23,"15min":1.1679999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.65200000000000002},"max":{"1min":1.6830000000000001,"5min":1.837,"15min":1.863},"last":0.81200000000000006},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2729999999999999,"15min":1.1870000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.571,"5min":1.879,"15min":1.879},"last":0.76600000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2090000000000001,"15min":1.0629999999999999},"min":{"1min":0.76200000000000001,"5min":0.70899999999999996,"15min":0.65700000000000003},"max":{"1min":1.782,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1599999999999999,"15min":1.0660000000000001},"min":{"1min":0.81899999999999995,"5min":0.71299999999999997,"15min":0.61299999999999999},"max":{"1min":1.5640000000000001,"5min":1.6359999999999999,"15min":1.843},"last":1.2509999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.155,"15min":1.0860000000000001},"min":{"1min":0.81799999999999995,"5min":0.74299999999999999,"15min":0.59499999999999997},"max":{"1min":1.3069999999999999,"5min":1.794,"15min":1.794},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.258,"15min":1.147},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.65200000000000002},"max":{"1min":1.5920000000000001,"5min":2.036,"15min":2.036},"last":0.93400000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2010000000000001,"15min":1.097},"min":{"1min":0.83699999999999997,"5min":0.71999999999999997,"15min":0.68899999999999995},"max":{"1min":1.72,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.101},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3320000000000001,"15min":1.23},"min":{"1min":0.91300000000000003,"5min":0.75700000000000001,"15min":0.70399999999999996},"max":{"1min":1.595,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.226}]},{"osd":4,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3069999999999999,"15min":1.179},"min":{"1min":0.95299999999999996,"5min":0.60899999999999999,"15min":0.46700000000000003},"max":{"1min":1.6479999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.0509999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.1990000000000001,"15min":1.131},"min":{"1min":0.90500000000000003,"5min":0.73999999999999999,"15min":0.57099999999999995},"max":{"1min":1.7689999999999999,"5min":1.804,"15min":1.901},"last":1.526}]},{"osd":6,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4299999999999999,"15min":1.3360000000000001},"min":{"1min":1.054,"5min":0.66600000000000004,"15min":0.54400000000000004},"max":{"1min":1.839,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.319,"15min":1.2470000000000001},"min":{"1min":1.163,"5min":0.81200000000000006,"15min":0.49299999999999999},"max":{"1min":1.702,"5min":1.7949999999999999,"15min":1.976},"last":1.3999999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.274,"15min":1.2290000000000001},"min":{"1min":1.0189999999999999,"5min":0.68300000000000005,"15min":0.50900000000000001},"max":{"1min":1.6779999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4570000000000001,"15min":1.4119999999999999},"min":{"1min":1.2310000000000001,"5min":0.84999999999999998,"15min":0.623},"max":{"1min":1.8120000000000001,"5min":2.117,"15min":2.117},"last":1.6120000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574941,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":736,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199245902,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98899999999999999,"15min":0.998},"min":{"1min":0.68500000000000005,"5min":0.62,"15min":0.62},"max":{"1min":1.3899999999999999,"5min":1.579,"15min":1.579},"last":1.45},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0409999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4259999999999999,"5min":1.7470000000000001,"15min":1.7909999999999999},"last":1.502}]},{"osd":1,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1679999999999999,"15min":1.149},"min":{"1min":0.67200000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.589,"5min":1.7130000000000001,"15min":1.8160000000000001},"last":1.3839999999999999},{"interface":"front","average":{"1min":1.29,"5min":1.2549999999999999,"15min":1.1850000000000001},"min":{"1min":0.77400000000000002,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.696},"last":1.2050000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1499999999999999,"15min":1.127},"min":{"1min":0.82399999999999995,"5min":0.749,"15min":0.69799999999999995},"max":{"1min":1.407,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.159},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.1060000000000001},"min":{"1min":0.872,"5min":0.72199999999999998,"15min":0.70499999999999996},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.2430000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.294,"15min":1.26},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.357},{"interface":"front","average":{"1min":1.369,"5min":1.3340000000000001,"15min":1.304},"min":{"1min":0.93100000000000005,"5min":0.82399999999999995,"15min":0.74399999999999999},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.5780000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.294,"15min":1.2030000000000001},"min":{"1min":0.90000000000000002,"5min":0.68600000000000005,"15min":0.44500000000000001},"max":{"1min":1.46,"5min":1.76,"15min":1.8740000000000001},"last":1.331},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.413,"15min":1.353},"min":{"1min":0.76500000000000001,"5min":0.625,"15min":0.504},"max":{"1min":1.827,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.93200000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3160000000000001,"15min":1.304},"min":{"1min":0.98399999999999999,"5min":0.71299999999999997,"15min":0.432},"max":{"1min":1.5509999999999999,"5min":1.7809999999999999,"15min":1.853},"last":1.296},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4630000000000001,"15min":1.3959999999999999},"min":{"1min":1.2170000000000001,"5min":0.73599999999999999,"15min":0.64500000000000002},"max":{"1min":1.7829999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.4650000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.3999999999999999,"15min":1.3480000000000001},"min":{"1min":1.0249999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.7589999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4379999999999999,"15min":1.4079999999999999},"min":{"1min":0.88500000000000001,"5min":0.60399999999999998,"15min":0.57399999999999995},"max":{"1min":1.8,"5min":1.905,"15min":1.9330000000000001},"last":1.407}]}]},{"osd":3,"up_from":13,"seq":55834574942,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":736,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199246290,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.52900000000000003,"5min":0.38100000000000001,"15min":0.35299999999999998},"max":{"1min":2.516,"5min":3.3140000000000001,"15min":3.4350000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1,"15min":0.94499999999999995},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.3490000000000002,"5min":4.3490000000000002,"15min":4.3490000000000002},"last":1.0209999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.0629999999999999,"15min":1.0620000000000001},"min":{"1min":0.45300000000000001,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":5.2510000000000003,"5min":5.2510000000000003,"15min":5.2510000000000003},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.403,"5min":1.1719999999999999,"15min":1.1160000000000001},"min":{"1min":0.63800000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":5.4119999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.2250000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.377,"15min":1.25},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.335,"5min":13.833,"15min":13.833},"last":1.417},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5349999999999999,"15min":1.306},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.53200000000000003},"max":{"1min":5.4480000000000004,"5min":16.748999999999999,"15min":16.748999999999999},"last":1.556}]},{"osd":4,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.3620000000000001,"15min":1.2330000000000001},"min":{"1min":0.81899999999999995,"5min":0.77400000000000002,"15min":0.65600000000000003},"max":{"1min":5.1790000000000003,"5min":5.1790000000000003,"15min":5.1790000000000003},"last":1.6419999999999999},{"interface":"front","average":{"1min":1.591,"5min":1.4239999999999999,"15min":1.409},"min":{"1min":0.873,"5min":0.80800000000000005,"15min":0.73199999999999998},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":1.1719999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.4750000000000001,"15min":1.45},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.76000000000000001},"max":{"1min":5.3289999999999997,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.367,"15min":1.2789999999999999},"min":{"1min":0.83799999999999997,"5min":0.80600000000000005,"15min":0.69999999999999996},"max":{"1min":4.6369999999999996,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.7470000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.4630000000000001,"15min":1.476},"min":{"1min":0.81999999999999995,"5min":0.81799999999999995,"15min":0.81599999999999995},"max":{"1min":4.5039999999999996,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":1.52},{"interface":"front","average":{"1min":1.667,"5min":1.548,"15min":1.514},"min":{"1min":0.83999999999999997,"5min":0.83199999999999996,"15min":0.80500000000000005},"max":{"1min":5.3739999999999997,"5min":5.3739999999999997,"15min":5.3739999999999997},"last":1.679}]},{"osd":7,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.476,"15min":1.385},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.73199999999999998},"max":{"1min":5.1349999999999998,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.843},{"interface":"front","average":{"1min":1.712,"5min":1.585,"15min":1.554},"min":{"1min":0.92200000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":4.4619999999999997,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":1.621}]}]},{"osd":2,"up_from":13,"seq":55834574941,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":735,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199246414,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.075,"15min":1.0109999999999999},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.34399999999999997},"max":{"1min":1.506,"5min":9.2550000000000008,"15min":9.2550000000000008},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0409999999999999,"15min":1.0389999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.7649999999999999,"5min":2.387,"15min":4.7850000000000001},"last":1.1679999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95999999999999996,"15min":1.165},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.46000000000000002},"max":{"1min":1.675,"5min":2.4390000000000001,"15min":8.2720000000000002},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.107,"15min":1.1619999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6519999999999999,"5min":2.7200000000000002,"15min":5.0620000000000003},"last":1.802}]},{"osd":3,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1830000000000001,"15min":1.3620000000000001},"min":{"1min":0.63300000000000001,"5min":0.54200000000000004,"15min":0.51900000000000002},"max":{"1min":1.708,"5min":2.238,"15min":8.3109999999999999},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.163,"15min":1.327},"min":{"1min":0.69599999999999995,"5min":0.46300000000000002,"15min":0.41799999999999998},"max":{"1min":1.73,"5min":2.323,"15min":8.0690000000000008},"last":1.3340000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.417,"15min":1.4610000000000001},"min":{"1min":0.93000000000000005,"5min":0.83399999999999996,"15min":0.79300000000000004},"max":{"1min":1.8009999999999999,"5min":2.6139999999999999,"15min":5.3929999999999998},"last":1.623},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.468,"15min":1.5489999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.78300000000000003},"max":{"1min":2.1949999999999998,"5min":3.1070000000000002,"15min":7.3570000000000002},"last":2.1179999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3939999999999999,"15min":1.546},"min":{"1min":0.93200000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9199999999999999,"5min":2.5579999999999998,"15min":8.1839999999999993},"last":1.6890000000000001},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.417},"min":{"1min":0.99299999999999999,"5min":0.81000000000000005,"15min":0.72199999999999998},"max":{"1min":1.845,"5min":2.4940000000000002,"15min":5.165},"last":1.8240000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.4350000000000001,"15min":1.454},"min":{"1min":0.82399999999999995,"5min":0.752,"15min":0.72999999999999998},"max":{"1min":2.5139999999999998,"5min":3.0179999999999998,"15min":5.1059999999999999},"last":1.4810000000000001},{"interface":"front","average":{"1min":1.446,"5min":1.4650000000000001,"15min":1.5589999999999999},"min":{"1min":1.0049999999999999,"5min":0.81200000000000006,"15min":0.77000000000000002},"max":{"1min":1.879,"5min":2.9580000000000002,"15min":8.1370000000000005},"last":1.915}]},{"osd":7,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.5149999999999999,"15min":1.552},"min":{"1min":0.98499999999999999,"5min":0.79200000000000004,"15min":0.78200000000000003},"max":{"1min":2.3700000000000001,"5min":3.2189999999999999,"15min":5.1449999999999996},"last":1.7090000000000001},{"interface":"front","average":{"1min":1.696,"5min":1.548,"15min":1.659},"min":{"1min":1.054,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":4.8289999999999997,"5min":4.8289999999999997,"15min":8.2300000000000004},"last":1.7629999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1314,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193998,"statfs":{"total":200000000,"available":198654440,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.84599999999999997},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3660000000000001,"5min":1.52,"15min":1.52},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.46899999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0720000000000001,"15min":1.0609999999999999},"min":{"1min":0.52500000000000002,"5min":0.42299999999999999,"15min":0.41399999999999998},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.5739999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.026,"15min":0.93600000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":1.6839999999999999,"5min":1.845,"15min":1.845},"last":0.52200000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3049999999999999,"15min":1.1719999999999999},"min":{"1min":0.64200000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.1040000000000001,"5min":2.137,"15min":2.137},"last":0.59599999999999997},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.2170000000000001,"15min":1.1930000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.44900000000000001},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.3959999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3520000000000001,"15min":1.2989999999999999},"min":{"1min":0.83999999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1440000000000001,"5min":2.444,"15min":2.444},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4039999999999999,"15min":1.3100000000000001},"min":{"1min":0.95299999999999996,"5min":0.76900000000000002,"15min":0.71099999999999997},"max":{"1min":1.7569999999999999,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":0.96499999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.425,"15min":1.3100000000000001},"min":{"1min":0.92800000000000005,"5min":0.88,"15min":0.88},"max":{"1min":1.915,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5049999999999999,"15min":1.423},"min":{"1min":1.0009999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.371,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.107}]},{"osd":6,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4530000000000001,"15min":1.3899999999999999},"min":{"1min":0.88400000000000001,"5min":0.875,"15min":0.76900000000000002},"max":{"1min":2.3610000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.538,"15min":1.46},"min":{"1min":1.032,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.2719999999999998,"5min":2.895,"15min":2.895},"last":1.014}]},{"osd":7,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.571,"15min":1.397},"min":{"1min":1.018,"5min":0.879,"15min":0.69999999999999996},"max":{"1min":1.8129999999999999,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":1.002},{"interface":"front","average":{"1min":1.468,"5min":1.659,"15min":1.508},"min":{"1min":0.97399999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.2360000000000002,"5min":9.4109999999999996,"15min":9.4109999999999996},"last":0.98099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574941,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":735,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199246484,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.99399999999999999,"15min":0.90600000000000003},"min":{"1min":0.64000000000000001,"5min":0.46700000000000003,"15min":0.39900000000000002},"max":{"1min":1.3220000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.143,"15min":1.0429999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.0369999999999999,"5min":2.1040000000000001,"15min":2.4180000000000001},"last":1.3460000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0660000000000001,"15min":0.96299999999999997},"min":{"1min":0.73099999999999998,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":1.4810000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0389999999999999,"15min":0.94699999999999995},"min":{"1min":0.55100000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.335,"5min":2.7810000000000001,"15min":2.7810000000000001},"last":0.70299999999999996}]},{"osd":3,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0880000000000001,"15min":1.0209999999999999},"min":{"1min":0.61299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.732,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.999},{"interface":"front","average":{"1min":1.306,"5min":1.3580000000000001,"15min":1.2390000000000001},"min":{"1min":0.746,"5min":0.63100000000000001,"15min":0.47499999999999998},"max":{"1min":1.982,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.411}]},{"osd":4,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.335,"15min":1.216},"min":{"1min":0.86299999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6739999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.282},{"interface":"front","average":{"1min":1.343,"5min":1.3360000000000001,"15min":1.2929999999999999},"min":{"1min":1.0409999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.663,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.119}]},{"osd":5,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.5489999999999999,"15min":1.4059999999999999},"min":{"1min":1.0169999999999999,"5min":0.71099999999999997,"15min":0.64500000000000002},"max":{"1min":1.877,"5min":16.298999999999999,"15min":16.298999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.385,"15min":1.2909999999999999},"min":{"1min":0.94899999999999995,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8109999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.696}]},{"osd":6,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.5029999999999999,"15min":1.409},"min":{"1min":1.171,"5min":0.85499999999999998,"15min":0.75900000000000001},"max":{"1min":1.9119999999999999,"5min":2.2559999999999998,"15min":2.5910000000000002},"last":1.776},{"interface":"front","average":{"1min":1.391,"5min":1.462,"15min":1.369},"min":{"1min":1.0349999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.776,"5min":2.46,"15min":2.46},"last":1.3819999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5369999999999999,"15min":1.409},"min":{"1min":1.1279999999999999,"5min":0.89200000000000002,"15min":0.70899999999999996},"max":{"1min":1.8740000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.4730000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.466,"15min":1.4330000000000001},"min":{"1min":0.88400000000000001,"5min":0.873,"15min":0.81999999999999995},"max":{"1min":2.113,"5min":2.4580000000000002,"15min":3.1429999999999998},"last":1.3100000000000001}]}]}],"pool_statfs":[]}} 2023-12-19T18:22:24.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:22:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.623+0000 7f24098b9640 1 Processor -- start 2023-12-19T18:22:24.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.624+0000 7f24098b9640 1 -- start start 2023-12-19T18:22:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.625+0000 7f24098b9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f240415ad60 0x7f240415b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.625+0000 7f24098b9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f240415b680 0x7f24040b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.626+0000 7f24098b9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24040ce050 con 0x7f24040b3170 2023-12-19T18:22:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.626+0000 7f24098b9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240415bb20 con 0x7f240415ad60 2023-12-19T18:22:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.626+0000 7f24098b9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f240415be00 con 0x7f240415b680 2023-12-19T18:22:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.626+0000 7f24088b7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f240415ad60 0x7f240415b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.626+0000 7f24088b7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f240415ad60 0x7f240415b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45114/0 (socket says 172.21.15.27:45114) 2023-12-19T18:22:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.626+0000 7f24088b7640 1 -- 172.21.15.27:0/3854119173 learned_addr learned my addr 172.21.15.27:0/3854119173 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.626+0000 7f2403fff640 1 --2- 172.21.15.27:0/3854119173 >> v2:172.21.15.192:3300/0 conn(0x7f240415b680 0x7f24040b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.626+0000 7f24088b7640 1 -- 172.21.15.27:0/3854119173 >> v2:172.21.15.192:3300/0 conn(0x7f240415b680 msgr2=0x7f24040b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.626+0000 7f24088b7640 1 --2- 172.21.15.27:0/3854119173 >> v2:172.21.15.192:3300/0 conn(0x7f240415b680 0x7f24040b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.626+0000 7f24088b7640 1 -- 172.21.15.27:0/3854119173 >> v1:172.21.15.192:6789/0 conn(0x7f24040b3170 legacy=0x7f240415e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.626+0000 7f24088b7640 1 -- 172.21.15.27:0/3854119173 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f240415c0e0 con 0x7f240415ad60 2023-12-19T18:22:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.626+0000 7f2403fff640 1 --2- 172.21.15.27:0/3854119173 >> v2:172.21.15.192:3300/0 conn(0x7f240415b680 0x7f24040b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.628+0000 7f24088b7640 1 --2- 172.21.15.27:0/3854119173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f240415ad60 0x7f240415b140 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f240406e620 tx=0x7f23f802b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d35b017eaa39bbd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.628+0000 7f24037fe640 1 -- 172.21.15.27:0/3854119173 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8031890 con 0x7f240415ad60 2023-12-19T18:22:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.628+0000 7f24037fe640 1 -- 172.21.15.27:0/3854119173 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23f80319f0 con 0x7f240415ad60 2023-12-19T18:22:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.628+0000 7f24037fe640 1 -- 172.21.15.27:0/3854119173 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23f8031d80 con 0x7f240415ad60 2023-12-19T18:22:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.629+0000 7f24098b9640 1 -- 172.21.15.27:0/3854119173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f240415ad60 msgr2=0x7f240415b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.629+0000 7f24098b9640 1 --2- 172.21.15.27:0/3854119173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f240415ad60 0x7f240415b140 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f240406e620 tx=0x7f23f802b930 comp rx=0 tx=0).stop 2023-12-19T18:22:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.629+0000 7f24098b9640 1 -- 172.21.15.27:0/3854119173 shutdown_connections 2023-12-19T18:22:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.629+0000 7f24098b9640 1 --2- 172.21.15.27:0/3854119173 >> v2:172.21.15.192:3300/0 conn(0x7f240415b680 0x7f24040b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.629+0000 7f24098b9640 1 --2- 172.21.15.27:0/3854119173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f240415ad60 0x7f240415b140 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.629+0000 7f24098b9640 1 -- 172.21.15.27:0/3854119173 >> 172.21.15.27:0/3854119173 conn(0x7f240406d5f0 msgr2=0x7f240406d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.629+0000 7f24098b9640 1 -- 172.21.15.27:0/3854119173 shutdown_connections 2023-12-19T18:22:24.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.629+0000 7f24098b9640 1 -- 172.21.15.27:0/3854119173 wait complete. 2023-12-19T18:22:24.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.631+0000 7f24098b9640 1 Processor -- start 2023-12-19T18:22:24.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.631+0000 7f24098b9640 1 -- start start 2023-12-19T18:22:24.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.632+0000 7f24098b9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f24040b3170 0x7f240415c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.632+0000 7f24098b9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f240415ad60 0x7f24040c7c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.632+0000 7f24098b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f240415b680 0x7f24041f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.632+0000 7f24098b9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240415d040 con 0x7f240415b680 2023-12-19T18:22:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.632+0000 7f24098b9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f240415d320 con 0x7f240415ad60 2023-12-19T18:22:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.632+0000 7f24098b9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24041f5b60 con 0x7f24040b3170 2023-12-19T18:22:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.632+0000 7f24090b8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f240415b680 0x7f24041f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.632+0000 7f24090b8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f240415b680 0x7f24041f6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45122/0 (socket says 172.21.15.27:45122) 2023-12-19T18:22:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.632+0000 7f24090b8640 1 -- 172.21.15.27:0/436975885 learned_addr learned my addr 172.21.15.27:0/436975885 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.632+0000 7f24088b7640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f24040b3170 0x7f240415c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.633+0000 7f2403fff640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f240415ad60 0x7f24040c7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.633+0000 7f24090b8640 1 -- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f24040b3170 msgr2=0x7f240415c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.633+0000 7f24090b8640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f24040b3170 0x7f240415c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:24.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.633+0000 7f24090b8640 1 -- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f240415ad60 msgr2=0x7f24040c7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.633+0000 7f24090b8640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f240415ad60 0x7f24040c7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.633+0000 7f24090b8640 1 -- 172.21.15.27:0/436975885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f80094d0 con 0x7f240415b680 2023-12-19T18:22:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.633+0000 7f2403fff640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f240415ad60 0x7f24040c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:24.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.633+0000 7f24090b8640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f240415b680 0x7f24041f6080 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00b990 tx=0x7f23fc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.633+0000 7f24017fa640 1 -- 172.21.15.27:0/436975885 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc00cba0 con 0x7f240415b680 2023-12-19T18:22:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.633+0000 7f24017fa640 1 -- 172.21.15.27:0/436975885 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23fc00cd00 con 0x7f240415b680 2023-12-19T18:22:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.633+0000 7f24017fa640 1 -- 172.21.15.27:0/436975885 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23fc014680 con 0x7f240415b680 2023-12-19T18:22:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.633+0000 7f24098b9640 1 -- 172.21.15.27:0/436975885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f240415d600 con 0x7f240415b680 2023-12-19T18:22:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.634+0000 7f24098b9640 1 -- 172.21.15.27:0/436975885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24040af6d0 con 0x7f240415b680 2023-12-19T18:22:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.635+0000 7f24098b9640 1 -- 172.21.15.27:0/436975885 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24041f5d80 con 0x7f240415b680 2023-12-19T18:22:24.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.636+0000 7f24017fa640 1 -- 172.21.15.27:0/436975885 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f23fc01d3b0 con 0x7f240415b680 2023-12-19T18:22:24.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.637+0000 7f24017fa640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f23e806a460 0x7f23e806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:24.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.637+0000 7f24017fa640 1 -- 172.21.15.27:0/436975885 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23fc088900 con 0x7f240415b680 2023-12-19T18:22:24.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.638+0000 7f24088b7640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f23e806a460 0x7f23e806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:24.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.639+0000 7f24088b7640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f23e806a460 0x7f23e806c920 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f23f8036000 tx=0x7f23f803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:24.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.642+0000 7f24017fa640 1 -- 172.21.15.27:0/436975885 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f23fc053e50 con 0x7f240415b680 2023-12-19T18:22:24.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.832+0000 7f24098b9640 1 -- 172.21.15.27:0/436975885 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f240415d490 con 0x7f23e806a460 2023-12-19T18:22:24.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.856+0000 7f24017fa640 1 -- 172.21.15.27:0/436975885 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+283031 (secure 0 0 0) 0x7f240415d490 con 0x7f23e806a460 2023-12-19T18:22:24.857 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:24.861 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:22:24.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.863+0000 7f24098b9640 1 -- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f23e806a460 msgr2=0x7f23e806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:24.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.863+0000 7f24098b9640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f23e806a460 0x7f23e806c920 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f23f8036000 tx=0x7f23f803c040 comp rx=0 tx=0).stop 2023-12-19T18:22:24.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.863+0000 7f24098b9640 1 -- 172.21.15.27:0/436975885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f240415b680 msgr2=0x7f24041f6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:24.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.863+0000 7f24098b9640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f240415b680 0x7f24041f6080 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f23fc00b990 tx=0x7f23fc00be50 comp rx=0 tx=0).stop 2023-12-19T18:22:24.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.863+0000 7f24098b9640 1 -- 172.21.15.27:0/436975885 shutdown_connections 2023-12-19T18:22:24.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.863+0000 7f24098b9640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f23e806a460 0x7f23e806c920 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:24.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.863+0000 7f24098b9640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f240415b680 0x7f24041f6080 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:24.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.863+0000 7f24098b9640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f240415ad60 0x7f24040c7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:24.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.863+0000 7f24098b9640 1 --2- 172.21.15.27:0/436975885 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f24040b3170 0x7f240415c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:24.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.863+0000 7f24098b9640 1 -- 172.21.15.27:0/436975885 >> 172.21.15.27:0/436975885 conn(0x7f240406d5f0 msgr2=0x7f24040ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:24.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.863+0000 7f24098b9640 1 -- 172.21.15.27:0/436975885 shutdown_connections 2023-12-19T18:22:24.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:24.863+0000 7f24098b9640 1 -- 172.21.15.27:0/436975885 wait complete. 2023-12-19T18:22:24.884 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":357,"stamp":"2023-12-19T18:22:23.868181+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7044,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1555455,"statfs":{"total":1600000000,"available":1592786412,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.640268"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258377+0000","last_change":"2023-12-19T18:22:14.257437+0000","last_active":"2023-12-19T18:22:14.258377+0000","last_peered":"2023-12-19T18:22:14.258377+0000","last_clean":"2023-12-19T18:22:14.258377+0000","last_became_active":"2023-12-19T18:22:14.256527+0000","last_became_peered":"2023-12-19T18:22:14.256527+0000","last_unstale":"2023-12-19T18:22:14.258377+0000","last_undegraded":"2023-12-19T18:22:14.258377+0000","last_fullsized":"2023-12-19T18:22:14.258377+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:40.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258346+0000","last_change":"2023-12-19T18:22:14.255774+0000","last_active":"2023-12-19T18:22:14.258346+0000","last_peered":"2023-12-19T18:22:14.258346+0000","last_clean":"2023-12-19T18:22:14.258346+0000","last_became_active":"2023-12-19T18:22:14.255220+0000","last_became_peered":"2023-12-19T18:22:14.255220+0000","last_unstale":"2023-12-19T18:22:14.258346+0000","last_undegraded":"2023-12-19T18:22:14.258346+0000","last_fullsized":"2023-12-19T18:22:14.258346+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:15.726948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.256564+0000","last_change":"2023-12-19T18:22:14.256359+0000","last_active":"2023-12-19T18:22:14.256564+0000","last_peered":"2023-12-19T18:22:14.256564+0000","last_clean":"2023-12-19T18:22:14.256564+0000","last_became_active":"2023-12-19T18:22:14.255780+0000","last_became_peered":"2023-12-19T18:22:14.255780+0000","last_unstale":"2023-12-19T18:22:14.256564+0000","last_undegraded":"2023-12-19T18:22:14.256564+0000","last_fullsized":"2023-12-19T18:22:14.256564+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:16.333492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258589+0000","last_change":"2023-12-19T18:22:14.258495+0000","last_active":"2023-12-19T18:22:14.258589+0000","last_peered":"2023-12-19T18:22:14.258589+0000","last_clean":"2023-12-19T18:22:14.258589+0000","last_became_active":"2023-12-19T18:22:14.258217+0000","last_became_peered":"2023-12-19T18:22:14.258217+0000","last_unstale":"2023-12-19T18:22:14.258589+0000","last_undegraded":"2023-12-19T18:22:14.258589+0000","last_fullsized":"2023-12-19T18:22:14.258589+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:09.942336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.244066+0000","last_change":"2023-12-19T18:22:14.243278+0000","last_active":"2023-12-19T18:22:14.244066+0000","last_peered":"2023-12-19T18:22:14.244066+0000","last_clean":"2023-12-19T18:22:14.244066+0000","last_became_active":"2023-12-19T18:22:14.242452+0000","last_became_peered":"2023-12-19T18:22:14.242452+0000","last_unstale":"2023-12-19T18:22:14.244066+0000","last_undegraded":"2023-12-19T18:22:14.244066+0000","last_fullsized":"2023-12-19T18:22:14.244066+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:56:26.506333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.234713+0000","last_change":"2023-12-19T18:22:14.234640+0000","last_active":"2023-12-19T18:22:14.234713+0000","last_peered":"2023-12-19T18:22:14.234713+0000","last_clean":"2023-12-19T18:22:14.234713+0000","last_became_active":"2023-12-19T18:22:14.234044+0000","last_became_peered":"2023-12-19T18:22:14.234044+0000","last_unstale":"2023-12-19T18:22:14.234713+0000","last_undegraded":"2023-12-19T18:22:14.234713+0000","last_fullsized":"2023-12-19T18:22:14.234713+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:02:55.424114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239891+0000","last_change":"2023-12-19T18:22:14.239762+0000","last_active":"2023-12-19T18:22:14.239891+0000","last_peered":"2023-12-19T18:22:14.239891+0000","last_clean":"2023-12-19T18:22:14.239891+0000","last_became_active":"2023-12-19T18:22:14.239355+0000","last_became_peered":"2023-12-19T18:22:14.239355+0000","last_unstale":"2023-12-19T18:22:14.239891+0000","last_undegraded":"2023-12-19T18:22:14.239891+0000","last_fullsized":"2023-12-19T18:22:14.239891+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:37:37.391952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.256258+0000","last_change":"2023-12-19T18:22:14.256157+0000","last_active":"2023-12-19T18:22:14.256258+0000","last_peered":"2023-12-19T18:22:14.256258+0000","last_clean":"2023-12-19T18:22:14.256258+0000","last_became_active":"2023-12-19T18:22:14.255443+0000","last_became_peered":"2023-12-19T18:22:14.255443+0000","last_unstale":"2023-12-19T18:22:14.256258+0000","last_undegraded":"2023-12-19T18:22:14.256258+0000","last_fullsized":"2023-12-19T18:22:14.256258+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:23.915911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252425+0000","last_change":"2023-12-19T18:22:14.252334+0000","last_active":"2023-12-19T18:22:14.252425+0000","last_peered":"2023-12-19T18:22:14.252425+0000","last_clean":"2023-12-19T18:22:14.252425+0000","last_became_active":"2023-12-19T18:22:14.252176+0000","last_became_peered":"2023-12-19T18:22:14.252176+0000","last_unstale":"2023-12-19T18:22:14.252425+0000","last_undegraded":"2023-12-19T18:22:14.252425+0000","last_fullsized":"2023-12-19T18:22:14.252425+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:29.637020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259112+0000","last_change":"2023-12-19T18:22:14.258799+0000","last_active":"2023-12-19T18:22:14.259112+0000","last_peered":"2023-12-19T18:22:14.259112+0000","last_clean":"2023-12-19T18:22:14.259112+0000","last_became_active":"2023-12-19T18:22:14.258575+0000","last_became_peered":"2023-12-19T18:22:14.258575+0000","last_unstale":"2023-12-19T18:22:14.259112+0000","last_undegraded":"2023-12-19T18:22:14.259112+0000","last_fullsized":"2023-12-19T18:22:14.259112+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:11:57.860995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.234856+0000","last_change":"2023-12-19T18:22:14.234773+0000","last_active":"2023-12-19T18:22:14.234856+0000","last_peered":"2023-12-19T18:22:14.234856+0000","last_clean":"2023-12-19T18:22:14.234856+0000","last_became_active":"2023-12-19T18:22:14.234418+0000","last_became_peered":"2023-12-19T18:22:14.234418+0000","last_unstale":"2023-12-19T18:22:14.234856+0000","last_undegraded":"2023-12-19T18:22:14.234856+0000","last_fullsized":"2023-12-19T18:22:14.234856+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:07:23.903727+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.256370+0000","last_change":"2023-12-19T18:22:14.256302+0000","last_active":"2023-12-19T18:22:14.256370+0000","last_peered":"2023-12-19T18:22:14.256370+0000","last_clean":"2023-12-19T18:22:14.256370+0000","last_became_active":"2023-12-19T18:22:14.256075+0000","last_became_peered":"2023-12-19T18:22:14.256075+0000","last_unstale":"2023-12-19T18:22:14.256370+0000","last_undegraded":"2023-12-19T18:22:14.256370+0000","last_fullsized":"2023-12-19T18:22:14.256370+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:31.105594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245932+0000","last_change":"2023-12-19T18:22:14.244697+0000","last_active":"2023-12-19T18:22:14.245932+0000","last_peered":"2023-12-19T18:22:14.245932+0000","last_clean":"2023-12-19T18:22:14.245932+0000","last_became_active":"2023-12-19T18:22:14.244184+0000","last_became_peered":"2023-12-19T18:22:14.244184+0000","last_unstale":"2023-12-19T18:22:14.245932+0000","last_undegraded":"2023-12-19T18:22:14.245932+0000","last_fullsized":"2023-12-19T18:22:14.245932+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:49:33.150825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.256363+0000","last_change":"2023-12-19T18:22:14.254104+0000","last_active":"2023-12-19T18:22:14.256363+0000","last_peered":"2023-12-19T18:22:14.256363+0000","last_clean":"2023-12-19T18:22:14.256363+0000","last_became_active":"2023-12-19T18:22:14.253879+0000","last_became_peered":"2023-12-19T18:22:14.253879+0000","last_unstale":"2023-12-19T18:22:14.256363+0000","last_undegraded":"2023-12-19T18:22:14.256363+0000","last_fullsized":"2023-12-19T18:22:14.256363+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:35:16.333492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259380+0000","last_change":"2023-12-19T18:22:14.258835+0000","last_active":"2023-12-19T18:22:14.259380+0000","last_peered":"2023-12-19T18:22:14.259380+0000","last_clean":"2023-12-19T18:22:14.259380+0000","last_became_active":"2023-12-19T18:22:14.257437+0000","last_became_peered":"2023-12-19T18:22:14.257437+0000","last_unstale":"2023-12-19T18:22:14.259380+0000","last_undegraded":"2023-12-19T18:22:14.259380+0000","last_fullsized":"2023-12-19T18:22:14.259380+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:34.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.240074+0000","last_change":"2023-12-19T18:22:14.239965+0000","last_active":"2023-12-19T18:22:14.240074+0000","last_peered":"2023-12-19T18:22:14.240074+0000","last_clean":"2023-12-19T18:22:14.240074+0000","last_became_active":"2023-12-19T18:22:14.239741+0000","last_became_peered":"2023-12-19T18:22:14.239741+0000","last_unstale":"2023-12-19T18:22:14.240074+0000","last_undegraded":"2023-12-19T18:22:14.240074+0000","last_fullsized":"2023-12-19T18:22:14.240074+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:02.299174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.237521+0000","last_change":"2023-12-19T18:22:14.237215+0000","last_active":"2023-12-19T18:22:14.237521+0000","last_peered":"2023-12-19T18:22:14.237521+0000","last_clean":"2023-12-19T18:22:14.237521+0000","last_became_active":"2023-12-19T18:22:14.236055+0000","last_became_peered":"2023-12-19T18:22:14.236055+0000","last_unstale":"2023-12-19T18:22:14.237521+0000","last_undegraded":"2023-12-19T18:22:14.237521+0000","last_fullsized":"2023-12-19T18:22:14.237521+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:41:21.616217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.261591+0000","last_change":"2023-12-19T18:22:14.261506+0000","last_active":"2023-12-19T18:22:14.261591+0000","last_peered":"2023-12-19T18:22:14.261591+0000","last_clean":"2023-12-19T18:22:14.261591+0000","last_became_active":"2023-12-19T18:22:14.259331+0000","last_became_peered":"2023-12-19T18:22:14.259331+0000","last_unstale":"2023-12-19T18:22:14.261591+0000","last_undegraded":"2023-12-19T18:22:14.261591+0000","last_fullsized":"2023-12-19T18:22:14.261591+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:44.388621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.260554+0000","last_change":"2023-12-19T18:22:14.260473+0000","last_active":"2023-12-19T18:22:14.260554+0000","last_peered":"2023-12-19T18:22:14.260554+0000","last_clean":"2023-12-19T18:22:14.260554+0000","last_became_active":"2023-12-19T18:22:14.259952+0000","last_became_peered":"2023-12-19T18:22:14.259952+0000","last_unstale":"2023-12-19T18:22:14.260554+0000","last_undegraded":"2023-12-19T18:22:14.260554+0000","last_fullsized":"2023-12-19T18:22:14.260554+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:41:30.585976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246133+0000","last_change":"2023-12-19T18:22:14.246068+0000","last_active":"2023-12-19T18:22:14.246133+0000","last_peered":"2023-12-19T18:22:14.246133+0000","last_clean":"2023-12-19T18:22:14.246133+0000","last_became_active":"2023-12-19T18:22:14.245845+0000","last_became_peered":"2023-12-19T18:22:14.245845+0000","last_unstale":"2023-12-19T18:22:14.246133+0000","last_undegraded":"2023-12-19T18:22:14.246133+0000","last_fullsized":"2023-12-19T18:22:14.246133+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:40.700106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.254074+0000","last_change":"2023-12-19T18:22:14.253786+0000","last_active":"2023-12-19T18:22:14.254074+0000","last_peered":"2023-12-19T18:22:14.254074+0000","last_clean":"2023-12-19T18:22:14.254074+0000","last_became_active":"2023-12-19T18:22:14.253409+0000","last_became_peered":"2023-12-19T18:22:14.253409+0000","last_unstale":"2023-12-19T18:22:14.254074+0000","last_undegraded":"2023-12-19T18:22:14.254074+0000","last_fullsized":"2023-12-19T18:22:14.254074+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:11.523949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.257498+0000","last_change":"2023-12-19T18:22:14.255178+0000","last_active":"2023-12-19T18:22:14.257498+0000","last_peered":"2023-12-19T18:22:14.257498+0000","last_clean":"2023-12-19T18:22:14.257498+0000","last_became_active":"2023-12-19T18:22:14.254755+0000","last_became_peered":"2023-12-19T18:22:14.254755+0000","last_unstale":"2023-12-19T18:22:14.257498+0000","last_undegraded":"2023-12-19T18:22:14.257498+0000","last_fullsized":"2023-12-19T18:22:14.257498+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:13:15.250452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252608+0000","last_change":"2023-12-19T18:22:14.250276+0000","last_active":"2023-12-19T18:22:14.252608+0000","last_peered":"2023-12-19T18:22:14.252608+0000","last_clean":"2023-12-19T18:22:14.252608+0000","last_became_active":"2023-12-19T18:22:14.250040+0000","last_became_peered":"2023-12-19T18:22:14.250040+0000","last_unstale":"2023-12-19T18:22:14.252608+0000","last_undegraded":"2023-12-19T18:22:14.252608+0000","last_fullsized":"2023-12-19T18:22:14.252608+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:08.465323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259701+0000","last_change":"2023-12-19T18:22:14.259592+0000","last_active":"2023-12-19T18:22:14.259701+0000","last_peered":"2023-12-19T18:22:14.259701+0000","last_clean":"2023-12-19T18:22:14.259701+0000","last_became_active":"2023-12-19T18:22:14.259307+0000","last_became_peered":"2023-12-19T18:22:14.259307+0000","last_unstale":"2023-12-19T18:22:14.259701+0000","last_undegraded":"2023-12-19T18:22:14.259701+0000","last_fullsized":"2023-12-19T18:22:14.259701+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:02:16.250351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239849+0000","last_change":"2023-12-19T18:22:14.236493+0000","last_active":"2023-12-19T18:22:14.239849+0000","last_peered":"2023-12-19T18:22:14.239849+0000","last_clean":"2023-12-19T18:22:14.239849+0000","last_became_active":"2023-12-19T18:22:14.236218+0000","last_became_peered":"2023-12-19T18:22:14.236218+0000","last_unstale":"2023-12-19T18:22:14.239849+0000","last_undegraded":"2023-12-19T18:22:14.239849+0000","last_fullsized":"2023-12-19T18:22:14.239849+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:32.485695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245899+0000","last_change":"2023-12-19T18:22:14.242241+0000","last_active":"2023-12-19T18:22:14.245899+0000","last_peered":"2023-12-19T18:22:14.245899+0000","last_clean":"2023-12-19T18:22:14.245899+0000","last_became_active":"2023-12-19T18:22:14.241678+0000","last_became_peered":"2023-12-19T18:22:14.241678+0000","last_unstale":"2023-12-19T18:22:14.245899+0000","last_undegraded":"2023-12-19T18:22:14.245899+0000","last_fullsized":"2023-12-19T18:22:14.245899+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:08.366785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248454+0000","last_change":"2023-12-19T18:22:14.247740+0000","last_active":"2023-12-19T18:22:14.248454+0000","last_peered":"2023-12-19T18:22:14.248454+0000","last_clean":"2023-12-19T18:22:14.248454+0000","last_became_active":"2023-12-19T18:22:14.247511+0000","last_became_peered":"2023-12-19T18:22:14.247511+0000","last_unstale":"2023-12-19T18:22:14.248454+0000","last_undegraded":"2023-12-19T18:22:14.248454+0000","last_fullsized":"2023-12-19T18:22:14.248454+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:57:01.585693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.257231+0000","last_change":"2023-12-19T18:22:14.257151+0000","last_active":"2023-12-19T18:22:14.257231+0000","last_peered":"2023-12-19T18:22:14.257231+0000","last_clean":"2023-12-19T18:22:14.257231+0000","last_became_active":"2023-12-19T18:22:14.256854+0000","last_became_peered":"2023-12-19T18:22:14.256854+0000","last_unstale":"2023-12-19T18:22:14.257231+0000","last_undegraded":"2023-12-19T18:22:14.257231+0000","last_fullsized":"2023-12-19T18:22:14.257231+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:01:59.548259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.238540+0000","last_change":"2023-12-19T18:22:14.236261+0000","last_active":"2023-12-19T18:22:14.238540+0000","last_peered":"2023-12-19T18:22:14.238540+0000","last_clean":"2023-12-19T18:22:14.238540+0000","last_became_active":"2023-12-19T18:22:14.234425+0000","last_became_peered":"2023-12-19T18:22:14.234425+0000","last_unstale":"2023-12-19T18:22:14.238540+0000","last_undegraded":"2023-12-19T18:22:14.238540+0000","last_fullsized":"2023-12-19T18:22:14.238540+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:32.485695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245871+0000","last_change":"2023-12-19T18:22:14.241276+0000","last_active":"2023-12-19T18:22:14.245871+0000","last_peered":"2023-12-19T18:22:14.245871+0000","last_clean":"2023-12-19T18:22:14.245871+0000","last_became_active":"2023-12-19T18:22:14.240498+0000","last_became_peered":"2023-12-19T18:22:14.240498+0000","last_unstale":"2023-12-19T18:22:14.245871+0000","last_undegraded":"2023-12-19T18:22:14.245871+0000","last_fullsized":"2023-12-19T18:22:14.245871+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:47:09.595830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252788+0000","last_change":"2023-12-19T18:22:14.252269+0000","last_active":"2023-12-19T18:22:14.252788+0000","last_peered":"2023-12-19T18:22:14.252788+0000","last_clean":"2023-12-19T18:22:14.252788+0000","last_became_active":"2023-12-19T18:22:14.251959+0000","last_became_peered":"2023-12-19T18:22:14.251959+0000","last_unstale":"2023-12-19T18:22:14.252788+0000","last_undegraded":"2023-12-19T18:22:14.252788+0000","last_fullsized":"2023-12-19T18:22:14.252788+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:01:36.203567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258305+0000","last_change":"2023-12-19T18:22:14.255516+0000","last_active":"2023-12-19T18:22:14.258305+0000","last_peered":"2023-12-19T18:22:14.258305+0000","last_clean":"2023-12-19T18:22:14.258305+0000","last_became_active":"2023-12-19T18:22:14.254491+0000","last_became_peered":"2023-12-19T18:22:14.254491+0000","last_unstale":"2023-12-19T18:22:14.258305+0000","last_undegraded":"2023-12-19T18:22:14.258305+0000","last_fullsized":"2023-12-19T18:22:14.258305+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:16.254938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.234589+0000","last_change":"2023-12-19T18:22:14.232017+0000","last_active":"2023-12-19T18:22:14.234589+0000","last_peered":"2023-12-19T18:22:14.234589+0000","last_clean":"2023-12-19T18:22:14.234589+0000","last_became_active":"2023-12-19T18:22:14.231247+0000","last_became_peered":"2023-12-19T18:22:14.231247+0000","last_unstale":"2023-12-19T18:22:14.234589+0000","last_undegraded":"2023-12-19T18:22:14.234589+0000","last_fullsized":"2023-12-19T18:22:14.234589+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:47:33.697121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1f","version":"134'4","reported_seq":15,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.257068+0000","last_change":"2023-12-19T18:22:13.732827+0000","last_active":"2023-12-19T18:22:14.257068+0000","last_peered":"2023-12-19T18:22:14.257068+0000","last_clean":"2023-12-19T18:22:14.257068+0000","last_became_active":"2023-12-19T18:22:13.732603+0000","last_became_peered":"2023-12-19T18:22:13.732603+0000","last_unstale":"2023-12-19T18:22:14.257068+0000","last_undegraded":"2023-12-19T18:22:14.257068+0000","last_fullsized":"2023-12-19T18:22:14.257068+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:41:12.315002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245917+0000","last_change":"2023-12-19T18:22:14.243673+0000","last_active":"2023-12-19T18:22:14.245917+0000","last_peered":"2023-12-19T18:22:14.245917+0000","last_clean":"2023-12-19T18:22:14.245917+0000","last_became_active":"2023-12-19T18:22:14.243147+0000","last_became_peered":"2023-12-19T18:22:14.243147+0000","last_unstale":"2023-12-19T18:22:14.245917+0000","last_undegraded":"2023-12-19T18:22:14.245917+0000","last_fullsized":"2023-12-19T18:22:14.245917+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:39.384923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1e","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248064+0000","last_change":"2023-12-19T18:22:13.731624+0000","last_active":"2023-12-19T18:22:14.248064+0000","last_peered":"2023-12-19T18:22:14.248064+0000","last_clean":"2023-12-19T18:22:14.248064+0000","last_became_active":"2023-12-19T18:22:13.731335+0000","last_became_peered":"2023-12-19T18:22:13.731335+0000","last_unstale":"2023-12-19T18:22:14.248064+0000","last_undegraded":"2023-12-19T18:22:14.248064+0000","last_fullsized":"2023-12-19T18:22:14.248064+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:53:16.279722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258361+0000","last_change":"2023-12-19T18:22:14.257304+0000","last_active":"2023-12-19T18:22:14.258361+0000","last_peered":"2023-12-19T18:22:14.258361+0000","last_clean":"2023-12-19T18:22:14.258361+0000","last_became_active":"2023-12-19T18:22:14.256258+0000","last_became_peered":"2023-12-19T18:22:14.256258+0000","last_unstale":"2023-12-19T18:22:14.258361+0000","last_undegraded":"2023-12-19T18:22:14.258361+0000","last_fullsized":"2023-12-19T18:22:14.258361+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:16.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.262268+0000","last_change":"2023-12-19T18:22:13.733176+0000","last_active":"2023-12-19T18:22:14.262268+0000","last_peered":"2023-12-19T18:22:14.262268+0000","last_clean":"2023-12-19T18:22:14.262268+0000","last_became_active":"2023-12-19T18:22:13.732940+0000","last_became_peered":"2023-12-19T18:22:13.732940+0000","last_unstale":"2023-12-19T18:22:14.262268+0000","last_undegraded":"2023-12-19T18:22:14.262268+0000","last_fullsized":"2023-12-19T18:22:14.262268+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:30.091059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.244211+0000","last_change":"2023-12-19T18:22:14.243888+0000","last_active":"2023-12-19T18:22:14.244211+0000","last_peered":"2023-12-19T18:22:14.244211+0000","last_clean":"2023-12-19T18:22:14.244211+0000","last_became_active":"2023-12-19T18:22:14.243667+0000","last_became_peered":"2023-12-19T18:22:14.243667+0000","last_unstale":"2023-12-19T18:22:14.244211+0000","last_undegraded":"2023-12-19T18:22:14.244211+0000","last_fullsized":"2023-12-19T18:22:14.244211+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:12:39.384923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1c","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.257151+0000","last_change":"2023-12-19T18:22:13.733119+0000","last_active":"2023-12-19T18:22:14.257151+0000","last_peered":"2023-12-19T18:22:14.257151+0000","last_clean":"2023-12-19T18:22:14.257151+0000","last_became_active":"2023-12-19T18:22:13.732868+0000","last_became_peered":"2023-12-19T18:22:13.732868+0000","last_unstale":"2023-12-19T18:22:14.257151+0000","last_undegraded":"2023-12-19T18:22:14.257151+0000","last_fullsized":"2023-12-19T18:22:14.257151+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:07.352249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.235616+0000","last_change":"2023-12-19T18:22:14.235017+0000","last_active":"2023-12-19T18:22:14.235616+0000","last_peered":"2023-12-19T18:22:14.235616+0000","last_clean":"2023-12-19T18:22:14.235616+0000","last_became_active":"2023-12-19T18:22:14.234807+0000","last_became_peered":"2023-12-19T18:22:14.234807+0000","last_unstale":"2023-12-19T18:22:14.235616+0000","last_undegraded":"2023-12-19T18:22:14.235616+0000","last_fullsized":"2023-12-19T18:22:14.235616+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:53.745021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1b","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252491+0000","last_change":"2023-12-19T18:22:13.730941+0000","last_active":"2023-12-19T18:22:14.252491+0000","last_peered":"2023-12-19T18:22:14.252491+0000","last_clean":"2023-12-19T18:22:14.252491+0000","last_became_active":"2023-12-19T18:22:13.730684+0000","last_became_peered":"2023-12-19T18:22:13.730684+0000","last_unstale":"2023-12-19T18:22:14.252491+0000","last_undegraded":"2023-12-19T18:22:14.252491+0000","last_fullsized":"2023-12-19T18:22:14.252491+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:43:06.634088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239743+0000","last_change":"2023-12-19T18:22:14.239588+0000","last_active":"2023-12-19T18:22:14.239743+0000","last_peered":"2023-12-19T18:22:14.239743+0000","last_clean":"2023-12-19T18:22:14.239743+0000","last_became_active":"2023-12-19T18:22:14.238776+0000","last_became_peered":"2023-12-19T18:22:14.238776+0000","last_unstale":"2023-12-19T18:22:14.239743+0000","last_undegraded":"2023-12-19T18:22:14.239743+0000","last_fullsized":"2023-12-19T18:22:14.239743+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:59:41.584580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1a","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259067+0000","last_change":"2023-12-19T18:22:13.731736+0000","last_active":"2023-12-19T18:22:14.259067+0000","last_peered":"2023-12-19T18:22:14.259067+0000","last_clean":"2023-12-19T18:22:14.259067+0000","last_became_active":"2023-12-19T18:22:13.731153+0000","last_became_peered":"2023-12-19T18:22:13.731153+0000","last_unstale":"2023-12-19T18:22:14.259067+0000","last_undegraded":"2023-12-19T18:22:14.259067+0000","last_fullsized":"2023-12-19T18:22:14.259067+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:52.730485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.254538+0000","last_change":"2023-12-19T18:22:14.254248+0000","last_active":"2023-12-19T18:22:14.254538+0000","last_peered":"2023-12-19T18:22:14.254538+0000","last_clean":"2023-12-19T18:22:14.254538+0000","last_became_active":"2023-12-19T18:22:14.254005+0000","last_became_peered":"2023-12-19T18:22:14.254005+0000","last_unstale":"2023-12-19T18:22:14.254538+0000","last_undegraded":"2023-12-19T18:22:14.254538+0000","last_fullsized":"2023-12-19T18:22:14.254538+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:11:57.860995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.235224+0000","last_change":"2023-12-19T18:22:13.730291+0000","last_active":"2023-12-19T18:22:14.235224+0000","last_peered":"2023-12-19T18:22:14.235224+0000","last_clean":"2023-12-19T18:22:14.235224+0000","last_became_active":"2023-12-19T18:22:13.730015+0000","last_became_peered":"2023-12-19T18:22:13.730015+0000","last_unstale":"2023-12-19T18:22:14.235224+0000","last_undegraded":"2023-12-19T18:22:14.235224+0000","last_fullsized":"2023-12-19T18:22:14.235224+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:02:54.040583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.241518+0000","last_change":"2023-12-19T18:22:14.241428+0000","last_active":"2023-12-19T18:22:14.241518+0000","last_peered":"2023-12-19T18:22:14.241518+0000","last_clean":"2023-12-19T18:22:14.241518+0000","last_became_active":"2023-12-19T18:22:14.241208+0000","last_became_peered":"2023-12-19T18:22:14.241208+0000","last_unstale":"2023-12-19T18:22:14.241518+0000","last_undegraded":"2023-12-19T18:22:14.241518+0000","last_fullsized":"2023-12-19T18:22:14.241518+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:45.553627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.224445+0000","last_change":"2023-12-19T18:22:13.730656+0000","last_active":"2023-12-19T18:22:14.224445+0000","last_peered":"2023-12-19T18:22:14.224445+0000","last_clean":"2023-12-19T18:22:14.224445+0000","last_became_active":"2023-12-19T18:22:13.730408+0000","last_became_peered":"2023-12-19T18:22:13.730408+0000","last_unstale":"2023-12-19T18:22:14.224445+0000","last_undegraded":"2023-12-19T18:22:14.224445+0000","last_fullsized":"2023-12-19T18:22:14.224445+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:33.562949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258327+0000","last_change":"2023-12-19T18:22:14.255646+0000","last_active":"2023-12-19T18:22:14.258327+0000","last_peered":"2023-12-19T18:22:14.258327+0000","last_clean":"2023-12-19T18:22:14.258327+0000","last_became_active":"2023-12-19T18:22:14.254964+0000","last_became_peered":"2023-12-19T18:22:14.254964+0000","last_unstale":"2023-12-19T18:22:14.258327+0000","last_undegraded":"2023-12-19T18:22:14.258327+0000","last_fullsized":"2023-12-19T18:22:14.258327+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:13:15.250452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245923+0000","last_change":"2023-12-19T18:22:13.724536+0000","last_active":"2023-12-19T18:22:14.245923+0000","last_peered":"2023-12-19T18:22:14.245923+0000","last_clean":"2023-12-19T18:22:14.245923+0000","last_became_active":"2023-12-19T18:22:13.724110+0000","last_became_peered":"2023-12-19T18:22:13.724110+0000","last_unstale":"2023-12-19T18:22:14.245923+0000","last_undegraded":"2023-12-19T18:22:14.245923+0000","last_fullsized":"2023-12-19T18:22:14.245923+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:44.539092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.245061+0000","last_change":"2023-12-19T18:22:14.244657+0000","last_active":"2023-12-19T18:22:14.245061+0000","last_peered":"2023-12-19T18:22:14.245061+0000","last_clean":"2023-12-19T18:22:14.245061+0000","last_became_active":"2023-12-19T18:22:14.244264+0000","last_became_peered":"2023-12-19T18:22:14.244264+0000","last_unstale":"2023-12-19T18:22:14.245061+0000","last_undegraded":"2023-12-19T18:22:14.245061+0000","last_fullsized":"2023-12-19T18:22:14.245061+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:15.726948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248854+0000","last_change":"2023-12-19T18:22:13.733261+0000","last_active":"2023-12-19T18:22:14.248854+0000","last_peered":"2023-12-19T18:22:14.248854+0000","last_clean":"2023-12-19T18:22:14.248854+0000","last_became_active":"2023-12-19T18:22:13.733015+0000","last_became_peered":"2023-12-19T18:22:13.733015+0000","last_unstale":"2023-12-19T18:22:14.248854+0000","last_undegraded":"2023-12-19T18:22:14.248854+0000","last_fullsized":"2023-12-19T18:22:14.248854+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:52.730485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259593+0000","last_change":"2023-12-19T18:22:14.259137+0000","last_active":"2023-12-19T18:22:14.259593+0000","last_peered":"2023-12-19T18:22:14.259593+0000","last_clean":"2023-12-19T18:22:14.259593+0000","last_became_active":"2023-12-19T18:22:14.258878+0000","last_became_peered":"2023-12-19T18:22:14.258878+0000","last_unstale":"2023-12-19T18:22:14.259593+0000","last_undegraded":"2023-12-19T18:22:14.259593+0000","last_fullsized":"2023-12-19T18:22:14.259593+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:32:56.103686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.15","version":"134'3","reported_seq":14,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.247294+0000","last_change":"2023-12-19T18:22:13.727917+0000","last_active":"2023-12-19T18:22:14.247294+0000","last_peered":"2023-12-19T18:22:14.247294+0000","last_clean":"2023-12-19T18:22:14.247294+0000","last_became_active":"2023-12-19T18:22:13.727633+0000","last_became_peered":"2023-12-19T18:22:13.727633+0000","last_unstale":"2023-12-19T18:22:14.247294+0000","last_undegraded":"2023-12-19T18:22:14.247294+0000","last_fullsized":"2023-12-19T18:22:14.247294+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:36:19.046380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.247094+0000","last_change":"2023-12-19T18:22:14.246791+0000","last_active":"2023-12-19T18:22:14.247094+0000","last_peered":"2023-12-19T18:22:14.247094+0000","last_clean":"2023-12-19T18:22:14.247094+0000","last_became_active":"2023-12-19T18:22:14.246271+0000","last_became_peered":"2023-12-19T18:22:14.246271+0000","last_unstale":"2023-12-19T18:22:14.247094+0000","last_undegraded":"2023-12-19T18:22:14.247094+0000","last_fullsized":"2023-12-19T18:22:14.247094+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:40.620758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.14","version":"134'6","reported_seq":17,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259355+0000","last_change":"2023-12-19T18:22:13.725517+0000","last_active":"2023-12-19T18:22:14.259355+0000","last_peered":"2023-12-19T18:22:14.259355+0000","last_clean":"2023-12-19T18:22:14.259355+0000","last_became_active":"2023-12-19T18:22:13.724094+0000","last_became_peered":"2023-12-19T18:22:13.724094+0000","last_unstale":"2023-12-19T18:22:14.259355+0000","last_undegraded":"2023-12-19T18:22:14.259355+0000","last_fullsized":"2023-12-19T18:22:14.259355+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:29:20.466956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252592+0000","last_change":"2023-12-19T18:22:14.250414+0000","last_active":"2023-12-19T18:22:14.252592+0000","last_peered":"2023-12-19T18:22:14.252592+0000","last_clean":"2023-12-19T18:22:14.252592+0000","last_became_active":"2023-12-19T18:22:14.249822+0000","last_became_peered":"2023-12-19T18:22:14.249822+0000","last_unstale":"2023-12-19T18:22:14.252592+0000","last_undegraded":"2023-12-19T18:22:14.252592+0000","last_fullsized":"2023-12-19T18:22:14.252592+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:37.118051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.13","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248641+0000","last_change":"2023-12-19T18:22:13.730064+0000","last_active":"2023-12-19T18:22:14.248641+0000","last_peered":"2023-12-19T18:22:14.248641+0000","last_clean":"2023-12-19T18:22:14.248641+0000","last_became_active":"2023-12-19T18:22:13.729780+0000","last_became_peered":"2023-12-19T18:22:13.729780+0000","last_unstale":"2023-12-19T18:22:14.248641+0000","last_undegraded":"2023-12-19T18:22:14.248641+0000","last_fullsized":"2023-12-19T18:22:14.248641+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:19:14.712413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.254201+0000","last_change":"2023-12-19T18:22:14.253925+0000","last_active":"2023-12-19T18:22:14.254201+0000","last_peered":"2023-12-19T18:22:14.254201+0000","last_clean":"2023-12-19T18:22:14.254201+0000","last_became_active":"2023-12-19T18:22:14.253159+0000","last_became_peered":"2023-12-19T18:22:14.253159+0000","last_unstale":"2023-12-19T18:22:14.254201+0000","last_undegraded":"2023-12-19T18:22:14.254201+0000","last_fullsized":"2023-12-19T18:22:14.254201+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:16.254938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246679+0000","last_change":"2023-12-19T18:22:13.732632+0000","last_active":"2023-12-19T18:22:14.246679+0000","last_peered":"2023-12-19T18:22:14.246679+0000","last_clean":"2023-12-19T18:22:14.246679+0000","last_became_active":"2023-12-19T18:22:13.732365+0000","last_became_peered":"2023-12-19T18:22:13.732365+0000","last_unstale":"2023-12-19T18:22:14.246679+0000","last_undegraded":"2023-12-19T18:22:14.246679+0000","last_fullsized":"2023-12-19T18:22:14.246679+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.235724+0000","last_change":"2023-12-19T18:22:14.235442+0000","last_active":"2023-12-19T18:22:14.235724+0000","last_peered":"2023-12-19T18:22:14.235724+0000","last_clean":"2023-12-19T18:22:14.235724+0000","last_became_active":"2023-12-19T18:22:14.235055+0000","last_became_peered":"2023-12-19T18:22:14.235055+0000","last_unstale":"2023-12-19T18:22:14.235724+0000","last_undegraded":"2023-12-19T18:22:14.235724+0000","last_fullsized":"2023-12-19T18:22:14.235724+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:06:47.768050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.229590+0000","last_change":"2023-12-19T18:22:13.729287+0000","last_active":"2023-12-19T18:22:14.229590+0000","last_peered":"2023-12-19T18:22:14.229590+0000","last_clean":"2023-12-19T18:22:14.229590+0000","last_became_active":"2023-12-19T18:22:13.725118+0000","last_became_peered":"2023-12-19T18:22:13.725118+0000","last_unstale":"2023-12-19T18:22:14.229590+0000","last_undegraded":"2023-12-19T18:22:14.229590+0000","last_fullsized":"2023-12-19T18:22:14.229590+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:13:44.939138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.255529+0000","last_change":"2023-12-19T18:22:14.255009+0000","last_active":"2023-12-19T18:22:14.255529+0000","last_peered":"2023-12-19T18:22:14.255529+0000","last_clean":"2023-12-19T18:22:14.255529+0000","last_became_active":"2023-12-19T18:22:14.254396+0000","last_became_peered":"2023-12-19T18:22:14.254396+0000","last_unstale":"2023-12-19T18:22:14.255529+0000","last_undegraded":"2023-12-19T18:22:14.255529+0000","last_fullsized":"2023-12-19T18:22:14.255529+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:53:08.465323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.10","version":"134'8","reported_seq":19,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.263107+0000","last_change":"2023-12-19T18:22:13.734908+0000","last_active":"2023-12-19T18:22:14.263107+0000","last_peered":"2023-12-19T18:22:14.263107+0000","last_clean":"2023-12-19T18:22:14.263107+0000","last_became_active":"2023-12-19T18:22:13.734666+0000","last_became_peered":"2023-12-19T18:22:13.734666+0000","last_unstale":"2023-12-19T18:22:14.263107+0000","last_undegraded":"2023-12-19T18:22:14.263107+0000","last_fullsized":"2023-12-19T18:22:14.263107+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:37:44.539092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.243196+0000","last_change":"2023-12-19T18:22:14.243113+0000","last_active":"2023-12-19T18:22:14.243196+0000","last_peered":"2023-12-19T18:22:14.243196+0000","last_clean":"2023-12-19T18:22:14.243196+0000","last_became_active":"2023-12-19T18:22:14.242834+0000","last_became_peered":"2023-12-19T18:22:14.242834+0000","last_unstale":"2023-12-19T18:22:14.243196+0000","last_undegraded":"2023-12-19T18:22:14.243196+0000","last_fullsized":"2023-12-19T18:22:14.243196+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:12.207051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.e","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.243309+0000","last_change":"2023-12-19T18:22:13.724787+0000","last_active":"2023-12-19T18:22:14.243309+0000","last_peered":"2023-12-19T18:22:14.243309+0000","last_clean":"2023-12-19T18:22:14.243309+0000","last_became_active":"2023-12-19T18:22:13.724547+0000","last_became_peered":"2023-12-19T18:22:13.724547+0000","last_unstale":"2023-12-19T18:22:14.243309+0000","last_undegraded":"2023-12-19T18:22:14.243309+0000","last_fullsized":"2023-12-19T18:22:14.243309+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:11.095769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.243792+0000","last_change":"2023-12-19T18:22:14.243719+0000","last_active":"2023-12-19T18:22:14.243792+0000","last_peered":"2023-12-19T18:22:14.243792+0000","last_clean":"2023-12-19T18:22:14.243792+0000","last_became_active":"2023-12-19T18:22:14.242848+0000","last_became_peered":"2023-12-19T18:22:14.242848+0000","last_unstale":"2023-12-19T18:22:14.243792+0000","last_undegraded":"2023-12-19T18:22:14.243792+0000","last_fullsized":"2023-12-19T18:22:14.243792+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:34.577484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.d","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246567+0000","last_change":"2023-12-19T18:22:13.728901+0000","last_active":"2023-12-19T18:22:14.246567+0000","last_peered":"2023-12-19T18:22:14.246567+0000","last_clean":"2023-12-19T18:22:14.246567+0000","last_became_active":"2023-12-19T18:22:13.728001+0000","last_became_peered":"2023-12-19T18:22:13.728001+0000","last_unstale":"2023-12-19T18:22:14.246567+0000","last_undegraded":"2023-12-19T18:22:14.246567+0000","last_fullsized":"2023-12-19T18:22:14.246567+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:59.426392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246154+0000","last_change":"2023-12-19T18:22:14.246103+0000","last_active":"2023-12-19T18:22:14.246154+0000","last_peered":"2023-12-19T18:22:14.246154+0000","last_clean":"2023-12-19T18:22:14.246154+0000","last_became_active":"2023-12-19T18:22:14.245847+0000","last_became_peered":"2023-12-19T18:22:14.245847+0000","last_unstale":"2023-12-19T18:22:14.246154+0000","last_undegraded":"2023-12-19T18:22:14.246154+0000","last_fullsized":"2023-12-19T18:22:14.246154+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:01.033125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.226401+0000","last_change":"2023-12-19T18:22:13.729021+0000","last_active":"2023-12-19T18:22:14.226401+0000","last_peered":"2023-12-19T18:22:14.226401+0000","last_clean":"2023-12-19T18:22:14.226401+0000","last_became_active":"2023-12-19T18:22:13.728792+0000","last_became_peered":"2023-12-19T18:22:13.728792+0000","last_unstale":"2023-12-19T18:22:14.226401+0000","last_undegraded":"2023-12-19T18:22:14.226401+0000","last_fullsized":"2023-12-19T18:22:14.226401+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:46:39.606223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252679+0000","last_change":"2023-12-19T18:22:14.252307+0000","last_active":"2023-12-19T18:22:14.252679+0000","last_peered":"2023-12-19T18:22:14.252679+0000","last_clean":"2023-12-19T18:22:14.252679+0000","last_became_active":"2023-12-19T18:22:14.251969+0000","last_became_peered":"2023-12-19T18:22:14.251969+0000","last_unstale":"2023-12-19T18:22:14.252679+0000","last_undegraded":"2023-12-19T18:22:14.252679+0000","last_fullsized":"2023-12-19T18:22:14.252679+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:50:35.183653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.b","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.263780+0000","last_change":"2023-12-19T18:22:13.734402+0000","last_active":"2023-12-19T18:22:14.263780+0000","last_peered":"2023-12-19T18:22:14.263780+0000","last_clean":"2023-12-19T18:22:14.263780+0000","last_became_active":"2023-12-19T18:22:13.733935+0000","last_became_peered":"2023-12-19T18:22:13.733935+0000","last_unstale":"2023-12-19T18:22:14.263780+0000","last_undegraded":"2023-12-19T18:22:14.263780+0000","last_fullsized":"2023-12-19T18:22:14.263780+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:21:30.091059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258421+0000","last_change":"2023-12-19T18:22:14.257490+0000","last_active":"2023-12-19T18:22:14.258421+0000","last_peered":"2023-12-19T18:22:14.258421+0000","last_clean":"2023-12-19T18:22:14.258421+0000","last_became_active":"2023-12-19T18:22:14.256714+0000","last_became_peered":"2023-12-19T18:22:14.256714+0000","last_unstale":"2023-12-19T18:22:14.258421+0000","last_undegraded":"2023-12-19T18:22:14.258421+0000","last_fullsized":"2023-12-19T18:22:14.258421+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:29.507646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.238218+0000","last_change":"2023-12-19T18:22:13.730665+0000","last_active":"2023-12-19T18:22:14.238218+0000","last_peered":"2023-12-19T18:22:14.238218+0000","last_clean":"2023-12-19T18:22:14.238218+0000","last_became_active":"2023-12-19T18:22:13.729630+0000","last_became_peered":"2023-12-19T18:22:13.729630+0000","last_unstale":"2023-12-19T18:22:14.238218+0000","last_undegraded":"2023-12-19T18:22:14.238218+0000","last_fullsized":"2023-12-19T18:22:14.238218+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:01:35.189032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.258658+0000","last_change":"2023-12-19T18:22:14.257612+0000","last_active":"2023-12-19T18:22:14.258658+0000","last_peered":"2023-12-19T18:22:14.258658+0000","last_clean":"2023-12-19T18:22:14.258658+0000","last_became_active":"2023-12-19T18:22:14.256820+0000","last_became_peered":"2023-12-19T18:22:14.256820+0000","last_unstale":"2023-12-19T18:22:14.258658+0000","last_undegraded":"2023-12-19T18:22:14.258658+0000","last_fullsized":"2023-12-19T18:22:14.258658+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:12.207051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.231579+0000","last_change":"2023-12-19T18:22:13.731257+0000","last_active":"2023-12-19T18:22:14.231579+0000","last_peered":"2023-12-19T18:22:14.231579+0000","last_clean":"2023-12-19T18:22:14.231579+0000","last_became_active":"2023-12-19T18:22:13.730910+0000","last_became_peered":"2023-12-19T18:22:13.730910+0000","last_unstale":"2023-12-19T18:22:14.231579+0000","last_undegraded":"2023-12-19T18:22:14.231579+0000","last_fullsized":"2023-12-19T18:22:14.231579+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.254075+0000","last_change":"2023-12-19T18:22:14.253991+0000","last_active":"2023-12-19T18:22:14.254075+0000","last_peered":"2023-12-19T18:22:14.254075+0000","last_clean":"2023-12-19T18:22:14.254075+0000","last_became_active":"2023-12-19T18:22:14.253757+0000","last_became_peered":"2023-12-19T18:22:14.253757+0000","last_unstale":"2023-12-19T18:22:14.254075+0000","last_undegraded":"2023-12-19T18:22:14.254075+0000","last_fullsized":"2023-12-19T18:22:14.254075+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:46:44.388621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246101+0000","last_change":"2023-12-19T18:22:13.714716+0000","last_active":"2023-12-19T18:22:14.246101+0000","last_peered":"2023-12-19T18:22:14.246101+0000","last_clean":"2023-12-19T18:22:14.246101+0000","last_became_active":"2023-12-19T18:22:13.713977+0000","last_became_peered":"2023-12-19T18:22:13.713977+0000","last_unstale":"2023-12-19T18:22:14.246101+0000","last_undegraded":"2023-12-19T18:22:14.246101+0000","last_fullsized":"2023-12-19T18:22:14.246101+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:11:39.685571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246372+0000","last_change":"2023-12-19T18:22:14.245059+0000","last_active":"2023-12-19T18:22:14.246372+0000","last_peered":"2023-12-19T18:22:14.246372+0000","last_clean":"2023-12-19T18:22:14.246372+0000","last_became_active":"2023-12-19T18:22:14.244526+0000","last_became_peered":"2023-12-19T18:22:14.244526+0000","last_unstale":"2023-12-19T18:22:14.246372+0000","last_undegraded":"2023-12-19T18:22:14.246372+0000","last_fullsized":"2023-12-19T18:22:14.246372+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:45:23.784185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.7","version":"134'2","reported_seq":13,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.241662+0000","last_change":"2023-12-19T18:22:13.729500+0000","last_active":"2023-12-19T18:22:14.241662+0000","last_peered":"2023-12-19T18:22:14.241662+0000","last_clean":"2023-12-19T18:22:14.241662+0000","last_became_active":"2023-12-19T18:22:13.725715+0000","last_became_peered":"2023-12-19T18:22:13.725715+0000","last_unstale":"2023-12-19T18:22:14.241662+0000","last_undegraded":"2023-12-19T18:22:14.241662+0000","last_fullsized":"2023-12-19T18:22:14.241662+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:30:23.979686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259484+0000","last_change":"2023-12-19T18:22:14.258603+0000","last_active":"2023-12-19T18:22:14.259484+0000","last_peered":"2023-12-19T18:22:14.259484+0000","last_clean":"2023-12-19T18:22:14.259484+0000","last_became_active":"2023-12-19T18:22:14.258065+0000","last_became_peered":"2023-12-19T18:22:14.258065+0000","last_unstale":"2023-12-19T18:22:14.259484+0000","last_undegraded":"2023-12-19T18:22:14.259484+0000","last_fullsized":"2023-12-19T18:22:14.259484+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:43:39.731660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.6","version":"135'1","reported_seq":12,"reported_epoch":135,"state":"active+clean","last_fresh":"2023-12-19T18:22:15.295280+0000","last_change":"2023-12-19T18:22:13.726768+0000","last_active":"2023-12-19T18:22:15.295280+0000","last_peered":"2023-12-19T18:22:15.295280+0000","last_clean":"2023-12-19T18:22:15.295280+0000","last_became_active":"2023-12-19T18:22:13.722319+0000","last_became_peered":"2023-12-19T18:22:13.722319+0000","last_unstale":"2023-12-19T18:22:15.295280+0000","last_undegraded":"2023-12-19T18:22:15.295280+0000","last_fullsized":"2023-12-19T18:22:15.295280+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:05:11.192516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.255546+0000","last_change":"2023-12-19T18:22:14.255042+0000","last_active":"2023-12-19T18:22:14.255546+0000","last_peered":"2023-12-19T18:22:14.255546+0000","last_clean":"2023-12-19T18:22:14.255546+0000","last_became_active":"2023-12-19T18:22:14.253623+0000","last_became_peered":"2023-12-19T18:22:14.253623+0000","last_unstale":"2023-12-19T18:22:14.255546+0000","last_undegraded":"2023-12-19T18:22:14.255546+0000","last_fullsized":"2023-12-19T18:22:14.255546+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:09.942336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.227165+0000","last_change":"2023-12-19T18:22:13.713351+0000","last_active":"2023-12-19T18:22:14.227165+0000","last_peered":"2023-12-19T18:22:14.227165+0000","last_clean":"2023-12-19T18:22:14.227165+0000","last_became_active":"2023-12-19T18:22:13.712581+0000","last_became_peered":"2023-12-19T18:22:13.712581+0000","last_unstale":"2023-12-19T18:22:14.227165+0000","last_undegraded":"2023-12-19T18:22:14.227165+0000","last_fullsized":"2023-12-19T18:22:14.227165+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:19:36.103516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.242422+0000","last_change":"2023-12-19T18:22:14.241466+0000","last_active":"2023-12-19T18:22:14.242422+0000","last_peered":"2023-12-19T18:22:14.242422+0000","last_clean":"2023-12-19T18:22:14.242422+0000","last_became_active":"2023-12-19T18:22:14.241205+0000","last_became_peered":"2023-12-19T18:22:14.241205+0000","last_unstale":"2023-12-19T18:22:14.242422+0000","last_undegraded":"2023-12-19T18:22:14.242422+0000","last_fullsized":"2023-12-19T18:22:14.242422+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:08.710594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239506+0000","last_change":"2023-12-19T18:22:13.718488+0000","last_active":"2023-12-19T18:22:14.239506+0000","last_peered":"2023-12-19T18:22:14.239506+0000","last_clean":"2023-12-19T18:22:14.239506+0000","last_became_active":"2023-12-19T18:22:13.717351+0000","last_became_peered":"2023-12-19T18:22:13.717351+0000","last_unstale":"2023-12-19T18:22:14.239506+0000","last_undegraded":"2023-12-19T18:22:14.239506+0000","last_fullsized":"2023-12-19T18:22:14.239506+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:49:11.599052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248088+0000","last_change":"2023-12-19T18:22:14.247933+0000","last_active":"2023-12-19T18:22:14.248088+0000","last_peered":"2023-12-19T18:22:14.248088+0000","last_clean":"2023-12-19T18:22:14.248088+0000","last_became_active":"2023-12-19T18:22:14.247461+0000","last_became_peered":"2023-12-19T18:22:14.247461+0000","last_unstale":"2023-12-19T18:22:14.248088+0000","last_undegraded":"2023-12-19T18:22:14.248088+0000","last_fullsized":"2023-12-19T18:22:14.248088+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:24.011514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246268+0000","last_change":"2023-12-19T18:22:13.733542+0000","last_active":"2023-12-19T18:22:14.246268+0000","last_peered":"2023-12-19T18:22:14.246268+0000","last_clean":"2023-12-19T18:22:14.246268+0000","last_became_active":"2023-12-19T18:22:13.733273+0000","last_became_peered":"2023-12-19T18:22:13.733273+0000","last_unstale":"2023-12-19T18:22:14.246268+0000","last_undegraded":"2023-12-19T18:22:14.246268+0000","last_fullsized":"2023-12-19T18:22:14.246268+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:07.352249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.252415+0000","last_change":"2023-12-19T18:22:14.252315+0000","last_active":"2023-12-19T18:22:14.252415+0000","last_peered":"2023-12-19T18:22:14.252415+0000","last_clean":"2023-12-19T18:22:14.252415+0000","last_became_active":"2023-12-19T18:22:14.252108+0000","last_became_peered":"2023-12-19T18:22:14.252108+0000","last_unstale":"2023-12-19T18:22:14.252415+0000","last_undegraded":"2023-12-19T18:22:14.252415+0000","last_fullsized":"2023-12-19T18:22:14.252415+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:08.470720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.239128+0000","last_change":"2023-12-19T18:22:13.723012+0000","last_active":"2023-12-19T18:22:14.239128+0000","last_peered":"2023-12-19T18:22:14.239128+0000","last_clean":"2023-12-19T18:22:14.239128+0000","last_became_active":"2023-12-19T18:22:13.717308+0000","last_became_peered":"2023-12-19T18:22:13.717308+0000","last_unstale":"2023-12-19T18:22:14.239128+0000","last_undegraded":"2023-12-19T18:22:14.239128+0000","last_fullsized":"2023-12-19T18:22:14.239128+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:17:01.277914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.246337+0000","last_change":"2023-12-19T18:22:14.244959+0000","last_active":"2023-12-19T18:22:14.246337+0000","last_peered":"2023-12-19T18:22:14.246337+0000","last_clean":"2023-12-19T18:22:14.246337+0000","last_became_active":"2023-12-19T18:22:14.238592+0000","last_became_peered":"2023-12-19T18:22:14.238592+0000","last_unstale":"2023-12-19T18:22:14.246337+0000","last_undegraded":"2023-12-19T18:22:14.246337+0000","last_fullsized":"2023-12-19T18:22:14.246337+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:30:01.499864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.232648+0000","last_change":"2023-12-19T18:22:13.736355+0000","last_active":"2023-12-19T18:22:14.232648+0000","last_peered":"2023-12-19T18:22:14.232648+0000","last_clean":"2023-12-19T18:22:14.232648+0000","last_became_active":"2023-12-19T18:22:13.736056+0000","last_became_peered":"2023-12-19T18:22:13.736056+0000","last_unstale":"2023-12-19T18:22:14.232648+0000","last_undegraded":"2023-12-19T18:22:14.232648+0000","last_fullsized":"2023-12-19T18:22:14.232648+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:47.766382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.255816+0000","last_change":"2023-12-19T18:22:14.255696+0000","last_active":"2023-12-19T18:22:14.255816+0000","last_peered":"2023-12-19T18:22:14.255816+0000","last_clean":"2023-12-19T18:22:14.255816+0000","last_became_active":"2023-12-19T18:22:14.255205+0000","last_became_peered":"2023-12-19T18:22:14.255205+0000","last_unstale":"2023-12-19T18:22:14.255816+0000","last_undegraded":"2023-12-19T18:22:14.255816+0000","last_fullsized":"2023-12-19T18:22:14.255816+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:59:08.366785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":11,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.248750+0000","last_change":"2023-12-19T18:22:13.715051+0000","last_active":"2023-12-19T18:22:14.248750+0000","last_peered":"2023-12-19T18:22:14.248750+0000","last_clean":"2023-12-19T18:22:14.248750+0000","last_became_active":"2023-12-19T18:22:13.714392+0000","last_became_peered":"2023-12-19T18:22:13.714392+0000","last_unstale":"2023-12-19T18:22:14.248750+0000","last_undegraded":"2023-12-19T18:22:14.248750+0000","last_fullsized":"2023-12-19T18:22:14.248750+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:19:00.018590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":10,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.259379+0000","last_change":"2023-12-19T18:22:14.259186+0000","last_active":"2023-12-19T18:22:14.259379+0000","last_peered":"2023-12-19T18:22:14.259379+0000","last_clean":"2023-12-19T18:22:14.259379+0000","last_became_active":"2023-12-19T18:22:14.258862+0000","last_became_peered":"2023-12-19T18:22:14.258862+0000","last_unstale":"2023-12-19T18:22:14.259379+0000","last_undegraded":"2023-12-19T18:22:14.259379+0000","last_fullsized":"2023-12-19T18:22:14.259379+0000","mapping_epoch":133,"log_start":"0'0","ondisk_log_start":"0'0","created":133,"last_epoch_clean":134,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:13.691897+0000","last_clean_scrub_stamp":"2023-12-19T18:22:13.691897+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:00:16.089328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.f","version":"134'1","reported_seq":12,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.247514+0000","last_change":"2023-12-19T18:22:13.725717+0000","last_active":"2023-12-19T18:22:14.247514+0000","last_peered":"2023-12-19T18:22:14.247514+0000","last_clean":"2023-12-19T18:22:14.247514+0000","last_became_active":"2023-12-19T18:22:13.724435+0000","last_became_peered":"2023-12-19T18:22:13.724435+0000","last_unstale":"2023-12-19T18:22:14.247514+0000","last_undegraded":"2023-12-19T18:22:14.247514+0000","last_fullsized":"2023-12-19T18:22:14.247514+0000","mapping_epoch":132,"log_start":"0'0","ondisk_log_start":"0'0","created":132,"last_epoch_clean":133,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:22:12.677362+0000","last_clean_scrub_stamp":"2023-12-19T18:22:12.677362+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:00:05.094962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":267,"reported_epoch":134,"state":"active+clean","last_fresh":"2023-12-19T18:22:14.229283+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:22:14.229283+0000","last_peered":"2023-12-19T18:22:14.229283+0000","last_clean":"2023-12-19T18:22:14.229283+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:22:14.229283+0000","last_undegraded":"2023-12-19T18:22:14.229283+0000","last_fullsized":"2023-12-19T18:22:14.229283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574942,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":736,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199245902,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.2310000000000001,"15min":1.1519999999999999},"min":{"1min":0.57899999999999996,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.9630000000000001,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":1.161},{"interface":"front","average":{"1min":1.272,"5min":1.278,"15min":1.1890000000000001},"min":{"1min":0.76900000000000002,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.6990000000000001,"5min":1.79,"15min":1.79},"last":1.254}]},{"osd":1,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.1399999999999999,"5min":1.3480000000000001,"15min":1.23},"min":{"1min":0.752,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.7010000000000001,"5min":10.926,"15min":10.926},"last":1.3799999999999999},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.365,"15min":1.244},"min":{"1min":0.80900000000000005,"5min":0.71899999999999997,"15min":0.67900000000000005},"max":{"1min":1.6240000000000001,"5min":10.904999999999999,"15min":10.904999999999999},"last":1.411}]},{"osd":2,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.0369999999999999,"5min":1.161,"15min":1.1040000000000001},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.427,"5min":3.1960000000000002,"15min":3.1960000000000002},"last":1.29},{"interface":"front","average":{"1min":1.24,"5min":1.266,"15min":1.1930000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0670000000000002,"5min":6.008,"15min":6.008},"last":0.88500000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.266,"5min":1.28,"15min":1.1619999999999999},"min":{"1min":0.66900000000000004,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.595,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":1.077},{"interface":"front","average":{"1min":1.4239999999999999,"5min":1.4179999999999999,"15min":1.294},"min":{"1min":0.85699999999999998,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":2.157,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":1.3080000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.258,"15min":1.123},"min":{"1min":0.88600000000000001,"5min":0.65400000000000003,"15min":0.47899999999999998},"max":{"1min":1.8009999999999999,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.222,"5min":1.304,"15min":1.2410000000000001},"min":{"1min":0.94599999999999995,"5min":0.72999999999999998,"15min":0.57599999999999996},"max":{"1min":1.6759999999999999,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.2070000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.399,"15min":1.333},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.55000000000000004},"max":{"1min":1.8620000000000001,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":0.99299999999999999},{"interface":"front","average":{"1min":1.385,"5min":1.4339999999999999,"15min":1.337},"min":{"1min":0.80700000000000005,"5min":0.71599999999999997,"15min":0.53500000000000003},"max":{"1min":2.1179999999999999,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":1.5629999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:21:58 2023","interfaces":[{"interface":"back","average":{"1min":1.585,"5min":1.5669999999999999,"15min":1.4690000000000001},"min":{"1min":1.036,"5min":0.78600000000000003,"15min":0.59899999999999998},"max":{"1min":2.423,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.476},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.49,"15min":1.3799999999999999},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.63200000000000001},"max":{"1min":2.0179999999999998,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":1.6850000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574942,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":735,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199246484,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2509999999999999,"5min":1.121,"15min":1.012},"min":{"1min":0.79800000000000004,"5min":0.66700000000000004,"15min":0.65800000000000003},"max":{"1min":1.6259999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":1.3260000000000001},{"interface":"front","average":{"1min":1.163,"5min":1.1870000000000001,"15min":1.075},"min":{"1min":0.78400000000000003,"5min":0.64300000000000002,"15min":0.55600000000000005},"max":{"1min":1.663,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":1.2010000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.1699999999999999,"15min":1.1319999999999999},"min":{"1min":0.86599999999999999,"5min":0.71399999999999997,"15min":0.68200000000000005},"max":{"1min":1.5609999999999999,"5min":4.5540000000000003,"15min":4.5540000000000003},"last":1.532},{"interface":"front","average":{"1min":1.214,"5min":1.226,"15min":1.1699999999999999},"min":{"1min":0.86599999999999999,"5min":0.625,"15min":0.60299999999999998},"max":{"1min":1.627,"5min":1.885,"15min":1.885},"last":1.1619999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.254,"15min":1.151},"min":{"1min":0.84799999999999998,"5min":0.626,"15min":0.626},"max":{"1min":1.583,"5min":1.821,"15min":1.821},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.171,"5min":1.218,"15min":1.2310000000000001},"min":{"1min":0.83599999999999997,"5min":0.81999999999999995,"15min":0.79900000000000004},"max":{"1min":1.821,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":1.4950000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.294,"15min":1.202},"min":{"1min":1.0129999999999999,"5min":0.81599999999999995,"15min":0.81599999999999995},"max":{"1min":1.722,"5min":3.8820000000000001,"15min":3.8820000000000001},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2170000000000001,"15min":1.1699999999999999},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6559999999999999,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":1.4139999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.296,"15min":1.2450000000000001},"min":{"1min":0.74099999999999999,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.6299999999999999,"5min":3.0680000000000001,"15min":3.0680000000000001},"last":1.522},{"interface":"front","average":{"1min":1.3959999999999999,"5min":1.431,"15min":1.347},"min":{"1min":0.76700000000000002,"5min":0.70799999999999996,"15min":0.57999999999999996},"max":{"1min":1.774,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.587}]},{"osd":5,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.2829999999999999,"15min":1.1879999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.7330000000000001,"5min":4.6680000000000001,"15min":4.6680000000000001},"last":1.107},{"interface":"front","average":{"1min":1.5840000000000001,"5min":1.4450000000000001,"15min":1.3560000000000001},"min":{"1min":0.84799999999999998,"5min":0.626,"15min":0.626},"max":{"1min":4.1159999999999997,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.4510000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:21:56 2023","interfaces":[{"interface":"back","average":{"1min":1.415,"5min":1.4430000000000001,"15min":1.349},"min":{"1min":0.92700000000000005,"5min":0.82199999999999995,"15min":0.61899999999999999},"max":{"1min":1.7010000000000001,"5min":3.9670000000000001,"15min":3.9670000000000001},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.393,"5min":1.4470000000000001,"15min":1.357},"min":{"1min":0.81899999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.8169999999999999,"5min":5.6639999999999997,"15min":5.6639999999999997},"last":1.2509999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574942,"num_pgs":30,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1313,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193998,"statfs":{"total":200000000,"available":198654496,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.23,"15min":1.1679999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.65200000000000002},"max":{"1min":1.6830000000000001,"5min":1.837,"15min":1.863},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.2729999999999999,"15min":1.1870000000000001},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.571,"5min":1.879,"15min":1.879},"last":0.69399999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2090000000000001,"15min":1.0629999999999999},"min":{"1min":0.76200000000000001,"5min":0.70899999999999996,"15min":0.65700000000000003},"max":{"1min":1.782,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.1759999999999999,"5min":1.1599999999999999,"15min":1.0660000000000001},"min":{"1min":0.81899999999999995,"5min":0.71299999999999997,"15min":0.61299999999999999},"max":{"1min":1.5640000000000001,"5min":1.6359999999999999,"15min":1.843},"last":1.339}]},{"osd":2,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.155,"15min":1.0860000000000001},"min":{"1min":0.81799999999999995,"5min":0.74299999999999999,"15min":0.59499999999999997},"max":{"1min":1.3069999999999999,"5min":1.794,"15min":1.794},"last":1.127},{"interface":"front","average":{"1min":1.2350000000000001,"5min":1.258,"15min":1.147},"min":{"1min":0.82599999999999996,"5min":0.78400000000000003,"15min":0.65200000000000002},"max":{"1min":1.5920000000000001,"5min":2.036,"15min":2.036},"last":1.2729999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2010000000000001,"15min":1.097},"min":{"1min":0.83699999999999997,"5min":0.71999999999999997,"15min":0.68899999999999995},"max":{"1min":1.72,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.21},{"interface":"front","average":{"1min":1.2809999999999999,"5min":1.3320000000000001,"15min":1.23},"min":{"1min":0.91300000000000003,"5min":0.75700000000000001,"15min":0.70399999999999996},"max":{"1min":1.595,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.484}]},{"osd":4,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3420000000000001,"5min":1.3069999999999999,"15min":1.179},"min":{"1min":0.95299999999999996,"5min":0.60899999999999999,"15min":0.46700000000000003},"max":{"1min":1.6479999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.409},{"interface":"front","average":{"1min":1.25,"5min":1.1990000000000001,"15min":1.131},"min":{"1min":0.90500000000000003,"5min":0.73999999999999999,"15min":0.57099999999999995},"max":{"1min":1.7689999999999999,"5min":1.804,"15min":1.901},"last":1.0800000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.429,"5min":1.4299999999999999,"15min":1.3360000000000001},"min":{"1min":1.054,"5min":0.66600000000000004,"15min":0.54400000000000004},"max":{"1min":1.839,"5min":2.1589999999999998,"15min":2.1589999999999998},"last":1.5589999999999999},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.319,"15min":1.2470000000000001},"min":{"1min":1.163,"5min":0.81200000000000006,"15min":0.49299999999999999},"max":{"1min":1.702,"5min":1.7949999999999999,"15min":1.976},"last":1.381}]},{"osd":7,"last update":"Tue Dec 19 18:21:57 2023","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.274,"15min":1.2290000000000001},"min":{"1min":1.0189999999999999,"5min":0.68300000000000005,"15min":0.50900000000000001},"max":{"1min":1.6779999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.528},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4570000000000001,"15min":1.4119999999999999},"min":{"1min":1.2310000000000001,"5min":0.84999999999999998,"15min":0.623},"max":{"1min":1.8120000000000001,"5min":2.117,"15min":2.117},"last":1.4430000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574942,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":736,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199245902,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":0.98899999999999999,"15min":0.998},"min":{"1min":0.68500000000000005,"5min":0.62,"15min":0.62},"max":{"1min":1.3899999999999999,"5min":1.579,"15min":1.579},"last":1.0189999999999999},{"interface":"front","average":{"1min":1.0600000000000001,"5min":1.0409999999999999,"15min":1.034},"min":{"1min":0.80000000000000004,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.4259999999999999,"5min":1.7470000000000001,"15min":1.7909999999999999},"last":0.82299999999999995}]},{"osd":1,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1679999999999999,"15min":1.149},"min":{"1min":0.67200000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.589,"5min":1.7130000000000001,"15min":1.8160000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.29,"5min":1.2549999999999999,"15min":1.1850000000000001},"min":{"1min":0.77400000000000002,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.696},"last":1.141}]},{"osd":2,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.173,"5min":1.1499999999999999,"15min":1.127},"min":{"1min":0.82399999999999995,"5min":0.749,"15min":0.69799999999999995},"max":{"1min":1.407,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.167,"5min":1.167,"15min":1.1060000000000001},"min":{"1min":0.872,"5min":0.72199999999999998,"15min":0.70499999999999996},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.3879999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.321,"5min":1.294,"15min":1.26},"min":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3340000000000001,"15min":1.304},"min":{"1min":0.93100000000000005,"5min":0.82399999999999995,"15min":0.74399999999999999},"max":{"1min":1.8420000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.28}]},{"osd":5,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.294,"15min":1.2030000000000001},"min":{"1min":0.90000000000000002,"5min":0.68600000000000005,"15min":0.44500000000000001},"max":{"1min":1.46,"5min":1.76,"15min":1.8740000000000001},"last":1.508},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.413,"15min":1.353},"min":{"1min":0.76500000000000001,"5min":0.625,"15min":0.504},"max":{"1min":1.827,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.302}]},{"osd":6,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.3160000000000001,"15min":1.304},"min":{"1min":0.98399999999999999,"5min":0.71299999999999997,"15min":0.432},"max":{"1min":1.5509999999999999,"5min":1.7809999999999999,"15min":1.853},"last":1.347},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.4630000000000001,"15min":1.3959999999999999},"min":{"1min":1.2170000000000001,"5min":0.73599999999999999,"15min":0.64500000000000002},"max":{"1min":1.7829999999999999,"5min":1.8160000000000001,"15min":1.8160000000000001},"last":1.599}]},{"osd":7,"last update":"Tue Dec 19 18:21:59 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.3999999999999999,"15min":1.3480000000000001},"min":{"1min":1.0249999999999999,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":1.7589999999999999,"5min":1.9910000000000001,"15min":1.9910000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.4510000000000001,"5min":1.4379999999999999,"15min":1.4079999999999999},"min":{"1min":0.88500000000000001,"5min":0.60399999999999998,"15min":0.57399999999999995},"max":{"1min":1.8,"5min":1.905,"15min":1.9330000000000001},"last":1.466}]}]},{"osd":3,"up_from":13,"seq":55834574943,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":736,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199246290,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":0.94199999999999995,"5min":0.94399999999999995,"15min":0.96399999999999997},"min":{"1min":0.52900000000000003,"5min":0.38100000000000001,"15min":0.35299999999999998},"max":{"1min":2.516,"5min":3.3140000000000001,"15min":3.4350000000000001},"last":1.181},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1,"15min":0.94499999999999995},"min":{"1min":0.438,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":4.3490000000000002,"5min":4.3490000000000002,"15min":4.3490000000000002},"last":1.655}]},{"osd":1,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":1.0629999999999999,"15min":1.0620000000000001},"min":{"1min":0.45300000000000001,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":5.2510000000000003,"5min":5.2510000000000003,"15min":5.2510000000000003},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.403,"5min":1.1719999999999999,"15min":1.1160000000000001},"min":{"1min":0.63800000000000001,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":5.4119999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.4750000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.2050000000000001,"5min":1.377,"15min":1.25},"min":{"1min":0.502,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":2.335,"5min":13.833,"15min":13.833},"last":1.3500000000000001},{"interface":"front","average":{"1min":1.5429999999999999,"5min":1.5349999999999999,"15min":1.306},"min":{"1min":0.60299999999999998,"5min":0.57999999999999996,"15min":0.53200000000000003},"max":{"1min":5.4480000000000004,"5min":16.748999999999999,"15min":16.748999999999999},"last":1.7509999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4570000000000001,"5min":1.3620000000000001,"15min":1.2330000000000001},"min":{"1min":0.81899999999999995,"5min":0.77400000000000002,"15min":0.65600000000000003},"max":{"1min":5.1790000000000003,"5min":5.1790000000000003,"15min":5.1790000000000003},"last":1.415},{"interface":"front","average":{"1min":1.591,"5min":1.4239999999999999,"15min":1.409},"min":{"1min":0.873,"5min":0.80800000000000005,"15min":0.73199999999999998},"max":{"1min":4.569,"5min":4.569,"15min":4.569},"last":1.798}]},{"osd":5,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.595,"5min":1.4750000000000001,"15min":1.45},"min":{"1min":0.96199999999999997,"5min":0.82399999999999995,"15min":0.76000000000000001},"max":{"1min":5.3289999999999997,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":1.6020000000000001},{"interface":"front","average":{"1min":1.46,"5min":1.367,"15min":1.2789999999999999},"min":{"1min":0.83799999999999997,"5min":0.80600000000000005,"15min":0.69999999999999996},"max":{"1min":4.6369999999999996,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.5649999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.5620000000000001,"5min":1.4630000000000001,"15min":1.476},"min":{"1min":0.81999999999999995,"5min":0.81799999999999995,"15min":0.81599999999999995},"max":{"1min":4.5039999999999996,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":1.6919999999999999},{"interface":"front","average":{"1min":1.667,"5min":1.548,"15min":1.514},"min":{"1min":0.83999999999999997,"5min":0.83199999999999996,"15min":0.80500000000000005},"max":{"1min":5.3739999999999997,"5min":5.3739999999999997,"15min":5.3739999999999997},"last":1.879}]},{"osd":7,"last update":"Tue Dec 19 18:21:52 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.476,"15min":1.385},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.73199999999999998},"max":{"1min":5.1349999999999998,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.8440000000000001},{"interface":"front","average":{"1min":1.712,"5min":1.585,"15min":1.554},"min":{"1min":0.92200000000000004,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":4.4619999999999997,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":1.9279999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574942,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":735,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199246414,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":1.075,"15min":1.0109999999999999},"min":{"1min":0.42399999999999999,"5min":0.35299999999999998,"15min":0.34399999999999997},"max":{"1min":1.506,"5min":9.2550000000000008,"15min":9.2550000000000008},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.97999999999999998,"5min":1.0409999999999999,"15min":1.0389999999999999},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.7649999999999999,"5min":2.387,"15min":4.7850000000000001},"last":0.93100000000000005}]},{"osd":1,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.95999999999999996,"15min":1.165},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.46000000000000002},"max":{"1min":1.675,"5min":2.4390000000000001,"15min":8.2720000000000002},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.1040000000000001,"5min":1.107,"15min":1.1619999999999999},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6519999999999999,"5min":2.7200000000000002,"15min":5.0620000000000003},"last":1.2370000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1830000000000001,"15min":1.3620000000000001},"min":{"1min":0.63300000000000001,"5min":0.54200000000000004,"15min":0.51900000000000002},"max":{"1min":1.708,"5min":2.238,"15min":8.3109999999999999},"last":1.107},{"interface":"front","average":{"1min":1.1200000000000001,"5min":1.163,"15min":1.327},"min":{"1min":0.69599999999999995,"5min":0.46300000000000002,"15min":0.41799999999999998},"max":{"1min":1.73,"5min":2.323,"15min":8.0690000000000008},"last":1}]},{"osd":4,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.417,"15min":1.4610000000000001},"min":{"1min":0.93000000000000005,"5min":0.83399999999999996,"15min":0.79300000000000004},"max":{"1min":1.8009999999999999,"5min":2.6139999999999999,"15min":5.3929999999999998},"last":1.4390000000000001},{"interface":"front","average":{"1min":1.5089999999999999,"5min":1.468,"15min":1.5489999999999999},"min":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.78300000000000003},"max":{"1min":2.1949999999999998,"5min":3.1070000000000002,"15min":7.3570000000000002},"last":1.373}]},{"osd":5,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.419,"5min":1.3939999999999999,"15min":1.546},"min":{"1min":0.93200000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.9199999999999999,"5min":2.5579999999999998,"15min":8.1839999999999993},"last":1.208},{"interface":"front","average":{"1min":1.395,"5min":1.423,"15min":1.417},"min":{"1min":0.99299999999999999,"5min":0.81000000000000005,"15min":0.72199999999999998},"max":{"1min":1.845,"5min":2.4940000000000002,"15min":5.165},"last":1.286}]},{"osd":6,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4710000000000001,"5min":1.4350000000000001,"15min":1.454},"min":{"1min":0.82399999999999995,"5min":0.752,"15min":0.72999999999999998},"max":{"1min":2.5139999999999998,"5min":3.0179999999999998,"15min":5.1059999999999999},"last":1.532},{"interface":"front","average":{"1min":1.446,"5min":1.4650000000000001,"15min":1.5589999999999999},"min":{"1min":1.0049999999999999,"5min":0.81200000000000006,"15min":0.77000000000000002},"max":{"1min":1.879,"5min":2.9580000000000002,"15min":8.1370000000000005},"last":1.351}]},{"osd":7,"last update":"Tue Dec 19 18:21:53 2023","interfaces":[{"interface":"back","average":{"1min":1.4830000000000001,"5min":1.5149999999999999,"15min":1.552},"min":{"1min":0.98499999999999999,"5min":0.79200000000000004,"15min":0.78200000000000003},"max":{"1min":2.3700000000000001,"5min":3.2189999999999999,"15min":5.1449999999999996},"last":1.48},{"interface":"front","average":{"1min":1.696,"5min":1.548,"15min":1.659},"min":{"1min":1.054,"5min":0.86499999999999999,"15min":0.86499999999999999},"max":{"1min":4.8289999999999997,"5min":4.8289999999999997,"15min":8.2300000000000004},"last":1.496}]}]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1314,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193998,"statfs":{"total":200000000,"available":198654440,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.84499999999999997,"15min":0.84599999999999997},"min":{"1min":0.42499999999999999,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.3660000000000001,"5min":1.52,"15min":1.52},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.95299999999999996,"15min":0.96099999999999997},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.7929999999999999,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.46899999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.0960000000000001,"5min":1.0720000000000001,"15min":1.0609999999999999},"min":{"1min":0.52500000000000002,"5min":0.42299999999999999,"15min":0.41399999999999998},"max":{"1min":2.4590000000000001,"5min":2.4590000000000001,"15min":2.5739999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.026,"15min":0.93600000000000005},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":1.6839999999999999,"5min":1.845,"15min":1.845},"last":0.52200000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3049999999999999,"15min":1.1719999999999999},"min":{"1min":0.64200000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":2.1040000000000001,"5min":2.137,"15min":2.137},"last":0.59599999999999997},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.2170000000000001,"15min":1.1930000000000001},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.44900000000000001},"max":{"1min":2.1320000000000001,"5min":2.1320000000000001,"15min":2.3959999999999999},"last":0.68300000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.3520000000000001,"15min":1.2989999999999999},"min":{"1min":0.83999999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1440000000000001,"5min":2.444,"15min":2.444},"last":1.1639999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4039999999999999,"15min":1.3100000000000001},"min":{"1min":0.95299999999999996,"5min":0.76900000000000002,"15min":0.71099999999999997},"max":{"1min":1.7569999999999999,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":0.96499999999999997}]},{"osd":5,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3779999999999999,"5min":1.425,"15min":1.3100000000000001},"min":{"1min":0.92800000000000005,"5min":0.88,"15min":0.88},"max":{"1min":1.915,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":0.76000000000000001},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.5049999999999999,"15min":1.423},"min":{"1min":1.0009999999999999,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":2.371,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.107}]},{"osd":6,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.3660000000000001,"5min":1.4530000000000001,"15min":1.3899999999999999},"min":{"1min":0.88400000000000001,"5min":0.875,"15min":0.76900000000000002},"max":{"1min":2.3610000000000002,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.4910000000000001,"5min":1.538,"15min":1.46},"min":{"1min":1.032,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":2.2719999999999998,"5min":2.895,"15min":2.895},"last":1.014}]},{"osd":7,"last update":"Tue Dec 19 18:21:55 2023","interfaces":[{"interface":"back","average":{"1min":1.405,"5min":1.571,"15min":1.397},"min":{"1min":1.018,"5min":0.879,"15min":0.69999999999999996},"max":{"1min":1.8129999999999999,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":1.002},{"interface":"front","average":{"1min":1.468,"5min":1.659,"15min":1.508},"min":{"1min":0.97399999999999998,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.2360000000000002,"5min":9.4109999999999996,"15min":9.4109999999999996},"last":0.98099999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574942,"num_pgs":12,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":735,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194576,"statfs":{"total":200000000,"available":199246484,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.97799999999999998,"5min":0.99399999999999999,"15min":0.90600000000000003},"min":{"1min":0.64000000000000001,"5min":0.46700000000000003,"15min":0.39900000000000002},"max":{"1min":1.3220000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0780000000000001,"5min":1.143,"15min":1.0429999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.0369999999999999,"5min":2.1040000000000001,"15min":2.4180000000000001},"last":1.036}]},{"osd":2,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":0.997,"5min":1.0660000000000001,"15min":0.96299999999999997},"min":{"1min":0.73099999999999998,"5min":0.47799999999999998,"15min":0.45600000000000002},"max":{"1min":1.4810000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.96199999999999997,"5min":1.0389999999999999,"15min":0.94699999999999995},"min":{"1min":0.55100000000000005,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.335,"5min":2.7810000000000001,"15min":2.7810000000000001},"last":0.88300000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.0880000000000001,"15min":1.0209999999999999},"min":{"1min":0.61299999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.732,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.306,"5min":1.3580000000000001,"15min":1.2390000000000001},"min":{"1min":0.746,"5min":0.63100000000000001,"15min":0.47499999999999998},"max":{"1min":1.982,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":1.1439999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.335,"15min":1.216},"min":{"1min":0.86299999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.6739999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.343,"5min":1.3360000000000001,"15min":1.2929999999999999},"min":{"1min":1.0409999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"max":{"1min":1.663,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":1.2529999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.5489999999999999,"15min":1.4059999999999999},"min":{"1min":1.0169999999999999,"5min":0.71099999999999997,"15min":0.64500000000000002},"max":{"1min":1.877,"5min":16.298999999999999,"15min":16.298999999999999},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.385,"15min":1.2909999999999999},"min":{"1min":0.94899999999999995,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.8109999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.3160000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.478,"5min":1.5029999999999999,"15min":1.409},"min":{"1min":1.171,"5min":0.85499999999999998,"15min":0.75900000000000001},"max":{"1min":1.9119999999999999,"5min":2.2559999999999998,"15min":2.5910000000000002},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.391,"5min":1.462,"15min":1.369},"min":{"1min":1.0349999999999999,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.776,"5min":2.46,"15min":2.46},"last":1.224}]},{"osd":7,"last update":"Tue Dec 19 18:22:01 2023","interfaces":[{"interface":"back","average":{"1min":1.5049999999999999,"5min":1.5369999999999999,"15min":1.409},"min":{"1min":1.1279999999999999,"5min":0.89200000000000002,"15min":0.70899999999999996},"max":{"1min":1.8740000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.4970000000000001},{"interface":"front","average":{"1min":1.4630000000000001,"5min":1.466,"15min":1.4330000000000001},"min":{"1min":0.88400000000000001,"5min":0.873,"15min":0.81999999999999995},"max":{"1min":2.113,"5min":2.4580000000000002,"15min":3.1429999999999998},"last":1.3}]}]}],"pool_statfs":[]}} 2023-12-19T18:22:24.889 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:22:24.889 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:22:25.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.174+0000 7f804b558640 1 Processor -- start 2023-12-19T18:22:25.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.174+0000 7f804b558640 1 -- start start 2023-12-19T18:22:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.176+0000 7f804b558640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8044070650 0x7f804415a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.176+0000 7f804b558640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80440ccba0 con 0x7f804406c030 2023-12-19T18:22:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.176+0000 7f804b558640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f804415b070 con 0x7f8044070ac0 2023-12-19T18:22:25.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.176+0000 7f804b558640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f804406f960 con 0x7f8044070650 2023-12-19T18:22:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.176+0000 7f804a556640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f804406c030 0x7f804406c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37686/0 (socket says 172.21.15.27:37686) 2023-12-19T18:22:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.176+0000 7f8049d55640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f8044070650 0x7f804415a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.176+0000 7f804a556640 1 -- 172.21.15.27:0/2223952988 learned_addr learned my addr 172.21.15.27:0/2223952988 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.177+0000 7f8049d55640 1 -- 172.21.15.27:0/2223952988 >> v1:172.21.15.192:6789/0 conn(0x7f8044070ac0 legacy=0x7f80440cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.177+0000 7f8049d55640 1 -- 172.21.15.27:0/2223952988 >> v1:172.21.15.27:6789/0 conn(0x7f804406c030 legacy=0x7f804406c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:25.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.177+0000 7f8049d55640 1 -- 172.21.15.27:0/2223952988 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80440701c0 con 0x7f8044070650 2023-12-19T18:22:25.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.178+0000 7f8049d55640 1 --2- 172.21.15.27:0/2223952988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8044070650 0x7f804415a140 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f80340097a0 tx=0x7f803402da50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28bdbda05ee29efc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:25.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.178+0000 7f8049554640 1 -- 172.21.15.27:0/2223952988 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034031870 con 0x7f8044070650 2023-12-19T18:22:25.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.178+0000 7f8049554640 1 -- 172.21.15.27:0/2223952988 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80340319d0 con 0x7f8044070650 2023-12-19T18:22:25.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.179+0000 7f8049554640 1 -- 172.21.15.27:0/2223952988 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034031d60 con 0x7f8044070650 2023-12-19T18:22:25.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.179+0000 7f804b558640 1 -- 172.21.15.27:0/2223952988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8044070650 msgr2=0x7f804415a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:25.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.179+0000 7f804b558640 1 --2- 172.21.15.27:0/2223952988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8044070650 0x7f804415a140 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f80340097a0 tx=0x7f803402da50 comp rx=0 tx=0).stop 2023-12-19T18:22:25.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.179+0000 7f804b558640 1 -- 172.21.15.27:0/2223952988 shutdown_connections 2023-12-19T18:22:25.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.179+0000 7f804b558640 1 --2- 172.21.15.27:0/2223952988 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8044070650 0x7f804415a140 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.179+0000 7f804b558640 1 -- 172.21.15.27:0/2223952988 >> 172.21.15.27:0/2223952988 conn(0x7f80440cb3f0 msgr2=0x7f80440cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:25.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.179+0000 7f804b558640 1 -- 172.21.15.27:0/2223952988 shutdown_connections 2023-12-19T18:22:25.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.179+0000 7f804b558640 1 -- 172.21.15.27:0/2223952988 wait complete. 2023-12-19T18:22:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.181+0000 7f804b558640 1 Processor -- start 2023-12-19T18:22:25.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.181+0000 7f804b558640 1 -- start start 2023-12-19T18:22:25.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.182+0000 7f804b558640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f804406c030 0x7f8044152050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:25.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.182+0000 7f804b558640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8044070650 0x7f80440b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:25.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f804b558640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8044070ac0 0x7f8044154d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:25.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f804b558640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80441f38c0 con 0x7f8044070650 2023-12-19T18:22:25.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f804b558640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044152590 con 0x7f8044070ac0 2023-12-19T18:22:25.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f804b558640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80441f2980 con 0x7f804406c030 2023-12-19T18:22:25.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f8049d55640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8044070650 0x7f80440b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:25.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f8049d55640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8044070650 0x7f80440b0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45128/0 (socket says 172.21.15.27:45128) 2023-12-19T18:22:25.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f8049d55640 1 -- 172.21.15.27:0/1669734891 learned_addr learned my addr 172.21.15.27:0/1669734891 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:25.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f804ad57640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8044070ac0 0x7f8044154d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:25.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f8049d55640 1 -- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f804406c030 msgr2=0x7f8044152050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:25.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f8049d55640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f804406c030 0x7f8044152050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f8049d55640 1 -- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8044070ac0 msgr2=0x7f8044154d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:25.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f8049d55640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8044070ac0 0x7f8044154d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f8049d55640 1 -- 172.21.15.27:0/1669734891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8034009400 con 0x7f8044070650 2023-12-19T18:22:25.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.183+0000 7f804ad57640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8044070ac0 0x7f8044154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:25.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.184+0000 7f8049d55640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8044070650 0x7f80440b0cb0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f803402da90 tx=0x7f80340046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:25.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.184+0000 7f803affd640 1 -- 172.21.15.27:0/1669734891 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803403d040 con 0x7f8044070650 2023-12-19T18:22:25.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.184+0000 7f803affd640 1 -- 172.21.15.27:0/1669734891 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8034039c40 con 0x7f8044070650 2023-12-19T18:22:25.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.184+0000 7f803affd640 1 -- 172.21.15.27:0/1669734891 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f803402f400 con 0x7f8044070650 2023-12-19T18:22:25.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.184+0000 7f804b558640 1 -- 172.21.15.27:0/1669734891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80441f2c40 con 0x7f8044070650 2023-12-19T18:22:25.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.184+0000 7f804b558640 1 -- 172.21.15.27:0/1669734891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80441f3200 con 0x7f8044070650 2023-12-19T18:22:25.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.185+0000 7f804b558640 1 -- 172.21.15.27:0/1669734891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8044066fe0 con 0x7f8044070650 2023-12-19T18:22:25.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.186+0000 7f803affd640 1 -- 172.21.15.27:0/1669734891 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f803402f580 con 0x7f8044070650 2023-12-19T18:22:25.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.188+0000 7f803affd640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f802806a430 0x7f802806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:25.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.188+0000 7f803affd640 1 -- 172.21.15.27:0/1669734891 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80340ab5d0 con 0x7f8044070650 2023-12-19T18:22:25.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.188+0000 7f804a556640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f802806a430 0x7f802806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:25.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.190+0000 7f804a556640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f802806a430 0x7f802806c8f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8040000f60 tx=0x7f8040007a80 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:25.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.192+0000 7f803affd640 1 -- 172.21.15.27:0/1669734891 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8034076cd0 con 0x7f8044070650 2023-12-19T18:22:25.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.453+0000 7f804b558640 1 -- 172.21.15.27:0/1669734891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f804404e840 con 0x7f8044070650 2023-12-19T18:22:25.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.461+0000 7f803affd640 1 -- 172.21.15.27:0/1669734891 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7f803407a2f0 con 0x7f8044070650 2023-12-19T18:22:25.461 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2023-12-19T18:22:25.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.463+0000 7f804b558640 1 -- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f802806a430 msgr2=0x7f802806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:25.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.463+0000 7f804b558640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f802806a430 0x7f802806c8f0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8040000f60 tx=0x7f8040007a80 comp rx=0 tx=0).stop 2023-12-19T18:22:25.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.463+0000 7f804b558640 1 -- 172.21.15.27:0/1669734891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8044070650 msgr2=0x7f80440b0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:25.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.463+0000 7f804b558640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8044070650 0x7f80440b0cb0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f803402da90 tx=0x7f80340046c0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.463+0000 7f804b558640 1 -- 172.21.15.27:0/1669734891 shutdown_connections 2023-12-19T18:22:25.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.463+0000 7f804b558640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f802806a430 0x7f802806c8f0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.463+0000 7f804b558640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8044070ac0 0x7f8044154d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.463+0000 7f804b558640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8044070650 0x7f80440b0cb0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.463+0000 7f804b558640 1 --2- 172.21.15.27:0/1669734891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f804406c030 0x7f8044152050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.463+0000 7f804b558640 1 -- 172.21.15.27:0/1669734891 >> 172.21.15.27:0/1669734891 conn(0x7f80440cb3f0 msgr2=0x7f804415cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:25.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.463+0000 7f804b558640 1 -- 172.21.15.27:0/1669734891 shutdown_connections 2023-12-19T18:22:25.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.464+0000 7f804b558640 1 -- 172.21.15.27:0/1669734891 wait complete. 2023-12-19T18:22:25.480 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:22:25.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.697+0000 7fbb21969640 1 Processor -- start 2023-12-19T18:22:25.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.698+0000 7fbb21969640 1 -- start start 2023-12-19T18:22:25.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.699+0000 7fbb21969640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbb1c0cbce0 0x7fbb1c0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:25.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.700+0000 7fbb21969640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1c0cb850 con 0x7fbb1c15af70 2023-12-19T18:22:25.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.700+0000 7fbb21969640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb1c15b350 con 0x7fbb1c0adc20 2023-12-19T18:22:25.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.700+0000 7fbb21969640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1c0cac40 con 0x7fbb1c0cbce0 2023-12-19T18:22:25.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.700+0000 7fbb1affd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbb1c15af70 0x7fbb1c0ad510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37694/0 (socket says 172.21.15.27:37694) 2023-12-19T18:22:25.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.700+0000 7fbb1affd640 1 -- 172.21.15.27:0/1434413459 learned_addr learned my addr 172.21.15.27:0/1434413459 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.700+0000 7fbb13fff640 1 --2- 172.21.15.27:0/1434413459 >> v2:172.21.15.192:3300/0 conn(0x7fbb1c0cbce0 0x7fbb1c0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028427620 0 0) 0x7fbb1c0cb850 con 0x7fbb1c15af70 2023-12-19T18:22:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb08003520 con 0x7fbb1c15af70 2023-12-19T18:22:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3895025173 0 0) 0x7fbb08003520 con 0x7fbb1c15af70 2023-12-19T18:22:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 >> v2:172.21.15.192:3300/0 conn(0x7fbb1c0cbce0 msgr2=0x7fbb1c0ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 --2- 172.21.15.27:0/1434413459 >> v2:172.21.15.192:3300/0 conn(0x7fbb1c0cbce0 0x7fbb1c0ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 >> v1:172.21.15.192:6789/0 conn(0x7fbb1c0adc20 legacy=0x7fbb1c06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:25.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c0caf20 con 0x7fbb1c15af70 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fbb0c001670 con 0x7fbb1c15af70 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 >> v1:172.21.15.27:6789/0 conn(0x7fbb1c15af70 legacy=0x7fbb1c0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 --2- 172.21.15.27:0/1434413459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb08003970 0x7fbb08003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 --2- 172.21.15.27:0/1434413459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb080047c0 0x7fbb08004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 --2- 172.21.15.27:0/1434413459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb08005180 0x7fbb0800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c0cac40 con 0x7fbb080047c0 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0800c020 con 0x7fbb08003970 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0800c1e0 con 0x7fbb08005180 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.701+0000 7fbb13fff640 1 --2- 172.21.15.27:0/1434413459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb080047c0 0x7fbb08004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.702+0000 7fbb1affd640 1 --2- 172.21.15.27:0/1434413459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb08003970 0x7fbb08003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.702+0000 7fbb13fff640 1 -- 172.21.15.27:0/1434413459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb08005180 msgr2=0x7fbb0800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.702+0000 7fbb13fff640 1 --2- 172.21.15.27:0/1434413459 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb08005180 0x7fbb0800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.702+0000 7fbb13fff640 1 -- 172.21.15.27:0/1434413459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb08003970 msgr2=0x7fbb08003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.702+0000 7fbb13fff640 1 --2- 172.21.15.27:0/1434413459 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb08003970 0x7fbb08003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.702+0000 7fbb13fff640 1 -- 172.21.15.27:0/1434413459 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb08003520 con 0x7fbb080047c0 2023-12-19T18:22:25.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.702+0000 7fbb1b7fe640 1 -- 172.21.15.27:0/1434413459 reap_dead start 2023-12-19T18:22:25.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.703+0000 7fbb13fff640 1 --2- 172.21.15.27:0/1434413459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb080047c0 0x7fbb08004c40 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009a90 tx=0x7fbb080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fc5e8311507ec0f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.703+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0402c680 con 0x7fbb080047c0 2023-12-19T18:22:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.703+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0402c7e0 con 0x7fbb080047c0 2023-12-19T18:22:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.703+0000 7fbb1a7fc640 1 -- 172.21.15.27:0/1434413459 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0402cb70 con 0x7fbb080047c0 2023-12-19T18:22:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.704+0000 7fbb21969640 1 -- 172.21.15.27:0/1434413459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb080047c0 msgr2=0x7fbb08004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.704+0000 7fbb21969640 1 --2- 172.21.15.27:0/1434413459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb080047c0 0x7fbb08004c40 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009a90 tx=0x7fbb080057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.704+0000 7fbb21969640 1 -- 172.21.15.27:0/1434413459 shutdown_connections 2023-12-19T18:22:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.704+0000 7fbb21969640 1 --2- 172.21.15.27:0/1434413459 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb080047c0 0x7fbb08004c40 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.704+0000 7fbb21969640 1 -- 172.21.15.27:0/1434413459 >> 172.21.15.27:0/1434413459 conn(0x7fbb1c06a330 msgr2=0x7fbb1c0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.704+0000 7fbb21969640 1 -- 172.21.15.27:0/1434413459 shutdown_connections 2023-12-19T18:22:25.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.704+0000 7fbb21969640 1 -- 172.21.15.27:0/1434413459 wait complete. 2023-12-19T18:22:25.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.706+0000 7fbb21969640 1 Processor -- start 2023-12-19T18:22:25.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.706+0000 7fbb21969640 1 -- start start 2023-12-19T18:22:25.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.707+0000 7fbb21969640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb080047c0 0x7fbb1c1f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:25.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.707+0000 7fbb21969640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb1c157530 0x7fbb1c154470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:25.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.707+0000 7fbb21969640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb1c1549b0 0x7fbb1c143380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:25.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.707+0000 7fbb21969640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c154e30 con 0x7fbb1c1549b0 2023-12-19T18:22:25.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.707+0000 7fbb21969640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1570a0 con 0x7fbb080047c0 2023-12-19T18:22:25.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.707+0000 7fbb21969640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1c1f4de0 con 0x7fbb1c157530 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.707+0000 7fbb1b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb1c1549b0 0x7fbb1c143380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.707+0000 7fbb1b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb1c1549b0 0x7fbb1c143380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45148/0 (socket says 172.21.15.27:45148) 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.707+0000 7fbb1b7fe640 1 -- 172.21.15.27:0/2775695348 learned_addr learned my addr 172.21.15.27:0/2775695348 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.707+0000 7fbb1affd640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb080047c0 0x7fbb1c1f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.707+0000 7fbb13fff640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb1c157530 0x7fbb1c154470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.708+0000 7fbb1b7fe640 1 -- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb1c157530 msgr2=0x7fbb1c154470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.708+0000 7fbb1b7fe640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb1c157530 0x7fbb1c154470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.708+0000 7fbb1b7fe640 1 -- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb080047c0 msgr2=0x7fbb1c1f48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.708+0000 7fbb1b7fe640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb080047c0 0x7fbb1c1f48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.708+0000 7fbb1b7fe640 1 -- 172.21.15.27:0/2775695348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb1c144100 con 0x7fbb1c1549b0 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.708+0000 7fbb1b7fe640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb1c1549b0 0x7fbb1c143380 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0b4830 tx=0x7fbb1400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.708+0000 7fbb18ff9640 1 -- 172.21.15.27:0/2775695348 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb140070d0 con 0x7fbb1c1549b0 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.708+0000 7fbb18ff9640 1 -- 172.21.15.27:0/2775695348 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb14007250 con 0x7fbb1c1549b0 2023-12-19T18:22:25.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.708+0000 7fbb21969640 1 -- 172.21.15.27:0/2775695348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1c156a50 con 0x7fbb1c1549b0 2023-12-19T18:22:25.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.708+0000 7fbb18ff9640 1 -- 172.21.15.27:0/2775695348 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb14007600 con 0x7fbb1c1549b0 2023-12-19T18:22:25.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.708+0000 7fbb21969640 1 -- 172.21.15.27:0/2775695348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb1c1566d0 con 0x7fbb1c1549b0 2023-12-19T18:22:25.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.709+0000 7fbb21969640 1 -- 172.21.15.27:0/2775695348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1c156890 con 0x7fbb1c1549b0 2023-12-19T18:22:25.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.710+0000 7fbb18ff9640 1 -- 172.21.15.27:0/2775695348 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbb14007760 con 0x7fbb1c1549b0 2023-12-19T18:22:25.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.711+0000 7fbb18ff9640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbafc06a430 0x7fbafc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:25.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.711+0000 7fbb18ff9640 1 -- 172.21.15.27:0/2775695348 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb14087de0 con 0x7fbb1c1549b0 2023-12-19T18:22:25.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.712+0000 7fbb1affd640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbafc06a430 0x7fbafc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:25.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.713+0000 7fbb1affd640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbafc06a430 0x7fbafc06c8f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c001bd0 tx=0x7fbb0c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:25.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.716+0000 7fbb18ff9640 1 -- 172.21.15.27:0/2775695348 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbb14053330 con 0x7fbb1c1549b0 2023-12-19T18:22:25.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.952+0000 7fbb21969640 1 -- 172.21.15.27:0/2775695348 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb1c068e90 con 0x7fbb1c1549b0 2023-12-19T18:22:25.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.953+0000 7fbb18ff9640 1 -- 172.21.15.27:0/2775695348 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4711 (secure 0 0 0) 0x7fbb14017090 con 0x7fbb1c1549b0 2023-12-19T18:22:25.953 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:25.953 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2713867269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2713867269},{"type":"v1","addr":"172.21.15.192:6837","nonce":2713867269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":5534,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/919878747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":919878747},{"type":"v1","addr":"172.21.15.27:6835","nonce":919878747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6160,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/861111763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":861111763},{"type":"v1","addr":"172.21.15.192:6836","nonce":861111763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:22:14.217224+0000","modified":"2023-12-19T18:22:17.743185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7320":{"gid":7320,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3670863819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3670863819},{"type":"v1","addr":"172.21.15.27:6834","nonce":3670863819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T18:22:25.954 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 54 2023-12-19T18:22:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.955+0000 7fbb21969640 1 -- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbafc06a430 msgr2=0x7fbafc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:25.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.955+0000 7fbb21969640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbafc06a430 0x7fbafc06c8f0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c001bd0 tx=0x7fbb0c009040 comp rx=0 tx=0).stop 2023-12-19T18:22:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.955+0000 7fbb21969640 1 -- 172.21.15.27:0/2775695348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb1c1549b0 msgr2=0x7fbb1c143380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.955+0000 7fbb21969640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb1c1549b0 0x7fbb1c143380 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7fbb1c0b4830 tx=0x7fbb1400c960 comp rx=0 tx=0).stop 2023-12-19T18:22:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.956+0000 7fbb21969640 1 -- 172.21.15.27:0/2775695348 shutdown_connections 2023-12-19T18:22:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.956+0000 7fbb21969640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbafc06a430 0x7fbafc06c8f0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.956+0000 7fbb21969640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb1c1549b0 0x7fbb1c143380 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.956+0000 7fbb21969640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb1c157530 0x7fbb1c154470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.956+0000 7fbb21969640 1 --2- 172.21.15.27:0/2775695348 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb080047c0 0x7fbb1c1f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.956+0000 7fbb21969640 1 -- 172.21.15.27:0/2775695348 >> 172.21.15.27:0/2775695348 conn(0x7fbb1c06a330 msgr2=0x7fbb1c070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.956+0000 7fbb21969640 1 -- 172.21.15.27:0/2775695348 shutdown_connections 2023-12-19T18:22:25.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:25.956+0000 7fbb21969640 1 -- 172.21.15.27:0/2775695348 wait complete. 2023-12-19T18:22:26.378 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:22:14.217224+0000', 'modified': '2023-12-19T18:22:17.743185+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7320}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7320': {'gid': 7320, 'name': 'c', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6834/3670863819', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6832', 'nonce': 3670863819}, {'type': 'v1', 'addr': '172.21.15.27:6834', 'nonce': 3670863819}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:22:26.379 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:22:26.379 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-19T18:22:26.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.622+0000 7fcea34dc640 1 Processor -- start 2023-12-19T18:22:26.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.623+0000 7fcea34dc640 1 -- start start 2023-12-19T18:22:26.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.624+0000 7fcea34dc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fce9c15ac50 0x7fce9c070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.624+0000 7fcea34dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce9c0aef20 0x7fce9c06b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.624+0000 7fcea34dc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce9c0adf10 con 0x7fce9c0af360 2023-12-19T18:22:26.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.624+0000 7fcea34dc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce9c0adac0 con 0x7fce9c15ac50 2023-12-19T18:22:26.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.624+0000 7fcea34dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c0ae160 con 0x7fce9c0aef20 2023-12-19T18:22:26.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.625+0000 7fcea1cd9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce9c0aef20 0x7fce9c06b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.625+0000 7fcea1cd9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce9c0aef20 0x7fce9c06b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:42708/0 (socket says 172.21.15.27:42708) 2023-12-19T18:22:26.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.625+0000 7fcea1cd9640 1 -- 172.21.15.27:0/1286367164 learned_addr learned my addr 172.21.15.27:0/1286367164 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:26.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.625+0000 7fcea24da640 1 --2- 172.21.15.27:0/1286367164 >> v2:172.21.15.192:3300/0 conn(0x7fce9c15ac50 0x7fce9c070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.626+0000 7fcea1cd9640 1 -- 172.21.15.27:0/1286367164 >> v2:172.21.15.192:3300/0 conn(0x7fce9c15ac50 msgr2=0x7fce9c070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:26.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.626+0000 7fcea1cd9640 1 --2- 172.21.15.27:0/1286367164 >> v2:172.21.15.192:3300/0 conn(0x7fce9c15ac50 0x7fce9c070fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.626+0000 7fcea1cd9640 1 -- 172.21.15.27:0/1286367164 >> v1:172.21.15.192:6789/0 conn(0x7fce9c0af360 legacy=0x7fce9c06b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.626+0000 7fcea1cd9640 1 -- 172.21.15.27:0/1286367164 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9c0ae440 con 0x7fce9c0aef20 2023-12-19T18:22:26.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.627+0000 7fcea1cd9640 1 --2- 172.21.15.27:0/1286367164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce9c0aef20 0x7fce9c06b310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009870 tx=0x7fce8c02b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86cabacf343442a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:26.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.628+0000 7fcea14d8640 1 -- 172.21.15.27:0/1286367164 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c0317c0 con 0x7fce9c0aef20 2023-12-19T18:22:26.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.629+0000 7fcea14d8640 1 -- 172.21.15.27:0/1286367164 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c031920 con 0x7fce9c0aef20 2023-12-19T18:22:26.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.629+0000 7fcea14d8640 1 -- 172.21.15.27:0/1286367164 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c031cb0 con 0x7fce9c0aef20 2023-12-19T18:22:26.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.629+0000 7fcea34dc640 1 -- 172.21.15.27:0/1286367164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce9c0aef20 msgr2=0x7fce9c06b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.629+0000 7fcea34dc640 1 --2- 172.21.15.27:0/1286367164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce9c0aef20 0x7fce9c06b310 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fce8c009870 tx=0x7fce8c02b860 comp rx=0 tx=0).stop 2023-12-19T18:22:26.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.629+0000 7fcea34dc640 1 -- 172.21.15.27:0/1286367164 shutdown_connections 2023-12-19T18:22:26.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.629+0000 7fcea34dc640 1 --2- 172.21.15.27:0/1286367164 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce9c0aef20 0x7fce9c06b310 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.629+0000 7fcea34dc640 1 --2- 172.21.15.27:0/1286367164 >> v2:172.21.15.192:3300/0 conn(0x7fce9c15ac50 0x7fce9c070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.629+0000 7fcea34dc640 1 -- 172.21.15.27:0/1286367164 >> 172.21.15.27:0/1286367164 conn(0x7fce9c06a7d0 msgr2=0x7fce9c0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:26.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.629+0000 7fcea34dc640 1 -- 172.21.15.27:0/1286367164 shutdown_connections 2023-12-19T18:22:26.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.630+0000 7fcea34dc640 1 -- 172.21.15.27:0/1286367164 wait complete. 2023-12-19T18:22:26.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.632+0000 7fcea34dc640 1 Processor -- start 2023-12-19T18:22:26.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.632+0000 7fcea34dc640 1 -- start start 2023-12-19T18:22:26.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.633+0000 7fcea34dc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce9c0aef20 0x7fce9c15dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.633+0000 7fcea34dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce9c0af360 0x7fce9c14c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.633+0000 7fcea34dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce9c15ac50 0x7fce9c0c8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.633+0000 7fcea34dc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c0b1c10 con 0x7fce9c0aef20 2023-12-19T18:22:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.633+0000 7fcea34dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c14cb10 con 0x7fce9c0af360 2023-12-19T18:22:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.633+0000 7fcea34dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce9c15e460 con 0x7fce9c15ac50 2023-12-19T18:22:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.633+0000 7fcea24da640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce9c0aef20 0x7fce9c15dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.634+0000 7fcea24da640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce9c0aef20 0x7fce9c15dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45156/0 (socket says 172.21.15.27:45156) 2023-12-19T18:22:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.634+0000 7fcea24da640 1 -- 172.21.15.27:0/174485871 learned_addr learned my addr 172.21.15.27:0/174485871 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:26.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.634+0000 7fcea1cd9640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce9c0af360 0x7fce9c14c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.634+0000 7fcea2cdb640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce9c15ac50 0x7fce9c0c8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.634+0000 7fcea24da640 1 -- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce9c15ac50 msgr2=0x7fce9c0c8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.634+0000 7fcea24da640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce9c15ac50 0x7fce9c0c8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.634+0000 7fcea24da640 1 -- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce9c0af360 msgr2=0x7fce9c14c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.634+0000 7fcea24da640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce9c0af360 0x7fce9c14c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.634+0000 7fcea24da640 1 -- 172.21.15.27:0/174485871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c0094d0 con 0x7fce9c0aef20 2023-12-19T18:22:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.634+0000 7fcea24da640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce9c0aef20 0x7fce9c15dd80 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fce9800e9b0 tx=0x7fce9800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.635+0000 7fce92ffd640 1 -- 172.21.15.27:0/174485871 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce98009670 con 0x7fce9c0aef20 2023-12-19T18:22:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.635+0000 7fce92ffd640 1 -- 172.21.15.27:0/174485871 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9800c9a0 con 0x7fce9c0aef20 2023-12-19T18:22:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.635+0000 7fce92ffd640 1 -- 172.21.15.27:0/174485871 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9800cd50 con 0x7fce9c0aef20 2023-12-19T18:22:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.635+0000 7fcea34dc640 1 -- 172.21.15.27:0/174485871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce9c0b1e60 con 0x7fce9c0aef20 2023-12-19T18:22:26.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.635+0000 7fcea34dc640 1 -- 172.21.15.27:0/174485871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c0b2420 con 0x7fce9c0aef20 2023-12-19T18:22:26.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.637+0000 7fce92ffd640 1 -- 172.21.15.27:0/174485871 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fce98013470 con 0x7fce9c0aef20 2023-12-19T18:22:26.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.638+0000 7fce92ffd640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce8006a430 0x7fce8006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.638+0000 7fce92ffd640 1 -- 172.21.15.27:0/174485871 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce98088bf0 con 0x7fce9c0aef20 2023-12-19T18:22:26.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.639+0000 7fcea1cd9640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce8006a430 0x7fce8006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.639+0000 7fcea738c740 1 Processor -- start 2023-12-19T18:22:26.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.639+0000 7fcea738c740 1 -- start start 2023-12-19T18:22:26.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.640+0000 7fcea1cd9640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce8006a430 0x7fce8006c8f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fce8c036000 tx=0x7fce8c03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:26.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.640+0000 7fcea738c740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198f110 0x55d31198f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.640+0000 7fcea738c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31195f090 0x55d31198c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.640+0000 7fcea738c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 0x55d31198c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.640+0000 7fcea738c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d31193a650 con 0x55d31195f090 2023-12-19T18:22:26.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.640+0000 7fcea738c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3118cf000 con 0x55d311962c90 2023-12-19T18:22:26.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.640+0000 7fcea738c740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3118cc790 con 0x55d31198f110 2023-12-19T18:22:26.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.640+0000 7fcea1cd9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31195f090 0x55d31198c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.640+0000 7fcea1cd9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31195f090 0x55d31198c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45166/0 (socket says 172.21.15.27:45166) 2023-12-19T18:22:26.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.640+0000 7fcea1cd9640 1 -- 172.21.15.27:0/438331191 learned_addr learned my addr 172.21.15.27:0/438331191 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:26.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.641+0000 7fcea24da640 1 --2- 172.21.15.27:0/438331191 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198f110 0x55d31198f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.641+0000 7fcea1cd9640 1 -- 172.21.15.27:0/438331191 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198f110 msgr2=0x55d31198f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:26.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.641+0000 7fcea1cd9640 1 --2- 172.21.15.27:0/438331191 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198f110 0x55d31198f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.641+0000 7fcea2cdb640 1 --2- 172.21.15.27:0/438331191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 0x55d31198c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.641+0000 7fcea1cd9640 1 -- 172.21.15.27:0/438331191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 msgr2=0x55d31198c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:26.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.641+0000 7fcea1cd9640 1 --2- 172.21.15.27:0/438331191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 0x55d31198c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.641+0000 7fcea1cd9640 1 -- 172.21.15.27:0/438331191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d31188f710 con 0x55d31195f090 2023-12-19T18:22:26.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.641+0000 7fcea24da640 1 --2- 172.21.15.27:0/438331191 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198f110 0x55d31198f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:26.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.641+0000 7fcea1cd9640 1 --2- 172.21.15.27:0/438331191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31195f090 0x55d31198c380 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fce8c034040 tx=0x7fce8c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e84aa1895640520 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:26.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.642+0000 7fce7ffff640 1 -- 172.21.15.27:0/438331191 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c04a070 con 0x55d31195f090 2023-12-19T18:22:26.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.642+0000 7fce7ffff640 1 -- 172.21.15.27:0/438331191 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c0329e0 con 0x55d31195f090 2023-12-19T18:22:26.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.642+0000 7fce7ffff640 1 -- 172.21.15.27:0/438331191 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c032d50 con 0x55d31195f090 2023-12-19T18:22:26.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.642+0000 7fcea738c740 1 -- 172.21.15.27:0/438331191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31195f090 msgr2=0x55d31198c380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.642+0000 7fcea738c740 1 --2- 172.21.15.27:0/438331191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31195f090 0x55d31198c380 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fce8c034040 tx=0x7fce8c030000 comp rx=0 tx=0).stop 2023-12-19T18:22:26.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.642+0000 7fcea738c740 1 -- 172.21.15.27:0/438331191 shutdown_connections 2023-12-19T18:22:26.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.642+0000 7fcea738c740 1 --2- 172.21.15.27:0/438331191 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 0x55d31198c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.642+0000 7fcea738c740 1 --2- 172.21.15.27:0/438331191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31195f090 0x55d31198c380 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.642+0000 7fcea738c740 1 --2- 172.21.15.27:0/438331191 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198f110 0x55d31198f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.642+0000 7fcea738c740 1 -- 172.21.15.27:0/438331191 >> 172.21.15.27:0/438331191 conn(0x55d3119613a0 msgr2=0x55d3119a9f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:26.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.643+0000 7fcea738c740 1 -- 172.21.15.27:0/438331191 shutdown_connections 2023-12-19T18:22:26.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.643+0000 7fcea738c740 1 -- 172.21.15.27:0/438331191 wait complete. 2023-12-19T18:22:26.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.643+0000 7fcea738c740 1 -- start start 2023-12-19T18:22:26.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.643+0000 7fcea738c740 1 Processor -- start 2023-12-19T18:22:26.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea738c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d311962c90 0x55d3119a1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea738c740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198f110 0x55d3119a3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea738c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d31198a9e0 0x55d31198ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea738c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3118c8080 con 0x55d311962c90 2023-12-19T18:22:26.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea738c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d311869850 con 0x55d31198a9e0 2023-12-19T18:22:26.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea738c740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d3118694f0 con 0x55d31198f110 2023-12-19T18:22:26.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea738c740 10 client.0 _ll_register_callbacks cb 0x55d3118c19c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:22:26.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea24da640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d311962c90 0x55d3119a1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea24da640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d311962c90 0x55d3119a1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45172/0 (socket says 172.21.15.27:45172) 2023-12-19T18:22:26.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea24da640 1 -- 172.21.15.27:0/2250739667 learned_addr learned my addr 172.21.15.27:0/2250739667 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:26.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea24da640 1 -- 172.21.15.27:0/2250739667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198f110 msgr2=0x55d3119a3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:26.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea24da640 1 --2- 172.21.15.27:0/2250739667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198f110 0x55d3119a3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea24da640 1 -- 172.21.15.27:0/2250739667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d31198a9e0 msgr2=0x55d31198ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:26.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea24da640 1 --2- 172.21.15.27:0/2250739667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d31198a9e0 0x55d31198ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.645+0000 7fcea24da640 1 -- 172.21.15.27:0/2250739667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c0ae440 con 0x55d311962c90 2023-12-19T18:22:26.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.646+0000 7fcea24da640 1 --2- 172.21.15.27:0/2250739667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d311962c90 0x55d3119a1010 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fce98004770 tx=0x7fce98057fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6621e8ef60015317 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:26.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.646+0000 7fce7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:22:26.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.646+0000 7fce7d7fa640 1 -- 172.21.15.27:0/2250739667 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9806f070 con 0x55d311962c90 2023-12-19T18:22:26.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.646+0000 7fce7d7fa640 1 -- 172.21.15.27:0/2250739667 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce980562f0 con 0x55d311962c90 2023-12-19T18:22:26.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.646+0000 7fcea738c740 1 -- 172.21.15.27:0/2250739667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3118f25d0 con 0x55d311962c90 2023-12-19T18:22:26.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.646+0000 7fce7d7fa640 1 -- 172.21.15.27:0/2250739667 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce98058ea0 con 0x55d311962c90 2023-12-19T18:22:26.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.646+0000 7fcea738c740 10 client.7497 fetch_fsmap learned FSMap version 54 2023-12-19T18:22:26.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.646+0000 7fcea738c740 1 -- 172.21.15.27:0/2250739667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55d3118c6700 con 0x55d311962c90 2023-12-19T18:22:26.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.646+0000 7fce7d7fa640 1 -- 172.21.15.27:0/2250739667 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce98073020 con 0x55d311962c90 2023-12-19T18:22:26.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.646+0000 7fce7d7fa640 1 -- 172.21.15.27:0/2250739667 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 54) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fce98011070 con 0x55d311962c90 2023-12-19T18:22:26.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.647+0000 7fcea738c740 10 client.7497 fetch_fsmap finished waiting for FSMap version 54 2023-12-19T18:22:26.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.647+0000 7fcea738c740 10 client.7497 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:22:26.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.647+0000 7fcea738c740 20 client.7497 populate_metadata read hostname 'smithi027' 2023-12-19T18:22:26.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.647+0000 7fcea738c740 1 --2- 172.21.15.27:0/2250739667 >> [v2:172.21.15.27:6832/3670863819,v1:172.21.15.27:6834/3670863819] conn(0x55d3119ba3c0 0x55d311990fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.647+0000 7fcea738c740 4 client.7497 mds_command: new command op to 7320 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:22:26.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.647+0000 7fcea738c740 1 -- 172.21.15.27:0/2250739667 --> [v2:172.21.15.27:6832/3670863819,v1:172.21.15.27:6834/3670863819] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d3118ba8d0 con 0x55d3119ba3c0 2023-12-19T18:22:26.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.647+0000 7fcea1cd9640 1 --2- 172.21.15.27:0/2250739667 >> [v2:172.21.15.27:6832/3670863819,v1:172.21.15.27:6834/3670863819] conn(0x55d3119ba3c0 0x55d311990fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.648+0000 7fce7d7fa640 10 client.7497 ms_handle_connect on v2:172.21.15.27:6832/3670863819 2023-12-19T18:22:26.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.648+0000 7fcea1cd9640 1 --2- 172.21.15.27:0/2250739667 >> [v2:172.21.15.27:6832/3670863819,v1:172.21.15.27:6834/3670863819] conn(0x55d3119ba3c0 0x55d311990fa0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f7b5da55a967cf0 server_cookie=d3aa16093874a4da in_seq=0 out_seq=0 2023-12-19T18:22:26.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fce7d7fa640 1 -- 172.21.15.27:0/2250739667 <== mds.0 v2:172.21.15.27:6832/3670863819 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d3118ba8d0 con 0x55d3119ba3c0 2023-12-19T18:22:26.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fce7d7fa640 10 client.7497 handle_command_reply: tid=0 2023-12-19T18:22:26.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fcea738c740 1 client.7497 shutdown 2023-12-19T18:22:26.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fcea738c740 1 -- 172.21.15.27:0/2250739667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d311962c90 msgr2=0x55d3119a1010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fcea738c740 1 --2- 172.21.15.27:0/2250739667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d311962c90 0x55d3119a1010 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fce98004770 tx=0x7fce98057fd0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fcea738c740 1 -- 172.21.15.27:0/2250739667 shutdown_connections 2023-12-19T18:22:26.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fcea738c740 1 --2- 172.21.15.27:0/2250739667 >> [v2:172.21.15.27:6832/3670863819,v1:172.21.15.27:6834/3670863819] conn(0x55d3119ba3c0 0x55d311990fa0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fcea738c740 1 --2- 172.21.15.27:0/2250739667 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d31198a9e0 0x55d31198ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fcea738c740 1 --2- 172.21.15.27:0/2250739667 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198f110 0x55d3119a3a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fcea738c740 1 --2- 172.21.15.27:0/2250739667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d311962c90 0x55d3119a1010 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fce7d7fa640 0 client.7497 ms_handle_reset on v2:172.21.15.27:6832/3670863819 2023-12-19T18:22:26.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fcea738c740 1 -- 172.21.15.27:0/2250739667 >> 172.21.15.27:0/2250739667 conn(0x55d31195f090 msgr2=0x55d3119ad430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:26.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fcea738c740 1 -- 172.21.15.27:0/2250739667 shutdown_connections 2023-12-19T18:22:26.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fcea738c740 1 -- 172.21.15.27:0/2250739667 wait complete. 2023-12-19T18:22:26.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.649+0000 7fcea738c740 20 client.7497 trim_cache size 0 max 16384 2023-12-19T18:22:26.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.666+0000 7fcea738c740 1 Processor -- start 2023-12-19T18:22:26.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.666+0000 7fcea738c740 1 -- start start 2023-12-19T18:22:26.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.667+0000 7fcea738c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 0x55d311992450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.667+0000 7fcea738c740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198a9e0 0x55d31198bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.667+0000 7fcea738c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 0x55d31198c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.667+0000 7fcea738c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d31162ff20 con 0x55d31198f110 2023-12-19T18:22:26.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.667+0000 7fcea738c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3119a4740 con 0x55d311962c90 2023-12-19T18:22:26.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.667+0000 7fcea738c740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d31198a860 con 0x55d31198a9e0 2023-12-19T18:22:26.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.667+0000 7fcea2cdb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 0x55d31198c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.667+0000 7fcea2cdb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 0x55d31198c470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45180/0 (socket says 172.21.15.27:45180) 2023-12-19T18:22:26.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.667+0000 7fcea2cdb640 1 -- 172.21.15.27:0/1513167607 learned_addr learned my addr 172.21.15.27:0/1513167607 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:26.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.668+0000 7fcea2cdb640 1 -- 172.21.15.27:0/1513167607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198a9e0 msgr2=0x55d31198bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:26.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.668+0000 7fcea2cdb640 1 --2- 172.21.15.27:0/1513167607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198a9e0 0x55d31198bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.668+0000 7fcea2cdb640 1 -- 172.21.15.27:0/1513167607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 msgr2=0x55d311992450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:26.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.668+0000 7fcea2cdb640 1 --2- 172.21.15.27:0/1513167607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 0x55d311992450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.668+0000 7fcea2cdb640 1 -- 172.21.15.27:0/1513167607 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d3118fc280 con 0x55d31198f110 2023-12-19T18:22:26.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.668+0000 7fcea2cdb640 1 --2- 172.21.15.27:0/1513167607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 0x55d31198c470 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fce94001a30 tx=0x7fce94001ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975f3a0e7c3e9a81 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:26.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.668+0000 7fce90ff9640 1 -- 172.21.15.27:0/1513167607 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94003b80 con 0x55d31198f110 2023-12-19T18:22:26.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.668+0000 7fce90ff9640 1 -- 172.21.15.27:0/1513167607 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce9400d050 con 0x55d31198f110 2023-12-19T18:22:26.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.668+0000 7fce90ff9640 1 -- 172.21.15.27:0/1513167607 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce9400f7b0 con 0x55d31198f110 2023-12-19T18:22:26.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.669+0000 7fcea738c740 1 -- 172.21.15.27:0/1513167607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 msgr2=0x55d31198c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.669+0000 7fcea738c740 1 --2- 172.21.15.27:0/1513167607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 0x55d31198c470 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fce94001a30 tx=0x7fce94001ef0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.669+0000 7fcea738c740 1 -- 172.21.15.27:0/1513167607 shutdown_connections 2023-12-19T18:22:26.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.669+0000 7fcea738c740 1 --2- 172.21.15.27:0/1513167607 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 0x55d31198c470 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.669+0000 7fcea738c740 1 --2- 172.21.15.27:0/1513167607 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198a9e0 0x55d31198bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.669+0000 7fcea738c740 1 --2- 172.21.15.27:0/1513167607 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 0x55d311992450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.669+0000 7fcea738c740 1 -- 172.21.15.27:0/1513167607 >> 172.21.15.27:0/1513167607 conn(0x55d31195f090 msgr2=0x55d3119a3840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:26.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.669+0000 7fcea738c740 1 -- 172.21.15.27:0/1513167607 shutdown_connections 2023-12-19T18:22:26.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.669+0000 7fcea738c740 1 -- 172.21.15.27:0/1513167607 wait complete. 2023-12-19T18:22:26.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.670+0000 7fcea738c740 1 -- start start 2023-12-19T18:22:26.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.670+0000 7fcea738c740 1 Processor -- start 2023-12-19T18:22:26.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.671+0000 7fcea738c740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 0x55d311a0e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.671+0000 7fcea738c740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198a9e0 0x55d311988a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.671+0000 7fcea738c740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 0x55d311988fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.671+0000 7fcea738c740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d311963480 con 0x55d31198f110 2023-12-19T18:22:26.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.671+0000 7fcea738c740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d31198fa00 con 0x55d311962c90 2023-12-19T18:22:26.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.671+0000 7fcea738c740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d31198a040 con 0x55d31198a9e0 2023-12-19T18:22:26.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.671+0000 7fcea738c740 10 client.0 _ll_register_callbacks cb 0x7fce980541d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:22:26.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.672+0000 7fcea2cdb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 0x55d311988fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.672+0000 7fcea2cdb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 0x55d311988fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45186/0 (socket says 172.21.15.27:45186) 2023-12-19T18:22:26.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.672+0000 7fcea2cdb640 1 -- 172.21.15.27:0/4151268773 learned_addr learned my addr 172.21.15.27:0/4151268773 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:26.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.672+0000 7fcea1cd9640 1 --2- 172.21.15.27:0/4151268773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198a9e0 0x55d311988a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.672+0000 7fcea24da640 1 --2- 172.21.15.27:0/4151268773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 0x55d311a0e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.672+0000 7fcea2cdb640 1 -- 172.21.15.27:0/4151268773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198a9e0 msgr2=0x55d311988a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:26.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.672+0000 7fcea2cdb640 1 --2- 172.21.15.27:0/4151268773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198a9e0 0x55d311988a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.672+0000 7fcea2cdb640 1 -- 172.21.15.27:0/4151268773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 msgr2=0x55d311a0e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:26.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.672+0000 7fcea2cdb640 1 --2- 172.21.15.27:0/4151268773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 0x55d311a0e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.672+0000 7fcea2cdb640 1 -- 172.21.15.27:0/4151268773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9c0adf10 con 0x55d31198f110 2023-12-19T18:22:26.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.672+0000 7fcea2cdb640 1 --2- 172.21.15.27:0/4151268773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 0x55d311988fc0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fce94009010 tx=0x7fce94009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf69a1ef02698154 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:26.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.672+0000 7fce7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:22:26.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fce7d7fa640 1 -- 172.21.15.27:0/4151268773 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce94006d50 con 0x55d31198f110 2023-12-19T18:22:26.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fce7d7fa640 1 -- 172.21.15.27:0/4151268773 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce940050d0 con 0x55d31198f110 2023-12-19T18:22:26.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fcea738c740 1 -- 172.21.15.27:0/4151268773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce8c0329e0 con 0x55d31198f110 2023-12-19T18:22:26.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fce7d7fa640 1 -- 172.21.15.27:0/4151268773 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce94005290 con 0x55d31198f110 2023-12-19T18:22:26.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fcea738c740 10 client.7503 fetch_fsmap learned FSMap version 54 2023-12-19T18:22:26.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fcea738c740 1 -- 172.21.15.27:0/4151268773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55d311868a60 con 0x55d31198f110 2023-12-19T18:22:26.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fce7d7fa640 1 -- 172.21.15.27:0/4151268773 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c0329e0 con 0x55d31198f110 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fce7d7fa640 1 -- 172.21.15.27:0/4151268773 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 54) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fce94012070 con 0x55d31198f110 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fcea738c740 10 client.7503 fetch_fsmap finished waiting for FSMap version 54 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fcea738c740 10 client.7503 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fcea738c740 20 client.7503 populate_metadata read hostname 'smithi027' 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fcea738c740 1 --2- 172.21.15.27:0/4151268773 >> [v2:172.21.15.27:6832/3670863819,v1:172.21.15.27:6834/3670863819] conn(0x55d3119b8cd0 0x55d3119b90b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fcea738c740 4 client.7503 mds_command: new command op to 7320 tid=0[{"prefix": "status"}] 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.673+0000 7fcea738c740 1 -- 172.21.15.27:0/4151268773 --> [v2:172.21.15.27:6832/3670863819,v1:172.21.15.27:6834/3670863819] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d3118c1100 con 0x55d3119b8cd0 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.674+0000 7fcea24da640 1 --2- 172.21.15.27:0/4151268773 >> [v2:172.21.15.27:6832/3670863819,v1:172.21.15.27:6834/3670863819] conn(0x55d3119b8cd0 0x55d3119b90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.674+0000 7fce7d7fa640 10 client.7503 ms_handle_connect on v2:172.21.15.27:6832/3670863819 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.674+0000 7fcea24da640 1 --2- 172.21.15.27:0/4151268773 >> [v2:172.21.15.27:6832/3670863819,v1:172.21.15.27:6834/3670863819] conn(0x55d3119b8cd0 0x55d3119b90b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd601c448c429c74 server_cookie=64b3bd5515d92ad in_seq=0 out_seq=0 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fce7d7fa640 1 -- 172.21.15.27:0/4151268773 <== mds.0 v2:172.21.15.27:6832/3670863819 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x55d3118c1100 con 0x55d3119b8cd0 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fce7d7fa640 10 client.7503 handle_command_reply: tid=0 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fcea738c740 1 client.7503 shutdown 2023-12-19T18:22:26.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fcea738c740 1 -- 172.21.15.27:0/4151268773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 msgr2=0x55d311988fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fcea738c740 1 --2- 172.21.15.27:0/4151268773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 0x55d311988fc0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fce94009010 tx=0x7fce94009040 comp rx=0 tx=0).stop 2023-12-19T18:22:26.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fcea738c740 1 -- 172.21.15.27:0/4151268773 shutdown_connections 2023-12-19T18:22:26.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fcea738c740 1 --2- 172.21.15.27:0/4151268773 >> [v2:172.21.15.27:6832/3670863819,v1:172.21.15.27:6834/3670863819] conn(0x55d3119b8cd0 0x55d3119b90b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fcea738c740 1 --2- 172.21.15.27:0/4151268773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55d31198f110 0x55d311988fc0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fcea738c740 1 --2- 172.21.15.27:0/4151268773 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55d31198a9e0 0x55d311988a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fce7d7fa640 0 client.7503 ms_handle_reset on v2:172.21.15.27:6832/3670863819 2023-12-19T18:22:26.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fcea738c740 1 --2- 172.21.15.27:0/4151268773 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55d311962c90 0x55d311a0e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fcea738c740 1 -- 172.21.15.27:0/4151268773 >> 172.21.15.27:0/4151268773 conn(0x55d31195f090 msgr2=0x55d31198bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:26.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fcea738c740 1 -- 172.21.15.27:0/4151268773 shutdown_connections 2023-12-19T18:22:26.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fcea738c740 1 -- 172.21.15.27:0/4151268773 wait complete. 2023-12-19T18:22:26.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.675+0000 7fcea738c740 20 client.7503 trim_cache size 0 max 16384 2023-12-19T18:22:26.680 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:22:26.680 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:22:26.680 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-12-19T18:22:26.680 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 7320, 2023-12-19T18:22:26.680 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-19T18:22:26.680 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-19T18:22:26.680 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-19T18:22:26.680 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 12.45397837, 2023-12-19T18:22:26.680 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 54, 2023-12-19T18:22:26.680 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 135, 2023-12-19T18:22:26.681 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 135, 2023-12-19T18:22:26.681 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 17.880458223000002 2023-12-19T18:22:26.681 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:22:26.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.676+0000 7fcea34dc640 1 -- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce8006a430 msgr2=0x7fce8006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.676+0000 7fcea34dc640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce8006a430 0x7fce8006c8f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fce8c036000 tx=0x7fce8c03d040 comp rx=0 tx=0).stop 2023-12-19T18:22:26.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.676+0000 7fcea34dc640 1 -- 172.21.15.27:0/174485871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce9c0aef20 msgr2=0x7fce9c15dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.676+0000 7fcea34dc640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce9c0aef20 0x7fce9c15dd80 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fce9800e9b0 tx=0x7fce9800ee70 comp rx=0 tx=0).stop 2023-12-19T18:22:26.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.676+0000 7fcea34dc640 1 -- 172.21.15.27:0/174485871 shutdown_connections 2023-12-19T18:22:26.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.676+0000 7fcea34dc640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce8006a430 0x7fce8006c8f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.676+0000 7fcea34dc640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce9c15ac50 0x7fce9c0c8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.676+0000 7fcea34dc640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce9c0af360 0x7fce9c14c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.676+0000 7fcea34dc640 1 --2- 172.21.15.27:0/174485871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce9c0aef20 0x7fce9c15dd80 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.676+0000 7fcea34dc640 1 -- 172.21.15.27:0/174485871 >> 172.21.15.27:0/174485871 conn(0x7fce9c06a7d0 msgr2=0x7fce9c06c490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:26.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.676+0000 7fcea34dc640 1 -- 172.21.15.27:0/174485871 shutdown_connections 2023-12-19T18:22:26.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.676+0000 7fcea34dc640 1 -- 172.21.15.27:0/174485871 wait complete. 2023-12-19T18:22:26.693 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:22:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.916+0000 7fdab3315640 1 Processor -- start 2023-12-19T18:22:26.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.916+0000 7fdab3315640 1 -- start start 2023-12-19T18:22:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.917+0000 7fdab3315640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdaac1553a0 0x7fdaac0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.917+0000 7fdab3315640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdaac15ee30 0x7fdaac0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.917+0000 7fdab3315640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaac0b0360 con 0x7fdaac0afeb0 2023-12-19T18:22:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.918+0000 7fdab3315640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdaac0cbcf0 con 0x7fdaac1553a0 2023-12-19T18:22:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.918+0000 7fdab3315640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac0cb350 con 0x7fdaac15ee30 2023-12-19T18:22:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.918+0000 7fdab1b12640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdaac0afeb0 0x7fdaac15e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:37704/0 (socket says 172.21.15.27:37704) 2023-12-19T18:22:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.918+0000 7fdab1b12640 1 -- 172.21.15.27:0/3277845109 learned_addr learned my addr 172.21.15.27:0/3277845109 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:26.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.918+0000 7fdab2313640 1 --2- 172.21.15.27:0/3277845109 >> v2:172.21.15.27:3300/0 conn(0x7fdaac1553a0 0x7fdaac0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.918+0000 7fdab2b14640 1 --2- 172.21.15.27:0/3277845109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdaac15ee30 0x7fdaac0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.918+0000 7fdab1311640 1 -- 172.21.15.27:0/3277845109 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3440007497 0 0) 0x7fdaac0b0360 con 0x7fdaac0afeb0 2023-12-19T18:22:26.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.918+0000 7fdab1311640 1 -- 172.21.15.27:0/3277845109 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda98003520 con 0x7fdaac0afeb0 2023-12-19T18:22:26.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.918+0000 7fdab2313640 1 -- 172.21.15.27:0/3277845109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdaac15ee30 msgr2=0x7fdaac0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:26.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.918+0000 7fdab2313640 1 --2- 172.21.15.27:0/3277845109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdaac15ee30 0x7fdaac0c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.918+0000 7fdab2313640 1 -- 172.21.15.27:0/3277845109 >> v1:172.21.15.27:6789/0 conn(0x7fdaac0afeb0 legacy=0x7fdaac15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.918+0000 7fdab2313640 1 -- 172.21.15.27:0/3277845109 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaac0cb630 con 0x7fdaac1553a0 2023-12-19T18:22:26.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.920+0000 7fdab2313640 1 --2- 172.21.15.27:0/3277845109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaac1553a0 0x7fdaac0c50a0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80097a0 tx=0x7fdaa802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84c764efea4c0f78 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:26.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.920+0000 7fdab1311640 1 -- 172.21.15.27:0/3277845109 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8031780 con 0x7fdaac1553a0 2023-12-19T18:22:26.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.920+0000 7fdab1311640 1 -- 172.21.15.27:0/3277845109 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa80318e0 con 0x7fdaac1553a0 2023-12-19T18:22:26.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.920+0000 7fdab1311640 1 -- 172.21.15.27:0/3277845109 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa8031c70 con 0x7fdaac1553a0 2023-12-19T18:22:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.920+0000 7fdab3315640 1 -- 172.21.15.27:0/3277845109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaac1553a0 msgr2=0x7fdaac0c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.920+0000 7fdab3315640 1 --2- 172.21.15.27:0/3277845109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaac1553a0 0x7fdaac0c50a0 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fdaa80097a0 tx=0x7fdaa802b820 comp rx=0 tx=0).stop 2023-12-19T18:22:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.921+0000 7fdab3315640 1 -- 172.21.15.27:0/3277845109 shutdown_connections 2023-12-19T18:22:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.921+0000 7fdab3315640 1 --2- 172.21.15.27:0/3277845109 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdaac15ee30 0x7fdaac0c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.921+0000 7fdab3315640 1 --2- 172.21.15.27:0/3277845109 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaac1553a0 0x7fdaac0c50a0 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.921+0000 7fdab3315640 1 -- 172.21.15.27:0/3277845109 >> 172.21.15.27:0/3277845109 conn(0x7fdaac069c70 msgr2=0x7fdaac06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.921+0000 7fdab3315640 1 -- 172.21.15.27:0/3277845109 shutdown_connections 2023-12-19T18:22:26.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.921+0000 7fdab3315640 1 -- 172.21.15.27:0/3277845109 wait complete. 2023-12-19T18:22:26.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.923+0000 7fdab3315640 1 Processor -- start 2023-12-19T18:22:26.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.923+0000 7fdab3315640 1 -- start start 2023-12-19T18:22:26.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab3315640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdaac0afeb0 0x7fdaac0ca7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab3315640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdaac1553a0 0x7fdaac0cace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab3315640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaac15ee30 0x7fdaac143e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab3315640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac157370 con 0x7fdaac15ee30 2023-12-19T18:22:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab3315640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac0c87b0 con 0x7fdaac0afeb0 2023-12-19T18:22:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab3315640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaac1f3e10 con 0x7fdaac1553a0 2023-12-19T18:22:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab2b14640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaac15ee30 0x7fdaac143e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab2b14640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaac15ee30 0x7fdaac143e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45206/0 (socket says 172.21.15.27:45206) 2023-12-19T18:22:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab2b14640 1 -- 172.21.15.27:0/2914734409 learned_addr learned my addr 172.21.15.27:0/2914734409 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab2b14640 1 -- 172.21.15.27:0/2914734409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdaac1553a0 msgr2=0x7fdaac0cace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab2b14640 1 --2- 172.21.15.27:0/2914734409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdaac1553a0 0x7fdaac0cace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab2b14640 1 -- 172.21.15.27:0/2914734409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdaac0afeb0 msgr2=0x7fdaac0ca7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab2313640 1 --2- 172.21.15.27:0/2914734409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdaac0afeb0 0x7fdaac0ca7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab2b14640 1 --2- 172.21.15.27:0/2914734409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdaac0afeb0 0x7fdaac0ca7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:26.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.924+0000 7fdab2b14640 1 -- 172.21.15.27:0/2914734409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa8009400 con 0x7fdaac15ee30 2023-12-19T18:22:26.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.925+0000 7fdab2b14640 1 --2- 172.21.15.27:0/2914734409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaac15ee30 0x7fdaac143e50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4007d40 tx=0x7fdaa400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:26.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.925+0000 7fdaa2ffd640 1 -- 172.21.15.27:0/2914734409 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4014420 con 0x7fdaac15ee30 2023-12-19T18:22:26.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.925+0000 7fdaa2ffd640 1 -- 172.21.15.27:0/2914734409 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa40145a0 con 0x7fdaac15ee30 2023-12-19T18:22:26.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.925+0000 7fdaa2ffd640 1 -- 172.21.15.27:0/2914734409 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa4014950 con 0x7fdaac15ee30 2023-12-19T18:22:26.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.925+0000 7fdab3315640 1 -- 172.21.15.27:0/2914734409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaac1f40f0 con 0x7fdaac15ee30 2023-12-19T18:22:26.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.925+0000 7fdab3315640 1 -- 172.21.15.27:0/2914734409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaac1f46b0 con 0x7fdaac15ee30 2023-12-19T18:22:26.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.926+0000 7fdab3315640 1 -- 172.21.15.27:0/2914734409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaac1574b0 con 0x7fdaac15ee30 2023-12-19T18:22:26.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.928+0000 7fdaa2ffd640 1 -- 172.21.15.27:0/2914734409 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdaa4014ab0 con 0x7fdaac15ee30 2023-12-19T18:22:26.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.928+0000 7fdaa2ffd640 1 --2- 172.21.15.27:0/2914734409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda9006a430 0x7fda9006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:26.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.929+0000 7fdaa2ffd640 1 -- 172.21.15.27:0/2914734409 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdaa40891b0 con 0x7fdaac15ee30 2023-12-19T18:22:26.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.929+0000 7fdab2313640 1 --2- 172.21.15.27:0/2914734409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda9006a430 0x7fda9006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:26.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.930+0000 7fdab2313640 1 --2- 172.21.15.27:0/2914734409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda9006a430 0x7fda9006c8f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8036000 tx=0x7fdaa803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:26.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:26.933+0000 7fdaa2ffd640 1 -- 172.21.15.27:0/2914734409 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdaac1574b0 con 0x7fdaac15ee30 2023-12-19T18:22:27.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.163+0000 7fdab3315640 1 -- 172.21.15.27:0/2914734409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdaac1574b0 con 0x7fdaac15ee30 2023-12-19T18:22:27.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.164+0000 7fdaa2ffd640 1 -- 172.21.15.27:0/2914734409 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdaac1574b0 con 0x7fdaac15ee30 2023-12-19T18:22:27.164 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:22:27.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.167+0000 7fdab3315640 1 -- 172.21.15.27:0/2914734409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda9006a430 msgr2=0x7fda9006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:27.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.167+0000 7fdab3315640 1 --2- 172.21.15.27:0/2914734409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda9006a430 0x7fda9006c8f0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fdaa8036000 tx=0x7fdaa803d040 comp rx=0 tx=0).stop 2023-12-19T18:22:27.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.167+0000 7fdab3315640 1 -- 172.21.15.27:0/2914734409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaac15ee30 msgr2=0x7fdaac143e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:27.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.167+0000 7fdab3315640 1 --2- 172.21.15.27:0/2914734409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaac15ee30 0x7fdaac143e50 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7fdaa4007d40 tx=0x7fdaa400e4b0 comp rx=0 tx=0).stop 2023-12-19T18:22:27.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.167+0000 7fdab3315640 1 -- 172.21.15.27:0/2914734409 shutdown_connections 2023-12-19T18:22:27.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.167+0000 7fdab3315640 1 --2- 172.21.15.27:0/2914734409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda9006a430 0x7fda9006c8f0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:27.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.167+0000 7fdab3315640 1 --2- 172.21.15.27:0/2914734409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaac15ee30 0x7fdaac143e50 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:27.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.167+0000 7fdab3315640 1 --2- 172.21.15.27:0/2914734409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdaac1553a0 0x7fdaac0cace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:27.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.167+0000 7fdab3315640 1 --2- 172.21.15.27:0/2914734409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdaac0afeb0 0x7fdaac0ca7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:27.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.167+0000 7fdab3315640 1 -- 172.21.15.27:0/2914734409 >> 172.21.15.27:0/2914734409 conn(0x7fdaac069c70 msgr2=0x7fdaac15e2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:27.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.167+0000 7fdab3315640 1 -- 172.21.15.27:0/2914734409 shutdown_connections 2023-12-19T18:22:27.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:27.167+0000 7fdab3315640 1 -- 172.21.15.27:0/2914734409 wait complete. 2023-12-19T18:22:27.186 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:22:27.187 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:22:27.187 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-12-19T18:22:27.187 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:22:27.187 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:22:27.187 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:22:27.187 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:22:27.187 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:22:27.187 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:22:27.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:27.187 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-12-19T18:22:27.242 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:22:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:22:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:22:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:22:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:27.242 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:22:27.242 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:22:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-12-19T18:22:27.243 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:22:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:22:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-12-19T18:22:27.243 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:22:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:22:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-12-19T18:22:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-19T18:22:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41893sec preferred_lft 41893sec 2023-12-19T18:22:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2023-12-19T18:22:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:27.243 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:22:27.243 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::8053:76ff:fe69:c66c/64 scope link 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::781d:d6ff:feb4:7a2d/64 scope link 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether f2:29:63:cc:14:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:22:27.244 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::f029:63ff:fecc:140e/64 scope link 2023-12-19T18:22:27.245 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:27.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:27.246 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:22:27.342 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-12-19T18:22:27.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:27.346 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-12-19T18:22:27.403 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-12-19T18:22:27.403 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:22:27.403 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-19T18:22:27.403 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-19T18:22:27.403 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:22:27.404 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:27.404 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:22:27.404 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:22:27.404 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:22:27.404 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:22:27.404 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:22:27.404 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:22:27.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:27.616 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:22:27.674 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:22:27.674 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:22:27.675 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:22:27.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:27.675 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-12-19T18:22:27.783 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-19T18:22:27.784 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:22:27.784 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:22:27.842 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:22:27.842 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:22:27.842 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:22:27.899 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:22:28.015 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:22:28.016 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:22:28.127 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:22:28.132 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:22:28.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:28.132 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:22:28.195 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:22:28.195 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-19T18:22:28.202 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:22:28.297 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:22:28.297 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:22:28.381 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:22:28.384 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:22:28.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:28.385 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:22:28.411 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-12-19T18:22:28.411+0000 7f88f5f31480 -1 init, newargv = 0x557ea3f765f0 newargc=15 2023-12-19T18:22:28.412 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[109914]: starting ceph client 2023-12-19T18:22:28.421 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[109914]: starting fuse 2023-12-19T18:22:28.433 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:22:28.433 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-19T18:22:28.433 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:28.434 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:22:28.435 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:22:28.435 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:22:28.435 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:22:28.435 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:22:28.435 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:22:28.435 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:22:28.435 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:22:28.435 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:28.435 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:22:28.435 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:22:28.654 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.109914.asok 2023-12-19T18:22:28.834 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.109914.asok 2023-12-19T18:22:28.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:28.834 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.109914.asok status 2023-12-19T18:22:29.094 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:22:29.094 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:22:29.094 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:22:29.094 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:22:29.094 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:22:29.094 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:22:29.094 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:22:29.094 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "109914", 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 7521, 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 7521 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3064875531 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:29.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3064875531 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.7521 192.168.0.1:0/3064875531", 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3064875531", 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 54, 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 135, 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:22:29.096 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:22:29.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:29.110 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-12-19T18:22:29.111 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-19T18:22:29.168 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-12-19T18:22:29.169 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-19T18:22:29.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:29.169 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:22:29.279 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:22:29.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.508+0000 7f121759e640 1 Processor -- start 2023-12-19T18:22:29.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.508+0000 7f121759e640 1 -- start start 2023-12-19T18:22:29.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.510+0000 7f121759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12180cb300 0x7f12180cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:29.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.510+0000 7f121759e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f12180c8f90 0x7f1218160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:29.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.510+0000 7f121759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1218160e40 0x7f12180ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:29.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.510+0000 7f121759e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12180c4480 con 0x7f1218160e40 2023-12-19T18:22:29.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.510+0000 7f121759e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12181468c0 con 0x7f12180c8f90 2023-12-19T18:22:29.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.510+0000 7f121759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12181612d0 con 0x7f12180cb300 2023-12-19T18:22:29.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.510+0000 7f1216d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1218160e40 0x7f12180ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:29.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.510+0000 7f1216d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1218160e40 0x7f12180ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45220/0 (socket says 172.21.15.27:45220) 2023-12-19T18:22:29.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.510+0000 7f1216d9d640 1 -- 172.21.15.27:0/2204960655 learned_addr learned my addr 172.21.15.27:0/2204960655 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:29.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.511+0000 7f121659c640 1 --2- 172.21.15.27:0/2204960655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12180cb300 0x7f12180cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:29.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.511+0000 7f120ffff640 1 --2- 172.21.15.27:0/2204960655 >> v2:172.21.15.192:3300/0 conn(0x7f12180c8f90 0x7f1218160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:29.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.511+0000 7f1216d9d640 1 -- 172.21.15.27:0/2204960655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12180cb300 msgr2=0x7f12180cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:29.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.511+0000 7f1216d9d640 1 --2- 172.21.15.27:0/2204960655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12180cb300 0x7f12180cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:29.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.511+0000 7f1216d9d640 1 -- 172.21.15.27:0/2204960655 >> v2:172.21.15.192:3300/0 conn(0x7f12180c8f90 msgr2=0x7f1218160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:29.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.511+0000 7f1216d9d640 1 --2- 172.21.15.27:0/2204960655 >> v2:172.21.15.192:3300/0 conn(0x7f12180c8f90 0x7f1218160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:29.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.511+0000 7f1216d9d640 1 -- 172.21.15.27:0/2204960655 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12180cbc20 con 0x7f1218160e40 2023-12-19T18:22:29.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.512+0000 7f1216d9d640 1 --2- 172.21.15.27:0/2204960655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1218160e40 0x7f12180ac8c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f121806d540 tx=0x7f121002bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9702b57389a4f454 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:29.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.512+0000 7f1215d9b640 1 -- 172.21.15.27:0/2204960655 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121002ca70 con 0x7f1218160e40 2023-12-19T18:22:29.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.513+0000 7f1215d9b640 1 -- 172.21.15.27:0/2204960655 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f121002cbd0 con 0x7f1218160e40 2023-12-19T18:22:29.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.513+0000 7f1215d9b640 1 -- 172.21.15.27:0/2204960655 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12100343f0 con 0x7f1218160e40 2023-12-19T18:22:29.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.513+0000 7f121759e640 1 -- 172.21.15.27:0/2204960655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1218160e40 msgr2=0x7f12180ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.513+0000 7f121759e640 1 --2- 172.21.15.27:0/2204960655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1218160e40 0x7f12180ac8c0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f121806d540 tx=0x7f121002bed0 comp rx=0 tx=0).stop 2023-12-19T18:22:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.513+0000 7f121759e640 1 -- 172.21.15.27:0/2204960655 shutdown_connections 2023-12-19T18:22:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.513+0000 7f121759e640 1 --2- 172.21.15.27:0/2204960655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1218160e40 0x7f12180ac8c0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.513+0000 7f121759e640 1 --2- 172.21.15.27:0/2204960655 >> v2:172.21.15.192:3300/0 conn(0x7f12180c8f90 0x7f1218160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.513+0000 7f121759e640 1 --2- 172.21.15.27:0/2204960655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f12180cb300 0x7f12180cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.513+0000 7f121759e640 1 -- 172.21.15.27:0/2204960655 >> 172.21.15.27:0/2204960655 conn(0x7f1218069c70 msgr2=0x7f121806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.513+0000 7f121759e640 1 -- 172.21.15.27:0/2204960655 shutdown_connections 2023-12-19T18:22:29.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.513+0000 7f121759e640 1 -- 172.21.15.27:0/2204960655 wait complete. 2023-12-19T18:22:29.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.515+0000 7f121759e640 1 Processor -- start 2023-12-19T18:22:29.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.515+0000 7f121759e640 1 -- start start 2023-12-19T18:22:29.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.516+0000 7f121759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12180c8f90 0x7f12180c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:29.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.516+0000 7f121759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12180cb300 0x7f1218144a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:29.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.516+0000 7f121759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1218160e40 0x7f12180b2760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:29.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.516+0000 7f121759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12180c6be0 con 0x7f12180c8f90 2023-12-19T18:22:29.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.516+0000 7f121759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12180c5fd0 con 0x7f12180cb300 2023-12-19T18:22:29.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.516+0000 7f121759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12180c62b0 con 0x7f1218160e40 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.517+0000 7f121659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12180c8f90 0x7f12180c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.517+0000 7f121659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12180c8f90 0x7f12180c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45222/0 (socket says 172.21.15.27:45222) 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.517+0000 7f121659c640 1 -- 172.21.15.27:0/3371918686 learned_addr learned my addr 172.21.15.27:0/3371918686 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.517+0000 7f120ffff640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12180cb300 0x7f1218144a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.517+0000 7f1216d9d640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1218160e40 0x7f12180b2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.517+0000 7f121659c640 1 -- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1218160e40 msgr2=0x7f12180b2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.517+0000 7f121659c640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1218160e40 0x7f12180b2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.517+0000 7f121659c640 1 -- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12180cb300 msgr2=0x7f1218144a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.517+0000 7f121659c640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12180cb300 0x7f1218144a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.517+0000 7f121659c640 1 -- 172.21.15.27:0/3371918686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1210009b30 con 0x7f12180c8f90 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.517+0000 7f1216d9d640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1218160e40 0x7f12180b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.517+0000 7f120ffff640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12180cb300 0x7f1218144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.518+0000 7f121659c640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12180c8f90 0x7f12180c7070 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f120800d800 tx=0x7f120800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.518+0000 7f120effd640 1 -- 172.21.15.27:0/3371918686 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120800b6c0 con 0x7f12180c8f90 2023-12-19T18:22:29.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.518+0000 7f120effd640 1 -- 172.21.15.27:0/3371918686 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f120800b820 con 0x7f12180c8f90 2023-12-19T18:22:29.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.518+0000 7f121759e640 1 -- 172.21.15.27:0/3371918686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12180c6590 con 0x7f12180c8f90 2023-12-19T18:22:29.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.518+0000 7f120effd640 1 -- 172.21.15.27:0/3371918686 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120800bbb0 con 0x7f12180c8f90 2023-12-19T18:22:29.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.518+0000 7f121759e640 1 -- 172.21.15.27:0/3371918686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1218160720 con 0x7f12180c8f90 2023-12-19T18:22:29.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.519+0000 7f121759e640 1 -- 172.21.15.27:0/3371918686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f121804e840 con 0x7f12180c8f90 2023-12-19T18:22:29.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.520+0000 7f120effd640 1 -- 172.21.15.27:0/3371918686 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f120800bd10 con 0x7f12180c8f90 2023-12-19T18:22:29.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.521+0000 7f120effd640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11f806a430 0x7f11f806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:29.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.522+0000 7f120effd640 1 -- 172.21.15.27:0/3371918686 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1208088e70 con 0x7f12180c8f90 2023-12-19T18:22:29.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.522+0000 7f120ffff640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11f806a430 0x7f11f806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:29.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.523+0000 7f120ffff640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11f806a430 0x7f11f806c8f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f12180b4760 tx=0x7f1200008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:29.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.526+0000 7f120effd640 1 -- 172.21.15.27:0/3371918686 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1208054450 con 0x7f12180c8f90 2023-12-19T18:22:29.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.762+0000 7f121759e640 1 -- 172.21.15.27:0/3371918686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f12180adc50 con 0x7f12180c8f90 2023-12-19T18:22:29.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.763+0000 7f120effd640 1 -- 172.21.15.27:0/3371918686 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1208057a70 con 0x7f12180c8f90 2023-12-19T18:22:29.763 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:22:29.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.766+0000 7f121759e640 1 -- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11f806a430 msgr2=0x7f11f806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:29.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.766+0000 7f121759e640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11f806a430 0x7f11f806c8f0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f12180b4760 tx=0x7f1200008040 comp rx=0 tx=0).stop 2023-12-19T18:22:29.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.766+0000 7f121759e640 1 -- 172.21.15.27:0/3371918686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12180c8f90 msgr2=0x7f12180c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:29.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.766+0000 7f121759e640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12180c8f90 0x7f12180c7070 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f120800d800 tx=0x7f120800dcc0 comp rx=0 tx=0).stop 2023-12-19T18:22:29.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.766+0000 7f121759e640 1 -- 172.21.15.27:0/3371918686 shutdown_connections 2023-12-19T18:22:29.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.766+0000 7f121759e640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11f806a430 0x7f11f806c8f0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:29.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.766+0000 7f121759e640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1218160e40 0x7f12180b2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:29.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.766+0000 7f121759e640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f12180cb300 0x7f1218144a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:29.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.766+0000 7f121759e640 1 --2- 172.21.15.27:0/3371918686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12180c8f90 0x7f12180c7070 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:29.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.766+0000 7f121759e640 1 -- 172.21.15.27:0/3371918686 >> 172.21.15.27:0/3371918686 conn(0x7f1218069c70 msgr2=0x7f121806c450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:29.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.766+0000 7f121759e640 1 -- 172.21.15.27:0/3371918686 shutdown_connections 2023-12-19T18:22:29.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:29.766+0000 7f121759e640 1 -- 172.21.15.27:0/3371918686 wait complete. 2023-12-19T18:22:29.786 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:22:29.786 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:22:29.786 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-19T18:22:29.786 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-19T18:22:29.786 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T18:22:29.787 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:22:29.787 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:22:29.787 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:22:29.787 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:22:29.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:29.787 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-19T18:22:29.851 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:22:29.851 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:22:29.851 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:22:29.851 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:29.851 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:22:29.851 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:29.851 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:22:29.851 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 41853sec preferred_lft 41853sec 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:22:29.852 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:22:29.853 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:22:29.853 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:29.853 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::b892:1cff:fedc:4bed/64 scope link 2023-12-19T18:22:29.853 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:29.853 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:22:29.853 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether a6:e1:fc:e6:56:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:22:29.853 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::a4e1:fcff:fee6:56fa/64 scope link 2023-12-19T18:22:29.853 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:29.853 INFO:teuthology.orchestra.run.smithi192.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:22:29.853 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T18:22:29.854 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::404f:6cff:fe73:9aa2/64 scope link 2023-12-19T18:22:29.854 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:22:29.854 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:29.854 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:22:29.944 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-19T18:22:29.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:29.949 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-19T18:22:30.000 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-19T18:22:30.001 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:22:30.001 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T18:22:30.001 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T18:22:30.001 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:22:30.001 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:30.002 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:22:30.002 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:22:30.002 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T18:22:30.002 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T18:22:30.002 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T18:22:30.002 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:22:30.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:30.183 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:22:30.235 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T18:22:30.235 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-19T18:22:30.236 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:22:30.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:30.237 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-19T18:22:30.331 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-19T18:22:30.331 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:22:30.331 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-19T18:22:30.384 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:22:30.384 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:22:30.384 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:22:30.434 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:22:30.523 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:22:30.523 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:22:30.616 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:22:30.620 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:22:30.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:30.620 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:22:30.673 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:22:30.674 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-19T18:22:30.680 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:22:30.755 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:22:30.755 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:22:30.824 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:22:30.827 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:22:30.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:30.828 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:22:30.863 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-19T18:22:30.863+0000 7f2f74741480 -1 init, newargv = 0x557cba005820 newargc=15 2023-12-19T18:22:30.863 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[89975]: starting ceph client 2023-12-19T18:22:30.873 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[89975]: starting fuse 2023-12-19T18:22:31.972 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:22:32.074 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:22:32.074 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:22:32.164 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:22:32.168 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:22:32.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:32.168 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:22:32.222 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:22:32.223 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-12-19T18:22:32.223 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:22:32.223 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-19T18:22:32.223 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-19T18:22:32.223 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-19T18:22:32.223 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-19T18:22:32.223 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:22:32.223 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-19T18:22:32.223 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:22:32.224 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:22:32.225 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-19T18:22:32.225 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-19T18:22:32.403 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.89975.asok 2023-12-19T18:22:32.583 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.89975.asok 2023-12-19T18:22:32.584 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:32.584 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.89975.asok status 2023-12-19T18:22:32.825 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-19T18:22:32.825 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-19T18:22:32.825 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:22:32.825 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:22:32.825 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "89975", 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5627, 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5627 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:22:32.826 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 104978444 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 104978444 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5627 192.168.0.1:0/104978444", 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/104978444", 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 54, 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 135, 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:22:32.827 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-19T18:22:32.828 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-19T18:22:32.828 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-19T18:22:32.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:32.840 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-19T18:22:32.840 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-19T18:22:32.893 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-19T18:22:32.893 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-19T18:22:32.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:32.893 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:22:32.989 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:22:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.232+0000 7f7a747fb640 1 Processor -- start 2023-12-19T18:22:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.232+0000 7f7a747fb640 1 -- start start 2023-12-19T18:22:33.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.233+0000 7f7a747fb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7a6c06c210 0x7f7a6c06ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.234+0000 7f7a747fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a6c0cde90 0x7f7a6c06b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.234+0000 7f7a747fb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a6c0d0710 con 0x7f7a6c0affc0 2023-12-19T18:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.234+0000 7f7a747fb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a6c0cf2c0 con 0x7f7a6c06c210 2023-12-19T18:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.234+0000 7f7a747fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0cb3e0 con 0x7f7a6c0cde90 2023-12-19T18:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.234+0000 7f7a72597640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7a6c0affc0 0x7f7a6c0cd780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44506/0 (socket says 172.21.15.27:44506) 2023-12-19T18:22:33.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.234+0000 7f7a72597640 1 -- 172.21.15.27:0/1222391237 learned_addr learned my addr 172.21.15.27:0/1222391237 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:33.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.234+0000 7f7a71d96640 1 --2- 172.21.15.27:0/1222391237 >> v2:172.21.15.192:3300/0 conn(0x7f7a6c06c210 0x7f7a6c06ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:33.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.234+0000 7f7a72d98640 1 --2- 172.21.15.27:0/1222391237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a6c0cde90 0x7f7a6c06b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:33.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.234+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376073211 0 0) 0x7f7a6c0d0710 con 0x7f7a6c0affc0 2023-12-19T18:22:33.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.234+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a5c003520 con 0x7f7a6c0affc0 2023-12-19T18:22:33.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3525899653 0 0) 0x7f7a5c003520 con 0x7f7a6c0affc0 2023-12-19T18:22:33.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a6c0cde90 msgr2=0x7f7a6c06b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:33.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 --2- 172.21.15.27:0/1222391237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a6c0cde90 0x7f7a6c06b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:33.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 >> v2:172.21.15.192:3300/0 conn(0x7f7a6c06c210 msgr2=0x7f7a6c06ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:33.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 --2- 172.21.15.27:0/1222391237 >> v2:172.21.15.192:3300/0 conn(0x7f7a6c06c210 0x7f7a6c06ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:33.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c161cf0 con 0x7f7a6c0affc0 2023-12-19T18:22:33.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f7a600019f0 con 0x7f7a6c0affc0 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 >> v1:172.21.15.27:6789/0 conn(0x7f7a6c0affc0 legacy=0x7f7a6c0cd780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 --2- 172.21.15.27:0/1222391237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a5c003970 0x7f7a5c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 --2- 172.21.15.27:0/1222391237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a6c0cde90 0x7f7a5c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 --2- 172.21.15.27:0/1222391237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a5c004d00 0x7f7a5c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0cf2c0 con 0x7f7a5c003970 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c00b9b0 con 0x7f7a5c004d00 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5c00bb70 con 0x7f7a6c0cde90 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.235+0000 7f7a72597640 1 --2- 172.21.15.27:0/1222391237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a5c003970 0x7f7a5c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.236+0000 7f7a71d96640 1 --2- 172.21.15.27:0/1222391237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a6c0cde90 0x7f7a5c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.236+0000 7f7a72d98640 1 --2- 172.21.15.27:0/1222391237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a5c004d00 0x7f7a5c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.236+0000 7f7a72597640 1 -- 172.21.15.27:0/1222391237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a6c0cde90 msgr2=0x7f7a5c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.236+0000 7f7a72597640 1 --2- 172.21.15.27:0/1222391237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a6c0cde90 0x7f7a5c0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.236+0000 7f7a72597640 1 -- 172.21.15.27:0/1222391237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a5c004d00 msgr2=0x7f7a5c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:33.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.236+0000 7f7a72597640 1 --2- 172.21.15.27:0/1222391237 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a5c004d00 0x7f7a5c005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.236+0000 7f7a72597640 1 -- 172.21.15.27:0/1222391237 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5c003520 con 0x7f7a5c003970 2023-12-19T18:22:33.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.237+0000 7f7a72597640 1 --2- 172.21.15.27:0/1222391237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a5c003970 0x7f7a5c003d90 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000a860 tx=0x7f7a5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e353c44b93c84a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.237+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60034630 con 0x7f7a5c003970 2023-12-19T18:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.237+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a6002f070 con 0x7f7a5c003970 2023-12-19T18:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.237+0000 7f7a71595640 1 -- 172.21.15.27:0/1222391237 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a600349f0 con 0x7f7a5c003970 2023-12-19T18:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.238+0000 7f7a747fb640 1 -- 172.21.15.27:0/1222391237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a5c003970 msgr2=0x7f7a5c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.238+0000 7f7a747fb640 1 --2- 172.21.15.27:0/1222391237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a5c003970 0x7f7a5c003d90 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f7a6000a860 tx=0x7f7a5c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.238+0000 7f7a72d98640 1 -- 172.21.15.27:0/1222391237 reap_dead start 2023-12-19T18:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.238+0000 7f7a747fb640 1 -- 172.21.15.27:0/1222391237 shutdown_connections 2023-12-19T18:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.238+0000 7f7a747fb640 1 -- 172.21.15.27:0/1222391237 >> 172.21.15.27:0/1222391237 conn(0x7f7a6c06f1f0 msgr2=0x7f7a6c06f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.238+0000 7f7a747fb640 1 -- 172.21.15.27:0/1222391237 shutdown_connections 2023-12-19T18:22:33.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.238+0000 7f7a747fb640 1 -- 172.21.15.27:0/1222391237 wait complete. 2023-12-19T18:22:33.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.240+0000 7f7a747fb640 1 Processor -- start 2023-12-19T18:22:33.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.240+0000 7f7a747fb640 1 -- start start 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.241+0000 7f7a747fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a5c003970 0x7f7a6c158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.241+0000 7f7a747fb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a6c0ccbc0 0x7f7a6c158e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.241+0000 7f7a747fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a6c15a5b0 0x7f7a6c15aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.241+0000 7f7a747fb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c0ad920 con 0x7f7a5c003970 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.241+0000 7f7a747fb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c15afa0 con 0x7f7a6c0ccbc0 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.241+0000 7f7a747fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6c159780 con 0x7f7a6c15a5b0 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.241+0000 7f7a72597640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a5c003970 0x7f7a6c158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.241+0000 7f7a72597640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a5c003970 0x7f7a6c158210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42110/0 (socket says 172.21.15.27:42110) 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.241+0000 7f7a72597640 1 -- 172.21.15.27:0/4087296422 learned_addr learned my addr 172.21.15.27:0/4087296422 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.242+0000 7f7a71d96640 1 --2- 172.21.15.27:0/4087296422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a6c0ccbc0 0x7f7a6c158e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.242+0000 7f7a72597640 1 -- 172.21.15.27:0/4087296422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a6c15a5b0 msgr2=0x7f7a6c15aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.242+0000 7f7a72597640 1 --2- 172.21.15.27:0/4087296422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a6c15a5b0 0x7f7a6c15aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.242+0000 7f7a72597640 1 -- 172.21.15.27:0/4087296422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a6c0ccbc0 msgr2=0x7f7a6c158e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:33.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.242+0000 7f7a72597640 1 --2- 172.21.15.27:0/4087296422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a6c0ccbc0 0x7f7a6c158e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.242+0000 7f7a72597640 1 -- 172.21.15.27:0/4087296422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6c159a60 con 0x7f7a5c003970 2023-12-19T18:22:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.242+0000 7f7a72597640 1 --2- 172.21.15.27:0/4087296422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a5c003970 0x7f7a6c158210 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7a6002cc70 tx=0x7f7a60001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.242+0000 7f7a6affd640 1 -- 172.21.15.27:0/4087296422 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60005030 con 0x7f7a5c003970 2023-12-19T18:22:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.242+0000 7f7a6affd640 1 -- 172.21.15.27:0/4087296422 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a60005190 con 0x7f7a5c003970 2023-12-19T18:22:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.242+0000 7f7a6affd640 1 -- 172.21.15.27:0/4087296422 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a60005500 con 0x7f7a5c003970 2023-12-19T18:22:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.242+0000 7f7a747fb640 1 -- 172.21.15.27:0/4087296422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6c159d40 con 0x7f7a5c003970 2023-12-19T18:22:33.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.242+0000 7f7a747fb640 1 -- 172.21.15.27:0/4087296422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6c158750 con 0x7f7a5c003970 2023-12-19T18:22:33.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.244+0000 7f7a747fb640 1 -- 172.21.15.27:0/4087296422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6c067110 con 0x7f7a5c003970 2023-12-19T18:22:33.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.245+0000 7f7a6affd640 1 -- 172.21.15.27:0/4087296422 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7a60031030 con 0x7f7a5c003970 2023-12-19T18:22:33.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.245+0000 7f7a6affd640 1 --2- 172.21.15.27:0/4087296422 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a5006a460 0x7f7a5006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:33.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.246+0000 7f7a6affd640 1 -- 172.21.15.27:0/4087296422 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a600aadd0 con 0x7f7a5c003970 2023-12-19T18:22:33.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.246+0000 7f7a71d96640 1 --2- 172.21.15.27:0/4087296422 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a5006a460 0x7f7a5006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:33.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.247+0000 7f7a71d96640 1 --2- 172.21.15.27:0/4087296422 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a5006a460 0x7f7a5006c920 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0c8860 tx=0x7f7a58008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:33.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.250+0000 7f7a6affd640 1 -- 172.21.15.27:0/4087296422 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7a60079e40 con 0x7f7a5c003970 2023-12-19T18:22:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.484+0000 7f7a747fb640 1 -- 172.21.15.27:0/4087296422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7a6c067110 con 0x7f7a5c003970 2023-12-19T18:22:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.486+0000 7f7a6affd640 1 -- 172.21.15.27:0/4087296422 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f7a600799d0 con 0x7f7a5c003970 2023-12-19T18:22:33.486 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:33.486 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2713867269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2713867269},{"type":"v1","addr":"172.21.15.192:6837","nonce":2713867269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":5534,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/919878747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":919878747},{"type":"v1","addr":"172.21.15.27:6835","nonce":919878747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6160,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/861111763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":861111763},{"type":"v1","addr":"172.21.15.192:6836","nonce":861111763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:22:14.217224+0000","modified":"2023-12-19T18:22:17.743185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7320":{"gid":7320,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3670863819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3670863819},{"type":"v1","addr":"172.21.15.27:6834","nonce":3670863819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T18:22:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 54 2023-12-19T18:22:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.488+0000 7f7a747fb640 1 -- 172.21.15.27:0/4087296422 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a5006a460 msgr2=0x7f7a5006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.488+0000 7f7a747fb640 1 --2- 172.21.15.27:0/4087296422 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a5006a460 0x7f7a5006c920 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7a6c0c8860 tx=0x7f7a58008040 comp rx=0 tx=0).stop 2023-12-19T18:22:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.488+0000 7f7a747fb640 1 -- 172.21.15.27:0/4087296422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a5c003970 msgr2=0x7f7a6c158210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.488+0000 7f7a747fb640 1 --2- 172.21.15.27:0/4087296422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a5c003970 0x7f7a6c158210 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f7a6002cc70 tx=0x7f7a60001d80 comp rx=0 tx=0).stop 2023-12-19T18:22:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.488+0000 7f7a747fb640 1 -- 172.21.15.27:0/4087296422 shutdown_connections 2023-12-19T18:22:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.488+0000 7f7a747fb640 1 --2- 172.21.15.27:0/4087296422 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7a5006a460 0x7f7a5006c920 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.488+0000 7f7a747fb640 1 --2- 172.21.15.27:0/4087296422 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a6c15a5b0 0x7f7a6c15aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.488+0000 7f7a747fb640 1 --2- 172.21.15.27:0/4087296422 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a6c0ccbc0 0x7f7a6c158e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.488+0000 7f7a747fb640 1 --2- 172.21.15.27:0/4087296422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a5c003970 0x7f7a6c158210 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.488+0000 7f7a747fb640 1 -- 172.21.15.27:0/4087296422 >> 172.21.15.27:0/4087296422 conn(0x7f7a6c06f1f0 msgr2=0x7f7a6c0c75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.488+0000 7f7a747fb640 1 -- 172.21.15.27:0/4087296422 shutdown_connections 2023-12-19T18:22:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:33.489+0000 7f7a747fb640 1 -- 172.21.15.27:0/4087296422 wait complete. 2023-12-19T18:22:33.510 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-12-19T18:22:33.757 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":7320,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":19.519582098000001,"mdsmap_epoch":54,"osdmap_epoch":135,"osdmap_epoch_barrier":135,"uptime":24.946064433} 2023-12-19T18:22:33.758 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 7320, "mdsmap_epoch": 54, "osdmap_epoch": 135, "osdmap_epoch_barrier": 135, "rank_uptime": 19.519582098, "state": "up:active", "uptime": 24.946064433, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:22:33.758 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-12-19T18:22:34.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.000+0000 7fcecd726640 1 Processor -- start 2023-12-19T18:22:34.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.001+0000 7fcecd726640 1 -- start start 2023-12-19T18:22:34.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.002+0000 7fcecd726640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcec80c55b0 0x7fcec80cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:34.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.002+0000 7fcecd726640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcec80ce520 0x7fcec80cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.002+0000 7fcecd726640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcec80cb100 con 0x7fcec8161030 2023-12-19T18:22:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.003+0000 7fcecd726640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcec80c95f0 con 0x7fcec80c55b0 2023-12-19T18:22:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.003+0000 7fcecd726640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80cae20 con 0x7fcec80ce520 2023-12-19T18:22:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.003+0000 7fcec77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcec80c55b0 0x7fcec80cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.003+0000 7fcec7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcec8161030 0x7fcec81657c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44510/0 (socket says 172.21.15.27:44510) 2023-12-19T18:22:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.003+0000 7fcec7fff640 1 -- 172.21.15.27:0/767492978 learned_addr learned my addr 172.21.15.27:0/767492978 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.003+0000 7fceccf25640 1 --2- 172.21.15.27:0/767492978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcec80ce520 0x7fcec80cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.003+0000 7fcec77fe640 1 -- 172.21.15.27:0/767492978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcec80ce520 msgr2=0x7fcec80cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.003+0000 7fcec6ffd640 1 -- 172.21.15.27:0/767492978 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436839148 0 0) 0x7fcec80cb100 con 0x7fcec8161030 2023-12-19T18:22:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.003+0000 7fcec77fe640 1 --2- 172.21.15.27:0/767492978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcec80ce520 0x7fcec80cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.003+0000 7fcec77fe640 1 -- 172.21.15.27:0/767492978 >> v1:172.21.15.27:6789/0 conn(0x7fcec8161030 legacy=0x7fcec81657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.004+0000 7fcec77fe640 1 -- 172.21.15.27:0/767492978 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec80c9030 con 0x7fcec80c55b0 2023-12-19T18:22:34.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.004+0000 7fceccf25640 1 --2- 172.21.15.27:0/767492978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcec80ce520 0x7fcec80cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:34.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.005+0000 7fcec77fe640 1 --2- 172.21.15.27:0/767492978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcec80c55b0 0x7fcec80cb350 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0097a0 tx=0x7fcebc02b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f55ff6170b5167 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:34.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.005+0000 7fcec6ffd640 1 -- 172.21.15.27:0/767492978 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc02d040 con 0x7fcec80c55b0 2023-12-19T18:22:34.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.005+0000 7fcec6ffd640 1 -- 172.21.15.27:0/767492978 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcebc031810 con 0x7fcec80c55b0 2023-12-19T18:22:34.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.005+0000 7fcec6ffd640 1 -- 172.21.15.27:0/767492978 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcebc031ba0 con 0x7fcec80c55b0 2023-12-19T18:22:34.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.005+0000 7fcecd726640 1 -- 172.21.15.27:0/767492978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcec80c55b0 msgr2=0x7fcec80cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:34.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.005+0000 7fcecd726640 1 --2- 172.21.15.27:0/767492978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcec80c55b0 0x7fcec80cb350 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7fcebc0097a0 tx=0x7fcebc02b8b0 comp rx=0 tx=0).stop 2023-12-19T18:22:34.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.006+0000 7fcecd726640 1 -- 172.21.15.27:0/767492978 shutdown_connections 2023-12-19T18:22:34.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.006+0000 7fcecd726640 1 --2- 172.21.15.27:0/767492978 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcec80ce520 0x7fcec80cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:34.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.006+0000 7fcecd726640 1 --2- 172.21.15.27:0/767492978 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcec80c55b0 0x7fcec80cb350 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:34.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.006+0000 7fcecd726640 1 -- 172.21.15.27:0/767492978 >> 172.21.15.27:0/767492978 conn(0x7fcec80b05b0 msgr2=0x7fcec80b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:34.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.006+0000 7fcecd726640 1 -- 172.21.15.27:0/767492978 shutdown_connections 2023-12-19T18:22:34.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.006+0000 7fcecd726640 1 -- 172.21.15.27:0/767492978 wait complete. 2023-12-19T18:22:34.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.007+0000 7fcecd726640 1 Processor -- start 2023-12-19T18:22:34.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.008+0000 7fcecd726640 1 -- start start 2023-12-19T18:22:34.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.008+0000 7fcecd726640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcec80c55b0 0x7fcec806beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:34.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.008+0000 7fcecd726640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcec80ce520 0x7fcec806c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:34.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fcecd726640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcec8161030 0x7fcec815b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:34.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fcecd726640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80c74b0 con 0x7fcec8161030 2023-12-19T18:22:34.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fcecd726640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80c7790 con 0x7fcec80ce520 2023-12-19T18:22:34.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fcecd726640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcec80c7a70 con 0x7fcec80c55b0 2023-12-19T18:22:34.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fceccf25640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcec8161030 0x7fcec815b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:34.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fceccf25640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcec8161030 0x7fcec815b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42122/0 (socket says 172.21.15.27:42122) 2023-12-19T18:22:34.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fceccf25640 1 -- 172.21.15.27:0/1315241416 learned_addr learned my addr 172.21.15.27:0/1315241416 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fcec7fff640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcec80c55b0 0x7fcec806beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fcec77fe640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcec80ce520 0x7fcec806c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fceccf25640 1 -- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcec80c55b0 msgr2=0x7fcec806beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fceccf25640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcec80c55b0 0x7fcec806beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fceccf25640 1 -- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcec80ce520 msgr2=0x7fcec806c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fceccf25640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcec80ce520 0x7fcec806c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.009+0000 7fceccf25640 1 -- 172.21.15.27:0/1315241416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcebc009400 con 0x7fcec8161030 2023-12-19T18:22:34.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.010+0000 7fceccf25640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcec8161030 0x7fcec815b870 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fcec000cef0 tx=0x7fcec000a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:34.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.010+0000 7fcec4ff9640 1 -- 172.21.15.27:0/1315241416 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec000d840 con 0x7fcec8161030 2023-12-19T18:22:34.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.010+0000 7fcec4ff9640 1 -- 172.21.15.27:0/1315241416 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcec000d9c0 con 0x7fcec8161030 2023-12-19T18:22:34.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.010+0000 7fcec4ff9640 1 -- 172.21.15.27:0/1315241416 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcec000dd70 con 0x7fcec8161030 2023-12-19T18:22:34.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.010+0000 7fcecd726640 1 -- 172.21.15.27:0/1315241416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec80c7d50 con 0x7fcec8161030 2023-12-19T18:22:34.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.010+0000 7fcecd726640 1 -- 172.21.15.27:0/1315241416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec814d090 con 0x7fcec8161030 2023-12-19T18:22:34.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.011+0000 7fcecd726640 1 -- 172.21.15.27:0/1315241416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec804e840 con 0x7fcec8161030 2023-12-19T18:22:34.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.012+0000 7fcec4ff9640 1 -- 172.21.15.27:0/1315241416 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcec001e3b0 con 0x7fcec8161030 2023-12-19T18:22:34.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.013+0000 7fcec4ff9640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fceb406a430 0x7fceb406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:34.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.013+0000 7fcec4ff9640 1 -- 172.21.15.27:0/1315241416 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcec0089480 con 0x7fcec8161030 2023-12-19T18:22:34.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.014+0000 7fcec7fff640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fceb406a430 0x7fceb406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:34.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.015+0000 7fcec7fff640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fceb406a430 0x7fceb406c8f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fceb8000f60 tx=0x7fceb8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:34.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.017+0000 7fcec4ff9640 1 -- 172.21.15.27:0/1315241416 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcec0054a90 con 0x7fcec8161030 2023-12-19T18:22:34.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.217+0000 7fcecd726640 1 -- 172.21.15.27:0/1315241416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fcec80c4fd0 con 0x7fcec8161030 2023-12-19T18:22:34.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.706+0000 7fcec4ff9640 1 -- 172.21.15.27:0/1315241416 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v136) v1 ==== 181+0+0 (secure 0 0 0) 0x7fcec00580b0 con 0x7fcec8161030 2023-12-19T18:22:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-12-19T18:22:34.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.709+0000 7fcecd726640 1 -- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fceb406a430 msgr2=0x7fceb406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:34.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.709+0000 7fcecd726640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fceb406a430 0x7fceb406c8f0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fceb8000f60 tx=0x7fceb8009040 comp rx=0 tx=0).stop 2023-12-19T18:22:34.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.710+0000 7fcecd726640 1 -- 172.21.15.27:0/1315241416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcec8161030 msgr2=0x7fcec815b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:34.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.710+0000 7fcecd726640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcec8161030 0x7fcec815b870 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7fcec000cef0 tx=0x7fcec000a710 comp rx=0 tx=0).stop 2023-12-19T18:22:34.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.710+0000 7fcecd726640 1 -- 172.21.15.27:0/1315241416 shutdown_connections 2023-12-19T18:22:34.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.710+0000 7fcecd726640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fceb406a430 0x7fceb406c8f0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:34.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.710+0000 7fcecd726640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcec8161030 0x7fcec815b870 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:34.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.710+0000 7fcecd726640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcec80ce520 0x7fcec806c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:34.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.710+0000 7fcecd726640 1 --2- 172.21.15.27:0/1315241416 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcec80c55b0 0x7fcec806beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:34.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.710+0000 7fcecd726640 1 -- 172.21.15.27:0/1315241416 >> 172.21.15.27:0/1315241416 conn(0x7fcec80b05b0 msgr2=0x7fcec80aedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:34.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.710+0000 7fcecd726640 1 -- 172.21.15.27:0/1315241416 shutdown_connections 2023-12-19T18:22:34.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:34.710+0000 7fcecd726640 1 -- 172.21.15.27:0/1315241416 wait complete. 2023-12-19T18:22:34.730 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:22:34.730 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:22:34.730 DEBUG:teuthology.orchestra.run.smithi027:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-12-19T18:22:34.730 DEBUG:teuthology.orchestra.run.smithi027:> os.fchmod(fd, 0O666) 2023-12-19T18:22:34.730 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(fd) 2023-12-19T18:22:34.731 DEBUG:teuthology.orchestra.run.smithi027:> os.close(fd) 2023-12-19T18:22:34.731 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:22:35.049 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:22:35.049 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:22:35.049 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:22:35.049 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:22:35.049 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:22:35.049 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:35.049 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:22:35.049 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:22:35.049 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:22:35.049 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:22:35.049 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:35.049 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:22:35.049 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:22:35.050 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:22:35.307 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.109914.asok 2023-12-19T18:22:35.486 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.109914.asok 2023-12-19T18:22:35.486 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:35.486 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.109914.asok status 2023-12-19T18:22:35.747 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "109914", 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 1, 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 7521, 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:22:35.748 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 7521 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3064875531 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3064875531 2023-12-19T18:22:35.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:35.750 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.7521 192.168.0.1:0/3064875531", 2023-12-19T18:22:35.750 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3064875531", 2023-12-19T18:22:35.750 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 2, 2023-12-19T18:22:35.750 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 54, 2023-12-19T18:22:35.750 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 135, 2023-12-19T18:22:35.750 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 135, 2023-12-19T18:22:35.750 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:22:35.750 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:22:35.750 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:22:35.763 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set pause 2023-12-19T18:22:36.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.004+0000 7fbb8b2f8640 1 Processor -- start 2023-12-19T18:22:36.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.004+0000 7fbb8b2f8640 1 -- start start 2023-12-19T18:22:36.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.006+0000 7fbb8b2f8640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbb8415adf0 0x7fbb8406d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.006+0000 7fbb8b2f8640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8406b520 con 0x7fbb84070c00 2023-12-19T18:22:36.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.006+0000 7fbb8b2f8640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8415c1e0 con 0x7fbb8406fc10 2023-12-19T18:22:36.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.006+0000 7fbb8b2f8640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb840703c0 con 0x7fbb8415adf0 2023-12-19T18:22:36.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.006+0000 7fbb88893640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbb84070c00 0x7fbb8406f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44516/0 (socket says 172.21.15.27:44516) 2023-12-19T18:22:36.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.006+0000 7fbb88893640 1 -- 172.21.15.27:0/1787223775 learned_addr learned my addr 172.21.15.27:0/1787223775 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:36.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.007+0000 7fbb89094640 1 --2- 172.21.15.27:0/1787223775 >> v2:172.21.15.192:3300/0 conn(0x7fbb8415adf0 0x7fbb8406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.007+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557043914 0 0) 0x7fbb8406b520 con 0x7fbb84070c00 2023-12-19T18:22:36.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.007+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb70003520 con 0x7fbb84070c00 2023-12-19T18:22:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.007+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 329093749 0 0) 0x7fbb70003520 con 0x7fbb84070c00 2023-12-19T18:22:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.007+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 >> v2:172.21.15.192:3300/0 conn(0x7fbb8415adf0 msgr2=0x7fbb8406d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.007+0000 7fbb7bfff640 1 --2- 172.21.15.27:0/1787223775 >> v2:172.21.15.192:3300/0 conn(0x7fbb8415adf0 0x7fbb8406d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.007+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 >> v1:172.21.15.192:6789/0 conn(0x7fbb8406fc10 legacy=0x7fbb840c91a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.007+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8415f3a0 con 0x7fbb84070c00 2023-12-19T18:22:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.007+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fbb74001670 con 0x7fbb84070c00 2023-12-19T18:22:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.007+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 >> v1:172.21.15.27:6789/0 conn(0x7fbb84070c00 legacy=0x7fbb8406f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.007+0000 7fbb7bfff640 1 --2- 172.21.15.27:0/1787223775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb700039c0 0x7fbb70003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.007+0000 7fbb7bfff640 1 --2- 172.21.15.27:0/1787223775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb700047c0 0x7fbb70004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.007+0000 7fbb7bfff640 1 --2- 172.21.15.27:0/1787223775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb700051d0 0x7fbb7000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7000c0f0 con 0x7fbb700039c0 2023-12-19T18:22:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7000c260 con 0x7fbb700047c0 2023-12-19T18:22:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7000c420 con 0x7fbb700051d0 2023-12-19T18:22:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb89094640 1 --2- 172.21.15.27:0/1787223775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb700039c0 0x7fbb70003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb88893640 1 --2- 172.21.15.27:0/1787223775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb700047c0 0x7fbb70004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb89895640 1 --2- 172.21.15.27:0/1787223775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb700051d0 0x7fbb7000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb89094640 1 -- 172.21.15.27:0/1787223775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb700051d0 msgr2=0x7fbb7000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb89094640 1 --2- 172.21.15.27:0/1787223775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb700051d0 0x7fbb7000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb89094640 1 -- 172.21.15.27:0/1787223775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb700047c0 msgr2=0x7fbb70004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb89094640 1 --2- 172.21.15.27:0/1787223775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb700047c0 0x7fbb70004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb89094640 1 -- 172.21.15.27:0/1787223775 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb70003520 con 0x7fbb700039c0 2023-12-19T18:22:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb89895640 1 --2- 172.21.15.27:0/1787223775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb700051d0 0x7fbb7000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb88893640 1 --2- 172.21.15.27:0/1787223775 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb700047c0 0x7fbb70004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:36.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.008+0000 7fbb89895640 1 -- 172.21.15.27:0/1787223775 reap_dead start 2023-12-19T18:22:36.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.009+0000 7fbb89094640 1 --2- 172.21.15.27:0/1787223775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb700039c0 0x7fbb70003de0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009a90 tx=0x7fbb700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e49a477a072fd3e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:36.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.010+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8002c680 con 0x7fbb700039c0 2023-12-19T18:22:36.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.010+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb8002c7e0 con 0x7fbb700039c0 2023-12-19T18:22:36.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.010+0000 7fbb7bfff640 1 -- 172.21.15.27:0/1787223775 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb8002cb70 con 0x7fbb700039c0 2023-12-19T18:22:36.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.010+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1787223775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb700039c0 msgr2=0x7fbb70003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:36.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.010+0000 7fbb8b2f8640 1 --2- 172.21.15.27:0/1787223775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb700039c0 0x7fbb70003de0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7fbb80009a90 tx=0x7fbb700057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.010+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1787223775 shutdown_connections 2023-12-19T18:22:36.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.010+0000 7fbb8b2f8640 1 --2- 172.21.15.27:0/1787223775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb700039c0 0x7fbb70003de0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.010+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1787223775 >> 172.21.15.27:0/1787223775 conn(0x7fbb840ceb50 msgr2=0x7fbb840cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:36.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.010+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1787223775 shutdown_connections 2023-12-19T18:22:36.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.011+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1787223775 wait complete. 2023-12-19T18:22:36.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.012+0000 7fbb8b2f8640 1 Processor -- start 2023-12-19T18:22:36.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.013+0000 7fbb8b2f8640 1 -- start start 2023-12-19T18:22:36.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.013+0000 7fbb8b2f8640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb700039c0 0x7fbb840c2710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.013+0000 7fbb8b2f8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb840c87c0 0x7fbb840c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.013+0000 7fbb8b2f8640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb840c7200 0x7fbb840c76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.013+0000 7fbb8b2f8640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840c5a10 con 0x7fbb840c87c0 2023-12-19T18:22:36.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.013+0000 7fbb8b2f8640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb841ee0f0 con 0x7fbb840c7200 2023-12-19T18:22:36.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.013+0000 7fbb8b2f8640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb840c8d30 con 0x7fbb700039c0 2023-12-19T18:22:36.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.014+0000 7fbb88893640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb840c87c0 0x7fbb840c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.014+0000 7fbb88893640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb840c87c0 0x7fbb840c6cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42140/0 (socket says 172.21.15.27:42140) 2023-12-19T18:22:36.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.014+0000 7fbb88893640 1 -- 172.21.15.27:0/1262491143 learned_addr learned my addr 172.21.15.27:0/1262491143 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.014+0000 7fbb89094640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb700039c0 0x7fbb840c2710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.014+0000 7fbb89895640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb840c7200 0x7fbb840c76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.014+0000 7fbb88893640 1 -- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb700039c0 msgr2=0x7fbb840c2710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.014+0000 7fbb88893640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb700039c0 0x7fbb840c2710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.014+0000 7fbb88893640 1 -- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb840c7200 msgr2=0x7fbb840c76b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.014+0000 7fbb88893640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb840c7200 0x7fbb840c76b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.014+0000 7fbb88893640 1 -- 172.21.15.27:0/1262491143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb840c4f30 con 0x7fbb840c87c0 2023-12-19T18:22:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.014+0000 7fbb89895640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb840c7200 0x7fbb840c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.015+0000 7fbb88893640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb840c87c0 0x7fbb840c6cc0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400d810 tx=0x7fbb7400dcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.015+0000 7fbb79ffb640 1 -- 172.21.15.27:0/1262491143 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb740074e0 con 0x7fbb840c87c0 2023-12-19T18:22:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.015+0000 7fbb79ffb640 1 -- 172.21.15.27:0/1262491143 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb74007640 con 0x7fbb840c87c0 2023-12-19T18:22:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.015+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1262491143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb840c5210 con 0x7fbb840c87c0 2023-12-19T18:22:36.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.015+0000 7fbb79ffb640 1 -- 172.21.15.27:0/1262491143 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74004a00 con 0x7fbb840c87c0 2023-12-19T18:22:36.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.015+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1262491143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb840c2c50 con 0x7fbb840c87c0 2023-12-19T18:22:36.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.016+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1262491143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb840c2e70 con 0x7fbb840c87c0 2023-12-19T18:22:36.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.017+0000 7fbb79ffb640 1 -- 172.21.15.27:0/1262491143 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbb74004b60 con 0x7fbb840c87c0 2023-12-19T18:22:36.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.018+0000 7fbb79ffb640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb6806a460 0x7fbb6806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.018+0000 7fbb79ffb640 1 -- 172.21.15.27:0/1262491143 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb74088770 con 0x7fbb840c87c0 2023-12-19T18:22:36.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.019+0000 7fbb89094640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb6806a460 0x7fbb6806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.020+0000 7fbb89094640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb6806a460 0x7fbb6806c920 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbb80036000 tx=0x7fbb80030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:36.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.023+0000 7fbb79ffb640 1 -- 172.21.15.27:0/1262491143 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbb74053c30 con 0x7fbb840c87c0 2023-12-19T18:22:36.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.227+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1262491143 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd set", "key": "pause"} v 0) v1 -- 0x7fbb84073770 con 0x7fbb840c87c0 2023-12-19T18:22:36.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.718+0000 7fbb79ffb640 1 -- 172.21.15.27:0/1262491143 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd set", "key": "pause"}]=0 pauserd,pausewr is set v137) v1 ==== 93+0+0 (secure 0 0 0) 0x7fbb74019020 con 0x7fbb840c87c0 2023-12-19T18:22:36.719 INFO:teuthology.orchestra.run.smithi027.stderr:pauserd,pausewr is set 2023-12-19T18:22:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.722+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb6806a460 msgr2=0x7fbb6806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.722+0000 7fbb8b2f8640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb6806a460 0x7fbb6806c920 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fbb80036000 tx=0x7fbb80030000 comp rx=0 tx=0).stop 2023-12-19T18:22:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.722+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1262491143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb840c87c0 msgr2=0x7fbb840c6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:36.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.722+0000 7fbb8b2f8640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb840c87c0 0x7fbb840c6cc0 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fbb7400d810 tx=0x7fbb7400dcd0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.722+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1262491143 shutdown_connections 2023-12-19T18:22:36.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.722+0000 7fbb8b2f8640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb6806a460 0x7fbb6806c920 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.722+0000 7fbb8b2f8640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb840c7200 0x7fbb840c76b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.722+0000 7fbb8b2f8640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb840c87c0 0x7fbb840c6cc0 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.722+0000 7fbb8b2f8640 1 --2- 172.21.15.27:0/1262491143 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb700039c0 0x7fbb840c2710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.722+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1262491143 >> 172.21.15.27:0/1262491143 conn(0x7fbb840ceb50 msgr2=0x7fbb8415a000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:36.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.722+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1262491143 shutdown_connections 2023-12-19T18:22:36.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.723+0000 7fbb8b2f8640 1 -- 172.21.15.27:0/1262491143 wait complete. 2023-12-19T18:22:36.745 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset pause 2023-12-19T18:22:36.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.976+0000 7fea35f30640 1 Processor -- start 2023-12-19T18:22:36.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.976+0000 7fea35f30640 1 -- start start 2023-12-19T18:22:36.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.977+0000 7fea35f30640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fea300b49d0 0x7fea30160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.978+0000 7fea35f30640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea300b5ca0 con 0x7fea3015cf90 2023-12-19T18:22:36.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.978+0000 7fea35f30640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea30161800 con 0x7fea300b89b0 2023-12-19T18:22:36.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.978+0000 7fea35f30640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea30161240 con 0x7fea300b49d0 2023-12-19T18:22:36.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.978+0000 7fea3572f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fea3015cf90 0x7fea300d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44522/0 (socket says 172.21.15.27:44522) 2023-12-19T18:22:36.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.978+0000 7fea3572f640 1 -- 172.21.15.27:0/4255511310 learned_addr learned my addr 172.21.15.27:0/4255511310 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:36.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.978+0000 7fea34f2e640 1 --2- 172.21.15.27:0/4255511310 >> v2:172.21.15.192:3300/0 conn(0x7fea300b49d0 0x7fea30160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288378297 0 0) 0x7fea300b5ca0 con 0x7fea3015cf90 2023-12-19T18:22:36.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea18003520 con 0x7fea3015cf90 2023-12-19T18:22:36.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1234571505 0 0) 0x7fea18003520 con 0x7fea3015cf90 2023-12-19T18:22:36.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 >> v2:172.21.15.192:3300/0 conn(0x7fea300b49d0 msgr2=0x7fea30160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:36.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 --2- 172.21.15.27:0/4255511310 >> v2:172.21.15.192:3300/0 conn(0x7fea300b49d0 0x7fea30160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 >> v1:172.21.15.192:6789/0 conn(0x7fea300b89b0 legacy=0x7fea3015c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:36.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea30161520 con 0x7fea3015cf90 2023-12-19T18:22:36.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fea280019f0 con 0x7fea3015cf90 2023-12-19T18:22:36.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 >> v1:172.21.15.27:6789/0 conn(0x7fea3015cf90 legacy=0x7fea300d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:36.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 --2- 172.21.15.27:0/4255511310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea180039c0 0x7fea18003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 --2- 172.21.15.27:0/4255511310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea180047c0 0x7fea18004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 --2- 172.21.15.27:0/4255511310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea18005180 0x7fea1800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea30161240 con 0x7fea180039c0 2023-12-19T18:22:36.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea1800c060 con 0x7fea180047c0 2023-12-19T18:22:36.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea1800c220 con 0x7fea18005180 2023-12-19T18:22:36.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.979+0000 7fea34f2e640 1 --2- 172.21.15.27:0/4255511310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea180039c0 0x7fea18003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.980+0000 7fea34f2e640 1 -- 172.21.15.27:0/4255511310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea18005180 msgr2=0x7fea1800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:36.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.980+0000 7fea34f2e640 1 --2- 172.21.15.27:0/4255511310 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea18005180 0x7fea1800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.980+0000 7fea34f2e640 1 -- 172.21.15.27:0/4255511310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea180047c0 msgr2=0x7fea18004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:36.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.980+0000 7fea2ffff640 1 --2- 172.21.15.27:0/4255511310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea180047c0 0x7fea18004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.980+0000 7fea34f2e640 1 --2- 172.21.15.27:0/4255511310 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea180047c0 0x7fea18004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.980+0000 7fea3572f640 1 -- 172.21.15.27:0/4255511310 reap_dead start 2023-12-19T18:22:36.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.980+0000 7fea34f2e640 1 -- 172.21.15.27:0/4255511310 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea18003520 con 0x7fea180039c0 2023-12-19T18:22:36.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.981+0000 7fea34f2e640 1 --2- 172.21.15.27:0/4255511310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea180039c0 0x7fea18003de0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fea20009ab0 tx=0x7fea180057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e48bef712bfdc63b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:36.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.981+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20005990 con 0x7fea180039c0 2023-12-19T18:22:36.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.981+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea20005af0 con 0x7fea180039c0 2023-12-19T18:22:36.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.981+0000 7fea2f7fe640 1 -- 172.21.15.27:0/4255511310 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea20005ea0 con 0x7fea180039c0 2023-12-19T18:22:36.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.982+0000 7fea35f30640 1 -- 172.21.15.27:0/4255511310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea180039c0 msgr2=0x7fea18003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:36.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.982+0000 7fea35f30640 1 --2- 172.21.15.27:0/4255511310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea180039c0 0x7fea18003de0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fea20009ab0 tx=0x7fea180057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.982+0000 7fea35f30640 1 -- 172.21.15.27:0/4255511310 shutdown_connections 2023-12-19T18:22:36.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.982+0000 7fea35f30640 1 --2- 172.21.15.27:0/4255511310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea180039c0 0x7fea18003de0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.982+0000 7fea35f30640 1 -- 172.21.15.27:0/4255511310 >> 172.21.15.27:0/4255511310 conn(0x7fea300cf760 msgr2=0x7fea300cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:36.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.982+0000 7fea35f30640 1 -- 172.21.15.27:0/4255511310 shutdown_connections 2023-12-19T18:22:36.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.982+0000 7fea35f30640 1 -- 172.21.15.27:0/4255511310 wait complete. 2023-12-19T18:22:36.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.984+0000 7fea35f30640 1 Processor -- start 2023-12-19T18:22:36.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.984+0000 7fea35f30640 1 -- start start 2023-12-19T18:22:36.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.985+0000 7fea35f30640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea180039c0 0x7fea30154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.985+0000 7fea35f30640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea301f2690 0x7fea3015e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.985+0000 7fea35f30640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea301f2b10 0x7fea3015ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.985+0000 7fea35f30640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea30159b10 con 0x7fea301f2b10 2023-12-19T18:22:36.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.985+0000 7fea35f30640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea30154660 con 0x7fea301f2690 2023-12-19T18:22:36.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.985+0000 7fea35f30640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea300cb440 con 0x7fea180039c0 2023-12-19T18:22:36.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.986+0000 7fea3572f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea301f2b10 0x7fea3015ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.986+0000 7fea3572f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea301f2b10 0x7fea3015ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42162/0 (socket says 172.21.15.27:42162) 2023-12-19T18:22:36.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.986+0000 7fea3572f640 1 -- 172.21.15.27:0/398356890 learned_addr learned my addr 172.21.15.27:0/398356890 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:36.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.986+0000 7fea2ffff640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea301f2690 0x7fea3015e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.986+0000 7fea34f2e640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea180039c0 0x7fea30154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.986+0000 7fea3572f640 1 -- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea180039c0 msgr2=0x7fea30154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:36.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.986+0000 7fea3572f640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea180039c0 0x7fea30154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.986+0000 7fea3572f640 1 -- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea301f2690 msgr2=0x7fea3015e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:36.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.986+0000 7fea3572f640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea301f2690 0x7fea3015e9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:36.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.986+0000 7fea3572f640 1 -- 172.21.15.27:0/398356890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea300cb720 con 0x7fea301f2b10 2023-12-19T18:22:36.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.986+0000 7fea3572f640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea301f2b10 0x7fea3015ef30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fea300ceda0 tx=0x7fea2800ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:36.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.987+0000 7fea2d7fa640 1 -- 172.21.15.27:0/398356890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2800a610 con 0x7fea301f2b10 2023-12-19T18:22:36.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.987+0000 7fea2d7fa640 1 -- 172.21.15.27:0/398356890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2800a7b0 con 0x7fea301f2b10 2023-12-19T18:22:36.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.987+0000 7fea2d7fa640 1 -- 172.21.15.27:0/398356890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2800ab60 con 0x7fea301f2b10 2023-12-19T18:22:36.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.987+0000 7fea35f30640 1 -- 172.21.15.27:0/398356890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea300cba00 con 0x7fea301f2b10 2023-12-19T18:22:36.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.987+0000 7fea35f30640 1 -- 172.21.15.27:0/398356890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea300cbfc0 con 0x7fea301f2b10 2023-12-19T18:22:36.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.988+0000 7fea35f30640 1 -- 172.21.15.27:0/398356890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea300cc1b0 con 0x7fea301f2b10 2023-12-19T18:22:36.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.989+0000 7fea2d7fa640 1 -- 172.21.15.27:0/398356890 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fea2800acc0 con 0x7fea301f2b10 2023-12-19T18:22:36.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.990+0000 7fea2d7fa640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea1c06a460 0x7fea1c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:36.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.990+0000 7fea2d7fa640 1 -- 172.21.15.27:0/398356890 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fea280881c0 con 0x7fea301f2b10 2023-12-19T18:22:36.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.991+0000 7fea2d7fa640 1 -- 172.21.15.27:0/398356890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=138+}) v3 -- 0x7fea1c071f90 con 0x7fea301f2b10 2023-12-19T18:22:36.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.991+0000 7fea34f2e640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea1c06a460 0x7fea1c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:36.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.992+0000 7fea34f2e640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea1c06a460 0x7fea1c06c920 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fea20035000 tx=0x7fea20030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:36.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:36.995+0000 7fea2d7fa640 1 -- 172.21.15.27:0/398356890 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fea2801a060 con 0x7fea301f2b10 2023-12-19T18:22:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.202+0000 7fea35f30640 1 -- 172.21.15.27:0/398356890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd unset", "key": "pause"} v 0) v1 -- 0x7fea3004e840 con 0x7fea301f2b10 2023-12-19T18:22:37.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.723+0000 7fea2d7fa640 1 -- 172.21.15.27:0/398356890 <== mon.0 v2:172.21.15.27:3300/0 7 ==== osd_map(138..138 src has 1..138) v4 ==== 288+0+0 (secure 0 0 0) 0x7fea28057b70 con 0x7fea301f2b10 2023-12-19T18:22:37.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.723+0000 7fea2d7fa640 1 -- 172.21.15.27:0/398356890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=139}) v3 -- 0x7fea1c072840 con 0x7fea301f2b10 2023-12-19T18:22:37.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.728+0000 7fea2d7fa640 1 -- 172.21.15.27:0/398356890 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "osd unset", "key": "pause"}]=0 pauserd,pausewr is unset v138) v1 ==== 97+0+0 (secure 0 0 0) 0x7fea2801a070 con 0x7fea301f2b10 2023-12-19T18:22:37.729 INFO:teuthology.orchestra.run.smithi027.stderr:pauserd,pausewr is unset 2023-12-19T18:22:37.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.733+0000 7fea35f30640 1 -- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea1c06a460 msgr2=0x7fea1c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.733+0000 7fea35f30640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea1c06a460 0x7fea1c06c920 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fea20035000 tx=0x7fea20030000 comp rx=0 tx=0).stop 2023-12-19T18:22:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.733+0000 7fea35f30640 1 -- 172.21.15.27:0/398356890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea301f2b10 msgr2=0x7fea3015ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.733+0000 7fea35f30640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea301f2b10 0x7fea3015ef30 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fea300ceda0 tx=0x7fea2800ccd0 comp rx=0 tx=0).stop 2023-12-19T18:22:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.733+0000 7fea35f30640 1 -- 172.21.15.27:0/398356890 shutdown_connections 2023-12-19T18:22:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.733+0000 7fea35f30640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea1c06a460 0x7fea1c06c920 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.733+0000 7fea35f30640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea301f2b10 0x7fea3015ef30 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.733+0000 7fea35f30640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea301f2690 0x7fea3015e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.733+0000 7fea35f30640 1 --2- 172.21.15.27:0/398356890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea180039c0 0x7fea30154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.733+0000 7fea35f30640 1 -- 172.21.15.27:0/398356890 >> 172.21.15.27:0/398356890 conn(0x7fea300cf760 msgr2=0x7fea3006bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.733+0000 7fea35f30640 1 -- 172.21.15.27:0/398356890 shutdown_connections 2023-12-19T18:22:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.733+0000 7fea35f30640 1 -- 172.21.15.27:0/398356890 wait complete. 2023-12-19T18:22:37.758 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:22:37.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.987+0000 7f4b0f82c640 1 Processor -- start 2023-12-19T18:22:37.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.987+0000 7f4b0f82c640 1 -- start start 2023-12-19T18:22:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.988+0000 7f4b0f82c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4b0815ce90 0x7f4b080cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.988+0000 7f4b0f82c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080ade10 0x7f4b080ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.988+0000 7f4b0f82c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b0806b4e0 con 0x7f4b0815d270 2023-12-19T18:22:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.988+0000 7f4b0f82c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b0806ce90 con 0x7f4b0815ce90 2023-12-19T18:22:37.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.988+0000 7f4b0f82c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080704f0 con 0x7f4b080ade10 2023-12-19T18:22:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.989+0000 7f4b0cdc7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4b0815d270 0x7f4b080cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44538/0 (socket says 172.21.15.27:44538) 2023-12-19T18:22:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.989+0000 7f4b0cdc7640 1 -- 172.21.15.27:0/2918308493 learned_addr learned my addr 172.21.15.27:0/2918308493 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.989+0000 7f4b0d5c8640 1 --2- 172.21.15.27:0/2918308493 >> v2:172.21.15.192:3300/0 conn(0x7f4b0815ce90 0x7f4b080cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.989+0000 7f4b0ddc9640 1 --2- 172.21.15.27:0/2918308493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080ade10 0x7f4b080ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.989+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3186442638 0 0) 0x7f4b0806b4e0 con 0x7f4b0815d270 2023-12-19T18:22:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.989+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4af4003520 con 0x7f4b0815d270 2023-12-19T18:22:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3168181541 0 0) 0x7f4af4003520 con 0x7f4b0815d270 2023-12-19T18:22:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080ade10 msgr2=0x7f4b080ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 --2- 172.21.15.27:0/2918308493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080ade10 0x7f4b080ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 >> v2:172.21.15.192:3300/0 conn(0x7f4b0815ce90 msgr2=0x7f4b080cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 --2- 172.21.15.27:0/2918308493 >> v2:172.21.15.192:3300/0 conn(0x7f4b0815ce90 0x7f4b080cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b0806b7c0 con 0x7f4b0815d270 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f4af8001670 con 0x7f4b0815d270 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 >> v1:172.21.15.27:6789/0 conn(0x7f4b0815d270 legacy=0x7f4b080cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 --2- 172.21.15.27:0/2918308493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080ade10 0x7f4af4003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 --2- 172.21.15.27:0/2918308493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4af4003f40 0x7f4af40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 --2- 172.21.15.27:0/2918308493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af4004d00 0x7f4af4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0806ce90 con 0x7f4af4003f40 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080704f0 con 0x7f4af4004d00 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af400b980 con 0x7f4b080ade10 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.990+0000 7f4b0cdc7640 1 --2- 172.21.15.27:0/2918308493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4af4003f40 0x7f4af40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.991+0000 7f4b0ddc9640 1 --2- 172.21.15.27:0/2918308493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af4004d00 0x7f4af4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.991+0000 7f4b0d5c8640 1 --2- 172.21.15.27:0/2918308493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080ade10 0x7f4af4003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.991+0000 7f4b0cdc7640 1 -- 172.21.15.27:0/2918308493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080ade10 msgr2=0x7f4af4003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.991+0000 7f4b0cdc7640 1 --2- 172.21.15.27:0/2918308493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080ade10 0x7f4af4003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.991+0000 7f4b0cdc7640 1 -- 172.21.15.27:0/2918308493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af4004d00 msgr2=0x7f4af4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:37.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.991+0000 7f4b0cdc7640 1 --2- 172.21.15.27:0/2918308493 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4af4004d00 0x7f4af4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:37.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.991+0000 7f4b0cdc7640 1 -- 172.21.15.27:0/2918308493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af4003520 con 0x7f4af4003f40 2023-12-19T18:22:37.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.991+0000 7f4b0d5c8640 1 --2- 172.21.15.27:0/2918308493 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080ade10 0x7f4af4003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:37.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.992+0000 7f4b0cdc7640 1 --2- 172.21.15.27:0/2918308493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4af4003f40 0x7f4af40047c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f4af800c110 tx=0x7f4af40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=123051defbdd1bc5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:37.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.992+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af8005ad0 con 0x7f4af4003f40 2023-12-19T18:22:37.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.992+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af8005c50 con 0x7f4af4003f40 2023-12-19T18:22:37.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.992+0000 7f4afffff640 1 -- 172.21.15.27:0/2918308493 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af802c7e0 con 0x7f4af4003f40 2023-12-19T18:22:37.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.993+0000 7f4b0f82c640 1 -- 172.21.15.27:0/2918308493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4af4003f40 msgr2=0x7f4af40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:37.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.993+0000 7f4b0f82c640 1 --2- 172.21.15.27:0/2918308493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4af4003f40 0x7f4af40047c0 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f4af800c110 tx=0x7f4af40057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:37.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.993+0000 7f4b0ddc9640 1 -- 172.21.15.27:0/2918308493 reap_dead start 2023-12-19T18:22:37.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.993+0000 7f4b0f82c640 1 -- 172.21.15.27:0/2918308493 shutdown_connections 2023-12-19T18:22:37.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.993+0000 7f4b0f82c640 1 -- 172.21.15.27:0/2918308493 >> 172.21.15.27:0/2918308493 conn(0x7f4b0806a330 msgr2=0x7f4b080b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:37.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.993+0000 7f4b0f82c640 1 -- 172.21.15.27:0/2918308493 shutdown_connections 2023-12-19T18:22:37.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.993+0000 7f4b0f82c640 1 -- 172.21.15.27:0/2918308493 wait complete. 2023-12-19T18:22:37.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.995+0000 7f4b0f82c640 1 Processor -- start 2023-12-19T18:22:37.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.995+0000 7f4b0f82c640 1 -- start start 2023-12-19T18:22:37.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.996+0000 7f4b0f82c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b0814df20 0x7f4b080c6580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:37.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.996+0000 7f4b0f82c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b0814e300 0x7f4b080c6ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:37.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.996+0000 7f4b0f82c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080c3b80 0x7f4b080c4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:37.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.996+0000 7f4b0f82c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080c4870 con 0x7f4b0814df20 2023-12-19T18:22:37.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.996+0000 7f4b0f82c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080c4b50 con 0x7f4b0814e300 2023-12-19T18:22:37.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.996+0000 7f4b0f82c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080c4e30 con 0x7f4b080c3b80 2023-12-19T18:22:37.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.996+0000 7f4b0d5c8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b0814df20 0x7f4b080c6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:37.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.996+0000 7f4b0d5c8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b0814df20 0x7f4b080c6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42186/0 (socket says 172.21.15.27:42186) 2023-12-19T18:22:37.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.996+0000 7f4b0d5c8640 1 -- 172.21.15.27:0/1682955110 learned_addr learned my addr 172.21.15.27:0/1682955110 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:37.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.997+0000 7f4b0ddc9640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080c3b80 0x7f4b080c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:37.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.997+0000 7f4b0cdc7640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b0814e300 0x7f4b080c6ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:37.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.997+0000 7f4b0d5c8640 1 -- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080c3b80 msgr2=0x7f4b080c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:37.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.997+0000 7f4b0d5c8640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080c3b80 0x7f4b080c4030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:37.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.997+0000 7f4b0d5c8640 1 -- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b0814e300 msgr2=0x7f4b080c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:37.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.997+0000 7f4b0d5c8640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b0814e300 0x7f4b080c6ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:37.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.997+0000 7f4b0d5c8640 1 -- 172.21.15.27:0/1682955110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b080c5110 con 0x7f4b0814df20 2023-12-19T18:22:37.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.997+0000 7f4b0cdc7640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b0814e300 0x7f4b080c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:37.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.997+0000 7f4b0d5c8640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b0814df20 0x7f4b080c6580 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f4b040109d0 tx=0x7f4b04010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:37.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.997+0000 7f4afdffb640 1 -- 172.21.15.27:0/1682955110 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0400ec20 con 0x7f4b0814df20 2023-12-19T18:22:37.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.997+0000 7f4afdffb640 1 -- 172.21.15.27:0/1682955110 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b0400ed80 con 0x7f4b0814df20 2023-12-19T18:22:37.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.997+0000 7f4afdffb640 1 -- 172.21.15.27:0/1682955110 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b0401c620 con 0x7f4b0814df20 2023-12-19T18:22:37.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.998+0000 7f4b0f82c640 1 -- 172.21.15.27:0/1682955110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b080c53f0 con 0x7f4b0814df20 2023-12-19T18:22:37.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.998+0000 7f4b0f82c640 1 -- 172.21.15.27:0/1682955110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b081f5f70 con 0x7f4b0814df20 2023-12-19T18:22:38.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:37.999+0000 7f4b0f82c640 1 -- 172.21.15.27:0/1682955110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b080c5580 con 0x7f4b0814df20 2023-12-19T18:22:38.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.000+0000 7f4afdffb640 1 -- 172.21.15.27:0/1682955110 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4b0401c780 con 0x7f4b0814df20 2023-12-19T18:22:38.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.001+0000 7f4afdffb640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4aec06a430 0x7f4aec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:38.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.001+0000 7f4afdffb640 1 -- 172.21.15.27:0/1682955110 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4b040890d0 con 0x7f4b0814df20 2023-12-19T18:22:38.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.002+0000 7f4b0cdc7640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4aec06a430 0x7f4aec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:38.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.003+0000 7f4b0cdc7640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4aec06a430 0x7f4aec06c8f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4af8034000 tx=0x7f4af8007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:38.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.006+0000 7f4afdffb640 1 -- 172.21.15.27:0/1682955110 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4b040546d0 con 0x7f4b0814df20 2023-12-19T18:22:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.201+0000 7f4b0f82c640 1 -- 172.21.15.27:0/1682955110 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4b08068e90 con 0x7f4b0814df20 2023-12-19T18:22:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.203+0000 7f4afdffb640 1 -- 172.21.15.27:0/1682955110 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16059 (secure 0 0 0) 0x7f4b04057cf0 con 0x7f4b0814df20 2023-12-19T18:22:38.204 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:38.204 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":138,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:22:37.716476+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:22:12.328518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T18:22:13.251708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:22:38.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.206+0000 7f4b0f82c640 1 -- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4aec06a430 msgr2=0x7f4aec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.206+0000 7f4b0f82c640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4aec06a430 0x7f4aec06c8f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f4af8034000 tx=0x7f4af8007000 comp rx=0 tx=0).stop 2023-12-19T18:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.206+0000 7f4b0f82c640 1 -- 172.21.15.27:0/1682955110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b0814df20 msgr2=0x7f4b080c6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.206+0000 7f4b0f82c640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b0814df20 0x7f4b080c6580 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f4b040109d0 tx=0x7f4b04010e90 comp rx=0 tx=0).stop 2023-12-19T18:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.206+0000 7f4b0f82c640 1 -- 172.21.15.27:0/1682955110 shutdown_connections 2023-12-19T18:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.206+0000 7f4b0f82c640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4aec06a430 0x7f4aec06c8f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.206+0000 7f4b0f82c640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b080c3b80 0x7f4b080c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.206+0000 7f4b0f82c640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b0814e300 0x7f4b080c6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.206+0000 7f4b0f82c640 1 --2- 172.21.15.27:0/1682955110 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b0814df20 0x7f4b080c6580 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.206+0000 7f4b0f82c640 1 -- 172.21.15.27:0/1682955110 >> 172.21.15.27:0/1682955110 conn(0x7f4b0806a330 msgr2=0x7f4b0815c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.206+0000 7f4b0f82c640 1 -- 172.21.15.27:0/1682955110 shutdown_connections 2023-12-19T18:22:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:38.207+0000 7f4b0f82c640 1 -- 172.21.15.27:0/1682955110 wait complete. 2023-12-19T18:22:38.225 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:22:38.226 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:22:38.226 DEBUG:teuthology.orchestra.run.smithi027:> fd = os.open("/home/ubuntu/cephtest/mnt.0/foo", os.O_CREAT | os.O_RDWR, 0O600) 2023-12-19T18:22:38.226 DEBUG:teuthology.orchestra.run.smithi027:> os.fchmod(fd, 0O666) 2023-12-19T18:22:38.226 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(fd) 2023-12-19T18:22:38.226 DEBUG:teuthology.orchestra.run.smithi027:> os.close(fd) 2023-12-19T18:22:38.226 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:22:38.541 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:22:38.541 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:22:38.541 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:22:38.541 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:22:38.541 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:22:38.541 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:38.541 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:22:38.541 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:22:38.541 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:22:38.541 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:22:38.541 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:38.542 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:22:38.542 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:22:38.542 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:22:38.542 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:22:38.542 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:38.542 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:22:38.542 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:22:38.542 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:22:38.542 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:22:38.542 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:22:38.542 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:22:38.543 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:22:38.543 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:22:38.543 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:38.543 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:22:38.543 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:22:38.787 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.109914.asok 2023-12-19T18:22:38.967 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.109914.asok 2023-12-19T18:22:38.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:38.968 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.109914.asok status 2023-12-19T18:22:39.230 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:22:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:22:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:22:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:22:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:22:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:22:39.230 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "109914", 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 1, 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 7521, 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 7521 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:22:39.231 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3064875531 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3064875531 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.7521 192.168.0.1:0/3064875531", 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3064875531", 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 2, 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 54, 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 135, 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 135, 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:22:39.232 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:22:39.233 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:22:39.246 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:22:39.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.486+0000 7f7c85ae7640 1 Processor -- start 2023-12-19T18:22:39.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.486+0000 7f7c85ae7640 1 -- start start 2023-12-19T18:22:39.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.488+0000 7f7c85ae7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c800ce520 0x7f7c800cb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:39.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.488+0000 7f7c85ae7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c800ce980 con 0x7f7c800cba90 2023-12-19T18:22:39.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.488+0000 7f7c85ae7640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c800caba0 con 0x7f7c80164a80 2023-12-19T18:22:39.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.488+0000 7f7c85ae7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800cbf00 con 0x7f7c800ce520 2023-12-19T18:22:39.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.488+0000 7f7c7ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7c800cba90 0x7f7c800cc710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44550/0 (socket says 172.21.15.27:44550) 2023-12-19T18:22:39.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.488+0000 7f7c7ffff640 1 -- 172.21.15.27:0/2024732401 learned_addr learned my addr 172.21.15.27:0/2024732401 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.488+0000 7f7c76ffd640 1 --2- 172.21.15.27:0/2024732401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c800ce520 0x7f7c800cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285420218 0 0) 0x7f7c800ce980 con 0x7f7c800cba90 2023-12-19T18:22:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7c6c003520 con 0x7f7c800cba90 2023-12-19T18:22:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1713812076 0 0) 0x7f7c6c003520 con 0x7f7c800cba90 2023-12-19T18:22:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c800ce520 msgr2=0x7f7c800cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 --2- 172.21.15.27:0/2024732401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c800ce520 0x7f7c800cb550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:39.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 >> v1:172.21.15.192:6789/0 conn(0x7f7c80164a80 legacy=0x7f7c800c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c80164e60 con 0x7f7c800cba90 2023-12-19T18:22:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f7c780019f0 con 0x7f7c800cba90 2023-12-19T18:22:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 >> v1:172.21.15.27:6789/0 conn(0x7f7c800cba90 legacy=0x7f7c800cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 --2- 172.21.15.27:0/2024732401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c6c0039a0 0x7f7c6c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 --2- 172.21.15.27:0/2024732401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c6c004350 0x7f7c6c004820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:39.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 --2- 172.21.15.27:0/2024732401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c800ce520 0x7f7c6c005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800cbf00 con 0x7f7c6c004350 2023-12-19T18:22:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c00ba30 con 0x7f7c6c0039a0 2023-12-19T18:22:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.489+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c00bbf0 con 0x7f7c800ce520 2023-12-19T18:22:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.490+0000 7f7c76ffd640 1 --2- 172.21.15.27:0/2024732401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c6c004350 0x7f7c6c004820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.490+0000 7f7c7ffff640 1 --2- 172.21.15.27:0/2024732401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c800ce520 0x7f7c6c005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.490+0000 7f7c76ffd640 1 -- 172.21.15.27:0/2024732401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c800ce520 msgr2=0x7f7c6c005120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.490+0000 7f7c76ffd640 1 --2- 172.21.15.27:0/2024732401 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c800ce520 0x7f7c6c005120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.490+0000 7f7c76ffd640 1 -- 172.21.15.27:0/2024732401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c6c0039a0 msgr2=0x7f7c6c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.490+0000 7f7c7f7fe640 1 --2- 172.21.15.27:0/2024732401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c6c0039a0 0x7f7c6c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.490+0000 7f7c76ffd640 1 --2- 172.21.15.27:0/2024732401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c6c0039a0 0x7f7c6c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.490+0000 7f7c76ffd640 1 -- 172.21.15.27:0/2024732401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c003520 con 0x7f7c6c004350 2023-12-19T18:22:39.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.490+0000 7f7c7f7fe640 1 --2- 172.21.15.27:0/2024732401 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c6c0039a0 0x7f7c6c003d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:39.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.491+0000 7f7c76ffd640 1 --2- 172.21.15.27:0/2024732401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c6c004350 0x7f7c6c004820 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009ac0 tx=0x7f7c6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7e95c1e739af5be server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.492+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6802c6b0 con 0x7f7c6c004350 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.492+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c6802c810 con 0x7f7c6c004350 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.492+0000 7f7c7effd640 1 -- 172.21.15.27:0/2024732401 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c6802cba0 con 0x7f7c6c004350 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.492+0000 7f7c85ae7640 1 -- 172.21.15.27:0/2024732401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c6c004350 msgr2=0x7f7c6c004820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.492+0000 7f7c85ae7640 1 --2- 172.21.15.27:0/2024732401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c6c004350 0x7f7c6c004820 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f7c68009ac0 tx=0x7f7c6c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.492+0000 7f7c7ffff640 1 -- 172.21.15.27:0/2024732401 reap_dead start 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.492+0000 7f7c85ae7640 1 -- 172.21.15.27:0/2024732401 shutdown_connections 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.492+0000 7f7c85ae7640 1 -- 172.21.15.27:0/2024732401 >> 172.21.15.27:0/2024732401 conn(0x7f7c8006a330 msgr2=0x7f7c800b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.492+0000 7f7c85ae7640 1 -- 172.21.15.27:0/2024732401 shutdown_connections 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.492+0000 7f7c85ae7640 1 -- 172.21.15.27:0/2024732401 wait complete. 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.494+0000 7f7c85ae7640 1 Processor -- start 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.495+0000 7f7c85ae7640 1 -- start start 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c85ae7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c6c004350 0x7f7c801f73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:39.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c85ae7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8006b4e0 0x7f7c80162aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c85ae7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c800c4870 0x7f7c801f7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c85ae7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c801f7920 con 0x7f7c8006b4e0 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c85ae7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c80162fe0 con 0x7f7c800c4870 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c85ae7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c800c37d0 con 0x7f7c6c004350 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c76ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8006b4e0 0x7f7c80162aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c76ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8006b4e0 0x7f7c80162aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42198/0 (socket says 172.21.15.27:42198) 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c76ffd640 1 -- 172.21.15.27:0/1369780413 learned_addr learned my addr 172.21.15.27:0/1369780413 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c7f7fe640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c6c004350 0x7f7c801f73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c7ffff640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c800c4870 0x7f7c801f7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c76ffd640 1 -- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c6c004350 msgr2=0x7f7c801f73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c76ffd640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c6c004350 0x7f7c801f73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c76ffd640 1 -- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c800c4870 msgr2=0x7f7c801f7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c76ffd640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c800c4870 0x7f7c801f7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c76ffd640 1 -- 172.21.15.27:0/1369780413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c800c3ab0 con 0x7f7c8006b4e0 2023-12-19T18:22:39.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.496+0000 7f7c7ffff640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c800c4870 0x7f7c801f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.497+0000 7f7c76ffd640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8006b4e0 0x7f7c80162aa0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7c68037040 tx=0x7f7c68004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.497+0000 7f7c7cff9640 1 -- 172.21.15.27:0/1369780413 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68003810 con 0x7f7c8006b4e0 2023-12-19T18:22:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.497+0000 7f7c7cff9640 1 -- 172.21.15.27:0/1369780413 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c68003970 con 0x7f7c8006b4e0 2023-12-19T18:22:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.497+0000 7f7c7cff9640 1 -- 172.21.15.27:0/1369780413 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c68003ce0 con 0x7f7c8006b4e0 2023-12-19T18:22:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.497+0000 7f7c85ae7640 1 -- 172.21.15.27:0/1369780413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c800c3d90 con 0x7f7c8006b4e0 2023-12-19T18:22:39.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.497+0000 7f7c85ae7640 1 -- 172.21.15.27:0/1369780413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c800c4350 con 0x7f7c8006b4e0 2023-12-19T18:22:39.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.498+0000 7f7c85ae7640 1 -- 172.21.15.27:0/1369780413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c800c44e0 con 0x7f7c8006b4e0 2023-12-19T18:22:39.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.499+0000 7f7c7cff9640 1 -- 172.21.15.27:0/1369780413 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7c68034440 con 0x7f7c8006b4e0 2023-12-19T18:22:39.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.500+0000 7f7c7cff9640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c6006a430 0x7f7c6006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:39.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.500+0000 7f7c7cff9640 1 -- 172.21.15.27:0/1369780413 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c680aabf0 con 0x7f7c8006b4e0 2023-12-19T18:22:39.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.501+0000 7f7c7f7fe640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c6006a430 0x7f7c6006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:39.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.502+0000 7f7c7f7fe640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c6006a430 0x7f7c6006c8f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7c70000f60 tx=0x7f7c70009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:39.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.504+0000 7f7c7cff9640 1 -- 172.21.15.27:0/1369780413 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7c68076280 con 0x7f7c8006b4e0 2023-12-19T18:22:39.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.735+0000 7f7c85ae7640 1 -- 172.21.15.27:0/1369780413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7c80068e90 con 0x7f7c8006b4e0 2023-12-19T18:22:39.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.737+0000 7f7c7cff9640 1 -- 172.21.15.27:0/1369780413 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f7c68039090 con 0x7f7c8006b4e0 2023-12-19T18:22:39.737 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:39.737 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2713867269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2713867269},{"type":"v1","addr":"172.21.15.192:6837","nonce":2713867269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":5534,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/919878747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":919878747},{"type":"v1","addr":"172.21.15.27:6835","nonce":919878747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6160,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/861111763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":861111763},{"type":"v1","addr":"172.21.15.192:6836","nonce":861111763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:22:14.217224+0000","modified":"2023-12-19T18:22:17.743185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7320":{"gid":7320,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3670863819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3670863819},{"type":"v1","addr":"172.21.15.27:6834","nonce":3670863819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T18:22:39.737 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 54 2023-12-19T18:22:39.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.739+0000 7f7c85ae7640 1 -- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c6006a430 msgr2=0x7f7c6006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:39.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.739+0000 7f7c85ae7640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c6006a430 0x7f7c6006c8f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f7c70000f60 tx=0x7f7c70009040 comp rx=0 tx=0).stop 2023-12-19T18:22:39.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.739+0000 7f7c85ae7640 1 -- 172.21.15.27:0/1369780413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8006b4e0 msgr2=0x7f7c80162aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:39.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.739+0000 7f7c85ae7640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8006b4e0 0x7f7c80162aa0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f7c68037040 tx=0x7f7c68004730 comp rx=0 tx=0).stop 2023-12-19T18:22:39.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.739+0000 7f7c85ae7640 1 -- 172.21.15.27:0/1369780413 shutdown_connections 2023-12-19T18:22:39.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.739+0000 7f7c85ae7640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c6006a430 0x7f7c6006c8f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:39.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.739+0000 7f7c85ae7640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c800c4870 0x7f7c801f7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:39.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.739+0000 7f7c85ae7640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8006b4e0 0x7f7c80162aa0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:39.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.739+0000 7f7c85ae7640 1 --2- 172.21.15.27:0/1369780413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c6c004350 0x7f7c801f73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:39.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.739+0000 7f7c85ae7640 1 -- 172.21.15.27:0/1369780413 >> 172.21.15.27:0/1369780413 conn(0x7f7c8006a330 msgr2=0x7f7c800aecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:39.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.740+0000 7f7c85ae7640 1 -- 172.21.15.27:0/1369780413 shutdown_connections 2023-12-19T18:22:39.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:39.740+0000 7f7c85ae7640 1 -- 172.21.15.27:0/1369780413 wait complete. 2023-12-19T18:22:39.760 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json osdmap barrier 138 2023-12-19T18:22:40.007 DEBUG:tasks.cephfs.filesystem:_json_asok output empty 2023-12-19T18:22:40.007 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:22:40.008 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:22:40.008 DEBUG:teuthology.orchestra.run.smithi027:> fd = os.open("/home/ubuntu/cephtest/mnt.0/baz", os.O_CREAT | os.O_RDWR, 0O600) 2023-12-19T18:22:40.008 DEBUG:teuthology.orchestra.run.smithi027:> os.fchmod(fd, 0O666) 2023-12-19T18:22:40.008 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(fd) 2023-12-19T18:22:40.008 DEBUG:teuthology.orchestra.run.smithi027:> os.close(fd) 2023-12-19T18:22:40.008 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:22:40.336 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:22:40.336 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:22:40.337 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:40.338 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:22:40.338 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:22:40.338 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:22:40.338 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:22:40.338 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:22:40.338 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:22:40.338 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:22:40.338 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:22:40.338 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:40.338 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:22:40.338 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:22:40.582 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.109914.asok 2023-12-19T18:22:40.762 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.109914.asok 2023-12-19T18:22:40.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:40.762 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.109914.asok status 2023-12-19T18:22:41.021 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:22:41.021 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:22:41.022 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:22:41.022 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:22:41.022 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:22:41.022 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:22:41.022 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:22:41.022 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "109914", 2023-12-19T18:22:41.022 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:22:41.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:41.023 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 2, 2023-12-19T18:22:41.023 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:22:41.023 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 7521, 2023-12-19T18:22:41.023 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:22:41.023 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:22:41.023 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:22:41.024 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 7521 2023-12-19T18:22:41.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:41.024 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:22:41.024 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:22:41.024 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:22:41.024 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3064875531 2023-12-19T18:22:41.024 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:41.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:41.025 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:22:41.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:22:41.025 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:22:41.025 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3064875531 2023-12-19T18:22:41.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:41.025 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.7521 192.168.0.1:0/3064875531", 2023-12-19T18:22:41.025 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3064875531", 2023-12-19T18:22:41.025 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 3, 2023-12-19T18:22:41.026 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 54, 2023-12-19T18:22:41.026 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 138, 2023-12-19T18:22:41.026 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 138, 2023-12-19T18:22:41.026 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:22:41.026 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:22:41.026 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:22:41.038 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:22:41.038 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:22:41.038 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:22:41.038 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:22:41.038 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:22:41.038 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:41.038 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:22:41.038 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:22:41.039 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:22:41.040 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:22:41.040 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:22:41.040 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:22:41.236 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.109914.asok 2023-12-19T18:22:41.417 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.109914.asok 2023-12-19T18:22:41.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:41.418 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.109914.asok status 2023-12-19T18:22:41.678 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "109914", 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 2, 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 7521, 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:22:41.679 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 7521 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3064875531 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3064875531 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:22:41.680 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.7521 192.168.0.1:0/3064875531", 2023-12-19T18:22:41.681 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3064875531", 2023-12-19T18:22:41.681 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 3, 2023-12-19T18:22:41.681 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 54, 2023-12-19T18:22:41.681 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 138, 2023-12-19T18:22:41.681 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 138, 2023-12-19T18:22:41.681 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:22:41.681 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:22:41.681 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:22:41.697 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-12-19T18:22:41.724 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:22:41.724 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:22:41.846 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:22:41.977 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:22:41.977 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:22:41.977 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:22:42.043 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:22:42.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.274+0000 7f52effff640 1 Processor -- start 2023-12-19T18:22:42.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.275+0000 7f52effff640 1 -- start start 2023-12-19T18:22:42.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.277+0000 7f52effff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52f01604d0 0x7f52f00b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:42.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.277+0000 7f52effff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52f00b7310 0x7f52f00b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:42.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.277+0000 7f52effff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f00b5e90 con 0x7f52f00d2350 2023-12-19T18:22:42.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.277+0000 7f52effff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f00d2b30 con 0x7f52f01604d0 2023-12-19T18:22:42.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.277+0000 7f52effff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f00d2850 con 0x7f52f00b7310 2023-12-19T18:22:42.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.277+0000 7f52eeffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52f01604d0 0x7f52f00b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:42.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.277+0000 7f52eeffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52f01604d0 0x7f52f00b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35664/0 (socket says 172.21.15.27:35664) 2023-12-19T18:22:42.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.277+0000 7f52eeffd640 1 -- 172.21.15.27:0/888051466 learned_addr learned my addr 172.21.15.27:0/888051466 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:42.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.278+0000 7f52ef7fe640 1 --2- 172.21.15.27:0/888051466 >> v2:172.21.15.192:3300/0 conn(0x7f52f00b7310 0x7f52f00b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:42.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.278+0000 7f52eeffd640 1 -- 172.21.15.27:0/888051466 >> v2:172.21.15.192:3300/0 conn(0x7f52f00b7310 msgr2=0x7f52f00b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:42.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.278+0000 7f52eeffd640 1 --2- 172.21.15.27:0/888051466 >> v2:172.21.15.192:3300/0 conn(0x7f52f00b7310 0x7f52f00b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.278+0000 7f52eeffd640 1 -- 172.21.15.27:0/888051466 >> v1:172.21.15.192:6789/0 conn(0x7f52f00d2350 legacy=0x7f52f00d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:42.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.278+0000 7f52eeffd640 1 -- 172.21.15.27:0/888051466 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f00cded0 con 0x7f52f01604d0 2023-12-19T18:22:42.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.279+0000 7f52eeffd640 1 --2- 172.21.15.27:0/888051466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f01604d0 0x7f52f00b47e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f52f00cd540 tx=0x7f52e002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32db23465665fd58 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:42.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.280+0000 7f52ee7fc640 1 -- 172.21.15.27:0/888051466 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0031870 con 0x7f52f01604d0 2023-12-19T18:22:42.279 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.280+0000 7f52ee7fc640 1 -- 172.21.15.27:0/888051466 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e00319d0 con 0x7f52f01604d0 2023-12-19T18:22:42.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.280+0000 7f52ee7fc640 1 -- 172.21.15.27:0/888051466 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e0031d60 con 0x7f52f01604d0 2023-12-19T18:22:42.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.280+0000 7f52effff640 1 -- 172.21.15.27:0/888051466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f01604d0 msgr2=0x7f52f00b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:42.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.280+0000 7f52effff640 1 --2- 172.21.15.27:0/888051466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f01604d0 0x7f52f00b47e0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f52f00cd540 tx=0x7f52e002b910 comp rx=0 tx=0).stop 2023-12-19T18:22:42.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.280+0000 7f52effff640 1 -- 172.21.15.27:0/888051466 shutdown_connections 2023-12-19T18:22:42.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.280+0000 7f52effff640 1 --2- 172.21.15.27:0/888051466 >> v2:172.21.15.192:3300/0 conn(0x7f52f00b7310 0x7f52f00b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.280+0000 7f52effff640 1 --2- 172.21.15.27:0/888051466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f01604d0 0x7f52f00b47e0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.280+0000 7f52effff640 1 -- 172.21.15.27:0/888051466 >> 172.21.15.27:0/888051466 conn(0x7f52f00cf7e0 msgr2=0x7f52f00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:42.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.281+0000 7f52effff640 1 -- 172.21.15.27:0/888051466 shutdown_connections 2023-12-19T18:22:42.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.281+0000 7f52effff640 1 -- 172.21.15.27:0/888051466 wait complete. 2023-12-19T18:22:42.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.282+0000 7f52effff640 1 Processor -- start 2023-12-19T18:22:42.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.282+0000 7f52effff640 1 -- start start 2023-12-19T18:22:42.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.283+0000 7f52effff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f00b7310 0x7f52f00d12c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:42.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.283+0000 7f52effff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52f00d2350 0x7f52f0150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:42.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.283+0000 7f52effff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f01604d0 0x7f52f01506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:42.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.283+0000 7f52effff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f0150c00 con 0x7f52f01604d0 2023-12-19T18:22:42.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.283+0000 7f52effff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f0159bb0 con 0x7f52f00b7310 2023-12-19T18:22:42.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.283+0000 7f52effff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f00c9ed0 con 0x7f52f00d2350 2023-12-19T18:22:42.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.284+0000 7f52ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f01604d0 0x7f52f01506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:42.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.284+0000 7f52ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f01604d0 0x7f52f01506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35672/0 (socket says 172.21.15.27:35672) 2023-12-19T18:22:42.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.284+0000 7f52ef7fe640 1 -- 172.21.15.27:0/3921059537 learned_addr learned my addr 172.21.15.27:0/3921059537 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:42.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.284+0000 7f52eeffd640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f00b7310 0x7f52f00d12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.284+0000 7f52ef7fe640 1 -- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52f00d2350 msgr2=0x7f52f0150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.284+0000 7f52ef7fe640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52f00d2350 0x7f52f0150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.284+0000 7f52ef7fe640 1 -- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f00b7310 msgr2=0x7f52f00d12c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.284+0000 7f52ef7fe640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f00b7310 0x7f52f00d12c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.284+0000 7f52ef7fe640 1 -- 172.21.15.27:0/3921059537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e00094d0 con 0x7f52f01604d0 2023-12-19T18:22:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.284+0000 7f52eeffd640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f00b7310 0x7f52f00d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.285+0000 7f52ef7fe640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f01604d0 0x7f52f01506c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f52e8007e60 tx=0x7f52e800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.285+0000 7f52ecff9640 1 -- 172.21.15.27:0/3921059537 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8014420 con 0x7f52f01604d0 2023-12-19T18:22:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.285+0000 7f52ecff9640 1 -- 172.21.15.27:0/3921059537 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e80145a0 con 0x7f52f01604d0 2023-12-19T18:22:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.285+0000 7f52ecff9640 1 -- 172.21.15.27:0/3921059537 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e8014950 con 0x7f52f01604d0 2023-12-19T18:22:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.285+0000 7f52effff640 1 -- 172.21.15.27:0/3921059537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f00ca1b0 con 0x7f52f01604d0 2023-12-19T18:22:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.285+0000 7f52effff640 1 -- 172.21.15.27:0/3921059537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f00ca770 con 0x7f52f01604d0 2023-12-19T18:22:42.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.286+0000 7f52effff640 1 -- 172.21.15.27:0/3921059537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f0066f80 con 0x7f52f01604d0 2023-12-19T18:22:42.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.287+0000 7f52ecff9640 1 -- 172.21.15.27:0/3921059537 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f52e8014ab0 con 0x7f52f01604d0 2023-12-19T18:22:42.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.288+0000 7f52ecff9640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f52d006a430 0x7f52d006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:42.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.288+0000 7f52ecff9640 1 -- 172.21.15.27:0/3921059537 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f52e80892d0 con 0x7f52f01604d0 2023-12-19T18:22:42.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.289+0000 7f52eeffd640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f52d006a430 0x7f52d006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:42.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.290+0000 7f52eeffd640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f52d006a430 0x7f52d006c8f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f52e0036000 tx=0x7f52e003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:42.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.293+0000 7f52ecff9640 1 -- 172.21.15.27:0/3921059537 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f52e8054820 con 0x7f52f01604d0 2023-12-19T18:22:42.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.486+0000 7f52effff640 1 -- 172.21.15.27:0/3921059537 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52f004e840 con 0x7f52f01604d0 2023-12-19T18:22:42.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.488+0000 7f52ecff9640 1 -- 172.21.15.27:0/3921059537 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16059 (secure 0 0 0) 0x7f52e8057e40 con 0x7f52f01604d0 2023-12-19T18:22:42.488 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:42.488 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":138,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:22:37.716476+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:22:12.328518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T18:22:13.251708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:22:42.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.491+0000 7f52effff640 1 -- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f52d006a430 msgr2=0x7f52d006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:42.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.491+0000 7f52effff640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f52d006a430 0x7f52d006c8f0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f52e0036000 tx=0x7f52e003c040 comp rx=0 tx=0).stop 2023-12-19T18:22:42.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.491+0000 7f52effff640 1 -- 172.21.15.27:0/3921059537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f01604d0 msgr2=0x7f52f01506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:42.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.491+0000 7f52effff640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f01604d0 0x7f52f01506c0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f52e8007e60 tx=0x7f52e800e4b0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.491+0000 7f52effff640 1 -- 172.21.15.27:0/3921059537 shutdown_connections 2023-12-19T18:22:42.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.491+0000 7f52effff640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f52d006a430 0x7f52d006c8f0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.491+0000 7f52effff640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f01604d0 0x7f52f01506c0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.491+0000 7f52effff640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52f00d2350 0x7f52f0150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.491+0000 7f52effff640 1 --2- 172.21.15.27:0/3921059537 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52f00b7310 0x7f52f00d12c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.491+0000 7f52effff640 1 -- 172.21.15.27:0/3921059537 >> 172.21.15.27:0/3921059537 conn(0x7f52f00cf7e0 msgr2=0x7f52f00b2460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:42.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.491+0000 7f52effff640 1 -- 172.21.15.27:0/3921059537 shutdown_connections 2023-12-19T18:22:42.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.491+0000 7f52effff640 1 -- 172.21.15.27:0/3921059537 wait complete. 2023-12-19T18:22:42.511 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:22:12.328518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:22:13.251708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:22:42.512 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:22:42.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.746+0000 7fa0c8a8c640 1 Processor -- start 2023-12-19T18:22:42.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.746+0000 7fa0c8a8c640 1 -- start start 2023-12-19T18:22:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.748+0000 7fa0c8a8c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa0c4159330 0x7fa0c415da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.748+0000 7fa0c8a8c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa0c40b2fc0 0x7fa0c40b3470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.748+0000 7fa0c8a8c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0c40b2b30 con 0x7fa0c406f320 2023-12-19T18:22:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.748+0000 7fa0c8a8c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c415df90 con 0x7fa0c4159330 2023-12-19T18:22:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.748+0000 7fa0c8a8c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0c406b000 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.748+0000 7fa0c37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa0c4159330 0x7fa0c415da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.748+0000 7fa0c37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa0c4159330 0x7fa0c415da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35680/0 (socket says 172.21.15.27:35680) 2023-12-19T18:22:42.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.748+0000 7fa0c37fe640 1 -- 172.21.15.27:0/1930925347 learned_addr learned my addr 172.21.15.27:0/1930925347 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:42.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.749+0000 7fa0bbfff640 1 --2- 172.21.15.27:0/1930925347 >> v2:172.21.15.192:3300/0 conn(0x7fa0c40b2fc0 0x7fa0c40b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:42.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.749+0000 7fa0c37fe640 1 -- 172.21.15.27:0/1930925347 >> v2:172.21.15.192:3300/0 conn(0x7fa0c40b2fc0 msgr2=0x7fa0c40b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:42.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.749+0000 7fa0c37fe640 1 --2- 172.21.15.27:0/1930925347 >> v2:172.21.15.192:3300/0 conn(0x7fa0c40b2fc0 0x7fa0c40b3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.749+0000 7fa0c37fe640 1 -- 172.21.15.27:0/1930925347 >> v1:172.21.15.192:6789/0 conn(0x7fa0c406f320 legacy=0x7fa0c406b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:42.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.749+0000 7fa0c37fe640 1 -- 172.21.15.27:0/1930925347 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0c406b2a0 con 0x7fa0c4159330 2023-12-19T18:22:42.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.749+0000 7fa0bbfff640 1 --2- 172.21.15.27:0/1930925347 >> v2:172.21.15.192:3300/0 conn(0x7fa0c40b2fc0 0x7fa0c40b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:42.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.750+0000 7fa0c37fe640 1 --2- 172.21.15.27:0/1930925347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c4159330 0x7fa0c415da50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40cb410 tx=0x7fa0b402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0945546728782ca server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.751+0000 7fa0c2ffd640 1 -- 172.21.15.27:0/1930925347 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4031870 con 0x7fa0c4159330 2023-12-19T18:22:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.751+0000 7fa0c2ffd640 1 -- 172.21.15.27:0/1930925347 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0b40319d0 con 0x7fa0c4159330 2023-12-19T18:22:42.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.751+0000 7fa0c2ffd640 1 -- 172.21.15.27:0/1930925347 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4031d60 con 0x7fa0c4159330 2023-12-19T18:22:42.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.752+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1930925347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c4159330 msgr2=0x7fa0c415da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:42.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.752+0000 7fa0c8a8c640 1 --2- 172.21.15.27:0/1930925347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c4159330 0x7fa0c415da50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa0c40cb410 tx=0x7fa0b402b910 comp rx=0 tx=0).stop 2023-12-19T18:22:42.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.752+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1930925347 shutdown_connections 2023-12-19T18:22:42.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.752+0000 7fa0c8a8c640 1 --2- 172.21.15.27:0/1930925347 >> v2:172.21.15.192:3300/0 conn(0x7fa0c40b2fc0 0x7fa0c40b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.752+0000 7fa0c8a8c640 1 --2- 172.21.15.27:0/1930925347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c4159330 0x7fa0c415da50 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.752+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1930925347 >> 172.21.15.27:0/1930925347 conn(0x7fa0c406a060 msgr2=0x7fa0c406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:42.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.752+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1930925347 shutdown_connections 2023-12-19T18:22:42.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.752+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1930925347 wait complete. 2023-12-19T18:22:42.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.754+0000 7fa0c8a8c640 1 Processor -- start 2023-12-19T18:22:42.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.754+0000 7fa0c8a8c640 1 -- start start 2023-12-19T18:22:42.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.755+0000 7fa0c8a8c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c406f320 0x7fa0c4157a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:42.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.755+0000 7fa0c8a8c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c40b2fc0 0x7fa0c4157f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:42.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.755+0000 7fa0c8a8c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0c4159330 0x7fa0c41f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:42.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.755+0000 7fa0c8a8c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4153790 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.755+0000 7fa0c8a8c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c40b05a0 con 0x7fa0c406f320 2023-12-19T18:22:42.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.755+0000 7fa0c8a8c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0c4152b80 con 0x7fa0c4159330 2023-12-19T18:22:42.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.755+0000 7fa0bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c40b2fc0 0x7fa0c4157f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:42.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.755+0000 7fa0bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c40b2fc0 0x7fa0c4157f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35694/0 (socket says 172.21.15.27:35694) 2023-12-19T18:22:42.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.755+0000 7fa0bbfff640 1 -- 172.21.15.27:0/1332323567 learned_addr learned my addr 172.21.15.27:0/1332323567 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:42.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.756+0000 7fa0c37fe640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c406f320 0x7fa0c4157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:42.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.756+0000 7fa0c3fff640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0c4159330 0x7fa0c41f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:42.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.756+0000 7fa0bbfff640 1 -- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0c4159330 msgr2=0x7fa0c41f6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:42.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.756+0000 7fa0bbfff640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0c4159330 0x7fa0c41f6320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.756+0000 7fa0bbfff640 1 -- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c406f320 msgr2=0x7fa0c4157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:42.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.756+0000 7fa0bbfff640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c406f320 0x7fa0c4157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.756+0000 7fa0bbfff640 1 -- 172.21.15.27:0/1332323567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0b40094d0 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.756+0000 7fa0c37fe640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c406f320 0x7fa0c4157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:42.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.756+0000 7fa0bbfff640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c40b2fc0 0x7fa0c4157f80 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00c7f0 tx=0x7fa0ac00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:42.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.756+0000 7fa0c0ff9640 1 -- 172.21.15.27:0/1332323567 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac007820 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.756+0000 7fa0c0ff9640 1 -- 172.21.15.27:0/1332323567 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0ac007980 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.756+0000 7fa0c0ff9640 1 -- 172.21.15.27:0/1332323567 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac007cf0 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.757+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1332323567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0c4152e60 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.757+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1332323567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0c4153420 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.758+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1332323567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0c404e840 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.759+0000 7fa0c0ff9640 1 -- 172.21.15.27:0/1332323567 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa0ac0113f0 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.760+0000 7fa0c0ff9640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0a406a430 0x7fa0a406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:42.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.760+0000 7fa0c0ff9640 1 -- 172.21.15.27:0/1332323567 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa0ac088ce0 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.760+0000 7fa0c37fe640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0a406a430 0x7fa0a406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:42.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.761+0000 7fa0c37fe640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0a406a430 0x7fa0a406c8f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4036000 tx=0x7fa0b403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:42.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.764+0000 7fa0c0ff9640 1 -- 172.21.15.27:0/1332323567 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa0ac054230 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.959+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1332323567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa0c40cae10 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.960+0000 7fa0c0ff9640 1 -- 172.21.15.27:0/1332323567 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v138) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa0ac01f0a0 con 0x7fa0c40b2fc0 2023-12-19T18:22:42.960 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:22:42.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.963+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0a406a430 msgr2=0x7fa0a406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:42.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.963+0000 7fa0c8a8c640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0a406a430 0x7fa0a406c8f0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fa0b4036000 tx=0x7fa0b403c040 comp rx=0 tx=0).stop 2023-12-19T18:22:42.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.963+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1332323567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c40b2fc0 msgr2=0x7fa0c4157f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:42.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.963+0000 7fa0c8a8c640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c40b2fc0 0x7fa0c4157f80 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac00c7f0 tx=0x7fa0ac00ccb0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.963+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1332323567 shutdown_connections 2023-12-19T18:22:42.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.963+0000 7fa0c8a8c640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa0a406a430 0x7fa0a406c8f0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.963+0000 7fa0c8a8c640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa0c4159330 0x7fa0c41f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.963+0000 7fa0c8a8c640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0c40b2fc0 0x7fa0c4157f80 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.963+0000 7fa0c8a8c640 1 --2- 172.21.15.27:0/1332323567 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa0c406f320 0x7fa0c4157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:42.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.963+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1332323567 >> 172.21.15.27:0/1332323567 conn(0x7fa0c406a060 msgr2=0x7fa0c406f770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:42.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.963+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1332323567 shutdown_connections 2023-12-19T18:22:42.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:42.963+0000 7fa0c8a8c640 1 -- 172.21.15.27:0/1332323567 wait complete. 2023-12-19T18:22:42.985 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:22:43.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.229+0000 7f38c78df640 1 Processor -- start 2023-12-19T18:22:43.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.229+0000 7f38c78df640 1 -- start start 2023-12-19T18:22:43.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.230+0000 7f38c78df640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f38c0160b70 0x7f38c00ae0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:43.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.230+0000 7f38c78df640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38c00cbb20 0x7f38c00ae630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:43.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.230+0000 7f38c78df640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38c00c9e80 con 0x7f38c00adc60 2023-12-19T18:22:43.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.230+0000 7f38c78df640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38c0160f50 con 0x7f38c0160b70 2023-12-19T18:22:43.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.230+0000 7f38c78df640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c00c9be0 con 0x7f38c00cbb20 2023-12-19T18:22:43.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.231+0000 7f38c567b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f38c0160b70 0x7f38c00ae0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:43.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.231+0000 7f38c567b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f38c0160b70 0x7f38c00ae0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35708/0 (socket says 172.21.15.27:35708) 2023-12-19T18:22:43.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.231+0000 7f38c567b640 1 -- 172.21.15.27:0/3503665679 learned_addr learned my addr 172.21.15.27:0/3503665679 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:43.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.231+0000 7f38c5e7c640 1 --2- 172.21.15.27:0/3503665679 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38c00cbb20 0x7f38c00ae630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:43.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.231+0000 7f38c567b640 1 -- 172.21.15.27:0/3503665679 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38c00cbb20 msgr2=0x7f38c00ae630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:43.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.231+0000 7f38c567b640 1 --2- 172.21.15.27:0/3503665679 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38c00cbb20 0x7f38c00ae630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.231+0000 7f38c567b640 1 -- 172.21.15.27:0/3503665679 >> v1:172.21.15.27:6789/0 conn(0x7f38c00adc60 legacy=0x7f38c00cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:43.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.231+0000 7f38c567b640 1 -- 172.21.15.27:0/3503665679 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38c00ad180 con 0x7f38c0160b70 2023-12-19T18:22:43.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.231+0000 7f38c5e7c640 1 --2- 172.21.15.27:0/3503665679 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38c00cbb20 0x7f38c00ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.232+0000 7f38c567b640 1 --2- 172.21.15.27:0/3503665679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c0160b70 0x7f38c00ae0f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009870 tx=0x7f38bc02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f3d6efbd7bbc48 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.233+0000 7f38b7fff640 1 -- 172.21.15.27:0/3503665679 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0317c0 con 0x7f38c0160b70 2023-12-19T18:22:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.233+0000 7f38b7fff640 1 -- 172.21.15.27:0/3503665679 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc031920 con 0x7f38c0160b70 2023-12-19T18:22:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.233+0000 7f38b7fff640 1 -- 172.21.15.27:0/3503665679 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc031cb0 con 0x7f38c0160b70 2023-12-19T18:22:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.233+0000 7f38c78df640 1 -- 172.21.15.27:0/3503665679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c0160b70 msgr2=0x7f38c00ae0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:43.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.233+0000 7f38c78df640 1 --2- 172.21.15.27:0/3503665679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c0160b70 0x7f38c00ae0f0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f38bc009870 tx=0x7f38bc02b860 comp rx=0 tx=0).stop 2023-12-19T18:22:43.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.233+0000 7f38c78df640 1 -- 172.21.15.27:0/3503665679 shutdown_connections 2023-12-19T18:22:43.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.233+0000 7f38c78df640 1 --2- 172.21.15.27:0/3503665679 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38c00cbb20 0x7f38c00ae630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.233+0000 7f38c78df640 1 --2- 172.21.15.27:0/3503665679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c0160b70 0x7f38c00ae0f0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.233+0000 7f38c78df640 1 -- 172.21.15.27:0/3503665679 >> 172.21.15.27:0/3503665679 conn(0x7f38c00cf3f0 msgr2=0x7f38c00cf800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:43.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.234+0000 7f38c78df640 1 -- 172.21.15.27:0/3503665679 shutdown_connections 2023-12-19T18:22:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.234+0000 7f38c78df640 1 -- 172.21.15.27:0/3503665679 wait complete. 2023-12-19T18:22:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.235+0000 7f38c78df640 1 Processor -- start 2023-12-19T18:22:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.236+0000 7f38c78df640 1 -- start start 2023-12-19T18:22:43.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c78df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00adc60 0x7f38c015a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c78df640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c00cbb20 0x7f38c00b4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c78df640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38c0160b70 0x7f38c00c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c78df640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c015a8e0 con 0x7f38c00adc60 2023-12-19T18:22:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c78df640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c00b4950 con 0x7f38c00cbb20 2023-12-19T18:22:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c78df640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38c00b4c30 con 0x7f38c0160b70 2023-12-19T18:22:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c567b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00adc60 0x7f38c015a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c567b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00adc60 0x7f38c015a3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35710/0 (socket says 172.21.15.27:35710) 2023-12-19T18:22:43.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c567b640 1 -- 172.21.15.27:0/2003207804 learned_addr learned my addr 172.21.15.27:0/2003207804 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c567b640 1 -- 172.21.15.27:0/2003207804 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38c0160b70 msgr2=0x7f38c00c7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c567b640 1 --2- 172.21.15.27:0/2003207804 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38c0160b70 0x7f38c00c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c567b640 1 -- 172.21.15.27:0/2003207804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c00cbb20 msgr2=0x7f38c00b4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c4e7a640 1 --2- 172.21.15.27:0/2003207804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c00cbb20 0x7f38c00b4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c567b640 1 --2- 172.21.15.27:0/2003207804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c00cbb20 0x7f38c00b4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.237+0000 7f38c567b640 1 -- 172.21.15.27:0/2003207804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc0094d0 con 0x7f38c00adc60 2023-12-19T18:22:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.238+0000 7f38c567b640 1 --2- 172.21.15.27:0/2003207804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00adc60 0x7f38c015a3a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f38bc000c00 tx=0x7f38bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.238+0000 7f38b5ffb640 1 -- 172.21.15.27:0/2003207804 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0329e0 con 0x7f38c00adc60 2023-12-19T18:22:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.238+0000 7f38b5ffb640 1 -- 172.21.15.27:0/2003207804 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38bc032b60 con 0x7f38c00adc60 2023-12-19T18:22:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.238+0000 7f38b5ffb640 1 -- 172.21.15.27:0/2003207804 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38bc0453f0 con 0x7f38c00adc60 2023-12-19T18:22:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.238+0000 7f38c78df640 1 -- 172.21.15.27:0/2003207804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c00aef40 con 0x7f38c00adc60 2023-12-19T18:22:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.238+0000 7f38c78df640 1 -- 172.21.15.27:0/2003207804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c00b4640 con 0x7f38c00adc60 2023-12-19T18:22:43.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.239+0000 7f38c78df640 1 -- 172.21.15.27:0/2003207804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c00aeda0 con 0x7f38c00adc60 2023-12-19T18:22:43.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.240+0000 7f38b5ffb640 1 -- 172.21.15.27:0/2003207804 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f38bc04d3f0 con 0x7f38c00adc60 2023-12-19T18:22:43.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.241+0000 7f38b5ffb640 1 --2- 172.21.15.27:0/2003207804 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f38a406a430 0x7f38a406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:43.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.242+0000 7f38b5ffb640 1 -- 172.21.15.27:0/2003207804 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f38bc0aa940 con 0x7f38c00adc60 2023-12-19T18:22:43.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.242+0000 7f38c4e7a640 1 --2- 172.21.15.27:0/2003207804 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f38a406a430 0x7f38a406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:43.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.243+0000 7f38c4e7a640 1 --2- 172.21.15.27:0/2003207804 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f38a406a430 0x7f38a406c8f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f38c00cb5f0 tx=0x7f38b0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:43.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.246+0000 7f38b5ffb640 1 -- 172.21.15.27:0/2003207804 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f38c00aeda0 con 0x7f38c00adc60 2023-12-19T18:22:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.444+0000 7f38c78df640 1 -- 172.21.15.27:0/2003207804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f38c00af460 con 0x7f38c00adc60 2023-12-19T18:22:43.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.445+0000 7f38b5ffb640 1 -- 172.21.15.27:0/2003207804 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v138) v1 ==== 104+0+11 (secure 0 0 0) 0x7f38c00af460 con 0x7f38c00adc60 2023-12-19T18:22:43.445 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:22:43.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.448+0000 7f38c78df640 1 -- 172.21.15.27:0/2003207804 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f38a406a430 msgr2=0x7f38a406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:43.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.448+0000 7f38c78df640 1 --2- 172.21.15.27:0/2003207804 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f38a406a430 0x7f38a406c8f0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f38c00cb5f0 tx=0x7f38b0009040 comp rx=0 tx=0).stop 2023-12-19T18:22:43.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.448+0000 7f38c78df640 1 -- 172.21.15.27:0/2003207804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00adc60 msgr2=0x7f38c015a3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:43.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.448+0000 7f38c78df640 1 --2- 172.21.15.27:0/2003207804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00adc60 0x7f38c015a3a0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f38bc000c00 tx=0x7f38bc0046c0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.448+0000 7f38c78df640 1 -- 172.21.15.27:0/2003207804 shutdown_connections 2023-12-19T18:22:43.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.448+0000 7f38c78df640 1 --2- 172.21.15.27:0/2003207804 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f38a406a430 0x7f38a406c8f0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.448+0000 7f38c78df640 1 --2- 172.21.15.27:0/2003207804 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f38c0160b70 0x7f38c00c7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.448+0000 7f38c78df640 1 --2- 172.21.15.27:0/2003207804 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f38c00cbb20 0x7f38c00b4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.448+0000 7f38c78df640 1 --2- 172.21.15.27:0/2003207804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c00adc60 0x7f38c015a3a0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.448+0000 7f38c78df640 1 -- 172.21.15.27:0/2003207804 >> 172.21.15.27:0/2003207804 conn(0x7f38c00cf3f0 msgr2=0x7f38c00cd050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:43.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.448+0000 7f38c78df640 1 -- 172.21.15.27:0/2003207804 shutdown_connections 2023-12-19T18:22:43.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.448+0000 7f38c78df640 1 -- 172.21.15.27:0/2003207804 wait complete. 2023-12-19T18:22:43.469 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:22:43.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.705+0000 7f8368b70640 1 Processor -- start 2023-12-19T18:22:43.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.706+0000 7f8368b70640 1 -- start start 2023-12-19T18:22:43.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.707+0000 7f8368b70640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83640cf580 0x7f836406d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:43.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.707+0000 7f8368b70640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83640b2020 con 0x7f836406baa0 2023-12-19T18:22:43.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.708+0000 7f8368b70640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83641654c0 con 0x7f836414ab60 2023-12-19T18:22:43.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.708+0000 7f8368b70640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83641657a0 con 0x7f83640cf580 2023-12-19T18:22:43.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.708+0000 7f835bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83640cf580 0x7f836406d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:43.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.708+0000 7f835bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f83640cf580 0x7f836406d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35726/0 (socket says 172.21.15.27:35726) 2023-12-19T18:22:43.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.708+0000 7f835bfff640 1 -- 172.21.15.27:0/1227970891 learned_addr learned my addr 172.21.15.27:0/1227970891 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.708+0000 7f835bfff640 1 -- 172.21.15.27:0/1227970891 >> v1:172.21.15.192:6789/0 conn(0x7f836414ab60 legacy=0x7f83640cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.708+0000 7f835bfff640 1 -- 172.21.15.27:0/1227970891 >> v1:172.21.15.27:6789/0 conn(0x7f836406baa0 legacy=0x7f83640b2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.708+0000 7f835bfff640 1 -- 172.21.15.27:0/1227970891 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8364165a80 con 0x7f83640cf580 2023-12-19T18:22:43.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.710+0000 7f835bfff640 1 --2- 172.21.15.27:0/1227970891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83640cf580 0x7f836406d170 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f834c009870 tx=0x7f834c02ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69521706f384f14d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.710+0000 7f8362ffd640 1 -- 172.21.15.27:0/1227970891 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c02f840 con 0x7f83640cf580 2023-12-19T18:22:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.710+0000 7f8362ffd640 1 -- 172.21.15.27:0/1227970891 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f834c02f9a0 con 0x7f83640cf580 2023-12-19T18:22:43.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.710+0000 7f8362ffd640 1 -- 172.21.15.27:0/1227970891 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834c02fd30 con 0x7f83640cf580 2023-12-19T18:22:43.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.711+0000 7f8368b70640 1 -- 172.21.15.27:0/1227970891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83640cf580 msgr2=0x7f836406d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:43.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.711+0000 7f8368b70640 1 --2- 172.21.15.27:0/1227970891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83640cf580 0x7f836406d170 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f834c009870 tx=0x7f834c02ba20 comp rx=0 tx=0).stop 2023-12-19T18:22:43.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.711+0000 7f8368b70640 1 -- 172.21.15.27:0/1227970891 shutdown_connections 2023-12-19T18:22:43.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.711+0000 7f8368b70640 1 --2- 172.21.15.27:0/1227970891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83640cf580 0x7f836406d170 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.711+0000 7f8368b70640 1 -- 172.21.15.27:0/1227970891 >> 172.21.15.27:0/1227970891 conn(0x7f836406a060 msgr2=0x7f836406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:43.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.711+0000 7f8368b70640 1 -- 172.21.15.27:0/1227970891 shutdown_connections 2023-12-19T18:22:43.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.711+0000 7f8368b70640 1 -- 172.21.15.27:0/1227970891 wait complete. 2023-12-19T18:22:43.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.713+0000 7f8368b70640 1 Processor -- start 2023-12-19T18:22:43.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.713+0000 7f8368b70640 1 -- start start 2023-12-19T18:22:43.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.714+0000 7f8368b70640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f836406baa0 0x7f83641f6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:43.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.714+0000 7f8368b70640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83640cf580 0x7f83641fadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:43.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.714+0000 7f8368b70640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f836414ab60 0x7f8364163f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:43.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.714+0000 7f8368b70640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83641f6cc0 con 0x7f836414ab60 2023-12-19T18:22:43.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.714+0000 7f8368b70640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83640acec0 con 0x7f836406baa0 2023-12-19T18:22:43.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.714+0000 7f8368b70640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83640aa560 con 0x7f83640cf580 2023-12-19T18:22:43.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.714+0000 7f8363fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f836414ab60 0x7f8364163f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:43.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.714+0000 7f8363fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f836414ab60 0x7f8364163f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35736/0 (socket says 172.21.15.27:35736) 2023-12-19T18:22:43.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.714+0000 7f8363fff640 1 -- 172.21.15.27:0/2480548699 learned_addr learned my addr 172.21.15.27:0/2480548699 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:43.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.714+0000 7f83637fe640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f836406baa0 0x7f83641f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:43.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.714+0000 7f835bfff640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83640cf580 0x7f83641fadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.715+0000 7f8363fff640 1 -- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83640cf580 msgr2=0x7f83641fadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.715+0000 7f8363fff640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83640cf580 0x7f83641fadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.715+0000 7f8363fff640 1 -- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f836406baa0 msgr2=0x7f83641f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.715+0000 7f8363fff640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f836406baa0 0x7f83641f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.715+0000 7f8363fff640 1 -- 172.21.15.27:0/2480548699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f834c0094d0 con 0x7f836414ab60 2023-12-19T18:22:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.715+0000 7f8363fff640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f836414ab60 0x7f8364163f10 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c730 tx=0x7f835c00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.715+0000 7f8360ff9640 1 -- 172.21.15.27:0/2480548699 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c003b70 con 0x7f836414ab60 2023-12-19T18:22:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.715+0000 7f8360ff9640 1 -- 172.21.15.27:0/2480548699 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f835c003cd0 con 0x7f836414ab60 2023-12-19T18:22:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.715+0000 7f8360ff9640 1 -- 172.21.15.27:0/2480548699 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f835c005310 con 0x7f836414ab60 2023-12-19T18:22:43.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.715+0000 7f8368b70640 1 -- 172.21.15.27:0/2480548699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83641fb2f0 con 0x7f836414ab60 2023-12-19T18:22:43.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.715+0000 7f8368b70640 1 -- 172.21.15.27:0/2480548699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83640ad440 con 0x7f836414ab60 2023-12-19T18:22:43.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.717+0000 7f8368b70640 1 -- 172.21.15.27:0/2480548699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f836404e840 con 0x7f836414ab60 2023-12-19T18:22:43.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.718+0000 7f8360ff9640 1 -- 172.21.15.27:0/2480548699 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f835c005470 con 0x7f836414ab60 2023-12-19T18:22:43.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.719+0000 7f8360ff9640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f834406a430 0x7f834406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:43.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.719+0000 7f8360ff9640 1 -- 172.21.15.27:0/2480548699 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f835c088750 con 0x7f836414ab60 2023-12-19T18:22:43.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.720+0000 7f83637fe640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f834406a430 0x7f834406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:43.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.721+0000 7f83637fe640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f834406a430 0x7f834406c8f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8354000f60 tx=0x7f8354009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:43.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.724+0000 7f8360ff9640 1 -- 172.21.15.27:0/2480548699 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f835c053ca0 con 0x7f836414ab60 2023-12-19T18:22:43.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.924+0000 7f8368b70640 1 -- 172.21.15.27:0/2480548699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f836406c790 con 0x7f836414ab60 2023-12-19T18:22:43.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.925+0000 7f8360ff9640 1 -- 172.21.15.27:0/2480548699 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v139) v1 ==== 100+0+11 (secure 0 0 0) 0x7f835c0572c0 con 0x7f836414ab60 2023-12-19T18:22:43.925 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:22:43.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.928+0000 7f8368b70640 1 -- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f834406a430 msgr2=0x7f834406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:43.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.928+0000 7f8368b70640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f834406a430 0x7f834406c8f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8354000f60 tx=0x7f8354009040 comp rx=0 tx=0).stop 2023-12-19T18:22:43.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.928+0000 7f8368b70640 1 -- 172.21.15.27:0/2480548699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f836414ab60 msgr2=0x7f8364163f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:43.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.928+0000 7f8368b70640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f836414ab60 0x7f8364163f10 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f835c00c730 tx=0x7f835c00cb30 comp rx=0 tx=0).stop 2023-12-19T18:22:43.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.928+0000 7f8368b70640 1 -- 172.21.15.27:0/2480548699 shutdown_connections 2023-12-19T18:22:43.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.928+0000 7f8368b70640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f834406a430 0x7f834406c8f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.928+0000 7f8368b70640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f836414ab60 0x7f8364163f10 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.928+0000 7f8368b70640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f83640cf580 0x7f83641fadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.928+0000 7f8368b70640 1 --2- 172.21.15.27:0/2480548699 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f836406baa0 0x7f83641f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:43.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.928+0000 7f8368b70640 1 -- 172.21.15.27:0/2480548699 >> 172.21.15.27:0/2480548699 conn(0x7f836406a060 msgr2=0x7f83640afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:43.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.929+0000 7f8368b70640 1 -- 172.21.15.27:0/2480548699 shutdown_connections 2023-12-19T18:22:43.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:43.929+0000 7f8368b70640 1 -- 172.21.15.27:0/2480548699 wait complete. 2023-12-19T18:22:43.947 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:22:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.173+0000 7f9f6384b640 1 Processor -- start 2023-12-19T18:22:44.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.173+0000 7f9f6384b640 1 -- start start 2023-12-19T18:22:44.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.174+0000 7f9f6384b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9f5c0d0b90 0x7f9f5c166020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.174+0000 7f9f6384b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5c0ce130 con 0x7f9f5c0cb3d0 2023-12-19T18:22:44.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.175+0000 7f9f6384b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f5c0d0250 con 0x7f9f5c15b020 2023-12-19T18:22:44.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.175+0000 7f9f6384b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f5c15c2c0 con 0x7f9f5c0d0b90 2023-12-19T18:22:44.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.175+0000 7f9f60de6640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9f5c0cb3d0 0x7f9f5c0d0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47908/0 (socket says 172.21.15.27:47908) 2023-12-19T18:22:44.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.175+0000 7f9f60de6640 1 -- 172.21.15.27:0/1980913666 learned_addr learned my addr 172.21.15.27:0/1980913666 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:44.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.175+0000 7f9f61de8640 1 --2- 172.21.15.27:0/1980913666 >> v2:172.21.15.192:3300/0 conn(0x7f9f5c0d0b90 0x7f9f5c166020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:44.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.175+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163602092 0 0) 0x7f9f5c0ce130 con 0x7f9f5c0cb3d0 2023-12-19T18:22:44.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.175+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f44003520 con 0x7f9f5c0cb3d0 2023-12-19T18:22:44.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3496639785 0 0) 0x7f9f44003520 con 0x7f9f5c0cb3d0 2023-12-19T18:22:44.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 >> v2:172.21.15.192:3300/0 conn(0x7f9f5c0d0b90 msgr2=0x7f9f5c166020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:44.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f5bfff640 1 --2- 172.21.15.27:0/1980913666 >> v2:172.21.15.192:3300/0 conn(0x7f9f5c0d0b90 0x7f9f5c166020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 >> v1:172.21.15.192:6789/0 conn(0x7f9f5c15b020 legacy=0x7f9f5c0cda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:44.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c15c5a0 con 0x7f9f5c0cb3d0 2023-12-19T18:22:44.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f9f50001670 con 0x7f9f5c0cb3d0 2023-12-19T18:22:44.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 >> v1:172.21.15.27:6789/0 conn(0x7f9f5c0cb3d0 legacy=0x7f9f5c0d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:44.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f5bfff640 1 --2- 172.21.15.27:0/1980913666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f440039c0 0x7f9f44003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f5bfff640 1 --2- 172.21.15.27:0/1980913666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f440047c0 0x7f9f44004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f5bfff640 1 --2- 172.21.15.27:0/1980913666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f44005180 0x7f9f4400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c15c2c0 con 0x7f9f440047c0 2023-12-19T18:22:44.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4400c080 con 0x7f9f440039c0 2023-12-19T18:22:44.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f4400c240 con 0x7f9f44005180 2023-12-19T18:22:44.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.176+0000 7f9f60de6640 1 --2- 172.21.15.27:0/1980913666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f440047c0 0x7f9f44004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:44.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.177+0000 7f9f615e7640 1 --2- 172.21.15.27:0/1980913666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f440039c0 0x7f9f44003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:44.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.177+0000 7f9f61de8640 1 --2- 172.21.15.27:0/1980913666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f44005180 0x7f9f4400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:44.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.177+0000 7f9f60de6640 1 -- 172.21.15.27:0/1980913666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f44005180 msgr2=0x7f9f4400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:44.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.177+0000 7f9f60de6640 1 --2- 172.21.15.27:0/1980913666 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f44005180 0x7f9f4400b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.177+0000 7f9f60de6640 1 -- 172.21.15.27:0/1980913666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f440039c0 msgr2=0x7f9f44003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:44.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.177+0000 7f9f60de6640 1 --2- 172.21.15.27:0/1980913666 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f440039c0 0x7f9f44003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.177+0000 7f9f60de6640 1 -- 172.21.15.27:0/1980913666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f44003520 con 0x7f9f440047c0 2023-12-19T18:22:44.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.177+0000 7f9f61de8640 1 -- 172.21.15.27:0/1980913666 reap_dead start 2023-12-19T18:22:44.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.178+0000 7f9f60de6640 1 --2- 172.21.15.27:0/1980913666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f440047c0 0x7f9f44004c40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000c0e0 tx=0x7f9f440057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93490686f1755aa5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.178+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50005ad0 con 0x7f9f440047c0 2023-12-19T18:22:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.178+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f50005c50 con 0x7f9f440047c0 2023-12-19T18:22:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.178+0000 7f9f5bfff640 1 -- 172.21.15.27:0/1980913666 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f50007070 con 0x7f9f440047c0 2023-12-19T18:22:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.179+0000 7f9f6384b640 1 -- 172.21.15.27:0/1980913666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f440047c0 msgr2=0x7f9f44004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.179+0000 7f9f6384b640 1 --2- 172.21.15.27:0/1980913666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f440047c0 0x7f9f44004c40 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f9f5000c0e0 tx=0x7f9f440057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.179+0000 7f9f6384b640 1 -- 172.21.15.27:0/1980913666 shutdown_connections 2023-12-19T18:22:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.179+0000 7f9f6384b640 1 --2- 172.21.15.27:0/1980913666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f440047c0 0x7f9f44004c40 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.179+0000 7f9f6384b640 1 -- 172.21.15.27:0/1980913666 >> 172.21.15.27:0/1980913666 conn(0x7f9f5c072c40 msgr2=0x7f9f5c073020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.179+0000 7f9f6384b640 1 -- 172.21.15.27:0/1980913666 shutdown_connections 2023-12-19T18:22:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.179+0000 7f9f6384b640 1 -- 172.21.15.27:0/1980913666 wait complete. 2023-12-19T18:22:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.181+0000 7f9f6384b640 1 Processor -- start 2023-12-19T18:22:44.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.181+0000 7f9f6384b640 1 -- start start 2023-12-19T18:22:44.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.182+0000 7f9f6384b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f440047c0 0x7f9f5c0c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.182+0000 7f9f6384b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5c14e270 0x7f9f5c1f0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.182+0000 7f9f6384b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c158490 0x7f9f5c158910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.182+0000 7f9f6384b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c0c4950 con 0x7f9f5c158490 2023-12-19T18:22:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.182+0000 7f9f6384b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c158e50 con 0x7f9f440047c0 2023-12-19T18:22:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.182+0000 7f9f6384b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c0c4b60 con 0x7f9f5c14e270 2023-12-19T18:22:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.182+0000 7f9f61de8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c158490 0x7f9f5c158910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.182+0000 7f9f61de8640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c158490 0x7f9f5c158910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35754/0 (socket says 172.21.15.27:35754) 2023-12-19T18:22:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.182+0000 7f9f61de8640 1 -- 172.21.15.27:0/2850277296 learned_addr learned my addr 172.21.15.27:0/2850277296 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.182+0000 7f9f615e7640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f440047c0 0x7f9f5c0c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.182+0000 7f9f60de6640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5c14e270 0x7f9f5c1f0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.183+0000 7f9f61de8640 1 -- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5c14e270 msgr2=0x7f9f5c1f0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.183+0000 7f9f61de8640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5c14e270 0x7f9f5c1f0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.183+0000 7f9f61de8640 1 -- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f440047c0 msgr2=0x7f9f5c0c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.183+0000 7f9f61de8640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f440047c0 0x7f9f5c0c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.183+0000 7f9f61de8640 1 -- 172.21.15.27:0/2850277296 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c0c4e40 con 0x7f9f5c158490 2023-12-19T18:22:44.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.183+0000 7f9f61de8640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c158490 0x7f9f5c158910 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0d1d60 tx=0x7f9f54010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:44.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.183+0000 7f9f59ffb640 1 -- 172.21.15.27:0/2850277296 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400e840 con 0x7f9f5c158490 2023-12-19T18:22:44.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.183+0000 7f9f59ffb640 1 -- 172.21.15.27:0/2850277296 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5400e9c0 con 0x7f9f5c158490 2023-12-19T18:22:44.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.183+0000 7f9f6384b640 1 -- 172.21.15.27:0/2850277296 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5c0c5120 con 0x7f9f5c158490 2023-12-19T18:22:44.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.183+0000 7f9f59ffb640 1 -- 172.21.15.27:0/2850277296 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5400ed70 con 0x7f9f5c158490 2023-12-19T18:22:44.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.183+0000 7f9f6384b640 1 -- 172.21.15.27:0/2850277296 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c0c56e0 con 0x7f9f5c158490 2023-12-19T18:22:44.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.184+0000 7f9f6384b640 1 -- 172.21.15.27:0/2850277296 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5c0c5900 con 0x7f9f5c158490 2023-12-19T18:22:44.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.185+0000 7f9f59ffb640 1 -- 172.21.15.27:0/2850277296 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9f540253f0 con 0x7f9f5c158490 2023-12-19T18:22:44.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.186+0000 7f9f59ffb640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f4806a460 0x7f9f4806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.187+0000 7f9f59ffb640 1 -- 172.21.15.27:0/2850277296 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f9f54088b60 con 0x7f9f5c158490 2023-12-19T18:22:44.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.187+0000 7f9f615e7640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f4806a460 0x7f9f4806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:44.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.188+0000 7f9f615e7640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f4806a460 0x7f9f4806c920 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c005290 tx=0x7f9f4c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:44.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.191+0000 7f9f59ffb640 1 -- 172.21.15.27:0/2850277296 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9f54054180 con 0x7f9f5c158490 2023-12-19T18:22:44.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.396+0000 7f9f6384b640 1 -- 172.21.15.27:0/2850277296 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9f5c068e90 con 0x7f9f5c158490 2023-12-19T18:22:44.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.397+0000 7f9f59ffb640 1 -- 172.21.15.27:0/2850277296 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v139) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9f54000e60 con 0x7f9f5c158490 2023-12-19T18:22:44.397 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:22:44.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.400+0000 7f9f6384b640 1 -- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f4806a460 msgr2=0x7f9f4806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:44.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.400+0000 7f9f6384b640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f4806a460 0x7f9f4806c920 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c005290 tx=0x7f9f4c009040 comp rx=0 tx=0).stop 2023-12-19T18:22:44.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.400+0000 7f9f6384b640 1 -- 172.21.15.27:0/2850277296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c158490 msgr2=0x7f9f5c158910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:44.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.400+0000 7f9f6384b640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c158490 0x7f9f5c158910 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0d1d60 tx=0x7f9f54010c20 comp rx=0 tx=0).stop 2023-12-19T18:22:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.400+0000 7f9f6384b640 1 -- 172.21.15.27:0/2850277296 shutdown_connections 2023-12-19T18:22:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.400+0000 7f9f6384b640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f4806a460 0x7f9f4806c920 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.400+0000 7f9f6384b640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c158490 0x7f9f5c158910 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.400+0000 7f9f6384b640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f5c14e270 0x7f9f5c1f0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.400+0000 7f9f6384b640 1 --2- 172.21.15.27:0/2850277296 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f440047c0 0x7f9f5c0c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.400+0000 7f9f6384b640 1 -- 172.21.15.27:0/2850277296 >> 172.21.15.27:0/2850277296 conn(0x7f9f5c072c40 msgr2=0x7f9f5c15a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.400+0000 7f9f6384b640 1 -- 172.21.15.27:0/2850277296 shutdown_connections 2023-12-19T18:22:44.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.400+0000 7f9f6384b640 1 -- 172.21.15.27:0/2850277296 wait complete. 2023-12-19T18:22:44.418 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-12-19T18:22:44.418 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:44.418 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-19T18:22:44.518 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[109914]: fuse finished with error 0 and tester_r 0 2023-12-19T18:22:44.525 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:22:44.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:44.525 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:22:44.611 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:22:44.611 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:22:44.657 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:22:44.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.882+0000 7ffa752b9640 1 Processor -- start 2023-12-19T18:22:44.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.882+0000 7ffa752b9640 1 -- start start 2023-12-19T18:22:44.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.883+0000 7ffa752b9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa701604d0 0x7ffa700b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.883+0000 7ffa752b9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffa700d2350 0x7ffa700b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.883+0000 7ffa752b9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa700b5e90 con 0x7ffa700d0300 2023-12-19T18:22:44.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.883+0000 7ffa752b9640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa700d2b30 con 0x7ffa700d2350 2023-12-19T18:22:44.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.883+0000 7ffa752b9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700d2850 con 0x7ffa701604d0 2023-12-19T18:22:44.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.883+0000 7ffa74ab8640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ffa700d0300 0x7ffa700d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47924/0 (socket says 172.21.15.27:47924) 2023-12-19T18:22:44.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.883+0000 7ffa6f7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ffa700d2350 0x7ffa700b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:44.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.884+0000 7ffa74ab8640 1 -- 172.21.15.27:0/1523423984 learned_addr learned my addr 172.21.15.27:0/1523423984 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:44.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.884+0000 7ffa6ffff640 1 --2- 172.21.15.27:0/1523423984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa701604d0 0x7ffa700b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:44.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.884+0000 7ffa6f7fe640 1 -- 172.21.15.27:0/1523423984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa701604d0 msgr2=0x7ffa700b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:44.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.884+0000 7ffa6f7fe640 1 --2- 172.21.15.27:0/1523423984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa701604d0 0x7ffa700b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.884+0000 7ffa6f7fe640 1 -- 172.21.15.27:0/1523423984 >> v1:172.21.15.27:6789/0 conn(0x7ffa700d0300 legacy=0x7ffa700d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:44.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.884+0000 7ffa6effd640 1 -- 172.21.15.27:0/1523423984 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952820396 0 0) 0x7ffa700b5e90 con 0x7ffa700d0300 2023-12-19T18:22:44.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.884+0000 7ffa6f7fe640 1 -- 172.21.15.27:0/1523423984 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa700cded0 con 0x7ffa700d2350 2023-12-19T18:22:44.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.884+0000 7ffa6ffff640 1 --2- 172.21.15.27:0/1523423984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa701604d0 0x7ffa700b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:44.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.885+0000 7ffa6f7fe640 1 --2- 172.21.15.27:0/1523423984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa700d2350 0x7ffa700b4d20 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ffa580097a0 tx=0x7ffa5802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80fd48bc65718a32 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:44.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.885+0000 7ffa6effd640 1 -- 172.21.15.27:0/1523423984 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa5802d040 con 0x7ffa700d2350 2023-12-19T18:22:44.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.885+0000 7ffa6effd640 1 -- 172.21.15.27:0/1523423984 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa58031810 con 0x7ffa700d2350 2023-12-19T18:22:44.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.885+0000 7ffa6effd640 1 -- 172.21.15.27:0/1523423984 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58031ba0 con 0x7ffa700d2350 2023-12-19T18:22:44.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.886+0000 7ffa752b9640 1 -- 172.21.15.27:0/1523423984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa700d2350 msgr2=0x7ffa700b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:44.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.886+0000 7ffa752b9640 1 --2- 172.21.15.27:0/1523423984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa700d2350 0x7ffa700b4d20 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7ffa580097a0 tx=0x7ffa5802b8b0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.886+0000 7ffa752b9640 1 -- 172.21.15.27:0/1523423984 shutdown_connections 2023-12-19T18:22:44.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.886+0000 7ffa752b9640 1 --2- 172.21.15.27:0/1523423984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa700d2350 0x7ffa700b4d20 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.886+0000 7ffa752b9640 1 --2- 172.21.15.27:0/1523423984 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa701604d0 0x7ffa700b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.886+0000 7ffa752b9640 1 -- 172.21.15.27:0/1523423984 >> 172.21.15.27:0/1523423984 conn(0x7ffa700cf7e0 msgr2=0x7ffa700cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:44.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.886+0000 7ffa752b9640 1 -- 172.21.15.27:0/1523423984 shutdown_connections 2023-12-19T18:22:44.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.886+0000 7ffa752b9640 1 -- 172.21.15.27:0/1523423984 wait complete. 2023-12-19T18:22:44.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.888+0000 7ffa752b9640 1 Processor -- start 2023-12-19T18:22:44.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.888+0000 7ffa752b9640 1 -- start start 2023-12-19T18:22:44.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.889+0000 7ffa752b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa700d0300 0x7ffa700c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.889+0000 7ffa752b9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa700d2350 0x7ffa701f1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.889+0000 7ffa752b9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa701604d0 0x7ffa701f2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.889+0000 7ffa752b9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa70149500 con 0x7ffa700d0300 2023-12-19T18:22:44.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.889+0000 7ffa752b9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700c6e50 con 0x7ffa700d2350 2023-12-19T18:22:44.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.889+0000 7ffa752b9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa700b3090 con 0x7ffa701604d0 2023-12-19T18:22:44.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.889+0000 7ffa6ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa700d0300 0x7ffa700c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:44.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.889+0000 7ffa6ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa700d0300 0x7ffa700c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35770/0 (socket says 172.21.15.27:35770) 2023-12-19T18:22:44.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.889+0000 7ffa6ffff640 1 -- 172.21.15.27:0/1581254954 learned_addr learned my addr 172.21.15.27:0/1581254954 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:44.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.890+0000 7ffa6ffff640 1 -- 172.21.15.27:0/1581254954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa701604d0 msgr2=0x7ffa701f2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:44.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.890+0000 7ffa6ffff640 1 --2- 172.21.15.27:0/1581254954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa701604d0 0x7ffa701f2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.890+0000 7ffa6ffff640 1 -- 172.21.15.27:0/1581254954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa700d2350 msgr2=0x7ffa701f1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:44.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.890+0000 7ffa6f7fe640 1 --2- 172.21.15.27:0/1581254954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa700d2350 0x7ffa701f1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:44.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.890+0000 7ffa6ffff640 1 --2- 172.21.15.27:0/1581254954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa700d2350 0x7ffa701f1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:44.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.890+0000 7ffa6ffff640 1 -- 172.21.15.27:0/1581254954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa58009400 con 0x7ffa700d0300 2023-12-19T18:22:44.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.890+0000 7ffa6ffff640 1 --2- 172.21.15.27:0/1581254954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa700d0300 0x7ffa700c6870 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000c7f0 tx=0x7ffa6000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:44.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.890+0000 7ffa6cff9640 1 -- 172.21.15.27:0/1581254954 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60007a30 con 0x7ffa700d0300 2023-12-19T18:22:44.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.891+0000 7ffa6cff9640 1 -- 172.21.15.27:0/1581254954 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa60007b90 con 0x7ffa700d0300 2023-12-19T18:22:44.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.891+0000 7ffa752b9640 1 -- 172.21.15.27:0/1581254954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa700b3370 con 0x7ffa700d0300 2023-12-19T18:22:44.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.891+0000 7ffa6cff9640 1 -- 172.21.15.27:0/1581254954 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000ce40 con 0x7ffa700d0300 2023-12-19T18:22:44.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.891+0000 7ffa752b9640 1 -- 172.21.15.27:0/1581254954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa700b2cd0 con 0x7ffa700d0300 2023-12-19T18:22:44.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.892+0000 7ffa752b9640 1 -- 172.21.15.27:0/1581254954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa70066f80 con 0x7ffa700d0300 2023-12-19T18:22:44.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.893+0000 7ffa6cff9640 1 -- 172.21.15.27:0/1581254954 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ffa600113f0 con 0x7ffa700d0300 2023-12-19T18:22:44.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.894+0000 7ffa6cff9640 1 --2- 172.21.15.27:0/1581254954 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffa5006a430 0x7ffa5006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:44.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.894+0000 7ffa6cff9640 1 -- 172.21.15.27:0/1581254954 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7ffa60088cc0 con 0x7ffa700d0300 2023-12-19T18:22:44.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.894+0000 7ffa6f7fe640 1 --2- 172.21.15.27:0/1581254954 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffa5006a430 0x7ffa5006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:44.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.895+0000 7ffa6f7fe640 1 --2- 172.21.15.27:0/1581254954 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffa5006a430 0x7ffa5006c8f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ffa58036000 tx=0x7ffa5803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:44.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:44.898+0000 7ffa6cff9640 1 -- 172.21.15.27:0/1581254954 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ffa6008f080 con 0x7ffa700d0300 2023-12-19T18:22:45.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.094+0000 7ffa752b9640 1 -- 172.21.15.27:0/1581254954 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa7004e840 con 0x7ffa700d0300 2023-12-19T18:22:45.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.097+0000 7ffa6cff9640 1 -- 172.21.15.27:0/1581254954 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16203 (secure 0 0 0) 0x7ffa6001f020 con 0x7ffa700d0300 2023-12-19T18:22:45.097 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:45.098 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":140,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:22:44.753680+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:22:12.328518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T18:22:13.251708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":5,"to":3}]},{"pgid":"14.5","mappings":[{"from":5,"to":0}]},{"pgid":"14.1f","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:22:45.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.102+0000 7ffa752b9640 1 -- 172.21.15.27:0/1581254954 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffa5006a430 msgr2=0x7ffa5006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.102+0000 7ffa752b9640 1 --2- 172.21.15.27:0/1581254954 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffa5006a430 0x7ffa5006c8f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7ffa58036000 tx=0x7ffa5803c040 comp rx=0 tx=0).stop 2023-12-19T18:22:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.102+0000 7ffa752b9640 1 -- 172.21.15.27:0/1581254954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa700d0300 msgr2=0x7ffa700c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.102+0000 7ffa752b9640 1 --2- 172.21.15.27:0/1581254954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa700d0300 0x7ffa700c6870 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7ffa6000c7f0 tx=0x7ffa6000cbf0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.102+0000 7ffa752b9640 1 -- 172.21.15.27:0/1581254954 shutdown_connections 2023-12-19T18:22:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.102+0000 7ffa752b9640 1 --2- 172.21.15.27:0/1581254954 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffa5006a430 0x7ffa5006c8f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.102+0000 7ffa752b9640 1 --2- 172.21.15.27:0/1581254954 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffa701604d0 0x7ffa701f2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.102+0000 7ffa752b9640 1 --2- 172.21.15.27:0/1581254954 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffa700d2350 0x7ffa701f1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.102+0000 7ffa752b9640 1 --2- 172.21.15.27:0/1581254954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa700d0300 0x7ffa700c6870 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.102+0000 7ffa752b9640 1 -- 172.21.15.27:0/1581254954 >> 172.21.15.27:0/1581254954 conn(0x7ffa700cf7e0 msgr2=0x7ffa700b4a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.103+0000 7ffa752b9640 1 -- 172.21.15.27:0/1581254954 shutdown_connections 2023-12-19T18:22:45.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.103+0000 7ffa752b9640 1 -- 172.21.15.27:0/1581254954 wait complete. 2023-12-19T18:22:45.121 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:22:12.328518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:22:13.251708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:22:45.121 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:22:45.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.351+0000 7fd87759e640 1 Processor -- start 2023-12-19T18:22:45.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.351+0000 7fd87759e640 1 -- start start 2023-12-19T18:22:45.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.353+0000 7fd87759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd87815d120 0x7fd8780cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:45.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.353+0000 7fd87759e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8780d3850 con 0x7fd8780ce950 2023-12-19T18:22:45.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.353+0000 7fd87759e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8780d0b10 con 0x7fd8780d1310 2023-12-19T18:22:45.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.353+0000 7fd87759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd87815d500 con 0x7fd87815d120 2023-12-19T18:22:45.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.353+0000 7fd876d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd8780ce950 0x7fd8780b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47940/0 (socket says 172.21.15.27:47940) 2023-12-19T18:22:45.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.353+0000 7fd876d9d640 1 -- 172.21.15.27:0/545531077 learned_addr learned my addr 172.21.15.27:0/545531077 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:45.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.353+0000 7fd87659c640 1 --2- 172.21.15.27:0/545531077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd87815d120 0x7fd8780cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:45.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.354+0000 7fd87559a640 1 -- 172.21.15.27:0/545531077 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736939459 0 0) 0x7fd8780d3850 con 0x7fd8780ce950 2023-12-19T18:22:45.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.354+0000 7fd87559a640 1 -- 172.21.15.27:0/545531077 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd860003520 con 0x7fd8780ce950 2023-12-19T18:22:45.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.354+0000 7fd87659c640 1 -- 172.21.15.27:0/545531077 >> v1:172.21.15.192:6789/0 conn(0x7fd8780d1310 legacy=0x7fd8780b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.354+0000 7fd87559a640 1 -- 172.21.15.27:0/545531077 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674128583 0 0) 0x7fd8780d0b10 con 0x7fd8780d1310 2023-12-19T18:22:45.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.354+0000 7fd87659c640 1 -- 172.21.15.27:0/545531077 >> v1:172.21.15.27:6789/0 conn(0x7fd8780ce950 legacy=0x7fd8780b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.354+0000 7fd87659c640 1 -- 172.21.15.27:0/545531077 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8780d1040 con 0x7fd87815d120 2023-12-19T18:22:45.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.356+0000 7fd87659c640 1 --2- 172.21.15.27:0/545531077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd87815d120 0x7fd8780cfd70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd8680097d0 tx=0x7fd86802b930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dca2104c35334e6b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:45.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.356+0000 7fd87559a640 1 -- 172.21.15.27:0/545531077 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868031890 con 0x7fd87815d120 2023-12-19T18:22:45.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.357+0000 7fd87559a640 1 -- 172.21.15.27:0/545531077 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8680319f0 con 0x7fd87815d120 2023-12-19T18:22:45.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.357+0000 7fd87559a640 1 -- 172.21.15.27:0/545531077 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd868031d60 con 0x7fd87815d120 2023-12-19T18:22:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.357+0000 7fd87759e640 1 -- 172.21.15.27:0/545531077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd87815d120 msgr2=0x7fd8780cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.357+0000 7fd87759e640 1 --2- 172.21.15.27:0/545531077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd87815d120 0x7fd8780cfd70 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd8680097d0 tx=0x7fd86802b930 comp rx=0 tx=0).stop 2023-12-19T18:22:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.357+0000 7fd87759e640 1 -- 172.21.15.27:0/545531077 shutdown_connections 2023-12-19T18:22:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.357+0000 7fd87759e640 1 --2- 172.21.15.27:0/545531077 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd87815d120 0x7fd8780cfd70 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.357+0000 7fd87759e640 1 -- 172.21.15.27:0/545531077 >> 172.21.15.27:0/545531077 conn(0x7fd8780d2d30 msgr2=0x7fd8780d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.357+0000 7fd87759e640 1 -- 172.21.15.27:0/545531077 shutdown_connections 2023-12-19T18:22:45.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.357+0000 7fd87759e640 1 -- 172.21.15.27:0/545531077 wait complete. 2023-12-19T18:22:45.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.359+0000 7fd87759e640 1 Processor -- start 2023-12-19T18:22:45.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.359+0000 7fd87759e640 1 -- start start 2023-12-19T18:22:45.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.360+0000 7fd87759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8780ce950 0x7fd878158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:45.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.360+0000 7fd87759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8780d1310 0x7fd87815ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:45.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.360+0000 7fd87759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87815d120 0x7fd87815bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:45.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.360+0000 7fd87759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8781589d0 con 0x7fd87815d120 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.360+0000 7fd87759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8780b5950 con 0x7fd8780ce950 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.360+0000 7fd87759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8780b5bf0 con 0x7fd8780d1310 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.361+0000 7fd876d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87815d120 0x7fd87815bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.361+0000 7fd876d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87815d120 0x7fd87815bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35784/0 (socket says 172.21.15.27:35784) 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.361+0000 7fd876d9d640 1 -- 172.21.15.27:0/1343281026 learned_addr learned my addr 172.21.15.27:0/1343281026 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.361+0000 7fd875d9b640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8780d1310 0x7fd87815ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.361+0000 7fd87659c640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8780ce950 0x7fd878158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.361+0000 7fd876d9d640 1 -- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8780d1310 msgr2=0x7fd87815ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.361+0000 7fd876d9d640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8780d1310 0x7fd87815ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.361+0000 7fd876d9d640 1 -- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8780ce950 msgr2=0x7fd878158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.361+0000 7fd876d9d640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8780ce950 0x7fd878158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.361+0000 7fd876d9d640 1 -- 172.21.15.27:0/1343281026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd868009400 con 0x7fd87815d120 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.361+0000 7fd87659c640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8780ce950 0x7fd878158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:45.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.361+0000 7fd875d9b640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8780d1310 0x7fd87815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:45.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.361+0000 7fd876d9d640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87815d120 0x7fd87815bfb0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd87000ca20 tx=0x7fd87000cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:45.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.362+0000 7fd866ffd640 1 -- 172.21.15.27:0/1343281026 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd87000abd0 con 0x7fd87815d120 2023-12-19T18:22:45.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.362+0000 7fd866ffd640 1 -- 172.21.15.27:0/1343281026 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd87000ad30 con 0x7fd87815d120 2023-12-19T18:22:45.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.362+0000 7fd866ffd640 1 -- 172.21.15.27:0/1343281026 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd870012620 con 0x7fd87815d120 2023-12-19T18:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.362+0000 7fd87759e640 1 -- 172.21.15.27:0/1343281026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8780b5ed0 con 0x7fd87815d120 2023-12-19T18:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.362+0000 7fd87759e640 1 -- 172.21.15.27:0/1343281026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8780b6490 con 0x7fd87815d120 2023-12-19T18:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.363+0000 7fd87759e640 1 -- 172.21.15.27:0/1343281026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd87804e840 con 0x7fd87815d120 2023-12-19T18:22:45.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.364+0000 7fd866ffd640 1 -- 172.21.15.27:0/1343281026 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd870007070 con 0x7fd87815d120 2023-12-19T18:22:45.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.365+0000 7fd866ffd640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd85806a430 0x7fd85806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:45.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.365+0000 7fd866ffd640 1 -- 172.21.15.27:0/1343281026 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fd870088950 con 0x7fd87815d120 2023-12-19T18:22:45.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.366+0000 7fd87659c640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd85806a430 0x7fd85806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:45.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.367+0000 7fd87659c640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd85806a430 0x7fd85806c8f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd868036000 tx=0x7fd868008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:45.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.369+0000 7fd866ffd640 1 -- 172.21.15.27:0/1343281026 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd870053e40 con 0x7fd87815d120 2023-12-19T18:22:45.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.562+0000 7fd87759e640 1 -- 172.21.15.27:0/1343281026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8780b1910 con 0x7fd87815d120 2023-12-19T18:22:45.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.563+0000 7fd866ffd640 1 -- 172.21.15.27:0/1343281026 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd870017070 con 0x7fd87815d120 2023-12-19T18:22:45.563 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:22:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.565+0000 7fd87759e640 1 -- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd85806a430 msgr2=0x7fd85806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.565+0000 7fd87759e640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd85806a430 0x7fd85806c8f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd868036000 tx=0x7fd868008030 comp rx=0 tx=0).stop 2023-12-19T18:22:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.565+0000 7fd87759e640 1 -- 172.21.15.27:0/1343281026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87815d120 msgr2=0x7fd87815bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.565+0000 7fd87759e640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87815d120 0x7fd87815bfb0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fd87000ca20 tx=0x7fd87000cee0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.566+0000 7fd87759e640 1 -- 172.21.15.27:0/1343281026 shutdown_connections 2023-12-19T18:22:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.566+0000 7fd87759e640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd85806a430 0x7fd85806c8f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.566+0000 7fd87759e640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd87815d120 0x7fd87815bfb0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.566+0000 7fd87759e640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8780d1310 0x7fd87815ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.566+0000 7fd87759e640 1 --2- 172.21.15.27:0/1343281026 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8780ce950 0x7fd878158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.566+0000 7fd87759e640 1 -- 172.21.15.27:0/1343281026 >> 172.21.15.27:0/1343281026 conn(0x7fd8780d2d30 msgr2=0x7fd87806a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:45.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.566+0000 7fd87759e640 1 -- 172.21.15.27:0/1343281026 shutdown_connections 2023-12-19T18:22:45.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.566+0000 7fd87759e640 1 -- 172.21.15.27:0/1343281026 wait complete. 2023-12-19T18:22:45.585 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:22:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.821+0000 7fe2d6b55640 1 Processor -- start 2023-12-19T18:22:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.821+0000 7fe2d6b55640 1 -- start start 2023-12-19T18:22:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.822+0000 7fe2d6b55640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe2d006caa0 0x7fe2d006ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.822+0000 7fe2d6b55640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d00af3a0 con 0x7fe2d00cf810 2023-12-19T18:22:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.822+0000 7fe2d6b55640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2d0071370 con 0x7fe2d00cbb70 2023-12-19T18:22:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.822+0000 7fe2d6b55640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2d0071930 con 0x7fe2d006caa0 2023-12-19T18:22:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.823+0000 7fe2cffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe2d00cf810 0x7fe2d00aec90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47942/0 (socket says 172.21.15.27:47942) 2023-12-19T18:22:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.823+0000 7fe2cffff640 1 -- 172.21.15.27:0/2957509529 learned_addr learned my addr 172.21.15.27:0/2957509529 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.823+0000 7fe2d50f2640 1 --2- 172.21.15.27:0/2957509529 >> v2:172.21.15.27:3300/0 conn(0x7fe2d006caa0 0x7fe2d006ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.823+0000 7fe2cf7fe640 1 -- 172.21.15.27:0/2957509529 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195386662 0 0) 0x7fe2d00af3a0 con 0x7fe2d00cf810 2023-12-19T18:22:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.823+0000 7fe2d50f2640 1 -- 172.21.15.27:0/2957509529 >> v1:172.21.15.192:6789/0 conn(0x7fe2d00cbb70 legacy=0x7fe2d0160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.823+0000 7fe2d50f2640 1 -- 172.21.15.27:0/2957509529 >> v1:172.21.15.27:6789/0 conn(0x7fe2d00cf810 legacy=0x7fe2d00aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.823+0000 7fe2d50f2640 1 -- 172.21.15.27:0/2957509529 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2d0071650 con 0x7fe2d006caa0 2023-12-19T18:22:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.825+0000 7fe2d50f2640 1 --2- 172.21.15.27:0/2957509529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d006caa0 0x7fe2d006ced0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80097a0 tx=0x7fe2c802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943f6040c8957966 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.825+0000 7fe2cf7fe640 1 -- 172.21.15.27:0/2957509529 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8030610 con 0x7fe2d006caa0 2023-12-19T18:22:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.825+0000 7fe2cf7fe640 1 -- 172.21.15.27:0/2957509529 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c8030770 con 0x7fe2d006caa0 2023-12-19T18:22:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.825+0000 7fe2cf7fe640 1 -- 172.21.15.27:0/2957509529 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c8030b00 con 0x7fe2d006caa0 2023-12-19T18:22:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.825+0000 7fe2d6b55640 1 -- 172.21.15.27:0/2957509529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d006caa0 msgr2=0x7fe2d006ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.825+0000 7fe2d6b55640 1 --2- 172.21.15.27:0/2957509529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d006caa0 0x7fe2d006ced0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fe2c80097a0 tx=0x7fe2c802b960 comp rx=0 tx=0).stop 2023-12-19T18:22:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.826+0000 7fe2d6b55640 1 -- 172.21.15.27:0/2957509529 shutdown_connections 2023-12-19T18:22:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.826+0000 7fe2d6b55640 1 --2- 172.21.15.27:0/2957509529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d006caa0 0x7fe2d006ced0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.826+0000 7fe2d6b55640 1 -- 172.21.15.27:0/2957509529 >> 172.21.15.27:0/2957509529 conn(0x7fe2d00cc6d0 msgr2=0x7fe2d00ccab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.826+0000 7fe2d6b55640 1 -- 172.21.15.27:0/2957509529 shutdown_connections 2023-12-19T18:22:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.826+0000 7fe2d6b55640 1 -- 172.21.15.27:0/2957509529 wait complete. 2023-12-19T18:22:45.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.827+0000 7fe2d6b55640 1 Processor -- start 2023-12-19T18:22:45.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.828+0000 7fe2d6b55640 1 -- start start 2023-12-19T18:22:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d6b55640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d006caa0 0x7fe2d00b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d6b55640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2d00cbb70 0x7fe2d00b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d6b55640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2d00cf810 0x7fe2d01f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d6b55640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d01f64d0 con 0x7fe2d006caa0 2023-12-19T18:22:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d6b55640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d01f67b0 con 0x7fe2d00cf810 2023-12-19T18:22:45.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d6b55640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2d01f6a90 con 0x7fe2d00cbb70 2023-12-19T18:22:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d48f1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d006caa0 0x7fe2d00b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d48f1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d006caa0 0x7fe2d00b1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35792/0 (socket says 172.21.15.27:35792) 2023-12-19T18:22:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d48f1640 1 -- 172.21.15.27:0/474263465 learned_addr learned my addr 172.21.15.27:0/474263465 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d48f1640 1 -- 172.21.15.27:0/474263465 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2d00cbb70 msgr2=0x7fe2d00b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d48f1640 1 --2- 172.21.15.27:0/474263465 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2d00cbb70 0x7fe2d00b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d48f1640 1 -- 172.21.15.27:0/474263465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2d00cf810 msgr2=0x7fe2d01f5bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d48f1640 1 --2- 172.21.15.27:0/474263465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2d00cf810 0x7fe2d01f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.829+0000 7fe2d48f1640 1 -- 172.21.15.27:0/474263465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c8009400 con 0x7fe2d006caa0 2023-12-19T18:22:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.830+0000 7fe2d48f1640 1 --2- 172.21.15.27:0/474263465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d006caa0 0x7fe2d00b1f00 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000c790 tx=0x7fe2c000cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.830+0000 7fe2cd7fa640 1 -- 172.21.15.27:0/474263465 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c000cea0 con 0x7fe2d006caa0 2023-12-19T18:22:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.830+0000 7fe2cd7fa640 1 -- 172.21.15.27:0/474263465 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c0006d50 con 0x7fe2d006caa0 2023-12-19T18:22:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.830+0000 7fe2cd7fa640 1 -- 172.21.15.27:0/474263465 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c0003bb0 con 0x7fe2d006caa0 2023-12-19T18:22:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.830+0000 7fe2d6b55640 1 -- 172.21.15.27:0/474263465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2d00af660 con 0x7fe2d006caa0 2023-12-19T18:22:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.830+0000 7fe2d6b55640 1 -- 172.21.15.27:0/474263465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2d00b3cf0 con 0x7fe2d006caa0 2023-12-19T18:22:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.831+0000 7fe2d6b55640 1 -- 172.21.15.27:0/474263465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2d00afb80 con 0x7fe2d006caa0 2023-12-19T18:22:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.832+0000 7fe2cd7fa640 1 -- 172.21.15.27:0/474263465 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe2c0003d10 con 0x7fe2d006caa0 2023-12-19T18:22:45.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.833+0000 7fe2cd7fa640 1 --2- 172.21.15.27:0/474263465 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2bc06a430 0x7fe2bc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:45.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.833+0000 7fe2cd7fa640 1 -- 172.21.15.27:0/474263465 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fe2c0088540 con 0x7fe2d006caa0 2023-12-19T18:22:45.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.834+0000 7fe2cffff640 1 --2- 172.21.15.27:0/474263465 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2bc06a430 0x7fe2bc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:45.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.835+0000 7fe2cffff640 1 --2- 172.21.15.27:0/474263465 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2bc06a430 0x7fe2bc06c8f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4000c00 tx=0x7fe2c40079c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:45.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:45.838+0000 7fe2cd7fa640 1 -- 172.21.15.27:0/474263465 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe2c00575f0 con 0x7fe2d006caa0 2023-12-19T18:22:46.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.038+0000 7fe2d6b55640 1 -- 172.21.15.27:0/474263465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2d0067110 con 0x7fe2d006caa0 2023-12-19T18:22:46.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.039+0000 7fe2cd7fa640 1 -- 172.21.15.27:0/474263465 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v140) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2c0057180 con 0x7fe2d006caa0 2023-12-19T18:22:46.040 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:22:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.042+0000 7fe2d6b55640 1 -- 172.21.15.27:0/474263465 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2bc06a430 msgr2=0x7fe2bc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.042+0000 7fe2d6b55640 1 --2- 172.21.15.27:0/474263465 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2bc06a430 0x7fe2bc06c8f0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe2c4000c00 tx=0x7fe2c40079c0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.042+0000 7fe2d6b55640 1 -- 172.21.15.27:0/474263465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d006caa0 msgr2=0x7fe2d00b1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.042+0000 7fe2d6b55640 1 --2- 172.21.15.27:0/474263465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d006caa0 0x7fe2d00b1f00 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7fe2c000c790 tx=0x7fe2c000cc50 comp rx=0 tx=0).stop 2023-12-19T18:22:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.043+0000 7fe2d6b55640 1 -- 172.21.15.27:0/474263465 shutdown_connections 2023-12-19T18:22:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.043+0000 7fe2d6b55640 1 --2- 172.21.15.27:0/474263465 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2bc06a430 0x7fe2bc06c8f0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.043+0000 7fe2d6b55640 1 --2- 172.21.15.27:0/474263465 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2d00cf810 0x7fe2d01f5bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.043+0000 7fe2d6b55640 1 --2- 172.21.15.27:0/474263465 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2d00cbb70 0x7fe2d00b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.043+0000 7fe2d6b55640 1 --2- 172.21.15.27:0/474263465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2d006caa0 0x7fe2d00b1f00 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.043+0000 7fe2d6b55640 1 -- 172.21.15.27:0/474263465 >> 172.21.15.27:0/474263465 conn(0x7fe2d00cc6d0 msgr2=0x7fe2d00c6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:46.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.043+0000 7fe2d6b55640 1 -- 172.21.15.27:0/474263465 shutdown_connections 2023-12-19T18:22:46.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.043+0000 7fe2d6b55640 1 -- 172.21.15.27:0/474263465 wait complete. 2023-12-19T18:22:46.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:22:46.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.305+0000 7f9fe2977640 1 Processor -- start 2023-12-19T18:22:46.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.306+0000 7f9fe2977640 1 -- start start 2023-12-19T18:22:46.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.307+0000 7f9fe2977640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9fdc0cf9b0 0x7f9fdc0cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:46.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.307+0000 7f9fe2977640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fdc0cda20 0x7f9fdc0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:46.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.307+0000 7f9fe2977640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc0d26d0 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.307+0000 7f9fe2977640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fdc0cea80 con 0x7f9fdc0cf9b0 2023-12-19T18:22:46.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.307+0000 7f9fe2977640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0b5f10 con 0x7f9fdc0cda20 2023-12-19T18:22:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.308+0000 7f9fdbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9fdc0b8c30 0x7f9fdc0d1fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47956/0 (socket says 172.21.15.27:47956) 2023-12-19T18:22:46.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.308+0000 7f9fdbfff640 1 -- 172.21.15.27:0/895552573 learned_addr learned my addr 172.21.15.27:0/895552573 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.308+0000 7f9fdb7fe640 1 --2- 172.21.15.27:0/895552573 >> v2:172.21.15.27:3300/0 conn(0x7f9fdc0cf9b0 0x7f9fdc0cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.308+0000 7f9fe0f14640 1 --2- 172.21.15.27:0/895552573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fdc0cda20 0x7f9fdc0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.308+0000 7f9fdaffd640 1 -- 172.21.15.27:0/895552573 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3299404080 0 0) 0x7f9fdc0d26d0 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.308+0000 7f9fdaffd640 1 -- 172.21.15.27:0/895552573 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fc4003520 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.309+0000 7f9fdb7fe640 1 -- 172.21.15.27:0/895552573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fdc0cda20 msgr2=0x7f9fdc0cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.309+0000 7f9fdb7fe640 1 --2- 172.21.15.27:0/895552573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fdc0cda20 0x7f9fdc0cde50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.309+0000 7f9fdb7fe640 1 -- 172.21.15.27:0/895552573 >> v1:172.21.15.27:6789/0 conn(0x7f9fdc0b8c30 legacy=0x7f9fdc0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.309+0000 7f9fdaffd640 1 -- 172.21.15.27:0/895552573 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3110586375 0 0) 0x7f9fc4003520 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.309+0000 7f9fdb7fe640 1 -- 172.21.15.27:0/895552573 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0ce7a0 con 0x7f9fdc0cf9b0 2023-12-19T18:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.310+0000 7f9fdb7fe640 1 --2- 172.21.15.27:0/895552573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc0cf9b0 0x7f9fdc0cfe40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00097d0 tx=0x7f9fd002b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2008a26badca80d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.311+0000 7f9fdaffd640 1 -- 172.21.15.27:0/895552573 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc001670 con 0x7f9fdc0cf9b0 2023-12-19T18:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.311+0000 7f9fdaffd640 1 -- 172.21.15.27:0/895552573 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd002e050 con 0x7f9fdc0cf9b0 2023-12-19T18:22:46.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.311+0000 7f9fdaffd640 1 -- 172.21.15.27:0/895552573 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd0030730 con 0x7f9fdc0cf9b0 2023-12-19T18:22:46.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.311+0000 7f9fe2977640 1 -- 172.21.15.27:0/895552573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc0cf9b0 msgr2=0x7f9fdc0cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.311+0000 7f9fe2977640 1 --2- 172.21.15.27:0/895552573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc0cf9b0 0x7f9fdc0cfe40 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f9fd00097d0 tx=0x7f9fd002b6e0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.312+0000 7f9fe2977640 1 -- 172.21.15.27:0/895552573 shutdown_connections 2023-12-19T18:22:46.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.312+0000 7f9fe2977640 1 --2- 172.21.15.27:0/895552573 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fdc0cda20 0x7f9fdc0cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.312+0000 7f9fe2977640 1 --2- 172.21.15.27:0/895552573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc0cf9b0 0x7f9fdc0cfe40 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.312+0000 7f9fe2977640 1 -- 172.21.15.27:0/895552573 >> 172.21.15.27:0/895552573 conn(0x7f9fdc0d2fa0 msgr2=0x7f9fdc0d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:46.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.312+0000 7f9fe2977640 1 -- 172.21.15.27:0/895552573 shutdown_connections 2023-12-19T18:22:46.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.312+0000 7f9fe2977640 1 -- 172.21.15.27:0/895552573 wait complete. 2023-12-19T18:22:46.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.314+0000 7f9fe2977640 1 Processor -- start 2023-12-19T18:22:46.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.314+0000 7f9fe2977640 1 -- start start 2023-12-19T18:22:46.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.315+0000 7f9fe2977640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc0b8c30 0x7f9fdc15cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:46.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.315+0000 7f9fe2977640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fdc0cda20 0x7f9fdc157df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:46.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.315+0000 7f9fe2977640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fdc0cf9b0 0x7f9fdc156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:46.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.315+0000 7f9fe2977640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0cb0c0 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.315+0000 7f9fe2977640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc158330 con 0x7f9fdc0cda20 2023-12-19T18:22:46.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.315+0000 7f9fe2977640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc15a530 con 0x7f9fdc0cf9b0 2023-12-19T18:22:46.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.315+0000 7f9fdbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc0b8c30 0x7f9fdc15cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:46.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.315+0000 7f9fdbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc0b8c30 0x7f9fdc15cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35820/0 (socket says 172.21.15.27:35820) 2023-12-19T18:22:46.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.315+0000 7f9fdbfff640 1 -- 172.21.15.27:0/3524851377 learned_addr learned my addr 172.21.15.27:0/3524851377 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.316+0000 7f9fdbfff640 1 -- 172.21.15.27:0/3524851377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fdc0cf9b0 msgr2=0x7f9fdc156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.316+0000 7f9fdbfff640 1 --2- 172.21.15.27:0/3524851377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fdc0cf9b0 0x7f9fdc156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.316+0000 7f9fdbfff640 1 -- 172.21.15.27:0/3524851377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fdc0cda20 msgr2=0x7f9fdc157df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.316+0000 7f9fdbfff640 1 --2- 172.21.15.27:0/3524851377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fdc0cda20 0x7f9fdc157df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.316+0000 7f9fdbfff640 1 -- 172.21.15.27:0/3524851377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd0009400 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.316+0000 7f9fdbfff640 1 --2- 172.21.15.27:0/3524851377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc0b8c30 0x7f9fdc15cb40 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00c760 tx=0x7f9fcc00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:46.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.316+0000 7f9fd8ff9640 1 -- 172.21.15.27:0/3524851377 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc005ad0 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.316+0000 7f9fd8ff9640 1 -- 172.21.15.27:0/3524851377 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc005c30 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.316+0000 7f9fe2977640 1 -- 172.21.15.27:0/3524851377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc15d1d0 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.316+0000 7f9fd8ff9640 1 -- 172.21.15.27:0/3524851377 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc007070 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.316+0000 7f9fe2977640 1 -- 172.21.15.27:0/3524851377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc0cb5f0 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.318+0000 7f9fe2977640 1 -- 172.21.15.27:0/3524851377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fdc068e90 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.318+0000 7f9fd8ff9640 1 -- 172.21.15.27:0/3524851377 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9fcc0071d0 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.319+0000 7f9fd8ff9640 1 --2- 172.21.15.27:0/3524851377 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fc806a430 0x7f9fc806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:46.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.319+0000 7f9fd8ff9640 1 -- 172.21.15.27:0/3524851377 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f9fcc0887c0 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.320+0000 7f9fdb7fe640 1 --2- 172.21.15.27:0/3524851377 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fc806a430 0x7f9fc806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:46.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.321+0000 7f9fdb7fe640 1 --2- 172.21.15.27:0/3524851377 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fc806a430 0x7f9fc806c8f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0034000 tx=0x7f9fd002f000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:46.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.324+0000 7f9fd8ff9640 1 -- 172.21.15.27:0/3524851377 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9fcc053cb0 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.517+0000 7f9fe2977640 1 -- 172.21.15.27:0/3524851377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9fdc0cd360 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.519+0000 7f9fd8ff9640 1 -- 172.21.15.27:0/3524851377 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v140) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9fcc0572d0 con 0x7f9fdc0b8c30 2023-12-19T18:22:46.519 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:22:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.521+0000 7f9fe2977640 1 -- 172.21.15.27:0/3524851377 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fc806a430 msgr2=0x7f9fc806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.521+0000 7f9fe2977640 1 --2- 172.21.15.27:0/3524851377 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fc806a430 0x7f9fc806c8f0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9fd0034000 tx=0x7f9fd002f000 comp rx=0 tx=0).stop 2023-12-19T18:22:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.521+0000 7f9fe2977640 1 -- 172.21.15.27:0/3524851377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc0b8c30 msgr2=0x7f9fdc15cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.521+0000 7f9fe2977640 1 --2- 172.21.15.27:0/3524851377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc0b8c30 0x7f9fdc15cb40 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc00c760 tx=0x7f9fcc00cc20 comp rx=0 tx=0).stop 2023-12-19T18:22:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.521+0000 7f9fe2977640 1 -- 172.21.15.27:0/3524851377 shutdown_connections 2023-12-19T18:22:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.521+0000 7f9fe2977640 1 --2- 172.21.15.27:0/3524851377 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9fc806a430 0x7f9fc806c8f0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.521+0000 7f9fe2977640 1 --2- 172.21.15.27:0/3524851377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9fdc0cf9b0 0x7f9fdc156340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.521+0000 7f9fe2977640 1 --2- 172.21.15.27:0/3524851377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9fdc0cda20 0x7f9fdc157df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.521+0000 7f9fe2977640 1 --2- 172.21.15.27:0/3524851377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fdc0b8c30 0x7f9fdc15cb40 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.521+0000 7f9fe2977640 1 -- 172.21.15.27:0/3524851377 >> 172.21.15.27:0/3524851377 conn(0x7f9fdc0d2fa0 msgr2=0x7f9fdc0cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:46.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.522+0000 7f9fe2977640 1 -- 172.21.15.27:0/3524851377 shutdown_connections 2023-12-19T18:22:46.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.522+0000 7f9fe2977640 1 -- 172.21.15.27:0/3524851377 wait complete. 2023-12-19T18:22:46.540 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:22:46.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.761+0000 7f4e40d99640 1 Processor -- start 2023-12-19T18:22:46.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.761+0000 7f4e40d99640 1 -- start start 2023-12-19T18:22:46.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.762+0000 7f4e40d99640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4e3c0b49d0 0x7f4e3c160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:46.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.762+0000 7f4e40d99640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4e3c15cf90 0x7f4e3c0d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:46.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.762+0000 7f4e40d99640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e3c0b5ca0 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.762+0000 7f4e40d99640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e3c161800 con 0x7f4e3c15cf90 2023-12-19T18:22:46.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.762+0000 7f4e40d99640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c161240 con 0x7f4e3c0b49d0 2023-12-19T18:22:46.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.763+0000 7f4e3b7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4e3c0b49d0 0x7f4e3c160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:46.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.763+0000 7f4e3b7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4e3c0b49d0 0x7f4e3c160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:38722/0 (socket says 172.21.15.27:38722) 2023-12-19T18:22:46.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.763+0000 7f4e3b7fe640 1 -- 172.21.15.27:0/2304931034 learned_addr learned my addr 172.21.15.27:0/2304931034 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:46.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.763+0000 7f4e3bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4e3c15cf90 0x7f4e3c0d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:46.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.764+0000 7f4e3b7fe640 1 -- 172.21.15.27:0/2304931034 >> v2:172.21.15.192:3300/0 conn(0x7f4e3c15cf90 msgr2=0x7f4e3c0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:46.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.764+0000 7f4e3b7fe640 1 --2- 172.21.15.27:0/2304931034 >> v2:172.21.15.192:3300/0 conn(0x7f4e3c15cf90 0x7f4e3c0d15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.764+0000 7f4e3b7fe640 1 -- 172.21.15.27:0/2304931034 >> v1:172.21.15.192:6789/0 conn(0x7f4e3c0b89b0 legacy=0x7f4e3c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.764+0000 7f4e3b7fe640 1 -- 172.21.15.27:0/2304931034 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e3c161520 con 0x7f4e3c0b49d0 2023-12-19T18:22:46.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.765+0000 7f4e3b7fe640 1 --2- 172.21.15.27:0/2304931034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4e3c0b49d0 0x7f4e3c160940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009850 tx=0x7f4e2c02b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a757099469127b44 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:46.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.766+0000 7f4e3a7fc640 1 -- 172.21.15.27:0/2304931034 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c004230 con 0x7f4e3c0b49d0 2023-12-19T18:22:46.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.766+0000 7f4e3a7fc640 1 -- 172.21.15.27:0/2304931034 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e2c004390 con 0x7f4e3c0b49d0 2023-12-19T18:22:46.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.766+0000 7f4e3a7fc640 1 -- 172.21.15.27:0/2304931034 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e2c003810 con 0x7f4e3c0b49d0 2023-12-19T18:22:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.767+0000 7f4e40d99640 1 -- 172.21.15.27:0/2304931034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4e3c0b49d0 msgr2=0x7f4e3c160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.767+0000 7f4e40d99640 1 --2- 172.21.15.27:0/2304931034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4e3c0b49d0 0x7f4e3c160940 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c009850 tx=0x7f4e2c02b9f0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.767+0000 7f4e40d99640 1 -- 172.21.15.27:0/2304931034 shutdown_connections 2023-12-19T18:22:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.767+0000 7f4e40d99640 1 --2- 172.21.15.27:0/2304931034 >> v2:172.21.15.192:3300/0 conn(0x7f4e3c15cf90 0x7f4e3c0d15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.767+0000 7f4e40d99640 1 --2- 172.21.15.27:0/2304931034 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4e3c0b49d0 0x7f4e3c160940 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.767+0000 7f4e40d99640 1 -- 172.21.15.27:0/2304931034 >> 172.21.15.27:0/2304931034 conn(0x7f4e3c0cf760 msgr2=0x7f4e3c0cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.767+0000 7f4e40d99640 1 -- 172.21.15.27:0/2304931034 shutdown_connections 2023-12-19T18:22:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.767+0000 7f4e40d99640 1 -- 172.21.15.27:0/2304931034 wait complete. 2023-12-19T18:22:46.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.769+0000 7f4e40d99640 1 Processor -- start 2023-12-19T18:22:46.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.769+0000 7f4e40d99640 1 -- start start 2023-12-19T18:22:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.770+0000 7f4e40d99640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4e3c0b49d0 0x7f4e3c0c8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.770+0000 7f4e40d99640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e3c0b89b0 0x7f4e3c0cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.770+0000 7f4e40d99640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e3c15cf90 0x7f4e3c155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.770+0000 7f4e40d99640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c0c8a10 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.770+0000 7f4e40d99640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c0c7e00 con 0x7f4e3c15cf90 2023-12-19T18:22:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.770+0000 7f4e40d99640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e3c0c80e0 con 0x7f4e3c0b49d0 2023-12-19T18:22:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.770+0000 7f4e3affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e3c0b89b0 0x7f4e3c0cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.770+0000 7f4e3affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e3c0b89b0 0x7f4e3c0cac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35822/0 (socket says 172.21.15.27:35822) 2023-12-19T18:22:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.770+0000 7f4e3affd640 1 -- 172.21.15.27:0/3608597218 learned_addr learned my addr 172.21.15.27:0/3608597218 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.770+0000 7f4e3b7fe640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4e3c0b49d0 0x7f4e3c0c8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.770+0000 7f4e3bfff640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e3c15cf90 0x7f4e3c155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.771+0000 7f4e3affd640 1 -- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4e3c0b49d0 msgr2=0x7f4e3c0c8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.771+0000 7f4e3affd640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4e3c0b49d0 0x7f4e3c0c8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.771+0000 7f4e3affd640 1 -- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e3c15cf90 msgr2=0x7f4e3c155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.771+0000 7f4e3affd640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e3c15cf90 0x7f4e3c155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.771+0000 7f4e3affd640 1 -- 172.21.15.27:0/3608597218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e2c009460 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.771+0000 7f4e3bfff640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e3c15cf90 0x7f4e3c155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:46.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.771+0000 7f4e3b7fe640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4e3c0b49d0 0x7f4e3c0c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:46.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.771+0000 7f4e3affd640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e3c0b89b0 0x7f4e3c0cac70 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000c400 tx=0x7f4e3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:46.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.771+0000 7f4e23fff640 1 -- 172.21.15.27:0/3608597218 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e300052d0 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.771+0000 7f4e23fff640 1 -- 172.21.15.27:0/3608597218 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e30005430 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.771+0000 7f4e40d99640 1 -- 172.21.15.27:0/3608597218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e3c0c83c0 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.771+0000 7f4e23fff640 1 -- 172.21.15.27:0/3608597218 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e30003830 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.771+0000 7f4e40d99640 1 -- 172.21.15.27:0/3608597218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e3c0c93e0 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.773+0000 7f4e40d99640 1 -- 172.21.15.27:0/3608597218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e3c0c8bc0 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.774+0000 7f4e23fff640 1 -- 172.21.15.27:0/3608597218 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4e30003990 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.775+0000 7f4e23fff640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4e2806a430 0x7f4e2806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:46.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.775+0000 7f4e23fff640 1 -- 172.21.15.27:0/3608597218 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f4e30087f80 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.775+0000 7f4e3b7fe640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4e2806a430 0x7f4e2806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:46.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.777+0000 7f4e3b7fe640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4e2806a430 0x7f4e2806c8f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c035000 tx=0x7f4e2c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:46.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.779+0000 7f4e23fff640 1 -- 172.21.15.27:0/3608597218 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4e3c0c8bc0 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.834 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:22:46.844 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:22:46.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.991+0000 7f4e40d99640 1 -- 172.21.15.27:0/3608597218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4e3c066120 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.992+0000 7f4e23fff640 1 -- 172.21.15.27:0/3608597218 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v140) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4e30053500 con 0x7f4e3c0b89b0 2023-12-19T18:22:46.992 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:22:46.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.995+0000 7f4e40d99640 1 -- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4e2806a430 msgr2=0x7f4e2806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.995+0000 7f4e40d99640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4e2806a430 0x7f4e2806c8f0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f4e2c035000 tx=0x7f4e2c008030 comp rx=0 tx=0).stop 2023-12-19T18:22:46.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.995+0000 7f4e40d99640 1 -- 172.21.15.27:0/3608597218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e3c0b89b0 msgr2=0x7f4e3c0cac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:46.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.995+0000 7f4e40d99640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e3c0b89b0 0x7f4e3c0cac70 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000c400 tx=0x7f4e3000c8c0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.995+0000 7f4e40d99640 1 -- 172.21.15.27:0/3608597218 shutdown_connections 2023-12-19T18:22:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.995+0000 7f4e40d99640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4e2806a430 0x7f4e2806c8f0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.995+0000 7f4e40d99640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4e3c15cf90 0x7f4e3c155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.995+0000 7f4e40d99640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e3c0b89b0 0x7f4e3c0cac70 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.995+0000 7f4e40d99640 1 --2- 172.21.15.27:0/3608597218 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4e3c0b49d0 0x7f4e3c0c8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.995+0000 7f4e40d99640 1 -- 172.21.15.27:0/3608597218 >> 172.21.15.27:0/3608597218 conn(0x7f4e3c0cf760 msgr2=0x7f4e3c065a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.995+0000 7f4e40d99640 1 -- 172.21.15.27:0/3608597218 shutdown_connections 2023-12-19T18:22:46.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:46.995+0000 7f4e40d99640 1 -- 172.21.15.27:0/3608597218 wait complete. 2023-12-19T18:22:47.014 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-19T18:22:47.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:47.014 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-19T18:22:47.111 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[89975]: fuse finished with error 0 and tester_r 0 2023-12-19T18:22:47.115 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:22:47.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:47.116 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-19T18:22:47.280 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:22:47.280 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:22:47.345 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:22:47.345 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:22:47.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:47.345 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-19T18:22:47.405 INFO:teuthology.orchestra.run.smithi027.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-19T18:22:47.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:22:47.406 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:22:47.407 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:22:47.467 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:22:47.467 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:22:47.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:22:47.467 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-19T18:22:47.522 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-19T18:22:47.523 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:22:47.523 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:22:47.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.769+0000 7fd390a50640 1 Processor -- start 2023-12-19T18:22:47.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.769+0000 7fd390a50640 1 -- start start 2023-12-19T18:22:47.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.770+0000 7fd390a50640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd38c0b4ef0 0x7fd38c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:47.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.770+0000 7fd390a50640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd38c0d2350 0x7fd38c0b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:47.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.770+0000 7fd390a50640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd38c0b5e90 con 0x7fd38c1604d0 2023-12-19T18:22:47.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.770+0000 7fd390a50640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c0d2b30 con 0x7fd38c0d2350 2023-12-19T18:22:47.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.770+0000 7fd390a50640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd38c0d2850 con 0x7fd38c0b4ef0 2023-12-19T18:22:47.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.770+0000 7fd38ad9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd38c1604d0 0x7fd38c0b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47966/0 (socket says 172.21.15.27:47966) 2023-12-19T18:22:47.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.770+0000 7fd38ad9d640 1 -- 172.21.15.27:0/866246509 learned_addr learned my addr 172.21.15.27:0/866246509 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:47.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.771+0000 7fd38b59e640 1 --2- 172.21.15.27:0/866246509 >> v2:172.21.15.27:3300/0 conn(0x7fd38c0d2350 0x7fd38c0b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:47.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.771+0000 7fd383fff640 1 --2- 172.21.15.27:0/866246509 >> v2:172.21.15.192:3300/0 conn(0x7fd38c0b4ef0 0x7fd38c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:47.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.771+0000 7fd38a59c640 1 -- 172.21.15.27:0/866246509 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477468428 0 0) 0x7fd38c0b5e90 con 0x7fd38c1604d0 2023-12-19T18:22:47.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.771+0000 7fd38a59c640 1 -- 172.21.15.27:0/866246509 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd378003520 con 0x7fd38c1604d0 2023-12-19T18:22:47.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.771+0000 7fd38b59e640 1 -- 172.21.15.27:0/866246509 >> v2:172.21.15.192:3300/0 conn(0x7fd38c0b4ef0 msgr2=0x7fd38c0b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:47.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.771+0000 7fd38b59e640 1 --2- 172.21.15.27:0/866246509 >> v2:172.21.15.192:3300/0 conn(0x7fd38c0b4ef0 0x7fd38c0b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:47.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.771+0000 7fd38b59e640 1 -- 172.21.15.27:0/866246509 >> v1:172.21.15.27:6789/0 conn(0x7fd38c1604d0 legacy=0x7fd38c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:47.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.771+0000 7fd38b59e640 1 -- 172.21.15.27:0/866246509 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd38c0cded0 con 0x7fd38c0d2350 2023-12-19T18:22:47.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.772+0000 7fd38b59e640 1 --2- 172.21.15.27:0/866246509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd38c0d2350 0x7fd38c0b6840 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd38c072120 tx=0x7fd38402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f63c139deb853ae server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:47.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.772+0000 7fd38a59c640 1 -- 172.21.15.27:0/866246509 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd38402cc80 con 0x7fd38c0d2350 2023-12-19T18:22:47.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.773+0000 7fd38a59c640 1 -- 172.21.15.27:0/866246509 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd38402cde0 con 0x7fd38c0d2350 2023-12-19T18:22:47.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.773+0000 7fd38a59c640 1 -- 172.21.15.27:0/866246509 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd384034660 con 0x7fd38c0d2350 2023-12-19T18:22:47.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.773+0000 7fd390a50640 1 -- 172.21.15.27:0/866246509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd38c0d2350 msgr2=0x7fd38c0b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:47.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.773+0000 7fd390a50640 1 --2- 172.21.15.27:0/866246509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd38c0d2350 0x7fd38c0b6840 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7fd38c072120 tx=0x7fd38402bed0 comp rx=0 tx=0).stop 2023-12-19T18:22:47.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.773+0000 7fd390a50640 1 -- 172.21.15.27:0/866246509 shutdown_connections 2023-12-19T18:22:47.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.773+0000 7fd390a50640 1 --2- 172.21.15.27:0/866246509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd38c0d2350 0x7fd38c0b6840 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:47.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.773+0000 7fd390a50640 1 --2- 172.21.15.27:0/866246509 >> v2:172.21.15.192:3300/0 conn(0x7fd38c0b4ef0 0x7fd38c0b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:47.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.773+0000 7fd390a50640 1 -- 172.21.15.27:0/866246509 >> 172.21.15.27:0/866246509 conn(0x7fd38c0cf7e0 msgr2=0x7fd38c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:47.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.773+0000 7fd390a50640 1 -- 172.21.15.27:0/866246509 shutdown_connections 2023-12-19T18:22:47.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.773+0000 7fd390a50640 1 -- 172.21.15.27:0/866246509 wait complete. 2023-12-19T18:22:47.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.775+0000 7fd390a50640 1 Processor -- start 2023-12-19T18:22:47.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.775+0000 7fd390a50640 1 -- start start 2023-12-19T18:22:47.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.776+0000 7fd390a50640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd38c0b4ef0 0x7fd38c0c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:47.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.776+0000 7fd390a50640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd38c0d2350 0x7fd38c1f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:47.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.776+0000 7fd390a50640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd38c1604d0 0x7fd38c0c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:47.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.776+0000 7fd390a50640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0d21a0 con 0x7fd38c0d2350 2023-12-19T18:22:47.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.776+0000 7fd390a50640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0d1620 con 0x7fd38c1604d0 2023-12-19T18:22:47.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.776+0000 7fd390a50640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd38c0d1900 con 0x7fd38c0b4ef0 2023-12-19T18:22:47.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.777+0000 7fd383fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd38c0d2350 0x7fd38c1f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:47.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.777+0000 7fd383fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd38c0d2350 0x7fd38c1f63f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35834/0 (socket says 172.21.15.27:35834) 2023-12-19T18:22:47.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.777+0000 7fd383fff640 1 -- 172.21.15.27:0/1670017360 learned_addr learned my addr 172.21.15.27:0/1670017360 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:47.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.777+0000 7fd383fff640 1 -- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd38c0b4ef0 msgr2=0x7fd38c0c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:47.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.777+0000 7fd38ad9d640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd38c0b4ef0 0x7fd38c0c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:47.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.777+0000 7fd38b59e640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd38c1604d0 0x7fd38c0c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:47.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.777+0000 7fd383fff640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd38c0b4ef0 0x7fd38c0c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:47.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.777+0000 7fd383fff640 1 -- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd38c1604d0 msgr2=0x7fd38c0c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:47.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.777+0000 7fd383fff640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd38c1604d0 0x7fd38c0c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:47.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.777+0000 7fd383fff640 1 -- 172.21.15.27:0/1670017360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd384009a70 con 0x7fd38c0d2350 2023-12-19T18:22:47.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.777+0000 7fd38ad9d640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd38c0b4ef0 0x7fd38c0c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:47.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.777+0000 7fd383fff640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd38c0d2350 0x7fd38c1f63f0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fd37400d800 tx=0x7fd37400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:47.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.778+0000 7fd3837fe640 1 -- 172.21.15.27:0/1670017360 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37400b6c0 con 0x7fd38c0d2350 2023-12-19T18:22:47.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.778+0000 7fd3837fe640 1 -- 172.21.15.27:0/1670017360 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd37400b820 con 0x7fd38c0d2350 2023-12-19T18:22:47.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.778+0000 7fd3837fe640 1 -- 172.21.15.27:0/1670017360 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd37400bbb0 con 0x7fd38c0d2350 2023-12-19T18:22:47.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.778+0000 7fd390a50640 1 -- 172.21.15.27:0/1670017360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd38c0d1be0 con 0x7fd38c0d2350 2023-12-19T18:22:47.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.778+0000 7fd390a50640 1 -- 172.21.15.27:0/1670017360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd38c0c6db0 con 0x7fd38c0d2350 2023-12-19T18:22:47.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.779+0000 7fd390a50640 1 -- 172.21.15.27:0/1670017360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd38c0c6f70 con 0x7fd38c0d2350 2023-12-19T18:22:47.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.780+0000 7fd3837fe640 1 -- 172.21.15.27:0/1670017360 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd37400bd10 con 0x7fd38c0d2350 2023-12-19T18:22:47.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.781+0000 7fd3837fe640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd36c06a430 0x7fd36c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:47.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.781+0000 7fd3837fe640 1 -- 172.21.15.27:0/1670017360 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fd374088ed0 con 0x7fd38c0d2350 2023-12-19T18:22:47.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.782+0000 7fd38ad9d640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd36c06a430 0x7fd36c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:47.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.783+0000 7fd38ad9d640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd36c06a430 0x7fd36c06c8f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0011e0 tx=0x7fd37c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:47.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:47.785+0000 7fd3837fe640 1 -- 172.21.15.27:0/1670017360 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd374054450 con 0x7fd38c0d2350 2023-12-19T18:22:48.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.024+0000 7fd390a50640 1 -- 172.21.15.27:0/1670017360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd38c066f80 con 0x7fd38c0d2350 2023-12-19T18:22:48.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.025+0000 7fd3837fe640 1 -- 172.21.15.27:0/1670017360 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4711 (secure 0 0 0) 0x7fd374057a70 con 0x7fd38c0d2350 2023-12-19T18:22:48.025 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:48.025 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2713867269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2713867269},{"type":"v1","addr":"172.21.15.192:6837","nonce":2713867269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":5534,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/919878747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":919878747},{"type":"v1","addr":"172.21.15.27:6835","nonce":919878747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6160,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/861111763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":861111763},{"type":"v1","addr":"172.21.15.192:6836","nonce":861111763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:22:14.217224+0000","modified":"2023-12-19T18:22:17.743185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7320":{"gid":7320,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3670863819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3670863819},{"type":"v1","addr":"172.21.15.27:6834","nonce":3670863819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T18:22:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 54 2023-12-19T18:22:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.028+0000 7fd390a50640 1 -- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd36c06a430 msgr2=0x7fd36c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.028+0000 7fd390a50640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd36c06a430 0x7fd36c06c8f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fd37c0011e0 tx=0x7fd37c009040 comp rx=0 tx=0).stop 2023-12-19T18:22:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.028+0000 7fd390a50640 1 -- 172.21.15.27:0/1670017360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd38c0d2350 msgr2=0x7fd38c1f63f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.028+0000 7fd390a50640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd38c0d2350 0x7fd38c1f63f0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7fd37400d800 tx=0x7fd37400dcc0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.028+0000 7fd390a50640 1 -- 172.21.15.27:0/1670017360 shutdown_connections 2023-12-19T18:22:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.028+0000 7fd390a50640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd36c06a430 0x7fd36c06c8f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.028+0000 7fd390a50640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd38c1604d0 0x7fd38c0c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.028+0000 7fd390a50640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd38c0d2350 0x7fd38c1f63f0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.028+0000 7fd390a50640 1 --2- 172.21.15.27:0/1670017360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd38c0b4ef0 0x7fd38c0c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.028+0000 7fd390a50640 1 -- 172.21.15.27:0/1670017360 >> 172.21.15.27:0/1670017360 conn(0x7fd38c0cf7e0 msgr2=0x7fd38c0b3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.028+0000 7fd390a50640 1 -- 172.21.15.27:0/1670017360 shutdown_connections 2023-12-19T18:22:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.028+0000 7fd390a50640 1 -- 172.21.15.27:0/1670017360 wait complete. 2023-12-19T18:22:48.049 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:22:48.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.279+0000 7fba12bcb640 1 Processor -- start 2023-12-19T18:22:48.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.280+0000 7fba12bcb640 1 -- start start 2023-12-19T18:22:48.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.281+0000 7fba12bcb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fba0c0ce9b0 0x7fba0c0cee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:48.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.281+0000 7fba12bcb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0c06fda0 con 0x7fba0c0c99d0 2023-12-19T18:22:48.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.281+0000 7fba12bcb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba0c0b1180 con 0x7fba0c070510 2023-12-19T18:22:48.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.281+0000 7fba12bcb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0c0b1460 con 0x7fba0c0ce9b0 2023-12-19T18:22:48.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.281+0000 7fba10967640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fba0c0c99d0 0x7fba0c0c9db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47968/0 (socket says 172.21.15.27:47968) 2023-12-19T18:22:48.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.282+0000 7fba10967640 1 -- 172.21.15.27:0/3355671047 learned_addr learned my addr 172.21.15.27:0/3355671047 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:48.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.282+0000 7fba0bfff640 1 --2- 172.21.15.27:0/3355671047 >> v2:172.21.15.27:3300/0 conn(0x7fba0c0ce9b0 0x7fba0c0cee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:48.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.282+0000 7fba0b7fe640 1 -- 172.21.15.27:0/3355671047 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3017896879 0 0) 0x7fba0c06fda0 con 0x7fba0c0c99d0 2023-12-19T18:22:48.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.282+0000 7fba0b7fe640 1 -- 172.21.15.27:0/3355671047 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9f4003520 con 0x7fba0c0c99d0 2023-12-19T18:22:48.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.282+0000 7fba0bfff640 1 -- 172.21.15.27:0/3355671047 >> v1:172.21.15.192:6789/0 conn(0x7fba0c070510 legacy=0x7fba0c0709a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.282+0000 7fba0bfff640 1 -- 172.21.15.27:0/3355671047 >> v1:172.21.15.27:6789/0 conn(0x7fba0c0c99d0 legacy=0x7fba0c0c9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.282+0000 7fba0bfff640 1 -- 172.21.15.27:0/3355671047 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba0c0b1740 con 0x7fba0c0ce9b0 2023-12-19T18:22:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.283+0000 7fba0bfff640 1 --2- 172.21.15.27:0/3355671047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba0c0ce9b0 0x7fba0c0cee30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fba000097a0 tx=0x7fba0002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7788ca7d8d2a83a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:48.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.284+0000 7fba0b7fe640 1 -- 172.21.15.27:0/3355671047 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00031830 con 0x7fba0c0ce9b0 2023-12-19T18:22:48.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.284+0000 7fba0b7fe640 1 -- 172.21.15.27:0/3355671047 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba00031990 con 0x7fba0c0ce9b0 2023-12-19T18:22:48.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.284+0000 7fba0b7fe640 1 -- 172.21.15.27:0/3355671047 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba00031d20 con 0x7fba0c0ce9b0 2023-12-19T18:22:48.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.284+0000 7fba12bcb640 1 -- 172.21.15.27:0/3355671047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba0c0ce9b0 msgr2=0x7fba0c0cee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.284+0000 7fba12bcb640 1 --2- 172.21.15.27:0/3355671047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba0c0ce9b0 0x7fba0c0cee30 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fba000097a0 tx=0x7fba0002b8d0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.284+0000 7fba12bcb640 1 -- 172.21.15.27:0/3355671047 shutdown_connections 2023-12-19T18:22:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.284+0000 7fba12bcb640 1 --2- 172.21.15.27:0/3355671047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba0c0ce9b0 0x7fba0c0cee30 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.284+0000 7fba12bcb640 1 -- 172.21.15.27:0/3355671047 >> 172.21.15.27:0/3355671047 conn(0x7fba0c0cbe00 msgr2=0x7fba0c0cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.285+0000 7fba12bcb640 1 -- 172.21.15.27:0/3355671047 shutdown_connections 2023-12-19T18:22:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.285+0000 7fba12bcb640 1 -- 172.21.15.27:0/3355671047 wait complete. 2023-12-19T18:22:48.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.286+0000 7fba12bcb640 1 Processor -- start 2023-12-19T18:22:48.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.286+0000 7fba12bcb640 1 -- start start 2023-12-19T18:22:48.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.287+0000 7fba12bcb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba0c070510 0x7fba0c14e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:48.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.287+0000 7fba12bcb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba0c0c99d0 0x7fba0c09aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.287+0000 7fba12bcb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba0c0ce9b0 0x7fba0c09b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.287+0000 7fba12bcb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c09c2e0 con 0x7fba0c0c99d0 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.287+0000 7fba12bcb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c09c700 con 0x7fba0c070510 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.287+0000 7fba12bcb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0c14ed50 con 0x7fba0c0ce9b0 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.288+0000 7fba0bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba0c0c99d0 0x7fba0c09aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.288+0000 7fba0bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba0c0c99d0 0x7fba0c09aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35854/0 (socket says 172.21.15.27:35854) 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.288+0000 7fba0bfff640 1 -- 172.21.15.27:0/772540563 learned_addr learned my addr 172.21.15.27:0/772540563 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.288+0000 7fba10967640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba0c070510 0x7fba0c14e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.288+0000 7fba0bfff640 1 -- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba0c0ce9b0 msgr2=0x7fba0c09b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.288+0000 7fba11168640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba0c0ce9b0 0x7fba0c09b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.288+0000 7fba0bfff640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba0c0ce9b0 0x7fba0c09b030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.288+0000 7fba0bfff640 1 -- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba0c070510 msgr2=0x7fba0c14e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.288+0000 7fba0bfff640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba0c070510 0x7fba0c14e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.288+0000 7fba0bfff640 1 -- 172.21.15.27:0/772540563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba00009400 con 0x7fba0c0c99d0 2023-12-19T18:22:48.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.288+0000 7fba10967640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba0c070510 0x7fba0c14e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:48.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.289+0000 7fba0bfff640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba0c0c99d0 0x7fba0c09aaf0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fba00034040 tx=0x7fba000046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:48.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.289+0000 7fba097fa640 1 -- 172.21.15.27:0/772540563 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0003a8f0 con 0x7fba0c0c99d0 2023-12-19T18:22:48.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.289+0000 7fba097fa640 1 -- 172.21.15.27:0/772540563 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba0003aa70 con 0x7fba0c0c99d0 2023-12-19T18:22:48.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.289+0000 7fba097fa640 1 -- 172.21.15.27:0/772540563 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba0003ae20 con 0x7fba0c0c99d0 2023-12-19T18:22:48.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.289+0000 7fba12bcb640 1 -- 172.21.15.27:0/772540563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0c09b660 con 0x7fba0c0c99d0 2023-12-19T18:22:48.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.289+0000 7fba12bcb640 1 -- 172.21.15.27:0/772540563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba0c09bc20 con 0x7fba0c0c99d0 2023-12-19T18:22:48.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.290+0000 7fba12bcb640 1 -- 172.21.15.27:0/772540563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba0c09c420 con 0x7fba0c0c99d0 2023-12-19T18:22:48.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.291+0000 7fba097fa640 1 -- 172.21.15.27:0/772540563 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fba0004d440 con 0x7fba0c0c99d0 2023-12-19T18:22:48.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.293+0000 7fba097fa640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb9f806a430 0x7fb9f806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:48.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.293+0000 7fba097fa640 1 -- 172.21.15.27:0/772540563 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fba000aaa60 con 0x7fba0c0c99d0 2023-12-19T18:22:48.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.293+0000 7fba10967640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb9f806a430 0x7fb9f806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:48.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.294+0000 7fba10967640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb9f806a430 0x7fb9f806c8f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc001560 tx=0x7fb9fc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.298+0000 7fba097fa640 1 -- 172.21.15.27:0/772540563 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fba0c09c420 con 0x7fba0c0c99d0 2023-12-19T18:22:48.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.497+0000 7fba12bcb640 1 -- 172.21.15.27:0/772540563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba0c09c420 con 0x7fba0c0c99d0 2023-12-19T18:22:48.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.499+0000 7fba097fa640 1 -- 172.21.15.27:0/772540563 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16203 (secure 0 0 0) 0x7fba0c09c420 con 0x7fba0c0c99d0 2023-12-19T18:22:48.500 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:48.500 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":140,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:22:44.753680+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:22:12.328518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T18:22:13.251708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":5,"to":3}]},{"pgid":"14.5","mappings":[{"from":5,"to":0}]},{"pgid":"14.1f","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:22:48.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.502+0000 7fba12bcb640 1 -- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb9f806a430 msgr2=0x7fb9f806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.502+0000 7fba12bcb640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb9f806a430 0x7fb9f806c8f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc001560 tx=0x7fb9fc009040 comp rx=0 tx=0).stop 2023-12-19T18:22:48.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.502+0000 7fba12bcb640 1 -- 172.21.15.27:0/772540563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba0c0c99d0 msgr2=0x7fba0c09aaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.502+0000 7fba12bcb640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba0c0c99d0 0x7fba0c09aaf0 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7fba00034040 tx=0x7fba000046c0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.502+0000 7fba12bcb640 1 -- 172.21.15.27:0/772540563 shutdown_connections 2023-12-19T18:22:48.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.502+0000 7fba12bcb640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb9f806a430 0x7fb9f806c8f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.502+0000 7fba12bcb640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fba0c0ce9b0 0x7fba0c09b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.502+0000 7fba12bcb640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba0c0c99d0 0x7fba0c09aaf0 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.503+0000 7fba12bcb640 1 --2- 172.21.15.27:0/772540563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fba0c070510 0x7fba0c14e810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.503+0000 7fba12bcb640 1 -- 172.21.15.27:0/772540563 >> 172.21.15.27:0/772540563 conn(0x7fba0c0cbe00 msgr2=0x7fba0c0c9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:48.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.503+0000 7fba12bcb640 1 -- 172.21.15.27:0/772540563 shutdown_connections 2023-12-19T18:22:48.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.503+0000 7fba12bcb640 1 -- 172.21.15.27:0/772540563 wait complete. 2023-12-19T18:22:48.526 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:22:12.328518+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '134', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:22:13.251708+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '136', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:22:48.526 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:22:48.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.765+0000 7fce98569640 1 Processor -- start 2023-12-19T18:22:48.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.765+0000 7fce98569640 1 -- start start 2023-12-19T18:22:48.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.767+0000 7fce98569640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fce900c9750 0x7fce9015e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:48.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.767+0000 7fce98569640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fce900ac9d0 0x7fce900ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:48.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.767+0000 7fce98569640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce900ce0b0 con 0x7fce900ccfa0 2023-12-19T18:22:48.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.767+0000 7fce98569640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce900ce920 con 0x7fce900c9750 2023-12-19T18:22:48.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.767+0000 7fce98569640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce9015e9e0 con 0x7fce900ac9d0 2023-12-19T18:22:48.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.767+0000 7fce96305640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fce900c9750 0x7fce9015e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:48.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.767+0000 7fce95b04640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fce900ccfa0 0x7fce900c9b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47974/0 (socket says 172.21.15.27:47974) 2023-12-19T18:22:48.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.767+0000 7fce95b04640 1 -- 172.21.15.27:0/1130816650 learned_addr learned my addr 172.21.15.27:0/1130816650 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:48.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.768+0000 7fce96305640 1 -- 172.21.15.27:0/1130816650 >> v2:172.21.15.192:3300/0 conn(0x7fce900ac9d0 msgr2=0x7fce900ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:48.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.768+0000 7fce96b06640 1 --2- 172.21.15.27:0/1130816650 >> v2:172.21.15.192:3300/0 conn(0x7fce900ac9d0 0x7fce900ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:48.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.768+0000 7fce96305640 1 --2- 172.21.15.27:0/1130816650 >> v2:172.21.15.192:3300/0 conn(0x7fce900ac9d0 0x7fce900ace40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.768+0000 7fce96305640 1 -- 172.21.15.27:0/1130816650 >> v1:172.21.15.27:6789/0 conn(0x7fce900ccfa0 legacy=0x7fce900c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.768+0000 7fce95303640 1 -- 172.21.15.27:0/1130816650 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512284208 0 0) 0x7fce900ce0b0 con 0x7fce900ccfa0 2023-12-19T18:22:48.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.768+0000 7fce96305640 1 -- 172.21.15.27:0/1130816650 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce900cd4a0 con 0x7fce900c9750 2023-12-19T18:22:48.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.769+0000 7fce96305640 1 --2- 172.21.15.27:0/1130816650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce900c9750 0x7fce9015e4a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fce900b1720 tx=0x7fce8c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b078ae69c4256ad9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:48.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.769+0000 7fce95303640 1 -- 172.21.15.27:0/1130816650 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c031830 con 0x7fce900c9750 2023-12-19T18:22:48.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.770+0000 7fce95303640 1 -- 172.21.15.27:0/1130816650 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce8c031990 con 0x7fce900c9750 2023-12-19T18:22:48.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.770+0000 7fce95303640 1 -- 172.21.15.27:0/1130816650 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8c031d20 con 0x7fce900c9750 2023-12-19T18:22:48.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.770+0000 7fce98569640 1 -- 172.21.15.27:0/1130816650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce900c9750 msgr2=0x7fce9015e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.770+0000 7fce98569640 1 --2- 172.21.15.27:0/1130816650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce900c9750 0x7fce9015e4a0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7fce900b1720 tx=0x7fce8c02b8d0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.770+0000 7fce98569640 1 -- 172.21.15.27:0/1130816650 shutdown_connections 2023-12-19T18:22:48.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.770+0000 7fce98569640 1 --2- 172.21.15.27:0/1130816650 >> v2:172.21.15.192:3300/0 conn(0x7fce900ac9d0 0x7fce900ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.770+0000 7fce98569640 1 --2- 172.21.15.27:0/1130816650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce900c9750 0x7fce9015e4a0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.770+0000 7fce98569640 1 -- 172.21.15.27:0/1130816650 >> 172.21.15.27:0/1130816650 conn(0x7fce900cebd0 msgr2=0x7fce900cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:48.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.770+0000 7fce98569640 1 -- 172.21.15.27:0/1130816650 shutdown_connections 2023-12-19T18:22:48.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.770+0000 7fce98569640 1 -- 172.21.15.27:0/1130816650 wait complete. 2023-12-19T18:22:48.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.772+0000 7fce98569640 1 Processor -- start 2023-12-19T18:22:48.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.772+0000 7fce98569640 1 -- start start 2023-12-19T18:22:48.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.773+0000 7fce98569640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce900ac9d0 0x7fce90159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:48.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.773+0000 7fce98569640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce900c9750 0x7fce900b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:48.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.773+0000 7fce98569640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce900ccfa0 0x7fce900b4430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.773+0000 7fce98569640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce900c4580 con 0x7fce900ccfa0 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.773+0000 7fce98569640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce900c4860 con 0x7fce900c9750 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.773+0000 7fce98569640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce900c4b40 con 0x7fce900ac9d0 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.774+0000 7fce96b06640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce900ccfa0 0x7fce900b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.774+0000 7fce96b06640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce900ccfa0 0x7fce900b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35878/0 (socket says 172.21.15.27:35878) 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.774+0000 7fce96b06640 1 -- 172.21.15.27:0/3507111789 learned_addr learned my addr 172.21.15.27:0/3507111789 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.774+0000 7fce96305640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce900ac9d0 0x7fce90159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.774+0000 7fce95b04640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce900c9750 0x7fce900b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.774+0000 7fce96b06640 1 -- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce900ac9d0 msgr2=0x7fce90159bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.774+0000 7fce96b06640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce900ac9d0 0x7fce90159bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.774+0000 7fce96b06640 1 -- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce900c9750 msgr2=0x7fce900b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.774+0000 7fce96b06640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce900c9750 0x7fce900b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.774+0000 7fce96b06640 1 -- 172.21.15.27:0/3507111789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce8c009400 con 0x7fce900ccfa0 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.774+0000 7fce95b04640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce900c9750 0x7fce900b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:48.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.775+0000 7fce96b06640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce900ccfa0 0x7fce900b4430 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fce8800cd70 tx=0x7fce8800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:48.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.775+0000 7fce86ffd640 1 -- 172.21.15.27:0/3507111789 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88003b90 con 0x7fce900ccfa0 2023-12-19T18:22:48.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.775+0000 7fce86ffd640 1 -- 172.21.15.27:0/3507111789 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce88003cf0 con 0x7fce900ccfa0 2023-12-19T18:22:48.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.775+0000 7fce86ffd640 1 -- 172.21.15.27:0/3507111789 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce88004240 con 0x7fce900ccfa0 2023-12-19T18:22:48.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.775+0000 7fce98569640 1 -- 172.21.15.27:0/3507111789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce900c4e20 con 0x7fce900ccfa0 2023-12-19T18:22:48.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.775+0000 7fce98569640 1 -- 172.21.15.27:0/3507111789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9015a0f0 con 0x7fce900ccfa0 2023-12-19T18:22:48.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.776+0000 7fce98569640 1 -- 172.21.15.27:0/3507111789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce900c5340 con 0x7fce900ccfa0 2023-12-19T18:22:48.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.777+0000 7fce86ffd640 1 -- 172.21.15.27:0/3507111789 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fce880043a0 con 0x7fce900ccfa0 2023-12-19T18:22:48.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.778+0000 7fce86ffd640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce7406a460 0x7fce7406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:48.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.778+0000 7fce86ffd640 1 -- 172.21.15.27:0/3507111789 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fce88012070 con 0x7fce900ccfa0 2023-12-19T18:22:48.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.779+0000 7fce96305640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce7406a460 0x7fce7406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:48.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.780+0000 7fce96305640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce7406a460 0x7fce7406c920 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fce8c036000 tx=0x7fce8c008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:48.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.783+0000 7fce86ffd640 1 -- 172.21.15.27:0/3507111789 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fce900c5340 con 0x7fce900ccfa0 2023-12-19T18:22:48.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.979+0000 7fce98569640 1 -- 172.21.15.27:0/3507111789 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fce900c5340 con 0x7fce900ccfa0 2023-12-19T18:22:48.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.980+0000 7fce86ffd640 1 -- 172.21.15.27:0/3507111789 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v140) v1 ==== 93+0+10 (secure 0 0 0) 0x7fce900c5340 con 0x7fce900ccfa0 2023-12-19T18:22:48.980 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:22:48.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.982+0000 7fce98569640 1 -- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce7406a460 msgr2=0x7fce7406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.982+0000 7fce98569640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce7406a460 0x7fce7406c920 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fce8c036000 tx=0x7fce8c008030 comp rx=0 tx=0).stop 2023-12-19T18:22:48.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.982+0000 7fce98569640 1 -- 172.21.15.27:0/3507111789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce900ccfa0 msgr2=0x7fce900b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:48.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.982+0000 7fce98569640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce900ccfa0 0x7fce900b4430 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fce8800cd70 tx=0x7fce8800a660 comp rx=0 tx=0).stop 2023-12-19T18:22:48.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.983+0000 7fce98569640 1 -- 172.21.15.27:0/3507111789 shutdown_connections 2023-12-19T18:22:48.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.983+0000 7fce98569640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce7406a460 0x7fce7406c920 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.983+0000 7fce98569640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce900ccfa0 0x7fce900b4430 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.983+0000 7fce98569640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce900c9750 0x7fce900b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.983+0000 7fce98569640 1 --2- 172.21.15.27:0/3507111789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce900ac9d0 0x7fce90159bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:48.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.983+0000 7fce98569640 1 -- 172.21.15.27:0/3507111789 >> 172.21.15.27:0/3507111789 conn(0x7fce900cebd0 msgr2=0x7fce90070f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:48.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.983+0000 7fce98569640 1 -- 172.21.15.27:0/3507111789 shutdown_connections 2023-12-19T18:22:48.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:48.983+0000 7fce98569640 1 -- 172.21.15.27:0/3507111789 wait complete. 2023-12-19T18:22:49.001 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:22:49.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.236+0000 7f82ab59e640 1 Processor -- start 2023-12-19T18:22:49.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.236+0000 7f82ab59e640 1 -- start start 2023-12-19T18:22:49.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.237+0000 7f82ab59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f82ac143450 0x7f82ac0ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:49.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.238+0000 7f82ab59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82ac0cf430 0x7f82ac06c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:49.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.238+0000 7f82ab59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f82ac06c680 0x7f82ac06b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:49.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.238+0000 7f82ab59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ac0b33e0 con 0x7f82ac06c680 2023-12-19T18:22:49.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.238+0000 7f82ab59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82ac0b27d0 con 0x7f82ac143450 2023-12-19T18:22:49.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.238+0000 7f82ab59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0cf850 con 0x7f82ac0cf430 2023-12-19T18:22:49.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.238+0000 7f82aad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f82ac06c680 0x7f82ac06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:49.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.238+0000 7f82aad9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f82ac06c680 0x7f82ac06b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35890/0 (socket says 172.21.15.27:35890) 2023-12-19T18:22:49.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.238+0000 7f82aad9d640 1 -- 172.21.15.27:0/2250196449 learned_addr learned my addr 172.21.15.27:0/2250196449 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:49.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.238+0000 7f82a3fff640 1 --2- 172.21.15.27:0/2250196449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82ac0cf430 0x7f82ac06c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:49.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.238+0000 7f82aa59c640 1 --2- 172.21.15.27:0/2250196449 >> v2:172.21.15.192:3300/0 conn(0x7f82ac143450 0x7f82ac0ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:49.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.239+0000 7f82aad9d640 1 -- 172.21.15.27:0/2250196449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82ac0cf430 msgr2=0x7f82ac06c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:49.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.239+0000 7f82aad9d640 1 --2- 172.21.15.27:0/2250196449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82ac0cf430 0x7f82ac06c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.239+0000 7f82aad9d640 1 -- 172.21.15.27:0/2250196449 >> v2:172.21.15.192:3300/0 conn(0x7f82ac143450 msgr2=0x7f82ac0ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:49.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.239+0000 7f82aad9d640 1 --2- 172.21.15.27:0/2250196449 >> v2:172.21.15.192:3300/0 conn(0x7f82ac143450 0x7f82ac0ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.239+0000 7f82aad9d640 1 -- 172.21.15.27:0/2250196449 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82ac0b2a40 con 0x7f82ac06c680 2023-12-19T18:22:49.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.240+0000 7f82aad9d640 1 --2- 172.21.15.27:0/2250196449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82ac06c680 0x7f82ac06b000 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f82ac071580 tx=0x7f82a402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbd34fc80cebc960 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:49.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.241+0000 7f82a9d9b640 1 -- 172.21.15.27:0/2250196449 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a402c9e0 con 0x7f82ac06c680 2023-12-19T18:22:49.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.241+0000 7f82a9d9b640 1 -- 172.21.15.27:0/2250196449 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82a402cb40 con 0x7f82ac06c680 2023-12-19T18:22:49.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.241+0000 7f82a9d9b640 1 -- 172.21.15.27:0/2250196449 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82a40343f0 con 0x7f82ac06c680 2023-12-19T18:22:49.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.241+0000 7f82ab59e640 1 -- 172.21.15.27:0/2250196449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82ac06c680 msgr2=0x7f82ac06b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:49.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.241+0000 7f82ab59e640 1 --2- 172.21.15.27:0/2250196449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82ac06c680 0x7f82ac06b000 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f82ac071580 tx=0x7f82a402be40 comp rx=0 tx=0).stop 2023-12-19T18:22:49.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.241+0000 7f82ab59e640 1 -- 172.21.15.27:0/2250196449 shutdown_connections 2023-12-19T18:22:49.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.241+0000 7f82ab59e640 1 --2- 172.21.15.27:0/2250196449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82ac06c680 0x7f82ac06b000 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.241+0000 7f82ab59e640 1 --2- 172.21.15.27:0/2250196449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82ac0cf430 0x7f82ac06c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.241+0000 7f82ab59e640 1 --2- 172.21.15.27:0/2250196449 >> v2:172.21.15.192:3300/0 conn(0x7f82ac143450 0x7f82ac0ceef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.241+0000 7f82ab59e640 1 -- 172.21.15.27:0/2250196449 >> 172.21.15.27:0/2250196449 conn(0x7f82ac06a060 msgr2=0x7f82ac06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:49.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.241+0000 7f82ab59e640 1 -- 172.21.15.27:0/2250196449 shutdown_connections 2023-12-19T18:22:49.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.242+0000 7f82ab59e640 1 -- 172.21.15.27:0/2250196449 wait complete. 2023-12-19T18:22:49.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.243+0000 7f82ab59e640 1 Processor -- start 2023-12-19T18:22:49.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.243+0000 7f82ab59e640 1 -- start start 2023-12-19T18:22:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.244+0000 7f82ab59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82ac06c680 0x7f82ac140370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.244+0000 7f82ab59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82ac0cf430 0x7f82ac0acb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.244+0000 7f82ab59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ac143450 0x7f82ac15b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.244+0000 7f82ab59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0cd940 con 0x7f82ac0cf430 2023-12-19T18:22:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.244+0000 7f82ab59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0cccc0 con 0x7f82ac143450 2023-12-19T18:22:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.244+0000 7f82ab59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82ac0ccfa0 con 0x7f82ac06c680 2023-12-19T18:22:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.245+0000 7f82a3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82ac0cf430 0x7f82ac0acb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.245+0000 7f82a3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82ac0cf430 0x7f82ac0acb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35892/0 (socket says 172.21.15.27:35892) 2023-12-19T18:22:49.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.245+0000 7f82a3fff640 1 -- 172.21.15.27:0/1668368391 learned_addr learned my addr 172.21.15.27:0/1668368391 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:49.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.245+0000 7f82aa59c640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82ac06c680 0x7f82ac140370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:49.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.245+0000 7f82a3fff640 1 -- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82ac06c680 msgr2=0x7f82ac140370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:49.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.245+0000 7f82aad9d640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ac143450 0x7f82ac15b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:49.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.245+0000 7f82a3fff640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82ac06c680 0x7f82ac140370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.245+0000 7f82a3fff640 1 -- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ac143450 msgr2=0x7f82ac15b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:49.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.245+0000 7f82a3fff640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ac143450 0x7f82ac15b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.245+0000 7f82a3fff640 1 -- 172.21.15.27:0/1668368391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82a4009b30 con 0x7f82ac0cf430 2023-12-19T18:22:49.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.245+0000 7f82a3fff640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82ac0cf430 0x7f82ac0acb30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f829400e870 tx=0x7f829400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:49.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.246+0000 7f82a2ffd640 1 -- 172.21.15.27:0/1668368391 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829400ee60 con 0x7f82ac0cf430 2023-12-19T18:22:49.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.246+0000 7f82a2ffd640 1 -- 172.21.15.27:0/1668368391 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f829400c940 con 0x7f82ac0cf430 2023-12-19T18:22:49.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.246+0000 7f82a2ffd640 1 -- 172.21.15.27:0/1668368391 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f829400ccf0 con 0x7f82ac0cf430 2023-12-19T18:22:49.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.246+0000 7f82ab59e640 1 -- 172.21.15.27:0/1668368391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82ac0cd280 con 0x7f82ac0cf430 2023-12-19T18:22:49.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.246+0000 7f82ab59e640 1 -- 172.21.15.27:0/1668368391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82ac1408b0 con 0x7f82ac0cf430 2023-12-19T18:22:49.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.247+0000 7f82ab59e640 1 -- 172.21.15.27:0/1668368391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82ac04e840 con 0x7f82ac0cf430 2023-12-19T18:22:49.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.248+0000 7f82a2ffd640 1 -- 172.21.15.27:0/1668368391 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8294013470 con 0x7f82ac0cf430 2023-12-19T18:22:49.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.249+0000 7f82a2ffd640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f828c06a430 0x7f828c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:49.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.249+0000 7f82a2ffd640 1 -- 172.21.15.27:0/1668368391 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f82940889c0 con 0x7f82ac0cf430 2023-12-19T18:22:49.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.250+0000 7f82aa59c640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f828c06a430 0x7f828c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:49.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.251+0000 7f82aa59c640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f828c06a430 0x7f828c06c8f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f829c009a10 tx=0x7f829c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:49.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.253+0000 7f82a2ffd640 1 -- 172.21.15.27:0/1668368391 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8294053fe0 con 0x7f82ac0cf430 2023-12-19T18:22:49.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.446+0000 7f82ab59e640 1 -- 172.21.15.27:0/1668368391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f82ac0b3c10 con 0x7f82ac0cf430 2023-12-19T18:22:49.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.447+0000 7f82a2ffd640 1 -- 172.21.15.27:0/1668368391 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v140) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8294021020 con 0x7f82ac0cf430 2023-12-19T18:22:49.447 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:22:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.449+0000 7f82ab59e640 1 -- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f828c06a430 msgr2=0x7f828c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.449+0000 7f82ab59e640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f828c06a430 0x7f828c06c8f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f829c009a10 tx=0x7f829c008040 comp rx=0 tx=0).stop 2023-12-19T18:22:49.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.449+0000 7f82ab59e640 1 -- 172.21.15.27:0/1668368391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82ac0cf430 msgr2=0x7f82ac0acb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.449+0000 7f82ab59e640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82ac0cf430 0x7f82ac0acb30 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f829400e870 tx=0x7f829400ed30 comp rx=0 tx=0).stop 2023-12-19T18:22:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.449+0000 7f82ab59e640 1 -- 172.21.15.27:0/1668368391 shutdown_connections 2023-12-19T18:22:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.449+0000 7f82ab59e640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f828c06a430 0x7f828c06c8f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.449+0000 7f82ab59e640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82ac143450 0x7f82ac15b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.449+0000 7f82ab59e640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82ac0cf430 0x7f82ac0acb30 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.449+0000 7f82ab59e640 1 --2- 172.21.15.27:0/1668368391 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f82ac06c680 0x7f82ac140370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.449+0000 7f82ab59e640 1 -- 172.21.15.27:0/1668368391 >> 172.21.15.27:0/1668368391 conn(0x7f82ac06a060 msgr2=0x7f82ac06bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.450+0000 7f82ab59e640 1 -- 172.21.15.27:0/1668368391 shutdown_connections 2023-12-19T18:22:49.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.450+0000 7f82ab59e640 1 -- 172.21.15.27:0/1668368391 wait complete. 2023-12-19T18:22:49.468 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:22:49.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.700+0000 7ff42e8c4640 1 Processor -- start 2023-12-19T18:22:49.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.700+0000 7ff42e8c4640 1 -- start start 2023-12-19T18:22:49.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.702+0000 7ff42e8c4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff42815b1a0 0x7ff42806f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.702+0000 7ff42e8c4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff428071420 0x7ff42815ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.702+0000 7ff42e8c4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff42806bc80 con 0x7ff4280c2720 2023-12-19T18:22:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.702+0000 7ff42e8c4640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff42806d690 con 0x7ff42815b1a0 2023-12-19T18:22:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.702+0000 7ff42e8c4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff42806d0a0 con 0x7ff428071420 2023-12-19T18:22:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.702+0000 7ff427fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff42815b1a0 0x7ff42806f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.702+0000 7ff427fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff42815b1a0 0x7ff42806f520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:55008/0 (socket says 172.21.15.27:55008) 2023-12-19T18:22:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.702+0000 7ff427fff640 1 -- 172.21.15.27:0/3607349387 learned_addr learned my addr 172.21.15.27:0/3607349387 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.702+0000 7ff41f7fe640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7ff4280c2720 0x7ff42806fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:60970/0 (socket says 172.21.15.27:60970) 2023-12-19T18:22:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.702+0000 7ff42ce61640 1 --2- 172.21.15.27:0/3607349387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff428071420 0x7ff42815ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.703+0000 7ff427fff640 1 -- 172.21.15.27:0/3607349387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff428071420 msgr2=0x7ff42815ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.703+0000 7ff427fff640 1 --2- 172.21.15.27:0/3607349387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff428071420 0x7ff42815ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.703+0000 7ff427fff640 1 -- 172.21.15.27:0/3607349387 >> v1:172.21.15.192:6789/0 conn(0x7ff4280c2720 legacy=0x7ff42806fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.703+0000 7ff427fff640 1 -- 172.21.15.27:0/3607349387 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff42806b6c0 con 0x7ff42815b1a0 2023-12-19T18:22:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.704+0000 7ff427fff640 1 --2- 172.21.15.27:0/3607349387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff42815b1a0 0x7ff42806f520 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff418009870 tx=0x7ff41802b8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d0cc86cb042786e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.705+0000 7ff4277fe640 1 -- 172.21.15.27:0/3607349387 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418031640 con 0x7ff42815b1a0 2023-12-19T18:22:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.705+0000 7ff4277fe640 1 -- 172.21.15.27:0/3607349387 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4180317a0 con 0x7ff42815b1a0 2023-12-19T18:22:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.705+0000 7ff4277fe640 1 -- 172.21.15.27:0/3607349387 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff418031b30 con 0x7ff42815b1a0 2023-12-19T18:22:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.706+0000 7ff42e8c4640 1 -- 172.21.15.27:0/3607349387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff42815b1a0 msgr2=0x7ff42806f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.706+0000 7ff42e8c4640 1 --2- 172.21.15.27:0/3607349387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff42815b1a0 0x7ff42806f520 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7ff418009870 tx=0x7ff41802b8f0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.706+0000 7ff42e8c4640 1 -- 172.21.15.27:0/3607349387 shutdown_connections 2023-12-19T18:22:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.706+0000 7ff42e8c4640 1 --2- 172.21.15.27:0/3607349387 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff428071420 0x7ff42815ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.706+0000 7ff42e8c4640 1 --2- 172.21.15.27:0/3607349387 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff42815b1a0 0x7ff42806f520 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.706+0000 7ff42e8c4640 1 -- 172.21.15.27:0/3607349387 >> 172.21.15.27:0/3607349387 conn(0x7ff4280cebd0 msgr2=0x7ff4280cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:49.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.706+0000 7ff42e8c4640 1 -- 172.21.15.27:0/3607349387 shutdown_connections 2023-12-19T18:22:49.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.706+0000 7ff42e8c4640 1 -- 172.21.15.27:0/3607349387 wait complete. 2023-12-19T18:22:49.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.708+0000 7ff42e8c4640 1 Processor -- start 2023-12-19T18:22:49.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.708+0000 7ff42e8c4640 1 -- start start 2023-12-19T18:22:49.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.709+0000 7ff42e8c4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff428071420 0x7ff4281466a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.709+0000 7ff42e8c4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4280c2720 0x7ff4281f66e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.709+0000 7ff42e8c4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff42815b1a0 0x7ff4280acbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.709+0000 7ff42e8c4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff428147390 con 0x7ff428071420 2023-12-19T18:22:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.709+0000 7ff42e8c4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff428146be0 con 0x7ff42815b1a0 2023-12-19T18:22:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.709+0000 7ff42e8c4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4281f6fe0 con 0x7ff4280c2720 2023-12-19T18:22:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.710+0000 7ff427fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff428071420 0x7ff4281466a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.710+0000 7ff427fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff428071420 0x7ff4281466a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35900/0 (socket says 172.21.15.27:35900) 2023-12-19T18:22:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.710+0000 7ff427fff640 1 -- 172.21.15.27:0/1218427762 learned_addr learned my addr 172.21.15.27:0/1218427762 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:49.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.710+0000 7ff41f7fe640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4280c2720 0x7ff4281f66e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:49.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.710+0000 7ff427fff640 1 -- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4280c2720 msgr2=0x7ff4281f66e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.710+0000 7ff42ce61640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff42815b1a0 0x7ff4280acbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.710+0000 7ff427fff640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4280c2720 0x7ff4281f66e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.710+0000 7ff427fff640 1 -- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff42815b1a0 msgr2=0x7ff4280acbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.710+0000 7ff427fff640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff42815b1a0 0x7ff4280acbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.710+0000 7ff427fff640 1 -- 172.21.15.27:0/1218427762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4180094d0 con 0x7ff428071420 2023-12-19T18:22:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.710+0000 7ff41f7fe640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4280c2720 0x7ff4281f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.710+0000 7ff42ce61640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff42815b1a0 0x7ff4280acbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.711+0000 7ff427fff640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff428071420 0x7ff4281466a0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff418000c00 tx=0x7ff4180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.711+0000 7ff4257fa640 1 -- 172.21.15.27:0/1218427762 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41803a930 con 0x7ff428071420 2023-12-19T18:22:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.711+0000 7ff4257fa640 1 -- 172.21.15.27:0/1218427762 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff41803aa90 con 0x7ff428071420 2023-12-19T18:22:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.711+0000 7ff42e8c4640 1 -- 172.21.15.27:0/1218427762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4281f72c0 con 0x7ff428071420 2023-12-19T18:22:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.711+0000 7ff4257fa640 1 -- 172.21.15.27:0/1218427762 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff41803ae40 con 0x7ff428071420 2023-12-19T18:22:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.711+0000 7ff42e8c4640 1 -- 172.21.15.27:0/1218427762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff428146fb0 con 0x7ff428071420 2023-12-19T18:22:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.712+0000 7ff42e8c4640 1 -- 172.21.15.27:0/1218427762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4281474d0 con 0x7ff428071420 2023-12-19T18:22:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.713+0000 7ff4257fa640 1 -- 172.21.15.27:0/1218427762 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff41802f050 con 0x7ff428071420 2023-12-19T18:22:49.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.714+0000 7ff4257fa640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff40c06a430 0x7ff40c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:49.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.714+0000 7ff4257fa640 1 -- 172.21.15.27:0/1218427762 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7ff4180aa670 con 0x7ff428071420 2023-12-19T18:22:49.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.715+0000 7ff41f7fe640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff40c06a430 0x7ff40c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:49.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.716+0000 7ff41f7fe640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff40c06a430 0x7ff40c06c8f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff4280ad870 tx=0x7ff410009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.719+0000 7ff4257fa640 1 -- 172.21.15.27:0/1218427762 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff4281474d0 con 0x7ff428071420 2023-12-19T18:22:49.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.916+0000 7ff42e8c4640 1 -- 172.21.15.27:0/1218427762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff4281474d0 con 0x7ff428071420 2023-12-19T18:22:49.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.917+0000 7ff4257fa640 1 -- 172.21.15.27:0/1218427762 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v140) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff4281474d0 con 0x7ff428071420 2023-12-19T18:22:49.917 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:22:49.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.919+0000 7ff42e8c4640 1 -- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff40c06a430 msgr2=0x7ff40c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:49.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.919+0000 7ff42e8c4640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff40c06a430 0x7ff40c06c8f0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff4280ad870 tx=0x7ff410009040 comp rx=0 tx=0).stop 2023-12-19T18:22:49.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.919+0000 7ff42e8c4640 1 -- 172.21.15.27:0/1218427762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff428071420 msgr2=0x7ff4281466a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:49.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.919+0000 7ff42e8c4640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff428071420 0x7ff4281466a0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7ff418000c00 tx=0x7ff4180046c0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.920+0000 7ff42e8c4640 1 -- 172.21.15.27:0/1218427762 shutdown_connections 2023-12-19T18:22:49.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.920+0000 7ff42e8c4640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff40c06a430 0x7ff40c06c8f0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.920+0000 7ff42e8c4640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff42815b1a0 0x7ff4280acbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.920+0000 7ff42e8c4640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff4280c2720 0x7ff4281f66e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.920+0000 7ff42e8c4640 1 --2- 172.21.15.27:0/1218427762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff428071420 0x7ff4281466a0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:49.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.920+0000 7ff42e8c4640 1 -- 172.21.15.27:0/1218427762 >> 172.21.15.27:0/1218427762 conn(0x7ff4280cebd0 msgr2=0x7ff4280c4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:49.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.920+0000 7ff42e8c4640 1 -- 172.21.15.27:0/1218427762 shutdown_connections 2023-12-19T18:22:49.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:49.920+0000 7ff42e8c4640 1 -- 172.21.15.27:0/1218427762 wait complete. 2023-12-19T18:22:49.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:22:50.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.164+0000 7fe1f4ffd640 1 Processor -- start 2023-12-19T18:22:50.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.164+0000 7fe1f4ffd640 1 -- start start 2023-12-19T18:22:50.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.165+0000 7fe1f4ffd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe1f01604d0 0x7fe1f00b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.165+0000 7fe1f4ffd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f00d2350 0x7fe1f00b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.166+0000 7fe1f4ffd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f00b5e90 con 0x7fe1f00d0300 2023-12-19T18:22:50.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.166+0000 7fe1f4ffd640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f00d2b30 con 0x7fe1f01604d0 2023-12-19T18:22:50.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.166+0000 7fe1f4ffd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00d2850 con 0x7fe1f00d2350 2023-12-19T18:22:50.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.166+0000 7fe1effff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe1f00d0300 0x7fe1f00d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47978/0 (socket says 172.21.15.27:47978) 2023-12-19T18:22:50.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.166+0000 7fe1effff640 1 -- 172.21.15.27:0/3049993605 learned_addr learned my addr 172.21.15.27:0/3049993605 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:50.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.166+0000 7fe1ef7fe640 1 --2- 172.21.15.27:0/3049993605 >> v2:172.21.15.192:3300/0 conn(0x7fe1f01604d0 0x7fe1f00b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:50.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.166+0000 7fe1eeffd640 1 --2- 172.21.15.27:0/3049993605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f00d2350 0x7fe1f00b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:50.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245525766 0 0) 0x7fe1f00b5e90 con 0x7fe1f00d0300 2023-12-19T18:22:50.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1dc003520 con 0x7fe1f00d0300 2023-12-19T18:22:50.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2551426423 0 0) 0x7fe1dc003520 con 0x7fe1f00d0300 2023-12-19T18:22:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f00d2350 msgr2=0x7fe1f00b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 --2- 172.21.15.27:0/3049993605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f00d2350 0x7fe1f00b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 >> v2:172.21.15.192:3300/0 conn(0x7fe1f01604d0 msgr2=0x7fe1f00b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 --2- 172.21.15.27:0/3049993605 >> v2:172.21.15.192:3300/0 conn(0x7fe1f01604d0 0x7fe1f00b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f00cded0 con 0x7fe1f00d0300 2023-12-19T18:22:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe1e8001670 con 0x7fe1f00d0300 2023-12-19T18:22:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 >> v1:172.21.15.27:6789/0 conn(0x7fe1f00d0300 legacy=0x7fe1f00d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 --2- 172.21.15.27:0/3049993605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f00d2350 0x7fe1dc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 --2- 172.21.15.27:0/3049993605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1dc003f40 0x7fe1dc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 --2- 172.21.15.27:0/3049993605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1dc004d00 0x7fe1dc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00d2b30 con 0x7fe1dc004d00 2023-12-19T18:22:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00d2850 con 0x7fe1dc003f40 2023-12-19T18:22:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc00b980 con 0x7fe1f00d2350 2023-12-19T18:22:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.167+0000 7fe1ef7fe640 1 --2- 172.21.15.27:0/3049993605 >> v2:172.21.15.192:3300/0 conn(0x7fe1f01604d0 0x7fe1f00b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:22:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.168+0000 7fe1effff640 1 --2- 172.21.15.27:0/3049993605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1dc004d00 0x7fe1dc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.168+0000 7fe1eeffd640 1 --2- 172.21.15.27:0/3049993605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1dc003f40 0x7fe1dc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.168+0000 7fe1effff640 1 -- 172.21.15.27:0/3049993605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f00d2350 msgr2=0x7fe1dc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.168+0000 7fe1effff640 1 --2- 172.21.15.27:0/3049993605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1f00d2350 0x7fe1dc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.168+0000 7fe1effff640 1 -- 172.21.15.27:0/3049993605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1dc003f40 msgr2=0x7fe1dc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.168+0000 7fe1effff640 1 --2- 172.21.15.27:0/3049993605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1dc003f40 0x7fe1dc0047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.168+0000 7fe1effff640 1 -- 172.21.15.27:0/3049993605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc003520 con 0x7fe1dc004d00 2023-12-19T18:22:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.169+0000 7fe1effff640 1 --2- 172.21.15.27:0/3049993605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1dc004d00 0x7fe1dc005180 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800c140 tx=0x7fe1dc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4eb118695eeb1bd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.169+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e8005ad0 con 0x7fe1dc004d00 2023-12-19T18:22:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.169+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e8005c50 con 0x7fe1dc004d00 2023-12-19T18:22:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.169+0000 7fe1ee7fc640 1 -- 172.21.15.27:0/3049993605 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e802c810 con 0x7fe1dc004d00 2023-12-19T18:22:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.170+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/3049993605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1dc004d00 msgr2=0x7fe1dc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.170+0000 7fe1f4ffd640 1 --2- 172.21.15.27:0/3049993605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1dc004d00 0x7fe1dc005180 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800c140 tx=0x7fe1dc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.170+0000 7fe1effff640 1 -- 172.21.15.27:0/3049993605 reap_dead start 2023-12-19T18:22:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.170+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/3049993605 shutdown_connections 2023-12-19T18:22:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.170+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/3049993605 >> 172.21.15.27:0/3049993605 conn(0x7fe1f00cf7e0 msgr2=0x7fe1f00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.170+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/3049993605 shutdown_connections 2023-12-19T18:22:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.170+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/3049993605 wait complete. 2023-12-19T18:22:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.172+0000 7fe1f4ffd640 1 Processor -- start 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.172+0000 7fe1f4ffd640 1 -- start start 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.173+0000 7fe1f4ffd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1dc004d00 0x7fe1f0154160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.173+0000 7fe1f4ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f01f5990 0x7fe1f00cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.173+0000 7fe1f4ffd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f01f5df0 0x7fe1f00cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.173+0000 7fe1f4ffd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f01f6940 con 0x7fe1f01f5990 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.173+0000 7fe1f4ffd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00c8010 con 0x7fe1f01f5df0 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.173+0000 7fe1f4ffd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f01f6380 con 0x7fe1dc004d00 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.173+0000 7fe1eeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f01f5990 0x7fe1f00cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.173+0000 7fe1eeffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f01f5990 0x7fe1f00cc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35924/0 (socket says 172.21.15.27:35924) 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.173+0000 7fe1eeffd640 1 -- 172.21.15.27:0/712964449 learned_addr learned my addr 172.21.15.27:0/712964449 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.174+0000 7fe1eeffd640 1 -- 172.21.15.27:0/712964449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1dc004d00 msgr2=0x7fe1f0154160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.174+0000 7fe1effff640 1 --2- 172.21.15.27:0/712964449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f01f5df0 0x7fe1f00cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.174+0000 7fe1eeffd640 1 --2- 172.21.15.27:0/712964449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1dc004d00 0x7fe1f0154160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.174+0000 7fe1eeffd640 1 -- 172.21.15.27:0/712964449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f01f5df0 msgr2=0x7fe1f00cc540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.174+0000 7fe1eeffd640 1 --2- 172.21.15.27:0/712964449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f01f5df0 0x7fe1f00cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.174+0000 7fe1eeffd640 1 -- 172.21.15.27:0/712964449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f01f6660 con 0x7fe1f01f5990 2023-12-19T18:22:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.174+0000 7fe1eeffd640 1 --2- 172.21.15.27:0/712964449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f01f5990 0x7fe1f00cc000 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8010810 tx=0x7fe1d8010cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.174+0000 7fe1e7fff640 1 -- 172.21.15.27:0/712964449 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d800eb60 con 0x7fe1f01f5990 2023-12-19T18:22:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.174+0000 7fe1e7fff640 1 -- 172.21.15.27:0/712964449 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1d800ecc0 con 0x7fe1f01f5990 2023-12-19T18:22:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.174+0000 7fe1e7fff640 1 -- 172.21.15.27:0/712964449 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1d8010ec0 con 0x7fe1f01f5990 2023-12-19T18:22:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.174+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/712964449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f00c82b0 con 0x7fe1f01f5990 2023-12-19T18:22:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.175+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/712964449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f015e2c0 con 0x7fe1f01f5990 2023-12-19T18:22:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.176+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/712964449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f00c7e70 con 0x7fe1f01f5990 2023-12-19T18:22:50.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.177+0000 7fe1e7fff640 1 -- 172.21.15.27:0/712964449 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe1d8005990 con 0x7fe1f01f5990 2023-12-19T18:22:50.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.178+0000 7fe1e7fff640 1 --2- 172.21.15.27:0/712964449 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe1d006a460 0x7fe1d006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.178+0000 7fe1e7fff640 1 -- 172.21.15.27:0/712964449 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7fe1d80889e0 con 0x7fe1f01f5990 2023-12-19T18:22:50.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.178+0000 7fe1ef7fe640 1 --2- 172.21.15.27:0/712964449 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe1d006a460 0x7fe1d006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:50.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.180+0000 7fe1ef7fe640 1 --2- 172.21.15.27:0/712964449 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe1d006a460 0x7fe1d006c920 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000e330 tx=0x7fe1e000b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:50.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.182+0000 7fe1e7fff640 1 -- 172.21.15.27:0/712964449 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe1f00c7e70 con 0x7fe1f01f5990 2023-12-19T18:22:50.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.413+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/712964449 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1f00c7e70 con 0x7fe1f01f5990 2023-12-19T18:22:50.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.415+0000 7fe1e7fff640 1 -- 172.21.15.27:0/712964449 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4711 (secure 0 0 0) 0x7fe1f00c7e70 con 0x7fe1f01f5990 2023-12-19T18:22:50.415 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:50.415 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2713867269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2713867269},{"type":"v1","addr":"172.21.15.192:6837","nonce":2713867269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":5534,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/919878747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":919878747},{"type":"v1","addr":"172.21.15.27:6835","nonce":919878747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6160,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/861111763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":861111763},{"type":"v1","addr":"172.21.15.192:6836","nonce":861111763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:22:14.217224+0000","modified":"2023-12-19T18:22:17.743185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7320":{"gid":7320,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3670863819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3670863819},{"type":"v1","addr":"172.21.15.27:6834","nonce":3670863819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T18:22:50.415 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 54 2023-12-19T18:22:50.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.417+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/712964449 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe1d006a460 msgr2=0x7fe1d006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:50.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.417+0000 7fe1f4ffd640 1 --2- 172.21.15.27:0/712964449 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe1d006a460 0x7fe1d006c920 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe1e000e330 tx=0x7fe1e000b040 comp rx=0 tx=0).stop 2023-12-19T18:22:50.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.418+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/712964449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f01f5990 msgr2=0x7fe1f00cc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:50.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.418+0000 7fe1f4ffd640 1 --2- 172.21.15.27:0/712964449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f01f5990 0x7fe1f00cc000 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7fe1d8010810 tx=0x7fe1d8010cd0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.418+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/712964449 shutdown_connections 2023-12-19T18:22:50.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.418+0000 7fe1f4ffd640 1 --2- 172.21.15.27:0/712964449 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe1d006a460 0x7fe1d006c920 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.418+0000 7fe1f4ffd640 1 --2- 172.21.15.27:0/712964449 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1f01f5df0 0x7fe1f00cc540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.418+0000 7fe1f4ffd640 1 --2- 172.21.15.27:0/712964449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f01f5990 0x7fe1f00cc000 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.418+0000 7fe1f4ffd640 1 --2- 172.21.15.27:0/712964449 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1dc004d00 0x7fe1f0154160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.418+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/712964449 >> 172.21.15.27:0/712964449 conn(0x7fe1f00cf7e0 msgr2=0x7fe1f00b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:50.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.418+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/712964449 shutdown_connections 2023-12-19T18:22:50.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.418+0000 7fe1f4ffd640 1 -- 172.21.15.27:0/712964449 wait complete. 2023-12-19T18:22:50.437 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:22:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.664+0000 7f99ea4c5640 1 Processor -- start 2023-12-19T18:22:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.664+0000 7f99ea4c5640 1 -- start start 2023-12-19T18:22:50.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.666+0000 7f99ea4c5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f99e41553a0 0x7f99e40c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.666+0000 7f99ea4c5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f99e415ee30 0x7f99e40c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.666+0000 7f99ea4c5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e40b0360 con 0x7f99e40afeb0 2023-12-19T18:22:50.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.666+0000 7f99ea4c5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e40cbcf0 con 0x7f99e41553a0 2023-12-19T18:22:50.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.666+0000 7f99ea4c5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e40cb350 con 0x7f99e415ee30 2023-12-19T18:22:50.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.666+0000 7f99e94c3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f99e41553a0 0x7f99e40c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:50.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.666+0000 7f99e8cc2640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f99e40afeb0 0x7f99e415e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47982/0 (socket says 172.21.15.27:47982) 2023-12-19T18:22:50.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.666+0000 7f99e8cc2640 1 -- 172.21.15.27:0/3229572061 learned_addr learned my addr 172.21.15.27:0/3229572061 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:50.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.667+0000 7f99e9cc4640 1 --2- 172.21.15.27:0/3229572061 >> v2:172.21.15.192:3300/0 conn(0x7f99e415ee30 0x7f99e40c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:50.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.667+0000 7f99e94c3640 1 -- 172.21.15.27:0/3229572061 >> v2:172.21.15.192:3300/0 conn(0x7f99e415ee30 msgr2=0x7f99e40c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:50.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.667+0000 7f99e94c3640 1 --2- 172.21.15.27:0/3229572061 >> v2:172.21.15.192:3300/0 conn(0x7f99e415ee30 0x7f99e40c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.667+0000 7f99e94c3640 1 -- 172.21.15.27:0/3229572061 >> v1:172.21.15.27:6789/0 conn(0x7f99e40afeb0 legacy=0x7f99e415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:50.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.667+0000 7f99e94c3640 1 -- 172.21.15.27:0/3229572061 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e40cb630 con 0x7f99e41553a0 2023-12-19T18:22:50.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.668+0000 7f99e94c3640 1 --2- 172.21.15.27:0/3229572061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e41553a0 0x7f99e40c50a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f99e4158d00 tx=0x7f99e002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2568a5355cf7ac2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:50.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.668+0000 7f99dbfff640 1 -- 172.21.15.27:0/3229572061 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e002e070 con 0x7f99e41553a0 2023-12-19T18:22:50.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.668+0000 7f99dbfff640 1 -- 172.21.15.27:0/3229572061 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e00315a0 con 0x7f99e41553a0 2023-12-19T18:22:50.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.668+0000 7f99dbfff640 1 -- 172.21.15.27:0/3229572061 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0031930 con 0x7f99e41553a0 2023-12-19T18:22:50.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.669+0000 7f99ea4c5640 1 -- 172.21.15.27:0/3229572061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e41553a0 msgr2=0x7f99e40c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:50.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.669+0000 7f99ea4c5640 1 --2- 172.21.15.27:0/3229572061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e41553a0 0x7f99e40c50a0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f99e4158d00 tx=0x7f99e002b840 comp rx=0 tx=0).stop 2023-12-19T18:22:50.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.669+0000 7f99ea4c5640 1 -- 172.21.15.27:0/3229572061 shutdown_connections 2023-12-19T18:22:50.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.669+0000 7f99ea4c5640 1 --2- 172.21.15.27:0/3229572061 >> v2:172.21.15.192:3300/0 conn(0x7f99e415ee30 0x7f99e40c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.669+0000 7f99ea4c5640 1 --2- 172.21.15.27:0/3229572061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e41553a0 0x7f99e40c50a0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.669+0000 7f99ea4c5640 1 -- 172.21.15.27:0/3229572061 >> 172.21.15.27:0/3229572061 conn(0x7f99e4069c70 msgr2=0x7f99e406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:50.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.669+0000 7f99ea4c5640 1 -- 172.21.15.27:0/3229572061 shutdown_connections 2023-12-19T18:22:50.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.669+0000 7f99ea4c5640 1 -- 172.21.15.27:0/3229572061 wait complete. 2023-12-19T18:22:50.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.671+0000 7f99ea4c5640 1 Processor -- start 2023-12-19T18:22:50.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.671+0000 7f99ea4c5640 1 -- start start 2023-12-19T18:22:50.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.672+0000 7f99ea4c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e40afeb0 0x7f99e41f7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.672+0000 7f99ea4c5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99e41553a0 0x7f99e40cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.672+0000 7f99ea4c5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99e415ee30 0x7f99e4153480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.672+0000 7f99ea4c5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e40cca50 con 0x7f99e40afeb0 2023-12-19T18:22:50.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.672+0000 7f99ea4c5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e40cccd0 con 0x7f99e41553a0 2023-12-19T18:22:50.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.672+0000 7f99ea4c5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e40ccfb0 con 0x7f99e415ee30 2023-12-19T18:22:50.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.673+0000 7f99e94c3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e40afeb0 0x7f99e41f7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:50.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.673+0000 7f99e94c3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e40afeb0 0x7f99e41f7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35936/0 (socket says 172.21.15.27:35936) 2023-12-19T18:22:50.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.673+0000 7f99e94c3640 1 -- 172.21.15.27:0/2465159370 learned_addr learned my addr 172.21.15.27:0/2465159370 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:50.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.673+0000 7f99e8cc2640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99e41553a0 0x7f99e40cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:50.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.673+0000 7f99e9cc4640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99e415ee30 0x7f99e4153480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:50.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.673+0000 7f99e94c3640 1 -- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99e415ee30 msgr2=0x7f99e4153480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:50.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.673+0000 7f99e94c3640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99e415ee30 0x7f99e4153480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.673+0000 7f99e94c3640 1 -- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99e41553a0 msgr2=0x7f99e40cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:50.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.673+0000 7f99e94c3640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99e41553a0 0x7f99e40cc510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.673+0000 7f99e94c3640 1 -- 172.21.15.27:0/2465159370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e0009400 con 0x7f99e40afeb0 2023-12-19T18:22:50.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.673+0000 7f99e8cc2640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99e41553a0 0x7f99e40cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:50.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.674+0000 7f99e94c3640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e40afeb0 0x7f99e41f7520 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f99e0000c00 tx=0x7f99e00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:50.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.674+0000 7f99d9ffb640 1 -- 172.21.15.27:0/2465159370 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0032840 con 0x7f99e40afeb0 2023-12-19T18:22:50.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.674+0000 7f99d9ffb640 1 -- 172.21.15.27:0/2465159370 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e00329a0 con 0x7f99e40afeb0 2023-12-19T18:22:50.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.674+0000 7f99ea4c5640 1 -- 172.21.15.27:0/2465159370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e40cd290 con 0x7f99e40afeb0 2023-12-19T18:22:50.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.674+0000 7f99d9ffb640 1 -- 172.21.15.27:0/2465159370 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0032d30 con 0x7f99e40afeb0 2023-12-19T18:22:50.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.674+0000 7f99ea4c5640 1 -- 172.21.15.27:0/2465159370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e40cd850 con 0x7f99e40afeb0 2023-12-19T18:22:50.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.675+0000 7f99ea4c5640 1 -- 172.21.15.27:0/2465159370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e404e840 con 0x7f99e40afeb0 2023-12-19T18:22:50.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.677+0000 7f99d9ffb640 1 -- 172.21.15.27:0/2465159370 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f99e004d3f0 con 0x7f99e40afeb0 2023-12-19T18:22:50.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.678+0000 7f99d9ffb640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99c806a430 0x7f99c806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:50.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.678+0000 7f99d9ffb640 1 -- 172.21.15.27:0/2465159370 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f99e00aa6b0 con 0x7f99e40afeb0 2023-12-19T18:22:50.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.679+0000 7f99e8cc2640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99c806a430 0x7f99c806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:50.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.680+0000 7f99e8cc2640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99c806a430 0x7f99c806c8f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f99e4154410 tx=0x7f99d4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:50.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.683+0000 7f99d9ffb640 1 -- 172.21.15.27:0/2465159370 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f99e0075ba0 con 0x7f99e40afeb0 2023-12-19T18:22:50.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.884+0000 7f99ea4c5640 1 -- 172.21.15.27:0/2465159370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99e40af6a0 con 0x7f99e40afeb0 2023-12-19T18:22:50.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.887+0000 7f99d9ffb640 1 -- 172.21.15.27:0/2465159370 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16203 (secure 0 0 0) 0x7f99e00791c0 con 0x7f99e40afeb0 2023-12-19T18:22:50.888 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:50.888 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":140,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:22:44.753680+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:22:12.328518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T18:22:13.251708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":5,"to":3}]},{"pgid":"14.5","mappings":[{"from":5,"to":0}]},{"pgid":"14.1f","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:22:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.890+0000 7f99ea4c5640 1 -- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99c806a430 msgr2=0x7f99c806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.890+0000 7f99ea4c5640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99c806a430 0x7f99c806c8f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f99e4154410 tx=0x7f99d4009040 comp rx=0 tx=0).stop 2023-12-19T18:22:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.890+0000 7f99ea4c5640 1 -- 172.21.15.27:0/2465159370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e40afeb0 msgr2=0x7f99e41f7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.890+0000 7f99ea4c5640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e40afeb0 0x7f99e41f7520 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f99e0000c00 tx=0x7f99e00046c0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.890+0000 7f99ea4c5640 1 -- 172.21.15.27:0/2465159370 shutdown_connections 2023-12-19T18:22:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.890+0000 7f99ea4c5640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99c806a430 0x7f99c806c8f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.890+0000 7f99ea4c5640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99e415ee30 0x7f99e4153480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.890+0000 7f99ea4c5640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99e41553a0 0x7f99e40cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.890+0000 7f99ea4c5640 1 --2- 172.21.15.27:0/2465159370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e40afeb0 0x7f99e41f7520 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.890+0000 7f99ea4c5640 1 -- 172.21.15.27:0/2465159370 >> 172.21.15.27:0/2465159370 conn(0x7f99e4069c70 msgr2=0x7f99e406b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:50.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.890+0000 7f99ea4c5640 1 -- 172.21.15.27:0/2465159370 shutdown_connections 2023-12-19T18:22:50.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:50.891+0000 7f99ea4c5640 1 -- 172.21.15.27:0/2465159370 wait complete. 2023-12-19T18:22:50.912 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:22:50.912 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:22:51.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.120+0000 7f375ce83640 1 Processor -- start 2023-12-19T18:22:51.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.120+0000 7f375ce83640 1 -- start start 2023-12-19T18:22:51.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.121+0000 7f375ce83640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f37580cb300 0x7f37580cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:51.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.121+0000 7f375ce83640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3758161010 0x7f37580cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:51.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.121+0000 7f375ce83640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37580cbc20 con 0x7f37580c8f90 2023-12-19T18:22:51.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.122+0000 7f375ce83640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37580c4480 con 0x7f37580cb300 2023-12-19T18:22:51.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.122+0000 7f375ce83640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37581468c0 con 0x7f3758161010 2023-12-19T18:22:51.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.122+0000 7f37577fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f37580cb300 0x7f37580cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:51.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.122+0000 7f37577fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f37580cb300 0x7f37580cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35948/0 (socket says 172.21.15.27:35948) 2023-12-19T18:22:51.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.122+0000 7f37577fe640 1 -- 172.21.15.27:0/1744296906 learned_addr learned my addr 172.21.15.27:0/1744296906 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:51.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.122+0000 7f3757fff640 1 --2- 172.21.15.27:0/1744296906 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3758161010 0x7f37580cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:51.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.122+0000 7f37577fe640 1 -- 172.21.15.27:0/1744296906 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3758161010 msgr2=0x7f37580cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:51.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.122+0000 7f37577fe640 1 --2- 172.21.15.27:0/1744296906 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3758161010 0x7f37580cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.122+0000 7f37577fe640 1 -- 172.21.15.27:0/1744296906 >> v1:172.21.15.192:6789/0 conn(0x7f37580c8f90 legacy=0x7f3758160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:51.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.122+0000 7f37577fe640 1 -- 172.21.15.27:0/1744296906 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3758145d60 con 0x7f37580cb300 2023-12-19T18:22:51.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.122+0000 7f3757fff640 1 --2- 172.21.15.27:0/1744296906 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3758161010 0x7f37580cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:51.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.124+0000 7f37577fe640 1 --2- 172.21.15.27:0/1744296906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37580cb300 0x7f37580cb6e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f37480098a0 tx=0x7f374802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c33e3c554b552 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:51.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.124+0000 7f37567fc640 1 -- 172.21.15.27:0/1744296906 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748031640 con 0x7f37580cb300 2023-12-19T18:22:51.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.124+0000 7f37567fc640 1 -- 172.21.15.27:0/1744296906 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37480317a0 con 0x7f37580cb300 2023-12-19T18:22:51.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.124+0000 7f37567fc640 1 -- 172.21.15.27:0/1744296906 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3748031b30 con 0x7f37580cb300 2023-12-19T18:22:51.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.125+0000 7f375ce83640 1 -- 172.21.15.27:0/1744296906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37580cb300 msgr2=0x7f37580cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:51.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.125+0000 7f375ce83640 1 --2- 172.21.15.27:0/1744296906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37580cb300 0x7f37580cb6e0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f37480098a0 tx=0x7f374802b8f0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.125+0000 7f375ce83640 1 -- 172.21.15.27:0/1744296906 shutdown_connections 2023-12-19T18:22:51.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.125+0000 7f375ce83640 1 --2- 172.21.15.27:0/1744296906 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3758161010 0x7f37580cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.125+0000 7f375ce83640 1 --2- 172.21.15.27:0/1744296906 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37580cb300 0x7f37580cb6e0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.125+0000 7f375ce83640 1 -- 172.21.15.27:0/1744296906 >> 172.21.15.27:0/1744296906 conn(0x7f3758069c70 msgr2=0x7f375806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:51.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.125+0000 7f375ce83640 1 -- 172.21.15.27:0/1744296906 shutdown_connections 2023-12-19T18:22:51.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.125+0000 7f375ce83640 1 -- 172.21.15.27:0/1744296906 wait complete. 2023-12-19T18:22:51.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.127+0000 7f375ce83640 1 Processor -- start 2023-12-19T18:22:51.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.127+0000 7f375ce83640 1 -- start start 2023-12-19T18:22:51.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.128+0000 7f375ce83640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37580c8f90 0x7f37581f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:51.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.128+0000 7f375ce83640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37580cb300 0x7f37580703c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:51.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.128+0000 7f375ce83640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3758161010 0x7f3758070900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:51.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.128+0000 7f375ce83640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37581f24f0 con 0x7f3758161010 2023-12-19T18:22:51.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.128+0000 7f375ce83640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375815eb10 con 0x7f37580cb300 2023-12-19T18:22:51.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.128+0000 7f375ce83640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375815df00 con 0x7f37580c8f90 2023-12-19T18:22:51.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.128+0000 7f3757fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3758161010 0x7f3758070900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:51.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.128+0000 7f3757fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3758161010 0x7f3758070900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35964/0 (socket says 172.21.15.27:35964) 2023-12-19T18:22:51.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.128+0000 7f3757fff640 1 -- 172.21.15.27:0/323666479 learned_addr learned my addr 172.21.15.27:0/323666479 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:51.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.129+0000 7f37577fe640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37580c8f90 0x7f37581f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:51.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.129+0000 7f3756ffd640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37580cb300 0x7f37580703c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:51.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.129+0000 7f3757fff640 1 -- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37580c8f90 msgr2=0x7f37581f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:51.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.129+0000 7f3757fff640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37580c8f90 0x7f37581f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.129+0000 7f3757fff640 1 -- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37580cb300 msgr2=0x7f37580703c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:51.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.129+0000 7f3757fff640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37580cb300 0x7f37580703c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.129+0000 7f3757fff640 1 -- 172.21.15.27:0/323666479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37480094d0 con 0x7f3758161010 2023-12-19T18:22:51.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.129+0000 7f3757fff640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3758161010 0x7f3758070900 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f375000cf20 tx=0x7f375000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:51.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.129+0000 7f373ffff640 1 -- 172.21.15.27:0/323666479 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375000d840 con 0x7f3758161010 2023-12-19T18:22:51.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.129+0000 7f373ffff640 1 -- 172.21.15.27:0/323666479 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f375000d9a0 con 0x7f3758161010 2023-12-19T18:22:51.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.129+0000 7f373ffff640 1 -- 172.21.15.27:0/323666479 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f375000dd50 con 0x7f3758161010 2023-12-19T18:22:51.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.129+0000 7f375ce83640 1 -- 172.21.15.27:0/323666479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375815e1e0 con 0x7f3758161010 2023-12-19T18:22:51.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.130+0000 7f375ce83640 1 -- 172.21.15.27:0/323666479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375815e7a0 con 0x7f3758161010 2023-12-19T18:22:51.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.131+0000 7f375ce83640 1 -- 172.21.15.27:0/323666479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375815eca0 con 0x7f3758161010 2023-12-19T18:22:51.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.132+0000 7f373ffff640 1 -- 172.21.15.27:0/323666479 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f375001e3b0 con 0x7f3758161010 2023-12-19T18:22:51.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.133+0000 7f373ffff640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f374406a460 0x7f374406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:51.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.133+0000 7f373ffff640 1 -- 172.21.15.27:0/323666479 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f37500893c0 con 0x7f3758161010 2023-12-19T18:22:51.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.133+0000 7f37577fe640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f374406a460 0x7f374406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:51.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.134+0000 7f37577fe640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f374406a460 0x7f374406c920 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3748036000 tx=0x7f374803c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:51.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.137+0000 7f373ffff640 1 -- 172.21.15.27:0/323666479 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3750054940 con 0x7f3758161010 2023-12-19T18:22:51.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.371+0000 7f375ce83640 1 -- 172.21.15.27:0/323666479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37580671b0 con 0x7f3758161010 2023-12-19T18:22:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.372+0000 7f373ffff640 1 -- 172.21.15.27:0/323666479 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f37580671b0 con 0x7f3758161010 2023-12-19T18:22:51.372 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:51.373 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2713867269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2713867269},{"type":"v1","addr":"172.21.15.192:6837","nonce":2713867269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":5534,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/919878747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":919878747},{"type":"v1","addr":"172.21.15.27:6835","nonce":919878747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6160,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/861111763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":861111763},{"type":"v1","addr":"172.21.15.192:6836","nonce":861111763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:22:14.217224+0000","modified":"2023-12-19T18:22:17.743185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7320":{"gid":7320,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3670863819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3670863819},{"type":"v1","addr":"172.21.15.27:6834","nonce":3670863819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T18:22:51.373 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 54 2023-12-19T18:22:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.375+0000 7f375ce83640 1 -- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f374406a460 msgr2=0x7f374406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.375+0000 7f375ce83640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f374406a460 0x7f374406c920 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f3748036000 tx=0x7f374803c040 comp rx=0 tx=0).stop 2023-12-19T18:22:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.375+0000 7f375ce83640 1 -- 172.21.15.27:0/323666479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3758161010 msgr2=0x7f3758070900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.375+0000 7f375ce83640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3758161010 0x7f3758070900 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f375000cf20 tx=0x7f375000a7d0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.375+0000 7f375ce83640 1 -- 172.21.15.27:0/323666479 shutdown_connections 2023-12-19T18:22:51.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.375+0000 7f375ce83640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f374406a460 0x7f374406c920 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.375+0000 7f375ce83640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3758161010 0x7f3758070900 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.375+0000 7f375ce83640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f37580cb300 0x7f37580703c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.375+0000 7f375ce83640 1 --2- 172.21.15.27:0/323666479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f37580c8f90 0x7f37581f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.375+0000 7f375ce83640 1 -- 172.21.15.27:0/323666479 >> 172.21.15.27:0/323666479 conn(0x7f3758069c70 msgr2=0x7f375806bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:51.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.375+0000 7f375ce83640 1 -- 172.21.15.27:0/323666479 shutdown_connections 2023-12-19T18:22:51.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.375+0000 7f375ce83640 1 -- 172.21.15.27:0/323666479 wait complete. 2023-12-19T18:22:51.394 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:22:51.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.618+0000 7f333b2c0640 1 Processor -- start 2023-12-19T18:22:51.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.619+0000 7f333b2c0640 1 -- start start 2023-12-19T18:22:51.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.620+0000 7f333b2c0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33340cb300 0x7f33340cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.620+0000 7f333b2c0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3334161010 0x7f33340cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.621+0000 7f333b2c0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33340cbc20 con 0x7f33340c8f90 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.621+0000 7f333b2c0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33340c4480 con 0x7f3334161010 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.621+0000 7f333b2c0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33341468c0 con 0x7f33340cb300 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.621+0000 7f3339abd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f33340c8f90 0x7f3334160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47984/0 (socket says 172.21.15.27:47984) 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.621+0000 7f3339abd640 1 -- 172.21.15.27:0/4198207886 learned_addr learned my addr 172.21.15.27:0/4198207886 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.621+0000 7f333aabf640 1 --2- 172.21.15.27:0/4198207886 >> v2:172.21.15.27:3300/0 conn(0x7f3334161010 0x7f33340cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.621+0000 7f333a2be640 1 --2- 172.21.15.27:0/4198207886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33340cb300 0x7f33340cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.621+0000 7f33392bc640 1 -- 172.21.15.27:0/4198207886 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648680285 0 0) 0x7f33340cbc20 con 0x7f33340c8f90 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.622+0000 7f33392bc640 1 -- 172.21.15.27:0/4198207886 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3320003520 con 0x7f33340c8f90 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.622+0000 7f333aabf640 1 -- 172.21.15.27:0/4198207886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33340cb300 msgr2=0x7f33340cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.622+0000 7f333aabf640 1 --2- 172.21.15.27:0/4198207886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33340cb300 0x7f33340cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.622+0000 7f333aabf640 1 -- 172.21.15.27:0/4198207886 >> v1:172.21.15.27:6789/0 conn(0x7f33340c8f90 legacy=0x7f3334160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:51.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.622+0000 7f333aabf640 1 -- 172.21.15.27:0/4198207886 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3334145d60 con 0x7f3334161010 2023-12-19T18:22:51.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.623+0000 7f333aabf640 1 --2- 172.21.15.27:0/4198207886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3334161010 0x7f33340cd090 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f332c009d90 tx=0x7f332c02beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a21067b1410c8d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:51.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.623+0000 7f33392bc640 1 -- 172.21.15.27:0/4198207886 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c02cc60 con 0x7f3334161010 2023-12-19T18:22:51.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.623+0000 7f33392bc640 1 -- 172.21.15.27:0/4198207886 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f332c02cdc0 con 0x7f3334161010 2023-12-19T18:22:51.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.623+0000 7f33392bc640 1 -- 172.21.15.27:0/4198207886 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f332c034660 con 0x7f3334161010 2023-12-19T18:22:51.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.624+0000 7f333b2c0640 1 -- 172.21.15.27:0/4198207886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3334161010 msgr2=0x7f33340cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:51.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.624+0000 7f333b2c0640 1 --2- 172.21.15.27:0/4198207886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3334161010 0x7f33340cd090 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f332c009d90 tx=0x7f332c02beb0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.624+0000 7f333b2c0640 1 -- 172.21.15.27:0/4198207886 shutdown_connections 2023-12-19T18:22:51.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.624+0000 7f333b2c0640 1 --2- 172.21.15.27:0/4198207886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3334161010 0x7f33340cd090 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.624+0000 7f333b2c0640 1 --2- 172.21.15.27:0/4198207886 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f33340cb300 0x7f33340cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.624+0000 7f333b2c0640 1 -- 172.21.15.27:0/4198207886 >> 172.21.15.27:0/4198207886 conn(0x7f3334069c70 msgr2=0x7f333406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:51.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.624+0000 7f333b2c0640 1 -- 172.21.15.27:0/4198207886 shutdown_connections 2023-12-19T18:22:51.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.624+0000 7f333b2c0640 1 -- 172.21.15.27:0/4198207886 wait complete. 2023-12-19T18:22:51.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.626+0000 7f333b2c0640 1 Processor -- start 2023-12-19T18:22:51.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.627+0000 7f333b2c0640 1 -- start start 2023-12-19T18:22:51.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.628+0000 7f333b2c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33340c8f90 0x7f33340ae140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:51.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.628+0000 7f333b2c0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33340cb300 0x7f333415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:51.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.628+0000 7f333b2c0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3334161010 0x7f33340af1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:51.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.628+0000 7f333b2c0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f333415e200 con 0x7f33340cb300 2023-12-19T18:22:51.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.628+0000 7f333b2c0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33340ae760 con 0x7f33340c8f90 2023-12-19T18:22:51.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.628+0000 7f333b2c0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f333415d9e0 con 0x7f3334161010 2023-12-19T18:22:51.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.628+0000 7f3339abd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33340cb300 0x7f333415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:51.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.628+0000 7f3339abd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33340cb300 0x7f333415d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35988/0 (socket says 172.21.15.27:35988) 2023-12-19T18:22:51.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.628+0000 7f3339abd640 1 -- 172.21.15.27:0/4256156374 learned_addr learned my addr 172.21.15.27:0/4256156374 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:51.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.628+0000 7f333a2be640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33340c8f90 0x7f33340ae140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:51.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.629+0000 7f333aabf640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3334161010 0x7f33340af1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:51.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.629+0000 7f3339abd640 1 -- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3334161010 msgr2=0x7f33340af1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:51.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.629+0000 7f3339abd640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3334161010 0x7f33340af1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.629+0000 7f3339abd640 1 -- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33340c8f90 msgr2=0x7f33340ae140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:51.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.629+0000 7f3339abd640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33340c8f90 0x7f33340ae140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.629+0000 7f3339abd640 1 -- 172.21.15.27:0/4256156374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f332c009a70 con 0x7f33340cb300 2023-12-19T18:22:51.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.629+0000 7f3339abd640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33340cb300 0x7f333415d4a0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f332400c410 tx=0x7f332400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:51.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.629+0000 7f332affd640 1 -- 172.21.15.27:0/4256156374 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3324005ce0 con 0x7f33340cb300 2023-12-19T18:22:51.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.629+0000 7f332affd640 1 -- 172.21.15.27:0/4256156374 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3324005e40 con 0x7f33340cb300 2023-12-19T18:22:51.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.629+0000 7f333b2c0640 1 -- 172.21.15.27:0/4256156374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33340af6f0 con 0x7f33340cb300 2023-12-19T18:22:51.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.629+0000 7f332affd640 1 -- 172.21.15.27:0/4256156374 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33240072f0 con 0x7f33340cb300 2023-12-19T18:22:51.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.630+0000 7f333b2c0640 1 -- 172.21.15.27:0/4256156374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f333415df20 con 0x7f33340cb300 2023-12-19T18:22:51.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.631+0000 7f333b2c0640 1 -- 172.21.15.27:0/4256156374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f333404e840 con 0x7f33340cb300 2023-12-19T18:22:51.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.632+0000 7f332affd640 1 -- 172.21.15.27:0/4256156374 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3324007450 con 0x7f33340cb300 2023-12-19T18:22:51.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.633+0000 7f332affd640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f331806a430 0x7f331806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:51.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.633+0000 7f332affd640 1 -- 172.21.15.27:0/4256156374 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f3324087fd0 con 0x7f33340cb300 2023-12-19T18:22:51.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.633+0000 7f333a2be640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f331806a430 0x7f331806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:51.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.635+0000 7f333a2be640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f331806a430 0x7f331806c8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3330009a30 tx=0x7f3330008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:51.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.637+0000 7f332affd640 1 -- 172.21.15.27:0/4256156374 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f33240570b0 con 0x7f33340cb300 2023-12-19T18:22:51.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.873+0000 7f333b2c0640 1 -- 172.21.15.27:0/4256156374 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f33340adc50 con 0x7f33340cb300 2023-12-19T18:22:51.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.874+0000 7f332affd640 1 -- 172.21.15.27:0/4256156374 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f3324056c40 con 0x7f33340cb300 2023-12-19T18:22:51.874 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:51.875 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2713867269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2713867269},{"type":"v1","addr":"172.21.15.192:6837","nonce":2713867269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":5534,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/919878747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":919878747},{"type":"v1","addr":"172.21.15.27:6835","nonce":919878747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6160,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/861111763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":861111763},{"type":"v1","addr":"172.21.15.192:6836","nonce":861111763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:22:14.217224+0000","modified":"2023-12-19T18:22:17.743185+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7320},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7320":{"gid":7320,"name":"c","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6834/3670863819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3670863819},{"type":"v1","addr":"172.21.15.27:6834","nonce":3670863819}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-19T18:22:51.875 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 54 2023-12-19T18:22:51.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.877+0000 7f333b2c0640 1 -- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f331806a430 msgr2=0x7f331806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:51.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.877+0000 7f333b2c0640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f331806a430 0x7f331806c8f0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3330009a30 tx=0x7f3330008040 comp rx=0 tx=0).stop 2023-12-19T18:22:51.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.877+0000 7f333b2c0640 1 -- 172.21.15.27:0/4256156374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33340cb300 msgr2=0x7f333415d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:51.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.877+0000 7f333b2c0640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33340cb300 0x7f333415d4a0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f332400c410 tx=0x7f332400c8d0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.877+0000 7f333b2c0640 1 -- 172.21.15.27:0/4256156374 shutdown_connections 2023-12-19T18:22:51.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.877+0000 7f333b2c0640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f331806a430 0x7f331806c8f0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.877+0000 7f333b2c0640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3334161010 0x7f33340af1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.877+0000 7f333b2c0640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33340cb300 0x7f333415d4a0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.877+0000 7f333b2c0640 1 --2- 172.21.15.27:0/4256156374 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f33340c8f90 0x7f33340ae140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:51.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.877+0000 7f333b2c0640 1 -- 172.21.15.27:0/4256156374 >> 172.21.15.27:0/4256156374 conn(0x7f3334069c70 msgr2=0x7f3334065910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:51.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.878+0000 7f333b2c0640 1 -- 172.21.15.27:0/4256156374 shutdown_connections 2023-12-19T18:22:51.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:51.878+0000 7f333b2c0640 1 -- 172.21.15.27:0/4256156374 wait complete. 2023-12-19T18:22:51.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:22:52.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.125+0000 7f6628849640 1 Processor -- start 2023-12-19T18:22:52.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.125+0000 7f6628849640 1 -- start start 2023-12-19T18:22:52.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.127+0000 7f6628849640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66240b4ee0 0x7f66240d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.127+0000 7f6628849640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f66240d1f60 0x7f66240b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.127+0000 7f6628849640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66240b6540 con 0x7f662415ceb0 2023-12-19T18:22:52.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.127+0000 7f6628849640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66240d23f0 con 0x7f66240d1f60 2023-12-19T18:22:52.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.127+0000 7f6628849640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66240cdfb0 con 0x7f66240b4ee0 2023-12-19T18:22:52.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.127+0000 7f6622d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f662415ceb0 0x7f66240b47d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60426/0 (socket says 172.21.15.27:60426) 2023-12-19T18:22:52.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.127+0000 7f6622d9d640 1 -- 172.21.15.27:0/2009466646 learned_addr learned my addr 172.21.15.27:0/2009466646 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.128+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3521113361 0 0) 0x7f66240b6540 con 0x7f662415ceb0 2023-12-19T18:22:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.128+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6610003520 con 0x7f662415ceb0 2023-12-19T18:22:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.128+0000 7f661bfff640 1 --2- 172.21.15.27:0/2009466646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66240b4ee0 0x7f66240d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.128+0000 7f662359e640 1 --2- 172.21.15.27:0/2009466646 >> v2:172.21.15.192:3300/0 conn(0x7f66240d1f60 0x7f66240b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.128+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3046155338 0 0) 0x7f6610003520 con 0x7f662415ceb0 2023-12-19T18:22:52.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.128+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66240b4ee0 msgr2=0x7f66240d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.128+0000 7f662259c640 1 --2- 172.21.15.27:0/2009466646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66240b4ee0 0x7f66240d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.128+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 >> v2:172.21.15.192:3300/0 conn(0x7f66240d1f60 msgr2=0x7f66240b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.128+0000 7f662259c640 1 --2- 172.21.15.27:0/2009466646 >> v2:172.21.15.192:3300/0 conn(0x7f66240d1f60 0x7f66240b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.128+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66240d2600 con 0x7f662415ceb0 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.128+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f6614001670 con 0x7f662415ceb0 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.128+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 >> v1:172.21.15.27:6789/0 conn(0x7f662415ceb0 legacy=0x7f66240b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.129+0000 7f662259c640 1 --2- 172.21.15.27:0/2009466646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610003970 0x7f6610003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.129+0000 7f662259c640 1 --2- 172.21.15.27:0/2009466646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66100047c0 0x7f6610004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.129+0000 7f662259c640 1 --2- 172.21.15.27:0/2009466646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66240b4ee0 0x7f6610005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.129+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66240d23f0 con 0x7f6610003970 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.129+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66240cdfb0 con 0x7f66100047c0 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.129+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f661000ba30 con 0x7f66240b4ee0 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.129+0000 7f6622d9d640 1 --2- 172.21.15.27:0/2009466646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610003970 0x7f6610003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.129+0000 7f662359e640 1 --2- 172.21.15.27:0/2009466646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66240b4ee0 0x7f6610005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.129+0000 7f661bfff640 1 --2- 172.21.15.27:0/2009466646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66100047c0 0x7f6610004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.130+0000 7f6622d9d640 1 -- 172.21.15.27:0/2009466646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66240b4ee0 msgr2=0x7f6610005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.130+0000 7f6622d9d640 1 --2- 172.21.15.27:0/2009466646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66240b4ee0 0x7f6610005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.130+0000 7f6622d9d640 1 -- 172.21.15.27:0/2009466646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66100047c0 msgr2=0x7f6610004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.130+0000 7f6622d9d640 1 --2- 172.21.15.27:0/2009466646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66100047c0 0x7f6610004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.130+0000 7f6622d9d640 1 -- 172.21.15.27:0/2009466646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6610003520 con 0x7f6610003970 2023-12-19T18:22:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.131+0000 7f6622d9d640 1 --2- 172.21.15.27:0/2009466646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610003970 0x7f6610003d90 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f661400c210 tx=0x7f66100057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8d59bb79c70a3ea server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.132+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6614005ad0 con 0x7f6610003970 2023-12-19T18:22:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.132+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6614005c30 con 0x7f6610003970 2023-12-19T18:22:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.132+0000 7f662259c640 1 -- 172.21.15.27:0/2009466646 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661402c8a0 con 0x7f6610003970 2023-12-19T18:22:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.132+0000 7f6628849640 1 -- 172.21.15.27:0/2009466646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610003970 msgr2=0x7f6610003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.132+0000 7f6628849640 1 --2- 172.21.15.27:0/2009466646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6610003970 0x7f6610003d90 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f661400c210 tx=0x7f66100057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.132+0000 7f662359e640 1 -- 172.21.15.27:0/2009466646 reap_dead start 2023-12-19T18:22:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.132+0000 7f6628849640 1 -- 172.21.15.27:0/2009466646 shutdown_connections 2023-12-19T18:22:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.132+0000 7f6628849640 1 -- 172.21.15.27:0/2009466646 >> 172.21.15.27:0/2009466646 conn(0x7f66240cf7d0 msgr2=0x7f66240cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.132+0000 7f6628849640 1 -- 172.21.15.27:0/2009466646 shutdown_connections 2023-12-19T18:22:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.132+0000 7f6628849640 1 -- 172.21.15.27:0/2009466646 wait complete. 2023-12-19T18:22:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.134+0000 7f6628849640 1 Processor -- start 2023-12-19T18:22:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.134+0000 7f6628849640 1 -- start start 2023-12-19T18:22:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.135+0000 7f6628849640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6610003970 0x7f66240c9e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.135+0000 7f6628849640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f662414e1d0 0x7f66240cb550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.135+0000 7f6628849640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66240c9350 0x7f66240cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.135+0000 7f6628849640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66240cba90 con 0x7f66240c9350 2023-12-19T18:22:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.135+0000 7f6628849640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66240ca4b0 con 0x7f662414e1d0 2023-12-19T18:22:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.135+0000 7f6628849640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66240ca790 con 0x7f6610003970 2023-12-19T18:22:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.136+0000 7f662359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66240c9350 0x7f66240cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.136+0000 7f662359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66240c9350 0x7f66240cc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42048/0 (socket says 172.21.15.27:42048) 2023-12-19T18:22:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.136+0000 7f662359e640 1 -- 172.21.15.27:0/2282808827 learned_addr learned my addr 172.21.15.27:0/2282808827 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.136+0000 7f662359e640 1 -- 172.21.15.27:0/2282808827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6610003970 msgr2=0x7f66240c9e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.136+0000 7f662359e640 1 --2- 172.21.15.27:0/2282808827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6610003970 0x7f66240c9e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.136+0000 7f662359e640 1 -- 172.21.15.27:0/2282808827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f662414e1d0 msgr2=0x7f66240cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.136+0000 7f661bfff640 1 --2- 172.21.15.27:0/2282808827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f662414e1d0 0x7f66240cb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.136+0000 7f662359e640 1 --2- 172.21.15.27:0/2282808827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f662414e1d0 0x7f66240cb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.136+0000 7f662359e640 1 -- 172.21.15.27:0/2282808827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66240caa70 con 0x7f66240c9350 2023-12-19T18:22:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.136+0000 7f662359e640 1 --2- 172.21.15.27:0/2282808827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66240c9350 0x7f66240cc660 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f66240cd300 tx=0x7f661c007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.137+0000 7f661b7fe640 1 -- 172.21.15.27:0/2282808827 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c0046a0 con 0x7f66240c9350 2023-12-19T18:22:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.137+0000 7f661b7fe640 1 -- 172.21.15.27:0/2282808827 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f661c004800 con 0x7f66240c9350 2023-12-19T18:22:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.137+0000 7f661b7fe640 1 -- 172.21.15.27:0/2282808827 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f661c015660 con 0x7f66240c9350 2023-12-19T18:22:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.137+0000 7f6628849640 1 -- 172.21.15.27:0/2282808827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66240cad50 con 0x7f66240c9350 2023-12-19T18:22:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.137+0000 7f6628849640 1 -- 172.21.15.27:0/2282808827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f662414e9c0 con 0x7f66240c9350 2023-12-19T18:22:52.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.138+0000 7f6628849640 1 -- 172.21.15.27:0/2282808827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66240cb270 con 0x7f66240c9350 2023-12-19T18:22:52.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.139+0000 7f661b7fe640 1 -- 172.21.15.27:0/2282808827 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f661c0157c0 con 0x7f66240c9350 2023-12-19T18:22:52.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.140+0000 7f661b7fe640 1 --2- 172.21.15.27:0/2282808827 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f660406a460 0x7f660406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.140+0000 7f661b7fe640 1 -- 172.21.15.27:0/2282808827 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f661c088ed0 con 0x7f66240c9350 2023-12-19T18:22:52.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.141+0000 7f6622d9d640 1 --2- 172.21.15.27:0/2282808827 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f660406a460 0x7f660406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.142+0000 7f6622d9d640 1 --2- 172.21.15.27:0/2282808827 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f660406a460 0x7f660406c920 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6614034000 tx=0x7f661402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:52.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.145+0000 7f661b7fe640 1 -- 172.21.15.27:0/2282808827 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f66240cb270 con 0x7f66240c9350 2023-12-19T18:22:52.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.342+0000 7f6628849640 1 -- 172.21.15.27:0/2282808827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66240cb270 con 0x7f66240c9350 2023-12-19T18:22:52.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.344+0000 7f661b7fe640 1 -- 172.21.15.27:0/2282808827 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16203 (secure 0 0 0) 0x7f661c057f60 con 0x7f66240c9350 2023-12-19T18:22:52.345 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:22:52.345 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":140,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:22:44.753680+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:22:12.328518+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"134","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-19T18:22:13.251708+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"136","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"14.2","mappings":[{"from":5,"to":3}]},{"pgid":"14.5","mappings":[{"from":5,"to":0}]},{"pgid":"14.1f","mappings":[{"from":4,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:22:52.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.347+0000 7f6628849640 1 -- 172.21.15.27:0/2282808827 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f660406a460 msgr2=0x7f660406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:52.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.347+0000 7f6628849640 1 --2- 172.21.15.27:0/2282808827 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f660406a460 0x7f660406c920 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f6614034000 tx=0x7f661402e000 comp rx=0 tx=0).stop 2023-12-19T18:22:52.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.347+0000 7f6628849640 1 -- 172.21.15.27:0/2282808827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66240c9350 msgr2=0x7f66240cc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:52.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.347+0000 7f6628849640 1 --2- 172.21.15.27:0/2282808827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66240c9350 0x7f66240cc660 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f66240cd300 tx=0x7f661c007700 comp rx=0 tx=0).stop 2023-12-19T18:22:52.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.347+0000 7f6628849640 1 -- 172.21.15.27:0/2282808827 shutdown_connections 2023-12-19T18:22:52.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.347+0000 7f6628849640 1 --2- 172.21.15.27:0/2282808827 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f660406a460 0x7f660406c920 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.347+0000 7f6628849640 1 --2- 172.21.15.27:0/2282808827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66240c9350 0x7f66240cc660 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.347+0000 7f6628849640 1 --2- 172.21.15.27:0/2282808827 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f662414e1d0 0x7f66240cb550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.347+0000 7f6628849640 1 --2- 172.21.15.27:0/2282808827 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6610003970 0x7f66240c9e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.347+0000 7f6628849640 1 -- 172.21.15.27:0/2282808827 >> 172.21.15.27:0/2282808827 conn(0x7f66240cf7d0 msgr2=0x7f66241570b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:52.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.347+0000 7f6628849640 1 -- 172.21.15.27:0/2282808827 shutdown_connections 2023-12-19T18:22:52.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.347+0000 7f6628849640 1 -- 172.21.15.27:0/2282808827 wait complete. 2023-12-19T18:22:52.367 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:22:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.606+0000 7f4d83641640 1 Processor -- start 2023-12-19T18:22:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.607+0000 7f4d83641640 1 -- start start 2023-12-19T18:22:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.608+0000 7f4d83641640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4d7c15adf0 0x7f4d7c06d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.608+0000 7f4d83641640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7c06fc10 0x7f4d7c0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.608+0000 7f4d83641640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d7c071820 con 0x7f4d7c070c00 2023-12-19T18:22:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.608+0000 7f4d83641640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d7c0703c0 con 0x7f4d7c15adf0 2023-12-19T18:22:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.608+0000 7f4d83641640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c06d8c0 con 0x7f4d7c06fc10 2023-12-19T18:22:52.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.609+0000 7f4d813dd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4d7c15adf0 0x7f4d7c06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.609+0000 7f4d813dd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4d7c15adf0 0x7f4d7c06d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42058/0 (socket says 172.21.15.27:42058) 2023-12-19T18:22:52.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.609+0000 7f4d813dd640 1 -- 172.21.15.27:0/1200698 learned_addr learned my addr 172.21.15.27:0/1200698 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:52.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.609+0000 7f4d81bde640 1 --2- 172.21.15.27:0/1200698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7c06fc10 0x7f4d7c0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.609+0000 7f4d813dd640 1 -- 172.21.15.27:0/1200698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7c06fc10 msgr2=0x7f4d7c0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:52.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.609+0000 7f4d813dd640 1 --2- 172.21.15.27:0/1200698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7c06fc10 0x7f4d7c0712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.609+0000 7f4d813dd640 1 -- 172.21.15.27:0/1200698 >> v1:172.21.15.192:6789/0 conn(0x7f4d7c070c00 legacy=0x7f4d7c06f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:52.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.609+0000 7f4d813dd640 1 -- 172.21.15.27:0/1200698 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d7c06b520 con 0x7f4d7c15adf0 2023-12-19T18:22:52.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.610+0000 7f4d813dd640 1 --2- 172.21.15.27:0/1200698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7c15adf0 0x7f4d7c06d380 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009e50 tx=0x7f4d7802be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=857ec20273edb0b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:52.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.611+0000 7f4d73fff640 1 -- 172.21.15.27:0/1200698 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7802c9e0 con 0x7f4d7c15adf0 2023-12-19T18:22:52.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.611+0000 7f4d73fff640 1 -- 172.21.15.27:0/1200698 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7802cb40 con 0x7f4d7c15adf0 2023-12-19T18:22:52.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.611+0000 7f4d73fff640 1 -- 172.21.15.27:0/1200698 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d780343f0 con 0x7f4d7c15adf0 2023-12-19T18:22:52.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.611+0000 7f4d83641640 1 -- 172.21.15.27:0/1200698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7c15adf0 msgr2=0x7f4d7c06d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:52.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.611+0000 7f4d83641640 1 --2- 172.21.15.27:0/1200698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7c15adf0 0x7f4d7c06d380 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f4d78009e50 tx=0x7f4d7802be40 comp rx=0 tx=0).stop 2023-12-19T18:22:52.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.612+0000 7f4d83641640 1 -- 172.21.15.27:0/1200698 shutdown_connections 2023-12-19T18:22:52.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.612+0000 7f4d83641640 1 --2- 172.21.15.27:0/1200698 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7c06fc10 0x7f4d7c0712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.612+0000 7f4d83641640 1 --2- 172.21.15.27:0/1200698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7c15adf0 0x7f4d7c06d380 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.612+0000 7f4d83641640 1 -- 172.21.15.27:0/1200698 >> 172.21.15.27:0/1200698 conn(0x7f4d7c0ceb50 msgr2=0x7f4d7c0cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:52.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.612+0000 7f4d83641640 1 -- 172.21.15.27:0/1200698 shutdown_connections 2023-12-19T18:22:52.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.612+0000 7f4d83641640 1 -- 172.21.15.27:0/1200698 wait complete. 2023-12-19T18:22:52.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.614+0000 7f4d83641640 1 Processor -- start 2023-12-19T18:22:52.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.614+0000 7f4d83641640 1 -- start start 2023-12-19T18:22:52.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.616+0000 7f4d83641640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7c06fc10 0x7f4d7c157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.616+0000 7f4d83641640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d7c070c00 0x7f4d7c0c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.616+0000 7f4d83641640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7c15adf0 0x7f4d7c0b2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.616+0000 7f4d83641640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c159ce0 con 0x7f4d7c15adf0 2023-12-19T18:22:52.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.616+0000 7f4d83641640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c158df0 con 0x7f4d7c070c00 2023-12-19T18:22:52.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.616+0000 7f4d83641640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7c1590d0 con 0x7f4d7c06fc10 2023-12-19T18:22:52.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.616+0000 7f4d81bde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7c15adf0 0x7f4d7c0b2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.616+0000 7f4d81bde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7c15adf0 0x7f4d7c0b2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42074/0 (socket says 172.21.15.27:42074) 2023-12-19T18:22:52.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.616+0000 7f4d81bde640 1 -- 172.21.15.27:0/522044068 learned_addr learned my addr 172.21.15.27:0/522044068 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:52.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.617+0000 7f4d813dd640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7c06fc10 0x7f4d7c157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.617+0000 7f4d80bdc640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d7c070c00 0x7f4d7c0c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.617+0000 7f4d81bde640 1 -- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7c06fc10 msgr2=0x7f4d7c157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:52.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.617+0000 7f4d81bde640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7c06fc10 0x7f4d7c157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.617+0000 7f4d81bde640 1 -- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d7c070c00 msgr2=0x7f4d7c0c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:52.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.617+0000 7f4d81bde640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d7c070c00 0x7f4d7c0c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.617+0000 7f4d81bde640 1 -- 172.21.15.27:0/522044068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d78009b30 con 0x7f4d7c15adf0 2023-12-19T18:22:52.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.617+0000 7f4d813dd640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7c06fc10 0x7f4d7c157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:52.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.617+0000 7f4d80bdc640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d7c070c00 0x7f4d7c0c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:52.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.617+0000 7f4d81bde640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7c15adf0 0x7f4d7c0b2e70 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400d5d0 tx=0x7f4d7400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:52.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.617+0000 7f4d71ffb640 1 -- 172.21.15.27:0/522044068 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d740125f0 con 0x7f4d7c15adf0 2023-12-19T18:22:52.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.618+0000 7f4d71ffb640 1 -- 172.21.15.27:0/522044068 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d74012750 con 0x7f4d7c15adf0 2023-12-19T18:22:52.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.618+0000 7f4d71ffb640 1 -- 172.21.15.27:0/522044068 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74012b00 con 0x7f4d7c15adf0 2023-12-19T18:22:52.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.618+0000 7f4d83641640 1 -- 172.21.15.27:0/522044068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7c1593b0 con 0x7f4d7c15adf0 2023-12-19T18:22:52.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.618+0000 7f4d83641640 1 -- 172.21.15.27:0/522044068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7c159970 con 0x7f4d7c15adf0 2023-12-19T18:22:52.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.619+0000 7f4d83641640 1 -- 172.21.15.27:0/522044068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d7c159e50 con 0x7f4d7c15adf0 2023-12-19T18:22:52.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.620+0000 7f4d71ffb640 1 -- 172.21.15.27:0/522044068 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4d74012c60 con 0x7f4d7c15adf0 2023-12-19T18:22:52.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.621+0000 7f4d71ffb640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4d6006a430 0x7f4d6006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:52.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.621+0000 7f4d71ffb640 1 -- 172.21.15.27:0/522044068 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 5796+0+0 (secure 0 0 0) 0x7f4d74088ee0 con 0x7f4d7c15adf0 2023-12-19T18:22:52.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.622+0000 7f4d813dd640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4d6006a430 0x7f4d6006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:52.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.623+0000 7f4d813dd640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4d6006a430 0x7f4d6006c8f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4d78035000 tx=0x7f4d78030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:52.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.626+0000 7f4d71ffb640 1 -- 172.21.15.27:0/522044068 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4d740543d0 con 0x7f4d7c15adf0 2023-12-19T18:22:52.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.860+0000 7f4d83641640 1 -- 172.21.15.27:0/522044068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4d7c068e90 con 0x7f4d7c15adf0 2023-12-19T18:22:52.862 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:22:52.862+0000 7f8f3865c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:22:52.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.878+0000 7f4d71ffb640 1 -- 172.21.15.27:0/522044068 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4d740579f0 con 0x7f4d7c15adf0 2023-12-19T18:22:52.879 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:22:52.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.882+0000 7f4d83641640 1 -- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4d6006a430 msgr2=0x7f4d6006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:52.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.882+0000 7f4d83641640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4d6006a430 0x7f4d6006c8f0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4d78035000 tx=0x7f4d78030000 comp rx=0 tx=0).stop 2023-12-19T18:22:52.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.882+0000 7f4d83641640 1 -- 172.21.15.27:0/522044068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7c15adf0 msgr2=0x7f4d7c0b2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:52.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.882+0000 7f4d83641640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7c15adf0 0x7f4d7c0b2e70 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400d5d0 tx=0x7f4d7400da90 comp rx=0 tx=0).stop 2023-12-19T18:22:52.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.882+0000 7f4d83641640 1 -- 172.21.15.27:0/522044068 shutdown_connections 2023-12-19T18:22:52.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.882+0000 7f4d83641640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4d6006a430 0x7f4d6006c8f0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.882+0000 7f4d83641640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d7c15adf0 0x7f4d7c0b2e70 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.882+0000 7f4d83641640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4d7c070c00 0x7f4d7c0c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.882+0000 7f4d83641640 1 --2- 172.21.15.27:0/522044068 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4d7c06fc10 0x7f4d7c157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:52.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.882+0000 7f4d83641640 1 -- 172.21.15.27:0/522044068 >> 172.21.15.27:0/522044068 conn(0x7f4d7c0ceb50 msgr2=0x7f4d7c0cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:52.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.882+0000 7f4d83641640 1 -- 172.21.15.27:0/522044068 shutdown_connections 2023-12-19T18:22:52.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:52.882+0000 7f4d83641640 1 -- 172.21.15.27:0/522044068 wait complete. 2023-12-19T18:22:52.887 INFO:tasks.ceph.mds.c.smithi027.stderr: -3781> 2023-12-19T18:22:19.511+0000 7fc78cfa1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:22:52.906 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:22:52.959 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:22:53.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.115+0000 7f7036ce5640 1 Processor -- start 2023-12-19T18:22:53.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.115+0000 7f7036ce5640 1 -- start start 2023-12-19T18:22:53.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.117+0000 7f7036ce5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f703015dc00 0x7f70300cc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:53.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.117+0000 7f7036ce5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f70300cb840 0x7f70300ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:53.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.117+0000 7f7036ce5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7030160920 con 0x7f703015dfe0 2023-12-19T18:22:53.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.117+0000 7f7036ce5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70300c9fa0 con 0x7f703015dc00 2023-12-19T18:22:53.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.117+0000 7f7036ce5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70300cd220 con 0x7f70300cb840 2023-12-19T18:22:53.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.117+0000 7f7035ce3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f703015dc00 0x7f70300cc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:53.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.117+0000 7f7035ce3640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f703015dc00 0x7f70300cc6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42112/0 (socket says 172.21.15.27:42112) 2023-12-19T18:22:53.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.118+0000 7f7035ce3640 1 -- 172.21.15.27:0/230775088 learned_addr learned my addr 172.21.15.27:0/230775088 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:53.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.118+0000 7f70364e4640 1 --2- 172.21.15.27:0/230775088 >> v2:172.21.15.192:3300/0 conn(0x7f70300cb840 0x7f70300ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:53.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.118+0000 7f7035ce3640 1 -- 172.21.15.27:0/230775088 >> v2:172.21.15.192:3300/0 conn(0x7f70300cb840 msgr2=0x7f70300ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:53.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.118+0000 7f7035ce3640 1 --2- 172.21.15.27:0/230775088 >> v2:172.21.15.192:3300/0 conn(0x7f70300cb840 0x7f70300ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:53.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.118+0000 7f7035ce3640 1 -- 172.21.15.27:0/230775088 >> v1:172.21.15.192:6789/0 conn(0x7f703015dfe0 legacy=0x7f70300cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:53.119 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.118+0000 7f7035ce3640 1 -- 172.21.15.27:0/230775088 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7030160be0 con 0x7f703015dc00 2023-12-19T18:22:53.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.120+0000 7f7035ce3640 1 --2- 172.21.15.27:0/230775088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f703015dc00 0x7f70300cc6d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f70300ce380 tx=0x7f702c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f226265204a6d439 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:53.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.120+0000 7f7034ce1640 1 -- 172.21.15.27:0/230775088 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c0317e0 con 0x7f703015dc00 2023-12-19T18:22:53.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.120+0000 7f7034ce1640 1 -- 172.21.15.27:0/230775088 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c031940 con 0x7f703015dc00 2023-12-19T18:22:53.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.120+0000 7f7034ce1640 1 -- 172.21.15.27:0/230775088 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c031cd0 con 0x7f703015dc00 2023-12-19T18:22:53.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.121+0000 7f7036ce5640 1 -- 172.21.15.27:0/230775088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f703015dc00 msgr2=0x7f70300cc6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:53.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.121+0000 7f7036ce5640 1 --2- 172.21.15.27:0/230775088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f703015dc00 0x7f70300cc6d0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f70300ce380 tx=0x7f702c02b880 comp rx=0 tx=0).stop 2023-12-19T18:22:53.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.121+0000 7f7036ce5640 1 -- 172.21.15.27:0/230775088 shutdown_connections 2023-12-19T18:22:53.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.121+0000 7f7036ce5640 1 --2- 172.21.15.27:0/230775088 >> v2:172.21.15.192:3300/0 conn(0x7f70300cb840 0x7f70300ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:53.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.121+0000 7f7036ce5640 1 --2- 172.21.15.27:0/230775088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f703015dc00 0x7f70300cc6d0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:53.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.121+0000 7f7036ce5640 1 -- 172.21.15.27:0/230775088 >> 172.21.15.27:0/230775088 conn(0x7f70300b05b0 msgr2=0x7f70300b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:53.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.121+0000 7f7036ce5640 1 -- 172.21.15.27:0/230775088 shutdown_connections 2023-12-19T18:22:53.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.121+0000 7f7036ce5640 1 -- 172.21.15.27:0/230775088 wait complete. 2023-12-19T18:22:53.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.123+0000 7f7036ce5640 1 Processor -- start 2023-12-19T18:22:53.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.123+0000 7f7036ce5640 1 -- start start 2023-12-19T18:22:53.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.124+0000 7f7036ce5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70300cb840 0x7f70301f68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:53.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.124+0000 7f7036ce5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f703015dc00 0x7f703015a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:53.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.124+0000 7f7036ce5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f703015dfe0 0x7f70301f2930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:53.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.124+0000 7f7036ce5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70301f38d0 con 0x7f70300cb840 2023-12-19T18:22:53.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.124+0000 7f7036ce5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70301f6e00 con 0x7f703015dc00 2023-12-19T18:22:53.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.124+0000 7f7036ce5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70301f3310 con 0x7f703015dfe0 2023-12-19T18:22:53.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.124+0000 7f7035ce3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70300cb840 0x7f70301f68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:53.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.125+0000 7f7035ce3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70300cb840 0x7f70301f68c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42124/0 (socket says 172.21.15.27:42124) 2023-12-19T18:22:53.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.125+0000 7f7035ce3640 1 -- 172.21.15.27:0/358144289 learned_addr learned my addr 172.21.15.27:0/358144289 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:53.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.125+0000 7f70354e2640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f703015dc00 0x7f703015a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:53.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.125+0000 7f7035ce3640 1 -- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f703015dfe0 msgr2=0x7f70301f2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:53.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.125+0000 7f7035ce3640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f703015dfe0 0x7f70301f2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:53.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.125+0000 7f7035ce3640 1 -- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f703015dc00 msgr2=0x7f703015a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:53.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.125+0000 7f7035ce3640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f703015dc00 0x7f703015a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:53.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.125+0000 7f7035ce3640 1 -- 172.21.15.27:0/358144289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f702c0094d0 con 0x7f70300cb840 2023-12-19T18:22:53.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.125+0000 7f70354e2640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f703015dc00 0x7f703015a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:53.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.125+0000 7f7035ce3640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70300cb840 0x7f70301f68c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f702c02b8c0 tx=0x7f702c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:53.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.126+0000 7f70267fc640 1 -- 172.21.15.27:0/358144289 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c032880 con 0x7f70300cb840 2023-12-19T18:22:53.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.126+0000 7f70267fc640 1 -- 172.21.15.27:0/358144289 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f702c032a00 con 0x7f70300cb840 2023-12-19T18:22:53.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.126+0000 7f7036ce5640 1 -- 172.21.15.27:0/358144289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70301f35f0 con 0x7f70300cb840 2023-12-19T18:22:53.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.126+0000 7f70267fc640 1 -- 172.21.15.27:0/358144289 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f702c032db0 con 0x7f70300cb840 2023-12-19T18:22:53.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.126+0000 7f7036ce5640 1 -- 172.21.15.27:0/358144289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70301f63a0 con 0x7f70300cb840 2023-12-19T18:22:53.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.127+0000 7f7036ce5640 1 -- 172.21.15.27:0/358144289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70301f5f20 con 0x7f70300cb840 2023-12-19T18:22:53.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.128+0000 7f70267fc640 1 -- 172.21.15.27:0/358144289 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f702c04e450 con 0x7f70300cb840 2023-12-19T18:22:53.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.129+0000 7f70267fc640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f701406a430 0x7f701406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:53.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.129+0000 7f70267fc640 1 -- 172.21.15.27:0/358144289 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f702c0aab00 con 0x7f70300cb840 2023-12-19T18:22:53.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.130+0000 7f70354e2640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f701406a430 0x7f701406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:53.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.131+0000 7f70354e2640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f701406a430 0x7f701406c8f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f703004e1f0 tx=0x7f7020009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:53.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.134+0000 7f70267fc640 1 -- 172.21.15.27:0/358144289 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f70301f5f20 con 0x7f70300cb840 2023-12-19T18:22:53.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.364+0000 7f7036ce5640 1 -- 172.21.15.27:0/358144289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f70301f5f20 con 0x7f70300cb840 2023-12-19T18:22:53.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.881+0000 7f70267fc640 1 -- 172.21.15.27:0/358144289 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7f70301f5f20 con 0x7f70300cb840 2023-12-19T18:22:53.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.884+0000 7f7036ce5640 1 -- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f701406a430 msgr2=0x7f701406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:53.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.884+0000 7f7036ce5640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f701406a430 0x7f701406c8f0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f703004e1f0 tx=0x7f7020009040 comp rx=0 tx=0).stop 2023-12-19T18:22:53.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.885+0000 7f7036ce5640 1 -- 172.21.15.27:0/358144289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70300cb840 msgr2=0x7f70301f68c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:53.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.885+0000 7f7036ce5640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70300cb840 0x7f70301f68c0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f702c02b8c0 tx=0x7f702c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:22:53.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.885+0000 7f7036ce5640 1 -- 172.21.15.27:0/358144289 shutdown_connections 2023-12-19T18:22:53.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.885+0000 7f7036ce5640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f701406a430 0x7f701406c8f0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:53.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.885+0000 7f7036ce5640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f703015dfe0 0x7f70301f2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:53.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.885+0000 7f7036ce5640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f703015dc00 0x7f703015a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:53.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.885+0000 7f7036ce5640 1 --2- 172.21.15.27:0/358144289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70300cb840 0x7f70301f68c0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:53.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.885+0000 7f7036ce5640 1 -- 172.21.15.27:0/358144289 >> 172.21.15.27:0/358144289 conn(0x7f70300b05b0 msgr2=0x7f703015b760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:53.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.885+0000 7f7036ce5640 1 -- 172.21.15.27:0/358144289 shutdown_connections 2023-12-19T18:22:53.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:53.885+0000 7f7036ce5640 1 -- 172.21.15.27:0/358144289 wait complete. 2023-12-19T18:22:53.907 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:22:54.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.147+0000 7f9c34df0640 1 Processor -- start 2023-12-19T18:22:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.147+0000 7f9c34df0640 1 -- start start 2023-12-19T18:22:54.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.149+0000 7f9c34df0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9c3015d4d0 0x7f9c300cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:54.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.149+0000 7f9c34df0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3006bea0 con 0x7f9c300ac8d0 2023-12-19T18:22:54.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.149+0000 7f9c34df0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c3015d8b0 con 0x7f9c300cd050 2023-12-19T18:22:54.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.149+0000 7f9c34df0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c3006ccb0 con 0x7f9c3015d4d0 2023-12-19T18:22:54.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.149+0000 7f9c25d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9c300ac8d0 0x7f9c300cb5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60430/0 (socket says 172.21.15.27:60430) 2023-12-19T18:22:54.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.149+0000 7f9c25d9b640 1 -- 172.21.15.27:0/2738696951 learned_addr learned my addr 172.21.15.27:0/2738696951 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:54.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.150+0000 7f9c2e59c640 1 --2- 172.21.15.27:0/2738696951 >> v2:172.21.15.192:3300/0 conn(0x7f9c3015d4d0 0x7f9c300cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:54.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.150+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 295992735 0 0) 0x7f9c3006bea0 con 0x7f9c300ac8d0 2023-12-19T18:22:54.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.150+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9c1c003520 con 0x7f9c300ac8d0 2023-12-19T18:22:54.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 325235049 0 0) 0x7f9c1c003520 con 0x7f9c300ac8d0 2023-12-19T18:22:54.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 >> v2:172.21.15.192:3300/0 conn(0x7f9c3015d4d0 msgr2=0x7f9c300cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:54.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2dd9b640 1 --2- 172.21.15.27:0/2738696951 >> v2:172.21.15.192:3300/0 conn(0x7f9c3015d4d0 0x7f9c300cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:54.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 >> v1:172.21.15.192:6789/0 conn(0x7f9c300cd050 legacy=0x7f9c300c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:54.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3006c120 con 0x7f9c300ac8d0 2023-12-19T18:22:54.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f9c18001670 con 0x7f9c300ac8d0 2023-12-19T18:22:54.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 >> v1:172.21.15.27:6789/0 conn(0x7f9c300ac8d0 legacy=0x7f9c300cb5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:54.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2dd9b640 1 --2- 172.21.15.27:0/2738696951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c1c003970 0x7f9c1c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:54.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2dd9b640 1 --2- 172.21.15.27:0/2738696951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c1c0047c0 0x7f9c1c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:54.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2dd9b640 1 --2- 172.21.15.27:0/2738696951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c1c005180 0x7f9c1c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:54.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3006ccb0 con 0x7f9c1c0047c0 2023-12-19T18:22:54.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c00c030 con 0x7f9c1c003970 2023-12-19T18:22:54.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1c00c1f0 con 0x7f9c1c005180 2023-12-19T18:22:54.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c25d9b640 1 --2- 172.21.15.27:0/2738696951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c1c0047c0 0x7f9c1c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:54.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.151+0000 7f9c2e59c640 1 --2- 172.21.15.27:0/2738696951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c1c003970 0x7f9c1c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:54.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.152+0000 7f9c2ed9d640 1 --2- 172.21.15.27:0/2738696951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c1c005180 0x7f9c1c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:54.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.152+0000 7f9c25d9b640 1 -- 172.21.15.27:0/2738696951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c1c005180 msgr2=0x7f9c1c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:54.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.152+0000 7f9c25d9b640 1 --2- 172.21.15.27:0/2738696951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c1c005180 0x7f9c1c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:54.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.152+0000 7f9c25d9b640 1 -- 172.21.15.27:0/2738696951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c1c003970 msgr2=0x7f9c1c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:54.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.152+0000 7f9c25d9b640 1 --2- 172.21.15.27:0/2738696951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c1c003970 0x7f9c1c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:54.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.152+0000 7f9c25d9b640 1 -- 172.21.15.27:0/2738696951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c1c003520 con 0x7f9c1c0047c0 2023-12-19T18:22:54.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.152+0000 7f9c2ed9d640 1 -- 172.21.15.27:0/2738696951 reap_dead start 2023-12-19T18:22:54.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.153+0000 7f9c25d9b640 1 --2- 172.21.15.27:0/2738696951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c1c0047c0 0x7f9c1c004c40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800c210 tx=0x7f9c1c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d67eaeff542ec0dd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:54.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.153+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c18005ad0 con 0x7f9c1c0047c0 2023-12-19T18:22:54.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.153+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c18005c30 con 0x7f9c1c0047c0 2023-12-19T18:22:54.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.153+0000 7f9c2dd9b640 1 -- 172.21.15.27:0/2738696951 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c1802cad0 con 0x7f9c1c0047c0 2023-12-19T18:22:54.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.154+0000 7f9c34df0640 1 -- 172.21.15.27:0/2738696951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c1c0047c0 msgr2=0x7f9c1c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:54.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.154+0000 7f9c34df0640 1 --2- 172.21.15.27:0/2738696951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c1c0047c0 0x7f9c1c004c40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f9c1800c210 tx=0x7f9c1c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:54.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.154+0000 7f9c34df0640 1 -- 172.21.15.27:0/2738696951 shutdown_connections 2023-12-19T18:22:54.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.154+0000 7f9c34df0640 1 --2- 172.21.15.27:0/2738696951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c1c0047c0 0x7f9c1c004c40 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.154+0000 7f9c34df0640 1 -- 172.21.15.27:0/2738696951 >> 172.21.15.27:0/2738696951 conn(0x7f9c300b0620 msgr2=0x7f9c300b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.154+0000 7f9c34df0640 1 -- 172.21.15.27:0/2738696951 shutdown_connections 2023-12-19T18:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.154+0000 7f9c34df0640 1 -- 172.21.15.27:0/2738696951 wait complete. 2023-12-19T18:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.156+0000 7f9c34df0640 1 Processor -- start 2023-12-19T18:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.156+0000 7f9c34df0640 1 -- start start 2023-12-19T18:22:54.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.157+0000 7f9c34df0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c1c0047c0 0x7f9c301538e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:54.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.157+0000 7f9c34df0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c301f3f50 0x7f9c3015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:54.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.157+0000 7f9c34df0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c300c5b40 0x7f9c300ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:54.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.157+0000 7f9c34df0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c3015c300 con 0x7f9c1c0047c0 2023-12-19T18:22:54.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.157+0000 7f9c34df0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c301f44f0 con 0x7f9c300c5b40 2023-12-19T18:22:54.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.157+0000 7f9c34df0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c301f47d0 con 0x7f9c301f3f50 2023-12-19T18:22:54.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.157+0000 7f9c2e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c1c0047c0 0x7f9c301538e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:54.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.158+0000 7f9c2e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c1c0047c0 0x7f9c301538e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42144/0 (socket says 172.21.15.27:42144) 2023-12-19T18:22:54.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.158+0000 7f9c2e59c640 1 -- 172.21.15.27:0/2217252901 learned_addr learned my addr 172.21.15.27:0/2217252901 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:54.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.158+0000 7f9c2ed9d640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c300c5b40 0x7f9c300ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:54.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.158+0000 7f9c25d9b640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c301f3f50 0x7f9c3015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:54.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.158+0000 7f9c2e59c640 1 -- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c301f3f50 msgr2=0x7f9c3015bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:54.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.158+0000 7f9c2e59c640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c301f3f50 0x7f9c3015bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:54.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.158+0000 7f9c2e59c640 1 -- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c300c5b40 msgr2=0x7f9c300ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:54.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.158+0000 7f9c2e59c640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c300c5b40 0x7f9c300ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:54.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.158+0000 7f9c2e59c640 1 -- 172.21.15.27:0/2217252901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c301f4ab0 con 0x7f9c1c0047c0 2023-12-19T18:22:54.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.158+0000 7f9c2ed9d640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c300c5b40 0x7f9c300ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:54.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.158+0000 7f9c2e59c640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c1c0047c0 0x7f9c301538e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009f30 tx=0x7f9c200115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:54.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.159+0000 7f9c277fe640 1 -- 172.21.15.27:0/2217252901 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2000e030 con 0x7f9c1c0047c0 2023-12-19T18:22:54.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.159+0000 7f9c277fe640 1 -- 172.21.15.27:0/2217252901 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c20016470 con 0x7f9c1c0047c0 2023-12-19T18:22:54.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.159+0000 7f9c277fe640 1 -- 172.21.15.27:0/2217252901 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c20016820 con 0x7f9c1c0047c0 2023-12-19T18:22:54.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.159+0000 7f9c34df0640 1 -- 172.21.15.27:0/2217252901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c301f4d90 con 0x7f9c1c0047c0 2023-12-19T18:22:54.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.159+0000 7f9c34df0640 1 -- 172.21.15.27:0/2217252901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c300c5fc0 con 0x7f9c1c0047c0 2023-12-19T18:22:54.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.160+0000 7f9c34df0640 1 -- 172.21.15.27:0/2217252901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c300ccec0 con 0x7f9c1c0047c0 2023-12-19T18:22:54.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.161+0000 7f9c277fe640 1 -- 172.21.15.27:0/2217252901 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9c20016980 con 0x7f9c1c0047c0 2023-12-19T18:22:54.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.163+0000 7f9c277fe640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c1006a430 0x7f9c1006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:54.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.163+0000 7f9c277fe640 1 -- 172.21.15.27:0/2217252901 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 5882+0+0 (secure 0 0 0) 0x7f9c2000f070 con 0x7f9c1c0047c0 2023-12-19T18:22:54.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.163+0000 7f9c25d9b640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c1006a430 0x7f9c1006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:54.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.164+0000 7f9c25d9b640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c1006a430 0x7f9c1006c8f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9c18034000 tx=0x7f9c18009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:54.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.167+0000 7f9c277fe640 1 -- 172.21.15.27:0/2217252901 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9c20054b40 con 0x7f9c1c0047c0 2023-12-19T18:22:54.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.364+0000 7f9c34df0640 1 -- 172.21.15.27:0/2217252901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9c30068e90 con 0x7f9c1c0047c0 2023-12-19T18:22:54.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.894+0000 7f9c277fe640 1 -- 172.21.15.27:0/2217252901 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v142) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9c20058160 con 0x7f9c1c0047c0 2023-12-19T18:22:54.894 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:22:54.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.899+0000 7f9c34df0640 1 -- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c1006a430 msgr2=0x7f9c1006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:54.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.899+0000 7f9c34df0640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c1006a430 0x7f9c1006c8f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f9c18034000 tx=0x7f9c18009030 comp rx=0 tx=0).stop 2023-12-19T18:22:54.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.899+0000 7f9c34df0640 1 -- 172.21.15.27:0/2217252901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c1c0047c0 msgr2=0x7f9c301538e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:54.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.899+0000 7f9c34df0640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c1c0047c0 0x7f9c301538e0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009f30 tx=0x7f9c200115a0 comp rx=0 tx=0).stop 2023-12-19T18:22:54.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.899+0000 7f9c34df0640 1 -- 172.21.15.27:0/2217252901 shutdown_connections 2023-12-19T18:22:54.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.899+0000 7f9c34df0640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9c1006a430 0x7f9c1006c8f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:54.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.899+0000 7f9c34df0640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9c300c5b40 0x7f9c300ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:54.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.899+0000 7f9c34df0640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9c301f3f50 0x7f9c3015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:54.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.899+0000 7f9c34df0640 1 --2- 172.21.15.27:0/2217252901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c1c0047c0 0x7f9c301538e0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:54.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.899+0000 7f9c34df0640 1 -- 172.21.15.27:0/2217252901 >> 172.21.15.27:0/2217252901 conn(0x7f9c300b0620 msgr2=0x7f9c3006b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:54.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.900+0000 7f9c34df0640 1 -- 172.21.15.27:0/2217252901 shutdown_connections 2023-12-19T18:22:54.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:54.900+0000 7f9c34df0640 1 -- 172.21.15.27:0/2217252901 wait complete. 2023-12-19T18:22:54.921 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:22:55.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.158+0000 7fe57b2b4640 1 Processor -- start 2023-12-19T18:22:55.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.158+0000 7fe57b2b4640 1 -- start start 2023-12-19T18:22:55.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.160+0000 7fe57b2b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5740ade20 0x7fe57415b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:55.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.160+0000 7fe57b2b4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe57415b810 0x7fe5740c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:55.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.160+0000 7fe57b2b4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe57415c030 con 0x7fe574158790 2023-12-19T18:22:55.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.160+0000 7fe57b2b4640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5740c99d0 con 0x7fe57415b810 2023-12-19T18:22:55.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.160+0000 7fe57b2b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe574158b70 con 0x7fe5740ade20 2023-12-19T18:22:55.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.160+0000 7fe57a2b2640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe574158790 0x7fe5740ad710 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60432/0 (socket says 172.21.15.27:60432) 2023-12-19T18:22:55.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.160+0000 7fe57a2b2640 1 -- 172.21.15.27:0/2116499466 learned_addr learned my addr 172.21.15.27:0/2116499466 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:55.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.161+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977181223 0 0) 0x7fe57415c030 con 0x7fe574158790 2023-12-19T18:22:55.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.161+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe560003520 con 0x7fe574158790 2023-12-19T18:22:55.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.161+0000 7fe57aab3640 1 --2- 172.21.15.27:0/2116499466 >> v2:172.21.15.192:3300/0 conn(0x7fe57415b810 0x7fe5740c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:55.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.161+0000 7fe579ab1640 1 --2- 172.21.15.27:0/2116499466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5740ade20 0x7fe57415b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:55.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.161+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1071413596 0 0) 0x7fe560003520 con 0x7fe574158790 2023-12-19T18:22:55.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.161+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5740ade20 msgr2=0x7fe57415b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:55.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.161+0000 7fe5792b0640 1 --2- 172.21.15.27:0/2116499466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5740ade20 0x7fe57415b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:55.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.161+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 >> v2:172.21.15.192:3300/0 conn(0x7fe57415b810 msgr2=0x7fe5740c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:55.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.161+0000 7fe5792b0640 1 --2- 172.21.15.27:0/2116499466 >> v2:172.21.15.192:3300/0 conn(0x7fe57415b810 0x7fe5740c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:55.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.162+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5740c9030 con 0x7fe574158790 2023-12-19T18:22:55.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.162+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe570001670 con 0x7fe574158790 2023-12-19T18:22:55.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.162+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 >> v1:172.21.15.27:6789/0 conn(0x7fe574158790 legacy=0x7fe5740ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:55.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.162+0000 7fe5792b0640 1 --2- 172.21.15.27:0/2116499466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe560003970 0x7fe560003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:55.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.162+0000 7fe5792b0640 1 --2- 172.21.15.27:0/2116499466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5740ade20 0x7fe5600047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:55.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.162+0000 7fe5792b0640 1 --2- 172.21.15.27:0/2116499466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe560004d00 0x7fe560005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:55.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.162+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740c99d0 con 0x7fe560003970 2023-12-19T18:22:55.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.162+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe574158b70 con 0x7fe560004d00 2023-12-19T18:22:55.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.162+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe56000b980 con 0x7fe5740ade20 2023-12-19T18:22:55.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.162+0000 7fe57a2b2640 1 --2- 172.21.15.27:0/2116499466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe560003970 0x7fe560003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:55.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.162+0000 7fe579ab1640 1 --2- 172.21.15.27:0/2116499466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5740ade20 0x7fe5600047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:55.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.162+0000 7fe57aab3640 1 --2- 172.21.15.27:0/2116499466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe560004d00 0x7fe560005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:55.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.163+0000 7fe57a2b2640 1 -- 172.21.15.27:0/2116499466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5740ade20 msgr2=0x7fe5600047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:55.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.163+0000 7fe57a2b2640 1 --2- 172.21.15.27:0/2116499466 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5740ade20 0x7fe5600047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:55.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.163+0000 7fe57a2b2640 1 -- 172.21.15.27:0/2116499466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe560004d00 msgr2=0x7fe560005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:55.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.163+0000 7fe57a2b2640 1 --2- 172.21.15.27:0/2116499466 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe560004d00 0x7fe560005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:55.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.163+0000 7fe57a2b2640 1 -- 172.21.15.27:0/2116499466 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe560003520 con 0x7fe560003970 2023-12-19T18:22:55.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.164+0000 7fe57a2b2640 1 --2- 172.21.15.27:0/2116499466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe560003970 0x7fe560003d90 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe57000c1c0 tx=0x7fe5600057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a9076ebba9e9820 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:55.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.164+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe570005ad0 con 0x7fe560003970 2023-12-19T18:22:55.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.164+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe570005c50 con 0x7fe560003970 2023-12-19T18:22:55.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.164+0000 7fe5792b0640 1 -- 172.21.15.27:0/2116499466 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe57002c810 con 0x7fe560003970 2023-12-19T18:22:55.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.164+0000 7fe57b2b4640 1 -- 172.21.15.27:0/2116499466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe560003970 msgr2=0x7fe560003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:55.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.164+0000 7fe57b2b4640 1 --2- 172.21.15.27:0/2116499466 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe560003970 0x7fe560003d90 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fe57000c1c0 tx=0x7fe5600057f0 comp rx=0 tx=0).stop 2023-12-19T18:22:55.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.165+0000 7fe57aab3640 1 -- 172.21.15.27:0/2116499466 reap_dead start 2023-12-19T18:22:55.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.165+0000 7fe57b2b4640 1 -- 172.21.15.27:0/2116499466 shutdown_connections 2023-12-19T18:22:55.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.165+0000 7fe57b2b4640 1 -- 172.21.15.27:0/2116499466 >> 172.21.15.27:0/2116499466 conn(0x7fe5740b05b0 msgr2=0x7fe5740b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:55.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.165+0000 7fe57b2b4640 1 -- 172.21.15.27:0/2116499466 shutdown_connections 2023-12-19T18:22:55.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.165+0000 7fe57b2b4640 1 -- 172.21.15.27:0/2116499466 wait complete. 2023-12-19T18:22:55.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.167+0000 7fe57b2b4640 1 Processor -- start 2023-12-19T18:22:55.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.167+0000 7fe57b2b4640 1 -- start start 2023-12-19T18:22:55.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.168+0000 7fe57b2b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560003970 0x7fe57406d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:55.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.168+0000 7fe57b2b4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe574156e50 0x7fe5740c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:55.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.168+0000 7fe57b2b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740af250 0x7fe5740af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:55.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.168+0000 7fe57b2b4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740b2740 con 0x7fe5740af250 2023-12-19T18:22:55.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.168+0000 7fe57b2b4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740afc10 con 0x7fe574156e50 2023-12-19T18:22:55.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.168+0000 7fe57b2b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5740c6240 con 0x7fe560003970 2023-12-19T18:22:55.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.168+0000 7fe57aab3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740af250 0x7fe5740af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:55.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.168+0000 7fe57aab3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740af250 0x7fe5740af6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42156/0 (socket says 172.21.15.27:42156) 2023-12-19T18:22:55.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.168+0000 7fe57aab3640 1 -- 172.21.15.27:0/3480828830 learned_addr learned my addr 172.21.15.27:0/3480828830 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:55.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.168+0000 7fe57a2b2640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560003970 0x7fe57406d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:55.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.168+0000 7fe579ab1640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe574156e50 0x7fe5740c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:55.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.169+0000 7fe57aab3640 1 -- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560003970 msgr2=0x7fe57406d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:55.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.169+0000 7fe57aab3640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560003970 0x7fe57406d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:55.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.169+0000 7fe57aab3640 1 -- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe574156e50 msgr2=0x7fe5740c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:55.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.169+0000 7fe57aab3640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe574156e50 0x7fe5740c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:55.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.169+0000 7fe57aab3640 1 -- 172.21.15.27:0/3480828830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5740c6520 con 0x7fe5740af250 2023-12-19T18:22:55.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.169+0000 7fe579ab1640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe574156e50 0x7fe5740c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:55.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.169+0000 7fe57aab3640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740af250 0x7fe5740af6d0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe5740c4b30 tx=0x7fe56c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:55.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.169+0000 7fe56affd640 1 -- 172.21.15.27:0/3480828830 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c00e070 con 0x7fe5740af250 2023-12-19T18:22:55.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.169+0000 7fe56affd640 1 -- 172.21.15.27:0/3480828830 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56c011e10 con 0x7fe5740af250 2023-12-19T18:22:55.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.169+0000 7fe57b2b4640 1 -- 172.21.15.27:0/3480828830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5740c6800 con 0x7fe5740af250 2023-12-19T18:22:55.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.169+0000 7fe56affd640 1 -- 172.21.15.27:0/3480828830 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56c003990 con 0x7fe5740af250 2023-12-19T18:22:55.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.169+0000 7fe57b2b4640 1 -- 172.21.15.27:0/3480828830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5740c6ae0 con 0x7fe5740af250 2023-12-19T18:22:55.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.171+0000 7fe57b2b4640 1 -- 172.21.15.27:0/3480828830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe57404e840 con 0x7fe5740af250 2023-12-19T18:22:55.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.172+0000 7fe56affd640 1 -- 172.21.15.27:0/3480828830 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe56c003af0 con 0x7fe5740af250 2023-12-19T18:22:55.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.173+0000 7fe56affd640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe55806a430 0x7fe55806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:55.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.173+0000 7fe56affd640 1 -- 172.21.15.27:0/3480828830 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(142..142 src has 1..142) v4 ==== 5357+0+0 (secure 0 0 0) 0x7fe56c088ec0 con 0x7fe5740af250 2023-12-19T18:22:55.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.173+0000 7fe57a2b2640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe55806a430 0x7fe55806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:55.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.175+0000 7fe57a2b2640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe55806a430 0x7fe55806c8f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe570034000 tx=0x7fe570007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:55.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.177+0000 7fe56affd640 1 -- 172.21.15.27:0/3480828830 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe56c054570 con 0x7fe5740af250 2023-12-19T18:22:55.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.372+0000 7fe57b2b4640 1 -- 172.21.15.27:0/3480828830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe5740c92e0 con 0x7fe5740af250 2023-12-19T18:22:55.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.906+0000 7fe56affd640 1 -- 172.21.15.27:0/3480828830 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v143) v1 ==== 169+0+0 (secure 0 0 0) 0x7fe56c057b90 con 0x7fe5740af250 2023-12-19T18:22:55.907 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-12-19T18:22:55.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.909+0000 7fe57b2b4640 1 -- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe55806a430 msgr2=0x7fe55806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:55.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.909+0000 7fe57b2b4640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe55806a430 0x7fe55806c8f0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe570034000 tx=0x7fe570007000 comp rx=0 tx=0).stop 2023-12-19T18:22:55.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.909+0000 7fe57b2b4640 1 -- 172.21.15.27:0/3480828830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740af250 msgr2=0x7fe5740af6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:55.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.909+0000 7fe57b2b4640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740af250 0x7fe5740af6d0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fe5740c4b30 tx=0x7fe56c0115a0 comp rx=0 tx=0).stop 2023-12-19T18:22:55.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.910+0000 7fe57b2b4640 1 -- 172.21.15.27:0/3480828830 shutdown_connections 2023-12-19T18:22:55.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.910+0000 7fe57b2b4640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe55806a430 0x7fe55806c8f0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:55.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.910+0000 7fe57b2b4640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5740af250 0x7fe5740af6d0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:55.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.910+0000 7fe57b2b4640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe574156e50 0x7fe5740c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:55.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.910+0000 7fe57b2b4640 1 --2- 172.21.15.27:0/3480828830 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe560003970 0x7fe57406d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:55.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.910+0000 7fe57b2b4640 1 -- 172.21.15.27:0/3480828830 >> 172.21.15.27:0/3480828830 conn(0x7fe5740b05b0 msgr2=0x7fe5740b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:55.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.910+0000 7fe57b2b4640 1 -- 172.21.15.27:0/3480828830 shutdown_connections 2023-12-19T18:22:55.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:55.910+0000 7fe57b2b4640 1 -- 172.21.15.27:0/3480828830 wait complete. 2023-12-19T18:22:55.935 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:22:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.166+0000 7f9dc7fff640 1 Processor -- start 2023-12-19T18:22:56.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.166+0000 7f9dc7fff640 1 -- start start 2023-12-19T18:22:56.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.168+0000 7f9dc7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9dc815cfc0 0x7f9dc8160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:56.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.168+0000 7f9dc7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9dc80cdb30 0x7f9dc80cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:56.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.168+0000 7f9dc7fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9dc80ce4d0 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.168+0000 7f9dc7fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9dc8161a50 con 0x7f9dc815cfc0 2023-12-19T18:22:56.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.168+0000 7f9dc7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc8161490 con 0x7f9dc80cdb30 2023-12-19T18:22:56.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.168+0000 7f9dc6ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9dc815cfc0 0x7f9dc8160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:56.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.168+0000 7f9dc6ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9dc815cfc0 0x7f9dc8160b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42170/0 (socket says 172.21.15.27:42170) 2023-12-19T18:22:56.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.168+0000 7f9dc6ffd640 1 -- 172.21.15.27:0/2938076206 learned_addr learned my addr 172.21.15.27:0/2938076206 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:56.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.168+0000 7f9dc67fc640 1 --2- 172.21.15.27:0/2938076206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9dc80cdb30 0x7f9dc80cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:56.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.169+0000 7f9dc6ffd640 1 -- 172.21.15.27:0/2938076206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9dc80cdb30 msgr2=0x7f9dc80cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:56.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.169+0000 7f9dc6ffd640 1 --2- 172.21.15.27:0/2938076206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9dc80cdb30 0x7f9dc80cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.169+0000 7f9dc6ffd640 1 -- 172.21.15.27:0/2938076206 >> v1:172.21.15.192:6789/0 conn(0x7f9dc80d2cf0 legacy=0x7f9dc80d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:56.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.169+0000 7f9dc6ffd640 1 -- 172.21.15.27:0/2938076206 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dc8161770 con 0x7f9dc815cfc0 2023-12-19T18:22:56.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.170+0000 7f9dc6ffd640 1 --2- 172.21.15.27:0/2938076206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dc815cfc0 0x7f9dc8160b40 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009870 tx=0x7f9db802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd349e73de84772 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:56.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.170+0000 7f9dc5ffb640 1 -- 172.21.15.27:0/2938076206 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db80317c0 con 0x7f9dc815cfc0 2023-12-19T18:22:56.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.170+0000 7f9dc5ffb640 1 -- 172.21.15.27:0/2938076206 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db8031920 con 0x7f9dc815cfc0 2023-12-19T18:22:56.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.171+0000 7f9dc5ffb640 1 -- 172.21.15.27:0/2938076206 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db8031cb0 con 0x7f9dc815cfc0 2023-12-19T18:22:56.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.171+0000 7f9dc7fff640 1 -- 172.21.15.27:0/2938076206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dc815cfc0 msgr2=0x7f9dc8160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:56.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.171+0000 7f9dc7fff640 1 --2- 172.21.15.27:0/2938076206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dc815cfc0 0x7f9dc8160b40 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f9db8009870 tx=0x7f9db802b860 comp rx=0 tx=0).stop 2023-12-19T18:22:56.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.171+0000 7f9dc7fff640 1 -- 172.21.15.27:0/2938076206 shutdown_connections 2023-12-19T18:22:56.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.171+0000 7f9dc7fff640 1 --2- 172.21.15.27:0/2938076206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9dc80cdb30 0x7f9dc80cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.171+0000 7f9dc7fff640 1 --2- 172.21.15.27:0/2938076206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dc815cfc0 0x7f9dc8160b40 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.171+0000 7f9dc7fff640 1 -- 172.21.15.27:0/2938076206 >> 172.21.15.27:0/2938076206 conn(0x7f9dc8072e20 msgr2=0x7f9dc8073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:56.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.171+0000 7f9dc7fff640 1 -- 172.21.15.27:0/2938076206 shutdown_connections 2023-12-19T18:22:56.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.171+0000 7f9dc7fff640 1 -- 172.21.15.27:0/2938076206 wait complete. 2023-12-19T18:22:56.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.173+0000 7f9dc7fff640 1 Processor -- start 2023-12-19T18:22:56.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.173+0000 7f9dc7fff640 1 -- start start 2023-12-19T18:22:56.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.174+0000 7f9dc7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9dc80cdb30 0x7f9dc81f2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:56.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.174+0000 7f9dc7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dc80d2cf0 0x7f9dc80ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:56.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.174+0000 7f9dc7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9dc815cfc0 0x7f9dc815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:56.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.174+0000 7f9dc7fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc81f2c90 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.174+0000 7f9dc7fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc81f2680 con 0x7f9dc80cdb30 2023-12-19T18:22:56.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.174+0000 7f9dc7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc80cb140 con 0x7f9dc815cfc0 2023-12-19T18:22:56.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.175+0000 7f9dc67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dc80d2cf0 0x7f9dc80ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:56.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.175+0000 7f9dc67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dc80d2cf0 0x7f9dc80ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42172/0 (socket says 172.21.15.27:42172) 2023-12-19T18:22:56.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.175+0000 7f9dc67fc640 1 -- 172.21.15.27:0/3967855595 learned_addr learned my addr 172.21.15.27:0/3967855595 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:56.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.175+0000 7f9dc67fc640 1 -- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9dc815cfc0 msgr2=0x7f9dc815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:56.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.175+0000 7f9dc67fc640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9dc815cfc0 0x7f9dc815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.175+0000 7f9dc67fc640 1 -- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9dc80cdb30 msgr2=0x7f9dc81f2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:56.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.175+0000 7f9dc6ffd640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9dc80cdb30 0x7f9dc81f2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:56.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.175+0000 7f9dc67fc640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9dc80cdb30 0x7f9dc81f2140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.175+0000 7f9dc67fc640 1 -- 172.21.15.27:0/3967855595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db80094d0 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.175+0000 7f9dc6ffd640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9dc80cdb30 0x7f9dc81f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:56.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.176+0000 7f9dc67fc640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dc80d2cf0 0x7f9dc80ca7f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f9db0007e40 tx=0x7f9db000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:56.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.176+0000 7f9dbf7fe640 1 -- 172.21.15.27:0/3967855595 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db0014620 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.176+0000 7f9dbf7fe640 1 -- 172.21.15.27:0/3967855595 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db000ee90 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.176+0000 7f9dbf7fe640 1 -- 172.21.15.27:0/3967855595 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db00149c0 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.176+0000 7f9dc7fff640 1 -- 172.21.15.27:0/3967855595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc80cb420 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.176+0000 7f9dc7fff640 1 -- 172.21.15.27:0/3967855595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc81f28b0 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.177+0000 7f9dc7fff640 1 -- 172.21.15.27:0/3967855595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dc81f2dd0 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.178+0000 7f9dbf7fe640 1 -- 172.21.15.27:0/3967855595 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9db0014b20 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.181+0000 7f9dbf7fe640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9dac06a430 0x7f9dac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:56.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.181+0000 7f9dbf7fe640 1 -- 172.21.15.27:0/3967855595 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9db0088f50 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.181+0000 7f9dc6ffd640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9dac06a430 0x7f9dac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:56.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.181+0000 7f9dc6ffd640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9dac06a430 0x7f9dac06c8f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9db8036000 tx=0x7f9db803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:56.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.185+0000 7f9dbf7fe640 1 -- 172.21.15.27:0/3967855595 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9dc81f2dd0 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.380+0000 7f9dc7fff640 1 -- 172.21.15.27:0/3967855595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9dc81f2a40 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.387+0000 7f9dbf7fe640 1 -- 172.21.15.27:0/3967855595 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9db0054830 con 0x7f9dc80d2cf0 2023-12-19T18:22:56.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.390+0000 7f9dc7fff640 1 -- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9dac06a430 msgr2=0x7f9dac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:56.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.390+0000 7f9dc7fff640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9dac06a430 0x7f9dac06c8f0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f9db8036000 tx=0x7f9db803d040 comp rx=0 tx=0).stop 2023-12-19T18:22:56.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.390+0000 7f9dc7fff640 1 -- 172.21.15.27:0/3967855595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dc80d2cf0 msgr2=0x7f9dc80ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:56.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.390+0000 7f9dc7fff640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dc80d2cf0 0x7f9dc80ca7f0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f9db0007e40 tx=0x7f9db000e4b0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.390+0000 7f9dc7fff640 1 -- 172.21.15.27:0/3967855595 shutdown_connections 2023-12-19T18:22:56.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.390+0000 7f9dc7fff640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9dac06a430 0x7f9dac06c8f0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.390+0000 7f9dc7fff640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9dc815cfc0 0x7f9dc815b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.390+0000 7f9dc7fff640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dc80d2cf0 0x7f9dc80ca7f0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.390+0000 7f9dc7fff640 1 --2- 172.21.15.27:0/3967855595 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9dc80cdb30 0x7f9dc81f2140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.390+0000 7f9dc7fff640 1 -- 172.21.15.27:0/3967855595 >> 172.21.15.27:0/3967855595 conn(0x7f9dc8072e20 msgr2=0x7f9dc80b27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:56.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.390+0000 7f9dc7fff640 1 -- 172.21.15.27:0/3967855595 shutdown_connections 2023-12-19T18:22:56.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.390+0000 7f9dc7fff640 1 -- 172.21.15.27:0/3967855595 wait complete. 2023-12-19T18:22:56.408 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier' 2023-12-19T18:22:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.644+0000 7fe8a2111640 1 Processor -- start 2023-12-19T18:22:56.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.645+0000 7fe8a2111640 1 -- start start 2023-12-19T18:22:56.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.646+0000 7fe8a2111640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe89c15ee30 0x7fe89c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.646+0000 7fe8a2111640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe89c0afeb0 0x7fe89c0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.646+0000 7fe8a2111640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe89c0cb350 con 0x7fe89c1553a0 2023-12-19T18:22:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.646+0000 7fe8a2111640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe89c0cbcf0 con 0x7fe89c15ee30 2023-12-19T18:22:56.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.646+0000 7fe8a2111640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0cb630 con 0x7fe89c0afeb0 2023-12-19T18:22:56.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.647+0000 7fe8a090e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe89c15ee30 0x7fe89c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:56.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.647+0000 7fe8a1910640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe89c0afeb0 0x7fe89c0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:56.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.647+0000 7fe8a090e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe89c15ee30 0x7fe89c0c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:33984/0 (socket says 172.21.15.27:33984) 2023-12-19T18:22:56.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.647+0000 7fe8a090e640 1 -- 172.21.15.27:0/2661145001 learned_addr learned my addr 172.21.15.27:0/2661145001 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:56.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.648+0000 7fe8a1910640 1 -- 172.21.15.27:0/2661145001 >> v2:172.21.15.192:3300/0 conn(0x7fe89c15ee30 msgr2=0x7fe89c0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:56.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.648+0000 7fe8a1910640 1 --2- 172.21.15.27:0/2661145001 >> v2:172.21.15.192:3300/0 conn(0x7fe89c15ee30 0x7fe89c0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.648+0000 7fe8a1910640 1 -- 172.21.15.27:0/2661145001 >> v1:172.21.15.192:6789/0 conn(0x7fe89c1553a0 legacy=0x7fe89c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:56.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.648+0000 7fe8a1910640 1 -- 172.21.15.27:0/2661145001 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe89c0cb910 con 0x7fe89c0afeb0 2023-12-19T18:22:56.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.648+0000 7fe8a090e640 1 --2- 172.21.15.27:0/2661145001 >> v2:172.21.15.192:3300/0 conn(0x7fe89c15ee30 0x7fe89c0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:22:56.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.649+0000 7fe8a1910640 1 --2- 172.21.15.27:0/2661145001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe89c0afeb0 0x7fe89c0c39d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe894009d90 tx=0x7fe89402be20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7c7d04e5f5d2be5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.650+0000 7fe893fff640 1 -- 172.21.15.27:0/2661145001 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89402cbd0 con 0x7fe89c0afeb0 2023-12-19T18:22:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.650+0000 7fe893fff640 1 -- 172.21.15.27:0/2661145001 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe89402cd30 con 0x7fe89c0afeb0 2023-12-19T18:22:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.650+0000 7fe893fff640 1 -- 172.21.15.27:0/2661145001 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89402cea0 con 0x7fe89c0afeb0 2023-12-19T18:22:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.650+0000 7fe8a2111640 1 -- 172.21.15.27:0/2661145001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe89c0afeb0 msgr2=0x7fe89c0c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:56.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.650+0000 7fe8a2111640 1 --2- 172.21.15.27:0/2661145001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe89c0afeb0 0x7fe89c0c39d0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe894009d90 tx=0x7fe89402be20 comp rx=0 tx=0).stop 2023-12-19T18:22:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.651+0000 7fe8a2111640 1 -- 172.21.15.27:0/2661145001 shutdown_connections 2023-12-19T18:22:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.651+0000 7fe8a2111640 1 --2- 172.21.15.27:0/2661145001 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe89c0afeb0 0x7fe89c0c39d0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.651+0000 7fe8a2111640 1 --2- 172.21.15.27:0/2661145001 >> v2:172.21.15.192:3300/0 conn(0x7fe89c15ee30 0x7fe89c0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.651+0000 7fe8a2111640 1 -- 172.21.15.27:0/2661145001 >> 172.21.15.27:0/2661145001 conn(0x7fe89c069c70 msgr2=0x7fe89c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:56.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.651+0000 7fe8a2111640 1 -- 172.21.15.27:0/2661145001 shutdown_connections 2023-12-19T18:22:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.651+0000 7fe8a2111640 1 -- 172.21.15.27:0/2661145001 wait complete. 2023-12-19T18:22:56.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.652+0000 7fe8a2111640 1 Processor -- start 2023-12-19T18:22:56.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.653+0000 7fe8a2111640 1 -- start start 2023-12-19T18:22:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a2111640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe89c0afeb0 0x7fe89c148400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a2111640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe89c1553a0 0x7fe89c0cdea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a2111640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89c15ee30 0x7fe89c0ad910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a2111640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0cd730 con 0x7fe89c15ee30 2023-12-19T18:22:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a2111640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c148940 con 0x7fe89c1553a0 2023-12-19T18:22:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a2111640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe89c0ccab0 con 0x7fe89c0afeb0 2023-12-19T18:22:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a1910640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89c15ee30 0x7fe89c0ad910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a1910640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89c15ee30 0x7fe89c0ad910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42184/0 (socket says 172.21.15.27:42184) 2023-12-19T18:22:56.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a1910640 1 -- 172.21.15.27:0/2957291961 learned_addr learned my addr 172.21.15.27:0/2957291961 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a090e640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe89c1553a0 0x7fe89c0cdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a110f640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe89c0afeb0 0x7fe89c148400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a1910640 1 -- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe89c0afeb0 msgr2=0x7fe89c148400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a1910640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe89c0afeb0 0x7fe89c148400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a1910640 1 -- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe89c1553a0 msgr2=0x7fe89c0cdea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a1910640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe89c1553a0 0x7fe89c0cdea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.654+0000 7fe8a1910640 1 -- 172.21.15.27:0/2957291961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe88c0094d0 con 0x7fe89c15ee30 2023-12-19T18:22:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.655+0000 7fe8a110f640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe89c0afeb0 0x7fe89c148400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:22:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.655+0000 7fe8a1910640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89c15ee30 0x7fe89c0ad910 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe894009d90 tx=0x7fe894004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.655+0000 7fe891ffb640 1 -- 172.21.15.27:0/2957291961 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe89402cea0 con 0x7fe89c15ee30 2023-12-19T18:22:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.655+0000 7fe891ffb640 1 -- 172.21.15.27:0/2957291961 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe894040650 con 0x7fe89c15ee30 2023-12-19T18:22:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.655+0000 7fe8a2111640 1 -- 172.21.15.27:0/2957291961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe894009a70 con 0x7fe89c15ee30 2023-12-19T18:22:56.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.655+0000 7fe891ffb640 1 -- 172.21.15.27:0/2957291961 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe894040a00 con 0x7fe89c15ee30 2023-12-19T18:22:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.655+0000 7fe8a2111640 1 -- 172.21.15.27:0/2957291961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe89c0cd070 con 0x7fe89c15ee30 2023-12-19T18:22:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.657+0000 7fe8a2111640 1 -- 172.21.15.27:0/2957291961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe89c04e840 con 0x7fe89c15ee30 2023-12-19T18:22:56.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.658+0000 7fe891ffb640 1 -- 172.21.15.27:0/2957291961 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe894040b60 con 0x7fe89c15ee30 2023-12-19T18:22:56.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.659+0000 7fe891ffb640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe88006a430 0x7fe88006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:56.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.659+0000 7fe891ffb640 1 -- 172.21.15.27:0/2957291961 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe8940ab250 con 0x7fe89c15ee30 2023-12-19T18:22:56.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.659+0000 7fe8a110f640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe88006a430 0x7fe88006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:56.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.660+0000 7fe8a110f640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe88006a430 0x7fe88006c8f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe898000f60 tx=0x7fe898009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:56.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.663+0000 7fe891ffb640 1 -- 172.21.15.27:0/2957291961 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe894076c50 con 0x7fe89c15ee30 2023-12-19T18:22:56.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:56.933+0000 7fe8a2111640 1 -- 172.21.15.27:0/2957291961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]} v 0) v1 -- 0x7fe89c0af6a0 con 0x7fe89c15ee30 2023-12-19T18:22:57.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:57.910+0000 7fe891ffb640 1 -- 172.21.15.27:0/2957291961 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_barrier"]}]=0 v412) v1 ==== 128+0+0 (secure 0 0 0) 0x7fe894009450 con 0x7fe89c15ee30 2023-12-19T18:22:57.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:57.914+0000 7fe8a2111640 1 -- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe88006a430 msgr2=0x7fe88006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:57.914+0000 7fe8a2111640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe88006a430 0x7fe88006c8f0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fe898000f60 tx=0x7fe898009040 comp rx=0 tx=0).stop 2023-12-19T18:22:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:57.914+0000 7fe8a2111640 1 -- 172.21.15.27:0/2957291961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89c15ee30 msgr2=0x7fe89c0ad910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:57.914+0000 7fe8a2111640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89c15ee30 0x7fe89c0ad910 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fe894009d90 tx=0x7fe894004a40 comp rx=0 tx=0).stop 2023-12-19T18:22:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:57.914+0000 7fe8a2111640 1 -- 172.21.15.27:0/2957291961 shutdown_connections 2023-12-19T18:22:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:57.914+0000 7fe8a2111640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe88006a430 0x7fe88006c8f0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:57.914+0000 7fe8a2111640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe89c15ee30 0x7fe89c0ad910 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:57.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:57.914+0000 7fe8a2111640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe89c1553a0 0x7fe89c0cdea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:57.914+0000 7fe8a2111640 1 --2- 172.21.15.27:0/2957291961 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe89c0afeb0 0x7fe89c148400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:57.914+0000 7fe8a2111640 1 -- 172.21.15.27:0/2957291961 >> 172.21.15.27:0/2957291961 conn(0x7fe89c069c70 msgr2=0x7fe89c159d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:57.914+0000 7fe8a2111640 1 -- 172.21.15.27:0/2957291961 shutdown_connections 2023-12-19T18:22:57.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:57.915+0000 7fe8a2111640 1 -- 172.21.15.27:0/2957291961 wait complete. 2023-12-19T18:22:57.936 INFO:tasks.cephfs_test_runner:That when an OSD epoch barrier is set on an MDS, subsequently ... ok 2023-12-19T18:22:57.936 INFO:tasks.cephfs_test_runner:Starting test: test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) 2023-12-19T18:22:57.937 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-12-19T18:22:58.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.173+0000 7f9f7913c640 1 Processor -- start 2023-12-19T18:22:58.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.173+0000 7f9f7913c640 1 -- start start 2023-12-19T18:22:58.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.175+0000 7f9f7913c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9f7415b020 0x7f9f740cda20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:58.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.175+0000 7f9f7913c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f740d0250 con 0x7f9f740cb3d0 2023-12-19T18:22:58.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.175+0000 7f9f7913c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f7415c2c0 con 0x7f9f740d0b90 2023-12-19T18:22:58.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.175+0000 7f9f7913c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f7415c5a0 con 0x7f9f7415b020 2023-12-19T18:22:58.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.175+0000 7f9f7259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9f740cb3d0 0x7f9f740d0480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60440/0 (socket says 172.21.15.27:60440) 2023-12-19T18:22:58.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.175+0000 7f9f7259c640 1 -- 172.21.15.27:0/3375415423 learned_addr learned my addr 172.21.15.27:0/3375415423 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:58.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.175+0000 7f9f72d9d640 1 --2- 172.21.15.27:0/3375415423 >> v2:172.21.15.27:3300/0 conn(0x7f9f7415b020 0x7f9f740cda20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:58.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.176+0000 7f9f71d9b640 1 -- 172.21.15.27:0/3375415423 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1238946073 0 0) 0x7f9f740d0250 con 0x7f9f740cb3d0 2023-12-19T18:22:58.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.176+0000 7f9f72d9d640 1 -- 172.21.15.27:0/3375415423 >> v1:172.21.15.192:6789/0 conn(0x7f9f740d0b90 legacy=0x7f9f7415ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:58.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.176+0000 7f9f72d9d640 1 -- 172.21.15.27:0/3375415423 >> v1:172.21.15.27:6789/0 conn(0x7f9f740cb3d0 legacy=0x7f9f740d0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:58.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.176+0000 7f9f72d9d640 1 -- 172.21.15.27:0/3375415423 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f740ccf40 con 0x7f9f7415b020 2023-12-19T18:22:58.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.178+0000 7f9f72d9d640 1 --2- 172.21.15.27:0/3375415423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f7415b020 0x7f9f740cda20 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0097a0 tx=0x7f9f5c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ce065532cb763e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:58.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.178+0000 7f9f71d9b640 1 -- 172.21.15.27:0/3375415423 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c031830 con 0x7f9f7415b020 2023-12-19T18:22:58.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.178+0000 7f9f71d9b640 1 -- 172.21.15.27:0/3375415423 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f5c031990 con 0x7f9f7415b020 2023-12-19T18:22:58.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.178+0000 7f9f71d9b640 1 -- 172.21.15.27:0/3375415423 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f5c031d20 con 0x7f9f7415b020 2023-12-19T18:22:58.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.179+0000 7f9f7913c640 1 -- 172.21.15.27:0/3375415423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f7415b020 msgr2=0x7f9f740cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:58.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.179+0000 7f9f7913c640 1 --2- 172.21.15.27:0/3375415423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f7415b020 0x7f9f740cda20 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c0097a0 tx=0x7f9f5c02b8d0 comp rx=0 tx=0).stop 2023-12-19T18:22:58.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.179+0000 7f9f7913c640 1 -- 172.21.15.27:0/3375415423 shutdown_connections 2023-12-19T18:22:58.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.179+0000 7f9f7913c640 1 --2- 172.21.15.27:0/3375415423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f7415b020 0x7f9f740cda20 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:58.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.179+0000 7f9f7913c640 1 -- 172.21.15.27:0/3375415423 >> 172.21.15.27:0/3375415423 conn(0x7f9f74072c40 msgr2=0x7f9f74073020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:58.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.179+0000 7f9f7913c640 1 -- 172.21.15.27:0/3375415423 shutdown_connections 2023-12-19T18:22:58.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.179+0000 7f9f7913c640 1 -- 172.21.15.27:0/3375415423 wait complete. 2023-12-19T18:22:58.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.181+0000 7f9f7913c640 1 Processor -- start 2023-12-19T18:22:58.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.181+0000 7f9f7913c640 1 -- start start 2023-12-19T18:22:58.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.182+0000 7f9f7913c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f740cb3d0 0x7f9f740b1770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:58.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.182+0000 7f9f7913c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f740d0b90 0x7f9f740c7f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:58.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.182+0000 7f9f7913c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f7415b020 0x7f9f7406aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:58.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.182+0000 7f9f7913c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740c9fe0 con 0x7f9f7415b020 2023-12-19T18:22:58.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.182+0000 7f9f7913c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740ca2c0 con 0x7f9f740cb3d0 2023-12-19T18:22:58.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.182+0000 7f9f7913c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f740ca5a0 con 0x7f9f740d0b90 2023-12-19T18:22:58.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.182+0000 7f9f7359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f7415b020 0x7f9f7406aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:58.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.182+0000 7f9f7359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f7415b020 0x7f9f7406aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42208/0 (socket says 172.21.15.27:42208) 2023-12-19T18:22:58.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.182+0000 7f9f7359e640 1 -- 172.21.15.27:0/2037451127 learned_addr learned my addr 172.21.15.27:0/2037451127 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:58.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.183+0000 7f9f7259c640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f740d0b90 0x7f9f740c7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:58.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.183+0000 7f9f72d9d640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f740cb3d0 0x7f9f740b1770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:58.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.183+0000 7f9f7359e640 1 -- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f740d0b90 msgr2=0x7f9f740c7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:58.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.183+0000 7f9f7359e640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f740d0b90 0x7f9f740c7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:58.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.183+0000 7f9f7359e640 1 -- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f740cb3d0 msgr2=0x7f9f740b1770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:58.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.183+0000 7f9f7359e640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f740cb3d0 0x7f9f740b1770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:58.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.183+0000 7f9f7359e640 1 -- 172.21.15.27:0/2037451127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f5c009400 con 0x7f9f7415b020 2023-12-19T18:22:58.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.183+0000 7f9f7359e640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f7415b020 0x7f9f7406aff0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c780 tx=0x7f9f6c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:58.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.184+0000 7f9f6b7fe640 1 -- 172.21.15.27:0/2037451127 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c004b60 con 0x7f9f7415b020 2023-12-19T18:22:58.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.184+0000 7f9f6b7fe640 1 -- 172.21.15.27:0/2037451127 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f6c00ce90 con 0x7f9f7415b020 2023-12-19T18:22:58.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.184+0000 7f9f7913c640 1 -- 172.21.15.27:0/2037451127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f740ca880 con 0x7f9f7415b020 2023-12-19T18:22:58.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.184+0000 7f9f6b7fe640 1 -- 172.21.15.27:0/2037451127 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f6c00a850 con 0x7f9f7415b020 2023-12-19T18:22:58.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.184+0000 7f9f7913c640 1 -- 172.21.15.27:0/2037451127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f740b1cb0 con 0x7f9f7415b020 2023-12-19T18:22:58.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.185+0000 7f9f7913c640 1 -- 172.21.15.27:0/2037451127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f740cada0 con 0x7f9f7415b020 2023-12-19T18:22:58.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.186+0000 7f9f6b7fe640 1 -- 172.21.15.27:0/2037451127 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9f6c00a9b0 con 0x7f9f7415b020 2023-12-19T18:22:58.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.187+0000 7f9f6b7fe640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f5406a430 0x7f9f5406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:58.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.187+0000 7f9f6b7fe640 1 -- 172.21.15.27:0/2037451127 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9f6c0888b0 con 0x7f9f7415b020 2023-12-19T18:22:58.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.188+0000 7f9f72d9d640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f5406a430 0x7f9f5406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:58.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.189+0000 7f9f72d9d640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f5406a430 0x7f9f5406c8f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c036000 tx=0x7f9f5c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:58.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.192+0000 7f9f6b7fe640 1 -- 172.21.15.27:0/2037451127 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9f6c054100 con 0x7f9f7415b020 2023-12-19T18:22:58.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.453+0000 7f9f7913c640 1 -- 172.21.15.27:0/2037451127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f9f740cd1f0 con 0x7f9f7415b020 2023-12-19T18:22:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.914+0000 7f9f6b7fe640 1 -- 172.21.15.27:0/2037451127 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v413) v1 ==== 143+0+0 (secure 0 0 0) 0x7f9f6c057720 con 0x7f9f7415b020 2023-12-19T18:22:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.917+0000 7f9f7913c640 1 -- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f5406a430 msgr2=0x7f9f5406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.917+0000 7f9f7913c640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f5406a430 0x7f9f5406c8f0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f9f5c036000 tx=0x7f9f5c03c040 comp rx=0 tx=0).stop 2023-12-19T18:22:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.917+0000 7f9f7913c640 1 -- 172.21.15.27:0/2037451127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f7415b020 msgr2=0x7f9f7406aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.917+0000 7f9f7913c640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f7415b020 0x7f9f7406aff0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7f9f6c00c780 tx=0x7f9f6c00cc40 comp rx=0 tx=0).stop 2023-12-19T18:22:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.917+0000 7f9f7913c640 1 -- 172.21.15.27:0/2037451127 shutdown_connections 2023-12-19T18:22:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.917+0000 7f9f7913c640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f5406a430 0x7f9f5406c8f0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.917+0000 7f9f7913c640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f7415b020 0x7f9f7406aff0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.917+0000 7f9f7913c640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f740d0b90 0x7f9f740c7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.917+0000 7f9f7913c640 1 --2- 172.21.15.27:0/2037451127 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f740cb3d0 0x7f9f740b1770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.917+0000 7f9f7913c640 1 -- 172.21.15.27:0/2037451127 >> 172.21.15.27:0/2037451127 conn(0x7f9f74072c40 msgr2=0x7f9f7406bc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:58.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.918+0000 7f9f7913c640 1 -- 172.21.15.27:0/2037451127 shutdown_connections 2023-12-19T18:22:58.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:58.918+0000 7f9f7913c640 1 -- 172.21.15.27:0/2037451127 wait complete. 2023-12-19T18:22:58.939 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:22:59.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.177+0000 7f7c8cdfb640 1 Processor -- start 2023-12-19T18:22:59.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.177+0000 7f7c8cdfb640 1 -- start start 2023-12-19T18:22:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.179+0000 7f7c8cdfb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c88161870 0x7f7c880c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.179+0000 7f7c8cdfb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7c880c19e0 0x7f7c8806f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.179+0000 7f7c8cdfb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c880cc960 con 0x7f7c8806fa60 2023-12-19T18:22:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.179+0000 7f7c8cdfb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c880c1e40 con 0x7f7c880c19e0 2023-12-19T18:22:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.179+0000 7f7c8cdfb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8806fef0 con 0x7f7c88161870 2023-12-19T18:22:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.179+0000 7f7c85d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7c880c19e0 0x7f7c8806f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.179+0000 7f7c86d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7c8806fa60 0x7f7c88165380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60442/0 (socket says 172.21.15.27:60442) 2023-12-19T18:22:59.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.180+0000 7f7c86d9d640 1 -- 172.21.15.27:0/4064363549 learned_addr learned my addr 172.21.15.27:0/4064363549 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:22:59.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.180+0000 7f7c8659c640 1 --2- 172.21.15.27:0/4064363549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c88161870 0x7f7c880c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:59.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.180+0000 7f7c85d9b640 1 -- 172.21.15.27:0/4064363549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c88161870 msgr2=0x7f7c880c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:22:59.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.180+0000 7f7c85d9b640 1 --2- 172.21.15.27:0/4064363549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c88161870 0x7f7c880c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:59.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.180+0000 7f7c85d9b640 1 -- 172.21.15.27:0/4064363549 >> v1:172.21.15.27:6789/0 conn(0x7f7c8806fa60 legacy=0x7f7c88165380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:59.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.180+0000 7f7c8559a640 1 -- 172.21.15.27:0/4064363549 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2167825007 0 0) 0x7f7c880cc960 con 0x7f7c8806fa60 2023-12-19T18:22:59.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.180+0000 7f7c85d9b640 1 -- 172.21.15.27:0/4064363549 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c880caab0 con 0x7f7c880c19e0 2023-12-19T18:22:59.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.181+0000 7f7c85d9b640 1 --2- 172.21.15.27:0/4064363549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c880c19e0 0x7f7c8806f520 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7c700097a0 tx=0x7f7c7002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c37ed9aa6ead91 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:59.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.182+0000 7f7c8559a640 1 -- 172.21.15.27:0/4064363549 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c7002e070 con 0x7f7c880c19e0 2023-12-19T18:22:59.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.182+0000 7f7c8559a640 1 -- 172.21.15.27:0/4064363549 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c700315a0 con 0x7f7c880c19e0 2023-12-19T18:22:59.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.182+0000 7f7c8559a640 1 -- 172.21.15.27:0/4064363549 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c70031930 con 0x7f7c880c19e0 2023-12-19T18:22:59.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.182+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/4064363549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c880c19e0 msgr2=0x7f7c8806f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:59.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.182+0000 7f7c8cdfb640 1 --2- 172.21.15.27:0/4064363549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c880c19e0 0x7f7c8806f520 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f7c700097a0 tx=0x7f7c7002b840 comp rx=0 tx=0).stop 2023-12-19T18:22:59.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.182+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/4064363549 shutdown_connections 2023-12-19T18:22:59.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.182+0000 7f7c8cdfb640 1 --2- 172.21.15.27:0/4064363549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c880c19e0 0x7f7c8806f520 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:59.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.182+0000 7f7c8cdfb640 1 --2- 172.21.15.27:0/4064363549 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c88161870 0x7f7c880c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:59.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.182+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/4064363549 >> 172.21.15.27:0/4064363549 conn(0x7f7c880cebd0 msgr2=0x7f7c880cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:59.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.182+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/4064363549 shutdown_connections 2023-12-19T18:22:59.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.183+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/4064363549 wait complete. 2023-12-19T18:22:59.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.184+0000 7f7c8cdfb640 1 Processor -- start 2023-12-19T18:22:59.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.185+0000 7f7c8cdfb640 1 -- start start 2023-12-19T18:22:59.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8cdfb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8806fa60 0x7f7c88164360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:59.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8cdfb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c880c19e0 0x7f7c880c2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:59.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8cdfb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c88161870 0x7f7c88162eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:59.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8cdfb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c880c2f30 con 0x7f7c8806fa60 2023-12-19T18:22:59.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8cdfb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c881648a0 con 0x7f7c880c19e0 2023-12-19T18:22:59.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8cdfb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c880c6640 con 0x7f7c88161870 2023-12-19T18:22:59.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8806fa60 0x7f7c88164360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:59.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8806fa60 0x7f7c88164360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42230/0 (socket says 172.21.15.27:42230) 2023-12-19T18:22:59.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8659c640 1 -- 172.21.15.27:0/2013206716 learned_addr learned my addr 172.21.15.27:0/2013206716 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:22:59.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8659c640 1 -- 172.21.15.27:0/2013206716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c88161870 msgr2=0x7f7c88162eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:59.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8659c640 1 --2- 172.21.15.27:0/2013206716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c88161870 0x7f7c88162eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:59.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8659c640 1 -- 172.21.15.27:0/2013206716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c880c19e0 msgr2=0x7f7c880c2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:59.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8659c640 1 --2- 172.21.15.27:0/2013206716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c880c19e0 0x7f7c880c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:59.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.186+0000 7f7c8659c640 1 -- 172.21.15.27:0/2013206716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c70009400 con 0x7f7c8806fa60 2023-12-19T18:22:59.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.187+0000 7f7c8659c640 1 --2- 172.21.15.27:0/2013206716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8806fa60 0x7f7c88164360 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7c78007ef0 tx=0x7f7c7800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.187+0000 7f7c7effd640 1 -- 172.21.15.27:0/2013206716 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78014620 con 0x7f7c8806fa60 2023-12-19T18:22:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.187+0000 7f7c7effd640 1 -- 172.21.15.27:0/2013206716 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c78014780 con 0x7f7c8806fa60 2023-12-19T18:22:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.187+0000 7f7c7effd640 1 -- 172.21.15.27:0/2013206716 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c78014b10 con 0x7f7c8806fa60 2023-12-19T18:22:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.187+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/2013206716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c880c6920 con 0x7f7c8806fa60 2023-12-19T18:22:59.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.187+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/2013206716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c880c6ee0 con 0x7f7c8806fa60 2023-12-19T18:22:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.188+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/2013206716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c880c7070 con 0x7f7c8806fa60 2023-12-19T18:22:59.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.190+0000 7f7c7effd640 1 -- 172.21.15.27:0/2013206716 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7c78014c70 con 0x7f7c8806fa60 2023-12-19T18:22:59.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.190+0000 7f7c7effd640 1 --2- 172.21.15.27:0/2013206716 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c6806a430 0x7f7c6806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:22:59.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.191+0000 7f7c7effd640 1 -- 172.21.15.27:0/2013206716 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f7c78089050 con 0x7f7c8806fa60 2023-12-19T18:22:59.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.191+0000 7f7c85d9b640 1 --2- 172.21.15.27:0/2013206716 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c6806a430 0x7f7c6806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:22:59.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.192+0000 7f7c85d9b640 1 --2- 172.21.15.27:0/2013206716 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c6806a430 0x7f7c6806c8f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7c70036000 tx=0x7f7c70030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:22:59.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.195+0000 7f7c7effd640 1 -- 172.21.15.27:0/2013206716 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7c78054940 con 0x7f7c8806fa60 2023-12-19T18:22:59.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.387+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/2013206716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f7c88068e90 con 0x7f7c8806fa60 2023-12-19T18:22:59.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.396+0000 7f7c7effd640 1 -- 172.21.15.27:0/2013206716 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f7c7805e020 con 0x7f7c8806fa60 2023-12-19T18:22:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.399+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/2013206716 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c6806a430 msgr2=0x7f7c6806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.399+0000 7f7c8cdfb640 1 --2- 172.21.15.27:0/2013206716 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c6806a430 0x7f7c6806c8f0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7c70036000 tx=0x7f7c70030000 comp rx=0 tx=0).stop 2023-12-19T18:22:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.399+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/2013206716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8806fa60 msgr2=0x7f7c88164360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:22:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.399+0000 7f7c8cdfb640 1 --2- 172.21.15.27:0/2013206716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8806fa60 0x7f7c88164360 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f7c78007ef0 tx=0x7f7c7800e4b0 comp rx=0 tx=0).stop 2023-12-19T18:22:59.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.399+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/2013206716 shutdown_connections 2023-12-19T18:22:59.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.399+0000 7f7c8cdfb640 1 --2- 172.21.15.27:0/2013206716 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7c6806a430 0x7f7c6806c8f0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:59.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.399+0000 7f7c8cdfb640 1 --2- 172.21.15.27:0/2013206716 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7c88161870 0x7f7c88162eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:59.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.399+0000 7f7c8cdfb640 1 --2- 172.21.15.27:0/2013206716 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7c880c19e0 0x7f7c880c2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:59.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.399+0000 7f7c8cdfb640 1 --2- 172.21.15.27:0/2013206716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c8806fa60 0x7f7c88164360 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:22:59.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.399+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/2013206716 >> 172.21.15.27:0/2013206716 conn(0x7f7c880cebd0 msgr2=0x7f7c8806b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:22:59.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.399+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/2013206716 shutdown_connections 2023-12-19T18:22:59.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:22:59.399+0000 7f7c8cdfb640 1 -- 172.21.15.27:0/2013206716 wait complete. 2023-12-19T18:22:59.419 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:22:59.420 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:22:59.531 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:22:59.645 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:22:59.645 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:22:59.645 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:22:59.709 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:22:59.709 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:22:59.709 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:22:59.768 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:22:59.768 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:22:59.768 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:22:59.833 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:22:59.833 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:22:59.833 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:22:59.894 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:22:59.894 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:22:59.895 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:23:00.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.135+0000 7f86c2f6c640 1 Processor -- start 2023-12-19T18:23:00.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.135+0000 7f86c2f6c640 1 -- start start 2023-12-19T18:23:00.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.136+0000 7f86c2f6c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86bc0b3170 0x7f86bc0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:00.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.136+0000 7f86c2f6c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f86bc0b2b10 0x7f86bc0cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:00.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.136+0000 7f86c2f6c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86bc15b850 con 0x7f86bc15ad60 2023-12-19T18:23:00.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.136+0000 7f86c2f6c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86bc0b2fa0 con 0x7f86bc0b2b10 2023-12-19T18:23:00.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.136+0000 7f86c2f6c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc15bb20 con 0x7f86bc0b3170 2023-12-19T18:23:00.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.137+0000 7f86c1769640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86bc0b3170 0x7f86bc0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:00.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.137+0000 7f86c276b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f86bc0b2b10 0x7f86bc0cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:00.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.137+0000 7f86c1769640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86bc0b3170 0x7f86bc0b25d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:58636/0 (socket says 172.21.15.27:58636) 2023-12-19T18:23:00.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.137+0000 7f86c276b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f86bc0b2b10 0x7f86bc0cd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:34028/0 (socket says 172.21.15.27:34028) 2023-12-19T18:23:00.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.137+0000 7f86c1769640 1 -- 172.21.15.27:0/336861988 learned_addr learned my addr 172.21.15.27:0/336861988 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:00.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.138+0000 7f86c1769640 1 -- 172.21.15.27:0/336861988 >> v2:172.21.15.192:3300/0 conn(0x7f86bc0b2b10 msgr2=0x7f86bc0cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:00.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.138+0000 7f86c1769640 1 --2- 172.21.15.27:0/336861988 >> v2:172.21.15.192:3300/0 conn(0x7f86bc0b2b10 0x7f86bc0cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.138+0000 7f86c1769640 1 -- 172.21.15.27:0/336861988 >> v1:172.21.15.192:6789/0 conn(0x7f86bc15ad60 legacy=0x7f86bc15b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:00.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.138+0000 7f86c1769640 1 -- 172.21.15.27:0/336861988 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86bc15be00 con 0x7f86bc0b3170 2023-12-19T18:23:00.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.138+0000 7f86c0f68640 1 -- 172.21.15.27:0/336861988 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1629909114 0 0) 0x7f86bc15b850 con 0x7f86bc15ad60 2023-12-19T18:23:00.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.138+0000 7f86c276b640 1 --2- 172.21.15.27:0/336861988 >> v2:172.21.15.192:3300/0 conn(0x7f86bc0b2b10 0x7f86bc0cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:23:00.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.140+0000 7f86c1769640 1 --2- 172.21.15.27:0/336861988 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86bc0b3170 0x7f86bc0b25d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009870 tx=0x7f86b002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54ad9c425126902e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:00.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.140+0000 7f86c0f68640 1 -- 172.21.15.27:0/336861988 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b0030840 con 0x7f86bc0b3170 2023-12-19T18:23:00.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.140+0000 7f86c0f68640 1 -- 172.21.15.27:0/336861988 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b00309a0 con 0x7f86bc0b3170 2023-12-19T18:23:00.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.140+0000 7f86c0f68640 1 -- 172.21.15.27:0/336861988 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b0030d30 con 0x7f86bc0b3170 2023-12-19T18:23:00.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.141+0000 7f86c2f6c640 1 -- 172.21.15.27:0/336861988 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86bc0b3170 msgr2=0x7f86bc0b25d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:00.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.141+0000 7f86c2f6c640 1 --2- 172.21.15.27:0/336861988 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86bc0b3170 0x7f86bc0b25d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f86b0009870 tx=0x7f86b002b980 comp rx=0 tx=0).stop 2023-12-19T18:23:00.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.141+0000 7f86c2f6c640 1 -- 172.21.15.27:0/336861988 shutdown_connections 2023-12-19T18:23:00.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.141+0000 7f86c2f6c640 1 --2- 172.21.15.27:0/336861988 >> v2:172.21.15.192:3300/0 conn(0x7f86bc0b2b10 0x7f86bc0cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.141+0000 7f86c2f6c640 1 --2- 172.21.15.27:0/336861988 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86bc0b3170 0x7f86bc0b25d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.141+0000 7f86c2f6c640 1 -- 172.21.15.27:0/336861988 >> 172.21.15.27:0/336861988 conn(0x7f86bc06d5f0 msgr2=0x7f86bc06d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:00.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.141+0000 7f86c2f6c640 1 -- 172.21.15.27:0/336861988 shutdown_connections 2023-12-19T18:23:00.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.141+0000 7f86c2f6c640 1 -- 172.21.15.27:0/336861988 wait complete. 2023-12-19T18:23:00.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.143+0000 7f86c2f6c640 1 Processor -- start 2023-12-19T18:23:00.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.143+0000 7f86c2f6c640 1 -- start start 2023-12-19T18:23:00.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.144+0000 7f86c2f6c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86bc0b2b10 0x7f86bc0c97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:00.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.144+0000 7f86c2f6c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86bc0b3170 0x7f86bc0b0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:00.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.144+0000 7f86c2f6c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86bc15ad60 0x7f86bc1eecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:00.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.144+0000 7f86c2f6c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0c9350 con 0x7f86bc0b2b10 2023-12-19T18:23:00.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.144+0000 7f86c2f6c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc0b1a10 con 0x7f86bc15ad60 2023-12-19T18:23:00.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.144+0000 7f86c2f6c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86bc1f28a0 con 0x7f86bc0b3170 2023-12-19T18:23:00.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.144+0000 7f86c1f6a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86bc0b2b10 0x7f86bc0c97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:00.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.144+0000 7f86c1f6a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86bc0b2b10 0x7f86bc0c97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42238/0 (socket says 172.21.15.27:42238) 2023-12-19T18:23:00.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.144+0000 7f86c1f6a640 1 -- 172.21.15.27:0/1519314088 learned_addr learned my addr 172.21.15.27:0/1519314088 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:00.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.145+0000 7f86c1769640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86bc0b3170 0x7f86bc0b0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:00.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.145+0000 7f86c276b640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86bc15ad60 0x7f86bc1eecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:00.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.145+0000 7f86c1f6a640 1 -- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86bc0b3170 msgr2=0x7f86bc0b0ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:00.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.145+0000 7f86c1f6a640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86bc0b3170 0x7f86bc0b0ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.145+0000 7f86c1f6a640 1 -- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86bc15ad60 msgr2=0x7f86bc1eecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:00.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.145+0000 7f86c1f6a640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86bc15ad60 0x7f86bc1eecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.145+0000 7f86c1f6a640 1 -- 172.21.15.27:0/1519314088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b4009a70 con 0x7f86bc0b2b10 2023-12-19T18:23:00.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.145+0000 7f86c1f6a640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86bc0b2b10 0x7f86bc0c97e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c4b0 tx=0x7f86ac00c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:00.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.145+0000 7f86ba7fc640 1 -- 172.21.15.27:0/1519314088 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac005480 con 0x7f86bc0b2b10 2023-12-19T18:23:00.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.146+0000 7f86ba7fc640 1 -- 172.21.15.27:0/1519314088 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86ac0035f0 con 0x7f86bc0b2b10 2023-12-19T18:23:00.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.146+0000 7f86c2f6c640 1 -- 172.21.15.27:0/1519314088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86b00094d0 con 0x7f86bc0b2b10 2023-12-19T18:23:00.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.146+0000 7f86ba7fc640 1 -- 172.21.15.27:0/1519314088 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86ac0039e0 con 0x7f86bc0b2b10 2023-12-19T18:23:00.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.146+0000 7f86c2f6c640 1 -- 172.21.15.27:0/1519314088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86bc0c8fe0 con 0x7f86bc0b2b10 2023-12-19T18:23:00.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.147+0000 7f86c2f6c640 1 -- 172.21.15.27:0/1519314088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86bc04e840 con 0x7f86bc0b2b10 2023-12-19T18:23:00.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.148+0000 7f86ba7fc640 1 -- 172.21.15.27:0/1519314088 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f86ac003b40 con 0x7f86bc0b2b10 2023-12-19T18:23:00.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.149+0000 7f86ba7fc640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f86a806a430 0x7f86a806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:00.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.149+0000 7f86ba7fc640 1 -- 172.21.15.27:0/1519314088 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f86ac087ce0 con 0x7f86bc0b2b10 2023-12-19T18:23:00.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.149+0000 7f86c1769640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f86a806a430 0x7f86a806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:00.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.150+0000 7f86c1769640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f86a806a430 0x7f86a806c8f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f86b0036000 tx=0x7f86b003b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:00.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.153+0000 7f86ba7fc640 1 -- 172.21.15.27:0/1519314088 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f86ac053530 con 0x7f86bc0b2b10 2023-12-19T18:23:00.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.391+0000 7f86c2f6c640 1 -- 172.21.15.27:0/1519314088 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f86bc06f8e0 con 0x7f86bc0b2b10 2023-12-19T18:23:00.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.392+0000 7f86ba7fc640 1 -- 172.21.15.27:0/1519314088 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f86ac017020 con 0x7f86bc0b2b10 2023-12-19T18:23:00.393 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:00.393 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5519,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/2713867269","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":2713867269},{"type":"v1","addr":"172.21.15.192:6837","nonce":2713867269}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":5534,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/919878747","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6833","nonce":919878747},{"type":"v1","addr":"172.21.15.27:6835","nonce":919878747}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6160,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/861111763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":861111763},{"type":"v1","addr":"172.21.15.192:6836","nonce":861111763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7725,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6834/191095169","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":191095169},{"type":"v1","addr":"172.21.15.27:6834","nonce":191095169}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[]} 2023-12-19T18:23:00.393 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 56 2023-12-19T18:23:00.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.395+0000 7f86c2f6c640 1 -- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f86a806a430 msgr2=0x7f86a806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:00.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.395+0000 7f86c2f6c640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f86a806a430 0x7f86a806c8f0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f86b0036000 tx=0x7f86b003b040 comp rx=0 tx=0).stop 2023-12-19T18:23:00.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.395+0000 7f86c2f6c640 1 -- 172.21.15.27:0/1519314088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86bc0b2b10 msgr2=0x7f86bc0c97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:00.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.395+0000 7f86c2f6c640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86bc0b2b10 0x7f86bc0c97e0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f86ac00c4b0 tx=0x7f86ac00c970 comp rx=0 tx=0).stop 2023-12-19T18:23:00.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.395+0000 7f86c2f6c640 1 -- 172.21.15.27:0/1519314088 shutdown_connections 2023-12-19T18:23:00.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.395+0000 7f86c2f6c640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f86a806a430 0x7f86a806c8f0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.395+0000 7f86c2f6c640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f86bc15ad60 0x7f86bc1eecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.395+0000 7f86c2f6c640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f86bc0b3170 0x7f86bc0b0ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.395+0000 7f86c2f6c640 1 --2- 172.21.15.27:0/1519314088 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86bc0b2b10 0x7f86bc0c97e0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.395+0000 7f86c2f6c640 1 -- 172.21.15.27:0/1519314088 >> 172.21.15.27:0/1519314088 conn(0x7f86bc06d5f0 msgr2=0x7f86bc06ad30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:00.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.396+0000 7f86c2f6c640 1 -- 172.21.15.27:0/1519314088 shutdown_connections 2023-12-19T18:23:00.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.396+0000 7f86c2f6c640 1 -- 172.21.15.27:0/1519314088 wait complete. 2023-12-19T18:23:00.415 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:23:00.415 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:23:00.415 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:23:00.415 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:23:00.415 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:23:00.416 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:23:00.416 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:23:00.416 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:23:00.416 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:23:00.416 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:23:00.416 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:23:00.416 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:23:00.416 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:23:00.417 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:23:00.417 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:23:00.417 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:23:00.432 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:23:00.433 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:23:00.434 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:23:00.434 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:23:00.438 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:23:00.438 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:23:00.441 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:23:00.441 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:23:00.482 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:23:00.482 DEBUG:teuthology.parallel:result is None 2023-12-19T18:23:00.489 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:23:00.489 DEBUG:teuthology.parallel:result is None 2023-12-19T18:23:00.490 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:23:00.490 DEBUG:teuthology.parallel:result is None 2023-12-19T18:23:00.496 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:23:00.497 DEBUG:teuthology.parallel:result is None 2023-12-19T18:23:00.497 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:23:00.650 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:23:00.655 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-19T18:23:00.658 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:23:00.660 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-19T18:23:00.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.703+0000 7fe0bbfff640 1 Processor -- start 2023-12-19T18:23:00.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.703+0000 7fe0bbfff640 1 -- start start 2023-12-19T18:23:00.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.704+0000 7fe0bbfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0bc15d180 0x7fe0bc0d0270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:00.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.704+0000 7fe0bbfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe0bc0b4e60 0x7fe0bc0b20c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:00.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.704+0000 7fe0bbfff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0bc0d1dc0 con 0x7fe0bc0d2250 2023-12-19T18:23:00.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.704+0000 7fe0bbfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0bc15d560 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.704+0000 7fe0bbfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc0d2690 con 0x7fe0bc15d180 2023-12-19T18:23:00.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.705+0000 7fe0bb7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe0bc0b4e60 0x7fe0bc0b20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:00.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.705+0000 7fe0baffd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0bc15d180 0x7fe0bc0d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:00.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.705+0000 7fe0ba7fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe0bc0d2250 0x7fe0bc0b4750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:60458/0 (socket says 172.21.15.27:60458) 2023-12-19T18:23:00.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.705+0000 7fe0ba7fc640 1 -- 172.21.15.27:0/3152762810 learned_addr learned my addr 172.21.15.27:0/3152762810 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:00.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.705+0000 7fe0b9ffb640 1 -- 172.21.15.27:0/3152762810 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965611339 0 0) 0x7fe0bc0d1dc0 con 0x7fe0bc0d2250 2023-12-19T18:23:00.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.705+0000 7fe0b9ffb640 1 -- 172.21.15.27:0/3152762810 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0a4003520 con 0x7fe0bc0d2250 2023-12-19T18:23:00.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.706+0000 7fe0bb7fe640 1 -- 172.21.15.27:0/3152762810 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0bc15d180 msgr2=0x7fe0bc0d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:00.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.706+0000 7fe0bb7fe640 1 --2- 172.21.15.27:0/3152762810 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0bc15d180 0x7fe0bc0d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.706+0000 7fe0bb7fe640 1 -- 172.21.15.27:0/3152762810 >> v1:172.21.15.27:6789/0 conn(0x7fe0bc0d2250 legacy=0x7fe0bc0b4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:00.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.706+0000 7fe0bb7fe640 1 -- 172.21.15.27:0/3152762810 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0bc0cd5d0 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.706+0000 7fe0baffd640 1 --2- 172.21.15.27:0/3152762810 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0bc15d180 0x7fe0bc0d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:23:00.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.707+0000 7fe0bb7fe640 1 --2- 172.21.15.27:0/3152762810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0bc0b4e60 0x7fe0bc0b20c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009d90 tx=0x7fe0b402bf30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a620608216c1232c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:00.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.707+0000 7fe0b9ffb640 1 -- 172.21.15.27:0/3152762810 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b402cce0 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.707+0000 7fe0b9ffb640 1 -- 172.21.15.27:0/3152762810 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0b402ce40 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.707+0000 7fe0b9ffb640 1 -- 172.21.15.27:0/3152762810 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b4034660 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.708+0000 7fe0bbfff640 1 -- 172.21.15.27:0/3152762810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0bc0b4e60 msgr2=0x7fe0bc0b20c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:00.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.708+0000 7fe0bbfff640 1 --2- 172.21.15.27:0/3152762810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0bc0b4e60 0x7fe0bc0b20c0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fe0b4009d90 tx=0x7fe0b402bf30 comp rx=0 tx=0).stop 2023-12-19T18:23:00.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.708+0000 7fe0bbfff640 1 -- 172.21.15.27:0/3152762810 shutdown_connections 2023-12-19T18:23:00.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.708+0000 7fe0bbfff640 1 --2- 172.21.15.27:0/3152762810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0bc0b4e60 0x7fe0bc0b20c0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.708+0000 7fe0bbfff640 1 --2- 172.21.15.27:0/3152762810 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0bc15d180 0x7fe0bc0d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.708+0000 7fe0bbfff640 1 -- 172.21.15.27:0/3152762810 >> 172.21.15.27:0/3152762810 conn(0x7fe0bc0cf750 msgr2=0x7fe0bc0cfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:00.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.708+0000 7fe0bbfff640 1 -- 172.21.15.27:0/3152762810 shutdown_connections 2023-12-19T18:23:00.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.708+0000 7fe0bbfff640 1 -- 172.21.15.27:0/3152762810 wait complete. 2023-12-19T18:23:00.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.710+0000 7fe0bbfff640 1 Processor -- start 2023-12-19T18:23:00.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.710+0000 7fe0bbfff640 1 -- start start 2023-12-19T18:23:00.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.711+0000 7fe0bbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0b4e60 0x7fe0bc14b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:00.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.711+0000 7fe0bbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0bc0d2250 0x7fe0bc1e6530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:00.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.711+0000 7fe0bbfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0bc15d180 0x7fe0bc15b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:00.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.711+0000 7fe0bbfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc1f59c0 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.711+0000 7fe0bbfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc1f5c60 con 0x7fe0bc0d2250 2023-12-19T18:23:00.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.711+0000 7fe0bbfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0bc1f5f40 con 0x7fe0bc15d180 2023-12-19T18:23:00.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.712+0000 7fe0baffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0b4e60 0x7fe0bc14b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.712+0000 7fe0baffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0b4e60 0x7fe0bc14b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42266/0 (socket says 172.21.15.27:42266) 2023-12-19T18:23:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.712+0000 7fe0baffd640 1 -- 172.21.15.27:0/931993496 learned_addr learned my addr 172.21.15.27:0/931993496 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.712+0000 7fe0ba7fc640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0bc0d2250 0x7fe0bc1e6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.712+0000 7fe0bb7fe640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0bc15d180 0x7fe0bc15b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.712+0000 7fe0baffd640 1 -- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0bc15d180 msgr2=0x7fe0bc15b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.712+0000 7fe0baffd640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0bc15d180 0x7fe0bc15b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.712+0000 7fe0baffd640 1 -- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0bc0d2250 msgr2=0x7fe0bc1e6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.712+0000 7fe0baffd640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0bc0d2250 0x7fe0bc1e6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.712+0000 7fe0baffd640 1 -- 172.21.15.27:0/931993496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac009400 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.712+0000 7fe0bb7fe640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0bc15d180 0x7fe0bc15b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.712+0000 7fe0baffd640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0b4e60 0x7fe0bc14b1f0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00e960 tx=0x7fe0ac00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:00.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.713+0000 7fe0a37fe640 1 -- 172.21.15.27:0/931993496 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac00ca00 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.713+0000 7fe0a37fe640 1 -- 172.21.15.27:0/931993496 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac00cb60 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.713+0000 7fe0a37fe640 1 -- 172.21.15.27:0/931993496 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac013470 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.713+0000 7fe0bbfff640 1 -- 172.21.15.27:0/931993496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b4009a70 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.713+0000 7fe0bbfff640 1 -- 172.21.15.27:0/931993496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0bc1f6500 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.714+0000 7fe0bbfff640 1 -- 172.21.15.27:0/931993496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0bc04e840 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.715+0000 7fe0a37fe640 1 -- 172.21.15.27:0/931993496 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe0ac0135d0 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.716+0000 7fe0a37fe640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe0a806a430 0x7fe0a806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:00.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.716+0000 7fe0a37fe640 1 -- 172.21.15.27:0/931993496 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe0ac0889a0 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.717+0000 7fe0ba7fc640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe0a806a430 0x7fe0a806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:00.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.718+0000 7fe0ba7fc640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe0a806a430 0x7fe0a806c8f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc0cd0d0 tx=0x7fe0b0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:00.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.721+0000 7fe0a37fe640 1 -- 172.21.15.27:0/931993496 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe0ac0541f0 con 0x7fe0bc0b4e60 2023-12-19T18:23:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:00.929+0000 7fe0bbfff640 1 -- 172.21.15.27:0/931993496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe0bc0b53b0 con 0x7fe0bc0b4e60 2023-12-19T18:23:01.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:01.930+0000 7fe0a37fe640 1 -- 172.21.15.27:0/931993496 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v148) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe0ac057810 con 0x7fe0bc0b4e60 2023-12-19T18:23:01.931 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2023-12-19T18:23:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:01.934+0000 7fe0bbfff640 1 -- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe0a806a430 msgr2=0x7fe0a806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:01.934+0000 7fe0bbfff640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe0a806a430 0x7fe0a806c8f0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc0cd0d0 tx=0x7fe0b0009040 comp rx=0 tx=0).stop 2023-12-19T18:23:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:01.934+0000 7fe0bbfff640 1 -- 172.21.15.27:0/931993496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0b4e60 msgr2=0x7fe0bc14b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:01.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:01.934+0000 7fe0bbfff640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0b4e60 0x7fe0bc14b1f0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac00e960 tx=0x7fe0ac00ee20 comp rx=0 tx=0).stop 2023-12-19T18:23:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:01.934+0000 7fe0bbfff640 1 -- 172.21.15.27:0/931993496 shutdown_connections 2023-12-19T18:23:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:01.934+0000 7fe0bbfff640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe0a806a430 0x7fe0a806c8f0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:01.934+0000 7fe0bbfff640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0bc15d180 0x7fe0bc15b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:01.934+0000 7fe0bbfff640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0bc0d2250 0x7fe0bc1e6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:01.934+0000 7fe0bbfff640 1 --2- 172.21.15.27:0/931993496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0bc0b4e60 0x7fe0bc14b1f0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:01.934+0000 7fe0bbfff640 1 -- 172.21.15.27:0/931993496 >> 172.21.15.27:0/931993496 conn(0x7fe0bc0cf750 msgr2=0x7fe0bc15c180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:01.934+0000 7fe0bbfff640 1 -- 172.21.15.27:0/931993496 shutdown_connections 2023-12-19T18:23:01.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:01.934+0000 7fe0bbfff640 1 -- 172.21.15.27:0/931993496 wait complete. 2023-12-19T18:23:01.955 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:23:02.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.184+0000 7f1ecc9ef640 1 Processor -- start 2023-12-19T18:23:02.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.184+0000 7f1ecc9ef640 1 -- start start 2023-12-19T18:23:02.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.186+0000 7f1ecc9ef640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1ec8160c10 0x7f1ec8160ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.186+0000 7f1ecc9ef640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec8161530 0x7f1ec815db40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.186+0000 7f1ecc9ef640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec80719a0 con 0x7f1ec806e150 2023-12-19T18:23:02.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.186+0000 7f1ecc9ef640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec815aef0 con 0x7f1ec8160c10 2023-12-19T18:23:02.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.186+0000 7f1ecc9ef640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec806df30 con 0x7f1ec8161530 2023-12-19T18:23:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.187+0000 7f1ec77fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1ec8160c10 0x7f1ec8160ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.187+0000 7f1ec77fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1ec8160c10 0x7f1ec8160ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:56276/0 (socket says 172.21.15.27:56276) 2023-12-19T18:23:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.187+0000 7f1ec77fe640 1 -- 172.21.15.27:0/708834583 learned_addr learned my addr 172.21.15.27:0/708834583 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.187+0000 7f1ec7fff640 1 --2- 172.21.15.27:0/708834583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec8161530 0x7f1ec815db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.188+0000 7f1ec7fff640 1 -- 172.21.15.27:0/708834583 >> v2:172.21.15.192:3300/0 conn(0x7f1ec8160c10 msgr2=0x7f1ec8160ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.188+0000 7f1ec7fff640 1 --2- 172.21.15.27:0/708834583 >> v2:172.21.15.192:3300/0 conn(0x7f1ec8160c10 0x7f1ec8160ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.188+0000 7f1ec7fff640 1 -- 172.21.15.27:0/708834583 >> v1:172.21.15.192:6789/0 conn(0x7f1ec806e150 legacy=0x7f1ec815d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.188+0000 7f1ec7fff640 1 -- 172.21.15.27:0/708834583 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec80cce80 con 0x7f1ec8161530 2023-12-19T18:23:02.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.188+0000 7f1ec77fe640 1 --2- 172.21.15.27:0/708834583 >> v2:172.21.15.192:3300/0 conn(0x7f1ec8160c10 0x7f1ec8160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:23:02.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.189+0000 7f1ec7fff640 1 --2- 172.21.15.27:0/708834583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec8161530 0x7f1ec815db40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009d90 tx=0x7f1ec002bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5638a476e889ce70 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.190+0000 7f1ec6ffd640 1 -- 172.21.15.27:0/708834583 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec002cb30 con 0x7f1ec8161530 2023-12-19T18:23:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.190+0000 7f1ec6ffd640 1 -- 172.21.15.27:0/708834583 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ec002cc90 con 0x7f1ec8161530 2023-12-19T18:23:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.190+0000 7f1ec6ffd640 1 -- 172.21.15.27:0/708834583 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ec0034440 con 0x7f1ec8161530 2023-12-19T18:23:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.191+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/708834583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec8161530 msgr2=0x7f1ec815db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.191+0000 7f1ecc9ef640 1 --2- 172.21.15.27:0/708834583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec8161530 0x7f1ec815db40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1ec0009d90 tx=0x7f1ec002bd80 comp rx=0 tx=0).stop 2023-12-19T18:23:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.191+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/708834583 shutdown_connections 2023-12-19T18:23:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.191+0000 7f1ecc9ef640 1 --2- 172.21.15.27:0/708834583 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec8161530 0x7f1ec815db40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.191+0000 7f1ecc9ef640 1 --2- 172.21.15.27:0/708834583 >> v2:172.21.15.192:3300/0 conn(0x7f1ec8160c10 0x7f1ec8160ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.191+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/708834583 >> 172.21.15.27:0/708834583 conn(0x7f1ec80cbb70 msgr2=0x7f1ec80cbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.191+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/708834583 shutdown_connections 2023-12-19T18:23:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.191+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/708834583 wait complete. 2023-12-19T18:23:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.193+0000 7f1ecc9ef640 1 Processor -- start 2023-12-19T18:23:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.193+0000 7f1ecc9ef640 1 -- start start 2023-12-19T18:23:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.194+0000 7f1ecc9ef640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec806e150 0x7f1ec814c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.194+0000 7f1ecc9ef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8160c10 0x7f1ec80c52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.194+0000 7f1ecc9ef640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec8161530 0x7f1ec815d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.194+0000 7f1ecc9ef640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c2fd0 con 0x7f1ec8160c10 2023-12-19T18:23:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.194+0000 7f1ecc9ef640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80b3550 con 0x7f1ec8161530 2023-12-19T18:23:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.194+0000 7f1ecc9ef640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec80c2cf0 con 0x7f1ec806e150 2023-12-19T18:23:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.194+0000 7f1ebffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8160c10 0x7f1ec80c52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.195+0000 7f1ebffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8160c10 0x7f1ec80c52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52700/0 (socket says 172.21.15.27:52700) 2023-12-19T18:23:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.195+0000 7f1ebffff640 1 -- 172.21.15.27:0/2012331965 learned_addr learned my addr 172.21.15.27:0/2012331965 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.195+0000 7f1ec7fff640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec8161530 0x7f1ec815d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.195+0000 7f1ec77fe640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec806e150 0x7f1ec814c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.195+0000 7f1ebffff640 1 -- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec806e150 msgr2=0x7f1ec814c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.195+0000 7f1ebffff640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec806e150 0x7f1ec814c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.195+0000 7f1ebffff640 1 -- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec8161530 msgr2=0x7f1ec815d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.195+0000 7f1ebffff640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec8161530 0x7f1ec815d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.195+0000 7f1ebffff640 1 -- 172.21.15.27:0/2012331965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb80094d0 con 0x7f1ec8160c10 2023-12-19T18:23:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.195+0000 7f1ec7fff640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec8161530 0x7f1ec815d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:23:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.195+0000 7f1ebffff640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8160c10 0x7f1ec80c52d0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c400 tx=0x7f1eb000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.196+0000 7f1ec4ff9640 1 -- 172.21.15.27:0/2012331965 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00052d0 con 0x7f1ec8160c10 2023-12-19T18:23:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.196+0000 7f1ec4ff9640 1 -- 172.21.15.27:0/2012331965 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0005430 con 0x7f1ec8160c10 2023-12-19T18:23:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.196+0000 7f1ec4ff9640 1 -- 172.21.15.27:0/2012331965 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0003830 con 0x7f1ec8160c10 2023-12-19T18:23:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.196+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/2012331965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec0009a70 con 0x7f1ec8160c10 2023-12-19T18:23:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.196+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/2012331965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec80b31e0 con 0x7f1ec8160c10 2023-12-19T18:23:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.197+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/2012331965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec80b2d60 con 0x7f1ec8160c10 2023-12-19T18:23:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.198+0000 7f1ec4ff9640 1 -- 172.21.15.27:0/2012331965 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1eb0003990 con 0x7f1ec8160c10 2023-12-19T18:23:02.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.200+0000 7f1ec4ff9640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ea806a430 0x7f1ea806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.200+0000 7f1ec4ff9640 1 -- 172.21.15.27:0/2012331965 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1eb0087c90 con 0x7f1ec8160c10 2023-12-19T18:23:02.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.200+0000 7f1ec77fe640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ea806a430 0x7f1ea806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.201+0000 7f1ec77fe640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ea806a430 0x7f1ea806c8f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009c50 tx=0x7f1eb8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:02.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.205+0000 7f1ec4ff9640 1 -- 172.21.15.27:0/2012331965 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1ec80b2d60 con 0x7f1ec8160c10 2023-12-19T18:23:02.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.472+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/2012331965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1ec80b2d60 con 0x7f1ec8160c10 2023-12-19T18:23:02.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.474+0000 7f1ec4ff9640 1 -- 172.21.15.27:0/2012331965 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1ec80b2d60 con 0x7f1ec8160c10 2023-12-19T18:23:02.474 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:02.474 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:23:02.474 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2023-12-19T18:23:02.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.474 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2023-12-19T18:23:02.474 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFl+4ruHBAAXwbpmT43SmlvNbuZj5e+bQ==", 2023-12-19T18:23:02.474 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.474 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:23:02.474 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:23:02.474 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmZyUBhAAG3kZkXvJIo7N9f1j+DPCSQ==", 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.475 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlChNOMBAAX4GV3iIGkSQdAsYu9oCPDw==", 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlTmFXFhAABA0NMUxKBV1ExAhYc5RFbQ==", 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:23:02.476 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFluleFIBAAs1goFVMOX4CbjwFhg0KoBg==", 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.477 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFlInW3MxAAI4hY1GBxXYXDswKKrEyDTg==", 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFlAWZ5CRAA82/2r9/sT0L9VlGHbXksQw==", 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.478 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFl1IvaGRAAqVacDDPLx+NqBOGp6MeX2w==", 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlr0BBHRAA+7gfs/RG6DtABnYPg059Tw==", 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.479 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlXZbELRAAKD0qOkhk7skfc4xcAC8Lhg==", 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.480 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlF07MARAA2TJJ+5C1HIU+SVhqmKaVHA==", 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlISPGEhAAdFIRRkXlLyOx/oMZ4MgL9Q==", 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:23:02.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmNM9JRAARgS20O8zpm6H70WDv+lgGw==", 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlWoZ8AhAADWvKVzeDhDGHtYNUFIOWsA==", 2023-12-19T18:23:02.482 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFllWFVMhAAD9ebDOppHTODVQjg6ppP3g==", 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:23:02.483 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlwbIQDRAAt/h55zBNCEId0vkYSUQ3TQ==", 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2023-12-19T18:23:02.484 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB53YFloCb6CxAAmZQyz5XOEqDOOiROJHLzxQ==", 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFltBhxDxAAiAS9CYPY86IMmJ7NGgA2lw==", 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:23:02.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.486 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:02.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:02.486 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2023-12-19T18:23:02.486 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlUnAhBBAAi++W9XK3o3Y/185hB3LuIg==", 2023-12-19T18:23:02.486 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:23:02.486 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:23:02.486 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:23:02.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:23:02.486 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.486 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:02.486 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:02.487 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:23:02.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.476+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ea806a430 msgr2=0x7f1ea806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:02.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.476+0000 7f1ecc9ef640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ea806a430 0x7f1ea806c8f0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009c50 tx=0x7f1eb8008040 comp rx=0 tx=0).stop 2023-12-19T18:23:02.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.476+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/2012331965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8160c10 msgr2=0x7f1ec80c52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:02.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.476+0000 7f1ecc9ef640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8160c10 0x7f1ec80c52d0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c400 tx=0x7f1eb000c8c0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.477+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/2012331965 shutdown_connections 2023-12-19T18:23:02.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.477+0000 7f1ecc9ef640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1ea806a430 0x7f1ea806c8f0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.477+0000 7f1ecc9ef640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1ec8161530 0x7f1ec815d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.477+0000 7f1ecc9ef640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec8160c10 0x7f1ec80c52d0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.477+0000 7f1ecc9ef640 1 --2- 172.21.15.27:0/2012331965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1ec806e150 0x7f1ec814c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.477+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/2012331965 >> 172.21.15.27:0/2012331965 conn(0x7f1ec80cbb70 msgr2=0x7f1ec80b00d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:02.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.477+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/2012331965 shutdown_connections 2023-12-19T18:23:02.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.477+0000 7f1ecc9ef640 1 -- 172.21.15.27:0/2012331965 wait complete. 2023-12-19T18:23:02.497 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:23:02.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.725+0000 7ff504ad5640 1 Processor -- start 2023-12-19T18:23:02.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.725+0000 7ff504ad5640 1 -- start start 2023-12-19T18:23:02.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.726+0000 7ff504ad5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff5000cdf90 0x7ff5000ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.726+0000 7ff504ad5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5000d1e00 0x7ff5000d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.727+0000 7ff504ad5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5000d1970 con 0x7ff50015d240 2023-12-19T18:23:02.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.727+0000 7ff504ad5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5000ce960 con 0x7ff5000cdf90 2023-12-19T18:23:02.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.727+0000 7ff504ad5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50006a5e0 con 0x7ff5000d1e00 2023-12-19T18:23:02.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.727+0000 7ff4fe59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff50015d240 0x7ff5001618d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51720/0 (socket says 172.21.15.27:51720) 2023-12-19T18:23:02.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.727+0000 7ff4fe59c640 1 -- 172.21.15.27:0/889587294 learned_addr learned my addr 172.21.15.27:0/889587294 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:02.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.727+0000 7ff4f7fff640 1 --2- 172.21.15.27:0/889587294 >> v2:172.21.15.192:3300/0 conn(0x7ff5000cdf90 0x7ff5000ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.727+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465565675 0 0) 0x7ff5000d1970 con 0x7ff50015d240 2023-12-19T18:23:02.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.727+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4ec003520 con 0x7ff50015d240 2023-12-19T18:23:02.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.727+0000 7ff4fed9d640 1 --2- 172.21.15.27:0/889587294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5000d1e00 0x7ff5000d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3506354651 0 0) 0x7ff4ec003520 con 0x7ff50015d240 2023-12-19T18:23:02.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5000d1e00 msgr2=0x7ff5000d2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:02.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 --2- 172.21.15.27:0/889587294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5000d1e00 0x7ff5000d2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 >> v2:172.21.15.192:3300/0 conn(0x7ff5000cdf90 msgr2=0x7ff5000ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:02.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 --2- 172.21.15.27:0/889587294 >> v2:172.21.15.192:3300/0 conn(0x7ff5000cdf90 0x7ff5000ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50006a880 con 0x7ff50015d240 2023-12-19T18:23:02.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7ff4f0001670 con 0x7ff50015d240 2023-12-19T18:23:02.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 >> v1:172.21.15.27:6789/0 conn(0x7ff50015d240 legacy=0x7ff5001618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:02.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 --2- 172.21.15.27:0/889587294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4ec003970 0x7ff4ec003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 --2- 172.21.15.27:0/889587294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4ec0047c0 0x7ff4ec004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 --2- 172.21.15.27:0/889587294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5000d1e00 0x7ff4ec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000ce960 con 0x7ff4ec003970 2023-12-19T18:23:02.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50006a5e0 con 0x7ff4ec0047c0 2023-12-19T18:23:02.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4ec00b9b0 con 0x7ff5000d1e00 2023-12-19T18:23:02.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.728+0000 7ff4fe59c640 1 --2- 172.21.15.27:0/889587294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4ec003970 0x7ff4ec003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.729+0000 7ff4f7fff640 1 --2- 172.21.15.27:0/889587294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4ec0047c0 0x7ff4ec004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.729+0000 7ff4fed9d640 1 --2- 172.21.15.27:0/889587294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5000d1e00 0x7ff4ec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.729+0000 7ff4fe59c640 1 -- 172.21.15.27:0/889587294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5000d1e00 msgr2=0x7ff4ec005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:02.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.729+0000 7ff4fe59c640 1 --2- 172.21.15.27:0/889587294 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5000d1e00 0x7ff4ec005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.729+0000 7ff4fe59c640 1 -- 172.21.15.27:0/889587294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4ec0047c0 msgr2=0x7ff4ec004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:02.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.729+0000 7ff4fe59c640 1 --2- 172.21.15.27:0/889587294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4ec0047c0 0x7ff4ec004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.729+0000 7ff4fe59c640 1 -- 172.21.15.27:0/889587294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4ec003520 con 0x7ff4ec003970 2023-12-19T18:23:02.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.729+0000 7ff4f7fff640 1 --2- 172.21.15.27:0/889587294 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff4ec0047c0 0x7ff4ec004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:23:02.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.730+0000 7ff4fe59c640 1 --2- 172.21.15.27:0/889587294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4ec003970 0x7ff4ec003d90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000c160 tx=0x7ff4ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d36989be9e01b5e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:02.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.731+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0005ad0 con 0x7ff4ec003970 2023-12-19T18:23:02.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.731+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f0005c30 con 0x7ff4ec003970 2023-12-19T18:23:02.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.731+0000 7ff4fdd9b640 1 -- 172.21.15.27:0/889587294 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f002c810 con 0x7ff4ec003970 2023-12-19T18:23:02.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.731+0000 7ff504ad5640 1 -- 172.21.15.27:0/889587294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4ec003970 msgr2=0x7ff4ec003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:02.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.731+0000 7ff504ad5640 1 --2- 172.21.15.27:0/889587294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4ec003970 0x7ff4ec003d90 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7ff4f000c160 tx=0x7ff4ec0057f0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.731+0000 7ff4fed9d640 1 -- 172.21.15.27:0/889587294 reap_dead start 2023-12-19T18:23:02.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.731+0000 7ff504ad5640 1 -- 172.21.15.27:0/889587294 shutdown_connections 2023-12-19T18:23:02.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.731+0000 7ff504ad5640 1 -- 172.21.15.27:0/889587294 >> 172.21.15.27:0/889587294 conn(0x7ff5000d31e0 msgr2=0x7ff5000d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:02.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.732+0000 7ff504ad5640 1 -- 172.21.15.27:0/889587294 shutdown_connections 2023-12-19T18:23:02.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.732+0000 7ff504ad5640 1 -- 172.21.15.27:0/889587294 wait complete. 2023-12-19T18:23:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.733+0000 7ff504ad5640 1 Processor -- start 2023-12-19T18:23:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.734+0000 7ff504ad5640 1 -- start start 2023-12-19T18:23:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff504ad5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4ec003970 0x7ff5000b5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff504ad5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff50014adf0 0x7ff50015adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff504ad5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5000acb90 0x7ff5000ad010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff504ad5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5000ad550 con 0x7ff4ec003970 2023-12-19T18:23:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff504ad5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff500148480 con 0x7ff50014adf0 2023-12-19T18:23:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff504ad5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50014b270 con 0x7ff5000acb90 2023-12-19T18:23:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff4fe59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4ec003970 0x7ff5000b5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff4fe59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4ec003970 0x7ff5000b5640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52722/0 (socket says 172.21.15.27:52722) 2023-12-19T18:23:02.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff4fe59c640 1 -- 172.21.15.27:0/2989870850 learned_addr learned my addr 172.21.15.27:0/2989870850 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff4f7fff640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff50014adf0 0x7ff50015adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff4fe59c640 1 -- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5000acb90 msgr2=0x7ff5000ad010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff4fe59c640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5000acb90 0x7ff5000ad010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff4fe59c640 1 -- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff50014adf0 msgr2=0x7ff50015adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff4fe59c640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff50014adf0 0x7ff50015adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff4fe59c640 1 -- 172.21.15.27:0/2989870850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50015b300 con 0x7ff4ec003970 2023-12-19T18:23:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.735+0000 7ff4f7fff640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff50014adf0 0x7ff50015adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.736+0000 7ff4fe59c640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4ec003970 0x7ff5000b5640 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff4f002c3b0 tx=0x7ff4f0000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.736+0000 7ff4f6ffd640 1 -- 172.21.15.27:0/2989870850 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f003c040 con 0x7ff4ec003970 2023-12-19T18:23:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.736+0000 7ff4f6ffd640 1 -- 172.21.15.27:0/2989870850 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4f002cc20 con 0x7ff4ec003970 2023-12-19T18:23:02.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.736+0000 7ff504ad5640 1 -- 172.21.15.27:0/2989870850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50015be30 con 0x7ff4ec003970 2023-12-19T18:23:02.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.736+0000 7ff4f6ffd640 1 -- 172.21.15.27:0/2989870850 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4f0007070 con 0x7ff4ec003970 2023-12-19T18:23:02.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.736+0000 7ff504ad5640 1 -- 172.21.15.27:0/2989870850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50015b870 con 0x7ff4ec003970 2023-12-19T18:23:02.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.738+0000 7ff504ad5640 1 -- 172.21.15.27:0/2989870850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5000d1580 con 0x7ff4ec003970 2023-12-19T18:23:02.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.739+0000 7ff4f6ffd640 1 -- 172.21.15.27:0/2989870850 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff4f0032070 con 0x7ff4ec003970 2023-12-19T18:23:02.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.740+0000 7ff4f6ffd640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff4e006a460 0x7ff4e006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:02.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.740+0000 7ff4f6ffd640 1 -- 172.21.15.27:0/2989870850 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff4f00a9a60 con 0x7ff4ec003970 2023-12-19T18:23:02.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.740+0000 7ff4f7fff640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff4e006a460 0x7ff4e006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:02.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.742+0000 7ff4f7fff640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff4e006a460 0x7ff4e006c920 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff5000aa230 tx=0x7ff4e8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:02.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.744+0000 7ff4f6ffd640 1 -- 172.21.15.27:0/2989870850 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff4f0075390 con 0x7ff4ec003970 2023-12-19T18:23:02.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.940+0000 7ff504ad5640 1 -- 172.21.15.27:0/2989870850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff500068e90 con 0x7ff4ec003970 2023-12-19T18:23:02.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.941+0000 7ff4f6ffd640 1 -- 172.21.15.27:0/2989870850 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) v1 ==== 74+0+12400 (secure 0 0 0) 0x7ff4f0046020 con 0x7ff4ec003970 2023-12-19T18:23:02.941 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:02.941 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":148,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:23:01.916021+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":139,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:23:02.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.943+0000 7ff504ad5640 1 -- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff4e006a460 msgr2=0x7ff4e006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:02.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.943+0000 7ff504ad5640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff4e006a460 0x7ff4e006c920 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ff5000aa230 tx=0x7ff4e8008040 comp rx=0 tx=0).stop 2023-12-19T18:23:02.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.944+0000 7ff504ad5640 1 -- 172.21.15.27:0/2989870850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4ec003970 msgr2=0x7ff5000b5640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:02.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.944+0000 7ff504ad5640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4ec003970 0x7ff5000b5640 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7ff4f002c3b0 tx=0x7ff4f0000fe0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.944+0000 7ff504ad5640 1 -- 172.21.15.27:0/2989870850 shutdown_connections 2023-12-19T18:23:02.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.944+0000 7ff504ad5640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff4e006a460 0x7ff4e006c920 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.944+0000 7ff504ad5640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5000acb90 0x7ff5000ad010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.944+0000 7ff504ad5640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff50014adf0 0x7ff50015adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.944+0000 7ff504ad5640 1 --2- 172.21.15.27:0/2989870850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff4ec003970 0x7ff5000b5640 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:02.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.944+0000 7ff504ad5640 1 -- 172.21.15.27:0/2989870850 >> 172.21.15.27:0/2989870850 conn(0x7ff5000d31e0 msgr2=0x7ff5000cc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:02.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.944+0000 7ff504ad5640 1 -- 172.21.15.27:0/2989870850 shutdown_connections 2023-12-19T18:23:02.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:02.944+0000 7ff504ad5640 1 -- 172.21.15.27:0/2989870850 wait complete. 2023-12-19T18:23:02.963 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:23:02.964 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:23:03.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.200+0000 7fa2448a0640 1 Processor -- start 2023-12-19T18:23:03.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.200+0000 7fa2448a0640 1 -- start start 2023-12-19T18:23:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.202+0000 7fa2448a0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa240151c90 0x7fa24006b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.202+0000 7fa2448a0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa24015e9d0 0x7fa2400cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.202+0000 7fa2448a0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa24015b3e0 con 0x7fa240152070 2023-12-19T18:23:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.202+0000 7fa2448a0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa24015eed0 con 0x7fa24015e9d0 2023-12-19T18:23:03.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.202+0000 7fa2448a0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa24015f1b0 con 0x7fa240151c90 2023-12-19T18:23:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.202+0000 7fa237fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa24015e9d0 0x7fa2400cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.202+0000 7fa237fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa24015e9d0 0x7fa2400cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52726/0 (socket says 172.21.15.27:52726) 2023-12-19T18:23:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.202+0000 7fa237fff640 1 -- 172.21.15.27:0/858795540 learned_addr learned my addr 172.21.15.27:0/858795540 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.203+0000 7fa23ed9d640 1 --2- 172.21.15.27:0/858795540 >> v2:172.21.15.192:3300/0 conn(0x7fa240151c90 0x7fa24006b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.203+0000 7fa237fff640 1 -- 172.21.15.27:0/858795540 >> v2:172.21.15.192:3300/0 conn(0x7fa240151c90 msgr2=0x7fa24006b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.203+0000 7fa237fff640 1 --2- 172.21.15.27:0/858795540 >> v2:172.21.15.192:3300/0 conn(0x7fa240151c90 0x7fa24006b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.203+0000 7fa237fff640 1 -- 172.21.15.27:0/858795540 >> v1:172.21.15.192:6789/0 conn(0x7fa240152070 legacy=0x7fa24015b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.203+0000 7fa237fff640 1 -- 172.21.15.27:0/858795540 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24015f490 con 0x7fa24015e9d0 2023-12-19T18:23:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.204+0000 7fa237fff640 1 --2- 172.21.15.27:0/858795540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24015e9d0 0x7fa2400cee40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa228009870 tx=0x7fa22802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89fdcab9c21bdf35 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.204+0000 7fa23e59c640 1 -- 172.21.15.27:0/858795540 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2280317e0 con 0x7fa24015e9d0 2023-12-19T18:23:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.204+0000 7fa23e59c640 1 -- 172.21.15.27:0/858795540 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa228031940 con 0x7fa24015e9d0 2023-12-19T18:23:03.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.204+0000 7fa23e59c640 1 -- 172.21.15.27:0/858795540 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa228031cd0 con 0x7fa24015e9d0 2023-12-19T18:23:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.205+0000 7fa2448a0640 1 -- 172.21.15.27:0/858795540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24015e9d0 msgr2=0x7fa2400cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.205+0000 7fa2448a0640 1 --2- 172.21.15.27:0/858795540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24015e9d0 0x7fa2400cee40 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7fa228009870 tx=0x7fa22802b880 comp rx=0 tx=0).stop 2023-12-19T18:23:03.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.205+0000 7fa2448a0640 1 -- 172.21.15.27:0/858795540 shutdown_connections 2023-12-19T18:23:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.205+0000 7fa2448a0640 1 --2- 172.21.15.27:0/858795540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24015e9d0 0x7fa2400cee40 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.205+0000 7fa2448a0640 1 --2- 172.21.15.27:0/858795540 >> v2:172.21.15.192:3300/0 conn(0x7fa240151c90 0x7fa24006b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.205+0000 7fa2448a0640 1 -- 172.21.15.27:0/858795540 >> 172.21.15.27:0/858795540 conn(0x7fa24006a060 msgr2=0x7fa24006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.205+0000 7fa2448a0640 1 -- 172.21.15.27:0/858795540 shutdown_connections 2023-12-19T18:23:03.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.205+0000 7fa2448a0640 1 -- 172.21.15.27:0/858795540 wait complete. 2023-12-19T18:23:03.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.208+0000 7fa2448a0640 1 Processor -- start 2023-12-19T18:23:03.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.208+0000 7fa2448a0640 1 -- start start 2023-12-19T18:23:03.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.209+0000 7fa2448a0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa240151c90 0x7fa2400b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:03.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.209+0000 7fa2448a0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa240152070 0x7fa24014e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:03.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.209+0000 7fa2448a0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24015e9d0 0x7fa2400cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:03.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.209+0000 7fa2448a0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2400b2dc0 con 0x7fa24015e9d0 2023-12-19T18:23:03.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.209+0000 7fa2448a0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2400b0790 con 0x7fa240152070 2023-12-19T18:23:03.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.209+0000 7fa2448a0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2400af1b0 con 0x7fa240151c90 2023-12-19T18:23:03.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.210+0000 7fa23f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24015e9d0 0x7fa2400cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:03.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.210+0000 7fa23f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24015e9d0 0x7fa2400cd4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52736/0 (socket says 172.21.15.27:52736) 2023-12-19T18:23:03.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.210+0000 7fa23f59e640 1 -- 172.21.15.27:0/2411372970 learned_addr learned my addr 172.21.15.27:0/2411372970 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:03.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.210+0000 7fa23ed9d640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa240151c90 0x7fa2400b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:03.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.210+0000 7fa237fff640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa240152070 0x7fa24014e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:03.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.210+0000 7fa23f59e640 1 -- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa240151c90 msgr2=0x7fa2400b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:03.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.210+0000 7fa23f59e640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa240151c90 0x7fa2400b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.210+0000 7fa23f59e640 1 -- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa240152070 msgr2=0x7fa24014e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:03.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.210+0000 7fa23f59e640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa240152070 0x7fa24014e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.210+0000 7fa23f59e640 1 -- 172.21.15.27:0/2411372970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2280094d0 con 0x7fa24015e9d0 2023-12-19T18:23:03.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.211+0000 7fa23f59e640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24015e9d0 0x7fa2400cd4f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa23800c780 tx=0x7fa23800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:03.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.211+0000 7fa2377fe640 1 -- 172.21.15.27:0/2411372970 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238003d80 con 0x7fa24015e9d0 2023-12-19T18:23:03.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.211+0000 7fa2377fe640 1 -- 172.21.15.27:0/2411372970 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa23800ce30 con 0x7fa24015e9d0 2023-12-19T18:23:03.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.211+0000 7fa2377fe640 1 -- 172.21.15.27:0/2411372970 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa238005290 con 0x7fa24015e9d0 2023-12-19T18:23:03.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.211+0000 7fa2448a0640 1 -- 172.21.15.27:0/2411372970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2400af490 con 0x7fa24015e9d0 2023-12-19T18:23:03.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.211+0000 7fa2448a0640 1 -- 172.21.15.27:0/2411372970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2400afa50 con 0x7fa24015e9d0 2023-12-19T18:23:03.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.212+0000 7fa2448a0640 1 -- 172.21.15.27:0/2411372970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa24004e840 con 0x7fa24015e9d0 2023-12-19T18:23:03.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.213+0000 7fa2377fe640 1 -- 172.21.15.27:0/2411372970 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa2380053f0 con 0x7fa24015e9d0 2023-12-19T18:23:03.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.214+0000 7fa2377fe640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa22406a430 0x7fa22406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:03.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.215+0000 7fa2377fe640 1 -- 172.21.15.27:0/2411372970 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa238087af0 con 0x7fa24015e9d0 2023-12-19T18:23:03.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.215+0000 7fa23ed9d640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa22406a430 0x7fa22406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:03.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.217+0000 7fa23ed9d640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa22406a430 0x7fa22406c8f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa230006fd0 tx=0x7fa230008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:03.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.219+0000 7fa2377fe640 1 -- 172.21.15.27:0/2411372970 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa238053390 con 0x7fa24015e9d0 2023-12-19T18:23:03.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.411+0000 7fa2448a0640 1 -- 172.21.15.27:0/2411372970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2400ccb70 con 0x7fa24015e9d0 2023-12-19T18:23:03.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.411+0000 7fa2377fe640 1 -- 172.21.15.27:0/2411372970 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa2380170a0 con 0x7fa24015e9d0 2023-12-19T18:23:03.411 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:23:03.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.414+0000 7fa2448a0640 1 -- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa22406a430 msgr2=0x7fa22406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.414+0000 7fa2448a0640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa22406a430 0x7fa22406c8f0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fa230006fd0 tx=0x7fa230008040 comp rx=0 tx=0).stop 2023-12-19T18:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.414+0000 7fa2448a0640 1 -- 172.21.15.27:0/2411372970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24015e9d0 msgr2=0x7fa2400cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.414+0000 7fa2448a0640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24015e9d0 0x7fa2400cd4f0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7fa23800c780 tx=0x7fa23800cc40 comp rx=0 tx=0).stop 2023-12-19T18:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.414+0000 7fa2448a0640 1 -- 172.21.15.27:0/2411372970 shutdown_connections 2023-12-19T18:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.414+0000 7fa2448a0640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa22406a430 0x7fa22406c8f0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.415+0000 7fa2448a0640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa24015e9d0 0x7fa2400cd4f0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.415+0000 7fa2448a0640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa240152070 0x7fa24014e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.415+0000 7fa2448a0640 1 --2- 172.21.15.27:0/2411372970 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa240151c90 0x7fa2400b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.415+0000 7fa2448a0640 1 -- 172.21.15.27:0/2411372970 >> 172.21.15.27:0/2411372970 conn(0x7fa24006a060 msgr2=0x7fa24014fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.415+0000 7fa2448a0640 1 -- 172.21.15.27:0/2411372970 shutdown_connections 2023-12-19T18:23:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.415+0000 7fa2448a0640 1 -- 172.21.15.27:0/2411372970 wait complete. 2023-12-19T18:23:03.433 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:23:03.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.671+0000 7f5d1759e640 1 Processor -- start 2023-12-19T18:23:03.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.671+0000 7f5d1759e640 1 -- start start 2023-12-19T18:23:03.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.672+0000 7f5d1759e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5d180b49d0 0x7f5d18160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:03.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.673+0000 7f5d1759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5d180b89b0 0x7f5d1815c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:03.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.673+0000 7f5d1759e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d180b5ca0 con 0x7f5d180b4db0 2023-12-19T18:23:03.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.673+0000 7f5d1759e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d18161800 con 0x7f5d180b89b0 2023-12-19T18:23:03.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.673+0000 7f5d1759e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d18161240 con 0x7f5d180b49d0 2023-12-19T18:23:03.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.673+0000 7f5d1659c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5d180b49d0 0x7f5d18160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:03.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.673+0000 7f5d15d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5d180b89b0 0x7f5d1815c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:03.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.673+0000 7f5d15d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5d180b89b0 0x7f5d1815c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52752/0 (socket says 172.21.15.27:52752) 2023-12-19T18:23:03.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.673+0000 7f5d15d9b640 1 -- 172.21.15.27:0/856077336 learned_addr learned my addr 172.21.15.27:0/856077336 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:03.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.674+0000 7f5d15d9b640 1 -- 172.21.15.27:0/856077336 >> v2:172.21.15.192:3300/0 conn(0x7f5d180b49d0 msgr2=0x7f5d18160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:03.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.674+0000 7f5d15d9b640 1 --2- 172.21.15.27:0/856077336 >> v2:172.21.15.192:3300/0 conn(0x7f5d180b49d0 0x7f5d18160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.674+0000 7f5d15d9b640 1 -- 172.21.15.27:0/856077336 >> v1:172.21.15.192:6789/0 conn(0x7f5d180b4db0 legacy=0x7f5d1806bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:03.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.674+0000 7f5d15d9b640 1 -- 172.21.15.27:0/856077336 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d18161520 con 0x7f5d180b89b0 2023-12-19T18:23:03.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.675+0000 7f5d15d9b640 1 --2- 172.21.15.27:0/856077336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d180b89b0 0x7f5d1815c880 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f5d1806cd00 tx=0x7f5d0002ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55351d88cf79029a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:03.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.675+0000 7f5d1559a640 1 -- 172.21.15.27:0/856077336 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0002f850 con 0x7f5d180b89b0 2023-12-19T18:23:03.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.675+0000 7f5d1559a640 1 -- 172.21.15.27:0/856077336 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d0002f9b0 con 0x7f5d180b89b0 2023-12-19T18:23:03.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.675+0000 7f5d1559a640 1 -- 172.21.15.27:0/856077336 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d0002fd40 con 0x7f5d180b89b0 2023-12-19T18:23:03.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.676+0000 7f5d1759e640 1 -- 172.21.15.27:0/856077336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d180b89b0 msgr2=0x7f5d1815c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:03.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.676+0000 7f5d1759e640 1 --2- 172.21.15.27:0/856077336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d180b89b0 0x7f5d1815c880 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f5d1806cd00 tx=0x7f5d0002ba30 comp rx=0 tx=0).stop 2023-12-19T18:23:03.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.676+0000 7f5d1759e640 1 -- 172.21.15.27:0/856077336 shutdown_connections 2023-12-19T18:23:03.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.676+0000 7f5d1759e640 1 --2- 172.21.15.27:0/856077336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d180b89b0 0x7f5d1815c880 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.676+0000 7f5d1759e640 1 --2- 172.21.15.27:0/856077336 >> v2:172.21.15.192:3300/0 conn(0x7f5d180b49d0 0x7f5d18160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.676+0000 7f5d1759e640 1 -- 172.21.15.27:0/856077336 >> 172.21.15.27:0/856077336 conn(0x7f5d180cf760 msgr2=0x7f5d180cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:03.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.676+0000 7f5d1759e640 1 -- 172.21.15.27:0/856077336 shutdown_connections 2023-12-19T18:23:03.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.676+0000 7f5d1759e640 1 -- 172.21.15.27:0/856077336 wait complete. 2023-12-19T18:23:03.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.678+0000 7f5d1759e640 1 Processor -- start 2023-12-19T18:23:03.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.678+0000 7f5d1759e640 1 -- start start 2023-12-19T18:23:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.679+0000 7f5d1759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d180b49d0 0x7f5d181f6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.679+0000 7f5d1759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d180b4db0 0x7f5d1806be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.679+0000 7f5d1759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d180b89b0 0x7f5d1815c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.679+0000 7f5d1759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d181f6ea0 con 0x7f5d180b89b0 2023-12-19T18:23:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.679+0000 7f5d1759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d181f5f10 con 0x7f5d180b49d0 2023-12-19T18:23:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.679+0000 7f5d1759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d1815d010 con 0x7f5d180b4db0 2023-12-19T18:23:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.680+0000 7f5d16d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d180b89b0 0x7f5d1815c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:03.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.680+0000 7f5d16d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d180b89b0 0x7f5d1815c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52758/0 (socket says 172.21.15.27:52758) 2023-12-19T18:23:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.680+0000 7f5d16d9d640 1 -- 172.21.15.27:0/3824528949 learned_addr learned my addr 172.21.15.27:0/3824528949 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.680+0000 7f5d15d9b640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d180b4db0 0x7f5d1806be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.680+0000 7f5d1659c640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d180b49d0 0x7f5d181f6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.680+0000 7f5d16d9d640 1 -- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d180b4db0 msgr2=0x7f5d1806be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.680+0000 7f5d16d9d640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d180b4db0 0x7f5d1806be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.680+0000 7f5d16d9d640 1 -- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d180b49d0 msgr2=0x7f5d181f6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.680+0000 7f5d16d9d640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d180b49d0 0x7f5d181f6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.680+0000 7f5d16d9d640 1 -- 172.21.15.27:0/3824528949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d000094d0 con 0x7f5d180b89b0 2023-12-19T18:23:03.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.680+0000 7f5d15d9b640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d180b4db0 0x7f5d1806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.681+0000 7f5d16d9d640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d180b89b0 0x7f5d1815c650 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5d10001eb0 tx=0x7f5d1000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.681+0000 7f5d0effd640 1 -- 172.21.15.27:0/3824528949 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d100052d0 con 0x7f5d180b89b0 2023-12-19T18:23:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.681+0000 7f5d0effd640 1 -- 172.21.15.27:0/3824528949 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d10005430 con 0x7f5d180b89b0 2023-12-19T18:23:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.681+0000 7f5d1759e640 1 -- 172.21.15.27:0/3824528949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d1815d2f0 con 0x7f5d180b89b0 2023-12-19T18:23:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.681+0000 7f5d0effd640 1 -- 172.21.15.27:0/3824528949 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d10003810 con 0x7f5d180b89b0 2023-12-19T18:23:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.681+0000 7f5d1759e640 1 -- 172.21.15.27:0/3824528949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d181f6440 con 0x7f5d180b89b0 2023-12-19T18:23:03.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.682+0000 7f5d1759e640 1 -- 172.21.15.27:0/3824528949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d180b8520 con 0x7f5d180b89b0 2023-12-19T18:23:03.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.683+0000 7f5d0effd640 1 -- 172.21.15.27:0/3824528949 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5d10003970 con 0x7f5d180b89b0 2023-12-19T18:23:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.684+0000 7f5d0effd640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5cf806a430 0x7f5cf806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:03.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.684+0000 7f5d0effd640 1 -- 172.21.15.27:0/3824528949 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5d10087c10 con 0x7f5d180b89b0 2023-12-19T18:23:03.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.685+0000 7f5d1659c640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5cf806a430 0x7f5cf806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:03.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.686+0000 7f5d1659c640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5cf806a430 0x7f5cf806c8f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5d080074e0 tx=0x7f5d0800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:03.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.689+0000 7f5d0effd640 1 -- 172.21.15.27:0/3824528949 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5d100534b0 con 0x7f5d180b89b0 2023-12-19T18:23:03.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.888+0000 7f5d1759e640 1 -- 172.21.15.27:0/3824528949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5d181f6630 con 0x7f5d180b89b0 2023-12-19T18:23:03.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.889+0000 7f5d0effd640 1 -- 172.21.15.27:0/3824528949 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v148) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5d10056ad0 con 0x7f5d180b89b0 2023-12-19T18:23:03.889 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:03.890 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-12-19T18:23:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-12-19T18:23:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-12-19T18:23:03.890 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:03.890 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-12-19T18:23:03.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.892+0000 7f5d1759e640 1 -- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5cf806a430 msgr2=0x7f5cf806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:03.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.892+0000 7f5d1759e640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5cf806a430 0x7f5cf806c8f0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f5d080074e0 tx=0x7f5d0800b040 comp rx=0 tx=0).stop 2023-12-19T18:23:03.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.892+0000 7f5d1759e640 1 -- 172.21.15.27:0/3824528949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d180b89b0 msgr2=0x7f5d1815c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:03.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.892+0000 7f5d1759e640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d180b89b0 0x7f5d1815c650 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f5d10001eb0 tx=0x7f5d1000c890 comp rx=0 tx=0).stop 2023-12-19T18:23:03.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.892+0000 7f5d1759e640 1 -- 172.21.15.27:0/3824528949 shutdown_connections 2023-12-19T18:23:03.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.892+0000 7f5d1759e640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5cf806a430 0x7f5cf806c8f0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.892+0000 7f5d1759e640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d180b89b0 0x7f5d1815c650 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.892+0000 7f5d1759e640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5d180b4db0 0x7f5d1806be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.892+0000 7f5d1759e640 1 --2- 172.21.15.27:0/3824528949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5d180b49d0 0x7f5d181f6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:03.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.892+0000 7f5d1759e640 1 -- 172.21.15.27:0/3824528949 >> 172.21.15.27:0/3824528949 conn(0x7f5d180cf760 msgr2=0x7f5d180d2550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:03.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.892+0000 7f5d1759e640 1 -- 172.21.15.27:0/3824528949 shutdown_connections 2023-12-19T18:23:03.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:03.892+0000 7f5d1759e640 1 -- 172.21.15.27:0/3824528949 wait complete. 2023-12-19T18:23:03.913 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:23:03.913 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:23:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.153+0000 7f076876d640 1 Processor -- start 2023-12-19T18:23:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.153+0000 7f076876d640 1 -- start start 2023-12-19T18:23:04.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.154+0000 7f076876d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f076015b0d0 0x7f07600cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:04.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.154+0000 7f076876d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07600cbb60 0x7f07600c91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:04.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.154+0000 7f076876d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0760070fb0 con 0x7f076015b4b0 2023-12-19T18:23:04.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.155+0000 7f076876d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f076006fab0 con 0x7f076015b0d0 2023-12-19T18:23:04.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.155+0000 7f076876d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076006f4f0 con 0x7f07600cbb60 2023-12-19T18:23:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.155+0000 7f0765d08640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f076015b4b0 0x7f07600cb450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51724/0 (socket says 172.21.15.27:51724) 2023-12-19T18:23:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.155+0000 7f0766509640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f076015b0d0 0x7f07600cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.155+0000 7f0765d08640 1 -- 172.21.15.27:0/4080790479 learned_addr learned my addr 172.21.15.27:0/4080790479 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.155+0000 7f0766d0a640 1 --2- 172.21.15.27:0/4080790479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07600cbb60 0x7f07600c91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.155+0000 7f0765507640 1 -- 172.21.15.27:0/4080790479 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 338177618 0 0) 0x7f0760070fb0 con 0x7f076015b4b0 2023-12-19T18:23:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.155+0000 7f0765507640 1 -- 172.21.15.27:0/4080790479 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f074c003520 con 0x7f076015b4b0 2023-12-19T18:23:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.156+0000 7f0766509640 1 -- 172.21.15.27:0/4080790479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07600cbb60 msgr2=0x7f07600c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.156+0000 7f0766509640 1 --2- 172.21.15.27:0/4080790479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07600cbb60 0x7f07600c91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.156+0000 7f0766509640 1 -- 172.21.15.27:0/4080790479 >> v1:172.21.15.27:6789/0 conn(0x7f076015b4b0 legacy=0x7f07600cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:04.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.156+0000 7f0766509640 1 -- 172.21.15.27:0/4080790479 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f076006f7d0 con 0x7f076015b0d0 2023-12-19T18:23:04.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.157+0000 7f0766509640 1 --2- 172.21.15.27:0/4080790479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015b0d0 0x7f07600cabf0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f075c0097d0 tx=0x7f075c02b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4936a02c1dc2226a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:04.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.157+0000 7f0765507640 1 -- 172.21.15.27:0/4080790479 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c031890 con 0x7f076015b0d0 2023-12-19T18:23:04.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.157+0000 7f0765507640 1 -- 172.21.15.27:0/4080790479 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c0319f0 con 0x7f076015b0d0 2023-12-19T18:23:04.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.157+0000 7f0765507640 1 -- 172.21.15.27:0/4080790479 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c031d60 con 0x7f076015b0d0 2023-12-19T18:23:04.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.157+0000 7f076876d640 1 -- 172.21.15.27:0/4080790479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015b0d0 msgr2=0x7f07600cabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:04.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.157+0000 7f076876d640 1 --2- 172.21.15.27:0/4080790479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015b0d0 0x7f07600cabf0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f075c0097d0 tx=0x7f075c02b930 comp rx=0 tx=0).stop 2023-12-19T18:23:04.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.158+0000 7f076876d640 1 -- 172.21.15.27:0/4080790479 shutdown_connections 2023-12-19T18:23:04.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.158+0000 7f076876d640 1 --2- 172.21.15.27:0/4080790479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07600cbb60 0x7f07600c91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:04.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.158+0000 7f076876d640 1 --2- 172.21.15.27:0/4080790479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f076015b0d0 0x7f07600cabf0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:04.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.158+0000 7f076876d640 1 -- 172.21.15.27:0/4080790479 >> 172.21.15.27:0/4080790479 conn(0x7f07600b0620 msgr2=0x7f07600b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:04.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.158+0000 7f076876d640 1 -- 172.21.15.27:0/4080790479 shutdown_connections 2023-12-19T18:23:04.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.158+0000 7f076876d640 1 -- 172.21.15.27:0/4080790479 wait complete. 2023-12-19T18:23:04.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.160+0000 7f076876d640 1 Processor -- start 2023-12-19T18:23:04.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.160+0000 7f076876d640 1 -- start start 2023-12-19T18:23:04.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.161+0000 7f076876d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07600cbb60 0x7f076014eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.161+0000 7f076876d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f076015b0d0 0x7f076015b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.161+0000 7f076876d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f076015b4b0 0x7f076015d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.161+0000 7f076876d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076015be80 con 0x7f07600cbb60 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.161+0000 7f076876d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f076015c0e0 con 0x7f076015b0d0 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.161+0000 7f076876d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f076015c3c0 con 0x7f076015b4b0 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.161+0000 7f0766509640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07600cbb60 0x7f076014eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.161+0000 7f0766509640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07600cbb60 0x7f076014eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52786/0 (socket says 172.21.15.27:52786) 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.161+0000 7f0766509640 1 -- 172.21.15.27:0/347005542 learned_addr learned my addr 172.21.15.27:0/347005542 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.161+0000 7f0766d0a640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f076015b4b0 0x7f076015d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.162+0000 7f0765d08640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f076015b0d0 0x7f076015b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.162+0000 7f0766509640 1 -- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f076015b4b0 msgr2=0x7f076015d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.162+0000 7f0766509640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f076015b4b0 0x7f076015d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.162+0000 7f0766509640 1 -- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f076015b0d0 msgr2=0x7f076015b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:04.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.162+0000 7f0766509640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f076015b0d0 0x7f076015b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:04.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.162+0000 7f0766509640 1 -- 172.21.15.27:0/347005542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f075c009400 con 0x7f07600cbb60 2023-12-19T18:23:04.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.162+0000 7f0766d0a640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f076015b4b0 0x7f076015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:04.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.162+0000 7f0765d08640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f076015b0d0 0x7f076015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.162+0000 7f0766509640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07600cbb60 0x7f076014eae0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f075c0098f0 tx=0x7f075c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.162+0000 7f0756ffd640 1 -- 172.21.15.27:0/347005542 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c03bd30 con 0x7f07600cbb60 2023-12-19T18:23:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.162+0000 7f0756ffd640 1 -- 172.21.15.27:0/347005542 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f075c03be90 con 0x7f07600cbb60 2023-12-19T18:23:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.162+0000 7f0756ffd640 1 -- 172.21.15.27:0/347005542 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075c045630 con 0x7f07600cbb60 2023-12-19T18:23:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.162+0000 7f076876d640 1 -- 172.21.15.27:0/347005542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f076015c6a0 con 0x7f07600cbb60 2023-12-19T18:23:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.163+0000 7f076876d640 1 -- 172.21.15.27:0/347005542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07600ac970 con 0x7f07600cbb60 2023-12-19T18:23:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.164+0000 7f076876d640 1 -- 172.21.15.27:0/347005542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07600acb60 con 0x7f07600cbb60 2023-12-19T18:23:04.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.165+0000 7f0756ffd640 1 -- 172.21.15.27:0/347005542 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f075c04e450 con 0x7f07600cbb60 2023-12-19T18:23:04.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.166+0000 7f0756ffd640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f074406a430 0x7f074406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:04.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.166+0000 7f0756ffd640 1 -- 172.21.15.27:0/347005542 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f075c0aad90 con 0x7f07600cbb60 2023-12-19T18:23:04.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.166+0000 7f0765d08640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f074406a430 0x7f074406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:04.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.167+0000 7f0765d08640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f074406a430 0x7f074406c8f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f076015d250 tx=0x7f0750009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.170+0000 7f0756ffd640 1 -- 172.21.15.27:0/347005542 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f075c076630 con 0x7f07600cbb60 2023-12-19T18:23:04.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.363+0000 7f076876d640 1 -- 172.21.15.27:0/347005542 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f0760068e90 con 0x7f07600cbb60 2023-12-19T18:23:04.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.961+0000 7f0756ffd640 1 -- 172.21.15.27:0/347005542 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v149) v1 ==== 138+0+0 (secure 0 0 0) 0x7f075c0350b0 con 0x7f07600cbb60 2023-12-19T18:23:04.962 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-12-19T18:23:04.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.966+0000 7f076876d640 1 -- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f074406a430 msgr2=0x7f074406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.966+0000 7f076876d640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f074406a430 0x7f074406c8f0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f076015d250 tx=0x7f0750009040 comp rx=0 tx=0).stop 2023-12-19T18:23:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.966+0000 7f076876d640 1 -- 172.21.15.27:0/347005542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07600cbb60 msgr2=0x7f076014eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.966+0000 7f076876d640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07600cbb60 0x7f076014eae0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f075c0098f0 tx=0x7f075c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:23:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.966+0000 7f076876d640 1 -- 172.21.15.27:0/347005542 shutdown_connections 2023-12-19T18:23:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.966+0000 7f076876d640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f074406a430 0x7f074406c8f0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.966+0000 7f076876d640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f076015b4b0 0x7f076015d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.966+0000 7f076876d640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f076015b0d0 0x7f076015b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.966+0000 7f076876d640 1 --2- 172.21.15.27:0/347005542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07600cbb60 0x7f076014eae0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.966+0000 7f076876d640 1 -- 172.21.15.27:0/347005542 >> 172.21.15.27:0/347005542 conn(0x7f07600b0620 msgr2=0x7f07600c4330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.966+0000 7f076876d640 1 -- 172.21.15.27:0/347005542 shutdown_connections 2023-12-19T18:23:04.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:04.967+0000 7f076876d640 1 -- 172.21.15.27:0/347005542 wait complete. 2023-12-19T18:23:04.989 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:23:05.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.229+0000 7f689cf3a640 1 Processor -- start 2023-12-19T18:23:05.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.230+0000 7f689cf3a640 1 -- start start 2023-12-19T18:23:05.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.231+0000 7f689cf3a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f689815ade0 0x7f68980b3990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:05.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.231+0000 7f689cf3a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f68980cce00 0x7f68980d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:05.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.231+0000 7f689cf3a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68980b3040 con 0x7f68980b2c00 2023-12-19T18:23:05.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.231+0000 7f689cf3a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68980cfea0 con 0x7f68980cce00 2023-12-19T18:23:05.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.231+0000 7f689cf3a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68980b6940 con 0x7f689815ade0 2023-12-19T18:23:05.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.232+0000 7f6897fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f68980cce00 0x7f68980d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:05.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.232+0000 7f68977fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f689815ade0 0x7f68980b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:05.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.232+0000 7f6897fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f68980cce00 0x7f68980d0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:56390/0 (socket says 172.21.15.27:56390) 2023-12-19T18:23:05.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.232+0000 7f6897fff640 1 -- 172.21.15.27:0/3526460278 learned_addr learned my addr 172.21.15.27:0/3526460278 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:05.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.233+0000 7f68977fe640 1 -- 172.21.15.27:0/3526460278 >> v2:172.21.15.192:3300/0 conn(0x7f68980cce00 msgr2=0x7f68980d0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.233+0000 7f68977fe640 1 --2- 172.21.15.27:0/3526460278 >> v2:172.21.15.192:3300/0 conn(0x7f68980cce00 0x7f68980d0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.233+0000 7f68977fe640 1 -- 172.21.15.27:0/3526460278 >> v1:172.21.15.192:6789/0 conn(0x7f68980b2c00 legacy=0x7f68980cc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.233+0000 7f68977fe640 1 -- 172.21.15.27:0/3526460278 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68980d00f0 con 0x7f689815ade0 2023-12-19T18:23:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.233+0000 7f6897fff640 1 --2- 172.21.15.27:0/3526460278 >> v2:172.21.15.192:3300/0 conn(0x7f68980cce00 0x7f68980d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:23:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.234+0000 7f68977fe640 1 --2- 172.21.15.27:0/3526460278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f689815ade0 0x7f68980b3990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f68880097a0 tx=0x7f688802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=506b7ac2bee49e75 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.235+0000 7f6896ffd640 1 -- 172.21.15.27:0/3526460278 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888031780 con 0x7f689815ade0 2023-12-19T18:23:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.235+0000 7f6896ffd640 1 -- 172.21.15.27:0/3526460278 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68880318e0 con 0x7f689815ade0 2023-12-19T18:23:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.235+0000 7f6896ffd640 1 -- 172.21.15.27:0/3526460278 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6888031c70 con 0x7f689815ade0 2023-12-19T18:23:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.235+0000 7f689cf3a640 1 -- 172.21.15.27:0/3526460278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f689815ade0 msgr2=0x7f68980b3990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.235+0000 7f689cf3a640 1 --2- 172.21.15.27:0/3526460278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f689815ade0 0x7f68980b3990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f68880097a0 tx=0x7f688802b820 comp rx=0 tx=0).stop 2023-12-19T18:23:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.235+0000 7f689cf3a640 1 -- 172.21.15.27:0/3526460278 shutdown_connections 2023-12-19T18:23:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.235+0000 7f689cf3a640 1 --2- 172.21.15.27:0/3526460278 >> v2:172.21.15.192:3300/0 conn(0x7f68980cce00 0x7f68980d0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.235+0000 7f689cf3a640 1 --2- 172.21.15.27:0/3526460278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f689815ade0 0x7f68980b3990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.235+0000 7f689cf3a640 1 -- 172.21.15.27:0/3526460278 >> 172.21.15.27:0/3526460278 conn(0x7f6898072c40 msgr2=0x7f6898073050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.235+0000 7f689cf3a640 1 -- 172.21.15.27:0/3526460278 shutdown_connections 2023-12-19T18:23:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.236+0000 7f689cf3a640 1 -- 172.21.15.27:0/3526460278 wait complete. 2023-12-19T18:23:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.237+0000 7f689cf3a640 1 Processor -- start 2023-12-19T18:23:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.237+0000 7f689cf3a640 1 -- start start 2023-12-19T18:23:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.238+0000 7f689cf3a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68980b2c00 0x7f689806ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.238+0000 7f689cf3a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68980cce00 0x7f68980c70a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.238+0000 7f689cf3a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f689815ade0 0x7f6898158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.238+0000 7f689cf3a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980cf710 con 0x7f68980cce00 2023-12-19T18:23:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.238+0000 7f689cf3a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68980c75e0 con 0x7f689815ade0 2023-12-19T18:23:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.238+0000 7f689cf3a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6898156e90 con 0x7f68980b2c00 2023-12-19T18:23:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.239+0000 7f688effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68980cce00 0x7f68980c70a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.239+0000 7f688effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68980cce00 0x7f68980c70a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52788/0 (socket says 172.21.15.27:52788) 2023-12-19T18:23:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.239+0000 7f688effd640 1 -- 172.21.15.27:0/3267039613 learned_addr learned my addr 172.21.15.27:0/3267039613 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.239+0000 7f68977fe640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68980b2c00 0x7f689806ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.239+0000 7f6897fff640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f689815ade0 0x7f6898158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.239+0000 7f688effd640 1 -- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68980b2c00 msgr2=0x7f689806ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.239+0000 7f688effd640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68980b2c00 0x7f689806ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.239+0000 7f688effd640 1 -- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f689815ade0 msgr2=0x7f6898158140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.239+0000 7f688effd640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f689815ade0 0x7f6898158140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.239+0000 7f688effd640 1 -- 172.21.15.27:0/3267039613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6890009b30 con 0x7f68980cce00 2023-12-19T18:23:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.239+0000 7f6897fff640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f689815ade0 0x7f6898158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.240+0000 7f688effd640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68980cce00 0x7f68980c70a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f688000c430 tx=0x7f688000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.240+0000 7f6894ff9640 1 -- 172.21.15.27:0/3267039613 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68800052d0 con 0x7f68980cce00 2023-12-19T18:23:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.240+0000 7f6894ff9640 1 -- 172.21.15.27:0/3267039613 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6880005430 con 0x7f68980cce00 2023-12-19T18:23:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.240+0000 7f6894ff9640 1 -- 172.21.15.27:0/3267039613 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6880003830 con 0x7f68980cce00 2023-12-19T18:23:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.240+0000 7f689cf3a640 1 -- 172.21.15.27:0/3267039613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6888009400 con 0x7f68980cce00 2023-12-19T18:23:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.240+0000 7f689cf3a640 1 -- 172.21.15.27:0/3267039613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6898157450 con 0x7f68980cce00 2023-12-19T18:23:05.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.241+0000 7f689cf3a640 1 -- 172.21.15.27:0/3267039613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689804e840 con 0x7f68980cce00 2023-12-19T18:23:05.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.243+0000 7f6894ff9640 1 -- 172.21.15.27:0/3267039613 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6880003990 con 0x7f68980cce00 2023-12-19T18:23:05.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.243+0000 7f6894ff9640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f687806a430 0x7f687806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.244+0000 7f6894ff9640 1 -- 172.21.15.27:0/3267039613 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f6880087b40 con 0x7f68980cce00 2023-12-19T18:23:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.244+0000 7f68977fe640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f687806a430 0x7f687806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:05.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.245+0000 7f68977fe640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f687806a430 0x7f687806c8f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6888036000 tx=0x7f688803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:05.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.249+0000 7f6894ff9640 1 -- 172.21.15.27:0/3267039613 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6880053300 con 0x7f68980cce00 2023-12-19T18:23:05.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.441+0000 7f689cf3a640 1 -- 172.21.15.27:0/3267039613 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f68980b3040 con 0x7f68980cce00 2023-12-19T18:23:05.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.980+0000 7f6894ff9640 1 -- 172.21.15.27:0/3267039613 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v150) v1 ==== 170+0+0 (secure 0 0 0) 0x7f68980b3040 con 0x7f68980cce00 2023-12-19T18:23:05.985 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-12-19T18:23:05.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.991+0000 7f689cf3a640 1 -- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f687806a430 msgr2=0x7f687806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:05.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.991+0000 7f689cf3a640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f687806a430 0x7f687806c8f0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6888036000 tx=0x7f688803d040 comp rx=0 tx=0).stop 2023-12-19T18:23:05.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.991+0000 7f689cf3a640 1 -- 172.21.15.27:0/3267039613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68980cce00 msgr2=0x7f68980c70a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:05.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.991+0000 7f689cf3a640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68980cce00 0x7f68980c70a0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f688000c430 tx=0x7f688000c8f0 comp rx=0 tx=0).stop 2023-12-19T18:23:05.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.992+0000 7f689cf3a640 1 -- 172.21.15.27:0/3267039613 shutdown_connections 2023-12-19T18:23:05.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.992+0000 7f689cf3a640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f687806a430 0x7f687806c8f0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:05.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.992+0000 7f689cf3a640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f689815ade0 0x7f6898158140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:05.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.992+0000 7f689cf3a640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68980cce00 0x7f68980c70a0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:05.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.992+0000 7f689cf3a640 1 --2- 172.21.15.27:0/3267039613 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68980b2c00 0x7f689806ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:05.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.992+0000 7f689cf3a640 1 -- 172.21.15.27:0/3267039613 >> 172.21.15.27:0/3267039613 conn(0x7f6898072c40 msgr2=0x7f68980b59b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:05.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.993+0000 7f689cf3a640 1 -- 172.21.15.27:0/3267039613 shutdown_connections 2023-12-19T18:23:05.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:05.994+0000 7f689cf3a640 1 -- 172.21.15.27:0/3267039613 wait complete. 2023-12-19T18:23:06.018 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:23:06.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.214+0000 7fd652185640 1 Processor -- start 2023-12-19T18:23:06.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.214+0000 7fd652185640 1 -- start start 2023-12-19T18:23:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.216+0000 7fd652185640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd64c151c90 0x7fd64c06b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.216+0000 7fd652185640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd64c0cf550 0x7fd64c15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.216+0000 7fd652185640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd64c15b3e0 con 0x7fd64c15e9d0 2023-12-19T18:23:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.216+0000 7fd652185640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64c15eed0 con 0x7fd64c0cf550 2023-12-19T18:23:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.216+0000 7fd652185640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd64c15f1b0 con 0x7fd64c151c90 2023-12-19T18:23:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.217+0000 7fd650982640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd64c15e9d0 0x7fd64c0cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51734/0 (socket says 172.21.15.27:51734) 2023-12-19T18:23:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.217+0000 7fd650982640 1 -- 172.21.15.27:0/565425152 learned_addr learned my addr 172.21.15.27:0/565425152 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.217+0000 7fd651984640 1 --2- 172.21.15.27:0/565425152 >> v2:172.21.15.27:3300/0 conn(0x7fd64c0cf550 0x7fd64c15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:06.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.217+0000 7fd651183640 1 --2- 172.21.15.27:0/565425152 >> v2:172.21.15.192:3300/0 conn(0x7fd64c151c90 0x7fd64c06b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:06.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.217+0000 7fd643fff640 1 -- 172.21.15.27:0/565425152 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3226626136 0 0) 0x7fd64c15b3e0 con 0x7fd64c15e9d0 2023-12-19T18:23:06.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.217+0000 7fd643fff640 1 -- 172.21.15.27:0/565425152 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd638003520 con 0x7fd64c15e9d0 2023-12-19T18:23:06.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.217+0000 7fd651984640 1 -- 172.21.15.27:0/565425152 >> v2:172.21.15.192:3300/0 conn(0x7fd64c151c90 msgr2=0x7fd64c06b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:06.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.217+0000 7fd651984640 1 --2- 172.21.15.27:0/565425152 >> v2:172.21.15.192:3300/0 conn(0x7fd64c151c90 0x7fd64c06b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.217+0000 7fd651984640 1 -- 172.21.15.27:0/565425152 >> v1:172.21.15.27:6789/0 conn(0x7fd64c15e9d0 legacy=0x7fd64c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:06.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.217+0000 7fd651984640 1 -- 172.21.15.27:0/565425152 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd64c15f490 con 0x7fd64c0cf550 2023-12-19T18:23:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.218+0000 7fd651984640 1 --2- 172.21.15.27:0/565425152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64c0cf550 0x7fd64c15b8a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fd644009d90 tx=0x7fd64402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac6d6ce5d681d16 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.219+0000 7fd643fff640 1 -- 172.21.15.27:0/565425152 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64402cb40 con 0x7fd64c0cf550 2023-12-19T18:23:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.219+0000 7fd643fff640 1 -- 172.21.15.27:0/565425152 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64402cca0 con 0x7fd64c0cf550 2023-12-19T18:23:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.219+0000 7fd643fff640 1 -- 172.21.15.27:0/565425152 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd644034660 con 0x7fd64c0cf550 2023-12-19T18:23:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.219+0000 7fd652185640 1 -- 172.21.15.27:0/565425152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64c0cf550 msgr2=0x7fd64c15b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.219+0000 7fd652185640 1 --2- 172.21.15.27:0/565425152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64c0cf550 0x7fd64c15b8a0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7fd644009d90 tx=0x7fd64402bd90 comp rx=0 tx=0).stop 2023-12-19T18:23:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.219+0000 7fd652185640 1 -- 172.21.15.27:0/565425152 shutdown_connections 2023-12-19T18:23:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.219+0000 7fd652185640 1 --2- 172.21.15.27:0/565425152 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64c0cf550 0x7fd64c15b8a0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.219+0000 7fd652185640 1 --2- 172.21.15.27:0/565425152 >> v2:172.21.15.192:3300/0 conn(0x7fd64c151c90 0x7fd64c06b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.219+0000 7fd652185640 1 -- 172.21.15.27:0/565425152 >> 172.21.15.27:0/565425152 conn(0x7fd64c06a060 msgr2=0x7fd64c06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.219+0000 7fd652185640 1 -- 172.21.15.27:0/565425152 shutdown_connections 2023-12-19T18:23:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.220+0000 7fd652185640 1 -- 172.21.15.27:0/565425152 wait complete. 2023-12-19T18:23:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.221+0000 7fd652185640 1 Processor -- start 2023-12-19T18:23:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.221+0000 7fd652185640 1 -- start start 2023-12-19T18:23:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.222+0000 7fd652185640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64c0cf550 0x7fd64c141470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.222+0000 7fd652185640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd64c151c90 0x7fd64c15a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.222+0000 7fd652185640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd64c15e9d0 0x7fd64c1f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.222+0000 7fd652185640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c1419b0 con 0x7fd64c0cf550 2023-12-19T18:23:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.222+0000 7fd652185640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c1f0d50 con 0x7fd64c151c90 2023-12-19T18:23:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.222+0000 7fd652185640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd64c159640 con 0x7fd64c15e9d0 2023-12-19T18:23:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.222+0000 7fd651183640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64c0cf550 0x7fd64c141470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.223+0000 7fd651183640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64c0cf550 0x7fd64c141470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52804/0 (socket says 172.21.15.27:52804) 2023-12-19T18:23:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.223+0000 7fd651183640 1 -- 172.21.15.27:0/711389293 learned_addr learned my addr 172.21.15.27:0/711389293 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.223+0000 7fd651183640 1 -- 172.21.15.27:0/711389293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd64c15e9d0 msgr2=0x7fd64c1f10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.223+0000 7fd651183640 1 --2- 172.21.15.27:0/711389293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd64c15e9d0 0x7fd64c1f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.223+0000 7fd651183640 1 -- 172.21.15.27:0/711389293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd64c151c90 msgr2=0x7fd64c15a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.223+0000 7fd650982640 1 --2- 172.21.15.27:0/711389293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd64c151c90 0x7fd64c15a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.223+0000 7fd651183640 1 --2- 172.21.15.27:0/711389293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd64c151c90 0x7fd64c15a120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.223+0000 7fd651183640 1 -- 172.21.15.27:0/711389293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd644009a70 con 0x7fd64c0cf550 2023-12-19T18:23:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.223+0000 7fd651183640 1 --2- 172.21.15.27:0/711389293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64c0cf550 0x7fd64c141470 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fd64800d800 tx=0x7fd64800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.224+0000 7fd641ffb640 1 -- 172.21.15.27:0/711389293 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64800b6c0 con 0x7fd64c0cf550 2023-12-19T18:23:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.224+0000 7fd641ffb640 1 -- 172.21.15.27:0/711389293 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd64800b820 con 0x7fd64c0cf550 2023-12-19T18:23:06.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.224+0000 7fd652185640 1 -- 172.21.15.27:0/711389293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd64c159920 con 0x7fd64c0cf550 2023-12-19T18:23:06.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.224+0000 7fd641ffb640 1 -- 172.21.15.27:0/711389293 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd64800bbb0 con 0x7fd64c0cf550 2023-12-19T18:23:06.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.224+0000 7fd652185640 1 -- 172.21.15.27:0/711389293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd64c159260 con 0x7fd64c0cf550 2023-12-19T18:23:06.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.225+0000 7fd652185640 1 -- 172.21.15.27:0/711389293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd64c159480 con 0x7fd64c0cf550 2023-12-19T18:23:06.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.226+0000 7fd641ffb640 1 -- 172.21.15.27:0/711389293 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd64800bd10 con 0x7fd64c0cf550 2023-12-19T18:23:06.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.227+0000 7fd641ffb640 1 --2- 172.21.15.27:0/711389293 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd63006a460 0x7fd63006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:06.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.227+0000 7fd641ffb640 1 -- 172.21.15.27:0/711389293 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fd648088dc0 con 0x7fd64c0cf550 2023-12-19T18:23:06.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.228+0000 7fd650982640 1 --2- 172.21.15.27:0/711389293 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd63006a460 0x7fd63006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:06.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.229+0000 7fd650982640 1 --2- 172.21.15.27:0/711389293 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd63006a460 0x7fd63006c920 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd64c06c5d0 tx=0x7fd63c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:06.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.232+0000 7fd641ffb640 1 -- 172.21.15.27:0/711389293 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd648054410 con 0x7fd64c0cf550 2023-12-19T18:23:06.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.463+0000 7fd652185640 1 -- 172.21.15.27:0/711389293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd64c04e840 con 0x7fd64c0cf550 2023-12-19T18:23:06.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.472+0000 7fd641ffb640 1 -- 172.21.15.27:0/711389293 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7fd648057a30 con 0x7fd64c0cf550 2023-12-19T18:23:06.473 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 16 and data pool 17 2023-12-19T18:23:06.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.476+0000 7fd652185640 1 -- 172.21.15.27:0/711389293 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd63006a460 msgr2=0x7fd63006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:06.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.476+0000 7fd652185640 1 --2- 172.21.15.27:0/711389293 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd63006a460 0x7fd63006c920 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fd64c06c5d0 tx=0x7fd63c009040 comp rx=0 tx=0).stop 2023-12-19T18:23:06.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.476+0000 7fd652185640 1 -- 172.21.15.27:0/711389293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64c0cf550 msgr2=0x7fd64c141470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:06.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.476+0000 7fd652185640 1 --2- 172.21.15.27:0/711389293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64c0cf550 0x7fd64c141470 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7fd64800d800 tx=0x7fd64800dcc0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.476+0000 7fd652185640 1 -- 172.21.15.27:0/711389293 shutdown_connections 2023-12-19T18:23:06.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.476+0000 7fd652185640 1 --2- 172.21.15.27:0/711389293 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd63006a460 0x7fd63006c920 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.476+0000 7fd652185640 1 --2- 172.21.15.27:0/711389293 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd64c15e9d0 0x7fd64c1f10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.476+0000 7fd652185640 1 --2- 172.21.15.27:0/711389293 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd64c151c90 0x7fd64c15a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.476+0000 7fd652185640 1 --2- 172.21.15.27:0/711389293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd64c0cf550 0x7fd64c141470 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.476+0000 7fd652185640 1 -- 172.21.15.27:0/711389293 >> 172.21.15.27:0/711389293 conn(0x7fd64c06a060 msgr2=0x7fd64c06a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:06.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.477+0000 7fd652185640 1 -- 172.21.15.27:0/711389293 shutdown_connections 2023-12-19T18:23:06.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.477+0000 7fd652185640 1 -- 172.21.15.27:0/711389293 wait complete. 2023-12-19T18:23:06.505 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:23:06.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.696+0000 7fc23f577640 1 Processor -- start 2023-12-19T18:23:06.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.696+0000 7fc23f577640 1 -- start start 2023-12-19T18:23:06.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.698+0000 7fc23f577640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc2380cbce0 0x7fc2380ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:06.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.698+0000 7fc23f577640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc2380adc20 0x7fc23806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:06.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.698+0000 7fc23f577640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2380cb850 con 0x7fc23815af70 2023-12-19T18:23:06.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.698+0000 7fc23f577640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc23815b350 con 0x7fc2380adc20 2023-12-19T18:23:06.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.698+0000 7fc23f577640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2380cac40 con 0x7fc2380cbce0 2023-12-19T18:23:06.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.698+0000 7fc23db14640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc2380adc20 0x7fc23806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:06.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.698+0000 7fc23db14640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc2380adc20 0x7fc23806b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52816/0 (socket says 172.21.15.27:52816) 2023-12-19T18:23:06.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.698+0000 7fc23db14640 1 -- 172.21.15.27:0/2237538021 learned_addr learned my addr 172.21.15.27:0/2237538021 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:06.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.698+0000 7fc23cb12640 1 --2- 172.21.15.27:0/2237538021 >> v2:172.21.15.192:3300/0 conn(0x7fc2380cbce0 0x7fc2380ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:06.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.699+0000 7fc23db14640 1 -- 172.21.15.27:0/2237538021 >> v2:172.21.15.192:3300/0 conn(0x7fc2380cbce0 msgr2=0x7fc2380ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:06.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.699+0000 7fc23db14640 1 --2- 172.21.15.27:0/2237538021 >> v2:172.21.15.192:3300/0 conn(0x7fc2380cbce0 0x7fc2380ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.699+0000 7fc23db14640 1 -- 172.21.15.27:0/2237538021 >> v1:172.21.15.192:6789/0 conn(0x7fc23815af70 legacy=0x7fc2380ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:06.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.699+0000 7fc23db14640 1 -- 172.21.15.27:0/2237538021 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2380caf20 con 0x7fc2380adc20 2023-12-19T18:23:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.700+0000 7fc23db14640 1 --2- 172.21.15.27:0/2237538021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2380adc20 0x7fc23806b4e0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fc2380cc620 tx=0x7fc23002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e18a325069b5349b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.701+0000 7fc22ffff640 1 -- 172.21.15.27:0/2237538021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23002ca80 con 0x7fc2380adc20 2023-12-19T18:23:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.701+0000 7fc22ffff640 1 -- 172.21.15.27:0/2237538021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23002cbe0 con 0x7fc2380adc20 2023-12-19T18:23:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.701+0000 7fc22ffff640 1 -- 172.21.15.27:0/2237538021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2300343f0 con 0x7fc2380adc20 2023-12-19T18:23:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.701+0000 7fc23f577640 1 -- 172.21.15.27:0/2237538021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2380adc20 msgr2=0x7fc23806b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.701+0000 7fc23f577640 1 --2- 172.21.15.27:0/2237538021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2380adc20 0x7fc23806b4e0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7fc2380cc620 tx=0x7fc23002bee0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.701+0000 7fc23f577640 1 -- 172.21.15.27:0/2237538021 shutdown_connections 2023-12-19T18:23:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.701+0000 7fc23f577640 1 --2- 172.21.15.27:0/2237538021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2380adc20 0x7fc23806b4e0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.701+0000 7fc23f577640 1 --2- 172.21.15.27:0/2237538021 >> v2:172.21.15.192:3300/0 conn(0x7fc2380cbce0 0x7fc2380ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.702+0000 7fc23f577640 1 -- 172.21.15.27:0/2237538021 >> 172.21.15.27:0/2237538021 conn(0x7fc23806a330 msgr2=0x7fc2380b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.702+0000 7fc23f577640 1 -- 172.21.15.27:0/2237538021 shutdown_connections 2023-12-19T18:23:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.702+0000 7fc23f577640 1 -- 172.21.15.27:0/2237538021 wait complete. 2023-12-19T18:23:06.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.703+0000 7fc23f577640 1 Processor -- start 2023-12-19T18:23:06.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.704+0000 7fc23f577640 1 -- start start 2023-12-19T18:23:06.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.704+0000 7fc23f577640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2380adc20 0x7fc238151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:06.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.705+0000 7fc23f577640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2380cbce0 0x7fc2380c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:06.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.705+0000 7fc23f577640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc23815af70 0x7fc2380cd570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:06.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.705+0000 7fc23f577640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23815c240 con 0x7fc2380cbce0 2023-12-19T18:23:06.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.705+0000 7fc23f577640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23815b630 con 0x7fc2380adc20 2023-12-19T18:23:06.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.705+0000 7fc23f577640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23815b910 con 0x7fc23815af70 2023-12-19T18:23:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.705+0000 7fc23cb12640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2380cbce0 0x7fc2380c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.705+0000 7fc23cb12640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2380cbce0 0x7fc2380c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52830/0 (socket says 172.21.15.27:52830) 2023-12-19T18:23:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.705+0000 7fc23cb12640 1 -- 172.21.15.27:0/3753178849 learned_addr learned my addr 172.21.15.27:0/3753178849 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.705+0000 7fc23cb12640 1 -- 172.21.15.27:0/3753178849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc23815af70 msgr2=0x7fc2380cd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.705+0000 7fc23cb12640 1 --2- 172.21.15.27:0/3753178849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc23815af70 0x7fc2380cd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.705+0000 7fc23cb12640 1 -- 172.21.15.27:0/3753178849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2380adc20 msgr2=0x7fc238151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.705+0000 7fc23cb12640 1 --2- 172.21.15.27:0/3753178849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2380adc20 0x7fc238151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.705+0000 7fc23cb12640 1 -- 172.21.15.27:0/3753178849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc230009b30 con 0x7fc2380cbce0 2023-12-19T18:23:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.706+0000 7fc23cb12640 1 --2- 172.21.15.27:0/3753178849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2380cbce0 0x7fc2380c5d50 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fc22800d7d0 tx=0x7fc22800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.706+0000 7fc22dffb640 1 -- 172.21.15.27:0/3753178849 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22800b6c0 con 0x7fc2380cbce0 2023-12-19T18:23:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.706+0000 7fc22dffb640 1 -- 172.21.15.27:0/3753178849 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22800b840 con 0x7fc2380cbce0 2023-12-19T18:23:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.706+0000 7fc23f577640 1 -- 172.21.15.27:0/3753178849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc23815bbf0 con 0x7fc2380cbce0 2023-12-19T18:23:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.706+0000 7fc22dffb640 1 -- 172.21.15.27:0/3753178849 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22800bbf0 con 0x7fc2380cbce0 2023-12-19T18:23:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.707+0000 7fc23f577640 1 -- 172.21.15.27:0/3753178849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2381520e0 con 0x7fc2380cbce0 2023-12-19T18:23:06.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.708+0000 7fc23f577640 1 -- 172.21.15.27:0/3753178849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc238068e90 con 0x7fc2380cbce0 2023-12-19T18:23:06.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.709+0000 7fc22dffb640 1 -- 172.21.15.27:0/3753178849 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc22800bd50 con 0x7fc2380cbce0 2023-12-19T18:23:06.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.710+0000 7fc22dffb640 1 --2- 172.21.15.27:0/3753178849 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc21c06a430 0x7fc21c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:06.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.710+0000 7fc22dffb640 1 -- 172.21.15.27:0/3753178849 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc228088e60 con 0x7fc2380cbce0 2023-12-19T18:23:06.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.710+0000 7fc23d313640 1 --2- 172.21.15.27:0/3753178849 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc21c06a430 0x7fc21c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:06.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.711+0000 7fc23d313640 1 --2- 172.21.15.27:0/3753178849 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc21c06a430 0x7fc21c06c8f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc234006fd0 tx=0x7fc234006bc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:06.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.715+0000 7fc22dffb640 1 -- 172.21.15.27:0/3753178849 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc228054450 con 0x7fc2380cbce0 2023-12-19T18:23:06.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.907+0000 7fc23f577640 1 -- 172.21.15.27:0/3753178849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc23815a650 con 0x7fc2380cbce0 2023-12-19T18:23:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.908+0000 7fc22dffb640 1 -- 172.21.15.27:0/3753178849 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16129 (secure 0 0 0) 0x7fc228057a70 con 0x7fc2380cbce0 2023-12-19T18:23:06.909 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:06.909 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":151,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:23:06.465222+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:23:04.431250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T18:23:05.506926+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:23:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.910+0000 7fc23f577640 1 -- 172.21.15.27:0/3753178849 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc21c06a430 msgr2=0x7fc21c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.910+0000 7fc23f577640 1 --2- 172.21.15.27:0/3753178849 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc21c06a430 0x7fc21c06c8f0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fc234006fd0 tx=0x7fc234006bc0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.911+0000 7fc23f577640 1 -- 172.21.15.27:0/3753178849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2380cbce0 msgr2=0x7fc2380c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.911+0000 7fc23f577640 1 --2- 172.21.15.27:0/3753178849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2380cbce0 0x7fc2380c5d50 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7fc22800d7d0 tx=0x7fc22800dc90 comp rx=0 tx=0).stop 2023-12-19T18:23:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.911+0000 7fc23f577640 1 -- 172.21.15.27:0/3753178849 shutdown_connections 2023-12-19T18:23:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.911+0000 7fc23f577640 1 --2- 172.21.15.27:0/3753178849 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc21c06a430 0x7fc21c06c8f0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.911+0000 7fc23f577640 1 --2- 172.21.15.27:0/3753178849 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc23815af70 0x7fc2380cd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.911+0000 7fc23f577640 1 --2- 172.21.15.27:0/3753178849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2380cbce0 0x7fc2380c5d50 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.911+0000 7fc23f577640 1 --2- 172.21.15.27:0/3753178849 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2380adc20 0x7fc238151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:06.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.911+0000 7fc23f577640 1 -- 172.21.15.27:0/3753178849 >> 172.21.15.27:0/3753178849 conn(0x7fc23806a330 msgr2=0x7fc238070dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:06.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.911+0000 7fc23f577640 1 -- 172.21.15.27:0/3753178849 shutdown_connections 2023-12-19T18:23:06.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:06.911+0000 7fc23f577640 1 -- 172.21.15.27:0/3753178849 wait complete. 2023-12-19T18:23:06.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:23:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.145+0000 7f6020c6c640 1 Processor -- start 2023-12-19T18:23:07.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.145+0000 7f6020c6c640 1 -- start start 2023-12-19T18:23:07.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.147+0000 7f6020c6c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f601c0b4ef0 0x7f601c0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:07.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.147+0000 7f6020c6c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f601c0d2350 0x7f601c0b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:07.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.147+0000 7f6020c6c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f601c0b5e90 con 0x7f601c1604d0 2023-12-19T18:23:07.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.147+0000 7f6020c6c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f601c0d2b30 con 0x7f601c0b4ef0 2023-12-19T18:23:07.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.147+0000 7f6020c6c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601c0d2850 con 0x7f601c0d2350 2023-12-19T18:23:07.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.148+0000 7f6013fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f601c0b4ef0 0x7f601c0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:07.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.148+0000 7f6013fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f601c0b4ef0 0x7f601c0b6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:56444/0 (socket says 172.21.15.27:56444) 2023-12-19T18:23:07.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.148+0000 7f6013fff640 1 -- 172.21.15.27:0/1203385813 learned_addr learned my addr 172.21.15.27:0/1203385813 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:07.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.148+0000 7f601bfff640 1 --2- 172.21.15.27:0/1203385813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f601c0d2350 0x7f601c0b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:07.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.149+0000 7f6013fff640 1 -- 172.21.15.27:0/1203385813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f601c0d2350 msgr2=0x7f601c0b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:07.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.149+0000 7f6013fff640 1 --2- 172.21.15.27:0/1203385813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f601c0d2350 0x7f601c0b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.149+0000 7f6013fff640 1 -- 172.21.15.27:0/1203385813 >> v1:172.21.15.192:6789/0 conn(0x7f601c1604d0 legacy=0x7f601c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:07.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.149+0000 7f601affd640 1 -- 172.21.15.27:0/1203385813 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 853807523 0 0) 0x7f601c0b5e90 con 0x7f601c1604d0 2023-12-19T18:23:07.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.149+0000 7f6013fff640 1 -- 172.21.15.27:0/1203385813 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f601c0cded0 con 0x7f601c0b4ef0 2023-12-19T18:23:07.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.149+0000 7f601bfff640 1 --2- 172.21.15.27:0/1203385813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f601c0d2350 0x7f601c0b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:23:07.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.150+0000 7f6013fff640 1 --2- 172.21.15.27:0/1203385813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f601c0b4ef0 0x7f601c0b6300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6004009870 tx=0x7f600402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42f52e8ed1b3abf8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.151+0000 7f601affd640 1 -- 172.21.15.27:0/1203385813 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004030840 con 0x7f601c0b4ef0 2023-12-19T18:23:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.151+0000 7f601affd640 1 -- 172.21.15.27:0/1203385813 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60040309a0 con 0x7f601c0b4ef0 2023-12-19T18:23:07.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.151+0000 7f601affd640 1 -- 172.21.15.27:0/1203385813 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6004030d30 con 0x7f601c0b4ef0 2023-12-19T18:23:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.151+0000 7f6020c6c640 1 -- 172.21.15.27:0/1203385813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f601c0b4ef0 msgr2=0x7f601c0b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.151+0000 7f6020c6c640 1 --2- 172.21.15.27:0/1203385813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f601c0b4ef0 0x7f601c0b6300 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6004009870 tx=0x7f600402b980 comp rx=0 tx=0).stop 2023-12-19T18:23:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.152+0000 7f6020c6c640 1 -- 172.21.15.27:0/1203385813 shutdown_connections 2023-12-19T18:23:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.152+0000 7f6020c6c640 1 --2- 172.21.15.27:0/1203385813 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f601c0d2350 0x7f601c0b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.152+0000 7f6020c6c640 1 --2- 172.21.15.27:0/1203385813 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f601c0b4ef0 0x7f601c0b6300 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.152+0000 7f6020c6c640 1 -- 172.21.15.27:0/1203385813 >> 172.21.15.27:0/1203385813 conn(0x7f601c0cf7e0 msgr2=0x7f601c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.152+0000 7f6020c6c640 1 -- 172.21.15.27:0/1203385813 shutdown_connections 2023-12-19T18:23:07.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.152+0000 7f6020c6c640 1 -- 172.21.15.27:0/1203385813 wait complete. 2023-12-19T18:23:07.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.154+0000 7f6020c6c640 1 Processor -- start 2023-12-19T18:23:07.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.154+0000 7f6020c6c640 1 -- start start 2023-12-19T18:23:07.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.155+0000 7f6020c6c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f601c0b4ef0 0x7f601c1543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:07.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.155+0000 7f6020c6c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f601c0d2350 0x7f601c0cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:07.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.155+0000 7f6020c6c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f601c1604d0 0x7f601c0cc630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:07.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.155+0000 7f6020c6c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c0c7980 con 0x7f601c0d2350 2023-12-19T18:23:07.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.155+0000 7f6020c6c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f601c0c6d70 con 0x7f601c1604d0 2023-12-19T18:23:07.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.155+0000 7f6020c6c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f601c0c7050 con 0x7f601c0b4ef0 2023-12-19T18:23:07.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.155+0000 7f6013fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f601c0d2350 0x7f601c0cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:07.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.155+0000 7f6013fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f601c0d2350 0x7f601c0cc0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52840/0 (socket says 172.21.15.27:52840) 2023-12-19T18:23:07.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.155+0000 7f6013fff640 1 -- 172.21.15.27:0/2010911967 learned_addr learned my addr 172.21.15.27:0/2010911967 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:07.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.156+0000 7f601b7fe640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f601c0b4ef0 0x7f601c1543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:07.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.156+0000 7f601bfff640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f601c1604d0 0x7f601c0cc630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:07.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.156+0000 7f6013fff640 1 -- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f601c0b4ef0 msgr2=0x7f601c1543a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:07.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.156+0000 7f6013fff640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f601c0b4ef0 0x7f601c1543a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.156+0000 7f6013fff640 1 -- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f601c1604d0 msgr2=0x7f601c0cc630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:07.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.156+0000 7f6013fff640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f601c1604d0 0x7f601c0cc630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.156+0000 7f6013fff640 1 -- 172.21.15.27:0/2010911967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6014009a70 con 0x7f601c0d2350 2023-12-19T18:23:07.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.156+0000 7f601b7fe640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f601c0b4ef0 0x7f601c1543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:23:07.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.156+0000 7f6013fff640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f601c0d2350 0x7f601c0cc0f0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6004009870 tx=0x7f60040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:07.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.157+0000 7f6018ff9640 1 -- 172.21.15.27:0/2010911967 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600403e070 con 0x7f601c0d2350 2023-12-19T18:23:07.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.157+0000 7f6018ff9640 1 -- 172.21.15.27:0/2010911967 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6004004030 con 0x7f601c0d2350 2023-12-19T18:23:07.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.157+0000 7f6018ff9640 1 -- 172.21.15.27:0/2010911967 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60040043e0 con 0x7f601c0d2350 2023-12-19T18:23:07.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.157+0000 7f6020c6c640 1 -- 172.21.15.27:0/2010911967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60040094d0 con 0x7f601c0d2350 2023-12-19T18:23:07.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.157+0000 7f6020c6c640 1 -- 172.21.15.27:0/2010911967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f601c0c7610 con 0x7f601c0d2350 2023-12-19T18:23:07.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.158+0000 7f6020c6c640 1 -- 172.21.15.27:0/2010911967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f601c066f80 con 0x7f601c0d2350 2023-12-19T18:23:07.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.159+0000 7f6018ff9640 1 -- 172.21.15.27:0/2010911967 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6004042070 con 0x7f601c0d2350 2023-12-19T18:23:07.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.160+0000 7f6018ff9640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ffc06a430 0x7f5ffc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:07.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.160+0000 7f6018ff9640 1 -- 172.21.15.27:0/2010911967 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(151..151 src has 1..151) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f60040aa850 con 0x7f601c0d2350 2023-12-19T18:23:07.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.160+0000 7f601b7fe640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ffc06a430 0x7f5ffc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:07.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.162+0000 7f601b7fe640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ffc06a430 0x7f5ffc06c8f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f600c000f60 tx=0x7f600c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:07.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.164+0000 7f6018ff9640 1 -- 172.21.15.27:0/2010911967 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6004075f50 con 0x7f601c0d2350 2023-12-19T18:23:07.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.359+0000 7f6020c6c640 1 -- 172.21.15.27:0/2010911967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f601c04e840 con 0x7f601c0d2350 2023-12-19T18:23:07.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.362+0000 7f6018ff9640 1 -- 172.21.15.27:0/2010911967 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v151) v1 ==== 74+0+16129 (secure 0 0 0) 0x7f6004079570 con 0x7f601c0d2350 2023-12-19T18:23:07.363 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:07.363 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":151,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:23:06.465222+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:23:04.431250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T18:23:05.506926+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:23:07.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.365+0000 7f6020c6c640 1 -- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ffc06a430 msgr2=0x7f5ffc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:07.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.365+0000 7f6020c6c640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ffc06a430 0x7f5ffc06c8f0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f600c000f60 tx=0x7f600c009040 comp rx=0 tx=0).stop 2023-12-19T18:23:07.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.365+0000 7f6020c6c640 1 -- 172.21.15.27:0/2010911967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f601c0d2350 msgr2=0x7f601c0cc0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:07.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.365+0000 7f6020c6c640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f601c0d2350 0x7f601c0cc0f0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f6004009870 tx=0x7f60040046c0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.365+0000 7f6020c6c640 1 -- 172.21.15.27:0/2010911967 shutdown_connections 2023-12-19T18:23:07.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.365+0000 7f6020c6c640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ffc06a430 0x7f5ffc06c8f0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.365+0000 7f6020c6c640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f601c1604d0 0x7f601c0cc630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.365+0000 7f6020c6c640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f601c0d2350 0x7f601c0cc0f0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.365+0000 7f6020c6c640 1 --2- 172.21.15.27:0/2010911967 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f601c0b4ef0 0x7f601c1543a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.365+0000 7f6020c6c640 1 -- 172.21.15.27:0/2010911967 >> 172.21.15.27:0/2010911967 conn(0x7f601c0cf7e0 msgr2=0x7f601c15f7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:07.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.365+0000 7f6020c6c640 1 -- 172.21.15.27:0/2010911967 shutdown_connections 2023-12-19T18:23:07.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.365+0000 7f6020c6c640 1 -- 172.21.15.27:0/2010911967 wait complete. 2023-12-19T18:23:07.386 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:23:07.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.624+0000 7fae6759e640 1 Processor -- start 2023-12-19T18:23:07.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.624+0000 7fae6759e640 1 -- start start 2023-12-19T18:23:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.625+0000 7fae6759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fae6815cfc0 0x7fae68160b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.625+0000 7fae6759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae680cdb30 0x7fae680cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.625+0000 7fae6759e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae680ce4d0 con 0x7fae680d2cf0 2023-12-19T18:23:07.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.625+0000 7fae6759e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae68161a50 con 0x7fae6815cfc0 2023-12-19T18:23:07.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.625+0000 7fae6759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae68161490 con 0x7fae680cdb30 2023-12-19T18:23:07.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.626+0000 7fae66d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fae680d2cf0 0x7fae680d3140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51736/0 (socket says 172.21.15.27:51736) 2023-12-19T18:23:07.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.626+0000 7fae66d9d640 1 -- 172.21.15.27:0/1440919676 learned_addr learned my addr 172.21.15.27:0/1440919676 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.626+0000 7fae6659c640 1 --2- 172.21.15.27:0/1440919676 >> v2:172.21.15.27:3300/0 conn(0x7fae6815cfc0 0x7fae68160b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.626+0000 7fae65d9b640 1 --2- 172.21.15.27:0/1440919676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae680cdb30 0x7fae680cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.626+0000 7fae6559a640 1 -- 172.21.15.27:0/1440919676 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636378700 0 0) 0x7fae680ce4d0 con 0x7fae680d2cf0 2023-12-19T18:23:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.626+0000 7fae6559a640 1 -- 172.21.15.27:0/1440919676 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae54003520 con 0x7fae680d2cf0 2023-12-19T18:23:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.626+0000 7fae6659c640 1 -- 172.21.15.27:0/1440919676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae680cdb30 msgr2=0x7fae680cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.626+0000 7fae6659c640 1 --2- 172.21.15.27:0/1440919676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae680cdb30 0x7fae680cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.626+0000 7fae6659c640 1 -- 172.21.15.27:0/1440919676 >> v1:172.21.15.27:6789/0 conn(0x7fae680d2cf0 legacy=0x7fae680d3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:07.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.626+0000 7fae6659c640 1 -- 172.21.15.27:0/1440919676 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae68161770 con 0x7fae6815cfc0 2023-12-19T18:23:07.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.628+0000 7fae6659c640 1 --2- 172.21.15.27:0/1440919676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae6815cfc0 0x7fae68160b40 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fae58009d90 tx=0x7fae5802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9842b5ca267994f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:07.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.628+0000 7fae6559a640 1 -- 172.21.15.27:0/1440919676 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5802cbd0 con 0x7fae6815cfc0 2023-12-19T18:23:07.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.628+0000 7fae6559a640 1 -- 172.21.15.27:0/1440919676 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae5802cd30 con 0x7fae6815cfc0 2023-12-19T18:23:07.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.628+0000 7fae6559a640 1 -- 172.21.15.27:0/1440919676 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5802cea0 con 0x7fae6815cfc0 2023-12-19T18:23:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.629+0000 7fae6759e640 1 -- 172.21.15.27:0/1440919676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae6815cfc0 msgr2=0x7fae68160b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.629+0000 7fae6759e640 1 --2- 172.21.15.27:0/1440919676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae6815cfc0 0x7fae68160b40 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fae58009d90 tx=0x7fae5802be20 comp rx=0 tx=0).stop 2023-12-19T18:23:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.629+0000 7fae6759e640 1 -- 172.21.15.27:0/1440919676 shutdown_connections 2023-12-19T18:23:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.629+0000 7fae6759e640 1 --2- 172.21.15.27:0/1440919676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae680cdb30 0x7fae680cdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.629+0000 7fae6759e640 1 --2- 172.21.15.27:0/1440919676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae6815cfc0 0x7fae68160b40 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.629+0000 7fae6759e640 1 -- 172.21.15.27:0/1440919676 >> 172.21.15.27:0/1440919676 conn(0x7fae68072e20 msgr2=0x7fae68073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:07.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.629+0000 7fae6759e640 1 -- 172.21.15.27:0/1440919676 shutdown_connections 2023-12-19T18:23:07.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.629+0000 7fae6759e640 1 -- 172.21.15.27:0/1440919676 wait complete. 2023-12-19T18:23:07.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.631+0000 7fae6759e640 1 Processor -- start 2023-12-19T18:23:07.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.631+0000 7fae6759e640 1 -- start start 2023-12-19T18:23:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae680cdb30 0x7fae680c8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae680d2cf0 0x7fae680cc700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae6815cfc0 0x7fae68142d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae680cb7a0 con 0x7fae680cdb30 2023-12-19T18:23:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae68145cd0 con 0x7fae680d2cf0 2023-12-19T18:23:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae680ca840 con 0x7fae6815cfc0 2023-12-19T18:23:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae680cdb30 0x7fae680c8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae680cdb30 0x7fae680c8a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52868/0 (socket says 172.21.15.27:52868) 2023-12-19T18:23:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6659c640 1 -- 172.21.15.27:0/1662854408 learned_addr learned my addr 172.21.15.27:0/1662854408 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:07.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae66d9d640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae6815cfc0 0x7fae68142d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6659c640 1 -- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae6815cfc0 msgr2=0x7fae68142d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6659c640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae6815cfc0 0x7fae68142d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae65d9b640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae680d2cf0 0x7fae680cc700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6659c640 1 -- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae680d2cf0 msgr2=0x7fae680cc700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6659c640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae680d2cf0 0x7fae680cc700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.632+0000 7fae6659c640 1 -- 172.21.15.27:0/1662854408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae58009a70 con 0x7fae680cdb30 2023-12-19T18:23:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.633+0000 7fae6659c640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae680cdb30 0x7fae680c8a70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fae58009d90 tx=0x7fae58004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.633+0000 7fae5effd640 1 -- 172.21.15.27:0/1662854408 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae5802cea0 con 0x7fae680cdb30 2023-12-19T18:23:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.633+0000 7fae5effd640 1 -- 172.21.15.27:0/1662854408 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae58040650 con 0x7fae680cdb30 2023-12-19T18:23:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.633+0000 7fae6759e640 1 -- 172.21.15.27:0/1662854408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae680cab20 con 0x7fae680cdb30 2023-12-19T18:23:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.633+0000 7fae5effd640 1 -- 172.21.15.27:0/1662854408 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae58040a00 con 0x7fae680cdb30 2023-12-19T18:23:07.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.633+0000 7fae6759e640 1 -- 172.21.15.27:0/1662854408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae680cb0e0 con 0x7fae680cdb30 2023-12-19T18:23:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.635+0000 7fae6759e640 1 -- 172.21.15.27:0/1662854408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae680d1260 con 0x7fae680cdb30 2023-12-19T18:23:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.635+0000 7fae5effd640 1 -- 172.21.15.27:0/1662854408 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fae58040b60 con 0x7fae680cdb30 2023-12-19T18:23:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.636+0000 7fae5effd640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae4806a430 0x7fae4806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:07.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.636+0000 7fae5effd640 1 -- 172.21.15.27:0/1662854408 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae580ab560 con 0x7fae680cdb30 2023-12-19T18:23:07.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.637+0000 7fae65d9b640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae4806a430 0x7fae4806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.638+0000 7fae65d9b640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae4806a430 0x7fae4806c8f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fae50006fd0 tx=0x7fae50008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:07.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.640+0000 7fae5effd640 1 -- 172.21.15.27:0/1662854408 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fae680d1260 con 0x7fae680cdb30 2023-12-19T18:23:07.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:07.873+0000 7fae6759e640 1 -- 172.21.15.27:0/1662854408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fae680d1260 con 0x7fae680cdb30 2023-12-19T18:23:08.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.480+0000 7fae5effd640 1 -- 172.21.15.27:0/1662854408 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v60) v1 ==== 118+0+0 (secure 0 0 0) 0x7fae680d1260 con 0x7fae680cdb30 2023-12-19T18:23:08.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.483+0000 7fae6759e640 1 -- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae4806a430 msgr2=0x7fae4806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:08.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.483+0000 7fae6759e640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae4806a430 0x7fae4806c8f0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fae50006fd0 tx=0x7fae50008040 comp rx=0 tx=0).stop 2023-12-19T18:23:08.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.483+0000 7fae6759e640 1 -- 172.21.15.27:0/1662854408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae680cdb30 msgr2=0x7fae680c8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:08.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.483+0000 7fae6759e640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae680cdb30 0x7fae680c8a70 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fae58009d90 tx=0x7fae58004a40 comp rx=0 tx=0).stop 2023-12-19T18:23:08.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.484+0000 7fae6759e640 1 -- 172.21.15.27:0/1662854408 shutdown_connections 2023-12-19T18:23:08.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.484+0000 7fae6759e640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fae4806a430 0x7fae4806c8f0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:08.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.484+0000 7fae6759e640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fae6815cfc0 0x7fae68142d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:08.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.484+0000 7fae6759e640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fae680d2cf0 0x7fae680cc700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:08.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.484+0000 7fae6759e640 1 --2- 172.21.15.27:0/1662854408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae680cdb30 0x7fae680c8a70 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:08.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.484+0000 7fae6759e640 1 -- 172.21.15.27:0/1662854408 >> 172.21.15.27:0/1662854408 conn(0x7fae68072e20 msgr2=0x7fae680cfba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:08.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.484+0000 7fae6759e640 1 -- 172.21.15.27:0/1662854408 shutdown_connections 2023-12-19T18:23:08.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.484+0000 7fae6759e640 1 -- 172.21.15.27:0/1662854408 wait complete. 2023-12-19T18:23:08.507 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:23:08.507 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:23:08.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.743+0000 7fa3f1b57640 1 Processor -- start 2023-12-19T18:23:08.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.743+0000 7fa3f1b57640 1 -- start start 2023-12-19T18:23:08.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.745+0000 7fa3f1b57640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa3ec158790 0x7fa3ec0ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:08.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.745+0000 7fa3f1b57640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3ec0adc50 0x7fa3ec15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:08.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.745+0000 7fa3f1b57640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa3ec15b810 0x7fa3ec15e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:08.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.745+0000 7fa3f1b57640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ec0c99d0 con 0x7fa3ec15b810 2023-12-19T18:23:08.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.745+0000 7fa3f1b57640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3ec0ae100 con 0x7fa3ec158790 2023-12-19T18:23:08.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.745+0000 7fa3f1b57640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec158b70 con 0x7fa3ec0adc50 2023-12-19T18:23:08.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.745+0000 7fa3f1356640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa3ec15b810 0x7fa3ec15e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:08.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.746+0000 7fa3f1356640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa3ec15b810 0x7fa3ec15e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52882/0 (socket says 172.21.15.27:52882) 2023-12-19T18:23:08.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.746+0000 7fa3f1356640 1 -- 172.21.15.27:0/408409789 learned_addr learned my addr 172.21.15.27:0/408409789 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:08.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.746+0000 7fa3e3fff640 1 --2- 172.21.15.27:0/408409789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3ec0adc50 0x7fa3ec15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:08.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.746+0000 7fa3f0b55640 1 --2- 172.21.15.27:0/408409789 >> v2:172.21.15.192:3300/0 conn(0x7fa3ec158790 0x7fa3ec0ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:08.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.746+0000 7fa3f1356640 1 -- 172.21.15.27:0/408409789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3ec0adc50 msgr2=0x7fa3ec15b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:08.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.746+0000 7fa3f1356640 1 --2- 172.21.15.27:0/408409789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3ec0adc50 0x7fa3ec15b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:08.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.746+0000 7fa3f1356640 1 -- 172.21.15.27:0/408409789 >> v2:172.21.15.192:3300/0 conn(0x7fa3ec158790 msgr2=0x7fa3ec0ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:08.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.746+0000 7fa3f1356640 1 --2- 172.21.15.27:0/408409789 >> v2:172.21.15.192:3300/0 conn(0x7fa3ec158790 0x7fa3ec0ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:08.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.746+0000 7fa3f1356640 1 -- 172.21.15.27:0/408409789 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3ec15c030 con 0x7fa3ec15b810 2023-12-19T18:23:08.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.746+0000 7fa3f0b55640 1 --2- 172.21.15.27:0/408409789 >> v2:172.21.15.192:3300/0 conn(0x7fa3ec158790 0x7fa3ec0ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:08.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.746+0000 7fa3e3fff640 1 --2- 172.21.15.27:0/408409789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3ec0adc50 0x7fa3ec15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:08.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.747+0000 7fa3f1356640 1 --2- 172.21.15.27:0/408409789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ec15b810 0x7fa3ec15e730 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec06f8d0 tx=0x7fa3e402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c1f525aa83b75e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:08.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.747+0000 7fa3e37fe640 1 -- 172.21.15.27:0/408409789 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e402ca70 con 0x7fa3ec15b810 2023-12-19T18:23:08.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.748+0000 7fa3e37fe640 1 -- 172.21.15.27:0/408409789 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e402cbd0 con 0x7fa3ec15b810 2023-12-19T18:23:08.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.748+0000 7fa3e37fe640 1 -- 172.21.15.27:0/408409789 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e40343f0 con 0x7fa3ec15b810 2023-12-19T18:23:08.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.748+0000 7fa3f1b57640 1 -- 172.21.15.27:0/408409789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ec15b810 msgr2=0x7fa3ec15e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:08.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.748+0000 7fa3f1b57640 1 --2- 172.21.15.27:0/408409789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ec15b810 0x7fa3ec15e730 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec06f8d0 tx=0x7fa3e402bed0 comp rx=0 tx=0).stop 2023-12-19T18:23:08.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.748+0000 7fa3f1b57640 1 -- 172.21.15.27:0/408409789 shutdown_connections 2023-12-19T18:23:08.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.748+0000 7fa3f1b57640 1 --2- 172.21.15.27:0/408409789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ec15b810 0x7fa3ec15e730 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:08.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.748+0000 7fa3f1b57640 1 --2- 172.21.15.27:0/408409789 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3ec0adc50 0x7fa3ec15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:08.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.748+0000 7fa3f1b57640 1 --2- 172.21.15.27:0/408409789 >> v2:172.21.15.192:3300/0 conn(0x7fa3ec158790 0x7fa3ec0ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:08.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.748+0000 7fa3f1b57640 1 -- 172.21.15.27:0/408409789 >> 172.21.15.27:0/408409789 conn(0x7fa3ec0b05b0 msgr2=0x7fa3ec0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:08.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.748+0000 7fa3f1b57640 1 -- 172.21.15.27:0/408409789 shutdown_connections 2023-12-19T18:23:08.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.748+0000 7fa3f1b57640 1 -- 172.21.15.27:0/408409789 wait complete. 2023-12-19T18:23:08.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.750+0000 7fa3f1b57640 1 Processor -- start 2023-12-19T18:23:08.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.750+0000 7fa3f1b57640 1 -- start start 2023-12-19T18:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.751+0000 7fa3f1b57640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ec0adc50 0x7fa3ec150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.751+0000 7fa3f1b57640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3ec158790 0x7fa3ec0acf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.751+0000 7fa3f1b57640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3ec15b810 0x7fa3ec06c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.751+0000 7fa3f1b57640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec06ca60 con 0x7fa3ec0adc50 2023-12-19T18:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.751+0000 7fa3f1b57640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec0ad8f0 con 0x7fa3ec15b810 2023-12-19T18:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.751+0000 7fa3f1b57640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3ec1f2270 con 0x7fa3ec158790 2023-12-19T18:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.751+0000 7fa3f0b55640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ec0adc50 0x7fa3ec150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.752+0000 7fa3f0b55640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ec0adc50 0x7fa3ec150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52892/0 (socket says 172.21.15.27:52892) 2023-12-19T18:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.752+0000 7fa3f0b55640 1 -- 172.21.15.27:0/3347659891 learned_addr learned my addr 172.21.15.27:0/3347659891 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.752+0000 7fa3f0b55640 1 -- 172.21.15.27:0/3347659891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3ec158790 msgr2=0x7fa3ec0acf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.752+0000 7fa3f0b55640 1 --2- 172.21.15.27:0/3347659891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3ec158790 0x7fa3ec0acf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:08.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.752+0000 7fa3f0b55640 1 -- 172.21.15.27:0/3347659891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3ec15b810 msgr2=0x7fa3ec06c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:08.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.752+0000 7fa3f0b55640 1 --2- 172.21.15.27:0/3347659891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3ec15b810 0x7fa3ec06c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:08.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.752+0000 7fa3f0b55640 1 -- 172.21.15.27:0/3347659891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3e4009b30 con 0x7fa3ec0adc50 2023-12-19T18:23:08.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.752+0000 7fa3f0b55640 1 --2- 172.21.15.27:0/3347659891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ec0adc50 0x7fa3ec150030 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800c820 tx=0x7fa3e800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:08.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.752+0000 7fa3e17fa640 1 -- 172.21.15.27:0/3347659891 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8007a30 con 0x7fa3ec0adc50 2023-12-19T18:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.753+0000 7fa3e17fa640 1 -- 172.21.15.27:0/3347659891 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e8007b90 con 0x7fa3ec0adc50 2023-12-19T18:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.753+0000 7fa3e17fa640 1 -- 172.21.15.27:0/3347659891 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e800ce70 con 0x7fa3ec0adc50 2023-12-19T18:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.753+0000 7fa3f1b57640 1 -- 172.21.15.27:0/3347659891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3ec0ad610 con 0x7fa3ec0adc50 2023-12-19T18:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.753+0000 7fa3f1b57640 1 -- 172.21.15.27:0/3347659891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ec0cc900 con 0x7fa3ec0adc50 2023-12-19T18:23:08.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.754+0000 7fa3f1b57640 1 -- 172.21.15.27:0/3347659891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3ec04e840 con 0x7fa3ec0adc50 2023-12-19T18:23:08.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.755+0000 7fa3e17fa640 1 -- 172.21.15.27:0/3347659891 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa3e80113f0 con 0x7fa3ec0adc50 2023-12-19T18:23:08.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.756+0000 7fa3e17fa640 1 --2- 172.21.15.27:0/3347659891 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa3d806a430 0x7fa3d806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:08.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.756+0000 7fa3e17fa640 1 -- 172.21.15.27:0/3347659891 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa3e8088b50 con 0x7fa3ec0adc50 2023-12-19T18:23:08.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.757+0000 7fa3e3fff640 1 --2- 172.21.15.27:0/3347659891 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa3d806a430 0x7fa3d806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:08.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.758+0000 7fa3e3fff640 1 --2- 172.21.15.27:0/3347659891 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa3d806a430 0x7fa3d806c8f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc006b10 tx=0x7fa3dc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:08.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.761+0000 7fa3e17fa640 1 -- 172.21.15.27:0/3347659891 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa3e80540a0 con 0x7fa3ec0adc50 2023-12-19T18:23:08.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:08.993+0000 7fa3f1b57640 1 -- 172.21.15.27:0/3347659891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fa3ec0c4e20 con 0x7fa3ec0adc50 2023-12-19T18:23:09.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:09.987+0000 7fa3e17fa640 1 -- 172.21.15.27:0/3347659891 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v61) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa3e801f0c0 con 0x7fa3ec0adc50 2023-12-19T18:23:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:09.990+0000 7fa3f1b57640 1 -- 172.21.15.27:0/3347659891 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa3d806a430 msgr2=0x7fa3d806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:09.990+0000 7fa3f1b57640 1 --2- 172.21.15.27:0/3347659891 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa3d806a430 0x7fa3d806c8f0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa3dc006b10 tx=0x7fa3dc008040 comp rx=0 tx=0).stop 2023-12-19T18:23:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:09.990+0000 7fa3f1b57640 1 -- 172.21.15.27:0/3347659891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ec0adc50 msgr2=0x7fa3ec150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:09.990+0000 7fa3f1b57640 1 --2- 172.21.15.27:0/3347659891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ec0adc50 0x7fa3ec150030 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fa3e800c820 tx=0x7fa3e800cc20 comp rx=0 tx=0).stop 2023-12-19T18:23:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:09.990+0000 7fa3f1b57640 1 -- 172.21.15.27:0/3347659891 shutdown_connections 2023-12-19T18:23:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:09.990+0000 7fa3f1b57640 1 --2- 172.21.15.27:0/3347659891 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa3d806a430 0x7fa3d806c8f0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:09.990+0000 7fa3f1b57640 1 --2- 172.21.15.27:0/3347659891 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa3ec15b810 0x7fa3ec06c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:09.990+0000 7fa3f1b57640 1 --2- 172.21.15.27:0/3347659891 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa3ec158790 0x7fa3ec0acf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:09.990+0000 7fa3f1b57640 1 --2- 172.21.15.27:0/3347659891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3ec0adc50 0x7fa3ec150030 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:09.990+0000 7fa3f1b57640 1 -- 172.21.15.27:0/3347659891 >> 172.21.15.27:0/3347659891 conn(0x7fa3ec0b05b0 msgr2=0x7fa3ec156140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:09.991+0000 7fa3f1b57640 1 -- 172.21.15.27:0/3347659891 shutdown_connections 2023-12-19T18:23:09.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:09.991+0000 7fa3f1b57640 1 -- 172.21.15.27:0/3347659891 wait complete. 2023-12-19T18:23:10.013 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:23:10.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:23:10.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.249+0000 7fdc9f1ce640 1 Processor -- start 2023-12-19T18:23:10.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.249+0000 7fdc9f1ce640 1 -- start start 2023-12-19T18:23:10.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.251+0000 7fdc9f1ce640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fdc9815af70 0x7fdc980ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:10.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.251+0000 7fdc9f1ce640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdc980adc20 0x7fdc9806b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:10.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.251+0000 7fdc9f1ce640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc980cb850 con 0x7fdc980cbce0 2023-12-19T18:23:10.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.251+0000 7fdc9f1ce640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc9815b350 con 0x7fdc980adc20 2023-12-19T18:23:10.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.251+0000 7fdc9f1ce640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc980cac40 con 0x7fdc9815af70 2023-12-19T18:23:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.251+0000 7fdc9d76b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdc980adc20 0x7fdc9806b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.251+0000 7fdc9d76b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdc980adc20 0x7fdc9806b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52904/0 (socket says 172.21.15.27:52904) 2023-12-19T18:23:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.251+0000 7fdc9d76b640 1 -- 172.21.15.27:0/398667246 learned_addr learned my addr 172.21.15.27:0/398667246 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.251+0000 7fdc9cf6a640 1 --2- 172.21.15.27:0/398667246 >> v2:172.21.15.192:3300/0 conn(0x7fdc9815af70 0x7fdc980ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.252+0000 7fdc9d76b640 1 -- 172.21.15.27:0/398667246 >> v2:172.21.15.192:3300/0 conn(0x7fdc9815af70 msgr2=0x7fdc980ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.252+0000 7fdc9d76b640 1 --2- 172.21.15.27:0/398667246 >> v2:172.21.15.192:3300/0 conn(0x7fdc9815af70 0x7fdc980ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.252+0000 7fdc9d76b640 1 -- 172.21.15.27:0/398667246 >> v1:172.21.15.192:6789/0 conn(0x7fdc980cbce0 legacy=0x7fdc980ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.252+0000 7fdc9d76b640 1 -- 172.21.15.27:0/398667246 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc980caf20 con 0x7fdc980adc20 2023-12-19T18:23:10.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.253+0000 7fdc9d76b640 1 --2- 172.21.15.27:0/398667246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc980adc20 0x7fdc9806b4e0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fdc9806d940 tx=0x7fdc9002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f0ea78599f77740 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:10.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.253+0000 7fdc977fe640 1 -- 172.21.15.27:0/398667246 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc9002ca80 con 0x7fdc980adc20 2023-12-19T18:23:10.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.253+0000 7fdc977fe640 1 -- 172.21.15.27:0/398667246 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc9002cbe0 con 0x7fdc980adc20 2023-12-19T18:23:10.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.253+0000 7fdc977fe640 1 -- 172.21.15.27:0/398667246 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc900343f0 con 0x7fdc980adc20 2023-12-19T18:23:10.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.254+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/398667246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc980adc20 msgr2=0x7fdc9806b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:10.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.254+0000 7fdc9f1ce640 1 --2- 172.21.15.27:0/398667246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc980adc20 0x7fdc9806b4e0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fdc9806d940 tx=0x7fdc9002bee0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.254+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/398667246 shutdown_connections 2023-12-19T18:23:10.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.254+0000 7fdc9f1ce640 1 --2- 172.21.15.27:0/398667246 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc980adc20 0x7fdc9806b4e0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.254+0000 7fdc9f1ce640 1 --2- 172.21.15.27:0/398667246 >> v2:172.21.15.192:3300/0 conn(0x7fdc9815af70 0x7fdc980ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.254+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/398667246 >> 172.21.15.27:0/398667246 conn(0x7fdc9806a330 msgr2=0x7fdc980b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:10.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.254+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/398667246 shutdown_connections 2023-12-19T18:23:10.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.254+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/398667246 wait complete. 2023-12-19T18:23:10.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.256+0000 7fdc9f1ce640 1 Processor -- start 2023-12-19T18:23:10.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.256+0000 7fdc9f1ce640 1 -- start start 2023-12-19T18:23:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.257+0000 7fdc9f1ce640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc980adc20 0x7fdc98158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.257+0000 7fdc9f1ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc980cbce0 0x7fdc9815bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.257+0000 7fdc9f1ce640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc9815af70 0x7fdc980af8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.257+0000 7fdc9f1ce640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9815da40 con 0x7fdc980cbce0 2023-12-19T18:23:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.257+0000 7fdc9f1ce640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc981587b0 con 0x7fdc980adc20 2023-12-19T18:23:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.257+0000 7fdc9f1ce640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc9815c6b0 con 0x7fdc9815af70 2023-12-19T18:23:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.257+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc980cbce0 0x7fdc9815bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.257+0000 7fdc97fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc980cbce0 0x7fdc9815bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52906/0 (socket says 172.21.15.27:52906) 2023-12-19T18:23:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.257+0000 7fdc97fff640 1 -- 172.21.15.27:0/2757287531 learned_addr learned my addr 172.21.15.27:0/2757287531 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.258+0000 7fdc9cf6a640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc980adc20 0x7fdc98158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.258+0000 7fdc9d76b640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc9815af70 0x7fdc980af8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.258+0000 7fdc97fff640 1 -- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc9815af70 msgr2=0x7fdc980af8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.258+0000 7fdc97fff640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc9815af70 0x7fdc980af8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.258+0000 7fdc97fff640 1 -- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc980adc20 msgr2=0x7fdc98158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.258+0000 7fdc97fff640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc980adc20 0x7fdc98158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.258+0000 7fdc97fff640 1 -- 172.21.15.27:0/2757287531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc90009b30 con 0x7fdc980cbce0 2023-12-19T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.258+0000 7fdc97fff640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc980cbce0 0x7fdc9815bd80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c400 tx=0x7fdc8400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.258+0000 7fdc957fa640 1 -- 172.21.15.27:0/2757287531 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84011790 con 0x7fdc980cbce0 2023-12-19T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.259+0000 7fdc957fa640 1 -- 172.21.15.27:0/2757287531 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc84011930 con 0x7fdc980cbce0 2023-12-19T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.259+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/2757287531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc980ccaa0 con 0x7fdc980cbce0 2023-12-19T18:23:10.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.259+0000 7fdc957fa640 1 -- 172.21.15.27:0/2757287531 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc84011ce0 con 0x7fdc980cbce0 2023-12-19T18:23:10.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.259+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/2757287531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc980cd060 con 0x7fdc980cbce0 2023-12-19T18:23:10.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.260+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/2757287531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc980cd280 con 0x7fdc980cbce0 2023-12-19T18:23:10.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.261+0000 7fdc957fa640 1 -- 172.21.15.27:0/2757287531 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdc8401c460 con 0x7fdc980cbce0 2023-12-19T18:23:10.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.262+0000 7fdc957fa640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc7c06a430 0x7fdc7c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:10.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.262+0000 7fdc957fa640 1 -- 172.21.15.27:0/2757287531 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc84088940 con 0x7fdc980cbce0 2023-12-19T18:23:10.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.263+0000 7fdc9cf6a640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc7c06a430 0x7fdc7c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:10.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.264+0000 7fdc9cf6a640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc7c06a430 0x7fdc7c06c8f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c006fd0 tx=0x7fdc8c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:10.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.267+0000 7fdc957fa640 1 -- 172.21.15.27:0/2757287531 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdc84053e90 con 0x7fdc980cbce0 2023-12-19T18:23:10.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.498+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/2757287531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdc980b3760 con 0x7fdc980cbce0 2023-12-19T18:23:10.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.499+0000 7fdc957fa640 1 -- 172.21.15.27:0/2757287531 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fdc840574b0 con 0x7fdc980cbce0 2023-12-19T18:23:10.500 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:10.500 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":61,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3298228307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3298228307},{"type":"v1","addr":"172.21.15.192:6835","nonce":3298228307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5726,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1760483999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1760483999},{"type":"v1","addr":"172.21.15.192:6837","nonce":1760483999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3000637822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3000637822},{"type":"v1","addr":"172.21.15.27:6833","nonce":3000637822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:23:06.465885+0000","modified":"2023-12-19T18:23:09.973228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1593995318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1593995318},{"type":"v1","addr":"172.21.15.27:6835","nonce":1593995318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T18:23:10.501 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 61 2023-12-19T18:23:10.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.502+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc7c06a430 msgr2=0x7fdc7c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.502+0000 7fdc9f1ce640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc7c06a430 0x7fdc7c06c8f0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fdc8c006fd0 tx=0x7fdc8c008040 comp rx=0 tx=0).stop 2023-12-19T18:23:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.502+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/2757287531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc980cbce0 msgr2=0x7fdc9815bd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.502+0000 7fdc9f1ce640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc980cbce0 0x7fdc9815bd80 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fdc8400c400 tx=0x7fdc8400c8c0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.502+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/2757287531 shutdown_connections 2023-12-19T18:23:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.502+0000 7fdc9f1ce640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc7c06a430 0x7fdc7c06c8f0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.502+0000 7fdc9f1ce640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc9815af70 0x7fdc980af8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.502+0000 7fdc9f1ce640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc980cbce0 0x7fdc9815bd80 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.502+0000 7fdc9f1ce640 1 --2- 172.21.15.27:0/2757287531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc980adc20 0x7fdc98158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.502+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/2757287531 >> 172.21.15.27:0/2757287531 conn(0x7fdc9806a330 msgr2=0x7fdc98159000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.503+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/2757287531 shutdown_connections 2023-12-19T18:23:10.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.503+0000 7fdc9f1ce640 1 -- 172.21.15.27:0/2757287531 wait complete. 2023-12-19T18:23:10.521 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:23:10.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.753+0000 7f66966fb640 1 Processor -- start 2023-12-19T18:23:10.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.754+0000 7f66966fb640 1 -- start start 2023-12-19T18:23:10.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.755+0000 7f66966fb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f66900ce420 0x7f66900d2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.755+0000 7f66966fb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66900d1970 con 0x7f66900cdf90 2023-12-19T18:23:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.755+0000 7f66966fb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f669006a5e0 con 0x7f669015d240 2023-12-19T18:23:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.755+0000 7f66966fb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66900ce890 con 0x7f66900ce420 2023-12-19T18:23:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.756+0000 7f6687fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f66900cdf90 0x7f66900d1e00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51742/0 (socket says 172.21.15.27:51742) 2023-12-19T18:23:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.756+0000 7f6687fff640 1 -- 172.21.15.27:0/1706751185 learned_addr learned my addr 172.21.15.27:0/1706751185 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.756+0000 7f6694c98640 1 --2- 172.21.15.27:0/1706751185 >> v2:172.21.15.27:3300/0 conn(0x7f66900ce420 0x7f66900d2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.756+0000 7f668f7fe640 1 -- 172.21.15.27:0/1706751185 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105666971 0 0) 0x7f66900d1970 con 0x7f66900cdf90 2023-12-19T18:23:10.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.756+0000 7f6694c98640 1 -- 172.21.15.27:0/1706751185 >> v1:172.21.15.192:6789/0 conn(0x7f669015d240 legacy=0x7f66901618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:10.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.756+0000 7f6694c98640 1 -- 172.21.15.27:0/1706751185 >> v1:172.21.15.27:6789/0 conn(0x7f66900cdf90 legacy=0x7f66900d1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:10.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.756+0000 7f6694c98640 1 -- 172.21.15.27:0/1706751185 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f669006a880 con 0x7f66900ce420 2023-12-19T18:23:10.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.757+0000 7f6694c98640 1 --2- 172.21.15.27:0/1706751185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66900ce420 0x7f66900d2510 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6688009d90 tx=0x7f668802bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76148fc6a785e43e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:10.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.758+0000 7f668f7fe640 1 -- 172.21.15.27:0/1706751185 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668802cd00 con 0x7f66900ce420 2023-12-19T18:23:10.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.758+0000 7f668f7fe640 1 -- 172.21.15.27:0/1706751185 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668802ce60 con 0x7f66900ce420 2023-12-19T18:23:10.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.758+0000 7f668f7fe640 1 -- 172.21.15.27:0/1706751185 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6688034660 con 0x7f66900ce420 2023-12-19T18:23:10.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.758+0000 7f66966fb640 1 -- 172.21.15.27:0/1706751185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66900ce420 msgr2=0x7f66900d2510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:10.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.758+0000 7f66966fb640 1 --2- 172.21.15.27:0/1706751185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66900ce420 0x7f66900d2510 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f6688009d90 tx=0x7f668802bf50 comp rx=0 tx=0).stop 2023-12-19T18:23:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.758+0000 7f66966fb640 1 -- 172.21.15.27:0/1706751185 shutdown_connections 2023-12-19T18:23:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.758+0000 7f66966fb640 1 --2- 172.21.15.27:0/1706751185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66900ce420 0x7f66900d2510 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.758+0000 7f66966fb640 1 -- 172.21.15.27:0/1706751185 >> 172.21.15.27:0/1706751185 conn(0x7f66900d31e0 msgr2=0x7f66900d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.759+0000 7f66966fb640 1 -- 172.21.15.27:0/1706751185 shutdown_connections 2023-12-19T18:23:10.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.759+0000 7f66966fb640 1 -- 172.21.15.27:0/1706751185 wait complete. 2023-12-19T18:23:10.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.760+0000 7f66966fb640 1 Processor -- start 2023-12-19T18:23:10.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.761+0000 7f66966fb640 1 -- start start 2023-12-19T18:23:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f66966fb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66900cdf90 0x7f6690154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f66966fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66900ce420 0x7f6690152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f66966fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f669015d240 0x7f66901576b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:10.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f66966fb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6690153770 con 0x7f66900ce420 2023-12-19T18:23:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f66966fb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6690153a50 con 0x7f66900cdf90 2023-12-19T18:23:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f66966fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6690153d30 con 0x7f669015d240 2023-12-19T18:23:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f6687fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66900ce420 0x7f6690152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f6687fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66900ce420 0x7f6690152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52922/0 (socket says 172.21.15.27:52922) 2023-12-19T18:23:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f6687fff640 1 -- 172.21.15.27:0/1154206155 learned_addr learned my addr 172.21.15.27:0/1154206155 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f6694c98640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f669015d240 0x7f66901576b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f668ffff640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66900cdf90 0x7f6690154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f6687fff640 1 -- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f669015d240 msgr2=0x7f66901576b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f6687fff640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f669015d240 0x7f66901576b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f6687fff640 1 -- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66900cdf90 msgr2=0x7f6690154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f6687fff640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66900cdf90 0x7f6690154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.762+0000 7f6687fff640 1 -- 172.21.15.27:0/1154206155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6688009a70 con 0x7f66900ce420 2023-12-19T18:23:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.763+0000 7f668ffff640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66900cdf90 0x7f6690154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.763+0000 7f6694c98640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f669015d240 0x7f66901576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.763+0000 7f6687fff640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66900ce420 0x7f6690152e70 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f667800c4d0 tx=0x7f667800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.763+0000 7f668d7fa640 1 -- 172.21.15.27:0/1154206155 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66780074e0 con 0x7f66900ce420 2023-12-19T18:23:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.763+0000 7f668d7fa640 1 -- 172.21.15.27:0/1154206155 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6678007660 con 0x7f66900ce420 2023-12-19T18:23:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.763+0000 7f66966fb640 1 -- 172.21.15.27:0/1154206155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6690154010 con 0x7f66900ce420 2023-12-19T18:23:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.763+0000 7f668d7fa640 1 -- 172.21.15.27:0/1154206155 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678004a20 con 0x7f66900ce420 2023-12-19T18:23:10.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.763+0000 7f66966fb640 1 -- 172.21.15.27:0/1154206155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66901533b0 con 0x7f66900ce420 2023-12-19T18:23:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.765+0000 7f66966fb640 1 -- 172.21.15.27:0/1154206155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6690154530 con 0x7f66900ce420 2023-12-19T18:23:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.766+0000 7f668d7fa640 1 -- 172.21.15.27:0/1154206155 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6678004b80 con 0x7f66900ce420 2023-12-19T18:23:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.766+0000 7f668d7fa640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f667006a430 0x7f667006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:10.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.767+0000 7f668d7fa640 1 -- 172.21.15.27:0/1154206155 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6678087af0 con 0x7f66900ce420 2023-12-19T18:23:10.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.767+0000 7f668ffff640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f667006a430 0x7f667006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:10.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.768+0000 7f668ffff640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f667006a430 0x7f667006c8f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f668000a830 tx=0x7f6680008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:10.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.771+0000 7f668d7fa640 1 -- 172.21.15.27:0/1154206155 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6690154530 con 0x7f66900ce420 2023-12-19T18:23:10.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.965+0000 7f66966fb640 1 -- 172.21.15.27:0/1154206155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6690154530 con 0x7f66900ce420 2023-12-19T18:23:10.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.968+0000 7f668d7fa640 1 -- 172.21.15.27:0/1154206155 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v152) v1 ==== 74+0+16116 (secure 0 0 0) 0x7f6690154530 con 0x7f66900ce420 2023-12-19T18:23:10.968 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:10.968 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":152,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:23:07.469802+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:23:04.431250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T18:23:05.506926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"152","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:23:10.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.971+0000 7f66966fb640 1 -- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f667006a430 msgr2=0x7f667006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:10.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.971+0000 7f66966fb640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f667006a430 0x7f667006c8f0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f668000a830 tx=0x7f6680008040 comp rx=0 tx=0).stop 2023-12-19T18:23:10.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.971+0000 7f66966fb640 1 -- 172.21.15.27:0/1154206155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66900ce420 msgr2=0x7f6690152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:10.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.971+0000 7f66966fb640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66900ce420 0x7f6690152e70 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f667800c4d0 tx=0x7f667800c990 comp rx=0 tx=0).stop 2023-12-19T18:23:10.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.971+0000 7f66966fb640 1 -- 172.21.15.27:0/1154206155 shutdown_connections 2023-12-19T18:23:10.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.971+0000 7f66966fb640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f667006a430 0x7f667006c8f0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.971+0000 7f66966fb640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f669015d240 0x7f66901576b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.971+0000 7f66966fb640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66900ce420 0x7f6690152e70 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.971+0000 7f66966fb640 1 --2- 172.21.15.27:0/1154206155 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f66900cdf90 0x7f6690154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.971+0000 7f66966fb640 1 -- 172.21.15.27:0/1154206155 >> 172.21.15.27:0/1154206155 conn(0x7f66900d31e0 msgr2=0x7f669014f070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.971+0000 7f66966fb640 1 -- 172.21.15.27:0/1154206155 shutdown_connections 2023-12-19T18:23:10.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:10.971+0000 7f66966fb640 1 -- 172.21.15.27:0/1154206155 wait complete. 2023-12-19T18:23:10.992 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:23:10.993 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:23:11.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.229+0000 7fef19c69640 1 Processor -- start 2023-12-19T18:23:11.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.229+0000 7fef19c69640 1 -- start start 2023-12-19T18:23:11.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.231+0000 7fef19c69640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fef1415af70 0x7fef140ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.231+0000 7fef19c69640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef140adc20 0x7fef1406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.231+0000 7fef19c69640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef140cb850 con 0x7fef140cbce0 2023-12-19T18:23:11.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.231+0000 7fef19c69640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef1415b350 con 0x7fef140adc20 2023-12-19T18:23:11.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.231+0000 7fef19c69640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef140cac40 con 0x7fef1415af70 2023-12-19T18:23:11.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.231+0000 7fef13fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef140adc20 0x7fef1406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.231+0000 7fef13fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fef140adc20 0x7fef1406b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52932/0 (socket says 172.21.15.27:52932) 2023-12-19T18:23:11.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.231+0000 7fef13fff640 1 -- 172.21.15.27:0/1101792745 learned_addr learned my addr 172.21.15.27:0/1101792745 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:11.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.231+0000 7fef137fe640 1 --2- 172.21.15.27:0/1101792745 >> v2:172.21.15.192:3300/0 conn(0x7fef1415af70 0x7fef140ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.232+0000 7fef13fff640 1 -- 172.21.15.27:0/1101792745 >> v2:172.21.15.192:3300/0 conn(0x7fef1415af70 msgr2=0x7fef140ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:11.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.232+0000 7fef13fff640 1 --2- 172.21.15.27:0/1101792745 >> v2:172.21.15.192:3300/0 conn(0x7fef1415af70 0x7fef140ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.232+0000 7fef13fff640 1 -- 172.21.15.27:0/1101792745 >> v1:172.21.15.192:6789/0 conn(0x7fef140cbce0 legacy=0x7fef140ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:11.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.232+0000 7fef13fff640 1 -- 172.21.15.27:0/1101792745 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef140caf20 con 0x7fef140adc20 2023-12-19T18:23:11.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.233+0000 7fef13fff640 1 --2- 172.21.15.27:0/1101792745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef140adc20 0x7fef1406b4e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fef1406d940 tx=0x7fef0c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e46a57577172c23a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:11.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.233+0000 7fef127fc640 1 -- 172.21.15.27:0/1101792745 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c031870 con 0x7fef140adc20 2023-12-19T18:23:11.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.234+0000 7fef127fc640 1 -- 172.21.15.27:0/1101792745 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0c0319d0 con 0x7fef140adc20 2023-12-19T18:23:11.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.234+0000 7fef127fc640 1 -- 172.21.15.27:0/1101792745 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c031d60 con 0x7fef140adc20 2023-12-19T18:23:11.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.234+0000 7fef19c69640 1 -- 172.21.15.27:0/1101792745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef140adc20 msgr2=0x7fef1406b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:11.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.234+0000 7fef19c69640 1 --2- 172.21.15.27:0/1101792745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef140adc20 0x7fef1406b4e0 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fef1406d940 tx=0x7fef0c02b910 comp rx=0 tx=0).stop 2023-12-19T18:23:11.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.234+0000 7fef19c69640 1 -- 172.21.15.27:0/1101792745 shutdown_connections 2023-12-19T18:23:11.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.234+0000 7fef19c69640 1 --2- 172.21.15.27:0/1101792745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef140adc20 0x7fef1406b4e0 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.234+0000 7fef19c69640 1 --2- 172.21.15.27:0/1101792745 >> v2:172.21.15.192:3300/0 conn(0x7fef1415af70 0x7fef140ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.234+0000 7fef19c69640 1 -- 172.21.15.27:0/1101792745 >> 172.21.15.27:0/1101792745 conn(0x7fef1406a330 msgr2=0x7fef140b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:11.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.234+0000 7fef19c69640 1 -- 172.21.15.27:0/1101792745 shutdown_connections 2023-12-19T18:23:11.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.234+0000 7fef19c69640 1 -- 172.21.15.27:0/1101792745 wait complete. 2023-12-19T18:23:11.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.236+0000 7fef19c69640 1 Processor -- start 2023-12-19T18:23:11.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.236+0000 7fef19c69640 1 -- start start 2023-12-19T18:23:11.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.237+0000 7fef19c69640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef140adc20 0x7fef1415c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.237+0000 7fef19c69640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef140cbce0 0x7fef1414ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.237+0000 7fef19c69640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef1415af70 0x7fef140b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.237+0000 7fef19c69640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1415b530 con 0x7fef140adc20 2023-12-19T18:23:11.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.237+0000 7fef19c69640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1415b7d0 con 0x7fef1415af70 2023-12-19T18:23:11.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.237+0000 7fef19c69640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1415bab0 con 0x7fef140cbce0 2023-12-19T18:23:11.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.238+0000 7fef137fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef140adc20 0x7fef1415c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.238+0000 7fef137fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef140adc20 0x7fef1415c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52936/0 (socket says 172.21.15.27:52936) 2023-12-19T18:23:11.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.238+0000 7fef137fe640 1 -- 172.21.15.27:0/3792921154 learned_addr learned my addr 172.21.15.27:0/3792921154 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:11.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.238+0000 7fef12ffd640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef140cbce0 0x7fef1414ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.238+0000 7fef13fff640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef1415af70 0x7fef140b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.238+0000 7fef137fe640 1 -- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef140cbce0 msgr2=0x7fef1414ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:11.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.238+0000 7fef137fe640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef140cbce0 0x7fef1414ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.238+0000 7fef137fe640 1 -- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef1415af70 msgr2=0x7fef140b39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:11.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.238+0000 7fef137fe640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef1415af70 0x7fef140b39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.238+0000 7fef137fe640 1 -- 172.21.15.27:0/3792921154 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef0c0094d0 con 0x7fef140adc20 2023-12-19T18:23:11.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.238+0000 7fef13fff640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef1415af70 0x7fef140b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:23:11.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.239+0000 7fef137fe640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef140adc20 0x7fef1415c590 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fef0400efc0 tx=0x7fef0400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:11.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.239+0000 7fef0bfff640 1 -- 172.21.15.27:0/3792921154 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef04009b80 con 0x7fef140adc20 2023-12-19T18:23:11.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.239+0000 7fef0bfff640 1 -- 172.21.15.27:0/3792921154 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef0400f040 con 0x7fef140adc20 2023-12-19T18:23:11.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.239+0000 7fef19c69640 1 -- 172.21.15.27:0/3792921154 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1415bd90 con 0x7fef140adc20 2023-12-19T18:23:11.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.239+0000 7fef0bfff640 1 -- 172.21.15.27:0/3792921154 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef040146a0 con 0x7fef140adc20 2023-12-19T18:23:11.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.239+0000 7fef19c69640 1 -- 172.21.15.27:0/3792921154 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1414f4b0 con 0x7fef140adc20 2023-12-19T18:23:11.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.240+0000 7fef19c69640 1 -- 172.21.15.27:0/3792921154 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef14068e90 con 0x7fef140adc20 2023-12-19T18:23:11.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.241+0000 7fef0bfff640 1 -- 172.21.15.27:0/3792921154 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fef04014800 con 0x7fef140adc20 2023-12-19T18:23:11.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.242+0000 7fef0bfff640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feef406a460 0x7feef406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.242+0000 7fef0bfff640 1 -- 172.21.15.27:0/3792921154 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fef040893b0 con 0x7fef140adc20 2023-12-19T18:23:11.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.243+0000 7fef12ffd640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feef406a460 0x7feef406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.244+0000 7fef12ffd640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feef406a460 0x7feef406c920 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fef140ad140 tx=0x7feefc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:11.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.247+0000 7fef0bfff640 1 -- 172.21.15.27:0/3792921154 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fef04054a30 con 0x7fef140adc20 2023-12-19T18:23:11.416 INFO:tasks.ceph.mds.a.smithi027.stderr:2023-12-19T18:23:11.416+0000 7f442b6c6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:23:11.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.441+0000 7fef19c69640 1 -- 172.21.15.27:0/3792921154 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef140cb4e0 con 0x7feef406a460 2023-12-19T18:23:11.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.465+0000 7fef0bfff640 1 -- 172.21.15.27:0/3792921154 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282986 (secure 0 0 0) 0x7feefc0053f0 con 0x7feef406a460 2023-12-19T18:23:11.469 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:11.471 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:23:11.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.470+0000 7fef19c69640 1 -- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feef406a460 msgr2=0x7feef406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.470+0000 7fef19c69640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feef406a460 0x7feef406c920 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fef140ad140 tx=0x7feefc009040 comp rx=0 tx=0).stop 2023-12-19T18:23:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.471+0000 7fef19c69640 1 -- 172.21.15.27:0/3792921154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef140adc20 msgr2=0x7fef1415c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.471+0000 7fef19c69640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef140adc20 0x7fef1415c590 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7fef0400efc0 tx=0x7fef0400c490 comp rx=0 tx=0).stop 2023-12-19T18:23:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.471+0000 7fef19c69640 1 -- 172.21.15.27:0/3792921154 shutdown_connections 2023-12-19T18:23:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.471+0000 7fef19c69640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feef406a460 0x7feef406c920 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.471+0000 7fef19c69640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fef1415af70 0x7fef140b39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.471+0000 7fef19c69640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fef140cbce0 0x7fef1414ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.471+0000 7fef19c69640 1 --2- 172.21.15.27:0/3792921154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fef140adc20 0x7fef1415c590 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.471+0000 7fef19c69640 1 -- 172.21.15.27:0/3792921154 >> 172.21.15.27:0/3792921154 conn(0x7fef1406a330 msgr2=0x7fef140ad3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.471+0000 7fef19c69640 1 -- 172.21.15.27:0/3792921154 shutdown_connections 2023-12-19T18:23:11.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.471+0000 7fef19c69640 1 -- 172.21.15.27:0/3792921154 wait complete. 2023-12-19T18:23:11.493 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":397,"stamp":"2023-12-19T18:23:09.884382+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7810,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554689,"statfs":{"total":1600000000,"available":1592001720,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.924681"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196527+0000","last_change":"2023-12-19T18:23:06.507531+0000","last_active":"2023-12-19T18:23:08.196527+0000","last_peered":"2023-12-19T18:23:08.196527+0000","last_clean":"2023-12-19T18:23:08.196527+0000","last_became_active":"2023-12-19T18:23:06.506056+0000","last_became_peered":"2023-12-19T18:23:06.506056+0000","last_unstale":"2023-12-19T18:23:08.196527+0000","last_undegraded":"2023-12-19T18:23:08.196527+0000","last_fullsized":"2023-12-19T18:23:08.196527+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:56.690910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.477351+0000","last_change":"2023-12-19T18:23:06.492270+0000","last_active":"2023-12-19T18:23:07.477351+0000","last_peered":"2023-12-19T18:23:07.477351+0000","last_clean":"2023-12-19T18:23:07.477351+0000","last_became_active":"2023-12-19T18:23:06.491935+0000","last_became_peered":"2023-12-19T18:23:06.491935+0000","last_unstale":"2023-12-19T18:23:07.477351+0000","last_undegraded":"2023-12-19T18:23:07.477351+0000","last_fullsized":"2023-12-19T18:23:07.477351+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:41:32.488817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195894+0000","last_change":"2023-12-19T18:23:06.485808+0000","last_active":"2023-12-19T18:23:08.195894+0000","last_peered":"2023-12-19T18:23:08.195894+0000","last_clean":"2023-12-19T18:23:08.195894+0000","last_became_active":"2023-12-19T18:23:06.485575+0000","last_became_peered":"2023-12-19T18:23:06.485575+0000","last_unstale":"2023-12-19T18:23:08.195894+0000","last_undegraded":"2023-12-19T18:23:08.195894+0000","last_fullsized":"2023-12-19T18:23:08.195894+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:55:21.903528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479661+0000","last_change":"2023-12-19T18:23:06.495516+0000","last_active":"2023-12-19T18:23:07.479661+0000","last_peered":"2023-12-19T18:23:07.479661+0000","last_clean":"2023-12-19T18:23:07.479661+0000","last_became_active":"2023-12-19T18:23:06.494716+0000","last_became_peered":"2023-12-19T18:23:06.494716+0000","last_unstale":"2023-12-19T18:23:07.479661+0000","last_undegraded":"2023-12-19T18:23:07.479661+0000","last_fullsized":"2023-12-19T18:23:07.479661+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:21.096114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196988+0000","last_change":"2023-12-19T18:23:06.510753+0000","last_active":"2023-12-19T18:23:08.196988+0000","last_peered":"2023-12-19T18:23:08.196988+0000","last_clean":"2023-12-19T18:23:08.196988+0000","last_became_active":"2023-12-19T18:23:06.510070+0000","last_became_peered":"2023-12-19T18:23:06.510070+0000","last_unstale":"2023-12-19T18:23:08.196988+0000","last_undegraded":"2023-12-19T18:23:08.196988+0000","last_fullsized":"2023-12-19T18:23:08.196988+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:24:06.331884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475511+0000","last_change":"2023-12-19T18:23:06.496021+0000","last_active":"2023-12-19T18:23:07.475511+0000","last_peered":"2023-12-19T18:23:07.475511+0000","last_clean":"2023-12-19T18:23:07.475511+0000","last_became_active":"2023-12-19T18:23:06.495714+0000","last_became_peered":"2023-12-19T18:23:06.495714+0000","last_unstale":"2023-12-19T18:23:07.475511+0000","last_undegraded":"2023-12-19T18:23:07.475511+0000","last_fullsized":"2023-12-19T18:23:07.475511+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:43:41.974906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196569+0000","last_change":"2023-12-19T18:23:06.494420+0000","last_active":"2023-12-19T18:23:08.196569+0000","last_peered":"2023-12-19T18:23:08.196569+0000","last_clean":"2023-12-19T18:23:08.196569+0000","last_became_active":"2023-12-19T18:23:06.494147+0000","last_became_peered":"2023-12-19T18:23:06.494147+0000","last_unstale":"2023-12-19T18:23:08.196569+0000","last_undegraded":"2023-12-19T18:23:08.196569+0000","last_fullsized":"2023-12-19T18:23:08.196569+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:16.036766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196925+0000","last_change":"2023-12-19T18:23:06.493298+0000","last_active":"2023-12-19T18:23:08.196925+0000","last_peered":"2023-12-19T18:23:08.196925+0000","last_clean":"2023-12-19T18:23:08.196925+0000","last_became_active":"2023-12-19T18:23:06.493071+0000","last_became_peered":"2023-12-19T18:23:06.493071+0000","last_unstale":"2023-12-19T18:23:08.196925+0000","last_undegraded":"2023-12-19T18:23:08.196925+0000","last_fullsized":"2023-12-19T18:23:08.196925+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:17.841146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195747+0000","last_change":"2023-12-19T18:23:06.507449+0000","last_active":"2023-12-19T18:23:08.195747+0000","last_peered":"2023-12-19T18:23:08.195747+0000","last_clean":"2023-12-19T18:23:08.195747+0000","last_became_active":"2023-12-19T18:23:06.506616+0000","last_became_peered":"2023-12-19T18:23:06.506616+0000","last_unstale":"2023-12-19T18:23:08.195747+0000","last_undegraded":"2023-12-19T18:23:08.195747+0000","last_fullsized":"2023-12-19T18:23:08.195747+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:06:03.251131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196611+0000","last_change":"2023-12-19T18:23:06.508916+0000","last_active":"2023-12-19T18:23:08.196611+0000","last_peered":"2023-12-19T18:23:08.196611+0000","last_clean":"2023-12-19T18:23:08.196611+0000","last_became_active":"2023-12-19T18:23:06.508129+0000","last_became_peered":"2023-12-19T18:23:06.508129+0000","last_unstale":"2023-12-19T18:23:08.196611+0000","last_undegraded":"2023-12-19T18:23:08.196611+0000","last_fullsized":"2023-12-19T18:23:08.196611+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:16.673265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197160+0000","last_change":"2023-12-19T18:23:06.491479+0000","last_active":"2023-12-19T18:23:08.197160+0000","last_peered":"2023-12-19T18:23:08.197160+0000","last_clean":"2023-12-19T18:23:08.197160+0000","last_became_active":"2023-12-19T18:23:06.491250+0000","last_became_peered":"2023-12-19T18:23:06.491250+0000","last_unstale":"2023-12-19T18:23:08.197160+0000","last_undegraded":"2023-12-19T18:23:08.197160+0000","last_fullsized":"2023-12-19T18:23:08.197160+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:41:59.650926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195570+0000","last_change":"2023-12-19T18:23:06.501465+0000","last_active":"2023-12-19T18:23:08.195570+0000","last_peered":"2023-12-19T18:23:08.195570+0000","last_clean":"2023-12-19T18:23:08.195570+0000","last_became_active":"2023-12-19T18:23:06.501120+0000","last_became_peered":"2023-12-19T18:23:06.501120+0000","last_unstale":"2023-12-19T18:23:08.195570+0000","last_undegraded":"2023-12-19T18:23:08.195570+0000","last_fullsized":"2023-12-19T18:23:08.195570+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:00.294833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.477083+0000","last_change":"2023-12-19T18:23:06.498953+0000","last_active":"2023-12-19T18:23:07.477083+0000","last_peered":"2023-12-19T18:23:07.477083+0000","last_clean":"2023-12-19T18:23:07.477083+0000","last_became_active":"2023-12-19T18:23:06.498730+0000","last_became_peered":"2023-12-19T18:23:06.498730+0000","last_unstale":"2023-12-19T18:23:07.477083+0000","last_undegraded":"2023-12-19T18:23:07.477083+0000","last_fullsized":"2023-12-19T18:23:07.477083+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:14.279141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.919697+0000","last_change":"2023-12-19T18:23:06.492231+0000","last_active":"2023-12-19T18:23:07.919697+0000","last_peered":"2023-12-19T18:23:07.919697+0000","last_clean":"2023-12-19T18:23:07.919697+0000","last_became_active":"2023-12-19T18:23:06.490824+0000","last_became_peered":"2023-12-19T18:23:06.490824+0000","last_unstale":"2023-12-19T18:23:07.919697+0000","last_undegraded":"2023-12-19T18:23:07.919697+0000","last_fullsized":"2023-12-19T18:23:07.919697+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:47:27.154269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.198324+0000","last_change":"2023-12-19T18:23:06.510155+0000","last_active":"2023-12-19T18:23:08.198324+0000","last_peered":"2023-12-19T18:23:08.198324+0000","last_clean":"2023-12-19T18:23:08.198324+0000","last_became_active":"2023-12-19T18:23:06.509735+0000","last_became_peered":"2023-12-19T18:23:06.509735+0000","last_unstale":"2023-12-19T18:23:08.198324+0000","last_undegraded":"2023-12-19T18:23:08.198324+0000","last_fullsized":"2023-12-19T18:23:08.198324+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:18:26.847665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197882+0000","last_change":"2023-12-19T18:23:06.496927+0000","last_active":"2023-12-19T18:23:08.197882+0000","last_peered":"2023-12-19T18:23:08.197882+0000","last_clean":"2023-12-19T18:23:08.197882+0000","last_became_active":"2023-12-19T18:23:06.496418+0000","last_became_peered":"2023-12-19T18:23:06.496418+0000","last_unstale":"2023-12-19T18:23:08.197882+0000","last_undegraded":"2023-12-19T18:23:08.197882+0000","last_fullsized":"2023-12-19T18:23:08.197882+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:58.009794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195527+0000","last_change":"2023-12-19T18:23:06.478414+0000","last_active":"2023-12-19T18:23:08.195527+0000","last_peered":"2023-12-19T18:23:08.195527+0000","last_clean":"2023-12-19T18:23:08.195527+0000","last_became_active":"2023-12-19T18:23:06.478164+0000","last_became_peered":"2023-12-19T18:23:06.478164+0000","last_unstale":"2023-12-19T18:23:08.195527+0000","last_undegraded":"2023-12-19T18:23:08.195527+0000","last_fullsized":"2023-12-19T18:23:08.195527+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:00.737229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197219+0000","last_change":"2023-12-19T18:23:06.507509+0000","last_active":"2023-12-19T18:23:08.197219+0000","last_peered":"2023-12-19T18:23:08.197219+0000","last_clean":"2023-12-19T18:23:08.197219+0000","last_became_active":"2023-12-19T18:23:06.507264+0000","last_became_peered":"2023-12-19T18:23:06.507264+0000","last_unstale":"2023-12-19T18:23:08.197219+0000","last_undegraded":"2023-12-19T18:23:08.197219+0000","last_fullsized":"2023-12-19T18:23:08.197219+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:32.338248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195360+0000","last_change":"2023-12-19T18:23:06.498747+0000","last_active":"2023-12-19T18:23:08.195360+0000","last_peered":"2023-12-19T18:23:08.195360+0000","last_clean":"2023-12-19T18:23:08.195360+0000","last_became_active":"2023-12-19T18:23:06.498505+0000","last_became_peered":"2023-12-19T18:23:06.498505+0000","last_unstale":"2023-12-19T18:23:08.195360+0000","last_undegraded":"2023-12-19T18:23:08.195360+0000","last_fullsized":"2023-12-19T18:23:08.195360+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:33.851089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195785+0000","last_change":"2023-12-19T18:23:06.492840+0000","last_active":"2023-12-19T18:23:08.195785+0000","last_peered":"2023-12-19T18:23:08.195785+0000","last_clean":"2023-12-19T18:23:08.195785+0000","last_became_active":"2023-12-19T18:23:06.492576+0000","last_became_peered":"2023-12-19T18:23:06.492576+0000","last_unstale":"2023-12-19T18:23:08.195785+0000","last_undegraded":"2023-12-19T18:23:08.195785+0000","last_fullsized":"2023-12-19T18:23:08.195785+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:57:18.772842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197417+0000","last_change":"2023-12-19T18:23:06.496011+0000","last_active":"2023-12-19T18:23:08.197417+0000","last_peered":"2023-12-19T18:23:08.197417+0000","last_clean":"2023-12-19T18:23:08.197417+0000","last_became_active":"2023-12-19T18:23:06.495763+0000","last_became_peered":"2023-12-19T18:23:06.495763+0000","last_unstale":"2023-12-19T18:23:08.197417+0000","last_undegraded":"2023-12-19T18:23:08.197417+0000","last_fullsized":"2023-12-19T18:23:08.197417+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:05:43.826190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475963+0000","last_change":"2023-12-19T18:23:06.496804+0000","last_active":"2023-12-19T18:23:07.475963+0000","last_peered":"2023-12-19T18:23:07.475963+0000","last_clean":"2023-12-19T18:23:07.475963+0000","last_became_active":"2023-12-19T18:23:06.496355+0000","last_became_peered":"2023-12-19T18:23:06.496355+0000","last_unstale":"2023-12-19T18:23:07.475963+0000","last_undegraded":"2023-12-19T18:23:07.475963+0000","last_fullsized":"2023-12-19T18:23:07.475963+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:58.389092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.919714+0000","last_change":"2023-12-19T18:23:06.484607+0000","last_active":"2023-12-19T18:23:07.919714+0000","last_peered":"2023-12-19T18:23:07.919714+0000","last_clean":"2023-12-19T18:23:07.919714+0000","last_became_active":"2023-12-19T18:23:06.484265+0000","last_became_peered":"2023-12-19T18:23:06.484265+0000","last_unstale":"2023-12-19T18:23:07.919714+0000","last_undegraded":"2023-12-19T18:23:07.919714+0000","last_fullsized":"2023-12-19T18:23:07.919714+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:58.009794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479791+0000","last_change":"2023-12-19T18:23:06.501397+0000","last_active":"2023-12-19T18:23:07.479791+0000","last_peered":"2023-12-19T18:23:07.479791+0000","last_clean":"2023-12-19T18:23:07.479791+0000","last_became_active":"2023-12-19T18:23:06.500482+0000","last_became_peered":"2023-12-19T18:23:06.500482+0000","last_unstale":"2023-12-19T18:23:07.479791+0000","last_undegraded":"2023-12-19T18:23:07.479791+0000","last_fullsized":"2023-12-19T18:23:07.479791+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:23:03.423398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196947+0000","last_change":"2023-12-19T18:23:06.492341+0000","last_active":"2023-12-19T18:23:08.196947+0000","last_peered":"2023-12-19T18:23:08.196947+0000","last_clean":"2023-12-19T18:23:08.196947+0000","last_became_active":"2023-12-19T18:23:06.492052+0000","last_became_peered":"2023-12-19T18:23:06.492052+0000","last_unstale":"2023-12-19T18:23:08.196947+0000","last_undegraded":"2023-12-19T18:23:08.196947+0000","last_fullsized":"2023-12-19T18:23:08.196947+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:18:32.166889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475986+0000","last_change":"2023-12-19T18:23:06.479557+0000","last_active":"2023-12-19T18:23:07.475986+0000","last_peered":"2023-12-19T18:23:07.475986+0000","last_clean":"2023-12-19T18:23:07.475986+0000","last_became_active":"2023-12-19T18:23:06.478928+0000","last_became_peered":"2023-12-19T18:23:06.478928+0000","last_unstale":"2023-12-19T18:23:07.475986+0000","last_undegraded":"2023-12-19T18:23:07.475986+0000","last_fullsized":"2023-12-19T18:23:07.475986+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:56:50.440480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.478906+0000","last_change":"2023-12-19T18:23:06.502465+0000","last_active":"2023-12-19T18:23:07.478906+0000","last_peered":"2023-12-19T18:23:07.478906+0000","last_clean":"2023-12-19T18:23:07.478906+0000","last_became_active":"2023-12-19T18:23:06.502151+0000","last_became_peered":"2023-12-19T18:23:06.502151+0000","last_unstale":"2023-12-19T18:23:07.478906+0000","last_undegraded":"2023-12-19T18:23:07.478906+0000","last_fullsized":"2023-12-19T18:23:07.478906+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:56.690910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475428+0000","last_change":"2023-12-19T18:23:06.496410+0000","last_active":"2023-12-19T18:23:07.475428+0000","last_peered":"2023-12-19T18:23:07.475428+0000","last_clean":"2023-12-19T18:23:07.475428+0000","last_became_active":"2023-12-19T18:23:06.496079+0000","last_became_peered":"2023-12-19T18:23:06.496079+0000","last_unstale":"2023-12-19T18:23:07.475428+0000","last_undegraded":"2023-12-19T18:23:07.475428+0000","last_fullsized":"2023-12-19T18:23:07.475428+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:39.745480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.919937+0000","last_change":"2023-12-19T18:23:06.493740+0000","last_active":"2023-12-19T18:23:07.919937+0000","last_peered":"2023-12-19T18:23:07.919937+0000","last_clean":"2023-12-19T18:23:07.919937+0000","last_became_active":"2023-12-19T18:23:06.491814+0000","last_became_peered":"2023-12-19T18:23:06.491814+0000","last_unstale":"2023-12-19T18:23:07.919937+0000","last_undegraded":"2023-12-19T18:23:07.919937+0000","last_fullsized":"2023-12-19T18:23:07.919937+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:03:11.918746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197262+0000","last_change":"2023-12-19T18:23:06.494995+0000","last_active":"2023-12-19T18:23:08.197262+0000","last_peered":"2023-12-19T18:23:08.197262+0000","last_clean":"2023-12-19T18:23:08.197262+0000","last_became_active":"2023-12-19T18:23:06.494722+0000","last_became_peered":"2023-12-19T18:23:06.494722+0000","last_unstale":"2023-12-19T18:23:08.197262+0000","last_undegraded":"2023-12-19T18:23:08.197262+0000","last_fullsized":"2023-12-19T18:23:08.197262+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:26.443334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196674+0000","last_change":"2023-12-19T18:23:06.506386+0000","last_active":"2023-12-19T18:23:08.196674+0000","last_peered":"2023-12-19T18:23:08.196674+0000","last_clean":"2023-12-19T18:23:08.196674+0000","last_became_active":"2023-12-19T18:23:06.505131+0000","last_became_peered":"2023-12-19T18:23:06.505131+0000","last_unstale":"2023-12-19T18:23:08.196674+0000","last_undegraded":"2023-12-19T18:23:08.196674+0000","last_fullsized":"2023-12-19T18:23:08.196674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:21.096114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196004+0000","last_change":"2023-12-19T18:23:06.496827+0000","last_active":"2023-12-19T18:23:08.196004+0000","last_peered":"2023-12-19T18:23:08.196004+0000","last_clean":"2023-12-19T18:23:08.196004+0000","last_became_active":"2023-12-19T18:23:06.496561+0000","last_became_peered":"2023-12-19T18:23:06.496561+0000","last_unstale":"2023-12-19T18:23:08.196004+0000","last_undegraded":"2023-12-19T18:23:08.196004+0000","last_fullsized":"2023-12-19T18:23:08.196004+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:08.187832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195975+0000","last_change":"2023-12-19T18:23:06.506914+0000","last_active":"2023-12-19T18:23:08.195975+0000","last_peered":"2023-12-19T18:23:08.195975+0000","last_clean":"2023-12-19T18:23:08.195975+0000","last_became_active":"2023-12-19T18:23:06.505804+0000","last_became_peered":"2023-12-19T18:23:06.505804+0000","last_unstale":"2023-12-19T18:23:08.195975+0000","last_undegraded":"2023-12-19T18:23:08.195975+0000","last_fullsized":"2023-12-19T18:23:08.195975+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:14.279141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"151'4","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920370+0000","last_change":"2023-12-19T18:23:05.995846+0000","last_active":"2023-12-19T18:23:07.920370+0000","last_peered":"2023-12-19T18:23:07.920370+0000","last_clean":"2023-12-19T18:23:07.920370+0000","last_became_active":"2023-12-19T18:23:05.995558+0000","last_became_peered":"2023-12-19T18:23:05.995558+0000","last_unstale":"2023-12-19T18:23:07.920370+0000","last_undegraded":"2023-12-19T18:23:07.920370+0000","last_fullsized":"2023-12-19T18:23:07.920370+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:46.674483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.919962+0000","last_change":"2023-12-19T18:23:06.484462+0000","last_active":"2023-12-19T18:23:07.919962+0000","last_peered":"2023-12-19T18:23:07.919962+0000","last_clean":"2023-12-19T18:23:07.919962+0000","last_became_active":"2023-12-19T18:23:06.483907+0000","last_became_peered":"2023-12-19T18:23:06.483907+0000","last_unstale":"2023-12-19T18:23:07.919962+0000","last_undegraded":"2023-12-19T18:23:07.919962+0000","last_fullsized":"2023-12-19T18:23:07.919962+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:08.187832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"151'1","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.477145+0000","last_change":"2023-12-19T18:23:05.998193+0000","last_active":"2023-12-19T18:23:07.477145+0000","last_peered":"2023-12-19T18:23:07.477145+0000","last_clean":"2023-12-19T18:23:07.477145+0000","last_became_active":"2023-12-19T18:23:05.997924+0000","last_became_peered":"2023-12-19T18:23:05.997924+0000","last_unstale":"2023-12-19T18:23:07.477145+0000","last_undegraded":"2023-12-19T18:23:07.477145+0000","last_fullsized":"2023-12-19T18:23:07.477145+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:24:44.640215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.476224+0000","last_change":"2023-12-19T18:23:06.508750+0000","last_active":"2023-12-19T18:23:07.476224+0000","last_peered":"2023-12-19T18:23:07.476224+0000","last_clean":"2023-12-19T18:23:07.476224+0000","last_became_active":"2023-12-19T18:23:06.508256+0000","last_became_peered":"2023-12-19T18:23:06.508256+0000","last_unstale":"2023-12-19T18:23:07.476224+0000","last_undegraded":"2023-12-19T18:23:07.476224+0000","last_fullsized":"2023-12-19T18:23:07.476224+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:10:57.367499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"151'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479328+0000","last_change":"2023-12-19T18:23:06.006602+0000","last_active":"2023-12-19T18:23:07.479328+0000","last_peered":"2023-12-19T18:23:07.479328+0000","last_clean":"2023-12-19T18:23:07.479328+0000","last_became_active":"2023-12-19T18:23:06.006377+0000","last_became_peered":"2023-12-19T18:23:06.006377+0000","last_unstale":"2023-12-19T18:23:07.479328+0000","last_undegraded":"2023-12-19T18:23:07.479328+0000","last_fullsized":"2023-12-19T18:23:07.479328+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:59.278694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196096+0000","last_change":"2023-12-19T18:23:06.503903+0000","last_active":"2023-12-19T18:23:08.196096+0000","last_peered":"2023-12-19T18:23:08.196096+0000","last_clean":"2023-12-19T18:23:08.196096+0000","last_became_active":"2023-12-19T18:23:06.503680+0000","last_became_peered":"2023-12-19T18:23:06.503680+0000","last_unstale":"2023-12-19T18:23:08.196096+0000","last_undegraded":"2023-12-19T18:23:08.196096+0000","last_fullsized":"2023-12-19T18:23:08.196096+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:48.370195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"151'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.478888+0000","last_change":"2023-12-19T18:23:06.005582+0000","last_active":"2023-12-19T18:23:07.478888+0000","last_peered":"2023-12-19T18:23:07.478888+0000","last_clean":"2023-12-19T18:23:07.478888+0000","last_became_active":"2023-12-19T18:23:06.004916+0000","last_became_peered":"2023-12-19T18:23:06.004916+0000","last_unstale":"2023-12-19T18:23:07.478888+0000","last_undegraded":"2023-12-19T18:23:07.478888+0000","last_fullsized":"2023-12-19T18:23:07.478888+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:49.313792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.478311+0000","last_change":"2023-12-19T18:23:06.504917+0000","last_active":"2023-12-19T18:23:07.478311+0000","last_peered":"2023-12-19T18:23:07.478311+0000","last_clean":"2023-12-19T18:23:07.478311+0000","last_became_active":"2023-12-19T18:23:06.502637+0000","last_became_peered":"2023-12-19T18:23:06.502637+0000","last_unstale":"2023-12-19T18:23:07.478311+0000","last_undegraded":"2023-12-19T18:23:07.478311+0000","last_fullsized":"2023-12-19T18:23:07.478311+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:06:03.251131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"151'1","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196308+0000","last_change":"2023-12-19T18:23:05.999686+0000","last_active":"2023-12-19T18:23:08.196308+0000","last_peered":"2023-12-19T18:23:08.196308+0000","last_clean":"2023-12-19T18:23:08.196308+0000","last_became_active":"2023-12-19T18:23:05.997802+0000","last_became_peered":"2023-12-19T18:23:05.997802+0000","last_unstale":"2023-12-19T18:23:08.196308+0000","last_undegraded":"2023-12-19T18:23:08.196308+0000","last_fullsized":"2023-12-19T18:23:08.196308+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:56:18.105452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196725+0000","last_change":"2023-12-19T18:23:06.498015+0000","last_active":"2023-12-19T18:23:08.196725+0000","last_peered":"2023-12-19T18:23:08.196725+0000","last_clean":"2023-12-19T18:23:08.196725+0000","last_became_active":"2023-12-19T18:23:06.497786+0000","last_became_peered":"2023-12-19T18:23:06.497786+0000","last_unstale":"2023-12-19T18:23:08.196725+0000","last_undegraded":"2023-12-19T18:23:08.196725+0000","last_fullsized":"2023-12-19T18:23:08.196725+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:30:36.322169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"151'1","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197084+0000","last_change":"2023-12-19T18:23:06.006189+0000","last_active":"2023-12-19T18:23:08.197084+0000","last_peered":"2023-12-19T18:23:08.197084+0000","last_clean":"2023-12-19T18:23:08.197084+0000","last_became_active":"2023-12-19T18:23:06.005474+0000","last_became_peered":"2023-12-19T18:23:06.005474+0000","last_unstale":"2023-12-19T18:23:08.197084+0000","last_undegraded":"2023-12-19T18:23:08.197084+0000","last_fullsized":"2023-12-19T18:23:08.197084+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:15.657126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196222+0000","last_change":"2023-12-19T18:23:06.508377+0000","last_active":"2023-12-19T18:23:08.196222+0000","last_peered":"2023-12-19T18:23:08.196222+0000","last_clean":"2023-12-19T18:23:08.196222+0000","last_became_active":"2023-12-19T18:23:06.506498+0000","last_became_peered":"2023-12-19T18:23:06.506498+0000","last_unstale":"2023-12-19T18:23:08.196222+0000","last_undegraded":"2023-12-19T18:23:08.196222+0000","last_fullsized":"2023-12-19T18:23:08.196222+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:24:45.656354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196426+0000","last_change":"2023-12-19T18:23:06.002642+0000","last_active":"2023-12-19T18:23:08.196426+0000","last_peered":"2023-12-19T18:23:08.196426+0000","last_clean":"2023-12-19T18:23:08.196426+0000","last_became_active":"2023-12-19T18:23:06.002468+0000","last_became_peered":"2023-12-19T18:23:06.002468+0000","last_unstale":"2023-12-19T18:23:08.196426+0000","last_undegraded":"2023-12-19T18:23:08.196426+0000","last_fullsized":"2023-12-19T18:23:08.196426+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:14:02.961282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195982+0000","last_change":"2023-12-19T18:23:06.489901+0000","last_active":"2023-12-19T18:23:08.195982+0000","last_peered":"2023-12-19T18:23:08.195982+0000","last_clean":"2023-12-19T18:23:08.195982+0000","last_became_active":"2023-12-19T18:23:06.489689+0000","last_became_peered":"2023-12-19T18:23:06.489689+0000","last_unstale":"2023-12-19T18:23:08.195982+0000","last_undegraded":"2023-12-19T18:23:08.195982+0000","last_fullsized":"2023-12-19T18:23:08.195982+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:03:14.058758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196875+0000","last_change":"2023-12-19T18:23:06.007538+0000","last_active":"2023-12-19T18:23:08.196875+0000","last_peered":"2023-12-19T18:23:08.196875+0000","last_clean":"2023-12-19T18:23:08.196875+0000","last_became_active":"2023-12-19T18:23:06.005765+0000","last_became_peered":"2023-12-19T18:23:06.005765+0000","last_unstale":"2023-12-19T18:23:08.196875+0000","last_undegraded":"2023-12-19T18:23:08.196875+0000","last_fullsized":"2023-12-19T18:23:08.196875+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:34:36.645513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.477469+0000","last_change":"2023-12-19T18:23:06.499805+0000","last_active":"2023-12-19T18:23:07.477469+0000","last_peered":"2023-12-19T18:23:07.477469+0000","last_clean":"2023-12-19T18:23:07.477469+0000","last_became_active":"2023-12-19T18:23:06.499641+0000","last_became_peered":"2023-12-19T18:23:06.499641+0000","last_unstale":"2023-12-19T18:23:07.477469+0000","last_undegraded":"2023-12-19T18:23:07.477469+0000","last_fullsized":"2023-12-19T18:23:07.477469+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:21.096114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479105+0000","last_change":"2023-12-19T18:23:06.005442+0000","last_active":"2023-12-19T18:23:07.479105+0000","last_peered":"2023-12-19T18:23:07.479105+0000","last_clean":"2023-12-19T18:23:07.479105+0000","last_became_active":"2023-12-19T18:23:06.004430+0000","last_became_peered":"2023-12-19T18:23:06.004430+0000","last_unstale":"2023-12-19T18:23:07.479105+0000","last_undegraded":"2023-12-19T18:23:07.479105+0000","last_fullsized":"2023-12-19T18:23:07.479105+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:15.657126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.477077+0000","last_change":"2023-12-19T18:23:06.495638+0000","last_active":"2023-12-19T18:23:07.477077+0000","last_peered":"2023-12-19T18:23:07.477077+0000","last_clean":"2023-12-19T18:23:07.477077+0000","last_became_active":"2023-12-19T18:23:06.495398+0000","last_became_peered":"2023-12-19T18:23:06.495398+0000","last_unstale":"2023-12-19T18:23:07.477077+0000","last_undegraded":"2023-12-19T18:23:07.477077+0000","last_fullsized":"2023-12-19T18:23:07.477077+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:03.827467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196417+0000","last_change":"2023-12-19T18:23:06.006199+0000","last_active":"2023-12-19T18:23:08.196417+0000","last_peered":"2023-12-19T18:23:08.196417+0000","last_clean":"2023-12-19T18:23:08.196417+0000","last_became_active":"2023-12-19T18:23:06.005495+0000","last_became_peered":"2023-12-19T18:23:06.005495+0000","last_unstale":"2023-12-19T18:23:08.196417+0000","last_undegraded":"2023-12-19T18:23:08.196417+0000","last_fullsized":"2023-12-19T18:23:08.196417+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:32.834949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475416+0000","last_change":"2023-12-19T18:23:06.485604+0000","last_active":"2023-12-19T18:23:07.475416+0000","last_peered":"2023-12-19T18:23:07.475416+0000","last_clean":"2023-12-19T18:23:07.475416+0000","last_became_active":"2023-12-19T18:23:06.484970+0000","last_became_peered":"2023-12-19T18:23:06.484970+0000","last_unstale":"2023-12-19T18:23:07.475416+0000","last_undegraded":"2023-12-19T18:23:07.475416+0000","last_fullsized":"2023-12-19T18:23:07.475416+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:20:31.373582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"151'3","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195729+0000","last_change":"2023-12-19T18:23:06.008799+0000","last_active":"2023-12-19T18:23:08.195729+0000","last_peered":"2023-12-19T18:23:08.195729+0000","last_clean":"2023-12-19T18:23:08.195729+0000","last_became_active":"2023-12-19T18:23:06.008565+0000","last_became_peered":"2023-12-19T18:23:06.008565+0000","last_unstale":"2023-12-19T18:23:08.195729+0000","last_undegraded":"2023-12-19T18:23:08.195729+0000","last_fullsized":"2023-12-19T18:23:08.195729+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:02.807770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197448+0000","last_change":"2023-12-19T18:23:06.490824+0000","last_active":"2023-12-19T18:23:08.197448+0000","last_peered":"2023-12-19T18:23:08.197448+0000","last_clean":"2023-12-19T18:23:08.197448+0000","last_became_active":"2023-12-19T18:23:06.490607+0000","last_became_peered":"2023-12-19T18:23:06.490607+0000","last_unstale":"2023-12-19T18:23:08.197448+0000","last_undegraded":"2023-12-19T18:23:08.197448+0000","last_fullsized":"2023-12-19T18:23:08.197448+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:54:24.376646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"151'6","reported_seq":17,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479992+0000","last_change":"2023-12-19T18:23:06.003698+0000","last_active":"2023-12-19T18:23:07.479992+0000","last_peered":"2023-12-19T18:23:07.479992+0000","last_clean":"2023-12-19T18:23:07.479992+0000","last_became_active":"2023-12-19T18:23:06.003232+0000","last_became_peered":"2023-12-19T18:23:06.003232+0000","last_unstale":"2023-12-19T18:23:07.479992+0000","last_undegraded":"2023-12-19T18:23:07.479992+0000","last_fullsized":"2023-12-19T18:23:07.479992+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:01.166002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197490+0000","last_change":"2023-12-19T18:23:06.496783+0000","last_active":"2023-12-19T18:23:08.197490+0000","last_peered":"2023-12-19T18:23:08.197490+0000","last_clean":"2023-12-19T18:23:08.197490+0000","last_became_active":"2023-12-19T18:23:06.496089+0000","last_became_peered":"2023-12-19T18:23:06.496089+0000","last_unstale":"2023-12-19T18:23:08.197490+0000","last_undegraded":"2023-12-19T18:23:08.197490+0000","last_fullsized":"2023-12-19T18:23:08.197490+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:02.182142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"151'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197102+0000","last_change":"2023-12-19T18:23:06.007242+0000","last_active":"2023-12-19T18:23:08.197102+0000","last_peered":"2023-12-19T18:23:08.197102+0000","last_clean":"2023-12-19T18:23:08.197102+0000","last_became_active":"2023-12-19T18:23:06.005442+0000","last_became_peered":"2023-12-19T18:23:06.005442+0000","last_unstale":"2023-12-19T18:23:08.197102+0000","last_undegraded":"2023-12-19T18:23:08.197102+0000","last_fullsized":"2023-12-19T18:23:08.197102+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:47.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479078+0000","last_change":"2023-12-19T18:23:06.487983+0000","last_active":"2023-12-19T18:23:07.479078+0000","last_peered":"2023-12-19T18:23:07.479078+0000","last_clean":"2023-12-19T18:23:07.479078+0000","last_became_active":"2023-12-19T18:23:06.487122+0000","last_became_peered":"2023-12-19T18:23:06.487122+0000","last_unstale":"2023-12-19T18:23:07.479078+0000","last_undegraded":"2023-12-19T18:23:07.479078+0000","last_fullsized":"2023-12-19T18:23:07.479078+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:03.827467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196589+0000","last_change":"2023-12-19T18:23:05.990252+0000","last_active":"2023-12-19T18:23:08.196589+0000","last_peered":"2023-12-19T18:23:08.196589+0000","last_clean":"2023-12-19T18:23:08.196589+0000","last_became_active":"2023-12-19T18:23:05.989928+0000","last_became_peered":"2023-12-19T18:23:05.989928+0000","last_unstale":"2023-12-19T18:23:08.196589+0000","last_undegraded":"2023-12-19T18:23:08.196589+0000","last_fullsized":"2023-12-19T18:23:08.196589+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:02:23.605737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479129+0000","last_change":"2023-12-19T18:23:06.488679+0000","last_active":"2023-12-19T18:23:07.479129+0000","last_peered":"2023-12-19T18:23:07.479129+0000","last_clean":"2023-12-19T18:23:07.479129+0000","last_became_active":"2023-12-19T18:23:06.487705+0000","last_became_peered":"2023-12-19T18:23:06.487705+0000","last_unstale":"2023-12-19T18:23:07.479129+0000","last_undegraded":"2023-12-19T18:23:07.479129+0000","last_fullsized":"2023-12-19T18:23:07.479129+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:31.720170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"151'8","reported_seq":19,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197583+0000","last_change":"2023-12-19T18:23:06.006003+0000","last_active":"2023-12-19T18:23:08.197583+0000","last_peered":"2023-12-19T18:23:08.197583+0000","last_clean":"2023-12-19T18:23:08.197583+0000","last_became_active":"2023-12-19T18:23:06.005068+0000","last_became_peered":"2023-12-19T18:23:06.005068+0000","last_unstale":"2023-12-19T18:23:08.197583+0000","last_undegraded":"2023-12-19T18:23:08.197583+0000","last_fullsized":"2023-12-19T18:23:08.197583+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:20.334858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.198552+0000","last_change":"2023-12-19T18:23:06.507941+0000","last_active":"2023-12-19T18:23:08.198552+0000","last_peered":"2023-12-19T18:23:08.198552+0000","last_clean":"2023-12-19T18:23:08.198552+0000","last_became_active":"2023-12-19T18:23:06.506481+0000","last_became_peered":"2023-12-19T18:23:06.506481+0000","last_unstale":"2023-12-19T18:23:08.198552+0000","last_undegraded":"2023-12-19T18:23:08.198552+0000","last_fullsized":"2023-12-19T18:23:08.198552+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:01:54.556435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"151'1","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475769+0000","last_change":"2023-12-19T18:23:06.000376+0000","last_active":"2023-12-19T18:23:07.475769+0000","last_peered":"2023-12-19T18:23:07.475769+0000","last_clean":"2023-12-19T18:23:07.475769+0000","last_became_active":"2023-12-19T18:23:05.999715+0000","last_became_peered":"2023-12-19T18:23:05.999715+0000","last_unstale":"2023-12-19T18:23:07.475769+0000","last_undegraded":"2023-12-19T18:23:07.475769+0000","last_fullsized":"2023-12-19T18:23:07.475769+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:20.079974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196204+0000","last_change":"2023-12-19T18:23:06.497446+0000","last_active":"2023-12-19T18:23:08.196204+0000","last_peered":"2023-12-19T18:23:08.196204+0000","last_clean":"2023-12-19T18:23:08.196204+0000","last_became_active":"2023-12-19T18:23:06.497070+0000","last_became_peered":"2023-12-19T18:23:06.497070+0000","last_unstale":"2023-12-19T18:23:08.196204+0000","last_undegraded":"2023-12-19T18:23:08.196204+0000","last_fullsized":"2023-12-19T18:23:08.196204+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:41:32.488817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"151'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197225+0000","last_change":"2023-12-19T18:23:06.007751+0000","last_active":"2023-12-19T18:23:08.197225+0000","last_peered":"2023-12-19T18:23:08.197225+0000","last_clean":"2023-12-19T18:23:08.197225+0000","last_became_active":"2023-12-19T18:23:06.006705+0000","last_became_peered":"2023-12-19T18:23:06.006705+0000","last_unstale":"2023-12-19T18:23:08.197225+0000","last_undegraded":"2023-12-19T18:23:08.197225+0000","last_fullsized":"2023-12-19T18:23:08.197225+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:47.354055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196872+0000","last_change":"2023-12-19T18:23:06.500822+0000","last_active":"2023-12-19T18:23:08.196872+0000","last_peered":"2023-12-19T18:23:08.196872+0000","last_clean":"2023-12-19T18:23:08.196872+0000","last_became_active":"2023-12-19T18:23:06.500680+0000","last_became_peered":"2023-12-19T18:23:06.500680+0000","last_unstale":"2023-12-19T18:23:08.196872+0000","last_undegraded":"2023-12-19T18:23:08.196872+0000","last_fullsized":"2023-12-19T18:23:08.196872+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:07:19.550672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"151'1","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.921225+0000","last_change":"2023-12-19T18:23:05.999099+0000","last_active":"2023-12-19T18:23:07.921225+0000","last_peered":"2023-12-19T18:23:07.921225+0000","last_clean":"2023-12-19T18:23:07.921225+0000","last_became_active":"2023-12-19T18:23:05.998714+0000","last_became_peered":"2023-12-19T18:23:05.998714+0000","last_unstale":"2023-12-19T18:23:07.921225+0000","last_undegraded":"2023-12-19T18:23:07.921225+0000","last_fullsized":"2023-12-19T18:23:07.921225+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:59.721089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920569+0000","last_change":"2023-12-19T18:23:06.480947+0000","last_active":"2023-12-19T18:23:07.920569+0000","last_peered":"2023-12-19T18:23:07.920569+0000","last_clean":"2023-12-19T18:23:07.920569+0000","last_became_active":"2023-12-19T18:23:06.480673+0000","last_became_peered":"2023-12-19T18:23:06.480673+0000","last_unstale":"2023-12-19T18:23:07.920569+0000","last_undegraded":"2023-12-19T18:23:07.920569+0000","last_fullsized":"2023-12-19T18:23:07.920569+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:14.161699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920536+0000","last_change":"2023-12-19T18:23:05.991986+0000","last_active":"2023-12-19T18:23:07.920536+0000","last_peered":"2023-12-19T18:23:07.920536+0000","last_clean":"2023-12-19T18:23:07.920536+0000","last_became_active":"2023-12-19T18:23:05.991293+0000","last_became_peered":"2023-12-19T18:23:05.991293+0000","last_unstale":"2023-12-19T18:23:07.920536+0000","last_undegraded":"2023-12-19T18:23:07.920536+0000","last_fullsized":"2023-12-19T18:23:07.920536+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:57:17.756702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196059+0000","last_change":"2023-12-19T18:23:06.484800+0000","last_active":"2023-12-19T18:23:08.196059+0000","last_peered":"2023-12-19T18:23:08.196059+0000","last_clean":"2023-12-19T18:23:08.196059+0000","last_became_active":"2023-12-19T18:23:06.484571+0000","last_became_peered":"2023-12-19T18:23:06.484571+0000","last_unstale":"2023-12-19T18:23:08.196059+0000","last_undegraded":"2023-12-19T18:23:08.196059+0000","last_fullsized":"2023-12-19T18:23:08.196059+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:54.565682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"151'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196009+0000","last_change":"2023-12-19T18:23:06.005564+0000","last_active":"2023-12-19T18:23:08.196009+0000","last_peered":"2023-12-19T18:23:08.196009+0000","last_clean":"2023-12-19T18:23:08.196009+0000","last_became_active":"2023-12-19T18:23:06.005261+0000","last_became_peered":"2023-12-19T18:23:06.005261+0000","last_unstale":"2023-12-19T18:23:08.196009+0000","last_undegraded":"2023-12-19T18:23:08.196009+0000","last_fullsized":"2023-12-19T18:23:08.196009+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:53.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197350+0000","last_change":"2023-12-19T18:23:06.503882+0000","last_active":"2023-12-19T18:23:08.197350+0000","last_peered":"2023-12-19T18:23:08.197350+0000","last_clean":"2023-12-19T18:23:08.197350+0000","last_became_active":"2023-12-19T18:23:06.502559+0000","last_became_peered":"2023-12-19T18:23:06.502559+0000","last_unstale":"2023-12-19T18:23:08.197350+0000","last_undegraded":"2023-12-19T18:23:08.197350+0000","last_fullsized":"2023-12-19T18:23:08.197350+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:03.827467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196458+0000","last_change":"2023-12-19T18:23:06.004039+0000","last_active":"2023-12-19T18:23:08.196458+0000","last_peered":"2023-12-19T18:23:08.196458+0000","last_clean":"2023-12-19T18:23:08.196458+0000","last_became_active":"2023-12-19T18:23:06.003485+0000","last_became_peered":"2023-12-19T18:23:06.003485+0000","last_unstale":"2023-12-19T18:23:08.196458+0000","last_undegraded":"2023-12-19T18:23:08.196458+0000","last_fullsized":"2023-12-19T18:23:08.196458+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:56.993654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920901+0000","last_change":"2023-12-19T18:23:06.507662+0000","last_active":"2023-12-19T18:23:07.920901+0000","last_peered":"2023-12-19T18:23:07.920901+0000","last_clean":"2023-12-19T18:23:07.920901+0000","last_became_active":"2023-12-19T18:23:06.507232+0000","last_became_peered":"2023-12-19T18:23:06.507232+0000","last_unstale":"2023-12-19T18:23:07.920901+0000","last_undegraded":"2023-12-19T18:23:07.920901+0000","last_fullsized":"2023-12-19T18:23:07.920901+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:50.329932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479537+0000","last_change":"2023-12-19T18:23:06.008189+0000","last_active":"2023-12-19T18:23:07.479537+0000","last_peered":"2023-12-19T18:23:07.479537+0000","last_clean":"2023-12-19T18:23:07.479537+0000","last_became_active":"2023-12-19T18:23:06.007954+0000","last_became_peered":"2023-12-19T18:23:06.007954+0000","last_unstale":"2023-12-19T18:23:07.479537+0000","last_undegraded":"2023-12-19T18:23:07.479537+0000","last_fullsized":"2023-12-19T18:23:07.479537+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:07.171692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196469+0000","last_change":"2023-12-19T18:23:06.508644+0000","last_active":"2023-12-19T18:23:08.196469+0000","last_peered":"2023-12-19T18:23:08.196469+0000","last_clean":"2023-12-19T18:23:08.196469+0000","last_became_active":"2023-12-19T18:23:06.507865+0000","last_became_peered":"2023-12-19T18:23:06.507865+0000","last_unstale":"2023-12-19T18:23:08.196469+0000","last_undegraded":"2023-12-19T18:23:08.196469+0000","last_fullsized":"2023-12-19T18:23:08.196469+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:50.329932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479331+0000","last_change":"2023-12-19T18:23:05.992912+0000","last_active":"2023-12-19T18:23:07.479331+0000","last_peered":"2023-12-19T18:23:07.479331+0000","last_clean":"2023-12-19T18:23:07.479331+0000","last_became_active":"2023-12-19T18:23:05.991729+0000","last_became_peered":"2023-12-19T18:23:05.991729+0000","last_unstale":"2023-12-19T18:23:07.479331+0000","last_undegraded":"2023-12-19T18:23:07.479331+0000","last_fullsized":"2023-12-19T18:23:07.479331+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:56.993654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920216+0000","last_change":"2023-12-19T18:23:06.501125+0000","last_active":"2023-12-19T18:23:07.920216+0000","last_peered":"2023-12-19T18:23:07.920216+0000","last_clean":"2023-12-19T18:23:07.920216+0000","last_became_active":"2023-12-19T18:23:06.500985+0000","last_became_peered":"2023-12-19T18:23:06.500985+0000","last_unstale":"2023-12-19T18:23:07.920216+0000","last_undegraded":"2023-12-19T18:23:07.920216+0000","last_fullsized":"2023-12-19T18:23:07.920216+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:00.294833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"151'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479357+0000","last_change":"2023-12-19T18:23:06.006204+0000","last_active":"2023-12-19T18:23:07.479357+0000","last_peered":"2023-12-19T18:23:07.479357+0000","last_clean":"2023-12-19T18:23:07.479357+0000","last_became_active":"2023-12-19T18:23:06.005971+0000","last_became_peered":"2023-12-19T18:23:06.005971+0000","last_unstale":"2023-12-19T18:23:07.479357+0000","last_undegraded":"2023-12-19T18:23:07.479357+0000","last_fullsized":"2023-12-19T18:23:07.479357+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:20.334858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.476221+0000","last_change":"2023-12-19T18:23:06.496656+0000","last_active":"2023-12-19T18:23:07.476221+0000","last_peered":"2023-12-19T18:23:07.476221+0000","last_clean":"2023-12-19T18:23:07.476221+0000","last_became_active":"2023-12-19T18:23:06.495309+0000","last_became_peered":"2023-12-19T18:23:06.495309+0000","last_unstale":"2023-12-19T18:23:07.476221+0000","last_undegraded":"2023-12-19T18:23:07.476221+0000","last_fullsized":"2023-12-19T18:23:07.476221+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:23.170269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197647+0000","last_change":"2023-12-19T18:23:05.993480+0000","last_active":"2023-12-19T18:23:08.197647+0000","last_peered":"2023-12-19T18:23:08.197647+0000","last_clean":"2023-12-19T18:23:08.197647+0000","last_became_active":"2023-12-19T18:23:05.993067+0000","last_became_peered":"2023-12-19T18:23:05.993067+0000","last_unstale":"2023-12-19T18:23:08.197647+0000","last_undegraded":"2023-12-19T18:23:08.197647+0000","last_fullsized":"2023-12-19T18:23:08.197647+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:25:04.595553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.198775+0000","last_change":"2023-12-19T18:23:06.509957+0000","last_active":"2023-12-19T18:23:08.198775+0000","last_peered":"2023-12-19T18:23:08.198775+0000","last_clean":"2023-12-19T18:23:08.198775+0000","last_became_active":"2023-12-19T18:23:06.509444+0000","last_became_peered":"2023-12-19T18:23:06.509444+0000","last_unstale":"2023-12-19T18:23:08.198775+0000","last_undegraded":"2023-12-19T18:23:08.198775+0000","last_fullsized":"2023-12-19T18:23:08.198775+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:23:03.423398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.480110+0000","last_change":"2023-12-19T18:23:05.993729+0000","last_active":"2023-12-19T18:23:07.480110+0000","last_peered":"2023-12-19T18:23:07.480110+0000","last_clean":"2023-12-19T18:23:07.480110+0000","last_became_active":"2023-12-19T18:23:05.992551+0000","last_became_peered":"2023-12-19T18:23:05.992551+0000","last_unstale":"2023-12-19T18:23:07.480110+0000","last_undegraded":"2023-12-19T18:23:07.480110+0000","last_fullsized":"2023-12-19T18:23:07.480110+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:03:10.902606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920117+0000","last_change":"2023-12-19T18:23:06.492085+0000","last_active":"2023-12-19T18:23:07.920117+0000","last_peered":"2023-12-19T18:23:07.920117+0000","last_clean":"2023-12-19T18:23:07.920117+0000","last_became_active":"2023-12-19T18:23:06.490512+0000","last_became_peered":"2023-12-19T18:23:06.490512+0000","last_unstale":"2023-12-19T18:23:07.920117+0000","last_undegraded":"2023-12-19T18:23:07.920117+0000","last_fullsized":"2023-12-19T18:23:07.920117+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:02.182142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479733+0000","last_change":"2023-12-19T18:23:05.998603+0000","last_active":"2023-12-19T18:23:07.479733+0000","last_peered":"2023-12-19T18:23:07.479733+0000","last_clean":"2023-12-19T18:23:07.479733+0000","last_became_active":"2023-12-19T18:23:05.998276+0000","last_became_peered":"2023-12-19T18:23:05.998276+0000","last_unstale":"2023-12-19T18:23:07.479733+0000","last_undegraded":"2023-12-19T18:23:07.479733+0000","last_fullsized":"2023-12-19T18:23:07.479733+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:13.145559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195999+0000","last_change":"2023-12-19T18:23:06.492826+0000","last_active":"2023-12-19T18:23:08.195999+0000","last_peered":"2023-12-19T18:23:08.195999+0000","last_clean":"2023-12-19T18:23:08.195999+0000","last_became_active":"2023-12-19T18:23:06.492593+0000","last_became_peered":"2023-12-19T18:23:06.492593+0000","last_unstale":"2023-12-19T18:23:08.195999+0000","last_undegraded":"2023-12-19T18:23:08.195999+0000","last_fullsized":"2023-12-19T18:23:08.195999+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:47.690623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197644+0000","last_change":"2023-12-19T18:23:05.994816+0000","last_active":"2023-12-19T18:23:08.197644+0000","last_peered":"2023-12-19T18:23:08.197644+0000","last_clean":"2023-12-19T18:23:08.197644+0000","last_became_active":"2023-12-19T18:23:05.994458+0000","last_became_peered":"2023-12-19T18:23:05.994458+0000","last_unstale":"2023-12-19T18:23:08.197644+0000","last_undegraded":"2023-12-19T18:23:08.197644+0000","last_fullsized":"2023-12-19T18:23:08.197644+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:55:37.048248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.921080+0000","last_change":"2023-12-19T18:23:06.495667+0000","last_active":"2023-12-19T18:23:07.921080+0000","last_peered":"2023-12-19T18:23:07.921080+0000","last_clean":"2023-12-19T18:23:07.921080+0000","last_became_active":"2023-12-19T18:23:06.495424+0000","last_became_peered":"2023-12-19T18:23:06.495424+0000","last_unstale":"2023-12-19T18:23:07.921080+0000","last_undegraded":"2023-12-19T18:23:07.921080+0000","last_fullsized":"2023-12-19T18:23:07.921080+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:16.673265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920785+0000","last_change":"2023-12-19T18:23:06.005973+0000","last_active":"2023-12-19T18:23:07.920785+0000","last_peered":"2023-12-19T18:23:07.920785+0000","last_clean":"2023-12-19T18:23:07.920785+0000","last_became_active":"2023-12-19T18:23:06.005622+0000","last_became_peered":"2023-12-19T18:23:06.005622+0000","last_unstale":"2023-12-19T18:23:07.920785+0000","last_undegraded":"2023-12-19T18:23:07.920785+0000","last_fullsized":"2023-12-19T18:23:07.920785+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:55:20.887389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920275+0000","last_change":"2023-12-19T18:23:06.483661+0000","last_active":"2023-12-19T18:23:07.920275+0000","last_peered":"2023-12-19T18:23:07.920275+0000","last_clean":"2023-12-19T18:23:07.920275+0000","last_became_active":"2023-12-19T18:23:06.483421+0000","last_became_peered":"2023-12-19T18:23:06.483421+0000","last_unstale":"2023-12-19T18:23:07.920275+0000","last_undegraded":"2023-12-19T18:23:07.920275+0000","last_fullsized":"2023-12-19T18:23:07.920275+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:21.350998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197793+0000","last_change":"2023-12-19T18:23:06.005758+0000","last_active":"2023-12-19T18:23:08.197793+0000","last_peered":"2023-12-19T18:23:08.197793+0000","last_clean":"2023-12-19T18:23:08.197793+0000","last_became_active":"2023-12-19T18:23:06.004713+0000","last_became_peered":"2023-12-19T18:23:06.004713+0000","last_unstale":"2023-12-19T18:23:08.197793+0000","last_undegraded":"2023-12-19T18:23:08.197793+0000","last_fullsized":"2023-12-19T18:23:08.197793+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:01.166002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475695+0000","last_change":"2023-12-19T18:23:06.503953+0000","last_active":"2023-12-19T18:23:07.475695+0000","last_peered":"2023-12-19T18:23:07.475695+0000","last_clean":"2023-12-19T18:23:07.475695+0000","last_became_active":"2023-12-19T18:23:06.503708+0000","last_became_peered":"2023-12-19T18:23:06.503708+0000","last_unstale":"2023-12-19T18:23:07.475695+0000","last_undegraded":"2023-12-19T18:23:07.475695+0000","last_fullsized":"2023-12-19T18:23:07.475695+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:02:27.597298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.476019+0000","last_change":"2023-12-19T18:23:06.000092+0000","last_active":"2023-12-19T18:23:07.476019+0000","last_peered":"2023-12-19T18:23:07.476019+0000","last_clean":"2023-12-19T18:23:07.476019+0000","last_became_active":"2023-12-19T18:23:05.999231+0000","last_became_peered":"2023-12-19T18:23:05.999231+0000","last_unstale":"2023-12-19T18:23:07.476019+0000","last_undegraded":"2023-12-19T18:23:07.476019+0000","last_fullsized":"2023-12-19T18:23:07.476019+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:13.263001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.478694+0000","last_change":"2023-12-19T18:23:06.493560+0000","last_active":"2023-12-19T18:23:07.478694+0000","last_peered":"2023-12-19T18:23:07.478694+0000","last_clean":"2023-12-19T18:23:07.478694+0000","last_became_active":"2023-12-19T18:23:06.493154+0000","last_became_peered":"2023-12-19T18:23:06.493154+0000","last_unstale":"2023-12-19T18:23:07.478694+0000","last_undegraded":"2023-12-19T18:23:07.478694+0000","last_fullsized":"2023-12-19T18:23:07.478694+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:14.279141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.477578+0000","last_change":"2023-12-19T18:23:05.998582+0000","last_active":"2023-12-19T18:23:07.477578+0000","last_peered":"2023-12-19T18:23:07.477578+0000","last_clean":"2023-12-19T18:23:07.477578+0000","last_became_active":"2023-12-19T18:23:05.998355+0000","last_became_peered":"2023-12-19T18:23:05.998355+0000","last_unstale":"2023-12-19T18:23:07.477578+0000","last_undegraded":"2023-12-19T18:23:07.477578+0000","last_fullsized":"2023-12-19T18:23:07.477578+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:24:05.315745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":283,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-12-19T18:23:06.474964+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:23:06.474964+0000","last_peered":"2023-12-19T18:23:06.474964+0000","last_clean":"2023-12-19T18:23:06.474964+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:23:06.474964+0000","last_undegraded":"2023-12-19T18:23:06.474964+0000","last_fullsized":"2023-12-19T18:23:06.474964+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":834,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194478,"statfs":{"total":200000000,"available":199145922,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.23,"15min":1.153},"min":{"1min":0.68000000000000005,"5min":0.57899999999999996,"15min":0.45900000000000002},"max":{"1min":1.587,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.286,"15min":1.196},"min":{"1min":0.76000000000000001,"5min":0.73499999999999999,"15min":0.55800000000000005},"max":{"1min":1.631,"5min":1.7330000000000001,"15min":1.79},"last":1.115}]},{"osd":1,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.238,"15min":1.2270000000000001},"min":{"1min":0.70899999999999996,"5min":0.63600000000000001,"15min":0.54400000000000004},"max":{"1min":1.6259999999999999,"5min":5.9029999999999996,"15min":10.926},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.2669999999999999,"15min":1.2450000000000001},"min":{"1min":0.85599999999999998,"5min":0.71899999999999997,"15min":0.67900000000000005},"max":{"1min":1.7190000000000001,"5min":1.9039999999999999,"15min":10.904999999999999},"last":1.349}]},{"osd":2,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.139,"15min":1.0980000000000001},"min":{"1min":0.66000000000000003,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5620000000000001,"5min":3.1960000000000002,"15min":3.1960000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.109,"5min":1.2430000000000001,"15min":1.1899999999999999},"min":{"1min":0.754,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4490000000000001,"5min":6.008,"15min":6.008},"last":1.1519999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.282,"15min":1.1679999999999999},"min":{"1min":0.84499999999999997,"5min":0.66900000000000004,"15min":0.61399999999999999},"max":{"1min":1.659,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":1.194},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.403,"15min":1.298},"min":{"1min":0.88200000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.5549999999999999,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":1.3300000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.2589999999999999,"15min":1.1319999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.47899999999999998},"max":{"1min":1.7549999999999999,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2969999999999999,"15min":1.24},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.57599999999999996},"max":{"1min":1.736,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.86099999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.3819999999999999,"15min":1.331},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.956,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.4370000000000001,"15min":1.3400000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.53500000000000003},"max":{"1min":1.8280000000000001,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":1.272}]},{"osd":6,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.5620000000000001,"15min":1.472},"min":{"1min":0.628,"5min":0.628,"15min":0.59899999999999998},"max":{"1min":1.742,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.4710000000000001,"15min":1.3819999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.8169999999999999,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":1.0720000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":832,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194480,"statfs":{"total":200000000,"available":199147610,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0900000000000001,"15min":1.0169999999999999},"min":{"1min":0.72799999999999998,"5min":0.68899999999999995,"15min":0.65800000000000003},"max":{"1min":1.333,"5min":1.6259999999999999,"15min":1.8129999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.006,"5min":1.1599999999999999,"15min":1.0740000000000001},"min":{"1min":0.64800000000000002,"5min":0.64300000000000002,"15min":0.55600000000000005},"max":{"1min":1.542,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":1.0429999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.1379999999999999,"15min":1.1299999999999999},"min":{"1min":0.77100000000000002,"5min":0.71399999999999997,"15min":0.68200000000000005},"max":{"1min":1.532,"5min":1.5609999999999999,"15min":4.5540000000000003},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.127,"5min":1.208,"15min":1.1679999999999999},"min":{"1min":0.79300000000000004,"5min":0.625,"15min":0.60299999999999998},"max":{"1min":1.583,"5min":1.885,"15min":1.885},"last":1.2729999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.2230000000000001,"15min":1.1539999999999999},"min":{"1min":0.871,"5min":0.74099999999999999,"15min":0.626},"max":{"1min":1.389,"5min":1.7330000000000001,"15min":1.821},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.2190000000000001,"15min":1.2250000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4950000000000001,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":0.90700000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2749999999999999,"15min":1.202},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5580000000000001,"5min":3.8820000000000001,"15min":3.8820000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.181,"15min":1.171},"min":{"1min":0.83799999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4510000000000001,"5min":1.6559999999999999,"15min":4.6189999999999998},"last":1.083}]},{"osd":4,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.2749999999999999,"15min":1.2529999999999999},"min":{"1min":0.90200000000000002,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.621,"5min":1.9670000000000001,"15min":3.0680000000000001},"last":1.476},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.415,"15min":1.353},"min":{"1min":0.95999999999999996,"5min":0.70799999999999996,"15min":0.57999999999999996},"max":{"1min":1.669,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.3999999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2609999999999999,"15min":1.1970000000000001},"min":{"1min":0.82999999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.734,"5min":2.2400000000000002,"15min":4.6680000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.419,"15min":1.361},"min":{"1min":1.145,"5min":0.626,"15min":0.626},"max":{"1min":1.5549999999999999,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.3600000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.425,"15min":1.3520000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.61899999999999999},"max":{"1min":1.702,"5min":3.9670000000000001,"15min":3.9670000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.409,"15min":1.365},"min":{"1min":1.0189999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.6319999999999999,"5min":2.3519999999999999,"15min":5.6639999999999997},"last":1.4339999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1409,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193903,"statfs":{"total":200000000,"available":198557150,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2130000000000001,"15min":1.1639999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.863},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.228,"5min":1.27,"15min":1.194},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.2669999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.222,"15min":1.079},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.65700000000000003},"max":{"1min":1.7929999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.179,"15min":1.0800000000000001},"min":{"1min":0.71899999999999997,"5min":0.71299999999999997,"15min":0.61299999999999999},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.843},"last":1.3380000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.155,"15min":1.093},"min":{"1min":0.745,"5min":0.74299999999999999,"15min":0.59499999999999997},"max":{"1min":1.5580000000000001,"5min":1.728,"15min":1.794},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.25,"15min":1.1599999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.65200000000000002},"max":{"1min":1.6779999999999999,"5min":1.768,"15min":2.036},"last":1.304}]},{"osd":3,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1930000000000001,"15min":1.105},"min":{"1min":0.78600000000000003,"5min":0.71999999999999997,"15min":0.68899999999999995},"max":{"1min":1.6319999999999999,"5min":1.72,"15min":1.7450000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.337,"15min":1.244},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.70399999999999996},"max":{"1min":1.9590000000000001,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.1830000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3049999999999999,"15min":1.198},"min":{"1min":0.93400000000000005,"5min":0.60899999999999999,"15min":0.46700000000000003},"max":{"1min":1.944,"5min":1.944,"15min":2.0329999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.218,"15min":1.143},"min":{"1min":0.97299999999999998,"5min":0.75900000000000001,"15min":0.57099999999999995},"max":{"1min":1.764,"5min":1.804,"15min":1.901},"last":1.228}]},{"osd":6,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.456,"15min":1.359},"min":{"1min":1.347,"5min":0.66600000000000004,"15min":0.54400000000000004},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.1589999999999998},"last":1.357},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3320000000000001,"15min":1.2569999999999999},"min":{"1min":0.93999999999999995,"5min":0.81200000000000006,"15min":0.49299999999999999},"max":{"1min":1.585,"5min":1.702,"15min":1.976},"last":1.45}]},{"osd":7,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3140000000000001,"15min":1.242},"min":{"1min":0.90000000000000002,"5min":0.68300000000000005,"15min":0.50900000000000001},"max":{"1min":1.863,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.399},{"interface":"front","average":{"1min":1.53,"5min":1.4790000000000001,"15min":1.4219999999999999},"min":{"1min":1.1559999999999999,"5min":0.86099999999999999,"15min":0.623},"max":{"1min":1.8620000000000001,"5min":2.117,"15min":2.117},"last":1.3700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":834,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194478,"statfs":{"total":200000000,"available":199145956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.998,"15min":1},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.45,"5min":1.532,"15min":1.579},"last":1.133},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0720000000000001,"15min":1.042},"min":{"1min":0.68999999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5680000000000001,"5min":1.7470000000000001,"15min":1.7909999999999999},"last":1.7330000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.153,"15min":1.1479999999999999},"min":{"1min":0.748,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.8160000000000001},"last":1.476},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.242,"15min":1.1850000000000001},"min":{"1min":0.69299999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.3720000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.1279999999999999,"15min":1.123},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.484,"5min":1.627,"15min":1.7010000000000001},"last":1.284},{"interface":"front","average":{"1min":1.143,"5min":1.1559999999999999,"15min":1.1100000000000001},"min":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.70499999999999996},"max":{"1min":1.633,"5min":1.675,"15min":1.675},"last":1.3400000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.29,"15min":1.264},"min":{"1min":0.877,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.351,"15min":1.3069999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.74399999999999999},"max":{"1min":1.5780000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.849}]},{"osd":5,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2869999999999999,"15min":1.212},"min":{"1min":0.82299999999999995,"5min":0.68600000000000005,"15min":0.44500000000000001},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8740000000000001},"last":1.157},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.395,"15min":1.353},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.504},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.93400000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.323,"15min":1.3029999999999999},"min":{"1min":1.008,"5min":0.71299999999999997,"15min":0.432},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.853},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.4590000000000001,"15min":1.3999999999999999},"min":{"1min":0.90800000000000003,"5min":0.73599999999999999,"15min":0.64500000000000002},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.7749999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.379,"15min":1.349},"min":{"1min":0.66000000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4419999999999999,"15min":1.4099999999999999},"min":{"1min":1.0880000000000001,"5min":0.60399999999999998,"15min":0.57399999999999995},"max":{"1min":1.8109999999999999,"5min":1.905,"15min":1.9330000000000001},"last":1.427}]}]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":832,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194480,"statfs":{"total":200000000,"available":199147610,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.95699999999999996},"min":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.35299999999999998},"max":{"1min":1.5469999999999999,"5min":3.3140000000000001,"15min":3.4350000000000001},"last":0.435},{"interface":"front","average":{"1min":1.032,"5min":1.01,"15min":0.95199999999999996},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7150000000000001,"5min":4.3490000000000002,"15min":4.3490000000000002},"last":0.65900000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.0209999999999999,"15min":1.05},"min":{"1min":0.66900000000000004,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":1.405,"5min":5.2510000000000003,"15min":5.2510000000000003},"last":0.624},{"interface":"front","average":{"1min":1.032,"5min":1.1519999999999999,"15min":1.111},"min":{"1min":0.53200000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.6319999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":0.69499999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.21,"15min":1.242},"min":{"1min":0.61299999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.417,"5min":2.5369999999999999,"15min":13.833},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3660000000000001,"15min":1.3120000000000001},"min":{"1min":0.79700000000000004,"5min":0.57999999999999996,"15min":0.53200000000000003},"max":{"1min":1.7509999999999999,"5min":5.4480000000000004,"15min":16.748999999999999},"last":0.93899999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3420000000000001,"15min":1.2410000000000001},"min":{"1min":0.87,"5min":0.77400000000000002,"15min":0.65600000000000003},"max":{"1min":1.8140000000000001,"5min":5.1790000000000003,"15min":5.1790000000000003},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.393,"5min":1.4099999999999999,"15min":1.407},"min":{"1min":1.0309999999999999,"5min":0.80800000000000005,"15min":0.73199999999999998},"max":{"1min":1.798,"5min":4.569,"15min":4.569},"last":0.97399999999999998}]},{"osd":5,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4490000000000001,"15min":1.444},"min":{"1min":0.95499999999999996,"5min":0.82399999999999995,"15min":0.76000000000000001},"max":{"1min":1.901,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.371,"15min":1.288},"min":{"1min":0.81899999999999995,"5min":0.80600000000000005,"15min":0.69999999999999996},"max":{"1min":1.9470000000000001,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.133}]},{"osd":6,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.446,"15min":1.47},"min":{"1min":0.97599999999999998,"5min":0.81799999999999995,"15min":0.81599999999999995},"max":{"1min":1.841,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.536,"15min":1.5129999999999999},"min":{"1min":0.92800000000000005,"5min":0.83199999999999996,"15min":0.80500000000000005},"max":{"1min":1.9970000000000001,"5min":5.3739999999999997,"15min":5.3739999999999997},"last":0.999}]},{"osd":7,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4750000000000001,"15min":1.395},"min":{"1min":0.89700000000000002,"5min":0.78500000000000003,"15min":0.73199999999999998},"max":{"1min":2.0720000000000001,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.22},{"interface":"front","average":{"1min":1.54,"5min":1.5640000000000001,"15min":1.5529999999999999},"min":{"1min":0.98099999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.0329999999999999,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":1.1020000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":833,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194479,"statfs":{"total":200000000,"available":199146972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.88200000000000001,"15min":0.997},"min":{"1min":0.47199999999999998,"5min":0.35299999999999998,"15min":0.34399999999999997},"max":{"1min":1.137,"5min":2.879,"15min":9.2550000000000008},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.0049999999999999,"15min":1.0289999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.361,"5min":2.387,"15min":4.7850000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.94899999999999995,"15min":1.137},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.46000000000000002},"max":{"1min":1.3340000000000001,"5min":2.4390000000000001,"15min":8.2720000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.093,"15min":1.151},"min":{"1min":0.67500000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.802,"5min":2.7200000000000002,"15min":5.0620000000000003},"last":0.745}]},{"osd":3,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.133,"15min":1.327},"min":{"1min":0.66000000000000003,"5min":0.56000000000000005,"15min":0.51900000000000002},"max":{"1min":1.5429999999999999,"5min":2.238,"15min":8.3109999999999999},"last":1.038},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.117,"15min":1.2949999999999999},"min":{"1min":0.60299999999999998,"5min":0.46300000000000002,"15min":0.41799999999999998},"max":{"1min":1.4490000000000001,"5min":2.323,"15min":8.0690000000000008},"last":0.61699999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.391,"15min":1.448},"min":{"1min":1.0760000000000001,"5min":0.83399999999999996,"15min":0.79300000000000004},"max":{"1min":1.7,"5min":2.6139999999999999,"15min":5.3929999999999998},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.454,"15min":1.5289999999999999},"min":{"1min":0.85399999999999998,"5min":0.82099999999999995,"15min":0.78300000000000003},"max":{"1min":2.1179999999999999,"5min":3.1070000000000002,"15min":7.3570000000000002},"last":1.1599999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.363,"15min":1.522},"min":{"1min":0.92800000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.7330000000000001,"5min":2.5579999999999998,"15min":8.1839999999999993},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.397,"15min":1.409},"min":{"1min":0.82299999999999995,"5min":0.81000000000000005,"15min":0.72199999999999998},"max":{"1min":1.9339999999999999,"5min":2.4940000000000002,"15min":5.165},"last":0.89200000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.411,"15min":1.4430000000000001},"min":{"1min":0.92500000000000004,"5min":0.752,"15min":0.72999999999999998},"max":{"1min":1.5680000000000001,"5min":3.0179999999999998,"15min":5.1059999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.4419999999999999,"15min":1.5449999999999999},"min":{"1min":0.98499999999999999,"5min":0.81200000000000006,"15min":0.77000000000000002},"max":{"1min":1.915,"5min":2.9580000000000002,"15min":8.1370000000000005},"last":1.246}]},{"osd":7,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.482,"15min":1.534},"min":{"1min":0.93500000000000005,"5min":0.79200000000000004,"15min":0.78200000000000003},"max":{"1min":1.714,"5min":3.2189999999999999,"15min":5.1449999999999996},"last":1.232},{"interface":"front","average":{"1min":1.377,"5min":1.5049999999999999,"15min":1.635},"min":{"1min":0.88500000000000001,"5min":0.878,"15min":0.86499999999999999},"max":{"1min":1.8600000000000001,"5min":4.8289999999999997,"15min":8.2300000000000004},"last":1.4970000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1402,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193909,"statfs":{"total":200000000,"available":198563542,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.85999999999999999,"15min":0.84799999999999998},"min":{"1min":0.54400000000000004,"5min":0.38,"15min":0.34699999999999998},"max":{"1min":1.4610000000000001,"5min":1.52,"15min":1.52},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.94699999999999995,"15min":0.95299999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.5820000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.434}]},{"osd":2,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.0529999999999999,"15min":1.052},"min":{"1min":0.56399999999999995,"5min":0.42299999999999999,"15min":0.41399999999999998},"max":{"1min":1.6120000000000001,"5min":2.4590000000000001,"15min":2.5739999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.016,"15min":0.93300000000000005},"min":{"1min":0.52200000000000002,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":1.1699999999999999,"5min":1.845,"15min":1.845},"last":0.626}]},{"osd":3,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.292,"15min":1.1699999999999999},"min":{"1min":0.59599999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.613,"5min":2.137,"15min":2.137},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.196,"15min":1.1839999999999999},"min":{"1min":0.63600000000000001,"5min":0.56100000000000005,"15min":0.44900000000000001},"max":{"1min":1.6200000000000001,"5min":2.1320000000000001,"15min":2.3959999999999999},"last":0.80900000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3440000000000001,"15min":1.2989999999999999},"min":{"1min":0.76400000000000001,"5min":0.72199999999999998,"15min":0.69099999999999995},"max":{"1min":1.696,"5min":2.444,"15min":2.444},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.3859999999999999,"15min":1.3089999999999999},"min":{"1min":0.95599999999999996,"5min":0.76900000000000002,"15min":0.71099999999999997},"max":{"1min":1.5369999999999999,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.3300000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.379,"15min":1.3049999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.506,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.476,"15min":1.4179999999999999},"min":{"1min":0.96399999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.76,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.1279999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.417,"15min":1.381},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.76900000000000002},"max":{"1min":1.714,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.355,"5min":1.5169999999999999,"15min":1.456},"min":{"1min":1.014,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.8200000000000001,"5min":2.895,"15min":2.895},"last":1.2829999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.4570000000000001,"15min":1.397},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.69999999999999996},"max":{"1min":1.6850000000000001,"5min":2.5670000000000002,"15min":9.4830000000000005},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.528,"15min":1.502},"min":{"1min":0.874,"5min":0.874,"15min":0.71899999999999997},"max":{"1min":1.7,"5min":2.456,"15min":9.4109999999999996},"last":1.3819999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":833,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194479,"statfs":{"total":200000000,"available":199146958,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.95599999999999996,"15min":0.89800000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1870000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":1.1040000000000001,"15min":1.0329999999999999},"min":{"1min":0.438,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3460000000000001,"5min":2.1040000000000001,"15min":2.4180000000000001},"last":1.3080000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.0189999999999999,"15min":0.95999999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4219999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":1.0009999999999999,"15min":0.94299999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.585,"5min":2.0950000000000002,"15min":2.7810000000000001},"last":1.1719999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.073,"15min":1.012},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.6719999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.44},{"interface":"front","average":{"1min":1,"5min":1.306,"15min":1.226},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.47499999999999998},"max":{"1min":1.603,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.83199999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.3140000000000001,"15min":1.2150000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.7569999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.409},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.3089999999999999,"15min":1.2809999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6160000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.90500000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3640000000000001,"15min":1.393},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.64500000000000002},"max":{"1min":1.806,"5min":2.3940000000000001,"15min":16.298999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.3600000000000001,"15min":1.286},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72199999999999998},"max":{"1min":1.696,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.6060000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.478,"15min":1.4019999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.776,"5min":2.2559999999999998,"15min":2.5910000000000002},"last":1.7},{"interface":"front","average":{"1min":1.2,"5min":1.4219999999999999,"15min":1.3620000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.77000000000000002},"max":{"1min":1.746,"5min":2.4359999999999999,"15min":2.46},"last":1.502}]},{"osd":7,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.488,"15min":1.4019999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.70899999999999996},"max":{"1min":1.609,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.55},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.4379999999999999,"15min":1.421},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7050000000000001,"5min":2.4580000000000002,"15min":3.1429999999999998},"last":0.98799999999999999}]}]}],"pool_statfs":[]}} 2023-12-19T18:23:11.501 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:23:11.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.771+0000 7f7a04a8a640 1 Processor -- start 2023-12-19T18:23:11.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.771+0000 7f7a04a8a640 1 -- start start 2023-12-19T18:23:11.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.773+0000 7f7a04a8a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7a000cb300 0x7f7a000cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.773+0000 7f7a04a8a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c8f90 0x7f7a00160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.773+0000 7f7a04a8a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a001612d0 con 0x7f7a00160e40 2023-12-19T18:23:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.773+0000 7f7a04a8a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a000c4480 con 0x7f7a000cb300 2023-12-19T18:23:11.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.773+0000 7f7a04a8a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a001468c0 con 0x7f7a000c8f90 2023-12-19T18:23:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.774+0000 7f79fffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7a00160e40 0x7f7a000ac8c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:51752/0 (socket says 172.21.15.27:51752) 2023-12-19T18:23:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.774+0000 7f79fffff640 1 -- 172.21.15.27:0/2535203800 learned_addr learned my addr 172.21.15.27:0/2535203800 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.774+0000 7f79ff7fe640 1 --2- 172.21.15.27:0/2535203800 >> v2:172.21.15.192:3300/0 conn(0x7f7a000cb300 0x7f7a000cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.774+0000 7f79feffd640 1 --2- 172.21.15.27:0/2535203800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c8f90 0x7f7a00160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3529823584 0 0) 0x7f79f8001490 con 0x7f7a00160e40 2023-12-19T18:23:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79e8003520 con 0x7f7a00160e40 2023-12-19T18:23:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2809157046 0 0) 0x7f79e8003520 con 0x7f7a00160e40 2023-12-19T18:23:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c8f90 msgr2=0x7f7a00160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 --2- 172.21.15.27:0/2535203800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c8f90 0x7f7a00160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 >> v2:172.21.15.192:3300/0 conn(0x7f7a000cb300 msgr2=0x7f7a000cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 --2- 172.21.15.27:0/2535203800 >> v2:172.21.15.192:3300/0 conn(0x7f7a000cb300 0x7f7a000cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a000cbc20 con 0x7f7a00160e40 2023-12-19T18:23:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f79f8001b80 con 0x7f7a00160e40 2023-12-19T18:23:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 >> v1:172.21.15.27:6789/0 conn(0x7f7a00160e40 legacy=0x7f7a000ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 --2- 172.21.15.27:0/2535203800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e8003970 0x7f79e8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 --2- 172.21.15.27:0/2535203800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c8f90 0x7f79e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 --2- 172.21.15.27:0/2535203800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79e8004d00 0x7f79e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a001468c0 con 0x7f79e8004d00 2023-12-19T18:23:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e800b980 con 0x7f79e8003970 2023-12-19T18:23:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.775+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79e800bac0 con 0x7f7a000c8f90 2023-12-19T18:23:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.776+0000 7f79fffff640 1 --2- 172.21.15.27:0/2535203800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79e8004d00 0x7f79e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.776+0000 7f79feffd640 1 --2- 172.21.15.27:0/2535203800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c8f90 0x7f79e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.776+0000 7f79ff7fe640 1 --2- 172.21.15.27:0/2535203800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e8003970 0x7f79e8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.776+0000 7f79fffff640 1 -- 172.21.15.27:0/2535203800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c8f90 msgr2=0x7f79e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.776+0000 7f79fffff640 1 --2- 172.21.15.27:0/2535203800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000c8f90 0x7f79e80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.776+0000 7f79fffff640 1 -- 172.21.15.27:0/2535203800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e8003970 msgr2=0x7f79e8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.776+0000 7f79fffff640 1 --2- 172.21.15.27:0/2535203800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79e8003970 0x7f79e8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.776+0000 7f79fffff640 1 -- 172.21.15.27:0/2535203800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e8003520 con 0x7f79e8004d00 2023-12-19T18:23:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.777+0000 7f79fffff640 1 --2- 172.21.15.27:0/2535203800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79e8004d00 0x7f79e8005180 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f79f800a7b0 tx=0x7f79e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d61367e1cfc89cc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.778+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f8034630 con 0x7f79e8004d00 2023-12-19T18:23:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.778+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f802f070 con 0x7f79e8004d00 2023-12-19T18:23:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.778+0000 7f79fe7fc640 1 -- 172.21.15.27:0/2535203800 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f80349f0 con 0x7f79e8004d00 2023-12-19T18:23:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.778+0000 7f7a04a8a640 1 -- 172.21.15.27:0/2535203800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79e8004d00 msgr2=0x7f79e8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.778+0000 7f7a04a8a640 1 --2- 172.21.15.27:0/2535203800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79e8004d00 0x7f79e8005180 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f79f800a7b0 tx=0x7f79e80057f0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.779+0000 7f79fffff640 1 -- 172.21.15.27:0/2535203800 reap_dead start 2023-12-19T18:23:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.779+0000 7f7a04a8a640 1 -- 172.21.15.27:0/2535203800 shutdown_connections 2023-12-19T18:23:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.779+0000 7f7a04a8a640 1 -- 172.21.15.27:0/2535203800 >> 172.21.15.27:0/2535203800 conn(0x7f7a00069c70 msgr2=0x7f7a0006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.779+0000 7f7a04a8a640 1 -- 172.21.15.27:0/2535203800 shutdown_connections 2023-12-19T18:23:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.779+0000 7f7a04a8a640 1 -- 172.21.15.27:0/2535203800 wait complete. 2023-12-19T18:23:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.781+0000 7f7a04a8a640 1 Processor -- start 2023-12-19T18:23:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.781+0000 7f7a04a8a640 1 -- start start 2023-12-19T18:23:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.782+0000 7f7a04a8a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79e8004d00 0x7f7a000c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.782+0000 7f7a04a8a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000b3960 0x7f7a00142380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.782+0000 7f7a04a8a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a0006f240 0x7f7a0006f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.782+0000 7f7a04a8a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a000c8a70 con 0x7f79e8004d00 2023-12-19T18:23:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.782+0000 7f7a04a8a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0006fc30 con 0x7f7a0006f240 2023-12-19T18:23:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.782+0000 7f7a04a8a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a000b4930 con 0x7f7a000b3960 2023-12-19T18:23:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.782+0000 7f79ff7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79e8004d00 0x7f7a000c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.782+0000 7f79ff7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79e8004d00 0x7f7a000c8530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52956/0 (socket says 172.21.15.27:52956) 2023-12-19T18:23:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.782+0000 7f79ff7fe640 1 -- 172.21.15.27:0/1326848402 learned_addr learned my addr 172.21.15.27:0/1326848402 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.782+0000 7f79feffd640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000b3960 0x7f7a00142380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.783+0000 7f79fffff640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a0006f240 0x7f7a0006f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.783+0000 7f79ff7fe640 1 -- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000b3960 msgr2=0x7f7a00142380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.783+0000 7f79ff7fe640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000b3960 0x7f7a00142380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.783+0000 7f79ff7fe640 1 -- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a0006f240 msgr2=0x7f7a0006f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.783+0000 7f79ff7fe640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a0006f240 0x7f7a0006f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.783+0000 7f79ff7fe640 1 -- 172.21.15.27:0/1326848402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a000b3de0 con 0x7f79e8004d00 2023-12-19T18:23:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.783+0000 7f79fffff640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a0006f240 0x7f7a0006f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:11.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.783+0000 7f79ff7fe640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79e8004d00 0x7f7a000c8530 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f79f0011700 tx=0x7f79f0011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:11.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.783+0000 7f79e7fff640 1 -- 172.21.15.27:0/1326848402 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f000f9c0 con 0x7f79e8004d00 2023-12-19T18:23:11.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.783+0000 7f79e7fff640 1 -- 172.21.15.27:0/1326848402 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79f000fb20 con 0x7f79e8004d00 2023-12-19T18:23:11.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.783+0000 7f79e7fff640 1 -- 172.21.15.27:0/1326848402 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79f000fe90 con 0x7f79e8004d00 2023-12-19T18:23:11.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.783+0000 7f7a04a8a640 1 -- 172.21.15.27:0/1326848402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a001428c0 con 0x7f79e8004d00 2023-12-19T18:23:11.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.784+0000 7f7a04a8a640 1 -- 172.21.15.27:0/1326848402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a000b4370 con 0x7f79e8004d00 2023-12-19T18:23:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.785+0000 7f7a04a8a640 1 -- 172.21.15.27:0/1326848402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0004e840 con 0x7f79e8004d00 2023-12-19T18:23:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.786+0000 7f79e7fff640 1 -- 172.21.15.27:0/1326848402 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f79f0015460 con 0x7f79e8004d00 2023-12-19T18:23:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.787+0000 7f79e7fff640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79ec06a430 0x7f79ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.787+0000 7f79e7fff640 1 -- 172.21.15.27:0/1326848402 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f79f0088de0 con 0x7f79e8004d00 2023-12-19T18:23:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.788+0000 7f79feffd640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79ec06a430 0x7f79ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:11.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.789+0000 7f79feffd640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79ec06a430 0x7f79ec06c8f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7a000ca040 tx=0x7f79f4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:11.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.792+0000 7f79e7fff640 1 -- 172.21.15.27:0/1326848402 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f79f0054460 con 0x7f79e8004d00 2023-12-19T18:23:11.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:11.981+0000 7f7a04a8a640 1 -- 172.21.15.27:0/1326848402 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a00161510 con 0x7f79ec06a430 2023-12-19T18:23:12.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.005+0000 7f79e7fff640 1 -- 172.21.15.27:0/1326848402 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282986 (secure 0 0 0) 0x7f79f4004030 con 0x7f79ec06a430 2023-12-19T18:23:12.006 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:12.010 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:23:12.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.012+0000 7f7a04a8a640 1 -- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79ec06a430 msgr2=0x7f79ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:12.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.012+0000 7f7a04a8a640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79ec06a430 0x7f79ec06c8f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7a000ca040 tx=0x7f79f4008040 comp rx=0 tx=0).stop 2023-12-19T18:23:12.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.012+0000 7f7a04a8a640 1 -- 172.21.15.27:0/1326848402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79e8004d00 msgr2=0x7f7a000c8530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:12.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.012+0000 7f7a04a8a640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79e8004d00 0x7f7a000c8530 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f79f0011700 tx=0x7f79f0011bc0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.012+0000 7f7a04a8a640 1 -- 172.21.15.27:0/1326848402 shutdown_connections 2023-12-19T18:23:12.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.012+0000 7f7a04a8a640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79ec06a430 0x7f79ec06c8f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.012+0000 7f7a04a8a640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a0006f240 0x7f7a0006f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.012+0000 7f7a04a8a640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a000b3960 0x7f7a00142380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.012+0000 7f7a04a8a640 1 --2- 172.21.15.27:0/1326848402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79e8004d00 0x7f7a000c8530 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.012+0000 7f7a04a8a640 1 -- 172.21.15.27:0/1326848402 >> 172.21.15.27:0/1326848402 conn(0x7f7a00069c70 msgr2=0x7f7a00070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:12.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.012+0000 7f7a04a8a640 1 -- 172.21.15.27:0/1326848402 shutdown_connections 2023-12-19T18:23:12.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.013+0000 7f7a04a8a640 1 -- 172.21.15.27:0/1326848402 wait complete. 2023-12-19T18:23:12.034 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":398,"stamp":"2023-12-19T18:23:11.885227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":98,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":7810,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1554689,"statfs":{"total":1600000000,"available":1592001720,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.002039"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196527+0000","last_change":"2023-12-19T18:23:06.507531+0000","last_active":"2023-12-19T18:23:08.196527+0000","last_peered":"2023-12-19T18:23:08.196527+0000","last_clean":"2023-12-19T18:23:08.196527+0000","last_became_active":"2023-12-19T18:23:06.506056+0000","last_became_peered":"2023-12-19T18:23:06.506056+0000","last_unstale":"2023-12-19T18:23:08.196527+0000","last_undegraded":"2023-12-19T18:23:08.196527+0000","last_fullsized":"2023-12-19T18:23:08.196527+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:56.690910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.477351+0000","last_change":"2023-12-19T18:23:06.492270+0000","last_active":"2023-12-19T18:23:07.477351+0000","last_peered":"2023-12-19T18:23:07.477351+0000","last_clean":"2023-12-19T18:23:07.477351+0000","last_became_active":"2023-12-19T18:23:06.491935+0000","last_became_peered":"2023-12-19T18:23:06.491935+0000","last_unstale":"2023-12-19T18:23:07.477351+0000","last_undegraded":"2023-12-19T18:23:07.477351+0000","last_fullsized":"2023-12-19T18:23:07.477351+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:41:32.488817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195894+0000","last_change":"2023-12-19T18:23:06.485808+0000","last_active":"2023-12-19T18:23:08.195894+0000","last_peered":"2023-12-19T18:23:08.195894+0000","last_clean":"2023-12-19T18:23:08.195894+0000","last_became_active":"2023-12-19T18:23:06.485575+0000","last_became_peered":"2023-12-19T18:23:06.485575+0000","last_unstale":"2023-12-19T18:23:08.195894+0000","last_undegraded":"2023-12-19T18:23:08.195894+0000","last_fullsized":"2023-12-19T18:23:08.195894+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:55:21.903528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479661+0000","last_change":"2023-12-19T18:23:06.495516+0000","last_active":"2023-12-19T18:23:07.479661+0000","last_peered":"2023-12-19T18:23:07.479661+0000","last_clean":"2023-12-19T18:23:07.479661+0000","last_became_active":"2023-12-19T18:23:06.494716+0000","last_became_peered":"2023-12-19T18:23:06.494716+0000","last_unstale":"2023-12-19T18:23:07.479661+0000","last_undegraded":"2023-12-19T18:23:07.479661+0000","last_fullsized":"2023-12-19T18:23:07.479661+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:21.096114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196988+0000","last_change":"2023-12-19T18:23:06.510753+0000","last_active":"2023-12-19T18:23:08.196988+0000","last_peered":"2023-12-19T18:23:08.196988+0000","last_clean":"2023-12-19T18:23:08.196988+0000","last_became_active":"2023-12-19T18:23:06.510070+0000","last_became_peered":"2023-12-19T18:23:06.510070+0000","last_unstale":"2023-12-19T18:23:08.196988+0000","last_undegraded":"2023-12-19T18:23:08.196988+0000","last_fullsized":"2023-12-19T18:23:08.196988+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:24:06.331884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475511+0000","last_change":"2023-12-19T18:23:06.496021+0000","last_active":"2023-12-19T18:23:07.475511+0000","last_peered":"2023-12-19T18:23:07.475511+0000","last_clean":"2023-12-19T18:23:07.475511+0000","last_became_active":"2023-12-19T18:23:06.495714+0000","last_became_peered":"2023-12-19T18:23:06.495714+0000","last_unstale":"2023-12-19T18:23:07.475511+0000","last_undegraded":"2023-12-19T18:23:07.475511+0000","last_fullsized":"2023-12-19T18:23:07.475511+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:43:41.974906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196569+0000","last_change":"2023-12-19T18:23:06.494420+0000","last_active":"2023-12-19T18:23:08.196569+0000","last_peered":"2023-12-19T18:23:08.196569+0000","last_clean":"2023-12-19T18:23:08.196569+0000","last_became_active":"2023-12-19T18:23:06.494147+0000","last_became_peered":"2023-12-19T18:23:06.494147+0000","last_unstale":"2023-12-19T18:23:08.196569+0000","last_undegraded":"2023-12-19T18:23:08.196569+0000","last_fullsized":"2023-12-19T18:23:08.196569+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:22:16.036766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196925+0000","last_change":"2023-12-19T18:23:06.493298+0000","last_active":"2023-12-19T18:23:08.196925+0000","last_peered":"2023-12-19T18:23:08.196925+0000","last_clean":"2023-12-19T18:23:08.196925+0000","last_became_active":"2023-12-19T18:23:06.493071+0000","last_became_peered":"2023-12-19T18:23:06.493071+0000","last_unstale":"2023-12-19T18:23:08.196925+0000","last_undegraded":"2023-12-19T18:23:08.196925+0000","last_fullsized":"2023-12-19T18:23:08.196925+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:51:17.841146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195747+0000","last_change":"2023-12-19T18:23:06.507449+0000","last_active":"2023-12-19T18:23:08.195747+0000","last_peered":"2023-12-19T18:23:08.195747+0000","last_clean":"2023-12-19T18:23:08.195747+0000","last_became_active":"2023-12-19T18:23:06.506616+0000","last_became_peered":"2023-12-19T18:23:06.506616+0000","last_unstale":"2023-12-19T18:23:08.195747+0000","last_undegraded":"2023-12-19T18:23:08.195747+0000","last_fullsized":"2023-12-19T18:23:08.195747+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:06:03.251131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196611+0000","last_change":"2023-12-19T18:23:06.508916+0000","last_active":"2023-12-19T18:23:08.196611+0000","last_peered":"2023-12-19T18:23:08.196611+0000","last_clean":"2023-12-19T18:23:08.196611+0000","last_became_active":"2023-12-19T18:23:06.508129+0000","last_became_peered":"2023-12-19T18:23:06.508129+0000","last_unstale":"2023-12-19T18:23:08.196611+0000","last_undegraded":"2023-12-19T18:23:08.196611+0000","last_fullsized":"2023-12-19T18:23:08.196611+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:16.673265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197160+0000","last_change":"2023-12-19T18:23:06.491479+0000","last_active":"2023-12-19T18:23:08.197160+0000","last_peered":"2023-12-19T18:23:08.197160+0000","last_clean":"2023-12-19T18:23:08.197160+0000","last_became_active":"2023-12-19T18:23:06.491250+0000","last_became_peered":"2023-12-19T18:23:06.491250+0000","last_unstale":"2023-12-19T18:23:08.197160+0000","last_undegraded":"2023-12-19T18:23:08.197160+0000","last_fullsized":"2023-12-19T18:23:08.197160+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:41:59.650926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195570+0000","last_change":"2023-12-19T18:23:06.501465+0000","last_active":"2023-12-19T18:23:08.195570+0000","last_peered":"2023-12-19T18:23:08.195570+0000","last_clean":"2023-12-19T18:23:08.195570+0000","last_became_active":"2023-12-19T18:23:06.501120+0000","last_became_peered":"2023-12-19T18:23:06.501120+0000","last_unstale":"2023-12-19T18:23:08.195570+0000","last_undegraded":"2023-12-19T18:23:08.195570+0000","last_fullsized":"2023-12-19T18:23:08.195570+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:00.294833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.477083+0000","last_change":"2023-12-19T18:23:06.498953+0000","last_active":"2023-12-19T18:23:07.477083+0000","last_peered":"2023-12-19T18:23:07.477083+0000","last_clean":"2023-12-19T18:23:07.477083+0000","last_became_active":"2023-12-19T18:23:06.498730+0000","last_became_peered":"2023-12-19T18:23:06.498730+0000","last_unstale":"2023-12-19T18:23:07.477083+0000","last_undegraded":"2023-12-19T18:23:07.477083+0000","last_fullsized":"2023-12-19T18:23:07.477083+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:14.279141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.919697+0000","last_change":"2023-12-19T18:23:06.492231+0000","last_active":"2023-12-19T18:23:07.919697+0000","last_peered":"2023-12-19T18:23:07.919697+0000","last_clean":"2023-12-19T18:23:07.919697+0000","last_became_active":"2023-12-19T18:23:06.490824+0000","last_became_peered":"2023-12-19T18:23:06.490824+0000","last_unstale":"2023-12-19T18:23:07.919697+0000","last_undegraded":"2023-12-19T18:23:07.919697+0000","last_fullsized":"2023-12-19T18:23:07.919697+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:47:27.154269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.198324+0000","last_change":"2023-12-19T18:23:06.510155+0000","last_active":"2023-12-19T18:23:08.198324+0000","last_peered":"2023-12-19T18:23:08.198324+0000","last_clean":"2023-12-19T18:23:08.198324+0000","last_became_active":"2023-12-19T18:23:06.509735+0000","last_became_peered":"2023-12-19T18:23:06.509735+0000","last_unstale":"2023-12-19T18:23:08.198324+0000","last_undegraded":"2023-12-19T18:23:08.198324+0000","last_fullsized":"2023-12-19T18:23:08.198324+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:18:26.847665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197882+0000","last_change":"2023-12-19T18:23:06.496927+0000","last_active":"2023-12-19T18:23:08.197882+0000","last_peered":"2023-12-19T18:23:08.197882+0000","last_clean":"2023-12-19T18:23:08.197882+0000","last_became_active":"2023-12-19T18:23:06.496418+0000","last_became_peered":"2023-12-19T18:23:06.496418+0000","last_unstale":"2023-12-19T18:23:08.197882+0000","last_undegraded":"2023-12-19T18:23:08.197882+0000","last_fullsized":"2023-12-19T18:23:08.197882+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:58.009794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195527+0000","last_change":"2023-12-19T18:23:06.478414+0000","last_active":"2023-12-19T18:23:08.195527+0000","last_peered":"2023-12-19T18:23:08.195527+0000","last_clean":"2023-12-19T18:23:08.195527+0000","last_became_active":"2023-12-19T18:23:06.478164+0000","last_became_peered":"2023-12-19T18:23:06.478164+0000","last_unstale":"2023-12-19T18:23:08.195527+0000","last_undegraded":"2023-12-19T18:23:08.195527+0000","last_fullsized":"2023-12-19T18:23:08.195527+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:51:00.737229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197219+0000","last_change":"2023-12-19T18:23:06.507509+0000","last_active":"2023-12-19T18:23:08.197219+0000","last_peered":"2023-12-19T18:23:08.197219+0000","last_clean":"2023-12-19T18:23:08.197219+0000","last_became_active":"2023-12-19T18:23:06.507264+0000","last_became_peered":"2023-12-19T18:23:06.507264+0000","last_unstale":"2023-12-19T18:23:08.197219+0000","last_undegraded":"2023-12-19T18:23:08.197219+0000","last_fullsized":"2023-12-19T18:23:08.197219+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:32:32.338248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195360+0000","last_change":"2023-12-19T18:23:06.498747+0000","last_active":"2023-12-19T18:23:08.195360+0000","last_peered":"2023-12-19T18:23:08.195360+0000","last_clean":"2023-12-19T18:23:08.195360+0000","last_became_active":"2023-12-19T18:23:06.498505+0000","last_became_peered":"2023-12-19T18:23:06.498505+0000","last_unstale":"2023-12-19T18:23:08.195360+0000","last_undegraded":"2023-12-19T18:23:08.195360+0000","last_fullsized":"2023-12-19T18:23:08.195360+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:33.851089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195785+0000","last_change":"2023-12-19T18:23:06.492840+0000","last_active":"2023-12-19T18:23:08.195785+0000","last_peered":"2023-12-19T18:23:08.195785+0000","last_clean":"2023-12-19T18:23:08.195785+0000","last_became_active":"2023-12-19T18:23:06.492576+0000","last_became_peered":"2023-12-19T18:23:06.492576+0000","last_unstale":"2023-12-19T18:23:08.195785+0000","last_undegraded":"2023-12-19T18:23:08.195785+0000","last_fullsized":"2023-12-19T18:23:08.195785+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:57:18.772842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197417+0000","last_change":"2023-12-19T18:23:06.496011+0000","last_active":"2023-12-19T18:23:08.197417+0000","last_peered":"2023-12-19T18:23:08.197417+0000","last_clean":"2023-12-19T18:23:08.197417+0000","last_became_active":"2023-12-19T18:23:06.495763+0000","last_became_peered":"2023-12-19T18:23:06.495763+0000","last_unstale":"2023-12-19T18:23:08.197417+0000","last_undegraded":"2023-12-19T18:23:08.197417+0000","last_fullsized":"2023-12-19T18:23:08.197417+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:05:43.826190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475963+0000","last_change":"2023-12-19T18:23:06.496804+0000","last_active":"2023-12-19T18:23:07.475963+0000","last_peered":"2023-12-19T18:23:07.475963+0000","last_clean":"2023-12-19T18:23:07.475963+0000","last_became_active":"2023-12-19T18:23:06.496355+0000","last_became_peered":"2023-12-19T18:23:06.496355+0000","last_unstale":"2023-12-19T18:23:07.475963+0000","last_undegraded":"2023-12-19T18:23:07.475963+0000","last_fullsized":"2023-12-19T18:23:07.475963+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:00:58.389092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.919714+0000","last_change":"2023-12-19T18:23:06.484607+0000","last_active":"2023-12-19T18:23:07.919714+0000","last_peered":"2023-12-19T18:23:07.919714+0000","last_clean":"2023-12-19T18:23:07.919714+0000","last_became_active":"2023-12-19T18:23:06.484265+0000","last_became_peered":"2023-12-19T18:23:06.484265+0000","last_unstale":"2023-12-19T18:23:07.919714+0000","last_undegraded":"2023-12-19T18:23:07.919714+0000","last_fullsized":"2023-12-19T18:23:07.919714+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:58.009794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479791+0000","last_change":"2023-12-19T18:23:06.501397+0000","last_active":"2023-12-19T18:23:07.479791+0000","last_peered":"2023-12-19T18:23:07.479791+0000","last_clean":"2023-12-19T18:23:07.479791+0000","last_became_active":"2023-12-19T18:23:06.500482+0000","last_became_peered":"2023-12-19T18:23:06.500482+0000","last_unstale":"2023-12-19T18:23:07.479791+0000","last_undegraded":"2023-12-19T18:23:07.479791+0000","last_fullsized":"2023-12-19T18:23:07.479791+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:23:03.423398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196947+0000","last_change":"2023-12-19T18:23:06.492341+0000","last_active":"2023-12-19T18:23:08.196947+0000","last_peered":"2023-12-19T18:23:08.196947+0000","last_clean":"2023-12-19T18:23:08.196947+0000","last_became_active":"2023-12-19T18:23:06.492052+0000","last_became_peered":"2023-12-19T18:23:06.492052+0000","last_unstale":"2023-12-19T18:23:08.196947+0000","last_undegraded":"2023-12-19T18:23:08.196947+0000","last_fullsized":"2023-12-19T18:23:08.196947+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:18:32.166889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475986+0000","last_change":"2023-12-19T18:23:06.479557+0000","last_active":"2023-12-19T18:23:07.475986+0000","last_peered":"2023-12-19T18:23:07.475986+0000","last_clean":"2023-12-19T18:23:07.475986+0000","last_became_active":"2023-12-19T18:23:06.478928+0000","last_became_peered":"2023-12-19T18:23:06.478928+0000","last_unstale":"2023-12-19T18:23:07.475986+0000","last_undegraded":"2023-12-19T18:23:07.475986+0000","last_fullsized":"2023-12-19T18:23:07.475986+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:56:50.440480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.478906+0000","last_change":"2023-12-19T18:23:06.502465+0000","last_active":"2023-12-19T18:23:07.478906+0000","last_peered":"2023-12-19T18:23:07.478906+0000","last_clean":"2023-12-19T18:23:07.478906+0000","last_became_active":"2023-12-19T18:23:06.502151+0000","last_became_peered":"2023-12-19T18:23:06.502151+0000","last_unstale":"2023-12-19T18:23:07.478906+0000","last_undegraded":"2023-12-19T18:23:07.478906+0000","last_fullsized":"2023-12-19T18:23:07.478906+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:56.690910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475428+0000","last_change":"2023-12-19T18:23:06.496410+0000","last_active":"2023-12-19T18:23:07.475428+0000","last_peered":"2023-12-19T18:23:07.475428+0000","last_clean":"2023-12-19T18:23:07.475428+0000","last_became_active":"2023-12-19T18:23:06.496079+0000","last_became_peered":"2023-12-19T18:23:06.496079+0000","last_unstale":"2023-12-19T18:23:07.475428+0000","last_undegraded":"2023-12-19T18:23:07.475428+0000","last_fullsized":"2023-12-19T18:23:07.475428+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:49:39.745480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.919937+0000","last_change":"2023-12-19T18:23:06.493740+0000","last_active":"2023-12-19T18:23:07.919937+0000","last_peered":"2023-12-19T18:23:07.919937+0000","last_clean":"2023-12-19T18:23:07.919937+0000","last_became_active":"2023-12-19T18:23:06.491814+0000","last_became_peered":"2023-12-19T18:23:06.491814+0000","last_unstale":"2023-12-19T18:23:07.919937+0000","last_undegraded":"2023-12-19T18:23:07.919937+0000","last_fullsized":"2023-12-19T18:23:07.919937+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:03:11.918746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197262+0000","last_change":"2023-12-19T18:23:06.494995+0000","last_active":"2023-12-19T18:23:08.197262+0000","last_peered":"2023-12-19T18:23:08.197262+0000","last_clean":"2023-12-19T18:23:08.197262+0000","last_became_active":"2023-12-19T18:23:06.494722+0000","last_became_peered":"2023-12-19T18:23:06.494722+0000","last_unstale":"2023-12-19T18:23:08.197262+0000","last_undegraded":"2023-12-19T18:23:08.197262+0000","last_fullsized":"2023-12-19T18:23:08.197262+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:50:26.443334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196674+0000","last_change":"2023-12-19T18:23:06.506386+0000","last_active":"2023-12-19T18:23:08.196674+0000","last_peered":"2023-12-19T18:23:08.196674+0000","last_clean":"2023-12-19T18:23:08.196674+0000","last_became_active":"2023-12-19T18:23:06.505131+0000","last_became_peered":"2023-12-19T18:23:06.505131+0000","last_unstale":"2023-12-19T18:23:08.196674+0000","last_undegraded":"2023-12-19T18:23:08.196674+0000","last_fullsized":"2023-12-19T18:23:08.196674+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:21.096114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196004+0000","last_change":"2023-12-19T18:23:06.496827+0000","last_active":"2023-12-19T18:23:08.196004+0000","last_peered":"2023-12-19T18:23:08.196004+0000","last_clean":"2023-12-19T18:23:08.196004+0000","last_became_active":"2023-12-19T18:23:06.496561+0000","last_became_peered":"2023-12-19T18:23:06.496561+0000","last_unstale":"2023-12-19T18:23:08.196004+0000","last_undegraded":"2023-12-19T18:23:08.196004+0000","last_fullsized":"2023-12-19T18:23:08.196004+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:08.187832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195975+0000","last_change":"2023-12-19T18:23:06.506914+0000","last_active":"2023-12-19T18:23:08.195975+0000","last_peered":"2023-12-19T18:23:08.195975+0000","last_clean":"2023-12-19T18:23:08.195975+0000","last_became_active":"2023-12-19T18:23:06.505804+0000","last_became_peered":"2023-12-19T18:23:06.505804+0000","last_unstale":"2023-12-19T18:23:08.195975+0000","last_undegraded":"2023-12-19T18:23:08.195975+0000","last_fullsized":"2023-12-19T18:23:08.195975+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:14.279141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"151'4","reported_seq":15,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920370+0000","last_change":"2023-12-19T18:23:05.995846+0000","last_active":"2023-12-19T18:23:07.920370+0000","last_peered":"2023-12-19T18:23:07.920370+0000","last_clean":"2023-12-19T18:23:07.920370+0000","last_became_active":"2023-12-19T18:23:05.995558+0000","last_became_peered":"2023-12-19T18:23:05.995558+0000","last_unstale":"2023-12-19T18:23:07.920370+0000","last_undegraded":"2023-12-19T18:23:07.920370+0000","last_fullsized":"2023-12-19T18:23:07.920370+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:46.674483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.919962+0000","last_change":"2023-12-19T18:23:06.484462+0000","last_active":"2023-12-19T18:23:07.919962+0000","last_peered":"2023-12-19T18:23:07.919962+0000","last_clean":"2023-12-19T18:23:07.919962+0000","last_became_active":"2023-12-19T18:23:06.483907+0000","last_became_peered":"2023-12-19T18:23:06.483907+0000","last_unstale":"2023-12-19T18:23:07.919962+0000","last_undegraded":"2023-12-19T18:23:07.919962+0000","last_fullsized":"2023-12-19T18:23:07.919962+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:08.187832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"151'1","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.477145+0000","last_change":"2023-12-19T18:23:05.998193+0000","last_active":"2023-12-19T18:23:07.477145+0000","last_peered":"2023-12-19T18:23:07.477145+0000","last_clean":"2023-12-19T18:23:07.477145+0000","last_became_active":"2023-12-19T18:23:05.997924+0000","last_became_peered":"2023-12-19T18:23:05.997924+0000","last_unstale":"2023-12-19T18:23:07.477145+0000","last_undegraded":"2023-12-19T18:23:07.477145+0000","last_fullsized":"2023-12-19T18:23:07.477145+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:24:44.640215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.476224+0000","last_change":"2023-12-19T18:23:06.508750+0000","last_active":"2023-12-19T18:23:07.476224+0000","last_peered":"2023-12-19T18:23:07.476224+0000","last_clean":"2023-12-19T18:23:07.476224+0000","last_became_active":"2023-12-19T18:23:06.508256+0000","last_became_peered":"2023-12-19T18:23:06.508256+0000","last_unstale":"2023-12-19T18:23:07.476224+0000","last_undegraded":"2023-12-19T18:23:07.476224+0000","last_fullsized":"2023-12-19T18:23:07.476224+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:10:57.367499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"151'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479328+0000","last_change":"2023-12-19T18:23:06.006602+0000","last_active":"2023-12-19T18:23:07.479328+0000","last_peered":"2023-12-19T18:23:07.479328+0000","last_clean":"2023-12-19T18:23:07.479328+0000","last_became_active":"2023-12-19T18:23:06.006377+0000","last_became_peered":"2023-12-19T18:23:06.006377+0000","last_unstale":"2023-12-19T18:23:07.479328+0000","last_undegraded":"2023-12-19T18:23:07.479328+0000","last_fullsized":"2023-12-19T18:23:07.479328+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:59.278694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196096+0000","last_change":"2023-12-19T18:23:06.503903+0000","last_active":"2023-12-19T18:23:08.196096+0000","last_peered":"2023-12-19T18:23:08.196096+0000","last_clean":"2023-12-19T18:23:08.196096+0000","last_became_active":"2023-12-19T18:23:06.503680+0000","last_became_peered":"2023-12-19T18:23:06.503680+0000","last_unstale":"2023-12-19T18:23:08.196096+0000","last_undegraded":"2023-12-19T18:23:08.196096+0000","last_fullsized":"2023-12-19T18:23:08.196096+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:48.370195+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"151'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.478888+0000","last_change":"2023-12-19T18:23:06.005582+0000","last_active":"2023-12-19T18:23:07.478888+0000","last_peered":"2023-12-19T18:23:07.478888+0000","last_clean":"2023-12-19T18:23:07.478888+0000","last_became_active":"2023-12-19T18:23:06.004916+0000","last_became_peered":"2023-12-19T18:23:06.004916+0000","last_unstale":"2023-12-19T18:23:07.478888+0000","last_undegraded":"2023-12-19T18:23:07.478888+0000","last_fullsized":"2023-12-19T18:23:07.478888+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:49.313792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.478311+0000","last_change":"2023-12-19T18:23:06.504917+0000","last_active":"2023-12-19T18:23:07.478311+0000","last_peered":"2023-12-19T18:23:07.478311+0000","last_clean":"2023-12-19T18:23:07.478311+0000","last_became_active":"2023-12-19T18:23:06.502637+0000","last_became_peered":"2023-12-19T18:23:06.502637+0000","last_unstale":"2023-12-19T18:23:07.478311+0000","last_undegraded":"2023-12-19T18:23:07.478311+0000","last_fullsized":"2023-12-19T18:23:07.478311+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:06:03.251131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"151'1","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196308+0000","last_change":"2023-12-19T18:23:05.999686+0000","last_active":"2023-12-19T18:23:08.196308+0000","last_peered":"2023-12-19T18:23:08.196308+0000","last_clean":"2023-12-19T18:23:08.196308+0000","last_became_active":"2023-12-19T18:23:05.997802+0000","last_became_peered":"2023-12-19T18:23:05.997802+0000","last_unstale":"2023-12-19T18:23:08.196308+0000","last_undegraded":"2023-12-19T18:23:08.196308+0000","last_fullsized":"2023-12-19T18:23:08.196308+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:56:18.105452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196725+0000","last_change":"2023-12-19T18:23:06.498015+0000","last_active":"2023-12-19T18:23:08.196725+0000","last_peered":"2023-12-19T18:23:08.196725+0000","last_clean":"2023-12-19T18:23:08.196725+0000","last_became_active":"2023-12-19T18:23:06.497786+0000","last_became_peered":"2023-12-19T18:23:06.497786+0000","last_unstale":"2023-12-19T18:23:08.196725+0000","last_undegraded":"2023-12-19T18:23:08.196725+0000","last_fullsized":"2023-12-19T18:23:08.196725+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:30:36.322169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"151'1","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197084+0000","last_change":"2023-12-19T18:23:06.006189+0000","last_active":"2023-12-19T18:23:08.197084+0000","last_peered":"2023-12-19T18:23:08.197084+0000","last_clean":"2023-12-19T18:23:08.197084+0000","last_became_active":"2023-12-19T18:23:06.005474+0000","last_became_peered":"2023-12-19T18:23:06.005474+0000","last_unstale":"2023-12-19T18:23:08.197084+0000","last_undegraded":"2023-12-19T18:23:08.197084+0000","last_fullsized":"2023-12-19T18:23:08.197084+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:15.657126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196222+0000","last_change":"2023-12-19T18:23:06.508377+0000","last_active":"2023-12-19T18:23:08.196222+0000","last_peered":"2023-12-19T18:23:08.196222+0000","last_clean":"2023-12-19T18:23:08.196222+0000","last_became_active":"2023-12-19T18:23:06.506498+0000","last_became_peered":"2023-12-19T18:23:06.506498+0000","last_unstale":"2023-12-19T18:23:08.196222+0000","last_undegraded":"2023-12-19T18:23:08.196222+0000","last_fullsized":"2023-12-19T18:23:08.196222+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:24:45.656354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196426+0000","last_change":"2023-12-19T18:23:06.002642+0000","last_active":"2023-12-19T18:23:08.196426+0000","last_peered":"2023-12-19T18:23:08.196426+0000","last_clean":"2023-12-19T18:23:08.196426+0000","last_became_active":"2023-12-19T18:23:06.002468+0000","last_became_peered":"2023-12-19T18:23:06.002468+0000","last_unstale":"2023-12-19T18:23:08.196426+0000","last_undegraded":"2023-12-19T18:23:08.196426+0000","last_fullsized":"2023-12-19T18:23:08.196426+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:14:02.961282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195982+0000","last_change":"2023-12-19T18:23:06.489901+0000","last_active":"2023-12-19T18:23:08.195982+0000","last_peered":"2023-12-19T18:23:08.195982+0000","last_clean":"2023-12-19T18:23:08.195982+0000","last_became_active":"2023-12-19T18:23:06.489689+0000","last_became_peered":"2023-12-19T18:23:06.489689+0000","last_unstale":"2023-12-19T18:23:08.195982+0000","last_undegraded":"2023-12-19T18:23:08.195982+0000","last_fullsized":"2023-12-19T18:23:08.195982+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:03:14.058758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196875+0000","last_change":"2023-12-19T18:23:06.007538+0000","last_active":"2023-12-19T18:23:08.196875+0000","last_peered":"2023-12-19T18:23:08.196875+0000","last_clean":"2023-12-19T18:23:08.196875+0000","last_became_active":"2023-12-19T18:23:06.005765+0000","last_became_peered":"2023-12-19T18:23:06.005765+0000","last_unstale":"2023-12-19T18:23:08.196875+0000","last_undegraded":"2023-12-19T18:23:08.196875+0000","last_fullsized":"2023-12-19T18:23:08.196875+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:34:36.645513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.477469+0000","last_change":"2023-12-19T18:23:06.499805+0000","last_active":"2023-12-19T18:23:07.477469+0000","last_peered":"2023-12-19T18:23:07.477469+0000","last_clean":"2023-12-19T18:23:07.477469+0000","last_became_active":"2023-12-19T18:23:06.499641+0000","last_became_peered":"2023-12-19T18:23:06.499641+0000","last_unstale":"2023-12-19T18:23:07.477469+0000","last_undegraded":"2023-12-19T18:23:07.477469+0000","last_fullsized":"2023-12-19T18:23:07.477469+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:21.096114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479105+0000","last_change":"2023-12-19T18:23:06.005442+0000","last_active":"2023-12-19T18:23:07.479105+0000","last_peered":"2023-12-19T18:23:07.479105+0000","last_clean":"2023-12-19T18:23:07.479105+0000","last_became_active":"2023-12-19T18:23:06.004430+0000","last_became_peered":"2023-12-19T18:23:06.004430+0000","last_unstale":"2023-12-19T18:23:07.479105+0000","last_undegraded":"2023-12-19T18:23:07.479105+0000","last_fullsized":"2023-12-19T18:23:07.479105+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:15.657126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.477077+0000","last_change":"2023-12-19T18:23:06.495638+0000","last_active":"2023-12-19T18:23:07.477077+0000","last_peered":"2023-12-19T18:23:07.477077+0000","last_clean":"2023-12-19T18:23:07.477077+0000","last_became_active":"2023-12-19T18:23:06.495398+0000","last_became_peered":"2023-12-19T18:23:06.495398+0000","last_unstale":"2023-12-19T18:23:07.477077+0000","last_undegraded":"2023-12-19T18:23:07.477077+0000","last_fullsized":"2023-12-19T18:23:07.477077+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:03.827467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196417+0000","last_change":"2023-12-19T18:23:06.006199+0000","last_active":"2023-12-19T18:23:08.196417+0000","last_peered":"2023-12-19T18:23:08.196417+0000","last_clean":"2023-12-19T18:23:08.196417+0000","last_became_active":"2023-12-19T18:23:06.005495+0000","last_became_peered":"2023-12-19T18:23:06.005495+0000","last_unstale":"2023-12-19T18:23:08.196417+0000","last_undegraded":"2023-12-19T18:23:08.196417+0000","last_fullsized":"2023-12-19T18:23:08.196417+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:32.834949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475416+0000","last_change":"2023-12-19T18:23:06.485604+0000","last_active":"2023-12-19T18:23:07.475416+0000","last_peered":"2023-12-19T18:23:07.475416+0000","last_clean":"2023-12-19T18:23:07.475416+0000","last_became_active":"2023-12-19T18:23:06.484970+0000","last_became_peered":"2023-12-19T18:23:06.484970+0000","last_unstale":"2023-12-19T18:23:07.475416+0000","last_undegraded":"2023-12-19T18:23:07.475416+0000","last_fullsized":"2023-12-19T18:23:07.475416+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:20:31.373582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"151'3","reported_seq":14,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195729+0000","last_change":"2023-12-19T18:23:06.008799+0000","last_active":"2023-12-19T18:23:08.195729+0000","last_peered":"2023-12-19T18:23:08.195729+0000","last_clean":"2023-12-19T18:23:08.195729+0000","last_became_active":"2023-12-19T18:23:06.008565+0000","last_became_peered":"2023-12-19T18:23:06.008565+0000","last_unstale":"2023-12-19T18:23:08.195729+0000","last_undegraded":"2023-12-19T18:23:08.195729+0000","last_fullsized":"2023-12-19T18:23:08.195729+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:50:02.807770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197448+0000","last_change":"2023-12-19T18:23:06.490824+0000","last_active":"2023-12-19T18:23:08.197448+0000","last_peered":"2023-12-19T18:23:08.197448+0000","last_clean":"2023-12-19T18:23:08.197448+0000","last_became_active":"2023-12-19T18:23:06.490607+0000","last_became_peered":"2023-12-19T18:23:06.490607+0000","last_unstale":"2023-12-19T18:23:08.197448+0000","last_undegraded":"2023-12-19T18:23:08.197448+0000","last_fullsized":"2023-12-19T18:23:08.197448+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:54:24.376646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"151'6","reported_seq":17,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479992+0000","last_change":"2023-12-19T18:23:06.003698+0000","last_active":"2023-12-19T18:23:07.479992+0000","last_peered":"2023-12-19T18:23:07.479992+0000","last_clean":"2023-12-19T18:23:07.479992+0000","last_became_active":"2023-12-19T18:23:06.003232+0000","last_became_peered":"2023-12-19T18:23:06.003232+0000","last_unstale":"2023-12-19T18:23:07.479992+0000","last_undegraded":"2023-12-19T18:23:07.479992+0000","last_fullsized":"2023-12-19T18:23:07.479992+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:01.166002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197490+0000","last_change":"2023-12-19T18:23:06.496783+0000","last_active":"2023-12-19T18:23:08.197490+0000","last_peered":"2023-12-19T18:23:08.197490+0000","last_clean":"2023-12-19T18:23:08.197490+0000","last_became_active":"2023-12-19T18:23:06.496089+0000","last_became_peered":"2023-12-19T18:23:06.496089+0000","last_unstale":"2023-12-19T18:23:08.197490+0000","last_undegraded":"2023-12-19T18:23:08.197490+0000","last_fullsized":"2023-12-19T18:23:08.197490+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:02.182142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"151'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197102+0000","last_change":"2023-12-19T18:23:06.007242+0000","last_active":"2023-12-19T18:23:08.197102+0000","last_peered":"2023-12-19T18:23:08.197102+0000","last_clean":"2023-12-19T18:23:08.197102+0000","last_became_active":"2023-12-19T18:23:06.005442+0000","last_became_peered":"2023-12-19T18:23:06.005442+0000","last_unstale":"2023-12-19T18:23:08.197102+0000","last_undegraded":"2023-12-19T18:23:08.197102+0000","last_fullsized":"2023-12-19T18:23:08.197102+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:03:47.097303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479078+0000","last_change":"2023-12-19T18:23:06.487983+0000","last_active":"2023-12-19T18:23:07.479078+0000","last_peered":"2023-12-19T18:23:07.479078+0000","last_clean":"2023-12-19T18:23:07.479078+0000","last_became_active":"2023-12-19T18:23:06.487122+0000","last_became_peered":"2023-12-19T18:23:06.487122+0000","last_unstale":"2023-12-19T18:23:07.479078+0000","last_undegraded":"2023-12-19T18:23:07.479078+0000","last_fullsized":"2023-12-19T18:23:07.479078+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:03.827467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196589+0000","last_change":"2023-12-19T18:23:05.990252+0000","last_active":"2023-12-19T18:23:08.196589+0000","last_peered":"2023-12-19T18:23:08.196589+0000","last_clean":"2023-12-19T18:23:08.196589+0000","last_became_active":"2023-12-19T18:23:05.989928+0000","last_became_peered":"2023-12-19T18:23:05.989928+0000","last_unstale":"2023-12-19T18:23:08.196589+0000","last_undegraded":"2023-12-19T18:23:08.196589+0000","last_fullsized":"2023-12-19T18:23:08.196589+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:02:23.605737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479129+0000","last_change":"2023-12-19T18:23:06.488679+0000","last_active":"2023-12-19T18:23:07.479129+0000","last_peered":"2023-12-19T18:23:07.479129+0000","last_clean":"2023-12-19T18:23:07.479129+0000","last_became_active":"2023-12-19T18:23:06.487705+0000","last_became_peered":"2023-12-19T18:23:06.487705+0000","last_unstale":"2023-12-19T18:23:07.479129+0000","last_undegraded":"2023-12-19T18:23:07.479129+0000","last_fullsized":"2023-12-19T18:23:07.479129+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:31.720170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"151'8","reported_seq":19,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197583+0000","last_change":"2023-12-19T18:23:06.006003+0000","last_active":"2023-12-19T18:23:08.197583+0000","last_peered":"2023-12-19T18:23:08.197583+0000","last_clean":"2023-12-19T18:23:08.197583+0000","last_became_active":"2023-12-19T18:23:06.005068+0000","last_became_peered":"2023-12-19T18:23:06.005068+0000","last_unstale":"2023-12-19T18:23:08.197583+0000","last_undegraded":"2023-12-19T18:23:08.197583+0000","last_fullsized":"2023-12-19T18:23:08.197583+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:20.334858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.198552+0000","last_change":"2023-12-19T18:23:06.507941+0000","last_active":"2023-12-19T18:23:08.198552+0000","last_peered":"2023-12-19T18:23:08.198552+0000","last_clean":"2023-12-19T18:23:08.198552+0000","last_became_active":"2023-12-19T18:23:06.506481+0000","last_became_peered":"2023-12-19T18:23:06.506481+0000","last_unstale":"2023-12-19T18:23:08.198552+0000","last_undegraded":"2023-12-19T18:23:08.198552+0000","last_fullsized":"2023-12-19T18:23:08.198552+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:01:54.556435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"151'1","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475769+0000","last_change":"2023-12-19T18:23:06.000376+0000","last_active":"2023-12-19T18:23:07.475769+0000","last_peered":"2023-12-19T18:23:07.475769+0000","last_clean":"2023-12-19T18:23:07.475769+0000","last_became_active":"2023-12-19T18:23:05.999715+0000","last_became_peered":"2023-12-19T18:23:05.999715+0000","last_unstale":"2023-12-19T18:23:07.475769+0000","last_undegraded":"2023-12-19T18:23:07.475769+0000","last_fullsized":"2023-12-19T18:23:07.475769+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:25:20.079974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196204+0000","last_change":"2023-12-19T18:23:06.497446+0000","last_active":"2023-12-19T18:23:08.196204+0000","last_peered":"2023-12-19T18:23:08.196204+0000","last_clean":"2023-12-19T18:23:08.196204+0000","last_became_active":"2023-12-19T18:23:06.497070+0000","last_became_peered":"2023-12-19T18:23:06.497070+0000","last_unstale":"2023-12-19T18:23:08.196204+0000","last_undegraded":"2023-12-19T18:23:08.196204+0000","last_fullsized":"2023-12-19T18:23:08.196204+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:41:32.488817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"151'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197225+0000","last_change":"2023-12-19T18:23:06.007751+0000","last_active":"2023-12-19T18:23:08.197225+0000","last_peered":"2023-12-19T18:23:08.197225+0000","last_clean":"2023-12-19T18:23:08.197225+0000","last_became_active":"2023-12-19T18:23:06.006705+0000","last_became_peered":"2023-12-19T18:23:06.006705+0000","last_unstale":"2023-12-19T18:23:08.197225+0000","last_undegraded":"2023-12-19T18:23:08.197225+0000","last_fullsized":"2023-12-19T18:23:08.197225+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:47.354055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196872+0000","last_change":"2023-12-19T18:23:06.500822+0000","last_active":"2023-12-19T18:23:08.196872+0000","last_peered":"2023-12-19T18:23:08.196872+0000","last_clean":"2023-12-19T18:23:08.196872+0000","last_became_active":"2023-12-19T18:23:06.500680+0000","last_became_peered":"2023-12-19T18:23:06.500680+0000","last_unstale":"2023-12-19T18:23:08.196872+0000","last_undegraded":"2023-12-19T18:23:08.196872+0000","last_fullsized":"2023-12-19T18:23:08.196872+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:07:19.550672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"151'1","reported_seq":12,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.921225+0000","last_change":"2023-12-19T18:23:05.999099+0000","last_active":"2023-12-19T18:23:07.921225+0000","last_peered":"2023-12-19T18:23:07.921225+0000","last_clean":"2023-12-19T18:23:07.921225+0000","last_became_active":"2023-12-19T18:23:05.998714+0000","last_became_peered":"2023-12-19T18:23:05.998714+0000","last_unstale":"2023-12-19T18:23:07.921225+0000","last_undegraded":"2023-12-19T18:23:07.921225+0000","last_fullsized":"2023-12-19T18:23:07.921225+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:50:59.721089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920569+0000","last_change":"2023-12-19T18:23:06.480947+0000","last_active":"2023-12-19T18:23:07.920569+0000","last_peered":"2023-12-19T18:23:07.920569+0000","last_clean":"2023-12-19T18:23:07.920569+0000","last_became_active":"2023-12-19T18:23:06.480673+0000","last_became_peered":"2023-12-19T18:23:06.480673+0000","last_unstale":"2023-12-19T18:23:07.920569+0000","last_undegraded":"2023-12-19T18:23:07.920569+0000","last_fullsized":"2023-12-19T18:23:07.920569+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:14.161699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920536+0000","last_change":"2023-12-19T18:23:05.991986+0000","last_active":"2023-12-19T18:23:07.920536+0000","last_peered":"2023-12-19T18:23:07.920536+0000","last_clean":"2023-12-19T18:23:07.920536+0000","last_became_active":"2023-12-19T18:23:05.991293+0000","last_became_peered":"2023-12-19T18:23:05.991293+0000","last_unstale":"2023-12-19T18:23:07.920536+0000","last_undegraded":"2023-12-19T18:23:07.920536+0000","last_fullsized":"2023-12-19T18:23:07.920536+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:57:17.756702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196059+0000","last_change":"2023-12-19T18:23:06.484800+0000","last_active":"2023-12-19T18:23:08.196059+0000","last_peered":"2023-12-19T18:23:08.196059+0000","last_clean":"2023-12-19T18:23:08.196059+0000","last_became_active":"2023-12-19T18:23:06.484571+0000","last_became_peered":"2023-12-19T18:23:06.484571+0000","last_unstale":"2023-12-19T18:23:08.196059+0000","last_undegraded":"2023-12-19T18:23:08.196059+0000","last_fullsized":"2023-12-19T18:23:08.196059+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:54.565682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"151'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196009+0000","last_change":"2023-12-19T18:23:06.005564+0000","last_active":"2023-12-19T18:23:08.196009+0000","last_peered":"2023-12-19T18:23:08.196009+0000","last_clean":"2023-12-19T18:23:08.196009+0000","last_became_active":"2023-12-19T18:23:06.005261+0000","last_became_peered":"2023-12-19T18:23:06.005261+0000","last_unstale":"2023-12-19T18:23:08.196009+0000","last_undegraded":"2023-12-19T18:23:08.196009+0000","last_fullsized":"2023-12-19T18:23:08.196009+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:06:53.549543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197350+0000","last_change":"2023-12-19T18:23:06.503882+0000","last_active":"2023-12-19T18:23:08.197350+0000","last_peered":"2023-12-19T18:23:08.197350+0000","last_clean":"2023-12-19T18:23:08.197350+0000","last_became_active":"2023-12-19T18:23:06.502559+0000","last_became_peered":"2023-12-19T18:23:06.502559+0000","last_unstale":"2023-12-19T18:23:08.197350+0000","last_undegraded":"2023-12-19T18:23:08.197350+0000","last_fullsized":"2023-12-19T18:23:08.197350+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:34:03.827467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196458+0000","last_change":"2023-12-19T18:23:06.004039+0000","last_active":"2023-12-19T18:23:08.196458+0000","last_peered":"2023-12-19T18:23:08.196458+0000","last_clean":"2023-12-19T18:23:08.196458+0000","last_became_active":"2023-12-19T18:23:06.003485+0000","last_became_peered":"2023-12-19T18:23:06.003485+0000","last_unstale":"2023-12-19T18:23:08.196458+0000","last_undegraded":"2023-12-19T18:23:08.196458+0000","last_fullsized":"2023-12-19T18:23:08.196458+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:56.993654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920901+0000","last_change":"2023-12-19T18:23:06.507662+0000","last_active":"2023-12-19T18:23:07.920901+0000","last_peered":"2023-12-19T18:23:07.920901+0000","last_clean":"2023-12-19T18:23:07.920901+0000","last_became_active":"2023-12-19T18:23:06.507232+0000","last_became_peered":"2023-12-19T18:23:06.507232+0000","last_unstale":"2023-12-19T18:23:07.920901+0000","last_undegraded":"2023-12-19T18:23:07.920901+0000","last_fullsized":"2023-12-19T18:23:07.920901+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:50.329932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479537+0000","last_change":"2023-12-19T18:23:06.008189+0000","last_active":"2023-12-19T18:23:07.479537+0000","last_peered":"2023-12-19T18:23:07.479537+0000","last_clean":"2023-12-19T18:23:07.479537+0000","last_became_active":"2023-12-19T18:23:06.007954+0000","last_became_peered":"2023-12-19T18:23:06.007954+0000","last_unstale":"2023-12-19T18:23:07.479537+0000","last_undegraded":"2023-12-19T18:23:07.479537+0000","last_fullsized":"2023-12-19T18:23:07.479537+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:17:07.171692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.196469+0000","last_change":"2023-12-19T18:23:06.508644+0000","last_active":"2023-12-19T18:23:08.196469+0000","last_peered":"2023-12-19T18:23:08.196469+0000","last_clean":"2023-12-19T18:23:08.196469+0000","last_became_active":"2023-12-19T18:23:06.507865+0000","last_became_peered":"2023-12-19T18:23:06.507865+0000","last_unstale":"2023-12-19T18:23:08.196469+0000","last_undegraded":"2023-12-19T18:23:08.196469+0000","last_fullsized":"2023-12-19T18:23:08.196469+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:09:50.329932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479331+0000","last_change":"2023-12-19T18:23:05.992912+0000","last_active":"2023-12-19T18:23:07.479331+0000","last_peered":"2023-12-19T18:23:07.479331+0000","last_clean":"2023-12-19T18:23:07.479331+0000","last_became_active":"2023-12-19T18:23:05.991729+0000","last_became_peered":"2023-12-19T18:23:05.991729+0000","last_unstale":"2023-12-19T18:23:07.479331+0000","last_undegraded":"2023-12-19T18:23:07.479331+0000","last_fullsized":"2023-12-19T18:23:07.479331+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:18:56.993654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920216+0000","last_change":"2023-12-19T18:23:06.501125+0000","last_active":"2023-12-19T18:23:07.920216+0000","last_peered":"2023-12-19T18:23:07.920216+0000","last_clean":"2023-12-19T18:23:07.920216+0000","last_became_active":"2023-12-19T18:23:06.500985+0000","last_became_peered":"2023-12-19T18:23:06.500985+0000","last_unstale":"2023-12-19T18:23:07.920216+0000","last_undegraded":"2023-12-19T18:23:07.920216+0000","last_fullsized":"2023-12-19T18:23:07.920216+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:35:00.294833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"151'2","reported_seq":13,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479357+0000","last_change":"2023-12-19T18:23:06.006204+0000","last_active":"2023-12-19T18:23:07.479357+0000","last_peered":"2023-12-19T18:23:07.479357+0000","last_clean":"2023-12-19T18:23:07.479357+0000","last_became_active":"2023-12-19T18:23:06.005971+0000","last_became_peered":"2023-12-19T18:23:06.005971+0000","last_unstale":"2023-12-19T18:23:07.479357+0000","last_undegraded":"2023-12-19T18:23:07.479357+0000","last_fullsized":"2023-12-19T18:23:07.479357+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:20.334858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.476221+0000","last_change":"2023-12-19T18:23:06.496656+0000","last_active":"2023-12-19T18:23:07.476221+0000","last_peered":"2023-12-19T18:23:07.476221+0000","last_clean":"2023-12-19T18:23:07.476221+0000","last_became_active":"2023-12-19T18:23:06.495309+0000","last_became_peered":"2023-12-19T18:23:06.495309+0000","last_unstale":"2023-12-19T18:23:07.476221+0000","last_undegraded":"2023-12-19T18:23:07.476221+0000","last_fullsized":"2023-12-19T18:23:07.476221+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:23.170269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197647+0000","last_change":"2023-12-19T18:23:05.993480+0000","last_active":"2023-12-19T18:23:08.197647+0000","last_peered":"2023-12-19T18:23:08.197647+0000","last_clean":"2023-12-19T18:23:08.197647+0000","last_became_active":"2023-12-19T18:23:05.993067+0000","last_became_peered":"2023-12-19T18:23:05.993067+0000","last_unstale":"2023-12-19T18:23:08.197647+0000","last_undegraded":"2023-12-19T18:23:08.197647+0000","last_fullsized":"2023-12-19T18:23:08.197647+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:25:04.595553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.198775+0000","last_change":"2023-12-19T18:23:06.509957+0000","last_active":"2023-12-19T18:23:08.198775+0000","last_peered":"2023-12-19T18:23:08.198775+0000","last_clean":"2023-12-19T18:23:08.198775+0000","last_became_active":"2023-12-19T18:23:06.509444+0000","last_became_peered":"2023-12-19T18:23:06.509444+0000","last_unstale":"2023-12-19T18:23:08.198775+0000","last_undegraded":"2023-12-19T18:23:08.198775+0000","last_fullsized":"2023-12-19T18:23:08.198775+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:23:03.423398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.480110+0000","last_change":"2023-12-19T18:23:05.993729+0000","last_active":"2023-12-19T18:23:07.480110+0000","last_peered":"2023-12-19T18:23:07.480110+0000","last_clean":"2023-12-19T18:23:07.480110+0000","last_became_active":"2023-12-19T18:23:05.992551+0000","last_became_peered":"2023-12-19T18:23:05.992551+0000","last_unstale":"2023-12-19T18:23:07.480110+0000","last_undegraded":"2023-12-19T18:23:07.480110+0000","last_fullsized":"2023-12-19T18:23:07.480110+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:03:10.902606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920117+0000","last_change":"2023-12-19T18:23:06.492085+0000","last_active":"2023-12-19T18:23:07.920117+0000","last_peered":"2023-12-19T18:23:07.920117+0000","last_clean":"2023-12-19T18:23:07.920117+0000","last_became_active":"2023-12-19T18:23:06.490512+0000","last_became_peered":"2023-12-19T18:23:06.490512+0000","last_unstale":"2023-12-19T18:23:07.920117+0000","last_undegraded":"2023-12-19T18:23:07.920117+0000","last_fullsized":"2023-12-19T18:23:07.920117+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:02.182142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.479733+0000","last_change":"2023-12-19T18:23:05.998603+0000","last_active":"2023-12-19T18:23:07.479733+0000","last_peered":"2023-12-19T18:23:07.479733+0000","last_clean":"2023-12-19T18:23:07.479733+0000","last_became_active":"2023-12-19T18:23:05.998276+0000","last_became_peered":"2023-12-19T18:23:05.998276+0000","last_unstale":"2023-12-19T18:23:07.479733+0000","last_undegraded":"2023-12-19T18:23:07.479733+0000","last_fullsized":"2023-12-19T18:23:07.479733+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:58:13.145559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.195999+0000","last_change":"2023-12-19T18:23:06.492826+0000","last_active":"2023-12-19T18:23:08.195999+0000","last_peered":"2023-12-19T18:23:08.195999+0000","last_clean":"2023-12-19T18:23:08.195999+0000","last_became_active":"2023-12-19T18:23:06.492593+0000","last_became_peered":"2023-12-19T18:23:06.492593+0000","last_unstale":"2023-12-19T18:23:08.195999+0000","last_undegraded":"2023-12-19T18:23:08.195999+0000","last_fullsized":"2023-12-19T18:23:08.195999+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:03:47.690623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197644+0000","last_change":"2023-12-19T18:23:05.994816+0000","last_active":"2023-12-19T18:23:08.197644+0000","last_peered":"2023-12-19T18:23:08.197644+0000","last_clean":"2023-12-19T18:23:08.197644+0000","last_became_active":"2023-12-19T18:23:05.994458+0000","last_became_peered":"2023-12-19T18:23:05.994458+0000","last_unstale":"2023-12-19T18:23:08.197644+0000","last_undegraded":"2023-12-19T18:23:08.197644+0000","last_fullsized":"2023-12-19T18:23:08.197644+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:55:37.048248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.921080+0000","last_change":"2023-12-19T18:23:06.495667+0000","last_active":"2023-12-19T18:23:07.921080+0000","last_peered":"2023-12-19T18:23:07.921080+0000","last_clean":"2023-12-19T18:23:07.921080+0000","last_became_active":"2023-12-19T18:23:06.495424+0000","last_became_peered":"2023-12-19T18:23:06.495424+0000","last_unstale":"2023-12-19T18:23:07.921080+0000","last_undegraded":"2023-12-19T18:23:07.921080+0000","last_fullsized":"2023-12-19T18:23:07.921080+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:58:16.673265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920785+0000","last_change":"2023-12-19T18:23:06.005973+0000","last_active":"2023-12-19T18:23:07.920785+0000","last_peered":"2023-12-19T18:23:07.920785+0000","last_clean":"2023-12-19T18:23:07.920785+0000","last_became_active":"2023-12-19T18:23:06.005622+0000","last_became_peered":"2023-12-19T18:23:06.005622+0000","last_unstale":"2023-12-19T18:23:07.920785+0000","last_undegraded":"2023-12-19T18:23:07.920785+0000","last_fullsized":"2023-12-19T18:23:07.920785+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:55:20.887389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.920275+0000","last_change":"2023-12-19T18:23:06.483661+0000","last_active":"2023-12-19T18:23:07.920275+0000","last_peered":"2023-12-19T18:23:07.920275+0000","last_clean":"2023-12-19T18:23:07.920275+0000","last_became_active":"2023-12-19T18:23:06.483421+0000","last_became_peered":"2023-12-19T18:23:06.483421+0000","last_unstale":"2023-12-19T18:23:07.920275+0000","last_undegraded":"2023-12-19T18:23:07.920275+0000","last_fullsized":"2023-12-19T18:23:07.920275+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:21.350998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:08.197793+0000","last_change":"2023-12-19T18:23:06.005758+0000","last_active":"2023-12-19T18:23:08.197793+0000","last_peered":"2023-12-19T18:23:08.197793+0000","last_clean":"2023-12-19T18:23:08.197793+0000","last_became_active":"2023-12-19T18:23:06.004713+0000","last_became_peered":"2023-12-19T18:23:06.004713+0000","last_unstale":"2023-12-19T18:23:08.197793+0000","last_undegraded":"2023-12-19T18:23:08.197793+0000","last_fullsized":"2023-12-19T18:23:08.197793+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:01.166002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.475695+0000","last_change":"2023-12-19T18:23:06.503953+0000","last_active":"2023-12-19T18:23:07.475695+0000","last_peered":"2023-12-19T18:23:07.475695+0000","last_clean":"2023-12-19T18:23:07.475695+0000","last_became_active":"2023-12-19T18:23:06.503708+0000","last_became_peered":"2023-12-19T18:23:06.503708+0000","last_unstale":"2023-12-19T18:23:07.475695+0000","last_undegraded":"2023-12-19T18:23:07.475695+0000","last_fullsized":"2023-12-19T18:23:07.475695+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:02:27.597298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.476019+0000","last_change":"2023-12-19T18:23:06.000092+0000","last_active":"2023-12-19T18:23:07.476019+0000","last_peered":"2023-12-19T18:23:07.476019+0000","last_clean":"2023-12-19T18:23:07.476019+0000","last_became_active":"2023-12-19T18:23:05.999231+0000","last_became_peered":"2023-12-19T18:23:05.999231+0000","last_unstale":"2023-12-19T18:23:07.476019+0000","last_undegraded":"2023-12-19T18:23:07.476019+0000","last_fullsized":"2023-12-19T18:23:07.476019+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:13.263001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.478694+0000","last_change":"2023-12-19T18:23:06.493560+0000","last_active":"2023-12-19T18:23:07.478694+0000","last_peered":"2023-12-19T18:23:07.478694+0000","last_clean":"2023-12-19T18:23:07.478694+0000","last_became_active":"2023-12-19T18:23:06.493154+0000","last_became_peered":"2023-12-19T18:23:06.493154+0000","last_unstale":"2023-12-19T18:23:07.478694+0000","last_undegraded":"2023-12-19T18:23:07.478694+0000","last_fullsized":"2023-12-19T18:23:07.478694+0000","mapping_epoch":150,"log_start":"0'0","ondisk_log_start":"0'0","created":150,"last_epoch_clean":151,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:05.958405+0000","last_clean_scrub_stamp":"2023-12-19T18:23:05.958405+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:24:14.279141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":152,"state":"active+clean","last_fresh":"2023-12-19T18:23:07.477578+0000","last_change":"2023-12-19T18:23:05.998582+0000","last_active":"2023-12-19T18:23:07.477578+0000","last_peered":"2023-12-19T18:23:07.477578+0000","last_clean":"2023-12-19T18:23:07.477578+0000","last_became_active":"2023-12-19T18:23:05.998355+0000","last_became_peered":"2023-12-19T18:23:05.998355+0000","last_unstale":"2023-12-19T18:23:07.477578+0000","last_undegraded":"2023-12-19T18:23:07.477578+0000","last_fullsized":"2023-12-19T18:23:07.477578+0000","mapping_epoch":149,"log_start":"0'0","ondisk_log_start":"0'0","created":149,"last_epoch_clean":150,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:23:04.942266+0000","last_clean_scrub_stamp":"2023-12-19T18:23:04.942266+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:24:05.315745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":283,"reported_epoch":151,"state":"active+clean","last_fresh":"2023-12-19T18:23:06.474964+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:23:06.474964+0000","last_peered":"2023-12-19T18:23:06.474964+0000","last_clean":"2023-12-19T18:23:06.474964+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:23:06.474964+0000","last_undegraded":"2023-12-19T18:23:06.474964+0000","last_fullsized":"2023-12-19T18:23:06.474964+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":834,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194478,"statfs":{"total":200000000,"available":199145922,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.1319999999999999,"5min":1.23,"15min":1.153},"min":{"1min":0.68000000000000005,"5min":0.57899999999999996,"15min":0.45900000000000002},"max":{"1min":1.587,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.286,"15min":1.196},"min":{"1min":0.76000000000000001,"5min":0.73499999999999999,"15min":0.55800000000000005},"max":{"1min":1.631,"5min":1.7330000000000001,"15min":1.79},"last":1.115}]},{"osd":1,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.117,"5min":1.238,"15min":1.2270000000000001},"min":{"1min":0.70899999999999996,"5min":0.63600000000000001,"15min":0.54400000000000004},"max":{"1min":1.6259999999999999,"5min":5.9029999999999996,"15min":10.926},"last":1.4039999999999999},{"interface":"front","average":{"1min":1.216,"5min":1.2669999999999999,"15min":1.2450000000000001},"min":{"1min":0.85599999999999998,"5min":0.71899999999999997,"15min":0.67900000000000005},"max":{"1min":1.7190000000000001,"5min":1.9039999999999999,"15min":10.904999999999999},"last":1.349}]},{"osd":2,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":1.139,"15min":1.0980000000000001},"min":{"1min":0.66000000000000003,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.5620000000000001,"5min":3.1960000000000002,"15min":3.1960000000000002},"last":0.66000000000000003},{"interface":"front","average":{"1min":1.109,"5min":1.2430000000000001,"15min":1.1899999999999999},"min":{"1min":0.754,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4490000000000001,"5min":6.008,"15min":6.008},"last":1.1519999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.282,"15min":1.1679999999999999},"min":{"1min":0.84499999999999997,"5min":0.66900000000000004,"15min":0.61399999999999999},"max":{"1min":1.659,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":1.194},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.403,"15min":1.298},"min":{"1min":0.88200000000000001,"5min":0.69599999999999995,"15min":0.69599999999999995},"max":{"1min":1.5549999999999999,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":1.3300000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.169,"5min":1.2589999999999999,"15min":1.1319999999999999},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.47899999999999998},"max":{"1min":1.7549999999999999,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":1.1839999999999999,"5min":1.2969999999999999,"15min":1.24},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.57599999999999996},"max":{"1min":1.736,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":0.86099999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.3819999999999999,"15min":1.331},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.956,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.4370000000000001,"15min":1.3400000000000001},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.53500000000000003},"max":{"1min":1.8280000000000001,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":1.272}]},{"osd":6,"last update":"Tue Dec 19 18:23:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4550000000000001,"5min":1.5620000000000001,"15min":1.472},"min":{"1min":0.628,"5min":0.628,"15min":0.59899999999999998},"max":{"1min":1.742,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.4710000000000001,"15min":1.3819999999999999},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.8169999999999999,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":1.0720000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":832,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194480,"statfs":{"total":200000000,"available":199147610,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.0900000000000001,"15min":1.0169999999999999},"min":{"1min":0.72799999999999998,"5min":0.68899999999999995,"15min":0.65800000000000003},"max":{"1min":1.333,"5min":1.6259999999999999,"15min":1.8129999999999999},"last":0.96999999999999997},{"interface":"front","average":{"1min":1.006,"5min":1.1599999999999999,"15min":1.0740000000000001},"min":{"1min":0.64800000000000002,"5min":0.64300000000000002,"15min":0.55600000000000005},"max":{"1min":1.542,"5min":3.5529999999999999,"15min":3.5529999999999999},"last":1.0429999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.1379999999999999,"15min":1.1299999999999999},"min":{"1min":0.77100000000000002,"5min":0.71399999999999997,"15min":0.68200000000000005},"max":{"1min":1.532,"5min":1.5609999999999999,"15min":4.5540000000000003},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.127,"5min":1.208,"15min":1.1679999999999999},"min":{"1min":0.79300000000000004,"5min":0.625,"15min":0.60299999999999998},"max":{"1min":1.583,"5min":1.885,"15min":1.885},"last":1.2729999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.137,"5min":1.2230000000000001,"15min":1.1539999999999999},"min":{"1min":0.871,"5min":0.74099999999999999,"15min":0.626},"max":{"1min":1.389,"5min":1.7330000000000001,"15min":1.821},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.1519999999999999,"5min":1.2190000000000001,"15min":1.2250000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.4950000000000001,"5min":3.7909999999999999,"15min":3.7909999999999999},"last":0.90700000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1519999999999999,"5min":1.2749999999999999,"15min":1.202},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5580000000000001,"5min":3.8820000000000001,"15min":3.8820000000000001},"last":1.2250000000000001},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.181,"15min":1.171},"min":{"1min":0.83799999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4510000000000001,"5min":1.6559999999999999,"15min":4.6189999999999998},"last":1.083}]},{"osd":4,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.2749999999999999,"15min":1.2529999999999999},"min":{"1min":0.90200000000000002,"5min":0.65600000000000003,"15min":0.65600000000000003},"max":{"1min":1.621,"5min":1.9670000000000001,"15min":3.0680000000000001},"last":1.476},{"interface":"front","average":{"1min":1.3879999999999999,"5min":1.415,"15min":1.353},"min":{"1min":0.95999999999999996,"5min":0.70799999999999996,"15min":0.57999999999999996},"max":{"1min":1.669,"5min":3.7000000000000002,"15min":3.7000000000000002},"last":1.3999999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2609999999999999,"15min":1.1970000000000001},"min":{"1min":0.82999999999999996,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.734,"5min":2.2400000000000002,"15min":4.6680000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.385,"5min":1.419,"15min":1.361},"min":{"1min":1.145,"5min":0.626,"15min":0.626},"max":{"1min":1.5549999999999999,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.3600000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.425,"15min":1.3520000000000001},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.61899999999999999},"max":{"1min":1.702,"5min":3.9670000000000001,"15min":3.9670000000000001},"last":1.3109999999999999},{"interface":"front","average":{"1min":1.423,"5min":1.409,"15min":1.365},"min":{"1min":1.0189999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.6319999999999999,"5min":2.3519999999999999,"15min":5.6639999999999997},"last":1.4339999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574951,"num_pgs":20,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1409,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193903,"statfs":{"total":200000000,"available":198557150,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.2130000000000001,"15min":1.1639999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.863},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.228,"5min":1.27,"15min":1.194},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.913,"5min":1.913,"15min":1.913},"last":1.2669999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2350000000000001,"5min":1.222,"15min":1.079},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.65700000000000003},"max":{"1min":1.7929999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.179,"15min":1.0800000000000001},"min":{"1min":0.71899999999999997,"5min":0.71299999999999997,"15min":0.61299999999999999},"max":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.843},"last":1.3380000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.155,"15min":1.093},"min":{"1min":0.745,"5min":0.74299999999999999,"15min":0.59499999999999997},"max":{"1min":1.5580000000000001,"5min":1.728,"15min":1.794},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.284,"5min":1.25,"15min":1.1599999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.65200000000000002},"max":{"1min":1.6779999999999999,"5min":1.768,"15min":2.036},"last":1.304}]},{"osd":3,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.149,"5min":1.1930000000000001,"15min":1.105},"min":{"1min":0.78600000000000003,"5min":0.71999999999999997,"15min":0.68899999999999995},"max":{"1min":1.6319999999999999,"5min":1.72,"15min":1.7450000000000001},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.337,"15min":1.244},"min":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.70399999999999996},"max":{"1min":1.9590000000000001,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.1830000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.367,"5min":1.3049999999999999,"15min":1.198},"min":{"1min":0.93400000000000005,"5min":0.60899999999999999,"15min":0.46700000000000003},"max":{"1min":1.944,"5min":1.944,"15min":2.0329999999999999},"last":1.1120000000000001},{"interface":"front","average":{"1min":1.2729999999999999,"5min":1.218,"15min":1.143},"min":{"1min":0.97299999999999998,"5min":0.75900000000000001,"15min":0.57099999999999995},"max":{"1min":1.764,"5min":1.804,"15min":1.901},"last":1.228}]},{"osd":6,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.6060000000000001,"5min":1.456,"15min":1.359},"min":{"1min":1.347,"5min":0.66600000000000004,"15min":0.54400000000000004},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.1589999999999998},"last":1.357},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3320000000000001,"15min":1.2569999999999999},"min":{"1min":0.93999999999999995,"5min":0.81200000000000006,"15min":0.49299999999999999},"max":{"1min":1.585,"5min":1.702,"15min":1.976},"last":1.45}]},{"osd":7,"last update":"Tue Dec 19 18:23:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3140000000000001,"15min":1.242},"min":{"1min":0.90000000000000002,"5min":0.68300000000000005,"15min":0.50900000000000001},"max":{"1min":1.863,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.399},{"interface":"front","average":{"1min":1.53,"5min":1.4790000000000001,"15min":1.4219999999999999},"min":{"1min":1.1559999999999999,"5min":0.86099999999999999,"15min":0.623},"max":{"1min":1.8620000000000001,"5min":2.117,"15min":2.117},"last":1.3700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574951,"num_pgs":33,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":834,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194478,"statfs":{"total":200000000,"available":199145956,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.998,"15min":1},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.45,"5min":1.532,"15min":1.579},"last":1.133},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.0720000000000001,"15min":1.042},"min":{"1min":0.68999999999999995,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.5680000000000001,"5min":1.7470000000000001,"15min":1.7909999999999999},"last":1.7330000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.153,"15min":1.1479999999999999},"min":{"1min":0.748,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.8160000000000001},"last":1.476},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.242,"15min":1.1850000000000001},"min":{"1min":0.69299999999999995,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.875,"5min":1.875,"15min":1.875},"last":1.3720000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.1279999999999999,"15min":1.123},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.484,"5min":1.627,"15min":1.7010000000000001},"last":1.284},{"interface":"front","average":{"1min":1.143,"5min":1.1559999999999999,"15min":1.1100000000000001},"min":{"1min":0.77700000000000002,"5min":0.72199999999999998,"15min":0.70499999999999996},"max":{"1min":1.633,"5min":1.675,"15min":1.675},"last":1.3400000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.29,"15min":1.264},"min":{"1min":0.877,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.946,"5min":1.946,"15min":1.946},"last":1.5069999999999999},{"interface":"front","average":{"1min":1.329,"5min":1.351,"15min":1.3069999999999999},"min":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.74399999999999999},"max":{"1min":1.5780000000000001,"5min":1.8420000000000001,"15min":1.8420000000000001},"last":1.849}]},{"osd":5,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.29,"5min":1.2869999999999999,"15min":1.212},"min":{"1min":0.82299999999999995,"5min":0.68600000000000005,"15min":0.44500000000000001},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8740000000000001},"last":1.157},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.395,"15min":1.353},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.504},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":0.93400000000000005}]},{"osd":6,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.323,"15min":1.3029999999999999},"min":{"1min":1.008,"5min":0.71299999999999997,"15min":0.432},"max":{"1min":1.6870000000000001,"5min":1.6870000000000001,"15min":1.853},"last":1.5669999999999999},{"interface":"front","average":{"1min":1.409,"5min":1.4590000000000001,"15min":1.3999999999999999},"min":{"1min":0.90800000000000003,"5min":0.73599999999999999,"15min":0.64500000000000002},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.7749999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:23:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3200000000000001,"5min":1.379,"15min":1.349},"min":{"1min":0.66000000000000003,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.4419999999999999,"15min":1.4099999999999999},"min":{"1min":1.0880000000000001,"5min":0.60399999999999998,"15min":0.57399999999999995},"max":{"1min":1.8109999999999999,"5min":1.905,"15min":1.9330000000000001},"last":1.427}]}]},{"osd":3,"up_from":13,"seq":55834574952,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":832,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194480,"statfs":{"total":200000000,"available":199147610,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.92500000000000004,"15min":0.95699999999999996},"min":{"1min":0.45200000000000001,"5min":0.38100000000000001,"15min":0.35299999999999998},"max":{"1min":1.5469999999999999,"5min":3.3140000000000001,"15min":3.4350000000000001},"last":0.435},{"interface":"front","average":{"1min":1.032,"5min":1.01,"15min":0.95199999999999996},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.7150000000000001,"5min":4.3490000000000002,"15min":4.3490000000000002},"last":0.65900000000000003}]},{"osd":1,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":1.0209999999999999,"15min":1.05},"min":{"1min":0.66900000000000004,"5min":0.41499999999999998,"15min":0.41299999999999998},"max":{"1min":1.405,"5min":5.2510000000000003,"15min":5.2510000000000003},"last":0.624},{"interface":"front","average":{"1min":1.032,"5min":1.1519999999999999,"15min":1.111},"min":{"1min":0.53200000000000003,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.6319999999999999,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":0.69499999999999995}]},{"osd":2,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.0940000000000001,"5min":1.21,"15min":1.242},"min":{"1min":0.61299999999999999,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.417,"5min":2.5369999999999999,"15min":13.833},"last":0.72499999999999998},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.3660000000000001,"15min":1.3120000000000001},"min":{"1min":0.79700000000000004,"5min":0.57999999999999996,"15min":0.53200000000000003},"max":{"1min":1.7509999999999999,"5min":5.4480000000000004,"15min":16.748999999999999},"last":0.93899999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3420000000000001,"15min":1.2410000000000001},"min":{"1min":0.87,"5min":0.77400000000000002,"15min":0.65600000000000003},"max":{"1min":1.8140000000000001,"5min":5.1790000000000003,"15min":5.1790000000000003},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.393,"5min":1.4099999999999999,"15min":1.407},"min":{"1min":1.0309999999999999,"5min":0.80800000000000005,"15min":0.73199999999999998},"max":{"1min":1.798,"5min":4.569,"15min":4.569},"last":0.97399999999999998}]},{"osd":5,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.4490000000000001,"15min":1.444},"min":{"1min":0.95499999999999996,"5min":0.82399999999999995,"15min":0.76000000000000001},"max":{"1min":1.901,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.3700000000000001,"5min":1.371,"15min":1.288},"min":{"1min":0.81899999999999995,"5min":0.80600000000000005,"15min":0.69999999999999996},"max":{"1min":1.9470000000000001,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.133}]},{"osd":6,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.4119999999999999,"5min":1.446,"15min":1.47},"min":{"1min":0.97599999999999998,"5min":0.81799999999999995,"15min":0.81599999999999995},"max":{"1min":1.841,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.536,"15min":1.5129999999999999},"min":{"1min":0.92800000000000005,"5min":0.83199999999999996,"15min":0.80500000000000005},"max":{"1min":1.9970000000000001,"5min":5.3739999999999997,"15min":5.3739999999999997},"last":0.999}]},{"osd":7,"last update":"Tue Dec 19 18:22:52 2023","interfaces":[{"interface":"back","average":{"1min":1.498,"5min":1.4750000000000001,"15min":1.395},"min":{"1min":0.89700000000000002,"5min":0.78500000000000003,"15min":0.73199999999999998},"max":{"1min":2.0720000000000001,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.22},{"interface":"front","average":{"1min":1.54,"5min":1.5640000000000001,"15min":1.5529999999999999},"min":{"1min":0.98099999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.0329999999999999,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":1.1020000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574951,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":833,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194479,"statfs":{"total":200000000,"available":199146972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.88200000000000001,"15min":0.997},"min":{"1min":0.47199999999999998,"5min":0.35299999999999998,"15min":0.34399999999999997},"max":{"1min":1.137,"5min":2.879,"15min":9.2550000000000008},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.0049999999999999,"15min":1.0289999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.361,"5min":2.387,"15min":4.7850000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.94899999999999995,"15min":1.137},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.46000000000000002},"max":{"1min":1.3340000000000001,"5min":2.4390000000000001,"15min":8.2720000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.093,"15min":1.151},"min":{"1min":0.67500000000000004,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.802,"5min":2.7200000000000002,"15min":5.0620000000000003},"last":0.745}]},{"osd":3,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.98299999999999998,"5min":1.133,"15min":1.327},"min":{"1min":0.66000000000000003,"5min":0.56000000000000005,"15min":0.51900000000000002},"max":{"1min":1.5429999999999999,"5min":2.238,"15min":8.3109999999999999},"last":1.038},{"interface":"front","average":{"1min":0.97399999999999998,"5min":1.117,"15min":1.2949999999999999},"min":{"1min":0.60299999999999998,"5min":0.46300000000000002,"15min":0.41799999999999998},"max":{"1min":1.4490000000000001,"5min":2.323,"15min":8.0690000000000008},"last":0.61699999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3220000000000001,"5min":1.391,"15min":1.448},"min":{"1min":1.0760000000000001,"5min":0.83399999999999996,"15min":0.79300000000000004},"max":{"1min":1.7,"5min":2.6139999999999999,"15min":5.3929999999999998},"last":0.98299999999999998},{"interface":"front","average":{"1min":1.3280000000000001,"5min":1.454,"15min":1.5289999999999999},"min":{"1min":0.85399999999999998,"5min":0.82099999999999995,"15min":0.78300000000000003},"max":{"1min":2.1179999999999999,"5min":3.1070000000000002,"15min":7.3570000000000002},"last":1.1599999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.363,"15min":1.522},"min":{"1min":0.92800000000000005,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":1.7330000000000001,"5min":2.5579999999999998,"15min":8.1839999999999993},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.397,"15min":1.409},"min":{"1min":0.82299999999999995,"5min":0.81000000000000005,"15min":0.72199999999999998},"max":{"1min":1.9339999999999999,"5min":2.4940000000000002,"15min":5.165},"last":0.89200000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.411,"15min":1.4430000000000001},"min":{"1min":0.92500000000000004,"5min":0.752,"15min":0.72999999999999998},"max":{"1min":1.5680000000000001,"5min":3.0179999999999998,"15min":5.1059999999999999},"last":1.2050000000000001},{"interface":"front","average":{"1min":1.411,"5min":1.4419999999999999,"15min":1.5449999999999999},"min":{"1min":0.98499999999999999,"5min":0.81200000000000006,"15min":0.77000000000000002},"max":{"1min":1.915,"5min":2.9580000000000002,"15min":8.1370000000000005},"last":1.246}]},{"osd":7,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.482,"15min":1.534},"min":{"1min":0.93500000000000005,"5min":0.79200000000000004,"15min":0.78200000000000003},"max":{"1min":1.714,"5min":3.2189999999999999,"15min":5.1449999999999996},"last":1.232},{"interface":"front","average":{"1min":1.377,"5min":1.5049999999999999,"15min":1.635},"min":{"1min":0.88500000000000001,"5min":0.878,"15min":0.86499999999999999},"max":{"1min":1.8600000000000001,"5min":4.8289999999999997,"15min":8.2300000000000004},"last":1.4970000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574952,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1402,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193909,"statfs":{"total":200000000,"available":198563542,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.85999999999999999,"15min":0.84799999999999998},"min":{"1min":0.54400000000000004,"5min":0.38,"15min":0.34699999999999998},"max":{"1min":1.4610000000000001,"5min":1.52,"15min":1.52},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.94699999999999995,"15min":0.95299999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.5820000000000001,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":0.434}]},{"osd":2,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":0.92800000000000005,"5min":1.0529999999999999,"15min":1.052},"min":{"1min":0.56399999999999995,"5min":0.42299999999999999,"15min":0.41399999999999998},"max":{"1min":1.6120000000000001,"5min":2.4590000000000001,"15min":2.5739999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":1.016,"15min":0.93300000000000005},"min":{"1min":0.52200000000000002,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":1.1699999999999999,"5min":1.845,"15min":1.845},"last":0.626}]},{"osd":3,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0700000000000001,"5min":1.292,"15min":1.1699999999999999},"min":{"1min":0.59599999999999997,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.613,"5min":2.137,"15min":2.137},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.0449999999999999,"5min":1.196,"15min":1.1839999999999999},"min":{"1min":0.63600000000000001,"5min":0.56100000000000005,"15min":0.44900000000000001},"max":{"1min":1.6200000000000001,"5min":2.1320000000000001,"15min":2.3959999999999999},"last":0.80900000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3440000000000001,"15min":1.2989999999999999},"min":{"1min":0.76400000000000001,"5min":0.72199999999999998,"15min":0.69099999999999995},"max":{"1min":1.696,"5min":2.444,"15min":2.444},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.3859999999999999,"15min":1.3089999999999999},"min":{"1min":0.95599999999999996,"5min":0.76900000000000002,"15min":0.71099999999999997},"max":{"1min":1.5369999999999999,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.3300000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.379,"15min":1.3049999999999999},"min":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.506,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.304,"5min":1.476,"15min":1.4179999999999999},"min":{"1min":0.96399999999999997,"5min":0.70499999999999996,"15min":0.70499999999999996},"max":{"1min":1.76,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.1279999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.417,"15min":1.381},"min":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.76900000000000002},"max":{"1min":1.714,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.355,"5min":1.5169999999999999,"15min":1.456},"min":{"1min":1.014,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.8200000000000001,"5min":2.895,"15min":2.895},"last":1.2829999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:22:56 2023","interfaces":[{"interface":"back","average":{"1min":1.302,"5min":1.4570000000000001,"15min":1.397},"min":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.69999999999999996},"max":{"1min":1.6850000000000001,"5min":2.5670000000000002,"15min":9.4830000000000005},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.528,"15min":1.502},"min":{"1min":0.874,"5min":0.874,"15min":0.71899999999999997},"max":{"1min":1.7,"5min":2.456,"15min":9.4109999999999996},"last":1.3819999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574951,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":833,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194479,"statfs":{"total":200000000,"available":199146958,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.95599999999999996,"15min":0.89800000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.1870000000000001,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":1.0600000000000001},{"interface":"front","average":{"1min":0.84599999999999997,"5min":1.1040000000000001,"15min":1.0329999999999999},"min":{"1min":0.438,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.3460000000000001,"5min":2.1040000000000001,"15min":2.4180000000000001},"last":1.3080000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":1.0189999999999999,"15min":0.95999999999999996},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.4219999999999999,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":1.4550000000000001},{"interface":"front","average":{"1min":0.83299999999999996,"5min":1.0009999999999999,"15min":0.94299999999999995},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.585,"5min":2.0950000000000002,"15min":2.7810000000000001},"last":1.1719999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":1.073,"15min":1.012},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.6719999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.44},{"interface":"front","average":{"1min":1,"5min":1.306,"15min":1.226},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.47499999999999998},"max":{"1min":1.603,"5min":2.0089999999999999,"15min":2.0089999999999999},"last":0.83199999999999996}]},{"osd":4,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.119,"5min":1.3140000000000001,"15min":1.2150000000000001},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.7569999999999999,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.409},{"interface":"front","average":{"1min":1.0940000000000001,"5min":1.3089999999999999,"15min":1.2809999999999999},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.6160000000000001,"5min":2.9529999999999998,"15min":2.9529999999999998},"last":0.90500000000000003}]},{"osd":5,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.3640000000000001,"15min":1.393},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.64500000000000002},"max":{"1min":1.806,"5min":2.3940000000000001,"15min":16.298999999999999},"last":1.3640000000000001},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.3600000000000001,"15min":1.286},"min":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72199999999999998},"max":{"1min":1.696,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":1.6060000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.478,"15min":1.4019999999999999},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.776,"5min":2.2559999999999998,"15min":2.5910000000000002},"last":1.7},{"interface":"front","average":{"1min":1.2,"5min":1.4219999999999999,"15min":1.3620000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.77000000000000002},"max":{"1min":1.746,"5min":2.4359999999999999,"15min":2.46},"last":1.502}]},{"osd":7,"last update":"Tue Dec 19 18:23:03 2023","interfaces":[{"interface":"back","average":{"1min":1.218,"5min":1.488,"15min":1.4019999999999999},"min":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.70899999999999996},"max":{"1min":1.609,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.55},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.4379999999999999,"15min":1.421},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":1.7050000000000001,"5min":2.4580000000000002,"15min":3.1429999999999998},"last":0.98799999999999999}]}]}],"pool_statfs":[]}} 2023-12-19T18:23:12.039 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:23:12.039 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:23:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.333+0000 7fd911887640 1 Processor -- start 2023-12-19T18:23:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.333+0000 7fd911887640 1 -- start start 2023-12-19T18:23:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.335+0000 7fd911887640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd90c15d160 0x7fd90c0cc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.335+0000 7fd911887640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90c0ccdb0 0x7fd90c0c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.335+0000 7fd911887640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd90c0cd210 con 0x7fd90c0c9570 2023-12-19T18:23:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.335+0000 7fd911887640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd90c06f360 con 0x7fd90c15d160 2023-12-19T18:23:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.335+0000 7fd911887640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0c9da0 con 0x7fd90c0ccdb0 2023-12-19T18:23:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.336+0000 7fd910885640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd90c15d160 0x7fd90c0cc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.336+0000 7fd911086640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd90c0c9570 0x7fd90c0ac8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59402/0 (socket says 172.21.15.27:59402) 2023-12-19T18:23:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.336+0000 7fd911086640 1 -- 172.21.15.27:0/3605322743 learned_addr learned my addr 172.21.15.27:0/3605322743 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.336+0000 7fd903fff640 1 --2- 172.21.15.27:0/3605322743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90c0ccdb0 0x7fd90c0c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.336+0000 7fd910885640 1 -- 172.21.15.27:0/3605322743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90c0ccdb0 msgr2=0x7fd90c0c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.336+0000 7fd910885640 1 --2- 172.21.15.27:0/3605322743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90c0ccdb0 0x7fd90c0c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.336+0000 7fd910885640 1 -- 172.21.15.27:0/3605322743 >> v1:172.21.15.27:6789/0 conn(0x7fd90c0c9570 legacy=0x7fd90c0ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.336+0000 7fd910885640 1 -- 172.21.15.27:0/3605322743 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd90c0ca080 con 0x7fd90c15d160 2023-12-19T18:23:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.336+0000 7fd903fff640 1 --2- 172.21.15.27:0/3605322743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90c0ccdb0 0x7fd90c0c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:23:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.338+0000 7fd910885640 1 --2- 172.21.15.27:0/3605322743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c15d160 0x7fd90c0cc870 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fd9080097a0 tx=0x7fd90802b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61eaf4f189440efe server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.338+0000 7fd9037fe640 1 -- 172.21.15.27:0/3605322743 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90802d040 con 0x7fd90c15d160 2023-12-19T18:23:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.338+0000 7fd9037fe640 1 -- 172.21.15.27:0/3605322743 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd908031810 con 0x7fd90c15d160 2023-12-19T18:23:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.338+0000 7fd9037fe640 1 -- 172.21.15.27:0/3605322743 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd908031ba0 con 0x7fd90c15d160 2023-12-19T18:23:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.339+0000 7fd911887640 1 -- 172.21.15.27:0/3605322743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c15d160 msgr2=0x7fd90c0cc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.339+0000 7fd911887640 1 --2- 172.21.15.27:0/3605322743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c15d160 0x7fd90c0cc870 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fd9080097a0 tx=0x7fd90802b8b0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.339+0000 7fd911887640 1 -- 172.21.15.27:0/3605322743 shutdown_connections 2023-12-19T18:23:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.339+0000 7fd911887640 1 --2- 172.21.15.27:0/3605322743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90c0ccdb0 0x7fd90c0c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.339+0000 7fd911887640 1 --2- 172.21.15.27:0/3605322743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c15d160 0x7fd90c0cc870 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.339+0000 7fd911887640 1 -- 172.21.15.27:0/3605322743 >> 172.21.15.27:0/3605322743 conn(0x7fd90c0cb3f0 msgr2=0x7fd90c0cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.339+0000 7fd911887640 1 -- 172.21.15.27:0/3605322743 shutdown_connections 2023-12-19T18:23:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.339+0000 7fd911887640 1 -- 172.21.15.27:0/3605322743 wait complete. 2023-12-19T18:23:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.341+0000 7fd911887640 1 Processor -- start 2023-12-19T18:23:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.341+0000 7fd911887640 1 -- start start 2023-12-19T18:23:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd911887640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c0c9570 0x7fd90c153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd911887640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90c0ccdb0 0x7fd90c06c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd911887640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90c06c850 0x7fd90c15fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd911887640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0b3350 con 0x7fd90c0c9570 2023-12-19T18:23:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd911887640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c06cd00 con 0x7fd90c06c850 2023-12-19T18:23:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd911887640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd90c0b3770 con 0x7fd90c0ccdb0 2023-12-19T18:23:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd910885640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c0c9570 0x7fd90c153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd910885640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c0c9570 0x7fd90c153dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54272/0 (socket says 172.21.15.27:54272) 2023-12-19T18:23:12.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd910885640 1 -- 172.21.15.27:0/3906647907 learned_addr learned my addr 172.21.15.27:0/3906647907 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:12.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd903fff640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90c0ccdb0 0x7fd90c06c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:12.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd910885640 1 -- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90c0ccdb0 msgr2=0x7fd90c06c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:12.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd910885640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90c0ccdb0 0x7fd90c06c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd910885640 1 -- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90c06c850 msgr2=0x7fd90c15fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:12.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.342+0000 7fd911086640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90c06c850 0x7fd90c15fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:12.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.343+0000 7fd910885640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90c06c850 0x7fd90c15fa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.343+0000 7fd910885640 1 -- 172.21.15.27:0/3906647907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd908009400 con 0x7fd90c0c9570 2023-12-19T18:23:12.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.343+0000 7fd910885640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c0c9570 0x7fd90c153dd0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fd9080097a0 tx=0x7fd9080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:12.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.343+0000 7fd9017fa640 1 -- 172.21.15.27:0/3906647907 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90803ad60 con 0x7fd90c0c9570 2023-12-19T18:23:12.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.343+0000 7fd9017fa640 1 -- 172.21.15.27:0/3906647907 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9080443f0 con 0x7fd90c0c9570 2023-12-19T18:23:12.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.343+0000 7fd9017fa640 1 -- 172.21.15.27:0/3906647907 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9080447a0 con 0x7fd90c0c9570 2023-12-19T18:23:12.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.343+0000 7fd911887640 1 -- 172.21.15.27:0/3906647907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd90c0b26d0 con 0x7fd90c0c9570 2023-12-19T18:23:12.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.343+0000 7fd911887640 1 -- 172.21.15.27:0/3906647907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd90c0b2c90 con 0x7fd90c0c9570 2023-12-19T18:23:12.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.344+0000 7fd911887640 1 -- 172.21.15.27:0/3906647907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90c0b3490 con 0x7fd90c0c9570 2023-12-19T18:23:12.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.346+0000 7fd9017fa640 1 -- 172.21.15.27:0/3906647907 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd90804d450 con 0x7fd90c0c9570 2023-12-19T18:23:12.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.347+0000 7fd9017fa640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd8f806a430 0x7fd8f806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:12.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.347+0000 7fd9017fa640 1 -- 172.21.15.27:0/3906647907 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9080aabf0 con 0x7fd90c0c9570 2023-12-19T18:23:12.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.347+0000 7fd903fff640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd8f806a430 0x7fd8f806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:12.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.348+0000 7fd903fff640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd8f806a430 0x7fd8f806c8f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd90c15b6e0 tx=0x7fd8fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:12.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.353+0000 7fd9017fa640 1 -- 172.21.15.27:0/3906647907 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd90c0b3490 con 0x7fd90c0c9570 2023-12-19T18:23:12.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.618+0000 7fd911887640 1 -- 172.21.15.27:0/3906647907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd90c0b3490 con 0x7fd90c0c9570 2023-12-19T18:23:12.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.623+0000 7fd9017fa640 1 -- 172.21.15.27:0/3906647907 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7fd90c0b3490 con 0x7fd90c0c9570 2023-12-19T18:23:12.623 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2023-12-19T18:23:12.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.626+0000 7fd911887640 1 -- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd8f806a430 msgr2=0x7fd8f806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:12.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.626+0000 7fd911887640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd8f806a430 0x7fd8f806c8f0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fd90c15b6e0 tx=0x7fd8fc008040 comp rx=0 tx=0).stop 2023-12-19T18:23:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.626+0000 7fd911887640 1 -- 172.21.15.27:0/3906647907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c0c9570 msgr2=0x7fd90c153dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.626+0000 7fd911887640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c0c9570 0x7fd90c153dd0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fd9080097a0 tx=0x7fd9080046c0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.626+0000 7fd911887640 1 -- 172.21.15.27:0/3906647907 shutdown_connections 2023-12-19T18:23:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.626+0000 7fd911887640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd8f806a430 0x7fd8f806c8f0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.626+0000 7fd911887640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd90c06c850 0x7fd90c15fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.626+0000 7fd911887640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd90c0ccdb0 0x7fd90c06c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.626+0000 7fd911887640 1 --2- 172.21.15.27:0/3906647907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd90c0c9570 0x7fd90c153dd0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.626+0000 7fd911887640 1 -- 172.21.15.27:0/3906647907 >> 172.21.15.27:0/3906647907 conn(0x7fd90c0cb3f0 msgr2=0x7fd90c0c7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.626+0000 7fd911887640 1 -- 172.21.15.27:0/3906647907 shutdown_connections 2023-12-19T18:23:12.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.626+0000 7fd911887640 1 -- 172.21.15.27:0/3906647907 wait complete. 2023-12-19T18:23:12.647 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:23:12.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.879+0000 7f9cf09c6640 1 Processor -- start 2023-12-19T18:23:12.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.879+0000 7f9cf09c6640 1 -- start start 2023-12-19T18:23:12.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.881+0000 7f9cf09c6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9cec154030 0x7f9cec0cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:12.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.881+0000 7f9cf09c6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cec15e440 con 0x7f9cec0af630 2023-12-19T18:23:12.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.881+0000 7f9cf09c6640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cec0cc190 con 0x7f9cec0c5560 2023-12-19T18:23:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.881+0000 7f9cf09c6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cec0cb2a0 con 0x7f9cec154030 2023-12-19T18:23:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.881+0000 7f9ceb7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9cec154030 0x7f9cec0cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.881+0000 7f9ceaffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9cec0af630 0x7f9cec15d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59410/0 (socket says 172.21.15.27:59410) 2023-12-19T18:23:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.881+0000 7f9ceaffd640 1 -- 172.21.15.27:0/819927965 learned_addr learned my addr 172.21.15.27:0/819927965 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.882+0000 7f9ceb7fe640 1 -- 172.21.15.27:0/819927965 >> v1:172.21.15.192:6789/0 conn(0x7f9cec0c5560 legacy=0x7f9cec15dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.882+0000 7f9ceb7fe640 1 -- 172.21.15.27:0/819927965 >> v1:172.21.15.27:6789/0 conn(0x7f9cec0af630 legacy=0x7f9cec15d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.882+0000 7f9ceb7fe640 1 -- 172.21.15.27:0/819927965 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cec0cb580 con 0x7f9cec154030 2023-12-19T18:23:12.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.882+0000 7f9cea7fc640 1 -- 172.21.15.27:0/819927965 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805447268 0 0) 0x7f9cec15e440 con 0x7f9cec0af630 2023-12-19T18:23:12.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.883+0000 7f9ceb7fe640 1 --2- 172.21.15.27:0/819927965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cec154030 0x7f9cec0cd4f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0097d0 tx=0x7f9cdc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd0819a9c98da86 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:12.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.883+0000 7f9cea7fc640 1 -- 172.21.15.27:0/819927965 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc031830 con 0x7f9cec154030 2023-12-19T18:23:12.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.883+0000 7f9cea7fc640 1 -- 172.21.15.27:0/819927965 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cdc031990 con 0x7f9cec154030 2023-12-19T18:23:12.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.883+0000 7f9cea7fc640 1 -- 172.21.15.27:0/819927965 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cdc031d20 con 0x7f9cec154030 2023-12-19T18:23:12.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.884+0000 7f9cf09c6640 1 -- 172.21.15.27:0/819927965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cec154030 msgr2=0x7f9cec0cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:12.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.884+0000 7f9cf09c6640 1 --2- 172.21.15.27:0/819927965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cec154030 0x7f9cec0cd4f0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc0097d0 tx=0x7f9cdc02b8d0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.884+0000 7f9cf09c6640 1 -- 172.21.15.27:0/819927965 shutdown_connections 2023-12-19T18:23:12.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.884+0000 7f9cf09c6640 1 --2- 172.21.15.27:0/819927965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cec154030 0x7f9cec0cd4f0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.884+0000 7f9cf09c6640 1 -- 172.21.15.27:0/819927965 >> 172.21.15.27:0/819927965 conn(0x7f9cec069c70 msgr2=0x7f9cec06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:12.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.884+0000 7f9cf09c6640 1 -- 172.21.15.27:0/819927965 shutdown_connections 2023-12-19T18:23:12.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.884+0000 7f9cf09c6640 1 -- 172.21.15.27:0/819927965 wait complete. 2023-12-19T18:23:12.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.886+0000 7f9cf09c6640 1 Processor -- start 2023-12-19T18:23:12.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.886+0000 7f9cf09c6640 1 -- start start 2023-12-19T18:23:12.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cf09c6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9cec0af630 0x7f9cec14c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:12.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cf09c6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cec0c5560 0x7f9cec141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:12.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cf09c6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cec154030 0x7f9cec141f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:12.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cf09c6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec149810 con 0x7f9cec154030 2023-12-19T18:23:12.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cf09c6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec06c460 con 0x7f9cec0c5560 2023-12-19T18:23:12.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cf09c6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cec06c740 con 0x7f9cec0af630 2023-12-19T18:23:12.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cec154030 0x7f9cec141f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:12.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cec154030 0x7f9cec141f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54282/0 (socket says 172.21.15.27:54282) 2023-12-19T18:23:12.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cebfff640 1 -- 172.21.15.27:0/1367755951 learned_addr learned my addr 172.21.15.27:0/1367755951 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:12.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9ceaffd640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cec0c5560 0x7f9cec141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:12.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9ceb7fe640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9cec0af630 0x7f9cec14c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:12.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cebfff640 1 -- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9cec0af630 msgr2=0x7f9cec14c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:12.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cebfff640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9cec0af630 0x7f9cec14c380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cebfff640 1 -- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cec0c5560 msgr2=0x7f9cec141a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:12.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cebfff640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cec0c5560 0x7f9cec141a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:12.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.887+0000 7f9cebfff640 1 -- 172.21.15.27:0/1367755951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cdc009400 con 0x7f9cec154030 2023-12-19T18:23:12.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.888+0000 7f9ceaffd640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cec0c5560 0x7f9cec141a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:12.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.888+0000 7f9cebfff640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cec154030 0x7f9cec141f60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400c780 tx=0x7f9ce400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:12.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.888+0000 7f9cd3fff640 1 -- 172.21.15.27:0/1367755951 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce4003d80 con 0x7f9cec154030 2023-12-19T18:23:12.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.888+0000 7f9cd3fff640 1 -- 172.21.15.27:0/1367755951 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ce400ce90 con 0x7f9cec154030 2023-12-19T18:23:12.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.888+0000 7f9cd3fff640 1 -- 172.21.15.27:0/1367755951 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ce40052b0 con 0x7f9cec154030 2023-12-19T18:23:12.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.888+0000 7f9cf09c6640 1 -- 172.21.15.27:0/1367755951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cec149a80 con 0x7f9cec154030 2023-12-19T18:23:12.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.888+0000 7f9cf09c6640 1 -- 172.21.15.27:0/1367755951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cec14c8c0 con 0x7f9cec154030 2023-12-19T18:23:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.889+0000 7f9cf09c6640 1 -- 172.21.15.27:0/1367755951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cec149ca0 con 0x7f9cec154030 2023-12-19T18:23:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.890+0000 7f9cd3fff640 1 -- 172.21.15.27:0/1367755951 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9ce4005410 con 0x7f9cec154030 2023-12-19T18:23:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.891+0000 7f9cd3fff640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cd806a430 0x7f9cd806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.892+0000 7f9cd3fff640 1 -- 172.21.15.27:0/1367755951 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ce4087ea0 con 0x7f9cec154030 2023-12-19T18:23:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.892+0000 7f9ceb7fe640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cd806a430 0x7f9cd806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.893+0000 7f9ceb7fe640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cd806a430 0x7f9cd806c8f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc036000 tx=0x7f9cdc008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:12.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:12.897+0000 7f9cd3fff640 1 -- 172.21.15.27:0/1367755951 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9ce40533f0 con 0x7f9cec154030 2023-12-19T18:23:13.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.129+0000 7f9cf09c6640 1 -- 172.21.15.27:0/1367755951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9cec04e840 con 0x7f9cec154030 2023-12-19T18:23:13.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.130+0000 7f9cd3fff640 1 -- 172.21.15.27:0/1367755951 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f9ce4017090 con 0x7f9cec154030 2023-12-19T18:23:13.130 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:13.130 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":61,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3298228307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3298228307},{"type":"v1","addr":"172.21.15.192:6835","nonce":3298228307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5726,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1760483999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1760483999},{"type":"v1","addr":"172.21.15.192:6837","nonce":1760483999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3000637822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3000637822},{"type":"v1","addr":"172.21.15.27:6833","nonce":3000637822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:23:06.465885+0000","modified":"2023-12-19T18:23:09.973228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1593995318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1593995318},{"type":"v1","addr":"172.21.15.27:6835","nonce":1593995318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T18:23:13.131 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 61 2023-12-19T18:23:13.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.132+0000 7f9cf09c6640 1 -- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cd806a430 msgr2=0x7f9cd806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:13.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.132+0000 7f9cf09c6640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cd806a430 0x7f9cd806c8f0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9cdc036000 tx=0x7f9cdc008030 comp rx=0 tx=0).stop 2023-12-19T18:23:13.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.132+0000 7f9cf09c6640 1 -- 172.21.15.27:0/1367755951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cec154030 msgr2=0x7f9cec141f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:13.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.132+0000 7f9cf09c6640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cec154030 0x7f9cec141f60 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7f9ce400c780 tx=0x7f9ce400cc40 comp rx=0 tx=0).stop 2023-12-19T18:23:13.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.133+0000 7f9cf09c6640 1 -- 172.21.15.27:0/1367755951 shutdown_connections 2023-12-19T18:23:13.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.133+0000 7f9cf09c6640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9cd806a430 0x7f9cd806c8f0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.133+0000 7f9cf09c6640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cec154030 0x7f9cec141f60 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.133+0000 7f9cf09c6640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9cec0c5560 0x7f9cec141a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.133+0000 7f9cf09c6640 1 --2- 172.21.15.27:0/1367755951 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9cec0af630 0x7f9cec14c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.133+0000 7f9cf09c6640 1 -- 172.21.15.27:0/1367755951 >> 172.21.15.27:0/1367755951 conn(0x7f9cec069c70 msgr2=0x7f9cec0cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:13.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.133+0000 7f9cf09c6640 1 -- 172.21.15.27:0/1367755951 shutdown_connections 2023-12-19T18:23:13.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.133+0000 7f9cf09c6640 1 -- 172.21.15.27:0/1367755951 wait complete. 2023-12-19T18:23:13.554 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 61, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:23:06.465885+0000', 'modified': '2023-12-19T18:23:09.973228+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7794}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7794': {'gid': 7794, 'name': 'a', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/1593995318', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6834', 'nonce': 1593995318}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 1593995318}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:23:13.555 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:23:13.555 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-19T18:23:13.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.801+0000 7f900b59e640 1 Processor -- start 2023-12-19T18:23:13.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.801+0000 7f900b59e640 1 -- start start 2023-12-19T18:23:13.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.802+0000 7f900b59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f900c0b89b0 0x7f900c15c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.802+0000 7f900b59e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900c0b5ca0 con 0x7f900c0b49d0 2023-12-19T18:23:13.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.802+0000 7f900b59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900c161800 con 0x7f900c0b4db0 2023-12-19T18:23:13.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.802+0000 7f900b59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f900c161050 con 0x7f900c0b89b0 2023-12-19T18:23:13.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.802+0000 7f900a59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f900c0b49d0 0x7f900c160940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59424/0 (socket says 172.21.15.27:59424) 2023-12-19T18:23:13.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.802+0000 7f900a59c640 1 -- 172.21.15.27:0/3706748047 learned_addr learned my addr 172.21.15.27:0/3706748047 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:13.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 111853171 0 0) 0x7f900c0b5ca0 con 0x7f900c0b49d0 2023-12-19T18:23:13.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ff8003520 con 0x7f900c0b49d0 2023-12-19T18:23:13.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f9009d9b640 1 --2- 172.21.15.27:0/3706748047 >> v2:172.21.15.192:3300/0 conn(0x7f900c0b89b0 0x7f900c15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2647845361 0 0) 0x7f8ff8003520 con 0x7f900c0b49d0 2023-12-19T18:23:13.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 >> v2:172.21.15.192:3300/0 conn(0x7f900c0b89b0 msgr2=0x7f900c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 --2- 172.21.15.27:0/3706748047 >> v2:172.21.15.192:3300/0 conn(0x7f900c0b89b0 0x7f900c15c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 >> v1:172.21.15.192:6789/0 conn(0x7f900c0b4db0 legacy=0x7f900c0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c161240 con 0x7f900c0b49d0 2023-12-19T18:23:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f8ffc001b70 con 0x7f900c0b49d0 2023-12-19T18:23:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 >> v1:172.21.15.27:6789/0 conn(0x7f900c0b49d0 legacy=0x7f900c160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 --2- 172.21.15.27:0/3706748047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff80039c0 0x7f8ff8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 --2- 172.21.15.27:0/3706748047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff80047c0 0x7f8ff8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 --2- 172.21.15.27:0/3706748047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff80051d0 0x7f8ff800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff800c0e0 con 0x7f8ff80047c0 2023-12-19T18:23:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff800c220 con 0x7f8ff80051d0 2023-12-19T18:23:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.803+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff800c3e0 con 0x7f8ff80039c0 2023-12-19T18:23:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.804+0000 7f9009d9b640 1 --2- 172.21.15.27:0/3706748047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff80047c0 0x7f8ff8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.804+0000 7f900a59c640 1 --2- 172.21.15.27:0/3706748047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff80039c0 0x7f8ff8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.804+0000 7f900ad9d640 1 --2- 172.21.15.27:0/3706748047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff80051d0 0x7f8ff800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.804+0000 7f9009d9b640 1 -- 172.21.15.27:0/3706748047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff80039c0 msgr2=0x7f8ff8003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.804+0000 7f9009d9b640 1 --2- 172.21.15.27:0/3706748047 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8ff80039c0 0x7f8ff8003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.804+0000 7f9009d9b640 1 -- 172.21.15.27:0/3706748047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff80051d0 msgr2=0x7f8ff800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.804+0000 7f9009d9b640 1 --2- 172.21.15.27:0/3706748047 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff80051d0 0x7f8ff800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.804+0000 7f9009d9b640 1 -- 172.21.15.27:0/3706748047 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff8003520 con 0x7f8ff80047c0 2023-12-19T18:23:13.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.804+0000 7f900ad9d640 1 -- 172.21.15.27:0/3706748047 reap_dead start 2023-12-19T18:23:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.805+0000 7f9009d9b640 1 --2- 172.21.15.27:0/3706748047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff80047c0 0x7f8ff8004c40 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400ba30 tx=0x7f8ff80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ae9a3ec566ccbb7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.806+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4032640 con 0x7f8ff80047c0 2023-12-19T18:23:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.806+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff40327a0 con 0x7f8ff80047c0 2023-12-19T18:23:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.806+0000 7f900959a640 1 -- 172.21.15.27:0/3706748047 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4032b30 con 0x7f8ff80047c0 2023-12-19T18:23:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.806+0000 7f900b59e640 1 -- 172.21.15.27:0/3706748047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff80047c0 msgr2=0x7f8ff8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.806+0000 7f900b59e640 1 --2- 172.21.15.27:0/3706748047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff80047c0 0x7f8ff8004c40 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f8ff400ba30 tx=0x7f8ff80057f0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.806+0000 7f900b59e640 1 -- 172.21.15.27:0/3706748047 shutdown_connections 2023-12-19T18:23:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.806+0000 7f900b59e640 1 --2- 172.21.15.27:0/3706748047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff80047c0 0x7f8ff8004c40 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.806+0000 7f900b59e640 1 -- 172.21.15.27:0/3706748047 >> 172.21.15.27:0/3706748047 conn(0x7f900c0cf760 msgr2=0x7f900c0cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.807+0000 7f900b59e640 1 -- 172.21.15.27:0/3706748047 shutdown_connections 2023-12-19T18:23:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.807+0000 7f900b59e640 1 -- 172.21.15.27:0/3706748047 wait complete. 2023-12-19T18:23:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.808+0000 7f900b59e640 1 Processor -- start 2023-12-19T18:23:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.809+0000 7f900b59e640 1 -- start start 2023-12-19T18:23:13.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.809+0000 7f900b59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff80047c0 0x7f900c156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.809+0000 7f900b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f900c1f6b30 0x7f900c0d13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.809+0000 7f900b59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f900c0d1930 0x7f900c1f2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.810+0000 7f900b59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c1f3860 con 0x7f900c1f6b30 2023-12-19T18:23:13.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.810+0000 7f900b59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900c15f5c0 con 0x7f8ff80047c0 2023-12-19T18:23:13.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.810+0000 7f900b59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900c0d1dc0 con 0x7f900c0d1930 2023-12-19T18:23:13.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.810+0000 7f9009d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f900c1f6b30 0x7f900c0d13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.810+0000 7f9009d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f900c1f6b30 0x7f900c0d13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54306/0 (socket says 172.21.15.27:54306) 2023-12-19T18:23:13.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.810+0000 7f9009d9b640 1 -- 172.21.15.27:0/1400319646 learned_addr learned my addr 172.21.15.27:0/1400319646 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:13.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.810+0000 7f900ad9d640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f900c0d1930 0x7f900c1f2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.810+0000 7f9009d9b640 1 -- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f900c0d1930 msgr2=0x7f900c1f2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.810+0000 7f9009d9b640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f900c0d1930 0x7f900c1f2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.810+0000 7f9009d9b640 1 -- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff80047c0 msgr2=0x7f900c156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.810+0000 7f900a59c640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff80047c0 0x7f900c156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.810+0000 7f9009d9b640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff80047c0 0x7f900c156100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.810+0000 7f9009d9b640 1 -- 172.21.15.27:0/1400319646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f900c1f66a0 con 0x7f900c1f6b30 2023-12-19T18:23:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.811+0000 7f9009d9b640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f900c1f6b30 0x7f900c0d13f0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f8ff402bec0 tx=0x7f8ff4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.811+0000 7f9002ffd640 1 -- 172.21.15.27:0/1400319646 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4004220 con 0x7f900c1f6b30 2023-12-19T18:23:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.811+0000 7f9002ffd640 1 -- 172.21.15.27:0/1400319646 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff40043a0 con 0x7f900c1f6b30 2023-12-19T18:23:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.811+0000 7f9002ffd640 1 -- 172.21.15.27:0/1400319646 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff403ca80 con 0x7f900c1f6b30 2023-12-19T18:23:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.811+0000 7f900b59e640 1 -- 172.21.15.27:0/1400319646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f900c1f3580 con 0x7f900c1f6b30 2023-12-19T18:23:13.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.811+0000 7f900b59e640 1 -- 172.21.15.27:0/1400319646 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c1f6330 con 0x7f900c1f6b30 2023-12-19T18:23:13.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.814+0000 7f9002ffd640 1 -- 172.21.15.27:0/1400319646 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8ff403cbe0 con 0x7f900c1f6b30 2023-12-19T18:23:13.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.814+0000 7f9002ffd640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8fec06a430 0x7f8fec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.815+0000 7f9002ffd640 1 -- 172.21.15.27:0/1400319646 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ff40ab380 con 0x7f900c1f6b30 2023-12-19T18:23:13.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.815+0000 7f900a59c640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8fec06a430 0x7f8fec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.815+0000 7f90137ca740 1 Processor -- start 2023-12-19T18:23:13.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.815+0000 7f90137ca740 1 -- start start 2023-12-19T18:23:13.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.816+0000 7f90137ca740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d4110 0x5633db1d44f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.816+0000 7f90137ca740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1a4090 0x5633db1d1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.816+0000 7f90137ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 0x5633db1d18c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.816+0000 7f90137ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633db17f650 con 0x5633db1a7c90 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.816+0000 7f90137ca740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633db114000 con 0x5633db1a4090 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.816+0000 7f90137ca740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633db111790 con 0x5633db1d4110 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.816+0000 7f900a59c640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8fec06a430 0x7f8fec06c8f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc001550 tx=0x7f8ffc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.817+0000 7f900ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 0x5633db1d18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.817+0000 7f900ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 0x5633db1d18c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54310/0 (socket says 172.21.15.27:54310) 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.817+0000 7f900ad9d640 1 -- 172.21.15.27:0/563726240 learned_addr learned my addr 172.21.15.27:0/563726240 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.817+0000 7f900a59c640 1 --2- 172.21.15.27:0/563726240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d4110 0x5633db1d44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.817+0000 7f9009d9b640 1 --2- 172.21.15.27:0/563726240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1a4090 0x5633db1d1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.817+0000 7f900ad9d640 1 -- 172.21.15.27:0/563726240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d4110 msgr2=0x5633db1d44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.817+0000 7f900ad9d640 1 --2- 172.21.15.27:0/563726240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d4110 0x5633db1d44f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.817+0000 7f900ad9d640 1 -- 172.21.15.27:0/563726240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1a4090 msgr2=0x5633db1d1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:13.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.817+0000 7f900ad9d640 1 --2- 172.21.15.27:0/563726240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1a4090 0x5633db1d1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.817+0000 7f900ad9d640 1 -- 172.21.15.27:0/563726240 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633db0d4710 con 0x5633db1a7c90 2023-12-19T18:23:13.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.817+0000 7f900ad9d640 1 --2- 172.21.15.27:0/563726240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 0x5633db1d18c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f900c0ced70 tx=0x7f9004004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bdbf0b159e936e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:13.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.818+0000 7f8febfff640 1 -- 172.21.15.27:0/563726240 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9004007230 con 0x5633db1a7c90 2023-12-19T18:23:13.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.818+0000 7f8febfff640 1 -- 172.21.15.27:0/563726240 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900400d040 con 0x5633db1a7c90 2023-12-19T18:23:13.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.818+0000 7f8febfff640 1 -- 172.21.15.27:0/563726240 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040075f0 con 0x5633db1a7c90 2023-12-19T18:23:13.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.818+0000 7f90137ca740 1 -- 172.21.15.27:0/563726240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 msgr2=0x5633db1d18c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:13.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.818+0000 7f90137ca740 1 --2- 172.21.15.27:0/563726240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 0x5633db1d18c0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f900c0ced70 tx=0x7f9004004040 comp rx=0 tx=0).stop 2023-12-19T18:23:13.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.818+0000 7f90137ca740 1 -- 172.21.15.27:0/563726240 shutdown_connections 2023-12-19T18:23:13.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.818+0000 7f90137ca740 1 --2- 172.21.15.27:0/563726240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 0x5633db1d18c0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.818+0000 7f90137ca740 1 --2- 172.21.15.27:0/563726240 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1a4090 0x5633db1d1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.818+0000 7f90137ca740 1 --2- 172.21.15.27:0/563726240 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d4110 0x5633db1d44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.818+0000 7f90137ca740 1 -- 172.21.15.27:0/563726240 >> 172.21.15.27:0/563726240 conn(0x5633db1a63a0 msgr2=0x5633db1eef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:13.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.819+0000 7f90137ca740 1 -- 172.21.15.27:0/563726240 shutdown_connections 2023-12-19T18:23:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.819+0000 7f90137ca740 1 -- 172.21.15.27:0/563726240 wait complete. 2023-12-19T18:23:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.819+0000 7f90137ca740 1 -- start start 2023-12-19T18:23:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.820+0000 7f90137ca740 1 Processor -- start 2023-12-19T18:23:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.821+0000 7f90137ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 0x5633db1f26c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.821+0000 7f90137ca740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1d4110 0x5633db1e8780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.821+0000 7f90137ca740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d0170 0x5633db1d05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.821+0000 7f90137ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633db10d080 con 0x5633db1a7c90 2023-12-19T18:23:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.821+0000 7f90137ca740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633db0ae850 con 0x5633db1d4110 2023-12-19T18:23:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.821+0000 7f90137ca740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633db0ae4f0 con 0x5633db1d0170 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.821+0000 7f90137ca740 10 client.0 _ll_register_callbacks cb 0x5633db1069c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.821+0000 7f900a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 0x5633db1f26c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.821+0000 7f900a59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 0x5633db1f26c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54318/0 (socket says 172.21.15.27:54318) 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.821+0000 7f900a59c640 1 -- 172.21.15.27:0/4114057633 learned_addr learned my addr 172.21.15.27:0/4114057633 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.821+0000 7f9009d9b640 1 --2- 172.21.15.27:0/4114057633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1d4110 0x5633db1e8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.821+0000 7f900ad9d640 1 --2- 172.21.15.27:0/4114057633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d0170 0x5633db1d05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.822+0000 7f900a59c640 1 -- 172.21.15.27:0/4114057633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d0170 msgr2=0x5633db1d05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.822+0000 7f900a59c640 1 --2- 172.21.15.27:0/4114057633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d0170 0x5633db1d05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.822+0000 7f900a59c640 1 -- 172.21.15.27:0/4114057633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1d4110 msgr2=0x5633db1e8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.822+0000 7f900a59c640 1 --2- 172.21.15.27:0/4114057633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1d4110 0x5633db1e8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.822+0000 7f900a59c640 1 -- 172.21.15.27:0/4114057633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9004003860 con 0x5633db1a7c90 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.822+0000 7f900a59c640 1 --2- 172.21.15.27:0/4114057633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 0x5633db1f26c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0019d0 tx=0x7f8ffc000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fd5f50430686a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.822+0000 7f8fe97fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:23:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.822+0000 7f8fe97fa640 1 -- 172.21.15.27:0/4114057633 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc020830 con 0x5633db1a7c90 2023-12-19T18:23:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.822+0000 7f8fe97fa640 1 -- 172.21.15.27:0/4114057633 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ffc020990 con 0x5633db1a7c90 2023-12-19T18:23:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.822+0000 7f90137ca740 1 -- 172.21.15.27:0/4114057633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5633db1375d0 con 0x5633db1a7c90 2023-12-19T18:23:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.823+0000 7f8fe97fa640 1 -- 172.21.15.27:0/4114057633 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8ffc020b50 con 0x5633db1a7c90 2023-12-19T18:23:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.823+0000 7f90137ca740 10 client.7926 fetch_fsmap learned FSMap version 61 2023-12-19T18:23:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.823+0000 7f90137ca740 1 -- 172.21.15.27:0/4114057633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x5633db10b700 con 0x5633db1a7c90 2023-12-19T18:23:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.823+0000 7f8fe97fa640 1 -- 172.21.15.27:0/4114057633 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5633db1375d0 con 0x5633db1a7c90 2023-12-19T18:23:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.823+0000 7f8fe97fa640 1 -- 172.21.15.27:0/4114057633 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 61) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f8ffc020cb0 con 0x5633db1a7c90 2023-12-19T18:23:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.823+0000 7f90137ca740 10 client.7926 fetch_fsmap finished waiting for FSMap version 61 2023-12-19T18:23:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.823+0000 7f90137ca740 10 client.7926 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T18:23:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.823+0000 7f90137ca740 20 client.7926 populate_metadata read hostname 'smithi027' 2023-12-19T18:23:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.823+0000 7f90137ca740 1 --2- 172.21.15.27:0/4114057633 >> [v2:172.21.15.27:6834/1593995318,v1:172.21.15.27:6835/1593995318] conn(0x5633db1eb170 0x5633db1eb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.823+0000 7f90137ca740 4 client.7926 mds_command: new command op to 7794 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:23:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.823+0000 7f90137ca740 1 -- 172.21.15.27:0/4114057633 --> [v2:172.21.15.27:6834/1593995318,v1:172.21.15.27:6835/1593995318] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5633db0ff8d0 con 0x5633db1eb170 2023-12-19T18:23:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.824+0000 7f9009d9b640 1 --2- 172.21.15.27:0/4114057633 >> [v2:172.21.15.27:6834/1593995318,v1:172.21.15.27:6835/1593995318] conn(0x5633db1eb170 0x5633db1eb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.824+0000 7f9009d9b640 1 --2- 172.21.15.27:0/4114057633 >> [v2:172.21.15.27:6834/1593995318,v1:172.21.15.27:6835/1593995318] conn(0x5633db1eb170 0x5633db1eb550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bda02582437f6eec server_cookie=6e3f87a360c79918 in_seq=0 out_seq=0 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.824+0000 7f8fe97fa640 10 client.7926 ms_handle_connect on v2:172.21.15.27:6834/1593995318 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.825+0000 7f8fe97fa640 1 -- 172.21.15.27:0/4114057633 <== mds.0 v2:172.21.15.27:6834/1593995318 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5633db0ff8d0 con 0x5633db1eb170 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.825+0000 7f8fe97fa640 10 client.7926 handle_command_reply: tid=0 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.825+0000 7f90137ca740 1 client.7926 shutdown 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.825+0000 7f90137ca740 1 -- 172.21.15.27:0/4114057633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 msgr2=0x5633db1f26c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.825+0000 7f90137ca740 1 --2- 172.21.15.27:0/4114057633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 0x5633db1f26c0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0019d0 tx=0x7f8ffc000f40 comp rx=0 tx=0).stop 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.825+0000 7f90137ca740 1 -- 172.21.15.27:0/4114057633 shutdown_connections 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.825+0000 7f90137ca740 1 --2- 172.21.15.27:0/4114057633 >> [v2:172.21.15.27:6834/1593995318,v1:172.21.15.27:6835/1593995318] conn(0x5633db1eb170 0x5633db1eb550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.825+0000 7f90137ca740 1 --2- 172.21.15.27:0/4114057633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d0170 0x5633db1d05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.825+0000 7f90137ca740 1 --2- 172.21.15.27:0/4114057633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1d4110 0x5633db1e8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.825+0000 7f90137ca740 1 --2- 172.21.15.27:0/4114057633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1a7c90 0x5633db1f26c0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.825+0000 7f8fe97fa640 0 client.7926 ms_handle_reset on v2:172.21.15.27:6834/1593995318 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.825+0000 7f90137ca740 1 -- 172.21.15.27:0/4114057633 >> 172.21.15.27:0/4114057633 conn(0x5633db1a4090 msgr2=0x5633db1a8540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.826+0000 7f90137ca740 1 -- 172.21.15.27:0/4114057633 shutdown_connections 2023-12-19T18:23:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.826+0000 7f90137ca740 1 -- 172.21.15.27:0/4114057633 wait complete. 2023-12-19T18:23:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.826+0000 7f90137ca740 20 client.7926 trim_cache size 0 max 16384 2023-12-19T18:23:13.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.842+0000 7f90137ca740 1 Processor -- start 2023-12-19T18:23:13.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.842+0000 7f90137ca740 1 -- start start 2023-12-19T18:23:13.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.843+0000 7f90137ca740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1a7c90 0x5633db1a8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.843+0000 7f90137ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d0170 0x5633db1d0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.843+0000 7f90137ca740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d4110 0x5633db1a3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.843+0000 7f90137ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633dae74f20 con 0x5633db1d0170 2023-12-19T18:23:13.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.843+0000 7f90137ca740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633db1a4b40 con 0x5633db1a7c90 2023-12-19T18:23:13.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.843+0000 7f90137ca740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633db1a8c10 con 0x5633db1d4110 2023-12-19T18:23:13.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.844+0000 7f9009d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d0170 0x5633db1d0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.844+0000 7f9009d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d0170 0x5633db1d0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54320/0 (socket says 172.21.15.27:54320) 2023-12-19T18:23:13.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.844+0000 7f9009d9b640 1 -- 172.21.15.27:0/2769136096 learned_addr learned my addr 172.21.15.27:0/2769136096 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:13.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.844+0000 7f900a59c640 1 --2- 172.21.15.27:0/2769136096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1a7c90 0x5633db1a8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.844+0000 7f9009d9b640 1 -- 172.21.15.27:0/2769136096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d4110 msgr2=0x5633db1a3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:13.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.844+0000 7f9009d9b640 1 --2- 172.21.15.27:0/2769136096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d4110 0x5633db1a3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.844+0000 7f9009d9b640 1 -- 172.21.15.27:0/2769136096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1a7c90 msgr2=0x5633db1a8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:13.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.844+0000 7f9009d9b640 1 --2- 172.21.15.27:0/2769136096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1a7c90 0x5633db1a8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.844+0000 7f9009d9b640 1 -- 172.21.15.27:0/2769136096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5633db141280 con 0x5633db1d0170 2023-12-19T18:23:13.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.844+0000 7f9009d9b640 1 --2- 172.21.15.27:0/2769136096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d0170 0x5633db1d0570 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4076b30 tx=0x7f8ff40ab040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45be09e66800dc9e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:13.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.845+0000 7f9000ff9640 1 -- 172.21.15.27:0/2769136096 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff407a630 con 0x5633db1d0170 2023-12-19T18:23:13.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.845+0000 7f9000ff9640 1 -- 172.21.15.27:0/2769136096 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff407e460 con 0x5633db1d0170 2023-12-19T18:23:13.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.845+0000 7f9000ff9640 1 -- 172.21.15.27:0/2769136096 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4082610 con 0x5633db1d0170 2023-12-19T18:23:13.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.845+0000 7f90137ca740 1 -- 172.21.15.27:0/2769136096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d0170 msgr2=0x5633db1d0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:13.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.845+0000 7f90137ca740 1 --2- 172.21.15.27:0/2769136096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d0170 0x5633db1d0570 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4076b30 tx=0x7f8ff40ab040 comp rx=0 tx=0).stop 2023-12-19T18:23:13.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.845+0000 7f90137ca740 1 -- 172.21.15.27:0/2769136096 shutdown_connections 2023-12-19T18:23:13.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.845+0000 7f90137ca740 1 --2- 172.21.15.27:0/2769136096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1d4110 0x5633db1a3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.845+0000 7f90137ca740 1 --2- 172.21.15.27:0/2769136096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d0170 0x5633db1d0570 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.845+0000 7f90137ca740 1 --2- 172.21.15.27:0/2769136096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1a7c90 0x5633db1a8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.845+0000 7f90137ca740 1 -- 172.21.15.27:0/2769136096 >> 172.21.15.27:0/2769136096 conn(0x5633db1a4090 msgr2=0x5633db1f24c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:13.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.845+0000 7f90137ca740 1 -- 172.21.15.27:0/2769136096 shutdown_connections 2023-12-19T18:23:13.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.845+0000 7f90137ca740 1 -- 172.21.15.27:0/2769136096 wait complete. 2023-12-19T18:23:13.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.846+0000 7f90137ca740 1 -- start start 2023-12-19T18:23:13.847 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.846+0000 7f90137ca740 1 Processor -- start 2023-12-19T18:23:13.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.847+0000 7f90137ca740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1a7c90 0x5633db1e7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.847+0000 7f90137ca740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1d0170 0x5633db1e7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.847+0000 7f90137ca740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d4110 0x5633db25d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.847+0000 7f90137ca740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633db25d430 con 0x5633db1d4110 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.847+0000 7f90137ca740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633db1a95d0 con 0x5633db1d0170 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.847+0000 7f90137ca740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5633db1f1560 con 0x5633db1a7c90 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.847+0000 7f90137ca740 10 client.0 _ll_register_callbacks cb 0x7f8ffc017040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.847+0000 7f900ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d4110 0x5633db25d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.848+0000 7f900ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d4110 0x5633db25d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54330/0 (socket says 172.21.15.27:54330) 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.848+0000 7f900ad9d640 1 -- 172.21.15.27:0/4193418377 learned_addr learned my addr 172.21.15.27:0/4193418377 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.848+0000 7f900ad9d640 1 -- 172.21.15.27:0/4193418377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1a7c90 msgr2=0x5633db1e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.848+0000 7f900a59c640 1 --2- 172.21.15.27:0/4193418377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1a7c90 0x5633db1e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.848+0000 7f900ad9d640 1 --2- 172.21.15.27:0/4193418377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1a7c90 0x5633db1e7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.848+0000 7f900ad9d640 1 -- 172.21.15.27:0/4193418377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1d0170 msgr2=0x5633db1e7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.848+0000 7f900ad9d640 1 --2- 172.21.15.27:0/4193418377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1d0170 0x5633db1e7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.848+0000 7f900ad9d640 1 -- 172.21.15.27:0/4193418377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f900c1f6330 con 0x5633db1d4110 2023-12-19T18:23:13.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.848+0000 7f900ad9d640 1 --2- 172.21.15.27:0/4193418377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d4110 0x5633db25d880 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9004001eb0 tx=0x7f90040036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be45bacb2ec0f1b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:13.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.848+0000 7f8fe97fa640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:23:13.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f8fe97fa640 1 -- 172.21.15.27:0/4193418377 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90040069e0 con 0x5633db1d4110 2023-12-19T18:23:13.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f8fe97fa640 1 -- 172.21.15.27:0/4193418377 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9004006b40 con 0x5633db1d4110 2023-12-19T18:23:13.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f90137ca740 1 -- 172.21.15.27:0/4193418377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f900400d040 con 0x5633db1d4110 2023-12-19T18:23:13.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f8fe97fa640 1 -- 172.21.15.27:0/4193418377 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9004006d00 con 0x5633db1d4110 2023-12-19T18:23:13.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f90137ca740 10 client.7932 fetch_fsmap learned FSMap version 61 2023-12-19T18:23:13.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f90137ca740 1 -- 172.21.15.27:0/4193418377 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=61,monmap=2+}) v3 -- 0x7f900c0cacd0 con 0x5633db1d4110 2023-12-19T18:23:13.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f8fe97fa640 1 -- 172.21.15.27:0/4193418377 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f900400d040 con 0x5633db1d4110 2023-12-19T18:23:13.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f8fe97fa640 1 -- 172.21.15.27:0/4193418377 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 61) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f90040125f0 con 0x5633db1d4110 2023-12-19T18:23:13.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f90137ca740 10 client.7932 fetch_fsmap finished waiting for FSMap version 61 2023-12-19T18:23:13.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f90137ca740 10 client.7932 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T18:23:13.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f90137ca740 20 client.7932 populate_metadata read hostname 'smithi027' 2023-12-19T18:23:13.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f90137ca740 1 --2- 172.21.15.27:0/4193418377 >> [v2:172.21.15.27:6834/1593995318,v1:172.21.15.27:6835/1593995318] conn(0x5633db25ddc0 0x5633db260a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:13.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f90137ca740 4 client.7932 mds_command: new command op to 7794 tid=0[{"prefix": "status"}] 2023-12-19T18:23:13.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.849+0000 7f90137ca740 1 -- 172.21.15.27:0/4193418377 --> [v2:172.21.15.27:6834/1593995318,v1:172.21.15.27:6835/1593995318] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5633db106100 con 0x5633db25ddc0 2023-12-19T18:23:13.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.850+0000 7f900a59c640 1 --2- 172.21.15.27:0/4193418377 >> [v2:172.21.15.27:6834/1593995318,v1:172.21.15.27:6835/1593995318] conn(0x5633db25ddc0 0x5633db260a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.850+0000 7f900a59c640 1 --2- 172.21.15.27:0/4193418377 >> [v2:172.21.15.27:6834/1593995318,v1:172.21.15.27:6835/1593995318] conn(0x5633db25ddc0 0x5633db260a20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ebd421f3b8b9cca server_cookie=2df55d25d798116d in_seq=0 out_seq=0 2023-12-19T18:23:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.850+0000 7f8fe97fa640 10 client.7932 ms_handle_connect on v2:172.21.15.27:6834/1593995318 2023-12-19T18:23:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.850+0000 7f8fe97fa640 1 -- 172.21.15.27:0/4193418377 <== mds.0 v2:172.21.15.27:6834/1593995318 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x5633db106100 con 0x5633db25ddc0 2023-12-19T18:23:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.850+0000 7f8fe97fa640 10 client.7932 handle_command_reply: tid=0 2023-12-19T18:23:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.850+0000 7f90137ca740 1 client.7932 shutdown 2023-12-19T18:23:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.851+0000 7f90137ca740 1 -- 172.21.15.27:0/4193418377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d4110 msgr2=0x5633db25d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.851+0000 7f90137ca740 1 --2- 172.21.15.27:0/4193418377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d4110 0x5633db25d880 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7f9004001eb0 tx=0x7f90040036f0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.851+0000 7f90137ca740 1 -- 172.21.15.27:0/4193418377 shutdown_connections 2023-12-19T18:23:13.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.851+0000 7f90137ca740 1 --2- 172.21.15.27:0/4193418377 >> [v2:172.21.15.27:6834/1593995318,v1:172.21.15.27:6835/1593995318] conn(0x5633db25ddc0 0x5633db260a20 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.851+0000 7f90137ca740 1 --2- 172.21.15.27:0/4193418377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x5633db1d4110 0x5633db25d880 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.851+0000 7f90137ca740 1 --2- 172.21.15.27:0/4193418377 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x5633db1d0170 0x5633db1e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.851+0000 7f90137ca740 1 --2- 172.21.15.27:0/4193418377 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x5633db1a7c90 0x5633db1e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.851+0000 7f90137ca740 1 -- 172.21.15.27:0/4193418377 >> 172.21.15.27:0/4193418377 conn(0x5633db1a4090 msgr2=0x5633db1e6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:13.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.851+0000 7f8fe97fa640 0 client.7932 ms_handle_reset on v2:172.21.15.27:6834/1593995318 2023-12-19T18:23:13.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.851+0000 7f90137ca740 1 -- 172.21.15.27:0/4193418377 shutdown_connections 2023-12-19T18:23:13.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.851+0000 7f90137ca740 1 -- 172.21.15.27:0/4193418377 wait complete. 2023-12-19T18:23:13.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.851+0000 7f90137ca740 20 client.7932 trim_cache size 0 max 16384 2023-12-19T18:23:13.854 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:23:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:23:13.854 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-12-19T18:23:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 7794, 2023-12-19T18:23:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-19T18:23:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-19T18:23:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-19T18:23:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 7.3802758620000004, 2023-12-19T18:23:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 61, 2023-12-19T18:23:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 152, 2023-12-19T18:23:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 152, 2023-12-19T18:23:13.855 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 13.187321715 2023-12-19T18:23:13.855 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:23:13.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.852+0000 7f900b59e640 1 -- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8fec06a430 msgr2=0x7f8fec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:13.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.852+0000 7f900b59e640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8fec06a430 0x7f8fec06c8f0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc001550 tx=0x7f8ffc009040 comp rx=0 tx=0).stop 2023-12-19T18:23:13.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.852+0000 7f900b59e640 1 -- 172.21.15.27:0/1400319646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f900c1f6b30 msgr2=0x7f900c0d13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:13.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.852+0000 7f900b59e640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f900c1f6b30 0x7f900c0d13f0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f8ff402bec0 tx=0x7f8ff4004730 comp rx=0 tx=0).stop 2023-12-19T18:23:13.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.852+0000 7f900b59e640 1 -- 172.21.15.27:0/1400319646 shutdown_connections 2023-12-19T18:23:13.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.852+0000 7f900b59e640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8fec06a430 0x7f8fec06c8f0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.852+0000 7f900b59e640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f900c0d1930 0x7f900c1f2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.852+0000 7f900b59e640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f900c1f6b30 0x7f900c0d13f0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.852+0000 7f900b59e640 1 --2- 172.21.15.27:0/1400319646 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8ff80047c0 0x7f900c156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:13.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.852+0000 7f900b59e640 1 -- 172.21.15.27:0/1400319646 >> 172.21.15.27:0/1400319646 conn(0x7f900c0cf760 msgr2=0x7f900c0cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:13.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.852+0000 7f900b59e640 1 -- 172.21.15.27:0/1400319646 shutdown_connections 2023-12-19T18:23:13.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:13.852+0000 7f900b59e640 1 -- 172.21.15.27:0/1400319646 wait complete. 2023-12-19T18:23:13.869 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:23:14.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.099+0000 7feede257640 1 Processor -- start 2023-12-19T18:23:14.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.099+0000 7feede257640 1 -- start start 2023-12-19T18:23:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.100+0000 7feede257640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7feed81604d0 0x7feed80b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.100+0000 7feede257640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feed80d2350 0x7feed80b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.101+0000 7feede257640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80d0300 0x7feed80d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.101+0000 7feede257640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed80d2b30 con 0x7feed80d2350 2023-12-19T18:23:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.101+0000 7feede257640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feed80b5e90 con 0x7feed81604d0 2023-12-19T18:23:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.101+0000 7feede257640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed80d2850 con 0x7feed80d0300 2023-12-19T18:23:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.101+0000 7feedca54640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feed80d2350 0x7feed80b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.101+0000 7feedca54640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7feed80d2350 0x7feed80b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54336/0 (socket says 172.21.15.27:54336) 2023-12-19T18:23:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.101+0000 7feedca54640 1 -- 172.21.15.27:0/487693551 learned_addr learned my addr 172.21.15.27:0/487693551 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:14.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.101+0000 7feedda56640 1 --2- 172.21.15.27:0/487693551 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80d0300 0x7feed80d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:14.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.101+0000 7feedca54640 1 -- 172.21.15.27:0/487693551 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80d0300 msgr2=0x7feed80d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:14.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.101+0000 7feedd255640 1 --2- 172.21.15.27:0/487693551 >> v2:172.21.15.192:3300/0 conn(0x7feed81604d0 0x7feed80b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:14.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.101+0000 7feedca54640 1 --2- 172.21.15.27:0/487693551 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80d0300 0x7feed80d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:14.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.101+0000 7feedca54640 1 -- 172.21.15.27:0/487693551 >> v2:172.21.15.192:3300/0 conn(0x7feed81604d0 msgr2=0x7feed80b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:14.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.102+0000 7feedca54640 1 --2- 172.21.15.27:0/487693551 >> v2:172.21.15.192:3300/0 conn(0x7feed81604d0 0x7feed80b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:14.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.102+0000 7feedca54640 1 -- 172.21.15.27:0/487693551 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feed80cded0 con 0x7feed80d2350 2023-12-19T18:23:14.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.102+0000 7feedda56640 1 --2- 172.21.15.27:0/487693551 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80d0300 0x7feed80d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:14.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.102+0000 7feedd255640 1 --2- 172.21.15.27:0/487693551 >> v2:172.21.15.192:3300/0 conn(0x7feed81604d0 0x7feed80b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:14.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.103+0000 7feedca54640 1 --2- 172.21.15.27:0/487693551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d2350 0x7feed80b4d20 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7feed806a040 tx=0x7feec402b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=142400803eaa3cf5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:14.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.103+0000 7feed7fff640 1 -- 172.21.15.27:0/487693551 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4031860 con 0x7feed80d2350 2023-12-19T18:23:14.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.103+0000 7feed7fff640 1 -- 172.21.15.27:0/487693551 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec40319c0 con 0x7feed80d2350 2023-12-19T18:23:14.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.103+0000 7feed7fff640 1 -- 172.21.15.27:0/487693551 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4031d50 con 0x7feed80d2350 2023-12-19T18:23:14.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.104+0000 7feede257640 1 -- 172.21.15.27:0/487693551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d2350 msgr2=0x7feed80b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:14.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.104+0000 7feede257640 1 --2- 172.21.15.27:0/487693551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d2350 0x7feed80b4d20 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7feed806a040 tx=0x7feec402b900 comp rx=0 tx=0).stop 2023-12-19T18:23:14.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.104+0000 7feede257640 1 -- 172.21.15.27:0/487693551 shutdown_connections 2023-12-19T18:23:14.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.104+0000 7feede257640 1 --2- 172.21.15.27:0/487693551 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80d0300 0x7feed80d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:14.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.104+0000 7feede257640 1 --2- 172.21.15.27:0/487693551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d2350 0x7feed80b4d20 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:14.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.104+0000 7feede257640 1 --2- 172.21.15.27:0/487693551 >> v2:172.21.15.192:3300/0 conn(0x7feed81604d0 0x7feed80b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:14.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.104+0000 7feede257640 1 -- 172.21.15.27:0/487693551 >> 172.21.15.27:0/487693551 conn(0x7feed80cf7e0 msgr2=0x7feed80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:14.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.104+0000 7feede257640 1 -- 172.21.15.27:0/487693551 shutdown_connections 2023-12-19T18:23:14.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.104+0000 7feede257640 1 -- 172.21.15.27:0/487693551 wait complete. 2023-12-19T18:23:14.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.106+0000 7feede257640 1 Processor -- start 2023-12-19T18:23:14.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.106+0000 7feede257640 1 -- start start 2023-12-19T18:23:14.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feede257640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80d0300 0x7feed81f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feede257640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d2350 0x7feed8159d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feede257640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed81604d0 0x7feed815a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feede257640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed81f5840 con 0x7feed80d2350 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feede257640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feed815cc90 con 0x7feed81604d0 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feede257640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feed815cf70 con 0x7feed80d0300 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feedca54640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d2350 0x7feed8159d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feedca54640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d2350 0x7feed8159d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54346/0 (socket says 172.21.15.27:54346) 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feedca54640 1 -- 172.21.15.27:0/337842899 learned_addr learned my addr 172.21.15.27:0/337842899 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feedd255640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80d0300 0x7feed81f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feedda56640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed81604d0 0x7feed815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feedca54640 1 -- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80d0300 msgr2=0x7feed81f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feedca54640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80d0300 0x7feed81f5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feedca54640 1 -- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed81604d0 msgr2=0x7feed815a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:14.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.108+0000 7feedca54640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed81604d0 0x7feed815a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:14.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.109+0000 7feedca54640 1 -- 172.21.15.27:0/337842899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec40094d0 con 0x7feed80d2350 2023-12-19T18:23:14.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.109+0000 7feedd255640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80d0300 0x7feed81f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:14.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.109+0000 7feedca54640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d2350 0x7feed8159d20 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7feec402b940 tx=0x7feec40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:14.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.109+0000 7feed5ffb640 1 -- 172.21.15.27:0/337842899 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec4031850 con 0x7feed80d2350 2023-12-19T18:23:14.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.109+0000 7feed5ffb640 1 -- 172.21.15.27:0/337842899 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feec40443f0 con 0x7feed80d2350 2023-12-19T18:23:14.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.109+0000 7feede257640 1 -- 172.21.15.27:0/337842899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feed80cae80 con 0x7feed80d2350 2023-12-19T18:23:14.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.109+0000 7feed5ffb640 1 -- 172.21.15.27:0/337842899 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feec40447a0 con 0x7feed80d2350 2023-12-19T18:23:14.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.109+0000 7feede257640 1 -- 172.21.15.27:0/337842899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feed80cb440 con 0x7feed80d2350 2023-12-19T18:23:14.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.111+0000 7feede257640 1 -- 172.21.15.27:0/337842899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feed8066f80 con 0x7feed80d2350 2023-12-19T18:23:14.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.111+0000 7feed5ffb640 1 -- 172.21.15.27:0/337842899 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7feec404d450 con 0x7feed80d2350 2023-12-19T18:23:14.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.112+0000 7feed5ffb640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feebc06a430 0x7feebc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:14.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.113+0000 7feed5ffb640 1 -- 172.21.15.27:0/337842899 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feec40ab100 con 0x7feed80d2350 2023-12-19T18:23:14.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.113+0000 7feedd255640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feebc06a430 0x7feebc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:14.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.114+0000 7feedd255640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feebc06a430 0x7feebc06c8f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7feecc00d430 tx=0x7feecc007530 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:14.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.117+0000 7feed5ffb640 1 -- 172.21.15.27:0/337842899 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7feec4076650 con 0x7feed80d2350 2023-12-19T18:23:14.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.353+0000 7feede257640 1 -- 172.21.15.27:0/337842899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7feed804e840 con 0x7feed80d2350 2023-12-19T18:23:14.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.354+0000 7feed5ffb640 1 -- 172.21.15.27:0/337842899 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) v1 ==== 53+0+73 (secure 0 0 0) 0x7feec4079c70 con 0x7feed80d2350 2023-12-19T18:23:14.354 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:23:14.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.357+0000 7feede257640 1 -- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feebc06a430 msgr2=0x7feebc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:14.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.357+0000 7feede257640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feebc06a430 0x7feebc06c8f0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7feecc00d430 tx=0x7feecc007530 comp rx=0 tx=0).stop 2023-12-19T18:23:14.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.357+0000 7feede257640 1 -- 172.21.15.27:0/337842899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d2350 msgr2=0x7feed8159d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:14.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.357+0000 7feede257640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d2350 0x7feed8159d20 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7feec402b940 tx=0x7feec40046c0 comp rx=0 tx=0).stop 2023-12-19T18:23:14.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.357+0000 7feede257640 1 -- 172.21.15.27:0/337842899 shutdown_connections 2023-12-19T18:23:14.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.357+0000 7feede257640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7feebc06a430 0x7feebc06c8f0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:14.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.357+0000 7feede257640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7feed81604d0 0x7feed815a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:14.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.357+0000 7feede257640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feed80d2350 0x7feed8159d20 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:14.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.357+0000 7feede257640 1 --2- 172.21.15.27:0/337842899 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7feed80d0300 0x7feed81f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:14.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.357+0000 7feede257640 1 -- 172.21.15.27:0/337842899 >> 172.21.15.27:0/337842899 conn(0x7feed80cf7e0 msgr2=0x7feed80b4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:14.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.357+0000 7feede257640 1 -- 172.21.15.27:0/337842899 shutdown_connections 2023-12-19T18:23:14.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:14.357+0000 7feede257640 1 -- 172.21.15.27:0/337842899 wait complete. 2023-12-19T18:23:14.380 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:23:14.380 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:23:14.381 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-12-19T18:23:14.381 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:23:14.381 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:23:14.381 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:23:14.381 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:23:14.381 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:23:14.381 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:23:14.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:14.381 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-12-19T18:23:14.436 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:23:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:23:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:23:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:23:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:14.436 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:23:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:23:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-12-19T18:23:14.436 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:23:14.436 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41846sec preferred_lft 41846sec 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:23:14.437 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::8053:76ff:fe69:c66c/64 scope link 2023-12-19T18:23:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:14.438 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:23:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:23:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::781d:d6ff:feb4:7a2d/64 scope link 2023-12-19T18:23:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:14.438 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:23:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether f2:29:63:cc:14:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:23:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::f029:63ff:fecc:140e/64 scope link 2023-12-19T18:23:14.438 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:14.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:14.439 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:23:14.546 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-12-19T18:23:14.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:14.551 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-12-19T18:23:14.610 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-12-19T18:23:14.611 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:23:14.611 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-19T18:23:14.611 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-19T18:23:14.611 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:23:14.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:14.612 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:23:14.612 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:23:14.612 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:23:14.612 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:23:14.612 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:23:14.612 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:23:14.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:14.832 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:23:14.890 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:23:14.890 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:23:14.890 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:23:14.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:14.891 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-12-19T18:23:14.992 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-19T18:23:14.992 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:23:14.992 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:23:15.050 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:23:15.050 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:23:15.051 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:23:15.108 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:23:15.216 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:23:15.216 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:23:15.320 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:23:15.324 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:23:15.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:15.325 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:23:15.388 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:23:15.388 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-19T18:23:15.394 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:23:15.481 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:23:15.482 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:23:15.564 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:23:15.568 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:23:15.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:15.568 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:23:15.602 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-12-19T18:23:15.602+0000 7fe31ca39480 -1 init, newargv = 0x55ae22667320 newargc=15 2023-12-19T18:23:15.602 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[113201]: starting ceph client 2023-12-19T18:23:15.610 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[113201]: starting fuse 2023-12-19T18:23:15.615 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:23:15.616 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-19T18:23:15.616 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:23:15.616 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:23:15.616 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:23:15.616 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:23:15.616 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:23:15.616 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:23:15.616 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:23:15.616 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:23:15.616 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:23:15.616 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:23:15.616 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:23:15.616 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:23:15.617 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:23:15.831 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.113201.asok 2023-12-19T18:23:16.010 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.113201.asok 2023-12-19T18:23:16.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:16.011 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.113201.asok status 2023-12-19T18:23:16.270 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:23:16.270 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:23:16.270 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:23:16.270 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:23:16.270 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:23:16.270 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:23:16.270 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:23:16.270 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "113201", 2023-12-19T18:23:16.270 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:23:16.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:16.270 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 7944, 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 7944 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 832578755 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:16.271 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:23:16.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:16.272 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:23:16.272 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 832578755 2023-12-19T18:23:16.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:16.272 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.7944 192.168.0.1:0/832578755", 2023-12-19T18:23:16.272 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/832578755", 2023-12-19T18:23:16.272 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-12-19T18:23:16.272 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 61, 2023-12-19T18:23:16.272 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 152, 2023-12-19T18:23:16.272 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:23:16.272 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:23:16.272 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:23:16.272 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:23:16.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:16.287 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-12-19T18:23:16.287 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-19T18:23:16.347 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-12-19T18:23:16.347 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-19T18:23:16.347 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:16.347 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:23:16.458 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:23:16.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.697+0000 7f060018e640 1 Processor -- start 2023-12-19T18:23:16.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.697+0000 7f060018e640 1 -- start start 2023-12-19T18:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.698+0000 7f060018e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05f80b4c80 0x7f05f8160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:16.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.698+0000 7f060018e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f05f80b5060 0x7f05f80d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:16.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.698+0000 7f060018e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05f80d0d20 con 0x7f05f80b8920 2023-12-19T18:23:16.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.698+0000 7f060018e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05f80b6130 con 0x7f05f80b5060 2023-12-19T18:23:16.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.699+0000 7f060018e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f8161bf0 con 0x7f05f80b4c80 2023-12-19T18:23:16.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.699+0000 7f05fe72b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f05f80b5060 0x7f05f80d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:16.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.699+0000 7f05fdf2a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05f80b4c80 0x7f05f8160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:16.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.699+0000 7f05fdf2a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05f80b4c80 0x7f05f8160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:55350/0 (socket says 172.21.15.27:55350) 2023-12-19T18:23:16.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.699+0000 7f05fdf2a640 1 -- 172.21.15.27:0/1145299979 learned_addr learned my addr 172.21.15.27:0/1145299979 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:16.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.700+0000 7f05fe72b640 1 -- 172.21.15.27:0/1145299979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05f80b4c80 msgr2=0x7f05f8160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:16.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.700+0000 7f05fe72b640 1 --2- 172.21.15.27:0/1145299979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05f80b4c80 0x7f05f8160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:16.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.700+0000 7f05fe72b640 1 -- 172.21.15.27:0/1145299979 >> v1:172.21.15.27:6789/0 conn(0x7f05f80b8920 legacy=0x7f05f815cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:16.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.700+0000 7f05fe72b640 1 -- 172.21.15.27:0/1145299979 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f8161350 con 0x7f05f80b5060 2023-12-19T18:23:16.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.701+0000 7f05fe72b640 1 --2- 172.21.15.27:0/1145299979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f80b5060 0x7f05f80d1a20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009d90 tx=0x7f05f002be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51c5293907176ff3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:16.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.703+0000 7f05fcf28640 1 -- 172.21.15.27:0/1145299979 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f002cbd0 con 0x7f05f80b5060 2023-12-19T18:23:16.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.703+0000 7f05fcf28640 1 -- 172.21.15.27:0/1145299979 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f002cd30 con 0x7f05f80b5060 2023-12-19T18:23:16.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.703+0000 7f05fcf28640 1 -- 172.21.15.27:0/1145299979 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f002cea0 con 0x7f05f80b5060 2023-12-19T18:23:16.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.704+0000 7f060018e640 1 -- 172.21.15.27:0/1145299979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f80b5060 msgr2=0x7f05f80d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:16.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.704+0000 7f060018e640 1 --2- 172.21.15.27:0/1145299979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f80b5060 0x7f05f80d1a20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f05f0009d90 tx=0x7f05f002be20 comp rx=0 tx=0).stop 2023-12-19T18:23:16.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.704+0000 7f060018e640 1 -- 172.21.15.27:0/1145299979 shutdown_connections 2023-12-19T18:23:16.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.704+0000 7f060018e640 1 --2- 172.21.15.27:0/1145299979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f80b5060 0x7f05f80d1a20 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:16.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.704+0000 7f060018e640 1 --2- 172.21.15.27:0/1145299979 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05f80b4c80 0x7f05f8160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:16.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.704+0000 7f060018e640 1 -- 172.21.15.27:0/1145299979 >> 172.21.15.27:0/1145299979 conn(0x7f05f80d2f90 msgr2=0x7f05f80d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:16.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.704+0000 7f060018e640 1 -- 172.21.15.27:0/1145299979 shutdown_connections 2023-12-19T18:23:16.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.704+0000 7f060018e640 1 -- 172.21.15.27:0/1145299979 wait complete. 2023-12-19T18:23:16.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.707+0000 7f060018e640 1 Processor -- start 2023-12-19T18:23:16.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.708+0000 7f060018e640 1 -- start start 2023-12-19T18:23:16.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.709+0000 7f060018e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f80b4c80 0x7f05f81f2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:16.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.709+0000 7f060018e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05f80b5060 0x7f05f81f35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:16.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.709+0000 7f060018e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05f80b8920 0x7f05f8159030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:16.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.709+0000 7f060018e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80caa50 con 0x7f05f80b4c80 2023-12-19T18:23:16.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.709+0000 7f060018e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05f80c7b10 con 0x7f05f80b8920 2023-12-19T18:23:16.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.709+0000 7f060018e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05f8159980 con 0x7f05f80b5060 2023-12-19T18:23:16.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.710+0000 7f05fdf2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f80b4c80 0x7f05f81f2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:16.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.710+0000 7f05fdf2a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f80b4c80 0x7f05f81f2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54354/0 (socket says 172.21.15.27:54354) 2023-12-19T18:23:16.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.710+0000 7f05fdf2a640 1 -- 172.21.15.27:0/1807777280 learned_addr learned my addr 172.21.15.27:0/1807777280 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:16.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.710+0000 7f05fd729640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05f80b5060 0x7f05f81f35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:16.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.710+0000 7f05fdf2a640 1 -- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05f80b5060 msgr2=0x7f05f81f35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:16.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.710+0000 7f05fdf2a640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05f80b5060 0x7f05f81f35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:16.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.710+0000 7f05fdf2a640 1 -- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05f80b8920 msgr2=0x7f05f8159030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:16.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.710+0000 7f05fdf2a640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05f80b8920 0x7f05f8159030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:16.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.710+0000 7f05fdf2a640 1 -- 172.21.15.27:0/1807777280 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f0009a70 con 0x7f05f80b4c80 2023-12-19T18:23:16.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.710+0000 7f05fd729640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05f80b5060 0x7f05f81f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:16.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.711+0000 7f05fdf2a640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f80b4c80 0x7f05f81f2b60 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009fd0 tx=0x7f05e800ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:16.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.711+0000 7f05f67fc640 1 -- 172.21.15.27:0/1807777280 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e800c940 con 0x7f05f80b4c80 2023-12-19T18:23:16.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.711+0000 7f05f67fc640 1 -- 172.21.15.27:0/1807777280 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05e800caa0 con 0x7f05f80b4c80 2023-12-19T18:23:16.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.711+0000 7f060018e640 1 -- 172.21.15.27:0/1807777280 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05f8159c60 con 0x7f05f80b4c80 2023-12-19T18:23:16.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.711+0000 7f05f67fc640 1 -- 172.21.15.27:0/1807777280 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05e800ce30 con 0x7f05f80b4c80 2023-12-19T18:23:16.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.711+0000 7f060018e640 1 -- 172.21.15.27:0/1807777280 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05f81f3af0 con 0x7f05f80b4c80 2023-12-19T18:23:16.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.713+0000 7f060018e640 1 -- 172.21.15.27:0/1807777280 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05f80cddf0 con 0x7f05f80b4c80 2023-12-19T18:23:16.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.715+0000 7f05f67fc640 1 -- 172.21.15.27:0/1807777280 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f05e8013470 con 0x7f05f80b4c80 2023-12-19T18:23:16.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.716+0000 7f05f67fc640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f05e406a430 0x7f05e406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:16.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.716+0000 7f05f67fc640 1 -- 172.21.15.27:0/1807777280 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05e8088b00 con 0x7f05f80b4c80 2023-12-19T18:23:16.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.717+0000 7f05fd729640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f05e406a430 0x7f05e406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:16.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.719+0000 7f05fd729640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f05e406a430 0x7f05e406c8f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f05f80ca980 tx=0x7f05ec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:16.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.721+0000 7f05f67fc640 1 -- 172.21.15.27:0/1807777280 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f05e808ae60 con 0x7f05f80b4c80 2023-12-19T18:23:16.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.955+0000 7f060018e640 1 -- 172.21.15.27:0/1807777280 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f05f8068e90 con 0x7f05f80b4c80 2023-12-19T18:23:16.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.956+0000 7f05f67fc640 1 -- 172.21.15.27:0/1807777280 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v61) v1 ==== 53+0+73 (secure 0 0 0) 0x7f05e8017070 con 0x7f05f80b4c80 2023-12-19T18:23:16.956 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:23:16.959 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.959+0000 7f060018e640 1 -- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f05e406a430 msgr2=0x7f05e406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:16.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.959+0000 7f060018e640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f05e406a430 0x7f05e406c8f0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f05f80ca980 tx=0x7f05ec009040 comp rx=0 tx=0).stop 2023-12-19T18:23:16.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.959+0000 7f060018e640 1 -- 172.21.15.27:0/1807777280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f80b4c80 msgr2=0x7f05f81f2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:16.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.959+0000 7f060018e640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f80b4c80 0x7f05f81f2b60 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f05e8009fd0 tx=0x7f05e800ed40 comp rx=0 tx=0).stop 2023-12-19T18:23:16.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.959+0000 7f060018e640 1 -- 172.21.15.27:0/1807777280 shutdown_connections 2023-12-19T18:23:16.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.959+0000 7f060018e640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f05e406a430 0x7f05e406c8f0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:16.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.959+0000 7f060018e640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f05f80b8920 0x7f05f8159030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:16.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.959+0000 7f060018e640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f05f80b5060 0x7f05f81f35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:16.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.959+0000 7f060018e640 1 --2- 172.21.15.27:0/1807777280 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05f80b4c80 0x7f05f81f2b60 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:16.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.959+0000 7f060018e640 1 -- 172.21.15.27:0/1807777280 >> 172.21.15.27:0/1807777280 conn(0x7f05f80d2f90 msgr2=0x7f05f80cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:16.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.959+0000 7f060018e640 1 -- 172.21.15.27:0/1807777280 shutdown_connections 2023-12-19T18:23:16.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:16.960+0000 7f060018e640 1 -- 172.21.15.27:0/1807777280 wait complete. 2023-12-19T18:23:16.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:23:16.984 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:23:16.985 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:23:16.985 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:23:16.985 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-19T18:23:16.985 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-19T18:23:16.985 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T18:23:16.985 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:23:16.985 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:23:16.985 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:23:16.985 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:23:16.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:16.986 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-19T18:23:17.042 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:23:17.043 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:23:17.043 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:23:17.043 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:17.043 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:23:17.043 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:17.043 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:23:17.043 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-19T18:23:17.043 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-19T18:23:17.043 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:23:17.043 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-19T18:23:17.043 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-19T18:23:17.043 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 41806sec preferred_lft 41806sec 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::b892:1cff:fedc:4bed/64 scope link 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:23:17.044 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether a6:e1:fc:e6:56:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:23:17.045 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::a4e1:fcff:fee6:56fa/64 scope link 2023-12-19T18:23:17.045 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:17.045 INFO:teuthology.orchestra.run.smithi192.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:23:17.045 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T18:23:17.045 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::404f:6cff:fe73:9aa2/64 scope link 2023-12-19T18:23:17.045 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:23:17.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:17.045 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:23:17.175 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-19T18:23:17.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:17.177 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-19T18:23:17.231 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-19T18:23:17.231 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:23:17.231 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T18:23:17.231 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T18:23:17.231 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:23:17.231 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:17.231 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:23:17.231 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:23:17.232 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T18:23:17.232 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T18:23:17.232 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T18:23:17.232 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:23:17.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:17.403 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:23:17.454 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T18:23:17.454 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-19T18:23:17.454 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:23:17.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:17.455 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-19T18:23:17.546 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-19T18:23:17.546 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:23:17.546 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-19T18:23:17.602 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:23:17.602 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:23:17.602 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:23:17.653 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:23:17.747 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:23:17.747 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:23:17.834 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:23:17.837 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:23:17.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:17.838 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:23:17.891 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:23:17.892 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-19T18:23:17.897 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:23:17.976 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:23:17.976 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:23:18.054 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:23:18.057 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:23:18.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:18.058 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:23:18.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-19T18:23:18.083+0000 7ff6f8c4b480 -1 init, newargv = 0x5575cc6e6820 newargc=15 2023-12-19T18:23:18.083 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[90674]: starting ceph client 2023-12-19T18:23:18.092 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[90674]: starting fuse 2023-12-19T18:23:18.103 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:23:18.103 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-12-19T18:23:18.103 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:23:18.103 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-19T18:23:18.103 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-19T18:23:18.103 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-19T18:23:18.103 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-19T18:23:18.103 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:23:18.103 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-19T18:23:18.104 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-19T18:23:18.105 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:23:18.105 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:23:18.105 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-19T18:23:18.105 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-19T18:23:18.304 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.90674.asok 2023-12-19T18:23:18.488 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.90674.asok 2023-12-19T18:23:18.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:18.489 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.90674.asok status 2023-12-19T18:23:18.730 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-19T18:23:18.730 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-19T18:23:18.730 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:23:18.730 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:23:18.730 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-19T18:23:18.730 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-19T18:23:18.730 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-19T18:23:18.730 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "90674", 2023-12-19T18:23:18.730 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5786, 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5786 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 834850445 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-19T18:23:18.731 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 834850445 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5786 192.168.0.1:0/834850445", 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/834850445", 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 61, 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 152, 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-19T18:23:18.732 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-19T18:23:18.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:18.745 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-19T18:23:18.745 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-19T18:23:18.796 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-19T18:23:18.796 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-19T18:23:18.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:18.796 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:23:18.886 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:23:19.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.129+0000 7f68d2e73640 1 Processor -- start 2023-12-19T18:23:19.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.129+0000 7f68d2e73640 1 -- start start 2023-12-19T18:23:19.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.130+0000 7f68d2e73640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc0b4ef0 0x7f68cc0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.130+0000 7f68d2e73640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f68cc0d2350 0x7f68cc0b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.131+0000 7f68d2e73640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68cc0b5e90 con 0x7f68cc1604d0 2023-12-19T18:23:19.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.131+0000 7f68d2e73640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68cc0d2b30 con 0x7f68cc0d2350 2023-12-19T18:23:19.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.131+0000 7f68d2e73640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0d2850 con 0x7f68cc0b4ef0 2023-12-19T18:23:19.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.131+0000 7f68d1e71640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f68cc1604d0 0x7f68cc0b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59436/0 (socket says 172.21.15.27:59436) 2023-12-19T18:23:19.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.131+0000 7f68d1e71640 1 -- 172.21.15.27:0/2031691097 learned_addr learned my addr 172.21.15.27:0/2031691097 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:19.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.131+0000 7f68d1670640 1 --2- 172.21.15.27:0/2031691097 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc0b4ef0 0x7f68cc0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.131+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2551944477 0 0) 0x7f68cc0b5e90 con 0x7f68cc1604d0 2023-12-19T18:23:19.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.131+0000 7f68d2672640 1 --2- 172.21.15.27:0/2031691097 >> v2:172.21.15.192:3300/0 conn(0x7f68cc0d2350 0x7f68cc0b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.131+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68bc003520 con 0x7f68cc1604d0 2023-12-19T18:23:19.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2221462280 0 0) 0x7f68bc003520 con 0x7f68cc1604d0 2023-12-19T18:23:19.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc0b4ef0 msgr2=0x7f68cc0b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:19.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 --2- 172.21.15.27:0/2031691097 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc0b4ef0 0x7f68cc0b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 >> v2:172.21.15.192:3300/0 conn(0x7f68cc0d2350 msgr2=0x7f68cc0b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:19.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 --2- 172.21.15.27:0/2031691097 >> v2:172.21.15.192:3300/0 conn(0x7f68cc0d2350 0x7f68cc0b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc0cded0 con 0x7f68cc1604d0 2023-12-19T18:23:19.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f68c00019f0 con 0x7f68cc1604d0 2023-12-19T18:23:19.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 >> v1:172.21.15.27:6789/0 conn(0x7f68cc1604d0 legacy=0x7f68cc0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:19.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 --2- 172.21.15.27:0/2031691097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68bc003970 0x7f68bc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 --2- 172.21.15.27:0/2031691097 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc0b4ef0 0x7f68bc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 --2- 172.21.15.27:0/2031691097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68bc004d00 0x7f68bc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0d2b30 con 0x7f68bc004d00 2023-12-19T18:23:19.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0d2850 con 0x7f68bc003970 2023-12-19T18:23:19.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68bc00b980 con 0x7f68cc0b4ef0 2023-12-19T18:23:19.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.132+0000 7f68d2672640 1 --2- 172.21.15.27:0/2031691097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68bc004d00 0x7f68bc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.133+0000 7f68d1670640 1 --2- 172.21.15.27:0/2031691097 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc0b4ef0 0x7f68bc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.133+0000 7f68d1e71640 1 --2- 172.21.15.27:0/2031691097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68bc003970 0x7f68bc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.133+0000 7f68d2672640 1 -- 172.21.15.27:0/2031691097 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc0b4ef0 msgr2=0x7f68bc0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:19.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.133+0000 7f68d2672640 1 --2- 172.21.15.27:0/2031691097 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68cc0b4ef0 0x7f68bc0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.133+0000 7f68d2672640 1 -- 172.21.15.27:0/2031691097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68bc003970 msgr2=0x7f68bc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:19.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.133+0000 7f68d2672640 1 --2- 172.21.15.27:0/2031691097 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68bc003970 0x7f68bc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.133+0000 7f68d2672640 1 -- 172.21.15.27:0/2031691097 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc003520 con 0x7f68bc004d00 2023-12-19T18:23:19.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.134+0000 7f68d2672640 1 --2- 172.21.15.27:0/2031691097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68bc004d00 0x7f68bc005180 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0d3e30 tx=0x7f68bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be4d70e22329595e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:19.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.134+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c402c680 con 0x7f68bc004d00 2023-12-19T18:23:19.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.135+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c402c7e0 con 0x7f68bc004d00 2023-12-19T18:23:19.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.135+0000 7f68d0e6f640 1 -- 172.21.15.27:0/2031691097 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c402cb90 con 0x7f68bc004d00 2023-12-19T18:23:19.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.135+0000 7f68d2e73640 1 -- 172.21.15.27:0/2031691097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68bc004d00 msgr2=0x7f68bc005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:19.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.135+0000 7f68d2e73640 1 --2- 172.21.15.27:0/2031691097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68bc004d00 0x7f68bc005180 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7f68cc0d3e30 tx=0x7f68bc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.135+0000 7f68d2672640 1 -- 172.21.15.27:0/2031691097 reap_dead start 2023-12-19T18:23:19.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.135+0000 7f68d2e73640 1 -- 172.21.15.27:0/2031691097 shutdown_connections 2023-12-19T18:23:19.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.135+0000 7f68d2e73640 1 -- 172.21.15.27:0/2031691097 >> 172.21.15.27:0/2031691097 conn(0x7f68cc0cf7e0 msgr2=0x7f68cc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:19.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.135+0000 7f68d2e73640 1 -- 172.21.15.27:0/2031691097 shutdown_connections 2023-12-19T18:23:19.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.135+0000 7f68d2e73640 1 -- 172.21.15.27:0/2031691097 wait complete. 2023-12-19T18:23:19.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.137+0000 7f68d2e73640 1 Processor -- start 2023-12-19T18:23:19.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.137+0000 7f68d2e73640 1 -- start start 2023-12-19T18:23:19.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.138+0000 7f68d2e73640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68bc004d00 0x7f68cc0aa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.138+0000 7f68d2e73640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc0a9780 0x7f68cc0b3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.138+0000 7f68d2e73640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68cc15c620 0x7f68cc15caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.138+0000 7f68d2e73640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc0aa8d0 con 0x7f68cc15c620 2023-12-19T18:23:19.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.138+0000 7f68d2e73640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc15b860 con 0x7f68cc0a9780 2023-12-19T18:23:19.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.138+0000 7f68d2e73640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68cc15bb40 con 0x7f68bc004d00 2023-12-19T18:23:19.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.139+0000 7f68d2672640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68cc15c620 0x7f68cc15caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.139+0000 7f68d2672640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68cc15c620 0x7f68cc15caa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54358/0 (socket says 172.21.15.27:54358) 2023-12-19T18:23:19.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.139+0000 7f68d2672640 1 -- 172.21.15.27:0/1369680029 learned_addr learned my addr 172.21.15.27:0/1369680029 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:19.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.139+0000 7f68d1e71640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68bc004d00 0x7f68cc0aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.139+0000 7f68d1670640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc0a9780 0x7f68cc0b3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.139+0000 7f68d2672640 1 -- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68bc004d00 msgr2=0x7f68cc0aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:19.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.139+0000 7f68d2672640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68bc004d00 0x7f68cc0aa080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.139+0000 7f68d2672640 1 -- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc0a9780 msgr2=0x7f68cc0b3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.139+0000 7f68d2672640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc0a9780 0x7f68cc0b3950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.139+0000 7f68d2672640 1 -- 172.21.15.27:0/1369680029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68cc15be20 con 0x7f68cc15c620 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.139+0000 7f68d1e71640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68bc004d00 0x7f68cc0aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.139+0000 7f68d2672640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68cc15c620 0x7f68cc15caa0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f68c402c220 tx=0x7f68c4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.140+0000 7f68ca7fc640 1 -- 172.21.15.27:0/1369680029 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c403e850 con 0x7f68cc15c620 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.140+0000 7f68ca7fc640 1 -- 172.21.15.27:0/1369680029 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68c403e9b0 con 0x7f68cc15c620 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.140+0000 7f68ca7fc640 1 -- 172.21.15.27:0/1369680029 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68c403ed60 con 0x7f68cc15c620 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.140+0000 7f68d2e73640 1 -- 172.21.15.27:0/1369680029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68cc15c100 con 0x7f68cc15c620 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.140+0000 7f68d2e73640 1 -- 172.21.15.27:0/1369680029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68cc0b3e90 con 0x7f68cc15c620 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.141+0000 7f68d2e73640 1 -- 172.21.15.27:0/1369680029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68cc066f80 con 0x7f68cc15c620 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.142+0000 7f68ca7fc640 1 -- 172.21.15.27:0/1369680029 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f68c4034440 con 0x7f68cc15c620 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.143+0000 7f68ca7fc640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f68b006a460 0x7f68b006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.143+0000 7f68ca7fc640 1 -- 172.21.15.27:0/1369680029 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68c40a9980 con 0x7f68cc15c620 2023-12-19T18:23:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.144+0000 7f68d1e71640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f68b006a460 0x7f68b006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.145+0000 7f68d1e71640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f68b006a460 0x7f68b006c920 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f68c0005d10 tx=0x7f68c0001310 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:19.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.148+0000 7f68ca7fc640 1 -- 172.21.15.27:0/1369680029 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f68c4074ec0 con 0x7f68cc15c620 2023-12-19T18:23:19.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.380+0000 7f68d2e73640 1 -- 172.21.15.27:0/1369680029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f68cc15c320 con 0x7f68cc15c620 2023-12-19T18:23:19.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.380+0000 7f68ca7fc640 1 -- 172.21.15.27:0/1369680029 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f68c4039020 con 0x7f68cc15c620 2023-12-19T18:23:19.380 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:19.381 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":61,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3298228307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3298228307},{"type":"v1","addr":"172.21.15.192:6835","nonce":3298228307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5726,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1760483999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1760483999},{"type":"v1","addr":"172.21.15.192:6837","nonce":1760483999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3000637822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3000637822},{"type":"v1","addr":"172.21.15.27:6833","nonce":3000637822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:23:06.465885+0000","modified":"2023-12-19T18:23:09.973228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1593995318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1593995318},{"type":"v1","addr":"172.21.15.27:6835","nonce":1593995318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T18:23:19.381 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 61 2023-12-19T18:23:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.383+0000 7f68d2e73640 1 -- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f68b006a460 msgr2=0x7f68b006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.383+0000 7f68d2e73640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f68b006a460 0x7f68b006c920 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f68c0005d10 tx=0x7f68c0001310 comp rx=0 tx=0).stop 2023-12-19T18:23:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.383+0000 7f68d2e73640 1 -- 172.21.15.27:0/1369680029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68cc15c620 msgr2=0x7f68cc15caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.383+0000 7f68d2e73640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68cc15c620 0x7f68cc15caa0 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7f68c402c220 tx=0x7f68c4004730 comp rx=0 tx=0).stop 2023-12-19T18:23:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.383+0000 7f68d2e73640 1 -- 172.21.15.27:0/1369680029 shutdown_connections 2023-12-19T18:23:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.383+0000 7f68d2e73640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f68b006a460 0x7f68b006c920 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.383+0000 7f68d2e73640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68cc15c620 0x7f68cc15caa0 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.383+0000 7f68d2e73640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f68cc0a9780 0x7f68cc0b3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.383+0000 7f68d2e73640 1 --2- 172.21.15.27:0/1369680029 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f68bc004d00 0x7f68cc0aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.383+0000 7f68d2e73640 1 -- 172.21.15.27:0/1369680029 >> 172.21.15.27:0/1369680029 conn(0x7f68cc0cf7e0 msgr2=0x7f68cc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.383+0000 7f68d2e73640 1 -- 172.21.15.27:0/1369680029 shutdown_connections 2023-12-19T18:23:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.384+0000 7f68d2e73640 1 -- 172.21.15.27:0/1369680029 wait complete. 2023-12-19T18:23:19.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-12-19T18:23:19.655 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":7794,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":13.168251065,"mdsmap_epoch":61,"osdmap_epoch":152,"osdmap_epoch_barrier":152,"uptime":18.975299869000001} 2023-12-19T18:23:19.656 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 7794, "mdsmap_epoch": 61, "osdmap_epoch": 152, "osdmap_epoch_barrier": 152, "rank_uptime": 13.168251065, "state": "up:active", "uptime": 18.975299869, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:23:19.656 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-12-19T18:23:19.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.895+0000 7ff9363ae640 1 Processor -- start 2023-12-19T18:23:19.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.895+0000 7ff9363ae640 1 -- start start 2023-12-19T18:23:19.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.897+0000 7ff9363ae640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9300b0060 0x7ff930161870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.897+0000 7ff9363ae640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff9300cefe0 0x7ff9300cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.897+0000 7ff9363ae640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9300d0400 con 0x7ff9300cdbf0 2023-12-19T18:23:19.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.897+0000 7ff9363ae640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9300ce080 con 0x7ff9300cefe0 2023-12-19T18:23:19.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.897+0000 7ff9363ae640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9300cf7f0 con 0x7ff9300b0060 2023-12-19T18:23:19.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.898+0000 7ff92ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9300b0060 0x7ff930161870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.898+0000 7ff92f7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff9300cefe0 0x7ff9300cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.898+0000 7ff92ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9300b0060 0x7ff930161870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:55384/0 (socket says 172.21.15.27:55384) 2023-12-19T18:23:19.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.898+0000 7ff92f7fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff9300cefe0 0x7ff9300cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:49388/0 (socket says 172.21.15.27:49388) 2023-12-19T18:23:19.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.898+0000 7ff92ffff640 1 -- 172.21.15.27:0/3600513819 learned_addr learned my addr 172.21.15.27:0/3600513819 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:19.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.899+0000 7ff92ffff640 1 -- 172.21.15.27:0/3600513819 >> v2:172.21.15.192:3300/0 conn(0x7ff9300cefe0 msgr2=0x7ff9300cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:19.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.899+0000 7ff92ffff640 1 --2- 172.21.15.27:0/3600513819 >> v2:172.21.15.192:3300/0 conn(0x7ff9300cefe0 0x7ff9300cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.899+0000 7ff92ffff640 1 -- 172.21.15.27:0/3600513819 >> v1:172.21.15.192:6789/0 conn(0x7ff9300cdbf0 legacy=0x7ff93015c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:19.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.899+0000 7ff92ffff640 1 -- 172.21.15.27:0/3600513819 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9300cfad0 con 0x7ff9300b0060 2023-12-19T18:23:19.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.899+0000 7ff92f7fe640 1 --2- 172.21.15.27:0/3600513819 >> v2:172.21.15.192:3300/0 conn(0x7ff9300cefe0 0x7ff9300cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:23:19.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.900+0000 7ff92ffff640 1 --2- 172.21.15.27:0/3600513819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9300b0060 0x7ff930161870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff920009850 tx=0x7ff92002b960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5ade419dfffb301 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:19.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.901+0000 7ff92effd640 1 -- 172.21.15.27:0/3600513819 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff920004230 con 0x7ff9300b0060 2023-12-19T18:23:19.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.901+0000 7ff92effd640 1 -- 172.21.15.27:0/3600513819 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff920004390 con 0x7ff9300b0060 2023-12-19T18:23:19.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.901+0000 7ff92effd640 1 -- 172.21.15.27:0/3600513819 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff920003810 con 0x7ff9300b0060 2023-12-19T18:23:19.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.901+0000 7ff9363ae640 1 -- 172.21.15.27:0/3600513819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9300b0060 msgr2=0x7ff930161870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:19.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.901+0000 7ff9363ae640 1 --2- 172.21.15.27:0/3600513819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9300b0060 0x7ff930161870 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff920009850 tx=0x7ff92002b960 comp rx=0 tx=0).stop 2023-12-19T18:23:19.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.901+0000 7ff9363ae640 1 -- 172.21.15.27:0/3600513819 shutdown_connections 2023-12-19T18:23:19.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.901+0000 7ff9363ae640 1 --2- 172.21.15.27:0/3600513819 >> v2:172.21.15.192:3300/0 conn(0x7ff9300cefe0 0x7ff9300cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.901+0000 7ff9363ae640 1 --2- 172.21.15.27:0/3600513819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9300b0060 0x7ff930161870 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.901+0000 7ff9363ae640 1 -- 172.21.15.27:0/3600513819 >> 172.21.15.27:0/3600513819 conn(0x7ff93009a000 msgr2=0x7ff9300d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:19.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.902+0000 7ff9363ae640 1 -- 172.21.15.27:0/3600513819 shutdown_connections 2023-12-19T18:23:19.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.902+0000 7ff9363ae640 1 -- 172.21.15.27:0/3600513819 wait complete. 2023-12-19T18:23:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.903+0000 7ff9363ae640 1 Processor -- start 2023-12-19T18:23:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.903+0000 7ff9363ae640 1 -- start start 2023-12-19T18:23:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.904+0000 7ff9363ae640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9300b0060 0x7ff9300c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.904+0000 7ff9363ae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300cdbf0 0x7ff9300cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.904+0000 7ff9363ae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9300cefe0 0x7ff9300cbf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.904+0000 7ff9363ae640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff930152a90 con 0x7ff9300cdbf0 2023-12-19T18:23:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.904+0000 7ff9363ae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9300c6c20 con 0x7ff9300cefe0 2023-12-19T18:23:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.904+0000 7ff9363ae640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9300c75f0 con 0x7ff9300b0060 2023-12-19T18:23:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.905+0000 7ff92f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300cdbf0 0x7ff9300cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.905+0000 7ff92f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300cdbf0 0x7ff9300cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54366/0 (socket says 172.21.15.27:54366) 2023-12-19T18:23:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.905+0000 7ff92f7fe640 1 -- 172.21.15.27:0/394018996 learned_addr learned my addr 172.21.15.27:0/394018996 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:19.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.905+0000 7ff92ffff640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9300b0060 0x7ff9300c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.905+0000 7ff93494b640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9300cefe0 0x7ff9300cbf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.905+0000 7ff92f7fe640 1 -- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9300b0060 msgr2=0x7ff9300c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.905+0000 7ff92f7fe640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9300b0060 0x7ff9300c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.905+0000 7ff92f7fe640 1 -- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9300cefe0 msgr2=0x7ff9300cbf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.905+0000 7ff92f7fe640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9300cefe0 0x7ff9300cbf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.905+0000 7ff92f7fe640 1 -- 172.21.15.27:0/394018996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff924009400 con 0x7ff9300cdbf0 2023-12-19T18:23:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.905+0000 7ff92ffff640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9300b0060 0x7ff9300c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.905+0000 7ff93494b640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9300cefe0 0x7ff9300cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.906+0000 7ff92f7fe640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300cdbf0 0x7ff9300cba00 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7ff92400e960 tx=0x7ff92400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.906+0000 7ff92cff9640 1 -- 172.21.15.27:0/394018996 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92400ca00 con 0x7ff9300cdbf0 2023-12-19T18:23:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.906+0000 7ff92cff9640 1 -- 172.21.15.27:0/394018996 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92400cb60 con 0x7ff9300cdbf0 2023-12-19T18:23:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.906+0000 7ff92cff9640 1 -- 172.21.15.27:0/394018996 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff924013470 con 0x7ff9300cdbf0 2023-12-19T18:23:19.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.906+0000 7ff9363ae640 1 -- 172.21.15.27:0/394018996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff920009460 con 0x7ff9300cdbf0 2023-12-19T18:23:19.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.906+0000 7ff9363ae640 1 -- 172.21.15.27:0/394018996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9300c62f0 con 0x7ff9300cdbf0 2023-12-19T18:23:19.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.907+0000 7ff9363ae640 1 -- 172.21.15.27:0/394018996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9300c6d90 con 0x7ff9300cdbf0 2023-12-19T18:23:19.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.909+0000 7ff92cff9640 1 -- 172.21.15.27:0/394018996 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff9240135d0 con 0x7ff9300cdbf0 2023-12-19T18:23:19.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.909+0000 7ff92cff9640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff91c06a460 0x7ff91c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:19.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.910+0000 7ff92cff9640 1 -- 172.21.15.27:0/394018996 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff924088c70 con 0x7ff9300cdbf0 2023-12-19T18:23:19.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.910+0000 7ff92ffff640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff91c06a460 0x7ff91c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:19.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.911+0000 7ff92ffff640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff91c06a460 0x7ff91c06c920 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff920035000 tx=0x7ff920008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:19.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:19.914+0000 7ff92cff9640 1 -- 172.21.15.27:0/394018996 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff9240541d0 con 0x7ff9300cdbf0 2023-12-19T18:23:20.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.110+0000 7ff9363ae640 1 -- 172.21.15.27:0/394018996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7ff930068e90 con 0x7ff9300cdbf0 2023-12-19T18:23:20.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.761+0000 7ff92cff9640 1 -- 172.21.15.27:0/394018996 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v153) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff9240170c0 con 0x7ff9300cdbf0 2023-12-19T18:23:20.762 INFO:teuthology.orchestra.run.smithi027.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-12-19T18:23:20.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.764+0000 7ff9363ae640 1 -- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff91c06a460 msgr2=0x7ff91c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:20.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.764+0000 7ff9363ae640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff91c06a460 0x7ff91c06c920 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff920035000 tx=0x7ff920008030 comp rx=0 tx=0).stop 2023-12-19T18:23:20.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.764+0000 7ff9363ae640 1 -- 172.21.15.27:0/394018996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300cdbf0 msgr2=0x7ff9300cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:20.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.765+0000 7ff9363ae640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300cdbf0 0x7ff9300cba00 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7ff92400e960 tx=0x7ff92400ee20 comp rx=0 tx=0).stop 2023-12-19T18:23:20.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.765+0000 7ff9363ae640 1 -- 172.21.15.27:0/394018996 shutdown_connections 2023-12-19T18:23:20.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.765+0000 7ff9363ae640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff91c06a460 0x7ff91c06c920 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:20.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.765+0000 7ff9363ae640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff9300cefe0 0x7ff9300cbf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:20.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.765+0000 7ff9363ae640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300cdbf0 0x7ff9300cba00 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:20.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.765+0000 7ff9363ae640 1 --2- 172.21.15.27:0/394018996 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff9300b0060 0x7ff9300c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:20.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.765+0000 7ff9363ae640 1 -- 172.21.15.27:0/394018996 >> 172.21.15.27:0/394018996 conn(0x7ff93009a000 msgr2=0x7ff93015f7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:20.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.765+0000 7ff9363ae640 1 -- 172.21.15.27:0/394018996 shutdown_connections 2023-12-19T18:23:20.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:20.765+0000 7ff9363ae640 1 -- 172.21.15.27:0/394018996 wait complete. 2023-12-19T18:23:20.788 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-12-19T18:23:21.035 INFO:teuthology.orchestra.run.smithi027.stdout:{"osd_mon_report_interval":"5"} 2023-12-19T18:23:21.035 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-12-19T18:23:21.036 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-12-19T18:23:21.036 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:23:21.036 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:23:21.096 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:21.097 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-12-19T18:23:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:16+0 records in 2023-12-19T18:23:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:16+0 records out 2023-12-19T18:23:21.497 INFO:teuthology.orchestra.run.smithi027.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.33242 s, 50.5 MB/s 2023-12-19T18:23:21.499 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:23:21.499 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:23:21.562 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:21.562 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-12-19T18:23:21.980 INFO:teuthology.orchestra.run.smithi027.stderr:17+0 records in 2023-12-19T18:23:21.980 INFO:teuthology.orchestra.run.smithi027.stderr:17+0 records out 2023-12-19T18:23:21.980 INFO:teuthology.orchestra.run.smithi027.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.353884 s, 50.4 MB/s 2023-12-19T18:23:21.982 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-12-19T18:23:21.982 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:23:22.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.220+0000 7fc655315640 1 Processor -- start 2023-12-19T18:23:22.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.220+0000 7fc655315640 1 -- start start 2023-12-19T18:23:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.222+0000 7fc655315640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc650070fb0 0x7fc65006b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.222+0000 7fc655315640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc65006b850 0x7fc65006f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.222+0000 7fc655315640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6500adf10 con 0x7fc65015ac50 2023-12-19T18:23:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.222+0000 7fc655315640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6500adac0 con 0x7fc650070fb0 2023-12-19T18:23:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.222+0000 7fc655315640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6500af630 con 0x7fc65006b850 2023-12-19T18:23:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.222+0000 7fc64ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc65015ac50 0x7fc6500aef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47704/0 (socket says 172.21.15.27:47704) 2023-12-19T18:23:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.222+0000 7fc64ffff640 1 -- 172.21.15.27:0/1481422258 learned_addr learned my addr 172.21.15.27:0/1481422258 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.222+0000 7fc6477fe640 1 --2- 172.21.15.27:0/1481422258 >> v2:172.21.15.27:3300/0 conn(0x7fc650070fb0 0x7fc65006b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.223+0000 7fc654b14640 1 --2- 172.21.15.27:0/1481422258 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc65006b850 0x7fc65006f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.223+0000 7fc6477fe640 1 -- 172.21.15.27:0/1481422258 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc65006b850 msgr2=0x7fc65006f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.223+0000 7fc64f7fe640 1 -- 172.21.15.27:0/1481422258 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2314911944 0 0) 0x7fc6500adf10 con 0x7fc65015ac50 2023-12-19T18:23:22.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.223+0000 7fc6477fe640 1 --2- 172.21.15.27:0/1481422258 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc65006b850 0x7fc65006f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.223+0000 7fc6477fe640 1 -- 172.21.15.27:0/1481422258 >> v1:172.21.15.27:6789/0 conn(0x7fc65015ac50 legacy=0x7fc6500aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.223+0000 7fc6477fe640 1 -- 172.21.15.27:0/1481422258 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6500713f0 con 0x7fc650070fb0 2023-12-19T18:23:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.223+0000 7fc654b14640 1 --2- 172.21.15.27:0/1481422258 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc65006b850 0x7fc65006f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:22.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.224+0000 7fc6477fe640 1 --2- 172.21.15.27:0/1481422258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc650070fb0 0x7fc65006b310 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc6380097a0 tx=0x7fc63802b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3addd7ac89b7096 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:22.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.225+0000 7fc64f7fe640 1 -- 172.21.15.27:0/1481422258 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6380316f0 con 0x7fc650070fb0 2023-12-19T18:23:22.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.225+0000 7fc64f7fe640 1 -- 172.21.15.27:0/1481422258 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc638031850 con 0x7fc650070fb0 2023-12-19T18:23:22.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.225+0000 7fc64f7fe640 1 -- 172.21.15.27:0/1481422258 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc638031be0 con 0x7fc650070fb0 2023-12-19T18:23:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.225+0000 7fc655315640 1 -- 172.21.15.27:0/1481422258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc650070fb0 msgr2=0x7fc65006b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.225+0000 7fc655315640 1 --2- 172.21.15.27:0/1481422258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc650070fb0 0x7fc65006b310 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fc6380097a0 tx=0x7fc63802b790 comp rx=0 tx=0).stop 2023-12-19T18:23:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.225+0000 7fc655315640 1 -- 172.21.15.27:0/1481422258 shutdown_connections 2023-12-19T18:23:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.225+0000 7fc655315640 1 --2- 172.21.15.27:0/1481422258 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc65006b850 0x7fc65006f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.225+0000 7fc655315640 1 --2- 172.21.15.27:0/1481422258 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc650070fb0 0x7fc65006b310 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.225+0000 7fc655315640 1 -- 172.21.15.27:0/1481422258 >> 172.21.15.27:0/1481422258 conn(0x7fc65006a7d0 msgr2=0x7fc6500ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.226+0000 7fc655315640 1 -- 172.21.15.27:0/1481422258 shutdown_connections 2023-12-19T18:23:22.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.226+0000 7fc655315640 1 -- 172.21.15.27:0/1481422258 wait complete. 2023-12-19T18:23:22.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.227+0000 7fc655315640 1 Processor -- start 2023-12-19T18:23:22.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.228+0000 7fc655315640 1 -- start start 2023-12-19T18:23:22.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.228+0000 7fc655315640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65006b850 0x7fc65014e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:22.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.228+0000 7fc655315640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc650070fb0 0x7fc6500ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:22.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.228+0000 7fc655315640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc65015ac50 0x7fc6500ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:22.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.229+0000 7fc655315640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65015a0a0 con 0x7fc65006b850 2023-12-19T18:23:22.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.229+0000 7fc655315640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65014e700 con 0x7fc65015ac50 2023-12-19T18:23:22.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.229+0000 7fc655315640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc650159520 con 0x7fc650070fb0 2023-12-19T18:23:22.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.229+0000 7fc64ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65006b850 0x7fc65014e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:22.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.229+0000 7fc64ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65006b850 0x7fc65014e1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60360/0 (socket says 172.21.15.27:60360) 2023-12-19T18:23:22.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.229+0000 7fc64ffff640 1 -- 172.21.15.27:0/2162366608 learned_addr learned my addr 172.21.15.27:0/2162366608 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:22.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.229+0000 7fc64ffff640 1 -- 172.21.15.27:0/2162366608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc650070fb0 msgr2=0x7fc6500ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:22.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.229+0000 7fc64ffff640 1 --2- 172.21.15.27:0/2162366608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc650070fb0 0x7fc6500ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:22.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.229+0000 7fc64ffff640 1 -- 172.21.15.27:0/2162366608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc65015ac50 msgr2=0x7fc6500ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:22.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.229+0000 7fc64ffff640 1 --2- 172.21.15.27:0/2162366608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc65015ac50 0x7fc6500ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.229+0000 7fc64ffff640 1 -- 172.21.15.27:0/2162366608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc638009400 con 0x7fc65006b850 2023-12-19T18:23:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.230+0000 7fc64ffff640 1 --2- 172.21.15.27:0/2162366608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65006b850 0x7fc65014e1c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc64000c4d0 tx=0x7fc64000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.230+0000 7fc64d7fa640 1 -- 172.21.15.27:0/2162366608 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6400074e0 con 0x7fc65006b850 2023-12-19T18:23:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.230+0000 7fc64d7fa640 1 -- 172.21.15.27:0/2162366608 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc640007660 con 0x7fc65006b850 2023-12-19T18:23:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.230+0000 7fc64d7fa640 1 -- 172.21.15.27:0/2162366608 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc640004a20 con 0x7fc65006b850 2023-12-19T18:23:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.230+0000 7fc655315640 1 -- 172.21.15.27:0/2162366608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc650159800 con 0x7fc65006b850 2023-12-19T18:23:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.230+0000 7fc655315640 1 -- 172.21.15.27:0/2162366608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc650159dc0 con 0x7fc65006b850 2023-12-19T18:23:22.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.231+0000 7fc655315640 1 -- 172.21.15.27:0/2162366608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6501517d0 con 0x7fc65006b850 2023-12-19T18:23:22.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.232+0000 7fc64d7fa640 1 -- 172.21.15.27:0/2162366608 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc640004b80 con 0x7fc65006b850 2023-12-19T18:23:22.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.234+0000 7fc64d7fa640 1 --2- 172.21.15.27:0/2162366608 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc63006a430 0x7fc63006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.234+0000 7fc64d7fa640 1 -- 172.21.15.27:0/2162366608 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(153..153 src has 1..153) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc640087ab0 con 0x7fc65006b850 2023-12-19T18:23:22.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.234+0000 7fc6477fe640 1 --2- 172.21.15.27:0/2162366608 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc63006a430 0x7fc63006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:22.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.236+0000 7fc6477fe640 1 --2- 172.21.15.27:0/2162366608 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc63006a430 0x7fc63006c8f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc638036000 tx=0x7fc638030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:22.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.239+0000 7fc64d7fa640 1 -- 172.21.15.27:0/2162366608 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc6501517d0 con 0x7fc65006b850 2023-12-19T18:23:22.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.432+0000 7fc655315640 1 -- 172.21.15.27:0/2162366608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc6501517d0 con 0x7fc65006b850 2023-12-19T18:23:22.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.436+0000 7fc64d7fa640 1 -- 172.21.15.27:0/2162366608 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v153) v1 ==== 81+0+33558 (secure 0 0 0) 0x7fc6501517d0 con 0x7fc65006b850 2023-12-19T18:23:22.437 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:22.437 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:23:22.437 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 153, 2023-12-19T18:23:22.437 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:23:22.437 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:23:22.437 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:23:20.743799+0000", 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:23:22.438 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 17, 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:22.439 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:22.440 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:22.441 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:22.442 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:22.443 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:22.443 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:22.443 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:22.443 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:22.443 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:22.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.443 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:22.443 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:22.443 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:22.443 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:22.443 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.444 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 16, 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:23:22.445 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:04.431250+0000", 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:23:22.446 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "151", 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:22.447 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:22.448 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:22.448 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:22.448 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:22.448 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:22.448 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:22.448 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:22.448 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:22.448 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:22.449 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:22.450 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.25, 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.25, 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2023-12-19T18:23:22.451 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 17, 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:05.506926+0000", 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:22.452 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:22.453 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:22.453 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:22.453 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:22.453 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:22.453 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:22.453 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:23:22.453 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:23:22.453 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:23:22.453 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:23:22.453 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:23:22.453 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:22.453 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:22.453 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:22.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:22.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:22.454 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:22.454 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:22.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "153", 2023-12-19T18:23:22.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:22.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:22.455 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:22.455 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:22.455 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:22.455 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:22.455 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:22.455 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:22.455 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:22.455 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:23:22.455 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:22.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:22.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:22.455 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:22.456 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.457 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:22.458 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:22.458 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:23:22.458 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.458 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:22.458 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:23:22.458 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:23:22.458 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:22.458 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:23:22.458 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:23:22.459 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:22.459 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:22.459 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:22.459 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.459 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.459 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:22.459 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:23:22.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.459 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:22.459 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:22.459 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:22.460 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:22.460 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:22.460 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:22.460 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:22.460 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:22.460 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:22.460 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:22.460 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:22.460 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:22.460 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:22.460 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:23:22.462 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:22.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.462 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:23:22.462 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:22.462 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.462 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.462 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:22.462 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:23:22.464 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:22.465 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:22.465 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:22.465 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.465 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:22.465 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:22.465 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:22.465 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:22.465 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:22.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.468 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:23:22.468 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:22.468 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.468 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.468 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:22.468 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.468 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:23:22.468 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:22.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.469 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:23:22.469 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:22.469 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.469 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:22.471 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.473 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:23:22.473 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:22.473 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.473 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.473 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:22.473 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.473 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:23:22.473 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:22.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.474 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:23:22.474 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:22.474 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.474 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.474 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:22.474 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:22.475 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.476 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:23:22.476 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:22.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.476 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:23:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:23:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:23:22.477 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:23:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:23:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:22.478 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:22.479 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:22.479 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:22.479 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:22.479 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:22.479 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:22.479 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:22.479 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:22.480 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:22.480 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.480 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.480 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:23:22.480 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:22.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.480 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.480 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:23:22.481 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:22.481 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.481 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.481 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:22.481 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.481 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:23:22.481 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:22.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.482 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:23:22.482 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:22.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.482 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.482 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:22.482 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.482 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:23:22.483 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:22.484 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:22.485 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:22.485 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:22.485 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:22.485 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:22.485 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:22.485 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:22.485 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:22.485 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:22.485 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:22.486 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:22.486 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:22.486 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:22.486 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:22.486 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.486 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:23:22.486 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:22.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.486 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:23:22.487 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:22.487 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.487 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.487 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:22.487 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.487 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:23:22.488 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:22.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.490 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.490 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:23:22.490 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:22.490 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.490 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.490 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:23:22.490 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:23:22.490 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:23:22.491 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:23:22.491 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:22.491 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:22.491 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:22.491 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:22.491 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:22.491 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:22.491 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:22.491 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:22.492 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:22.492 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:22.492 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:22.492 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:22.492 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:22.492 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:22.492 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:22.492 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:22.493 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.493 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:23:22.493 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:22.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.494 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:23:22.494 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:22.494 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.494 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.494 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:22.495 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.495 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.495 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:23:22.495 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:22.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.496 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.496 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:23:22.496 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:22.496 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.496 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.496 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:22.497 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.497 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:23:22.497 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:22.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.498 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:23:22.498 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:22.498 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.498 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.498 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:22.498 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.499 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:23:22.499 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:22.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.499 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:23:22.500 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:22.500 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.500 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.500 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:23:22.500 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:23:22.501 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:23:22.501 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:23:22.501 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:22.501 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:22.501 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:22.501 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.501 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.501 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.502 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:23:22.503 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.505 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:23:22.505 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:22.505 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.505 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.505 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:22.505 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.505 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.505 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.505 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:22.506 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:22.507 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.510 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:22.510 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.510 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:23:22.510 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:22.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.510 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.510 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:23:22.510 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:22.510 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.510 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.511 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:22.511 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:22.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:22.511 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:23:22.511 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:22.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.511 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:22.511 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:23:22.511 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:22.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.512 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.512 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:23:22.512 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:23:22.512 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:23:22.512 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:23:22.512 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:22.512 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:22.512 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:22.512 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:22.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.512 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:22.513 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:23:22.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.513 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:22.513 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:22.513 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:22.513 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:22.513 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:22.513 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:22.513 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:23:22.513 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:22.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.513 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.513 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:22.514 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:22.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:22.516 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:23:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:22.517 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.518 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:23:22.519 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.520 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:22.520 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:23:22.520 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:23:22.520 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:23:22.520 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:23:22.520 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:23:22.520 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:23:22.520 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:23:22.520 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:23:22.520 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:23:22.520 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:23:22.520 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:23:22.520 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:22.521 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:23:22.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.439+0000 7fc655315640 1 -- 172.21.15.27:0/2162366608 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc63006a430 msgr2=0x7fc63006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:22.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.439+0000 7fc655315640 1 --2- 172.21.15.27:0/2162366608 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc63006a430 0x7fc63006c8f0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc638036000 tx=0x7fc638030000 comp rx=0 tx=0).stop 2023-12-19T18:23:22.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.439+0000 7fc655315640 1 -- 172.21.15.27:0/2162366608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65006b850 msgr2=0x7fc65014e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:22.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.439+0000 7fc655315640 1 --2- 172.21.15.27:0/2162366608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65006b850 0x7fc65014e1c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7fc64000c4d0 tx=0x7fc64000c990 comp rx=0 tx=0).stop 2023-12-19T18:23:22.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.439+0000 7fc655315640 1 -- 172.21.15.27:0/2162366608 shutdown_connections 2023-12-19T18:23:22.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.439+0000 7fc655315640 1 --2- 172.21.15.27:0/2162366608 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc63006a430 0x7fc63006c8f0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:22.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.439+0000 7fc655315640 1 --2- 172.21.15.27:0/2162366608 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc65015ac50 0x7fc6500ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:22.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.439+0000 7fc655315640 1 --2- 172.21.15.27:0/2162366608 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc650070fb0 0x7fc6500ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:22.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.439+0000 7fc655315640 1 --2- 172.21.15.27:0/2162366608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc65006b850 0x7fc65014e1c0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:22.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.439+0000 7fc655315640 1 -- 172.21.15.27:0/2162366608 >> 172.21.15.27:0/2162366608 conn(0x7fc65006a7d0 msgr2=0x7fc6500cc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:22.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.439+0000 7fc655315640 1 -- 172.21.15.27:0/2162366608 shutdown_connections 2023-12-19T18:23:22.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:22.439+0000 7fc655315640 1 -- 172.21.15.27:0/2162366608 wait complete. 2023-12-19T18:23:22.525 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-12-19T18:23:27.526 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:23:27.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.764+0000 7fe6b88b4640 1 Processor -- start 2023-12-19T18:23:27.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.765+0000 7fe6b88b4640 1 -- start start 2023-12-19T18:23:27.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.766+0000 7fe6b88b4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe6b40b49d0 0x7fe6b4160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:27.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.766+0000 7fe6b88b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40b89b0 0x7fe6b415c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:27.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.766+0000 7fe6b88b4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe6b40b4db0 0x7fe6b415cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:27.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.766+0000 7fe6b88b4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b415d300 con 0x7fe6b40b49d0 2023-12-19T18:23:27.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.766+0000 7fe6b88b4640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6b40b5ca0 con 0x7fe6b40b4db0 2023-12-19T18:23:27.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.766+0000 7fe6b88b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b4161800 con 0x7fe6b40b89b0 2023-12-19T18:23:27.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.766+0000 7fe6b37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe6b40b49d0 0x7fe6b4160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:27.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.767+0000 7fe6b37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe6b40b49d0 0x7fe6b4160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60370/0 (socket says 172.21.15.27:60370) 2023-12-19T18:23:27.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.767+0000 7fe6b37fe640 1 -- 172.21.15.27:0/3277202376 learned_addr learned my addr 172.21.15.27:0/3277202376 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:27.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.767+0000 7fe6abfff640 1 --2- 172.21.15.27:0/3277202376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40b89b0 0x7fe6b415c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:27.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.767+0000 7fe6b3fff640 1 --2- 172.21.15.27:0/3277202376 >> v2:172.21.15.192:3300/0 conn(0x7fe6b40b4db0 0x7fe6b415cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:27.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.767+0000 7fe6b37fe640 1 -- 172.21.15.27:0/3277202376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40b89b0 msgr2=0x7fe6b415c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:27.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.767+0000 7fe6b37fe640 1 --2- 172.21.15.27:0/3277202376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40b89b0 0x7fe6b415c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:27.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.767+0000 7fe6b37fe640 1 -- 172.21.15.27:0/3277202376 >> v2:172.21.15.192:3300/0 conn(0x7fe6b40b4db0 msgr2=0x7fe6b415cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:27.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.767+0000 7fe6b37fe640 1 --2- 172.21.15.27:0/3277202376 >> v2:172.21.15.192:3300/0 conn(0x7fe6b40b4db0 0x7fe6b415cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:27.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.767+0000 7fe6b37fe640 1 -- 172.21.15.27:0/3277202376 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6b4161240 con 0x7fe6b40b49d0 2023-12-19T18:23:27.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.767+0000 7fe6b3fff640 1 --2- 172.21.15.27:0/3277202376 >> v2:172.21.15.192:3300/0 conn(0x7fe6b40b4db0 0x7fe6b415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:27.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.768+0000 7fe6b37fe640 1 --2- 172.21.15.27:0/3277202376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b40b49d0 0x7fe6b4160940 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40b5bc0 tx=0x7fe6a402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f6b7aa514d48639 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:27.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.769+0000 7fe6b2ffd640 1 -- 172.21.15.27:0/3277202376 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4004230 con 0x7fe6b40b49d0 2023-12-19T18:23:27.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.769+0000 7fe6b2ffd640 1 -- 172.21.15.27:0/3277202376 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a4004390 con 0x7fe6b40b49d0 2023-12-19T18:23:27.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.769+0000 7fe6b2ffd640 1 -- 172.21.15.27:0/3277202376 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4030840 con 0x7fe6b40b49d0 2023-12-19T18:23:27.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.769+0000 7fe6b88b4640 1 -- 172.21.15.27:0/3277202376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b40b49d0 msgr2=0x7fe6b4160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:27.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.769+0000 7fe6b88b4640 1 --2- 172.21.15.27:0/3277202376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b40b49d0 0x7fe6b4160940 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7fe6b40b5bc0 tx=0x7fe6a402b980 comp rx=0 tx=0).stop 2023-12-19T18:23:27.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.769+0000 7fe6b88b4640 1 -- 172.21.15.27:0/3277202376 shutdown_connections 2023-12-19T18:23:27.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.769+0000 7fe6b88b4640 1 --2- 172.21.15.27:0/3277202376 >> v2:172.21.15.192:3300/0 conn(0x7fe6b40b4db0 0x7fe6b415cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:27.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.769+0000 7fe6b88b4640 1 --2- 172.21.15.27:0/3277202376 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40b89b0 0x7fe6b415c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:27.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.769+0000 7fe6b88b4640 1 --2- 172.21.15.27:0/3277202376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b40b49d0 0x7fe6b4160940 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:27.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.769+0000 7fe6b88b4640 1 -- 172.21.15.27:0/3277202376 >> 172.21.15.27:0/3277202376 conn(0x7fe6b40cf760 msgr2=0x7fe6b40cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:27.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.770+0000 7fe6b88b4640 1 -- 172.21.15.27:0/3277202376 shutdown_connections 2023-12-19T18:23:27.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.770+0000 7fe6b88b4640 1 -- 172.21.15.27:0/3277202376 wait complete. 2023-12-19T18:23:27.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.771+0000 7fe6b88b4640 1 Processor -- start 2023-12-19T18:23:27.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.772+0000 7fe6b88b4640 1 -- start start 2023-12-19T18:23:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b88b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b40b49d0 0x7fe6b415f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b88b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40b4db0 0x7fe6b415b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b88b4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b40b89b0 0x7fe6b413a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b88b4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b415e5c0 con 0x7fe6b40b49d0 2023-12-19T18:23:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b88b4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b415e860 con 0x7fe6b40b89b0 2023-12-19T18:23:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b88b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6b415eb40 con 0x7fe6b40b4db0 2023-12-19T18:23:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b40b49d0 0x7fe6b415f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b40b49d0 0x7fe6b415f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60382/0 (socket says 172.21.15.27:60382) 2023-12-19T18:23:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b37fe640 1 -- 172.21.15.27:0/667026684 learned_addr learned my addr 172.21.15.27:0/667026684 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b3fff640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b40b89b0 0x7fe6b413a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6abfff640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40b4db0 0x7fe6b415b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b37fe640 1 -- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40b4db0 msgr2=0x7fe6b415b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b37fe640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40b4db0 0x7fe6b415b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:27.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b37fe640 1 -- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b40b89b0 msgr2=0x7fe6b413a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:27.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.773+0000 7fe6b37fe640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b40b89b0 0x7fe6b413a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:27.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.774+0000 7fe6b37fe640 1 -- 172.21.15.27:0/667026684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a4009460 con 0x7fe6b40b49d0 2023-12-19T18:23:27.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.774+0000 7fe6b3fff640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b40b89b0 0x7fe6b413a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:27.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.774+0000 7fe6b37fe640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b40b49d0 0x7fe6b415f620 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4000c00 tx=0x7fe6a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:27.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.774+0000 7fe6b0ff9640 1 -- 172.21.15.27:0/667026684 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a403e070 con 0x7fe6b40b49d0 2023-12-19T18:23:27.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.774+0000 7fe6b0ff9640 1 -- 172.21.15.27:0/667026684 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6a4004500 con 0x7fe6b40b49d0 2023-12-19T18:23:27.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.774+0000 7fe6b88b4640 1 -- 172.21.15.27:0/667026684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6b415ee20 con 0x7fe6b40b49d0 2023-12-19T18:23:27.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.774+0000 7fe6b0ff9640 1 -- 172.21.15.27:0/667026684 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6a4030e40 con 0x7fe6b40b49d0 2023-12-19T18:23:27.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.774+0000 7fe6b88b4640 1 -- 172.21.15.27:0/667026684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6b415fb60 con 0x7fe6b40b49d0 2023-12-19T18:23:27.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.776+0000 7fe6b88b4640 1 -- 172.21.15.27:0/667026684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6b404e840 con 0x7fe6b40b49d0 2023-12-19T18:23:27.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.777+0000 7fe6b0ff9640 1 -- 172.21.15.27:0/667026684 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe6a4003610 con 0x7fe6b40b49d0 2023-12-19T18:23:27.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.778+0000 7fe6b0ff9640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe69406a430 0x7fe69406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:27.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.778+0000 7fe6b0ff9640 1 -- 172.21.15.27:0/667026684 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe6a40aa9c0 con 0x7fe6b40b49d0 2023-12-19T18:23:27.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.778+0000 7fe6b0ff9640 1 -- 172.21.15.27:0/667026684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7fe694071f60 con 0x7fe6b40b49d0 2023-12-19T18:23:27.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.779+0000 7fe6abfff640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe69406a430 0x7fe69406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:27.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.780+0000 7fe6abfff640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe69406a430 0x7fe69406c8f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe69c006fd0 tx=0x7fe69c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:27.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.783+0000 7fe6b0ff9640 1 -- 172.21.15.27:0/667026684 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe6a4076040 con 0x7fe6b40b49d0 2023-12-19T18:23:27.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.980+0000 7fe6b88b4640 1 -- 172.21.15.27:0/667026684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe6b40d08b0 con 0x7fe6b40b49d0 2023-12-19T18:23:27.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.982+0000 7fe6b0ff9640 1 -- 172.21.15.27:0/667026684 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v154) v1 ==== 81+0+33577 (secure 0 0 0) 0x7fe6a4048070 con 0x7fe6b40b49d0 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 154, 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:23:26.707491+0000", 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:23:27.983 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:23:27.984 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:23:27.984 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:27.984 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:23:27.984 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:23:27.984 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:23:27.984 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:23:27.984 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:23:27.984 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 17, 2023-12-19T18:23:27.984 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:23:27.984 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:23:27.984 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:23:27.984 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:27.985 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:27.986 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:27.986 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:27.986 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:27.986 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:23:27.986 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:23:27.986 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:23:27.986 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:23:27.986 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:23:27.986 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:27.986 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:27.986 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:27.986 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:27.987 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:27.988 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:27.989 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:27.990 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 16, 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:04.431250+0000", 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:27.991 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:27.992 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:27.992 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:27.992 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:27.992 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:23:27.992 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:23:27.992 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:23:27.992 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:23:27.992 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:23:27.992 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:27.992 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:27.992 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:27.992 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "151", 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:27.993 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:27.994 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:27.995 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.25, 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.25, 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2023-12-19T18:23:27.996 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2023-12-19T18:23:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 17, 2023-12-19T18:23:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:23:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:05.506926+0000", 2023-12-19T18:23:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1027, 2023-12-19T18:23:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full,full_quota", 2023-12-19T18:23:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:27.997 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:27.998 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "154", 2023-12-19T18:23:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:27.999 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:28.000 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:28.001 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:28.002 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.003 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.004 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:23:28.005 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.006 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.007 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.008 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.010 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.011 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:23:28.012 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:28.013 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.014 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.015 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.016 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.018 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.019 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.020 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.021 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.022 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.023 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.024 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.025 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.026 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.027 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:23:28.028 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.029 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.030 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.031 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:23:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:23:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.033 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.034 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.036 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.037 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.038 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:28.039 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.040 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:23:28.041 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.042 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.043 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.044 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.045 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.046 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.047 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.048 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.049 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:23:28.050 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:23:28.051 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:23:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:23:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:23:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:23:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:23:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:23:28.052 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.052 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:23:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.985+0000 7fe6b88b4640 1 -- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe69406a430 msgr2=0x7fe69406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.985+0000 7fe6b88b4640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe69406a430 0x7fe69406c8f0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fe69c006fd0 tx=0x7fe69c008040 comp rx=0 tx=0).stop 2023-12-19T18:23:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.985+0000 7fe6b88b4640 1 -- 172.21.15.27:0/667026684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b40b49d0 msgr2=0x7fe6b415f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.985+0000 7fe6b88b4640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b40b49d0 0x7fe6b415f620 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4000c00 tx=0x7fe6a40046c0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.985+0000 7fe6b88b4640 1 -- 172.21.15.27:0/667026684 shutdown_connections 2023-12-19T18:23:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.985+0000 7fe6b88b4640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe69406a430 0x7fe69406c8f0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.985+0000 7fe6b88b4640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe6b40b89b0 0x7fe6b413a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.985+0000 7fe6b88b4640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe6b40b4db0 0x7fe6b415b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.985+0000 7fe6b88b4640 1 --2- 172.21.15.27:0/667026684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6b40b49d0 0x7fe6b415f620 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.985+0000 7fe6b88b4640 1 -- 172.21.15.27:0/667026684 >> 172.21.15.27:0/667026684 conn(0x7fe6b40cf760 msgr2=0x7fe6b415c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.985+0000 7fe6b88b4640 1 -- 172.21.15.27:0/667026684 shutdown_connections 2023-12-19T18:23:28.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:27.985+0000 7fe6b88b4640 1 -- 172.21.15.27:0/667026684 wait complete. 2023-12-19T18:23:28.055 DEBUG:tasks.ceph_test_case:wait_until_true: success in 5s and 0 retries 2023-12-19T18:23:28.055 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:23:28.055 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:23:28.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:28.118 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-12-19T18:23:28.364 INFO:teuthology.orchestra.run.smithi027.stderr:dd: error writing 'large_file_b': No space left on device 2023-12-19T18:23:28.364 INFO:teuthology.orchestra.run.smithi027.stderr:dd: closing output file 'large_file_b': No space left on device 2023-12-19T18:23:28.366 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:23:28.367 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:23:28.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.606+0000 7fabb2f1f640 1 Processor -- start 2023-12-19T18:23:28.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.606+0000 7fabb2f1f640 1 -- start start 2023-12-19T18:23:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.607+0000 7fabb2f1f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabac0b49d0 0x7fabac160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.607+0000 7fabb2f1f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fabac0b89b0 0x7fabac15c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.607+0000 7fabb2f1f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabac0b5ca0 con 0x7fabac0b4db0 2023-12-19T18:23:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.607+0000 7fabb2f1f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabac161800 con 0x7fabac0b89b0 2023-12-19T18:23:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.607+0000 7fabb2f1f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac161240 con 0x7fabac0b49d0 2023-12-19T18:23:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.608+0000 7fabb171c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fabac0b89b0 0x7fabac15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.608+0000 7fabb171c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fabac0b89b0 0x7fabac15c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60386/0 (socket says 172.21.15.27:60386) 2023-12-19T18:23:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.608+0000 7fabb171c640 1 -- 172.21.15.27:0/4272448122 learned_addr learned my addr 172.21.15.27:0/4272448122 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.608+0000 7fabb271e640 1 --1- 172.21.15.27:0/4272448122 >> v1:172.21.15.27:6789/0 conn(0x7fabac0b4db0 0x7fabac06bc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47712/0 (socket says 172.21.15.27:47712) 2023-12-19T18:23:28.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.608+0000 7fabb1f1d640 1 --2- 172.21.15.27:0/4272448122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabac0b49d0 0x7fabac160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:28.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.608+0000 7fabb171c640 1 -- 172.21.15.27:0/4272448122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabac0b49d0 msgr2=0x7fabac160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:28.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.608+0000 7fabb171c640 1 --2- 172.21.15.27:0/4272448122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabac0b49d0 0x7fabac160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.608+0000 7fabb171c640 1 -- 172.21.15.27:0/4272448122 >> v1:172.21.15.27:6789/0 conn(0x7fabac0b4db0 legacy=0x7fabac06bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:28.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.608+0000 7fabb171c640 1 -- 172.21.15.27:0/4272448122 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabac161520 con 0x7fabac0b89b0 2023-12-19T18:23:28.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.608+0000 7fabb0f1b640 1 -- 172.21.15.27:0/4272448122 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2230775859 0 0) 0x7fabac0b5ca0 con 0x7fabac0b4db0 2023-12-19T18:23:28.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.609+0000 7fabb171c640 1 --2- 172.21.15.27:0/4272448122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabac0b89b0 0x7fabac15c880 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fab98009870 tx=0x7fab9802ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b60e5257f44cb13 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:28.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.610+0000 7fabb0f1b640 1 -- 172.21.15.27:0/4272448122 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9802f830 con 0x7fabac0b89b0 2023-12-19T18:23:28.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.610+0000 7fabb0f1b640 1 -- 172.21.15.27:0/4272448122 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab9802f990 con 0x7fabac0b89b0 2023-12-19T18:23:28.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.610+0000 7fabb0f1b640 1 -- 172.21.15.27:0/4272448122 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9802fd20 con 0x7fabac0b89b0 2023-12-19T18:23:28.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.610+0000 7fabb2f1f640 1 -- 172.21.15.27:0/4272448122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabac0b89b0 msgr2=0x7fabac15c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:28.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.610+0000 7fabb2f1f640 1 --2- 172.21.15.27:0/4272448122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabac0b89b0 0x7fabac15c880 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fab98009870 tx=0x7fab9802ba10 comp rx=0 tx=0).stop 2023-12-19T18:23:28.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.610+0000 7fabb2f1f640 1 -- 172.21.15.27:0/4272448122 shutdown_connections 2023-12-19T18:23:28.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.610+0000 7fabb2f1f640 1 --2- 172.21.15.27:0/4272448122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabac0b89b0 0x7fabac15c880 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.610+0000 7fabb2f1f640 1 --2- 172.21.15.27:0/4272448122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabac0b49d0 0x7fabac160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.610+0000 7fabb2f1f640 1 -- 172.21.15.27:0/4272448122 >> 172.21.15.27:0/4272448122 conn(0x7fabac0cf760 msgr2=0x7fabac0cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:28.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.610+0000 7fabb2f1f640 1 -- 172.21.15.27:0/4272448122 shutdown_connections 2023-12-19T18:23:28.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.611+0000 7fabb2f1f640 1 -- 172.21.15.27:0/4272448122 wait complete. 2023-12-19T18:23:28.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.612+0000 7fabb2f1f640 1 Processor -- start 2023-12-19T18:23:28.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.612+0000 7fabb2f1f640 1 -- start start 2023-12-19T18:23:28.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.613+0000 7fabb2f1f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabac0b49d0 0x7fabac1f2000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:28.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.613+0000 7fabb2f1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabac0b4db0 0x7fabac148f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:28.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.613+0000 7fabb2f1f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabac0b89b0 0x7fabac157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:28.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.613+0000 7fabb2f1f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0ca990 con 0x7fabac0b4db0 2023-12-19T18:23:28.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.613+0000 7fabb2f1f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0cac30 con 0x7fabac0b89b0 2023-12-19T18:23:28.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.613+0000 7fabb2f1f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabac0caf10 con 0x7fabac0b49d0 2023-12-19T18:23:28.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.614+0000 7fabb171c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabac0b4db0 0x7fabac148f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:28.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.614+0000 7fabb171c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabac0b4db0 0x7fabac148f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60396/0 (socket says 172.21.15.27:60396) 2023-12-19T18:23:28.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.614+0000 7fabb171c640 1 -- 172.21.15.27:0/3785913485 learned_addr learned my addr 172.21.15.27:0/3785913485 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:28.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.614+0000 7fabb1f1d640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabac0b49d0 0x7fabac1f2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:28.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.614+0000 7fabb271e640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabac0b89b0 0x7fabac157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.614+0000 7fabb171c640 1 -- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabac0b49d0 msgr2=0x7fabac1f2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.614+0000 7fabb171c640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabac0b49d0 0x7fabac1f2000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.614+0000 7fabb171c640 1 -- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabac0b89b0 msgr2=0x7fabac157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.614+0000 7fabb171c640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabac0b89b0 0x7fabac157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.614+0000 7fabb171c640 1 -- 172.21.15.27:0/3785913485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab980094d0 con 0x7fabac0b4db0 2023-12-19T18:23:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.614+0000 7fabb271e640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabac0b89b0 0x7fabac157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.615+0000 7fabb171c640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabac0b4db0 0x7fabac148f90 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fab9802ba50 tx=0x7fab98004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.615+0000 7fabaa7fc640 1 -- 172.21.15.27:0/3785913485 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab9803c040 con 0x7fabac0b4db0 2023-12-19T18:23:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.615+0000 7fabaa7fc640 1 -- 172.21.15.27:0/3785913485 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab98037c60 con 0x7fabac0b4db0 2023-12-19T18:23:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.615+0000 7fabb2f1f640 1 -- 172.21.15.27:0/3785913485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabac0cb1f0 con 0x7fabac0b4db0 2023-12-19T18:23:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.615+0000 7fabaa7fc640 1 -- 172.21.15.27:0/3785913485 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab980423f0 con 0x7fabac0b4db0 2023-12-19T18:23:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.615+0000 7fabb2f1f640 1 -- 172.21.15.27:0/3785913485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac1494d0 con 0x7fabac0b4db0 2023-12-19T18:23:28.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.616+0000 7fabb2f1f640 1 -- 172.21.15.27:0/3785913485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabac0d0d80 con 0x7fabac0b4db0 2023-12-19T18:23:28.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.617+0000 7fabaa7fc640 1 -- 172.21.15.27:0/3785913485 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fab9802d3c0 con 0x7fabac0b4db0 2023-12-19T18:23:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.618+0000 7fabaa7fc640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fab9006a430 0x7fab9006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.618+0000 7fabaa7fc640 1 -- 172.21.15.27:0/3785913485 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab980aaa20 con 0x7fabac0b4db0 2023-12-19T18:23:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.619+0000 7fabaa7fc640 1 -- 172.21.15.27:0/3785913485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7fab90071f60 con 0x7fabac0b4db0 2023-12-19T18:23:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.619+0000 7fabb1f1d640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fab9006a430 0x7fab9006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:28.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.620+0000 7fabb1f1d640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fab9006a430 0x7fab9006c8f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7faba0009f80 tx=0x7faba000b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.623+0000 7fabaa7fc640 1 -- 172.21.15.27:0/3785913485 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fab98047060 con 0x7fabac0b4db0 2023-12-19T18:23:28.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.818+0000 7fabb2f1f640 1 -- 172.21.15.27:0/3785913485 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fabac0cb710 con 0x7fabac0b4db0 2023-12-19T18:23:28.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.821+0000 7fabaa7fc640 1 -- 172.21.15.27:0/3785913485 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v154) v1 ==== 81+0+33577 (secure 0 0 0) 0x7fabac0cb710 con 0x7fabac0b4db0 2023-12-19T18:23:28.822 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:28.822 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:23:28.822 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 154, 2023-12-19T18:23:28.822 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:23:28.822 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:23:28.822 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:23:26.707491+0000", 2023-12-19T18:23:28.822 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:23:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 17, 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:28.829 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 16, 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:04.431250+0000", 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:28.830 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:28.831 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "151", 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:28.832 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:28.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:23:28.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.25, 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.25, 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:28.835 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 17, 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:05.506926+0000", 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1027, 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full,full_quota", 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:28.836 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.837 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "154", 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:28.838 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:28.839 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:23:28.840 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:23:28.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.842 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:28.842 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.842 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.842 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.842 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.842 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.842 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.842 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.842 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.842 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.842 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.842 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.843 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.843 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.843 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:23:28.843 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.843 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:23:28.844 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.845 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.847 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.850 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:28.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.852 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:23:28.853 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.854 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:23:28.854 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.854 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.854 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.854 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.854 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:23:28.856 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.857 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.858 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.859 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.860 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.861 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.862 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.863 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:23:28.866 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.867 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.868 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.869 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.870 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.871 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.872 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.873 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.874 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.875 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:23:28.877 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:28.878 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:28.880 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:28.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:23:28.882 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:23:28.883 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.884 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.885 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.886 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:23:28.888 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:23:28.889 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:23:28.890 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:23:28.890 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:23:28.890 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:23:28.890 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:23:28.890 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:23:28.890 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:23:28.890 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:28.890 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:23:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.825+0000 7fabb2f1f640 1 -- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fab9006a430 msgr2=0x7fab9006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:28.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.825+0000 7fabb2f1f640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fab9006a430 0x7fab9006c8f0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7faba0009f80 tx=0x7faba000b040 comp rx=0 tx=0).stop 2023-12-19T18:23:28.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.825+0000 7fabb2f1f640 1 -- 172.21.15.27:0/3785913485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabac0b4db0 msgr2=0x7fabac148f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:28.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.825+0000 7fabb2f1f640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabac0b4db0 0x7fabac148f90 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fab9802ba50 tx=0x7fab98004730 comp rx=0 tx=0).stop 2023-12-19T18:23:28.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.826+0000 7fabb2f1f640 1 -- 172.21.15.27:0/3785913485 shutdown_connections 2023-12-19T18:23:28.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.826+0000 7fabb2f1f640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fab9006a430 0x7fab9006c8f0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.826+0000 7fabb2f1f640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fabac0b89b0 0x7fabac157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.826+0000 7fabb2f1f640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabac0b4db0 0x7fabac148f90 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.826+0000 7fabb2f1f640 1 --2- 172.21.15.27:0/3785913485 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fabac0b49d0 0x7fabac1f2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:28.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.826+0000 7fabb2f1f640 1 -- 172.21.15.27:0/3785913485 >> 172.21.15.27:0/3785913485 conn(0x7fabac0cf760 msgr2=0x7fabac0b6370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:28.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.826+0000 7fabb2f1f640 1 -- 172.21.15.27:0/3785913485 shutdown_connections 2023-12-19T18:23:28.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:28.826+0000 7fabb2f1f640 1 -- 172.21.15.27:0/3785913485 wait complete. 2023-12-19T18:23:28.893 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:23:29.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.134+0000 7fce3ffff640 1 Processor -- start 2023-12-19T18:23:29.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.134+0000 7fce3ffff640 1 -- start start 2023-12-19T18:23:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.136+0000 7fce3ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fce400c50a0 0x7fce400afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.136+0000 7fce3ffff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce400c54f0 con 0x7fce401553a0 2023-12-19T18:23:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.136+0000 7fce3ffff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce400cb350 con 0x7fce4015ee30 2023-12-19T18:23:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.136+0000 7fce3ffff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce400cbcf0 con 0x7fce400c50a0 2023-12-19T18:23:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.136+0000 7fce3effd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fce401553a0 0x7fce4015e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47718/0 (socket says 172.21.15.27:47718) 2023-12-19T18:23:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.136+0000 7fce3effd640 1 -- 172.21.15.27:0/1604666923 learned_addr learned my addr 172.21.15.27:0/1604666923 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.136+0000 7fce3f7fe640 1 --2- 172.21.15.27:0/1604666923 >> v2:172.21.15.27:3300/0 conn(0x7fce400c50a0 0x7fce400afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.136+0000 7fce3dffb640 1 -- 172.21.15.27:0/1604666923 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196198428 0 0) 0x7fce30001110 con 0x7fce401553a0 2023-12-19T18:23:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.136+0000 7fce3dffb640 1 -- 172.21.15.27:0/1604666923 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce28003520 con 0x7fce401553a0 2023-12-19T18:23:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.137+0000 7fce3f7fe640 1 -- 172.21.15.27:0/1604666923 >> v1:172.21.15.192:6789/0 conn(0x7fce4015ee30 legacy=0x7fce400c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.137+0000 7fce3f7fe640 1 -- 172.21.15.27:0/1604666923 >> v1:172.21.15.27:6789/0 conn(0x7fce401553a0 legacy=0x7fce4015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.137+0000 7fce3f7fe640 1 -- 172.21.15.27:0/1604666923 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce400cb630 con 0x7fce400c50a0 2023-12-19T18:23:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.138+0000 7fce3f7fe640 1 --2- 172.21.15.27:0/1604666923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce400c50a0 0x7fce400afeb0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fce38009d90 tx=0x7fce3802be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1951248dd72996ab server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:29.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.138+0000 7fce3dffb640 1 -- 172.21.15.27:0/1604666923 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3802cbd0 con 0x7fce400c50a0 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.138+0000 7fce3dffb640 1 -- 172.21.15.27:0/1604666923 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3802cd30 con 0x7fce400c50a0 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.138+0000 7fce3dffb640 1 -- 172.21.15.27:0/1604666923 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3802cea0 con 0x7fce400c50a0 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.139+0000 7fce3ffff640 1 -- 172.21.15.27:0/1604666923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce400c50a0 msgr2=0x7fce400afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.139+0000 7fce3ffff640 1 --2- 172.21.15.27:0/1604666923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce400c50a0 0x7fce400afeb0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7fce38009d90 tx=0x7fce3802be20 comp rx=0 tx=0).stop 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.139+0000 7fce3ffff640 1 -- 172.21.15.27:0/1604666923 shutdown_connections 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.139+0000 7fce3ffff640 1 --2- 172.21.15.27:0/1604666923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce400c50a0 0x7fce400afeb0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.139+0000 7fce3ffff640 1 -- 172.21.15.27:0/1604666923 >> 172.21.15.27:0/1604666923 conn(0x7fce40069c70 msgr2=0x7fce4006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.139+0000 7fce3ffff640 1 -- 172.21.15.27:0/1604666923 shutdown_connections 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.139+0000 7fce3ffff640 1 -- 172.21.15.27:0/1604666923 wait complete. 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.141+0000 7fce3ffff640 1 Processor -- start 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.141+0000 7fce3ffff640 1 -- start start 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.142+0000 7fce3ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce400c50a0 0x7fce400b4120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.142+0000 7fce3ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce401553a0 0x7fce4015c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:29.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.142+0000 7fce3ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4015ee30 0x7fce4014c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.142+0000 7fce3ffff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4015daa0 con 0x7fce4015ee30 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.142+0000 7fce3ffff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce4015cbc0 con 0x7fce401553a0 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.142+0000 7fce3ffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce4015ce90 con 0x7fce400c50a0 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.142+0000 7fce3f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4015ee30 0x7fce4014c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.142+0000 7fce3f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4015ee30 0x7fce4014c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60408/0 (socket says 172.21.15.27:60408) 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.142+0000 7fce3f7fe640 1 -- 172.21.15.27:0/2270897452 learned_addr learned my addr 172.21.15.27:0/2270897452 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce3e7fc640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce401553a0 0x7fce4015c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce3effd640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce400c50a0 0x7fce400b4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce3f7fe640 1 -- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce400c50a0 msgr2=0x7fce400b4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce3f7fe640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce400c50a0 0x7fce400b4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce3f7fe640 1 -- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce401553a0 msgr2=0x7fce4015c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce3f7fe640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce401553a0 0x7fce4015c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce3f7fe640 1 -- 172.21.15.27:0/2270897452 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce38009a70 con 0x7fce4015ee30 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce3e7fc640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce401553a0 0x7fce4015c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:29.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce3effd640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce400c50a0 0x7fce400b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:29.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce3f7fe640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4015ee30 0x7fce4014c310 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fce38009d90 tx=0x7fce38004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:29.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce277fe640 1 -- 172.21.15.27:0/2270897452 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3802cea0 con 0x7fce4015ee30 2023-12-19T18:23:29.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce277fe640 1 -- 172.21.15.27:0/2270897452 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce38040650 con 0x7fce4015ee30 2023-12-19T18:23:29.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce277fe640 1 -- 172.21.15.27:0/2270897452 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce38040a00 con 0x7fce4015ee30 2023-12-19T18:23:29.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.143+0000 7fce3ffff640 1 -- 172.21.15.27:0/2270897452 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce4015d170 con 0x7fce4015ee30 2023-12-19T18:23:29.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.144+0000 7fce3ffff640 1 -- 172.21.15.27:0/2270897452 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce4015d730 con 0x7fce4015ee30 2023-12-19T18:23:29.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.145+0000 7fce3ffff640 1 -- 172.21.15.27:0/2270897452 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce4015dc50 con 0x7fce4015ee30 2023-12-19T18:23:29.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.146+0000 7fce277fe640 1 -- 172.21.15.27:0/2270897452 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fce38040b60 con 0x7fce4015ee30 2023-12-19T18:23:29.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.146+0000 7fce277fe640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce2c06a430 0x7fce2c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:29.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.147+0000 7fce277fe640 1 -- 172.21.15.27:0/2270897452 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce380ab4c0 con 0x7fce4015ee30 2023-12-19T18:23:29.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.147+0000 7fce277fe640 1 -- 172.21.15.27:0/2270897452 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7fce2c071f60 con 0x7fce4015ee30 2023-12-19T18:23:29.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.147+0000 7fce3effd640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce2c06a430 0x7fce2c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:29.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.148+0000 7fce3effd640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce2c06a430 0x7fce2c06c8f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fce30001370 tx=0x7fce30009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:29.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.151+0000 7fce277fe640 1 -- 172.21.15.27:0/2270897452 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fce38076c00 con 0x7fce4015ee30 2023-12-19T18:23:29.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.381+0000 7fce3ffff640 1 -- 172.21.15.27:0/2270897452 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce4004e840 con 0x7fce4015ee30 2023-12-19T18:23:29.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.382+0000 7fce277fe640 1 -- 172.21.15.27:0/2270897452 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fce38044020 con 0x7fce4015ee30 2023-12-19T18:23:29.382 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:29.382 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":61,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3298228307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3298228307},{"type":"v1","addr":"172.21.15.192:6835","nonce":3298228307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5726,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1760483999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1760483999},{"type":"v1","addr":"172.21.15.192:6837","nonce":1760483999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3000637822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3000637822},{"type":"v1","addr":"172.21.15.27:6833","nonce":3000637822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:23:06.465885+0000","modified":"2023-12-19T18:23:09.973228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1593995318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1593995318},{"type":"v1","addr":"172.21.15.27:6835","nonce":1593995318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T18:23:29.382 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 61 2023-12-19T18:23:29.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.385+0000 7fce3ffff640 1 -- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce2c06a430 msgr2=0x7fce2c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:29.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.385+0000 7fce3ffff640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce2c06a430 0x7fce2c06c8f0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fce30001370 tx=0x7fce30009040 comp rx=0 tx=0).stop 2023-12-19T18:23:29.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.385+0000 7fce3ffff640 1 -- 172.21.15.27:0/2270897452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4015ee30 msgr2=0x7fce4014c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:29.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.385+0000 7fce3ffff640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4015ee30 0x7fce4014c310 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7fce38009d90 tx=0x7fce38004a40 comp rx=0 tx=0).stop 2023-12-19T18:23:29.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.385+0000 7fce3ffff640 1 -- 172.21.15.27:0/2270897452 shutdown_connections 2023-12-19T18:23:29.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.385+0000 7fce3ffff640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce2c06a430 0x7fce2c06c8f0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:29.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.385+0000 7fce3ffff640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce4015ee30 0x7fce4014c310 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:29.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.385+0000 7fce3ffff640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce401553a0 0x7fce4015c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:29.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.385+0000 7fce3ffff640 1 --2- 172.21.15.27:0/2270897452 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce400c50a0 0x7fce400b4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:29.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.385+0000 7fce3ffff640 1 -- 172.21.15.27:0/2270897452 >> 172.21.15.27:0/2270897452 conn(0x7fce40069c70 msgr2=0x7fce4006b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:29.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.385+0000 7fce3ffff640 1 -- 172.21.15.27:0/2270897452 shutdown_connections 2023-12-19T18:23:29.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:29.385+0000 7fce3ffff640 1 -- 172.21.15.27:0/2270897452 wait complete. 2023-12-19T18:23:29.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-12-19T18:23:29.654 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":7794,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":23.167320496999999,"mdsmap_epoch":61,"osdmap_epoch":154,"osdmap_epoch_barrier":152,"uptime":28.974388046000001} 2023-12-19T18:23:29.655 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 7794, "mdsmap_epoch": 61, "osdmap_epoch": 154, "osdmap_epoch_barrier": 152, "rank_uptime": 23.167320497, "state": "up:active", "uptime": 28.974388046, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:23:29.655 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-12-19T18:23:29.655 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:29.655 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-12-19T18:23:29.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:29.724 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-12-19T18:23:29.793 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:23:30.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.027+0000 7f3e97fff640 1 Processor -- start 2023-12-19T18:23:30.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.027+0000 7f3e97fff640 1 -- start start 2023-12-19T18:23:30.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.029+0000 7f3e97fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e98070aa0 0x7f3e98070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:30.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.029+0000 7f3e97fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3e9815a3a0 0x7f3e9815a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:30.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.029+0000 7f3e97fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e98071440 con 0x7f3e98154170 2023-12-19T18:23:30.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.029+0000 7f3e97fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9815b110 con 0x7f3e9815a3a0 2023-12-19T18:23:30.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.029+0000 7f3e97fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e9806fce0 con 0x7f3e98070aa0 2023-12-19T18:23:30.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.030+0000 7f3e977fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3e9815a3a0 0x7f3e9815a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:30.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.030+0000 7f3e967fc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e98070aa0 0x7f3e98070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:30.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.030+0000 7f3e977fe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3e9815a3a0 0x7f3e9815a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:41042/0 (socket says 172.21.15.27:41042) 2023-12-19T18:23:30.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.030+0000 7f3e967fc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e98070aa0 0x7f3e98070f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:46728/0 (socket says 172.21.15.27:46728) 2023-12-19T18:23:30.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.030+0000 7f3e977fe640 1 -- 172.21.15.27:0/955734733 learned_addr learned my addr 172.21.15.27:0/955734733 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:30.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.030+0000 7f3e96ffd640 1 --1- 172.21.15.27:0/955734733 >> v1:172.21.15.192:6789/0 conn(0x7f3e98154170 0x7f3e98070170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:59796/0 (socket says 172.21.15.27:59796) 2023-12-19T18:23:30.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.030+0000 7f3e95ffb640 1 -- 172.21.15.27:0/955734733 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 189554994 0 0) 0x7f3e98071440 con 0x7f3e98154170 2023-12-19T18:23:30.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.030+0000 7f3e95ffb640 1 -- 172.21.15.27:0/955734733 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e80003520 con 0x7f3e98154170 2023-12-19T18:23:30.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.030+0000 7f3e967fc640 1 -- 172.21.15.27:0/955734733 >> v2:172.21.15.192:3300/0 conn(0x7f3e9815a3a0 msgr2=0x7f3e9815a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:30.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.031+0000 7f3e967fc640 1 --2- 172.21.15.27:0/955734733 >> v2:172.21.15.192:3300/0 conn(0x7f3e9815a3a0 0x7f3e9815a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:30.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.031+0000 7f3e967fc640 1 -- 172.21.15.27:0/955734733 >> v1:172.21.15.192:6789/0 conn(0x7f3e98154170 legacy=0x7f3e98070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:30.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.031+0000 7f3e967fc640 1 -- 172.21.15.27:0/955734733 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e98070880 con 0x7f3e98070aa0 2023-12-19T18:23:30.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.031+0000 7f3e977fe640 1 --2- 172.21.15.27:0/955734733 >> v2:172.21.15.192:3300/0 conn(0x7f3e9815a3a0 0x7f3e9815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:23:30.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.032+0000 7f3e967fc640 1 --2- 172.21.15.27:0/955734733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e98070aa0 0x7f3e98070f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0097a0 tx=0x7f3e8c02b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=811fe8cca75c409b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:30.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.033+0000 7f3e95ffb640 1 -- 172.21.15.27:0/955734733 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c02e070 con 0x7f3e98070aa0 2023-12-19T18:23:30.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.033+0000 7f3e95ffb640 1 -- 172.21.15.27:0/955734733 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8c0315a0 con 0x7f3e98070aa0 2023-12-19T18:23:30.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.033+0000 7f3e95ffb640 1 -- 172.21.15.27:0/955734733 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8c031930 con 0x7f3e98070aa0 2023-12-19T18:23:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.033+0000 7f3e97fff640 1 -- 172.21.15.27:0/955734733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e98070aa0 msgr2=0x7f3e98070f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.033+0000 7f3e97fff640 1 --2- 172.21.15.27:0/955734733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e98070aa0 0x7f3e98070f00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3e8c0097a0 tx=0x7f3e8c02b840 comp rx=0 tx=0).stop 2023-12-19T18:23:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.033+0000 7f3e97fff640 1 -- 172.21.15.27:0/955734733 shutdown_connections 2023-12-19T18:23:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.033+0000 7f3e97fff640 1 --2- 172.21.15.27:0/955734733 >> v2:172.21.15.192:3300/0 conn(0x7f3e9815a3a0 0x7f3e9815a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.033+0000 7f3e97fff640 1 --2- 172.21.15.27:0/955734733 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e98070aa0 0x7f3e98070f00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.034+0000 7f3e97fff640 1 -- 172.21.15.27:0/955734733 >> 172.21.15.27:0/955734733 conn(0x7f3e9806a7d0 msgr2=0x7f3e980ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.034+0000 7f3e97fff640 1 -- 172.21.15.27:0/955734733 shutdown_connections 2023-12-19T18:23:30.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.034+0000 7f3e97fff640 1 -- 172.21.15.27:0/955734733 wait complete. 2023-12-19T18:23:30.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.035+0000 7f3e97fff640 1 Processor -- start 2023-12-19T18:23:30.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.036+0000 7f3e97fff640 1 -- start start 2023-12-19T18:23:30.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e97fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e98070aa0 0x7f3e981508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:30.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e97fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e98154170 0x7f3e98143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:30.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e97fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9815a3a0 0x7f3e980a84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e97fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e98151120 con 0x7f3e9815a3a0 2023-12-19T18:23:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e97fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e98151400 con 0x7f3e98070aa0 2023-12-19T18:23:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e97fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e981516e0 con 0x7f3e98154170 2023-12-19T18:23:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e977fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9815a3a0 0x7f3e980a84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e977fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9815a3a0 0x7f3e980a84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60424/0 (socket says 172.21.15.27:60424) 2023-12-19T18:23:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e977fe640 1 -- 172.21.15.27:0/227297688 learned_addr learned my addr 172.21.15.27:0/227297688 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e96ffd640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e98070aa0 0x7f3e981508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e967fc640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e98154170 0x7f3e98143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e977fe640 1 -- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e98154170 msgr2=0x7f3e98143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e977fe640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e98154170 0x7f3e98143a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e977fe640 1 -- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e98070aa0 msgr2=0x7f3e981508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.037+0000 7f3e977fe640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e98070aa0 0x7f3e981508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.038+0000 7f3e977fe640 1 -- 172.21.15.27:0/227297688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e90009b30 con 0x7f3e9815a3a0 2023-12-19T18:23:30.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.038+0000 7f3e977fe640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9815a3a0 0x7f3e980a84b0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000efc0 tx=0x7f3e9000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:30.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.038+0000 7f3e7f7fe640 1 -- 172.21.15.27:0/227297688 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e90018070 con 0x7f3e9815a3a0 2023-12-19T18:23:30.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.038+0000 7f3e7f7fe640 1 -- 172.21.15.27:0/227297688 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9000f040 con 0x7f3e9815a3a0 2023-12-19T18:23:30.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.038+0000 7f3e97fff640 1 -- 172.21.15.27:0/227297688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e8c009400 con 0x7f3e9815a3a0 2023-12-19T18:23:30.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.038+0000 7f3e7f7fe640 1 -- 172.21.15.27:0/227297688 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000ce00 con 0x7f3e9815a3a0 2023-12-19T18:23:30.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.038+0000 7f3e97fff640 1 -- 172.21.15.27:0/227297688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e98150e10 con 0x7f3e9815a3a0 2023-12-19T18:23:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.040+0000 7f3e97fff640 1 -- 172.21.15.27:0/227297688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e980b2230 con 0x7f3e9815a3a0 2023-12-19T18:23:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.040+0000 7f3e7f7fe640 1 -- 172.21.15.27:0/227297688 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e90014480 con 0x7f3e9815a3a0 2023-12-19T18:23:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.041+0000 7f3e7f7fe640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e8406a430 0x7f3e8406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.042+0000 7f3e7f7fe640 1 -- 172.21.15.27:0/227297688 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e90088ca0 con 0x7f3e9815a3a0 2023-12-19T18:23:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.042+0000 7f3e7f7fe640 1 -- 172.21.15.27:0/227297688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7f3e84071f60 con 0x7f3e9815a3a0 2023-12-19T18:23:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.042+0000 7f3e96ffd640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e8406a430 0x7f3e8406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.043+0000 7f3e96ffd640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e8406a430 0x7f3e8406c8f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3e880011e0 tx=0x7f3e88007b70 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.046+0000 7f3e7f7fe640 1 -- 172.21.15.27:0/227297688 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3e980b2230 con 0x7f3e9815a3a0 2023-12-19T18:23:30.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.239+0000 7f3e97fff640 1 -- 172.21.15.27:0/227297688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f3e980b2230 con 0x7f3e9815a3a0 2023-12-19T18:23:30.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.243+0000 7f3e7f7fe640 1 -- 172.21.15.27:0/227297688 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v154) v1 ==== 81+0+33577 (secure 0 0 0) 0x7f3e980b2230 con 0x7f3e9815a3a0 2023-12-19T18:23:30.243 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:30.243 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 154, 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:23:26.707491+0000", 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:30.244 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 17, 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:23:30.245 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:23:30.246 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:30.247 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:30.248 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:30.249 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:30.250 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 16, 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:04.431250+0000", 2023-12-19T18:23:30.251 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:30.252 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:30.252 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:30.252 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:30.252 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:30.252 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:30.253 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:30.253 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:30.253 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:30.253 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:30.253 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:30.253 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:30.253 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:23:30.253 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:23:30.253 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:23:30.253 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:23:30.253 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:23:30.253 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:30.253 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "151", 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:30.254 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:30.255 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:30.256 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:30.256 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:30.256 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:30.256 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:30.256 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:30.256 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.256 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:30.256 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:30.256 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:30.256 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:30.256 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:30.256 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:30.256 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.257 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.25, 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.25, 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 17, 2023-12-19T18:23:30.258 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:05.506926+0000", 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1027, 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full,full_quota", 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:30.259 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:23:30.260 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:23:30.260 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:23:30.260 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:23:30.260 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:23:30.260 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:30.260 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:30.260 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:30.260 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:30.260 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:30.260 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:30.260 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:30.260 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "154", 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:30.261 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:30.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:23:30.263 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:30.264 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:30.265 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.266 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.267 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:30.269 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.269 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:23:30.269 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:30.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.269 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.269 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.269 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:23:30.269 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:30.269 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:30.270 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.271 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:23:30.272 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:30.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.274 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.275 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.276 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.277 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.283 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.285 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:30.286 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.287 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.288 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:23:30.288 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:30.288 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.288 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.288 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:30.288 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.288 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.288 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:23:30.288 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:30.288 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.288 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.289 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.290 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.291 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.292 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.293 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.294 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.295 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:23:30.296 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:30.297 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:30.298 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.299 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.301 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:30.302 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.303 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.304 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:23:30.304 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:30.304 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.304 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.304 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.304 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:30.304 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.304 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.304 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.304 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:23:30.304 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:30.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.305 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.305 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:23:30.305 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:30.305 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.305 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.305 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.305 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:30.305 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.305 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.306 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:30.307 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:23:30.307 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:30.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.307 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.307 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:30.307 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:23:30.307 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:30.307 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.307 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.307 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.307 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:23:30.307 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:23:30.307 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:30.308 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:30.309 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:30.309 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:23:30.309 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:30.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.309 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.309 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:30.309 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:30.309 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:30.309 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:30.309 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:30.309 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:30.309 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:23:30.309 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:30.310 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:23:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:23:30.315 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:23:30.316 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:23:30.316 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:23:30.316 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:30.316 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:23:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.245+0000 7f3e97fff640 1 -- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e8406a430 msgr2=0x7f3e8406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.245+0000 7f3e97fff640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e8406a430 0x7f3e8406c8f0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3e880011e0 tx=0x7f3e88007b70 comp rx=0 tx=0).stop 2023-12-19T18:23:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.245+0000 7f3e97fff640 1 -- 172.21.15.27:0/227297688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9815a3a0 msgr2=0x7f3e980a84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.245+0000 7f3e97fff640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9815a3a0 0x7f3e980a84b0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f3e9000efc0 tx=0x7f3e9000c490 comp rx=0 tx=0).stop 2023-12-19T18:23:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.246+0000 7f3e97fff640 1 -- 172.21.15.27:0/227297688 shutdown_connections 2023-12-19T18:23:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.246+0000 7f3e97fff640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e8406a430 0x7f3e8406c8f0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.246+0000 7f3e97fff640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e9815a3a0 0x7f3e980a84b0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.246+0000 7f3e97fff640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e98154170 0x7f3e98143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.246+0000 7f3e97fff640 1 --2- 172.21.15.27:0/227297688 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e98070aa0 0x7f3e981508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.246+0000 7f3e97fff640 1 -- 172.21.15.27:0/227297688 >> 172.21.15.27:0/227297688 conn(0x7f3e9806a7d0 msgr2=0x7f3e9815dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.246+0000 7f3e97fff640 1 -- 172.21.15.27:0/227297688 shutdown_connections 2023-12-19T18:23:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:30.246+0000 7f3e97fff640 1 -- 172.21.15.27:0/227297688 wait complete. 2023-12-19T18:23:30.319 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-12-19T18:23:35.319 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:23:35.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.568+0000 7f1f7a88c640 1 Processor -- start 2023-12-19T18:23:35.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.568+0000 7f1f7a88c640 1 -- start start 2023-12-19T18:23:35.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.569+0000 7f1f7a88c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f7415ac50 0x7f1f74070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:35.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.570+0000 7f1f7a88c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1f740af360 0x7f1f7406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:35.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.570+0000 7f1f7a88c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f740adf10 con 0x7f1f740aef20 2023-12-19T18:23:35.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.570+0000 7f1f7a88c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f740adac0 con 0x7f1f740af360 2023-12-19T18:23:35.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.570+0000 7f1f7a88c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740ae160 con 0x7f1f7415ac50 2023-12-19T18:23:35.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.570+0000 7f1f7988a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f7415ac50 0x7f1f74070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:35.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.570+0000 7f1f7a08b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1f740af360 0x7f1f7406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:35.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.571+0000 7f1f7988a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f7415ac50 0x7f1f74070fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:46848/0 (socket says 172.21.15.27:46848) 2023-12-19T18:23:35.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.571+0000 7f1f7988a640 1 -- 172.21.15.27:0/3994013966 learned_addr learned my addr 172.21.15.27:0/3994013966 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:35.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.571+0000 7f1f79089640 1 --1- 172.21.15.27:0/3994013966 >> v1:172.21.15.192:6789/0 conn(0x7f1f740aef20 0x7f1f7406b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:56946/0 (socket says 172.21.15.27:56946) 2023-12-19T18:23:35.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.571+0000 7f1f78888640 1 -- 172.21.15.27:0/3994013966 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2636116869 0 0) 0x7f1f740adf10 con 0x7f1f740aef20 2023-12-19T18:23:35.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.571+0000 7f1f78888640 1 -- 172.21.15.27:0/3994013966 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f60003520 con 0x7f1f740aef20 2023-12-19T18:23:35.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.571+0000 7f1f7988a640 1 -- 172.21.15.27:0/3994013966 >> v2:172.21.15.192:3300/0 conn(0x7f1f740af360 msgr2=0x7f1f7406ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:35.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.571+0000 7f1f7988a640 1 --2- 172.21.15.27:0/3994013966 >> v2:172.21.15.192:3300/0 conn(0x7f1f740af360 0x7f1f7406ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:35.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.572+0000 7f1f7988a640 1 -- 172.21.15.27:0/3994013966 >> v1:172.21.15.192:6789/0 conn(0x7f1f740aef20 legacy=0x7f1f7406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:35.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.572+0000 7f1f7988a640 1 -- 172.21.15.27:0/3994013966 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f740ae440 con 0x7f1f7415ac50 2023-12-19T18:23:35.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.572+0000 7f1f7a08b640 1 --2- 172.21.15.27:0/3994013966 >> v2:172.21.15.192:3300/0 conn(0x7f1f740af360 0x7f1f7406ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:23:35.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.573+0000 7f1f7988a640 1 --2- 172.21.15.27:0/3994013966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f7415ac50 0x7f1f74070fb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009870 tx=0x7f1f7002b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=321fe8c151550326 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:35.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.573+0000 7f1f78888640 1 -- 172.21.15.27:0/3994013966 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f700317c0 con 0x7f1f7415ac50 2023-12-19T18:23:35.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.573+0000 7f1f78888640 1 -- 172.21.15.27:0/3994013966 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f70031920 con 0x7f1f7415ac50 2023-12-19T18:23:35.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.573+0000 7f1f78888640 1 -- 172.21.15.27:0/3994013966 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70031c90 con 0x7f1f7415ac50 2023-12-19T18:23:35.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.574+0000 7f1f7a88c640 1 -- 172.21.15.27:0/3994013966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f7415ac50 msgr2=0x7f1f74070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:35.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.574+0000 7f1f7a88c640 1 --2- 172.21.15.27:0/3994013966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f7415ac50 0x7f1f74070fb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009870 tx=0x7f1f7002b860 comp rx=0 tx=0).stop 2023-12-19T18:23:35.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.574+0000 7f1f7a88c640 1 -- 172.21.15.27:0/3994013966 shutdown_connections 2023-12-19T18:23:35.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.574+0000 7f1f7a88c640 1 --2- 172.21.15.27:0/3994013966 >> v2:172.21.15.192:3300/0 conn(0x7f1f740af360 0x7f1f7406ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:35.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.574+0000 7f1f7a88c640 1 --2- 172.21.15.27:0/3994013966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f7415ac50 0x7f1f74070fb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:35.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.574+0000 7f1f7a88c640 1 -- 172.21.15.27:0/3994013966 >> 172.21.15.27:0/3994013966 conn(0x7f1f7406a7d0 msgr2=0x7f1f740ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:35.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.574+0000 7f1f7a88c640 1 -- 172.21.15.27:0/3994013966 shutdown_connections 2023-12-19T18:23:35.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.574+0000 7f1f7a88c640 1 -- 172.21.15.27:0/3994013966 wait complete. 2023-12-19T18:23:35.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.577+0000 7f1f7a88c640 1 Processor -- start 2023-12-19T18:23:35.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.577+0000 7f1f7a88c640 1 -- start start 2023-12-19T18:23:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.578+0000 7f1f7a88c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f740aef20 0x7f1f740ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.578+0000 7f1f7a88c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f740af360 0x7f1f740c39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.578+0000 7f1f7a88c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f7415ac50 0x7f1f741f19b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.578+0000 7f1f7a88c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f740c6450 con 0x7f1f740aef20 2023-12-19T18:23:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.578+0000 7f1f7a88c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f741f6bd0 con 0x7f1f7415ac50 2023-12-19T18:23:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.578+0000 7f1f7a88c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f741f6050 con 0x7f1f740af360 2023-12-19T18:23:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.579+0000 7f1f7988a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f740aef20 0x7f1f740ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.579+0000 7f1f7988a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f740aef20 0x7f1f740ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33582/0 (socket says 172.21.15.27:33582) 2023-12-19T18:23:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.579+0000 7f1f7988a640 1 -- 172.21.15.27:0/637400323 learned_addr learned my addr 172.21.15.27:0/637400323 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:35.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.579+0000 7f1f79089640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f740af360 0x7f1f740c39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:35.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.579+0000 7f1f7a08b640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f7415ac50 0x7f1f741f19b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:35.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.579+0000 7f1f7988a640 1 -- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f740af360 msgr2=0x7f1f740c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:35.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.579+0000 7f1f7988a640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f740af360 0x7f1f740c39b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:35.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.579+0000 7f1f7988a640 1 -- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f7415ac50 msgr2=0x7f1f741f19b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:35.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.579+0000 7f1f7988a640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f7415ac50 0x7f1f741f19b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:35.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.579+0000 7f1f7988a640 1 -- 172.21.15.27:0/637400323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c009a70 con 0x7f1f740aef20 2023-12-19T18:23:35.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.580+0000 7f1f7988a640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f740aef20 0x7f1f740ca610 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f1f70000c00 tx=0x7f1f700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:35.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.580+0000 7f1f6a7fc640 1 -- 172.21.15.27:0/637400323 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70032860 con 0x7f1f740aef20 2023-12-19T18:23:35.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.580+0000 7f1f6a7fc640 1 -- 172.21.15.27:0/637400323 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f700329c0 con 0x7f1f740aef20 2023-12-19T18:23:35.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.580+0000 7f1f6a7fc640 1 -- 172.21.15.27:0/637400323 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70032d70 con 0x7f1f740aef20 2023-12-19T18:23:35.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.580+0000 7f1f7a88c640 1 -- 172.21.15.27:0/637400323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f700094d0 con 0x7f1f740aef20 2023-12-19T18:23:35.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.580+0000 7f1f7a88c640 1 -- 172.21.15.27:0/637400323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f741f6610 con 0x7f1f740aef20 2023-12-19T18:23:35.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.582+0000 7f1f7a88c640 1 -- 172.21.15.27:0/637400323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f740c65c0 con 0x7f1f740aef20 2023-12-19T18:23:35.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.583+0000 7f1f6a7fc640 1 -- 172.21.15.27:0/637400323 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1f7004e450 con 0x7f1f740aef20 2023-12-19T18:23:35.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.584+0000 7f1f6a7fc640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f5806a430 0x7f1f5806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:35.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.584+0000 7f1f6a7fc640 1 -- 172.21.15.27:0/637400323 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f700aaa00 con 0x7f1f740aef20 2023-12-19T18:23:35.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.584+0000 7f1f6a7fc640 1 -- 172.21.15.27:0/637400323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=155}) v3 -- 0x7f1f58071f60 con 0x7f1f740aef20 2023-12-19T18:23:35.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.584+0000 7f1f79089640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f5806a430 0x7f1f5806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:35.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.586+0000 7f1f79089640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f5806a430 0x7f1f5806c8f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1f740b2200 tx=0x7f1f64007b50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:35.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.589+0000 7f1f6a7fc640 1 -- 172.21.15.27:0/637400323 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1f740c65c0 con 0x7f1f740aef20 2023-12-19T18:23:35.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.794+0000 7f1f7a88c640 1 -- 172.21.15.27:0/637400323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f1f740c6590 con 0x7f1f740aef20 2023-12-19T18:23:35.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.798+0000 7f1f6a7fc640 1 -- 172.21.15.27:0/637400323 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v154) v1 ==== 81+0+33577 (secure 0 0 0) 0x7f1f700760a0 con 0x7f1f740aef20 2023-12-19T18:23:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:35.799 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:23:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 154, 2023-12-19T18:23:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:23:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:23:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:23:26.707491+0000", 2023-12-19T18:23:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:23:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:23:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:23:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:23:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:23:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:23:35.799 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 17, 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:23:35.800 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:23:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:23:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:23:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:35.801 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:35.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:35.803 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.804 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:23:35.805 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.806 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 16, 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:04.431250+0000", 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:35.807 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:35.808 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "151", 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:35.809 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:35.810 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:23:35.811 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.25, 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.25, 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:35.812 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 17, 2023-12-19T18:23:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:23:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:05.506926+0000", 2023-12-19T18:23:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1027, 2023-12-19T18:23:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full,full_quota", 2023-12-19T18:23:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:35.813 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:35.814 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "154", 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:35.815 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:35.816 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:35.817 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:35.818 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:35.819 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.820 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.821 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:23:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:23:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.822 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:23:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:23:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.823 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:35.825 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.826 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:23:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:23:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.827 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.828 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:23:35.829 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:35.830 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:35.831 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.832 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.834 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:35.835 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.836 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.837 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:35.838 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.839 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:35.840 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.841 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.842 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:23:35.843 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:23:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.844 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.845 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.847 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.848 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.849 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:35.850 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:35.851 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.853 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:35.854 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:23:35.855 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:35.856 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:23:35.857 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.858 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.859 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:23:35.860 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:35.861 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.862 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:35.863 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:35.864 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:23:35.865 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:23:35.866 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:23:35.867 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:23:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:23:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:23:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:23:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:23:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:23:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:23:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:23:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:23:35.868 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:35.868 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:23:35.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.800+0000 7f1f7a88c640 1 -- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f5806a430 msgr2=0x7f1f5806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:35.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.800+0000 7f1f7a88c640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f5806a430 0x7f1f5806c8f0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f1f740b2200 tx=0x7f1f64007b50 comp rx=0 tx=0).stop 2023-12-19T18:23:35.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.800+0000 7f1f7a88c640 1 -- 172.21.15.27:0/637400323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f740aef20 msgr2=0x7f1f740ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:35.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.800+0000 7f1f7a88c640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f740aef20 0x7f1f740ca610 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f1f70000c00 tx=0x7f1f700046c0 comp rx=0 tx=0).stop 2023-12-19T18:23:35.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.800+0000 7f1f7a88c640 1 -- 172.21.15.27:0/637400323 shutdown_connections 2023-12-19T18:23:35.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.800+0000 7f1f7a88c640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f5806a430 0x7f1f5806c8f0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:35.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.800+0000 7f1f7a88c640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f7415ac50 0x7f1f741f19b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:35.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.800+0000 7f1f7a88c640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f740af360 0x7f1f740c39b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:35.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.800+0000 7f1f7a88c640 1 --2- 172.21.15.27:0/637400323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f740aef20 0x7f1f740ca610 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:35.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.800+0000 7f1f7a88c640 1 -- 172.21.15.27:0/637400323 >> 172.21.15.27:0/637400323 conn(0x7f1f7406a7d0 msgr2=0x7f1f74159a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:35.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.801+0000 7f1f7a88c640 1 -- 172.21.15.27:0/637400323 shutdown_connections 2023-12-19T18:23:35.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:35.801+0000 7f1f7a88c640 1 -- 172.21.15.27:0/637400323 wait complete. 2023-12-19T18:23:35.871 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-12-19T18:23:40.872 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:23:41.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.118+0000 7fc6f56d5640 1 Processor -- start 2023-12-19T18:23:41.118 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.118+0000 7fc6f56d5640 1 -- start start 2023-12-19T18:23:41.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.120+0000 7fc6f56d5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6f01540e0 0x7fc6f0160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.120+0000 7fc6f56d5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc6f006b310 0x7fc6f006b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.120+0000 7fc6f56d5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6f015d600 con 0x7fc6f006c6e0 2023-12-19T18:23:41.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.120+0000 7fc6f56d5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6f006bcc0 con 0x7fc6f006b310 2023-12-19T18:23:41.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.120+0000 7fc6f56d5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f0161130 con 0x7fc6f01540e0 2023-12-19T18:23:41.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.121+0000 7fc6effff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6f01540e0 0x7fc6f0160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.121+0000 7fc6effff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6f01540e0 0x7fc6f0160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:46866/0 (socket says 172.21.15.27:46866) 2023-12-19T18:23:41.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.121+0000 7fc6effff640 1 -- 172.21.15.27:0/4167993793 learned_addr learned my addr 172.21.15.27:0/4167993793 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:41.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.121+0000 7fc6f4ed4640 1 --2- 172.21.15.27:0/4167993793 >> v2:172.21.15.192:3300/0 conn(0x7fc6f006b310 0x7fc6f006b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.122+0000 7fc6effff640 1 -- 172.21.15.27:0/4167993793 >> v2:172.21.15.192:3300/0 conn(0x7fc6f006b310 msgr2=0x7fc6f006b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.122+0000 7fc6effff640 1 --2- 172.21.15.27:0/4167993793 >> v2:172.21.15.192:3300/0 conn(0x7fc6f006b310 0x7fc6f006b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.122+0000 7fc6effff640 1 -- 172.21.15.27:0/4167993793 >> v1:172.21.15.192:6789/0 conn(0x7fc6f006c6e0 legacy=0x7fc6f015dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.122+0000 7fc6effff640 1 -- 172.21.15.27:0/4167993793 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6f01613d0 con 0x7fc6f01540e0 2023-12-19T18:23:41.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.122+0000 7fc6f4ed4640 1 --2- 172.21.15.27:0/4167993793 >> v2:172.21.15.192:3300/0 conn(0x7fc6f006b310 0x7fc6f006b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:23:41.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.123+0000 7fc6effff640 1 --2- 172.21.15.27:0/4167993793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6f01540e0 0x7fc6f0160bf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009870 tx=0x7fc6e002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1ced0289a509ef7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.124+0000 7fc6eeffd640 1 -- 172.21.15.27:0/4167993793 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0030840 con 0x7fc6f01540e0 2023-12-19T18:23:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.124+0000 7fc6eeffd640 1 -- 172.21.15.27:0/4167993793 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e00309a0 con 0x7fc6f01540e0 2023-12-19T18:23:41.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.124+0000 7fc6eeffd640 1 -- 172.21.15.27:0/4167993793 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e0030d30 con 0x7fc6f01540e0 2023-12-19T18:23:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.125+0000 7fc6f56d5640 1 -- 172.21.15.27:0/4167993793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6f01540e0 msgr2=0x7fc6f0160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.125+0000 7fc6f56d5640 1 --2- 172.21.15.27:0/4167993793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6f01540e0 0x7fc6f0160bf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0009870 tx=0x7fc6e002b980 comp rx=0 tx=0).stop 2023-12-19T18:23:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.125+0000 7fc6f56d5640 1 -- 172.21.15.27:0/4167993793 shutdown_connections 2023-12-19T18:23:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.125+0000 7fc6f56d5640 1 --2- 172.21.15.27:0/4167993793 >> v2:172.21.15.192:3300/0 conn(0x7fc6f006b310 0x7fc6f006b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.125+0000 7fc6f56d5640 1 --2- 172.21.15.27:0/4167993793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6f01540e0 0x7fc6f0160bf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.125+0000 7fc6f56d5640 1 -- 172.21.15.27:0/4167993793 >> 172.21.15.27:0/4167993793 conn(0x7fc6f006a7d0 msgr2=0x7fc6f00ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:41.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.125+0000 7fc6f56d5640 1 -- 172.21.15.27:0/4167993793 shutdown_connections 2023-12-19T18:23:41.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.125+0000 7fc6f56d5640 1 -- 172.21.15.27:0/4167993793 wait complete. 2023-12-19T18:23:41.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.127+0000 7fc6f56d5640 1 Processor -- start 2023-12-19T18:23:41.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.127+0000 7fc6f56d5640 1 -- start start 2023-12-19T18:23:41.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.128+0000 7fc6f56d5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6f006b310 0x7fc6f01f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.128+0000 7fc6f56d5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6f006c6e0 0x7fc6f00b42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.128+0000 7fc6f56d5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f01540e0 0x7fc6f00b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.128+0000 7fc6f56d5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00b3e10 con 0x7fc6f006c6e0 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.128+0000 7fc6f56d5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00b1b70 con 0x7fc6f01540e0 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.128+0000 7fc6f56d5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6f00b34e0 con 0x7fc6f006b310 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.129+0000 7fc6ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6f006c6e0 0x7fc6f00b42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.129+0000 7fc6ef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6f006c6e0 0x7fc6f00b42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33592/0 (socket says 172.21.15.27:33592) 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.129+0000 7fc6ef7fe640 1 -- 172.21.15.27:0/1764847278 learned_addr learned my addr 172.21.15.27:0/1764847278 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.129+0000 7fc6effff640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6f006b310 0x7fc6f01f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.129+0000 7fc6f4ed4640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f01540e0 0x7fc6f00b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.129+0000 7fc6ef7fe640 1 -- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6f006b310 msgr2=0x7fc6f01f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.129+0000 7fc6ef7fe640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6f006b310 0x7fc6f01f34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.129+0000 7fc6ef7fe640 1 -- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f01540e0 msgr2=0x7fc6f00b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.129+0000 7fc6ef7fe640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f01540e0 0x7fc6f00b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.129+0000 7fc6ef7fe640 1 -- 172.21.15.27:0/1764847278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6e8009a70 con 0x7fc6f006c6e0 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.129+0000 7fc6f4ed4640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f01540e0 0x7fc6f00b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:41.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.130+0000 7fc6ef7fe640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6f006c6e0 0x7fc6f00b42a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400c400 tx=0x7fc6e400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.130+0000 7fc6ecff9640 1 -- 172.21.15.27:0/1764847278 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e40052d0 con 0x7fc6f006c6e0 2023-12-19T18:23:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.130+0000 7fc6ecff9640 1 -- 172.21.15.27:0/1764847278 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6e4005430 con 0x7fc6f006c6e0 2023-12-19T18:23:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.130+0000 7fc6f56d5640 1 -- 172.21.15.27:0/1764847278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6e00094d0 con 0x7fc6f006c6e0 2023-12-19T18:23:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.130+0000 7fc6ecff9640 1 -- 172.21.15.27:0/1764847278 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6e4003830 con 0x7fc6f006c6e0 2023-12-19T18:23:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.130+0000 7fc6f56d5640 1 -- 172.21.15.27:0/1764847278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6f00b3aa0 con 0x7fc6f006c6e0 2023-12-19T18:23:41.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.131+0000 7fc6f56d5640 1 -- 172.21.15.27:0/1764847278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6f00b4b30 con 0x7fc6f006c6e0 2023-12-19T18:23:41.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.132+0000 7fc6ecff9640 1 -- 172.21.15.27:0/1764847278 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc6e4003990 con 0x7fc6f006c6e0 2023-12-19T18:23:41.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.133+0000 7fc6ecff9640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc6dc06a430 0x7fc6dc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.133+0000 7fc6ecff9640 1 -- 172.21.15.27:0/1764847278 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc6e4087ef0 con 0x7fc6f006c6e0 2023-12-19T18:23:41.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.134+0000 7fc6effff640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc6dc06a430 0x7fc6dc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.135+0000 7fc6effff640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc6dc06a430 0x7fc6dc06c8f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0036000 tx=0x7fc6e0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:41.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.138+0000 7fc6ecff9640 1 -- 172.21.15.27:0/1764847278 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc6e40534e0 con 0x7fc6f006c6e0 2023-12-19T18:23:41.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.343+0000 7fc6f56d5640 1 -- 172.21.15.27:0/1764847278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fc6f004e840 con 0x7fc6f006c6e0 2023-12-19T18:23:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.346+0000 7fc6ecff9640 1 -- 172.21.15.27:0/1764847278 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v155) v1 ==== 81+0+33558 (secure 0 0 0) 0x7fc6e4056b00 con 0x7fc6f006c6e0 2023-12-19T18:23:41.347 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:41.347 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:23:41.347 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 155, 2023-12-19T18:23:41.347 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:23:41.347 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:23:41.347 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:23:36.712355+0000", 2023-12-19T18:23:41.347 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:23:41.347 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:23:41.347 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 17, 2023-12-19T18:23:41.348 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:23:41.349 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:23:41.349 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:23:41.349 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:23:41.349 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:23:41.349 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:23:41.349 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.349 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:23:41.349 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:23:41.349 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:23:41.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:41.349 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:41.349 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:41.349 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:41.350 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:41.350 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:41.350 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:41.350 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:41.350 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:41.350 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:41.350 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:41.350 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:41.350 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:23:41.350 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:23:41.350 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:23:41.351 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:23:41.351 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:23:41.351 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:41.351 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:41.351 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:41.351 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:41.351 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:41.351 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:41.351 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:41.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.351 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:23:41.351 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:41.351 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:41.352 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:41.353 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:23:41.354 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 16, 2023-12-19T18:23:41.355 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:04.431250+0000", 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:41.356 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:23:41.357 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:23:41.357 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:23:41.357 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:23:41.357 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:23:41.357 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:41.357 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:41.357 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:41.357 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:41.357 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:41.357 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:41.357 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:41.357 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.357 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "151", 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:41.358 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:41.359 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.360 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.25, 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.25, 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 17, 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:23:41.361 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:05.506926+0000", 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:23:41.362 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:23:41.363 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:23:41.363 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:23:41.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:41.363 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:41.363 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:41.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:41.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:41.363 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:41.363 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:41.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "155", 2023-12-19T18:23:41.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:41.363 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:41.364 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:41.365 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:41.365 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:41.365 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:41.365 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:41.365 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:41.365 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:41.365 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:41.365 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:41.365 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:41.365 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:41.365 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.365 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:41.365 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:41.366 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:41.367 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:23:41.367 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.367 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.367 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:41.367 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:23:41.367 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:23:41.367 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:41.367 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:23:41.367 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:23:41.367 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:41.367 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:41.367 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:41.367 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.368 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.368 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:41.368 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:23:41.368 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.368 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:41.368 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.368 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.368 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.368 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.368 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.368 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.368 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.368 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:23:41.369 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:23:41.370 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:23:41.371 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.372 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.373 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.374 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.375 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.376 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.377 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.379 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.380 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.380 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:23:41.380 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.380 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.380 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.380 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:23:41.380 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.380 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.381 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.381 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.381 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:23:41.381 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.381 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.381 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.381 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:23:41.381 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.381 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.382 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.382 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.382 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.382 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:23:41.382 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.382 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.382 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.382 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.382 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:23:41.382 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.383 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.383 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.383 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.383 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.383 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:23:41.383 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.383 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.383 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.384 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.385 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:41.385 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.385 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.385 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.385 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.385 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.385 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.385 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.385 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.385 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.385 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.385 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.385 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.386 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.387 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.388 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:23:41.388 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.388 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.388 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.388 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:23:41.388 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.388 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.388 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.388 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.389 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.389 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.390 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:23:41.390 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.390 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.390 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.390 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:23:41.390 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.390 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.390 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.391 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:23:41.392 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:23:41.393 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:23:41.394 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:23:41.395 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.396 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.397 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.398 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.400 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:23:41.400 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.400 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.401 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.402 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.403 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.404 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:23:41.405 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.407 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.408 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.409 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:23:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:23:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.410 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.411 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.411 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:23:41.411 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.413 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.413 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:23:41.413 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.413 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.413 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.413 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:23:41.413 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:23:41.413 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:23:41.413 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:23:41.413 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.413 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:23:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.349+0000 7fc6f56d5640 1 -- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc6dc06a430 msgr2=0x7fc6dc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.349+0000 7fc6f56d5640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc6dc06a430 0x7fc6dc06c8f0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fc6e0036000 tx=0x7fc6e0008030 comp rx=0 tx=0).stop 2023-12-19T18:23:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.349+0000 7fc6f56d5640 1 -- 172.21.15.27:0/1764847278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6f006c6e0 msgr2=0x7fc6f00b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.349+0000 7fc6f56d5640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6f006c6e0 0x7fc6f00b42a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fc6e400c400 tx=0x7fc6e400c8c0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.349+0000 7fc6f56d5640 1 -- 172.21.15.27:0/1764847278 shutdown_connections 2023-12-19T18:23:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.349+0000 7fc6f56d5640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc6dc06a430 0x7fc6dc06c8f0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.349+0000 7fc6f56d5640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc6f01540e0 0x7fc6f00b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.349+0000 7fc6f56d5640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6f006c6e0 0x7fc6f00b42a0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.349+0000 7fc6f56d5640 1 --2- 172.21.15.27:0/1764847278 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc6f006b310 0x7fc6f01f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.349+0000 7fc6f56d5640 1 -- 172.21.15.27:0/1764847278 >> 172.21.15.27:0/1764847278 conn(0x7fc6f006a7d0 msgr2=0x7fc6f00ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.349+0000 7fc6f56d5640 1 -- 172.21.15.27:0/1764847278 shutdown_connections 2023-12-19T18:23:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.349+0000 7fc6f56d5640 1 -- 172.21.15.27:0/1764847278 wait complete. 2023-12-19T18:23:41.425 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-12-19T18:23:41.425 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:23:41.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.665+0000 7f5cb30f7640 1 Processor -- start 2023-12-19T18:23:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.666+0000 7f5cb30f7640 1 -- start start 2023-12-19T18:23:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.667+0000 7f5cb30f7640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5cac15adf0 0x7f5cac06d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.667+0000 7f5cb30f7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cac06fc10 0x7f5cac0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.667+0000 7f5cb30f7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cac071820 con 0x7f5cac070c00 2023-12-19T18:23:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.667+0000 7f5cb30f7640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cac0703c0 con 0x7f5cac15adf0 2023-12-19T18:23:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.667+0000 7f5cb30f7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac06d8c0 con 0x7f5cac06fc10 2023-12-19T18:23:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.668+0000 7f5cabfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5cac070c00 0x7f5cac06f500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57236/0 (socket says 172.21.15.27:57236) 2023-12-19T18:23:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.668+0000 7f5cabfff640 1 -- 172.21.15.27:0/1400244058 learned_addr learned my addr 172.21.15.27:0/1400244058 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:41.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.668+0000 7f5cb0e93640 1 --2- 172.21.15.27:0/1400244058 >> v2:172.21.15.192:3300/0 conn(0x7f5cac15adf0 0x7f5cac06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.668+0000 7f5cb1694640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cac06fc10 0x7f5cac0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.668+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451192384 0 0) 0x7f5cac071820 con 0x7f5cac070c00 2023-12-19T18:23:41.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.668+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c9c003520 con 0x7f5cac070c00 2023-12-19T18:23:41.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2415466484 0 0) 0x7f5c9c003520 con 0x7f5cac070c00 2023-12-19T18:23:41.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cac06fc10 msgr2=0x7f5cac0712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:41.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cac06fc10 0x7f5cac0712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 >> v2:172.21.15.192:3300/0 conn(0x7f5cac15adf0 msgr2=0x7f5cac06d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 --2- 172.21.15.27:0/1400244058 >> v2:172.21.15.192:3300/0 conn(0x7f5cac15adf0 0x7f5cac06d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac06b520 con 0x7f5cac070c00 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f5c98001670 con 0x7f5cac070c00 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 >> v1:172.21.15.27:6789/0 conn(0x7f5cac070c00 legacy=0x7f5cac06f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c9c003970 0x7f5c9c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cac06fc10 0x7f5c9c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c9c004d00 0x7f5c9c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac06d8c0 con 0x7f5c9c003970 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c00b980 con 0x7f5c9c004d00 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c00bb40 con 0x7f5cac06fc10 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.669+0000 7f5cb0e93640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c9c003970 0x7f5c9c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.670+0000 7f5cb0e93640 1 -- 172.21.15.27:0/1400244058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cac06fc10 msgr2=0x7f5c9c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.670+0000 7f5cabfff640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cac06fc10 0x7f5c9c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.670+0000 7f5cb1694640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c9c004d00 0x7f5c9c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.670+0000 7f5cb0e93640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cac06fc10 0x7f5c9c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.670+0000 7f5cb0e93640 1 -- 172.21.15.27:0/1400244058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c9c004d00 msgr2=0x7f5c9c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.670+0000 7f5cb0e93640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c9c004d00 0x7f5c9c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.670+0000 7f5cb0e93640 1 -- 172.21.15.27:0/1400244058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c003520 con 0x7f5c9c003970 2023-12-19T18:23:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.670+0000 7f5cabfff640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5cac06fc10 0x7f5c9c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:41.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.670+0000 7f5cb1694640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c9c004d00 0x7f5c9c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.671+0000 7f5cb0e93640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c9c003970 0x7f5c9c003d90 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0009ac0 tx=0x7f5c9c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2ead176e1143cb5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.671+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca002c710 con 0x7f5c9c003970 2023-12-19T18:23:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.671+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca002c870 con 0x7f5c9c003970 2023-12-19T18:23:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.671+0000 7f5cab7fe640 1 -- 172.21.15.27:0/1400244058 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca002cc00 con 0x7f5c9c003970 2023-12-19T18:23:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.672+0000 7f5cb30f7640 1 -- 172.21.15.27:0/1400244058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c9c003970 msgr2=0x7f5c9c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.672+0000 7f5cb30f7640 1 --2- 172.21.15.27:0/1400244058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c9c003970 0x7f5c9c003d90 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0009ac0 tx=0x7f5c9c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.672+0000 7f5cb1694640 1 -- 172.21.15.27:0/1400244058 reap_dead start 2023-12-19T18:23:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.672+0000 7f5cb30f7640 1 -- 172.21.15.27:0/1400244058 shutdown_connections 2023-12-19T18:23:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.672+0000 7f5cb30f7640 1 -- 172.21.15.27:0/1400244058 >> 172.21.15.27:0/1400244058 conn(0x7f5cac0ceb50 msgr2=0x7f5cac0cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.672+0000 7f5cb30f7640 1 -- 172.21.15.27:0/1400244058 shutdown_connections 2023-12-19T18:23:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.672+0000 7f5cb30f7640 1 -- 172.21.15.27:0/1400244058 wait complete. 2023-12-19T18:23:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.674+0000 7f5cb30f7640 1 Processor -- start 2023-12-19T18:23:41.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.674+0000 7f5cb30f7640 1 -- start start 2023-12-19T18:23:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.675+0000 7f5cb30f7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c9c003970 0x7f5cac14e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.675+0000 7f5cb30f7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cac0b1b00 0x7f5cac1f4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.675+0000 7f5cb30f7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cac0c9180 0x7f5cac0c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.675+0000 7f5cb30f7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0b3060 con 0x7f5cac0b1b00 2023-12-19T18:23:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.675+0000 7f5cb30f7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0c9b50 con 0x7f5cac0c9180 2023-12-19T18:23:41.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.675+0000 7f5cb30f7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cac0b2bd0 con 0x7f5c9c003970 2023-12-19T18:23:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.675+0000 7f5cabfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cac0b1b00 0x7f5cac1f4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.675+0000 7f5cabfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cac0b1b00 0x7f5cac1f4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33614/0 (socket says 172.21.15.27:33614) 2023-12-19T18:23:41.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.675+0000 7f5cabfff640 1 -- 172.21.15.27:0/87440320 learned_addr learned my addr 172.21.15.27:0/87440320 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.676+0000 7f5cb1694640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cac0c9180 0x7f5cac0c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.676+0000 7f5cb0e93640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c9c003970 0x7f5cac14e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.676+0000 7f5cabfff640 1 -- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c9c003970 msgr2=0x7f5cac14e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.676+0000 7f5cabfff640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c9c003970 0x7f5cac14e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.676+0000 7f5cabfff640 1 -- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cac0c9180 msgr2=0x7f5cac0c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.676+0000 7f5cabfff640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cac0c9180 0x7f5cac0c9610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.676+0000 7f5cabfff640 1 -- 172.21.15.27:0/87440320 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac14f290 con 0x7f5cac0b1b00 2023-12-19T18:23:41.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.676+0000 7f5cabfff640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cac0b1b00 0x7f5cac1f4210 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800d7a0 tx=0x7f5c9800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.676+0000 7f5ca97fa640 1 -- 172.21.15.27:0/87440320 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c980074e0 con 0x7f5cac0b1b00 2023-12-19T18:23:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.676+0000 7f5ca97fa640 1 -- 172.21.15.27:0/87440320 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c98007640 con 0x7f5cac0b1b00 2023-12-19T18:23:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.676+0000 7f5ca97fa640 1 -- 172.21.15.27:0/87440320 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c98004a20 con 0x7f5cac0b1b00 2023-12-19T18:23:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.676+0000 7f5cb30f7640 1 -- 172.21.15.27:0/87440320 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cac1f4750 con 0x7f5cac0b1b00 2023-12-19T18:23:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.677+0000 7f5cb30f7640 1 -- 172.21.15.27:0/87440320 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cac0b22a0 con 0x7f5cac0b1b00 2023-12-19T18:23:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.678+0000 7f5cb30f7640 1 -- 172.21.15.27:0/87440320 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cac068e90 con 0x7f5cac0b1b00 2023-12-19T18:23:41.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.679+0000 7f5ca97fa640 1 -- 172.21.15.27:0/87440320 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5c98004b80 con 0x7f5cac0b1b00 2023-12-19T18:23:41.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.680+0000 7f5ca97fa640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c9006a460 0x7f5c9006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:41.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.680+0000 7f5ca97fa640 1 -- 172.21.15.27:0/87440320 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5c980881a0 con 0x7f5cac0b1b00 2023-12-19T18:23:41.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.680+0000 7f5cb0e93640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c9006a460 0x7f5c9006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:41.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.681+0000 7f5cb0e93640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c9006a460 0x7f5c9006c920 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0036000 tx=0x7f5ca0030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:41.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.684+0000 7f5ca97fa640 1 -- 172.21.15.27:0/87440320 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5c98053780 con 0x7f5cac0b1b00 2023-12-19T18:23:41.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.888+0000 7f5cb30f7640 1 -- 172.21.15.27:0/87440320 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f5cac0b2d10 con 0x7f5cac0b1b00 2023-12-19T18:23:41.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.890+0000 7f5ca97fa640 1 -- 172.21.15.27:0/87440320 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v155) v1 ==== 81+0+33558 (secure 0 0 0) 0x7f5c98056da0 con 0x7f5cac0b1b00 2023-12-19T18:23:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:41.891 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:23:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 155, 2023-12-19T18:23:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:23:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:23:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:23:36.712355+0000", 2023-12-19T18:23:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:23:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:23:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:23:41.891 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:23:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:23:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:23:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:23:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:23:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:23:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:23:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:23:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:23:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:23:41.892 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:23:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 17, 2023-12-19T18:23:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:23:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:23:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:23:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:23:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:23:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:23:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:23:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:23:41.893 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:23:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:41.894 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:23:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:23:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:23:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:23:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:23:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:41.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:23:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:41.896 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:41.897 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:41.898 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:23:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:23:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:23:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:23:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:23:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:23:41.899 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:23:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:23:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 16, 2023-12-19T18:23:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:23:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:04.431250+0000", 2023-12-19T18:23:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:41.900 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:23:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:23:41.901 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:23:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:23:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:23:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "151", 2023-12-19T18:23:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:41.902 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:23:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:41.903 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:41.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:41.905 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:23:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:23:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:23:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:23:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 2.25, 2023-12-19T18:23:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 2.25, 2023-12-19T18:23:41.906 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2.25, 2023-12-19T18:23:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2.25, 2023-12-19T18:23:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 17, 2023-12-19T18:23:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:23:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:23:05.506926+0000", 2023-12-19T18:23:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:23:41.907 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:23:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:23:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:23:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:23:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:23:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:23:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:23:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:23:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:23:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:23:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:23:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:23:41.908 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:23:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:23:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:23:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:23:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:23:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:23:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:23:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:23:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:23:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:23:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:23:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.909 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "155", 2023-12-19T18:23:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:23:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:23:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:23:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:23:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:23:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:23:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:23:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:23:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:23:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:23:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:23:41.910 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:23:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:23:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:23:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:23:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:23:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:23:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:23:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:23:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:23:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:23:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:23:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:23:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:23:41.911 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:23:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:23:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:23:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:23:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:23:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:23:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:23:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:23:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:23:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:23:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:23:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:23:41.912 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:23:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:23:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:23:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:23:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:23:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:23:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:23:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:23:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:23:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:23:41.913 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:23:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:23:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:23:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:23:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:23:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:23:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:23:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.914 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.915 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.916 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:23:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:23:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.917 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:23:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:23:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.918 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.919 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:23:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:23:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:23:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:23:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.920 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.921 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:23:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:23:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.922 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.923 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:23:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:23:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.924 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:23:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:23:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:23:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.925 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:23:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:23:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:23:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:23:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.926 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:23:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:23:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.928 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:23:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:23:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:23:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:23:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.930 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:23:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.931 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:23:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:23:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:23:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:23:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:23:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:23:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.933 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:23:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:23:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.934 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:23:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:23:41.935 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:23:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.936 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:23:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:23:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:23:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:23:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:23:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:23:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:23:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:23:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.938 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.939 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:23:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.940 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:23:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:23:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:23:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:23:41.942 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:23:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.943 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:23:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:23:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:23:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:23:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:23:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:23:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:23:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.945 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:23:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:23:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.949 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.949 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:23:41.949 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.949 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.949 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:23:41.949 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.949 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.949 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.949 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.949 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.950 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:23:41.950 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.950 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:23:41.950 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:23:41.950 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.950 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.951 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.952 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.952 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.952 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.952 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.952 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.952 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.952 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.952 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.952 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.952 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.954 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.955 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.955 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:23:41.955 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.955 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.955 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:23:41.955 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.955 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.955 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.955 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.955 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.956 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.956 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:23:41.956 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.956 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:23:41.956 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:23:41.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.956 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.957 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:23:41.957 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:23:41.957 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:23:41.957 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:23:41.957 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.957 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.957 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.957 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.957 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:41.957 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:23:41.958 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:23:41.958 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:23:41.958 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:23:41.958 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:23:41.958 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:23:41.958 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:23:41.958 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:23:41.958 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 150, 2023-12-19T18:23:41.958 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:23:41.958 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:23:41.958 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:23:41.958 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.959 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.959 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:23:41.959 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.959 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.959 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:23:41.959 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.959 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.959 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.959 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:23:41.960 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.960 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.960 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:23:41.960 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.960 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.960 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:23:41.960 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.960 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.960 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.961 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:23:41.961 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.961 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:23:41.961 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.961 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:23:41.961 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.961 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.962 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.962 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:23:41.962 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:23:41.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:23:41.962 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:23:41.962 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:23:41.962 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:23:41.962 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:23:41.963 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.963 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.963 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:23:41.963 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:23:41.963 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:23:41.963 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:23:41.963 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:23:41.963 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:23:41.963 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:23:41.963 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:23:41.963 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.963 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:41.964 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:23:41.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.964 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:23:41.964 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.964 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.964 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.964 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.964 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.964 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:23:41.964 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.964 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:23:41.965 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.965 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.965 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.965 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.965 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.965 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:23:41.965 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.965 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:23:41.965 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.966 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.966 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.966 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.966 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.966 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:23:41.966 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.966 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:23:41.966 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.966 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.966 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.967 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.967 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.967 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:23:41.967 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.967 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:23:41.967 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.967 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.967 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.967 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.967 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.968 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:23:41.968 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.968 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:23:41.968 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.968 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.968 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.968 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.968 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.968 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:23:41.968 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:23:41.969 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.969 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.969 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.969 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.969 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.969 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:23:41.969 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:23:41.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:23:41.969 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:23:41.970 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:23:41.970 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:23:41.970 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:23:41.970 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:23:41.970 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:23:41.970 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:23:41.970 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.970 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:23:41.970 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:23:41.970 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:23:41.970 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:23:41.970 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:23:41.970 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:23:41.971 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:23:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:23:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:23:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:23:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:23:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:23:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:23:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:23:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:23:41.972 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:23:41.972 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:23:41.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.893+0000 7f5cb30f7640 1 -- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c9006a460 msgr2=0x7f5c9006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:41.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.893+0000 7f5cb30f7640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c9006a460 0x7f5c9006c920 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0036000 tx=0x7f5ca0030000 comp rx=0 tx=0).stop 2023-12-19T18:23:41.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.893+0000 7f5cb30f7640 1 -- 172.21.15.27:0/87440320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cac0b1b00 msgr2=0x7f5cac1f4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:41.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.893+0000 7f5cb30f7640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cac0b1b00 0x7f5cac1f4210 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f5c9800d7a0 tx=0x7f5c9800dc60 comp rx=0 tx=0).stop 2023-12-19T18:23:41.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.893+0000 7f5cb30f7640 1 -- 172.21.15.27:0/87440320 shutdown_connections 2023-12-19T18:23:41.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.893+0000 7f5cb30f7640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5c9006a460 0x7f5c9006c920 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.893+0000 7f5cb30f7640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5cac0c9180 0x7f5cac0c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.893+0000 7f5cb30f7640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cac0b1b00 0x7f5cac1f4210 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.893+0000 7f5cb30f7640 1 --2- 172.21.15.27:0/87440320 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c9c003970 0x7f5cac14e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:41.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.893+0000 7f5cb30f7640 1 -- 172.21.15.27:0/87440320 >> 172.21.15.27:0/87440320 conn(0x7f5cac0ceb50 msgr2=0x7f5cac0cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:41.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.893+0000 7f5cb30f7640 1 -- 172.21.15.27:0/87440320 shutdown_connections 2023-12-19T18:23:41.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:41.893+0000 7f5cb30f7640 1 -- 172.21.15.27:0/87440320 wait complete. 2023-12-19T18:23:41.975 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:23:42.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.215+0000 7fa83ffff640 1 Processor -- start 2023-12-19T18:23:42.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.215+0000 7fa83ffff640 1 -- start start 2023-12-19T18:23:42.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.218+0000 7fa83ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa8400afac0 0x7fa8400c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:42.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.218+0000 7fa83ffff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8400cc190 con 0x7fa840154030 2023-12-19T18:23:42.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.218+0000 7fa83ffff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8400aff30 con 0x7fa8400af630 2023-12-19T18:23:42.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.218+0000 7fa83ffff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8400cdc00 con 0x7fa8400afac0 2023-12-19T18:23:42.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.218+0000 7fa83effd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa840154030 0x7fa8400cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59132/0 (socket says 172.21.15.27:59132) 2023-12-19T18:23:42.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.218+0000 7fa83effd640 1 -- 172.21.15.27:0/3302536414 learned_addr learned my addr 172.21.15.27:0/3302536414 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.218+0000 7fa83f7fe640 1 --2- 172.21.15.27:0/3302536414 >> v2:172.21.15.27:3300/0 conn(0x7fa8400afac0 0x7fa8400c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.218+0000 7fa83e7fc640 1 -- 172.21.15.27:0/3302536414 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1839048145 0 0) 0x7fa8400cc190 con 0x7fa840154030 2023-12-19T18:23:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.219+0000 7fa83e7fc640 1 -- 172.21.15.27:0/3302536414 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa82c003520 con 0x7fa840154030 2023-12-19T18:23:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.219+0000 7fa83f7fe640 1 -- 172.21.15.27:0/3302536414 >> v1:172.21.15.192:6789/0 conn(0x7fa8400af630 legacy=0x7fa8400c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.219+0000 7fa83f7fe640 1 -- 172.21.15.27:0/3302536414 >> v1:172.21.15.27:6789/0 conn(0x7fa840154030 legacy=0x7fa8400cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.219+0000 7fa83f7fe640 1 -- 172.21.15.27:0/3302536414 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8400cb2a0 con 0x7fa8400afac0 2023-12-19T18:23:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.220+0000 7fa83f7fe640 1 --2- 172.21.15.27:0/3302536414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8400afac0 0x7fa8400c4c60 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fa838009dc0 tx=0x7fa83802be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd42f77e9a28d19 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:42.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.220+0000 7fa83e7fc640 1 -- 172.21.15.27:0/3302536414 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83802cc40 con 0x7fa8400afac0 2023-12-19T18:23:42.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.220+0000 7fa83e7fc640 1 -- 172.21.15.27:0/3302536414 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83802cda0 con 0x7fa8400afac0 2023-12-19T18:23:42.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.220+0000 7fa83e7fc640 1 -- 172.21.15.27:0/3302536414 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa838034660 con 0x7fa8400afac0 2023-12-19T18:23:42.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.221+0000 7fa83ffff640 1 -- 172.21.15.27:0/3302536414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8400afac0 msgr2=0x7fa8400c4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:42.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.221+0000 7fa83ffff640 1 --2- 172.21.15.27:0/3302536414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8400afac0 0x7fa8400c4c60 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7fa838009dc0 tx=0x7fa83802be90 comp rx=0 tx=0).stop 2023-12-19T18:23:42.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.221+0000 7fa83ffff640 1 -- 172.21.15.27:0/3302536414 shutdown_connections 2023-12-19T18:23:42.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.221+0000 7fa83ffff640 1 --2- 172.21.15.27:0/3302536414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8400afac0 0x7fa8400c4c60 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:42.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.221+0000 7fa83ffff640 1 -- 172.21.15.27:0/3302536414 >> 172.21.15.27:0/3302536414 conn(0x7fa840069c70 msgr2=0x7fa84006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:42.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.221+0000 7fa83ffff640 1 -- 172.21.15.27:0/3302536414 shutdown_connections 2023-12-19T18:23:42.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.221+0000 7fa83ffff640 1 -- 172.21.15.27:0/3302536414 wait complete. 2023-12-19T18:23:42.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.223+0000 7fa83ffff640 1 Processor -- start 2023-12-19T18:23:42.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.223+0000 7fa83ffff640 1 -- start start 2023-12-19T18:23:42.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.224+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8400af630 0x7fa840147860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:42.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.224+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8400afac0 0x7fa84013b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:42.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.224+0000 7fa83ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa840154030 0x7fa840144950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:42.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.224+0000 7fa83ffff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa840145d40 con 0x7fa8400af630 2023-12-19T18:23:42.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.224+0000 7fa83ffff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84013f200 con 0x7fa8400afac0 2023-12-19T18:23:42.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.224+0000 7fa83ffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8401454a0 con 0x7fa840154030 2023-12-19T18:23:42.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.224+0000 7fa83effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8400af630 0x7fa840147860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:42.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.224+0000 7fa83effd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8400af630 0x7fa840147860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37376/0 (socket says 172.21.15.27:37376) 2023-12-19T18:23:42.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.224+0000 7fa83effd640 1 -- 172.21.15.27:0/1920889073 learned_addr learned my addr 172.21.15.27:0/1920889073 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:42.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.224+0000 7fa8367fc640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8400afac0 0x7fa84013b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:42.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.224+0000 7fa83f7fe640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa840154030 0x7fa840144950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:42.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.225+0000 7fa83effd640 1 -- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa840154030 msgr2=0x7fa840144950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:42.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.225+0000 7fa83effd640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa840154030 0x7fa840144950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:42.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.225+0000 7fa83effd640 1 -- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8400afac0 msgr2=0x7fa84013b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:42.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.225+0000 7fa83effd640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8400afac0 0x7fa84013b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:42.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.225+0000 7fa83effd640 1 -- 172.21.15.27:0/1920889073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa838009a70 con 0x7fa8400af630 2023-12-19T18:23:42.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.225+0000 7fa8367fc640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8400afac0 0x7fa84013b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:42.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.225+0000 7fa83effd640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8400af630 0x7fa840147860 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fa83000c3c0 tx=0x7fa83000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:42.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.225+0000 7fa83cff9640 1 -- 172.21.15.27:0/1920889073 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830005ce0 con 0x7fa8400af630 2023-12-19T18:23:42.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.225+0000 7fa83cff9640 1 -- 172.21.15.27:0/1920889073 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa830005e40 con 0x7fa8400af630 2023-12-19T18:23:42.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.225+0000 7fa83cff9640 1 -- 172.21.15.27:0/1920889073 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8300072f0 con 0x7fa8400af630 2023-12-19T18:23:42.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.225+0000 7fa83ffff640 1 -- 172.21.15.27:0/1920889073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa840145780 con 0x7fa8400af630 2023-12-19T18:23:42.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.225+0000 7fa83ffff640 1 -- 172.21.15.27:0/1920889073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa84006f970 con 0x7fa8400af630 2023-12-19T18:23:42.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.226+0000 7fa83ffff640 1 -- 172.21.15.27:0/1920889073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa84004e840 con 0x7fa8400af630 2023-12-19T18:23:42.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.228+0000 7fa83cff9640 1 -- 172.21.15.27:0/1920889073 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa830007450 con 0x7fa8400af630 2023-12-19T18:23:42.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.229+0000 7fa83cff9640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa82006a430 0x7fa82006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:42.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.229+0000 7fa83cff9640 1 -- 172.21.15.27:0/1920889073 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa830087f00 con 0x7fa8400af630 2023-12-19T18:23:42.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.230+0000 7fa8367fc640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa82006a430 0x7fa82006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:42.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.231+0000 7fa8367fc640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa82006a430 0x7fa82006c8f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa84006c510 tx=0x7fa828009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:42.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.234+0000 7fa83cff9640 1 -- 172.21.15.27:0/1920889073 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa8300534e0 con 0x7fa8400af630 2023-12-19T18:23:42.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.472+0000 7fa83ffff640 1 -- 172.21.15.27:0/1920889073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa8400c3c50 con 0x7fa8400af630 2023-12-19T18:23:42.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.473+0000 7fa83cff9640 1 -- 172.21.15.27:0/1920889073 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fa830017090 con 0x7fa8400af630 2023-12-19T18:23:42.474 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:42.474 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":61,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3298228307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3298228307},{"type":"v1","addr":"172.21.15.192:6835","nonce":3298228307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5726,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1760483999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1760483999},{"type":"v1","addr":"172.21.15.192:6837","nonce":1760483999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3000637822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3000637822},{"type":"v1","addr":"172.21.15.27:6833","nonce":3000637822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:23:06.465885+0000","modified":"2023-12-19T18:23:09.973228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1593995318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1593995318},{"type":"v1","addr":"172.21.15.27:6835","nonce":1593995318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T18:23:42.474 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 61 2023-12-19T18:23:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.476+0000 7fa83ffff640 1 -- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa82006a430 msgr2=0x7fa82006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.476+0000 7fa83ffff640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa82006a430 0x7fa82006c8f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fa84006c510 tx=0x7fa828009040 comp rx=0 tx=0).stop 2023-12-19T18:23:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.476+0000 7fa83ffff640 1 -- 172.21.15.27:0/1920889073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8400af630 msgr2=0x7fa840147860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.476+0000 7fa83ffff640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8400af630 0x7fa840147860 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fa83000c3c0 tx=0x7fa83000c880 comp rx=0 tx=0).stop 2023-12-19T18:23:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.476+0000 7fa83ffff640 1 -- 172.21.15.27:0/1920889073 shutdown_connections 2023-12-19T18:23:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.476+0000 7fa83ffff640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa82006a430 0x7fa82006c8f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.476+0000 7fa83ffff640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa840154030 0x7fa840144950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.476+0000 7fa83ffff640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8400afac0 0x7fa84013b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.476+0000 7fa83ffff640 1 --2- 172.21.15.27:0/1920889073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8400af630 0x7fa840147860 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.476+0000 7fa83ffff640 1 -- 172.21.15.27:0/1920889073 >> 172.21.15.27:0/1920889073 conn(0x7fa840069c70 msgr2=0x7fa840146c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.476+0000 7fa83ffff640 1 -- 172.21.15.27:0/1920889073 shutdown_connections 2023-12-19T18:23:42.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:42.476+0000 7fa83ffff640 1 -- 172.21.15.27:0/1920889073 wait complete. 2023-12-19T18:23:42.497 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-12-19T18:23:42.747 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":7794,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":36.259844098999999,"mdsmap_epoch":61,"osdmap_epoch":155,"osdmap_epoch_barrier":154,"uptime":42.066902075000002} 2023-12-19T18:23:42.747 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 7794, "mdsmap_epoch": 61, "osdmap_epoch": 155, "osdmap_epoch_barrier": 154, "rank_uptime": 36.259844099, "state": "up:active", "uptime": 42.066902075, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:23:42.748 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-12-19T18:23:42.748 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:23:42.748 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:23:42.803 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:42.803 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-12-19T18:23:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:50+0 records in 2023-12-19T18:23:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:50+0 records out 2023-12-19T18:23:43.940 INFO:teuthology.orchestra.run.smithi027.stderr:52428800 bytes (52 MB, 50 MiB) copied, 1.07563 s, 48.7 MB/s 2023-12-19T18:23:43.942 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:23:43.942 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:23:44.080 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:23:44.199 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:23:44.199 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:23:44.199 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:23:44.263 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:23:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.497+0000 7fea58e96640 1 Processor -- start 2023-12-19T18:23:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.497+0000 7fea58e96640 1 -- start start 2023-12-19T18:23:44.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.499+0000 7fea58e96640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea5415ce90 0x7fea540cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.499+0000 7fea58e96640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fea5415d270 0x7fea540ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.499+0000 7fea58e96640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea5406b4e0 con 0x7fea540ade10 2023-12-19T18:23:44.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.499+0000 7fea58e96640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea5406ce90 con 0x7fea5415d270 2023-12-19T18:23:44.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.499+0000 7fea58e96640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea540704f0 con 0x7fea5415ce90 2023-12-19T18:23:44.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.499+0000 7fea51d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fea5415d270 0x7fea540ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:44.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.500+0000 7fea51d9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fea5415d270 0x7fea540ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37386/0 (socket says 172.21.15.27:37386) 2023-12-19T18:23:44.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.500+0000 7fea51d9b640 1 -- 172.21.15.27:0/2499885640 learned_addr learned my addr 172.21.15.27:0/2499885640 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.500+0000 7fea5259c640 1 --2- 172.21.15.27:0/2499885640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea5415ce90 0x7fea540cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.500+0000 7fea51d9b640 1 -- 172.21.15.27:0/2499885640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea5415ce90 msgr2=0x7fea540cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.500+0000 7fea51d9b640 1 --2- 172.21.15.27:0/2499885640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea5415ce90 0x7fea540cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.500+0000 7fea51d9b640 1 -- 172.21.15.27:0/2499885640 >> v1:172.21.15.192:6789/0 conn(0x7fea540ade10 legacy=0x7fea540cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.500+0000 7fea51d9b640 1 -- 172.21.15.27:0/2499885640 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea5406b7c0 con 0x7fea5415d270 2023-12-19T18:23:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.500+0000 7fea5259c640 1 --2- 172.21.15.27:0/2499885640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea5415ce90 0x7fea540cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:44.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.501+0000 7fea51d9b640 1 --2- 172.21.15.27:0/2499885640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea5415d270 0x7fea540ccc50 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009e50 tx=0x7fea3c02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1819f74ac10d370e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:44.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.502+0000 7fea5159a640 1 -- 172.21.15.27:0/2499885640 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c02ca80 con 0x7fea5415d270 2023-12-19T18:23:44.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.502+0000 7fea5159a640 1 -- 172.21.15.27:0/2499885640 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c02cbe0 con 0x7fea5415d270 2023-12-19T18:23:44.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.502+0000 7fea5159a640 1 -- 172.21.15.27:0/2499885640 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c0343f0 con 0x7fea5415d270 2023-12-19T18:23:44.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.502+0000 7fea58e96640 1 -- 172.21.15.27:0/2499885640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea5415d270 msgr2=0x7fea540ccc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:44.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.502+0000 7fea58e96640 1 --2- 172.21.15.27:0/2499885640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea5415d270 0x7fea540ccc50 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009e50 tx=0x7fea3c02bee0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.502+0000 7fea58e96640 1 -- 172.21.15.27:0/2499885640 shutdown_connections 2023-12-19T18:23:44.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.502+0000 7fea58e96640 1 --2- 172.21.15.27:0/2499885640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea5415d270 0x7fea540ccc50 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.502+0000 7fea58e96640 1 --2- 172.21.15.27:0/2499885640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea5415ce90 0x7fea540cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.502+0000 7fea58e96640 1 -- 172.21.15.27:0/2499885640 >> 172.21.15.27:0/2499885640 conn(0x7fea5406a330 msgr2=0x7fea540b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:44.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.502+0000 7fea58e96640 1 -- 172.21.15.27:0/2499885640 shutdown_connections 2023-12-19T18:23:44.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.503+0000 7fea58e96640 1 -- 172.21.15.27:0/2499885640 wait complete. 2023-12-19T18:23:44.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.504+0000 7fea58e96640 1 Processor -- start 2023-12-19T18:23:44.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.504+0000 7fea58e96640 1 -- start start 2023-12-19T18:23:44.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.505+0000 7fea58e96640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea540ade10 0x7fea540b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.505+0000 7fea58e96640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea5415ce90 0x7fea540c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.505+0000 7fea58e96640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea540c49d0 0x7fea540b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.505+0000 7fea58e96640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea540c4e60 con 0x7fea5415ce90 2023-12-19T18:23:44.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.505+0000 7fea58e96640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea540b1de0 con 0x7fea540ade10 2023-12-19T18:23:44.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.505+0000 7fea58e96640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea540b3670 con 0x7fea540c49d0 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.506+0000 7fea51d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea5415ce90 0x7fea540c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.506+0000 7fea51d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea5415ce90 0x7fea540c4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37400/0 (socket says 172.21.15.27:37400) 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.506+0000 7fea51d9b640 1 -- 172.21.15.27:0/3093483601 learned_addr learned my addr 172.21.15.27:0/3093483601 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.506+0000 7fea51d9b640 1 -- 172.21.15.27:0/3093483601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea540c49d0 msgr2=0x7fea540b2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.506+0000 7fea5259c640 1 --2- 172.21.15.27:0/3093483601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea540ade10 0x7fea540b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.506+0000 7fea51d9b640 1 --2- 172.21.15.27:0/3093483601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea540c49d0 0x7fea540b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.506+0000 7fea51d9b640 1 -- 172.21.15.27:0/3093483601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea540ade10 msgr2=0x7fea540b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.506+0000 7fea51d9b640 1 --2- 172.21.15.27:0/3093483601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea540ade10 0x7fea540b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.506+0000 7fea51d9b640 1 -- 172.21.15.27:0/3093483601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c009b30 con 0x7fea5415ce90 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.506+0000 7fea51d9b640 1 --2- 172.21.15.27:0/3093483601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea5415ce90 0x7fea540c4490 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009e50 tx=0x7fea3c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.507+0000 7fea4affd640 1 -- 172.21.15.27:0/3093483601 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c03e450 con 0x7fea5415ce90 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.507+0000 7fea4affd640 1 -- 172.21.15.27:0/3093483601 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c03e5f0 con 0x7fea5415ce90 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.507+0000 7fea4affd640 1 -- 172.21.15.27:0/3093483601 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c03e9a0 con 0x7fea5415ce90 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.507+0000 7fea58e96640 1 -- 172.21.15.27:0/3093483601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5414efc0 con 0x7fea5415ce90 2023-12-19T18:23:44.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.507+0000 7fea58e96640 1 -- 172.21.15.27:0/3093483601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea540b1790 con 0x7fea5415ce90 2023-12-19T18:23:44.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.508+0000 7fea58e96640 1 -- 172.21.15.27:0/3093483601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea540b1f70 con 0x7fea5415ce90 2023-12-19T18:23:44.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.509+0000 7fea4affd640 1 -- 172.21.15.27:0/3093483601 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fea3c02cd50 con 0x7fea5415ce90 2023-12-19T18:23:44.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.510+0000 7fea4affd640 1 --2- 172.21.15.27:0/3093483601 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea3406a430 0x7fea3406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.510+0000 7fea4affd640 1 -- 172.21.15.27:0/3093483601 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fea3c0ab6c0 con 0x7fea5415ce90 2023-12-19T18:23:44.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.511+0000 7fea5259c640 1 --2- 172.21.15.27:0/3093483601 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea3406a430 0x7fea3406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:44.512 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.512+0000 7fea5259c640 1 --2- 172.21.15.27:0/3093483601 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea3406a430 0x7fea3406c8f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fea4400b840 tx=0x7fea44008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:44.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.515+0000 7fea4affd640 1 -- 172.21.15.27:0/3093483601 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fea3c076b40 con 0x7fea5415ce90 2023-12-19T18:23:44.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.712+0000 7fea58e96640 1 -- 172.21.15.27:0/3093483601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea54068e90 con 0x7fea5415ce90 2023-12-19T18:23:44.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.713+0000 7fea4affd640 1 -- 172.21.15.27:0/3093483601 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) v1 ==== 74+0+16220 (secure 0 0 0) 0x7fea3c07a160 con 0x7fea5415ce90 2023-12-19T18:23:44.713 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:44.714 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":156,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:23:43.989507+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:23:04.431250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T18:23:05.506926+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":4,"to":3}]},{"pgid":"17.f","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:23:44.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.716+0000 7fea58e96640 1 -- 172.21.15.27:0/3093483601 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea3406a430 msgr2=0x7fea3406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:44.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.716+0000 7fea58e96640 1 --2- 172.21.15.27:0/3093483601 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea3406a430 0x7fea3406c8f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fea4400b840 tx=0x7fea44008040 comp rx=0 tx=0).stop 2023-12-19T18:23:44.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.716+0000 7fea58e96640 1 -- 172.21.15.27:0/3093483601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea5415ce90 msgr2=0x7fea540c4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:44.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.716+0000 7fea58e96640 1 --2- 172.21.15.27:0/3093483601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea5415ce90 0x7fea540c4490 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fea3c009e50 tx=0x7fea3c004a40 comp rx=0 tx=0).stop 2023-12-19T18:23:44.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.716+0000 7fea58e96640 1 -- 172.21.15.27:0/3093483601 shutdown_connections 2023-12-19T18:23:44.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.716+0000 7fea58e96640 1 --2- 172.21.15.27:0/3093483601 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea3406a430 0x7fea3406c8f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.716+0000 7fea58e96640 1 --2- 172.21.15.27:0/3093483601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea540c49d0 0x7fea540b2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.716+0000 7fea58e96640 1 --2- 172.21.15.27:0/3093483601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea5415ce90 0x7fea540c4490 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.716+0000 7fea58e96640 1 --2- 172.21.15.27:0/3093483601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea540ade10 0x7fea540b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.716+0000 7fea58e96640 1 -- 172.21.15.27:0/3093483601 >> 172.21.15.27:0/3093483601 conn(0x7fea5406a330 msgr2=0x7fea541608a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:44.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.716+0000 7fea58e96640 1 -- 172.21.15.27:0/3093483601 shutdown_connections 2023-12-19T18:23:44.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.716+0000 7fea58e96640 1 -- 172.21.15.27:0/3093483601 wait complete. 2023-12-19T18:23:44.737 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:23:04.431250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:23:05.506926+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:23:44.737 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:23:44.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.968+0000 7f5a40fd1640 1 Processor -- start 2023-12-19T18:23:44.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.968+0000 7f5a40fd1640 1 -- start start 2023-12-19T18:23:44.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.970+0000 7f5a40fd1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5a3c151c90 0x7f5a3c06b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.970+0000 7f5a40fd1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a3c0cf550 0x7f5a3c15b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.970+0000 7f5a40fd1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a3c15b3e0 con 0x7f5a3c15e9d0 2023-12-19T18:23:44.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.970+0000 7f5a40fd1640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a3c15eed0 con 0x7f5a3c151c90 2023-12-19T18:23:44.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.970+0000 7f5a40fd1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c15f1b0 con 0x7f5a3c0cf550 2023-12-19T18:23:44.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.970+0000 7f5a3affd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5a3c15e9d0 0x7f5a3c0cee40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59140/0 (socket says 172.21.15.27:59140) 2023-12-19T18:23:44.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.970+0000 7f5a3affd640 1 -- 172.21.15.27:0/89502272 learned_addr learned my addr 172.21.15.27:0/89502272 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3b7fe640 1 --2- 172.21.15.27:0/89502272 >> v2:172.21.15.192:3300/0 conn(0x7f5a3c151c90 0x7f5a3c06b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3bfff640 1 --2- 172.21.15.27:0/89502272 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a3c0cf550 0x7f5a3c15b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 401541787 0 0) 0x7f5a3c15b3e0 con 0x7f5a3c15e9d0 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a28003520 con 0x7f5a3c15e9d0 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1810645490 0 0) 0x7f5a28003520 con 0x7f5a3c15e9d0 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a3c0cf550 msgr2=0x7f5a3c15b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 --2- 172.21.15.27:0/89502272 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a3c0cf550 0x7f5a3c15b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 >> v2:172.21.15.192:3300/0 conn(0x7f5a3c151c90 msgr2=0x7f5a3c06b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 --2- 172.21.15.27:0/89502272 >> v2:172.21.15.192:3300/0 conn(0x7f5a3c151c90 0x7f5a3c06b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c15f490 con 0x7f5a3c15e9d0 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f5a24001670 con 0x7f5a3c15e9d0 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 >> v1:172.21.15.27:6789/0 conn(0x7f5a3c15e9d0 legacy=0x7f5a3c0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 --2- 172.21.15.27:0/89502272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a28003970 0x7f5a28003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 --2- 172.21.15.27:0/89502272 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a3c0cf550 0x7f5a280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 --2- 172.21.15.27:0/89502272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a28004d00 0x7f5a28005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c15eed0 con 0x7f5a28004d00 2023-12-19T18:23:44.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c15f1b0 con 0x7f5a28003970 2023-12-19T18:23:44.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.971+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2800b9b0 con 0x7f5a3c0cf550 2023-12-19T18:23:44.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.972+0000 7f5a3b7fe640 1 --2- 172.21.15.27:0/89502272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a28003970 0x7f5a28003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:44.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.972+0000 7f5a3bfff640 1 --2- 172.21.15.27:0/89502272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a28004d00 0x7f5a28005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:44.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.972+0000 7f5a3affd640 1 --2- 172.21.15.27:0/89502272 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a3c0cf550 0x7f5a280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.972+0000 7f5a3bfff640 1 -- 172.21.15.27:0/89502272 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a3c0cf550 msgr2=0x7f5a280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.972+0000 7f5a3bfff640 1 --2- 172.21.15.27:0/89502272 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a3c0cf550 0x7f5a280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.972+0000 7f5a3bfff640 1 -- 172.21.15.27:0/89502272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a28003970 msgr2=0x7f5a28003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.972+0000 7f5a3bfff640 1 --2- 172.21.15.27:0/89502272 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a28003970 0x7f5a28003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.972+0000 7f5a3bfff640 1 -- 172.21.15.27:0/89502272 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a28003520 con 0x7f5a28004d00 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.973+0000 7f5a3bfff640 1 --2- 172.21.15.27:0/89502272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a28004d00 0x7f5a28005180 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009a90 tx=0x7f5a280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783992d236c7ebea server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.974+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3402c680 con 0x7f5a28004d00 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.974+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a3402c7e0 con 0x7f5a28004d00 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.974+0000 7f5a3a7fc640 1 -- 172.21.15.27:0/89502272 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3402cb70 con 0x7f5a28004d00 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.974+0000 7f5a40fd1640 1 -- 172.21.15.27:0/89502272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a28004d00 msgr2=0x7f5a28005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.974+0000 7f5a40fd1640 1 --2- 172.21.15.27:0/89502272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a28004d00 0x7f5a28005180 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009a90 tx=0x7f5a280057f0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.974+0000 7f5a3bfff640 1 -- 172.21.15.27:0/89502272 reap_dead start 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.975+0000 7f5a40fd1640 1 -- 172.21.15.27:0/89502272 shutdown_connections 2023-12-19T18:23:44.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.975+0000 7f5a40fd1640 1 -- 172.21.15.27:0/89502272 >> 172.21.15.27:0/89502272 conn(0x7f5a3c06a060 msgr2=0x7f5a3c06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:44.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.975+0000 7f5a40fd1640 1 -- 172.21.15.27:0/89502272 shutdown_connections 2023-12-19T18:23:44.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.975+0000 7f5a40fd1640 1 -- 172.21.15.27:0/89502272 wait complete. 2023-12-19T18:23:44.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.976+0000 7f5a40fd1640 1 Processor -- start 2023-12-19T18:23:44.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.977+0000 7f5a40fd1640 1 -- start start 2023-12-19T18:23:44.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a40fd1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a28004d00 0x7f5a3c15c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a40fd1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a3c15de70 0x7f5a3c1e73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a40fd1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a3c13e6d0 0x7f5a3c1e47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a40fd1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c15cdd0 con 0x7f5a3c15de70 2023-12-19T18:23:44.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a40fd1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c15d0b0 con 0x7f5a28004d00 2023-12-19T18:23:44.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a40fd1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a3c15d390 con 0x7f5a3c13e6d0 2023-12-19T18:23:44.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a3affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a3c15de70 0x7f5a3c1e73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:44.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a3affd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a3c15de70 0x7f5a3c1e73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37410/0 (socket says 172.21.15.27:37410) 2023-12-19T18:23:44.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a3affd640 1 -- 172.21.15.27:0/1107445044 learned_addr learned my addr 172.21.15.27:0/1107445044 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:44.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a3affd640 1 -- 172.21.15.27:0/1107445044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a3c13e6d0 msgr2=0x7f5a3c1e47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:44.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a3affd640 1 --2- 172.21.15.27:0/1107445044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a3c13e6d0 0x7f5a3c1e47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a3affd640 1 -- 172.21.15.27:0/1107445044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a28004d00 msgr2=0x7f5a3c15c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:44.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a3affd640 1 --2- 172.21.15.27:0/1107445044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a28004d00 0x7f5a3c15c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:44.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.978+0000 7f5a3affd640 1 -- 172.21.15.27:0/1107445044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a3c15d670 con 0x7f5a3c15de70 2023-12-19T18:23:44.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.979+0000 7f5a3affd640 1 --2- 172.21.15.27:0/1107445044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a3c15de70 0x7f5a3c1e73b0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400e780 tx=0x7f5a2400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:44.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.979+0000 7f5a33fff640 1 -- 172.21.15.27:0/1107445044 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2400ee90 con 0x7f5a3c15de70 2023-12-19T18:23:44.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.979+0000 7f5a33fff640 1 -- 172.21.15.27:0/1107445044 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a2400c8d0 con 0x7f5a3c15de70 2023-12-19T18:23:44.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.979+0000 7f5a33fff640 1 -- 172.21.15.27:0/1107445044 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a2400cc60 con 0x7f5a3c15de70 2023-12-19T18:23:44.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.979+0000 7f5a40fd1640 1 -- 172.21.15.27:0/1107445044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a3c15d950 con 0x7f5a3c15de70 2023-12-19T18:23:44.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.979+0000 7f5a40fd1640 1 -- 172.21.15.27:0/1107445044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a3c1e4d00 con 0x7f5a3c15de70 2023-12-19T18:23:44.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.980+0000 7f5a40fd1640 1 -- 172.21.15.27:0/1107445044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a3c04e840 con 0x7f5a3c15de70 2023-12-19T18:23:44.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.981+0000 7f5a33fff640 1 -- 172.21.15.27:0/1107445044 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5a2401d460 con 0x7f5a3c15de70 2023-12-19T18:23:44.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.983+0000 7f5a33fff640 1 --2- 172.21.15.27:0/1107445044 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5a1c06a460 0x7f5a1c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:44.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.983+0000 7f5a33fff640 1 -- 172.21.15.27:0/1107445044 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5a24088980 con 0x7f5a3c15de70 2023-12-19T18:23:44.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.983+0000 7f5a3b7fe640 1 --2- 172.21.15.27:0/1107445044 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5a1c06a460 0x7f5a1c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:44.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.985+0000 7f5a3b7fe640 1 --2- 172.21.15.27:0/1107445044 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5a1c06a460 0x7f5a1c06c920 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009f40 tx=0x7f5a2c009750 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:44.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:44.987+0000 7f5a33fff640 1 -- 172.21.15.27:0/1107445044 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5a240579b0 con 0x7f5a3c15de70 2023-12-19T18:23:45.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.189+0000 7f5a40fd1640 1 -- 172.21.15.27:0/1107445044 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5a3c06c9c0 con 0x7f5a3c15de70 2023-12-19T18:23:45.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.191+0000 7f5a33fff640 1 -- 172.21.15.27:0/1107445044 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v157) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5a24057540 con 0x7f5a3c15de70 2023-12-19T18:23:45.191 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:23:45.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.193+0000 7f5a40fd1640 1 -- 172.21.15.27:0/1107445044 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5a1c06a460 msgr2=0x7f5a1c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:45.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.193+0000 7f5a40fd1640 1 --2- 172.21.15.27:0/1107445044 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5a1c06a460 0x7f5a1c06c920 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009f40 tx=0x7f5a2c009750 comp rx=0 tx=0).stop 2023-12-19T18:23:45.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.193+0000 7f5a40fd1640 1 -- 172.21.15.27:0/1107445044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a3c15de70 msgr2=0x7f5a3c1e73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:45.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.193+0000 7f5a40fd1640 1 --2- 172.21.15.27:0/1107445044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a3c15de70 0x7f5a3c1e73b0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5a2400e780 tx=0x7f5a2400ec40 comp rx=0 tx=0).stop 2023-12-19T18:23:45.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.193+0000 7f5a40fd1640 1 -- 172.21.15.27:0/1107445044 shutdown_connections 2023-12-19T18:23:45.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.193+0000 7f5a40fd1640 1 --2- 172.21.15.27:0/1107445044 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5a1c06a460 0x7f5a1c06c920 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.193+0000 7f5a40fd1640 1 --2- 172.21.15.27:0/1107445044 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5a3c13e6d0 0x7f5a3c1e47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.193+0000 7f5a40fd1640 1 --2- 172.21.15.27:0/1107445044 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a3c15de70 0x7f5a3c1e73b0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.193+0000 7f5a40fd1640 1 --2- 172.21.15.27:0/1107445044 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5a28004d00 0x7f5a3c15c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.193+0000 7f5a40fd1640 1 -- 172.21.15.27:0/1107445044 >> 172.21.15.27:0/1107445044 conn(0x7f5a3c06a060 msgr2=0x7f5a3c0cee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:45.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.193+0000 7f5a40fd1640 1 -- 172.21.15.27:0/1107445044 shutdown_connections 2023-12-19T18:23:45.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.193+0000 7f5a40fd1640 1 -- 172.21.15.27:0/1107445044 wait complete. 2023-12-19T18:23:45.212 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:23:45.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.450+0000 7f6ccd327640 1 Processor -- start 2023-12-19T18:23:45.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.450+0000 7f6ccd327640 1 -- start start 2023-12-19T18:23:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.451+0000 7f6ccd327640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cc815a0b0 0x7f6cc80cdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.451+0000 7f6ccd327640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc80c9880 con 0x7f6cc80ce210 2023-12-19T18:23:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.452+0000 7f6ccd327640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cc80c9b20 con 0x7f6cc806cee0 2023-12-19T18:23:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.452+0000 7f6ccd327640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80c9de0 con 0x7f6cc815a0b0 2023-12-19T18:23:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.452+0000 7f6cc67fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6cc80ce210 0x7f6cc80cd400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59144/0 (socket says 172.21.15.27:59144) 2023-12-19T18:23:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.452+0000 7f6cc67fc640 1 -- 172.21.15.27:0/4251029810 learned_addr learned my addr 172.21.15.27:0/4251029810 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.452+0000 7f6cc6ffd640 1 --2- 172.21.15.27:0/4251029810 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cc815a0b0 0x7f6cc80cdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.452+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 36322568 0 0) 0x7f6cc80c9880 con 0x7f6cc80ce210 2023-12-19T18:23:45.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.452+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cb4003520 con 0x7f6cc80ce210 2023-12-19T18:23:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2297045642 0 0) 0x7f6cb4003520 con 0x7f6cc80ce210 2023-12-19T18:23:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cc815a0b0 msgr2=0x7f6cc80cdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc5ffb640 1 --2- 172.21.15.27:0/4251029810 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cc815a0b0 0x7f6cc80cdc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 >> v1:172.21.15.192:6789/0 conn(0x7f6cc806cee0 legacy=0x7f6cc8070030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:45.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80cc920 con 0x7f6cc80ce210 2023-12-19T18:23:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f6cb0001670 con 0x7f6cc80ce210 2023-12-19T18:23:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 >> v1:172.21.15.27:6789/0 conn(0x7f6cc80ce210 legacy=0x7f6cc80cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc5ffb640 1 --2- 172.21.15.27:0/4251029810 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cc815a0b0 0x7f6cb40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc5ffb640 1 --2- 172.21.15.27:0/4251029810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cb4003f70 0x7f6cb40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc5ffb640 1 --2- 172.21.15.27:0/4251029810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb4004d00 0x7f6cb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80c9de0 con 0x7f6cb4003f70 2023-12-19T18:23:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb400bac0 con 0x7f6cb4004d00 2023-12-19T18:23:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cb400bc80 con 0x7f6cc815a0b0 2023-12-19T18:23:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.453+0000 7f6cc67fc640 1 --2- 172.21.15.27:0/4251029810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cb4003f70 0x7f6cb40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.454+0000 7f6cc6ffd640 1 --2- 172.21.15.27:0/4251029810 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cc815a0b0 0x7f6cb40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.454+0000 7f6cc77fe640 1 --2- 172.21.15.27:0/4251029810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb4004d00 0x7f6cb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.454+0000 7f6cc67fc640 1 -- 172.21.15.27:0/4251029810 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cc815a0b0 msgr2=0x7f6cb40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:45.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.454+0000 7f6cc67fc640 1 --2- 172.21.15.27:0/4251029810 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cc815a0b0 0x7f6cb40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.454+0000 7f6cc67fc640 1 -- 172.21.15.27:0/4251029810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb4004d00 msgr2=0x7f6cb4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:45.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.454+0000 7f6cc67fc640 1 --2- 172.21.15.27:0/4251029810 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb4004d00 0x7f6cb4005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.454+0000 7f6cc67fc640 1 -- 172.21.15.27:0/4251029810 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cb4003520 con 0x7f6cb4003f70 2023-12-19T18:23:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.455+0000 7f6cc67fc640 1 --2- 172.21.15.27:0/4251029810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cb4003f70 0x7f6cb40047c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000c0e0 tx=0x7f6cb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d30616871d563c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.455+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb0005ad0 con 0x7f6cb4003f70 2023-12-19T18:23:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.455+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb0005c30 con 0x7f6cb4003f70 2023-12-19T18:23:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.455+0000 7f6cc5ffb640 1 -- 172.21.15.27:0/4251029810 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb00072b0 con 0x7f6cb4003f70 2023-12-19T18:23:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.456+0000 7f6ccd327640 1 -- 172.21.15.27:0/4251029810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cb4003f70 msgr2=0x7f6cb40047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.456+0000 7f6ccd327640 1 --2- 172.21.15.27:0/4251029810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cb4003f70 0x7f6cb40047c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000c0e0 tx=0x7f6cb40057f0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.456+0000 7f6cc77fe640 1 -- 172.21.15.27:0/4251029810 reap_dead start 2023-12-19T18:23:45.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.456+0000 7f6ccd327640 1 -- 172.21.15.27:0/4251029810 shutdown_connections 2023-12-19T18:23:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.456+0000 7f6ccd327640 1 -- 172.21.15.27:0/4251029810 >> 172.21.15.27:0/4251029810 conn(0x7f6cc80cebd0 msgr2=0x7f6cc80cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.456+0000 7f6ccd327640 1 -- 172.21.15.27:0/4251029810 shutdown_connections 2023-12-19T18:23:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.456+0000 7f6ccd327640 1 -- 172.21.15.27:0/4251029810 wait complete. 2023-12-19T18:23:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.458+0000 7f6ccd327640 1 Processor -- start 2023-12-19T18:23:45.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.458+0000 7f6ccd327640 1 -- start start 2023-12-19T18:23:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.459+0000 7f6ccd327640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb4003f70 0x7f6cc814b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.459+0000 7f6ccd327640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc815ae40 0x7f6cc815b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.459+0000 7f6ccd327640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cc81f1a50 0x7f6cc81f1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.459+0000 7f6ccd327640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc80b38c0 con 0x7f6cc815ae40 2023-12-19T18:23:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.459+0000 7f6ccd327640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc815b800 con 0x7f6cb4003f70 2023-12-19T18:23:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.459+0000 7f6ccd327640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cc81f27c0 con 0x7f6cc81f1a50 2023-12-19T18:23:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.459+0000 7f6cc67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc815ae40 0x7f6cc815b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.459+0000 7f6cc67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc815ae40 0x7f6cc815b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37426/0 (socket says 172.21.15.27:37426) 2023-12-19T18:23:45.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.459+0000 7f6cc67fc640 1 -- 172.21.15.27:0/2054075909 learned_addr learned my addr 172.21.15.27:0/2054075909 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.460+0000 7f6cc77fe640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cc81f1a50 0x7f6cc81f1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.460+0000 7f6cc6ffd640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb4003f70 0x7f6cc814b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.460+0000 7f6cc67fc640 1 -- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cc81f1a50 msgr2=0x7f6cc81f1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.460+0000 7f6cc67fc640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cc81f1a50 0x7f6cc81f1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.460+0000 7f6cc67fc640 1 -- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb4003f70 msgr2=0x7f6cc814b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.460+0000 7f6cc67fc640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb4003f70 0x7f6cc814b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.460+0000 7f6cc67fc640 1 -- 172.21.15.27:0/2054075909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc80b2f20 con 0x7f6cc815ae40 2023-12-19T18:23:45.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.460+0000 7f6cc67fc640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc815ae40 0x7f6cc815b2c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000c0e0 tx=0x7f6cb0001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.461+0000 7f6cbf7fe640 1 -- 172.21.15.27:0/2054075909 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb003c040 con 0x7f6cc815ae40 2023-12-19T18:23:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.461+0000 7f6cbf7fe640 1 -- 172.21.15.27:0/2054075909 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb0007580 con 0x7f6cc815ae40 2023-12-19T18:23:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.461+0000 7f6cbf7fe640 1 -- 172.21.15.27:0/2054075909 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb002c740 con 0x7f6cc815ae40 2023-12-19T18:23:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.461+0000 7f6ccd327640 1 -- 172.21.15.27:0/2054075909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cc80b3200 con 0x7f6cc815ae40 2023-12-19T18:23:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.461+0000 7f6ccd327640 1 -- 172.21.15.27:0/2054075909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cc80b3ce0 con 0x7f6cc815ae40 2023-12-19T18:23:45.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.462+0000 7f6ccd327640 1 -- 172.21.15.27:0/2054075909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cc8068e90 con 0x7f6cc815ae40 2023-12-19T18:23:45.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.463+0000 7f6cbf7fe640 1 -- 172.21.15.27:0/2054075909 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6cb0032070 con 0x7f6cc815ae40 2023-12-19T18:23:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.464+0000 7f6cbf7fe640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6ca806a420 0x7f6ca806c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.464+0000 7f6cbf7fe640 1 -- 172.21.15.27:0/2054075909 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6cb00a9a90 con 0x7f6cc815ae40 2023-12-19T18:23:45.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.465+0000 7f6cc6ffd640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6ca806a420 0x7f6ca806c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.466+0000 7f6cc6ffd640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6ca806a420 0x7f6ca806c8e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8007380 tx=0x7f6cb800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:45.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.468+0000 7f6cbf7fe640 1 -- 172.21.15.27:0/2054075909 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6cb0075030 con 0x7f6cc815ae40 2023-12-19T18:23:45.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.660+0000 7f6ccd327640 1 -- 172.21.15.27:0/2054075909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6cc806b6f0 con 0x7f6cc815ae40 2023-12-19T18:23:45.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.661+0000 7f6cbf7fe640 1 -- 172.21.15.27:0/2054075909 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v157) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6cb00460a0 con 0x7f6cc815ae40 2023-12-19T18:23:45.661 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:23:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.664+0000 7f6ccd327640 1 -- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6ca806a420 msgr2=0x7f6ca806c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.664+0000 7f6ccd327640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6ca806a420 0x7f6ca806c8e0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f6cb8007380 tx=0x7f6cb800b040 comp rx=0 tx=0).stop 2023-12-19T18:23:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.664+0000 7f6ccd327640 1 -- 172.21.15.27:0/2054075909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc815ae40 msgr2=0x7f6cc815b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.664+0000 7f6ccd327640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc815ae40 0x7f6cc815b2c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f6cb000c0e0 tx=0x7f6cb0001930 comp rx=0 tx=0).stop 2023-12-19T18:23:45.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.664+0000 7f6ccd327640 1 -- 172.21.15.27:0/2054075909 shutdown_connections 2023-12-19T18:23:45.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.664+0000 7f6ccd327640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6ca806a420 0x7f6ca806c8e0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.664+0000 7f6ccd327640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6cc81f1a50 0x7f6cc81f1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.664+0000 7f6ccd327640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cc815ae40 0x7f6cc815b2c0 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.664+0000 7f6ccd327640 1 --2- 172.21.15.27:0/2054075909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6cb4003f70 0x7f6cc814b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.664+0000 7f6ccd327640 1 -- 172.21.15.27:0/2054075909 >> 172.21.15.27:0/2054075909 conn(0x7f6cc80cebd0 msgr2=0x7f6cc8159250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:45.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.664+0000 7f6ccd327640 1 -- 172.21.15.27:0/2054075909 shutdown_connections 2023-12-19T18:23:45.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.664+0000 7f6ccd327640 1 -- 172.21.15.27:0/2054075909 wait complete. 2023-12-19T18:23:45.682 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:23:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.907+0000 7f880acec640 1 Processor -- start 2023-12-19T18:23:45.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.907+0000 7f880acec640 1 -- start start 2023-12-19T18:23:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.908+0000 7f880acec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880415b1a0 0x7f880406f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.908+0000 7f880acec640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f88040c2720 0x7f880406fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.908+0000 7f880acec640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f880406bc80 con 0x7f8804071420 2023-12-19T18:23:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.908+0000 7f880acec640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f880415fd90 con 0x7f88040c2720 2023-12-19T18:23:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.909+0000 7f880acec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f880406d690 con 0x7f880415b1a0 2023-12-19T18:23:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.909+0000 7f8803fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f88040c2720 0x7f880406fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.909+0000 7f8808a88640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880415b1a0 0x7f880406f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.909+0000 7f8809289640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f8804071420 0x7f88040c9d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:46978/0 (socket says 172.21.15.27:46978) 2023-12-19T18:23:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.909+0000 7f8803fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f88040c2720 0x7f880406fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:33506/0 (socket says 172.21.15.27:33506) 2023-12-19T18:23:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.909+0000 7f8809289640 1 -- 172.21.15.27:0/2019029727 learned_addr learned my addr 172.21.15.27:0/2019029727 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.910+0000 7f8808a88640 1 -- 172.21.15.27:0/2019029727 >> v2:172.21.15.192:3300/0 conn(0x7f88040c2720 msgr2=0x7f880406fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.910+0000 7f8808a88640 1 --2- 172.21.15.27:0/2019029727 >> v2:172.21.15.192:3300/0 conn(0x7f88040c2720 0x7f880406fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.910+0000 7f8808a88640 1 -- 172.21.15.27:0/2019029727 >> v1:172.21.15.192:6789/0 conn(0x7f8804071420 legacy=0x7f88040c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.910+0000 7f8808a88640 1 -- 172.21.15.27:0/2019029727 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f880406d0a0 con 0x7f880415b1a0 2023-12-19T18:23:45.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.910+0000 7f8803fff640 1 --2- 172.21.15.27:0/2019029727 >> v2:172.21.15.192:3300/0 conn(0x7f88040c2720 0x7f880406fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:23:45.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.912+0000 7f8808a88640 1 --2- 172.21.15.27:0/2019029727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880415b1a0 0x7f880406f520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f87f80097a0 tx=0x7f87f802b820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ada3921ee535436 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:45.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.912+0000 7f88037fe640 1 -- 172.21.15.27:0/2019029727 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8031780 con 0x7f880415b1a0 2023-12-19T18:23:45.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.913+0000 7f88037fe640 1 -- 172.21.15.27:0/2019029727 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f80318e0 con 0x7f880415b1a0 2023-12-19T18:23:45.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.913+0000 7f88037fe640 1 -- 172.21.15.27:0/2019029727 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f8031c70 con 0x7f880415b1a0 2023-12-19T18:23:45.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.913+0000 7f880acec640 1 -- 172.21.15.27:0/2019029727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880415b1a0 msgr2=0x7f880406f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:45.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.913+0000 7f880acec640 1 --2- 172.21.15.27:0/2019029727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880415b1a0 0x7f880406f520 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f87f80097a0 tx=0x7f87f802b820 comp rx=0 tx=0).stop 2023-12-19T18:23:45.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.913+0000 7f880acec640 1 -- 172.21.15.27:0/2019029727 shutdown_connections 2023-12-19T18:23:45.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.913+0000 7f880acec640 1 --2- 172.21.15.27:0/2019029727 >> v2:172.21.15.192:3300/0 conn(0x7f88040c2720 0x7f880406fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.913+0000 7f880acec640 1 --2- 172.21.15.27:0/2019029727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f880415b1a0 0x7f880406f520 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.913+0000 7f880acec640 1 -- 172.21.15.27:0/2019029727 >> 172.21.15.27:0/2019029727 conn(0x7f88040cebd0 msgr2=0x7f88040cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:45.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.914+0000 7f880acec640 1 -- 172.21.15.27:0/2019029727 shutdown_connections 2023-12-19T18:23:45.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.914+0000 7f880acec640 1 -- 172.21.15.27:0/2019029727 wait complete. 2023-12-19T18:23:45.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.915+0000 7f880acec640 1 Processor -- start 2023-12-19T18:23:45.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.916+0000 7f880acec640 1 -- start start 2023-12-19T18:23:45.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.916+0000 7f880acec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8804071420 0x7f8804158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.916+0000 7f880acec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88040c2720 0x7f88041f27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.916+0000 7f880acec640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f880415b1a0 0x7f88040aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.916+0000 7f880acec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88040aa220 con 0x7f88040c2720 2023-12-19T18:23:45.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.917+0000 7f880acec640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88041f3280 con 0x7f880415b1a0 2023-12-19T18:23:45.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.917+0000 7f880acec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88041f3560 con 0x7f8804071420 2023-12-19T18:23:45.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.917+0000 7f8803fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88040c2720 0x7f88041f27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.917+0000 7f8803fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88040c2720 0x7f88041f27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37442/0 (socket says 172.21.15.27:37442) 2023-12-19T18:23:45.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.917+0000 7f8803fff640 1 -- 172.21.15.27:0/1889730500 learned_addr learned my addr 172.21.15.27:0/1889730500 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:45.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.917+0000 7f8808a88640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8804071420 0x7f8804158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.917+0000 7f8809289640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f880415b1a0 0x7f88040aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.917+0000 7f8803fff640 1 -- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8804071420 msgr2=0x7f8804158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:45.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.917+0000 7f8803fff640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8804071420 0x7f8804158220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.917+0000 7f8803fff640 1 -- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f880415b1a0 msgr2=0x7f88040aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:45.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.917+0000 7f8803fff640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f880415b1a0 0x7f88040aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.917+0000 7f8803fff640 1 -- 172.21.15.27:0/1889730500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f4009400 con 0x7f88040c2720 2023-12-19T18:23:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.918+0000 7f8803fff640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88040c2720 0x7f88041f27c0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f87f400e9b0 tx=0x7f87f400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.918+0000 7f88017fa640 1 -- 172.21.15.27:0/1889730500 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f400ca00 con 0x7f88040c2720 2023-12-19T18:23:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.918+0000 7f88017fa640 1 -- 172.21.15.27:0/1889730500 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87f400cb60 con 0x7f88040c2720 2023-12-19T18:23:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.918+0000 7f88017fa640 1 -- 172.21.15.27:0/1889730500 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87f4013470 con 0x7f88040c2720 2023-12-19T18:23:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.918+0000 7f880acec640 1 -- 172.21.15.27:0/1889730500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87f8009400 con 0x7f88040c2720 2023-12-19T18:23:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.918+0000 7f880acec640 1 -- 172.21.15.27:0/1889730500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88040c8d20 con 0x7f88040c2720 2023-12-19T18:23:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.920+0000 7f880acec640 1 -- 172.21.15.27:0/1889730500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8804068e90 con 0x7f88040c2720 2023-12-19T18:23:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.920+0000 7f88017fa640 1 -- 172.21.15.27:0/1889730500 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f87f40135d0 con 0x7f88040c2720 2023-12-19T18:23:45.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.921+0000 7f88017fa640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f87f006a430 0x7f87f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:45.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.922+0000 7f88017fa640 1 -- 172.21.15.27:0/1889730500 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f87f4088ca0 con 0x7f88040c2720 2023-12-19T18:23:45.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.922+0000 7f8808a88640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f87f006a430 0x7f87f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:45.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.923+0000 7f8808a88640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f87f006a430 0x7f87f006c8f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f87f8036000 tx=0x7f87f803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:45.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:45.926+0000 7f88017fa640 1 -- 172.21.15.27:0/1889730500 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f87f4054250 con 0x7f88040c2720 2023-12-19T18:23:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.133+0000 7f880acec640 1 -- 172.21.15.27:0/1889730500 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f88040cad60 con 0x7f88040c2720 2023-12-19T18:23:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.134+0000 7f88017fa640 1 -- 172.21.15.27:0/1889730500 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v157) v1 ==== 100+0+11 (secure 0 0 0) 0x7f87f4057870 con 0x7f88040c2720 2023-12-19T18:23:46.134 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:23:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.136+0000 7f880acec640 1 -- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f87f006a430 msgr2=0x7f87f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.136+0000 7f880acec640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f87f006a430 0x7f87f006c8f0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f87f8036000 tx=0x7f87f803d040 comp rx=0 tx=0).stop 2023-12-19T18:23:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.136+0000 7f880acec640 1 -- 172.21.15.27:0/1889730500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88040c2720 msgr2=0x7f88041f27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.136+0000 7f880acec640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88040c2720 0x7f88041f27c0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f87f400e9b0 tx=0x7f87f400ee70 comp rx=0 tx=0).stop 2023-12-19T18:23:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.137+0000 7f880acec640 1 -- 172.21.15.27:0/1889730500 shutdown_connections 2023-12-19T18:23:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.137+0000 7f880acec640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f87f006a430 0x7f87f006c8f0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.137+0000 7f880acec640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f880415b1a0 0x7f88040aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.137+0000 7f880acec640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88040c2720 0x7f88041f27c0 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.137+0000 7f880acec640 1 --2- 172.21.15.27:0/1889730500 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8804071420 0x7f8804158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.137+0000 7f880acec640 1 -- 172.21.15.27:0/1889730500 >> 172.21.15.27:0/1889730500 conn(0x7f88040cebd0 msgr2=0x7f88040cddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:46.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.137+0000 7f880acec640 1 -- 172.21.15.27:0/1889730500 shutdown_connections 2023-12-19T18:23:46.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.137+0000 7f880acec640 1 -- 172.21.15.27:0/1889730500 wait complete. 2023-12-19T18:23:46.156 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:23:46.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.388+0000 7f6c98ca0640 1 Processor -- start 2023-12-19T18:23:46.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.388+0000 7f6c98ca0640 1 -- start start 2023-12-19T18:23:46.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.389+0000 7f6c98ca0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6c9415adf0 0x7f6c9406d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:46.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.390+0000 7f6c98ca0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6c9406fc10 0x7f6c940712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:46.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.390+0000 7f6c98ca0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c94071820 con 0x7f6c94070c00 2023-12-19T18:23:46.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.390+0000 7f6c98ca0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c940703c0 con 0x7f6c9415adf0 2023-12-19T18:23:46.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.390+0000 7f6c98ca0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c9406d8c0 con 0x7f6c9406fc10 2023-12-19T18:23:46.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.390+0000 7f6c9259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6c9415adf0 0x7f6c9406d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:46.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.390+0000 7f6c9259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6c9415adf0 0x7f6c9406d380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37446/0 (socket says 172.21.15.27:37446) 2023-12-19T18:23:46.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.390+0000 7f6c9259c640 1 -- 172.21.15.27:0/1119956655 learned_addr learned my addr 172.21.15.27:0/1119956655 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.390+0000 7f6c92d9d640 1 --2- 172.21.15.27:0/1119956655 >> v2:172.21.15.192:3300/0 conn(0x7f6c9406fc10 0x7f6c940712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.391+0000 7f6c9259c640 1 -- 172.21.15.27:0/1119956655 >> v2:172.21.15.192:3300/0 conn(0x7f6c9406fc10 msgr2=0x7f6c940712e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.391+0000 7f6c9259c640 1 --2- 172.21.15.27:0/1119956655 >> v2:172.21.15.192:3300/0 conn(0x7f6c9406fc10 0x7f6c940712e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.391+0000 7f6c9259c640 1 -- 172.21.15.27:0/1119956655 >> v1:172.21.15.192:6789/0 conn(0x7f6c94070c00 legacy=0x7f6c9406f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.391+0000 7f6c9259c640 1 -- 172.21.15.27:0/1119956655 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c9406b520 con 0x7f6c9415adf0 2023-12-19T18:23:46.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.392+0000 7f6c9259c640 1 --2- 172.21.15.27:0/1119956655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c9415adf0 0x7f6c9406d380 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f6c9406aff0 tx=0x7f6c8402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=209007641d6af39b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:46.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.392+0000 7f6c9159a640 1 -- 172.21.15.27:0/1119956655 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84031870 con 0x7f6c9415adf0 2023-12-19T18:23:46.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.392+0000 7f6c9159a640 1 -- 172.21.15.27:0/1119956655 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c840319d0 con 0x7f6c9415adf0 2023-12-19T18:23:46.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.392+0000 7f6c9159a640 1 -- 172.21.15.27:0/1119956655 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c84031d60 con 0x7f6c9415adf0 2023-12-19T18:23:46.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.393+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1119956655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c9415adf0 msgr2=0x7f6c9406d380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:46.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.393+0000 7f6c98ca0640 1 --2- 172.21.15.27:0/1119956655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c9415adf0 0x7f6c9406d380 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7f6c9406aff0 tx=0x7f6c8402b910 comp rx=0 tx=0).stop 2023-12-19T18:23:46.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.393+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1119956655 shutdown_connections 2023-12-19T18:23:46.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.393+0000 7f6c98ca0640 1 --2- 172.21.15.27:0/1119956655 >> v2:172.21.15.192:3300/0 conn(0x7f6c9406fc10 0x7f6c940712e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.393+0000 7f6c98ca0640 1 --2- 172.21.15.27:0/1119956655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c9415adf0 0x7f6c9406d380 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.393+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1119956655 >> 172.21.15.27:0/1119956655 conn(0x7f6c940ceb50 msgr2=0x7f6c940cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:46.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.393+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1119956655 shutdown_connections 2023-12-19T18:23:46.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.393+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1119956655 wait complete. 2023-12-19T18:23:46.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.395+0000 7f6c98ca0640 1 Processor -- start 2023-12-19T18:23:46.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.395+0000 7f6c98ca0640 1 -- start start 2023-12-19T18:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.396+0000 7f6c98ca0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c9406fc10 0x7f6c940c6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.396+0000 7f6c98ca0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c94070c00 0x7f6c940b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:46.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.396+0000 7f6c98ca0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c9415adf0 0x7f6c941f5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.396+0000 7f6c98ca0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940c7450 con 0x7f6c94070c00 2023-12-19T18:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.396+0000 7f6c98ca0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940b2870 con 0x7f6c9415adf0 2023-12-19T18:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.396+0000 7f6c98ca0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940c7640 con 0x7f6c9406fc10 2023-12-19T18:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.396+0000 7f6c91d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c94070c00 0x7f6c940b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.397+0000 7f6c91d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c94070c00 0x7f6c940b2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37458/0 (socket says 172.21.15.27:37458) 2023-12-19T18:23:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.397+0000 7f6c91d9b640 1 -- 172.21.15.27:0/1314506399 learned_addr learned my addr 172.21.15.27:0/1314506399 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.397+0000 7f6c92d9d640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c9415adf0 0x7f6c941f5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.397+0000 7f6c9259c640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c9406fc10 0x7f6c940c6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.397+0000 7f6c91d9b640 1 -- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c9406fc10 msgr2=0x7f6c940c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.397+0000 7f6c91d9b640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c9406fc10 0x7f6c940c6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.397+0000 7f6c91d9b640 1 -- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c9415adf0 msgr2=0x7f6c941f5cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.397+0000 7f6c91d9b640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c9415adf0 0x7f6c941f5cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.397+0000 7f6c91d9b640 1 -- 172.21.15.27:0/1314506399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c840094d0 con 0x7f6c94070c00 2023-12-19T18:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.397+0000 7f6c92d9d640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c9415adf0 0x7f6c941f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.397+0000 7f6c9259c640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c9406fc10 0x7f6c940c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.397+0000 7f6c91d9b640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c94070c00 0x7f6c940b2330 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00c430 tx=0x7f6c7c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.398+0000 7f6c8affd640 1 -- 172.21.15.27:0/1314506399 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c0052d0 con 0x7f6c94070c00 2023-12-19T18:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.398+0000 7f6c8affd640 1 -- 172.21.15.27:0/1314506399 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c7c005450 con 0x7f6c94070c00 2023-12-19T18:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.398+0000 7f6c8affd640 1 -- 172.21.15.27:0/1314506399 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c007260 con 0x7f6c94070c00 2023-12-19T18:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.398+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1314506399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c940c7920 con 0x7f6c94070c00 2023-12-19T18:23:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.398+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1314506399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c940c7ee0 con 0x7f6c94070c00 2023-12-19T18:23:46.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.399+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1314506399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c94068e90 con 0x7f6c94070c00 2023-12-19T18:23:46.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.400+0000 7f6c8affd640 1 -- 172.21.15.27:0/1314506399 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6c7c0073c0 con 0x7f6c94070c00 2023-12-19T18:23:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.401+0000 7f6c8affd640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c7406a430 0x7f6c7406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.401+0000 7f6c8affd640 1 -- 172.21.15.27:0/1314506399 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6c7c087df0 con 0x7f6c94070c00 2023-12-19T18:23:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.402+0000 7f6c9259c640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c7406a430 0x7f6c7406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:46.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.403+0000 7f6c9259c640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c7406a430 0x7f6c7406c8f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6c84036000 tx=0x7f6c8403c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:46.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.405+0000 7f6c8affd640 1 -- 172.21.15.27:0/1314506399 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6c7c053300 con 0x7f6c94070c00 2023-12-19T18:23:46.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.621+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1314506399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f6c940b1e30 con 0x7f6c94070c00 2023-12-19T18:23:46.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.622+0000 7f6c8affd640 1 -- 172.21.15.27:0/1314506399 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v157) v1 ==== 80+0+0 (secure 0 0 0) 0x7f6c7c056920 con 0x7f6c94070c00 2023-12-19T18:23:46.622 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:23:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.624+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c7406a430 msgr2=0x7f6c7406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.624+0000 7f6c98ca0640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c7406a430 0x7f6c7406c8f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f6c84036000 tx=0x7f6c8403c040 comp rx=0 tx=0).stop 2023-12-19T18:23:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.624+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1314506399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c94070c00 msgr2=0x7f6c940b2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.624+0000 7f6c98ca0640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c94070c00 0x7f6c940b2330 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00c430 tx=0x7f6c7c00c8f0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.625+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1314506399 shutdown_connections 2023-12-19T18:23:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.625+0000 7f6c98ca0640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c7406a430 0x7f6c7406c8f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.625+0000 7f6c98ca0640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c9415adf0 0x7f6c941f5cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.625+0000 7f6c98ca0640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c94070c00 0x7f6c940b2330 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.625+0000 7f6c98ca0640 1 --2- 172.21.15.27:0/1314506399 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c9406fc10 0x7f6c940c6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.625+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1314506399 >> 172.21.15.27:0/1314506399 conn(0x7f6c940ceb50 msgr2=0x7f6c9415e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.625+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1314506399 shutdown_connections 2023-12-19T18:23:46.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:46.625+0000 7f6c98ca0640 1 -- 172.21.15.27:0/1314506399 wait complete. 2023-12-19T18:23:46.645 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-12-19T18:23:46.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:46.645 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-19T18:23:46.745 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[113201]: fuse finished with error 0 and tester_r 0 2023-12-19T18:23:46.766 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:23:46.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:46.766 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:23:46.968 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:23:46.968 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:23:47.027 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:23:47.114 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:23:47.123 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:23:47.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.259+0000 7f6259c9e640 1 Processor -- start 2023-12-19T18:23:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.260+0000 7f6259c9e640 1 -- start start 2023-12-19T18:23:47.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.261+0000 7f6259c9e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f625415ceb0 0x7f62540b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:47.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.261+0000 7f6259c9e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f62540d2130 0x7f62540b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:47.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.261+0000 7f6259c9e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62540b5170 con 0x7f62540b4d10 2023-12-19T18:23:47.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.261+0000 7f6259c9e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62540b6540 con 0x7f62540d2130 2023-12-19T18:23:47.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.261+0000 7f6259c9e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62540cdfb0 con 0x7f625415ceb0 2023-12-19T18:23:47.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.262+0000 7f625949d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f62540d2130 0x7f62540b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:47.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.262+0000 7f6253fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f62540b4d10 0x7f62540d1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59158/0 (socket says 172.21.15.27:59158) 2023-12-19T18:23:47.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.262+0000 7f6253fff640 1 -- 172.21.15.27:0/666583472 learned_addr learned my addr 172.21.15.27:0/666583472 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:47.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.262+0000 7f6258c9c640 1 --2- 172.21.15.27:0/666583472 >> v2:172.21.15.192:3300/0 conn(0x7f625415ceb0 0x7f62540b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:47.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.262+0000 7f625949d640 1 -- 172.21.15.27:0/666583472 >> v2:172.21.15.192:3300/0 conn(0x7f625415ceb0 msgr2=0x7f62540b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:47.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.262+0000 7f625949d640 1 --2- 172.21.15.27:0/666583472 >> v2:172.21.15.192:3300/0 conn(0x7f625415ceb0 0x7f62540b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.262+0000 7f625949d640 1 -- 172.21.15.27:0/666583472 >> v1:172.21.15.27:6789/0 conn(0x7f62540b4d10 legacy=0x7f62540d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:47.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.262+0000 7f625949d640 1 -- 172.21.15.27:0/666583472 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62540d2600 con 0x7f62540d2130 2023-12-19T18:23:47.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.263+0000 7f625949d640 1 --2- 172.21.15.27:0/666583472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62540d2130 0x7f62540b69d0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f62540668b0 tx=0x7f624c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71c5d0731d127cf server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:47.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.264+0000 7f62537fe640 1 -- 172.21.15.27:0/666583472 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c02cc70 con 0x7f62540d2130 2023-12-19T18:23:47.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.264+0000 7f62537fe640 1 -- 172.21.15.27:0/666583472 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c02cdd0 con 0x7f62540d2130 2023-12-19T18:23:47.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.264+0000 7f62537fe640 1 -- 172.21.15.27:0/666583472 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c034660 con 0x7f62540d2130 2023-12-19T18:23:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.264+0000 7f6259c9e640 1 -- 172.21.15.27:0/666583472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62540d2130 msgr2=0x7f62540b69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.264+0000 7f6259c9e640 1 --2- 172.21.15.27:0/666583472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62540d2130 0x7f62540b69d0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7f62540668b0 tx=0x7f624c02bec0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.265+0000 7f6259c9e640 1 -- 172.21.15.27:0/666583472 shutdown_connections 2023-12-19T18:23:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.265+0000 7f6259c9e640 1 --2- 172.21.15.27:0/666583472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62540d2130 0x7f62540b69d0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.265+0000 7f6259c9e640 1 --2- 172.21.15.27:0/666583472 >> v2:172.21.15.192:3300/0 conn(0x7f625415ceb0 0x7f62540b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.265+0000 7f6259c9e640 1 -- 172.21.15.27:0/666583472 >> 172.21.15.27:0/666583472 conn(0x7f62540cf7d0 msgr2=0x7f62540cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.265+0000 7f6259c9e640 1 -- 172.21.15.27:0/666583472 shutdown_connections 2023-12-19T18:23:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.265+0000 7f6259c9e640 1 -- 172.21.15.27:0/666583472 wait complete. 2023-12-19T18:23:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.267+0000 7f6259c9e640 1 Processor -- start 2023-12-19T18:23:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.267+0000 7f6259c9e640 1 -- start start 2023-12-19T18:23:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6259c9e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62540b4d10 0x7f62541575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6259c9e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62540d2130 0x7f625415a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6259c9e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f625415ceb0 0x7f62540aa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6259c9e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62541f3430 con 0x7f62540b4d10 2023-12-19T18:23:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6259c9e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6254157b00 con 0x7f625415ceb0 2023-12-19T18:23:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6259c9e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f625415ac10 con 0x7f62540d2130 2023-12-19T18:23:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6258c9c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62540b4d10 0x7f62541575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6258c9c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62540b4d10 0x7f62541575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37480/0 (socket says 172.21.15.27:37480) 2023-12-19T18:23:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6258c9c640 1 -- 172.21.15.27:0/756810392 learned_addr learned my addr 172.21.15.27:0/756810392 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6258c9c640 1 -- 172.21.15.27:0/756810392 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62540d2130 msgr2=0x7f625415a6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6258c9c640 1 --2- 172.21.15.27:0/756810392 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62540d2130 0x7f625415a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6258c9c640 1 -- 172.21.15.27:0/756810392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f625415ceb0 msgr2=0x7f62540aa080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6258c9c640 1 --2- 172.21.15.27:0/756810392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f625415ceb0 0x7f62540aa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.268+0000 7f6258c9c640 1 -- 172.21.15.27:0/756810392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f624c009a70 con 0x7f62540b4d10 2023-12-19T18:23:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.269+0000 7f6258c9c640 1 --2- 172.21.15.27:0/756810392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62540b4d10 0x7f62541575c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f624800d5f0 tx=0x7f624800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.269+0000 7f62517fa640 1 -- 172.21.15.27:0/756810392 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62480125f0 con 0x7f62540b4d10 2023-12-19T18:23:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.269+0000 7f62517fa640 1 -- 172.21.15.27:0/756810392 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6248012750 con 0x7f62540b4d10 2023-12-19T18:23:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.269+0000 7f6259c9e640 1 -- 172.21.15.27:0/756810392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62541f2820 con 0x7f62540b4d10 2023-12-19T18:23:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.269+0000 7f62517fa640 1 -- 172.21.15.27:0/756810392 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248012b00 con 0x7f62540b4d10 2023-12-19T18:23:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.269+0000 7f6259c9e640 1 -- 172.21.15.27:0/756810392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62541f2de0 con 0x7f62540b4d10 2023-12-19T18:23:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.271+0000 7f6259c9e640 1 -- 172.21.15.27:0/756810392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f625404e840 con 0x7f62540b4d10 2023-12-19T18:23:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.271+0000 7f62517fa640 1 -- 172.21.15.27:0/756810392 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6248012c60 con 0x7f62540b4d10 2023-12-19T18:23:47.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.273+0000 7f62517fa640 1 --2- 172.21.15.27:0/756810392 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f623806a430 0x7f623806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:47.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.273+0000 7f62517fa640 1 -- 172.21.15.27:0/756810392 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f6248088d20 con 0x7f62540b4d10 2023-12-19T18:23:47.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.273+0000 7f62517fa640 1 -- 172.21.15.27:0/756810392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7f6238072020 con 0x7f62540b4d10 2023-12-19T18:23:47.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.273+0000 7f6253fff640 1 --2- 172.21.15.27:0/756810392 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f623806a430 0x7f623806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:47.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.275+0000 7f6253fff640 1 --2- 172.21.15.27:0/756810392 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f623806a430 0x7f623806c8f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f62540b4ae0 tx=0x7f6240007a40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:47.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.278+0000 7f62517fa640 1 -- 172.21.15.27:0/756810392 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6248054360 con 0x7f62540b4d10 2023-12-19T18:23:47.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.476+0000 7f6259c9e640 1 -- 172.21.15.27:0/756810392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f625415ef80 con 0x7f62540b4d10 2023-12-19T18:23:47.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.479+0000 7f62517fa640 1 -- 172.21.15.27:0/756810392 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16239 (secure 0 0 0) 0x7f6248057980 con 0x7f62540b4d10 2023-12-19T18:23:47.480 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:47.480 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":158,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:23:46.716988+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:23:04.431250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T18:23:05.506926+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":4,"to":3}]},{"pgid":"17.f","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:23:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.482+0000 7f6259c9e640 1 -- 172.21.15.27:0/756810392 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f623806a430 msgr2=0x7f623806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.482+0000 7f6259c9e640 1 --2- 172.21.15.27:0/756810392 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f623806a430 0x7f623806c8f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f62540b4ae0 tx=0x7f6240007a40 comp rx=0 tx=0).stop 2023-12-19T18:23:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.483+0000 7f6259c9e640 1 -- 172.21.15.27:0/756810392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62540b4d10 msgr2=0x7f62541575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.483+0000 7f6259c9e640 1 --2- 172.21.15.27:0/756810392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62540b4d10 0x7f62541575c0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f624800d5f0 tx=0x7f624800dab0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.483+0000 7f6259c9e640 1 -- 172.21.15.27:0/756810392 shutdown_connections 2023-12-19T18:23:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.483+0000 7f6259c9e640 1 --2- 172.21.15.27:0/756810392 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f623806a430 0x7f623806c8f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.483+0000 7f6259c9e640 1 --2- 172.21.15.27:0/756810392 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f625415ceb0 0x7f62540aa080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.483+0000 7f6259c9e640 1 --2- 172.21.15.27:0/756810392 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62540d2130 0x7f625415a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.483+0000 7f6259c9e640 1 --2- 172.21.15.27:0/756810392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62540b4d10 0x7f62541575c0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.483+0000 7f6259c9e640 1 -- 172.21.15.27:0/756810392 >> 172.21.15.27:0/756810392 conn(0x7f62540cf7d0 msgr2=0x7f625415e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:47.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.483+0000 7f6259c9e640 1 -- 172.21.15.27:0/756810392 shutdown_connections 2023-12-19T18:23:47.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.483+0000 7f6259c9e640 1 -- 172.21.15.27:0/756810392 wait complete. 2023-12-19T18:23:47.504 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:23:04.431250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:23:05.506926+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:23:47.505 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:23:47.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.788+0000 7fcbb8577640 1 Processor -- start 2023-12-19T18:23:47.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.788+0000 7fcbb8577640 1 -- start start 2023-12-19T18:23:47.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.790+0000 7fcbb8577640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcbb00ad9e0 0x7fcbb01683a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:47.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.790+0000 7fcbb8577640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb00ce0b0 con 0x7fcbb00c9750 2023-12-19T18:23:47.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.790+0000 7fcbb8577640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbb00ce920 con 0x7fcbb00ccfa0 2023-12-19T18:23:47.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.790+0000 7fcbb8577640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbb00cd4a0 con 0x7fcbb00ad9e0 2023-12-19T18:23:47.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.791+0000 7fcbb6313640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcbb00c9750 0x7fcbb015e4a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59168/0 (socket says 172.21.15.27:59168) 2023-12-19T18:23:47.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.791+0000 7fcbb6313640 1 -- 172.21.15.27:0/930082492 learned_addr learned my addr 172.21.15.27:0/930082492 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:47.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.791+0000 7fcbb6b14640 1 --2- 172.21.15.27:0/930082492 >> v2:172.21.15.192:3300/0 conn(0x7fcbb00ad9e0 0x7fcbb01683a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:47.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.791+0000 7fcbb5311640 1 -- 172.21.15.27:0/930082492 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 700504822 0 0) 0x7fcbb00ce0b0 con 0x7fcbb00c9750 2023-12-19T18:23:47.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.791+0000 7fcbb5311640 1 -- 172.21.15.27:0/930082492 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb9c003520 con 0x7fcbb00c9750 2023-12-19T18:23:47.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.792+0000 7fcbb6b14640 1 -- 172.21.15.27:0/930082492 >> v1:172.21.15.192:6789/0 conn(0x7fcbb00ccfa0 legacy=0x7fcbb00c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:47.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.792+0000 7fcbb6b14640 1 -- 172.21.15.27:0/930082492 >> v1:172.21.15.27:6789/0 conn(0x7fcbb00c9750 legacy=0x7fcbb015e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:47.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.792+0000 7fcbb5311640 1 -- 172.21.15.27:0/930082492 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 368143722 0 0) 0x7fcb9c003520 con 0x7fcbb00c9750 2023-12-19T18:23:47.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.792+0000 7fcbb6b14640 1 -- 172.21.15.27:0/930082492 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb00cd780 con 0x7fcbb00ad9e0 2023-12-19T18:23:47.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.793+0000 7fcbb6b14640 1 --2- 172.21.15.27:0/930082492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb00ad9e0 0x7fcbb01683a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009d90 tx=0x7fcba802bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b93fd17834a6018b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:47.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.793+0000 7fcbb5311640 1 -- 172.21.15.27:0/930082492 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba802ccf0 con 0x7fcbb00ad9e0 2023-12-19T18:23:47.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.794+0000 7fcbb5311640 1 -- 172.21.15.27:0/930082492 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba802ce50 con 0x7fcbb00ad9e0 2023-12-19T18:23:47.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.794+0000 7fcbb5311640 1 -- 172.21.15.27:0/930082492 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba8034640 con 0x7fcbb00ad9e0 2023-12-19T18:23:47.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.794+0000 7fcbb8577640 1 -- 172.21.15.27:0/930082492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb00ad9e0 msgr2=0x7fcbb01683a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:47.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.794+0000 7fcbb8577640 1 --2- 172.21.15.27:0/930082492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb00ad9e0 0x7fcbb01683a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fcba8009d90 tx=0x7fcba802bf40 comp rx=0 tx=0).stop 2023-12-19T18:23:47.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.794+0000 7fcbb8577640 1 -- 172.21.15.27:0/930082492 shutdown_connections 2023-12-19T18:23:47.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.794+0000 7fcbb8577640 1 --2- 172.21.15.27:0/930082492 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb00ad9e0 0x7fcbb01683a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.794+0000 7fcbb8577640 1 -- 172.21.15.27:0/930082492 >> 172.21.15.27:0/930082492 conn(0x7fcbb00cebd0 msgr2=0x7fcbb00cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:47.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.794+0000 7fcbb8577640 1 -- 172.21.15.27:0/930082492 shutdown_connections 2023-12-19T18:23:47.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.794+0000 7fcbb8577640 1 -- 172.21.15.27:0/930082492 wait complete. 2023-12-19T18:23:47.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.796+0000 7fcbb8577640 1 Processor -- start 2023-12-19T18:23:47.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.796+0000 7fcbb8577640 1 -- start start 2023-12-19T18:23:47.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.797+0000 7fcbb8577640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb00ad9e0 0x7fcbb00c8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:47.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.797+0000 7fcbb8577640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb00c9750 0x7fcbb00c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:47.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.797+0000 7fcbb8577640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcbb00ccfa0 0x7fcbb015c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:47.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.797+0000 7fcbb8577640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb00c63b0 con 0x7fcbb00c9750 2023-12-19T18:23:47.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.797+0000 7fcbb8577640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb00c6690 con 0x7fcbb00ad9e0 2023-12-19T18:23:47.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.797+0000 7fcbb8577640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbb00c6970 con 0x7fcbb00ccfa0 2023-12-19T18:23:47.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.798+0000 7fcbb5b12640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb00c9750 0x7fcbb00c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.798+0000 7fcbb5b12640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb00c9750 0x7fcbb00c8870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37484/0 (socket says 172.21.15.27:37484) 2023-12-19T18:23:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.798+0000 7fcbb5b12640 1 -- 172.21.15.27:0/1029860560 learned_addr learned my addr 172.21.15.27:0/1029860560 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.798+0000 7fcbb6b14640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcbb00ccfa0 0x7fcbb015c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.798+0000 7fcbb6313640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb00ad9e0 0x7fcbb00c8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.798+0000 7fcbb5b12640 1 -- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcbb00ccfa0 msgr2=0x7fcbb015c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.798+0000 7fcbb5b12640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcbb00ccfa0 0x7fcbb015c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.798+0000 7fcbb5b12640 1 -- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb00ad9e0 msgr2=0x7fcbb00c8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.798+0000 7fcbb5b12640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb00ad9e0 0x7fcbb00c8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.798+0000 7fcbb5b12640 1 -- 172.21.15.27:0/1029860560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba8009a70 con 0x7fcbb00c9750 2023-12-19T18:23:47.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.798+0000 7fcbb6b14640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcbb00ccfa0 0x7fcbb015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:47.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.799+0000 7fcbb5b12640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb00c9750 0x7fcbb00c8870 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c3c0 tx=0x7fcba000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:47.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.799+0000 7fcba6ffd640 1 -- 172.21.15.27:0/1029860560 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba00052d0 con 0x7fcbb00c9750 2023-12-19T18:23:47.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.799+0000 7fcba6ffd640 1 -- 172.21.15.27:0/1029860560 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcba0005450 con 0x7fcbb00c9750 2023-12-19T18:23:47.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.799+0000 7fcbb8577640 1 -- 172.21.15.27:0/1029860560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbb00c6c50 con 0x7fcbb00c9750 2023-12-19T18:23:47.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.799+0000 7fcba6ffd640 1 -- 172.21.15.27:0/1029860560 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcba0003830 con 0x7fcbb00c9750 2023-12-19T18:23:47.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.799+0000 7fcbb8577640 1 -- 172.21.15.27:0/1029860560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbb00c7ab0 con 0x7fcbb00c9750 2023-12-19T18:23:47.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.801+0000 7fcbb8577640 1 -- 172.21.15.27:0/1029860560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbb00c7170 con 0x7fcbb00c9750 2023-12-19T18:23:47.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.801+0000 7fcba6ffd640 1 -- 172.21.15.27:0/1029860560 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcba0003990 con 0x7fcbb00c9750 2023-12-19T18:23:47.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.802+0000 7fcba6ffd640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb9406a430 0x7fcb9406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:47.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.803+0000 7fcba6ffd640 1 -- 172.21.15.27:0/1029860560 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcba0087f10 con 0x7fcbb00c9750 2023-12-19T18:23:47.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.803+0000 7fcba6ffd640 1 -- 172.21.15.27:0/1029860560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7fcb94072020 con 0x7fcbb00c9750 2023-12-19T18:23:47.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.803+0000 7fcbb6313640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb9406a430 0x7fcb9406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:47.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.804+0000 7fcbb6313640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb9406a430 0x7fcb9406c8f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcbac005880 tx=0x7fcbac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:47.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:47.807+0000 7fcba6ffd640 1 -- 172.21.15.27:0/1029860560 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcbb00c7170 con 0x7fcbb00c9750 2023-12-19T18:23:48.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.001+0000 7fcbb8577640 1 -- 172.21.15.27:0/1029860560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcbb00c7170 con 0x7fcbb00c9750 2023-12-19T18:23:48.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.002+0000 7fcba6ffd640 1 -- 172.21.15.27:0/1029860560 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcbb00c7170 con 0x7fcbb00c9750 2023-12-19T18:23:48.002 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:23:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.005+0000 7fcbb8577640 1 -- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb9406a430 msgr2=0x7fcb9406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.005+0000 7fcbb8577640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb9406a430 0x7fcb9406c8f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fcbac005880 tx=0x7fcbac009040 comp rx=0 tx=0).stop 2023-12-19T18:23:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.005+0000 7fcbb8577640 1 -- 172.21.15.27:0/1029860560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb00c9750 msgr2=0x7fcbb00c8870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.005+0000 7fcbb8577640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb00c9750 0x7fcbb00c8870 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fcba000c3c0 tx=0x7fcba000c880 comp rx=0 tx=0).stop 2023-12-19T18:23:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.005+0000 7fcbb8577640 1 -- 172.21.15.27:0/1029860560 shutdown_connections 2023-12-19T18:23:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.005+0000 7fcbb8577640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb9406a430 0x7fcb9406c8f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.005+0000 7fcbb8577640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcbb00ccfa0 0x7fcbb015c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.005+0000 7fcbb8577640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbb00c9750 0x7fcbb00c8870 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.005+0000 7fcbb8577640 1 --2- 172.21.15.27:0/1029860560 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcbb00ad9e0 0x7fcbb00c8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.005+0000 7fcbb8577640 1 -- 172.21.15.27:0/1029860560 >> 172.21.15.27:0/1029860560 conn(0x7fcbb00cebd0 msgr2=0x7fcbb00cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.005+0000 7fcbb8577640 1 -- 172.21.15.27:0/1029860560 shutdown_connections 2023-12-19T18:23:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.006+0000 7fcbb8577640 1 -- 172.21.15.27:0/1029860560 wait complete. 2023-12-19T18:23:48.024 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:23:48.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.255+0000 7f43dc901640 1 Processor -- start 2023-12-19T18:23:48.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.255+0000 7f43dc901640 1 -- start start 2023-12-19T18:23:48.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.257+0000 7f43dc901640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f43d815cf80 0x7f43d80d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:48.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.257+0000 7f43dc901640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43d80b8a60 0x7f43d80b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:48.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.257+0000 7f43dc901640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43d80d0e10 con 0x7f43d80b85e0 2023-12-19T18:23:48.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.257+0000 7f43dc901640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43d80d1c70 con 0x7f43d815cf80 2023-12-19T18:23:48.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.257+0000 7f43dc901640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d815d360 con 0x7f43d80b8a60 2023-12-19T18:23:48.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.257+0000 7f43d77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f43d815cf80 0x7f43d80d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:48.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.257+0000 7f43d77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f43d815cf80 0x7f43d80d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37496/0 (socket says 172.21.15.27:37496) 2023-12-19T18:23:48.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.257+0000 7f43d77fe640 1 -- 172.21.15.27:0/984496901 learned_addr learned my addr 172.21.15.27:0/984496901 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:48.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.258+0000 7f43d77fe640 1 -- 172.21.15.27:0/984496901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43d80b8a60 msgr2=0x7f43d80b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:48.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.258+0000 7f43d7fff640 1 --2- 172.21.15.27:0/984496901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43d80b8a60 0x7f43d80b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:48.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.258+0000 7f43d77fe640 1 --2- 172.21.15.27:0/984496901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43d80b8a60 0x7f43d80b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.258+0000 7f43d77fe640 1 -- 172.21.15.27:0/984496901 >> v1:172.21.15.192:6789/0 conn(0x7f43d80b85e0 legacy=0x7f43d80cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:48.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.258+0000 7f43d77fe640 1 -- 172.21.15.27:0/984496901 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43d80d1060 con 0x7f43d815cf80 2023-12-19T18:23:48.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.259+0000 7f43d77fe640 1 --2- 172.21.15.27:0/984496901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d815cf80 0x7f43d80d2100 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009870 tx=0x7f43c802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb64ccfaa031278 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:48.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.259+0000 7f43d67fc640 1 -- 172.21.15.27:0/984496901 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8030840 con 0x7f43d815cf80 2023-12-19T18:23:48.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.259+0000 7f43d67fc640 1 -- 172.21.15.27:0/984496901 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c80309a0 con 0x7f43d815cf80 2023-12-19T18:23:48.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.259+0000 7f43d67fc640 1 -- 172.21.15.27:0/984496901 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c8030d30 con 0x7f43d815cf80 2023-12-19T18:23:48.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.260+0000 7f43dc901640 1 -- 172.21.15.27:0/984496901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d815cf80 msgr2=0x7f43d80d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:48.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.260+0000 7f43dc901640 1 --2- 172.21.15.27:0/984496901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d815cf80 0x7f43d80d2100 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f43c8009870 tx=0x7f43c802b980 comp rx=0 tx=0).stop 2023-12-19T18:23:48.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.260+0000 7f43dc901640 1 -- 172.21.15.27:0/984496901 shutdown_connections 2023-12-19T18:23:48.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.260+0000 7f43dc901640 1 --2- 172.21.15.27:0/984496901 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43d80b8a60 0x7f43d80b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.260+0000 7f43dc901640 1 --2- 172.21.15.27:0/984496901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d815cf80 0x7f43d80d2100 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.260+0000 7f43dc901640 1 -- 172.21.15.27:0/984496901 >> 172.21.15.27:0/984496901 conn(0x7f43d8072e20 msgr2=0x7f43d8073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:48.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.260+0000 7f43dc901640 1 -- 172.21.15.27:0/984496901 shutdown_connections 2023-12-19T18:23:48.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.260+0000 7f43dc901640 1 -- 172.21.15.27:0/984496901 wait complete. 2023-12-19T18:23:48.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.262+0000 7f43dc901640 1 Processor -- start 2023-12-19T18:23:48.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.262+0000 7f43dc901640 1 -- start start 2023-12-19T18:23:48.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.263+0000 7f43dc901640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43d80b85e0 0x7f43d81428b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:48.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.263+0000 7f43dc901640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d815cf80 0x7f43d815ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:48.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.263+0000 7f43dc901640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43d8153090 0x7f43d8155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:48.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.263+0000 7f43dc901640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d80caee0 con 0x7f43d815cf80 2023-12-19T18:23:48.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.263+0000 7f43dc901640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43d80ca610 con 0x7f43d8153090 2023-12-19T18:23:48.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.263+0000 7f43dc901640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43d8155c70 con 0x7f43d80b85e0 2023-12-19T18:23:48.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.264+0000 7f43d6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d815cf80 0x7f43d815ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:48.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.264+0000 7f43d6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d815cf80 0x7f43d815ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37500/0 (socket says 172.21.15.27:37500) 2023-12-19T18:23:48.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.264+0000 7f43d6ffd640 1 -- 172.21.15.27:0/3224752111 learned_addr learned my addr 172.21.15.27:0/3224752111 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:48.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.264+0000 7f43d7fff640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43d8153090 0x7f43d8155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:48.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.264+0000 7f43d77fe640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43d80b85e0 0x7f43d81428b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.264+0000 7f43d6ffd640 1 -- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43d80b85e0 msgr2=0x7f43d81428b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.264+0000 7f43d6ffd640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43d80b85e0 0x7f43d81428b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.264+0000 7f43d6ffd640 1 -- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43d8153090 msgr2=0x7f43d8155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.264+0000 7f43d6ffd640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43d8153090 0x7f43d8155300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.264+0000 7f43d6ffd640 1 -- 172.21.15.27:0/3224752111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43c80094d0 con 0x7f43d815cf80 2023-12-19T18:23:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.265+0000 7f43d6ffd640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d815cf80 0x7f43d815ffd0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c7c0 tx=0x7f43c000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.265+0000 7f43cffff640 1 -- 172.21.15.27:0/3224752111 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c000aa50 con 0x7f43d815cf80 2023-12-19T18:23:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.265+0000 7f43cffff640 1 -- 172.21.15.27:0/3224752111 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43c000abb0 con 0x7f43d815cf80 2023-12-19T18:23:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.265+0000 7f43cffff640 1 -- 172.21.15.27:0/3224752111 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43c0012420 con 0x7f43d815cf80 2023-12-19T18:23:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.265+0000 7f43dc901640 1 -- 172.21.15.27:0/3224752111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43d80ca890 con 0x7f43d815cf80 2023-12-19T18:23:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.265+0000 7f43dc901640 1 -- 172.21.15.27:0/3224752111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43d8160510 con 0x7f43d815cf80 2023-12-19T18:23:48.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.266+0000 7f43dc901640 1 -- 172.21.15.27:0/3224752111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43d80cb090 con 0x7f43d815cf80 2023-12-19T18:23:48.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.267+0000 7f43cffff640 1 -- 172.21.15.27:0/3224752111 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f43c0012580 con 0x7f43d815cf80 2023-12-19T18:23:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.268+0000 7f43cffff640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f43b806a460 0x7f43b806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.268+0000 7f43cffff640 1 -- 172.21.15.27:0/3224752111 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f43c0088ba0 con 0x7f43d815cf80 2023-12-19T18:23:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.268+0000 7f43cffff640 1 -- 172.21.15.27:0/3224752111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7f43b8072050 con 0x7f43d815cf80 2023-12-19T18:23:48.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.269+0000 7f43d77fe640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f43b806a460 0x7f43b806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:48.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.270+0000 7f43d77fe640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f43b806a460 0x7f43b806c920 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f43c8036000 tx=0x7f43c803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:48.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.272+0000 7f43cffff640 1 -- 172.21.15.27:0/3224752111 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f43c0016060 con 0x7f43d815cf80 2023-12-19T18:23:48.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.471+0000 7f43dc901640 1 -- 172.21.15.27:0/3224752111 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f43d804e840 con 0x7f43d815cf80 2023-12-19T18:23:48.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.472+0000 7f43cffff640 1 -- 172.21.15.27:0/3224752111 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43c0016070 con 0x7f43d815cf80 2023-12-19T18:23:48.472 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:23:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.476+0000 7f43dc901640 1 -- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f43b806a460 msgr2=0x7f43b806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.476+0000 7f43dc901640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f43b806a460 0x7f43b806c920 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f43c8036000 tx=0x7f43c803b040 comp rx=0 tx=0).stop 2023-12-19T18:23:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.476+0000 7f43dc901640 1 -- 172.21.15.27:0/3224752111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d815cf80 msgr2=0x7f43d815ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.476+0000 7f43dc901640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d815cf80 0x7f43d815ffd0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f43c000c7c0 tx=0x7f43c000cc80 comp rx=0 tx=0).stop 2023-12-19T18:23:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.476+0000 7f43dc901640 1 -- 172.21.15.27:0/3224752111 shutdown_connections 2023-12-19T18:23:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.476+0000 7f43dc901640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f43b806a460 0x7f43b806c920 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.476+0000 7f43dc901640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f43d8153090 0x7f43d8155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.476+0000 7f43dc901640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43d815cf80 0x7f43d815ffd0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.476+0000 7f43dc901640 1 --2- 172.21.15.27:0/3224752111 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f43d80b85e0 0x7f43d81428b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.476+0000 7f43dc901640 1 -- 172.21.15.27:0/3224752111 >> 172.21.15.27:0/3224752111 conn(0x7f43d8072e20 msgr2=0x7f43d80b4ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.476+0000 7f43dc901640 1 -- 172.21.15.27:0/3224752111 shutdown_connections 2023-12-19T18:23:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.476+0000 7f43dc901640 1 -- 172.21.15.27:0/3224752111 wait complete. 2023-12-19T18:23:48.495 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:23:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.732+0000 7efcd6e8a640 1 Processor -- start 2023-12-19T18:23:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.732+0000 7efcd6e8a640 1 -- start start 2023-12-19T18:23:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.734+0000 7efcd6e8a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efcd015e9d0 0x7efcd00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.734+0000 7efcd6e8a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcd0152070 0x7efcd00cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.734+0000 7efcd6e8a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcd015b3e0 con 0x7efcd0151c90 2023-12-19T18:23:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.734+0000 7efcd6e8a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcd015eed0 con 0x7efcd015e9d0 2023-12-19T18:23:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.734+0000 7efcd6e8a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd015f1b0 con 0x7efcd0152070 2023-12-19T18:23:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.734+0000 7efcd5687640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efcd015e9d0 0x7efcd00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.734+0000 7efcd5687640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efcd015e9d0 0x7efcd00cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37508/0 (socket says 172.21.15.27:37508) 2023-12-19T18:23:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.734+0000 7efcd5687640 1 -- 172.21.15.27:0/2026894863 learned_addr learned my addr 172.21.15.27:0/2026894863 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.735+0000 7efcd6689640 1 --2- 172.21.15.27:0/2026894863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcd0152070 0x7efcd00cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.735+0000 7efcd5687640 1 -- 172.21.15.27:0/2026894863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcd0152070 msgr2=0x7efcd00cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.735+0000 7efcd5687640 1 --2- 172.21.15.27:0/2026894863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcd0152070 0x7efcd00cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.735+0000 7efcd5687640 1 -- 172.21.15.27:0/2026894863 >> v1:172.21.15.192:6789/0 conn(0x7efcd0151c90 legacy=0x7efcd006b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.735+0000 7efcd5687640 1 -- 172.21.15.27:0/2026894863 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcd015f490 con 0x7efcd015e9d0 2023-12-19T18:23:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.735+0000 7efcd6689640 1 --2- 172.21.15.27:0/2026894863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcd0152070 0x7efcd00cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.736+0000 7efcd5687640 1 --2- 172.21.15.27:0/2026894863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcd015e9d0 0x7efcd00cee40 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7efcc00098a0 tx=0x7efcc002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d696e6aa3fab5964 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.736+0000 7efcd4e86640 1 -- 172.21.15.27:0/2026894863 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc00317c0 con 0x7efcd015e9d0 2023-12-19T18:23:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.736+0000 7efcd4e86640 1 -- 172.21.15.27:0/2026894863 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc0031920 con 0x7efcd015e9d0 2023-12-19T18:23:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.736+0000 7efcd4e86640 1 -- 172.21.15.27:0/2026894863 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc0031cb0 con 0x7efcd015e9d0 2023-12-19T18:23:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.737+0000 7efcd6e8a640 1 -- 172.21.15.27:0/2026894863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcd015e9d0 msgr2=0x7efcd00cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.737+0000 7efcd6e8a640 1 --2- 172.21.15.27:0/2026894863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcd015e9d0 0x7efcd00cee40 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7efcc00098a0 tx=0x7efcc002b860 comp rx=0 tx=0).stop 2023-12-19T18:23:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.737+0000 7efcd6e8a640 1 -- 172.21.15.27:0/2026894863 shutdown_connections 2023-12-19T18:23:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.737+0000 7efcd6e8a640 1 --2- 172.21.15.27:0/2026894863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcd0152070 0x7efcd00cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.737+0000 7efcd6e8a640 1 --2- 172.21.15.27:0/2026894863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcd015e9d0 0x7efcd00cee40 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.737+0000 7efcd6e8a640 1 -- 172.21.15.27:0/2026894863 >> 172.21.15.27:0/2026894863 conn(0x7efcd006a060 msgr2=0x7efcd006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.737+0000 7efcd6e8a640 1 -- 172.21.15.27:0/2026894863 shutdown_connections 2023-12-19T18:23:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.737+0000 7efcd6e8a640 1 -- 172.21.15.27:0/2026894863 wait complete. 2023-12-19T18:23:48.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.739+0000 7efcd6e8a640 1 Processor -- start 2023-12-19T18:23:48.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.739+0000 7efcd6e8a640 1 -- start start 2023-12-19T18:23:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.740+0000 7efcd6e8a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcd0151c90 0x7efcd0140560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.740+0000 7efcd6e8a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcd0152070 0x7efcd0140eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.740+0000 7efcd6e8a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcd015e9d0 0x7efcd01413f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:48.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.740+0000 7efcd6e8a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd0146200 con 0x7efcd015e9d0 2023-12-19T18:23:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.740+0000 7efcd6e8a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd01426d0 con 0x7efcd0152070 2023-12-19T18:23:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.740+0000 7efcd6e8a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcd0141ac0 con 0x7efcd0151c90 2023-12-19T18:23:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.740+0000 7efcd6689640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcd015e9d0 0x7efcd01413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.741+0000 7efcd6689640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcd015e9d0 0x7efcd01413f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37524/0 (socket says 172.21.15.27:37524) 2023-12-19T18:23:48.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.741+0000 7efcd6689640 1 -- 172.21.15.27:0/1553351117 learned_addr learned my addr 172.21.15.27:0/1553351117 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.741+0000 7efcd5e88640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcd0151c90 0x7efcd0140560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.741+0000 7efcd5687640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcd0152070 0x7efcd0140eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.741+0000 7efcd6689640 1 -- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcd0151c90 msgr2=0x7efcd0140560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.741+0000 7efcd6689640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcd0151c90 0x7efcd0140560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.741+0000 7efcd6689640 1 -- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcd0152070 msgr2=0x7efcd0140eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.741+0000 7efcd6689640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcd0152070 0x7efcd0140eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.741+0000 7efcd6689640 1 -- 172.21.15.27:0/1553351117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc00094d0 con 0x7efcd015e9d0 2023-12-19T18:23:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.741+0000 7efcd5687640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcd0152070 0x7efcd0140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.741+0000 7efcd6689640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcd015e9d0 0x7efcd01413f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efcc800ce40 tx=0x7efcc800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.742+0000 7efcc67fc640 1 -- 172.21.15.27:0/1553351117 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc800d840 con 0x7efcd015e9d0 2023-12-19T18:23:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.742+0000 7efcc67fc640 1 -- 172.21.15.27:0/1553351117 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcc800d9a0 con 0x7efcd015e9d0 2023-12-19T18:23:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.742+0000 7efcc67fc640 1 -- 172.21.15.27:0/1553351117 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc800dd50 con 0x7efcd015e9d0 2023-12-19T18:23:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.742+0000 7efcd6e8a640 1 -- 172.21.15.27:0/1553351117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd0141da0 con 0x7efcd015e9d0 2023-12-19T18:23:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.742+0000 7efcd6e8a640 1 -- 172.21.15.27:0/1553351117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd0142360 con 0x7efcd015e9d0 2023-12-19T18:23:48.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.743+0000 7efcd6e8a640 1 -- 172.21.15.27:0/1553351117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd004e840 con 0x7efcd015e9d0 2023-12-19T18:23:48.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.744+0000 7efcc67fc640 1 -- 172.21.15.27:0/1553351117 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efcc801e3b0 con 0x7efcd015e9d0 2023-12-19T18:23:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.745+0000 7efcc67fc640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efcb406a430 0x7efcb406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.745+0000 7efcc67fc640 1 -- 172.21.15.27:0/1553351117 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efcc8089430 con 0x7efcd015e9d0 2023-12-19T18:23:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.745+0000 7efcc67fc640 1 -- 172.21.15.27:0/1553351117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7efcb4072020 con 0x7efcd015e9d0 2023-12-19T18:23:48.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.746+0000 7efcd5e88640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efcb406a430 0x7efcb406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:48.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.747+0000 7efcd5e88640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efcb406a430 0x7efcb406c8f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efccc004ea0 tx=0x7efccc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:48.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.749+0000 7efcc67fc640 1 -- 172.21.15.27:0/1553351117 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efcc801a060 con 0x7efcd015e9d0 2023-12-19T18:23:48.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.946+0000 7efcd6e8a640 1 -- 172.21.15.27:0/1553351117 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efcd015f740 con 0x7efcd015e9d0 2023-12-19T18:23:48.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.947+0000 7efcc67fc640 1 -- 172.21.15.27:0/1553351117 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) v1 ==== 100+0+11 (secure 0 0 0) 0x7efcc805e030 con 0x7efcd015e9d0 2023-12-19T18:23:48.947 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:23:48.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.949+0000 7efcd6e8a640 1 -- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efcb406a430 msgr2=0x7efcb406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:48.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.949+0000 7efcd6e8a640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efcb406a430 0x7efcb406c8f0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efccc004ea0 tx=0x7efccc009040 comp rx=0 tx=0).stop 2023-12-19T18:23:48.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.949+0000 7efcd6e8a640 1 -- 172.21.15.27:0/1553351117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcd015e9d0 msgr2=0x7efcd01413f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:48.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.949+0000 7efcd6e8a640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcd015e9d0 0x7efcd01413f0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7efcc800ce40 tx=0x7efcc800a710 comp rx=0 tx=0).stop 2023-12-19T18:23:48.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.950+0000 7efcd6e8a640 1 -- 172.21.15.27:0/1553351117 shutdown_connections 2023-12-19T18:23:48.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.950+0000 7efcd6e8a640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efcb406a430 0x7efcb406c8f0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.950+0000 7efcd6e8a640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcd015e9d0 0x7efcd01413f0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.950+0000 7efcd6e8a640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcd0152070 0x7efcd0140eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.950+0000 7efcd6e8a640 1 --2- 172.21.15.27:0/1553351117 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcd0151c90 0x7efcd0140560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:48.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.950+0000 7efcd6e8a640 1 -- 172.21.15.27:0/1553351117 >> 172.21.15.27:0/1553351117 conn(0x7efcd006a060 msgr2=0x7efcd006b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:48.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.950+0000 7efcd6e8a640 1 -- 172.21.15.27:0/1553351117 shutdown_connections 2023-12-19T18:23:48.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:48.950+0000 7efcd6e8a640 1 -- 172.21.15.27:0/1553351117 wait complete. 2023-12-19T18:23:48.970 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:23:49.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.204+0000 7f3b431ef640 1 Processor -- start 2023-12-19T18:23:49.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.204+0000 7f3b431ef640 1 -- start start 2023-12-19T18:23:49.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.206+0000 7f3b431ef640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b3c06ba20 0x7f3b3c165c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:49.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.206+0000 7f3b431ef640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3c0adf10 con 0x7f3b3c070fb0 2023-12-19T18:23:49.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.206+0000 7f3b431ef640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b3c0adac0 con 0x7f3b3c15ac50 2023-12-19T18:23:49.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.206+0000 7f3b431ef640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0af630 con 0x7f3b3c06ba20 2023-12-19T18:23:49.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.206+0000 7f3b419ec640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3b3c070fb0 0x7f3b3c06b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59170/0 (socket says 172.21.15.27:59170) 2023-12-19T18:23:49.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.206+0000 7f3b419ec640 1 -- 172.21.15.27:0/1874632786 learned_addr learned my addr 172.21.15.27:0/1874632786 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:49.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.206+0000 7f3b429ee640 1 --2- 172.21.15.27:0/1874632786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b3c06ba20 0x7f3b3c165c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:49.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438910112 0 0) 0x7f3b3c0adf10 con 0x7f3b3c070fb0 2023-12-19T18:23:49.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b28003520 con 0x7f3b3c070fb0 2023-12-19T18:23:49.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2423972482 0 0) 0x7f3b28003520 con 0x7f3b3c070fb0 2023-12-19T18:23:49.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b3c06ba20 msgr2=0x7f3b3c165c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:49.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 --2- 172.21.15.27:0/1874632786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b3c06ba20 0x7f3b3c165c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:49.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 >> v1:172.21.15.192:6789/0 conn(0x7f3b3c15ac50 legacy=0x7f3b3c0aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:49.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c0713f0 con 0x7f3b3c070fb0 2023-12-19T18:23:49.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f3b30001670 con 0x7f3b3c070fb0 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 >> v1:172.21.15.27:6789/0 conn(0x7f3b3c070fb0 legacy=0x7f3b3c06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 --2- 172.21.15.27:0/1874632786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b280039a0 0x7f3b28003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 --2- 172.21.15.27:0/1874632786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b3c06ba20 0x7f3b280047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 --2- 172.21.15.27:0/1874632786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b28004d00 0x7f3b28005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0af630 con 0x7f3b28004d00 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2800bb40 con 0x7f3b280039a0 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.207+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2800bd00 con 0x7f3b3c06ba20 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.208+0000 7f3b429ee640 1 --2- 172.21.15.27:0/1874632786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b28004d00 0x7f3b28005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.208+0000 7f3b421ed640 1 --2- 172.21.15.27:0/1874632786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b280039a0 0x7f3b28003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.208+0000 7f3b419ec640 1 --2- 172.21.15.27:0/1874632786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b3c06ba20 0x7f3b280047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.208+0000 7f3b429ee640 1 -- 172.21.15.27:0/1874632786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b3c06ba20 msgr2=0x7f3b280047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.208+0000 7f3b429ee640 1 --2- 172.21.15.27:0/1874632786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b3c06ba20 0x7f3b280047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.208+0000 7f3b429ee640 1 -- 172.21.15.27:0/1874632786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b280039a0 msgr2=0x7f3b28003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:49.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.208+0000 7f3b429ee640 1 --2- 172.21.15.27:0/1874632786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b280039a0 0x7f3b28003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:49.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.208+0000 7f3b429ee640 1 -- 172.21.15.27:0/1874632786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b28003520 con 0x7f3b28004d00 2023-12-19T18:23:49.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.208+0000 7f3b419ec640 1 --2- 172.21.15.27:0/1874632786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b3c06ba20 0x7f3b280047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:49.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.209+0000 7f3b429ee640 1 --2- 172.21.15.27:0/1874632786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b28004d00 0x7f3b28005180 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009fc0 tx=0x7f3b280057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2117657409ae54c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:49.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.209+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b3402cbf0 con 0x7f3b28004d00 2023-12-19T18:23:49.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.210+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b3400f040 con 0x7f3b28004d00 2023-12-19T18:23:49.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.210+0000 7f3b411eb640 1 -- 172.21.15.27:0/1874632786 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b34034440 con 0x7f3b28004d00 2023-12-19T18:23:49.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.210+0000 7f3b431ef640 1 -- 172.21.15.27:0/1874632786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b28004d00 msgr2=0x7f3b28005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:49.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.210+0000 7f3b431ef640 1 --2- 172.21.15.27:0/1874632786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b28004d00 0x7f3b28005180 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f3b34009fc0 tx=0x7f3b280057f0 comp rx=0 tx=0).stop 2023-12-19T18:23:49.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.210+0000 7f3b429ee640 1 -- 172.21.15.27:0/1874632786 reap_dead start 2023-12-19T18:23:49.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.210+0000 7f3b431ef640 1 -- 172.21.15.27:0/1874632786 shutdown_connections 2023-12-19T18:23:49.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.210+0000 7f3b431ef640 1 -- 172.21.15.27:0/1874632786 >> 172.21.15.27:0/1874632786 conn(0x7f3b3c06a7d0 msgr2=0x7f3b3c0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:49.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.210+0000 7f3b431ef640 1 -- 172.21.15.27:0/1874632786 shutdown_connections 2023-12-19T18:23:49.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.210+0000 7f3b431ef640 1 -- 172.21.15.27:0/1874632786 wait complete. 2023-12-19T18:23:49.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.212+0000 7f3b431ef640 1 Processor -- start 2023-12-19T18:23:49.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.212+0000 7f3b431ef640 1 -- start start 2023-12-19T18:23:49.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.213+0000 7f3b431ef640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b28004d00 0x7f3b3c14f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:49.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.213+0000 7f3b431ef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b3c157ab0 0x7f3b3c157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:49.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.213+0000 7f3b431ef640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b3c159160 0x7f3b3c1595e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.213+0000 7f3b431ef640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c159b20 con 0x7f3b3c157ab0 2023-12-19T18:23:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.213+0000 7f3b431ef640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0a8650 con 0x7f3b28004d00 2023-12-19T18:23:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.213+0000 7f3b431ef640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b3c0b0a90 con 0x7f3b3c159160 2023-12-19T18:23:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.214+0000 7f3b419ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b3c157ab0 0x7f3b3c157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.214+0000 7f3b419ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b3c157ab0 0x7f3b3c157f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37542/0 (socket says 172.21.15.27:37542) 2023-12-19T18:23:49.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.214+0000 7f3b419ec640 1 -- 172.21.15.27:0/1517947868 learned_addr learned my addr 172.21.15.27:0/1517947868 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.214+0000 7f3b421ed640 1 --2- 172.21.15.27:0/1517947868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b28004d00 0x7f3b3c14f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.214+0000 7f3b419ec640 1 -- 172.21.15.27:0/1517947868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b3c159160 msgr2=0x7f3b3c1595e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.214+0000 7f3b419ec640 1 --2- 172.21.15.27:0/1517947868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b3c159160 0x7f3b3c1595e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.214+0000 7f3b419ec640 1 -- 172.21.15.27:0/1517947868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b28004d00 msgr2=0x7f3b3c14f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.214+0000 7f3b419ec640 1 --2- 172.21.15.27:0/1517947868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b28004d00 0x7f3b3c14f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.214+0000 7f3b419ec640 1 -- 172.21.15.27:0/1517947868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b3c158620 con 0x7f3b3c157ab0 2023-12-19T18:23:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.214+0000 7f3b419ec640 1 --2- 172.21.15.27:0/1517947868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b3c157ab0 0x7f3b3c157f30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000d860 tx=0x7f3b3000dd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:49.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.215+0000 7f3b2effd640 1 -- 172.21.15.27:0/1517947868 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b300074e0 con 0x7f3b3c157ab0 2023-12-19T18:23:49.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.215+0000 7f3b2effd640 1 -- 172.21.15.27:0/1517947868 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b30007640 con 0x7f3b3c157ab0 2023-12-19T18:23:49.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.215+0000 7f3b2effd640 1 -- 172.21.15.27:0/1517947868 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b30004a00 con 0x7f3b3c157ab0 2023-12-19T18:23:49.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.215+0000 7f3b431ef640 1 -- 172.21.15.27:0/1517947868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b3c158900 con 0x7f3b3c157ab0 2023-12-19T18:23:49.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.215+0000 7f3b431ef640 1 -- 172.21.15.27:0/1517947868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b3c0b0fc0 con 0x7f3b3c157ab0 2023-12-19T18:23:49.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.216+0000 7f3b431ef640 1 -- 172.21.15.27:0/1517947868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b3c04e840 con 0x7f3b3c157ab0 2023-12-19T18:23:49.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.217+0000 7f3b2effd640 1 -- 172.21.15.27:0/1517947868 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3b30004b60 con 0x7f3b3c157ab0 2023-12-19T18:23:49.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.218+0000 7f3b2effd640 1 --2- 172.21.15.27:0/1517947868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3b2006a460 0x7f3b2006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:49.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.219+0000 7f3b2effd640 1 -- 172.21.15.27:0/1517947868 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3b30088290 con 0x7f3b3c157ab0 2023-12-19T18:23:49.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.219+0000 7f3b2effd640 1 -- 172.21.15.27:0/1517947868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7f3b20072050 con 0x7f3b3c157ab0 2023-12-19T18:23:49.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.219+0000 7f3b421ed640 1 --2- 172.21.15.27:0/1517947868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3b2006a460 0x7f3b2006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:49.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.220+0000 7f3b421ed640 1 --2- 172.21.15.27:0/1517947868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3b2006a460 0x7f3b2006c920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3b38007fd0 tx=0x7f3b38009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:49.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.224+0000 7f3b2effd640 1 -- 172.21.15.27:0/1517947868 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3b30019060 con 0x7f3b3c157ab0 2023-12-19T18:23:49.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.435+0000 7f3b431ef640 1 -- 172.21.15.27:0/1517947868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3b3c06f430 con 0x7f3b3c157ab0 2023-12-19T18:23:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.436+0000 7f3b2effd640 1 -- 172.21.15.27:0/1517947868 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v158) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3b30053840 con 0x7f3b3c157ab0 2023-12-19T18:23:49.436 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:23:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.439+0000 7f3b431ef640 1 -- 172.21.15.27:0/1517947868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3b2006a460 msgr2=0x7f3b2006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:49.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.439+0000 7f3b431ef640 1 --2- 172.21.15.27:0/1517947868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3b2006a460 0x7f3b2006c920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f3b38007fd0 tx=0x7f3b38009040 comp rx=0 tx=0).stop 2023-12-19T18:23:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.439+0000 7f3b431ef640 1 -- 172.21.15.27:0/1517947868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b3c157ab0 msgr2=0x7f3b3c157f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.439+0000 7f3b431ef640 1 --2- 172.21.15.27:0/1517947868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b3c157ab0 0x7f3b3c157f30 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f3b3000d860 tx=0x7f3b3000dd20 comp rx=0 tx=0).stop 2023-12-19T18:23:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.439+0000 7f3b431ef640 1 -- 172.21.15.27:0/1517947868 shutdown_connections 2023-12-19T18:23:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.439+0000 7f3b431ef640 1 --2- 172.21.15.27:0/1517947868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3b2006a460 0x7f3b2006c920 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:49.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.439+0000 7f3b431ef640 1 --2- 172.21.15.27:0/1517947868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3b3c159160 0x7f3b3c1595e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:49.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.439+0000 7f3b431ef640 1 --2- 172.21.15.27:0/1517947868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b3c157ab0 0x7f3b3c157f30 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:49.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.439+0000 7f3b431ef640 1 --2- 172.21.15.27:0/1517947868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3b28004d00 0x7f3b3c14f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:49.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.439+0000 7f3b431ef640 1 -- 172.21.15.27:0/1517947868 >> 172.21.15.27:0/1517947868 conn(0x7f3b3c06a7d0 msgr2=0x7f3b3c0cc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:49.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.439+0000 7f3b431ef640 1 -- 172.21.15.27:0/1517947868 shutdown_connections 2023-12-19T18:23:49.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:49.439+0000 7f3b431ef640 1 -- 172.21.15.27:0/1517947868 wait complete. 2023-12-19T18:23:49.460 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-19T18:23:49.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:49.460 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-19T18:23:49.558 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[90674]: fuse finished with error 0 and tester_r 0 2023-12-19T18:23:49.566 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:23:49.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:49.566 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-19T18:23:49.727 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:23:49.727 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:23:49.792 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:23:49.792 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:23:49.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:49.793 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-19T18:23:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-19T18:23:49.851 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:23:49.852 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:23:49.852 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:23:49.911 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:23:49.911 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:23:49.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:23:49.911 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-19T18:23:49.966 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-19T18:23:49.967 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:23:49.967 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:23:50.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.209+0000 7fc345a2d640 1 Processor -- start 2023-12-19T18:23:50.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.210+0000 7fc345a2d640 1 -- start start 2023-12-19T18:23:50.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.211+0000 7fc345a2d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc3400b4ee0 0x7fc3400d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:50.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.211+0000 7fc345a2d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc3400d1f60 0x7fc3400b69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:50.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.211+0000 7fc345a2d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3400b6540 con 0x7fc34015ceb0 2023-12-19T18:23:50.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.211+0000 7fc345a2d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3400cdfb0 con 0x7fc3400b4ee0 2023-12-19T18:23:50.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.211+0000 7fc345a2d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3400d2380 con 0x7fc3400d1f60 2023-12-19T18:23:50.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.212+0000 7fc344a2b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc34015ceb0 0x7fc3400b47d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59184/0 (socket says 172.21.15.27:59184) 2023-12-19T18:23:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.212+0000 7fc344a2b640 1 -- 172.21.15.27:0/1843771009 learned_addr learned my addr 172.21.15.27:0/1843771009 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.212+0000 7fc33ffff640 1 --2- 172.21.15.27:0/1843771009 >> v2:172.21.15.27:3300/0 conn(0x7fc3400b4ee0 0x7fc3400d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.212+0000 7fc33f7fe640 1 -- 172.21.15.27:0/1843771009 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 609475690 0 0) 0x7fc3400b6540 con 0x7fc34015ceb0 2023-12-19T18:23:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.212+0000 7fc34522c640 1 --2- 172.21.15.27:0/1843771009 >> v2:172.21.15.192:3300/0 conn(0x7fc3400d1f60 0x7fc3400b69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.212+0000 7fc33f7fe640 1 -- 172.21.15.27:0/1843771009 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc330003520 con 0x7fc34015ceb0 2023-12-19T18:23:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.212+0000 7fc33ffff640 1 -- 172.21.15.27:0/1843771009 >> v2:172.21.15.192:3300/0 conn(0x7fc3400d1f60 msgr2=0x7fc3400b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.212+0000 7fc33ffff640 1 --2- 172.21.15.27:0/1843771009 >> v2:172.21.15.192:3300/0 conn(0x7fc3400d1f60 0x7fc3400b69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.212+0000 7fc33ffff640 1 -- 172.21.15.27:0/1843771009 >> v1:172.21.15.27:6789/0 conn(0x7fc34015ceb0 legacy=0x7fc3400b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.212+0000 7fc33ffff640 1 -- 172.21.15.27:0/1843771009 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3400d2600 con 0x7fc3400b4ee0 2023-12-19T18:23:50.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.214+0000 7fc33ffff640 1 --2- 172.21.15.27:0/1843771009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3400b4ee0 0x7fc3400d1a20 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc3400d0590 tx=0x7fc32c02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85be483287f50ad4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:50.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.214+0000 7fc33f7fe640 1 -- 172.21.15.27:0/1843771009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c031830 con 0x7fc3400b4ee0 2023-12-19T18:23:50.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.214+0000 7fc33f7fe640 1 -- 172.21.15.27:0/1843771009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c031990 con 0x7fc3400b4ee0 2023-12-19T18:23:50.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.214+0000 7fc33f7fe640 1 -- 172.21.15.27:0/1843771009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c031d00 con 0x7fc3400b4ee0 2023-12-19T18:23:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.215+0000 7fc345a2d640 1 -- 172.21.15.27:0/1843771009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3400b4ee0 msgr2=0x7fc3400d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.215+0000 7fc345a2d640 1 --2- 172.21.15.27:0/1843771009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3400b4ee0 0x7fc3400d1a20 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7fc3400d0590 tx=0x7fc32c02b8d0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.215+0000 7fc345a2d640 1 -- 172.21.15.27:0/1843771009 shutdown_connections 2023-12-19T18:23:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.215+0000 7fc345a2d640 1 --2- 172.21.15.27:0/1843771009 >> v2:172.21.15.192:3300/0 conn(0x7fc3400d1f60 0x7fc3400b69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.215+0000 7fc345a2d640 1 --2- 172.21.15.27:0/1843771009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3400b4ee0 0x7fc3400d1a20 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.215+0000 7fc345a2d640 1 -- 172.21.15.27:0/1843771009 >> 172.21.15.27:0/1843771009 conn(0x7fc3400cf7d0 msgr2=0x7fc3400cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.215+0000 7fc345a2d640 1 -- 172.21.15.27:0/1843771009 shutdown_connections 2023-12-19T18:23:50.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.215+0000 7fc345a2d640 1 -- 172.21.15.27:0/1843771009 wait complete. 2023-12-19T18:23:50.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.217+0000 7fc345a2d640 1 Processor -- start 2023-12-19T18:23:50.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.217+0000 7fc345a2d640 1 -- start start 2023-12-19T18:23:50.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.218+0000 7fc345a2d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3400b4ee0 0x7fc34014e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:50.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.218+0000 7fc345a2d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3400d1f60 0x7fc3400adaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:50.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.218+0000 7fc345a2d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34015ceb0 0x7fc3400adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:50.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.218+0000 7fc345a2d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3401f22d0 con 0x7fc34015ceb0 2023-12-19T18:23:50.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.218+0000 7fc345a2d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3401f2540 con 0x7fc3400b4ee0 2023-12-19T18:23:50.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.218+0000 7fc345a2d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3401f2820 con 0x7fc3400d1f60 2023-12-19T18:23:50.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.219+0000 7fc34522c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34015ceb0 0x7fc3400adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:50.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.219+0000 7fc34522c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34015ceb0 0x7fc3400adfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37558/0 (socket says 172.21.15.27:37558) 2023-12-19T18:23:50.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.219+0000 7fc34522c640 1 -- 172.21.15.27:0/3716933426 learned_addr learned my addr 172.21.15.27:0/3716933426 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:50.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.219+0000 7fc33ffff640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3400d1f60 0x7fc3400adaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:50.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.219+0000 7fc344a2b640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3400b4ee0 0x7fc34014e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:50.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.219+0000 7fc34522c640 1 -- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3400d1f60 msgr2=0x7fc3400adaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:50.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.219+0000 7fc34522c640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3400d1f60 0x7fc3400adaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.219+0000 7fc34522c640 1 -- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3400b4ee0 msgr2=0x7fc34014e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:50.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.219+0000 7fc34522c640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3400b4ee0 0x7fc34014e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.219+0000 7fc34522c640 1 -- 172.21.15.27:0/3716933426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c009400 con 0x7fc34015ceb0 2023-12-19T18:23:50.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.219+0000 7fc344a2b640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3400b4ee0 0x7fc34014e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:50.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.220+0000 7fc34522c640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34015ceb0 0x7fc3400adfe0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc338007e40 tx=0x7fc33800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:50.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.220+0000 7fc33d7fa640 1 -- 172.21.15.27:0/3716933426 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338014620 con 0x7fc34015ceb0 2023-12-19T18:23:50.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.220+0000 7fc33d7fa640 1 -- 172.21.15.27:0/3716933426 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc33800ee90 con 0x7fc34015ceb0 2023-12-19T18:23:50.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.220+0000 7fc33d7fa640 1 -- 172.21.15.27:0/3716933426 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3380149e0 con 0x7fc34015ceb0 2023-12-19T18:23:50.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.220+0000 7fc345a2d640 1 -- 172.21.15.27:0/3716933426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3401f2b00 con 0x7fc34015ceb0 2023-12-19T18:23:50.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.220+0000 7fc345a2d640 1 -- 172.21.15.27:0/3716933426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc340151820 con 0x7fc34015ceb0 2023-12-19T18:23:50.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.221+0000 7fc345a2d640 1 -- 172.21.15.27:0/3716933426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3401f2cf0 con 0x7fc34015ceb0 2023-12-19T18:23:50.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.222+0000 7fc33d7fa640 1 -- 172.21.15.27:0/3716933426 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc338014b40 con 0x7fc34015ceb0 2023-12-19T18:23:50.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.223+0000 7fc33d7fa640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc32406a430 0x7fc32406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:50.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.223+0000 7fc33d7fa640 1 -- 172.21.15.27:0/3716933426 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc3380893a0 con 0x7fc34015ceb0 2023-12-19T18:23:50.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.223+0000 7fc33d7fa640 1 -- 172.21.15.27:0/3716933426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7fc324072020 con 0x7fc34015ceb0 2023-12-19T18:23:50.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.224+0000 7fc344a2b640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc32406a430 0x7fc32406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:50.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.225+0000 7fc344a2b640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc32406a430 0x7fc32406c8f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc334005880 tx=0x7fc334009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:50.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.227+0000 7fc33d7fa640 1 -- 172.21.15.27:0/3716933426 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc3380548b0 con 0x7fc34015ceb0 2023-12-19T18:23:50.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.457+0000 7fc345a2d640 1 -- 172.21.15.27:0/3716933426 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc34004e840 con 0x7fc34015ceb0 2023-12-19T18:23:50.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.458+0000 7fc33d7fa640 1 -- 172.21.15.27:0/3716933426 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fc338057ed0 con 0x7fc34015ceb0 2023-12-19T18:23:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:50.458 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":61,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3298228307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3298228307},{"type":"v1","addr":"172.21.15.192:6835","nonce":3298228307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5726,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1760483999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1760483999},{"type":"v1","addr":"172.21.15.192:6837","nonce":1760483999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3000637822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3000637822},{"type":"v1","addr":"172.21.15.27:6833","nonce":3000637822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:23:06.465885+0000","modified":"2023-12-19T18:23:09.973228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1593995318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1593995318},{"type":"v1","addr":"172.21.15.27:6835","nonce":1593995318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T18:23:50.459 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 61 2023-12-19T18:23:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.461+0000 7fc345a2d640 1 -- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc32406a430 msgr2=0x7fc32406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.461+0000 7fc345a2d640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc32406a430 0x7fc32406c8f0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fc334005880 tx=0x7fc334009040 comp rx=0 tx=0).stop 2023-12-19T18:23:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.461+0000 7fc345a2d640 1 -- 172.21.15.27:0/3716933426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34015ceb0 msgr2=0x7fc3400adfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.461+0000 7fc345a2d640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34015ceb0 0x7fc3400adfe0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fc338007e40 tx=0x7fc33800e4b0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.461+0000 7fc345a2d640 1 -- 172.21.15.27:0/3716933426 shutdown_connections 2023-12-19T18:23:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.461+0000 7fc345a2d640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc32406a430 0x7fc32406c8f0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.461+0000 7fc345a2d640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34015ceb0 0x7fc3400adfe0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.461+0000 7fc345a2d640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3400d1f60 0x7fc3400adaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.461+0000 7fc345a2d640 1 --2- 172.21.15.27:0/3716933426 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3400b4ee0 0x7fc34014e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.461+0000 7fc345a2d640 1 -- 172.21.15.27:0/3716933426 >> 172.21.15.27:0/3716933426 conn(0x7fc3400cf7d0 msgr2=0x7fc3400c9e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:50.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.461+0000 7fc345a2d640 1 -- 172.21.15.27:0/3716933426 shutdown_connections 2023-12-19T18:23:50.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.461+0000 7fc345a2d640 1 -- 172.21.15.27:0/3716933426 wait complete. 2023-12-19T18:23:50.481 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:23:50.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.702+0000 7f0c428ee640 1 Processor -- start 2023-12-19T18:23:50.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.703+0000 7f0c428ee640 1 -- start start 2023-12-19T18:23:50.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.704+0000 7f0c428ee640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0c3c15af70 0x7f0c3c0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:50.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.704+0000 7f0c428ee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c3c0cbce0 0x7f0c3c0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:50.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.704+0000 7f0c428ee640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0c3c0ada50 0x7f0c3c0adf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:50.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.704+0000 7f0c428ee640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c3c0cb850 con 0x7f0c3c15af70 2023-12-19T18:23:50.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.704+0000 7f0c428ee640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c3c15b350 con 0x7f0c3c0ada50 2023-12-19T18:23:50.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.704+0000 7f0c428ee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c0cac40 con 0x7f0c3c0cbce0 2023-12-19T18:23:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.705+0000 7f0c3bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0c3c15af70 0x7f0c3c0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.705+0000 7f0c3bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0c3c15af70 0x7f0c3c0ca540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37562/0 (socket says 172.21.15.27:37562) 2023-12-19T18:23:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.705+0000 7f0c3bfff640 1 -- 172.21.15.27:0/558254563 learned_addr learned my addr 172.21.15.27:0/558254563 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.705+0000 7f0c3b7fe640 1 --2- 172.21.15.27:0/558254563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c3c0cbce0 0x7f0c3c0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.705+0000 7f0c40e8b640 1 --2- 172.21.15.27:0/558254563 >> v2:172.21.15.192:3300/0 conn(0x7f0c3c0ada50 0x7f0c3c0adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.705+0000 7f0c3bfff640 1 -- 172.21.15.27:0/558254563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c3c0cbce0 msgr2=0x7f0c3c0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.705+0000 7f0c3bfff640 1 --2- 172.21.15.27:0/558254563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c3c0cbce0 0x7f0c3c0ad510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.705+0000 7f0c3bfff640 1 -- 172.21.15.27:0/558254563 >> v2:172.21.15.192:3300/0 conn(0x7f0c3c0ada50 msgr2=0x7f0c3c0adf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.705+0000 7f0c3bfff640 1 --2- 172.21.15.27:0/558254563 >> v2:172.21.15.192:3300/0 conn(0x7f0c3c0ada50 0x7f0c3c0adf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.705+0000 7f0c3bfff640 1 -- 172.21.15.27:0/558254563 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c3c0caf20 con 0x7f0c3c15af70 2023-12-19T18:23:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.707+0000 7f0c3bfff640 1 --2- 172.21.15.27:0/558254563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c3c15af70 0x7f0c3c0ca540 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0b1730 tx=0x7f0c2c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5af2e478955b4e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:50.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.707+0000 7f0c3affd640 1 -- 172.21.15.27:0/558254563 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c02ca70 con 0x7f0c3c15af70 2023-12-19T18:23:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.707+0000 7f0c3affd640 1 -- 172.21.15.27:0/558254563 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c2c02cbd0 con 0x7f0c3c15af70 2023-12-19T18:23:50.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.707+0000 7f0c3affd640 1 -- 172.21.15.27:0/558254563 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c0343f0 con 0x7f0c3c15af70 2023-12-19T18:23:50.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.708+0000 7f0c428ee640 1 -- 172.21.15.27:0/558254563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c3c15af70 msgr2=0x7f0c3c0ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:50.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.708+0000 7f0c428ee640 1 --2- 172.21.15.27:0/558254563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c3c15af70 0x7f0c3c0ca540 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0b1730 tx=0x7f0c2c02bed0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.708+0000 7f0c428ee640 1 -- 172.21.15.27:0/558254563 shutdown_connections 2023-12-19T18:23:50.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.708+0000 7f0c428ee640 1 --2- 172.21.15.27:0/558254563 >> v2:172.21.15.192:3300/0 conn(0x7f0c3c0ada50 0x7f0c3c0adf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.708+0000 7f0c428ee640 1 --2- 172.21.15.27:0/558254563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c3c0cbce0 0x7f0c3c0ad510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.708+0000 7f0c428ee640 1 --2- 172.21.15.27:0/558254563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c3c15af70 0x7f0c3c0ca540 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.708+0000 7f0c428ee640 1 -- 172.21.15.27:0/558254563 >> 172.21.15.27:0/558254563 conn(0x7f0c3c06a330 msgr2=0x7f0c3c0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:50.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.708+0000 7f0c428ee640 1 -- 172.21.15.27:0/558254563 shutdown_connections 2023-12-19T18:23:50.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.708+0000 7f0c428ee640 1 -- 172.21.15.27:0/558254563 wait complete. 2023-12-19T18:23:50.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.711+0000 7f0c428ee640 1 Processor -- start 2023-12-19T18:23:50.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.711+0000 7f0c428ee640 1 -- start start 2023-12-19T18:23:50.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.712+0000 7f0c428ee640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c3c0ada50 0x7f0c3c0b2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:50.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.712+0000 7f0c428ee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c3c0cbce0 0x7f0c3c15cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:50.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.712+0000 7f0c428ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c3c15af70 0x7f0c3c15c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.712+0000 7f0c428ee640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c15d330 con 0x7f0c3c15af70 2023-12-19T18:23:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.712+0000 7f0c428ee640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c15d5d0 con 0x7f0c3c0ada50 2023-12-19T18:23:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.712+0000 7f0c428ee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c15b670 con 0x7f0c3c0cbce0 2023-12-19T18:23:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.712+0000 7f0c40e8b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c3c15af70 0x7f0c3c15c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.712+0000 7f0c40e8b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c3c15af70 0x7f0c3c15c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37564/0 (socket says 172.21.15.27:37564) 2023-12-19T18:23:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.712+0000 7f0c40e8b640 1 -- 172.21.15.27:0/829807558 learned_addr learned my addr 172.21.15.27:0/829807558 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:50.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.713+0000 7f0c3b7fe640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c3c0cbce0 0x7f0c3c15cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:50.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.713+0000 7f0c3bfff640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c3c0ada50 0x7f0c3c0b2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:50.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.713+0000 7f0c40e8b640 1 -- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c3c0cbce0 msgr2=0x7f0c3c15cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:50.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.713+0000 7f0c40e8b640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c3c0cbce0 0x7f0c3c15cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.713+0000 7f0c40e8b640 1 -- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c3c0ada50 msgr2=0x7f0c3c0b2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:50.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.713+0000 7f0c40e8b640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c3c0ada50 0x7f0c3c0b2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.713+0000 7f0c40e8b640 1 -- 172.21.15.27:0/829807558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c009b30 con 0x7f0c3c15af70 2023-12-19T18:23:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.713+0000 7f0c40e8b640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c3c15af70 0x7f0c3c15c150 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400d800 tx=0x7f0c3400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.714+0000 7f0c38ff9640 1 -- 172.21.15.27:0/829807558 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c340096c0 con 0x7f0c3c15af70 2023-12-19T18:23:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.714+0000 7f0c38ff9640 1 -- 172.21.15.27:0/829807558 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c34009840 con 0x7f0c3c15af70 2023-12-19T18:23:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.714+0000 7f0c428ee640 1 -- 172.21.15.27:0/829807558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3c15b950 con 0x7f0c3c15af70 2023-12-19T18:23:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.714+0000 7f0c38ff9640 1 -- 172.21.15.27:0/829807558 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34009bf0 con 0x7f0c3c15af70 2023-12-19T18:23:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.714+0000 7f0c428ee640 1 -- 172.21.15.27:0/829807558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3c15c690 con 0x7f0c3c15af70 2023-12-19T18:23:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.715+0000 7f0c428ee640 1 -- 172.21.15.27:0/829807558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c3c068e90 con 0x7f0c3c15af70 2023-12-19T18:23:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.716+0000 7f0c38ff9640 1 -- 172.21.15.27:0/829807558 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0c34009d50 con 0x7f0c3c15af70 2023-12-19T18:23:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.717+0000 7f0c38ff9640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0c2806a430 0x7f0c2806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.717+0000 7f0c38ff9640 1 -- 172.21.15.27:0/829807558 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0c34088da0 con 0x7f0c3c15af70 2023-12-19T18:23:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.717+0000 7f0c38ff9640 1 -- 172.21.15.27:0/829807558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7f0c28072020 con 0x7f0c3c15af70 2023-12-19T18:23:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.718+0000 7f0c3bfff640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0c2806a430 0x7f0c2806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.719+0000 7f0c3bfff640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0c2806a430 0x7f0c2806c8f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c035000 tx=0x7f0c2c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:50.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.721+0000 7f0c38ff9640 1 -- 172.21.15.27:0/829807558 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0c34054340 con 0x7f0c3c15af70 2023-12-19T18:23:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.917+0000 7f0c428ee640 1 -- 172.21.15.27:0/829807558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0c3c1529b0 con 0x7f0c3c15af70 2023-12-19T18:23:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.918+0000 7f0c38ff9640 1 -- 172.21.15.27:0/829807558 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16239 (secure 0 0 0) 0x7f0c34018070 con 0x7f0c3c15af70 2023-12-19T18:23:50.918 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:50.919 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":158,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:23:46.716988+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:23:04.431250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T18:23:05.506926+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":4,"to":3}]},{"pgid":"17.f","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:23:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.921+0000 7f0c428ee640 1 -- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0c2806a430 msgr2=0x7f0c2806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.921+0000 7f0c428ee640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0c2806a430 0x7f0c2806c8f0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c035000 tx=0x7f0c2c030000 comp rx=0 tx=0).stop 2023-12-19T18:23:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.921+0000 7f0c428ee640 1 -- 172.21.15.27:0/829807558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c3c15af70 msgr2=0x7f0c3c15c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.921+0000 7f0c428ee640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c3c15af70 0x7f0c3c15c150 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7f0c3400d800 tx=0x7f0c3400dcc0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.921+0000 7f0c428ee640 1 -- 172.21.15.27:0/829807558 shutdown_connections 2023-12-19T18:23:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.921+0000 7f0c428ee640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0c2806a430 0x7f0c2806c8f0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.921+0000 7f0c428ee640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c3c15af70 0x7f0c3c15c150 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.921+0000 7f0c428ee640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c3c0cbce0 0x7f0c3c15cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.921+0000 7f0c428ee640 1 --2- 172.21.15.27:0/829807558 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c3c0ada50 0x7f0c3c0b2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.921+0000 7f0c428ee640 1 -- 172.21.15.27:0/829807558 >> 172.21.15.27:0/829807558 conn(0x7f0c3c06a330 msgr2=0x7f0c3c0afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.922+0000 7f0c428ee640 1 -- 172.21.15.27:0/829807558 shutdown_connections 2023-12-19T18:23:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:50.922+0000 7f0c428ee640 1 -- 172.21.15.27:0/829807558 wait complete. 2023-12-19T18:23:50.942 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:23:04.431250+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '151', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:23:05.506926+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:23:50.943 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:23:51.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.175+0000 7efd15c51640 1 Processor -- start 2023-12-19T18:23:51.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.175+0000 7efd15c51640 1 -- start start 2023-12-19T18:23:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.177+0000 7efd15c51640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1015b1a0 0x7efd1006f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.177+0000 7efd15c51640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efd10071420 0x7efd1015ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.177+0000 7efd15c51640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1006bc80 con 0x7efd100c2720 2023-12-19T18:23:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.177+0000 7efd15c51640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1006d690 con 0x7efd10071420 2023-12-19T18:23:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.177+0000 7efd15c51640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1006d0a0 con 0x7efd1015b1a0 2023-12-19T18:23:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.177+0000 7efd07fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efd100c2720 0x7efd1006fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59194/0 (socket says 172.21.15.27:59194) 2023-12-19T18:23:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.177+0000 7efd07fff640 1 -- 172.21.15.27:0/3830744907 learned_addr learned my addr 172.21.15.27:0/3830744907 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.178+0000 7efd0ffff640 1 --2- 172.21.15.27:0/3830744907 >> v2:172.21.15.192:3300/0 conn(0x7efd10071420 0x7efd1015ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.178+0000 7efd0f7fe640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1015b1a0 0x7efd1006f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.178+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 288261841 0 0) 0x7efcf8001110 con 0x7efd100c2720 2023-12-19T18:23:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.178+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efcfc003520 con 0x7efd100c2720 2023-12-19T18:23:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.178+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3504074381 0 0) 0x7efcfc003520 con 0x7efd100c2720 2023-12-19T18:23:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.178+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1015b1a0 msgr2=0x7efd1006f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.178+0000 7efd0effd640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1015b1a0 0x7efd1006f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.178+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 >> v2:172.21.15.192:3300/0 conn(0x7efd10071420 msgr2=0x7efd1015ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:51.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.178+0000 7efd0effd640 1 --2- 172.21.15.27:0/3830744907 >> v2:172.21.15.192:3300/0 conn(0x7efd10071420 0x7efd1015ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.178+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1006b6c0 con 0x7efd100c2720 2023-12-19T18:23:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.178+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7efcf8001800 con 0x7efd100c2720 2023-12-19T18:23:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 >> v1:172.21.15.27:6789/0 conn(0x7efd100c2720 legacy=0x7efd1006fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0effd640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcfc003970 0x7efcfc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0effd640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc0047c0 0x7efcfc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0effd640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1015b1a0 0x7efcfc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1006d690 con 0x7efcfc003970 2023-12-19T18:23:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1006d0a0 con 0x7efcfc0047c0 2023-12-19T18:23:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc00fd90 con 0x7efd1015b1a0 2023-12-19T18:23:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0f7fe640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcfc003970 0x7efcfc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd07fff640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc0047c0 0x7efcfc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0f7fe640 1 -- 172.21.15.27:0/3830744907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1015b1a0 msgr2=0x7efcfc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0ffff640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1015b1a0 0x7efcfc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0f7fe640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1015b1a0 0x7efcfc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0f7fe640 1 -- 172.21.15.27:0/3830744907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc0047c0 msgr2=0x7efcfc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0f7fe640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc0047c0 0x7efcfc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0f7fe640 1 -- 172.21.15.27:0/3830744907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcfc003520 con 0x7efcfc003970 2023-12-19T18:23:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd07fff640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efcfc0047c0 0x7efcfc004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.179+0000 7efd0ffff640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd1015b1a0 0x7efcfc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:51.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.181+0000 7efd0f7fe640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcfc003970 0x7efcfc003d90 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7efd00009af0 tx=0x7efcfc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7818af4b58d811c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:51.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.181+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0002c6b0 con 0x7efcfc003970 2023-12-19T18:23:51.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.181+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0002c810 con 0x7efcfc003970 2023-12-19T18:23:51.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.181+0000 7efd0effd640 1 -- 172.21.15.27:0/3830744907 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0002cba0 con 0x7efcfc003970 2023-12-19T18:23:51.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.181+0000 7efd15c51640 1 -- 172.21.15.27:0/3830744907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcfc003970 msgr2=0x7efcfc003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:51.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.181+0000 7efd15c51640 1 --2- 172.21.15.27:0/3830744907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efcfc003970 0x7efcfc003d90 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7efd00009af0 tx=0x7efcfc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.181+0000 7efd0ffff640 1 -- 172.21.15.27:0/3830744907 reap_dead start 2023-12-19T18:23:51.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.182+0000 7efd15c51640 1 -- 172.21.15.27:0/3830744907 shutdown_connections 2023-12-19T18:23:51.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.182+0000 7efd15c51640 1 -- 172.21.15.27:0/3830744907 >> 172.21.15.27:0/3830744907 conn(0x7efd100cebd0 msgr2=0x7efd100cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:51.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.182+0000 7efd15c51640 1 -- 172.21.15.27:0/3830744907 shutdown_connections 2023-12-19T18:23:51.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.182+0000 7efd15c51640 1 -- 172.21.15.27:0/3830744907 wait complete. 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.183+0000 7efd15c51640 1 Processor -- start 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.184+0000 7efd15c51640 1 -- start start 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.184+0000 7efd15c51640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcfc003970 0x7efd10149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.184+0000 7efd15c51640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd10156630 0x7efd10159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd15c51640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd101584c0 0x7efd100c7e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd15c51640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd101495c0 con 0x7efd101584c0 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd15c51640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd101595d0 con 0x7efd10156630 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd15c51640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd101589c0 con 0x7efcfc003970 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd0ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd101584c0 0x7efd100c7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd0ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd101584c0 0x7efd100c7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37586/0 (socket says 172.21.15.27:37586) 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd0ffff640 1 -- 172.21.15.27:0/3031130959 learned_addr learned my addr 172.21.15.27:0/3031130959 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd0f7fe640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcfc003970 0x7efd10149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd07fff640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd10156630 0x7efd10159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd0ffff640 1 -- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcfc003970 msgr2=0x7efd10149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd0ffff640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcfc003970 0x7efd10149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd0ffff640 1 -- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd10156630 msgr2=0x7efd10159a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd0ffff640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd10156630 0x7efd10159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd0ffff640 1 -- 172.21.15.27:0/3031130959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd10158ca0 con 0x7efd101584c0 2023-12-19T18:23:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.185+0000 7efd0f7fe640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcfc003970 0x7efd10149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.186+0000 7efd0ffff640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd101584c0 0x7efd100c7e70 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7efd100cea40 tx=0x7efd0800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.186+0000 7efd0cff9640 1 -- 172.21.15.27:0/3031130959 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08017400 con 0x7efd101584c0 2023-12-19T18:23:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.186+0000 7efd0cff9640 1 -- 172.21.15.27:0/3031130959 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd08011040 con 0x7efd101584c0 2023-12-19T18:23:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.186+0000 7efd0cff9640 1 -- 172.21.15.27:0/3031130959 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08017780 con 0x7efd101584c0 2023-12-19T18:23:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.186+0000 7efd15c51640 1 -- 172.21.15.27:0/3031130959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd10158f80 con 0x7efd101584c0 2023-12-19T18:23:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.186+0000 7efd15c51640 1 -- 172.21.15.27:0/3031130959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd10159fa0 con 0x7efd101584c0 2023-12-19T18:23:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.187+0000 7efd15c51640 1 -- 172.21.15.27:0/3031130959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd10159760 con 0x7efd101584c0 2023-12-19T18:23:51.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.188+0000 7efd0cff9640 1 -- 172.21.15.27:0/3031130959 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efd080203b0 con 0x7efd101584c0 2023-12-19T18:23:51.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.189+0000 7efd0cff9640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efcf006a430 0x7efcf006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.189+0000 7efd0cff9640 1 -- 172.21.15.27:0/3031130959 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efd08088b80 con 0x7efd101584c0 2023-12-19T18:23:51.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.190+0000 7efd0cff9640 1 -- 172.21.15.27:0/3031130959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7efcf0072020 con 0x7efd101584c0 2023-12-19T18:23:51.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.190+0000 7efd0f7fe640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efcf006a430 0x7efcf006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.191+0000 7efd0f7fe640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efcf006a430 0x7efcf006c8f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efd00036000 tx=0x7efd00030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:51.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.193+0000 7efd0cff9640 1 -- 172.21.15.27:0/3031130959 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efd08054090 con 0x7efd101584c0 2023-12-19T18:23:51.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.387+0000 7efd15c51640 1 -- 172.21.15.27:0/3031130959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd10068e90 con 0x7efd101584c0 2023-12-19T18:23:51.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.388+0000 7efd0cff9640 1 -- 172.21.15.27:0/3031130959 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v158) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd08088e60 con 0x7efd101584c0 2023-12-19T18:23:51.388 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:23:51.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.390+0000 7efd15c51640 1 -- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efcf006a430 msgr2=0x7efcf006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:51.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.390+0000 7efd15c51640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efcf006a430 0x7efcf006c8f0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7efd00036000 tx=0x7efd00030000 comp rx=0 tx=0).stop 2023-12-19T18:23:51.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.391+0000 7efd15c51640 1 -- 172.21.15.27:0/3031130959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd101584c0 msgr2=0x7efd100c7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:51.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.391+0000 7efd15c51640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd101584c0 0x7efd100c7e70 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7efd100cea40 tx=0x7efd0800f4a0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.391+0000 7efd15c51640 1 -- 172.21.15.27:0/3031130959 shutdown_connections 2023-12-19T18:23:51.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.391+0000 7efd15c51640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efcf006a430 0x7efcf006c8f0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.391+0000 7efd15c51640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd101584c0 0x7efd100c7e70 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.391+0000 7efd15c51640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd10156630 0x7efd10159a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.391+0000 7efd15c51640 1 --2- 172.21.15.27:0/3031130959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efcfc003970 0x7efd10149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.391+0000 7efd15c51640 1 -- 172.21.15.27:0/3031130959 >> 172.21.15.27:0/3031130959 conn(0x7efd100cebd0 msgr2=0x7efd100cdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:51.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.391+0000 7efd15c51640 1 -- 172.21.15.27:0/3031130959 shutdown_connections 2023-12-19T18:23:51.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.391+0000 7efd15c51640 1 -- 172.21.15.27:0/3031130959 wait complete. 2023-12-19T18:23:51.409 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:23:51.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.642+0000 7fece1afa640 1 Processor -- start 2023-12-19T18:23:51.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.643+0000 7fece1afa640 1 -- start start 2023-12-19T18:23:51.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.644+0000 7fece1afa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fecdc15d120 0x7fecdc0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.644+0000 7fece1afa640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecdc0ce950 0x7fecdc0d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.644+0000 7fece1afa640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecdc0d3850 con 0x7fecdc0d1310 2023-12-19T18:23:51.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.644+0000 7fece1afa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecdc0d0b10 con 0x7fecdc15d120 2023-12-19T18:23:51.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.644+0000 7fece1afa640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0d1c90 con 0x7fecdc0ce950 2023-12-19T18:23:51.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.645+0000 7fece0af8640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fecdc15d120 0x7fecdc0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.645+0000 7fecdbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fecdc0d1310 0x7fecdc0b2970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59202/0 (socket says 172.21.15.27:59202) 2023-12-19T18:23:51.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.645+0000 7fecdbfff640 1 -- 172.21.15.27:0/3140384041 learned_addr learned my addr 172.21.15.27:0/3140384041 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:51.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.645+0000 7fece12f9640 1 --2- 172.21.15.27:0/3140384041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecdc0ce950 0x7fecdc0d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.645+0000 7fece0af8640 1 -- 172.21.15.27:0/3140384041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecdc0ce950 msgr2=0x7fecdc0d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:51.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.645+0000 7fece0af8640 1 --2- 172.21.15.27:0/3140384041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecdc0ce950 0x7fecdc0d1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.645+0000 7fece0af8640 1 -- 172.21.15.27:0/3140384041 >> v1:172.21.15.27:6789/0 conn(0x7fecdc0d1310 legacy=0x7fecdc0b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:51.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.645+0000 7fece0af8640 1 -- 172.21.15.27:0/3140384041 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc15d500 con 0x7fecdc15d120 2023-12-19T18:23:51.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.646+0000 7fece0af8640 1 --2- 172.21.15.27:0/3140384041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecdc15d120 0x7fecdc0cfd70 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fecd00097a0 tx=0x7fecd002b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e003271a5e10682 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:51.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.646+0000 7fecdb7fe640 1 -- 172.21.15.27:0/3140384041 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd002d040 con 0x7fecdc15d120 2023-12-19T18:23:51.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.647+0000 7fecdb7fe640 1 -- 172.21.15.27:0/3140384041 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd0031810 con 0x7fecdc15d120 2023-12-19T18:23:51.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.647+0000 7fecdb7fe640 1 -- 172.21.15.27:0/3140384041 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd0031ba0 con 0x7fecdc15d120 2023-12-19T18:23:51.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.647+0000 7fece1afa640 1 -- 172.21.15.27:0/3140384041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecdc15d120 msgr2=0x7fecdc0cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:51.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.647+0000 7fece1afa640 1 --2- 172.21.15.27:0/3140384041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecdc15d120 0x7fecdc0cfd70 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7fecd00097a0 tx=0x7fecd002b8b0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.647+0000 7fece1afa640 1 -- 172.21.15.27:0/3140384041 shutdown_connections 2023-12-19T18:23:51.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.647+0000 7fece1afa640 1 --2- 172.21.15.27:0/3140384041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecdc0ce950 0x7fecdc0d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.647+0000 7fece1afa640 1 --2- 172.21.15.27:0/3140384041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecdc15d120 0x7fecdc0cfd70 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.647+0000 7fece1afa640 1 -- 172.21.15.27:0/3140384041 >> 172.21.15.27:0/3140384041 conn(0x7fecdc0d2d30 msgr2=0x7fecdc0d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:51.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.647+0000 7fece1afa640 1 -- 172.21.15.27:0/3140384041 shutdown_connections 2023-12-19T18:23:51.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.647+0000 7fece1afa640 1 -- 172.21.15.27:0/3140384041 wait complete. 2023-12-19T18:23:51.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.649+0000 7fece1afa640 1 Processor -- start 2023-12-19T18:23:51.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.649+0000 7fece1afa640 1 -- start start 2023-12-19T18:23:51.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.650+0000 7fece1afa640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecdc0ce950 0x7fecdc1f3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.650+0000 7fece1afa640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecdc0d1310 0x7fecdc0af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.650+0000 7fece1afa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecdc15d120 0x7fecdc1593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.650+0000 7fece1afa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0afe40 con 0x7fecdc15d120 2023-12-19T18:23:51.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.650+0000 7fece1afa640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0b08d0 con 0x7fecdc0ce950 2023-12-19T18:23:51.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.650+0000 7fece1afa640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc0b7840 con 0x7fecdc0d1310 2023-12-19T18:23:51.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.651+0000 7fece12f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecdc15d120 0x7fecdc1593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.651+0000 7fece12f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecdc15d120 0x7fecdc1593c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37610/0 (socket says 172.21.15.27:37610) 2023-12-19T18:23:51.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.651+0000 7fece12f9640 1 -- 172.21.15.27:0/1823491665 learned_addr learned my addr 172.21.15.27:0/1823491665 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:51.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.651+0000 7fece12f9640 1 -- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecdc0d1310 msgr2=0x7fecdc0af900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:51.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.651+0000 7fece12f9640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecdc0d1310 0x7fecdc0af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.651+0000 7fece12f9640 1 -- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecdc0ce950 msgr2=0x7fecdc1f3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:51.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.651+0000 7fece0af8640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecdc0ce950 0x7fecdc1f3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.651+0000 7fece12f9640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecdc0ce950 0x7fecdc1f3080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.651+0000 7fece12f9640 1 -- 172.21.15.27:0/1823491665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd0009400 con 0x7fecdc15d120 2023-12-19T18:23:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.651+0000 7fece0af8640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecdc0ce950 0x7fecdc1f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.652+0000 7fece12f9640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecdc15d120 0x7fecdc1593c0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fecd4007e90 tx=0x7fecd400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.652+0000 7fecd97fa640 1 -- 172.21.15.27:0/1823491665 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4014420 con 0x7fecdc15d120 2023-12-19T18:23:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.652+0000 7fecd97fa640 1 -- 172.21.15.27:0/1823491665 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd4014580 con 0x7fecdc15d120 2023-12-19T18:23:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.652+0000 7fecd97fa640 1 -- 172.21.15.27:0/1823491665 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd4014930 con 0x7fecdc15d120 2023-12-19T18:23:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.652+0000 7fece1afa640 1 -- 172.21.15.27:0/1823491665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc0b0030 con 0x7fecdc15d120 2023-12-19T18:23:51.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.652+0000 7fece1afa640 1 -- 172.21.15.27:0/1823491665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc0b05f0 con 0x7fecdc15d120 2023-12-19T18:23:51.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.654+0000 7fece1afa640 1 -- 172.21.15.27:0/1823491665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecdc04e840 con 0x7fecdc15d120 2023-12-19T18:23:51.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.654+0000 7fecd97fa640 1 -- 172.21.15.27:0/1823491665 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fecd4014a90 con 0x7fecdc15d120 2023-12-19T18:23:51.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.655+0000 7fecd97fa640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fecc006a460 0x7fecc006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:51.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.656+0000 7fecd97fa640 1 -- 172.21.15.27:0/1823491665 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fecd4089140 con 0x7fecdc15d120 2023-12-19T18:23:51.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.656+0000 7fecd97fa640 1 -- 172.21.15.27:0/1823491665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7fecc0072050 con 0x7fecdc15d120 2023-12-19T18:23:51.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.656+0000 7fece0af8640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fecc006a460 0x7fecc006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:51.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.657+0000 7fece0af8640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fecc006a460 0x7fecc006c920 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fecd0036000 tx=0x7fecd003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:51.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.660+0000 7fecd97fa640 1 -- 172.21.15.27:0/1823491665 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fecd4054780 con 0x7fecdc15d120 2023-12-19T18:23:51.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.853+0000 7fece1afa640 1 -- 172.21.15.27:0/1823491665 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fecdc0b7980 con 0x7fecdc15d120 2023-12-19T18:23:51.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.854+0000 7fecd97fa640 1 -- 172.21.15.27:0/1823491665 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v158) v1 ==== 104+0+11 (secure 0 0 0) 0x7fecd4057da0 con 0x7fecdc15d120 2023-12-19T18:23:51.855 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:23:51.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.857+0000 7fece1afa640 1 -- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fecc006a460 msgr2=0x7fecc006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:51.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.857+0000 7fece1afa640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fecc006a460 0x7fecc006c920 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fecd0036000 tx=0x7fecd003c040 comp rx=0 tx=0).stop 2023-12-19T18:23:51.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.857+0000 7fece1afa640 1 -- 172.21.15.27:0/1823491665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecdc15d120 msgr2=0x7fecdc1593c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:51.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.857+0000 7fece1afa640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecdc15d120 0x7fecdc1593c0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7fecd4007e90 tx=0x7fecd400e4b0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.857+0000 7fece1afa640 1 -- 172.21.15.27:0/1823491665 shutdown_connections 2023-12-19T18:23:51.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.857+0000 7fece1afa640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fecc006a460 0x7fecc006c920 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.857+0000 7fece1afa640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fecdc15d120 0x7fecdc1593c0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.857+0000 7fece1afa640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fecdc0d1310 0x7fecdc0af900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.857+0000 7fece1afa640 1 --2- 172.21.15.27:0/1823491665 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fecdc0ce950 0x7fecdc1f3080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.857+0000 7fece1afa640 1 -- 172.21.15.27:0/1823491665 >> 172.21.15.27:0/1823491665 conn(0x7fecdc0d2d30 msgr2=0x7fecdc0cd890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.857+0000 7fece1afa640 1 -- 172.21.15.27:0/1823491665 shutdown_connections 2023-12-19T18:23:51.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:51.857+0000 7fece1afa640 1 -- 172.21.15.27:0/1823491665 wait complete. 2023-12-19T18:23:51.878 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:23:52.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.109+0000 7f030a3fd640 1 Processor -- start 2023-12-19T18:23:52.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.110+0000 7f030a3fd640 1 -- start start 2023-12-19T18:23:52.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.111+0000 7f030a3fd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f030415e9d0 0x7f03040cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:52.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.112+0000 7f030a3fd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0304152070 0x7f03040cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:52.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.112+0000 7f030a3fd640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f030415b3e0 con 0x7f0304151c90 2023-12-19T18:23:52.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.112+0000 7f030a3fd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f030415eed0 con 0x7f030415e9d0 2023-12-19T18:23:52.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.112+0000 7f030a3fd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030415f1b0 con 0x7f0304152070 2023-12-19T18:23:52.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.112+0000 7f0308bfa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f030415e9d0 0x7f03040cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:52.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.112+0000 7f0308bfa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f030415e9d0 0x7f03040cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49902/0 (socket says 172.21.15.27:49902) 2023-12-19T18:23:52.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.112+0000 7f0308bfa640 1 -- 172.21.15.27:0/4213775661 learned_addr learned my addr 172.21.15.27:0/4213775661 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:52.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.112+0000 7f0309bfc640 1 --2- 172.21.15.27:0/4213775661 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0304152070 0x7f03040cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:52.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.113+0000 7f0308bfa640 1 -- 172.21.15.27:0/4213775661 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0304152070 msgr2=0x7f03040cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:52.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.113+0000 7f0308bfa640 1 --2- 172.21.15.27:0/4213775661 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0304152070 0x7f03040cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.113+0000 7f0308bfa640 1 -- 172.21.15.27:0/4213775661 >> v1:172.21.15.192:6789/0 conn(0x7f0304151c90 legacy=0x7f030406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:52.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.113+0000 7f0308bfa640 1 -- 172.21.15.27:0/4213775661 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f030415f490 con 0x7f030415e9d0 2023-12-19T18:23:52.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.114+0000 7f0308bfa640 1 --2- 172.21.15.27:0/4213775661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f030415e9d0 0x7f03040cee40 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f02f40098a0 tx=0x7f02f402b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0eb49498a879501 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:52.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.114+0000 7f02fbfff640 1 -- 172.21.15.27:0/4213775661 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4031640 con 0x7f030415e9d0 2023-12-19T18:23:52.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.115+0000 7f02fbfff640 1 -- 172.21.15.27:0/4213775661 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f40317a0 con 0x7f030415e9d0 2023-12-19T18:23:52.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.115+0000 7f02fbfff640 1 -- 172.21.15.27:0/4213775661 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f4031b30 con 0x7f030415e9d0 2023-12-19T18:23:52.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.115+0000 7f030a3fd640 1 -- 172.21.15.27:0/4213775661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f030415e9d0 msgr2=0x7f03040cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:52.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.115+0000 7f030a3fd640 1 --2- 172.21.15.27:0/4213775661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f030415e9d0 0x7f03040cee40 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f02f40098a0 tx=0x7f02f402b8e0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.115+0000 7f030a3fd640 1 -- 172.21.15.27:0/4213775661 shutdown_connections 2023-12-19T18:23:52.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.115+0000 7f030a3fd640 1 --2- 172.21.15.27:0/4213775661 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0304152070 0x7f03040cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.115+0000 7f030a3fd640 1 --2- 172.21.15.27:0/4213775661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f030415e9d0 0x7f03040cee40 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.115+0000 7f030a3fd640 1 -- 172.21.15.27:0/4213775661 >> 172.21.15.27:0/4213775661 conn(0x7f030406a060 msgr2=0x7f030406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:52.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.116+0000 7f030a3fd640 1 -- 172.21.15.27:0/4213775661 shutdown_connections 2023-12-19T18:23:52.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.116+0000 7f030a3fd640 1 -- 172.21.15.27:0/4213775661 wait complete. 2023-12-19T18:23:52.120 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.117+0000 7f030a3fd640 1 Processor -- start 2023-12-19T18:23:52.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.118+0000 7f030a3fd640 1 -- start start 2023-12-19T18:23:52.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.118+0000 7f030a3fd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0304151c90 0x7f0304152ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:52.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.118+0000 7f030a3fd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0304152070 0x7f03040ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:52.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f030a3fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f030415e9d0 0x7f03040ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:52.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f030a3fd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041f30f0 con 0x7f030415e9d0 2023-12-19T18:23:52.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f030a3fd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03041f2280 con 0x7f0304151c90 2023-12-19T18:23:52.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f030a3fd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03041f2470 con 0x7f0304152070 2023-12-19T18:23:52.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f0309bfc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f030415e9d0 0x7f03040ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:52.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f0309bfc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f030415e9d0 0x7f03040ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49918/0 (socket says 172.21.15.27:49918) 2023-12-19T18:23:52.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f0309bfc640 1 -- 172.21.15.27:0/1547644529 learned_addr learned my addr 172.21.15.27:0/1547644529 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:52.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f03093fb640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0304151c90 0x7f0304152ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:52.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f0308bfa640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0304152070 0x7f03040ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:52.121 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f0309bfc640 1 -- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0304152070 msgr2=0x7f03040ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:52.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f0309bfc640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0304152070 0x7f03040ac8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f0309bfc640 1 -- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0304151c90 msgr2=0x7f0304152ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:52.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f0309bfc640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0304151c90 0x7f0304152ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.119+0000 7f0309bfc640 1 -- 172.21.15.27:0/1547644529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f40094d0 con 0x7f030415e9d0 2023-12-19T18:23:52.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.120+0000 7f0309bfc640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f030415e9d0 0x7f03040ace00 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00efc0 tx=0x7f02fc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:52.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.120+0000 7f02f9ffb640 1 -- 172.21.15.27:0/1547644529 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc009b10 con 0x7f030415e9d0 2023-12-19T18:23:52.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.120+0000 7f02f9ffb640 1 -- 172.21.15.27:0/1547644529 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc00f040 con 0x7f030415e9d0 2023-12-19T18:23:52.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.120+0000 7f02f9ffb640 1 -- 172.21.15.27:0/1547644529 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc0146c0 con 0x7f030415e9d0 2023-12-19T18:23:52.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.120+0000 7f030a3fd640 1 -- 172.21.15.27:0/1547644529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03041f2750 con 0x7f030415e9d0 2023-12-19T18:23:52.122 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.120+0000 7f030a3fd640 1 -- 172.21.15.27:0/1547644529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03041f2d10 con 0x7f030415e9d0 2023-12-19T18:23:52.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.122+0000 7f030a3fd640 1 -- 172.21.15.27:0/1547644529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f030404e840 con 0x7f030415e9d0 2023-12-19T18:23:52.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.122+0000 7f02f9ffb640 1 -- 172.21.15.27:0/1547644529 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f02fc014820 con 0x7f030415e9d0 2023-12-19T18:23:52.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.123+0000 7f02f9ffb640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f02e806a460 0x7f02e806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:52.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.123+0000 7f02f9ffb640 1 -- 172.21.15.27:0/1547644529 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f02fc0894b0 con 0x7f030415e9d0 2023-12-19T18:23:52.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.124+0000 7f02f9ffb640 1 -- 172.21.15.27:0/1547644529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7f02e8072050 con 0x7f030415e9d0 2023-12-19T18:23:52.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.124+0000 7f03093fb640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f02e806a460 0x7f02e806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:52.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.125+0000 7f03093fb640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f02e806a460 0x7f02e806c920 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f03040ad630 tx=0x7f0300009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:52.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.128+0000 7f02f9ffb640 1 -- 172.21.15.27:0/1547644529 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f02fc018060 con 0x7f030415e9d0 2023-12-19T18:23:52.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.319+0000 7f030a3fd640 1 -- 172.21.15.27:0/1547644529 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f03040b1670 con 0x7f030415e9d0 2023-12-19T18:23:52.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.320+0000 7f02f9ffb640 1 -- 172.21.15.27:0/1547644529 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v158) v1 ==== 100+0+11 (secure 0 0 0) 0x7f02fc0549c0 con 0x7f030415e9d0 2023-12-19T18:23:52.320 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:23:52.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.323+0000 7f030a3fd640 1 -- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f02e806a460 msgr2=0x7f02e806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.323+0000 7f030a3fd640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f02e806a460 0x7f02e806c920 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f03040ad630 tx=0x7f0300009040 comp rx=0 tx=0).stop 2023-12-19T18:23:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.323+0000 7f030a3fd640 1 -- 172.21.15.27:0/1547644529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f030415e9d0 msgr2=0x7f03040ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.323+0000 7f030a3fd640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f030415e9d0 0x7f03040ace00 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00efc0 tx=0x7f02fc00c490 comp rx=0 tx=0).stop 2023-12-19T18:23:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.323+0000 7f030a3fd640 1 -- 172.21.15.27:0/1547644529 shutdown_connections 2023-12-19T18:23:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.323+0000 7f030a3fd640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f02e806a460 0x7f02e806c920 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.323+0000 7f030a3fd640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f030415e9d0 0x7f03040ace00 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.323+0000 7f030a3fd640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0304152070 0x7f03040ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.323+0000 7f030a3fd640 1 --2- 172.21.15.27:0/1547644529 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0304151c90 0x7f0304152ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.323+0000 7f030a3fd640 1 -- 172.21.15.27:0/1547644529 >> 172.21.15.27:0/1547644529 conn(0x7f030406a060 msgr2=0x7f0304151160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.323+0000 7f030a3fd640 1 -- 172.21.15.27:0/1547644529 shutdown_connections 2023-12-19T18:23:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.323+0000 7f030a3fd640 1 -- 172.21.15.27:0/1547644529 wait complete. 2023-12-19T18:23:52.344 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:23:52.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.577+0000 7fda53fff640 1 Processor -- start 2023-12-19T18:23:52.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.577+0000 7fda53fff640 1 -- start start 2023-12-19T18:23:52.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.579+0000 7fda53fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fda541553a0 0x7fda540c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:52.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.579+0000 7fda53fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fda540afeb0 0x7fda5415e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:52.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.579+0000 7fda53fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda5415f090 con 0x7fda5415ec60 2023-12-19T18:23:52.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.579+0000 7fda53fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda540cbcf0 con 0x7fda540afeb0 2023-12-19T18:23:52.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.579+0000 7fda53fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda540b0360 con 0x7fda541553a0 2023-12-19T18:23:52.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.580+0000 7fda4a7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fda540afeb0 0x7fda5415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:52.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.580+0000 7fda4a7fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fda540afeb0 0x7fda5415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49932/0 (socket says 172.21.15.27:49932) 2023-12-19T18:23:52.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.580+0000 7fda4a7fc640 1 -- 172.21.15.27:0/357018270 learned_addr learned my addr 172.21.15.27:0/357018270 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:52.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.580+0000 7fda52ffd640 1 --2- 172.21.15.27:0/357018270 >> v2:172.21.15.192:3300/0 conn(0x7fda541553a0 0x7fda540c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:52.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.580+0000 7fda4a7fc640 1 -- 172.21.15.27:0/357018270 >> v2:172.21.15.192:3300/0 conn(0x7fda541553a0 msgr2=0x7fda540c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:52.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.580+0000 7fda4a7fc640 1 --2- 172.21.15.27:0/357018270 >> v2:172.21.15.192:3300/0 conn(0x7fda541553a0 0x7fda540c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.580+0000 7fda4a7fc640 1 -- 172.21.15.27:0/357018270 >> v1:172.21.15.192:6789/0 conn(0x7fda5415ec60 legacy=0x7fda5415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:52.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.580+0000 7fda4a7fc640 1 -- 172.21.15.27:0/357018270 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda540cb350 con 0x7fda540afeb0 2023-12-19T18:23:52.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.580+0000 7fda52ffd640 1 --2- 172.21.15.27:0/357018270 >> v2:172.21.15.192:3300/0 conn(0x7fda541553a0 0x7fda540c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:52.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.581+0000 7fda4a7fc640 1 --2- 172.21.15.27:0/357018270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda540afeb0 0x7fda5415e720 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fda5406d4a0 tx=0x7fda3c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=789d9119fece037b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:52.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.582+0000 7fda527fc640 1 -- 172.21.15.27:0/357018270 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0317e0 con 0x7fda540afeb0 2023-12-19T18:23:52.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.582+0000 7fda527fc640 1 -- 172.21.15.27:0/357018270 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c031940 con 0x7fda540afeb0 2023-12-19T18:23:52.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.582+0000 7fda527fc640 1 -- 172.21.15.27:0/357018270 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c031cd0 con 0x7fda540afeb0 2023-12-19T18:23:52.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.582+0000 7fda53fff640 1 -- 172.21.15.27:0/357018270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda540afeb0 msgr2=0x7fda5415e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:52.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.582+0000 7fda53fff640 1 --2- 172.21.15.27:0/357018270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda540afeb0 0x7fda5415e720 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fda5406d4a0 tx=0x7fda3c02b880 comp rx=0 tx=0).stop 2023-12-19T18:23:52.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.583+0000 7fda53fff640 1 -- 172.21.15.27:0/357018270 shutdown_connections 2023-12-19T18:23:52.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.583+0000 7fda53fff640 1 --2- 172.21.15.27:0/357018270 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda540afeb0 0x7fda5415e720 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.583+0000 7fda53fff640 1 --2- 172.21.15.27:0/357018270 >> v2:172.21.15.192:3300/0 conn(0x7fda541553a0 0x7fda540c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.583+0000 7fda53fff640 1 -- 172.21.15.27:0/357018270 >> 172.21.15.27:0/357018270 conn(0x7fda54069c70 msgr2=0x7fda5406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:52.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.583+0000 7fda53fff640 1 -- 172.21.15.27:0/357018270 shutdown_connections 2023-12-19T18:23:52.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.583+0000 7fda53fff640 1 -- 172.21.15.27:0/357018270 wait complete. 2023-12-19T18:23:52.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.585+0000 7fda53fff640 1 Processor -- start 2023-12-19T18:23:52.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.585+0000 7fda53fff640 1 -- start start 2023-12-19T18:23:52.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda53fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda540afeb0 0x7fda540ae950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:52.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda53fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda541553a0 0x7fda5415a300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:52.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda53fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda5415ec60 0x7fda54147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:52.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda53fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda541513c0 con 0x7fda541553a0 2023-12-19T18:23:52.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda53fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda541516a0 con 0x7fda5415ec60 2023-12-19T18:23:52.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda53fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda54151980 con 0x7fda540afeb0 2023-12-19T18:23:52.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda4a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda541553a0 0x7fda5415a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:52.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda4a7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda541553a0 0x7fda5415a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49936/0 (socket says 172.21.15.27:49936) 2023-12-19T18:23:52.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda4a7fc640 1 -- 172.21.15.27:0/2908527933 learned_addr learned my addr 172.21.15.27:0/2908527933 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:52.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda537fe640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda5415ec60 0x7fda54147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:52.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda52ffd640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda540afeb0 0x7fda540ae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:52.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda4a7fc640 1 -- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda540afeb0 msgr2=0x7fda540ae950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:52.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda4a7fc640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda540afeb0 0x7fda540ae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda4a7fc640 1 -- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda5415ec60 msgr2=0x7fda54147fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:52.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.586+0000 7fda4a7fc640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda5415ec60 0x7fda54147fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.587+0000 7fda4a7fc640 1 -- 172.21.15.27:0/2908527933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c0094d0 con 0x7fda541553a0 2023-12-19T18:23:52.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.587+0000 7fda537fe640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda5415ec60 0x7fda54147fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:52.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.587+0000 7fda4a7fc640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda541553a0 0x7fda5415a300 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fda3c02b8c0 tx=0x7fda3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:52.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.587+0000 7fda50ff9640 1 -- 172.21.15.27:0/2908527933 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0329c0 con 0x7fda541553a0 2023-12-19T18:23:52.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.587+0000 7fda50ff9640 1 -- 172.21.15.27:0/2908527933 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c032b20 con 0x7fda541553a0 2023-12-19T18:23:52.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.587+0000 7fda50ff9640 1 -- 172.21.15.27:0/2908527933 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0453f0 con 0x7fda541553a0 2023-12-19T18:23:52.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.587+0000 7fda53fff640 1 -- 172.21.15.27:0/2908527933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda54151c60 con 0x7fda541553a0 2023-12-19T18:23:52.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.587+0000 7fda53fff640 1 -- 172.21.15.27:0/2908527933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda540aee90 con 0x7fda541553a0 2023-12-19T18:23:52.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.588+0000 7fda53fff640 1 -- 172.21.15.27:0/2908527933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda54152180 con 0x7fda541553a0 2023-12-19T18:23:52.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.589+0000 7fda50ff9640 1 -- 172.21.15.27:0/2908527933 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fda3c04e450 con 0x7fda541553a0 2023-12-19T18:23:52.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.590+0000 7fda50ff9640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda3406a430 0x7fda3406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:52.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.590+0000 7fda50ff9640 1 -- 172.21.15.27:0/2908527933 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fda3c0aacb0 con 0x7fda541553a0 2023-12-19T18:23:52.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.591+0000 7fda50ff9640 1 -- 172.21.15.27:0/2908527933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7fda34072020 con 0x7fda541553a0 2023-12-19T18:23:52.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.591+0000 7fda52ffd640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda3406a430 0x7fda3406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:52.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.592+0000 7fda52ffd640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda3406a430 0x7fda3406c8f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fda44008850 tx=0x7fda44008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:52.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.595+0000 7fda50ff9640 1 -- 172.21.15.27:0/2908527933 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fda54152180 con 0x7fda541553a0 2023-12-19T18:23:52.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.826+0000 7fda53fff640 1 -- 172.21.15.27:0/2908527933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fda54152180 con 0x7fda541553a0 2023-12-19T18:23:52.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.827+0000 7fda50ff9640 1 -- 172.21.15.27:0/2908527933 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fda54152180 con 0x7fda541553a0 2023-12-19T18:23:52.827 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:52.827 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":61,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3298228307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3298228307},{"type":"v1","addr":"172.21.15.192:6835","nonce":3298228307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5726,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1760483999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1760483999},{"type":"v1","addr":"172.21.15.192:6837","nonce":1760483999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3000637822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3000637822},{"type":"v1","addr":"172.21.15.27:6833","nonce":3000637822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:23:06.465885+0000","modified":"2023-12-19T18:23:09.973228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1593995318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1593995318},{"type":"v1","addr":"172.21.15.27:6835","nonce":1593995318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T18:23:52.827 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 61 2023-12-19T18:23:52.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.830+0000 7fda53fff640 1 -- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda3406a430 msgr2=0x7fda3406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:52.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.830+0000 7fda53fff640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda3406a430 0x7fda3406c8f0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fda44008850 tx=0x7fda44008040 comp rx=0 tx=0).stop 2023-12-19T18:23:52.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.830+0000 7fda53fff640 1 -- 172.21.15.27:0/2908527933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda541553a0 msgr2=0x7fda5415a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:52.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.830+0000 7fda53fff640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda541553a0 0x7fda5415a300 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fda3c02b8c0 tx=0x7fda3c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.830+0000 7fda53fff640 1 -- 172.21.15.27:0/2908527933 shutdown_connections 2023-12-19T18:23:52.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.830+0000 7fda53fff640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fda3406a430 0x7fda3406c8f0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.830+0000 7fda53fff640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fda5415ec60 0x7fda54147fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.830+0000 7fda53fff640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda541553a0 0x7fda5415a300 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.830+0000 7fda53fff640 1 --2- 172.21.15.27:0/2908527933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fda540afeb0 0x7fda540ae950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:52.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.830+0000 7fda53fff640 1 -- 172.21.15.27:0/2908527933 >> 172.21.15.27:0/2908527933 conn(0x7fda54069c70 msgr2=0x7fda54159680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:52.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.830+0000 7fda53fff640 1 -- 172.21.15.27:0/2908527933 shutdown_connections 2023-12-19T18:23:52.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:52.830+0000 7fda53fff640 1 -- 172.21.15.27:0/2908527933 wait complete. 2023-12-19T18:23:52.851 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:23:53.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.088+0000 7f5103ec4640 1 Processor -- start 2023-12-19T18:23:53.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.088+0000 7f5103ec4640 1 -- start start 2023-12-19T18:23:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.090+0000 7f5103ec4640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f50fc0cbce0 0x7f50fc0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.090+0000 7f5103ec4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50fc0cb850 con 0x7f50fc0adc20 2023-12-19T18:23:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.090+0000 7f5103ec4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50fc15b350 con 0x7f50fc15af70 2023-12-19T18:23:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.090+0000 7f5103ec4640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50fc0cac40 con 0x7f50fc0cbce0 2023-12-19T18:23:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.090+0000 7f5102461640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f50fc0adc20 0x7f50fc06b4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34854/0 (socket says 172.21.15.27:34854) 2023-12-19T18:23:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.090+0000 7f5102461640 1 -- 172.21.15.27:0/3142405031 learned_addr learned my addr 172.21.15.27:0/3142405031 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:53.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.090+0000 7f510145f640 1 --2- 172.21.15.27:0/3142405031 >> v2:172.21.15.27:3300/0 conn(0x7f50fc0cbce0 0x7f50fc0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.090+0000 7f5100c5e640 1 -- 172.21.15.27:0/3142405031 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165199819 0 0) 0x7f50fc0cb850 con 0x7f50fc0adc20 2023-12-19T18:23:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.091+0000 7f5100c5e640 1 -- 172.21.15.27:0/3142405031 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f50e4003520 con 0x7f50fc0adc20 2023-12-19T18:23:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.091+0000 7f510145f640 1 -- 172.21.15.27:0/3142405031 >> v1:172.21.15.192:6789/0 conn(0x7f50fc15af70 legacy=0x7f50fc0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.091+0000 7f510145f640 1 -- 172.21.15.27:0/3142405031 >> v1:172.21.15.27:6789/0 conn(0x7f50fc0adc20 legacy=0x7f50fc06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.091+0000 7f510145f640 1 -- 172.21.15.27:0/3142405031 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc0caf20 con 0x7f50fc0cbce0 2023-12-19T18:23:53.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.092+0000 7f510145f640 1 --2- 172.21.15.27:0/3142405031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50fc0cbce0 0x7f50fc0ca540 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f50f00097a0 tx=0x7f50f002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aea33c96033305af server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.092+0000 7f5100c5e640 1 -- 172.21.15.27:0/3142405031 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0030610 con 0x7f50fc0cbce0 2023-12-19T18:23:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.092+0000 7f5100c5e640 1 -- 172.21.15.27:0/3142405031 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f0030770 con 0x7f50fc0cbce0 2023-12-19T18:23:53.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.092+0000 7f5100c5e640 1 -- 172.21.15.27:0/3142405031 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0030b00 con 0x7f50fc0cbce0 2023-12-19T18:23:53.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.093+0000 7f5103ec4640 1 -- 172.21.15.27:0/3142405031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50fc0cbce0 msgr2=0x7f50fc0ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:53.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.093+0000 7f5103ec4640 1 --2- 172.21.15.27:0/3142405031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50fc0cbce0 0x7f50fc0ca540 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f50f00097a0 tx=0x7f50f002b960 comp rx=0 tx=0).stop 2023-12-19T18:23:53.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.093+0000 7f5103ec4640 1 -- 172.21.15.27:0/3142405031 shutdown_connections 2023-12-19T18:23:53.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.093+0000 7f5103ec4640 1 --2- 172.21.15.27:0/3142405031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50fc0cbce0 0x7f50fc0ca540 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.093+0000 7f5103ec4640 1 -- 172.21.15.27:0/3142405031 >> 172.21.15.27:0/3142405031 conn(0x7f50fc06a330 msgr2=0x7f50fc0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:53.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.093+0000 7f5103ec4640 1 -- 172.21.15.27:0/3142405031 shutdown_connections 2023-12-19T18:23:53.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.093+0000 7f5103ec4640 1 -- 172.21.15.27:0/3142405031 wait complete. 2023-12-19T18:23:53.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.095+0000 7f5103ec4640 1 Processor -- start 2023-12-19T18:23:53.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.095+0000 7f5103ec4640 1 -- start start 2023-12-19T18:23:53.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.096+0000 7f5103ec4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50fc0adc20 0x7f50fc0b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:53.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.096+0000 7f5103ec4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50fc0cbce0 0x7f50fc155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:53.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.096+0000 7f5103ec4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50fc15af70 0x7f50fc154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:53.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.096+0000 7f5103ec4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1571c0 con 0x7f50fc0cbce0 2023-12-19T18:23:53.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.096+0000 7f5103ec4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc0b3150 con 0x7f50fc15af70 2023-12-19T18:23:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.096+0000 7f5103ec4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc0b3430 con 0x7f50fc0adc20 2023-12-19T18:23:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.096+0000 7f510145f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50fc0cbce0 0x7f50fc155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.097+0000 7f510145f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50fc0cbce0 0x7f50fc155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49968/0 (socket says 172.21.15.27:49968) 2023-12-19T18:23:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.097+0000 7f510145f640 1 -- 172.21.15.27:0/724934365 learned_addr learned my addr 172.21.15.27:0/724934365 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.097+0000 7f510145f640 1 -- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50fc0adc20 msgr2=0x7f50fc0b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.097+0000 7f5101c60640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50fc0adc20 0x7f50fc0b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.097+0000 7f5102461640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50fc15af70 0x7f50fc154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.097+0000 7f510145f640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50fc0adc20 0x7f50fc0b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.097+0000 7f510145f640 1 -- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50fc15af70 msgr2=0x7f50fc154bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.097+0000 7f510145f640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50fc15af70 0x7f50fc154bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.097+0000 7f510145f640 1 -- 172.21.15.27:0/724934365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f0009400 con 0x7f50fc0cbce0 2023-12-19T18:23:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.097+0000 7f5102461640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50fc15af70 0x7f50fc154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:53.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.097+0000 7f510145f640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50fc0cbce0 0x7f50fc155740 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f50f002bf70 tx=0x7f50f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:53.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.098+0000 7f50fa7fc640 1 -- 172.21.15.27:0/724934365 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f003e040 con 0x7f50fc0cbce0 2023-12-19T18:23:53.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.098+0000 7f50fa7fc640 1 -- 172.21.15.27:0/724934365 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f0039cb0 con 0x7f50fc0cbce0 2023-12-19T18:23:53.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.098+0000 7f50fa7fc640 1 -- 172.21.15.27:0/724934365 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f0044630 con 0x7f50fc0cbce0 2023-12-19T18:23:53.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.098+0000 7f5103ec4640 1 -- 172.21.15.27:0/724934365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50fc157460 con 0x7f50fc0cbce0 2023-12-19T18:23:53.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.098+0000 7f5103ec4640 1 -- 172.21.15.27:0/724934365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50fc15cd70 con 0x7f50fc0cbce0 2023-12-19T18:23:53.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.099+0000 7f5103ec4640 1 -- 172.21.15.27:0/724934365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50fc157980 con 0x7f50fc0cbce0 2023-12-19T18:23:53.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.100+0000 7f50fa7fc640 1 -- 172.21.15.27:0/724934365 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f50f0043390 con 0x7f50fc0cbce0 2023-12-19T18:23:53.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.101+0000 7f50fa7fc640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f50e806a430 0x7f50e806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:53.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.101+0000 7f50fa7fc640 1 -- 172.21.15.27:0/724934365 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f50f00aaac0 con 0x7f50fc0cbce0 2023-12-19T18:23:53.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.101+0000 7f50fa7fc640 1 -- 172.21.15.27:0/724934365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7f50e8072020 con 0x7f50fc0cbce0 2023-12-19T18:23:53.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.102+0000 7f5101c60640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f50e806a430 0x7f50e806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:53.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.103+0000 7f5101c60640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f50e806a430 0x7f50e806c8f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f50ec007fd0 tx=0x7f50ec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:53.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.105+0000 7f50fa7fc640 1 -- 172.21.15.27:0/724934365 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f50fc157980 con 0x7f50fc0cbce0 2023-12-19T18:23:53.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.299+0000 7f5103ec4640 1 -- 172.21.15.27:0/724934365 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50fc157980 con 0x7f50fc0cbce0 2023-12-19T18:23:53.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.302+0000 7f50fa7fc640 1 -- 172.21.15.27:0/724934365 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16239 (secure 0 0 0) 0x7f50fc157980 con 0x7f50fc0cbce0 2023-12-19T18:23:53.302 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:53.302 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":158,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:23:46.716988+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:23:04.431250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T18:23:05.506926+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":4,"to":3}]},{"pgid":"17.f","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:23:53.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.304+0000 7f5103ec4640 1 -- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f50e806a430 msgr2=0x7f50e806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:53.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.304+0000 7f5103ec4640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f50e806a430 0x7f50e806c8f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f50ec007fd0 tx=0x7f50ec009040 comp rx=0 tx=0).stop 2023-12-19T18:23:53.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.304+0000 7f5103ec4640 1 -- 172.21.15.27:0/724934365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50fc0cbce0 msgr2=0x7f50fc155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:53.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.304+0000 7f5103ec4640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50fc0cbce0 0x7f50fc155740 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f50f002bf70 tx=0x7f50f0004730 comp rx=0 tx=0).stop 2023-12-19T18:23:53.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.305+0000 7f5103ec4640 1 -- 172.21.15.27:0/724934365 shutdown_connections 2023-12-19T18:23:53.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.305+0000 7f5103ec4640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f50e806a430 0x7f50e806c8f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.305+0000 7f5103ec4640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f50fc15af70 0x7f50fc154bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.305+0000 7f5103ec4640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50fc0cbce0 0x7f50fc155740 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.305+0000 7f5103ec4640 1 --2- 172.21.15.27:0/724934365 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f50fc0adc20 0x7f50fc0b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.305+0000 7f5103ec4640 1 -- 172.21.15.27:0/724934365 >> 172.21.15.27:0/724934365 conn(0x7f50fc06a330 msgr2=0x7f50fc0ceab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:53.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.305+0000 7f5103ec4640 1 -- 172.21.15.27:0/724934365 shutdown_connections 2023-12-19T18:23:53.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.305+0000 7f5103ec4640 1 -- 172.21.15.27:0/724934365 wait complete. 2023-12-19T18:23:53.323 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:23:53.324 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:23:53.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.560+0000 7f0e1a2fa640 1 Processor -- start 2023-12-19T18:23:53.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.560+0000 7f0e1a2fa640 1 -- start start 2023-12-19T18:23:53.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.561+0000 7f0e1a2fa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0e140b4ef0 0x7f0e140b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:53.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.561+0000 7f0e1a2fa640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0e140d2350 0x7f0e140b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:53.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.561+0000 7f0e1a2fa640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e140b5e90 con 0x7f0e141604d0 2023-12-19T18:23:53.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.562+0000 7f0e1a2fa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e140d2b30 con 0x7f0e140b4ef0 2023-12-19T18:23:53.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.562+0000 7f0e1a2fa640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e140d2850 con 0x7f0e140d2350 2023-12-19T18:23:53.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.562+0000 7f0e192f8640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0e141604d0 0x7f0e140b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34870/0 (socket says 172.21.15.27:34870) 2023-12-19T18:23:53.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.562+0000 7f0e192f8640 1 -- 172.21.15.27:0/1149545507 learned_addr learned my addr 172.21.15.27:0/1149545507 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:53.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.562+0000 7f0e18af7640 1 --2- 172.21.15.27:0/1149545507 >> v2:172.21.15.27:3300/0 conn(0x7f0e140b4ef0 0x7f0e140b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:53.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.562+0000 7f0e19af9640 1 --2- 172.21.15.27:0/1149545507 >> v2:172.21.15.192:3300/0 conn(0x7f0e140d2350 0x7f0e140b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:53.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.562+0000 7f0e13fff640 1 -- 172.21.15.27:0/1149545507 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3648359714 0 0) 0x7f0e140b5e90 con 0x7f0e141604d0 2023-12-19T18:23:53.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.562+0000 7f0e13fff640 1 -- 172.21.15.27:0/1149545507 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0dfc003520 con 0x7f0e141604d0 2023-12-19T18:23:53.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.562+0000 7f0e18af7640 1 -- 172.21.15.27:0/1149545507 >> v2:172.21.15.192:3300/0 conn(0x7f0e140d2350 msgr2=0x7f0e140b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:53.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.562+0000 7f0e18af7640 1 --2- 172.21.15.27:0/1149545507 >> v2:172.21.15.192:3300/0 conn(0x7f0e140d2350 0x7f0e140b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.562+0000 7f0e18af7640 1 -- 172.21.15.27:0/1149545507 >> v1:172.21.15.27:6789/0 conn(0x7f0e141604d0 legacy=0x7f0e140b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:53.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.562+0000 7f0e18af7640 1 -- 172.21.15.27:0/1149545507 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e140cded0 con 0x7f0e140b4ef0 2023-12-19T18:23:53.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.564+0000 7f0e18af7640 1 --2- 172.21.15.27:0/1149545507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e140b4ef0 0x7f0e140b6300 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f0e14071e80 tx=0x7f0e0802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b02f47661f369765 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:53.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.564+0000 7f0e13fff640 1 -- 172.21.15.27:0/1149545507 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08031830 con 0x7f0e140b4ef0 2023-12-19T18:23:53.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.564+0000 7f0e13fff640 1 -- 172.21.15.27:0/1149545507 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e08031990 con 0x7f0e140b4ef0 2023-12-19T18:23:53.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.564+0000 7f0e13fff640 1 -- 172.21.15.27:0/1149545507 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e08031d20 con 0x7f0e140b4ef0 2023-12-19T18:23:53.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.565+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/1149545507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e140b4ef0 msgr2=0x7f0e140b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:53.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.565+0000 7f0e1a2fa640 1 --2- 172.21.15.27:0/1149545507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e140b4ef0 0x7f0e140b6300 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f0e14071e80 tx=0x7f0e0802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.565+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/1149545507 shutdown_connections 2023-12-19T18:23:53.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.565+0000 7f0e1a2fa640 1 --2- 172.21.15.27:0/1149545507 >> v2:172.21.15.192:3300/0 conn(0x7f0e140d2350 0x7f0e140b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.565+0000 7f0e1a2fa640 1 --2- 172.21.15.27:0/1149545507 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e140b4ef0 0x7f0e140b6300 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.565+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/1149545507 >> 172.21.15.27:0/1149545507 conn(0x7f0e140cf7e0 msgr2=0x7f0e140cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:53.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.565+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/1149545507 shutdown_connections 2023-12-19T18:23:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.565+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/1149545507 wait complete. 2023-12-19T18:23:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.566+0000 7f0e1a2fa640 1 Processor -- start 2023-12-19T18:23:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.567+0000 7f0e1a2fa640 1 -- start start 2023-12-19T18:23:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e1a2fa640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e140b4ef0 0x7f0e141f2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e1a2fa640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e140d2350 0x7f0e140c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e1a2fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e141604d0 0x7f0e1415ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e1a2fa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e14142ef0 con 0x7f0e141604d0 2023-12-19T18:23:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e1a2fa640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e141431d0 con 0x7f0e140b4ef0 2023-12-19T18:23:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e1a2fa640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140c8f30 con 0x7f0e140d2350 2023-12-19T18:23:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e19af9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e141604d0 0x7f0e1415ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e19af9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e141604d0 0x7f0e1415ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:49998/0 (socket says 172.21.15.27:49998) 2023-12-19T18:23:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e19af9640 1 -- 172.21.15.27:0/3470576916 learned_addr learned my addr 172.21.15.27:0/3470576916 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e192f8640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e140b4ef0 0x7f0e141f2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e18af7640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e140d2350 0x7f0e140c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e19af9640 1 -- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e140d2350 msgr2=0x7f0e140c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e19af9640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e140d2350 0x7f0e140c84e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e19af9640 1 -- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e140b4ef0 msgr2=0x7f0e141f2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e19af9640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e140b4ef0 0x7f0e141f2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e19af9640 1 -- 172.21.15.27:0/3470576916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e08009400 con 0x7f0e141604d0 2023-12-19T18:23:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.568+0000 7f0e192f8640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e140b4ef0 0x7f0e141f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.569+0000 7f0e19af9640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e141604d0 0x7f0e1415ce50 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00d830 tx=0x7f0e0c00dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.569+0000 7f0e11ffb640 1 -- 172.21.15.27:0/3470576916 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00b6c0 con 0x7f0e141604d0 2023-12-19T18:23:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.569+0000 7f0e11ffb640 1 -- 172.21.15.27:0/3470576916 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c00b840 con 0x7f0e141604d0 2023-12-19T18:23:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.569+0000 7f0e11ffb640 1 -- 172.21.15.27:0/3470576916 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00bbf0 con 0x7f0e141604d0 2023-12-19T18:23:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.569+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/3470576916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e140c9210 con 0x7f0e141604d0 2023-12-19T18:23:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.569+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/3470576916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e140c6950 con 0x7f0e141604d0 2023-12-19T18:23:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.570+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/3470576916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e140c9730 con 0x7f0e141604d0 2023-12-19T18:23:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.571+0000 7f0e11ffb640 1 -- 172.21.15.27:0/3470576916 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0e0c01e460 con 0x7f0e141604d0 2023-12-19T18:23:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.572+0000 7f0e11ffb640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0e0006a430 0x7f0e0006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.572+0000 7f0e11ffb640 1 -- 172.21.15.27:0/3470576916 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0e0c089060 con 0x7f0e141604d0 2023-12-19T18:23:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.572+0000 7f0e11ffb640 1 -- 172.21.15.27:0/3470576916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7f0e00072020 con 0x7f0e141604d0 2023-12-19T18:23:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.573+0000 7f0e192f8640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0e0006a430 0x7f0e0006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.574+0000 7f0e192f8640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0e0006a430 0x7f0e0006c8f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0e04001560 tx=0x7f0e04009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:53.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.576+0000 7f0e11ffb640 1 -- 172.21.15.27:0/3470576916 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0e0c0546a0 con 0x7f0e141604d0 2023-12-19T18:23:53.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.806+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/3470576916 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0e14066f80 con 0x7f0e141604d0 2023-12-19T18:23:53.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.808+0000 7f0e11ffb640 1 -- 172.21.15.27:0/3470576916 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f0e0c057cc0 con 0x7f0e141604d0 2023-12-19T18:23:53.808 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:53.808 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":61,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3298228307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3298228307},{"type":"v1","addr":"172.21.15.192:6835","nonce":3298228307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5726,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1760483999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1760483999},{"type":"v1","addr":"172.21.15.192:6837","nonce":1760483999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3000637822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3000637822},{"type":"v1","addr":"172.21.15.27:6833","nonce":3000637822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:23:06.465885+0000","modified":"2023-12-19T18:23:09.973228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1593995318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1593995318},{"type":"v1","addr":"172.21.15.27:6835","nonce":1593995318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T18:23:53.808 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 61 2023-12-19T18:23:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.810+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0e0006a430 msgr2=0x7f0e0006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:53.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.810+0000 7f0e1a2fa640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0e0006a430 0x7f0e0006c8f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0e04001560 tx=0x7f0e04009040 comp rx=0 tx=0).stop 2023-12-19T18:23:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.810+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/3470576916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e141604d0 msgr2=0x7f0e1415ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.810+0000 7f0e1a2fa640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e141604d0 0x7f0e1415ce50 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c00d830 tx=0x7f0e0c00dcf0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.810+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/3470576916 shutdown_connections 2023-12-19T18:23:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.810+0000 7f0e1a2fa640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0e0006a430 0x7f0e0006c8f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.810+0000 7f0e1a2fa640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e141604d0 0x7f0e1415ce50 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.810+0000 7f0e1a2fa640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0e140d2350 0x7f0e140c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.810+0000 7f0e1a2fa640 1 --2- 172.21.15.27:0/3470576916 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0e140b4ef0 0x7f0e141f2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.810+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/3470576916 >> 172.21.15.27:0/3470576916 conn(0x7f0e140cf7e0 msgr2=0x7f0e140b2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.810+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/3470576916 shutdown_connections 2023-12-19T18:23:53.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:53.810+0000 7f0e1a2fa640 1 -- 172.21.15.27:0/3470576916 wait complete. 2023-12-19T18:23:53.829 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:23:54.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.051+0000 7fc336cbe640 1 Processor -- start 2023-12-19T18:23:54.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.051+0000 7fc336cbe640 1 -- start start 2023-12-19T18:23:54.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.053+0000 7fc336cbe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc3301553a0 0x7fc3300c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:54.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.053+0000 7fc336cbe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3300afeb0 0x7fc33015e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:54.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.053+0000 7fc336cbe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc33015ec60 0x7fc33015b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:54.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.053+0000 7fc336cbe640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3300cbcf0 con 0x7fc3301553a0 2023-12-19T18:23:54.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.053+0000 7fc336cbe640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc33015f0d0 con 0x7fc33015ec60 2023-12-19T18:23:54.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.053+0000 7fc336cbe640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3300b0330 con 0x7fc3300afeb0 2023-12-19T18:23:54.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.053+0000 7fc335cbc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc3301553a0 0x7fc3300c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:54.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.053+0000 7fc335cbc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc3301553a0 0x7fc3300c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50012/0 (socket says 172.21.15.27:50012) 2023-12-19T18:23:54.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.053+0000 7fc335cbc640 1 -- 172.21.15.27:0/210162521 learned_addr learned my addr 172.21.15.27:0/210162521 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:54.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.054+0000 7fc3354bb640 1 --2- 172.21.15.27:0/210162521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3300afeb0 0x7fc33015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.054+0000 7fc3364bd640 1 --2- 172.21.15.27:0/210162521 >> v2:172.21.15.192:3300/0 conn(0x7fc33015ec60 0x7fc33015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.054+0000 7fc335cbc640 1 -- 172.21.15.27:0/210162521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3300afeb0 msgr2=0x7fc33015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.054+0000 7fc335cbc640 1 --2- 172.21.15.27:0/210162521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3300afeb0 0x7fc33015e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.054+0000 7fc335cbc640 1 -- 172.21.15.27:0/210162521 >> v2:172.21.15.192:3300/0 conn(0x7fc33015ec60 msgr2=0x7fc33015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.054+0000 7fc335cbc640 1 --2- 172.21.15.27:0/210162521 >> v2:172.21.15.192:3300/0 conn(0x7fc33015ec60 0x7fc33015b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.054+0000 7fc335cbc640 1 -- 172.21.15.27:0/210162521 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3300cb350 con 0x7fc3301553a0 2023-12-19T18:23:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.054+0000 7fc3354bb640 1 --2- 172.21.15.27:0/210162521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3300afeb0 0x7fc33015e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:54.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.054+0000 7fc3364bd640 1 --2- 172.21.15.27:0/210162521 >> v2:172.21.15.192:3300/0 conn(0x7fc33015ec60 0x7fc33015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.055+0000 7fc335cbc640 1 --2- 172.21.15.27:0/210162521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3301553a0 0x7fc3300c50a0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fc33006f2d0 tx=0x7fc32c02b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b2e67625a704c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.055+0000 7fc334cba640 1 -- 172.21.15.27:0/210162521 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0317c0 con 0x7fc3301553a0 2023-12-19T18:23:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.056+0000 7fc334cba640 1 -- 172.21.15.27:0/210162521 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c031920 con 0x7fc3301553a0 2023-12-19T18:23:54.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.056+0000 7fc334cba640 1 -- 172.21.15.27:0/210162521 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c031cb0 con 0x7fc3301553a0 2023-12-19T18:23:54.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.056+0000 7fc336cbe640 1 -- 172.21.15.27:0/210162521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3301553a0 msgr2=0x7fc3300c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:54.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.056+0000 7fc336cbe640 1 --2- 172.21.15.27:0/210162521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3301553a0 0x7fc3300c50a0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7fc33006f2d0 tx=0x7fc32c02b860 comp rx=0 tx=0).stop 2023-12-19T18:23:54.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.056+0000 7fc336cbe640 1 -- 172.21.15.27:0/210162521 shutdown_connections 2023-12-19T18:23:54.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.056+0000 7fc336cbe640 1 --2- 172.21.15.27:0/210162521 >> v2:172.21.15.192:3300/0 conn(0x7fc33015ec60 0x7fc33015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.056+0000 7fc336cbe640 1 --2- 172.21.15.27:0/210162521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc3300afeb0 0x7fc33015e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.056+0000 7fc336cbe640 1 --2- 172.21.15.27:0/210162521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3301553a0 0x7fc3300c50a0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.056+0000 7fc336cbe640 1 -- 172.21.15.27:0/210162521 >> 172.21.15.27:0/210162521 conn(0x7fc330069c70 msgr2=0x7fc33006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:54.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.056+0000 7fc336cbe640 1 -- 172.21.15.27:0/210162521 shutdown_connections 2023-12-19T18:23:54.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.056+0000 7fc336cbe640 1 -- 172.21.15.27:0/210162521 wait complete. 2023-12-19T18:23:54.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.058+0000 7fc336cbe640 1 Processor -- start 2023-12-19T18:23:54.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.058+0000 7fc336cbe640 1 -- start start 2023-12-19T18:23:54.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.059+0000 7fc336cbe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3300afeb0 0x7fc3301f20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:54.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.059+0000 7fc336cbe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3301553a0 0x7fc33014bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:54.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.059+0000 7fc336cbe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc33015ec60 0x7fc3300c7810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:54.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.059+0000 7fc336cbe640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc330147eb0 con 0x7fc3300afeb0 2023-12-19T18:23:54.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.059+0000 7fc336cbe640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3301f6010 con 0x7fc3301553a0 2023-12-19T18:23:54.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.059+0000 7fc336cbe640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3301f62f0 con 0x7fc33015ec60 2023-12-19T18:23:54.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.059+0000 7fc335cbc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3300afeb0 0x7fc3301f20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:54.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.059+0000 7fc335cbc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3300afeb0 0x7fc3301f20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50024/0 (socket says 172.21.15.27:50024) 2023-12-19T18:23:54.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.059+0000 7fc335cbc640 1 -- 172.21.15.27:0/3945941540 learned_addr learned my addr 172.21.15.27:0/3945941540 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:54.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.060+0000 7fc3354bb640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3301553a0 0x7fc33014bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:54.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.060+0000 7fc3364bd640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc33015ec60 0x7fc3300c7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:54.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.060+0000 7fc335cbc640 1 -- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc33015ec60 msgr2=0x7fc3300c7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:54.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.060+0000 7fc335cbc640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc33015ec60 0x7fc3300c7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.060+0000 7fc335cbc640 1 -- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3301553a0 msgr2=0x7fc33014bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:54.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.060+0000 7fc335cbc640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3301553a0 0x7fc33014bec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.060+0000 7fc335cbc640 1 -- 172.21.15.27:0/3945941540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc32c0094d0 con 0x7fc3300afeb0 2023-12-19T18:23:54.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.060+0000 7fc335cbc640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3300afeb0 0x7fc3301f20b0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc32c02b8a0 tx=0x7fc32c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:54.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.060+0000 7fc3267fc640 1 -- 172.21.15.27:0/3945941540 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0329a0 con 0x7fc3300afeb0 2023-12-19T18:23:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.061+0000 7fc3267fc640 1 -- 172.21.15.27:0/3945941540 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32c032b40 con 0x7fc3300afeb0 2023-12-19T18:23:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.061+0000 7fc3267fc640 1 -- 172.21.15.27:0/3945941540 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32c0453f0 con 0x7fc3300afeb0 2023-12-19T18:23:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.061+0000 7fc336cbe640 1 -- 172.21.15.27:0/3945941540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3301f65d0 con 0x7fc3300afeb0 2023-12-19T18:23:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.061+0000 7fc336cbe640 1 -- 172.21.15.27:0/3945941540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33014bc10 con 0x7fc3300afeb0 2023-12-19T18:23:54.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.062+0000 7fc336cbe640 1 -- 172.21.15.27:0/3945941540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc330148020 con 0x7fc3300afeb0 2023-12-19T18:23:54.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.063+0000 7fc3267fc640 1 -- 172.21.15.27:0/3945941540 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc32c04e450 con 0x7fc3300afeb0 2023-12-19T18:23:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.064+0000 7fc3267fc640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc31406a460 0x7fc31406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.064+0000 7fc3267fc640 1 -- 172.21.15.27:0/3945941540 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fc32c0aac10 con 0x7fc3300afeb0 2023-12-19T18:23:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.064+0000 7fc3267fc640 1 -- 172.21.15.27:0/3945941540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7fc314072050 con 0x7fc3300afeb0 2023-12-19T18:23:54.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.064+0000 7fc3354bb640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc31406a460 0x7fc31406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:54.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.065+0000 7fc3354bb640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc31406a460 0x7fc31406c920 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc320006fd0 tx=0x7fc320008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:54.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.069+0000 7fc3267fc640 1 -- 172.21.15.27:0/3945941540 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc32c076230 con 0x7fc3300afeb0 2023-12-19T18:23:54.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.300+0000 7fc336cbe640 1 -- 172.21.15.27:0/3945941540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc33004e840 con 0x7fc3300afeb0 2023-12-19T18:23:54.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.301+0000 7fc3267fc640 1 -- 172.21.15.27:0/3945941540 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fc32c079850 con 0x7fc3300afeb0 2023-12-19T18:23:54.302 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:54.302 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":61,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3298228307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3298228307},{"type":"v1","addr":"172.21.15.192:6835","nonce":3298228307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5726,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1760483999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1760483999},{"type":"v1","addr":"172.21.15.192:6837","nonce":1760483999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3000637822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3000637822},{"type":"v1","addr":"172.21.15.27:6833","nonce":3000637822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":61,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:23:06.465885+0000","modified":"2023-12-19T18:23:09.973228+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7794},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7794":{"gid":7794,"name":"a","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/1593995318","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1593995318},{"type":"v1","addr":"172.21.15.27:6835","nonce":1593995318}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-19T18:23:54.302 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 61 2023-12-19T18:23:54.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.304+0000 7fc336cbe640 1 -- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc31406a460 msgr2=0x7fc31406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:54.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.304+0000 7fc336cbe640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc31406a460 0x7fc31406c920 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fc320006fd0 tx=0x7fc320008040 comp rx=0 tx=0).stop 2023-12-19T18:23:54.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.304+0000 7fc336cbe640 1 -- 172.21.15.27:0/3945941540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3300afeb0 msgr2=0x7fc3301f20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:54.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.304+0000 7fc336cbe640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3300afeb0 0x7fc3301f20b0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7fc32c02b8a0 tx=0x7fc32c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.304+0000 7fc336cbe640 1 -- 172.21.15.27:0/3945941540 shutdown_connections 2023-12-19T18:23:54.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.304+0000 7fc336cbe640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc31406a460 0x7fc31406c920 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.304+0000 7fc336cbe640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc33015ec60 0x7fc3300c7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.304+0000 7fc336cbe640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc3301553a0 0x7fc33014bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.304+0000 7fc336cbe640 1 --2- 172.21.15.27:0/3945941540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3300afeb0 0x7fc3301f20b0 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.304+0000 7fc336cbe640 1 -- 172.21.15.27:0/3945941540 >> 172.21.15.27:0/3945941540 conn(0x7fc330069c70 msgr2=0x7fc3300ca760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:54.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.304+0000 7fc336cbe640 1 -- 172.21.15.27:0/3945941540 shutdown_connections 2023-12-19T18:23:54.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.304+0000 7fc336cbe640 1 -- 172.21.15.27:0/3945941540 wait complete. 2023-12-19T18:23:54.323 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:23:54.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.564+0000 7f63fffff640 1 Processor -- start 2023-12-19T18:23:54.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.565+0000 7f63fffff640 1 -- start start 2023-12-19T18:23:54.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.566+0000 7f63fffff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6400154030 0x7f64000cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:54.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.566+0000 7f63fffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f64000c5560 0x7f64000afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:54.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.566+0000 7f63fffff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64000cc190 con 0x7f64000af630 2023-12-19T18:23:54.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.566+0000 7f63fffff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64000b0000 con 0x7f64000c5560 2023-12-19T18:23:54.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.566+0000 7f63fffff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64000cb2a0 con 0x7f6400154030 2023-12-19T18:23:54.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.567+0000 7f63f7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f64000af630 0x7f640015d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34882/0 (socket says 172.21.15.27:34882) 2023-12-19T18:23:54.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.567+0000 7f63f7fff640 1 -- 172.21.15.27:0/1419521908 learned_addr learned my addr 172.21.15.27:0/1419521908 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.567+0000 7f63ff7fe640 1 --2- 172.21.15.27:0/1419521908 >> v2:172.21.15.27:3300/0 conn(0x7f64000c5560 0x7f64000afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.567+0000 7f63feffd640 1 --2- 172.21.15.27:0/1419521908 >> v2:172.21.15.192:3300/0 conn(0x7f6400154030 0x7f64000cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.567+0000 7f63fe7fc640 1 -- 172.21.15.27:0/1419521908 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 123797858 0 0) 0x7f64000cc190 con 0x7f64000af630 2023-12-19T18:23:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.567+0000 7f63fe7fc640 1 -- 172.21.15.27:0/1419521908 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63ec005730 con 0x7f64000af630 2023-12-19T18:23:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.567+0000 7f63ff7fe640 1 -- 172.21.15.27:0/1419521908 >> v2:172.21.15.192:3300/0 conn(0x7f6400154030 msgr2=0x7f64000cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.567+0000 7f63ff7fe640 1 --2- 172.21.15.27:0/1419521908 >> v2:172.21.15.192:3300/0 conn(0x7f6400154030 0x7f64000cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.567+0000 7f63ff7fe640 1 -- 172.21.15.27:0/1419521908 >> v1:172.21.15.27:6789/0 conn(0x7f64000af630 legacy=0x7f640015d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.568+0000 7f63ff7fe640 1 -- 172.21.15.27:0/1419521908 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64000cb580 con 0x7f64000c5560 2023-12-19T18:23:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.569+0000 7f63ff7fe640 1 --2- 172.21.15.27:0/1419521908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64000c5560 0x7f64000afac0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009d90 tx=0x7f63f802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d363e6913a1abb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.569+0000 7f63fe7fc640 1 -- 172.21.15.27:0/1419521908 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f802cc60 con 0x7f64000c5560 2023-12-19T18:23:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.569+0000 7f63fe7fc640 1 -- 172.21.15.27:0/1419521908 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f802cdc0 con 0x7f64000c5560 2023-12-19T18:23:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.569+0000 7f63fe7fc640 1 -- 172.21.15.27:0/1419521908 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f8034640 con 0x7f64000c5560 2023-12-19T18:23:54.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.570+0000 7f63fffff640 1 -- 172.21.15.27:0/1419521908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64000c5560 msgr2=0x7f64000afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:54.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.570+0000 7f63fffff640 1 --2- 172.21.15.27:0/1419521908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64000c5560 0x7f64000afac0 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009d90 tx=0x7f63f802beb0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.570+0000 7f63fffff640 1 -- 172.21.15.27:0/1419521908 shutdown_connections 2023-12-19T18:23:54.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.570+0000 7f63fffff640 1 --2- 172.21.15.27:0/1419521908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64000c5560 0x7f64000afac0 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.570+0000 7f63fffff640 1 --2- 172.21.15.27:0/1419521908 >> v2:172.21.15.192:3300/0 conn(0x7f6400154030 0x7f64000cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.570+0000 7f63fffff640 1 -- 172.21.15.27:0/1419521908 >> 172.21.15.27:0/1419521908 conn(0x7f6400069c70 msgr2=0x7f640006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:54.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.570+0000 7f63fffff640 1 -- 172.21.15.27:0/1419521908 shutdown_connections 2023-12-19T18:23:54.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.570+0000 7f63fffff640 1 -- 172.21.15.27:0/1419521908 wait complete. 2023-12-19T18:23:54.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.572+0000 7f63fffff640 1 Processor -- start 2023-12-19T18:23:54.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.572+0000 7f63fffff640 1 -- start start 2023-12-19T18:23:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.573+0000 7f63fffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64000af630 0x7f64001481d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.573+0000 7f63fffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64000c5560 0x7f64000c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.573+0000 7f63fffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6400154030 0x7f64000c64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.573+0000 7f63fffff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f640006f240 con 0x7f64000c5560 2023-12-19T18:23:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.573+0000 7f63fffff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64001522d0 con 0x7f6400154030 2023-12-19T18:23:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.573+0000 7f63fffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64001525b0 con 0x7f64000af630 2023-12-19T18:23:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.573+0000 7f63f7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64000c5560 0x7f64000c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.573+0000 7f63f7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64000c5560 0x7f64000c5b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50046/0 (socket says 172.21.15.27:50046) 2023-12-19T18:23:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.573+0000 7f63f7fff640 1 -- 172.21.15.27:0/1378342570 learned_addr learned my addr 172.21.15.27:0/1378342570 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.574+0000 7f63f7fff640 1 -- 172.21.15.27:0/1378342570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64000af630 msgr2=0x7f64001481d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.574+0000 7f63f7fff640 1 --2- 172.21.15.27:0/1378342570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64000af630 0x7f64001481d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.574+0000 7f63ff7fe640 1 --2- 172.21.15.27:0/1378342570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6400154030 0x7f64000c64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:54.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.574+0000 7f63f7fff640 1 -- 172.21.15.27:0/1378342570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6400154030 msgr2=0x7f64000c64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:54.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.574+0000 7f63f7fff640 1 --2- 172.21.15.27:0/1378342570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6400154030 0x7f64000c64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.574+0000 7f63f7fff640 1 -- 172.21.15.27:0/1378342570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f8009a70 con 0x7f64000c5560 2023-12-19T18:23:54.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.574+0000 7f63f7fff640 1 --2- 172.21.15.27:0/1378342570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64000c5560 0x7f64000c5b30 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c430 tx=0x7f63e800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:54.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.574+0000 7f63f77fe640 1 -- 172.21.15.27:0/1378342570 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8005ce0 con 0x7f64000c5560 2023-12-19T18:23:54.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.575+0000 7f63f77fe640 1 -- 172.21.15.27:0/1378342570 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e8005e40 con 0x7f64000c5560 2023-12-19T18:23:54.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.575+0000 7f63f77fe640 1 -- 172.21.15.27:0/1378342570 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e80072f0 con 0x7f64000c5560 2023-12-19T18:23:54.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.575+0000 7f63fffff640 1 -- 172.21.15.27:0/1378342570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6400152890 con 0x7f64000c5560 2023-12-19T18:23:54.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.575+0000 7f63fffff640 1 -- 172.21.15.27:0/1378342570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64000c7200 con 0x7f64000c5560 2023-12-19T18:23:54.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.576+0000 7f63fffff640 1 -- 172.21.15.27:0/1378342570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f640004e840 con 0x7f64000c5560 2023-12-19T18:23:54.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.577+0000 7f63f77fe640 1 -- 172.21.15.27:0/1378342570 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f63e8007450 con 0x7f64000c5560 2023-12-19T18:23:54.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.578+0000 7f63f77fe640 1 --2- 172.21.15.27:0/1378342570 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f63e006a460 0x7f63e006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:54.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.578+0000 7f63f77fe640 1 -- 172.21.15.27:0/1378342570 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f63e8087ff0 con 0x7f64000c5560 2023-12-19T18:23:54.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.578+0000 7f63f77fe640 1 -- 172.21.15.27:0/1378342570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7f63e0072050 con 0x7f64000c5560 2023-12-19T18:23:54.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.578+0000 7f63feffd640 1 --2- 172.21.15.27:0/1378342570 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f63e006a460 0x7f63e006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:54.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.579+0000 7f63feffd640 1 --2- 172.21.15.27:0/1378342570 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f63e006a460 0x7f63e006c920 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f63f0006a90 tx=0x7f63f0006a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:54.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.582+0000 7f63f77fe640 1 -- 172.21.15.27:0/1378342570 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f63e8017060 con 0x7f64000c5560 2023-12-19T18:23:54.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.776+0000 7f63fffff640 1 -- 172.21.15.27:0/1378342570 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f64000c3c50 con 0x7f64000c5560 2023-12-19T18:23:54.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.779+0000 7f63f77fe640 1 -- 172.21.15.27:0/1378342570 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16239 (secure 0 0 0) 0x7f63e8053590 con 0x7f64000c5560 2023-12-19T18:23:54.779 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:23:54.779 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":158,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:23:46.716988+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:23:04.431250+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"151","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-19T18:23:05.506926+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.11","mappings":[{"from":4,"to":3}]},{"pgid":"17.f","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:23:54.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.782+0000 7f63fffff640 1 -- 172.21.15.27:0/1378342570 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f63e006a460 msgr2=0x7f63e006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:54.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.782+0000 7f63fffff640 1 --2- 172.21.15.27:0/1378342570 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f63e006a460 0x7f63e006c920 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f63f0006a90 tx=0x7f63f0006a20 comp rx=0 tx=0).stop 2023-12-19T18:23:54.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.782+0000 7f63fffff640 1 -- 172.21.15.27:0/1378342570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64000c5560 msgr2=0x7f64000c5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.782+0000 7f63fffff640 1 --2- 172.21.15.27:0/1378342570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64000c5560 0x7f64000c5b30 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f63e800c430 tx=0x7f63e800c8f0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.782+0000 7f63fffff640 1 -- 172.21.15.27:0/1378342570 shutdown_connections 2023-12-19T18:23:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.782+0000 7f63fffff640 1 --2- 172.21.15.27:0/1378342570 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f63e006a460 0x7f63e006c920 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.782+0000 7f63fffff640 1 --2- 172.21.15.27:0/1378342570 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6400154030 0x7f64000c64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.782+0000 7f63fffff640 1 --2- 172.21.15.27:0/1378342570 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64000c5560 0x7f64000c5b30 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.782+0000 7f63fffff640 1 --2- 172.21.15.27:0/1378342570 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64000af630 0x7f64001481d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.782+0000 7f63fffff640 1 -- 172.21.15.27:0/1378342570 >> 172.21.15.27:0/1378342570 conn(0x7f6400069c70 msgr2=0x7f64000ce8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.782+0000 7f63fffff640 1 -- 172.21.15.27:0/1378342570 shutdown_connections 2023-12-19T18:23:54.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:54.782+0000 7f63fffff640 1 -- 172.21.15.27:0/1378342570 wait complete. 2023-12-19T18:23:54.805 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:23:55.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.032+0000 7f17bb59e640 1 Processor -- start 2023-12-19T18:23:55.033 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.032+0000 7f17bb59e640 1 -- start start 2023-12-19T18:23:55.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.034+0000 7f17bb59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17bc0b49d0 0x7f17bc160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.034+0000 7f17bb59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f17bc0b89b0 0x7f17bc15c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.034+0000 7f17bb59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f17bc0b4db0 0x7f17bc15cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.034+0000 7f17bb59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17bc15d300 con 0x7f17bc0b89b0 2023-12-19T18:23:55.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.034+0000 7f17bb59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17bc0b5ca0 con 0x7f17bc0b4db0 2023-12-19T18:23:55.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.034+0000 7f17bb59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc161800 con 0x7f17bc0b49d0 2023-12-19T18:23:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.034+0000 7f17b3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f17bc0b89b0 0x7f17bc15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.034+0000 7f17b3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f17bc0b89b0 0x7f17bc15c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50062/0 (socket says 172.21.15.27:50062) 2023-12-19T18:23:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.034+0000 7f17b3fff640 1 -- 172.21.15.27:0/2678395873 learned_addr learned my addr 172.21.15.27:0/2678395873 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.035+0000 7f17ba59c640 1 --2- 172.21.15.27:0/2678395873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17bc0b49d0 0x7f17bc160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.035+0000 7f17b3fff640 1 -- 172.21.15.27:0/2678395873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17bc0b49d0 msgr2=0x7f17bc160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.035+0000 7f17b3fff640 1 --2- 172.21.15.27:0/2678395873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17bc0b49d0 0x7f17bc160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.035+0000 7f17b3fff640 1 -- 172.21.15.27:0/2678395873 >> v2:172.21.15.192:3300/0 conn(0x7f17bc0b4db0 msgr2=0x7f17bc15cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.035+0000 7f17b3fff640 1 --2- 172.21.15.27:0/2678395873 >> v2:172.21.15.192:3300/0 conn(0x7f17bc0b4db0 0x7f17bc15cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.035+0000 7f17b3fff640 1 -- 172.21.15.27:0/2678395873 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc161240 con 0x7f17bc0b89b0 2023-12-19T18:23:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.036+0000 7f17b3fff640 1 --2- 172.21.15.27:0/2678395873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17bc0b89b0 0x7f17bc15c880 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0b5bc0 tx=0x7f17a402b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9690f4e3b37181a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.036+0000 7f17b9d9b640 1 -- 172.21.15.27:0/2678395873 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4030850 con 0x7f17bc0b89b0 2023-12-19T18:23:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.037+0000 7f17b9d9b640 1 -- 172.21.15.27:0/2678395873 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a40309b0 con 0x7f17bc0b89b0 2023-12-19T18:23:55.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.037+0000 7f17b9d9b640 1 -- 172.21.15.27:0/2678395873 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a4030d40 con 0x7f17bc0b89b0 2023-12-19T18:23:55.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.037+0000 7f17bb59e640 1 -- 172.21.15.27:0/2678395873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17bc0b89b0 msgr2=0x7f17bc15c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:55.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.037+0000 7f17bb59e640 1 --2- 172.21.15.27:0/2678395873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17bc0b89b0 0x7f17bc15c880 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f17bc0b5bc0 tx=0x7f17a402b990 comp rx=0 tx=0).stop 2023-12-19T18:23:55.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.037+0000 7f17bb59e640 1 -- 172.21.15.27:0/2678395873 shutdown_connections 2023-12-19T18:23:55.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.037+0000 7f17bb59e640 1 --2- 172.21.15.27:0/2678395873 >> v2:172.21.15.192:3300/0 conn(0x7f17bc0b4db0 0x7f17bc15cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.037+0000 7f17bb59e640 1 --2- 172.21.15.27:0/2678395873 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17bc0b89b0 0x7f17bc15c880 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.037+0000 7f17bb59e640 1 --2- 172.21.15.27:0/2678395873 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17bc0b49d0 0x7f17bc160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.037+0000 7f17bb59e640 1 -- 172.21.15.27:0/2678395873 >> 172.21.15.27:0/2678395873 conn(0x7f17bc0cf760 msgr2=0x7f17bc0cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:55.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.037+0000 7f17bb59e640 1 -- 172.21.15.27:0/2678395873 shutdown_connections 2023-12-19T18:23:55.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.038+0000 7f17bb59e640 1 -- 172.21.15.27:0/2678395873 wait complete. 2023-12-19T18:23:55.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.040+0000 7f17bb59e640 1 Processor -- start 2023-12-19T18:23:55.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.040+0000 7f17bb59e640 1 -- start start 2023-12-19T18:23:55.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.041+0000 7f17bb59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17bc0b49d0 0x7f17bc14b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.041+0000 7f17bb59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17bc0b4db0 0x7f17bc15d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.041+0000 7f17bb59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17bc0b89b0 0x7f17bc153740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.041+0000 7f17bb59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc1532b0 con 0x7f17bc0b89b0 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.041+0000 7f17bb59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc14b6d0 con 0x7f17bc0b4db0 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.041+0000 7f17bb59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc0c7df0 con 0x7f17bc0b49d0 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.042+0000 7f17bad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17bc0b89b0 0x7f17bc153740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.042+0000 7f17bad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17bc0b89b0 0x7f17bc153740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50074/0 (socket says 172.21.15.27:50074) 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.042+0000 7f17bad9d640 1 -- 172.21.15.27:0/2780508230 learned_addr learned my addr 172.21.15.27:0/2780508230 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.042+0000 7f17b3fff640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17bc0b4db0 0x7f17bc15d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.042+0000 7f17ba59c640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17bc0b49d0 0x7f17bc14b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.042+0000 7f17bad9d640 1 -- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17bc0b49d0 msgr2=0x7f17bc14b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.042+0000 7f17bad9d640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17bc0b49d0 0x7f17bc14b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.042+0000 7f17bad9d640 1 -- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17bc0b4db0 msgr2=0x7f17bc15d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.042+0000 7f17bad9d640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17bc0b4db0 0x7f17bc15d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.042+0000 7f17bad9d640 1 -- 172.21.15.27:0/2780508230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a40094d0 con 0x7f17bc0b89b0 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.042+0000 7f17ba59c640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17bc0b49d0 0x7f17bc14b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:55.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.042+0000 7f17b3fff640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17bc0b4db0 0x7f17bc15d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:55.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.043+0000 7f17bad9d640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17bc0b89b0 0x7f17bc153740 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b970 tx=0x7f17b400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:55.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.043+0000 7f17b2ffd640 1 -- 172.21.15.27:0/2780508230 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b400cb40 con 0x7f17bc0b89b0 2023-12-19T18:23:55.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.043+0000 7f17b2ffd640 1 -- 172.21.15.27:0/2780508230 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17b400ccc0 con 0x7f17bc0b89b0 2023-12-19T18:23:55.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.043+0000 7f17bb59e640 1 -- 172.21.15.27:0/2780508230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17bc15ad30 con 0x7f17bc0b89b0 2023-12-19T18:23:55.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.043+0000 7f17b2ffd640 1 -- 172.21.15.27:0/2780508230 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b4014680 con 0x7f17bc0b89b0 2023-12-19T18:23:55.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.043+0000 7f17bb59e640 1 -- 172.21.15.27:0/2780508230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17bc15aa50 con 0x7f17bc0b89b0 2023-12-19T18:23:55.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.044+0000 7f17bb59e640 1 -- 172.21.15.27:0/2780508230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17bc153420 con 0x7f17bc0b89b0 2023-12-19T18:23:55.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.045+0000 7f17b2ffd640 1 -- 172.21.15.27:0/2780508230 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f17b401d3b0 con 0x7f17bc0b89b0 2023-12-19T18:23:55.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.046+0000 7f17b2ffd640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f179c06a460 0x7f179c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.047+0000 7f17b2ffd640 1 -- 172.21.15.27:0/2780508230 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f17b4088880 con 0x7f17bc0b89b0 2023-12-19T18:23:55.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.047+0000 7f17b2ffd640 1 -- 172.21.15.27:0/2780508230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=159}) v3 -- 0x7f179c072050 con 0x7f17bc0b89b0 2023-12-19T18:23:55.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.047+0000 7f17ba59c640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f179c06a460 0x7f179c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.048+0000 7f17ba59c640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f179c06a460 0x7f179c06c920 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009820 tx=0x7f17ac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:55.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.051+0000 7f17b2ffd640 1 -- 172.21.15.27:0/2780508230 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f17b4019020 con 0x7f17bc0b89b0 2023-12-19T18:23:55.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.283+0000 7f17bb59e640 1 -- 172.21.15.27:0/2780508230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f17bc04e840 con 0x7f17bc0b89b0 2023-12-19T18:23:55.284 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:23:55.285+0000 7f8f3865c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:23:55.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.290+0000 7f17b2ffd640 1 -- 172.21.15.27:0/2780508230 <== mon.0 v2:172.21.15.27:3300/0 7 ==== osd_map(159..159 src has 1..159) v4 ==== 374+0+0 (secure 0 0 0) 0x7f17b4088070 con 0x7f17bc0b89b0 2023-12-19T18:23:55.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.290+0000 7f17b2ffd640 1 -- 172.21.15.27:0/2780508230 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f179c072900 con 0x7f17bc0b89b0 2023-12-19T18:23:55.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.307+0000 7f17b2ffd640 1 -- 172.21.15.27:0/2780508230 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) v1 ==== 161+0+0 (secure 0 0 0) 0x7f17b40578c0 con 0x7f17bc0b89b0 2023-12-19T18:23:55.307 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:23:55.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.310+0000 7f17bb59e640 1 -- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f179c06a460 msgr2=0x7f179c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:55.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.310+0000 7f17bb59e640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f179c06a460 0x7f179c06c920 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009820 tx=0x7f17ac008040 comp rx=0 tx=0).stop 2023-12-19T18:23:55.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.310+0000 7f17bb59e640 1 -- 172.21.15.27:0/2780508230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17bc0b89b0 msgr2=0x7f17bc153740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:55.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.310+0000 7f17bb59e640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17bc0b89b0 0x7f17bc153740 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f17b400b970 tx=0x7f17b400be30 comp rx=0 tx=0).stop 2023-12-19T18:23:55.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.311+0000 7f17bb59e640 1 -- 172.21.15.27:0/2780508230 shutdown_connections 2023-12-19T18:23:55.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.311+0000 7f17bb59e640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f179c06a460 0x7f179c06c920 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.311+0000 7f17bb59e640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17bc0b89b0 0x7f17bc153740 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.311+0000 7f17bb59e640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f17bc0b4db0 0x7f17bc15d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.311+0000 7f17bb59e640 1 --2- 172.21.15.27:0/2780508230 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f17bc0b49d0 0x7f17bc14b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.311+0000 7f17bb59e640 1 -- 172.21.15.27:0/2780508230 >> 172.21.15.27:0/2780508230 conn(0x7f17bc0cf760 msgr2=0x7f17bc0b5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:55.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.311+0000 7f17bb59e640 1 -- 172.21.15.27:0/2780508230 shutdown_connections 2023-12-19T18:23:55.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.311+0000 7f17bb59e640 1 -- 172.21.15.27:0/2780508230 wait complete. 2023-12-19T18:23:55.321 INFO:tasks.ceph.mds.a.smithi027.stderr: -8042> 2023-12-19T18:23:11.416+0000 7f442b6c6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:23:55.330 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:23:55.383 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:23:55.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.538+0000 7f0777570640 1 Processor -- start 2023-12-19T18:23:55.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.539+0000 7f0777570640 1 -- start start 2023-12-19T18:23:55.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.540+0000 7f0777570640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07700cdb30 0x7f07700cdf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.540+0000 7f0777570640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07700ce4d0 con 0x7f077015cfc0 2023-12-19T18:23:55.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.540+0000 7f0777570640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0770161250 con 0x7f07700d17a0 2023-12-19T18:23:55.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.540+0000 7f0777570640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0770161a50 con 0x7f07700cdb30 2023-12-19T18:23:55.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.541+0000 7f077656e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f077015cfc0 0x7f0770160b40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34886/0 (socket says 172.21.15.27:34886) 2023-12-19T18:23:55.541 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.541+0000 7f077656e640 1 -- 172.21.15.27:0/746143633 learned_addr learned my addr 172.21.15.27:0/746143633 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:55.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.541+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2856847734 0 0) 0x7f07700ce4d0 con 0x7f077015cfc0 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.541+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0758003520 con 0x7f077015cfc0 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.541+0000 7f0775d6d640 1 --2- 172.21.15.27:0/746143633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07700cdb30 0x7f07700cdf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.541+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1927012639 0 0) 0x7f0758003520 con 0x7f077015cfc0 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.541+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07700cdb30 msgr2=0x7f07700cdf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.541+0000 7f077556c640 1 --2- 172.21.15.27:0/746143633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07700cdb30 0x7f07700cdf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.541+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 >> v1:172.21.15.192:6789/0 conn(0x7f07700d17a0 legacy=0x7f07700d1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.541+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0770161490 con 0x7f077015cfc0 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.541+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f0760001670 con 0x7f077015cfc0 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.541+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 >> v1:172.21.15.27:6789/0 conn(0x7f077015cfc0 legacy=0x7f0770160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f077556c640 1 --2- 172.21.15.27:0/746143633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07580039a0 0x7f0758003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f077556c640 1 --2- 172.21.15.27:0/746143633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07580047c0 0x7f0758004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f077556c640 1 --2- 172.21.15.27:0/746143633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07700cdb30 0x7f075800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0770161a50 con 0x7f07580047c0 2023-12-19T18:23:55.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07580054b0 con 0x7f07580039a0 2023-12-19T18:23:55.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0758005670 con 0x7f07700cdb30 2023-12-19T18:23:55.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f0775d6d640 1 --2- 172.21.15.27:0/746143633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07580047c0 0x7f0758004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f077656e640 1 --2- 172.21.15.27:0/746143633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07580039a0 0x7f0758003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f0775d6d640 1 -- 172.21.15.27:0/746143633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07700cdb30 msgr2=0x7f075800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:55.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f0776d6f640 1 --2- 172.21.15.27:0/746143633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07700cdb30 0x7f075800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f0775d6d640 1 --2- 172.21.15.27:0/746143633 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07700cdb30 0x7f075800b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f0775d6d640 1 -- 172.21.15.27:0/746143633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07580039a0 msgr2=0x7f0758003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:55.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f0775d6d640 1 --2- 172.21.15.27:0/746143633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07580039a0 0x7f0758003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f0775d6d640 1 -- 172.21.15.27:0/746143633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0758003520 con 0x7f07580047c0 2023-12-19T18:23:55.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.542+0000 7f077656e640 1 --2- 172.21.15.27:0/746143633 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f07580039a0 0x7f0758003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:55.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.544+0000 7f0775d6d640 1 --2- 172.21.15.27:0/746143633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07580047c0 0x7f0758004c40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f076400ba30 tx=0x7f07580057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc2fe8a069e734b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:55.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.544+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764032640 con 0x7f07580047c0 2023-12-19T18:23:55.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.544+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07640327a0 con 0x7f07580047c0 2023-12-19T18:23:55.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.544+0000 7f077556c640 1 -- 172.21.15.27:0/746143633 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0764032b30 con 0x7f07580047c0 2023-12-19T18:23:55.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.544+0000 7f0777570640 1 -- 172.21.15.27:0/746143633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07580047c0 msgr2=0x7f0758004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:55.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.544+0000 7f0777570640 1 --2- 172.21.15.27:0/746143633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07580047c0 0x7f0758004c40 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f076400ba30 tx=0x7f07580057f0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.544+0000 7f0776d6f640 1 -- 172.21.15.27:0/746143633 reap_dead start 2023-12-19T18:23:55.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.545+0000 7f0777570640 1 -- 172.21.15.27:0/746143633 shutdown_connections 2023-12-19T18:23:55.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.545+0000 7f0777570640 1 -- 172.21.15.27:0/746143633 >> 172.21.15.27:0/746143633 conn(0x7f0770072e20 msgr2=0x7f0770073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:55.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.545+0000 7f0777570640 1 -- 172.21.15.27:0/746143633 shutdown_connections 2023-12-19T18:23:55.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.545+0000 7f0777570640 1 -- 172.21.15.27:0/746143633 wait complete. 2023-12-19T18:23:55.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.547+0000 7f0777570640 1 Processor -- start 2023-12-19T18:23:55.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.547+0000 7f0777570640 1 -- start start 2023-12-19T18:23:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0777570640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07580047c0 0x7f07700c8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0777570640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077015ddb0 0x7f07700b6f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0777570640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f077015e230 0x7f07700b7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0777570640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077015e790 con 0x7f077015e230 2023-12-19T18:23:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0777570640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f077015ea70 con 0x7f077015ddb0 2023-12-19T18:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0777570640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f077015ed50 con 0x7f07580047c0 2023-12-19T18:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0776d6f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f077015e230 0x7f07700b7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0776d6f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f077015e230 0x7f07700b7450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50114/0 (socket says 172.21.15.27:50114) 2023-12-19T18:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0776d6f640 1 -- 172.21.15.27:0/942124737 learned_addr learned my addr 172.21.15.27:0/942124737 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f077656e640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07580047c0 0x7f07700c8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0775d6d640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077015ddb0 0x7f07700b6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0776d6f640 1 -- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07580047c0 msgr2=0x7f07700c8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0776d6f640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07580047c0 0x7f07700c8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0776d6f640 1 -- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077015ddb0 msgr2=0x7f07700b6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0776d6f640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077015ddb0 0x7f07700b6f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.548+0000 7f0776d6f640 1 -- 172.21.15.27:0/942124737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07701f3ea0 con 0x7f077015e230 2023-12-19T18:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.549+0000 7f0776d6f640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f077015e230 0x7f07700b7450 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f07700d3e00 tx=0x7f076800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.549+0000 7f076effd640 1 -- 172.21.15.27:0/942124737 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768003b70 con 0x7f077015e230 2023-12-19T18:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.549+0000 7f076effd640 1 -- 172.21.15.27:0/942124737 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0768003cd0 con 0x7f077015e230 2023-12-19T18:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.549+0000 7f076effd640 1 -- 172.21.15.27:0/942124737 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0768005290 con 0x7f077015e230 2023-12-19T18:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.549+0000 7f0777570640 1 -- 172.21.15.27:0/942124737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f077015f030 con 0x7f077015e230 2023-12-19T18:23:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.549+0000 7f0777570640 1 -- 172.21.15.27:0/942124737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07700cb7f0 con 0x7f077015e230 2023-12-19T18:23:55.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.551+0000 7f0777570640 1 -- 172.21.15.27:0/942124737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07701f4090 con 0x7f077015e230 2023-12-19T18:23:55.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.551+0000 7f076effd640 1 -- 172.21.15.27:0/942124737 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f07680053f0 con 0x7f077015e230 2023-12-19T18:23:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.552+0000 7f076effd640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f075c06a460 0x7f075c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:55.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.552+0000 7f076effd640 1 -- 172.21.15.27:0/942124737 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f07680880f0 con 0x7f077015e230 2023-12-19T18:23:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.553+0000 7f076effd640 1 -- 172.21.15.27:0/942124737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7f075c072160 con 0x7f077015e230 2023-12-19T18:23:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.553+0000 7f077656e640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f075c06a460 0x7f075c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:55.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.554+0000 7f077656e640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f075c06a460 0x7f075c06c920 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0760000c00 tx=0x7f0760009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.557+0000 7f076effd640 1 -- 172.21.15.27:0/942124737 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0768018060 con 0x7f077015e230 2023-12-19T18:23:55.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:55.785+0000 7f0777570640 1 -- 172.21.15.27:0/942124737 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0770161250 con 0x7f077015e230 2023-12-19T18:23:56.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.310+0000 7f076effd640 1 -- 172.21.15.27:0/942124737 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0770161250 con 0x7f077015e230 2023-12-19T18:23:56.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.314+0000 7f0777570640 1 -- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f075c06a460 msgr2=0x7f075c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:56.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.314+0000 7f0777570640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f075c06a460 0x7f075c06c920 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f0760000c00 tx=0x7f0760009040 comp rx=0 tx=0).stop 2023-12-19T18:23:56.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.314+0000 7f0777570640 1 -- 172.21.15.27:0/942124737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f077015e230 msgr2=0x7f07700b7450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:56.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.314+0000 7f0777570640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f077015e230 0x7f07700b7450 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f07700d3e00 tx=0x7f076800dc00 comp rx=0 tx=0).stop 2023-12-19T18:23:56.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.314+0000 7f0777570640 1 -- 172.21.15.27:0/942124737 shutdown_connections 2023-12-19T18:23:56.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.314+0000 7f0777570640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f075c06a460 0x7f075c06c920 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:56.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.314+0000 7f0777570640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f077015e230 0x7f07700b7450 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:56.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.314+0000 7f0777570640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f077015ddb0 0x7f07700b6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:56.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.314+0000 7f0777570640 1 --2- 172.21.15.27:0/942124737 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f07580047c0 0x7f07700c8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:56.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.314+0000 7f0777570640 1 -- 172.21.15.27:0/942124737 >> 172.21.15.27:0/942124737 conn(0x7f0770072e20 msgr2=0x7f0770073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:56.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.315+0000 7f0777570640 1 -- 172.21.15.27:0/942124737 shutdown_connections 2023-12-19T18:23:56.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.315+0000 7f0777570640 1 -- 172.21.15.27:0/942124737 wait complete. 2023-12-19T18:23:56.337 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:23:56.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.567+0000 7fea42d20640 1 Processor -- start 2023-12-19T18:23:56.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.568+0000 7fea42d20640 1 -- start start 2023-12-19T18:23:56.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.569+0000 7fea42d20640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fea3c15b4b0 0x7fea3c0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:56.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.569+0000 7fea42d20640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fea3c15fa70 0x7fea3c0cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:56.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.569+0000 7fea42d20640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea3c070fb0 con 0x7fea3c15b0d0 2023-12-19T18:23:56.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.569+0000 7fea42d20640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3c06fab0 con 0x7fea3c15b4b0 2023-12-19T18:23:56.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.569+0000 7fea42d20640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea3c06f4f0 con 0x7fea3c15fa70 2023-12-19T18:23:56.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.570+0000 7fea3bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fea3c15b4b0 0x7fea3c0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:56.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.570+0000 7fea40abc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fea3c15b0d0 0x7fea3c0cabf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34888/0 (socket says 172.21.15.27:34888) 2023-12-19T18:23:56.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.570+0000 7fea40abc640 1 -- 172.21.15.27:0/2519665196 learned_addr learned my addr 172.21.15.27:0/2519665196 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:56.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.570+0000 7fea412bd640 1 --2- 172.21.15.27:0/2519665196 >> v2:172.21.15.192:3300/0 conn(0x7fea3c15fa70 0x7fea3c0cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:56.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.570+0000 7fea3bfff640 1 -- 172.21.15.27:0/2519665196 >> v2:172.21.15.192:3300/0 conn(0x7fea3c15fa70 msgr2=0x7fea3c0cb990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:56.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.570+0000 7fea3bfff640 1 --2- 172.21.15.27:0/2519665196 >> v2:172.21.15.192:3300/0 conn(0x7fea3c15fa70 0x7fea3c0cb990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:56.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.570+0000 7fea3bfff640 1 -- 172.21.15.27:0/2519665196 >> v1:172.21.15.27:6789/0 conn(0x7fea3c15b0d0 legacy=0x7fea3c0cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:56.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.570+0000 7fea3bfff640 1 -- 172.21.15.27:0/2519665196 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c06f7d0 con 0x7fea3c15b4b0 2023-12-19T18:23:56.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.570+0000 7fea3b7fe640 1 -- 172.21.15.27:0/2519665196 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 830932773 0 0) 0x7fea3c070fb0 con 0x7fea3c15b0d0 2023-12-19T18:23:56.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.572+0000 7fea3bfff640 1 --2- 172.21.15.27:0/2519665196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea3c15b4b0 0x7fea3c0cb450 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0097a0 tx=0x7fea2c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0b3ee8c671da35 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:56.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.572+0000 7fea3b7fe640 1 -- 172.21.15.27:0/2519665196 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c030610 con 0x7fea3c15b4b0 2023-12-19T18:23:56.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.572+0000 7fea3b7fe640 1 -- 172.21.15.27:0/2519665196 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea2c030770 con 0x7fea3c15b4b0 2023-12-19T18:23:56.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.572+0000 7fea3b7fe640 1 -- 172.21.15.27:0/2519665196 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea2c030b00 con 0x7fea3c15b4b0 2023-12-19T18:23:56.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.573+0000 7fea42d20640 1 -- 172.21.15.27:0/2519665196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea3c15b4b0 msgr2=0x7fea3c0cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:56.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.573+0000 7fea42d20640 1 --2- 172.21.15.27:0/2519665196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea3c15b4b0 0x7fea3c0cb450 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0097a0 tx=0x7fea2c02b960 comp rx=0 tx=0).stop 2023-12-19T18:23:56.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.573+0000 7fea42d20640 1 -- 172.21.15.27:0/2519665196 shutdown_connections 2023-12-19T18:23:56.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.573+0000 7fea42d20640 1 --2- 172.21.15.27:0/2519665196 >> v2:172.21.15.192:3300/0 conn(0x7fea3c15fa70 0x7fea3c0cb990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:56.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.573+0000 7fea42d20640 1 --2- 172.21.15.27:0/2519665196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea3c15b4b0 0x7fea3c0cb450 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:56.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.573+0000 7fea42d20640 1 -- 172.21.15.27:0/2519665196 >> 172.21.15.27:0/2519665196 conn(0x7fea3c0b0620 msgr2=0x7fea3c0b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:56.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.573+0000 7fea42d20640 1 -- 172.21.15.27:0/2519665196 shutdown_connections 2023-12-19T18:23:56.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.573+0000 7fea42d20640 1 -- 172.21.15.27:0/2519665196 wait complete. 2023-12-19T18:23:56.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.575+0000 7fea42d20640 1 Processor -- start 2023-12-19T18:23:56.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.575+0000 7fea42d20640 1 -- start start 2023-12-19T18:23:56.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.576+0000 7fea42d20640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea3c15b0d0 0x7fea3c14b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:56.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.576+0000 7fea42d20640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea3c15b4b0 0x7fea3c0c8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:56.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.576+0000 7fea42d20640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea3c15fa70 0x7fea3c0c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:56.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.576+0000 7fea42d20640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c15d1d0 con 0x7fea3c15b0d0 2023-12-19T18:23:56.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.576+0000 7fea42d20640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c15c5c0 con 0x7fea3c15fa70 2023-12-19T18:23:56.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.576+0000 7fea42d20640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea3c15c8a0 con 0x7fea3c15b4b0 2023-12-19T18:23:56.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.576+0000 7fea40abc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea3c15b0d0 0x7fea3c14b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:56.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.576+0000 7fea40abc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea3c15b0d0 0x7fea3c14b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50146/0 (socket says 172.21.15.27:50146) 2023-12-19T18:23:56.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.576+0000 7fea40abc640 1 -- 172.21.15.27:0/4156525482 learned_addr learned my addr 172.21.15.27:0/4156525482 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:56.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.577+0000 7fea3bfff640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea3c15b4b0 0x7fea3c0c8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.577+0000 7fea40abc640 1 -- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea3c15b4b0 msgr2=0x7fea3c0c8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.577+0000 7fea412bd640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea3c15fa70 0x7fea3c0c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.577+0000 7fea40abc640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea3c15b4b0 0x7fea3c0c8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.577+0000 7fea40abc640 1 -- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea3c15fa70 msgr2=0x7fea3c0c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.577+0000 7fea40abc640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea3c15fa70 0x7fea3c0c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.577+0000 7fea40abc640 1 -- 172.21.15.27:0/4156525482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2c009400 con 0x7fea3c15b0d0 2023-12-19T18:23:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.577+0000 7fea3bfff640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea3c15b4b0 0x7fea3c0c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:56.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.577+0000 7fea40abc640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea3c15b0d0 0x7fea3c14b1d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fea3000c4d0 tx=0x7fea3000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:56.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.577+0000 7fea397fa640 1 -- 172.21.15.27:0/4156525482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea300074e0 con 0x7fea3c15b0d0 2023-12-19T18:23:56.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.577+0000 7fea397fa640 1 -- 172.21.15.27:0/4156525482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea30007660 con 0x7fea3c15b0d0 2023-12-19T18:23:56.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.577+0000 7fea397fa640 1 -- 172.21.15.27:0/4156525482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea30004a20 con 0x7fea3c15b0d0 2023-12-19T18:23:56.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.577+0000 7fea42d20640 1 -- 172.21.15.27:0/4156525482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea3c15cb80 con 0x7fea3c15b0d0 2023-12-19T18:23:56.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.578+0000 7fea42d20640 1 -- 172.21.15.27:0/4156525482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea3c14b710 con 0x7fea3c15b0d0 2023-12-19T18:23:56.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.579+0000 7fea42d20640 1 -- 172.21.15.27:0/4156525482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea3c14b990 con 0x7fea3c15b0d0 2023-12-19T18:23:56.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.580+0000 7fea397fa640 1 -- 172.21.15.27:0/4156525482 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fea30004b80 con 0x7fea3c15b0d0 2023-12-19T18:23:56.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.581+0000 7fea397fa640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea2806a430 0x7fea2806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:56.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.581+0000 7fea397fa640 1 -- 172.21.15.27:0/4156525482 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fea30087c70 con 0x7fea3c15b0d0 2023-12-19T18:23:56.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.581+0000 7fea397fa640 1 -- 172.21.15.27:0/4156525482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=160}) v3 -- 0x7fea28072130 con 0x7fea3c15b0d0 2023-12-19T18:23:56.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.581+0000 7fea3bfff640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea2806a430 0x7fea2806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:56.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.583+0000 7fea3bfff640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea2806a430 0x7fea2806c8f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fea2c036000 tx=0x7fea2c03b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:56.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.585+0000 7fea397fa640 1 -- 172.21.15.27:0/4156525482 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fea30018020 con 0x7fea3c15b0d0 2023-12-19T18:23:56.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:56.785+0000 7fea42d20640 1 -- 172.21.15.27:0/4156525482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fea3c15d310 con 0x7fea3c15b0d0 2023-12-19T18:23:57.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.727+0000 7fea397fa640 1 -- 172.21.15.27:0/4156525482 <== mon.0 v2:172.21.15.27:3300/0 7 ==== osd_map(160..160 src has 1..160) v4 ==== 313+0+0 (secure 0 0 0) 0x7fea30004e30 con 0x7fea3c15b0d0 2023-12-19T18:23:57.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.727+0000 7fea397fa640 1 -- 172.21.15.27:0/4156525482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=161}) v3 -- 0x7fea28070560 con 0x7fea3c15b0d0 2023-12-19T18:23:57.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.735+0000 7fea397fa640 1 -- 172.21.15.27:0/4156525482 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v160) v1 ==== 181+0+0 (secure 0 0 0) 0x7fea30056bc0 con 0x7fea3c15b0d0 2023-12-19T18:23:57.736 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:23:57.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.741+0000 7fea42d20640 1 -- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea2806a430 msgr2=0x7fea2806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:57.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.741+0000 7fea42d20640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea2806a430 0x7fea2806c8f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fea2c036000 tx=0x7fea2c03b040 comp rx=0 tx=0).stop 2023-12-19T18:23:57.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.741+0000 7fea42d20640 1 -- 172.21.15.27:0/4156525482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea3c15b0d0 msgr2=0x7fea3c14b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:57.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.741+0000 7fea42d20640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea3c15b0d0 0x7fea3c14b1d0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7fea3000c4d0 tx=0x7fea3000c990 comp rx=0 tx=0).stop 2023-12-19T18:23:57.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.741+0000 7fea42d20640 1 -- 172.21.15.27:0/4156525482 shutdown_connections 2023-12-19T18:23:57.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.741+0000 7fea42d20640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fea2806a430 0x7fea2806c8f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:57.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.741+0000 7fea42d20640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fea3c15fa70 0x7fea3c0c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:57.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.741+0000 7fea42d20640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fea3c15b4b0 0x7fea3c0c8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:57.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.741+0000 7fea42d20640 1 --2- 172.21.15.27:0/4156525482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea3c15b0d0 0x7fea3c14b1d0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:57.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.741+0000 7fea42d20640 1 -- 172.21.15.27:0/4156525482 >> 172.21.15.27:0/4156525482 conn(0x7fea3c0b0620 msgr2=0x7fea3c0c2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:57.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.742+0000 7fea42d20640 1 -- 172.21.15.27:0/4156525482 shutdown_connections 2023-12-19T18:23:57.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.742+0000 7fea42d20640 1 -- 172.21.15.27:0/4156525482 wait complete. 2023-12-19T18:23:57.764 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:23:57.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.998+0000 7fd2fb5f0640 1 Processor -- start 2023-12-19T18:23:57.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.998+0000 7fd2fb5f0640 1 -- start start 2023-12-19T18:23:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:57.999+0000 7fd2fb5f0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd2f415d060 0x7fd2f41616e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.000+0000 7fd2fb5f0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd2f406a8a0 0x7fd2f40d1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.000+0000 7fd2fb5f0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2f4161c20 con 0x7fd2f40b6270 2023-12-19T18:23:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.000+0000 7fd2fb5f0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f40ce7d0 con 0x7fd2f406a8a0 2023-12-19T18:23:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.000+0000 7fd2fb5f0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2f40cdbd0 con 0x7fd2f415d060 2023-12-19T18:23:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.000+0000 7fd2f9b8d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd2f406a8a0 0x7fd2f40d1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.000+0000 7fd2f9b8d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd2f406a8a0 0x7fd2f40d1d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50154/0 (socket says 172.21.15.27:50154) 2023-12-19T18:23:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.000+0000 7fd2f9b8d640 1 -- 172.21.15.27:0/296664581 learned_addr learned my addr 172.21.15.27:0/296664581 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.000+0000 7fd2f938c640 1 --2- 172.21.15.27:0/296664581 >> v2:172.21.15.192:3300/0 conn(0x7fd2f415d060 0x7fd2f41616e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.000+0000 7fd2f9b8d640 1 -- 172.21.15.27:0/296664581 >> v2:172.21.15.192:3300/0 conn(0x7fd2f415d060 msgr2=0x7fd2f41616e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.000+0000 7fd2f9b8d640 1 --2- 172.21.15.27:0/296664581 >> v2:172.21.15.192:3300/0 conn(0x7fd2f415d060 0x7fd2f41616e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.000+0000 7fd2f9b8d640 1 -- 172.21.15.27:0/296664581 >> v1:172.21.15.192:6789/0 conn(0x7fd2f40b6270 legacy=0x7fd2f406a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.001+0000 7fd2f9b8d640 1 -- 172.21.15.27:0/296664581 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f40ce4f0 con 0x7fd2f406a8a0 2023-12-19T18:23:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.001+0000 7fd2f938c640 1 --2- 172.21.15.27:0/296664581 >> v2:172.21.15.192:3300/0 conn(0x7fd2f415d060 0x7fd2f41616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.002+0000 7fd2f9b8d640 1 --2- 172.21.15.27:0/296664581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f406a8a0 0x7fd2f40d1d70 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40d2ef0 tx=0x7fd2ec02be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a686edc92ec4492 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.002+0000 7fd2ebfff640 1 -- 172.21.15.27:0/296664581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec02c9f0 con 0x7fd2f406a8a0 2023-12-19T18:23:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.002+0000 7fd2ebfff640 1 -- 172.21.15.27:0/296664581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2ec02cb50 con 0x7fd2f406a8a0 2023-12-19T18:23:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.002+0000 7fd2ebfff640 1 -- 172.21.15.27:0/296664581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2ec0343f0 con 0x7fd2f406a8a0 2023-12-19T18:23:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.003+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/296664581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f406a8a0 msgr2=0x7fd2f40d1d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.003+0000 7fd2fb5f0640 1 --2- 172.21.15.27:0/296664581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f406a8a0 0x7fd2f40d1d70 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7fd2f40d2ef0 tx=0x7fd2ec02be50 comp rx=0 tx=0).stop 2023-12-19T18:23:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.003+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/296664581 shutdown_connections 2023-12-19T18:23:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.003+0000 7fd2fb5f0640 1 --2- 172.21.15.27:0/296664581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f406a8a0 0x7fd2f40d1d70 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.003+0000 7fd2fb5f0640 1 --2- 172.21.15.27:0/296664581 >> v2:172.21.15.192:3300/0 conn(0x7fd2f415d060 0x7fd2f41616e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.003+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/296664581 >> 172.21.15.27:0/296664581 conn(0x7fd2f40cfa10 msgr2=0x7fd2f40cfdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.003+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/296664581 shutdown_connections 2023-12-19T18:23:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.003+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/296664581 wait complete. 2023-12-19T18:23:58.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.005+0000 7fd2fb5f0640 1 Processor -- start 2023-12-19T18:23:58.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.005+0000 7fd2fb5f0640 1 -- start start 2023-12-19T18:23:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.006+0000 7fd2fb5f0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f406a8a0 0x7fd2f41603c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.006+0000 7fd2fb5f0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2f40b6270 0x7fd2f414ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.006+0000 7fd2fb5f0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2f415d060 0x7fd2f4158120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.006+0000 7fd2fb5f0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f415e270 con 0x7fd2f406a8a0 2023-12-19T18:23:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.006+0000 7fd2fb5f0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f4150460 con 0x7fd2f415d060 2023-12-19T18:23:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.006+0000 7fd2fb5f0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2f40ca690 con 0x7fd2f40b6270 2023-12-19T18:23:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.006+0000 7fd2f938c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f406a8a0 0x7fd2f41603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.006+0000 7fd2f938c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f406a8a0 0x7fd2f41603c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50162/0 (socket says 172.21.15.27:50162) 2023-12-19T18:23:58.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.006+0000 7fd2f938c640 1 -- 172.21.15.27:0/2240992745 learned_addr learned my addr 172.21.15.27:0/2240992745 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.007+0000 7fd2f9b8d640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2f415d060 0x7fd2f4158120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.007+0000 7fd2f8b8b640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2f40b6270 0x7fd2f414ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.007+0000 7fd2f938c640 1 -- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2f40b6270 msgr2=0x7fd2f414ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.007+0000 7fd2f938c640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2f40b6270 0x7fd2f414ff20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.007+0000 7fd2f938c640 1 -- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2f415d060 msgr2=0x7fd2f4158120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.007+0000 7fd2f938c640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2f415d060 0x7fd2f4158120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.007+0000 7fd2f938c640 1 -- 172.21.15.27:0/2240992745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec009b30 con 0x7fd2f406a8a0 2023-12-19T18:23:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.007+0000 7fd2f9b8d640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2f415d060 0x7fd2f4158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.007+0000 7fd2f938c640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f406a8a0 0x7fd2f41603c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000c820 tx=0x7fd2f000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.007+0000 7fd2e9ffb640 1 -- 172.21.15.27:0/2240992745 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f00079d0 con 0x7fd2f406a8a0 2023-12-19T18:23:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.007+0000 7fd2e9ffb640 1 -- 172.21.15.27:0/2240992745 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2f0007b30 con 0x7fd2f406a8a0 2023-12-19T18:23:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.007+0000 7fd2e9ffb640 1 -- 172.21.15.27:0/2240992745 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2f000ce70 con 0x7fd2f406a8a0 2023-12-19T18:23:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.007+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/2240992745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2f40ca3b0 con 0x7fd2f406a8a0 2023-12-19T18:23:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.008+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/2240992745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2f415df90 con 0x7fd2f406a8a0 2023-12-19T18:23:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.009+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/2240992745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2f4068e90 con 0x7fd2f406a8a0 2023-12-19T18:23:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.010+0000 7fd2e9ffb640 1 -- 172.21.15.27:0/2240992745 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd2f00113f0 con 0x7fd2f406a8a0 2023-12-19T18:23:58.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.011+0000 7fd2e9ffb640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd2d806a460 0x7fd2d806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:58.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.011+0000 7fd2e9ffb640 1 -- 172.21.15.27:0/2240992745 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5386+0+0 (secure 0 0 0) 0x7fd2f00888f0 con 0x7fd2f406a8a0 2023-12-19T18:23:58.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.011+0000 7fd2e9ffb640 1 -- 172.21.15.27:0/2240992745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=161}) v3 -- 0x7fd2d8071290 con 0x7fd2f406a8a0 2023-12-19T18:23:58.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.011+0000 7fd2f8b8b640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd2d806a460 0x7fd2d806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:58.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.013+0000 7fd2f8b8b640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd2d806a460 0x7fd2d806c920 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4153ec0 tx=0x7fd2e4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:58.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.015+0000 7fd2e9ffb640 1 -- 172.21.15.27:0/2240992745 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd2f001f060 con 0x7fd2f406a8a0 2023-12-19T18:23:58.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.212+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/2240992745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd2f4156c10 con 0x7fd2f406a8a0 2023-12-19T18:23:58.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.731+0000 7fd2e9ffb640 1 -- 172.21.15.27:0/2240992745 <== mon.0 v2:172.21.15.27:3300/0 7 ==== osd_map(161..161 src has 1..161) v4 ==== 313+0+0 (secure 0 0 0) 0x7fd2f00583d0 con 0x7fd2f406a8a0 2023-12-19T18:23:58.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.731+0000 7fd2e9ffb640 1 -- 172.21.15.27:0/2240992745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=162}) v3 -- 0x7fd2d8070020 con 0x7fd2f406a8a0 2023-12-19T18:23:58.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.737+0000 7fd2e9ffb640 1 -- 172.21.15.27:0/2240992745 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v161) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd2f0054010 con 0x7fd2f406a8a0 2023-12-19T18:23:58.738 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-12-19T18:23:58.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.743+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd2d806a460 msgr2=0x7fd2d806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.743+0000 7fd2fb5f0640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd2d806a460 0x7fd2d806c920 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fd2f4153ec0 tx=0x7fd2e4009040 comp rx=0 tx=0).stop 2023-12-19T18:23:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.744+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/2240992745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f406a8a0 msgr2=0x7fd2f41603c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:58.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.744+0000 7fd2fb5f0640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f406a8a0 0x7fd2f41603c0 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7fd2f000c820 tx=0x7fd2f000cc20 comp rx=0 tx=0).stop 2023-12-19T18:23:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.747+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/2240992745 shutdown_connections 2023-12-19T18:23:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.747+0000 7fd2fb5f0640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd2d806a460 0x7fd2d806c920 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.747+0000 7fd2fb5f0640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2f415d060 0x7fd2f4158120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.747+0000 7fd2fb5f0640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2f40b6270 0x7fd2f414ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:58.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.747+0000 7fd2fb5f0640 1 --2- 172.21.15.27:0/2240992745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2f406a8a0 0x7fd2f41603c0 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.747+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/2240992745 >> 172.21.15.27:0/2240992745 conn(0x7fd2f40cfa10 msgr2=0x7fd2f40b7e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.748+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/2240992745 shutdown_connections 2023-12-19T18:23:58.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.748+0000 7fd2fb5f0640 1 -- 172.21.15.27:0/2240992745 wait complete. 2023-12-19T18:23:58.775 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:23:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.988+0000 7f6c6355c640 1 Processor -- start 2023-12-19T18:23:58.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.988+0000 7f6c6355c640 1 -- start start 2023-12-19T18:23:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.989+0000 7f6c6355c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6c5c159de0 0x7f6c5c0b1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.989+0000 7f6c6355c640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5c15d810 con 0x7f6c5c15df80 2023-12-19T18:23:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.990+0000 7f6c6355c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c5c0ca0c0 con 0x7f6c5c159a00 2023-12-19T18:23:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.990+0000 7f6c6355c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c5c0ca3a0 con 0x7f6c5c159de0 2023-12-19T18:23:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.990+0000 7f6c61af9640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6c5c15df80 0x7f6c5c06c0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34894/0 (socket says 172.21.15.27:34894) 2023-12-19T18:23:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.990+0000 7f6c61af9640 1 -- 172.21.15.27:0/2985809612 learned_addr learned my addr 172.21.15.27:0/2985809612 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:58.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.990+0000 7f6c60af7640 1 --2- 172.21.15.27:0/2985809612 >> v2:172.21.15.27:3300/0 conn(0x7f6c5c159de0 0x7f6c5c0b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.990+0000 7f6c53fff640 1 -- 172.21.15.27:0/2985809612 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3243330044 0 0) 0x7f6c5c15d810 con 0x7f6c5c15df80 2023-12-19T18:23:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.990+0000 7f6c53fff640 1 -- 172.21.15.27:0/2985809612 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c48003520 con 0x7f6c5c15df80 2023-12-19T18:23:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.991+0000 7f6c60af7640 1 -- 172.21.15.27:0/2985809612 >> v1:172.21.15.192:6789/0 conn(0x7f6c5c159a00 legacy=0x7f6c5c0c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.991+0000 7f6c60af7640 1 -- 172.21.15.27:0/2985809612 >> v1:172.21.15.27:6789/0 conn(0x7f6c5c15df80 legacy=0x7f6c5c06c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.991+0000 7f6c60af7640 1 -- 172.21.15.27:0/2985809612 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c0ca680 con 0x7f6c5c159de0 2023-12-19T18:23:58.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.992+0000 7f6c60af7640 1 --2- 172.21.15.27:0/2985809612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c5c159de0 0x7f6c5c0b1050 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0097a0 tx=0x7f6c4c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4f96c0272624da4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:58.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.992+0000 7f6c53fff640 1 -- 172.21.15.27:0/2985809612 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c030610 con 0x7f6c5c159de0 2023-12-19T18:23:58.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.992+0000 7f6c53fff640 1 -- 172.21.15.27:0/2985809612 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c030770 con 0x7f6c5c159de0 2023-12-19T18:23:58.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.992+0000 7f6c53fff640 1 -- 172.21.15.27:0/2985809612 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c030b00 con 0x7f6c5c159de0 2023-12-19T18:23:58.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.993+0000 7f6c6355c640 1 -- 172.21.15.27:0/2985809612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c5c159de0 msgr2=0x7f6c5c0b1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:58.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.993+0000 7f6c6355c640 1 --2- 172.21.15.27:0/2985809612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c5c159de0 0x7f6c5c0b1050 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c0097a0 tx=0x7f6c4c02b960 comp rx=0 tx=0).stop 2023-12-19T18:23:58.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.993+0000 7f6c6355c640 1 -- 172.21.15.27:0/2985809612 shutdown_connections 2023-12-19T18:23:58.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.993+0000 7f6c6355c640 1 --2- 172.21.15.27:0/2985809612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c5c159de0 0x7f6c5c0b1050 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:58.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.993+0000 7f6c6355c640 1 -- 172.21.15.27:0/2985809612 >> 172.21.15.27:0/2985809612 conn(0x7f6c5c0cb5b0 msgr2=0x7f6c5c0cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:58.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.993+0000 7f6c6355c640 1 -- 172.21.15.27:0/2985809612 shutdown_connections 2023-12-19T18:23:58.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.993+0000 7f6c6355c640 1 -- 172.21.15.27:0/2985809612 wait complete. 2023-12-19T18:23:58.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.995+0000 7f6c6355c640 1 Processor -- start 2023-12-19T18:23:58.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.995+0000 7f6c6355c640 1 -- start start 2023-12-19T18:23:58.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.996+0000 7f6c6355c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c5c159a00 0x7f6c5c14af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:58.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.996+0000 7f6c6355c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c5c159de0 0x7f6c5c0c2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:58.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.996+0000 7f6c6355c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c5c15df80 0x7f6c5c0cdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:58.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.996+0000 7f6c6355c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c15d170 con 0x7f6c5c159de0 2023-12-19T18:23:58.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.996+0000 7f6c6355c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c15c950 con 0x7f6c5c159a00 2023-12-19T18:23:58.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.996+0000 7f6c6355c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c5c15cbf0 con 0x7f6c5c15df80 2023-12-19T18:23:58.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.996+0000 7f6c60af7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c5c159de0 0x7f6c5c0c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:58.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.996+0000 7f6c60af7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c5c159de0 0x7f6c5c0c2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50182/0 (socket says 172.21.15.27:50182) 2023-12-19T18:23:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.996+0000 7f6c60af7640 1 -- 172.21.15.27:0/1238300511 learned_addr learned my addr 172.21.15.27:0/1238300511 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.996+0000 7f6c612f8640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c5c159a00 0x7f6c5c14af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.996+0000 7f6c61af9640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c5c15df80 0x7f6c5c0cdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.997+0000 7f6c60af7640 1 -- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c5c15df80 msgr2=0x7f6c5c0cdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.997+0000 7f6c60af7640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c5c15df80 0x7f6c5c0cdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.997+0000 7f6c60af7640 1 -- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c5c159a00 msgr2=0x7f6c5c14af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.997+0000 7f6c60af7640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c5c159a00 0x7f6c5c14af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.997+0000 7f6c60af7640 1 -- 172.21.15.27:0/1238300511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c009400 con 0x7f6c5c159de0 2023-12-19T18:23:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.997+0000 7f6c61af9640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c5c15df80 0x7f6c5c0cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.997+0000 7f6c60af7640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c5c159de0 0x7f6c5c0c2660 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c02bf70 tx=0x7f6c4c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:58.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.997+0000 7f6c51ffb640 1 -- 172.21.15.27:0/1238300511 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c03e040 con 0x7f6c5c159de0 2023-12-19T18:23:59.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.997+0000 7f6c51ffb640 1 -- 172.21.15.27:0/1238300511 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c039cb0 con 0x7f6c5c159de0 2023-12-19T18:23:59.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.997+0000 7f6c51ffb640 1 -- 172.21.15.27:0/1238300511 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c044630 con 0x7f6c5c159de0 2023-12-19T18:23:59.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.997+0000 7f6c6355c640 1 -- 172.21.15.27:0/1238300511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c5c0c6ea0 con 0x7f6c5c159de0 2023-12-19T18:23:59.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.998+0000 7f6c6355c640 1 -- 172.21.15.27:0/1238300511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c5c15adf0 con 0x7f6c5c159de0 2023-12-19T18:23:59.000 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:58.999+0000 7f6c6355c640 1 -- 172.21.15.27:0/1238300511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c5c15ac50 con 0x7f6c5c159de0 2023-12-19T18:23:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.000+0000 7f6c51ffb640 1 -- 172.21.15.27:0/1238300511 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6c4c043390 con 0x7f6c5c159de0 2023-12-19T18:23:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.001+0000 7f6c51ffb640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c4006a430 0x7f6c4006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.001+0000 7f6c51ffb640 1 -- 172.21.15.27:0/1238300511 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6c4c0aa730 con 0x7f6c5c159de0 2023-12-19T18:23:59.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.001+0000 7f6c612f8640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c4006a430 0x7f6c4006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:59.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.002+0000 7f6c612f8640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c4006a430 0x7f6c4006c8f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6c580012e0 tx=0x7f6c58009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:59.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.005+0000 7f6c51ffb640 1 -- 172.21.15.27:0/1238300511 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6c5c15ac50 con 0x7f6c5c159de0 2023-12-19T18:23:59.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.199+0000 7f6c6355c640 1 -- 172.21.15.27:0/1238300511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6c5c0c7030 con 0x7f6c5c159de0 2023-12-19T18:23:59.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.205+0000 7f6c51ffb640 1 -- 172.21.15.27:0/1238300511 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6c5c0c7030 con 0x7f6c5c159de0 2023-12-19T18:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.207+0000 7f6c6355c640 1 -- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c4006a430 msgr2=0x7f6c4006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.207+0000 7f6c6355c640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c4006a430 0x7f6c4006c8f0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f6c580012e0 tx=0x7f6c58009040 comp rx=0 tx=0).stop 2023-12-19T18:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.207+0000 7f6c6355c640 1 -- 172.21.15.27:0/1238300511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c5c159de0 msgr2=0x7f6c5c0c2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.207+0000 7f6c6355c640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c5c159de0 0x7f6c5c0c2660 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f6c4c02bf70 tx=0x7f6c4c004730 comp rx=0 tx=0).stop 2023-12-19T18:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.207+0000 7f6c6355c640 1 -- 172.21.15.27:0/1238300511 shutdown_connections 2023-12-19T18:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.207+0000 7f6c6355c640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6c4006a430 0x7f6c4006c8f0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.207+0000 7f6c6355c640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c5c15df80 0x7f6c5c0cdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.207+0000 7f6c6355c640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c5c159de0 0x7f6c5c0c2660 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.207+0000 7f6c6355c640 1 --2- 172.21.15.27:0/1238300511 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c5c159a00 0x7f6c5c14af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.207+0000 7f6c6355c640 1 -- 172.21.15.27:0/1238300511 >> 172.21.15.27:0/1238300511 conn(0x7f6c5c0cb5b0 msgr2=0x7f6c5c0ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.208+0000 7f6c6355c640 1 -- 172.21.15.27:0/1238300511 shutdown_connections 2023-12-19T18:23:59.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.208+0000 7f6c6355c640 1 -- 172.21.15.27:0/1238300511 wait complete. 2023-12-19T18:23:59.228 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file' 2023-12-19T18:23:59.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.462+0000 7f2f2324d640 1 Processor -- start 2023-12-19T18:23:59.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.463+0000 7f2f2324d640 1 -- start start 2023-12-19T18:23:59.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.464+0000 7f2f2324d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f1c0d2350 0x7f2f1c0b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:59.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.464+0000 7f2f2324d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c0b5e90 con 0x7f2f1c0b4ef0 2023-12-19T18:23:59.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.464+0000 7f2f2324d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f1c0d2850 con 0x7f2f1c1604d0 2023-12-19T18:23:59.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.464+0000 7f2f2324d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0d2b30 con 0x7f2f1c0d2350 2023-12-19T18:23:59.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.465+0000 7f2f21a4a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2f1c0b4ef0 0x7f2f1c0b6300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34900/0 (socket says 172.21.15.27:34900) 2023-12-19T18:23:59.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.465+0000 7f2f21a4a640 1 -- 172.21.15.27:0/1367932748 learned_addr learned my addr 172.21.15.27:0/1367932748 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:23:59.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.465+0000 7f2f22a4c640 1 --2- 172.21.15.27:0/1367932748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f1c0d2350 0x7f2f1c0b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:59.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.465+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2861797740 0 0) 0x7f2f1c0b5e90 con 0x7f2f1c0b4ef0 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.465+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f04003520 con 0x7f2f1c0b4ef0 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.465+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 327063484 0 0) 0x7f2f04003520 con 0x7f2f1c0b4ef0 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.465+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f1c0d2350 msgr2=0x7f2f1c0b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.465+0000 7f2f21249640 1 --2- 172.21.15.27:0/1367932748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f1c0d2350 0x7f2f1c0b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.465+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 >> v1:172.21.15.192:6789/0 conn(0x7f2f1c1604d0 legacy=0x7f2f1c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c0cded0 con 0x7f2f1c0b4ef0 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f2f10001670 con 0x7f2f1c0b4ef0 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 >> v1:172.21.15.27:6789/0 conn(0x7f2f1c0b4ef0 legacy=0x7f2f1c0b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f21249640 1 --2- 172.21.15.27:0/1367932748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f040039a0 0x7f2f04003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f21249640 1 --2- 172.21.15.27:0/1367932748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f040047c0 0x7f2f04004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f21249640 1 --2- 172.21.15.27:0/1367932748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f1c0d2350 0x7f2f04005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0d2b30 con 0x7f2f040039a0 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0400ba60 con 0x7f2f040047c0 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f0400bc20 con 0x7f2f1c0d2350 2023-12-19T18:23:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f2224b640 1 --2- 172.21.15.27:0/1367932748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f040039a0 0x7f2f04003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:59.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f21a4a640 1 --2- 172.21.15.27:0/1367932748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f040047c0 0x7f2f04004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:59.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f22a4c640 1 --2- 172.21.15.27:0/1367932748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f1c0d2350 0x7f2f04005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:59.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f2224b640 1 -- 172.21.15.27:0/1367932748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f1c0d2350 msgr2=0x7f2f04005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:59.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f2224b640 1 --2- 172.21.15.27:0/1367932748 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f1c0d2350 0x7f2f04005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f2224b640 1 -- 172.21.15.27:0/1367932748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f040047c0 msgr2=0x7f2f04004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:23:59.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f2224b640 1 --2- 172.21.15.27:0/1367932748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f040047c0 0x7f2f04004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f2224b640 1 -- 172.21.15.27:0/1367932748 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f04003520 con 0x7f2f040039a0 2023-12-19T18:23:59.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.466+0000 7f2f21a4a640 1 --2- 172.21.15.27:0/1367932748 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f040047c0 0x7f2f04004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:23:59.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.468+0000 7f2f2224b640 1 --2- 172.21.15.27:0/1367932748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f040039a0 0x7f2f04003dc0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c001e20 tx=0x7f2f040057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a339e238cd810273 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:59.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.468+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c003b70 con 0x7f2f040039a0 2023-12-19T18:23:59.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.468+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0c003cd0 con 0x7f2f040039a0 2023-12-19T18:23:59.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.468+0000 7f2f21249640 1 -- 172.21.15.27:0/1367932748 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c005310 con 0x7f2f040039a0 2023-12-19T18:23:59.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.468+0000 7f2f2324d640 1 -- 172.21.15.27:0/1367932748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f040039a0 msgr2=0x7f2f04003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:59.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.468+0000 7f2f2324d640 1 --2- 172.21.15.27:0/1367932748 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f040039a0 0x7f2f04003dc0 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c001e20 tx=0x7f2f040057f0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.468+0000 7f2f22a4c640 1 -- 172.21.15.27:0/1367932748 reap_dead start 2023-12-19T18:23:59.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.469+0000 7f2f2324d640 1 -- 172.21.15.27:0/1367932748 shutdown_connections 2023-12-19T18:23:59.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.469+0000 7f2f2324d640 1 -- 172.21.15.27:0/1367932748 >> 172.21.15.27:0/1367932748 conn(0x7f2f1c0cf7e0 msgr2=0x7f2f1c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:59.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.469+0000 7f2f2324d640 1 -- 172.21.15.27:0/1367932748 shutdown_connections 2023-12-19T18:23:59.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.469+0000 7f2f2324d640 1 -- 172.21.15.27:0/1367932748 wait complete. 2023-12-19T18:23:59.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.470+0000 7f2f2324d640 1 Processor -- start 2023-12-19T18:23:59.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.471+0000 7f2f2324d640 1 -- start start 2023-12-19T18:23:59.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2324d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f040039a0 0x7f2f1c0c68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:59.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2324d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f1c158ad0 0x7f2f1c1f1c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:59.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2324d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f1c1f2150 0x7f2f1c0c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:59.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2324d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0cc5b0 con 0x7f2f040039a0 2023-12-19T18:23:59.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2324d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0c6e20 con 0x7f2f1c158ad0 2023-12-19T18:23:59.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2324d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1c0cb930 con 0x7f2f1c1f2150 2023-12-19T18:23:59.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2224b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f040039a0 0x7f2f1c0c68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:59.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2224b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f040039a0 0x7f2f1c0c68e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50200/0 (socket says 172.21.15.27:50200) 2023-12-19T18:23:59.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2224b640 1 -- 172.21.15.27:0/4232277980 learned_addr learned my addr 172.21.15.27:0/4232277980 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:23:59.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2224b640 1 -- 172.21.15.27:0/4232277980 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f1c1f2150 msgr2=0x7f2f1c0c8240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:59.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2224b640 1 --2- 172.21.15.27:0/4232277980 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f1c1f2150 0x7f2f1c0c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f21a4a640 1 --2- 172.21.15.27:0/4232277980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f1c158ad0 0x7f2f1c1f1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:59.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2224b640 1 -- 172.21.15.27:0/4232277980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f1c158ad0 msgr2=0x7f2f1c1f1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:59.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2224b640 1 --2- 172.21.15.27:0/4232277980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f1c158ad0 0x7f2f1c1f1c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.472+0000 7f2f2224b640 1 -- 172.21.15.27:0/4232277980 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c0cbc10 con 0x7f2f040039a0 2023-12-19T18:23:59.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.473+0000 7f2f2224b640 1 --2- 172.21.15.27:0/4232277980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f040039a0 0x7f2f1c0c68e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c02bf50 tx=0x7f2f0c0014a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:59.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.473+0000 7f2f1affd640 1 -- 172.21.15.27:0/4232277980 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c03e070 con 0x7f2f040039a0 2023-12-19T18:23:59.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.473+0000 7f2f1affd640 1 -- 172.21.15.27:0/4232277980 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0c006b50 con 0x7f2f040039a0 2023-12-19T18:23:59.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.473+0000 7f2f1affd640 1 -- 172.21.15.27:0/4232277980 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0c007030 con 0x7f2f040039a0 2023-12-19T18:23:59.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.473+0000 7f2f2324d640 1 -- 172.21.15.27:0/4232277980 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f1c0cbef0 con 0x7f2f040039a0 2023-12-19T18:23:59.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.473+0000 7f2f2324d640 1 -- 172.21.15.27:0/4232277980 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f1c158f50 con 0x7f2f040039a0 2023-12-19T18:23:59.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.474+0000 7f2f2324d640 1 -- 172.21.15.27:0/4232277980 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1c066f80 con 0x7f2f040039a0 2023-12-19T18:23:59.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.475+0000 7f2f1affd640 1 -- 172.21.15.27:0/4232277980 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2f0c007190 con 0x7f2f040039a0 2023-12-19T18:23:59.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.476+0000 7f2f1affd640 1 --2- 172.21.15.27:0/4232277980 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2f0806a430 0x7f2f0806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:23:59.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.476+0000 7f2f1affd640 1 -- 172.21.15.27:0/4232277980 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f2f0c0aa120 con 0x7f2f040039a0 2023-12-19T18:23:59.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.477+0000 7f2f21a4a640 1 --2- 172.21.15.27:0/4232277980 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2f0806a430 0x7f2f0806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:23:59.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.478+0000 7f2f21a4a640 1 --2- 172.21.15.27:0/4232277980 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2f0806a430 0x7f2f0806c8f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0ceb90 tx=0x7f2f10009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:23:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.481+0000 7f2f1affd640 1 -- 172.21.15.27:0/4232277980 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2f0c075a10 con 0x7f2f040039a0 2023-12-19T18:23:59.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.745+0000 7f2f2324d640 1 -- 172.21.15.27:0/4232277980 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]} v 0) v1 -- 0x7f2f1c04e840 con 0x7f2f040039a0 2023-12-19T18:23:59.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.762+0000 7f2f1affd640 1 -- 172.21.15.27:0/4232277980 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_different_file"]}]=0 v466) v1 ==== 140+0+0 (secure 0 0 0) 0x7f2f0c079030 con 0x7f2f040039a0 2023-12-19T18:23:59.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.765+0000 7f2f2324d640 1 -- 172.21.15.27:0/4232277980 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2f0806a430 msgr2=0x7f2f0806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:59.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.765+0000 7f2f2324d640 1 --2- 172.21.15.27:0/4232277980 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2f0806a430 0x7f2f0806c8f0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f2f1c0ceb90 tx=0x7f2f10009040 comp rx=0 tx=0).stop 2023-12-19T18:23:59.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.765+0000 7f2f2324d640 1 -- 172.21.15.27:0/4232277980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f040039a0 msgr2=0x7f2f1c0c68e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:23:59.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.765+0000 7f2f2324d640 1 --2- 172.21.15.27:0/4232277980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f040039a0 0x7f2f1c0c68e0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f2f0c02bf50 tx=0x7f2f0c0014a0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.765+0000 7f2f2324d640 1 -- 172.21.15.27:0/4232277980 shutdown_connections 2023-12-19T18:23:59.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.765+0000 7f2f2324d640 1 --2- 172.21.15.27:0/4232277980 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2f0806a430 0x7f2f0806c8f0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.765+0000 7f2f2324d640 1 --2- 172.21.15.27:0/4232277980 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f1c1f2150 0x7f2f1c0c8240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.765+0000 7f2f2324d640 1 --2- 172.21.15.27:0/4232277980 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f1c158ad0 0x7f2f1c1f1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.765+0000 7f2f2324d640 1 --2- 172.21.15.27:0/4232277980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f040039a0 0x7f2f1c0c68e0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:23:59.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.765+0000 7f2f2324d640 1 -- 172.21.15.27:0/4232277980 >> 172.21.15.27:0/4232277980 conn(0x7f2f1c0cf7e0 msgr2=0x7f2f1c0d18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:23:59.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.765+0000 7f2f2324d640 1 -- 172.21.15.27:0/4232277980 shutdown_connections 2023-12-19T18:23:59.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:23:59.765+0000 7f2f2324d640 1 -- 172.21.15.27:0/4232277980 wait complete. 2023-12-19T18:23:59.786 INFO:tasks.cephfs_test_runner:test_full_different_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-12-19T18:23:59.786 INFO:tasks.cephfs_test_runner:Starting test: test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) 2023-12-19T18:23:59.787 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-12-19T18:24:00.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.015+0000 7fb07b127640 1 Processor -- start 2023-12-19T18:24:00.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.015+0000 7fb07b127640 1 -- start start 2023-12-19T18:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.016+0000 7fb07b127640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb0741604d0 0x7fb0740b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.016+0000 7fb07b127640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb0740b7310 0x7fb0740b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.016+0000 7fb07b127640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0740b5e90 con 0x7fb0740d2350 2023-12-19T18:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.016+0000 7fb07b127640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0740d2b30 con 0x7fb0741604d0 2023-12-19T18:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.016+0000 7fb07b127640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0740d2850 con 0x7fb0740b7310 2023-12-19T18:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.017+0000 7fb079924640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb0740d2350 0x7fb0740d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34908/0 (socket says 172.21.15.27:34908) 2023-12-19T18:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.017+0000 7fb079924640 1 -- 172.21.15.27:0/2948731201 learned_addr learned my addr 172.21.15.27:0/2948731201 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:00.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.017+0000 7fb07a125640 1 --2- 172.21.15.27:0/2948731201 >> v2:172.21.15.27:3300/0 conn(0x7fb0741604d0 0x7fb0740b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.017+0000 7fb07a926640 1 --2- 172.21.15.27:0/2948731201 >> v2:172.21.15.192:3300/0 conn(0x7fb0740b7310 0x7fb0740b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.017+0000 7fb07a125640 1 -- 172.21.15.27:0/2948731201 >> v2:172.21.15.192:3300/0 conn(0x7fb0740b7310 msgr2=0x7fb0740b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.017+0000 7fb07a125640 1 --2- 172.21.15.27:0/2948731201 >> v2:172.21.15.192:3300/0 conn(0x7fb0740b7310 0x7fb0740b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.017+0000 7fb079123640 1 -- 172.21.15.27:0/2948731201 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323386735 0 0) 0x7fb0740b5e90 con 0x7fb0740d2350 2023-12-19T18:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.017+0000 7fb07a125640 1 -- 172.21.15.27:0/2948731201 >> v1:172.21.15.27:6789/0 conn(0x7fb0740d2350 legacy=0x7fb0740d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:00.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.017+0000 7fb07a125640 1 -- 172.21.15.27:0/2948731201 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0740cded0 con 0x7fb0741604d0 2023-12-19T18:24:00.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.019+0000 7fb07a125640 1 --2- 172.21.15.27:0/2948731201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0741604d0 0x7fb0740b47e0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb0740cd540 tx=0x7fb07002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=204246553d2677f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:00.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.019+0000 7fb079123640 1 -- 172.21.15.27:0/2948731201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07002e070 con 0x7fb0741604d0 2023-12-19T18:24:00.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.019+0000 7fb079123640 1 -- 172.21.15.27:0/2948731201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0700315a0 con 0x7fb0741604d0 2023-12-19T18:24:00.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.019+0000 7fb079123640 1 -- 172.21.15.27:0/2948731201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb070031930 con 0x7fb0741604d0 2023-12-19T18:24:00.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.020+0000 7fb07b127640 1 -- 172.21.15.27:0/2948731201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0741604d0 msgr2=0x7fb0740b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:00.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.020+0000 7fb07b127640 1 --2- 172.21.15.27:0/2948731201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0741604d0 0x7fb0740b47e0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fb0740cd540 tx=0x7fb07002b840 comp rx=0 tx=0).stop 2023-12-19T18:24:00.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.020+0000 7fb07b127640 1 -- 172.21.15.27:0/2948731201 shutdown_connections 2023-12-19T18:24:00.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.020+0000 7fb07b127640 1 --2- 172.21.15.27:0/2948731201 >> v2:172.21.15.192:3300/0 conn(0x7fb0740b7310 0x7fb0740b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:00.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.020+0000 7fb07b127640 1 --2- 172.21.15.27:0/2948731201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0741604d0 0x7fb0740b47e0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:00.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.020+0000 7fb07b127640 1 -- 172.21.15.27:0/2948731201 >> 172.21.15.27:0/2948731201 conn(0x7fb0740cf7e0 msgr2=0x7fb0740cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:00.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.020+0000 7fb07b127640 1 -- 172.21.15.27:0/2948731201 shutdown_connections 2023-12-19T18:24:00.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.020+0000 7fb07b127640 1 -- 172.21.15.27:0/2948731201 wait complete. 2023-12-19T18:24:00.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.023+0000 7fb07b127640 1 Processor -- start 2023-12-19T18:24:00.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.023+0000 7fb07b127640 1 -- start start 2023-12-19T18:24:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.024+0000 7fb07b127640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0740b7310 0x7fb074155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.024+0000 7fb07b127640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0740d2350 0x7fb074158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.024+0000 7fb07b127640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0741604d0 0x7fb074153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.024+0000 7fb07b127640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb074156a30 con 0x7fb0740d2350 2023-12-19T18:24:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.024+0000 7fb07b127640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb07415c2a0 con 0x7fb0740b7310 2023-12-19T18:24:00.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.024+0000 7fb07b127640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb07415c540 con 0x7fb0741604d0 2023-12-19T18:24:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.024+0000 7fb079924640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0740d2350 0x7fb074158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.025+0000 7fb079924640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0740d2350 0x7fb074158a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50218/0 (socket says 172.21.15.27:50218) 2023-12-19T18:24:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.025+0000 7fb079924640 1 -- 172.21.15.27:0/792628823 learned_addr learned my addr 172.21.15.27:0/792628823 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.025+0000 7fb07a125640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0740b7310 0x7fb074155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.025+0000 7fb07a926640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0741604d0 0x7fb074153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.025+0000 7fb079924640 1 -- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0741604d0 msgr2=0x7fb074153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.025+0000 7fb079924640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0741604d0 0x7fb074153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.025+0000 7fb079924640 1 -- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0740b7310 msgr2=0x7fb074155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.025+0000 7fb079924640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0740b7310 0x7fb074155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.025+0000 7fb079924640 1 -- 172.21.15.27:0/792628823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb070009400 con 0x7fb0740d2350 2023-12-19T18:24:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.025+0000 7fb07a125640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0740b7310 0x7fb074155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:00.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.025+0000 7fb079924640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0740d2350 0x7fb074158a10 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fb06800c4b0 tx=0x7fb06800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.026+0000 7fb066ffd640 1 -- 172.21.15.27:0/792628823 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0680074e0 con 0x7fb0740d2350 2023-12-19T18:24:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.026+0000 7fb066ffd640 1 -- 172.21.15.27:0/792628823 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb068007660 con 0x7fb0740d2350 2023-12-19T18:24:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.026+0000 7fb066ffd640 1 -- 172.21.15.27:0/792628823 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb068004a20 con 0x7fb0740d2350 2023-12-19T18:24:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.026+0000 7fb07b127640 1 -- 172.21.15.27:0/792628823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb07415c820 con 0x7fb0740d2350 2023-12-19T18:24:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.026+0000 7fb07b127640 1 -- 172.21.15.27:0/792628823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb074154e50 con 0x7fb0740d2350 2023-12-19T18:24:00.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.027+0000 7fb07b127640 1 -- 172.21.15.27:0/792628823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb074066f80 con 0x7fb0740d2350 2023-12-19T18:24:00.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.028+0000 7fb066ffd640 1 -- 172.21.15.27:0/792628823 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb068004b80 con 0x7fb0740d2350 2023-12-19T18:24:00.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.029+0000 7fb066ffd640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb05806a430 0x7fb05806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:00.029 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.029+0000 7fb066ffd640 1 -- 172.21.15.27:0/792628823 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb068087840 con 0x7fb0740d2350 2023-12-19T18:24:00.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.030+0000 7fb07a125640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb05806a430 0x7fb05806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:00.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.031+0000 7fb07a125640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb05806a430 0x7fb05806c8f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb070036000 tx=0x7fb07003d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:00.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.035+0000 7fb066ffd640 1 -- 172.21.15.27:0/792628823 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb0680531e0 con 0x7fb0740d2350 2023-12-19T18:24:00.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.302+0000 7fb07b127640 1 -- 172.21.15.27:0/792628823 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7fb07404e840 con 0x7fb0740d2350 2023-12-19T18:24:00.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.773+0000 7fb066ffd640 1 -- 172.21.15.27:0/792628823 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v467) v1 ==== 135+0+0 (secure 0 0 0) 0x7fb0680180c0 con 0x7fb0740d2350 2023-12-19T18:24:00.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.777+0000 7fb07b127640 1 -- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb05806a430 msgr2=0x7fb05806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:00.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.777+0000 7fb07b127640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb05806a430 0x7fb05806c8f0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fb070036000 tx=0x7fb07003d040 comp rx=0 tx=0).stop 2023-12-19T18:24:00.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.777+0000 7fb07b127640 1 -- 172.21.15.27:0/792628823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0740d2350 msgr2=0x7fb074158a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:00.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.777+0000 7fb07b127640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0740d2350 0x7fb074158a10 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7fb06800c4b0 tx=0x7fb06800c970 comp rx=0 tx=0).stop 2023-12-19T18:24:00.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.777+0000 7fb07b127640 1 -- 172.21.15.27:0/792628823 shutdown_connections 2023-12-19T18:24:00.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.777+0000 7fb07b127640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb05806a430 0x7fb05806c8f0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:00.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.777+0000 7fb07b127640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0741604d0 0x7fb074153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:00.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.777+0000 7fb07b127640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0740d2350 0x7fb074158a10 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:00.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.777+0000 7fb07b127640 1 --2- 172.21.15.27:0/792628823 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0740b7310 0x7fb074155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:00.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.777+0000 7fb07b127640 1 -- 172.21.15.27:0/792628823 >> 172.21.15.27:0/792628823 conn(0x7fb0740cf7e0 msgr2=0x7fb0740cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:00.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.777+0000 7fb07b127640 1 -- 172.21.15.27:0/792628823 shutdown_connections 2023-12-19T18:24:00.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:00.777+0000 7fb07b127640 1 -- 172.21.15.27:0/792628823 wait complete. 2023-12-19T18:24:00.802 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:24:01.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.042+0000 7fbf6ad8a640 1 Processor -- start 2023-12-19T18:24:01.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.042+0000 7fbf6ad8a640 1 -- start start 2023-12-19T18:24:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.043+0000 7fbf6ad8a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf6415cf80 0x7fbf640d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.043+0000 7fbf6ad8a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbf640b85e0 0x7fbf640cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.043+0000 7fbf6ad8a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbf640b8a60 0x7fbf640cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.043+0000 7fbf6ad8a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf640d0e10 con 0x7fbf640b85e0 2023-12-19T18:24:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.043+0000 7fbf6ad8a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf640d1c70 con 0x7fbf640b8a60 2023-12-19T18:24:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.043+0000 7fbf6ad8a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf6415d360 con 0x7fbf6415cf80 2023-12-19T18:24:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.044+0000 7fbf69587640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbf640b85e0 0x7fbf640cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.044+0000 7fbf69587640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbf640b85e0 0x7fbf640cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50234/0 (socket says 172.21.15.27:50234) 2023-12-19T18:24:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.044+0000 7fbf69587640 1 -- 172.21.15.27:0/859601662 learned_addr learned my addr 172.21.15.27:0/859601662 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.044+0000 7fbf6a589640 1 --2- 172.21.15.27:0/859601662 >> v2:172.21.15.192:3300/0 conn(0x7fbf640b8a60 0x7fbf640cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.044+0000 7fbf69d88640 1 --2- 172.21.15.27:0/859601662 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf6415cf80 0x7fbf640d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.044+0000 7fbf69587640 1 -- 172.21.15.27:0/859601662 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf6415cf80 msgr2=0x7fbf640d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.044+0000 7fbf69587640 1 --2- 172.21.15.27:0/859601662 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf6415cf80 0x7fbf640d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.044+0000 7fbf69587640 1 -- 172.21.15.27:0/859601662 >> v2:172.21.15.192:3300/0 conn(0x7fbf640b8a60 msgr2=0x7fbf640cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.044+0000 7fbf69587640 1 --2- 172.21.15.27:0/859601662 >> v2:172.21.15.192:3300/0 conn(0x7fbf640b8a60 0x7fbf640cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:01.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.044+0000 7fbf69587640 1 -- 172.21.15.27:0/859601662 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf640d1060 con 0x7fbf640b85e0 2023-12-19T18:24:01.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.044+0000 7fbf6a589640 1 --2- 172.21.15.27:0/859601662 >> v2:172.21.15.192:3300/0 conn(0x7fbf640b8a60 0x7fbf640cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:01.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.044+0000 7fbf69d88640 1 --2- 172.21.15.27:0/859601662 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf6415cf80 0x7fbf640d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.046+0000 7fbf69587640 1 --2- 172.21.15.27:0/859601662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf640b85e0 0x7fbf640cda40 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fbf64073bb0 tx=0x7fbf5802b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77b21c813e1444f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.046+0000 7fbf68d86640 1 -- 172.21.15.27:0/859601662 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58030850 con 0x7fbf640b85e0 2023-12-19T18:24:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.046+0000 7fbf68d86640 1 -- 172.21.15.27:0/859601662 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf580309b0 con 0x7fbf640b85e0 2023-12-19T18:24:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.046+0000 7fbf68d86640 1 -- 172.21.15.27:0/859601662 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58030d40 con 0x7fbf640b85e0 2023-12-19T18:24:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.047+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/859601662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf640b85e0 msgr2=0x7fbf640cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.047+0000 7fbf6ad8a640 1 --2- 172.21.15.27:0/859601662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf640b85e0 0x7fbf640cda40 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7fbf64073bb0 tx=0x7fbf5802b990 comp rx=0 tx=0).stop 2023-12-19T18:24:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.047+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/859601662 shutdown_connections 2023-12-19T18:24:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.047+0000 7fbf6ad8a640 1 --2- 172.21.15.27:0/859601662 >> v2:172.21.15.192:3300/0 conn(0x7fbf640b8a60 0x7fbf640cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.047+0000 7fbf6ad8a640 1 --2- 172.21.15.27:0/859601662 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf640b85e0 0x7fbf640cda40 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.047+0000 7fbf6ad8a640 1 --2- 172.21.15.27:0/859601662 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf6415cf80 0x7fbf640d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.047+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/859601662 >> 172.21.15.27:0/859601662 conn(0x7fbf64072e20 msgr2=0x7fbf64073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:01.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.047+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/859601662 shutdown_connections 2023-12-19T18:24:01.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.047+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/859601662 wait complete. 2023-12-19T18:24:01.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.049+0000 7fbf6ad8a640 1 Processor -- start 2023-12-19T18:24:01.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.049+0000 7fbf6ad8a640 1 -- start start 2023-12-19T18:24:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.050+0000 7fbf6ad8a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf640b85e0 0x7fbf641f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.050+0000 7fbf6ad8a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6415cf80 0x7fbf641f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.050+0000 7fbf6ad8a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf640c93b0 0x7fbf640ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.050+0000 7fbf6ad8a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640cb2a0 con 0x7fbf6415cf80 2023-12-19T18:24:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.050+0000 7fbf6ad8a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640cb580 con 0x7fbf640b85e0 2023-12-19T18:24:01.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.050+0000 7fbf6ad8a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf640cb860 con 0x7fbf640c93b0 2023-12-19T18:24:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.050+0000 7fbf69587640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6415cf80 0x7fbf641f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.050+0000 7fbf69587640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6415cf80 0x7fbf641f5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:50240/0 (socket says 172.21.15.27:50240) 2023-12-19T18:24:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.050+0000 7fbf69587640 1 -- 172.21.15.27:0/518110761 learned_addr learned my addr 172.21.15.27:0/518110761 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.050+0000 7fbf6a589640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf640c93b0 0x7fbf640ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:01.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.051+0000 7fbf69d88640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf640b85e0 0x7fbf641f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.051+0000 7fbf69587640 1 -- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf640c93b0 msgr2=0x7fbf640ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.051+0000 7fbf69587640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf640c93b0 0x7fbf640ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.051+0000 7fbf69587640 1 -- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf640b85e0 msgr2=0x7fbf641f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.051+0000 7fbf69587640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf640b85e0 0x7fbf641f5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.051+0000 7fbf69587640 1 -- 172.21.15.27:0/518110761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf580094d0 con 0x7fbf6415cf80 2023-12-19T18:24:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.051+0000 7fbf6a589640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf640c93b0 0x7fbf640ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.051+0000 7fbf69587640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6415cf80 0x7fbf641f5d90 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fbf5802b9d0 tx=0x7fbf58004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.051+0000 7fbf627fc640 1 -- 172.21.15.27:0/518110761 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf58030840 con 0x7fbf6415cf80 2023-12-19T18:24:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.051+0000 7fbf627fc640 1 -- 172.21.15.27:0/518110761 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf58043400 con 0x7fbf6415cf80 2023-12-19T18:24:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.051+0000 7fbf627fc640 1 -- 172.21.15.27:0/518110761 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf580437b0 con 0x7fbf6415cf80 2023-12-19T18:24:01.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.051+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/518110761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf640cbb40 con 0x7fbf6415cf80 2023-12-19T18:24:01.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.052+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/518110761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf640caa90 con 0x7fbf6415cf80 2023-12-19T18:24:01.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.053+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/518110761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf6404e840 con 0x7fbf6415cf80 2023-12-19T18:24:01.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.054+0000 7fbf627fc640 1 -- 172.21.15.27:0/518110761 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbf5802e070 con 0x7fbf6415cf80 2023-12-19T18:24:01.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.055+0000 7fbf627fc640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbf5006a430 0x7fbf5006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:01.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.055+0000 7fbf627fc640 1 -- 172.21.15.27:0/518110761 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbf5803e070 con 0x7fbf6415cf80 2023-12-19T18:24:01.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.055+0000 7fbf69d88640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbf5006a430 0x7fbf5006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:01.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.057+0000 7fbf69d88640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbf5006a430 0x7fbf5006c8f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbf54007130 tx=0x7fbf54007060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:01.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.059+0000 7fbf627fc640 1 -- 172.21.15.27:0/518110761 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbf58077040 con 0x7fbf6415cf80 2023-12-19T18:24:01.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.245+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/518110761 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fbf640b2800 con 0x7fbf6415cf80 2023-12-19T18:24:01.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.251+0000 7fbf627fc640 1 -- 172.21.15.27:0/518110761 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbf5807a660 con 0x7fbf6415cf80 2023-12-19T18:24:01.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.254+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbf5006a430 msgr2=0x7fbf5006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:01.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.254+0000 7fbf6ad8a640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbf5006a430 0x7fbf5006c8f0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fbf54007130 tx=0x7fbf54007060 comp rx=0 tx=0).stop 2023-12-19T18:24:01.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.254+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/518110761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6415cf80 msgr2=0x7fbf641f5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:01.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.254+0000 7fbf6ad8a640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6415cf80 0x7fbf641f5d90 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7fbf5802b9d0 tx=0x7fbf58004730 comp rx=0 tx=0).stop 2023-12-19T18:24:01.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.254+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/518110761 shutdown_connections 2023-12-19T18:24:01.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.254+0000 7fbf6ad8a640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbf5006a430 0x7fbf5006c8f0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:01.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.254+0000 7fbf6ad8a640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf640c93b0 0x7fbf640ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:01.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.254+0000 7fbf6ad8a640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf6415cf80 0x7fbf641f5d90 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:01.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.254+0000 7fbf6ad8a640 1 --2- 172.21.15.27:0/518110761 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf640b85e0 0x7fbf641f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:01.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.254+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/518110761 >> 172.21.15.27:0/518110761 conn(0x7fbf64072e20 msgr2=0x7fbf640b73e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:01.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.254+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/518110761 shutdown_connections 2023-12-19T18:24:01.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:01.254+0000 7fbf6ad8a640 1 -- 172.21.15.27:0/518110761 wait complete. 2023-12-19T18:24:01.275 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:24:01.275 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:24:01.405 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:24:01.529 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:24:01.529 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:24:01.529 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:24:01.594 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:24:01.594 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:24:01.594 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:24:01.655 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:24:01.655 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:24:01.655 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:24:01.721 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:24:01.721 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:24:01.721 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:24:01.780 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:24:01.780 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:24:01.780 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:24:02.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.026+0000 7f082a155640 1 Processor -- start 2023-12-19T18:24:02.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.027+0000 7f082a155640 1 -- start start 2023-12-19T18:24:02.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.028+0000 7f082a155640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f08240b4c80 0x7f0824160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:02.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.028+0000 7f082a155640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f08240b5060 0x7f08240d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:02.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.028+0000 7f082a155640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08240d0d20 con 0x7f08240b8920 2023-12-19T18:24:02.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.028+0000 7f082a155640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08240b6130 con 0x7f08240b4c80 2023-12-19T18:24:02.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.028+0000 7f082a155640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0824161bf0 con 0x7f08240b5060 2023-12-19T18:24:02.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.029+0000 7f08237fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f08240b4c80 0x7f0824160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:02.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.029+0000 7f08237fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f08240b4c80 0x7f0824160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38040/0 (socket says 172.21.15.27:38040) 2023-12-19T18:24:02.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.029+0000 7f08237fe640 1 -- 172.21.15.27:0/1408242750 learned_addr learned my addr 172.21.15.27:0/1408242750 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:02.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.029+0000 7f0823fff640 1 --2- 172.21.15.27:0/1408242750 >> v2:172.21.15.192:3300/0 conn(0x7f08240b5060 0x7f08240d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:02.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.029+0000 7f08237fe640 1 -- 172.21.15.27:0/1408242750 >> v2:172.21.15.192:3300/0 conn(0x7f08240b5060 msgr2=0x7f08240d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:02.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.029+0000 7f08237fe640 1 --2- 172.21.15.27:0/1408242750 >> v2:172.21.15.192:3300/0 conn(0x7f08240b5060 0x7f08240d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.029+0000 7f08237fe640 1 -- 172.21.15.27:0/1408242750 >> v1:172.21.15.192:6789/0 conn(0x7f08240b8920 legacy=0x7f082415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:02.030 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.029+0000 7f08237fe640 1 -- 172.21.15.27:0/1408242750 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0824161350 con 0x7f08240b4c80 2023-12-19T18:24:02.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.031+0000 7f08237fe640 1 --2- 172.21.15.27:0/1408242750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08240b4c80 0x7f0824160c70 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f08240cf170 tx=0x7f081402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97e42455268ca890 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:02.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.031+0000 7f0822ffd640 1 -- 172.21.15.27:0/1408242750 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08140317e0 con 0x7f08240b4c80 2023-12-19T18:24:02.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.031+0000 7f0822ffd640 1 -- 172.21.15.27:0/1408242750 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0814031940 con 0x7f08240b4c80 2023-12-19T18:24:02.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.031+0000 7f0822ffd640 1 -- 172.21.15.27:0/1408242750 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0814031cd0 con 0x7f08240b4c80 2023-12-19T18:24:02.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.032+0000 7f082a155640 1 -- 172.21.15.27:0/1408242750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08240b4c80 msgr2=0x7f0824160c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:02.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.032+0000 7f082a155640 1 --2- 172.21.15.27:0/1408242750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08240b4c80 0x7f0824160c70 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f08240cf170 tx=0x7f081402b880 comp rx=0 tx=0).stop 2023-12-19T18:24:02.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.032+0000 7f082a155640 1 -- 172.21.15.27:0/1408242750 shutdown_connections 2023-12-19T18:24:02.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.032+0000 7f082a155640 1 --2- 172.21.15.27:0/1408242750 >> v2:172.21.15.192:3300/0 conn(0x7f08240b5060 0x7f08240d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.032+0000 7f082a155640 1 --2- 172.21.15.27:0/1408242750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08240b4c80 0x7f0824160c70 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.032+0000 7f082a155640 1 -- 172.21.15.27:0/1408242750 >> 172.21.15.27:0/1408242750 conn(0x7f08240d2f90 msgr2=0x7f08240d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:02.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.032+0000 7f082a155640 1 -- 172.21.15.27:0/1408242750 shutdown_connections 2023-12-19T18:24:02.032 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.032+0000 7f082a155640 1 -- 172.21.15.27:0/1408242750 wait complete. 2023-12-19T18:24:02.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.034+0000 7f082a155640 1 Processor -- start 2023-12-19T18:24:02.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.034+0000 7f082a155640 1 -- start start 2023-12-19T18:24:02.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.036+0000 7f082a155640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08240b4c80 0x7f082414a270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.036+0000 7f082a155640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08240b5060 0x7f08240cabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.036+0000 7f082a155640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08240b8920 0x7f08240cb120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.036+0000 7f082a155640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082415a7c0 con 0x7f08240b8920 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.036+0000 7f082a155640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082415aaa0 con 0x7f08240b4c80 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.036+0000 7f082a155640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08240ca100 con 0x7f08240b5060 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.036+0000 7f0823fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08240b8920 0x7f08240cb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.036+0000 7f0823fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08240b8920 0x7f08240cb120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38052/0 (socket says 172.21.15.27:38052) 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.036+0000 7f0823fff640 1 -- 172.21.15.27:0/3430051040 learned_addr learned my addr 172.21.15.27:0/3430051040 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.036+0000 7f08237fe640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08240b4c80 0x7f082414a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.036+0000 7f081bfff640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08240b5060 0x7f08240cabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.036+0000 7f0823fff640 1 -- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08240b5060 msgr2=0x7f08240cabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.036+0000 7f0823fff640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08240b5060 0x7f08240cabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.037+0000 7f0823fff640 1 -- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08240b4c80 msgr2=0x7f082414a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:02.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.037+0000 7f0823fff640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08240b4c80 0x7f082414a270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.037+0000 7f0823fff640 1 -- 172.21.15.27:0/3430051040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08140094d0 con 0x7f08240b8920 2023-12-19T18:24:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.037+0000 7f08237fe640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08240b4c80 0x7f082414a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.037+0000 7f081bfff640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08240b5060 0x7f08240cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.037+0000 7f0823fff640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08240b8920 0x7f08240cb120 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f081c007e30 tx=0x7f081c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.037+0000 7f0820ff9640 1 -- 172.21.15.27:0/3430051040 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c014620 con 0x7f08240b8920 2023-12-19T18:24:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.037+0000 7f0820ff9640 1 -- 172.21.15.27:0/3430051040 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f081c00ee90 con 0x7f08240b8920 2023-12-19T18:24:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.037+0000 7f0820ff9640 1 -- 172.21.15.27:0/3430051040 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f081c0149c0 con 0x7f08240b8920 2023-12-19T18:24:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.037+0000 7f082a155640 1 -- 172.21.15.27:0/3430051040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08240ca3e0 con 0x7f08240b8920 2023-12-19T18:24:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.037+0000 7f082a155640 1 -- 172.21.15.27:0/3430051040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082414a7b0 con 0x7f08240b8920 2023-12-19T18:24:02.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.039+0000 7f082a155640 1 -- 172.21.15.27:0/3430051040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08240ca900 con 0x7f08240b8920 2023-12-19T18:24:02.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.040+0000 7f0820ff9640 1 -- 172.21.15.27:0/3430051040 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f081c014b20 con 0x7f08240b8920 2023-12-19T18:24:02.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.040+0000 7f0820ff9640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f080406a430 0x7f080406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:02.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.041+0000 7f0820ff9640 1 -- 172.21.15.27:0/3430051040 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f081c088fb0 con 0x7f08240b8920 2023-12-19T18:24:02.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.041+0000 7f08237fe640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f080406a430 0x7f080406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:02.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.042+0000 7f08237fe640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f080406a430 0x7f080406c8f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0814036000 tx=0x7f081403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:02.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.045+0000 7f0820ff9640 1 -- 172.21.15.27:0/3430051040 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f081c0bbca0 con 0x7f08240b8920 2023-12-19T18:24:02.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.277+0000 7f082a155640 1 -- 172.21.15.27:0/3430051040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f08241618e0 con 0x7f08240b8920 2023-12-19T18:24:02.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.278+0000 7f0820ff9640 1 -- 172.21.15.27:0/3430051040 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f08240ca900 con 0x7f08240b8920 2023-12-19T18:24:02.279 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:24:02.279 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":63,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5720,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6835/3298228307","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3298228307},{"type":"v1","addr":"172.21.15.192:6835","nonce":3298228307}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":5726,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1760483999","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6836","nonce":1760483999},{"type":"v1","addr":"172.21.15.192:6837","nonce":1760483999}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":6388,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3000637822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3000637822},{"type":"v1","addr":"172.21.15.27:6833","nonce":3000637822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8163,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/1277213001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":1277213001},{"type":"v1","addr":"172.21.15.27:6835","nonce":1277213001}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":63}],"filesystems":[]} 2023-12-19T18:24:02.279 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 63 2023-12-19T18:24:02.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.282+0000 7f082a155640 1 -- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f080406a430 msgr2=0x7f080406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:02.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.282+0000 7f082a155640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f080406a430 0x7f080406c8f0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f0814036000 tx=0x7f081403d040 comp rx=0 tx=0).stop 2023-12-19T18:24:02.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.282+0000 7f082a155640 1 -- 172.21.15.27:0/3430051040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08240b8920 msgr2=0x7f08240cb120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:02.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.282+0000 7f082a155640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08240b8920 0x7f08240cb120 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f081c007e30 tx=0x7f081c00e4b0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.282+0000 7f082a155640 1 -- 172.21.15.27:0/3430051040 shutdown_connections 2023-12-19T18:24:02.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.282+0000 7f082a155640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f080406a430 0x7f080406c8f0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.282+0000 7f082a155640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08240b8920 0x7f08240cb120 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.282+0000 7f082a155640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08240b5060 0x7f08240cabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.282+0000 7f082a155640 1 --2- 172.21.15.27:0/3430051040 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08240b4c80 0x7f082414a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.282+0000 7f082a155640 1 -- 172.21.15.27:0/3430051040 >> 172.21.15.27:0/3430051040 conn(0x7f08240d2f90 msgr2=0x7f08240c6ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:02.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.282+0000 7f082a155640 1 -- 172.21.15.27:0/3430051040 shutdown_connections 2023-12-19T18:24:02.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.282+0000 7f082a155640 1 -- 172.21.15.27:0/3430051040 wait complete. 2023-12-19T18:24:02.303 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:24:02.303 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:24:02.304 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:24:02.304 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:24:02.304 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:24:02.304 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:24:02.304 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:24:02.304 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:24:02.304 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:24:02.304 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:24:02.305 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:24:02.305 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:24:02.305 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:24:02.305 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:24:02.305 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:24:02.305 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:24:02.320 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:24:02.321 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:24:02.327 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:24:02.328 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:24:02.329 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:24:02.329 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:24:02.330 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:24:02.330 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:24:02.377 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:24:02.378 DEBUG:teuthology.parallel:result is None 2023-12-19T18:24:02.380 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:24:02.380 DEBUG:teuthology.parallel:result is None 2023-12-19T18:24:02.385 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:24:02.385 DEBUG:teuthology.parallel:result is None 2023-12-19T18:24:02.388 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:24:02.388 DEBUG:teuthology.parallel:result is None 2023-12-19T18:24:02.388 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:24:02.545 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-19T18:24:02.545 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-19T18:24:02.547 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:24:02.553 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:24:02.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.591+0000 7f71b6969640 1 Processor -- start 2023-12-19T18:24:02.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.591+0000 7f71b6969640 1 -- start start 2023-12-19T18:24:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.593+0000 7f71b6969640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b0071140 0x7f71b00cd420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.593+0000 7f71b6969640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f71b00ac8d0 0x7f71b00acd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.593+0000 7f71b6969640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71b00ce7c0 con 0x7f71b015d1d0 2023-12-19T18:24:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.593+0000 7f71b6969640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71b0070150 con 0x7f71b00ac8d0 2023-12-19T18:24:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.593+0000 7f71b6969640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b015d5b0 con 0x7f71b0071140 2023-12-19T18:24:02.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.593+0000 7f71a7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b0071140 0x7f71b00cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.593+0000 7f71b4f06640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f71b00ac8d0 0x7f71b00acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.593+0000 7f71a7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b0071140 0x7f71b00cd420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:44236/0 (socket says 172.21.15.27:44236) 2023-12-19T18:24:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.593+0000 7f71a7fff640 1 -- 172.21.15.27:0/3334423335 learned_addr learned my addr 172.21.15.27:0/3334423335 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.594+0000 7f71a7fff640 1 -- 172.21.15.27:0/3334423335 >> v2:172.21.15.192:3300/0 conn(0x7f71b00ac8d0 msgr2=0x7f71b00acd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:02.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.594+0000 7f71a7fff640 1 --2- 172.21.15.27:0/3334423335 >> v2:172.21.15.192:3300/0 conn(0x7f71b00ac8d0 0x7f71b00acd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.594+0000 7f71a7fff640 1 -- 172.21.15.27:0/3334423335 >> v1:172.21.15.192:6789/0 conn(0x7f71b015d1d0 legacy=0x7f71b00ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.594+0000 7f71af7fe640 1 -- 172.21.15.27:0/3334423335 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3740015934 0 0) 0x7f71b00ce7c0 con 0x7f71b015d1d0 2023-12-19T18:24:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.594+0000 7f71a7fff640 1 -- 172.21.15.27:0/3334423335 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b0070360 con 0x7f71b0071140 2023-12-19T18:24:02.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.595+0000 7f71a7fff640 1 --2- 172.21.15.27:0/3334423335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b0071140 0x7f71b00cd420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7198009870 tx=0x7f719802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dce0f723f7824f76 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.595+0000 7f71af7fe640 1 -- 172.21.15.27:0/3334423335 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198031640 con 0x7f71b0071140 2023-12-19T18:24:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.595+0000 7f71af7fe640 1 -- 172.21.15.27:0/3334423335 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71980317a0 con 0x7f71b0071140 2023-12-19T18:24:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.595+0000 7f71af7fe640 1 -- 172.21.15.27:0/3334423335 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198031b30 con 0x7f71b0071140 2023-12-19T18:24:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.596+0000 7f71b6969640 1 -- 172.21.15.27:0/3334423335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b0071140 msgr2=0x7f71b00cd420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.596+0000 7f71b6969640 1 --2- 172.21.15.27:0/3334423335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b0071140 0x7f71b00cd420 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7198009870 tx=0x7f719802b8f0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.596+0000 7f71b6969640 1 -- 172.21.15.27:0/3334423335 shutdown_connections 2023-12-19T18:24:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.596+0000 7f71b6969640 1 --2- 172.21.15.27:0/3334423335 >> v2:172.21.15.192:3300/0 conn(0x7f71b00ac8d0 0x7f71b00acd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.596+0000 7f71b6969640 1 --2- 172.21.15.27:0/3334423335 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b0071140 0x7f71b00cd420 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.596+0000 7f71b6969640 1 -- 172.21.15.27:0/3334423335 >> 172.21.15.27:0/3334423335 conn(0x7f71b00ceb50 msgr2=0x7f71b00cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.596+0000 7f71b6969640 1 -- 172.21.15.27:0/3334423335 shutdown_connections 2023-12-19T18:24:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.596+0000 7f71b6969640 1 -- 172.21.15.27:0/3334423335 wait complete. 2023-12-19T18:24:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.598+0000 7f71b6969640 1 Processor -- start 2023-12-19T18:24:02.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.598+0000 7f71b6969640 1 -- start start 2023-12-19T18:24:02.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.599+0000 7f71b6969640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b0071140 0x7f71b00b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.599+0000 7f71b6969640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b00ac8d0 0x7f71b00c4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.599+0000 7f71b6969640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71b015d1d0 0x7f71b00acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.599+0000 7f71b6969640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00b2d00 con 0x7f71b0071140 2023-12-19T18:24:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.599+0000 7f71b6969640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00b20f0 con 0x7f71b015d1d0 2023-12-19T18:24:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.599+0000 7f71b6969640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71b00b23d0 con 0x7f71b00ac8d0 2023-12-19T18:24:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.599+0000 7f71affff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b0071140 0x7f71b00b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.599+0000 7f71affff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b0071140 0x7f71b00b1b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38092/0 (socket says 172.21.15.27:38092) 2023-12-19T18:24:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.599+0000 7f71affff640 1 -- 172.21.15.27:0/2288473843 learned_addr learned my addr 172.21.15.27:0/2288473843 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.599+0000 7f71a7fff640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b00ac8d0 0x7f71b00c4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.599+0000 7f71b4f06640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71b015d1d0 0x7f71b00acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.600+0000 7f71a7fff640 1 -- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71b015d1d0 msgr2=0x7f71b00acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:02.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.600+0000 7f71a7fff640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71b015d1d0 0x7f71b00acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.600+0000 7f71a7fff640 1 -- 172.21.15.27:0/2288473843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b0071140 msgr2=0x7f71b00b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:02.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.600+0000 7f71a7fff640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b0071140 0x7f71b00b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:02.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.600+0000 7f71a7fff640 1 -- 172.21.15.27:0/2288473843 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71980094d0 con 0x7f71b00ac8d0 2023-12-19T18:24:02.601 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.600+0000 7f71a7fff640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b00ac8d0 0x7f71b00c4ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7198033000 tx=0x7f71980046c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:02.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.600+0000 7f71ad7fa640 1 -- 172.21.15.27:0/2288473843 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719803ac60 con 0x7f71b00ac8d0 2023-12-19T18:24:02.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.600+0000 7f71ad7fa640 1 -- 172.21.15.27:0/2288473843 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719803ade0 con 0x7f71b00ac8d0 2023-12-19T18:24:02.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.600+0000 7f71b6969640 1 -- 172.21.15.27:0/2288473843 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71b00b26b0 con 0x7f71b00ac8d0 2023-12-19T18:24:02.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.600+0000 7f71ad7fa640 1 -- 172.21.15.27:0/2288473843 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7198044630 con 0x7f71b00ac8d0 2023-12-19T18:24:02.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.600+0000 7f71b6969640 1 -- 172.21.15.27:0/2288473843 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71b00c5430 con 0x7f71b00ac8d0 2023-12-19T18:24:02.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.601+0000 7f71b6969640 1 -- 172.21.15.27:0/2288473843 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71b00c5b20 con 0x7f71b00ac8d0 2023-12-19T18:24:02.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.602+0000 7f71ad7fa640 1 -- 172.21.15.27:0/2288473843 <== mon.2 v2:172.21.15.192:3301/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f719802f050 con 0x7f71b00ac8d0 2023-12-19T18:24:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.603+0000 7f71ad7fa640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f719006a430 0x7f719006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.603+0000 7f71ad7fa640 1 -- 172.21.15.27:0/2288473843 <== mon.2 v2:172.21.15.192:3301/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f71980aa280 con 0x7f71b00ac8d0 2023-12-19T18:24:02.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.604+0000 7f71affff640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f719006a430 0x7f719006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:02.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.605+0000 7f71affff640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f719006a430 0x7f719006c8f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f71a0000f60 tx=0x7f71a0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:02.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.609+0000 7f71ad7fa640 1 -- 172.21.15.27:0/2288473843 <== mon.2 v2:172.21.15.192:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f71b00c5b20 con 0x7f71b00ac8d0 2023-12-19T18:24:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:02.819+0000 7f71b6969640 1 -- 172.21.15.27:0/2288473843 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f71b00c5b20 con 0x7f71b00ac8d0 2023-12-19T18:24:03.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:03.792+0000 7f71ad7fa640 1 -- 172.21.15.27:0/2288473843 <== mon.2 v2:172.21.15.192:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v166) v1 ==== 97+0+0 (secure 0 0 0) 0x7f71b00c5b20 con 0x7f71b00ac8d0 2023-12-19T18:24:03.792 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2023-12-19T18:24:03.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:03.796+0000 7f71b6969640 1 -- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f719006a430 msgr2=0x7f719006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:03.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:03.796+0000 7f71b6969640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f719006a430 0x7f719006c8f0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f71a0000f60 tx=0x7f71a0009040 comp rx=0 tx=0).stop 2023-12-19T18:24:03.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:03.796+0000 7f71b6969640 1 -- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b00ac8d0 msgr2=0x7f71b00c4ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:03.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:03.796+0000 7f71b6969640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b00ac8d0 0x7f71b00c4ef0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7198033000 tx=0x7f71980046c0 comp rx=0 tx=0).stop 2023-12-19T18:24:03.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:03.796+0000 7f71b6969640 1 -- 172.21.15.27:0/2288473843 shutdown_connections 2023-12-19T18:24:03.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:03.796+0000 7f71b6969640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f719006a430 0x7f719006c8f0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:03.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:03.796+0000 7f71b6969640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f71b015d1d0 0x7f71b00acd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:03.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:03.796+0000 7f71b6969640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f71b00ac8d0 0x7f71b00c4ef0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:03.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:03.796+0000 7f71b6969640 1 --2- 172.21.15.27:0/2288473843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71b0071140 0x7f71b00b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:03.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:03.796+0000 7f71b6969640 1 -- 172.21.15.27:0/2288473843 >> 172.21.15.27:0/2288473843 conn(0x7f71b00ceb50 msgr2=0x7f71b00afbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:03.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:03.796+0000 7f71b6969640 1 -- 172.21.15.27:0/2288473843 shutdown_connections 2023-12-19T18:24:03.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:03.796+0000 7f71b6969640 1 -- 172.21.15.27:0/2288473843 wait complete. 2023-12-19T18:24:03.816 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:24:04.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.033+0000 7efd7359e640 1 Processor -- start 2023-12-19T18:24:04.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.034+0000 7efd7359e640 1 -- start start 2023-12-19T18:24:04.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.035+0000 7efd7359e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efd740b49d0 0x7efd74160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:04.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.035+0000 7efd7359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efd7415cf90 0x7efd740d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:04.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.035+0000 7efd7359e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd740b5ca0 con 0x7efd740b89b0 2023-12-19T18:24:04.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.035+0000 7efd7359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd74161800 con 0x7efd7415cf90 2023-12-19T18:24:04.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.035+0000 7efd7359e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd74161240 con 0x7efd740b49d0 2023-12-19T18:24:04.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.036+0000 7efd72d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efd7415cf90 0x7efd740d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:04.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.036+0000 7efd72d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efd7415cf90 0x7efd740d15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38098/0 (socket says 172.21.15.27:38098) 2023-12-19T18:24:04.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.036+0000 7efd72d9d640 1 -- 172.21.15.27:0/2897876699 learned_addr learned my addr 172.21.15.27:0/2897876699 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:04.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.036+0000 7efd7259c640 1 --2- 172.21.15.27:0/2897876699 >> v2:172.21.15.192:3300/0 conn(0x7efd740b49d0 0x7efd74160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:04.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.036+0000 7efd72d9d640 1 -- 172.21.15.27:0/2897876699 >> v2:172.21.15.192:3300/0 conn(0x7efd740b49d0 msgr2=0x7efd74160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:04.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.036+0000 7efd72d9d640 1 --2- 172.21.15.27:0/2897876699 >> v2:172.21.15.192:3300/0 conn(0x7efd740b49d0 0x7efd74160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.036+0000 7efd72d9d640 1 -- 172.21.15.27:0/2897876699 >> v1:172.21.15.192:6789/0 conn(0x7efd740b89b0 legacy=0x7efd7415c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:04.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.036+0000 7efd72d9d640 1 -- 172.21.15.27:0/2897876699 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd74161520 con 0x7efd7415cf90 2023-12-19T18:24:04.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.037+0000 7efd72d9d640 1 --2- 172.21.15.27:0/2897876699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd7415cf90 0x7efd740d15a0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7efd7406cd00 tx=0x7efd6c02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591ab7b36dbb5662 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:04.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.038+0000 7efd7159a640 1 -- 172.21.15.27:0/2897876699 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c030860 con 0x7efd7415cf90 2023-12-19T18:24:04.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.038+0000 7efd7159a640 1 -- 172.21.15.27:0/2897876699 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6c0309c0 con 0x7efd7415cf90 2023-12-19T18:24:04.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.038+0000 7efd7159a640 1 -- 172.21.15.27:0/2897876699 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6c030d50 con 0x7efd7415cf90 2023-12-19T18:24:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.038+0000 7efd7359e640 1 -- 172.21.15.27:0/2897876699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd7415cf90 msgr2=0x7efd740d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.038+0000 7efd7359e640 1 --2- 172.21.15.27:0/2897876699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd7415cf90 0x7efd740d15a0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7efd7406cd00 tx=0x7efd6c02b9a0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.039+0000 7efd7359e640 1 -- 172.21.15.27:0/2897876699 shutdown_connections 2023-12-19T18:24:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.039+0000 7efd7359e640 1 --2- 172.21.15.27:0/2897876699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd7415cf90 0x7efd740d15a0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.039+0000 7efd7359e640 1 --2- 172.21.15.27:0/2897876699 >> v2:172.21.15.192:3300/0 conn(0x7efd740b49d0 0x7efd74160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.039+0000 7efd7359e640 1 -- 172.21.15.27:0/2897876699 >> 172.21.15.27:0/2897876699 conn(0x7efd740cf760 msgr2=0x7efd740cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.039+0000 7efd7359e640 1 -- 172.21.15.27:0/2897876699 shutdown_connections 2023-12-19T18:24:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.039+0000 7efd7359e640 1 -- 172.21.15.27:0/2897876699 wait complete. 2023-12-19T18:24:04.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.041+0000 7efd7359e640 1 Processor -- start 2023-12-19T18:24:04.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.041+0000 7efd7359e640 1 -- start start 2023-12-19T18:24:04.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.042+0000 7efd7359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd740b49d0 0x7efd741f1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:04.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.042+0000 7efd7359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd740b89b0 0x7efd740c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:04.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.042+0000 7efd7359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd7415cf90 0x7efd7415b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:04.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.042+0000 7efd7359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd740c92f0 con 0x7efd740b89b0 2023-12-19T18:24:04.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.042+0000 7efd7359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd740c86e0 con 0x7efd740b49d0 2023-12-19T18:24:04.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.042+0000 7efd7359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd740c89c0 con 0x7efd7415cf90 2023-12-19T18:24:04.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.042+0000 7efd71d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd740b89b0 0x7efd740c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:04.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.042+0000 7efd71d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd740b89b0 0x7efd740c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38110/0 (socket says 172.21.15.27:38110) 2023-12-19T18:24:04.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.042+0000 7efd71d9b640 1 -- 172.21.15.27:0/3355568524 learned_addr learned my addr 172.21.15.27:0/3355568524 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:04.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.043+0000 7efd72d9d640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd7415cf90 0x7efd7415b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:04.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.043+0000 7efd7259c640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd740b49d0 0x7efd741f1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:04.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.043+0000 7efd71d9b640 1 -- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd7415cf90 msgr2=0x7efd7415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:04.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.043+0000 7efd71d9b640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd7415cf90 0x7efd7415b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.043+0000 7efd71d9b640 1 -- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd740b49d0 msgr2=0x7efd741f1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:04.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.043+0000 7efd71d9b640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd740b49d0 0x7efd741f1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.043+0000 7efd71d9b640 1 -- 172.21.15.27:0/3355568524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd6c0094d0 con 0x7efd740b89b0 2023-12-19T18:24:04.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.043+0000 7efd71d9b640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd740b89b0 0x7efd740c7c90 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00c430 tx=0x7efd5c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:04.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.043+0000 7efd6affd640 1 -- 172.21.15.27:0/3355568524 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c011790 con 0x7efd740b89b0 2023-12-19T18:24:04.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.043+0000 7efd6affd640 1 -- 172.21.15.27:0/3355568524 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd5c0118f0 con 0x7efd740b89b0 2023-12-19T18:24:04.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.043+0000 7efd7359e640 1 -- 172.21.15.27:0/3355568524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd740c8ca0 con 0x7efd740b89b0 2023-12-19T18:24:04.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.043+0000 7efd6affd640 1 -- 172.21.15.27:0/3355568524 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd5c011ce0 con 0x7efd740b89b0 2023-12-19T18:24:04.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.044+0000 7efd7359e640 1 -- 172.21.15.27:0/3355568524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd741f2480 con 0x7efd740b89b0 2023-12-19T18:24:04.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.045+0000 7efd7359e640 1 -- 172.21.15.27:0/3355568524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7404e840 con 0x7efd740b89b0 2023-12-19T18:24:04.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.046+0000 7efd6affd640 1 -- 172.21.15.27:0/3355568524 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efd5c01c460 con 0x7efd740b89b0 2023-12-19T18:24:04.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.047+0000 7efd6affd640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd5406a430 0x7efd5406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:04.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.047+0000 7efd6affd640 1 -- 172.21.15.27:0/3355568524 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efd5c088500 con 0x7efd740b89b0 2023-12-19T18:24:04.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.048+0000 7efd7259c640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd5406a430 0x7efd5406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:04.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.049+0000 7efd7259c640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd5406a430 0x7efd5406c8f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efd64009ab0 tx=0x7efd64003990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:04.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.052+0000 7efd6affd640 1 -- 172.21.15.27:0/3355568524 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efd5c053e40 con 0x7efd740b89b0 2023-12-19T18:24:04.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.319+0000 7efd7359e640 1 -- 172.21.15.27:0/3355568524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7efd740d08b0 con 0x7efd740b89b0 2023-12-19T18:24:04.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.320+0000 7efd6affd640 1 -- 172.21.15.27:0/3355568524 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v18) v1 ==== 80+0+5304 (secure 0 0 0) 0x7efd5c057460 con 0x7efd740b89b0 2023-12-19T18:24:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:24:04.321 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:24:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2023-12-19T18:24:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2023-12-19T18:24:04.321 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFl+4ruHBAAXwbpmT43SmlvNbuZj5e+bQ==", 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmZyUBhAAG3kZkXvJIo7N9f1j+DPCSQ==", 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:24:04.322 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlChNOMBAAX4GV3iIGkSQdAsYu9oCPDw==", 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2023-12-19T18:24:04.323 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlTmFXFhAABA0NMUxKBV1ExAhYc5RFbQ==", 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFluleFIBAAs1goFVMOX4CbjwFhg0KoBg==", 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:24:04.324 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFlInW3MxAAI4hY1GBxXYXDswKKrEyDTg==", 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFlAWZ5CRAA82/2r9/sT0L9VlGHbXksQw==", 2023-12-19T18:24:04.325 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFl1IvaGRAAqVacDDPLx+NqBOGp6MeX2w==", 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.326 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2023-12-19T18:24:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlr0BBHRAA+7gfs/RG6DtABnYPg059Tw==", 2023-12-19T18:24:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:24:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:24:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.327 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlXZbELRAAKD0qOkhk7skfc4xcAC8Lhg==", 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlF07MARAA2TJJ+5C1HIU+SVhqmKaVHA==", 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:24:04.328 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlISPGEhAAdFIRRkXlLyOx/oMZ4MgL9Q==", 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2023-12-19T18:24:04.329 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmNM9JRAARgS20O8zpm6H70WDv+lgGw==", 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlWoZ8AhAADWvKVzeDhDGHtYNUFIOWsA==", 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:24:04.330 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFllWFVMhAAD9ebDOppHTODVQjg6ppP3g==", 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2023-12-19T18:24:04.331 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlwbIQDRAAt/h55zBNCEId0vkYSUQ3TQ==", 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB53YFloCb6CxAAmZQyz5XOEqDOOiROJHLzxQ==", 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:24:04.332 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFltBhxDxAAiAS9CYPY86IMmJ7NGgA2lw==", 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlUnAhBBAAi++W9XK3o3Y/185hB3LuIg==", 2023-12-19T18:24:04.333 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:24:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:24:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:24:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:24:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:04.334 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:24:04.334 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:24:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.323+0000 7efd7359e640 1 -- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd5406a430 msgr2=0x7efd5406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.323+0000 7efd7359e640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd5406a430 0x7efd5406c8f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7efd64009ab0 tx=0x7efd64003990 comp rx=0 tx=0).stop 2023-12-19T18:24:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.323+0000 7efd7359e640 1 -- 172.21.15.27:0/3355568524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd740b89b0 msgr2=0x7efd740c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.323+0000 7efd7359e640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd740b89b0 0x7efd740c7c90 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00c430 tx=0x7efd5c00c8f0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.323+0000 7efd7359e640 1 -- 172.21.15.27:0/3355568524 shutdown_connections 2023-12-19T18:24:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.323+0000 7efd7359e640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd5406a430 0x7efd5406c8f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.323+0000 7efd7359e640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd7415cf90 0x7efd7415b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.323+0000 7efd7359e640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd740b89b0 0x7efd740c7c90 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.323+0000 7efd7359e640 1 --2- 172.21.15.27:0/3355568524 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd740b49d0 0x7efd741f1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.323+0000 7efd7359e640 1 -- 172.21.15.27:0/3355568524 >> 172.21.15.27:0/3355568524 conn(0x7efd740cf760 msgr2=0x7efd740d2150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:04.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.323+0000 7efd7359e640 1 -- 172.21.15.27:0/3355568524 shutdown_connections 2023-12-19T18:24:04.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.323+0000 7efd7359e640 1 -- 172.21.15.27:0/3355568524 wait complete. 2023-12-19T18:24:04.344 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:24:04.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.572+0000 7f461e5e1640 1 Processor -- start 2023-12-19T18:24:04.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.572+0000 7f461e5e1640 1 -- start start 2023-12-19T18:24:04.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.574+0000 7f461e5e1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f46180cb300 0x7f46180cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.574+0000 7f461e5e1640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4618161010 0x7f46180cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.574+0000 7f461e5e1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46180cbc20 con 0x7f46180c8f90 2023-12-19T18:24:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.574+0000 7f461e5e1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46180c4480 con 0x7f46180cb300 2023-12-19T18:24:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.574+0000 7f461e5e1640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46181468c0 con 0x7f4618161010 2023-12-19T18:24:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.575+0000 7f461cdde640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f46180c8f90 0x7f4618160900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49210/0 (socket says 172.21.15.27:49210) 2023-12-19T18:24:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.575+0000 7f461cdde640 1 -- 172.21.15.27:0/1770562288 learned_addr learned my addr 172.21.15.27:0/1770562288 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:04.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.575+0000 7f461d5df640 1 --2- 172.21.15.27:0/1770562288 >> v2:172.21.15.27:3300/0 conn(0x7f46180cb300 0x7f46180cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.575+0000 7f461dde0640 1 --2- 172.21.15.27:0/1770562288 >> v2:172.21.15.192:3300/0 conn(0x7f4618161010 0x7f46180cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.575+0000 7f460ffff640 1 -- 172.21.15.27:0/1770562288 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2832088125 0 0) 0x7f46180cbc20 con 0x7f46180c8f90 2023-12-19T18:24:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.575+0000 7f460ffff640 1 -- 172.21.15.27:0/1770562288 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4604003520 con 0x7f46180c8f90 2023-12-19T18:24:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.575+0000 7f461d5df640 1 -- 172.21.15.27:0/1770562288 >> v2:172.21.15.192:3300/0 conn(0x7f4618161010 msgr2=0x7f46180cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.575+0000 7f461d5df640 1 --2- 172.21.15.27:0/1770562288 >> v2:172.21.15.192:3300/0 conn(0x7f4618161010 0x7f46180cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.576+0000 7f461d5df640 1 -- 172.21.15.27:0/1770562288 >> v1:172.21.15.27:6789/0 conn(0x7f46180c8f90 legacy=0x7f4618160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:04.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.576+0000 7f461d5df640 1 -- 172.21.15.27:0/1770562288 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4618145d60 con 0x7f46180cb300 2023-12-19T18:24:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.576+0000 7f461dde0640 1 --2- 172.21.15.27:0/1770562288 >> v2:172.21.15.192:3300/0 conn(0x7f4618161010 0x7f46180cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:24:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.577+0000 7f461d5df640 1 --2- 172.21.15.27:0/1770562288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46180cb300 0x7f46180cb6e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f46140097a0 tx=0x7f461402b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46bb92e06e62a58c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.578+0000 7f460ffff640 1 -- 172.21.15.27:0/1770562288 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614031830 con 0x7f46180cb300 2023-12-19T18:24:04.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.578+0000 7f460ffff640 1 -- 172.21.15.27:0/1770562288 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4614031990 con 0x7f46180cb300 2023-12-19T18:24:04.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.578+0000 7f460ffff640 1 -- 172.21.15.27:0/1770562288 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4614031d00 con 0x7f46180cb300 2023-12-19T18:24:04.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.578+0000 7f461e5e1640 1 -- 172.21.15.27:0/1770562288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46180cb300 msgr2=0x7f46180cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:04.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.578+0000 7f461e5e1640 1 --2- 172.21.15.27:0/1770562288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46180cb300 0x7f46180cb6e0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f46140097a0 tx=0x7f461402b8d0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.578+0000 7f461e5e1640 1 -- 172.21.15.27:0/1770562288 shutdown_connections 2023-12-19T18:24:04.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.578+0000 7f461e5e1640 1 --2- 172.21.15.27:0/1770562288 >> v2:172.21.15.192:3300/0 conn(0x7f4618161010 0x7f46180cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.578+0000 7f461e5e1640 1 --2- 172.21.15.27:0/1770562288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46180cb300 0x7f46180cb6e0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.578+0000 7f461e5e1640 1 -- 172.21.15.27:0/1770562288 >> 172.21.15.27:0/1770562288 conn(0x7f4618069c70 msgr2=0x7f461806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:04.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.578+0000 7f461e5e1640 1 -- 172.21.15.27:0/1770562288 shutdown_connections 2023-12-19T18:24:04.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.579+0000 7f461e5e1640 1 -- 172.21.15.27:0/1770562288 wait complete. 2023-12-19T18:24:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.580+0000 7f461e5e1640 1 Processor -- start 2023-12-19T18:24:04.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.580+0000 7f461e5e1640 1 -- start start 2023-12-19T18:24:04.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.581+0000 7f461e5e1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46180c8f90 0x7f46181f1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.581+0000 7f461e5e1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46180cb300 0x7f46180aeec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.581+0000 7f461e5e1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4618161010 0x7f46180af400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.581+0000 7f461e5e1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46180b1830 con 0x7f46180cb300 2023-12-19T18:24:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.581+0000 7f461e5e1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461806c240 con 0x7f4618161010 2023-12-19T18:24:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.581+0000 7f461e5e1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f461806bf60 con 0x7f46180c8f90 2023-12-19T18:24:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.582+0000 7f461cdde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46180cb300 0x7f46180aeec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.582+0000 7f461cdde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46180cb300 0x7f46180aeec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38134/0 (socket says 172.21.15.27:38134) 2023-12-19T18:24:04.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.582+0000 7f461cdde640 1 -- 172.21.15.27:0/1244261837 learned_addr learned my addr 172.21.15.27:0/1244261837 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.582+0000 7f461d5df640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46180c8f90 0x7f46181f1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.582+0000 7f461dde0640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4618161010 0x7f46180af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.582+0000 7f461cdde640 1 -- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46180c8f90 msgr2=0x7f46181f1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.582+0000 7f461cdde640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46180c8f90 0x7f46181f1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.582+0000 7f461cdde640 1 -- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4618161010 msgr2=0x7f46180af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.582+0000 7f461cdde640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4618161010 0x7f46180af400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.582+0000 7f461cdde640 1 -- 172.21.15.27:0/1244261837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4614009400 con 0x7f46180cb300 2023-12-19T18:24:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.583+0000 7f461cdde640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46180cb300 0x7f46180aeec0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f460800c390 tx=0x7f460800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.583+0000 7f460dffb640 1 -- 172.21.15.27:0/1244261837 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4608005ce0 con 0x7f46180cb300 2023-12-19T18:24:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.583+0000 7f460dffb640 1 -- 172.21.15.27:0/1244261837 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4608005e40 con 0x7f46180cb300 2023-12-19T18:24:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.583+0000 7f460dffb640 1 -- 172.21.15.27:0/1244261837 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46080072f0 con 0x7f46180cb300 2023-12-19T18:24:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.583+0000 7f461e5e1640 1 -- 172.21.15.27:0/1244261837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461815d480 con 0x7f46180cb300 2023-12-19T18:24:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.583+0000 7f461e5e1640 1 -- 172.21.15.27:0/1244261837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461815da40 con 0x7f46180cb300 2023-12-19T18:24:04.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.584+0000 7f461e5e1640 1 -- 172.21.15.27:0/1244261837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f461804e840 con 0x7f46180cb300 2023-12-19T18:24:04.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.585+0000 7f460dffb640 1 -- 172.21.15.27:0/1244261837 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4608007450 con 0x7f46180cb300 2023-12-19T18:24:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.586+0000 7f460dffb640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45fc06a430 0x7f45fc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.586+0000 7f460dffb640 1 -- 172.21.15.27:0/1244261837 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4608087c30 con 0x7f46180cb300 2023-12-19T18:24:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.587+0000 7f461d5df640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45fc06a430 0x7f45fc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:04.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.588+0000 7f461d5df640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45fc06a430 0x7f45fc06c8f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4614036000 tx=0x7f4614008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:04.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.591+0000 7f460dffb640 1 -- 172.21.15.27:0/1244261837 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4608053560 con 0x7f46180cb300 2023-12-19T18:24:04.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.785+0000 7f461e5e1640 1 -- 172.21.15.27:0/1244261837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f46180cbbd0 con 0x7f46180cb300 2023-12-19T18:24:04.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.787+0000 7f460dffb640 1 -- 172.21.15.27:0/1244261837 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+12400 (secure 0 0 0) 0x7f4608017090 con 0x7f46180cb300 2023-12-19T18:24:04.787 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:24:04.787 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":166,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:24:03.782967+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":156,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":150,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:24:04.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.789+0000 7f461e5e1640 1 -- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45fc06a430 msgr2=0x7f45fc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:04.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.789+0000 7f461e5e1640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45fc06a430 0x7f45fc06c8f0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f4614036000 tx=0x7f4614008030 comp rx=0 tx=0).stop 2023-12-19T18:24:04.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.789+0000 7f461e5e1640 1 -- 172.21.15.27:0/1244261837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46180cb300 msgr2=0x7f46180aeec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:04.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.789+0000 7f461e5e1640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46180cb300 0x7f46180aeec0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f460800c390 tx=0x7f460800c850 comp rx=0 tx=0).stop 2023-12-19T18:24:04.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.790+0000 7f461e5e1640 1 -- 172.21.15.27:0/1244261837 shutdown_connections 2023-12-19T18:24:04.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.790+0000 7f461e5e1640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45fc06a430 0x7f45fc06c8f0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.790+0000 7f461e5e1640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4618161010 0x7f46180af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.790+0000 7f461e5e1640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46180cb300 0x7f46180aeec0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.790+0000 7f461e5e1640 1 --2- 172.21.15.27:0/1244261837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f46180c8f90 0x7f46181f1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:04.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.790+0000 7f461e5e1640 1 -- 172.21.15.27:0/1244261837 >> 172.21.15.27:0/1244261837 conn(0x7f4618069c70 msgr2=0x7f46180c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:04.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.790+0000 7f461e5e1640 1 -- 172.21.15.27:0/1244261837 shutdown_connections 2023-12-19T18:24:04.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:04.790+0000 7f461e5e1640 1 -- 172.21.15.27:0/1244261837 wait complete. 2023-12-19T18:24:04.811 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:24:04.812 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:24:05.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.043+0000 7f00508b4640 1 Processor -- start 2023-12-19T18:24:05.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.044+0000 7f00508b4640 1 -- start start 2023-12-19T18:24:05.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.045+0000 7f00508b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f004c15cf80 0x7f004c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:05.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.045+0000 7f00508b4640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004c0d0e10 con 0x7f004c0b8a60 2023-12-19T18:24:05.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.045+0000 7f00508b4640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f004c0d1c70 con 0x7f004c0b85e0 2023-12-19T18:24:05.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.045+0000 7f00508b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c15d360 con 0x7f004c15cf80 2023-12-19T18:24:05.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.046+0000 7f004bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f004c0b8a60 0x7f004c0b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49214/0 (socket says 172.21.15.27:49214) 2023-12-19T18:24:05.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.046+0000 7f004bfff640 1 -- 172.21.15.27:0/1757236795 learned_addr learned my addr 172.21.15.27:0/1757236795 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:05.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.046+0000 7f004b7fe640 1 --2- 172.21.15.27:0/1757236795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f004c15cf80 0x7f004c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:05.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.046+0000 7f004a7fc640 1 -- 172.21.15.27:0/1757236795 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4077047196 0 0) 0x7f004c0d0e10 con 0x7f004c0b8a60 2023-12-19T18:24:05.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.046+0000 7f004a7fc640 1 -- 172.21.15.27:0/1757236795 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0038003520 con 0x7f004c0b8a60 2023-12-19T18:24:05.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.047+0000 7f004b7fe640 1 -- 172.21.15.27:0/1757236795 >> v1:172.21.15.192:6789/0 conn(0x7f004c0b85e0 legacy=0x7f004c0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.047+0000 7f004b7fe640 1 -- 172.21.15.27:0/1757236795 >> v1:172.21.15.27:6789/0 conn(0x7f004c0b8a60 legacy=0x7f004c0b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.047+0000 7f004b7fe640 1 -- 172.21.15.27:0/1757236795 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f004c0d1060 con 0x7f004c15cf80 2023-12-19T18:24:05.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.048+0000 7f004b7fe640 1 --2- 172.21.15.27:0/1757236795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f004c15cf80 0x7f004c0d2100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f003c009dc0 tx=0x7f003c02bfa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cfb740d5512963c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:05.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.049+0000 7f004a7fc640 1 -- 172.21.15.27:0/1757236795 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c02cd50 con 0x7f004c15cf80 2023-12-19T18:24:05.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.049+0000 7f004a7fc640 1 -- 172.21.15.27:0/1757236795 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c034440 con 0x7f004c15cf80 2023-12-19T18:24:05.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.049+0000 7f004a7fc640 1 -- 172.21.15.27:0/1757236795 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c0347d0 con 0x7f004c15cf80 2023-12-19T18:24:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.050+0000 7f00508b4640 1 -- 172.21.15.27:0/1757236795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f004c15cf80 msgr2=0x7f004c0d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.050+0000 7f00508b4640 1 --2- 172.21.15.27:0/1757236795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f004c15cf80 0x7f004c0d2100 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f003c009dc0 tx=0x7f003c02bfa0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.050+0000 7f00508b4640 1 -- 172.21.15.27:0/1757236795 shutdown_connections 2023-12-19T18:24:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.050+0000 7f00508b4640 1 --2- 172.21.15.27:0/1757236795 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f004c15cf80 0x7f004c0d2100 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.050+0000 7f00508b4640 1 -- 172.21.15.27:0/1757236795 >> 172.21.15.27:0/1757236795 conn(0x7f004c072e20 msgr2=0x7f004c073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.050+0000 7f00508b4640 1 -- 172.21.15.27:0/1757236795 shutdown_connections 2023-12-19T18:24:05.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.050+0000 7f00508b4640 1 -- 172.21.15.27:0/1757236795 wait complete. 2023-12-19T18:24:05.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.052+0000 7f00508b4640 1 Processor -- start 2023-12-19T18:24:05.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.052+0000 7f00508b4640 1 -- start start 2023-12-19T18:24:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.053+0000 7f00508b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c0b85e0 0x7f004c155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.053+0000 7f00508b4640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f004c15cf80 0x7f004c0c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.053+0000 7f00508b4640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f004c15fbc0 0x7f004c0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.053+0000 7f00508b4640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c155cc0 con 0x7f004c0b85e0 2023-12-19T18:24:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.053+0000 7f00508b4640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004c1523c0 con 0x7f004c15fbc0 2023-12-19T18:24:05.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.053+0000 7f00508b4640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f004c0cb050 con 0x7f004c15cf80 2023-12-19T18:24:05.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.053+0000 7f004b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c0b85e0 0x7f004c155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:05.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.053+0000 7f004b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c0b85e0 0x7f004c155780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38138/0 (socket says 172.21.15.27:38138) 2023-12-19T18:24:05.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.053+0000 7f004b7fe640 1 -- 172.21.15.27:0/317251868 learned_addr learned my addr 172.21.15.27:0/317251868 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.054+0000 7f004affd640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f004c15cf80 0x7f004c0c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.054+0000 7f004bfff640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f004c15fbc0 0x7f004c0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.054+0000 7f004b7fe640 1 -- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f004c15cf80 msgr2=0x7f004c0c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.054+0000 7f004b7fe640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f004c15cf80 0x7f004c0c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.054+0000 7f004b7fe640 1 -- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f004c15fbc0 msgr2=0x7f004c0ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.054+0000 7f004b7fe640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f004c15fbc0 0x7f004c0ca280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.054+0000 7f004b7fe640 1 -- 172.21.15.27:0/317251868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f003c009a70 con 0x7f004c0b85e0 2023-12-19T18:24:05.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.054+0000 7f004b7fe640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c0b85e0 0x7f004c155780 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f003c000f80 tx=0x7f003c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.055+0000 7f0043fff640 1 -- 172.21.15.27:0/317251868 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c040650 con 0x7f004c0b85e0 2023-12-19T18:24:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.055+0000 7f0043fff640 1 -- 172.21.15.27:0/317251868 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f003c0407b0 con 0x7f004c0b85e0 2023-12-19T18:24:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.055+0000 7f00508b4640 1 -- 172.21.15.27:0/317251868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f004c0cad70 con 0x7f004c0b85e0 2023-12-19T18:24:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.055+0000 7f0043fff640 1 -- 172.21.15.27:0/317251868 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003c040b60 con 0x7f004c0b85e0 2023-12-19T18:24:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.055+0000 7f00508b4640 1 -- 172.21.15.27:0/317251868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f004c15f3c0 con 0x7f004c0b85e0 2023-12-19T18:24:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.056+0000 7f00508b4640 1 -- 172.21.15.27:0/317251868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f004c04e840 con 0x7f004c0b85e0 2023-12-19T18:24:05.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.057+0000 7f0043fff640 1 -- 172.21.15.27:0/317251868 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f003c040cc0 con 0x7f004c0b85e0 2023-12-19T18:24:05.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.058+0000 7f0043fff640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f002c06a430 0x7f002c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:05.058 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.058+0000 7f0043fff640 1 -- 172.21.15.27:0/317251868 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f003c0ab390 con 0x7f004c0b85e0 2023-12-19T18:24:05.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.059+0000 7f004affd640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f002c06a430 0x7f002c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:05.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.060+0000 7f004affd640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f002c06a430 0x7f002c06c8f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f004c0b1cd0 tx=0x7f0034009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:05.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.063+0000 7f0043fff640 1 -- 172.21.15.27:0/317251868 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f003c076cd0 con 0x7f004c0b85e0 2023-12-19T18:24:05.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.255+0000 7f00508b4640 1 -- 172.21.15.27:0/317251868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f004c0d02c0 con 0x7f004c0b85e0 2023-12-19T18:24:05.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.256+0000 7f0043fff640 1 -- 172.21.15.27:0/317251868 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v166) v1 ==== 93+0+10 (secure 0 0 0) 0x7f003c009450 con 0x7f004c0b85e0 2023-12-19T18:24:05.256 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:24:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.258+0000 7f00508b4640 1 -- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f002c06a430 msgr2=0x7f002c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.258+0000 7f00508b4640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f002c06a430 0x7f002c06c8f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f004c0b1cd0 tx=0x7f0034009040 comp rx=0 tx=0).stop 2023-12-19T18:24:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.259+0000 7f00508b4640 1 -- 172.21.15.27:0/317251868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c0b85e0 msgr2=0x7f004c155780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.259+0000 7f00508b4640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c0b85e0 0x7f004c155780 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f003c000f80 tx=0x7f003c004a40 comp rx=0 tx=0).stop 2023-12-19T18:24:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.259+0000 7f00508b4640 1 -- 172.21.15.27:0/317251868 shutdown_connections 2023-12-19T18:24:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.259+0000 7f00508b4640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f002c06a430 0x7f002c06c8f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.259+0000 7f00508b4640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f004c15fbc0 0x7f004c0ca280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.259+0000 7f00508b4640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f004c15cf80 0x7f004c0c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.259+0000 7f00508b4640 1 --2- 172.21.15.27:0/317251868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f004c0b85e0 0x7f004c155780 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.259+0000 7f00508b4640 1 -- 172.21.15.27:0/317251868 >> 172.21.15.27:0/317251868 conn(0x7f004c072e20 msgr2=0x7f004c0cd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:05.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.259+0000 7f00508b4640 1 -- 172.21.15.27:0/317251868 shutdown_connections 2023-12-19T18:24:05.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.259+0000 7f00508b4640 1 -- 172.21.15.27:0/317251868 wait complete. 2023-12-19T18:24:05.280 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:24:05.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.515+0000 7f627e9a2640 1 Processor -- start 2023-12-19T18:24:05.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.515+0000 7f627e9a2640 1 -- start start 2023-12-19T18:24:05.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.516+0000 7f627e9a2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62780b49d0 0x7f6278160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:05.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.516+0000 7f627e9a2640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62780b5ca0 con 0x7f627815cf90 2023-12-19T18:24:05.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.516+0000 7f627e9a2640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6278161800 con 0x7f62780b89b0 2023-12-19T18:24:05.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.516+0000 7f627e9a2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6278161240 con 0x7f62780b49d0 2023-12-19T18:24:05.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.517+0000 7f627e1a1640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f627815cf90 0x7f62780d15a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49228/0 (socket says 172.21.15.27:49228) 2023-12-19T18:24:05.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.517+0000 7f627e1a1640 1 -- 172.21.15.27:0/3282127869 learned_addr learned my addr 172.21.15.27:0/3282127869 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:05.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.517+0000 7f627d9a0640 1 --2- 172.21.15.27:0/3282127869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62780b49d0 0x7f6278160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.518+0000 7f627c99e640 1 -- 172.21.15.27:0/3282127869 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1201784216 0 0) 0x7f62780b5ca0 con 0x7f627815cf90 2023-12-19T18:24:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.518+0000 7f627c99e640 1 -- 172.21.15.27:0/3282127869 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6264003520 con 0x7f627815cf90 2023-12-19T18:24:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.518+0000 7f627d9a0640 1 -- 172.21.15.27:0/3282127869 >> v1:172.21.15.192:6789/0 conn(0x7f62780b89b0 legacy=0x7f627815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.518+0000 7f627d9a0640 1 -- 172.21.15.27:0/3282127869 >> v1:172.21.15.27:6789/0 conn(0x7f627815cf90 legacy=0x7f62780d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.518+0000 7f627d9a0640 1 -- 172.21.15.27:0/3282127869 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6278161520 con 0x7f62780b49d0 2023-12-19T18:24:05.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.519+0000 7f627d9a0640 1 --2- 172.21.15.27:0/3282127869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62780b49d0 0x7f6278160940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6274009e10 tx=0x7f627402bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb1650597f93d377 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:05.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.520+0000 7f627c99e640 1 -- 172.21.15.27:0/3282127869 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f627402cda0 con 0x7f62780b49d0 2023-12-19T18:24:05.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.520+0000 7f627c99e640 1 -- 172.21.15.27:0/3282127869 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6274034440 con 0x7f62780b49d0 2023-12-19T18:24:05.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.520+0000 7f627c99e640 1 -- 172.21.15.27:0/3282127869 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62740347b0 con 0x7f62780b49d0 2023-12-19T18:24:05.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.520+0000 7f627e9a2640 1 -- 172.21.15.27:0/3282127869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62780b49d0 msgr2=0x7f6278160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.520+0000 7f627e9a2640 1 --2- 172.21.15.27:0/3282127869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62780b49d0 0x7f6278160940 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6274009e10 tx=0x7f627402bff0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.520+0000 7f627e9a2640 1 -- 172.21.15.27:0/3282127869 shutdown_connections 2023-12-19T18:24:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.520+0000 7f627e9a2640 1 --2- 172.21.15.27:0/3282127869 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62780b49d0 0x7f6278160940 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.520+0000 7f627e9a2640 1 -- 172.21.15.27:0/3282127869 >> 172.21.15.27:0/3282127869 conn(0x7f62780cf760 msgr2=0x7f62780cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.520+0000 7f627e9a2640 1 -- 172.21.15.27:0/3282127869 shutdown_connections 2023-12-19T18:24:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.521+0000 7f627e9a2640 1 -- 172.21.15.27:0/3282127869 wait complete. 2023-12-19T18:24:05.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.522+0000 7f627e9a2640 1 Processor -- start 2023-12-19T18:24:05.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.522+0000 7f627e9a2640 1 -- start start 2023-12-19T18:24:05.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.523+0000 7f627e9a2640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62780b49d0 0x7f627815ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:05.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.523+0000 7f627e9a2640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62780b89b0 0x7f62781f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.523+0000 7f627e9a2640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f627815cf90 0x7f62781f5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.523+0000 7f627e9a2640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627815f4c0 con 0x7f62780b89b0 2023-12-19T18:24:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.523+0000 7f627e9a2640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f627815f7a0 con 0x7f627815cf90 2023-12-19T18:24:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.523+0000 7f627e9a2640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f627815fa80 con 0x7f62780b49d0 2023-12-19T18:24:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.524+0000 7f627d19f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62780b89b0 0x7f62781f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.524+0000 7f627d19f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62780b89b0 0x7f62781f1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38140/0 (socket says 172.21.15.27:38140) 2023-12-19T18:24:05.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.524+0000 7f627d19f640 1 -- 172.21.15.27:0/3450926722 learned_addr learned my addr 172.21.15.27:0/3450926722 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:05.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.524+0000 7f627d9a0640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62780b49d0 0x7f627815ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:05.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.524+0000 7f627e1a1640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f627815cf90 0x7f62781f5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:05.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.524+0000 7f627d19f640 1 -- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62780b49d0 msgr2=0x7f627815ebc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.524+0000 7f627d19f640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62780b49d0 0x7f627815ebc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.524+0000 7f627d19f640 1 -- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f627815cf90 msgr2=0x7f62781f5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.524+0000 7f627d19f640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f627815cf90 0x7f62781f5ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.524+0000 7f627d19f640 1 -- 172.21.15.27:0/3450926722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6274009a70 con 0x7f62780b89b0 2023-12-19T18:24:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.524+0000 7f627d9a0640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62780b49d0 0x7f627815ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.524+0000 7f627d19f640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62780b89b0 0x7f62781f1d90 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f626800c440 tx=0x7f6268001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.525+0000 7f626e7fc640 1 -- 172.21.15.27:0/3450926722 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62680052d0 con 0x7f62780b89b0 2023-12-19T18:24:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.525+0000 7f626e7fc640 1 -- 172.21.15.27:0/3450926722 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6268005430 con 0x7f62780b89b0 2023-12-19T18:24:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.525+0000 7f627e9a2640 1 -- 172.21.15.27:0/3450926722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f627814f120 con 0x7f62780b89b0 2023-12-19T18:24:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.525+0000 7f626e7fc640 1 -- 172.21.15.27:0/3450926722 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6268007260 con 0x7f62780b89b0 2023-12-19T18:24:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.525+0000 7f627e9a2640 1 -- 172.21.15.27:0/3450926722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627815f100 con 0x7f62780b89b0 2023-12-19T18:24:05.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.526+0000 7f627e9a2640 1 -- 172.21.15.27:0/3450926722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627814f640 con 0x7f62780b89b0 2023-12-19T18:24:05.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.527+0000 7f626e7fc640 1 -- 172.21.15.27:0/3450926722 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f62680073c0 con 0x7f62780b89b0 2023-12-19T18:24:05.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.528+0000 7f626e7fc640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f625c06a430 0x7f625c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:05.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.529+0000 7f626e7fc640 1 -- 172.21.15.27:0/3450926722 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6268087a80 con 0x7f62780b89b0 2023-12-19T18:24:05.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.529+0000 7f627d9a0640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f625c06a430 0x7f625c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:05.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.530+0000 7f627d9a0640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f625c06a430 0x7f625c06c8f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6274036000 tx=0x7f6274030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:05.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.533+0000 7f626e7fc640 1 -- 172.21.15.27:0/3450926722 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f627814f640 con 0x7f62780b89b0 2023-12-19T18:24:05.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.730+0000 7f627e9a2640 1 -- 172.21.15.27:0/3450926722 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f627814f640 con 0x7f62780b89b0 2023-12-19T18:24:05.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.731+0000 7f626e7fc640 1 -- 172.21.15.27:0/3450926722 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v166) v1 ==== 84+0+65 (secure 0 0 0) 0x7f627814f640 con 0x7f62780b89b0 2023-12-19T18:24:05.731 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:24:05.732 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-12-19T18:24:05.732 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:24:05.732 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-12-19T18:24:05.732 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-12-19T18:24:05.732 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:05.732 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-12-19T18:24:05.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.734+0000 7f627e9a2640 1 -- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f625c06a430 msgr2=0x7f625c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.734+0000 7f627e9a2640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f625c06a430 0x7f625c06c8f0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f6274036000 tx=0x7f6274030000 comp rx=0 tx=0).stop 2023-12-19T18:24:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.734+0000 7f627e9a2640 1 -- 172.21.15.27:0/3450926722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62780b89b0 msgr2=0x7f62781f1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.734+0000 7f627e9a2640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62780b89b0 0x7f62781f1d90 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f626800c440 tx=0x7f6268001e70 comp rx=0 tx=0).stop 2023-12-19T18:24:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.734+0000 7f627e9a2640 1 -- 172.21.15.27:0/3450926722 shutdown_connections 2023-12-19T18:24:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.734+0000 7f627e9a2640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f625c06a430 0x7f625c06c8f0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.734+0000 7f627e9a2640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f627815cf90 0x7f62781f5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.734+0000 7f627e9a2640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62780b89b0 0x7f62781f1d90 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.734+0000 7f627e9a2640 1 --2- 172.21.15.27:0/3450926722 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f62780b49d0 0x7f627815ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:05.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.734+0000 7f627e9a2640 1 -- 172.21.15.27:0/3450926722 >> 172.21.15.27:0/3450926722 conn(0x7f62780cf760 msgr2=0x7f627815ac20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:05.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.735+0000 7f627e9a2640 1 -- 172.21.15.27:0/3450926722 shutdown_connections 2023-12-19T18:24:05.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:05.735+0000 7f627e9a2640 1 -- 172.21.15.27:0/3450926722 wait complete. 2023-12-19T18:24:05.755 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:24:05.756 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:24:06.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.003+0000 7f6fdb434640 1 Processor -- start 2023-12-19T18:24:06.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.004+0000 7f6fdb434640 1 -- start start 2023-12-19T18:24:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.005+0000 7f6fdb434640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd40cb840 0x7f6fd4167cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.005+0000 7f6fdb434640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd4160920 con 0x7f6fd415dc00 2023-12-19T18:24:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.005+0000 7f6fdb434640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd40cd220 con 0x7f6fd415dfe0 2023-12-19T18:24:06.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.005+0000 7f6fdb434640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd40c9fa0 con 0x7f6fd40cb840 2023-12-19T18:24:06.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.005+0000 7f6fda432640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6fd415dc00 0x7f6fd40cc6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49232/0 (socket says 172.21.15.27:49232) 2023-12-19T18:24:06.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.005+0000 7f6fda432640 1 -- 172.21.15.27:0/3080861686 learned_addr learned my addr 172.21.15.27:0/3080861686 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078732232 0 0) 0x7f6fd4160920 con 0x7f6fd415dc00 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fc0003520 con 0x7f6fd415dc00 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fdac33640 1 --2- 172.21.15.27:0/3080861686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd40cb840 0x7f6fd4167cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 730076906 0 0) 0x7f6fc0003520 con 0x7f6fd415dc00 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd40cb840 msgr2=0x7f6fd4167cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 --2- 172.21.15.27:0/3080861686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd40cb840 0x7f6fd4167cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 >> v1:172.21.15.192:6789/0 conn(0x7f6fd415dfe0 legacy=0x7f6fd40cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd4160be0 con 0x7f6fd415dc00 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f6fd0001670 con 0x7f6fd415dc00 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 >> v1:172.21.15.27:6789/0 conn(0x7f6fd415dc00 legacy=0x7f6fd40cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 --2- 172.21.15.27:0/3080861686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fc00039a0 0x7f6fc0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 --2- 172.21.15.27:0/3080861686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc00047c0 0x7f6fc0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 --2- 172.21.15.27:0/3080861686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc0005250 0x7f6fc000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd40c9fa0 con 0x7f6fc0005250 2023-12-19T18:24:06.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc000c170 con 0x7f6fc00047c0 2023-12-19T18:24:06.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fc000c330 con 0x7f6fc00039a0 2023-12-19T18:24:06.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.006+0000 7f6fdac33640 1 --2- 172.21.15.27:0/3080861686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fd40cb840 0x7f6fd4167cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:24:06.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.007+0000 7f6fdac33640 1 --2- 172.21.15.27:0/3080861686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc0005250 0x7f6fc000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:06.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.007+0000 7f6fda432640 1 --2- 172.21.15.27:0/3080861686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fc00039a0 0x7f6fc0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:06.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.007+0000 7f6fd9c31640 1 --2- 172.21.15.27:0/3080861686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc00047c0 0x7f6fc0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:06.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.007+0000 7f6fdac33640 1 -- 172.21.15.27:0/3080861686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fc00039a0 msgr2=0x7f6fc0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:06.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.007+0000 7f6fdac33640 1 --2- 172.21.15.27:0/3080861686 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fc00039a0 0x7f6fc0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:06.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.007+0000 7f6fdac33640 1 -- 172.21.15.27:0/3080861686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc00047c0 msgr2=0x7f6fc0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:06.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.007+0000 7f6fdac33640 1 --2- 172.21.15.27:0/3080861686 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fc00047c0 0x7f6fc0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:06.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.007+0000 7f6fdac33640 1 -- 172.21.15.27:0/3080861686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fc0003520 con 0x7f6fc0005250 2023-12-19T18:24:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.008+0000 7f6fdac33640 1 --2- 172.21.15.27:0/3080861686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc0005250 0x7f6fc000b980 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009f00 tx=0x7f6fc00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e098d462a0df847d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.009+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc02cc70 con 0x7f6fc0005250 2023-12-19T18:24:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.009+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc02cdd0 con 0x7f6fc0005250 2023-12-19T18:24:06.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.009+0000 7f6fd9430640 1 -- 172.21.15.27:0/3080861686 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc003bb0 con 0x7f6fc0005250 2023-12-19T18:24:06.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.009+0000 7f6fdb434640 1 -- 172.21.15.27:0/3080861686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc0005250 msgr2=0x7f6fc000b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:06.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.009+0000 7f6fdb434640 1 --2- 172.21.15.27:0/3080861686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fc0005250 0x7f6fc000b980 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc009f00 tx=0x7f6fc00057f0 comp rx=0 tx=0).stop 2023-12-19T18:24:06.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.009+0000 7f6fdac33640 1 -- 172.21.15.27:0/3080861686 reap_dead start 2023-12-19T18:24:06.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.009+0000 7f6fdb434640 1 -- 172.21.15.27:0/3080861686 shutdown_connections 2023-12-19T18:24:06.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.009+0000 7f6fdb434640 1 -- 172.21.15.27:0/3080861686 >> 172.21.15.27:0/3080861686 conn(0x7f6fd40b05b0 msgr2=0x7f6fd40b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:06.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.009+0000 7f6fdb434640 1 -- 172.21.15.27:0/3080861686 shutdown_connections 2023-12-19T18:24:06.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.010+0000 7f6fdb434640 1 -- 172.21.15.27:0/3080861686 wait complete. 2023-12-19T18:24:06.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.012+0000 7f6fdb434640 1 Processor -- start 2023-12-19T18:24:06.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.012+0000 7f6fdb434640 1 -- start start 2023-12-19T18:24:06.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.013+0000 7f6fdb434640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fc0005250 0x7f6fd414d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:06.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.013+0000 7f6fdb434640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fd414f5c0 0x7f6fd41f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:06.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.013+0000 7f6fdb434640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fd40c51b0 0x7f6fd41f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:06.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.013+0000 7f6fdb434640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd41f24f0 con 0x7f6fd40c51b0 2023-12-19T18:24:06.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.013+0000 7f6fdb434640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd41f5fe0 con 0x7f6fd414f5c0 2023-12-19T18:24:06.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.013+0000 7f6fdb434640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd41f62c0 con 0x7f6fc0005250 2023-12-19T18:24:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.013+0000 7f6fdac33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fd40c51b0 0x7f6fd41f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.013+0000 7f6fdac33640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fd40c51b0 0x7f6fd41f1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38144/0 (socket says 172.21.15.27:38144) 2023-12-19T18:24:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.013+0000 7f6fdac33640 1 -- 172.21.15.27:0/1334762726 learned_addr learned my addr 172.21.15.27:0/1334762726 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.013+0000 7f6fda432640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fc0005250 0x7f6fd414d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.013+0000 7f6fd9c31640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fd414f5c0 0x7f6fd41f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.014+0000 7f6fdac33640 1 -- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fc0005250 msgr2=0x7f6fd414d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.014+0000 7f6fdac33640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fc0005250 0x7f6fd414d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.014+0000 7f6fdac33640 1 -- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fd414f5c0 msgr2=0x7f6fd41f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.014+0000 7f6fdac33640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fd414f5c0 0x7f6fd41f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.014+0000 7f6fdac33640 1 -- 172.21.15.27:0/1334762726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd41f65a0 con 0x7f6fd40c51b0 2023-12-19T18:24:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.014+0000 7f6fdac33640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fd40c51b0 0x7f6fd41f1fb0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00d0a0 tx=0x7f6fcc004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:06.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.014+0000 7f6fcaffd640 1 -- 172.21.15.27:0/1334762726 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc03c070 con 0x7f6fd40c51b0 2023-12-19T18:24:06.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.014+0000 7f6fcaffd640 1 -- 172.21.15.27:0/1334762726 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc003e50 con 0x7f6fd40c51b0 2023-12-19T18:24:06.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.014+0000 7f6fdb434640 1 -- 172.21.15.27:0/1334762726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd41f6880 con 0x7f6fd40c51b0 2023-12-19T18:24:06.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.014+0000 7f6fcaffd640 1 -- 172.21.15.27:0/1334762726 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0046f0 con 0x7f6fd40c51b0 2023-12-19T18:24:06.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.015+0000 7f6fdb434640 1 -- 172.21.15.27:0/1334762726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd414df00 con 0x7f6fd40c51b0 2023-12-19T18:24:06.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.016+0000 7f6fdb434640 1 -- 172.21.15.27:0/1334762726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd41f6a10 con 0x7f6fd40c51b0 2023-12-19T18:24:06.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.017+0000 7f6fcaffd640 1 -- 172.21.15.27:0/1334762726 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6fcc037460 con 0x7f6fd40c51b0 2023-12-19T18:24:06.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.018+0000 7f6fcaffd640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6fb806a430 0x7f6fb806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:06.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.018+0000 7f6fcaffd640 1 -- 172.21.15.27:0/1334762726 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6fcc0a9be0 con 0x7f6fd40c51b0 2023-12-19T18:24:06.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.018+0000 7f6fda432640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6fb806a430 0x7f6fb806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:06.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.019+0000 7f6fda432640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6fb806a430 0x7f6fb806c8f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0001eb0 tx=0x7f6fd0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:06.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.022+0000 7f6fcaffd640 1 -- 172.21.15.27:0/1334762726 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6fcc075510 con 0x7f6fd40c51b0 2023-12-19T18:24:06.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.214+0000 7f6fdb434640 1 -- 172.21.15.27:0/1334762726 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6fd414df00 con 0x7f6fd40c51b0 2023-12-19T18:24:06.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.822+0000 7f6fcaffd640 1 -- 172.21.15.27:0/1334762726 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v167) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6fd414df00 con 0x7f6fd40c51b0 2023-12-19T18:24:06.823 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-12-19T18:24:06.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.829+0000 7f6fdb434640 1 -- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6fb806a430 msgr2=0x7f6fb806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:06.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.829+0000 7f6fdb434640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6fb806a430 0x7f6fb806c8f0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f6fd0001eb0 tx=0x7f6fd0009040 comp rx=0 tx=0).stop 2023-12-19T18:24:06.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.829+0000 7f6fdb434640 1 -- 172.21.15.27:0/1334762726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fd40c51b0 msgr2=0x7f6fd41f1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:06.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.829+0000 7f6fdb434640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fd40c51b0 0x7f6fd41f1fb0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00d0a0 tx=0x7f6fcc004ab0 comp rx=0 tx=0).stop 2023-12-19T18:24:06.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.829+0000 7f6fdb434640 1 -- 172.21.15.27:0/1334762726 shutdown_connections 2023-12-19T18:24:06.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.829+0000 7f6fdb434640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6fb806a430 0x7f6fb806c8f0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:06.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.829+0000 7f6fdb434640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6fd40c51b0 0x7f6fd41f1fb0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:06.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.829+0000 7f6fdb434640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6fd414f5c0 0x7f6fd41f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:06.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.829+0000 7f6fdb434640 1 --2- 172.21.15.27:0/1334762726 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6fc0005250 0x7f6fd414d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:06.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.829+0000 7f6fdb434640 1 -- 172.21.15.27:0/1334762726 >> 172.21.15.27:0/1334762726 conn(0x7f6fd40b05b0 msgr2=0x7f6fd415e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:06.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.830+0000 7f6fdb434640 1 -- 172.21.15.27:0/1334762726 shutdown_connections 2023-12-19T18:24:06.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:06.830+0000 7f6fdb434640 1 -- 172.21.15.27:0/1334762726 wait complete. 2023-12-19T18:24:06.852 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:24:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.096+0000 7f1f87fff640 1 Processor -- start 2023-12-19T18:24:07.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.097+0000 7f1f87fff640 1 -- start start 2023-12-19T18:24:07.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.098+0000 7f1f87fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1f88070650 0x7f1f8815a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:07.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.098+0000 7f1f87fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f880ccba0 con 0x7f1f88070ac0 2023-12-19T18:24:07.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.098+0000 7f1f87fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f8815b070 con 0x7f1f8806c030 2023-12-19T18:24:07.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.098+0000 7f1f87fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8806f960 con 0x7f1f88070650 2023-12-19T18:24:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.099+0000 7f1f877fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1f88070ac0 0x7f1f880cd030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49248/0 (socket says 172.21.15.27:49248) 2023-12-19T18:24:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.099+0000 7f1f877fe640 1 -- 172.21.15.27:0/1914326413 learned_addr learned my addr 172.21.15.27:0/1914326413 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:07.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.099+0000 7f1f7ffff640 1 --2- 172.21.15.27:0/1914326413 >> v2:172.21.15.27:3300/0 conn(0x7f1f88070650 0x7f1f8815a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.099+0000 7f1f867fc640 1 -- 172.21.15.27:0/1914326413 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2023513505 0 0) 0x7f1f880ccba0 con 0x7f1f88070ac0 2023-12-19T18:24:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.099+0000 7f1f867fc640 1 -- 172.21.15.27:0/1914326413 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f74003520 con 0x7f1f88070ac0 2023-12-19T18:24:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.099+0000 7f1f7ffff640 1 -- 172.21.15.27:0/1914326413 >> v1:172.21.15.192:6789/0 conn(0x7f1f8806c030 legacy=0x7f1f8806c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.099+0000 7f1f7ffff640 1 -- 172.21.15.27:0/1914326413 >> v1:172.21.15.27:6789/0 conn(0x7f1f88070ac0 legacy=0x7f1f880cd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:07.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.099+0000 7f1f7ffff640 1 -- 172.21.15.27:0/1914326413 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f880701c0 con 0x7f1f88070650 2023-12-19T18:24:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.101+0000 7f1f7ffff640 1 --2- 172.21.15.27:0/1914326413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f88070650 0x7f1f8815a140 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f1f700097a0 tx=0x7f1f7002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0725cd586c76a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.101+0000 7f1f867fc640 1 -- 172.21.15.27:0/1914326413 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70031830 con 0x7f1f88070650 2023-12-19T18:24:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.101+0000 7f1f867fc640 1 -- 172.21.15.27:0/1914326413 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f70031990 con 0x7f1f88070650 2023-12-19T18:24:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.101+0000 7f1f867fc640 1 -- 172.21.15.27:0/1914326413 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70031d20 con 0x7f1f88070650 2023-12-19T18:24:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.101+0000 7f1f87fff640 1 -- 172.21.15.27:0/1914326413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f88070650 msgr2=0x7f1f8815a140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:07.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.101+0000 7f1f87fff640 1 --2- 172.21.15.27:0/1914326413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f88070650 0x7f1f8815a140 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f1f700097a0 tx=0x7f1f7002b8d0 comp rx=0 tx=0).stop 2023-12-19T18:24:07.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.102+0000 7f1f87fff640 1 -- 172.21.15.27:0/1914326413 shutdown_connections 2023-12-19T18:24:07.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.102+0000 7f1f87fff640 1 --2- 172.21.15.27:0/1914326413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f88070650 0x7f1f8815a140 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:07.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.102+0000 7f1f87fff640 1 -- 172.21.15.27:0/1914326413 >> 172.21.15.27:0/1914326413 conn(0x7f1f880cb3f0 msgr2=0x7f1f880cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:07.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.102+0000 7f1f87fff640 1 -- 172.21.15.27:0/1914326413 shutdown_connections 2023-12-19T18:24:07.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.102+0000 7f1f87fff640 1 -- 172.21.15.27:0/1914326413 wait complete. 2023-12-19T18:24:07.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.104+0000 7f1f87fff640 1 Processor -- start 2023-12-19T18:24:07.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.104+0000 7f1f87fff640 1 -- start start 2023-12-19T18:24:07.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.105+0000 7f1f87fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8806c030 0x7f1f8814f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:07.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.105+0000 7f1f87fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f88070650 0x7f1f880b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:07.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.105+0000 7f1f87fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f88070ac0 0x7f1f8814eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:07.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.105+0000 7f1f87fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f880b2b40 con 0x7f1f88070650 2023-12-19T18:24:07.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.105+0000 7f1f87fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f88152d90 con 0x7f1f8806c030 2023-12-19T18:24:07.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.105+0000 7f1f87fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8814f0b0 con 0x7f1f88070ac0 2023-12-19T18:24:07.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.105+0000 7f1f7ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f88070650 0x7f1f880b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:07.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.105+0000 7f1f7ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f88070650 0x7f1f880b2600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38168/0 (socket says 172.21.15.27:38168) 2023-12-19T18:24:07.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.105+0000 7f1f7ffff640 1 -- 172.21.15.27:0/3371799725 learned_addr learned my addr 172.21.15.27:0/3371799725 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.105+0000 7f1f86ffd640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8806c030 0x7f1f8814f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.106+0000 7f1f877fe640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f88070ac0 0x7f1f8814eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.106+0000 7f1f7ffff640 1 -- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f88070ac0 msgr2=0x7f1f8814eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.106+0000 7f1f7ffff640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f88070ac0 0x7f1f8814eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.106+0000 7f1f7ffff640 1 -- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8806c030 msgr2=0x7f1f8814f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.106+0000 7f1f7ffff640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8806c030 0x7f1f8814f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.106+0000 7f1f7ffff640 1 -- 172.21.15.27:0/3371799725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f70009400 con 0x7f1f88070650 2023-12-19T18:24:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.106+0000 7f1f86ffd640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8806c030 0x7f1f8814f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:24:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.106+0000 7f1f877fe640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f88070ac0 0x7f1f8814eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:07.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.106+0000 7f1f7ffff640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f88070650 0x7f1f880b2600 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f1f7002bee0 tx=0x7f1f700046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:07.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.106+0000 7f1f84ff9640 1 -- 172.21.15.27:0/3371799725 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7003a6f0 con 0x7f1f88070650 2023-12-19T18:24:07.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.106+0000 7f1f84ff9640 1 -- 172.21.15.27:0/3371799725 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f7003a870 con 0x7f1f88070650 2023-12-19T18:24:07.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.106+0000 7f1f84ff9640 1 -- 172.21.15.27:0/3371799725 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7003ac20 con 0x7f1f88070650 2023-12-19T18:24:07.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.106+0000 7f1f87fff640 1 -- 172.21.15.27:0/3371799725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f88153030 con 0x7f1f88070650 2023-12-19T18:24:07.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.107+0000 7f1f87fff640 1 -- 172.21.15.27:0/3371799725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f880b3fe0 con 0x7f1f88070650 2023-12-19T18:24:07.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.108+0000 7f1f87fff640 1 -- 172.21.15.27:0/3371799725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8804e840 con 0x7f1f88070650 2023-12-19T18:24:07.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.109+0000 7f1f84ff9640 1 -- 172.21.15.27:0/3371799725 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1f70030380 con 0x7f1f88070650 2023-12-19T18:24:07.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.110+0000 7f1f84ff9640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6806a430 0x7f1f6806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:07.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.110+0000 7f1f84ff9640 1 -- 172.21.15.27:0/3371799725 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(167..167 src has 1..167) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f1f700aa450 con 0x7f1f88070650 2023-12-19T18:24:07.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.111+0000 7f1f86ffd640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6806a430 0x7f1f6806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:07.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.112+0000 7f1f86ffd640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6806a430 0x7f1f6806c8f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1f88159cb0 tx=0x7f1f78009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:07.114 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.114+0000 7f1f84ff9640 1 -- 172.21.15.27:0/3371799725 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1f70075d00 con 0x7f1f88070650 2023-12-19T18:24:07.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.307+0000 7f1f87fff640 1 -- 172.21.15.27:0/3371799725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1f880b48f0 con 0x7f1f88070650 2023-12-19T18:24:07.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.837+0000 7f1f84ff9640 1 -- 172.21.15.27:0/3371799725 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v168) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1f70079320 con 0x7f1f88070650 2023-12-19T18:24:07.840 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-12-19T18:24:07.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.843+0000 7f1f87fff640 1 -- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6806a430 msgr2=0x7f1f6806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:07.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.843+0000 7f1f87fff640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6806a430 0x7f1f6806c8f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1f88159cb0 tx=0x7f1f78009040 comp rx=0 tx=0).stop 2023-12-19T18:24:07.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.843+0000 7f1f87fff640 1 -- 172.21.15.27:0/3371799725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f88070650 msgr2=0x7f1f880b2600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:07.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.843+0000 7f1f87fff640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f88070650 0x7f1f880b2600 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f1f7002bee0 tx=0x7f1f700046c0 comp rx=0 tx=0).stop 2023-12-19T18:24:07.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.846+0000 7f1f87fff640 1 -- 172.21.15.27:0/3371799725 shutdown_connections 2023-12-19T18:24:07.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.846+0000 7f1f87fff640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1f6806a430 0x7f1f6806c8f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:07.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.846+0000 7f1f87fff640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1f88070ac0 0x7f1f8814eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:07.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.846+0000 7f1f87fff640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f88070650 0x7f1f880b2600 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:07.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.846+0000 7f1f87fff640 1 --2- 172.21.15.27:0/3371799725 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1f8806c030 0x7f1f8814f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:07.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.846+0000 7f1f87fff640 1 -- 172.21.15.27:0/3371799725 >> 172.21.15.27:0/3371799725 conn(0x7f1f880cb3f0 msgr2=0x7f1f88157fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:07.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.847+0000 7f1f87fff640 1 -- 172.21.15.27:0/3371799725 shutdown_connections 2023-12-19T18:24:07.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:07.848+0000 7f1f87fff640 1 -- 172.21.15.27:0/3371799725 wait complete. 2023-12-19T18:24:07.874 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:24:08.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.077+0000 7f665359e640 1 Processor -- start 2023-12-19T18:24:08.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.078+0000 7f665359e640 1 -- start start 2023-12-19T18:24:08.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.079+0000 7f665359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f665415e9d0 0x7f66540cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.079+0000 7f665359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665415b3e0 con 0x7f6654152070 2023-12-19T18:24:08.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.079+0000 7f665359e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f665415eed0 con 0x7f6654151c90 2023-12-19T18:24:08.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.079+0000 7f665359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665415f1b0 con 0x7f665415e9d0 2023-12-19T18:24:08.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.080+0000 7f6652d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6654152070 0x7f665415b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49262/0 (socket says 172.21.15.27:49262) 2023-12-19T18:24:08.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.080+0000 7f6652d9d640 1 -- 172.21.15.27:0/1473420351 learned_addr learned my addr 172.21.15.27:0/1473420351 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:08.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.080+0000 7f664bfff640 1 --2- 172.21.15.27:0/1473420351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f665415e9d0 0x7f66540cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.080+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041932453 0 0) 0x7f665415b3e0 con 0x7f6654152070 2023-12-19T18:24:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.080+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6640003520 con 0x7f6654152070 2023-12-19T18:24:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2272228559 0 0) 0x7f6640003520 con 0x7f6654152070 2023-12-19T18:24:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f665415e9d0 msgr2=0x7f66540cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f6651d9b640 1 --2- 172.21.15.27:0/1473420351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f665415e9d0 0x7f66540cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 >> v1:172.21.15.192:6789/0 conn(0x7f6654151c90 legacy=0x7f665406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f665415f490 con 0x7f6654152070 2023-12-19T18:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f664c0019f0 con 0x7f6654152070 2023-12-19T18:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 >> v1:172.21.15.27:6789/0 conn(0x7f6654152070 legacy=0x7f665415b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f6651d9b640 1 --2- 172.21.15.27:0/1473420351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66400039a0 0x7f6640003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f6651d9b640 1 --2- 172.21.15.27:0/1473420351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f665415e9d0 0x7f66400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f6651d9b640 1 --2- 172.21.15.27:0/1473420351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6640004d00 0x7f6640005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665415f1b0 con 0x7f66400039a0 2023-12-19T18:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f664000ba30 con 0x7f6640004d00 2023-12-19T18:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f664000bbf0 con 0x7f665415e9d0 2023-12-19T18:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.081+0000 7f665259c640 1 --2- 172.21.15.27:0/1473420351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66400039a0 0x7f6640003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.082+0000 7f664bfff640 1 --2- 172.21.15.27:0/1473420351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f665415e9d0 0x7f66400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.082+0000 7f665259c640 1 -- 172.21.15.27:0/1473420351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f665415e9d0 msgr2=0x7f66400047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.082+0000 7f665259c640 1 --2- 172.21.15.27:0/1473420351 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f665415e9d0 0x7f66400047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.082+0000 7f665259c640 1 -- 172.21.15.27:0/1473420351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6640004d00 msgr2=0x7f6640005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.082+0000 7f6652d9d640 1 --2- 172.21.15.27:0/1473420351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6640004d00 0x7f6640005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.082+0000 7f665259c640 1 --2- 172.21.15.27:0/1473420351 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6640004d00 0x7f6640005180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.082+0000 7f665259c640 1 -- 172.21.15.27:0/1473420351 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6640003520 con 0x7f66400039a0 2023-12-19T18:24:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.083+0000 7f665259c640 1 --2- 172.21.15.27:0/1473420351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66400039a0 0x7f6640003d80 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f6644001860 tx=0x7f66400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b495d2765e04402e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.083+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644033630 con 0x7f66400039a0 2023-12-19T18:24:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.083+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66440337b0 con 0x7f66400039a0 2023-12-19T18:24:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.083+0000 7f6651d9b640 1 -- 172.21.15.27:0/1473420351 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6644033b60 con 0x7f66400039a0 2023-12-19T18:24:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.084+0000 7f665359e640 1 -- 172.21.15.27:0/1473420351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66400039a0 msgr2=0x7f6640003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.084+0000 7f665359e640 1 --2- 172.21.15.27:0/1473420351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66400039a0 0x7f6640003d80 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7f6644001860 tx=0x7f66400057f0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.084+0000 7f6652d9d640 1 -- 172.21.15.27:0/1473420351 reap_dead start 2023-12-19T18:24:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.084+0000 7f665359e640 1 -- 172.21.15.27:0/1473420351 shutdown_connections 2023-12-19T18:24:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.084+0000 7f665359e640 1 -- 172.21.15.27:0/1473420351 >> 172.21.15.27:0/1473420351 conn(0x7f665406a060 msgr2=0x7f665406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.084+0000 7f665359e640 1 -- 172.21.15.27:0/1473420351 shutdown_connections 2023-12-19T18:24:08.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.084+0000 7f665359e640 1 -- 172.21.15.27:0/1473420351 wait complete. 2023-12-19T18:24:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.086+0000 7f665359e640 1 Processor -- start 2023-12-19T18:24:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.086+0000 7f665359e640 1 -- start start 2023-12-19T18:24:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.087+0000 7f665359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66400039a0 0x7f665415b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.087+0000 7f665359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6654148090 0x7f6654149a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.087+0000 7f665359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6654149fc0 0x7f66540b0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.087+0000 7f665359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66541e5c20 con 0x7f6654149fc0 2023-12-19T18:24:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.087+0000 7f665359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66541e5f00 con 0x7f6654148090 2023-12-19T18:24:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.087+0000 7f665359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66541e61e0 con 0x7f66400039a0 2023-12-19T18:24:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.088+0000 7f6652d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6654149fc0 0x7f66540b0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.088+0000 7f6652d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6654149fc0 0x7f66540b0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38174/0 (socket says 172.21.15.27:38174) 2023-12-19T18:24:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.088+0000 7f6652d9d640 1 -- 172.21.15.27:0/3724628724 learned_addr learned my addr 172.21.15.27:0/3724628724 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.088+0000 7f665259c640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66400039a0 0x7f665415b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.088+0000 7f664bfff640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6654148090 0x7f6654149a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.088+0000 7f6652d9d640 1 -- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66400039a0 msgr2=0x7f665415b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.088+0000 7f6652d9d640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66400039a0 0x7f665415b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.088+0000 7f6652d9d640 1 -- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6654148090 msgr2=0x7f6654149a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.088+0000 7f6652d9d640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6654148090 0x7f6654149a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.088+0000 7f6652d9d640 1 -- 172.21.15.27:0/3724628724 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66541e64c0 con 0x7f6654149fc0 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.089+0000 7f6652d9d640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6654149fc0 0x7f66540b0ab0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f664c001460 tx=0x7f664c00b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.089+0000 7f664affd640 1 -- 172.21.15.27:0/3724628724 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c00b8d0 con 0x7f6654149fc0 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.089+0000 7f664affd640 1 -- 172.21.15.27:0/3724628724 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f664c00ba30 con 0x7f6654149fc0 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.089+0000 7f664affd640 1 -- 172.21.15.27:0/3724628724 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664c00bde0 con 0x7f6654149fc0 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.089+0000 7f665359e640 1 -- 172.21.15.27:0/3724628724 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f665414a540 con 0x7f6654149fc0 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.089+0000 7f665359e640 1 -- 172.21.15.27:0/3724628724 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6654148510 con 0x7f6654149fc0 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.090+0000 7f665359e640 1 -- 172.21.15.27:0/3724628724 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f665404e840 con 0x7f6654149fc0 2023-12-19T18:24:08.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.091+0000 7f664affd640 1 -- 172.21.15.27:0/3724628724 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f664c007070 con 0x7f6654149fc0 2023-12-19T18:24:08.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.092+0000 7f664affd640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f663406a430 0x7f663406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.092+0000 7f664affd640 1 -- 172.21.15.27:0/3724628724 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f664c088b90 con 0x7f6654149fc0 2023-12-19T18:24:08.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.093+0000 7f665259c640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f663406a430 0x7f663406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.094+0000 7f665259c640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f663406a430 0x7f663406c8f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6644036000 tx=0x7f6644035040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:08.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.097+0000 7f664affd640 1 -- 172.21.15.27:0/3724628724 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f664c0540b0 con 0x7f6654149fc0 2023-12-19T18:24:08.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.340+0000 7f665359e640 1 -- 172.21.15.27:0/3724628724 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f66541e5d60 con 0x7f6654149fc0 2023-12-19T18:24:08.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.349+0000 7f664affd640 1 -- 172.21.15.27:0/3724628724 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v65) v1 ==== 174+0+0 (secure 0 0 0) 0x7f66541e5d60 con 0x7f6654149fc0 2023-12-19T18:24:08.355 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 18 and data pool 19 2023-12-19T18:24:08.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.356+0000 7f665359e640 1 -- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f663406a430 msgr2=0x7f663406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:08.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.356+0000 7f665359e640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f663406a430 0x7f663406c8f0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6644036000 tx=0x7f6644035040 comp rx=0 tx=0).stop 2023-12-19T18:24:08.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.356+0000 7f665359e640 1 -- 172.21.15.27:0/3724628724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6654149fc0 msgr2=0x7f66540b0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:08.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.356+0000 7f665359e640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6654149fc0 0x7f66540b0ab0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f664c001460 tx=0x7f664c00b0d0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.356+0000 7f665359e640 1 -- 172.21.15.27:0/3724628724 shutdown_connections 2023-12-19T18:24:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.356+0000 7f665359e640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f663406a430 0x7f663406c8f0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.356+0000 7f665359e640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6654149fc0 0x7f66540b0ab0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.356+0000 7f665359e640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6654148090 0x7f6654149a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.356+0000 7f665359e640 1 --2- 172.21.15.27:0/3724628724 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f66400039a0 0x7f665415b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.357+0000 7f665359e640 1 -- 172.21.15.27:0/3724628724 >> 172.21.15.27:0/3724628724 conn(0x7f665406a060 msgr2=0x7f66540ac8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.358+0000 7f665359e640 1 -- 172.21.15.27:0/3724628724 shutdown_connections 2023-12-19T18:24:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.358+0000 7f665359e640 1 -- 172.21.15.27:0/3724628724 wait complete. 2023-12-19T18:24:08.387 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:24:08.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.582+0000 7f9919316640 1 Processor -- start 2023-12-19T18:24:08.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.582+0000 7f9919316640 1 -- start start 2023-12-19T18:24:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.584+0000 7f9919316640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f991415ce90 0x7f99140cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.584+0000 7f9919316640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f991415d270 0x7f99140ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.584+0000 7f9919316640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991406b4e0 con 0x7f99140ade10 2023-12-19T18:24:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.584+0000 7f9919316640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991406ce90 con 0x7f991415ce90 2023-12-19T18:24:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.584+0000 7f9919316640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99140704f0 con 0x7f991415d270 2023-12-19T18:24:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.584+0000 7f99137fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f99140ade10 0x7f99140cb2b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49278/0 (socket says 172.21.15.27:49278) 2023-12-19T18:24:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.584+0000 7f99137fe640 1 -- 172.21.15.27:0/1014599863 learned_addr learned my addr 172.21.15.27:0/1014599863 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:08.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f990bfff640 1 --2- 172.21.15.27:0/1014599863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f991415d270 0x7f99140ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285500920 0 0) 0x7f991406b4e0 con 0x7f99140ade10 2023-12-19T18:24:08.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f9912ffd640 1 --2- 172.21.15.27:0/1014599863 >> v2:172.21.15.192:3300/0 conn(0x7f991415ce90 0x7f99140cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9900003520 con 0x7f99140ade10 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2838660826 0 0) 0x7f9900003520 con 0x7f99140ade10 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f991415d270 msgr2=0x7f99140ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f99127fc640 1 --2- 172.21.15.27:0/1014599863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f991415d270 0x7f99140ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 >> v2:172.21.15.192:3300/0 conn(0x7f991415ce90 msgr2=0x7f99140cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f99127fc640 1 --2- 172.21.15.27:0/1014599863 >> v2:172.21.15.192:3300/0 conn(0x7f991415ce90 0x7f99140cc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991406b7c0 con 0x7f99140ade10 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f990c0019f0 con 0x7f99140ade10 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 >> v1:172.21.15.27:6789/0 conn(0x7f99140ade10 legacy=0x7f99140cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f99127fc640 1 --2- 172.21.15.27:0/1014599863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900003970 0x7f9900003d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f99127fc640 1 --2- 172.21.15.27:0/1014599863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99000042d0 0x7f99000047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.585+0000 7f99127fc640 1 --2- 172.21.15.27:0/1014599863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9900004e10 0x7f9900005290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.586+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991406ce90 con 0x7f99000042d0 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.586+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99140704f0 con 0x7f9900004e10 2023-12-19T18:24:08.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.586+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990000bd00 con 0x7f9900003970 2023-12-19T18:24:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.586+0000 7f990bfff640 1 --2- 172.21.15.27:0/1014599863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99000042d0 0x7f99000047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.586+0000 7f9912ffd640 1 --2- 172.21.15.27:0/1014599863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900003970 0x7f9900003d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.586+0000 7f990bfff640 1 -- 172.21.15.27:0/1014599863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900003970 msgr2=0x7f9900003d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.586+0000 7f990bfff640 1 --2- 172.21.15.27:0/1014599863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900003970 0x7f9900003d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.586+0000 7f990bfff640 1 -- 172.21.15.27:0/1014599863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9900004e10 msgr2=0x7f9900005290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.586+0000 7f990bfff640 1 --2- 172.21.15.27:0/1014599863 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9900004e10 0x7f9900005290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.586+0000 7f990bfff640 1 -- 172.21.15.27:0/1014599863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9900003520 con 0x7f99000042d0 2023-12-19T18:24:08.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.586+0000 7f9912ffd640 1 --2- 172.21.15.27:0/1014599863 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9900003970 0x7f9900003d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.587+0000 7f990bfff640 1 --2- 172.21.15.27:0/1014599863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99000042d0 0x7f99000047c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009ac0 tx=0x7f99000057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b5b9804279282 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.588+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc02c6b0 con 0x7f99000042d0 2023-12-19T18:24:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.588+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98fc02c810 con 0x7f99000042d0 2023-12-19T18:24:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.588+0000 7f99127fc640 1 -- 172.21.15.27:0/1014599863 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc02cba0 con 0x7f99000042d0 2023-12-19T18:24:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.588+0000 7f9919316640 1 -- 172.21.15.27:0/1014599863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99000042d0 msgr2=0x7f99000047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:08.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.588+0000 7f9919316640 1 --2- 172.21.15.27:0/1014599863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99000042d0 0x7f99000047c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009ac0 tx=0x7f99000057f0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.588+0000 7f99137fe640 1 -- 172.21.15.27:0/1014599863 reap_dead start 2023-12-19T18:24:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.588+0000 7f9919316640 1 -- 172.21.15.27:0/1014599863 shutdown_connections 2023-12-19T18:24:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.588+0000 7f9919316640 1 -- 172.21.15.27:0/1014599863 >> 172.21.15.27:0/1014599863 conn(0x7f991406a330 msgr2=0x7f99140b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.588+0000 7f9919316640 1 -- 172.21.15.27:0/1014599863 shutdown_connections 2023-12-19T18:24:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.588+0000 7f9919316640 1 -- 172.21.15.27:0/1014599863 wait complete. 2023-12-19T18:24:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.590+0000 7f9919316640 1 Processor -- start 2023-12-19T18:24:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.590+0000 7f9919316640 1 -- start start 2023-12-19T18:24:08.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.591+0000 7f9919316640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99000042d0 0x7f99141f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.591+0000 7f9919316640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99141f6480 0x7f99140b3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.591+0000 7f9919316640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9914150710 0x7f99141f2400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.591+0000 7f9919316640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99140b2b70 con 0x7f9914150710 2023-12-19T18:24:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.591+0000 7f9919316640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99140b2e50 con 0x7f99141f6480 2023-12-19T18:24:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.591+0000 7f9919316640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99140b3130 con 0x7f99000042d0 2023-12-19T18:24:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.592+0000 7f99137fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9914150710 0x7f99141f2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.592+0000 7f99137fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9914150710 0x7f99141f2400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38188/0 (socket says 172.21.15.27:38188) 2023-12-19T18:24:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.592+0000 7f99137fe640 1 -- 172.21.15.27:0/2248569442 learned_addr learned my addr 172.21.15.27:0/2248569442 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:08.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.592+0000 7f990bfff640 1 --2- 172.21.15.27:0/2248569442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99141f6480 0x7f99140b3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.592+0000 7f99137fe640 1 -- 172.21.15.27:0/2248569442 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99000042d0 msgr2=0x7f99141f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:08.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.592+0000 7f99137fe640 1 --2- 172.21.15.27:0/2248569442 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99000042d0 0x7f99141f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.592+0000 7f99137fe640 1 -- 172.21.15.27:0/2248569442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99141f6480 msgr2=0x7f99140b3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:08.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.592+0000 7f99137fe640 1 --2- 172.21.15.27:0/2248569442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99141f6480 0x7f99140b3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.592+0000 7f99137fe640 1 -- 172.21.15.27:0/2248569442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99140b3410 con 0x7f9914150710 2023-12-19T18:24:08.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.593+0000 7f99137fe640 1 --2- 172.21.15.27:0/2248569442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9914150710 0x7f99141f2400 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f990c0012e0 tx=0x7f990c00cf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:08.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.593+0000 7f990b7fe640 1 -- 172.21.15.27:0/2248569442 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00a610 con 0x7f9914150710 2023-12-19T18:24:08.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.593+0000 7f990b7fe640 1 -- 172.21.15.27:0/2248569442 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f990c00a7b0 con 0x7f9914150710 2023-12-19T18:24:08.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.593+0000 7f9919316640 1 -- 172.21.15.27:0/2248569442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99140b36f0 con 0x7f9914150710 2023-12-19T18:24:08.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.593+0000 7f990b7fe640 1 -- 172.21.15.27:0/2248569442 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990c00ab60 con 0x7f9914150710 2023-12-19T18:24:08.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.593+0000 7f9919316640 1 -- 172.21.15.27:0/2248569442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99140b4150 con 0x7f9914150710 2023-12-19T18:24:08.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.594+0000 7f9919316640 1 -- 172.21.15.27:0/2248569442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9914068e90 con 0x7f9914150710 2023-12-19T18:24:08.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.595+0000 7f990b7fe640 1 -- 172.21.15.27:0/2248569442 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f990c00acc0 con 0x7f9914150710 2023-12-19T18:24:08.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.596+0000 7f990b7fe640 1 --2- 172.21.15.27:0/2248569442 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f98f406a460 0x7f98f406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:08.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.596+0000 7f990b7fe640 1 -- 172.21.15.27:0/2248569442 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f990c0886c0 con 0x7f9914150710 2023-12-19T18:24:08.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.597+0000 7f9912ffd640 1 --2- 172.21.15.27:0/2248569442 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f98f406a460 0x7f98f406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:08.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.598+0000 7f9912ffd640 1 --2- 172.21.15.27:0/2248569442 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f98f406a460 0x7f98f406c920 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f99140c55f0 tx=0x7f9904008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:08.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.601+0000 7f990b7fe640 1 -- 172.21.15.27:0/2248569442 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f990c053ca0 con 0x7f9914150710 2023-12-19T18:24:08.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.796+0000 7f9919316640 1 -- 172.21.15.27:0/2248569442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f99140b4ac0 con 0x7f9914150710 2023-12-19T18:24:08.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.799+0000 7f990b7fe640 1 -- 172.21.15.27:0/2248569442 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+16073 (secure 0 0 0) 0x7f990c0572c0 con 0x7f9914150710 2023-12-19T18:24:08.799 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:24:08.799 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":169,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:24:08.342463+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:24:06.280682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T18:24:07.374789+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:24:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.801+0000 7f9919316640 1 -- 172.21.15.27:0/2248569442 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f98f406a460 msgr2=0x7f98f406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.801+0000 7f9919316640 1 --2- 172.21.15.27:0/2248569442 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f98f406a460 0x7f98f406c920 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f99140c55f0 tx=0x7f9904008040 comp rx=0 tx=0).stop 2023-12-19T18:24:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.801+0000 7f9919316640 1 -- 172.21.15.27:0/2248569442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9914150710 msgr2=0x7f99141f2400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.801+0000 7f9919316640 1 --2- 172.21.15.27:0/2248569442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9914150710 0x7f99141f2400 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f990c0012e0 tx=0x7f990c00cf00 comp rx=0 tx=0).stop 2023-12-19T18:24:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.802+0000 7f9919316640 1 -- 172.21.15.27:0/2248569442 shutdown_connections 2023-12-19T18:24:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.802+0000 7f9919316640 1 --2- 172.21.15.27:0/2248569442 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f98f406a460 0x7f98f406c920 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.802+0000 7f9919316640 1 --2- 172.21.15.27:0/2248569442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9914150710 0x7f99141f2400 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.802+0000 7f9919316640 1 --2- 172.21.15.27:0/2248569442 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99141f6480 0x7f99140b3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.802+0000 7f9919316640 1 --2- 172.21.15.27:0/2248569442 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99000042d0 0x7f99141f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.802+0000 7f9919316640 1 -- 172.21.15.27:0/2248569442 >> 172.21.15.27:0/2248569442 conn(0x7f991406a330 msgr2=0x7f99140af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.802+0000 7f9919316640 1 -- 172.21.15.27:0/2248569442 shutdown_connections 2023-12-19T18:24:08.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:08.802+0000 7f9919316640 1 -- 172.21.15.27:0/2248569442 wait complete. 2023-12-19T18:24:08.823 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:24:09.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.049+0000 7f08525a9640 1 Processor -- start 2023-12-19T18:24:09.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.049+0000 7f08525a9640 1 -- start start 2023-12-19T18:24:09.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.051+0000 7f08525a9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f084c15ee30 0x7f084c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.051+0000 7f08525a9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f084c0cb350 con 0x7f084c1553a0 2023-12-19T18:24:09.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.051+0000 7f08525a9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f084c0cb630 con 0x7f084c0afeb0 2023-12-19T18:24:09.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.051+0000 7f08525a9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f084c0cbcf0 con 0x7f084c15ee30 2023-12-19T18:24:09.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.051+0000 7f08515a7640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f084c1553a0 0x7f084c15e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49286/0 (socket says 172.21.15.27:49286) 2023-12-19T18:24:09.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.051+0000 7f08515a7640 1 -- 172.21.15.27:0/2805130908 learned_addr learned my addr 172.21.15.27:0/2805130908 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:09.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.051+0000 7f0850da6640 1 --2- 172.21.15.27:0/2805130908 >> v2:172.21.15.192:3300/0 conn(0x7f084c15ee30 0x7f084c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.051+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2356701441 0 0) 0x7f084c0cb350 con 0x7f084c1553a0 2023-12-19T18:24:09.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0838003520 con 0x7f084c1553a0 2023-12-19T18:24:09.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2400539680 0 0) 0x7f0838003520 con 0x7f084c1553a0 2023-12-19T18:24:09.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 >> v2:172.21.15.192:3300/0 conn(0x7f084c15ee30 msgr2=0x7f084c0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 --2- 172.21.15.27:0/2805130908 >> v2:172.21.15.192:3300/0 conn(0x7f084c15ee30 0x7f084c0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 >> v1:172.21.15.192:6789/0 conn(0x7f084c0afeb0 legacy=0x7f084c0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c0cb910 con 0x7f084c1553a0 2023-12-19T18:24:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f0848001670 con 0x7f084c1553a0 2023-12-19T18:24:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 >> v1:172.21.15.27:6789/0 conn(0x7f084c1553a0 legacy=0x7f084c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 --2- 172.21.15.27:0/2805130908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0838003970 0x7f0838003d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 --2- 172.21.15.27:0/2805130908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08380047c0 0x7f0838004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 --2- 172.21.15.27:0/2805130908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08380052a0 0x7f083800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c0cbcf0 con 0x7f08380047c0 2023-12-19T18:24:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f083800c150 con 0x7f08380052a0 2023-12-19T18:24:09.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f083800c310 con 0x7f0838003970 2023-12-19T18:24:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.052+0000 7f0850da6640 1 --2- 172.21.15.27:0/2805130908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08380047c0 0x7f0838004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.053+0000 7f08515a7640 1 --2- 172.21.15.27:0/2805130908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0838003970 0x7f0838003d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.053+0000 7f0850da6640 1 -- 172.21.15.27:0/2805130908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0838003970 msgr2=0x7f0838003d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.053+0000 7f0850da6640 1 --2- 172.21.15.27:0/2805130908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0838003970 0x7f0838003d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.053+0000 7f0850da6640 1 -- 172.21.15.27:0/2805130908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08380052a0 msgr2=0x7f083800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.053+0000 7f0850da6640 1 --2- 172.21.15.27:0/2805130908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f08380052a0 0x7f083800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.053+0000 7f0850da6640 1 -- 172.21.15.27:0/2805130908 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0838003520 con 0x7f08380047c0 2023-12-19T18:24:09.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.053+0000 7f0851da8640 1 -- 172.21.15.27:0/2805130908 reap_dead start 2023-12-19T18:24:09.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.054+0000 7f0850da6640 1 --2- 172.21.15.27:0/2805130908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08380047c0 0x7f0838004c40 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f083c00ba00 tx=0x7f08380057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c70dd6459ea8dd42 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:09.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.054+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c032640 con 0x7f08380047c0 2023-12-19T18:24:09.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.054+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083c0327a0 con 0x7f08380047c0 2023-12-19T18:24:09.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.054+0000 7f0843fff640 1 -- 172.21.15.27:0/2805130908 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c032b30 con 0x7f08380047c0 2023-12-19T18:24:09.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.055+0000 7f08525a9640 1 -- 172.21.15.27:0/2805130908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08380047c0 msgr2=0x7f0838004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:09.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.055+0000 7f08525a9640 1 --2- 172.21.15.27:0/2805130908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08380047c0 0x7f0838004c40 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f083c00ba00 tx=0x7f08380057f0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.055+0000 7f08525a9640 1 -- 172.21.15.27:0/2805130908 shutdown_connections 2023-12-19T18:24:09.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.055+0000 7f08525a9640 1 --2- 172.21.15.27:0/2805130908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08380047c0 0x7f0838004c40 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.055+0000 7f08525a9640 1 -- 172.21.15.27:0/2805130908 >> 172.21.15.27:0/2805130908 conn(0x7f084c069c70 msgr2=0x7f084c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:09.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.055+0000 7f08525a9640 1 -- 172.21.15.27:0/2805130908 shutdown_connections 2023-12-19T18:24:09.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.055+0000 7f08525a9640 1 -- 172.21.15.27:0/2805130908 wait complete. 2023-12-19T18:24:09.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.057+0000 7f08525a9640 1 Processor -- start 2023-12-19T18:24:09.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.057+0000 7f08525a9640 1 -- start start 2023-12-19T18:24:09.059 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.058+0000 7f08525a9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08380047c0 0x7f084c147460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.058+0000 7f08525a9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f084c0b4210 0x7f084c1557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.058+0000 7f08525a9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f084c1572e0 0x7f084c157770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.058+0000 7f08525a9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c1479a0 con 0x7f084c0b4210 2023-12-19T18:24:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.058+0000 7f08525a9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f084c1588e0 con 0x7f084c1572e0 2023-12-19T18:24:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.058+0000 7f08525a9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f084c157fb0 con 0x7f08380047c0 2023-12-19T18:24:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.058+0000 7f0850da6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f084c0b4210 0x7f084c1557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.058+0000 7f0850da6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f084c0b4210 0x7f084c1557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38210/0 (socket says 172.21.15.27:38210) 2023-12-19T18:24:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.058+0000 7f0850da6640 1 -- 172.21.15.27:0/370466530 learned_addr learned my addr 172.21.15.27:0/370466530 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.059+0000 7f08515a7640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08380047c0 0x7f084c147460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.059+0000 7f0850da6640 1 -- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08380047c0 msgr2=0x7f084c147460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.059+0000 7f0850da6640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08380047c0 0x7f084c147460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.059+0000 7f0850da6640 1 -- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f084c1572e0 msgr2=0x7f084c157770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.059+0000 7f0851da8640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f084c1572e0 0x7f084c157770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.059+0000 7f0850da6640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f084c1572e0 0x7f084c157770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.059+0000 7f0850da6640 1 -- 172.21.15.27:0/370466530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f084c158290 con 0x7f084c0b4210 2023-12-19T18:24:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.059+0000 7f0850da6640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f084c0b4210 0x7f084c1557f0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f083c02bec0 tx=0x7f083c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.059+0000 7f0841ffb640 1 -- 172.21.15.27:0/370466530 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c004220 con 0x7f084c0b4210 2023-12-19T18:24:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.059+0000 7f0841ffb640 1 -- 172.21.15.27:0/370466530 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f083c004380 con 0x7f084c0b4210 2023-12-19T18:24:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.059+0000 7f0841ffb640 1 -- 172.21.15.27:0/370466530 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f083c03ba70 con 0x7f084c0b4210 2023-12-19T18:24:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.059+0000 7f08525a9640 1 -- 172.21.15.27:0/370466530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f084c158570 con 0x7f084c0b4210 2023-12-19T18:24:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.060+0000 7f08525a9640 1 -- 172.21.15.27:0/370466530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f084c0b4780 con 0x7f084c0b4210 2023-12-19T18:24:09.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.061+0000 7f08525a9640 1 -- 172.21.15.27:0/370466530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084c04e840 con 0x7f084c0b4210 2023-12-19T18:24:09.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.062+0000 7f0841ffb640 1 -- 172.21.15.27:0/370466530 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f083c03bbd0 con 0x7f084c0b4210 2023-12-19T18:24:09.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.062+0000 7f0841ffb640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f083006a430 0x7f083006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.063+0000 7f0841ffb640 1 -- 172.21.15.27:0/370466530 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f083c0ab300 con 0x7f084c0b4210 2023-12-19T18:24:09.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.063+0000 7f08515a7640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f083006a430 0x7f083006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.064 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.064+0000 7f08515a7640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f083006a430 0x7f083006c8f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0848001eb0 tx=0x7f0848001de0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:09.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.067+0000 7f0841ffb640 1 -- 172.21.15.27:0/370466530 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f083c076a10 con 0x7f084c0b4210 2023-12-19T18:24:09.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.261+0000 7f08525a9640 1 -- 172.21.15.27:0/370466530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f084c04e840 con 0x7f084c0b4210 2023-12-19T18:24:09.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.263+0000 7f0841ffb640 1 -- 172.21.15.27:0/370466530 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+16073 (secure 0 0 0) 0x7f083c07a030 con 0x7f084c0b4210 2023-12-19T18:24:09.264 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:24:09.264 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":169,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:24:08.342463+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:24:06.280682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T18:24:07.374789+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:24:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.266+0000 7f08525a9640 1 -- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f083006a430 msgr2=0x7f083006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.266+0000 7f08525a9640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f083006a430 0x7f083006c8f0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f0848001eb0 tx=0x7f0848001de0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.266+0000 7f08525a9640 1 -- 172.21.15.27:0/370466530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f084c0b4210 msgr2=0x7f084c1557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.266+0000 7f08525a9640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f084c0b4210 0x7f084c1557f0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f083c02bec0 tx=0x7f083c004730 comp rx=0 tx=0).stop 2023-12-19T18:24:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.266+0000 7f08525a9640 1 -- 172.21.15.27:0/370466530 shutdown_connections 2023-12-19T18:24:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.266+0000 7f08525a9640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f083006a430 0x7f083006c8f0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.266+0000 7f08525a9640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f084c1572e0 0x7f084c157770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.266+0000 7f08525a9640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f084c0b4210 0x7f084c1557f0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.266+0000 7f08525a9640 1 --2- 172.21.15.27:0/370466530 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f08380047c0 0x7f084c147460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.266+0000 7f08525a9640 1 -- 172.21.15.27:0/370466530 >> 172.21.15.27:0/370466530 conn(0x7f084c069c70 msgr2=0x7f084c0ca830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.266+0000 7f08525a9640 1 -- 172.21.15.27:0/370466530 shutdown_connections 2023-12-19T18:24:09.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.266+0000 7f08525a9640 1 -- 172.21.15.27:0/370466530 wait complete. 2023-12-19T18:24:09.284 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:24:09.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.509+0000 7fc8a1822640 1 Processor -- start 2023-12-19T18:24:09.509 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.509+0000 7fc8a1822640 1 -- start start 2023-12-19T18:24:09.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.511+0000 7fc8a1822640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c158790 0x7fc89c0ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.511+0000 7fc8a1822640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c15c030 con 0x7fc89c0adc50 2023-12-19T18:24:09.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.511+0000 7fc8a1822640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c0c9030 con 0x7fc89c15b9e0 2023-12-19T18:24:09.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.511+0000 7fc8a1822640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0c99d0 con 0x7fc89c158790 2023-12-19T18:24:09.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.511+0000 7fc89bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc89c0adc50 0x7fc89c15b2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49290/0 (socket says 172.21.15.27:49290) 2023-12-19T18:24:09.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.511+0000 7fc89bfff640 1 -- 172.21.15.27:0/962847358 learned_addr learned my addr 172.21.15.27:0/962847358 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc8a0820640 1 --2- 172.21.15.27:0/962847358 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c158790 0x7fc89c0ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454614972 0 0) 0x7fc89c15c030 con 0x7fc89c0adc50 2023-12-19T18:24:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc88c003520 con 0x7fc89c0adc50 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3807776430 0 0) 0x7fc88c003520 con 0x7fc89c0adc50 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c158790 msgr2=0x7fc89c0ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 --2- 172.21.15.27:0/962847358 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c158790 0x7fc89c0ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 >> v1:172.21.15.192:6789/0 conn(0x7fc89c15b9e0 legacy=0x7fc89c0c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c0c9310 con 0x7fc89c0adc50 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fc888001670 con 0x7fc89c0adc50 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 >> v1:172.21.15.27:6789/0 conn(0x7fc89c0adc50 legacy=0x7fc89c15b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 --2- 172.21.15.27:0/962847358 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c158790 0x7fc88c0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 --2- 172.21.15.27:0/962847358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc88c003f20 0x7fc88c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 --2- 172.21.15.27:0/962847358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc88c004d00 0x7fc88c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0c99d0 con 0x7fc88c003f20 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c00ba60 con 0x7fc88c004d00 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.512+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc88c00bc20 con 0x7fc89c158790 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.513+0000 7fc89bfff640 1 --2- 172.21.15.27:0/962847358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc88c003f20 0x7fc88c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.513+0000 7fc8a0820640 1 --2- 172.21.15.27:0/962847358 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c158790 0x7fc88c0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.513+0000 7fc8a1021640 1 --2- 172.21.15.27:0/962847358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc88c004d00 0x7fc88c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.513+0000 7fc89bfff640 1 -- 172.21.15.27:0/962847358 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c158790 msgr2=0x7fc88c0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:09.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.513+0000 7fc89bfff640 1 --2- 172.21.15.27:0/962847358 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c158790 0x7fc88c0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.513+0000 7fc89bfff640 1 -- 172.21.15.27:0/962847358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc88c004d00 msgr2=0x7fc88c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:09.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.513+0000 7fc89bfff640 1 --2- 172.21.15.27:0/962847358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc88c004d00 0x7fc88c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.513+0000 7fc89bfff640 1 -- 172.21.15.27:0/962847358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c003520 con 0x7fc88c003f20 2023-12-19T18:24:09.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.513+0000 7fc8a1021640 1 --2- 172.21.15.27:0/962847358 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc88c004d00 0x7fc88c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:09.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.515+0000 7fc89bfff640 1 --2- 172.21.15.27:0/962847358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc88c003f20 0x7fc88c0047c0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fc88800c190 tx=0x7fc88c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5490e5a597676fae server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:09.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.515+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc888005ad0 con 0x7fc88c003f20 2023-12-19T18:24:09.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.515+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc888005c30 con 0x7fc88c003f20 2023-12-19T18:24:09.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.515+0000 7fc89b7fe640 1 -- 172.21.15.27:0/962847358 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88802cae0 con 0x7fc88c003f20 2023-12-19T18:24:09.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.515+0000 7fc8a1822640 1 -- 172.21.15.27:0/962847358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc88c003f20 msgr2=0x7fc88c0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:09.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.515+0000 7fc8a1822640 1 --2- 172.21.15.27:0/962847358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc88c003f20 0x7fc88c0047c0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7fc88800c190 tx=0x7fc88c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.515+0000 7fc8a1021640 1 -- 172.21.15.27:0/962847358 reap_dead start 2023-12-19T18:24:09.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.516+0000 7fc8a1822640 1 -- 172.21.15.27:0/962847358 shutdown_connections 2023-12-19T18:24:09.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.516+0000 7fc8a1822640 1 -- 172.21.15.27:0/962847358 >> 172.21.15.27:0/962847358 conn(0x7fc89c0b05b0 msgr2=0x7fc89c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:09.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.516+0000 7fc8a1822640 1 -- 172.21.15.27:0/962847358 shutdown_connections 2023-12-19T18:24:09.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.516+0000 7fc8a1822640 1 -- 172.21.15.27:0/962847358 wait complete. 2023-12-19T18:24:09.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.517+0000 7fc8a1822640 1 Processor -- start 2023-12-19T18:24:09.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.518+0000 7fc8a1822640 1 -- start start 2023-12-19T18:24:09.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1822640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc88c003f20 0x7fc89c1547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1822640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c0c3820 0x7fc89c0c3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1822640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c156210 0x7fc89c156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1822640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c0c41e0 con 0x7fc89c156210 2023-12-19T18:24:09.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1822640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c156fb0 con 0x7fc89c0c3820 2023-12-19T18:24:09.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1822640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1595d0 con 0x7fc88c003f20 2023-12-19T18:24:09.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1021640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c156210 0x7fc89c156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1021640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c156210 0x7fc89c156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38236/0 (socket says 172.21.15.27:38236) 2023-12-19T18:24:09.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1021640 1 -- 172.21.15.27:0/1657035546 learned_addr learned my addr 172.21.15.27:0/1657035546 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:09.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a0820640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc88c003f20 0x7fc89c1547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc89bfff640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c0c3820 0x7fc89c0c3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1021640 1 -- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc88c003f20 msgr2=0x7fc89c1547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:09.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1021640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc88c003f20 0x7fc89c1547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1021640 1 -- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c0c3820 msgr2=0x7fc89c0c3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:09.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1021640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c0c3820 0x7fc89c0c3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:09.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc8a1021640 1 -- 172.21.15.27:0/1657035546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c158ca0 con 0x7fc89c156210 2023-12-19T18:24:09.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.519+0000 7fc89bfff640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c0c3820 0x7fc89c0c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:09.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.520+0000 7fc8a1021640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c156210 0x7fc89c156690 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0b0540 tx=0x7fc89400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:09.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.520+0000 7fc8997fa640 1 -- 172.21.15.27:0/1657035546 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894006d50 con 0x7fc89c156210 2023-12-19T18:24:09.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.520+0000 7fc8997fa640 1 -- 172.21.15.27:0/1657035546 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc894003970 con 0x7fc89c156210 2023-12-19T18:24:09.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.520+0000 7fc8997fa640 1 -- 172.21.15.27:0/1657035546 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894003d60 con 0x7fc89c156210 2023-12-19T18:24:09.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.520+0000 7fc8a1822640 1 -- 172.21.15.27:0/1657035546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c158f80 con 0x7fc89c156210 2023-12-19T18:24:09.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.520+0000 7fc8a1822640 1 -- 172.21.15.27:0/1657035546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c154d00 con 0x7fc89c156210 2023-12-19T18:24:09.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.522+0000 7fc8a1822640 1 -- 172.21.15.27:0/1657035546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc89c04e840 con 0x7fc89c156210 2023-12-19T18:24:09.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.523+0000 7fc8997fa640 1 -- 172.21.15.27:0/1657035546 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc894005070 con 0x7fc89c156210 2023-12-19T18:24:09.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.524+0000 7fc8997fa640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc88006a430 0x7fc88006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:09.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.524+0000 7fc8997fa640 1 -- 172.21.15.27:0/1657035546 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc894088740 con 0x7fc89c156210 2023-12-19T18:24:09.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.524+0000 7fc8a0820640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc88006a430 0x7fc88006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:09.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.525+0000 7fc8a0820640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc88006a430 0x7fc88006c8f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc890009a10 tx=0x7fc890008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:09.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.528+0000 7fc8997fa640 1 -- 172.21.15.27:0/1657035546 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc894053dc0 con 0x7fc89c156210 2023-12-19T18:24:09.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:09.760+0000 7fc8a1822640 1 -- 172.21.15.27:0/1657035546 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc89c0c4e20 con 0x7fc89c156210 2023-12-19T18:24:10.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.377+0000 7fc8997fa640 1 -- 172.21.15.27:0/1657035546 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v68) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc8940573e0 con 0x7fc89c156210 2023-12-19T18:24:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.381+0000 7fc8a1822640 1 -- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc88006a430 msgr2=0x7fc88006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.381+0000 7fc8a1822640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc88006a430 0x7fc88006c8f0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fc890009a10 tx=0x7fc890008040 comp rx=0 tx=0).stop 2023-12-19T18:24:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.381+0000 7fc8a1822640 1 -- 172.21.15.27:0/1657035546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c156210 msgr2=0x7fc89c156690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.381+0000 7fc8a1822640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c156210 0x7fc89c156690 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0b0540 tx=0x7fc89400dc00 comp rx=0 tx=0).stop 2023-12-19T18:24:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.381+0000 7fc8a1822640 1 -- 172.21.15.27:0/1657035546 shutdown_connections 2023-12-19T18:24:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.381+0000 7fc8a1822640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc88006a430 0x7fc88006c8f0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.381+0000 7fc8a1822640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c156210 0x7fc89c156690 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.381+0000 7fc8a1822640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c0c3820 0x7fc89c0c3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.381+0000 7fc8a1822640 1 --2- 172.21.15.27:0/1657035546 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc88c003f20 0x7fc89c1547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.381+0000 7fc8a1822640 1 -- 172.21.15.27:0/1657035546 >> 172.21.15.27:0/1657035546 conn(0x7fc89c0b05b0 msgr2=0x7fc89c0ca1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:10.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.381+0000 7fc8a1822640 1 -- 172.21.15.27:0/1657035546 shutdown_connections 2023-12-19T18:24:10.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.381+0000 7fc8a1822640 1 -- 172.21.15.27:0/1657035546 wait complete. 2023-12-19T18:24:10.402 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:24:10.402 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:24:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.639+0000 7f4fdb3fd640 1 Processor -- start 2023-12-19T18:24:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.640+0000 7f4fdb3fd640 1 -- start start 2023-12-19T18:24:10.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.641+0000 7f4fdb3fd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4fd40c5040 0x7f4fd40ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:10.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.641+0000 7f4fdb3fd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd415d780 con 0x7f4fd40adc50 2023-12-19T18:24:10.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.642+0000 7f4fdb3fd640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fd40cc200 con 0x7f4fd40cbe20 2023-12-19T18:24:10.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.642+0000 7f4fdb3fd640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4fd415d9d0 con 0x7f4fd40c5040 2023-12-19T18:24:10.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.642+0000 7f4fdabfc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4fd40adc50 0x7f4fd4159ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:49304/0 (socket says 172.21.15.27:49304) 2023-12-19T18:24:10.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.642+0000 7f4fdabfc640 1 -- 172.21.15.27:0/3073628367 learned_addr learned my addr 172.21.15.27:0/3073628367 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:10.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.642+0000 7f4fd9bfa640 1 --2- 172.21.15.27:0/3073628367 >> v2:172.21.15.192:3300/0 conn(0x7f4fd40c5040 0x7f4fd40ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:10.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.642+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1026377770 0 0) 0x7f4fd415d780 con 0x7f4fd40adc50 2023-12-19T18:24:10.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fbc003520 con 0x7f4fd40adc50 2023-12-19T18:24:10.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3469251503 0 0) 0x7f4fbc003520 con 0x7f4fd40adc50 2023-12-19T18:24:10.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 >> v2:172.21.15.192:3300/0 conn(0x7f4fd40c5040 msgr2=0x7f4fd40ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:10.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 --2- 172.21.15.27:0/3073628367 >> v2:172.21.15.192:3300/0 conn(0x7f4fd40c5040 0x7f4fd40ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 >> v1:172.21.15.192:6789/0 conn(0x7f4fd40cbe20 legacy=0x7f4fd40c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd415dcb0 con 0x7f4fd40adc50 2023-12-19T18:24:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f4fcc001670 con 0x7f4fd40adc50 2023-12-19T18:24:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 >> v1:172.21.15.27:6789/0 conn(0x7f4fd40adc50 legacy=0x7f4fd4159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 --2- 172.21.15.27:0/3073628367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fbc003970 0x7f4fbc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 --2- 172.21.15.27:0/3073628367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fbc0047c0 0x7f4fbc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 --2- 172.21.15.27:0/3073628367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fbc0052a0 0x7f4fbc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd415d9d0 con 0x7f4fbc0047c0 2023-12-19T18:24:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc00c170 con 0x7f4fbc003970 2023-12-19T18:24:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fbc00c330 con 0x7f4fbc0052a0 2023-12-19T18:24:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.643+0000 7f4fd9bfa640 1 --2- 172.21.15.27:0/3073628367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fbc0047c0 0x7f4fbc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.644+0000 7f4fda3fb640 1 --2- 172.21.15.27:0/3073628367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fbc003970 0x7f4fbc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:10.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.644+0000 7f4fdabfc640 1 --2- 172.21.15.27:0/3073628367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fbc0052a0 0x7f4fbc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.644+0000 7f4fd9bfa640 1 -- 172.21.15.27:0/3073628367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fbc0052a0 msgr2=0x7f4fbc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.644+0000 7f4fd9bfa640 1 --2- 172.21.15.27:0/3073628367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fbc0052a0 0x7f4fbc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.644+0000 7f4fd9bfa640 1 -- 172.21.15.27:0/3073628367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fbc003970 msgr2=0x7f4fbc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.644+0000 7f4fd9bfa640 1 --2- 172.21.15.27:0/3073628367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fbc003970 0x7f4fbc003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.644+0000 7f4fd9bfa640 1 -- 172.21.15.27:0/3073628367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fbc003520 con 0x7f4fbc0047c0 2023-12-19T18:24:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.644+0000 7f4fdabfc640 1 --2- 172.21.15.27:0/3073628367 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fbc0052a0 0x7f4fbc00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.644+0000 7f4fda3fb640 1 --2- 172.21.15.27:0/3073628367 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fbc003970 0x7f4fbc003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:24:10.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.644+0000 7f4fdabfc640 1 -- 172.21.15.27:0/3073628367 reap_dead start 2023-12-19T18:24:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.645+0000 7f4fd9bfa640 1 --2- 172.21.15.27:0/3073628367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fbc0047c0 0x7f4fbc004c40 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009a90 tx=0x7f4fbc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3ae8e226333d509 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.645+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc802c660 con 0x7f4fbc0047c0 2023-12-19T18:24:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.646+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc802c7c0 con 0x7f4fbc0047c0 2023-12-19T18:24:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.646+0000 7f4fd93f9640 1 -- 172.21.15.27:0/3073628367 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc802cb50 con 0x7f4fbc0047c0 2023-12-19T18:24:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.646+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/3073628367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fbc0047c0 msgr2=0x7f4fbc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.646+0000 7f4fdb3fd640 1 --2- 172.21.15.27:0/3073628367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fbc0047c0 0x7f4fbc004c40 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8009a90 tx=0x7f4fbc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:24:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.646+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/3073628367 shutdown_connections 2023-12-19T18:24:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.646+0000 7f4fdb3fd640 1 --2- 172.21.15.27:0/3073628367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fbc0047c0 0x7f4fbc004c40 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:10.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.646+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/3073628367 >> 172.21.15.27:0/3073628367 conn(0x7f4fd40b05b0 msgr2=0x7f4fd40b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:10.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.646+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/3073628367 shutdown_connections 2023-12-19T18:24:10.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.646+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/3073628367 wait complete. 2023-12-19T18:24:10.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.648+0000 7f4fdb3fd640 1 Processor -- start 2023-12-19T18:24:10.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.648+0000 7f4fdb3fd640 1 -- start start 2023-12-19T18:24:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.649+0000 7f4fdb3fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fbc0047c0 0x7f4fd41f1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.649+0000 7f4fdb3fd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fd4159e50 0x7f4fd415a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.649+0000 7f4fdb3fd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fd41f59a0 0x7f4fd40c3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.649+0000 7f4fdb3fd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd415b050 con 0x7f4fbc0047c0 2023-12-19T18:24:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.649+0000 7f4fdb3fd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd41f2280 con 0x7f4fd41f59a0 2023-12-19T18:24:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.649+0000 7f4fdb3fd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fd415c730 con 0x7f4fd4159e50 2023-12-19T18:24:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.650+0000 7f4fda3fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fbc0047c0 0x7f4fd41f1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.650+0000 7f4fda3fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fbc0047c0 0x7f4fd41f1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38254/0 (socket says 172.21.15.27:38254) 2023-12-19T18:24:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.650+0000 7f4fda3fb640 1 -- 172.21.15.27:0/1047037809 learned_addr learned my addr 172.21.15.27:0/1047037809 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.650+0000 7f4fd9bfa640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fd4159e50 0x7f4fd415a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:10.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.650+0000 7f4fdabfc640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fd41f59a0 0x7f4fd40c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.650+0000 7f4fda3fb640 1 -- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fd4159e50 msgr2=0x7f4fd415a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.650+0000 7f4fda3fb640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fd4159e50 0x7f4fd415a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.650+0000 7f4fda3fb640 1 -- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fd41f59a0 msgr2=0x7f4fd40c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:10.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.650+0000 7f4fda3fb640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fd41f59a0 0x7f4fd40c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:10.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.650+0000 7f4fda3fb640 1 -- 172.21.15.27:0/1047037809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fd415ca10 con 0x7f4fbc0047c0 2023-12-19T18:24:10.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.650+0000 7f4fd9bfa640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fd4159e50 0x7f4fd415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:24:10.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.651+0000 7f4fda3fb640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fbc0047c0 0x7f4fd41f1d40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4000f80 tx=0x7f4fc400ef80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:10.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.651+0000 7f4fd2ffd640 1 -- 172.21.15.27:0/1047037809 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4003b80 con 0x7f4fbc0047c0 2023-12-19T18:24:10.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.651+0000 7f4fd2ffd640 1 -- 172.21.15.27:0/1047037809 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc4003ce0 con 0x7f4fbc0047c0 2023-12-19T18:24:10.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.651+0000 7f4fd2ffd640 1 -- 172.21.15.27:0/1047037809 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc4005310 con 0x7f4fbc0047c0 2023-12-19T18:24:10.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.651+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/1047037809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fd415ccf0 con 0x7f4fbc0047c0 2023-12-19T18:24:10.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.651+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/1047037809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fd415c440 con 0x7f4fbc0047c0 2023-12-19T18:24:10.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.652+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/1047037809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fd404e840 con 0x7f4fbc0047c0 2023-12-19T18:24:10.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.653+0000 7f4fd2ffd640 1 -- 172.21.15.27:0/1047037809 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4fc4005470 con 0x7f4fbc0047c0 2023-12-19T18:24:10.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.654+0000 7f4fd2ffd640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4fc006a460 0x7f4fc006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:10.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.654+0000 7f4fd2ffd640 1 -- 172.21.15.27:0/1047037809 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4fc4088a30 con 0x7f4fbc0047c0 2023-12-19T18:24:10.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.655+0000 7f4fd9bfa640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4fc006a460 0x7f4fc006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:10.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.656+0000 7f4fd9bfa640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4fc006a460 0x7f4fc006c920 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8036000 tx=0x7f4fc8030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.659+0000 7f4fd2ffd640 1 -- 172.21.15.27:0/1047037809 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4fc4054020 con 0x7f4fbc0047c0 2023-12-19T18:24:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:10.891+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/1047037809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4fd40ccc40 con 0x7f4fbc0047c0 2023-12-19T18:24:11.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:11.869+0000 7f4fd2ffd640 1 -- 172.21.15.27:0/1047037809 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v69) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4fc4057640 con 0x7f4fbc0047c0 2023-12-19T18:24:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:11.874+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4fc006a460 msgr2=0x7f4fc006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:11.874+0000 7f4fdb3fd640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4fc006a460 0x7f4fc006c920 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8036000 tx=0x7f4fc8030000 comp rx=0 tx=0).stop 2023-12-19T18:24:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:11.874+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/1047037809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fbc0047c0 msgr2=0x7f4fd41f1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:11.874+0000 7f4fdb3fd640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fbc0047c0 0x7f4fd41f1d40 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f4fc4000f80 tx=0x7f4fc400ef80 comp rx=0 tx=0).stop 2023-12-19T18:24:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:11.874+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/1047037809 shutdown_connections 2023-12-19T18:24:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:11.874+0000 7f4fdb3fd640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4fc006a460 0x7f4fc006c920 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:11.874+0000 7f4fdb3fd640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4fd41f59a0 0x7f4fd40c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:11.874+0000 7f4fdb3fd640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4fd4159e50 0x7f4fd415a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:11.874+0000 7f4fdb3fd640 1 --2- 172.21.15.27:0/1047037809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fbc0047c0 0x7f4fd41f1d40 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:11.874+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/1047037809 >> 172.21.15.27:0/1047037809 conn(0x7f4fd40b05b0 msgr2=0x7f4fd4070c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:11.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:11.874+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/1047037809 shutdown_connections 2023-12-19T18:24:11.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:11.874+0000 7f4fdb3fd640 1 -- 172.21.15.27:0/1047037809 wait complete. 2023-12-19T18:24:11.894 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:24:11.894 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:24:12.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.123+0000 7f9eff59e640 1 Processor -- start 2023-12-19T18:24:12.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.123+0000 7f9eff59e640 1 -- start start 2023-12-19T18:24:12.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.124+0000 7f9eff59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9f001553a0 0x7f9f000c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:12.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.124+0000 7f9eff59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f000afeb0 0x7f9f0015e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:12.124 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.124+0000 7f9eff59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9f0015ec60 0x7f9f0015b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:12.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.124+0000 7f9eff59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f000cbcf0 con 0x7f9f001553a0 2023-12-19T18:24:12.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.125+0000 7f9eff59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f0015f0d0 con 0x7f9f0015ec60 2023-12-19T18:24:12.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.125+0000 7f9eff59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000b0330 con 0x7f9f000afeb0 2023-12-19T18:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.125+0000 7f9efe59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9f001553a0 0x7f9f000c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.125+0000 7f9efe59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9f001553a0 0x7f9f000c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60046/0 (socket says 172.21.15.27:60046) 2023-12-19T18:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.125+0000 7f9efe59c640 1 -- 172.21.15.27:0/913453921 learned_addr learned my addr 172.21.15.27:0/913453921 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.125+0000 7f9efdd9b640 1 --2- 172.21.15.27:0/913453921 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f000afeb0 0x7f9f0015e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.125+0000 7f9efed9d640 1 --2- 172.21.15.27:0/913453921 >> v2:172.21.15.192:3300/0 conn(0x7f9f0015ec60 0x7f9f0015b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.125+0000 7f9efe59c640 1 -- 172.21.15.27:0/913453921 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f000afeb0 msgr2=0x7f9f0015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.125+0000 7f9efe59c640 1 --2- 172.21.15.27:0/913453921 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f000afeb0 0x7f9f0015e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.125+0000 7f9efe59c640 1 -- 172.21.15.27:0/913453921 >> v2:172.21.15.192:3300/0 conn(0x7f9f0015ec60 msgr2=0x7f9f0015b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.125+0000 7f9efe59c640 1 --2- 172.21.15.27:0/913453921 >> v2:172.21.15.192:3300/0 conn(0x7f9f0015ec60 0x7f9f0015b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.125+0000 7f9efe59c640 1 -- 172.21.15.27:0/913453921 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f000cb350 con 0x7f9f001553a0 2023-12-19T18:24:12.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.125+0000 7f9efed9d640 1 --2- 172.21.15.27:0/913453921 >> v2:172.21.15.192:3300/0 conn(0x7f9f0015ec60 0x7f9f0015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:12.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.127+0000 7f9efe59c640 1 --2- 172.21.15.27:0/913453921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f001553a0 0x7f9f000c50a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f9f0006f2d0 tx=0x7f9ef002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6d59dea09c3a0c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:12.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.127+0000 7f9efd59a640 1 -- 172.21.15.27:0/913453921 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00317c0 con 0x7f9f001553a0 2023-12-19T18:24:12.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.127+0000 7f9efd59a640 1 -- 172.21.15.27:0/913453921 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef0031920 con 0x7f9f001553a0 2023-12-19T18:24:12.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.127+0000 7f9efd59a640 1 -- 172.21.15.27:0/913453921 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef0031cb0 con 0x7f9f001553a0 2023-12-19T18:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.127+0000 7f9eff59e640 1 -- 172.21.15.27:0/913453921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f001553a0 msgr2=0x7f9f000c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.127+0000 7f9eff59e640 1 --2- 172.21.15.27:0/913453921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f001553a0 0x7f9f000c50a0 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f9f0006f2d0 tx=0x7f9ef002b860 comp rx=0 tx=0).stop 2023-12-19T18:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.128+0000 7f9eff59e640 1 -- 172.21.15.27:0/913453921 shutdown_connections 2023-12-19T18:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.128+0000 7f9eff59e640 1 --2- 172.21.15.27:0/913453921 >> v2:172.21.15.192:3300/0 conn(0x7f9f0015ec60 0x7f9f0015b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.128+0000 7f9eff59e640 1 --2- 172.21.15.27:0/913453921 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f000afeb0 0x7f9f0015e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.128+0000 7f9eff59e640 1 --2- 172.21.15.27:0/913453921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f001553a0 0x7f9f000c50a0 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.128+0000 7f9eff59e640 1 -- 172.21.15.27:0/913453921 >> 172.21.15.27:0/913453921 conn(0x7f9f00069c70 msgr2=0x7f9f0006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:12.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.128+0000 7f9eff59e640 1 -- 172.21.15.27:0/913453921 shutdown_connections 2023-12-19T18:24:12.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.128+0000 7f9eff59e640 1 -- 172.21.15.27:0/913453921 wait complete. 2023-12-19T18:24:12.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.130+0000 7f9eff59e640 1 Processor -- start 2023-12-19T18:24:12.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.130+0000 7f9eff59e640 1 -- start start 2023-12-19T18:24:12.131 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.131+0000 7f9eff59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f000afeb0 0x7f9f00150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.131+0000 7f9eff59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f001553a0 0x7f9f00152280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.131+0000 7f9eff59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f0015ec60 0x7f9f001f3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.131+0000 7f9eff59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000cc5b0 con 0x7f9f000afeb0 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.131+0000 7f9eff59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000cc890 con 0x7f9f001553a0 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.131+0000 7f9eff59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f000ccb70 con 0x7f9f0015ec60 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.131+0000 7f9efe59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f000afeb0 0x7f9f00150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.131+0000 7f9efe59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f000afeb0 0x7f9f00150e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60048/0 (socket says 172.21.15.27:60048) 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.131+0000 7f9efe59c640 1 -- 172.21.15.27:0/1522666670 learned_addr learned my addr 172.21.15.27:0/1522666670 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.131+0000 7f9efdd9b640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f001553a0 0x7f9f00152280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.132+0000 7f9efed9d640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f0015ec60 0x7f9f001f3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.132+0000 7f9efe59c640 1 -- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f0015ec60 msgr2=0x7f9f001f3790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.132+0000 7f9efe59c640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f0015ec60 0x7f9f001f3790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.132+0000 7f9efe59c640 1 -- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f001553a0 msgr2=0x7f9f00152280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.132+0000 7f9efe59c640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f001553a0 0x7f9f00152280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.132+0000 7f9efe59c640 1 -- 172.21.15.27:0/1522666670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ef00094d0 con 0x7f9f000afeb0 2023-12-19T18:24:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.132+0000 7f9efdd9b640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f001553a0 0x7f9f00152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.132+0000 7f9efe59c640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f000afeb0 0x7f9f00150e70 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f9ef002b8a0 tx=0x7f9ef00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.132+0000 7f9eeeffd640 1 -- 172.21.15.27:0/1522666670 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00329a0 con 0x7f9f000afeb0 2023-12-19T18:24:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.132+0000 7f9eeeffd640 1 -- 172.21.15.27:0/1522666670 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ef0032b40 con 0x7f9f000afeb0 2023-12-19T18:24:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.132+0000 7f9eeeffd640 1 -- 172.21.15.27:0/1522666670 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ef00453f0 con 0x7f9f000afeb0 2023-12-19T18:24:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.132+0000 7f9eff59e640 1 -- 172.21.15.27:0/1522666670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f000cce50 con 0x7f9f000afeb0 2023-12-19T18:24:12.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.132+0000 7f9eff59e640 1 -- 172.21.15.27:0/1522666670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f00151450 con 0x7f9f000afeb0 2023-12-19T18:24:12.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.134+0000 7f9eff59e640 1 -- 172.21.15.27:0/1522666670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f000cd070 con 0x7f9f000afeb0 2023-12-19T18:24:12.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.135+0000 7f9eeeffd640 1 -- 172.21.15.27:0/1522666670 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9ef004e450 con 0x7f9f000afeb0 2023-12-19T18:24:12.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.136+0000 7f9eeeffd640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ee006a480 0x7f9ee006c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:12.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.136+0000 7f9eeeffd640 1 -- 172.21.15.27:0/1522666670 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9ef00aabd0 con 0x7f9f000afeb0 2023-12-19T18:24:12.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.136+0000 7f9efdd9b640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ee006a480 0x7f9ee006c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:12.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.138+0000 7f9efdd9b640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ee006a480 0x7f9ee006c940 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9f000acc30 tx=0x7f9ef4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:12.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.140+0000 7f9eeeffd640 1 -- 172.21.15.27:0/1522666670 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9ef00762a0 con 0x7f9f000afeb0 2023-12-19T18:24:12.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.370+0000 7f9eff59e640 1 -- 172.21.15.27:0/1522666670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9f0004e840 con 0x7f9f000afeb0 2023-12-19T18:24:12.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.371+0000 7f9eeeffd640 1 -- 172.21.15.27:0/1522666670 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f9ef00798c0 con 0x7f9f000afeb0 2023-12-19T18:24:12.371 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:24:12.371 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":69,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3291476389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3291476389},{"type":"v1","addr":"172.21.15.192:6836","nonce":3291476389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3526330256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3526330256},{"type":"v1","addr":"172.21.15.192:6837","nonce":3526330256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/899234003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899234003},{"type":"v1","addr":"172.21.15.27:6835","nonce":899234003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:24:08.343302+0000","modified":"2023-12-19T18:24:11.863775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8247":{"gid":8247,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6833/2519815591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2519815591},{"type":"v1","addr":"172.21.15.27:6833","nonce":2519815591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-19T18:24:12.371 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 69 2023-12-19T18:24:12.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.373+0000 7f9eff59e640 1 -- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ee006a480 msgr2=0x7f9ee006c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:12.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.373+0000 7f9eff59e640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ee006a480 0x7f9ee006c940 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f9f000acc30 tx=0x7f9ef4008040 comp rx=0 tx=0).stop 2023-12-19T18:24:12.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.373+0000 7f9eff59e640 1 -- 172.21.15.27:0/1522666670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f000afeb0 msgr2=0x7f9f00150e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:12.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.373+0000 7f9eff59e640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f000afeb0 0x7f9f00150e70 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f9ef002b8a0 tx=0x7f9ef00046c0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.373+0000 7f9eff59e640 1 -- 172.21.15.27:0/1522666670 shutdown_connections 2023-12-19T18:24:12.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.373+0000 7f9eff59e640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9ee006a480 0x7f9ee006c940 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.373+0000 7f9eff59e640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f0015ec60 0x7f9f001f3790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.373+0000 7f9eff59e640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f001553a0 0x7f9f00152280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.373+0000 7f9eff59e640 1 --2- 172.21.15.27:0/1522666670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f000afeb0 0x7f9f00150e70 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.373+0000 7f9eff59e640 1 -- 172.21.15.27:0/1522666670 >> 172.21.15.27:0/1522666670 conn(0x7f9f00069c70 msgr2=0x7f9f000c4e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:12.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.374+0000 7f9eff59e640 1 -- 172.21.15.27:0/1522666670 shutdown_connections 2023-12-19T18:24:12.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.374+0000 7f9eff59e640 1 -- 172.21.15.27:0/1522666670 wait complete. 2023-12-19T18:24:12.393 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:24:12.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.621+0000 7fb5f129e640 1 Processor -- start 2023-12-19T18:24:12.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.621+0000 7fb5f129e640 1 -- start start 2023-12-19T18:24:12.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.623+0000 7fb5f129e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb5ec0c8f90 0x7fb5ec160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:12.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.623+0000 7fb5f129e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ec160e40 0x7fb5ec0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:12.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.623+0000 7fb5f129e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5ec1612d0 con 0x7fb5ec0cb300 2023-12-19T18:24:12.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.623+0000 7fb5f129e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5ec0c4480 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.623+0000 7fb5f129e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec1468c0 con 0x7fb5ec160e40 2023-12-19T18:24:12.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.623+0000 7fb5e3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb5ec0c8f90 0x7fb5ec160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:12.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.623+0000 7fb5f0a9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ec160e40 0x7fb5ec0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:12.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.623+0000 7fb5e3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb5ec0c8f90 0x7fb5ec160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:36740/0 (socket says 172.21.15.27:36740) 2023-12-19T18:24:12.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.623+0000 7fb5ebfff640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fb5ec0cb300 0x7fb5ec0cb6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:52206/0 (socket says 172.21.15.27:52206) 2023-12-19T18:24:12.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.623+0000 7fb5e3fff640 1 -- 172.21.15.27:0/1422701200 learned_addr learned my addr 172.21.15.27:0/1422701200 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:12.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.624+0000 7fb5e3fff640 1 -- 172.21.15.27:0/1422701200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ec160e40 msgr2=0x7fb5ec0cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:12.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.624+0000 7fb5e3fff640 1 --2- 172.21.15.27:0/1422701200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ec160e40 0x7fb5ec0cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.624+0000 7fb5e3fff640 1 -- 172.21.15.27:0/1422701200 >> v1:172.21.15.192:6789/0 conn(0x7fb5ec0cb300 legacy=0x7fb5ec0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:12.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.624+0000 7fb5e3fff640 1 -- 172.21.15.27:0/1422701200 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec145d60 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.624+0000 7fb5eb7fe640 1 -- 172.21.15.27:0/1422701200 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2027253572 0 0) 0x7fb5dc001110 con 0x7fb5ec0cb300 2023-12-19T18:24:12.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.626+0000 7fb5e3fff640 1 --2- 172.21.15.27:0/1422701200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ec0c8f90 0x7fb5ec160900 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40098a0 tx=0x7fb5d402b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1a5fa99f9374034f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:12.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.626+0000 7fb5eb7fe640 1 -- 172.21.15.27:0/1422701200 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4030840 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.626+0000 7fb5eb7fe640 1 -- 172.21.15.27:0/1422701200 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d40309a0 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.626+0000 7fb5eb7fe640 1 -- 172.21.15.27:0/1422701200 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4030d30 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.627+0000 7fb5f129e640 1 -- 172.21.15.27:0/1422701200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ec0c8f90 msgr2=0x7fb5ec160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.627+0000 7fb5f129e640 1 --2- 172.21.15.27:0/1422701200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ec0c8f90 0x7fb5ec160900 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fb5d40098a0 tx=0x7fb5d402b980 comp rx=0 tx=0).stop 2023-12-19T18:24:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.627+0000 7fb5f129e640 1 -- 172.21.15.27:0/1422701200 shutdown_connections 2023-12-19T18:24:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.627+0000 7fb5f129e640 1 --2- 172.21.15.27:0/1422701200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ec160e40 0x7fb5ec0cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.627+0000 7fb5f129e640 1 --2- 172.21.15.27:0/1422701200 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ec0c8f90 0x7fb5ec160900 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.627+0000 7fb5f129e640 1 -- 172.21.15.27:0/1422701200 >> 172.21.15.27:0/1422701200 conn(0x7fb5ec069c70 msgr2=0x7fb5ec06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:12.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.627+0000 7fb5f129e640 1 -- 172.21.15.27:0/1422701200 shutdown_connections 2023-12-19T18:24:12.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.627+0000 7fb5f129e640 1 -- 172.21.15.27:0/1422701200 wait complete. 2023-12-19T18:24:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.629+0000 7fb5f129e640 1 Processor -- start 2023-12-19T18:24:12.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.629+0000 7fb5f129e640 1 -- start start 2023-12-19T18:24:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.630+0000 7fb5f129e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5ec0c8f90 0x7fb5ec0c70d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.630+0000 7fb5f129e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ec0cb300 0x7fb5ec15fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.630+0000 7fb5f129e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ec160e40 0x7fb5ec142350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.630+0000 7fb5f129e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec15d350 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.630+0000 7fb5f129e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec0c8580 con 0x7fb5ec0cb300 2023-12-19T18:24:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.630+0000 7fb5f129e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec0c8860 con 0x7fb5ec160e40 2023-12-19T18:24:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.630+0000 7fb5ebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5ec0c8f90 0x7fb5ec0c70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.630+0000 7fb5ebfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5ec0c8f90 0x7fb5ec0c70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60064/0 (socket says 172.21.15.27:60064) 2023-12-19T18:24:12.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.630+0000 7fb5ebfff640 1 -- 172.21.15.27:0/122851503 learned_addr learned my addr 172.21.15.27:0/122851503 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:12.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.630+0000 7fb5e3fff640 1 --2- 172.21.15.27:0/122851503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ec0cb300 0x7fb5ec15fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.631+0000 7fb5ebfff640 1 -- 172.21.15.27:0/122851503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ec160e40 msgr2=0x7fb5ec142350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.631+0000 7fb5ebfff640 1 --2- 172.21.15.27:0/122851503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ec160e40 0x7fb5ec142350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.631+0000 7fb5ebfff640 1 -- 172.21.15.27:0/122851503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ec0cb300 msgr2=0x7fb5ec15fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.631+0000 7fb5ebfff640 1 --2- 172.21.15.27:0/122851503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ec0cb300 0x7fb5ec15fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.631+0000 7fb5ebfff640 1 -- 172.21.15.27:0/122851503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d40094d0 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.631+0000 7fb5ebfff640 1 --2- 172.21.15.27:0/122851503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5ec0c8f90 0x7fb5ec0c70d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00c780 tx=0x7fb5dc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.631+0000 7fb5e97fa640 1 -- 172.21.15.27:0/122851503 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0072d0 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.631+0000 7fb5e97fa640 1 -- 172.21.15.27:0/122851503 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc007450 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.631+0000 7fb5f129e640 1 -- 172.21.15.27:0/122851503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5ec0c8b40 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.631+0000 7fb5e97fa640 1 -- 172.21.15.27:0/122851503 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0047e0 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.631+0000 7fb5f129e640 1 -- 172.21.15.27:0/122851503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5ec160410 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.632+0000 7fb5f129e640 1 -- 172.21.15.27:0/122851503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5ec160600 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.633+0000 7fb5e97fa640 1 -- 172.21.15.27:0/122851503 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb5dc004960 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.634+0000 7fb5e97fa640 1 --2- 172.21.15.27:0/122851503 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb5cc06a430 0x7fb5cc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:12.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.634+0000 7fb5e97fa640 1 -- 172.21.15.27:0/122851503 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5dc0879a0 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.635+0000 7fb5e3fff640 1 --2- 172.21.15.27:0/122851503 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb5cc06a430 0x7fb5cc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:12.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.636+0000 7fb5e3fff640 1 --2- 172.21.15.27:0/122851503 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb5cc06a430 0x7fb5cc06c8f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4036000 tx=0x7fb5d403b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:12.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.638+0000 7fb5e97fa640 1 -- 172.21.15.27:0/122851503 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb5dc053020 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.798 INFO:tasks.ceph.mds.a.smithi027.stderr:2023-12-19T18:24:12.798+0000 7f36cbd41640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:24:12.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.833+0000 7fb5f129e640 1 -- 172.21.15.27:0/122851503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5ec1612d0 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.835+0000 7fb5e97fa640 1 -- 172.21.15.27:0/122851503 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) v1 ==== 74+0+16060 (secure 0 0 0) 0x7fb5ec1612d0 con 0x7fb5ec0c8f90 2023-12-19T18:24:12.835 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:24:12.835 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":170,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:24:09.346922+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:24:06.280682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T18:24:07.374789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:24:12.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.838+0000 7fb5f129e640 1 -- 172.21.15.27:0/122851503 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb5cc06a430 msgr2=0x7fb5cc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:12.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.838+0000 7fb5f129e640 1 --2- 172.21.15.27:0/122851503 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb5cc06a430 0x7fb5cc06c8f0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4036000 tx=0x7fb5d403b040 comp rx=0 tx=0).stop 2023-12-19T18:24:12.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.838+0000 7fb5f129e640 1 -- 172.21.15.27:0/122851503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5ec0c8f90 msgr2=0x7fb5ec0c70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:12.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.838+0000 7fb5f129e640 1 --2- 172.21.15.27:0/122851503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5ec0c8f90 0x7fb5ec0c70d0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc00c780 tx=0x7fb5dc00cc40 comp rx=0 tx=0).stop 2023-12-19T18:24:12.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.838+0000 7fb5f129e640 1 -- 172.21.15.27:0/122851503 shutdown_connections 2023-12-19T18:24:12.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.838+0000 7fb5f129e640 1 --2- 172.21.15.27:0/122851503 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb5cc06a430 0x7fb5cc06c8f0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.838+0000 7fb5f129e640 1 --2- 172.21.15.27:0/122851503 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb5ec160e40 0x7fb5ec142350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.838+0000 7fb5f129e640 1 --2- 172.21.15.27:0/122851503 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb5ec0cb300 0x7fb5ec15fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.838+0000 7fb5f129e640 1 --2- 172.21.15.27:0/122851503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5ec0c8f90 0x7fb5ec0c70d0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:12.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.838+0000 7fb5f129e640 1 -- 172.21.15.27:0/122851503 >> 172.21.15.27:0/122851503 conn(0x7fb5ec069c70 msgr2=0x7fb5ec0ccf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:12.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.838+0000 7fb5f129e640 1 -- 172.21.15.27:0/122851503 shutdown_connections 2023-12-19T18:24:12.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:12.838+0000 7fb5f129e640 1 -- 172.21.15.27:0/122851503 wait complete. 2023-12-19T18:24:12.858 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:24:12.859 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:24:13.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.084+0000 7fcdd5abe640 1 Processor -- start 2023-12-19T18:24:13.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.084+0000 7fcdd5abe640 1 -- start start 2023-12-19T18:24:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.085+0000 7fcdd5abe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdd015ee30 0x7fcdd00c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.085+0000 7fcdd5abe640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd00b0320 con 0x7fcdd00afeb0 2023-12-19T18:24:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.085+0000 7fcdd5abe640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdd00cb350 con 0x7fcdd01553a0 2023-12-19T18:24:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.085+0000 7fcdd5abe640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00cbcf0 con 0x7fcdd015ee30 2023-12-19T18:24:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.085+0000 7fcdd52bd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcdd00afeb0 0x7fcdd00c39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40420/0 (socket says 172.21.15.27:40420) 2023-12-19T18:24:13.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.085+0000 7fcdd52bd640 1 -- 172.21.15.27:0/3966560603 learned_addr learned my addr 172.21.15.27:0/3966560603 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.085+0000 7fcdcffff640 1 --2- 172.21.15.27:0/3966560603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdd015ee30 0x7fcdd00c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 485069977 0 0) 0x7fcdd00b0320 con 0x7fcdd00afeb0 2023-12-19T18:24:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdb8003520 con 0x7fcdd00afeb0 2023-12-19T18:24:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2076246760 0 0) 0x7fcdd00cb350 con 0x7fcdd01553a0 2023-12-19T18:24:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdd00b0320 con 0x7fcdd01553a0 2023-12-19T18:24:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4047294394 0 0) 0x7fcdb8003520 con 0x7fcdd00afeb0 2023-12-19T18:24:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdd015ee30 msgr2=0x7fcdd00c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 --2- 172.21.15.27:0/3966560603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdd015ee30 0x7fcdd00c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 >> v1:172.21.15.192:6789/0 conn(0x7fcdd01553a0 legacy=0x7fcdd015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd00cb630 con 0x7fcdd00afeb0 2023-12-19T18:24:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fcdc8001970 con 0x7fcdd00afeb0 2023-12-19T18:24:13.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 >> v1:172.21.15.27:6789/0 conn(0x7fcdd00afeb0 legacy=0x7fcdd00c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 --2- 172.21.15.27:0/3966560603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdb8003b30 0x7fcdb8003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 --2- 172.21.15.27:0/3966560603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb80047c0 0x7fcdb8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 --2- 172.21.15.27:0/3966560603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdd015ee30 0x7fcdb80051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00cbcf0 con 0x7fcdb8003b30 2023-12-19T18:24:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb800bb10 con 0x7fcdb80047c0 2023-12-19T18:24:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.086+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb800bca0 con 0x7fcdd015ee30 2023-12-19T18:24:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.087+0000 7fcdd4abc640 1 --2- 172.21.15.27:0/3966560603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdb8003b30 0x7fcdb8003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.087+0000 7fcdcffff640 1 --2- 172.21.15.27:0/3966560603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb80047c0 0x7fcdb8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.087+0000 7fcdd52bd640 1 --2- 172.21.15.27:0/3966560603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdd015ee30 0x7fcdb80051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.087+0000 7fcdd4abc640 1 -- 172.21.15.27:0/3966560603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdd015ee30 msgr2=0x7fcdb80051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.087+0000 7fcdd4abc640 1 --2- 172.21.15.27:0/3966560603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdd015ee30 0x7fcdb80051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.087+0000 7fcdd4abc640 1 -- 172.21.15.27:0/3966560603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb80047c0 msgr2=0x7fcdb8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.087+0000 7fcdd4abc640 1 --2- 172.21.15.27:0/3966560603 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdb80047c0 0x7fcdb8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.087+0000 7fcdd4abc640 1 -- 172.21.15.27:0/3966560603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb8003520 con 0x7fcdb8003b30 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.087+0000 7fcdd52bd640 1 --2- 172.21.15.27:0/3966560603 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdd015ee30 0x7fcdb80051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.089+0000 7fcdd4abc640 1 --2- 172.21.15.27:0/3966560603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdb8003b30 0x7fcdb8003f30 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4001900 tx=0x7fcdb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c8497c8983c1bc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.089+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc40052d0 con 0x7fcdb8003b30 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.089+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc4005430 con 0x7fcdb8003b30 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.089+0000 7fcdcf7fe640 1 -- 172.21.15.27:0/3966560603 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc4003810 con 0x7fcdb8003b30 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.089+0000 7fcdd5abe640 1 -- 172.21.15.27:0/3966560603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdb8003b30 msgr2=0x7fcdb8003f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.089+0000 7fcdd5abe640 1 --2- 172.21.15.27:0/3966560603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdb8003b30 0x7fcdb8003f30 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4001900 tx=0x7fcdb80057f0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.089+0000 7fcdd52bd640 1 -- 172.21.15.27:0/3966560603 reap_dead start 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.089+0000 7fcdd5abe640 1 -- 172.21.15.27:0/3966560603 shutdown_connections 2023-12-19T18:24:13.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.089+0000 7fcdd5abe640 1 -- 172.21.15.27:0/3966560603 >> 172.21.15.27:0/3966560603 conn(0x7fcdd0069c70 msgr2=0x7fcdd006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:13.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.090+0000 7fcdd5abe640 1 -- 172.21.15.27:0/3966560603 shutdown_connections 2023-12-19T18:24:13.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.090+0000 7fcdd5abe640 1 -- 172.21.15.27:0/3966560603 wait complete. 2023-12-19T18:24:13.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.091+0000 7fcdd5abe640 1 Processor -- start 2023-12-19T18:24:13.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd5abe640 1 -- start start 2023-12-19T18:24:13.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd5abe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdb8003b30 0x7fcdd00c8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd5abe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdd0147a30 0x7fcdd00c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd5abe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdd00c7490 0x7fcdd006fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd5abe640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00c7000 con 0x7fcdd00c7490 2023-12-19T18:24:13.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd5abe640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00c63f0 con 0x7fcdd0147a30 2023-12-19T18:24:13.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd5abe640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdd00c66d0 con 0x7fcdb8003b30 2023-12-19T18:24:13.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd52bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdd00c7490 0x7fcdd006fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd52bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdd00c7490 0x7fcdd006fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60080/0 (socket says 172.21.15.27:60080) 2023-12-19T18:24:13.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd52bd640 1 -- 172.21.15.27:0/2462526003 learned_addr learned my addr 172.21.15.27:0/2462526003 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdcffff640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdd0147a30 0x7fcdd00c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd52bd640 1 -- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdb8003b30 msgr2=0x7fcdd00c8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd52bd640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdb8003b30 0x7fcdd00c8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd52bd640 1 -- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdd0147a30 msgr2=0x7fcdd00c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd52bd640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdd0147a30 0x7fcdd00c5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdd52bd640 1 -- 172.21.15.27:0/2462526003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdd00c69b0 con 0x7fcdd00c7490 2023-12-19T18:24:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.093+0000 7fcdcffff640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdd0147a30 0x7fcdd00c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.094+0000 7fcdd52bd640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdd00c7490 0x7fcdd006fe60 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0071090 tx=0x7fcdc800b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.094+0000 7fcdcd7fa640 1 -- 172.21.15.27:0/2462526003 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc800b870 con 0x7fcdd00c7490 2023-12-19T18:24:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.094+0000 7fcdcd7fa640 1 -- 172.21.15.27:0/2462526003 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdc800b9d0 con 0x7fcdd00c7490 2023-12-19T18:24:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.094+0000 7fcdcd7fa640 1 -- 172.21.15.27:0/2462526003 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdc800bd80 con 0x7fcdd00c7490 2023-12-19T18:24:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.094+0000 7fcdd5abe640 1 -- 172.21.15.27:0/2462526003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdd00c6c90 con 0x7fcdd00c7490 2023-12-19T18:24:13.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.094+0000 7fcdd5abe640 1 -- 172.21.15.27:0/2462526003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdd00c7910 con 0x7fcdd00c7490 2023-12-19T18:24:13.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.095+0000 7fcdd5abe640 1 -- 172.21.15.27:0/2462526003 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdd00c5f50 con 0x7fcdd00c7490 2023-12-19T18:24:13.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.096+0000 7fcdcd7fa640 1 -- 172.21.15.27:0/2462526003 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcdc8007070 con 0x7fcdd00c7490 2023-12-19T18:24:13.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.097+0000 7fcdcd7fa640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdbc06a430 0x7fcdbc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.097+0000 7fcdcd7fa640 1 -- 172.21.15.27:0/2462526003 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdc8088c70 con 0x7fcdd00c7490 2023-12-19T18:24:13.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.098+0000 7fcdd4abc640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdbc06a430 0x7fcdbc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.099+0000 7fcdd4abc640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdbc06a430 0x7fcdbc06c8f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4034000 tx=0x7fcdc4009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:13.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.102+0000 7fcdcd7fa640 1 -- 172.21.15.27:0/2462526003 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcdc80541c0 con 0x7fcdd00c7490 2023-12-19T18:24:13.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.292+0000 7fcdd5abe640 1 -- 172.21.15.27:0/2462526003 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcdd00c5730 con 0x7fcdbc06a430 2023-12-19T18:24:13.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.315+0000 7fcdcd7fa640 1 -- 172.21.15.27:0/2462526003 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282896 (secure 0 0 0) 0x7fcdc40067d0 con 0x7fcdbc06a430 2023-12-19T18:24:13.318 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:24:13.319 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:24:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.320+0000 7fcdd5abe640 1 -- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdbc06a430 msgr2=0x7fcdbc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.320+0000 7fcdd5abe640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdbc06a430 0x7fcdbc06c8f0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fcdc4034000 tx=0x7fcdc4009030 comp rx=0 tx=0).stop 2023-12-19T18:24:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.320+0000 7fcdd5abe640 1 -- 172.21.15.27:0/2462526003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdd00c7490 msgr2=0x7fcdd006fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.320+0000 7fcdd5abe640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdd00c7490 0x7fcdd006fe60 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fcdd0071090 tx=0x7fcdc800b0d0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.321+0000 7fcdd5abe640 1 -- 172.21.15.27:0/2462526003 shutdown_connections 2023-12-19T18:24:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.321+0000 7fcdd5abe640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdbc06a430 0x7fcdbc06c8f0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.321+0000 7fcdd5abe640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdd00c7490 0x7fcdd006fe60 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.321+0000 7fcdd5abe640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdd0147a30 0x7fcdd00c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.321+0000 7fcdd5abe640 1 --2- 172.21.15.27:0/2462526003 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdb8003b30 0x7fcdd00c8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.321+0000 7fcdd5abe640 1 -- 172.21.15.27:0/2462526003 >> 172.21.15.27:0/2462526003 conn(0x7fcdd0069c70 msgr2=0x7fcdd006b9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.321+0000 7fcdd5abe640 1 -- 172.21.15.27:0/2462526003 shutdown_connections 2023-12-19T18:24:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.321+0000 7fcdd5abe640 1 -- 172.21.15.27:0/2462526003 wait complete. 2023-12-19T18:24:13.339 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2023-12-19T18:24:11.907372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553954,"statfs":{"total":1600000000,"available":1591249047,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.088912"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381356+0000","last_change":"2023-12-19T18:24:08.376161+0000","last_active":"2023-12-19T18:24:08.381356+0000","last_peered":"2023-12-19T18:24:08.381356+0000","last_clean":"2023-12-19T18:24:08.381356+0000","last_became_active":"2023-12-19T18:24:08.375608+0000","last_became_peered":"2023-12-19T18:24:08.375608+0000","last_unstale":"2023-12-19T18:24:08.381356+0000","last_undegraded":"2023-12-19T18:24:08.381356+0000","last_fullsized":"2023-12-19T18:24:08.381356+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:26:49.103272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382526+0000","last_change":"2023-12-19T18:24:08.381626+0000","last_active":"2023-12-19T18:24:08.382526+0000","last_peered":"2023-12-19T18:24:08.382526+0000","last_clean":"2023-12-19T18:24:08.382526+0000","last_became_active":"2023-12-19T18:24:08.376947+0000","last_became_peered":"2023-12-19T18:24:08.376947+0000","last_unstale":"2023-12-19T18:24:08.382526+0000","last_undegraded":"2023-12-19T18:24:08.382526+0000","last_fullsized":"2023-12-19T18:24:08.382526+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:52:26.286720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381156+0000","last_change":"2023-12-19T18:24:08.380971+0000","last_active":"2023-12-19T18:24:08.381156+0000","last_peered":"2023-12-19T18:24:08.381156+0000","last_clean":"2023-12-19T18:24:08.381156+0000","last_became_active":"2023-12-19T18:24:08.380611+0000","last_became_peered":"2023-12-19T18:24:08.380611+0000","last_unstale":"2023-12-19T18:24:08.381156+0000","last_undegraded":"2023-12-19T18:24:08.381156+0000","last_fullsized":"2023-12-19T18:24:08.381156+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:49:18.668947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.360440+0000","last_change":"2023-12-19T18:24:08.360188+0000","last_active":"2023-12-19T18:24:08.360440+0000","last_peered":"2023-12-19T18:24:08.360440+0000","last_clean":"2023-12-19T18:24:08.360440+0000","last_became_active":"2023-12-19T18:24:08.359539+0000","last_became_peered":"2023-12-19T18:24:08.359539+0000","last_unstale":"2023-12-19T18:24:08.360440+0000","last_undegraded":"2023-12-19T18:24:08.360440+0000","last_fullsized":"2023-12-19T18:24:08.360440+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:26:49.103272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.369871+0000","last_change":"2023-12-19T18:24:08.367100+0000","last_active":"2023-12-19T18:24:08.369871+0000","last_peered":"2023-12-19T18:24:08.369871+0000","last_clean":"2023-12-19T18:24:08.369871+0000","last_became_active":"2023-12-19T18:24:08.366848+0000","last_became_peered":"2023-12-19T18:24:08.366848+0000","last_unstale":"2023-12-19T18:24:08.369871+0000","last_undegraded":"2023-12-19T18:24:08.369871+0000","last_fullsized":"2023-12-19T18:24:08.369871+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:04.052385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.376628+0000","last_change":"2023-12-19T18:24:08.376363+0000","last_active":"2023-12-19T18:24:08.376628+0000","last_peered":"2023-12-19T18:24:08.376628+0000","last_clean":"2023-12-19T18:24:08.376628+0000","last_became_active":"2023-12-19T18:24:08.376012+0000","last_became_peered":"2023-12-19T18:24:08.376012+0000","last_unstale":"2023-12-19T18:24:08.376628+0000","last_undegraded":"2023-12-19T18:24:08.376628+0000","last_fullsized":"2023-12-19T18:24:08.376628+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:00.467931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.369898+0000","last_change":"2023-12-19T18:24:08.369499+0000","last_active":"2023-12-19T18:24:08.369898+0000","last_peered":"2023-12-19T18:24:08.369898+0000","last_clean":"2023-12-19T18:24:08.369898+0000","last_became_active":"2023-12-19T18:24:08.369266+0000","last_became_peered":"2023-12-19T18:24:08.369266+0000","last_unstale":"2023-12-19T18:24:08.369898+0000","last_undegraded":"2023-12-19T18:24:08.369898+0000","last_fullsized":"2023-12-19T18:24:08.369898+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:13.768073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.375567+0000","last_change":"2023-12-19T18:24:08.375450+0000","last_active":"2023-12-19T18:24:08.375567+0000","last_peered":"2023-12-19T18:24:08.375567+0000","last_clean":"2023-12-19T18:24:08.375567+0000","last_became_active":"2023-12-19T18:24:08.375154+0000","last_became_peered":"2023-12-19T18:24:08.375154+0000","last_unstale":"2023-12-19T18:24:08.375567+0000","last_undegraded":"2023-12-19T18:24:08.375567+0000","last_fullsized":"2023-12-19T18:24:08.375567+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:05:27.086846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381620+0000","last_change":"2023-12-19T18:24:08.381011+0000","last_active":"2023-12-19T18:24:08.381620+0000","last_peered":"2023-12-19T18:24:08.381620+0000","last_clean":"2023-12-19T18:24:08.381620+0000","last_became_active":"2023-12-19T18:24:08.380464+0000","last_became_peered":"2023-12-19T18:24:08.380464+0000","last_unstale":"2023-12-19T18:24:08.381620+0000","last_undegraded":"2023-12-19T18:24:08.381620+0000","last_fullsized":"2023-12-19T18:24:08.381620+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:33:27.734476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381390+0000","last_change":"2023-12-19T18:24:08.378192+0000","last_active":"2023-12-19T18:24:08.381390+0000","last_peered":"2023-12-19T18:24:08.381390+0000","last_clean":"2023-12-19T18:24:08.381390+0000","last_became_active":"2023-12-19T18:24:08.377755+0000","last_became_peered":"2023-12-19T18:24:08.377755+0000","last_unstale":"2023-12-19T18:24:08.381390+0000","last_undegraded":"2023-12-19T18:24:08.381390+0000","last_fullsized":"2023-12-19T18:24:08.381390+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:39:11.014721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380008+0000","last_change":"2023-12-19T18:24:08.379394+0000","last_active":"2023-12-19T18:24:08.380008+0000","last_peered":"2023-12-19T18:24:08.380008+0000","last_clean":"2023-12-19T18:24:08.380008+0000","last_became_active":"2023-12-19T18:24:08.378548+0000","last_became_peered":"2023-12-19T18:24:08.378548+0000","last_unstale":"2023-12-19T18:24:08.380008+0000","last_undegraded":"2023-12-19T18:24:08.380008+0000","last_fullsized":"2023-12-19T18:24:08.380008+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:25:03.694013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.357805+0000","last_change":"2023-12-19T18:24:08.357388+0000","last_active":"2023-12-19T18:24:08.357805+0000","last_peered":"2023-12-19T18:24:08.357805+0000","last_clean":"2023-12-19T18:24:08.357805+0000","last_became_active":"2023-12-19T18:24:08.356987+0000","last_became_peered":"2023-12-19T18:24:08.356987+0000","last_unstale":"2023-12-19T18:24:08.357805+0000","last_undegraded":"2023-12-19T18:24:08.357805+0000","last_fullsized":"2023-12-19T18:24:08.357805+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:39.310759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.360135+0000","last_change":"2023-12-19T18:24:08.359944+0000","last_active":"2023-12-19T18:24:08.360135+0000","last_peered":"2023-12-19T18:24:08.360135+0000","last_clean":"2023-12-19T18:24:08.360135+0000","last_became_active":"2023-12-19T18:24:08.358066+0000","last_became_peered":"2023-12-19T18:24:08.358066+0000","last_unstale":"2023-12-19T18:24:08.360135+0000","last_undegraded":"2023-12-19T18:24:08.360135+0000","last_fullsized":"2023-12-19T18:24:08.360135+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:03.637381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.383899+0000","last_change":"2023-12-19T18:24:08.383826+0000","last_active":"2023-12-19T18:24:08.383899+0000","last_peered":"2023-12-19T18:24:08.383899+0000","last_clean":"2023-12-19T18:24:08.383899+0000","last_became_active":"2023-12-19T18:24:08.383582+0000","last_became_peered":"2023-12-19T18:24:08.383582+0000","last_unstale":"2023-12-19T18:24:08.383899+0000","last_undegraded":"2023-12-19T18:24:08.383899+0000","last_fullsized":"2023-12-19T18:24:08.383899+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:21:25.284071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381625+0000","last_change":"2023-12-19T18:24:08.379784+0000","last_active":"2023-12-19T18:24:08.381625+0000","last_peered":"2023-12-19T18:24:08.381625+0000","last_clean":"2023-12-19T18:24:08.381625+0000","last_became_active":"2023-12-19T18:24:08.379409+0000","last_became_peered":"2023-12-19T18:24:08.379409+0000","last_unstale":"2023-12-19T18:24:08.381625+0000","last_undegraded":"2023-12-19T18:24:08.381625+0000","last_fullsized":"2023-12-19T18:24:08.381625+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:16.359936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380102+0000","last_change":"2023-12-19T18:24:08.378521+0000","last_active":"2023-12-19T18:24:08.380102+0000","last_peered":"2023-12-19T18:24:08.380102+0000","last_clean":"2023-12-19T18:24:08.380102+0000","last_became_active":"2023-12-19T18:24:08.376186+0000","last_became_peered":"2023-12-19T18:24:08.376186+0000","last_unstale":"2023-12-19T18:24:08.380102+0000","last_undegraded":"2023-12-19T18:24:08.380102+0000","last_fullsized":"2023-12-19T18:24:08.380102+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:25:03.694013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.362467+0000","last_change":"2023-12-19T18:24:08.362264+0000","last_active":"2023-12-19T18:24:08.362467+0000","last_peered":"2023-12-19T18:24:08.362467+0000","last_clean":"2023-12-19T18:24:08.362467+0000","last_became_active":"2023-12-19T18:24:08.362010+0000","last_became_peered":"2023-12-19T18:24:08.362010+0000","last_unstale":"2023-12-19T18:24:08.362467+0000","last_undegraded":"2023-12-19T18:24:08.362467+0000","last_fullsized":"2023-12-19T18:24:08.362467+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:56.632742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381510+0000","last_change":"2023-12-19T18:24:08.379242+0000","last_active":"2023-12-19T18:24:08.381510+0000","last_peered":"2023-12-19T18:24:08.381510+0000","last_clean":"2023-12-19T18:24:08.381510+0000","last_became_active":"2023-12-19T18:24:08.378871+0000","last_became_peered":"2023-12-19T18:24:08.378871+0000","last_unstale":"2023-12-19T18:24:08.381510+0000","last_undegraded":"2023-12-19T18:24:08.381510+0000","last_fullsized":"2023-12-19T18:24:08.381510+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:49.502235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.372875+0000","last_change":"2023-12-19T18:24:08.372223+0000","last_active":"2023-12-19T18:24:08.372875+0000","last_peered":"2023-12-19T18:24:08.372875+0000","last_clean":"2023-12-19T18:24:08.372875+0000","last_became_active":"2023-12-19T18:24:08.371748+0000","last_became_peered":"2023-12-19T18:24:08.371748+0000","last_unstale":"2023-12-19T18:24:08.372875+0000","last_undegraded":"2023-12-19T18:24:08.372875+0000","last_fullsized":"2023-12-19T18:24:08.372875+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:40:03.158824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.384544+0000","last_change":"2023-12-19T18:24:08.384331+0000","last_active":"2023-12-19T18:24:08.384544+0000","last_peered":"2023-12-19T18:24:08.384544+0000","last_clean":"2023-12-19T18:24:08.384544+0000","last_became_active":"2023-12-19T18:24:08.383917+0000","last_became_peered":"2023-12-19T18:24:08.383917+0000","last_unstale":"2023-12-19T18:24:08.384544+0000","last_undegraded":"2023-12-19T18:24:08.384544+0000","last_fullsized":"2023-12-19T18:24:08.384544+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:40:41.401902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.366916+0000","last_change":"2023-12-19T18:24:08.366507+0000","last_active":"2023-12-19T18:24:08.366916+0000","last_peered":"2023-12-19T18:24:08.366916+0000","last_clean":"2023-12-19T18:24:08.366916+0000","last_became_active":"2023-12-19T18:24:08.365865+0000","last_became_peered":"2023-12-19T18:24:08.365865+0000","last_unstale":"2023-12-19T18:24:08.366916+0000","last_undegraded":"2023-12-19T18:24:08.366916+0000","last_fullsized":"2023-12-19T18:24:08.366916+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:40:42.739778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382573+0000","last_change":"2023-12-19T18:24:08.381894+0000","last_active":"2023-12-19T18:24:08.382573+0000","last_peered":"2023-12-19T18:24:08.382573+0000","last_clean":"2023-12-19T18:24:08.382573+0000","last_became_active":"2023-12-19T18:24:08.377864+0000","last_became_peered":"2023-12-19T18:24:08.377864+0000","last_unstale":"2023-12-19T18:24:08.382573+0000","last_undegraded":"2023-12-19T18:24:08.382573+0000","last_fullsized":"2023-12-19T18:24:08.382573+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:44.600564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.365796+0000","last_change":"2023-12-19T18:24:08.365592+0000","last_active":"2023-12-19T18:24:08.365796+0000","last_peered":"2023-12-19T18:24:08.365796+0000","last_clean":"2023-12-19T18:24:08.365796+0000","last_became_active":"2023-12-19T18:24:08.364907+0000","last_became_peered":"2023-12-19T18:24:08.364907+0000","last_unstale":"2023-12-19T18:24:08.365796+0000","last_undegraded":"2023-12-19T18:24:08.365796+0000","last_fullsized":"2023-12-19T18:24:08.365796+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:48:55.273272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382776+0000","last_change":"2023-12-19T18:24:08.382596+0000","last_active":"2023-12-19T18:24:08.382776+0000","last_peered":"2023-12-19T18:24:08.382776+0000","last_clean":"2023-12-19T18:24:08.382776+0000","last_became_active":"2023-12-19T18:24:08.381056+0000","last_became_peered":"2023-12-19T18:24:08.381056+0000","last_unstale":"2023-12-19T18:24:08.382776+0000","last_undegraded":"2023-12-19T18:24:08.382776+0000","last_fullsized":"2023-12-19T18:24:08.382776+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:03:55.875007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.353647+0000","last_change":"2023-12-19T18:24:08.353308+0000","last_active":"2023-12-19T18:24:08.353647+0000","last_peered":"2023-12-19T18:24:08.353647+0000","last_clean":"2023-12-19T18:24:08.353647+0000","last_became_active":"2023-12-19T18:24:08.352641+0000","last_became_peered":"2023-12-19T18:24:08.352641+0000","last_unstale":"2023-12-19T18:24:08.353647+0000","last_undegraded":"2023-12-19T18:24:08.353647+0000","last_fullsized":"2023-12-19T18:24:08.353647+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:20:20.916791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382621+0000","last_change":"2023-12-19T18:24:08.382138+0000","last_active":"2023-12-19T18:24:08.382621+0000","last_peered":"2023-12-19T18:24:08.382621+0000","last_clean":"2023-12-19T18:24:08.382621+0000","last_became_active":"2023-12-19T18:24:08.380759+0000","last_became_peered":"2023-12-19T18:24:08.380759+0000","last_unstale":"2023-12-19T18:24:08.382621+0000","last_undegraded":"2023-12-19T18:24:08.382621+0000","last_fullsized":"2023-12-19T18:24:08.382621+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:03:06.021100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380018+0000","last_change":"2023-12-19T18:24:08.377689+0000","last_active":"2023-12-19T18:24:08.380018+0000","last_peered":"2023-12-19T18:24:08.380018+0000","last_clean":"2023-12-19T18:24:08.380018+0000","last_became_active":"2023-12-19T18:24:08.377399+0000","last_became_peered":"2023-12-19T18:24:08.377399+0000","last_unstale":"2023-12-19T18:24:08.380018+0000","last_undegraded":"2023-12-19T18:24:08.380018+0000","last_fullsized":"2023-12-19T18:24:08.380018+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:42:11.977275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.376396+0000","last_change":"2023-12-19T18:24:08.376259+0000","last_active":"2023-12-19T18:24:08.376396+0000","last_peered":"2023-12-19T18:24:08.376396+0000","last_clean":"2023-12-19T18:24:08.376396+0000","last_became_active":"2023-12-19T18:24:08.375953+0000","last_became_peered":"2023-12-19T18:24:08.375953+0000","last_unstale":"2023-12-19T18:24:08.376396+0000","last_undegraded":"2023-12-19T18:24:08.376396+0000","last_fullsized":"2023-12-19T18:24:08.376396+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:23:53.234858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382589+0000","last_change":"2023-12-19T18:24:08.382017+0000","last_active":"2023-12-19T18:24:08.382589+0000","last_peered":"2023-12-19T18:24:08.382589+0000","last_clean":"2023-12-19T18:24:08.382589+0000","last_became_active":"2023-12-19T18:24:08.381085+0000","last_became_peered":"2023-12-19T18:24:08.381085+0000","last_unstale":"2023-12-19T18:24:08.382589+0000","last_undegraded":"2023-12-19T18:24:08.382589+0000","last_fullsized":"2023-12-19T18:24:08.382589+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:48:00.856741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.373452+0000","last_change":"2023-12-19T18:24:08.373348+0000","last_active":"2023-12-19T18:24:08.373452+0000","last_peered":"2023-12-19T18:24:08.373452+0000","last_clean":"2023-12-19T18:24:08.373452+0000","last_became_active":"2023-12-19T18:24:08.370846+0000","last_became_peered":"2023-12-19T18:24:08.370846+0000","last_unstale":"2023-12-19T18:24:08.373452+0000","last_undegraded":"2023-12-19T18:24:08.373452+0000","last_fullsized":"2023-12-19T18:24:08.373452+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:20:20.916791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380252+0000","last_change":"2023-12-19T18:24:08.380176+0000","last_active":"2023-12-19T18:24:08.380252+0000","last_peered":"2023-12-19T18:24:08.380252+0000","last_clean":"2023-12-19T18:24:08.380252+0000","last_became_active":"2023-12-19T18:24:08.379946+0000","last_became_peered":"2023-12-19T18:24:08.379946+0000","last_unstale":"2023-12-19T18:24:08.380252+0000","last_undegraded":"2023-12-19T18:24:08.380252+0000","last_fullsized":"2023-12-19T18:24:08.380252+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:04:15.916411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.367912+0000","last_change":"2023-12-19T18:24:08.365585+0000","last_active":"2023-12-19T18:24:08.367912+0000","last_peered":"2023-12-19T18:24:08.367912+0000","last_clean":"2023-12-19T18:24:08.367912+0000","last_became_active":"2023-12-19T18:24:08.365347+0000","last_became_peered":"2023-12-19T18:24:08.365347+0000","last_unstale":"2023-12-19T18:24:08.367912+0000","last_undegraded":"2023-12-19T18:24:08.367912+0000","last_fullsized":"2023-12-19T18:24:08.367912+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:59:21.973339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.376952+0000","last_change":"2023-12-19T18:24:08.374976+0000","last_active":"2023-12-19T18:24:08.376952+0000","last_peered":"2023-12-19T18:24:08.376952+0000","last_clean":"2023-12-19T18:24:08.376952+0000","last_became_active":"2023-12-19T18:24:08.374720+0000","last_became_peered":"2023-12-19T18:24:08.374720+0000","last_unstale":"2023-12-19T18:24:08.376952+0000","last_undegraded":"2023-12-19T18:24:08.376952+0000","last_fullsized":"2023-12-19T18:24:08.376952+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:48:00.856741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"169'4","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.373957+0000","last_change":"2023-12-19T18:24:07.835645+0000","last_active":"2023-12-19T18:24:08.373957+0000","last_peered":"2023-12-19T18:24:08.373957+0000","last_clean":"2023-12-19T18:24:08.373957+0000","last_became_active":"2023-12-19T18:24:07.834985+0000","last_became_peered":"2023-12-19T18:24:07.834985+0000","last_unstale":"2023-12-19T18:24:08.373957+0000","last_undegraded":"2023-12-19T18:24:08.373957+0000","last_fullsized":"2023-12-19T18:24:08.373957+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:03:54.865602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.362458+0000","last_change":"2023-12-19T18:24:08.362138+0000","last_active":"2023-12-19T18:24:08.362458+0000","last_peered":"2023-12-19T18:24:08.362458+0000","last_clean":"2023-12-19T18:24:08.362458+0000","last_became_active":"2023-12-19T18:24:08.361859+0000","last_became_peered":"2023-12-19T18:24:08.361859+0000","last_unstale":"2023-12-19T18:24:08.362458+0000","last_undegraded":"2023-12-19T18:24:08.362458+0000","last_fullsized":"2023-12-19T18:24:08.362458+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:21.693486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.367073+0000","last_change":"2023-12-19T18:24:07.858702+0000","last_active":"2023-12-19T18:24:08.367073+0000","last_peered":"2023-12-19T18:24:08.367073+0000","last_clean":"2023-12-19T18:24:08.367073+0000","last_became_active":"2023-12-19T18:24:07.857348+0000","last_became_peered":"2023-12-19T18:24:07.857348+0000","last_unstale":"2023-12-19T18:24:08.367073+0000","last_undegraded":"2023-12-19T18:24:08.367073+0000","last_fullsized":"2023-12-19T18:24:08.367073+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:27:14.666538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.358847+0000","last_change":"2023-12-19T18:24:08.358773+0000","last_active":"2023-12-19T18:24:08.358847+0000","last_peered":"2023-12-19T18:24:08.358847+0000","last_clean":"2023-12-19T18:24:08.358847+0000","last_became_active":"2023-12-19T18:24:08.358552+0000","last_became_peered":"2023-12-19T18:24:08.358552+0000","last_unstale":"2023-12-19T18:24:08.358847+0000","last_undegraded":"2023-12-19T18:24:08.358847+0000","last_fullsized":"2023-12-19T18:24:08.358847+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:27:57.787243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.385358+0000","last_change":"2023-12-19T18:24:07.846327+0000","last_active":"2023-12-19T18:24:08.385358+0000","last_peered":"2023-12-19T18:24:08.385358+0000","last_clean":"2023-12-19T18:24:08.385358+0000","last_became_active":"2023-12-19T18:24:07.845959+0000","last_became_peered":"2023-12-19T18:24:07.845959+0000","last_unstale":"2023-12-19T18:24:08.385358+0000","last_undegraded":"2023-12-19T18:24:08.385358+0000","last_fullsized":"2023-12-19T18:24:08.385358+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:31.012210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.379008+0000","last_change":"2023-12-19T18:24:08.378628+0000","last_active":"2023-12-19T18:24:08.379008+0000","last_peered":"2023-12-19T18:24:08.379008+0000","last_clean":"2023-12-19T18:24:08.379008+0000","last_became_active":"2023-12-19T18:24:08.378376+0000","last_became_peered":"2023-12-19T18:24:08.378376+0000","last_unstale":"2023-12-19T18:24:08.379008+0000","last_undegraded":"2023-12-19T18:24:08.379008+0000","last_fullsized":"2023-12-19T18:24:08.379008+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:27:35.903794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.383465+0000","last_change":"2023-12-19T18:24:07.851480+0000","last_active":"2023-12-19T18:24:08.383465+0000","last_peered":"2023-12-19T18:24:08.383465+0000","last_clean":"2023-12-19T18:24:08.383465+0000","last_became_active":"2023-12-19T18:24:07.851210+0000","last_became_peered":"2023-12-19T18:24:07.851210+0000","last_unstale":"2023-12-19T18:24:08.383465+0000","last_undegraded":"2023-12-19T18:24:08.383465+0000","last_fullsized":"2023-12-19T18:24:08.383465+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:40:41.730373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.384137+0000","last_change":"2023-12-19T18:24:08.384058+0000","last_active":"2023-12-19T18:24:08.384137+0000","last_peered":"2023-12-19T18:24:08.384137+0000","last_clean":"2023-12-19T18:24:08.384137+0000","last_became_active":"2023-12-19T18:24:08.383597+0000","last_became_peered":"2023-12-19T18:24:08.383597+0000","last_unstale":"2023-12-19T18:24:08.384137+0000","last_undegraded":"2023-12-19T18:24:08.384137+0000","last_fullsized":"2023-12-19T18:24:08.384137+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:16:01.456865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.377538+0000","last_change":"2023-12-19T18:24:07.851233+0000","last_active":"2023-12-19T18:24:08.377538+0000","last_peered":"2023-12-19T18:24:08.377538+0000","last_clean":"2023-12-19T18:24:08.377538+0000","last_became_active":"2023-12-19T18:24:07.850845+0000","last_became_peered":"2023-12-19T18:24:07.850845+0000","last_unstale":"2023-12-19T18:24:08.377538+0000","last_undegraded":"2023-12-19T18:24:08.377538+0000","last_fullsized":"2023-12-19T18:24:08.377538+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:48.492830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.379041+0000","last_change":"2023-12-19T18:24:08.378855+0000","last_active":"2023-12-19T18:24:08.379041+0000","last_peered":"2023-12-19T18:24:08.379041+0000","last_clean":"2023-12-19T18:24:08.379041+0000","last_became_active":"2023-12-19T18:24:08.378436+0000","last_became_peered":"2023-12-19T18:24:08.378436+0000","last_unstale":"2023-12-19T18:24:08.379041+0000","last_undegraded":"2023-12-19T18:24:08.379041+0000","last_fullsized":"2023-12-19T18:24:08.379041+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:15.049131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.383879+0000","last_change":"2023-12-19T18:24:07.849975+0000","last_active":"2023-12-19T18:24:08.383879+0000","last_peered":"2023-12-19T18:24:08.383879+0000","last_clean":"2023-12-19T18:24:08.383879+0000","last_became_active":"2023-12-19T18:24:07.849713+0000","last_became_peered":"2023-12-19T18:24:07.849713+0000","last_unstale":"2023-12-19T18:24:08.383879+0000","last_undegraded":"2023-12-19T18:24:08.383879+0000","last_fullsized":"2023-12-19T18:24:08.383879+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:20:35.641800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.362587+0000","last_change":"2023-12-19T18:24:08.362514+0000","last_active":"2023-12-19T18:24:08.362587+0000","last_peered":"2023-12-19T18:24:08.362587+0000","last_clean":"2023-12-19T18:24:08.362587+0000","last_became_active":"2023-12-19T18:24:08.362110+0000","last_became_peered":"2023-12-19T18:24:08.362110+0000","last_unstale":"2023-12-19T18:24:08.362587+0000","last_undegraded":"2023-12-19T18:24:08.362587+0000","last_fullsized":"2023-12-19T18:24:08.362587+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:45:36.959427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.356579+0000","last_change":"2023-12-19T18:24:07.858845+0000","last_active":"2023-12-19T18:24:08.356579+0000","last_peered":"2023-12-19T18:24:08.356579+0000","last_clean":"2023-12-19T18:24:08.356579+0000","last_became_active":"2023-12-19T18:24:07.858422+0000","last_became_peered":"2023-12-19T18:24:07.858422+0000","last_unstale":"2023-12-19T18:24:08.356579+0000","last_undegraded":"2023-12-19T18:24:08.356579+0000","last_fullsized":"2023-12-19T18:24:08.356579+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:17:50.446767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382558+0000","last_change":"2023-12-19T18:24:08.381888+0000","last_active":"2023-12-19T18:24:08.382558+0000","last_peered":"2023-12-19T18:24:08.382558+0000","last_clean":"2023-12-19T18:24:08.382558+0000","last_became_active":"2023-12-19T18:24:08.380687+0000","last_became_peered":"2023-12-19T18:24:08.380687+0000","last_unstale":"2023-12-19T18:24:08.382558+0000","last_undegraded":"2023-12-19T18:24:08.382558+0000","last_fullsized":"2023-12-19T18:24:08.382558+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:52:36.884988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.368203+0000","last_change":"2023-12-19T18:24:07.848158+0000","last_active":"2023-12-19T18:24:08.368203+0000","last_peered":"2023-12-19T18:24:08.368203+0000","last_clean":"2023-12-19T18:24:08.368203+0000","last_became_active":"2023-12-19T18:24:07.846783+0000","last_became_peered":"2023-12-19T18:24:07.846783+0000","last_unstale":"2023-12-19T18:24:08.368203+0000","last_undegraded":"2023-12-19T18:24:08.368203+0000","last_fullsized":"2023-12-19T18:24:08.368203+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:14.039726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382604+0000","last_change":"2023-12-19T18:24:08.382112+0000","last_active":"2023-12-19T18:24:08.382604+0000","last_peered":"2023-12-19T18:24:08.382604+0000","last_clean":"2023-12-19T18:24:08.382604+0000","last_became_active":"2023-12-19T18:24:08.381245+0000","last_became_peered":"2023-12-19T18:24:08.381245+0000","last_unstale":"2023-12-19T18:24:08.382604+0000","last_undegraded":"2023-12-19T18:24:08.382604+0000","last_fullsized":"2023-12-19T18:24:08.382604+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:11:45.151948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.355423+0000","last_change":"2023-12-19T18:24:07.850532+0000","last_active":"2023-12-19T18:24:08.355423+0000","last_peered":"2023-12-19T18:24:08.355423+0000","last_clean":"2023-12-19T18:24:08.355423+0000","last_became_active":"2023-12-19T18:24:07.848933+0000","last_became_peered":"2023-12-19T18:24:07.848933+0000","last_unstale":"2023-12-19T18:24:08.355423+0000","last_undegraded":"2023-12-19T18:24:08.355423+0000","last_fullsized":"2023-12-19T18:24:08.355423+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:54:25.319062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.374930+0000","last_change":"2023-12-19T18:24:08.374768+0000","last_active":"2023-12-19T18:24:08.374930+0000","last_peered":"2023-12-19T18:24:08.374930+0000","last_clean":"2023-12-19T18:24:08.374930+0000","last_became_active":"2023-12-19T18:24:08.374537+0000","last_became_peered":"2023-12-19T18:24:08.374537+0000","last_unstale":"2023-12-19T18:24:08.374930+0000","last_undegraded":"2023-12-19T18:24:08.374930+0000","last_fullsized":"2023-12-19T18:24:08.374930+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:27:35.903794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.373387+0000","last_change":"2023-12-19T18:24:07.846574+0000","last_active":"2023-12-19T18:24:08.373387+0000","last_peered":"2023-12-19T18:24:08.373387+0000","last_clean":"2023-12-19T18:24:08.373387+0000","last_became_active":"2023-12-19T18:24:07.846248+0000","last_became_peered":"2023-12-19T18:24:07.846248+0000","last_unstale":"2023-12-19T18:24:08.373387+0000","last_undegraded":"2023-12-19T18:24:08.373387+0000","last_fullsized":"2023-12-19T18:24:08.373387+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:44:53.499837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.358528+0000","last_change":"2023-12-19T18:24:08.358423+0000","last_active":"2023-12-19T18:24:08.358528+0000","last_peered":"2023-12-19T18:24:08.358528+0000","last_clean":"2023-12-19T18:24:08.358528+0000","last_became_active":"2023-12-19T18:24:08.358131+0000","last_became_peered":"2023-12-19T18:24:08.358131+0000","last_unstale":"2023-12-19T18:24:08.358528+0000","last_undegraded":"2023-12-19T18:24:08.358528+0000","last_fullsized":"2023-12-19T18:24:08.358528+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:16:01.456865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"169'3","reported_seq":13,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.373347+0000","last_change":"2023-12-19T18:24:07.838418+0000","last_active":"2023-12-19T18:24:08.373347+0000","last_peered":"2023-12-19T18:24:08.373347+0000","last_clean":"2023-12-19T18:24:08.373347+0000","last_became_active":"2023-12-19T18:24:07.837665+0000","last_became_peered":"2023-12-19T18:24:07.837665+0000","last_unstale":"2023-12-19T18:24:08.373347+0000","last_undegraded":"2023-12-19T18:24:08.373347+0000","last_fullsized":"2023-12-19T18:24:08.373347+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:08:30.083697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.361496+0000","last_change":"2023-12-19T18:24:08.361428+0000","last_active":"2023-12-19T18:24:08.361496+0000","last_peered":"2023-12-19T18:24:08.361496+0000","last_clean":"2023-12-19T18:24:08.361496+0000","last_became_active":"2023-12-19T18:24:08.361099+0000","last_became_peered":"2023-12-19T18:24:08.361099+0000","last_unstale":"2023-12-19T18:24:08.361496+0000","last_undegraded":"2023-12-19T18:24:08.361496+0000","last_fullsized":"2023-12-19T18:24:08.361496+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:59:10.167390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"169'6","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.379451+0000","last_change":"2023-12-19T18:24:07.849608+0000","last_active":"2023-12-19T18:24:08.379451+0000","last_peered":"2023-12-19T18:24:08.379451+0000","last_clean":"2023-12-19T18:24:08.379451+0000","last_became_active":"2023-12-19T18:24:07.848082+0000","last_became_peered":"2023-12-19T18:24:07.848082+0000","last_unstale":"2023-12-19T18:24:08.379451+0000","last_undegraded":"2023-12-19T18:24:08.379451+0000","last_fullsized":"2023-12-19T18:24:08.379451+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:25.224894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.384803+0000","last_change":"2023-12-19T18:24:08.384725+0000","last_active":"2023-12-19T18:24:08.384803+0000","last_peered":"2023-12-19T18:24:08.384803+0000","last_clean":"2023-12-19T18:24:08.384803+0000","last_became_active":"2023-12-19T18:24:08.384452+0000","last_became_peered":"2023-12-19T18:24:08.384452+0000","last_unstale":"2023-12-19T18:24:08.384803+0000","last_undegraded":"2023-12-19T18:24:08.384803+0000","last_fullsized":"2023-12-19T18:24:08.384803+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:32.021615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.370031+0000","last_change":"2023-12-19T18:24:07.843854+0000","last_active":"2023-12-19T18:24:08.370031+0000","last_peered":"2023-12-19T18:24:08.370031+0000","last_clean":"2023-12-19T18:24:08.370031+0000","last_became_active":"2023-12-19T18:24:07.841863+0000","last_became_peered":"2023-12-19T18:24:07.841863+0000","last_unstale":"2023-12-19T18:24:08.370031+0000","last_undegraded":"2023-12-19T18:24:08.370031+0000","last_fullsized":"2023-12-19T18:24:08.370031+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:16:00.447460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380051+0000","last_change":"2023-12-19T18:24:08.378476+0000","last_active":"2023-12-19T18:24:08.380051+0000","last_peered":"2023-12-19T18:24:08.380051+0000","last_clean":"2023-12-19T18:24:08.380051+0000","last_became_active":"2023-12-19T18:24:08.374053+0000","last_became_peered":"2023-12-19T18:24:08.374053+0000","last_unstale":"2023-12-19T18:24:08.380051+0000","last_undegraded":"2023-12-19T18:24:08.380051+0000","last_fullsized":"2023-12-19T18:24:08.380051+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:40:03.158824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.349186+0000","last_change":"2023-12-19T18:24:07.842757+0000","last_active":"2023-12-19T18:24:08.349186+0000","last_peered":"2023-12-19T18:24:08.349186+0000","last_clean":"2023-12-19T18:24:08.349186+0000","last_became_active":"2023-12-19T18:24:07.842202+0000","last_became_peered":"2023-12-19T18:24:07.842202+0000","last_unstale":"2023-12-19T18:24:08.349186+0000","last_undegraded":"2023-12-19T18:24:08.349186+0000","last_fullsized":"2023-12-19T18:24:08.349186+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:07.710175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.357727+0000","last_change":"2023-12-19T18:24:08.356702+0000","last_active":"2023-12-19T18:24:08.357727+0000","last_peered":"2023-12-19T18:24:08.357727+0000","last_clean":"2023-12-19T18:24:08.357727+0000","last_became_active":"2023-12-19T18:24:08.356141+0000","last_became_peered":"2023-12-19T18:24:08.356141+0000","last_unstale":"2023-12-19T18:24:08.357727+0000","last_undegraded":"2023-12-19T18:24:08.357727+0000","last_fullsized":"2023-12-19T18:24:08.357727+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:49.502235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"169'8","reported_seq":18,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382916+0000","last_change":"2023-12-19T18:24:07.850309+0000","last_active":"2023-12-19T18:24:08.382916+0000","last_peered":"2023-12-19T18:24:08.382916+0000","last_clean":"2023-12-19T18:24:08.382916+0000","last_became_active":"2023-12-19T18:24:07.850064+0000","last_became_peered":"2023-12-19T18:24:07.850064+0000","last_unstale":"2023-12-19T18:24:08.382916+0000","last_undegraded":"2023-12-19T18:24:08.382916+0000","last_fullsized":"2023-12-19T18:24:08.382916+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:30:48.439388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.379403+0000","last_change":"2023-12-19T18:24:08.379327+0000","last_active":"2023-12-19T18:24:08.379403+0000","last_peered":"2023-12-19T18:24:08.379403+0000","last_clean":"2023-12-19T18:24:08.379403+0000","last_became_active":"2023-12-19T18:24:08.378520+0000","last_became_peered":"2023-12-19T18:24:08.378520+0000","last_unstale":"2023-12-19T18:24:08.379403+0000","last_undegraded":"2023-12-19T18:24:08.379403+0000","last_fullsized":"2023-12-19T18:24:08.379403+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:45.539977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.367608+0000","last_change":"2023-12-19T18:24:07.847424+0000","last_active":"2023-12-19T18:24:08.367608+0000","last_peered":"2023-12-19T18:24:08.367608+0000","last_clean":"2023-12-19T18:24:08.367608+0000","last_became_active":"2023-12-19T18:24:07.845842+0000","last_became_peered":"2023-12-19T18:24:07.845842+0000","last_unstale":"2023-12-19T18:24:08.367608+0000","last_undegraded":"2023-12-19T18:24:08.367608+0000","last_fullsized":"2023-12-19T18:24:08.367608+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:31:41.263967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.371903+0000","last_change":"2023-12-19T18:24:08.371828+0000","last_active":"2023-12-19T18:24:08.371903+0000","last_peered":"2023-12-19T18:24:08.371903+0000","last_clean":"2023-12-19T18:24:08.371903+0000","last_became_active":"2023-12-19T18:24:08.371552+0000","last_became_peered":"2023-12-19T18:24:08.371552+0000","last_unstale":"2023-12-19T18:24:08.371903+0000","last_undegraded":"2023-12-19T18:24:08.371903+0000","last_fullsized":"2023-12-19T18:24:08.371903+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:00.467931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.385037+0000","last_change":"2023-12-19T18:24:07.841045+0000","last_active":"2023-12-19T18:24:08.385037+0000","last_peered":"2023-12-19T18:24:08.385037+0000","last_clean":"2023-12-19T18:24:08.385037+0000","last_became_active":"2023-12-19T18:24:07.840408+0000","last_became_peered":"2023-12-19T18:24:07.840408+0000","last_unstale":"2023-12-19T18:24:08.385037+0000","last_undegraded":"2023-12-19T18:24:08.385037+0000","last_fullsized":"2023-12-19T18:24:08.385037+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:12.758668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382543+0000","last_change":"2023-12-19T18:24:08.381772+0000","last_active":"2023-12-19T18:24:08.382543+0000","last_peered":"2023-12-19T18:24:08.382543+0000","last_clean":"2023-12-19T18:24:08.382543+0000","last_became_active":"2023-12-19T18:24:08.377348+0000","last_became_peered":"2023-12-19T18:24:08.377348+0000","last_unstale":"2023-12-19T18:24:08.382543+0000","last_undegraded":"2023-12-19T18:24:08.382543+0000","last_fullsized":"2023-12-19T18:24:08.382543+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:18:53.695825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.371134+0000","last_change":"2023-12-19T18:24:07.857366+0000","last_active":"2023-12-19T18:24:08.371134+0000","last_peered":"2023-12-19T18:24:08.371134+0000","last_clean":"2023-12-19T18:24:08.371134+0000","last_became_active":"2023-12-19T18:24:07.856567+0000","last_became_peered":"2023-12-19T18:24:07.856567+0000","last_unstale":"2023-12-19T18:24:08.371134+0000","last_undegraded":"2023-12-19T18:24:08.371134+0000","last_fullsized":"2023-12-19T18:24:08.371134+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:53.645430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381374+0000","last_change":"2023-12-19T18:24:08.376304+0000","last_active":"2023-12-19T18:24:08.381374+0000","last_peered":"2023-12-19T18:24:08.381374+0000","last_clean":"2023-12-19T18:24:08.381374+0000","last_became_active":"2023-12-19T18:24:08.375932+0000","last_became_peered":"2023-12-19T18:24:08.375932+0000","last_unstale":"2023-12-19T18:24:08.381374+0000","last_undegraded":"2023-12-19T18:24:08.381374+0000","last_fullsized":"2023-12-19T18:24:08.381374+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:31:57.093000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.366069+0000","last_change":"2023-12-19T18:24:07.853555+0000","last_active":"2023-12-19T18:24:08.366069+0000","last_peered":"2023-12-19T18:24:08.366069+0000","last_clean":"2023-12-19T18:24:08.366069+0000","last_became_active":"2023-12-19T18:24:07.853122+0000","last_became_peered":"2023-12-19T18:24:07.853122+0000","last_unstale":"2023-12-19T18:24:08.366069+0000","last_undegraded":"2023-12-19T18:24:08.366069+0000","last_fullsized":"2023-12-19T18:24:08.366069+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:40:40.392497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.372432+0000","last_change":"2023-12-19T18:24:08.372024+0000","last_active":"2023-12-19T18:24:08.372432+0000","last_peered":"2023-12-19T18:24:08.372432+0000","last_clean":"2023-12-19T18:24:08.372432+0000","last_became_active":"2023-12-19T18:24:08.371331+0000","last_became_peered":"2023-12-19T18:24:08.371331+0000","last_unstale":"2023-12-19T18:24:08.372432+0000","last_undegraded":"2023-12-19T18:24:08.372432+0000","last_fullsized":"2023-12-19T18:24:08.372432+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:04.052385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.371315+0000","last_change":"2023-12-19T18:24:07.857978+0000","last_active":"2023-12-19T18:24:08.371315+0000","last_peered":"2023-12-19T18:24:08.371315+0000","last_clean":"2023-12-19T18:24:08.371315+0000","last_became_active":"2023-12-19T18:24:07.856803+0000","last_became_peered":"2023-12-19T18:24:07.856803+0000","last_unstale":"2023-12-19T18:24:08.371315+0000","last_undegraded":"2023-12-19T18:24:08.371315+0000","last_fullsized":"2023-12-19T18:24:08.371315+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:36.587232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.358706+0000","last_change":"2023-12-19T18:24:08.358612+0000","last_active":"2023-12-19T18:24:08.358706+0000","last_peered":"2023-12-19T18:24:08.358706+0000","last_clean":"2023-12-19T18:24:08.358706+0000","last_became_active":"2023-12-19T18:24:08.358377+0000","last_became_peered":"2023-12-19T18:24:08.358377+0000","last_unstale":"2023-12-19T18:24:08.358706+0000","last_undegraded":"2023-12-19T18:24:08.358706+0000","last_fullsized":"2023-12-19T18:24:08.358706+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:25:52.888125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.371122+0000","last_change":"2023-12-19T18:24:07.859555+0000","last_active":"2023-12-19T18:24:08.371122+0000","last_peered":"2023-12-19T18:24:08.371122+0000","last_clean":"2023-12-19T18:24:08.371122+0000","last_became_active":"2023-12-19T18:24:07.859155+0000","last_became_peered":"2023-12-19T18:24:07.859155+0000","last_unstale":"2023-12-19T18:24:08.371122+0000","last_undegraded":"2023-12-19T18:24:08.371122+0000","last_fullsized":"2023-12-19T18:24:08.371122+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:33.080490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.383670+0000","last_change":"2023-12-19T18:24:08.383592+0000","last_active":"2023-12-19T18:24:08.383670+0000","last_peered":"2023-12-19T18:24:08.383670+0000","last_clean":"2023-12-19T18:24:08.383670+0000","last_became_active":"2023-12-19T18:24:08.383354+0000","last_became_peered":"2023-12-19T18:24:08.383354+0000","last_unstale":"2023-12-19T18:24:08.383670+0000","last_undegraded":"2023-12-19T18:24:08.383670+0000","last_fullsized":"2023-12-19T18:24:08.383670+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:52:36.884988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.376687+0000","last_change":"2023-12-19T18:24:07.851951+0000","last_active":"2023-12-19T18:24:08.376687+0000","last_peered":"2023-12-19T18:24:08.376687+0000","last_clean":"2023-12-19T18:24:08.376687+0000","last_became_active":"2023-12-19T18:24:07.851579+0000","last_became_peered":"2023-12-19T18:24:07.851579+0000","last_unstale":"2023-12-19T18:24:08.376687+0000","last_undegraded":"2023-12-19T18:24:08.376687+0000","last_fullsized":"2023-12-19T18:24:08.376687+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:48.492830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381387+0000","last_change":"2023-12-19T18:24:08.381338+0000","last_active":"2023-12-19T18:24:08.381387+0000","last_peered":"2023-12-19T18:24:08.381387+0000","last_clean":"2023-12-19T18:24:08.381387+0000","last_became_active":"2023-12-19T18:24:08.381141+0000","last_became_peered":"2023-12-19T18:24:08.381141+0000","last_unstale":"2023-12-19T18:24:08.381387+0000","last_undegraded":"2023-12-19T18:24:08.381387+0000","last_fullsized":"2023-12-19T18:24:08.381387+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:08:31.093102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.350469+0000","last_change":"2023-12-19T18:24:07.856117+0000","last_active":"2023-12-19T18:24:08.350469+0000","last_peered":"2023-12-19T18:24:08.350469+0000","last_clean":"2023-12-19T18:24:08.350469+0000","last_became_active":"2023-12-19T18:24:07.855943+0000","last_became_peered":"2023-12-19T18:24:07.855943+0000","last_unstale":"2023-12-19T18:24:08.350469+0000","last_undegraded":"2023-12-19T18:24:08.350469+0000","last_fullsized":"2023-12-19T18:24:08.350469+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:44.737120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.360046+0000","last_change":"2023-12-19T18:24:08.359967+0000","last_active":"2023-12-19T18:24:08.360046+0000","last_peered":"2023-12-19T18:24:08.360046+0000","last_clean":"2023-12-19T18:24:08.360046+0000","last_became_active":"2023-12-19T18:24:08.359701+0000","last_became_peered":"2023-12-19T18:24:08.359701+0000","last_unstale":"2023-12-19T18:24:08.360046+0000","last_undegraded":"2023-12-19T18:24:08.360046+0000","last_fullsized":"2023-12-19T18:24:08.360046+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:11:00.980095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382006+0000","last_change":"2023-12-19T18:24:07.857582+0000","last_active":"2023-12-19T18:24:08.382006+0000","last_peered":"2023-12-19T18:24:08.382006+0000","last_clean":"2023-12-19T18:24:08.382006+0000","last_became_active":"2023-12-19T18:24:07.857401+0000","last_became_peered":"2023-12-19T18:24:07.857401+0000","last_unstale":"2023-12-19T18:24:08.382006+0000","last_undegraded":"2023-12-19T18:24:08.382006+0000","last_fullsized":"2023-12-19T18:24:08.382006+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:33:23.371165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.370639+0000","last_change":"2023-12-19T18:24:08.370512+0000","last_active":"2023-12-19T18:24:08.370639+0000","last_peered":"2023-12-19T18:24:08.370639+0000","last_clean":"2023-12-19T18:24:08.370639+0000","last_became_active":"2023-12-19T18:24:08.369842+0000","last_became_peered":"2023-12-19T18:24:08.369842+0000","last_unstale":"2023-12-19T18:24:08.370639+0000","last_undegraded":"2023-12-19T18:24:08.370639+0000","last_fullsized":"2023-12-19T18:24:08.370639+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:11:12.380314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.363062+0000","last_change":"2023-12-19T18:24:07.850755+0000","last_active":"2023-12-19T18:24:08.363062+0000","last_peered":"2023-12-19T18:24:08.363062+0000","last_clean":"2023-12-19T18:24:08.363062+0000","last_became_active":"2023-12-19T18:24:07.850250+0000","last_became_peered":"2023-12-19T18:24:07.850250+0000","last_unstale":"2023-12-19T18:24:08.363062+0000","last_undegraded":"2023-12-19T18:24:08.363062+0000","last_fullsized":"2023-12-19T18:24:08.363062+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:14.183993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380036+0000","last_change":"2023-12-19T18:24:08.377781+0000","last_active":"2023-12-19T18:24:08.380036+0000","last_peered":"2023-12-19T18:24:08.380036+0000","last_clean":"2023-12-19T18:24:08.380036+0000","last_became_active":"2023-12-19T18:24:08.377264+0000","last_became_peered":"2023-12-19T18:24:08.377264+0000","last_unstale":"2023-12-19T18:24:08.380036+0000","last_undegraded":"2023-12-19T18:24:08.380036+0000","last_fullsized":"2023-12-19T18:24:08.380036+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:56:06.338402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.371279+0000","last_change":"2023-12-19T18:24:07.847872+0000","last_active":"2023-12-19T18:24:08.371279+0000","last_peered":"2023-12-19T18:24:08.371279+0000","last_clean":"2023-12-19T18:24:08.371279+0000","last_became_active":"2023-12-19T18:24:07.847549+0000","last_became_peered":"2023-12-19T18:24:07.847549+0000","last_unstale":"2023-12-19T18:24:08.371279+0000","last_undegraded":"2023-12-19T18:24:08.371279+0000","last_fullsized":"2023-12-19T18:24:08.371279+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:10:57.945137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381820+0000","last_change":"2023-12-19T18:24:08.381731+0000","last_active":"2023-12-19T18:24:08.381820+0000","last_peered":"2023-12-19T18:24:08.381820+0000","last_clean":"2023-12-19T18:24:08.381820+0000","last_became_active":"2023-12-19T18:24:08.380797+0000","last_became_peered":"2023-12-19T18:24:08.380797+0000","last_unstale":"2023-12-19T18:24:08.381820+0000","last_undegraded":"2023-12-19T18:24:08.381820+0000","last_fullsized":"2023-12-19T18:24:08.381820+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:04:29.837453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.376826+0000","last_change":"2023-12-19T18:24:07.836260+0000","last_active":"2023-12-19T18:24:08.376826+0000","last_peered":"2023-12-19T18:24:08.376826+0000","last_clean":"2023-12-19T18:24:08.376826+0000","last_became_active":"2023-12-19T18:24:07.835697+0000","last_became_peered":"2023-12-19T18:24:07.835697+0000","last_unstale":"2023-12-19T18:24:08.376826+0000","last_undegraded":"2023-12-19T18:24:08.376826+0000","last_fullsized":"2023-12-19T18:24:08.376826+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:15.350531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.370046+0000","last_change":"2023-12-19T18:24:08.368396+0000","last_active":"2023-12-19T18:24:08.370046+0000","last_peered":"2023-12-19T18:24:08.370046+0000","last_clean":"2023-12-19T18:24:08.370046+0000","last_became_active":"2023-12-19T18:24:08.367761+0000","last_became_peered":"2023-12-19T18:24:08.367761+0000","last_unstale":"2023-12-19T18:24:08.370046+0000","last_undegraded":"2023-12-19T18:24:08.370046+0000","last_fullsized":"2023-12-19T18:24:08.370046+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:16.359936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.365199+0000","last_change":"2023-12-19T18:24:07.845815+0000","last_active":"2023-12-19T18:24:08.365199+0000","last_peered":"2023-12-19T18:24:08.365199+0000","last_clean":"2023-12-19T18:24:08.365199+0000","last_became_active":"2023-12-19T18:24:07.845462+0000","last_became_peered":"2023-12-19T18:24:07.845462+0000","last_unstale":"2023-12-19T18:24:08.365199+0000","last_undegraded":"2023-12-19T18:24:08.365199+0000","last_fullsized":"2023-12-19T18:24:08.365199+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:15.350531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.357788+0000","last_change":"2023-12-19T18:24:08.357269+0000","last_active":"2023-12-19T18:24:08.357788+0000","last_peered":"2023-12-19T18:24:08.357788+0000","last_clean":"2023-12-19T18:24:08.357788+0000","last_became_active":"2023-12-19T18:24:08.357021+0000","last_became_peered":"2023-12-19T18:24:08.357021+0000","last_unstale":"2023-12-19T18:24:08.357788+0000","last_undegraded":"2023-12-19T18:24:08.357788+0000","last_fullsized":"2023-12-19T18:24:08.357788+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:15.193398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.352810+0000","last_change":"2023-12-19T18:24:07.853773+0000","last_active":"2023-12-19T18:24:08.352810+0000","last_peered":"2023-12-19T18:24:08.352810+0000","last_clean":"2023-12-19T18:24:08.352810+0000","last_became_active":"2023-12-19T18:24:07.852959+0000","last_became_peered":"2023-12-19T18:24:07.852959+0000","last_unstale":"2023-12-19T18:24:08.352810+0000","last_undegraded":"2023-12-19T18:24:08.352810+0000","last_fullsized":"2023-12-19T18:24:08.352810+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:43:34.775884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.383140+0000","last_change":"2023-12-19T18:24:08.382807+0000","last_active":"2023-12-19T18:24:08.383140+0000","last_peered":"2023-12-19T18:24:08.383140+0000","last_clean":"2023-12-19T18:24:08.383140+0000","last_became_active":"2023-12-19T18:24:08.381396+0000","last_became_peered":"2023-12-19T18:24:08.381396+0000","last_unstale":"2023-12-19T18:24:08.383140+0000","last_undegraded":"2023-12-19T18:24:08.383140+0000","last_fullsized":"2023-12-19T18:24:08.383140+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:08:31.093102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.350915+0000","last_change":"2023-12-19T18:24:07.848616+0000","last_active":"2023-12-19T18:24:08.350915+0000","last_peered":"2023-12-19T18:24:08.350915+0000","last_clean":"2023-12-19T18:24:08.350915+0000","last_became_active":"2023-12-19T18:24:07.848119+0000","last_became_peered":"2023-12-19T18:24:07.848119+0000","last_unstale":"2023-12-19T18:24:08.350915+0000","last_undegraded":"2023-12-19T18:24:08.350915+0000","last_fullsized":"2023-12-19T18:24:08.350915+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:03.042980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380067+0000","last_change":"2023-12-19T18:24:08.379786+0000","last_active":"2023-12-19T18:24:08.380067+0000","last_peered":"2023-12-19T18:24:08.380067+0000","last_clean":"2023-12-19T18:24:08.380067+0000","last_became_active":"2023-12-19T18:24:08.376557+0000","last_became_peered":"2023-12-19T18:24:08.376557+0000","last_unstale":"2023-12-19T18:24:08.380067+0000","last_undegraded":"2023-12-19T18:24:08.380067+0000","last_fullsized":"2023-12-19T18:24:08.380067+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:08.719580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.368048+0000","last_change":"2023-12-19T18:24:07.839076+0000","last_active":"2023-12-19T18:24:08.368048+0000","last_peered":"2023-12-19T18:24:08.368048+0000","last_clean":"2023-12-19T18:24:08.368048+0000","last_became_active":"2023-12-19T18:24:07.838687+0000","last_became_peered":"2023-12-19T18:24:07.838687+0000","last_unstale":"2023-12-19T18:24:08.368048+0000","last_undegraded":"2023-12-19T18:24:08.368048+0000","last_fullsized":"2023-12-19T18:24:08.368048+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:40:40.392497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.369179+0000","last_change":"2023-12-19T18:24:08.369081+0000","last_active":"2023-12-19T18:24:08.369179+0000","last_peered":"2023-12-19T18:24:08.369179+0000","last_clean":"2023-12-19T18:24:08.369179+0000","last_became_active":"2023-12-19T18:24:08.368867+0000","last_became_peered":"2023-12-19T18:24:08.368867+0000","last_unstale":"2023-12-19T18:24:08.369179+0000","last_undegraded":"2023-12-19T18:24:08.369179+0000","last_fullsized":"2023-12-19T18:24:08.369179+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:10.506959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382564+0000","last_change":"2023-12-19T18:24:07.852598+0000","last_active":"2023-12-19T18:24:08.382564+0000","last_peered":"2023-12-19T18:24:08.382564+0000","last_clean":"2023-12-19T18:24:08.382564+0000","last_became_active":"2023-12-19T18:24:07.851919+0000","last_became_peered":"2023-12-19T18:24:07.851919+0000","last_unstale":"2023-12-19T18:24:08.382564+0000","last_undegraded":"2023-12-19T18:24:08.382564+0000","last_fullsized":"2023-12-19T18:24:08.382564+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:01:38.785106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":298,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-12-19T18:24:07.835486+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:24:07.835486+0000","last_peered":"2023-12-19T18:24:07.835486+0000","last_clean":"2023-12-19T18:24:07.835486+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:24:07.835486+0000","last_undegraded":"2023-12-19T18:24:07.835486+0000","last_fullsized":"2023-12-19T18:24:07.835486+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":929,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194382,"statfs":{"total":200000000,"available":199047974,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.2110000000000001,"15min":1.149},"min":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.45900000000000002},"max":{"1min":1.488,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.169,"5min":1.264,"15min":1.198},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.55800000000000005},"max":{"1min":1.587,"5min":1.7330000000000001,"15min":1.79},"last":0.79600000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.206,"15min":1.2190000000000001},"min":{"1min":0.65800000000000003,"5min":0.63600000000000001,"15min":0.54400000000000004},"max":{"1min":1.5940000000000001,"5min":5.9029999999999996,"15min":10.926},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.256,"5min":1.258,"15min":1.25},"min":{"1min":0.80700000000000005,"5min":0.71899999999999997,"15min":0.67900000000000005},"max":{"1min":1.736,"5min":1.736,"15min":10.904999999999999},"last":1.1990000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1240000000000001,"15min":1.097},"min":{"1min":0.66000000000000003,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.444,"5min":3.1960000000000002,"15min":3.1960000000000002},"last":1.016},{"interface":"front","average":{"1min":1.173,"5min":1.2450000000000001,"15min":1.1910000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.6799999999999999,"5min":6.008,"15min":6.008},"last":1.069}]},{"osd":3,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.278,"15min":1.1759999999999999},"min":{"1min":0.89800000000000002,"5min":0.66900000000000004,"15min":0.61399999999999999},"max":{"1min":1.643,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.284,"5min":1.401,"15min":1.3029999999999999},"min":{"1min":0.77500000000000002,"5min":0.77300000000000002,"15min":0.69599999999999995},"max":{"1min":1.671,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":1.1699999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.274,"15min":1.1499999999999999},"min":{"1min":0.74299999999999999,"5min":0.55700000000000005,"15min":0.47899999999999998},"max":{"1min":1.847,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2909999999999999,"15min":1.246},"min":{"1min":0.66200000000000003,"5min":0.59299999999999997,"15min":0.57599999999999996},"max":{"1min":1.7450000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.3300000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3660000000000001,"15min":1.331},"min":{"1min":0.68999999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.8140000000000001,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.4370000000000001,"15min":1.347},"min":{"1min":0.627,"5min":0.627,"15min":0.53500000000000003},"max":{"1min":1.7969999999999999,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":1.3839999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.548,"15min":1.4790000000000001},"min":{"1min":0.72499999999999998,"5min":0.628,"15min":0.59899999999999998},"max":{"1min":1.8959999999999999,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.468,"15min":1.3899999999999999},"min":{"1min":0.79900000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.929,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":1.7809999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":929,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194382,"statfs":{"total":200000000,"available":199048008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1040000000000001,"15min":1.028},"min":{"1min":0.75800000000000001,"5min":0.68899999999999995,"15min":0.65800000000000003},"max":{"1min":1.522,"5min":1.6259999999999999,"15min":1.8129999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.1339999999999999,"15min":1.0860000000000001},"min":{"1min":0.73399999999999999,"5min":0.64300000000000002,"15min":0.55600000000000005},"max":{"1min":1.6000000000000001,"5min":1.748,"15min":3.5529999999999999},"last":1.01}]},{"osd":1,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.1279999999999999,"15min":1.127},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68200000000000005},"max":{"1min":1.3759999999999999,"5min":1.5609999999999999,"15min":4.5540000000000003},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.1899999999999999,"15min":1.1619999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.60299999999999998},"max":{"1min":1.5960000000000001,"5min":1.885,"15min":1.885},"last":1.2190000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2110000000000001,"15min":1.159},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.626},"max":{"1min":1.4319999999999999,"5min":1.7330000000000001,"15min":1.821},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1830000000000001,"15min":1.218},"min":{"1min":0.79800000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.573,"5min":1.944,"15min":3.7909999999999999},"last":1.1499999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.2709999999999999,"15min":1.214},"min":{"1min":0.95699999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.641,"5min":1.722,"15min":3.8820000000000001},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.1839999999999999,"15min":1.171},"min":{"1min":0.79200000000000004,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4319999999999999,"5min":1.6559999999999999,"15min":4.6189999999999998},"last":1.306}]},{"osd":4,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.28,"15min":1.2549999999999999},"min":{"1min":0.81799999999999995,"5min":0.74099999999999999,"15min":0.65600000000000003},"max":{"1min":1.6519999999999999,"5min":1.9670000000000001,"15min":3.0680000000000001},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4159999999999999,"15min":1.365},"min":{"1min":1.2210000000000001,"5min":0.70799999999999996,"15min":0.57999999999999996},"max":{"1min":1.746,"5min":2.0430000000000001,"15min":3.7000000000000002},"last":1.5009999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2529999999999999,"15min":1.204},"min":{"1min":0.89500000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6579999999999999,"5min":2.2400000000000002,"15min":4.6680000000000001},"last":1.421},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.419,"15min":1.3640000000000001},"min":{"1min":1.018,"5min":0.626,"15min":0.626},"max":{"1min":1.597,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.794}]},{"osd":7,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.409,"15min":1.361},"min":{"1min":1.1659999999999999,"5min":0.72299999999999998,"15min":0.61899999999999999},"max":{"1min":1.698,"5min":2.133,"15min":3.9670000000000001},"last":1.841},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4139999999999999,"15min":1.371},"min":{"1min":1.0049999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7430000000000001,"5min":2.3519999999999999,"15min":5.6639999999999997},"last":1.607}]}]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1493,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193819,"statfs":{"total":200000000,"available":198470690,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.196,"15min":1.1679999999999999},"min":{"1min":0.69999999999999996,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.6739999999999999,"5min":1.8540000000000001,"15min":1.863},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.2729999999999999,"15min":1.2010000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.478,"5min":1.913,"15min":1.913},"last":1.292}]},{"osd":1,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.212,"15min":1.0900000000000001},"min":{"1min":0.71999999999999997,"5min":0.71599999999999997,"15min":0.65700000000000003},"max":{"1min":1.5249999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1759999999999999,"15min":1.0880000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.546,"5min":1.8300000000000001,"15min":1.843},"last":0.99399999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.155,"15min":1.101},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.59499999999999997},"max":{"1min":1.536,"5min":1.5580000000000001,"15min":1.794},"last":1.077},{"interface":"front","average":{"1min":1.278,"5min":1.26,"15min":1.1719999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8,"5min":1.8,"15min":2.036},"last":1.407}]},{"osd":3,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1990000000000001,"15min":1.1140000000000001},"min":{"1min":0.879,"5min":0.71999999999999997,"15min":0.68899999999999995},"max":{"1min":1.635,"5min":1.72,"15min":1.7450000000000001},"last":1.351},{"interface":"front","average":{"1min":1.286,"5min":1.3340000000000001,"15min":1.252},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.619,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.6000000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3149999999999999,"15min":1.216},"min":{"1min":0.626,"5min":0.626,"15min":0.46700000000000003},"max":{"1min":1.7250000000000001,"5min":1.944,"15min":2.0329999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.216,"15min":1.149},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.57099999999999995},"max":{"1min":1.5449999999999999,"5min":1.7689999999999999,"15min":1.901},"last":0.83899999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.46,"15min":1.369},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.7729999999999999,"5min":2.0329999999999999,"15min":2.1589999999999998},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.347,"15min":1.2669999999999999},"min":{"1min":0.84699999999999998,"5min":0.81999999999999995,"15min":0.49299999999999999},"max":{"1min":1.6419999999999999,"5min":1.702,"15min":1.976},"last":1.107}]},{"osd":7,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3200000000000001,"15min":1.25},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.50900000000000001},"max":{"1min":1.6839999999999999,"5min":1.881,"15min":2.0270000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4850000000000001,"15min":1.427},"min":{"1min":0.96199999999999997,"5min":0.95099999999999996,"15min":0.623},"max":{"1min":1.732,"5min":2.117,"15min":2.117},"last":1.4399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":916,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194395,"statfs":{"total":200000000,"available":199061058,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0349999999999999,"15min":1.006},"min":{"1min":0.78300000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.083,"15min":1.0509999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7909999999999999},"last":0.71199999999999997}]},{"osd":1,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.159,"15min":1.1499999999999999},"min":{"1min":0.745,"5min":0.67200000000000004,"15min":0.66900000000000004},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.2509999999999999,"15min":1.1919999999999999},"min":{"1min":0.92300000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.7450000000000001,"5min":1.875,"15min":1.875},"last":1.355}]},{"osd":2,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1459999999999999,"15min":1.1240000000000001},"min":{"1min":0.86799999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.7010000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.165,"5min":1.1739999999999999,"15min":1.1160000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.70499999999999996},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.93100000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2949999999999999,"15min":1.2669999999999999},"min":{"1min":0.85499999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.534,"5min":1.946,"15min":1.946},"last":0.875},{"interface":"front","average":{"1min":1.429,"5min":1.3759999999999999,"15min":1.3169999999999999},"min":{"1min":1.0029999999999999,"5min":0.83399999999999996,"15min":0.74399999999999999},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.002}]},{"osd":5,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.292,"15min":1.2190000000000001},"min":{"1min":0.84499999999999997,"5min":0.82299999999999995,"15min":0.44500000000000001},"max":{"1min":1.593,"5min":1.8129999999999999,"15min":1.8740000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.3919999999999999,"15min":1.3560000000000001},"min":{"1min":0.79800000000000004,"5min":0.61899999999999999,"15min":0.504},"max":{"1min":1.7150000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.2649999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3120000000000001,"15min":1.304},"min":{"1min":0.85999999999999999,"5min":0.76700000000000002,"15min":0.432},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.45,"15min":1.4019999999999999},"min":{"1min":1.0580000000000001,"5min":0.90800000000000003,"15min":0.64500000000000002},"max":{"1min":1.7749999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.4950000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.373,"15min":1.3500000000000001},"min":{"1min":0.95099999999999996,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":1.8200000000000001,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.45,"15min":1.415},"min":{"1min":1.1519999999999999,"5min":0.88500000000000001,"15min":0.57399999999999995},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.3149999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":923,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194389,"statfs":{"total":200000000,"available":199054783,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91900000000000004,"15min":0.94999999999999996},"min":{"1min":0.435,"5min":0.38100000000000001,"15min":0.35299999999999998},"max":{"1min":1.3939999999999999,"5min":3.3140000000000001,"15min":3.4350000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.99399999999999999,"15min":0.95299999999999996},"min":{"1min":0.54600000000000004,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.9159999999999999,"5min":4.3490000000000002,"15min":4.3490000000000002},"last":1.268}]},{"osd":1,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0429999999999999,"15min":1.05},"min":{"1min":0.624,"5min":0.439,"15min":0.41299999999999998},"max":{"1min":1.619,"5min":5.2510000000000003,"15min":5.2510000000000003},"last":1.133},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.143,"15min":1.1080000000000001},"min":{"1min":0.64400000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.494,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.0980000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2210000000000001,"15min":1.244},"min":{"1min":0.72499999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.7350000000000001,"5min":2.5369999999999999,"15min":13.833},"last":1.028},{"interface":"front","average":{"1min":1.292,"5min":1.381,"15min":1.3180000000000001},"min":{"1min":0.83799999999999997,"5min":0.60299999999999998,"15min":0.53200000000000003},"max":{"1min":1.7709999999999999,"5min":5.4480000000000004,"15min":16.748999999999999},"last":0.93100000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.321,"15min":1.244},"min":{"1min":0.85499999999999998,"5min":0.77400000000000002,"15min":0.65600000000000003},"max":{"1min":1.4970000000000001,"5min":5.1790000000000003,"15min":5.1790000000000003},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4199999999999999,"15min":1.409},"min":{"1min":0.97399999999999998,"5min":0.84599999999999997,"15min":0.73199999999999998},"max":{"1min":1.97,"5min":4.569,"15min":4.569},"last":1.3879999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.4399999999999999,"15min":1.4359999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.76000000000000001},"max":{"1min":1.679,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":1.8720000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.371,"15min":1.296},"min":{"1min":0.95899999999999996,"5min":0.80600000000000005,"15min":0.69999999999999996},"max":{"1min":1.804,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.4399999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4550000000000001,"15min":1.4670000000000001},"min":{"1min":0.86499999999999999,"5min":0.81799999999999995,"15min":0.81599999999999995},"max":{"1min":2.0550000000000002,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.5269999999999999,"15min":1.508},"min":{"1min":0.999,"5min":0.83199999999999996,"15min":0.80500000000000005},"max":{"1min":1.7270000000000001,"5min":5.3739999999999997,"15min":5.3739999999999997},"last":1.9830000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.482,"15min":1.4019999999999999},"min":{"1min":1.0580000000000001,"5min":0.78500000000000003,"15min":0.73199999999999998},"max":{"1min":1.6830000000000001,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.5740000000000001,"15min":1.5549999999999999},"min":{"1min":1.1020000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.113,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":1.575}]}]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194382,"statfs":{"total":200000000,"available":199047336,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.92800000000000005,"15min":0.998},"min":{"1min":0.68200000000000005,"5min":0.35299999999999998,"15min":0.34399999999999997},"max":{"1min":1.506,"5min":2.879,"15min":9.2550000000000008},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.98999999999999999,"15min":1.0229999999999999},"min":{"1min":0.47499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.776,"5min":2.387,"15min":4.7850000000000001},"last":1.379}]},{"osd":1,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.96599999999999997,"15min":1.1120000000000001},"min":{"1min":0.76000000000000001,"5min":0.47599999999999998,"15min":0.46000000000000002},"max":{"1min":1.1910000000000001,"5min":2.4390000000000001,"15min":8.2720000000000002},"last":1.079},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1020000000000001,"15min":1.141},"min":{"1min":0.622,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6279999999999999,"5min":2.7200000000000002,"15min":5.0620000000000003},"last":1.214}]},{"osd":3,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1479999999999999,"15min":1.304},"min":{"1min":0.81599999999999995,"5min":0.56000000000000005,"15min":0.51900000000000002},"max":{"1min":1.524,"5min":2.238,"15min":8.3109999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.1279999999999999,"15min":1.274},"min":{"1min":0.61699999999999999,"5min":0.55500000000000005,"15min":0.41799999999999998},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":8.0690000000000008},"last":0.628}]},{"osd":4,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.387,"15min":1.4359999999999999},"min":{"1min":0.88100000000000001,"5min":0.83399999999999996,"15min":0.79300000000000004},"max":{"1min":2.2160000000000002,"5min":2.6139999999999999,"15min":5.3929999999999998},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.456,"15min":1.5149999999999999},"min":{"1min":0.90100000000000002,"5min":0.82099999999999995,"15min":0.78300000000000003},"max":{"1min":1.8720000000000001,"5min":3.1070000000000002,"15min":7.3570000000000002},"last":1.6240000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3799999999999999,"15min":1.502},"min":{"1min":1.0820000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.0019999999999998,"5min":2.5579999999999998,"15min":8.1839999999999993},"last":1.323},{"interface":"front","average":{"1min":1.353,"5min":1.399,"15min":1.4039999999999999},"min":{"1min":0.89200000000000002,"5min":0.82299999999999995,"15min":0.72199999999999998},"max":{"1min":1.7849999999999999,"5min":2.4940000000000002,"15min":5.165},"last":1.044}]},{"osd":6,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4379999999999999,"15min":1.4430000000000001},"min":{"1min":0.94899999999999995,"5min":0.82399999999999995,"15min":0.72999999999999998},"max":{"1min":2.2570000000000001,"5min":3.0179999999999998,"15min":5.1059999999999999},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4790000000000001,"15min":1.536},"min":{"1min":1.0529999999999999,"5min":0.93300000000000005,"15min":0.77000000000000002},"max":{"1min":2.1539999999999999,"5min":2.9580000000000002,"15min":8.1370000000000005},"last":1.4330000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.5109999999999999,"15min":1.5349999999999999},"min":{"1min":0.95099999999999996,"5min":0.91100000000000003,"15min":0.78200000000000003},"max":{"1min":2.367,"5min":3.2189999999999999,"15min":5.1449999999999996},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.5589999999999999,"15min":1.6279999999999999},"min":{"1min":1.232,"5min":0.88100000000000001,"15min":0.86499999999999999},"max":{"1min":2.5790000000000002,"5min":4.8289999999999997,"15min":8.2300000000000004},"last":1.665}]}]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1499,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193812,"statfs":{"total":200000000,"available":198464415,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.86299999999999999,"15min":0.84699999999999998},"min":{"1min":0.41199999999999998,"5min":0.38,"15min":0.34699999999999998},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.94899999999999995},"min":{"1min":0.434,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.768,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.2430000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.0409999999999999,"15min":1.046},"min":{"1min":0.57499999999999996,"5min":0.42299999999999999,"15min":0.41399999999999998},"max":{"1min":1.5129999999999999,"5min":2.4590000000000001,"15min":2.5739999999999998},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.99399999999999999,"15min":0.93799999999999994},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":1.6839999999999999,"5min":1.716,"15min":1.845},"last":1.379}]},{"osd":3,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.288,"15min":1.179},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.6759999999999999,"5min":2.137,"15min":2.137},"last":2.089},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.204,"15min":1.1819999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.44900000000000001},"max":{"1min":1.8360000000000001,"5min":2.1320000000000001,"15min":2.3959999999999999},"last":1.135}]},{"osd":4,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3460000000000001,"15min":1.3},"min":{"1min":0.84099999999999997,"5min":0.72199999999999998,"15min":0.69099999999999995},"max":{"1min":1.917,"5min":2.444,"15min":2.444},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.383,"15min":1.3120000000000001},"min":{"1min":0.92500000000000004,"5min":0.79600000000000004,"15min":0.71099999999999997},"max":{"1min":1.8660000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.6180000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.391,"15min":1.3120000000000001},"min":{"1min":0.76700000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.998,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.8560000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.4590000000000001,"15min":1.4159999999999999},"min":{"1min":0.96199999999999997,"5min":0.95399999999999996,"15min":0.70499999999999996},"max":{"1min":2.141,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.302}]},{"osd":6,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.411,"15min":1.379},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9119999999999999,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":2.016},{"interface":"front","average":{"1min":1.329,"5min":1.4850000000000001,"15min":1.45},"min":{"1min":0.98499999999999999,"5min":0.91400000000000003,"15min":0.76400000000000001},"max":{"1min":2.2509999999999999,"5min":2.895,"15min":2.895},"last":1.5209999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.4350000000000001,"15min":1.397},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1629999999999998,"5min":2.5670000000000002,"15min":9.4830000000000005},"last":1.98},{"interface":"front","average":{"1min":1.462,"5min":1.5209999999999999,"15min":1.504},"min":{"1min":0.92400000000000004,"5min":0.874,"15min":0.71899999999999997},"max":{"1min":2.0720000000000001,"5min":2.456,"15min":9.4109999999999996},"last":1.9359999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":923,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194389,"statfs":{"total":200000000,"available":199054783,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.92900000000000005,"15min":0.89700000000000002},"min":{"1min":0.34499999999999997,"5min":0.311,"15min":0.311},"max":{"1min":1.355,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":1,"5min":1.073,"15min":1.034},"min":{"1min":0.48699999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.6459999999999999,"5min":2.0369999999999999,"15min":2.4180000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.99299999999999999,"15min":0.95799999999999996},"min":{"1min":0.52900000000000003,"5min":0.434,"15min":0.434},"max":{"1min":1.4550000000000001,"5min":1.536,"15min":1.7450000000000001},"last":0.998},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.0089999999999999,"15min":0.94699999999999995},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.5700000000000001,"5min":2.0950000000000002,"15min":2.7810000000000001},"last":0.53700000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.0209999999999999,"15min":1.006},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8069999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.748},{"interface":"front","average":{"1min":1.151,"5min":1.2529999999999999,"15min":1.2230000000000001},"min":{"1min":0.66100000000000003,"5min":0.53000000000000003,"15min":0.47499999999999998},"max":{"1min":1.9159999999999999,"5min":1.982,"15min":2.0089999999999999},"last":0.85099999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2949999999999999,"15min":1.2190000000000001},"min":{"1min":0.67600000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.49,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.165},{"interface":"front","average":{"1min":1.234,"5min":1.274,"15min":1.278},"min":{"1min":0.70399999999999996,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.056,"5min":2.2450000000000001,"15min":2.9529999999999998},"last":0.89000000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3540000000000001,"15min":1.3919999999999999},"min":{"1min":0.74099999999999999,"5min":0.68500000000000005,"15min":0.64500000000000002},"max":{"1min":1.887,"5min":2.3940000000000001,"15min":16.298999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.23,"5min":1.339,"15min":1.286},"min":{"1min":0.80800000000000005,"5min":0.72799999999999998,"15min":0.72199999999999998},"max":{"1min":1.6060000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.91300000000000003}]},{"osd":6,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4570000000000001,"15min":1.4019999999999999},"min":{"1min":1.044,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.706,"5min":2.2559999999999998,"15min":2.5910000000000002},"last":1.044},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.399,"15min":1.3680000000000001},"min":{"1min":0.89100000000000001,"5min":0.84999999999999998,"15min":0.77000000000000002},"max":{"1min":1.8819999999999999,"5min":2.4359999999999999,"15min":2.46},"last":1.1040000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.468,"15min":1.4099999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.70899999999999996},"max":{"1min":2.0230000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.218},{"interface":"front","average":{"1min":1.3,"5min":1.4139999999999999,"15min":1.413},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9370000000000001,"5min":2.4580000000000002,"15min":3.1429999999999998},"last":1.0189999999999999}]}]}],"pool_statfs":[]}} 2023-12-19T18:24:13.345 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:24:13.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.634+0000 7f2d94dfd640 1 Processor -- start 2023-12-19T18:24:13.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.634+0000 7f2d94dfd640 1 -- start start 2023-12-19T18:24:13.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.636+0000 7f2d94dfd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2d9015d270 0x7f2d900cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.636+0000 7f2d94dfd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d900704f0 con 0x7f2d900cce20 2023-12-19T18:24:13.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.636+0000 7f2d94dfd640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d900cbf00 con 0x7f2d9015ce90 2023-12-19T18:24:13.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.636+0000 7f2d94dfd640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9006ce90 con 0x7f2d9015d270 2023-12-19T18:24:13.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.636+0000 7f2d8ed9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2d900cce20 0x7f2d900cb7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40426/0 (socket says 172.21.15.27:40426) 2023-12-19T18:24:13.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.636+0000 7f2d8ed9d640 1 -- 172.21.15.27:0/3838484505 learned_addr learned my addr 172.21.15.27:0/3838484505 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:13.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.637+0000 7f2d85d9b640 1 --2- 172.21.15.27:0/3838484505 >> v2:172.21.15.192:3300/0 conn(0x7f2d9015d270 0x7f2d900cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.637+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482035703 0 0) 0x7f2d900704f0 con 0x7f2d900cce20 2023-12-19T18:24:13.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.637+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d7c003520 con 0x7f2d900cce20 2023-12-19T18:24:13.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.637+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2827632163 0 0) 0x7f2d7c003520 con 0x7f2d900cce20 2023-12-19T18:24:13.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.637+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 >> v2:172.21.15.192:3300/0 conn(0x7f2d9015d270 msgr2=0x7f2d900cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:13.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.637+0000 7f2d8dd9b640 1 --2- 172.21.15.27:0/3838484505 >> v2:172.21.15.192:3300/0 conn(0x7f2d9015d270 0x7f2d900cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.637+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 >> v1:172.21.15.192:6789/0 conn(0x7f2d9015ce90 legacy=0x7f2d900cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.637+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9006b4e0 con 0x7f2d900cce20 2023-12-19T18:24:13.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.637+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f2d880019f0 con 0x7f2d900cce20 2023-12-19T18:24:13.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.637+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 >> v1:172.21.15.27:6789/0 conn(0x7f2d900cce20 legacy=0x7f2d900cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d8dd9b640 1 --2- 172.21.15.27:0/3838484505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7c0039c0 0x7f2d7c003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d8dd9b640 1 --2- 172.21.15.27:0/3838484505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d7c0047c0 0x7f2d7c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d8dd9b640 1 --2- 172.21.15.27:0/3838484505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d7c005180 0x7f2d7c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9006ce90 con 0x7f2d7c005180 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c00c080 con 0x7f2d7c0039c0 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c00c240 con 0x7f2d7c0047c0 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d8ed9d640 1 --2- 172.21.15.27:0/3838484505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d7c005180 0x7f2d7c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d8e59c640 1 --2- 172.21.15.27:0/3838484505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7c0039c0 0x7f2d7c003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d85d9b640 1 --2- 172.21.15.27:0/3838484505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d7c0047c0 0x7f2d7c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d8ed9d640 1 -- 172.21.15.27:0/3838484505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d7c0047c0 msgr2=0x7f2d7c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d8ed9d640 1 --2- 172.21.15.27:0/3838484505 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d7c0047c0 0x7f2d7c004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d8ed9d640 1 -- 172.21.15.27:0/3838484505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7c0039c0 msgr2=0x7f2d7c003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d8ed9d640 1 --2- 172.21.15.27:0/3838484505 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7c0039c0 0x7f2d7c003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.638+0000 7f2d8ed9d640 1 -- 172.21.15.27:0/3838484505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c003520 con 0x7f2d7c005180 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.639+0000 7f2d8ed9d640 1 -- 172.21.15.27:0/3838484505 reap_dead start 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.640+0000 7f2d8ed9d640 1 --2- 172.21.15.27:0/3838484505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d7c005180 0x7f2d7c00b980 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800a760 tx=0x7f2d7c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d33bec1ab9c25e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:13.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.640+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88034630 con 0x7f2d7c005180 2023-12-19T18:24:13.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.640+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8802f070 con 0x7f2d7c005180 2023-12-19T18:24:13.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.640+0000 7f2d8dd9b640 1 -- 172.21.15.27:0/3838484505 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88034a10 con 0x7f2d7c005180 2023-12-19T18:24:13.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.640+0000 7f2d94dfd640 1 -- 172.21.15.27:0/3838484505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d7c005180 msgr2=0x7f2d7c00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.640+0000 7f2d94dfd640 1 --2- 172.21.15.27:0/3838484505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d7c005180 0x7f2d7c00b980 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800a760 tx=0x7f2d7c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.641+0000 7f2d94dfd640 1 -- 172.21.15.27:0/3838484505 shutdown_connections 2023-12-19T18:24:13.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.641+0000 7f2d94dfd640 1 --2- 172.21.15.27:0/3838484505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d7c005180 0x7f2d7c00b980 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.641+0000 7f2d94dfd640 1 -- 172.21.15.27:0/3838484505 >> 172.21.15.27:0/3838484505 conn(0x7f2d9006a330 msgr2=0x7f2d900b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:13.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.641+0000 7f2d94dfd640 1 -- 172.21.15.27:0/3838484505 shutdown_connections 2023-12-19T18:24:13.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.641+0000 7f2d94dfd640 1 -- 172.21.15.27:0/3838484505 wait complete. 2023-12-19T18:24:13.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.643+0000 7f2d94dfd640 1 Processor -- start 2023-12-19T18:24:13.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.643+0000 7f2d94dfd640 1 -- start start 2023-12-19T18:24:13.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.644+0000 7f2d94dfd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7c005180 0x7f2d9014c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.644+0000 7f2d94dfd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d900b2cc0 0x7f2d900b3140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.644+0000 7f2d94dfd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d900ae940 0x7f2d900aedc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.644+0000 7f2d94dfd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d900af300 con 0x7f2d900ae940 2023-12-19T18:24:13.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.644+0000 7f2d94dfd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9015e0c0 con 0x7f2d7c005180 2023-12-19T18:24:13.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.644+0000 7f2d94dfd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d90150800 con 0x7f2d900b2cc0 2023-12-19T18:24:13.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.644+0000 7f2d8ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d900ae940 0x7f2d900aedc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.644+0000 7f2d8ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d900ae940 0x7f2d900aedc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60104/0 (socket says 172.21.15.27:60104) 2023-12-19T18:24:13.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.644+0000 7f2d8ed9d640 1 -- 172.21.15.27:0/206856479 learned_addr learned my addr 172.21.15.27:0/206856479 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.644+0000 7f2d8e59c640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7c005180 0x7f2d9014c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.644+0000 7f2d85d9b640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d900b2cc0 0x7f2d900b3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.645+0000 7f2d8ed9d640 1 -- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d900b2cc0 msgr2=0x7f2d900b3140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.645+0000 7f2d8ed9d640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d900b2cc0 0x7f2d900b3140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.645+0000 7f2d8ed9d640 1 -- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7c005180 msgr2=0x7f2d9014c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.645+0000 7f2d8ed9d640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7c005180 0x7f2d9014c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.645+0000 7f2d8ed9d640 1 -- 172.21.15.27:0/206856479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9015d790 con 0x7f2d900ae940 2023-12-19T18:24:13.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.645+0000 7f2d8ed9d640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d900ae940 0x7f2d900aedc0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800a760 tx=0x7f2d88001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:13.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.645+0000 7f2d877fe640 1 -- 172.21.15.27:0/206856479 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d880347b0 con 0x7f2d900ae940 2023-12-19T18:24:13.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.645+0000 7f2d877fe640 1 -- 172.21.15.27:0/206856479 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d88034910 con 0x7f2d900ae940 2023-12-19T18:24:13.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.645+0000 7f2d877fe640 1 -- 172.21.15.27:0/206856479 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d88034ca0 con 0x7f2d900ae940 2023-12-19T18:24:13.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.645+0000 7f2d94dfd640 1 -- 172.21.15.27:0/206856479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9015da70 con 0x7f2d900ae940 2023-12-19T18:24:13.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.646+0000 7f2d94dfd640 1 -- 172.21.15.27:0/206856479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d900b3810 con 0x7f2d900ae940 2023-12-19T18:24:13.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.647+0000 7f2d94dfd640 1 -- 172.21.15.27:0/206856479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d90068e90 con 0x7f2d900ae940 2023-12-19T18:24:13.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.648+0000 7f2d877fe640 1 -- 172.21.15.27:0/206856479 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2d88031030 con 0x7f2d900ae940 2023-12-19T18:24:13.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.649+0000 7f2d877fe640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2d7006a460 0x7f2d7006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:13.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.649+0000 7f2d877fe640 1 -- 172.21.15.27:0/206856479 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d880aac00 con 0x7f2d900ae940 2023-12-19T18:24:13.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.649+0000 7f2d8e59c640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2d7006a460 0x7f2d7006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:13.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.651+0000 7f2d8e59c640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2d7006a460 0x7f2d7006c920 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2d80007fd0 tx=0x7f2d80009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:13.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.654+0000 7f2d877fe640 1 -- 172.21.15.27:0/206856479 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2d880761e0 con 0x7f2d900ae940 2023-12-19T18:24:13.848 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.849+0000 7f2d94dfd640 1 -- 172.21.15.27:0/206856479 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2d9006baa0 con 0x7f2d7006a460 2023-12-19T18:24:13.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.871+0000 7f2d877fe640 1 -- 172.21.15.27:0/206856479 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282896 (secure 0 0 0) 0x7f2d80014400 con 0x7f2d7006a460 2023-12-19T18:24:13.874 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:24:13.876 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:24:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.877+0000 7f2d94dfd640 1 -- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2d7006a460 msgr2=0x7f2d7006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.877+0000 7f2d94dfd640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2d7006a460 0x7f2d7006c920 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f2d80007fd0 tx=0x7f2d80009040 comp rx=0 tx=0).stop 2023-12-19T18:24:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.877+0000 7f2d94dfd640 1 -- 172.21.15.27:0/206856479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d900ae940 msgr2=0x7f2d900aedc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.877+0000 7f2d94dfd640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d900ae940 0x7f2d900aedc0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f2d8800a760 tx=0x7f2d88001cb0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.877+0000 7f2d94dfd640 1 -- 172.21.15.27:0/206856479 shutdown_connections 2023-12-19T18:24:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.877+0000 7f2d94dfd640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2d7006a460 0x7f2d7006c920 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.877+0000 7f2d94dfd640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d900ae940 0x7f2d900aedc0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.877+0000 7f2d94dfd640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2d900b2cc0 0x7f2d900b3140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.877+0000 7f2d94dfd640 1 --2- 172.21.15.27:0/206856479 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2d7c005180 0x7f2d9014c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.877+0000 7f2d94dfd640 1 -- 172.21.15.27:0/206856479 >> 172.21.15.27:0/206856479 conn(0x7f2d9006a330 msgr2=0x7f2d900af4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.878+0000 7f2d94dfd640 1 -- 172.21.15.27:0/206856479 shutdown_connections 2023-12-19T18:24:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:13.878+0000 7f2d94dfd640 1 -- 172.21.15.27:0/206856479 wait complete. 2023-12-19T18:24:13.897 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":446,"stamp":"2023-12-19T18:24:11.907372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":48,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":8545,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553954,"statfs":{"total":1600000000,"available":1591249047,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.088912"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381356+0000","last_change":"2023-12-19T18:24:08.376161+0000","last_active":"2023-12-19T18:24:08.381356+0000","last_peered":"2023-12-19T18:24:08.381356+0000","last_clean":"2023-12-19T18:24:08.381356+0000","last_became_active":"2023-12-19T18:24:08.375608+0000","last_became_peered":"2023-12-19T18:24:08.375608+0000","last_unstale":"2023-12-19T18:24:08.381356+0000","last_undegraded":"2023-12-19T18:24:08.381356+0000","last_fullsized":"2023-12-19T18:24:08.381356+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:26:49.103272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382526+0000","last_change":"2023-12-19T18:24:08.381626+0000","last_active":"2023-12-19T18:24:08.382526+0000","last_peered":"2023-12-19T18:24:08.382526+0000","last_clean":"2023-12-19T18:24:08.382526+0000","last_became_active":"2023-12-19T18:24:08.376947+0000","last_became_peered":"2023-12-19T18:24:08.376947+0000","last_unstale":"2023-12-19T18:24:08.382526+0000","last_undegraded":"2023-12-19T18:24:08.382526+0000","last_fullsized":"2023-12-19T18:24:08.382526+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:52:26.286720+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381156+0000","last_change":"2023-12-19T18:24:08.380971+0000","last_active":"2023-12-19T18:24:08.381156+0000","last_peered":"2023-12-19T18:24:08.381156+0000","last_clean":"2023-12-19T18:24:08.381156+0000","last_became_active":"2023-12-19T18:24:08.380611+0000","last_became_peered":"2023-12-19T18:24:08.380611+0000","last_unstale":"2023-12-19T18:24:08.381156+0000","last_undegraded":"2023-12-19T18:24:08.381156+0000","last_fullsized":"2023-12-19T18:24:08.381156+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:49:18.668947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.360440+0000","last_change":"2023-12-19T18:24:08.360188+0000","last_active":"2023-12-19T18:24:08.360440+0000","last_peered":"2023-12-19T18:24:08.360440+0000","last_clean":"2023-12-19T18:24:08.360440+0000","last_became_active":"2023-12-19T18:24:08.359539+0000","last_became_peered":"2023-12-19T18:24:08.359539+0000","last_unstale":"2023-12-19T18:24:08.360440+0000","last_undegraded":"2023-12-19T18:24:08.360440+0000","last_fullsized":"2023-12-19T18:24:08.360440+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:26:49.103272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.369871+0000","last_change":"2023-12-19T18:24:08.367100+0000","last_active":"2023-12-19T18:24:08.369871+0000","last_peered":"2023-12-19T18:24:08.369871+0000","last_clean":"2023-12-19T18:24:08.369871+0000","last_became_active":"2023-12-19T18:24:08.366848+0000","last_became_peered":"2023-12-19T18:24:08.366848+0000","last_unstale":"2023-12-19T18:24:08.369871+0000","last_undegraded":"2023-12-19T18:24:08.369871+0000","last_fullsized":"2023-12-19T18:24:08.369871+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:04.052385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.376628+0000","last_change":"2023-12-19T18:24:08.376363+0000","last_active":"2023-12-19T18:24:08.376628+0000","last_peered":"2023-12-19T18:24:08.376628+0000","last_clean":"2023-12-19T18:24:08.376628+0000","last_became_active":"2023-12-19T18:24:08.376012+0000","last_became_peered":"2023-12-19T18:24:08.376012+0000","last_unstale":"2023-12-19T18:24:08.376628+0000","last_undegraded":"2023-12-19T18:24:08.376628+0000","last_fullsized":"2023-12-19T18:24:08.376628+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:00.467931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.369898+0000","last_change":"2023-12-19T18:24:08.369499+0000","last_active":"2023-12-19T18:24:08.369898+0000","last_peered":"2023-12-19T18:24:08.369898+0000","last_clean":"2023-12-19T18:24:08.369898+0000","last_became_active":"2023-12-19T18:24:08.369266+0000","last_became_peered":"2023-12-19T18:24:08.369266+0000","last_unstale":"2023-12-19T18:24:08.369898+0000","last_undegraded":"2023-12-19T18:24:08.369898+0000","last_fullsized":"2023-12-19T18:24:08.369898+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:13.768073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.375567+0000","last_change":"2023-12-19T18:24:08.375450+0000","last_active":"2023-12-19T18:24:08.375567+0000","last_peered":"2023-12-19T18:24:08.375567+0000","last_clean":"2023-12-19T18:24:08.375567+0000","last_became_active":"2023-12-19T18:24:08.375154+0000","last_became_peered":"2023-12-19T18:24:08.375154+0000","last_unstale":"2023-12-19T18:24:08.375567+0000","last_undegraded":"2023-12-19T18:24:08.375567+0000","last_fullsized":"2023-12-19T18:24:08.375567+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:05:27.086846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381620+0000","last_change":"2023-12-19T18:24:08.381011+0000","last_active":"2023-12-19T18:24:08.381620+0000","last_peered":"2023-12-19T18:24:08.381620+0000","last_clean":"2023-12-19T18:24:08.381620+0000","last_became_active":"2023-12-19T18:24:08.380464+0000","last_became_peered":"2023-12-19T18:24:08.380464+0000","last_unstale":"2023-12-19T18:24:08.381620+0000","last_undegraded":"2023-12-19T18:24:08.381620+0000","last_fullsized":"2023-12-19T18:24:08.381620+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:33:27.734476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381390+0000","last_change":"2023-12-19T18:24:08.378192+0000","last_active":"2023-12-19T18:24:08.381390+0000","last_peered":"2023-12-19T18:24:08.381390+0000","last_clean":"2023-12-19T18:24:08.381390+0000","last_became_active":"2023-12-19T18:24:08.377755+0000","last_became_peered":"2023-12-19T18:24:08.377755+0000","last_unstale":"2023-12-19T18:24:08.381390+0000","last_undegraded":"2023-12-19T18:24:08.381390+0000","last_fullsized":"2023-12-19T18:24:08.381390+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:39:11.014721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380008+0000","last_change":"2023-12-19T18:24:08.379394+0000","last_active":"2023-12-19T18:24:08.380008+0000","last_peered":"2023-12-19T18:24:08.380008+0000","last_clean":"2023-12-19T18:24:08.380008+0000","last_became_active":"2023-12-19T18:24:08.378548+0000","last_became_peered":"2023-12-19T18:24:08.378548+0000","last_unstale":"2023-12-19T18:24:08.380008+0000","last_undegraded":"2023-12-19T18:24:08.380008+0000","last_fullsized":"2023-12-19T18:24:08.380008+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:25:03.694013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.357805+0000","last_change":"2023-12-19T18:24:08.357388+0000","last_active":"2023-12-19T18:24:08.357805+0000","last_peered":"2023-12-19T18:24:08.357805+0000","last_clean":"2023-12-19T18:24:08.357805+0000","last_became_active":"2023-12-19T18:24:08.356987+0000","last_became_peered":"2023-12-19T18:24:08.356987+0000","last_unstale":"2023-12-19T18:24:08.357805+0000","last_undegraded":"2023-12-19T18:24:08.357805+0000","last_fullsized":"2023-12-19T18:24:08.357805+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:39.310759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.360135+0000","last_change":"2023-12-19T18:24:08.359944+0000","last_active":"2023-12-19T18:24:08.360135+0000","last_peered":"2023-12-19T18:24:08.360135+0000","last_clean":"2023-12-19T18:24:08.360135+0000","last_became_active":"2023-12-19T18:24:08.358066+0000","last_became_peered":"2023-12-19T18:24:08.358066+0000","last_unstale":"2023-12-19T18:24:08.360135+0000","last_undegraded":"2023-12-19T18:24:08.360135+0000","last_fullsized":"2023-12-19T18:24:08.360135+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:18:03.637381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.383899+0000","last_change":"2023-12-19T18:24:08.383826+0000","last_active":"2023-12-19T18:24:08.383899+0000","last_peered":"2023-12-19T18:24:08.383899+0000","last_clean":"2023-12-19T18:24:08.383899+0000","last_became_active":"2023-12-19T18:24:08.383582+0000","last_became_peered":"2023-12-19T18:24:08.383582+0000","last_unstale":"2023-12-19T18:24:08.383899+0000","last_undegraded":"2023-12-19T18:24:08.383899+0000","last_fullsized":"2023-12-19T18:24:08.383899+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:21:25.284071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381625+0000","last_change":"2023-12-19T18:24:08.379784+0000","last_active":"2023-12-19T18:24:08.381625+0000","last_peered":"2023-12-19T18:24:08.381625+0000","last_clean":"2023-12-19T18:24:08.381625+0000","last_became_active":"2023-12-19T18:24:08.379409+0000","last_became_peered":"2023-12-19T18:24:08.379409+0000","last_unstale":"2023-12-19T18:24:08.381625+0000","last_undegraded":"2023-12-19T18:24:08.381625+0000","last_fullsized":"2023-12-19T18:24:08.381625+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:16.359936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380102+0000","last_change":"2023-12-19T18:24:08.378521+0000","last_active":"2023-12-19T18:24:08.380102+0000","last_peered":"2023-12-19T18:24:08.380102+0000","last_clean":"2023-12-19T18:24:08.380102+0000","last_became_active":"2023-12-19T18:24:08.376186+0000","last_became_peered":"2023-12-19T18:24:08.376186+0000","last_unstale":"2023-12-19T18:24:08.380102+0000","last_undegraded":"2023-12-19T18:24:08.380102+0000","last_fullsized":"2023-12-19T18:24:08.380102+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:25:03.694013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.362467+0000","last_change":"2023-12-19T18:24:08.362264+0000","last_active":"2023-12-19T18:24:08.362467+0000","last_peered":"2023-12-19T18:24:08.362467+0000","last_clean":"2023-12-19T18:24:08.362467+0000","last_became_active":"2023-12-19T18:24:08.362010+0000","last_became_peered":"2023-12-19T18:24:08.362010+0000","last_unstale":"2023-12-19T18:24:08.362467+0000","last_undegraded":"2023-12-19T18:24:08.362467+0000","last_fullsized":"2023-12-19T18:24:08.362467+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:08:56.632742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381510+0000","last_change":"2023-12-19T18:24:08.379242+0000","last_active":"2023-12-19T18:24:08.381510+0000","last_peered":"2023-12-19T18:24:08.381510+0000","last_clean":"2023-12-19T18:24:08.381510+0000","last_became_active":"2023-12-19T18:24:08.378871+0000","last_became_peered":"2023-12-19T18:24:08.378871+0000","last_unstale":"2023-12-19T18:24:08.381510+0000","last_undegraded":"2023-12-19T18:24:08.381510+0000","last_fullsized":"2023-12-19T18:24:08.381510+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:49.502235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.372875+0000","last_change":"2023-12-19T18:24:08.372223+0000","last_active":"2023-12-19T18:24:08.372875+0000","last_peered":"2023-12-19T18:24:08.372875+0000","last_clean":"2023-12-19T18:24:08.372875+0000","last_became_active":"2023-12-19T18:24:08.371748+0000","last_became_peered":"2023-12-19T18:24:08.371748+0000","last_unstale":"2023-12-19T18:24:08.372875+0000","last_undegraded":"2023-12-19T18:24:08.372875+0000","last_fullsized":"2023-12-19T18:24:08.372875+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:40:03.158824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.384544+0000","last_change":"2023-12-19T18:24:08.384331+0000","last_active":"2023-12-19T18:24:08.384544+0000","last_peered":"2023-12-19T18:24:08.384544+0000","last_clean":"2023-12-19T18:24:08.384544+0000","last_became_active":"2023-12-19T18:24:08.383917+0000","last_became_peered":"2023-12-19T18:24:08.383917+0000","last_unstale":"2023-12-19T18:24:08.384544+0000","last_undegraded":"2023-12-19T18:24:08.384544+0000","last_fullsized":"2023-12-19T18:24:08.384544+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:40:41.401902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.366916+0000","last_change":"2023-12-19T18:24:08.366507+0000","last_active":"2023-12-19T18:24:08.366916+0000","last_peered":"2023-12-19T18:24:08.366916+0000","last_clean":"2023-12-19T18:24:08.366916+0000","last_became_active":"2023-12-19T18:24:08.365865+0000","last_became_peered":"2023-12-19T18:24:08.365865+0000","last_unstale":"2023-12-19T18:24:08.366916+0000","last_undegraded":"2023-12-19T18:24:08.366916+0000","last_fullsized":"2023-12-19T18:24:08.366916+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:40:42.739778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382573+0000","last_change":"2023-12-19T18:24:08.381894+0000","last_active":"2023-12-19T18:24:08.382573+0000","last_peered":"2023-12-19T18:24:08.382573+0000","last_clean":"2023-12-19T18:24:08.382573+0000","last_became_active":"2023-12-19T18:24:08.377864+0000","last_became_peered":"2023-12-19T18:24:08.377864+0000","last_unstale":"2023-12-19T18:24:08.382573+0000","last_undegraded":"2023-12-19T18:24:08.382573+0000","last_fullsized":"2023-12-19T18:24:08.382573+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:44.600564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.365796+0000","last_change":"2023-12-19T18:24:08.365592+0000","last_active":"2023-12-19T18:24:08.365796+0000","last_peered":"2023-12-19T18:24:08.365796+0000","last_clean":"2023-12-19T18:24:08.365796+0000","last_became_active":"2023-12-19T18:24:08.364907+0000","last_became_peered":"2023-12-19T18:24:08.364907+0000","last_unstale":"2023-12-19T18:24:08.365796+0000","last_undegraded":"2023-12-19T18:24:08.365796+0000","last_fullsized":"2023-12-19T18:24:08.365796+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:48:55.273272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382776+0000","last_change":"2023-12-19T18:24:08.382596+0000","last_active":"2023-12-19T18:24:08.382776+0000","last_peered":"2023-12-19T18:24:08.382776+0000","last_clean":"2023-12-19T18:24:08.382776+0000","last_became_active":"2023-12-19T18:24:08.381056+0000","last_became_peered":"2023-12-19T18:24:08.381056+0000","last_unstale":"2023-12-19T18:24:08.382776+0000","last_undegraded":"2023-12-19T18:24:08.382776+0000","last_fullsized":"2023-12-19T18:24:08.382776+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:03:55.875007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.353647+0000","last_change":"2023-12-19T18:24:08.353308+0000","last_active":"2023-12-19T18:24:08.353647+0000","last_peered":"2023-12-19T18:24:08.353647+0000","last_clean":"2023-12-19T18:24:08.353647+0000","last_became_active":"2023-12-19T18:24:08.352641+0000","last_became_peered":"2023-12-19T18:24:08.352641+0000","last_unstale":"2023-12-19T18:24:08.353647+0000","last_undegraded":"2023-12-19T18:24:08.353647+0000","last_fullsized":"2023-12-19T18:24:08.353647+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:20:20.916791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382621+0000","last_change":"2023-12-19T18:24:08.382138+0000","last_active":"2023-12-19T18:24:08.382621+0000","last_peered":"2023-12-19T18:24:08.382621+0000","last_clean":"2023-12-19T18:24:08.382621+0000","last_became_active":"2023-12-19T18:24:08.380759+0000","last_became_peered":"2023-12-19T18:24:08.380759+0000","last_unstale":"2023-12-19T18:24:08.382621+0000","last_undegraded":"2023-12-19T18:24:08.382621+0000","last_fullsized":"2023-12-19T18:24:08.382621+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:03:06.021100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380018+0000","last_change":"2023-12-19T18:24:08.377689+0000","last_active":"2023-12-19T18:24:08.380018+0000","last_peered":"2023-12-19T18:24:08.380018+0000","last_clean":"2023-12-19T18:24:08.380018+0000","last_became_active":"2023-12-19T18:24:08.377399+0000","last_became_peered":"2023-12-19T18:24:08.377399+0000","last_unstale":"2023-12-19T18:24:08.380018+0000","last_undegraded":"2023-12-19T18:24:08.380018+0000","last_fullsized":"2023-12-19T18:24:08.380018+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:42:11.977275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.376396+0000","last_change":"2023-12-19T18:24:08.376259+0000","last_active":"2023-12-19T18:24:08.376396+0000","last_peered":"2023-12-19T18:24:08.376396+0000","last_clean":"2023-12-19T18:24:08.376396+0000","last_became_active":"2023-12-19T18:24:08.375953+0000","last_became_peered":"2023-12-19T18:24:08.375953+0000","last_unstale":"2023-12-19T18:24:08.376396+0000","last_undegraded":"2023-12-19T18:24:08.376396+0000","last_fullsized":"2023-12-19T18:24:08.376396+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:23:53.234858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382589+0000","last_change":"2023-12-19T18:24:08.382017+0000","last_active":"2023-12-19T18:24:08.382589+0000","last_peered":"2023-12-19T18:24:08.382589+0000","last_clean":"2023-12-19T18:24:08.382589+0000","last_became_active":"2023-12-19T18:24:08.381085+0000","last_became_peered":"2023-12-19T18:24:08.381085+0000","last_unstale":"2023-12-19T18:24:08.382589+0000","last_undegraded":"2023-12-19T18:24:08.382589+0000","last_fullsized":"2023-12-19T18:24:08.382589+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:48:00.856741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.373452+0000","last_change":"2023-12-19T18:24:08.373348+0000","last_active":"2023-12-19T18:24:08.373452+0000","last_peered":"2023-12-19T18:24:08.373452+0000","last_clean":"2023-12-19T18:24:08.373452+0000","last_became_active":"2023-12-19T18:24:08.370846+0000","last_became_peered":"2023-12-19T18:24:08.370846+0000","last_unstale":"2023-12-19T18:24:08.373452+0000","last_undegraded":"2023-12-19T18:24:08.373452+0000","last_fullsized":"2023-12-19T18:24:08.373452+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:20:20.916791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380252+0000","last_change":"2023-12-19T18:24:08.380176+0000","last_active":"2023-12-19T18:24:08.380252+0000","last_peered":"2023-12-19T18:24:08.380252+0000","last_clean":"2023-12-19T18:24:08.380252+0000","last_became_active":"2023-12-19T18:24:08.379946+0000","last_became_peered":"2023-12-19T18:24:08.379946+0000","last_unstale":"2023-12-19T18:24:08.380252+0000","last_undegraded":"2023-12-19T18:24:08.380252+0000","last_fullsized":"2023-12-19T18:24:08.380252+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:04:15.916411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.367912+0000","last_change":"2023-12-19T18:24:08.365585+0000","last_active":"2023-12-19T18:24:08.367912+0000","last_peered":"2023-12-19T18:24:08.367912+0000","last_clean":"2023-12-19T18:24:08.367912+0000","last_became_active":"2023-12-19T18:24:08.365347+0000","last_became_peered":"2023-12-19T18:24:08.365347+0000","last_unstale":"2023-12-19T18:24:08.367912+0000","last_undegraded":"2023-12-19T18:24:08.367912+0000","last_fullsized":"2023-12-19T18:24:08.367912+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:59:21.973339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.376952+0000","last_change":"2023-12-19T18:24:08.374976+0000","last_active":"2023-12-19T18:24:08.376952+0000","last_peered":"2023-12-19T18:24:08.376952+0000","last_clean":"2023-12-19T18:24:08.376952+0000","last_became_active":"2023-12-19T18:24:08.374720+0000","last_became_peered":"2023-12-19T18:24:08.374720+0000","last_unstale":"2023-12-19T18:24:08.376952+0000","last_undegraded":"2023-12-19T18:24:08.376952+0000","last_fullsized":"2023-12-19T18:24:08.376952+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:48:00.856741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"169'4","reported_seq":14,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.373957+0000","last_change":"2023-12-19T18:24:07.835645+0000","last_active":"2023-12-19T18:24:08.373957+0000","last_peered":"2023-12-19T18:24:08.373957+0000","last_clean":"2023-12-19T18:24:08.373957+0000","last_became_active":"2023-12-19T18:24:07.834985+0000","last_became_peered":"2023-12-19T18:24:07.834985+0000","last_unstale":"2023-12-19T18:24:08.373957+0000","last_undegraded":"2023-12-19T18:24:08.373957+0000","last_fullsized":"2023-12-19T18:24:08.373957+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:03:54.865602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.362458+0000","last_change":"2023-12-19T18:24:08.362138+0000","last_active":"2023-12-19T18:24:08.362458+0000","last_peered":"2023-12-19T18:24:08.362458+0000","last_clean":"2023-12-19T18:24:08.362458+0000","last_became_active":"2023-12-19T18:24:08.361859+0000","last_became_peered":"2023-12-19T18:24:08.361859+0000","last_unstale":"2023-12-19T18:24:08.362458+0000","last_undegraded":"2023-12-19T18:24:08.362458+0000","last_fullsized":"2023-12-19T18:24:08.362458+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:59:21.693486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.367073+0000","last_change":"2023-12-19T18:24:07.858702+0000","last_active":"2023-12-19T18:24:08.367073+0000","last_peered":"2023-12-19T18:24:08.367073+0000","last_clean":"2023-12-19T18:24:08.367073+0000","last_became_active":"2023-12-19T18:24:07.857348+0000","last_became_peered":"2023-12-19T18:24:07.857348+0000","last_unstale":"2023-12-19T18:24:08.367073+0000","last_undegraded":"2023-12-19T18:24:08.367073+0000","last_fullsized":"2023-12-19T18:24:08.367073+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:27:14.666538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.358847+0000","last_change":"2023-12-19T18:24:08.358773+0000","last_active":"2023-12-19T18:24:08.358847+0000","last_peered":"2023-12-19T18:24:08.358847+0000","last_clean":"2023-12-19T18:24:08.358847+0000","last_became_active":"2023-12-19T18:24:08.358552+0000","last_became_peered":"2023-12-19T18:24:08.358552+0000","last_unstale":"2023-12-19T18:24:08.358847+0000","last_undegraded":"2023-12-19T18:24:08.358847+0000","last_fullsized":"2023-12-19T18:24:08.358847+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:27:57.787243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.385358+0000","last_change":"2023-12-19T18:24:07.846327+0000","last_active":"2023-12-19T18:24:08.385358+0000","last_peered":"2023-12-19T18:24:08.385358+0000","last_clean":"2023-12-19T18:24:08.385358+0000","last_became_active":"2023-12-19T18:24:07.845959+0000","last_became_peered":"2023-12-19T18:24:07.845959+0000","last_unstale":"2023-12-19T18:24:08.385358+0000","last_undegraded":"2023-12-19T18:24:08.385358+0000","last_fullsized":"2023-12-19T18:24:08.385358+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:31.012210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.379008+0000","last_change":"2023-12-19T18:24:08.378628+0000","last_active":"2023-12-19T18:24:08.379008+0000","last_peered":"2023-12-19T18:24:08.379008+0000","last_clean":"2023-12-19T18:24:08.379008+0000","last_became_active":"2023-12-19T18:24:08.378376+0000","last_became_peered":"2023-12-19T18:24:08.378376+0000","last_unstale":"2023-12-19T18:24:08.379008+0000","last_undegraded":"2023-12-19T18:24:08.379008+0000","last_fullsized":"2023-12-19T18:24:08.379008+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:27:35.903794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.383465+0000","last_change":"2023-12-19T18:24:07.851480+0000","last_active":"2023-12-19T18:24:08.383465+0000","last_peered":"2023-12-19T18:24:08.383465+0000","last_clean":"2023-12-19T18:24:08.383465+0000","last_became_active":"2023-12-19T18:24:07.851210+0000","last_became_peered":"2023-12-19T18:24:07.851210+0000","last_unstale":"2023-12-19T18:24:08.383465+0000","last_undegraded":"2023-12-19T18:24:08.383465+0000","last_fullsized":"2023-12-19T18:24:08.383465+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:40:41.730373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.384137+0000","last_change":"2023-12-19T18:24:08.384058+0000","last_active":"2023-12-19T18:24:08.384137+0000","last_peered":"2023-12-19T18:24:08.384137+0000","last_clean":"2023-12-19T18:24:08.384137+0000","last_became_active":"2023-12-19T18:24:08.383597+0000","last_became_peered":"2023-12-19T18:24:08.383597+0000","last_unstale":"2023-12-19T18:24:08.384137+0000","last_undegraded":"2023-12-19T18:24:08.384137+0000","last_fullsized":"2023-12-19T18:24:08.384137+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:16:01.456865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.377538+0000","last_change":"2023-12-19T18:24:07.851233+0000","last_active":"2023-12-19T18:24:08.377538+0000","last_peered":"2023-12-19T18:24:08.377538+0000","last_clean":"2023-12-19T18:24:08.377538+0000","last_became_active":"2023-12-19T18:24:07.850845+0000","last_became_peered":"2023-12-19T18:24:07.850845+0000","last_unstale":"2023-12-19T18:24:08.377538+0000","last_undegraded":"2023-12-19T18:24:08.377538+0000","last_fullsized":"2023-12-19T18:24:08.377538+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:48.492830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.379041+0000","last_change":"2023-12-19T18:24:08.378855+0000","last_active":"2023-12-19T18:24:08.379041+0000","last_peered":"2023-12-19T18:24:08.379041+0000","last_clean":"2023-12-19T18:24:08.379041+0000","last_became_active":"2023-12-19T18:24:08.378436+0000","last_became_peered":"2023-12-19T18:24:08.378436+0000","last_unstale":"2023-12-19T18:24:08.379041+0000","last_undegraded":"2023-12-19T18:24:08.379041+0000","last_fullsized":"2023-12-19T18:24:08.379041+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:15.049131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.383879+0000","last_change":"2023-12-19T18:24:07.849975+0000","last_active":"2023-12-19T18:24:08.383879+0000","last_peered":"2023-12-19T18:24:08.383879+0000","last_clean":"2023-12-19T18:24:08.383879+0000","last_became_active":"2023-12-19T18:24:07.849713+0000","last_became_peered":"2023-12-19T18:24:07.849713+0000","last_unstale":"2023-12-19T18:24:08.383879+0000","last_undegraded":"2023-12-19T18:24:08.383879+0000","last_fullsized":"2023-12-19T18:24:08.383879+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:20:35.641800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.362587+0000","last_change":"2023-12-19T18:24:08.362514+0000","last_active":"2023-12-19T18:24:08.362587+0000","last_peered":"2023-12-19T18:24:08.362587+0000","last_clean":"2023-12-19T18:24:08.362587+0000","last_became_active":"2023-12-19T18:24:08.362110+0000","last_became_peered":"2023-12-19T18:24:08.362110+0000","last_unstale":"2023-12-19T18:24:08.362587+0000","last_undegraded":"2023-12-19T18:24:08.362587+0000","last_fullsized":"2023-12-19T18:24:08.362587+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:45:36.959427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.356579+0000","last_change":"2023-12-19T18:24:07.858845+0000","last_active":"2023-12-19T18:24:08.356579+0000","last_peered":"2023-12-19T18:24:08.356579+0000","last_clean":"2023-12-19T18:24:08.356579+0000","last_became_active":"2023-12-19T18:24:07.858422+0000","last_became_peered":"2023-12-19T18:24:07.858422+0000","last_unstale":"2023-12-19T18:24:08.356579+0000","last_undegraded":"2023-12-19T18:24:08.356579+0000","last_fullsized":"2023-12-19T18:24:08.356579+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:17:50.446767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382558+0000","last_change":"2023-12-19T18:24:08.381888+0000","last_active":"2023-12-19T18:24:08.382558+0000","last_peered":"2023-12-19T18:24:08.382558+0000","last_clean":"2023-12-19T18:24:08.382558+0000","last_became_active":"2023-12-19T18:24:08.380687+0000","last_became_peered":"2023-12-19T18:24:08.380687+0000","last_unstale":"2023-12-19T18:24:08.382558+0000","last_undegraded":"2023-12-19T18:24:08.382558+0000","last_fullsized":"2023-12-19T18:24:08.382558+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:52:36.884988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.368203+0000","last_change":"2023-12-19T18:24:07.848158+0000","last_active":"2023-12-19T18:24:08.368203+0000","last_peered":"2023-12-19T18:24:08.368203+0000","last_clean":"2023-12-19T18:24:08.368203+0000","last_became_active":"2023-12-19T18:24:07.846783+0000","last_became_peered":"2023-12-19T18:24:07.846783+0000","last_unstale":"2023-12-19T18:24:08.368203+0000","last_undegraded":"2023-12-19T18:24:08.368203+0000","last_fullsized":"2023-12-19T18:24:08.368203+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:14.039726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382604+0000","last_change":"2023-12-19T18:24:08.382112+0000","last_active":"2023-12-19T18:24:08.382604+0000","last_peered":"2023-12-19T18:24:08.382604+0000","last_clean":"2023-12-19T18:24:08.382604+0000","last_became_active":"2023-12-19T18:24:08.381245+0000","last_became_peered":"2023-12-19T18:24:08.381245+0000","last_unstale":"2023-12-19T18:24:08.382604+0000","last_undegraded":"2023-12-19T18:24:08.382604+0000","last_fullsized":"2023-12-19T18:24:08.382604+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:11:45.151948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.355423+0000","last_change":"2023-12-19T18:24:07.850532+0000","last_active":"2023-12-19T18:24:08.355423+0000","last_peered":"2023-12-19T18:24:08.355423+0000","last_clean":"2023-12-19T18:24:08.355423+0000","last_became_active":"2023-12-19T18:24:07.848933+0000","last_became_peered":"2023-12-19T18:24:07.848933+0000","last_unstale":"2023-12-19T18:24:08.355423+0000","last_undegraded":"2023-12-19T18:24:08.355423+0000","last_fullsized":"2023-12-19T18:24:08.355423+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:54:25.319062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.374930+0000","last_change":"2023-12-19T18:24:08.374768+0000","last_active":"2023-12-19T18:24:08.374930+0000","last_peered":"2023-12-19T18:24:08.374930+0000","last_clean":"2023-12-19T18:24:08.374930+0000","last_became_active":"2023-12-19T18:24:08.374537+0000","last_became_peered":"2023-12-19T18:24:08.374537+0000","last_unstale":"2023-12-19T18:24:08.374930+0000","last_undegraded":"2023-12-19T18:24:08.374930+0000","last_fullsized":"2023-12-19T18:24:08.374930+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:27:35.903794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.373387+0000","last_change":"2023-12-19T18:24:07.846574+0000","last_active":"2023-12-19T18:24:08.373387+0000","last_peered":"2023-12-19T18:24:08.373387+0000","last_clean":"2023-12-19T18:24:08.373387+0000","last_became_active":"2023-12-19T18:24:07.846248+0000","last_became_peered":"2023-12-19T18:24:07.846248+0000","last_unstale":"2023-12-19T18:24:08.373387+0000","last_undegraded":"2023-12-19T18:24:08.373387+0000","last_fullsized":"2023-12-19T18:24:08.373387+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:44:53.499837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.358528+0000","last_change":"2023-12-19T18:24:08.358423+0000","last_active":"2023-12-19T18:24:08.358528+0000","last_peered":"2023-12-19T18:24:08.358528+0000","last_clean":"2023-12-19T18:24:08.358528+0000","last_became_active":"2023-12-19T18:24:08.358131+0000","last_became_peered":"2023-12-19T18:24:08.358131+0000","last_unstale":"2023-12-19T18:24:08.358528+0000","last_undegraded":"2023-12-19T18:24:08.358528+0000","last_fullsized":"2023-12-19T18:24:08.358528+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:16:01.456865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"169'3","reported_seq":13,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.373347+0000","last_change":"2023-12-19T18:24:07.838418+0000","last_active":"2023-12-19T18:24:08.373347+0000","last_peered":"2023-12-19T18:24:08.373347+0000","last_clean":"2023-12-19T18:24:08.373347+0000","last_became_active":"2023-12-19T18:24:07.837665+0000","last_became_peered":"2023-12-19T18:24:07.837665+0000","last_unstale":"2023-12-19T18:24:08.373347+0000","last_undegraded":"2023-12-19T18:24:08.373347+0000","last_fullsized":"2023-12-19T18:24:08.373347+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:08:30.083697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.361496+0000","last_change":"2023-12-19T18:24:08.361428+0000","last_active":"2023-12-19T18:24:08.361496+0000","last_peered":"2023-12-19T18:24:08.361496+0000","last_clean":"2023-12-19T18:24:08.361496+0000","last_became_active":"2023-12-19T18:24:08.361099+0000","last_became_peered":"2023-12-19T18:24:08.361099+0000","last_unstale":"2023-12-19T18:24:08.361496+0000","last_undegraded":"2023-12-19T18:24:08.361496+0000","last_fullsized":"2023-12-19T18:24:08.361496+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:59:10.167390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"169'6","reported_seq":16,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.379451+0000","last_change":"2023-12-19T18:24:07.849608+0000","last_active":"2023-12-19T18:24:08.379451+0000","last_peered":"2023-12-19T18:24:08.379451+0000","last_clean":"2023-12-19T18:24:08.379451+0000","last_became_active":"2023-12-19T18:24:07.848082+0000","last_became_peered":"2023-12-19T18:24:07.848082+0000","last_unstale":"2023-12-19T18:24:08.379451+0000","last_undegraded":"2023-12-19T18:24:08.379451+0000","last_fullsized":"2023-12-19T18:24:08.379451+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:25.224894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.384803+0000","last_change":"2023-12-19T18:24:08.384725+0000","last_active":"2023-12-19T18:24:08.384803+0000","last_peered":"2023-12-19T18:24:08.384803+0000","last_clean":"2023-12-19T18:24:08.384803+0000","last_became_active":"2023-12-19T18:24:08.384452+0000","last_became_peered":"2023-12-19T18:24:08.384452+0000","last_unstale":"2023-12-19T18:24:08.384803+0000","last_undegraded":"2023-12-19T18:24:08.384803+0000","last_fullsized":"2023-12-19T18:24:08.384803+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:51:32.021615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.370031+0000","last_change":"2023-12-19T18:24:07.843854+0000","last_active":"2023-12-19T18:24:08.370031+0000","last_peered":"2023-12-19T18:24:08.370031+0000","last_clean":"2023-12-19T18:24:08.370031+0000","last_became_active":"2023-12-19T18:24:07.841863+0000","last_became_peered":"2023-12-19T18:24:07.841863+0000","last_unstale":"2023-12-19T18:24:08.370031+0000","last_undegraded":"2023-12-19T18:24:08.370031+0000","last_fullsized":"2023-12-19T18:24:08.370031+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:16:00.447460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380051+0000","last_change":"2023-12-19T18:24:08.378476+0000","last_active":"2023-12-19T18:24:08.380051+0000","last_peered":"2023-12-19T18:24:08.380051+0000","last_clean":"2023-12-19T18:24:08.380051+0000","last_became_active":"2023-12-19T18:24:08.374053+0000","last_became_peered":"2023-12-19T18:24:08.374053+0000","last_unstale":"2023-12-19T18:24:08.380051+0000","last_undegraded":"2023-12-19T18:24:08.380051+0000","last_fullsized":"2023-12-19T18:24:08.380051+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:40:03.158824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.349186+0000","last_change":"2023-12-19T18:24:07.842757+0000","last_active":"2023-12-19T18:24:08.349186+0000","last_peered":"2023-12-19T18:24:08.349186+0000","last_clean":"2023-12-19T18:24:08.349186+0000","last_became_active":"2023-12-19T18:24:07.842202+0000","last_became_peered":"2023-12-19T18:24:07.842202+0000","last_unstale":"2023-12-19T18:24:08.349186+0000","last_undegraded":"2023-12-19T18:24:08.349186+0000","last_fullsized":"2023-12-19T18:24:08.349186+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:07.710175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.357727+0000","last_change":"2023-12-19T18:24:08.356702+0000","last_active":"2023-12-19T18:24:08.357727+0000","last_peered":"2023-12-19T18:24:08.357727+0000","last_clean":"2023-12-19T18:24:08.357727+0000","last_became_active":"2023-12-19T18:24:08.356141+0000","last_became_peered":"2023-12-19T18:24:08.356141+0000","last_unstale":"2023-12-19T18:24:08.357727+0000","last_undegraded":"2023-12-19T18:24:08.357727+0000","last_fullsized":"2023-12-19T18:24:08.357727+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:49.502235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"169'8","reported_seq":18,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382916+0000","last_change":"2023-12-19T18:24:07.850309+0000","last_active":"2023-12-19T18:24:08.382916+0000","last_peered":"2023-12-19T18:24:08.382916+0000","last_clean":"2023-12-19T18:24:08.382916+0000","last_became_active":"2023-12-19T18:24:07.850064+0000","last_became_peered":"2023-12-19T18:24:07.850064+0000","last_unstale":"2023-12-19T18:24:08.382916+0000","last_undegraded":"2023-12-19T18:24:08.382916+0000","last_fullsized":"2023-12-19T18:24:08.382916+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:30:48.439388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.379403+0000","last_change":"2023-12-19T18:24:08.379327+0000","last_active":"2023-12-19T18:24:08.379403+0000","last_peered":"2023-12-19T18:24:08.379403+0000","last_clean":"2023-12-19T18:24:08.379403+0000","last_became_active":"2023-12-19T18:24:08.378520+0000","last_became_peered":"2023-12-19T18:24:08.378520+0000","last_unstale":"2023-12-19T18:24:08.379403+0000","last_undegraded":"2023-12-19T18:24:08.379403+0000","last_fullsized":"2023-12-19T18:24:08.379403+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:01:45.539977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.367608+0000","last_change":"2023-12-19T18:24:07.847424+0000","last_active":"2023-12-19T18:24:08.367608+0000","last_peered":"2023-12-19T18:24:08.367608+0000","last_clean":"2023-12-19T18:24:08.367608+0000","last_became_active":"2023-12-19T18:24:07.845842+0000","last_became_peered":"2023-12-19T18:24:07.845842+0000","last_unstale":"2023-12-19T18:24:08.367608+0000","last_undegraded":"2023-12-19T18:24:08.367608+0000","last_fullsized":"2023-12-19T18:24:08.367608+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:31:41.263967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.371903+0000","last_change":"2023-12-19T18:24:08.371828+0000","last_active":"2023-12-19T18:24:08.371903+0000","last_peered":"2023-12-19T18:24:08.371903+0000","last_clean":"2023-12-19T18:24:08.371903+0000","last_became_active":"2023-12-19T18:24:08.371552+0000","last_became_peered":"2023-12-19T18:24:08.371552+0000","last_unstale":"2023-12-19T18:24:08.371903+0000","last_undegraded":"2023-12-19T18:24:08.371903+0000","last_fullsized":"2023-12-19T18:24:08.371903+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:00:00.467931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.385037+0000","last_change":"2023-12-19T18:24:07.841045+0000","last_active":"2023-12-19T18:24:08.385037+0000","last_peered":"2023-12-19T18:24:08.385037+0000","last_clean":"2023-12-19T18:24:08.385037+0000","last_became_active":"2023-12-19T18:24:07.840408+0000","last_became_peered":"2023-12-19T18:24:07.840408+0000","last_unstale":"2023-12-19T18:24:08.385037+0000","last_undegraded":"2023-12-19T18:24:08.385037+0000","last_fullsized":"2023-12-19T18:24:08.385037+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:12.758668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382543+0000","last_change":"2023-12-19T18:24:08.381772+0000","last_active":"2023-12-19T18:24:08.382543+0000","last_peered":"2023-12-19T18:24:08.382543+0000","last_clean":"2023-12-19T18:24:08.382543+0000","last_became_active":"2023-12-19T18:24:08.377348+0000","last_became_peered":"2023-12-19T18:24:08.377348+0000","last_unstale":"2023-12-19T18:24:08.382543+0000","last_undegraded":"2023-12-19T18:24:08.382543+0000","last_fullsized":"2023-12-19T18:24:08.382543+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:18:53.695825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"169'1","reported_seq":11,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.371134+0000","last_change":"2023-12-19T18:24:07.857366+0000","last_active":"2023-12-19T18:24:08.371134+0000","last_peered":"2023-12-19T18:24:08.371134+0000","last_clean":"2023-12-19T18:24:08.371134+0000","last_became_active":"2023-12-19T18:24:07.856567+0000","last_became_peered":"2023-12-19T18:24:07.856567+0000","last_unstale":"2023-12-19T18:24:08.371134+0000","last_undegraded":"2023-12-19T18:24:08.371134+0000","last_fullsized":"2023-12-19T18:24:08.371134+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:53.645430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381374+0000","last_change":"2023-12-19T18:24:08.376304+0000","last_active":"2023-12-19T18:24:08.381374+0000","last_peered":"2023-12-19T18:24:08.381374+0000","last_clean":"2023-12-19T18:24:08.381374+0000","last_became_active":"2023-12-19T18:24:08.375932+0000","last_became_peered":"2023-12-19T18:24:08.375932+0000","last_unstale":"2023-12-19T18:24:08.381374+0000","last_undegraded":"2023-12-19T18:24:08.381374+0000","last_fullsized":"2023-12-19T18:24:08.381374+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:31:57.093000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.366069+0000","last_change":"2023-12-19T18:24:07.853555+0000","last_active":"2023-12-19T18:24:08.366069+0000","last_peered":"2023-12-19T18:24:08.366069+0000","last_clean":"2023-12-19T18:24:08.366069+0000","last_became_active":"2023-12-19T18:24:07.853122+0000","last_became_peered":"2023-12-19T18:24:07.853122+0000","last_unstale":"2023-12-19T18:24:08.366069+0000","last_undegraded":"2023-12-19T18:24:08.366069+0000","last_fullsized":"2023-12-19T18:24:08.366069+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:40:40.392497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.372432+0000","last_change":"2023-12-19T18:24:08.372024+0000","last_active":"2023-12-19T18:24:08.372432+0000","last_peered":"2023-12-19T18:24:08.372432+0000","last_clean":"2023-12-19T18:24:08.372432+0000","last_became_active":"2023-12-19T18:24:08.371331+0000","last_became_peered":"2023-12-19T18:24:08.371331+0000","last_unstale":"2023-12-19T18:24:08.372432+0000","last_undegraded":"2023-12-19T18:24:08.372432+0000","last_fullsized":"2023-12-19T18:24:08.372432+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:04.052385+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.371315+0000","last_change":"2023-12-19T18:24:07.857978+0000","last_active":"2023-12-19T18:24:08.371315+0000","last_peered":"2023-12-19T18:24:08.371315+0000","last_clean":"2023-12-19T18:24:08.371315+0000","last_became_active":"2023-12-19T18:24:07.856803+0000","last_became_peered":"2023-12-19T18:24:07.856803+0000","last_unstale":"2023-12-19T18:24:08.371315+0000","last_undegraded":"2023-12-19T18:24:08.371315+0000","last_fullsized":"2023-12-19T18:24:08.371315+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:44:36.587232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.358706+0000","last_change":"2023-12-19T18:24:08.358612+0000","last_active":"2023-12-19T18:24:08.358706+0000","last_peered":"2023-12-19T18:24:08.358706+0000","last_clean":"2023-12-19T18:24:08.358706+0000","last_became_active":"2023-12-19T18:24:08.358377+0000","last_became_peered":"2023-12-19T18:24:08.358377+0000","last_unstale":"2023-12-19T18:24:08.358706+0000","last_undegraded":"2023-12-19T18:24:08.358706+0000","last_fullsized":"2023-12-19T18:24:08.358706+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:25:52.888125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.371122+0000","last_change":"2023-12-19T18:24:07.859555+0000","last_active":"2023-12-19T18:24:08.371122+0000","last_peered":"2023-12-19T18:24:08.371122+0000","last_clean":"2023-12-19T18:24:08.371122+0000","last_became_active":"2023-12-19T18:24:07.859155+0000","last_became_peered":"2023-12-19T18:24:07.859155+0000","last_unstale":"2023-12-19T18:24:08.371122+0000","last_undegraded":"2023-12-19T18:24:08.371122+0000","last_fullsized":"2023-12-19T18:24:08.371122+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:11:33.080490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.383670+0000","last_change":"2023-12-19T18:24:08.383592+0000","last_active":"2023-12-19T18:24:08.383670+0000","last_peered":"2023-12-19T18:24:08.383670+0000","last_clean":"2023-12-19T18:24:08.383670+0000","last_became_active":"2023-12-19T18:24:08.383354+0000","last_became_peered":"2023-12-19T18:24:08.383354+0000","last_unstale":"2023-12-19T18:24:08.383670+0000","last_undegraded":"2023-12-19T18:24:08.383670+0000","last_fullsized":"2023-12-19T18:24:08.383670+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:52:36.884988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.376687+0000","last_change":"2023-12-19T18:24:07.851951+0000","last_active":"2023-12-19T18:24:08.376687+0000","last_peered":"2023-12-19T18:24:08.376687+0000","last_clean":"2023-12-19T18:24:08.376687+0000","last_became_active":"2023-12-19T18:24:07.851579+0000","last_became_peered":"2023-12-19T18:24:07.851579+0000","last_unstale":"2023-12-19T18:24:08.376687+0000","last_undegraded":"2023-12-19T18:24:08.376687+0000","last_fullsized":"2023-12-19T18:24:08.376687+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:20:48.492830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381387+0000","last_change":"2023-12-19T18:24:08.381338+0000","last_active":"2023-12-19T18:24:08.381387+0000","last_peered":"2023-12-19T18:24:08.381387+0000","last_clean":"2023-12-19T18:24:08.381387+0000","last_became_active":"2023-12-19T18:24:08.381141+0000","last_became_peered":"2023-12-19T18:24:08.381141+0000","last_unstale":"2023-12-19T18:24:08.381387+0000","last_undegraded":"2023-12-19T18:24:08.381387+0000","last_fullsized":"2023-12-19T18:24:08.381387+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:08:31.093102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.350469+0000","last_change":"2023-12-19T18:24:07.856117+0000","last_active":"2023-12-19T18:24:08.350469+0000","last_peered":"2023-12-19T18:24:08.350469+0000","last_clean":"2023-12-19T18:24:08.350469+0000","last_became_active":"2023-12-19T18:24:07.855943+0000","last_became_peered":"2023-12-19T18:24:07.855943+0000","last_unstale":"2023-12-19T18:24:08.350469+0000","last_undegraded":"2023-12-19T18:24:08.350469+0000","last_fullsized":"2023-12-19T18:24:08.350469+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:05:44.737120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.360046+0000","last_change":"2023-12-19T18:24:08.359967+0000","last_active":"2023-12-19T18:24:08.360046+0000","last_peered":"2023-12-19T18:24:08.360046+0000","last_clean":"2023-12-19T18:24:08.360046+0000","last_became_active":"2023-12-19T18:24:08.359701+0000","last_became_peered":"2023-12-19T18:24:08.359701+0000","last_unstale":"2023-12-19T18:24:08.360046+0000","last_undegraded":"2023-12-19T18:24:08.360046+0000","last_fullsized":"2023-12-19T18:24:08.360046+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:11:00.980095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382006+0000","last_change":"2023-12-19T18:24:07.857582+0000","last_active":"2023-12-19T18:24:08.382006+0000","last_peered":"2023-12-19T18:24:08.382006+0000","last_clean":"2023-12-19T18:24:08.382006+0000","last_became_active":"2023-12-19T18:24:07.857401+0000","last_became_peered":"2023-12-19T18:24:07.857401+0000","last_unstale":"2023-12-19T18:24:08.382006+0000","last_undegraded":"2023-12-19T18:24:08.382006+0000","last_fullsized":"2023-12-19T18:24:08.382006+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:33:23.371165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.370639+0000","last_change":"2023-12-19T18:24:08.370512+0000","last_active":"2023-12-19T18:24:08.370639+0000","last_peered":"2023-12-19T18:24:08.370639+0000","last_clean":"2023-12-19T18:24:08.370639+0000","last_became_active":"2023-12-19T18:24:08.369842+0000","last_became_peered":"2023-12-19T18:24:08.369842+0000","last_unstale":"2023-12-19T18:24:08.370639+0000","last_undegraded":"2023-12-19T18:24:08.370639+0000","last_fullsized":"2023-12-19T18:24:08.370639+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:11:12.380314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.363062+0000","last_change":"2023-12-19T18:24:07.850755+0000","last_active":"2023-12-19T18:24:08.363062+0000","last_peered":"2023-12-19T18:24:08.363062+0000","last_clean":"2023-12-19T18:24:08.363062+0000","last_became_active":"2023-12-19T18:24:07.850250+0000","last_became_peered":"2023-12-19T18:24:07.850250+0000","last_unstale":"2023-12-19T18:24:08.363062+0000","last_undegraded":"2023-12-19T18:24:08.363062+0000","last_fullsized":"2023-12-19T18:24:08.363062+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:14.183993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380036+0000","last_change":"2023-12-19T18:24:08.377781+0000","last_active":"2023-12-19T18:24:08.380036+0000","last_peered":"2023-12-19T18:24:08.380036+0000","last_clean":"2023-12-19T18:24:08.380036+0000","last_became_active":"2023-12-19T18:24:08.377264+0000","last_became_peered":"2023-12-19T18:24:08.377264+0000","last_unstale":"2023-12-19T18:24:08.380036+0000","last_undegraded":"2023-12-19T18:24:08.380036+0000","last_fullsized":"2023-12-19T18:24:08.380036+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:56:06.338402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.371279+0000","last_change":"2023-12-19T18:24:07.847872+0000","last_active":"2023-12-19T18:24:08.371279+0000","last_peered":"2023-12-19T18:24:08.371279+0000","last_clean":"2023-12-19T18:24:08.371279+0000","last_became_active":"2023-12-19T18:24:07.847549+0000","last_became_peered":"2023-12-19T18:24:07.847549+0000","last_unstale":"2023-12-19T18:24:08.371279+0000","last_undegraded":"2023-12-19T18:24:08.371279+0000","last_fullsized":"2023-12-19T18:24:08.371279+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:10:57.945137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.381820+0000","last_change":"2023-12-19T18:24:08.381731+0000","last_active":"2023-12-19T18:24:08.381820+0000","last_peered":"2023-12-19T18:24:08.381820+0000","last_clean":"2023-12-19T18:24:08.381820+0000","last_became_active":"2023-12-19T18:24:08.380797+0000","last_became_peered":"2023-12-19T18:24:08.380797+0000","last_unstale":"2023-12-19T18:24:08.381820+0000","last_undegraded":"2023-12-19T18:24:08.381820+0000","last_fullsized":"2023-12-19T18:24:08.381820+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:04:29.837453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.376826+0000","last_change":"2023-12-19T18:24:07.836260+0000","last_active":"2023-12-19T18:24:08.376826+0000","last_peered":"2023-12-19T18:24:08.376826+0000","last_clean":"2023-12-19T18:24:08.376826+0000","last_became_active":"2023-12-19T18:24:07.835697+0000","last_became_peered":"2023-12-19T18:24:07.835697+0000","last_unstale":"2023-12-19T18:24:08.376826+0000","last_undegraded":"2023-12-19T18:24:08.376826+0000","last_fullsized":"2023-12-19T18:24:08.376826+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:15.350531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.370046+0000","last_change":"2023-12-19T18:24:08.368396+0000","last_active":"2023-12-19T18:24:08.370046+0000","last_peered":"2023-12-19T18:24:08.370046+0000","last_clean":"2023-12-19T18:24:08.370046+0000","last_became_active":"2023-12-19T18:24:08.367761+0000","last_became_peered":"2023-12-19T18:24:08.367761+0000","last_unstale":"2023-12-19T18:24:08.370046+0000","last_undegraded":"2023-12-19T18:24:08.370046+0000","last_fullsized":"2023-12-19T18:24:08.370046+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:16.359936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.365199+0000","last_change":"2023-12-19T18:24:07.845815+0000","last_active":"2023-12-19T18:24:08.365199+0000","last_peered":"2023-12-19T18:24:08.365199+0000","last_clean":"2023-12-19T18:24:08.365199+0000","last_became_active":"2023-12-19T18:24:07.845462+0000","last_became_peered":"2023-12-19T18:24:07.845462+0000","last_unstale":"2023-12-19T18:24:08.365199+0000","last_undegraded":"2023-12-19T18:24:08.365199+0000","last_fullsized":"2023-12-19T18:24:08.365199+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:31:15.350531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.357788+0000","last_change":"2023-12-19T18:24:08.357269+0000","last_active":"2023-12-19T18:24:08.357788+0000","last_peered":"2023-12-19T18:24:08.357788+0000","last_clean":"2023-12-19T18:24:08.357788+0000","last_became_active":"2023-12-19T18:24:08.357021+0000","last_became_peered":"2023-12-19T18:24:08.357021+0000","last_unstale":"2023-12-19T18:24:08.357788+0000","last_undegraded":"2023-12-19T18:24:08.357788+0000","last_fullsized":"2023-12-19T18:24:08.357788+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:45:15.193398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.352810+0000","last_change":"2023-12-19T18:24:07.853773+0000","last_active":"2023-12-19T18:24:08.352810+0000","last_peered":"2023-12-19T18:24:08.352810+0000","last_clean":"2023-12-19T18:24:08.352810+0000","last_became_active":"2023-12-19T18:24:07.852959+0000","last_became_peered":"2023-12-19T18:24:07.852959+0000","last_unstale":"2023-12-19T18:24:08.352810+0000","last_undegraded":"2023-12-19T18:24:08.352810+0000","last_fullsized":"2023-12-19T18:24:08.352810+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:43:34.775884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.383140+0000","last_change":"2023-12-19T18:24:08.382807+0000","last_active":"2023-12-19T18:24:08.383140+0000","last_peered":"2023-12-19T18:24:08.383140+0000","last_clean":"2023-12-19T18:24:08.383140+0000","last_became_active":"2023-12-19T18:24:08.381396+0000","last_became_peered":"2023-12-19T18:24:08.381396+0000","last_unstale":"2023-12-19T18:24:08.383140+0000","last_undegraded":"2023-12-19T18:24:08.383140+0000","last_fullsized":"2023-12-19T18:24:08.383140+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:08:31.093102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.350915+0000","last_change":"2023-12-19T18:24:07.848616+0000","last_active":"2023-12-19T18:24:08.350915+0000","last_peered":"2023-12-19T18:24:08.350915+0000","last_clean":"2023-12-19T18:24:08.350915+0000","last_became_active":"2023-12-19T18:24:07.848119+0000","last_became_peered":"2023-12-19T18:24:07.848119+0000","last_unstale":"2023-12-19T18:24:08.350915+0000","last_undegraded":"2023-12-19T18:24:08.350915+0000","last_fullsized":"2023-12-19T18:24:08.350915+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:32:03.042980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.380067+0000","last_change":"2023-12-19T18:24:08.379786+0000","last_active":"2023-12-19T18:24:08.380067+0000","last_peered":"2023-12-19T18:24:08.380067+0000","last_clean":"2023-12-19T18:24:08.380067+0000","last_became_active":"2023-12-19T18:24:08.376557+0000","last_became_peered":"2023-12-19T18:24:08.376557+0000","last_unstale":"2023-12-19T18:24:08.380067+0000","last_undegraded":"2023-12-19T18:24:08.380067+0000","last_fullsized":"2023-12-19T18:24:08.380067+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:12:08.719580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.368048+0000","last_change":"2023-12-19T18:24:07.839076+0000","last_active":"2023-12-19T18:24:08.368048+0000","last_peered":"2023-12-19T18:24:08.368048+0000","last_clean":"2023-12-19T18:24:08.368048+0000","last_became_active":"2023-12-19T18:24:07.838687+0000","last_became_peered":"2023-12-19T18:24:07.838687+0000","last_unstale":"2023-12-19T18:24:08.368048+0000","last_undegraded":"2023-12-19T18:24:08.368048+0000","last_fullsized":"2023-12-19T18:24:08.368048+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:40:40.392497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.369179+0000","last_change":"2023-12-19T18:24:08.369081+0000","last_active":"2023-12-19T18:24:08.369179+0000","last_peered":"2023-12-19T18:24:08.369179+0000","last_clean":"2023-12-19T18:24:08.369179+0000","last_became_active":"2023-12-19T18:24:08.368867+0000","last_became_peered":"2023-12-19T18:24:08.368867+0000","last_unstale":"2023-12-19T18:24:08.369179+0000","last_undegraded":"2023-12-19T18:24:08.369179+0000","last_fullsized":"2023-12-19T18:24:08.369179+0000","mapping_epoch":168,"log_start":"0'0","ondisk_log_start":"0'0","created":168,"last_epoch_clean":169,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:07.816059+0000","last_clean_scrub_stamp":"2023-12-19T18:24:07.816059+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:53:10.506959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"169'2","reported_seq":12,"reported_epoch":169,"state":"active+clean","last_fresh":"2023-12-19T18:24:08.382564+0000","last_change":"2023-12-19T18:24:07.852598+0000","last_active":"2023-12-19T18:24:08.382564+0000","last_peered":"2023-12-19T18:24:08.382564+0000","last_clean":"2023-12-19T18:24:08.382564+0000","last_became_active":"2023-12-19T18:24:07.851919+0000","last_became_peered":"2023-12-19T18:24:07.851919+0000","last_unstale":"2023-12-19T18:24:08.382564+0000","last_undegraded":"2023-12-19T18:24:08.382564+0000","last_fullsized":"2023-12-19T18:24:08.382564+0000","mapping_epoch":167,"log_start":"0'0","ondisk_log_start":"0'0","created":167,"last_epoch_clean":168,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:24:06.806654+0000","last_clean_scrub_stamp":"2023-12-19T18:24:06.806654+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:01:38.785106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":298,"reported_epoch":168,"state":"active+clean","last_fresh":"2023-12-19T18:24:07.835486+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:24:07.835486+0000","last_peered":"2023-12-19T18:24:07.835486+0000","last_clean":"2023-12-19T18:24:07.835486+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:24:07.835486+0000","last_undegraded":"2023-12-19T18:24:07.835486+0000","last_fullsized":"2023-12-19T18:24:07.835486+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":929,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194382,"statfs":{"total":200000000,"available":199047974,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":1.2110000000000001,"15min":1.149},"min":{"1min":0.59299999999999997,"5min":0.57899999999999996,"15min":0.45900000000000002},"max":{"1min":1.488,"5min":5.6879999999999997,"15min":5.6879999999999997},"last":0.95799999999999996},{"interface":"front","average":{"1min":1.169,"5min":1.264,"15min":1.198},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.55800000000000005},"max":{"1min":1.587,"5min":1.7330000000000001,"15min":1.79},"last":0.79600000000000004}]},{"osd":1,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":1.206,"15min":1.2190000000000001},"min":{"1min":0.65800000000000003,"5min":0.63600000000000001,"15min":0.54400000000000004},"max":{"1min":1.5940000000000001,"5min":5.9029999999999996,"15min":10.926},"last":0.73199999999999998},{"interface":"front","average":{"1min":1.256,"5min":1.258,"15min":1.25},"min":{"1min":0.80700000000000005,"5min":0.71899999999999997,"15min":0.67900000000000005},"max":{"1min":1.736,"5min":1.736,"15min":10.904999999999999},"last":1.1990000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.1240000000000001,"15min":1.097},"min":{"1min":0.66000000000000003,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.444,"5min":3.1960000000000002,"15min":3.1960000000000002},"last":1.016},{"interface":"front","average":{"1min":1.173,"5min":1.2450000000000001,"15min":1.1910000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.6799999999999999,"5min":6.008,"15min":6.008},"last":1.069}]},{"osd":3,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.206,"5min":1.278,"15min":1.1759999999999999},"min":{"1min":0.89800000000000002,"5min":0.66900000000000004,"15min":0.61399999999999999},"max":{"1min":1.643,"5min":3.0379999999999998,"15min":3.0379999999999998},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.284,"5min":1.401,"15min":1.3029999999999999},"min":{"1min":0.77500000000000002,"5min":0.77300000000000002,"15min":0.69599999999999995},"max":{"1min":1.671,"5min":5.8120000000000003,"15min":5.8120000000000003},"last":1.1699999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3160000000000001,"5min":1.274,"15min":1.1499999999999999},"min":{"1min":0.74299999999999999,"5min":0.55700000000000005,"15min":0.47899999999999998},"max":{"1min":1.847,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.2949999999999999,"5min":1.2909999999999999,"15min":1.246},"min":{"1min":0.66200000000000003,"5min":0.59299999999999997,"15min":0.57599999999999996},"max":{"1min":1.7450000000000001,"5min":3.3180000000000001,"15min":3.3180000000000001},"last":1.3300000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.3660000000000001,"15min":1.331},"min":{"1min":0.68999999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.8140000000000001,"5min":7.6429999999999998,"15min":7.6429999999999998},"last":1.2569999999999999},{"interface":"front","average":{"1min":1.399,"5min":1.4370000000000001,"15min":1.347},"min":{"1min":0.627,"5min":0.627,"15min":0.53500000000000003},"max":{"1min":1.7969999999999999,"5min":6.0839999999999996,"15min":6.0839999999999996},"last":1.3839999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":1.516,"5min":1.548,"15min":1.4790000000000001},"min":{"1min":0.72499999999999998,"5min":0.628,"15min":0.59899999999999998},"max":{"1min":1.8959999999999999,"5min":3.1309999999999998,"15min":3.1309999999999998},"last":1.6519999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.468,"15min":1.3899999999999999},"min":{"1min":0.79900000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.929,"5min":4.1479999999999997,"15min":4.1479999999999997},"last":1.7809999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574963,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":929,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194382,"statfs":{"total":200000000,"available":199048008,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1040000000000001,"15min":1.028},"min":{"1min":0.75800000000000001,"5min":0.68899999999999995,"15min":0.65800000000000003},"max":{"1min":1.522,"5min":1.6259999999999999,"15min":1.8129999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.175,"5min":1.1339999999999999,"15min":1.0860000000000001},"min":{"1min":0.73399999999999999,"5min":0.64300000000000002,"15min":0.55600000000000005},"max":{"1min":1.6000000000000001,"5min":1.748,"15min":3.5529999999999999},"last":1.01}]},{"osd":1,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.1279999999999999,"15min":1.127},"min":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68200000000000005},"max":{"1min":1.3759999999999999,"5min":1.5609999999999999,"15min":4.5540000000000003},"last":0.94199999999999995},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.1899999999999999,"15min":1.1619999999999999},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.60299999999999998},"max":{"1min":1.5960000000000001,"5min":1.885,"15min":1.885},"last":1.2190000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.2110000000000001,"15min":1.159},"min":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.626},"max":{"1min":1.4319999999999999,"5min":1.7330000000000001,"15min":1.821},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.1379999999999999,"5min":1.1830000000000001,"15min":1.218},"min":{"1min":0.79800000000000004,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.573,"5min":1.944,"15min":3.7909999999999999},"last":1.1499999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.2709999999999999,"15min":1.214},"min":{"1min":0.95699999999999996,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.641,"5min":1.722,"15min":3.8820000000000001},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.145,"5min":1.1839999999999999,"15min":1.171},"min":{"1min":0.79200000000000004,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.4319999999999999,"5min":1.6559999999999999,"15min":4.6189999999999998},"last":1.306}]},{"osd":4,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.28,"15min":1.2549999999999999},"min":{"1min":0.81799999999999995,"5min":0.74099999999999999,"15min":0.65600000000000003},"max":{"1min":1.6519999999999999,"5min":1.9670000000000001,"15min":3.0680000000000001},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.4710000000000001,"5min":1.4159999999999999,"15min":1.365},"min":{"1min":1.2210000000000001,"5min":0.70799999999999996,"15min":0.57999999999999996},"max":{"1min":1.746,"5min":2.0430000000000001,"15min":3.7000000000000002},"last":1.5009999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2529999999999999,"15min":1.204},"min":{"1min":0.89500000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.6579999999999999,"5min":2.2400000000000002,"15min":4.6680000000000001},"last":1.421},{"interface":"front","average":{"1min":1.3500000000000001,"5min":1.419,"15min":1.3640000000000001},"min":{"1min":1.018,"5min":0.626,"15min":0.626},"max":{"1min":1.597,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.794}]},{"osd":7,"last update":"Tue Dec 19 18:23:56 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.409,"15min":1.361},"min":{"1min":1.1659999999999999,"5min":0.72299999999999998,"15min":0.61899999999999999},"max":{"1min":1.698,"5min":2.133,"15min":3.9670000000000001},"last":1.841},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.4139999999999999,"15min":1.371},"min":{"1min":1.0049999999999999,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.7430000000000001,"5min":2.3519999999999999,"15min":5.6639999999999997},"last":1.607}]}]},{"osd":5,"up_from":13,"seq":55834574963,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1493,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193819,"statfs":{"total":200000000,"available":198470690,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.181,"5min":1.196,"15min":1.1679999999999999},"min":{"1min":0.69999999999999996,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.6739999999999999,"5min":1.8540000000000001,"15min":1.863},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.2729999999999999,"15min":1.2010000000000001},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.478,"5min":1.913,"15min":1.913},"last":1.292}]},{"osd":1,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.212,"15min":1.0900000000000001},"min":{"1min":0.71999999999999997,"5min":0.71599999999999997,"15min":0.65700000000000003},"max":{"1min":1.5249999999999999,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":1.5009999999999999},{"interface":"front","average":{"1min":1.1299999999999999,"5min":1.1759999999999999,"15min":1.0880000000000001},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.546,"5min":1.8300000000000001,"15min":1.843},"last":0.99399999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.155,"15min":1.101},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.59499999999999997},"max":{"1min":1.536,"5min":1.5580000000000001,"15min":1.794},"last":1.077},{"interface":"front","average":{"1min":1.278,"5min":1.26,"15min":1.1719999999999999},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8,"5min":1.8,"15min":2.036},"last":1.407}]},{"osd":3,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.1990000000000001,"15min":1.1140000000000001},"min":{"1min":0.879,"5min":0.71999999999999997,"15min":0.68899999999999995},"max":{"1min":1.635,"5min":1.72,"15min":1.7450000000000001},"last":1.351},{"interface":"front","average":{"1min":1.286,"5min":1.3340000000000001,"15min":1.252},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.619,"5min":2.0590000000000002,"15min":2.0590000000000002},"last":1.6000000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3740000000000001,"5min":1.3149999999999999,"15min":1.216},"min":{"1min":0.626,"5min":0.626,"15min":0.46700000000000003},"max":{"1min":1.7250000000000001,"5min":1.944,"15min":2.0329999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.216,"15min":1.149},"min":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.57099999999999995},"max":{"1min":1.5449999999999999,"5min":1.7689999999999999,"15min":1.901},"last":0.83899999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4279999999999999,"5min":1.46,"15min":1.369},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.7729999999999999,"5min":2.0329999999999999,"15min":2.1589999999999998},"last":1.5529999999999999},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.347,"15min":1.2669999999999999},"min":{"1min":0.84699999999999998,"5min":0.81999999999999995,"15min":0.49299999999999999},"max":{"1min":1.6419999999999999,"5min":1.702,"15min":1.976},"last":1.107}]},{"osd":7,"last update":"Tue Dec 19 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3200000000000001,"15min":1.25},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.50900000000000001},"max":{"1min":1.6839999999999999,"5min":1.881,"15min":2.0270000000000001},"last":0.88800000000000001},{"interface":"front","average":{"1min":1.4610000000000001,"5min":1.4850000000000001,"15min":1.427},"min":{"1min":0.96199999999999997,"5min":0.95099999999999996,"15min":0.623},"max":{"1min":1.732,"5min":2.117,"15min":2.117},"last":1.4399999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574963,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":916,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194395,"statfs":{"total":200000000,"available":199061058,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0900000000000001,"5min":1.0349999999999999,"15min":1.006},"min":{"1min":0.78300000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.972,"5min":1.972,"15min":1.972},"last":1.0640000000000001},{"interface":"front","average":{"1min":1.1679999999999999,"5min":1.083,"15min":1.0509999999999999},"min":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"max":{"1min":1.7330000000000001,"5min":1.7330000000000001,"15min":1.7909999999999999},"last":0.71199999999999997}]},{"osd":1,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.159,"15min":1.1499999999999999},"min":{"1min":0.745,"5min":0.67200000000000004,"15min":0.66900000000000004},"max":{"1min":1.8400000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.2110000000000001},{"interface":"front","average":{"1min":1.238,"5min":1.2509999999999999,"15min":1.1919999999999999},"min":{"1min":0.92300000000000004,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.7450000000000001,"5min":1.875,"15min":1.875},"last":1.355}]},{"osd":2,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1419999999999999,"5min":1.1459999999999999,"15min":1.1240000000000001},"min":{"1min":0.86799999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.7010000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.165,"5min":1.1739999999999999,"15min":1.1160000000000001},"min":{"1min":0.747,"5min":0.747,"15min":0.70499999999999996},"max":{"1min":2.0270000000000001,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.93100000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.2949999999999999,"15min":1.2669999999999999},"min":{"1min":0.85499999999999998,"5min":0.76300000000000001,"15min":0.76300000000000001},"max":{"1min":1.534,"5min":1.946,"15min":1.946},"last":0.875},{"interface":"front","average":{"1min":1.429,"5min":1.3759999999999999,"15min":1.3169999999999999},"min":{"1min":1.0029999999999999,"5min":0.83399999999999996,"15min":0.74399999999999999},"max":{"1min":1.927,"5min":1.927,"15min":1.927},"last":1.002}]},{"osd":5,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.292,"15min":1.2190000000000001},"min":{"1min":0.84499999999999997,"5min":0.82299999999999995,"15min":0.44500000000000001},"max":{"1min":1.593,"5min":1.8129999999999999,"15min":1.8740000000000001},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.359,"5min":1.3919999999999999,"15min":1.3560000000000001},"min":{"1min":0.79800000000000004,"5min":0.61899999999999999,"15min":0.504},"max":{"1min":1.7150000000000001,"5min":1.8540000000000001,"15min":1.8540000000000001},"last":1.2649999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.3120000000000001,"15min":1.304},"min":{"1min":0.85999999999999999,"5min":0.76700000000000002,"15min":0.432},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.45,"15min":1.4019999999999999},"min":{"1min":1.0580000000000001,"5min":0.90800000000000003,"15min":0.64500000000000002},"max":{"1min":1.7749999999999999,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.4950000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.373,"15min":1.3500000000000001},"min":{"1min":0.95099999999999996,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":1.8200000000000001,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.466,"5min":1.45,"15min":1.415},"min":{"1min":1.1519999999999999,"5min":0.88500000000000001,"15min":0.57399999999999995},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.3149999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574964,"num_pgs":16,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":923,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194389,"statfs":{"total":200000000,"available":199054783,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.91900000000000004,"15min":0.94999999999999996},"min":{"1min":0.435,"5min":0.38100000000000001,"15min":0.35299999999999998},"max":{"1min":1.3939999999999999,"5min":3.3140000000000001,"15min":3.4350000000000001},"last":1.3380000000000001},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.99399999999999999,"15min":0.95299999999999996},"min":{"1min":0.54600000000000004,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.9159999999999999,"5min":4.3490000000000002,"15min":4.3490000000000002},"last":1.268}]},{"osd":1,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.087,"5min":1.0429999999999999,"15min":1.05},"min":{"1min":0.624,"5min":0.439,"15min":0.41299999999999998},"max":{"1min":1.619,"5min":5.2510000000000003,"15min":5.2510000000000003},"last":1.133},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.143,"15min":1.1080000000000001},"min":{"1min":0.64400000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.494,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.0980000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.25,"5min":1.2210000000000001,"15min":1.244},"min":{"1min":0.72499999999999998,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.7350000000000001,"5min":2.5369999999999999,"15min":13.833},"last":1.028},{"interface":"front","average":{"1min":1.292,"5min":1.381,"15min":1.3180000000000001},"min":{"1min":0.83799999999999997,"5min":0.60299999999999998,"15min":0.53200000000000003},"max":{"1min":1.7709999999999999,"5min":5.4480000000000004,"15min":16.748999999999999},"last":0.93100000000000005}]},{"osd":4,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.2150000000000001,"5min":1.321,"15min":1.244},"min":{"1min":0.85499999999999998,"5min":0.77400000000000002,"15min":0.65600000000000003},"max":{"1min":1.4970000000000001,"5min":5.1790000000000003,"15min":5.1790000000000003},"last":1.6060000000000001},{"interface":"front","average":{"1min":1.4410000000000001,"5min":1.4199999999999999,"15min":1.409},"min":{"1min":0.97399999999999998,"5min":0.84599999999999997,"15min":0.73199999999999998},"max":{"1min":1.97,"5min":4.569,"15min":4.569},"last":1.3879999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.337,"5min":1.4399999999999999,"15min":1.4359999999999999},"min":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.76000000000000001},"max":{"1min":1.679,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":1.8720000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.371,"15min":1.296},"min":{"1min":0.95899999999999996,"5min":0.80600000000000005,"15min":0.69999999999999996},"max":{"1min":1.804,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.4399999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.4650000000000001,"5min":1.4550000000000001,"15min":1.4670000000000001},"min":{"1min":0.86499999999999999,"5min":0.81799999999999995,"15min":0.81599999999999995},"max":{"1min":2.0550000000000002,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.4470000000000001,"5min":1.5269999999999999,"15min":1.508},"min":{"1min":0.999,"5min":0.83199999999999996,"15min":0.80500000000000005},"max":{"1min":1.7270000000000001,"5min":5.3739999999999997,"15min":5.3739999999999997},"last":1.9830000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:23:54 2023","interfaces":[{"interface":"back","average":{"1min":1.446,"5min":1.482,"15min":1.4019999999999999},"min":{"1min":1.0580000000000001,"5min":0.78500000000000003,"15min":0.73199999999999998},"max":{"1min":1.6830000000000001,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.5189999999999999},{"interface":"front","average":{"1min":1.587,"5min":1.5740000000000001,"15min":1.5549999999999999},"min":{"1min":1.1020000000000001,"5min":0.85399999999999998,"15min":0.85399999999999998},"max":{"1min":2.113,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":1.575}]}]},{"osd":2,"up_from":13,"seq":55834574963,"num_pgs":15,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":930,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194382,"statfs":{"total":200000000,"available":199047336,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.92800000000000005,"15min":0.998},"min":{"1min":0.68200000000000005,"5min":0.35299999999999998,"15min":0.34399999999999997},"max":{"1min":1.506,"5min":2.879,"15min":9.2550000000000008},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.98999999999999999,"15min":1.0229999999999999},"min":{"1min":0.47499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.776,"5min":2.387,"15min":4.7850000000000001},"last":1.379}]},{"osd":1,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":0.93400000000000005,"5min":0.96599999999999997,"15min":1.1120000000000001},"min":{"1min":0.76000000000000001,"5min":0.47599999999999998,"15min":0.46000000000000002},"max":{"1min":1.1910000000000001,"5min":2.4390000000000001,"15min":8.2720000000000002},"last":1.079},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.1020000000000001,"15min":1.141},"min":{"1min":0.622,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.6279999999999999,"5min":2.7200000000000002,"15min":5.0620000000000003},"last":1.214}]},{"osd":3,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":1.151,"5min":1.1479999999999999,"15min":1.304},"min":{"1min":0.81599999999999995,"5min":0.56000000000000005,"15min":0.51900000000000002},"max":{"1min":1.524,"5min":2.238,"15min":8.3109999999999999},"last":1.5649999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.1279999999999999,"15min":1.274},"min":{"1min":0.61699999999999999,"5min":0.55500000000000005,"15min":0.41799999999999998},"max":{"1min":2.3999999999999999,"5min":2.3999999999999999,"15min":8.0690000000000008},"last":0.628}]},{"osd":4,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.387,"15min":1.4359999999999999},"min":{"1min":0.88100000000000001,"5min":0.83399999999999996,"15min":0.79300000000000004},"max":{"1min":2.2160000000000002,"5min":2.6139999999999999,"15min":5.3929999999999998},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.4159999999999999,"5min":1.456,"15min":1.5149999999999999},"min":{"1min":0.90100000000000002,"5min":0.82099999999999995,"15min":0.78300000000000003},"max":{"1min":1.8720000000000001,"5min":3.1070000000000002,"15min":7.3570000000000002},"last":1.6240000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3600000000000001,"5min":1.3799999999999999,"15min":1.502},"min":{"1min":1.0820000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.0019999999999998,"5min":2.5579999999999998,"15min":8.1839999999999993},"last":1.323},{"interface":"front","average":{"1min":1.353,"5min":1.399,"15min":1.4039999999999999},"min":{"1min":0.89200000000000002,"5min":0.82299999999999995,"15min":0.72199999999999998},"max":{"1min":1.7849999999999999,"5min":2.4940000000000002,"15min":5.165},"last":1.044}]},{"osd":6,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":1.4810000000000001,"5min":1.4379999999999999,"15min":1.4430000000000001},"min":{"1min":0.94899999999999995,"5min":0.82399999999999995,"15min":0.72999999999999998},"max":{"1min":2.2570000000000001,"5min":3.0179999999999998,"15min":5.1059999999999999},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.5009999999999999,"5min":1.4790000000000001,"15min":1.536},"min":{"1min":1.0529999999999999,"5min":0.93300000000000005,"15min":0.77000000000000002},"max":{"1min":2.1539999999999999,"5min":2.9580000000000002,"15min":8.1370000000000005},"last":1.4330000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:24:00 2023","interfaces":[{"interface":"back","average":{"1min":1.603,"5min":1.5109999999999999,"15min":1.5349999999999999},"min":{"1min":0.95099999999999996,"5min":0.91100000000000003,"15min":0.78200000000000003},"max":{"1min":2.367,"5min":3.2189999999999999,"15min":5.1449999999999996},"last":1.7210000000000001},{"interface":"front","average":{"1min":1.6439999999999999,"5min":1.5589999999999999,"15min":1.6279999999999999},"min":{"1min":1.232,"5min":0.88100000000000001,"15min":0.86499999999999999},"max":{"1min":2.5790000000000002,"5min":4.8289999999999997,"15min":8.2300000000000004},"last":1.665}]}]},{"osd":1,"up_from":13,"seq":55834574964,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1499,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193812,"statfs":{"total":200000000,"available":198464415,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.86299999999999999,"15min":0.84699999999999998},"min":{"1min":0.41199999999999998,"5min":0.38,"15min":0.34699999999999998},"max":{"1min":1.804,"5min":1.804,"15min":1.804},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.92200000000000004,"5min":0.95499999999999996,"15min":0.94899999999999995},"min":{"1min":0.434,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.768,"5min":1.7929999999999999,"15min":1.7929999999999999},"last":1.2430000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":1.0409999999999999,"15min":1.046},"min":{"1min":0.57499999999999996,"5min":0.42299999999999999,"15min":0.41399999999999998},"max":{"1min":1.5129999999999999,"5min":2.4590000000000001,"15min":2.5739999999999998},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.96399999999999997,"5min":0.99399999999999999,"15min":0.93799999999999994},"min":{"1min":0.39900000000000002,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":1.6839999999999999,"5min":1.716,"15min":1.845},"last":1.379}]},{"osd":3,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.226,"5min":1.288,"15min":1.179},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.6759999999999999,"5min":2.137,"15min":2.137},"last":2.089},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.204,"15min":1.1819999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.44900000000000001},"max":{"1min":1.8360000000000001,"5min":2.1320000000000001,"15min":2.3959999999999999},"last":1.135}]},{"osd":4,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.288,"5min":1.3460000000000001,"15min":1.3},"min":{"1min":0.84099999999999997,"5min":0.72199999999999998,"15min":0.69099999999999995},"max":{"1min":1.917,"5min":2.444,"15min":2.444},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.3089999999999999,"5min":1.383,"15min":1.3120000000000001},"min":{"1min":0.92500000000000004,"5min":0.79600000000000004,"15min":0.71099999999999997},"max":{"1min":1.8660000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.6180000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.355,"5min":1.391,"15min":1.3120000000000001},"min":{"1min":0.76700000000000002,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.998,"5min":2.5459999999999998,"15min":2.5459999999999998},"last":1.8560000000000001},{"interface":"front","average":{"1min":1.349,"5min":1.4590000000000001,"15min":1.4159999999999999},"min":{"1min":0.96199999999999997,"5min":0.95399999999999996,"15min":0.70499999999999996},"max":{"1min":2.141,"5min":2.6179999999999999,"15min":2.6179999999999999},"last":1.302}]},{"osd":6,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.335,"5min":1.411,"15min":1.379},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.9119999999999999,"5min":2.7000000000000002,"15min":2.7000000000000002},"last":2.016},{"interface":"front","average":{"1min":1.329,"5min":1.4850000000000001,"15min":1.45},"min":{"1min":0.98499999999999999,"5min":0.91400000000000003,"15min":0.76400000000000001},"max":{"1min":2.2509999999999999,"5min":2.895,"15min":2.895},"last":1.5209999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:23:59 2023","interfaces":[{"interface":"back","average":{"1min":1.3069999999999999,"5min":1.4350000000000001,"15min":1.397},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":2.1629999999999998,"5min":2.5670000000000002,"15min":9.4830000000000005},"last":1.98},{"interface":"front","average":{"1min":1.462,"5min":1.5209999999999999,"15min":1.504},"min":{"1min":0.92400000000000004,"5min":0.874,"15min":0.71899999999999997},"max":{"1min":2.0720000000000001,"5min":2.456,"15min":9.4109999999999996},"last":1.9359999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574963,"num_pgs":5,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":923,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194389,"statfs":{"total":200000000,"available":199054783,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.92900000000000005,"15min":0.89700000000000002},"min":{"1min":0.34499999999999997,"5min":0.311,"15min":0.311},"max":{"1min":1.355,"5min":2.3199999999999998,"15min":2.3199999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":1,"5min":1.073,"15min":1.034},"min":{"1min":0.48699999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.6459999999999999,"5min":2.0369999999999999,"15min":2.4180000000000001},"last":0.60999999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.99299999999999999,"15min":0.95799999999999996},"min":{"1min":0.52900000000000003,"5min":0.434,"15min":0.434},"max":{"1min":1.4550000000000001,"5min":1.536,"15min":1.7450000000000001},"last":0.998},{"interface":"front","average":{"1min":0.97299999999999998,"5min":1.0089999999999999,"15min":0.94699999999999995},"min":{"1min":0.441,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.5700000000000001,"5min":2.0950000000000002,"15min":2.7810000000000001},"last":0.53700000000000003}]},{"osd":3,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":1.0209999999999999,"15min":1.006},"min":{"1min":0.44,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.8069999999999999,"5min":2.5190000000000001,"15min":2.5190000000000001},"last":0.748},{"interface":"front","average":{"1min":1.151,"5min":1.2529999999999999,"15min":1.2230000000000001},"min":{"1min":0.66100000000000003,"5min":0.53000000000000003,"15min":0.47499999999999998},"max":{"1min":1.9159999999999999,"5min":1.982,"15min":2.0089999999999999},"last":0.85099999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2030000000000001,"5min":1.2949999999999999,"15min":1.2190000000000001},"min":{"1min":0.67600000000000005,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.49,"5min":2.5529999999999999,"15min":2.5529999999999999},"last":1.165},{"interface":"front","average":{"1min":1.234,"5min":1.274,"15min":1.278},"min":{"1min":0.70399999999999996,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":2.056,"5min":2.2450000000000001,"15min":2.9529999999999998},"last":0.89000000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.3540000000000001,"15min":1.3919999999999999},"min":{"1min":0.74099999999999999,"5min":0.68500000000000005,"15min":0.64500000000000002},"max":{"1min":1.887,"5min":2.3940000000000001,"15min":16.298999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.23,"5min":1.339,"15min":1.286},"min":{"1min":0.80800000000000005,"5min":0.72799999999999998,"15min":0.72199999999999998},"max":{"1min":1.6060000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.91300000000000003}]},{"osd":6,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.4570000000000001,"15min":1.4019999999999999},"min":{"1min":1.044,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.706,"5min":2.2559999999999998,"15min":2.5910000000000002},"last":1.044},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.399,"15min":1.3680000000000001},"min":{"1min":0.89100000000000001,"5min":0.84999999999999998,"15min":0.77000000000000002},"max":{"1min":1.8819999999999999,"5min":2.4359999999999999,"15min":2.46},"last":1.1040000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:24:06 2023","interfaces":[{"interface":"back","average":{"1min":1.456,"5min":1.468,"15min":1.4099999999999999},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.70899999999999996},"max":{"1min":2.0230000000000001,"5min":2.4969999999999999,"15min":2.4969999999999999},"last":1.218},{"interface":"front","average":{"1min":1.3,"5min":1.4139999999999999,"15min":1.413},"min":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"max":{"1min":1.9370000000000001,"5min":2.4580000000000002,"15min":3.1429999999999998},"last":1.0189999999999999}]}]}],"pool_statfs":[]}} 2023-12-19T18:24:13.902 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:24:13.902 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:24:14.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.187+0000 7fd9a6220640 1 Processor -- start 2023-12-19T18:24:14.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.187+0000 7fd9a6220640 1 -- start start 2023-12-19T18:24:14.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.189+0000 7fd9a6220640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a015ae60 0x7fd9a0070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.189+0000 7fd9a6220640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd9a00cc870 0x7fd9a0071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.189+0000 7fd9a6220640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a00b03f0 con 0x7fd9a00ad9d0 2023-12-19T18:24:14.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.189+0000 7fd9a6220640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a00b0c90 con 0x7fd9a00cc870 2023-12-19T18:24:14.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.189+0000 7fd9a6220640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a0071550 con 0x7fd9a015ae60 2023-12-19T18:24:14.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.190+0000 7fd9a5a1f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd9a00ad9d0 0x7fd9a015e9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40440/0 (socket says 172.21.15.27:40440) 2023-12-19T18:24:14.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.190+0000 7fd9a5a1f640 1 -- 172.21.15.27:0/31196819 learned_addr learned my addr 172.21.15.27:0/31196819 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:14.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.190+0000 7fd9a521e640 1 --2- 172.21.15.27:0/31196819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a015ae60 0x7fd9a0070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.190+0000 7fd9a4a1d640 1 --2- 172.21.15.27:0/31196819 >> v2:172.21.15.192:3300/0 conn(0x7fd9a00cc870 0x7fd9a0071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.190+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1467708288 0 0) 0x7fd9a00b03f0 con 0x7fd9a00ad9d0 2023-12-19T18:24:14.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.190+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd98c003520 con 0x7fd9a00ad9d0 2023-12-19T18:24:14.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1394828345 0 0) 0x7fd98c003520 con 0x7fd9a00ad9d0 2023-12-19T18:24:14.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a015ae60 msgr2=0x7fd9a0070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:14.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 --2- 172.21.15.27:0/31196819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a015ae60 0x7fd9a0070ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 >> v2:172.21.15.192:3300/0 conn(0x7fd9a00cc870 msgr2=0x7fd9a0071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:14.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 --2- 172.21.15.27:0/31196819 >> v2:172.21.15.192:3300/0 conn(0x7fd9a00cc870 0x7fd9a0071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a00b4740 con 0x7fd9a00ad9d0 2023-12-19T18:24:14.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fd998001670 con 0x7fd9a00ad9d0 2023-12-19T18:24:14.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 >> v1:172.21.15.27:6789/0 conn(0x7fd9a00ad9d0 legacy=0x7fd9a015e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:14.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 --2- 172.21.15.27:0/31196819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd98c003970 0x7fd98c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 --2- 172.21.15.27:0/31196819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd98c0047c0 0x7fd98c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 --2- 172.21.15.27:0/31196819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a015ae60 0x7fd98c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a00b0c90 con 0x7fd98c0047c0 2023-12-19T18:24:14.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a0071550 con 0x7fd98c003970 2023-12-19T18:24:14.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd98c00ba60 con 0x7fd9a015ae60 2023-12-19T18:24:14.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.191+0000 7fd9a4a1d640 1 --2- 172.21.15.27:0/31196819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd98c0047c0 0x7fd98c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.192+0000 7fd9a521e640 1 --2- 172.21.15.27:0/31196819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd98c003970 0x7fd98c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.192+0000 7fd9a5a1f640 1 --2- 172.21.15.27:0/31196819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a015ae60 0x7fd98c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.192+0000 7fd9a4a1d640 1 -- 172.21.15.27:0/31196819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a015ae60 msgr2=0x7fd98c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:14.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.192+0000 7fd9a4a1d640 1 --2- 172.21.15.27:0/31196819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a015ae60 0x7fd98c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.192+0000 7fd9a4a1d640 1 -- 172.21.15.27:0/31196819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd98c003970 msgr2=0x7fd98c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:14.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.192+0000 7fd9a4a1d640 1 --2- 172.21.15.27:0/31196819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd98c003970 0x7fd98c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.192+0000 7fd9a4a1d640 1 -- 172.21.15.27:0/31196819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c003520 con 0x7fd98c0047c0 2023-12-19T18:24:14.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.192+0000 7fd9a4a1d640 1 --2- 172.21.15.27:0/31196819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd98c0047c0 0x7fd98c004c40 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00ccc50 tx=0x7fd98c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a1b75a4394c206 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:14.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.192+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99002c680 con 0x7fd98c0047c0 2023-12-19T18:24:14.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.192+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99002c7e0 con 0x7fd98c0047c0 2023-12-19T18:24:14.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.192+0000 7fd997fff640 1 -- 172.21.15.27:0/31196819 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99002cb70 con 0x7fd98c0047c0 2023-12-19T18:24:14.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.193+0000 7fd9a6220640 1 -- 172.21.15.27:0/31196819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd98c0047c0 msgr2=0x7fd98c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:14.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.193+0000 7fd9a6220640 1 --2- 172.21.15.27:0/31196819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd98c0047c0 0x7fd98c004c40 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fd9a00ccc50 tx=0x7fd98c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.193+0000 7fd9a5a1f640 1 -- 172.21.15.27:0/31196819 reap_dead start 2023-12-19T18:24:14.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.193+0000 7fd9a6220640 1 -- 172.21.15.27:0/31196819 shutdown_connections 2023-12-19T18:24:14.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.193+0000 7fd9a6220640 1 -- 172.21.15.27:0/31196819 >> 172.21.15.27:0/31196819 conn(0x7fd9a00cb3f0 msgr2=0x7fd9a00cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:14.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.193+0000 7fd9a6220640 1 -- 172.21.15.27:0/31196819 shutdown_connections 2023-12-19T18:24:14.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.193+0000 7fd9a6220640 1 -- 172.21.15.27:0/31196819 wait complete. 2023-12-19T18:24:14.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.195+0000 7fd9a6220640 1 Processor -- start 2023-12-19T18:24:14.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.195+0000 7fd9a6220640 1 -- start start 2023-12-19T18:24:14.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.196+0000 7fd9a6220640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd98c0047c0 0x7fd9a015e360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.196+0000 7fd9a6220640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a015b480 0x7fd9a00c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.196+0000 7fd9a6220640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9a00caa30 0x7fd9a00ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.196+0000 7fd9a6220640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a015b900 con 0x7fd98c0047c0 2023-12-19T18:24:14.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.196+0000 7fd9a6220640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a00c7f90 con 0x7fd9a00caa30 2023-12-19T18:24:14.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.196+0000 7fd9a6220640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a00c8270 con 0x7fd9a015b480 2023-12-19T18:24:14.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.196+0000 7fd9a521e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd98c0047c0 0x7fd9a015e360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.196+0000 7fd9a521e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd98c0047c0 0x7fd9a015e360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60124/0 (socket says 172.21.15.27:60124) 2023-12-19T18:24:14.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.196+0000 7fd9a521e640 1 -- 172.21.15.27:0/3541385108 learned_addr learned my addr 172.21.15.27:0/3541385108 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:14.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.197+0000 7fd9a5a1f640 1 --2- 172.21.15.27:0/3541385108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9a00caa30 0x7fd9a00ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.197+0000 7fd9a521e640 1 -- 172.21.15.27:0/3541385108 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a015b480 msgr2=0x7fd9a00c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:14.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.197+0000 7fd9a521e640 1 --2- 172.21.15.27:0/3541385108 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a015b480 0x7fd9a00c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.197+0000 7fd9a521e640 1 -- 172.21.15.27:0/3541385108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9a00caa30 msgr2=0x7fd9a00ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:14.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.197+0000 7fd9a521e640 1 --2- 172.21.15.27:0/3541385108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9a00caa30 0x7fd9a00ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.197+0000 7fd9a521e640 1 -- 172.21.15.27:0/3541385108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a00c8550 con 0x7fd98c0047c0 2023-12-19T18:24:14.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.197+0000 7fd9a521e640 1 --2- 172.21.15.27:0/3541385108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd98c0047c0 0x7fd9a015e360 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd99c007f30 tx=0x7fd99c011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:14.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.197+0000 7fd995ffb640 1 -- 172.21.15.27:0/3541385108 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c00e030 con 0x7fd98c0047c0 2023-12-19T18:24:14.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.197+0000 7fd995ffb640 1 -- 172.21.15.27:0/3541385108 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd99c016470 con 0x7fd98c0047c0 2023-12-19T18:24:14.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.198+0000 7fd9a6220640 1 -- 172.21.15.27:0/3541385108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a00c8830 con 0x7fd98c0047c0 2023-12-19T18:24:14.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.198+0000 7fd995ffb640 1 -- 172.21.15.27:0/3541385108 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd99c016820 con 0x7fd98c0047c0 2023-12-19T18:24:14.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.198+0000 7fd9a6220640 1 -- 172.21.15.27:0/3541385108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a00caee0 con 0x7fd98c0047c0 2023-12-19T18:24:14.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.199+0000 7fd9a6220640 1 -- 172.21.15.27:0/3541385108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a004e840 con 0x7fd98c0047c0 2023-12-19T18:24:14.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.200+0000 7fd995ffb640 1 -- 172.21.15.27:0/3541385108 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd99c016980 con 0x7fd98c0047c0 2023-12-19T18:24:14.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.201+0000 7fd995ffb640 1 --2- 172.21.15.27:0/3541385108 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd98406a460 0x7fd98406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.201+0000 7fd995ffb640 1 -- 172.21.15.27:0/3541385108 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd99c00f070 con 0x7fd98c0047c0 2023-12-19T18:24:14.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.201+0000 7fd9a4a1d640 1 --2- 172.21.15.27:0/3541385108 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd98406a460 0x7fd98406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.202+0000 7fd9a4a1d640 1 --2- 172.21.15.27:0/3541385108 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd98406a460 0x7fd98406c920 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd990036000 tx=0x7fd99002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:14.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.205+0000 7fd995ffb640 1 -- 172.21.15.27:0/3541385108 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd99c054de0 con 0x7fd98c0047c0 2023-12-19T18:24:14.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.476+0000 7fd9a6220640 1 -- 172.21.15.27:0/3541385108 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd9a015a4a0 con 0x7fd98c0047c0 2023-12-19T18:24:14.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.485+0000 7fd995ffb640 1 -- 172.21.15.27:0/3541385108 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v19) v1 ==== 190+0+0 (secure 0 0 0) 0x7fd99c058400 con 0x7fd98c0047c0 2023-12-19T18:24:14.485 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2023-12-19T18:24:14.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.488+0000 7fd9a6220640 1 -- 172.21.15.27:0/3541385108 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd98406a460 msgr2=0x7fd98406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:14.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.488+0000 7fd9a6220640 1 --2- 172.21.15.27:0/3541385108 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd98406a460 0x7fd98406c920 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fd990036000 tx=0x7fd99002e000 comp rx=0 tx=0).stop 2023-12-19T18:24:14.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.488+0000 7fd9a6220640 1 -- 172.21.15.27:0/3541385108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd98c0047c0 msgr2=0x7fd9a015e360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:14.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.488+0000 7fd9a6220640 1 --2- 172.21.15.27:0/3541385108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd98c0047c0 0x7fd9a015e360 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fd99c007f30 tx=0x7fd99c011660 comp rx=0 tx=0).stop 2023-12-19T18:24:14.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.488+0000 7fd9a6220640 1 -- 172.21.15.27:0/3541385108 shutdown_connections 2023-12-19T18:24:14.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.488+0000 7fd9a6220640 1 --2- 172.21.15.27:0/3541385108 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd98406a460 0x7fd98406c920 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.488+0000 7fd9a6220640 1 --2- 172.21.15.27:0/3541385108 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd9a00caa30 0x7fd9a00ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.488+0000 7fd9a6220640 1 --2- 172.21.15.27:0/3541385108 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd9a015b480 0x7fd9a00c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.488+0000 7fd9a6220640 1 --2- 172.21.15.27:0/3541385108 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd98c0047c0 0x7fd9a015e360 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.488+0000 7fd9a6220640 1 -- 172.21.15.27:0/3541385108 >> 172.21.15.27:0/3541385108 conn(0x7fd9a00cb3f0 msgr2=0x7fd9a00cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:14.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.488+0000 7fd9a6220640 1 -- 172.21.15.27:0/3541385108 shutdown_connections 2023-12-19T18:24:14.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.488+0000 7fd9a6220640 1 -- 172.21.15.27:0/3541385108 wait complete. 2023-12-19T18:24:14.506 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:24:14.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.725+0000 7f3680a16640 1 Processor -- start 2023-12-19T18:24:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.725+0000 7f3680a16640 1 -- start start 2023-12-19T18:24:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.727+0000 7f3680a16640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f367c1604d0 0x7f367c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.727+0000 7f3680a16640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f367c0d2350 0x7f367c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.727+0000 7f3680a16640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f367c0b5e90 con 0x7f367c0d0300 2023-12-19T18:24:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.727+0000 7f3680a16640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f367c0d2b30 con 0x7f367c0d2350 2023-12-19T18:24:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.727+0000 7f3680a16640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367c0d2850 con 0x7f367c1604d0 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.727+0000 7f3673fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f367c0d2350 0x7f367c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.727+0000 7f3673fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f367c0d2350 0x7f367c0b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60140/0 (socket says 172.21.15.27:60140) 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.727+0000 7f3673fff640 1 -- 172.21.15.27:0/3774588990 learned_addr learned my addr 172.21.15.27:0/3774588990 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.728+0000 7f367ad9d640 1 --2- 172.21.15.27:0/3774588990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f367c1604d0 0x7f367c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.728+0000 7f3673fff640 1 -- 172.21.15.27:0/3774588990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f367c1604d0 msgr2=0x7f367c0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.728+0000 7f3673fff640 1 --2- 172.21.15.27:0/3774588990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f367c1604d0 0x7f367c0b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.728+0000 7f3673fff640 1 -- 172.21.15.27:0/3774588990 >> v1:172.21.15.192:6789/0 conn(0x7f367c0d0300 legacy=0x7f367c0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.728+0000 7f3673fff640 1 -- 172.21.15.27:0/3774588990 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c0cded0 con 0x7f367c0d2350 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.729+0000 7f3673fff640 1 --2- 172.21.15.27:0/3774588990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f367c0d2350 0x7f367c0b4d20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3664009870 tx=0x7f366402b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3b854a4a6a6611 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.730+0000 7f367a59c640 1 -- 172.21.15.27:0/3774588990 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664030840 con 0x7f367c0d2350 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.730+0000 7f367a59c640 1 -- 172.21.15.27:0/3774588990 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36640309a0 con 0x7f367c0d2350 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.730+0000 7f367a59c640 1 -- 172.21.15.27:0/3774588990 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3664030d30 con 0x7f367c0d2350 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.730+0000 7f3680a16640 1 -- 172.21.15.27:0/3774588990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f367c0d2350 msgr2=0x7f367c0b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.730+0000 7f3680a16640 1 --2- 172.21.15.27:0/3774588990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f367c0d2350 0x7f367c0b4d20 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f3664009870 tx=0x7f366402b980 comp rx=0 tx=0).stop 2023-12-19T18:24:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.730+0000 7f3680a16640 1 -- 172.21.15.27:0/3774588990 shutdown_connections 2023-12-19T18:24:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.730+0000 7f3680a16640 1 --2- 172.21.15.27:0/3774588990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f367c0d2350 0x7f367c0b4d20 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.730+0000 7f3680a16640 1 --2- 172.21.15.27:0/3774588990 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f367c1604d0 0x7f367c0b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.730+0000 7f3680a16640 1 -- 172.21.15.27:0/3774588990 >> 172.21.15.27:0/3774588990 conn(0x7f367c0cf7e0 msgr2=0x7f367c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.731+0000 7f3680a16640 1 -- 172.21.15.27:0/3774588990 shutdown_connections 2023-12-19T18:24:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.731+0000 7f3680a16640 1 -- 172.21.15.27:0/3774588990 wait complete. 2023-12-19T18:24:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.732+0000 7f3680a16640 1 Processor -- start 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.733+0000 7f3680a16640 1 -- start start 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f3680a16640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f367c0d0300 0x7f367c0b81e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f3680a16640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f367c0d2350 0x7f367c0c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f3680a16640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f367c1604d0 0x7f367c1f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f3680a16640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c1f2c90 con 0x7f367c1604d0 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f3680a16640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c0c6cc0 con 0x7f367c0d0300 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f3680a16640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367c0c6fa0 con 0x7f367c0d2350 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f367b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f367c1604d0 0x7f367c1f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f367b59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f367c1604d0 0x7f367c1f62e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60154/0 (socket says 172.21.15.27:60154) 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f367b59e640 1 -- 172.21.15.27:0/4027629617 learned_addr learned my addr 172.21.15.27:0/4027629617 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f3673fff640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f367c0d2350 0x7f367c0c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f367ad9d640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f367c0d0300 0x7f367c0b81e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f367b59e640 1 -- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f367c0d2350 msgr2=0x7f367c0c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f367b59e640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f367c0d2350 0x7f367c0c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f367b59e640 1 -- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f367c0d0300 msgr2=0x7f367c0b81e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f367b59e640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f367c0d0300 0x7f367c0b81e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.734+0000 7f367b59e640 1 -- 172.21.15.27:0/4027629617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36640094d0 con 0x7f367c1604d0 2023-12-19T18:24:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.735+0000 7f367b59e640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f367c1604d0 0x7f367c1f62e0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f367400c6b0 tx=0x7f367400cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.735+0000 7f36737fe640 1 -- 172.21.15.27:0/4027629617 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674003b70 con 0x7f367c1604d0 2023-12-19T18:24:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.735+0000 7f36737fe640 1 -- 172.21.15.27:0/4027629617 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3674003cf0 con 0x7f367c1604d0 2023-12-19T18:24:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.735+0000 7f3680a16640 1 -- 172.21.15.27:0/4027629617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367c0c7280 con 0x7f367c1604d0 2023-12-19T18:24:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.735+0000 7f36737fe640 1 -- 172.21.15.27:0/4027629617 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674003ea0 con 0x7f367c1604d0 2023-12-19T18:24:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.735+0000 7f3680a16640 1 -- 172.21.15.27:0/4027629617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367c159a00 con 0x7f367c1604d0 2023-12-19T18:24:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.736+0000 7f3680a16640 1 -- 172.21.15.27:0/4027629617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f367c066f80 con 0x7f367c1604d0 2023-12-19T18:24:14.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.737+0000 7f36737fe640 1 -- 172.21.15.27:0/4027629617 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3674005310 con 0x7f367c1604d0 2023-12-19T18:24:14.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.738+0000 7f36737fe640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f365c06a430 0x7f365c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:14.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.739+0000 7f36737fe640 1 -- 172.21.15.27:0/4027629617 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36740886b0 con 0x7f367c1604d0 2023-12-19T18:24:14.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.739+0000 7f367ad9d640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f365c06a430 0x7f365c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:14.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.740+0000 7f367ad9d640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f365c06a430 0x7f365c06c8f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f366c009ad0 tx=0x7f366c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:14.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.743+0000 7f36737fe640 1 -- 172.21.15.27:0/4027629617 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3674053ca0 con 0x7f367c1604d0 2023-12-19T18:24:14.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.977+0000 7f3680a16640 1 -- 172.21.15.27:0/4027629617 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f367c04e840 con 0x7f367c1604d0 2023-12-19T18:24:14.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.978+0000 7f36737fe640 1 -- 172.21.15.27:0/4027629617 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f36740572c0 con 0x7f367c1604d0 2023-12-19T18:24:14.979 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:24:14.979 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":69,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3291476389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3291476389},{"type":"v1","addr":"172.21.15.192:6836","nonce":3291476389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3526330256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3526330256},{"type":"v1","addr":"172.21.15.192:6837","nonce":3526330256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/899234003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899234003},{"type":"v1","addr":"172.21.15.27:6835","nonce":899234003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:24:08.343302+0000","modified":"2023-12-19T18:24:11.863775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8247":{"gid":8247,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6833/2519815591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2519815591},{"type":"v1","addr":"172.21.15.27:6833","nonce":2519815591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-19T18:24:14.979 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 69 2023-12-19T18:24:14.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.981+0000 7f3680a16640 1 -- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f365c06a430 msgr2=0x7f365c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:14.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.981+0000 7f3680a16640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f365c06a430 0x7f365c06c8f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f366c009ad0 tx=0x7f366c008040 comp rx=0 tx=0).stop 2023-12-19T18:24:14.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.982+0000 7f3680a16640 1 -- 172.21.15.27:0/4027629617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f367c1604d0 msgr2=0x7f367c1f62e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:14.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.982+0000 7f3680a16640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f367c1604d0 0x7f367c1f62e0 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f367400c6b0 tx=0x7f367400cb70 comp rx=0 tx=0).stop 2023-12-19T18:24:14.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.982+0000 7f3680a16640 1 -- 172.21.15.27:0/4027629617 shutdown_connections 2023-12-19T18:24:14.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.982+0000 7f3680a16640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f365c06a430 0x7f365c06c8f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.982+0000 7f3680a16640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f367c1604d0 0x7f367c1f62e0 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.982+0000 7f3680a16640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f367c0d2350 0x7f367c0c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.982+0000 7f3680a16640 1 --2- 172.21.15.27:0/4027629617 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f367c0d0300 0x7f367c0b81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:14.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.982+0000 7f3680a16640 1 -- 172.21.15.27:0/4027629617 >> 172.21.15.27:0/4027629617 conn(0x7f367c0cf7e0 msgr2=0x7f367c0d13a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:14.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.982+0000 7f3680a16640 1 -- 172.21.15.27:0/4027629617 shutdown_connections 2023-12-19T18:24:14.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:14.982+0000 7f3680a16640 1 -- 172.21.15.27:0/4027629617 wait complete. 2023-12-19T18:24:15.404 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 69, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:24:08.343302+0000', 'modified': '2023-12-19T18:24:11.863775+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8247}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8247': {'gid': 8247, 'name': 'a', 'rank': 0, 'incarnation': 65, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6833/2519815591', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6832', 'nonce': 2519815591}, {'type': 'v1', 'addr': '172.21.15.27:6833', 'nonce': 2519815591}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:24:15.404 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:24:15.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-19T18:24:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.653+0000 7f7807fff640 1 Processor -- start 2023-12-19T18:24:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.653+0000 7f7807fff640 1 -- start start 2023-12-19T18:24:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.655+0000 7f7807fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f780815ac50 0x7f7808070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.655+0000 7f7807fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78080adf10 con 0x7f78080af360 2023-12-19T18:24:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.655+0000 7f7807fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78080adac0 con 0x7f78080aef20 2023-12-19T18:24:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.655+0000 7f7807fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78080ae160 con 0x7f780815ac50 2023-12-19T18:24:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.655+0000 7f7806ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f780815ac50 0x7f7808070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.655+0000 7f78077fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f78080af360 0x7f780806f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40446/0 (socket says 172.21.15.27:40446) 2023-12-19T18:24:15.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.655+0000 7f78077fe640 1 -- 172.21.15.27:0/1980171306 learned_addr learned my addr 172.21.15.27:0/1980171306 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:15.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.656+0000 7f7806ffd640 1 -- 172.21.15.27:0/1980171306 >> v1:172.21.15.192:6789/0 conn(0x7f78080aef20 legacy=0x7f780806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:15.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.656+0000 7f7806ffd640 1 -- 172.21.15.27:0/1980171306 >> v1:172.21.15.27:6789/0 conn(0x7f78080af360 legacy=0x7f780806f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:15.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.656+0000 7f7806ffd640 1 -- 172.21.15.27:0/1980171306 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78080ae440 con 0x7f780815ac50 2023-12-19T18:24:15.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.656+0000 7f7805ffb640 1 -- 172.21.15.27:0/1980171306 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3327587572 0 0) 0x7f78080adf10 con 0x7f78080af360 2023-12-19T18:24:15.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.657+0000 7f7806ffd640 1 --2- 172.21.15.27:0/1980171306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f780815ac50 0x7f7808070fb0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f77f80097a0 tx=0x7f77f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfc4cd4a63f3ed0e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:15.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.657+0000 7f7805ffb640 1 -- 172.21.15.27:0/1980171306 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f802e070 con 0x7f780815ac50 2023-12-19T18:24:15.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.658+0000 7f7805ffb640 1 -- 172.21.15.27:0/1980171306 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77f80315a0 con 0x7f780815ac50 2023-12-19T18:24:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.658+0000 7f7805ffb640 1 -- 172.21.15.27:0/1980171306 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f8031930 con 0x7f780815ac50 2023-12-19T18:24:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.658+0000 7f7807fff640 1 -- 172.21.15.27:0/1980171306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f780815ac50 msgr2=0x7f7808070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.658+0000 7f7807fff640 1 --2- 172.21.15.27:0/1980171306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f780815ac50 0x7f7808070fb0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f77f80097a0 tx=0x7f77f802b840 comp rx=0 tx=0).stop 2023-12-19T18:24:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.658+0000 7f7807fff640 1 -- 172.21.15.27:0/1980171306 shutdown_connections 2023-12-19T18:24:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.658+0000 7f7807fff640 1 --2- 172.21.15.27:0/1980171306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f780815ac50 0x7f7808070fb0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.658+0000 7f7807fff640 1 -- 172.21.15.27:0/1980171306 >> 172.21.15.27:0/1980171306 conn(0x7f780806a7d0 msgr2=0x7f78080ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.658+0000 7f7807fff640 1 -- 172.21.15.27:0/1980171306 shutdown_connections 2023-12-19T18:24:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.658+0000 7f7807fff640 1 -- 172.21.15.27:0/1980171306 wait complete. 2023-12-19T18:24:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.660+0000 7f7807fff640 1 Processor -- start 2023-12-19T18:24:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.660+0000 7f7807fff640 1 -- start start 2023-12-19T18:24:15.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.661+0000 7f7807fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78080aef20 0x7f780813b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.661+0000 7f7807fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78080af360 0x7f7808156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.661+0000 7f7807fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f780815ac50 0x7f7808157170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.661+0000 7f7807fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780813c760 con 0x7f78080af360 2023-12-19T18:24:15.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.661+0000 7f7807fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780813ca40 con 0x7f780815ac50 2023-12-19T18:24:15.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.661+0000 7f7807fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f780813cd20 con 0x7f78080aef20 2023-12-19T18:24:15.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.661+0000 7f78067fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78080af360 0x7f7808156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.661+0000 7f78067fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78080af360 0x7f7808156c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60168/0 (socket says 172.21.15.27:60168) 2023-12-19T18:24:15.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.661+0000 7f78067fc640 1 -- 172.21.15.27:0/2395381676 learned_addr learned my addr 172.21.15.27:0/2395381676 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:15.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.662+0000 7f78077fe640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f780815ac50 0x7f7808157170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.662+0000 7f7806ffd640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78080aef20 0x7f780813b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.662+0000 7f78067fc640 1 -- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78080aef20 msgr2=0x7f780813b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:15.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.662+0000 7f78067fc640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78080aef20 0x7f780813b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.662+0000 7f78067fc640 1 -- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f780815ac50 msgr2=0x7f7808157170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:15.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.662+0000 7f78067fc640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f780815ac50 0x7f7808157170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.662+0000 7f78067fc640 1 -- 172.21.15.27:0/2395381676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f8009400 con 0x7f78080af360 2023-12-19T18:24:15.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.662+0000 7f78067fc640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78080af360 0x7f7808156c30 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c400 tx=0x7f77fc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:15.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.662+0000 7f77ef7fe640 1 -- 172.21.15.27:0/2395381676 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0052d0 con 0x7f78080af360 2023-12-19T18:24:15.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.662+0000 7f77ef7fe640 1 -- 172.21.15.27:0/2395381676 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc005430 con 0x7f78080af360 2023-12-19T18:24:15.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.663+0000 7f77ef7fe640 1 -- 172.21.15.27:0/2395381676 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc003830 con 0x7f78080af360 2023-12-19T18:24:15.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.663+0000 7f7807fff640 1 -- 172.21.15.27:0/2395381676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780813d000 con 0x7f78080af360 2023-12-19T18:24:15.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.663+0000 7f7807fff640 1 -- 172.21.15.27:0/2395381676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780813b9f0 con 0x7f78080af360 2023-12-19T18:24:15.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.665+0000 7f77ef7fe640 1 -- 172.21.15.27:0/2395381676 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f77fc003990 con 0x7f78080af360 2023-12-19T18:24:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.666+0000 7f77ef7fe640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f77f406a460 0x7f77f406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.666+0000 7f77ef7fe640 1 -- 172.21.15.27:0/2395381676 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77fc087f50 con 0x7f78080af360 2023-12-19T18:24:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.666+0000 7f780fe4b740 1 Processor -- start 2023-12-19T18:24:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.667+0000 7f7806ffd640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f77f406a460 0x7f77f406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.667+0000 7f780fe4b740 1 -- start start 2023-12-19T18:24:15.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.667+0000 7f780fe4b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 0x557356eb6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f780fe4b740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed2960 0x557356eb7460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f780fe4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb4550 0x557356e8ac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f780fe4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356e62650 con 0x557356eb4550 2023-12-19T18:24:15.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f780fe4b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356df7000 con 0x557356ed4e40 2023-12-19T18:24:15.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f780fe4b740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x557356df4790 con 0x557356ed2960 2023-12-19T18:24:15.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f78077fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb4550 0x557356e8ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f7806ffd640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f77f406a460 0x7f77f406c920 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f77f8036000 tx=0x7f77f8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:15.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f78077fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb4550 0x557356e8ac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60172/0 (socket says 172.21.15.27:60172) 2023-12-19T18:24:15.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f78077fe640 1 -- 172.21.15.27:0/1853187927 learned_addr learned my addr 172.21.15.27:0/1853187927 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:15.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f7806ffd640 1 --2- 172.21.15.27:0/1853187927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 0x557356eb6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f78067fc640 1 --2- 172.21.15.27:0/1853187927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed2960 0x557356eb7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f78077fe640 1 -- 172.21.15.27:0/1853187927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed2960 msgr2=0x557356eb7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:15.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f78077fe640 1 --2- 172.21.15.27:0/1853187927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed2960 0x557356eb7460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f78077fe640 1 -- 172.21.15.27:0/1853187927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 msgr2=0x557356eb6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:15.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f78077fe640 1 --2- 172.21.15.27:0/1853187927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 0x557356eb6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.668+0000 7f78077fe640 1 -- 172.21.15.27:0/1853187927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557356db7710 con 0x557356eb4550 2023-12-19T18:24:15.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.669+0000 7f78077fe640 1 --2- 172.21.15.27:0/1853187927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb4550 0x557356e8ac90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f7800001da0 tx=0x7f780000d150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2082bb2a4b913cc9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:15.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.669+0000 7f77ecff9640 1 -- 172.21.15.27:0/1853187927 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780000d980 con 0x557356eb4550 2023-12-19T18:24:15.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.669+0000 7f77ecff9640 1 -- 172.21.15.27:0/1853187927 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780000dae0 con 0x557356eb4550 2023-12-19T18:24:15.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.669+0000 7f77ecff9640 1 -- 172.21.15.27:0/1853187927 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f780000de50 con 0x557356eb4550 2023-12-19T18:24:15.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.670+0000 7f780fe4b740 1 -- 172.21.15.27:0/1853187927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb4550 msgr2=0x557356e8ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:15.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.670+0000 7f780fe4b740 1 --2- 172.21.15.27:0/1853187927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb4550 0x557356e8ac90 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f7800001da0 tx=0x7f780000d150 comp rx=0 tx=0).stop 2023-12-19T18:24:15.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.670+0000 7f780fe4b740 1 -- 172.21.15.27:0/1853187927 shutdown_connections 2023-12-19T18:24:15.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.670+0000 7f780fe4b740 1 --2- 172.21.15.27:0/1853187927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb4550 0x557356e8ac90 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.670+0000 7f780fe4b740 1 --2- 172.21.15.27:0/1853187927 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed2960 0x557356eb7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.670+0000 7f780fe4b740 1 --2- 172.21.15.27:0/1853187927 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 0x557356eb6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.670+0000 7f780fe4b740 1 -- 172.21.15.27:0/1853187927 >> 172.21.15.27:0/1853187927 conn(0x557356e893a0 msgr2=0x557356ed1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.670+0000 7f780fe4b740 1 -- 172.21.15.27:0/1853187927 shutdown_connections 2023-12-19T18:24:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.670+0000 7f780fe4b740 1 -- 172.21.15.27:0/1853187927 wait complete. 2023-12-19T18:24:15.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.671+0000 7f780fe4b740 1 -- start start 2023-12-19T18:24:15.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.671+0000 7f780fe4b740 1 Processor -- start 2023-12-19T18:24:15.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.672+0000 7f780fe4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed2960 0x557356ed4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.672+0000 7f780fe4b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 0x557356eb0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.672+0000 7f780fe4b740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed4840 0x557356eb14a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.672+0000 7f780fe4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356df0080 con 0x557356ed2960 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.672+0000 7f780fe4b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356d91850 con 0x557356ed4e40 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.672+0000 7f780fe4b740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x557356d914f0 con 0x557356ed4840 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.672+0000 7f780fe4b740 10 client.0 _ll_register_callbacks cb 0x557356de99c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.672+0000 7f7806ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed2960 0x557356ed4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.672+0000 7f7806ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed2960 0x557356ed4300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60186/0 (socket says 172.21.15.27:60186) 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.672+0000 7f7806ffd640 1 -- 172.21.15.27:0/718764674 learned_addr learned my addr 172.21.15.27:0/718764674 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.673+0000 7f78067fc640 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 0x557356eb0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.673+0000 7f78077fe640 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed4840 0x557356eb14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.673+0000 7f7806ffd640 1 -- 172.21.15.27:0/718764674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed4840 msgr2=0x557356eb14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.673+0000 7f7806ffd640 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed4840 0x557356eb14a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.673+0000 7f7806ffd640 1 -- 172.21.15.27:0/718764674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 msgr2=0x557356eb0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.673+0000 7f7806ffd640 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 0x557356eb0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.673+0000 7f7806ffd640 1 -- 172.21.15.27:0/718764674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78000019d0 con 0x557356ed2960 2023-12-19T18:24:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.673+0000 7f78077fe640 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed4840 0x557356eb14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.673+0000 7f7806ffd640 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed2960 0x557356ed4300 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006eb0 tx=0x7f77f8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a581a1ddad4fd97c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.673+0000 7f77e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:24:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.673+0000 7f77e5ffb640 1 -- 172.21.15.27:0/718764674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f8040070 con 0x557356ed2960 2023-12-19T18:24:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.673+0000 7f780fe4b740 1 -- 172.21.15.27:0/718764674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557356e1a5d0 con 0x557356ed2960 2023-12-19T18:24:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.673+0000 7f77e5ffb640 1 -- 172.21.15.27:0/718764674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77f8032cc0 con 0x557356ed2960 2023-12-19T18:24:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.674+0000 7f77e5ffb640 1 -- 172.21.15.27:0/718764674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77f8032e80 con 0x557356ed2960 2023-12-19T18:24:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.674+0000 7f780fe4b740 10 client.8373 fetch_fsmap learned FSMap version 69 2023-12-19T18:24:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.674+0000 7f780fe4b740 1 -- 172.21.15.27:0/718764674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x557356dee700 con 0x557356ed2960 2023-12-19T18:24:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.674+0000 7f77e5ffb640 1 -- 172.21.15.27:0/718764674 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557356e1a5d0 con 0x557356ed2960 2023-12-19T18:24:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.674+0000 7f77e5ffb640 1 -- 172.21.15.27:0/718764674 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 69) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f77f8046410 con 0x557356ed2960 2023-12-19T18:24:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.674+0000 7f780fe4b740 10 client.8373 fetch_fsmap finished waiting for FSMap version 69 2023-12-19T18:24:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.674+0000 7f780fe4b740 10 client.8373 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T18:24:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.674+0000 7f780fe4b740 20 client.8373 populate_metadata read hostname 'smithi027' 2023-12-19T18:24:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.674+0000 7f780fe4b740 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.27:6832/2519815591,v1:172.21.15.27:6833/2519815591] conn(0x557356e99f70 0x557356e9aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.674+0000 7f780fe4b740 4 client.8373 mds_command: new command op to 8247 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:24:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.674+0000 7f780fe4b740 1 -- 172.21.15.27:0/718764674 --> [v2:172.21.15.27:6832/2519815591,v1:172.21.15.27:6833/2519815591] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557356de28d0 con 0x557356e99f70 2023-12-19T18:24:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.675+0000 7f78067fc640 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.27:6832/2519815591,v1:172.21.15.27:6833/2519815591] conn(0x557356e99f70 0x557356e9aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.675+0000 7f78067fc640 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.27:6832/2519815591,v1:172.21.15.27:6833/2519815591] conn(0x557356e99f70 0x557356e9aea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e5f05b70764fec3 server_cookie=5fb87d7a938b9f57 in_seq=0 out_seq=0 2023-12-19T18:24:15.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.675+0000 7f77e5ffb640 10 client.8373 ms_handle_connect on v2:172.21.15.27:6832/2519815591 2023-12-19T18:24:15.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f77e5ffb640 1 -- 172.21.15.27:0/718764674 <== mds.0 v2:172.21.15.27:6832/2519815591 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557356de28d0 con 0x557356e99f70 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f77e5ffb640 10 client.8373 handle_command_reply: tid=0 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f780fe4b740 1 client.8373 shutdown 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f780fe4b740 1 -- 172.21.15.27:0/718764674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed2960 msgr2=0x557356ed4300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f780fe4b740 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed2960 0x557356ed4300 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006eb0 tx=0x7f77f8004730 comp rx=0 tx=0).stop 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f780fe4b740 1 -- 172.21.15.27:0/718764674 shutdown_connections 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f780fe4b740 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.27:6832/2519815591,v1:172.21.15.27:6833/2519815591] conn(0x557356e99f70 0x557356e9aea0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f780fe4b740 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed4840 0x557356eb14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f780fe4b740 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 0x557356eb0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f780fe4b740 1 --2- 172.21.15.27:0/718764674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed2960 0x557356ed4300 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f77e5ffb640 0 client.8373 ms_handle_reset on v2:172.21.15.27:6832/2519815591 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f780fe4b740 1 -- 172.21.15.27:0/718764674 >> 172.21.15.27:0/718764674 conn(0x557356eb4550 msgr2=0x557356e868c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f780fe4b740 1 -- 172.21.15.27:0/718764674 shutdown_connections 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.676+0000 7f780fe4b740 1 -- 172.21.15.27:0/718764674 wait complete. 2023-12-19T18:24:15.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.677+0000 7f780fe4b740 20 client.8373 trim_cache size 0 max 16384 2023-12-19T18:24:15.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.693+0000 7f780fe4b740 1 Processor -- start 2023-12-19T18:24:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.694+0000 7f780fe4b740 1 -- start start 2023-12-19T18:24:15.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.694+0000 7f780fe4b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed2960 0x557356ead030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.694+0000 7f780fe4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed4e40 0x557356eb0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.694+0000 7f780fe4b740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356eb05d0 0x557356eef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.694+0000 7f780fe4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356b57f20 con 0x557356ed4e40 2023-12-19T18:24:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.694+0000 7f780fe4b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356eb1880 con 0x557356ed2960 2023-12-19T18:24:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.695+0000 7f780fe4b740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x557356eacd00 con 0x557356eb05d0 2023-12-19T18:24:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.695+0000 7f78067fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed4e40 0x557356eb0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.695+0000 7f78067fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed4e40 0x557356eb0090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60200/0 (socket says 172.21.15.27:60200) 2023-12-19T18:24:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.695+0000 7f78067fc640 1 -- 172.21.15.27:0/2844498227 learned_addr learned my addr 172.21.15.27:0/2844498227 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.695+0000 7f78067fc640 1 -- 172.21.15.27:0/2844498227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356eb05d0 msgr2=0x557356eef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.695+0000 7f78067fc640 1 --2- 172.21.15.27:0/2844498227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356eb05d0 0x557356eef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.695+0000 7f78067fc640 1 -- 172.21.15.27:0/2844498227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed2960 msgr2=0x557356ead030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.695+0000 7f7806ffd640 1 --2- 172.21.15.27:0/2844498227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed2960 0x557356ead030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.695+0000 7f78067fc640 1 --2- 172.21.15.27:0/2844498227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed2960 0x557356ead030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.695+0000 7f78067fc640 1 -- 172.21.15.27:0/2844498227 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557356e24280 con 0x557356ed4e40 2023-12-19T18:24:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.696+0000 7f78067fc640 1 --2- 172.21.15.27:0/2844498227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed4e40 0x557356eb0090 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f77fc000f30 tx=0x7f77fc05d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=268df605c9ee98ef server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.696+0000 7f77ed7fa640 1 -- 172.21.15.27:0/2844498227 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc0864f0 con 0x557356ed4e40 2023-12-19T18:24:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.696+0000 7f77ed7fa640 1 -- 172.21.15.27:0/2844498227 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77fc06e070 con 0x557356ed4e40 2023-12-19T18:24:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.696+0000 7f77ed7fa640 1 -- 172.21.15.27:0/2844498227 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77fc053870 con 0x557356ed4e40 2023-12-19T18:24:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.696+0000 7f780fe4b740 1 -- 172.21.15.27:0/2844498227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed4e40 msgr2=0x557356eb0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.696+0000 7f780fe4b740 1 --2- 172.21.15.27:0/2844498227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed4e40 0x557356eb0090 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f77fc000f30 tx=0x7f77fc05d040 comp rx=0 tx=0).stop 2023-12-19T18:24:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.696+0000 7f780fe4b740 1 -- 172.21.15.27:0/2844498227 shutdown_connections 2023-12-19T18:24:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.696+0000 7f780fe4b740 1 --2- 172.21.15.27:0/2844498227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356eb05d0 0x557356eef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.696+0000 7f780fe4b740 1 --2- 172.21.15.27:0/2844498227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356ed4e40 0x557356eb0090 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.696+0000 7f780fe4b740 1 --2- 172.21.15.27:0/2844498227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed2960 0x557356ead030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.696+0000 7f780fe4b740 1 -- 172.21.15.27:0/2844498227 >> 172.21.15.27:0/2844498227 conn(0x557356eb4550 msgr2=0x557356e88050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.697+0000 7f780fe4b740 1 -- 172.21.15.27:0/2844498227 shutdown_connections 2023-12-19T18:24:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.697+0000 7f780fe4b740 1 -- 172.21.15.27:0/2844498227 wait complete. 2023-12-19T18:24:15.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.697+0000 7f780fe4b740 1 -- start start 2023-12-19T18:24:15.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.698+0000 7f780fe4b740 1 Processor -- start 2023-12-19T18:24:15.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f780fe4b740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb05d0 0x557356eb1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f780fe4b740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed2960 0x557356eef2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f780fe4b740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 0x557356eef830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f780fe4b740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356e8c720 con 0x557356eb05d0 2023-12-19T18:24:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f780fe4b740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557356f3da00 con 0x557356ed4e40 2023-12-19T18:24:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f780fe4b740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x557356ed1c60 con 0x557356ed2960 2023-12-19T18:24:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f780fe4b740 10 client.0 _ll_register_callbacks cb 0x557356eb4210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:24:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f7806ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb05d0 0x557356eb1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f7806ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb05d0 0x557356eb1320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60206/0 (socket says 172.21.15.27:60206) 2023-12-19T18:24:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f7806ffd640 1 -- 172.21.15.27:0/2064710890 learned_addr learned my addr 172.21.15.27:0/2064710890 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f7806ffd640 1 -- 172.21.15.27:0/2064710890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed2960 msgr2=0x557356eef2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f7806ffd640 1 --2- 172.21.15.27:0/2064710890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed2960 0x557356eef2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f7806ffd640 1 -- 172.21.15.27:0/2064710890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 msgr2=0x557356eef830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f7806ffd640 1 --2- 172.21.15.27:0/2064710890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 0x557356eef830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.699+0000 7f7806ffd640 1 -- 172.21.15.27:0/2064710890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780813b9f0 con 0x557356eb05d0 2023-12-19T18:24:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.700+0000 7f7806ffd640 1 --2- 172.21.15.27:0/2064710890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb05d0 0x557356eb1320 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006ca0 tx=0x7f77f8009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caa6b785e41dff18 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.700+0000 7f77e5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:24:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.700+0000 7f77e5ffb640 1 -- 172.21.15.27:0/2064710890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77f803b5f0 con 0x557356eb05d0 2023-12-19T18:24:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.700+0000 7f77e5ffb640 1 -- 172.21.15.27:0/2064710890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f77f8046920 con 0x557356eb05d0 2023-12-19T18:24:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.700+0000 7f780fe4b740 1 -- 172.21.15.27:0/2064710890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f780000dae0 con 0x557356eb05d0 2023-12-19T18:24:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.700+0000 7f77e5ffb640 1 -- 172.21.15.27:0/2064710890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77f803b750 con 0x557356eb05d0 2023-12-19T18:24:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.700+0000 7f780fe4b740 10 client.8379 fetch_fsmap learned FSMap version 69 2023-12-19T18:24:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.700+0000 7f780fe4b740 1 -- 172.21.15.27:0/2064710890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=2+}) v3 -- 0x557356d90a60 con 0x557356eb05d0 2023-12-19T18:24:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.700+0000 7f77e5ffb640 1 -- 172.21.15.27:0/2064710890 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f780000dae0 con 0x557356eb05d0 2023-12-19T18:24:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.701+0000 7f77e5ffb640 1 -- 172.21.15.27:0/2064710890 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 69) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f77f8058070 con 0x557356eb05d0 2023-12-19T18:24:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.701+0000 7f780fe4b740 10 client.8379 fetch_fsmap finished waiting for FSMap version 69 2023-12-19T18:24:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.701+0000 7f780fe4b740 10 client.8379 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-19T18:24:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.701+0000 7f780fe4b740 20 client.8379 populate_metadata read hostname 'smithi027' 2023-12-19T18:24:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.701+0000 7f780fe4b740 1 --2- 172.21.15.27:0/2064710890 >> [v2:172.21.15.27:6832/2519815591,v1:172.21.15.27:6833/2519815591] conn(0x557356eba6a0 0x557356ecffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.701+0000 7f780fe4b740 4 client.8379 mds_command: new command op to 8247 tid=0[{"prefix": "status"}] 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.701+0000 7f780fe4b740 1 -- 172.21.15.27:0/2064710890 --> [v2:172.21.15.27:6832/2519815591,v1:172.21.15.27:6833/2519815591] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557356de9100 con 0x557356eba6a0 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.701+0000 7f78067fc640 1 --2- 172.21.15.27:0/2064710890 >> [v2:172.21.15.27:6832/2519815591,v1:172.21.15.27:6833/2519815591] conn(0x557356eba6a0 0x557356ecffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.701+0000 7f78067fc640 1 --2- 172.21.15.27:0/2064710890 >> [v2:172.21.15.27:6832/2519815591,v1:172.21.15.27:6833/2519815591] conn(0x557356eba6a0 0x557356ecffc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b69350c028f39b5 server_cookie=f461141151588e5e in_seq=0 out_seq=0 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.701+0000 7f77e5ffb640 10 client.8379 ms_handle_connect on v2:172.21.15.27:6832/2519815591 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f77e5ffb640 1 -- 172.21.15.27:0/2064710890 <== mds.0 v2:172.21.15.27:6832/2519815591 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x557356de9100 con 0x557356eba6a0 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f77e5ffb640 10 client.8379 handle_command_reply: tid=0 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f780fe4b740 1 client.8379 shutdown 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f780fe4b740 1 -- 172.21.15.27:0/2064710890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb05d0 msgr2=0x557356eb1320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f780fe4b740 1 --2- 172.21.15.27:0/2064710890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb05d0 0x557356eb1320 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f77f8006ca0 tx=0x7f77f8009190 comp rx=0 tx=0).stop 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f780fe4b740 1 -- 172.21.15.27:0/2064710890 shutdown_connections 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f780fe4b740 1 --2- 172.21.15.27:0/2064710890 >> [v2:172.21.15.27:6832/2519815591,v1:172.21.15.27:6833/2519815591] conn(0x557356eba6a0 0x557356ecffc0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f780fe4b740 1 --2- 172.21.15.27:0/2064710890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557356ed4e40 0x557356eef830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f780fe4b740 1 --2- 172.21.15.27:0/2064710890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557356ed2960 0x557356eef2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f780fe4b740 1 --2- 172.21.15.27:0/2064710890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557356eb05d0 0x557356eb1320 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f77e5ffb640 0 client.8379 ms_handle_reset on v2:172.21.15.27:6832/2519815591 2023-12-19T18:24:15.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f780fe4b740 1 -- 172.21.15.27:0/2064710890 >> 172.21.15.27:0/2064710890 conn(0x557356eb4550 msgr2=0x557356e88050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:15.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f780fe4b740 1 -- 172.21.15.27:0/2064710890 shutdown_connections 2023-12-19T18:24:15.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f780fe4b740 1 -- 172.21.15.27:0/2064710890 wait complete. 2023-12-19T18:24:15.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.702+0000 7f780fe4b740 20 client.8379 trim_cache size 0 max 16384 2023-12-19T18:24:15.705 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:24:15.705 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:24:15.705 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-12-19T18:24:15.705 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 8247, 2023-12-19T18:24:15.705 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-19T18:24:15.705 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-19T18:24:15.706 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-19T18:24:15.706 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 7.3550774240000001, 2023-12-19T18:24:15.706 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 69, 2023-12-19T18:24:15.706 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 170, 2023-12-19T18:24:15.706 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 170, 2023-12-19T18:24:15.706 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 13.149349755999999 2023-12-19T18:24:15.706 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:24:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.703+0000 7f7807fff640 1 -- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f77f406a460 msgr2=0x7f77f406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.703+0000 7f7807fff640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f77f406a460 0x7f77f406c920 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f77f8036000 tx=0x7f77f8008030 comp rx=0 tx=0).stop 2023-12-19T18:24:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.703+0000 7f7807fff640 1 -- 172.21.15.27:0/2395381676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78080af360 msgr2=0x7f7808156c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.703+0000 7f7807fff640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78080af360 0x7f7808156c30 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f77fc00c400 tx=0x7f77fc00c8c0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.703+0000 7f7807fff640 1 -- 172.21.15.27:0/2395381676 shutdown_connections 2023-12-19T18:24:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.703+0000 7f7807fff640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f77f406a460 0x7f77f406c920 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.703+0000 7f7807fff640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f780815ac50 0x7f7808157170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.703+0000 7f7807fff640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78080af360 0x7f7808156c30 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.703+0000 7f7807fff640 1 --2- 172.21.15.27:0/2395381676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78080aef20 0x7f780813b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.703+0000 7f7807fff640 1 -- 172.21.15.27:0/2395381676 >> 172.21.15.27:0/2395381676 conn(0x7f780806a7d0 msgr2=0x7f78080cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:15.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.703+0000 7f7807fff640 1 -- 172.21.15.27:0/2395381676 shutdown_connections 2023-12-19T18:24:15.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.703+0000 7f7807fff640 1 -- 172.21.15.27:0/2395381676 wait complete. 2023-12-19T18:24:15.723 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:24:15.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.943+0000 7fe5e1511640 1 Processor -- start 2023-12-19T18:24:15.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.943+0000 7fe5e1511640 1 -- start start 2023-12-19T18:24:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.945+0000 7fe5e1511640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe5dc0b4ef0 0x7fe5dc0b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.945+0000 7fe5e1511640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe5dc0d2350 0x7fe5dc0b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.945+0000 7fe5e1511640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5dc0b5e90 con 0x7fe5dc1604d0 2023-12-19T18:24:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.945+0000 7fe5e1511640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5dc0d2b30 con 0x7fe5dc0d2350 2023-12-19T18:24:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.945+0000 7fe5e1511640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5dc0d2850 con 0x7fe5dc0b4ef0 2023-12-19T18:24:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.945+0000 7fe5e0d10640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe5dc0d2350 0x7fe5dc0b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.945+0000 7fe5e0d10640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe5dc0d2350 0x7fe5dc0b6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60222/0 (socket says 172.21.15.27:60222) 2023-12-19T18:24:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.945+0000 7fe5e0d10640 1 -- 172.21.15.27:0/3770513474 learned_addr learned my addr 172.21.15.27:0/3770513474 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.946+0000 7fe5d3fff640 1 --2- 172.21.15.27:0/3770513474 >> v2:172.21.15.192:3300/0 conn(0x7fe5dc0b4ef0 0x7fe5dc0b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.946+0000 7fe5e0d10640 1 -- 172.21.15.27:0/3770513474 >> v2:172.21.15.192:3300/0 conn(0x7fe5dc0b4ef0 msgr2=0x7fe5dc0b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:15.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.946+0000 7fe5e0d10640 1 --2- 172.21.15.27:0/3770513474 >> v2:172.21.15.192:3300/0 conn(0x7fe5dc0b4ef0 0x7fe5dc0b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.946+0000 7fe5e0d10640 1 -- 172.21.15.27:0/3770513474 >> v1:172.21.15.192:6789/0 conn(0x7fe5dc1604d0 legacy=0x7fe5dc0b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:15.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.946+0000 7fe5e0d10640 1 -- 172.21.15.27:0/3770513474 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5dc0cded0 con 0x7fe5dc0d2350 2023-12-19T18:24:15.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.946+0000 7fe5d3fff640 1 --2- 172.21.15.27:0/3770513474 >> v2:172.21.15.192:3300/0 conn(0x7fe5dc0b4ef0 0x7fe5dc0b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:15.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.947+0000 7fe5e0d10640 1 --2- 172.21.15.27:0/3770513474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5dc0d2350 0x7fe5dc0b6840 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc072120 tx=0x7fe5d402bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338a1e6a137dc0ce server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:15.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.948+0000 7fe5db7fe640 1 -- 172.21.15.27:0/3770513474 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d402ca80 con 0x7fe5dc0d2350 2023-12-19T18:24:15.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.948+0000 7fe5db7fe640 1 -- 172.21.15.27:0/3770513474 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d402cbe0 con 0x7fe5dc0d2350 2023-12-19T18:24:15.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.948+0000 7fe5db7fe640 1 -- 172.21.15.27:0/3770513474 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d40343f0 con 0x7fe5dc0d2350 2023-12-19T18:24:15.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.948+0000 7fe5e1511640 1 -- 172.21.15.27:0/3770513474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5dc0d2350 msgr2=0x7fe5dc0b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:15.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.948+0000 7fe5e1511640 1 --2- 172.21.15.27:0/3770513474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5dc0d2350 0x7fe5dc0b6840 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fe5dc072120 tx=0x7fe5d402bee0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.948+0000 7fe5e1511640 1 -- 172.21.15.27:0/3770513474 shutdown_connections 2023-12-19T18:24:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.948+0000 7fe5e1511640 1 --2- 172.21.15.27:0/3770513474 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5dc0d2350 0x7fe5dc0b6840 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.948+0000 7fe5e1511640 1 --2- 172.21.15.27:0/3770513474 >> v2:172.21.15.192:3300/0 conn(0x7fe5dc0b4ef0 0x7fe5dc0b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.948+0000 7fe5e1511640 1 -- 172.21.15.27:0/3770513474 >> 172.21.15.27:0/3770513474 conn(0x7fe5dc0cf7e0 msgr2=0x7fe5dc0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.948+0000 7fe5e1511640 1 -- 172.21.15.27:0/3770513474 shutdown_connections 2023-12-19T18:24:15.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.948+0000 7fe5e1511640 1 -- 172.21.15.27:0/3770513474 wait complete. 2023-12-19T18:24:15.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.950+0000 7fe5e1511640 1 Processor -- start 2023-12-19T18:24:15.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.950+0000 7fe5e1511640 1 -- start start 2023-12-19T18:24:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.951+0000 7fe5e1511640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5dc0b4ef0 0x7fe5dc0cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.951+0000 7fe5e1511640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5dc0d2350 0x7fe5dc14f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.951+0000 7fe5e1511640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5dc1604d0 0x7fe5dc0a9980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.951+0000 7fe5e1511640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc14fd20 con 0x7fe5dc0d2350 2023-12-19T18:24:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.951+0000 7fe5e1511640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc0cb540 con 0x7fe5dc0b4ef0 2023-12-19T18:24:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.951+0000 7fe5e1511640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5dc0cb820 con 0x7fe5dc1604d0 2023-12-19T18:24:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.952+0000 7fe5d3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5dc0d2350 0x7fe5dc14f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.952+0000 7fe5d3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5dc0d2350 0x7fe5dc14f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60224/0 (socket says 172.21.15.27:60224) 2023-12-19T18:24:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.952+0000 7fe5d3fff640 1 -- 172.21.15.27:0/3530603008 learned_addr learned my addr 172.21.15.27:0/3530603008 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.952+0000 7fe5dbfff640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5dc0b4ef0 0x7fe5dc0cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.952+0000 7fe5e0d10640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5dc1604d0 0x7fe5dc0a9980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.952+0000 7fe5d3fff640 1 -- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5dc1604d0 msgr2=0x7fe5dc0a9980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.952+0000 7fe5d3fff640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5dc1604d0 0x7fe5dc0a9980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.952+0000 7fe5d3fff640 1 -- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5dc0b4ef0 msgr2=0x7fe5dc0cae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.952+0000 7fe5d3fff640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5dc0b4ef0 0x7fe5dc0cae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.952+0000 7fe5d3fff640 1 -- 172.21.15.27:0/3530603008 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d4009b30 con 0x7fe5dc0d2350 2023-12-19T18:24:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.953+0000 7fe5d3fff640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5dc0d2350 0x7fe5dc14f7e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c7f0 tx=0x7fe5c400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.953+0000 7fe5d97fa640 1 -- 172.21.15.27:0/3530603008 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4007820 con 0x7fe5dc0d2350 2023-12-19T18:24:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.953+0000 7fe5d97fa640 1 -- 172.21.15.27:0/3530603008 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c4007980 con 0x7fe5dc0d2350 2023-12-19T18:24:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.953+0000 7fe5d97fa640 1 -- 172.21.15.27:0/3530603008 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4007cf0 con 0x7fe5dc0d2350 2023-12-19T18:24:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.953+0000 7fe5e1511640 1 -- 172.21.15.27:0/3530603008 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5dc0cbb00 con 0x7fe5dc0d2350 2023-12-19T18:24:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.953+0000 7fe5e1511640 1 -- 172.21.15.27:0/3530603008 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5dc0cc0c0 con 0x7fe5dc0d2350 2023-12-19T18:24:15.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.954+0000 7fe5e1511640 1 -- 172.21.15.27:0/3530603008 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5dc066f80 con 0x7fe5dc0d2350 2023-12-19T18:24:15.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.955+0000 7fe5d97fa640 1 -- 172.21.15.27:0/3530603008 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe5c40113f0 con 0x7fe5dc0d2350 2023-12-19T18:24:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.956+0000 7fe5d97fa640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5bc06a430 0x7fe5bc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.956+0000 7fe5d97fa640 1 -- 172.21.15.27:0/3530603008 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe5c4088c50 con 0x7fe5dc0d2350 2023-12-19T18:24:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.957+0000 7fe5dbfff640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5bc06a430 0x7fe5bc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:15.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.958+0000 7fe5dbfff640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5bc06a430 0x7fe5bc06c8f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc007fd0 tx=0x7fe5cc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:15.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:15.961+0000 7fe5d97fa640 1 -- 172.21.15.27:0/3530603008 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe5c4054230 con 0x7fe5dc0d2350 2023-12-19T18:24:16.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.196+0000 7fe5e1511640 1 -- 172.21.15.27:0/3530603008 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe5dc04e840 con 0x7fe5dc0d2350 2023-12-19T18:24:16.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.197+0000 7fe5d97fa640 1 -- 172.21.15.27:0/3530603008 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe5c401f070 con 0x7fe5dc0d2350 2023-12-19T18:24:16.197 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:24:16.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.199+0000 7fe5e1511640 1 -- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5bc06a430 msgr2=0x7fe5bc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:16.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.199+0000 7fe5e1511640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5bc06a430 0x7fe5bc06c8f0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc007fd0 tx=0x7fe5cc009040 comp rx=0 tx=0).stop 2023-12-19T18:24:16.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.200+0000 7fe5e1511640 1 -- 172.21.15.27:0/3530603008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5dc0d2350 msgr2=0x7fe5dc14f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:16.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.200+0000 7fe5e1511640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5dc0d2350 0x7fe5dc14f7e0 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c7f0 tx=0x7fe5c400ccb0 comp rx=0 tx=0).stop 2023-12-19T18:24:16.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.200+0000 7fe5e1511640 1 -- 172.21.15.27:0/3530603008 shutdown_connections 2023-12-19T18:24:16.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.200+0000 7fe5e1511640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5bc06a430 0x7fe5bc06c8f0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:16.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.200+0000 7fe5e1511640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5dc1604d0 0x7fe5dc0a9980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:16.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.200+0000 7fe5e1511640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5dc0d2350 0x7fe5dc14f7e0 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:16.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.200+0000 7fe5e1511640 1 --2- 172.21.15.27:0/3530603008 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5dc0b4ef0 0x7fe5dc0cae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:16.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.200+0000 7fe5e1511640 1 -- 172.21.15.27:0/3530603008 >> 172.21.15.27:0/3530603008 conn(0x7fe5dc0cf7e0 msgr2=0x7fe5dc15ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:16.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.200+0000 7fe5e1511640 1 -- 172.21.15.27:0/3530603008 shutdown_connections 2023-12-19T18:24:16.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:16.200+0000 7fe5e1511640 1 -- 172.21.15.27:0/3530603008 wait complete. 2023-12-19T18:24:16.220 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:24:16.220 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:24:16.220 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-12-19T18:24:16.220 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:24:16.220 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:24:16.220 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:24:16.220 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:24:16.220 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:24:16.220 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:24:16.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:16.220 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-12-19T18:24:16.278 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:24:16.278 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:24:16.278 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:24:16.278 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41784sec preferred_lft 41784sec 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2023-12-19T18:24:16.279 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::8053:76ff:fe69:c66c/64 scope link 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::781d:d6ff:feb4:7a2d/64 scope link 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:24:16.280 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether f2:29:63:cc:14:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:24:16.281 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::f029:63ff:fecc:140e/64 scope link 2023-12-19T18:24:16.281 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:16.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:16.281 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:24:16.387 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-12-19T18:24:16.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:16.392 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-12-19T18:24:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-12-19T18:24:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:24:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-19T18:24:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-19T18:24:16.448 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:24:16.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:16.449 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:24:16.449 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:24:16.449 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:24:16.449 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:24:16.449 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:24:16.449 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:24:16.666 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:16.667 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:24:16.723 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:24:16.724 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:24:16.724 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:24:16.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:16.724 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-12-19T18:24:16.833 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-19T18:24:16.833 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:24:16.833 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:24:16.898 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:24:16.898 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:24:16.898 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:24:16.954 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:24:17.050 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:24:17.050 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:24:17.151 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:24:17.155 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:24:17.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:17.156 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:24:17.213 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:24:17.213 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-19T18:24:17.219 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:24:17.265 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:24:17.269 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:24:17.306 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:24:17.307 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:24:17.418 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-12-19T18:24:17.419+0000 7fe8ea445480 -1 init, newargv = 0x55fe46c35690 newargc=15 2023-12-19T18:24:17.419 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[116729]: starting ceph client 2023-12-19T18:24:17.426 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[116729]: starting fuse 2023-12-19T18:24:17.429 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:24:17.432 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:24:17.433 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:17.433 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:24:17.474 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:24:17.475 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-19T18:24:17.475 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:24:17.475 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:24:17.475 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:24:17.476 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:24:17.476 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:24:17.476 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:24:17.476 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:24:17.476 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:24:17.476 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:24:17.476 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:24:17.476 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:24:17.476 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:24:17.476 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:24:17.476 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:24:17.476 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:24:17.477 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:24:17.477 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:24:17.477 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:24:17.477 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:24:17.477 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:24:17.477 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:24:17.477 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:24:17.477 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:24:17.477 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:24:17.477 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:24:17.477 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:24:17.477 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:24:17.703 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.116729.asok 2023-12-19T18:24:17.882 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.116729.asok 2023-12-19T18:24:17.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:17.882 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.116729.asok status 2023-12-19T18:24:18.142 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:24:18.142 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "116729", 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 8391, 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:24:18.143 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:24:18.144 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 8391 2023-12-19T18:24:18.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:18.144 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:24:18.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:24:18.144 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:24:18.144 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 562398352 2023-12-19T18:24:18.144 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:24:18.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:18.144 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:24:18.144 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:24:18.144 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:24:18.144 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 562398352 2023-12-19T18:24:18.144 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:24:18.145 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.8391 192.168.0.1:0/562398352", 2023-12-19T18:24:18.145 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/562398352", 2023-12-19T18:24:18.145 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-12-19T18:24:18.145 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 69, 2023-12-19T18:24:18.145 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 170, 2023-12-19T18:24:18.145 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:24:18.145 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:24:18.145 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:24:18.145 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:24:18.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:18.158 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-12-19T18:24:18.158 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-19T18:24:18.217 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-12-19T18:24:18.217 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-19T18:24:18.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:18.217 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:24:18.329 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:24:18.562 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.562+0000 7f99e9c22640 1 Processor -- start 2023-12-19T18:24:18.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.562+0000 7f99e9c22640 1 -- start start 2023-12-19T18:24:18.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.564+0000 7f99e9c22640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f99e40c50a0 0x7f99e40afeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:18.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.564+0000 7f99e9c22640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e40c54f0 con 0x7f99e415ee30 2023-12-19T18:24:18.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.564+0000 7f99e9c22640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99e40cb350 con 0x7f99e41553a0 2023-12-19T18:24:18.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.564+0000 7f99e9c22640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99e40cbcf0 con 0x7f99e40c50a0 2023-12-19T18:24:18.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.565+0000 7f99e9421640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f99e40c50a0 0x7f99e40afeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:18.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.565+0000 7f99e9421640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f99e40c50a0 0x7f99e40afeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60226/0 (socket says 172.21.15.27:60226) 2023-12-19T18:24:18.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.565+0000 7f99e9421640 1 -- 172.21.15.27:0/2887551842 learned_addr learned my addr 172.21.15.27:0/2887551842 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:18.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.565+0000 7f99e9421640 1 -- 172.21.15.27:0/2887551842 >> v1:172.21.15.192:6789/0 conn(0x7f99e41553a0 legacy=0x7f99e415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:18.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.565+0000 7f99e9421640 1 -- 172.21.15.27:0/2887551842 >> v1:172.21.15.27:6789/0 conn(0x7f99e415ee30 legacy=0x7f99e40c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:18.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.565+0000 7f99e9421640 1 -- 172.21.15.27:0/2887551842 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e40cb630 con 0x7f99e40c50a0 2023-12-19T18:24:18.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.566+0000 7f99e9421640 1 --2- 172.21.15.27:0/2887551842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e40c50a0 0x7f99e40afeb0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009e50 tx=0x7f99dc02bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d492d44715151c0f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:18.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.567+0000 7f99e37fe640 1 -- 172.21.15.27:0/2887551842 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc02ca80 con 0x7f99e40c50a0 2023-12-19T18:24:18.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.567+0000 7f99e37fe640 1 -- 172.21.15.27:0/2887551842 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99dc02cbe0 con 0x7f99e40c50a0 2023-12-19T18:24:18.567 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.567+0000 7f99e37fe640 1 -- 172.21.15.27:0/2887551842 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99dc0343f0 con 0x7f99e40c50a0 2023-12-19T18:24:18.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.567+0000 7f99e9c22640 1 -- 172.21.15.27:0/2887551842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e40c50a0 msgr2=0x7f99e40afeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:18.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.567+0000 7f99e9c22640 1 --2- 172.21.15.27:0/2887551842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e40c50a0 0x7f99e40afeb0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7f99dc009e50 tx=0x7f99dc02bee0 comp rx=0 tx=0).stop 2023-12-19T18:24:18.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.568+0000 7f99e9c22640 1 -- 172.21.15.27:0/2887551842 shutdown_connections 2023-12-19T18:24:18.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.568+0000 7f99e9c22640 1 --2- 172.21.15.27:0/2887551842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e40c50a0 0x7f99e40afeb0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:18.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.568+0000 7f99e9c22640 1 -- 172.21.15.27:0/2887551842 >> 172.21.15.27:0/2887551842 conn(0x7f99e4069c70 msgr2=0x7f99e406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:18.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.568+0000 7f99e9c22640 1 -- 172.21.15.27:0/2887551842 shutdown_connections 2023-12-19T18:24:18.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.568+0000 7f99e9c22640 1 -- 172.21.15.27:0/2887551842 wait complete. 2023-12-19T18:24:18.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.570+0000 7f99e9c22640 1 Processor -- start 2023-12-19T18:24:18.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.570+0000 7f99e9c22640 1 -- start start 2023-12-19T18:24:18.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.571+0000 7f99e9c22640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99e40c50a0 0x7f99e40cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:18.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.571+0000 7f99e9c22640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e41553a0 0x7f99e40aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:18.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.571+0000 7f99e9c22640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99e415ee30 0x7f99e40cc510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:18.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.571+0000 7f99e9c22640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e41e6740 con 0x7f99e41553a0 2023-12-19T18:24:18.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.571+0000 7f99e9c22640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99e41e6a00 con 0x7f99e40c50a0 2023-12-19T18:24:18.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.571+0000 7f99e9c22640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99e41e6ce0 con 0x7f99e415ee30 2023-12-19T18:24:18.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.571+0000 7f99e3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e41553a0 0x7f99e40aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:18.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.571+0000 7f99e3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e41553a0 0x7f99e40aeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60238/0 (socket says 172.21.15.27:60238) 2023-12-19T18:24:18.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.571+0000 7f99e3fff640 1 -- 172.21.15.27:0/3425648408 learned_addr learned my addr 172.21.15.27:0/3425648408 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.572+0000 7f99e9421640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99e415ee30 0x7f99e40cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.572+0000 7f99e8c20640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99e40c50a0 0x7f99e40cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.572+0000 7f99e3fff640 1 -- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99e415ee30 msgr2=0x7f99e40cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.572+0000 7f99e3fff640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99e415ee30 0x7f99e40cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.572+0000 7f99e3fff640 1 -- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99e40c50a0 msgr2=0x7f99e40cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.572+0000 7f99e3fff640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99e40c50a0 0x7f99e40cd6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.572+0000 7f99e3fff640 1 -- 172.21.15.27:0/3425648408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99dc009b30 con 0x7f99e41553a0 2023-12-19T18:24:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.572+0000 7f99e3fff640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e41553a0 0x7f99e40aeca0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f99d000c400 tx=0x7f99d000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.572+0000 7f99e17fa640 1 -- 172.21.15.27:0/3425648408 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d00052d0 con 0x7f99e41553a0 2023-12-19T18:24:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.572+0000 7f99e17fa640 1 -- 172.21.15.27:0/3425648408 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99d0005430 con 0x7f99e41553a0 2023-12-19T18:24:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.572+0000 7f99e17fa640 1 -- 172.21.15.27:0/3425648408 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99d0003830 con 0x7f99e41553a0 2023-12-19T18:24:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.572+0000 7f99e9c22640 1 -- 172.21.15.27:0/3425648408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99e40cd1c0 con 0x7f99e41553a0 2023-12-19T18:24:18.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.573+0000 7f99e9c22640 1 -- 172.21.15.27:0/3425648408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e41547e0 con 0x7f99e41553a0 2023-12-19T18:24:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.574+0000 7f99e9c22640 1 -- 172.21.15.27:0/3425648408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99e40cd3e0 con 0x7f99e41553a0 2023-12-19T18:24:18.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.575+0000 7f99e17fa640 1 -- 172.21.15.27:0/3425648408 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f99d0003990 con 0x7f99e41553a0 2023-12-19T18:24:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.576+0000 7f99e17fa640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99c806a430 0x7f99c806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:18.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.576+0000 7f99e17fa640 1 -- 172.21.15.27:0/3425648408 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f99d0087cc0 con 0x7f99e41553a0 2023-12-19T18:24:18.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.577+0000 7f99e8c20640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99c806a430 0x7f99c806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:18.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.578+0000 7f99e8c20640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99c806a430 0x7f99c806c8f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f99d80015d0 tx=0x7f99d8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:18.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.581+0000 7f99e17fa640 1 -- 172.21.15.27:0/3425648408 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f99d0053340 con 0x7f99e41553a0 2023-12-19T18:24:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.811+0000 7f99e9c22640 1 -- 172.21.15.27:0/3425648408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f99e40c54f0 con 0x7f99e41553a0 2023-12-19T18:24:18.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.812+0000 7f99e17fa640 1 -- 172.21.15.27:0/3425648408 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v69) v1 ==== 53+0+73 (secure 0 0 0) 0x7f99d0017060 con 0x7f99e41553a0 2023-12-19T18:24:18.813 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:24:18.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.815+0000 7f99e9c22640 1 -- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99c806a430 msgr2=0x7f99c806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:18.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.815+0000 7f99e9c22640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99c806a430 0x7f99c806c8f0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f99d80015d0 tx=0x7f99d8009040 comp rx=0 tx=0).stop 2023-12-19T18:24:18.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.815+0000 7f99e9c22640 1 -- 172.21.15.27:0/3425648408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e41553a0 msgr2=0x7f99e40aeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:18.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.815+0000 7f99e9c22640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e41553a0 0x7f99e40aeca0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f99d000c400 tx=0x7f99d000c8c0 comp rx=0 tx=0).stop 2023-12-19T18:24:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.815+0000 7f99e9c22640 1 -- 172.21.15.27:0/3425648408 shutdown_connections 2023-12-19T18:24:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.815+0000 7f99e9c22640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f99c806a430 0x7f99c806c8f0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.815+0000 7f99e9c22640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99e415ee30 0x7f99e40cc510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.815+0000 7f99e9c22640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99e41553a0 0x7f99e40aeca0 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.815+0000 7f99e9c22640 1 --2- 172.21.15.27:0/3425648408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99e40c50a0 0x7f99e40cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.815+0000 7f99e9c22640 1 -- 172.21.15.27:0/3425648408 >> 172.21.15.27:0/3425648408 conn(0x7f99e4069c70 msgr2=0x7f99e415b8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.815+0000 7f99e9c22640 1 -- 172.21.15.27:0/3425648408 shutdown_connections 2023-12-19T18:24:18.816 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:18.815+0000 7f99e9c22640 1 -- 172.21.15.27:0/3425648408 wait complete. 2023-12-19T18:24:18.834 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:24:18.834 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:24:18.834 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-19T18:24:18.834 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-19T18:24:18.834 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T18:24:18.834 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:24:18.834 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:24:18.834 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:24:18.834 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:24:18.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:18.835 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-19T18:24:18.894 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:24:18.895 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 41744sec preferred_lft 41744sec 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::b892:1cff:fedc:4bed/64 scope link 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether a6:e1:fc:e6:56:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::a4e1:fcff:fee6:56fa/64 scope link 2023-12-19T18:24:18.896 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:18.897 INFO:teuthology.orchestra.run.smithi192.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:24:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T18:24:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::404f:6cff:fe73:9aa2/64 scope link 2023-12-19T18:24:18.897 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:24:18.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:18.898 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:24:18.985 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-19T18:24:18.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:18.990 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-19T18:24:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-19T18:24:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:24:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T18:24:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T18:24:19.044 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:24:19.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:19.045 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:24:19.045 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:24:19.045 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T18:24:19.045 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T18:24:19.045 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T18:24:19.045 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:24:19.223 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:19.223 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:24:19.273 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T18:24:19.274 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-19T18:24:19.275 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:24:19.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:19.275 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-19T18:24:19.369 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-19T18:24:19.370 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:24:19.370 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-19T18:24:19.427 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:24:19.427 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:24:19.427 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:24:19.480 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:24:19.576 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:24:19.576 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:24:19.671 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:24:19.674 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:24:19.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:19.675 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:24:19.729 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:24:19.729 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-19T18:24:19.738 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:24:19.815 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:24:19.815 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:24:19.885 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:24:19.888 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:24:19.888 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:19.888 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:24:19.922 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-19T18:24:19.922+0000 7f595da37480 -1 init, newargv = 0x5579435f2780 newargc=15 2023-12-19T18:24:19.922 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[91323]: starting ceph client 2023-12-19T18:24:19.934 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[91323]: starting fuse 2023-12-19T18:24:21.032 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:24:21.134 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:24:21.134 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:24:21.224 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:24:21.228 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:24:21.228 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:21.228 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:24:21.281 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:24:21.281 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-12-19T18:24:21.281 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-19T18:24:21.282 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:24:21.283 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:24:21.283 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:24:21.283 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-19T18:24:21.283 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-19T18:24:21.283 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-19T18:24:21.283 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:24:21.283 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:24:21.283 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-19T18:24:21.283 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-19T18:24:21.461 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.91323.asok 2023-12-19T18:24:21.642 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.91323.asok 2023-12-19T18:24:21.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:21.642 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.91323.asok status 2023-12-19T18:24:21.877 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "91323", 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 5948, 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-19T18:24:21.878 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 5948 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3685369193 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 3685369193 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.5948 192.168.0.1:0/3685369193", 2023-12-19T18:24:21.879 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/3685369193", 2023-12-19T18:24:21.880 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-19T18:24:21.880 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 69, 2023-12-19T18:24:21.880 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 170, 2023-12-19T18:24:21.880 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:24:21.880 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-19T18:24:21.880 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-19T18:24:21.880 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-19T18:24:21.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:21.892 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-19T18:24:21.893 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-19T18:24:21.948 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-19T18:24:21.948 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-19T18:24:21.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:24:21.948 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:24:22.044 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:24:22.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.289+0000 7f3f813cb640 1 Processor -- start 2023-12-19T18:24:22.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.289+0000 7f3f813cb640 1 -- start start 2023-12-19T18:24:22.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.291+0000 7f3f813cb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f7c15d240 0x7f3f7c1618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:22.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.291+0000 7f3f813cb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f7c0ce420 0x7f3f7c0b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:22.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.291+0000 7f3f813cb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f7c0d1970 con 0x7f3f7c0cdf90 2023-12-19T18:24:22.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.291+0000 7f3f813cb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f7c161e10 con 0x7f3f7c15d240 2023-12-19T18:24:22.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.291+0000 7f3f813cb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c06a5e0 con 0x7f3f7c0ce420 2023-12-19T18:24:22.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.291+0000 7f3f7affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f7c15d240 0x7f3f7c1618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:22.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.291+0000 7f3f7affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f7c15d240 0x7f3f7c1618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42412/0 (socket says 172.21.15.27:42412) 2023-12-19T18:24:22.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.291+0000 7f3f7affd640 1 -- 172.21.15.27:0/3604948965 learned_addr learned my addr 172.21.15.27:0/3604948965 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:22.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.292+0000 7f3f7b7fe640 1 --2- 172.21.15.27:0/3604948965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f7c0ce420 0x7f3f7c0b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:22.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.292+0000 7f3f7affd640 1 -- 172.21.15.27:0/3604948965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f7c0ce420 msgr2=0x7f3f7c0b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:22.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.292+0000 7f3f7affd640 1 --2- 172.21.15.27:0/3604948965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f7c0ce420 0x7f3f7c0b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:22.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.292+0000 7f3f7affd640 1 -- 172.21.15.27:0/3604948965 >> v1:172.21.15.192:6789/0 conn(0x7f3f7c0cdf90 legacy=0x7f3f7c0b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:22.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.292+0000 7f3f7affd640 1 -- 172.21.15.27:0/3604948965 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f7c0ce8b0 con 0x7f3f7c15d240 2023-12-19T18:24:22.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.293+0000 7f3f7affd640 1 --2- 172.21.15.27:0/3604948965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c15d240 0x7f3f7c1618d0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009870 tx=0x7f3f6c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25f69b2eaab2512e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:22.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.293+0000 7f3f7a7fc640 1 -- 172.21.15.27:0/3604948965 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c030840 con 0x7f3f7c15d240 2023-12-19T18:24:22.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.293+0000 7f3f7a7fc640 1 -- 172.21.15.27:0/3604948965 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f6c0309a0 con 0x7f3f7c15d240 2023-12-19T18:24:22.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.293+0000 7f3f7a7fc640 1 -- 172.21.15.27:0/3604948965 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f6c030d30 con 0x7f3f7c15d240 2023-12-19T18:24:22.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.294+0000 7f3f813cb640 1 -- 172.21.15.27:0/3604948965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c15d240 msgr2=0x7f3f7c1618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:22.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.294+0000 7f3f813cb640 1 --2- 172.21.15.27:0/3604948965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c15d240 0x7f3f7c1618d0 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c009870 tx=0x7f3f6c02b980 comp rx=0 tx=0).stop 2023-12-19T18:24:22.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.294+0000 7f3f813cb640 1 -- 172.21.15.27:0/3604948965 shutdown_connections 2023-12-19T18:24:22.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.294+0000 7f3f813cb640 1 --2- 172.21.15.27:0/3604948965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f7c0ce420 0x7f3f7c0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:22.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.294+0000 7f3f813cb640 1 --2- 172.21.15.27:0/3604948965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c15d240 0x7f3f7c1618d0 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:22.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.294+0000 7f3f813cb640 1 -- 172.21.15.27:0/3604948965 >> 172.21.15.27:0/3604948965 conn(0x7f3f7c0d31e0 msgr2=0x7f3f7c0d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:22.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.294+0000 7f3f813cb640 1 -- 172.21.15.27:0/3604948965 shutdown_connections 2023-12-19T18:24:22.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.294+0000 7f3f813cb640 1 -- 172.21.15.27:0/3604948965 wait complete. 2023-12-19T18:24:22.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.296+0000 7f3f813cb640 1 Processor -- start 2023-12-19T18:24:22.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.296+0000 7f3f813cb640 1 -- start start 2023-12-19T18:24:22.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.297+0000 7f3f813cb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f7c0cdf90 0x7f3f7c151600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:22.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.297+0000 7f3f813cb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c0ce420 0x7f3f7c0cbf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:22.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.297+0000 7f3f813cb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f7c15d240 0x7f3f7c0cc490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:22.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.297+0000 7f3f813cb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c14f560 con 0x7f3f7c0ce420 2023-12-19T18:24:22.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.297+0000 7f3f813cb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c159de0 con 0x7f3f7c15d240 2023-12-19T18:24:22.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.297+0000 7f3f813cb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f7c15a0c0 con 0x7f3f7c0cdf90 2023-12-19T18:24:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.298+0000 7f3f73fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c0ce420 0x7f3f7c0cbf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.298+0000 7f3f73fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c0ce420 0x7f3f7c0cbf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42424/0 (socket says 172.21.15.27:42424) 2023-12-19T18:24:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.298+0000 7f3f73fff640 1 -- 172.21.15.27:0/371633379 learned_addr learned my addr 172.21.15.27:0/371633379 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.298+0000 7f3f7b7fe640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f7c15d240 0x7f3f7c0cc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.298+0000 7f3f7affd640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f7c0cdf90 0x7f3f7c151600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.298+0000 7f3f73fff640 1 -- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f7c0cdf90 msgr2=0x7f3f7c151600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.298+0000 7f3f73fff640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f7c0cdf90 0x7f3f7c151600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.298+0000 7f3f73fff640 1 -- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f7c15d240 msgr2=0x7f3f7c0cc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.298+0000 7f3f73fff640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f7c15d240 0x7f3f7c0cc490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.298+0000 7f3f73fff640 1 -- 172.21.15.27:0/371633379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f6c0094d0 con 0x7f3f7c0ce420 2023-12-19T18:24:22.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.298+0000 7f3f7affd640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f7c0cdf90 0x7f3f7c151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.298+0000 7f3f7b7fe640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f7c15d240 0x7f3f7c0cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.299+0000 7f3f73fff640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c0ce420 0x7f3f7c0cbf50 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400c430 tx=0x7f3f6400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.299+0000 7f3f737fe640 1 -- 172.21.15.27:0/371633379 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64011790 con 0x7f3f7c0ce420 2023-12-19T18:24:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.299+0000 7f3f737fe640 1 -- 172.21.15.27:0/371633379 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f640118f0 con 0x7f3f7c0ce420 2023-12-19T18:24:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.299+0000 7f3f737fe640 1 -- 172.21.15.27:0/371633379 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f64011ce0 con 0x7f3f7c0ce420 2023-12-19T18:24:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.299+0000 7f3f813cb640 1 -- 172.21.15.27:0/371633379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f7c15a3a0 con 0x7f3f7c0ce420 2023-12-19T18:24:22.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.299+0000 7f3f813cb640 1 -- 172.21.15.27:0/371633379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7c0ccdf0 con 0x7f3f7c0ce420 2023-12-19T18:24:22.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.300+0000 7f3f813cb640 1 -- 172.21.15.27:0/371633379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f7c15a8c0 con 0x7f3f7c0ce420 2023-12-19T18:24:22.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.301+0000 7f3f737fe640 1 -- 172.21.15.27:0/371633379 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3f6401c460 con 0x7f3f7c0ce420 2023-12-19T18:24:22.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.302+0000 7f3f737fe640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3f5c06a430 0x7f3f5c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:22.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.302+0000 7f3f737fe640 1 -- 172.21.15.27:0/371633379 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3f64088920 con 0x7f3f7c0ce420 2023-12-19T18:24:22.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.303+0000 7f3f7affd640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3f5c06a430 0x7f3f5c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:22.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.304+0000 7f3f7affd640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3f5c06a430 0x7f3f5c06c8f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c036000 tx=0x7f3f6c03b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:22.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.307+0000 7f3f737fe640 1 -- 172.21.15.27:0/371633379 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3f64053e70 con 0x7f3f7c0ce420 2023-12-19T18:24:22.542 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.542+0000 7f3f813cb640 1 -- 172.21.15.27:0/371633379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f7c068e90 con 0x7f3f7c0ce420 2023-12-19T18:24:22.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.543+0000 7f3f737fe640 1 -- 172.21.15.27:0/371633379 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f3f64057490 con 0x7f3f7c0ce420 2023-12-19T18:24:22.544 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:24:22.544 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":69,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3291476389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3291476389},{"type":"v1","addr":"172.21.15.192:6836","nonce":3291476389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3526330256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3526330256},{"type":"v1","addr":"172.21.15.192:6837","nonce":3526330256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/899234003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899234003},{"type":"v1","addr":"172.21.15.27:6835","nonce":899234003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:24:08.343302+0000","modified":"2023-12-19T18:24:11.863775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8247":{"gid":8247,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6833/2519815591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2519815591},{"type":"v1","addr":"172.21.15.27:6833","nonce":2519815591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-19T18:24:22.544 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 69 2023-12-19T18:24:22.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.546+0000 7f3f813cb640 1 -- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3f5c06a430 msgr2=0x7f3f5c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:22.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.546+0000 7f3f813cb640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3f5c06a430 0x7f3f5c06c8f0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3f6c036000 tx=0x7f3f6c03b040 comp rx=0 tx=0).stop 2023-12-19T18:24:22.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.546+0000 7f3f813cb640 1 -- 172.21.15.27:0/371633379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c0ce420 msgr2=0x7f3f7c0cbf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:22.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.546+0000 7f3f813cb640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c0ce420 0x7f3f7c0cbf50 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f3f6400c430 tx=0x7f3f6400c8f0 comp rx=0 tx=0).stop 2023-12-19T18:24:22.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.546+0000 7f3f813cb640 1 -- 172.21.15.27:0/371633379 shutdown_connections 2023-12-19T18:24:22.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.546+0000 7f3f813cb640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3f5c06a430 0x7f3f5c06c8f0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:22.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.546+0000 7f3f813cb640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f7c15d240 0x7f3f7c0cc490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.546+0000 7f3f813cb640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f7c0ce420 0x7f3f7c0cbf50 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.546+0000 7f3f813cb640 1 --2- 172.21.15.27:0/371633379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f7c0cdf90 0x7f3f7c151600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.546+0000 7f3f813cb640 1 -- 172.21.15.27:0/371633379 >> 172.21.15.27:0/371633379 conn(0x7f3f7c0d31e0 msgr2=0x7f3f7c0b6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.546+0000 7f3f813cb640 1 -- 172.21.15.27:0/371633379 shutdown_connections 2023-12-19T18:24:22.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:22.546+0000 7f3f813cb640 1 -- 172.21.15.27:0/371633379 wait complete. 2023-12-19T18:24:22.568 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.a.asok --format=json status 2023-12-19T18:24:22.826 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":8247,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.462181729999999,"mdsmap_epoch":69,"osdmap_epoch":170,"osdmap_epoch_barrier":170,"uptime":20.256468124000001} 2023-12-19T18:24:22.826 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 8247, "mdsmap_epoch": 69, "osdmap_epoch": 170, "osdmap_epoch_barrier": 170, "rank_uptime": 14.46218173, "state": "up:active", "uptime": 20.256468124, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:24:22.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-12-19T18:24:23.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.066+0000 7f022f2fb640 1 Processor -- start 2023-12-19T18:24:23.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.066+0000 7f022f2fb640 1 -- start start 2023-12-19T18:24:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.068+0000 7f022f2fb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f022815aeb0 0x7f02280cfd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.068+0000 7f022f2fb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f02280cf310 0x7f022815ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.068+0000 7f022f2fb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f022815f400 con 0x7f022815efb0 2023-12-19T18:24:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.068+0000 7f022f2fb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02280b3b80 con 0x7f022815aeb0 2023-12-19T18:24:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.068+0000 7f022f2fb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f022815f8d0 con 0x7f02280cf310 2023-12-19T18:24:23.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.068+0000 7f022d898640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f022815efb0 0x7f02280b27b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:54546/0 (socket says 172.21.15.27:54546) 2023-12-19T18:24:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.068+0000 7f022d898640 1 -- 172.21.15.27:0/1784482958 learned_addr learned my addr 172.21.15.27:0/1784482958 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:24:23.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.069+0000 7f022d097640 1 --2- 172.21.15.27:0/1784482958 >> v2:172.21.15.27:3300/0 conn(0x7f022815aeb0 0x7f02280cfd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:23.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.069+0000 7f021ffff640 1 -- 172.21.15.27:0/1784482958 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1614303595 0 0) 0x7f0220001490 con 0x7f022815efb0 2023-12-19T18:24:23.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.069+0000 7f022c896640 1 --2- 172.21.15.27:0/1784482958 >> v2:172.21.15.192:3300/0 conn(0x7f02280cf310 0x7f022815ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:23.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.069+0000 7f021ffff640 1 -- 172.21.15.27:0/1784482958 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0214003520 con 0x7f022815efb0 2023-12-19T18:24:23.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.069+0000 7f022d097640 1 -- 172.21.15.27:0/1784482958 >> v2:172.21.15.192:3300/0 conn(0x7f02280cf310 msgr2=0x7f022815ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:24:23.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.069+0000 7f022d097640 1 --2- 172.21.15.27:0/1784482958 >> v2:172.21.15.192:3300/0 conn(0x7f02280cf310 0x7f022815ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:23.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.069+0000 7f022d097640 1 -- 172.21.15.27:0/1784482958 >> v1:172.21.15.27:6789/0 conn(0x7f022815efb0 legacy=0x7f02280b27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:23.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.069+0000 7f022d097640 1 -- 172.21.15.27:0/1784482958 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02280d0490 con 0x7f022815aeb0 2023-12-19T18:24:23.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.070+0000 7f022d097640 1 --2- 172.21.15.27:0/1784482958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022815aeb0 0x7f02280cfd20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f02280c7d90 tx=0x7f022402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80988d7cbc81eaf0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:23.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.071+0000 7f021ffff640 1 -- 172.21.15.27:0/1784482958 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022402e070 con 0x7f022815aeb0 2023-12-19T18:24:23.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.071+0000 7f021ffff640 1 -- 172.21.15.27:0/1784482958 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02240315a0 con 0x7f022815aeb0 2023-12-19T18:24:23.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.071+0000 7f021ffff640 1 -- 172.21.15.27:0/1784482958 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0224031930 con 0x7f022815aeb0 2023-12-19T18:24:23.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.071+0000 7f022f2fb640 1 -- 172.21.15.27:0/1784482958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022815aeb0 msgr2=0x7f02280cfd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:23.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.071+0000 7f022f2fb640 1 --2- 172.21.15.27:0/1784482958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022815aeb0 0x7f02280cfd20 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7f02280c7d90 tx=0x7f022402b840 comp rx=0 tx=0).stop 2023-12-19T18:24:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.071+0000 7f022f2fb640 1 -- 172.21.15.27:0/1784482958 shutdown_connections 2023-12-19T18:24:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.071+0000 7f022f2fb640 1 --2- 172.21.15.27:0/1784482958 >> v2:172.21.15.192:3300/0 conn(0x7f02280cf310 0x7f022815ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.071+0000 7f022f2fb640 1 --2- 172.21.15.27:0/1784482958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022815aeb0 0x7f02280cfd20 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.071+0000 7f022f2fb640 1 -- 172.21.15.27:0/1784482958 >> 172.21.15.27:0/1784482958 conn(0x7f022806f1f0 msgr2=0x7f022806f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.072+0000 7f022f2fb640 1 -- 172.21.15.27:0/1784482958 shutdown_connections 2023-12-19T18:24:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.072+0000 7f022f2fb640 1 -- 172.21.15.27:0/1784482958 wait complete. 2023-12-19T18:24:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.073+0000 7f022f2fb640 1 Processor -- start 2023-12-19T18:24:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.074+0000 7f022f2fb640 1 -- start start 2023-12-19T18:24:23.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.074+0000 7f022f2fb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02280cf310 0x7f02281f01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:23.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.074+0000 7f022f2fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022815aeb0 0x7f02281f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:23.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.074+0000 7f022f2fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022815efb0 0x7f02281f1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:23.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.074+0000 7f022f2fb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02281f1720 con 0x7f022815aeb0 2023-12-19T18:24:23.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.074+0000 7f022f2fb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02280c9640 con 0x7f02280cf310 2023-12-19T18:24:23.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.074+0000 7f022f2fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02280c9920 con 0x7f022815efb0 2023-12-19T18:24:23.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.075+0000 7f022c896640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022815aeb0 0x7f02281f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:23.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.075+0000 7f022c896640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022815aeb0 0x7f02281f11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42448/0 (socket says 172.21.15.27:42448) 2023-12-19T18:24:23.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.075+0000 7f022c896640 1 -- 172.21.15.27:0/2404059428 learned_addr learned my addr 172.21.15.27:0/2404059428 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:24:23.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.075+0000 7f022d097640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02280cf310 0x7f02281f01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:23.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.075+0000 7f022d898640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022815efb0 0x7f02281f1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:23.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.075+0000 7f022c896640 1 -- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022815efb0 msgr2=0x7f02281f1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:23.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.075+0000 7f022c896640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022815efb0 0x7f02281f1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:23.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.075+0000 7f022c896640 1 -- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02280cf310 msgr2=0x7f02281f01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:23.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.075+0000 7f022c896640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02280cf310 0x7f02281f01c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:23.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.075+0000 7f022c896640 1 -- 172.21.15.27:0/2404059428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0224009400 con 0x7f022815aeb0 2023-12-19T18:24:23.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.075+0000 7f022d898640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022815efb0 0x7f02281f1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:24:23.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.076+0000 7f022c896640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022815aeb0 0x7f02281f11e0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f021800d800 tx=0x7f021800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:23.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.076+0000 7f021dffb640 1 -- 172.21.15.27:0/2404059428 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021800b6c0 con 0x7f022815aeb0 2023-12-19T18:24:23.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.076+0000 7f021dffb640 1 -- 172.21.15.27:0/2404059428 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f021800b820 con 0x7f022815aeb0 2023-12-19T18:24:23.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.076+0000 7f021dffb640 1 -- 172.21.15.27:0/2404059428 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f021800bbb0 con 0x7f022815aeb0 2023-12-19T18:24:23.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.076+0000 7f022f2fb640 1 -- 172.21.15.27:0/2404059428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02280c9c00 con 0x7f022815aeb0 2023-12-19T18:24:23.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.076+0000 7f022f2fb640 1 -- 172.21.15.27:0/2404059428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02280ca1c0 con 0x7f022815aeb0 2023-12-19T18:24:23.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.077+0000 7f022f2fb640 1 -- 172.21.15.27:0/2404059428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02280712c0 con 0x7f022815aeb0 2023-12-19T18:24:23.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.078+0000 7f021dffb640 1 -- 172.21.15.27:0/2404059428 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f021800bd10 con 0x7f022815aeb0 2023-12-19T18:24:23.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.079+0000 7f021dffb640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020c06a430 0x7f020c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:24:23.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.079+0000 7f021dffb640 1 -- 172.21.15.27:0/2404059428 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0218088f00 con 0x7f022815aeb0 2023-12-19T18:24:23.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.080+0000 7f022d097640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020c06a430 0x7f020c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:24:23.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.081+0000 7f022d097640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020c06a430 0x7f020c06c8f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0224036000 tx=0x7f022403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:24:23.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.083+0000 7f021dffb640 1 -- 172.21.15.27:0/2404059428 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0218054450 con 0x7f022815aeb0 2023-12-19T18:24:23.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.284+0000 7f022f2fb640 1 -- 172.21.15.27:0/2404059428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f02280ca350 con 0x7f022815aeb0 2023-12-19T18:24:23.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.755+0000 7f021dffb640 1 -- 172.21.15.27:0/2404059428 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v171) v1 ==== 181+0+0 (secure 0 0 0) 0x7f0218057a70 con 0x7f022815aeb0 2023-12-19T18:24:23.755 INFO:teuthology.orchestra.run.smithi027.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-12-19T18:24:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.759+0000 7f022f2fb640 1 -- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020c06a430 msgr2=0x7f020c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.759+0000 7f022f2fb640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020c06a430 0x7f020c06c8f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0224036000 tx=0x7f022403d040 comp rx=0 tx=0).stop 2023-12-19T18:24:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.759+0000 7f022f2fb640 1 -- 172.21.15.27:0/2404059428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022815aeb0 msgr2=0x7f02281f11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:24:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.759+0000 7f022f2fb640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022815aeb0 0x7f02281f11e0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f021800d800 tx=0x7f021800dcc0 comp rx=0 tx=0).stop 2023-12-19T18:24:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.759+0000 7f022f2fb640 1 -- 172.21.15.27:0/2404059428 shutdown_connections 2023-12-19T18:24:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.759+0000 7f022f2fb640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f020c06a430 0x7f020c06c8f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.759+0000 7f022f2fb640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f022815efb0 0x7f02281f1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.759+0000 7f022f2fb640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f022815aeb0 0x7f02281f11e0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.759+0000 7f022f2fb640 1 --2- 172.21.15.27:0/2404059428 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f02280cf310 0x7f02281f01c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:24:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.759+0000 7f022f2fb640 1 -- 172.21.15.27:0/2404059428 >> 172.21.15.27:0/2404059428 conn(0x7f022806f1f0 msgr2=0x7f02280b5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:24:23.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.760+0000 7f022f2fb640 1 -- 172.21.15.27:0/2404059428 shutdown_connections 2023-12-19T18:24:23.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:24:23.760+0000 7f022f2fb640 1 -- 172.21.15.27:0/2404059428 wait complete. 2023-12-19T18:24:23.783 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-12-19T18:24:24.028 INFO:teuthology.orchestra.run.smithi027.stdout:{"osd_mon_report_interval":"5"} 2023-12-19T18:24:24.029 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-12-19T18:24:24.029 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-12-19T18:24:24.272 INFO:teuthology.orchestra.run.smithi027.stdout:{"mon_tick_interval":"5"} 2023-12-19T18:24:24.272 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-12-19T18:24:24.273 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-12-19T18:24:24.273 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:24:24.273 DEBUG:teuthology.orchestra.run.smithi027:> import time 2023-12-19T18:24:24.273 DEBUG:teuthology.orchestra.run.smithi027:> import datetime 2023-12-19T18:24:24.273 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:24:24.273 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:24:24.273 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:24:24.273 DEBUG:teuthology.orchestra.run.smithi027:> # Write some buffered data through before going full, all should be well 2023-12-19T18:24:24.273 DEBUG:teuthology.orchestra.run.smithi027:> print("writing some data through which we expect to succeed") 2023-12-19T18:24:24.273 DEBUG:teuthology.orchestra.run.smithi027:> bytes = 0 2023-12-19T18:24:24.273 DEBUG:teuthology.orchestra.run.smithi027:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-12-19T18:24:24.274 DEBUG:teuthology.orchestra.run.smithi027:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-12-19T18:24:24.274 DEBUG:teuthology.orchestra.run.smithi027:> bytes += os.write(f, b'"'"'a'"'"' * 512 * 1024) 2023-12-19T18:24:24.274 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(f) 2023-12-19T18:24:24.274 DEBUG:teuthology.orchestra.run.smithi027:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-12-19T18:24:24.274 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:24:24.274 DEBUG:teuthology.orchestra.run.smithi027:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-12-19T18:24:24.274 DEBUG:teuthology.orchestra.run.smithi027:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-12-19T18:24:24.274 DEBUG:teuthology.orchestra.run.smithi027:> # buffered IO, the error should always only appear from fsync and not 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> # from write 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> full = False 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> for n in range(0, int(32 * 0.9)): 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> print("wrote {0} bytes via buffered write, may repeat".format(bytes)) 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> print("done writing {0} bytes".format(bytes)) 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> # OK, now we should sneak in under the full condition 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> # due to the time it takes the OSDs to report to the 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> # mons, and get a successful fsync on our full-making data 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(f) 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> print("successfully fsync'"'"'ed prior to getting full state reported") 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> # buffered write, add more dirty data to the buffer 2023-12-19T18:24:24.275 DEBUG:teuthology.orchestra.run.smithi027:> print("starting buffered write") 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> try: 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> for n in range(0, int(32 * 0.2)): 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> time.sleep(15.0) 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> except OSError: 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> pass; 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> print("wrote, now waiting 30s and then doing a close we expect to fail") 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> # Wait long enough for a background flush that should fail 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> time.sleep(30) 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> if True: 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> # ...and check that the failed background flush is reflected in fclose 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> try: 2023-12-19T18:24:24.276 DEBUG:teuthology.orchestra.run.smithi027:> os.close(f) 2023-12-19T18:24:24.277 DEBUG:teuthology.orchestra.run.smithi027:> except OSError: 2023-12-19T18:24:24.277 DEBUG:teuthology.orchestra.run.smithi027:> print("close() returned an error as expected") 2023-12-19T18:24:24.277 DEBUG:teuthology.orchestra.run.smithi027:> else: 2023-12-19T18:24:24.277 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("close() failed to raise error") 2023-12-19T18:24:24.277 DEBUG:teuthology.orchestra.run.smithi027:> else: 2023-12-19T18:24:24.277 DEBUG:teuthology.orchestra.run.smithi027:> # The kernel cephfs client does not raise errors on fclose 2023-12-19T18:24:24.277 DEBUG:teuthology.orchestra.run.smithi027:> os.close(f) 2023-12-19T18:24:24.277 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:24:24.277 DEBUG:teuthology.orchestra.run.smithi027:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-12-19T18:24:24.277 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:24:47.401 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:24:47.411 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:25:17.550 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:25:17.560 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:25:47.698 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:25:47.708 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:26:13.091 INFO:teuthology.orchestra.run.smithi027.stdout:writing some data through which we expect to succeed 2023-12-19T18:26:13.092 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-12-19T18:26:13.092 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 1572864 bytes via buffered write, may repeat 2023-12-19T18:26:13.092 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 2621440 bytes via buffered write, may repeat 2023-12-19T18:26:13.092 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 3670016 bytes via buffered write, may repeat 2023-12-19T18:26:13.092 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 4718592 bytes via buffered write, may repeat 2023-12-19T18:26:13.092 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 5767168 bytes via buffered write, may repeat 2023-12-19T18:26:13.092 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 6815744 bytes via buffered write, may repeat 2023-12-19T18:26:13.092 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 7864320 bytes via buffered write, may repeat 2023-12-19T18:26:13.092 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 8912896 bytes via buffered write, may repeat 2023-12-19T18:26:13.092 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 9961472 bytes via buffered write, may repeat 2023-12-19T18:26:13.092 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 11010048 bytes via buffered write, may repeat 2023-12-19T18:26:13.093 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 12058624 bytes via buffered write, may repeat 2023-12-19T18:26:13.093 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 13107200 bytes via buffered write, may repeat 2023-12-19T18:26:13.093 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 14155776 bytes via buffered write, may repeat 2023-12-19T18:26:13.093 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 15204352 bytes via buffered write, may repeat 2023-12-19T18:26:13.093 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 16252928 bytes via buffered write, may repeat 2023-12-19T18:26:13.093 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 17301504 bytes via buffered write, may repeat 2023-12-19T18:26:13.093 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 18350080 bytes via buffered write, may repeat 2023-12-19T18:26:13.093 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 19398656 bytes via buffered write, may repeat 2023-12-19T18:26:13.093 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 20447232 bytes via buffered write, may repeat 2023-12-19T18:26:13.093 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 21495808 bytes via buffered write, may repeat 2023-12-19T18:26:13.093 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 22544384 bytes via buffered write, may repeat 2023-12-19T18:26:13.093 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 23592960 bytes via buffered write, may repeat 2023-12-19T18:26:13.093 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 24641536 bytes via buffered write, may repeat 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 25690112 bytes via buffered write, may repeat 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 26738688 bytes via buffered write, may repeat 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 27787264 bytes via buffered write, may repeat 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 28835840 bytes via buffered write, may repeat 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:wrote 29884416 bytes via buffered write, may repeat 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:done writing 29884416 bytes 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:successfully fsync'ed prior to getting full state reported 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:starting buffered write 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:wrote, now waiting 30s and then doing a close we expect to fail 2023-12-19T18:26:13.094 INFO:teuthology.orchestra.run.smithi027.stdout:close() returned an error as expected 2023-12-19T18:26:13.203 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:26:13.204 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:26:13.341 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:26:13.465 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:26:13.466 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:26:13.466 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:26:13.531 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:26:13.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.769+0000 7fbe0882b640 1 Processor -- start 2023-12-19T18:26:13.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.769+0000 7fbe0882b640 1 -- start start 2023-12-19T18:26:13.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.771+0000 7fbe0882b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe040cb300 0x7fbe040cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:13.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.771+0000 7fbe0882b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe040cbc20 con 0x7fbe04161010 2023-12-19T18:26:13.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.771+0000 7fbe0882b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe04145d60 con 0x7fbe040c8f90 2023-12-19T18:26:13.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.771+0000 7fbe0882b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe040c4480 con 0x7fbe040cb300 2023-12-19T18:26:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.772+0000 7fbe0359e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbe04161010 0x7fbe040cd090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40034/0 (socket says 172.21.15.27:40034) 2023-12-19T18:26:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.772+0000 7fbe0359e640 1 -- 172.21.15.27:0/3116585658 learned_addr learned my addr 172.21.15.27:0/3116585658 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.772+0000 7fbe02d9d640 1 --2- 172.21.15.27:0/3116585658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe040cb300 0x7fbe040cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:13.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 392602927 0 0) 0x7fbe040cbc20 con 0x7fbe04161010 2023-12-19T18:26:13.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdec003520 con 0x7fbe04161010 2023-12-19T18:26:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4048095506 0 0) 0x7fbe04145d60 con 0x7fbe040c8f90 2023-12-19T18:26:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe040cbc20 con 0x7fbe040c8f90 2023-12-19T18:26:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2426207320 0 0) 0x7fbdec003520 con 0x7fbe04161010 2023-12-19T18:26:13.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe040cb300 msgr2=0x7fbe040cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 --2- 172.21.15.27:0/3116585658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe040cb300 0x7fbe040cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 >> v1:172.21.15.192:6789/0 conn(0x7fbe040c8f90 legacy=0x7fbe04160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe04161510 con 0x7fbe04161010 2023-12-19T18:26:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fbdfc0019f0 con 0x7fbe04161010 2023-12-19T18:26:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 >> v1:172.21.15.27:6789/0 conn(0x7fbe04161010 legacy=0x7fbe040cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 --2- 172.21.15.27:0/3116585658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe040cb300 0x7fbdec003b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 --2- 172.21.15.27:0/3116585658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdec0040e0 0x7fbdec0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 --2- 172.21.15.27:0/3116585658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdec004d00 0x7fbdec005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec00ba90 con 0x7fbdec004d00 2023-12-19T18:26:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec00bc00 con 0x7fbdec0040e0 2023-12-19T18:26:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec00bdc0 con 0x7fbe040cb300 2023-12-19T18:26:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.773+0000 7fbe0359e640 1 --2- 172.21.15.27:0/3116585658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdec004d00 0x7fbdec005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:13.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.774+0000 7fbe02d9d640 1 --2- 172.21.15.27:0/3116585658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe040cb300 0x7fbdec003b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:13.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.774+0000 7fbe0259c640 1 --2- 172.21.15.27:0/3116585658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdec0040e0 0x7fbdec0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:13.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.774+0000 7fbe0359e640 1 -- 172.21.15.27:0/3116585658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe040cb300 msgr2=0x7fbdec003b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:13.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.774+0000 7fbe0359e640 1 --2- 172.21.15.27:0/3116585658 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe040cb300 0x7fbdec003b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:13.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.774+0000 7fbe0359e640 1 -- 172.21.15.27:0/3116585658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdec0040e0 msgr2=0x7fbdec0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:13.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.774+0000 7fbe0359e640 1 --2- 172.21.15.27:0/3116585658 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdec0040e0 0x7fbdec0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:13.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.774+0000 7fbe0359e640 1 -- 172.21.15.27:0/3116585658 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdec003520 con 0x7fbdec004d00 2023-12-19T18:26:13.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.775+0000 7fbe0359e640 1 --2- 172.21.15.27:0/3116585658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdec004d00 0x7fbdec005180 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fbe04161440 tx=0x7fbdec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec7dfb5ad412dd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:13.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.776+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc034630 con 0x7fbdec004d00 2023-12-19T18:26:13.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.776+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdfc02f070 con 0x7fbdec004d00 2023-12-19T18:26:13.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.776+0000 7fbe01d9b640 1 -- 172.21.15.27:0/3116585658 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdfc034a30 con 0x7fbdec004d00 2023-12-19T18:26:13.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.776+0000 7fbe0882b640 1 -- 172.21.15.27:0/3116585658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdec004d00 msgr2=0x7fbdec005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:13.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.776+0000 7fbe0882b640 1 --2- 172.21.15.27:0/3116585658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdec004d00 0x7fbdec005180 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fbe04161440 tx=0x7fbdec0057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:13.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.776+0000 7fbe0359e640 1 -- 172.21.15.27:0/3116585658 reap_dead start 2023-12-19T18:26:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.776+0000 7fbe0882b640 1 -- 172.21.15.27:0/3116585658 shutdown_connections 2023-12-19T18:26:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.776+0000 7fbe0882b640 1 -- 172.21.15.27:0/3116585658 >> 172.21.15.27:0/3116585658 conn(0x7fbe04069c70 msgr2=0x7fbe0406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.776+0000 7fbe0882b640 1 -- 172.21.15.27:0/3116585658 shutdown_connections 2023-12-19T18:26:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.776+0000 7fbe0882b640 1 -- 172.21.15.27:0/3116585658 wait complete. 2023-12-19T18:26:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.778+0000 7fbe0882b640 1 Processor -- start 2023-12-19T18:26:13.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.778+0000 7fbe0882b640 1 -- start start 2023-12-19T18:26:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.779+0000 7fbe0882b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdec004d00 0x7fbe040c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.779+0000 7fbe0882b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe040c7260 0x7fbe0413d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:13.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.779+0000 7fbe0882b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe041f5fb0 0x7fbe040ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.779+0000 7fbe0882b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe041f6520 con 0x7fbdec004d00 2023-12-19T18:26:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.779+0000 7fbe0882b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe041f6800 con 0x7fbe041f5fb0 2023-12-19T18:26:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.779+0000 7fbe0882b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe041f6ae0 con 0x7fbe040c7260 2023-12-19T18:26:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.780+0000 7fbe02d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdec004d00 0x7fbe040c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.780+0000 7fbe02d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdec004d00 0x7fbe040c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35198/0 (socket says 172.21.15.27:35198) 2023-12-19T18:26:13.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.780+0000 7fbe02d9d640 1 -- 172.21.15.27:0/4099352441 learned_addr learned my addr 172.21.15.27:0/4099352441 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.780+0000 7fbe02d9d640 1 -- 172.21.15.27:0/4099352441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe040c7260 msgr2=0x7fbe0413d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.780+0000 7fbe02d9d640 1 --2- 172.21.15.27:0/4099352441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe040c7260 0x7fbe0413d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.780+0000 7fbe0359e640 1 --2- 172.21.15.27:0/4099352441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe041f5fb0 0x7fbe040ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.780+0000 7fbe02d9d640 1 -- 172.21.15.27:0/4099352441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe041f5fb0 msgr2=0x7fbe040ac930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.780+0000 7fbe02d9d640 1 --2- 172.21.15.27:0/4099352441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe041f5fb0 0x7fbe040ac930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:13.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.780+0000 7fbe02d9d640 1 -- 172.21.15.27:0/4099352441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe041f6dc0 con 0x7fbdec004d00 2023-12-19T18:26:13.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.781+0000 7fbe02d9d640 1 --2- 172.21.15.27:0/4099352441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdec004d00 0x7fbe040c61e0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4011600 tx=0x7fbdf4011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:13.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.781+0000 7fbdeb7fe640 1 -- 172.21.15.27:0/4099352441 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400f920 con 0x7fbdec004d00 2023-12-19T18:26:13.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.781+0000 7fbdeb7fe640 1 -- 172.21.15.27:0/4099352441 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdf400fa80 con 0x7fbdec004d00 2023-12-19T18:26:13.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.781+0000 7fbdeb7fe640 1 -- 172.21.15.27:0/4099352441 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdf400fdf0 con 0x7fbdec004d00 2023-12-19T18:26:13.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.781+0000 7fbe0882b640 1 -- 172.21.15.27:0/4099352441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe040aebf0 con 0x7fbdec004d00 2023-12-19T18:26:13.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.781+0000 7fbe0882b640 1 -- 172.21.15.27:0/4099352441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe040ace70 con 0x7fbdec004d00 2023-12-19T18:26:13.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.782+0000 7fbe0882b640 1 -- 172.21.15.27:0/4099352441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe0404e840 con 0x7fbdec004d00 2023-12-19T18:26:13.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.783+0000 7fbdeb7fe640 1 -- 172.21.15.27:0/4099352441 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbdf4015460 con 0x7fbdec004d00 2023-12-19T18:26:13.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.784+0000 7fbdeb7fe640 1 --2- 172.21.15.27:0/4099352441 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbdf006a460 0x7fbdf006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:13.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.784+0000 7fbdeb7fe640 1 -- 172.21.15.27:0/4099352441 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbdf4088ed0 con 0x7fbdec004d00 2023-12-19T18:26:13.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.785+0000 7fbdeb7fe640 1 -- 172.21.15.27:0/4099352441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fbdf0071f90 con 0x7fbdec004d00 2023-12-19T18:26:13.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.785+0000 7fbe0259c640 1 --2- 172.21.15.27:0/4099352441 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbdf006a460 0x7fbdf006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.786+0000 7fbe0259c640 1 --2- 172.21.15.27:0/4099352441 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbdf006a460 0x7fbdf006c920 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8007fd0 tx=0x7fbdf8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:13.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.792+0000 7fbdeb7fe640 1 -- 172.21.15.27:0/4099352441 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbdf4054420 con 0x7fbdec004d00 2023-12-19T18:26:13.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.989+0000 7fbe0882b640 1 -- 172.21.15.27:0/4099352441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe0406d240 con 0x7fbdec004d00 2023-12-19T18:26:13.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.992+0000 7fbdeb7fe640 1 -- 172.21.15.27:0/4099352441 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16086 (secure 0 0 0) 0x7fbdf4022030 con 0x7fbdec004d00 2023-12-19T18:26:13.992 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:13.992 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":172,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:25:26.756190+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:24:06.280682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T18:24:07.374789+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:26:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.995+0000 7fbe0882b640 1 -- 172.21.15.27:0/4099352441 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbdf006a460 msgr2=0x7fbdf006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.995+0000 7fbe0882b640 1 --2- 172.21.15.27:0/4099352441 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbdf006a460 0x7fbdf006c920 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fbdf8007fd0 tx=0x7fbdf8009040 comp rx=0 tx=0).stop 2023-12-19T18:26:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.995+0000 7fbe0882b640 1 -- 172.21.15.27:0/4099352441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdec004d00 msgr2=0x7fbe040c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.995+0000 7fbe0882b640 1 --2- 172.21.15.27:0/4099352441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdec004d00 0x7fbe040c61e0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4011600 tx=0x7fbdf4011ac0 comp rx=0 tx=0).stop 2023-12-19T18:26:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.995+0000 7fbe0882b640 1 -- 172.21.15.27:0/4099352441 shutdown_connections 2023-12-19T18:26:13.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.995+0000 7fbe0882b640 1 --2- 172.21.15.27:0/4099352441 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbdf006a460 0x7fbdf006c920 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.995+0000 7fbe0882b640 1 --2- 172.21.15.27:0/4099352441 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbe041f5fb0 0x7fbe040ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.995+0000 7fbe0882b640 1 --2- 172.21.15.27:0/4099352441 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbe040c7260 0x7fbe0413d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.995+0000 7fbe0882b640 1 --2- 172.21.15.27:0/4099352441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdec004d00 0x7fbe040c61e0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.995+0000 7fbe0882b640 1 -- 172.21.15.27:0/4099352441 >> 172.21.15.27:0/4099352441 conn(0x7fbe04069c70 msgr2=0x7fbe0406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.995+0000 7fbe0882b640 1 -- 172.21.15.27:0/4099352441 shutdown_connections 2023-12-19T18:26:13.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:13.995+0000 7fbe0882b640 1 -- 172.21.15.27:0/4099352441 wait complete. 2023-12-19T18:26:14.016 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:24:06.280682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:24:07.374789+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:26:14.016 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:26:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.256+0000 7fe79f59e640 1 Processor -- start 2023-12-19T18:26:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.256+0000 7fe79f59e640 1 -- start start 2023-12-19T18:26:14.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.258+0000 7fe79f59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe7a00cf600 0x7fe7a006c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.258+0000 7fe79f59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe7a006c680 0x7fe7a00cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.258+0000 7fe79f59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a00b27d0 con 0x7fe7a0143450 2023-12-19T18:26:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.258+0000 7fe79f59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00b33e0 con 0x7fe7a00cf600 2023-12-19T18:26:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.258+0000 7fe79f59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a00b2a40 con 0x7fe7a006c680 2023-12-19T18:26:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.258+0000 7fe797fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe7a00cf600 0x7fe7a006c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.258+0000 7fe797fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe7a00cf600 0x7fe7a006c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35202/0 (socket says 172.21.15.27:35202) 2023-12-19T18:26:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.258+0000 7fe797fff640 1 -- 172.21.15.27:0/2994532595 learned_addr learned my addr 172.21.15.27:0/2994532595 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.258+0000 7fe79ed9d640 1 --2- 172.21.15.27:0/2994532595 >> v2:172.21.15.192:3300/0 conn(0x7fe7a006c680 0x7fe7a00cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.259+0000 7fe797fff640 1 -- 172.21.15.27:0/2994532595 >> v2:172.21.15.192:3300/0 conn(0x7fe7a006c680 msgr2=0x7fe7a00cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.259+0000 7fe797fff640 1 --2- 172.21.15.27:0/2994532595 >> v2:172.21.15.192:3300/0 conn(0x7fe7a006c680 0x7fe7a00cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.259+0000 7fe797fff640 1 -- 172.21.15.27:0/2994532595 >> v1:172.21.15.192:6789/0 conn(0x7fe7a0143450 legacy=0x7fe7a00ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.259+0000 7fe797fff640 1 -- 172.21.15.27:0/2994532595 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00b2d20 con 0x7fe7a00cf600 2023-12-19T18:26:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.260+0000 7fe797fff640 1 --2- 172.21.15.27:0/2994532595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00cf600 0x7fe7a006c140 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00b22c0 tx=0x7fe78802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b59bdf094250170 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:14.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.260+0000 7fe79dd9b640 1 -- 172.21.15.27:0/2994532595 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7880317e0 con 0x7fe7a00cf600 2023-12-19T18:26:14.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.260+0000 7fe79dd9b640 1 -- 172.21.15.27:0/2994532595 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe788031940 con 0x7fe7a00cf600 2023-12-19T18:26:14.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.260+0000 7fe79dd9b640 1 -- 172.21.15.27:0/2994532595 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe788031cd0 con 0x7fe7a00cf600 2023-12-19T18:26:14.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.261+0000 7fe79f59e640 1 -- 172.21.15.27:0/2994532595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00cf600 msgr2=0x7fe7a006c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:14.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.261+0000 7fe79f59e640 1 --2- 172.21.15.27:0/2994532595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00cf600 0x7fe7a006c140 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00b22c0 tx=0x7fe78802b880 comp rx=0 tx=0).stop 2023-12-19T18:26:14.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.261+0000 7fe79f59e640 1 -- 172.21.15.27:0/2994532595 shutdown_connections 2023-12-19T18:26:14.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.261+0000 7fe79f59e640 1 --2- 172.21.15.27:0/2994532595 >> v2:172.21.15.192:3300/0 conn(0x7fe7a006c680 0x7fe7a00cbdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.261+0000 7fe79f59e640 1 --2- 172.21.15.27:0/2994532595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00cf600 0x7fe7a006c140 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.261+0000 7fe79f59e640 1 -- 172.21.15.27:0/2994532595 >> 172.21.15.27:0/2994532595 conn(0x7fe7a006a060 msgr2=0x7fe7a006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:14.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.261+0000 7fe79f59e640 1 -- 172.21.15.27:0/2994532595 shutdown_connections 2023-12-19T18:26:14.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.261+0000 7fe79f59e640 1 -- 172.21.15.27:0/2994532595 wait complete. 2023-12-19T18:26:14.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.263+0000 7fe79f59e640 1 Processor -- start 2023-12-19T18:26:14.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.263+0000 7fe79f59e640 1 -- start start 2023-12-19T18:26:14.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.264+0000 7fe79f59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a006c680 0x7fe7a0152f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.264+0000 7fe79f59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7a00cf600 0x7fe7a013a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.264+0000 7fe79f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a0143450 0x7fe7a0137440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.264+0000 7fe79f59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a0151b00 con 0x7fe7a0143450 2023-12-19T18:26:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.264+0000 7fe79f59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a01520c0 con 0x7fe7a006c680 2023-12-19T18:26:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.264+0000 7fe79f59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a00cdcb0 con 0x7fe7a00cf600 2023-12-19T18:26:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.265+0000 7fe79ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a0143450 0x7fe7a0137440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.265+0000 7fe79ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a0143450 0x7fe7a0137440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35210/0 (socket says 172.21.15.27:35210) 2023-12-19T18:26:14.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.265+0000 7fe79ed9d640 1 -- 172.21.15.27:0/3624922 learned_addr learned my addr 172.21.15.27:0/3624922 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.265+0000 7fe79e59c640 1 --2- 172.21.15.27:0/3624922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a006c680 0x7fe7a0152f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.265+0000 7fe79ed9d640 1 -- 172.21.15.27:0/3624922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7a00cf600 msgr2=0x7fe7a013a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.265+0000 7fe79ed9d640 1 --2- 172.21.15.27:0/3624922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7a00cf600 0x7fe7a013a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.265+0000 7fe79ed9d640 1 -- 172.21.15.27:0/3624922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a006c680 msgr2=0x7fe7a0152f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.265+0000 7fe79ed9d640 1 --2- 172.21.15.27:0/3624922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a006c680 0x7fe7a0152f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.265+0000 7fe79ed9d640 1 -- 172.21.15.27:0/3624922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7880094d0 con 0x7fe7a0143450 2023-12-19T18:26:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.266+0000 7fe79ed9d640 1 --2- 172.21.15.27:0/3624922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a0143450 0x7fe7a0137440 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fe798007d90 tx=0x7fe79800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.266+0000 7fe796ffd640 1 -- 172.21.15.27:0/3624922 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798014420 con 0x7fe7a0143450 2023-12-19T18:26:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.266+0000 7fe796ffd640 1 -- 172.21.15.27:0/3624922 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7980145a0 con 0x7fe7a0143450 2023-12-19T18:26:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.266+0000 7fe79f59e640 1 -- 172.21.15.27:0/3624922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a00cdf90 con 0x7fe7a0143450 2023-12-19T18:26:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.266+0000 7fe796ffd640 1 -- 172.21.15.27:0/3624922 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe798014950 con 0x7fe7a0143450 2023-12-19T18:26:14.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.266+0000 7fe79f59e640 1 -- 172.21.15.27:0/3624922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a00af7b0 con 0x7fe7a0143450 2023-12-19T18:26:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.267+0000 7fe79f59e640 1 -- 172.21.15.27:0/3624922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a0152230 con 0x7fe7a0143450 2023-12-19T18:26:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.268+0000 7fe796ffd640 1 -- 172.21.15.27:0/3624922 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe798014ab0 con 0x7fe7a0143450 2023-12-19T18:26:14.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.269+0000 7fe796ffd640 1 --2- 172.21.15.27:0/3624922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe78006a430 0x7fe78006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.269+0000 7fe796ffd640 1 -- 172.21.15.27:0/3624922 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe798089240 con 0x7fe7a0143450 2023-12-19T18:26:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.270+0000 7fe796ffd640 1 -- 172.21.15.27:0/3624922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fe780071f60 con 0x7fe7a0143450 2023-12-19T18:26:14.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.270+0000 7fe79e59c640 1 --2- 172.21.15.27:0/3624922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe78006a430 0x7fe78006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:14.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.271+0000 7fe79e59c640 1 --2- 172.21.15.27:0/3624922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe78006a430 0x7fe78006c8f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe790006fd0 tx=0x7fe790008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:14.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.274+0000 7fe796ffd640 1 -- 172.21.15.27:0/3624922 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe798054820 con 0x7fe7a0143450 2023-12-19T18:26:14.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.465+0000 7fe79f59e640 1 -- 172.21.15.27:0/3624922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe7a004e840 con 0x7fe7a0143450 2023-12-19T18:26:14.466 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.466+0000 7fe796ffd640 1 -- 172.21.15.27:0/3624922 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v172) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe798057e40 con 0x7fe7a0143450 2023-12-19T18:26:14.466 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:26:14.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.468+0000 7fe79f59e640 1 -- 172.21.15.27:0/3624922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe78006a430 msgr2=0x7fe78006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:14.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.468+0000 7fe79f59e640 1 --2- 172.21.15.27:0/3624922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe78006a430 0x7fe78006c8f0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fe790006fd0 tx=0x7fe790008040 comp rx=0 tx=0).stop 2023-12-19T18:26:14.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.468+0000 7fe79f59e640 1 -- 172.21.15.27:0/3624922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a0143450 msgr2=0x7fe7a0137440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:14.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.468+0000 7fe79f59e640 1 --2- 172.21.15.27:0/3624922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a0143450 0x7fe7a0137440 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fe798007d90 tx=0x7fe79800e4b0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.468+0000 7fe79f59e640 1 -- 172.21.15.27:0/3624922 shutdown_connections 2023-12-19T18:26:14.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.468+0000 7fe79f59e640 1 --2- 172.21.15.27:0/3624922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe78006a430 0x7fe78006c8f0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.468+0000 7fe79f59e640 1 --2- 172.21.15.27:0/3624922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a0143450 0x7fe7a0137440 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.468+0000 7fe79f59e640 1 --2- 172.21.15.27:0/3624922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7a00cf600 0x7fe7a013a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.468+0000 7fe79f59e640 1 --2- 172.21.15.27:0/3624922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a006c680 0x7fe7a0152f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.468+0000 7fe79f59e640 1 -- 172.21.15.27:0/3624922 >> 172.21.15.27:0/3624922 conn(0x7fe7a006a060 msgr2=0x7fe7a00ad240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:14.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.469+0000 7fe79f59e640 1 -- 172.21.15.27:0/3624922 shutdown_connections 2023-12-19T18:26:14.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.469+0000 7fe79f59e640 1 -- 172.21.15.27:0/3624922 wait complete. 2023-12-19T18:26:14.488 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:26:14.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.725+0000 7fce99589640 1 Processor -- start 2023-12-19T18:26:14.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.725+0000 7fce99589640 1 -- start start 2023-12-19T18:26:14.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.727+0000 7fce99589640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce94163b00 0x7fce94165ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.727+0000 7fce99589640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce940cd6b0 con 0x7fce9415ae80 2023-12-19T18:26:14.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.727+0000 7fce99589640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce940cdc70 con 0x7fce940cb3e0 2023-12-19T18:26:14.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.727+0000 7fce99589640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce940b3630 con 0x7fce94163b00 2023-12-19T18:26:14.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.728+0000 7fce92ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fce9415ae80 0x7fce940b2810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40048/0 (socket says 172.21.15.27:40048) 2023-12-19T18:26:14.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.728+0000 7fce92ffd640 1 -- 172.21.15.27:0/3991703139 learned_addr learned my addr 172.21.15.27:0/3991703139 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.728+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3406938917 0 0) 0x7fce940cd6b0 con 0x7fce9415ae80 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.728+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce80003520 con 0x7fce9415ae80 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.728+0000 7fce937fe640 1 --2- 172.21.15.27:0/3991703139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce94163b00 0x7fce94165ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 638353686 0 0) 0x7fce80003520 con 0x7fce9415ae80 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce94163b00 msgr2=0x7fce94165ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce91ffb640 1 --2- 172.21.15.27:0/3991703139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce94163b00 0x7fce94165ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 >> v1:172.21.15.192:6789/0 conn(0x7fce940cb3e0 legacy=0x7fce940b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce940d0450 con 0x7fce9415ae80 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fce84001670 con 0x7fce9415ae80 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 >> v1:172.21.15.27:6789/0 conn(0x7fce9415ae80 legacy=0x7fce940b2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce91ffb640 1 --2- 172.21.15.27:0/3991703139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce800039a0 0x7fce80003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce91ffb640 1 --2- 172.21.15.27:0/3991703139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce94163b00 0x7fce800047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce91ffb640 1 --2- 172.21.15.27:0/3991703139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce80004d00 0x7fce80005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce940b3630 con 0x7fce800039a0 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce8000ff70 con 0x7fce80004d00 2023-12-19T18:26:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce80010130 con 0x7fce94163b00 2023-12-19T18:26:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce92ffd640 1 --2- 172.21.15.27:0/3991703139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce800039a0 0x7fce80003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.729+0000 7fce927fc640 1 --2- 172.21.15.27:0/3991703139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce94163b00 0x7fce800047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.730+0000 7fce937fe640 1 --2- 172.21.15.27:0/3991703139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce80004d00 0x7fce80005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.730+0000 7fce92ffd640 1 -- 172.21.15.27:0/3991703139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce94163b00 msgr2=0x7fce800047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.730+0000 7fce92ffd640 1 --2- 172.21.15.27:0/3991703139 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce94163b00 0x7fce800047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.730+0000 7fce92ffd640 1 -- 172.21.15.27:0/3991703139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce80004d00 msgr2=0x7fce80005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:14.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.730+0000 7fce92ffd640 1 --2- 172.21.15.27:0/3991703139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce80004d00 0x7fce80005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.730+0000 7fce92ffd640 1 -- 172.21.15.27:0/3991703139 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce80003520 con 0x7fce800039a0 2023-12-19T18:26:14.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.730+0000 7fce937fe640 1 --2- 172.21.15.27:0/3991703139 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce80004d00 0x7fce80005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.731+0000 7fce92ffd640 1 --2- 172.21.15.27:0/3991703139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce800039a0 0x7fce80003dc0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fce8400c0e0 tx=0x7fce800057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d759c5332fe4a82 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.732+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce84005ad0 con 0x7fce800039a0 2023-12-19T18:26:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.732+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce84005c30 con 0x7fce800039a0 2023-12-19T18:26:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.732+0000 7fce91ffb640 1 -- 172.21.15.27:0/3991703139 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce8402c810 con 0x7fce800039a0 2023-12-19T18:26:14.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.732+0000 7fce99589640 1 -- 172.21.15.27:0/3991703139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce800039a0 msgr2=0x7fce80003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:14.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.732+0000 7fce99589640 1 --2- 172.21.15.27:0/3991703139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce800039a0 0x7fce80003dc0 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fce8400c0e0 tx=0x7fce800057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.732+0000 7fce937fe640 1 -- 172.21.15.27:0/3991703139 reap_dead start 2023-12-19T18:26:14.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.732+0000 7fce99589640 1 -- 172.21.15.27:0/3991703139 shutdown_connections 2023-12-19T18:26:14.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.732+0000 7fce99589640 1 -- 172.21.15.27:0/3991703139 >> 172.21.15.27:0/3991703139 conn(0x7fce9406f1f0 msgr2=0x7fce9406f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:14.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.732+0000 7fce99589640 1 -- 172.21.15.27:0/3991703139 shutdown_connections 2023-12-19T18:26:14.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.732+0000 7fce99589640 1 -- 172.21.15.27:0/3991703139 wait complete. 2023-12-19T18:26:14.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.734+0000 7fce99589640 1 Processor -- start 2023-12-19T18:26:14.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.734+0000 7fce99589640 1 -- start start 2023-12-19T18:26:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.735+0000 7fce99589640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce800039a0 0x7fce940c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.735+0000 7fce99589640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce94148ef0 0x7fce940ccb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.735+0000 7fce99589640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce940cd0a0 0x7fce94154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.735+0000 7fce99589640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce9414ff10 con 0x7fce94148ef0 2023-12-19T18:26:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.735+0000 7fce99589640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce940c5aa0 con 0x7fce800039a0 2023-12-19T18:26:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.735+0000 7fce99589640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce940cd520 con 0x7fce940cd0a0 2023-12-19T18:26:14.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.736+0000 7fce927fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce94148ef0 0x7fce940ccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.736+0000 7fce927fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce94148ef0 0x7fce940ccb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35226/0 (socket says 172.21.15.27:35226) 2023-12-19T18:26:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.736+0000 7fce927fc640 1 -- 172.21.15.27:0/2543059880 learned_addr learned my addr 172.21.15.27:0/2543059880 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.736+0000 7fce92ffd640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce800039a0 0x7fce940c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.736+0000 7fce937fe640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce940cd0a0 0x7fce94154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.736+0000 7fce927fc640 1 -- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce940cd0a0 msgr2=0x7fce94154670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.736+0000 7fce927fc640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce940cd0a0 0x7fce94154670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.736+0000 7fce927fc640 1 -- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce800039a0 msgr2=0x7fce940c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.736+0000 7fce927fc640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce800039a0 0x7fce940c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.736+0000 7fce927fc640 1 -- 172.21.15.27:0/2543059880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce9414fc30 con 0x7fce94148ef0 2023-12-19T18:26:14.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.736+0000 7fce937fe640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce940cd0a0 0x7fce94154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.737+0000 7fce927fc640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce94148ef0 0x7fce940ccb60 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00fb90 tx=0x7fce7c00d520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.737+0000 7fce8b7fe640 1 -- 172.21.15.27:0/2543059880 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c001a60 con 0x7fce94148ef0 2023-12-19T18:26:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.737+0000 7fce8b7fe640 1 -- 172.21.15.27:0/2543059880 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7c00fe90 con 0x7fce94148ef0 2023-12-19T18:26:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.737+0000 7fce8b7fe640 1 -- 172.21.15.27:0/2543059880 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7c00de10 con 0x7fce94148ef0 2023-12-19T18:26:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.737+0000 7fce99589640 1 -- 172.21.15.27:0/2543059880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce940c53e0 con 0x7fce94148ef0 2023-12-19T18:26:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.737+0000 7fce99589640 1 -- 172.21.15.27:0/2543059880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce9414f850 con 0x7fce94148ef0 2023-12-19T18:26:14.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.738+0000 7fce99589640 1 -- 172.21.15.27:0/2543059880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce940c5240 con 0x7fce94148ef0 2023-12-19T18:26:14.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.739+0000 7fce8b7fe640 1 -- 172.21.15.27:0/2543059880 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fce7c003970 con 0x7fce94148ef0 2023-12-19T18:26:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.740+0000 7fce8b7fe640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce7406a430 0x7fce7406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.740+0000 7fce8b7fe640 1 -- 172.21.15.27:0/2543059880 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fce7c088b60 con 0x7fce94148ef0 2023-12-19T18:26:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.740+0000 7fce8b7fe640 1 -- 172.21.15.27:0/2543059880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fce74071f60 con 0x7fce94148ef0 2023-12-19T18:26:14.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.741+0000 7fce92ffd640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce7406a430 0x7fce7406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:14.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.742+0000 7fce92ffd640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce7406a430 0x7fce7406c8f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fce84034000 tx=0x7fce84009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:14.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.744+0000 7fce8b7fe640 1 -- 172.21.15.27:0/2543059880 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fce940c5240 con 0x7fce94148ef0 2023-12-19T18:26:14.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.938+0000 7fce99589640 1 -- 172.21.15.27:0/2543059880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce940c5240 con 0x7fce94148ef0 2023-12-19T18:26:14.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.939+0000 7fce8b7fe640 1 -- 172.21.15.27:0/2543059880 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v172) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce940c5240 con 0x7fce94148ef0 2023-12-19T18:26:14.939 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:26:14.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.941+0000 7fce99589640 1 -- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce7406a430 msgr2=0x7fce7406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.941+0000 7fce99589640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce7406a430 0x7fce7406c8f0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fce84034000 tx=0x7fce84009030 comp rx=0 tx=0).stop 2023-12-19T18:26:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.941+0000 7fce99589640 1 -- 172.21.15.27:0/2543059880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce94148ef0 msgr2=0x7fce940ccb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.941+0000 7fce99589640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce94148ef0 0x7fce940ccb60 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fce7c00fb90 tx=0x7fce7c00d520 comp rx=0 tx=0).stop 2023-12-19T18:26:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.941+0000 7fce99589640 1 -- 172.21.15.27:0/2543059880 shutdown_connections 2023-12-19T18:26:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.941+0000 7fce99589640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fce7406a430 0x7fce7406c8f0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.941+0000 7fce99589640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fce940cd0a0 0x7fce94154670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.941+0000 7fce99589640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce94148ef0 0x7fce940ccb60 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.941+0000 7fce99589640 1 --2- 172.21.15.27:0/2543059880 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fce800039a0 0x7fce940c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.941+0000 7fce99589640 1 -- 172.21.15.27:0/2543059880 >> 172.21.15.27:0/2543059880 conn(0x7fce9406f1f0 msgr2=0x7fce940b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.942+0000 7fce99589640 1 -- 172.21.15.27:0/2543059880 shutdown_connections 2023-12-19T18:26:14.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:14.942+0000 7fce99589640 1 -- 172.21.15.27:0/2543059880 wait complete. 2023-12-19T18:26:14.960 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:26:15.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.190+0000 7ff5c29fd640 1 Processor -- start 2023-12-19T18:26:15.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.190+0000 7ff5c29fd640 1 -- start start 2023-12-19T18:26:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.191+0000 7ff5c29fd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff5bc15aed0 0x7ff5bc0cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.191+0000 7ff5c29fd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5bc0cdaa0 0x7ff5bc0b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.191+0000 7ff5c29fd640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5bc0b3fd0 con 0x7ff5bc0d1450 2023-12-19T18:26:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.191+0000 7ff5c29fd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5bc0cf6f0 con 0x7ff5bc15aed0 2023-12-19T18:26:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.191+0000 7ff5c29fd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc0cf9d0 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.192+0000 7ff5c19fb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff5bc15aed0 0x7ff5bc0cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:15.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.192+0000 7ff5c19fb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff5bc15aed0 0x7ff5bc0cf110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35228/0 (socket says 172.21.15.27:35228) 2023-12-19T18:26:15.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.192+0000 7ff5c19fb640 1 -- 172.21.15.27:0/149066386 learned_addr learned my addr 172.21.15.27:0/149066386 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:15.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.192+0000 7ff5c19fb640 1 -- 172.21.15.27:0/149066386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5bc0cdaa0 msgr2=0x7ff5bc0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:15.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.192+0000 7ff5c19fb640 1 --2- 172.21.15.27:0/149066386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5bc0cdaa0 0x7ff5bc0b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.192+0000 7ff5c19fb640 1 -- 172.21.15.27:0/149066386 >> v1:172.21.15.192:6789/0 conn(0x7ff5bc0d1450 legacy=0x7ff5bc0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:15.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.192+0000 7ff5c19fb640 1 -- 172.21.15.27:0/149066386 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc0b3980 con 0x7ff5bc15aed0 2023-12-19T18:26:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.193+0000 7ff5c19fb640 1 --2- 172.21.15.27:0/149066386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5bc15aed0 0x7ff5bc0cf110 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009870 tx=0x7ff5b002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5277d58d293388 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.194+0000 7ff5c09f9640 1 -- 172.21.15.27:0/149066386 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00317c0 con 0x7ff5bc15aed0 2023-12-19T18:26:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.194+0000 7ff5c09f9640 1 -- 172.21.15.27:0/149066386 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b0031920 con 0x7ff5bc15aed0 2023-12-19T18:26:15.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.194+0000 7ff5c09f9640 1 -- 172.21.15.27:0/149066386 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b0031cb0 con 0x7ff5bc15aed0 2023-12-19T18:26:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.194+0000 7ff5c29fd640 1 -- 172.21.15.27:0/149066386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5bc15aed0 msgr2=0x7ff5bc0cf110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.194+0000 7ff5c29fd640 1 --2- 172.21.15.27:0/149066386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5bc15aed0 0x7ff5bc0cf110 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009870 tx=0x7ff5b002b860 comp rx=0 tx=0).stop 2023-12-19T18:26:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.194+0000 7ff5c29fd640 1 -- 172.21.15.27:0/149066386 shutdown_connections 2023-12-19T18:26:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.194+0000 7ff5c29fd640 1 --2- 172.21.15.27:0/149066386 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5bc0cdaa0 0x7ff5bc0b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.194+0000 7ff5c29fd640 1 --2- 172.21.15.27:0/149066386 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5bc15aed0 0x7ff5bc0cf110 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.194+0000 7ff5c29fd640 1 -- 172.21.15.27:0/149066386 >> 172.21.15.27:0/149066386 conn(0x7ff5bc099ee0 msgr2=0x7ff5bc0d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:15.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.194+0000 7ff5c29fd640 1 -- 172.21.15.27:0/149066386 shutdown_connections 2023-12-19T18:26:15.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.194+0000 7ff5c29fd640 1 -- 172.21.15.27:0/149066386 wait complete. 2023-12-19T18:26:15.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.196+0000 7ff5c29fd640 1 Processor -- start 2023-12-19T18:26:15.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.196+0000 7ff5c29fd640 1 -- start start 2023-12-19T18:26:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.197+0000 7ff5c29fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5bc0cdaa0 0x7ff5bc0ca620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.197+0000 7ff5c29fd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5bc0d1450 0x7ff5bc152920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.197+0000 7ff5c29fd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0b5cd0 0x7ff5bc0b6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.197+0000 7ff5c29fd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc0b66c0 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.197+0000 7ff5c29fd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc152e60 con 0x7ff5bc0b5cd0 2023-12-19T18:26:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.197+0000 7ff5c29fd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5bc06b3f0 con 0x7ff5bc0d1450 2023-12-19T18:26:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.198+0000 7ff5c19fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5bc0cdaa0 0x7ff5bc0ca620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.198+0000 7ff5c19fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5bc0cdaa0 0x7ff5bc0ca620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35242/0 (socket says 172.21.15.27:35242) 2023-12-19T18:26:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.198+0000 7ff5c19fb640 1 -- 172.21.15.27:0/237873647 learned_addr learned my addr 172.21.15.27:0/237873647 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.198+0000 7ff5c21fc640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0b5cd0 0x7ff5bc0b6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.198+0000 7ff5c19fb640 1 -- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5bc0d1450 msgr2=0x7ff5bc152920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.198+0000 7ff5c19fb640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5bc0d1450 0x7ff5bc152920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.198+0000 7ff5c19fb640 1 -- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0b5cd0 msgr2=0x7ff5bc0b6180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.198+0000 7ff5c19fb640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0b5cd0 0x7ff5bc0b6180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.198+0000 7ff5c19fb640 1 -- 172.21.15.27:0/237873647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5b00094d0 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.198+0000 7ff5c21fc640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0b5cd0 0x7ff5bc0b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.198+0000 7ff5c19fb640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5bc0cdaa0 0x7ff5bc0ca620 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7ff5b002b8a0 tx=0x7ff5b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.199+0000 7ff5ae7fc640 1 -- 172.21.15.27:0/237873647 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00329a0 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.199+0000 7ff5ae7fc640 1 -- 172.21.15.27:0/237873647 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b0032b20 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.199+0000 7ff5ae7fc640 1 -- 172.21.15.27:0/237873647 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b00453f0 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.199+0000 7ff5c29fd640 1 -- 172.21.15.27:0/237873647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5bc06b690 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.199+0000 7ff5c29fd640 1 -- 172.21.15.27:0/237873647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5bc06bc50 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.200+0000 7ff5c29fd640 1 -- 172.21.15.27:0/237873647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5bc04e840 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.201+0000 7ff5ae7fc640 1 -- 172.21.15.27:0/237873647 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff5b004e450 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.202+0000 7ff5ae7fc640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5a006a430 0x7ff5a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.202+0000 7ff5ae7fc640 1 -- 172.21.15.27:0/237873647 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5b00aadd0 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.203+0000 7ff5ae7fc640 1 -- 172.21.15.27:0/237873647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7ff5a0071f60 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.203+0000 7ff5c11fa640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5a006a430 0x7ff5a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:15.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.204+0000 7ff5c11fa640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5a006a430 0x7ff5a006c8f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc09ea90 tx=0x7ff5b80079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:15.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.206+0000 7ff5ae7fc640 1 -- 172.21.15.27:0/237873647 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff5b0076440 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.404+0000 7ff5c29fd640 1 -- 172.21.15.27:0/237873647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff5bc0ce900 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.405+0000 7ff5ae7fc640 1 -- 172.21.15.27:0/237873647 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5b0079a60 con 0x7ff5bc0cdaa0 2023-12-19T18:26:15.405 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:26:15.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.408+0000 7ff5c29fd640 1 -- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5a006a430 msgr2=0x7ff5a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.408+0000 7ff5c29fd640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5a006a430 0x7ff5a006c8f0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc09ea90 tx=0x7ff5b80079b0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.408+0000 7ff5c29fd640 1 -- 172.21.15.27:0/237873647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5bc0cdaa0 msgr2=0x7ff5bc0ca620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.408+0000 7ff5c29fd640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5bc0cdaa0 0x7ff5bc0ca620 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7ff5b002b8a0 tx=0x7ff5b00046c0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.408+0000 7ff5c29fd640 1 -- 172.21.15.27:0/237873647 shutdown_connections 2023-12-19T18:26:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.408+0000 7ff5c29fd640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5a006a430 0x7ff5a006c8f0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.408+0000 7ff5c29fd640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5bc0b5cd0 0x7ff5bc0b6180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.408+0000 7ff5c29fd640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5bc0d1450 0x7ff5bc152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.408+0000 7ff5c29fd640 1 --2- 172.21.15.27:0/237873647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5bc0cdaa0 0x7ff5bc0ca620 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.408+0000 7ff5c29fd640 1 -- 172.21.15.27:0/237873647 >> 172.21.15.27:0/237873647 conn(0x7ff5bc099ee0 msgr2=0x7ff5bc0cb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:15.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.408+0000 7ff5c29fd640 1 -- 172.21.15.27:0/237873647 shutdown_connections 2023-12-19T18:26:15.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.408+0000 7ff5c29fd640 1 -- 172.21.15.27:0/237873647 wait complete. 2023-12-19T18:26:15.426 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:26:15.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.661+0000 7f01a0ef3640 1 Processor -- start 2023-12-19T18:26:15.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.662+0000 7f01a0ef3640 1 -- start start 2023-12-19T18:26:15.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.663+0000 7f01a0ef3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f019c15cf90 0x7f019c167c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.663+0000 7f01a0ef3640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c0b5ca0 con 0x7f019c0b49d0 2023-12-19T18:26:15.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.663+0000 7f01a0ef3640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c161800 con 0x7f019c0b89b0 2023-12-19T18:26:15.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.663+0000 7f01a0ef3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019c161050 con 0x7f019c15cf90 2023-12-19T18:26:15.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.664+0000 7f019b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f019c0b49d0 0x7f019c160940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40054/0 (socket says 172.21.15.27:40054) 2023-12-19T18:26:15.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.664+0000 7f019b7fe640 1 -- 172.21.15.27:0/3362055635 learned_addr learned my addr 172.21.15.27:0/3362055635 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.664+0000 7f019bfff640 1 --2- 172.21.15.27:0/3362055635 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f019c15cf90 0x7f019c167c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.664+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2722101707 0 0) 0x7f019c0b5ca0 con 0x7f019c0b49d0 2023-12-19T18:26:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.664+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0188003520 con 0x7f019c0b49d0 2023-12-19T18:26:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1152859569 0 0) 0x7f0188003520 con 0x7f019c0b49d0 2023-12-19T18:26:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f019c15cf90 msgr2=0x7f019c167c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019affd640 1 --2- 172.21.15.27:0/3362055635 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f019c15cf90 0x7f019c167c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 >> v1:172.21.15.192:6789/0 conn(0x7f019c0b89b0 legacy=0x7f019c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c161240 con 0x7f019c0b49d0 2023-12-19T18:26:15.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f018c0017f0 con 0x7f019c0b49d0 2023-12-19T18:26:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 >> v1:172.21.15.27:6789/0 conn(0x7f019c0b49d0 legacy=0x7f019c160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019affd640 1 --2- 172.21.15.27:0/3362055635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01880039a0 0x7f0188003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019affd640 1 --2- 172.21.15.27:0/3362055635 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f019c15cf90 0x7f01880047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019affd640 1 --2- 172.21.15.27:0/3362055635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0188004d00 0x7f0188005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018800b980 con 0x7f0188004d00 2023-12-19T18:26:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018800bac0 con 0x7f01880039a0 2023-12-19T18:26:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018800bc80 con 0x7f019c15cf90 2023-12-19T18:26:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.665+0000 7f019bfff640 1 --2- 172.21.15.27:0/3362055635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0188004d00 0x7f0188005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.666+0000 7f019b7fe640 1 --2- 172.21.15.27:0/3362055635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01880039a0 0x7f0188003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.666+0000 7f0193fff640 1 --2- 172.21.15.27:0/3362055635 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f019c15cf90 0x7f01880047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.666+0000 7f019bfff640 1 -- 172.21.15.27:0/3362055635 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f019c15cf90 msgr2=0x7f01880047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.666+0000 7f019bfff640 1 --2- 172.21.15.27:0/3362055635 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f019c15cf90 0x7f01880047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.666+0000 7f019bfff640 1 -- 172.21.15.27:0/3362055635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01880039a0 msgr2=0x7f0188003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:15.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.666+0000 7f019bfff640 1 --2- 172.21.15.27:0/3362055635 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01880039a0 0x7f0188003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.666+0000 7f019bfff640 1 -- 172.21.15.27:0/3362055635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0188003520 con 0x7f0188004d00 2023-12-19T18:26:15.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.667+0000 7f019bfff640 1 --2- 172.21.15.27:0/3362055635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0188004d00 0x7f0188005180 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f0194009fc0 tx=0x7f01880057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33edb338256c2ec0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:15.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.667+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019402cc20 con 0x7f0188004d00 2023-12-19T18:26:15.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.667+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f019400f040 con 0x7f0188004d00 2023-12-19T18:26:15.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.667+0000 7f019affd640 1 -- 172.21.15.27:0/3362055635 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194034440 con 0x7f0188004d00 2023-12-19T18:26:15.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.668+0000 7f01a0ef3640 1 -- 172.21.15.27:0/3362055635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0188004d00 msgr2=0x7f0188005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:15.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.668+0000 7f01a0ef3640 1 --2- 172.21.15.27:0/3362055635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0188004d00 0x7f0188005180 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f0194009fc0 tx=0x7f01880057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.668+0000 7f019bfff640 1 -- 172.21.15.27:0/3362055635 reap_dead start 2023-12-19T18:26:15.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.668+0000 7f01a0ef3640 1 -- 172.21.15.27:0/3362055635 shutdown_connections 2023-12-19T18:26:15.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.668+0000 7f01a0ef3640 1 -- 172.21.15.27:0/3362055635 >> 172.21.15.27:0/3362055635 conn(0x7f019c0cf760 msgr2=0x7f019c0cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:15.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.668+0000 7f01a0ef3640 1 -- 172.21.15.27:0/3362055635 shutdown_connections 2023-12-19T18:26:15.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.668+0000 7f01a0ef3640 1 -- 172.21.15.27:0/3362055635 wait complete. 2023-12-19T18:26:15.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.670+0000 7f01a0ef3640 1 Processor -- start 2023-12-19T18:26:15.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.670+0000 7f01a0ef3640 1 -- start start 2023-12-19T18:26:15.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.671+0000 7f01a0ef3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0188004d00 0x7f019c0c5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.671+0000 7f01a0ef3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f019c0b6280 0x7f019c0b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.671+0000 7f01a0ef3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0c8700 0x7f019c0c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.671+0000 7f01a0ef3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0ca1f0 con 0x7f019c0c8700 2023-12-19T18:26:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.671+0000 7f01a0ef3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0b6c00 con 0x7f0188004d00 2023-12-19T18:26:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.671+0000 7f01a0ef3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0cc190 con 0x7f019c0b6280 2023-12-19T18:26:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.671+0000 7f019bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0c8700 0x7f019c0c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.671+0000 7f019bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0c8700 0x7f019c0c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35250/0 (socket says 172.21.15.27:35250) 2023-12-19T18:26:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.671+0000 7f019bfff640 1 -- 172.21.15.27:0/118081674 learned_addr learned my addr 172.21.15.27:0/118081674 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.672+0000 7f019b7fe640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0188004d00 0x7f019c0c5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.672+0000 7f0193fff640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f019c0b6280 0x7f019c0b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.672+0000 7f019bfff640 1 -- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f019c0b6280 msgr2=0x7f019c0b66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.672+0000 7f019bfff640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f019c0b6280 0x7f019c0b66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.672+0000 7f019bfff640 1 -- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0188004d00 msgr2=0x7f019c0c5110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:15.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.672+0000 7f019bfff640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0188004d00 0x7f019c0c5110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.672+0000 7f019bfff640 1 -- 172.21.15.27:0/118081674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c0c9450 con 0x7f019c0c8700 2023-12-19T18:26:15.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.672+0000 7f019bfff640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0c8700 0x7f019c0c8b80 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f0194002230 tx=0x7f0194009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:15.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.672+0000 7f0198ff9640 1 -- 172.21.15.27:0/118081674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194043070 con 0x7f019c0c8700 2023-12-19T18:26:15.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.673+0000 7f0198ff9640 1 -- 172.21.15.27:0/118081674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0194034ad0 con 0x7f019c0c8700 2023-12-19T18:26:15.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.673+0000 7f01a0ef3640 1 -- 172.21.15.27:0/118081674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019c0ca400 con 0x7f019c0c8700 2023-12-19T18:26:15.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.673+0000 7f0198ff9640 1 -- 172.21.15.27:0/118081674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0194034e80 con 0x7f019c0c8700 2023-12-19T18:26:15.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.673+0000 7f01a0ef3640 1 -- 172.21.15.27:0/118081674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019c0ca9c0 con 0x7f019c0c8700 2023-12-19T18:26:15.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.674+0000 7f01a0ef3640 1 -- 172.21.15.27:0/118081674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019c04e840 con 0x7f019c0c8700 2023-12-19T18:26:15.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.675+0000 7f0198ff9640 1 -- 172.21.15.27:0/118081674 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f019403e440 con 0x7f019c0c8700 2023-12-19T18:26:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.676+0000 7f0198ff9640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f017c06a430 0x7f017c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.676+0000 7f0198ff9640 1 -- 172.21.15.27:0/118081674 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01940aa560 con 0x7f019c0c8700 2023-12-19T18:26:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.676+0000 7f0198ff9640 1 -- 172.21.15.27:0/118081674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f017c071f60 con 0x7f019c0c8700 2023-12-19T18:26:15.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.677+0000 7f019b7fe640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f017c06a430 0x7f017c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:15.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.678+0000 7f019b7fe640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f017c06a430 0x7f017c06c8f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f018c001920 tx=0x7f018c001710 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:15.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.680+0000 7f0198ff9640 1 -- 172.21.15.27:0/118081674 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0194075b50 con 0x7f019c0c8700 2023-12-19T18:26:15.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.890+0000 7f01a0ef3640 1 -- 172.21.15.27:0/118081674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f019c0d08b0 con 0x7f019c0c8700 2023-12-19T18:26:15.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.891+0000 7f0198ff9640 1 -- 172.21.15.27:0/118081674 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v172) v1 ==== 80+0+0 (secure 0 0 0) 0x7f0194039070 con 0x7f019c0c8700 2023-12-19T18:26:15.891 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:26:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.894+0000 7f01a0ef3640 1 -- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f017c06a430 msgr2=0x7f017c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.894+0000 7f01a0ef3640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f017c06a430 0x7f017c06c8f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f018c001920 tx=0x7f018c001710 comp rx=0 tx=0).stop 2023-12-19T18:26:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.894+0000 7f01a0ef3640 1 -- 172.21.15.27:0/118081674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0c8700 msgr2=0x7f019c0c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.894+0000 7f01a0ef3640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0c8700 0x7f019c0c8b80 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f0194002230 tx=0x7f0194009750 comp rx=0 tx=0).stop 2023-12-19T18:26:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.894+0000 7f01a0ef3640 1 -- 172.21.15.27:0/118081674 shutdown_connections 2023-12-19T18:26:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.894+0000 7f01a0ef3640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f017c06a430 0x7f017c06c8f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.894+0000 7f01a0ef3640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f019c0c8700 0x7f019c0c8b80 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.894+0000 7f01a0ef3640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f019c0b6280 0x7f019c0b66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.894+0000 7f01a0ef3640 1 --2- 172.21.15.27:0/118081674 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0188004d00 0x7f019c0c5110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.894+0000 7f01a0ef3640 1 -- 172.21.15.27:0/118081674 >> 172.21.15.27:0/118081674 conn(0x7f019c0cf760 msgr2=0x7f019c0b7510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.894+0000 7f01a0ef3640 1 -- 172.21.15.27:0/118081674 shutdown_connections 2023-12-19T18:26:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:15.894+0000 7f01a0ef3640 1 -- 172.21.15.27:0/118081674 wait complete. 2023-12-19T18:26:15.915 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-12-19T18:26:15.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:15.915 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-19T18:26:16.016 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[116729]: fuse finished with error 0 and tester_r 0 2023-12-19T18:26:16.023 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:26:16.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:16.023 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:26:16.205 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:26:16.205 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:26:16.264 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:26:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.514+0000 7fc0c4638640 1 Processor -- start 2023-12-19T18:26:16.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.514+0000 7fc0c4638640 1 -- start start 2023-12-19T18:26:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.516+0000 7fc0c4638640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc0bc15af70 0x7fc0bc0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.516+0000 7fc0c4638640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0bc0cbce0 0x7fc0bc0ad510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.516+0000 7fc0c4638640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0bc0cb850 con 0x7fc0bc0ada50 2023-12-19T18:26:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.516+0000 7fc0c4638640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0bc15b350 con 0x7fc0bc15af70 2023-12-19T18:26:16.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.516+0000 7fc0c4638640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc0cac40 con 0x7fc0bc0cbce0 2023-12-19T18:26:16.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.517+0000 7fc0c1bd3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0bc0cbce0 0x7fc0bc0ad510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:16.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.517+0000 7fc0c23d4640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc0bc15af70 0x7fc0bc0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:16.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.517+0000 7fc0c1bd3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0bc0cbce0 0x7fc0bc0ad510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:56798/0 (socket says 172.21.15.27:56798) 2023-12-19T18:26:16.517 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.517+0000 7fc0c1bd3640 1 -- 172.21.15.27:0/1440142557 learned_addr learned my addr 172.21.15.27:0/1440142557 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:16.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.518+0000 7fc0c1bd3640 1 -- 172.21.15.27:0/1440142557 >> v2:172.21.15.192:3300/0 conn(0x7fc0bc15af70 msgr2=0x7fc0bc0ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:16.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.518+0000 7fc0c1bd3640 1 --2- 172.21.15.27:0/1440142557 >> v2:172.21.15.192:3300/0 conn(0x7fc0bc15af70 0x7fc0bc0ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:16.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.518+0000 7fc0c1bd3640 1 -- 172.21.15.27:0/1440142557 >> v1:172.21.15.192:6789/0 conn(0x7fc0bc0ada50 legacy=0x7fc0bc0c9190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:16.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.518+0000 7fc0c13d2640 1 -- 172.21.15.27:0/1440142557 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2087353641 0 0) 0x7fc0bc0cb850 con 0x7fc0bc0ada50 2023-12-19T18:26:16.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.518+0000 7fc0c1bd3640 1 -- 172.21.15.27:0/1440142557 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc0caf20 con 0x7fc0bc0cbce0 2023-12-19T18:26:16.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.519+0000 7fc0c1bd3640 1 --2- 172.21.15.27:0/1440142557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0bc0cbce0 0x7fc0bc0ad510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009870 tx=0x7fc0ac02b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=92dc05340a457f37 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:16.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.520+0000 7fc0c13d2640 1 -- 172.21.15.27:0/1440142557 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac030840 con 0x7fc0bc0cbce0 2023-12-19T18:26:16.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.520+0000 7fc0c13d2640 1 -- 172.21.15.27:0/1440142557 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ac0309a0 con 0x7fc0bc0cbce0 2023-12-19T18:26:16.520 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.520+0000 7fc0c13d2640 1 -- 172.21.15.27:0/1440142557 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac030d30 con 0x7fc0bc0cbce0 2023-12-19T18:26:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.521+0000 7fc0c4638640 1 -- 172.21.15.27:0/1440142557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0bc0cbce0 msgr2=0x7fc0bc0ad510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.521+0000 7fc0c4638640 1 --2- 172.21.15.27:0/1440142557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0bc0cbce0 0x7fc0bc0ad510 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac009870 tx=0x7fc0ac02b980 comp rx=0 tx=0).stop 2023-12-19T18:26:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.521+0000 7fc0c4638640 1 -- 172.21.15.27:0/1440142557 shutdown_connections 2023-12-19T18:26:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.521+0000 7fc0c4638640 1 --2- 172.21.15.27:0/1440142557 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0bc0cbce0 0x7fc0bc0ad510 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.521+0000 7fc0c4638640 1 --2- 172.21.15.27:0/1440142557 >> v2:172.21.15.192:3300/0 conn(0x7fc0bc15af70 0x7fc0bc0ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.521+0000 7fc0c4638640 1 -- 172.21.15.27:0/1440142557 >> 172.21.15.27:0/1440142557 conn(0x7fc0bc06a330 msgr2=0x7fc0bc0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.521+0000 7fc0c4638640 1 -- 172.21.15.27:0/1440142557 shutdown_connections 2023-12-19T18:26:16.521 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.521+0000 7fc0c4638640 1 -- 172.21.15.27:0/1440142557 wait complete. 2023-12-19T18:26:16.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.523+0000 7fc0c4638640 1 Processor -- start 2023-12-19T18:26:16.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.523+0000 7fc0c4638640 1 -- start start 2023-12-19T18:26:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.524+0000 7fc0c4638640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0bc0ada50 0x7fc0bc1500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.524+0000 7fc0c4638640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0bc0cbce0 0x7fc0bc0c4940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.524+0000 7fc0c4638640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0bc15af70 0x7fc0bc0ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.524+0000 7fc0c4638640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc150630 con 0x7fc0bc15af70 2023-12-19T18:26:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.524+0000 7fc0c4638640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc0c4e80 con 0x7fc0bc0cbce0 2023-12-19T18:26:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.524+0000 7fc0c4638640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0bc0ccad0 con 0x7fc0bc0ada50 2023-12-19T18:26:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.524+0000 7fc0c2bd5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0bc15af70 0x7fc0bc0ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.524+0000 7fc0c2bd5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0bc15af70 0x7fc0bc0ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35258/0 (socket says 172.21.15.27:35258) 2023-12-19T18:26:16.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.524+0000 7fc0c2bd5640 1 -- 172.21.15.27:0/3220209178 learned_addr learned my addr 172.21.15.27:0/3220209178 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.524+0000 7fc0c23d4640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0bc0ada50 0x7fc0bc1500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.524+0000 7fc0c1bd3640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0bc0cbce0 0x7fc0bc0c4940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.525+0000 7fc0c2bd5640 1 -- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0bc0ada50 msgr2=0x7fc0bc1500f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.525+0000 7fc0c2bd5640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0bc0ada50 0x7fc0bc1500f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.525+0000 7fc0c2bd5640 1 -- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0bc0cbce0 msgr2=0x7fc0bc0c4940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.525+0000 7fc0c2bd5640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0bc0cbce0 0x7fc0bc0c4940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.525+0000 7fc0c2bd5640 1 -- 172.21.15.27:0/3220209178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ac0094d0 con 0x7fc0bc15af70 2023-12-19T18:26:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.525+0000 7fc0c2bd5640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0bc15af70 0x7fc0bc0ac8d0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c760 tx=0x7fc0b400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.525+0000 7fc0b2ffd640 1 -- 172.21.15.27:0/3220209178 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4003da0 con 0x7fc0bc15af70 2023-12-19T18:26:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.525+0000 7fc0b2ffd640 1 -- 172.21.15.27:0/3220209178 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0b400ce70 con 0x7fc0bc15af70 2023-12-19T18:26:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.525+0000 7fc0b2ffd640 1 -- 172.21.15.27:0/3220209178 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0b4005290 con 0x7fc0bc15af70 2023-12-19T18:26:16.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.525+0000 7fc0c4638640 1 -- 172.21.15.27:0/3220209178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0bc0ccdb0 con 0x7fc0bc15af70 2023-12-19T18:26:16.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.526+0000 7fc0c4638640 1 -- 172.21.15.27:0/3220209178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0bc0cd370 con 0x7fc0bc15af70 2023-12-19T18:26:16.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.527+0000 7fc0c4638640 1 -- 172.21.15.27:0/3220209178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0bc068e90 con 0x7fc0bc15af70 2023-12-19T18:26:16.527 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.528+0000 7fc0b2ffd640 1 -- 172.21.15.27:0/3220209178 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc0b40053f0 con 0x7fc0bc15af70 2023-12-19T18:26:16.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.529+0000 7fc0b2ffd640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0a006a460 0x7fc0a006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:16.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.529+0000 7fc0b2ffd640 1 -- 172.21.15.27:0/3220209178 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc0b40886f0 con 0x7fc0bc15af70 2023-12-19T18:26:16.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.529+0000 7fc0b2ffd640 1 -- 172.21.15.27:0/3220209178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fc0a0071f90 con 0x7fc0bc15af70 2023-12-19T18:26:16.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.529+0000 7fc0c23d4640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0a006a460 0x7fc0a006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:16.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.530+0000 7fc0c23d4640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0a006a460 0x7fc0a006c920 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009aa0 tx=0x7fc0b8009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:16.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.533+0000 7fc0b2ffd640 1 -- 172.21.15.27:0/3220209178 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc0b4053c40 con 0x7fc0bc15af70 2023-12-19T18:26:16.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.734+0000 7fc0c4638640 1 -- 172.21.15.27:0/3220209178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0bc0ca260 con 0x7fc0bc15af70 2023-12-19T18:26:16.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.736+0000 7fc0b2ffd640 1 -- 172.21.15.27:0/3220209178 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16086 (secure 0 0 0) 0x7fc0b4057260 con 0x7fc0bc15af70 2023-12-19T18:26:16.736 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:16.736 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":172,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:25:26.756190+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:24:06.280682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T18:24:07.374789+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:26:16.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.738+0000 7fc0c4638640 1 -- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0a006a460 msgr2=0x7fc0a006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:16.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.738+0000 7fc0c4638640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0a006a460 0x7fc0a006c920 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fc0b8009aa0 tx=0x7fc0b8009190 comp rx=0 tx=0).stop 2023-12-19T18:26:16.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.739+0000 7fc0c4638640 1 -- 172.21.15.27:0/3220209178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0bc15af70 msgr2=0x7fc0bc0ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:16.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.739+0000 7fc0c4638640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0bc15af70 0x7fc0bc0ac8d0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7fc0b400c760 tx=0x7fc0b400cc20 comp rx=0 tx=0).stop 2023-12-19T18:26:16.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.739+0000 7fc0c4638640 1 -- 172.21.15.27:0/3220209178 shutdown_connections 2023-12-19T18:26:16.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.739+0000 7fc0c4638640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0a006a460 0x7fc0a006c920 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:16.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.739+0000 7fc0c4638640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0bc15af70 0x7fc0bc0ac8d0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:16.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.739+0000 7fc0c4638640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0bc0cbce0 0x7fc0bc0c4940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:16.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.739+0000 7fc0c4638640 1 --2- 172.21.15.27:0/3220209178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0bc0ada50 0x7fc0bc1500f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:16.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.739+0000 7fc0c4638640 1 -- 172.21.15.27:0/3220209178 >> 172.21.15.27:0/3220209178 conn(0x7fc0bc06a330 msgr2=0x7fc0bc070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:16.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.739+0000 7fc0c4638640 1 -- 172.21.15.27:0/3220209178 shutdown_connections 2023-12-19T18:26:16.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.739+0000 7fc0c4638640 1 -- 172.21.15.27:0/3220209178 wait complete. 2023-12-19T18:26:16.758 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:24:06.280682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:24:07.374789+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:26:16.759 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:26:16.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.991+0000 7ff5cf3b3640 1 Processor -- start 2023-12-19T18:26:16.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.991+0000 7ff5cf3b3640 1 -- start start 2023-12-19T18:26:16.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.992+0000 7ff5cf3b3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c80d2100 0x7ff5c80cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:16.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.992+0000 7ff5cf3b3640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff5c80cdf80 0x7ff5c80b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:16.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.992+0000 7ff5cf3b3640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c80d0e10 con 0x7ff5c815cf80 2023-12-19T18:26:16.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.992+0000 7ff5cf3b3640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c80b8cf0 con 0x7ff5c80cdf80 2023-12-19T18:26:16.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.992+0000 7ff5cf3b3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80ce410 con 0x7ff5c80d2100 2023-12-19T18:26:16.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.993+0000 7ff5cebb2640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff5c80cdf80 0x7ff5c80b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:16.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.993+0000 7ff5cebb2640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ff5c80cdf80 0x7ff5c80b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:44562/0 (socket says 172.21.15.27:44562) 2023-12-19T18:26:16.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.993+0000 7ff5cebb2640 1 -- 172.21.15.27:0/1253491354 learned_addr learned my addr 172.21.15.27:0/1253491354 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:16.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.993+0000 7ff5cdbb0640 1 --2- 172.21.15.27:0/1253491354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c80d2100 0x7ff5c80cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:16.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.994+0000 7ff5cebb2640 1 -- 172.21.15.27:0/1253491354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c80d2100 msgr2=0x7ff5c80cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:16.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.994+0000 7ff5cebb2640 1 --2- 172.21.15.27:0/1253491354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c80d2100 0x7ff5c80cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:16.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.994+0000 7ff5cebb2640 1 -- 172.21.15.27:0/1253491354 >> v1:172.21.15.192:6789/0 conn(0x7ff5c815cf80 legacy=0x7ff5c80b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:16.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.994+0000 7ff5cebb2640 1 -- 172.21.15.27:0/1253491354 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c80d1c70 con 0x7ff5c80cdf80 2023-12-19T18:26:16.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.995+0000 7ff5cebb2640 1 --2- 172.21.15.27:0/1253491354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c80cdf80 0x7ff5c80b5d60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009870 tx=0x7ff5c002ba10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db2525bfdb119ac7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:16.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.996+0000 7ff5cd3af640 1 -- 172.21.15.27:0/1253491354 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c002f830 con 0x7ff5c80cdf80 2023-12-19T18:26:16.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.996+0000 7ff5cd3af640 1 -- 172.21.15.27:0/1253491354 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5c002f990 con 0x7ff5c80cdf80 2023-12-19T18:26:16.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.996+0000 7ff5cd3af640 1 -- 172.21.15.27:0/1253491354 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5c002fd20 con 0x7ff5c80cdf80 2023-12-19T18:26:16.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.997+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/1253491354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c80cdf80 msgr2=0x7ff5c80b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:16.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.997+0000 7ff5cf3b3640 1 --2- 172.21.15.27:0/1253491354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c80cdf80 0x7ff5c80b5d60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff5c0009870 tx=0x7ff5c002ba10 comp rx=0 tx=0).stop 2023-12-19T18:26:16.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.997+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/1253491354 shutdown_connections 2023-12-19T18:26:16.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.997+0000 7ff5cf3b3640 1 --2- 172.21.15.27:0/1253491354 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c80cdf80 0x7ff5c80b5d60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:16.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.997+0000 7ff5cf3b3640 1 --2- 172.21.15.27:0/1253491354 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c80d2100 0x7ff5c80cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:16.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.997+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/1253491354 >> 172.21.15.27:0/1253491354 conn(0x7ff5c8072e20 msgr2=0x7ff5c8073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:16.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.997+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/1253491354 shutdown_connections 2023-12-19T18:26:16.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.997+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/1253491354 wait complete. 2023-12-19T18:26:16.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:16.999+0000 7ff5cf3b3640 1 Processor -- start 2023-12-19T18:26:16.999 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.000+0000 7ff5cf3b3640 1 -- start start 2023-12-19T18:26:17.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.001+0000 7ff5cf3b3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c80cdf80 0x7ff5c81f2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.001+0000 7ff5cf3b3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c80d2100 0x7ff5c80c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.001+0000 7ff5cf3b3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c815cf80 0x7ff5c80cb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.001+0000 7ff5cf3b3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80c67a0 con 0x7ff5c80d2100 2023-12-19T18:26:17.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.001+0000 7ff5cf3b3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80cae60 con 0x7ff5c815cf80 2023-12-19T18:26:17.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.001+0000 7ff5cf3b3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c80cbee0 con 0x7ff5c80cdf80 2023-12-19T18:26:17.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.001+0000 7ff5cdbb0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c80d2100 0x7ff5c80c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.001+0000 7ff5cdbb0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c80d2100 0x7ff5c80c8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35270/0 (socket says 172.21.15.27:35270) 2023-12-19T18:26:17.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.001+0000 7ff5cdbb0640 1 -- 172.21.15.27:0/2305922035 learned_addr learned my addr 172.21.15.27:0/2305922035 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:17.002 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.002+0000 7ff5ce3b1640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c80cdf80 0x7ff5c81f2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.002+0000 7ff5cebb2640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c815cf80 0x7ff5c80cb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.002+0000 7ff5cdbb0640 1 -- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c80cdf80 msgr2=0x7ff5c81f2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.002+0000 7ff5cdbb0640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c80cdf80 0x7ff5c81f2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.002+0000 7ff5cdbb0640 1 -- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c815cf80 msgr2=0x7ff5c80cb3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.002+0000 7ff5cdbb0640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c815cf80 0x7ff5c80cb3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.002+0000 7ff5cdbb0640 1 -- 172.21.15.27:0/2305922035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c00094d0 con 0x7ff5c80d2100 2023-12-19T18:26:17.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.002+0000 7ff5cdbb0640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c80d2100 0x7ff5c80c8d90 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00e9b0 tx=0x7ff5bc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:17.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.003+0000 7ff5c6ffd640 1 -- 172.21.15.27:0/2305922035 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc009670 con 0x7ff5c80d2100 2023-12-19T18:26:17.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.003+0000 7ff5c6ffd640 1 -- 172.21.15.27:0/2305922035 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc00c9a0 con 0x7ff5c80d2100 2023-12-19T18:26:17.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.003+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/2305922035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c80c69f0 con 0x7ff5c80d2100 2023-12-19T18:26:17.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.003+0000 7ff5c6ffd640 1 -- 172.21.15.27:0/2305922035 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc00cd50 con 0x7ff5c80d2100 2023-12-19T18:26:17.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.003+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/2305922035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c80c6fb0 con 0x7ff5c80d2100 2023-12-19T18:26:17.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.004+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/2305922035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c804e840 con 0x7ff5c80d2100 2023-12-19T18:26:17.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.005+0000 7ff5c6ffd640 1 -- 172.21.15.27:0/2305922035 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff5bc013470 con 0x7ff5c80d2100 2023-12-19T18:26:17.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.006+0000 7ff5c6ffd640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5b406a430 0x7ff5b406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.007+0000 7ff5c6ffd640 1 -- 172.21.15.27:0/2305922035 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff5bc088bf0 con 0x7ff5c80d2100 2023-12-19T18:26:17.006 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.007+0000 7ff5c6ffd640 1 -- 172.21.15.27:0/2305922035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7ff5b4071f60 con 0x7ff5c80d2100 2023-12-19T18:26:17.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.007+0000 7ff5ce3b1640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5b406a430 0x7ff5b406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.008+0000 7ff5ce3b1640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5b406a430 0x7ff5b406c8f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80012e0 tx=0x7ff5b8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:17.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.011+0000 7ff5c6ffd640 1 -- 172.21.15.27:0/2305922035 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff5bc017060 con 0x7ff5c80d2100 2023-12-19T18:26:17.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.203+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/2305922035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5c80cd8e0 con 0x7ff5c80d2100 2023-12-19T18:26:17.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.205+0000 7ff5c6ffd640 1 -- 172.21.15.27:0/2305922035 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v172) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5bc021070 con 0x7ff5c80d2100 2023-12-19T18:26:17.205 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:26:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.207+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5b406a430 msgr2=0x7ff5b406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.207+0000 7ff5cf3b3640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5b406a430 0x7ff5b406c8f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff5b80012e0 tx=0x7ff5b8009040 comp rx=0 tx=0).stop 2023-12-19T18:26:17.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.207+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/2305922035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c80d2100 msgr2=0x7ff5c80c8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.207+0000 7ff5cf3b3640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c80d2100 0x7ff5c80c8d90 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc00e9b0 tx=0x7ff5bc00ee70 comp rx=0 tx=0).stop 2023-12-19T18:26:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.207+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/2305922035 shutdown_connections 2023-12-19T18:26:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.207+0000 7ff5cf3b3640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5b406a430 0x7ff5b406c8f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.207+0000 7ff5cf3b3640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c815cf80 0x7ff5c80cb3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.207+0000 7ff5cf3b3640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c80d2100 0x7ff5c80c8d90 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.207+0000 7ff5cf3b3640 1 --2- 172.21.15.27:0/2305922035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c80cdf80 0x7ff5c81f2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.207+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/2305922035 >> 172.21.15.27:0/2305922035 conn(0x7ff5c8072e20 msgr2=0x7ff5c80d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.207+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/2305922035 shutdown_connections 2023-12-19T18:26:17.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.207+0000 7ff5cf3b3640 1 -- 172.21.15.27:0/2305922035 wait complete. 2023-12-19T18:26:17.228 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:26:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.468+0000 7f45ca33e640 1 Processor -- start 2023-12-19T18:26:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.468+0000 7f45ca33e640 1 -- start start 2023-12-19T18:26:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.469+0000 7f45ca33e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f45c415b1a0 0x7f45c406f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.469+0000 7f45ca33e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f45c40c2720 0x7f45c406fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.469+0000 7f45ca33e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45c406bc80 con 0x7f45c4071420 2023-12-19T18:26:17.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.469+0000 7f45ca33e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c415fd90 con 0x7f45c40c2720 2023-12-19T18:26:17.470 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.469+0000 7f45ca33e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45c406d690 con 0x7f45c415b1a0 2023-12-19T18:26:17.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.470+0000 7f45c37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f45c40c2720 0x7f45c406fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.470+0000 7f45c37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f45c40c2720 0x7f45c406fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35274/0 (socket says 172.21.15.27:35274) 2023-12-19T18:26:17.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.470+0000 7f45c37fe640 1 -- 172.21.15.27:0/2463517599 learned_addr learned my addr 172.21.15.27:0/2463517599 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:17.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.470+0000 7f45c3fff640 1 --2- 172.21.15.27:0/2463517599 >> v2:172.21.15.192:3300/0 conn(0x7f45c415b1a0 0x7f45c406f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.471+0000 7f45c37fe640 1 -- 172.21.15.27:0/2463517599 >> v2:172.21.15.192:3300/0 conn(0x7f45c415b1a0 msgr2=0x7f45c406f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:17.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.471+0000 7f45c37fe640 1 --2- 172.21.15.27:0/2463517599 >> v2:172.21.15.192:3300/0 conn(0x7f45c415b1a0 0x7f45c406f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.471+0000 7f45c37fe640 1 -- 172.21.15.27:0/2463517599 >> v1:172.21.15.192:6789/0 conn(0x7f45c4071420 legacy=0x7f45c40c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.471+0000 7f45c37fe640 1 -- 172.21.15.27:0/2463517599 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45c406d0a0 con 0x7f45c40c2720 2023-12-19T18:26:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.473+0000 7f45c37fe640 1 --2- 172.21.15.27:0/2463517599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c40c2720 0x7f45c406fa60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f45c40cea40 tx=0x7f45b802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcfd486d7d5da70 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:17.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.473+0000 7f45c2ffd640 1 -- 172.21.15.27:0/2463517599 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8031870 con 0x7f45c40c2720 2023-12-19T18:26:17.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.473+0000 7f45c2ffd640 1 -- 172.21.15.27:0/2463517599 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45b80319d0 con 0x7f45c40c2720 2023-12-19T18:26:17.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.473+0000 7f45c2ffd640 1 -- 172.21.15.27:0/2463517599 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8031d60 con 0x7f45c40c2720 2023-12-19T18:26:17.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.474+0000 7f45ca33e640 1 -- 172.21.15.27:0/2463517599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c40c2720 msgr2=0x7f45c406fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.474+0000 7f45ca33e640 1 --2- 172.21.15.27:0/2463517599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c40c2720 0x7f45c406fa60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f45c40cea40 tx=0x7f45b802b910 comp rx=0 tx=0).stop 2023-12-19T18:26:17.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.474+0000 7f45ca33e640 1 -- 172.21.15.27:0/2463517599 shutdown_connections 2023-12-19T18:26:17.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.474+0000 7f45ca33e640 1 --2- 172.21.15.27:0/2463517599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c40c2720 0x7f45c406fa60 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.474+0000 7f45ca33e640 1 --2- 172.21.15.27:0/2463517599 >> v2:172.21.15.192:3300/0 conn(0x7f45c415b1a0 0x7f45c406f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.474+0000 7f45ca33e640 1 -- 172.21.15.27:0/2463517599 >> 172.21.15.27:0/2463517599 conn(0x7f45c40cebd0 msgr2=0x7f45c40cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:17.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.474+0000 7f45ca33e640 1 -- 172.21.15.27:0/2463517599 shutdown_connections 2023-12-19T18:26:17.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.474+0000 7f45ca33e640 1 -- 172.21.15.27:0/2463517599 wait complete. 2023-12-19T18:26:17.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.476+0000 7f45ca33e640 1 Processor -- start 2023-12-19T18:26:17.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.476+0000 7f45ca33e640 1 -- start start 2023-12-19T18:26:17.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.477+0000 7f45ca33e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45c4071420 0x7f45c40c8710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.477+0000 7f45ca33e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45c40c2720 0x7f45c40c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.477+0000 7f45ca33e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c415b1a0 0x7f45c40c7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.477+0000 7f45ca33e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c4147810 con 0x7f45c415b1a0 2023-12-19T18:26:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.477+0000 7f45ca33e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c41e5900 con 0x7f45c4071420 2023-12-19T18:26:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.477+0000 7f45ca33e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45c41e4cf0 con 0x7f45c40c2720 2023-12-19T18:26:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.477+0000 7f45c88db640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c415b1a0 0x7f45c40c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.478+0000 7f45c88db640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c415b1a0 0x7f45c40c7230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35276/0 (socket says 172.21.15.27:35276) 2023-12-19T18:26:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.478+0000 7f45c88db640 1 -- 172.21.15.27:0/173274215 learned_addr learned my addr 172.21.15.27:0/173274215 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:17.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.478+0000 7f45c88db640 1 -- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45c40c2720 msgr2=0x7f45c40c6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.478+0000 7f45c3fff640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45c4071420 0x7f45c40c8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.478+0000 7f45c37fe640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45c40c2720 0x7f45c40c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.478+0000 7f45c88db640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45c40c2720 0x7f45c40c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.478+0000 7f45c88db640 1 -- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45c4071420 msgr2=0x7f45c40c8710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.478+0000 7f45c88db640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45c4071420 0x7f45c40c8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.478+0000 7f45c88db640 1 -- 172.21.15.27:0/173274215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b80094d0 con 0x7f45c415b1a0 2023-12-19T18:26:17.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.478+0000 7f45c88db640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c415b1a0 0x7f45c40c7230 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00c730 tx=0x7f45bc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:17.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.479+0000 7f45c0ff9640 1 -- 172.21.15.27:0/173274215 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc003b70 con 0x7f45c415b1a0 2023-12-19T18:26:17.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.479+0000 7f45c0ff9640 1 -- 172.21.15.27:0/173274215 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45bc003cd0 con 0x7f45c415b1a0 2023-12-19T18:26:17.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.479+0000 7f45c0ff9640 1 -- 172.21.15.27:0/173274215 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00aa90 con 0x7f45c415b1a0 2023-12-19T18:26:17.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.479+0000 7f45ca33e640 1 -- 172.21.15.27:0/173274215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c41e4fd0 con 0x7f45c415b1a0 2023-12-19T18:26:17.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.479+0000 7f45ca33e640 1 -- 172.21.15.27:0/173274215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c41e5590 con 0x7f45c415b1a0 2023-12-19T18:26:17.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.480+0000 7f45ca33e640 1 -- 172.21.15.27:0/173274215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c41e5a70 con 0x7f45c415b1a0 2023-12-19T18:26:17.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.481+0000 7f45c0ff9640 1 -- 172.21.15.27:0/173274215 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f45bc00abf0 con 0x7f45c415b1a0 2023-12-19T18:26:17.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.482+0000 7f45c0ff9640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45b006a430 0x7f45b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.482+0000 7f45c0ff9640 1 -- 172.21.15.27:0/173274215 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f45bc0885c0 con 0x7f45c415b1a0 2023-12-19T18:26:17.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.482+0000 7f45c0ff9640 1 -- 172.21.15.27:0/173274215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f45b0071f60 con 0x7f45c415b1a0 2023-12-19T18:26:17.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.483+0000 7f45c3fff640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45b006a430 0x7f45b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.484+0000 7f45c3fff640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45b006a430 0x7f45b006c8f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009970 tx=0x7f45b4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:17.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.486+0000 7f45c0ff9640 1 -- 172.21.15.27:0/173274215 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f45bc053ba0 con 0x7f45c415b1a0 2023-12-19T18:26:17.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.685+0000 7f45ca33e640 1 -- 172.21.15.27:0/173274215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f45c406bc80 con 0x7f45c415b1a0 2023-12-19T18:26:17.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.686+0000 7f45c0ff9640 1 -- 172.21.15.27:0/173274215 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v172) v1 ==== 104+0+11 (secure 0 0 0) 0x7f45c406bc80 con 0x7f45c415b1a0 2023-12-19T18:26:17.686 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:26:17.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.689+0000 7f45ca33e640 1 -- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45b006a430 msgr2=0x7f45b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.689+0000 7f45ca33e640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45b006a430 0x7f45b006c8f0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f45b4009970 tx=0x7f45b4008040 comp rx=0 tx=0).stop 2023-12-19T18:26:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.689+0000 7f45ca33e640 1 -- 172.21.15.27:0/173274215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c415b1a0 msgr2=0x7f45c40c7230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.689+0000 7f45ca33e640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c415b1a0 0x7f45c40c7230 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f45bc00c730 tx=0x7f45bc00cbf0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.689+0000 7f45ca33e640 1 -- 172.21.15.27:0/173274215 shutdown_connections 2023-12-19T18:26:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.689+0000 7f45ca33e640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f45b006a430 0x7f45b006c8f0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.689+0000 7f45ca33e640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c415b1a0 0x7f45c40c7230 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.689+0000 7f45ca33e640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f45c40c2720 0x7f45c40c6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.689+0000 7f45ca33e640 1 --2- 172.21.15.27:0/173274215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f45c4071420 0x7f45c40c8710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.689+0000 7f45ca33e640 1 -- 172.21.15.27:0/173274215 >> 172.21.15.27:0/173274215 conn(0x7f45c40cebd0 msgr2=0x7f45c40c4b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.689+0000 7f45ca33e640 1 -- 172.21.15.27:0/173274215 shutdown_connections 2023-12-19T18:26:17.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.689+0000 7f45ca33e640 1 -- 172.21.15.27:0/173274215 wait complete. 2023-12-19T18:26:17.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:26:17.855 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:26:17.900 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:26:17.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.944+0000 7fae01c21640 1 Processor -- start 2023-12-19T18:26:17.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.944+0000 7fae01c21640 1 -- start start 2023-12-19T18:26:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.945+0000 7fae01c21640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadfc15b1a0 0x7fadfc06f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.945+0000 7fae01c21640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadfc06bc80 con 0x7fadfc0c2720 2023-12-19T18:26:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.945+0000 7fae01c21640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadfc06d690 con 0x7fadfc071420 2023-12-19T18:26:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.945+0000 7fae01c21640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc06d0a0 con 0x7fadfc15b1a0 2023-12-19T18:26:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.945+0000 7fadf3fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fadfc0c2720 0x7fadfc06fa60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40058/0 (socket says 172.21.15.27:40058) 2023-12-19T18:26:17.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.945+0000 7fadf3fff640 1 -- 172.21.15.27:0/3897422894 learned_addr learned my addr 172.21.15.27:0/3897422894 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.946+0000 7fadfb7fe640 1 --2- 172.21.15.27:0/3897422894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadfc15b1a0 0x7fadfc06f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.946+0000 7fadfaffd640 1 -- 172.21.15.27:0/3897422894 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2459321982 0 0) 0x7fade4001110 con 0x7fadfc0c2720 2023-12-19T18:26:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.946+0000 7fadfaffd640 1 -- 172.21.15.27:0/3897422894 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade8003520 con 0x7fadfc0c2720 2023-12-19T18:26:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.946+0000 7fadfaffd640 1 -- 172.21.15.27:0/3897422894 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916873383 0 0) 0x7fadfc06d690 con 0x7fadfc071420 2023-12-19T18:26:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.946+0000 7fadfaffd640 1 -- 172.21.15.27:0/3897422894 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fade4001110 con 0x7fadfc071420 2023-12-19T18:26:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.946+0000 7fadfb7fe640 1 -- 172.21.15.27:0/3897422894 >> v1:172.21.15.192:6789/0 conn(0x7fadfc071420 legacy=0x7fadfc15ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.946+0000 7fadfb7fe640 1 -- 172.21.15.27:0/3897422894 >> v1:172.21.15.27:6789/0 conn(0x7fadfc0c2720 legacy=0x7fadfc06fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.946+0000 7fadfaffd640 1 -- 172.21.15.27:0/3897422894 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2558922753 0 0) 0x7fade8003520 con 0x7fadfc0c2720 2023-12-19T18:26:17.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.946+0000 7fadfb7fe640 1 -- 172.21.15.27:0/3897422894 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadfc06b6c0 con 0x7fadfc15b1a0 2023-12-19T18:26:17.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.948+0000 7fadfb7fe640 1 --2- 172.21.15.27:0/3897422894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadfc15b1a0 0x7fadfc06f520 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fadec0097d0 tx=0x7fadec02b8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4ad766f7dc21140 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:17.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.948+0000 7fadfaffd640 1 -- 172.21.15.27:0/3897422894 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec031800 con 0x7fadfc15b1a0 2023-12-19T18:26:17.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.948+0000 7fadfaffd640 1 -- 172.21.15.27:0/3897422894 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadec031960 con 0x7fadfc15b1a0 2023-12-19T18:26:17.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.948+0000 7fadfaffd640 1 -- 172.21.15.27:0/3897422894 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadec031cd0 con 0x7fadfc15b1a0 2023-12-19T18:26:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.948+0000 7fae01c21640 1 -- 172.21.15.27:0/3897422894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadfc15b1a0 msgr2=0x7fadfc06f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.948+0000 7fae01c21640 1 --2- 172.21.15.27:0/3897422894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadfc15b1a0 0x7fadfc06f520 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fadec0097d0 tx=0x7fadec02b8a0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.949+0000 7fae01c21640 1 -- 172.21.15.27:0/3897422894 shutdown_connections 2023-12-19T18:26:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.949+0000 7fae01c21640 1 --2- 172.21.15.27:0/3897422894 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadfc15b1a0 0x7fadfc06f520 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.949+0000 7fae01c21640 1 -- 172.21.15.27:0/3897422894 >> 172.21.15.27:0/3897422894 conn(0x7fadfc0cebd0 msgr2=0x7fadfc0cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.949+0000 7fae01c21640 1 -- 172.21.15.27:0/3897422894 shutdown_connections 2023-12-19T18:26:17.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.949+0000 7fae01c21640 1 -- 172.21.15.27:0/3897422894 wait complete. 2023-12-19T18:26:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.951+0000 7fae01c21640 1 Processor -- start 2023-12-19T18:26:17.950 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.951+0000 7fae01c21640 1 -- start start 2023-12-19T18:26:17.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fae01c21640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadfc071420 0x7fadfc0c4d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fae01c21640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc0c2720 0x7fadfc0af900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fae01c21640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadfc15b1a0 0x7fadfc159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fae01c21640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1f62d0 con 0x7fadfc0c2720 2023-12-19T18:26:17.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fae01c21640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1f65b0 con 0x7fadfc15b1a0 2023-12-19T18:26:17.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fae01c21640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadfc1f6890 con 0x7fadfc071420 2023-12-19T18:26:17.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fadf3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc0c2720 0x7fadfc0af900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fadf3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc0c2720 0x7fadfc0af900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35280/0 (socket says 172.21.15.27:35280) 2023-12-19T18:26:17.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fadf3fff640 1 -- 172.21.15.27:0/466195083 learned_addr learned my addr 172.21.15.27:0/466195083 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:17.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fadfb7fe640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadfc071420 0x7fadfc0c4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fadfbfff640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadfc15b1a0 0x7fadfc159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fadf3fff640 1 -- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadfc071420 msgr2=0x7fadfc0c4d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fadf3fff640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadfc071420 0x7fadfc0c4d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.952+0000 7fadf3fff640 1 -- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadfc15b1a0 msgr2=0x7fadfc159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.953+0000 7fadf3fff640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadfc15b1a0 0x7fadfc159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.953+0000 7fadf3fff640 1 -- 172.21.15.27:0/466195083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec009400 con 0x7fadfc0c2720 2023-12-19T18:26:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.953+0000 7fadf3fff640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc0c2720 0x7fadfc0af900 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fade4000fd0 tx=0x7fade400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.953+0000 7fadf8ff9640 1 -- 172.21.15.27:0/466195083 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4005ab0 con 0x7fadfc0c2720 2023-12-19T18:26:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.953+0000 7fadf8ff9640 1 -- 172.21.15.27:0/466195083 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade4005c30 con 0x7fadfc0c2720 2023-12-19T18:26:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.953+0000 7fae01c21640 1 -- 172.21.15.27:0/466195083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadfc1f6b70 con 0x7fadfc0c2720 2023-12-19T18:26:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.953+0000 7fadf8ff9640 1 -- 172.21.15.27:0/466195083 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4007070 con 0x7fadfc0c2720 2023-12-19T18:26:17.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.953+0000 7fae01c21640 1 -- 172.21.15.27:0/466195083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadfc0afe40 con 0x7fadfc0c2720 2023-12-19T18:26:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.955+0000 7fae01c21640 1 -- 172.21.15.27:0/466195083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadfc068e90 con 0x7fadfc0c2720 2023-12-19T18:26:17.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.955+0000 7fadf8ff9640 1 -- 172.21.15.27:0/466195083 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fade40071d0 con 0x7fadfc0c2720 2023-12-19T18:26:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.956+0000 7fadf8ff9640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faddc06a430 0x7faddc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.956+0000 7fadf8ff9640 1 -- 172.21.15.27:0/466195083 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fade4087e10 con 0x7fadfc0c2720 2023-12-19T18:26:17.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.957+0000 7fadf8ff9640 1 -- 172.21.15.27:0/466195083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7faddc071f60 con 0x7fadfc0c2720 2023-12-19T18:26:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.957+0000 7fadfb7fe640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faddc06a430 0x7faddc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:17.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.958+0000 7fadfb7fe640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faddc06a430 0x7faddc06c8f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fadec036000 tx=0x7fadec03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:17.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:17.961+0000 7fadf8ff9640 1 -- 172.21.15.27:0/466195083 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fade4017060 con 0x7fadfc0c2720 2023-12-19T18:26:18.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.155+0000 7fae01c21640 1 -- 172.21.15.27:0/466195083 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fadfc06bc80 con 0x7fadfc0c2720 2023-12-19T18:26:18.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.156+0000 7fadf8ff9640 1 -- 172.21.15.27:0/466195083 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7fadfc06bc80 con 0x7fadfc0c2720 2023-12-19T18:26:18.156 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:26:18.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.159+0000 7fae01c21640 1 -- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faddc06a430 msgr2=0x7faddc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:18.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.159+0000 7fae01c21640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faddc06a430 0x7faddc06c8f0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fadec036000 tx=0x7fadec03c040 comp rx=0 tx=0).stop 2023-12-19T18:26:18.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.159+0000 7fae01c21640 1 -- 172.21.15.27:0/466195083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc0c2720 msgr2=0x7fadfc0af900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:18.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.159+0000 7fae01c21640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc0c2720 0x7fadfc0af900 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7fade4000fd0 tx=0x7fade400cbe0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.159+0000 7fae01c21640 1 -- 172.21.15.27:0/466195083 shutdown_connections 2023-12-19T18:26:18.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.159+0000 7fae01c21640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faddc06a430 0x7faddc06c8f0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.159+0000 7fae01c21640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadfc15b1a0 0x7fadfc159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.159+0000 7fae01c21640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadfc0c2720 0x7fadfc0af900 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.159+0000 7fae01c21640 1 --2- 172.21.15.27:0/466195083 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadfc071420 0x7fadfc0c4d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.159+0000 7fae01c21640 1 -- 172.21.15.27:0/466195083 >> 172.21.15.27:0/466195083 conn(0x7fadfc0cebd0 msgr2=0x7fadfc0cefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:18.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.159+0000 7fae01c21640 1 -- 172.21.15.27:0/466195083 shutdown_connections 2023-12-19T18:26:18.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.159+0000 7fae01c21640 1 -- 172.21.15.27:0/466195083 wait complete. 2023-12-19T18:26:18.179 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:26:18.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.414+0000 7f1509fbc640 1 Processor -- start 2023-12-19T18:26:18.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.415+0000 7f1509fbc640 1 -- start start 2023-12-19T18:26:18.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.416+0000 7f1509fbc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f15041540e0 0x7f1504160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:18.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.416+0000 7f1509fbc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f150406b310 0x7f150406b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:18.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.416+0000 7f1509fbc640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f150415d600 con 0x7f150406c6e0 2023-12-19T18:26:18.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.416+0000 7f1509fbc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f150406bcc0 con 0x7f150406b310 2023-12-19T18:26:18.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.416+0000 7f1509fbc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1504161130 con 0x7f15041540e0 2023-12-19T18:26:18.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.417+0000 7f1503fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f150406c6e0 0x7f150415dda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40064/0 (socket says 172.21.15.27:40064) 2023-12-19T18:26:18.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.417+0000 7f15097bb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f150406b310 0x7f150406b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:18.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.417+0000 7f1503fff640 1 -- 172.21.15.27:0/2418858677 learned_addr learned my addr 172.21.15.27:0/2418858677 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:18.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.417+0000 7f1508fba640 1 --2- 172.21.15.27:0/2418858677 >> v2:172.21.15.192:3300/0 conn(0x7f15041540e0 0x7f1504160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:18.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.417+0000 7f15037fe640 1 -- 172.21.15.27:0/2418858677 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3365510939 0 0) 0x7f150415d600 con 0x7f150406c6e0 2023-12-19T18:26:18.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.417+0000 7f15097bb640 1 -- 172.21.15.27:0/2418858677 >> v2:172.21.15.192:3300/0 conn(0x7f15041540e0 msgr2=0x7f1504160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:18.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.417+0000 7f15097bb640 1 --2- 172.21.15.27:0/2418858677 >> v2:172.21.15.192:3300/0 conn(0x7f15041540e0 0x7f1504160bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.417+0000 7f15097bb640 1 -- 172.21.15.27:0/2418858677 >> v1:172.21.15.27:6789/0 conn(0x7f150406c6e0 legacy=0x7f150415dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:18.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.418+0000 7f15097bb640 1 -- 172.21.15.27:0/2418858677 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15041613d0 con 0x7f150406b310 2023-12-19T18:26:18.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.419+0000 7f15097bb640 1 --2- 172.21.15.27:0/2418858677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f150406b310 0x7f150406b780 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f15040c8b70 tx=0x7f14fc02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f183aba930eae3b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:18.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.419+0000 7f15037fe640 1 -- 172.21.15.27:0/2418858677 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc02cc80 con 0x7f150406b310 2023-12-19T18:26:18.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.419+0000 7f15037fe640 1 -- 172.21.15.27:0/2418858677 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14fc02cde0 con 0x7f150406b310 2023-12-19T18:26:18.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.419+0000 7f15037fe640 1 -- 172.21.15.27:0/2418858677 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14fc034660 con 0x7f150406b310 2023-12-19T18:26:18.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.419+0000 7f1509fbc640 1 -- 172.21.15.27:0/2418858677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f150406b310 msgr2=0x7f150406b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:18.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.419+0000 7f1509fbc640 1 --2- 172.21.15.27:0/2418858677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f150406b310 0x7f150406b780 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f15040c8b70 tx=0x7f14fc02bed0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.420+0000 7f1509fbc640 1 -- 172.21.15.27:0/2418858677 shutdown_connections 2023-12-19T18:26:18.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.420+0000 7f1509fbc640 1 --2- 172.21.15.27:0/2418858677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f150406b310 0x7f150406b780 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.420+0000 7f1509fbc640 1 --2- 172.21.15.27:0/2418858677 >> v2:172.21.15.192:3300/0 conn(0x7f15041540e0 0x7f1504160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.420+0000 7f1509fbc640 1 -- 172.21.15.27:0/2418858677 >> 172.21.15.27:0/2418858677 conn(0x7f150406a7d0 msgr2=0x7f15040ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:18.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.420+0000 7f1509fbc640 1 -- 172.21.15.27:0/2418858677 shutdown_connections 2023-12-19T18:26:18.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.420+0000 7f1509fbc640 1 -- 172.21.15.27:0/2418858677 wait complete. 2023-12-19T18:26:18.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.421+0000 7f1509fbc640 1 Processor -- start 2023-12-19T18:26:18.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.422+0000 7f1509fbc640 1 -- start start 2023-12-19T18:26:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1509fbc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f150406b310 0x7f1504150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1509fbc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f150406c6e0 0x7f15040c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1509fbc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15041540e0 0x7f15040c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1509fbc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1504154e90 con 0x7f150406b310 2023-12-19T18:26:18.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1509fbc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1504150bd0 con 0x7f15041540e0 2023-12-19T18:26:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1509fbc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1504150670 con 0x7f150406c6e0 2023-12-19T18:26:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1508fba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f150406b310 0x7f1504150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1508fba640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f150406b310 0x7f1504150130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35292/0 (socket says 172.21.15.27:35292) 2023-12-19T18:26:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1508fba640 1 -- 172.21.15.27:0/3007255149 learned_addr learned my addr 172.21.15.27:0/3007255149 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1508fba640 1 -- 172.21.15.27:0/3007255149 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f150406c6e0 msgr2=0x7f15040c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1508fba640 1 --2- 172.21.15.27:0/3007255149 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f150406c6e0 0x7f15040c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1508fba640 1 -- 172.21.15.27:0/3007255149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15041540e0 msgr2=0x7f15040c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:18.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1508fba640 1 --2- 172.21.15.27:0/3007255149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15041540e0 0x7f15040c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.423+0000 7f1508fba640 1 -- 172.21.15.27:0/3007255149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14fc009a70 con 0x7f150406b310 2023-12-19T18:26:18.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.424+0000 7f1508fba640 1 --2- 172.21.15.27:0/3007255149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f150406b310 0x7f1504150130 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f14f800d830 tx=0x7f14f800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:18.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.424+0000 7f15017fa640 1 -- 172.21.15.27:0/3007255149 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f800b6c0 con 0x7f150406b310 2023-12-19T18:26:18.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.424+0000 7f15017fa640 1 -- 172.21.15.27:0/3007255149 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14f800b820 con 0x7f150406b310 2023-12-19T18:26:18.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.424+0000 7f15017fa640 1 -- 172.21.15.27:0/3007255149 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14f800bbb0 con 0x7f150406b310 2023-12-19T18:26:18.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.424+0000 7f1509fbc640 1 -- 172.21.15.27:0/3007255149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1504160460 con 0x7f150406b310 2023-12-19T18:26:18.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.424+0000 7f1509fbc640 1 -- 172.21.15.27:0/3007255149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1504154840 con 0x7f150406b310 2023-12-19T18:26:18.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.425+0000 7f1509fbc640 1 -- 172.21.15.27:0/3007255149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f150404e840 con 0x7f150406b310 2023-12-19T18:26:18.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.426+0000 7f15017fa640 1 -- 172.21.15.27:0/3007255149 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f14f801e460 con 0x7f150406b310 2023-12-19T18:26:18.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.428+0000 7f15017fa640 1 --2- 172.21.15.27:0/3007255149 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f14e806a430 0x7f14e806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:18.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.428+0000 7f15017fa640 1 -- 172.21.15.27:0/3007255149 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f14f80890a0 con 0x7f150406b310 2023-12-19T18:26:18.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.428+0000 7f15017fa640 1 -- 172.21.15.27:0/3007255149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f14e8071f60 con 0x7f150406b310 2023-12-19T18:26:18.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.428+0000 7f1503fff640 1 --2- 172.21.15.27:0/3007255149 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f14e806a430 0x7f14e806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:18.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.430+0000 7f1503fff640 1 --2- 172.21.15.27:0/3007255149 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f14e806a430 0x7f14e806c8f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f14f0007fd0 tx=0x7f14f0007920 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:18.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.433+0000 7f15017fa640 1 -- 172.21.15.27:0/3007255149 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f14f8054680 con 0x7f150406b310 2023-12-19T18:26:18.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.640+0000 7f1509fbc640 1 -- 172.21.15.27:0/3007255149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f150406af60 con 0x7f150406b310 2023-12-19T18:26:18.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.641+0000 7f15017fa640 1 -- 172.21.15.27:0/3007255149 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v172) v1 ==== 80+0+0 (secure 0 0 0) 0x7f14f8057ca0 con 0x7f150406b310 2023-12-19T18:26:18.641 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:26:18.644 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.644+0000 7f1509fbc640 1 -- 172.21.15.27:0/3007255149 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f14e806a430 msgr2=0x7f14e806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:18.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.644+0000 7f1509fbc640 1 --2- 172.21.15.27:0/3007255149 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f14e806a430 0x7f14e806c8f0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f14f0007fd0 tx=0x7f14f0007920 comp rx=0 tx=0).stop 2023-12-19T18:26:18.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.644+0000 7f1509fbc640 1 -- 172.21.15.27:0/3007255149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f150406b310 msgr2=0x7f1504150130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:18.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.644+0000 7f1509fbc640 1 --2- 172.21.15.27:0/3007255149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f150406b310 0x7f1504150130 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f14f800d830 tx=0x7f14f800dcf0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.644+0000 7f1509fbc640 1 -- 172.21.15.27:0/3007255149 shutdown_connections 2023-12-19T18:26:18.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.644+0000 7f1509fbc640 1 --2- 172.21.15.27:0/3007255149 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f14e806a430 0x7f14e806c8f0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.644+0000 7f1509fbc640 1 --2- 172.21.15.27:0/3007255149 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f15041540e0 0x7f15040c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.644+0000 7f1509fbc640 1 --2- 172.21.15.27:0/3007255149 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f150406c6e0 0x7f15040c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.644+0000 7f1509fbc640 1 --2- 172.21.15.27:0/3007255149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f150406b310 0x7f1504150130 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:18.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.644+0000 7f1509fbc640 1 -- 172.21.15.27:0/3007255149 >> 172.21.15.27:0/3007255149 conn(0x7f150406a7d0 msgr2=0x7f15040cdd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:18.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.644+0000 7f1509fbc640 1 -- 172.21.15.27:0/3007255149 shutdown_connections 2023-12-19T18:26:18.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:18.644+0000 7f1509fbc640 1 -- 172.21.15.27:0/3007255149 wait complete. 2023-12-19T18:26:18.662 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-19T18:26:18.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:18.663 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-19T18:26:18.761 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[91323]: fuse finished with error 0 and tester_r 0 2023-12-19T18:26:18.767 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:26:18.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:18.767 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-19T18:26:18.900 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:26:18.900 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:26:18.966 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:26:18.966 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:26:18.966 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:18.966 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-19T18:26:19.026 INFO:teuthology.orchestra.run.smithi027.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-19T18:26:19.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:26:19.027 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:26:19.027 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:26:19.087 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:26:19.087 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:26:19.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:19.087 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-19T18:26:19.142 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-19T18:26:19.142 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:26:19.143 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:26:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.382+0000 7f5b3787a640 1 Processor -- start 2023-12-19T18:26:19.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.383+0000 7f5b3787a640 1 -- start start 2023-12-19T18:26:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.384+0000 7f5b3787a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5b3015ae80 0x7f5b300d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:19.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.384+0000 7f5b3787a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5b300cea40 0x7f5b3006bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.384+0000 7f5b3787a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b3006ac80 0x7f5b3006b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.384+0000 7f5b3787a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3006c3e0 con 0x7f5b300cea40 2023-12-19T18:26:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.384+0000 7f5b3787a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3015e7a0 con 0x7f5b3015ae80 2023-12-19T18:26:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.384+0000 7f5b3787a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b300b0530 con 0x7f5b3006ac80 2023-12-19T18:26:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.385+0000 7f5b34e15640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5b300cea40 0x7f5b3006bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.385+0000 7f5b34e15640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5b300cea40 0x7f5b3006bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35308/0 (socket says 172.21.15.27:35308) 2023-12-19T18:26:19.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.385+0000 7f5b34e15640 1 -- 172.21.15.27:0/1295994919 learned_addr learned my addr 172.21.15.27:0/1295994919 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.385+0000 7f5b35616640 1 --2- 172.21.15.27:0/1295994919 >> v2:172.21.15.192:3300/0 conn(0x7f5b3015ae80 0x7f5b300d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.385+0000 7f5b35e17640 1 --2- 172.21.15.27:0/1295994919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b3006ac80 0x7f5b3006b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.385+0000 7f5b34e15640 1 -- 172.21.15.27:0/1295994919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b3006ac80 msgr2=0x7f5b3006b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.385+0000 7f5b34e15640 1 --2- 172.21.15.27:0/1295994919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b3006ac80 0x7f5b3006b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.385+0000 7f5b34e15640 1 -- 172.21.15.27:0/1295994919 >> v2:172.21.15.192:3300/0 conn(0x7f5b3015ae80 msgr2=0x7f5b300d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.385+0000 7f5b34e15640 1 --2- 172.21.15.27:0/1295994919 >> v2:172.21.15.192:3300/0 conn(0x7f5b3015ae80 0x7f5b300d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.385+0000 7f5b34e15640 1 -- 172.21.15.27:0/1295994919 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3006b600 con 0x7f5b300cea40 2023-12-19T18:26:19.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.385+0000 7f5b35616640 1 --2- 172.21.15.27:0/1295994919 >> v2:172.21.15.192:3300/0 conn(0x7f5b3015ae80 0x7f5b300d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:19.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.387+0000 7f5b34e15640 1 --2- 172.21.15.27:0/1295994919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b300cea40 0x7f5b3006bea0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f5b3006e6c0 tx=0x7f5b2002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c022897d3673af9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:19.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.387+0000 7f5b27fff640 1 -- 172.21.15.27:0/1295994919 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b200317c0 con 0x7f5b300cea40 2023-12-19T18:26:19.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.387+0000 7f5b27fff640 1 -- 172.21.15.27:0/1295994919 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b20031920 con 0x7f5b300cea40 2023-12-19T18:26:19.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.387+0000 7f5b27fff640 1 -- 172.21.15.27:0/1295994919 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b20031cb0 con 0x7f5b300cea40 2023-12-19T18:26:19.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.388+0000 7f5b3787a640 1 -- 172.21.15.27:0/1295994919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b300cea40 msgr2=0x7f5b3006bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:19.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.388+0000 7f5b3787a640 1 --2- 172.21.15.27:0/1295994919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b300cea40 0x7f5b3006bea0 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7f5b3006e6c0 tx=0x7f5b2002b860 comp rx=0 tx=0).stop 2023-12-19T18:26:19.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.388+0000 7f5b3787a640 1 -- 172.21.15.27:0/1295994919 shutdown_connections 2023-12-19T18:26:19.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.388+0000 7f5b3787a640 1 --2- 172.21.15.27:0/1295994919 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b3006ac80 0x7f5b3006b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.388+0000 7f5b3787a640 1 --2- 172.21.15.27:0/1295994919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b300cea40 0x7f5b3006bea0 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.388+0000 7f5b3787a640 1 --2- 172.21.15.27:0/1295994919 >> v2:172.21.15.192:3300/0 conn(0x7f5b3015ae80 0x7f5b300d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.388+0000 7f5b3787a640 1 -- 172.21.15.27:0/1295994919 >> 172.21.15.27:0/1295994919 conn(0x7f5b300cd810 msgr2=0x7f5b300cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:19.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.388+0000 7f5b3787a640 1 -- 172.21.15.27:0/1295994919 shutdown_connections 2023-12-19T18:26:19.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.388+0000 7f5b3787a640 1 -- 172.21.15.27:0/1295994919 wait complete. 2023-12-19T18:26:19.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.390+0000 7f5b3787a640 1 Processor -- start 2023-12-19T18:26:19.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.390+0000 7f5b3787a640 1 -- start start 2023-12-19T18:26:19.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.391+0000 7f5b3787a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b3006ac80 0x7f5b301f0380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:19.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.391+0000 7f5b3787a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b300cea40 0x7f5b300c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:19.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.391+0000 7f5b3787a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b3015ae80 0x7f5b300b59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:19.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.391+0000 7f5b3787a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b301f08c0 con 0x7f5b300cea40 2023-12-19T18:26:19.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.391+0000 7f5b3787a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b300b4990 con 0x7f5b3015ae80 2023-12-19T18:26:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.391+0000 7f5b3787a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b300b3d80 con 0x7f5b3006ac80 2023-12-19T18:26:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.391+0000 7f5b34e15640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b300cea40 0x7f5b300c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.391+0000 7f5b34e15640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b300cea40 0x7f5b300c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35312/0 (socket says 172.21.15.27:35312) 2023-12-19T18:26:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.391+0000 7f5b34e15640 1 -- 172.21.15.27:0/1670887818 learned_addr learned my addr 172.21.15.27:0/1670887818 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.392+0000 7f5b35616640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b3006ac80 0x7f5b301f0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.392+0000 7f5b35e17640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b3015ae80 0x7f5b300b59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.392+0000 7f5b34e15640 1 -- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b3006ac80 msgr2=0x7f5b301f0380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.392+0000 7f5b34e15640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b3006ac80 0x7f5b301f0380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.392+0000 7f5b34e15640 1 -- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b3015ae80 msgr2=0x7f5b300b59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.392+0000 7f5b34e15640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b3015ae80 0x7f5b300b59c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.392+0000 7f5b34e15640 1 -- 172.21.15.27:0/1670887818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b200094d0 con 0x7f5b300cea40 2023-12-19T18:26:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.392+0000 7f5b35616640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b3006ac80 0x7f5b301f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.392+0000 7f5b34e15640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b300cea40 0x7f5b300c6fa0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f5b2002b8a0 tx=0x7f5b200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:19.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.392+0000 7f5b25ffb640 1 -- 172.21.15.27:0/1670887818 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b200329a0 con 0x7f5b300cea40 2023-12-19T18:26:19.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.392+0000 7f5b25ffb640 1 -- 172.21.15.27:0/1670887818 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b20032b40 con 0x7f5b300cea40 2023-12-19T18:26:19.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.392+0000 7f5b3787a640 1 -- 172.21.15.27:0/1670887818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b300b4060 con 0x7f5b300cea40 2023-12-19T18:26:19.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.392+0000 7f5b25ffb640 1 -- 172.21.15.27:0/1670887818 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b200453f0 con 0x7f5b300cea40 2023-12-19T18:26:19.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.393+0000 7f5b3787a640 1 -- 172.21.15.27:0/1670887818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b300b4620 con 0x7f5b300cea40 2023-12-19T18:26:19.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.394+0000 7f5b3787a640 1 -- 172.21.15.27:0/1670887818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3006c920 con 0x7f5b300cea40 2023-12-19T18:26:19.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.395+0000 7f5b25ffb640 1 -- 172.21.15.27:0/1670887818 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5b2004e450 con 0x7f5b300cea40 2023-12-19T18:26:19.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.396+0000 7f5b25ffb640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5b1406a460 0x7f5b1406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:19.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.396+0000 7f5b25ffb640 1 -- 172.21.15.27:0/1670887818 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5b200aace0 con 0x7f5b300cea40 2023-12-19T18:26:19.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.396+0000 7f5b25ffb640 1 -- 172.21.15.27:0/1670887818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f5b14071f90 con 0x7f5b300cea40 2023-12-19T18:26:19.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.396+0000 7f5b35616640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5b1406a460 0x7f5b1406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.398+0000 7f5b35616640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5b1406a460 0x7f5b1406c920 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5b30160530 tx=0x7f5b2c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:19.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.400+0000 7f5b25ffb640 1 -- 172.21.15.27:0/1670887818 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5b3006c920 con 0x7f5b300cea40 2023-12-19T18:26:19.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.629+0000 7f5b3787a640 1 -- 172.21.15.27:0/1670887818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5b3006c920 con 0x7f5b300cea40 2023-12-19T18:26:19.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.630+0000 7f5b25ffb640 1 -- 172.21.15.27:0/1670887818 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f5b3006c920 con 0x7f5b300cea40 2023-12-19T18:26:19.630 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:19.630 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":69,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3291476389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3291476389},{"type":"v1","addr":"172.21.15.192:6836","nonce":3291476389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3526330256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3526330256},{"type":"v1","addr":"172.21.15.192:6837","nonce":3526330256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/899234003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899234003},{"type":"v1","addr":"172.21.15.27:6835","nonce":899234003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:24:08.343302+0000","modified":"2023-12-19T18:24:11.863775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8247":{"gid":8247,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6833/2519815591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2519815591},{"type":"v1","addr":"172.21.15.27:6833","nonce":2519815591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-19T18:26:19.631 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 69 2023-12-19T18:26:19.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.633+0000 7f5b3787a640 1 -- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5b1406a460 msgr2=0x7f5b1406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:19.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.633+0000 7f5b3787a640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5b1406a460 0x7f5b1406c920 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5b30160530 tx=0x7f5b2c008040 comp rx=0 tx=0).stop 2023-12-19T18:26:19.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.633+0000 7f5b3787a640 1 -- 172.21.15.27:0/1670887818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b300cea40 msgr2=0x7f5b300c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:19.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.633+0000 7f5b3787a640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b300cea40 0x7f5b300c6fa0 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7f5b2002b8a0 tx=0x7f5b200046c0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.633+0000 7f5b3787a640 1 -- 172.21.15.27:0/1670887818 shutdown_connections 2023-12-19T18:26:19.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.633+0000 7f5b3787a640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5b1406a460 0x7f5b1406c920 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.633+0000 7f5b3787a640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5b3015ae80 0x7f5b300b59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.633+0000 7f5b3787a640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b300cea40 0x7f5b300c6fa0 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.633+0000 7f5b3787a640 1 --2- 172.21.15.27:0/1670887818 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5b3006ac80 0x7f5b301f0380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.633+0000 7f5b3787a640 1 -- 172.21.15.27:0/1670887818 >> 172.21.15.27:0/1670887818 conn(0x7f5b300cd810 msgr2=0x7f5b3015e900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:19.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.633+0000 7f5b3787a640 1 -- 172.21.15.27:0/1670887818 shutdown_connections 2023-12-19T18:26:19.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.633+0000 7f5b3787a640 1 -- 172.21.15.27:0/1670887818 wait complete. 2023-12-19T18:26:19.651 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:26:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.874+0000 7fcb60aaf640 1 Processor -- start 2023-12-19T18:26:19.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.874+0000 7fcb60aaf640 1 -- start start 2023-12-19T18:26:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.876+0000 7fcb60aaf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcb5c15af70 0x7fcb5c0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.876+0000 7fcb60aaf640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcb5c0adc20 0x7fcb5c06b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.876+0000 7fcb60aaf640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb5c0cb850 con 0x7fcb5c0cbce0 2023-12-19T18:26:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.876+0000 7fcb60aaf640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5c15b350 con 0x7fcb5c0adc20 2023-12-19T18:26:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.876+0000 7fcb60aaf640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb5c0cac40 con 0x7fcb5c15af70 2023-12-19T18:26:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.876+0000 7fcb59d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcb5c0cbce0 0x7fcb5c0ad510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40080/0 (socket says 172.21.15.27:40080) 2023-12-19T18:26:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.876+0000 7fcb59d9b640 1 -- 172.21.15.27:0/902514209 learned_addr learned my addr 172.21.15.27:0/902514209 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:19.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.876+0000 7fcb5ad9d640 1 --2- 172.21.15.27:0/902514209 >> v2:172.21.15.27:3300/0 conn(0x7fcb5c0adc20 0x7fcb5c06b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.877+0000 7fcb5a59c640 1 --2- 172.21.15.27:0/902514209 >> v2:172.21.15.192:3300/0 conn(0x7fcb5c15af70 0x7fcb5c0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.877+0000 7fcb5959a640 1 -- 172.21.15.27:0/902514209 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897589117 0 0) 0x7fcb5c0cb850 con 0x7fcb5c0cbce0 2023-12-19T18:26:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.877+0000 7fcb5959a640 1 -- 172.21.15.27:0/902514209 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb44003520 con 0x7fcb5c0cbce0 2023-12-19T18:26:19.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.877+0000 7fcb5ad9d640 1 -- 172.21.15.27:0/902514209 >> v2:172.21.15.192:3300/0 conn(0x7fcb5c15af70 msgr2=0x7fcb5c0ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:19.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.877+0000 7fcb5ad9d640 1 --2- 172.21.15.27:0/902514209 >> v2:172.21.15.192:3300/0 conn(0x7fcb5c15af70 0x7fcb5c0ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.877+0000 7fcb5ad9d640 1 -- 172.21.15.27:0/902514209 >> v1:172.21.15.27:6789/0 conn(0x7fcb5c0cbce0 legacy=0x7fcb5c0ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:19.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.877+0000 7fcb5ad9d640 1 -- 172.21.15.27:0/902514209 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb5c0caf20 con 0x7fcb5c0adc20 2023-12-19T18:26:19.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.878+0000 7fcb5ad9d640 1 --2- 172.21.15.27:0/902514209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb5c0adc20 0x7fcb5c06b4e0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c06d940 tx=0x7fcb5402bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fc8883d8dafe0a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:19.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.878+0000 7fcb5959a640 1 -- 172.21.15.27:0/902514209 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5402cc80 con 0x7fcb5c0adc20 2023-12-19T18:26:19.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.878+0000 7fcb5959a640 1 -- 172.21.15.27:0/902514209 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5402cde0 con 0x7fcb5c0adc20 2023-12-19T18:26:19.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.878+0000 7fcb5959a640 1 -- 172.21.15.27:0/902514209 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54034640 con 0x7fcb5c0adc20 2023-12-19T18:26:19.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.879+0000 7fcb60aaf640 1 -- 172.21.15.27:0/902514209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb5c0adc20 msgr2=0x7fcb5c06b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:19.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.879+0000 7fcb60aaf640 1 --2- 172.21.15.27:0/902514209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb5c0adc20 0x7fcb5c06b4e0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fcb5c06d940 tx=0x7fcb5402bed0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.879+0000 7fcb60aaf640 1 -- 172.21.15.27:0/902514209 shutdown_connections 2023-12-19T18:26:19.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.879+0000 7fcb60aaf640 1 --2- 172.21.15.27:0/902514209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb5c0adc20 0x7fcb5c06b4e0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.879+0000 7fcb60aaf640 1 --2- 172.21.15.27:0/902514209 >> v2:172.21.15.192:3300/0 conn(0x7fcb5c15af70 0x7fcb5c0ca540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.879+0000 7fcb60aaf640 1 -- 172.21.15.27:0/902514209 >> 172.21.15.27:0/902514209 conn(0x7fcb5c06a330 msgr2=0x7fcb5c0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:19.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.879+0000 7fcb60aaf640 1 -- 172.21.15.27:0/902514209 shutdown_connections 2023-12-19T18:26:19.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.879+0000 7fcb60aaf640 1 -- 172.21.15.27:0/902514209 wait complete. 2023-12-19T18:26:19.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.881+0000 7fcb60aaf640 1 Processor -- start 2023-12-19T18:26:19.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.881+0000 7fcb60aaf640 1 -- start start 2023-12-19T18:26:19.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.882+0000 7fcb60aaf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb5c0adc20 0x7fcb5c15b490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:19.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.882+0000 7fcb60aaf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb5c0cbce0 0x7fcb5c0ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:19.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.882+0000 7fcb60aaf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb5c15af70 0x7fcb5c0c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:19.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.882+0000 7fcb60aaf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c0c7d40 con 0x7fcb5c15af70 2023-12-19T18:26:19.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.882+0000 7fcb60aaf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c0c7130 con 0x7fcb5c0cbce0 2023-12-19T18:26:19.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.882+0000 7fcb60aaf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb5c0c7410 con 0x7fcb5c0adc20 2023-12-19T18:26:19.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.883+0000 7fcb5ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb5c15af70 0x7fcb5c0c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:19.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.883+0000 7fcb5ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb5c15af70 0x7fcb5c0c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35334/0 (socket says 172.21.15.27:35334) 2023-12-19T18:26:19.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.883+0000 7fcb5ad9d640 1 -- 172.21.15.27:0/2455451332 learned_addr learned my addr 172.21.15.27:0/2455451332 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:19.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.883+0000 7fcb59d9b640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb5c0cbce0 0x7fcb5c0ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:19.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.883+0000 7fcb5a59c640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb5c0adc20 0x7fcb5c15b490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:19.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.883+0000 7fcb5ad9d640 1 -- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb5c0adc20 msgr2=0x7fcb5c15b490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:19.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.883+0000 7fcb5ad9d640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb5c0adc20 0x7fcb5c15b490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.883+0000 7fcb5ad9d640 1 -- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb5c0cbce0 msgr2=0x7fcb5c0ad220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:19.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.883+0000 7fcb5ad9d640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb5c0cbce0 0x7fcb5c0ad220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:19.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.883+0000 7fcb5ad9d640 1 -- 172.21.15.27:0/2455451332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb54009a70 con 0x7fcb5c15af70 2023-12-19T18:26:19.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.883+0000 7fcb59d9b640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb5c0cbce0 0x7fcb5c0ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:19.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.883+0000 7fcb5a59c640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb5c0adc20 0x7fcb5c15b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:19.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.884+0000 7fcb5ad9d640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb5c15af70 0x7fcb5c0c81d0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fcb5402c4e0 tx=0x7fcb54004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:19.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.884+0000 7fcb4affd640 1 -- 172.21.15.27:0/2455451332 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5403e610 con 0x7fcb5c15af70 2023-12-19T18:26:19.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.884+0000 7fcb4affd640 1 -- 172.21.15.27:0/2455451332 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5403e770 con 0x7fcb5c15af70 2023-12-19T18:26:19.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.884+0000 7fcb4affd640 1 -- 172.21.15.27:0/2455451332 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5403eb60 con 0x7fcb5c15af70 2023-12-19T18:26:19.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.884+0000 7fcb60aaf640 1 -- 172.21.15.27:0/2455451332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb5c0c76f0 con 0x7fcb5c15af70 2023-12-19T18:26:19.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.884+0000 7fcb60aaf640 1 -- 172.21.15.27:0/2455451332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb5c15b9d0 con 0x7fcb5c15af70 2023-12-19T18:26:19.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.885+0000 7fcb60aaf640 1 -- 172.21.15.27:0/2455451332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb5c0c7eb0 con 0x7fcb5c15af70 2023-12-19T18:26:19.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.886+0000 7fcb4affd640 1 -- 172.21.15.27:0/2455451332 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcb5403b050 con 0x7fcb5c15af70 2023-12-19T18:26:19.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.888+0000 7fcb4affd640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb3c06a430 0x7fcb3c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:19.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.888+0000 7fcb4affd640 1 -- 172.21.15.27:0/2455451332 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcb540aab60 con 0x7fcb5c15af70 2023-12-19T18:26:19.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.888+0000 7fcb4affd640 1 -- 172.21.15.27:0/2455451332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fcb3c071f60 con 0x7fcb5c15af70 2023-12-19T18:26:19.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.889+0000 7fcb5a59c640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb3c06a430 0x7fcb3c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:19.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.890+0000 7fcb5a59c640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb3c06a430 0x7fcb3c06c8f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c006fd0 tx=0x7fcb4c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:19.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:19.893+0000 7fcb4affd640 1 -- 172.21.15.27:0/2455451332 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcb54039060 con 0x7fcb5c15af70 2023-12-19T18:26:20.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.083+0000 7fcb60aaf640 1 -- 172.21.15.27:0/2455451332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb5c068e90 con 0x7fcb5c15af70 2023-12-19T18:26:20.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.086+0000 7fcb4affd640 1 -- 172.21.15.27:0/2455451332 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16086 (secure 0 0 0) 0x7fcb5404d030 con 0x7fcb5c15af70 2023-12-19T18:26:20.087 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:20.087 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":172,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:25:26.756190+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:24:06.280682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T18:24:07.374789+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:26:20.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.089+0000 7fcb60aaf640 1 -- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb3c06a430 msgr2=0x7fcb3c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.089+0000 7fcb60aaf640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb3c06a430 0x7fcb3c06c8f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fcb4c006fd0 tx=0x7fcb4c008040 comp rx=0 tx=0).stop 2023-12-19T18:26:20.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.089+0000 7fcb60aaf640 1 -- 172.21.15.27:0/2455451332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb5c15af70 msgr2=0x7fcb5c0c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.089+0000 7fcb60aaf640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb5c15af70 0x7fcb5c0c81d0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7fcb5402c4e0 tx=0x7fcb54004a40 comp rx=0 tx=0).stop 2023-12-19T18:26:20.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.089+0000 7fcb60aaf640 1 -- 172.21.15.27:0/2455451332 shutdown_connections 2023-12-19T18:26:20.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.089+0000 7fcb60aaf640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcb3c06a430 0x7fcb3c06c8f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.089+0000 7fcb60aaf640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb5c15af70 0x7fcb5c0c81d0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.089+0000 7fcb60aaf640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcb5c0cbce0 0x7fcb5c0ad220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.089+0000 7fcb60aaf640 1 --2- 172.21.15.27:0/2455451332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcb5c0adc20 0x7fcb5c15b490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.089+0000 7fcb60aaf640 1 -- 172.21.15.27:0/2455451332 >> 172.21.15.27:0/2455451332 conn(0x7fcb5c06a330 msgr2=0x7fcb5c159e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:20.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.089+0000 7fcb60aaf640 1 -- 172.21.15.27:0/2455451332 shutdown_connections 2023-12-19T18:26:20.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.089+0000 7fcb60aaf640 1 -- 172.21.15.27:0/2455451332 wait complete. 2023-12-19T18:26:20.106 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:24:06.280682+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '169', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.25, 'score_stable': 1.25, 'optimal_score': 1, 'raw_score_acting': 1.25, 'raw_score_stable': 1.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:24:07.374789+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:26:20.107 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:26:20.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.324+0000 7f0a3259b640 1 Processor -- start 2023-12-19T18:26:20.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.324+0000 7f0a3259b640 1 -- start start 2023-12-19T18:26:20.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.325+0000 7f0a3259b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0cdaa0 0x7f0a2c0cf110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.325+0000 7f0a3259b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a2c0b3fd0 con 0x7f0a2c15aed0 2023-12-19T18:26:20.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.325+0000 7f0a3259b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a2c0cf6f0 con 0x7f0a2c0d1450 2023-12-19T18:26:20.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.325+0000 7f0a3259b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c0cf9d0 con 0x7f0a2c0cdaa0 2023-12-19T18:26:20.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.326+0000 7f0a31599640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0a2c15aed0 0x7f0a2c0cd390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40090/0 (socket says 172.21.15.27:40090) 2023-12-19T18:26:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.326+0000 7f0a31599640 1 -- 172.21.15.27:0/1259647125 learned_addr learned my addr 172.21.15.27:0/1259647125 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.326+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2097179833 0 0) 0x7f0a2c0b3fd0 con 0x7f0a2c15aed0 2023-12-19T18:26:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.326+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a14003520 con 0x7f0a2c15aed0 2023-12-19T18:26:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.326+0000 7f0a30d98640 1 --2- 172.21.15.27:0/1259647125 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0cdaa0 0x7f0a2c0cf110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.326+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2419680363 0 0) 0x7f0a14003520 con 0x7f0a2c15aed0 2023-12-19T18:26:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.326+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0cdaa0 msgr2=0x7f0a2c0cf110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a2bfff640 1 --2- 172.21.15.27:0/1259647125 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0cdaa0 0x7f0a2c0cf110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 >> v1:172.21.15.192:6789/0 conn(0x7f0a2c0d1450 legacy=0x7f0a2c0b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2c0b3980 con 0x7f0a2c15aed0 2023-12-19T18:26:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f0a1c001670 con 0x7f0a2c15aed0 2023-12-19T18:26:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 >> v1:172.21.15.27:6789/0 conn(0x7f0a2c15aed0 legacy=0x7f0a2c0cd390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a2bfff640 1 --2- 172.21.15.27:0/1259647125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a140039a0 0x7f0a14003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a2bfff640 1 --2- 172.21.15.27:0/1259647125 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0cdaa0 0x7f0a140047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a2bfff640 1 --2- 172.21.15.27:0/1259647125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a14004d70 0x7f0a140051f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c0cf9d0 con 0x7f0a14004d70 2023-12-19T18:26:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1400bc50 con 0x7f0a140039a0 2023-12-19T18:26:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1400be10 con 0x7f0a2c0cdaa0 2023-12-19T18:26:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a31d9a640 1 --2- 172.21.15.27:0/1259647125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a14004d70 0x7f0a140051f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a30d98640 1 --2- 172.21.15.27:0/1259647125 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0cdaa0 0x7f0a140047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a31599640 1 --2- 172.21.15.27:0/1259647125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a140039a0 0x7f0a14003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.327+0000 7f0a31d9a640 1 -- 172.21.15.27:0/1259647125 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0cdaa0 msgr2=0x7f0a140047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.328+0000 7f0a31d9a640 1 --2- 172.21.15.27:0/1259647125 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0cdaa0 0x7f0a140047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.328+0000 7f0a31d9a640 1 -- 172.21.15.27:0/1259647125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a140039a0 msgr2=0x7f0a14003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.328+0000 7f0a31d9a640 1 --2- 172.21.15.27:0/1259647125 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a140039a0 0x7f0a14003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.328+0000 7f0a31d9a640 1 -- 172.21.15.27:0/1259647125 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a14003520 con 0x7f0a14004d70 2023-12-19T18:26:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.329+0000 7f0a31d9a640 1 --2- 172.21.15.27:0/1259647125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a14004d70 0x7f0a140051f0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f0a240044b0 tx=0x7f0a140057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dfb26020f0d592a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.329+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24030620 con 0x7f0a14004d70 2023-12-19T18:26:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.329+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a24030780 con 0x7f0a14004d70 2023-12-19T18:26:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.329+0000 7f0a2bfff640 1 -- 172.21.15.27:0/1259647125 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24030b30 con 0x7f0a14004d70 2023-12-19T18:26:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.330+0000 7f0a3259b640 1 -- 172.21.15.27:0/1259647125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a14004d70 msgr2=0x7f0a140051f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.330+0000 7f0a3259b640 1 --2- 172.21.15.27:0/1259647125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a14004d70 0x7f0a140051f0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f0a240044b0 tx=0x7f0a140057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.330+0000 7f0a31d9a640 1 -- 172.21.15.27:0/1259647125 reap_dead start 2023-12-19T18:26:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.330+0000 7f0a3259b640 1 -- 172.21.15.27:0/1259647125 shutdown_connections 2023-12-19T18:26:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.330+0000 7f0a3259b640 1 -- 172.21.15.27:0/1259647125 >> 172.21.15.27:0/1259647125 conn(0x7f0a2c099ee0 msgr2=0x7f0a2c0d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.330+0000 7f0a3259b640 1 -- 172.21.15.27:0/1259647125 shutdown_connections 2023-12-19T18:26:20.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.330+0000 7f0a3259b640 1 -- 172.21.15.27:0/1259647125 wait complete. 2023-12-19T18:26:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.332+0000 7f0a3259b640 1 Processor -- start 2023-12-19T18:26:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.332+0000 7f0a3259b640 1 -- start start 2023-12-19T18:26:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a3259b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a14004d70 0x7f0a2c158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a3259b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0c8c50 0x7f0a2c1543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a3259b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2c06e7f0 0x7f0a2c06ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a3259b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c06f1b0 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a3259b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c0c9f90 con 0x7f0a14004d70 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a3259b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a2c0c90d0 con 0x7f0a2c0c8c50 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a31d9a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2c06e7f0 0x7f0a2c06ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a31d9a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2c06e7f0 0x7f0a2c06ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35350/0 (socket says 172.21.15.27:35350) 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a31d9a640 1 -- 172.21.15.27:0/866708215 learned_addr learned my addr 172.21.15.27:0/866708215 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a30d98640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0c8c50 0x7f0a2c1543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a31599640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a14004d70 0x7f0a2c158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a31d9a640 1 -- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0c8c50 msgr2=0x7f0a2c1543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a31d9a640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0c8c50 0x7f0a2c1543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a31d9a640 1 -- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a14004d70 msgr2=0x7f0a2c158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a31d9a640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a14004d70 0x7f0a2c158610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a31d9a640 1 -- 172.21.15.27:0/866708215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a2c1548f0 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.333+0000 7f0a30d98640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0c8c50 0x7f0a2c1543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.334+0000 7f0a31d9a640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2c06e7f0 0x7f0a2c06ec70 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f0a240045d0 tx=0x7f0a24003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:20.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.334+0000 7f0a29ffb640 1 -- 172.21.15.27:0/866708215 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a2403f070 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.334+0000 7f0a29ffb640 1 -- 172.21.15.27:0/866708215 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a24030b00 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.334+0000 7f0a3259b640 1 -- 172.21.15.27:0/866708215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a2c0c9380 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.334+0000 7f0a29ffb640 1 -- 172.21.15.27:0/866708215 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a24030e70 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.334+0000 7f0a3259b640 1 -- 172.21.15.27:0/866708215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a2c0c9940 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.335+0000 7f0a3259b640 1 -- 172.21.15.27:0/866708215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a2c04e840 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.336+0000 7f0a29ffb640 1 -- 172.21.15.27:0/866708215 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0a24039450 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.337+0000 7f0a29ffb640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a1806a430 0x7f0a1806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.337+0000 7f0a29ffb640 1 -- 172.21.15.27:0/866708215 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a240aa5d0 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.338+0000 7f0a29ffb640 1 -- 172.21.15.27:0/866708215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f0a18071f60 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.338+0000 7f0a31599640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a1806a430 0x7f0a1806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:20.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.339+0000 7f0a31599640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a1806a430 0x7f0a1806c8f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0016b0 tx=0x7f0a1c001c90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:20.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.341+0000 7f0a29ffb640 1 -- 172.21.15.27:0/866708215 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0a24034020 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.534 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.534+0000 7f0a3259b640 1 -- 172.21.15.27:0/866708215 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0a2c15a130 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.536+0000 7f0a29ffb640 1 -- 172.21.15.27:0/866708215 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v172) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0a24079640 con 0x7f0a2c06e7f0 2023-12-19T18:26:20.536 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:26:20.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.538+0000 7f0a3259b640 1 -- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a1806a430 msgr2=0x7f0a1806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.538+0000 7f0a3259b640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a1806a430 0x7f0a1806c8f0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f0a1c0016b0 tx=0x7f0a1c001c90 comp rx=0 tx=0).stop 2023-12-19T18:26:20.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.538+0000 7f0a3259b640 1 -- 172.21.15.27:0/866708215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2c06e7f0 msgr2=0x7f0a2c06ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.538+0000 7f0a3259b640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2c06e7f0 0x7f0a2c06ec70 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f0a240045d0 tx=0x7f0a24003a70 comp rx=0 tx=0).stop 2023-12-19T18:26:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.538+0000 7f0a3259b640 1 -- 172.21.15.27:0/866708215 shutdown_connections 2023-12-19T18:26:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.538+0000 7f0a3259b640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0a1806a430 0x7f0a1806c8f0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.538+0000 7f0a3259b640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0a2c06e7f0 0x7f0a2c06ec70 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.538+0000 7f0a3259b640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0a2c0c8c50 0x7f0a2c1543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.538+0000 7f0a3259b640 1 --2- 172.21.15.27:0/866708215 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0a14004d70 0x7f0a2c158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.538+0000 7f0a3259b640 1 -- 172.21.15.27:0/866708215 >> 172.21.15.27:0/866708215 conn(0x7f0a2c099ee0 msgr2=0x7f0a2c15d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.538+0000 7f0a3259b640 1 -- 172.21.15.27:0/866708215 shutdown_connections 2023-12-19T18:26:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.539+0000 7f0a3259b640 1 -- 172.21.15.27:0/866708215 wait complete. 2023-12-19T18:26:20.556 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:26:20.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.779+0000 7ff2ade7e640 1 Processor -- start 2023-12-19T18:26:20.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.779+0000 7ff2ade7e640 1 -- start start 2023-12-19T18:26:20.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.780+0000 7ff2ade7e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2a815cf80 0x7ff2a80d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.780+0000 7ff2ade7e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff2a80b85e0 0x7ff2a80cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.780+0000 7ff2ade7e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2a80d0e10 con 0x7ff2a80b8a60 2023-12-19T18:26:20.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.780+0000 7ff2ade7e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2a80d1c70 con 0x7ff2a80b85e0 2023-12-19T18:26:20.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.780+0000 7ff2ade7e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a815d360 con 0x7ff2a815cf80 2023-12-19T18:26:20.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.781+0000 7ff2ad67d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff2a80b8a60 0x7ff2a80cf500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40104/0 (socket says 172.21.15.27:40104) 2023-12-19T18:26:20.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.781+0000 7ff2ad67d640 1 -- 172.21.15.27:0/720214333 learned_addr learned my addr 172.21.15.27:0/720214333 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:20.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.781+0000 7ff2a7fff640 1 --2- 172.21.15.27:0/720214333 >> v2:172.21.15.27:3300/0 conn(0x7ff2a80b85e0 0x7ff2a80cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.781+0000 7ff2ace7c640 1 --2- 172.21.15.27:0/720214333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2a815cf80 0x7ff2a80d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.781+0000 7ff2a77fe640 1 -- 172.21.15.27:0/720214333 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472561278 0 0) 0x7ff2a80d0e10 con 0x7ff2a80b8a60 2023-12-19T18:26:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.781+0000 7ff2a7fff640 1 -- 172.21.15.27:0/720214333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2a815cf80 msgr2=0x7ff2a80d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.781+0000 7ff2a7fff640 1 --2- 172.21.15.27:0/720214333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2a815cf80 0x7ff2a80d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.781+0000 7ff2a7fff640 1 -- 172.21.15.27:0/720214333 >> v1:172.21.15.27:6789/0 conn(0x7ff2a80b8a60 legacy=0x7ff2a80cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.781+0000 7ff2a7fff640 1 -- 172.21.15.27:0/720214333 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a80d1060 con 0x7ff2a80b85e0 2023-12-19T18:26:20.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.783+0000 7ff2a7fff640 1 --2- 172.21.15.27:0/720214333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a80b85e0 0x7ff2a80cda40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0097a0 tx=0x7ff29c02b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fcb75496dd63eaa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:20.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.783+0000 7ff2a77fe640 1 -- 172.21.15.27:0/720214333 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c030610 con 0x7ff2a80b85e0 2023-12-19T18:26:20.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.783+0000 7ff2a77fe640 1 -- 172.21.15.27:0/720214333 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c030770 con 0x7ff2a80b85e0 2023-12-19T18:26:20.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.783+0000 7ff2a77fe640 1 -- 172.21.15.27:0/720214333 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c030b00 con 0x7ff2a80b85e0 2023-12-19T18:26:20.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.783+0000 7ff2ade7e640 1 -- 172.21.15.27:0/720214333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a80b85e0 msgr2=0x7ff2a80cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.783+0000 7ff2ade7e640 1 --2- 172.21.15.27:0/720214333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a80b85e0 0x7ff2a80cda40 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0097a0 tx=0x7ff29c02b960 comp rx=0 tx=0).stop 2023-12-19T18:26:20.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.784+0000 7ff2ade7e640 1 -- 172.21.15.27:0/720214333 shutdown_connections 2023-12-19T18:26:20.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.784+0000 7ff2ade7e640 1 --2- 172.21.15.27:0/720214333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a80b85e0 0x7ff2a80cda40 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.784+0000 7ff2ade7e640 1 --2- 172.21.15.27:0/720214333 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2a815cf80 0x7ff2a80d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.784+0000 7ff2ade7e640 1 -- 172.21.15.27:0/720214333 >> 172.21.15.27:0/720214333 conn(0x7ff2a8072e20 msgr2=0x7ff2a8073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:20.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.784+0000 7ff2ade7e640 1 -- 172.21.15.27:0/720214333 shutdown_connections 2023-12-19T18:26:20.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.784+0000 7ff2ade7e640 1 -- 172.21.15.27:0/720214333 wait complete. 2023-12-19T18:26:20.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.785+0000 7ff2ade7e640 1 Processor -- start 2023-12-19T18:26:20.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.786+0000 7ff2ade7e640 1 -- start start 2023-12-19T18:26:20.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.786+0000 7ff2ade7e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2a80b85e0 0x7ff2a80cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.786+0000 7ff2ade7e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a815cf80 0x7ff2a8152220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.786+0000 7ff2ade7e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2a80cb2b0 0x7ff2a815d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.786+0000 7ff2ade7e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80cc3c0 con 0x7ff2a815cf80 2023-12-19T18:26:20.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.786+0000 7ff2ade7e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a8152760 con 0x7ff2a80cb2b0 2023-12-19T18:26:20.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.786+0000 7ff2ade7e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a80cb7b0 con 0x7ff2a80b85e0 2023-12-19T18:26:20.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.787+0000 7ff2a7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a815cf80 0x7ff2a8152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:20.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.787+0000 7ff2a7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a815cf80 0x7ff2a8152220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35356/0 (socket says 172.21.15.27:35356) 2023-12-19T18:26:20.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.787+0000 7ff2a7fff640 1 -- 172.21.15.27:0/449652427 learned_addr learned my addr 172.21.15.27:0/449652427 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:20.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.787+0000 7ff2ace7c640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2a80b85e0 0x7ff2a80cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:20.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.787+0000 7ff2a7fff640 1 -- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2a80b85e0 msgr2=0x7ff2a80cc850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.787+0000 7ff2a7fff640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2a80b85e0 0x7ff2a80cc850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.787+0000 7ff2a7fff640 1 -- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2a80cb2b0 msgr2=0x7ff2a815d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.787+0000 7ff2a7fff640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2a80cb2b0 0x7ff2a815d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.787+0000 7ff2a7fff640 1 -- 172.21.15.27:0/449652427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c009400 con 0x7ff2a815cf80 2023-12-19T18:26:20.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.787+0000 7ff2ace7c640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2a80b85e0 0x7ff2a80cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:20.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.788+0000 7ff2a7fff640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a815cf80 0x7ff2a8152220 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0098c0 tx=0x7ff29c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:20.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.788+0000 7ff2a57fa640 1 -- 172.21.15.27:0/449652427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c03e040 con 0x7ff2a815cf80 2023-12-19T18:26:20.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.788+0000 7ff2a57fa640 1 -- 172.21.15.27:0/449652427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff29c039cb0 con 0x7ff2a815cf80 2023-12-19T18:26:20.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.788+0000 7ff2a57fa640 1 -- 172.21.15.27:0/449652427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff29c044630 con 0x7ff2a815cf80 2023-12-19T18:26:20.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.788+0000 7ff2ade7e640 1 -- 172.21.15.27:0/449652427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a80cba90 con 0x7ff2a815cf80 2023-12-19T18:26:20.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.788+0000 7ff2ade7e640 1 -- 172.21.15.27:0/449652427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a80cc050 con 0x7ff2a815cf80 2023-12-19T18:26:20.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.789+0000 7ff2ade7e640 1 -- 172.21.15.27:0/449652427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2a80cc530 con 0x7ff2a815cf80 2023-12-19T18:26:20.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.790+0000 7ff2a57fa640 1 -- 172.21.15.27:0/449652427 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff29c043390 con 0x7ff2a815cf80 2023-12-19T18:26:20.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.791+0000 7ff2a57fa640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff29406a430 0x7ff29406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:20.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.791+0000 7ff2a57fa640 1 -- 172.21.15.27:0/449652427 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff29c0aab80 con 0x7ff2a815cf80 2023-12-19T18:26:20.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.791+0000 7ff2a57fa640 1 -- 172.21.15.27:0/449652427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7ff294071f60 con 0x7ff2a815cf80 2023-12-19T18:26:20.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.792+0000 7ff2ace7c640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff29406a430 0x7ff29406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:20.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.793+0000 7ff2ace7c640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff29406a430 0x7ff29406c8f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff298009f50 tx=0x7ff29800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:20.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.795+0000 7ff2a57fa640 1 -- 172.21.15.27:0/449652427 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff29c076170 con 0x7ff2a815cf80 2023-12-19T18:26:20.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.992+0000 7ff2ade7e640 1 -- 172.21.15.27:0/449652427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff2a804e840 con 0x7ff2a815cf80 2023-12-19T18:26:20.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.993+0000 7ff2a57fa640 1 -- 172.21.15.27:0/449652427 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v172) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff29c0340a0 con 0x7ff2a815cf80 2023-12-19T18:26:20.993 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:26:20.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.995+0000 7ff2ade7e640 1 -- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff29406a430 msgr2=0x7ff29406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.995+0000 7ff2ade7e640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff29406a430 0x7ff29406c8f0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff298009f50 tx=0x7ff29800b040 comp rx=0 tx=0).stop 2023-12-19T18:26:20.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.995+0000 7ff2ade7e640 1 -- 172.21.15.27:0/449652427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a815cf80 msgr2=0x7ff2a8152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:20.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.995+0000 7ff2ade7e640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a815cf80 0x7ff2a8152220 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7ff29c0098c0 tx=0x7ff29c004730 comp rx=0 tx=0).stop 2023-12-19T18:26:20.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.996+0000 7ff2ade7e640 1 -- 172.21.15.27:0/449652427 shutdown_connections 2023-12-19T18:26:20.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.996+0000 7ff2ade7e640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff29406a430 0x7ff29406c8f0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.996+0000 7ff2ade7e640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff2a80cb2b0 0x7ff2a815d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.996+0000 7ff2ade7e640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2a815cf80 0x7ff2a8152220 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.996+0000 7ff2ade7e640 1 --2- 172.21.15.27:0/449652427 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff2a80b85e0 0x7ff2a80cc850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:20.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.996+0000 7ff2ade7e640 1 -- 172.21.15.27:0/449652427 >> 172.21.15.27:0/449652427 conn(0x7ff2a8072e20 msgr2=0x7ff2a815c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:20.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.996+0000 7ff2ade7e640 1 -- 172.21.15.27:0/449652427 shutdown_connections 2023-12-19T18:26:20.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:20.996+0000 7ff2ade7e640 1 -- 172.21.15.27:0/449652427 wait complete. 2023-12-19T18:26:21.014 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:26:21.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.251+0000 7f0d926dc640 1 Processor -- start 2023-12-19T18:26:21.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.251+0000 7f0d926dc640 1 -- start start 2023-12-19T18:26:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.253+0000 7f0d926dc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0d8c0d2100 0x7f0d8c0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.253+0000 7f0d926dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d8c0cdf80 0x7f0d8c0b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.253+0000 7f0d926dc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8c0d0e10 con 0x7f0d8c15cf80 2023-12-19T18:26:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.253+0000 7f0d926dc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8c0b8cf0 con 0x7f0d8c0d2100 2023-12-19T18:26:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.253+0000 7f0d926dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c0ce410 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.253+0000 7f0d90ed9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0d8c0d2100 0x7f0d8c0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.254+0000 7f0d90ed9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0d8c0d2100 0x7f0d8c0cda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35364/0 (socket says 172.21.15.27:35364) 2023-12-19T18:26:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.254+0000 7f0d90ed9640 1 -- 172.21.15.27:0/2257323144 learned_addr learned my addr 172.21.15.27:0/2257323144 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.254+0000 7f0d91edb640 1 --2- 172.21.15.27:0/2257323144 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d8c0cdf80 0x7f0d8c0b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:21.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.254+0000 7f0d90ed9640 1 -- 172.21.15.27:0/2257323144 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d8c0cdf80 msgr2=0x7f0d8c0b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:21.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.254+0000 7f0d90ed9640 1 --2- 172.21.15.27:0/2257323144 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d8c0cdf80 0x7f0d8c0b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.254+0000 7f0d90ed9640 1 -- 172.21.15.27:0/2257323144 >> v1:172.21.15.192:6789/0 conn(0x7f0d8c15cf80 legacy=0x7f0d8c0b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:21.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.254+0000 7f0d90ed9640 1 -- 172.21.15.27:0/2257323144 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c0d1c70 con 0x7f0d8c0d2100 2023-12-19T18:26:21.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.255+0000 7f0d90ed9640 1 --2- 172.21.15.27:0/2257323144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c0d2100 0x7f0d8c0cda40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009870 tx=0x7f0d7c02b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d49f02b7d45e5bc6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:21.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.256+0000 7f0d83fff640 1 -- 172.21.15.27:0/2257323144 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c030840 con 0x7f0d8c0d2100 2023-12-19T18:26:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.256+0000 7f0d83fff640 1 -- 172.21.15.27:0/2257323144 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c0309a0 con 0x7f0d8c0d2100 2023-12-19T18:26:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.256+0000 7f0d83fff640 1 -- 172.21.15.27:0/2257323144 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c030d30 con 0x7f0d8c0d2100 2023-12-19T18:26:21.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.256+0000 7f0d926dc640 1 -- 172.21.15.27:0/2257323144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c0d2100 msgr2=0x7f0d8c0cda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.256+0000 7f0d926dc640 1 --2- 172.21.15.27:0/2257323144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c0d2100 0x7f0d8c0cda40 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c009870 tx=0x7f0d7c02b980 comp rx=0 tx=0).stop 2023-12-19T18:26:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.256+0000 7f0d926dc640 1 -- 172.21.15.27:0/2257323144 shutdown_connections 2023-12-19T18:26:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.256+0000 7f0d926dc640 1 --2- 172.21.15.27:0/2257323144 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d8c0cdf80 0x7f0d8c0b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.257+0000 7f0d926dc640 1 --2- 172.21.15.27:0/2257323144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c0d2100 0x7f0d8c0cda40 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.257+0000 7f0d926dc640 1 -- 172.21.15.27:0/2257323144 >> 172.21.15.27:0/2257323144 conn(0x7f0d8c072e20 msgr2=0x7f0d8c073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.257+0000 7f0d926dc640 1 -- 172.21.15.27:0/2257323144 shutdown_connections 2023-12-19T18:26:21.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.257+0000 7f0d926dc640 1 -- 172.21.15.27:0/2257323144 wait complete. 2023-12-19T18:26:21.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.258+0000 7f0d926dc640 1 Processor -- start 2023-12-19T18:26:21.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.259+0000 7f0d926dc640 1 -- start start 2023-12-19T18:26:21.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d926dc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c0cdf80 0x7f0d8c0cc810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d926dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d8c0d2100 0x7f0d8c14d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d926dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d8c15cf80 0x7f0d8c14a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d926dc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c0ccd50 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d926dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c0ca8f0 con 0x7f0d8c15cf80 2023-12-19T18:26:21.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d926dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c0cab90 con 0x7f0d8c0d2100 2023-12-19T18:26:21.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d916da640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c0cdf80 0x7f0d8c0cc810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:21.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d916da640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c0cdf80 0x7f0d8c0cc810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35378/0 (socket says 172.21.15.27:35378) 2023-12-19T18:26:21.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d916da640 1 -- 172.21.15.27:0/3168789965 learned_addr learned my addr 172.21.15.27:0/3168789965 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:21.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d90ed9640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d8c0d2100 0x7f0d8c14d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d91edb640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d8c15cf80 0x7f0d8c14a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d916da640 1 -- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d8c0d2100 msgr2=0x7f0d8c14d920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d916da640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d8c0d2100 0x7f0d8c14d920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d916da640 1 -- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d8c15cf80 msgr2=0x7f0d8c14a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d916da640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d8c15cf80 0x7f0d8c14a830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d916da640 1 -- 172.21.15.27:0/3168789965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c0094d0 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.260+0000 7f0d90ed9640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d8c0d2100 0x7f0d8c14d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.261+0000 7f0d916da640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c0cdf80 0x7f0d8c0cc810 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800ba50 tx=0x7f0d8800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.261+0000 7f0d81ffb640 1 -- 172.21.15.27:0/3168789965 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8800cbe0 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.261+0000 7f0d81ffb640 1 -- 172.21.15.27:0/3168789965 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8800cd80 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.261+0000 7f0d81ffb640 1 -- 172.21.15.27:0/3168789965 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d88014680 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.261+0000 7f0d926dc640 1 -- 172.21.15.27:0/3168789965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8c0cae70 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.261+0000 7f0d926dc640 1 -- 172.21.15.27:0/3168789965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8c0cb430 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.263+0000 7f0d926dc640 1 -- 172.21.15.27:0/3168789965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8c0cb650 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.264+0000 7f0d81ffb640 1 -- 172.21.15.27:0/3168789965 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0d880147e0 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.264+0000 7f0d81ffb640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d7006a430 0x7f0d7006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.265+0000 7f0d81ffb640 1 -- 172.21.15.27:0/3168789965 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0d88088810 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.265+0000 7f0d81ffb640 1 -- 172.21.15.27:0/3168789965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7f0d70071f60 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.265+0000 7f0d90ed9640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d7006a430 0x7f0d7006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:21.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.266+0000 7f0d90ed9640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d7006a430 0x7f0d7006c8f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c036000 tx=0x7f0d7c03b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:21.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.269+0000 7f0d81ffb640 1 -- 172.21.15.27:0/3168789965 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0d88019060 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.464+0000 7f0d926dc640 1 -- 172.21.15.27:0/3168789965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0d8c04e840 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.465+0000 7f0d81ffb640 1 -- 172.21.15.27:0/3168789965 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v172) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0d88053eb0 con 0x7f0d8c0cdf80 2023-12-19T18:26:21.465 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:26:21.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.467+0000 7f0d926dc640 1 -- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d7006a430 msgr2=0x7f0d7006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:21.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.468+0000 7f0d926dc640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d7006a430 0x7f0d7006c8f0 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c036000 tx=0x7f0d7c03b040 comp rx=0 tx=0).stop 2023-12-19T18:26:21.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.468+0000 7f0d926dc640 1 -- 172.21.15.27:0/3168789965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c0cdf80 msgr2=0x7f0d8c0cc810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:21.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.468+0000 7f0d926dc640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c0cdf80 0x7f0d8c0cc810 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800ba50 tx=0x7f0d8800bf10 comp rx=0 tx=0).stop 2023-12-19T18:26:21.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.468+0000 7f0d926dc640 1 -- 172.21.15.27:0/3168789965 shutdown_connections 2023-12-19T18:26:21.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.468+0000 7f0d926dc640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d7006a430 0x7f0d7006c8f0 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.468+0000 7f0d926dc640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d8c15cf80 0x7f0d8c14a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.468+0000 7f0d926dc640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d8c0d2100 0x7f0d8c14d920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.468+0000 7f0d926dc640 1 --2- 172.21.15.27:0/3168789965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d8c0cdf80 0x7f0d8c0cc810 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.468+0000 7f0d926dc640 1 -- 172.21.15.27:0/3168789965 >> 172.21.15.27:0/3168789965 conn(0x7f0d8c072e20 msgr2=0x7f0d8c073200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:21.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.468+0000 7f0d926dc640 1 -- 172.21.15.27:0/3168789965 shutdown_connections 2023-12-19T18:26:21.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.468+0000 7f0d926dc640 1 -- 172.21.15.27:0/3168789965 wait complete. 2023-12-19T18:26:21.487 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:26:21.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.715+0000 7fa301b2e640 1 Processor -- start 2023-12-19T18:26:21.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.715+0000 7fa301b2e640 1 -- start start 2023-12-19T18:26:21.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.717+0000 7fa301b2e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa2fc06bbf0 0x7fa2fc0cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.717+0000 7fa301b2e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2fc15b180 con 0x7fa2fc06c880 2023-12-19T18:26:21.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.717+0000 7fa301b2e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2fc0ca850 con 0x7fa2fc159c40 2023-12-19T18:26:21.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.717+0000 7fa301b2e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2fc15a570 con 0x7fa2fc06bbf0 2023-12-19T18:26:21.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.717+0000 7fa2fbfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa2fc06c880 0x7fa2fc0cdd40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40106/0 (socket says 172.21.15.27:40106) 2023-12-19T18:26:21.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.717+0000 7fa2fbfff640 1 -- 172.21.15.27:0/343507687 learned_addr learned my addr 172.21.15.27:0/343507687 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:21.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.718+0000 7fa2f3fff640 1 --2- 172.21.15.27:0/343507687 >> v2:172.21.15.192:3300/0 conn(0x7fa2fc06bbf0 0x7fa2fc0cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.718+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2929321439 0 0) 0x7fa2fc15b180 con 0x7fa2fc06c880 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.718+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2e8005730 con 0x7fa2fc06c880 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.718+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3740707127 0 0) 0x7fa2e8005730 con 0x7fa2fc06c880 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.718+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 >> v2:172.21.15.192:3300/0 conn(0x7fa2fc06bbf0 msgr2=0x7fa2fc0cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.718+0000 7fa2faffd640 1 --2- 172.21.15.27:0/343507687 >> v2:172.21.15.192:3300/0 conn(0x7fa2fc06bbf0 0x7fa2fc0cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.718+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 >> v1:172.21.15.192:6789/0 conn(0x7fa2fc159c40 legacy=0x7fa2fc06b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.718+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc15a850 con 0x7fa2fc06c880 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.718+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fa2f40019f0 con 0x7fa2fc06c880 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.718+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 >> v1:172.21.15.27:6789/0 conn(0x7fa2fc06c880 legacy=0x7fa2fc0cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.718+0000 7fa2faffd640 1 --2- 172.21.15.27:0/343507687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8005bd0 0x7fa2e8005ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.718+0000 7fa2faffd640 1 --2- 172.21.15.27:0/343507687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e80069d0 0x7fa2e8006e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.719+0000 7fa2faffd640 1 --2- 172.21.15.27:0/343507687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2e8007390 0x7fa2e800db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.719+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc15a570 con 0x7fa2e8005bd0 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.719+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e800e290 con 0x7fa2e80069d0 2023-12-19T18:26:21.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.719+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e800e450 con 0x7fa2e8007390 2023-12-19T18:26:21.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.719+0000 7fa2fb7fe640 1 --2- 172.21.15.27:0/343507687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8005bd0 0x7fa2e8005ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:21.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.719+0000 7fa2f3fff640 1 --2- 172.21.15.27:0/343507687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e80069d0 0x7fa2e8006e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:21.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.719+0000 7fa2fb7fe640 1 -- 172.21.15.27:0/343507687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2e8007390 msgr2=0x7fa2e800db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:21.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.719+0000 7fa2fb7fe640 1 --2- 172.21.15.27:0/343507687 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2e8007390 0x7fa2e800db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.719+0000 7fa2fb7fe640 1 -- 172.21.15.27:0/343507687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e80069d0 msgr2=0x7fa2e8006e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:21.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.719+0000 7fa2fb7fe640 1 --2- 172.21.15.27:0/343507687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e80069d0 0x7fa2e8006e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.719+0000 7fa2fb7fe640 1 -- 172.21.15.27:0/343507687 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e8005730 con 0x7fa2e8005bd0 2023-12-19T18:26:21.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.719+0000 7fa2f3fff640 1 --2- 172.21.15.27:0/343507687 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2e80069d0 0x7fa2e8006e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:21.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.719+0000 7fa2fbfff640 1 -- 172.21.15.27:0/343507687 reap_dead start 2023-12-19T18:26:21.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.720+0000 7fa2fb7fe640 1 --2- 172.21.15.27:0/343507687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8005bd0 0x7fa2e8005ff0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec001330 tx=0x7fa2e8007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573e1c100051acb6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:21.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.721+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec0052d0 con 0x7fa2e8005bd0 2023-12-19T18:26:21.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.721+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2ec005430 con 0x7fa2e8005bd0 2023-12-19T18:26:21.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.721+0000 7fa2faffd640 1 -- 172.21.15.27:0/343507687 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2ec003830 con 0x7fa2e8005bd0 2023-12-19T18:26:21.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.721+0000 7fa301b2e640 1 -- 172.21.15.27:0/343507687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8005bd0 msgr2=0x7fa2e8005ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:21.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.721+0000 7fa301b2e640 1 --2- 172.21.15.27:0/343507687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8005bd0 0x7fa2e8005ff0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec001330 tx=0x7fa2e8007a00 comp rx=0 tx=0).stop 2023-12-19T18:26:21.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.721+0000 7fa301b2e640 1 -- 172.21.15.27:0/343507687 shutdown_connections 2023-12-19T18:26:21.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.721+0000 7fa301b2e640 1 --2- 172.21.15.27:0/343507687 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2e8005bd0 0x7fa2e8005ff0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.721+0000 7fa301b2e640 1 -- 172.21.15.27:0/343507687 >> 172.21.15.27:0/343507687 conn(0x7fa2fc06a330 msgr2=0x7fa2fc0b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:21.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.722+0000 7fa301b2e640 1 -- 172.21.15.27:0/343507687 shutdown_connections 2023-12-19T18:26:21.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.722+0000 7fa301b2e640 1 -- 172.21.15.27:0/343507687 wait complete. 2023-12-19T18:26:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.723+0000 7fa301b2e640 1 Processor -- start 2023-12-19T18:26:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.723+0000 7fa301b2e640 1 -- start start 2023-12-19T18:26:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.724+0000 7fa301b2e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2e8005bd0 0x7fa2fc0cdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.724+0000 7fa301b2e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2fc0cc360 0x7fa2fc0b3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.724+0000 7fa301b2e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0b40b0 0x7fa2fc1f1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.724+0000 7fa301b2e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0b4530 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.724+0000 7fa301b2e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0c3ee0 con 0x7fa2fc0cc360 2023-12-19T18:26:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.724+0000 7fa301b2e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc1f2840 con 0x7fa2e8005bd0 2023-12-19T18:26:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.725+0000 7fa2fbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0b40b0 0x7fa2fc1f1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.725+0000 7fa2fbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0b40b0 0x7fa2fc1f1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35398/0 (socket says 172.21.15.27:35398) 2023-12-19T18:26:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.725+0000 7fa2fbfff640 1 -- 172.21.15.27:0/2149405609 learned_addr learned my addr 172.21.15.27:0/2149405609 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.725+0000 7fa2f3fff640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2fc0cc360 0x7fa2fc0b3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.725+0000 7fa2fb7fe640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2e8005bd0 0x7fa2fc0cdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.725+0000 7fa2fbfff640 1 -- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2e8005bd0 msgr2=0x7fa2fc0cdee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.725+0000 7fa2fbfff640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2e8005bd0 0x7fa2fc0cdee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.725+0000 7fa2fbfff640 1 -- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2fc0cc360 msgr2=0x7fa2fc0b3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.725+0000 7fa2fbfff640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2fc0cc360 0x7fa2fc0b3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.725+0000 7fa2fbfff640 1 -- 172.21.15.27:0/2149405609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc0b1f00 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.725+0000 7fa2fb7fe640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2e8005bd0 0x7fa2fc0cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.725+0000 7fa2f3fff640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2fc0cc360 0x7fa2fc0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.726+0000 7fa2fbfff640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0b40b0 0x7fa2fc1f1910 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc06d880 tx=0x7fa2f400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.726+0000 7fa2f8ff9640 1 -- 172.21.15.27:0/2149405609 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f400a610 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.726+0000 7fa2f8ff9640 1 -- 172.21.15.27:0/2149405609 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2f400a770 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.726+0000 7fa2f8ff9640 1 -- 172.21.15.27:0/2149405609 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2f400ab20 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.726+0000 7fa301b2e640 1 -- 172.21.15.27:0/2149405609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2fc0ce420 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.726+0000 7fa301b2e640 1 -- 172.21.15.27:0/2149405609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2fc0b18b0 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.727+0000 7fa301b2e640 1 -- 172.21.15.27:0/2149405609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2fc0b2090 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.729+0000 7fa2f8ff9640 1 -- 172.21.15.27:0/2149405609 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa2f400ac80 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.730+0000 7fa2f8ff9640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa2dc06a460 0x7fa2dc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:21.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.730+0000 7fa2f8ff9640 1 -- 172.21.15.27:0/2149405609 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa2f4088110 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.730+0000 7fa2f8ff9640 1 -- 172.21.15.27:0/2149405609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=173}) v3 -- 0x7fa2dc071f90 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.730+0000 7fa2fb7fe640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa2dc06a460 0x7fa2dc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:21.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.732+0000 7fa2fb7fe640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa2dc06a460 0x7fa2dc06c920 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec038000 tx=0x7fa2ec009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:21.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.734+0000 7fa2f8ff9640 1 -- 172.21.15.27:0/2149405609 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa2f40537a0 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.781+0000 7fa2f8ff9640 1 -- 172.21.15.27:0/2149405609 <== mon.0 v2:172.21.15.27:3300/0 7 ==== osd_map(173..173 src has 1..173) v4 ==== 679+0+0 (secure 0 0 0) 0x7fa2f4057460 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.781+0000 7fa2f8ff9640 1 -- 172.21.15.27:0/2149405609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=174}) v3 -- 0x7fa2dc072ab0 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.979+0000 7fa301b2e640 1 -- 172.21.15.27:0/2149405609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2fc068e90 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.980+0000 7fa2f8ff9640 1 -- 172.21.15.27:0/2149405609 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4714 (secure 0 0 0) 0x7fa2f4056dc0 con 0x7fa2fc0b40b0 2023-12-19T18:26:21.980 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:21.980 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":69,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3291476389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3291476389},{"type":"v1","addr":"172.21.15.192:6836","nonce":3291476389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3526330256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3526330256},{"type":"v1","addr":"172.21.15.192:6837","nonce":3526330256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/899234003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899234003},{"type":"v1","addr":"172.21.15.27:6835","nonce":899234003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:24:08.343302+0000","modified":"2023-12-19T18:24:11.863775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8247":{"gid":8247,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6833/2519815591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2519815591},{"type":"v1","addr":"172.21.15.27:6833","nonce":2519815591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-19T18:26:21.980 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 69 2023-12-19T18:26:21.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.982+0000 7fa301b2e640 1 -- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa2dc06a460 msgr2=0x7fa2dc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:21.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.982+0000 7fa301b2e640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa2dc06a460 0x7fa2dc06c920 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa2ec038000 tx=0x7fa2ec009030 comp rx=0 tx=0).stop 2023-12-19T18:26:21.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.982+0000 7fa301b2e640 1 -- 172.21.15.27:0/2149405609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0b40b0 msgr2=0x7fa2fc1f1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:21.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.982+0000 7fa301b2e640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0b40b0 0x7fa2fc1f1910 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7fa2fc06d880 tx=0x7fa2f400ccd0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.983+0000 7fa301b2e640 1 -- 172.21.15.27:0/2149405609 shutdown_connections 2023-12-19T18:26:21.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.983+0000 7fa301b2e640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa2dc06a460 0x7fa2dc06c920 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.983+0000 7fa301b2e640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0b40b0 0x7fa2fc1f1910 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.983+0000 7fa301b2e640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa2fc0cc360 0x7fa2fc0b3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.983+0000 7fa301b2e640 1 --2- 172.21.15.27:0/2149405609 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa2e8005bd0 0x7fa2fc0cdee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:21.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.983+0000 7fa301b2e640 1 -- 172.21.15.27:0/2149405609 >> 172.21.15.27:0/2149405609 conn(0x7fa2fc06a330 msgr2=0x7fa2fc0b0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:21.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.983+0000 7fa301b2e640 1 -- 172.21.15.27:0/2149405609 shutdown_connections 2023-12-19T18:26:21.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:21.983+0000 7fa301b2e640 1 -- 172.21.15.27:0/2149405609 wait complete. 2023-12-19T18:26:22.002 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:26:22.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.245+0000 7f3f05eda640 1 Processor -- start 2023-12-19T18:26:22.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.245+0000 7f3f05eda640 1 -- start start 2023-12-19T18:26:22.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.247+0000 7f3f05eda640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3f000b4ef0 0x7f3f000b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:22.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.247+0000 7f3f05eda640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f000d2350 0x7f3f000b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:22.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.247+0000 7f3f05eda640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f000b5e90 con 0x7f3f001604d0 2023-12-19T18:26:22.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.247+0000 7f3f05eda640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f000d2b30 con 0x7f3f000d2350 2023-12-19T18:26:22.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.247+0000 7f3f05eda640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f000d2850 con 0x7f3f000b4ef0 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.247+0000 7f3f04ed8640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3f001604d0 0x7f3f000b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34868/0 (socket says 172.21.15.27:34868) 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.247+0000 7f3f04ed8640 1 -- 172.21.15.27:0/2868103220 learned_addr learned my addr 172.21.15.27:0/2868103220 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.247+0000 7f3f056d9640 1 --2- 172.21.15.27:0/2868103220 >> v2:172.21.15.27:3300/0 conn(0x7f3f000d2350 0x7f3f000b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.247+0000 7f3efffff640 1 --2- 172.21.15.27:0/2868103220 >> v2:172.21.15.192:3300/0 conn(0x7f3f000b4ef0 0x7f3f000b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.247+0000 7f3eff7fe640 1 -- 172.21.15.27:0/2868103220 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695909883 0 0) 0x7f3f000b5e90 con 0x7f3f001604d0 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.248+0000 7f3eff7fe640 1 -- 172.21.15.27:0/2868103220 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ee8003520 con 0x7f3f001604d0 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.248+0000 7f3f056d9640 1 -- 172.21.15.27:0/2868103220 >> v2:172.21.15.192:3300/0 conn(0x7f3f000b4ef0 msgr2=0x7f3f000b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.248+0000 7f3f056d9640 1 --2- 172.21.15.27:0/2868103220 >> v2:172.21.15.192:3300/0 conn(0x7f3f000b4ef0 0x7f3f000b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.248+0000 7f3f056d9640 1 -- 172.21.15.27:0/2868103220 >> v1:172.21.15.27:6789/0 conn(0x7f3f001604d0 legacy=0x7f3f000b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.248+0000 7f3f056d9640 1 -- 172.21.15.27:0/2868103220 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f000cded0 con 0x7f3f000d2350 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.248+0000 7f3efffff640 1 --2- 172.21.15.27:0/2868103220 >> v2:172.21.15.192:3300/0 conn(0x7f3f000b4ef0 0x7f3f000b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.249+0000 7f3f056d9640 1 --2- 172.21.15.27:0/2868103220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f000d2350 0x7f3f000b6840 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f3f00072120 tx=0x7f3ef802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d38c4ab0f0c993a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.249+0000 7f3eff7fe640 1 -- 172.21.15.27:0/2868103220 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8031830 con 0x7f3f000d2350 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.249+0000 7f3eff7fe640 1 -- 172.21.15.27:0/2868103220 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef8031990 con 0x7f3f000d2350 2023-12-19T18:26:22.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.249+0000 7f3eff7fe640 1 -- 172.21.15.27:0/2868103220 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef8031d00 con 0x7f3f000d2350 2023-12-19T18:26:22.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.250+0000 7f3f05eda640 1 -- 172.21.15.27:0/2868103220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f000d2350 msgr2=0x7f3f000b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:22.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.250+0000 7f3f05eda640 1 --2- 172.21.15.27:0/2868103220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f000d2350 0x7f3f000b6840 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f3f00072120 tx=0x7f3ef802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.250+0000 7f3f05eda640 1 -- 172.21.15.27:0/2868103220 shutdown_connections 2023-12-19T18:26:22.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.250+0000 7f3f05eda640 1 --2- 172.21.15.27:0/2868103220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f000d2350 0x7f3f000b6840 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.250+0000 7f3f05eda640 1 --2- 172.21.15.27:0/2868103220 >> v2:172.21.15.192:3300/0 conn(0x7f3f000b4ef0 0x7f3f000b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.250+0000 7f3f05eda640 1 -- 172.21.15.27:0/2868103220 >> 172.21.15.27:0/2868103220 conn(0x7f3f000cf7e0 msgr2=0x7f3f000cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:22.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.250+0000 7f3f05eda640 1 -- 172.21.15.27:0/2868103220 shutdown_connections 2023-12-19T18:26:22.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.250+0000 7f3f05eda640 1 -- 172.21.15.27:0/2868103220 wait complete. 2023-12-19T18:26:22.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.252+0000 7f3f05eda640 1 Processor -- start 2023-12-19T18:26:22.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.252+0000 7f3f05eda640 1 -- start start 2023-12-19T18:26:22.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.253+0000 7f3f05eda640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f000b4ef0 0x7f3f001f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:22.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.253+0000 7f3f05eda640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f000d2350 0x7f3f000b7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:22.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.253+0000 7f3f05eda640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f001604d0 0x7f3f00145f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:22.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.253+0000 7f3f05eda640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f000b6d60 con 0x7f3f001604d0 2023-12-19T18:26:22.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.253+0000 7f3f05eda640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f000b7000 con 0x7f3f000d2350 2023-12-19T18:26:22.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.253+0000 7f3f05eda640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f000b72e0 con 0x7f3f000b4ef0 2023-12-19T18:26:22.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.254+0000 7f3f056d9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f001604d0 0x7f3f00145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:22.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.254+0000 7f3f056d9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f001604d0 0x7f3f00145f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58684/0 (socket says 172.21.15.27:58684) 2023-12-19T18:26:22.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.254+0000 7f3f056d9640 1 -- 172.21.15.27:0/4121277876 learned_addr learned my addr 172.21.15.27:0/4121277876 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.254+0000 7f3efffff640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f000d2350 0x7f3f000b7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.254+0000 7f3f04ed8640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f000b4ef0 0x7f3f001f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:22.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.254+0000 7f3f056d9640 1 -- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f000b4ef0 msgr2=0x7f3f001f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.254+0000 7f3f056d9640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f000b4ef0 0x7f3f001f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.254+0000 7f3f056d9640 1 -- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f000d2350 msgr2=0x7f3f000b7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.254+0000 7f3f056d9640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f000d2350 0x7f3f000b7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.254+0000 7f3f056d9640 1 -- 172.21.15.27:0/4121277876 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ef8009400 con 0x7f3f001604d0 2023-12-19T18:26:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.254+0000 7f3f04ed8640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f000b4ef0 0x7f3f001f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.254+0000 7f3efffff640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f000d2350 0x7f3f000b7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.254+0000 7f3f056d9640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f001604d0 0x7f3f00145f70 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009890 tx=0x7f3ef80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.255+0000 7f3efd7fa640 1 -- 172.21.15.27:0/4121277876 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef803a630 con 0x7f3f001604d0 2023-12-19T18:26:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.255+0000 7f3efd7fa640 1 -- 172.21.15.27:0/4121277876 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ef803a790 con 0x7f3f001604d0 2023-12-19T18:26:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.255+0000 7f3efd7fa640 1 -- 172.21.15.27:0/4121277876 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ef803ab20 con 0x7f3f001604d0 2023-12-19T18:26:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.255+0000 7f3f05eda640 1 -- 172.21.15.27:0/4121277876 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f000b75c0 con 0x7f3f001604d0 2023-12-19T18:26:22.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.255+0000 7f3f05eda640 1 -- 172.21.15.27:0/4121277876 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f000b8300 con 0x7f3f001604d0 2023-12-19T18:26:22.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.256+0000 7f3f05eda640 1 -- 172.21.15.27:0/4121277876 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f000b7ae0 con 0x7f3f001604d0 2023-12-19T18:26:22.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.257+0000 7f3efd7fa640 1 -- 172.21.15.27:0/4121277876 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3ef803ac80 con 0x7f3f001604d0 2023-12-19T18:26:22.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.258+0000 7f3efd7fa640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3eec06a430 0x7f3eec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:22.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.258+0000 7f3efd7fa640 1 -- 172.21.15.27:0/4121277876 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ef80aa5e0 con 0x7f3f001604d0 2023-12-19T18:26:22.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.259+0000 7f3f04ed8640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3eec06a430 0x7f3eec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:22.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.260+0000 7f3f04ed8640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3eec06a430 0x7f3eec06c8f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0005c00 tx=0x7f3ef0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:22.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.263+0000 7f3efd7fa640 1 -- 172.21.15.27:0/4121277876 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3f000b7ae0 con 0x7f3f001604d0 2023-12-19T18:26:22.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.459+0000 7f3f05eda640 1 -- 172.21.15.27:0/4121277876 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3f000b7ae0 con 0x7f3f001604d0 2023-12-19T18:26:22.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.460+0000 7f3efd7fa640 1 -- 172.21.15.27:0/4121277876 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+16067 (secure 0 0 0) 0x7f3f000b7ae0 con 0x7f3f001604d0 2023-12-19T18:26:22.461 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:22.461 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":173,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:26:21.777719+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:24:06.280682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T18:24:07.374789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:26:22.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.463+0000 7f3f05eda640 1 -- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3eec06a430 msgr2=0x7f3eec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.463+0000 7f3f05eda640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3eec06a430 0x7f3eec06c8f0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f3ef0005c00 tx=0x7f3ef0009040 comp rx=0 tx=0).stop 2023-12-19T18:26:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.463+0000 7f3f05eda640 1 -- 172.21.15.27:0/4121277876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f001604d0 msgr2=0x7f3f00145f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.463+0000 7f3f05eda640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f001604d0 0x7f3f00145f70 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f3ef8009890 tx=0x7f3ef80046c0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.463+0000 7f3f05eda640 1 -- 172.21.15.27:0/4121277876 shutdown_connections 2023-12-19T18:26:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.463+0000 7f3f05eda640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3eec06a430 0x7f3eec06c8f0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.463+0000 7f3f05eda640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f001604d0 0x7f3f00145f70 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.463+0000 7f3f05eda640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f000d2350 0x7f3f000b7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.463+0000 7f3f05eda640 1 --2- 172.21.15.27:0/4121277876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f000b4ef0 0x7f3f001f1fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.463+0000 7f3f05eda640 1 -- 172.21.15.27:0/4121277876 >> 172.21.15.27:0/4121277876 conn(0x7f3f000cf7e0 msgr2=0x7f3f000cff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:22.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.463+0000 7f3f05eda640 1 -- 172.21.15.27:0/4121277876 shutdown_connections 2023-12-19T18:26:22.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.463+0000 7f3f05eda640 1 -- 172.21.15.27:0/4121277876 wait complete. 2023-12-19T18:26:22.484 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:26:22.484 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:26:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.703+0000 7f7ffe79b640 1 Processor -- start 2023-12-19T18:26:22.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.703+0000 7f7ffe79b640 1 -- start start 2023-12-19T18:26:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.705+0000 7f7ffe79b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7ff815ee30 0x7f7ff80c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.705+0000 7f7ffe79b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff80afeb0 0x7f7ff80c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.705+0000 7f7ffe79b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ff80cb350 con 0x7f7ff81553a0 2023-12-19T18:26:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.705+0000 7f7ffe79b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ff80cbcf0 con 0x7f7ff815ee30 2023-12-19T18:26:22.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.705+0000 7f7ffe79b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff80cb630 con 0x7f7ff80afeb0 2023-12-19T18:26:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.706+0000 7f7ffdf9a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff80afeb0 0x7f7ff80c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.706+0000 7f7ffdf9a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff80afeb0 0x7f7ff80c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:54998/0 (socket says 172.21.15.27:54998) 2023-12-19T18:26:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.706+0000 7f7ffdf9a640 1 -- 172.21.15.27:0/3458796026 learned_addr learned my addr 172.21.15.27:0/3458796026 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:22.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.706+0000 7f7ffcf98640 1 --2- 172.21.15.27:0/3458796026 >> v2:172.21.15.192:3300/0 conn(0x7f7ff815ee30 0x7f7ff80c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.707+0000 7f7ffdf9a640 1 -- 172.21.15.27:0/3458796026 >> v2:172.21.15.192:3300/0 conn(0x7f7ff815ee30 msgr2=0x7f7ff80c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.707+0000 7f7ffdf9a640 1 --2- 172.21.15.27:0/3458796026 >> v2:172.21.15.192:3300/0 conn(0x7f7ff815ee30 0x7f7ff80c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.707+0000 7f7ffdf9a640 1 -- 172.21.15.27:0/3458796026 >> v1:172.21.15.192:6789/0 conn(0x7f7ff81553a0 legacy=0x7f7ff815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:22.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.707+0000 7f7ffdf9a640 1 -- 172.21.15.27:0/3458796026 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff80cb910 con 0x7f7ff80afeb0 2023-12-19T18:26:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.708+0000 7f7ffdf9a640 1 --2- 172.21.15.27:0/3458796026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff80afeb0 0x7f7ff80c39d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009e50 tx=0x7f7ff002bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce695bf89488238c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.709+0000 7f7feffff640 1 -- 172.21.15.27:0/3458796026 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff002ca60 con 0x7f7ff80afeb0 2023-12-19T18:26:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.709+0000 7f7feffff640 1 -- 172.21.15.27:0/3458796026 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff002cbc0 con 0x7f7ff80afeb0 2023-12-19T18:26:22.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.709+0000 7f7feffff640 1 -- 172.21.15.27:0/3458796026 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff00343f0 con 0x7f7ff80afeb0 2023-12-19T18:26:22.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.709+0000 7f7ffe79b640 1 -- 172.21.15.27:0/3458796026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff80afeb0 msgr2=0x7f7ff80c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.709+0000 7f7ffe79b640 1 --2- 172.21.15.27:0/3458796026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff80afeb0 0x7f7ff80c39d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0009e50 tx=0x7f7ff002bec0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.710+0000 7f7ffe79b640 1 -- 172.21.15.27:0/3458796026 shutdown_connections 2023-12-19T18:26:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.710+0000 7f7ffe79b640 1 --2- 172.21.15.27:0/3458796026 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff80afeb0 0x7f7ff80c39d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.710+0000 7f7ffe79b640 1 --2- 172.21.15.27:0/3458796026 >> v2:172.21.15.192:3300/0 conn(0x7f7ff815ee30 0x7f7ff80c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.710+0000 7f7ffe79b640 1 -- 172.21.15.27:0/3458796026 >> 172.21.15.27:0/3458796026 conn(0x7f7ff8069c70 msgr2=0x7f7ff806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.710+0000 7f7ffe79b640 1 -- 172.21.15.27:0/3458796026 shutdown_connections 2023-12-19T18:26:22.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.710+0000 7f7ffe79b640 1 -- 172.21.15.27:0/3458796026 wait complete. 2023-12-19T18:26:22.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.712+0000 7f7ffe79b640 1 Processor -- start 2023-12-19T18:26:22.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.712+0000 7f7ffe79b640 1 -- start start 2023-12-19T18:26:22.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.714+0000 7f7ffe79b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff80afeb0 0x7f7ff8154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:22.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.714+0000 7f7ffe79b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff81553a0 0x7f7ff806b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:22.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.714+0000 7f7ffe79b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff815ee30 0x7f7ff806b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:22.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.714+0000 7f7ffe79b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff80cf740 con 0x7f7ff80afeb0 2023-12-19T18:26:22.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.714+0000 7f7ffe79b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff80b19d0 con 0x7f7ff815ee30 2023-12-19T18:26:22.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.714+0000 7f7ffe79b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff8154a50 con 0x7f7ff81553a0 2023-12-19T18:26:22.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.714+0000 7f7ffd799640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff80afeb0 0x7f7ff8154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:22.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.714+0000 7f7ffd799640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff80afeb0 0x7f7ff8154510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58692/0 (socket says 172.21.15.27:58692) 2023-12-19T18:26:22.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.714+0000 7f7ffd799640 1 -- 172.21.15.27:0/641132738 learned_addr learned my addr 172.21.15.27:0/641132738 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.714+0000 7f7ffcf98640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff81553a0 0x7f7ff806b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.715+0000 7f7ffdf9a640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff815ee30 0x7f7ff806b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.715+0000 7f7ffd799640 1 -- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff81553a0 msgr2=0x7f7ff806b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.715+0000 7f7ffd799640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff81553a0 0x7f7ff806b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.715+0000 7f7ffd799640 1 -- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff815ee30 msgr2=0x7f7ff806b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.715+0000 7f7ffd799640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff815ee30 0x7f7ff806b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.715+0000 7f7ffd799640 1 -- 172.21.15.27:0/641132738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff0009b30 con 0x7f7ff80afeb0 2023-12-19T18:26:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.715+0000 7f7ffcf98640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff81553a0 0x7f7ff806b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:26:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.715+0000 7f7ffd799640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff80afeb0 0x7f7ff8154510 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400c450 tx=0x7f7ff400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.716+0000 7f7fedffb640 1 -- 172.21.15.27:0/641132738 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff40052d0 con 0x7f7ff80afeb0 2023-12-19T18:26:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.716+0000 7f7fedffb640 1 -- 172.21.15.27:0/641132738 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff4005430 con 0x7f7ff80afeb0 2023-12-19T18:26:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.716+0000 7f7fedffb640 1 -- 172.21.15.27:0/641132738 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff4012870 con 0x7f7ff80afeb0 2023-12-19T18:26:22.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.716+0000 7f7ffe79b640 1 -- 172.21.15.27:0/641132738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff80c7a80 con 0x7f7ff80afeb0 2023-12-19T18:26:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.716+0000 7f7ffe79b640 1 -- 172.21.15.27:0/641132738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff80c8040 con 0x7f7ff80afeb0 2023-12-19T18:26:22.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.717+0000 7f7ffe79b640 1 -- 172.21.15.27:0/641132738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff804e840 con 0x7f7ff80afeb0 2023-12-19T18:26:22.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.718+0000 7f7fedffb640 1 -- 172.21.15.27:0/641132738 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7ff40129d0 con 0x7f7ff80afeb0 2023-12-19T18:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.719+0000 7f7fedffb640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fdc06a430 0x7f7fdc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.719+0000 7f7fedffb640 1 -- 172.21.15.27:0/641132738 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ff4088420 con 0x7f7ff80afeb0 2023-12-19T18:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.720+0000 7f7ffcf98640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fdc06a430 0x7f7fdc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:22.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.721+0000 7f7ffcf98640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fdc06a430 0x7f7fdc06c8f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80c5320 tx=0x7f7fe8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:22.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.724+0000 7f7fedffb640 1 -- 172.21.15.27:0/641132738 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7ff4053970 con 0x7f7ff80afeb0 2023-12-19T18:26:22.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.957+0000 7f7ffe79b640 1 -- 172.21.15.27:0/641132738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ff8158a50 con 0x7f7ff80afeb0 2023-12-19T18:26:22.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.958+0000 7f7fedffb640 1 -- 172.21.15.27:0/641132738 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4714 (secure 0 0 0) 0x7f7ff4056f90 con 0x7f7ff80afeb0 2023-12-19T18:26:22.958 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:22.958 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":69,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3291476389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3291476389},{"type":"v1","addr":"172.21.15.192:6836","nonce":3291476389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3526330256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3526330256},{"type":"v1","addr":"172.21.15.192:6837","nonce":3526330256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/899234003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899234003},{"type":"v1","addr":"172.21.15.27:6835","nonce":899234003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:24:08.343302+0000","modified":"2023-12-19T18:24:11.863775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8247":{"gid":8247,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6833/2519815591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2519815591},{"type":"v1","addr":"172.21.15.27:6833","nonce":2519815591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-19T18:26:22.959 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 69 2023-12-19T18:26:22.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.961+0000 7f7ffe79b640 1 -- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fdc06a430 msgr2=0x7f7fdc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:22.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.961+0000 7f7ffe79b640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fdc06a430 0x7f7fdc06c8f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7ff80c5320 tx=0x7f7fe8008040 comp rx=0 tx=0).stop 2023-12-19T18:26:22.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.961+0000 7f7ffe79b640 1 -- 172.21.15.27:0/641132738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff80afeb0 msgr2=0x7f7ff8154510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:22.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.961+0000 7f7ffe79b640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff80afeb0 0x7f7ff8154510 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400c450 tx=0x7f7ff400c910 comp rx=0 tx=0).stop 2023-12-19T18:26:22.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.961+0000 7f7ffe79b640 1 -- 172.21.15.27:0/641132738 shutdown_connections 2023-12-19T18:26:22.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.961+0000 7f7ffe79b640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fdc06a430 0x7f7fdc06c8f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.961 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.961+0000 7f7ffe79b640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff815ee30 0x7f7ff806b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.961+0000 7f7ffe79b640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7ff81553a0 0x7f7ff806b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.961+0000 7f7ffe79b640 1 --2- 172.21.15.27:0/641132738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff80afeb0 0x7f7ff8154510 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:22.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.961+0000 7f7ffe79b640 1 -- 172.21.15.27:0/641132738 >> 172.21.15.27:0/641132738 conn(0x7f7ff8069c70 msgr2=0x7f7ff8159d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:22.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.961+0000 7f7ffe79b640 1 -- 172.21.15.27:0/641132738 shutdown_connections 2023-12-19T18:26:22.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:22.961+0000 7f7ffe79b640 1 -- 172.21.15.27:0/641132738 wait complete. 2023-12-19T18:26:22.982 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:26:23.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.209+0000 7fe388a37640 1 Processor -- start 2023-12-19T18:26:23.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.209+0000 7fe388a37640 1 -- start start 2023-12-19T18:26:23.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.211+0000 7fe388a37640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe384154030 0x7fe3840cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:23.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.211+0000 7fe388a37640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe3840af630 0x7fe3840afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:23.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.211+0000 7fe388a37640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3840affd0 con 0x7fe38415d620 2023-12-19T18:26:23.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.211+0000 7fe388a37640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3840cc190 con 0x7fe3840af630 2023-12-19T18:26:23.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.211+0000 7fe388a37640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3840cb2a0 con 0x7fe384154030 2023-12-19T18:26:23.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.211+0000 7fe38259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe3840af630 0x7fe3840afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:23.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.211+0000 7fe38259c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe3840af630 0x7fe3840afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58708/0 (socket says 172.21.15.27:58708) 2023-12-19T18:26:23.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.211+0000 7fe38259c640 1 -- 172.21.15.27:0/756672775 learned_addr learned my addr 172.21.15.27:0/756672775 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:23.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.211+0000 7fe382d9d640 1 --2- 172.21.15.27:0/756672775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe384154030 0x7fe3840cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:23.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.212+0000 7fe38259c640 1 -- 172.21.15.27:0/756672775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe384154030 msgr2=0x7fe3840cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:23.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.212+0000 7fe38259c640 1 --2- 172.21.15.27:0/756672775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe384154030 0x7fe3840cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.212+0000 7fe38259c640 1 -- 172.21.15.27:0/756672775 >> v1:172.21.15.192:6789/0 conn(0x7fe38415d620 legacy=0x7fe38415da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:23.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.212+0000 7fe38259c640 1 -- 172.21.15.27:0/756672775 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3840cb580 con 0x7fe3840af630 2023-12-19T18:26:23.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.213+0000 7fe38259c640 1 --2- 172.21.15.27:0/756672775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3840af630 0x7fe3840afa90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe378009870 tx=0x7fe37802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b79e107af6336af server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:23.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.213+0000 7fe381d9b640 1 -- 172.21.15.27:0/756672775 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378031640 con 0x7fe3840af630 2023-12-19T18:26:23.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.213+0000 7fe381d9b640 1 -- 172.21.15.27:0/756672775 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3780317a0 con 0x7fe3840af630 2023-12-19T18:26:23.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.213+0000 7fe381d9b640 1 -- 172.21.15.27:0/756672775 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378031b30 con 0x7fe3840af630 2023-12-19T18:26:23.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.214+0000 7fe388a37640 1 -- 172.21.15.27:0/756672775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3840af630 msgr2=0x7fe3840afa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:23.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.214+0000 7fe388a37640 1 --2- 172.21.15.27:0/756672775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3840af630 0x7fe3840afa90 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fe378009870 tx=0x7fe37802b8f0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.214+0000 7fe388a37640 1 -- 172.21.15.27:0/756672775 shutdown_connections 2023-12-19T18:26:23.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.214+0000 7fe388a37640 1 --2- 172.21.15.27:0/756672775 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3840af630 0x7fe3840afa90 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.214+0000 7fe388a37640 1 --2- 172.21.15.27:0/756672775 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe384154030 0x7fe3840cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.214+0000 7fe388a37640 1 -- 172.21.15.27:0/756672775 >> 172.21.15.27:0/756672775 conn(0x7fe384069c70 msgr2=0x7fe38406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:23.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.214+0000 7fe388a37640 1 -- 172.21.15.27:0/756672775 shutdown_connections 2023-12-19T18:26:23.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.214+0000 7fe388a37640 1 -- 172.21.15.27:0/756672775 wait complete. 2023-12-19T18:26:23.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.216+0000 7fe388a37640 1 Processor -- start 2023-12-19T18:26:23.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.216+0000 7fe388a37640 1 -- start start 2023-12-19T18:26:23.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.217+0000 7fe388a37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3840af630 0x7fe38413f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:23.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.217+0000 7fe388a37640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe384154030 0x7fe3840b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:23.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.217+0000 7fe388a37640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe38415d620 0x7fe38406f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:23.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.217+0000 7fe388a37640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38414a640 con 0x7fe384154030 2023-12-19T18:26:23.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.217+0000 7fe388a37640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe38413f9b0 con 0x7fe3840af630 2023-12-19T18:26:23.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.217+0000 7fe388a37640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3841423e0 con 0x7fe38415d620 2023-12-19T18:26:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.218+0000 7fe38259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe384154030 0x7fe3840b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.218+0000 7fe38259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe384154030 0x7fe3840b3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58720/0 (socket says 172.21.15.27:58720) 2023-12-19T18:26:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.218+0000 7fe38259c640 1 -- 172.21.15.27:0/1924607986 learned_addr learned my addr 172.21.15.27:0/1924607986 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.218+0000 7fe382d9d640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3840af630 0x7fe38413f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.218+0000 7fe38359e640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe38415d620 0x7fe38406f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.218+0000 7fe38259c640 1 -- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe38415d620 msgr2=0x7fe38406f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.218+0000 7fe38259c640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe38415d620 0x7fe38406f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.218+0000 7fe38259c640 1 -- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3840af630 msgr2=0x7fe38413f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.218+0000 7fe38259c640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3840af630 0x7fe38413f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.218+0000 7fe38259c640 1 -- 172.21.15.27:0/1924607986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3780094d0 con 0x7fe384154030 2023-12-19T18:26:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.218+0000 7fe382d9d640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3840af630 0x7fe38413f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:26:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.219+0000 7fe38259c640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe384154030 0x7fe3840b3dd0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe378009870 tx=0x7fe3780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.219+0000 7fe36b7fe640 1 -- 172.21.15.27:0/1924607986 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe37803ac60 con 0x7fe384154030 2023-12-19T18:26:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.219+0000 7fe36b7fe640 1 -- 172.21.15.27:0/1924607986 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe37803adc0 con 0x7fe384154030 2023-12-19T18:26:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.219+0000 7fe36b7fe640 1 -- 172.21.15.27:0/1924607986 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe378044630 con 0x7fe384154030 2023-12-19T18:26:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.219+0000 7fe388a37640 1 -- 172.21.15.27:0/1924607986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe384147fb0 con 0x7fe384154030 2023-12-19T18:26:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.219+0000 7fe388a37640 1 -- 172.21.15.27:0/1924607986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe384149f80 con 0x7fe384154030 2023-12-19T18:26:23.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.221+0000 7fe388a37640 1 -- 172.21.15.27:0/1924607986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe38414a780 con 0x7fe384154030 2023-12-19T18:26:23.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.221+0000 7fe36b7fe640 1 -- 172.21.15.27:0/1924607986 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe37802f050 con 0x7fe384154030 2023-12-19T18:26:23.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.222+0000 7fe36b7fe640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe37006a430 0x7fe37006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:23.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.222+0000 7fe36b7fe640 1 -- 172.21.15.27:0/1924607986 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3780aa7a0 con 0x7fe384154030 2023-12-19T18:26:23.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.223+0000 7fe382d9d640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe37006a430 0x7fe37006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:23.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.224+0000 7fe382d9d640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe37006a430 0x7fe37006c8f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe384149e70 tx=0x7fe3740039f0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:23.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.227+0000 7fe36b7fe640 1 -- 172.21.15.27:0/1924607986 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe38414a780 con 0x7fe384154030 2023-12-19T18:26:23.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.457+0000 7fe388a37640 1 -- 172.21.15.27:0/1924607986 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe38414a780 con 0x7fe384154030 2023-12-19T18:26:23.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.458+0000 7fe36b7fe640 1 -- 172.21.15.27:0/1924607986 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 69 v69) v1 ==== 94+0+4714 (secure 0 0 0) 0x7fe38414a780 con 0x7fe384154030 2023-12-19T18:26:23.458 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:23.458 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":69,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3291476389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3291476389},{"type":"v1","addr":"172.21.15.192:6836","nonce":3291476389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3526330256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3526330256},{"type":"v1","addr":"172.21.15.192:6837","nonce":3526330256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/899234003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899234003},{"type":"v1","addr":"172.21.15.27:6835","nonce":899234003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64}],"filesystems":[{"mdsmap":{"epoch":69,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:24:08.343302+0000","modified":"2023-12-19T18:24:11.863775+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8247},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8247":{"gid":8247,"name":"a","rank":0,"incarnation":65,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6833/2519815591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2519815591},{"type":"v1","addr":"172.21.15.27:6833","nonce":2519815591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-19T18:26:23.459 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 69 2023-12-19T18:26:23.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.461+0000 7fe388a37640 1 -- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe37006a430 msgr2=0x7fe37006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:23.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.461+0000 7fe388a37640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe37006a430 0x7fe37006c8f0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fe384149e70 tx=0x7fe3740039f0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.461+0000 7fe388a37640 1 -- 172.21.15.27:0/1924607986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe384154030 msgr2=0x7fe3840b3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:23.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.461+0000 7fe388a37640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe384154030 0x7fe3840b3dd0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7fe378009870 tx=0x7fe3780046c0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.461+0000 7fe388a37640 1 -- 172.21.15.27:0/1924607986 shutdown_connections 2023-12-19T18:26:23.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.461+0000 7fe388a37640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe37006a430 0x7fe37006c8f0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.461+0000 7fe388a37640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe38415d620 0x7fe38406f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.461+0000 7fe388a37640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe384154030 0x7fe3840b3dd0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.461+0000 7fe388a37640 1 --2- 172.21.15.27:0/1924607986 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3840af630 0x7fe38413f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.461+0000 7fe388a37640 1 -- 172.21.15.27:0/1924607986 >> 172.21.15.27:0/1924607986 conn(0x7fe384069c70 msgr2=0x7fe3840c9940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:23.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.461+0000 7fe388a37640 1 -- 172.21.15.27:0/1924607986 shutdown_connections 2023-12-19T18:26:23.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.461+0000 7fe388a37640 1 -- 172.21.15.27:0/1924607986 wait complete. 2023-12-19T18:26:23.482 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:26:23.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.707+0000 7f633b43e640 1 Processor -- start 2023-12-19T18:26:23.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.708+0000 7f633b43e640 1 -- start start 2023-12-19T18:26:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.709+0000 7f633b43e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63340b4ee0 0x7f63340d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.709+0000 7f633b43e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63340b6540 con 0x7f63340d1f60 2023-12-19T18:26:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.709+0000 7f633b43e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63340cdfb0 con 0x7f633415ceb0 2023-12-19T18:26:23.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.709+0000 7f633b43e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63340d2380 con 0x7f63340b4ee0 2023-12-19T18:26:23.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.709+0000 7f6339c3b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f63340b4ee0 0x7f63340d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:23.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.709+0000 7f633ac3d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f63340d1f60 0x7f63340b69d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34882/0 (socket says 172.21.15.27:34882) 2023-12-19T18:26:23.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.709+0000 7f633ac3d640 1 -- 172.21.15.27:0/3791943483 learned_addr learned my addr 172.21.15.27:0/3791943483 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:23.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.710+0000 7f6339c3b640 1 -- 172.21.15.27:0/3791943483 >> v1:172.21.15.192:6789/0 conn(0x7f633415ceb0 legacy=0x7f63340b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:23.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.710+0000 7f6339c3b640 1 -- 172.21.15.27:0/3791943483 >> v1:172.21.15.27:6789/0 conn(0x7f63340d1f60 legacy=0x7f63340b69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:23.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.710+0000 7f6339c3b640 1 -- 172.21.15.27:0/3791943483 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340d2600 con 0x7f63340b4ee0 2023-12-19T18:26:23.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.710+0000 7f633943a640 1 -- 172.21.15.27:0/3791943483 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971282292 0 0) 0x7f63340b6540 con 0x7f63340d1f60 2023-12-19T18:26:23.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.711+0000 7f6339c3b640 1 --2- 172.21.15.27:0/3791943483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63340b4ee0 0x7f63340d1a20 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f63240097a0 tx=0x7f632402b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aad381c71c8f14f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:23.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.711+0000 7f633943a640 1 -- 172.21.15.27:0/3791943483 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324030610 con 0x7f63340b4ee0 2023-12-19T18:26:23.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.711+0000 7f633943a640 1 -- 172.21.15.27:0/3791943483 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6324030770 con 0x7f63340b4ee0 2023-12-19T18:26:23.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.712+0000 7f633943a640 1 -- 172.21.15.27:0/3791943483 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324030b00 con 0x7f63340b4ee0 2023-12-19T18:26:23.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.712+0000 7f633b43e640 1 -- 172.21.15.27:0/3791943483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63340b4ee0 msgr2=0x7f63340d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:23.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.712+0000 7f633b43e640 1 --2- 172.21.15.27:0/3791943483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63340b4ee0 0x7f63340d1a20 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f63240097a0 tx=0x7f632402b960 comp rx=0 tx=0).stop 2023-12-19T18:26:23.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.712+0000 7f633b43e640 1 -- 172.21.15.27:0/3791943483 shutdown_connections 2023-12-19T18:26:23.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.712+0000 7f633b43e640 1 --2- 172.21.15.27:0/3791943483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63340b4ee0 0x7f63340d1a20 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.712+0000 7f633b43e640 1 -- 172.21.15.27:0/3791943483 >> 172.21.15.27:0/3791943483 conn(0x7f63340cf7d0 msgr2=0x7f63340cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:23.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.712+0000 7f633b43e640 1 -- 172.21.15.27:0/3791943483 shutdown_connections 2023-12-19T18:26:23.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.712+0000 7f633b43e640 1 -- 172.21.15.27:0/3791943483 wait complete. 2023-12-19T18:26:23.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.714+0000 7f633b43e640 1 Processor -- start 2023-12-19T18:26:23.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.714+0000 7f633b43e640 1 -- start start 2023-12-19T18:26:23.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.715+0000 7f633b43e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63340b4ee0 0x7f63341549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:23.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.715+0000 7f633b43e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63340d1f60 0x7f6334157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:23.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.715+0000 7f633b43e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633415ceb0 0x7f633415b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:23.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.715+0000 7f633b43e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63341f22d0 con 0x7f63340d1f60 2023-12-19T18:26:23.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.715+0000 7f633b43e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63341f2570 con 0x7f63340b4ee0 2023-12-19T18:26:23.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.715+0000 7f633b43e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63341f2850 con 0x7f633415ceb0 2023-12-19T18:26:23.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.715+0000 7f6339c3b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63340d1f60 0x7f6334157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:23.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.716+0000 7f6339c3b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63340d1f60 0x7f6334157460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58732/0 (socket says 172.21.15.27:58732) 2023-12-19T18:26:23.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.716+0000 7f6339c3b640 1 -- 172.21.15.27:0/745748103 learned_addr learned my addr 172.21.15.27:0/745748103 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:23.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.716+0000 7f633a43c640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63340b4ee0 0x7f63341549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:23.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.716+0000 7f633ac3d640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633415ceb0 0x7f633415b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:23.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.716+0000 7f6339c3b640 1 -- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633415ceb0 msgr2=0x7f633415b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:23.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.716+0000 7f6339c3b640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633415ceb0 0x7f633415b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.716+0000 7f6339c3b640 1 -- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63340b4ee0 msgr2=0x7f63341549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:23.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.716+0000 7f6339c3b640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63340b4ee0 0x7f63341549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.716+0000 7f6339c3b640 1 -- 172.21.15.27:0/745748103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6324009400 con 0x7f63340d1f60 2023-12-19T18:26:23.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.716+0000 7f633a43c640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63340b4ee0 0x7f63341549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:23.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.716+0000 7f6339c3b640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63340d1f60 0x7f6334157460 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f63240098c0 tx=0x7f63240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:23.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.717+0000 7f632affd640 1 -- 172.21.15.27:0/745748103 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632403e070 con 0x7f63340d1f60 2023-12-19T18:26:23.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.717+0000 7f632affd640 1 -- 172.21.15.27:0/745748103 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6324039a10 con 0x7f63340d1f60 2023-12-19T18:26:23.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.717+0000 7f632affd640 1 -- 172.21.15.27:0/745748103 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324039da0 con 0x7f63340d1f60 2023-12-19T18:26:23.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.717+0000 7f633b43e640 1 -- 172.21.15.27:0/745748103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63341f2b30 con 0x7f63340d1f60 2023-12-19T18:26:23.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.717+0000 7f633b43e640 1 -- 172.21.15.27:0/745748103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6334157c90 con 0x7f63340d1f60 2023-12-19T18:26:23.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.718+0000 7f633b43e640 1 -- 172.21.15.27:0/745748103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633404e840 con 0x7f63340d1f60 2023-12-19T18:26:23.719 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.719+0000 7f632affd640 1 -- 172.21.15.27:0/745748103 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f632404b400 con 0x7f63340d1f60 2023-12-19T18:26:23.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.720+0000 7f632affd640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f631806a430 0x7f631806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:23.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.720+0000 7f632affd640 1 -- 172.21.15.27:0/745748103 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63240aae30 con 0x7f63340d1f60 2023-12-19T18:26:23.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.721+0000 7f633a43c640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f631806a430 0x7f631806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:23.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.722+0000 7f633a43c640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f631806a430 0x7f631806c8f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f633415b980 tx=0x7f6330009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:23.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.724+0000 7f632affd640 1 -- 172.21.15.27:0/745748103 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6324076420 con 0x7f63340d1f60 2023-12-19T18:26:23.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.922+0000 7f633b43e640 1 -- 172.21.15.27:0/745748103 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63341f2cf0 con 0x7f63340d1f60 2023-12-19T18:26:23.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.925+0000 7f632affd640 1 -- 172.21.15.27:0/745748103 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+16067 (secure 0 0 0) 0x7f6324034090 con 0x7f63340d1f60 2023-12-19T18:26:23.926 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:23.926 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":173,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:26:21.777719+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:24:06.280682+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"169","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-19T18:24:07.374789+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:26:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.928+0000 7f633b43e640 1 -- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f631806a430 msgr2=0x7f631806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.928+0000 7f633b43e640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f631806a430 0x7f631806c8f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f633415b980 tx=0x7f6330009040 comp rx=0 tx=0).stop 2023-12-19T18:26:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.928+0000 7f633b43e640 1 -- 172.21.15.27:0/745748103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63340d1f60 msgr2=0x7f6334157460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:23.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.928+0000 7f633b43e640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63340d1f60 0x7f6334157460 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f63240098c0 tx=0x7f63240046c0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.928+0000 7f633b43e640 1 -- 172.21.15.27:0/745748103 shutdown_connections 2023-12-19T18:26:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.928+0000 7f633b43e640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f631806a430 0x7f631806c8f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.928+0000 7f633b43e640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f633415ceb0 0x7f633415b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.928+0000 7f633b43e640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63340d1f60 0x7f6334157460 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.928+0000 7f633b43e640 1 --2- 172.21.15.27:0/745748103 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f63340b4ee0 0x7f63341549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.928+0000 7f633b43e640 1 -- 172.21.15.27:0/745748103 >> 172.21.15.27:0/745748103 conn(0x7f63340cf7d0 msgr2=0x7f633415eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.929+0000 7f633b43e640 1 -- 172.21.15.27:0/745748103 shutdown_connections 2023-12-19T18:26:23.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:23.929+0000 7f633b43e640 1 -- 172.21.15.27:0/745748103 wait complete. 2023-12-19T18:26:23.948 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:26:24.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.177+0000 7fa19dc8c640 1 Processor -- start 2023-12-19T18:26:24.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.177+0000 7fa19dc8c640 1 -- start start 2023-12-19T18:26:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.179+0000 7fa19dc8c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1980b49d0 0x7fa198160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.179+0000 7fa19dc8c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa19815cf90 0x7fa1980d15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.179+0000 7fa19dc8c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1980b5ca0 con 0x7fa1980b89b0 2023-12-19T18:26:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.179+0000 7fa19dc8c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa198161800 con 0x7fa19815cf90 2023-12-19T18:26:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.179+0000 7fa19dc8c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa198161240 con 0x7fa1980b49d0 2023-12-19T18:26:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.179+0000 7fa19d48b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa19815cf90 0x7fa1980d15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.179+0000 7fa19d48b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa19815cf90 0x7fa1980d15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58734/0 (socket says 172.21.15.27:58734) 2023-12-19T18:26:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.179+0000 7fa19d48b640 1 -- 172.21.15.27:0/1835396525 learned_addr learned my addr 172.21.15.27:0/1835396525 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.180+0000 7fa19cc8a640 1 --2- 172.21.15.27:0/1835396525 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1980b49d0 0x7fa198160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.180+0000 7fa19d48b640 1 -- 172.21.15.27:0/1835396525 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1980b49d0 msgr2=0x7fa198160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:24.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.180+0000 7fa19d48b640 1 --2- 172.21.15.27:0/1835396525 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1980b49d0 0x7fa198160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.180+0000 7fa19d48b640 1 -- 172.21.15.27:0/1835396525 >> v1:172.21.15.192:6789/0 conn(0x7fa1980b89b0 legacy=0x7fa19815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:24.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.180+0000 7fa19d48b640 1 -- 172.21.15.27:0/1835396525 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa198161520 con 0x7fa19815cf90 2023-12-19T18:26:24.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.181+0000 7fa19d48b640 1 --2- 172.21.15.27:0/1835396525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa19815cf90 0x7fa1980d15a0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa190009e50 tx=0x7fa19002bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785cafdcb8cab097 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:24.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.182+0000 7fa1977fe640 1 -- 172.21.15.27:0/1835396525 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa19002cb90 con 0x7fa19815cf90 2023-12-19T18:26:24.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.182+0000 7fa1977fe640 1 -- 172.21.15.27:0/1835396525 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa19002ccf0 con 0x7fa19815cf90 2023-12-19T18:26:24.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.182+0000 7fa1977fe640 1 -- 172.21.15.27:0/1835396525 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa190034610 con 0x7fa19815cf90 2023-12-19T18:26:24.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.182+0000 7fa19dc8c640 1 -- 172.21.15.27:0/1835396525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa19815cf90 msgr2=0x7fa1980d15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:24.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.182+0000 7fa19dc8c640 1 --2- 172.21.15.27:0/1835396525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa19815cf90 0x7fa1980d15a0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fa190009e50 tx=0x7fa19002bff0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.182+0000 7fa19dc8c640 1 -- 172.21.15.27:0/1835396525 shutdown_connections 2023-12-19T18:26:24.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.182+0000 7fa19dc8c640 1 --2- 172.21.15.27:0/1835396525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa19815cf90 0x7fa1980d15a0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.182+0000 7fa19dc8c640 1 --2- 172.21.15.27:0/1835396525 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa1980b49d0 0x7fa198160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.182+0000 7fa19dc8c640 1 -- 172.21.15.27:0/1835396525 >> 172.21.15.27:0/1835396525 conn(0x7fa1980cf760 msgr2=0x7fa1980cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:24.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.182+0000 7fa19dc8c640 1 -- 172.21.15.27:0/1835396525 shutdown_connections 2023-12-19T18:26:24.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.182+0000 7fa19dc8c640 1 -- 172.21.15.27:0/1835396525 wait complete. 2023-12-19T18:26:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.184+0000 7fa19dc8c640 1 Processor -- start 2023-12-19T18:26:24.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.184+0000 7fa19dc8c640 1 -- start start 2023-12-19T18:26:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.185+0000 7fa19dc8c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1980b49d0 0x7fa198157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:24.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.185+0000 7fa19dc8c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1980b89b0 0x7fa198159470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.185+0000 7fa19dc8c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa19815cf90 0x7fa198142d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.185+0000 7fa19dc8c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa198157580 con 0x7fa1980b89b0 2023-12-19T18:26:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.185+0000 7fa19dc8c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1981599b0 con 0x7fa1980b49d0 2023-12-19T18:26:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.185+0000 7fa19dc8c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1980c65b0 con 0x7fa19815cf90 2023-12-19T18:26:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.186+0000 7fa197fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1980b89b0 0x7fa198159470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.186+0000 7fa197fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1980b89b0 0x7fa198159470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58750/0 (socket says 172.21.15.27:58750) 2023-12-19T18:26:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.186+0000 7fa197fff640 1 -- 172.21.15.27:0/2827400206 learned_addr learned my addr 172.21.15.27:0/2827400206 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.186+0000 7fa197fff640 1 -- 172.21.15.27:0/2827400206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa19815cf90 msgr2=0x7fa198142d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.186+0000 7fa197fff640 1 --2- 172.21.15.27:0/2827400206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa19815cf90 0x7fa198142d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.186+0000 7fa19cc8a640 1 --2- 172.21.15.27:0/2827400206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1980b49d0 0x7fa198157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.186+0000 7fa197fff640 1 -- 172.21.15.27:0/2827400206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1980b49d0 msgr2=0x7fa198157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.186+0000 7fa197fff640 1 --2- 172.21.15.27:0/2827400206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1980b49d0 0x7fa198157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.186+0000 7fa197fff640 1 -- 172.21.15.27:0/2827400206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa190009b30 con 0x7fa1980b89b0 2023-12-19T18:26:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.186+0000 7fa197fff640 1 --2- 172.21.15.27:0/2827400206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1980b89b0 0x7fa198159470 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c400 tx=0x7fa18800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.187+0000 7fa1957fa640 1 -- 172.21.15.27:0/2827400206 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1880052d0 con 0x7fa1980b89b0 2023-12-19T18:26:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.187+0000 7fa1957fa640 1 -- 172.21.15.27:0/2827400206 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa188005430 con 0x7fa1980b89b0 2023-12-19T18:26:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.187+0000 7fa1957fa640 1 -- 172.21.15.27:0/2827400206 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa188003830 con 0x7fa1980b89b0 2023-12-19T18:26:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.187+0000 7fa19dc8c640 1 -- 172.21.15.27:0/2827400206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1980c7ba0 con 0x7fa1980b89b0 2023-12-19T18:26:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.187+0000 7fa19dc8c640 1 -- 172.21.15.27:0/2827400206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1980c8160 con 0x7fa1980b89b0 2023-12-19T18:26:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.188+0000 7fa19dc8c640 1 -- 172.21.15.27:0/2827400206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa19804e840 con 0x7fa1980b89b0 2023-12-19T18:26:24.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.189+0000 7fa1957fa640 1 -- 172.21.15.27:0/2827400206 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa188003990 con 0x7fa1980b89b0 2023-12-19T18:26:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.190+0000 7fa1957fa640 1 --2- 172.21.15.27:0/2827400206 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa18406a430 0x7fa18406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.190+0000 7fa1957fa640 1 -- 172.21.15.27:0/2827400206 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa188087ef0 con 0x7fa1980b89b0 2023-12-19T18:26:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.191+0000 7fa19cc8a640 1 --2- 172.21.15.27:0/2827400206 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa18406a430 0x7fa18406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:24.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.192+0000 7fa19cc8a640 1 --2- 172.21.15.27:0/2827400206 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa18406a430 0x7fa18406c8f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa198153e00 tx=0x7fa18c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:24.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.194+0000 7fa1957fa640 1 -- 172.21.15.27:0/2827400206 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa1880534d0 con 0x7fa1980b89b0 2023-12-19T18:26:24.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.433+0000 7fa19dc8c640 1 -- 172.21.15.27:0/2827400206 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa1980d08b0 con 0x7fa1980b89b0 2023-12-19T18:26:24.434 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:26:24.435+0000 7f8f3865c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:26:24.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.452+0000 7fa1957fa640 1 -- 172.21.15.27:0/2827400206 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v70) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa188056af0 con 0x7fa1980b89b0 2023-12-19T18:26:24.452 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:26:24.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.455+0000 7fa19dc8c640 1 -- 172.21.15.27:0/2827400206 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa18406a430 msgr2=0x7fa18406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:24.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.455+0000 7fa19dc8c640 1 --2- 172.21.15.27:0/2827400206 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa18406a430 0x7fa18406c8f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fa198153e00 tx=0x7fa18c008040 comp rx=0 tx=0).stop 2023-12-19T18:26:24.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.455+0000 7fa19dc8c640 1 -- 172.21.15.27:0/2827400206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1980b89b0 msgr2=0x7fa198159470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:24.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.455+0000 7fa19dc8c640 1 --2- 172.21.15.27:0/2827400206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1980b89b0 0x7fa198159470 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c400 tx=0x7fa18800c8c0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.455+0000 7fa19dc8c640 1 -- 172.21.15.27:0/2827400206 shutdown_connections 2023-12-19T18:26:24.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.455+0000 7fa19dc8c640 1 --2- 172.21.15.27:0/2827400206 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa18406a430 0x7fa18406c8f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.455+0000 7fa19dc8c640 1 --2- 172.21.15.27:0/2827400206 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa19815cf90 0x7fa198142d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.455+0000 7fa19dc8c640 1 --2- 172.21.15.27:0/2827400206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1980b89b0 0x7fa198159470 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.455+0000 7fa19dc8c640 1 --2- 172.21.15.27:0/2827400206 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa1980b49d0 0x7fa198157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.455+0000 7fa19dc8c640 1 -- 172.21.15.27:0/2827400206 >> 172.21.15.27:0/2827400206 conn(0x7fa1980cf760 msgr2=0x7fa1981561f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:24.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.455+0000 7fa19dc8c640 1 -- 172.21.15.27:0/2827400206 shutdown_connections 2023-12-19T18:26:24.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.455+0000 7fa19dc8c640 1 -- 172.21.15.27:0/2827400206 wait complete. 2023-12-19T18:26:24.463 INFO:tasks.ceph.mds.a.smithi027.stderr: -9463> 2023-12-19T18:24:12.798+0000 7f36cbd41640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:26:24.476 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:26:24.521 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:26:24.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.676+0000 7f6c07fff640 1 Processor -- start 2023-12-19T18:26:24.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.676+0000 7f6c07fff640 1 -- start start 2023-12-19T18:26:24.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.678+0000 7f6c07fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6c08158790 0x7f6c080ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:24.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.678+0000 7f6c07fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6c0815b9e0 0x7f6c080c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:24.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.678+0000 7f6c07fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c0815c030 con 0x7f6c080adc50 2023-12-19T18:26:24.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.678+0000 7f6c07fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c080c99d0 con 0x7f6c0815b9e0 2023-12-19T18:26:24.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.678+0000 7f6c07fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c080ae100 con 0x7f6c08158790 2023-12-19T18:26:24.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.678+0000 7f6c077fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6c0815b9e0 0x7f6c080c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:24.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.678+0000 7f6c077fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6c0815b9e0 0x7f6c080c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58776/0 (socket says 172.21.15.27:58776) 2023-12-19T18:26:24.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.678+0000 7f6c077fe640 1 -- 172.21.15.27:0/2388436628 learned_addr learned my addr 172.21.15.27:0/2388436628 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:24.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.679+0000 7f6c06ffd640 1 --2- 172.21.15.27:0/2388436628 >> v2:172.21.15.192:3300/0 conn(0x7f6c08158790 0x7f6c080ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:24.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.679+0000 7f6c077fe640 1 -- 172.21.15.27:0/2388436628 >> v2:172.21.15.192:3300/0 conn(0x7f6c08158790 msgr2=0x7f6c080ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:24.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.679+0000 7f6c077fe640 1 --2- 172.21.15.27:0/2388436628 >> v2:172.21.15.192:3300/0 conn(0x7f6c08158790 0x7f6c080ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.679+0000 7f6c077fe640 1 -- 172.21.15.27:0/2388436628 >> v1:172.21.15.192:6789/0 conn(0x7f6c080adc50 legacy=0x7f6c0815b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:24.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.679+0000 7f6c077fe640 1 -- 172.21.15.27:0/2388436628 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c080c9030 con 0x7f6c0815b9e0 2023-12-19T18:26:24.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.680+0000 7f6c077fe640 1 --2- 172.21.15.27:0/2388436628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0815b9e0 0x7f6c080c3820 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009e50 tx=0x7f6c0002bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ec729282fa2b4f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:24.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.680+0000 7f6c067fc640 1 -- 172.21.15.27:0/2388436628 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0002ca80 con 0x7f6c0815b9e0 2023-12-19T18:26:24.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.680+0000 7f6c067fc640 1 -- 172.21.15.27:0/2388436628 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0002cbe0 con 0x7f6c0815b9e0 2023-12-19T18:26:24.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.680+0000 7f6c067fc640 1 -- 172.21.15.27:0/2388436628 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c000343f0 con 0x7f6c0815b9e0 2023-12-19T18:26:24.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.681+0000 7f6c07fff640 1 -- 172.21.15.27:0/2388436628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0815b9e0 msgr2=0x7f6c080c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:24.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.681+0000 7f6c07fff640 1 --2- 172.21.15.27:0/2388436628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0815b9e0 0x7f6c080c3820 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f6c00009e50 tx=0x7f6c0002bee0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.681+0000 7f6c07fff640 1 -- 172.21.15.27:0/2388436628 shutdown_connections 2023-12-19T18:26:24.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.681+0000 7f6c07fff640 1 --2- 172.21.15.27:0/2388436628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0815b9e0 0x7f6c080c3820 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.681+0000 7f6c07fff640 1 --2- 172.21.15.27:0/2388436628 >> v2:172.21.15.192:3300/0 conn(0x7f6c08158790 0x7f6c080ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.681+0000 7f6c07fff640 1 -- 172.21.15.27:0/2388436628 >> 172.21.15.27:0/2388436628 conn(0x7f6c080b05b0 msgr2=0x7f6c080b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:24.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.681+0000 7f6c07fff640 1 -- 172.21.15.27:0/2388436628 shutdown_connections 2023-12-19T18:26:24.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.681+0000 7f6c07fff640 1 -- 172.21.15.27:0/2388436628 wait complete. 2023-12-19T18:26:24.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.683+0000 7f6c07fff640 1 Processor -- start 2023-12-19T18:26:24.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.683+0000 7f6c07fff640 1 -- start start 2023-12-19T18:26:24.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.684+0000 7f6c07fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c080adc50 0x7f6c080b2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:24.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.684+0000 7f6c07fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c08158790 0x7f6c0814c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:24.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.684+0000 7f6c07fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0815b9e0 0x7f6c0806c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:24.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.684+0000 7f6c07fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c080b3180 con 0x7f6c0815b9e0 2023-12-19T18:26:24.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.684+0000 7f6c07fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c081f0410 con 0x7f6c080adc50 2023-12-19T18:26:24.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.684+0000 7f6c07fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c0814ca20 con 0x7f6c08158790 2023-12-19T18:26:24.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.685+0000 7f6c077fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0815b9e0 0x7f6c0806c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:24.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.685+0000 7f6c077fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0815b9e0 0x7f6c0806c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58784/0 (socket says 172.21.15.27:58784) 2023-12-19T18:26:24.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.685+0000 7f6c077fe640 1 -- 172.21.15.27:0/3287153602 learned_addr learned my addr 172.21.15.27:0/3287153602 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:24.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.685+0000 7f6bfffff640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c08158790 0x7f6c0814c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.685+0000 7f6c06ffd640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c080adc50 0x7f6c080b2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.685+0000 7f6c077fe640 1 -- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c08158790 msgr2=0x7f6c0814c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.685+0000 7f6c077fe640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c08158790 0x7f6c0814c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.685+0000 7f6c077fe640 1 -- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c080adc50 msgr2=0x7f6c080b2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.685+0000 7f6c077fe640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c080adc50 0x7f6c080b2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.685+0000 7f6c077fe640 1 -- 172.21.15.27:0/3287153602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c00009b30 con 0x7f6c0815b9e0 2023-12-19T18:26:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.685+0000 7f6c06ffd640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c080adc50 0x7f6c080b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.685+0000 7f6c077fe640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0815b9e0 0x7f6c0806c2a0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f6c00036040 tx=0x7f6c00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.686+0000 7f6c04ff9640 1 -- 172.21.15.27:0/3287153602 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c00009510 con 0x7f6c0815b9e0 2023-12-19T18:26:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.686+0000 7f6c04ff9640 1 -- 172.21.15.27:0/3287153602 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c0003e450 con 0x7f6c0815b9e0 2023-12-19T18:26:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.686+0000 7f6c07fff640 1 -- 172.21.15.27:0/3287153602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c080b33b0 con 0x7f6c0815b9e0 2023-12-19T18:26:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.686+0000 7f6c04ff9640 1 -- 172.21.15.27:0/3287153602 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c0003e7e0 con 0x7f6c0815b9e0 2023-12-19T18:26:24.687 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.686+0000 7f6c07fff640 1 -- 172.21.15.27:0/3287153602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c080b3970 con 0x7f6c0815b9e0 2023-12-19T18:26:24.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.687+0000 7f6c07fff640 1 -- 172.21.15.27:0/3287153602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c0804e840 con 0x7f6c0815b9e0 2023-12-19T18:26:24.688 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.688+0000 7f6c04ff9640 1 -- 172.21.15.27:0/3287153602 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6c0002cd50 con 0x7f6c0815b9e0 2023-12-19T18:26:24.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.689+0000 7f6c04ff9640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6be806a430 0x7f6be806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:24.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.689+0000 7f6c04ff9640 1 -- 172.21.15.27:0/3287153602 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f6c000ab470 con 0x7f6c0815b9e0 2023-12-19T18:26:24.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.690+0000 7f6c06ffd640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6be806a430 0x7f6be806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.691+0000 7f6c06ffd640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6be806a430 0x7f6be806c8f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8006fd0 tx=0x7f6bf8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:24.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.693+0000 7f6c04ff9640 1 -- 172.21.15.27:0/3287153602 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6c000769f0 con 0x7f6c0815b9e0 2023-12-19T18:26:24.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:24.927+0000 7f6c07fff640 1 -- 172.21.15.27:0/3287153602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6c080c4e20 con 0x7f6c0815b9e0 2023-12-19T18:26:25.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.457+0000 7f6c04ff9640 1 -- 172.21.15.27:0/3287153602 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v71) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6c0007a010 con 0x7f6c0815b9e0 2023-12-19T18:26:25.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.460+0000 7f6c07fff640 1 -- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6be806a430 msgr2=0x7f6be806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:25.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.460+0000 7f6c07fff640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6be806a430 0x7f6be806c8f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f6bf8006fd0 tx=0x7f6bf8008040 comp rx=0 tx=0).stop 2023-12-19T18:26:25.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.460+0000 7f6c07fff640 1 -- 172.21.15.27:0/3287153602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0815b9e0 msgr2=0x7f6c0806c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:25.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.460+0000 7f6c07fff640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0815b9e0 0x7f6c0806c2a0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f6c00036040 tx=0x7f6c00004a40 comp rx=0 tx=0).stop 2023-12-19T18:26:25.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.461+0000 7f6c07fff640 1 -- 172.21.15.27:0/3287153602 shutdown_connections 2023-12-19T18:26:25.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.461+0000 7f6c07fff640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6be806a430 0x7f6be806c8f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:25.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.461+0000 7f6c07fff640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c0815b9e0 0x7f6c0806c2a0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:25.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.461+0000 7f6c07fff640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6c08158790 0x7f6c0814c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:25.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.461+0000 7f6c07fff640 1 --2- 172.21.15.27:0/3287153602 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6c080adc50 0x7f6c080b2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:25.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.461+0000 7f6c07fff640 1 -- 172.21.15.27:0/3287153602 >> 172.21.15.27:0/3287153602 conn(0x7f6c080b05b0 msgr2=0x7f6c0815d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:25.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.461+0000 7f6c07fff640 1 -- 172.21.15.27:0/3287153602 shutdown_connections 2023-12-19T18:26:25.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.461+0000 7f6c07fff640 1 -- 172.21.15.27:0/3287153602 wait complete. 2023-12-19T18:26:25.481 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:26:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.763+0000 7f14e619b640 1 Processor -- start 2023-12-19T18:26:25.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.763+0000 7f14e619b640 1 -- start start 2023-12-19T18:26:25.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.765+0000 7f14e619b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f14e0158790 0x7f14e00ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:25.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.765+0000 7f14e619b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14e015b9e0 0x7f14e00c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:25.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.765+0000 7f14e619b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14e015c030 con 0x7f14e00adc50 2023-12-19T18:26:25.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.765+0000 7f14e619b640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e00c99d0 con 0x7f14e015b9e0 2023-12-19T18:26:25.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.765+0000 7f14e619b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14e00ae100 con 0x7f14e0158790 2023-12-19T18:26:25.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.765+0000 7f14e4998640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f14e00adc50 0x7f14e015b2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34902/0 (socket says 172.21.15.27:34902) 2023-12-19T18:26:25.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.765+0000 7f14e4998640 1 -- 172.21.15.27:0/2078950186 learned_addr learned my addr 172.21.15.27:0/2078950186 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:25.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.765+0000 7f14e599a640 1 --2- 172.21.15.27:0/2078950186 >> v2:172.21.15.27:3300/0 conn(0x7f14e015b9e0 0x7f14e00c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.766+0000 7f14e5199640 1 --2- 172.21.15.27:0/2078950186 >> v2:172.21.15.192:3300/0 conn(0x7f14e0158790 0x7f14e00ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.766+0000 7f14d7fff640 1 -- 172.21.15.27:0/2078950186 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3149354418 0 0) 0x7f14e015c030 con 0x7f14e00adc50 2023-12-19T18:26:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.766+0000 7f14e599a640 1 -- 172.21.15.27:0/2078950186 >> v2:172.21.15.192:3300/0 conn(0x7f14e0158790 msgr2=0x7f14e00ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.766+0000 7f14e599a640 1 --2- 172.21.15.27:0/2078950186 >> v2:172.21.15.192:3300/0 conn(0x7f14e0158790 0x7f14e00ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:25.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.766+0000 7f14e599a640 1 -- 172.21.15.27:0/2078950186 >> v1:172.21.15.27:6789/0 conn(0x7f14e00adc50 legacy=0x7f14e015b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:25.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.766+0000 7f14e599a640 1 -- 172.21.15.27:0/2078950186 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14e00c9030 con 0x7f14e015b9e0 2023-12-19T18:26:25.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.767+0000 7f14e599a640 1 --2- 172.21.15.27:0/2078950186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e015b9e0 0x7f14e00c3820 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009d90 tx=0x7f14d802beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca04ae34caaffd server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:25.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.767+0000 7f14d7fff640 1 -- 172.21.15.27:0/2078950186 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d802cc60 con 0x7f14e015b9e0 2023-12-19T18:26:25.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.767+0000 7f14d7fff640 1 -- 172.21.15.27:0/2078950186 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d802cdc0 con 0x7f14e015b9e0 2023-12-19T18:26:25.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.767+0000 7f14d7fff640 1 -- 172.21.15.27:0/2078950186 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d8034660 con 0x7f14e015b9e0 2023-12-19T18:26:25.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.768+0000 7f14e619b640 1 -- 172.21.15.27:0/2078950186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e015b9e0 msgr2=0x7f14e00c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:25.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.768+0000 7f14e619b640 1 --2- 172.21.15.27:0/2078950186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e015b9e0 0x7f14e00c3820 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f14d8009d90 tx=0x7f14d802beb0 comp rx=0 tx=0).stop 2023-12-19T18:26:25.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.768+0000 7f14e619b640 1 -- 172.21.15.27:0/2078950186 shutdown_connections 2023-12-19T18:26:25.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.768+0000 7f14e619b640 1 --2- 172.21.15.27:0/2078950186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e015b9e0 0x7f14e00c3820 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:25.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.768+0000 7f14e619b640 1 --2- 172.21.15.27:0/2078950186 >> v2:172.21.15.192:3300/0 conn(0x7f14e0158790 0x7f14e00ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:25.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.768+0000 7f14e619b640 1 -- 172.21.15.27:0/2078950186 >> 172.21.15.27:0/2078950186 conn(0x7f14e00b05b0 msgr2=0x7f14e00b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:25.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.768+0000 7f14e619b640 1 -- 172.21.15.27:0/2078950186 shutdown_connections 2023-12-19T18:26:25.769 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.768+0000 7f14e619b640 1 -- 172.21.15.27:0/2078950186 wait complete. 2023-12-19T18:26:25.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.770+0000 7f14e619b640 1 Processor -- start 2023-12-19T18:26:25.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.770+0000 7f14e619b640 1 -- start start 2023-12-19T18:26:25.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.771+0000 7f14e619b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14e00adc50 0x7f14e006cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:25.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.771+0000 7f14e619b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e0158790 0x7f14e015a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:25.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.771+0000 7f14e619b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14e015b9e0 0x7f14e015a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:25.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.771+0000 7f14e619b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e006c3a0 con 0x7f14e0158790 2023-12-19T18:26:25.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.771+0000 7f14e619b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14e00b1f00 con 0x7f14e015b9e0 2023-12-19T18:26:25.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.771+0000 7f14e619b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14e006c0c0 con 0x7f14e00adc50 2023-12-19T18:26:25.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.771+0000 7f14e4998640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e0158790 0x7f14e015a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:25.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.771+0000 7f14e4998640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e0158790 0x7f14e015a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58804/0 (socket says 172.21.15.27:58804) 2023-12-19T18:26:25.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.771+0000 7f14e4998640 1 -- 172.21.15.27:0/3358451911 learned_addr learned my addr 172.21.15.27:0/3358451911 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:25.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.772+0000 7f14e5199640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14e00adc50 0x7f14e006cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:25.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.772+0000 7f14e4998640 1 -- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14e00adc50 msgr2=0x7f14e006cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:25.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.772+0000 7f14e4998640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14e00adc50 0x7f14e006cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:25.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.772+0000 7f14e4998640 1 -- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14e015b9e0 msgr2=0x7f14e015a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:25.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.772+0000 7f14e599a640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14e015b9e0 0x7f14e015a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:25.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.772+0000 7f14e4998640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14e015b9e0 0x7f14e015a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:25.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.772+0000 7f14e4998640 1 -- 172.21.15.27:0/3358451911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14d8009a70 con 0x7f14e0158790 2023-12-19T18:26:25.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.772+0000 7f14e4998640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e0158790 0x7f14e015a0d0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f14d000c4d0 tx=0x7f14d000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:25.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.772+0000 7f14d5ffb640 1 -- 172.21.15.27:0/3358451911 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d00074e0 con 0x7f14e0158790 2023-12-19T18:26:25.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.772+0000 7f14d5ffb640 1 -- 172.21.15.27:0/3358451911 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14d0007660 con 0x7f14e0158790 2023-12-19T18:26:25.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.772+0000 7f14e619b640 1 -- 172.21.15.27:0/3358451911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14e00b1840 con 0x7f14e0158790 2023-12-19T18:26:25.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.773+0000 7f14d5ffb640 1 -- 172.21.15.27:0/3358451911 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14d0004a20 con 0x7f14e0158790 2023-12-19T18:26:25.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.773+0000 7f14e619b640 1 -- 172.21.15.27:0/3358451911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14e006d250 con 0x7f14e0158790 2023-12-19T18:26:25.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.774+0000 7f14e619b640 1 -- 172.21.15.27:0/3358451911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14e00b16a0 con 0x7f14e0158790 2023-12-19T18:26:25.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.775+0000 7f14d5ffb640 1 -- 172.21.15.27:0/3358451911 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f14d0004b80 con 0x7f14e0158790 2023-12-19T18:26:25.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.776+0000 7f14d5ffb640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f14c406a460 0x7f14c406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:25.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.776+0000 7f14d5ffb640 1 -- 172.21.15.27:0/3358451911 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5795+0+0 (secure 0 0 0) 0x7f14d0087e30 con 0x7f14e0158790 2023-12-19T18:26:25.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.776+0000 7f14e5199640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f14c406a460 0x7f14c406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:25.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.778+0000 7f14e5199640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f14c406a460 0x7f14c406c920 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f14dc006fd0 tx=0x7f14dc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:25.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.780+0000 7f14d5ffb640 1 -- 172.21.15.27:0/3358451911 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f14e00b16a0 con 0x7f14e0158790 2023-12-19T18:26:25.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:25.976+0000 7f14e619b640 1 -- 172.21.15.27:0/3358451911 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f14e00b2040 con 0x7f14e0158790 2023-12-19T18:26:26.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:26.875+0000 7f14d5ffb640 1 -- 172.21.15.27:0/3358451911 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v175) v1 ==== 181+0+0 (secure 0 0 0) 0x7f14e00b2040 con 0x7f14e0158790 2023-12-19T18:26:26.875 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:26:26.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:26.879+0000 7f14e619b640 1 -- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f14c406a460 msgr2=0x7f14c406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:26.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:26.879+0000 7f14e619b640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f14c406a460 0x7f14c406c920 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f14dc006fd0 tx=0x7f14dc008040 comp rx=0 tx=0).stop 2023-12-19T18:26:26.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:26.879+0000 7f14e619b640 1 -- 172.21.15.27:0/3358451911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e0158790 msgr2=0x7f14e015a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:26.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:26.879+0000 7f14e619b640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e0158790 0x7f14e015a0d0 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f14d000c4d0 tx=0x7f14d000c990 comp rx=0 tx=0).stop 2023-12-19T18:26:26.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:26.879+0000 7f14e619b640 1 -- 172.21.15.27:0/3358451911 shutdown_connections 2023-12-19T18:26:26.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:26.879+0000 7f14e619b640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f14c406a460 0x7f14c406c920 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:26.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:26.879+0000 7f14e619b640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14e015b9e0 0x7f14e015a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:26.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:26.879+0000 7f14e619b640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14e0158790 0x7f14e015a0d0 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:26.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:26.879+0000 7f14e619b640 1 --2- 172.21.15.27:0/3358451911 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14e00adc50 0x7f14e006cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:26.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:26.879+0000 7f14e619b640 1 -- 172.21.15.27:0/3358451911 >> 172.21.15.27:0/3358451911 conn(0x7f14e00b05b0 msgr2=0x7f14e0070be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:26.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:26.880+0000 7f14e619b640 1 -- 172.21.15.27:0/3358451911 shutdown_connections 2023-12-19T18:26:26.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:26.880+0000 7f14e619b640 1 -- 172.21.15.27:0/3358451911 wait complete. 2023-12-19T18:26:26.905 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:26:27.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.142+0000 7f7bfce37640 1 Processor -- start 2023-12-19T18:26:27.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.142+0000 7f7bfce37640 1 -- start start 2023-12-19T18:26:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.144+0000 7f7bfce37640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7bf815dfe0 0x7f7bf80cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.144+0000 7f7bfce37640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf80cb670 0x7f7bf80c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.144+0000 7f7bfce37640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf80cbaa0 con 0x7f7bf815dc00 2023-12-19T18:26:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.144+0000 7f7bfce37640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf8160920 con 0x7f7bf815dfe0 2023-12-19T18:26:27.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.144+0000 7f7bfce37640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf80cd220 con 0x7f7bf80cb670 2023-12-19T18:26:27.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.145+0000 7f7beffff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7bf815dfe0 0x7f7bf80cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:27.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.145+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf80cb670 0x7f7bf80c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:27.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.145+0000 7f7bf7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf80cb670 0x7f7bf80c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:55096/0 (socket says 172.21.15.27:55096) 2023-12-19T18:26:27.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.145+0000 7f7bf7fff640 1 -- 172.21.15.27:0/1218864925 learned_addr learned my addr 172.21.15.27:0/1218864925 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:27.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.145+0000 7f7bf77fe640 1 --1- 172.21.15.27:0/1218864925 >> v1:172.21.15.192:6789/0 conn(0x7f7bf815dc00 0x7f7bf80cc6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:39062/0 (socket says 172.21.15.27:39062) 2023-12-19T18:26:27.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.146+0000 7f7bf7fff640 1 -- 172.21.15.27:0/1218864925 >> v2:172.21.15.192:3300/0 conn(0x7f7bf815dfe0 msgr2=0x7f7bf80cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:27.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.146+0000 7f7bf7fff640 1 --2- 172.21.15.27:0/1218864925 >> v2:172.21.15.192:3300/0 conn(0x7f7bf815dfe0 0x7f7bf80cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:27.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.146+0000 7f7bf7fff640 1 -- 172.21.15.27:0/1218864925 >> v1:172.21.15.192:6789/0 conn(0x7f7bf815dc00 legacy=0x7f7bf80cc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:27.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.146+0000 7f7bf7fff640 1 -- 172.21.15.27:0/1218864925 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf8160be0 con 0x7f7bf80cb670 2023-12-19T18:26:27.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.146+0000 7f7beffff640 1 --2- 172.21.15.27:0/1218864925 >> v2:172.21.15.192:3300/0 conn(0x7f7bf815dfe0 0x7f7bf80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:26:27.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.147+0000 7f7bf7fff640 1 --2- 172.21.15.27:0/1218864925 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf80cb670 0x7f7bf80c8f90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009e50 tx=0x7f7bf002be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8c8ecf53b58ea33 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:27.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.148+0000 7f7bf6ffd640 1 -- 172.21.15.27:0/1218864925 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf002c9d0 con 0x7f7bf80cb670 2023-12-19T18:26:27.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.148+0000 7f7bf6ffd640 1 -- 172.21.15.27:0/1218864925 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bf002cb30 con 0x7f7bf80cb670 2023-12-19T18:26:27.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.148+0000 7f7bf6ffd640 1 -- 172.21.15.27:0/1218864925 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bf00343f0 con 0x7f7bf80cb670 2023-12-19T18:26:27.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.148+0000 7f7bfce37640 1 -- 172.21.15.27:0/1218864925 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf80cb670 msgr2=0x7f7bf80c8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:27.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.148+0000 7f7bfce37640 1 --2- 172.21.15.27:0/1218864925 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf80cb670 0x7f7bf80c8f90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0009e50 tx=0x7f7bf002be30 comp rx=0 tx=0).stop 2023-12-19T18:26:27.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.149+0000 7f7bfce37640 1 -- 172.21.15.27:0/1218864925 shutdown_connections 2023-12-19T18:26:27.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.149+0000 7f7bfce37640 1 --2- 172.21.15.27:0/1218864925 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf80cb670 0x7f7bf80c8f90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:27.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.149+0000 7f7bfce37640 1 --2- 172.21.15.27:0/1218864925 >> v2:172.21.15.192:3300/0 conn(0x7f7bf815dfe0 0x7f7bf80cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:27.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.149+0000 7f7bfce37640 1 -- 172.21.15.27:0/1218864925 >> 172.21.15.27:0/1218864925 conn(0x7f7bf80b05b0 msgr2=0x7f7bf80b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:27.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.149+0000 7f7bfce37640 1 -- 172.21.15.27:0/1218864925 shutdown_connections 2023-12-19T18:26:27.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.149+0000 7f7bfce37640 1 -- 172.21.15.27:0/1218864925 wait complete. 2023-12-19T18:26:27.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.151+0000 7f7bfce37640 1 Processor -- start 2023-12-19T18:26:27.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.151+0000 7f7bfce37640 1 -- start start 2023-12-19T18:26:27.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.152+0000 7f7bfce37640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf80cb670 0x7f7bf80c3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:27.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.152+0000 7f7bfce37640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf815dc00 0x7f7bf81f25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:27.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.152+0000 7f7bfce37640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf815dfe0 0x7f7bf80c4650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.152+0000 7f7bfce37640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf80c4b90 con 0x7f7bf815dc00 2023-12-19T18:26:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.152+0000 7f7bfce37640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81f2c60 con 0x7f7bf80cb670 2023-12-19T18:26:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.152+0000 7f7bfce37640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf81f2f40 con 0x7f7bf815dfe0 2023-12-19T18:26:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.152+0000 7f7beffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf815dc00 0x7f7bf81f25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.152+0000 7f7beffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf815dc00 0x7f7bf81f25e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58816/0 (socket says 172.21.15.27:58816) 2023-12-19T18:26:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.152+0000 7f7beffff640 1 -- 172.21.15.27:0/1168189933 learned_addr learned my addr 172.21.15.27:0/1168189933 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.152+0000 7f7bf7fff640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf815dfe0 0x7f7bf80c4650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.152+0000 7f7bf77fe640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf80cb670 0x7f7bf80c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.153+0000 7f7beffff640 1 -- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf815dfe0 msgr2=0x7f7bf80c4650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.153+0000 7f7beffff640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf815dfe0 0x7f7bf80c4650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.153+0000 7f7beffff640 1 -- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf80cb670 msgr2=0x7f7bf80c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.153+0000 7f7beffff640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf80cb670 0x7f7bf80c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.153+0000 7f7beffff640 1 -- 172.21.15.27:0/1168189933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be0009400 con 0x7f7bf815dc00 2023-12-19T18:26:27.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.153+0000 7f7beffff640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf815dc00 0x7f7bf81f25e0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f7be000ebe0 tx=0x7f7be000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:27.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.153+0000 7f7bf4ff9640 1 -- 172.21.15.27:0/1168189933 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0009760 con 0x7f7bf815dc00 2023-12-19T18:26:27.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.153+0000 7f7bf4ff9640 1 -- 172.21.15.27:0/1168189933 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be000cc30 con 0x7f7bf815dc00 2023-12-19T18:26:27.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.153+0000 7f7bf4ff9640 1 -- 172.21.15.27:0/1168189933 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be000eee0 con 0x7f7bf815dc00 2023-12-19T18:26:27.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.153+0000 7f7bfce37640 1 -- 172.21.15.27:0/1168189933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf0009b30 con 0x7f7bf815dc00 2023-12-19T18:26:27.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.154+0000 7f7bfce37640 1 -- 172.21.15.27:0/1168189933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf80ae430 con 0x7f7bf815dc00 2023-12-19T18:26:27.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.155+0000 7f7bfce37640 1 -- 172.21.15.27:0/1168189933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf804e840 con 0x7f7bf815dc00 2023-12-19T18:26:27.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.156+0000 7f7bf4ff9640 1 -- 172.21.15.27:0/1168189933 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7be0013470 con 0x7f7bf815dc00 2023-12-19T18:26:27.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.157+0000 7f7bf4ff9640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7bd806a430 0x7f7bd806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:27.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.157+0000 7f7bf4ff9640 1 -- 172.21.15.27:0/1168189933 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5357+0+0 (secure 0 0 0) 0x7f7be0088ad0 con 0x7f7bf815dc00 2023-12-19T18:26:27.157 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.157+0000 7f7bf77fe640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7bd806a430 0x7f7bd806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:27.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.158+0000 7f7bf77fe640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7bd806a430 0x7f7bd806c8f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7be8000f60 tx=0x7f7be8007ca0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:27.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.161+0000 7f7bf4ff9640 1 -- 172.21.15.27:0/1168189933 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7be0054210 con 0x7f7bf815dc00 2023-12-19T18:26:27.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.356+0000 7f7bfce37640 1 -- 172.21.15.27:0/1168189933 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7bf80ae070 con 0x7f7bf815dc00 2023-12-19T18:26:27.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.889+0000 7f7bf4ff9640 1 -- 172.21.15.27:0/1168189933 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v176) v1 ==== 169+0+0 (secure 0 0 0) 0x7f7be0057830 con 0x7f7bf815dc00 2023-12-19T18:26:27.890 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-12-19T18:26:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.894+0000 7f7bfce37640 1 -- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7bd806a430 msgr2=0x7f7bd806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.894+0000 7f7bfce37640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7bd806a430 0x7f7bd806c8f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f7be8000f60 tx=0x7f7be8007ca0 comp rx=0 tx=0).stop 2023-12-19T18:26:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.894+0000 7f7bfce37640 1 -- 172.21.15.27:0/1168189933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf815dc00 msgr2=0x7f7bf81f25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.894+0000 7f7bfce37640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf815dc00 0x7f7bf81f25e0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f7be000ebe0 tx=0x7f7be000c520 comp rx=0 tx=0).stop 2023-12-19T18:26:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.894+0000 7f7bfce37640 1 -- 172.21.15.27:0/1168189933 shutdown_connections 2023-12-19T18:26:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.894+0000 7f7bfce37640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7bd806a430 0x7f7bd806c8f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.894+0000 7f7bfce37640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf815dfe0 0x7f7bf80c4650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.894+0000 7f7bfce37640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf815dc00 0x7f7bf81f25e0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.894+0000 7f7bfce37640 1 --2- 172.21.15.27:0/1168189933 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf80cb670 0x7f7bf80c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.894+0000 7f7bfce37640 1 -- 172.21.15.27:0/1168189933 >> 172.21.15.27:0/1168189933 conn(0x7f7bf80b05b0 msgr2=0x7f7bf806ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.895+0000 7f7bfce37640 1 -- 172.21.15.27:0/1168189933 shutdown_connections 2023-12-19T18:26:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:27.895+0000 7f7bfce37640 1 -- 172.21.15.27:0/1168189933 wait complete. 2023-12-19T18:26:27.920 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:26:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.144+0000 7f1e921b0640 1 Processor -- start 2023-12-19T18:26:28.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.144+0000 7f1e921b0640 1 -- start start 2023-12-19T18:26:28.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.146+0000 7f1e921b0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e8c0ade20 0x7f1e8c15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:28.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.146+0000 7f1e921b0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1e8c15b810 0x7f1e8c0c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:28.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.146+0000 7f1e921b0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8c15c030 con 0x7f1e8c158790 2023-12-19T18:26:28.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.146+0000 7f1e921b0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e8c0c99d0 con 0x7f1e8c15b810 2023-12-19T18:26:28.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.146+0000 7f1e921b0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c158b70 con 0x7f1e8c0ade20 2023-12-19T18:26:28.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.147+0000 7f1e919af640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1e8c15b810 0x7f1e8c0c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:28.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.147+0000 7f1e919af640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1e8c15b810 0x7f1e8c0c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58830/0 (socket says 172.21.15.27:58830) 2023-12-19T18:26:28.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.147+0000 7f1e919af640 1 -- 172.21.15.27:0/1217960199 learned_addr learned my addr 172.21.15.27:0/1217960199 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:28.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.147+0000 7f1e909ad640 1 --2- 172.21.15.27:0/1217960199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e8c0ade20 0x7f1e8c15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:28.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.147+0000 7f1e919af640 1 -- 172.21.15.27:0/1217960199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e8c0ade20 msgr2=0x7f1e8c15b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:28.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.147+0000 7f1e919af640 1 --2- 172.21.15.27:0/1217960199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e8c0ade20 0x7f1e8c15b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.147+0000 7f1e919af640 1 -- 172.21.15.27:0/1217960199 >> v1:172.21.15.192:6789/0 conn(0x7f1e8c158790 legacy=0x7f1e8c0ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:28.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.147+0000 7f1e919af640 1 -- 172.21.15.27:0/1217960199 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8c0c9030 con 0x7f1e8c15b810 2023-12-19T18:26:28.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.148+0000 7f1e919af640 1 --2- 172.21.15.27:0/1217960199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e8c15b810 0x7f1e8c0c3820 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009e50 tx=0x7f1e8402bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ae5885d1b22554e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:28.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.149+0000 7f1e83fff640 1 -- 172.21.15.27:0/1217960199 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8402caf0 con 0x7f1e8c15b810 2023-12-19T18:26:28.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.149+0000 7f1e83fff640 1 -- 172.21.15.27:0/1217960199 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8402cc50 con 0x7f1e8c15b810 2023-12-19T18:26:28.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.149+0000 7f1e83fff640 1 -- 172.21.15.27:0/1217960199 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e840343f0 con 0x7f1e8c15b810 2023-12-19T18:26:28.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.149+0000 7f1e921b0640 1 -- 172.21.15.27:0/1217960199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e8c15b810 msgr2=0x7f1e8c0c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.149+0000 7f1e921b0640 1 --2- 172.21.15.27:0/1217960199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e8c15b810 0x7f1e8c0c3820 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f1e84009e50 tx=0x7f1e8402bf50 comp rx=0 tx=0).stop 2023-12-19T18:26:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.150+0000 7f1e921b0640 1 -- 172.21.15.27:0/1217960199 shutdown_connections 2023-12-19T18:26:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.150+0000 7f1e921b0640 1 --2- 172.21.15.27:0/1217960199 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e8c15b810 0x7f1e8c0c3820 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.150+0000 7f1e921b0640 1 --2- 172.21.15.27:0/1217960199 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e8c0ade20 0x7f1e8c15b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.150+0000 7f1e921b0640 1 -- 172.21.15.27:0/1217960199 >> 172.21.15.27:0/1217960199 conn(0x7f1e8c0b05b0 msgr2=0x7f1e8c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.150+0000 7f1e921b0640 1 -- 172.21.15.27:0/1217960199 shutdown_connections 2023-12-19T18:26:28.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.150+0000 7f1e921b0640 1 -- 172.21.15.27:0/1217960199 wait complete. 2023-12-19T18:26:28.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.151+0000 7f1e921b0640 1 Processor -- start 2023-12-19T18:26:28.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.152+0000 7f1e921b0640 1 -- start start 2023-12-19T18:26:28.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.153+0000 7f1e921b0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e8c0ade20 0x7f1e8c150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:28.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.153+0000 7f1e921b0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e8c158790 0x7f1e8c155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.153+0000 7f1e921b0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e8c15b810 0x7f1e8c15d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.153+0000 7f1e921b0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c150ea0 con 0x7f1e8c15b810 2023-12-19T18:26:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.153+0000 7f1e921b0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c155e80 con 0x7f1e8c158790 2023-12-19T18:26:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.153+0000 7f1e921b0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8c1548f0 con 0x7f1e8c0ade20 2023-12-19T18:26:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.153+0000 7f1e919af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e8c15b810 0x7f1e8c15d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.153+0000 7f1e919af640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e8c15b810 0x7f1e8c15d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58842/0 (socket says 172.21.15.27:58842) 2023-12-19T18:26:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.153+0000 7f1e919af640 1 -- 172.21.15.27:0/685704028 learned_addr learned my addr 172.21.15.27:0/685704028 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.153+0000 7f1e909ad640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e8c158790 0x7f1e8c155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.153+0000 7f1e911ae640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e8c0ade20 0x7f1e8c150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.154+0000 7f1e919af640 1 -- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e8c0ade20 msgr2=0x7f1e8c150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.154+0000 7f1e919af640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e8c0ade20 0x7f1e8c150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.154+0000 7f1e919af640 1 -- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e8c158790 msgr2=0x7f1e8c155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:28.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.154+0000 7f1e919af640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e8c158790 0x7f1e8c155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.154+0000 7f1e919af640 1 -- 172.21.15.27:0/685704028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e84009b30 con 0x7f1e8c15b810 2023-12-19T18:26:28.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.154+0000 7f1e919af640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e8c15b810 0x7f1e8c15d670 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f1e84036040 tx=0x7f1e84004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:28.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.154+0000 7f1e81ffb640 1 -- 172.21.15.27:0/685704028 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e84009510 con 0x7f1e8c15b810 2023-12-19T18:26:28.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.154+0000 7f1e81ffb640 1 -- 172.21.15.27:0/685704028 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8403e450 con 0x7f1e8c15b810 2023-12-19T18:26:28.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.154+0000 7f1e81ffb640 1 -- 172.21.15.27:0/685704028 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8403e800 con 0x7f1e8c15b810 2023-12-19T18:26:28.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.154+0000 7f1e921b0640 1 -- 172.21.15.27:0/685704028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e8c152030 con 0x7f1e8c15b810 2023-12-19T18:26:28.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.155+0000 7f1e921b0640 1 -- 172.21.15.27:0/685704028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e8c1525f0 con 0x7f1e8c15b810 2023-12-19T18:26:28.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.156+0000 7f1e921b0640 1 -- 172.21.15.27:0/685704028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e8c04e840 con 0x7f1e8c15b810 2023-12-19T18:26:28.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.157+0000 7f1e81ffb640 1 -- 172.21.15.27:0/685704028 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1e8403e960 con 0x7f1e8c15b810 2023-12-19T18:26:28.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.158+0000 7f1e81ffb640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1e7006a430 0x7f1e7006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:28.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.158+0000 7f1e81ffb640 1 -- 172.21.15.27:0/685704028 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1e840ab780 con 0x7f1e8c15b810 2023-12-19T18:26:28.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.159+0000 7f1e911ae640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1e7006a430 0x7f1e7006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:28.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.160+0000 7f1e911ae640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1e7006a430 0x7f1e7006c8f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0cbee0 tx=0x7f1e88009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:28.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.163+0000 7f1e81ffb640 1 -- 172.21.15.27:0/685704028 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1e84076fd0 con 0x7f1e8c15b810 2023-12-19T18:26:28.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.359+0000 7f1e921b0640 1 -- 172.21.15.27:0/685704028 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1e8c0c4e20 con 0x7f1e8c15b810 2023-12-19T18:26:28.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.368+0000 7f1e81ffb640 1 -- 172.21.15.27:0/685704028 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1e8407a5f0 con 0x7f1e8c15b810 2023-12-19T18:26:28.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.371+0000 7f1e921b0640 1 -- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1e7006a430 msgr2=0x7f1e7006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:28.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.371+0000 7f1e921b0640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1e7006a430 0x7f1e7006c8f0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c0cbee0 tx=0x7f1e88009040 comp rx=0 tx=0).stop 2023-12-19T18:26:28.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.371+0000 7f1e921b0640 1 -- 172.21.15.27:0/685704028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e8c15b810 msgr2=0x7f1e8c15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:28.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.371+0000 7f1e921b0640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e8c15b810 0x7f1e8c15d670 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f1e84036040 tx=0x7f1e84004a40 comp rx=0 tx=0).stop 2023-12-19T18:26:28.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.371+0000 7f1e921b0640 1 -- 172.21.15.27:0/685704028 shutdown_connections 2023-12-19T18:26:28.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.371+0000 7f1e921b0640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1e7006a430 0x7f1e7006c8f0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.371+0000 7f1e921b0640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e8c15b810 0x7f1e8c15d670 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.371+0000 7f1e921b0640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e8c158790 0x7f1e8c155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.371+0000 7f1e921b0640 1 --2- 172.21.15.27:0/685704028 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e8c0ade20 0x7f1e8c150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.371+0000 7f1e921b0640 1 -- 172.21.15.27:0/685704028 >> 172.21.15.27:0/685704028 conn(0x7f1e8c0b05b0 msgr2=0x7f1e8c15c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:28.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.372+0000 7f1e921b0640 1 -- 172.21.15.27:0/685704028 shutdown_connections 2023-12-19T18:26:28.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.372+0000 7f1e921b0640 1 -- 172.21.15.27:0/685704028 wait complete. 2023-12-19T18:26:28.390 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose' 2023-12-19T18:26:28.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.614+0000 7ffae744d640 1 Processor -- start 2023-12-19T18:26:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.614+0000 7ffae744d640 1 -- start start 2023-12-19T18:26:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.616+0000 7ffae744d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ffae01590c0 0x7ffae00c8f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.616+0000 7ffae744d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffae00706b0 0x7ffae00ad500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.616+0000 7ffae744d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffae00c9620 con 0x7ffae00ada40 2023-12-19T18:26:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.616+0000 7ffae744d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffae00710b0 con 0x7ffae01590c0 2023-12-19T18:26:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.616+0000 7ffae744d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae0070260 con 0x7ffae00706b0 2023-12-19T18:26:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.617+0000 7ffae644b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ffae01590c0 0x7ffae00c8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.617+0000 7ffae644b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7ffae01590c0 0x7ffae00c8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:45606/0 (socket says 172.21.15.27:45606) 2023-12-19T18:26:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.617+0000 7ffae5c4a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffae00706b0 0x7ffae00ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.617+0000 7ffae644b640 1 -- 172.21.15.27:0/4085029325 learned_addr learned my addr 172.21.15.27:0/4085029325 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.618+0000 7ffae5c4a640 1 -- 172.21.15.27:0/4085029325 >> v2:172.21.15.192:3300/0 conn(0x7ffae01590c0 msgr2=0x7ffae00c8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:28.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.618+0000 7ffae5c4a640 1 --2- 172.21.15.27:0/4085029325 >> v2:172.21.15.192:3300/0 conn(0x7ffae01590c0 0x7ffae00c8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.618+0000 7ffae5c4a640 1 -- 172.21.15.27:0/4085029325 >> v1:172.21.15.192:6789/0 conn(0x7ffae00ada40 legacy=0x7ffae00ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.618+0000 7ffae5c4a640 1 -- 172.21.15.27:0/4085029325 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae00c9ad0 con 0x7ffae00706b0 2023-12-19T18:26:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.618+0000 7ffae644b640 1 --2- 172.21.15.27:0/4085029325 >> v2:172.21.15.192:3300/0 conn(0x7ffae01590c0 0x7ffae00c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:26:28.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.619+0000 7ffae5c4a640 1 --2- 172.21.15.27:0/4085029325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffae00706b0 0x7ffae00ad500 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffad000b160 tx=0x7ffad002c020 comp rx=0 tx=0).ready entity=mon.2 client_cookie=100184dde76e543e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:28.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.620+0000 7ffae5449640 1 -- 172.21.15.27:0/4085029325 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad002cdd0 con 0x7ffae00706b0 2023-12-19T18:26:28.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.620+0000 7ffae5449640 1 -- 172.21.15.27:0/4085029325 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffad0034440 con 0x7ffae00706b0 2023-12-19T18:26:28.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.620+0000 7ffae5449640 1 -- 172.21.15.27:0/4085029325 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffad00347d0 con 0x7ffae00706b0 2023-12-19T18:26:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.621+0000 7ffae744d640 1 -- 172.21.15.27:0/4085029325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffae00706b0 msgr2=0x7ffae00ad500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.621+0000 7ffae744d640 1 --2- 172.21.15.27:0/4085029325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffae00706b0 0x7ffae00ad500 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7ffad000b160 tx=0x7ffad002c020 comp rx=0 tx=0).stop 2023-12-19T18:26:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.621+0000 7ffae744d640 1 -- 172.21.15.27:0/4085029325 shutdown_connections 2023-12-19T18:26:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.621+0000 7ffae744d640 1 --2- 172.21.15.27:0/4085029325 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffae00706b0 0x7ffae00ad500 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.621+0000 7ffae744d640 1 --2- 172.21.15.27:0/4085029325 >> v2:172.21.15.192:3300/0 conn(0x7ffae01590c0 0x7ffae00c8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.621+0000 7ffae744d640 1 -- 172.21.15.27:0/4085029325 >> 172.21.15.27:0/4085029325 conn(0x7ffae00ce940 msgr2=0x7ffae00ced50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:28.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.621+0000 7ffae744d640 1 -- 172.21.15.27:0/4085029325 shutdown_connections 2023-12-19T18:26:28.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.621+0000 7ffae744d640 1 -- 172.21.15.27:0/4085029325 wait complete. 2023-12-19T18:26:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.623+0000 7ffae744d640 1 Processor -- start 2023-12-19T18:26:28.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.623+0000 7ffae744d640 1 -- start start 2023-12-19T18:26:28.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.624+0000 7ffae744d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffae00706b0 0x7ffae00ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:28.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.624+0000 7ffae744d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffae00ada40 0x7ffae00ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:28.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.624+0000 7ffae744d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffae01590c0 0x7ffae00b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:28.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.624+0000 7ffae744d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae00c8a70 con 0x7ffae00706b0 2023-12-19T18:26:28.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.624+0000 7ffae744d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffae01e08c0 con 0x7ffae01590c0 2023-12-19T18:26:28.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.624+0000 7ffae744d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffae01e0b80 con 0x7ffae00ada40 2023-12-19T18:26:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.625+0000 7ffae644b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffae00706b0 0x7ffae00ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.625+0000 7ffae644b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffae00706b0 0x7ffae00ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58844/0 (socket says 172.21.15.27:58844) 2023-12-19T18:26:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.625+0000 7ffae644b640 1 -- 172.21.15.27:0/532450380 learned_addr learned my addr 172.21.15.27:0/532450380 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.625+0000 7ffae5c4a640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffae00ada40 0x7ffae00ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.625+0000 7ffae6c4c640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffae01590c0 0x7ffae00b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.625+0000 7ffae644b640 1 -- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffae00ada40 msgr2=0x7ffae00ace00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.625+0000 7ffae644b640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffae00ada40 0x7ffae00ace00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.625+0000 7ffae644b640 1 -- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffae01590c0 msgr2=0x7ffae00b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.625+0000 7ffae644b640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffae01590c0 0x7ffae00b2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.625+0000 7ffae644b640 1 -- 172.21.15.27:0/532450380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffadc0094d0 con 0x7ffae00706b0 2023-12-19T18:26:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.626+0000 7ffae644b640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffae00706b0 0x7ffae00ac8c0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00e9b0 tx=0x7ffadc00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:28.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.626+0000 7ffad6ffd640 1 -- 172.21.15.27:0/532450380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc00c9a0 con 0x7ffae00706b0 2023-12-19T18:26:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.626+0000 7ffad6ffd640 1 -- 172.21.15.27:0/532450380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffadc00cb00 con 0x7ffae00706b0 2023-12-19T18:26:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.626+0000 7ffae744d640 1 -- 172.21.15.27:0/532450380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad0009400 con 0x7ffae00706b0 2023-12-19T18:26:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.626+0000 7ffad6ffd640 1 -- 172.21.15.27:0/532450380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffadc00ce70 con 0x7ffae00706b0 2023-12-19T18:26:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.626+0000 7ffae744d640 1 -- 172.21.15.27:0/532450380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffae00b2a60 con 0x7ffae00706b0 2023-12-19T18:26:28.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.627+0000 7ffae744d640 1 -- 172.21.15.27:0/532450380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffae004e840 con 0x7ffae00706b0 2023-12-19T18:26:28.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.628+0000 7ffad6ffd640 1 -- 172.21.15.27:0/532450380 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ffadc013470 con 0x7ffae00706b0 2023-12-19T18:26:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.629+0000 7ffad6ffd640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffac406a430 0x7ffac406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:28.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.629+0000 7ffad6ffd640 1 -- 172.21.15.27:0/532450380 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 4943+0+0 (secure 0 0 0) 0x7ffadc088ca0 con 0x7ffae00706b0 2023-12-19T18:26:28.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.630+0000 7ffae5c4a640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffac406a430 0x7ffac406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.631+0000 7ffae5c4a640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffac406a430 0x7ffac406c8f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ffad0036000 tx=0x7ffad0030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:28.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.633+0000 7ffad6ffd640 1 -- 172.21.15.27:0/532450380 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ffadc0544f0 con 0x7ffae00706b0 2023-12-19T18:26:28.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:28.898+0000 7ffae744d640 1 -- 172.21.15.27:0/532450380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]} v 0) v1 -- 0x7ffae00cc5e0 con 0x7ffae00706b0 2023-12-19T18:26:29.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:29.905+0000 7ffad6ffd640 1 -- 172.21.15.27:0/532450380 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fclose"]}]=0 v559) v1 ==== 132+0+0 (secure 0 0 0) 0x7ffadc057b10 con 0x7ffae00706b0 2023-12-19T18:26:29.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:29.909+0000 7ffae744d640 1 -- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffac406a430 msgr2=0x7ffac406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:29.909+0000 7ffae744d640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffac406a430 0x7ffac406c8f0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ffad0036000 tx=0x7ffad0030000 comp rx=0 tx=0).stop 2023-12-19T18:26:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:29.909+0000 7ffae744d640 1 -- 172.21.15.27:0/532450380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffae00706b0 msgr2=0x7ffae00ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:29.909+0000 7ffae744d640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffae00706b0 0x7ffae00ac8c0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7ffadc00e9b0 tx=0x7ffadc00ee70 comp rx=0 tx=0).stop 2023-12-19T18:26:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:29.909+0000 7ffae744d640 1 -- 172.21.15.27:0/532450380 shutdown_connections 2023-12-19T18:26:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:29.909+0000 7ffae744d640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffac406a430 0x7ffac406c8f0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:29.909+0000 7ffae744d640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffae01590c0 0x7ffae00b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:29.909+0000 7ffae744d640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffae00ada40 0x7ffae00ace00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:29.909+0000 7ffae744d640 1 --2- 172.21.15.27:0/532450380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffae00706b0 0x7ffae00ac8c0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:29.909+0000 7ffae744d640 1 -- 172.21.15.27:0/532450380 >> 172.21.15.27:0/532450380 conn(0x7ffae00ce940 msgr2=0x7ffae00ced20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:29.910+0000 7ffae744d640 1 -- 172.21.15.27:0/532450380 shutdown_connections 2023-12-19T18:26:29.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:29.910+0000 7ffae744d640 1 -- 172.21.15.27:0/532450380 wait complete. 2023-12-19T18:26:29.933 INFO:tasks.cephfs_test_runner:test_full_fclose (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-12-19T18:26:29.933 INFO:tasks.cephfs_test_runner:Starting test: test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) That when the full flag is encountered during asynchronous 2023-12-19T18:26:29.933 INFO:tasks.cephfs_test_runner:test_full_fsync (tasks.cephfs.test_full.TestQuotaFull) 2023-12-19T18:26:29.934 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-12-19T18:26:30.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.170+0000 7f55dcc07640 1 Processor -- start 2023-12-19T18:26:30.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.171+0000 7f55dcc07640 1 -- start start 2023-12-19T18:26:30.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.172+0000 7f55dcc07640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f55d8154030 0x7f55d80cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:30.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.172+0000 7f55dcc07640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55d815e440 con 0x7f55d80c5560 2023-12-19T18:26:30.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.172+0000 7f55dcc07640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55d80cc190 con 0x7f55d80af630 2023-12-19T18:26:30.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.172+0000 7f55dcc07640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55d80cb2a0 con 0x7f55d8154030 2023-12-19T18:26:30.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.173+0000 7f55d7fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f55d80c5560 0x7f55d815dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34910/0 (socket says 172.21.15.27:34910) 2023-12-19T18:26:30.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.173+0000 7f55d77fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f55d8154030 0x7f55d80cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:30.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.173+0000 7f55d7fff640 1 -- 172.21.15.27:0/1122376327 learned_addr learned my addr 172.21.15.27:0/1122376327 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:30.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.173+0000 7f55d77fe640 1 -- 172.21.15.27:0/1122376327 >> v1:172.21.15.192:6789/0 conn(0x7f55d80af630 legacy=0x7f55d815d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:30.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.173+0000 7f55d77fe640 1 -- 172.21.15.27:0/1122376327 >> v1:172.21.15.27:6789/0 conn(0x7f55d80c5560 legacy=0x7f55d815dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:30.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.173+0000 7f55d77fe640 1 -- 172.21.15.27:0/1122376327 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d80cb580 con 0x7f55d8154030 2023-12-19T18:26:30.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.173+0000 7f55d6ffd640 1 -- 172.21.15.27:0/1122376327 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3162471335 0 0) 0x7f55d815e440 con 0x7f55d80c5560 2023-12-19T18:26:30.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.175+0000 7f55d77fe640 1 --2- 172.21.15.27:0/1122376327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55d8154030 0x7f55d80cd4f0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f55c80097d0 tx=0x7f55c802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9403be5eb3acdb2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:30.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.175+0000 7f55d6ffd640 1 -- 172.21.15.27:0/1122376327 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8030610 con 0x7f55d8154030 2023-12-19T18:26:30.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.175+0000 7f55d6ffd640 1 -- 172.21.15.27:0/1122376327 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55c8030770 con 0x7f55d8154030 2023-12-19T18:26:30.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.175+0000 7f55d6ffd640 1 -- 172.21.15.27:0/1122376327 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c8030b00 con 0x7f55d8154030 2023-12-19T18:26:30.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.175+0000 7f55dcc07640 1 -- 172.21.15.27:0/1122376327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55d8154030 msgr2=0x7f55d80cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:30.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.175+0000 7f55dcc07640 1 --2- 172.21.15.27:0/1122376327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55d8154030 0x7f55d80cd4f0 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f55c80097d0 tx=0x7f55c802b960 comp rx=0 tx=0).stop 2023-12-19T18:26:30.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.176+0000 7f55dcc07640 1 -- 172.21.15.27:0/1122376327 shutdown_connections 2023-12-19T18:26:30.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.176+0000 7f55dcc07640 1 --2- 172.21.15.27:0/1122376327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55d8154030 0x7f55d80cd4f0 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:30.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.176+0000 7f55dcc07640 1 -- 172.21.15.27:0/1122376327 >> 172.21.15.27:0/1122376327 conn(0x7f55d8069c70 msgr2=0x7f55d806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:30.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.176+0000 7f55dcc07640 1 -- 172.21.15.27:0/1122376327 shutdown_connections 2023-12-19T18:26:30.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.176+0000 7f55dcc07640 1 -- 172.21.15.27:0/1122376327 wait complete. 2023-12-19T18:26:30.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.178+0000 7f55dcc07640 1 Processor -- start 2023-12-19T18:26:30.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.178+0000 7f55dcc07640 1 -- start start 2023-12-19T18:26:30.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.179+0000 7f55dcc07640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55d80af630 0x7f55d8130e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:30.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.179+0000 7f55dcc07640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55d80c5560 0x7f55d813aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.179+0000 7f55dcc07640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55d8154030 0x7f55d813afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.179+0000 7f55dcc07640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d8131380 con 0x7f55d8154030 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.179+0000 7f55dcc07640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55d8134b00 con 0x7f55d80af630 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.179+0000 7f55dcc07640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55d81538b0 con 0x7f55d80c5560 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.179+0000 7f55d7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55d8154030 0x7f55d813afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.179+0000 7f55d7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55d8154030 0x7f55d813afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58856/0 (socket says 172.21.15.27:58856) 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.179+0000 7f55d7fff640 1 -- 172.21.15.27:0/3454095295 learned_addr learned my addr 172.21.15.27:0/3454095295 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.179+0000 7f55d77fe640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55d80af630 0x7f55d8130e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.179+0000 7f55cffff640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55d80c5560 0x7f55d813aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.180+0000 7f55d7fff640 1 -- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55d80c5560 msgr2=0x7f55d813aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.180+0000 7f55d7fff640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55d80c5560 0x7f55d813aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.180+0000 7f55d7fff640 1 -- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55d80af630 msgr2=0x7f55d8130e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.180+0000 7f55d7fff640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55d80af630 0x7f55d8130e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:30.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.180+0000 7f55d7fff640 1 -- 172.21.15.27:0/3454095295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c8009400 con 0x7f55d8154030 2023-12-19T18:26:30.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.180+0000 7f55d7fff640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55d8154030 0x7f55d813afb0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f55d000c7a0 tx=0x7f55d000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:30.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.180+0000 7f55d4ff9640 1 -- 172.21.15.27:0/3454095295 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d000a640 con 0x7f55d8154030 2023-12-19T18:26:30.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.180+0000 7f55d4ff9640 1 -- 172.21.15.27:0/3454095295 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d000a7a0 con 0x7f55d8154030 2023-12-19T18:26:30.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.180+0000 7f55d4ff9640 1 -- 172.21.15.27:0/3454095295 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d000abd0 con 0x7f55d8154030 2023-12-19T18:26:30.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.180+0000 7f55dcc07640 1 -- 172.21.15.27:0/3454095295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55d806cae0 con 0x7f55d8154030 2023-12-19T18:26:30.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.180+0000 7f55dcc07640 1 -- 172.21.15.27:0/3454095295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55d806d0a0 con 0x7f55d8154030 2023-12-19T18:26:30.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.181+0000 7f55dcc07640 1 -- 172.21.15.27:0/3454095295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55d804e840 con 0x7f55d8154030 2023-12-19T18:26:30.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.182+0000 7f55d4ff9640 1 -- 172.21.15.27:0/3454095295 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f55d000ad30 con 0x7f55d8154030 2023-12-19T18:26:30.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.183+0000 7f55d4ff9640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f55b806a430 0x7f55b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:30.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.183+0000 7f55d4ff9640 1 -- 172.21.15.27:0/3454095295 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f55d0087b80 con 0x7f55d8154030 2023-12-19T18:26:30.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.184+0000 7f55d77fe640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f55b806a430 0x7f55b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:30.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.185+0000 7f55d77fe640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f55b806a430 0x7f55b806c8f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f55c8036000 tx=0x7f55c803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:30.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.188+0000 7f55d4ff9640 1 -- 172.21.15.27:0/3454095295 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f55d0053470 con 0x7f55d8154030 2023-12-19T18:26:30.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.451+0000 7f55dcc07640 1 -- 172.21.15.27:0/3454095295 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7f55d80ce940 con 0x7f55d8154030 2023-12-19T18:26:30.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.915+0000 7f55d4ff9640 1 -- 172.21.15.27:0/3454095295 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v560) v1 ==== 134+0+0 (secure 0 0 0) 0x7f55d0018020 con 0x7f55d8154030 2023-12-19T18:26:30.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.918+0000 7f55dcc07640 1 -- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f55b806a430 msgr2=0x7f55b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:30.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.918+0000 7f55dcc07640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f55b806a430 0x7f55b806c8f0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f55c8036000 tx=0x7f55c803b040 comp rx=0 tx=0).stop 2023-12-19T18:26:30.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.918+0000 7f55dcc07640 1 -- 172.21.15.27:0/3454095295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55d8154030 msgr2=0x7f55d813afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:30.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.918+0000 7f55dcc07640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55d8154030 0x7f55d813afb0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f55d000c7a0 tx=0x7f55d000cc60 comp rx=0 tx=0).stop 2023-12-19T18:26:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.918+0000 7f55dcc07640 1 -- 172.21.15.27:0/3454095295 shutdown_connections 2023-12-19T18:26:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.918+0000 7f55dcc07640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f55b806a430 0x7f55b806c8f0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.918+0000 7f55dcc07640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55d8154030 0x7f55d813afb0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.918+0000 7f55dcc07640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f55d80c5560 0x7f55d813aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.918+0000 7f55dcc07640 1 --2- 172.21.15.27:0/3454095295 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f55d80af630 0x7f55d8130e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.918+0000 7f55dcc07640 1 -- 172.21.15.27:0/3454095295 >> 172.21.15.27:0/3454095295 conn(0x7f55d8069c70 msgr2=0x7f55d80ca590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.919+0000 7f55dcc07640 1 -- 172.21.15.27:0/3454095295 shutdown_connections 2023-12-19T18:26:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:30.919+0000 7f55dcc07640 1 -- 172.21.15.27:0/3454095295 wait complete. 2023-12-19T18:26:30.943 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:26:31.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.183+0000 7fdcd359e640 1 Processor -- start 2023-12-19T18:26:31.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.183+0000 7fdcd359e640 1 -- start start 2023-12-19T18:26:31.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.185+0000 7fdcd359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd40d2100 0x7fdcd40cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:31.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.185+0000 7fdcd359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdcd40cdf80 0x7fdcd40b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:31.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.185+0000 7fdcd359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcd40d0e10 con 0x7fdcd415cf80 2023-12-19T18:26:31.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.185+0000 7fdcd359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcd40b8cf0 con 0x7fdcd40cdf80 2023-12-19T18:26:31.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.185+0000 7fdcd359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40ce410 con 0x7fdcd40d2100 2023-12-19T18:26:31.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.185+0000 7fdcd259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdcd415cf80 0x7fdcd40b85e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:34926/0 (socket says 172.21.15.27:34926) 2023-12-19T18:26:31.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.185+0000 7fdcd259c640 1 -- 172.21.15.27:0/1842413295 learned_addr learned my addr 172.21.15.27:0/1842413295 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:31.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.185+0000 7fdcd2d9d640 1 --2- 172.21.15.27:0/1842413295 >> v2:172.21.15.27:3300/0 conn(0x7fdcd40cdf80 0x7fdcd40b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.185+0000 7fdcd1d9b640 1 --2- 172.21.15.27:0/1842413295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd40d2100 0x7fdcd40cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.185+0000 7fdcd2d9d640 1 -- 172.21.15.27:0/1842413295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd40d2100 msgr2=0x7fdcd40cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.185+0000 7fdcd2d9d640 1 --2- 172.21.15.27:0/1842413295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd40d2100 0x7fdcd40cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.186+0000 7fdcd2d9d640 1 -- 172.21.15.27:0/1842413295 >> v1:172.21.15.27:6789/0 conn(0x7fdcd415cf80 legacy=0x7fdcd40b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.186+0000 7fdcd159a640 1 -- 172.21.15.27:0/1842413295 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2441002870 0 0) 0x7fdcd40d0e10 con 0x7fdcd415cf80 2023-12-19T18:26:31.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.186+0000 7fdcd2d9d640 1 -- 172.21.15.27:0/1842413295 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd40d1c70 con 0x7fdcd40cdf80 2023-12-19T18:26:31.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.187+0000 7fdcd2d9d640 1 --2- 172.21.15.27:0/1842413295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd40cdf80 0x7fdcd40b5d60 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0097a0 tx=0x7fdccc02b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba104e4420c637c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:31.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.187+0000 7fdcd159a640 1 -- 172.21.15.27:0/1842413295 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc030800 con 0x7fdcd40cdf80 2023-12-19T18:26:31.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.187+0000 7fdcd159a640 1 -- 172.21.15.27:0/1842413295 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdccc030960 con 0x7fdcd40cdf80 2023-12-19T18:26:31.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.187+0000 7fdcd159a640 1 -- 172.21.15.27:0/1842413295 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc030cf0 con 0x7fdcd40cdf80 2023-12-19T18:26:31.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.188+0000 7fdcd359e640 1 -- 172.21.15.27:0/1842413295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd40cdf80 msgr2=0x7fdcd40b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:31.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.188+0000 7fdcd359e640 1 --2- 172.21.15.27:0/1842413295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd40cdf80 0x7fdcd40b5d60 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0097a0 tx=0x7fdccc02b940 comp rx=0 tx=0).stop 2023-12-19T18:26:31.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.188+0000 7fdcd359e640 1 -- 172.21.15.27:0/1842413295 shutdown_connections 2023-12-19T18:26:31.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.188+0000 7fdcd359e640 1 --2- 172.21.15.27:0/1842413295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd40cdf80 0x7fdcd40b5d60 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:31.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.188+0000 7fdcd359e640 1 --2- 172.21.15.27:0/1842413295 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd40d2100 0x7fdcd40cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:31.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.188+0000 7fdcd359e640 1 -- 172.21.15.27:0/1842413295 >> 172.21.15.27:0/1842413295 conn(0x7fdcd4072e20 msgr2=0x7fdcd4073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:31.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.188+0000 7fdcd359e640 1 -- 172.21.15.27:0/1842413295 shutdown_connections 2023-12-19T18:26:31.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.188+0000 7fdcd359e640 1 -- 172.21.15.27:0/1842413295 wait complete. 2023-12-19T18:26:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.190+0000 7fdcd359e640 1 Processor -- start 2023-12-19T18:26:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.190+0000 7fdcd359e640 1 -- start start 2023-12-19T18:26:31.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.191+0000 7fdcd359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd40cdf80 0x7fdcd4152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:31.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.191+0000 7fdcd359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd40d2100 0x7fdcd4160000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:31.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.191+0000 7fdcd359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd415cf80 0x7fdcd4153570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.191+0000 7fdcd359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd41533b0 con 0x7fdcd415cf80 2023-12-19T18:26:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.191+0000 7fdcd359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40cbf40 con 0x7fdcd40cdf80 2023-12-19T18:26:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.191+0000 7fdcd359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcd40cc220 con 0x7fdcd40d2100 2023-12-19T18:26:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.191+0000 7fdcd2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd415cf80 0x7fdcd4153570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.191+0000 7fdcd2d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd415cf80 0x7fdcd4153570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58866/0 (socket says 172.21.15.27:58866) 2023-12-19T18:26:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.191+0000 7fdcd2d9d640 1 -- 172.21.15.27:0/4023691060 learned_addr learned my addr 172.21.15.27:0/4023691060 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.192+0000 7fdcd1d9b640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd40d2100 0x7fdcd4160000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.192+0000 7fdcd259c640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd40cdf80 0x7fdcd4152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.192+0000 7fdcd2d9d640 1 -- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd40d2100 msgr2=0x7fdcd4160000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.192+0000 7fdcd2d9d640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd40d2100 0x7fdcd4160000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.192+0000 7fdcd2d9d640 1 -- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd40cdf80 msgr2=0x7fdcd4152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.192+0000 7fdcd2d9d640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd40cdf80 0x7fdcd4152e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.192+0000 7fdcd2d9d640 1 -- 172.21.15.27:0/4023691060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdccc009400 con 0x7fdcd415cf80 2023-12-19T18:26:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.192+0000 7fdcd2d9d640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd415cf80 0x7fdcd4153570 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0097a0 tx=0x7fdccc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.193+0000 7fdccaffd640 1 -- 172.21.15.27:0/4023691060 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc039c10 con 0x7fdcd415cf80 2023-12-19T18:26:31.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.193+0000 7fdccaffd640 1 -- 172.21.15.27:0/4023691060 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdccc03f040 con 0x7fdcd415cf80 2023-12-19T18:26:31.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.193+0000 7fdccaffd640 1 -- 172.21.15.27:0/4023691060 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdccc006e10 con 0x7fdcd415cf80 2023-12-19T18:26:31.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.193+0000 7fdcd359e640 1 -- 172.21.15.27:0/4023691060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcd40cc500 con 0x7fdcd415cf80 2023-12-19T18:26:31.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.193+0000 7fdcd359e640 1 -- 172.21.15.27:0/4023691060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcd4154cd0 con 0x7fdcd415cf80 2023-12-19T18:26:31.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.194+0000 7fdcd359e640 1 -- 172.21.15.27:0/4023691060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcd415a2c0 con 0x7fdcd415cf80 2023-12-19T18:26:31.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.195+0000 7fdccaffd640 1 -- 172.21.15.27:0/4023691060 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdccc004030 con 0x7fdcd415cf80 2023-12-19T18:26:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.196+0000 7fdccaffd640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdcb406a460 0x7fdcb406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.196+0000 7fdccaffd640 1 -- 172.21.15.27:0/4023691060 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fdccc0aa710 con 0x7fdcd415cf80 2023-12-19T18:26:31.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.196+0000 7fdcd259c640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdcb406a460 0x7fdcb406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:31.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.198+0000 7fdcd259c640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdcb406a460 0x7fdcb406c920 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4007ed0 tx=0x7fdcc4007e60 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:31.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.200+0000 7fdccaffd640 1 -- 172.21.15.27:0/4023691060 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdccc075ff0 con 0x7fdcd415cf80 2023-12-19T18:26:31.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.399+0000 7fdcd359e640 1 -- 172.21.15.27:0/4023691060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdcd404e840 con 0x7fdcd415cf80 2023-12-19T18:26:31.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.406+0000 7fdccaffd640 1 -- 172.21.15.27:0/4023691060 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdccc079610 con 0x7fdcd415cf80 2023-12-19T18:26:31.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.408+0000 7fdcd359e640 1 -- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdcb406a460 msgr2=0x7fdcb406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:31.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.408+0000 7fdcd359e640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdcb406a460 0x7fdcb406c920 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fdcc4007ed0 tx=0x7fdcc4007e60 comp rx=0 tx=0).stop 2023-12-19T18:26:31.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.408+0000 7fdcd359e640 1 -- 172.21.15.27:0/4023691060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd415cf80 msgr2=0x7fdcd4153570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:31.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.408+0000 7fdcd359e640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd415cf80 0x7fdcd4153570 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7fdccc0097a0 tx=0x7fdccc0046c0 comp rx=0 tx=0).stop 2023-12-19T18:26:31.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.409+0000 7fdcd359e640 1 -- 172.21.15.27:0/4023691060 shutdown_connections 2023-12-19T18:26:31.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.409+0000 7fdcd359e640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdcb406a460 0x7fdcb406c920 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:31.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.409+0000 7fdcd359e640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdcd415cf80 0x7fdcd4153570 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:31.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.409+0000 7fdcd359e640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdcd40d2100 0x7fdcd4160000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:31.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.409+0000 7fdcd359e640 1 --2- 172.21.15.27:0/4023691060 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdcd40cdf80 0x7fdcd4152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:31.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.409+0000 7fdcd359e640 1 -- 172.21.15.27:0/4023691060 >> 172.21.15.27:0/4023691060 conn(0x7fdcd4072e20 msgr2=0x7fdcd415c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:31.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.409+0000 7fdcd359e640 1 -- 172.21.15.27:0/4023691060 shutdown_connections 2023-12-19T18:26:31.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:31.409+0000 7fdcd359e640 1 -- 172.21.15.27:0/4023691060 wait complete. 2023-12-19T18:26:31.429 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:26:31.429 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:26:31.558 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:26:31.682 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:26:31.682 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:26:31.682 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:26:31.747 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:26:31.747 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:26:31.747 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:26:31.804 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:26:31.804 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:26:31.804 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:26:31.868 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:26:31.868 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:26:31.868 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:26:31.928 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:26:31.928 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:26:31.928 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:26:32.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.172+0000 7fd2bae05640 1 Processor -- start 2023-12-19T18:26:32.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.173+0000 7fd2bae05640 1 -- start start 2023-12-19T18:26:32.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.174+0000 7fd2bae05640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2b40b4c80 0x7fd2b4160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:32.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.175+0000 7fd2bae05640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd2b40b5060 0x7fd2b40d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:32.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.175+0000 7fd2bae05640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b40d0d20 con 0x7fd2b40b8920 2023-12-19T18:26:32.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.175+0000 7fd2bae05640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b40b6130 con 0x7fd2b40b5060 2023-12-19T18:26:32.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.175+0000 7fd2bae05640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4161bf0 con 0x7fd2b40b4c80 2023-12-19T18:26:32.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.175+0000 7fd2b8ba1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2b40b4c80 0x7fd2b4160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:32.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.176+0000 7fd2b93a2640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd2b40b5060 0x7fd2b40d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:32.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.176+0000 7fd2b8ba1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2b40b4c80 0x7fd2b4160c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:55106/0 (socket says 172.21.15.27:55106) 2023-12-19T18:26:32.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.176+0000 7fd2b8ba1640 1 -- 172.21.15.27:0/1031314286 learned_addr learned my addr 172.21.15.27:0/1031314286 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:32.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.176+0000 7fd2b8ba1640 1 -- 172.21.15.27:0/1031314286 >> v2:172.21.15.192:3300/0 conn(0x7fd2b40b5060 msgr2=0x7fd2b40d1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:32.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.176+0000 7fd2b8ba1640 1 --2- 172.21.15.27:0/1031314286 >> v2:172.21.15.192:3300/0 conn(0x7fd2b40b5060 0x7fd2b40d1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.176+0000 7fd2b8ba1640 1 -- 172.21.15.27:0/1031314286 >> v1:172.21.15.192:6789/0 conn(0x7fd2b40b8920 legacy=0x7fd2b415cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:32.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.176+0000 7fd2b8ba1640 1 -- 172.21.15.27:0/1031314286 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b4161350 con 0x7fd2b40b4c80 2023-12-19T18:26:32.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.176+0000 7fd2b93a2640 1 --2- 172.21.15.27:0/1031314286 >> v2:172.21.15.192:3300/0 conn(0x7fd2b40b5060 0x7fd2b40d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:26:32.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.178+0000 7fd2b8ba1640 1 --2- 172.21.15.27:0/1031314286 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2b40b4c80 0x7fd2b4160c70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009870 tx=0x7fd2a802b8f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3468d389ca8d7cf3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:32.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.178+0000 7fd2b37fe640 1 -- 172.21.15.27:0/1031314286 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8031640 con 0x7fd2b40b4c80 2023-12-19T18:26:32.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.178+0000 7fd2b37fe640 1 -- 172.21.15.27:0/1031314286 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a80317a0 con 0x7fd2b40b4c80 2023-12-19T18:26:32.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.178+0000 7fd2b37fe640 1 -- 172.21.15.27:0/1031314286 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8031b30 con 0x7fd2b40b4c80 2023-12-19T18:26:32.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.179+0000 7fd2bae05640 1 -- 172.21.15.27:0/1031314286 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2b40b4c80 msgr2=0x7fd2b4160c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:32.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.179+0000 7fd2bae05640 1 --2- 172.21.15.27:0/1031314286 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2b40b4c80 0x7fd2b4160c70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009870 tx=0x7fd2a802b8f0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.179+0000 7fd2bae05640 1 -- 172.21.15.27:0/1031314286 shutdown_connections 2023-12-19T18:26:32.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.179+0000 7fd2bae05640 1 --2- 172.21.15.27:0/1031314286 >> v2:172.21.15.192:3300/0 conn(0x7fd2b40b5060 0x7fd2b40d1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.179+0000 7fd2bae05640 1 --2- 172.21.15.27:0/1031314286 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2b40b4c80 0x7fd2b4160c70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.179+0000 7fd2bae05640 1 -- 172.21.15.27:0/1031314286 >> 172.21.15.27:0/1031314286 conn(0x7fd2b40d2f90 msgr2=0x7fd2b40d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:32.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.180+0000 7fd2bae05640 1 -- 172.21.15.27:0/1031314286 shutdown_connections 2023-12-19T18:26:32.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.180+0000 7fd2bae05640 1 -- 172.21.15.27:0/1031314286 wait complete. 2023-12-19T18:26:32.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.181+0000 7fd2bae05640 1 Processor -- start 2023-12-19T18:26:32.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.181+0000 7fd2bae05640 1 -- start start 2023-12-19T18:26:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.182+0000 7fd2bae05640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2b40b4c80 0x7fd2b4157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.182+0000 7fd2bae05640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2b40b5060 0x7fd2b415fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.182+0000 7fd2bae05640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2b40b8920 0x7fd2b41602e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.182+0000 7fd2bae05640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b4158190 con 0x7fd2b40b4c80 2023-12-19T18:26:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.182+0000 7fd2bae05640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b41f29f0 con 0x7fd2b40b5060 2023-12-19T18:26:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.182+0000 7fd2bae05640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b41f2cd0 con 0x7fd2b40b8920 2023-12-19T18:26:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.183+0000 7fd2b8ba1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2b40b4c80 0x7fd2b4157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.183+0000 7fd2b8ba1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2b40b4c80 0x7fd2b4157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43814/0 (socket says 172.21.15.27:43814) 2023-12-19T18:26:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.183+0000 7fd2b8ba1640 1 -- 172.21.15.27:0/146315347 learned_addr learned my addr 172.21.15.27:0/146315347 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.183+0000 7fd2b93a2640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2b40b8920 0x7fd2b41602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:32.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.183+0000 7fd2b3fff640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2b40b5060 0x7fd2b415fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.183+0000 7fd2b8ba1640 1 -- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2b40b8920 msgr2=0x7fd2b41602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:32.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.183+0000 7fd2b8ba1640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2b40b8920 0x7fd2b41602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.183+0000 7fd2b8ba1640 1 -- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2b40b5060 msgr2=0x7fd2b415fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.183+0000 7fd2b8ba1640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2b40b5060 0x7fd2b415fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.183+0000 7fd2b8ba1640 1 -- 172.21.15.27:0/146315347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ac009a70 con 0x7fd2b40b4c80 2023-12-19T18:26:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.184+0000 7fd2b8ba1640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2b40b4c80 0x7fd2b4157c50 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8000c00 tx=0x7fd2a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.184+0000 7fd2b17fa640 1 -- 172.21.15.27:0/146315347 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a803a930 con 0x7fd2b40b4c80 2023-12-19T18:26:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.184+0000 7fd2b17fa640 1 -- 172.21.15.27:0/146315347 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a803aa90 con 0x7fd2b40b4c80 2023-12-19T18:26:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.184+0000 7fd2bae05640 1 -- 172.21.15.27:0/146315347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a80094d0 con 0x7fd2b40b4c80 2023-12-19T18:26:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.184+0000 7fd2b17fa640 1 -- 172.21.15.27:0/146315347 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a803ae40 con 0x7fd2b40b4c80 2023-12-19T18:26:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.184+0000 7fd2bae05640 1 -- 172.21.15.27:0/146315347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b41f3290 con 0x7fd2b40b4c80 2023-12-19T18:26:32.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.186+0000 7fd2bae05640 1 -- 172.21.15.27:0/146315347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b415c870 con 0x7fd2b40b4c80 2023-12-19T18:26:32.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.187+0000 7fd2b17fa640 1 -- 172.21.15.27:0/146315347 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd2a802f050 con 0x7fd2b40b4c80 2023-12-19T18:26:32.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.187+0000 7fd2b17fa640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd29806a430 0x7fd29806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:32.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.188+0000 7fd2b17fa640 1 -- 172.21.15.27:0/146315347 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd2a80aa470 con 0x7fd2b40b4c80 2023-12-19T18:26:32.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.188+0000 7fd2b3fff640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd29806a430 0x7fd29806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.189+0000 7fd2b3fff640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd29806a430 0x7fd29806c8f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40cb130 tx=0x7fd2a0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.192+0000 7fd2b17fa640 1 -- 172.21.15.27:0/146315347 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd2a8075cc0 con 0x7fd2b40b4c80 2023-12-19T18:26:32.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.427+0000 7fd2bae05640 1 -- 172.21.15.27:0/146315347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd2b4068e90 con 0x7fd2b40b4c80 2023-12-19T18:26:32.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.428+0000 7fd2b17fa640 1 -- 172.21.15.27:0/146315347 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fd2a8035de0 con 0x7fd2b40b4c80 2023-12-19T18:26:32.428 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:32.429 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":71,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5867,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3291476389","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3291476389},{"type":"v1","addr":"172.21.15.192:6836","nonce":3291476389}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5870,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3526330256","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3526330256},{"type":"v1","addr":"172.21.15.192:6837","nonce":3526330256}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":5876,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/899234003","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":899234003},{"type":"v1","addr":"172.21.15.27:6835","nonce":899234003}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":8559,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/2220157551","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":2220157551},{"type":"v1","addr":"172.21.15.27:6833","nonce":2220157551}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[]} 2023-12-19T18:26:32.429 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 71 2023-12-19T18:26:32.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.431+0000 7fd2bae05640 1 -- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd29806a430 msgr2=0x7fd29806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:32.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.431+0000 7fd2bae05640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd29806a430 0x7fd29806c8f0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fd2b40cb130 tx=0x7fd2a0009040 comp rx=0 tx=0).stop 2023-12-19T18:26:32.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.431+0000 7fd2bae05640 1 -- 172.21.15.27:0/146315347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2b40b4c80 msgr2=0x7fd2b4157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:32.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.431+0000 7fd2bae05640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2b40b4c80 0x7fd2b4157c50 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8000c00 tx=0x7fd2a80046c0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.431+0000 7fd2bae05640 1 -- 172.21.15.27:0/146315347 shutdown_connections 2023-12-19T18:26:32.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.431+0000 7fd2bae05640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd29806a430 0x7fd29806c8f0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.431+0000 7fd2bae05640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd2b40b8920 0x7fd2b41602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.431+0000 7fd2bae05640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd2b40b5060 0x7fd2b415fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.431+0000 7fd2bae05640 1 --2- 172.21.15.27:0/146315347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2b40b4c80 0x7fd2b4157c50 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.431+0000 7fd2bae05640 1 -- 172.21.15.27:0/146315347 >> 172.21.15.27:0/146315347 conn(0x7fd2b40d2f90 msgr2=0x7fd2b40cf860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.431+0000 7fd2bae05640 1 -- 172.21.15.27:0/146315347 shutdown_connections 2023-12-19T18:26:32.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.431+0000 7fd2bae05640 1 -- 172.21.15.27:0/146315347 wait complete. 2023-12-19T18:26:32.452 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:26:32.453 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:26:32.453 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:26:32.453 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:26:32.453 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:26:32.453 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:26:32.453 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:26:32.453 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:26:32.454 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:26:32.454 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:26:32.454 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:26:32.454 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:26:32.454 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:26:32.454 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:26:32.454 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:26:32.454 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:26:32.475 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:26:32.475 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:26:32.476 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:26:32.476 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:26:32.479 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:26:32.479 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:26:32.480 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:26:32.480 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:26:32.524 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:26:32.524 DEBUG:teuthology.parallel:result is None 2023-12-19T18:26:32.528 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:26:32.528 DEBUG:teuthology.parallel:result is None 2023-12-19T18:26:32.530 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:26:32.531 DEBUG:teuthology.parallel:result is None 2023-12-19T18:26:32.535 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:26:32.535 DEBUG:teuthology.parallel:result is None 2023-12-19T18:26:32.536 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:26:32.684 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-19T18:26:32.685 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-19T18:26:32.701 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:26:32.703 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:26:32.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.745+0000 7fc7f607d640 1 Processor -- start 2023-12-19T18:26:32.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.745+0000 7fc7f607d640 1 -- start start 2023-12-19T18:26:32.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.746+0000 7fc7f607d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f015acb0 0x7fc7f00cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:32.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.746+0000 7fc7f607d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc7f00cc130 0x7fc7f00cc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:32.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.746+0000 7fc7f607d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc7f015e880 0x7fc7f015ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:32.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.746+0000 7fc7f607d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f00cf1d0 con 0x7fc7f00cc130 2023-12-19T18:26:32.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.746+0000 7fc7f607d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7f015f250 con 0x7fc7f015e880 2023-12-19T18:26:32.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.746+0000 7fc7f607d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00ce5e0 con 0x7fc7f015acb0 2023-12-19T18:26:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.746+0000 7fc7f487a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc7f00cc130 0x7fc7f00cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.747+0000 7fc7f487a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc7f00cc130 0x7fc7f00cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43838/0 (socket says 172.21.15.27:43838) 2023-12-19T18:26:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.747+0000 7fc7f487a640 1 -- 172.21.15.27:0/3361777311 learned_addr learned my addr 172.21.15.27:0/3361777311 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.747+0000 7fc7f587c640 1 --2- 172.21.15.27:0/3361777311 >> v2:172.21.15.192:3300/0 conn(0x7fc7f015e880 0x7fc7f015ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.747+0000 7fc7f507b640 1 --2- 172.21.15.27:0/3361777311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f015acb0 0x7fc7f00cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:32.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.747+0000 7fc7f487a640 1 -- 172.21.15.27:0/3361777311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f015acb0 msgr2=0x7fc7f00cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.747+0000 7fc7f487a640 1 --2- 172.21.15.27:0/3361777311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f015acb0 0x7fc7f00cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.747+0000 7fc7f487a640 1 -- 172.21.15.27:0/3361777311 >> v2:172.21.15.192:3300/0 conn(0x7fc7f015e880 msgr2=0x7fc7f015ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.747+0000 7fc7f487a640 1 --2- 172.21.15.27:0/3361777311 >> v2:172.21.15.192:3300/0 conn(0x7fc7f015e880 0x7fc7f015ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.747+0000 7fc7f487a640 1 -- 172.21.15.27:0/3361777311 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f00ce830 con 0x7fc7f00cc130 2023-12-19T18:26:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.747+0000 7fc7f587c640 1 --2- 172.21.15.27:0/3361777311 >> v2:172.21.15.192:3300/0 conn(0x7fc7f015e880 0x7fc7f015ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:32.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.747+0000 7fc7f507b640 1 --2- 172.21.15.27:0/3361777311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f015acb0 0x7fc7f00cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.748+0000 7fc7f487a640 1 --2- 172.21.15.27:0/3361777311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f00cc130 0x7fc7f00cc590 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0071450 tx=0x7fc7e002b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b018ddbc20aa0e2a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.749+0000 7fc7e7fff640 1 -- 172.21.15.27:0/3361777311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e00317d0 con 0x7fc7f00cc130 2023-12-19T18:26:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.749+0000 7fc7e7fff640 1 -- 172.21.15.27:0/3361777311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e0031930 con 0x7fc7f00cc130 2023-12-19T18:26:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.749+0000 7fc7e7fff640 1 -- 172.21.15.27:0/3361777311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e0031cc0 con 0x7fc7f00cc130 2023-12-19T18:26:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.749+0000 7fc7f607d640 1 -- 172.21.15.27:0/3361777311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f00cc130 msgr2=0x7fc7f00cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.749+0000 7fc7f607d640 1 --2- 172.21.15.27:0/3361777311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f00cc130 0x7fc7f00cc590 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7fc7f0071450 tx=0x7fc7e002b870 comp rx=0 tx=0).stop 2023-12-19T18:26:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.749+0000 7fc7f607d640 1 -- 172.21.15.27:0/3361777311 shutdown_connections 2023-12-19T18:26:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.749+0000 7fc7f607d640 1 --2- 172.21.15.27:0/3361777311 >> v2:172.21.15.192:3300/0 conn(0x7fc7f015e880 0x7fc7f015ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.749+0000 7fc7f607d640 1 --2- 172.21.15.27:0/3361777311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f00cc130 0x7fc7f00cc590 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.749+0000 7fc7f607d640 1 --2- 172.21.15.27:0/3361777311 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f015acb0 0x7fc7f00cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.749+0000 7fc7f607d640 1 -- 172.21.15.27:0/3361777311 >> 172.21.15.27:0/3361777311 conn(0x7fc7f00d0aa0 msgr2=0x7fc7f00d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.749+0000 7fc7f607d640 1 -- 172.21.15.27:0/3361777311 shutdown_connections 2023-12-19T18:26:32.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.749+0000 7fc7f607d640 1 -- 172.21.15.27:0/3361777311 wait complete. 2023-12-19T18:26:32.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.751+0000 7fc7f607d640 1 Processor -- start 2023-12-19T18:26:32.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.751+0000 7fc7f607d640 1 -- start start 2023-12-19T18:26:32.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.752+0000 7fc7f607d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f00cc130 0x7fc7f014f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:32.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.752+0000 7fc7f607d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f015acb0 0x7fc7f00b4b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:32.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.752+0000 7fc7f607d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f015e880 0x7fc7f00c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:32.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.752+0000 7fc7f607d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00b6040 con 0x7fc7f015e880 2023-12-19T18:26:32.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.752+0000 7fc7f607d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00b53c0 con 0x7fc7f015acb0 2023-12-19T18:26:32.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.752+0000 7fc7f607d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f00b56a0 con 0x7fc7f00cc130 2023-12-19T18:26:32.754 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.752+0000 7fc7f587c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f015e880 0x7fc7f00c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.752+0000 7fc7f587c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f015e880 0x7fc7f00c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43840/0 (socket says 172.21.15.27:43840) 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.752+0000 7fc7f587c640 1 -- 172.21.15.27:0/1487079786 learned_addr learned my addr 172.21.15.27:0/1487079786 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.753+0000 7fc7f487a640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f015acb0 0x7fc7f00b4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.753+0000 7fc7f507b640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f00cc130 0x7fc7f014f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.753+0000 7fc7f587c640 1 -- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f00cc130 msgr2=0x7fc7f014f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.753+0000 7fc7f587c640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f00cc130 0x7fc7f014f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.753+0000 7fc7f587c640 1 -- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f015acb0 msgr2=0x7fc7f00b4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.753+0000 7fc7f587c640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f015acb0 0x7fc7f00b4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.753+0000 7fc7f587c640 1 -- 172.21.15.27:0/1487079786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7e00094d0 con 0x7fc7f015e880 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.753+0000 7fc7f587c640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f015e880 0x7fc7f00c66f0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800cd50 tx=0x7fc7e800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.753+0000 7fc7e5ffb640 1 -- 172.21.15.27:0/1487079786 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e800f680 con 0x7fc7f015e880 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.753+0000 7fc7e5ffb640 1 -- 172.21.15.27:0/1487079786 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e800f800 con 0x7fc7f015e880 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.753+0000 7fc7e5ffb640 1 -- 172.21.15.27:0/1487079786 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e800fbb0 con 0x7fc7f015e880 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.753+0000 7fc7f607d640 1 -- 172.21.15.27:0/1487079786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f00b5980 con 0x7fc7f015e880 2023-12-19T18:26:32.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.754+0000 7fc7f607d640 1 -- 172.21.15.27:0/1487079786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f00b5070 con 0x7fc7f015e880 2023-12-19T18:26:32.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.755+0000 7fc7f607d640 1 -- 172.21.15.27:0/1487079786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7f00d00f0 con 0x7fc7f015e880 2023-12-19T18:26:32.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.756+0000 7fc7e5ffb640 1 -- 172.21.15.27:0/1487079786 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc7e800b040 con 0x7fc7f015e880 2023-12-19T18:26:32.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.757+0000 7fc7e5ffb640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc7d406a430 0x7fc7d406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:32.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.757+0000 7fc7e5ffb640 1 -- 172.21.15.27:0/1487079786 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc7e8088ff0 con 0x7fc7f015e880 2023-12-19T18:26:32.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.757+0000 7fc7f507b640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc7d406a430 0x7fc7d406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:32.758 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.758+0000 7fc7f507b640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc7d406a430 0x7fc7d406c8f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec006fd0 tx=0x7fc7ec008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:32.761 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.761+0000 7fc7e5ffb640 1 -- 172.21.15.27:0/1487079786 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc7e80bbca0 con 0x7fc7f015e880 2023-12-19T18:26:32.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:32.974+0000 7fc7f607d640 1 -- 172.21.15.27:0/1487079786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fc7f00b6180 con 0x7fc7f015e880 2023-12-19T18:26:33.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:33.941+0000 7fc7e5ffb640 1 -- 172.21.15.27:0/1487079786 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v181) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc7f00b6180 con 0x7fc7f015e880 2023-12-19T18:26:33.941 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2023-12-19T18:26:33.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:33.944+0000 7fc7f607d640 1 -- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc7d406a430 msgr2=0x7fc7d406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:33.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:33.944+0000 7fc7f607d640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc7d406a430 0x7fc7d406c8f0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec006fd0 tx=0x7fc7ec008040 comp rx=0 tx=0).stop 2023-12-19T18:26:33.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:33.944+0000 7fc7f607d640 1 -- 172.21.15.27:0/1487079786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f015e880 msgr2=0x7fc7f00c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:33.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:33.944+0000 7fc7f607d640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f015e880 0x7fc7f00c66f0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x7fc7e800cd50 tx=0x7fc7e800a650 comp rx=0 tx=0).stop 2023-12-19T18:26:33.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:33.945+0000 7fc7f607d640 1 -- 172.21.15.27:0/1487079786 shutdown_connections 2023-12-19T18:26:33.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:33.945+0000 7fc7f607d640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc7d406a430 0x7fc7d406c8f0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:33.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:33.945+0000 7fc7f607d640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7f015e880 0x7fc7f00c66f0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:33.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:33.945+0000 7fc7f607d640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc7f015acb0 0x7fc7f00b4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:33.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:33.945+0000 7fc7f607d640 1 --2- 172.21.15.27:0/1487079786 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc7f00cc130 0x7fc7f014f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:33.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:33.945+0000 7fc7f607d640 1 -- 172.21.15.27:0/1487079786 >> 172.21.15.27:0/1487079786 conn(0x7fc7f00d0aa0 msgr2=0x7fc7f00b2750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:33.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:33.945+0000 7fc7f607d640 1 -- 172.21.15.27:0/1487079786 shutdown_connections 2023-12-19T18:26:33.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:33.945+0000 7fc7f607d640 1 -- 172.21.15.27:0/1487079786 wait complete. 2023-12-19T18:26:33.966 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:26:34.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.197+0000 7ffbd9def640 1 Processor -- start 2023-12-19T18:26:34.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.197+0000 7ffbd9def640 1 -- start start 2023-12-19T18:26:34.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.199+0000 7ffbd9def640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbd406ba20 0x7ffbd4165c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.199+0000 7ffbd9def640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd40adf10 con 0x7ffbd415ac50 2023-12-19T18:26:34.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.199+0000 7ffbd9def640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd40adac0 con 0x7ffbd4070fb0 2023-12-19T18:26:34.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.199+0000 7ffbd9def640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40af630 con 0x7ffbd406ba20 2023-12-19T18:26:34.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.199+0000 7ffbd8ded640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ffbd415ac50 0x7ffbd40aef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39974/0 (socket says 172.21.15.27:39974) 2023-12-19T18:26:34.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.199+0000 7ffbd8ded640 1 -- 172.21.15.27:0/4289355104 learned_addr learned my addr 172.21.15.27:0/4289355104 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:34.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd95ee640 1 --2- 172.21.15.27:0/4289355104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbd406ba20 0x7ffbd4165c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:34.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2886604413 0 0) 0x7ffbd40adf10 con 0x7ffbd415ac50 2023-12-19T18:26:34.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffbc4003520 con 0x7ffbd415ac50 2023-12-19T18:26:34.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2453764746 0 0) 0x7ffbc4003520 con 0x7ffbd415ac50 2023-12-19T18:26:34.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbd406ba20 msgr2=0x7ffbd4165c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 --2- 172.21.15.27:0/4289355104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbd406ba20 0x7ffbd4165c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 >> v1:172.21.15.192:6789/0 conn(0x7ffbd4070fb0 legacy=0x7ffbd406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd40713f0 con 0x7ffbd415ac50 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7ffbc8001670 con 0x7ffbd415ac50 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 >> v1:172.21.15.27:6789/0 conn(0x7ffbd415ac50 legacy=0x7ffbd40aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 --2- 172.21.15.27:0/4289355104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbc40039a0 0x7ffbc4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 --2- 172.21.15.27:0/4289355104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc40047c0 0x7ffbc4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 --2- 172.21.15.27:0/4289355104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbc40052a0 0x7ffbc400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40af630 con 0x7ffbc40039a0 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc400c1c0 con 0x7ffbc40052a0 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc400c380 con 0x7ffbc40047c0 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd95ee640 1 --2- 172.21.15.27:0/4289355104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbd406ba20 0x7ffbd4165c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.200+0000 7ffbd8ded640 1 --2- 172.21.15.27:0/4289355104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbc40039a0 0x7ffbc4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:34.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.201+0000 7ffbd8ded640 1 -- 172.21.15.27:0/4289355104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc40047c0 msgr2=0x7ffbc4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:34.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.201+0000 7ffbd8ded640 1 --2- 172.21.15.27:0/4289355104 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc40047c0 0x7ffbc4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.201+0000 7ffbd8ded640 1 -- 172.21.15.27:0/4289355104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbc40052a0 msgr2=0x7ffbc400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:34.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.201+0000 7ffbd8ded640 1 --2- 172.21.15.27:0/4289355104 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbc40052a0 0x7ffbc400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.201+0000 7ffbd8ded640 1 -- 172.21.15.27:0/4289355104 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc4003520 con 0x7ffbc40039a0 2023-12-19T18:26:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.202+0000 7ffbd8ded640 1 --2- 172.21.15.27:0/4289355104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbc40039a0 0x7ffbc4003dc0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c0e0 tx=0x7ffbc40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b332d32ecec4dd97 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.202+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc8005ad0 con 0x7ffbc40039a0 2023-12-19T18:26:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.202+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc8005c30 con 0x7ffbc40039a0 2023-12-19T18:26:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.202+0000 7ffbd37fe640 1 -- 172.21.15.27:0/4289355104 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc802c810 con 0x7ffbc40039a0 2023-12-19T18:26:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.203+0000 7ffbd9def640 1 -- 172.21.15.27:0/4289355104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbc40039a0 msgr2=0x7ffbc4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.203+0000 7ffbd9def640 1 --2- 172.21.15.27:0/4289355104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbc40039a0 0x7ffbc4003dc0 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7ffbc800c0e0 tx=0x7ffbc40057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.203+0000 7ffbd95ee640 1 -- 172.21.15.27:0/4289355104 reap_dead start 2023-12-19T18:26:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.203+0000 7ffbd9def640 1 -- 172.21.15.27:0/4289355104 shutdown_connections 2023-12-19T18:26:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.203+0000 7ffbd9def640 1 -- 172.21.15.27:0/4289355104 >> 172.21.15.27:0/4289355104 conn(0x7ffbd406a7d0 msgr2=0x7ffbd40ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:34.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.203+0000 7ffbd9def640 1 -- 172.21.15.27:0/4289355104 shutdown_connections 2023-12-19T18:26:34.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.203+0000 7ffbd9def640 1 -- 172.21.15.27:0/4289355104 wait complete. 2023-12-19T18:26:34.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.205+0000 7ffbd9def640 1 Processor -- start 2023-12-19T18:26:34.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.205+0000 7ffbd9def640 1 -- start start 2023-12-19T18:26:34.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.206+0000 7ffbd9def640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc40039a0 0x7ffbd413e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.206+0000 7ffbd9def640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbd414ee60 0x7ffbd413b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.206+0000 7ffbd9def640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbd40c9850 0x7ffbd40c9cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.206+0000 7ffbd9def640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40ca210 con 0x7ffbd40c9850 2023-12-19T18:26:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.206+0000 7ffbd9def640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd40b2150 con 0x7ffbd414ee60 2023-12-19T18:26:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.206+0000 7ffbd9def640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd413ecd0 con 0x7ffbc40039a0 2023-12-19T18:26:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.206+0000 7ffbd95ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbd40c9850 0x7ffbd40c9cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.207+0000 7ffbd95ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbd40c9850 0x7ffbd40c9cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43862/0 (socket says 172.21.15.27:43862) 2023-12-19T18:26:34.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.207+0000 7ffbd95ee640 1 -- 172.21.15.27:0/642033826 learned_addr learned my addr 172.21.15.27:0/642033826 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.207+0000 7ffbd8ded640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc40039a0 0x7ffbd413e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.207+0000 7ffbd95ee640 1 -- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc40039a0 msgr2=0x7ffbd413e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.207+0000 7ffbd95ee640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc40039a0 0x7ffbd413e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.207+0000 7ffbd3fff640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbd414ee60 0x7ffbd413b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:34.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.207+0000 7ffbd95ee640 1 -- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbd414ee60 msgr2=0x7ffbd413b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:34.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.207+0000 7ffbd95ee640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbd414ee60 0x7ffbd413b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.207+0000 7ffbd95ee640 1 -- 172.21.15.27:0/642033826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd413ba00 con 0x7ffbd40c9850 2023-12-19T18:26:34.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.207+0000 7ffbd8ded640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc40039a0 0x7ffbd413e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:34.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.207+0000 7ffbd3fff640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbd414ee60 0x7ffbd413b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:34.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.207+0000 7ffbd95ee640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbd40c9850 0x7ffbd40c9cd0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7ffbd406f0f0 tx=0x7ffbcc00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:34.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.208+0000 7ffbd17fa640 1 -- 172.21.15.27:0/642033826 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00dd80 con 0x7ffbd40c9850 2023-12-19T18:26:34.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.208+0000 7ffbd17fa640 1 -- 172.21.15.27:0/642033826 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc00f040 con 0x7ffbd40c9850 2023-12-19T18:26:34.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.208+0000 7ffbd17fa640 1 -- 172.21.15.27:0/642033826 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc00dee0 con 0x7ffbd40c9850 2023-12-19T18:26:34.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.208+0000 7ffbd9def640 1 -- 172.21.15.27:0/642033826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd41e2100 con 0x7ffbd40c9850 2023-12-19T18:26:34.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.208+0000 7ffbd9def640 1 -- 172.21.15.27:0/642033826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd40b23d0 con 0x7ffbd40c9850 2023-12-19T18:26:34.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.209+0000 7ffbd9def640 1 -- 172.21.15.27:0/642033826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd404e840 con 0x7ffbd40c9850 2023-12-19T18:26:34.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.210+0000 7ffbd17fa640 1 -- 172.21.15.27:0/642033826 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ffbcc003bd0 con 0x7ffbd40c9850 2023-12-19T18:26:34.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.211+0000 7ffbd17fa640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffbb806a430 0x7ffbb806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.211+0000 7ffbd17fa640 1 -- 172.21.15.27:0/642033826 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ffbcc088430 con 0x7ffbd40c9850 2023-12-19T18:26:34.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.212+0000 7ffbd8ded640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffbb806a430 0x7ffbb806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:34.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.213+0000 7ffbd8ded640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffbb806a430 0x7ffbb806c8f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8034000 tx=0x7ffbc8009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:34.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.215+0000 7ffbd17fa640 1 -- 172.21.15.27:0/642033826 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ffbcc053d60 con 0x7ffbd40c9850 2023-12-19T18:26:34.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.479+0000 7ffbd9def640 1 -- 172.21.15.27:0/642033826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ffbd40ae9f0 con 0x7ffbd40c9850 2023-12-19T18:26:34.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.481+0000 7ffbd17fa640 1 -- 172.21.15.27:0/642033826 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ffbcc057380 con 0x7ffbd40c9850 2023-12-19T18:26:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:34.481 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:26:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2023-12-19T18:26:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2023-12-19T18:26:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFl+4ruHBAAXwbpmT43SmlvNbuZj5e+bQ==", 2023-12-19T18:26:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:26:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:26:34.481 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmZyUBhAAG3kZkXvJIo7N9f1j+DPCSQ==", 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2023-12-19T18:26:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlChNOMBAAX4GV3iIGkSQdAsYu9oCPDw==", 2023-12-19T18:26:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:26:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:26:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:26:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlTmFXFhAABA0NMUxKBV1ExAhYc5RFbQ==", 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFluleFIBAAs1goFVMOX4CbjwFhg0KoBg==", 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.484 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:26:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:26:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2023-12-19T18:26:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFlInW3MxAAI4hY1GBxXYXDswKKrEyDTg==", 2023-12-19T18:26:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:26:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:26:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.485 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFlAWZ5CRAA82/2r9/sT0L9VlGHbXksQw==", 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFl1IvaGRAAqVacDDPLx+NqBOGp6MeX2w==", 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:26:34.486 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlr0BBHRAA+7gfs/RG6DtABnYPg059Tw==", 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.487 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2023-12-19T18:26:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlXZbELRAAKD0qOkhk7skfc4xcAC8Lhg==", 2023-12-19T18:26:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:26:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:26:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2023-12-19T18:26:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlF07MARAA2TJJ+5C1HIU+SVhqmKaVHA==", 2023-12-19T18:26:34.488 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:26:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:26:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2023-12-19T18:26:34.489 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlISPGEhAAdFIRRkXlLyOx/oMZ4MgL9Q==", 2023-12-19T18:26:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:26:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:26:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2023-12-19T18:26:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmNM9JRAARgS20O8zpm6H70WDv+lgGw==", 2023-12-19T18:26:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:26:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2023-12-19T18:26:34.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T18:26:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2023-12-19T18:26:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlWoZ8AhAADWvKVzeDhDGHtYNUFIOWsA==", 2023-12-19T18:26:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:26:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:26:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:26:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:26:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2023-12-19T18:26:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFllWFVMhAAD9ebDOppHTODVQjg6ppP3g==", 2023-12-19T18:26:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:26:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:26:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:26:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:26:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2023-12-19T18:26:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlwbIQDRAAt/h55zBNCEId0vkYSUQ3TQ==", 2023-12-19T18:26:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:26:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:26:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:26:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:26:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2023-12-19T18:26:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB53YFloCb6CxAAmZQyz5XOEqDOOiROJHLzxQ==", 2023-12-19T18:26:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:26:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:26:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:26:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2023-12-19T18:26:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFltBhxDxAAiAS9CYPY86IMmJ7NGgA2lw==", 2023-12-19T18:26:34.494 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:26:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:26:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:34.495 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2023-12-19T18:26:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlUnAhBBAAi++W9XK3o3Y/185hB3LuIg==", 2023-12-19T18:26:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:26:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:26:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:26:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:26:34.496 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:34.497 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:26:34.497 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:26:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.483+0000 7ffbd9def640 1 -- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffbb806a430 msgr2=0x7ffbb806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.483+0000 7ffbd9def640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffbb806a430 0x7ffbb806c8f0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ffbc8034000 tx=0x7ffbc8009030 comp rx=0 tx=0).stop 2023-12-19T18:26:34.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.483+0000 7ffbd9def640 1 -- 172.21.15.27:0/642033826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbd40c9850 msgr2=0x7ffbd40c9cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.483+0000 7ffbd9def640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbd40c9850 0x7ffbd40c9cd0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7ffbd406f0f0 tx=0x7ffbcc00d4a0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.483+0000 7ffbd9def640 1 -- 172.21.15.27:0/642033826 shutdown_connections 2023-12-19T18:26:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.483+0000 7ffbd9def640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ffbb806a430 0x7ffbb806c8f0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.483+0000 7ffbd9def640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbd40c9850 0x7ffbd40c9cd0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.483+0000 7ffbd9def640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ffbd414ee60 0x7ffbd413b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.483+0000 7ffbd9def640 1 --2- 172.21.15.27:0/642033826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ffbc40039a0 0x7ffbd413e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.483+0000 7ffbd9def640 1 -- 172.21.15.27:0/642033826 >> 172.21.15.27:0/642033826 conn(0x7ffbd406a7d0 msgr2=0x7ffbd4150750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:34.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.484+0000 7ffbd9def640 1 -- 172.21.15.27:0/642033826 shutdown_connections 2023-12-19T18:26:34.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.484+0000 7ffbd9def640 1 -- 172.21.15.27:0/642033826 wait complete. 2023-12-19T18:26:34.504 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:26:34.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.723+0000 7efe59326640 1 Processor -- start 2023-12-19T18:26:34.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.723+0000 7efe59326640 1 -- start start 2023-12-19T18:26:34.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.725+0000 7efe59326640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efe54154030 0x7efe540cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.725+0000 7efe59326640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe540af630 0x7efe540afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.725+0000 7efe59326640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe540cc190 con 0x7efe5415d620 2023-12-19T18:26:34.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.725+0000 7efe59326640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe540b0000 con 0x7efe540af630 2023-12-19T18:26:34.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.725+0000 7efe59326640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe540cb2a0 con 0x7efe54154030 2023-12-19T18:26:34.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.726+0000 7efe4bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe540af630 0x7efe540afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:34.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.726+0000 7efe4bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efe540af630 0x7efe540afac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43878/0 (socket says 172.21.15.27:43878) 2023-12-19T18:26:34.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.726+0000 7efe4bfff640 1 -- 172.21.15.27:0/3822365471 learned_addr learned my addr 172.21.15.27:0/3822365471 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:34.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.726+0000 7efe53fff640 1 --2- 172.21.15.27:0/3822365471 >> v2:172.21.15.192:3300/0 conn(0x7efe54154030 0x7efe540cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:34.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.726+0000 7efe4bfff640 1 -- 172.21.15.27:0/3822365471 >> v2:172.21.15.192:3300/0 conn(0x7efe54154030 msgr2=0x7efe540cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:34.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.726+0000 7efe4bfff640 1 --2- 172.21.15.27:0/3822365471 >> v2:172.21.15.192:3300/0 conn(0x7efe54154030 0x7efe540cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.726+0000 7efe4bfff640 1 -- 172.21.15.27:0/3822365471 >> v1:172.21.15.192:6789/0 conn(0x7efe5415d620 legacy=0x7efe5415da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:34.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.726+0000 7efe4bfff640 1 -- 172.21.15.27:0/3822365471 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe540cb580 con 0x7efe540af630 2023-12-19T18:26:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.727+0000 7efe4bfff640 1 --2- 172.21.15.27:0/3822365471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe540af630 0x7efe540afac0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009870 tx=0x7efe3c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=707f6f733adfcbe1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.728+0000 7efe537fe640 1 -- 172.21.15.27:0/3822365471 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c031870 con 0x7efe540af630 2023-12-19T18:26:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.728+0000 7efe537fe640 1 -- 172.21.15.27:0/3822365471 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c0319d0 con 0x7efe540af630 2023-12-19T18:26:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.728+0000 7efe537fe640 1 -- 172.21.15.27:0/3822365471 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c031d60 con 0x7efe540af630 2023-12-19T18:26:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.728+0000 7efe59326640 1 -- 172.21.15.27:0/3822365471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe540af630 msgr2=0x7efe540afac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.728+0000 7efe59326640 1 --2- 172.21.15.27:0/3822365471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe540af630 0x7efe540afac0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009870 tx=0x7efe3c02b910 comp rx=0 tx=0).stop 2023-12-19T18:26:34.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.729+0000 7efe59326640 1 -- 172.21.15.27:0/3822365471 shutdown_connections 2023-12-19T18:26:34.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.729+0000 7efe59326640 1 --2- 172.21.15.27:0/3822365471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe540af630 0x7efe540afac0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.729+0000 7efe59326640 1 --2- 172.21.15.27:0/3822365471 >> v2:172.21.15.192:3300/0 conn(0x7efe54154030 0x7efe540cd4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.729+0000 7efe59326640 1 -- 172.21.15.27:0/3822365471 >> 172.21.15.27:0/3822365471 conn(0x7efe54069c70 msgr2=0x7efe5406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:34.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.729+0000 7efe59326640 1 -- 172.21.15.27:0/3822365471 shutdown_connections 2023-12-19T18:26:34.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.729+0000 7efe59326640 1 -- 172.21.15.27:0/3822365471 wait complete. 2023-12-19T18:26:34.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.731+0000 7efe59326640 1 Processor -- start 2023-12-19T18:26:34.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.731+0000 7efe59326640 1 -- start start 2023-12-19T18:26:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.732+0000 7efe59326640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe540af630 0x7efe5406f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.732+0000 7efe59326640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe54154030 0x7efe54143760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.732+0000 7efe59326640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe5415d620 0x7efe5415c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.732+0000 7efe59326640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe54070710 con 0x7efe54154030 2023-12-19T18:26:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.732+0000 7efe59326640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5406f860 con 0x7efe540af630 2023-12-19T18:26:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.732+0000 7efe59326640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe54143ca0 con 0x7efe5415d620 2023-12-19T18:26:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.732+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe54154030 0x7efe54143760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.732+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe54154030 0x7efe54143760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43890/0 (socket says 172.21.15.27:43890) 2023-12-19T18:26:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.732+0000 7efe4bfff640 1 -- 172.21.15.27:0/2085481877 learned_addr learned my addr 172.21.15.27:0/2085481877 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.733+0000 7efe58b25640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe5415d620 0x7efe5415c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.733+0000 7efe53fff640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe540af630 0x7efe5406f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.733+0000 7efe4bfff640 1 -- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe5415d620 msgr2=0x7efe5415c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:34.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.733+0000 7efe4bfff640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe5415d620 0x7efe5415c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.733+0000 7efe4bfff640 1 -- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe540af630 msgr2=0x7efe5406f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.733+0000 7efe4bfff640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe540af630 0x7efe5406f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.733+0000 7efe4bfff640 1 -- 172.21.15.27:0/2085481877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe3c0094d0 con 0x7efe54154030 2023-12-19T18:26:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.733+0000 7efe58b25640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe5415d620 0x7efe5415c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.733+0000 7efe4bfff640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe54154030 0x7efe54143760 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009870 tx=0x7efe3c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.733+0000 7efe517fa640 1 -- 172.21.15.27:0/2085481877 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c03ac60 con 0x7efe54154030 2023-12-19T18:26:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.733+0000 7efe517fa640 1 -- 172.21.15.27:0/2085481877 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe3c03adc0 con 0x7efe54154030 2023-12-19T18:26:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.733+0000 7efe517fa640 1 -- 172.21.15.27:0/2085481877 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe3c044610 con 0x7efe54154030 2023-12-19T18:26:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.733+0000 7efe59326640 1 -- 172.21.15.27:0/2085481877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5406fa90 con 0x7efe54154030 2023-12-19T18:26:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.734+0000 7efe59326640 1 -- 172.21.15.27:0/2085481877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe54070050 con 0x7efe54154030 2023-12-19T18:26:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.735+0000 7efe59326640 1 -- 172.21.15.27:0/2085481877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe54070850 con 0x7efe54154030 2023-12-19T18:26:34.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.736+0000 7efe517fa640 1 -- 172.21.15.27:0/2085481877 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efe3c04d450 con 0x7efe54154030 2023-12-19T18:26:34.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.737+0000 7efe517fa640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efe3406a430 0x7efe3406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:34.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.737+0000 7efe517fa640 1 -- 172.21.15.27:0/2085481877 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efe3c02d040 con 0x7efe54154030 2023-12-19T18:26:34.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.737+0000 7efe53fff640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efe3406a430 0x7efe3406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:34.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.739+0000 7efe53fff640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efe3406a430 0x7efe3406c8f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7efe44006fd0 tx=0x7efe44008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:34.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.741+0000 7efe517fa640 1 -- 172.21.15.27:0/2085481877 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efe54070850 con 0x7efe54154030 2023-12-19T18:26:34.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.936+0000 7efe59326640 1 -- 172.21.15.27:0/2085481877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe54070850 con 0x7efe54154030 2023-12-19T18:26:34.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.938+0000 7efe517fa640 1 -- 172.21.15.27:0/2085481877 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v181) v1 ==== 74+0+12400 (secure 0 0 0) 0x7efe54070850 con 0x7efe54154030 2023-12-19T18:26:34.938 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:34.938 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":181,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:26:33.926681+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":168,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:26:34.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.941+0000 7efe59326640 1 -- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efe3406a430 msgr2=0x7efe3406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:34.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.941+0000 7efe59326640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efe3406a430 0x7efe3406c8f0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7efe44006fd0 tx=0x7efe44008040 comp rx=0 tx=0).stop 2023-12-19T18:26:34.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.941+0000 7efe59326640 1 -- 172.21.15.27:0/2085481877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe54154030 msgr2=0x7efe54143760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:34.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.941+0000 7efe59326640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe54154030 0x7efe54143760 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009870 tx=0x7efe3c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.941+0000 7efe59326640 1 -- 172.21.15.27:0/2085481877 shutdown_connections 2023-12-19T18:26:34.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.941+0000 7efe59326640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efe3406a430 0x7efe3406c8f0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.941+0000 7efe59326640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efe5415d620 0x7efe5415c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.941+0000 7efe59326640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe54154030 0x7efe54143760 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.941+0000 7efe59326640 1 --2- 172.21.15.27:0/2085481877 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efe540af630 0x7efe5406f320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:34.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.941+0000 7efe59326640 1 -- 172.21.15.27:0/2085481877 >> 172.21.15.27:0/2085481877 conn(0x7efe54069c70 msgr2=0x7efe540cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:34.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.941+0000 7efe59326640 1 -- 172.21.15.27:0/2085481877 shutdown_connections 2023-12-19T18:26:34.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:34.941+0000 7efe59326640 1 -- 172.21.15.27:0/2085481877 wait complete. 2023-12-19T18:26:34.962 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:26:34.963 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:26:35.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.191+0000 7f4ba1db5640 1 Processor -- start 2023-12-19T18:26:35.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.192+0000 7f4ba1db5640 1 -- start start 2023-12-19T18:26:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.193+0000 7f4ba1db5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b9c15cf80 0x7f4b9c0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.193+0000 7f4ba1db5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9c0d0e10 con 0x7f4b9c0b8a60 2023-12-19T18:26:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.193+0000 7f4ba1db5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b9c0d1c70 con 0x7f4b9c0b85e0 2023-12-19T18:26:35.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.193+0000 7f4ba1db5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c15d360 con 0x7f4b9c15cf80 2023-12-19T18:26:35.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.194+0000 7f4ba15b4640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4b9c0b8a60 0x7f4b9c0b5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39988/0 (socket says 172.21.15.27:39988) 2023-12-19T18:26:35.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.194+0000 7f4ba15b4640 1 -- 172.21.15.27:0/290826909 learned_addr learned my addr 172.21.15.27:0/290826909 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.194+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482312765 0 0) 0x7f4b9c0d0e10 con 0x7f4b9c0b8a60 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.194+0000 7f4ba0db3640 1 --2- 172.21.15.27:0/290826909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b9c15cf80 0x7f4b9c0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.194+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4b8c003520 con 0x7f4b9c0b8a60 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4224118121 0 0) 0x7f4b8c003520 con 0x7f4b9c0b8a60 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b9c15cf80 msgr2=0x7f4b9c0d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4b9b7fe640 1 --2- 172.21.15.27:0/290826909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b9c15cf80 0x7f4b9c0d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 >> v1:172.21.15.192:6789/0 conn(0x7f4b9c0b85e0 legacy=0x7f4b9c0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9c0d1060 con 0x7f4b9c0b8a60 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f4b940019f0 con 0x7f4b9c0b8a60 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 >> v1:172.21.15.27:6789/0 conn(0x7f4b9c0b8a60 legacy=0x7f4b9c0b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4b9b7fe640 1 --2- 172.21.15.27:0/290826909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b8c0039a0 0x7f4b8c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4b9b7fe640 1 --2- 172.21.15.27:0/290826909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b8c0047c0 0x7f4b8c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4b9b7fe640 1 --2- 172.21.15.27:0/290826909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b8c005180 0x7f4b8c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c15d360 con 0x7f4b8c0039a0 2023-12-19T18:26:35.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c00c030 con 0x7f4b8c005180 2023-12-19T18:26:35.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b8c00c1f0 con 0x7f4b8c0047c0 2023-12-19T18:26:35.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4ba0db3640 1 --2- 172.21.15.27:0/290826909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b9c15cf80 0x7f4b9c0d2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:26:35.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.195+0000 7f4ba0db3640 1 --2- 172.21.15.27:0/290826909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b8c0039a0 0x7f4b8c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.196+0000 7f4ba0db3640 1 -- 172.21.15.27:0/290826909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b8c0047c0 msgr2=0x7f4b8c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:35.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.196+0000 7f4ba0db3640 1 --2- 172.21.15.27:0/290826909 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b8c0047c0 0x7f4b8c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.196+0000 7f4ba15b4640 1 --2- 172.21.15.27:0/290826909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b8c005180 0x7f4b8c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.196+0000 7f4ba0db3640 1 -- 172.21.15.27:0/290826909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b8c005180 msgr2=0x7f4b8c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:35.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.196+0000 7f4ba0db3640 1 --2- 172.21.15.27:0/290826909 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b8c005180 0x7f4b8c00b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.196+0000 7f4ba0db3640 1 -- 172.21.15.27:0/290826909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c003520 con 0x7f4b8c0039a0 2023-12-19T18:26:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.197+0000 7f4ba0db3640 1 --2- 172.21.15.27:0/290826909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b8c0039a0 0x7f4b8c003dc0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f4b90006fd0 tx=0x7f4b8c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0efa974a9004b72 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.197+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90033630 con 0x7f4b8c0039a0 2023-12-19T18:26:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.197+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b90033790 con 0x7f4b8c0039a0 2023-12-19T18:26:35.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.197+0000 7f4b9b7fe640 1 -- 172.21.15.27:0/290826909 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b90033b20 con 0x7f4b8c0039a0 2023-12-19T18:26:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.198+0000 7f4ba1db5640 1 -- 172.21.15.27:0/290826909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b8c0039a0 msgr2=0x7f4b8c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.198+0000 7f4ba1db5640 1 --2- 172.21.15.27:0/290826909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b8c0039a0 0x7f4b8c003dc0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f4b90006fd0 tx=0x7f4b8c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.198+0000 7f4ba15b4640 1 -- 172.21.15.27:0/290826909 reap_dead start 2023-12-19T18:26:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.198+0000 7f4ba1db5640 1 -- 172.21.15.27:0/290826909 shutdown_connections 2023-12-19T18:26:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.198+0000 7f4ba1db5640 1 -- 172.21.15.27:0/290826909 >> 172.21.15.27:0/290826909 conn(0x7f4b9c072e20 msgr2=0x7f4b9c073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.198+0000 7f4ba1db5640 1 -- 172.21.15.27:0/290826909 shutdown_connections 2023-12-19T18:26:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.198+0000 7f4ba1db5640 1 -- 172.21.15.27:0/290826909 wait complete. 2023-12-19T18:26:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.200+0000 7f4ba1db5640 1 Processor -- start 2023-12-19T18:26:35.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.200+0000 7f4ba1db5640 1 -- start start 2023-12-19T18:26:35.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.201+0000 7f4ba1db5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b8c0039a0 0x7f4b9c156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.201+0000 7f4ba1db5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b9c0cddd0 0x7f4b9c1f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.201+0000 7f4ba1db5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b9c1f1d00 0x7f4b9c15ea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.201+0000 7f4ba1db5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c1f5fc0 con 0x7f4b9c1f1d00 2023-12-19T18:26:35.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.201+0000 7f4ba1db5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c157710 con 0x7f4b8c0039a0 2023-12-19T18:26:35.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.201+0000 7f4ba1db5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c0ce250 con 0x7f4b9c0cddd0 2023-12-19T18:26:35.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.201+0000 7f4ba15b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b9c1f1d00 0x7f4b9c15ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.201+0000 7f4ba15b4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b9c1f1d00 0x7f4b9c15ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43906/0 (socket says 172.21.15.27:43906) 2023-12-19T18:26:35.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.201+0000 7f4ba15b4640 1 -- 172.21.15.27:0/1073535670 learned_addr learned my addr 172.21.15.27:0/1073535670 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:35.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.202+0000 7f4ba15b4640 1 -- 172.21.15.27:0/1073535670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b9c0cddd0 msgr2=0x7f4b9c1f63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.202+0000 7f4ba15b4640 1 --2- 172.21.15.27:0/1073535670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b9c0cddd0 0x7f4b9c1f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.202+0000 7f4ba15b4640 1 -- 172.21.15.27:0/1073535670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b8c0039a0 msgr2=0x7f4b9c156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.202+0000 7f4ba0db3640 1 --2- 172.21.15.27:0/1073535670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b8c0039a0 0x7f4b9c156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.202+0000 7f4ba15b4640 1 --2- 172.21.15.27:0/1073535670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b8c0039a0 0x7f4b9c156ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.202+0000 7f4ba15b4640 1 -- 172.21.15.27:0/1073535670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b9c1f5c00 con 0x7f4b9c1f1d00 2023-12-19T18:26:35.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.202+0000 7f4ba15b4640 1 --2- 172.21.15.27:0/1073535670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b9c1f1d00 0x7f4b9c15ea90 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0d2c50 tx=0x7f4b9400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:35.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.202+0000 7f4b997fa640 1 -- 172.21.15.27:0/1073535670 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94004d60 con 0x7f4b9c1f1d00 2023-12-19T18:26:35.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.202+0000 7f4b997fa640 1 -- 172.21.15.27:0/1073535670 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b9400bb80 con 0x7f4b9c1f1d00 2023-12-19T18:26:35.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.202+0000 7f4b997fa640 1 -- 172.21.15.27:0/1073535670 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b94007110 con 0x7f4b9c1f1d00 2023-12-19T18:26:35.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.202+0000 7f4ba1db5640 1 -- 172.21.15.27:0/1073535670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9c157150 con 0x7f4b9c1f1d00 2023-12-19T18:26:35.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.202+0000 7f4ba1db5640 1 -- 172.21.15.27:0/1073535670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9c1f6920 con 0x7f4b9c1f1d00 2023-12-19T18:26:35.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.204+0000 7f4ba1db5640 1 -- 172.21.15.27:0/1073535670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9c04e840 con 0x7f4b9c1f1d00 2023-12-19T18:26:35.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.205+0000 7f4b997fa640 1 -- 172.21.15.27:0/1073535670 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4b94007270 con 0x7f4b9c1f1d00 2023-12-19T18:26:35.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.206+0000 7f4b997fa640 1 --2- 172.21.15.27:0/1073535670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4b8006a430 0x7f4b8006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.206+0000 7f4b997fa640 1 -- 172.21.15.27:0/1073535670 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f4b94088970 con 0x7f4b9c1f1d00 2023-12-19T18:26:35.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.206+0000 7f4ba0db3640 1 --2- 172.21.15.27:0/1073535670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4b8006a430 0x7f4b8006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.207+0000 7f4ba0db3640 1 --2- 172.21.15.27:0/1073535670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4b8006a430 0x7f4b8006c8f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4b90036000 tx=0x7f4b90035040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:35.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.210+0000 7f4b997fa640 1 -- 172.21.15.27:0/1073535670 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4b940542b0 con 0x7f4b9c1f1d00 2023-12-19T18:26:35.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.403+0000 7f4ba1db5640 1 -- 172.21.15.27:0/1073535670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4b9c0b7080 con 0x7f4b9c1f1d00 2023-12-19T18:26:35.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.404+0000 7f4b997fa640 1 -- 172.21.15.27:0/1073535670 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v181) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4b940578d0 con 0x7f4b9c1f1d00 2023-12-19T18:26:35.404 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:26:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.407+0000 7f4ba1db5640 1 -- 172.21.15.27:0/1073535670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4b8006a430 msgr2=0x7f4b8006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.407+0000 7f4ba1db5640 1 --2- 172.21.15.27:0/1073535670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4b8006a430 0x7f4b8006c8f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f4b90036000 tx=0x7f4b90035040 comp rx=0 tx=0).stop 2023-12-19T18:26:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.407+0000 7f4ba1db5640 1 -- 172.21.15.27:0/1073535670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b9c1f1d00 msgr2=0x7f4b9c15ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.407+0000 7f4ba1db5640 1 --2- 172.21.15.27:0/1073535670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b9c1f1d00 0x7f4b9c15ea90 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c0d2c50 tx=0x7f4b9400b4a0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.407+0000 7f4ba1db5640 1 -- 172.21.15.27:0/1073535670 shutdown_connections 2023-12-19T18:26:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.407+0000 7f4ba1db5640 1 --2- 172.21.15.27:0/1073535670 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4b8006a430 0x7f4b8006c8f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.407+0000 7f4ba1db5640 1 --2- 172.21.15.27:0/1073535670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b9c1f1d00 0x7f4b9c15ea90 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.407+0000 7f4ba1db5640 1 --2- 172.21.15.27:0/1073535670 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4b9c0cddd0 0x7f4b9c1f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.407+0000 7f4ba1db5640 1 --2- 172.21.15.27:0/1073535670 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4b8c0039a0 0x7f4b9c156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.407+0000 7f4ba1db5640 1 -- 172.21.15.27:0/1073535670 >> 172.21.15.27:0/1073535670 conn(0x7f4b9c072e20 msgr2=0x7f4b9c0b75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.407+0000 7f4ba1db5640 1 -- 172.21.15.27:0/1073535670 shutdown_connections 2023-12-19T18:26:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.407+0000 7f4ba1db5640 1 -- 172.21.15.27:0/1073535670 wait complete. 2023-12-19T18:26:35.427 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:26:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.660+0000 7f76aa30b640 1 Processor -- start 2023-12-19T18:26:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.660+0000 7f76aa30b640 1 -- start start 2023-12-19T18:26:35.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.662+0000 7f76aa30b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76a4070fb0 0x7f76a406b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.662+0000 7f76aa30b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a40adf10 con 0x7f76a406b850 2023-12-19T18:26:35.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.662+0000 7f76aa30b640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76a40adac0 con 0x7f76a415ac50 2023-12-19T18:26:35.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.662+0000 7f76aa30b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40af630 con 0x7f76a4070fb0 2023-12-19T18:26:35.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.663+0000 7f76a9b0a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f76a406b850 0x7f76a406f650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39996/0 (socket says 172.21.15.27:39996) 2023-12-19T18:26:35.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.663+0000 7f76a9b0a640 1 -- 172.21.15.27:0/1303863969 learned_addr learned my addr 172.21.15.27:0/1303863969 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:35.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.663+0000 7f76a8b08640 1 --2- 172.21.15.27:0/1303863969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76a4070fb0 0x7f76a406b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.663+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1358870718 0 0) 0x7f76a40adf10 con 0x7f76a406b850 2023-12-19T18:26:35.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.663+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7690003520 con 0x7f76a406b850 2023-12-19T18:26:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2335584268 0 0) 0x7f7690003520 con 0x7f76a406b850 2023-12-19T18:26:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76a4070fb0 msgr2=0x7f76a406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f769bfff640 1 --2- 172.21.15.27:0/1303863969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76a4070fb0 0x7f76a406b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 >> v1:172.21.15.192:6789/0 conn(0x7f76a415ac50 legacy=0x7f76a40aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a40713f0 con 0x7f76a406b850 2023-12-19T18:26:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f769c0019f0 con 0x7f76a406b850 2023-12-19T18:26:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 >> v1:172.21.15.27:6789/0 conn(0x7f76a406b850 legacy=0x7f76a406f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f769bfff640 1 --2- 172.21.15.27:0/1303863969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76900039a0 0x7f7690003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f769bfff640 1 --2- 172.21.15.27:0/1303863969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76a4070fb0 0x7f76900047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f769bfff640 1 --2- 172.21.15.27:0/1303863969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7690004d00 0x7f7690005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40af630 con 0x7f76900039a0 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769000bb10 con 0x7f7690004d00 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f769000bca0 con 0x7f76a4070fb0 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f76a9309640 1 --2- 172.21.15.27:0/1303863969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76900039a0 0x7f7690003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.664+0000 7f76a8b08640 1 --2- 172.21.15.27:0/1303863969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76a4070fb0 0x7f76900047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.665+0000 7f76a9b0a640 1 --2- 172.21.15.27:0/1303863969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7690004d00 0x7f7690005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.665+0000 7f76a9309640 1 -- 172.21.15.27:0/1303863969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76a4070fb0 msgr2=0x7f76900047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.665+0000 7f76a9309640 1 --2- 172.21.15.27:0/1303863969 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76a4070fb0 0x7f76900047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.665+0000 7f76a9309640 1 -- 172.21.15.27:0/1303863969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7690004d00 msgr2=0x7f7690005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.665+0000 7f76a9309640 1 --2- 172.21.15.27:0/1303863969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7690004d00 0x7f7690005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.665+0000 7f76a9309640 1 -- 172.21.15.27:0/1303863969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690003520 con 0x7f76900039a0 2023-12-19T18:26:35.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.665+0000 7f76a9b0a640 1 --2- 172.21.15.27:0/1303863969 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7690004d00 0x7f7690005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:35.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.666+0000 7f76a9309640 1 --2- 172.21.15.27:0/1303863969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76900039a0 0x7f7690003d80 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f76a0001890 tx=0x7f76900057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4a6d37276707346 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.666+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0033630 con 0x7f76900039a0 2023-12-19T18:26:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.666+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a00337b0 con 0x7f76900039a0 2023-12-19T18:26:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.666+0000 7f769bfff640 1 -- 172.21.15.27:0/1303863969 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a0033b60 con 0x7f76900039a0 2023-12-19T18:26:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.667+0000 7f76aa30b640 1 -- 172.21.15.27:0/1303863969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76900039a0 msgr2=0x7f7690003d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.667+0000 7f76aa30b640 1 --2- 172.21.15.27:0/1303863969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76900039a0 0x7f7690003d80 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f76a0001890 tx=0x7f76900057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.667+0000 7f76a9b0a640 1 -- 172.21.15.27:0/1303863969 reap_dead start 2023-12-19T18:26:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.667+0000 7f76aa30b640 1 -- 172.21.15.27:0/1303863969 shutdown_connections 2023-12-19T18:26:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.667+0000 7f76aa30b640 1 -- 172.21.15.27:0/1303863969 >> 172.21.15.27:0/1303863969 conn(0x7f76a406a7d0 msgr2=0x7f76a40ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.667+0000 7f76aa30b640 1 -- 172.21.15.27:0/1303863969 shutdown_connections 2023-12-19T18:26:35.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.667+0000 7f76aa30b640 1 -- 172.21.15.27:0/1303863969 wait complete. 2023-12-19T18:26:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.669+0000 7f76aa30b640 1 Processor -- start 2023-12-19T18:26:35.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.670+0000 7f76aa30b640 1 -- start start 2023-12-19T18:26:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.671+0000 7f76aa30b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76900039a0 0x7f76a40c9fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.671+0000 7f76aa30b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a414af60 0x7f76a414a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.671+0000 7f76aa30b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a41f6280 0x7f76a40ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.671+0000 7f76aa30b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a40ca500 con 0x7f76a414af60 2023-12-19T18:26:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.671+0000 7f76aa30b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76a41f5df0 con 0x7f76a41f6280 2023-12-19T18:26:35.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.671+0000 7f76aa30b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76a41f5ac0 con 0x7f76900039a0 2023-12-19T18:26:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.671+0000 7f76a8b08640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a414af60 0x7f76a414a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.671+0000 7f76a8b08640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a414af60 0x7f76a414a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43920/0 (socket says 172.21.15.27:43920) 2023-12-19T18:26:35.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.671+0000 7f76a8b08640 1 -- 172.21.15.27:0/1324381528 learned_addr learned my addr 172.21.15.27:0/1324381528 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:35.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.672+0000 7f76a9309640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76900039a0 0x7f76a40c9fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.672+0000 7f76a9b0a640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a41f6280 0x7f76a40ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.672+0000 7f76a8b08640 1 -- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76900039a0 msgr2=0x7f76a40c9fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.672+0000 7f76a8b08640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76900039a0 0x7f76a40c9fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.672+0000 7f76a8b08640 1 -- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a41f6280 msgr2=0x7f76a40ac8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.672+0000 7f76a8b08640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a41f6280 0x7f76a40ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.672+0000 7f76a8b08640 1 -- 172.21.15.27:0/1324381528 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a415d420 con 0x7f76a414af60 2023-12-19T18:26:35.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.672+0000 7f76a8b08640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a414af60 0x7f76a414a720 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f7694011600 tx=0x7f7694011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:35.673 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.672+0000 7f7699ffb640 1 -- 172.21.15.27:0/1324381528 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769400f920 con 0x7f76a414af60 2023-12-19T18:26:35.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.672+0000 7f7699ffb640 1 -- 172.21.15.27:0/1324381528 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f769400fa80 con 0x7f76a414af60 2023-12-19T18:26:35.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.672+0000 7f7699ffb640 1 -- 172.21.15.27:0/1324381528 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f769400fdf0 con 0x7f76a414af60 2023-12-19T18:26:35.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.673+0000 7f76aa30b640 1 -- 172.21.15.27:0/1324381528 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76a40ca730 con 0x7f76a414af60 2023-12-19T18:26:35.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.673+0000 7f76aa30b640 1 -- 172.21.15.27:0/1324381528 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76a40cacf0 con 0x7f76a414af60 2023-12-19T18:26:35.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.674+0000 7f76aa30b640 1 -- 172.21.15.27:0/1324381528 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76a404e840 con 0x7f76a414af60 2023-12-19T18:26:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.675+0000 7f7699ffb640 1 -- 172.21.15.27:0/1324381528 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7694015460 con 0x7f76a414af60 2023-12-19T18:26:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.676+0000 7f7699ffb640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f768806a430 0x7f768806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:35.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.676+0000 7f7699ffb640 1 -- 172.21.15.27:0/1324381528 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7694088a00 con 0x7f76a414af60 2023-12-19T18:26:35.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.676+0000 7f76a9309640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f768806a430 0x7f768806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:35.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.678+0000 7f76a9309640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f768806a430 0x7f768806c8f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f76a0036000 tx=0x7f76a0031000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:35.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.680+0000 7f7699ffb640 1 -- 172.21.15.27:0/1324381528 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7694054330 con 0x7f76a414af60 2023-12-19T18:26:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.876+0000 7f76aa30b640 1 -- 172.21.15.27:0/1324381528 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f76a406f430 con 0x7f76a414af60 2023-12-19T18:26:35.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.877+0000 7f7699ffb640 1 -- 172.21.15.27:0/1324381528 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v181) v1 ==== 84+0+65 (secure 0 0 0) 0x7f7694057950 con 0x7f76a414af60 2023-12-19T18:26:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:35.877 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-12-19T18:26:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:26:35.877 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-12-19T18:26:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-12-19T18:26:35.878 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:35.878 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-12-19T18:26:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.880+0000 7f76aa30b640 1 -- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f768806a430 msgr2=0x7f768806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.880+0000 7f76aa30b640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f768806a430 0x7f768806c8f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f76a0036000 tx=0x7f76a0031000 comp rx=0 tx=0).stop 2023-12-19T18:26:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.880+0000 7f76aa30b640 1 -- 172.21.15.27:0/1324381528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a414af60 msgr2=0x7f76a414a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.880+0000 7f76aa30b640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a414af60 0x7f76a414a720 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f7694011600 tx=0x7f7694011ac0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.880+0000 7f76aa30b640 1 -- 172.21.15.27:0/1324381528 shutdown_connections 2023-12-19T18:26:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.880+0000 7f76aa30b640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f768806a430 0x7f768806c8f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.880+0000 7f76aa30b640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f76a41f6280 0x7f76a40ac8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.880+0000 7f76aa30b640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76a414af60 0x7f76a414a720 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.880+0000 7f76aa30b640 1 --2- 172.21.15.27:0/1324381528 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f76900039a0 0x7f76a40c9fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.880+0000 7f76aa30b640 1 -- 172.21.15.27:0/1324381528 >> 172.21.15.27:0/1324381528 conn(0x7f76a406a7d0 msgr2=0x7f76a41579c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.880+0000 7f76aa30b640 1 -- 172.21.15.27:0/1324381528 shutdown_connections 2023-12-19T18:26:35.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:35.880+0000 7f76aa30b640 1 -- 172.21.15.27:0/1324381528 wait complete. 2023-12-19T18:26:35.898 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:26:35.899 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:26:36.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.134+0000 7fe7b9a4a640 1 Processor -- start 2023-12-19T18:26:36.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.134+0000 7fe7b9a4a640 1 -- start start 2023-12-19T18:26:36.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.136+0000 7fe7b9a4a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7b40cddc0 0x7fe7b415c980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:36.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.136+0000 7fe7b9a4a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b40d0400 con 0x7fe7b40b0060 2023-12-19T18:26:36.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.136+0000 7fe7b9a4a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7b4161d00 con 0x7fe7b4161870 2023-12-19T18:26:36.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.136+0000 7fe7b9a4a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40cf7f0 con 0x7fe7b40cddc0 2023-12-19T18:26:36.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.137+0000 7fe7b2ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe7b40b0060 0x7fe7b40cefe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40004/0 (socket says 172.21.15.27:40004) 2023-12-19T18:26:36.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.137+0000 7fe7b2ffd640 1 -- 172.21.15.27:0/2618412195 learned_addr learned my addr 172.21.15.27:0/2618412195 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:36.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.137+0000 7fe7b37fe640 1 --2- 172.21.15.27:0/2618412195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7b40cddc0 0x7fe7b415c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:36.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.137+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2137166394 0 0) 0x7fe7b40d0400 con 0x7fe7b40b0060 2023-12-19T18:26:36.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.137+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a0003520 con 0x7fe7b40b0060 2023-12-19T18:26:36.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.137+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3288744684 0 0) 0x7fe7a0003520 con 0x7fe7b40b0060 2023-12-19T18:26:36.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.137+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7b40cddc0 msgr2=0x7fe7b415c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:36.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.137+0000 7fe7b1ffb640 1 --2- 172.21.15.27:0/2618412195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7b40cddc0 0x7fe7b415c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:36.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.138+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 >> v1:172.21.15.192:6789/0 conn(0x7fe7b4161870 legacy=0x7fe7b40cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:36.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.138+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b40cfad0 con 0x7fe7b40b0060 2023-12-19T18:26:36.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.138+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe7a40017f0 con 0x7fe7b40b0060 2023-12-19T18:26:36.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.138+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 >> v1:172.21.15.27:6789/0 conn(0x7fe7b40b0060 legacy=0x7fe7b40cefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:36.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.138+0000 7fe7b1ffb640 1 --2- 172.21.15.27:0/2618412195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a00039a0 0x7fe7a0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:36.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.138+0000 7fe7b1ffb640 1 --2- 172.21.15.27:0/2618412195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00047c0 0x7fe7a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:36.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.138+0000 7fe7b1ffb640 1 --2- 172.21.15.27:0/2618412195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7b40cddc0 0x7fe7a0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:36.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.138+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40cf7f0 con 0x7fe7a00047c0 2023-12-19T18:26:36.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.138+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a000ba60 con 0x7fe7a00039a0 2023-12-19T18:26:36.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.138+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a000bc20 con 0x7fe7b40cddc0 2023-12-19T18:26:36.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.138+0000 7fe7b27fc640 1 --2- 172.21.15.27:0/2618412195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00047c0 0x7fe7a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.138+0000 7fe7b2ffd640 1 --2- 172.21.15.27:0/2618412195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a00039a0 0x7fe7a0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.138+0000 7fe7b37fe640 1 --2- 172.21.15.27:0/2618412195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7b40cddc0 0x7fe7a0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.139+0000 7fe7b27fc640 1 -- 172.21.15.27:0/2618412195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7b40cddc0 msgr2=0x7fe7a0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.139+0000 7fe7b27fc640 1 --2- 172.21.15.27:0/2618412195 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7b40cddc0 0x7fe7a0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.139+0000 7fe7b27fc640 1 -- 172.21.15.27:0/2618412195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a00039a0 msgr2=0x7fe7a0003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.139+0000 7fe7b27fc640 1 --2- 172.21.15.27:0/2618412195 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7a00039a0 0x7fe7a0003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.139+0000 7fe7b27fc640 1 -- 172.21.15.27:0/2618412195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a0003520 con 0x7fe7a00047c0 2023-12-19T18:26:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.140+0000 7fe7b27fc640 1 --2- 172.21.15.27:0/2618412195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00047c0 0x7fe7a0004c40 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe79c001720 tx=0x7fe7a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b6654e26923635 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.140+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c0052d0 con 0x7fe7a00047c0 2023-12-19T18:26:36.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.140+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c02e050 con 0x7fe7a00047c0 2023-12-19T18:26:36.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.140+0000 7fe7b1ffb640 1 -- 172.21.15.27:0/2618412195 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c0035f0 con 0x7fe7a00047c0 2023-12-19T18:26:36.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.141+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/2618412195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00047c0 msgr2=0x7fe7a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:36.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.141+0000 7fe7b9a4a640 1 --2- 172.21.15.27:0/2618412195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7a00047c0 0x7fe7a0004c40 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7fe79c001720 tx=0x7fe7a00057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:36.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.141+0000 7fe7b37fe640 1 -- 172.21.15.27:0/2618412195 reap_dead start 2023-12-19T18:26:36.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.141+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/2618412195 shutdown_connections 2023-12-19T18:26:36.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.141+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/2618412195 >> 172.21.15.27:0/2618412195 conn(0x7fe7b409a000 msgr2=0x7fe7b40d0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:36.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.141+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/2618412195 shutdown_connections 2023-12-19T18:26:36.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.141+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/2618412195 wait complete. 2023-12-19T18:26:36.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.143+0000 7fe7b9a4a640 1 Processor -- start 2023-12-19T18:26:36.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.143+0000 7fe7b9a4a640 1 -- start start 2023-12-19T18:26:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.144+0000 7fe7b9a4a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7b415e170 0x7fe7b40b4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.144+0000 7fe7b9a4a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b415e550 0x7fe7b41f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.144+0000 7fe7b9a4a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7b41f6ba0 0x7fe7b41564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.144+0000 7fe7b9a4a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40c6940 con 0x7fe7b415e550 2023-12-19T18:26:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.144+0000 7fe7b9a4a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b41f71d0 con 0x7fe7b415e170 2023-12-19T18:26:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.144+0000 7fe7b9a4a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7b40b39d0 con 0x7fe7b41f6ba0 2023-12-19T18:26:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.144+0000 7fe7b27fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b415e550 0x7fe7b41f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.145+0000 7fe7b27fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b415e550 0x7fe7b41f6660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43932/0 (socket says 172.21.15.27:43932) 2023-12-19T18:26:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.145+0000 7fe7b27fc640 1 -- 172.21.15.27:0/3717656106 learned_addr learned my addr 172.21.15.27:0/3717656106 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.145+0000 7fe7b27fc640 1 -- 172.21.15.27:0/3717656106 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7b41f6ba0 msgr2=0x7fe7b41564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.145+0000 7fe7b27fc640 1 --2- 172.21.15.27:0/3717656106 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7b41f6ba0 0x7fe7b41564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.145+0000 7fe7b27fc640 1 -- 172.21.15.27:0/3717656106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7b415e170 msgr2=0x7fe7b40b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.145+0000 7fe7b27fc640 1 --2- 172.21.15.27:0/3717656106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7b415e170 0x7fe7b40b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.145+0000 7fe7b27fc640 1 -- 172.21.15.27:0/3717656106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b40b3cb0 con 0x7fe7b415e550 2023-12-19T18:26:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.145+0000 7fe7b27fc640 1 --2- 172.21.15.27:0/3717656106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b415e550 0x7fe7b41f6660 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fe79c005040 tx=0x7fe79c001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.146+0000 7fe7abfff640 1 -- 172.21.15.27:0/3717656106 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c007640 con 0x7fe7b415e550 2023-12-19T18:26:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.146+0000 7fe7abfff640 1 -- 172.21.15.27:0/3717656106 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c0069e0 con 0x7fe7b415e550 2023-12-19T18:26:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.146+0000 7fe7abfff640 1 -- 172.21.15.27:0/3717656106 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c006d90 con 0x7fe7b415e550 2023-12-19T18:26:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.146+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/3717656106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7b41f74b0 con 0x7fe7b415e550 2023-12-19T18:26:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.146+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/3717656106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7b40b4bb0 con 0x7fe7b415e550 2023-12-19T18:26:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.147+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/3717656106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7b40b4300 con 0x7fe7b415e550 2023-12-19T18:26:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.148+0000 7fe7abfff640 1 -- 172.21.15.27:0/3717656106 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe79c0035f0 con 0x7fe7b415e550 2023-12-19T18:26:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.149+0000 7fe7abfff640 1 --2- 172.21.15.27:0/3717656106 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe79406a430 0x7fe79406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.149+0000 7fe7abfff640 1 -- 172.21.15.27:0/3717656106 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe79c0aa110 con 0x7fe7b415e550 2023-12-19T18:26:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.149+0000 7fe7b2ffd640 1 --2- 172.21.15.27:0/3717656106 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe79406a430 0x7fe79406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.151+0000 7fe7b2ffd640 1 --2- 172.21.15.27:0/3717656106 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe79406a430 0x7fe79406c8f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4001eb0 tx=0x7fe7a4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.155+0000 7fe7abfff640 1 -- 172.21.15.27:0/3717656106 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe79c075b60 con 0x7fe7b415e550 2023-12-19T18:26:36.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.356+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/3717656106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe7b4068e90 con 0x7fe7b415e550 2023-12-19T18:26:36.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.966+0000 7fe7abfff640 1 -- 172.21.15.27:0/3717656106 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v182) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe79c079180 con 0x7fe7b415e550 2023-12-19T18:26:36.966 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-12-19T18:26:36.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.971+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/3717656106 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe79406a430 msgr2=0x7fe79406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:36.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.971+0000 7fe7b9a4a640 1 --2- 172.21.15.27:0/3717656106 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe79406a430 0x7fe79406c8f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe7a4001eb0 tx=0x7fe7a4009040 comp rx=0 tx=0).stop 2023-12-19T18:26:36.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.971+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/3717656106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b415e550 msgr2=0x7fe7b41f6660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:36.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.971+0000 7fe7b9a4a640 1 --2- 172.21.15.27:0/3717656106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b415e550 0x7fe7b41f6660 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7fe79c005040 tx=0x7fe79c001120 comp rx=0 tx=0).stop 2023-12-19T18:26:36.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.972+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/3717656106 shutdown_connections 2023-12-19T18:26:36.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.972+0000 7fe7b9a4a640 1 --2- 172.21.15.27:0/3717656106 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe79406a430 0x7fe79406c8f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:36.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.972+0000 7fe7b9a4a640 1 --2- 172.21.15.27:0/3717656106 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe7b41f6ba0 0x7fe7b41564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:36.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.972+0000 7fe7b9a4a640 1 --2- 172.21.15.27:0/3717656106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7b415e550 0x7fe7b41f6660 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:36.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.972+0000 7fe7b9a4a640 1 --2- 172.21.15.27:0/3717656106 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe7b415e170 0x7fe7b40b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:36.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.972+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/3717656106 >> 172.21.15.27:0/3717656106 conn(0x7fe7b409a000 msgr2=0x7fe7b415f060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:36.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.972+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/3717656106 shutdown_connections 2023-12-19T18:26:36.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:36.973+0000 7fe7b9a4a640 1 -- 172.21.15.27:0/3717656106 wait complete. 2023-12-19T18:26:36.995 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:26:37.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.239+0000 7fa082dab640 1 Processor -- start 2023-12-19T18:26:37.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.239+0000 7fa082dab640 1 -- start start 2023-12-19T18:26:37.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.241+0000 7fa082dab640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa07c0b5160 0x7fa07c0b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:37.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.241+0000 7fa082dab640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07c15d750 con 0x7fa07c0b55f0 2023-12-19T18:26:37.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.241+0000 7fa082dab640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa07c0d1750 con 0x7fa07c15d370 2023-12-19T18:26:37.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.241+0000 7fa082dab640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa07c0b2880 con 0x7fa07c0b5160 2023-12-19T18:26:37.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.241+0000 7fa081348640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa07c0b55f0 0x7fa07c0b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40014/0 (socket says 172.21.15.27:40014) 2023-12-19T18:26:37.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.241+0000 7fa081348640 1 -- 172.21.15.27:0/1394011768 learned_addr learned my addr 172.21.15.27:0/1394011768 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:37.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.242+0000 7fa07bfff640 1 --2- 172.21.15.27:0/1394011768 >> v2:172.21.15.192:3300/0 conn(0x7fa07c0b5160 0x7fa07c0b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:37.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.242+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660906853 0 0) 0x7fa07c15d750 con 0x7fa07c0b55f0 2023-12-19T18:26:37.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.242+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa06c003520 con 0x7fa07c0b55f0 2023-12-19T18:26:37.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.242+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1458964853 0 0) 0x7fa06c003520 con 0x7fa07c0b55f0 2023-12-19T18:26:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.242+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 >> v2:172.21.15.192:3300/0 conn(0x7fa07c0b5160 msgr2=0x7fa07c0b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.242+0000 7fa07b7fe640 1 --2- 172.21.15.27:0/1394011768 >> v2:172.21.15.192:3300/0 conn(0x7fa07c0b5160 0x7fa07c0b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.242+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 >> v1:172.21.15.192:6789/0 conn(0x7fa07c15d370 legacy=0x7fa07c0b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.242+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c0d1a30 con 0x7fa07c0b55f0 2023-12-19T18:26:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.242+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fa0740019f0 con 0x7fa07c0b55f0 2023-12-19T18:26:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.242+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 >> v1:172.21.15.27:6789/0 conn(0x7fa07c0b55f0 legacy=0x7fa07c0b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.242+0000 7fa07b7fe640 1 --2- 172.21.15.27:0/1394011768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa06c0039a0 0x7fa06c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa07b7fe640 1 --2- 172.21.15.27:0/1394011768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa06c0047c0 0x7fa06c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa07b7fe640 1 --2- 172.21.15.27:0/1394011768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa06c005180 0x7fa06c00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c00c000 con 0x7fa06c0039a0 2023-12-19T18:26:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c00c170 con 0x7fa06c005180 2023-12-19T18:26:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c00c330 con 0x7fa06c0047c0 2023-12-19T18:26:37.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa080b47640 1 --2- 172.21.15.27:0/1394011768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa06c0039a0 0x7fa06c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa07bfff640 1 --2- 172.21.15.27:0/1394011768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa06c0047c0 0x7fa06c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa081348640 1 --2- 172.21.15.27:0/1394011768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa06c005180 0x7fa06c00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa080b47640 1 -- 172.21.15.27:0/1394011768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa06c0047c0 msgr2=0x7fa06c004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa080b47640 1 --2- 172.21.15.27:0/1394011768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa06c0047c0 0x7fa06c004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa080b47640 1 -- 172.21.15.27:0/1394011768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa06c005180 msgr2=0x7fa06c00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa080b47640 1 --2- 172.21.15.27:0/1394011768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa06c005180 0x7fa06c00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa080b47640 1 -- 172.21.15.27:0/1394011768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c003520 con 0x7fa06c0039a0 2023-12-19T18:26:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa081348640 1 --2- 172.21.15.27:0/1394011768 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa06c005180 0x7fa06c00b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:37.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.243+0000 7fa081348640 1 -- 172.21.15.27:0/1394011768 reap_dead start 2023-12-19T18:26:37.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.245+0000 7fa080b47640 1 --2- 172.21.15.27:0/1394011768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa06c0039a0 0x7fa06c003dc0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fa070001750 tx=0x7fa06c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f44a683eb0f78f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:37.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.245+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0700052d0 con 0x7fa06c0039a0 2023-12-19T18:26:37.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.245+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07002e050 con 0x7fa06c0039a0 2023-12-19T18:26:37.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.245+0000 7fa07b7fe640 1 -- 172.21.15.27:0/1394011768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070003830 con 0x7fa06c0039a0 2023-12-19T18:26:37.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.245+0000 7fa082dab640 1 -- 172.21.15.27:0/1394011768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa06c0039a0 msgr2=0x7fa06c003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.245+0000 7fa082dab640 1 --2- 172.21.15.27:0/1394011768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa06c0039a0 0x7fa06c003dc0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7fa070001750 tx=0x7fa06c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.246+0000 7fa082dab640 1 -- 172.21.15.27:0/1394011768 shutdown_connections 2023-12-19T18:26:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.246+0000 7fa082dab640 1 --2- 172.21.15.27:0/1394011768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa06c0039a0 0x7fa06c003dc0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.246+0000 7fa082dab640 1 -- 172.21.15.27:0/1394011768 >> 172.21.15.27:0/1394011768 conn(0x7fa07c06a480 msgr2=0x7fa07c06a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.246+0000 7fa082dab640 1 -- 172.21.15.27:0/1394011768 shutdown_connections 2023-12-19T18:26:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.246+0000 7fa082dab640 1 -- 172.21.15.27:0/1394011768 wait complete. 2023-12-19T18:26:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.248+0000 7fa082dab640 1 Processor -- start 2023-12-19T18:26:37.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.248+0000 7fa082dab640 1 -- start start 2023-12-19T18:26:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.249+0000 7fa082dab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa06c0039a0 0x7fa07c156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.249+0000 7fa082dab640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa07c0c8e80 0x7fa07c157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.249+0000 7fa082dab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa07c153590 0x7fa07c157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.249+0000 7fa082dab640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c159440 con 0x7fa06c0039a0 2023-12-19T18:26:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.249+0000 7fa082dab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c0c9380 con 0x7fa07c153590 2023-12-19T18:26:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.249+0000 7fa082dab640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa07c159690 con 0x7fa07c0c8e80 2023-12-19T18:26:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.249+0000 7fa080b47640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa06c0039a0 0x7fa07c156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.249+0000 7fa080b47640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa06c0039a0 0x7fa07c156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43962/0 (socket says 172.21.15.27:43962) 2023-12-19T18:26:37.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.249+0000 7fa080b47640 1 -- 172.21.15.27:0/1138054750 learned_addr learned my addr 172.21.15.27:0/1138054750 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:37.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.249+0000 7fa07bfff640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa07c0c8e80 0x7fa07c157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:37.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.249+0000 7fa081348640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa07c153590 0x7fa07c157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:37.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.250+0000 7fa080b47640 1 -- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa07c0c8e80 msgr2=0x7fa07c157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:37.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.250+0000 7fa080b47640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa07c0c8e80 0x7fa07c157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:37.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.250+0000 7fa080b47640 1 -- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa07c153590 msgr2=0x7fa07c157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:37.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.250+0000 7fa080b47640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa07c153590 0x7fa07c157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:37.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.250+0000 7fa080b47640 1 -- 172.21.15.27:0/1138054750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c159970 con 0x7fa06c0039a0 2023-12-19T18:26:37.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.250+0000 7fa080b47640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa06c0039a0 0x7fa07c156e30 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa07002bc00 tx=0x7fa070001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:37.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.250+0000 7fa0797fa640 1 -- 172.21.15.27:0/1138054750 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0700076c0 con 0x7fa06c0039a0 2023-12-19T18:26:37.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.250+0000 7fa0797fa640 1 -- 172.21.15.27:0/1138054750 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa070007a30 con 0x7fa06c0039a0 2023-12-19T18:26:37.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.250+0000 7fa0797fa640 1 -- 172.21.15.27:0/1138054750 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070006c20 con 0x7fa06c0039a0 2023-12-19T18:26:37.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.250+0000 7fa082dab640 1 -- 172.21.15.27:0/1138054750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa07c158080 con 0x7fa06c0039a0 2023-12-19T18:26:37.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.251+0000 7fa082dab640 1 -- 172.21.15.27:0/1138054750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa07c1f5cd0 con 0x7fa06c0039a0 2023-12-19T18:26:37.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.252+0000 7fa082dab640 1 -- 172.21.15.27:0/1138054750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa07c0ce7d0 con 0x7fa06c0039a0 2023-12-19T18:26:37.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.253+0000 7fa0797fa640 1 -- 172.21.15.27:0/1138054750 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa0700035f0 con 0x7fa06c0039a0 2023-12-19T18:26:37.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.255+0000 7fa0797fa640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa06006a460 0x7fa06006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:37.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.255+0000 7fa0797fa640 1 -- 172.21.15.27:0/1138054750 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fa0700aa750 con 0x7fa06c0039a0 2023-12-19T18:26:37.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.256+0000 7fa07bfff640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa06006a460 0x7fa06006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:37.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.257+0000 7fa07bfff640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa06006a460 0x7fa06006c920 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0d11d0 tx=0x7fa068008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:37.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.259+0000 7fa0797fa640 1 -- 172.21.15.27:0/1138054750 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa070079c40 con 0x7fa06c0039a0 2023-12-19T18:26:37.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.454+0000 7fa082dab640 1 -- 172.21.15.27:0/1138054750 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa07c068e90 con 0x7fa06c0039a0 2023-12-19T18:26:37.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.983+0000 7fa0797fa640 1 -- 172.21.15.27:0/1138054750 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v183) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa0700410d0 con 0x7fa06c0039a0 2023-12-19T18:26:37.985 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-12-19T18:26:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.992+0000 7fa082dab640 1 -- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa06006a460 msgr2=0x7fa06006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.992+0000 7fa082dab640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa06006a460 0x7fa06006c920 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fa07c0d11d0 tx=0x7fa068008040 comp rx=0 tx=0).stop 2023-12-19T18:26:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.992+0000 7fa082dab640 1 -- 172.21.15.27:0/1138054750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa06c0039a0 msgr2=0x7fa07c156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:37.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.992+0000 7fa082dab640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa06c0039a0 0x7fa07c156e30 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7fa07002bc00 tx=0x7fa070001120 comp rx=0 tx=0).stop 2023-12-19T18:26:37.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.993+0000 7fa082dab640 1 -- 172.21.15.27:0/1138054750 shutdown_connections 2023-12-19T18:26:37.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.993+0000 7fa082dab640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa06006a460 0x7fa06006c920 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:37.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.993+0000 7fa082dab640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa07c153590 0x7fa07c157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:37.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.993+0000 7fa082dab640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa07c0c8e80 0x7fa07c157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:37.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.993+0000 7fa082dab640 1 --2- 172.21.15.27:0/1138054750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa06c0039a0 0x7fa07c156e30 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:37.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.993+0000 7fa082dab640 1 -- 172.21.15.27:0/1138054750 >> 172.21.15.27:0/1138054750 conn(0x7fa07c06a480 msgr2=0x7fa07c15c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:37.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.994+0000 7fa082dab640 1 -- 172.21.15.27:0/1138054750 shutdown_connections 2023-12-19T18:26:37.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:37.994+0000 7fa082dab640 1 -- 172.21.15.27:0/1138054750 wait complete. 2023-12-19T18:26:38.022 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:26:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.230+0000 7f42ddce5640 1 Processor -- start 2023-12-19T18:26:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.230+0000 7f42ddce5640 1 -- start start 2023-12-19T18:26:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.232+0000 7f42ddce5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42d815e9d0 0x7f42d80cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.232+0000 7f42ddce5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42d8152070 0x7f42d80cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.232+0000 7f42ddce5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d815b3e0 con 0x7f42d8151c90 2023-12-19T18:26:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.232+0000 7f42ddce5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d815eed0 con 0x7f42d815e9d0 2023-12-19T18:26:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.232+0000 7f42ddce5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d815f1b0 con 0x7f42d8152070 2023-12-19T18:26:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.233+0000 7f42d7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42d815e9d0 0x7f42d80cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.233+0000 7f42d7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f42d815e9d0 0x7f42d80cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43974/0 (socket says 172.21.15.27:43974) 2023-12-19T18:26:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.233+0000 7f42d7fff640 1 -- 172.21.15.27:0/1664501005 learned_addr learned my addr 172.21.15.27:0/1664501005 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:38.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.233+0000 7f42dd4e4640 1 --2- 172.21.15.27:0/1664501005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42d8152070 0x7f42d80cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:38.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.233+0000 7f42d7fff640 1 -- 172.21.15.27:0/1664501005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42d8152070 msgr2=0x7f42d80cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:38.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.233+0000 7f42d7fff640 1 --2- 172.21.15.27:0/1664501005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42d8152070 0x7f42d80cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.233+0000 7f42d7fff640 1 -- 172.21.15.27:0/1664501005 >> v1:172.21.15.192:6789/0 conn(0x7f42d8151c90 legacy=0x7f42d806b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.233+0000 7f42d7fff640 1 -- 172.21.15.27:0/1664501005 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d815f490 con 0x7f42d815e9d0 2023-12-19T18:26:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.233+0000 7f42dd4e4640 1 --2- 172.21.15.27:0/1664501005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42d8152070 0x7f42d80cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.234+0000 7f42d7fff640 1 --2- 172.21.15.27:0/1664501005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d815e9d0 0x7f42d80cee40 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f42c40098a0 tx=0x7f42c402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62aaa1ddd660341 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.235+0000 7f42d77fe640 1 -- 172.21.15.27:0/1664501005 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c40317c0 con 0x7f42d815e9d0 2023-12-19T18:26:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.235+0000 7f42d77fe640 1 -- 172.21.15.27:0/1664501005 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c4031920 con 0x7f42d815e9d0 2023-12-19T18:26:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.235+0000 7f42d77fe640 1 -- 172.21.15.27:0/1664501005 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c4031cb0 con 0x7f42d815e9d0 2023-12-19T18:26:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.235+0000 7f42ddce5640 1 -- 172.21.15.27:0/1664501005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d815e9d0 msgr2=0x7f42d80cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.235+0000 7f42ddce5640 1 --2- 172.21.15.27:0/1664501005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d815e9d0 0x7f42d80cee40 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f42c40098a0 tx=0x7f42c402b860 comp rx=0 tx=0).stop 2023-12-19T18:26:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.236+0000 7f42ddce5640 1 -- 172.21.15.27:0/1664501005 shutdown_connections 2023-12-19T18:26:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.236+0000 7f42ddce5640 1 --2- 172.21.15.27:0/1664501005 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42d8152070 0x7f42d80cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.236+0000 7f42ddce5640 1 --2- 172.21.15.27:0/1664501005 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d815e9d0 0x7f42d80cee40 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.236+0000 7f42ddce5640 1 -- 172.21.15.27:0/1664501005 >> 172.21.15.27:0/1664501005 conn(0x7f42d806a060 msgr2=0x7f42d806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.236+0000 7f42ddce5640 1 -- 172.21.15.27:0/1664501005 shutdown_connections 2023-12-19T18:26:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.236+0000 7f42ddce5640 1 -- 172.21.15.27:0/1664501005 wait complete. 2023-12-19T18:26:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.238+0000 7f42ddce5640 1 Processor -- start 2023-12-19T18:26:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.238+0000 7f42ddce5640 1 -- start start 2023-12-19T18:26:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.239+0000 7f42ddce5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8151c90 0x7f42d81565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.239+0000 7f42ddce5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d8152070 0x7f42d8140a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.239+0000 7f42ddce5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42d815e9d0 0x7f42d8142270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.239+0000 7f42ddce5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8140fc0 con 0x7f42d8152070 2023-12-19T18:26:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.239+0000 7f42ddce5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d8141de0 con 0x7f42d8151c90 2023-12-19T18:26:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.239+0000 7f42ddce5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d81411d0 con 0x7f42d815e9d0 2023-12-19T18:26:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.239+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d8152070 0x7f42d8140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.239+0000 7f42d7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d8152070 0x7f42d8140a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43986/0 (socket says 172.21.15.27:43986) 2023-12-19T18:26:38.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.239+0000 7f42d7fff640 1 -- 172.21.15.27:0/3480917220 learned_addr learned my addr 172.21.15.27:0/3480917220 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.239+0000 7f42dcce3640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8151c90 0x7f42d81565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.239+0000 7f42d7fff640 1 -- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42d815e9d0 msgr2=0x7f42d8142270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.239+0000 7f42dd4e4640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42d815e9d0 0x7f42d8142270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.240+0000 7f42d7fff640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42d815e9d0 0x7f42d8142270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.240+0000 7f42d7fff640 1 -- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8151c90 msgr2=0x7f42d81565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.240+0000 7f42d7fff640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8151c90 0x7f42d81565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.240+0000 7f42d7fff640 1 -- 172.21.15.27:0/3480917220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c40094d0 con 0x7f42d8152070 2023-12-19T18:26:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.240+0000 7f42dcce3640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8151c90 0x7f42d81565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.240+0000 7f42d7fff640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d8152070 0x7f42d8140a80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f42c40090b0 tx=0x7f42c40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.240+0000 7f42d57fa640 1 -- 172.21.15.27:0/3480917220 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c40329a0 con 0x7f42d8152070 2023-12-19T18:26:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.240+0000 7f42d57fa640 1 -- 172.21.15.27:0/3480917220 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42c4032b20 con 0x7f42d8152070 2023-12-19T18:26:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.240+0000 7f42d57fa640 1 -- 172.21.15.27:0/3480917220 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42c40453f0 con 0x7f42d8152070 2023-12-19T18:26:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.240+0000 7f42ddce5640 1 -- 172.21.15.27:0/3480917220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d81414b0 con 0x7f42d8152070 2023-12-19T18:26:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.241+0000 7f42ddce5640 1 -- 172.21.15.27:0/3480917220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d8141a70 con 0x7f42d8152070 2023-12-19T18:26:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.242+0000 7f42ddce5640 1 -- 172.21.15.27:0/3480917220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d8141f90 con 0x7f42d8152070 2023-12-19T18:26:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.243+0000 7f42d57fa640 1 -- 172.21.15.27:0/3480917220 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f42c404d3f0 con 0x7f42d8152070 2023-12-19T18:26:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.244+0000 7f42d57fa640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f42bc06a430 0x7f42bc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.244+0000 7f42d57fa640 1 -- 172.21.15.27:0/3480917220 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f42c40aa920 con 0x7f42d8152070 2023-12-19T18:26:38.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.244+0000 7f42dcce3640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f42bc06a430 0x7f42bc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:38.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.245+0000 7f42dcce3640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f42bc06a430 0x7f42bc06c8f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f42cc003eb0 tx=0x7f42cc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:38.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.249+0000 7f42d57fa640 1 -- 172.21.15.27:0/3480917220 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f42d8141f90 con 0x7f42d8152070 2023-12-19T18:26:38.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.481+0000 7f42ddce5640 1 -- 172.21.15.27:0/3480917220 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f42d8141f50 con 0x7f42d8152070 2023-12-19T18:26:38.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.494+0000 7f42d57fa640 1 -- 172.21.15.27:0/3480917220 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v73) v1 ==== 174+0+0 (secure 0 0 0) 0x7f42c4075ed0 con 0x7f42d8152070 2023-12-19T18:26:38.495 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 20 and data pool 21 2023-12-19T18:26:38.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.500+0000 7f42ddce5640 1 -- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f42bc06a430 msgr2=0x7f42bc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:38.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.500+0000 7f42ddce5640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f42bc06a430 0x7f42bc06c8f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f42cc003eb0 tx=0x7f42cc009040 comp rx=0 tx=0).stop 2023-12-19T18:26:38.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.501+0000 7f42ddce5640 1 -- 172.21.15.27:0/3480917220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d8152070 msgr2=0x7f42d8140a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:38.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.501+0000 7f42ddce5640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d8152070 0x7f42d8140a80 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f42c40090b0 tx=0x7f42c40046c0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.503+0000 7f42ddce5640 1 -- 172.21.15.27:0/3480917220 shutdown_connections 2023-12-19T18:26:38.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.503+0000 7f42ddce5640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f42bc06a430 0x7f42bc06c8f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.503+0000 7f42ddce5640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f42d815e9d0 0x7f42d8142270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.503+0000 7f42ddce5640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42d8152070 0x7f42d8140a80 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.503+0000 7f42ddce5640 1 --2- 172.21.15.27:0/3480917220 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f42d8151c90 0x7f42d81565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.503+0000 7f42ddce5640 1 -- 172.21.15.27:0/3480917220 >> 172.21.15.27:0/3480917220 conn(0x7f42d806a060 msgr2=0x7f42d806b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:38.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.505+0000 7f42ddce5640 1 -- 172.21.15.27:0/3480917220 shutdown_connections 2023-12-19T18:26:38.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.506+0000 7f42ddce5640 1 -- 172.21.15.27:0/3480917220 wait complete. 2023-12-19T18:26:38.532 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:26:38.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.725+0000 7fcabafbf640 1 Processor -- start 2023-12-19T18:26:38.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.725+0000 7fcabafbf640 1 -- start start 2023-12-19T18:26:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.726+0000 7fcabafbf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab4159c40 0x7fcab406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.726+0000 7fcabafbf640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcab415a020 0x7fcab406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:38.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.726+0000 7fcabafbf640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab415b180 con 0x7fcab406c880 2023-12-19T18:26:38.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.726+0000 7fcabafbf640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab40ca850 con 0x7fcab415a020 2023-12-19T18:26:38.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.726+0000 7fcabafbf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab415a570 con 0x7fcab4159c40 2023-12-19T18:26:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.727+0000 7fcab8d5b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab4159c40 0x7fcab406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.727+0000 7fcab955c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcab415a020 0x7fcab406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.727+0000 7fcab3fff640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7fcab406c880 0x7fcab40cd800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:60166/0 (socket says 172.21.15.27:60166) 2023-12-19T18:26:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.727+0000 7fcab8d5b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab4159c40 0x7fcab406b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:55332/0 (socket says 172.21.15.27:55332) 2023-12-19T18:26:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.727+0000 7fcab3fff640 1 -- 172.21.15.27:0/134781941 learned_addr learned my addr 172.21.15.27:0/134781941 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.728+0000 7fcab955c640 1 -- 172.21.15.27:0/134781941 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab4159c40 msgr2=0x7fcab406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.728+0000 7fcab955c640 1 --2- 172.21.15.27:0/134781941 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab4159c40 0x7fcab406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.728+0000 7fcab955c640 1 -- 172.21.15.27:0/134781941 >> v1:172.21.15.192:6789/0 conn(0x7fcab406c880 legacy=0x7fcab40cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.728+0000 7fcab955c640 1 -- 172.21.15.27:0/134781941 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab415a850 con 0x7fcab415a020 2023-12-19T18:26:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.729+0000 7fcab955c640 1 --2- 172.21.15.27:0/134781941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab415a020 0x7fcab406ba20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009d90 tx=0x7fcaac02e0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c3ec9c10b64d5e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.730+0000 7fcab37fe640 1 -- 172.21.15.27:0/134781941 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac02ee70 con 0x7fcab415a020 2023-12-19T18:26:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.730+0000 7fcab37fe640 1 -- 172.21.15.27:0/134781941 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac036440 con 0x7fcab415a020 2023-12-19T18:26:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.730+0000 7fcab37fe640 1 -- 172.21.15.27:0/134781941 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac0367d0 con 0x7fcab415a020 2023-12-19T18:26:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.731+0000 7fcabafbf640 1 -- 172.21.15.27:0/134781941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab415a020 msgr2=0x7fcab406ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.731+0000 7fcabafbf640 1 --2- 172.21.15.27:0/134781941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab415a020 0x7fcab406ba20 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009d90 tx=0x7fcaac02e0c0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.731+0000 7fcabafbf640 1 -- 172.21.15.27:0/134781941 shutdown_connections 2023-12-19T18:26:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.731+0000 7fcabafbf640 1 --2- 172.21.15.27:0/134781941 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab415a020 0x7fcab406ba20 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.731+0000 7fcabafbf640 1 --2- 172.21.15.27:0/134781941 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab4159c40 0x7fcab406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.731+0000 7fcabafbf640 1 -- 172.21.15.27:0/134781941 >> 172.21.15.27:0/134781941 conn(0x7fcab406a330 msgr2=0x7fcab40b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:38.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.731+0000 7fcabafbf640 1 -- 172.21.15.27:0/134781941 shutdown_connections 2023-12-19T18:26:38.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.731+0000 7fcabafbf640 1 -- 172.21.15.27:0/134781941 wait complete. 2023-12-19T18:26:38.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.733+0000 7fcabafbf640 1 Processor -- start 2023-12-19T18:26:38.733 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.733+0000 7fcabafbf640 1 -- start start 2023-12-19T18:26:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.735+0000 7fcabafbf640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab406c880 0x7fcab40afa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.735+0000 7fcabafbf640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab4159c40 0x7fcab41468a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.735+0000 7fcabafbf640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab415a020 0x7fcab40ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.735+0000 7fcabafbf640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4146de0 con 0x7fcab415a020 2023-12-19T18:26:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.735+0000 7fcabafbf640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40c5690 con 0x7fcab4159c40 2023-12-19T18:26:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.735+0000 7fcabafbf640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40aef80 con 0x7fcab406c880 2023-12-19T18:26:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.735+0000 7fcab955c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab415a020 0x7fcab40ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.735+0000 7fcab955c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab415a020 0x7fcab40ac8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44000/0 (socket says 172.21.15.27:44000) 2023-12-19T18:26:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.735+0000 7fcab955c640 1 -- 172.21.15.27:0/1934622197 learned_addr learned my addr 172.21.15.27:0/1934622197 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.735+0000 7fcab8d5b640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab406c880 0x7fcab40afa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.735+0000 7fcab3fff640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab4159c40 0x7fcab41468a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.736+0000 7fcab955c640 1 -- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab406c880 msgr2=0x7fcab40afa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.736+0000 7fcab955c640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab406c880 0x7fcab40afa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.736+0000 7fcab955c640 1 -- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab4159c40 msgr2=0x7fcab41468a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.736+0000 7fcab955c640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab4159c40 0x7fcab41468a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.736+0000 7fcab955c640 1 -- 172.21.15.27:0/1934622197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac009a70 con 0x7fcab415a020 2023-12-19T18:26:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.736+0000 7fcab8d5b640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab406c880 0x7fcab40afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:26:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.736+0000 7fcab955c640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab415a020 0x7fcab40ac8d0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009d90 tx=0x7fcaac004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.736+0000 7fcab17fa640 1 -- 172.21.15.27:0/1934622197 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac03f850 con 0x7fcab415a020 2023-12-19T18:26:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.736+0000 7fcab17fa640 1 -- 172.21.15.27:0/1934622197 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac03f9b0 con 0x7fcab415a020 2023-12-19T18:26:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.736+0000 7fcab17fa640 1 -- 172.21.15.27:0/1934622197 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac03fd60 con 0x7fcab415a020 2023-12-19T18:26:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.736+0000 7fcabafbf640 1 -- 172.21.15.27:0/1934622197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab4147d50 con 0x7fcab415a020 2023-12-19T18:26:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.737+0000 7fcabafbf640 1 -- 172.21.15.27:0/1934622197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab40af220 con 0x7fcab415a020 2023-12-19T18:26:38.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.738+0000 7fcabafbf640 1 -- 172.21.15.27:0/1934622197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab4068e90 con 0x7fcab415a020 2023-12-19T18:26:38.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.739+0000 7fcab17fa640 1 -- 172.21.15.27:0/1934622197 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcaac03d070 con 0x7fcab415a020 2023-12-19T18:26:38.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.740+0000 7fcab17fa640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fca9806a430 0x7fca9806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:38.739 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.740+0000 7fcab17fa640 1 -- 172.21.15.27:0/1934622197 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcaac0ab150 con 0x7fcab415a020 2023-12-19T18:26:38.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.740+0000 7fcab8d5b640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fca9806a430 0x7fca9806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:38.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.741+0000 7fcab8d5b640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fca9806a430 0x7fca9806c8f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009b90 tx=0x7fcaa8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:38.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.744+0000 7fcab17fa640 1 -- 172.21.15.27:0/1934622197 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcaac076740 con 0x7fcab415a020 2023-12-19T18:26:38.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.937+0000 7fcabafbf640 1 -- 172.21.15.27:0/1934622197 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcab40c9980 con 0x7fcab415a020 2023-12-19T18:26:38.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.940+0000 7fcab17fa640 1 -- 172.21.15.27:0/1934622197 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16125 (secure 0 0 0) 0x7fcaac079d60 con 0x7fcab415a020 2023-12-19T18:26:38.940 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:38.940 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":184,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:26:38.483135+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:26:36.428093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T18:26:37.523993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:26:38.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.943+0000 7fcabafbf640 1 -- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fca9806a430 msgr2=0x7fca9806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:38.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.943+0000 7fcabafbf640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fca9806a430 0x7fca9806c8f0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8009b90 tx=0x7fcaa8008040 comp rx=0 tx=0).stop 2023-12-19T18:26:38.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.943+0000 7fcabafbf640 1 -- 172.21.15.27:0/1934622197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab415a020 msgr2=0x7fcab40ac8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:38.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.943+0000 7fcabafbf640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab415a020 0x7fcab40ac8d0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fcaac009d90 tx=0x7fcaac004a40 comp rx=0 tx=0).stop 2023-12-19T18:26:38.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.943+0000 7fcabafbf640 1 -- 172.21.15.27:0/1934622197 shutdown_connections 2023-12-19T18:26:38.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.943+0000 7fcabafbf640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fca9806a430 0x7fca9806c8f0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.943+0000 7fcabafbf640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcab415a020 0x7fcab40ac8d0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.943+0000 7fcabafbf640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcab4159c40 0x7fcab41468a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.943+0000 7fcabafbf640 1 --2- 172.21.15.27:0/1934622197 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcab406c880 0x7fcab40afa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:38.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.943+0000 7fcabafbf640 1 -- 172.21.15.27:0/1934622197 >> 172.21.15.27:0/1934622197 conn(0x7fcab406a330 msgr2=0x7fcab4158900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:38.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.943+0000 7fcabafbf640 1 -- 172.21.15.27:0/1934622197 shutdown_connections 2023-12-19T18:26:38.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:38.943+0000 7fcabafbf640 1 -- 172.21.15.27:0/1934622197 wait complete. 2023-12-19T18:26:38.961 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:26:39.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.191+0000 7f8b50641640 1 Processor -- start 2023-12-19T18:26:39.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.191+0000 7f8b50641640 1 -- start start 2023-12-19T18:26:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.192+0000 7f8b50641640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b4815af70 0x7f8b480ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.193+0000 7f8b50641640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b480cb850 con 0x7f8b480cbce0 2023-12-19T18:26:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.193+0000 7f8b50641640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b4815b350 con 0x7f8b480adc20 2023-12-19T18:26:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.193+0000 7f8b50641640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480cac40 con 0x7f8b4815af70 2023-12-19T18:26:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.193+0000 7f8b4dbdc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8b480cbce0 0x7f8b480ad510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40016/0 (socket says 172.21.15.27:40016) 2023-12-19T18:26:39.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.193+0000 7f8b4dbdc640 1 -- 172.21.15.27:0/2574026082 learned_addr learned my addr 172.21.15.27:0/2574026082 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:39.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.193+0000 7f8b4e3dd640 1 --2- 172.21.15.27:0/2574026082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b4815af70 0x7f8b480ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665312127 0 0) 0x7f8b480cb850 con 0x7f8b480cbce0 2023-12-19T18:26:39.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b34003520 con 0x7f8b480cbce0 2023-12-19T18:26:39.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220500000 0 0) 0x7f8b4815b350 con 0x7f8b480adc20 2023-12-19T18:26:39.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b480cb850 con 0x7f8b480adc20 2023-12-19T18:26:39.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3576591715 0 0) 0x7f8b34003520 con 0x7f8b480cbce0 2023-12-19T18:26:39.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b4815af70 msgr2=0x7f8b480ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:39.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 --2- 172.21.15.27:0/2574026082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b4815af70 0x7f8b480ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 >> v1:172.21.15.192:6789/0 conn(0x7f8b480adc20 legacy=0x7f8b4806b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:39.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b480caf20 con 0x7f8b480cbce0 2023-12-19T18:26:39.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f8b38001670 con 0x7f8b480cbce0 2023-12-19T18:26:39.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 >> v1:172.21.15.27:6789/0 conn(0x7f8b480cbce0 legacy=0x7f8b480ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:39.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 --2- 172.21.15.27:0/2574026082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b34003b30 0x7f8b34003f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 --2- 172.21.15.27:0/2574026082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b340047c0 0x7f8b34004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 --2- 172.21.15.27:0/2574026082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b34005180 0x7f8b3400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480cac40 con 0x7f8b340047c0 2023-12-19T18:26:39.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3400bf70 con 0x7f8b34005180 2023-12-19T18:26:39.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.194+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3400c130 con 0x7f8b34003b30 2023-12-19T18:26:39.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.195+0000 7f8b4dbdc640 1 --2- 172.21.15.27:0/2574026082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b340047c0 0x7f8b34004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.195+0000 7f8b4e3dd640 1 --2- 172.21.15.27:0/2574026082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b34003b30 0x7f8b34003f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.195+0000 7f8b4ebde640 1 --2- 172.21.15.27:0/2574026082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b34005180 0x7f8b3400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.195+0000 7f8b4dbdc640 1 -- 172.21.15.27:0/2574026082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b34003b30 msgr2=0x7f8b34003f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:39.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.195+0000 7f8b4dbdc640 1 --2- 172.21.15.27:0/2574026082 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b34003b30 0x7f8b34003f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.195+0000 7f8b4dbdc640 1 -- 172.21.15.27:0/2574026082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b34005180 msgr2=0x7f8b3400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:39.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.195+0000 7f8b4dbdc640 1 --2- 172.21.15.27:0/2574026082 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b34005180 0x7f8b3400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.195+0000 7f8b4dbdc640 1 -- 172.21.15.27:0/2574026082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b34003520 con 0x7f8b340047c0 2023-12-19T18:26:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.196+0000 7f8b4dbdc640 1 --2- 172.21.15.27:0/2574026082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b340047c0 0x7f8b34004c40 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f8b480cc0c0 tx=0x7f8b340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6573ab7c0431b627 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.197+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b38005ad0 con 0x7f8b340047c0 2023-12-19T18:26:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.197+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b38005c50 con 0x7f8b340047c0 2023-12-19T18:26:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.197+0000 7f8b4d3db640 1 -- 172.21.15.27:0/2574026082 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3802c810 con 0x7f8b340047c0 2023-12-19T18:26:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.197+0000 7f8b50641640 1 -- 172.21.15.27:0/2574026082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b340047c0 msgr2=0x7f8b34004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.197+0000 7f8b50641640 1 --2- 172.21.15.27:0/2574026082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b340047c0 0x7f8b34004c40 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f8b480cc0c0 tx=0x7f8b340057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.197+0000 7f8b4ebde640 1 -- 172.21.15.27:0/2574026082 reap_dead start 2023-12-19T18:26:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.197+0000 7f8b50641640 1 -- 172.21.15.27:0/2574026082 shutdown_connections 2023-12-19T18:26:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.197+0000 7f8b50641640 1 -- 172.21.15.27:0/2574026082 >> 172.21.15.27:0/2574026082 conn(0x7f8b4806a330 msgr2=0x7f8b480b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:39.199 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.198+0000 7f8b50641640 1 -- 172.21.15.27:0/2574026082 shutdown_connections 2023-12-19T18:26:39.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.198+0000 7f8b50641640 1 -- 172.21.15.27:0/2574026082 wait complete. 2023-12-19T18:26:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.199+0000 7f8b50641640 1 Processor -- start 2023-12-19T18:26:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.200+0000 7f8b50641640 1 -- start start 2023-12-19T18:26:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b50641640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b340047c0 0x7f8b4814bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b50641640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b480c9170 0x7f8b480c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b50641640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b480af820 0x7f8b480afca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b50641640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481f6720 con 0x7f8b480af820 2023-12-19T18:26:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b50641640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b480c9b30 con 0x7f8b480c9170 2023-12-19T18:26:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b50641640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b481f5aa0 con 0x7f8b340047c0 2023-12-19T18:26:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b4ebde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b480af820 0x7f8b480afca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b4ebde640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b480af820 0x7f8b480afca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44022/0 (socket says 172.21.15.27:44022) 2023-12-19T18:26:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b4ebde640 1 -- 172.21.15.27:0/3112894605 learned_addr learned my addr 172.21.15.27:0/3112894605 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b4dbdc640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b480c9170 0x7f8b480c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b4e3dd640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b340047c0 0x7f8b4814bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b4ebde640 1 -- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b340047c0 msgr2=0x7f8b4814bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b4ebde640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b340047c0 0x7f8b4814bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b4ebde640 1 -- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b480c9170 msgr2=0x7f8b480c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.201+0000 7f8b4ebde640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b480c9170 0x7f8b480c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.202+0000 7f8b4ebde640 1 -- 172.21.15.27:0/3112894605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b481f5d80 con 0x7f8b480af820 2023-12-19T18:26:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.202+0000 7f8b4ebde640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b480af820 0x7f8b480afca0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f8b4806d5f0 tx=0x7f8b4000eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.202+0000 7f8b3effd640 1 -- 172.21.15.27:0/3112894605 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40003970 con 0x7f8b480af820 2023-12-19T18:26:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.202+0000 7f8b3effd640 1 -- 172.21.15.27:0/3112894605 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b40003ad0 con 0x7f8b480af820 2023-12-19T18:26:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.202+0000 7f8b3effd640 1 -- 172.21.15.27:0/3112894605 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40003e80 con 0x7f8b480af820 2023-12-19T18:26:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.202+0000 7f8b50641640 1 -- 172.21.15.27:0/3112894605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b481f6060 con 0x7f8b480af820 2023-12-19T18:26:39.205 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.202+0000 7f8b50641640 1 -- 172.21.15.27:0/3112894605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b480b01e0 con 0x7f8b480af820 2023-12-19T18:26:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.203+0000 7f8b50641640 1 -- 172.21.15.27:0/3112894605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b48068e90 con 0x7f8b480af820 2023-12-19T18:26:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.204+0000 7f8b3effd640 1 -- 172.21.15.27:0/3112894605 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8b400050d0 con 0x7f8b480af820 2023-12-19T18:26:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.205+0000 7f8b3effd640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b2c06a430 0x7f8b2c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.205+0000 7f8b3effd640 1 -- 172.21.15.27:0/3112894605 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8b40088780 con 0x7f8b480af820 2023-12-19T18:26:39.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.206+0000 7f8b4e3dd640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b2c06a430 0x7f8b2c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.207+0000 7f8b4e3dd640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b2c06a430 0x7f8b2c06c8f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8b480c5630 tx=0x7f8b44008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:39.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.210+0000 7f8b3effd640 1 -- 172.21.15.27:0/3112894605 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8b40053d60 con 0x7f8b480af820 2023-12-19T18:26:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.404+0000 7f8b50641640 1 -- 172.21.15.27:0/3112894605 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b480ad8d0 con 0x7f8b480af820 2023-12-19T18:26:39.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.406+0000 7f8b3effd640 1 -- 172.21.15.27:0/3112894605 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+16125 (secure 0 0 0) 0x7f8b40057380 con 0x7f8b480af820 2023-12-19T18:26:39.406 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:39.406 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":184,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:26:38.483135+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:26:36.428093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T18:26:37.523993+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:26:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.408+0000 7f8b50641640 1 -- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b2c06a430 msgr2=0x7f8b2c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.408+0000 7f8b50641640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b2c06a430 0x7f8b2c06c8f0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f8b480c5630 tx=0x7f8b44008040 comp rx=0 tx=0).stop 2023-12-19T18:26:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.409+0000 7f8b50641640 1 -- 172.21.15.27:0/3112894605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b480af820 msgr2=0x7f8b480afca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.409+0000 7f8b50641640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b480af820 0x7f8b480afca0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f8b4806d5f0 tx=0x7f8b4000eb20 comp rx=0 tx=0).stop 2023-12-19T18:26:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.409+0000 7f8b50641640 1 -- 172.21.15.27:0/3112894605 shutdown_connections 2023-12-19T18:26:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.409+0000 7f8b50641640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8b2c06a430 0x7f8b2c06c8f0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.409+0000 7f8b50641640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b480af820 0x7f8b480afca0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.409+0000 7f8b50641640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8b480c9170 0x7f8b480c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.409+0000 7f8b50641640 1 --2- 172.21.15.27:0/3112894605 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8b340047c0 0x7f8b4814bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.409+0000 7f8b50641640 1 -- 172.21.15.27:0/3112894605 >> 172.21.15.27:0/3112894605 conn(0x7f8b4806a330 msgr2=0x7f8b480aeab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.409+0000 7f8b50641640 1 -- 172.21.15.27:0/3112894605 shutdown_connections 2023-12-19T18:26:39.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.409+0000 7f8b50641640 1 -- 172.21.15.27:0/3112894605 wait complete. 2023-12-19T18:26:39.430 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:26:39.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.669+0000 7f01cae5f640 1 Processor -- start 2023-12-19T18:26:39.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.670+0000 7f01cae5f640 1 -- start start 2023-12-19T18:26:39.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.671+0000 7f01cae5f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c415d370 0x7f01c40b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.671+0000 7f01cae5f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c415d750 con 0x7f01c40b5160 2023-12-19T18:26:39.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.671+0000 7f01cae5f640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01c40d1750 con 0x7f01c40b55f0 2023-12-19T18:26:39.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.671+0000 7f01cae5f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40b2880 con 0x7f01c415d370 2023-12-19T18:26:39.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.672+0000 7f01c93fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f01c40b5160 0x7f01c40b62b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40020/0 (socket says 172.21.15.27:40020) 2023-12-19T18:26:39.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.672+0000 7f01c93fc640 1 -- 172.21.15.27:0/2876262275 learned_addr learned my addr 172.21.15.27:0/2876262275 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:39.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.672+0000 7f01c8bfb640 1 --2- 172.21.15.27:0/2876262275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c415d370 0x7f01c40b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.672+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067071961 0 0) 0x7f01c415d750 con 0x7f01c40b5160 2023-12-19T18:26:39.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.672+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01b4003520 con 0x7f01c40b5160 2023-12-19T18:26:39.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1434358362 0 0) 0x7f01b4003520 con 0x7f01c40b5160 2023-12-19T18:26:39.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c415d370 msgr2=0x7f01c40b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:39.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c37fe640 1 --2- 172.21.15.27:0/2876262275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c415d370 0x7f01c40b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 >> v1:172.21.15.192:6789/0 conn(0x7f01c40b55f0 legacy=0x7f01c40b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:39.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c40d1a30 con 0x7f01c40b5160 2023-12-19T18:26:39.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f01bc0019f0 con 0x7f01c40b5160 2023-12-19T18:26:39.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 >> v1:172.21.15.27:6789/0 conn(0x7f01c40b5160 legacy=0x7f01c40b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:39.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c37fe640 1 --2- 172.21.15.27:0/2876262275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c415d370 0x7f01b40039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c37fe640 1 --2- 172.21.15.27:0/2876262275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01b4003f20 0x7f01b40047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c37fe640 1 --2- 172.21.15.27:0/2876262275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01b4004d00 0x7f01b4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b400bac0 con 0x7f01b4004d00 2023-12-19T18:26:39.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b400bc30 con 0x7f01b4003f20 2023-12-19T18:26:39.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b400bdf0 con 0x7f01c415d370 2023-12-19T18:26:39.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.673+0000 7f01c93fc640 1 --2- 172.21.15.27:0/2876262275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01b4004d00 0x7f01b4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.674+0000 7f01c8bfb640 1 --2- 172.21.15.27:0/2876262275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c415d370 0x7f01b40039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.674+0000 7f01c3fff640 1 --2- 172.21.15.27:0/2876262275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01b4003f20 0x7f01b40047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.674+0000 7f01c93fc640 1 -- 172.21.15.27:0/2876262275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c415d370 msgr2=0x7f01b40039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:39.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.674+0000 7f01c93fc640 1 --2- 172.21.15.27:0/2876262275 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01c415d370 0x7f01b40039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.674+0000 7f01c93fc640 1 -- 172.21.15.27:0/2876262275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01b4003f20 msgr2=0x7f01b40047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:39.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.674+0000 7f01c93fc640 1 --2- 172.21.15.27:0/2876262275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01b4003f20 0x7f01b40047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.674+0000 7f01c93fc640 1 -- 172.21.15.27:0/2876262275 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b4003520 con 0x7f01b4004d00 2023-12-19T18:26:39.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.674+0000 7f01c3fff640 1 --2- 172.21.15.27:0/2876262275 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01b4003f20 0x7f01b40047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:39.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.675+0000 7f01c93fc640 1 --2- 172.21.15.27:0/2876262275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01b4004d00 0x7f01b4005180 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00a830 tx=0x7f01b40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9b3350e620acfa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:39.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.675+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc0072d0 con 0x7f01b4004d00 2023-12-19T18:26:39.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.675+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01bc02f070 con 0x7f01b4004d00 2023-12-19T18:26:39.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.675+0000 7f01c37fe640 1 -- 172.21.15.27:0/2876262275 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01bc0076f0 con 0x7f01b4004d00 2023-12-19T18:26:39.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.676+0000 7f01cae5f640 1 -- 172.21.15.27:0/2876262275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01b4004d00 msgr2=0x7f01b4005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:39.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.676+0000 7f01cae5f640 1 --2- 172.21.15.27:0/2876262275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01b4004d00 0x7f01b4005180 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00a830 tx=0x7f01b40057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.676+0000 7f01c93fc640 1 -- 172.21.15.27:0/2876262275 reap_dead start 2023-12-19T18:26:39.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.676+0000 7f01cae5f640 1 -- 172.21.15.27:0/2876262275 shutdown_connections 2023-12-19T18:26:39.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.676+0000 7f01cae5f640 1 -- 172.21.15.27:0/2876262275 >> 172.21.15.27:0/2876262275 conn(0x7f01c406a480 msgr2=0x7f01c406a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:39.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.676+0000 7f01cae5f640 1 -- 172.21.15.27:0/2876262275 shutdown_connections 2023-12-19T18:26:39.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.676+0000 7f01cae5f640 1 -- 172.21.15.27:0/2876262275 wait complete. 2023-12-19T18:26:39.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.678+0000 7f01cae5f640 1 Processor -- start 2023-12-19T18:26:39.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.678+0000 7f01cae5f640 1 -- start start 2023-12-19T18:26:39.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.679+0000 7f01cae5f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01b4004d00 0x7f01c40ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.679+0000 7f01cae5f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c41f5420 0x7f01c41e83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.679+0000 7f01cae5f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c415b450 0x7f01c415b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.679+0000 7f01cae5f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40cb300 con 0x7f01c41f5420 2023-12-19T18:26:39.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.679+0000 7f01cae5f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01c40cd8a0 con 0x7f01c415b450 2023-12-19T18:26:39.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.679+0000 7f01cae5f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01c415be10 con 0x7f01b4004d00 2023-12-19T18:26:39.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.679+0000 7f01c3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c41f5420 0x7f01c41e83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.679+0000 7f01c3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c41f5420 0x7f01c41e83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44026/0 (socket says 172.21.15.27:44026) 2023-12-19T18:26:39.680 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.679+0000 7f01c3fff640 1 -- 172.21.15.27:0/1041878180 learned_addr learned my addr 172.21.15.27:0/1041878180 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:39.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.680+0000 7f01c8bfb640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01b4004d00 0x7f01c40ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.680+0000 7f01c3fff640 1 -- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01b4004d00 msgr2=0x7f01c40ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:39.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.680+0000 7f01c3fff640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01b4004d00 0x7f01c40ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.680+0000 7f01c93fc640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c415b450 0x7f01c415b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.680+0000 7f01c3fff640 1 -- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c415b450 msgr2=0x7f01c415b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:39.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.680+0000 7f01c3fff640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c415b450 0x7f01c415b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:39.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.680+0000 7f01c3fff640 1 -- 172.21.15.27:0/1041878180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c40ca680 con 0x7f01c41f5420 2023-12-19T18:26:39.681 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.680+0000 7f01c3fff640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c41f5420 0x7f01c41e83b0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f01b000d7a0 tx=0x7f01b000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:39.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.680+0000 7f01c17fa640 1 -- 172.21.15.27:0/1041878180 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b0017070 con 0x7f01c41f5420 2023-12-19T18:26:39.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.681+0000 7f01c17fa640 1 -- 172.21.15.27:0/1041878180 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01b0005070 con 0x7f01c41f5420 2023-12-19T18:26:39.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.681+0000 7f01c17fa640 1 -- 172.21.15.27:0/1041878180 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01b0005420 con 0x7f01c41f5420 2023-12-19T18:26:39.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.681+0000 7f01cae5f640 1 -- 172.21.15.27:0/1041878180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01c40ca960 con 0x7f01c41f5420 2023-12-19T18:26:39.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.681+0000 7f01cae5f640 1 -- 172.21.15.27:0/1041878180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01c40caf20 con 0x7f01c41f5420 2023-12-19T18:26:39.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.682+0000 7f01cae5f640 1 -- 172.21.15.27:0/1041878180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c4068e90 con 0x7f01c41f5420 2023-12-19T18:26:39.682 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.683+0000 7f01c17fa640 1 -- 172.21.15.27:0/1041878180 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f01b0007030 con 0x7f01c41f5420 2023-12-19T18:26:39.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.684+0000 7f01c17fa640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f01a806a460 0x7f01a806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:39.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.685+0000 7f01c17fa640 1 -- 172.21.15.27:0/1041878180 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01b0088150 con 0x7f01c41f5420 2023-12-19T18:26:39.685 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.685+0000 7f01c8bfb640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f01a806a460 0x7f01a806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:39.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.686+0000 7f01c8bfb640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f01a806a460 0x7f01a806c920 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009a10 tx=0x7f01b8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:39.689 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.689+0000 7f01c17fa640 1 -- 172.21.15.27:0/1041878180 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f01b00537d0 con 0x7f01c41f5420 2023-12-19T18:26:39.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:39.924+0000 7f01cae5f640 1 -- 172.21.15.27:0/1041878180 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f01c40b4a60 con 0x7f01c41f5420 2023-12-19T18:26:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.513+0000 7f01c17fa640 1 -- 172.21.15.27:0/1041878180 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v76) v1 ==== 118+0+0 (secure 0 0 0) 0x7f01b0056df0 con 0x7f01c41f5420 2023-12-19T18:26:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.517+0000 7f01cae5f640 1 -- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f01a806a460 msgr2=0x7f01a806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.517+0000 7f01cae5f640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f01a806a460 0x7f01a806c920 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f01b8009a10 tx=0x7f01b8008040 comp rx=0 tx=0).stop 2023-12-19T18:26:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.517+0000 7f01cae5f640 1 -- 172.21.15.27:0/1041878180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c41f5420 msgr2=0x7f01c41e83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.517+0000 7f01cae5f640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c41f5420 0x7f01c41e83b0 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f01b000d7a0 tx=0x7f01b000dc60 comp rx=0 tx=0).stop 2023-12-19T18:26:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.518+0000 7f01cae5f640 1 -- 172.21.15.27:0/1041878180 shutdown_connections 2023-12-19T18:26:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.518+0000 7f01cae5f640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f01a806a460 0x7f01a806c920 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.518+0000 7f01cae5f640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01c415b450 0x7f01c415b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.518+0000 7f01cae5f640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01c41f5420 0x7f01c41e83b0 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.518+0000 7f01cae5f640 1 --2- 172.21.15.27:0/1041878180 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f01b4004d00 0x7f01c40ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:40.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.518+0000 7f01cae5f640 1 -- 172.21.15.27:0/1041878180 >> 172.21.15.27:0/1041878180 conn(0x7f01c406a480 msgr2=0x7f01c415c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:40.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.518+0000 7f01cae5f640 1 -- 172.21.15.27:0/1041878180 shutdown_connections 2023-12-19T18:26:40.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.518+0000 7f01cae5f640 1 -- 172.21.15.27:0/1041878180 wait complete. 2023-12-19T18:26:40.538 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:26:40.538 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:26:40.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.777+0000 7fd70ffff640 1 Processor -- start 2023-12-19T18:26:40.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.778+0000 7fd70ffff640 1 -- start start 2023-12-19T18:26:40.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.780+0000 7fd70ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd710158790 0x7fd7100ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:40.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.780+0000 7fd70ffff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd71015c030 con 0x7fd71015b9e0 2023-12-19T18:26:40.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.780+0000 7fd70ffff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7100c9030 con 0x7fd7100adc50 2023-12-19T18:26:40.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.780+0000 7fd70ffff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7100c99d0 con 0x7fd710158790 2023-12-19T18:26:40.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.780+0000 7fd70effd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd710158790 0x7fd7100ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:40.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.780+0000 7fd70f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd71015b9e0 0x7fd7100c3820 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:40022/0 (socket says 172.21.15.27:40022) 2023-12-19T18:26:40.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.780+0000 7fd70f7fe640 1 -- 172.21.15.27:0/866031608 learned_addr learned my addr 172.21.15.27:0/866031608 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:40.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.781+0000 7fd70effd640 1 -- 172.21.15.27:0/866031608 >> v1:172.21.15.192:6789/0 conn(0x7fd7100adc50 legacy=0x7fd71015b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:40.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.781+0000 7fd70effd640 1 -- 172.21.15.27:0/866031608 >> v1:172.21.15.27:6789/0 conn(0x7fd71015b9e0 legacy=0x7fd7100c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:40.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.781+0000 7fd70effd640 1 -- 172.21.15.27:0/866031608 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7100c9310 con 0x7fd710158790 2023-12-19T18:26:40.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.782+0000 7fd70effd640 1 --2- 172.21.15.27:0/866031608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710158790 0x7fd7100ad710 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fd700009dc0 tx=0x7fd70002be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59576a688eb99df2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:40.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.782+0000 7fd70e7fc640 1 -- 172.21.15.27:0/866031608 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70002cc40 con 0x7fd710158790 2023-12-19T18:26:40.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.782+0000 7fd70e7fc640 1 -- 172.21.15.27:0/866031608 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70002cda0 con 0x7fd710158790 2023-12-19T18:26:40.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.782+0000 7fd70e7fc640 1 -- 172.21.15.27:0/866031608 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd700034660 con 0x7fd710158790 2023-12-19T18:26:40.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.783+0000 7fd70ffff640 1 -- 172.21.15.27:0/866031608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710158790 msgr2=0x7fd7100ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:40.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.783+0000 7fd70ffff640 1 --2- 172.21.15.27:0/866031608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710158790 0x7fd7100ad710 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7fd700009dc0 tx=0x7fd70002be90 comp rx=0 tx=0).stop 2023-12-19T18:26:40.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.783+0000 7fd70ffff640 1 -- 172.21.15.27:0/866031608 shutdown_connections 2023-12-19T18:26:40.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.783+0000 7fd70ffff640 1 --2- 172.21.15.27:0/866031608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710158790 0x7fd7100ad710 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:40.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.783+0000 7fd70ffff640 1 -- 172.21.15.27:0/866031608 >> 172.21.15.27:0/866031608 conn(0x7fd7100b05b0 msgr2=0x7fd7100b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:40.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.783+0000 7fd70ffff640 1 -- 172.21.15.27:0/866031608 shutdown_connections 2023-12-19T18:26:40.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.783+0000 7fd70ffff640 1 -- 172.21.15.27:0/866031608 wait complete. 2023-12-19T18:26:40.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.785+0000 7fd70ffff640 1 Processor -- start 2023-12-19T18:26:40.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.785+0000 7fd70ffff640 1 -- start start 2023-12-19T18:26:40.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.786+0000 7fd70ffff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7100adc50 0x7fd7100b2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:40.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.786+0000 7fd70ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710158790 0x7fd7100b2dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:40.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.786+0000 7fd70ffff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd71015b9e0 0x7fd7100ca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:40.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.786+0000 7fd70ffff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100ae100 con 0x7fd710158790 2023-12-19T18:26:40.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.786+0000 7fd70ffff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100b3610 con 0x7fd71015b9e0 2023-12-19T18:26:40.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.786+0000 7fd70ffff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7100b38f0 con 0x7fd7100adc50 2023-12-19T18:26:40.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.786+0000 7fd7067fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710158790 0x7fd7100b2dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:40.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.787+0000 7fd7067fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710158790 0x7fd7100b2dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:44040/0 (socket says 172.21.15.27:44040) 2023-12-19T18:26:40.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.787+0000 7fd7067fc640 1 -- 172.21.15.27:0/3040613734 learned_addr learned my addr 172.21.15.27:0/3040613734 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:40.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.787+0000 7fd7067fc640 1 -- 172.21.15.27:0/3040613734 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7100adc50 msgr2=0x7fd7100b2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:40.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.787+0000 7fd7067fc640 1 --2- 172.21.15.27:0/3040613734 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7100adc50 0x7fd7100b2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:40.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.787+0000 7fd7067fc640 1 -- 172.21.15.27:0/3040613734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd71015b9e0 msgr2=0x7fd7100ca0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:40.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.787+0000 7fd7067fc640 1 --2- 172.21.15.27:0/3040613734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd71015b9e0 0x7fd7100ca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:40.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.787+0000 7fd7067fc640 1 -- 172.21.15.27:0/3040613734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700009a70 con 0x7fd710158790 2023-12-19T18:26:40.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.787+0000 7fd7067fc640 1 --2- 172.21.15.27:0/3040613734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710158790 0x7fd7100b2dd0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c430 tx=0x7fd6f800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:40.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.788+0000 7fd70cff9640 1 -- 172.21.15.27:0/3040613734 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8011790 con 0x7fd710158790 2023-12-19T18:26:40.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.788+0000 7fd70cff9640 1 -- 172.21.15.27:0/3040613734 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f80118f0 con 0x7fd710158790 2023-12-19T18:26:40.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.788+0000 7fd70cff9640 1 -- 172.21.15.27:0/3040613734 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f8011ce0 con 0x7fd710158790 2023-12-19T18:26:40.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.788+0000 7fd70ffff640 1 -- 172.21.15.27:0/3040613734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7100b3bd0 con 0x7fd710158790 2023-12-19T18:26:40.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.788+0000 7fd70ffff640 1 -- 172.21.15.27:0/3040613734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7100b4190 con 0x7fd710158790 2023-12-19T18:26:40.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.789+0000 7fd70ffff640 1 -- 172.21.15.27:0/3040613734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7100c9dd0 con 0x7fd710158790 2023-12-19T18:26:40.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.790+0000 7fd70cff9640 1 -- 172.21.15.27:0/3040613734 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd6f801c460 con 0x7fd710158790 2023-12-19T18:26:40.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.791+0000 7fd70cff9640 1 --2- 172.21.15.27:0/3040613734 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6f006a430 0x7fd6f006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:40.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.791+0000 7fd70cff9640 1 -- 172.21.15.27:0/3040613734 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6f8088920 con 0x7fd710158790 2023-12-19T18:26:40.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.792+0000 7fd70effd640 1 --2- 172.21.15.27:0/3040613734 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6f006a430 0x7fd6f006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:40.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.793+0000 7fd70effd640 1 --2- 172.21.15.27:0/3040613734 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6f006a430 0x7fd6f006c8f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd700036000 tx=0x7fd70002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:40.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:40.795+0000 7fd70cff9640 1 -- 172.21.15.27:0/3040613734 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd7100c9dd0 con 0x7fd710158790 2023-12-19T18:26:41.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:41.027+0000 7fd70ffff640 1 -- 172.21.15.27:0/3040613734 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fd7100c9dd0 con 0x7fd710158790 2023-12-19T18:26:42.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.022+0000 7fd70cff9640 1 -- 172.21.15.27:0/3040613734 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v77) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd7100c9dd0 con 0x7fd710158790 2023-12-19T18:26:42.026 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.026+0000 7fd70ffff640 1 -- 172.21.15.27:0/3040613734 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6f006a430 msgr2=0x7fd6f006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.026+0000 7fd70ffff640 1 --2- 172.21.15.27:0/3040613734 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6f006a430 0x7fd6f006c8f0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fd700036000 tx=0x7fd70002e000 comp rx=0 tx=0).stop 2023-12-19T18:26:42.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.026+0000 7fd70ffff640 1 -- 172.21.15.27:0/3040613734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710158790 msgr2=0x7fd7100b2dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.026+0000 7fd70ffff640 1 --2- 172.21.15.27:0/3040613734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710158790 0x7fd7100b2dd0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7fd6f800c430 tx=0x7fd6f800c8f0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.026+0000 7fd70ffff640 1 -- 172.21.15.27:0/3040613734 shutdown_connections 2023-12-19T18:26:42.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.026+0000 7fd70ffff640 1 --2- 172.21.15.27:0/3040613734 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6f006a430 0x7fd6f006c8f0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.026+0000 7fd70ffff640 1 --2- 172.21.15.27:0/3040613734 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd71015b9e0 0x7fd7100ca0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.026+0000 7fd70ffff640 1 --2- 172.21.15.27:0/3040613734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd710158790 0x7fd7100b2dd0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.026+0000 7fd70ffff640 1 --2- 172.21.15.27:0/3040613734 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7100adc50 0x7fd7100b2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.026+0000 7fd70ffff640 1 -- 172.21.15.27:0/3040613734 >> 172.21.15.27:0/3040613734 conn(0x7fd7100b05b0 msgr2=0x7fd710065910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:42.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.027+0000 7fd70ffff640 1 -- 172.21.15.27:0/3040613734 shutdown_connections 2023-12-19T18:26:42.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.027+0000 7fd70ffff640 1 -- 172.21.15.27:0/3040613734 wait complete. 2023-12-19T18:26:42.050 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:26:42.050 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:26:42.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.283+0000 7f3bf2a34640 1 Processor -- start 2023-12-19T18:26:42.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.284+0000 7f3bf2a34640 1 -- start start 2023-12-19T18:26:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.285+0000 7f3bf2a34640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3bec1604d0 0x7f3bec0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.285+0000 7f3bf2a34640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bec0d2350 0x7f3bec0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.285+0000 7f3bf2a34640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3bec0b5e90 con 0x7f3bec0d0300 2023-12-19T18:26:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.285+0000 7f3bf2a34640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bec0d2b30 con 0x7f3bec1604d0 2023-12-19T18:26:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.285+0000 7f3bf2a34640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0d2850 con 0x7f3bec0d2350 2023-12-19T18:26:42.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.286+0000 7f3bf1231640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bec0d2350 0x7f3bec0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.286+0000 7f3bf2233640 1 --1- >> v1:172.21.15.192:6789/0 conn(0x7f3bec0d0300 0x7f3bec0d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.192:6789/0 says I am v1:172.21.15.27:51096/0 (socket says 172.21.15.27:51096) 2023-12-19T18:26:42.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.286+0000 7f3bf2233640 1 -- 172.21.15.27:0/713156143 learned_addr learned my addr 172.21.15.27:0/713156143 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:42.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.286+0000 7f3bf1a32640 1 --2- 172.21.15.27:0/713156143 >> v2:172.21.15.192:3300/0 conn(0x7f3bec1604d0 0x7f3bec0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.287+0000 7f3bf0a30640 1 -- 172.21.15.27:0/713156143 <== mon.1 v1:172.21.15.192:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2803687034 0 0) 0x7f3bec0b5e90 con 0x7f3bec0d0300 2023-12-19T18:26:42.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.287+0000 7f3bf0a30640 1 -- 172.21.15.27:0/713156143 --> v1:172.21.15.192:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3bd4003520 con 0x7f3bec0d0300 2023-12-19T18:26:42.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.287+0000 7f3bf1a32640 1 -- 172.21.15.27:0/713156143 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bec0d2350 msgr2=0x7f3bec0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:42.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.287+0000 7f3bf1a32640 1 --2- 172.21.15.27:0/713156143 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bec0d2350 0x7f3bec0b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.287+0000 7f3bf1a32640 1 -- 172.21.15.27:0/713156143 >> v1:172.21.15.192:6789/0 conn(0x7f3bec0d0300 legacy=0x7f3bec0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.287+0000 7f3bf1a32640 1 -- 172.21.15.27:0/713156143 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bec0cded0 con 0x7f3bec1604d0 2023-12-19T18:26:42.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.288+0000 7f3bf1a32640 1 --2- 172.21.15.27:0/713156143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bec1604d0 0x7f3bec0b47e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009d90 tx=0x7f3bdc02beb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76e353e731a7799d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:42.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.289+0000 7f3bf0a30640 1 -- 172.21.15.27:0/713156143 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc02cc60 con 0x7f3bec1604d0 2023-12-19T18:26:42.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.289+0000 7f3bf0a30640 1 -- 172.21.15.27:0/713156143 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bdc02cdc0 con 0x7f3bec1604d0 2023-12-19T18:26:42.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.289+0000 7f3bf0a30640 1 -- 172.21.15.27:0/713156143 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bdc034660 con 0x7f3bec1604d0 2023-12-19T18:26:42.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.289+0000 7f3bf2a34640 1 -- 172.21.15.27:0/713156143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bec1604d0 msgr2=0x7f3bec0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.289+0000 7f3bf2a34640 1 --2- 172.21.15.27:0/713156143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bec1604d0 0x7f3bec0b47e0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc009d90 tx=0x7f3bdc02beb0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.289+0000 7f3bf2a34640 1 -- 172.21.15.27:0/713156143 shutdown_connections 2023-12-19T18:26:42.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.289+0000 7f3bf2a34640 1 --2- 172.21.15.27:0/713156143 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bec0d2350 0x7f3bec0b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.289+0000 7f3bf2a34640 1 --2- 172.21.15.27:0/713156143 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bec1604d0 0x7f3bec0b47e0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.289+0000 7f3bf2a34640 1 -- 172.21.15.27:0/713156143 >> 172.21.15.27:0/713156143 conn(0x7f3bec0cf7e0 msgr2=0x7f3bec0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:42.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.290+0000 7f3bf2a34640 1 -- 172.21.15.27:0/713156143 shutdown_connections 2023-12-19T18:26:42.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.290+0000 7f3bf2a34640 1 -- 172.21.15.27:0/713156143 wait complete. 2023-12-19T18:26:42.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.291+0000 7f3bf2a34640 1 Processor -- start 2023-12-19T18:26:42.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.292+0000 7f3bf2a34640 1 -- start start 2023-12-19T18:26:42.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.293+0000 7f3bf2a34640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bec0d0300 0x7f3bec15cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.293+0000 7f3bf2a34640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bec0d2350 0x7f3bec0c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.293+0000 7f3bf2a34640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bec1604d0 0x7f3bec150120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.293+0000 7f3bf2a34640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec1f2a50 con 0x7f3bec0d2350 2023-12-19T18:26:42.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.293+0000 7f3bf2a34640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0c6db0 con 0x7f3bec1604d0 2023-12-19T18:26:42.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.293+0000 7f3bf2a34640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bec0c8060 con 0x7f3bec0d0300 2023-12-19T18:26:42.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.293+0000 7f3bf1231640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bec0d2350 0x7f3bec0c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.293+0000 7f3bf1231640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bec0d2350 0x7f3bec0c6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35974/0 (socket says 172.21.15.27:35974) 2023-12-19T18:26:42.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.293+0000 7f3bf1231640 1 -- 172.21.15.27:0/2150476327 learned_addr learned my addr 172.21.15.27:0/2150476327 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:42.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.293+0000 7f3bf1a32640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bec0d0300 0x7f3bec15cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.293+0000 7f3bf2233640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bec1604d0 0x7f3bec150120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.294+0000 7f3bf1231640 1 -- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bec0d0300 msgr2=0x7f3bec15cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.294+0000 7f3bf1231640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bec0d0300 0x7f3bec15cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.294+0000 7f3bf1231640 1 -- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bec1604d0 msgr2=0x7f3bec150120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.294+0000 7f3bf1231640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bec1604d0 0x7f3bec150120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.294+0000 7f3bf1231640 1 -- 172.21.15.27:0/2150476327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bdc009a70 con 0x7f3bec0d2350 2023-12-19T18:26:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.294+0000 7f3bf1231640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bec0d2350 0x7f3bec0c6870 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e9b0 tx=0x7f3be000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.294+0000 7f3bea7fc640 1 -- 172.21.15.27:0/2150476327 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be0009670 con 0x7f3bec0d2350 2023-12-19T18:26:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.294+0000 7f3bea7fc640 1 -- 172.21.15.27:0/2150476327 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be000c9a0 con 0x7f3bec0d2350 2023-12-19T18:26:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.294+0000 7f3bea7fc640 1 -- 172.21.15.27:0/2150476327 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be000cd50 con 0x7f3bec0d2350 2023-12-19T18:26:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.294+0000 7f3bf2a34640 1 -- 172.21.15.27:0/2150476327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bec0c7d60 con 0x7f3bec0d2350 2023-12-19T18:26:42.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.295+0000 7f3bf2a34640 1 -- 172.21.15.27:0/2150476327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bec0c72c0 con 0x7f3bec0d2350 2023-12-19T18:26:42.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.296+0000 7f3bf2a34640 1 -- 172.21.15.27:0/2150476327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bec0c8210 con 0x7f3bec0d2350 2023-12-19T18:26:42.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.297+0000 7f3bea7fc640 1 -- 172.21.15.27:0/2150476327 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3be0013470 con 0x7f3bec0d2350 2023-12-19T18:26:42.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.298+0000 7f3bea7fc640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3bd806a460 0x7f3bd806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.298+0000 7f3bea7fc640 1 -- 172.21.15.27:0/2150476327 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3be0088b20 con 0x7f3bec0d2350 2023-12-19T18:26:42.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.298+0000 7f3bf1a32640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3bd806a460 0x7f3bd806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.299+0000 7f3bf1a32640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3bd806a460 0x7f3bd806c920 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc036000 tx=0x7f3bdc030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:42.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.302+0000 7f3bea7fc640 1 -- 172.21.15.27:0/2150476327 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3bec0c8210 con 0x7f3bec0d2350 2023-12-19T18:26:42.525 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.525+0000 7f3bf2a34640 1 -- 172.21.15.27:0/2150476327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bec066f80 con 0x7f3bec0d2350 2023-12-19T18:26:42.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.527+0000 7f3bea7fc640 1 -- 172.21.15.27:0/2150476327 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4719 (secure 0 0 0) 0x7f3be0017090 con 0x7f3bec0d2350 2023-12-19T18:26:42.527 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:42.527 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3563146395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3563146395},{"type":"v1","addr":"172.21.15.192:6836","nonce":3563146395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3445095528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3445095528},{"type":"v1","addr":"172.21.15.192:6837","nonce":3445095528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3536033217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3536033217},{"type":"v1","addr":"172.21.15.27:6833","nonce":3536033217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:26:38.483498+0000","modified":"2023-12-19T18:26:42.008767+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2895889621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2895889621},{"type":"v1","addr":"172.21.15.27:6835","nonce":2895889621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-19T18:26:42.527 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 77 2023-12-19T18:26:42.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.529+0000 7f3bf2a34640 1 -- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3bd806a460 msgr2=0x7f3bd806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.529+0000 7f3bf2a34640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3bd806a460 0x7f3bd806c920 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3bdc036000 tx=0x7f3bdc030000 comp rx=0 tx=0).stop 2023-12-19T18:26:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.529+0000 7f3bf2a34640 1 -- 172.21.15.27:0/2150476327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bec0d2350 msgr2=0x7f3bec0c6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.529+0000 7f3bf2a34640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bec0d2350 0x7f3bec0c6870 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f3be000e9b0 tx=0x7f3be000ee70 comp rx=0 tx=0).stop 2023-12-19T18:26:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.529+0000 7f3bf2a34640 1 -- 172.21.15.27:0/2150476327 shutdown_connections 2023-12-19T18:26:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.529+0000 7f3bf2a34640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3bd806a460 0x7f3bd806c920 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.529+0000 7f3bf2a34640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3bec1604d0 0x7f3bec150120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.529+0000 7f3bf2a34640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bec0d2350 0x7f3bec0c6870 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.529+0000 7f3bf2a34640 1 --2- 172.21.15.27:0/2150476327 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3bec0d0300 0x7f3bec15cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.529+0000 7f3bf2a34640 1 -- 172.21.15.27:0/2150476327 >> 172.21.15.27:0/2150476327 conn(0x7f3bec0cf7e0 msgr2=0x7f3bec0d09f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.529+0000 7f3bf2a34640 1 -- 172.21.15.27:0/2150476327 shutdown_connections 2023-12-19T18:26:42.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.530+0000 7f3bf2a34640 1 -- 172.21.15.27:0/2150476327 wait complete. 2023-12-19T18:26:42.546 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:26:42.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.766+0000 7fe1fbfff640 1 Processor -- start 2023-12-19T18:26:42.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.766+0000 7fe1fbfff640 1 -- start start 2023-12-19T18:26:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.768+0000 7fe1fbfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe1fc1553a0 0x7fe1fc0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.768+0000 7fe1fbfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc0afeb0 0x7fe1fc15e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.768+0000 7fe1fbfff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1fc0b0330 con 0x7fe1fc15ec60 2023-12-19T18:26:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.768+0000 7fe1fbfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1fc0cbcf0 con 0x7fe1fc1553a0 2023-12-19T18:26:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.768+0000 7fe1fbfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc15f0d0 con 0x7fe1fc0afeb0 2023-12-19T18:26:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.768+0000 7fe1fb7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe1fc15ec60 0x7fe1fc15b130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43776/0 (socket says 172.21.15.27:43776) 2023-12-19T18:26:42.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.768+0000 7fe1fb7fe640 1 -- 172.21.15.27:0/3185865345 learned_addr learned my addr 172.21.15.27:0/3185865345 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.768+0000 7fe1faffd640 1 --2- 172.21.15.27:0/3185865345 >> v2:172.21.15.192:3300/0 conn(0x7fe1fc1553a0 0x7fe1fc0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1f3fff640 1 --2- 172.21.15.27:0/3185865345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc0afeb0 0x7fe1fc15e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358736838 0 0) 0x7fe1fc0b0330 con 0x7fe1fc15ec60 2023-12-19T18:26:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1e8003520 con 0x7fe1fc15ec60 2023-12-19T18:26:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3028314848 0 0) 0x7fe1e8003520 con 0x7fe1fc15ec60 2023-12-19T18:26:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc0afeb0 msgr2=0x7fe1fc15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:42.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 --2- 172.21.15.27:0/3185865345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc0afeb0 0x7fe1fc15e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 >> v2:172.21.15.192:3300/0 conn(0x7fe1fc1553a0 msgr2=0x7fe1fc0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 --2- 172.21.15.27:0/3185865345 >> v2:172.21.15.192:3300/0 conn(0x7fe1fc1553a0 0x7fe1fc0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc0cb350 con 0x7fe1fc15ec60 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe1f40019f0 con 0x7fe1fc15ec60 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 >> v1:172.21.15.27:6789/0 conn(0x7fe1fc15ec60 legacy=0x7fe1fc15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 --2- 172.21.15.27:0/3185865345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1e80039a0 0x7fe1e8003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 --2- 172.21.15.27:0/3185865345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc0afeb0 0x7fe1e80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 --2- 172.21.15.27:0/3185865345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e8004d00 0x7fe1e8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc0cbcf0 con 0x7fe1e80039a0 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc15f0d0 con 0x7fe1e8004d00 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.769+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1e800b9b0 con 0x7fe1fc0afeb0 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.770+0000 7fe1faffd640 1 --2- 172.21.15.27:0/3185865345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1e80039a0 0x7fe1e8003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.770+0000 7fe1f3fff640 1 --2- 172.21.15.27:0/3185865345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc0afeb0 0x7fe1e80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.770+0000 7fe1fb7fe640 1 --2- 172.21.15.27:0/3185865345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e8004d00 0x7fe1e8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.770+0000 7fe1faffd640 1 -- 172.21.15.27:0/3185865345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc0afeb0 msgr2=0x7fe1e80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:42.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.770+0000 7fe1faffd640 1 --2- 172.21.15.27:0/3185865345 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc0afeb0 0x7fe1e80047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.770+0000 7fe1faffd640 1 -- 172.21.15.27:0/3185865345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e8004d00 msgr2=0x7fe1e8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:42.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.770+0000 7fe1faffd640 1 --2- 172.21.15.27:0/3185865345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e8004d00 0x7fe1e8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.770+0000 7fe1faffd640 1 -- 172.21.15.27:0/3185865345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8003520 con 0x7fe1e80039a0 2023-12-19T18:26:42.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.770+0000 7fe1fb7fe640 1 --2- 172.21.15.27:0/3185865345 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1e8004d00 0x7fe1e8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:42.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.771+0000 7fe1faffd640 1 --2- 172.21.15.27:0/3185865345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1e80039a0 0x7fe1e8003dc0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009ac0 tx=0x7fe1e80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cf4869f7fcf6d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:42.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.772+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec02c710 con 0x7fe1e80039a0 2023-12-19T18:26:42.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.772+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec02c870 con 0x7fe1e80039a0 2023-12-19T18:26:42.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.772+0000 7fe1fa7fc640 1 -- 172.21.15.27:0/3185865345 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec02cc00 con 0x7fe1e80039a0 2023-12-19T18:26:42.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.772+0000 7fe1fbfff640 1 -- 172.21.15.27:0/3185865345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1e80039a0 msgr2=0x7fe1e8003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.772+0000 7fe1fbfff640 1 --2- 172.21.15.27:0/3185865345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1e80039a0 0x7fe1e8003dc0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec009ac0 tx=0x7fe1e80057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.772+0000 7fe1fb7fe640 1 -- 172.21.15.27:0/3185865345 reap_dead start 2023-12-19T18:26:42.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.772+0000 7fe1fbfff640 1 -- 172.21.15.27:0/3185865345 shutdown_connections 2023-12-19T18:26:42.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.772+0000 7fe1fbfff640 1 -- 172.21.15.27:0/3185865345 >> 172.21.15.27:0/3185865345 conn(0x7fe1fc069c70 msgr2=0x7fe1fc06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:42.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.772+0000 7fe1fbfff640 1 -- 172.21.15.27:0/3185865345 shutdown_connections 2023-12-19T18:26:42.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.772+0000 7fe1fbfff640 1 -- 172.21.15.27:0/3185865345 wait complete. 2023-12-19T18:26:42.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.774+0000 7fe1fbfff640 1 Processor -- start 2023-12-19T18:26:42.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.774+0000 7fe1fbfff640 1 -- start start 2023-12-19T18:26:42.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.775+0000 7fe1fbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1e80039a0 0x7fe1fc1f2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.775+0000 7fe1fbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1fc1f5010 0x7fe1fc15d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.775+0000 7fe1fbfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc1f5490 0x7fe1fc0c4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.775+0000 7fe1fbfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc1f28b0 con 0x7fe1e80039a0 2023-12-19T18:26:42.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.775+0000 7fe1fbfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc15dda0 con 0x7fe1fc1f5010 2023-12-19T18:26:42.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.775+0000 7fe1fbfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1fc15e080 con 0x7fe1fc1f5490 2023-12-19T18:26:42.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.776+0000 7fe1faffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1e80039a0 0x7fe1fc1f2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.776+0000 7fe1faffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1e80039a0 0x7fe1fc1f2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35984/0 (socket says 172.21.15.27:35984) 2023-12-19T18:26:42.776 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.776+0000 7fe1faffd640 1 -- 172.21.15.27:0/2029551926 learned_addr learned my addr 172.21.15.27:0/2029551926 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:42.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.776+0000 7fe1f3fff640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1fc1f5010 0x7fe1fc15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.776+0000 7fe1fb7fe640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc1f5490 0x7fe1fc0c4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.776+0000 7fe1faffd640 1 -- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc1f5490 msgr2=0x7fe1fc0c4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.776+0000 7fe1faffd640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc1f5490 0x7fe1fc0c4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.776+0000 7fe1faffd640 1 -- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1fc1f5010 msgr2=0x7fe1fc15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.776+0000 7fe1faffd640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1fc1f5010 0x7fe1fc15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.776+0000 7fe1faffd640 1 -- 172.21.15.27:0/2029551926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1fc15e360 con 0x7fe1e80039a0 2023-12-19T18:26:42.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.776+0000 7fe1fb7fe640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc1f5490 0x7fe1fc0c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:42.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.777+0000 7fe1faffd640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1e80039a0 0x7fe1fc1f2370 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec02c2b0 tx=0x7fe1ec004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:42.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.777+0000 7fe1f8ff9640 1 -- 172.21.15.27:0/2029551926 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec003810 con 0x7fe1e80039a0 2023-12-19T18:26:42.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.777+0000 7fe1f8ff9640 1 -- 172.21.15.27:0/2029551926 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1ec02de90 con 0x7fe1e80039a0 2023-12-19T18:26:42.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.777+0000 7fe1fbfff640 1 -- 172.21.15.27:0/2029551926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1fc0cce50 con 0x7fe1e80039a0 2023-12-19T18:26:42.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.777+0000 7fe1f8ff9640 1 -- 172.21.15.27:0/2029551926 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1ec003b70 con 0x7fe1e80039a0 2023-12-19T18:26:42.779 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.777+0000 7fe1fbfff640 1 -- 172.21.15.27:0/2029551926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1fc0cc7e0 con 0x7fe1e80039a0 2023-12-19T18:26:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.778+0000 7fe1fbfff640 1 -- 172.21.15.27:0/2029551926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1fc04e840 con 0x7fe1e80039a0 2023-12-19T18:26:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.779+0000 7fe1f8ff9640 1 -- 172.21.15.27:0/2029551926 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe1ec003cd0 con 0x7fe1e80039a0 2023-12-19T18:26:42.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.780+0000 7fe1f8ff9640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe1dc06a430 0x7fe1dc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:42.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.780+0000 7fe1f8ff9640 1 -- 172.21.15.27:0/2029551926 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe1ec0aa7c0 con 0x7fe1e80039a0 2023-12-19T18:26:42.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.781+0000 7fe1f3fff640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe1dc06a430 0x7fe1dc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.782+0000 7fe1f3fff640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe1dc06a430 0x7fe1dc06c8f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009a10 tx=0x7fe1e4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:42.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.785+0000 7fe1f8ff9640 1 -- 172.21.15.27:0/2029551926 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe1ec075db0 con 0x7fe1e80039a0 2023-12-19T18:26:42.942 INFO:tasks.ceph.mds.c.smithi027.stderr:2023-12-19T18:26:42.942+0000 7f7c0cba1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:26:42.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.985+0000 7fe1fbfff640 1 -- 172.21.15.27:0/2029551926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1fc0af6a0 con 0x7fe1e80039a0 2023-12-19T18:26:42.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.988+0000 7fe1f8ff9640 1 -- 172.21.15.27:0/2029551926 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v185) v1 ==== 74+0+16112 (secure 0 0 0) 0x7fe1ec0793d0 con 0x7fe1e80039a0 2023-12-19T18:26:42.989 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:42.989 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":185,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:26:39.487566+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:26:36.428093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T18:26:37.523993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"185","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:26:42.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.992+0000 7fe1fbfff640 1 -- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe1dc06a430 msgr2=0x7fe1dc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.992+0000 7fe1fbfff640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe1dc06a430 0x7fe1dc06c8f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fe1e4009a10 tx=0x7fe1e4008040 comp rx=0 tx=0).stop 2023-12-19T18:26:42.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.992+0000 7fe1fbfff640 1 -- 172.21.15.27:0/2029551926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1e80039a0 msgr2=0x7fe1fc1f2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:42.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.992+0000 7fe1fbfff640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1e80039a0 0x7fe1fc1f2370 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fe1ec02c2b0 tx=0x7fe1ec004730 comp rx=0 tx=0).stop 2023-12-19T18:26:42.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.992+0000 7fe1fbfff640 1 -- 172.21.15.27:0/2029551926 shutdown_connections 2023-12-19T18:26:42.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.992+0000 7fe1fbfff640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe1dc06a430 0x7fe1dc06c8f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.992+0000 7fe1fbfff640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe1fc1f5490 0x7fe1fc0c4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.992+0000 7fe1fbfff640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe1fc1f5010 0x7fe1fc15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.992+0000 7fe1fbfff640 1 --2- 172.21.15.27:0/2029551926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1e80039a0 0x7fe1fc1f2370 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:42.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.992+0000 7fe1fbfff640 1 -- 172.21.15.27:0/2029551926 >> 172.21.15.27:0/2029551926 conn(0x7fe1fc069c70 msgr2=0x7fe1fc158d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:42.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.992+0000 7fe1fbfff640 1 -- 172.21.15.27:0/2029551926 shutdown_connections 2023-12-19T18:26:42.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:42.992+0000 7fe1fbfff640 1 -- 172.21.15.27:0/2029551926 wait complete. 2023-12-19T18:26:43.013 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:26:43.013 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:26:43.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.247+0000 7f7bfbdae640 1 Processor -- start 2023-12-19T18:26:43.247 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.247+0000 7f7bfbdae640 1 -- start start 2023-12-19T18:26:43.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.249+0000 7f7bfbdae640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf40ae340 0x7f7bf40ccf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:43.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.249+0000 7f7bfbdae640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7bf40ae720 0x7f7bf40b31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:43.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.249+0000 7f7bfbdae640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bf40cad80 con 0x7f7bf40b3720 2023-12-19T18:26:43.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.249+0000 7f7bfbdae640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bf40cb210 con 0x7f7bf40ae720 2023-12-19T18:26:43.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.249+0000 7f7bfbdae640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf40caa80 con 0x7f7bf40ae340 2023-12-19T18:26:43.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.249+0000 7f7bf9349640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7bf40ae720 0x7f7bf40b31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:43.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.249+0000 7f7bfa34b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7bf40b3720 0x7f7bf40c99d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43792/0 (socket says 172.21.15.27:43792) 2023-12-19T18:26:43.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.249+0000 7f7bfa34b640 1 -- 172.21.15.27:0/1142377535 learned_addr learned my addr 172.21.15.27:0/1142377535 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:43.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.250+0000 7f7bf9b4a640 1 --2- 172.21.15.27:0/1142377535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf40ae340 0x7f7bf40ccf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:43.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.250+0000 7f7bf9349640 1 -- 172.21.15.27:0/1142377535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf40ae340 msgr2=0x7f7bf40ccf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:43.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.250+0000 7f7bf9349640 1 --2- 172.21.15.27:0/1142377535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf40ae340 0x7f7bf40ccf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.250+0000 7f7bf9349640 1 -- 172.21.15.27:0/1142377535 >> v1:172.21.15.27:6789/0 conn(0x7f7bf40b3720 legacy=0x7f7bf40c99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:43.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.250+0000 7f7bf9349640 1 -- 172.21.15.27:0/1142377535 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bf40ced50 con 0x7f7bf40ae720 2023-12-19T18:26:43.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.251+0000 7f7bf9349640 1 --2- 172.21.15.27:0/1142377535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf40ae720 0x7f7bf40b31e0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7be40097a0 tx=0x7f7be402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e79fd757bf82e71 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:43.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.251+0000 7f7bf8b48640 1 -- 172.21.15.27:0/1142377535 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be402d040 con 0x7f7bf40ae720 2023-12-19T18:26:43.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.251+0000 7f7bf8b48640 1 -- 172.21.15.27:0/1142377535 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be4031810 con 0x7f7bf40ae720 2023-12-19T18:26:43.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.252+0000 7f7bf8b48640 1 -- 172.21.15.27:0/1142377535 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be4031ba0 con 0x7f7bf40ae720 2023-12-19T18:26:43.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.252+0000 7f7bfbdae640 1 -- 172.21.15.27:0/1142377535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf40ae720 msgr2=0x7f7bf40b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:43.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.252+0000 7f7bfbdae640 1 --2- 172.21.15.27:0/1142377535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf40ae720 0x7f7bf40b31e0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f7be40097a0 tx=0x7f7be402b8b0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.252+0000 7f7bfbdae640 1 -- 172.21.15.27:0/1142377535 shutdown_connections 2023-12-19T18:26:43.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.252+0000 7f7bfbdae640 1 --2- 172.21.15.27:0/1142377535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf40ae720 0x7f7bf40b31e0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.252+0000 7f7bfbdae640 1 --2- 172.21.15.27:0/1142377535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf40ae340 0x7f7bf40ccf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.252+0000 7f7bfbdae640 1 -- 172.21.15.27:0/1142377535 >> 172.21.15.27:0/1142377535 conn(0x7f7bf40cbe00 msgr2=0x7f7bf40cc1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:43.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.252+0000 7f7bfbdae640 1 -- 172.21.15.27:0/1142377535 shutdown_connections 2023-12-19T18:26:43.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.252+0000 7f7bfbdae640 1 -- 172.21.15.27:0/1142377535 wait complete. 2023-12-19T18:26:43.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.254+0000 7f7bfbdae640 1 Processor -- start 2023-12-19T18:26:43.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.254+0000 7f7bfbdae640 1 -- start start 2023-12-19T18:26:43.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.255+0000 7f7bfbdae640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf40ae340 0x7f7bf40c2eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:43.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.255+0000 7f7bfbdae640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf40ae720 0x7f7bf40c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:43.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.255+0000 7f7bfbdae640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf40b3720 0x7f7bf4099a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:43.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.255+0000 7f7bfbdae640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf409a7f0 con 0x7f7bf40b3720 2023-12-19T18:26:43.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.255+0000 7f7bfbdae640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf41585b0 con 0x7f7bf40ae340 2023-12-19T18:26:43.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.255+0000 7f7bfbdae640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bf40c5240 con 0x7f7bf40ae720 2023-12-19T18:26:43.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.256+0000 7f7bfa34b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf40b3720 0x7f7bf4099a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:43.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.256+0000 7f7bfa34b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf40b3720 0x7f7bf4099a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36012/0 (socket says 172.21.15.27:36012) 2023-12-19T18:26:43.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.256+0000 7f7bfa34b640 1 -- 172.21.15.27:0/3439398966 learned_addr learned my addr 172.21.15.27:0/3439398966 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:43.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.256+0000 7f7bf9349640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf40ae720 0x7f7bf40c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:43.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.256+0000 7f7bf9b4a640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf40ae340 0x7f7bf40c2eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:43.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.256+0000 7f7bfa34b640 1 -- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf40ae720 msgr2=0x7f7bf40c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:43.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.256+0000 7f7bfa34b640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf40ae720 0x7f7bf40c4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.256+0000 7f7bfa34b640 1 -- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf40ae340 msgr2=0x7f7bf40c2eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:43.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.257+0000 7f7bfa34b640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf40ae340 0x7f7bf40c2eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.257+0000 7f7bfa34b640 1 -- 172.21.15.27:0/3439398966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be4009400 con 0x7f7bf40b3720 2023-12-19T18:26:43.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.257+0000 7f7bfa34b640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf40b3720 0x7f7bf4099a50 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00cbf0 tx=0x7f7bec00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:43.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.257+0000 7f7bea7fc640 1 -- 172.21.15.27:0/3439398966 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec00ad40 con 0x7f7bf40b3720 2023-12-19T18:26:43.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.257+0000 7f7bea7fc640 1 -- 172.21.15.27:0/3439398966 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bec012400 con 0x7f7bf40b3720 2023-12-19T18:26:43.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.257+0000 7f7bea7fc640 1 -- 172.21.15.27:0/3439398966 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bec00aea0 con 0x7f7bf40b3720 2023-12-19T18:26:43.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.257+0000 7f7bfbdae640 1 -- 172.21.15.27:0/3439398966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bf409a510 con 0x7f7bf40b3720 2023-12-19T18:26:43.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.257+0000 7f7bfbdae640 1 -- 172.21.15.27:0/3439398966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bf4158240 con 0x7f7bf40b3720 2023-12-19T18:26:43.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.258+0000 7f7bfbdae640 1 -- 172.21.15.27:0/3439398966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bf4068e90 con 0x7f7bf40b3720 2023-12-19T18:26:43.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.259+0000 7f7bea7fc640 1 -- 172.21.15.27:0/3439398966 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7bec007070 con 0x7f7bf40b3720 2023-12-19T18:26:43.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.261+0000 7f7bea7fc640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7bd806a430 0x7f7bd806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:43.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.261+0000 7f7bea7fc640 1 -- 172.21.15.27:0/3439398966 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7bec088740 con 0x7f7bf40b3720 2023-12-19T18:26:43.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.262+0000 7f7bf9b4a640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7bd806a430 0x7f7bd806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:43.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.263+0000 7f7bf9b4a640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7bd806a430 0x7f7bd806c8f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0006fd0 tx=0x7f7bf0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:43.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.265+0000 7f7bea7fc640 1 -- 172.21.15.27:0/3439398966 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7bec053e70 con 0x7f7bf40b3720 2023-12-19T18:26:43.454 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.454+0000 7f7bfbdae640 1 -- 172.21.15.27:0/3439398966 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7bf40cf030 con 0x7f7bd806a430 2023-12-19T18:26:43.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.478+0000 7f7bea7fc640 1 -- 172.21.15.27:0/3439398966 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282235 (secure 0 0 0) 0x7f7bf000f860 con 0x7f7bd806a430 2023-12-19T18:26:43.482 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:43.484 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:26:43.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.485+0000 7f7bfbdae640 1 -- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7bd806a430 msgr2=0x7f7bd806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:43.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.485+0000 7f7bfbdae640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7bd806a430 0x7f7bd806c8f0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f7bf0006fd0 tx=0x7f7bf0008040 comp rx=0 tx=0).stop 2023-12-19T18:26:43.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.485+0000 7f7bfbdae640 1 -- 172.21.15.27:0/3439398966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf40b3720 msgr2=0x7f7bf4099a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:43.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.485+0000 7f7bfbdae640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf40b3720 0x7f7bf4099a50 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f7bec00cbf0 tx=0x7f7bec00a430 comp rx=0 tx=0).stop 2023-12-19T18:26:43.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.485+0000 7f7bfbdae640 1 -- 172.21.15.27:0/3439398966 shutdown_connections 2023-12-19T18:26:43.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.485+0000 7f7bfbdae640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7bd806a430 0x7f7bd806c8f0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.485+0000 7f7bfbdae640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bf40b3720 0x7f7bf4099a50 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.485+0000 7f7bfbdae640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7bf40ae720 0x7f7bf40c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.485+0000 7f7bfbdae640 1 --2- 172.21.15.27:0/3439398966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7bf40ae340 0x7f7bf40c2eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.485+0000 7f7bfbdae640 1 -- 172.21.15.27:0/3439398966 >> 172.21.15.27:0/3439398966 conn(0x7f7bf40cbe00 msgr2=0x7f7bf406b730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:43.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.485+0000 7f7bfbdae640 1 -- 172.21.15.27:0/3439398966 shutdown_connections 2023-12-19T18:26:43.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.485+0000 7f7bfbdae640 1 -- 172.21.15.27:0/3439398966 wait complete. 2023-12-19T18:26:43.507 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":536,"stamp":"2023-12-19T18:26:41.965763+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":42,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9210,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553289,"statfs":{"total":1600000000,"available":1590568090,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.998660"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.516731+0000","last_change":"2023-12-19T18:26:38.511736+0000","last_active":"2023-12-19T18:26:38.516731+0000","last_peered":"2023-12-19T18:26:38.516731+0000","last_clean":"2023-12-19T18:26:38.516731+0000","last_became_active":"2023-12-19T18:26:38.508958+0000","last_became_peered":"2023-12-19T18:26:38.508958+0000","last_unstale":"2023-12-19T18:26:38.516731+0000","last_undegraded":"2023-12-19T18:26:38.516731+0000","last_fullsized":"2023-12-19T18:26:38.516731+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:34.370730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.524557+0000","last_change":"2023-12-19T18:26:38.523848+0000","last_active":"2023-12-19T18:26:38.524557+0000","last_peered":"2023-12-19T18:26:38.524557+0000","last_clean":"2023-12-19T18:26:38.524557+0000","last_became_active":"2023-12-19T18:26:38.523619+0000","last_became_peered":"2023-12-19T18:26:38.523619+0000","last_unstale":"2023-12-19T18:26:38.524557+0000","last_undegraded":"2023-12-19T18:26:38.524557+0000","last_fullsized":"2023-12-19T18:26:38.524557+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:47:44.420268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.513114+0000","last_change":"2023-12-19T18:26:38.512087+0000","last_active":"2023-12-19T18:26:38.513114+0000","last_peered":"2023-12-19T18:26:38.513114+0000","last_clean":"2023-12-19T18:26:38.513114+0000","last_became_active":"2023-12-19T18:26:38.510831+0000","last_became_peered":"2023-12-19T18:26:38.510831+0000","last_unstale":"2023-12-19T18:26:38.513114+0000","last_undegraded":"2023-12-19T18:26:38.513114+0000","last_fullsized":"2023-12-19T18:26:38.513114+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:29:48.773763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.514642+0000","last_change":"2023-12-19T18:26:38.512424+0000","last_active":"2023-12-19T18:26:38.514642+0000","last_peered":"2023-12-19T18:26:38.514642+0000","last_clean":"2023-12-19T18:26:38.514642+0000","last_became_active":"2023-12-19T18:26:38.512202+0000","last_became_peered":"2023-12-19T18:26:38.512202+0000","last_unstale":"2023-12-19T18:26:38.514642+0000","last_undegraded":"2023-12-19T18:26:38.514642+0000","last_fullsized":"2023-12-19T18:26:38.514642+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:27:22.514431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.507855+0000","last_change":"2023-12-19T18:26:38.507304+0000","last_active":"2023-12-19T18:26:38.507855+0000","last_peered":"2023-12-19T18:26:38.507855+0000","last_clean":"2023-12-19T18:26:38.507855+0000","last_became_active":"2023-12-19T18:26:38.507063+0000","last_became_peered":"2023-12-19T18:26:38.507063+0000","last_unstale":"2023-12-19T18:26:38.507855+0000","last_undegraded":"2023-12-19T18:26:38.507855+0000","last_fullsized":"2023-12-19T18:26:38.507855+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:25:11.204756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.521639+0000","last_change":"2023-12-19T18:26:38.518271+0000","last_active":"2023-12-19T18:26:38.521639+0000","last_peered":"2023-12-19T18:26:38.521639+0000","last_clean":"2023-12-19T18:26:38.521639+0000","last_became_active":"2023-12-19T18:26:38.517107+0000","last_became_peered":"2023-12-19T18:26:38.517107+0000","last_unstale":"2023-12-19T18:26:38.521639+0000","last_undegraded":"2023-12-19T18:26:38.521639+0000","last_fullsized":"2023-12-19T18:26:38.521639+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:07:27.156222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.516979+0000","last_change":"2023-12-19T18:26:38.511992+0000","last_active":"2023-12-19T18:26:38.516979+0000","last_peered":"2023-12-19T18:26:38.516979+0000","last_clean":"2023-12-19T18:26:38.516979+0000","last_became_active":"2023-12-19T18:26:38.511518+0000","last_became_peered":"2023-12-19T18:26:38.511518+0000","last_unstale":"2023-12-19T18:26:38.516979+0000","last_undegraded":"2023-12-19T18:26:38.516979+0000","last_fullsized":"2023-12-19T18:26:38.516979+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:09.636074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.520517+0000","last_change":"2023-12-19T18:26:38.520362+0000","last_active":"2023-12-19T18:26:38.520517+0000","last_peered":"2023-12-19T18:26:38.520517+0000","last_clean":"2023-12-19T18:26:38.520517+0000","last_became_active":"2023-12-19T18:26:38.519835+0000","last_became_peered":"2023-12-19T18:26:38.519835+0000","last_unstale":"2023-12-19T18:26:38.520517+0000","last_undegraded":"2023-12-19T18:26:38.520517+0000","last_fullsized":"2023-12-19T18:26:38.520517+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:22:24.538127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.521663+0000","last_change":"2023-12-19T18:26:38.520612+0000","last_active":"2023-12-19T18:26:38.521663+0000","last_peered":"2023-12-19T18:26:38.521663+0000","last_clean":"2023-12-19T18:26:38.521663+0000","last_became_active":"2023-12-19T18:26:38.520138+0000","last_became_peered":"2023-12-19T18:26:38.520138+0000","last_unstale":"2023-12-19T18:26:38.521663+0000","last_undegraded":"2023-12-19T18:26:38.521663+0000","last_fullsized":"2023-12-19T18:26:38.521663+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:13.074404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.503412+0000","last_change":"2023-12-19T18:26:38.502557+0000","last_active":"2023-12-19T18:26:38.503412+0000","last_peered":"2023-12-19T18:26:38.503412+0000","last_clean":"2023-12-19T18:26:38.503412+0000","last_became_active":"2023-12-19T18:26:38.502172+0000","last_became_peered":"2023-12-19T18:26:38.502172+0000","last_unstale":"2023-12-19T18:26:38.503412+0000","last_undegraded":"2023-12-19T18:26:38.503412+0000","last_fullsized":"2023-12-19T18:26:38.503412+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:21:44.849567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.513800+0000","last_change":"2023-12-19T18:26:38.513600+0000","last_active":"2023-12-19T18:26:38.513800+0000","last_peered":"2023-12-19T18:26:38.513800+0000","last_clean":"2023-12-19T18:26:38.513800+0000","last_became_active":"2023-12-19T18:26:38.512761+0000","last_became_peered":"2023-12-19T18:26:38.512761+0000","last_unstale":"2023-12-19T18:26:38.513800+0000","last_undegraded":"2023-12-19T18:26:38.513800+0000","last_fullsized":"2023-12-19T18:26:38.513800+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:39:34.284978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.514455+0000","last_change":"2023-12-19T18:26:38.514384+0000","last_active":"2023-12-19T18:26:38.514455+0000","last_peered":"2023-12-19T18:26:38.514455+0000","last_clean":"2023-12-19T18:26:38.514455+0000","last_became_active":"2023-12-19T18:26:38.512957+0000","last_became_peered":"2023-12-19T18:26:38.512957+0000","last_unstale":"2023-12-19T18:26:38.514455+0000","last_undegraded":"2023-12-19T18:26:38.514455+0000","last_fullsized":"2023-12-19T18:26:38.514455+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:13.074404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.509114+0000","last_change":"2023-12-19T18:26:38.508388+0000","last_active":"2023-12-19T18:26:38.509114+0000","last_peered":"2023-12-19T18:26:38.509114+0000","last_clean":"2023-12-19T18:26:38.509114+0000","last_became_active":"2023-12-19T18:26:38.508159+0000","last_became_peered":"2023-12-19T18:26:38.508159+0000","last_unstale":"2023-12-19T18:26:38.509114+0000","last_undegraded":"2023-12-19T18:26:38.509114+0000","last_fullsized":"2023-12-19T18:26:38.509114+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:57:41.432563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.503039+0000","last_change":"2023-12-19T18:26:38.500039+0000","last_active":"2023-12-19T18:26:38.503039+0000","last_peered":"2023-12-19T18:26:38.503039+0000","last_clean":"2023-12-19T18:26:38.503039+0000","last_became_active":"2023-12-19T18:26:38.499817+0000","last_became_peered":"2023-12-19T18:26:38.499817+0000","last_unstale":"2023-12-19T18:26:38.503039+0000","last_undegraded":"2023-12-19T18:26:38.503039+0000","last_fullsized":"2023-12-19T18:26:38.503039+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:09:57.453540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.505390+0000","last_change":"2023-12-19T18:26:38.504914+0000","last_active":"2023-12-19T18:26:38.505390+0000","last_peered":"2023-12-19T18:26:38.505390+0000","last_clean":"2023-12-19T18:26:38.505390+0000","last_became_active":"2023-12-19T18:26:38.504653+0000","last_became_peered":"2023-12-19T18:26:38.504653+0000","last_unstale":"2023-12-19T18:26:38.505390+0000","last_undegraded":"2023-12-19T18:26:38.505390+0000","last_fullsized":"2023-12-19T18:26:38.505390+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:57.647894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.524215+0000","last_change":"2023-12-19T18:26:38.522909+0000","last_active":"2023-12-19T18:26:38.524215+0000","last_peered":"2023-12-19T18:26:38.524215+0000","last_clean":"2023-12-19T18:26:38.524215+0000","last_became_active":"2023-12-19T18:26:38.522655+0000","last_became_peered":"2023-12-19T18:26:38.522655+0000","last_unstale":"2023-12-19T18:26:38.524215+0000","last_undegraded":"2023-12-19T18:26:38.524215+0000","last_fullsized":"2023-12-19T18:26:38.524215+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:59:50.875803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.511276+0000","last_change":"2023-12-19T18:26:38.507509+0000","last_active":"2023-12-19T18:26:38.511276+0000","last_peered":"2023-12-19T18:26:38.511276+0000","last_clean":"2023-12-19T18:26:38.511276+0000","last_became_active":"2023-12-19T18:26:38.507230+0000","last_became_peered":"2023-12-19T18:26:38.507230+0000","last_unstale":"2023-12-19T18:26:38.511276+0000","last_undegraded":"2023-12-19T18:26:38.511276+0000","last_fullsized":"2023-12-19T18:26:38.511276+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:44:57.712706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.513311+0000","last_change":"2023-12-19T18:26:38.512437+0000","last_active":"2023-12-19T18:26:38.513311+0000","last_peered":"2023-12-19T18:26:38.513311+0000","last_clean":"2023-12-19T18:26:38.513311+0000","last_became_active":"2023-12-19T18:26:38.512125+0000","last_became_peered":"2023-12-19T18:26:38.512125+0000","last_unstale":"2023-12-19T18:26:38.513311+0000","last_undegraded":"2023-12-19T18:26:38.513311+0000","last_fullsized":"2023-12-19T18:26:38.513311+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:49.646820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.523505+0000","last_change":"2023-12-19T18:26:38.522323+0000","last_active":"2023-12-19T18:26:38.523505+0000","last_peered":"2023-12-19T18:26:38.523505+0000","last_clean":"2023-12-19T18:26:38.523505+0000","last_became_active":"2023-12-19T18:26:38.514076+0000","last_became_peered":"2023-12-19T18:26:38.514076+0000","last_unstale":"2023-12-19T18:26:38.523505+0000","last_undegraded":"2023-12-19T18:26:38.523505+0000","last_fullsized":"2023-12-19T18:26:38.523505+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:15:59.240438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.524362+0000","last_change":"2023-12-19T18:26:38.524051+0000","last_active":"2023-12-19T18:26:38.524362+0000","last_peered":"2023-12-19T18:26:38.524362+0000","last_clean":"2023-12-19T18:26:38.524362+0000","last_became_active":"2023-12-19T18:26:38.523668+0000","last_became_peered":"2023-12-19T18:26:38.523668+0000","last_unstale":"2023-12-19T18:26:38.524362+0000","last_undegraded":"2023-12-19T18:26:38.524362+0000","last_fullsized":"2023-12-19T18:26:38.524362+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:16:40.828355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.513275+0000","last_change":"2023-12-19T18:26:38.513151+0000","last_active":"2023-12-19T18:26:38.513275+0000","last_peered":"2023-12-19T18:26:38.513275+0000","last_clean":"2023-12-19T18:26:38.513275+0000","last_became_active":"2023-12-19T18:26:38.511855+0000","last_became_peered":"2023-12-19T18:26:38.511855+0000","last_unstale":"2023-12-19T18:26:38.513275+0000","last_undegraded":"2023-12-19T18:26:38.513275+0000","last_fullsized":"2023-12-19T18:26:38.513275+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:23.624624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.505845+0000","last_change":"2023-12-19T18:26:38.505779+0000","last_active":"2023-12-19T18:26:38.505845+0000","last_peered":"2023-12-19T18:26:38.505845+0000","last_clean":"2023-12-19T18:26:38.505845+0000","last_became_active":"2023-12-19T18:26:38.505432+0000","last_became_peered":"2023-12-19T18:26:38.505432+0000","last_unstale":"2023-12-19T18:26:38.505845+0000","last_undegraded":"2023-12-19T18:26:38.505845+0000","last_fullsized":"2023-12-19T18:26:38.505845+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:39:34.284978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.515191+0000","last_change":"2023-12-19T18:26:38.514929+0000","last_active":"2023-12-19T18:26:38.515191+0000","last_peered":"2023-12-19T18:26:38.515191+0000","last_clean":"2023-12-19T18:26:38.515191+0000","last_became_active":"2023-12-19T18:26:38.514200+0000","last_became_peered":"2023-12-19T18:26:38.514200+0000","last_unstale":"2023-12-19T18:26:38.515191+0000","last_undegraded":"2023-12-19T18:26:38.515191+0000","last_fullsized":"2023-12-19T18:26:38.515191+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:17:35.926413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.503683+0000","last_change":"2023-12-19T18:26:38.501316+0000","last_active":"2023-12-19T18:26:38.503683+0000","last_peered":"2023-12-19T18:26:38.503683+0000","last_clean":"2023-12-19T18:26:38.503683+0000","last_became_active":"2023-12-19T18:26:38.501052+0000","last_became_peered":"2023-12-19T18:26:38.501052+0000","last_unstale":"2023-12-19T18:26:38.503683+0000","last_undegraded":"2023-12-19T18:26:38.503683+0000","last_fullsized":"2023-12-19T18:26:38.503683+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:55:51.941470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.525944+0000","last_change":"2023-12-19T18:26:38.525353+0000","last_active":"2023-12-19T18:26:38.525944+0000","last_peered":"2023-12-19T18:26:38.525944+0000","last_clean":"2023-12-19T18:26:38.525944+0000","last_became_active":"2023-12-19T18:26:38.524781+0000","last_became_peered":"2023-12-19T18:26:38.524781+0000","last_unstale":"2023-12-19T18:26:38.525944+0000","last_undegraded":"2023-12-19T18:26:38.525944+0000","last_fullsized":"2023-12-19T18:26:38.525944+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:30.507693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.521579+0000","last_change":"2023-12-19T18:26:38.521257+0000","last_active":"2023-12-19T18:26:38.521579+0000","last_peered":"2023-12-19T18:26:38.521579+0000","last_clean":"2023-12-19T18:26:38.521579+0000","last_became_active":"2023-12-19T18:26:38.521027+0000","last_became_peered":"2023-12-19T18:26:38.521027+0000","last_unstale":"2023-12-19T18:26:38.521579+0000","last_undegraded":"2023-12-19T18:26:38.521579+0000","last_fullsized":"2023-12-19T18:26:38.521579+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:13:51.785468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.526029+0000","last_change":"2023-12-19T18:26:38.525961+0000","last_active":"2023-12-19T18:26:38.526029+0000","last_peered":"2023-12-19T18:26:38.526029+0000","last_clean":"2023-12-19T18:26:38.526029+0000","last_became_active":"2023-12-19T18:26:38.525681+0000","last_became_peered":"2023-12-19T18:26:38.525681+0000","last_unstale":"2023-12-19T18:26:38.526029+0000","last_undegraded":"2023-12-19T18:26:38.526029+0000","last_fullsized":"2023-12-19T18:26:38.526029+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:09:57.453540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.524575+0000","last_change":"2023-12-19T18:26:38.524304+0000","last_active":"2023-12-19T18:26:38.524575+0000","last_peered":"2023-12-19T18:26:38.524575+0000","last_clean":"2023-12-19T18:26:38.524575+0000","last_became_active":"2023-12-19T18:26:38.524054+0000","last_became_peered":"2023-12-19T18:26:38.524054+0000","last_unstale":"2023-12-19T18:26:38.524575+0000","last_undegraded":"2023-12-19T18:26:38.524575+0000","last_fullsized":"2023-12-19T18:26:38.524575+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:38:02.035441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.513506+0000","last_change":"2023-12-19T18:26:38.509646+0000","last_active":"2023-12-19T18:26:38.513506+0000","last_peered":"2023-12-19T18:26:38.513506+0000","last_clean":"2023-12-19T18:26:38.513506+0000","last_became_active":"2023-12-19T18:26:38.509086+0000","last_became_peered":"2023-12-19T18:26:38.509086+0000","last_unstale":"2023-12-19T18:26:38.513506+0000","last_undegraded":"2023-12-19T18:26:38.513506+0000","last_fullsized":"2023-12-19T18:26:38.513506+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:31.383005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.511292+0000","last_change":"2023-12-19T18:26:38.510763+0000","last_active":"2023-12-19T18:26:38.511292+0000","last_peered":"2023-12-19T18:26:38.511292+0000","last_clean":"2023-12-19T18:26:38.511292+0000","last_became_active":"2023-12-19T18:26:38.503707+0000","last_became_peered":"2023-12-19T18:26:38.503707+0000","last_unstale":"2023-12-19T18:26:38.511292+0000","last_undegraded":"2023-12-19T18:26:38.511292+0000","last_fullsized":"2023-12-19T18:26:38.511292+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:57:04.171752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.521122+0000","last_change":"2023-12-19T18:26:38.520939+0000","last_active":"2023-12-19T18:26:38.521122+0000","last_peered":"2023-12-19T18:26:38.521122+0000","last_clean":"2023-12-19T18:26:38.521122+0000","last_became_active":"2023-12-19T18:26:38.520690+0000","last_became_peered":"2023-12-19T18:26:38.520690+0000","last_unstale":"2023-12-19T18:26:38.521122+0000","last_undegraded":"2023-12-19T18:26:38.521122+0000","last_fullsized":"2023-12-19T18:26:38.521122+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:53.886763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.520449+0000","last_change":"2023-12-19T18:26:38.520370+0000","last_active":"2023-12-19T18:26:38.520449+0000","last_peered":"2023-12-19T18:26:38.520449+0000","last_clean":"2023-12-19T18:26:38.520449+0000","last_became_active":"2023-12-19T18:26:38.520135+0000","last_became_peered":"2023-12-19T18:26:38.520135+0000","last_unstale":"2023-12-19T18:26:38.520449+0000","last_undegraded":"2023-12-19T18:26:38.520449+0000","last_fullsized":"2023-12-19T18:26:38.520449+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:31.201464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.510520+0000","last_change":"2023-12-19T18:26:38.510459+0000","last_active":"2023-12-19T18:26:38.510520+0000","last_peered":"2023-12-19T18:26:38.510520+0000","last_clean":"2023-12-19T18:26:38.510520+0000","last_became_active":"2023-12-19T18:26:38.510258+0000","last_became_peered":"2023-12-19T18:26:38.510258+0000","last_unstale":"2023-12-19T18:26:38.510520+0000","last_undegraded":"2023-12-19T18:26:38.510520+0000","last_fullsized":"2023-12-19T18:26:38.510520+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:57.578030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"184'4","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.521107+0000","last_change":"2023-12-19T18:26:38.000068+0000","last_active":"2023-12-19T18:26:38.521107+0000","last_peered":"2023-12-19T18:26:38.521107+0000","last_clean":"2023-12-19T18:26:38.521107+0000","last_became_active":"2023-12-19T18:26:37.999319+0000","last_became_peered":"2023-12-19T18:26:37.999319+0000","last_unstale":"2023-12-19T18:26:38.521107+0000","last_undegraded":"2023-12-19T18:26:38.521107+0000","last_fullsized":"2023-12-19T18:26:38.521107+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:19:49.762606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.505204+0000","last_change":"2023-12-19T18:26:38.504737+0000","last_active":"2023-12-19T18:26:38.505204+0000","last_peered":"2023-12-19T18:26:38.505204+0000","last_clean":"2023-12-19T18:26:38.505204+0000","last_became_active":"2023-12-19T18:26:38.502873+0000","last_became_peered":"2023-12-19T18:26:38.502873+0000","last_unstale":"2023-12-19T18:26:38.505204+0000","last_undegraded":"2023-12-19T18:26:38.505204+0000","last_fullsized":"2023-12-19T18:26:38.505204+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:42.815725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.515989+0000","last_change":"2023-12-19T18:26:38.002073+0000","last_active":"2023-12-19T18:26:38.515989+0000","last_peered":"2023-12-19T18:26:38.515989+0000","last_clean":"2023-12-19T18:26:38.515989+0000","last_became_active":"2023-12-19T18:26:38.001660+0000","last_became_peered":"2023-12-19T18:26:38.001660+0000","last_unstale":"2023-12-19T18:26:38.515989+0000","last_undegraded":"2023-12-19T18:26:38.515989+0000","last_fullsized":"2023-12-19T18:26:38.515989+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:47:43.408526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.521734+0000","last_change":"2023-12-19T18:26:38.520945+0000","last_active":"2023-12-19T18:26:38.521734+0000","last_peered":"2023-12-19T18:26:38.521734+0000","last_clean":"2023-12-19T18:26:38.521734+0000","last_became_active":"2023-12-19T18:26:38.517960+0000","last_became_peered":"2023-12-19T18:26:38.517960+0000","last_unstale":"2023-12-19T18:26:38.521734+0000","last_undegraded":"2023-12-19T18:26:38.521734+0000","last_fullsized":"2023-12-19T18:26:38.521734+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:59:50.875803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"184'2","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.515834+0000","last_change":"2023-12-19T18:26:38.002984+0000","last_active":"2023-12-19T18:26:38.515834+0000","last_peered":"2023-12-19T18:26:38.515834+0000","last_clean":"2023-12-19T18:26:38.515834+0000","last_became_active":"2023-12-19T18:26:38.001283+0000","last_became_peered":"2023-12-19T18:26:38.001283+0000","last_unstale":"2023-12-19T18:26:38.515834+0000","last_undegraded":"2023-12-19T18:26:38.515834+0000","last_fullsized":"2023-12-19T18:26:38.515834+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:52.628278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.514676+0000","last_change":"2023-12-19T18:26:38.514543+0000","last_active":"2023-12-19T18:26:38.514676+0000","last_peered":"2023-12-19T18:26:38.514676+0000","last_clean":"2023-12-19T18:26:38.514676+0000","last_became_active":"2023-12-19T18:26:38.514308+0000","last_became_peered":"2023-12-19T18:26:38.514308+0000","last_unstale":"2023-12-19T18:26:38.514676+0000","last_undegraded":"2023-12-19T18:26:38.514676+0000","last_fullsized":"2023-12-19T18:26:38.514676+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:16:39.937501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"184'2","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.514557+0000","last_change":"2023-12-19T18:26:38.000265+0000","last_active":"2023-12-19T18:26:38.514557+0000","last_peered":"2023-12-19T18:26:38.514557+0000","last_clean":"2023-12-19T18:26:38.514557+0000","last_became_active":"2023-12-19T18:26:37.999232+0000","last_became_peered":"2023-12-19T18:26:37.999232+0000","last_unstale":"2023-12-19T18:26:38.514557+0000","last_undegraded":"2023-12-19T18:26:38.514557+0000","last_fullsized":"2023-12-19T18:26:38.514557+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:43.500742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.521454+0000","last_change":"2023-12-19T18:26:38.521202+0000","last_active":"2023-12-19T18:26:38.521454+0000","last_peered":"2023-12-19T18:26:38.521454+0000","last_clean":"2023-12-19T18:26:38.521454+0000","last_became_active":"2023-12-19T18:26:38.518592+0000","last_became_peered":"2023-12-19T18:26:38.518592+0000","last_unstale":"2023-12-19T18:26:38.521454+0000","last_undegraded":"2023-12-19T18:26:38.521454+0000","last_fullsized":"2023-12-19T18:26:38.521454+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:06:42.420094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.524588+0000","last_change":"2023-12-19T18:26:37.995673+0000","last_active":"2023-12-19T18:26:38.524588+0000","last_peered":"2023-12-19T18:26:38.524588+0000","last_clean":"2023-12-19T18:26:38.524588+0000","last_became_active":"2023-12-19T18:26:37.995097+0000","last_became_peered":"2023-12-19T18:26:37.995097+0000","last_unstale":"2023-12-19T18:26:38.524588+0000","last_undegraded":"2023-12-19T18:26:38.524588+0000","last_fullsized":"2023-12-19T18:26:38.524588+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:59:49.864061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.516494+0000","last_change":"2023-12-19T18:26:38.516422+0000","last_active":"2023-12-19T18:26:38.516494+0000","last_peered":"2023-12-19T18:26:38.516494+0000","last_clean":"2023-12-19T18:26:38.516494+0000","last_became_active":"2023-12-19T18:26:38.515525+0000","last_became_peered":"2023-12-19T18:26:38.515525+0000","last_unstale":"2023-12-19T18:26:38.516494+0000","last_undegraded":"2023-12-19T18:26:38.516494+0000","last_fullsized":"2023-12-19T18:26:38.516494+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:13:40.079418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.512287+0000","last_change":"2023-12-19T18:26:38.002164+0000","last_active":"2023-12-19T18:26:38.512287+0000","last_peered":"2023-12-19T18:26:38.512287+0000","last_clean":"2023-12-19T18:26:38.512287+0000","last_became_active":"2023-12-19T18:26:38.001879+0000","last_became_peered":"2023-12-19T18:26:38.001879+0000","last_unstale":"2023-12-19T18:26:38.512287+0000","last_undegraded":"2023-12-19T18:26:38.512287+0000","last_fullsized":"2023-12-19T18:26:38.512287+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:21:34.578038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.525444+0000","last_change":"2023-12-19T18:26:38.522611+0000","last_active":"2023-12-19T18:26:38.525444+0000","last_peered":"2023-12-19T18:26:38.525444+0000","last_clean":"2023-12-19T18:26:38.525444+0000","last_became_active":"2023-12-19T18:26:38.522155+0000","last_became_peered":"2023-12-19T18:26:38.522155+0000","last_unstale":"2023-12-19T18:26:38.525444+0000","last_undegraded":"2023-12-19T18:26:38.525444+0000","last_fullsized":"2023-12-19T18:26:38.525444+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:16:40.950387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.506413+0000","last_change":"2023-12-19T18:26:38.000955+0000","last_active":"2023-12-19T18:26:38.506413+0000","last_peered":"2023-12-19T18:26:38.506413+0000","last_clean":"2023-12-19T18:26:38.506413+0000","last_became_active":"2023-12-19T18:26:37.999522+0000","last_became_peered":"2023-12-19T18:26:37.999522+0000","last_unstale":"2023-12-19T18:26:38.506413+0000","last_undegraded":"2023-12-19T18:26:38.506413+0000","last_fullsized":"2023-12-19T18:26:38.506413+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:38:01.023699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.509833+0000","last_change":"2023-12-19T18:26:38.509484+0000","last_active":"2023-12-19T18:26:38.509833+0000","last_peered":"2023-12-19T18:26:38.509833+0000","last_clean":"2023-12-19T18:26:38.509833+0000","last_became_active":"2023-12-19T18:26:38.508921+0000","last_became_peered":"2023-12-19T18:26:38.508921+0000","last_unstale":"2023-12-19T18:26:38.509833+0000","last_undegraded":"2023-12-19T18:26:38.509833+0000","last_fullsized":"2023-12-19T18:26:38.509833+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:21.211976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.495269+0000","last_change":"2023-12-19T18:26:38.005154+0000","last_active":"2023-12-19T18:26:38.495269+0000","last_peered":"2023-12-19T18:26:38.495269+0000","last_clean":"2023-12-19T18:26:38.495269+0000","last_became_active":"2023-12-19T18:26:38.004729+0000","last_became_peered":"2023-12-19T18:26:38.004729+0000","last_unstale":"2023-12-19T18:26:38.495269+0000","last_undegraded":"2023-12-19T18:26:38.495269+0000","last_fullsized":"2023-12-19T18:26:38.495269+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:10:29.677956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.516954+0000","last_change":"2023-12-19T18:26:38.516869+0000","last_active":"2023-12-19T18:26:38.516954+0000","last_peered":"2023-12-19T18:26:38.516954+0000","last_clean":"2023-12-19T18:26:38.516954+0000","last_became_active":"2023-12-19T18:26:38.516188+0000","last_became_peered":"2023-12-19T18:26:38.516188+0000","last_unstale":"2023-12-19T18:26:38.516954+0000","last_undegraded":"2023-12-19T18:26:38.516954+0000","last_fullsized":"2023-12-19T18:26:38.516954+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:59:50.875803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.508373+0000","last_change":"2023-12-19T18:26:37.999814+0000","last_active":"2023-12-19T18:26:38.508373+0000","last_peered":"2023-12-19T18:26:38.508373+0000","last_clean":"2023-12-19T18:26:38.508373+0000","last_became_active":"2023-12-19T18:26:37.999101+0000","last_became_peered":"2023-12-19T18:26:37.999101+0000","last_unstale":"2023-12-19T18:26:38.508373+0000","last_undegraded":"2023-12-19T18:26:38.508373+0000","last_fullsized":"2023-12-19T18:26:38.508373+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:29.495951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.509765+0000","last_change":"2023-12-19T18:26:38.509020+0000","last_active":"2023-12-19T18:26:38.509765+0000","last_peered":"2023-12-19T18:26:38.509765+0000","last_clean":"2023-12-19T18:26:38.509765+0000","last_became_active":"2023-12-19T18:26:38.505997+0000","last_became_peered":"2023-12-19T18:26:38.505997+0000","last_unstale":"2023-12-19T18:26:38.509765+0000","last_undegraded":"2023-12-19T18:26:38.509765+0000","last_fullsized":"2023-12-19T18:26:38.509765+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:07:27.156222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.496228+0000","last_change":"2023-12-19T18:26:37.997613+0000","last_active":"2023-12-19T18:26:38.496228+0000","last_peered":"2023-12-19T18:26:38.496228+0000","last_clean":"2023-12-19T18:26:38.496228+0000","last_became_active":"2023-12-19T18:26:37.995932+0000","last_became_peered":"2023-12-19T18:26:37.995932+0000","last_unstale":"2023-12-19T18:26:38.496228+0000","last_undegraded":"2023-12-19T18:26:38.496228+0000","last_fullsized":"2023-12-19T18:26:38.496228+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:06.356535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.524099+0000","last_change":"2023-12-19T18:26:38.520807+0000","last_active":"2023-12-19T18:26:38.524099+0000","last_peered":"2023-12-19T18:26:38.524099+0000","last_clean":"2023-12-19T18:26:38.524099+0000","last_became_active":"2023-12-19T18:26:38.520565+0000","last_became_peered":"2023-12-19T18:26:38.520565+0000","last_unstale":"2023-12-19T18:26:38.524099+0000","last_undegraded":"2023-12-19T18:26:38.524099+0000","last_fullsized":"2023-12-19T18:26:38.524099+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:09:06.034401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"184'6","reported_seq":16,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.527050+0000","last_change":"2023-12-19T18:26:37.994368+0000","last_active":"2023-12-19T18:26:38.527050+0000","last_peered":"2023-12-19T18:26:38.527050+0000","last_clean":"2023-12-19T18:26:38.527050+0000","last_became_active":"2023-12-19T18:26:37.993322+0000","last_became_peered":"2023-12-19T18:26:37.993322+0000","last_unstale":"2023-12-19T18:26:38.527050+0000","last_undegraded":"2023-12-19T18:26:38.527050+0000","last_fullsized":"2023-12-19T18:26:38.527050+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:38:01.023699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.514660+0000","last_change":"2023-12-19T18:26:38.513554+0000","last_active":"2023-12-19T18:26:38.514660+0000","last_peered":"2023-12-19T18:26:38.514660+0000","last_clean":"2023-12-19T18:26:38.514660+0000","last_became_active":"2023-12-19T18:26:38.512437+0000","last_became_peered":"2023-12-19T18:26:38.512437+0000","last_unstale":"2023-12-19T18:26:38.514660+0000","last_undegraded":"2023-12-19T18:26:38.514660+0000","last_fullsized":"2023-12-19T18:26:38.514660+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:24:00.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"184'2","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.516915+0000","last_change":"2023-12-19T18:26:38.002798+0000","last_active":"2023-12-19T18:26:38.516915+0000","last_peered":"2023-12-19T18:26:38.516915+0000","last_clean":"2023-12-19T18:26:38.516915+0000","last_became_active":"2023-12-19T18:26:38.000972+0000","last_became_peered":"2023-12-19T18:26:38.000972+0000","last_unstale":"2023-12-19T18:26:38.516915+0000","last_undegraded":"2023-12-19T18:26:38.516915+0000","last_fullsized":"2023-12-19T18:26:38.516915+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:10:48.802437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.517581+0000","last_change":"2023-12-19T18:26:38.517459+0000","last_active":"2023-12-19T18:26:38.517581+0000","last_peered":"2023-12-19T18:26:38.517581+0000","last_clean":"2023-12-19T18:26:38.517581+0000","last_became_active":"2023-12-19T18:26:38.517106+0000","last_became_peered":"2023-12-19T18:26:38.517106+0000","last_unstale":"2023-12-19T18:26:38.517581+0000","last_undegraded":"2023-12-19T18:26:38.517581+0000","last_fullsized":"2023-12-19T18:26:38.517581+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:20.080675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.504256+0000","last_change":"2023-12-19T18:26:38.006106+0000","last_active":"2023-12-19T18:26:38.504256+0000","last_peered":"2023-12-19T18:26:38.504256+0000","last_clean":"2023-12-19T18:26:38.504256+0000","last_became_active":"2023-12-19T18:26:38.005595+0000","last_became_peered":"2023-12-19T18:26:38.005595+0000","last_unstale":"2023-12-19T18:26:38.504256+0000","last_undegraded":"2023-12-19T18:26:38.504256+0000","last_fullsized":"2023-12-19T18:26:38.504256+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:18:28.254508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.515208+0000","last_change":"2023-12-19T18:26:38.515126+0000","last_active":"2023-12-19T18:26:38.515208+0000","last_peered":"2023-12-19T18:26:38.515208+0000","last_clean":"2023-12-19T18:26:38.515208+0000","last_became_active":"2023-12-19T18:26:38.514605+0000","last_became_peered":"2023-12-19T18:26:38.514605+0000","last_unstale":"2023-12-19T18:26:38.515208+0000","last_undegraded":"2023-12-19T18:26:38.515208+0000","last_fullsized":"2023-12-19T18:26:38.515208+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:09.636074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.515491+0000","last_change":"2023-12-19T18:26:38.001707+0000","last_active":"2023-12-19T18:26:38.515491+0000","last_peered":"2023-12-19T18:26:38.515491+0000","last_clean":"2023-12-19T18:26:38.515491+0000","last_became_active":"2023-12-19T18:26:38.001487+0000","last_became_peered":"2023-12-19T18:26:38.001487+0000","last_unstale":"2023-12-19T18:26:38.515491+0000","last_undegraded":"2023-12-19T18:26:38.515491+0000","last_fullsized":"2023-12-19T18:26:38.515491+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:13.178810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.516769+0000","last_change":"2023-12-19T18:26:38.516578+0000","last_active":"2023-12-19T18:26:38.516769+0000","last_peered":"2023-12-19T18:26:38.516769+0000","last_clean":"2023-12-19T18:26:38.516769+0000","last_became_active":"2023-12-19T18:26:38.515850+0000","last_became_peered":"2023-12-19T18:26:38.515850+0000","last_unstale":"2023-12-19T18:26:38.516769+0000","last_undegraded":"2023-12-19T18:26:38.516769+0000","last_fullsized":"2023-12-19T18:26:38.516769+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:42.815725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"184'8","reported_seq":18,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.527892+0000","last_change":"2023-12-19T18:26:37.997903+0000","last_active":"2023-12-19T18:26:38.527892+0000","last_peered":"2023-12-19T18:26:38.527892+0000","last_clean":"2023-12-19T18:26:38.527892+0000","last_became_active":"2023-12-19T18:26:37.997359+0000","last_became_peered":"2023-12-19T18:26:37.997359+0000","last_unstale":"2023-12-19T18:26:38.527892+0000","last_undegraded":"2023-12-19T18:26:38.527892+0000","last_fullsized":"2023-12-19T18:26:38.527892+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:41.803983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.503701+0000","last_change":"2023-12-19T18:26:38.501505+0000","last_active":"2023-12-19T18:26:38.503701+0000","last_peered":"2023-12-19T18:26:38.503701+0000","last_clean":"2023-12-19T18:26:38.503701+0000","last_became_active":"2023-12-19T18:26:38.501087+0000","last_became_peered":"2023-12-19T18:26:38.501087+0000","last_unstale":"2023-12-19T18:26:38.503701+0000","last_undegraded":"2023-12-19T18:26:38.503701+0000","last_fullsized":"2023-12-19T18:26:38.503701+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:49:29.813985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.520470+0000","last_change":"2023-12-19T18:26:37.996630+0000","last_active":"2023-12-19T18:26:38.520470+0000","last_peered":"2023-12-19T18:26:38.520470+0000","last_clean":"2023-12-19T18:26:38.520470+0000","last_became_active":"2023-12-19T18:26:37.996263+0000","last_became_peered":"2023-12-19T18:26:37.996263+0000","last_unstale":"2023-12-19T18:26:38.520470+0000","last_undegraded":"2023-12-19T18:26:38.520470+0000","last_fullsized":"2023-12-19T18:26:38.520470+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:15:07.042524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.519026+0000","last_change":"2023-12-19T18:26:38.518923+0000","last_active":"2023-12-19T18:26:38.519026+0000","last_peered":"2023-12-19T18:26:38.519026+0000","last_clean":"2023-12-19T18:26:38.519026+0000","last_became_active":"2023-12-19T18:26:38.518655+0000","last_became_peered":"2023-12-19T18:26:38.518655+0000","last_unstale":"2023-12-19T18:26:38.519026+0000","last_undegraded":"2023-12-19T18:26:38.519026+0000","last_fullsized":"2023-12-19T18:26:38.519026+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:50.192924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"184'2","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.523400+0000","last_change":"2023-12-19T18:26:38.002740+0000","last_active":"2023-12-19T18:26:38.523400+0000","last_peered":"2023-12-19T18:26:38.523400+0000","last_clean":"2023-12-19T18:26:38.523400+0000","last_became_active":"2023-12-19T18:26:38.002404+0000","last_became_peered":"2023-12-19T18:26:38.002404+0000","last_unstale":"2023-12-19T18:26:38.523400+0000","last_undegraded":"2023-12-19T18:26:38.523400+0000","last_fullsized":"2023-12-19T18:26:38.523400+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:09:05.022659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.505371+0000","last_change":"2023-12-19T18:26:38.504549+0000","last_active":"2023-12-19T18:26:38.505371+0000","last_peered":"2023-12-19T18:26:38.505371+0000","last_clean":"2023-12-19T18:26:38.505371+0000","last_became_active":"2023-12-19T18:26:38.504290+0000","last_became_peered":"2023-12-19T18:26:38.504290+0000","last_unstale":"2023-12-19T18:26:38.505371+0000","last_undegraded":"2023-12-19T18:26:38.505371+0000","last_fullsized":"2023-12-19T18:26:38.505371+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:26:35.126926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"184'1","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.524159+0000","last_change":"2023-12-19T18:26:37.992200+0000","last_active":"2023-12-19T18:26:38.524159+0000","last_peered":"2023-12-19T18:26:38.524159+0000","last_clean":"2023-12-19T18:26:38.524159+0000","last_became_active":"2023-12-19T18:26:37.991863+0000","last_became_peered":"2023-12-19T18:26:37.991863+0000","last_unstale":"2023-12-19T18:26:38.524159+0000","last_undegraded":"2023-12-19T18:26:38.524159+0000","last_fullsized":"2023-12-19T18:26:38.524159+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:47:43.408526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.521684+0000","last_change":"2023-12-19T18:26:38.520808+0000","last_active":"2023-12-19T18:26:38.521684+0000","last_peered":"2023-12-19T18:26:38.521684+0000","last_clean":"2023-12-19T18:26:38.521684+0000","last_became_active":"2023-12-19T18:26:38.517583+0000","last_became_peered":"2023-12-19T18:26:38.517583+0000","last_unstale":"2023-12-19T18:26:38.521684+0000","last_undegraded":"2023-12-19T18:26:38.521684+0000","last_fullsized":"2023-12-19T18:26:38.521684+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:42.815725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.515593+0000","last_change":"2023-12-19T18:26:38.000581+0000","last_active":"2023-12-19T18:26:38.515593+0000","last_peered":"2023-12-19T18:26:38.515593+0000","last_clean":"2023-12-19T18:26:38.515593+0000","last_became_active":"2023-12-19T18:26:38.000337+0000","last_became_peered":"2023-12-19T18:26:38.000337+0000","last_unstale":"2023-12-19T18:26:38.515593+0000","last_undegraded":"2023-12-19T18:26:38.515593+0000","last_fullsized":"2023-12-19T18:26:38.515593+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:56:03.771290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.512608+0000","last_change":"2023-12-19T18:26:38.512241+0000","last_active":"2023-12-19T18:26:38.512608+0000","last_peered":"2023-12-19T18:26:38.512608+0000","last_clean":"2023-12-19T18:26:38.512608+0000","last_became_active":"2023-12-19T18:26:38.512009+0000","last_became_peered":"2023-12-19T18:26:38.512009+0000","last_unstale":"2023-12-19T18:26:38.512608+0000","last_undegraded":"2023-12-19T18:26:38.512608+0000","last_fullsized":"2023-12-19T18:26:38.512608+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:47:41.584531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"184'2","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.517370+0000","last_change":"2023-12-19T18:26:38.002613+0000","last_active":"2023-12-19T18:26:38.517370+0000","last_peered":"2023-12-19T18:26:38.517370+0000","last_clean":"2023-12-19T18:26:38.517370+0000","last_became_active":"2023-12-19T18:26:38.002370+0000","last_became_peered":"2023-12-19T18:26:38.002370+0000","last_unstale":"2023-12-19T18:26:38.517370+0000","last_undegraded":"2023-12-19T18:26:38.517370+0000","last_fullsized":"2023-12-19T18:26:38.517370+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:16:39.938645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.521724+0000","last_change":"2023-12-19T18:26:38.521621+0000","last_active":"2023-12-19T18:26:38.521724+0000","last_peered":"2023-12-19T18:26:38.521724+0000","last_clean":"2023-12-19T18:26:38.521724+0000","last_became_active":"2023-12-19T18:26:38.518851+0000","last_became_peered":"2023-12-19T18:26:38.518851+0000","last_unstale":"2023-12-19T18:26:38.521724+0000","last_undegraded":"2023-12-19T18:26:38.521724+0000","last_fullsized":"2023-12-19T18:26:38.521724+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:47:41.584531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.507275+0000","last_change":"2023-12-19T18:26:37.979009+0000","last_active":"2023-12-19T18:26:38.507275+0000","last_peered":"2023-12-19T18:26:38.507275+0000","last_clean":"2023-12-19T18:26:38.507275+0000","last_became_active":"2023-12-19T18:26:37.978387+0000","last_became_peered":"2023-12-19T18:26:37.978387+0000","last_unstale":"2023-12-19T18:26:38.507275+0000","last_undegraded":"2023-12-19T18:26:38.507275+0000","last_fullsized":"2023-12-19T18:26:38.507275+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:27:55.676426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.512388+0000","last_change":"2023-12-19T18:26:38.511007+0000","last_active":"2023-12-19T18:26:38.512388+0000","last_peered":"2023-12-19T18:26:38.512388+0000","last_clean":"2023-12-19T18:26:38.512388+0000","last_became_active":"2023-12-19T18:26:38.510797+0000","last_became_peered":"2023-12-19T18:26:38.510797+0000","last_unstale":"2023-12-19T18:26:38.512388+0000","last_undegraded":"2023-12-19T18:26:38.512388+0000","last_fullsized":"2023-12-19T18:26:38.512388+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:31.201464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.503206+0000","last_change":"2023-12-19T18:26:37.997350+0000","last_active":"2023-12-19T18:26:38.503206+0000","last_peered":"2023-12-19T18:26:38.503206+0000","last_clean":"2023-12-19T18:26:38.503206+0000","last_became_active":"2023-12-19T18:26:37.997075+0000","last_became_peered":"2023-12-19T18:26:37.997075+0000","last_unstale":"2023-12-19T18:26:38.503206+0000","last_undegraded":"2023-12-19T18:26:38.503206+0000","last_fullsized":"2023-12-19T18:26:38.503206+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:33:41.275730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.522858+0000","last_change":"2023-12-19T18:26:38.522444+0000","last_active":"2023-12-19T18:26:38.522858+0000","last_peered":"2023-12-19T18:26:38.522858+0000","last_clean":"2023-12-19T18:26:38.522858+0000","last_became_active":"2023-12-19T18:26:38.521830+0000","last_became_peered":"2023-12-19T18:26:38.521830+0000","last_unstale":"2023-12-19T18:26:38.522858+0000","last_undegraded":"2023-12-19T18:26:38.522858+0000","last_fullsized":"2023-12-19T18:26:38.522858+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:57:04.171752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.503617+0000","last_change":"2023-12-19T18:26:37.978842+0000","last_active":"2023-12-19T18:26:38.503617+0000","last_peered":"2023-12-19T18:26:38.503617+0000","last_clean":"2023-12-19T18:26:38.503617+0000","last_became_active":"2023-12-19T18:26:37.978269+0000","last_became_peered":"2023-12-19T18:26:37.978269+0000","last_unstale":"2023-12-19T18:26:38.503617+0000","last_undegraded":"2023-12-19T18:26:38.503617+0000","last_fullsized":"2023-12-19T18:26:38.503617+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:25:56.253877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.507619+0000","last_change":"2023-12-19T18:26:38.507047+0000","last_active":"2023-12-19T18:26:38.507619+0000","last_peered":"2023-12-19T18:26:38.507619+0000","last_clean":"2023-12-19T18:26:38.507619+0000","last_became_active":"2023-12-19T18:26:38.506771+0000","last_became_peered":"2023-12-19T18:26:38.506771+0000","last_unstale":"2023-12-19T18:26:38.507619+0000","last_undegraded":"2023-12-19T18:26:38.507619+0000","last_fullsized":"2023-12-19T18:26:38.507619+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:06:35.754608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"184'2","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.509999+0000","last_change":"2023-12-19T18:26:37.992843+0000","last_active":"2023-12-19T18:26:38.509999+0000","last_peered":"2023-12-19T18:26:38.509999+0000","last_clean":"2023-12-19T18:26:38.509999+0000","last_became_active":"2023-12-19T18:26:37.992138+0000","last_became_peered":"2023-12-19T18:26:37.992138+0000","last_unstale":"2023-12-19T18:26:38.509999+0000","last_undegraded":"2023-12-19T18:26:38.509999+0000","last_fullsized":"2023-12-19T18:26:38.509999+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:18:28.254508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.507591+0000","last_change":"2023-12-19T18:26:38.506842+0000","last_active":"2023-12-19T18:26:38.507591+0000","last_peered":"2023-12-19T18:26:38.507591+0000","last_clean":"2023-12-19T18:26:38.507591+0000","last_became_active":"2023-12-19T18:26:38.506603+0000","last_became_peered":"2023-12-19T18:26:38.506603+0000","last_unstale":"2023-12-19T18:26:38.507591+0000","last_undegraded":"2023-12-19T18:26:38.507591+0000","last_fullsized":"2023-12-19T18:26:38.507591+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:14:38.491679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.511867+0000","last_change":"2023-12-19T18:26:37.999764+0000","last_active":"2023-12-19T18:26:38.511867+0000","last_peered":"2023-12-19T18:26:38.511867+0000","last_clean":"2023-12-19T18:26:38.511867+0000","last_became_active":"2023-12-19T18:26:37.999519+0000","last_became_peered":"2023-12-19T18:26:37.999519+0000","last_unstale":"2023-12-19T18:26:38.511867+0000","last_undegraded":"2023-12-19T18:26:38.511867+0000","last_fullsized":"2023-12-19T18:26:38.511867+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:15.829119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.524343+0000","last_change":"2023-12-19T18:26:38.523911+0000","last_active":"2023-12-19T18:26:38.524343+0000","last_peered":"2023-12-19T18:26:38.524343+0000","last_clean":"2023-12-19T18:26:38.524343+0000","last_became_active":"2023-12-19T18:26:38.523133+0000","last_became_peered":"2023-12-19T18:26:38.523133+0000","last_unstale":"2023-12-19T18:26:38.524343+0000","last_undegraded":"2023-12-19T18:26:38.524343+0000","last_fullsized":"2023-12-19T18:26:38.524343+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:48.610885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.490333+0000","last_change":"2023-12-19T18:26:37.996734+0000","last_active":"2023-12-19T18:26:38.490333+0000","last_peered":"2023-12-19T18:26:38.490333+0000","last_clean":"2023-12-19T18:26:38.490333+0000","last_became_active":"2023-12-19T18:26:37.996274+0000","last_became_peered":"2023-12-19T18:26:37.996274+0000","last_unstale":"2023-12-19T18:26:38.490333+0000","last_undegraded":"2023-12-19T18:26:38.490333+0000","last_fullsized":"2023-12-19T18:26:38.490333+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:29.495951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.523530+0000","last_change":"2023-12-19T18:26:38.522476+0000","last_active":"2023-12-19T18:26:38.523530+0000","last_peered":"2023-12-19T18:26:38.523530+0000","last_clean":"2023-12-19T18:26:38.523530+0000","last_became_active":"2023-12-19T18:26:38.515096+0000","last_became_peered":"2023-12-19T18:26:38.515096+0000","last_unstale":"2023-12-19T18:26:38.523530+0000","last_undegraded":"2023-12-19T18:26:38.523530+0000","last_fullsized":"2023-12-19T18:26:38.523530+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:16:39.937501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.490362+0000","last_change":"2023-12-19T18:26:37.997778+0000","last_active":"2023-12-19T18:26:38.490362+0000","last_peered":"2023-12-19T18:26:38.490362+0000","last_clean":"2023-12-19T18:26:38.490362+0000","last_became_active":"2023-12-19T18:26:37.997532+0000","last_became_peered":"2023-12-19T18:26:37.997532+0000","last_unstale":"2023-12-19T18:26:38.490362+0000","last_undegraded":"2023-12-19T18:26:38.490362+0000","last_fullsized":"2023-12-19T18:26:38.490362+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:49.181182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.508973+0000","last_change":"2023-12-19T18:26:38.506763+0000","last_active":"2023-12-19T18:26:38.508973+0000","last_peered":"2023-12-19T18:26:38.508973+0000","last_clean":"2023-12-19T18:26:38.508973+0000","last_became_active":"2023-12-19T18:26:38.506090+0000","last_became_peered":"2023-12-19T18:26:38.506090+0000","last_unstale":"2023-12-19T18:26:38.508973+0000","last_undegraded":"2023-12-19T18:26:38.508973+0000","last_fullsized":"2023-12-19T18:26:38.508973+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:07:27.156222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.515897+0000","last_change":"2023-12-19T18:26:38.006710+0000","last_active":"2023-12-19T18:26:38.515897+0000","last_peered":"2023-12-19T18:26:38.515897+0000","last_clean":"2023-12-19T18:26:38.515897+0000","last_became_active":"2023-12-19T18:26:38.006471+0000","last_became_peered":"2023-12-19T18:26:38.006471+0000","last_unstale":"2023-12-19T18:26:38.515897+0000","last_undegraded":"2023-12-19T18:26:38.515897+0000","last_fullsized":"2023-12-19T18:26:38.515897+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:14.752319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.507634+0000","last_change":"2023-12-19T18:26:38.507476+0000","last_active":"2023-12-19T18:26:38.507634+0000","last_peered":"2023-12-19T18:26:38.507634+0000","last_clean":"2023-12-19T18:26:38.507634+0000","last_became_active":"2023-12-19T18:26:38.507237+0000","last_became_peered":"2023-12-19T18:26:38.507237+0000","last_unstale":"2023-12-19T18:26:38.507634+0000","last_undegraded":"2023-12-19T18:26:38.507634+0000","last_fullsized":"2023-12-19T18:26:38.507634+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:44:27.103692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.494469+0000","last_change":"2023-12-19T18:26:37.999535+0000","last_active":"2023-12-19T18:26:38.494469+0000","last_peered":"2023-12-19T18:26:38.494469+0000","last_clean":"2023-12-19T18:26:38.494469+0000","last_became_active":"2023-12-19T18:26:37.999290+0000","last_became_peered":"2023-12-19T18:26:37.999290+0000","last_unstale":"2023-12-19T18:26:38.494469+0000","last_undegraded":"2023-12-19T18:26:38.494469+0000","last_fullsized":"2023-12-19T18:26:38.494469+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:03:38.154234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.509816+0000","last_change":"2023-12-19T18:26:38.509297+0000","last_active":"2023-12-19T18:26:38.509816+0000","last_peered":"2023-12-19T18:26:38.509816+0000","last_clean":"2023-12-19T18:26:38.509816+0000","last_became_active":"2023-12-19T18:26:38.508537+0000","last_became_peered":"2023-12-19T18:26:38.508537+0000","last_unstale":"2023-12-19T18:26:38.509816+0000","last_undegraded":"2023-12-19T18:26:38.509816+0000","last_fullsized":"2023-12-19T18:26:38.509816+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:26:08.400076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.496963+0000","last_change":"2023-12-19T18:26:38.003094+0000","last_active":"2023-12-19T18:26:38.496963+0000","last_peered":"2023-12-19T18:26:38.496963+0000","last_clean":"2023-12-19T18:26:38.496963+0000","last_became_active":"2023-12-19T18:26:38.001985+0000","last_became_peered":"2023-12-19T18:26:38.001985+0000","last_unstale":"2023-12-19T18:26:38.496963+0000","last_undegraded":"2023-12-19T18:26:38.496963+0000","last_fullsized":"2023-12-19T18:26:38.496963+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:27:55.676426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.502371+0000","last_change":"2023-12-19T18:26:38.502269+0000","last_active":"2023-12-19T18:26:38.502371+0000","last_peered":"2023-12-19T18:26:38.502371+0000","last_clean":"2023-12-19T18:26:38.502371+0000","last_became_active":"2023-12-19T18:26:38.502021+0000","last_became_peered":"2023-12-19T18:26:38.502021+0000","last_unstale":"2023-12-19T18:26:38.502371+0000","last_undegraded":"2023-12-19T18:26:38.502371+0000","last_fullsized":"2023-12-19T18:26:38.502371+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:36:51.412075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.506634+0000","last_change":"2023-12-19T18:26:38.001917+0000","last_active":"2023-12-19T18:26:38.506634+0000","last_peered":"2023-12-19T18:26:38.506634+0000","last_clean":"2023-12-19T18:26:38.506634+0000","last_became_active":"2023-12-19T18:26:38.001257+0000","last_became_peered":"2023-12-19T18:26:38.001257+0000","last_unstale":"2023-12-19T18:26:38.506634+0000","last_undegraded":"2023-12-19T18:26:38.506634+0000","last_fullsized":"2023-12-19T18:26:38.506634+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:06:41.408352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":9,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.495547+0000","last_change":"2023-12-19T18:26:38.495348+0000","last_active":"2023-12-19T18:26:38.495547+0000","last_peered":"2023-12-19T18:26:38.495547+0000","last_clean":"2023-12-19T18:26:38.495547+0000","last_became_active":"2023-12-19T18:26:38.494366+0000","last_became_peered":"2023-12-19T18:26:38.494366+0000","last_unstale":"2023-12-19T18:26:38.495547+0000","last_undegraded":"2023-12-19T18:26:38.495547+0000","last_fullsized":"2023-12-19T18:26:38.495547+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:44:27.103692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"184'3","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.518334+0000","last_change":"2023-12-19T18:26:38.002223+0000","last_active":"2023-12-19T18:26:38.518334+0000","last_peered":"2023-12-19T18:26:38.518334+0000","last_clean":"2023-12-19T18:26:38.518334+0000","last_became_active":"2023-12-19T18:26:38.001643+0000","last_became_peered":"2023-12-19T18:26:38.001643+0000","last_unstale":"2023-12-19T18:26:38.518334+0000","last_undegraded":"2023-12-19T18:26:38.518334+0000","last_fullsized":"2023-12-19T18:26:38.518334+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:18.539917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":312,"reported_epoch":183,"state":"active+clean","last_fresh":"2023-12-19T18:26:37.987291+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:26:37.987291+0000","last_peered":"2023-12-19T18:26:37.987291+0000","last_clean":"2023-12-19T18:26:37.987291+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:26:37.987291+0000","last_undegraded":"2023-12-19T18:26:37.987291+0000","last_fullsized":"2023-12-19T18:26:37.987291+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1008,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194304,"statfs":{"total":200000000,"available":198967368,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.179,"15min":1.169},"min":{"1min":0.748,"5min":0.57899999999999996,"15min":0.45900000000000002},"max":{"1min":1.597,"5min":1.9630000000000001,"15min":5.6879999999999997},"last":1.3},{"interface":"front","average":{"1min":1.25,"5min":1.2310000000000001,"15min":1.2110000000000001},"min":{"1min":1.05,"5min":0.71899999999999997,"15min":0.55800000000000005},"max":{"1min":1.526,"5min":1.7030000000000001,"15min":1.79},"last":1.506}]},{"osd":1,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.155,"15min":1.228},"min":{"1min":0.84299999999999997,"5min":0.65800000000000003,"15min":0.54400000000000004},"max":{"1min":1.619,"5min":1.7010000000000001,"15min":10.926},"last":1.4750000000000001},{"interface":"front","average":{"1min":1.294,"5min":1.2769999999999999,"15min":1.264},"min":{"1min":1.018,"5min":0.80700000000000005,"15min":0.67900000000000005},"max":{"1min":1.5980000000000001,"5min":1.736,"15min":10.904999999999999},"last":1.4590000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.0880000000000001,"15min":1.109},"min":{"1min":0.90700000000000003,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":3.1960000000000002},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.1779999999999999,"15min":1.196},"min":{"1min":0.93400000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.4810000000000001,"5min":2.0670000000000002,"15min":6.008},"last":1.409}]},{"osd":3,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.25,"15min":1.2030000000000001},"min":{"1min":0.83499999999999996,"5min":0.66900000000000004,"15min":0.61399999999999999},"max":{"1min":1.696,"5min":1.732,"15min":3.0379999999999998},"last":1.335},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3540000000000001,"15min":1.3260000000000001},"min":{"1min":1.2070000000000001,"5min":0.77500000000000002,"15min":0.69599999999999995},"max":{"1min":1.5529999999999999,"5min":2.157,"15min":5.8120000000000003},"last":1.522}]},{"osd":4,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2150000000000001,"15min":1.1639999999999999},"min":{"1min":0.88200000000000001,"5min":0.55700000000000005,"15min":0.47899999999999998},"max":{"1min":1.569,"5min":1.847,"15min":4.0110000000000001},"last":1.2450000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.246,"15min":1.254},"min":{"1min":0.88200000000000001,"5min":0.59299999999999997,"15min":0.57599999999999996},"max":{"1min":1.8,"5min":1.8,"15min":3.3180000000000001},"last":1.1579999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.2809999999999999,"15min":1.331},"min":{"1min":0.94599999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.657,"5min":1.956,"15min":7.6429999999999998},"last":1.1970000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.3580000000000001,"15min":1.355},"min":{"1min":0.96299999999999997,"5min":0.627,"15min":0.53500000000000003},"max":{"1min":1.6419999999999999,"5min":2.1179999999999999,"15min":6.0839999999999996},"last":1.367}]},{"osd":6,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.512,"15min":1.4910000000000001},"min":{"1min":1.1459999999999999,"5min":0.628,"15min":0.59899999999999998},"max":{"1min":1.716,"5min":2.423,"15min":3.1309999999999998},"last":1.554},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4379999999999999,"15min":1.413},"min":{"1min":1.1220000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.839,"5min":2.0179999999999998,"15min":4.1479999999999997},"last":1.4350000000000001}]}]},{"osd":6,"up_from":13,"seq":55834574993,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1002,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194310,"statfs":{"total":200000000,"available":198973643,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.119,"15min":1.0489999999999999},"min":{"1min":0.89100000000000001,"5min":0.69499999999999995,"15min":0.65800000000000003},"max":{"1min":1.502,"5min":1.6259999999999999,"15min":1.8129999999999999},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.1359999999999999,"15min":1.107},"min":{"1min":0.93200000000000005,"5min":0.64800000000000002,"15min":0.55600000000000005},"max":{"1min":1.548,"5min":1.663,"15min":3.5529999999999999},"last":1.1220000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.115,"15min":1.123},"min":{"1min":0.85199999999999998,"5min":0.68600000000000005,"15min":0.68200000000000005},"max":{"1min":1.4710000000000001,"5min":1.5609999999999999,"15min":4.5540000000000003},"last":1.2949999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.165,"15min":1.171},"min":{"1min":0.874,"5min":0.64600000000000002,"15min":0.60299999999999998},"max":{"1min":1.5660000000000001,"5min":1.627,"15min":1.885},"last":1}]},{"osd":2,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.21,"15min":1.1830000000000001},"min":{"1min":0.92300000000000004,"5min":0.78900000000000003,"15min":0.626},"max":{"1min":1.508,"5min":1.583,"15min":1.821},"last":1.155},{"interface":"front","average":{"1min":1.23,"5min":1.1910000000000001,"15min":1.218},"min":{"1min":0.89800000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7669999999999999,"5min":1.821,"15min":3.7909999999999999},"last":1.1899999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.28,"15min":1.236},"min":{"1min":1.0409999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5760000000000001,"5min":1.722,"15min":3.8820000000000001},"last":0.93500000000000005},{"interface":"front","average":{"1min":1.325,"5min":1.224,"15min":1.1919999999999999},"min":{"1min":0.97099999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.619,"5min":1.6559999999999999,"15min":4.6189999999999998},"last":0.71199999999999997}]},{"osd":4,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.3080000000000001,"15min":1.27},"min":{"1min":0.88400000000000001,"5min":0.74099999999999999,"15min":0.65600000000000003},"max":{"1min":1.627,"5min":1.6899999999999999,"15min":3.0680000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.413,"15min":1.3799999999999999},"min":{"1min":0.79900000000000004,"5min":0.76700000000000002,"15min":0.57999999999999996},"max":{"1min":1.758,"5min":1.774,"15min":3.7000000000000002},"last":1.0489999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.2629999999999999,"15min":1.226},"min":{"1min":0.85699999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.746,"5min":1.746,"15min":4.6680000000000001},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.401,"5min":1.431,"15min":1.3799999999999999},"min":{"1min":1.0960000000000001,"5min":0.84799999999999998,"15min":0.626},"max":{"1min":1.6479999999999999,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.2110000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.3839999999999999},"min":{"1min":0.90300000000000002,"5min":0.72299999999999998,"15min":0.61899999999999999},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":3.9670000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4239999999999999,"15min":1.391},"min":{"1min":1.129,"5min":0.81899999999999995,"15min":0.79300000000000004},"max":{"1min":1.7290000000000001,"5min":1.8169999999999999,"15min":5.6639999999999997},"last":1.4159999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574993,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1578,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193733,"statfs":{"total":200000000,"available":198383275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.2070000000000001,"15min":1.1930000000000001},"min":{"1min":0.80400000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.6279999999999999,"5min":1.8540000000000001,"15min":1.863},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.371,"5min":1.2729999999999999,"15min":1.226},"min":{"1min":1.214,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5660000000000001,"5min":1.913,"15min":1.913},"last":1.556}]},{"osd":1,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.163,"15min":1.1100000000000001},"min":{"1min":0.92500000000000004,"5min":0.71599999999999997,"15min":0.65700000000000003},"max":{"1min":1.5920000000000001,"5min":1.7929999999999999,"15min":1.9490000000000001},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1759999999999999,"15min":1.111},"min":{"1min":0.91900000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.696,"5min":1.8300000000000001,"15min":1.843},"last":1.621}]},{"osd":2,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.175,"15min":1.127},"min":{"1min":1.026,"5min":0.72199999999999998,"15min":0.59499999999999997},"max":{"1min":1.621,"5min":1.621,"15min":1.794},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2809999999999999,"15min":1.2},"min":{"1min":0.996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.677,"5min":1.8,"15min":2.036},"last":1.0429999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2030000000000001,"15min":1.141},"min":{"1min":0.94599999999999995,"5min":0.78600000000000003,"15min":0.68899999999999995},"max":{"1min":1.5429999999999999,"5min":1.72,"15min":1.7450000000000001},"last":1.589},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3480000000000001,"15min":1.284},"min":{"1min":1.137,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.694,"5min":1.9590000000000001,"15min":2.0590000000000002},"last":1.528}]},{"osd":4,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3160000000000001,"15min":1.2370000000000001},"min":{"1min":0.77900000000000003,"5min":0.626,"15min":0.46700000000000003},"max":{"1min":1.659,"5min":1.944,"15min":2.0329999999999999},"last":1.659},{"interface":"front","average":{"1min":1.105,"5min":1.1850000000000001,"15min":1.1519999999999999},"min":{"1min":0.88700000000000001,"5min":0.66000000000000003,"15min":0.57099999999999995},"max":{"1min":1.609,"5min":1.7689999999999999,"15min":1.901},"last":1.133}]},{"osd":6,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.46,"15min":1.3899999999999999},"min":{"1min":1.2210000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.631,"5min":2.0329999999999999,"15min":2.1589999999999998},"last":1.7230000000000001},{"interface":"front","average":{"1min":1.248,"5min":1.3340000000000001,"15min":1.28},"min":{"1min":1.0069999999999999,"5min":0.83199999999999996,"15min":0.49299999999999999},"max":{"1min":1.704,"5min":1.704,"15min":1.976},"last":1.325}]},{"osd":7,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.3049999999999999,"15min":1.2589999999999999},"min":{"1min":0.751,"5min":0.54700000000000004,"15min":0.50900000000000001},"max":{"1min":1.6399999999999999,"5min":1.863,"15min":2.0270000000000001},"last":1.6850000000000001},{"interface":"front","average":{"1min":1.419,"5min":1.466,"15min":1.4350000000000001},"min":{"1min":1.252,"5min":0.93300000000000005,"15min":0.623},"max":{"1min":1.661,"5min":1.8620000000000001,"15min":2.117},"last":1.4159999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1017,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194295,"statfs":{"total":200000000,"available":198958425,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.054,"15min":1.016},"min":{"1min":0.71299999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.468,"5min":1.972,"15min":1.972},"last":1.0169999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.095,"15min":1.0549999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3859999999999999,"5min":1.7330000000000001,"15min":1.7909999999999999},"last":1.403}]},{"osd":1,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1619999999999999,"15min":1.1579999999999999},"min":{"1min":0.871,"5min":0.67200000000000004,"15min":0.66900000000000004},"max":{"1min":1.6000000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.319,"5min":1.252,"15min":1.2130000000000001},"min":{"1min":0.95199999999999996,"5min":0.64700000000000002,"15min":0.61699999999999999},"max":{"1min":1.5600000000000001,"5min":1.875,"15min":1.875},"last":1.506}]},{"osd":2,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.125,"15min":1.125},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.536,"5min":1.6830000000000001,"15min":1.7010000000000001},"last":1.6950000000000001},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.165,"15min":1.1299999999999999},"min":{"1min":0.81599999999999995,"5min":0.747,"15min":0.70499999999999996},"max":{"1min":1.456,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.4530000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.266,"15min":1.2629999999999999},"min":{"1min":0.90900000000000003,"5min":0.754,"15min":0.754},"max":{"1min":1.5940000000000001,"5min":1.946,"15min":1.946},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.357,"15min":1.323},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.591,"5min":1.927,"15min":1.927},"last":1.788}]},{"osd":5,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2509999999999999,"15min":1.2310000000000001},"min":{"1min":0.83699999999999997,"5min":0.82299999999999995,"15min":0.44500000000000001},"max":{"1min":1.681,"5min":1.8129999999999999,"15min":1.8740000000000001},"last":1.196},{"interface":"front","average":{"1min":1.24,"5min":1.343,"15min":1.357},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.504},"max":{"1min":1.7509999999999999,"5min":1.867,"15min":1.867},"last":1.9359999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.28,"15min":1.298},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.432},"max":{"1min":1.651,"5min":1.921,"15min":1.921},"last":1.54},{"interface":"front","average":{"1min":1.397,"5min":1.413,"15min":1.4079999999999999},"min":{"1min":0.93700000000000006,"5min":0.90800000000000003,"15min":0.64500000000000002},"max":{"1min":1.744,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.653}]},{"osd":7,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3380000000000001,"15min":1.3460000000000001},"min":{"1min":0.77300000000000002,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":1.6739999999999999,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.6120000000000001},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.413,"15min":1.411},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.57399999999999995},"max":{"1min":1.611,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.0760000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1008,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194304,"statfs":{"total":200000000,"available":198967368,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.89400000000000002,"15min":0.93500000000000005},"min":{"1min":0.69399999999999995,"5min":0.435,"15min":0.35299999999999998},"max":{"1min":1.4299999999999999,"5min":2.516,"15min":3.4350000000000001},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0369999999999999,"15min":0.97599999999999998},"min":{"1min":0.69799999999999995,"5min":0.42999999999999999,"15min":0.39500000000000002},"max":{"1min":1.4219999999999999,"5min":4.3490000000000002,"15min":4.3490000000000002},"last":1.4199999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.0469999999999999,"15min":1.04},"min":{"1min":0.76600000000000001,"5min":0.45300000000000001,"15min":0.41299999999999998},"max":{"1min":1.3109999999999999,"5min":5.2510000000000003,"15min":5.2510000000000003},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.1439999999999999,"15min":1.109},"min":{"1min":0.93100000000000005,"5min":0.53200000000000003,"15min":0.47799999999999998},"max":{"1min":1.643,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.3839999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.204,"15min":1.2470000000000001},"min":{"1min":0.96699999999999997,"5min":0.502,"15min":0.46000000000000002},"max":{"1min":1.7,"5min":2.4649999999999999,"15min":13.833},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.3580000000000001,"15min":1.3360000000000001},"min":{"1min":1.105,"5min":0.60299999999999998,"15min":0.53200000000000003},"max":{"1min":1.798,"5min":5.4480000000000004,"15min":16.748999999999999},"last":1.5189999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.343,"15min":1.272},"min":{"1min":0.99299999999999999,"5min":0.77900000000000003,"15min":0.65600000000000003},"max":{"1min":1.7250000000000001,"5min":5.1790000000000003,"15min":5.1790000000000003},"last":1.127},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4510000000000001,"15min":1.409},"min":{"1min":1.117,"5min":0.873,"15min":0.73199999999999998},"max":{"1min":2.0819999999999999,"5min":4.569,"15min":4.569},"last":1.6890000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4359999999999999,"15min":1.4350000000000001},"min":{"1min":1.2390000000000001,"5min":0.80700000000000005,"15min":0.76000000000000001},"max":{"1min":1.7909999999999999,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":1.5760000000000001},{"interface":"front","average":{"1min":1.369,"5min":1.3959999999999999,"15min":1.3180000000000001},"min":{"1min":1.0840000000000001,"5min":0.81899999999999995,"15min":0.69999999999999996},"max":{"1min":1.8520000000000001,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.0089999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.478,"15min":1.464},"min":{"1min":1.2310000000000001,"5min":0.81999999999999995,"15min":0.81599999999999995},"max":{"1min":1.9410000000000001,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":1.7609999999999999},{"interface":"front","average":{"1min":1.538,"5min":1.53,"15min":1.5089999999999999},"min":{"1min":1.2190000000000001,"5min":0.83999999999999997,"15min":0.80500000000000005},"max":{"1min":1.9419999999999999,"5min":5.3739999999999997,"15min":5.3739999999999997},"last":1.9099999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5,"15min":1.4259999999999999},"min":{"1min":1.2090000000000001,"5min":0.78500000000000003,"15min":0.73199999999999998},"max":{"1min":1.8799999999999999,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.3080000000000001},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.601,"15min":1.5589999999999999},"min":{"1min":1.3700000000000001,"5min":0.92200000000000004,"15min":0.85399999999999998},"max":{"1min":2.02,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":1.8149999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574993,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1008,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194304,"statfs":{"total":200000000,"available":198967368,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90400000000000003,"15min":0.98599999999999999},"min":{"1min":0.68100000000000005,"5min":0.42399999999999999,"15min":0.34399999999999997},"max":{"1min":1.2589999999999999,"5min":1.506,"15min":9.2550000000000008},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.081,"5min":1.0129999999999999,"15min":1.0309999999999999},"min":{"1min":0.73199999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5620000000000001,"5min":1.7869999999999999,"15min":4.7850000000000001},"last":1.3280000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.96699999999999997,"15min":1.0720000000000001},"min":{"1min":0.70099999999999996,"5min":0.47599999999999998,"15min":0.46000000000000002},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":8.2720000000000002},"last":1.069},{"interface":"front","average":{"1min":1.194,"5min":1.1240000000000001,"15min":1.1399999999999999},"min":{"1min":0.86799999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.5609999999999999,"5min":1.8460000000000001,"15min":5.0620000000000003},"last":1.1539999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.2150000000000001,"15min":1.288},"min":{"1min":1.026,"5min":0.63300000000000001,"15min":0.51900000000000002},"max":{"1min":1.8200000000000001,"5min":2.8159999999999998,"15min":8.3109999999999999},"last":1.2589999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.103,"15min":1.234},"min":{"1min":0.88100000000000001,"5min":0.60299999999999998,"15min":0.41799999999999998},"max":{"1min":1.4610000000000001,"5min":2.6240000000000001,"15min":8.0690000000000008},"last":1.2889999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.3640000000000001,"15min":1.425},"min":{"1min":1.1990000000000001,"5min":0.81000000000000005,"15min":0.79300000000000004},"max":{"1min":1.7869999999999999,"5min":2.2160000000000002,"15min":5.3929999999999998},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.548,"5min":1.4650000000000001,"15min":1.504},"min":{"1min":1.052,"5min":0.82099999999999995,"15min":0.78300000000000003},"max":{"1min":1.8460000000000001,"5min":2.1949999999999998,"15min":7.3570000000000002},"last":1.611}]},{"osd":5,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.3839999999999999,"15min":1.47},"min":{"1min":1.133,"5min":0.91800000000000004,"15min":0.76200000000000001},"max":{"1min":1.802,"5min":2.0299999999999998,"15min":8.1839999999999993},"last":1.7190000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.369,"15min":1.401},"min":{"1min":1.095,"5min":0.82299999999999995,"15min":0.72199999999999998},"max":{"1min":1.702,"5min":1.9930000000000001,"15min":5.165},"last":1.496}]},{"osd":6,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.4279999999999999,"15min":1.4390000000000001},"min":{"1min":1.147,"5min":0.82399999999999995,"15min":0.72999999999999998},"max":{"1min":1.8680000000000001,"5min":2.5139999999999998,"15min":5.1059999999999999},"last":1.966},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4830000000000001,"15min":1.522},"min":{"1min":1.198,"5min":0.98499999999999999,"15min":0.77000000000000002},"max":{"1min":1.788,"5min":2.7559999999999998,"15min":8.1370000000000005},"last":1.996}]},{"osd":7,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.498,"15min":1.53},"min":{"1min":1.319,"5min":0.93500000000000005,"15min":0.78200000000000003},"max":{"1min":1.8029999999999999,"5min":2.3700000000000001,"15min":5.1449999999999996},"last":2.0419999999999998},{"interface":"front","average":{"1min":1.613,"5min":1.599,"15min":1.6140000000000001},"min":{"1min":1.4159999999999999,"5min":0.88500000000000001,"15min":0.86499999999999999},"max":{"1min":1.907,"5min":4.8289999999999997,"15min":8.2300000000000004},"last":1.921}]}]},{"osd":1,"up_from":13,"seq":55834574994,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1578,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193733,"statfs":{"total":200000000,"available":198383275,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.89800000000000002,"15min":0.86499999999999999},"min":{"1min":0.73699999999999999,"5min":0.41199999999999998,"15min":0.34699999999999998},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":0.90200000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":0.97899999999999998,"15min":0.96499999999999997},"min":{"1min":0.72099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.80900000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.004,"15min":1.0389999999999999},"min":{"1min":0.84299999999999997,"5min":0.52500000000000002,"15min":0.41399999999999998},"max":{"1min":1.452,"5min":2.4590000000000001,"15min":2.5739999999999998},"last":1.022},{"interface":"front","average":{"1min":1.1140000000000001,"5min":0.997,"15min":0.96499999999999997},"min":{"1min":0.82799999999999996,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":1.5860000000000001,"5min":1.718,"15min":1.845},"last":1.071}]},{"osd":3,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.2989999999999999,"15min":1.2270000000000001},"min":{"1min":1.3300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.8959999999999999,"5min":2.1040000000000001,"15min":2.137},"last":1.3740000000000001},{"interface":"front","average":{"1min":1.355,"5min":1.1950000000000001,"15min":1.2},"min":{"1min":0.91700000000000004,"5min":0.50800000000000001,"15min":0.44900000000000001},"max":{"1min":1.8260000000000001,"5min":2.1320000000000001,"15min":2.3959999999999999},"last":1.1890000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.304,"15min":1.3029999999999999},"min":{"1min":1.012,"5min":0.76400000000000001,"15min":0.69099999999999995},"max":{"1min":1.71,"5min":2.1440000000000001,"15min":2.444},"last":1.143},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.3640000000000001,"15min":1.3340000000000001},"min":{"1min":1.117,"5min":0.92500000000000004,"15min":0.71099999999999997},"max":{"1min":1.7789999999999999,"5min":1.8660000000000001,"15min":2.6440000000000001},"last":1.736}]},{"osd":5,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.371,"15min":1.3420000000000001},"min":{"1min":1.081,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.909,"5min":1.998,"15min":2.5459999999999998},"last":1.6859999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4119999999999999,"15min":1.425},"min":{"1min":1.218,"5min":0.85899999999999999,"15min":0.70499999999999996},"max":{"1min":1.821,"5min":2.371,"15min":2.6179999999999999},"last":1.516}]},{"osd":6,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.401,"15min":1.403},"min":{"1min":1.179,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.274,"5min":2.3610000000000002,"15min":2.7000000000000002},"last":1.4350000000000001},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.456,"15min":1.468},"min":{"1min":1.2769999999999999,"5min":0.98499999999999999,"15min":0.76400000000000001},"max":{"1min":1.8720000000000001,"5min":2.2719999999999998,"15min":2.895},"last":1.3240000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.413,"15min":1.427},"min":{"1min":1.3069999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.956,"5min":2.1629999999999998,"15min":9.4830000000000005},"last":1.6220000000000001},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.5069999999999999,"15min":1.53},"min":{"1min":1.331,"5min":0.874,"15min":0.71899999999999997},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":9.4109999999999996},"last":1.5920000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574993,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1008,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194304,"statfs":{"total":200000000,"available":198967368,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":0.89800000000000002,"15min":0.91300000000000003},"min":{"1min":0.746,"5min":0.311,"15min":0.311},"max":{"1min":1.671,"5min":1.671,"15min":2.3199999999999998},"last":1.256},{"interface":"front","average":{"1min":1.218,"5min":1.0549999999999999,"15min":1.0580000000000001},"min":{"1min":0.91900000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.4180000000000001},"last":1.1879999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":0.997,"15min":0.98299999999999998},"min":{"1min":0.81000000000000005,"5min":0.434,"15min":0.434},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.89800000000000002},{"interface":"front","average":{"1min":1.1619999999999999,"5min":0.98999999999999999,"15min":0.97299999999999998},"min":{"1min":0.81499999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.7810000000000001},"last":0.98699999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.0389999999999999,"15min":1.026},"min":{"1min":0.78600000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.7509999999999999,"5min":1.8200000000000001,"15min":2.5190000000000001},"last":1.1240000000000001},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.2949999999999999,"15min":1.2649999999999999},"min":{"1min":1.1200000000000001,"5min":0.53000000000000003,"15min":0.47499999999999998},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.401}]},{"osd":4,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.288,"15min":1.254},"min":{"1min":1.1819999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.5529999999999999},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.262,"15min":1.2829999999999999},"min":{"1min":1.0489999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.714,"5min":2.056,"15min":2.9529999999999998},"last":1.234}]},{"osd":5,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.319,"15min":1.3939999999999999},"min":{"1min":1.0700000000000001,"5min":0.68500000000000005,"15min":0.64500000000000002},"max":{"1min":1.867,"5min":1.97,"15min":16.298999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.3320000000000001,"15min":1.3129999999999999},"min":{"1min":1.1990000000000001,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.6220000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.4530000000000001,"15min":1.431},"min":{"1min":1.2769999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.5910000000000002},"last":1.52},{"interface":"front","average":{"1min":1.647,"5min":1.425,"15min":1.4019999999999999},"min":{"1min":1.1419999999999999,"5min":0.745,"15min":0.745},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.46},"last":1.502}]},{"osd":7,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.4530000000000001,"15min":1.4379999999999999},"min":{"1min":1.1699999999999999,"5min":0.72099999999999997,"15min":0.70899999999999996},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.4969999999999999},"last":1.383},{"interface":"front","average":{"1min":1.522,"5min":1.395,"15min":1.4199999999999999},"min":{"1min":1.165,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.913,"5min":2.113,"15min":3.1429999999999998},"last":1.5640000000000001}]}]}],"pool_statfs":[]}} 2023-12-19T18:26:43.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:26:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.791+0000 7fd7b759e640 1 Processor -- start 2023-12-19T18:26:43.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.792+0000 7fd7b759e640 1 -- start start 2023-12-19T18:26:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.793+0000 7fd7b759e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd7b8154030 0x7fd7b80cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.793+0000 7fd7b759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7b80af630 0x7fd7b80afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.793+0000 7fd7b759e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7b80affd0 con 0x7fd7b815d620 2023-12-19T18:26:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.793+0000 7fd7b759e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b80cc190 con 0x7fd7b8154030 2023-12-19T18:26:43.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.793+0000 7fd7b759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80cb2a0 con 0x7fd7b80af630 2023-12-19T18:26:43.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.794+0000 7fd7affff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7b80af630 0x7fd7b80afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:43.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.794+0000 7fd7affff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7b80af630 0x7fd7b80afa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:32938/0 (socket says 172.21.15.27:32938) 2023-12-19T18:26:43.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.794+0000 7fd7b659c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd7b8154030 0x7fd7b80cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:43.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.794+0000 7fd7affff640 1 -- 172.21.15.27:0/3965107497 learned_addr learned my addr 172.21.15.27:0/3965107497 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.795+0000 7fd7b659c640 1 -- 172.21.15.27:0/3965107497 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7b80af630 msgr2=0x7fd7b80afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.795+0000 7fd7b659c640 1 --2- 172.21.15.27:0/3965107497 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7b80af630 0x7fd7b80afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.795+0000 7fd7b659c640 1 -- 172.21.15.27:0/3965107497 >> v1:172.21.15.192:6789/0 conn(0x7fd7b815d620 legacy=0x7fd7b815da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:43.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.795+0000 7fd7b659c640 1 -- 172.21.15.27:0/3965107497 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80cb580 con 0x7fd7b8154030 2023-12-19T18:26:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.796+0000 7fd7b659c640 1 --2- 172.21.15.27:0/3965107497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b8154030 0x7fd7b80cd4f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80097a0 tx=0x7fd7a802b8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57198158ba914eab server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.797+0000 7fd7b5d9b640 1 -- 172.21.15.27:0/3965107497 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8031830 con 0x7fd7b8154030 2023-12-19T18:26:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.797+0000 7fd7b5d9b640 1 -- 172.21.15.27:0/3965107497 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a8031990 con 0x7fd7b8154030 2023-12-19T18:26:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.797+0000 7fd7b5d9b640 1 -- 172.21.15.27:0/3965107497 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a8031d20 con 0x7fd7b8154030 2023-12-19T18:26:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.798+0000 7fd7b759e640 1 -- 172.21.15.27:0/3965107497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b8154030 msgr2=0x7fd7b80cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.798+0000 7fd7b759e640 1 --2- 172.21.15.27:0/3965107497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b8154030 0x7fd7b80cd4f0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd7a80097a0 tx=0x7fd7a802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.798+0000 7fd7b759e640 1 -- 172.21.15.27:0/3965107497 shutdown_connections 2023-12-19T18:26:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.798+0000 7fd7b759e640 1 --2- 172.21.15.27:0/3965107497 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7b80af630 0x7fd7b80afa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.798+0000 7fd7b759e640 1 --2- 172.21.15.27:0/3965107497 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b8154030 0x7fd7b80cd4f0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.798+0000 7fd7b759e640 1 -- 172.21.15.27:0/3965107497 >> 172.21.15.27:0/3965107497 conn(0x7fd7b8069c70 msgr2=0x7fd7b806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.798+0000 7fd7b759e640 1 -- 172.21.15.27:0/3965107497 shutdown_connections 2023-12-19T18:26:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.798+0000 7fd7b759e640 1 -- 172.21.15.27:0/3965107497 wait complete. 2023-12-19T18:26:43.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.800+0000 7fd7b759e640 1 Processor -- start 2023-12-19T18:26:43.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.800+0000 7fd7b759e640 1 -- start start 2023-12-19T18:26:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.801+0000 7fd7b759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7b80af630 0x7fd7b80c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.801+0000 7fd7b759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b8154030 0x7fd7b814a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.801+0000 7fd7b759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7b815d620 0x7fd7b814a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.801+0000 7fd7b759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80c68b0 con 0x7fd7b80af630 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.801+0000 7fd7b759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80c6b90 con 0x7fd7b8154030 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.801+0000 7fd7b759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b80c6e70 con 0x7fd7b815d620 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.801+0000 7fd7b659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7b80af630 0x7fd7b80c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.801+0000 7fd7b659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7b80af630 0x7fd7b80c8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36026/0 (socket says 172.21.15.27:36026) 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.801+0000 7fd7b659c640 1 -- 172.21.15.27:0/1688363363 learned_addr learned my addr 172.21.15.27:0/1688363363 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.802+0000 7fd7affff640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b8154030 0x7fd7b814a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.802+0000 7fd7b659c640 1 -- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7b815d620 msgr2=0x7fd7b814a8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.802+0000 7fd7b6d9d640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7b815d620 0x7fd7b814a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.802+0000 7fd7b659c640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7b815d620 0x7fd7b814a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.802+0000 7fd7b659c640 1 -- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b8154030 msgr2=0x7fd7b814a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:43.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.802+0000 7fd7b659c640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b8154030 0x7fd7b814a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:43.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.802+0000 7fd7b659c640 1 -- 172.21.15.27:0/1688363363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a8009400 con 0x7fd7b80af630 2023-12-19T18:26:43.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.802+0000 7fd7b659c640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7b80af630 0x7fd7b80c8130 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fd7a802bee0 tx=0x7fd7a80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:43.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.802+0000 7fd7aeffd640 1 -- 172.21.15.27:0/1688363363 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a803a6f0 con 0x7fd7b80af630 2023-12-19T18:26:43.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.803+0000 7fd7aeffd640 1 -- 172.21.15.27:0/1688363363 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a803a870 con 0x7fd7b80af630 2023-12-19T18:26:43.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.803+0000 7fd7aeffd640 1 -- 172.21.15.27:0/1688363363 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a803ac20 con 0x7fd7b80af630 2023-12-19T18:26:43.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.803+0000 7fd7b759e640 1 -- 172.21.15.27:0/1688363363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b80c7150 con 0x7fd7b80af630 2023-12-19T18:26:43.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.803+0000 7fd7b759e640 1 -- 172.21.15.27:0/1688363363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b80c8670 con 0x7fd7b80af630 2023-12-19T18:26:43.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.804+0000 7fd7b759e640 1 -- 172.21.15.27:0/1688363363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b804e840 con 0x7fd7b80af630 2023-12-19T18:26:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.805+0000 7fd7aeffd640 1 -- 172.21.15.27:0/1688363363 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd7a8030380 con 0x7fd7b80af630 2023-12-19T18:26:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.806+0000 7fd7aeffd640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd79806a430 0x7fd79806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:43.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.807+0000 7fd7aeffd640 1 -- 172.21.15.27:0/1688363363 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7a80aa370 con 0x7fd7b80af630 2023-12-19T18:26:43.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.807+0000 7fd7affff640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd79806a430 0x7fd79806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:43.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.808+0000 7fd7affff640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd79806a430 0x7fd79806c8f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80b48f0 tx=0x7fd7a0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:43.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:43.811+0000 7fd7aeffd640 1 -- 172.21.15.27:0/1688363363 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd7a80758c0 con 0x7fd7b80af630 2023-12-19T18:26:44.005 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.005+0000 7fd7b759e640 1 -- 172.21.15.27:0/1688363363 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7b80cbb40 con 0x7fd79806a430 2023-12-19T18:26:44.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.029+0000 7fd7aeffd640 1 -- 172.21.15.27:0/1688363363 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282366 (secure 0 0 0) 0x7fd7a0011970 con 0x7fd79806a430 2023-12-19T18:26:44.032 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:44.035 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:26:44.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.036+0000 7fd7b759e640 1 -- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd79806a430 msgr2=0x7fd79806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:44.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.036+0000 7fd7b759e640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd79806a430 0x7fd79806c8f0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fd7b80b48f0 tx=0x7fd7a0008040 comp rx=0 tx=0).stop 2023-12-19T18:26:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.036+0000 7fd7b759e640 1 -- 172.21.15.27:0/1688363363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7b80af630 msgr2=0x7fd7b80c8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.036+0000 7fd7b759e640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7b80af630 0x7fd7b80c8130 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7fd7a802bee0 tx=0x7fd7a80046c0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.036+0000 7fd7b759e640 1 -- 172.21.15.27:0/1688363363 shutdown_connections 2023-12-19T18:26:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.036+0000 7fd7b759e640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd79806a430 0x7fd79806c8f0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.037+0000 7fd7b759e640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7b815d620 0x7fd7b814a8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.037+0000 7fd7b759e640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7b8154030 0x7fd7b814a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.037+0000 7fd7b759e640 1 --2- 172.21.15.27:0/1688363363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7b80af630 0x7fd7b80c8130 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.037+0000 7fd7b759e640 1 -- 172.21.15.27:0/1688363363 >> 172.21.15.27:0/1688363363 conn(0x7fd7b8069c70 msgr2=0x7fd7b806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.037+0000 7fd7b759e640 1 -- 172.21.15.27:0/1688363363 shutdown_connections 2023-12-19T18:26:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.037+0000 7fd7b759e640 1 -- 172.21.15.27:0/1688363363 wait complete. 2023-12-19T18:26:44.057 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":537,"stamp":"2023-12-19T18:26:43.966383+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9327,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1553172,"statfs":{"total":1600000000,"available":1590448884,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001996"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.813442+0000","last_change":"2023-12-19T18:26:38.511736+0000","last_active":"2023-12-19T18:26:38.813442+0000","last_peered":"2023-12-19T18:26:38.813442+0000","last_clean":"2023-12-19T18:26:38.813442+0000","last_became_active":"2023-12-19T18:26:38.508958+0000","last_became_peered":"2023-12-19T18:26:38.508958+0000","last_unstale":"2023-12-19T18:26:38.813442+0000","last_undegraded":"2023-12-19T18:26:38.813442+0000","last_fullsized":"2023-12-19T18:26:38.813442+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:34.370730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805233+0000","last_change":"2023-12-19T18:26:38.523848+0000","last_active":"2023-12-19T18:26:38.805233+0000","last_peered":"2023-12-19T18:26:38.805233+0000","last_clean":"2023-12-19T18:26:38.805233+0000","last_became_active":"2023-12-19T18:26:38.523619+0000","last_became_peered":"2023-12-19T18:26:38.523619+0000","last_unstale":"2023-12-19T18:26:38.805233+0000","last_undegraded":"2023-12-19T18:26:38.805233+0000","last_fullsized":"2023-12-19T18:26:38.805233+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:47:44.420268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.811576+0000","last_change":"2023-12-19T18:26:38.512087+0000","last_active":"2023-12-19T18:26:38.811576+0000","last_peered":"2023-12-19T18:26:38.811576+0000","last_clean":"2023-12-19T18:26:38.811576+0000","last_became_active":"2023-12-19T18:26:38.510831+0000","last_became_peered":"2023-12-19T18:26:38.510831+0000","last_unstale":"2023-12-19T18:26:38.811576+0000","last_undegraded":"2023-12-19T18:26:38.811576+0000","last_fullsized":"2023-12-19T18:26:38.811576+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:29:48.773763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804602+0000","last_change":"2023-12-19T18:26:38.512424+0000","last_active":"2023-12-19T18:26:38.804602+0000","last_peered":"2023-12-19T18:26:38.804602+0000","last_clean":"2023-12-19T18:26:38.804602+0000","last_became_active":"2023-12-19T18:26:38.512202+0000","last_became_peered":"2023-12-19T18:26:38.512202+0000","last_unstale":"2023-12-19T18:26:38.804602+0000","last_undegraded":"2023-12-19T18:26:38.804602+0000","last_fullsized":"2023-12-19T18:26:38.804602+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:27:22.514431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.796462+0000","last_change":"2023-12-19T18:26:38.507304+0000","last_active":"2023-12-19T18:26:38.796462+0000","last_peered":"2023-12-19T18:26:38.796462+0000","last_clean":"2023-12-19T18:26:38.796462+0000","last_became_active":"2023-12-19T18:26:38.507063+0000","last_became_peered":"2023-12-19T18:26:38.507063+0000","last_unstale":"2023-12-19T18:26:38.796462+0000","last_undegraded":"2023-12-19T18:26:38.796462+0000","last_fullsized":"2023-12-19T18:26:38.796462+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:25:11.204756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.812717+0000","last_change":"2023-12-19T18:26:38.518271+0000","last_active":"2023-12-19T18:26:38.812717+0000","last_peered":"2023-12-19T18:26:38.812717+0000","last_clean":"2023-12-19T18:26:38.812717+0000","last_became_active":"2023-12-19T18:26:38.517107+0000","last_became_peered":"2023-12-19T18:26:38.517107+0000","last_unstale":"2023-12-19T18:26:38.812717+0000","last_undegraded":"2023-12-19T18:26:38.812717+0000","last_fullsized":"2023-12-19T18:26:38.812717+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:07:27.156222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.795675+0000","last_change":"2023-12-19T18:26:38.511992+0000","last_active":"2023-12-19T18:26:38.795675+0000","last_peered":"2023-12-19T18:26:38.795675+0000","last_clean":"2023-12-19T18:26:38.795675+0000","last_became_active":"2023-12-19T18:26:38.511518+0000","last_became_peered":"2023-12-19T18:26:38.511518+0000","last_unstale":"2023-12-19T18:26:38.795675+0000","last_undegraded":"2023-12-19T18:26:38.795675+0000","last_fullsized":"2023-12-19T18:26:38.795675+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:09.636074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.813370+0000","last_change":"2023-12-19T18:26:38.520362+0000","last_active":"2023-12-19T18:26:38.813370+0000","last_peered":"2023-12-19T18:26:38.813370+0000","last_clean":"2023-12-19T18:26:38.813370+0000","last_became_active":"2023-12-19T18:26:38.519835+0000","last_became_peered":"2023-12-19T18:26:38.519835+0000","last_unstale":"2023-12-19T18:26:38.813370+0000","last_undegraded":"2023-12-19T18:26:38.813370+0000","last_fullsized":"2023-12-19T18:26:38.813370+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:22:24.538127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.812478+0000","last_change":"2023-12-19T18:26:38.520612+0000","last_active":"2023-12-19T18:26:38.812478+0000","last_peered":"2023-12-19T18:26:38.812478+0000","last_clean":"2023-12-19T18:26:38.812478+0000","last_became_active":"2023-12-19T18:26:38.520138+0000","last_became_peered":"2023-12-19T18:26:38.520138+0000","last_unstale":"2023-12-19T18:26:38.812478+0000","last_undegraded":"2023-12-19T18:26:38.812478+0000","last_fullsized":"2023-12-19T18:26:38.812478+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:13.074404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.811784+0000","last_change":"2023-12-19T18:26:38.502557+0000","last_active":"2023-12-19T18:26:38.811784+0000","last_peered":"2023-12-19T18:26:38.811784+0000","last_clean":"2023-12-19T18:26:38.811784+0000","last_became_active":"2023-12-19T18:26:38.502172+0000","last_became_peered":"2023-12-19T18:26:38.502172+0000","last_unstale":"2023-12-19T18:26:38.811784+0000","last_undegraded":"2023-12-19T18:26:38.811784+0000","last_fullsized":"2023-12-19T18:26:38.811784+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:21:44.849567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.812654+0000","last_change":"2023-12-19T18:26:38.513600+0000","last_active":"2023-12-19T18:26:38.812654+0000","last_peered":"2023-12-19T18:26:38.812654+0000","last_clean":"2023-12-19T18:26:38.812654+0000","last_became_active":"2023-12-19T18:26:38.512761+0000","last_became_peered":"2023-12-19T18:26:38.512761+0000","last_unstale":"2023-12-19T18:26:38.812654+0000","last_undegraded":"2023-12-19T18:26:38.812654+0000","last_fullsized":"2023-12-19T18:26:38.812654+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:39:34.284978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.798493+0000","last_change":"2023-12-19T18:26:38.514384+0000","last_active":"2023-12-19T18:26:38.798493+0000","last_peered":"2023-12-19T18:26:38.798493+0000","last_clean":"2023-12-19T18:26:38.798493+0000","last_became_active":"2023-12-19T18:26:38.512957+0000","last_became_peered":"2023-12-19T18:26:38.512957+0000","last_unstale":"2023-12-19T18:26:38.798493+0000","last_undegraded":"2023-12-19T18:26:38.798493+0000","last_fullsized":"2023-12-19T18:26:38.798493+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:13.074404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.796331+0000","last_change":"2023-12-19T18:26:38.508388+0000","last_active":"2023-12-19T18:26:38.796331+0000","last_peered":"2023-12-19T18:26:38.796331+0000","last_clean":"2023-12-19T18:26:38.796331+0000","last_became_active":"2023-12-19T18:26:38.508159+0000","last_became_peered":"2023-12-19T18:26:38.508159+0000","last_unstale":"2023-12-19T18:26:38.796331+0000","last_undegraded":"2023-12-19T18:26:38.796331+0000","last_fullsized":"2023-12-19T18:26:38.796331+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:57:41.432563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.798423+0000","last_change":"2023-12-19T18:26:38.500039+0000","last_active":"2023-12-19T18:26:38.798423+0000","last_peered":"2023-12-19T18:26:38.798423+0000","last_clean":"2023-12-19T18:26:38.798423+0000","last_became_active":"2023-12-19T18:26:38.499817+0000","last_became_peered":"2023-12-19T18:26:38.499817+0000","last_unstale":"2023-12-19T18:26:38.798423+0000","last_undegraded":"2023-12-19T18:26:38.798423+0000","last_fullsized":"2023-12-19T18:26:38.798423+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:09:57.453540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.796483+0000","last_change":"2023-12-19T18:26:38.504914+0000","last_active":"2023-12-19T18:26:38.796483+0000","last_peered":"2023-12-19T18:26:38.796483+0000","last_clean":"2023-12-19T18:26:38.796483+0000","last_became_active":"2023-12-19T18:26:38.504653+0000","last_became_peered":"2023-12-19T18:26:38.504653+0000","last_unstale":"2023-12-19T18:26:38.796483+0000","last_undegraded":"2023-12-19T18:26:38.796483+0000","last_fullsized":"2023-12-19T18:26:38.796483+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:29:57.647894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.795718+0000","last_change":"2023-12-19T18:26:38.522909+0000","last_active":"2023-12-19T18:26:38.795718+0000","last_peered":"2023-12-19T18:26:38.795718+0000","last_clean":"2023-12-19T18:26:38.795718+0000","last_became_active":"2023-12-19T18:26:38.522655+0000","last_became_peered":"2023-12-19T18:26:38.522655+0000","last_unstale":"2023-12-19T18:26:38.795718+0000","last_undegraded":"2023-12-19T18:26:38.795718+0000","last_fullsized":"2023-12-19T18:26:38.795718+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:59:50.875803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.811689+0000","last_change":"2023-12-19T18:26:38.507509+0000","last_active":"2023-12-19T18:26:38.811689+0000","last_peered":"2023-12-19T18:26:38.811689+0000","last_clean":"2023-12-19T18:26:38.811689+0000","last_became_active":"2023-12-19T18:26:38.507230+0000","last_became_peered":"2023-12-19T18:26:38.507230+0000","last_unstale":"2023-12-19T18:26:38.811689+0000","last_undegraded":"2023-12-19T18:26:38.811689+0000","last_fullsized":"2023-12-19T18:26:38.811689+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:44:57.712706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.796352+0000","last_change":"2023-12-19T18:26:38.512437+0000","last_active":"2023-12-19T18:26:38.796352+0000","last_peered":"2023-12-19T18:26:38.796352+0000","last_clean":"2023-12-19T18:26:38.796352+0000","last_became_active":"2023-12-19T18:26:38.512125+0000","last_became_peered":"2023-12-19T18:26:38.512125+0000","last_unstale":"2023-12-19T18:26:38.796352+0000","last_undegraded":"2023-12-19T18:26:38.796352+0000","last_fullsized":"2023-12-19T18:26:38.796352+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:24:49.646820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.813343+0000","last_change":"2023-12-19T18:26:38.522323+0000","last_active":"2023-12-19T18:26:38.813343+0000","last_peered":"2023-12-19T18:26:38.813343+0000","last_clean":"2023-12-19T18:26:38.813343+0000","last_became_active":"2023-12-19T18:26:38.514076+0000","last_became_peered":"2023-12-19T18:26:38.514076+0000","last_unstale":"2023-12-19T18:26:38.813343+0000","last_undegraded":"2023-12-19T18:26:38.813343+0000","last_fullsized":"2023-12-19T18:26:38.813343+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:15:59.240438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.813495+0000","last_change":"2023-12-19T18:26:38.524051+0000","last_active":"2023-12-19T18:26:38.813495+0000","last_peered":"2023-12-19T18:26:38.813495+0000","last_clean":"2023-12-19T18:26:38.813495+0000","last_became_active":"2023-12-19T18:26:38.523668+0000","last_became_peered":"2023-12-19T18:26:38.523668+0000","last_unstale":"2023-12-19T18:26:38.813495+0000","last_undegraded":"2023-12-19T18:26:38.813495+0000","last_fullsized":"2023-12-19T18:26:38.813495+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:16:40.828355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.811735+0000","last_change":"2023-12-19T18:26:38.513151+0000","last_active":"2023-12-19T18:26:38.811735+0000","last_peered":"2023-12-19T18:26:38.811735+0000","last_clean":"2023-12-19T18:26:38.811735+0000","last_became_active":"2023-12-19T18:26:38.511855+0000","last_became_peered":"2023-12-19T18:26:38.511855+0000","last_unstale":"2023-12-19T18:26:38.811735+0000","last_undegraded":"2023-12-19T18:26:38.811735+0000","last_fullsized":"2023-12-19T18:26:38.811735+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:44:23.624624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.798523+0000","last_change":"2023-12-19T18:26:38.505779+0000","last_active":"2023-12-19T18:26:38.798523+0000","last_peered":"2023-12-19T18:26:38.798523+0000","last_clean":"2023-12-19T18:26:38.798523+0000","last_became_active":"2023-12-19T18:26:38.505432+0000","last_became_peered":"2023-12-19T18:26:38.505432+0000","last_unstale":"2023-12-19T18:26:38.798523+0000","last_undegraded":"2023-12-19T18:26:38.798523+0000","last_fullsized":"2023-12-19T18:26:38.798523+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:39:34.284978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804744+0000","last_change":"2023-12-19T18:26:38.514929+0000","last_active":"2023-12-19T18:26:38.804744+0000","last_peered":"2023-12-19T18:26:38.804744+0000","last_clean":"2023-12-19T18:26:38.804744+0000","last_became_active":"2023-12-19T18:26:38.514200+0000","last_became_peered":"2023-12-19T18:26:38.514200+0000","last_unstale":"2023-12-19T18:26:38.804744+0000","last_undegraded":"2023-12-19T18:26:38.804744+0000","last_fullsized":"2023-12-19T18:26:38.804744+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:17:35.926413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.796251+0000","last_change":"2023-12-19T18:26:38.501316+0000","last_active":"2023-12-19T18:26:38.796251+0000","last_peered":"2023-12-19T18:26:38.796251+0000","last_clean":"2023-12-19T18:26:38.796251+0000","last_became_active":"2023-12-19T18:26:38.501052+0000","last_became_peered":"2023-12-19T18:26:38.501052+0000","last_unstale":"2023-12-19T18:26:38.796251+0000","last_undegraded":"2023-12-19T18:26:38.796251+0000","last_fullsized":"2023-12-19T18:26:38.796251+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:55:51.941470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805501+0000","last_change":"2023-12-19T18:26:38.525353+0000","last_active":"2023-12-19T18:26:38.805501+0000","last_peered":"2023-12-19T18:26:38.805501+0000","last_clean":"2023-12-19T18:26:38.805501+0000","last_became_active":"2023-12-19T18:26:38.524781+0000","last_became_peered":"2023-12-19T18:26:38.524781+0000","last_unstale":"2023-12-19T18:26:38.805501+0000","last_undegraded":"2023-12-19T18:26:38.805501+0000","last_fullsized":"2023-12-19T18:26:38.805501+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:30.507693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.813464+0000","last_change":"2023-12-19T18:26:38.521257+0000","last_active":"2023-12-19T18:26:38.813464+0000","last_peered":"2023-12-19T18:26:38.813464+0000","last_clean":"2023-12-19T18:26:38.813464+0000","last_became_active":"2023-12-19T18:26:38.521027+0000","last_became_peered":"2023-12-19T18:26:38.521027+0000","last_unstale":"2023-12-19T18:26:38.813464+0000","last_undegraded":"2023-12-19T18:26:38.813464+0000","last_fullsized":"2023-12-19T18:26:38.813464+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:13:51.785468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805253+0000","last_change":"2023-12-19T18:26:38.525961+0000","last_active":"2023-12-19T18:26:38.805253+0000","last_peered":"2023-12-19T18:26:38.805253+0000","last_clean":"2023-12-19T18:26:38.805253+0000","last_became_active":"2023-12-19T18:26:38.525681+0000","last_became_peered":"2023-12-19T18:26:38.525681+0000","last_unstale":"2023-12-19T18:26:38.805253+0000","last_undegraded":"2023-12-19T18:26:38.805253+0000","last_fullsized":"2023-12-19T18:26:38.805253+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:09:57.453540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805144+0000","last_change":"2023-12-19T18:26:38.524304+0000","last_active":"2023-12-19T18:26:38.805144+0000","last_peered":"2023-12-19T18:26:38.805144+0000","last_clean":"2023-12-19T18:26:38.805144+0000","last_became_active":"2023-12-19T18:26:38.524054+0000","last_became_peered":"2023-12-19T18:26:38.524054+0000","last_unstale":"2023-12-19T18:26:38.805144+0000","last_undegraded":"2023-12-19T18:26:38.805144+0000","last_fullsized":"2023-12-19T18:26:38.805144+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:38:02.035441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.813257+0000","last_change":"2023-12-19T18:26:38.509646+0000","last_active":"2023-12-19T18:26:38.813257+0000","last_peered":"2023-12-19T18:26:38.813257+0000","last_clean":"2023-12-19T18:26:38.813257+0000","last_became_active":"2023-12-19T18:26:38.509086+0000","last_became_peered":"2023-12-19T18:26:38.509086+0000","last_unstale":"2023-12-19T18:26:38.813257+0000","last_undegraded":"2023-12-19T18:26:38.813257+0000","last_fullsized":"2023-12-19T18:26:38.813257+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:05:31.383005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.798613+0000","last_change":"2023-12-19T18:26:38.510763+0000","last_active":"2023-12-19T18:26:38.798613+0000","last_peered":"2023-12-19T18:26:38.798613+0000","last_clean":"2023-12-19T18:26:38.798613+0000","last_became_active":"2023-12-19T18:26:38.503707+0000","last_became_peered":"2023-12-19T18:26:38.503707+0000","last_unstale":"2023-12-19T18:26:38.798613+0000","last_undegraded":"2023-12-19T18:26:38.798613+0000","last_fullsized":"2023-12-19T18:26:38.798613+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:57:04.171752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805449+0000","last_change":"2023-12-19T18:26:38.520939+0000","last_active":"2023-12-19T18:26:38.805449+0000","last_peered":"2023-12-19T18:26:38.805449+0000","last_clean":"2023-12-19T18:26:38.805449+0000","last_became_active":"2023-12-19T18:26:38.520690+0000","last_became_peered":"2023-12-19T18:26:38.520690+0000","last_unstale":"2023-12-19T18:26:38.805449+0000","last_undegraded":"2023-12-19T18:26:38.805449+0000","last_fullsized":"2023-12-19T18:26:38.805449+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:00:53.886763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805279+0000","last_change":"2023-12-19T18:26:38.520370+0000","last_active":"2023-12-19T18:26:38.805279+0000","last_peered":"2023-12-19T18:26:38.805279+0000","last_clean":"2023-12-19T18:26:38.805279+0000","last_became_active":"2023-12-19T18:26:38.520135+0000","last_became_peered":"2023-12-19T18:26:38.520135+0000","last_unstale":"2023-12-19T18:26:38.805279+0000","last_undegraded":"2023-12-19T18:26:38.805279+0000","last_fullsized":"2023-12-19T18:26:38.805279+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:31.201464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.796292+0000","last_change":"2023-12-19T18:26:38.510459+0000","last_active":"2023-12-19T18:26:38.796292+0000","last_peered":"2023-12-19T18:26:38.796292+0000","last_clean":"2023-12-19T18:26:38.796292+0000","last_became_active":"2023-12-19T18:26:38.510258+0000","last_became_peered":"2023-12-19T18:26:38.510258+0000","last_unstale":"2023-12-19T18:26:38.796292+0000","last_undegraded":"2023-12-19T18:26:38.796292+0000","last_fullsized":"2023-12-19T18:26:38.796292+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:57:57.578030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"184'4","reported_seq":15,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.811632+0000","last_change":"2023-12-19T18:26:38.000068+0000","last_active":"2023-12-19T18:26:38.811632+0000","last_peered":"2023-12-19T18:26:38.811632+0000","last_clean":"2023-12-19T18:26:38.811632+0000","last_became_active":"2023-12-19T18:26:37.999319+0000","last_became_peered":"2023-12-19T18:26:37.999319+0000","last_unstale":"2023-12-19T18:26:38.811632+0000","last_undegraded":"2023-12-19T18:26:38.811632+0000","last_fullsized":"2023-12-19T18:26:38.811632+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:19:49.762606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.795627+0000","last_change":"2023-12-19T18:26:38.504737+0000","last_active":"2023-12-19T18:26:38.795627+0000","last_peered":"2023-12-19T18:26:38.795627+0000","last_clean":"2023-12-19T18:26:38.795627+0000","last_became_active":"2023-12-19T18:26:38.502873+0000","last_became_peered":"2023-12-19T18:26:38.502873+0000","last_unstale":"2023-12-19T18:26:38.795627+0000","last_undegraded":"2023-12-19T18:26:38.795627+0000","last_fullsized":"2023-12-19T18:26:38.795627+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:42.815725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"184'1","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.812531+0000","last_change":"2023-12-19T18:26:38.002073+0000","last_active":"2023-12-19T18:26:38.812531+0000","last_peered":"2023-12-19T18:26:38.812531+0000","last_clean":"2023-12-19T18:26:38.812531+0000","last_became_active":"2023-12-19T18:26:38.001660+0000","last_became_peered":"2023-12-19T18:26:38.001660+0000","last_unstale":"2023-12-19T18:26:38.812531+0000","last_undegraded":"2023-12-19T18:26:38.812531+0000","last_fullsized":"2023-12-19T18:26:38.812531+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:47:43.408526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.812675+0000","last_change":"2023-12-19T18:26:38.520945+0000","last_active":"2023-12-19T18:26:38.812675+0000","last_peered":"2023-12-19T18:26:38.812675+0000","last_clean":"2023-12-19T18:26:38.812675+0000","last_became_active":"2023-12-19T18:26:38.517960+0000","last_became_peered":"2023-12-19T18:26:38.517960+0000","last_unstale":"2023-12-19T18:26:38.812675+0000","last_undegraded":"2023-12-19T18:26:38.812675+0000","last_fullsized":"2023-12-19T18:26:38.812675+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:59:50.875803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"184'2","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.811807+0000","last_change":"2023-12-19T18:26:38.002984+0000","last_active":"2023-12-19T18:26:38.811807+0000","last_peered":"2023-12-19T18:26:38.811807+0000","last_clean":"2023-12-19T18:26:38.811807+0000","last_became_active":"2023-12-19T18:26:38.001283+0000","last_became_peered":"2023-12-19T18:26:38.001283+0000","last_unstale":"2023-12-19T18:26:38.811807+0000","last_undegraded":"2023-12-19T18:26:38.811807+0000","last_fullsized":"2023-12-19T18:26:38.811807+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:52.628278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804995+0000","last_change":"2023-12-19T18:26:38.514543+0000","last_active":"2023-12-19T18:26:38.804995+0000","last_peered":"2023-12-19T18:26:38.804995+0000","last_clean":"2023-12-19T18:26:38.804995+0000","last_became_active":"2023-12-19T18:26:38.514308+0000","last_became_peered":"2023-12-19T18:26:38.514308+0000","last_unstale":"2023-12-19T18:26:38.804995+0000","last_undegraded":"2023-12-19T18:26:38.804995+0000","last_fullsized":"2023-12-19T18:26:38.804995+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:16:39.937501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"184'2","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.811758+0000","last_change":"2023-12-19T18:26:38.000265+0000","last_active":"2023-12-19T18:26:38.811758+0000","last_peered":"2023-12-19T18:26:38.811758+0000","last_clean":"2023-12-19T18:26:38.811758+0000","last_became_active":"2023-12-19T18:26:37.999232+0000","last_became_peered":"2023-12-19T18:26:37.999232+0000","last_unstale":"2023-12-19T18:26:38.811758+0000","last_undegraded":"2023-12-19T18:26:38.811758+0000","last_fullsized":"2023-12-19T18:26:38.811758+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:57:43.500742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805166+0000","last_change":"2023-12-19T18:26:38.521202+0000","last_active":"2023-12-19T18:26:38.805166+0000","last_peered":"2023-12-19T18:26:38.805166+0000","last_clean":"2023-12-19T18:26:38.805166+0000","last_became_active":"2023-12-19T18:26:38.518592+0000","last_became_peered":"2023-12-19T18:26:38.518592+0000","last_unstale":"2023-12-19T18:26:38.805166+0000","last_undegraded":"2023-12-19T18:26:38.805166+0000","last_fullsized":"2023-12-19T18:26:38.805166+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:06:42.420094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"184'1","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.813418+0000","last_change":"2023-12-19T18:26:37.995673+0000","last_active":"2023-12-19T18:26:38.813418+0000","last_peered":"2023-12-19T18:26:38.813418+0000","last_clean":"2023-12-19T18:26:38.813418+0000","last_became_active":"2023-12-19T18:26:37.995097+0000","last_became_peered":"2023-12-19T18:26:37.995097+0000","last_unstale":"2023-12-19T18:26:38.813418+0000","last_undegraded":"2023-12-19T18:26:38.813418+0000","last_fullsized":"2023-12-19T18:26:38.813418+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:59:49.864061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804811+0000","last_change":"2023-12-19T18:26:38.516422+0000","last_active":"2023-12-19T18:26:38.804811+0000","last_peered":"2023-12-19T18:26:38.804811+0000","last_clean":"2023-12-19T18:26:38.804811+0000","last_became_active":"2023-12-19T18:26:38.515525+0000","last_became_peered":"2023-12-19T18:26:38.515525+0000","last_unstale":"2023-12-19T18:26:38.804811+0000","last_undegraded":"2023-12-19T18:26:38.804811+0000","last_fullsized":"2023-12-19T18:26:38.804811+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:13:40.079418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"184'1","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.796380+0000","last_change":"2023-12-19T18:26:38.002164+0000","last_active":"2023-12-19T18:26:38.796380+0000","last_peered":"2023-12-19T18:26:38.796380+0000","last_clean":"2023-12-19T18:26:38.796380+0000","last_became_active":"2023-12-19T18:26:38.001879+0000","last_became_peered":"2023-12-19T18:26:38.001879+0000","last_unstale":"2023-12-19T18:26:38.796380+0000","last_undegraded":"2023-12-19T18:26:38.796380+0000","last_fullsized":"2023-12-19T18:26:38.796380+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:21:34.578038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805526+0000","last_change":"2023-12-19T18:26:38.522611+0000","last_active":"2023-12-19T18:26:38.805526+0000","last_peered":"2023-12-19T18:26:38.805526+0000","last_clean":"2023-12-19T18:26:38.805526+0000","last_became_active":"2023-12-19T18:26:38.522155+0000","last_became_peered":"2023-12-19T18:26:38.522155+0000","last_unstale":"2023-12-19T18:26:38.805526+0000","last_undegraded":"2023-12-19T18:26:38.805526+0000","last_fullsized":"2023-12-19T18:26:38.805526+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:16:40.950387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.812579+0000","last_change":"2023-12-19T18:26:38.000955+0000","last_active":"2023-12-19T18:26:38.812579+0000","last_peered":"2023-12-19T18:26:38.812579+0000","last_clean":"2023-12-19T18:26:38.812579+0000","last_became_active":"2023-12-19T18:26:37.999522+0000","last_became_peered":"2023-12-19T18:26:37.999522+0000","last_unstale":"2023-12-19T18:26:38.812579+0000","last_undegraded":"2023-12-19T18:26:38.812579+0000","last_fullsized":"2023-12-19T18:26:38.812579+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:38:01.023699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804632+0000","last_change":"2023-12-19T18:26:38.509484+0000","last_active":"2023-12-19T18:26:38.804632+0000","last_peered":"2023-12-19T18:26:38.804632+0000","last_clean":"2023-12-19T18:26:38.804632+0000","last_became_active":"2023-12-19T18:26:38.508921+0000","last_became_peered":"2023-12-19T18:26:38.508921+0000","last_unstale":"2023-12-19T18:26:38.804632+0000","last_undegraded":"2023-12-19T18:26:38.804632+0000","last_fullsized":"2023-12-19T18:26:38.804632+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:15:21.211976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.811829+0000","last_change":"2023-12-19T18:26:38.005154+0000","last_active":"2023-12-19T18:26:38.811829+0000","last_peered":"2023-12-19T18:26:38.811829+0000","last_clean":"2023-12-19T18:26:38.811829+0000","last_became_active":"2023-12-19T18:26:38.004729+0000","last_became_peered":"2023-12-19T18:26:38.004729+0000","last_unstale":"2023-12-19T18:26:38.811829+0000","last_undegraded":"2023-12-19T18:26:38.811829+0000","last_fullsized":"2023-12-19T18:26:38.811829+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:10:29.677956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804833+0000","last_change":"2023-12-19T18:26:38.516869+0000","last_active":"2023-12-19T18:26:38.804833+0000","last_peered":"2023-12-19T18:26:38.804833+0000","last_clean":"2023-12-19T18:26:38.804833+0000","last_became_active":"2023-12-19T18:26:38.516188+0000","last_became_peered":"2023-12-19T18:26:38.516188+0000","last_unstale":"2023-12-19T18:26:38.804833+0000","last_undegraded":"2023-12-19T18:26:38.804833+0000","last_fullsized":"2023-12-19T18:26:38.804833+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:59:50.875803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.812749+0000","last_change":"2023-12-19T18:26:37.999814+0000","last_active":"2023-12-19T18:26:38.812749+0000","last_peered":"2023-12-19T18:26:38.812749+0000","last_clean":"2023-12-19T18:26:38.812749+0000","last_became_active":"2023-12-19T18:26:37.999101+0000","last_became_peered":"2023-12-19T18:26:37.999101+0000","last_unstale":"2023-12-19T18:26:38.812749+0000","last_undegraded":"2023-12-19T18:26:38.812749+0000","last_fullsized":"2023-12-19T18:26:38.812749+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:29.495951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.798551+0000","last_change":"2023-12-19T18:26:38.509020+0000","last_active":"2023-12-19T18:26:38.798551+0000","last_peered":"2023-12-19T18:26:38.798551+0000","last_clean":"2023-12-19T18:26:38.798551+0000","last_became_active":"2023-12-19T18:26:38.505997+0000","last_became_peered":"2023-12-19T18:26:38.505997+0000","last_unstale":"2023-12-19T18:26:38.798551+0000","last_undegraded":"2023-12-19T18:26:38.798551+0000","last_fullsized":"2023-12-19T18:26:38.798551+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:07:27.156222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.796439+0000","last_change":"2023-12-19T18:26:37.997613+0000","last_active":"2023-12-19T18:26:38.796439+0000","last_peered":"2023-12-19T18:26:38.796439+0000","last_clean":"2023-12-19T18:26:38.796439+0000","last_became_active":"2023-12-19T18:26:37.995932+0000","last_became_peered":"2023-12-19T18:26:37.995932+0000","last_unstale":"2023-12-19T18:26:38.796439+0000","last_undegraded":"2023-12-19T18:26:38.796439+0000","last_fullsized":"2023-12-19T18:26:38.796439+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:07:06.356535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.795652+0000","last_change":"2023-12-19T18:26:38.520807+0000","last_active":"2023-12-19T18:26:38.795652+0000","last_peered":"2023-12-19T18:26:38.795652+0000","last_clean":"2023-12-19T18:26:38.795652+0000","last_became_active":"2023-12-19T18:26:38.520565+0000","last_became_peered":"2023-12-19T18:26:38.520565+0000","last_unstale":"2023-12-19T18:26:38.795652+0000","last_undegraded":"2023-12-19T18:26:38.795652+0000","last_fullsized":"2023-12-19T18:26:38.795652+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:09:06.034401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"184'6","reported_seq":17,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.795548+0000","last_change":"2023-12-19T18:26:37.994368+0000","last_active":"2023-12-19T18:26:38.795548+0000","last_peered":"2023-12-19T18:26:38.795548+0000","last_clean":"2023-12-19T18:26:38.795548+0000","last_became_active":"2023-12-19T18:26:37.993322+0000","last_became_peered":"2023-12-19T18:26:37.993322+0000","last_unstale":"2023-12-19T18:26:38.795548+0000","last_undegraded":"2023-12-19T18:26:38.795548+0000","last_fullsized":"2023-12-19T18:26:38.795548+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:38:01.023699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804924+0000","last_change":"2023-12-19T18:26:38.513554+0000","last_active":"2023-12-19T18:26:38.804924+0000","last_peered":"2023-12-19T18:26:38.804924+0000","last_clean":"2023-12-19T18:26:38.804924+0000","last_became_active":"2023-12-19T18:26:38.512437+0000","last_became_peered":"2023-12-19T18:26:38.512437+0000","last_unstale":"2023-12-19T18:26:38.804924+0000","last_undegraded":"2023-12-19T18:26:38.804924+0000","last_fullsized":"2023-12-19T18:26:38.804924+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:24:00.958700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"184'2","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.811854+0000","last_change":"2023-12-19T18:26:38.002798+0000","last_active":"2023-12-19T18:26:38.811854+0000","last_peered":"2023-12-19T18:26:38.811854+0000","last_clean":"2023-12-19T18:26:38.811854+0000","last_became_active":"2023-12-19T18:26:38.000972+0000","last_became_peered":"2023-12-19T18:26:38.000972+0000","last_unstale":"2023-12-19T18:26:38.811854+0000","last_undegraded":"2023-12-19T18:26:38.811854+0000","last_fullsized":"2023-12-19T18:26:38.811854+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:10:48.802437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.813542+0000","last_change":"2023-12-19T18:26:38.517459+0000","last_active":"2023-12-19T18:26:38.813542+0000","last_peered":"2023-12-19T18:26:38.813542+0000","last_clean":"2023-12-19T18:26:38.813542+0000","last_became_active":"2023-12-19T18:26:38.517106+0000","last_became_peered":"2023-12-19T18:26:38.517106+0000","last_unstale":"2023-12-19T18:26:38.813542+0000","last_undegraded":"2023-12-19T18:26:38.813542+0000","last_fullsized":"2023-12-19T18:26:38.813542+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:42:20.080675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.798593+0000","last_change":"2023-12-19T18:26:38.006106+0000","last_active":"2023-12-19T18:26:38.798593+0000","last_peered":"2023-12-19T18:26:38.798593+0000","last_clean":"2023-12-19T18:26:38.798593+0000","last_became_active":"2023-12-19T18:26:38.005595+0000","last_became_peered":"2023-12-19T18:26:38.005595+0000","last_unstale":"2023-12-19T18:26:38.798593+0000","last_undegraded":"2023-12-19T18:26:38.798593+0000","last_fullsized":"2023-12-19T18:26:38.798593+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:18:28.254508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804766+0000","last_change":"2023-12-19T18:26:38.515126+0000","last_active":"2023-12-19T18:26:38.804766+0000","last_peered":"2023-12-19T18:26:38.804766+0000","last_clean":"2023-12-19T18:26:38.804766+0000","last_became_active":"2023-12-19T18:26:38.514605+0000","last_became_peered":"2023-12-19T18:26:38.514605+0000","last_unstale":"2023-12-19T18:26:38.804766+0000","last_undegraded":"2023-12-19T18:26:38.804766+0000","last_fullsized":"2023-12-19T18:26:38.804766+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:09:09.636074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805368+0000","last_change":"2023-12-19T18:26:38.001707+0000","last_active":"2023-12-19T18:26:38.805368+0000","last_peered":"2023-12-19T18:26:38.805368+0000","last_clean":"2023-12-19T18:26:38.805368+0000","last_became_active":"2023-12-19T18:26:38.001487+0000","last_became_peered":"2023-12-19T18:26:38.001487+0000","last_unstale":"2023-12-19T18:26:38.805368+0000","last_undegraded":"2023-12-19T18:26:38.805368+0000","last_fullsized":"2023-12-19T18:26:38.805368+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:11:13.178810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804855+0000","last_change":"2023-12-19T18:26:38.516578+0000","last_active":"2023-12-19T18:26:38.804855+0000","last_peered":"2023-12-19T18:26:38.804855+0000","last_clean":"2023-12-19T18:26:38.804855+0000","last_became_active":"2023-12-19T18:26:38.515850+0000","last_became_peered":"2023-12-19T18:26:38.515850+0000","last_unstale":"2023-12-19T18:26:38.804855+0000","last_undegraded":"2023-12-19T18:26:38.804855+0000","last_fullsized":"2023-12-19T18:26:38.804855+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:42.815725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"184'8","reported_seq":19,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.813394+0000","last_change":"2023-12-19T18:26:37.997903+0000","last_active":"2023-12-19T18:26:38.813394+0000","last_peered":"2023-12-19T18:26:38.813394+0000","last_clean":"2023-12-19T18:26:38.813394+0000","last_became_active":"2023-12-19T18:26:37.997359+0000","last_became_peered":"2023-12-19T18:26:37.997359+0000","last_unstale":"2023-12-19T18:26:38.813394+0000","last_undegraded":"2023-12-19T18:26:38.813394+0000","last_fullsized":"2023-12-19T18:26:38.813394+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:41.803983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.796503+0000","last_change":"2023-12-19T18:26:38.501505+0000","last_active":"2023-12-19T18:26:38.796503+0000","last_peered":"2023-12-19T18:26:38.796503+0000","last_clean":"2023-12-19T18:26:38.796503+0000","last_became_active":"2023-12-19T18:26:38.501087+0000","last_became_peered":"2023-12-19T18:26:38.501087+0000","last_unstale":"2023-12-19T18:26:38.796503+0000","last_undegraded":"2023-12-19T18:26:38.796503+0000","last_fullsized":"2023-12-19T18:26:38.796503+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:49:29.813985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"184'1","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804681+0000","last_change":"2023-12-19T18:26:37.996630+0000","last_active":"2023-12-19T18:26:38.804681+0000","last_peered":"2023-12-19T18:26:38.804681+0000","last_clean":"2023-12-19T18:26:38.804681+0000","last_became_active":"2023-12-19T18:26:37.996263+0000","last_became_peered":"2023-12-19T18:26:37.996263+0000","last_unstale":"2023-12-19T18:26:38.804681+0000","last_undegraded":"2023-12-19T18:26:38.804681+0000","last_fullsized":"2023-12-19T18:26:38.804681+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:15:07.042524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805115+0000","last_change":"2023-12-19T18:26:38.518923+0000","last_active":"2023-12-19T18:26:38.805115+0000","last_peered":"2023-12-19T18:26:38.805115+0000","last_clean":"2023-12-19T18:26:38.805115+0000","last_became_active":"2023-12-19T18:26:38.518655+0000","last_became_peered":"2023-12-19T18:26:38.518655+0000","last_unstale":"2023-12-19T18:26:38.805115+0000","last_undegraded":"2023-12-19T18:26:38.805115+0000","last_fullsized":"2023-12-19T18:26:38.805115+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:50.192924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"184'2","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.813624+0000","last_change":"2023-12-19T18:26:38.002740+0000","last_active":"2023-12-19T18:26:38.813624+0000","last_peered":"2023-12-19T18:26:38.813624+0000","last_clean":"2023-12-19T18:26:38.813624+0000","last_became_active":"2023-12-19T18:26:38.002404+0000","last_became_peered":"2023-12-19T18:26:38.002404+0000","last_unstale":"2023-12-19T18:26:38.813624+0000","last_undegraded":"2023-12-19T18:26:38.813624+0000","last_fullsized":"2023-12-19T18:26:38.813624+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:09:05.022659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.796409+0000","last_change":"2023-12-19T18:26:38.504549+0000","last_active":"2023-12-19T18:26:38.796409+0000","last_peered":"2023-12-19T18:26:38.796409+0000","last_clean":"2023-12-19T18:26:38.796409+0000","last_became_active":"2023-12-19T18:26:38.504290+0000","last_became_peered":"2023-12-19T18:26:38.504290+0000","last_unstale":"2023-12-19T18:26:38.796409+0000","last_undegraded":"2023-12-19T18:26:38.796409+0000","last_fullsized":"2023-12-19T18:26:38.796409+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:26:35.126926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"184'1","reported_seq":12,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.795740+0000","last_change":"2023-12-19T18:26:37.992200+0000","last_active":"2023-12-19T18:26:38.795740+0000","last_peered":"2023-12-19T18:26:38.795740+0000","last_clean":"2023-12-19T18:26:38.795740+0000","last_became_active":"2023-12-19T18:26:37.991863+0000","last_became_peered":"2023-12-19T18:26:37.991863+0000","last_unstale":"2023-12-19T18:26:38.795740+0000","last_undegraded":"2023-12-19T18:26:38.795740+0000","last_fullsized":"2023-12-19T18:26:38.795740+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:47:43.408526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.812696+0000","last_change":"2023-12-19T18:26:38.520808+0000","last_active":"2023-12-19T18:26:38.812696+0000","last_peered":"2023-12-19T18:26:38.812696+0000","last_clean":"2023-12-19T18:26:38.812696+0000","last_became_active":"2023-12-19T18:26:38.517583+0000","last_became_peered":"2023-12-19T18:26:38.517583+0000","last_unstale":"2023-12-19T18:26:38.812696+0000","last_undegraded":"2023-12-19T18:26:38.812696+0000","last_fullsized":"2023-12-19T18:26:38.812696+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:39:42.815725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805409+0000","last_change":"2023-12-19T18:26:38.000581+0000","last_active":"2023-12-19T18:26:38.805409+0000","last_peered":"2023-12-19T18:26:38.805409+0000","last_clean":"2023-12-19T18:26:38.805409+0000","last_became_active":"2023-12-19T18:26:38.000337+0000","last_became_peered":"2023-12-19T18:26:38.000337+0000","last_unstale":"2023-12-19T18:26:38.805409+0000","last_undegraded":"2023-12-19T18:26:38.805409+0000","last_fullsized":"2023-12-19T18:26:38.805409+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:56:03.771290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.798394+0000","last_change":"2023-12-19T18:26:38.512241+0000","last_active":"2023-12-19T18:26:38.798394+0000","last_peered":"2023-12-19T18:26:38.798394+0000","last_clean":"2023-12-19T18:26:38.798394+0000","last_became_active":"2023-12-19T18:26:38.512009+0000","last_became_peered":"2023-12-19T18:26:38.512009+0000","last_unstale":"2023-12-19T18:26:38.798394+0000","last_undegraded":"2023-12-19T18:26:38.798394+0000","last_fullsized":"2023-12-19T18:26:38.798394+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:47:41.584531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"184'2","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.812630+0000","last_change":"2023-12-19T18:26:38.002613+0000","last_active":"2023-12-19T18:26:38.812630+0000","last_peered":"2023-12-19T18:26:38.812630+0000","last_clean":"2023-12-19T18:26:38.812630+0000","last_became_active":"2023-12-19T18:26:38.002370+0000","last_became_peered":"2023-12-19T18:26:38.002370+0000","last_unstale":"2023-12-19T18:26:38.812630+0000","last_undegraded":"2023-12-19T18:26:38.812630+0000","last_fullsized":"2023-12-19T18:26:38.812630+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:16:39.938645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805189+0000","last_change":"2023-12-19T18:26:38.521621+0000","last_active":"2023-12-19T18:26:38.805189+0000","last_peered":"2023-12-19T18:26:38.805189+0000","last_clean":"2023-12-19T18:26:38.805189+0000","last_became_active":"2023-12-19T18:26:38.518851+0000","last_became_peered":"2023-12-19T18:26:38.518851+0000","last_unstale":"2023-12-19T18:26:38.805189+0000","last_undegraded":"2023-12-19T18:26:38.805189+0000","last_fullsized":"2023-12-19T18:26:38.805189+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:47:41.584531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804706+0000","last_change":"2023-12-19T18:26:37.979009+0000","last_active":"2023-12-19T18:26:38.804706+0000","last_peered":"2023-12-19T18:26:38.804706+0000","last_clean":"2023-12-19T18:26:38.804706+0000","last_became_active":"2023-12-19T18:26:37.978387+0000","last_became_peered":"2023-12-19T18:26:37.978387+0000","last_unstale":"2023-12-19T18:26:38.804706+0000","last_undegraded":"2023-12-19T18:26:38.804706+0000","last_fullsized":"2023-12-19T18:26:38.804706+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:27:55.676426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.798572+0000","last_change":"2023-12-19T18:26:38.511007+0000","last_active":"2023-12-19T18:26:38.798572+0000","last_peered":"2023-12-19T18:26:38.798572+0000","last_clean":"2023-12-19T18:26:38.798572+0000","last_became_active":"2023-12-19T18:26:38.510797+0000","last_became_peered":"2023-12-19T18:26:38.510797+0000","last_unstale":"2023-12-19T18:26:38.798572+0000","last_undegraded":"2023-12-19T18:26:38.798572+0000","last_fullsized":"2023-12-19T18:26:38.798572+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:29:31.201464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805018+0000","last_change":"2023-12-19T18:26:37.997350+0000","last_active":"2023-12-19T18:26:38.805018+0000","last_peered":"2023-12-19T18:26:38.805018+0000","last_clean":"2023-12-19T18:26:38.805018+0000","last_became_active":"2023-12-19T18:26:37.997075+0000","last_became_peered":"2023-12-19T18:26:37.997075+0000","last_unstale":"2023-12-19T18:26:38.805018+0000","last_undegraded":"2023-12-19T18:26:38.805018+0000","last_fullsized":"2023-12-19T18:26:38.805018+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:33:41.275730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805547+0000","last_change":"2023-12-19T18:26:38.522444+0000","last_active":"2023-12-19T18:26:38.805547+0000","last_peered":"2023-12-19T18:26:38.805547+0000","last_clean":"2023-12-19T18:26:38.805547+0000","last_became_active":"2023-12-19T18:26:38.521830+0000","last_became_peered":"2023-12-19T18:26:38.521830+0000","last_unstale":"2023-12-19T18:26:38.805547+0000","last_undegraded":"2023-12-19T18:26:38.805547+0000","last_fullsized":"2023-12-19T18:26:38.805547+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:57:04.171752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804968+0000","last_change":"2023-12-19T18:26:37.978842+0000","last_active":"2023-12-19T18:26:38.804968+0000","last_peered":"2023-12-19T18:26:38.804968+0000","last_clean":"2023-12-19T18:26:38.804968+0000","last_became_active":"2023-12-19T18:26:37.978269+0000","last_became_peered":"2023-12-19T18:26:37.978269+0000","last_unstale":"2023-12-19T18:26:38.804968+0000","last_undegraded":"2023-12-19T18:26:38.804968+0000","last_fullsized":"2023-12-19T18:26:38.804968+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:25:56.253877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804789+0000","last_change":"2023-12-19T18:26:38.507047+0000","last_active":"2023-12-19T18:26:38.804789+0000","last_peered":"2023-12-19T18:26:38.804789+0000","last_clean":"2023-12-19T18:26:38.804789+0000","last_became_active":"2023-12-19T18:26:38.506771+0000","last_became_peered":"2023-12-19T18:26:38.506771+0000","last_unstale":"2023-12-19T18:26:38.804789+0000","last_undegraded":"2023-12-19T18:26:38.804789+0000","last_fullsized":"2023-12-19T18:26:38.804789+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:06:35.754608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"184'2","reported_seq":13,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804900+0000","last_change":"2023-12-19T18:26:37.992843+0000","last_active":"2023-12-19T18:26:38.804900+0000","last_peered":"2023-12-19T18:26:38.804900+0000","last_clean":"2023-12-19T18:26:38.804900+0000","last_became_active":"2023-12-19T18:26:37.992138+0000","last_became_peered":"2023-12-19T18:26:37.992138+0000","last_unstale":"2023-12-19T18:26:38.804900+0000","last_undegraded":"2023-12-19T18:26:38.804900+0000","last_fullsized":"2023-12-19T18:26:38.804900+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:18:28.254508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804877+0000","last_change":"2023-12-19T18:26:38.506842+0000","last_active":"2023-12-19T18:26:38.804877+0000","last_peered":"2023-12-19T18:26:38.804877+0000","last_clean":"2023-12-19T18:26:38.804877+0000","last_became_active":"2023-12-19T18:26:38.506603+0000","last_became_peered":"2023-12-19T18:26:38.506603+0000","last_unstale":"2023-12-19T18:26:38.804877+0000","last_undegraded":"2023-12-19T18:26:38.804877+0000","last_fullsized":"2023-12-19T18:26:38.804877+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:14:38.491679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"185'1","reported_seq":12,"reported_epoch":185,"state":"active+clean","last_fresh":"2023-12-19T18:26:39.650357+0000","last_change":"2023-12-19T18:26:37.999764+0000","last_active":"2023-12-19T18:26:39.650357+0000","last_peered":"2023-12-19T18:26:39.650357+0000","last_clean":"2023-12-19T18:26:39.650357+0000","last_became_active":"2023-12-19T18:26:37.999519+0000","last_became_peered":"2023-12-19T18:26:37.999519+0000","last_unstale":"2023-12-19T18:26:39.650357+0000","last_undegraded":"2023-12-19T18:26:39.650357+0000","last_fullsized":"2023-12-19T18:26:39.650357+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:28:15.829119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.813582+0000","last_change":"2023-12-19T18:26:38.523911+0000","last_active":"2023-12-19T18:26:38.813582+0000","last_peered":"2023-12-19T18:26:38.813582+0000","last_clean":"2023-12-19T18:26:38.813582+0000","last_became_active":"2023-12-19T18:26:38.523133+0000","last_became_peered":"2023-12-19T18:26:38.523133+0000","last_unstale":"2023-12-19T18:26:38.813582+0000","last_undegraded":"2023-12-19T18:26:38.813582+0000","last_fullsized":"2023-12-19T18:26:38.813582+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:48.610885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.795581+0000","last_change":"2023-12-19T18:26:37.996734+0000","last_active":"2023-12-19T18:26:38.795581+0000","last_peered":"2023-12-19T18:26:38.795581+0000","last_clean":"2023-12-19T18:26:38.795581+0000","last_became_active":"2023-12-19T18:26:37.996274+0000","last_became_peered":"2023-12-19T18:26:37.996274+0000","last_unstale":"2023-12-19T18:26:38.795581+0000","last_undegraded":"2023-12-19T18:26:38.795581+0000","last_fullsized":"2023-12-19T18:26:38.795581+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:24:29.495951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.813319+0000","last_change":"2023-12-19T18:26:38.522476+0000","last_active":"2023-12-19T18:26:38.813319+0000","last_peered":"2023-12-19T18:26:38.813319+0000","last_clean":"2023-12-19T18:26:38.813319+0000","last_became_active":"2023-12-19T18:26:38.515096+0000","last_became_peered":"2023-12-19T18:26:38.515096+0000","last_unstale":"2023-12-19T18:26:38.813319+0000","last_undegraded":"2023-12-19T18:26:38.813319+0000","last_fullsized":"2023-12-19T18:26:38.813319+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:16:39.937501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.795765+0000","last_change":"2023-12-19T18:26:37.997778+0000","last_active":"2023-12-19T18:26:38.795765+0000","last_peered":"2023-12-19T18:26:38.795765+0000","last_clean":"2023-12-19T18:26:38.795765+0000","last_became_active":"2023-12-19T18:26:37.997532+0000","last_became_peered":"2023-12-19T18:26:37.997532+0000","last_unstale":"2023-12-19T18:26:38.795765+0000","last_undegraded":"2023-12-19T18:26:38.795765+0000","last_fullsized":"2023-12-19T18:26:38.795765+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:27:49.181182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.795697+0000","last_change":"2023-12-19T18:26:38.506763+0000","last_active":"2023-12-19T18:26:38.795697+0000","last_peered":"2023-12-19T18:26:38.795697+0000","last_clean":"2023-12-19T18:26:38.795697+0000","last_became_active":"2023-12-19T18:26:38.506090+0000","last_became_peered":"2023-12-19T18:26:38.506090+0000","last_unstale":"2023-12-19T18:26:38.795697+0000","last_undegraded":"2023-12-19T18:26:38.795697+0000","last_fullsized":"2023-12-19T18:26:38.795697+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:07:27.156222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.805479+0000","last_change":"2023-12-19T18:26:38.006710+0000","last_active":"2023-12-19T18:26:38.805479+0000","last_peered":"2023-12-19T18:26:38.805479+0000","last_clean":"2023-12-19T18:26:38.805479+0000","last_became_active":"2023-12-19T18:26:38.006471+0000","last_became_peered":"2023-12-19T18:26:38.006471+0000","last_unstale":"2023-12-19T18:26:38.805479+0000","last_undegraded":"2023-12-19T18:26:38.805479+0000","last_fullsized":"2023-12-19T18:26:38.805479+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:57:14.752319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804946+0000","last_change":"2023-12-19T18:26:38.507476+0000","last_active":"2023-12-19T18:26:38.804946+0000","last_peered":"2023-12-19T18:26:38.804946+0000","last_clean":"2023-12-19T18:26:38.804946+0000","last_became_active":"2023-12-19T18:26:38.507237+0000","last_became_peered":"2023-12-19T18:26:38.507237+0000","last_unstale":"2023-12-19T18:26:38.804946+0000","last_undegraded":"2023-12-19T18:26:38.804946+0000","last_fullsized":"2023-12-19T18:26:38.804946+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:44:27.103692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.811713+0000","last_change":"2023-12-19T18:26:37.999535+0000","last_active":"2023-12-19T18:26:38.811713+0000","last_peered":"2023-12-19T18:26:38.811713+0000","last_clean":"2023-12-19T18:26:38.811713+0000","last_became_active":"2023-12-19T18:26:37.999290+0000","last_became_peered":"2023-12-19T18:26:37.999290+0000","last_unstale":"2023-12-19T18:26:38.811713+0000","last_undegraded":"2023-12-19T18:26:38.811713+0000","last_fullsized":"2023-12-19T18:26:38.811713+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:03:38.154234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.804655+0000","last_change":"2023-12-19T18:26:38.509297+0000","last_active":"2023-12-19T18:26:38.804655+0000","last_peered":"2023-12-19T18:26:38.804655+0000","last_clean":"2023-12-19T18:26:38.804655+0000","last_became_active":"2023-12-19T18:26:38.508537+0000","last_became_peered":"2023-12-19T18:26:38.508537+0000","last_unstale":"2023-12-19T18:26:38.804655+0000","last_undegraded":"2023-12-19T18:26:38.804655+0000","last_fullsized":"2023-12-19T18:26:38.804655+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:26:08.400076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.798472+0000","last_change":"2023-12-19T18:26:38.003094+0000","last_active":"2023-12-19T18:26:38.798472+0000","last_peered":"2023-12-19T18:26:38.798472+0000","last_clean":"2023-12-19T18:26:38.798472+0000","last_became_active":"2023-12-19T18:26:38.001985+0000","last_became_peered":"2023-12-19T18:26:38.001985+0000","last_unstale":"2023-12-19T18:26:38.798472+0000","last_undegraded":"2023-12-19T18:26:38.798472+0000","last_fullsized":"2023-12-19T18:26:38.798472+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:27:55.676426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.811527+0000","last_change":"2023-12-19T18:26:38.502269+0000","last_active":"2023-12-19T18:26:38.811527+0000","last_peered":"2023-12-19T18:26:38.811527+0000","last_clean":"2023-12-19T18:26:38.811527+0000","last_became_active":"2023-12-19T18:26:38.502021+0000","last_became_peered":"2023-12-19T18:26:38.502021+0000","last_unstale":"2023-12-19T18:26:38.811527+0000","last_undegraded":"2023-12-19T18:26:38.811527+0000","last_fullsized":"2023-12-19T18:26:38.811527+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:36:51.412075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":11,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.812606+0000","last_change":"2023-12-19T18:26:38.001917+0000","last_active":"2023-12-19T18:26:38.812606+0000","last_peered":"2023-12-19T18:26:38.812606+0000","last_clean":"2023-12-19T18:26:38.812606+0000","last_became_active":"2023-12-19T18:26:38.001257+0000","last_became_peered":"2023-12-19T18:26:38.001257+0000","last_unstale":"2023-12-19T18:26:38.812606+0000","last_undegraded":"2023-12-19T18:26:38.812606+0000","last_fullsized":"2023-12-19T18:26:38.812606+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:06:41.408352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.795604+0000","last_change":"2023-12-19T18:26:38.495348+0000","last_active":"2023-12-19T18:26:38.795604+0000","last_peered":"2023-12-19T18:26:38.795604+0000","last_clean":"2023-12-19T18:26:38.795604+0000","last_became_active":"2023-12-19T18:26:38.494366+0000","last_became_peered":"2023-12-19T18:26:38.494366+0000","last_unstale":"2023-12-19T18:26:38.795604+0000","last_undegraded":"2023-12-19T18:26:38.795604+0000","last_fullsized":"2023-12-19T18:26:38.795604+0000","mapping_epoch":183,"log_start":"0'0","ondisk_log_start":"0'0","created":183,"last_epoch_clean":184,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:37.962663+0000","last_clean_scrub_stamp":"2023-12-19T18:26:37.962663+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:44:27.103692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"184'3","reported_seq":14,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.798449+0000","last_change":"2023-12-19T18:26:38.002223+0000","last_active":"2023-12-19T18:26:38.798449+0000","last_peered":"2023-12-19T18:26:38.798449+0000","last_clean":"2023-12-19T18:26:38.798449+0000","last_became_active":"2023-12-19T18:26:38.001643+0000","last_became_peered":"2023-12-19T18:26:38.001643+0000","last_unstale":"2023-12-19T18:26:38.798449+0000","last_undegraded":"2023-12-19T18:26:38.798449+0000","last_fullsized":"2023-12-19T18:26:38.798449+0000","mapping_epoch":182,"log_start":"0'0","ondisk_log_start":"0'0","created":182,"last_epoch_clean":183,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:26:36.950921+0000","last_clean_scrub_stamp":"2023-12-19T18:26:36.950921+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T18:54:18.539917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":313,"reported_epoch":184,"state":"active+clean","last_fresh":"2023-12-19T18:26:38.496222+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:26:38.496222+0000","last_peered":"2023-12-19T18:26:38.496222+0000","last_clean":"2023-12-19T18:26:38.496222+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:26:38.496222+0000","last_undegraded":"2023-12-19T18:26:38.496222+0000","last_fullsized":"2023-12-19T18:26:38.496222+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574994,"num_pgs":22,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1021,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194291,"statfs":{"total":200000000,"available":198954293,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.179,"15min":1.169},"min":{"1min":0.748,"5min":0.57899999999999996,"15min":0.45900000000000002},"max":{"1min":1.597,"5min":1.9630000000000001,"15min":5.6879999999999997},"last":1.194},{"interface":"front","average":{"1min":1.25,"5min":1.2310000000000001,"15min":1.2110000000000001},"min":{"1min":1.05,"5min":0.71899999999999997,"15min":0.55800000000000005},"max":{"1min":1.526,"5min":1.7030000000000001,"15min":1.79},"last":1.1459999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.155,"15min":1.228},"min":{"1min":0.84299999999999997,"5min":0.65800000000000003,"15min":0.54400000000000004},"max":{"1min":1.619,"5min":1.7010000000000001,"15min":10.926},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.294,"5min":1.2769999999999999,"15min":1.264},"min":{"1min":1.018,"5min":0.80700000000000005,"15min":0.67900000000000005},"max":{"1min":1.5980000000000001,"5min":1.736,"15min":10.904999999999999},"last":0.95999999999999996}]},{"osd":2,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.0880000000000001,"15min":1.109},"min":{"1min":0.90700000000000003,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":3.1960000000000002},"last":1.0249999999999999},{"interface":"front","average":{"1min":1.198,"5min":1.1779999999999999,"15min":1.196},"min":{"1min":0.93400000000000005,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.4810000000000001,"5min":2.0670000000000002,"15min":6.008},"last":1.0760000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.292,"5min":1.25,"15min":1.2030000000000001},"min":{"1min":0.83499999999999996,"5min":0.66900000000000004,"15min":0.61399999999999999},"max":{"1min":1.696,"5min":1.732,"15min":3.0379999999999998},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3540000000000001,"15min":1.3260000000000001},"min":{"1min":1.2070000000000001,"5min":0.77500000000000002,"15min":0.69599999999999995},"max":{"1min":1.5529999999999999,"5min":2.157,"15min":5.8120000000000003},"last":1.4059999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.171,"5min":1.2150000000000001,"15min":1.1639999999999999},"min":{"1min":0.88200000000000001,"5min":0.55700000000000005,"15min":0.47899999999999998},"max":{"1min":1.569,"5min":1.847,"15min":4.0110000000000001},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.246,"15min":1.254},"min":{"1min":0.88200000000000001,"5min":0.59299999999999997,"15min":0.57599999999999996},"max":{"1min":1.8,"5min":1.8,"15min":3.3180000000000001},"last":1.3169999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.2809999999999999,"15min":1.331},"min":{"1min":0.94599999999999995,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.657,"5min":1.956,"15min":7.6429999999999998},"last":1.4830000000000001},{"interface":"front","average":{"1min":1.296,"5min":1.3580000000000001,"15min":1.355},"min":{"1min":0.96299999999999997,"5min":0.627,"15min":0.53500000000000003},"max":{"1min":1.6419999999999999,"5min":2.1179999999999999,"15min":6.0839999999999996},"last":1.359}]},{"osd":6,"last update":"Tue Dec 19 18:26:11 2023","interfaces":[{"interface":"back","average":{"1min":1.482,"5min":1.512,"15min":1.4910000000000001},"min":{"1min":1.1459999999999999,"5min":0.628,"15min":0.59899999999999998},"max":{"1min":1.716,"5min":2.423,"15min":3.1309999999999998},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.4890000000000001,"5min":1.4379999999999999,"15min":1.413},"min":{"1min":1.1220000000000001,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.839,"5min":2.0179999999999998,"15min":4.1479999999999997},"last":1.45}]}]},{"osd":6,"up_from":13,"seq":55834574994,"num_pgs":19,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1021,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194291,"statfs":{"total":200000000,"available":198954293,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.119,"15min":1.0489999999999999},"min":{"1min":0.89100000000000001,"5min":0.69499999999999995,"15min":0.65800000000000003},"max":{"1min":1.502,"5min":1.6259999999999999,"15min":1.8129999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.1359999999999999,"15min":1.107},"min":{"1min":0.93200000000000005,"5min":0.64800000000000002,"15min":0.55600000000000005},"max":{"1min":1.548,"5min":1.663,"15min":3.5529999999999999},"last":0.996}]},{"osd":1,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.115,"15min":1.123},"min":{"1min":0.85199999999999998,"5min":0.68600000000000005,"15min":0.68200000000000005},"max":{"1min":1.4710000000000001,"5min":1.5609999999999999,"15min":4.5540000000000003},"last":1.181},{"interface":"front","average":{"1min":1.276,"5min":1.165,"15min":1.171},"min":{"1min":0.874,"5min":0.64600000000000002,"15min":0.60299999999999998},"max":{"1min":1.5660000000000001,"5min":1.627,"15min":1.885},"last":1.3460000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.21,"15min":1.1830000000000001},"min":{"1min":0.92300000000000004,"5min":0.78900000000000003,"15min":0.626},"max":{"1min":1.508,"5min":1.583,"15min":1.821},"last":1.2290000000000001},{"interface":"front","average":{"1min":1.23,"5min":1.1910000000000001,"15min":1.218},"min":{"1min":0.89800000000000002,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.7669999999999999,"5min":1.821,"15min":3.7909999999999999},"last":1.526}]},{"osd":3,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.28,"15min":1.236},"min":{"1min":1.0409999999999999,"5min":0.67800000000000005,"15min":0.67800000000000005},"max":{"1min":1.5760000000000001,"5min":1.722,"15min":3.8820000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.325,"5min":1.224,"15min":1.1919999999999999},"min":{"1min":0.97099999999999997,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.619,"5min":1.6559999999999999,"15min":4.6189999999999998},"last":1.3009999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.3080000000000001,"15min":1.27},"min":{"1min":0.88400000000000001,"5min":0.74099999999999999,"15min":0.65600000000000003},"max":{"1min":1.627,"5min":1.6899999999999999,"15min":3.0680000000000001},"last":1.415},{"interface":"front","average":{"1min":1.3740000000000001,"5min":1.413,"15min":1.3799999999999999},"min":{"1min":0.79900000000000004,"5min":0.76700000000000002,"15min":0.57999999999999996},"max":{"1min":1.758,"5min":1.774,"15min":3.7000000000000002},"last":1.123}]},{"osd":5,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.2629999999999999,"15min":1.226},"min":{"1min":0.85699999999999998,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.746,"5min":1.746,"15min":4.6680000000000001},"last":1.431},{"interface":"front","average":{"1min":1.401,"5min":1.431,"15min":1.3799999999999999},"min":{"1min":1.0960000000000001,"5min":0.84799999999999998,"15min":0.626},"max":{"1min":1.6479999999999999,"5min":4.1159999999999997,"15min":4.1159999999999997},"last":1.47}]},{"osd":7,"last update":"Tue Dec 19 18:26:00 2023","interfaces":[{"interface":"back","average":{"1min":1.423,"5min":1.423,"15min":1.3839999999999999},"min":{"1min":0.90300000000000002,"5min":0.72299999999999998,"15min":0.61899999999999999},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":3.9670000000000001},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.4199999999999999,"5min":1.4239999999999999,"15min":1.391},"min":{"1min":1.129,"5min":0.81899999999999995,"15min":0.79300000000000004},"max":{"1min":1.7290000000000001,"5min":1.8169999999999999,"15min":5.6639999999999997},"last":1.488}]}]},{"osd":5,"up_from":13,"seq":55834574994,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1599,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193713,"statfs":{"total":200000000,"available":198362167,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.371,"5min":1.2070000000000001,"15min":1.1930000000000001},"min":{"1min":0.80400000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":1.6279999999999999,"5min":1.8540000000000001,"15min":1.863},"last":1.482},{"interface":"front","average":{"1min":1.371,"5min":1.2729999999999999,"15min":1.226},"min":{"1min":1.214,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.5660000000000001,"5min":1.913,"15min":1.913},"last":1.4330000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1459999999999999,"5min":1.163,"15min":1.1100000000000001},"min":{"1min":0.92500000000000004,"5min":0.71599999999999997,"15min":0.65700000000000003},"max":{"1min":1.5920000000000001,"5min":1.7929999999999999,"15min":1.9490000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":1.1970000000000001,"5min":1.1759999999999999,"15min":1.111},"min":{"1min":0.91900000000000004,"5min":0.505,"15min":0.505},"max":{"1min":1.696,"5min":1.8300000000000001,"15min":1.843},"last":1.5349999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.175,"15min":1.127},"min":{"1min":1.026,"5min":0.72199999999999998,"15min":0.59499999999999997},"max":{"1min":1.621,"5min":1.621,"15min":1.794},"last":1.276},{"interface":"front","average":{"1min":1.2789999999999999,"5min":1.2809999999999999,"15min":1.2},"min":{"1min":0.996,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.677,"5min":1.8,"15min":2.036},"last":1.1399999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.23,"5min":1.2030000000000001,"15min":1.141},"min":{"1min":0.94599999999999995,"5min":0.78600000000000003,"15min":0.68899999999999995},"max":{"1min":1.5429999999999999,"5min":1.72,"15min":1.7450000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.3480000000000001,"15min":1.284},"min":{"1min":1.137,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.694,"5min":1.9590000000000001,"15min":2.0590000000000002},"last":1.3080000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.3160000000000001,"15min":1.2370000000000001},"min":{"1min":0.77900000000000003,"5min":0.626,"15min":0.46700000000000003},"max":{"1min":1.659,"5min":1.944,"15min":2.0329999999999999},"last":0.97799999999999998},{"interface":"front","average":{"1min":1.105,"5min":1.1850000000000001,"15min":1.1519999999999999},"min":{"1min":0.88700000000000001,"5min":0.66000000000000003,"15min":0.57099999999999995},"max":{"1min":1.609,"5min":1.7689999999999999,"15min":1.901},"last":0.81200000000000006}]},{"osd":6,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.425,"5min":1.46,"15min":1.3899999999999999},"min":{"1min":1.2210000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.631,"5min":2.0329999999999999,"15min":2.1589999999999998},"last":1.25},{"interface":"front","average":{"1min":1.248,"5min":1.3340000000000001,"15min":1.28},"min":{"1min":1.0069999999999999,"5min":0.83199999999999996,"15min":0.49299999999999999},"max":{"1min":1.704,"5min":1.704,"15min":1.976},"last":1.206}]},{"osd":7,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.3049999999999999,"15min":1.2589999999999999},"min":{"1min":0.751,"5min":0.54700000000000004,"15min":0.50900000000000001},"max":{"1min":1.6399999999999999,"5min":1.863,"15min":2.0270000000000001},"last":1.171},{"interface":"front","average":{"1min":1.419,"5min":1.466,"15min":1.4350000000000001},"min":{"1min":1.252,"5min":0.93300000000000005,"15min":0.623},"max":{"1min":1.661,"5min":1.8620000000000001,"15min":2.117},"last":1.389}]}]},{"osd":4,"up_from":13,"seq":55834574994,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1023,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194289,"statfs":{"total":200000000,"available":198952125,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.054,"15min":1.016},"min":{"1min":0.71299999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.468,"5min":1.972,"15min":1.972},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.0669999999999999,"5min":1.095,"15min":1.0549999999999999},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.3859999999999999,"5min":1.7330000000000001,"15min":1.7909999999999999},"last":1.1100000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1910000000000001,"5min":1.1619999999999999,"15min":1.1579999999999999},"min":{"1min":0.871,"5min":0.67200000000000004,"15min":0.66900000000000004},"max":{"1min":1.6000000000000001,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.252,"15min":1.2130000000000001},"min":{"1min":0.95199999999999996,"5min":0.64700000000000002,"15min":0.61699999999999999},"max":{"1min":1.5600000000000001,"5min":1.875,"15min":1.875},"last":1.5289999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.101,"5min":1.125,"15min":1.125},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.536,"5min":1.6830000000000001,"15min":1.7010000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.2150000000000001,"5min":1.165,"15min":1.1299999999999999},"min":{"1min":0.81599999999999995,"5min":0.747,"15min":0.70499999999999996},"max":{"1min":1.456,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.0529999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.272,"5min":1.266,"15min":1.2629999999999999},"min":{"1min":0.90900000000000003,"5min":0.754,"15min":0.754},"max":{"1min":1.5940000000000001,"5min":1.946,"15min":1.946},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.357,"15min":1.323},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.591,"5min":1.927,"15min":1.927},"last":1.4079999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.264,"5min":1.2509999999999999,"15min":1.2310000000000001},"min":{"1min":0.83699999999999997,"5min":0.82299999999999995,"15min":0.44500000000000001},"max":{"1min":1.681,"5min":1.8129999999999999,"15min":1.8740000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.24,"5min":1.343,"15min":1.357},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.504},"max":{"1min":1.7509999999999999,"5min":1.867,"15min":1.867},"last":1.198}]},{"osd":6,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":1.28,"15min":1.298},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.432},"max":{"1min":1.651,"5min":1.921,"15min":1.921},"last":1.268},{"interface":"front","average":{"1min":1.397,"5min":1.413,"15min":1.4079999999999999},"min":{"1min":0.93700000000000006,"5min":0.90800000000000003,"15min":0.64500000000000002},"max":{"1min":1.744,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.1639999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:26:07 2023","interfaces":[{"interface":"back","average":{"1min":1.2849999999999999,"5min":1.3380000000000001,"15min":1.3460000000000001},"min":{"1min":0.77300000000000002,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":1.6739999999999999,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.4650000000000001},{"interface":"front","average":{"1min":1.2989999999999999,"5min":1.413,"15min":1.411},"min":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.57399999999999995},"max":{"1min":1.611,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.218}]}]},{"osd":3,"up_from":13,"seq":55834574995,"num_pgs":27,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1021,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194290,"statfs":{"total":200000000,"available":198953677,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.89400000000000002,"15min":0.93500000000000005},"min":{"1min":0.69399999999999995,"5min":0.435,"15min":0.35299999999999998},"max":{"1min":1.4299999999999999,"5min":2.516,"15min":3.4350000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.0369999999999999,"15min":0.97599999999999998},"min":{"1min":0.69799999999999995,"5min":0.42999999999999999,"15min":0.39500000000000002},"max":{"1min":1.4219999999999999,"5min":4.3490000000000002,"15min":4.3490000000000002},"last":1.6100000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":1.0469999999999999,"15min":1.04},"min":{"1min":0.76600000000000001,"5min":0.45300000000000001,"15min":0.41299999999999998},"max":{"1min":1.3109999999999999,"5min":5.2510000000000003,"15min":5.2510000000000003},"last":0.93200000000000005},{"interface":"front","average":{"1min":1.1879999999999999,"5min":1.1439999999999999,"15min":1.109},"min":{"1min":0.93100000000000005,"5min":0.53200000000000003,"15min":0.47799999999999998},"max":{"1min":1.643,"5min":5.4119999999999999,"15min":5.4119999999999999},"last":1.25}]},{"osd":2,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.204,"15min":1.2470000000000001},"min":{"1min":0.96699999999999997,"5min":0.502,"15min":0.46000000000000002},"max":{"1min":1.7,"5min":2.4649999999999999,"15min":13.833},"last":1.6819999999999999},{"interface":"front","average":{"1min":1.365,"5min":1.3580000000000001,"15min":1.3360000000000001},"min":{"1min":1.105,"5min":0.60299999999999998,"15min":0.53200000000000003},"max":{"1min":1.798,"5min":5.4480000000000004,"15min":16.748999999999999},"last":1.4970000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.343,"15min":1.272},"min":{"1min":0.99299999999999999,"5min":0.77900000000000003,"15min":0.65600000000000003},"max":{"1min":1.7250000000000001,"5min":5.1790000000000003,"15min":5.1790000000000003},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.4319999999999999,"5min":1.4510000000000001,"15min":1.409},"min":{"1min":1.117,"5min":0.873,"15min":0.73199999999999998},"max":{"1min":2.0819999999999999,"5min":4.569,"15min":4.569},"last":1.1839999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4690000000000001,"5min":1.4359999999999999,"15min":1.4350000000000001},"min":{"1min":1.2390000000000001,"5min":0.80700000000000005,"15min":0.76000000000000001},"max":{"1min":1.7909999999999999,"5min":5.3289999999999997,"15min":5.3289999999999997},"last":1.768},{"interface":"front","average":{"1min":1.369,"5min":1.3959999999999999,"15min":1.3180000000000001},"min":{"1min":1.0840000000000001,"5min":0.81899999999999995,"15min":0.69999999999999996},"max":{"1min":1.8520000000000001,"5min":4.6369999999999996,"15min":4.6369999999999996},"last":1.724}]},{"osd":6,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.4790000000000001,"5min":1.478,"15min":1.464},"min":{"1min":1.2310000000000001,"5min":0.81999999999999995,"15min":0.81599999999999995},"max":{"1min":1.9410000000000001,"5min":4.5039999999999996,"15min":4.5039999999999996},"last":1.798},{"interface":"front","average":{"1min":1.538,"5min":1.53,"15min":1.5089999999999999},"min":{"1min":1.2190000000000001,"5min":0.83999999999999997,"15min":0.80500000000000005},"max":{"1min":1.9419999999999999,"5min":5.3739999999999997,"15min":5.3739999999999997},"last":1.659}]},{"osd":7,"last update":"Tue Dec 19 18:25:58 2023","interfaces":[{"interface":"back","average":{"1min":1.5249999999999999,"5min":1.5,"15min":1.4259999999999999},"min":{"1min":1.2090000000000001,"5min":0.78500000000000003,"15min":0.73199999999999998},"max":{"1min":1.8799999999999999,"5min":5.1349999999999998,"15min":5.1349999999999998},"last":1.391},{"interface":"front","average":{"1min":1.5820000000000001,"5min":1.601,"15min":1.5589999999999999},"min":{"1min":1.3700000000000001,"5min":0.92200000000000004,"15min":0.85399999999999998},"max":{"1min":2.02,"5min":4.4619999999999997,"15min":4.4619999999999997},"last":1.552}]}]},{"osd":2,"up_from":13,"seq":55834574994,"num_pgs":31,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1021,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194291,"statfs":{"total":200000000,"available":198954271,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.90400000000000003,"15min":0.98599999999999999},"min":{"1min":0.68100000000000005,"5min":0.42399999999999999,"15min":0.34399999999999997},"max":{"1min":1.2589999999999999,"5min":1.506,"15min":9.2550000000000008},"last":0.79500000000000004},{"interface":"front","average":{"1min":1.081,"5min":1.0129999999999999,"15min":1.0309999999999999},"min":{"1min":0.73199999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.5620000000000001,"5min":1.7869999999999999,"15min":4.7850000000000001},"last":0.95599999999999996}]},{"osd":1,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.026,"5min":0.96699999999999997,"15min":1.0720000000000001},"min":{"1min":0.70099999999999996,"5min":0.47599999999999998,"15min":0.46000000000000002},"max":{"1min":1.9159999999999999,"5min":1.9159999999999999,"15min":8.2720000000000002},"last":0.72099999999999997},{"interface":"front","average":{"1min":1.194,"5min":1.1240000000000001,"15min":1.1399999999999999},"min":{"1min":0.86799999999999999,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.5609999999999999,"5min":1.8460000000000001,"15min":5.0620000000000003},"last":0.82499999999999996}]},{"osd":3,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.363,"5min":1.2150000000000001,"15min":1.288},"min":{"1min":1.026,"5min":0.63300000000000001,"15min":0.51900000000000002},"max":{"1min":1.8200000000000001,"5min":2.8159999999999998,"15min":8.3109999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.095,"5min":1.103,"15min":1.234},"min":{"1min":0.88100000000000001,"5min":0.60299999999999998,"15min":0.41799999999999998},"max":{"1min":1.4610000000000001,"5min":2.6240000000000001,"15min":8.0690000000000008},"last":1.0589999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.444,"5min":1.3640000000000001,"15min":1.425},"min":{"1min":1.1990000000000001,"5min":0.81000000000000005,"15min":0.79300000000000004},"max":{"1min":1.7869999999999999,"5min":2.2160000000000002,"15min":5.3929999999999998},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.548,"5min":1.4650000000000001,"15min":1.504},"min":{"1min":1.052,"5min":0.82099999999999995,"15min":0.78300000000000003},"max":{"1min":1.8460000000000001,"5min":2.1949999999999998,"15min":7.3570000000000002},"last":1.3080000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4239999999999999,"5min":1.3839999999999999,"15min":1.47},"min":{"1min":1.133,"5min":0.91800000000000004,"15min":0.76200000000000001},"max":{"1min":1.802,"5min":2.0299999999999998,"15min":8.1839999999999993},"last":1.2749999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.369,"15min":1.401},"min":{"1min":1.095,"5min":0.82299999999999995,"15min":0.72199999999999998},"max":{"1min":1.702,"5min":1.9930000000000001,"15min":5.165},"last":1.319}]},{"osd":6,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.4279999999999999,"15min":1.4390000000000001},"min":{"1min":1.147,"5min":0.82399999999999995,"15min":0.72999999999999998},"max":{"1min":1.8680000000000001,"5min":2.5139999999999998,"15min":5.1059999999999999},"last":1.109},{"interface":"front","average":{"1min":1.4790000000000001,"5min":1.4830000000000001,"15min":1.522},"min":{"1min":1.198,"5min":0.98499999999999999,"15min":0.77000000000000002},"max":{"1min":1.788,"5min":2.7559999999999998,"15min":8.1370000000000005},"last":1.3700000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.546,"5min":1.498,"15min":1.53},"min":{"1min":1.319,"5min":0.93500000000000005,"15min":0.78200000000000003},"max":{"1min":1.8029999999999999,"5min":2.3700000000000001,"15min":5.1449999999999996},"last":1.252},{"interface":"front","average":{"1min":1.613,"5min":1.599,"15min":1.6140000000000001},"min":{"1min":1.4159999999999999,"5min":0.88500000000000001,"15min":0.86499999999999999},"max":{"1min":1.907,"5min":4.8289999999999997,"15min":8.2300000000000004},"last":1.1699999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574995,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1597,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193714,"statfs":{"total":200000000,"available":198363855,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.018,"5min":0.89800000000000002,"15min":0.86499999999999999},"min":{"1min":0.73699999999999999,"5min":0.41199999999999998,"15min":0.34699999999999998},"max":{"1min":2.262,"5min":2.262,"15min":2.262},"last":0.65300000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":0.97899999999999998,"15min":0.96499999999999997},"min":{"1min":0.72099999999999997,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":2.5939999999999999,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.67000000000000004}]},{"osd":2,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.0660000000000001,"5min":1.004,"15min":1.0389999999999999},"min":{"1min":0.84299999999999997,"5min":0.52500000000000002,"15min":0.41399999999999998},"max":{"1min":1.452,"5min":2.4590000000000001,"15min":2.5739999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.1140000000000001,"5min":0.997,"15min":0.96499999999999997},"min":{"1min":0.82799999999999996,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":1.5860000000000001,"5min":1.718,"15min":1.845},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.5369999999999999,"5min":1.2989999999999999,"15min":1.2270000000000001},"min":{"1min":1.3300000000000001,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.8959999999999999,"5min":2.1040000000000001,"15min":2.137},"last":0.53000000000000003},{"interface":"front","average":{"1min":1.355,"5min":1.1950000000000001,"15min":1.2},"min":{"1min":0.91700000000000004,"5min":0.50800000000000001,"15min":0.44900000000000001},"max":{"1min":1.8260000000000001,"5min":2.1320000000000001,"15min":2.3959999999999999},"last":0.79500000000000004}]},{"osd":4,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.304,"15min":1.3029999999999999},"min":{"1min":1.012,"5min":0.76400000000000001,"15min":0.69099999999999995},"max":{"1min":1.71,"5min":2.1440000000000001,"15min":2.444},"last":1.022},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.3640000000000001,"15min":1.3340000000000001},"min":{"1min":1.117,"5min":0.92500000000000004,"15min":0.71099999999999997},"max":{"1min":1.7789999999999999,"5min":1.8660000000000001,"15min":2.6440000000000001},"last":0.92800000000000005}]},{"osd":5,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.371,"15min":1.3420000000000001},"min":{"1min":1.081,"5min":0.76000000000000001,"15min":0.76000000000000001},"max":{"1min":1.909,"5min":1.998,"15min":2.5459999999999998},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4119999999999999,"15min":1.425},"min":{"1min":1.218,"5min":0.85899999999999999,"15min":0.70499999999999996},"max":{"1min":1.821,"5min":2.371,"15min":2.6179999999999999},"last":0.96899999999999997}]},{"osd":6,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.401,"15min":1.403},"min":{"1min":1.179,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.274,"5min":2.3610000000000002,"15min":2.7000000000000002},"last":1.1879999999999999},{"interface":"front","average":{"1min":1.5660000000000001,"5min":1.456,"15min":1.468},"min":{"1min":1.2769999999999999,"5min":0.98499999999999999,"15min":0.76400000000000001},"max":{"1min":1.8720000000000001,"5min":2.2719999999999998,"15min":2.895},"last":0.88100000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:26:02 2023","interfaces":[{"interface":"back","average":{"1min":1.569,"5min":1.413,"15min":1.427},"min":{"1min":1.3069999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.956,"5min":2.1629999999999998,"15min":9.4830000000000005},"last":0.83199999999999996},{"interface":"front","average":{"1min":1.6180000000000001,"5min":1.5069999999999999,"15min":1.53},"min":{"1min":1.331,"5min":0.874,"15min":0.71899999999999997},"max":{"1min":2.3420000000000001,"5min":2.3420000000000001,"15min":9.4109999999999996},"last":1.1319999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574994,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1021,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194291,"statfs":{"total":200000000,"available":198954203,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0469999999999999,"5min":0.89800000000000002,"15min":0.91300000000000003},"min":{"1min":0.746,"5min":0.311,"15min":0.311},"max":{"1min":1.671,"5min":1.671,"15min":2.3199999999999998},"last":1.119},{"interface":"front","average":{"1min":1.218,"5min":1.0549999999999999,"15min":1.0580000000000001},"min":{"1min":0.91900000000000004,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":2.2170000000000001,"5min":2.2170000000000001,"15min":2.4180000000000001},"last":0.97399999999999998}]},{"osd":2,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1850000000000001,"5min":0.997,"15min":0.98299999999999998},"min":{"1min":0.81000000000000005,"5min":0.434,"15min":0.434},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.069},{"interface":"front","average":{"1min":1.1619999999999999,"5min":0.98999999999999999,"15min":0.97299999999999998},"min":{"1min":0.81499999999999995,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.3340000000000001,"5min":2.3340000000000001,"15min":2.7810000000000001},"last":0.92400000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.0389999999999999,"15min":1.026},"min":{"1min":0.78600000000000003,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.7509999999999999,"5min":1.8200000000000001,"15min":2.5190000000000001},"last":0.81899999999999995},{"interface":"front","average":{"1min":1.5900000000000001,"5min":1.2949999999999999,"15min":1.2649999999999999},"min":{"1min":1.1200000000000001,"5min":0.53000000000000003,"15min":0.47499999999999998},"max":{"1min":2.3620000000000001,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.875}]},{"osd":4,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.48,"5min":1.288,"15min":1.254},"min":{"1min":1.1819999999999999,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":2.1920000000000002,"5min":2.1920000000000002,"15min":2.5529999999999999},"last":1.2709999999999999},{"interface":"front","average":{"1min":1.3460000000000001,"5min":1.262,"15min":1.2829999999999999},"min":{"1min":1.0489999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.714,"5min":2.056,"15min":2.9529999999999998},"last":1.0900000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.319,"15min":1.3939999999999999},"min":{"1min":1.0700000000000001,"5min":0.68500000000000005,"15min":0.64500000000000002},"max":{"1min":1.867,"5min":1.97,"15min":16.298999999999999},"last":1.161},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.3320000000000001,"15min":1.3129999999999999},"min":{"1min":1.1990000000000001,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":2.2909999999999999,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.47}]},{"osd":6,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.6639999999999999,"5min":1.4530000000000001,"15min":1.431},"min":{"1min":1.2769999999999999,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":2.4169999999999998,"5min":2.4169999999999998,"15min":2.5910000000000002},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.647,"5min":1.425,"15min":1.4019999999999999},"min":{"1min":1.1419999999999999,"5min":0.745,"15min":0.745},"max":{"1min":2.1709999999999998,"5min":2.1709999999999998,"15min":2.46},"last":1.341}]},{"osd":7,"last update":"Tue Dec 19 18:26:10 2023","interfaces":[{"interface":"back","average":{"1min":1.579,"5min":1.4530000000000001,"15min":1.4379999999999999},"min":{"1min":1.1699999999999999,"5min":0.72099999999999997,"15min":0.70899999999999996},"max":{"1min":2.2549999999999999,"5min":2.2549999999999999,"15min":2.4969999999999999},"last":1.3959999999999999},{"interface":"front","average":{"1min":1.522,"5min":1.395,"15min":1.4199999999999999},"min":{"1min":1.165,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.913,"5min":2.113,"15min":3.1429999999999998},"last":1.3129999999999999}]}]}],"pool_statfs":[]}} 2023-12-19T18:26:44.062 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:26:44.062 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:26:44.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.343+0000 7fcdf4a8a640 1 Processor -- start 2023-12-19T18:26:44.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.343+0000 7fcdf4a8a640 1 -- start start 2023-12-19T18:26:44.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.345+0000 7fcdf4a8a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcdf00ccf80 0x7fcdf00c9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.345+0000 7fcdf4a8a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf006f360 con 0x7fcdf015d160 2023-12-19T18:26:44.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.345+0000 7fcdf4a8a640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf00c9da0 con 0x7fcdf00c9570 2023-12-19T18:26:44.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.345+0000 7fcdf4a8a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf00ca080 con 0x7fcdf00ccf80 2023-12-19T18:26:44.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.345+0000 7fcdef7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fcdf015d160 0x7fcdf00cc870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43806/0 (socket says 172.21.15.27:43806) 2023-12-19T18:26:44.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.345+0000 7fcdef7fe640 1 -- 172.21.15.27:0/2463588678 learned_addr learned my addr 172.21.15.27:0/2463588678 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.346+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 223904945 0 0) 0x7fcdf006f360 con 0x7fcdf015d160 2023-12-19T18:26:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.346+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcddc003520 con 0x7fcdf015d160 2023-12-19T18:26:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.346+0000 7fcdeeffd640 1 --2- 172.21.15.27:0/2463588678 >> v2:172.21.15.192:3300/0 conn(0x7fcdf00ccf80 0x7fcdf00c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.346+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2894829177 0 0) 0x7fcddc003520 con 0x7fcdf015d160 2023-12-19T18:26:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.346+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 >> v2:172.21.15.192:3300/0 conn(0x7fcdf00ccf80 msgr2=0x7fcdf00c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdee7fc640 1 --2- 172.21.15.27:0/2463588678 >> v2:172.21.15.192:3300/0 conn(0x7fcdf00ccf80 0x7fcdf00c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 >> v1:172.21.15.192:6789/0 conn(0x7fcdf00c9570 legacy=0x7fcdf006fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf00ca360 con 0x7fcdf015d160 2023-12-19T18:26:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fcde0001670 con 0x7fcdf015d160 2023-12-19T18:26:44.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 >> v1:172.21.15.27:6789/0 conn(0x7fcdf015d160 legacy=0x7fcdf00cc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdee7fc640 1 --2- 172.21.15.27:0/2463588678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcddc0039c0 0x7fcddc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdee7fc640 1 --2- 172.21.15.27:0/2463588678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcddc0047c0 0x7fcddc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdee7fc640 1 --2- 172.21.15.27:0/2463588678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcddc005180 0x7fcddc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00ca080 con 0x7fcddc0047c0 2023-12-19T18:26:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc00c080 con 0x7fcddc0039c0 2023-12-19T18:26:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcddc00c240 con 0x7fcddc005180 2023-12-19T18:26:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdeeffd640 1 --2- 172.21.15.27:0/2463588678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcddc0047c0 0x7fcddc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdef7fe640 1 --2- 172.21.15.27:0/2463588678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcddc0039c0 0x7fcddc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.347+0000 7fcdeffff640 1 --2- 172.21.15.27:0/2463588678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcddc005180 0x7fcddc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.348+0000 7fcdeeffd640 1 -- 172.21.15.27:0/2463588678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcddc005180 msgr2=0x7fcddc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.348+0000 7fcdeeffd640 1 --2- 172.21.15.27:0/2463588678 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcddc005180 0x7fcddc00b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.348+0000 7fcdeeffd640 1 -- 172.21.15.27:0/2463588678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcddc0039c0 msgr2=0x7fcddc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:44.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.348+0000 7fcdeeffd640 1 --2- 172.21.15.27:0/2463588678 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcddc0039c0 0x7fcddc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.348+0000 7fcdeeffd640 1 -- 172.21.15.27:0/2463588678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcddc003520 con 0x7fcddc0047c0 2023-12-19T18:26:44.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.348+0000 7fcdeffff640 1 -- 172.21.15.27:0/2463588678 reap_dead start 2023-12-19T18:26:44.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.349+0000 7fcdeeffd640 1 --2- 172.21.15.27:0/2463588678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcddc0047c0 0x7fcddc004c40 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009ac0 tx=0x7fcddc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d1766e7a26e3f9f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:44.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.349+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd802c6b0 con 0x7fcddc0047c0 2023-12-19T18:26:44.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.349+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdd802c810 con 0x7fcddc0047c0 2023-12-19T18:26:44.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.349+0000 7fcdee7fc640 1 -- 172.21.15.27:0/2463588678 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdd802cba0 con 0x7fcddc0047c0 2023-12-19T18:26:44.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.350+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/2463588678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcddc0047c0 msgr2=0x7fcddc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:44.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.350+0000 7fcdf4a8a640 1 --2- 172.21.15.27:0/2463588678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcddc0047c0 0x7fcddc004c40 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8009ac0 tx=0x7fcddc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.350+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/2463588678 shutdown_connections 2023-12-19T18:26:44.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.350+0000 7fcdf4a8a640 1 --2- 172.21.15.27:0/2463588678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcddc0047c0 0x7fcddc004c40 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.350+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/2463588678 >> 172.21.15.27:0/2463588678 conn(0x7fcdf00cb3f0 msgr2=0x7fcdf00cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:44.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.350+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/2463588678 shutdown_connections 2023-12-19T18:26:44.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.350+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/2463588678 wait complete. 2023-12-19T18:26:44.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.352+0000 7fcdf4a8a640 1 Processor -- start 2023-12-19T18:26:44.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.352+0000 7fcdf4a8a640 1 -- start start 2023-12-19T18:26:44.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.354+0000 7fcdf4a8a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcddc0047c0 0x7fcdf00c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.354+0000 7fcdf4a8a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf014a1e0 0x7fcdf015df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.354+0000 7fcdf4a8a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdf00c7c10 0x7fcdf00c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.354+0000 7fcdf4a8a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00c85d0 con 0x7fcddc0047c0 2023-12-19T18:26:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.354+0000 7fcdf4a8a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf014b430 con 0x7fcdf00c7c10 2023-12-19T18:26:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.354+0000 7fcdf4a8a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00ae370 con 0x7fcdf014a1e0 2023-12-19T18:26:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.354+0000 7fcdef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcddc0047c0 0x7fcdf00c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.354+0000 7fcdef7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcddc0047c0 0x7fcdf00c5320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36044/0 (socket says 172.21.15.27:36044) 2023-12-19T18:26:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.354+0000 7fcdef7fe640 1 -- 172.21.15.27:0/3139934922 learned_addr learned my addr 172.21.15.27:0/3139934922 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.354+0000 7fcdeeffd640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf014a1e0 0x7fcdf015df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.355+0000 7fcdef7fe640 1 -- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf014a1e0 msgr2=0x7fcdf015df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.355+0000 7fcdef7fe640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf014a1e0 0x7fcdf015df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.355+0000 7fcdef7fe640 1 -- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdf00c7c10 msgr2=0x7fcdf00c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.355+0000 7fcdef7fe640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdf00c7c10 0x7fcdf00c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.355+0000 7fcdef7fe640 1 -- 172.21.15.27:0/3139934922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf015e4a0 con 0x7fcddc0047c0 2023-12-19T18:26:44.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.355+0000 7fcdeeffd640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf014a1e0 0x7fcdf015df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:44.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.355+0000 7fcdef7fe640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcddc0047c0 0x7fcdf00c5320 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fcde000e7b0 tx=0x7fcde000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:44.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.355+0000 7fcde7fff640 1 -- 172.21.15.27:0/3139934922 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde000ee60 con 0x7fcddc0047c0 2023-12-19T18:26:44.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.355+0000 7fcde7fff640 1 -- 172.21.15.27:0/3139934922 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde000c8d0 con 0x7fcddc0047c0 2023-12-19T18:26:44.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.355+0000 7fcde7fff640 1 -- 172.21.15.27:0/3139934922 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde000cc80 con 0x7fcddc0047c0 2023-12-19T18:26:44.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.355+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/3139934922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf00ae610 con 0x7fcddc0047c0 2023-12-19T18:26:44.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.356+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/3139934922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf00aebd0 con 0x7fcddc0047c0 2023-12-19T18:26:44.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.357+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/3139934922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf00aedc0 con 0x7fcddc0047c0 2023-12-19T18:26:44.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.358+0000 7fcde7fff640 1 -- 172.21.15.27:0/3139934922 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcde001d460 con 0x7fcddc0047c0 2023-12-19T18:26:44.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.359+0000 7fcde7fff640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdd006a430 0x7fcdd006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.359+0000 7fcde7fff640 1 -- 172.21.15.27:0/3139934922 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcde00888a0 con 0x7fcddc0047c0 2023-12-19T18:26:44.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.360+0000 7fcdeeffd640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdd006a430 0x7fcdd006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:44.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.361+0000 7fcdeeffd640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdd006a430 0x7fcdd006c8f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8036000 tx=0x7fcdd8030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:44.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.364+0000 7fcde7fff640 1 -- 172.21.15.27:0/3139934922 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcde00579b0 con 0x7fcddc0047c0 2023-12-19T18:26:44.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.629+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/3139934922 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fcdf004e840 con 0x7fcddc0047c0 2023-12-19T18:26:44.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.636+0000 7fcde7fff640 1 -- 172.21.15.27:0/3139934922 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7fcde0057540 con 0x7fcddc0047c0 2023-12-19T18:26:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2023-12-19T18:26:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.639+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdd006a430 msgr2=0x7fcdd006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.639+0000 7fcdf4a8a640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdd006a430 0x7fcdd006c8f0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7fcdd8036000 tx=0x7fcdd8030000 comp rx=0 tx=0).stop 2023-12-19T18:26:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.639+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/3139934922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcddc0047c0 msgr2=0x7fcdf00c5320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.639+0000 7fcdf4a8a640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcddc0047c0 0x7fcdf00c5320 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fcde000e7b0 tx=0x7fcde000ec70 comp rx=0 tx=0).stop 2023-12-19T18:26:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.639+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/3139934922 shutdown_connections 2023-12-19T18:26:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.639+0000 7fcdf4a8a640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcdd006a430 0x7fcdd006c8f0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.639+0000 7fcdf4a8a640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcdf00c7c10 0x7fcdf00c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.639+0000 7fcdf4a8a640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcdf014a1e0 0x7fcdf015df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.639+0000 7fcdf4a8a640 1 --2- 172.21.15.27:0/3139934922 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcddc0047c0 0x7fcdf00c5320 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.639+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/3139934922 >> 172.21.15.27:0/3139934922 conn(0x7fcdf00cb3f0 msgr2=0x7fcdf006b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.639+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/3139934922 shutdown_connections 2023-12-19T18:26:44.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.640+0000 7fcdf4a8a640 1 -- 172.21.15.27:0/3139934922 wait complete. 2023-12-19T18:26:44.660 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:26:44.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.899+0000 7f34d54c5640 1 Processor -- start 2023-12-19T18:26:44.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.899+0000 7f34d54c5640 1 -- start start 2023-12-19T18:26:44.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.901+0000 7f34d54c5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f34d01553a0 0x7f34d00c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.901+0000 7f34d54c5640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f34d015ee30 0x7f34d00c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.901+0000 7f34d54c5640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d00b0360 con 0x7f34d00afeb0 2023-12-19T18:26:44.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.901+0000 7f34d54c5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d00cbcf0 con 0x7f34d01553a0 2023-12-19T18:26:44.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.901+0000 7f34d54c5640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d00cb350 con 0x7f34d015ee30 2023-12-19T18:26:44.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.901+0000 7f34cffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f34d01553a0 0x7f34d00c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:44.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.901+0000 7f34cf7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f34d00afeb0 0x7f34d015e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43812/0 (socket says 172.21.15.27:43812) 2023-12-19T18:26:44.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.901+0000 7f34cf7fe640 1 -- 172.21.15.27:0/1765092537 learned_addr learned my addr 172.21.15.27:0/1765092537 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:44.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.902+0000 7f34d4cc4640 1 --2- 172.21.15.27:0/1765092537 >> v2:172.21.15.192:3300/0 conn(0x7f34d015ee30 0x7f34d00c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:44.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.902+0000 7f34cffff640 1 -- 172.21.15.27:0/1765092537 >> v2:172.21.15.192:3300/0 conn(0x7f34d015ee30 msgr2=0x7f34d00c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:44.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.902+0000 7f34cffff640 1 --2- 172.21.15.27:0/1765092537 >> v2:172.21.15.192:3300/0 conn(0x7f34d015ee30 0x7f34d00c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.902+0000 7f34cffff640 1 -- 172.21.15.27:0/1765092537 >> v1:172.21.15.27:6789/0 conn(0x7f34d00afeb0 legacy=0x7f34d015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:44.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.902+0000 7f34cffff640 1 -- 172.21.15.27:0/1765092537 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d00cb630 con 0x7f34d01553a0 2023-12-19T18:26:44.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.902+0000 7f34ceffd640 1 -- 172.21.15.27:0/1765092537 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3049466005 0 0) 0x7f34d00b0360 con 0x7f34d00afeb0 2023-12-19T18:26:44.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.903+0000 7f34cffff640 1 --2- 172.21.15.27:0/1765092537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d01553a0 0x7f34d00c50a0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f34d0158d00 tx=0x7f34b802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b80f8487a550b14e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:44.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.904+0000 7f34ceffd640 1 -- 172.21.15.27:0/1765092537 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b802e070 con 0x7f34d01553a0 2023-12-19T18:26:44.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.904+0000 7f34ceffd640 1 -- 172.21.15.27:0/1765092537 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b80315a0 con 0x7f34d01553a0 2023-12-19T18:26:44.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.904+0000 7f34ceffd640 1 -- 172.21.15.27:0/1765092537 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b8031930 con 0x7f34d01553a0 2023-12-19T18:26:44.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.904+0000 7f34d54c5640 1 -- 172.21.15.27:0/1765092537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d01553a0 msgr2=0x7f34d00c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:44.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.904+0000 7f34d54c5640 1 --2- 172.21.15.27:0/1765092537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d01553a0 0x7f34d00c50a0 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f34d0158d00 tx=0x7f34b802b840 comp rx=0 tx=0).stop 2023-12-19T18:26:44.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.904+0000 7f34d54c5640 1 -- 172.21.15.27:0/1765092537 shutdown_connections 2023-12-19T18:26:44.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.904+0000 7f34d54c5640 1 --2- 172.21.15.27:0/1765092537 >> v2:172.21.15.192:3300/0 conn(0x7f34d015ee30 0x7f34d00c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.904+0000 7f34d54c5640 1 --2- 172.21.15.27:0/1765092537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d01553a0 0x7f34d00c50a0 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.904+0000 7f34d54c5640 1 -- 172.21.15.27:0/1765092537 >> 172.21.15.27:0/1765092537 conn(0x7f34d0069c70 msgr2=0x7f34d006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:44.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.904+0000 7f34d54c5640 1 -- 172.21.15.27:0/1765092537 shutdown_connections 2023-12-19T18:26:44.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.904+0000 7f34d54c5640 1 -- 172.21.15.27:0/1765092537 wait complete. 2023-12-19T18:26:44.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.906+0000 7f34d54c5640 1 Processor -- start 2023-12-19T18:26:44.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.906+0000 7f34d54c5640 1 -- start start 2023-12-19T18:26:44.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.907+0000 7f34d54c5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34d00afeb0 0x7f34d014c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.907+0000 7f34d54c5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34d01553a0 0x7f34d006b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34d54c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d015ee30 0x7f34d00ad3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34d54c5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d006bdd0 con 0x7f34d015ee30 2023-12-19T18:26:44.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34d54c5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d006c0b0 con 0x7f34d01553a0 2023-12-19T18:26:44.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34d54c5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d006c390 con 0x7f34d00afeb0 2023-12-19T18:26:44.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34d4cc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d015ee30 0x7f34d00ad3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:44.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34d4cc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d015ee30 0x7f34d00ad3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36066/0 (socket says 172.21.15.27:36066) 2023-12-19T18:26:44.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34d4cc4640 1 -- 172.21.15.27:0/108428381 learned_addr learned my addr 172.21.15.27:0/108428381 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:44.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34cf7fe640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34d01553a0 0x7f34d006b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:44.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34cffff640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34d00afeb0 0x7f34d014c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:44.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34d4cc4640 1 -- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34d00afeb0 msgr2=0x7f34d014c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:44.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34d4cc4640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34d00afeb0 0x7f34d014c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34d4cc4640 1 -- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34d01553a0 msgr2=0x7f34d006b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:44.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34d4cc4640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34d01553a0 0x7f34d006b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:44.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34d4cc4640 1 -- 172.21.15.27:0/108428381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b8009400 con 0x7f34d015ee30 2023-12-19T18:26:44.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.908+0000 7f34cffff640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34d00afeb0 0x7f34d014c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:44.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.909+0000 7f34d4cc4640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d015ee30 0x7f34d00ad3d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f34c8007e60 tx=0x7f34c800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:44.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.909+0000 7f34ccff9640 1 -- 172.21.15.27:0/108428381 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8014420 con 0x7f34d015ee30 2023-12-19T18:26:44.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.909+0000 7f34ccff9640 1 -- 172.21.15.27:0/108428381 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34c80145a0 con 0x7f34d015ee30 2023-12-19T18:26:44.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.909+0000 7f34ccff9640 1 -- 172.21.15.27:0/108428381 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34c8014950 con 0x7f34d015ee30 2023-12-19T18:26:44.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.909+0000 7f34d54c5640 1 -- 172.21.15.27:0/108428381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d006c670 con 0x7f34d015ee30 2023-12-19T18:26:44.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.909+0000 7f34d54c5640 1 -- 172.21.15.27:0/108428381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d014ca10 con 0x7f34d015ee30 2023-12-19T18:26:44.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.910+0000 7f34d54c5640 1 -- 172.21.15.27:0/108428381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d00671b0 con 0x7f34d015ee30 2023-12-19T18:26:44.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.911+0000 7f34ccff9640 1 -- 172.21.15.27:0/108428381 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f34c8014ab0 con 0x7f34d015ee30 2023-12-19T18:26:44.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.912+0000 7f34ccff9640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f34b006a480 0x7f34b006c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:44.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.912+0000 7f34ccff9640 1 -- 172.21.15.27:0/108428381 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(185..185 src has 1..185) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f34c8089160 con 0x7f34d015ee30 2023-12-19T18:26:44.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.913+0000 7f34cffff640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f34b006a480 0x7f34b006c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:44.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.914+0000 7f34cffff640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f34b006a480 0x7f34b006c940 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f34b8036000 tx=0x7f34b8008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:44.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:44.916+0000 7f34ccff9640 1 -- 172.21.15.27:0/108428381 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f34d00671b0 con 0x7f34d015ee30 2023-12-19T18:26:45.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.146+0000 7f34d54c5640 1 -- 172.21.15.27:0/108428381 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34d00671b0 con 0x7f34d015ee30 2023-12-19T18:26:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.147+0000 7f34ccff9640 1 -- 172.21.15.27:0/108428381 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4719 (secure 0 0 0) 0x7f34d00671b0 con 0x7f34d015ee30 2023-12-19T18:26:45.147 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:45.147 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3563146395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3563146395},{"type":"v1","addr":"172.21.15.192:6836","nonce":3563146395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3445095528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3445095528},{"type":"v1","addr":"172.21.15.192:6837","nonce":3445095528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3536033217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3536033217},{"type":"v1","addr":"172.21.15.27:6833","nonce":3536033217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:26:38.483498+0000","modified":"2023-12-19T18:26:42.008767+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2895889621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2895889621},{"type":"v1","addr":"172.21.15.27:6835","nonce":2895889621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-19T18:26:45.147 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 77 2023-12-19T18:26:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.149+0000 7f34d54c5640 1 -- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f34b006a480 msgr2=0x7f34b006c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.149+0000 7f34d54c5640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f34b006a480 0x7f34b006c940 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f34b8036000 tx=0x7f34b8008030 comp rx=0 tx=0).stop 2023-12-19T18:26:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.149+0000 7f34d54c5640 1 -- 172.21.15.27:0/108428381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d015ee30 msgr2=0x7f34d00ad3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.149+0000 7f34d54c5640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d015ee30 0x7f34d00ad3d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f34c8007e60 tx=0x7f34c800e4b0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.150+0000 7f34d54c5640 1 -- 172.21.15.27:0/108428381 shutdown_connections 2023-12-19T18:26:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.150+0000 7f34d54c5640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f34b006a480 0x7f34b006c940 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.150+0000 7f34d54c5640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34d015ee30 0x7f34d00ad3d0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.150+0000 7f34d54c5640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f34d01553a0 0x7f34d006b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.150+0000 7f34d54c5640 1 --2- 172.21.15.27:0/108428381 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f34d00afeb0 0x7f34d014c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.150+0000 7f34d54c5640 1 -- 172.21.15.27:0/108428381 >> 172.21.15.27:0/108428381 conn(0x7f34d0069c70 msgr2=0x7f34d0154670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:45.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.150+0000 7f34d54c5640 1 -- 172.21.15.27:0/108428381 shutdown_connections 2023-12-19T18:26:45.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.150+0000 7f34d54c5640 1 -- 172.21.15.27:0/108428381 wait complete. 2023-12-19T18:26:45.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 77, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:26:38.483498+0000', 'modified': '2023-12-19T18:26:42.008767+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8631}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8631': {'gid': 8631, 'name': 'c', 'rank': 0, 'incarnation': 73, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/2895889621', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6834', 'nonce': 2895889621}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 2895889621}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:26:45.572 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:26:45.572 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-19T18:26:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.818+0000 7f306493d640 1 Processor -- start 2023-12-19T18:26:45.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.820+0000 7f306493d640 1 -- start start 2023-12-19T18:26:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.820+0000 7f306493d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30600b49d0 0x7f3060160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.820+0000 7f306493d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f30600b89b0 0x7f306015c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.820+0000 7f306493d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30600b5ca0 con 0x7f30600b4db0 2023-12-19T18:26:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.820+0000 7f306493d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3060161800 con 0x7f30600b89b0 2023-12-19T18:26:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.820+0000 7f306493d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3060161240 con 0x7f30600b49d0 2023-12-19T18:26:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.821+0000 7f305ed9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30600b49d0 0x7f3060160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.821+0000 7f305ed9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30600b49d0 0x7f3060160940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:32974/0 (socket says 172.21.15.27:32974) 2023-12-19T18:26:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.821+0000 7f305ed9d640 1 -- 172.21.15.27:0/1307979473 learned_addr learned my addr 172.21.15.27:0/1307979473 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:45.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.821+0000 7f3057fff640 1 --2- 172.21.15.27:0/1307979473 >> v2:172.21.15.192:3300/0 conn(0x7f30600b89b0 0x7f306015c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.822+0000 7f305ed9d640 1 -- 172.21.15.27:0/1307979473 >> v2:172.21.15.192:3300/0 conn(0x7f30600b89b0 msgr2=0x7f306015c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.822+0000 7f305ed9d640 1 --2- 172.21.15.27:0/1307979473 >> v2:172.21.15.192:3300/0 conn(0x7f30600b89b0 0x7f306015c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.822+0000 7f305ed9d640 1 -- 172.21.15.27:0/1307979473 >> v1:172.21.15.192:6789/0 conn(0x7f30600b4db0 legacy=0x7f306006bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:45.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.822+0000 7f305ed9d640 1 -- 172.21.15.27:0/1307979473 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3060161520 con 0x7f30600b49d0 2023-12-19T18:26:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.824+0000 7f305ed9d640 1 --2- 172.21.15.27:0/1307979473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30600b49d0 0x7f3060160940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3050009850 tx=0x7f305002b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96b14d1cc9b88780 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.824+0000 7f305e59c640 1 -- 172.21.15.27:0/1307979473 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050004230 con 0x7f30600b49d0 2023-12-19T18:26:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.824+0000 7f305e59c640 1 -- 172.21.15.27:0/1307979473 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3050004390 con 0x7f30600b49d0 2023-12-19T18:26:45.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.824+0000 7f305e59c640 1 -- 172.21.15.27:0/1307979473 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050003810 con 0x7f30600b49d0 2023-12-19T18:26:45.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.825+0000 7f306493d640 1 -- 172.21.15.27:0/1307979473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30600b49d0 msgr2=0x7f3060160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:45.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.825+0000 7f306493d640 1 --2- 172.21.15.27:0/1307979473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30600b49d0 0x7f3060160940 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3050009850 tx=0x7f305002b9f0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.825+0000 7f306493d640 1 -- 172.21.15.27:0/1307979473 shutdown_connections 2023-12-19T18:26:45.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.825+0000 7f306493d640 1 --2- 172.21.15.27:0/1307979473 >> v2:172.21.15.192:3300/0 conn(0x7f30600b89b0 0x7f306015c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.825+0000 7f306493d640 1 --2- 172.21.15.27:0/1307979473 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30600b49d0 0x7f3060160940 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.825+0000 7f306493d640 1 -- 172.21.15.27:0/1307979473 >> 172.21.15.27:0/1307979473 conn(0x7f30600cf760 msgr2=0x7f30600cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.825+0000 7f306493d640 1 -- 172.21.15.27:0/1307979473 shutdown_connections 2023-12-19T18:26:45.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.825+0000 7f306493d640 1 -- 172.21.15.27:0/1307979473 wait complete. 2023-12-19T18:26:45.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.827+0000 7f306493d640 1 Processor -- start 2023-12-19T18:26:45.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.828+0000 7f306493d640 1 -- start start 2023-12-19T18:26:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.829+0000 7f306493d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30600b49d0 0x7f306015b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.829+0000 7f306493d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30600b4db0 0x7f30601575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.829+0000 7f306493d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30600b89b0 0x7f3060157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.829+0000 7f306493d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f306015afa0 con 0x7f30600b4db0 2023-12-19T18:26:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.829+0000 7f306493d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3060146160 con 0x7f30600b89b0 2023-12-19T18:26:45.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.829+0000 7f306493d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f306015a950 con 0x7f30600b49d0 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.829+0000 7f3057fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30600b4db0 0x7f30601575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.829+0000 7f3057fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30600b4db0 0x7f30601575c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36072/0 (socket says 172.21.15.27:36072) 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.829+0000 7f3057fff640 1 -- 172.21.15.27:0/3456254943 learned_addr learned my addr 172.21.15.27:0/3456254943 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.829+0000 7f305ed9d640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30600b49d0 0x7f306015b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.829+0000 7f305f59e640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30600b89b0 0x7f3060157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.830+0000 7f3057fff640 1 -- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30600b49d0 msgr2=0x7f306015b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.830+0000 7f3057fff640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30600b49d0 0x7f306015b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.830+0000 7f3057fff640 1 -- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30600b89b0 msgr2=0x7f3060157b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.830+0000 7f3057fff640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30600b89b0 0x7f3060157b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.830+0000 7f3057fff640 1 -- 172.21.15.27:0/3456254943 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3050009460 con 0x7f30600b4db0 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.830+0000 7f305f59e640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30600b89b0 0x7f3060157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.830+0000 7f3057fff640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30600b4db0 0x7f30601575c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f304800e840 tx=0x7f304800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.830+0000 7f30577fe640 1 -- 172.21.15.27:0/3456254943 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30480094f0 con 0x7f30600b4db0 2023-12-19T18:26:45.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.830+0000 7f30577fe640 1 -- 172.21.15.27:0/3456254943 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f304800c940 con 0x7f30600b4db0 2023-12-19T18:26:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.830+0000 7f30577fe640 1 -- 172.21.15.27:0/3456254943 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f304800ccd0 con 0x7f30600b4db0 2023-12-19T18:26:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.830+0000 7f306493d640 1 -- 172.21.15.27:0/3456254943 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f306015ac30 con 0x7f30600b4db0 2023-12-19T18:26:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.831+0000 7f306493d640 1 -- 172.21.15.27:0/3456254943 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f306015b970 con 0x7f30600b4db0 2023-12-19T18:26:45.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.833+0000 7f30577fe640 1 -- 172.21.15.27:0/3456254943 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3048013470 con 0x7f30600b4db0 2023-12-19T18:26:45.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.834+0000 7f30577fe640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f304006a430 0x7f304006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.834+0000 7f30577fe640 1 -- 172.21.15.27:0/3456254943 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3048088af0 con 0x7f30600b4db0 2023-12-19T18:26:45.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.834+0000 7f305ed9d640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f304006a430 0x7f304006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.834+0000 7f3067d82740 1 Processor -- start 2023-12-19T18:26:45.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.835+0000 7f3067d82740 1 -- start start 2023-12-19T18:26:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.836+0000 7f305ed9d640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f304006a430 0x7f304006c8f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3050035000 tx=0x7f3050008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:45.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.836+0000 7f3067d82740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 0x5573264894f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.836+0000 7f3067d82740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557326459090 0x557326486380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.836+0000 7f3067d82740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55732645cc90 0x5573264868c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.836+0000 7f3067d82740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557326434650 con 0x557326489110 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.836+0000 7f3067d82740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573263c9000 con 0x55732645cc90 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.836+0000 7f3067d82740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573263c6790 con 0x557326459090 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.836+0000 7f305ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 0x5573264894f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.836+0000 7f305ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 0x5573264894f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36082/0 (socket says 172.21.15.27:36082) 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.836+0000 7f305ed9d640 1 -- 172.21.15.27:0/3267170035 learned_addr learned my addr 172.21.15.27:0/3267170035 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.836+0000 7f305f59e640 1 --2- 172.21.15.27:0/3267170035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55732645cc90 0x5573264868c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.836+0000 7f3057fff640 1 --2- 172.21.15.27:0/3267170035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557326459090 0x557326486380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.836+0000 7f305ed9d640 1 -- 172.21.15.27:0/3267170035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557326459090 msgr2=0x557326486380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.837+0000 7f305ed9d640 1 --2- 172.21.15.27:0/3267170035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557326459090 0x557326486380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.837+0000 7f305ed9d640 1 -- 172.21.15.27:0/3267170035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55732645cc90 msgr2=0x5573264868c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:45.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.837+0000 7f305ed9d640 1 --2- 172.21.15.27:0/3267170035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55732645cc90 0x5573264868c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.837+0000 7f305ed9d640 1 -- 172.21.15.27:0/3267170035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557326389710 con 0x557326489110 2023-12-19T18:26:45.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.837+0000 7f305f59e640 1 --2- 172.21.15.27:0/3267170035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55732645cc90 0x5573264868c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:45.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.837+0000 7f305ed9d640 1 --2- 172.21.15.27:0/3267170035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 0x5573264894f0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f3050009970 tx=0x7f3050030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=effef884d8e67ff server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.837+0000 7f3054ff9640 1 -- 172.21.15.27:0/3267170035 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30500038d0 con 0x557326489110 2023-12-19T18:26:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.837+0000 7f3054ff9640 1 -- 172.21.15.27:0/3267170035 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305003e040 con 0x557326489110 2023-12-19T18:26:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.837+0000 7f3054ff9640 1 -- 172.21.15.27:0/3267170035 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050003c90 con 0x557326489110 2023-12-19T18:26:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.838+0000 7f3067d82740 1 -- 172.21.15.27:0/3267170035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 msgr2=0x5573264894f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.838+0000 7f3067d82740 1 --2- 172.21.15.27:0/3267170035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 0x5573264894f0 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f3050009970 tx=0x7f3050030000 comp rx=0 tx=0).stop 2023-12-19T18:26:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.838+0000 7f3067d82740 1 -- 172.21.15.27:0/3267170035 shutdown_connections 2023-12-19T18:26:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.838+0000 7f3067d82740 1 --2- 172.21.15.27:0/3267170035 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55732645cc90 0x5573264868c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.838+0000 7f3067d82740 1 --2- 172.21.15.27:0/3267170035 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x557326459090 0x557326486380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.838+0000 7f3067d82740 1 --2- 172.21.15.27:0/3267170035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 0x5573264894f0 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.838+0000 7f3067d82740 1 -- 172.21.15.27:0/3267170035 >> 172.21.15.27:0/3267170035 conn(0x55732645b3a0 msgr2=0x5573264a3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.838+0000 7f3067d82740 1 -- 172.21.15.27:0/3267170035 shutdown_connections 2023-12-19T18:26:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.838+0000 7f3067d82740 1 -- 172.21.15.27:0/3267170035 wait complete. 2023-12-19T18:26:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.839+0000 7f3067d82740 1 -- start start 2023-12-19T18:26:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.839+0000 7f3067d82740 1 Processor -- start 2023-12-19T18:26:45.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.840+0000 7f3067d82740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 0x55732647ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.840+0000 7f3067d82740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557326489110 0x5573264815b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.840+0000 7f3067d82740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55732647d250 0x557326481af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.840+0000 7f3067d82740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573263c2080 con 0x55732645cc90 2023-12-19T18:26:45.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.840+0000 7f3067d82740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x557326363850 con 0x557326489110 2023-12-19T18:26:45.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.840+0000 7f3067d82740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573263634f0 con 0x55732647d250 2023-12-19T18:26:45.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.840+0000 7f3067d82740 10 client.0 _ll_register_callbacks cb 0x5573263bb9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:26:45.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.840+0000 7f305ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 0x55732647ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.840+0000 7f305ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 0x55732647ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36086/0 (socket says 172.21.15.27:36086) 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.840+0000 7f305ed9d640 1 -- 172.21.15.27:0/872049655 learned_addr learned my addr 172.21.15.27:0/872049655 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.841+0000 7f305ed9d640 1 -- 172.21.15.27:0/872049655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55732647d250 msgr2=0x557326481af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.841+0000 7f305ed9d640 1 --2- 172.21.15.27:0/872049655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55732647d250 0x557326481af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.841+0000 7f305ed9d640 1 -- 172.21.15.27:0/872049655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557326489110 msgr2=0x5573264815b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.841+0000 7f305ed9d640 1 --2- 172.21.15.27:0/872049655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557326489110 0x5573264815b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.841+0000 7f305ed9d640 1 -- 172.21.15.27:0/872049655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3060161520 con 0x55732645cc90 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.841+0000 7f305ed9d640 1 --2- 172.21.15.27:0/872049655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 0x55732647ccb0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f3050009940 tx=0x7f3050004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466c970decb122e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.841+0000 7f303dffb640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.841+0000 7f303dffb640 1 -- 172.21.15.27:0/872049655 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305002f810 con 0x55732645cc90 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.841+0000 7f303dffb640 1 -- 172.21.15.27:0/872049655 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f305005bb30 con 0x55732645cc90 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.841+0000 7f3067d82740 1 -- 172.21.15.27:0/872049655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5573263ec5d0 con 0x55732645cc90 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.842+0000 7f303dffb640 1 -- 172.21.15.27:0/872049655 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3050003f20 con 0x55732645cc90 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.842+0000 7f3067d82740 10 client.8748 fetch_fsmap learned FSMap version 77 2023-12-19T18:26:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.842+0000 7f3067d82740 1 -- 172.21.15.27:0/872049655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x5573263c0700 con 0x55732645cc90 2023-12-19T18:26:45.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.842+0000 7f303dffb640 1 -- 172.21.15.27:0/872049655 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5573263ec5d0 con 0x55732645cc90 2023-12-19T18:26:45.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.842+0000 7f303dffb640 1 -- 172.21.15.27:0/872049655 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 77) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f3050041070 con 0x55732645cc90 2023-12-19T18:26:45.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.842+0000 7f3067d82740 10 client.8748 fetch_fsmap finished waiting for FSMap version 77 2023-12-19T18:26:45.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.842+0000 7f3067d82740 10 client.8748 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:26:45.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.842+0000 7f3067d82740 20 client.8748 populate_metadata read hostname 'smithi027' 2023-12-19T18:26:45.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.842+0000 7f3067d82740 1 --2- 172.21.15.27:0/872049655 >> [v2:172.21.15.27:6834/2895889621,v1:172.21.15.27:6835/2895889621] conn(0x557326482d80 0x557326483180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.842+0000 7f3067d82740 4 client.8748 mds_command: new command op to 8631 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:26:45.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.842+0000 7f3067d82740 1 -- 172.21.15.27:0/872049655 --> [v2:172.21.15.27:6834/2895889621,v1:172.21.15.27:6835/2895889621] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5573263b48d0 con 0x557326482d80 2023-12-19T18:26:45.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.842+0000 7f3057fff640 1 --2- 172.21.15.27:0/872049655 >> [v2:172.21.15.27:6834/2895889621,v1:172.21.15.27:6835/2895889621] conn(0x557326482d80 0x557326483180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.843+0000 7f303dffb640 10 client.8748 ms_handle_connect on v2:172.21.15.27:6834/2895889621 2023-12-19T18:26:45.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.843+0000 7f3057fff640 1 --2- 172.21.15.27:0/872049655 >> [v2:172.21.15.27:6834/2895889621,v1:172.21.15.27:6835/2895889621] conn(0x557326482d80 0x557326483180 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb1a6f971cfd98 server_cookie=c3fc53a4c5c0b492 in_seq=0 out_seq=0 2023-12-19T18:26:45.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f303dffb640 1 -- 172.21.15.27:0/872049655 <== mds.0 v2:172.21.15.27:6834/2895889621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5573263b48d0 con 0x557326482d80 2023-12-19T18:26:45.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f303dffb640 10 client.8748 handle_command_reply: tid=0 2023-12-19T18:26:45.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f3067d82740 1 client.8748 shutdown 2023-12-19T18:26:45.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f3067d82740 1 -- 172.21.15.27:0/872049655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 msgr2=0x55732647ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:45.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f3067d82740 1 --2- 172.21.15.27:0/872049655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 0x55732647ccb0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f3050009940 tx=0x7f3050004730 comp rx=0 tx=0).stop 2023-12-19T18:26:45.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f3067d82740 1 -- 172.21.15.27:0/872049655 shutdown_connections 2023-12-19T18:26:45.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f3067d82740 1 --2- 172.21.15.27:0/872049655 >> [v2:172.21.15.27:6834/2895889621,v1:172.21.15.27:6835/2895889621] conn(0x557326482d80 0x557326483180 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f3067d82740 1 --2- 172.21.15.27:0/872049655 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55732647d250 0x557326481af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f3067d82740 1 --2- 172.21.15.27:0/872049655 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557326489110 0x5573264815b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f3067d82740 1 --2- 172.21.15.27:0/872049655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 0x55732647ccb0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f303dffb640 0 client.8748 ms_handle_reset on v2:172.21.15.27:6834/2895889621 2023-12-19T18:26:45.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f3067d82740 1 -- 172.21.15.27:0/872049655 >> 172.21.15.27:0/872049655 conn(0x557326459090 msgr2=0x55732648ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:45.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f3067d82740 1 -- 172.21.15.27:0/872049655 shutdown_connections 2023-12-19T18:26:45.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f3067d82740 1 -- 172.21.15.27:0/872049655 wait complete. 2023-12-19T18:26:45.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.844+0000 7f3067d82740 20 client.8748 trim_cache size 0 max 16384 2023-12-19T18:26:45.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.861+0000 7f3067d82740 1 Processor -- start 2023-12-19T18:26:45.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.861+0000 7f3067d82740 1 -- start start 2023-12-19T18:26:45.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f3067d82740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 0x55732648a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f3067d82740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55732647d250 0x55732645d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f3067d82740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557326489110 0x55732645d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f3067d82740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x557326129f20 con 0x55732645cc90 2023-12-19T18:26:45.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f3067d82740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55732648ce30 con 0x557326489110 2023-12-19T18:26:45.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f3067d82740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573263c11f0 con 0x55732647d250 2023-12-19T18:26:45.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f305ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 0x55732648a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f305ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 0x55732648a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36088/0 (socket says 172.21.15.27:36088) 2023-12-19T18:26:45.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f305ed9d640 1 -- 172.21.15.27:0/2726287628 learned_addr learned my addr 172.21.15.27:0/2726287628 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:45.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f305ed9d640 1 -- 172.21.15.27:0/2726287628 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55732647d250 msgr2=0x55732645d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:45.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f305ed9d640 1 --2- 172.21.15.27:0/2726287628 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55732647d250 0x55732645d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f305ed9d640 1 -- 172.21.15.27:0/2726287628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557326489110 msgr2=0x55732645d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:45.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f305ed9d640 1 --2- 172.21.15.27:0/2726287628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557326489110 0x55732645d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f305ed9d640 1 -- 172.21.15.27:0/2726287628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5573263f6280 con 0x55732645cc90 2023-12-19T18:26:45.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.862+0000 7f305ed9d640 1 --2- 172.21.15.27:0/2726287628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 0x55732648a2f0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f3050000c00 tx=0x7f3050053040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5918e1bdfcb6dce9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:45.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.863+0000 7f30557fa640 1 -- 172.21.15.27:0/2726287628 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f305002f810 con 0x55732645cc90 2023-12-19T18:26:45.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.863+0000 7f30557fa640 1 -- 172.21.15.27:0/2726287628 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305002f970 con 0x55732645cc90 2023-12-19T18:26:45.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.863+0000 7f30557fa640 1 -- 172.21.15.27:0/2726287628 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3050003a80 con 0x55732645cc90 2023-12-19T18:26:45.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.863+0000 7f3067d82740 1 -- 172.21.15.27:0/2726287628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 msgr2=0x55732648a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:45.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.863+0000 7f3067d82740 1 --2- 172.21.15.27:0/2726287628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 0x55732648a2f0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f3050000c00 tx=0x7f3050053040 comp rx=0 tx=0).stop 2023-12-19T18:26:45.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.863+0000 7f3067d82740 1 -- 172.21.15.27:0/2726287628 shutdown_connections 2023-12-19T18:26:45.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.863+0000 7f3067d82740 1 --2- 172.21.15.27:0/2726287628 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x557326489110 0x55732645d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.863+0000 7f3067d82740 1 --2- 172.21.15.27:0/2726287628 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55732647d250 0x55732645d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.863+0000 7f3067d82740 1 --2- 172.21.15.27:0/2726287628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55732645cc90 0x55732648a2f0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.863+0000 7f3067d82740 1 -- 172.21.15.27:0/2726287628 >> 172.21.15.27:0/2726287628 conn(0x557326459090 msgr2=0x557326483160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.863+0000 7f3067d82740 1 -- 172.21.15.27:0/2726287628 shutdown_connections 2023-12-19T18:26:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.863+0000 7f3067d82740 1 -- 172.21.15.27:0/2726287628 wait complete. 2023-12-19T18:26:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.864+0000 7f3067d82740 1 -- start start 2023-12-19T18:26:45.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.864+0000 7f3067d82740 1 Processor -- start 2023-12-19T18:26:45.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.865+0000 7f3067d82740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55732645cc90 0x55732645dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.865+0000 7f3067d82740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55732647d250 0x557326473a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.865+0000 7f3067d82740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 0x557326509520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.865+0000 7f3067d82740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573264a4620 con 0x557326489110 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.865+0000 7f3067d82740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x5573264a3b10 con 0x55732647d250 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.865+0000 7f3067d82740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x5573264a4310 con 0x55732645cc90 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.865+0000 7f3067d82740 10 client.0 _ll_register_callbacks cb 0x7f3050009680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.866+0000 7f305f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 0x557326509520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.866+0000 7f305f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 0x557326509520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36098/0 (socket says 172.21.15.27:36098) 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.866+0000 7f305f59e640 1 -- 172.21.15.27:0/514567903 learned_addr learned my addr 172.21.15.27:0/514567903 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.866+0000 7f305ed9d640 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55732645cc90 0x55732645dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.866+0000 7f3057fff640 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55732647d250 0x557326473a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.866+0000 7f305f59e640 1 -- 172.21.15.27:0/514567903 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55732645cc90 msgr2=0x55732645dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.866+0000 7f305f59e640 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55732645cc90 0x55732645dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.866+0000 7f305f59e640 1 -- 172.21.15.27:0/514567903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55732647d250 msgr2=0x557326473a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:45.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.866+0000 7f305f59e640 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55732647d250 0x557326473a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.866+0000 7f305f59e640 1 -- 172.21.15.27:0/514567903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5573263c0700 con 0x557326489110 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.866+0000 7f3057fff640 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55732647d250 0x557326473a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.866+0000 7f305f59e640 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 0x557326509520 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f305800e420 tx=0x7f305800e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a39573e24382da50 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.866+0000 7f303dffb640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f303dffb640 1 -- 172.21.15.27:0/514567903 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3058003d80 con 0x557326489110 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f303dffb640 1 -- 172.21.15.27:0/514567903 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(186..186 src has 1..186) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f30580050d0 con 0x557326489110 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f3067d82740 1 -- 172.21.15.27:0/514567903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f305003e040 con 0x557326489110 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f303dffb640 1 -- 172.21.15.27:0/514567903 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=77) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30580016f0 con 0x557326489110 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f3067d82740 10 client.8754 fetch_fsmap learned FSMap version 77 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f3067d82740 1 -- 172.21.15.27:0/514567903 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=77,monmap=2+}) v3 -- 0x557326362a60 con 0x557326489110 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f303dffb640 1 -- 172.21.15.27:0/514567903 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f305003e040 con 0x557326489110 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f303dffb640 1 -- 172.21.15.27:0/514567903 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 77) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f3058005290 con 0x557326489110 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f3067d82740 10 client.8754 fetch_fsmap finished waiting for FSMap version 77 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f3067d82740 10 client.8754 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:26:45.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f3067d82740 20 client.8754 populate_metadata read hostname 'smithi027' 2023-12-19T18:26:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f3067d82740 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.27:6834/2895889621,v1:172.21.15.27:6835/2895889621] conn(0x5573264842e0 0x5573264b5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f3067d82740 4 client.8754 mds_command: new command op to 8631 tid=0[{"prefix": "status"}] 2023-12-19T18:26:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.867+0000 7f3067d82740 1 -- 172.21.15.27:0/514567903 --> [v2:172.21.15.27:6834/2895889621,v1:172.21.15.27:6835/2895889621] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5573263bb100 con 0x5573264842e0 2023-12-19T18:26:45.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.868+0000 7f305ed9d640 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.27:6834/2895889621,v1:172.21.15.27:6835/2895889621] conn(0x5573264842e0 0x5573264b5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.868+0000 7f305ed9d640 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.27:6834/2895889621,v1:172.21.15.27:6835/2895889621] conn(0x5573264842e0 0x5573264b5890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2156def297332f18 server_cookie=5b59c6d1d8b25301 in_seq=0 out_seq=0 2023-12-19T18:26:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.868+0000 7f303dffb640 10 client.8754 ms_handle_connect on v2:172.21.15.27:6834/2895889621 2023-12-19T18:26:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.868+0000 7f303dffb640 1 -- 172.21.15.27:0/514567903 <== mds.0 v2:172.21.15.27:6834/2895889621 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5573263bb100 con 0x5573264842e0 2023-12-19T18:26:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.868+0000 7f303dffb640 10 client.8754 handle_command_reply: tid=0 2023-12-19T18:26:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.868+0000 7f3067d82740 1 client.8754 shutdown 2023-12-19T18:26:45.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.868+0000 7f3067d82740 1 -- 172.21.15.27:0/514567903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 msgr2=0x557326509520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.868+0000 7f3067d82740 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 0x557326509520 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f305800e420 tx=0x7f305800e8e0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.869+0000 7f3067d82740 1 -- 172.21.15.27:0/514567903 shutdown_connections 2023-12-19T18:26:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.869+0000 7f3067d82740 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.27:6834/2895889621,v1:172.21.15.27:6835/2895889621] conn(0x5573264842e0 0x5573264b5890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.869+0000 7f3067d82740 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x557326489110 0x557326509520 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.869+0000 7f3067d82740 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55732647d250 0x557326473a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.869+0000 7f3067d82740 1 --2- 172.21.15.27:0/514567903 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55732645cc90 0x55732645dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.869+0000 7f3067d82740 1 -- 172.21.15.27:0/514567903 >> 172.21.15.27:0/514567903 conn(0x557326459090 msgr2=0x557326483160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.869+0000 7f303dffb640 0 client.8754 ms_handle_reset on v2:172.21.15.27:6834/2895889621 2023-12-19T18:26:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.869+0000 7f3067d82740 1 -- 172.21.15.27:0/514567903 shutdown_connections 2023-12-19T18:26:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.869+0000 7f3067d82740 1 -- 172.21.15.27:0/514567903 wait complete. 2023-12-19T18:26:45.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.869+0000 7f3067d82740 20 client.8754 trim_cache size 0 max 16384 2023-12-19T18:26:45.872 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:26:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:26:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-12-19T18:26:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 8631, 2023-12-19T18:26:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-19T18:26:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-19T18:26:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-19T18:26:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 7.3818809740000004, 2023-12-19T18:26:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 77, 2023-12-19T18:26:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 186, 2023-12-19T18:26:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 185, 2023-12-19T18:26:45.873 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 13.158968527000001 2023-12-19T18:26:45.873 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:26:45.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.870+0000 7f306493d640 1 -- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f304006a430 msgr2=0x7f304006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:45.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.870+0000 7f306493d640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f304006a430 0x7f304006c8f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3050035000 tx=0x7f3050008030 comp rx=0 tx=0).stop 2023-12-19T18:26:45.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.870+0000 7f306493d640 1 -- 172.21.15.27:0/3456254943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30600b4db0 msgr2=0x7f30601575c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:45.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.870+0000 7f306493d640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30600b4db0 0x7f30601575c0 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f304800e840 tx=0x7f304800ed00 comp rx=0 tx=0).stop 2023-12-19T18:26:45.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.870+0000 7f306493d640 1 -- 172.21.15.27:0/3456254943 shutdown_connections 2023-12-19T18:26:45.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.870+0000 7f306493d640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f304006a430 0x7f304006c8f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.870+0000 7f306493d640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30600b89b0 0x7f3060157b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.870+0000 7f306493d640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30600b4db0 0x7f30601575c0 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.870+0000 7f306493d640 1 --2- 172.21.15.27:0/3456254943 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30600b49d0 0x7f306015b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:45.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.870+0000 7f306493d640 1 -- 172.21.15.27:0/3456254943 >> 172.21.15.27:0/3456254943 conn(0x7f30600cf760 msgr2=0x7f30600d23c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:45.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.870+0000 7f306493d640 1 -- 172.21.15.27:0/3456254943 shutdown_connections 2023-12-19T18:26:45.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:45.870+0000 7f306493d640 1 -- 172.21.15.27:0/3456254943 wait complete. 2023-12-19T18:26:45.886 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:26:46.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.098+0000 7efde96e7640 1 Processor -- start 2023-12-19T18:26:46.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.098+0000 7efde96e7640 1 -- start start 2023-12-19T18:26:46.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.099+0000 7efde96e7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efde415ae80 0x7efde40cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:46.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.100+0000 7efde96e7640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efde40b2810 0x7efde40b2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:46.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.100+0000 7efde96e7640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efde40cd6b0 con 0x7efde40b31b0 2023-12-19T18:26:46.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.100+0000 7efde96e7640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efde40cdc70 con 0x7efde40b2810 2023-12-19T18:26:46.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.100+0000 7efde96e7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde40b3640 con 0x7efde415ae80 2023-12-19T18:26:46.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.100+0000 7efde37fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efde40b31b0 0x7efde40ce960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43814/0 (socket says 172.21.15.27:43814) 2023-12-19T18:26:46.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.100+0000 7efde37fe640 1 -- 172.21.15.27:0/4066427212 learned_addr learned my addr 172.21.15.27:0/4066427212 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.100+0000 7efde27fc640 1 --2- 172.21.15.27:0/4066427212 >> v2:172.21.15.192:3300/0 conn(0x7efde40b2810 0x7efde40b2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.100+0000 7efde2ffd640 1 --2- 172.21.15.27:0/4066427212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efde415ae80 0x7efde40cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.100+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 810125959 0 0) 0x7efde40cd6b0 con 0x7efde40b31b0 2023-12-19T18:26:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efdd0003520 con 0x7efde40b31b0 2023-12-19T18:26:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2987689735 0 0) 0x7efdd0003520 con 0x7efde40b31b0 2023-12-19T18:26:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efde415ae80 msgr2=0x7efde40cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 --2- 172.21.15.27:0/4066427212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efde415ae80 0x7efde40cb3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 >> v2:172.21.15.192:3300/0 conn(0x7efde40b2810 msgr2=0x7efde40b2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 --2- 172.21.15.27:0/4066427212 >> v2:172.21.15.192:3300/0 conn(0x7efde40b2810 0x7efde40b2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde40d0450 con 0x7efde40b31b0 2023-12-19T18:26:46.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7efddc001670 con 0x7efde40b31b0 2023-12-19T18:26:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 >> v1:172.21.15.27:6789/0 conn(0x7efde40b31b0 legacy=0x7efde40ce960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 --2- 172.21.15.27:0/4066427212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd0003970 0x7efdd0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 --2- 172.21.15.27:0/4066427212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd00047c0 0x7efdd0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 --2- 172.21.15.27:0/4066427212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efde415ae80 0x7efdd0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde40cdc70 con 0x7efdd0003970 2023-12-19T18:26:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde40b3640 con 0x7efdd00047c0 2023-12-19T18:26:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdd000b980 con 0x7efde415ae80 2023-12-19T18:26:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.101+0000 7efde2ffd640 1 --2- 172.21.15.27:0/4066427212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd0003970 0x7efdd0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.102+0000 7efde2ffd640 1 -- 172.21.15.27:0/4066427212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efde415ae80 msgr2=0x7efdd0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.102+0000 7efde2ffd640 1 --2- 172.21.15.27:0/4066427212 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efde415ae80 0x7efdd0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.102+0000 7efde2ffd640 1 -- 172.21.15.27:0/4066427212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd00047c0 msgr2=0x7efdd0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:46.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.102+0000 7efde2ffd640 1 --2- 172.21.15.27:0/4066427212 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd00047c0 0x7efdd0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:46.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.102+0000 7efde2ffd640 1 -- 172.21.15.27:0/4066427212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdd0003520 con 0x7efdd0003970 2023-12-19T18:26:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.103+0000 7efde2ffd640 1 --2- 172.21.15.27:0/4066427212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd0003970 0x7efdd0003d90 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009fc0 tx=0x7efdd00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d8f24ae2b2bd9d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.103+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd402cc60 con 0x7efdd0003970 2023-12-19T18:26:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.103+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdd400f040 con 0x7efdd0003970 2023-12-19T18:26:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.103+0000 7efde1ffb640 1 -- 172.21.15.27:0/4066427212 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdd4034440 con 0x7efdd0003970 2023-12-19T18:26:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.104+0000 7efde96e7640 1 -- 172.21.15.27:0/4066427212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd0003970 msgr2=0x7efdd0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.104+0000 7efde96e7640 1 --2- 172.21.15.27:0/4066427212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdd0003970 0x7efdd0003d90 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7efdd4009fc0 tx=0x7efdd00057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.104+0000 7efde37fe640 1 -- 172.21.15.27:0/4066427212 reap_dead start 2023-12-19T18:26:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.104+0000 7efde96e7640 1 -- 172.21.15.27:0/4066427212 shutdown_connections 2023-12-19T18:26:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.104+0000 7efde96e7640 1 -- 172.21.15.27:0/4066427212 >> 172.21.15.27:0/4066427212 conn(0x7efde406f1f0 msgr2=0x7efde406f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.104+0000 7efde96e7640 1 -- 172.21.15.27:0/4066427212 shutdown_connections 2023-12-19T18:26:46.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.104+0000 7efde96e7640 1 -- 172.21.15.27:0/4066427212 wait complete. 2023-12-19T18:26:46.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.106+0000 7efde96e7640 1 Processor -- start 2023-12-19T18:26:46.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.106+0000 7efde96e7640 1 -- start start 2023-12-19T18:26:46.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.107+0000 7efde96e7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd0003970 0x7efde4148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:46.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.107+0000 7efde96e7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efde40ad370 0x7efde40ad7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:46.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.107+0000 7efde96e7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efde40c87e0 0x7efde40c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:46.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.107+0000 7efde96e7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde41486b0 con 0x7efde40c87e0 2023-12-19T18:26:46.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.107+0000 7efde96e7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efde41f26f0 con 0x7efdd0003970 2023-12-19T18:26:46.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.107+0000 7efde96e7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efde40ae130 con 0x7efde40ad370 2023-12-19T18:26:46.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.107+0000 7efde37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efde40c87e0 0x7efde40c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.107+0000 7efde37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efde40c87e0 0x7efde40c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36120/0 (socket says 172.21.15.27:36120) 2023-12-19T18:26:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.107+0000 7efde37fe640 1 -- 172.21.15.27:0/846650033 learned_addr learned my addr 172.21.15.27:0/846650033 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.107+0000 7efde2ffd640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd0003970 0x7efde4148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.107+0000 7efde27fc640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efde40ad370 0x7efde40ad7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.108+0000 7efde37fe640 1 -- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efde40ad370 msgr2=0x7efde40ad7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.108+0000 7efde37fe640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efde40ad370 0x7efde40ad7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.108+0000 7efde37fe640 1 -- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd0003970 msgr2=0x7efde4148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.108+0000 7efde37fe640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd0003970 0x7efde4148170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.108+0000 7efde37fe640 1 -- 172.21.15.27:0/846650033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efde40ae410 con 0x7efde40c87e0 2023-12-19T18:26:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.108+0000 7efde2ffd640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd0003970 0x7efde4148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:26:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.108+0000 7efde37fe640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efde40c87e0 0x7efde40c8c60 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7efde40d0cf0 tx=0x7efddc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.108+0000 7efddb7fe640 1 -- 172.21.15.27:0/846650033 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc007230 con 0x7efde40c87e0 2023-12-19T18:26:46.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.108+0000 7efddb7fe640 1 -- 172.21.15.27:0/846650033 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efddc00ced0 con 0x7efde40c87e0 2023-12-19T18:26:46.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.108+0000 7efddb7fe640 1 -- 172.21.15.27:0/846650033 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efddc0075d0 con 0x7efde40c87e0 2023-12-19T18:26:46.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.108+0000 7efde96e7640 1 -- 172.21.15.27:0/846650033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efde40c7fe0 con 0x7efde40c87e0 2023-12-19T18:26:46.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.109+0000 7efde96e7640 1 -- 172.21.15.27:0/846650033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efde40add30 con 0x7efde40c87e0 2023-12-19T18:26:46.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.110+0000 7efde96e7640 1 -- 172.21.15.27:0/846650033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efde40c8500 con 0x7efde40c87e0 2023-12-19T18:26:46.110 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.111+0000 7efddb7fe640 1 -- 172.21.15.27:0/846650033 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efddc0047e0 con 0x7efde40c87e0 2023-12-19T18:26:46.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.112+0000 7efddb7fe640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efdc406a430 0x7efdc406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:46.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.112+0000 7efddb7fe640 1 -- 172.21.15.27:0/846650033 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efddc087e40 con 0x7efde40c87e0 2023-12-19T18:26:46.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.112+0000 7efde2ffd640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efdc406a430 0x7efdc406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:46.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.114+0000 7efde2ffd640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efdc406a430 0x7efdc406c8f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7efdd4036000 tx=0x7efdd402e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:46.116 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.116+0000 7efddb7fe640 1 -- 172.21.15.27:0/846650033 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efde40c8500 con 0x7efde40c87e0 2023-12-19T18:26:46.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.359+0000 7efde96e7640 1 -- 172.21.15.27:0/846650033 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7efde40c8500 con 0x7efde40c87e0 2023-12-19T18:26:46.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.360+0000 7efddb7fe640 1 -- 172.21.15.27:0/846650033 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7efde40c8500 con 0x7efde40c87e0 2023-12-19T18:26:46.360 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:26:46.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.363+0000 7efde96e7640 1 -- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efdc406a430 msgr2=0x7efdc406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:46.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.363+0000 7efde96e7640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efdc406a430 0x7efdc406c8f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7efdd4036000 tx=0x7efdd402e000 comp rx=0 tx=0).stop 2023-12-19T18:26:46.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.363+0000 7efde96e7640 1 -- 172.21.15.27:0/846650033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efde40c87e0 msgr2=0x7efde40c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:46.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.363+0000 7efde96e7640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efde40c87e0 0x7efde40c8c60 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7efde40d0cf0 tx=0x7efddc00cb90 comp rx=0 tx=0).stop 2023-12-19T18:26:46.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.364+0000 7efde96e7640 1 -- 172.21.15.27:0/846650033 shutdown_connections 2023-12-19T18:26:46.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.364+0000 7efde96e7640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efdc406a430 0x7efdc406c8f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:46.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.364+0000 7efde96e7640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efde40c87e0 0x7efde40c8c60 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:46.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.364+0000 7efde96e7640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efde40ad370 0x7efde40ad7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:46.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.364+0000 7efde96e7640 1 --2- 172.21.15.27:0/846650033 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdd0003970 0x7efde4148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:46.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.364+0000 7efde96e7640 1 -- 172.21.15.27:0/846650033 >> 172.21.15.27:0/846650033 conn(0x7efde406f1f0 msgr2=0x7efde415a090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:46.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.364+0000 7efde96e7640 1 -- 172.21.15.27:0/846650033 shutdown_connections 2023-12-19T18:26:46.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:46.364+0000 7efde96e7640 1 -- 172.21.15.27:0/846650033 wait complete. 2023-12-19T18:26:46.382 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:26:46.382 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:26:46.382 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-12-19T18:26:46.382 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:26:46.382 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:26:46.382 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:26:46.382 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:26:46.382 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:26:46.382 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:26:46.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:46.383 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-12-19T18:26:46.440 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:26:46.440 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:26:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:26:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:26:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:46.441 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:26:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:26:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-12-19T18:26:46.441 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:26:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:26:46.441 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-12-19T18:26:46.441 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:26:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:26:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-12-19T18:26:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-19T18:26:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41634sec preferred_lft 41634sec 2023-12-19T18:26:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2023-12-19T18:26:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:46.442 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:26:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:26:46.442 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-12-19T18:26:46.442 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:26:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff 2023-12-19T18:26:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:26:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:46.443 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::8053:76ff:fe69:c66c/64 scope link 2023-12-19T18:26:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:46.444 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:26:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:26:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::781d:d6ff:feb4:7a2d/64 scope link 2023-12-19T18:26:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:46.444 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:26:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether f2:29:63:cc:14:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:26:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::f029:63ff:fecc:140e/64 scope link 2023-12-19T18:26:46.444 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:46.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:46.445 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:26:46.550 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-12-19T18:26:46.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:46.555 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-12-19T18:26:46.616 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-12-19T18:26:46.616 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:26:46.616 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-19T18:26:46.616 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-19T18:26:46.616 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:26:46.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:46.617 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:26:46.617 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:26:46.617 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:26:46.617 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:26:46.617 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:26:46.617 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:26:46.830 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:46.831 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:26:46.886 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:26:46.886 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:26:46.887 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:26:46.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:46.887 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-12-19T18:26:47.001 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-19T18:26:47.001 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:26:47.001 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:26:47.063 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:26:47.063 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:26:47.063 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:26:47.119 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:26:47.231 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:26:47.231 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:26:47.331 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:26:47.336 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:26:47.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:47.337 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:26:47.396 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:26:47.396 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-19T18:26:47.401 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:26:47.483 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:26:47.484 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:26:47.563 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:26:47.566 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:26:47.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:47.566 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:26:47.602 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-12-19T18:26:47.603+0000 7f0615252480 -1 init, newargv = 0x55ca5641cf50 newargc=15 2023-12-19T18:26:47.602 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[119737]: starting ceph client 2023-12-19T18:26:47.611 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[119737]: starting fuse 2023-12-19T18:26:48.035 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:26:48.045 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:26:48.711 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:26:48.825 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:26:48.826 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:26:48.931 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:26:48.935 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:26:48.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:48.935 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:26:48.993 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:26:48.993 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:26:48.994 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:26:48.995 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:26:48.995 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:26:48.995 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:26:48.995 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:26:48.995 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:26:48.995 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:26:48.995 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:26:48.995 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:26:48.995 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:26:48.995 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:26:48.995 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:26:49.201 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.119737.asok 2023-12-19T18:26:49.379 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.119737.asok 2023-12-19T18:26:49.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:49.380 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.119737.asok status 2023-12-19T18:26:49.643 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:26:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:26:49.643 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "119737", 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 8766, 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 8766 2023-12-19T18:26:49.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3327340520 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 3327340520 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.8766 192.168.0.1:0/3327340520", 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/3327340520", 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-12-19T18:26:49.645 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 77, 2023-12-19T18:26:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 187, 2023-12-19T18:26:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:26:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:26:49.646 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:26:49.646 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:26:49.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:49.659 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-12-19T18:26:49.659 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-19T18:26:49.721 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-12-19T18:26:49.722 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-19T18:26:49.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:49.722 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:26:49.831 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:26:50.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.062+0000 7f71391fb640 1 Processor -- start 2023-12-19T18:26:50.062 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.062+0000 7f71391fb640 1 -- start start 2023-12-19T18:26:50.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.063+0000 7f71391fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713415d240 0x7f71341618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:50.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.063+0000 7f71391fb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f71340cdf90 0x7f71340ce3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:50.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.063+0000 7f71391fb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71340ce930 con 0x7f71340b4c60 2023-12-19T18:26:50.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.063+0000 7f71391fb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71340d1970 con 0x7f71340cdf90 2023-12-19T18:26:50.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.063+0000 7f71391fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7134161e10 con 0x7f713415d240 2023-12-19T18:26:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.064+0000 7f713359e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f71340b4c60 0x7f71340b50f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43816/0 (socket says 172.21.15.27:43816) 2023-12-19T18:26:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.064+0000 7f713359e640 1 -- 172.21.15.27:0/343277552 learned_addr learned my addr 172.21.15.27:0/343277552 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.064+0000 7f713259c640 1 --2- 172.21.15.27:0/343277552 >> v2:172.21.15.192:3300/0 conn(0x7f71340cdf90 0x7f71340ce3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.064+0000 7f7132d9d640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713415d240 0x7f71341618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.064+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1959534756 0 0) 0x7f71340ce930 con 0x7f71340b4c60 2023-12-19T18:26:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.064+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f711c003520 con 0x7f71340b4c60 2023-12-19T18:26:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2493883943 0 0) 0x7f711c003520 con 0x7f71340b4c60 2023-12-19T18:26:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713415d240 msgr2=0x7f71341618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713415d240 0x7f71341618d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:50.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 >> v2:172.21.15.192:3300/0 conn(0x7f71340cdf90 msgr2=0x7f71340ce3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:50.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 --2- 172.21.15.27:0/343277552 >> v2:172.21.15.192:3300/0 conn(0x7f71340cdf90 0x7f71340ce3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:50.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f713406a5e0 con 0x7f71340b4c60 2023-12-19T18:26:50.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f712c0019f0 con 0x7f71340b4c60 2023-12-19T18:26:50.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 >> v1:172.21.15.27:6789/0 conn(0x7f71340b4c60 legacy=0x7f71340b50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:50.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c003970 0x7f711c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:50.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713415d240 0x7f711c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:50.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f711c004d00 0x7f711c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:50.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71340d1970 con 0x7f711c003970 2023-12-19T18:26:50.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7134161e10 con 0x7f711c004d00 2023-12-19T18:26:50.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c00b9b0 con 0x7f713415d240 2023-12-19T18:26:50.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.065+0000 7f7132d9d640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c003970 0x7f711c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:50.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.066+0000 7f7132d9d640 1 -- 172.21.15.27:0/343277552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713415d240 msgr2=0x7f711c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:50.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.066+0000 7f713259c640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713415d240 0x7f711c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:50.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.066+0000 7f713359e640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f711c004d00 0x7f711c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:50.067 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.066+0000 7f7132d9d640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713415d240 0x7f711c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:50.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.066+0000 7f7132d9d640 1 -- 172.21.15.27:0/343277552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f711c004d00 msgr2=0x7f711c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:50.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.066+0000 7f7132d9d640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f711c004d00 0x7f711c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:50.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.066+0000 7f7132d9d640 1 -- 172.21.15.27:0/343277552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c003520 con 0x7f711c003970 2023-12-19T18:26:50.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.066+0000 7f713259c640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f713415d240 0x7f711c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:50.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.066+0000 7f713359e640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f711c004d00 0x7f711c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:50.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.067+0000 7f7132d9d640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c003970 0x7f711c003d90 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7124009ac0 tx=0x7f711c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d3a86bf6615af0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:50.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.068+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712402c6b0 con 0x7f711c003970 2023-12-19T18:26:50.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.068+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712402c810 con 0x7f711c003970 2023-12-19T18:26:50.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.068+0000 7f7131d9b640 1 -- 172.21.15.27:0/343277552 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712402cba0 con 0x7f711c003970 2023-12-19T18:26:50.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.069+0000 7f71391fb640 1 -- 172.21.15.27:0/343277552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c003970 msgr2=0x7f711c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:50.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.069+0000 7f71391fb640 1 --2- 172.21.15.27:0/343277552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f711c003970 0x7f711c003d90 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f7124009ac0 tx=0x7f711c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:26:50.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.069+0000 7f713359e640 1 -- 172.21.15.27:0/343277552 reap_dead start 2023-12-19T18:26:50.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.069+0000 7f71391fb640 1 -- 172.21.15.27:0/343277552 shutdown_connections 2023-12-19T18:26:50.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.069+0000 7f71391fb640 1 -- 172.21.15.27:0/343277552 >> 172.21.15.27:0/343277552 conn(0x7f71340d31e0 msgr2=0x7f71340d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:50.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.069+0000 7f71391fb640 1 -- 172.21.15.27:0/343277552 shutdown_connections 2023-12-19T18:26:50.069 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.069+0000 7f71391fb640 1 -- 172.21.15.27:0/343277552 wait complete. 2023-12-19T18:26:50.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.071+0000 7f71391fb640 1 Processor -- start 2023-12-19T18:26:50.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.071+0000 7f71391fb640 1 -- start start 2023-12-19T18:26:50.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.072+0000 7f71391fb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f711c003970 0x7f713415a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:50.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.072+0000 7f71391fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71341576a0 0x7f7134158dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:50.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.072+0000 7f71391fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7134159310 0x7f71340cb780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:50.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.072+0000 7f71391fb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7134159de0 con 0x7f71341576a0 2023-12-19T18:26:50.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.072+0000 7f71391fb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7134159820 con 0x7f711c003970 2023-12-19T18:26:50.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.072+0000 7f71391fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7134159b00 con 0x7f7134159310 2023-12-19T18:26:50.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.072+0000 7f713259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71341576a0 0x7f7134158dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:50.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.073+0000 7f713259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71341576a0 0x7f7134158dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36148/0 (socket says 172.21.15.27:36148) 2023-12-19T18:26:50.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.073+0000 7f713259c640 1 -- 172.21.15.27:0/1605566888 learned_addr learned my addr 172.21.15.27:0/1605566888 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.073+0000 7f7132d9d640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f711c003970 0x7f713415a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.073+0000 7f713259c640 1 -- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7134159310 msgr2=0x7f71340cb780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.073+0000 7f713259c640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7134159310 0x7f71340cb780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.073+0000 7f713259c640 1 -- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f711c003970 msgr2=0x7f713415a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.073+0000 7f713259c640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f711c003970 0x7f713415a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.073+0000 7f713259c640 1 -- 172.21.15.27:0/1605566888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71340a9c50 con 0x7f71341576a0 2023-12-19T18:26:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.073+0000 7f7132d9d640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f711c003970 0x7f713415a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.073+0000 7f713259c640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71341576a0 0x7f7134158dd0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f71280109d0 tx=0x7f7128010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:50.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.074+0000 7f711b7fe640 1 -- 172.21.15.27:0/1605566888 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712800ec20 con 0x7f71341576a0 2023-12-19T18:26:50.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.074+0000 7f711b7fe640 1 -- 172.21.15.27:0/1605566888 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712800ed80 con 0x7f71341576a0 2023-12-19T18:26:50.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.074+0000 7f711b7fe640 1 -- 172.21.15.27:0/1605566888 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712801c620 con 0x7f71341576a0 2023-12-19T18:26:50.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.074+0000 7f71391fb640 1 -- 172.21.15.27:0/1605566888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71340a9f30 con 0x7f71341576a0 2023-12-19T18:26:50.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.074+0000 7f71391fb640 1 -- 172.21.15.27:0/1605566888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7134157b20 con 0x7f71341576a0 2023-12-19T18:26:50.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.075+0000 7f71391fb640 1 -- 172.21.15.27:0/1605566888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71340a9ab0 con 0x7f71341576a0 2023-12-19T18:26:50.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.076+0000 7f711b7fe640 1 -- 172.21.15.27:0/1605566888 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f712801c780 con 0x7f71341576a0 2023-12-19T18:26:50.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.077+0000 7f711b7fe640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f712006a460 0x7f712006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:50.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.077+0000 7f711b7fe640 1 -- 172.21.15.27:0/1605566888 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f7128088f60 con 0x7f71341576a0 2023-12-19T18:26:50.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.078+0000 7f7132d9d640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f712006a460 0x7f712006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:50.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.079+0000 7f7132d9d640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f712006a460 0x7f712006c920 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7124009ac0 tx=0x7f7124030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:50.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.082+0000 7f711b7fe640 1 -- 172.21.15.27:0/1605566888 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f71340a9ab0 con 0x7f71341576a0 2023-12-19T18:26:50.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.311+0000 7f71391fb640 1 -- 172.21.15.27:0/1605566888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f71340a9ab0 con 0x7f71341576a0 2023-12-19T18:26:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.312+0000 7f711b7fe640 1 -- 172.21.15.27:0/1605566888 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v77) v1 ==== 53+0+73 (secure 0 0 0) 0x7f71340a9ab0 con 0x7f71341576a0 2023-12-19T18:26:50.312 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:26:50.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.315+0000 7f71391fb640 1 -- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f712006a460 msgr2=0x7f712006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:50.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.315+0000 7f71391fb640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f712006a460 0x7f712006c920 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f7124009ac0 tx=0x7f7124030000 comp rx=0 tx=0).stop 2023-12-19T18:26:50.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.315+0000 7f71391fb640 1 -- 172.21.15.27:0/1605566888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71341576a0 msgr2=0x7f7134158dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:50.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.315+0000 7f71391fb640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71341576a0 0x7f7134158dd0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7f71280109d0 tx=0x7f7128010e90 comp rx=0 tx=0).stop 2023-12-19T18:26:50.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.315+0000 7f71391fb640 1 -- 172.21.15.27:0/1605566888 shutdown_connections 2023-12-19T18:26:50.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.315+0000 7f71391fb640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f712006a460 0x7f712006c920 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:50.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.315+0000 7f71391fb640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7134159310 0x7f71340cb780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:50.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.315+0000 7f71391fb640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71341576a0 0x7f7134158dd0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:50.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.315+0000 7f71391fb640 1 --2- 172.21.15.27:0/1605566888 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f711c003970 0x7f713415a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:50.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.315+0000 7f71391fb640 1 -- 172.21.15.27:0/1605566888 >> 172.21.15.27:0/1605566888 conn(0x7f71340d31e0 msgr2=0x7f71340b79e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:50.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.315+0000 7f71391fb640 1 -- 172.21.15.27:0/1605566888 shutdown_connections 2023-12-19T18:26:50.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:50.315+0000 7f71391fb640 1 -- 172.21.15.27:0/1605566888 wait complete. 2023-12-19T18:26:50.338 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:26:50.338 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:26:50.338 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-19T18:26:50.338 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-19T18:26:50.338 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T18:26:50.338 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:26:50.338 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:26:50.338 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:26:50.338 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:26:50.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:50.339 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-19T18:26:50.399 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:26:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:26:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:26:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:26:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:50.399 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:26:50.399 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 41593sec preferred_lft 41593sec 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:26:50.400 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:26:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:26:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::b892:1cff:fedc:4bed/64 scope link 2023-12-19T18:26:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:50.401 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:26:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether a6:e1:fc:e6:56:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:26:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::a4e1:fcff:fee6:56fa/64 scope link 2023-12-19T18:26:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:50.401 INFO:teuthology.orchestra.run.smithi192.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:26:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T18:26:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::404f:6cff:fe73:9aa2/64 scope link 2023-12-19T18:26:50.401 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:26:50.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:50.403 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:26:50.494 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-19T18:26:50.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:50.498 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-19T18:26:50.556 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-19T18:26:50.557 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:26:50.557 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T18:26:50.557 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T18:26:50.557 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:26:50.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:50.557 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:26:50.557 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:26:50.557 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T18:26:50.557 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T18:26:50.558 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T18:26:50.558 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:26:50.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:50.744 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:26:50.795 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T18:26:50.795 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-19T18:26:50.795 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:26:50.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:50.795 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-19T18:26:50.880 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-19T18:26:50.880 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:26:50.880 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-19T18:26:50.934 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:26:50.934 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:26:50.934 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:26:50.984 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:26:51.072 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:26:51.072 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:26:51.164 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:26:51.167 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:26:51.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:51.168 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:26:51.221 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:26:51.221 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-19T18:26:51.227 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:26:51.306 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:26:51.306 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:26:51.381 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:26:51.384 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:26:51.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:51.384 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:26:51.415 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-19T18:26:51.415+0000 7fe36ce0f480 -1 init, newargv = 0x55bf1df0c5d0 newargc=15 2023-12-19T18:26:51.415 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[92078]: starting ceph client 2023-12-19T18:26:51.424 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[92078]: starting fuse 2023-12-19T18:26:51.427 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:26:51.428 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-12-19T18:26:51.428 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:26:51.428 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-19T18:26:51.428 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-19T18:26:51.428 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-19T18:26:51.428 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-19T18:26:51.428 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:26:51.428 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-19T18:26:51.428 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-19T18:26:51.428 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-19T18:26:51.428 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-19T18:26:51.428 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:26:51.428 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-19T18:26:51.429 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-19T18:26:51.617 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.92078.asok 2023-12-19T18:26:51.800 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.92078.asok 2023-12-19T18:26:51.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:51.800 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.92078.asok status 2023-12-19T18:26:52.040 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-19T18:26:52.040 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-19T18:26:52.040 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:26:52.040 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:26:52.040 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-19T18:26:52.040 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-19T18:26:52.040 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-19T18:26:52.040 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "92078", 2023-12-19T18:26:52.040 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 7000, 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 7000 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2610180659 2023-12-19T18:26:52.041 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 2610180659 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.7000 192.168.0.1:0/2610180659", 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/2610180659", 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 77, 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 187, 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-19T18:26:52.042 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-19T18:26:52.043 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-19T18:26:52.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:52.054 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-19T18:26:52.055 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-19T18:26:52.109 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-19T18:26:52.109 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-19T18:26:52.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:26:52.110 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:26:52.205 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:26:52.446 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.446+0000 7fcff453c640 1 Processor -- start 2023-12-19T18:26:52.447 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.447+0000 7fcff453c640 1 -- start start 2023-12-19T18:26:52.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.448+0000 7fcff453c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcfec0cdf90 0x7fcfec0ce420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:52.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.448+0000 7fcff453c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec0d1e00 0x7fcfec0d2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:52.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.448+0000 7fcff453c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfec0d1970 con 0x7fcfec15d240 2023-12-19T18:26:52.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.448+0000 7fcff453c640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfec0ce960 con 0x7fcfec0cdf90 2023-12-19T18:26:52.448 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.448+0000 7fcff453c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec06a5e0 con 0x7fcfec0d1e00 2023-12-19T18:26:52.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.449+0000 7fcff2ad9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec0d1e00 0x7fcfec0d2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:52.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.449+0000 7fcff2ad9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec0d1e00 0x7fcfec0d2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:54674/0 (socket says 172.21.15.27:54674) 2023-12-19T18:26:52.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.449+0000 7fcff2ad9640 1 -- 172.21.15.27:0/3800220630 learned_addr learned my addr 172.21.15.27:0/3800220630 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:52.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.449+0000 7fcff1ad7640 1 --2- 172.21.15.27:0/3800220630 >> v2:172.21.15.192:3300/0 conn(0x7fcfec0cdf90 0x7fcfec0ce420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:52.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.450+0000 7fcff2ad9640 1 -- 172.21.15.27:0/3800220630 >> v2:172.21.15.192:3300/0 conn(0x7fcfec0cdf90 msgr2=0x7fcfec0ce420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:52.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.450+0000 7fcff2ad9640 1 --2- 172.21.15.27:0/3800220630 >> v2:172.21.15.192:3300/0 conn(0x7fcfec0cdf90 0x7fcfec0ce420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:52.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.450+0000 7fcff2ad9640 1 -- 172.21.15.27:0/3800220630 >> v1:172.21.15.192:6789/0 conn(0x7fcfec15d240 legacy=0x7fcfec1618d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:52.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.450+0000 7fcff2ad9640 1 -- 172.21.15.27:0/3800220630 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec06a880 con 0x7fcfec0d1e00 2023-12-19T18:26:52.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.451+0000 7fcff2ad9640 1 --2- 172.21.15.27:0/3800220630 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec0d1e00 0x7fcfec0d2290 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009e50 tx=0x7fcfe402bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5de3935f1833df03 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:52.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.452+0000 7fcff12d6640 1 -- 172.21.15.27:0/3800220630 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe402cb90 con 0x7fcfec0d1e00 2023-12-19T18:26:52.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.452+0000 7fcff12d6640 1 -- 172.21.15.27:0/3800220630 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe402ccf0 con 0x7fcfec0d1e00 2023-12-19T18:26:52.452 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.452+0000 7fcff12d6640 1 -- 172.21.15.27:0/3800220630 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4034610 con 0x7fcfec0d1e00 2023-12-19T18:26:52.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.452+0000 7fcff453c640 1 -- 172.21.15.27:0/3800220630 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec0d1e00 msgr2=0x7fcfec0d2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:52.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.452+0000 7fcff453c640 1 --2- 172.21.15.27:0/3800220630 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec0d1e00 0x7fcfec0d2290 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009e50 tx=0x7fcfe402bff0 comp rx=0 tx=0).stop 2023-12-19T18:26:52.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.453+0000 7fcff453c640 1 -- 172.21.15.27:0/3800220630 shutdown_connections 2023-12-19T18:26:52.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.453+0000 7fcff453c640 1 --2- 172.21.15.27:0/3800220630 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec0d1e00 0x7fcfec0d2290 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:52.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.453+0000 7fcff453c640 1 --2- 172.21.15.27:0/3800220630 >> v2:172.21.15.192:3300/0 conn(0x7fcfec0cdf90 0x7fcfec0ce420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:52.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.453+0000 7fcff453c640 1 -- 172.21.15.27:0/3800220630 >> 172.21.15.27:0/3800220630 conn(0x7fcfec0d31e0 msgr2=0x7fcfec0d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:52.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.453+0000 7fcff453c640 1 -- 172.21.15.27:0/3800220630 shutdown_connections 2023-12-19T18:26:52.453 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.453+0000 7fcff453c640 1 -- 172.21.15.27:0/3800220630 wait complete. 2023-12-19T18:26:52.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.455+0000 7fcff453c640 1 Processor -- start 2023-12-19T18:26:52.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.455+0000 7fcff453c640 1 -- start start 2023-12-19T18:26:52.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.457+0000 7fcff453c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec0cdf90 0x7fcfec156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:52.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.457+0000 7fcff453c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfec0d1e00 0x7fcfec145410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:52.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.457+0000 7fcff453c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec15d240 0x7fcfec0cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:52.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.457+0000 7fcff453c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec156650 con 0x7fcfec15d240 2023-12-19T18:26:52.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.457+0000 7fcff453c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec0cbb50 con 0x7fcfec0d1e00 2023-12-19T18:26:52.457 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.457+0000 7fcff453c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfec0cbda0 con 0x7fcfec0cdf90 2023-12-19T18:26:52.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.457+0000 7fcff2ad9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec15d240 0x7fcfec0cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:52.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.457+0000 7fcff2ad9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec15d240 0x7fcfec0cce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40562/0 (socket says 172.21.15.27:40562) 2023-12-19T18:26:52.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.457+0000 7fcff2ad9640 1 -- 172.21.15.27:0/593878425 learned_addr learned my addr 172.21.15.27:0/593878425 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:52.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.457+0000 7fcff22d8640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec0cdf90 0x7fcfec156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:52.458 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.457+0000 7fcff1ad7640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfec0d1e00 0x7fcfec145410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.458+0000 7fcff2ad9640 1 -- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec0cdf90 msgr2=0x7fcfec156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.458+0000 7fcff2ad9640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec0cdf90 0x7fcfec156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.458+0000 7fcff2ad9640 1 -- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfec0d1e00 msgr2=0x7fcfec145410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.458+0000 7fcff2ad9640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfec0d1e00 0x7fcfec145410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.458+0000 7fcff2ad9640 1 -- 172.21.15.27:0/593878425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe4009b30 con 0x7fcfec15d240 2023-12-19T18:26:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.458+0000 7fcff22d8640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec0cdf90 0x7fcfec156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:26:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.458+0000 7fcff2ad9640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec15d240 0x7fcfec0cce40 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009e50 tx=0x7fcfe4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.458+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/593878425 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4009510 con 0x7fcfec15d240 2023-12-19T18:26:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.459+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/593878425 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe403e450 con 0x7fcfec15d240 2023-12-19T18:26:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.459+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/593878425 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe403e800 con 0x7fcfec15d240 2023-12-19T18:26:52.459 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.459+0000 7fcff453c640 1 -- 172.21.15.27:0/593878425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfec0cc080 con 0x7fcfec15d240 2023-12-19T18:26:52.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.459+0000 7fcff453c640 1 -- 172.21.15.27:0/593878425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfec0cc640 con 0x7fcfec15d240 2023-12-19T18:26:52.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.460+0000 7fcff453c640 1 -- 172.21.15.27:0/593878425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfec0d1580 con 0x7fcfec15d240 2023-12-19T18:26:52.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.461+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/593878425 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcfe403e960 con 0x7fcfec15d240 2023-12-19T18:26:52.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.462+0000 7fcfe2ffd640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcfd006a430 0x7fcfd006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:52.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.462+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/593878425 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcfe40ab4d0 con 0x7fcfec15d240 2023-12-19T18:26:52.462 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.462+0000 7fcff22d8640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcfd006a430 0x7fcfd006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:52.463 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.464+0000 7fcff22d8640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcfd006a430 0x7fcfd006c8f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8007fd0 tx=0x7fcfe8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:52.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.469+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/593878425 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcfe40769e0 con 0x7fcfec15d240 2023-12-19T18:26:52.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.712+0000 7fcff453c640 1 -- 172.21.15.27:0/593878425 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcfec068e90 con 0x7fcfec15d240 2023-12-19T18:26:52.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.713+0000 7fcfe2ffd640 1 -- 172.21.15.27:0/593878425 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4719 (secure 0 0 0) 0x7fcfe402ce60 con 0x7fcfec15d240 2023-12-19T18:26:52.713 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:26:52.713 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3563146395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3563146395},{"type":"v1","addr":"172.21.15.192:6836","nonce":3563146395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3445095528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3445095528},{"type":"v1","addr":"172.21.15.192:6837","nonce":3445095528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3536033217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3536033217},{"type":"v1","addr":"172.21.15.27:6833","nonce":3536033217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:26:38.483498+0000","modified":"2023-12-19T18:26:42.008767+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2895889621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2895889621},{"type":"v1","addr":"172.21.15.27:6835","nonce":2895889621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-19T18:26:52.713 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 77 2023-12-19T18:26:52.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.715+0000 7fcff453c640 1 -- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcfd006a430 msgr2=0x7fcfd006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:52.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.715+0000 7fcff453c640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcfd006a430 0x7fcfd006c8f0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7fcfe8007fd0 tx=0x7fcfe8009040 comp rx=0 tx=0).stop 2023-12-19T18:26:52.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.715+0000 7fcff453c640 1 -- 172.21.15.27:0/593878425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec15d240 msgr2=0x7fcfec0cce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:52.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.715+0000 7fcff453c640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec15d240 0x7fcfec0cce40 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4009e50 tx=0x7fcfe4004a40 comp rx=0 tx=0).stop 2023-12-19T18:26:52.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.716+0000 7fcff453c640 1 -- 172.21.15.27:0/593878425 shutdown_connections 2023-12-19T18:26:52.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.716+0000 7fcff453c640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcfd006a430 0x7fcfd006c8f0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:52.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.716+0000 7fcff453c640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfec15d240 0x7fcfec0cce40 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:52.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.716+0000 7fcff453c640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcfec0d1e00 0x7fcfec145410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:52.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.716+0000 7fcff453c640 1 --2- 172.21.15.27:0/593878425 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcfec0cdf90 0x7fcfec156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:52.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.716+0000 7fcff453c640 1 -- 172.21.15.27:0/593878425 >> 172.21.15.27:0/593878425 conn(0x7fcfec0d31e0 msgr2=0x7fcfec15e610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:52.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.716+0000 7fcff453c640 1 -- 172.21.15.27:0/593878425 shutdown_connections 2023-12-19T18:26:52.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:52.716+0000 7fcff453c640 1 -- 172.21.15.27:0/593878425 wait complete. 2023-12-19T18:26:52.735 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-12-19T18:26:52.986 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":8631,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.483301494999999,"mdsmap_epoch":77,"osdmap_epoch":187,"osdmap_epoch_barrier":185,"uptime":20.260392869} 2023-12-19T18:26:52.987 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 8631, "mdsmap_epoch": 77, "osdmap_epoch": 187, "osdmap_epoch_barrier": 185, "rank_uptime": 14.483301495, "state": "up:active", "uptime": 20.260392869, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:26:52.987 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-12-19T18:26:53.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.209+0000 7fbf531d0640 1 Processor -- start 2023-12-19T18:26:53.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.209+0000 7fbf531d0640 1 -- start start 2023-12-19T18:26:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.211+0000 7fbf531d0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf4c158790 0x7fbf4c0ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.211+0000 7fbf531d0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbf4c0adc50 0x7fbf4c15b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.211+0000 7fbf531d0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4c15c030 con 0x7fbf4c15b810 2023-12-19T18:26:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.211+0000 7fbf531d0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4c0c99d0 con 0x7fbf4c0adc50 2023-12-19T18:26:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.211+0000 7fbf531d0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c158b70 con 0x7fbf4c158790 2023-12-19T18:26:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.211+0000 7fbf529cf640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbf4c15b810 0x7fbf4c15e730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:39862/0 (socket says 172.21.15.27:39862) 2023-12-19T18:26:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.211+0000 7fbf529cf640 1 -- 172.21.15.27:0/4262795532 learned_addr learned my addr 172.21.15.27:0/4262795532 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:26:53.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.211+0000 7fbf519cd640 1 --2- 172.21.15.27:0/4262795532 >> v2:172.21.15.27:3300/0 conn(0x7fbf4c0adc50 0x7fbf4c15b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:53.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.211+0000 7fbf521ce640 1 --2- 172.21.15.27:0/4262795532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf4c158790 0x7fbf4c0ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:53.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.212+0000 7fbf511cc640 1 -- 172.21.15.27:0/4262795532 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2726963797 0 0) 0x7fbf4c15c030 con 0x7fbf4c15b810 2023-12-19T18:26:53.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.212+0000 7fbf511cc640 1 -- 172.21.15.27:0/4262795532 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf34003520 con 0x7fbf4c15b810 2023-12-19T18:26:53.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.212+0000 7fbf519cd640 1 -- 172.21.15.27:0/4262795532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf4c158790 msgr2=0x7fbf4c0ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:26:53.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.212+0000 7fbf519cd640 1 --2- 172.21.15.27:0/4262795532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf4c158790 0x7fbf4c0ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:53.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.212+0000 7fbf519cd640 1 -- 172.21.15.27:0/4262795532 >> v1:172.21.15.27:6789/0 conn(0x7fbf4c15b810 legacy=0x7fbf4c15e730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:53.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.212+0000 7fbf519cd640 1 -- 172.21.15.27:0/4262795532 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0c9030 con 0x7fbf4c0adc50 2023-12-19T18:26:53.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.213+0000 7fbf519cd640 1 --2- 172.21.15.27:0/4262795532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf4c0adc50 0x7fbf4c15b2d0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009d90 tx=0x7fbf4002beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c68ed90e7e84ae5 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:53.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.213+0000 7fbf511cc640 1 -- 172.21.15.27:0/4262795532 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4002cc60 con 0x7fbf4c0adc50 2023-12-19T18:26:53.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.213+0000 7fbf511cc640 1 -- 172.21.15.27:0/4262795532 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4002cdc0 con 0x7fbf4c0adc50 2023-12-19T18:26:53.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.213+0000 7fbf511cc640 1 -- 172.21.15.27:0/4262795532 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf40034640 con 0x7fbf4c0adc50 2023-12-19T18:26:53.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.214+0000 7fbf531d0640 1 -- 172.21.15.27:0/4262795532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf4c0adc50 msgr2=0x7fbf4c15b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:53.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.214+0000 7fbf531d0640 1 --2- 172.21.15.27:0/4262795532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf4c0adc50 0x7fbf4c15b2d0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fbf40009d90 tx=0x7fbf4002beb0 comp rx=0 tx=0).stop 2023-12-19T18:26:53.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.214+0000 7fbf531d0640 1 -- 172.21.15.27:0/4262795532 shutdown_connections 2023-12-19T18:26:53.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.214+0000 7fbf531d0640 1 --2- 172.21.15.27:0/4262795532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf4c0adc50 0x7fbf4c15b2d0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:53.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.214+0000 7fbf531d0640 1 --2- 172.21.15.27:0/4262795532 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf4c158790 0x7fbf4c0ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:53.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.214+0000 7fbf531d0640 1 -- 172.21.15.27:0/4262795532 >> 172.21.15.27:0/4262795532 conn(0x7fbf4c0b05b0 msgr2=0x7fbf4c0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:53.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.214+0000 7fbf531d0640 1 -- 172.21.15.27:0/4262795532 shutdown_connections 2023-12-19T18:26:53.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.214+0000 7fbf531d0640 1 -- 172.21.15.27:0/4262795532 wait complete. 2023-12-19T18:26:53.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.216+0000 7fbf531d0640 1 Processor -- start 2023-12-19T18:26:53.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.216+0000 7fbf531d0640 1 -- start start 2023-12-19T18:26:53.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.217+0000 7fbf531d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf4c0adc50 0x7fbf4c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:53.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.217+0000 7fbf531d0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf4c158790 0x7fbf4c06c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:53.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.217+0000 7fbf531d0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf4c15b810 0x7fbf4c06cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:53.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.217+0000 7fbf531d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0c73a0 con 0x7fbf4c0adc50 2023-12-19T18:26:53.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.217+0000 7fbf531d0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0c7680 con 0x7fbf4c15b810 2023-12-19T18:26:53.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.217+0000 7fbf531d0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0c7960 con 0x7fbf4c158790 2023-12-19T18:26:53.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.217+0000 7fbf521ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf4c0adc50 0x7fbf4c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:53.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.217+0000 7fbf521ce640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf4c0adc50 0x7fbf4c0c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:40578/0 (socket says 172.21.15.27:40578) 2023-12-19T18:26:53.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.217+0000 7fbf521ce640 1 -- 172.21.15.27:0/684452262 learned_addr learned my addr 172.21.15.27:0/684452262 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:26:53.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.217+0000 7fbf519cd640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf4c158790 0x7fbf4c06c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:53.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf529cf640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf4c15b810 0x7fbf4c06cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:53.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf521ce640 1 -- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf4c158790 msgr2=0x7fbf4c06c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:53.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf521ce640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf4c158790 0x7fbf4c06c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:53.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf521ce640 1 -- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf4c15b810 msgr2=0x7fbf4c06cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:53.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf521ce640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf4c15b810 0x7fbf4c06cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:53.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf521ce640 1 -- 172.21.15.27:0/684452262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf40009a70 con 0x7fbf4c0adc50 2023-12-19T18:26:53.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf519cd640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf4c158790 0x7fbf4c06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:53.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf529cf640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf4c15b810 0x7fbf4c06cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:26:53.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf521ce640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf4c0adc50 0x7fbf4c0c6dc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00d800 tx=0x7fbf3c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:53.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf4affd640 1 -- 172.21.15.27:0/684452262 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00b6c0 con 0x7fbf4c0adc50 2023-12-19T18:26:53.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf4affd640 1 -- 172.21.15.27:0/684452262 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf3c00b820 con 0x7fbf4c0adc50 2023-12-19T18:26:53.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf531d0640 1 -- 172.21.15.27:0/684452262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c0c7c40 con 0x7fbf4c0adc50 2023-12-19T18:26:53.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf4affd640 1 -- 172.21.15.27:0/684452262 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c00bbb0 con 0x7fbf4c0adc50 2023-12-19T18:26:53.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.218+0000 7fbf531d0640 1 -- 172.21.15.27:0/684452262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4c14c960 con 0x7fbf4c0adc50 2023-12-19T18:26:53.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.220+0000 7fbf531d0640 1 -- 172.21.15.27:0/684452262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4c04e840 con 0x7fbf4c0adc50 2023-12-19T18:26:53.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.221+0000 7fbf4affd640 1 -- 172.21.15.27:0/684452262 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbf3c00bd10 con 0x7fbf4c0adc50 2023-12-19T18:26:53.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.222+0000 7fbf4affd640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbf3806a430 0x7fbf3806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:26:53.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.222+0000 7fbf4affd640 1 -- 172.21.15.27:0/684452262 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbf3c088f40 con 0x7fbf4c0adc50 2023-12-19T18:26:53.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.222+0000 7fbf519cd640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbf3806a430 0x7fbf3806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:26:53.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.223+0000 7fbf519cd640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbf3806a430 0x7fbf3806c8f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbf40036000 tx=0x7fbf40030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:26:53.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.226+0000 7fbf4affd640 1 -- 172.21.15.27:0/684452262 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbf3c054450 con 0x7fbf4c0adc50 2023-12-19T18:26:53.420 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.420+0000 7fbf531d0640 1 -- 172.21.15.27:0/684452262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7fbf4c0c4e20 con 0x7fbf4c0adc50 2023-12-19T18:26:53.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.824+0000 7fbf4affd640 1 -- 172.21.15.27:0/684452262 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v188) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbf3c057a70 con 0x7fbf4c0adc50 2023-12-19T18:26:53.826 INFO:teuthology.orchestra.run.smithi027.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-12-19T18:26:53.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.828+0000 7fbf531d0640 1 -- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbf3806a430 msgr2=0x7fbf3806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:53.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.828+0000 7fbf531d0640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbf3806a430 0x7fbf3806c8f0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7fbf40036000 tx=0x7fbf40030000 comp rx=0 tx=0).stop 2023-12-19T18:26:53.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.828+0000 7fbf531d0640 1 -- 172.21.15.27:0/684452262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf4c0adc50 msgr2=0x7fbf4c0c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:26:53.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.828+0000 7fbf531d0640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf4c0adc50 0x7fbf4c0c6dc0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00d800 tx=0x7fbf3c00dcc0 comp rx=0 tx=0).stop 2023-12-19T18:26:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.828+0000 7fbf531d0640 1 -- 172.21.15.27:0/684452262 shutdown_connections 2023-12-19T18:26:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.828+0000 7fbf531d0640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbf3806a430 0x7fbf3806c8f0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.828+0000 7fbf531d0640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbf4c15b810 0x7fbf4c06cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.828+0000 7fbf531d0640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbf4c158790 0x7fbf4c06c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.828+0000 7fbf531d0640 1 --2- 172.21.15.27:0/684452262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf4c0adc50 0x7fbf4c0c6dc0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:26:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.828+0000 7fbf531d0640 1 -- 172.21.15.27:0/684452262 >> 172.21.15.27:0/684452262 conn(0x7fbf4c0b05b0 msgr2=0x7fbf4c0afe00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:26:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.829+0000 7fbf531d0640 1 -- 172.21.15.27:0/684452262 shutdown_connections 2023-12-19T18:26:53.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:26:53.829+0000 7fbf531d0640 1 -- 172.21.15.27:0/684452262 wait complete. 2023-12-19T18:26:53.850 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-12-19T18:26:54.097 INFO:teuthology.orchestra.run.smithi027.stdout:{"osd_mon_report_interval":"5"} 2023-12-19T18:26:54.098 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-12-19T18:26:54.098 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok --format=json config get mon_tick_interval 2023-12-19T18:26:54.323 INFO:teuthology.orchestra.run.smithi027.stdout:{"mon_tick_interval":"5"} 2023-12-19T18:26:54.323 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mon_tick_interval": "5" } 2023-12-19T18:26:54.324 INFO:tasks.cephfs.test_full:pool capacity 33554432, 32MB should be enough to fill it 2023-12-19T18:26:54.324 DEBUG:teuthology.orchestra.run.smithi027:> adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:26:54.324 DEBUG:teuthology.orchestra.run.smithi027:> import time 2023-12-19T18:26:54.324 DEBUG:teuthology.orchestra.run.smithi027:> import datetime 2023-12-19T18:26:54.324 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:26:54.324 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:26:54.324 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:26:54.324 DEBUG:teuthology.orchestra.run.smithi027:> # Write some buffered data through before going full, all should be well 2023-12-19T18:26:54.324 DEBUG:teuthology.orchestra.run.smithi027:> print("writing some data through which we expect to succeed") 2023-12-19T18:26:54.324 DEBUG:teuthology.orchestra.run.smithi027:> bytes = 0 2023-12-19T18:26:54.325 DEBUG:teuthology.orchestra.run.smithi027:> f = os.open("/home/ubuntu/cephtest/mnt.0/full_test_file", os.O_WRONLY | os.O_CREAT) 2023-12-19T18:26:54.325 DEBUG:teuthology.orchestra.run.smithi027:> os.setxattr("/home/ubuntu/cephtest/mnt.0/full_test_file", '"'"'ceph.file.layout'"'"', b'"'"'stripe_unit=1048576 stripe_count=1 object_size=1048576'"'"') 2023-12-19T18:26:54.325 DEBUG:teuthology.orchestra.run.smithi027:> bytes += os.write(f, b'"'"'a'"'"' * 4096) 2023-12-19T18:26:54.325 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(f) 2023-12-19T18:26:54.325 DEBUG:teuthology.orchestra.run.smithi027:> print("fsync'"'"'ed data successfully, will now attempt to fill fs") 2023-12-19T18:26:54.325 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:26:54.325 DEBUG:teuthology.orchestra.run.smithi027:> # Okay, now we'"'"'re going to fill up the filesystem, and then keep 2023-12-19T18:26:54.326 DEBUG:teuthology.orchestra.run.smithi027:> # writing until we see an error from fsync. As long as we'"'"'re doing 2023-12-19T18:26:54.326 DEBUG:teuthology.orchestra.run.smithi027:> # buffered IO, the error should always only appear from fsync and not 2023-12-19T18:26:54.326 DEBUG:teuthology.orchestra.run.smithi027:> # from write 2023-12-19T18:26:54.326 DEBUG:teuthology.orchestra.run.smithi027:> full = False 2023-12-19T18:26:54.326 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:26:54.326 DEBUG:teuthology.orchestra.run.smithi027:> for n in range(0, int(32 * 1.1)): 2023-12-19T18:26:54.326 DEBUG:teuthology.orchestra.run.smithi027:> try: 2023-12-19T18:26:54.327 DEBUG:teuthology.orchestra.run.smithi027:> bytes += os.write(f, b'"'"'x'"'"' * 1024 * 1024) 2023-12-19T18:26:54.327 DEBUG:teuthology.orchestra.run.smithi027:> print("wrote bytes via buffered write, moving on to fsync") 2023-12-19T18:26:54.327 DEBUG:teuthology.orchestra.run.smithi027:> except OSError as e: 2023-12-19T18:26:54.327 DEBUG:teuthology.orchestra.run.smithi027:> if True: 2023-12-19T18:26:54.327 DEBUG:teuthology.orchestra.run.smithi027:> print("Unexpected error %s from write() instead of fsync()" % e) 2023-12-19T18:26:54.327 DEBUG:teuthology.orchestra.run.smithi027:> raise 2023-12-19T18:26:54.328 DEBUG:teuthology.orchestra.run.smithi027:> else: 2023-12-19T18:26:54.328 DEBUG:teuthology.orchestra.run.smithi027:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-12-19T18:26:54.328 DEBUG:teuthology.orchestra.run.smithi027:> full = True 2023-12-19T18:26:54.328 DEBUG:teuthology.orchestra.run.smithi027:> break 2023-12-19T18:26:54.329 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:26:54.329 DEBUG:teuthology.orchestra.run.smithi027:> try: 2023-12-19T18:26:54.329 DEBUG:teuthology.orchestra.run.smithi027:> os.fsync(f) 2023-12-19T18:26:54.329 DEBUG:teuthology.orchestra.run.smithi027:> print("fsync'"'"'ed successfully") 2023-12-19T18:26:54.329 DEBUG:teuthology.orchestra.run.smithi027:> except OSError as e: 2023-12-19T18:26:54.330 DEBUG:teuthology.orchestra.run.smithi027:> print("Reached fullness after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-12-19T18:26:54.330 DEBUG:teuthology.orchestra.run.smithi027:> full = True 2023-12-19T18:26:54.330 DEBUG:teuthology.orchestra.run.smithi027:> break 2023-12-19T18:26:54.330 DEBUG:teuthology.orchestra.run.smithi027:> else: 2023-12-19T18:26:54.330 DEBUG:teuthology.orchestra.run.smithi027:> print("Not full yet after %.2f MB" % (bytes / (1024.0 * 1024.0))) 2023-12-19T18:26:54.330 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:26:54.331 DEBUG:teuthology.orchestra.run.smithi027:> if n > 32 * 0.9: 2023-12-19T18:26:54.331 DEBUG:teuthology.orchestra.run.smithi027:> # Be cautious in the last region where we expect to hit 2023-12-19T18:26:54.331 DEBUG:teuthology.orchestra.run.smithi027:> # the full condition, so that we don'"'"'t overshoot too dramatically 2023-12-19T18:26:54.331 DEBUG:teuthology.orchestra.run.smithi027:> print("sleeping a bit as we'"'"'ve exceeded 90% of our expected full ratio") 2023-12-19T18:26:54.331 DEBUG:teuthology.orchestra.run.smithi027:> time.sleep(15.0) 2023-12-19T18:26:54.331 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:26:54.331 DEBUG:teuthology.orchestra.run.smithi027:> if not full: 2023-12-19T18:26:54.332 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Failed to reach fullness after writing %d bytes" % bytes) 2023-12-19T18:26:54.332 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:26:54.332 DEBUG:teuthology.orchestra.run.smithi027:> # close() should not raise an error because we already caught it in 2023-12-19T18:26:54.332 DEBUG:teuthology.orchestra.run.smithi027:> # fsync. There shouldn'"'"'t have been any more writeback errors 2023-12-19T18:26:54.332 DEBUG:teuthology.orchestra.run.smithi027:> # since then because all IOs got cancelled on the full flag. 2023-12-19T18:26:54.332 DEBUG:teuthology.orchestra.run.smithi027:> print("calling close") 2023-12-19T18:26:54.333 DEBUG:teuthology.orchestra.run.smithi027:> os.close(f) 2023-12-19T18:26:54.333 DEBUG:teuthology.orchestra.run.smithi027:> print("close() did not raise error") 2023-12-19T18:26:54.333 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:26:54.333 DEBUG:teuthology.orchestra.run.smithi027:> os.unlink("/home/ubuntu/cephtest/mnt.0/full_test_file") 2023-12-19T18:26:54.333 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:27:18.190 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:27:18.201 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:27:43.408 INFO:teuthology.orchestra.run.smithi027.stdout:writing some data through which we expect to succeed 2023-12-19T18:27:43.409 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed data successfully, will now attempt to fill fs 2023-12-19T18:27:43.409 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.409 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.409 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 1.00 MB 2023-12-19T18:27:43.409 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.409 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.410 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 2.00 MB 2023-12-19T18:27:43.410 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.410 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.410 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 3.00 MB 2023-12-19T18:27:43.410 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.411 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.411 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 4.00 MB 2023-12-19T18:27:43.411 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.411 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.411 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 5.00 MB 2023-12-19T18:27:43.411 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.411 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.412 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 6.00 MB 2023-12-19T18:27:43.412 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.412 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.412 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 7.00 MB 2023-12-19T18:27:43.412 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.412 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 8.00 MB 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 9.00 MB 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 10.00 MB 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 11.00 MB 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 12.00 MB 2023-12-19T18:27:43.413 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 13.00 MB 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 14.00 MB 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 15.00 MB 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 16.00 MB 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 17.00 MB 2023-12-19T18:27:43.414 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 18.00 MB 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 19.00 MB 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 20.00 MB 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 21.00 MB 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 22.00 MB 2023-12-19T18:27:43.415 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 23.00 MB 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 24.00 MB 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 25.00 MB 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 26.00 MB 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 27.00 MB 2023-12-19T18:27:43.416 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 28.00 MB 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 29.00 MB 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 30.00 MB 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 31.00 MB 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.417 INFO:teuthology.orchestra.run.smithi027.stdout:fsync'ed successfully 2023-12-19T18:27:43.418 INFO:teuthology.orchestra.run.smithi027.stdout:Not full yet after 32.00 MB 2023-12-19T18:27:43.418 INFO:teuthology.orchestra.run.smithi027.stdout:sleeping a bit as we've exceeded 90% of our expected full ratio 2023-12-19T18:27:43.418 INFO:teuthology.orchestra.run.smithi027.stdout:wrote bytes via buffered write, moving on to fsync 2023-12-19T18:27:43.418 INFO:teuthology.orchestra.run.smithi027.stdout:Reached fullness after 33.00 MB 2023-12-19T18:27:43.418 INFO:teuthology.orchestra.run.smithi027.stdout:calling close 2023-12-19T18:27:43.418 INFO:teuthology.orchestra.run.smithi027.stdout:close() did not raise error 2023-12-19T18:27:43.558 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:27:43.558 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:27:43.700 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:27:43.831 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:27:43.831 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:27:43.832 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:27:43.897 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:27:44.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.133+0000 7f22df59e640 1 Processor -- start 2023-12-19T18:27:44.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.134+0000 7f22df59e640 1 -- start start 2023-12-19T18:27:44.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.135+0000 7f22df59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e00cb300 0x7f22e00cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:44.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.135+0000 7f22df59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f22e00c8f90 0x7f22e0160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:44.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.135+0000 7f22df59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22e0160e40 0x7f22e00ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:44.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.135+0000 7f22df59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e00c4480 con 0x7f22e0160e40 2023-12-19T18:27:44.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.135+0000 7f22df59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22e01468c0 con 0x7f22e00c8f90 2023-12-19T18:27:44.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.135+0000 7f22df59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e01612d0 con 0x7f22e00cb300 2023-12-19T18:27:44.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.135+0000 7f22ded9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22e0160e40 0x7f22e00ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:44.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.135+0000 7f22ded9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f22e0160e40 0x7f22e00ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42736/0 (socket says 172.21.15.27:42736) 2023-12-19T18:27:44.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.135+0000 7f22ded9d640 1 -- 172.21.15.27:0/1260640560 learned_addr learned my addr 172.21.15.27:0/1260640560 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:44.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.136+0000 7f22d5d9b640 1 --2- 172.21.15.27:0/1260640560 >> v2:172.21.15.192:3300/0 conn(0x7f22e00c8f90 0x7f22e0160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:44.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.136+0000 7f22de59c640 1 --2- 172.21.15.27:0/1260640560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e00cb300 0x7f22e00cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:44.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.136+0000 7f22ded9d640 1 -- 172.21.15.27:0/1260640560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e00cb300 msgr2=0x7f22e00cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:44.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.136+0000 7f22ded9d640 1 --2- 172.21.15.27:0/1260640560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e00cb300 0x7f22e00cb6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.136+0000 7f22ded9d640 1 -- 172.21.15.27:0/1260640560 >> v2:172.21.15.192:3300/0 conn(0x7f22e00c8f90 msgr2=0x7f22e0160900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:44.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.136+0000 7f22ded9d640 1 --2- 172.21.15.27:0/1260640560 >> v2:172.21.15.192:3300/0 conn(0x7f22e00c8f90 0x7f22e0160900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.136+0000 7f22ded9d640 1 -- 172.21.15.27:0/1260640560 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22e00cbc20 con 0x7f22e0160e40 2023-12-19T18:27:44.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.137+0000 7f22ded9d640 1 --2- 172.21.15.27:0/1260640560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e0160e40 0x7f22e00ac8c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f22e006d540 tx=0x7f22d802bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5626f84c58544a95 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.138+0000 7f22ddd9b640 1 -- 172.21.15.27:0/1260640560 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d802ca70 con 0x7f22e0160e40 2023-12-19T18:27:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.138+0000 7f22ddd9b640 1 -- 172.21.15.27:0/1260640560 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d802cbd0 con 0x7f22e0160e40 2023-12-19T18:27:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.138+0000 7f22ddd9b640 1 -- 172.21.15.27:0/1260640560 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d80343f0 con 0x7f22e0160e40 2023-12-19T18:27:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.138+0000 7f22df59e640 1 -- 172.21.15.27:0/1260640560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e0160e40 msgr2=0x7f22e00ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.138+0000 7f22df59e640 1 --2- 172.21.15.27:0/1260640560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e0160e40 0x7f22e00ac8c0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7f22e006d540 tx=0x7f22d802bed0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.138+0000 7f22df59e640 1 -- 172.21.15.27:0/1260640560 shutdown_connections 2023-12-19T18:27:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.138+0000 7f22df59e640 1 --2- 172.21.15.27:0/1260640560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e0160e40 0x7f22e00ac8c0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.138+0000 7f22df59e640 1 --2- 172.21.15.27:0/1260640560 >> v2:172.21.15.192:3300/0 conn(0x7f22e00c8f90 0x7f22e0160900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.138+0000 7f22df59e640 1 --2- 172.21.15.27:0/1260640560 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e00cb300 0x7f22e00cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.138+0000 7f22df59e640 1 -- 172.21.15.27:0/1260640560 >> 172.21.15.27:0/1260640560 conn(0x7f22e0069c70 msgr2=0x7f22e006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.139+0000 7f22df59e640 1 -- 172.21.15.27:0/1260640560 shutdown_connections 2023-12-19T18:27:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.139+0000 7f22df59e640 1 -- 172.21.15.27:0/1260640560 wait complete. 2023-12-19T18:27:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.140+0000 7f22df59e640 1 Processor -- start 2023-12-19T18:27:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.141+0000 7f22df59e640 1 -- start start 2023-12-19T18:27:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22df59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00c8f90 0x7f22e01444c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22df59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e00cb300 0x7f22e015d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22df59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e0160e40 0x7f22e00b2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22df59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e01607d0 con 0x7f22e00c8f90 2023-12-19T18:27:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22df59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22e00c5a10 con 0x7f22e0160e40 2023-12-19T18:27:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22df59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22e0144a00 con 0x7f22e00cb300 2023-12-19T18:27:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22de59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00c8f90 0x7f22e01444c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22de59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00c8f90 0x7f22e01444c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42744/0 (socket says 172.21.15.27:42744) 2023-12-19T18:27:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22de59c640 1 -- 172.21.15.27:0/4246209080 learned_addr learned my addr 172.21.15.27:0/4246209080 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22d5d9b640 1 --2- 172.21.15.27:0/4246209080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e00cb300 0x7f22e015d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22de59c640 1 -- 172.21.15.27:0/4246209080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e00cb300 msgr2=0x7f22e015d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22de59c640 1 --2- 172.21.15.27:0/4246209080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e00cb300 0x7f22e015d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22de59c640 1 -- 172.21.15.27:0/4246209080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e0160e40 msgr2=0x7f22e00b2ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22de59c640 1 --2- 172.21.15.27:0/4246209080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e0160e40 0x7f22e00b2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.142+0000 7f22de59c640 1 -- 172.21.15.27:0/4246209080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d8009b30 con 0x7f22e00c8f90 2023-12-19T18:27:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.143+0000 7f22de59c640 1 --2- 172.21.15.27:0/4246209080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00c8f90 0x7f22e01444c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f22d000d800 tx=0x7f22d000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.143+0000 7f22d77fe640 1 -- 172.21.15.27:0/4246209080 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000b6c0 con 0x7f22e00c8f90 2023-12-19T18:27:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.143+0000 7f22d77fe640 1 -- 172.21.15.27:0/4246209080 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d000b820 con 0x7f22e00c8f90 2023-12-19T18:27:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.143+0000 7f22d77fe640 1 -- 172.21.15.27:0/4246209080 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000bbb0 con 0x7f22e00c8f90 2023-12-19T18:27:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.143+0000 7f22df59e640 1 -- 172.21.15.27:0/4246209080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22e00c5c60 con 0x7f22e00c8f90 2023-12-19T18:27:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.143+0000 7f22df59e640 1 -- 172.21.15.27:0/4246209080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22e00c6220 con 0x7f22e00c8f90 2023-12-19T18:27:44.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.145+0000 7f22df59e640 1 -- 172.21.15.27:0/4246209080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22e004e840 con 0x7f22e00c8f90 2023-12-19T18:27:44.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.146+0000 7f22d77fe640 1 -- 172.21.15.27:0/4246209080 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f22d000bd10 con 0x7f22e00c8f90 2023-12-19T18:27:44.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.146+0000 7f22d77fe640 1 --2- 172.21.15.27:0/4246209080 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22c006a460 0x7f22c006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:44.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.147+0000 7f22d77fe640 1 -- 172.21.15.27:0/4246209080 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f22d0088e30 con 0x7f22e00c8f90 2023-12-19T18:27:44.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.147+0000 7f22d77fe640 1 -- 172.21.15.27:0/4246209080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7f22c0072050 con 0x7f22e00c8f90 2023-12-19T18:27:44.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.147+0000 7f22d5d9b640 1 --2- 172.21.15.27:0/4246209080 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22c006a460 0x7f22c006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:44.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.148+0000 7f22d5d9b640 1 --2- 172.21.15.27:0/4246209080 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22c006a460 0x7f22c006c920 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f22c8006fd0 tx=0x7f22c8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:44.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.151+0000 7f22d77fe640 1 -- 172.21.15.27:0/4246209080 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f22d00543d0 con 0x7f22e00c8f90 2023-12-19T18:27:44.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.349+0000 7f22df59e640 1 -- 172.21.15.27:0/4246209080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22e00adc50 con 0x7f22e00c8f90 2023-12-19T18:27:44.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.352+0000 7f22d77fe640 1 -- 172.21.15.27:0/4246209080 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+16236 (secure 0 0 0) 0x7f22d00579f0 con 0x7f22e00c8f90 2023-12-19T18:27:44.352 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:27:44.352 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":189,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:27:31.809427+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:26:36.428093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T18:26:37.523993+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1e","mappings":[{"from":2,"to":5}]},{"pgid":"21.33","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:27:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.354+0000 7f22df59e640 1 -- 172.21.15.27:0/4246209080 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22c006a460 msgr2=0x7f22c006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.354+0000 7f22df59e640 1 --2- 172.21.15.27:0/4246209080 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22c006a460 0x7f22c006c920 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f22c8006fd0 tx=0x7f22c8008040 comp rx=0 tx=0).stop 2023-12-19T18:27:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.354+0000 7f22df59e640 1 -- 172.21.15.27:0/4246209080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00c8f90 msgr2=0x7f22e01444c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.354+0000 7f22df59e640 1 --2- 172.21.15.27:0/4246209080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00c8f90 0x7f22e01444c0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f22d000d800 tx=0x7f22d000dcc0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.355+0000 7f22df59e640 1 -- 172.21.15.27:0/4246209080 shutdown_connections 2023-12-19T18:27:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.355+0000 7f22df59e640 1 --2- 172.21.15.27:0/4246209080 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f22c006a460 0x7f22c006c920 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.355+0000 7f22df59e640 1 --2- 172.21.15.27:0/4246209080 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f22e0160e40 0x7f22e00b2ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.355+0000 7f22df59e640 1 --2- 172.21.15.27:0/4246209080 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f22e00cb300 0x7f22e015d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.355+0000 7f22df59e640 1 --2- 172.21.15.27:0/4246209080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22e00c8f90 0x7f22e01444c0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.355+0000 7f22df59e640 1 -- 172.21.15.27:0/4246209080 >> 172.21.15.27:0/4246209080 conn(0x7f22e0069c70 msgr2=0x7f22e006b580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.355+0000 7f22df59e640 1 -- 172.21.15.27:0/4246209080 shutdown_connections 2023-12-19T18:27:44.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.355+0000 7f22df59e640 1 -- 172.21.15.27:0/4246209080 wait complete. 2023-12-19T18:27:44.377 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:26:36.428093+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:26:37.523993+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:27:44.378 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:27:44.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.614+0000 7efdb7769640 1 Processor -- start 2023-12-19T18:27:44.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.614+0000 7efdb7769640 1 -- start start 2023-12-19T18:27:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.615+0000 7efdb7769640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efdb015d370 0x7efdb00b2340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.615+0000 7efdb7769640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdb00b5160 0x7efdb00b62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.616+0000 7efdb7769640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb015d750 con 0x7efdb00b55f0 2023-12-19T18:27:44.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.616+0000 7efdb7769640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb00b2880 con 0x7efdb015d370 2023-12-19T18:27:44.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.616+0000 7efdb7769640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb00d1750 con 0x7efdb00b5160 2023-12-19T18:27:44.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.616+0000 7efdb4d04640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7efdb00b55f0 0x7efdb00b4a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53568/0 (socket says 172.21.15.27:53568) 2023-12-19T18:27:44.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.616+0000 7efdb4d04640 1 -- 172.21.15.27:0/2146964102 learned_addr learned my addr 172.21.15.27:0/2146964102 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:44.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.616+0000 7efdb5505640 1 --2- 172.21.15.27:0/2146964102 >> v2:172.21.15.192:3300/0 conn(0x7efdb015d370 0x7efdb00b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:44.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.616+0000 7efdb5d06640 1 --2- 172.21.15.27:0/2146964102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdb00b5160 0x7efdb00b62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:44.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.617+0000 7efda3fff640 1 -- 172.21.15.27:0/2146964102 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 453887452 0 0) 0x7efdb015d750 con 0x7efdb00b55f0 2023-12-19T18:27:44.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.617+0000 7efda3fff640 1 -- 172.21.15.27:0/2146964102 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd9c003520 con 0x7efdb00b55f0 2023-12-19T18:27:44.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.617+0000 7efdb5505640 1 -- 172.21.15.27:0/2146964102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdb00b5160 msgr2=0x7efdb00b62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:44.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.617+0000 7efdb5505640 1 --2- 172.21.15.27:0/2146964102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdb00b5160 0x7efdb00b62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.617+0000 7efdb5505640 1 -- 172.21.15.27:0/2146964102 >> v1:172.21.15.27:6789/0 conn(0x7efdb00b55f0 legacy=0x7efdb00b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:44.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.617+0000 7efdb5505640 1 -- 172.21.15.27:0/2146964102 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb00d1a30 con 0x7efdb015d370 2023-12-19T18:27:44.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.618+0000 7efdb5505640 1 --2- 172.21.15.27:0/2146964102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdb015d370 0x7efdb00b2340 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efdac0097d0 tx=0x7efdac02b930 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a72e38787a44b502 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:44.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.619+0000 7efda3fff640 1 -- 172.21.15.27:0/2146964102 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac031890 con 0x7efdb015d370 2023-12-19T18:27:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.619+0000 7efda3fff640 1 -- 172.21.15.27:0/2146964102 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdac0319f0 con 0x7efdb015d370 2023-12-19T18:27:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.619+0000 7efda3fff640 1 -- 172.21.15.27:0/2146964102 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac031d80 con 0x7efdb015d370 2023-12-19T18:27:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.619+0000 7efdb7769640 1 -- 172.21.15.27:0/2146964102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdb015d370 msgr2=0x7efdb00b2340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:44.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.619+0000 7efdb7769640 1 --2- 172.21.15.27:0/2146964102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdb015d370 0x7efdb00b2340 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efdac0097d0 tx=0x7efdac02b930 comp rx=0 tx=0).stop 2023-12-19T18:27:44.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.620+0000 7efdb7769640 1 -- 172.21.15.27:0/2146964102 shutdown_connections 2023-12-19T18:27:44.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.620+0000 7efdb7769640 1 --2- 172.21.15.27:0/2146964102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdb00b5160 0x7efdb00b62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.620+0000 7efdb7769640 1 --2- 172.21.15.27:0/2146964102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdb015d370 0x7efdb00b2340 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.620+0000 7efdb7769640 1 -- 172.21.15.27:0/2146964102 >> 172.21.15.27:0/2146964102 conn(0x7efdb006a480 msgr2=0x7efdb006a860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:44.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.620+0000 7efdb7769640 1 -- 172.21.15.27:0/2146964102 shutdown_connections 2023-12-19T18:27:44.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.620+0000 7efdb7769640 1 -- 172.21.15.27:0/2146964102 wait complete. 2023-12-19T18:27:44.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.622+0000 7efdb7769640 1 Processor -- start 2023-12-19T18:27:44.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.622+0000 7efdb7769640 1 -- start start 2023-12-19T18:27:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.623+0000 7efdb7769640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdb00b5160 0x7efdb0148100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.623+0000 7efdb7769640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb00b55f0 0x7efdb00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.623+0000 7efdb7769640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdb015d370 0x7efdb00c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.623+0000 7efdb7769640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb00c87a0 con 0x7efdb00b55f0 2023-12-19T18:27:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.623+0000 7efdb7769640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb0148640 con 0x7efdb00b5160 2023-12-19T18:27:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.623+0000 7efdb7769640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb0149090 con 0x7efdb015d370 2023-12-19T18:27:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.623+0000 7efdb4d04640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb00b55f0 0x7efdb00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.623+0000 7efdb4d04640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb00b55f0 0x7efdb00c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42754/0 (socket says 172.21.15.27:42754) 2023-12-19T18:27:44.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.623+0000 7efdb4d04640 1 -- 172.21.15.27:0/1500662323 learned_addr learned my addr 172.21.15.27:0/1500662323 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:44.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.623+0000 7efdb5505640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdb00b5160 0x7efdb0148100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:44.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.624+0000 7efdb4d04640 1 -- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdb015d370 msgr2=0x7efdb00c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:44.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.624+0000 7efdb5d06640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdb015d370 0x7efdb00c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:44.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.624+0000 7efdb4d04640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdb015d370 0x7efdb00c7ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.624+0000 7efdb4d04640 1 -- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdb00b5160 msgr2=0x7efdb0148100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:44.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.624+0000 7efdb4d04640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdb00b5160 0x7efdb0148100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.624+0000 7efdb4d04640 1 -- 172.21.15.27:0/1500662323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdac009400 con 0x7efdb00b55f0 2023-12-19T18:27:44.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.624+0000 7efdb5505640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdb00b5160 0x7efdb0148100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.624+0000 7efdb5d06640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdb015d370 0x7efdb00c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.624+0000 7efdb4d04640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb00b55f0 0x7efdb00c7760 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7efda400c3e0 tx=0x7efda400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.624+0000 7efda1ffb640 1 -- 172.21.15.27:0/1500662323 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda4005ce0 con 0x7efdb00b55f0 2023-12-19T18:27:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.624+0000 7efda1ffb640 1 -- 172.21.15.27:0/1500662323 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda4005e40 con 0x7efdb00b55f0 2023-12-19T18:27:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.624+0000 7efdb7769640 1 -- 172.21.15.27:0/1500662323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb0148a80 con 0x7efdb00b55f0 2023-12-19T18:27:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.624+0000 7efda1ffb640 1 -- 172.21.15.27:0/1500662323 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda40072d0 con 0x7efdb00b55f0 2023-12-19T18:27:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.625+0000 7efdb7769640 1 -- 172.21.15.27:0/1500662323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb0148d20 con 0x7efdb00b55f0 2023-12-19T18:27:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.626+0000 7efdb7769640 1 -- 172.21.15.27:0/1500662323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb01488e0 con 0x7efdb00b55f0 2023-12-19T18:27:44.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.627+0000 7efda1ffb640 1 -- 172.21.15.27:0/1500662323 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efda40074f0 con 0x7efdb00b55f0 2023-12-19T18:27:44.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.628+0000 7efda1ffb640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd9406a430 0x7efd9406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:44.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.628+0000 7efda1ffb640 1 -- 172.21.15.27:0/1500662323 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7efda4087e20 con 0x7efdb00b55f0 2023-12-19T18:27:44.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.628+0000 7efda1ffb640 1 -- 172.21.15.27:0/1500662323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7efd94072020 con 0x7efdb00b55f0 2023-12-19T18:27:44.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.628+0000 7efdb5505640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd9406a430 0x7efd9406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:44.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.630+0000 7efdb5505640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd9406a430 0x7efd9406c8f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efdac036000 tx=0x7efdac008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:44.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.632+0000 7efda1ffb640 1 -- 172.21.15.27:0/1500662323 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efdb01488e0 con 0x7efdb00b55f0 2023-12-19T18:27:44.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.828+0000 7efdb7769640 1 -- 172.21.15.27:0/1500662323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efdb01488e0 con 0x7efdb00b55f0 2023-12-19T18:27:44.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.829+0000 7efda1ffb640 1 -- 172.21.15.27:0/1500662323 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v189) v1 ==== 93+0+10 (secure 0 0 0) 0x7efdb01488e0 con 0x7efdb00b55f0 2023-12-19T18:27:44.829 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:27:44.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.832+0000 7efdb7769640 1 -- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd9406a430 msgr2=0x7efd9406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:44.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.832+0000 7efdb7769640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd9406a430 0x7efd9406c8f0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7efdac036000 tx=0x7efdac008030 comp rx=0 tx=0).stop 2023-12-19T18:27:44.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.832+0000 7efdb7769640 1 -- 172.21.15.27:0/1500662323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb00b55f0 msgr2=0x7efdb00c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:44.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.832+0000 7efdb7769640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb00b55f0 0x7efdb00c7760 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7efda400c3e0 tx=0x7efda400c8a0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.832+0000 7efdb7769640 1 -- 172.21.15.27:0/1500662323 shutdown_connections 2023-12-19T18:27:44.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.832+0000 7efdb7769640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd9406a430 0x7efd9406c8f0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.832+0000 7efdb7769640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efdb015d370 0x7efdb00c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.832+0000 7efdb7769640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efdb00b55f0 0x7efdb00c7760 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.832+0000 7efdb7769640 1 --2- 172.21.15.27:0/1500662323 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efdb00b5160 0x7efdb0148100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:44.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.832+0000 7efdb7769640 1 -- 172.21.15.27:0/1500662323 >> 172.21.15.27:0/1500662323 conn(0x7efdb006a480 msgr2=0x7efdb015c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:44.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.832+0000 7efdb7769640 1 -- 172.21.15.27:0/1500662323 shutdown_connections 2023-12-19T18:27:44.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:44.832+0000 7efdb7769640 1 -- 172.21.15.27:0/1500662323 wait complete. 2023-12-19T18:27:44.853 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:27:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.081+0000 7fccdaf8e640 1 Processor -- start 2023-12-19T18:27:45.081 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.081+0000 7fccdaf8e640 1 -- start start 2023-12-19T18:27:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.083+0000 7fccdaf8e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccd415d120 0x7fccd40cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.083+0000 7fccdaf8e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd40d3850 con 0x7fccd40d1310 2023-12-19T18:27:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.083+0000 7fccdaf8e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccd40d0b10 con 0x7fccd40ce950 2023-12-19T18:27:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.083+0000 7fccdaf8e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd415d500 con 0x7fccd415d120 2023-12-19T18:27:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.083+0000 7fccd978b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fccd40d1310 0x7fccd40b2970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53578/0 (socket says 172.21.15.27:53578) 2023-12-19T18:27:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.083+0000 7fccd978b640 1 -- 172.21.15.27:0/928562912 learned_addr learned my addr 172.21.15.27:0/928562912 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.083+0000 7fccd9f8c640 1 --2- 172.21.15.27:0/928562912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccd415d120 0x7fccd40cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1947285088 0 0) 0x7fccd40d3850 con 0x7fccd40d1310 2023-12-19T18:27:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccbc003520 con 0x7fccd40d1310 2023-12-19T18:27:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1168297454 0 0) 0x7fccbc003520 con 0x7fccd40d1310 2023-12-19T18:27:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccd415d120 msgr2=0x7fccd40cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 --2- 172.21.15.27:0/928562912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccd415d120 0x7fccd40cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 >> v1:172.21.15.192:6789/0 conn(0x7fccd40ce950 legacy=0x7fccd40b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd40d1040 con 0x7fccd40d1310 2023-12-19T18:27:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fccc8001670 con 0x7fccd40d1310 2023-12-19T18:27:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 >> v1:172.21.15.27:6789/0 conn(0x7fccd40d1310 legacy=0x7fccd40b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 --2- 172.21.15.27:0/928562912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccd415d120 0x7fccbc0039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 --2- 172.21.15.27:0/928562912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccbc003f70 0x7fccbc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 --2- 172.21.15.27:0/928562912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccbc004d00 0x7fccbc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd415d500 con 0x7fccbc003f70 2023-12-19T18:27:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc00bac0 con 0x7fccbc004d00 2023-12-19T18:27:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.084+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccbc00bc80 con 0x7fccd415d120 2023-12-19T18:27:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.085+0000 7fccd978b640 1 --2- 172.21.15.27:0/928562912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccbc003f70 0x7fccbc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.085+0000 7fccd9f8c640 1 --2- 172.21.15.27:0/928562912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccd415d120 0x7fccbc0039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.085+0000 7fccda78d640 1 --2- 172.21.15.27:0/928562912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccbc004d00 0x7fccbc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.085+0000 7fccd978b640 1 -- 172.21.15.27:0/928562912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccd415d120 msgr2=0x7fccbc0039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.085+0000 7fccd978b640 1 --2- 172.21.15.27:0/928562912 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccd415d120 0x7fccbc0039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.085+0000 7fccd978b640 1 -- 172.21.15.27:0/928562912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccbc004d00 msgr2=0x7fccbc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:45.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.085+0000 7fccd978b640 1 --2- 172.21.15.27:0/928562912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccbc004d00 0x7fccbc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.085+0000 7fccd978b640 1 -- 172.21.15.27:0/928562912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccbc003520 con 0x7fccbc003f70 2023-12-19T18:27:45.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.085+0000 7fccda78d640 1 --2- 172.21.15.27:0/928562912 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccbc004d00 0x7fccbc005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:45.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.086+0000 7fccd978b640 1 --2- 172.21.15.27:0/928562912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccbc003f70 0x7fccbc0047c0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fccc800c0e0 tx=0x7fccbc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebc0c88dcc5e255c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:45.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.087+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8005ad0 con 0x7fccbc003f70 2023-12-19T18:27:45.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.087+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc8005c50 con 0x7fccbc003f70 2023-12-19T18:27:45.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.087+0000 7fccd8f8a640 1 -- 172.21.15.27:0/928562912 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc8007070 con 0x7fccbc003f70 2023-12-19T18:27:45.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.087+0000 7fccdaf8e640 1 -- 172.21.15.27:0/928562912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccbc003f70 msgr2=0x7fccbc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:45.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.087+0000 7fccdaf8e640 1 --2- 172.21.15.27:0/928562912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccbc003f70 0x7fccbc0047c0 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7fccc800c0e0 tx=0x7fccbc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.087+0000 7fccda78d640 1 -- 172.21.15.27:0/928562912 reap_dead start 2023-12-19T18:27:45.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.087+0000 7fccdaf8e640 1 -- 172.21.15.27:0/928562912 shutdown_connections 2023-12-19T18:27:45.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.087+0000 7fccdaf8e640 1 -- 172.21.15.27:0/928562912 >> 172.21.15.27:0/928562912 conn(0x7fccd40d2d30 msgr2=0x7fccd40d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:45.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.087+0000 7fccdaf8e640 1 -- 172.21.15.27:0/928562912 shutdown_connections 2023-12-19T18:27:45.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.087+0000 7fccdaf8e640 1 -- 172.21.15.27:0/928562912 wait complete. 2023-12-19T18:27:45.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.089+0000 7fccdaf8e640 1 Processor -- start 2023-12-19T18:27:45.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.090+0000 7fccdaf8e640 1 -- start start 2023-12-19T18:27:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.090+0000 7fccdaf8e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccbc003f70 0x7fccd414f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccdaf8e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd415d640 0x7fccd40b32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccdaf8e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccd415dac0 0x7fccd40b3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccdaf8e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd40b6670 con 0x7fccd415dac0 2023-12-19T18:27:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccdaf8e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccd415ec30 con 0x7fccd415d640 2023-12-19T18:27:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccdaf8e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccd415e020 con 0x7fccbc003f70 2023-12-19T18:27:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccda78d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccd415dac0 0x7fccd40b3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccda78d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccd415dac0 0x7fccd40b3830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42774/0 (socket says 172.21.15.27:42774) 2023-12-19T18:27:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccda78d640 1 -- 172.21.15.27:0/1597867571 learned_addr learned my addr 172.21.15.27:0/1597867571 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccd978b640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd415d640 0x7fccd40b32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccda78d640 1 -- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccbc003f70 msgr2=0x7fccd414f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccda78d640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccbc003f70 0x7fccd414f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccda78d640 1 -- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd415d640 msgr2=0x7fccd40b32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccda78d640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd415d640 0x7fccd40b32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccda78d640 1 -- 172.21.15.27:0/1597867571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd415e300 con 0x7fccd415dac0 2023-12-19T18:27:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.091+0000 7fccd978b640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd415d640 0x7fccd40b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:45.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.092+0000 7fccda78d640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccd415dac0 0x7fccd40b3830 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fccd408d040 tx=0x7fcccc00d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:45.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.092+0000 7fccd27fc640 1 -- 172.21.15.27:0/1597867571 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc00b070 con 0x7fccd415dac0 2023-12-19T18:27:45.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.092+0000 7fccd27fc640 1 -- 172.21.15.27:0/1597867571 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcccc0054a0 con 0x7fccd415dac0 2023-12-19T18:27:45.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.092+0000 7fccd27fc640 1 -- 172.21.15.27:0/1597867571 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcccc007270 con 0x7fccd415dac0 2023-12-19T18:27:45.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.092+0000 7fccdaf8e640 1 -- 172.21.15.27:0/1597867571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccd415e5e0 con 0x7fccd415dac0 2023-12-19T18:27:45.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.092+0000 7fccdaf8e640 1 -- 172.21.15.27:0/1597867571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccd40c8d90 con 0x7fccd415dac0 2023-12-19T18:27:45.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.093+0000 7fccdaf8e640 1 -- 172.21.15.27:0/1597867571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccd404e840 con 0x7fccd415dac0 2023-12-19T18:27:45.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.094+0000 7fccd27fc640 1 -- 172.21.15.27:0/1597867571 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcccc0073d0 con 0x7fccd415dac0 2023-12-19T18:27:45.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.095+0000 7fccd27fc640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fccc006a460 0x7fccc006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.095+0000 7fccd27fc640 1 -- 172.21.15.27:0/1597867571 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fcccc0881a0 con 0x7fccd415dac0 2023-12-19T18:27:45.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.096+0000 7fccd27fc640 1 -- 172.21.15.27:0/1597867571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7fccc0072050 con 0x7fccd415dac0 2023-12-19T18:27:45.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.096+0000 7fccd9f8c640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fccc006a460 0x7fccc006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:45.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.097+0000 7fccd9f8c640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fccc006a460 0x7fccc006c920 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009fd0 tx=0x7fccc4009750 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:45.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.100+0000 7fccd27fc640 1 -- 172.21.15.27:0/1597867571 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcccc05be60 con 0x7fccd415dac0 2023-12-19T18:27:45.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.299+0000 7fccdaf8e640 1 -- 172.21.15.27:0/1597867571 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fccd40c8f20 con 0x7fccd415dac0 2023-12-19T18:27:45.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.300+0000 7fccd27fc640 1 -- 172.21.15.27:0/1597867571 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v189) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcccc05be60 con 0x7fccd415dac0 2023-12-19T18:27:45.300 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:27:45.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.303+0000 7fccdaf8e640 1 -- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fccc006a460 msgr2=0x7fccc006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:45.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.303+0000 7fccdaf8e640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fccc006a460 0x7fccc006c920 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fccc4009fd0 tx=0x7fccc4009750 comp rx=0 tx=0).stop 2023-12-19T18:27:45.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.303+0000 7fccdaf8e640 1 -- 172.21.15.27:0/1597867571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccd415dac0 msgr2=0x7fccd40b3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:45.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.303+0000 7fccdaf8e640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccd415dac0 0x7fccd40b3830 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7fccd408d040 tx=0x7fcccc00d890 comp rx=0 tx=0).stop 2023-12-19T18:27:45.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.303+0000 7fccdaf8e640 1 -- 172.21.15.27:0/1597867571 shutdown_connections 2023-12-19T18:27:45.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.303+0000 7fccdaf8e640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fccc006a460 0x7fccc006c920 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.303+0000 7fccdaf8e640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccd415dac0 0x7fccd40b3830 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.303+0000 7fccdaf8e640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fccd415d640 0x7fccd40b32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.303+0000 7fccdaf8e640 1 --2- 172.21.15.27:0/1597867571 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fccbc003f70 0x7fccd414f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.303+0000 7fccdaf8e640 1 -- 172.21.15.27:0/1597867571 >> 172.21.15.27:0/1597867571 conn(0x7fccd40d2d30 msgr2=0x7fccd40d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.303+0000 7fccdaf8e640 1 -- 172.21.15.27:0/1597867571 shutdown_connections 2023-12-19T18:27:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.304+0000 7fccdaf8e640 1 -- 172.21.15.27:0/1597867571 wait complete. 2023-12-19T18:27:45.323 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:27:45.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.561+0000 7f012f59e640 1 Processor -- start 2023-12-19T18:27:45.561 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.561+0000 7f012f59e640 1 -- start start 2023-12-19T18:27:45.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.563+0000 7f012f59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f013015d190 0x7f01300cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.563+0000 7f012f59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f013006af90 0x7f01300cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.563+0000 7f012f59e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f013006bca0 con 0x7f01300cbc00 2023-12-19T18:27:45.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.563+0000 7f012f59e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01300d15a0 con 0x7f013015d190 2023-12-19T18:27:45.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.563+0000 7f012f59e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01300d1880 con 0x7f013006af90 2023-12-19T18:27:45.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.563+0000 7f012e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f013015d190 0x7f01300cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:45.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.563+0000 7f012e59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f013015d190 0x7f01300cd3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42776/0 (socket says 172.21.15.27:42776) 2023-12-19T18:27:45.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.563+0000 7f012e59c640 1 -- 172.21.15.27:0/3201958972 learned_addr learned my addr 172.21.15.27:0/3201958972 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:45.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.564+0000 7f012dd9b640 1 --2- 172.21.15.27:0/3201958972 >> v2:172.21.15.192:3300/0 conn(0x7f013006af90 0x7f01300cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:45.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.564+0000 7f012e59c640 1 -- 172.21.15.27:0/3201958972 >> v2:172.21.15.192:3300/0 conn(0x7f013006af90 msgr2=0x7f01300cd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:45.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.564+0000 7f012e59c640 1 --2- 172.21.15.27:0/3201958972 >> v2:172.21.15.192:3300/0 conn(0x7f013006af90 0x7f01300cd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.564+0000 7f012e59c640 1 -- 172.21.15.27:0/3201958972 >> v1:172.21.15.192:6789/0 conn(0x7f01300cbc00 legacy=0x7f01300cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:45.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.564+0000 7f012e59c640 1 -- 172.21.15.27:0/3201958972 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f013006b930 con 0x7f013015d190 2023-12-19T18:27:45.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.565+0000 7f012e59c640 1 --2- 172.21.15.27:0/3201958972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f013015d190 0x7f01300cd3b0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f0120009e50 tx=0x7f012002be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acd265801ec49576 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:45.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.565+0000 7f012d59a640 1 -- 172.21.15.27:0/3201958972 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012002c9f0 con 0x7f013015d190 2023-12-19T18:27:45.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.565+0000 7f012d59a640 1 -- 172.21.15.27:0/3201958972 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012002cb50 con 0x7f013015d190 2023-12-19T18:27:45.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.565+0000 7f012d59a640 1 -- 172.21.15.27:0/3201958972 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01200343f0 con 0x7f013015d190 2023-12-19T18:27:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.566+0000 7f012f59e640 1 -- 172.21.15.27:0/3201958972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f013015d190 msgr2=0x7f01300cd3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.566+0000 7f012f59e640 1 --2- 172.21.15.27:0/3201958972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f013015d190 0x7f01300cd3b0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f0120009e50 tx=0x7f012002be50 comp rx=0 tx=0).stop 2023-12-19T18:27:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.566+0000 7f012f59e640 1 -- 172.21.15.27:0/3201958972 shutdown_connections 2023-12-19T18:27:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.566+0000 7f012f59e640 1 --2- 172.21.15.27:0/3201958972 >> v2:172.21.15.192:3300/0 conn(0x7f013006af90 0x7f01300cd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.566+0000 7f012f59e640 1 --2- 172.21.15.27:0/3201958972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f013015d190 0x7f01300cd3b0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.566+0000 7f012f59e640 1 -- 172.21.15.27:0/3201958972 >> 172.21.15.27:0/3201958972 conn(0x7f0130099ee0 msgr2=0x7f01300d0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.566+0000 7f012f59e640 1 -- 172.21.15.27:0/3201958972 shutdown_connections 2023-12-19T18:27:45.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.566+0000 7f012f59e640 1 -- 172.21.15.27:0/3201958972 wait complete. 2023-12-19T18:27:45.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.568+0000 7f012f59e640 1 Processor -- start 2023-12-19T18:27:45.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.568+0000 7f012f59e640 1 -- start start 2023-12-19T18:27:45.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.569+0000 7f012f59e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f013006af90 0x7f013015b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.569+0000 7f012f59e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01300cbc00 0x7f01301f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.569+0000 7f012f59e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f013015d190 0x7f01300c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.569+0000 7f012f59e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013015bb00 con 0x7f013006af90 2023-12-19T18:27:45.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.569+0000 7f012f59e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f013009a9f0 con 0x7f01300cbc00 2023-12-19T18:27:45.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.569+0000 7f012f59e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f013009acd0 con 0x7f013015d190 2023-12-19T18:27:45.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.569+0000 7f012e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f013006af90 0x7f013015b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:45.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.569+0000 7f012e59c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f013006af90 0x7f013015b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42786/0 (socket says 172.21.15.27:42786) 2023-12-19T18:27:45.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.569+0000 7f012e59c640 1 -- 172.21.15.27:0/915947764 learned_addr learned my addr 172.21.15.27:0/915947764 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.570+0000 7f012dd9b640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01300cbc00 0x7f01301f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.570+0000 7f012e59c640 1 -- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f013015d190 msgr2=0x7f01300c6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.570+0000 7f012ed9d640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f013015d190 0x7f01300c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.570+0000 7f012e59c640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f013015d190 0x7f01300c6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.570+0000 7f012e59c640 1 -- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01300cbc00 msgr2=0x7f01301f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.570+0000 7f012e59c640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01300cbc00 0x7f01301f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.570+0000 7f012e59c640 1 -- 172.21.15.27:0/915947764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0120009b30 con 0x7f013006af90 2023-12-19T18:27:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.570+0000 7f012dd9b640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01300cbc00 0x7f01301f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.570+0000 7f012ed9d640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f013015d190 0x7f01300c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:45.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.570+0000 7f012e59c640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f013006af90 0x7f013015b5c0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f0120009e50 tx=0x7f0120004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:45.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.571+0000 7f0126ffd640 1 -- 172.21.15.27:0/915947764 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120009510 con 0x7f013006af90 2023-12-19T18:27:45.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.571+0000 7f0126ffd640 1 -- 172.21.15.27:0/915947764 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f012003e450 con 0x7f013006af90 2023-12-19T18:27:45.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.571+0000 7f0126ffd640 1 -- 172.21.15.27:0/915947764 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012003e800 con 0x7f013006af90 2023-12-19T18:27:45.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.571+0000 7f012f59e640 1 -- 172.21.15.27:0/915947764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f013009afb0 con 0x7f013006af90 2023-12-19T18:27:45.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.571+0000 7f012f59e640 1 -- 172.21.15.27:0/915947764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f013009b570 con 0x7f013006af90 2023-12-19T18:27:45.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.572+0000 7f012f59e640 1 -- 172.21.15.27:0/915947764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01300d0930 con 0x7f013006af90 2023-12-19T18:27:45.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.573+0000 7f0126ffd640 1 -- 172.21.15.27:0/915947764 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f012003e960 con 0x7f013006af90 2023-12-19T18:27:45.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.574+0000 7f0126ffd640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f011006a430 0x7f011006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:45.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.574+0000 7f0126ffd640 1 -- 172.21.15.27:0/915947764 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f01200ab770 con 0x7f013006af90 2023-12-19T18:27:45.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.574+0000 7f0126ffd640 1 -- 172.21.15.27:0/915947764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7f0110072020 con 0x7f013006af90 2023-12-19T18:27:45.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.575+0000 7f012dd9b640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f011006a430 0x7f011006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:45.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.576+0000 7f012dd9b640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f011006a430 0x7f011006c8f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f01300cd940 tx=0x7f0118008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:45.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.578+0000 7f0126ffd640 1 -- 172.21.15.27:0/915947764 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0120076d10 con 0x7f013006af90 2023-12-19T18:27:45.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.777+0000 7f012f59e640 1 -- 172.21.15.27:0/915947764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f013004e840 con 0x7f013006af90 2023-12-19T18:27:45.778 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.778+0000 7f0126ffd640 1 -- 172.21.15.27:0/915947764 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v189) v1 ==== 100+0+11 (secure 0 0 0) 0x7f012007a330 con 0x7f013006af90 2023-12-19T18:27:45.778 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:27:45.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.781+0000 7f012f59e640 1 -- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f011006a430 msgr2=0x7f011006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:45.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.781+0000 7f012f59e640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f011006a430 0x7f011006c8f0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f01300cd940 tx=0x7f0118008040 comp rx=0 tx=0).stop 2023-12-19T18:27:45.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.781+0000 7f012f59e640 1 -- 172.21.15.27:0/915947764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f013006af90 msgr2=0x7f013015b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:45.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.781+0000 7f012f59e640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f013006af90 0x7f013015b5c0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f0120009e50 tx=0x7f0120004a40 comp rx=0 tx=0).stop 2023-12-19T18:27:45.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.781+0000 7f012f59e640 1 -- 172.21.15.27:0/915947764 shutdown_connections 2023-12-19T18:27:45.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.781+0000 7f012f59e640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f011006a430 0x7f011006c8f0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.781+0000 7f012f59e640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f013015d190 0x7f01300c6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.781+0000 7f012f59e640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f01300cbc00 0x7f01301f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.781+0000 7f012f59e640 1 --2- 172.21.15.27:0/915947764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f013006af90 0x7f013015b5c0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:45.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.781+0000 7f012f59e640 1 -- 172.21.15.27:0/915947764 >> 172.21.15.27:0/915947764 conn(0x7f0130099ee0 msgr2=0x7f013006e540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:45.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.781+0000 7f012f59e640 1 -- 172.21.15.27:0/915947764 shutdown_connections 2023-12-19T18:27:45.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:45.781+0000 7f012f59e640 1 -- 172.21.15.27:0/915947764 wait complete. 2023-12-19T18:27:45.802 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:27:46.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.037+0000 7fbd03c04640 1 Processor -- start 2023-12-19T18:27:46.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.037+0000 7fbd03c04640 1 -- start start 2023-12-19T18:27:46.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.039+0000 7fbd03c04640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fbcfc15d1d0 0x7fbcfc0ccd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.039+0000 7fbd03c04640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcfc0ac8d0 0x7fbcfc0acd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.039+0000 7fbd03c04640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcfc0ce7c0 con 0x7fbcfc071140 2023-12-19T18:27:46.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.039+0000 7fbd03c04640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcfc070150 con 0x7fbcfc15d1d0 2023-12-19T18:27:46.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.039+0000 7fbd03c04640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc15d5b0 con 0x7fbcfc0ac8d0 2023-12-19T18:27:46.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.039+0000 7fbd0119f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbcfc071140 0x7fbcfc0cd250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53580/0 (socket says 172.21.15.27:53580) 2023-12-19T18:27:46.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.039+0000 7fbd0119f640 1 -- 172.21.15.27:0/3637504535 learned_addr learned my addr 172.21.15.27:0/3637504535 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:46.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.040+0000 7fbd019a0640 1 --2- 172.21.15.27:0/3637504535 >> v2:172.21.15.192:3300/0 conn(0x7fbcfc15d1d0 0x7fbcfc0ccd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.040+0000 7fbd021a1640 1 --2- 172.21.15.27:0/3637504535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcfc0ac8d0 0x7fbcfc0acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.040+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075181301 0 0) 0x7fbcfc0ce7c0 con 0x7fbcfc071140 2023-12-19T18:27:46.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.040+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbce8003520 con 0x7fbcfc071140 2023-12-19T18:27:46.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.040+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3107839942 0 0) 0x7fbce8003520 con 0x7fbcfc071140 2023-12-19T18:27:46.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.040+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcfc0ac8d0 msgr2=0x7fbcfc0acd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:46.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0099e640 1 --2- 172.21.15.27:0/3637504535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcfc0ac8d0 0x7fbcfc0acd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 >> v2:172.21.15.192:3300/0 conn(0x7fbcfc15d1d0 msgr2=0x7fbcfc0ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:46.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0099e640 1 --2- 172.21.15.27:0/3637504535 >> v2:172.21.15.192:3300/0 conn(0x7fbcfc15d1d0 0x7fbcfc0ccd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc070360 con 0x7fbcfc071140 2023-12-19T18:27:46.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fbcec001670 con 0x7fbcfc071140 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 >> v1:172.21.15.27:6789/0 conn(0x7fbcfc071140 legacy=0x7fbcfc0cd250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0099e640 1 --2- 172.21.15.27:0/3637504535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbce8003970 0x7fbce8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0099e640 1 --2- 172.21.15.27:0/3637504535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbce80047c0 0x7fbce8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0099e640 1 --2- 172.21.15.27:0/3637504535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcfc0ac8d0 0x7fbce8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc070150 con 0x7fbce80047c0 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc15d5b0 con 0x7fbce8003970 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbce800b9d0 con 0x7fbcfc0ac8d0 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0119f640 1 --2- 172.21.15.27:0/3637504535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbce80047c0 0x7fbce8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd019a0640 1 --2- 172.21.15.27:0/3637504535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbce8003970 0x7fbce8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd021a1640 1 --2- 172.21.15.27:0/3637504535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcfc0ac8d0 0x7fbce8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.041+0000 7fbd0119f640 1 -- 172.21.15.27:0/3637504535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcfc0ac8d0 msgr2=0x7fbce8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.042+0000 7fbd0119f640 1 --2- 172.21.15.27:0/3637504535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcfc0ac8d0 0x7fbce8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.042+0000 7fbd0119f640 1 -- 172.21.15.27:0/3637504535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbce8003970 msgr2=0x7fbce8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:46.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.042+0000 7fbd0119f640 1 --2- 172.21.15.27:0/3637504535 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbce8003970 0x7fbce8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.042+0000 7fbd0119f640 1 -- 172.21.15.27:0/3637504535 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce8003520 con 0x7fbce80047c0 2023-12-19T18:27:46.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.042+0000 7fbd021a1640 1 --2- 172.21.15.27:0/3637504535 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbcfc0ac8d0 0x7fbce8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:46.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.043+0000 7fbd0119f640 1 --2- 172.21.15.27:0/3637504535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbce80047c0 0x7fbce8004c40 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00c140 tx=0x7fbce80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5badbdc3e8558b4b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:46.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.043+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec005ad0 con 0x7fbce80047c0 2023-12-19T18:27:46.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.043+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcec005c50 con 0x7fbce80047c0 2023-12-19T18:27:46.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.043+0000 7fbd0099e640 1 -- 172.21.15.27:0/3637504535 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcec02c810 con 0x7fbce80047c0 2023-12-19T18:27:46.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.044+0000 7fbd03c04640 1 -- 172.21.15.27:0/3637504535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbce80047c0 msgr2=0x7fbce8004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:46.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.044+0000 7fbd03c04640 1 --2- 172.21.15.27:0/3637504535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbce80047c0 0x7fbce8004c40 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00c140 tx=0x7fbce80057f0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.044+0000 7fbd021a1640 1 -- 172.21.15.27:0/3637504535 reap_dead start 2023-12-19T18:27:46.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.044+0000 7fbd03c04640 1 -- 172.21.15.27:0/3637504535 shutdown_connections 2023-12-19T18:27:46.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.044+0000 7fbd03c04640 1 -- 172.21.15.27:0/3637504535 >> 172.21.15.27:0/3637504535 conn(0x7fbcfc0ceb50 msgr2=0x7fbcfc0cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:46.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.044+0000 7fbd03c04640 1 -- 172.21.15.27:0/3637504535 shutdown_connections 2023-12-19T18:27:46.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.044+0000 7fbd03c04640 1 -- 172.21.15.27:0/3637504535 wait complete. 2023-12-19T18:27:46.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.046+0000 7fbd03c04640 1 Processor -- start 2023-12-19T18:27:46.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.046+0000 7fbd03c04640 1 -- start start 2023-12-19T18:27:46.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.047+0000 7fbd03c04640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbce80047c0 0x7fbcfc15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.047+0000 7fbd03c04640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcfc1525e0 0x7fbcfc15fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.047+0000 7fbd03c04640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcfc160410 0x7fbcfc15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.047+0000 7fbd03c04640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc15b530 con 0x7fbcfc160410 2023-12-19T18:27:46.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.047+0000 7fbd03c04640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc0c4bb0 con 0x7fbcfc1525e0 2023-12-19T18:27:46.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.047+0000 7fbd03c04640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcfc0c48d0 con 0x7fbce80047c0 2023-12-19T18:27:46.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.047+0000 7fbd021a1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcfc160410 0x7fbcfc15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.048+0000 7fbd021a1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcfc160410 0x7fbcfc15dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42804/0 (socket says 172.21.15.27:42804) 2023-12-19T18:27:46.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.048+0000 7fbd021a1640 1 -- 172.21.15.27:0/139338119 learned_addr learned my addr 172.21.15.27:0/139338119 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:46.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.048+0000 7fbd0119f640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcfc1525e0 0x7fbcfc15fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.048+0000 7fbd019a0640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbce80047c0 0x7fbcfc15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.048+0000 7fbd021a1640 1 -- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbce80047c0 msgr2=0x7fbcfc15aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:46.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.048+0000 7fbd021a1640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbce80047c0 0x7fbcfc15aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.048+0000 7fbd021a1640 1 -- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcfc1525e0 msgr2=0x7fbcfc15fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:46.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.048+0000 7fbd021a1640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcfc1525e0 0x7fbcfc15fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.048+0000 7fbd021a1640 1 -- 172.21.15.27:0/139338119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc15a510 con 0x7fbcfc160410 2023-12-19T18:27:46.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.048+0000 7fbd021a1640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcfc160410 0x7fbcfc15dbe0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0ca840 tx=0x7fbcf400f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:46.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.049+0000 7fbcf27fc640 1 -- 172.21.15.27:0/139338119 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4017400 con 0x7fbcfc160410 2023-12-19T18:27:46.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.049+0000 7fbcf27fc640 1 -- 172.21.15.27:0/139338119 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcf4011040 con 0x7fbcfc160410 2023-12-19T18:27:46.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.049+0000 7fbd03c04640 1 -- 172.21.15.27:0/139338119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcfc15a7f0 con 0x7fbcfc160410 2023-12-19T18:27:46.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.049+0000 7fbcf27fc640 1 -- 172.21.15.27:0/139338119 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcf4017780 con 0x7fbcfc160410 2023-12-19T18:27:46.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.049+0000 7fbd03c04640 1 -- 172.21.15.27:0/139338119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcfc15e120 con 0x7fbcfc160410 2023-12-19T18:27:46.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.050+0000 7fbd03c04640 1 -- 172.21.15.27:0/139338119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcfc068e90 con 0x7fbcfc160410 2023-12-19T18:27:46.051 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.051+0000 7fbcf27fc640 1 -- 172.21.15.27:0/139338119 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbcf40203b0 con 0x7fbcfc160410 2023-12-19T18:27:46.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.052+0000 7fbcf27fc640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbce006a430 0x7fbce006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.052+0000 7fbcf27fc640 1 -- 172.21.15.27:0/139338119 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbcf4088c80 con 0x7fbcfc160410 2023-12-19T18:27:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.053+0000 7fbcf27fc640 1 -- 172.21.15.27:0/139338119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7fbce0072020 con 0x7fbcfc160410 2023-12-19T18:27:46.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.053+0000 7fbd019a0640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbce006a430 0x7fbce006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.054 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.054+0000 7fbd019a0640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbce006a430 0x7fbce006c8f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009a90 tx=0x7fbcf8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:46.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.057+0000 7fbcf27fc640 1 -- 172.21.15.27:0/139338119 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbcf4054220 con 0x7fbcfc160410 2023-12-19T18:27:46.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.264+0000 7fbd03c04640 1 -- 172.21.15.27:0/139338119 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbcfc0c9cd0 con 0x7fbcfc160410 2023-12-19T18:27:46.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.265+0000 7fbcf27fc640 1 -- 172.21.15.27:0/139338119 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v189) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbcf401c020 con 0x7fbcfc160410 2023-12-19T18:27:46.264 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.268+0000 7fbd03c04640 1 -- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbce006a430 msgr2=0x7fbce006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.268+0000 7fbd03c04640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbce006a430 0x7fbce006c8f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fbcf8009a90 tx=0x7fbcf8008040 comp rx=0 tx=0).stop 2023-12-19T18:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.268+0000 7fbd03c04640 1 -- 172.21.15.27:0/139338119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcfc160410 msgr2=0x7fbcfc15dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.268+0000 7fbd03c04640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcfc160410 0x7fbcfc15dbe0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0ca840 tx=0x7fbcf400f4a0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.268+0000 7fbd03c04640 1 -- 172.21.15.27:0/139338119 shutdown_connections 2023-12-19T18:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.268+0000 7fbd03c04640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbce006a430 0x7fbce006c8f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.268+0000 7fbd03c04640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcfc160410 0x7fbcfc15dbe0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.268+0000 7fbd03c04640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbcfc1525e0 0x7fbcfc15fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.268+0000 7fbd03c04640 1 --2- 172.21.15.27:0/139338119 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbce80047c0 0x7fbcfc15aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.268+0000 7fbd03c04640 1 -- 172.21.15.27:0/139338119 >> 172.21.15.27:0/139338119 conn(0x7fbcfc0ceb50 msgr2=0x7fbcfc0ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:46.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.268+0000 7fbd03c04640 1 -- 172.21.15.27:0/139338119 shutdown_connections 2023-12-19T18:27:46.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.268+0000 7fbd03c04640 1 -- 172.21.15.27:0/139338119 wait complete. 2023-12-19T18:27:46.289 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-12-19T18:27:46.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:27:46.290 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-19T18:27:46.395 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[119737]: fuse finished with error 0 and tester_r 0 2023-12-19T18:27:46.399 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:27:46.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:27:46.399 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:27:46.475 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:27:46.476 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:27:46.539 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:27:46.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.780+0000 7fbb7a177640 1 Processor -- start 2023-12-19T18:27:46.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.780+0000 7fbb7a177640 1 -- start start 2023-12-19T18:27:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.782+0000 7fbb7a177640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbb74164a80 0x7fbb740ce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.782+0000 7fbb7a177640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb740cbc60 0x7fbb740cc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.782+0000 7fbb7a177640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb740caba0 con 0x7fbb740c9190 2023-12-19T18:27:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.782+0000 7fbb7a177640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb74164e60 con 0x7fbb74164a80 2023-12-19T18:27:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.782+0000 7fbb7a177640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7406c2d0 con 0x7fbb740cbc60 2023-12-19T18:27:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.782+0000 7fbb737fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbb74164a80 0x7fbb740ce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.782+0000 7fbb737fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fbb74164a80 0x7fbb740ce520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42810/0 (socket says 172.21.15.27:42810) 2023-12-19T18:27:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.782+0000 7fbb737fe640 1 -- 172.21.15.27:0/4247524103 learned_addr learned my addr 172.21.15.27:0/4247524103 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.782+0000 7fbb737fe640 1 -- 172.21.15.27:0/4247524103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb740cbc60 msgr2=0x7fbb740cc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.782+0000 7fbb737fe640 1 --2- 172.21.15.27:0/4247524103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb740cbc60 0x7fbb740cc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.782+0000 7fbb737fe640 1 -- 172.21.15.27:0/4247524103 >> v1:172.21.15.192:6789/0 conn(0x7fbb740c9190 legacy=0x7fbb740cb550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.782+0000 7fbb737fe640 1 -- 172.21.15.27:0/4247524103 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb740cae40 con 0x7fbb74164a80 2023-12-19T18:27:46.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.784+0000 7fbb737fe640 1 --2- 172.21.15.27:0/4247524103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb74164a80 0x7fbb740ce520 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009870 tx=0x7fbb6402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be15c70d972a8438 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:46.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.784+0000 7fbb72ffd640 1 -- 172.21.15.27:0/4247524103 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb640317c0 con 0x7fbb74164a80 2023-12-19T18:27:46.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.784+0000 7fbb72ffd640 1 -- 172.21.15.27:0/4247524103 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb64031920 con 0x7fbb74164a80 2023-12-19T18:27:46.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.784+0000 7fbb72ffd640 1 -- 172.21.15.27:0/4247524103 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb64031cb0 con 0x7fbb74164a80 2023-12-19T18:27:46.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.784+0000 7fbb7a177640 1 -- 172.21.15.27:0/4247524103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb74164a80 msgr2=0x7fbb740ce520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:46.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.784+0000 7fbb7a177640 1 --2- 172.21.15.27:0/4247524103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb74164a80 0x7fbb740ce520 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7fbb64009870 tx=0x7fbb6402b860 comp rx=0 tx=0).stop 2023-12-19T18:27:46.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.785+0000 7fbb7a177640 1 -- 172.21.15.27:0/4247524103 shutdown_connections 2023-12-19T18:27:46.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.785+0000 7fbb7a177640 1 --2- 172.21.15.27:0/4247524103 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb740cbc60 0x7fbb740cc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.785+0000 7fbb7a177640 1 --2- 172.21.15.27:0/4247524103 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb74164a80 0x7fbb740ce520 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.785+0000 7fbb7a177640 1 -- 172.21.15.27:0/4247524103 >> 172.21.15.27:0/4247524103 conn(0x7fbb7406a330 msgr2=0x7fbb740b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:46.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.785+0000 7fbb7a177640 1 -- 172.21.15.27:0/4247524103 shutdown_connections 2023-12-19T18:27:46.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.785+0000 7fbb7a177640 1 -- 172.21.15.27:0/4247524103 wait complete. 2023-12-19T18:27:46.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.786+0000 7fbb7a177640 1 Processor -- start 2023-12-19T18:27:46.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.787+0000 7fbb7a177640 1 -- start start 2023-12-19T18:27:46.790 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.788+0000 7fbb7a177640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb740c9190 0x7fbb740c2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.788+0000 7fbb7a177640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb740cbc60 0x7fbb74150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.788+0000 7fbb7a177640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb74164a80 0x7fbb74150c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.788+0000 7fbb7a177640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb74151190 con 0x7fbb740cbc60 2023-12-19T18:27:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.788+0000 7fbb7a177640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740c4e90 con 0x7fbb74164a80 2023-12-19T18:27:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.788+0000 7fbb7a177640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb740c5170 con 0x7fbb740c9190 2023-12-19T18:27:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.788+0000 7fbb6bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb740cbc60 0x7fbb74150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.788+0000 7fbb6bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb740cbc60 0x7fbb74150710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42822/0 (socket says 172.21.15.27:42822) 2023-12-19T18:27:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.788+0000 7fbb6bfff640 1 -- 172.21.15.27:0/1718008382 learned_addr learned my addr 172.21.15.27:0/1718008382 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.788+0000 7fbb737fe640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb740c9190 0x7fbb740c2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.788+0000 7fbb73fff640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb74164a80 0x7fbb74150c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.789+0000 7fbb6bfff640 1 -- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb740c9190 msgr2=0x7fbb740c2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:46.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.789+0000 7fbb6bfff640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb740c9190 0x7fbb740c2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.789+0000 7fbb6bfff640 1 -- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb74164a80 msgr2=0x7fbb74150c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.789+0000 7fbb6bfff640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb74164a80 0x7fbb74150c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.789+0000 7fbb6bfff640 1 -- 172.21.15.27:0/1718008382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb640094d0 con 0x7fbb740cbc60 2023-12-19T18:27:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.789+0000 7fbb737fe640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb740c9190 0x7fbb740c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:27:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.789+0000 7fbb6bfff640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb740cbc60 0x7fbb74150710 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00b3e0 tx=0x7fbb5c00b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.789+0000 7fbb70ff9640 1 -- 172.21.15.27:0/1718008382 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c011830 con 0x7fbb740cbc60 2023-12-19T18:27:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.789+0000 7fbb70ff9640 1 -- 172.21.15.27:0/1718008382 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5c0119b0 con 0x7fbb740cbc60 2023-12-19T18:27:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.789+0000 7fbb70ff9640 1 -- 172.21.15.27:0/1718008382 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c011d60 con 0x7fbb740cbc60 2023-12-19T18:27:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.789+0000 7fbb7a177640 1 -- 172.21.15.27:0/1718008382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb740c5450 con 0x7fbb740cbc60 2023-12-19T18:27:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.790+0000 7fbb7a177640 1 -- 172.21.15.27:0/1718008382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb741537b0 con 0x7fbb740cbc60 2023-12-19T18:27:46.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.791+0000 7fbb7a177640 1 -- 172.21.15.27:0/1718008382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb74068e90 con 0x7fbb740cbc60 2023-12-19T18:27:46.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.792+0000 7fbb70ff9640 1 -- 172.21.15.27:0/1718008382 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbb5c01a400 con 0x7fbb740cbc60 2023-12-19T18:27:46.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.793+0000 7fbb70ff9640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb5806a430 0x7fbb5806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:46.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.793+0000 7fbb70ff9640 1 -- 172.21.15.27:0/1718008382 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbb5c088ad0 con 0x7fbb740cbc60 2023-12-19T18:27:46.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.793+0000 7fbb70ff9640 1 -- 172.21.15.27:0/1718008382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7fbb58072020 con 0x7fbb740cbc60 2023-12-19T18:27:46.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.793+0000 7fbb737fe640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb5806a430 0x7fbb5806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:46.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.795+0000 7fbb737fe640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb5806a430 0x7fbb5806c8f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbb64036000 tx=0x7fbb6403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:46.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.797+0000 7fbb70ff9640 1 -- 172.21.15.27:0/1718008382 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbb5c015020 con 0x7fbb740cbc60 2023-12-19T18:27:46.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.993+0000 7fbb7a177640 1 -- 172.21.15.27:0/1718008382 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb740b4ac0 con 0x7fbb740cbc60 2023-12-19T18:27:46.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.995+0000 7fbb70ff9640 1 -- 172.21.15.27:0/1718008382 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+16236 (secure 0 0 0) 0x7fbb5c054070 con 0x7fbb740cbc60 2023-12-19T18:27:46.995 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:27:46.995 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":189,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:27:31.809427+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:26:36.428093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T18:26:37.523993+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1e","mappings":[{"from":2,"to":5}]},{"pgid":"21.33","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.997+0000 7fbb7a177640 1 -- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb5806a430 msgr2=0x7fbb5806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.997+0000 7fbb7a177640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb5806a430 0x7fbb5806c8f0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fbb64036000 tx=0x7fbb6403d040 comp rx=0 tx=0).stop 2023-12-19T18:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.997+0000 7fbb7a177640 1 -- 172.21.15.27:0/1718008382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb740cbc60 msgr2=0x7fbb74150710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.997+0000 7fbb7a177640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb740cbc60 0x7fbb74150710 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00b3e0 tx=0x7fbb5c00b8a0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.998+0000 7fbb7a177640 1 -- 172.21.15.27:0/1718008382 shutdown_connections 2023-12-19T18:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.998+0000 7fbb7a177640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbb5806a430 0x7fbb5806c8f0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.998+0000 7fbb7a177640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbb74164a80 0x7fbb74150c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.998+0000 7fbb7a177640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbb740cbc60 0x7fbb74150710 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.998+0000 7fbb7a177640 1 --2- 172.21.15.27:0/1718008382 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbb740c9190 0x7fbb740c2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.998+0000 7fbb7a177640 1 -- 172.21.15.27:0/1718008382 >> 172.21.15.27:0/1718008382 conn(0x7fbb7406a330 msgr2=0x7fbb740af110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.998+0000 7fbb7a177640 1 -- 172.21.15.27:0/1718008382 shutdown_connections 2023-12-19T18:27:46.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:46.998+0000 7fbb7a177640 1 -- 172.21.15.27:0/1718008382 wait complete. 2023-12-19T18:27:47.018 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:26:36.428093+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:26:37.523993+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:27:47.019 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:27:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.240+0000 7fb0e359e640 1 Processor -- start 2023-12-19T18:27:47.240 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.240+0000 7fb0e359e640 1 -- start start 2023-12-19T18:27:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.242+0000 7fb0e359e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb0e415e9d0 0x7fb0e40cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.242+0000 7fb0e359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e415b3e0 con 0x7fb0e4152070 2023-12-19T18:27:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.242+0000 7fb0e359e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e415eed0 con 0x7fb0e4151c90 2023-12-19T18:27:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.242+0000 7fb0e359e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e415f1b0 con 0x7fb0e415e9d0 2023-12-19T18:27:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.242+0000 7fb0e2d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb0e4152070 0x7fb0e415b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53584/0 (socket says 172.21.15.27:53584) 2023-12-19T18:27:47.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.242+0000 7fb0e2d9d640 1 -- 172.21.15.27:0/2571868634 learned_addr learned my addr 172.21.15.27:0/2571868634 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.243+0000 7fb0dbfff640 1 --2- 172.21.15.27:0/2571868634 >> v2:172.21.15.192:3300/0 conn(0x7fb0e415e9d0 0x7fb0e40cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.243+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87794930 0 0) 0x7fb0e415b3e0 con 0x7fb0e4152070 2023-12-19T18:27:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.243+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0d0003520 con 0x7fb0e4152070 2023-12-19T18:27:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.243+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4224235348 0 0) 0x7fb0d0003520 con 0x7fb0e4152070 2023-12-19T18:27:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.243+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 >> v2:172.21.15.192:3300/0 conn(0x7fb0e415e9d0 msgr2=0x7fb0e40cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.243+0000 7fb0e1d9b640 1 --2- 172.21.15.27:0/2571868634 >> v2:172.21.15.192:3300/0 conn(0x7fb0e415e9d0 0x7fb0e40cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.243+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 >> v1:172.21.15.192:6789/0 conn(0x7fb0e4151c90 legacy=0x7fb0e406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.243+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e415f490 con 0x7fb0e4152070 2023-12-19T18:27:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.243+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fb0dc0019f0 con 0x7fb0e4152070 2023-12-19T18:27:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 >> v1:172.21.15.27:6789/0 conn(0x7fb0e4152070 legacy=0x7fb0e415b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e1d9b640 1 --2- 172.21.15.27:0/2571868634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d00039a0 0x7fb0d0003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e1d9b640 1 --2- 172.21.15.27:0/2571868634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0d00047c0 0x7fb0d0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e1d9b640 1 --2- 172.21.15.27:0/2571868634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0d00051d0 0x7fb0d000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e415f1b0 con 0x7fb0d00039a0 2023-12-19T18:27:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d000c100 con 0x7fb0d00051d0 2023-12-19T18:27:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d000c2c0 con 0x7fb0d00047c0 2023-12-19T18:27:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e259c640 1 --2- 172.21.15.27:0/2571868634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d00039a0 0x7fb0d0003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e259c640 1 -- 172.21.15.27:0/2571868634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0d00047c0 msgr2=0x7fb0d0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0dbfff640 1 --2- 172.21.15.27:0/2571868634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0d00047c0 0x7fb0d0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e259c640 1 --2- 172.21.15.27:0/2571868634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0d00047c0 0x7fb0d0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e259c640 1 -- 172.21.15.27:0/2571868634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0d00051d0 msgr2=0x7fb0d000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e2d9d640 1 --2- 172.21.15.27:0/2571868634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0d00051d0 0x7fb0d000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e259c640 1 --2- 172.21.15.27:0/2571868634 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0d00051d0 0x7fb0d000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e2d9d640 1 -- 172.21.15.27:0/2571868634 reap_dead start 2023-12-19T18:27:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.244+0000 7fb0e259c640 1 -- 172.21.15.27:0/2571868634 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d0003520 con 0x7fb0d00039a0 2023-12-19T18:27:47.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.245+0000 7fb0dbfff640 1 --2- 172.21.15.27:0/2571868634 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0d00047c0 0x7fb0d0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:47.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.246+0000 7fb0e259c640 1 --2- 172.21.15.27:0/2571868634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d00039a0 0x7fb0d0003dc0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4001750 tx=0x7fb0d00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c729a8e1e396097 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:47.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.246+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d40052d0 con 0x7fb0d00039a0 2023-12-19T18:27:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.246+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d402e050 con 0x7fb0d00039a0 2023-12-19T18:27:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.246+0000 7fb0e1d9b640 1 -- 172.21.15.27:0/2571868634 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4003830 con 0x7fb0d00039a0 2023-12-19T18:27:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.246+0000 7fb0e359e640 1 -- 172.21.15.27:0/2571868634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d00039a0 msgr2=0x7fb0d0003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.246+0000 7fb0e359e640 1 --2- 172.21.15.27:0/2571868634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d00039a0 0x7fb0d0003dc0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4001750 tx=0x7fb0d00057f0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.246+0000 7fb0e359e640 1 -- 172.21.15.27:0/2571868634 shutdown_connections 2023-12-19T18:27:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.247+0000 7fb0e359e640 1 --2- 172.21.15.27:0/2571868634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d00039a0 0x7fb0d0003dc0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.247+0000 7fb0e359e640 1 -- 172.21.15.27:0/2571868634 >> 172.21.15.27:0/2571868634 conn(0x7fb0e406a060 msgr2=0x7fb0e406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.247+0000 7fb0e359e640 1 -- 172.21.15.27:0/2571868634 shutdown_connections 2023-12-19T18:27:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.247+0000 7fb0e359e640 1 -- 172.21.15.27:0/2571868634 wait complete. 2023-12-19T18:27:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.248+0000 7fb0e359e640 1 Processor -- start 2023-12-19T18:27:47.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.249+0000 7fb0e359e640 1 -- start start 2023-12-19T18:27:47.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.249+0000 7fb0e359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d00039a0 0x7fb0e414f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.249+0000 7fb0e359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e41effc0 0x7fb0e41f0440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e40c3900 0x7fb0e40c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e40c42c0 con 0x7fb0d00039a0 2023-12-19T18:27:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e414fe40 con 0x7fb0e41effc0 2023-12-19T18:27:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e40a8a40 con 0x7fb0e40c3900 2023-12-19T18:27:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d00039a0 0x7fb0e414f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d00039a0 0x7fb0e414f900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42842/0 (socket says 172.21.15.27:42842) 2023-12-19T18:27:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e259c640 1 -- 172.21.15.27:0/699348128 learned_addr learned my addr 172.21.15.27:0/699348128 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e259c640 1 -- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e40c3900 msgr2=0x7fb0e40c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e2d9d640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e40c3900 0x7fb0e40c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0dbfff640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e41effc0 0x7fb0e41f0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e259c640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e40c3900 0x7fb0e40c3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e259c640 1 -- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e41effc0 msgr2=0x7fb0e41f0440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e259c640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e41effc0 0x7fb0e41f0440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e259c640 1 -- 172.21.15.27:0/699348128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e40a8d20 con 0x7fb0d00039a0 2023-12-19T18:27:47.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0e2d9d640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e40c3900 0x7fb0e40c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:47.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.250+0000 7fb0dbfff640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e41effc0 0x7fb0e41f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:47.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.251+0000 7fb0e259c640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d00039a0 0x7fb0e414f900 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4005040 tx=0x7fb0d4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:47.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.251+0000 7fb0daffd640 1 -- 172.21.15.27:0/699348128 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d40076c0 con 0x7fb0d00039a0 2023-12-19T18:27:47.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.251+0000 7fb0daffd640 1 -- 172.21.15.27:0/699348128 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d4007a30 con 0x7fb0d00039a0 2023-12-19T18:27:47.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.251+0000 7fb0daffd640 1 -- 172.21.15.27:0/699348128 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d4006be0 con 0x7fb0d00039a0 2023-12-19T18:27:47.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.251+0000 7fb0e359e640 1 -- 172.21.15.27:0/699348128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e40a9000 con 0x7fb0d00039a0 2023-12-19T18:27:47.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.251+0000 7fb0e359e640 1 -- 172.21.15.27:0/699348128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e40a95c0 con 0x7fb0d00039a0 2023-12-19T18:27:47.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.252+0000 7fb0e359e640 1 -- 172.21.15.27:0/699348128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e404e840 con 0x7fb0d00039a0 2023-12-19T18:27:47.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.253+0000 7fb0daffd640 1 -- 172.21.15.27:0/699348128 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb0d4006d40 con 0x7fb0d00039a0 2023-12-19T18:27:47.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.254+0000 7fb0daffd640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb0c406a460 0x7fb0c406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.254+0000 7fb0daffd640 1 -- 172.21.15.27:0/699348128 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fb0d40aa900 con 0x7fb0d00039a0 2023-12-19T18:27:47.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.254+0000 7fb0daffd640 1 -- 172.21.15.27:0/699348128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7fb0c4072050 con 0x7fb0d00039a0 2023-12-19T18:27:47.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.255+0000 7fb0dbfff640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb0c406a460 0x7fb0c406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.256+0000 7fb0dbfff640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb0c406a460 0x7fb0c406c920 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb0e415a910 tx=0x7fb0cc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:47.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.259+0000 7fb0daffd640 1 -- 172.21.15.27:0/699348128 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb0d404b020 con 0x7fb0d00039a0 2023-12-19T18:27:47.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.451+0000 7fb0e359e640 1 -- 172.21.15.27:0/699348128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb0e415b8b0 con 0x7fb0d00039a0 2023-12-19T18:27:47.451 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.452+0000 7fb0daffd640 1 -- 172.21.15.27:0/699348128 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v189) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb0d4075ff0 con 0x7fb0d00039a0 2023-12-19T18:27:47.452 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:27:47.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.455+0000 7fb0e359e640 1 -- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb0c406a460 msgr2=0x7fb0c406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:47.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.455+0000 7fb0e359e640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb0c406a460 0x7fb0c406c920 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fb0e415a910 tx=0x7fb0cc008040 comp rx=0 tx=0).stop 2023-12-19T18:27:47.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.455+0000 7fb0e359e640 1 -- 172.21.15.27:0/699348128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d00039a0 msgr2=0x7fb0e414f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:47.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.455+0000 7fb0e359e640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d00039a0 0x7fb0e414f900 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7fb0d4005040 tx=0x7fb0d4001120 comp rx=0 tx=0).stop 2023-12-19T18:27:47.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.455+0000 7fb0e359e640 1 -- 172.21.15.27:0/699348128 shutdown_connections 2023-12-19T18:27:47.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.455+0000 7fb0e359e640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb0c406a460 0x7fb0c406c920 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.455+0000 7fb0e359e640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb0e40c3900 0x7fb0e40c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.455+0000 7fb0e359e640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb0e41effc0 0x7fb0e41f0440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.455 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.455+0000 7fb0e359e640 1 --2- 172.21.15.27:0/699348128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0d00039a0 0x7fb0e414f900 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.455+0000 7fb0e359e640 1 -- 172.21.15.27:0/699348128 >> 172.21.15.27:0/699348128 conn(0x7fb0e406a060 msgr2=0x7fb0e415db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:47.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.455+0000 7fb0e359e640 1 -- 172.21.15.27:0/699348128 shutdown_connections 2023-12-19T18:27:47.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.455+0000 7fb0e359e640 1 -- 172.21.15.27:0/699348128 wait complete. 2023-12-19T18:27:47.474 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:27:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.709+0000 7fd7f4e62640 1 Processor -- start 2023-12-19T18:27:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.709+0000 7fd7f4e62640 1 -- start start 2023-12-19T18:27:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.710+0000 7fd7f4e62640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7f0151c90 0x7fd7f006b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.710+0000 7fd7f4e62640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd7f015e9d0 0x7fd7f00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.710+0000 7fd7f4e62640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f015b3e0 con 0x7fd7f0152070 2023-12-19T18:27:47.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.710+0000 7fd7f4e62640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f015eed0 con 0x7fd7f015e9d0 2023-12-19T18:27:47.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.710+0000 7fd7f4e62640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f015f1b0 con 0x7fd7f0151c90 2023-12-19T18:27:47.711 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.711+0000 7fd7e7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd7f015e9d0 0x7fd7f00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.711+0000 7fd7e7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd7f015e9d0 0x7fd7f00cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42848/0 (socket says 172.21.15.27:42848) 2023-12-19T18:27:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.711+0000 7fd7e7fff640 1 -- 172.21.15.27:0/1169019237 learned_addr learned my addr 172.21.15.27:0/1169019237 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.711+0000 7fd7ef7fe640 1 --2- 172.21.15.27:0/1169019237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7f0151c90 0x7fd7f006b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.711+0000 7fd7e7fff640 1 -- 172.21.15.27:0/1169019237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7f0151c90 msgr2=0x7fd7f006b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.711+0000 7fd7e7fff640 1 --2- 172.21.15.27:0/1169019237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7f0151c90 0x7fd7f006b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.711+0000 7fd7e7fff640 1 -- 172.21.15.27:0/1169019237 >> v1:172.21.15.192:6789/0 conn(0x7fd7f0152070 legacy=0x7fd7f015b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.712+0000 7fd7e7fff640 1 -- 172.21.15.27:0/1169019237 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f015f490 con 0x7fd7f015e9d0 2023-12-19T18:27:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.713+0000 7fd7e7fff640 1 --2- 172.21.15.27:0/1169019237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7f015e9d0 0x7fd7f00cee40 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009870 tx=0x7fd7d802b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1cc9f591ab464e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:47.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.713+0000 7fd7eeffd640 1 -- 172.21.15.27:0/1169019237 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8031640 con 0x7fd7f015e9d0 2023-12-19T18:27:47.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.713+0000 7fd7eeffd640 1 -- 172.21.15.27:0/1169019237 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7d80317a0 con 0x7fd7f015e9d0 2023-12-19T18:27:47.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.713+0000 7fd7eeffd640 1 -- 172.21.15.27:0/1169019237 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7d8031b30 con 0x7fd7f015e9d0 2023-12-19T18:27:47.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.714+0000 7fd7f4e62640 1 -- 172.21.15.27:0/1169019237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7f015e9d0 msgr2=0x7fd7f00cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:47.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.714+0000 7fd7f4e62640 1 --2- 172.21.15.27:0/1169019237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7f015e9d0 0x7fd7f00cee40 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7fd7d8009870 tx=0x7fd7d802b8f0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.714+0000 7fd7f4e62640 1 -- 172.21.15.27:0/1169019237 shutdown_connections 2023-12-19T18:27:47.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.714+0000 7fd7f4e62640 1 --2- 172.21.15.27:0/1169019237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7f015e9d0 0x7fd7f00cee40 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.714+0000 7fd7f4e62640 1 --2- 172.21.15.27:0/1169019237 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7f0151c90 0x7fd7f006b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.714+0000 7fd7f4e62640 1 -- 172.21.15.27:0/1169019237 >> 172.21.15.27:0/1169019237 conn(0x7fd7f006a060 msgr2=0x7fd7f006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:47.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.714+0000 7fd7f4e62640 1 -- 172.21.15.27:0/1169019237 shutdown_connections 2023-12-19T18:27:47.714 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.714+0000 7fd7f4e62640 1 -- 172.21.15.27:0/1169019237 wait complete. 2023-12-19T18:27:47.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.716+0000 7fd7f4e62640 1 Processor -- start 2023-12-19T18:27:47.715 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.716+0000 7fd7f4e62640 1 -- start start 2023-12-19T18:27:47.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.717+0000 7fd7f4e62640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7f0151c90 0x7fd7f015aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.717 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.717+0000 7fd7f4e62640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7f0152070 0x7fd7f00b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.717+0000 7fd7f4e62640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7f015e9d0 0x7fd7f00b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.717+0000 7fd7f4e62640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f014de00 con 0x7fd7f015e9d0 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.717+0000 7fd7f4e62640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00b32b0 con 0x7fd7f0151c90 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.717+0000 7fd7f4e62640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00b3ac0 con 0x7fd7f0152070 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.717+0000 7fd7effff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7f015e9d0 0x7fd7f00b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.717+0000 7fd7effff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7f015e9d0 0x7fd7f00b10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42852/0 (socket says 172.21.15.27:42852) 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.717+0000 7fd7effff640 1 -- 172.21.15.27:0/2010774096 learned_addr learned my addr 172.21.15.27:0/2010774096 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.718+0000 7fd7ef7fe640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7f0151c90 0x7fd7f015aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.718+0000 7fd7e7fff640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7f0152070 0x7fd7f00b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.718+0000 7fd7effff640 1 -- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7f0152070 msgr2=0x7fd7f00b2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.718+0000 7fd7effff640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7f0152070 0x7fd7f00b2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.718+0000 7fd7effff640 1 -- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7f0151c90 msgr2=0x7fd7f015aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.718+0000 7fd7effff640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7f0151c90 0x7fd7f015aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.718+0000 7fd7effff640 1 -- 172.21.15.27:0/2010774096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d80094d0 con 0x7fd7f015e9d0 2023-12-19T18:27:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.718+0000 7fd7effff640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7f015e9d0 0x7fd7f00b10b0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800c760 tx=0x7fd7e800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.718+0000 7fd7ecff9640 1 -- 172.21.15.27:0/2010774096 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8003b70 con 0x7fd7f015e9d0 2023-12-19T18:27:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.719+0000 7fd7ecff9640 1 -- 172.21.15.27:0/2010774096 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e8003cf0 con 0x7fd7f015e9d0 2023-12-19T18:27:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.719+0000 7fd7ecff9640 1 -- 172.21.15.27:0/2010774096 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e800aa90 con 0x7fd7f015e9d0 2023-12-19T18:27:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.719+0000 7fd7f4e62640 1 -- 172.21.15.27:0/2010774096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f014f7e0 con 0x7fd7f015e9d0 2023-12-19T18:27:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.719+0000 7fd7f4e62640 1 -- 172.21.15.27:0/2010774096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f00b3500 con 0x7fd7f015e9d0 2023-12-19T18:27:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.720+0000 7fd7f4e62640 1 -- 172.21.15.27:0/2010774096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f01f57e0 con 0x7fd7f015e9d0 2023-12-19T18:27:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.721+0000 7fd7ecff9640 1 -- 172.21.15.27:0/2010774096 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd7e800abf0 con 0x7fd7f015e9d0 2023-12-19T18:27:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.722+0000 7fd7ecff9640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd7d006a430 0x7fd7d006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.722+0000 7fd7ecff9640 1 -- 172.21.15.27:0/2010774096 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd7e8088640 con 0x7fd7f015e9d0 2023-12-19T18:27:47.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.722+0000 7fd7ecff9640 1 -- 172.21.15.27:0/2010774096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7fd7d0072020 con 0x7fd7f015e9d0 2023-12-19T18:27:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.723+0000 7fd7ef7fe640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd7d006a430 0x7fd7d006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:47.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.724+0000 7fd7ef7fe640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd7d006a430 0x7fd7d006c8f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0006fd0 tx=0x7fd7e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:47.726 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.726+0000 7fd7ecff9640 1 -- 172.21.15.27:0/2010774096 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd7e8003e60 con 0x7fd7f015e9d0 2023-12-19T18:27:47.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.920+0000 7fd7f4e62640 1 -- 172.21.15.27:0/2010774096 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd7f004e840 con 0x7fd7f015e9d0 2023-12-19T18:27:47.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.921+0000 7fd7ecff9640 1 -- 172.21.15.27:0/2010774096 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v189) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd7f01f57e0 con 0x7fd7f015e9d0 2023-12-19T18:27:47.921 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:27:47.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.924+0000 7fd7f4e62640 1 -- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd7d006a430 msgr2=0x7fd7d006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.924+0000 7fd7f4e62640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd7d006a430 0x7fd7d006c8f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fd7e0006fd0 tx=0x7fd7e0008040 comp rx=0 tx=0).stop 2023-12-19T18:27:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.924+0000 7fd7f4e62640 1 -- 172.21.15.27:0/2010774096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7f015e9d0 msgr2=0x7fd7f00b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.924+0000 7fd7f4e62640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7f015e9d0 0x7fd7f00b10b0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7fd7e800c760 tx=0x7fd7e800cc20 comp rx=0 tx=0).stop 2023-12-19T18:27:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.924+0000 7fd7f4e62640 1 -- 172.21.15.27:0/2010774096 shutdown_connections 2023-12-19T18:27:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.924+0000 7fd7f4e62640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd7d006a430 0x7fd7d006c8f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.924+0000 7fd7f4e62640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7f015e9d0 0x7fd7f00b10b0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.924+0000 7fd7f4e62640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7f0152070 0x7fd7f00b2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.924+0000 7fd7f4e62640 1 --2- 172.21.15.27:0/2010774096 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7f0151c90 0x7fd7f015aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.924+0000 7fd7f4e62640 1 -- 172.21.15.27:0/2010774096 >> 172.21.15.27:0/2010774096 conn(0x7fd7f006a060 msgr2=0x7fd7f0150610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.925+0000 7fd7f4e62640 1 -- 172.21.15.27:0/2010774096 shutdown_connections 2023-12-19T18:27:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:47.925+0000 7fd7f4e62640 1 -- 172.21.15.27:0/2010774096 wait complete. 2023-12-19T18:27:47.946 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:27:48.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.179+0000 7f1fcbfff640 1 Processor -- start 2023-12-19T18:27:48.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.179+0000 7f1fcbfff640 1 -- start start 2023-12-19T18:27:48.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.180+0000 7f1fcbfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fcc15cf80 0x7f1fcc0d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.180+0000 7f1fcbfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f1fcc0b85e0 0x7f1fcc0cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.180+0000 7f1fcbfff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fcc0d0e10 con 0x7f1fcc0b8a60 2023-12-19T18:27:48.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.180+0000 7f1fcbfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fcc0d1c70 con 0x7f1fcc0b85e0 2023-12-19T18:27:48.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.180+0000 7f1fcbfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc15d360 con 0x7f1fcc15cf80 2023-12-19T18:27:48.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.181+0000 7f1fcb7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f1fcc0b8a60 0x7f1fcc0cf500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53600/0 (socket says 172.21.15.27:53600) 2023-12-19T18:27:48.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.181+0000 7f1fcb7fe640 1 -- 172.21.15.27:0/2447676676 learned_addr learned my addr 172.21.15.27:0/2447676676 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:48.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.181+0000 7f1fca7fc640 1 --2- 172.21.15.27:0/2447676676 >> v2:172.21.15.192:3300/0 conn(0x7f1fcc0b85e0 0x7f1fcc0cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.181+0000 7f1fcaffd640 1 --2- 172.21.15.27:0/2447676676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fcc15cf80 0x7f1fcc0d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.181+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2276553805 0 0) 0x7f1fcc0d0e10 con 0x7f1fcc0b8a60 2023-12-19T18:27:48.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.181+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1fb8003520 con 0x7f1fcc0b8a60 2023-12-19T18:27:48.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1820088234 0 0) 0x7f1fb8003520 con 0x7f1fcc0b8a60 2023-12-19T18:27:48.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fcc15cf80 msgr2=0x7f1fcc0d2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:48.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 --2- 172.21.15.27:0/2447676676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fcc15cf80 0x7f1fcc0d2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 >> v2:172.21.15.192:3300/0 conn(0x7f1fcc0b85e0 msgr2=0x7f1fcc0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:48.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 --2- 172.21.15.27:0/2447676676 >> v2:172.21.15.192:3300/0 conn(0x7f1fcc0b85e0 0x7f1fcc0cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc0d1060 con 0x7f1fcc0b8a60 2023-12-19T18:27:48.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f1fc4001670 con 0x7f1fcc0b8a60 2023-12-19T18:27:48.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 >> v1:172.21.15.27:6789/0 conn(0x7f1fcc0b8a60 legacy=0x7f1fcc0cf500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:48.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 --2- 172.21.15.27:0/2447676676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fb8003970 0x7f1fb8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 --2- 172.21.15.27:0/2447676676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fcc15cf80 0x7f1fb80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 --2- 172.21.15.27:0/2447676676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fb8004d00 0x7f1fb8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0d1c70 con 0x7f1fb8004d00 2023-12-19T18:27:48.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc15d360 con 0x7f1fb8003970 2023-12-19T18:27:48.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb800b980 con 0x7f1fcc15cf80 2023-12-19T18:27:48.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.182+0000 7f1fcb7fe640 1 --2- 172.21.15.27:0/2447676676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fb8004d00 0x7f1fb8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.183+0000 7f1fcaffd640 1 --2- 172.21.15.27:0/2447676676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fb8003970 0x7f1fb8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.183+0000 7f1fcb7fe640 1 -- 172.21.15.27:0/2447676676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fcc15cf80 msgr2=0x7f1fb80047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:48.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.183+0000 7f1fcb7fe640 1 --2- 172.21.15.27:0/2447676676 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fcc15cf80 0x7f1fb80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.183+0000 7f1fcb7fe640 1 -- 172.21.15.27:0/2447676676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fb8003970 msgr2=0x7f1fb8003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:48.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.183+0000 7f1fcb7fe640 1 --2- 172.21.15.27:0/2447676676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fb8003970 0x7f1fb8003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.183+0000 7f1fcb7fe640 1 -- 172.21.15.27:0/2447676676 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb8003520 con 0x7f1fb8004d00 2023-12-19T18:27:48.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.183+0000 7f1fcaffd640 1 --2- 172.21.15.27:0/2447676676 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fb8003970 0x7f1fb8003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:48.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.184+0000 7f1fcb7fe640 1 --2- 172.21.15.27:0/2447676676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fb8004d00 0x7f1fb8005180 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0c9cf0 tx=0x7f1fb80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc4357ab618ecfc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:48.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.184+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4005ad0 con 0x7f1fb8004d00 2023-12-19T18:27:48.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.184+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc4005c30 con 0x7f1fb8004d00 2023-12-19T18:27:48.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.184+0000 7f1fc9ffb640 1 -- 172.21.15.27:0/2447676676 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc402c890 con 0x7f1fb8004d00 2023-12-19T18:27:48.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.185+0000 7f1fcbfff640 1 -- 172.21.15.27:0/2447676676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fb8004d00 msgr2=0x7f1fb8005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:48.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.185+0000 7f1fcbfff640 1 --2- 172.21.15.27:0/2447676676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fb8004d00 0x7f1fb8005180 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc0c9cf0 tx=0x7f1fb80057f0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.185+0000 7f1fcb7fe640 1 -- 172.21.15.27:0/2447676676 reap_dead start 2023-12-19T18:27:48.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.185+0000 7f1fcbfff640 1 -- 172.21.15.27:0/2447676676 shutdown_connections 2023-12-19T18:27:48.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.185+0000 7f1fcbfff640 1 -- 172.21.15.27:0/2447676676 >> 172.21.15.27:0/2447676676 conn(0x7f1fcc072e20 msgr2=0x7f1fcc073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:48.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.185+0000 7f1fcbfff640 1 -- 172.21.15.27:0/2447676676 shutdown_connections 2023-12-19T18:27:48.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.185+0000 7f1fcbfff640 1 -- 172.21.15.27:0/2447676676 wait complete. 2023-12-19T18:27:48.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.187+0000 7f1fcbfff640 1 Processor -- start 2023-12-19T18:27:48.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.187+0000 7f1fcbfff640 1 -- start start 2023-12-19T18:27:48.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.188+0000 7f1fcbfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fb8004d00 0x7f1fcc15e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.188+0000 7f1fcbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fcc151d40 0x7f1fcc15f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.188+0000 7f1fcbfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fcc0b3860 0x7f1fcc0b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.188+0000 7f1fcbfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc1521c0 con 0x7f1fcc151d40 2023-12-19T18:27:48.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.188+0000 7f1fcbfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc0b4220 con 0x7f1fcc0b3860 2023-12-19T18:27:48.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.188+0000 7f1fcbfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fcc15fa50 con 0x7f1fb8004d00 2023-12-19T18:27:48.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.188+0000 7f1fca7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fcc151d40 0x7f1fcc15f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.188+0000 7f1fca7fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fcc151d40 0x7f1fcc15f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42870/0 (socket says 172.21.15.27:42870) 2023-12-19T18:27:48.189 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.188+0000 7f1fca7fc640 1 -- 172.21.15.27:0/4059348815 learned_addr learned my addr 172.21.15.27:0/4059348815 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:48.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.188+0000 7f1fcaffd640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fb8004d00 0x7f1fcc15e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fcb7fe640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fcc0b3860 0x7f1fcc0b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fca7fc640 1 -- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fb8004d00 msgr2=0x7f1fcc15e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:48.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fca7fc640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fb8004d00 0x7f1fcc15e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fca7fc640 1 -- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fcc0b3860 msgr2=0x7f1fcc0b3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:48.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fca7fc640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fcc0b3860 0x7f1fcc0b3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fca7fc640 1 -- 172.21.15.27:0/4059348815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fcc1571a0 con 0x7f1fcc151d40 2023-12-19T18:27:48.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fcaffd640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fb8004d00 0x7f1fcc15e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:48.190 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fcb7fe640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fcc0b3860 0x7f1fcc0b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:48.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fca7fc640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fcc151d40 0x7f1fcc15f510 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009cf0 tx=0x7f1fbc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:48.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fc37fe640 1 -- 172.21.15.27:0/4059348815 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc003990 con 0x7f1fcc151d40 2023-12-19T18:27:48.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fc37fe640 1 -- 172.21.15.27:0/4059348815 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc00f040 con 0x7f1fcc151d40 2023-12-19T18:27:48.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fc37fe640 1 -- 172.21.15.27:0/4059348815 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc003d50 con 0x7f1fcc151d40 2023-12-19T18:27:48.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fcbfff640 1 -- 172.21.15.27:0/4059348815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fcc152340 con 0x7f1fcc151d40 2023-12-19T18:27:48.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.189+0000 7f1fcbfff640 1 -- 172.21.15.27:0/4059348815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fcc15ff90 con 0x7f1fcc151d40 2023-12-19T18:27:48.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.191+0000 7f1fcbfff640 1 -- 172.21.15.27:0/4059348815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fcc04e840 con 0x7f1fcc151d40 2023-12-19T18:27:48.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.192+0000 7f1fc37fe640 1 -- 172.21.15.27:0/4059348815 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1fbc004030 con 0x7f1fcc151d40 2023-12-19T18:27:48.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.193+0000 7f1fc37fe640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fac06a460 0x7f1fac06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.193+0000 7f1fc37fe640 1 -- 172.21.15.27:0/4059348815 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f1fbc089370 con 0x7f1fcc151d40 2023-12-19T18:27:48.192 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.193+0000 7f1fc37fe640 1 -- 172.21.15.27:0/4059348815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7f1fac072050 con 0x7f1fcc151d40 2023-12-19T18:27:48.193 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.193+0000 7f1fcaffd640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fac06a460 0x7f1fac06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.194 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.194+0000 7f1fcaffd640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fac06a460 0x7f1fac06c920 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009aa0 tx=0x7f1fb4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:48.197 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.197+0000 7f1fc37fe640 1 -- 172.21.15.27:0/4059348815 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1fbc054910 con 0x7f1fcc151d40 2023-12-19T18:27:48.348 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:27:48.357 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:27:48.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.395+0000 7f1fcbfff640 1 -- 172.21.15.27:0/4059348815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1fcc15b3c0 con 0x7f1fcc151d40 2023-12-19T18:27:48.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.396+0000 7f1fc37fe640 1 -- 172.21.15.27:0/4059348815 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v189) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1fbc057f30 con 0x7f1fcc151d40 2023-12-19T18:27:48.396 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:27:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.399+0000 7f1fcbfff640 1 -- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fac06a460 msgr2=0x7f1fac06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.399+0000 7f1fcbfff640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fac06a460 0x7f1fac06c920 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f1fb4009aa0 tx=0x7f1fb4008040 comp rx=0 tx=0).stop 2023-12-19T18:27:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.399+0000 7f1fcbfff640 1 -- 172.21.15.27:0/4059348815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fcc151d40 msgr2=0x7f1fcc15f510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.399+0000 7f1fcbfff640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fcc151d40 0x7f1fcc15f510 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc009cf0 tx=0x7f1fbc00c490 comp rx=0 tx=0).stop 2023-12-19T18:27:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.399+0000 7f1fcbfff640 1 -- 172.21.15.27:0/4059348815 shutdown_connections 2023-12-19T18:27:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.399+0000 7f1fcbfff640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1fac06a460 0x7f1fac06c920 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.399+0000 7f1fcbfff640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1fcc0b3860 0x7f1fcc0b3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.399+0000 7f1fcbfff640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1fcc151d40 0x7f1fcc15f510 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.399+0000 7f1fcbfff640 1 --2- 172.21.15.27:0/4059348815 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1fb8004d00 0x7f1fcc15e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.399+0000 7f1fcbfff640 1 -- 172.21.15.27:0/4059348815 >> 172.21.15.27:0/4059348815 conn(0x7f1fcc072e20 msgr2=0x7f1fcc155370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:48.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.399+0000 7f1fcbfff640 1 -- 172.21.15.27:0/4059348815 shutdown_connections 2023-12-19T18:27:48.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.399+0000 7f1fcbfff640 1 -- 172.21.15.27:0/4059348815 wait complete. 2023-12-19T18:27:48.417 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:27:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.690+0000 7f3f20fc6640 1 Processor -- start 2023-12-19T18:27:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.691+0000 7f3f20fc6640 1 -- start start 2023-12-19T18:27:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.693+0000 7f3f20fc6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f1c06f8e0 0x7f3f1c06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.693+0000 7f3f20fc6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f1c161a40 0x7f3f1c161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.693+0000 7f3f20fc6640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c06b310 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.693+0000 7f3f20fc6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1c0702a0 con 0x7f3f1c06f8e0 2023-12-19T18:27:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.693+0000 7f3f20fc6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c162790 con 0x7f3f1c161a40 2023-12-19T18:27:48.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.693+0000 7f3f13fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f1c06f8e0 0x7f3f1c06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.693+0000 7f3f13fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f1c06f8e0 0x7f3f1c06fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42874/0 (socket says 172.21.15.27:42874) 2023-12-19T18:27:48.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.693+0000 7f3f13fff640 1 -- 172.21.15.27:0/3235823920 learned_addr learned my addr 172.21.15.27:0/3235823920 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:48.694 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.693+0000 7f3f1bfff640 1 --2- 172.21.15.27:0/3235823920 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f1c161a40 0x7f3f1c161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.694+0000 7f3f13fff640 1 -- 172.21.15.27:0/3235823920 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f1c161a40 msgr2=0x7f3f1c161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:48.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.694+0000 7f3f13fff640 1 --2- 172.21.15.27:0/3235823920 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f1c161a40 0x7f3f1c161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.694+0000 7f3f13fff640 1 -- 172.21.15.27:0/3235823920 >> v1:172.21.15.192:6789/0 conn(0x7f3f1c15b9e0 legacy=0x7f3f1c0c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:48.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.694+0000 7f3f13fff640 1 -- 172.21.15.27:0/3235823920 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c06b560 con 0x7f3f1c06f8e0 2023-12-19T18:27:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.695+0000 7f3f13fff640 1 --2- 172.21.15.27:0/3235823920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f1c06f8e0 0x7f3f1c06fd60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009870 tx=0x7f3f0402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34b58d67a3d78fc8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.695+0000 7f3f1affd640 1 -- 172.21.15.27:0/3235823920 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04031640 con 0x7f3f1c06f8e0 2023-12-19T18:27:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.695+0000 7f3f1affd640 1 -- 172.21.15.27:0/3235823920 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f040317a0 con 0x7f3f1c06f8e0 2023-12-19T18:27:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.695+0000 7f3f1affd640 1 -- 172.21.15.27:0/3235823920 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04031b30 con 0x7f3f1c06f8e0 2023-12-19T18:27:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.696+0000 7f3f20fc6640 1 -- 172.21.15.27:0/3235823920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f1c06f8e0 msgr2=0x7f3f1c06fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.696+0000 7f3f20fc6640 1 --2- 172.21.15.27:0/3235823920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f1c06f8e0 0x7f3f1c06fd60 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009870 tx=0x7f3f0402b8f0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.696+0000 7f3f20fc6640 1 -- 172.21.15.27:0/3235823920 shutdown_connections 2023-12-19T18:27:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.696+0000 7f3f20fc6640 1 --2- 172.21.15.27:0/3235823920 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f1c161a40 0x7f3f1c161eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.696+0000 7f3f20fc6640 1 --2- 172.21.15.27:0/3235823920 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f1c06f8e0 0x7f3f1c06fd60 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.696+0000 7f3f20fc6640 1 -- 172.21.15.27:0/3235823920 >> 172.21.15.27:0/3235823920 conn(0x7f3f1c06a7d0 msgr2=0x7f3f1c0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.696+0000 7f3f20fc6640 1 -- 172.21.15.27:0/3235823920 shutdown_connections 2023-12-19T18:27:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.696+0000 7f3f20fc6640 1 -- 172.21.15.27:0/3235823920 wait complete. 2023-12-19T18:27:48.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.699+0000 7f3f20fc6640 1 Processor -- start 2023-12-19T18:27:48.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.699+0000 7f3f20fc6640 1 -- start start 2023-12-19T18:27:48.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.700+0000 7f3f20fc6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f1c06f8e0 0x7f3f1c0b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f20fc6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f1c15b9e0 0x7f3f1c1fa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f20fc6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c161a40 0x7f3f1c0c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f20fc6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0b2260 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f20fc6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0b1650 con 0x7f3f1c161a40 2023-12-19T18:27:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f20fc6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0b1930 con 0x7f3f1c06f8e0 2023-12-19T18:27:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f13fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f1c15b9e0 0x7f3f1c1fa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f13fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f1c15b9e0 0x7f3f1c1fa1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42882/0 (socket says 172.21.15.27:42882) 2023-12-19T18:27:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f13fff640 1 -- 172.21.15.27:0/2371602547 learned_addr learned my addr 172.21.15.27:0/2371602547 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f1b7fe640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f1c06f8e0 0x7f3f1c0b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f1bfff640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c161a40 0x7f3f1c0c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f13fff640 1 -- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f1c06f8e0 msgr2=0x7f3f1c0b3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f13fff640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f1c06f8e0 0x7f3f1c0b3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f13fff640 1 -- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c161a40 msgr2=0x7f3f1c0c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f13fff640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c161a40 0x7f3f1c0c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f13fff640 1 -- 172.21.15.27:0/2371602547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f040094d0 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.701+0000 7f3f1b7fe640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f1c06f8e0 0x7f3f1c0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:27:48.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.702+0000 7f3f13fff640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f1c15b9e0 0x7f3f1c1fa1f0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009870 tx=0x7f3f040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:48.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.702+0000 7f3f18ff9640 1 -- 172.21.15.27:0/2371602547 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0403ac60 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.702+0000 7f3f18ff9640 1 -- 172.21.15.27:0/2371602547 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0403adc0 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.702+0000 7f3f18ff9640 1 -- 172.21.15.27:0/2371602547 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f04044630 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.702+0000 7f3f20fc6640 1 -- 172.21.15.27:0/2371602547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c0b1c10 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.702+0000 7f3f20fc6640 1 -- 172.21.15.27:0/2371602547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1c1fad40 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.704+0000 7f3f20fc6640 1 -- 172.21.15.27:0/2371602547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1c0b23d0 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.705+0000 7f3f18ff9640 1 -- 172.21.15.27:0/2371602547 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3f0402f050 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.706+0000 7f3f18ff9640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3efc06a430 0x7f3efc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:48.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.706+0000 7f3f18ff9640 1 -- 172.21.15.27:0/2371602547 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3f040aa870 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.706+0000 7f3f18ff9640 1 -- 172.21.15.27:0/2371602547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7f3efc072020 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.707+0000 7f3f1b7fe640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3efc06a430 0x7f3efc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:48.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.708+0000 7f3f1b7fe640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3efc06a430 0x7f3efc06c8f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c003eb0 tx=0x7f3f0c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:48.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.710+0000 7f3f18ff9640 1 -- 172.21.15.27:0/2371602547 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3f04035060 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.917+0000 7f3f20fc6640 1 -- 172.21.15.27:0/2371602547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3f1c04e840 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.918+0000 7f3f18ff9640 1 -- 172.21.15.27:0/2371602547 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v189) v1 ==== 80+0+0 (secure 0 0 0) 0x7f3f04035030 con 0x7f3f1c15b9e0 2023-12-19T18:27:48.918 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:27:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.920+0000 7f3f20fc6640 1 -- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3efc06a430 msgr2=0x7f3efc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.920+0000 7f3f20fc6640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3efc06a430 0x7f3efc06c8f0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c003eb0 tx=0x7f3f0c009040 comp rx=0 tx=0).stop 2023-12-19T18:27:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.921+0000 7f3f20fc6640 1 -- 172.21.15.27:0/2371602547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f1c15b9e0 msgr2=0x7f3f1c1fa1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.921+0000 7f3f20fc6640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f1c15b9e0 0x7f3f1c1fa1f0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f3f04009870 tx=0x7f3f040046c0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.921+0000 7f3f20fc6640 1 -- 172.21.15.27:0/2371602547 shutdown_connections 2023-12-19T18:27:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.921+0000 7f3f20fc6640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3efc06a430 0x7f3efc06c8f0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.921+0000 7f3f20fc6640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f1c161a40 0x7f3f1c0c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.921+0000 7f3f20fc6640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f1c15b9e0 0x7f3f1c1fa1f0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.921+0000 7f3f20fc6640 1 --2- 172.21.15.27:0/2371602547 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f1c06f8e0 0x7f3f1c0b3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.921+0000 7f3f20fc6640 1 -- 172.21.15.27:0/2371602547 >> 172.21.15.27:0/2371602547 conn(0x7f3f1c06a7d0 msgr2=0x7f3f1c0ca650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.921+0000 7f3f20fc6640 1 -- 172.21.15.27:0/2371602547 shutdown_connections 2023-12-19T18:27:48.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:48.921+0000 7f3f20fc6640 1 -- 172.21.15.27:0/2371602547 wait complete. 2023-12-19T18:27:48.940 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-19T18:27:48.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:27:48.941 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-19T18:27:49.039 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[92078]: fuse finished with error 0 and tester_r 0 2023-12-19T18:27:49.045 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:27:49.045 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:27:49.045 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-19T18:27:49.095 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:27:49.095 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:27:49.158 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:27:49.159 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:27:49.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:27:49.159 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-19T18:27:49.220 INFO:teuthology.orchestra.run.smithi027.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-19T18:27:49.221 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:27:49.221 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:27:49.221 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:27:49.281 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:27:49.281 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:27:49.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:27:49.282 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-19T18:27:49.334 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-19T18:27:49.335 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:27:49.336 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:27:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.574+0000 7fd0c5567640 1 Processor -- start 2023-12-19T18:27:49.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.575+0000 7fd0c5567640 1 -- start start 2023-12-19T18:27:49.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.576+0000 7fd0c5567640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd0c00d2350 0x7fd0c00b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.576+0000 7fd0c5567640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c00b5e90 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.577+0000 7fd0c5567640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0c00d2850 con 0x7fd0c01604d0 2023-12-19T18:27:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.577+0000 7fd0c5567640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0c00d2b30 con 0x7fd0c00d2350 2023-12-19T18:27:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.577+0000 7fd0bf7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd0c00b4ef0 0x7fd0c00b6300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53610/0 (socket says 172.21.15.27:53610) 2023-12-19T18:27:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.577+0000 7fd0bf7fe640 1 -- 172.21.15.27:0/654475008 learned_addr learned my addr 172.21.15.27:0/654475008 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:49.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.577+0000 7fd0c4d66640 1 --2- 172.21.15.27:0/654475008 >> v2:172.21.15.27:3300/0 conn(0x7fd0c00d2350 0x7fd0c00b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.577+0000 7fd0beffd640 1 -- 172.21.15.27:0/654475008 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1805436240 0 0) 0x7fd0c00b5e90 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.577+0000 7fd0beffd640 1 -- 172.21.15.27:0/654475008 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0ac003520 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.577+0000 7fd0c4d66640 1 -- 172.21.15.27:0/654475008 >> v1:172.21.15.192:6789/0 conn(0x7fd0c01604d0 legacy=0x7fd0c00b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.577+0000 7fd0c4d66640 1 -- 172.21.15.27:0/654475008 >> v1:172.21.15.27:6789/0 conn(0x7fd0c00b4ef0 legacy=0x7fd0c00b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:49.578 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.577+0000 7fd0c4d66640 1 -- 172.21.15.27:0/654475008 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c00cded0 con 0x7fd0c00d2350 2023-12-19T18:27:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.579+0000 7fd0c4d66640 1 --2- 172.21.15.27:0/654475008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c00d2350 0x7fd0c00b6a10 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80097a0 tx=0x7fd0b802b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab0aa8d5e597b7f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.579+0000 7fd0beffd640 1 -- 172.21.15.27:0/654475008 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8030610 con 0x7fd0c00d2350 2023-12-19T18:27:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.579+0000 7fd0beffd640 1 -- 172.21.15.27:0/654475008 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b8030770 con 0x7fd0c00d2350 2023-12-19T18:27:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.579+0000 7fd0beffd640 1 -- 172.21.15.27:0/654475008 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b8030b00 con 0x7fd0c00d2350 2023-12-19T18:27:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.579+0000 7fd0c5567640 1 -- 172.21.15.27:0/654475008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c00d2350 msgr2=0x7fd0c00b6a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.580+0000 7fd0c5567640 1 --2- 172.21.15.27:0/654475008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c00d2350 0x7fd0c00b6a10 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7fd0b80097a0 tx=0x7fd0b802b960 comp rx=0 tx=0).stop 2023-12-19T18:27:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.580+0000 7fd0c5567640 1 -- 172.21.15.27:0/654475008 shutdown_connections 2023-12-19T18:27:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.580+0000 7fd0c5567640 1 --2- 172.21.15.27:0/654475008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c00d2350 0x7fd0c00b6a10 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:49.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.580+0000 7fd0c5567640 1 -- 172.21.15.27:0/654475008 >> 172.21.15.27:0/654475008 conn(0x7fd0c00cf7e0 msgr2=0x7fd0c00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.580+0000 7fd0c5567640 1 -- 172.21.15.27:0/654475008 shutdown_connections 2023-12-19T18:27:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.580+0000 7fd0c5567640 1 -- 172.21.15.27:0/654475008 wait complete. 2023-12-19T18:27:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.582+0000 7fd0c5567640 1 Processor -- start 2023-12-19T18:27:49.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.582+0000 7fd0c5567640 1 -- start start 2023-12-19T18:27:49.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.583+0000 7fd0c5567640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c00b4ef0 0x7fd0c00cae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:49.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.583+0000 7fd0c5567640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0c00d2350 0x7fd0c00c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.583+0000 7fd0c5567640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0c01604d0 0x7fd0c01efca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.583+0000 7fd0c5567640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c01f27f0 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.583+0000 7fd0c5567640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c00c90c0 con 0x7fd0c00d2350 2023-12-19T18:27:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.583+0000 7fd0c5567640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c01f01e0 con 0x7fd0c01604d0 2023-12-19T18:27:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.583+0000 7fd0bffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c00b4ef0 0x7fd0c00cae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.583+0000 7fd0bffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c00b4ef0 0x7fd0c00cae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42900/0 (socket says 172.21.15.27:42900) 2023-12-19T18:27:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.583+0000 7fd0bffff640 1 -- 172.21.15.27:0/1669080793 learned_addr learned my addr 172.21.15.27:0/1669080793 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.583+0000 7fd0bf7fe640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0c00d2350 0x7fd0c00c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.584+0000 7fd0c4d66640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0c01604d0 0x7fd0c01efca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.584+0000 7fd0bffff640 1 -- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0c01604d0 msgr2=0x7fd0c01efca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.584+0000 7fd0bffff640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0c01604d0 0x7fd0c01efca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:49.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.584+0000 7fd0bffff640 1 -- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0c00d2350 msgr2=0x7fd0c00c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:49.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.584+0000 7fd0bffff640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0c00d2350 0x7fd0c00c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:49.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.584+0000 7fd0bffff640 1 -- 172.21.15.27:0/1669080793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0b8009400 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.584+0000 7fd0c4d66640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0c01604d0 0x7fd0c01efca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:49.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.584+0000 7fd0bffff640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c00b4ef0 0x7fd0c00cae80 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000c730 tx=0x7fd0b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:49.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.584+0000 7fd0bcff9640 1 -- 172.21.15.27:0/1669080793 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0003b70 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.584+0000 7fd0bcff9640 1 -- 172.21.15.27:0/1669080793 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0b0003cf0 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.584+0000 7fd0bcff9640 1 -- 172.21.15.27:0/1669080793 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0b0003ea0 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.584+0000 7fd0c5567640 1 -- 172.21.15.27:0/1669080793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c00cb510 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.585+0000 7fd0c5567640 1 -- 172.21.15.27:0/1669080793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c00cbad0 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.586+0000 7fd0c5567640 1 -- 172.21.15.27:0/1669080793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c0066f80 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.587+0000 7fd0bcff9640 1 -- 172.21.15.27:0/1669080793 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd0b000aa90 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.588+0000 7fd0bcff9640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd0a006a430 0x7fd0a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:49.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.588+0000 7fd0bcff9640 1 -- 172.21.15.27:0/1669080793 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fd0b0088770 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.588 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.589+0000 7fd0bcff9640 1 -- 172.21.15.27:0/1669080793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7fd0a0072020 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.589 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.589+0000 7fd0bf7fe640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd0a006a430 0x7fd0a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:49.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.590+0000 7fd0bf7fe640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd0a006a430 0x7fd0a006c8f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00b65c0 tx=0x7fd0a8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:49.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.593+0000 7fd0bcff9640 1 -- 172.21.15.27:0/1669080793 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd0b0053d20 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.826+0000 7fd0c5567640 1 -- 172.21.15.27:0/1669080793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd0c004e840 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.827+0000 7fd0bcff9640 1 -- 172.21.15.27:0/1669080793 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4719 (secure 0 0 0) 0x7fd0b0057340 con 0x7fd0c00b4ef0 2023-12-19T18:27:49.827 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:27:49.827 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3563146395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3563146395},{"type":"v1","addr":"172.21.15.192:6836","nonce":3563146395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3445095528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3445095528},{"type":"v1","addr":"172.21.15.192:6837","nonce":3445095528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3536033217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3536033217},{"type":"v1","addr":"172.21.15.27:6833","nonce":3536033217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:26:38.483498+0000","modified":"2023-12-19T18:26:42.008767+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2895889621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2895889621},{"type":"v1","addr":"172.21.15.27:6835","nonce":2895889621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-19T18:27:49.827 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 77 2023-12-19T18:27:49.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.830+0000 7fd0c5567640 1 -- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd0a006a430 msgr2=0x7fd0a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:49.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.830+0000 7fd0c5567640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd0a006a430 0x7fd0a006c8f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fd0c00b65c0 tx=0x7fd0a8009040 comp rx=0 tx=0).stop 2023-12-19T18:27:49.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.830+0000 7fd0c5567640 1 -- 172.21.15.27:0/1669080793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c00b4ef0 msgr2=0x7fd0c00cae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:49.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.830+0000 7fd0c5567640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c00b4ef0 0x7fd0c00cae80 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7fd0b000c730 tx=0x7fd0b000cbf0 comp rx=0 tx=0).stop 2023-12-19T18:27:49.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.830+0000 7fd0c5567640 1 -- 172.21.15.27:0/1669080793 shutdown_connections 2023-12-19T18:27:49.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.830+0000 7fd0c5567640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd0a006a430 0x7fd0a006c8f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:49.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.830+0000 7fd0c5567640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd0c01604d0 0x7fd0c01efca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:49.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.830+0000 7fd0c5567640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd0c00d2350 0x7fd0c00c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:49.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.830+0000 7fd0c5567640 1 --2- 172.21.15.27:0/1669080793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd0c00b4ef0 0x7fd0c00cae80 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:49.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.830+0000 7fd0c5567640 1 -- 172.21.15.27:0/1669080793 >> 172.21.15.27:0/1669080793 conn(0x7fd0c00cf7e0 msgr2=0x7fd0c00b2430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:49.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.830+0000 7fd0c5567640 1 -- 172.21.15.27:0/1669080793 shutdown_connections 2023-12-19T18:27:49.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:49.830+0000 7fd0c5567640 1 -- 172.21.15.27:0/1669080793 wait complete. 2023-12-19T18:27:49.851 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:27:50.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.089+0000 7f3533557640 1 Processor -- start 2023-12-19T18:27:50.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.089+0000 7f3533557640 1 -- start start 2023-12-19T18:27:50.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.091+0000 7f3533557640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f352c0c4260 0x7f352c06fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.091+0000 7f3533557640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f352c0af750 0x7f352c165470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.091+0000 7f3533557640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c1659b0 0x7f352c06ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.091+0000 7f3533557640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352c0ce0d0 con 0x7f352c0af750 2023-12-19T18:27:50.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.091+0000 7f3533557640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352c164f90 con 0x7f352c0c4260 2023-12-19T18:27:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.091+0000 7f3533557640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0cd830 con 0x7f352c1659b0 2023-12-19T18:27:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.091+0000 7f3531d54640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f352c0af750 0x7f352c165470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.091+0000 7f3531d54640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f352c0af750 0x7f352c165470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42910/0 (socket says 172.21.15.27:42910) 2023-12-19T18:27:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.091+0000 7f3531d54640 1 -- 172.21.15.27:0/1699194586 learned_addr learned my addr 172.21.15.27:0/1699194586 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.091+0000 7f3532555640 1 --2- 172.21.15.27:0/1699194586 >> v2:172.21.15.192:3300/0 conn(0x7f352c0c4260 0x7f352c06fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.092+0000 7f3532d56640 1 --2- 172.21.15.27:0/1699194586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c1659b0 0x7f352c06ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.092+0000 7f3531d54640 1 -- 172.21.15.27:0/1699194586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c1659b0 msgr2=0x7f352c06ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:50.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.092+0000 7f3531d54640 1 --2- 172.21.15.27:0/1699194586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c1659b0 0x7f352c06ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.092+0000 7f3531d54640 1 -- 172.21.15.27:0/1699194586 >> v2:172.21.15.192:3300/0 conn(0x7f352c0c4260 msgr2=0x7f352c06fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:50.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.092+0000 7f3531d54640 1 --2- 172.21.15.27:0/1699194586 >> v2:172.21.15.192:3300/0 conn(0x7f352c0c4260 0x7f352c06fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.092+0000 7f3531d54640 1 -- 172.21.15.27:0/1699194586 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c0cdb10 con 0x7f352c0af750 2023-12-19T18:27:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.093+0000 7f3531d54640 1 --2- 172.21.15.27:0/1699194586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0af750 0x7f352c165470 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f352c0ce490 tx=0x7f351c02b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb11be402791025 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.093+0000 7f3531553640 1 -- 172.21.15.27:0/1699194586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c0317e0 con 0x7f352c0af750 2023-12-19T18:27:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.094+0000 7f3531553640 1 -- 172.21.15.27:0/1699194586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c031940 con 0x7f352c0af750 2023-12-19T18:27:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.094+0000 7f3531553640 1 -- 172.21.15.27:0/1699194586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c031cd0 con 0x7f352c0af750 2023-12-19T18:27:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.094+0000 7f3533557640 1 -- 172.21.15.27:0/1699194586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0af750 msgr2=0x7f352c165470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.094+0000 7f3533557640 1 --2- 172.21.15.27:0/1699194586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0af750 0x7f352c165470 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7f352c0ce490 tx=0x7f351c02b880 comp rx=0 tx=0).stop 2023-12-19T18:27:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.094+0000 7f3533557640 1 -- 172.21.15.27:0/1699194586 shutdown_connections 2023-12-19T18:27:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.094+0000 7f3533557640 1 --2- 172.21.15.27:0/1699194586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c1659b0 0x7f352c06ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.094+0000 7f3533557640 1 --2- 172.21.15.27:0/1699194586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0af750 0x7f352c165470 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.094+0000 7f3533557640 1 --2- 172.21.15.27:0/1699194586 >> v2:172.21.15.192:3300/0 conn(0x7f352c0c4260 0x7f352c06fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.094+0000 7f3533557640 1 -- 172.21.15.27:0/1699194586 >> 172.21.15.27:0/1699194586 conn(0x7f352c06d5f0 msgr2=0x7f352c06d9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.094+0000 7f3533557640 1 -- 172.21.15.27:0/1699194586 shutdown_connections 2023-12-19T18:27:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.094+0000 7f3533557640 1 -- 172.21.15.27:0/1699194586 wait complete. 2023-12-19T18:27:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.096+0000 7f3533557640 1 Processor -- start 2023-12-19T18:27:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.096+0000 7f3533557640 1 -- start start 2023-12-19T18:27:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.097+0000 7f3533557640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c0af750 0x7f352c160170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.097+0000 7f3533557640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0c4260 0x7f352c0c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.097+0000 7f3533557640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c1659b0 0x7f352c0c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.097+0000 7f3533557640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0c6820 con 0x7f352c0c4260 2023-12-19T18:27:50.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.097+0000 7f3533557640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0c6b00 con 0x7f352c1659b0 2023-12-19T18:27:50.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.097+0000 7f3533557640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352c0c6de0 con 0x7f352c0af750 2023-12-19T18:27:50.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.098+0000 7f3531d54640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0c4260 0x7f352c0c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.098+0000 7f3531d54640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0c4260 0x7f352c0c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42918/0 (socket says 172.21.15.27:42918) 2023-12-19T18:27:50.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.098+0000 7f3531d54640 1 -- 172.21.15.27:0/2347101202 learned_addr learned my addr 172.21.15.27:0/2347101202 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.098+0000 7f3532d56640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c1659b0 0x7f352c0c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.098+0000 7f3532555640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c0af750 0x7f352c160170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.098+0000 7f3531d54640 1 -- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c0af750 msgr2=0x7f352c160170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.098+0000 7f3531d54640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c0af750 0x7f352c160170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.098+0000 7f3531d54640 1 -- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c1659b0 msgr2=0x7f352c0c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:50.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.098+0000 7f3531d54640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c1659b0 0x7f352c0c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.098+0000 7f3531d54640 1 -- 172.21.15.27:0/2347101202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c0094d0 con 0x7f352c0c4260 2023-12-19T18:27:50.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.098+0000 7f3532d56640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c1659b0 0x7f352c0c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:27:50.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.099+0000 7f3531d54640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0c4260 0x7f352c0c5a10 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f351c000c00 tx=0x7f351c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:50.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.099+0000 7f3522ffd640 1 -- 172.21.15.27:0/2347101202 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c032880 con 0x7f352c0c4260 2023-12-19T18:27:50.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.099+0000 7f3522ffd640 1 -- 172.21.15.27:0/2347101202 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c032a00 con 0x7f352c0c4260 2023-12-19T18:27:50.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.099+0000 7f3533557640 1 -- 172.21.15.27:0/2347101202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c0c70c0 con 0x7f352c0c4260 2023-12-19T18:27:50.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.099+0000 7f3522ffd640 1 -- 172.21.15.27:0/2347101202 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c032db0 con 0x7f352c0c4260 2023-12-19T18:27:50.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.099+0000 7f3533557640 1 -- 172.21.15.27:0/2347101202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c1606b0 con 0x7f352c0c4260 2023-12-19T18:27:50.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.100+0000 7f3533557640 1 -- 172.21.15.27:0/2347101202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352c04e840 con 0x7f352c0c4260 2023-12-19T18:27:50.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.102+0000 7f3522ffd640 1 -- 172.21.15.27:0/2347101202 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f351c04e450 con 0x7f352c0c4260 2023-12-19T18:27:50.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.103+0000 7f3522ffd640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f351006a430 0x7f351006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.103+0000 7f3522ffd640 1 -- 172.21.15.27:0/2347101202 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f351c0aaaa0 con 0x7f352c0c4260 2023-12-19T18:27:50.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.104+0000 7f3522ffd640 1 -- 172.21.15.27:0/2347101202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7f3510072020 con 0x7f352c0c4260 2023-12-19T18:27:50.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.104+0000 7f3532555640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f351006a430 0x7f351006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.105+0000 7f3532555640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f351006a430 0x7f351006c8f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3528006fd0 tx=0x7f3528008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:50.107 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.108+0000 7f3522ffd640 1 -- 172.21.15.27:0/2347101202 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f351c0760f0 con 0x7f352c0c4260 2023-12-19T18:27:50.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.306+0000 7f3533557640 1 -- 172.21.15.27:0/2347101202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f352c06f120 con 0x7f352c0c4260 2023-12-19T18:27:50.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.309+0000 7f3522ffd640 1 -- 172.21.15.27:0/2347101202 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v189) v1 ==== 74+0+16236 (secure 0 0 0) 0x7f351c079710 con 0x7f352c0c4260 2023-12-19T18:27:50.309 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:27:50.309 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":189,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:27:31.809427+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:26:36.428093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T18:26:37.523993+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"189","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1e","mappings":[{"from":2,"to":5}]},{"pgid":"21.33","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:27:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.311+0000 7f3533557640 1 -- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f351006a430 msgr2=0x7f351006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.311+0000 7f3533557640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f351006a430 0x7f351006c8f0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f3528006fd0 tx=0x7f3528008040 comp rx=0 tx=0).stop 2023-12-19T18:27:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.312+0000 7f3533557640 1 -- 172.21.15.27:0/2347101202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0c4260 msgr2=0x7f352c0c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.312+0000 7f3533557640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0c4260 0x7f352c0c5a10 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7f351c000c00 tx=0x7f351c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.312+0000 7f3533557640 1 -- 172.21.15.27:0/2347101202 shutdown_connections 2023-12-19T18:27:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.312+0000 7f3533557640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f351006a430 0x7f351006c8f0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.312+0000 7f3533557640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f352c1659b0 0x7f352c0c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.312+0000 7f3533557640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f352c0c4260 0x7f352c0c5a10 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.312+0000 7f3533557640 1 --2- 172.21.15.27:0/2347101202 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f352c0af750 0x7f352c160170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.312+0000 7f3533557640 1 -- 172.21.15.27:0/2347101202 >> 172.21.15.27:0/2347101202 conn(0x7f352c06d5f0 msgr2=0x7f352c161000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:50.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.312+0000 7f3533557640 1 -- 172.21.15.27:0/2347101202 shutdown_connections 2023-12-19T18:27:50.313 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.312+0000 7f3533557640 1 -- 172.21.15.27:0/2347101202 wait complete. 2023-12-19T18:27:50.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:26:36.428093+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '184', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:26:37.523993+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '189', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:27:50.332 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:27:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.563+0000 7f9f9f095640 1 Processor -- start 2023-12-19T18:27:50.563 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.563+0000 7f9f9f095640 1 -- start start 2023-12-19T18:27:50.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.565+0000 7f9f9f095640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f98143450 0x7f9f980ceef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.565+0000 7f9f9f095640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9f9806c850 0x7f9f980cbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.565+0000 7f9f9f095640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f980b27d0 con 0x7f9f980cf430 2023-12-19T18:27:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.565+0000 7f9f9f095640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f980b33e0 con 0x7f9f9806c850 2023-12-19T18:27:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.565+0000 7f9f9f095640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980cf850 con 0x7f9f98143450 2023-12-19T18:27:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.565+0000 7f9f9d892640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9f980cf430 0x7f9f9806c140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53622/0 (socket says 172.21.15.27:53622) 2023-12-19T18:27:50.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.565+0000 7f9f9d892640 1 -- 172.21.15.27:0/651676454 learned_addr learned my addr 172.21.15.27:0/651676454 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:50.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.565+0000 7f9f9e894640 1 --2- 172.21.15.27:0/651676454 >> v2:172.21.15.192:3300/0 conn(0x7f9f9806c850 0x7f9f980cbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.565+0000 7f9f9e093640 1 --2- 172.21.15.27:0/651676454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f98143450 0x7f9f980ceef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513394456 0 0) 0x7f9f980b27d0 con 0x7f9f980cf430 2023-12-19T18:27:50.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9f84005730 con 0x7f9f980cf430 2023-12-19T18:27:50.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 199239057 0 0) 0x7f9f84005730 con 0x7f9f980cf430 2023-12-19T18:27:50.570 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f98143450 msgr2=0x7f9f980ceef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 --2- 172.21.15.27:0/651676454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f98143450 0x7f9f980ceef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 >> v2:172.21.15.192:3300/0 conn(0x7f9f9806c850 msgr2=0x7f9f980cbdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 --2- 172.21.15.27:0/651676454 >> v2:172.21.15.192:3300/0 conn(0x7f9f9806c850 0x7f9f980cbdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f980b2a40 con 0x7f9f980cf430 2023-12-19T18:27:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f9f88001670 con 0x7f9f980cf430 2023-12-19T18:27:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 >> v1:172.21.15.27:6789/0 conn(0x7f9f980cf430 legacy=0x7f9f9806c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 --2- 172.21.15.27:0/651676454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f84005b80 0x7f9f84005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 --2- 172.21.15.27:0/651676454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f98143450 0x7f9f840069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 --2- 172.21.15.27:0/651676454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f840064e0 0x7f9f84006f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980b33e0 con 0x7f9f840064e0 2023-12-19T18:27:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f980cf850 con 0x7f9f84005b80 2023-12-19T18:27:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.566+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f84007810 con 0x7f9f98143450 2023-12-19T18:27:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.567+0000 7f9f9e894640 1 --2- 172.21.15.27:0/651676454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f840064e0 0x7f9f84006f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.567+0000 7f9f9d892640 1 --2- 172.21.15.27:0/651676454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f98143450 0x7f9f840069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.567+0000 7f9f9e894640 1 -- 172.21.15.27:0/651676454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f98143450 msgr2=0x7f9f840069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.567+0000 7f9f9e093640 1 --2- 172.21.15.27:0/651676454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f84005b80 0x7f9f84005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.567+0000 7f9f9e894640 1 --2- 172.21.15.27:0/651676454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f98143450 0x7f9f840069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.567+0000 7f9f9e894640 1 -- 172.21.15.27:0/651676454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f84005b80 msgr2=0x7f9f84005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.567+0000 7f9f9e894640 1 --2- 172.21.15.27:0/651676454 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f84005b80 0x7f9f84005fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.567+0000 7f9f9e894640 1 -- 172.21.15.27:0/651676454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f84005730 con 0x7f9f840064e0 2023-12-19T18:27:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.567+0000 7f9f9d892640 1 --2- 172.21.15.27:0/651676454 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f98143450 0x7f9f840069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:50.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.568+0000 7f9f9e894640 1 --2- 172.21.15.27:0/651676454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f840064e0 0x7f9f84006f10 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000e170 tx=0x7f9f84007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5b144995b5b7a7f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:50.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.568+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9002cc20 con 0x7f9f840064e0 2023-12-19T18:27:50.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.569+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f9000f040 con 0x7f9f840064e0 2023-12-19T18:27:50.574 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.569+0000 7f9f9d091640 1 -- 172.21.15.27:0/651676454 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90034440 con 0x7f9f840064e0 2023-12-19T18:27:50.579 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.569+0000 7f9f9f095640 1 -- 172.21.15.27:0/651676454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f840064e0 msgr2=0x7f9f84006f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.569+0000 7f9f9f095640 1 --2- 172.21.15.27:0/651676454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f840064e0 0x7f9f84006f10 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000e170 tx=0x7f9f84007a00 comp rx=0 tx=0).stop 2023-12-19T18:27:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.569+0000 7f9f9e894640 1 -- 172.21.15.27:0/651676454 reap_dead start 2023-12-19T18:27:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.569+0000 7f9f9f095640 1 -- 172.21.15.27:0/651676454 shutdown_connections 2023-12-19T18:27:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.569+0000 7f9f9f095640 1 -- 172.21.15.27:0/651676454 >> 172.21.15.27:0/651676454 conn(0x7f9f9806a060 msgr2=0x7f9f9806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.569+0000 7f9f9f095640 1 -- 172.21.15.27:0/651676454 shutdown_connections 2023-12-19T18:27:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.569+0000 7f9f9f095640 1 -- 172.21.15.27:0/651676454 wait complete. 2023-12-19T18:27:50.580 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.571+0000 7f9f9f095640 1 Processor -- start 2023-12-19T18:27:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.571+0000 7f9f9f095640 1 -- start start 2023-12-19T18:27:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.572+0000 7f9f9f095640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f840064e0 0x7f9f98159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.572+0000 7f9f9f095640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f9815b8b0 0x7f9f9815bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.572+0000 7f9f9f095640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98070740 0x7f9f98070bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.572+0000 7f9f9f095640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f981f2410 con 0x7f9f98070740 2023-12-19T18:27:50.581 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.573+0000 7f9f9f095640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f98071100 con 0x7f9f840064e0 2023-12-19T18:27:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.573+0000 7f9f9f095640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f98159940 con 0x7f9f9815b8b0 2023-12-19T18:27:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.573+0000 7f9f9e894640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98070740 0x7f9f98070bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.573+0000 7f9f9e894640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98070740 0x7f9f98070bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42944/0 (socket says 172.21.15.27:42944) 2023-12-19T18:27:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.573+0000 7f9f9e894640 1 -- 172.21.15.27:0/1031599845 learned_addr learned my addr 172.21.15.27:0/1031599845 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.573+0000 7f9f9e093640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f840064e0 0x7f9f98159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.573+0000 7f9f9e894640 1 -- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f9815b8b0 msgr2=0x7f9f9815bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:50.582 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.573+0000 7f9f9d892640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f9815b8b0 0x7f9f9815bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.573+0000 7f9f9e894640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f9815b8b0 0x7f9f9815bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.573+0000 7f9f9e894640 1 -- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f840064e0 msgr2=0x7f9f98159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:50.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.573+0000 7f9f9e894640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f840064e0 0x7f9f98159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.573+0000 7f9f9e894640 1 -- 172.21.15.27:0/1031599845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f9815aaf0 con 0x7f9f98070740 2023-12-19T18:27:50.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.573+0000 7f9f9e093640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f840064e0 0x7f9f98159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:50.583 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.574+0000 7f9f9e894640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98070740 0x7f9f98070bc0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000e170 tx=0x7f9f90009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:50.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.574+0000 7f9f8effd640 1 -- 172.21.15.27:0/1031599845 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f9002cea0 con 0x7f9f98070740 2023-12-19T18:27:50.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.574+0000 7f9f8effd640 1 -- 172.21.15.27:0/1031599845 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f90034b50 con 0x7f9f98070740 2023-12-19T18:27:50.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.574+0000 7f9f8effd640 1 -- 172.21.15.27:0/1031599845 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f90003990 con 0x7f9f98070740 2023-12-19T18:27:50.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.574+0000 7f9f9f095640 1 -- 172.21.15.27:0/1031599845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f9815add0 con 0x7f9f98070740 2023-12-19T18:27:50.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.574+0000 7f9f9f095640 1 -- 172.21.15.27:0/1031599845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f9815b390 con 0x7f9f98070740 2023-12-19T18:27:50.584 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.575+0000 7f9f9f095640 1 -- 172.21.15.27:0/1031599845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f9804e840 con 0x7f9f98070740 2023-12-19T18:27:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.576+0000 7f9f8effd640 1 -- 172.21.15.27:0/1031599845 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9f90047460 con 0x7f9f98070740 2023-12-19T18:27:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.577+0000 7f9f8effd640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f7c06a430 0x7f9f7c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.577+0000 7f9f8effd640 1 -- 172.21.15.27:0/1031599845 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f9f900aa7b0 con 0x7f9f98070740 2023-12-19T18:27:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.578+0000 7f9f8effd640 1 -- 172.21.15.27:0/1031599845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7f9f7c072020 con 0x7f9f98070740 2023-12-19T18:27:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.578+0000 7f9f9e093640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f7c06a430 0x7f9f7c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:50.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.579+0000 7f9f9e093640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f7c06a430 0x7f9f7c06c8f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9f940099e0 tx=0x7f9f94008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:50.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.583+0000 7f9f8effd640 1 -- 172.21.15.27:0/1031599845 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9f90039060 con 0x7f9f98070740 2023-12-19T18:27:50.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.776+0000 7f9f9f095640 1 -- 172.21.15.27:0/1031599845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9f9806c170 con 0x7f9f98070740 2023-12-19T18:27:50.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.777+0000 7f9f8effd640 1 -- 172.21.15.27:0/1031599845 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v189) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9f90042070 con 0x7f9f98070740 2023-12-19T18:27:50.778 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:27:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.780+0000 7f9f9f095640 1 -- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f7c06a430 msgr2=0x7f9f7c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.780+0000 7f9f9f095640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f7c06a430 0x7f9f7c06c8f0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f9f940099e0 tx=0x7f9f94008040 comp rx=0 tx=0).stop 2023-12-19T18:27:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.780+0000 7f9f9f095640 1 -- 172.21.15.27:0/1031599845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98070740 msgr2=0x7f9f98070bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.780+0000 7f9f9f095640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98070740 0x7f9f98070bc0 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7f9f9000e170 tx=0x7f9f90009750 comp rx=0 tx=0).stop 2023-12-19T18:27:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.780+0000 7f9f9f095640 1 -- 172.21.15.27:0/1031599845 shutdown_connections 2023-12-19T18:27:50.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.780+0000 7f9f9f095640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9f7c06a430 0x7f9f7c06c8f0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.780+0000 7f9f9f095640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98070740 0x7f9f98070bc0 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.780+0000 7f9f9f095640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9f9815b8b0 0x7f9f9815bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.780+0000 7f9f9f095640 1 --2- 172.21.15.27:0/1031599845 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9f840064e0 0x7f9f98159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:50.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.780+0000 7f9f9f095640 1 -- 172.21.15.27:0/1031599845 >> 172.21.15.27:0/1031599845 conn(0x7f9f9806a060 msgr2=0x7f9f9806b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:50.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.780+0000 7f9f9f095640 1 -- 172.21.15.27:0/1031599845 shutdown_connections 2023-12-19T18:27:50.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:50.781+0000 7f9f9f095640 1 -- 172.21.15.27:0/1031599845 wait complete. 2023-12-19T18:27:50.800 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:27:51.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.034+0000 7f30317c6640 1 Processor -- start 2023-12-19T18:27:51.035 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.035+0000 7f30317c6640 1 -- start start 2023-12-19T18:27:51.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.036+0000 7f30317c6640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f302c0ce150 0x7f302c167ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.036+0000 7f30317c6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f302c0d0e10 con 0x7f302c0d2100 2023-12-19T18:27:51.036 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.036+0000 7f30317c6640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f302c0d1c70 con 0x7f302c15cf80 2023-12-19T18:27:51.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.036+0000 7f30317c6640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302c0b8cf0 con 0x7f302c0ce150 2023-12-19T18:27:51.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.037+0000 7f302b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f302c0d2100 0x7f302c0cda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53624/0 (socket says 172.21.15.27:53624) 2023-12-19T18:27:51.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.037+0000 7f302b7fe640 1 -- 172.21.15.27:0/1598754950 learned_addr learned my addr 172.21.15.27:0/1598754950 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:51.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.037+0000 7f3030fc5640 1 --2- 172.21.15.27:0/1598754950 >> v2:172.21.15.27:3300/0 conn(0x7f302c0ce150 0x7f302c167ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.037+0000 7f302affd640 1 -- 172.21.15.27:0/1598754950 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2536249738 0 0) 0x7f302c0d0e10 con 0x7f302c0d2100 2023-12-19T18:27:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.037+0000 7f302affd640 1 -- 172.21.15.27:0/1598754950 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3018003520 con 0x7f302c0d2100 2023-12-19T18:27:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.037+0000 7f3030fc5640 1 -- 172.21.15.27:0/1598754950 >> v1:172.21.15.192:6789/0 conn(0x7f302c15cf80 legacy=0x7f302c0b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.037+0000 7f3030fc5640 1 -- 172.21.15.27:0/1598754950 >> v1:172.21.15.27:6789/0 conn(0x7f302c0d2100 legacy=0x7f302c0cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.037+0000 7f3030fc5640 1 -- 172.21.15.27:0/1598754950 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c15d360 con 0x7f302c0ce150 2023-12-19T18:27:51.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.039+0000 7f3030fc5640 1 --2- 172.21.15.27:0/1598754950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302c0ce150 0x7f302c167ab0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f30240097a0 tx=0x7f302402b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a875a66ae649cb4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:51.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.039+0000 7f302affd640 1 -- 172.21.15.27:0/1598754950 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302402f810 con 0x7f302c0ce150 2023-12-19T18:27:51.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.039+0000 7f302affd640 1 -- 172.21.15.27:0/1598754950 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302402f970 con 0x7f302c0ce150 2023-12-19T18:27:51.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.039+0000 7f302affd640 1 -- 172.21.15.27:0/1598754950 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302402fce0 con 0x7f302c0ce150 2023-12-19T18:27:51.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.040+0000 7f30317c6640 1 -- 172.21.15.27:0/1598754950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302c0ce150 msgr2=0x7f302c167ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.040+0000 7f30317c6640 1 --2- 172.21.15.27:0/1598754950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302c0ce150 0x7f302c167ab0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f30240097a0 tx=0x7f302402b9f0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.040+0000 7f30317c6640 1 -- 172.21.15.27:0/1598754950 shutdown_connections 2023-12-19T18:27:51.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.040+0000 7f30317c6640 1 --2- 172.21.15.27:0/1598754950 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302c0ce150 0x7f302c167ab0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.040+0000 7f30317c6640 1 -- 172.21.15.27:0/1598754950 >> 172.21.15.27:0/1598754950 conn(0x7f302c072e20 msgr2=0x7f302c073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:51.040 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.040+0000 7f30317c6640 1 -- 172.21.15.27:0/1598754950 shutdown_connections 2023-12-19T18:27:51.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.040+0000 7f30317c6640 1 -- 172.21.15.27:0/1598754950 wait complete. 2023-12-19T18:27:51.041 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.042+0000 7f30317c6640 1 Processor -- start 2023-12-19T18:27:51.042 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.042+0000 7f30317c6640 1 -- start start 2023-12-19T18:27:51.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.043+0000 7f30317c6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f302c0ce150 0x7f302c158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.043+0000 7f30317c6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302c0d2100 0x7f302c0a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.043+0000 7f30317c6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f302c15cf80 0x7f302c0ab1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.043+0000 7f30317c6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c1590a0 con 0x7f302c0d2100 2023-12-19T18:27:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.043+0000 7f30317c6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0aa720 con 0x7f302c15cf80 2023-12-19T18:27:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.043+0000 7f30317c6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0ac0c0 con 0x7f302c0ce150 2023-12-19T18:27:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.043+0000 7f302b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302c0d2100 0x7f302c0a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.043+0000 7f302b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302c0d2100 0x7f302c0a96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42972/0 (socket says 172.21.15.27:42972) 2023-12-19T18:27:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.043+0000 7f302b7fe640 1 -- 172.21.15.27:0/703574972 learned_addr learned my addr 172.21.15.27:0/703574972 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.044+0000 7f302bfff640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f302c0ce150 0x7f302c158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.044+0000 7f3030fc5640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f302c15cf80 0x7f302c0ab1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.044+0000 7f302b7fe640 1 -- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f302c0ce150 msgr2=0x7f302c158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.044+0000 7f302b7fe640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f302c0ce150 0x7f302c158b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.044+0000 7f302b7fe640 1 -- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f302c15cf80 msgr2=0x7f302c0ab1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.044+0000 7f302b7fe640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f302c15cf80 0x7f302c0ab1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.044+0000 7f302b7fe640 1 -- 172.21.15.27:0/703574972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3024009400 con 0x7f302c0d2100 2023-12-19T18:27:51.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.044+0000 7f302b7fe640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302c0d2100 0x7f302c0a96a0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f301400c500 tx=0x7f301400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:51.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.044+0000 7f3028ff9640 1 -- 172.21.15.27:0/703574972 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3014005ce0 con 0x7f302c0d2100 2023-12-19T18:27:51.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.044+0000 7f3028ff9640 1 -- 172.21.15.27:0/703574972 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3014005e40 con 0x7f302c0d2100 2023-12-19T18:27:51.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.044+0000 7f3028ff9640 1 -- 172.21.15.27:0/703574972 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3014007330 con 0x7f302c0d2100 2023-12-19T18:27:51.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.045+0000 7f30317c6640 1 -- 172.21.15.27:0/703574972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302c0a9be0 con 0x7f302c0d2100 2023-12-19T18:27:51.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.045+0000 7f30317c6640 1 -- 172.21.15.27:0/703574972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302c0aa160 con 0x7f302c0d2100 2023-12-19T18:27:51.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.046+0000 7f30317c6640 1 -- 172.21.15.27:0/703574972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302c04e840 con 0x7f302c0d2100 2023-12-19T18:27:51.047 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.047+0000 7f3028ff9640 1 -- 172.21.15.27:0/703574972 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3014007490 con 0x7f302c0d2100 2023-12-19T18:27:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.048+0000 7f3028ff9640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f300c06a460 0x7f300c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.048+0000 7f3028ff9640 1 -- 172.21.15.27:0/703574972 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f3014087fc0 con 0x7f302c0d2100 2023-12-19T18:27:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.048+0000 7f3028ff9640 1 -- 172.21.15.27:0/703574972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7f300c072050 con 0x7f302c0d2100 2023-12-19T18:27:51.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.049+0000 7f302bfff640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f300c06a460 0x7f300c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.050+0000 7f302bfff640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f300c06a460 0x7f300c06c920 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f301c007fd0 tx=0x7f301c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:51.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.052+0000 7f3028ff9640 1 -- 172.21.15.27:0/703574972 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f30140baca0 con 0x7f302c0d2100 2023-12-19T18:27:51.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.245+0000 7f30317c6640 1 -- 172.21.15.27:0/703574972 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f302c04dff0 con 0x7f302c0d2100 2023-12-19T18:27:51.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.246+0000 7f3028ff9640 1 -- 172.21.15.27:0/703574972 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v189) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3014017060 con 0x7f302c0d2100 2023-12-19T18:27:51.247 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:27:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.249+0000 7f30317c6640 1 -- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f300c06a460 msgr2=0x7f300c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.249+0000 7f30317c6640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f300c06a460 0x7f300c06c920 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f301c007fd0 tx=0x7f301c009040 comp rx=0 tx=0).stop 2023-12-19T18:27:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.249+0000 7f30317c6640 1 -- 172.21.15.27:0/703574972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302c0d2100 msgr2=0x7f302c0a96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.249+0000 7f30317c6640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302c0d2100 0x7f302c0a96a0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f301400c500 tx=0x7f301400c9c0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.249+0000 7f30317c6640 1 -- 172.21.15.27:0/703574972 shutdown_connections 2023-12-19T18:27:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.249+0000 7f30317c6640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f300c06a460 0x7f300c06c920 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.249+0000 7f30317c6640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f302c15cf80 0x7f302c0ab1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.249+0000 7f30317c6640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f302c0d2100 0x7f302c0a96a0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.249+0000 7f30317c6640 1 --2- 172.21.15.27:0/703574972 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f302c0ce150 0x7f302c158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.249+0000 7f30317c6640 1 -- 172.21.15.27:0/703574972 >> 172.21.15.27:0/703574972 conn(0x7f302c072e20 msgr2=0x7f302c15c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:51.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.249+0000 7f30317c6640 1 -- 172.21.15.27:0/703574972 shutdown_connections 2023-12-19T18:27:51.250 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.249+0000 7f30317c6640 1 -- 172.21.15.27:0/703574972 wait complete. 2023-12-19T18:27:51.268 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:27:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.491+0000 7f95b02d5640 1 Processor -- start 2023-12-19T18:27:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.491+0000 7f95b02d5640 1 -- start start 2023-12-19T18:27:51.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.493+0000 7f95b02d5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95a815b1a0 0x7f95a806f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.493+0000 7f95b02d5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95a80c2720 0x7f95a806fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.493+0000 7f95b02d5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95a806bc80 con 0x7f95a8071420 2023-12-19T18:27:51.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.493+0000 7f95b02d5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95a815fd90 con 0x7f95a80c2720 2023-12-19T18:27:51.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.493+0000 7f95b02d5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a806d690 con 0x7f95a815b1a0 2023-12-19T18:27:51.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.493+0000 7f95ad870640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95a80c2720 0x7f95a806fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.493+0000 7f95ad870640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f95a80c2720 0x7f95a806fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42974/0 (socket says 172.21.15.27:42974) 2023-12-19T18:27:51.493 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.493+0000 7f95ad870640 1 -- 172.21.15.27:0/3763087574 learned_addr learned my addr 172.21.15.27:0/3763087574 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.493+0000 7f95ae071640 1 --2- 172.21.15.27:0/3763087574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95a815b1a0 0x7f95a806f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.494+0000 7f95ad870640 1 -- 172.21.15.27:0/3763087574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95a815b1a0 msgr2=0x7f95a806f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.494+0000 7f95ad870640 1 --2- 172.21.15.27:0/3763087574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95a815b1a0 0x7f95a806f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.494+0000 7f95ad870640 1 -- 172.21.15.27:0/3763087574 >> v1:172.21.15.192:6789/0 conn(0x7f95a8071420 legacy=0x7f95a80c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.494+0000 7f95ad870640 1 -- 172.21.15.27:0/3763087574 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a806d0a0 con 0x7f95a80c2720 2023-12-19T18:27:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.495+0000 7f95ad870640 1 --2- 172.21.15.27:0/3763087574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95a80c2720 0x7f95a806fa60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f959c009870 tx=0x7f959c02b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338bf0a18296eb5b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.495+0000 7f95ad06f640 1 -- 172.21.15.27:0/3763087574 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c031640 con 0x7f95a80c2720 2023-12-19T18:27:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.495+0000 7f95ad06f640 1 -- 172.21.15.27:0/3763087574 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c0317a0 con 0x7f95a80c2720 2023-12-19T18:27:51.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.495+0000 7f95ad06f640 1 -- 172.21.15.27:0/3763087574 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c031b30 con 0x7f95a80c2720 2023-12-19T18:27:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.496+0000 7f95b02d5640 1 -- 172.21.15.27:0/3763087574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95a80c2720 msgr2=0x7f95a806fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.496+0000 7f95b02d5640 1 --2- 172.21.15.27:0/3763087574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95a80c2720 0x7f95a806fa60 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f959c009870 tx=0x7f959c02b8f0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.496+0000 7f95b02d5640 1 -- 172.21.15.27:0/3763087574 shutdown_connections 2023-12-19T18:27:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.496+0000 7f95b02d5640 1 --2- 172.21.15.27:0/3763087574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95a80c2720 0x7f95a806fa60 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.496+0000 7f95b02d5640 1 --2- 172.21.15.27:0/3763087574 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95a815b1a0 0x7f95a806f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.496+0000 7f95b02d5640 1 -- 172.21.15.27:0/3763087574 >> 172.21.15.27:0/3763087574 conn(0x7f95a80cebd0 msgr2=0x7f95a80cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:51.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.496+0000 7f95b02d5640 1 -- 172.21.15.27:0/3763087574 shutdown_connections 2023-12-19T18:27:51.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.496+0000 7f95b02d5640 1 -- 172.21.15.27:0/3763087574 wait complete. 2023-12-19T18:27:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.498+0000 7f95b02d5640 1 Processor -- start 2023-12-19T18:27:51.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.498+0000 7f95b02d5640 1 -- start start 2023-12-19T18:27:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.499+0000 7f95b02d5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95a8071420 0x7f95a80cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.499+0000 7f95b02d5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95a80c2720 0x7f95a81537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.499+0000 7f95b02d5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95a815b1a0 0x7f95a8153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.499+0000 7f95b02d5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a8156900 con 0x7f95a815b1a0 2023-12-19T18:27:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.499+0000 7f95b02d5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95a80ce460 con 0x7f95a8071420 2023-12-19T18:27:51.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.499+0000 7f95b02d5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95a8156620 con 0x7f95a80c2720 2023-12-19T18:27:51.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.499+0000 7f95ae872640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95a815b1a0 0x7f95a8153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.500+0000 7f95ae872640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95a815b1a0 0x7f95a8153ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42986/0 (socket says 172.21.15.27:42986) 2023-12-19T18:27:51.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.500+0000 7f95ae872640 1 -- 172.21.15.27:0/1622538501 learned_addr learned my addr 172.21.15.27:0/1622538501 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:51.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.500+0000 7f95ae872640 1 -- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95a80c2720 msgr2=0x7f95a81537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.500+0000 7f95ae872640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95a80c2720 0x7f95a81537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.500+0000 7f95ae872640 1 -- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95a8071420 msgr2=0x7f95a80cdf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.500+0000 7f95ae071640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95a8071420 0x7f95a80cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.500+0000 7f95ae872640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95a8071420 0x7f95a80cdf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.500+0000 7f95ae872640 1 -- 172.21.15.27:0/1622538501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c0094d0 con 0x7f95a815b1a0 2023-12-19T18:27:51.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.500+0000 7f95ae071640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95a8071420 0x7f95a80cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:51.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.500+0000 7f95ae872640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95a815b1a0 0x7f95a8153ce0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f95a000c390 tx=0x7f95a000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:51.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.501+0000 7f95a6ffd640 1 -- 172.21.15.27:0/1622538501 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a00052d0 con 0x7f95a815b1a0 2023-12-19T18:27:51.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.501+0000 7f95a6ffd640 1 -- 172.21.15.27:0/1622538501 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a0005430 con 0x7f95a815b1a0 2023-12-19T18:27:51.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.501+0000 7f95a6ffd640 1 -- 172.21.15.27:0/1622538501 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a0003810 con 0x7f95a815b1a0 2023-12-19T18:27:51.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.501+0000 7f95b02d5640 1 -- 172.21.15.27:0/1622538501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a8143dc0 con 0x7f95a815b1a0 2023-12-19T18:27:51.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.501+0000 7f95b02d5640 1 -- 172.21.15.27:0/1622538501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95a8144380 con 0x7f95a815b1a0 2023-12-19T18:27:51.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.502+0000 7f95b02d5640 1 -- 172.21.15.27:0/1622538501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95a8143c20 con 0x7f95a815b1a0 2023-12-19T18:27:51.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.503+0000 7f95a6ffd640 1 -- 172.21.15.27:0/1622538501 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f95a0003970 con 0x7f95a815b1a0 2023-12-19T18:27:51.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.504+0000 7f95a6ffd640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f959406a430 0x7f959406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.504+0000 7f95a6ffd640 1 -- 172.21.15.27:0/1622538501 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f95a0087ce0 con 0x7f95a815b1a0 2023-12-19T18:27:51.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.504+0000 7f95a6ffd640 1 -- 172.21.15.27:0/1622538501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=190}) v3 -- 0x7f9594072020 con 0x7f95a815b1a0 2023-12-19T18:27:51.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.505+0000 7f95ae071640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f959406a430 0x7f959406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.506+0000 7f95ae071640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f959406a430 0x7f959406c8f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9598007330 tx=0x7f959800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:51.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.508+0000 7f95a6ffd640 1 -- 172.21.15.27:0/1622538501 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f95a0017060 con 0x7f95a815b1a0 2023-12-19T18:27:51.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.705+0000 7f95b02d5640 1 -- 172.21.15.27:0/1622538501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f95a806bc80 con 0x7f95a815b1a0 2023-12-19T18:27:51.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.705+0000 7f95a6ffd640 1 -- 172.21.15.27:0/1622538501 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v189) v1 ==== 100+0+11 (secure 0 0 0) 0x7f95a806bc80 con 0x7f95a815b1a0 2023-12-19T18:27:51.706 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:27:51.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.708+0000 7f95b02d5640 1 -- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f959406a430 msgr2=0x7f959406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.708+0000 7f95b02d5640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f959406a430 0x7f959406c8f0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9598007330 tx=0x7f959800b040 comp rx=0 tx=0).stop 2023-12-19T18:27:51.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.708+0000 7f95b02d5640 1 -- 172.21.15.27:0/1622538501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95a815b1a0 msgr2=0x7f95a8153ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.708+0000 7f95b02d5640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95a815b1a0 0x7f95a8153ce0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f95a000c390 tx=0x7f95a000c850 comp rx=0 tx=0).stop 2023-12-19T18:27:51.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.708+0000 7f95b02d5640 1 -- 172.21.15.27:0/1622538501 shutdown_connections 2023-12-19T18:27:51.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.708+0000 7f95b02d5640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f959406a430 0x7f959406c8f0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.708+0000 7f95b02d5640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95a815b1a0 0x7f95a8153ce0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.708+0000 7f95b02d5640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f95a80c2720 0x7f95a81537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.708+0000 7f95b02d5640 1 --2- 172.21.15.27:0/1622538501 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f95a8071420 0x7f95a80cdf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.708+0000 7f95b02d5640 1 -- 172.21.15.27:0/1622538501 >> 172.21.15.27:0/1622538501 conn(0x7f95a80cebd0 msgr2=0x7f95a80ae650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:51.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.708+0000 7f95b02d5640 1 -- 172.21.15.27:0/1622538501 shutdown_connections 2023-12-19T18:27:51.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.708+0000 7f95b02d5640 1 -- 172.21.15.27:0/1622538501 wait complete. 2023-12-19T18:27:51.729 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:27:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.965+0000 7f0c7813e640 1 Processor -- start 2023-12-19T18:27:51.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.965+0000 7f0c7813e640 1 -- start start 2023-12-19T18:27:51.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.967+0000 7f0c7813e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c700cbce0 0x7f0c700ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.967+0000 7f0c7813e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0c700adc20 0x7f0c7006b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.967+0000 7f0c7813e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c700cb850 con 0x7f0c7015af70 2023-12-19T18:27:51.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.967+0000 7f0c7813e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c7015b350 con 0x7f0c700adc20 2023-12-19T18:27:51.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.967+0000 7f0c7813e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700cac40 con 0x7f0c700cbce0 2023-12-19T18:27:51.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.967+0000 7f0c75eda640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f0c7015af70 0x7f0c700ad510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53632/0 (socket says 172.21.15.27:53632) 2023-12-19T18:27:51.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.967+0000 7f0c75eda640 1 -- 172.21.15.27:0/2087423743 learned_addr learned my addr 172.21.15.27:0/2087423743 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:51.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.967+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524923537 0 0) 0x7f0c700cb850 con 0x7f0c7015af70 2023-12-19T18:27:51.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c5c003520 con 0x7f0c7015af70 2023-12-19T18:27:51.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c766db640 1 --2- 172.21.15.27:0/2087423743 >> v2:172.21.15.192:3300/0 conn(0x7f0c700adc20 0x7f0c7006b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c756d9640 1 --2- 172.21.15.27:0/2087423743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c700cbce0 0x7f0c700ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 981688475 0 0) 0x7f0c5c003520 con 0x7f0c7015af70 2023-12-19T18:27:51.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c700cbce0 msgr2=0x7f0c700ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:51.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 --2- 172.21.15.27:0/2087423743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c700cbce0 0x7f0c700ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 >> v2:172.21.15.192:3300/0 conn(0x7f0c700adc20 msgr2=0x7f0c7006b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 --2- 172.21.15.27:0/2087423743 >> v2:172.21.15.192:3300/0 conn(0x7f0c700adc20 0x7f0c7006b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c700caf20 con 0x7f0c7015af70 2023-12-19T18:27:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f0c6c001670 con 0x7f0c7015af70 2023-12-19T18:27:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 >> v1:172.21.15.27:6789/0 conn(0x7f0c7015af70 legacy=0x7f0c700ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 --2- 172.21.15.27:0/2087423743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c700cbce0 0x7f0c5c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 --2- 172.21.15.27:0/2087423743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c5c003fa0 0x7f0c5c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 --2- 172.21.15.27:0/2087423743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c5c004d00 0x7f0c5c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7015b350 con 0x7f0c5c004d00 2023-12-19T18:27:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c700cac40 con 0x7f0c5c003fa0 2023-12-19T18:27:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.968+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c5c00ba80 con 0x7f0c700cbce0 2023-12-19T18:27:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.969+0000 7f0c766db640 1 --2- 172.21.15.27:0/2087423743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c5c004d00 0x7f0c5c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.969+0000 7f0c756d9640 1 --2- 172.21.15.27:0/2087423743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c5c003fa0 0x7f0c5c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.969+0000 7f0c75eda640 1 --2- 172.21.15.27:0/2087423743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c700cbce0 0x7f0c5c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.969+0000 7f0c766db640 1 -- 172.21.15.27:0/2087423743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c700cbce0 msgr2=0x7f0c5c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:51.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.969+0000 7f0c766db640 1 --2- 172.21.15.27:0/2087423743 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c700cbce0 0x7f0c5c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.969+0000 7f0c766db640 1 -- 172.21.15.27:0/2087423743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c5c003fa0 msgr2=0x7f0c5c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:51.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.969+0000 7f0c766db640 1 --2- 172.21.15.27:0/2087423743 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c5c003fa0 0x7f0c5c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.971 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.969+0000 7f0c766db640 1 -- 172.21.15.27:0/2087423743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c003520 con 0x7f0c5c004d00 2023-12-19T18:27:51.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.970+0000 7f0c766db640 1 --2- 172.21.15.27:0/2087423743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c5c004d00 0x7f0c5c005180 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f0c68007360 tx=0x7f0c5c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f6ec9f1675eaedb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:51.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.970+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6802ccf0 con 0x7f0c5c004d00 2023-12-19T18:27:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.970+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c6802ce50 con 0x7f0c5c004d00 2023-12-19T18:27:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.970+0000 7f0c74ed8640 1 -- 172.21.15.27:0/2087423743 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c68034660 con 0x7f0c5c004d00 2023-12-19T18:27:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.971+0000 7f0c7813e640 1 -- 172.21.15.27:0/2087423743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c5c004d00 msgr2=0x7f0c5c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.971+0000 7f0c7813e640 1 --2- 172.21.15.27:0/2087423743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c5c004d00 0x7f0c5c005180 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f0c68007360 tx=0x7f0c5c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.971+0000 7f0c766db640 1 -- 172.21.15.27:0/2087423743 reap_dead start 2023-12-19T18:27:51.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.971+0000 7f0c7813e640 1 -- 172.21.15.27:0/2087423743 shutdown_connections 2023-12-19T18:27:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.971+0000 7f0c7813e640 1 -- 172.21.15.27:0/2087423743 >> 172.21.15.27:0/2087423743 conn(0x7f0c7006a330 msgr2=0x7f0c700b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.971+0000 7f0c7813e640 1 -- 172.21.15.27:0/2087423743 shutdown_connections 2023-12-19T18:27:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.971+0000 7f0c7813e640 1 -- 172.21.15.27:0/2087423743 wait complete. 2023-12-19T18:27:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.973+0000 7f0c7813e640 1 Processor -- start 2023-12-19T18:27:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.973+0000 7f0c7813e640 1 -- start start 2023-12-19T18:27:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.974+0000 7f0c7813e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c5c004d00 0x7f0c701ee040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.974+0000 7f0c7813e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7015bea0 0x7f0c701f0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.974+0000 7f0c7813e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c700b3c60 0x7f0c70070d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.974+0000 7f0c7813e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c701f13e0 con 0x7f0c7015bea0 2023-12-19T18:27:51.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.974+0000 7f0c7813e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7015c410 con 0x7f0c700b3c60 2023-12-19T18:27:51.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.974+0000 7f0c7813e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7015c6f0 con 0x7f0c5c004d00 2023-12-19T18:27:51.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.975+0000 7f0c756d9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7015bea0 0x7f0c701f0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.975+0000 7f0c756d9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7015bea0 0x7f0c701f0ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:43002/0 (socket says 172.21.15.27:43002) 2023-12-19T18:27:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.975+0000 7f0c756d9640 1 -- 172.21.15.27:0/2858352317 learned_addr learned my addr 172.21.15.27:0/2858352317 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.975+0000 7f0c766db640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c700b3c60 0x7f0c70070d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.975+0000 7f0c75eda640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c5c004d00 0x7f0c701ee040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.975+0000 7f0c756d9640 1 -- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c5c004d00 msgr2=0x7f0c701ee040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.975+0000 7f0c756d9640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c5c004d00 0x7f0c701ee040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.975+0000 7f0c756d9640 1 -- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c700b3c60 msgr2=0x7f0c70070d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.975+0000 7f0c756d9640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c700b3c60 0x7f0c70070d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.975+0000 7f0c756d9640 1 -- 172.21.15.27:0/2858352317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c7015c9d0 con 0x7f0c7015bea0 2023-12-19T18:27:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.976+0000 7f0c756d9640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7015bea0 0x7f0c701f0ea0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f0c60013b30 tx=0x7f0c6000bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:51.980 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.976+0000 7f0c667fc640 1 -- 172.21.15.27:0/2858352317 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c60011be0 con 0x7f0c7015bea0 2023-12-19T18:27:51.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.976+0000 7f0c667fc640 1 -- 172.21.15.27:0/2858352317 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c60011d40 con 0x7f0c7015bea0 2023-12-19T18:27:51.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.976+0000 7f0c667fc640 1 -- 172.21.15.27:0/2858352317 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c6001a660 con 0x7f0c7015bea0 2023-12-19T18:27:51.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.976+0000 7f0c7813e640 1 -- 172.21.15.27:0/2858352317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7015ccb0 con 0x7f0c7015bea0 2023-12-19T18:27:51.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.976+0000 7f0c7813e640 1 -- 172.21.15.27:0/2858352317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c700b40e0 con 0x7f0c7015bea0 2023-12-19T18:27:51.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.977+0000 7f0c7813e640 1 -- 172.21.15.27:0/2858352317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c700b42a0 con 0x7f0c7015bea0 2023-12-19T18:27:51.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.979+0000 7f0c667fc640 1 -- 172.21.15.27:0/2858352317 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0c6001a7c0 con 0x7f0c7015bea0 2023-12-19T18:27:51.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.980+0000 7f0c667fc640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0c5406a430 0x7f0c5406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:51.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.980+0000 7f0c667fc640 1 -- 172.21.15.27:0/2858352317 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f0c6008d310 con 0x7f0c7015bea0 2023-12-19T18:27:51.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.980+0000 7f0c75eda640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0c5406a430 0x7f0c5406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:51.982 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.981+0000 7f0c75eda640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0c5406a430 0x7f0c5406c8f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c001050 tx=0x7f0c6c001190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:51.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:51.984+0000 7f0c667fc640 1 -- 172.21.15.27:0/2858352317 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0c600588b0 con 0x7f0c7015bea0 2023-12-19T18:27:52.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.219+0000 7f0c7813e640 1 -- 172.21.15.27:0/2858352317 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c700cb1d0 con 0x7f0c7015bea0 2023-12-19T18:27:52.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.219+0000 7f0c667fc640 1 -- 172.21.15.27:0/2858352317 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4719 (secure 0 0 0) 0x7f0c6005bed0 con 0x7f0c7015bea0 2023-12-19T18:27:52.219 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:27:52.220 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3563146395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3563146395},{"type":"v1","addr":"172.21.15.192:6836","nonce":3563146395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3445095528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3445095528},{"type":"v1","addr":"172.21.15.192:6837","nonce":3445095528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3536033217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3536033217},{"type":"v1","addr":"172.21.15.27:6833","nonce":3536033217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:26:38.483498+0000","modified":"2023-12-19T18:26:42.008767+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2895889621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2895889621},{"type":"v1","addr":"172.21.15.27:6835","nonce":2895889621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-19T18:27:52.220 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 77 2023-12-19T18:27:52.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.222+0000 7f0c7813e640 1 -- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0c5406a430 msgr2=0x7f0c5406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:52.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.222+0000 7f0c7813e640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0c5406a430 0x7f0c5406c8f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0c6c001050 tx=0x7f0c6c001190 comp rx=0 tx=0).stop 2023-12-19T18:27:52.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.222+0000 7f0c7813e640 1 -- 172.21.15.27:0/2858352317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7015bea0 msgr2=0x7f0c701f0ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:52.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.222+0000 7f0c7813e640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7015bea0 0x7f0c701f0ea0 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f0c60013b30 tx=0x7f0c6000bec0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.223+0000 7f0c7813e640 1 -- 172.21.15.27:0/2858352317 shutdown_connections 2023-12-19T18:27:52.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.223+0000 7f0c7813e640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0c5406a430 0x7f0c5406c8f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.223+0000 7f0c7813e640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0c700b3c60 0x7f0c70070d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.223+0000 7f0c7813e640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c7015bea0 0x7f0c701f0ea0 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.223+0000 7f0c7813e640 1 --2- 172.21.15.27:0/2858352317 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0c5c004d00 0x7f0c701ee040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.223+0000 7f0c7813e640 1 -- 172.21.15.27:0/2858352317 >> 172.21.15.27:0/2858352317 conn(0x7f0c7006a330 msgr2=0x7f0c700c9170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:52.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.223+0000 7f0c7813e640 1 -- 172.21.15.27:0/2858352317 shutdown_connections 2023-12-19T18:27:52.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.223+0000 7f0c7813e640 1 -- 172.21.15.27:0/2858352317 wait complete. 2023-12-19T18:27:52.244 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:27:52.468 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.468+0000 7f6039e83640 1 Processor -- start 2023-12-19T18:27:52.469 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.469+0000 7f6039e83640 1 -- start start 2023-12-19T18:27:52.471 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.470+0000 7f6039e83640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f60341604d0 0x7f60340b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:52.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.470+0000 7f6039e83640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60340d2350 0x7f60340b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:52.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.470+0000 7f6039e83640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60340b5e90 con 0x7f60340d0300 2023-12-19T18:27:52.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.470+0000 7f6039e83640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60340d2b30 con 0x7f60341604d0 2023-12-19T18:27:52.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.470+0000 7f6039e83640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60340d2850 con 0x7f60340d2350 2023-12-19T18:27:52.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.471+0000 7f6038e81640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f60341604d0 0x7f60340b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:52.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.471+0000 7f6038e81640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f60341604d0 0x7f60340b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38120/0 (socket says 172.21.15.27:38120) 2023-12-19T18:27:52.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.471+0000 7f6038e81640 1 -- 172.21.15.27:0/925735250 learned_addr learned my addr 172.21.15.27:0/925735250 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:52.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.471+0000 7f6033fff640 1 --2- 172.21.15.27:0/925735250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60340d2350 0x7f60340b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:52.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.471+0000 7f6038e81640 1 -- 172.21.15.27:0/925735250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60340d2350 msgr2=0x7f60340b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:52.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.471+0000 7f6038e81640 1 --2- 172.21.15.27:0/925735250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60340d2350 0x7f60340b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.471+0000 7f6038e81640 1 -- 172.21.15.27:0/925735250 >> v1:172.21.15.27:6789/0 conn(0x7f60340d0300 legacy=0x7f60340d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:52.473 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.471+0000 7f6038e81640 1 -- 172.21.15.27:0/925735250 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60340cded0 con 0x7f60341604d0 2023-12-19T18:27:52.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.473+0000 7f6038e81640 1 --2- 172.21.15.27:0/925735250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60341604d0 0x7f60340b47e0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f6028009870 tx=0x7f602802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee77aea4acfa6fcb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:52.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.473+0000 7f60337fe640 1 -- 172.21.15.27:0/925735250 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028030840 con 0x7f60341604d0 2023-12-19T18:27:52.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.473+0000 7f60337fe640 1 -- 172.21.15.27:0/925735250 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60280309a0 con 0x7f60341604d0 2023-12-19T18:27:52.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.473+0000 7f60337fe640 1 -- 172.21.15.27:0/925735250 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028030d30 con 0x7f60341604d0 2023-12-19T18:27:52.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.474+0000 7f6039e83640 1 -- 172.21.15.27:0/925735250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60341604d0 msgr2=0x7f60340b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:52.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.474+0000 7f6039e83640 1 --2- 172.21.15.27:0/925735250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60341604d0 0x7f60340b47e0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f6028009870 tx=0x7f602802b980 comp rx=0 tx=0).stop 2023-12-19T18:27:52.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.474+0000 7f6039e83640 1 -- 172.21.15.27:0/925735250 shutdown_connections 2023-12-19T18:27:52.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.474+0000 7f6039e83640 1 --2- 172.21.15.27:0/925735250 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60340d2350 0x7f60340b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.474+0000 7f6039e83640 1 --2- 172.21.15.27:0/925735250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60341604d0 0x7f60340b47e0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.474+0000 7f6039e83640 1 -- 172.21.15.27:0/925735250 >> 172.21.15.27:0/925735250 conn(0x7f60340cf7e0 msgr2=0x7f60340cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:52.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.474+0000 7f6039e83640 1 -- 172.21.15.27:0/925735250 shutdown_connections 2023-12-19T18:27:52.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.474+0000 7f6039e83640 1 -- 172.21.15.27:0/925735250 wait complete. 2023-12-19T18:27:52.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.476+0000 7f6039e83640 1 Processor -- start 2023-12-19T18:27:52.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.476+0000 7f6039e83640 1 -- start start 2023-12-19T18:27:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.478+0000 7f6039e83640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60340d0300 0x7f60340cc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.478+0000 7f6039e83640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60340d2350 0x7f6034156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.478+0000 7f6039e83640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60341604d0 0x7f6034156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.478+0000 7f6039e83640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60341f3a20 con 0x7f60340d0300 2023-12-19T18:27:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.478+0000 7f6039e83640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60341581e0 con 0x7f60341604d0 2023-12-19T18:27:52.478 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.478+0000 7f6039e83640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60340cc7a0 con 0x7f60340d2350 2023-12-19T18:27:52.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.478+0000 7f6038e81640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60340d0300 0x7f60340cc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:52.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.478+0000 7f6038e81640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60340d0300 0x7f60340cc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38128/0 (socket says 172.21.15.27:38128) 2023-12-19T18:27:52.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.478+0000 7f6038e81640 1 -- 172.21.15.27:0/2861176198 learned_addr learned my addr 172.21.15.27:0/2861176198 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.479+0000 7f6038e81640 1 -- 172.21.15.27:0/2861176198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60340d2350 msgr2=0x7f6034156140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.479+0000 7f6038e81640 1 --2- 172.21.15.27:0/2861176198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60340d2350 0x7f6034156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.479+0000 7f6038e81640 1 -- 172.21.15.27:0/2861176198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60341604d0 msgr2=0x7f6034156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.479+0000 7f6038e81640 1 --2- 172.21.15.27:0/2861176198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60341604d0 0x7f6034156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.479+0000 7f6038e81640 1 -- 172.21.15.27:0/2861176198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60280094d0 con 0x7f60340d0300 2023-12-19T18:27:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.479+0000 7f6038e81640 1 --2- 172.21.15.27:0/2861176198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60340d0300 0x7f60340cc260 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f6028000c00 tx=0x7f6028004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.479+0000 7f60317fa640 1 -- 172.21.15.27:0/2861176198 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028039c90 con 0x7f60340d0300 2023-12-19T18:27:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.479+0000 7f60317fa640 1 -- 172.21.15.27:0/2861176198 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6028039df0 con 0x7f60340d0300 2023-12-19T18:27:52.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.479+0000 7f6039e83640 1 -- 172.21.15.27:0/2861176198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6034158480 con 0x7f60340d0300 2023-12-19T18:27:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.479+0000 7f60317fa640 1 -- 172.21.15.27:0/2861176198 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6028043640 con 0x7f60340d0300 2023-12-19T18:27:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.480+0000 7f6039e83640 1 -- 172.21.15.27:0/2861176198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6034158a40 con 0x7f60340d0300 2023-12-19T18:27:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.481+0000 7f6039e83640 1 -- 172.21.15.27:0/2861176198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6034066f80 con 0x7f60340d0300 2023-12-19T18:27:52.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.482+0000 7f60317fa640 1 -- 172.21.15.27:0/2861176198 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f60280437a0 con 0x7f60340d0300 2023-12-19T18:27:52.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.483+0000 7f60317fa640 1 --2- 172.21.15.27:0/2861176198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f601806a430 0x7f601806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:52.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.483+0000 7f60317fa640 1 -- 172.21.15.27:0/2861176198 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f602803e070 con 0x7f60340d0300 2023-12-19T18:27:52.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.483+0000 7f6033fff640 1 --2- 172.21.15.27:0/2861176198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f601806a430 0x7f601806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:52.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.484+0000 7f6033fff640 1 --2- 172.21.15.27:0/2861176198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f601806a430 0x7f601806c8f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f60200098f0 tx=0x7f6020008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:52.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.487+0000 7f60317fa640 1 -- 172.21.15.27:0/2861176198 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6028076fe0 con 0x7f60340d0300 2023-12-19T18:27:52.672 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.673+0000 7f6039e83640 1 -- 172.21.15.27:0/2861176198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f603404e840 con 0x7f60340d0300 2023-12-19T18:27:52.675 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.676+0000 7f60317fa640 1 -- 172.21.15.27:0/2861176198 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+16217 (secure 0 0 0) 0x7f602807a600 con 0x7f60340d0300 2023-12-19T18:27:52.676 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:27:52.676 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":190,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:27:51.817442+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:26:36.428093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T18:26:37.523993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1e","mappings":[{"from":2,"to":5}]},{"pgid":"21.33","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:27:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.678+0000 7f6039e83640 1 -- 172.21.15.27:0/2861176198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f601806a430 msgr2=0x7f601806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.678+0000 7f6039e83640 1 --2- 172.21.15.27:0/2861176198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f601806a430 0x7f601806c8f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f60200098f0 tx=0x7f6020008040 comp rx=0 tx=0).stop 2023-12-19T18:27:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.678+0000 7f6039e83640 1 -- 172.21.15.27:0/2861176198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60340d0300 msgr2=0x7f60340cc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:52.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.678+0000 7f6039e83640 1 --2- 172.21.15.27:0/2861176198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60340d0300 0x7f60340cc260 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f6028000c00 tx=0x7f6028004730 comp rx=0 tx=0).stop 2023-12-19T18:27:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.678+0000 7f6039e83640 1 -- 172.21.15.27:0/2861176198 shutdown_connections 2023-12-19T18:27:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.678+0000 7f6039e83640 1 --2- 172.21.15.27:0/2861176198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f601806a430 0x7f601806c8f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.678+0000 7f6039e83640 1 --2- 172.21.15.27:0/2861176198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f60341604d0 0x7f6034156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.678+0000 7f6039e83640 1 --2- 172.21.15.27:0/2861176198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f60340d2350 0x7f6034156140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.678+0000 7f6039e83640 1 --2- 172.21.15.27:0/2861176198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60340d0300 0x7f60340cc260 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.678+0000 7f6039e83640 1 -- 172.21.15.27:0/2861176198 >> 172.21.15.27:0/2861176198 conn(0x7f60340cf7e0 msgr2=0x7f60340b4850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.678+0000 7f6039e83640 1 -- 172.21.15.27:0/2861176198 shutdown_connections 2023-12-19T18:27:52.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.678+0000 7f6039e83640 1 -- 172.21.15.27:0/2861176198 wait complete. 2023-12-19T18:27:52.697 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:27:52.697 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:27:52.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.923+0000 7f5805be1640 1 Processor -- start 2023-12-19T18:27:52.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.924+0000 7f5805be1640 1 -- start start 2023-12-19T18:27:52.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.925+0000 7f5805be1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58000b4ef0 0x7f58000b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:52.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.925+0000 7f5805be1640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f58000d2350 0x7f58000b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:52.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.925+0000 7f5805be1640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58000b5e90 con 0x7f58001604d0 2023-12-19T18:27:52.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.925+0000 7f5805be1640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58000d2b30 con 0x7f58000d2350 2023-12-19T18:27:52.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.925+0000 7f5805be1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58000d2850 con 0x7f58000b4ef0 2023-12-19T18:27:52.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.925+0000 7f5804bdf640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f58001604d0 0x7f58000b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47676/0 (socket says 172.21.15.27:47676) 2023-12-19T18:27:52.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.926+0000 7f5804bdf640 1 -- 172.21.15.27:0/618734491 learned_addr learned my addr 172.21.15.27:0/618734491 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:52.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.926+0000 7f58053e0640 1 --2- 172.21.15.27:0/618734491 >> v2:172.21.15.27:3300/0 conn(0x7f58000d2350 0x7f58000b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:52.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.926+0000 7f57fffff640 1 --2- 172.21.15.27:0/618734491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58000b4ef0 0x7f58000b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:52.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.926+0000 7f58053e0640 1 -- 172.21.15.27:0/618734491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58000b4ef0 msgr2=0x7f58000b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:52.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.926+0000 7f58053e0640 1 --2- 172.21.15.27:0/618734491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58000b4ef0 0x7f58000b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.926+0000 7f58053e0640 1 -- 172.21.15.27:0/618734491 >> v1:172.21.15.27:6789/0 conn(0x7f58001604d0 legacy=0x7f58000b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:52.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.926+0000 7f58053e0640 1 -- 172.21.15.27:0/618734491 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58000cded0 con 0x7f58000d2350 2023-12-19T18:27:52.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.927+0000 7f58053e0640 1 --2- 172.21.15.27:0/618734491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58000d2350 0x7f58000b6840 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f57f80097a0 tx=0x7f57f802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2151bcfbf555f8ac server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:52.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.929+0000 7f57ff7fe640 1 -- 172.21.15.27:0/618734491 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8031830 con 0x7f58000d2350 2023-12-19T18:27:52.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.929+0000 7f57ff7fe640 1 -- 172.21.15.27:0/618734491 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f8031990 con 0x7f58000d2350 2023-12-19T18:27:52.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.929+0000 7f57ff7fe640 1 -- 172.21.15.27:0/618734491 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f8031d20 con 0x7f58000d2350 2023-12-19T18:27:52.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.930+0000 7f5805be1640 1 -- 172.21.15.27:0/618734491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58000d2350 msgr2=0x7f58000b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:52.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.930+0000 7f5805be1640 1 --2- 172.21.15.27:0/618734491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58000d2350 0x7f58000b6840 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f57f80097a0 tx=0x7f57f802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.930+0000 7f5805be1640 1 -- 172.21.15.27:0/618734491 shutdown_connections 2023-12-19T18:27:52.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.930+0000 7f5805be1640 1 --2- 172.21.15.27:0/618734491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58000d2350 0x7f58000b6840 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.930+0000 7f5805be1640 1 --2- 172.21.15.27:0/618734491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58000b4ef0 0x7f58000b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.930+0000 7f5805be1640 1 -- 172.21.15.27:0/618734491 >> 172.21.15.27:0/618734491 conn(0x7f58000cf7e0 msgr2=0x7f58000cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:52.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.930+0000 7f5805be1640 1 -- 172.21.15.27:0/618734491 shutdown_connections 2023-12-19T18:27:52.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.930+0000 7f5805be1640 1 -- 172.21.15.27:0/618734491 wait complete. 2023-12-19T18:27:52.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.932+0000 7f5805be1640 1 Processor -- start 2023-12-19T18:27:52.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.933+0000 7f5805be1640 1 -- start start 2023-12-19T18:27:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.934+0000 7f5805be1640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58000b4ef0 0x7f580015ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.934+0000 7f5805be1640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58000d2350 0x7f58000c8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.934+0000 7f5805be1640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58001604d0 0x7f5800154930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.934+0000 7f5805be1640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580015a670 con 0x7f58001604d0 2023-12-19T18:27:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.934+0000 7f5805be1640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58001f3460 con 0x7f58000b4ef0 2023-12-19T18:27:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.934+0000 7f5805be1640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f580015a020 con 0x7f58000d2350 2023-12-19T18:27:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.934+0000 7f58053e0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58001604d0 0x7f5800154930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.934+0000 7f58053e0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58001604d0 0x7f5800154930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38144/0 (socket says 172.21.15.27:38144) 2023-12-19T18:27:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.934+0000 7f58053e0640 1 -- 172.21.15.27:0/3965887723 learned_addr learned my addr 172.21.15.27:0/3965887723 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.934+0000 7f5804bdf640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58000b4ef0 0x7f580015ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.935+0000 7f57fffff640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58000d2350 0x7f58000c8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.935+0000 7f58053e0640 1 -- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58000d2350 msgr2=0x7f58000c8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:52.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.935+0000 7f58053e0640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58000d2350 0x7f58000c8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.935+0000 7f58053e0640 1 -- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58000b4ef0 msgr2=0x7f580015ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:52.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.935+0000 7f58053e0640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58000b4ef0 0x7f580015ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:52.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.935+0000 7f58053e0640 1 -- 172.21.15.27:0/3965887723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f8009400 con 0x7f58001604d0 2023-12-19T18:27:52.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.935+0000 7f58053e0640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58001604d0 0x7f5800154930 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f57f80098c0 tx=0x7f57f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:52.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.935+0000 7f57fd7fa640 1 -- 172.21.15.27:0/3965887723 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f80317d0 con 0x7f58001604d0 2023-12-19T18:27:52.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.935+0000 7f57fd7fa640 1 -- 172.21.15.27:0/3965887723 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f803b9f0 con 0x7f58001604d0 2023-12-19T18:27:52.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.936+0000 7f57fd7fa640 1 -- 172.21.15.27:0/3965887723 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f803bda0 con 0x7f58001604d0 2023-12-19T18:27:52.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.936+0000 7f5805be1640 1 -- 172.21.15.27:0/3965887723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580015a300 con 0x7f58001604d0 2023-12-19T18:27:52.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.936+0000 7f5805be1640 1 -- 172.21.15.27:0/3965887723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580015b040 con 0x7f58001604d0 2023-12-19T18:27:52.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.937+0000 7f5805be1640 1 -- 172.21.15.27:0/3965887723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5800066f80 con 0x7f58001604d0 2023-12-19T18:27:52.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.938+0000 7f57fd7fa640 1 -- 172.21.15.27:0/3965887723 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f57f80453f0 con 0x7f58001604d0 2023-12-19T18:27:52.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.940+0000 7f57fd7fa640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f57ec06a430 0x7f57ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:52.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.940+0000 7f57fd7fa640 1 -- 172.21.15.27:0/3965887723 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f57f80aabb0 con 0x7f58001604d0 2023-12-19T18:27:52.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.940+0000 7f5804bdf640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f57ec06a430 0x7f57ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.942+0000 7f5804bdf640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f57ec06a430 0x7f57ec06c8f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005c00 tx=0x7f57f0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:52.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:52.944+0000 7f57fd7fa640 1 -- 172.21.15.27:0/3965887723 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f57f80761f0 con 0x7f58001604d0 2023-12-19T18:27:53.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.174+0000 7f5805be1640 1 -- 172.21.15.27:0/3965887723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f580004e840 con 0x7f58001604d0 2023-12-19T18:27:53.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.175+0000 7f57fd7fa640 1 -- 172.21.15.27:0/3965887723 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4719 (secure 0 0 0) 0x7f57f8079810 con 0x7f58001604d0 2023-12-19T18:27:53.175 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:27:53.175 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3563146395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3563146395},{"type":"v1","addr":"172.21.15.192:6836","nonce":3563146395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3445095528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3445095528},{"type":"v1","addr":"172.21.15.192:6837","nonce":3445095528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3536033217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3536033217},{"type":"v1","addr":"172.21.15.27:6833","nonce":3536033217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:26:38.483498+0000","modified":"2023-12-19T18:26:42.008767+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2895889621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2895889621},{"type":"v1","addr":"172.21.15.27:6835","nonce":2895889621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-19T18:27:53.176 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 77 2023-12-19T18:27:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.178+0000 7f5805be1640 1 -- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f57ec06a430 msgr2=0x7f57ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.178+0000 7f5805be1640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f57ec06a430 0x7f57ec06c8f0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f57f0005c00 tx=0x7f57f0009040 comp rx=0 tx=0).stop 2023-12-19T18:27:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.178+0000 7f5805be1640 1 -- 172.21.15.27:0/3965887723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58001604d0 msgr2=0x7f5800154930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.178+0000 7f5805be1640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58001604d0 0x7f5800154930 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f57f80098c0 tx=0x7f57f80046c0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.178+0000 7f5805be1640 1 -- 172.21.15.27:0/3965887723 shutdown_connections 2023-12-19T18:27:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.178+0000 7f5805be1640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f57ec06a430 0x7f57ec06c8f0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.178+0000 7f5805be1640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58001604d0 0x7f5800154930 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.178+0000 7f5805be1640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f58000d2350 0x7f58000c8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.178+0000 7f5805be1640 1 --2- 172.21.15.27:0/3965887723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f58000b4ef0 0x7f580015ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.178+0000 7f5805be1640 1 -- 172.21.15.27:0/3965887723 >> 172.21.15.27:0/3965887723 conn(0x7f58000cf7e0 msgr2=0x7f58000d1820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.178+0000 7f5805be1640 1 -- 172.21.15.27:0/3965887723 shutdown_connections 2023-12-19T18:27:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.178+0000 7f5805be1640 1 -- 172.21.15.27:0/3965887723 wait complete. 2023-12-19T18:27:53.197 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:27:53.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.423+0000 7f239759e640 1 Processor -- start 2023-12-19T18:27:53.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.423+0000 7f239759e640 1 -- start start 2023-12-19T18:27:53.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.425+0000 7f239759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23980d2350 0x7f23980b6a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.425+0000 7f239759e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23980b5e90 con 0x7f23981604d0 2023-12-19T18:27:53.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.425+0000 7f239759e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23980d2850 con 0x7f23980b4ef0 2023-12-19T18:27:53.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.425+0000 7f239759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23980d2b30 con 0x7f23980d2350 2023-12-19T18:27:53.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.425+0000 7f239659c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f23981604d0 0x7f23980b47e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47684/0 (socket says 172.21.15.27:47684) 2023-12-19T18:27:53.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.425+0000 7f239659c640 1 -- 172.21.15.27:0/3193829092 learned_addr learned my addr 172.21.15.27:0/3193829092 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.425+0000 7f2396d9d640 1 --2- 172.21.15.27:0/3193829092 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23980d2350 0x7f23980b6a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.425+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2724620470 0 0) 0x7f23980b5e90 con 0x7f23981604d0 2023-12-19T18:27:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.425+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2384003520 con 0x7f23981604d0 2023-12-19T18:27:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2774266387 0 0) 0x7f2384003520 con 0x7f23981604d0 2023-12-19T18:27:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23980d2350 msgr2=0x7f23980b6a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f239559a640 1 --2- 172.21.15.27:0/3193829092 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23980d2350 0x7f23980b6a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 >> v1:172.21.15.192:6789/0 conn(0x7f23980b4ef0 legacy=0x7f23980b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23980cded0 con 0x7f23981604d0 2023-12-19T18:27:53.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f23880019f0 con 0x7f23981604d0 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 >> v1:172.21.15.27:6789/0 conn(0x7f23981604d0 legacy=0x7f23980b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f239559a640 1 --2- 172.21.15.27:0/3193829092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23840039a0 0x7f2384003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f239559a640 1 --2- 172.21.15.27:0/3193829092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23840047c0 0x7f2384004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f239559a640 1 --2- 172.21.15.27:0/3193829092 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23980d2350 0x7f2384005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23980d2b30 con 0x7f23840047c0 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238400ba60 con 0x7f23840039a0 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f238400bc20 con 0x7f23980d2350 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.426+0000 7f2395d9b640 1 --2- 172.21.15.27:0/3193829092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23840047c0 0x7f2384004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.427+0000 7f239659c640 1 --2- 172.21.15.27:0/3193829092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23840039a0 0x7f2384003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.427+0000 7f2396d9d640 1 --2- 172.21.15.27:0/3193829092 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23980d2350 0x7f2384005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.427+0000 7f2395d9b640 1 -- 172.21.15.27:0/3193829092 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23980d2350 msgr2=0x7f2384005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.427+0000 7f2395d9b640 1 --2- 172.21.15.27:0/3193829092 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23980d2350 0x7f2384005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.427+0000 7f2395d9b640 1 -- 172.21.15.27:0/3193829092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23840039a0 msgr2=0x7f2384003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:53.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.427+0000 7f2395d9b640 1 --2- 172.21.15.27:0/3193829092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23840039a0 0x7f2384003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.427+0000 7f2395d9b640 1 -- 172.21.15.27:0/3193829092 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2384003520 con 0x7f23840047c0 2023-12-19T18:27:53.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.427+0000 7f239659c640 1 --2- 172.21.15.27:0/3193829092 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23840039a0 0x7f2384003dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:27:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.428+0000 7f2395d9b640 1 --2- 172.21.15.27:0/3193829092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23840047c0 0x7f2384004c40 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f2380001720 tx=0x7f23840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f32c94a89af8fdc7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.428+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23800052d0 con 0x7f23840047c0 2023-12-19T18:27:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.428+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f238002e050 con 0x7f23840047c0 2023-12-19T18:27:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.428+0000 7f239559a640 1 -- 172.21.15.27:0/3193829092 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23800035f0 con 0x7f23840047c0 2023-12-19T18:27:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.429+0000 7f239759e640 1 -- 172.21.15.27:0/3193829092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23840047c0 msgr2=0x7f2384004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.429+0000 7f239759e640 1 --2- 172.21.15.27:0/3193829092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23840047c0 0x7f2384004c40 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7f2380001720 tx=0x7f23840057f0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.429+0000 7f2396d9d640 1 -- 172.21.15.27:0/3193829092 reap_dead start 2023-12-19T18:27:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.429+0000 7f239759e640 1 -- 172.21.15.27:0/3193829092 shutdown_connections 2023-12-19T18:27:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.429+0000 7f239759e640 1 -- 172.21.15.27:0/3193829092 >> 172.21.15.27:0/3193829092 conn(0x7f23980cf7e0 msgr2=0x7f23980cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:53.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.429+0000 7f239759e640 1 -- 172.21.15.27:0/3193829092 shutdown_connections 2023-12-19T18:27:53.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.429+0000 7f239759e640 1 -- 172.21.15.27:0/3193829092 wait complete. 2023-12-19T18:27:53.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.431+0000 7f239759e640 1 Processor -- start 2023-12-19T18:27:53.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.431+0000 7f239759e640 1 -- start start 2023-12-19T18:27:53.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.432+0000 7f239759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23840047c0 0x7f23981f2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.432+0000 7f239759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23981f6cf0 0x7f2398152180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.432+0000 7f239759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23980aba10 0x7f23980d0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f239759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23980a9710 con 0x7f23981f6cf0 2023-12-19T18:27:53.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f239759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23981f3010 con 0x7f23980aba10 2023-12-19T18:27:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f239759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23980ac310 con 0x7f23840047c0 2023-12-19T18:27:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f2395d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23981f6cf0 0x7f2398152180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f2395d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23981f6cf0 0x7f2398152180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38168/0 (socket says 172.21.15.27:38168) 2023-12-19T18:27:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f2395d9b640 1 -- 172.21.15.27:0/3929281868 learned_addr learned my addr 172.21.15.27:0/3929281868 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f239659c640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23840047c0 0x7f23981f2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f2396d9d640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23980aba10 0x7f23980d0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f2395d9b640 1 -- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23840047c0 msgr2=0x7f23981f2ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f2395d9b640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23840047c0 0x7f23981f2ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f2395d9b640 1 -- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23980aba10 msgr2=0x7f23980d0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f2395d9b640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23980aba10 0x7f23980d0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f2395d9b640 1 -- 172.21.15.27:0/3929281868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23980a98f0 con 0x7f23981f6cf0 2023-12-19T18:27:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f239659c640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23840047c0 0x7f23981f2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:53.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.433+0000 7f2396d9d640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23980aba10 0x7f23980d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:53.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.434+0000 7f2395d9b640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23981f6cf0 0x7f2398152180 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f2380008fd0 tx=0x7f2380001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:53.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.434+0000 7f238effd640 1 -- 172.21.15.27:0/3929281868 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23800076c0 con 0x7f23981f6cf0 2023-12-19T18:27:53.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.434+0000 7f238effd640 1 -- 172.21.15.27:0/3929281868 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2380007a30 con 0x7f23981f6cf0 2023-12-19T18:27:53.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.434+0000 7f238effd640 1 -- 172.21.15.27:0/3929281868 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2380006c00 con 0x7f23981f6cf0 2023-12-19T18:27:53.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.434+0000 7f239759e640 1 -- 172.21.15.27:0/3929281868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23980a9bd0 con 0x7f23981f6cf0 2023-12-19T18:27:53.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.434+0000 7f239759e640 1 -- 172.21.15.27:0/3929281868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23981526c0 con 0x7f23981f6cf0 2023-12-19T18:27:53.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.435+0000 7f239759e640 1 -- 172.21.15.27:0/3929281868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2398066f80 con 0x7f23981f6cf0 2023-12-19T18:27:53.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.436+0000 7f238effd640 1 -- 172.21.15.27:0/3929281868 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f23800035f0 con 0x7f23981f6cf0 2023-12-19T18:27:53.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.437+0000 7f238effd640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f237806a460 0x7f237806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.437+0000 7f238effd640 1 -- 172.21.15.27:0/3929281868 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f23800aab10 con 0x7f23981f6cf0 2023-12-19T18:27:53.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.438+0000 7f239659c640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f237806a460 0x7f237806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.439+0000 7f239659c640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f237806a460 0x7f237806c920 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f23980c9430 tx=0x7f2388001250 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:53.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.442+0000 7f238effd640 1 -- 172.21.15.27:0/3929281868 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2380079ab0 con 0x7f23981f6cf0 2023-12-19T18:27:53.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.674+0000 7f239759e640 1 -- 172.21.15.27:0/3929281868 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23980a9710 con 0x7f23981f6cf0 2023-12-19T18:27:53.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.676+0000 7f238effd640 1 -- 172.21.15.27:0/3929281868 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 77 v77) v1 ==== 94+0+4719 (secure 0 0 0) 0x7f238004b090 con 0x7f23981f6cf0 2023-12-19T18:27:53.676 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:27:53.676 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":77,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3563146395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3563146395},{"type":"v1","addr":"172.21.15.192:6836","nonce":3563146395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3445095528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3445095528},{"type":"v1","addr":"172.21.15.192:6837","nonce":3445095528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3536033217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3536033217},{"type":"v1","addr":"172.21.15.27:6833","nonce":3536033217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72}],"filesystems":[{"mdsmap":{"epoch":77,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:26:38.483498+0000","modified":"2023-12-19T18:26:42.008767+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8631},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8631":{"gid":8631,"name":"c","rank":0,"incarnation":73,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/2895889621","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":2895889621},{"type":"v1","addr":"172.21.15.27:6835","nonce":2895889621}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-19T18:27:53.676 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 77 2023-12-19T18:27:53.678 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.678+0000 7f239759e640 1 -- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f237806a460 msgr2=0x7f237806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:53.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.678+0000 7f239759e640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f237806a460 0x7f237806c920 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f23980c9430 tx=0x7f2388001250 comp rx=0 tx=0).stop 2023-12-19T18:27:53.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.678+0000 7f239759e640 1 -- 172.21.15.27:0/3929281868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23981f6cf0 msgr2=0x7f2398152180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:53.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.678+0000 7f239759e640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23981f6cf0 0x7f2398152180 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f2380008fd0 tx=0x7f2380001120 comp rx=0 tx=0).stop 2023-12-19T18:27:53.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.678+0000 7f239759e640 1 -- 172.21.15.27:0/3929281868 shutdown_connections 2023-12-19T18:27:53.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.678+0000 7f239759e640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f237806a460 0x7f237806c920 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.678+0000 7f239759e640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f23980aba10 0x7f23980d0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.678+0000 7f239759e640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23981f6cf0 0x7f2398152180 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.678+0000 7f239759e640 1 --2- 172.21.15.27:0/3929281868 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f23840047c0 0x7f23981f2ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.679+0000 7f239759e640 1 -- 172.21.15.27:0/3929281868 >> 172.21.15.27:0/3929281868 conn(0x7f23980cf7e0 msgr2=0x7f23980cbe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:53.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.679+0000 7f239759e640 1 -- 172.21.15.27:0/3929281868 shutdown_connections 2023-12-19T18:27:53.679 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.679+0000 7f239759e640 1 -- 172.21.15.27:0/3929281868 wait complete. 2023-12-19T18:27:53.699 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:27:53.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.926+0000 7f5c05d67640 1 Processor -- start 2023-12-19T18:27:53.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.926+0000 7f5c05d67640 1 -- start start 2023-12-19T18:27:53.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.928+0000 7f5c05d67640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f5c0015ee30 0x7f5c000c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.928+0000 7f5c05d67640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c000afeb0 0x7f5c000c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.928+0000 7f5c05d67640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c000cb350 con 0x7f5c001553a0 2023-12-19T18:27:53.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.928+0000 7f5c05d67640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c000cbcf0 con 0x7f5c0015ee30 2023-12-19T18:27:53.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.928+0000 7f5c05d67640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c000cb630 con 0x7f5c000afeb0 2023-12-19T18:27:53.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.928+0000 7f5c04d65640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5c001553a0 0x7f5c0015e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47690/0 (socket says 172.21.15.27:47690) 2023-12-19T18:27:53.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.928+0000 7f5c04d65640 1 -- 172.21.15.27:0/1115511268 learned_addr learned my addr 172.21.15.27:0/1115511268 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:53.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5c05566640 1 --2- 172.21.15.27:0/1115511268 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c000afeb0 0x7f5c000c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bfffff640 1 --2- 172.21.15.27:0/1115511268 >> v2:172.21.15.192:3300/0 conn(0x7f5c0015ee30 0x7f5c000c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3525186720 0 0) 0x7f5c000cb350 con 0x7f5c001553a0 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bf0003520 con 0x7f5c001553a0 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 167442760 0 0) 0x7f5bf0003520 con 0x7f5c001553a0 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c000afeb0 msgr2=0x7f5c000c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 --2- 172.21.15.27:0/1115511268 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c000afeb0 0x7f5c000c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 >> v2:172.21.15.192:3300/0 conn(0x7f5c0015ee30 msgr2=0x7f5c000c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 --2- 172.21.15.27:0/1115511268 >> v2:172.21.15.192:3300/0 conn(0x7f5c0015ee30 0x7f5c000c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c000cb910 con 0x7f5c001553a0 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f5bf4001670 con 0x7f5c001553a0 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 >> v1:172.21.15.27:6789/0 conn(0x7f5c001553a0 legacy=0x7f5c0015e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 --2- 172.21.15.27:0/1115511268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bf0003970 0x7f5bf0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 --2- 172.21.15.27:0/1115511268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bf00047c0 0x7f5bf0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 --2- 172.21.15.27:0/1115511268 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c000afeb0 0x7f5bf0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c000cbcf0 con 0x7f5bf0003970 2023-12-19T18:27:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c000cb630 con 0x7f5bf00047c0 2023-12-19T18:27:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.929+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bf000ba00 con 0x7f5c000afeb0 2023-12-19T18:27:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.930+0000 7f5c04d65640 1 --2- 172.21.15.27:0/1115511268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bf0003970 0x7f5bf0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.930+0000 7f5bfffff640 1 --2- 172.21.15.27:0/1115511268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bf00047c0 0x7f5bf0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.930+0000 7f5c04d65640 1 -- 172.21.15.27:0/1115511268 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c000afeb0 msgr2=0x7f5bf0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.930+0000 7f5c05566640 1 --2- 172.21.15.27:0/1115511268 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c000afeb0 0x7f5bf0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.930+0000 7f5c04d65640 1 --2- 172.21.15.27:0/1115511268 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c000afeb0 0x7f5bf0005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.930+0000 7f5c04d65640 1 -- 172.21.15.27:0/1115511268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bf00047c0 msgr2=0x7f5bf0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.930+0000 7f5c04d65640 1 --2- 172.21.15.27:0/1115511268 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5bf00047c0 0x7f5bf0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.930+0000 7f5c04d65640 1 -- 172.21.15.27:0/1115511268 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bf0003520 con 0x7f5bf0003970 2023-12-19T18:27:53.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.930+0000 7f5c05566640 1 --2- 172.21.15.27:0/1115511268 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5c000afeb0 0x7f5bf0005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:53.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.931+0000 7f5c04d65640 1 --2- 172.21.15.27:0/1115511268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bf0003970 0x7f5bf0003d90 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400c110 tx=0x7f5bf00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36f52651d579c560 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:53.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.931+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4005ad0 con 0x7f5bf0003970 2023-12-19T18:27:53.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.932+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf4005c50 con 0x7f5bf0003970 2023-12-19T18:27:53.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.932+0000 7f5bff7fe640 1 -- 172.21.15.27:0/1115511268 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf402c830 con 0x7f5bf0003970 2023-12-19T18:27:53.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.932+0000 7f5c05d67640 1 -- 172.21.15.27:0/1115511268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bf0003970 msgr2=0x7f5bf0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.932+0000 7f5c05d67640 1 --2- 172.21.15.27:0/1115511268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5bf0003970 0x7f5bf0003d90 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400c110 tx=0x7f5bf00057f0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.932+0000 7f5c05566640 1 -- 172.21.15.27:0/1115511268 reap_dead start 2023-12-19T18:27:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.932+0000 7f5c05d67640 1 -- 172.21.15.27:0/1115511268 shutdown_connections 2023-12-19T18:27:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.932+0000 7f5c05d67640 1 -- 172.21.15.27:0/1115511268 >> 172.21.15.27:0/1115511268 conn(0x7f5c00069c70 msgr2=0x7f5c0006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.932+0000 7f5c05d67640 1 -- 172.21.15.27:0/1115511268 shutdown_connections 2023-12-19T18:27:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.932+0000 7f5c05d67640 1 -- 172.21.15.27:0/1115511268 wait complete. 2023-12-19T18:27:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.934+0000 7f5c05d67640 1 Processor -- start 2023-12-19T18:27:53.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.934+0000 7f5c05d67640 1 -- start start 2023-12-19T18:27:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.935+0000 7f5c05d67640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bf0003970 0x7f5c0014c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.935+0000 7f5c05d67640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c000ad070 0x7f5c00157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.935+0000 7f5c05d67640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c000b1600 0x7f5c0006c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.935+0000 7f5c05d67640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c000ad6e0 con 0x7f5c000b1600 2023-12-19T18:27:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.935+0000 7f5c05d67640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c000ad9c0 con 0x7f5c000ad070 2023-12-19T18:27:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.935+0000 7f5c05d67640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c000adca0 con 0x7f5bf0003970 2023-12-19T18:27:53.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.936+0000 7f5c05566640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c000b1600 0x7f5c0006c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.936+0000 7f5c05566640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c000b1600 0x7f5c0006c570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38200/0 (socket says 172.21.15.27:38200) 2023-12-19T18:27:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.936+0000 7f5c05566640 1 -- 172.21.15.27:0/673718754 learned_addr learned my addr 172.21.15.27:0/673718754 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.936+0000 7f5bfffff640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c000ad070 0x7f5c00157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.936+0000 7f5c04d65640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bf0003970 0x7f5c0014c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.936+0000 7f5c05566640 1 -- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bf0003970 msgr2=0x7f5c0014c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.936+0000 7f5c05566640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bf0003970 0x7f5c0014c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.936+0000 7f5c05566640 1 -- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c000ad070 msgr2=0x7f5c00157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.936+0000 7f5c05566640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c000ad070 0x7f5c00157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.936+0000 7f5c05566640 1 -- 172.21.15.27:0/673718754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c000adf80 con 0x7f5c000b1600 2023-12-19T18:27:53.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.936+0000 7f5c05566640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c000b1600 0x7f5c0006c570 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f5c00158d00 tx=0x7f5bf800f4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:53.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.937+0000 7f5bfd7fa640 1 -- 172.21.15.27:0/673718754 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8017400 con 0x7f5c000b1600 2023-12-19T18:27:53.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.937+0000 7f5bfd7fa640 1 -- 172.21.15.27:0/673718754 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bf8011040 con 0x7f5c000b1600 2023-12-19T18:27:53.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.937+0000 7f5bfd7fa640 1 -- 172.21.15.27:0/673718754 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bf8017780 con 0x7f5c000b1600 2023-12-19T18:27:53.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.937+0000 7f5c05d67640 1 -- 172.21.15.27:0/673718754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c00158690 con 0x7f5c000b1600 2023-12-19T18:27:53.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.937+0000 7f5c05d67640 1 -- 172.21.15.27:0/673718754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c000b1ab0 con 0x7f5c000b1600 2023-12-19T18:27:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.939+0000 7f5c05d67640 1 -- 172.21.15.27:0/673718754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c000ad450 con 0x7f5c000b1600 2023-12-19T18:27:53.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.939+0000 7f5bfd7fa640 1 -- 172.21.15.27:0/673718754 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5bf80203b0 con 0x7f5c000b1600 2023-12-19T18:27:53.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.940+0000 7f5bfd7fa640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5be406a460 0x7f5be406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.941+0000 7f5bfd7fa640 1 -- 172.21.15.27:0/673718754 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5767+0+0 (secure 0 0 0) 0x7f5bf8088c10 con 0x7f5c000b1600 2023-12-19T18:27:53.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.941+0000 7f5c04d65640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5be406a460 0x7f5be406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:53.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.942+0000 7f5c04d65640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5be406a460 0x7f5be406c920 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400c110 tx=0x7f5bf4007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:53.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:53.946+0000 7f5bfd7fa640 1 -- 172.21.15.27:0/673718754 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5bf80541c0 con 0x7f5c000b1600 2023-12-19T18:27:54.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.142+0000 7f5c05d67640 1 -- 172.21.15.27:0/673718754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5c000ad820 con 0x7f5c000b1600 2023-12-19T18:27:54.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.144+0000 7f5bfd7fa640 1 -- 172.21.15.27:0/673718754 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v190) v1 ==== 74+0+16217 (secure 0 0 0) 0x7f5c000ad820 con 0x7f5c000b1600 2023-12-19T18:27:54.144 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:27:54.145 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":190,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:27:51.817442+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:26:36.428093+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-19T18:26:37.523993+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"190","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.1e","mappings":[{"from":2,"to":5}]},{"pgid":"21.33","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:27:54.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.147+0000 7f5c05d67640 1 -- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5be406a460 msgr2=0x7f5be406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:54.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.147+0000 7f5c05d67640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5be406a460 0x7f5be406c920 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f5bf400c110 tx=0x7f5bf4007000 comp rx=0 tx=0).stop 2023-12-19T18:27:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.147+0000 7f5c05d67640 1 -- 172.21.15.27:0/673718754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c000b1600 msgr2=0x7f5c0006c570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.147+0000 7f5c05d67640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c000b1600 0x7f5c0006c570 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f5c00158d00 tx=0x7f5bf800f4a0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.147+0000 7f5c05d67640 1 -- 172.21.15.27:0/673718754 shutdown_connections 2023-12-19T18:27:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.147+0000 7f5c05d67640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5be406a460 0x7f5be406c920 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.147+0000 7f5c05d67640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5c000b1600 0x7f5c0006c570 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.147+0000 7f5c05d67640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5c000ad070 0x7f5c00157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.147+0000 7f5c05d67640 1 --2- 172.21.15.27:0/673718754 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5bf0003970 0x7f5c0014c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.147+0000 7f5c05d67640 1 -- 172.21.15.27:0/673718754 >> 172.21.15.27:0/673718754 conn(0x7f5c00069c70 msgr2=0x7f5c000cecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.148+0000 7f5c05d67640 1 -- 172.21.15.27:0/673718754 shutdown_connections 2023-12-19T18:27:54.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.148+0000 7f5c05d67640 1 -- 172.21.15.27:0/673718754 wait complete. 2023-12-19T18:27:54.166 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:27:54.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.400+0000 7fbde2dfd640 1 Processor -- start 2023-12-19T18:27:54.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.400+0000 7fbde2dfd640 1 -- start start 2023-12-19T18:27:54.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.401+0000 7fbde2dfd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbddc154170 0x7fbddc070170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.402+0000 7fbde2dfd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc15b110 con 0x7fbddc15aab0 2023-12-19T18:27:54.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.402+0000 7fbde2dfd640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbddc06fce0 con 0x7fbddc070aa0 2023-12-19T18:27:54.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.402+0000 7fbde2dfd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc06f3b0 con 0x7fbddc154170 2023-12-19T18:27:54.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.402+0000 7fbde25fc640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbddc15aab0 0x7fbddc0af9f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47696/0 (socket says 172.21.15.27:47696) 2023-12-19T18:27:54.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.402+0000 7fbde25fc640 1 -- 172.21.15.27:0/2314261359 learned_addr learned my addr 172.21.15.27:0/2314261359 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:54.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.402+0000 7fbde1dfb640 1 --2- 172.21.15.27:0/2314261359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbddc154170 0x7fbddc070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 423118404 0 0) 0x7fbddc15b110 con 0x7fbddc15aab0 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbdc8003520 con 0x7fbddc15aab0 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 204003491 0 0) 0x7fbdc8003520 con 0x7fbddc15aab0 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbddc154170 msgr2=0x7fbddc070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 --2- 172.21.15.27:0/2314261359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbddc154170 0x7fbddc070170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 >> v1:172.21.15.192:6789/0 conn(0x7fbddc070aa0 legacy=0x7fbddc15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc06f690 con 0x7fbddc15aab0 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fbdd4001670 con 0x7fbddc15aab0 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 >> v1:172.21.15.27:6789/0 conn(0x7fbddc15aab0 legacy=0x7fbddc0af9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 --2- 172.21.15.27:0/2314261359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbddc154170 0x7fbdc80039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 --2- 172.21.15.27:0/2314261359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdc8003f20 0x7fbdc80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 --2- 172.21.15.27:0/2314261359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc8004d00 0x7fbdc8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc06f3b0 con 0x7fbdc8003f20 2023-12-19T18:27:54.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc800bb10 con 0x7fbdc8004d00 2023-12-19T18:27:54.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.403+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc800bcd0 con 0x7fbddc154170 2023-12-19T18:27:54.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.404+0000 7fbde15fa640 1 --2- 172.21.15.27:0/2314261359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdc8003f20 0x7fbdc80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.404+0000 7fbde1dfb640 1 --2- 172.21.15.27:0/2314261359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbddc154170 0x7fbdc80039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.404+0000 7fbde25fc640 1 --2- 172.21.15.27:0/2314261359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc8004d00 0x7fbdc8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.404+0000 7fbde15fa640 1 -- 172.21.15.27:0/2314261359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbddc154170 msgr2=0x7fbdc80039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:54.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.404+0000 7fbde15fa640 1 --2- 172.21.15.27:0/2314261359 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbddc154170 0x7fbdc80039a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.404+0000 7fbde15fa640 1 -- 172.21.15.27:0/2314261359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc8004d00 msgr2=0x7fbdc8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:54.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.404+0000 7fbde15fa640 1 --2- 172.21.15.27:0/2314261359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc8004d00 0x7fbdc8005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.404+0000 7fbde15fa640 1 -- 172.21.15.27:0/2314261359 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc8003520 con 0x7fbdc8003f20 2023-12-19T18:27:54.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.404+0000 7fbde25fc640 1 --2- 172.21.15.27:0/2314261359 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc8004d00 0x7fbdc8005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:27:54.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.405+0000 7fbde15fa640 1 --2- 172.21.15.27:0/2314261359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdc8003f20 0x7fbdc80047c0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc001750 tx=0x7fbdc80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bbc3d4dd80064c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:54.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.406+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc0052d0 con 0x7fbdc8003f20 2023-12-19T18:27:54.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.406+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdcc02e050 con 0x7fbdc8003f20 2023-12-19T18:27:54.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.406+0000 7fbde0df9640 1 -- 172.21.15.27:0/2314261359 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdcc0035f0 con 0x7fbdc8003f20 2023-12-19T18:27:54.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.406+0000 7fbde2dfd640 1 -- 172.21.15.27:0/2314261359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdc8003f20 msgr2=0x7fbdc80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:54.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.406+0000 7fbde2dfd640 1 --2- 172.21.15.27:0/2314261359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbdc8003f20 0x7fbdc80047c0 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7fbdcc001750 tx=0x7fbdc80057f0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.406+0000 7fbde25fc640 1 -- 172.21.15.27:0/2314261359 reap_dead start 2023-12-19T18:27:54.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.406+0000 7fbde2dfd640 1 -- 172.21.15.27:0/2314261359 shutdown_connections 2023-12-19T18:27:54.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.406+0000 7fbde2dfd640 1 -- 172.21.15.27:0/2314261359 >> 172.21.15.27:0/2314261359 conn(0x7fbddc06a7d0 msgr2=0x7fbddc0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:54.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.406+0000 7fbde2dfd640 1 -- 172.21.15.27:0/2314261359 shutdown_connections 2023-12-19T18:27:54.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.407+0000 7fbde2dfd640 1 -- 172.21.15.27:0/2314261359 wait complete. 2023-12-19T18:27:54.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.408+0000 7fbde2dfd640 1 Processor -- start 2023-12-19T18:27:54.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.409+0000 7fbde2dfd640 1 -- start start 2023-12-19T18:27:54.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.409+0000 7fbde2dfd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc8003f20 0x7fbddc14d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.409+0000 7fbde2dfd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbddc158280 0x7fbddc158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde2dfd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbddc1f20f0 0x7fbddc1f2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde2dfd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc1f31d0 con 0x7fbddc1f20f0 2023-12-19T18:27:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde2dfd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc0b2ee0 con 0x7fbdc8003f20 2023-12-19T18:27:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde2dfd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbddc158c40 con 0x7fbddc158280 2023-12-19T18:27:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde25fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbddc1f20f0 0x7fbddc1f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde25fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbddc1f20f0 0x7fbddc1f2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38214/0 (socket says 172.21.15.27:38214) 2023-12-19T18:27:54.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde25fc640 1 -- 172.21.15.27:0/1023556491 learned_addr learned my addr 172.21.15.27:0/1023556491 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde1dfb640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc8003f20 0x7fbddc14d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde15fa640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbddc158280 0x7fbddc158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde25fc640 1 -- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbddc158280 msgr2=0x7fbddc158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde25fc640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbddc158280 0x7fbddc158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde25fc640 1 -- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc8003f20 msgr2=0x7fbddc14d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde25fc640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc8003f20 0x7fbddc14d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde25fc640 1 -- 172.21.15.27:0/1023556491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc1f2ef0 con 0x7fbddc1f20f0 2023-12-19T18:27:54.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.410+0000 7fbde1dfb640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc8003f20 0x7fbddc14d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:54.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.411+0000 7fbde25fc640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbddc1f20f0 0x7fbddc1f2580 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0ae720 tx=0x7fbdd400eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:54.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.411+0000 7fbdd27fc640 1 -- 172.21.15.27:0/1023556491 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd400eed0 con 0x7fbddc1f20f0 2023-12-19T18:27:54.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.411+0000 7fbdd27fc640 1 -- 172.21.15.27:0/1023556491 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdd400c5b0 con 0x7fbddc1f20f0 2023-12-19T18:27:54.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.411+0000 7fbdd27fc640 1 -- 172.21.15.27:0/1023556491 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdd400c920 con 0x7fbddc1f20f0 2023-12-19T18:27:54.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.411+0000 7fbde2dfd640 1 -- 172.21.15.27:0/1023556491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbddc0b2820 con 0x7fbddc1f20f0 2023-12-19T18:27:54.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.411+0000 7fbde2dfd640 1 -- 172.21.15.27:0/1023556491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbddc14d970 con 0x7fbddc1f20f0 2023-12-19T18:27:54.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.412+0000 7fbde2dfd640 1 -- 172.21.15.27:0/1023556491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbddc0b2680 con 0x7fbddc1f20f0 2023-12-19T18:27:54.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.413+0000 7fbdd27fc640 1 -- 172.21.15.27:0/1023556491 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbdd400ca80 con 0x7fbddc1f20f0 2023-12-19T18:27:54.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.414+0000 7fbdd27fc640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbdc006a460 0x7fbdc006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.414+0000 7fbdd27fc640 1 -- 172.21.15.27:0/1023556491 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5767+0+0 (secure 0 0 0) 0x7fbdd40884d0 con 0x7fbddc1f20f0 2023-12-19T18:27:54.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.415+0000 7fbde1dfb640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbdc006a460 0x7fbdc006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.416+0000 7fbde1dfb640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbdc006a460 0x7fbdc006c920 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8007440 tx=0x7fbdd800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:54.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.418+0000 7fbdd27fc640 1 -- 172.21.15.27:0/1023556491 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbddc0b2680 con 0x7fbddc1f20f0 2023-12-19T18:27:54.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.644+0000 7fbde2dfd640 1 -- 172.21.15.27:0/1023556491 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fbddc0b3020 con 0x7fbddc1f20f0 2023-12-19T18:27:54.645 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:27:54.645+0000 7f8f3865c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:27:54.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.660+0000 7fbdd27fc640 1 -- 172.21.15.27:0/1023556491 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v78) v1 ==== 161+0+0 (secure 0 0 0) 0x7fbddc0b3020 con 0x7fbddc1f20f0 2023-12-19T18:27:54.660 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:27:54.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.662+0000 7fbde2dfd640 1 -- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbdc006a460 msgr2=0x7fbdc006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:54.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.662+0000 7fbde2dfd640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbdc006a460 0x7fbdc006c920 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8007440 tx=0x7fbdd800b040 comp rx=0 tx=0).stop 2023-12-19T18:27:54.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.662+0000 7fbde2dfd640 1 -- 172.21.15.27:0/1023556491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbddc1f20f0 msgr2=0x7fbddc1f2580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:54.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.662+0000 7fbde2dfd640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbddc1f20f0 0x7fbddc1f2580 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fbddc0ae720 tx=0x7fbdd400eb90 comp rx=0 tx=0).stop 2023-12-19T18:27:54.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.663+0000 7fbde2dfd640 1 -- 172.21.15.27:0/1023556491 shutdown_connections 2023-12-19T18:27:54.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.663+0000 7fbde2dfd640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbdc006a460 0x7fbdc006c920 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.663+0000 7fbde2dfd640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbddc1f20f0 0x7fbddc1f2580 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.663+0000 7fbde2dfd640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbddc158280 0x7fbddc158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.663+0000 7fbde2dfd640 1 --2- 172.21.15.27:0/1023556491 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbdc8003f20 0x7fbddc14d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.663+0000 7fbde2dfd640 1 -- 172.21.15.27:0/1023556491 >> 172.21.15.27:0/1023556491 conn(0x7fbddc06a7d0 msgr2=0x7fbddc15a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:54.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.663+0000 7fbde2dfd640 1 -- 172.21.15.27:0/1023556491 shutdown_connections 2023-12-19T18:27:54.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.663+0000 7fbde2dfd640 1 -- 172.21.15.27:0/1023556491 wait complete. 2023-12-19T18:27:54.681 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:27:54.698 INFO:tasks.ceph.mds.c.smithi027.stderr: -8553> 2023-12-19T18:26:42.942+0000 7f7c0cba1640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:27:54.767 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:27:54.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.887+0000 7f99a4b9f640 1 Processor -- start 2023-12-19T18:27:54.887 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.887+0000 7f99a4b9f640 1 -- start start 2023-12-19T18:27:54.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.888+0000 7f99a4b9f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99a0159de0 0x7f99a00b1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.888+0000 7f99a4b9f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f99a015df80 0x7f99a006c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.888+0000 7f99a4b9f640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99a015d810 con 0x7f99a0159a00 2023-12-19T18:27:54.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.889+0000 7f99a4b9f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99a00ca0c0 con 0x7f99a015df80 2023-12-19T18:27:54.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.889+0000 7f99a4b9f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a00ca3a0 con 0x7f99a0159de0 2023-12-19T18:27:54.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.889+0000 7f999e59c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f99a0159a00 0x7f99a00c9900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47702/0 (socket says 172.21.15.27:47702) 2023-12-19T18:27:54.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.889+0000 7f999e59c640 1 -- 172.21.15.27:0/71468680 learned_addr learned my addr 172.21.15.27:0/71468680 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:54.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.889+0000 7f9997fff640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99a0159de0 0x7f99a00b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.889+0000 7f999ed9d640 1 --2- 172.21.15.27:0/71468680 >> v2:172.21.15.192:3300/0 conn(0x7f99a015df80 0x7f99a006c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.889+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27641661 0 0) 0x7f99a015d810 con 0x7f99a0159a00 2023-12-19T18:27:54.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.889+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f998c003520 con 0x7f99a0159a00 2023-12-19T18:27:54.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4235664507 0 0) 0x7f998c003520 con 0x7f99a0159a00 2023-12-19T18:27:54.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99a0159de0 msgr2=0x7f99a00b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:54.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99a0159de0 0x7f99a00b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 >> v2:172.21.15.192:3300/0 conn(0x7f99a015df80 msgr2=0x7f99a006c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:54.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 --2- 172.21.15.27:0/71468680 >> v2:172.21.15.192:3300/0 conn(0x7f99a015df80 0x7f99a006c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a00ca680 con 0x7f99a0159a00 2023-12-19T18:27:54.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f9990001670 con 0x7f99a0159a00 2023-12-19T18:27:54.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 >> v1:172.21.15.27:6789/0 conn(0x7f99a0159a00 legacy=0x7f99a00c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:54.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f998c003970 0x7f998c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99a0159de0 0x7f998c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f998c004d00 0x7f998c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a00ca0c0 con 0x7f998c003970 2023-12-19T18:27:54.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a00ca3a0 con 0x7f998c004d00 2023-12-19T18:27:54.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f998c00b980 con 0x7f99a0159de0 2023-12-19T18:27:54.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f999e59c640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f998c003970 0x7f998c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.890+0000 7f9997fff640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99a0159de0 0x7f998c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.891+0000 7f999e59c640 1 -- 172.21.15.27:0/71468680 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99a0159de0 msgr2=0x7f998c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:54.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.891+0000 7f999e59c640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99a0159de0 0x7f998c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.891+0000 7f999e59c640 1 -- 172.21.15.27:0/71468680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f998c004d00 msgr2=0x7f998c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:54.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.891+0000 7f999ed9d640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f998c004d00 0x7f998c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.891+0000 7f999e59c640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f998c004d00 0x7f998c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.891+0000 7f999e59c640 1 -- 172.21.15.27:0/71468680 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f998c003520 con 0x7f998c003970 2023-12-19T18:27:54.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.891+0000 7f9997fff640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f99a0159de0 0x7f998c0047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:54.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.891+0000 7f999ed9d640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f998c004d00 0x7f998c005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:54.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.892+0000 7f999e59c640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f998c003970 0x7f998c003d90 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f999000c160 tx=0x7f998c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f51a30e940b9a2e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:54.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.892+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9990005ad0 con 0x7f998c003970 2023-12-19T18:27:54.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.892+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9990005c30 con 0x7f998c003970 2023-12-19T18:27:54.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.892+0000 7f999dd9b640 1 -- 172.21.15.27:0/71468680 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f999002c810 con 0x7f998c003970 2023-12-19T18:27:54.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.893+0000 7f99a4b9f640 1 -- 172.21.15.27:0/71468680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f998c003970 msgr2=0x7f998c003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:54.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.893+0000 7f99a4b9f640 1 --2- 172.21.15.27:0/71468680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f998c003970 0x7f998c003d90 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f999000c160 tx=0x7f998c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.893+0000 7f999ed9d640 1 -- 172.21.15.27:0/71468680 reap_dead start 2023-12-19T18:27:54.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.893+0000 7f99a4b9f640 1 -- 172.21.15.27:0/71468680 shutdown_connections 2023-12-19T18:27:54.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.893+0000 7f99a4b9f640 1 -- 172.21.15.27:0/71468680 >> 172.21.15.27:0/71468680 conn(0x7f99a00cb5b0 msgr2=0x7f99a00cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:54.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.893+0000 7f99a4b9f640 1 -- 172.21.15.27:0/71468680 shutdown_connections 2023-12-19T18:27:54.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.893+0000 7f99a4b9f640 1 -- 172.21.15.27:0/71468680 wait complete. 2023-12-19T18:27:54.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.895+0000 7f99a4b9f640 1 Processor -- start 2023-12-19T18:27:54.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.895+0000 7f99a4b9f640 1 -- start start 2023-12-19T18:27:54.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f99a4b9f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f998c003970 0x7f99a015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f99a4b9f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99a014cf20 0x7f99a006afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f99a4b9f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a006b4e0 0x7f99a00c37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f99a4b9f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a006b960 con 0x7f99a006b4e0 2023-12-19T18:27:54.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f99a4b9f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99a015b790 con 0x7f99a014cf20 2023-12-19T18:27:54.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f99a4b9f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99a00c47e0 con 0x7f998c003970 2023-12-19T18:27:54.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f999ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a006b4e0 0x7f99a00c37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f999ed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a006b4e0 0x7f99a00c37d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38250/0 (socket says 172.21.15.27:38250) 2023-12-19T18:27:54.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f999ed9d640 1 -- 172.21.15.27:0/858519332 learned_addr learned my addr 172.21.15.27:0/858519332 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:54.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f999e59c640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f998c003970 0x7f99a015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f999ed9d640 1 -- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f998c003970 msgr2=0x7f99a015b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:54.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f9997fff640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99a014cf20 0x7f99a006afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f999ed9d640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f998c003970 0x7f99a015b250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f999ed9d640 1 -- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99a014cf20 msgr2=0x7f99a006afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:54.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f999ed9d640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99a014cf20 0x7f99a006afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:54.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.896+0000 7f999ed9d640 1 -- 172.21.15.27:0/858519332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99a00c4500 con 0x7f99a006b4e0 2023-12-19T18:27:54.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.897+0000 7f999ed9d640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a006b4e0 0x7f99a00c37d0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f99a00c9100 tx=0x7f999800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:54.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.897+0000 7f9996ffd640 1 -- 172.21.15.27:0/858519332 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998016620 con 0x7f99a006b4e0 2023-12-19T18:27:54.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.897+0000 7f9996ffd640 1 -- 172.21.15.27:0/858519332 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9998016780 con 0x7f99a006b4e0 2023-12-19T18:27:54.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.897+0000 7f99a4b9f640 1 -- 172.21.15.27:0/858519332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99a00b1700 con 0x7f99a006b4e0 2023-12-19T18:27:54.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.897+0000 7f9996ffd640 1 -- 172.21.15.27:0/858519332 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9998016b10 con 0x7f99a006b4e0 2023-12-19T18:27:54.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.897+0000 7f99a4b9f640 1 -- 172.21.15.27:0/858519332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99a00b1cc0 con 0x7f99a006b4e0 2023-12-19T18:27:54.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.898+0000 7f99a4b9f640 1 -- 172.21.15.27:0/858519332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99a0068e90 con 0x7f99a006b4e0 2023-12-19T18:27:54.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.899+0000 7f9996ffd640 1 -- 172.21.15.27:0/858519332 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9998016c70 con 0x7f99a006b4e0 2023-12-19T18:27:54.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.900+0000 7f9996ffd640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f998006a430 0x7f998006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:54.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.900+0000 7f9996ffd640 1 -- 172.21.15.27:0/858519332 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5853+0+0 (secure 0 0 0) 0x7f9998089640 con 0x7f99a006b4e0 2023-12-19T18:27:54.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.901+0000 7f999e59c640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f998006a430 0x7f998006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:54.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.902+0000 7f999e59c640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f998006a430 0x7f998006c8f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9990001050 tx=0x7f9990007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:54.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:54.904+0000 7f9996ffd640 1 -- 172.21.15.27:0/858519332 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9998054c50 con 0x7f99a006b4e0 2023-12-19T18:27:55.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.128+0000 7f99a4b9f640 1 -- 172.21.15.27:0/858519332 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f99a0071310 con 0x7f99a006b4e0 2023-12-19T18:27:55.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.665+0000 7f9996ffd640 1 -- 172.21.15.27:0/858519332 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v79) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9998058270 con 0x7f99a006b4e0 2023-12-19T18:27:55.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.668+0000 7f99a4b9f640 1 -- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f998006a430 msgr2=0x7f998006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:55.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.668+0000 7f99a4b9f640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f998006a430 0x7f998006c8f0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f9990001050 tx=0x7f9990007000 comp rx=0 tx=0).stop 2023-12-19T18:27:55.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.668+0000 7f99a4b9f640 1 -- 172.21.15.27:0/858519332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a006b4e0 msgr2=0x7f99a00c37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:55.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.668+0000 7f99a4b9f640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a006b4e0 0x7f99a00c37d0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f99a00c9100 tx=0x7f999800f520 comp rx=0 tx=0).stop 2023-12-19T18:27:55.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.668+0000 7f99a4b9f640 1 -- 172.21.15.27:0/858519332 shutdown_connections 2023-12-19T18:27:55.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.668+0000 7f99a4b9f640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f998006a430 0x7f998006c8f0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:55.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.668+0000 7f99a4b9f640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99a006b4e0 0x7f99a00c37d0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:55.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.668+0000 7f99a4b9f640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f99a014cf20 0x7f99a006afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:55.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.668+0000 7f99a4b9f640 1 --2- 172.21.15.27:0/858519332 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f998c003970 0x7f99a015b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:55.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.668+0000 7f99a4b9f640 1 -- 172.21.15.27:0/858519332 >> 172.21.15.27:0/858519332 conn(0x7f99a00cb5b0 msgr2=0x7f99a00ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:55.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.668+0000 7f99a4b9f640 1 -- 172.21.15.27:0/858519332 shutdown_connections 2023-12-19T18:27:55.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.668+0000 7f99a4b9f640 1 -- 172.21.15.27:0/858519332 wait complete. 2023-12-19T18:27:55.691 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:27:55.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.981+0000 7fe3039fb640 1 Processor -- start 2023-12-19T18:27:55.981 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.981+0000 7fe3039fb640 1 -- start start 2023-12-19T18:27:55.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.983+0000 7fe3039fb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe2fc15b4b0 0x7fe2fc0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:55.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.983+0000 7fe3039fb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2fc070fb0 con 0x7fe2fc15fa70 2023-12-19T18:27:55.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.983+0000 7fe3039fb640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2fc06fab0 con 0x7fe2fc15b0d0 2023-12-19T18:27:55.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.983+0000 7fe3039fb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2fc06f4f0 con 0x7fe2fc15b4b0 2023-12-19T18:27:55.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.983+0000 7fe300f96640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe2fc15b4b0 0x7fe2fc0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:55.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.983+0000 7fe301f98640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe2fc15fa70 0x7fe2fc0c91c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47712/0 (socket says 172.21.15.27:47712) 2023-12-19T18:27:55.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.983+0000 7fe301f98640 1 -- 172.21.15.27:0/3950457310 learned_addr learned my addr 172.21.15.27:0/3950457310 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:55.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.984+0000 7fe300f96640 1 -- 172.21.15.27:0/3950457310 >> v1:172.21.15.192:6789/0 conn(0x7fe2fc15b0d0 legacy=0x7fe2fc0cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:55.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.984+0000 7fe300f96640 1 -- 172.21.15.27:0/3950457310 >> v1:172.21.15.27:6789/0 conn(0x7fe2fc15fa70 legacy=0x7fe2fc0c91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:55.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.984+0000 7fe2f3fff640 1 -- 172.21.15.27:0/3950457310 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204195659 0 0) 0x7fe2fc070fb0 con 0x7fe2fc15fa70 2023-12-19T18:27:55.984 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.984+0000 7fe300f96640 1 -- 172.21.15.27:0/3950457310 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc06f7d0 con 0x7fe2fc15b4b0 2023-12-19T18:27:55.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.985+0000 7fe300f96640 1 --2- 172.21.15.27:0/3950457310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2fc15b4b0 0x7fe2fc0cb450 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0097d0 tx=0x7fe2ec02b9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d742934d4aa2aa32 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:55.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.986+0000 7fe2f3fff640 1 -- 172.21.15.27:0/3950457310 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec030890 con 0x7fe2fc15b4b0 2023-12-19T18:27:55.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.986+0000 7fe2f3fff640 1 -- 172.21.15.27:0/3950457310 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec0309f0 con 0x7fe2fc15b4b0 2023-12-19T18:27:55.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.986+0000 7fe2f3fff640 1 -- 172.21.15.27:0/3950457310 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec030d80 con 0x7fe2fc15b4b0 2023-12-19T18:27:55.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.986+0000 7fe3039fb640 1 -- 172.21.15.27:0/3950457310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2fc15b4b0 msgr2=0x7fe2fc0cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:55.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.986+0000 7fe3039fb640 1 --2- 172.21.15.27:0/3950457310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2fc15b4b0 0x7fe2fc0cb450 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec0097d0 tx=0x7fe2ec02b9d0 comp rx=0 tx=0).stop 2023-12-19T18:27:55.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.986+0000 7fe3039fb640 1 -- 172.21.15.27:0/3950457310 shutdown_connections 2023-12-19T18:27:55.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.986+0000 7fe3039fb640 1 --2- 172.21.15.27:0/3950457310 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2fc15b4b0 0x7fe2fc0cb450 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:55.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.986+0000 7fe3039fb640 1 -- 172.21.15.27:0/3950457310 >> 172.21.15.27:0/3950457310 conn(0x7fe2fc0b0620 msgr2=0x7fe2fc0b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:55.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.987+0000 7fe3039fb640 1 -- 172.21.15.27:0/3950457310 shutdown_connections 2023-12-19T18:27:55.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.987+0000 7fe3039fb640 1 -- 172.21.15.27:0/3950457310 wait complete. 2023-12-19T18:27:55.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.988+0000 7fe3039fb640 1 Processor -- start 2023-12-19T18:27:55.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.989+0000 7fe3039fb640 1 -- start start 2023-12-19T18:27:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.989+0000 7fe3039fb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2fc15b0d0 0x7fe2fc1560a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.989+0000 7fe3039fb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2fc15b4b0 0x7fe2fc0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.989+0000 7fe3039fb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2fc15fa70 0x7fe2fc1f64e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.990+0000 7fe3039fb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc1f0b30 con 0x7fe2fc15b0d0 2023-12-19T18:27:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.990+0000 7fe3039fb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc1f10f0 con 0x7fe2fc15b4b0 2023-12-19T18:27:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.990+0000 7fe3039fb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc0c8020 con 0x7fe2fc15fa70 2023-12-19T18:27:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.990+0000 7fe301797640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2fc15b0d0 0x7fe2fc1560a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.990+0000 7fe301797640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2fc15b0d0 0x7fe2fc1560a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38280/0 (socket says 172.21.15.27:38280) 2023-12-19T18:27:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.990+0000 7fe301797640 1 -- 172.21.15.27:0/2638432169 learned_addr learned my addr 172.21.15.27:0/2638432169 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.990+0000 7fe301797640 1 -- 172.21.15.27:0/2638432169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2fc15fa70 msgr2=0x7fe2fc1f64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.990+0000 7fe301797640 1 --2- 172.21.15.27:0/2638432169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2fc15fa70 0x7fe2fc1f64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.990+0000 7fe300f96640 1 --2- 172.21.15.27:0/2638432169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2fc15b4b0 0x7fe2fc0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.990+0000 7fe301797640 1 -- 172.21.15.27:0/2638432169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2fc15b4b0 msgr2=0x7fe2fc0c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.990+0000 7fe301797640 1 --2- 172.21.15.27:0/2638432169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2fc15b4b0 0x7fe2fc0c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:55.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.990+0000 7fe301797640 1 -- 172.21.15.27:0/2638432169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ec009400 con 0x7fe2fc15b0d0 2023-12-19T18:27:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.991+0000 7fe301797640 1 --2- 172.21.15.27:0/2638432169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2fc15b0d0 0x7fe2fc1560a0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800c480 tx=0x7fe2f800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.991+0000 7fe2f1ffb640 1 -- 172.21.15.27:0/2638432169 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f80052d0 con 0x7fe2fc15b0d0 2023-12-19T18:27:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.991+0000 7fe2f1ffb640 1 -- 172.21.15.27:0/2638432169 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f8005430 con 0x7fe2fc15b0d0 2023-12-19T18:27:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.991+0000 7fe2f1ffb640 1 -- 172.21.15.27:0/2638432169 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f8012870 con 0x7fe2fc15b0d0 2023-12-19T18:27:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.991+0000 7fe3039fb640 1 -- 172.21.15.27:0/2638432169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2fc0c8300 con 0x7fe2fc15b0d0 2023-12-19T18:27:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.991+0000 7fe3039fb640 1 -- 172.21.15.27:0/2638432169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc1f7240 con 0x7fe2fc15b0d0 2023-12-19T18:27:55.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.992+0000 7fe3039fb640 1 -- 172.21.15.27:0/2638432169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2fc0c8540 con 0x7fe2fc15b0d0 2023-12-19T18:27:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.993+0000 7fe2f1ffb640 1 -- 172.21.15.27:0/2638432169 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe2f80129d0 con 0x7fe2fc15b0d0 2023-12-19T18:27:55.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.994+0000 7fe2f1ffb640 1 --2- 172.21.15.27:0/2638432169 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2e006a430 0x7fe2e006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:55.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.994+0000 7fe2f1ffb640 1 -- 172.21.15.27:0/2638432169 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5853+0+0 (secure 0 0 0) 0x7fe2f80884b0 con 0x7fe2fc15b0d0 2023-12-19T18:27:55.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.995+0000 7fe300f96640 1 --2- 172.21.15.27:0/2638432169 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2e006a430 0x7fe2e006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:55.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.996+0000 7fe300f96640 1 --2- 172.21.15.27:0/2638432169 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2e006a430 0x7fe2e006c8f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec036000 tx=0x7fe2ec03b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:55.999+0000 7fe2f1ffb640 1 -- 172.21.15.27:0/2638432169 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe2f8053aa0 con 0x7fe2fc15b0d0 2023-12-19T18:27:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.196+0000 7fe3039fb640 1 -- 172.21.15.27:0/2638432169 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fe2fc068e90 con 0x7fe2fc15b0d0 2023-12-19T18:27:56.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.972+0000 7fe2f1ffb640 1 -- 172.21.15.27:0/2638432169 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v192) v1 ==== 181+0+0 (secure 0 0 0) 0x7fe2f80570c0 con 0x7fe2fc15b0d0 2023-12-19T18:27:56.973 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:27:56.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.976+0000 7fe3039fb640 1 -- 172.21.15.27:0/2638432169 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2e006a430 msgr2=0x7fe2e006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:56.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.976+0000 7fe3039fb640 1 --2- 172.21.15.27:0/2638432169 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2e006a430 0x7fe2e006c8f0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec036000 tx=0x7fe2ec03b040 comp rx=0 tx=0).stop 2023-12-19T18:27:56.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.976+0000 7fe3039fb640 1 -- 172.21.15.27:0/2638432169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2fc15b0d0 msgr2=0x7fe2fc1560a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:56.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.976+0000 7fe3039fb640 1 --2- 172.21.15.27:0/2638432169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2fc15b0d0 0x7fe2fc1560a0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7fe2f800c480 tx=0x7fe2f800c940 comp rx=0 tx=0).stop 2023-12-19T18:27:56.977 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.976+0000 7fe3039fb640 1 -- 172.21.15.27:0/2638432169 shutdown_connections 2023-12-19T18:27:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.976+0000 7fe3039fb640 1 --2- 172.21.15.27:0/2638432169 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe2e006a430 0x7fe2e006c8f0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.976+0000 7fe3039fb640 1 --2- 172.21.15.27:0/2638432169 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe2fc15fa70 0x7fe2fc1f64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.976+0000 7fe3039fb640 1 --2- 172.21.15.27:0/2638432169 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe2fc15b4b0 0x7fe2fc0c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.976+0000 7fe3039fb640 1 --2- 172.21.15.27:0/2638432169 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2fc15b0d0 0x7fe2fc1560a0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.976+0000 7fe3039fb640 1 -- 172.21.15.27:0/2638432169 >> 172.21.15.27:0/2638432169 conn(0x7fe2fc0b0620 msgr2=0x7fe2fc0c3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.977+0000 7fe3039fb640 1 -- 172.21.15.27:0/2638432169 shutdown_connections 2023-12-19T18:27:56.978 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:56.977+0000 7fe3039fb640 1 -- 172.21.15.27:0/2638432169 wait complete. 2023-12-19T18:27:57.003 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:27:57.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.240+0000 7fc8de1b9640 1 Processor -- start 2023-12-19T18:27:57.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.241+0000 7fc8de1b9640 1 -- start start 2023-12-19T18:27:57.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.242+0000 7fc8de1b9640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc8d80cbb60 0x7fc8d8166170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:57.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.242+0000 7fc8de1b9640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d8070fb0 con 0x7fc8d815b0d0 2023-12-19T18:27:57.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.242+0000 7fc8de1b9640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d806fab0 con 0x7fc8d815b4b0 2023-12-19T18:27:57.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.242+0000 7fc8de1b9640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d806f4f0 con 0x7fc8d80cbb60 2023-12-19T18:27:57.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.243+0000 7fc8d77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc8d815b0d0 0x7fc8d80cabf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47726/0 (socket says 172.21.15.27:47726) 2023-12-19T18:27:57.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.243+0000 7fc8d77fe640 1 -- 172.21.15.27:0/3415428864 learned_addr learned my addr 172.21.15.27:0/3415428864 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:57.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.243+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3647931897 0 0) 0x7fc8d8070fb0 con 0x7fc8d815b0d0 2023-12-19T18:27:57.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.243+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8bc003520 con 0x7fc8d815b0d0 2023-12-19T18:27:57.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.243+0000 7fc8d7fff640 1 --2- 172.21.15.27:0/3415428864 >> v2:172.21.15.192:3300/0 conn(0x7fc8d80cbb60 0x7fc8d8166170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:57.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 93096309 0 0) 0x7fc8bc003520 con 0x7fc8d815b0d0 2023-12-19T18:27:57.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 >> v2:172.21.15.192:3300/0 conn(0x7fc8d80cbb60 msgr2=0x7fc8d8166170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:57.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d67fc640 1 --2- 172.21.15.27:0/3415428864 >> v2:172.21.15.192:3300/0 conn(0x7fc8d80cbb60 0x7fc8d8166170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:57.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 >> v1:172.21.15.192:6789/0 conn(0x7fc8d815b4b0 legacy=0x7fc8d80cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:57.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d806f7d0 con 0x7fc8d815b0d0 2023-12-19T18:27:57.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fc8c8001670 con 0x7fc8d815b0d0 2023-12-19T18:27:57.245 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 >> v1:172.21.15.27:6789/0 conn(0x7fc8d815b0d0 legacy=0x7fc8d80cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d67fc640 1 --2- 172.21.15.27:0/3415428864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8bc0039c0 0x7fc8bc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d67fc640 1 --2- 172.21.15.27:0/3415428864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc0047c0 0x7fc8bc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d67fc640 1 --2- 172.21.15.27:0/3415428864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8bc005180 0x7fc8bc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d806f4f0 con 0x7fc8bc0047c0 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc00c060 con 0x7fc8bc0039c0 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8bc00c220 con 0x7fc8bc005180 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d6ffd640 1 --2- 172.21.15.27:0/3415428864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc0047c0 0x7fc8bc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.244+0000 7fc8d77fe640 1 --2- 172.21.15.27:0/3415428864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8bc0039c0 0x7fc8bc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.245+0000 7fc8d6ffd640 1 -- 172.21.15.27:0/3415428864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8bc005180 msgr2=0x7fc8bc00b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.245+0000 7fc8d6ffd640 1 --2- 172.21.15.27:0/3415428864 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8bc005180 0x7fc8bc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.245+0000 7fc8d6ffd640 1 -- 172.21.15.27:0/3415428864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8bc0039c0 msgr2=0x7fc8bc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.245+0000 7fc8d6ffd640 1 --2- 172.21.15.27:0/3415428864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8bc0039c0 0x7fc8bc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.245+0000 7fc8d6ffd640 1 -- 172.21.15.27:0/3415428864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc003520 con 0x7fc8bc0047c0 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.245+0000 7fc8d77fe640 1 --2- 172.21.15.27:0/3415428864 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8bc0039c0 0x7fc8bc003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:27:57.246 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.245+0000 7fc8d7fff640 1 -- 172.21.15.27:0/3415428864 reap_dead start 2023-12-19T18:27:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.246+0000 7fc8d6ffd640 1 --2- 172.21.15.27:0/3415428864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc0047c0 0x7fc8bc004c40 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4001750 tx=0x7fc8bc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32f3a3c5e73c60cc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.246+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40052d0 con 0x7fc8bc0047c0 2023-12-19T18:27:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.246+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c402e050 con 0x7fc8bc0047c0 2023-12-19T18:27:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.246+0000 7fc8d67fc640 1 -- 172.21.15.27:0/3415428864 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4003830 con 0x7fc8bc0047c0 2023-12-19T18:27:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.247+0000 7fc8de1b9640 1 -- 172.21.15.27:0/3415428864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc0047c0 msgr2=0x7fc8bc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.247+0000 7fc8de1b9640 1 --2- 172.21.15.27:0/3415428864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc0047c0 0x7fc8bc004c40 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4001750 tx=0x7fc8bc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:27:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.247+0000 7fc8de1b9640 1 -- 172.21.15.27:0/3415428864 shutdown_connections 2023-12-19T18:27:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.247+0000 7fc8de1b9640 1 --2- 172.21.15.27:0/3415428864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8bc0047c0 0x7fc8bc004c40 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.247+0000 7fc8de1b9640 1 -- 172.21.15.27:0/3415428864 >> 172.21.15.27:0/3415428864 conn(0x7fc8d80b0620 msgr2=0x7fc8d80b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:57.248 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.247+0000 7fc8de1b9640 1 -- 172.21.15.27:0/3415428864 shutdown_connections 2023-12-19T18:27:57.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.247+0000 7fc8de1b9640 1 -- 172.21.15.27:0/3415428864 wait complete. 2023-12-19T18:27:57.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.249+0000 7fc8de1b9640 1 Processor -- start 2023-12-19T18:27:57.249 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.249+0000 7fc8de1b9640 1 -- start start 2023-12-19T18:27:57.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.250+0000 7fc8de1b9640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8bc0047c0 0x7fc8d814af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:57.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.250+0000 7fc8de1b9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d80c8240 0x7fc8d814eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:57.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.250+0000 7fc8de1b9640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d80c86c0 0x7fc8d815db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:57.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.250+0000 7fc8de1b9640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d814b4a0 con 0x7fc8d80c8240 2023-12-19T18:27:57.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.250+0000 7fc8de1b9640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d80ac8d0 con 0x7fc8d80c86c0 2023-12-19T18:27:57.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.250+0000 7fc8de1b9640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d80acb20 con 0x7fc8bc0047c0 2023-12-19T18:27:57.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.251+0000 7fc8d6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d80c8240 0x7fc8d814eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:57.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.251+0000 7fc8d6ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d80c8240 0x7fc8d814eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38296/0 (socket says 172.21.15.27:38296) 2023-12-19T18:27:57.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.251+0000 7fc8d6ffd640 1 -- 172.21.15.27:0/2517052158 learned_addr learned my addr 172.21.15.27:0/2517052158 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:57.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.251+0000 7fc8d77fe640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8bc0047c0 0x7fc8d814af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:57.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.251+0000 7fc8d7fff640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d80c86c0 0x7fc8d815db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:57.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.251+0000 7fc8d6ffd640 1 -- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8bc0047c0 msgr2=0x7fc8d814af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:57.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.251+0000 7fc8d6ffd640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8bc0047c0 0x7fc8d814af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:57.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.251+0000 7fc8d6ffd640 1 -- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d80c86c0 msgr2=0x7fc8d815db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:57.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.251+0000 7fc8d6ffd640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d80c86c0 0x7fc8d815db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:57.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.251+0000 7fc8d6ffd640 1 -- 172.21.15.27:0/2517052158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d80ace00 con 0x7fc8d80c8240 2023-12-19T18:27:57.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.251+0000 7fc8d7fff640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d80c86c0 0x7fc8d815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:57.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.252+0000 7fc8d6ffd640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d80c8240 0x7fc8d814eb70 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4000c00 tx=0x7fc8c4001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:57.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.252+0000 7fc8cffff640 1 -- 172.21.15.27:0/2517052158 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40076c0 con 0x7fc8d80c8240 2023-12-19T18:27:57.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.252+0000 7fc8cffff640 1 -- 172.21.15.27:0/2517052158 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c4007a30 con 0x7fc8d80c8240 2023-12-19T18:27:57.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.252+0000 7fc8cffff640 1 -- 172.21.15.27:0/2517052158 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4006be0 con 0x7fc8d80c8240 2023-12-19T18:27:57.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.252+0000 7fc8de1b9640 1 -- 172.21.15.27:0/2517052158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d80ad0e0 con 0x7fc8d80c8240 2023-12-19T18:27:57.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.252+0000 7fc8de1b9640 1 -- 172.21.15.27:0/2517052158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d80ad6a0 con 0x7fc8d80c8240 2023-12-19T18:27:57.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.253+0000 7fc8de1b9640 1 -- 172.21.15.27:0/2517052158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d80ad830 con 0x7fc8d80c8240 2023-12-19T18:27:57.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.254+0000 7fc8cffff640 1 -- 172.21.15.27:0/2517052158 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc8c4006d40 con 0x7fc8d80c8240 2023-12-19T18:27:57.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.255+0000 7fc8cffff640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc8c006a430 0x7fc8c006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:57.255 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.255+0000 7fc8cffff640 1 -- 172.21.15.27:0/2517052158 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5415+0+0 (secure 0 0 0) 0x7fc8c40aa430 con 0x7fc8d80c8240 2023-12-19T18:27:57.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.256+0000 7fc8d77fe640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc8c006a430 0x7fc8c006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:57.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.257+0000 7fc8d77fe640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc8c006a430 0x7fc8c006c8f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8001eb0 tx=0x7fc8c8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:57.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.260+0000 7fc8cffff640 1 -- 172.21.15.27:0/2517052158 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc8c4075aa0 con 0x7fc8d80c8240 2023-12-19T18:27:57.456 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.456+0000 7fc8de1b9640 1 -- 172.21.15.27:0/2517052158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc8d8068e90 con 0x7fc8d80c8240 2023-12-19T18:27:57.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.990+0000 7fc8cffff640 1 -- 172.21.15.27:0/2517052158 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v193) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc8c40790c0 con 0x7fc8d80c8240 2023-12-19T18:27:57.991 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-12-19T18:27:57.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.995+0000 7fc8de1b9640 1 -- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc8c006a430 msgr2=0x7fc8c006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:57.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.995+0000 7fc8de1b9640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc8c006a430 0x7fc8c006c8f0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc8c8001eb0 tx=0x7fc8c8009040 comp rx=0 tx=0).stop 2023-12-19T18:27:57.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.995+0000 7fc8de1b9640 1 -- 172.21.15.27:0/2517052158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d80c8240 msgr2=0x7fc8d814eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:57.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.995+0000 7fc8de1b9640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d80c8240 0x7fc8d814eb70 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4000c00 tx=0x7fc8c4001120 comp rx=0 tx=0).stop 2023-12-19T18:27:57.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.995+0000 7fc8de1b9640 1 -- 172.21.15.27:0/2517052158 shutdown_connections 2023-12-19T18:27:57.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.995+0000 7fc8de1b9640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc8c006a430 0x7fc8c006c8f0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:57.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.995+0000 7fc8de1b9640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc8d80c86c0 0x7fc8d815db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:57.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.995+0000 7fc8de1b9640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8d80c8240 0x7fc8d814eb70 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:57.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.995+0000 7fc8de1b9640 1 --2- 172.21.15.27:0/2517052158 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc8bc0047c0 0x7fc8d814af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:57.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.995+0000 7fc8de1b9640 1 -- 172.21.15.27:0/2517052158 >> 172.21.15.27:0/2517052158 conn(0x7fc8d80b0620 msgr2=0x7fc8d80c29f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:57.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.995+0000 7fc8de1b9640 1 -- 172.21.15.27:0/2517052158 shutdown_connections 2023-12-19T18:27:57.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:57.996+0000 7fc8de1b9640 1 -- 172.21.15.27:0/2517052158 wait complete. 2023-12-19T18:27:58.023 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:27:58.251 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.251+0000 7f31b8c0e640 1 Processor -- start 2023-12-19T18:27:58.252 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.252+0000 7f31b8c0e640 1 -- start start 2023-12-19T18:27:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.253+0000 7f31b8c0e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f31b40cdb30 0x7f31b40cdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.253+0000 7f31b8c0e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b4161250 con 0x7f31b40d17a0 2023-12-19T18:27:58.253 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.253+0000 7f31b8c0e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31b4161a50 con 0x7f31b415cfc0 2023-12-19T18:27:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.253+0000 7f31b8c0e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31b40ce500 con 0x7f31b40cdb30 2023-12-19T18:27:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.254+0000 7f31b3fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f31b40d17a0 0x7f31b40d1bb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47728/0 (socket says 172.21.15.27:47728) 2023-12-19T18:27:58.254 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.254+0000 7f31b3fff640 1 -- 172.21.15.27:0/3627947740 learned_addr learned my addr 172.21.15.27:0/3627947740 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.254+0000 7f31abfff640 1 --2- 172.21.15.27:0/3627947740 >> v2:172.21.15.192:3300/0 conn(0x7f31b40cdb30 0x7f31b40cdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.254+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1379824713 0 0) 0x7f31ac001490 con 0x7f31b40d17a0 2023-12-19T18:27:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.254+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f31a0003520 con 0x7f31b40d17a0 2023-12-19T18:27:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1426407906 0 0) 0x7f31a0003520 con 0x7f31b40d17a0 2023-12-19T18:27:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 >> v2:172.21.15.192:3300/0 conn(0x7f31b40cdb30 msgr2=0x7f31b40cdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31b2ffd640 1 --2- 172.21.15.27:0/3627947740 >> v2:172.21.15.192:3300/0 conn(0x7f31b40cdb30 0x7f31b40cdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 >> v1:172.21.15.192:6789/0 conn(0x7f31b415cfc0 legacy=0x7f31b4160b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b4161490 con 0x7f31b40d17a0 2023-12-19T18:27:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f31ac001b80 con 0x7f31b40d17a0 2023-12-19T18:27:58.256 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 >> v1:172.21.15.27:6789/0 conn(0x7f31b40d17a0 legacy=0x7f31b40d1bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31b2ffd640 1 --2- 172.21.15.27:0/3627947740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31a0003970 0x7f31a0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31b2ffd640 1 --2- 172.21.15.27:0/3627947740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a00047c0 0x7f31a0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31b2ffd640 1 --2- 172.21.15.27:0/3627947740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31a00052a0 0x7f31a000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b40ce500 con 0x7f31a00047c0 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a000c170 con 0x7f31a00052a0 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a000c330 con 0x7f31a0003970 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.255+0000 7f31abfff640 1 --2- 172.21.15.27:0/3627947740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a00047c0 0x7f31a0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.256+0000 7f31abfff640 1 -- 172.21.15.27:0/3627947740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31a0003970 msgr2=0x7f31a0003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.256+0000 7f31b3fff640 1 --2- 172.21.15.27:0/3627947740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31a00052a0 0x7f31a000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.256+0000 7f31b37fe640 1 --2- 172.21.15.27:0/3627947740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31a0003970 0x7f31a0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.256+0000 7f31abfff640 1 --2- 172.21.15.27:0/3627947740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31a0003970 0x7f31a0003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.256+0000 7f31abfff640 1 -- 172.21.15.27:0/3627947740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31a00052a0 msgr2=0x7f31a000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.256+0000 7f31abfff640 1 --2- 172.21.15.27:0/3627947740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31a00052a0 0x7f31a000b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.257 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.256+0000 7f31abfff640 1 -- 172.21.15.27:0/3627947740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a0003520 con 0x7f31a00047c0 2023-12-19T18:27:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.256+0000 7f31b3fff640 1 --2- 172.21.15.27:0/3627947740 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31a00052a0 0x7f31a000b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.256+0000 7f31b3fff640 1 -- 172.21.15.27:0/3627947740 reap_dead start 2023-12-19T18:27:58.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.256+0000 7f31b37fe640 1 --2- 172.21.15.27:0/3627947740 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31a0003970 0x7f31a0003d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:27:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.257+0000 7f31abfff640 1 --2- 172.21.15.27:0/3627947740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a00047c0 0x7f31a0004c40 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f319c00ba00 tx=0x7f31a00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db2705f6516c0d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.257+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c032620 con 0x7f31a00047c0 2023-12-19T18:27:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.257+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f319c032780 con 0x7f31a00047c0 2023-12-19T18:27:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.257+0000 7f31b2ffd640 1 -- 172.21.15.27:0/3627947740 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f319c032b10 con 0x7f31a00047c0 2023-12-19T18:27:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.258+0000 7f31b8c0e640 1 -- 172.21.15.27:0/3627947740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a00047c0 msgr2=0x7f31a0004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:58.259 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.258+0000 7f31b8c0e640 1 --2- 172.21.15.27:0/3627947740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a00047c0 0x7f31a0004c40 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7f319c00ba00 tx=0x7f31a00057f0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.258+0000 7f31b8c0e640 1 -- 172.21.15.27:0/3627947740 shutdown_connections 2023-12-19T18:27:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.258+0000 7f31b8c0e640 1 --2- 172.21.15.27:0/3627947740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a00047c0 0x7f31a0004c40 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.258+0000 7f31b8c0e640 1 -- 172.21.15.27:0/3627947740 >> 172.21.15.27:0/3627947740 conn(0x7f31b4072e20 msgr2=0x7f31b4073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.258+0000 7f31b8c0e640 1 -- 172.21.15.27:0/3627947740 shutdown_connections 2023-12-19T18:27:58.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.258+0000 7f31b8c0e640 1 -- 172.21.15.27:0/3627947740 wait complete. 2023-12-19T18:27:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.260+0000 7f31b8c0e640 1 Processor -- start 2023-12-19T18:27:58.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.260+0000 7f31b8c0e640 1 -- start start 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.261+0000 7f31b8c0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a00047c0 0x7f31b4151250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.261+0000 7f31b8c0e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b4151cc0 0x7f31b415a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.261+0000 7f31b8c0e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b415aea0 0x7f31b41f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.261+0000 7f31b8c0e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b41f62d0 con 0x7f31a00047c0 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.261+0000 7f31b8c0e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b4151790 con 0x7f31b415aea0 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.261+0000 7f31b8c0e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31b4152210 con 0x7f31b4151cc0 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.261+0000 7f31b37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a00047c0 0x7f31b4151250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.261+0000 7f31b37fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a00047c0 0x7f31b4151250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38302/0 (socket says 172.21.15.27:38302) 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.261+0000 7f31b37fe640 1 -- 172.21.15.27:0/691056890 learned_addr learned my addr 172.21.15.27:0/691056890 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.262+0000 7f31abfff640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b4151cc0 0x7f31b415a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.262+0000 7f31b37fe640 1 -- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b4151cc0 msgr2=0x7f31b415a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.262+0000 7f31b37fe640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b4151cc0 0x7f31b415a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.262+0000 7f31b37fe640 1 -- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b415aea0 msgr2=0x7f31b41f2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:58.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.262+0000 7f31b3fff640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b415aea0 0x7f31b41f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.262+0000 7f31b37fe640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b415aea0 0x7f31b41f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.262+0000 7f31b37fe640 1 -- 172.21.15.27:0/691056890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31b41524f0 con 0x7f31a00047c0 2023-12-19T18:27:58.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.262+0000 7f31b37fe640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a00047c0 0x7f31b4151250 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f31a400d4b0 tx=0x7f31a400d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:58.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.262+0000 7f31b0ff9640 1 -- 172.21.15.27:0/691056890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a4005480 con 0x7f31a00047c0 2023-12-19T18:27:58.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.262+0000 7f31b0ff9640 1 -- 172.21.15.27:0/691056890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31a4003610 con 0x7f31a00047c0 2023-12-19T18:27:58.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.262+0000 7f31b0ff9640 1 -- 172.21.15.27:0/691056890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31a4003a00 con 0x7f31a00047c0 2023-12-19T18:27:58.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.262+0000 7f31b8c0e640 1 -- 172.21.15.27:0/691056890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b41527d0 con 0x7f31a00047c0 2023-12-19T18:27:58.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.263+0000 7f31b8c0e640 1 -- 172.21.15.27:0/691056890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b41f3030 con 0x7f31a00047c0 2023-12-19T18:27:58.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.264+0000 7f31b8c0e640 1 -- 172.21.15.27:0/691056890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b404e840 con 0x7f31a00047c0 2023-12-19T18:27:58.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.265+0000 7f31b0ff9640 1 -- 172.21.15.27:0/691056890 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f31a4003b60 con 0x7f31a00047c0 2023-12-19T18:27:58.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.266+0000 7f31b0ff9640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f319406a430 0x7f319406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.266+0000 7f31b0ff9640 1 -- 172.21.15.27:0/691056890 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f31a40884e0 con 0x7f31a00047c0 2023-12-19T18:27:58.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.266+0000 7f31abfff640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f319406a430 0x7f319406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.267+0000 7f31abfff640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f319406a430 0x7f319406c8f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f31a0004740 tx=0x7f319c03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:58.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.270+0000 7f31b0ff9640 1 -- 172.21.15.27:0/691056890 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f31a4053df0 con 0x7f31a00047c0 2023-12-19T18:27:58.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.465+0000 7f31b8c0e640 1 -- 172.21.15.27:0/691056890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f31b40cd7a0 con 0x7f31a00047c0 2023-12-19T18:27:58.472 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.472+0000 7f31b0ff9640 1 -- 172.21.15.27:0/691056890 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7f31a4057410 con 0x7f31a00047c0 2023-12-19T18:27:58.475 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.474+0000 7f31b8c0e640 1 -- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f319406a430 msgr2=0x7f319406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.474+0000 7f31b8c0e640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f319406a430 0x7f319406c8f0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f31a0004740 tx=0x7f319c03d040 comp rx=0 tx=0).stop 2023-12-19T18:27:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.474+0000 7f31b8c0e640 1 -- 172.21.15.27:0/691056890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a00047c0 msgr2=0x7f31b4151250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.474+0000 7f31b8c0e640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a00047c0 0x7f31b4151250 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7f31a400d4b0 tx=0x7f31a400d970 comp rx=0 tx=0).stop 2023-12-19T18:27:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.475+0000 7f31b8c0e640 1 -- 172.21.15.27:0/691056890 shutdown_connections 2023-12-19T18:27:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.475+0000 7f31b8c0e640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f319406a430 0x7f319406c8f0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.475+0000 7f31b8c0e640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31b415aea0 0x7f31b41f2af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.476 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.475+0000 7f31b8c0e640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f31b4151cc0 0x7f31b415a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.475+0000 7f31b8c0e640 1 --2- 172.21.15.27:0/691056890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31a00047c0 0x7f31b4151250 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.475+0000 7f31b8c0e640 1 -- 172.21.15.27:0/691056890 >> 172.21.15.27:0/691056890 conn(0x7f31b4072e20 msgr2=0x7f31b4073200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:58.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.475+0000 7f31b8c0e640 1 -- 172.21.15.27:0/691056890 shutdown_connections 2023-12-19T18:27:58.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.475+0000 7f31b8c0e640 1 -- 172.21.15.27:0/691056890 wait complete. 2023-12-19T18:27:58.492 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync' 2023-12-19T18:27:58.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.716+0000 7f9749279640 1 Processor -- start 2023-12-19T18:27:58.716 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.716+0000 7f9749279640 1 -- start start 2023-12-19T18:27:58.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.718+0000 7f9749279640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9744154030 0x7f97440cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.718+0000 7f9749279640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f974415e440 con 0x7f97440c5560 2023-12-19T18:27:58.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.718+0000 7f9749279640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97440cc190 con 0x7f97440af630 2023-12-19T18:27:58.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.718+0000 7f9749279640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97440cb2a0 con 0x7f9744154030 2023-12-19T18:27:58.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.718+0000 7f9748a78640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f97440c5560 0x7f974415dd30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47736/0 (socket says 172.21.15.27:47736) 2023-12-19T18:27:58.718 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.718+0000 7f9748a78640 1 -- 172.21.15.27:0/2931623347 learned_addr learned my addr 172.21.15.27:0/2931623347 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:27:58.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.718+0000 7f9743fff640 1 --2- 172.21.15.27:0/2931623347 >> v2:172.21.15.192:3300/0 conn(0x7f9744154030 0x7f97440cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 580898338 0 0) 0x7f974415e440 con 0x7f97440c5560 2023-12-19T18:27:58.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9730003520 con 0x7f97440c5560 2023-12-19T18:27:58.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 76864151 0 0) 0x7f9730003520 con 0x7f97440c5560 2023-12-19T18:27:58.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 >> v2:172.21.15.192:3300/0 conn(0x7f9744154030 msgr2=0x7f97440cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:58.720 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 --2- 172.21.15.27:0/2931623347 >> v2:172.21.15.192:3300/0 conn(0x7f9744154030 0x7f97440cd4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 >> v1:172.21.15.192:6789/0 conn(0x7f97440af630 legacy=0x7f974415d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97440cb580 con 0x7f97440c5560 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f973c001670 con 0x7f97440c5560 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 >> v1:172.21.15.27:6789/0 conn(0x7f97440c5560 legacy=0x7f974415dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 --2- 172.21.15.27:0/2931623347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97300039c0 0x7f9730003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 --2- 172.21.15.27:0/2931623347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97300047c0 0x7f9730004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 --2- 172.21.15.27:0/2931623347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300051d0 0x7f973000db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97440cb2a0 con 0x7f97300051d0 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f973000e330 con 0x7f97300047c0 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.719+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f973000e4f0 con 0x7f97300039c0 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.720+0000 7f9748a78640 1 --2- 172.21.15.27:0/2931623347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300051d0 0x7f973000db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.720+0000 7f97437fe640 1 --2- 172.21.15.27:0/2931623347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97300047c0 0x7f9730004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.720+0000 7f9743fff640 1 --2- 172.21.15.27:0/2931623347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97300039c0 0x7f9730003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.721 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.720+0000 7f9748a78640 1 -- 172.21.15.27:0/2931623347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97300039c0 msgr2=0x7f9730003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:58.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.720+0000 7f9748a78640 1 --2- 172.21.15.27:0/2931623347 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97300039c0 0x7f9730003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.720+0000 7f9748a78640 1 -- 172.21.15.27:0/2931623347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97300047c0 msgr2=0x7f9730004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:27:58.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.720+0000 7f9748a78640 1 --2- 172.21.15.27:0/2931623347 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97300047c0 0x7f9730004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.720+0000 7f9748a78640 1 -- 172.21.15.27:0/2931623347 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9730003520 con 0x7f97300051d0 2023-12-19T18:27:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.721+0000 7f9748a78640 1 -- 172.21.15.27:0/2931623347 reap_dead start 2023-12-19T18:27:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.721+0000 7f9748a78640 1 --2- 172.21.15.27:0/2931623347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300051d0 0x7f973000db90 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f973c00c0e0 tx=0x7f97300057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5f6762517c5b36 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.722+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c005ad0 con 0x7f97300051d0 2023-12-19T18:27:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.722+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973c005c70 con 0x7f97300051d0 2023-12-19T18:27:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.722+0000 7f9742ffd640 1 -- 172.21.15.27:0/2931623347 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f973c02c810 con 0x7f97300051d0 2023-12-19T18:27:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.722+0000 7f9749279640 1 -- 172.21.15.27:0/2931623347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300051d0 msgr2=0x7f973000db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.722+0000 7f9749279640 1 --2- 172.21.15.27:0/2931623347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300051d0 0x7f973000db90 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7f973c00c0e0 tx=0x7f97300057f0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.722+0000 7f9749279640 1 -- 172.21.15.27:0/2931623347 shutdown_connections 2023-12-19T18:27:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.722+0000 7f9749279640 1 --2- 172.21.15.27:0/2931623347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300051d0 0x7f973000db90 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.723 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.722+0000 7f9749279640 1 -- 172.21.15.27:0/2931623347 >> 172.21.15.27:0/2931623347 conn(0x7f9744069c70 msgr2=0x7f974406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:27:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.722+0000 7f9749279640 1 -- 172.21.15.27:0/2931623347 shutdown_connections 2023-12-19T18:27:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.723+0000 7f9749279640 1 -- 172.21.15.27:0/2931623347 wait complete. 2023-12-19T18:27:58.724 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.725+0000 7f9749279640 1 Processor -- start 2023-12-19T18:27:58.725 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.725+0000 7f9749279640 1 -- start start 2023-12-19T18:27:58.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.726+0000 7f9749279640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300051d0 0x7f97440ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.727 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.726+0000 7f9749279640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f974414c940 0x7f9744147360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.726+0000 7f9749279640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f974415c3e0 0x7f974415c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.726+0000 7f9749279640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974415cdd0 con 0x7f97300051d0 2023-12-19T18:27:58.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.726+0000 7f9749279640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f974406c550 con 0x7f974415c3e0 2023-12-19T18:27:58.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.726+0000 7f9749279640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f974406c830 con 0x7f974414c940 2023-12-19T18:27:58.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.726+0000 7f9743fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300051d0 0x7f97440ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.726+0000 7f9743fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300051d0 0x7f97440ca950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38320/0 (socket says 172.21.15.27:38320) 2023-12-19T18:27:58.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.726+0000 7f9743fff640 1 -- 172.21.15.27:0/4262403913 learned_addr learned my addr 172.21.15.27:0/4262403913 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:27:58.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.727+0000 7f9748a78640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f974415c3e0 0x7f974415c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.727+0000 7f97437fe640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f974414c940 0x7f9744147360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.727+0000 7f9743fff640 1 -- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f974414c940 msgr2=0x7f9744147360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:58.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.727+0000 7f9743fff640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f974414c940 0x7f9744147360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.727+0000 7f9743fff640 1 -- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f974415c3e0 msgr2=0x7f974415c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:27:58.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.727+0000 7f9743fff640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f974415c3e0 0x7f974415c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:27:58.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.727+0000 7f9743fff640 1 -- 172.21.15.27:0/4262403913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f974406cb10 con 0x7f97300051d0 2023-12-19T18:27:58.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.727+0000 7f9743fff640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300051d0 0x7f97440ca950 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f9734009cb0 tx=0x7f9734007700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:58.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.727+0000 7f9740ff9640 1 -- 172.21.15.27:0/4262403913 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9734014610 con 0x7f97300051d0 2023-12-19T18:27:58.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.727+0000 7f9740ff9640 1 -- 172.21.15.27:0/4262403913 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f973400f050 con 0x7f97300051d0 2023-12-19T18:27:58.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.728+0000 7f9740ff9640 1 -- 172.21.15.27:0/4262403913 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97340149d0 con 0x7f97300051d0 2023-12-19T18:27:58.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.728+0000 7f9749279640 1 -- 172.21.15.27:0/4262403913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f974406cdf0 con 0x7f97300051d0 2023-12-19T18:27:58.729 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.728+0000 7f9749279640 1 -- 172.21.15.27:0/4262403913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97441478a0 con 0x7f97300051d0 2023-12-19T18:27:58.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.729+0000 7f9749279640 1 -- 172.21.15.27:0/4262403913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f974404e840 con 0x7f97300051d0 2023-12-19T18:27:58.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.730+0000 7f9740ff9640 1 -- 172.21.15.27:0/4262403913 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9734014b30 con 0x7f97300051d0 2023-12-19T18:27:58.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.731+0000 7f9740ff9640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f972406a430 0x7f972406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:27:58.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.731+0000 7f9740ff9640 1 -- 172.21.15.27:0/4262403913 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9734088e60 con 0x7f97300051d0 2023-12-19T18:27:58.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.731+0000 7f97437fe640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f972406a430 0x7f972406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:27:58.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.733+0000 7f97437fe640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f972406a430 0x7f972406c8f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f974414cd40 tx=0x7f972c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:27:58.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:58.735+0000 7f9740ff9640 1 -- 172.21.15.27:0/4262403913 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f97340547e0 con 0x7f97300051d0 2023-12-19T18:27:59.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:27:59.009+0000 7f9749279640 1 -- 172.21.15.27:0/4262403913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]} v 0) v1 -- 0x7f97440c3c50 con 0x7f97300051d0 2023-12-19T18:28:00.004 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.004+0000 7f9740ff9640 1 -- 172.21.15.27:0/4262403913 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_fsync"]}]=0 v622) v1 ==== 131+0+0 (secure 0 0 0) 0x7f9734057e00 con 0x7f97300051d0 2023-12-19T18:28:00.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.007+0000 7f9749279640 1 -- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f972406a430 msgr2=0x7f972406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:00.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.007+0000 7f9749279640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f972406a430 0x7f972406c8f0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f974414cd40 tx=0x7f972c009040 comp rx=0 tx=0).stop 2023-12-19T18:28:00.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.007+0000 7f9749279640 1 -- 172.21.15.27:0/4262403913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300051d0 msgr2=0x7f97440ca950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:00.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.007+0000 7f9749279640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300051d0 0x7f97440ca950 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f9734009cb0 tx=0x7f9734007700 comp rx=0 tx=0).stop 2023-12-19T18:28:00.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.007+0000 7f9749279640 1 -- 172.21.15.27:0/4262403913 shutdown_connections 2023-12-19T18:28:00.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.007+0000 7f9749279640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f972406a430 0x7f972406c8f0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:00.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.007+0000 7f9749279640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f974415c3e0 0x7f974415c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:00.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.007+0000 7f9749279640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f974414c940 0x7f9744147360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:00.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.007+0000 7f9749279640 1 --2- 172.21.15.27:0/4262403913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97300051d0 0x7f97440ca950 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:00.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.007+0000 7f9749279640 1 -- 172.21.15.27:0/4262403913 >> 172.21.15.27:0/4262403913 conn(0x7f9744069c70 msgr2=0x7f97440c96f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:00.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.008+0000 7f9749279640 1 -- 172.21.15.27:0/4262403913 shutdown_connections 2023-12-19T18:28:00.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.008+0000 7f9749279640 1 -- 172.21.15.27:0/4262403913 wait complete. 2023-12-19T18:28:00.030 INFO:tasks.cephfs_test_runner:That when the full flag is encountered during asynchronous ... ok 2023-12-19T18:28:00.030 INFO:tasks.cephfs_test_runner:Starting test: test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) 2023-12-19T18:28:00.031 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-12-19T18:28:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.258+0000 7f5208a34640 1 Processor -- start 2023-12-19T18:28:00.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.258+0000 7f5208a34640 1 -- start start 2023-12-19T18:28:00.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.260+0000 7f5208a34640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52040d27d0 0x7f52040d1030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:00.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.260+0000 7f5208a34640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520415d3b0 con 0x7f520415cfd0 2023-12-19T18:28:00.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.260+0000 7f5208a34640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52040d1730 con 0x7f52040b4ee0 2023-12-19T18:28:00.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.260+0000 7f5208a34640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52040d2c00 con 0x7f52040d27d0 2023-12-19T18:28:00.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.260+0000 7f52037fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f520415cfd0 0x7f52040b47d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47748/0 (socket says 172.21.15.27:47748) 2023-12-19T18:28:00.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.260+0000 7f52037fe640 1 -- 172.21.15.27:0/1628599429 learned_addr learned my addr 172.21.15.27:0/1628599429 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:00.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.260+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873058824 0 0) 0x7f520415d3b0 con 0x7f520415cfd0 2023-12-19T18:28:00.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51f0003520 con 0x7f520415cfd0 2023-12-19T18:28:00.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5203fff640 1 --2- 172.21.15.27:0/1628599429 >> v2:172.21.15.192:3300/0 conn(0x7f52040d27d0 0x7f52040d1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1804635716 0 0) 0x7f51f0003520 con 0x7f520415cfd0 2023-12-19T18:28:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 >> v2:172.21.15.192:3300/0 conn(0x7f52040d27d0 msgr2=0x7f52040d1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 --2- 172.21.15.27:0/1628599429 >> v2:172.21.15.192:3300/0 conn(0x7f52040d27d0 0x7f52040d1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 >> v1:172.21.15.192:6789/0 conn(0x7f52040b4ee0 legacy=0x7f52040b5e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52040d1a10 con 0x7f520415cfd0 2023-12-19T18:28:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f51f4001670 con 0x7f520415cfd0 2023-12-19T18:28:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 >> v1:172.21.15.27:6789/0 conn(0x7f520415cfd0 legacy=0x7f52040b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:00.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 --2- 172.21.15.27:0/1628599429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f00039c0 0x7f51f0003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 --2- 172.21.15.27:0/1628599429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f00047c0 0x7f51f0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 --2- 172.21.15.27:0/1628599429 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51f00051d0 0x7f51f000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f000c0f0 con 0x7f51f00039c0 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f000c260 con 0x7f51f00047c0 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51f000c420 con 0x7f51f00051d0 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.261+0000 7f52037fe640 1 --2- 172.21.15.27:0/1628599429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f00039c0 0x7f51f0003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.262+0000 7f52037fe640 1 -- 172.21.15.27:0/1628599429 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51f00051d0 msgr2=0x7f51f000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.262+0000 7f52037fe640 1 --2- 172.21.15.27:0/1628599429 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f51f00051d0 0x7f51f000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.262+0000 7f52037fe640 1 -- 172.21.15.27:0/1628599429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f00047c0 msgr2=0x7f51f0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.262+0000 7f51fbfff640 1 --2- 172.21.15.27:0/1628599429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f00047c0 0x7f51f0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.262+0000 7f52037fe640 1 --2- 172.21.15.27:0/1628599429 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f00047c0 0x7f51f0004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.262+0000 7f52037fe640 1 -- 172.21.15.27:0/1628599429 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0003520 con 0x7f51f00039c0 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.262+0000 7f5203fff640 1 -- 172.21.15.27:0/1628599429 reap_dead start 2023-12-19T18:28:00.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.263+0000 7f52037fe640 1 --2- 172.21.15.27:0/1628599429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f00039c0 0x7f51f0003de0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c0e0 tx=0x7f51f00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db33bcd2d697ff6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:00.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.263+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f4005ad0 con 0x7f51f00039c0 2023-12-19T18:28:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.263+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51f4005c30 con 0x7f51f00039c0 2023-12-19T18:28:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.263+0000 7f5202ffd640 1 -- 172.21.15.27:0/1628599429 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51f402ca50 con 0x7f51f00039c0 2023-12-19T18:28:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.264+0000 7f5208a34640 1 -- 172.21.15.27:0/1628599429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f00039c0 msgr2=0x7f51f0003de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.264+0000 7f5208a34640 1 --2- 172.21.15.27:0/1628599429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f00039c0 0x7f51f0003de0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f51f400c0e0 tx=0x7f51f00057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.264+0000 7f5208a34640 1 -- 172.21.15.27:0/1628599429 shutdown_connections 2023-12-19T18:28:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.264+0000 7f5208a34640 1 --2- 172.21.15.27:0/1628599429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f00039c0 0x7f51f0003de0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.264+0000 7f5208a34640 1 -- 172.21.15.27:0/1628599429 >> 172.21.15.27:0/1628599429 conn(0x7f52040cf7d0 msgr2=0x7f52040cfbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.264+0000 7f5208a34640 1 -- 172.21.15.27:0/1628599429 shutdown_connections 2023-12-19T18:28:00.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.264+0000 7f5208a34640 1 -- 172.21.15.27:0/1628599429 wait complete. 2023-12-19T18:28:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.266+0000 7f5208a34640 1 Processor -- start 2023-12-19T18:28:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.266+0000 7f5208a34640 1 -- start start 2023-12-19T18:28:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.267+0000 7f5208a34640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f00039c0 0x7f52040c9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.267+0000 7f5208a34640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f520415efc0 0x7f52041f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.267+0000 7f5208a34640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52040caaf0 0x7f52040caf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.267+0000 7f5208a34640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52040cb4b0 con 0x7f520415efc0 2023-12-19T18:28:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.267+0000 7f5208a34640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52041602d0 con 0x7f51f00039c0 2023-12-19T18:28:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.267+0000 7f5208a34640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f520415f440 con 0x7f52040caaf0 2023-12-19T18:28:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.268+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f520415efc0 0x7f52041f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.268+0000 7f51fbfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f520415efc0 0x7f52041f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38342/0 (socket says 172.21.15.27:38342) 2023-12-19T18:28:00.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.268+0000 7f51fbfff640 1 -- 172.21.15.27:0/1546486566 learned_addr learned my addr 172.21.15.27:0/1546486566 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:00.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.268+0000 7f52037fe640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f00039c0 0x7f52040c9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:00.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.268+0000 7f5203fff640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52040caaf0 0x7f52040caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:00.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.268+0000 7f51fbfff640 1 -- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52040caaf0 msgr2=0x7f52040caf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:00.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.268+0000 7f51fbfff640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52040caaf0 0x7f52040caf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:00.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.268+0000 7f51fbfff640 1 -- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f00039c0 msgr2=0x7f52040c9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:00.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.268+0000 7f51fbfff640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f00039c0 0x7f52040c9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:00.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.268+0000 7f51fbfff640 1 -- 172.21.15.27:0/1546486566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f520415f6c0 con 0x7f520415efc0 2023-12-19T18:28:00.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.268+0000 7f52037fe640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f00039c0 0x7f52040c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:00.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.269+0000 7f51fbfff640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f520415efc0 0x7f52041f5070 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00d5a0 tx=0x7f51ec00da60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:00.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.269+0000 7f5200ff9640 1 -- 172.21.15.27:0/1546486566 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec003610 con 0x7f520415efc0 2023-12-19T18:28:00.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.269+0000 7f5200ff9640 1 -- 172.21.15.27:0/1546486566 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ec003770 con 0x7f520415efc0 2023-12-19T18:28:00.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.269+0000 7f5200ff9640 1 -- 172.21.15.27:0/1546486566 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ec003b20 con 0x7f520415efc0 2023-12-19T18:28:00.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.269+0000 7f5208a34640 1 -- 172.21.15.27:0/1546486566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f520415f9a0 con 0x7f520415efc0 2023-12-19T18:28:00.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.269+0000 7f5208a34640 1 -- 172.21.15.27:0/1546486566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f520415ff60 con 0x7f520415efc0 2023-12-19T18:28:00.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.270+0000 7f5208a34640 1 -- 172.21.15.27:0/1546486566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5204160480 con 0x7f520415efc0 2023-12-19T18:28:00.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.271+0000 7f5200ff9640 1 -- 172.21.15.27:0/1546486566 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f51ec003c80 con 0x7f520415efc0 2023-12-19T18:28:00.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.273+0000 7f5200ff9640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f51e406a460 0x7f51e406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:00.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.273+0000 7f5200ff9640 1 -- 172.21.15.27:0/1546486566 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f51ec088430 con 0x7f520415efc0 2023-12-19T18:28:00.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.273+0000 7f52037fe640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f51e406a460 0x7f51e406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:00.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.275+0000 7f52037fe640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f51e406a460 0x7f51e406c920 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f51f4034000 tx=0x7f51f4009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:00.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.277+0000 7f5200ff9640 1 -- 172.21.15.27:0/1546486566 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5204160480 con 0x7f520415efc0 2023-12-19T18:28:00.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:00.543+0000 7f5208a34640 1 -- 172.21.15.27:0/1546486566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7f52041600f0 con 0x7f520415efc0 2023-12-19T18:28:01.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.008+0000 7f5200ff9640 1 -- 172.21.15.27:0/1546486566 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v623) v1 ==== 138+0+0 (secure 0 0 0) 0x7f51ec053e70 con 0x7f520415efc0 2023-12-19T18:28:01.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.012+0000 7f5208a34640 1 -- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f51e406a460 msgr2=0x7f51e406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:01.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.012+0000 7f5208a34640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f51e406a460 0x7f51e406c920 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f51f4034000 tx=0x7f51f4009030 comp rx=0 tx=0).stop 2023-12-19T18:28:01.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.012+0000 7f5208a34640 1 -- 172.21.15.27:0/1546486566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f520415efc0 msgr2=0x7f52041f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:01.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.012+0000 7f5208a34640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f520415efc0 0x7f52041f5070 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00d5a0 tx=0x7f51ec00da60 comp rx=0 tx=0).stop 2023-12-19T18:28:01.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.012+0000 7f5208a34640 1 -- 172.21.15.27:0/1546486566 shutdown_connections 2023-12-19T18:28:01.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.012+0000 7f5208a34640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f51e406a460 0x7f51e406c920 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.012+0000 7f5208a34640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f52040caaf0 0x7f52040caf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.012+0000 7f5208a34640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f520415efc0 0x7f52041f5070 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.012+0000 7f5208a34640 1 --2- 172.21.15.27:0/1546486566 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51f00039c0 0x7f52040c9f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.012+0000 7f5208a34640 1 -- 172.21.15.27:0/1546486566 >> 172.21.15.27:0/1546486566 conn(0x7f52040cf7d0 msgr2=0x7f52040b6490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:01.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.012+0000 7f5208a34640 1 -- 172.21.15.27:0/1546486566 shutdown_connections 2023-12-19T18:28:01.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.012+0000 7f5208a34640 1 -- 172.21.15.27:0/1546486566 wait complete. 2023-12-19T18:28:01.034 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-19T18:28:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.271+0000 7fe61e02c640 1 Processor -- start 2023-12-19T18:28:01.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.271+0000 7fe61e02c640 1 -- start start 2023-12-19T18:28:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.273+0000 7fe61e02c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe61815def0 0x7fe61806bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.273+0000 7fe61e02c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe618159a80 0x7fe618159f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.273+0000 7fe61e02c640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6180cdef0 con 0x7fe61815d400 2023-12-19T18:28:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.273+0000 7fe61e02c640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6180c9020 con 0x7fe61815def0 2023-12-19T18:28:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.273+0000 7fe61e02c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe61806c4d0 con 0x7fe618159a80 2023-12-19T18:28:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.274+0000 7fe60ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe61815def0 0x7fe61806bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.274+0000 7fe60ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe61815def0 0x7fe61806bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38344/0 (socket says 172.21.15.27:38344) 2023-12-19T18:28:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.274+0000 7fe60ffff640 1 -- 172.21.15.27:0/3872170200 learned_addr learned my addr 172.21.15.27:0/3872170200 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.274+0000 7fe617fff640 1 --2- 172.21.15.27:0/3872170200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe618159a80 0x7fe618159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.274+0000 7fe60ffff640 1 -- 172.21.15.27:0/3872170200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe618159a80 msgr2=0x7fe618159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.274+0000 7fe60ffff640 1 --2- 172.21.15.27:0/3872170200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe618159a80 0x7fe618159f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.274+0000 7fe60ffff640 1 -- 172.21.15.27:0/3872170200 >> v1:172.21.15.192:6789/0 conn(0x7fe61815d400 legacy=0x7fe61815d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.274+0000 7fe60ffff640 1 -- 172.21.15.27:0/3872170200 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61806c7b0 con 0x7fe61815def0 2023-12-19T18:28:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.275+0000 7fe60ffff640 1 --2- 172.21.15.27:0/3872170200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe61815def0 0x7fe61806bdd0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fe600009870 tx=0x7fe60002b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1921aeda00367472 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.276+0000 7fe616ffd640 1 -- 172.21.15.27:0/3872170200 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6000317c0 con 0x7fe61815def0 2023-12-19T18:28:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.276+0000 7fe616ffd640 1 -- 172.21.15.27:0/3872170200 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe600031920 con 0x7fe61815def0 2023-12-19T18:28:01.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.276+0000 7fe616ffd640 1 -- 172.21.15.27:0/3872170200 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe600031cb0 con 0x7fe61815def0 2023-12-19T18:28:01.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.276+0000 7fe61e02c640 1 -- 172.21.15.27:0/3872170200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe61815def0 msgr2=0x7fe61806bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:01.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.276+0000 7fe61e02c640 1 --2- 172.21.15.27:0/3872170200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe61815def0 0x7fe61806bdd0 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fe600009870 tx=0x7fe60002b860 comp rx=0 tx=0).stop 2023-12-19T18:28:01.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.277+0000 7fe61e02c640 1 -- 172.21.15.27:0/3872170200 shutdown_connections 2023-12-19T18:28:01.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.277+0000 7fe61e02c640 1 --2- 172.21.15.27:0/3872170200 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe618159a80 0x7fe618159f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.277+0000 7fe61e02c640 1 --2- 172.21.15.27:0/3872170200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe61815def0 0x7fe61806bdd0 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.277+0000 7fe61e02c640 1 -- 172.21.15.27:0/3872170200 >> 172.21.15.27:0/3872170200 conn(0x7fe6180cebd0 msgr2=0x7fe6180cefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:01.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.277+0000 7fe61e02c640 1 -- 172.21.15.27:0/3872170200 shutdown_connections 2023-12-19T18:28:01.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.277+0000 7fe61e02c640 1 -- 172.21.15.27:0/3872170200 wait complete. 2023-12-19T18:28:01.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.279+0000 7fe61e02c640 1 Processor -- start 2023-12-19T18:28:01.278 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.279+0000 7fe61e02c640 1 -- start start 2023-12-19T18:28:01.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.280+0000 7fe61e02c640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe618159a80 0x7fe61814d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:01.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.280+0000 7fe61e02c640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe61815d400 0x7fe618157e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:01.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.280+0000 7fe61e02c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe61815def0 0x7fe6180af6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:01.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.280+0000 7fe61e02c640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61814eac0 con 0x7fe61815def0 2023-12-19T18:28:01.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.280+0000 7fe61e02c640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe61814efd0 con 0x7fe61815d400 2023-12-19T18:28:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.280+0000 7fe61e02c640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe618151d30 con 0x7fe618159a80 2023-12-19T18:28:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.280+0000 7fe617fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe61815def0 0x7fe6180af6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.280+0000 7fe617fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe61815def0 0x7fe6180af6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:38354/0 (socket says 172.21.15.27:38354) 2023-12-19T18:28:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.280+0000 7fe617fff640 1 -- 172.21.15.27:0/592209420 learned_addr learned my addr 172.21.15.27:0/592209420 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.280+0000 7fe60ffff640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe61815d400 0x7fe618157e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.280+0000 7fe6177fe640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe618159a80 0x7fe61814d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.281+0000 7fe617fff640 1 -- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe618159a80 msgr2=0x7fe61814d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.281+0000 7fe617fff640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe618159a80 0x7fe61814d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.281+0000 7fe617fff640 1 -- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe61815d400 msgr2=0x7fe618157e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.281+0000 7fe617fff640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe61815d400 0x7fe618157e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.281+0000 7fe617fff640 1 -- 172.21.15.27:0/592209420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6000094d0 con 0x7fe61815def0 2023-12-19T18:28:01.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.281+0000 7fe617fff640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe61815def0 0x7fe6180af6c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fe610007e70 tx=0x7fe61000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:01.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.281+0000 7fe614ff9640 1 -- 172.21.15.27:0/592209420 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610014620 con 0x7fe61815def0 2023-12-19T18:28:01.282 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.281+0000 7fe614ff9640 1 -- 172.21.15.27:0/592209420 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6100147a0 con 0x7fe61815def0 2023-12-19T18:28:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.281+0000 7fe614ff9640 1 -- 172.21.15.27:0/592209420 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe610014b50 con 0x7fe61815def0 2023-12-19T18:28:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.281+0000 7fe61e02c640 1 -- 172.21.15.27:0/592209420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe618151a50 con 0x7fe61815def0 2023-12-19T18:28:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.281+0000 7fe61e02c640 1 -- 172.21.15.27:0/592209420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe61814f4c0 con 0x7fe61815def0 2023-12-19T18:28:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.282+0000 7fe61e02c640 1 -- 172.21.15.27:0/592209420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6181517d0 con 0x7fe61815def0 2023-12-19T18:28:01.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.283+0000 7fe614ff9640 1 -- 172.21.15.27:0/592209420 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe610014cb0 con 0x7fe61815def0 2023-12-19T18:28:01.284 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.285+0000 7fe614ff9640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5f806a430 0x7fe5f806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:01.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.285+0000 7fe614ff9640 1 -- 172.21.15.27:0/592209420 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe610088fe0 con 0x7fe61815def0 2023-12-19T18:28:01.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.285+0000 7fe6177fe640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5f806a430 0x7fe5f806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:01.286 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.287+0000 7fe6177fe640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5f806a430 0x7fe5f806c8f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe6180b0530 tx=0x7fe608009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:01.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.289+0000 7fe614ff9640 1 -- 172.21.15.27:0/592209420 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe6100548c0 con 0x7fe61815def0 2023-12-19T18:28:01.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.484+0000 7fe61e02c640 1 -- 172.21.15.27:0/592209420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe618068e90 con 0x7fe61815def0 2023-12-19T18:28:01.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.493+0000 7fe614ff9640 1 -- 172.21.15.27:0/592209420 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v20)=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe610057ee0 con 0x7fe61815def0 2023-12-19T18:28:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.496+0000 7fe61e02c640 1 -- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5f806a430 msgr2=0x7fe5f806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.496+0000 7fe61e02c640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5f806a430 0x7fe5f806c8f0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fe6180b0530 tx=0x7fe608009040 comp rx=0 tx=0).stop 2023-12-19T18:28:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.496+0000 7fe61e02c640 1 -- 172.21.15.27:0/592209420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe61815def0 msgr2=0x7fe6180af6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.496+0000 7fe61e02c640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe61815def0 0x7fe6180af6c0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fe610007e70 tx=0x7fe61000e4b0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.496+0000 7fe61e02c640 1 -- 172.21.15.27:0/592209420 shutdown_connections 2023-12-19T18:28:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.496+0000 7fe61e02c640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe5f806a430 0x7fe5f806c8f0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.496+0000 7fe61e02c640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe61815def0 0x7fe6180af6c0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.496+0000 7fe61e02c640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe61815d400 0x7fe618157e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.496+0000 7fe61e02c640 1 --2- 172.21.15.27:0/592209420 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe618159a80 0x7fe61814d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.496+0000 7fe61e02c640 1 -- 172.21.15.27:0/592209420 >> 172.21.15.27:0/592209420 conn(0x7fe6180cebd0 msgr2=0x7fe618153360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.496+0000 7fe61e02c640 1 -- 172.21.15.27:0/592209420 shutdown_connections 2023-12-19T18:28:01.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:01.496+0000 7fe61e02c640 1 -- 172.21.15.27:0/592209420 wait complete. 2023-12-19T18:28:01.516 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:28:01.517 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:28:01.643 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:28:01.768 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:28:01.768 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:28:01.768 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:28:01.832 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:28:01.832 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:28:01.832 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:28:01.890 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:28:01.890 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:28:01.890 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:28:01.954 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:28:01.954 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:28:01.954 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:28:02.012 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:28:02.012 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-19T18:28:02.012 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:28:02.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.258+0000 7f5ecac20640 1 Processor -- start 2023-12-19T18:28:02.258 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.258+0000 7f5ecac20640 1 -- start start 2023-12-19T18:28:02.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.260+0000 7f5ecac20640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5ec415ee30 0x7f5ec40c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:02.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.260+0000 7f5ecac20640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ec40afeb0 0x7f5ec40c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:02.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.260+0000 7f5ecac20640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ec40cb350 con 0x7f5ec41553a0 2023-12-19T18:28:02.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.260+0000 7f5ecac20640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec40cbcf0 con 0x7f5ec415ee30 2023-12-19T18:28:02.260 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.260+0000 7f5ecac20640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec40cb630 con 0x7f5ec40afeb0 2023-12-19T18:28:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.260+0000 7f5ec941d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5ec415ee30 0x7f5ec40c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.260+0000 7f5ec941d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f5ec415ee30 0x7f5ec40c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45534/0 (socket says 172.21.15.27:45534) 2023-12-19T18:28:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.260+0000 7f5ec941d640 1 -- 172.21.15.27:0/3565764768 learned_addr learned my addr 172.21.15.27:0/3565764768 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.261+0000 7f5eca41f640 1 --2- 172.21.15.27:0/3565764768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ec40afeb0 0x7f5ec40c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:02.261 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.261+0000 7f5ec941d640 1 -- 172.21.15.27:0/3565764768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ec40afeb0 msgr2=0x7f5ec40c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.261+0000 7f5ec941d640 1 --2- 172.21.15.27:0/3565764768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ec40afeb0 0x7f5ec40c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.261+0000 7f5ec941d640 1 -- 172.21.15.27:0/3565764768 >> v1:172.21.15.192:6789/0 conn(0x7f5ec41553a0 legacy=0x7f5ec415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.261+0000 7f5ec941d640 1 -- 172.21.15.27:0/3565764768 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec40cb910 con 0x7f5ec415ee30 2023-12-19T18:28:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.262+0000 7f5ec941d640 1 --2- 172.21.15.27:0/3565764768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec415ee30 0x7f5ec40c50a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009870 tx=0x7f5eb402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eade449da1849b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:02.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.262+0000 7f5ec8c1c640 1 -- 172.21.15.27:0/3565764768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb40317c0 con 0x7f5ec415ee30 2023-12-19T18:28:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.263+0000 7f5ec8c1c640 1 -- 172.21.15.27:0/3565764768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb4031920 con 0x7f5ec415ee30 2023-12-19T18:28:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.263+0000 7f5ec8c1c640 1 -- 172.21.15.27:0/3565764768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4031cb0 con 0x7f5ec415ee30 2023-12-19T18:28:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.263+0000 7f5ecac20640 1 -- 172.21.15.27:0/3565764768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec415ee30 msgr2=0x7f5ec40c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:02.263 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.263+0000 7f5ecac20640 1 --2- 172.21.15.27:0/3565764768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec415ee30 0x7f5ec40c50a0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009870 tx=0x7f5eb402b860 comp rx=0 tx=0).stop 2023-12-19T18:28:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.264+0000 7f5ecac20640 1 -- 172.21.15.27:0/3565764768 shutdown_connections 2023-12-19T18:28:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.264+0000 7f5ecac20640 1 --2- 172.21.15.27:0/3565764768 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ec40afeb0 0x7f5ec40c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.264+0000 7f5ecac20640 1 --2- 172.21.15.27:0/3565764768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec415ee30 0x7f5ec40c50a0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.264+0000 7f5ecac20640 1 -- 172.21.15.27:0/3565764768 >> 172.21.15.27:0/3565764768 conn(0x7f5ec4069c70 msgr2=0x7f5ec406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.264+0000 7f5ecac20640 1 -- 172.21.15.27:0/3565764768 shutdown_connections 2023-12-19T18:28:02.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.264+0000 7f5ecac20640 1 -- 172.21.15.27:0/3565764768 wait complete. 2023-12-19T18:28:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.265+0000 7f5ecac20640 1 Processor -- start 2023-12-19T18:28:02.265 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.266+0000 7f5ecac20640 1 -- start start 2023-12-19T18:28:02.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.266+0000 7f5ecac20640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec40afeb0 0x7f5ec40b41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.267+0000 7f5ecac20640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ec41553a0 0x7f5ec40c6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.267+0000 7f5ecac20640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ec415ee30 0x7f5ec41f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.267+0000 7f5ecac20640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec41f1d10 con 0x7f5ec40afeb0 2023-12-19T18:28:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.267+0000 7f5ecac20640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec41f1260 con 0x7f5ec41553a0 2023-12-19T18:28:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.267+0000 7f5ecac20640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec4151dc0 con 0x7f5ec415ee30 2023-12-19T18:28:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.267+0000 7f5ec9c1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec40afeb0 0x7f5ec40b41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.267+0000 7f5ec9c1e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec40afeb0 0x7f5ec40b41f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45538/0 (socket says 172.21.15.27:45538) 2023-12-19T18:28:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.267+0000 7f5ec9c1e640 1 -- 172.21.15.27:0/2385216078 learned_addr learned my addr 172.21.15.27:0/2385216078 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.267+0000 7f5ec9c1e640 1 -- 172.21.15.27:0/2385216078 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ec415ee30 msgr2=0x7f5ec41f0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.267+0000 7f5ec9c1e640 1 --2- 172.21.15.27:0/2385216078 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ec415ee30 0x7f5ec41f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.267+0000 7f5ec9c1e640 1 -- 172.21.15.27:0/2385216078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ec41553a0 msgr2=0x7f5ec40c6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.267+0000 7f5ec9c1e640 1 --2- 172.21.15.27:0/2385216078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ec41553a0 0x7f5ec40c6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.267+0000 7f5ec9c1e640 1 -- 172.21.15.27:0/2385216078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb40094d0 con 0x7f5ec40afeb0 2023-12-19T18:28:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.268+0000 7f5ec9c1e640 1 --2- 172.21.15.27:0/2385216078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec40afeb0 0x7f5ec40b41f0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000c430 tx=0x7f5ec000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.268+0000 7f5eba7fc640 1 -- 172.21.15.27:0/2385216078 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0011790 con 0x7f5ec40afeb0 2023-12-19T18:28:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.268+0000 7f5eba7fc640 1 -- 172.21.15.27:0/2385216078 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec00118f0 con 0x7f5ec40afeb0 2023-12-19T18:28:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.268+0000 7f5ecac20640 1 -- 172.21.15.27:0/2385216078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec41f1470 con 0x7f5ec40afeb0 2023-12-19T18:28:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.268+0000 7f5eba7fc640 1 -- 172.21.15.27:0/2385216078 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0011ce0 con 0x7f5ec40afeb0 2023-12-19T18:28:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.268+0000 7f5ecac20640 1 -- 172.21.15.27:0/2385216078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec41f1a30 con 0x7f5ec40afeb0 2023-12-19T18:28:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.269+0000 7f5ecac20640 1 -- 172.21.15.27:0/2385216078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec404e0a0 con 0x7f5ec40afeb0 2023-12-19T18:28:02.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.270+0000 7f5eba7fc640 1 -- 172.21.15.27:0/2385216078 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5ec001c460 con 0x7f5ec40afeb0 2023-12-19T18:28:02.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.271+0000 7f5eba7fc640 1 --2- 172.21.15.27:0/2385216078 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ea806a430 0x7f5ea806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:02.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.271+0000 7f5eba7fc640 1 -- 172.21.15.27:0/2385216078 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f5ec0088650 con 0x7f5ec40afeb0 2023-12-19T18:28:02.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.272+0000 7f5ec941d640 1 --2- 172.21.15.27:0/2385216078 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ea806a430 0x7f5ea806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:02.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.273+0000 7f5ec941d640 1 --2- 172.21.15.27:0/2385216078 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ea806a430 0x7f5ea806c8f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4036000 tx=0x7f5eb4030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:02.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.276+0000 7f5eba7fc640 1 -- 172.21.15.27:0/2385216078 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5ec0053ea0 con 0x7f5ec40afeb0 2023-12-19T18:28:02.510 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.510+0000 7f5ecac20640 1 -- 172.21.15.27:0/2385216078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5ec404e840 con 0x7f5ec40afeb0 2023-12-19T18:28:02.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.511+0000 7f5eba7fc640 1 -- 172.21.15.27:0/2385216078 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f5ec00574c0 con 0x7f5ec40afeb0 2023-12-19T18:28:02.511 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:02.512 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":79,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6044,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/3563146395","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":3563146395},{"type":"v1","addr":"172.21.15.192:6836","nonce":3563146395}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6880,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/3445095528","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":3445095528},{"type":"v1","addr":"172.21.15.192:6837","nonce":3445095528}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":6889,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/3536033217","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":3536033217},{"type":"v1","addr":"172.21.15.27:6833","nonce":3536033217}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":72},{"gid":8928,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/3489887575","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":3489887575},{"type":"v1","addr":"172.21.15.27:6835","nonce":3489887575}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":79}],"filesystems":[]} 2023-12-19T18:28:02.512 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 79 2023-12-19T18:28:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.515+0000 7f5ecac20640 1 -- 172.21.15.27:0/2385216078 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ea806a430 msgr2=0x7f5ea806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.515+0000 7f5ecac20640 1 --2- 172.21.15.27:0/2385216078 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ea806a430 0x7f5ea806c8f0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4036000 tx=0x7f5eb4030000 comp rx=0 tx=0).stop 2023-12-19T18:28:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.515+0000 7f5ecac20640 1 -- 172.21.15.27:0/2385216078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec40afeb0 msgr2=0x7f5ec40b41f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.515+0000 7f5ecac20640 1 --2- 172.21.15.27:0/2385216078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec40afeb0 0x7f5ec40b41f0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000c430 tx=0x7f5ec000c8f0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.515+0000 7f5ecac20640 1 -- 172.21.15.27:0/2385216078 shutdown_connections 2023-12-19T18:28:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.515+0000 7f5ecac20640 1 --2- 172.21.15.27:0/2385216078 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f5ea806a430 0x7f5ea806c8f0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.515+0000 7f5ecac20640 1 --2- 172.21.15.27:0/2385216078 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5ec415ee30 0x7f5ec41f0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.515+0000 7f5ecac20640 1 --2- 172.21.15.27:0/2385216078 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f5ec41553a0 0x7f5ec40c6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.515+0000 7f5ecac20640 1 --2- 172.21.15.27:0/2385216078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec40afeb0 0x7f5ec40b41f0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.515+0000 7f5ecac20640 1 -- 172.21.15.27:0/2385216078 >> 172.21.15.27:0/2385216078 conn(0x7f5ec4069c70 msgr2=0x7f5ec40c9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.515+0000 7f5ecac20640 1 -- 172.21.15.27:0/2385216078 shutdown_connections 2023-12-19T18:28:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.515+0000 7f5ecac20640 1 -- 172.21.15.27:0/2385216078 wait complete. 2023-12-19T18:28:02.536 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-19T18:28:02.537 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-19T18:28:02.537 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:28:02.537 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:28:02.537 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-19T18:28:02.537 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-19T18:28:02.537 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:28:02.537 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:28:02.538 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-19T18:28:02.538 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-19T18:28:02.538 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:28:02.538 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:28:02.538 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-19T18:28:02.538 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-19T18:28:02.538 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:28:02.539 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:28:02.554 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:28:02.554 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-19T18:28:02.558 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:28:02.558 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-19T18:28:02.560 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:28:02.561 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-19T18:28:02.565 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:28:02.565 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-19T18:28:02.606 INFO:tasks.ceph.mds.a:Started 2023-12-19T18:28:02.606 DEBUG:teuthology.parallel:result is None 2023-12-19T18:28:02.613 INFO:tasks.ceph.mds.d:Started 2023-12-19T18:28:02.613 DEBUG:teuthology.parallel:result is None 2023-12-19T18:28:02.614 INFO:tasks.ceph.mds.c:Started 2023-12-19T18:28:02.614 DEBUG:teuthology.parallel:result is None 2023-12-19T18:28:02.621 INFO:tasks.ceph.mds.b:Started 2023-12-19T18:28:02.621 DEBUG:teuthology.parallel:result is None 2023-12-19T18:28:02.621 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-19T18:28:02.779 INFO:tasks.ceph.mds.a.smithi027.stdout:starting mds.a at 2023-12-19T18:28:02.780 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:28:02.781 INFO:tasks.ceph.mds.d.smithi192.stdout:starting mds.d at 2023-12-19T18:28:02.784 INFO:tasks.ceph.mds.b.smithi192.stdout:starting mds.b at 2023-12-19T18:28:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.827+0000 7f1d5f069640 1 Processor -- start 2023-12-19T18:28:02.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.827+0000 7f1d5f069640 1 -- start start 2023-12-19T18:28:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.829+0000 7f1d5f069640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d5806ab00 0x7f1d580d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.829+0000 7f1d5f069640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1d580d2af0 0x7f1d580d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.829+0000 7f1d5f069640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d580d1a50 con 0x7f1d5815d260 2023-12-19T18:28:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.829+0000 7f1d5f069640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d580cdaf0 con 0x7f1d580d2af0 2023-12-19T18:28:02.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.829+0000 7f1d5f069640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d580b71b0 con 0x7f1d5806ab00 2023-12-19T18:28:02.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.829+0000 7f1d5d606640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1d580d2af0 0x7f1d580d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:02.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.829+0000 7f1d5d606640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1d580d2af0 0x7f1d580d13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45566/0 (socket says 172.21.15.27:45566) 2023-12-19T18:28:02.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.829+0000 7f1d5d606640 1 -- 172.21.15.27:0/2027109997 learned_addr learned my addr 172.21.15.27:0/2027109997 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:02.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.829+0000 7f1d5ce05640 1 --1- 172.21.15.27:0/2027109997 >> v1:172.21.15.27:6789/0 conn(0x7f1d5815d260 0x7f1d5806a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:43996/0 (socket says 172.21.15.27:43996) 2023-12-19T18:28:02.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.829+0000 7f1d57fff640 1 --2- 172.21.15.27:0/2027109997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d5806ab00 0x7f1d580d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:02.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.830+0000 7f1d5d606640 1 -- 172.21.15.27:0/2027109997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d5806ab00 msgr2=0x7f1d580d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:02.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.830+0000 7f1d5d606640 1 --2- 172.21.15.27:0/2027109997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d5806ab00 0x7f1d580d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.830+0000 7f1d5d606640 1 -- 172.21.15.27:0/2027109997 >> v1:172.21.15.27:6789/0 conn(0x7f1d5815d260 legacy=0x7f1d5806a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:02.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.830+0000 7f1d5d606640 1 -- 172.21.15.27:0/2027109997 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d580d1d30 con 0x7f1d580d2af0 2023-12-19T18:28:02.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.831+0000 7f1d5d606640 1 --2- 172.21.15.27:0/2027109997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d580d2af0 0x7f1d580d13b0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009e50 tx=0x7f1d5002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=300a58a9ae5d1212 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:02.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.831+0000 7f1d577fe640 1 -- 172.21.15.27:0/2027109997 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5002caf0 con 0x7f1d580d2af0 2023-12-19T18:28:02.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.831+0000 7f1d577fe640 1 -- 172.21.15.27:0/2027109997 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5002cc50 con 0x7f1d580d2af0 2023-12-19T18:28:02.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.831+0000 7f1d577fe640 1 -- 172.21.15.27:0/2027109997 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d500343f0 con 0x7f1d580d2af0 2023-12-19T18:28:02.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.832+0000 7f1d5f069640 1 -- 172.21.15.27:0/2027109997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d580d2af0 msgr2=0x7f1d580d13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:02.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.832+0000 7f1d5f069640 1 --2- 172.21.15.27:0/2027109997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d580d2af0 0x7f1d580d13b0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009e50 tx=0x7f1d5002bf50 comp rx=0 tx=0).stop 2023-12-19T18:28:02.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.832+0000 7f1d5f069640 1 -- 172.21.15.27:0/2027109997 shutdown_connections 2023-12-19T18:28:02.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.832+0000 7f1d5f069640 1 --2- 172.21.15.27:0/2027109997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d580d2af0 0x7f1d580d13b0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.832+0000 7f1d5f069640 1 --2- 172.21.15.27:0/2027109997 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d5806ab00 0x7f1d580d0e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.832+0000 7f1d5f069640 1 -- 172.21.15.27:0/2027109997 >> 172.21.15.27:0/2027109997 conn(0x7f1d580d2f70 msgr2=0x7f1d580d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:02.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.832+0000 7f1d5f069640 1 -- 172.21.15.27:0/2027109997 shutdown_connections 2023-12-19T18:28:02.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.832+0000 7f1d5f069640 1 -- 172.21.15.27:0/2027109997 wait complete. 2023-12-19T18:28:02.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.834+0000 7f1d5f069640 1 Processor -- start 2023-12-19T18:28:02.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.834+0000 7f1d5f069640 1 -- start start 2023-12-19T18:28:02.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.835+0000 7f1d5f069640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d5806ab00 0x7f1d5815fb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:02.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.835+0000 7f1d5f069640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d580d2af0 0x7f1d580cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:02.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.835+0000 7f1d5f069640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5815d260 0x7f1d5815b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:02.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.835+0000 7f1d5f069640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d5814ef80 con 0x7f1d5815d260 2023-12-19T18:28:02.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.835+0000 7f1d5f069640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d580cc080 con 0x7f1d5806ab00 2023-12-19T18:28:02.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.835+0000 7f1d5f069640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d580cb540 con 0x7f1d580d2af0 2023-12-19T18:28:02.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.835+0000 7f1d5d606640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5815d260 0x7f1d5815b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:02.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.835+0000 7f1d5d606640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5815d260 0x7f1d5815b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45576/0 (socket says 172.21.15.27:45576) 2023-12-19T18:28:02.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.835+0000 7f1d5d606640 1 -- 172.21.15.27:0/369025800 learned_addr learned my addr 172.21.15.27:0/369025800 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:02.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.835+0000 7f1d57fff640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d580d2af0 0x7f1d580cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:02.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.835+0000 7f1d5ce05640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d5806ab00 0x7f1d5815fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:02.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.836+0000 7f1d5d606640 1 -- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d580d2af0 msgr2=0x7f1d580cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:02.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.836+0000 7f1d5d606640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d580d2af0 0x7f1d580cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.836+0000 7f1d5d606640 1 -- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d5806ab00 msgr2=0x7f1d5815fb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:02.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.836+0000 7f1d5d606640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d5806ab00 0x7f1d5815fb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:02.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.836+0000 7f1d5d606640 1 -- 172.21.15.27:0/369025800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d50009b30 con 0x7f1d5815d260 2023-12-19T18:28:02.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.836+0000 7f1d5ce05640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d5806ab00 0x7f1d5815fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:28:02.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.836+0000 7f1d5d606640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5815d260 0x7f1d5815b860 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009e50 tx=0x7f1d50004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:02.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.836+0000 7f1d557fa640 1 -- 172.21.15.27:0/369025800 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d50009510 con 0x7f1d5815d260 2023-12-19T18:28:02.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.836+0000 7f1d557fa640 1 -- 172.21.15.27:0/369025800 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d5003e450 con 0x7f1d5815d260 2023-12-19T18:28:02.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.836+0000 7f1d557fa640 1 -- 172.21.15.27:0/369025800 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d5003e800 con 0x7f1d5815d260 2023-12-19T18:28:02.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.837+0000 7f1d5f069640 1 -- 172.21.15.27:0/369025800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d580cb7e0 con 0x7f1d5815d260 2023-12-19T18:28:02.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.837+0000 7f1d5f069640 1 -- 172.21.15.27:0/369025800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d580cbda0 con 0x7f1d5815d260 2023-12-19T18:28:02.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.838+0000 7f1d5f069640 1 -- 172.21.15.27:0/369025800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d58066810 con 0x7f1d5815d260 2023-12-19T18:28:02.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.839+0000 7f1d557fa640 1 -- 172.21.15.27:0/369025800 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1d5003e960 con 0x7f1d5815d260 2023-12-19T18:28:02.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.840+0000 7f1d557fa640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1d3c06a430 0x7f1d3c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:02.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.840+0000 7f1d557fa640 1 -- 172.21.15.27:0/369025800 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f1d500ab2e0 con 0x7f1d5815d260 2023-12-19T18:28:02.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.840+0000 7f1d5ce05640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1d3c06a430 0x7f1d3c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:02.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.841+0000 7f1d5ce05640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1d3c06a430 0x7f1d3c06c8f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c007fd0 tx=0x7f1d4c007d40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:02.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:02.844+0000 7f1d557fa640 1 -- 172.21.15.27:0/369025800 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1d50076bd0 con 0x7f1d5815d260 2023-12-19T18:28:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:03.055+0000 7f1d5f069640 1 -- 172.21.15.27:0/369025800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1d58068e90 con 0x7f1d5815d260 2023-12-19T18:28:04.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.035+0000 7f1d557fa640 1 -- 172.21.15.27:0/369025800 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v198) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1d5007a1f0 con 0x7f1d5815d260 2023-12-19T18:28:04.035 INFO:teuthology.orchestra.run.smithi027.stderr: removed all blocklist entries 2023-12-19T18:28:04.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.039+0000 7f1d5f069640 1 -- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1d3c06a430 msgr2=0x7f1d3c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:04.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.039+0000 7f1d5f069640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1d3c06a430 0x7f1d3c06c8f0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f1d4c007fd0 tx=0x7f1d4c007d40 comp rx=0 tx=0).stop 2023-12-19T18:28:04.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.039+0000 7f1d5f069640 1 -- 172.21.15.27:0/369025800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5815d260 msgr2=0x7f1d5815b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:04.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.039+0000 7f1d5f069640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5815d260 0x7f1d5815b860 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f1d50009e50 tx=0x7f1d50004a40 comp rx=0 tx=0).stop 2023-12-19T18:28:04.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.039+0000 7f1d5f069640 1 -- 172.21.15.27:0/369025800 shutdown_connections 2023-12-19T18:28:04.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.039+0000 7f1d5f069640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1d3c06a430 0x7f1d3c06c8f0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.039+0000 7f1d5f069640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d5815d260 0x7f1d5815b860 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.039+0000 7f1d5f069640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1d580d2af0 0x7f1d580cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.039+0000 7f1d5f069640 1 --2- 172.21.15.27:0/369025800 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1d5806ab00 0x7f1d5815fb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.039+0000 7f1d5f069640 1 -- 172.21.15.27:0/369025800 >> 172.21.15.27:0/369025800 conn(0x7f1d580d2f70 msgr2=0x7f1d5806a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.039+0000 7f1d5f069640 1 -- 172.21.15.27:0/369025800 shutdown_connections 2023-12-19T18:28:04.039 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.040+0000 7f1d5f069640 1 -- 172.21.15.27:0/369025800 wait complete. 2023-12-19T18:28:04.059 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-19T18:28:04.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.292+0000 7fb742c4d640 1 Processor -- start 2023-12-19T18:28:04.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.293+0000 7fb742c4d640 1 -- start start 2023-12-19T18:28:04.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.294+0000 7fb742c4d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb73c0b49d0 0x7fb73c160940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:04.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.294+0000 7fb742c4d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73c0b5ca0 con 0x7fb73c0b89b0 2023-12-19T18:28:04.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.294+0000 7fb742c4d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb73c161800 con 0x7fb73c15cf90 2023-12-19T18:28:04.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.294+0000 7fb742c4d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb73c161240 con 0x7fb73c0b49d0 2023-12-19T18:28:04.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.295+0000 7fb74144a640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb73c0b89b0 0x7fb73c15c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44004/0 (socket says 172.21.15.27:44004) 2023-12-19T18:28:04.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.295+0000 7fb74144a640 1 -- 172.21.15.27:0/1514040006 learned_addr learned my addr 172.21.15.27:0/1514040006 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:04.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.295+0000 7fb741c4b640 1 --2- 172.21.15.27:0/1514040006 >> v2:172.21.15.27:3300/0 conn(0x7fb73c0b49d0 0x7fb73c160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:04.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.295+0000 7fb740c49640 1 -- 172.21.15.27:0/1514040006 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1516599077 0 0) 0x7fb73c0b5ca0 con 0x7fb73c0b89b0 2023-12-19T18:28:04.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.295+0000 7fb741c4b640 1 -- 172.21.15.27:0/1514040006 >> v1:172.21.15.192:6789/0 conn(0x7fb73c15cf90 legacy=0x7fb73c0d15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:04.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.295+0000 7fb741c4b640 1 -- 172.21.15.27:0/1514040006 >> v1:172.21.15.27:6789/0 conn(0x7fb73c0b89b0 legacy=0x7fb73c15c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:04.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.295+0000 7fb741c4b640 1 -- 172.21.15.27:0/1514040006 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb73c161520 con 0x7fb73c0b49d0 2023-12-19T18:28:04.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.297+0000 7fb741c4b640 1 --2- 172.21.15.27:0/1514040006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73c0b49d0 0x7fb73c160940 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009d80 tx=0x7fb72c02bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30615805cc2f38a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:04.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.297+0000 7fb740c49640 1 -- 172.21.15.27:0/1514040006 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c02cd60 con 0x7fb73c0b49d0 2023-12-19T18:28:04.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.297+0000 7fb740c49640 1 -- 172.21.15.27:0/1514040006 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c034440 con 0x7fb73c0b49d0 2023-12-19T18:28:04.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.297+0000 7fb740c49640 1 -- 172.21.15.27:0/1514040006 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c0347d0 con 0x7fb73c0b49d0 2023-12-19T18:28:04.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.297+0000 7fb742c4d640 1 -- 172.21.15.27:0/1514040006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73c0b49d0 msgr2=0x7fb73c160940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:04.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.297+0000 7fb742c4d640 1 --2- 172.21.15.27:0/1514040006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73c0b49d0 0x7fb73c160940 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fb72c009d80 tx=0x7fb72c02bfb0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.298+0000 7fb742c4d640 1 -- 172.21.15.27:0/1514040006 shutdown_connections 2023-12-19T18:28:04.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.298+0000 7fb742c4d640 1 --2- 172.21.15.27:0/1514040006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73c0b49d0 0x7fb73c160940 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.298+0000 7fb742c4d640 1 -- 172.21.15.27:0/1514040006 >> 172.21.15.27:0/1514040006 conn(0x7fb73c0cf760 msgr2=0x7fb73c0cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:04.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.298+0000 7fb742c4d640 1 -- 172.21.15.27:0/1514040006 shutdown_connections 2023-12-19T18:28:04.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.298+0000 7fb742c4d640 1 -- 172.21.15.27:0/1514040006 wait complete. 2023-12-19T18:28:04.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.299+0000 7fb742c4d640 1 Processor -- start 2023-12-19T18:28:04.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.300+0000 7fb742c4d640 1 -- start start 2023-12-19T18:28:04.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb742c4d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73c0b49d0 0x7fb73c15bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb742c4d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb73c0b89b0 0x7fb73c0ccff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb742c4d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73c15cf90 0x7fb73c0cd530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb742c4d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c141d00 con 0x7fb73c0b49d0 2023-12-19T18:28:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb742c4d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c145b80 con 0x7fb73c15cf90 2023-12-19T18:28:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb742c4d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb73c142ab0 con 0x7fb73c0b89b0 2023-12-19T18:28:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb741c4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73c0b49d0 0x7fb73c15bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb741c4b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73c0b49d0 0x7fb73c15bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45596/0 (socket says 172.21.15.27:45596) 2023-12-19T18:28:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb741c4b640 1 -- 172.21.15.27:0/1609221471 learned_addr learned my addr 172.21.15.27:0/1609221471 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:04.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb741c4b640 1 -- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb73c0b89b0 msgr2=0x7fb73c0ccff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:04.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb74144a640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb73c0b89b0 0x7fb73c0ccff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb74244c640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73c15cf90 0x7fb73c0cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb741c4b640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb73c0b89b0 0x7fb73c0ccff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb741c4b640 1 -- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73c15cf90 msgr2=0x7fb73c0cd530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb741c4b640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73c15cf90 0x7fb73c0cd530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb741c4b640 1 -- 172.21.15.27:0/1609221471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb72c009a10 con 0x7fb73c0b49d0 2023-12-19T18:28:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.301+0000 7fb74144a640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb73c0b89b0 0x7fb73c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.302+0000 7fb74244c640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73c15cf90 0x7fb73c0cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.302+0000 7fb741c4b640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73c0b49d0 0x7fb73c15bba0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fb72c037040 tx=0x7fb72c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:04.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.302+0000 7fb73a7fc640 1 -- 172.21.15.27:0/1609221471 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c040650 con 0x7fb73c0b49d0 2023-12-19T18:28:04.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.302+0000 7fb73a7fc640 1 -- 172.21.15.27:0/1609221471 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb72c0407b0 con 0x7fb73c0b49d0 2023-12-19T18:28:04.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.302+0000 7fb73a7fc640 1 -- 172.21.15.27:0/1609221471 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72c040b60 con 0x7fb73c0b49d0 2023-12-19T18:28:04.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.302+0000 7fb742c4d640 1 -- 172.21.15.27:0/1609221471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb73c141ee0 con 0x7fb73c0b49d0 2023-12-19T18:28:04.303 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.302+0000 7fb742c4d640 1 -- 172.21.15.27:0/1609221471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb73c142180 con 0x7fb73c0b49d0 2023-12-19T18:28:04.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.304+0000 7fb742c4d640 1 -- 172.21.15.27:0/1609221471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73c160e00 con 0x7fb73c0b49d0 2023-12-19T18:28:04.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.304+0000 7fb73a7fc640 1 -- 172.21.15.27:0/1609221471 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb72c040cc0 con 0x7fb73c0b49d0 2023-12-19T18:28:04.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.305+0000 7fb73a7fc640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb72806a430 0x7fb72806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:04.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.306+0000 7fb73a7fc640 1 -- 172.21.15.27:0/1609221471 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb72c0ab560 con 0x7fb73c0b49d0 2023-12-19T18:28:04.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.306+0000 7fb74144a640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb72806a430 0x7fb72806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:04.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.307+0000 7fb74144a640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb72806a430 0x7fb72806c8f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb73c1563c0 tx=0x7fb730009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.310+0000 7fb73a7fc640 1 -- 172.21.15.27:0/1609221471 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb72c076f30 con 0x7fb73c0b49d0 2023-12-19T18:28:04.571 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.572+0000 7fb742c4d640 1 -- 172.21.15.27:0/1609221471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb73c04e840 con 0x7fb73c0b49d0 2023-12-19T18:28:04.572 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.573+0000 7fb73a7fc640 1 -- 172.21.15.27:0/1609221471 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fb72c07a550 con 0x7fb73c0b49d0 2023-12-19T18:28:04.572 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:04.573 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:28:04.573 INFO:teuthology.orchestra.run.smithi027.stdout: "auth_dump": [ 2023-12-19T18:28:04.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.573 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.a", 2023-12-19T18:28:04.573 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFl+4ruHBAAXwbpmT43SmlvNbuZj5e+bQ==", 2023-12-19T18:28:04.573 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.573 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:28:04.573 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:28:04.573 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:28:04.573 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.573 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.574 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.b", 2023-12-19T18:28:04.574 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmZyUBhAAG3kZkXvJIo7N9f1j+DPCSQ==", 2023-12-19T18:28:04.574 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.574 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:28:04.574 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:28:04.574 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:28:04.574 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.574 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.574 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.c", 2023-12-19T18:28:04.574 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlChNOMBAAX4GV3iIGkSQdAsYu9oCPDw==", 2023-12-19T18:28:04.574 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mds.d", 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlTmFXFhAABA0NMUxKBV1ExAhYc5RFbQ==", 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.575 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.0", 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFluleFIBAAs1goFVMOX4CbjwFhg0KoBg==", 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.1", 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCA3YFlInW3MxAAI4hY1GBxXYXDswKKrEyDTg==", 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.576 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.2", 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFlAWZ5CRAA82/2r9/sT0L9VlGHbXksQw==", 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.577 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.3", 2023-12-19T18:28:04.578 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCB3YFl1IvaGRAAqVacDDPLx+NqBOGp6MeX2w==", 2023-12-19T18:28:04.578 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.578 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:28:04.578 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:28:04.578 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.578 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.578 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.4", 2023-12-19T18:28:04.578 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlr0BBHRAA+7gfs/RG6DtABnYPg059Tw==", 2023-12-19T18:28:04.578 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.5", 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCF3YFlXZbELRAAKD0qOkhk7skfc4xcAC8Lhg==", 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.6", 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlF07MARAA2TJJ+5C1HIU+SVhqmKaVHA==", 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "osd.7", 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQCG3YFlISPGEhAAdFIRRkXlLyOx/oMZ4MgL9Q==", 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow profile osd", 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile osd", 2023-12-19T18:28:04.580 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.0", 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFlmNM9JRAARgS20O8zpm6H70WDv+lgGw==", 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow r", 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.1", 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlWoZ8AhAADWvKVzeDhDGHtYNUFIOWsA==", 2023-12-19T18:28:04.581 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.2", 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB73YFllWFVMhAAD9ebDOppHTODVQjg6ppP3g==", 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:28:04.582 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.583 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.3", 2023-12-19T18:28:04.583 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB83YFlwbIQDRAAt/h55zBNCEId0vkYSUQ3TQ==", 2023-12-19T18:28:04.583 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.583 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow", 2023-12-19T18:28:04.583 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow r", 2023-12-19T18:28:04.583 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow rw", 2023-12-19T18:28:04.583 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow rwx" 2023-12-19T18:28:04.583 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "client.admin", 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB53YFloCb6CxAAmZQyz5XOEqDOOiROJHLzxQ==", 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": "allow *", 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow *", 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.x", 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFltBhxDxAAiAS9CYPY86IMmJ7NGgA2lw==", 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:28:04.584 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout: "entity": "mgr.y", 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout: "key": "AQB63YFlUnAhBBAAi++W9XK3o3Y/185hB3LuIg==", 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout: "caps": { 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": "allow *", 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": "allow profile mgr", 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": "allow *" 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:04.585 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:28:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.575+0000 7fb742c4d640 1 -- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb72806a430 msgr2=0x7fb72806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.575+0000 7fb742c4d640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb72806a430 0x7fb72806c8f0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fb73c1563c0 tx=0x7fb730009040 comp rx=0 tx=0).stop 2023-12-19T18:28:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.576+0000 7fb742c4d640 1 -- 172.21.15.27:0/1609221471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73c0b49d0 msgr2=0x7fb73c15bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:04.586 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.576+0000 7fb742c4d640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73c0b49d0 0x7fb73c15bba0 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fb72c037040 tx=0x7fb72c004a40 comp rx=0 tx=0).stop 2023-12-19T18:28:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.576+0000 7fb742c4d640 1 -- 172.21.15.27:0/1609221471 shutdown_connections 2023-12-19T18:28:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.576+0000 7fb742c4d640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb72806a430 0x7fb72806c8f0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.576+0000 7fb742c4d640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb73c15cf90 0x7fb73c0cd530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.576+0000 7fb742c4d640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb73c0b89b0 0x7fb73c0ccff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.576+0000 7fb742c4d640 1 --2- 172.21.15.27:0/1609221471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb73c0b49d0 0x7fb73c15bba0 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.576+0000 7fb742c4d640 1 -- 172.21.15.27:0/1609221471 >> 172.21.15.27:0/1609221471 conn(0x7fb73c0cf760 msgr2=0x7fb73c15d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.576+0000 7fb742c4d640 1 -- 172.21.15.27:0/1609221471 shutdown_connections 2023-12-19T18:28:04.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.576+0000 7fb742c4d640 1 -- 172.21.15.27:0/1609221471 wait complete. 2023-12-19T18:28:04.597 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:28:04.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.831+0000 7fd3d7547640 1 Processor -- start 2023-12-19T18:28:04.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.831+0000 7fd3d7547640 1 -- start start 2023-12-19T18:28:04.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.833+0000 7fd3d7547640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd3d015b870 0x7fd3d00c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:04.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.833+0000 7fd3d7547640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd3d00cdf90 0x7fd3d0164d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:04.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.833+0000 7fd3d7547640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3d00c4a20 con 0x7fd3d0165290 2023-12-19T18:28:04.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.833+0000 7fd3d7547640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d0165740 con 0x7fd3d015b870 2023-12-19T18:28:04.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.833+0000 7fd3d7547640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3d00ce750 con 0x7fd3d00cdf90 2023-12-19T18:28:04.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.834+0000 7fd3d6545640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd3d015b870 0x7fd3d00c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:04.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.834+0000 7fd3d6545640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd3d015b870 0x7fd3d00c5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45602/0 (socket says 172.21.15.27:45602) 2023-12-19T18:28:04.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.834+0000 7fd3d6545640 1 -- 172.21.15.27:0/113281926 learned_addr learned my addr 172.21.15.27:0/113281926 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:04.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.834+0000 7fd3d5d44640 1 --2- 172.21.15.27:0/113281926 >> v2:172.21.15.192:3300/0 conn(0x7fd3d00cdf90 0x7fd3d0164d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:04.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.834+0000 7fd3d6545640 1 -- 172.21.15.27:0/113281926 >> v2:172.21.15.192:3300/0 conn(0x7fd3d00cdf90 msgr2=0x7fd3d0164d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:04.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.834+0000 7fd3d6545640 1 --2- 172.21.15.27:0/113281926 >> v2:172.21.15.192:3300/0 conn(0x7fd3d00cdf90 0x7fd3d0164d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.834+0000 7fd3d6545640 1 -- 172.21.15.27:0/113281926 >> v1:172.21.15.27:6789/0 conn(0x7fd3d0165290 legacy=0x7fd3d0161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:04.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.834+0000 7fd3d6545640 1 -- 172.21.15.27:0/113281926 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3d015bc50 con 0x7fd3d015b870 2023-12-19T18:28:04.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.834+0000 7fd3d5543640 1 -- 172.21.15.27:0/113281926 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3281048631 0 0) 0x7fd3d00c4a20 con 0x7fd3d0165290 2023-12-19T18:28:04.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.836+0000 7fd3d6545640 1 --2- 172.21.15.27:0/113281926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3d015b870 0x7fd3d00c5260 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00ccb90 tx=0x7fd3cc02b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac4dfe67b7f1d05 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:04.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.836+0000 7fd3d5543640 1 -- 172.21.15.27:0/113281926 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc030860 con 0x7fd3d015b870 2023-12-19T18:28:04.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.836+0000 7fd3d5543640 1 -- 172.21.15.27:0/113281926 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc0309c0 con 0x7fd3d015b870 2023-12-19T18:28:04.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.836+0000 7fd3d5543640 1 -- 172.21.15.27:0/113281926 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc030d50 con 0x7fd3d015b870 2023-12-19T18:28:04.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.836+0000 7fd3d7547640 1 -- 172.21.15.27:0/113281926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3d015b870 msgr2=0x7fd3d00c5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:04.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.836+0000 7fd3d7547640 1 --2- 172.21.15.27:0/113281926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3d015b870 0x7fd3d00c5260 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fd3d00ccb90 tx=0x7fd3cc02b9a0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.836+0000 7fd3d7547640 1 -- 172.21.15.27:0/113281926 shutdown_connections 2023-12-19T18:28:04.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.836+0000 7fd3d7547640 1 --2- 172.21.15.27:0/113281926 >> v2:172.21.15.192:3300/0 conn(0x7fd3d00cdf90 0x7fd3d0164d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.836+0000 7fd3d7547640 1 --2- 172.21.15.27:0/113281926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3d015b870 0x7fd3d00c5260 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.837+0000 7fd3d7547640 1 -- 172.21.15.27:0/113281926 >> 172.21.15.27:0/113281926 conn(0x7fd3d0069c70 msgr2=0x7fd3d006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:04.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.837+0000 7fd3d7547640 1 -- 172.21.15.27:0/113281926 shutdown_connections 2023-12-19T18:28:04.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.837+0000 7fd3d7547640 1 -- 172.21.15.27:0/113281926 wait complete. 2023-12-19T18:28:04.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.838+0000 7fd3d7547640 1 Processor -- start 2023-12-19T18:28:04.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.839+0000 7fd3d7547640 1 -- start start 2023-12-19T18:28:04.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.839+0000 7fd3d7547640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3d00cdf90 0x7fd3d00b33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:04.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d7547640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3d015b870 0x7fd3d01fa250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:04.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d7547640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d0165290 0x7fd3d0147290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:04.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d7547640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d00b4990 con 0x7fd3d00cdf90 2023-12-19T18:28:04.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d7547640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d00b3d10 con 0x7fd3d015b870 2023-12-19T18:28:04.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d7547640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3d00b3ff0 con 0x7fd3d0165290 2023-12-19T18:28:04.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d6545640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3d00cdf90 0x7fd3d00b33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:04.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d6545640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3d00cdf90 0x7fd3d00b33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45606/0 (socket says 172.21.15.27:45606) 2023-12-19T18:28:04.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d6545640 1 -- 172.21.15.27:0/3517687073 learned_addr learned my addr 172.21.15.27:0/3517687073 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:04.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d6d46640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d0165290 0x7fd3d0147290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:04.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d5d44640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3d015b870 0x7fd3d01fa250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d6545640 1 -- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d0165290 msgr2=0x7fd3d0147290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d6545640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d0165290 0x7fd3d0147290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d6545640 1 -- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3d015b870 msgr2=0x7fd3d01fa250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d6545640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3d015b870 0x7fd3d01fa250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.840+0000 7fd3d6545640 1 -- 172.21.15.27:0/3517687073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3cc0094d0 con 0x7fd3d00cdf90 2023-12-19T18:28:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.841+0000 7fd3d6545640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3d00cdf90 0x7fd3d00b33c0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0097f0 tx=0x7fd3cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.841+0000 7fd3c6ffd640 1 -- 172.21.15.27:0/3517687073 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc039c90 con 0x7fd3d00cdf90 2023-12-19T18:28:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.841+0000 7fd3c6ffd640 1 -- 172.21.15.27:0/3517687073 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3cc039df0 con 0x7fd3d00cdf90 2023-12-19T18:28:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.841+0000 7fd3d7547640 1 -- 172.21.15.27:0/3517687073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3d00b42d0 con 0x7fd3d00cdf90 2023-12-19T18:28:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.841+0000 7fd3c6ffd640 1 -- 172.21.15.27:0/3517687073 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3cc043620 con 0x7fd3d00cdf90 2023-12-19T18:28:04.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.841+0000 7fd3d7547640 1 -- 172.21.15.27:0/3517687073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3d00b3900 con 0x7fd3d00cdf90 2023-12-19T18:28:04.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.843+0000 7fd3d7547640 1 -- 172.21.15.27:0/3517687073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3d004e840 con 0x7fd3d00cdf90 2023-12-19T18:28:04.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.844+0000 7fd3c6ffd640 1 -- 172.21.15.27:0/3517687073 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd3cc043780 con 0x7fd3d00cdf90 2023-12-19T18:28:04.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.845+0000 7fd3c6ffd640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd3b406a430 0x7fd3b406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:04.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.845+0000 7fd3c6ffd640 1 -- 172.21.15.27:0/3517687073 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd3cc03e070 con 0x7fd3d00cdf90 2023-12-19T18:28:04.845 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.846+0000 7fd3d5d44640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd3b406a430 0x7fd3b406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:04.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.847+0000 7fd3d5d44640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd3b406a430 0x7fd3b406c8f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0006fd0 tx=0x7fd3c0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:04.849 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:04.849+0000 7fd3c6ffd640 1 -- 172.21.15.27:0/3517687073 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd3cc0770d0 con 0x7fd3d00cdf90 2023-12-19T18:28:05.044 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.044+0000 7fd3d7547640 1 -- 172.21.15.27:0/3517687073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3d01611a0 con 0x7fd3d00cdf90 2023-12-19T18:28:05.046 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.046+0000 7fd3c6ffd640 1 -- 172.21.15.27:0/3517687073 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+12400 (secure 0 0 0) 0x7fd3cc07a6f0 con 0x7fd3d00cdf90 2023-12-19T18:28:05.046 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:05.047 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":198,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:28:04.017127+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":186,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":183,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:28:05.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.049+0000 7fd3d7547640 1 -- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd3b406a430 msgr2=0x7fd3b406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:05.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.049+0000 7fd3d7547640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd3b406a430 0x7fd3b406c8f0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fd3c0006fd0 tx=0x7fd3c0008040 comp rx=0 tx=0).stop 2023-12-19T18:28:05.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.049+0000 7fd3d7547640 1 -- 172.21.15.27:0/3517687073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3d00cdf90 msgr2=0x7fd3d00b33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.049+0000 7fd3d7547640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3d00cdf90 0x7fd3d00b33c0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fd3cc0097f0 tx=0x7fd3cc004730 comp rx=0 tx=0).stop 2023-12-19T18:28:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.049+0000 7fd3d7547640 1 -- 172.21.15.27:0/3517687073 shutdown_connections 2023-12-19T18:28:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.049+0000 7fd3d7547640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd3b406a430 0x7fd3b406c8f0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.049+0000 7fd3d7547640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd3d0165290 0x7fd3d0147290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.049+0000 7fd3d7547640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd3d015b870 0x7fd3d01fa250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.049+0000 7fd3d7547640 1 --2- 172.21.15.27:0/3517687073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3d00cdf90 0x7fd3d00b33c0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.049+0000 7fd3d7547640 1 -- 172.21.15.27:0/3517687073 >> 172.21.15.27:0/3517687073 conn(0x7fd3d0069c70 msgr2=0x7fd3d00c38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.049+0000 7fd3d7547640 1 -- 172.21.15.27:0/3517687073 shutdown_connections 2023-12-19T18:28:05.050 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.049+0000 7fd3d7547640 1 -- 172.21.15.27:0/3517687073 wait complete. 2023-12-19T18:28:05.068 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:28:05.069 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:28:05.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.305+0000 7f772c86d640 1 Processor -- start 2023-12-19T18:28:05.305 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.305+0000 7f772c86d640 1 -- start start 2023-12-19T18:28:05.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.307+0000 7f772c86d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f772815d120 0x7f77280cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:05.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.307+0000 7f772c86d640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f77280ce950 0x7f77280d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:05.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.307+0000 7f772c86d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77280d3850 con 0x7f77280d1310 2023-12-19T18:28:05.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.307+0000 7f772c86d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77280d0b10 con 0x7f772815d120 2023-12-19T18:28:05.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.307+0000 7f772c86d640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77280d1c90 con 0x7f77280ce950 2023-12-19T18:28:05.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.307+0000 7f77277fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f772815d120 0x7f77280cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:05.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.307+0000 7f771ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f77280d1310 0x7f77280b2970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44018/0 (socket says 172.21.15.27:44018) 2023-12-19T18:28:05.307 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.307+0000 7f771ffff640 1 -- 172.21.15.27:0/379628073 learned_addr learned my addr 172.21.15.27:0/379628073 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:05.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.308+0000 7f7727fff640 1 --2- 172.21.15.27:0/379628073 >> v2:172.21.15.192:3300/0 conn(0x7f77280ce950 0x7f77280d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:05.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.308+0000 7f77277fe640 1 -- 172.21.15.27:0/379628073 >> v2:172.21.15.192:3300/0 conn(0x7f77280ce950 msgr2=0x7f77280d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:05.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.308+0000 7f77277fe640 1 --2- 172.21.15.27:0/379628073 >> v2:172.21.15.192:3300/0 conn(0x7f77280ce950 0x7f77280d1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.308+0000 7f77277fe640 1 -- 172.21.15.27:0/379628073 >> v1:172.21.15.27:6789/0 conn(0x7f77280d1310 legacy=0x7f77280b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:05.308 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.308+0000 7f77277fe640 1 -- 172.21.15.27:0/379628073 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f772815d500 con 0x7f772815d120 2023-12-19T18:28:05.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.309+0000 7f77277fe640 1 --2- 172.21.15.27:0/379628073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f772815d120 0x7f77280cfd70 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f772808d520 tx=0x7f771802bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=529aa62e03beeea0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:05.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.309+0000 7f7726ffd640 1 -- 172.21.15.27:0/379628073 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f771802cc70 con 0x7f772815d120 2023-12-19T18:28:05.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.309+0000 7f7726ffd640 1 -- 172.21.15.27:0/379628073 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f771802cdd0 con 0x7f772815d120 2023-12-19T18:28:05.309 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.309+0000 7f7726ffd640 1 -- 172.21.15.27:0/379628073 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7718034660 con 0x7f772815d120 2023-12-19T18:28:05.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.310+0000 7f772c86d640 1 -- 172.21.15.27:0/379628073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f772815d120 msgr2=0x7f77280cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:05.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.310+0000 7f772c86d640 1 --2- 172.21.15.27:0/379628073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f772815d120 0x7f77280cfd70 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7f772808d520 tx=0x7f771802bec0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.310+0000 7f772c86d640 1 -- 172.21.15.27:0/379628073 shutdown_connections 2023-12-19T18:28:05.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.310+0000 7f772c86d640 1 --2- 172.21.15.27:0/379628073 >> v2:172.21.15.192:3300/0 conn(0x7f77280ce950 0x7f77280d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.310 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.310+0000 7f772c86d640 1 --2- 172.21.15.27:0/379628073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f772815d120 0x7f77280cfd70 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.310+0000 7f772c86d640 1 -- 172.21.15.27:0/379628073 >> 172.21.15.27:0/379628073 conn(0x7f77280d2d30 msgr2=0x7f77280d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:05.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.310+0000 7f772c86d640 1 -- 172.21.15.27:0/379628073 shutdown_connections 2023-12-19T18:28:05.311 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.310+0000 7f772c86d640 1 -- 172.21.15.27:0/379628073 wait complete. 2023-12-19T18:28:05.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.313+0000 7f772c86d640 1 Processor -- start 2023-12-19T18:28:05.312 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.313+0000 7f772c86d640 1 -- start start 2023-12-19T18:28:05.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.314+0000 7f772c86d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77280ce950 0x7f772814d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:05.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.314+0000 7f772c86d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77280d1310 0x7f77280b8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:05.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.314+0000 7f772c86d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f772815d120 0x7f772806b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:05.314 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.314+0000 7f772c86d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772815da70 con 0x7f772815d120 2023-12-19T18:28:05.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.314+0000 7f772c86d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f772815fd20 con 0x7f77280d1310 2023-12-19T18:28:05.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.314+0000 7f772c86d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7728160000 con 0x7f77280ce950 2023-12-19T18:28:05.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.314+0000 7f7727fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f772815d120 0x7f772806b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:05.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.315+0000 7f7727fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f772815d120 0x7f772806b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45628/0 (socket says 172.21.15.27:45628) 2023-12-19T18:28:05.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.315+0000 7f7727fff640 1 -- 172.21.15.27:0/1412699093 learned_addr learned my addr 172.21.15.27:0/1412699093 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:05.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.315+0000 7f77277fe640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77280ce950 0x7f772814d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:05.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.315+0000 7f771ffff640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77280d1310 0x7f77280b8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:05.316 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.315+0000 7f7727fff640 1 -- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77280ce950 msgr2=0x7f772814d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:05.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.315+0000 7f7727fff640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77280ce950 0x7f772814d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.315+0000 7f7727fff640 1 -- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77280d1310 msgr2=0x7f77280b8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:05.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.315+0000 7f7727fff640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77280d1310 0x7f77280b8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.315+0000 7f7727fff640 1 -- 172.21.15.27:0/1412699093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7718009a70 con 0x7f772815d120 2023-12-19T18:28:05.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.315+0000 7f7727fff640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f772815d120 0x7f772806b3d0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f772000d5d0 tx=0x7f772000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:05.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.316+0000 7f7724ff9640 1 -- 172.21.15.27:0/1412699093 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77200125f0 con 0x7f772815d120 2023-12-19T18:28:05.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.316+0000 7f7724ff9640 1 -- 172.21.15.27:0/1412699093 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7720012770 con 0x7f772815d120 2023-12-19T18:28:05.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.316+0000 7f7724ff9640 1 -- 172.21.15.27:0/1412699093 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7720012b20 con 0x7f772815d120 2023-12-19T18:28:05.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.316+0000 7f772c86d640 1 -- 172.21.15.27:0/1412699093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77281602e0 con 0x7f772815d120 2023-12-19T18:28:05.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.316+0000 7f772c86d640 1 -- 172.21.15.27:0/1412699093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77280b59f0 con 0x7f772815d120 2023-12-19T18:28:05.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.317+0000 7f772c86d640 1 -- 172.21.15.27:0/1412699093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77280b5c10 con 0x7f772815d120 2023-12-19T18:28:05.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.318+0000 7f7724ff9640 1 -- 172.21.15.27:0/1412699093 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7720012c80 con 0x7f772815d120 2023-12-19T18:28:05.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.320+0000 7f7724ff9640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f770806a430 0x7f770806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:05.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.320+0000 7f7724ff9640 1 -- 172.21.15.27:0/1412699093 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f7720088b90 con 0x7f772815d120 2023-12-19T18:28:05.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.321+0000 7f77277fe640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f770806a430 0x7f770806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:05.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.322+0000 7f77277fe640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f770806a430 0x7f770806c8f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7718036000 tx=0x7f7718030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:05.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.323+0000 7f7724ff9640 1 -- 172.21.15.27:0/1412699093 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f77200543b0 con 0x7f772815d120 2023-12-19T18:28:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.519+0000 7f772c86d640 1 -- 172.21.15.27:0/1412699093 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f772804e840 con 0x7f772815d120 2023-12-19T18:28:05.519 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.520+0000 7f7724ff9640 1 -- 172.21.15.27:0/1412699093 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v198) v1 ==== 93+0+10 (secure 0 0 0) 0x7f77200579d0 con 0x7f772815d120 2023-12-19T18:28:05.519 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:28:05.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.522+0000 7f772c86d640 1 -- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f770806a430 msgr2=0x7f770806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:05.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.522+0000 7f772c86d640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f770806a430 0x7f770806c8f0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f7718036000 tx=0x7f7718030000 comp rx=0 tx=0).stop 2023-12-19T18:28:05.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.522+0000 7f772c86d640 1 -- 172.21.15.27:0/1412699093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f772815d120 msgr2=0x7f772806b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:05.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.522+0000 7f772c86d640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f772815d120 0x7f772806b3d0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f772000d5d0 tx=0x7f772000da90 comp rx=0 tx=0).stop 2023-12-19T18:28:05.522 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.522+0000 7f772c86d640 1 -- 172.21.15.27:0/1412699093 shutdown_connections 2023-12-19T18:28:05.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.522+0000 7f772c86d640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f770806a430 0x7f770806c8f0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.522+0000 7f772c86d640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f772815d120 0x7f772806b3d0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.522+0000 7f772c86d640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f77280d1310 0x7f77280b8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.522+0000 7f772c86d640 1 --2- 172.21.15.27:0/1412699093 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f77280ce950 0x7f772814d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.522+0000 7f772c86d640 1 -- 172.21.15.27:0/1412699093 >> 172.21.15.27:0/1412699093 conn(0x7f77280d2d30 msgr2=0x7f77280b1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:05.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.522+0000 7f772c86d640 1 -- 172.21.15.27:0/1412699093 shutdown_connections 2023-12-19T18:28:05.523 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.523+0000 7f772c86d640 1 -- 172.21.15.27:0/1412699093 wait complete. 2023-12-19T18:28:05.543 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-19T18:28:05.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.783+0000 7f801324d640 1 Processor -- start 2023-12-19T18:28:05.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.783+0000 7f801324d640 1 -- start start 2023-12-19T18:28:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.784+0000 7f801324d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c15d260 0x7f800c06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.784+0000 7f801324d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f800c0d2af0 0x7f800c06a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.784+0000 7f801324d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f800c0b71b0 con 0x7f800c0d0e70 2023-12-19T18:28:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.784+0000 7f801324d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f800c0cdaf0 con 0x7f800c0d2af0 2023-12-19T18:28:05.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.784+0000 7f801324d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c0d1a50 con 0x7f800c15d260 2023-12-19T18:28:05.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.785+0000 7f800bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f800c0d2af0 0x7f800c06a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:05.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.785+0000 7f800bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f800c0d2af0 0x7f800c06a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45634/0 (socket says 172.21.15.27:45634) 2023-12-19T18:28:05.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.785+0000 7f800bfff640 1 -- 172.21.15.27:0/3640493089 learned_addr learned my addr 172.21.15.27:0/3640493089 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.785+0000 7f80117ea640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f800c0d0e70 0x7f800c0b49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44030/0 (socket says 172.21.15.27:44030) 2023-12-19T18:28:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.785+0000 7f8010fe9640 1 --2- 172.21.15.27:0/3640493089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c15d260 0x7f800c06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.785+0000 7f800bfff640 1 -- 172.21.15.27:0/3640493089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c15d260 msgr2=0x7f800c06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.785+0000 7f800bfff640 1 --2- 172.21.15.27:0/3640493089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c15d260 0x7f800c06a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.785+0000 7f800bfff640 1 -- 172.21.15.27:0/3640493089 >> v1:172.21.15.27:6789/0 conn(0x7f800c0d0e70 legacy=0x7f800c0b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.785+0000 7f800bfff640 1 -- 172.21.15.27:0/3640493089 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f800c0d1d30 con 0x7f800c0d2af0 2023-12-19T18:28:05.786 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.785+0000 7f800b7fe640 1 -- 172.21.15.27:0/3640493089 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456552677 0 0) 0x7f8004001110 con 0x7f800c0d0e70 2023-12-19T18:28:05.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.787+0000 7f800bfff640 1 --2- 172.21.15.27:0/3640493089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800c0d2af0 0x7f800c06a930 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009e50 tx=0x7f7ffc02bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8393654975723641 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:05.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.787+0000 7f800b7fe640 1 -- 172.21.15.27:0/3640493089 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc02cb00 con 0x7f800c0d2af0 2023-12-19T18:28:05.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.787+0000 7f800b7fe640 1 -- 172.21.15.27:0/3640493089 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc02cc60 con 0x7f800c0d2af0 2023-12-19T18:28:05.787 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.787+0000 7f800b7fe640 1 -- 172.21.15.27:0/3640493089 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0343f0 con 0x7f800c0d2af0 2023-12-19T18:28:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.787+0000 7f801324d640 1 -- 172.21.15.27:0/3640493089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800c0d2af0 msgr2=0x7f800c06a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.788+0000 7f801324d640 1 --2- 172.21.15.27:0/3640493089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800c0d2af0 0x7f800c06a930 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7f7ffc009e50 tx=0x7f7ffc02bf60 comp rx=0 tx=0).stop 2023-12-19T18:28:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.788+0000 7f801324d640 1 -- 172.21.15.27:0/3640493089 shutdown_connections 2023-12-19T18:28:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.788+0000 7f801324d640 1 --2- 172.21.15.27:0/3640493089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800c0d2af0 0x7f800c06a930 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.788+0000 7f801324d640 1 --2- 172.21.15.27:0/3640493089 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c15d260 0x7f800c06a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.788+0000 7f801324d640 1 -- 172.21.15.27:0/3640493089 >> 172.21.15.27:0/3640493089 conn(0x7f800c0d2f70 msgr2=0x7f800c0d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.788+0000 7f801324d640 1 -- 172.21.15.27:0/3640493089 shutdown_connections 2023-12-19T18:28:05.788 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.788+0000 7f801324d640 1 -- 172.21.15.27:0/3640493089 wait complete. 2023-12-19T18:28:05.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.789+0000 7f801324d640 1 Processor -- start 2023-12-19T18:28:05.789 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.790+0000 7f801324d640 1 -- start start 2023-12-19T18:28:05.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.791+0000 7f801324d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c0d0e70 0x7f800c15fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:05.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.791+0000 7f801324d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800c0d2af0 0x7f800c1f21e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:05.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.791+0000 7f801324d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800c15d260 0x7f800c0ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:05.791 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.791+0000 7f801324d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c14ef80 con 0x7f800c15d260 2023-12-19T18:28:05.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.791+0000 7f801324d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f800c14f110 con 0x7f800c0d2af0 2023-12-19T18:28:05.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.791+0000 7f801324d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f800c1f2720 con 0x7f800c0d0e70 2023-12-19T18:28:05.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.791+0000 7f80117ea640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800c15d260 0x7f800c0ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:05.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.791+0000 7f80117ea640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800c15d260 0x7f800c0ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45644/0 (socket says 172.21.15.27:45644) 2023-12-19T18:28:05.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.791+0000 7f80117ea640 1 -- 172.21.15.27:0/528973718 learned_addr learned my addr 172.21.15.27:0/528973718 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:05.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.791+0000 7f8010fe9640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c0d0e70 0x7f800c15fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:05.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.791+0000 7f800bfff640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800c0d2af0 0x7f800c1f21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:05.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.791+0000 7f80117ea640 1 -- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c0d0e70 msgr2=0x7f800c15fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:05.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.792+0000 7f80117ea640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c0d0e70 0x7f800c15fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.792+0000 7f80117ea640 1 -- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800c0d2af0 msgr2=0x7f800c1f21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:05.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.792+0000 7f80117ea640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800c0d2af0 0x7f800c1f21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:05.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.792+0000 7f80117ea640 1 -- 172.21.15.27:0/528973718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ffc009b30 con 0x7f800c15d260 2023-12-19T18:28:05.792 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.792+0000 7f800bfff640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800c0d2af0 0x7f800c1f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:05.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.792+0000 7f80117ea640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800c15d260 0x7f800c0ca9c0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f800400c780 tx=0x7f800400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:05.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.792+0000 7f80097fa640 1 -- 172.21.15.27:0/528973718 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80040072d0 con 0x7f800c15d260 2023-12-19T18:28:05.793 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.792+0000 7f80097fa640 1 -- 172.21.15.27:0/528973718 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8004007450 con 0x7f800c15d260 2023-12-19T18:28:05.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.792+0000 7f80097fa640 1 -- 172.21.15.27:0/528973718 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8004001d10 con 0x7f800c15d260 2023-12-19T18:28:05.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.792+0000 7f801324d640 1 -- 172.21.15.27:0/528973718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f800c0cb210 con 0x7f800c15d260 2023-12-19T18:28:05.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.792+0000 7f801324d640 1 -- 172.21.15.27:0/528973718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800c0cb7d0 con 0x7f800c15d260 2023-12-19T18:28:05.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.793+0000 7f801324d640 1 -- 172.21.15.27:0/528973718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800c066810 con 0x7f800c15d260 2023-12-19T18:28:05.794 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.794+0000 7f80097fa640 1 -- 172.21.15.27:0/528973718 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f80040047e0 con 0x7f800c15d260 2023-12-19T18:28:05.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.795+0000 7f80097fa640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fec06a430 0x7f7fec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:05.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.795+0000 7f80097fa640 1 -- 172.21.15.27:0/528973718 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8004087780 con 0x7f800c15d260 2023-12-19T18:28:05.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.796+0000 7f8010fe9640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fec06a430 0x7f7fec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:05.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.797+0000 7f8010fe9640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fec06a430 0x7f7fec06c8f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4006fd0 tx=0x7f7ff4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:05.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:05.800+0000 7f80097fa640 1 -- 172.21.15.27:0/528973718 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8004053020 con 0x7f800c15d260 2023-12-19T18:28:06.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.003+0000 7f801324d640 1 -- 172.21.15.27:0/528973718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f800c0b7970 con 0x7f800c15d260 2023-12-19T18:28:06.003 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.004+0000 7f80097fa640 1 -- 172.21.15.27:0/528973718 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v198) v1 ==== 84+0+65 (secure 0 0 0) 0x7f800c0b7970 con 0x7f800c15d260 2023-12-19T18:28:06.004 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:06.004 INFO:teuthology.orchestra.run.smithi027.stdout:[ 2023-12-19T18:28:06.004 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:06.004 INFO:teuthology.orchestra.run.smithi027.stdout: "poolnum": 1, 2023-12-19T18:28:06.004 INFO:teuthology.orchestra.run.smithi027.stdout: "poolname": ".mgr" 2023-12-19T18:28:06.004 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:06.004 INFO:teuthology.orchestra.run.smithi027.stdout:] 2023-12-19T18:28:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.006+0000 7f801324d640 1 -- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fec06a430 msgr2=0x7f7fec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.006+0000 7f801324d640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fec06a430 0x7f7fec06c8f0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4006fd0 tx=0x7f7ff4008040 comp rx=0 tx=0).stop 2023-12-19T18:28:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.006+0000 7f801324d640 1 -- 172.21.15.27:0/528973718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800c15d260 msgr2=0x7f800c0ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.006+0000 7f801324d640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800c15d260 0x7f800c0ca9c0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7f800400c780 tx=0x7f800400cc40 comp rx=0 tx=0).stop 2023-12-19T18:28:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.007+0000 7f801324d640 1 -- 172.21.15.27:0/528973718 shutdown_connections 2023-12-19T18:28:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.007+0000 7f801324d640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fec06a430 0x7f7fec06c8f0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.007+0000 7f801324d640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f800c15d260 0x7f800c0ca9c0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.007+0000 7f801324d640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f800c0d2af0 0x7f800c1f21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.007+0000 7f801324d640 1 --2- 172.21.15.27:0/528973718 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800c0d0e70 0x7f800c15fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:06.007 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.007+0000 7f801324d640 1 -- 172.21.15.27:0/528973718 >> 172.21.15.27:0/528973718 conn(0x7f800c0d2f70 msgr2=0x7f800c15b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.007+0000 7f801324d640 1 -- 172.21.15.27:0/528973718 shutdown_connections 2023-12-19T18:28:06.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.007+0000 7f801324d640 1 -- 172.21.15.27:0/528973718 wait complete. 2023-12-19T18:28:06.028 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-19T18:28:06.028 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-19T18:28:06.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.262+0000 7fe8c2457640 1 Processor -- start 2023-12-19T18:28:06.262 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.262+0000 7fe8c2457640 1 -- start start 2023-12-19T18:28:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.263+0000 7fe8c2457640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe8bc0c5040 0x7fe8bc0ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.264+0000 7fe8c2457640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc15d780 con 0x7fe8bc0cbe20 2023-12-19T18:28:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.264+0000 7fe8c2457640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc0cc200 con 0x7fe8bc0adc50 2023-12-19T18:28:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.264+0000 7fe8c2457640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc15d9d0 con 0x7fe8bc0c5040 2023-12-19T18:28:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.264+0000 7fe8c1455640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe8bc0cbe20 0x7fe8bc0c4930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44036/0 (socket says 172.21.15.27:44036) 2023-12-19T18:28:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.264+0000 7fe8c1455640 1 -- 172.21.15.27:0/2007858336 learned_addr learned my addr 172.21.15.27:0/2007858336 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.264+0000 7fe8c0c54640 1 --2- 172.21.15.27:0/2007858336 >> v2:172.21.15.192:3300/0 conn(0x7fe8bc0c5040 0x7fe8bc0ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:06.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2466864052 0 0) 0x7fe8bc15d780 con 0x7fe8bc0cbe20 2023-12-19T18:28:06.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8a8003520 con 0x7fe8bc0cbe20 2023-12-19T18:28:06.266 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1426806526 0 0) 0x7fe8a8003520 con 0x7fe8bc0cbe20 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 >> v2:172.21.15.192:3300/0 conn(0x7fe8bc0c5040 msgr2=0x7fe8bc0ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 --2- 172.21.15.27:0/2007858336 >> v2:172.21.15.192:3300/0 conn(0x7fe8bc0c5040 0x7fe8bc0ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 >> v1:172.21.15.192:6789/0 conn(0x7fe8bc0adc50 legacy=0x7fe8bc159ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc15dcb0 con 0x7fe8bc0cbe20 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe8b8001670 con 0x7fe8bc0cbe20 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 >> v1:172.21.15.27:6789/0 conn(0x7fe8bc0cbe20 legacy=0x7fe8bc0c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 --2- 172.21.15.27:0/2007858336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a8003970 0x7fe8a8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 --2- 172.21.15.27:0/2007858336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8a80047c0 0x7fe8a8004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 --2- 172.21.15.27:0/2007858336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80052a0 0x7fe8a800b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc15d9d0 con 0x7fe8a8003970 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a800c170 con 0x7fe8a80047c0 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.265+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a800c330 con 0x7fe8a80052a0 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.266+0000 7fe8c1455640 1 --2- 172.21.15.27:0/2007858336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a8003970 0x7fe8a8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.266+0000 7fe8c0c54640 1 --2- 172.21.15.27:0/2007858336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8a80047c0 0x7fe8a8004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.266+0000 7fe8c1455640 1 -- 172.21.15.27:0/2007858336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80052a0 msgr2=0x7fe8a800b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.266+0000 7fe8c1455640 1 --2- 172.21.15.27:0/2007858336 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a80052a0 0x7fe8a800b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.266+0000 7fe8c1455640 1 -- 172.21.15.27:0/2007858336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8a80047c0 msgr2=0x7fe8a8004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.266+0000 7fe8c1455640 1 --2- 172.21.15.27:0/2007858336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8a80047c0 0x7fe8a8004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.266+0000 7fe8c1455640 1 -- 172.21.15.27:0/2007858336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a8003520 con 0x7fe8a8003970 2023-12-19T18:28:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.266+0000 7fe8c0c54640 1 --2- 172.21.15.27:0/2007858336 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8a80047c0 0x7fe8a8004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.266+0000 7fe8c1c56640 1 -- 172.21.15.27:0/2007858336 reap_dead start 2023-12-19T18:28:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.267+0000 7fe8c1455640 1 --2- 172.21.15.27:0/2007858336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a8003970 0x7fe8a8003d90 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c210 tx=0x7fe8a80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f863250b08a12719 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.268+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b8005ad0 con 0x7fe8a8003970 2023-12-19T18:28:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.268+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b8005c30 con 0x7fe8a8003970 2023-12-19T18:28:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.268+0000 7fe8b3fff640 1 -- 172.21.15.27:0/2007858336 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b802cab0 con 0x7fe8a8003970 2023-12-19T18:28:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.268+0000 7fe8c2457640 1 -- 172.21.15.27:0/2007858336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a8003970 msgr2=0x7fe8a8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.268+0000 7fe8c2457640 1 --2- 172.21.15.27:0/2007858336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a8003970 0x7fe8a8003d90 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7fe8b800c210 tx=0x7fe8a80057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.268+0000 7fe8c2457640 1 -- 172.21.15.27:0/2007858336 shutdown_connections 2023-12-19T18:28:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.268+0000 7fe8c2457640 1 --2- 172.21.15.27:0/2007858336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8a8003970 0x7fe8a8003d90 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.268+0000 7fe8c2457640 1 -- 172.21.15.27:0/2007858336 >> 172.21.15.27:0/2007858336 conn(0x7fe8bc0b05b0 msgr2=0x7fe8bc0b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.268+0000 7fe8c2457640 1 -- 172.21.15.27:0/2007858336 shutdown_connections 2023-12-19T18:28:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.268+0000 7fe8c2457640 1 -- 172.21.15.27:0/2007858336 wait complete. 2023-12-19T18:28:06.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.270+0000 7fe8c2457640 1 Processor -- start 2023-12-19T18:28:06.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.270+0000 7fe8c2457640 1 -- start start 2023-12-19T18:28:06.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.271+0000 7fe8c2457640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a8003970 0x7fe8bc06c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:06.271 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.271+0000 7fe8c2457640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc14fec0 0x7fe8bc15a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:06.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.271+0000 7fe8c2457640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc14e360 0x7fe8bc14e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:06.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.271+0000 7fe8c2457640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc14f360 con 0x7fe8bc14e360 2023-12-19T18:28:06.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.271+0000 7fe8c2457640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc150340 con 0x7fe8bc14fec0 2023-12-19T18:28:06.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.271+0000 7fe8c2457640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc06c8d0 con 0x7fe8a8003970 2023-12-19T18:28:06.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.272+0000 7fe8c1c56640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc14e360 0x7fe8bc14e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:06.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.272+0000 7fe8c1c56640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc14e360 0x7fe8bc14e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45660/0 (socket says 172.21.15.27:45660) 2023-12-19T18:28:06.272 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.272+0000 7fe8c1c56640 1 -- 172.21.15.27:0/824746770 learned_addr learned my addr 172.21.15.27:0/824746770 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.272+0000 7fe8c0c54640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc14fec0 0x7fe8bc15a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.272+0000 7fe8c1455640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a8003970 0x7fe8bc06c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.272+0000 7fe8c1c56640 1 -- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a8003970 msgr2=0x7fe8bc06c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.272+0000 7fe8c1c56640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a8003970 0x7fe8bc06c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.272+0000 7fe8c1c56640 1 -- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc14fec0 msgr2=0x7fe8bc15a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.272+0000 7fe8c1c56640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc14fec0 0x7fe8bc15a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:06.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.272+0000 7fe8c1c56640 1 -- 172.21.15.27:0/824746770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc06cbb0 con 0x7fe8bc14e360 2023-12-19T18:28:06.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.272+0000 7fe8c0c54640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc14fec0 0x7fe8bc15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:06.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.272+0000 7fe8c1c56640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc14e360 0x7fe8bc14e7e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0c5550 tx=0x7fe8b400c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:06.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.273+0000 7fe8b1ffb640 1 -- 172.21.15.27:0/824746770 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4007070 con 0x7fe8bc14e360 2023-12-19T18:28:06.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.273+0000 7fe8b1ffb640 1 -- 172.21.15.27:0/824746770 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b40071d0 con 0x7fe8bc14e360 2023-12-19T18:28:06.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.273+0000 7fe8b1ffb640 1 -- 172.21.15.27:0/824746770 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b4007580 con 0x7fe8bc14e360 2023-12-19T18:28:06.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.273+0000 7fe8c2457640 1 -- 172.21.15.27:0/824746770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc06ce90 con 0x7fe8bc14e360 2023-12-19T18:28:06.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.273+0000 7fe8c2457640 1 -- 172.21.15.27:0/824746770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc06d450 con 0x7fe8bc14e360 2023-12-19T18:28:06.274 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.274+0000 7fe8c2457640 1 -- 172.21.15.27:0/824746770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8bc04e840 con 0x7fe8bc14e360 2023-12-19T18:28:06.275 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.275+0000 7fe8b1ffb640 1 -- 172.21.15.27:0/824746770 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe8b40076e0 con 0x7fe8bc14e360 2023-12-19T18:28:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.276+0000 7fe8b1ffb640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe8a006a460 0x7fe8a006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.276+0000 7fe8b1ffb640 1 -- 172.21.15.27:0/824746770 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe8b4087370 con 0x7fe8bc14e360 2023-12-19T18:28:06.276 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.277+0000 7fe8c1455640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe8a006a460 0x7fe8a006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:06.277 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.278+0000 7fe8c1455640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe8a006a460 0x7fe8a006c920 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8034000 tx=0x7fe8b8009030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:06.280 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.280+0000 7fe8b1ffb640 1 -- 172.21.15.27:0/824746770 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe8b4052cb0 con 0x7fe8bc14e360 2023-12-19T18:28:06.474 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:06.475+0000 7fe8c2457640 1 -- 172.21.15.27:0/824746770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe8bc06d670 con 0x7fe8bc14e360 2023-12-19T18:28:07.061 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.061+0000 7fe8b1ffb640 1 -- 172.21.15.27:0/824746770 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v199) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe8b40170b0 con 0x7fe8bc14e360 2023-12-19T18:28:07.062 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' created 2023-12-19T18:28:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.068+0000 7fe8c2457640 1 -- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe8a006a460 msgr2=0x7fe8a006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.068+0000 7fe8c2457640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe8a006a460 0x7fe8a006c920 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8034000 tx=0x7fe8b8009030 comp rx=0 tx=0).stop 2023-12-19T18:28:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.068+0000 7fe8c2457640 1 -- 172.21.15.27:0/824746770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc14e360 msgr2=0x7fe8bc14e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.068+0000 7fe8c2457640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc14e360 0x7fe8bc14e7e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc0c5550 tx=0x7fe8b400c980 comp rx=0 tx=0).stop 2023-12-19T18:28:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.069+0000 7fe8c2457640 1 -- 172.21.15.27:0/824746770 shutdown_connections 2023-12-19T18:28:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.069+0000 7fe8c2457640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe8a006a460 0x7fe8a006c920 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.069+0000 7fe8c2457640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8bc14e360 0x7fe8bc14e7e0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.069+0000 7fe8c2457640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe8bc14fec0 0x7fe8bc15a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.069+0000 7fe8c2457640 1 --2- 172.21.15.27:0/824746770 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe8a8003970 0x7fe8bc06c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.069+0000 7fe8c2457640 1 -- 172.21.15.27:0/824746770 >> 172.21.15.27:0/824746770 conn(0x7fe8bc0b05b0 msgr2=0x7fe8bc0c4050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.070+0000 7fe8c2457640 1 -- 172.21.15.27:0/824746770 shutdown_connections 2023-12-19T18:28:07.070 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.070+0000 7fe8c2457640 1 -- 172.21.15.27:0/824746770 wait complete. 2023-12-19T18:28:07.094 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-19T18:28:07.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.338+0000 7f3e864c0640 1 Processor -- start 2023-12-19T18:28:07.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.338+0000 7f3e864c0640 1 -- start start 2023-12-19T18:28:07.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.340+0000 7f3e864c0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8015ae60 0x7f3e80070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:07.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.340+0000 7f3e864c0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3e800cc870 0x7f3e80071010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:07.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.340+0000 7f3e864c0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e800b03f0 con 0x7f3e800ad9d0 2023-12-19T18:28:07.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.340+0000 7f3e864c0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e800b0c90 con 0x7f3e800cc870 2023-12-19T18:28:07.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.340+0000 7f3e864c0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e80071550 con 0x7f3e8015ae60 2023-12-19T18:28:07.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.340+0000 7f3e85cbf640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3e800ad9d0 0x7f3e8015e9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44042/0 (socket says 172.21.15.27:44042) 2023-12-19T18:28:07.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.340+0000 7f3e85cbf640 1 -- 172.21.15.27:0/3264440640 learned_addr learned my addr 172.21.15.27:0/3264440640 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.340+0000 7f3e854be640 1 --2- 172.21.15.27:0/3264440640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8015ae60 0x7f3e80070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:07.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e84cbd640 1 --2- 172.21.15.27:0/3264440640 >> v2:172.21.15.192:3300/0 conn(0x7f3e800cc870 0x7f3e80071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202059396 0 0) 0x7f3e800b03f0 con 0x7f3e800ad9d0 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e6c003520 con 0x7f3e800ad9d0 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4045371724 0 0) 0x7f3e6c003520 con 0x7f3e800ad9d0 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8015ae60 msgr2=0x7f3e80070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 --2- 172.21.15.27:0/3264440640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8015ae60 0x7f3e80070ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 >> v2:172.21.15.192:3300/0 conn(0x7f3e800cc870 msgr2=0x7f3e80071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 --2- 172.21.15.27:0/3264440640 >> v2:172.21.15.192:3300/0 conn(0x7f3e800cc870 0x7f3e80071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e800b4740 con 0x7f3e800ad9d0 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f3e78001670 con 0x7f3e800ad9d0 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 >> v1:172.21.15.27:6789/0 conn(0x7f3e800ad9d0 legacy=0x7f3e8015e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 --2- 172.21.15.27:0/3264440640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e6c003970 0x7f3e6c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 --2- 172.21.15.27:0/3264440640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8015ae60 0x7f3e6c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 --2- 172.21.15.27:0/3264440640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e6c004d00 0x7f3e6c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e800b0c90 con 0x7f3e6c004d00 2023-12-19T18:28:07.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e80071550 con 0x7f3e6c003970 2023-12-19T18:28:07.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.341+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e6c00ba00 con 0x7f3e8015ae60 2023-12-19T18:28:07.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.342+0000 7f3e85cbf640 1 --2- 172.21.15.27:0/3264440640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e6c004d00 0x7f3e6c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:07.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.342+0000 7f3e84cbd640 1 --2- 172.21.15.27:0/3264440640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8015ae60 0x7f3e6c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:07.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.342+0000 7f3e854be640 1 --2- 172.21.15.27:0/3264440640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e6c003970 0x7f3e6c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:07.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.342+0000 7f3e85cbf640 1 -- 172.21.15.27:0/3264440640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8015ae60 msgr2=0x7f3e6c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:07.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.342+0000 7f3e85cbf640 1 --2- 172.21.15.27:0/3264440640 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8015ae60 0x7f3e6c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:07.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.342+0000 7f3e85cbf640 1 -- 172.21.15.27:0/3264440640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e6c003970 msgr2=0x7f3e6c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.342+0000 7f3e85cbf640 1 --2- 172.21.15.27:0/3264440640 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e6c003970 0x7f3e6c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.342+0000 7f3e85cbf640 1 -- 172.21.15.27:0/3264440640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e6c003520 con 0x7f3e6c004d00 2023-12-19T18:28:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.344+0000 7f3e85cbf640 1 --2- 172.21.15.27:0/3264440640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e6c004d00 0x7f3e6c005180 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800c210 tx=0x7f3e6c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19263446d1cbccf9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.344+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7802cae0 con 0x7f3e6c004d00 2023-12-19T18:28:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.344+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7802cc40 con 0x7f3e6c004d00 2023-12-19T18:28:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.344+0000 7f3e77fff640 1 -- 172.21.15.27:0/3264440640 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e780058d0 con 0x7f3e6c004d00 2023-12-19T18:28:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.344+0000 7f3e864c0640 1 -- 172.21.15.27:0/3264440640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e6c004d00 msgr2=0x7f3e6c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.344+0000 7f3e864c0640 1 --2- 172.21.15.27:0/3264440640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e6c004d00 0x7f3e6c005180 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f3e7800c210 tx=0x7f3e6c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.344+0000 7f3e85cbf640 1 -- 172.21.15.27:0/3264440640 reap_dead start 2023-12-19T18:28:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.344+0000 7f3e864c0640 1 -- 172.21.15.27:0/3264440640 shutdown_connections 2023-12-19T18:28:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.344+0000 7f3e864c0640 1 -- 172.21.15.27:0/3264440640 >> 172.21.15.27:0/3264440640 conn(0x7f3e800cb3f0 msgr2=0x7f3e800cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.345+0000 7f3e864c0640 1 -- 172.21.15.27:0/3264440640 shutdown_connections 2023-12-19T18:28:07.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.345+0000 7f3e864c0640 1 -- 172.21.15.27:0/3264440640 wait complete. 2023-12-19T18:28:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.346+0000 7f3e864c0640 1 Processor -- start 2023-12-19T18:28:07.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.347+0000 7f3e864c0640 1 -- start start 2023-12-19T18:28:07.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e864c0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e6c004d00 0x7f3e801525c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:07.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e864c0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8014f630 0x7f3e8015d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:07.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e864c0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e8015db70 0x7f3e800b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:07.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e864c0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8014fab0 con 0x7f3e6c004d00 2023-12-19T18:28:07.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e864c0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e800c67c0 con 0x7f3e8015db70 2023-12-19T18:28:07.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e864c0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8015e0a0 con 0x7f3e8014f630 2023-12-19T18:28:07.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e854be640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e6c004d00 0x7f3e801525c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:07.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e854be640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e6c004d00 0x7f3e801525c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45684/0 (socket says 172.21.15.27:45684) 2023-12-19T18:28:07.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e854be640 1 -- 172.21.15.27:0/2758492611 learned_addr learned my addr 172.21.15.27:0/2758492611 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:07.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e85cbf640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e8015db70 0x7f3e800b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e84cbd640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8014f630 0x7f3e8015d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e854be640 1 -- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8014f630 msgr2=0x7f3e8015d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e854be640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8014f630 0x7f3e8015d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e854be640 1 -- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e8015db70 msgr2=0x7f3e800b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e854be640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e8015db70 0x7f3e800b3760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.348+0000 7f3e854be640 1 -- 172.21.15.27:0/2758492611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e800c6170 con 0x7f3e6c004d00 2023-12-19T18:28:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.349+0000 7f3e854be640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e6c004d00 0x7f3e801525c0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009fb0 tx=0x7f3e7c0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.349+0000 7f3e75ffb640 1 -- 172.21.15.27:0/2758492611 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c00e030 con 0x7f3e6c004d00 2023-12-19T18:28:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.349+0000 7f3e75ffb640 1 -- 172.21.15.27:0/2758492611 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e7c016470 con 0x7f3e6c004d00 2023-12-19T18:28:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.349+0000 7f3e75ffb640 1 -- 172.21.15.27:0/2758492611 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e7c016820 con 0x7f3e6c004d00 2023-12-19T18:28:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.349+0000 7f3e864c0640 1 -- 172.21.15.27:0/2758492611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e800c6450 con 0x7f3e6c004d00 2023-12-19T18:28:07.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.349+0000 7f3e864c0640 1 -- 172.21.15.27:0/2758492611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e8014f290 con 0x7f3e6c004d00 2023-12-19T18:28:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.350+0000 7f3e864c0640 1 -- 172.21.15.27:0/2758492611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e8004e840 con 0x7f3e6c004d00 2023-12-19T18:28:07.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.352+0000 7f3e75ffb640 1 -- 172.21.15.27:0/2758492611 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e7c016980 con 0x7f3e6c004d00 2023-12-19T18:28:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.353+0000 7f3e75ffb640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e6406a430 0x7f3e6406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.353+0000 7f3e75ffb640 1 -- 172.21.15.27:0/2758492611 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f3e7c00f070 con 0x7f3e6c004d00 2023-12-19T18:28:07.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.354+0000 7f3e84cbd640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e6406a430 0x7f3e6406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:07.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.355+0000 7f3e84cbd640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e6406a430 0x7f3e6406c8f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3e800c8120 tx=0x7f3e70008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:07.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.358+0000 7f3e75ffb640 1 -- 172.21.15.27:0/2758492611 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3e7c054ea0 con 0x7f3e6c004d00 2023-12-19T18:28:07.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:07.552+0000 7f3e864c0640 1 -- 172.21.15.27:0/2758492611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f3e80159230 con 0x7f3e6c004d00 2023-12-19T18:28:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.082+0000 7f3e75ffb640 1 -- 172.21.15.27:0/2758492611 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v200) v1 ==== 170+0+0 (secure 0 0 0) 0x7f3e7c089070 con 0x7f3e6c004d00 2023-12-19T18:28:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' created 2023-12-19T18:28:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.088+0000 7f3e864c0640 1 -- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e6406a430 msgr2=0x7f3e6406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.088+0000 7f3e864c0640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e6406a430 0x7f3e6406c8f0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f3e800c8120 tx=0x7f3e70008040 comp rx=0 tx=0).stop 2023-12-19T18:28:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.088+0000 7f3e864c0640 1 -- 172.21.15.27:0/2758492611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e6c004d00 msgr2=0x7f3e801525c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:08.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.088+0000 7f3e864c0640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e6c004d00 0x7f3e801525c0 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f3e7c009fb0 tx=0x7f3e7c0115a0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.090+0000 7f3e864c0640 1 -- 172.21.15.27:0/2758492611 shutdown_connections 2023-12-19T18:28:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.091+0000 7f3e864c0640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e6406a430 0x7f3e6406c8f0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.091+0000 7f3e864c0640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e8015db70 0x7f3e800b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.091+0000 7f3e864c0640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e8014f630 0x7f3e8015d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.091+0000 7f3e864c0640 1 --2- 172.21.15.27:0/2758492611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e6c004d00 0x7f3e801525c0 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.091+0000 7f3e864c0640 1 -- 172.21.15.27:0/2758492611 >> 172.21.15.27:0/2758492611 conn(0x7f3e800cb3f0 msgr2=0x7f3e8006f260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:08.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.092+0000 7f3e864c0640 1 -- 172.21.15.27:0/2758492611 shutdown_connections 2023-12-19T18:28:08.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.094+0000 7f3e864c0640 1 -- 172.21.15.27:0/2758492611 wait complete. 2023-12-19T18:28:08.122 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-19T18:28:08.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.325+0000 7f4031ebe640 1 Processor -- start 2023-12-19T18:28:08.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.325+0000 7f4031ebe640 1 -- start start 2023-12-19T18:28:08.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.326+0000 7f4031ebe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f402c15d120 0x7f402c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:08.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.326+0000 7f4031ebe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f402c0d1310 0x7f402c0d1750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:08.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.326+0000 7f4031ebe640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f402c0b2970 0x7f402c0b2e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:08.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.326+0000 7f4031ebe640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402c0d3850 con 0x7f402c15d120 2023-12-19T18:28:08.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.326+0000 7f4031ebe640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f402c0d0b10 con 0x7f402c0b2970 2023-12-19T18:28:08.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.326+0000 7f4031ebe640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c0d1c90 con 0x7f402c0d1310 2023-12-19T18:28:08.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.327+0000 7f4030ebc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f402c15d120 0x7f402c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:08.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.327+0000 7f4030ebc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f402c15d120 0x7f402c0cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45700/0 (socket says 172.21.15.27:45700) 2023-12-19T18:28:08.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.327+0000 7f4030ebc640 1 -- 172.21.15.27:0/3829264972 learned_addr learned my addr 172.21.15.27:0/3829264972 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:08.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.327+0000 7f402bfff640 1 --2- 172.21.15.27:0/3829264972 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f402c0d1310 0x7f402c0d1750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:08.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.327+0000 7f40316bd640 1 --2- 172.21.15.27:0/3829264972 >> v2:172.21.15.192:3300/0 conn(0x7f402c0b2970 0x7f402c0b2e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:08.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.327+0000 7f4030ebc640 1 -- 172.21.15.27:0/3829264972 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f402c0d1310 msgr2=0x7f402c0d1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:08.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.327+0000 7f4030ebc640 1 --2- 172.21.15.27:0/3829264972 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f402c0d1310 0x7f402c0d1750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.327+0000 7f4030ebc640 1 -- 172.21.15.27:0/3829264972 >> v2:172.21.15.192:3300/0 conn(0x7f402c0b2970 msgr2=0x7f402c0b2e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:08.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.327+0000 7f4030ebc640 1 --2- 172.21.15.27:0/3829264972 >> v2:172.21.15.192:3300/0 conn(0x7f402c0b2970 0x7f402c0b2e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.328+0000 7f4030ebc640 1 -- 172.21.15.27:0/3829264972 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f402c15d500 con 0x7f402c15d120 2023-12-19T18:28:08.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.329+0000 7f4030ebc640 1 --2- 172.21.15.27:0/3829264972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c15d120 0x7f402c0cfd70 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f402c08bd20 tx=0x7f401c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186e38e4f6f9fbe8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:08.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.329+0000 7f402b7fe640 1 -- 172.21.15.27:0/3829264972 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c031870 con 0x7f402c15d120 2023-12-19T18:28:08.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.329+0000 7f402b7fe640 1 -- 172.21.15.27:0/3829264972 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f401c0319d0 con 0x7f402c15d120 2023-12-19T18:28:08.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.329+0000 7f402b7fe640 1 -- 172.21.15.27:0/3829264972 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f401c031d60 con 0x7f402c15d120 2023-12-19T18:28:08.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.330+0000 7f4031ebe640 1 -- 172.21.15.27:0/3829264972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c15d120 msgr2=0x7f402c0cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:08.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.330+0000 7f4031ebe640 1 --2- 172.21.15.27:0/3829264972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c15d120 0x7f402c0cfd70 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x7f402c08bd20 tx=0x7f401c02b910 comp rx=0 tx=0).stop 2023-12-19T18:28:08.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.330+0000 7f4031ebe640 1 -- 172.21.15.27:0/3829264972 shutdown_connections 2023-12-19T18:28:08.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.330+0000 7f4031ebe640 1 --2- 172.21.15.27:0/3829264972 >> v2:172.21.15.192:3300/0 conn(0x7f402c0b2970 0x7f402c0b2e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.330+0000 7f4031ebe640 1 --2- 172.21.15.27:0/3829264972 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f402c0d1310 0x7f402c0d1750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.330+0000 7f4031ebe640 1 --2- 172.21.15.27:0/3829264972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c15d120 0x7f402c0cfd70 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.330+0000 7f4031ebe640 1 -- 172.21.15.27:0/3829264972 >> 172.21.15.27:0/3829264972 conn(0x7f402c0d2d30 msgr2=0x7f402c0d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:08.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.330+0000 7f4031ebe640 1 -- 172.21.15.27:0/3829264972 shutdown_connections 2023-12-19T18:28:08.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.330+0000 7f4031ebe640 1 -- 172.21.15.27:0/3829264972 wait complete. 2023-12-19T18:28:08.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.332+0000 7f4031ebe640 1 Processor -- start 2023-12-19T18:28:08.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.332+0000 7f4031ebe640 1 -- start start 2023-12-19T18:28:08.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.333+0000 7f4031ebe640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f402c0b2970 0x7f402c15b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:08.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.333+0000 7f4031ebe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f402c0d1310 0x7f402c0cb400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:08.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.333+0000 7f4031ebe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c15d120 0x7f402c15fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:08.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.333+0000 7f4031ebe640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c1f2420 con 0x7f402c15d120 2023-12-19T18:28:08.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.333+0000 7f4031ebe640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c1f26c0 con 0x7f402c0b2970 2023-12-19T18:28:08.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.333+0000 7f4031ebe640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402c1f29a0 con 0x7f402c0d1310 2023-12-19T18:28:08.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.333+0000 7f40316bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c15d120 0x7f402c15fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:08.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.333+0000 7f40316bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c15d120 0x7f402c15fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45702/0 (socket says 172.21.15.27:45702) 2023-12-19T18:28:08.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.333+0000 7f40316bd640 1 -- 172.21.15.27:0/4114135443 learned_addr learned my addr 172.21.15.27:0/4114135443 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:08.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.334+0000 7f4030ebc640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f402c0b2970 0x7f402c15b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:08.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.334+0000 7f402bfff640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f402c0d1310 0x7f402c0cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:08.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.334+0000 7f40316bd640 1 -- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f402c0d1310 msgr2=0x7f402c0cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:08.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.334+0000 7f40316bd640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f402c0d1310 0x7f402c0cb400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.334+0000 7f40316bd640 1 -- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f402c0b2970 msgr2=0x7f402c15b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:08.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.334+0000 7f40316bd640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f402c0b2970 0x7f402c15b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.334+0000 7f40316bd640 1 -- 172.21.15.27:0/4114135443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f401c0094d0 con 0x7f402c15d120 2023-12-19T18:28:08.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.334+0000 7f4030ebc640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f402c0b2970 0x7f402c15b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:08.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.334+0000 7f40316bd640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c15d120 0x7f402c15fc30 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f402400d5d0 tx=0x7f402400da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:08.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.334+0000 7f40297fa640 1 -- 172.21.15.27:0/4114135443 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024014620 con 0x7f402c15d120 2023-12-19T18:28:08.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.335+0000 7f40297fa640 1 -- 172.21.15.27:0/4114135443 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40240147a0 con 0x7f402c15d120 2023-12-19T18:28:08.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.335+0000 7f40297fa640 1 -- 172.21.15.27:0/4114135443 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4024014b50 con 0x7f402c15d120 2023-12-19T18:28:08.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.335+0000 7f4031ebe640 1 -- 172.21.15.27:0/4114135443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402c1f2c80 con 0x7f402c15d120 2023-12-19T18:28:08.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.335+0000 7f4031ebe640 1 -- 172.21.15.27:0/4114135443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c0a0950 con 0x7f402c15d120 2023-12-19T18:28:08.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.336+0000 7f4031ebe640 1 -- 172.21.15.27:0/4114135443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402c04e840 con 0x7f402c15d120 2023-12-19T18:28:08.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.337+0000 7f40297fa640 1 -- 172.21.15.27:0/4114135443 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4024014cb0 con 0x7f402c15d120 2023-12-19T18:28:08.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.338+0000 7f40297fa640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f400c06a430 0x7f400c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:08.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.339+0000 7f40297fa640 1 -- 172.21.15.27:0/4114135443 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f4024088c10 con 0x7f402c15d120 2023-12-19T18:28:08.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.339+0000 7f4030ebc640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f400c06a430 0x7f400c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:08.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.340+0000 7f4030ebc640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f400c06a430 0x7f400c06c8f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f401c036000 tx=0x7f401c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.344+0000 7f40297fa640 1 -- 172.21.15.27:0/4114135443 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4024054260 con 0x7f402c15d120 2023-12-19T18:28:08.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.575+0000 7f4031ebe640 1 -- 172.21.15.27:0/4114135443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f402c1f2e70 con 0x7f402c15d120 2023-12-19T18:28:08.585 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.585+0000 7f40297fa640 1 -- 172.21.15.27:0/4114135443 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v81) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4024018020 con 0x7f402c15d120 2023-12-19T18:28:08.586 INFO:teuthology.orchestra.run.smithi027.stderr:new fs with metadata pool 22 and data pool 23 2023-12-19T18:28:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.589+0000 7f4031ebe640 1 -- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f400c06a430 msgr2=0x7f400c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.589+0000 7f4031ebe640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f400c06a430 0x7f400c06c8f0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f401c036000 tx=0x7f401c03c040 comp rx=0 tx=0).stop 2023-12-19T18:28:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.589+0000 7f4031ebe640 1 -- 172.21.15.27:0/4114135443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c15d120 msgr2=0x7f402c15fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.589+0000 7f4031ebe640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c15d120 0x7f402c15fc30 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f402400d5d0 tx=0x7f402400da90 comp rx=0 tx=0).stop 2023-12-19T18:28:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.590+0000 7f4031ebe640 1 -- 172.21.15.27:0/4114135443 shutdown_connections 2023-12-19T18:28:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.590+0000 7f4031ebe640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f400c06a430 0x7f400c06c8f0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.590+0000 7f4031ebe640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c15d120 0x7f402c15fc30 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.590+0000 7f4031ebe640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f402c0d1310 0x7f402c0cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.590+0000 7f4031ebe640 1 --2- 172.21.15.27:0/4114135443 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f402c0b2970 0x7f402c15b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.590+0000 7f4031ebe640 1 -- 172.21.15.27:0/4114135443 >> 172.21.15.27:0/4114135443 conn(0x7f402c0d2d30 msgr2=0x7f402c0b3d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.593+0000 7f4031ebe640 1 -- 172.21.15.27:0/4114135443 shutdown_connections 2023-12-19T18:28:08.592 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.593+0000 7f4031ebe640 1 -- 172.21.15.27:0/4114135443 wait complete. 2023-12-19T18:28:08.619 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:28:08.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.815+0000 7f11f9657640 1 Processor -- start 2023-12-19T18:28:08.815 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.816+0000 7f11f9657640 1 -- start start 2023-12-19T18:28:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.817+0000 7f11f9657640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f11f40b4ef0 0x7f11f40b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.817+0000 7f11f9657640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11f40d2350 0x7f11f40b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.817+0000 7f11f9657640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f40b5e90 con 0x7f11f41604d0 2023-12-19T18:28:08.817 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.817+0000 7f11f9657640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f40d2b30 con 0x7f11f40b4ef0 2023-12-19T18:28:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.817+0000 7f11f9657640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f40d2850 con 0x7f11f40d2350 2023-12-19T18:28:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.818+0000 7f11f37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f11f40b4ef0 0x7f11f40b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.818+0000 7f11f37fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f11f40b4ef0 0x7f11f40b6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45718/0 (socket says 172.21.15.27:45718) 2023-12-19T18:28:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.818+0000 7f11f37fe640 1 -- 172.21.15.27:0/2038794494 learned_addr learned my addr 172.21.15.27:0/2038794494 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.818+0000 7f11f8e56640 1 --2- 172.21.15.27:0/2038794494 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11f40d2350 0x7f11f40b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:08.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.818+0000 7f11f37fe640 1 -- 172.21.15.27:0/2038794494 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11f40d2350 msgr2=0x7f11f40b6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.818+0000 7f11f37fe640 1 --2- 172.21.15.27:0/2038794494 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11f40d2350 0x7f11f40b6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.818+0000 7f11f37fe640 1 -- 172.21.15.27:0/2038794494 >> v1:172.21.15.27:6789/0 conn(0x7f11f41604d0 legacy=0x7f11f40b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.818+0000 7f11f37fe640 1 -- 172.21.15.27:0/2038794494 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f40cded0 con 0x7f11f40b4ef0 2023-12-19T18:28:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.818+0000 7f11f2ffd640 1 -- 172.21.15.27:0/2038794494 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345712377 0 0) 0x7f11f40b5e90 con 0x7f11f41604d0 2023-12-19T18:28:08.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.820+0000 7f11f37fe640 1 --2- 172.21.15.27:0/2038794494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11f40b4ef0 0x7f11f40b6300 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009870 tx=0x7f11e802b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fca140d78aceda10 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.820+0000 7f11f2ffd640 1 -- 172.21.15.27:0/2038794494 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8030840 con 0x7f11f40b4ef0 2023-12-19T18:28:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.820+0000 7f11f2ffd640 1 -- 172.21.15.27:0/2038794494 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e80309a0 con 0x7f11f40b4ef0 2023-12-19T18:28:08.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.820+0000 7f11f2ffd640 1 -- 172.21.15.27:0/2038794494 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e8030d30 con 0x7f11f40b4ef0 2023-12-19T18:28:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.821+0000 7f11f9657640 1 -- 172.21.15.27:0/2038794494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11f40b4ef0 msgr2=0x7f11f40b6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.821+0000 7f11f9657640 1 --2- 172.21.15.27:0/2038794494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11f40b4ef0 0x7f11f40b6300 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f11e8009870 tx=0x7f11e802b980 comp rx=0 tx=0).stop 2023-12-19T18:28:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.821+0000 7f11f9657640 1 -- 172.21.15.27:0/2038794494 shutdown_connections 2023-12-19T18:28:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.821+0000 7f11f9657640 1 --2- 172.21.15.27:0/2038794494 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11f40d2350 0x7f11f40b6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.821+0000 7f11f9657640 1 --2- 172.21.15.27:0/2038794494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11f40b4ef0 0x7f11f40b6300 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.821+0000 7f11f9657640 1 -- 172.21.15.27:0/2038794494 >> 172.21.15.27:0/2038794494 conn(0x7f11f40cf7e0 msgr2=0x7f11f40cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:08.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.821+0000 7f11f9657640 1 -- 172.21.15.27:0/2038794494 shutdown_connections 2023-12-19T18:28:08.822 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.821+0000 7f11f9657640 1 -- 172.21.15.27:0/2038794494 wait complete. 2023-12-19T18:28:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.823+0000 7f11f9657640 1 Processor -- start 2023-12-19T18:28:08.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.823+0000 7f11f9657640 1 -- start start 2023-12-19T18:28:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.824+0000 7f11f9657640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11f40b4ef0 0x7f11f41f23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.824+0000 7f11f9657640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11f40d2350 0x7f11f40cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.824+0000 7f11f9657640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11f41604d0 0x7f11f40b3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.824+0000 7f11f9657640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f40cbb30 con 0x7f11f40b4ef0 2023-12-19T18:28:08.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.824+0000 7f11f9657640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f40caf20 con 0x7f11f40d2350 2023-12-19T18:28:08.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.824+0000 7f11f9657640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f40cb200 con 0x7f11f41604d0 2023-12-19T18:28:08.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.824+0000 7f11f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11f40b4ef0 0x7f11f41f23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:08.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.825+0000 7f11f3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11f40b4ef0 0x7f11f41f23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45730/0 (socket says 172.21.15.27:45730) 2023-12-19T18:28:08.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.825+0000 7f11f3fff640 1 -- 172.21.15.27:0/4187282222 learned_addr learned my addr 172.21.15.27:0/4187282222 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:08.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.825+0000 7f11f3fff640 1 -- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11f41604d0 msgr2=0x7f11f40b3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:08.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.825+0000 7f11f3fff640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11f41604d0 0x7f11f40b3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.825+0000 7f11f3fff640 1 -- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11f40d2350 msgr2=0x7f11f40cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:08.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.825+0000 7f11f37fe640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11f40d2350 0x7f11f40cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:08.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.825+0000 7f11f3fff640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11f40d2350 0x7f11f40cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:08.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.825+0000 7f11f3fff640 1 -- 172.21.15.27:0/4187282222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e80094d0 con 0x7f11f40b4ef0 2023-12-19T18:28:08.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.825+0000 7f11f37fe640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11f40d2350 0x7f11f40cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:08.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.825+0000 7f11f3fff640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11f40b4ef0 0x7f11f41f23b0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f11e400c4b0 tx=0x7f11e400c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:08.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.826+0000 7f11f0ff9640 1 -- 172.21.15.27:0/4187282222 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e4005480 con 0x7f11f40b4ef0 2023-12-19T18:28:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.826+0000 7f11f0ff9640 1 -- 172.21.15.27:0/4187282222 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e40035f0 con 0x7f11f40b4ef0 2023-12-19T18:28:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.826+0000 7f11f9657640 1 -- 172.21.15.27:0/4187282222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f40cb4e0 con 0x7f11f40b4ef0 2023-12-19T18:28:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.826+0000 7f11f0ff9640 1 -- 172.21.15.27:0/4187282222 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e40039e0 con 0x7f11f40b4ef0 2023-12-19T18:28:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.826+0000 7f11f9657640 1 -- 172.21.15.27:0/4187282222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f40ccaf0 con 0x7f11f40b4ef0 2023-12-19T18:28:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.827+0000 7f11f9657640 1 -- 172.21.15.27:0/4187282222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f4066f80 con 0x7f11f40b4ef0 2023-12-19T18:28:08.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.828+0000 7f11f0ff9640 1 -- 172.21.15.27:0/4187282222 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f11e4003b40 con 0x7f11f40b4ef0 2023-12-19T18:28:08.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.829+0000 7f11f0ff9640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11e006a430 0x7f11e006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:08.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.829+0000 7f11f0ff9640 1 -- 172.21.15.27:0/4187282222 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11e4087fe0 con 0x7f11f40b4ef0 2023-12-19T18:28:08.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.829+0000 7f11f37fe640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11e006a430 0x7f11e006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.831+0000 7f11f37fe640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11e006a430 0x7f11e006c8f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f11e8036000 tx=0x7f11e803b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:08.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:08.833+0000 7f11f0ff9640 1 -- 172.21.15.27:0/4187282222 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f11e4053530 con 0x7f11f40b4ef0 2023-12-19T18:28:09.031 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.031+0000 7f11f9657640 1 -- 172.21.15.27:0/4187282222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11f404e840 con 0x7f11f40b4ef0 2023-12-19T18:28:09.034 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.034+0000 7f11f0ff9640 1 -- 172.21.15.27:0/4187282222 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+16125 (secure 0 0 0) 0x7f11e4017020 con 0x7f11f40b4ef0 2023-12-19T18:28:09.036 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:09.036 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":201,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:28:08.577248+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:28:06.542489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:28:07.623298+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:28:09.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.037+0000 7f11f9657640 1 -- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11e006a430 msgr2=0x7f11e006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:09.037 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.037+0000 7f11f9657640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11e006a430 0x7f11e006c8f0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f11e8036000 tx=0x7f11e803b040 comp rx=0 tx=0).stop 2023-12-19T18:28:09.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.037+0000 7f11f9657640 1 -- 172.21.15.27:0/4187282222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11f40b4ef0 msgr2=0x7f11f41f23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:09.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.037+0000 7f11f9657640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11f40b4ef0 0x7f11f41f23b0 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f11e400c4b0 tx=0x7f11e400c970 comp rx=0 tx=0).stop 2023-12-19T18:28:09.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.037+0000 7f11f9657640 1 -- 172.21.15.27:0/4187282222 shutdown_connections 2023-12-19T18:28:09.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.037+0000 7f11f9657640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f11e006a430 0x7f11e006c8f0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.037+0000 7f11f9657640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f11f41604d0 0x7f11f40b3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.037+0000 7f11f9657640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f11f40d2350 0x7f11f40cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.037+0000 7f11f9657640 1 --2- 172.21.15.27:0/4187282222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11f40b4ef0 0x7f11f41f23b0 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.037+0000 7f11f9657640 1 -- 172.21.15.27:0/4187282222 >> 172.21.15.27:0/4187282222 conn(0x7f11f40cf7e0 msgr2=0x7f11f40b46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:09.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.037+0000 7f11f9657640 1 -- 172.21.15.27:0/4187282222 shutdown_connections 2023-12-19T18:28:09.038 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.037+0000 7f11f9657640 1 -- 172.21.15.27:0/4187282222 wait complete. 2023-12-19T18:28:09.057 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:28:09.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.288+0000 7f09611d0640 1 Processor -- start 2023-12-19T18:28:09.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.288+0000 7f09611d0640 1 -- start start 2023-12-19T18:28:09.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.290+0000 7f09611d0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f095c15d240 0x7f095c1618d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:09.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.290+0000 7f09611d0640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f095c0ce3f0 0x7f095c0b5370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:09.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.290+0000 7f09611d0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f095c0d1970 con 0x7f095c0cdf90 2023-12-19T18:28:09.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.290+0000 7f09611d0640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f095c161e10 con 0x7f095c0ce3f0 2023-12-19T18:28:09.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.290+0000 7f09611d0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095c06a5e0 con 0x7f095c15d240 2023-12-19T18:28:09.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.290+0000 7f095ad9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f095c15d240 0x7f095c1618d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:09.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.290+0000 7f095b59e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f095c0ce3f0 0x7f095c0b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:09.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.291+0000 7f095ad9d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f095c15d240 0x7f095c1618d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:46202/0 (socket says 172.21.15.27:46202) 2023-12-19T18:28:09.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.291+0000 7f095ad9d640 1 -- 172.21.15.27:0/2644323045 learned_addr learned my addr 172.21.15.27:0/2644323045 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:09.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.291+0000 7f095ad9d640 1 -- 172.21.15.27:0/2644323045 >> v2:172.21.15.192:3300/0 conn(0x7f095c0ce3f0 msgr2=0x7f095c0b5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:09.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.291+0000 7f095ad9d640 1 --2- 172.21.15.27:0/2644323045 >> v2:172.21.15.192:3300/0 conn(0x7f095c0ce3f0 0x7f095c0b5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.291+0000 7f095ad9d640 1 -- 172.21.15.27:0/2644323045 >> v1:172.21.15.192:6789/0 conn(0x7f095c0cdf90 legacy=0x7f095c0b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:09.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.291+0000 7f095ad9d640 1 -- 172.21.15.27:0/2644323045 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c06a880 con 0x7f095c15d240 2023-12-19T18:28:09.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.293+0000 7f095ad9d640 1 --2- 172.21.15.27:0/2644323045 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f095c15d240 0x7f095c1618d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f094c009e50 tx=0x7f094c02bff0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aee5ba701cdb305 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:09.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.293+0000 7f0959d9b640 1 -- 172.21.15.27:0/2644323045 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c02cb90 con 0x7f095c15d240 2023-12-19T18:28:09.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.293+0000 7f0959d9b640 1 -- 172.21.15.27:0/2644323045 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c02ccf0 con 0x7f095c15d240 2023-12-19T18:28:09.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.293+0000 7f0959d9b640 1 -- 172.21.15.27:0/2644323045 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c034610 con 0x7f095c15d240 2023-12-19T18:28:09.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.294+0000 7f09611d0640 1 -- 172.21.15.27:0/2644323045 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f095c15d240 msgr2=0x7f095c1618d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:09.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.294+0000 7f09611d0640 1 --2- 172.21.15.27:0/2644323045 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f095c15d240 0x7f095c1618d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f094c009e50 tx=0x7f094c02bff0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.294+0000 7f09611d0640 1 -- 172.21.15.27:0/2644323045 shutdown_connections 2023-12-19T18:28:09.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.294+0000 7f09611d0640 1 --2- 172.21.15.27:0/2644323045 >> v2:172.21.15.192:3300/0 conn(0x7f095c0ce3f0 0x7f095c0b5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.294+0000 7f09611d0640 1 --2- 172.21.15.27:0/2644323045 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f095c15d240 0x7f095c1618d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.294+0000 7f09611d0640 1 -- 172.21.15.27:0/2644323045 >> 172.21.15.27:0/2644323045 conn(0x7f095c0d31e0 msgr2=0x7f095c0d35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:09.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.294+0000 7f09611d0640 1 -- 172.21.15.27:0/2644323045 shutdown_connections 2023-12-19T18:28:09.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.294+0000 7f09611d0640 1 -- 172.21.15.27:0/2644323045 wait complete. 2023-12-19T18:28:09.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.296+0000 7f09611d0640 1 Processor -- start 2023-12-19T18:28:09.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.296+0000 7f09611d0640 1 -- start start 2023-12-19T18:28:09.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.297+0000 7f09611d0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c0cdf90 0x7f095c14d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:09.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.297+0000 7f09611d0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f095c0ce3f0 0x7f095c153f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:09.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.297+0000 7f09611d0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f095c15d240 0x7f095c1544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:09.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.297+0000 7f09611d0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c1520b0 con 0x7f095c0cdf90 2023-12-19T18:28:09.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.297+0000 7f09611d0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f095c1f71f0 con 0x7f095c15d240 2023-12-19T18:28:09.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.297+0000 7f09611d0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f095c154b10 con 0x7f095c0ce3f0 2023-12-19T18:28:09.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.297+0000 7f095ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c0cdf90 0x7f095c14d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:09.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.297+0000 7f095ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c0cdf90 0x7f095c14d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45732/0 (socket says 172.21.15.27:45732) 2023-12-19T18:28:09.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.297+0000 7f095ad9d640 1 -- 172.21.15.27:0/1919347238 learned_addr learned my addr 172.21.15.27:0/1919347238 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:09.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.298+0000 7f095a59c640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f095c0ce3f0 0x7f095c153f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:09.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.298+0000 7f095b59e640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f095c15d240 0x7f095c1544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:09.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.298+0000 7f095ad9d640 1 -- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f095c0ce3f0 msgr2=0x7f095c153f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:09.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.298+0000 7f095ad9d640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f095c0ce3f0 0x7f095c153f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.298+0000 7f095ad9d640 1 -- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f095c15d240 msgr2=0x7f095c1544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:09.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.298+0000 7f095ad9d640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f095c15d240 0x7f095c1544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.298+0000 7f095ad9d640 1 -- 172.21.15.27:0/1919347238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c009b30 con 0x7f095c0cdf90 2023-12-19T18:28:09.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.298+0000 7f095ad9d640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c0cdf90 0x7f095c14d690 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f094c009e50 tx=0x7f094c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:09.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.299+0000 7f09537fe640 1 -- 172.21.15.27:0/1919347238 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c040670 con 0x7f095c0cdf90 2023-12-19T18:28:09.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.299+0000 7f09537fe640 1 -- 172.21.15.27:0/1919347238 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c040810 con 0x7f095c0cdf90 2023-12-19T18:28:09.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.299+0000 7f09611d0640 1 -- 172.21.15.27:0/1919347238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f095c154df0 con 0x7f095c0cdf90 2023-12-19T18:28:09.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.299+0000 7f09537fe640 1 -- 172.21.15.27:0/1919347238 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c040bc0 con 0x7f095c0cdf90 2023-12-19T18:28:09.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.299+0000 7f09611d0640 1 -- 172.21.15.27:0/1919347238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f095c15da80 con 0x7f095c0cdf90 2023-12-19T18:28:09.300 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.300+0000 7f09611d0640 1 -- 172.21.15.27:0/1919347238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095c155030 con 0x7f095c0cdf90 2023-12-19T18:28:09.301 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.301+0000 7f09537fe640 1 -- 172.21.15.27:0/1919347238 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f094c040d20 con 0x7f095c0cdf90 2023-12-19T18:28:09.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.302+0000 7f09537fe640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f093c06a430 0x7f093c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:09.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.302+0000 7f09537fe640 1 -- 172.21.15.27:0/1919347238 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f094c0ab7d0 con 0x7f095c0cdf90 2023-12-19T18:28:09.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.303+0000 7f095a59c640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f093c06a430 0x7f093c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:09.304 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.304+0000 7f095a59c640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f093c06a430 0x7f093c06c8f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f095c1f3f20 tx=0x7f0944009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:09.306 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.306+0000 7f09537fe640 1 -- 172.21.15.27:0/1919347238 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f095c155030 con 0x7f095c0cdf90 2023-12-19T18:28:09.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.492+0000 7f09611d0640 1 -- 172.21.15.27:0/1919347238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f095c0d1580 con 0x7f095c0cdf90 2023-12-19T18:28:09.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.495+0000 7f09537fe640 1 -- 172.21.15.27:0/1919347238 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v201) v1 ==== 74+0+16125 (secure 0 0 0) 0x7f094c076ed0 con 0x7f095c0cdf90 2023-12-19T18:28:09.495 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:09.495 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":201,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:28:08.577248+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:28:06.542489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:28:07.623298+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:28:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.497+0000 7f09611d0640 1 -- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f093c06a430 msgr2=0x7f093c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.497+0000 7f09611d0640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f093c06a430 0x7f093c06c8f0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f095c1f3f20 tx=0x7f0944009040 comp rx=0 tx=0).stop 2023-12-19T18:28:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.497+0000 7f09611d0640 1 -- 172.21.15.27:0/1919347238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c0cdf90 msgr2=0x7f095c14d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.497+0000 7f09611d0640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c0cdf90 0x7f095c14d690 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f094c009e50 tx=0x7f094c004a40 comp rx=0 tx=0).stop 2023-12-19T18:28:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.498+0000 7f09611d0640 1 -- 172.21.15.27:0/1919347238 shutdown_connections 2023-12-19T18:28:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.498+0000 7f09611d0640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f093c06a430 0x7f093c06c8f0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.498+0000 7f09611d0640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f095c15d240 0x7f095c1544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.498+0000 7f09611d0640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f095c0ce3f0 0x7f095c153f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.498+0000 7f09611d0640 1 --2- 172.21.15.27:0/1919347238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f095c0cdf90 0x7f095c14d690 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.498+0000 7f09611d0640 1 -- 172.21.15.27:0/1919347238 >> 172.21.15.27:0/1919347238 conn(0x7f095c0d31e0 msgr2=0x7f095c0c7f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:09.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.498+0000 7f09611d0640 1 -- 172.21.15.27:0/1919347238 shutdown_connections 2023-12-19T18:28:09.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.498+0000 7f09611d0640 1 -- 172.21.15.27:0/1919347238 wait complete. 2023-12-19T18:28:09.517 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-19T18:28:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.741+0000 7fa12f4dc640 1 Processor -- start 2023-12-19T18:28:09.741 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.741+0000 7fa12f4dc640 1 -- start start 2023-12-19T18:28:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.743+0000 7fa12f4dc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa128158790 0x7fa1280ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.743+0000 7fa12f4dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa12815b9e0 0x7fa1280c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.743+0000 7fa12f4dc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa12815c030 con 0x7fa1280adc50 2023-12-19T18:28:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.743+0000 7fa12f4dc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1280c99d0 con 0x7fa128158790 2023-12-19T18:28:09.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.743+0000 7fa12f4dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12815be50 con 0x7fa12815b9e0 2023-12-19T18:28:09.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.744+0000 7fa12e4da640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa128158790 0x7fa1280ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:09.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.744+0000 7fa12e4da640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fa128158790 0x7fa1280ad710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45744/0 (socket says 172.21.15.27:45744) 2023-12-19T18:28:09.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.744+0000 7fa12e4da640 1 -- 172.21.15.27:0/1034672727 learned_addr learned my addr 172.21.15.27:0/1034672727 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:09.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.744+0000 7fa12ecdb640 1 --2- 172.21.15.27:0/1034672727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa12815b9e0 0x7fa1280c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:09.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.744+0000 7fa12e4da640 1 -- 172.21.15.27:0/1034672727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa12815b9e0 msgr2=0x7fa1280c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:09.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.744+0000 7fa12e4da640 1 --2- 172.21.15.27:0/1034672727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa12815b9e0 0x7fa1280c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.744+0000 7fa12e4da640 1 -- 172.21.15.27:0/1034672727 >> v1:172.21.15.192:6789/0 conn(0x7fa1280adc50 legacy=0x7fa12815b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:09.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.745+0000 7fa12e4da640 1 -- 172.21.15.27:0/1034672727 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1280c9030 con 0x7fa128158790 2023-12-19T18:28:09.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.746+0000 7fa12e4da640 1 --2- 172.21.15.27:0/1034672727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128158790 0x7fa1280ad710 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fa1240098a0 tx=0x7fa12402b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c41d287e1c298813 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:09.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.746+0000 7fa12d4d8640 1 -- 172.21.15.27:0/1034672727 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124031640 con 0x7fa128158790 2023-12-19T18:28:09.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.746+0000 7fa12d4d8640 1 -- 172.21.15.27:0/1034672727 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1240317a0 con 0x7fa128158790 2023-12-19T18:28:09.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.746+0000 7fa12d4d8640 1 -- 172.21.15.27:0/1034672727 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124031b30 con 0x7fa128158790 2023-12-19T18:28:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.747+0000 7fa12f4dc640 1 -- 172.21.15.27:0/1034672727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128158790 msgr2=0x7fa1280ad710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.747+0000 7fa12f4dc640 1 --2- 172.21.15.27:0/1034672727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128158790 0x7fa1280ad710 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7fa1240098a0 tx=0x7fa12402b8f0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.747+0000 7fa12f4dc640 1 -- 172.21.15.27:0/1034672727 shutdown_connections 2023-12-19T18:28:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.747+0000 7fa12f4dc640 1 --2- 172.21.15.27:0/1034672727 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa12815b9e0 0x7fa1280c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.747+0000 7fa12f4dc640 1 --2- 172.21.15.27:0/1034672727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa128158790 0x7fa1280ad710 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.747+0000 7fa12f4dc640 1 -- 172.21.15.27:0/1034672727 >> 172.21.15.27:0/1034672727 conn(0x7fa1280b05b0 msgr2=0x7fa1280b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.747+0000 7fa12f4dc640 1 -- 172.21.15.27:0/1034672727 shutdown_connections 2023-12-19T18:28:09.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.747+0000 7fa12f4dc640 1 -- 172.21.15.27:0/1034672727 wait complete. 2023-12-19T18:28:09.748 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.749+0000 7fa12f4dc640 1 Processor -- start 2023-12-19T18:28:09.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.749+0000 7fa12f4dc640 1 -- start start 2023-12-19T18:28:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.750+0000 7fa12f4dc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1280adc50 0x7fa12814adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.750+0000 7fa12f4dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa128158790 0x7fa1281565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.750+0000 7fa12f4dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa12815b9e0 0x7fa128156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.750+0000 7fa12f4dc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1280ae100 con 0x7fa1280adc50 2023-12-19T18:28:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.750+0000 7fa12f4dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa12806c3f0 con 0x7fa12815b9e0 2023-12-19T18:28:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.750+0000 7fa12f4dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa12806b7e0 con 0x7fa128158790 2023-12-19T18:28:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.750+0000 7fa12e4da640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1280adc50 0x7fa12814adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.750+0000 7fa12e4da640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1280adc50 0x7fa12814adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45746/0 (socket says 172.21.15.27:45746) 2023-12-19T18:28:09.750 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.750+0000 7fa12e4da640 1 -- 172.21.15.27:0/4146284610 learned_addr learned my addr 172.21.15.27:0/4146284610 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:09.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.751+0000 7fa12e4da640 1 -- 172.21.15.27:0/4146284610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa128158790 msgr2=0x7fa1281565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:09.751 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.751+0000 7fa12e4da640 1 --2- 172.21.15.27:0/4146284610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa128158790 0x7fa1281565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.751+0000 7fa12e4da640 1 -- 172.21.15.27:0/4146284610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa12815b9e0 msgr2=0x7fa128156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.751+0000 7fa12e4da640 1 --2- 172.21.15.27:0/4146284610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa12815b9e0 0x7fa128156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.751+0000 7fa12e4da640 1 -- 172.21.15.27:0/4146284610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1240094d0 con 0x7fa1280adc50 2023-12-19T18:28:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.751+0000 7fa12e4da640 1 --2- 172.21.15.27:0/4146284610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1280adc50 0x7fa12814adb0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fa124000c00 tx=0x7fa1240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.751+0000 7fa11effd640 1 -- 172.21.15.27:0/4146284610 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa12403ac60 con 0x7fa1280adc50 2023-12-19T18:28:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.751+0000 7fa11effd640 1 -- 172.21.15.27:0/4146284610 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa12403adc0 con 0x7fa1280adc50 2023-12-19T18:28:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.751+0000 7fa11effd640 1 -- 172.21.15.27:0/4146284610 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa124044630 con 0x7fa1280adc50 2023-12-19T18:28:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.751+0000 7fa12f4dc640 1 -- 172.21.15.27:0/4146284610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa12806bac0 con 0x7fa1280adc50 2023-12-19T18:28:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.751+0000 7fa12f4dc640 1 -- 172.21.15.27:0/4146284610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa12806c080 con 0x7fa1280adc50 2023-12-19T18:28:09.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.753+0000 7fa12f4dc640 1 -- 172.21.15.27:0/4146284610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa12804e840 con 0x7fa1280adc50 2023-12-19T18:28:09.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.753+0000 7fa11effd640 1 -- 172.21.15.27:0/4146284610 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa12402f050 con 0x7fa1280adc50 2023-12-19T18:28:09.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.755+0000 7fa11effd640 1 --2- 172.21.15.27:0/4146284610 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa10c06a430 0x7fa10c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:09.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.755+0000 7fa11effd640 1 -- 172.21.15.27:0/4146284610 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1240aa960 con 0x7fa1280adc50 2023-12-19T18:28:09.755 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.756+0000 7fa12dcd9640 1 --2- 172.21.15.27:0/4146284610 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa10c06a430 0x7fa10c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:09.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.757+0000 7fa12dcd9640 1 --2- 172.21.15.27:0/4146284610 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa10c06a430 0x7fa10c06c8f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa128156570 tx=0x7fa1180079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:09.759 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.760+0000 7fa11effd640 1 -- 172.21.15.27:0/4146284610 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa124075e20 con 0x7fa1280adc50 2023-12-19T18:28:09.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:09.983+0000 7fa12f4dc640 1 -- 172.21.15.27:0/4146284610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fa1280c4e20 con 0x7fa1280adc50 2023-12-19T18:28:10.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.610+0000 7fa11effd640 1 -- 172.21.15.27:0/4146284610 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v84) v1 ==== 118+0+0 (secure 0 0 0) 0x7fa124035020 con 0x7fa1280adc50 2023-12-19T18:28:10.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.614+0000 7fa12f4dc640 1 -- 172.21.15.27:0/4146284610 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa10c06a430 msgr2=0x7fa10c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:10.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.614+0000 7fa12f4dc640 1 --2- 172.21.15.27:0/4146284610 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa10c06a430 0x7fa10c06c8f0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fa128156570 tx=0x7fa1180079b0 comp rx=0 tx=0).stop 2023-12-19T18:28:10.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.614+0000 7fa12f4dc640 1 -- 172.21.15.27:0/4146284610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1280adc50 msgr2=0x7fa12814adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:10.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.614+0000 7fa12f4dc640 1 --2- 172.21.15.27:0/4146284610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1280adc50 0x7fa12814adb0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fa124000c00 tx=0x7fa1240046c0 comp rx=0 tx=0).stop 2023-12-19T18:28:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.614+0000 7fa12f4dc640 1 -- 172.21.15.27:0/4146284610 shutdown_connections 2023-12-19T18:28:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.614+0000 7fa12f4dc640 1 --2- 172.21.15.27:0/4146284610 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa10c06a430 0x7fa10c06c8f0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.614+0000 7fa12f4dc640 1 --2- 172.21.15.27:0/4146284610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa12815b9e0 0x7fa128156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.614+0000 7fa12f4dc640 1 --2- 172.21.15.27:0/4146284610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa128158790 0x7fa1281565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.614+0000 7fa12f4dc640 1 --2- 172.21.15.27:0/4146284610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1280adc50 0x7fa12814adb0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.614+0000 7fa12f4dc640 1 -- 172.21.15.27:0/4146284610 >> 172.21.15.27:0/4146284610 conn(0x7fa1280b05b0 msgr2=0x7fa12815c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.614+0000 7fa12f4dc640 1 -- 172.21.15.27:0/4146284610 shutdown_connections 2023-12-19T18:28:10.615 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.614+0000 7fa12f4dc640 1 -- 172.21.15.27:0/4146284610 wait complete. 2023-12-19T18:28:10.638 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-19T18:28:10.639 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-19T18:28:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.878+0000 7f0568a03640 1 Processor -- start 2023-12-19T18:28:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.879+0000 7f0568a03640 1 -- start start 2023-12-19T18:28:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.880+0000 7f0568a03640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0564158790 0x7f05640ad710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.880+0000 7f0568a03640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f056415b9e0 0x7f05640c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.880+0000 7f0568a03640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f056415c030 con 0x7f05640adc50 2023-12-19T18:28:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.881+0000 7f0568a03640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05640c99d0 con 0x7f056415b9e0 2023-12-19T18:28:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.881+0000 7f0568a03640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0564158b70 con 0x7f0564158790 2023-12-19T18:28:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.881+0000 7f055bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f05640adc50 0x7f056415b2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44058/0 (socket says 172.21.15.27:44058) 2023-12-19T18:28:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.881+0000 7f055bfff640 1 -- 172.21.15.27:0/3359511544 learned_addr learned my addr 172.21.15.27:0/3359511544 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.881+0000 7f0562d9d640 1 --2- 172.21.15.27:0/3359511544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0564158790 0x7f05640ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:10.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.881+0000 7f056359e640 1 --2- 172.21.15.27:0/3359511544 >> v2:172.21.15.192:3300/0 conn(0x7f056415b9e0 0x7f05640c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.881+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269524880 0 0) 0x7f056415c030 con 0x7f05640adc50 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.881+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0550003520 con 0x7f05640adc50 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3287127333 0 0) 0x7f0550003520 con 0x7f05640adc50 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0564158790 msgr2=0x7f05640ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 --2- 172.21.15.27:0/3359511544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0564158790 0x7f05640ad710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 >> v2:172.21.15.192:3300/0 conn(0x7f056415b9e0 msgr2=0x7f05640c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 --2- 172.21.15.27:0/3359511544 >> v2:172.21.15.192:3300/0 conn(0x7f056415b9e0 0x7f05640c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05640c9030 con 0x7f05640adc50 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f054c001670 con 0x7f05640adc50 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 >> v1:172.21.15.27:6789/0 conn(0x7f05640adc50 legacy=0x7f056415b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 --2- 172.21.15.27:0/3359511544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0550003970 0x7f0550003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 --2- 172.21.15.27:0/3359511544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0550004360 0x7f0550004840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 --2- 172.21.15.27:0/3359511544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0550004f60 0x7f055000b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05640c99d0 con 0x7f0550004360 2023-12-19T18:28:10.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0564158b70 con 0x7f0550004f60 2023-12-19T18:28:10.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f055000bef0 con 0x7f0550003970 2023-12-19T18:28:10.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f0562d9d640 1 --2- 172.21.15.27:0/3359511544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0564158790 0x7f05640ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:28:10.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.882+0000 7f055bfff640 1 --2- 172.21.15.27:0/3359511544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0550004360 0x7f0550004840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:10.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.883+0000 7f0562d9d640 1 --2- 172.21.15.27:0/3359511544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0550003970 0x7f0550003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:10.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.883+0000 7f055bfff640 1 -- 172.21.15.27:0/3359511544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0550003970 msgr2=0x7f0550003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:10.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.883+0000 7f055bfff640 1 --2- 172.21.15.27:0/3359511544 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0550003970 0x7f0550003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:10.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.883+0000 7f056359e640 1 --2- 172.21.15.27:0/3359511544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0550004f60 0x7f055000b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:10.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.883+0000 7f055bfff640 1 -- 172.21.15.27:0/3359511544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0550004f60 msgr2=0x7f055000b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:10.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.883+0000 7f055bfff640 1 --2- 172.21.15.27:0/3359511544 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0550004f60 0x7f055000b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:10.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.883+0000 7f055bfff640 1 -- 172.21.15.27:0/3359511544 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0550003520 con 0x7f0550004360 2023-12-19T18:28:10.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.884+0000 7f055bfff640 1 --2- 172.21.15.27:0/3359511544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0550004360 0x7f0550004840 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f054c00c210 tx=0x7f05500057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db9e654989bed4 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:10.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.884+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c02cae0 con 0x7f0550004360 2023-12-19T18:28:10.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.884+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c02cc40 con 0x7f0550004360 2023-12-19T18:28:10.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.884+0000 7f056259c640 1 -- 172.21.15.27:0/3359511544 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c005af0 con 0x7f0550004360 2023-12-19T18:28:10.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.885+0000 7f0568a03640 1 -- 172.21.15.27:0/3359511544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0550004360 msgr2=0x7f0550004840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:10.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.885+0000 7f0568a03640 1 --2- 172.21.15.27:0/3359511544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0550004360 0x7f0550004840 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f054c00c210 tx=0x7f05500057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:10.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.885+0000 7f056359e640 1 -- 172.21.15.27:0/3359511544 reap_dead start 2023-12-19T18:28:10.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.885+0000 7f0568a03640 1 -- 172.21.15.27:0/3359511544 shutdown_connections 2023-12-19T18:28:10.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.885+0000 7f0568a03640 1 -- 172.21.15.27:0/3359511544 >> 172.21.15.27:0/3359511544 conn(0x7f05640b05b0 msgr2=0x7f05640b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:10.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.885+0000 7f0568a03640 1 -- 172.21.15.27:0/3359511544 shutdown_connections 2023-12-19T18:28:10.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.885+0000 7f0568a03640 1 -- 172.21.15.27:0/3359511544 wait complete. 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.887+0000 7f0568a03640 1 Processor -- start 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.887+0000 7f0568a03640 1 -- start start 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.888+0000 7f0568a03640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0550004360 0x7f056406c240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.888+0000 7f0568a03640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0564158f10 0x7f05640c7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.888+0000 7f0568a03640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0564159390 0x7f05640c7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.888+0000 7f0568a03640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05640af250 con 0x7f0564158f10 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.888+0000 7f0568a03640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05640aea40 con 0x7f0550004360 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.888+0000 7f0568a03640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05640c6ae0 con 0x7f0564159390 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.888+0000 7f055bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0564158f10 0x7f05640c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.888+0000 7f055bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0564158f10 0x7f05640c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:45768/0 (socket says 172.21.15.27:45768) 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.888+0000 7f055bfff640 1 -- 172.21.15.27:0/666568198 learned_addr learned my addr 172.21.15.27:0/666568198 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.889+0000 7f056359e640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0564159390 0x7f05640c7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.889+0000 7f0562d9d640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0550004360 0x7f056406c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.889+0000 7f055bfff640 1 -- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0564159390 msgr2=0x7f05640c7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:10.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.889+0000 7f055bfff640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0564159390 0x7f05640c7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:10.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.889+0000 7f055bfff640 1 -- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0550004360 msgr2=0x7f056406c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:10.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.889+0000 7f055bfff640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0550004360 0x7f056406c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:10.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.889+0000 7f055bfff640 1 -- 172.21.15.27:0/666568198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f056406c780 con 0x7f0564158f10 2023-12-19T18:28:10.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.889+0000 7f0562d9d640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0550004360 0x7f056406c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:10.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.889+0000 7f055bfff640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0564158f10 0x7f05640c7580 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f054c02c480 tx=0x7f056415b140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:10.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.889+0000 7f055b7fe640 1 -- 172.21.15.27:0/666568198 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c039070 con 0x7f0564158f10 2023-12-19T18:28:10.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.889+0000 7f055b7fe640 1 -- 172.21.15.27:0/666568198 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f054c004bf0 con 0x7f0564158f10 2023-12-19T18:28:10.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.889+0000 7f0568a03640 1 -- 172.21.15.27:0/666568198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05640aec90 con 0x7f0564158f10 2023-12-19T18:28:10.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.889+0000 7f055b7fe640 1 -- 172.21.15.27:0/666568198 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f054c03c040 con 0x7f0564158f10 2023-12-19T18:28:10.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.890+0000 7f0568a03640 1 -- 172.21.15.27:0/666568198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05640c6d80 con 0x7f0564158f10 2023-12-19T18:28:10.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.891+0000 7f0568a03640 1 -- 172.21.15.27:0/666568198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f056404e840 con 0x7f0564158f10 2023-12-19T18:28:10.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.891+0000 7f055b7fe640 1 -- 172.21.15.27:0/666568198 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f054c032070 con 0x7f0564158f10 2023-12-19T18:28:10.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.893+0000 7f055b7fe640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f054406a430 0x7f054406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:10.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.893+0000 7f055b7fe640 1 -- 172.21.15.27:0/666568198 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f054c0a9cc0 con 0x7f0564158f10 2023-12-19T18:28:10.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.894+0000 7f0562d9d640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f054406a430 0x7f054406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:10.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.895+0000 7f0562d9d640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f054406a430 0x7f054406c8f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f056406f8d0 tx=0x7f0554008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:10.897+0000 7f055b7fe640 1 -- 172.21.15.27:0/666568198 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f054c075390 con 0x7f0564158f10 2023-12-19T18:28:11.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:11.132+0000 7f0568a03640 1 -- 172.21.15.27:0/666568198 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f056406d410 con 0x7f0564158f10 2023-12-19T18:28:12.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.095+0000 7f055b7fe640 1 -- 172.21.15.27:0/666568198 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v85) v1 ==== 122+0+0 (secure 0 0 0) 0x7f054c0789b0 con 0x7f0564158f10 2023-12-19T18:28:12.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.099+0000 7f0568a03640 1 -- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f054406a430 msgr2=0x7f054406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:12.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.099+0000 7f0568a03640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f054406a430 0x7f054406c8f0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f056406f8d0 tx=0x7f0554008040 comp rx=0 tx=0).stop 2023-12-19T18:28:12.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.099+0000 7f0568a03640 1 -- 172.21.15.27:0/666568198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0564158f10 msgr2=0x7f05640c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:12.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.099+0000 7f0568a03640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0564158f10 0x7f05640c7580 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f054c02c480 tx=0x7f056415b140 comp rx=0 tx=0).stop 2023-12-19T18:28:12.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.099+0000 7f0568a03640 1 -- 172.21.15.27:0/666568198 shutdown_connections 2023-12-19T18:28:12.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.099+0000 7f0568a03640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f054406a430 0x7f054406c8f0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.099+0000 7f0568a03640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0564159390 0x7f05640c7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.099+0000 7f0568a03640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0564158f10 0x7f05640c7580 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.099+0000 7f0568a03640 1 --2- 172.21.15.27:0/666568198 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0550004360 0x7f056406c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.099+0000 7f0568a03640 1 -- 172.21.15.27:0/666568198 >> 172.21.15.27:0/666568198 conn(0x7f05640b05b0 msgr2=0x7f05640cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:12.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.099+0000 7f0568a03640 1 -- 172.21.15.27:0/666568198 shutdown_connections 2023-12-19T18:28:12.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.099+0000 7f0568a03640 1 -- 172.21.15.27:0/666568198 wait complete. 2023-12-19T18:28:12.122 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-19T18:28:12.123 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:28:12.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.354+0000 7fe451be7640 1 Processor -- start 2023-12-19T18:28:12.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.354+0000 7fe451be7640 1 -- start start 2023-12-19T18:28:12.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe451be7640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe44c0b8c30 0x7fe44c0d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:12.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe451be7640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe44c0cf9b0 0x7fe44c0cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:12.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe451be7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44c0cda20 0x7fe44c0cde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:12.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe451be7640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe44c0cea80 con 0x7fe44c0cf9b0 2023-12-19T18:28:12.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe451be7640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe44c0b5f10 con 0x7fe44c0b8c30 2023-12-19T18:28:12.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe451be7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c0d0380 con 0x7fe44c0cda20 2023-12-19T18:28:12.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe44affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe44c0cf9b0 0x7fe44c0cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:12.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe44affd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe44c0cf9b0 0x7fe44c0cfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42228/0 (socket says 172.21.15.27:42228) 2023-12-19T18:28:12.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe44affd640 1 -- 172.21.15.27:0/1192986974 learned_addr learned my addr 172.21.15.27:0/1192986974 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:12.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe44bfff640 1 --2- 172.21.15.27:0/1192986974 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44c0cda20 0x7fe44c0cde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:12.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe44b7fe640 1 --2- 172.21.15.27:0/1192986974 >> v2:172.21.15.192:3300/0 conn(0x7fe44c0b8c30 0x7fe44c0d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:12.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe44affd640 1 -- 172.21.15.27:0/1192986974 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44c0cda20 msgr2=0x7fe44c0cde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:12.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe44affd640 1 --2- 172.21.15.27:0/1192986974 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44c0cda20 0x7fe44c0cde50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe44affd640 1 -- 172.21.15.27:0/1192986974 >> v2:172.21.15.192:3300/0 conn(0x7fe44c0b8c30 msgr2=0x7fe44c0d1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:12.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe44affd640 1 --2- 172.21.15.27:0/1192986974 >> v2:172.21.15.192:3300/0 conn(0x7fe44c0b8c30 0x7fe44c0d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.356+0000 7fe44affd640 1 -- 172.21.15.27:0/1192986974 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c0ce7a0 con 0x7fe44c0cf9b0 2023-12-19T18:28:12.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.358+0000 7fe44affd640 1 --2- 172.21.15.27:0/1192986974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44c0cf9b0 0x7fe44c0cfe40 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe44c073c20 tx=0x7fe43402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=715f2301f46be11d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:12.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.358+0000 7fe44a7fc640 1 -- 172.21.15.27:0/1192986974 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4340317c0 con 0x7fe44c0cf9b0 2023-12-19T18:28:12.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.358+0000 7fe44a7fc640 1 -- 172.21.15.27:0/1192986974 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe434031920 con 0x7fe44c0cf9b0 2023-12-19T18:28:12.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.358+0000 7fe44a7fc640 1 -- 172.21.15.27:0/1192986974 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe434031cb0 con 0x7fe44c0cf9b0 2023-12-19T18:28:12.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.359+0000 7fe451be7640 1 -- 172.21.15.27:0/1192986974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44c0cf9b0 msgr2=0x7fe44c0cfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:12.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.359+0000 7fe451be7640 1 --2- 172.21.15.27:0/1192986974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44c0cf9b0 0x7fe44c0cfe40 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7fe44c073c20 tx=0x7fe43402b860 comp rx=0 tx=0).stop 2023-12-19T18:28:12.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.359+0000 7fe451be7640 1 -- 172.21.15.27:0/1192986974 shutdown_connections 2023-12-19T18:28:12.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.359+0000 7fe451be7640 1 --2- 172.21.15.27:0/1192986974 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44c0cda20 0x7fe44c0cde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.359+0000 7fe451be7640 1 --2- 172.21.15.27:0/1192986974 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44c0cf9b0 0x7fe44c0cfe40 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.359+0000 7fe451be7640 1 --2- 172.21.15.27:0/1192986974 >> v2:172.21.15.192:3300/0 conn(0x7fe44c0b8c30 0x7fe44c0d1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.359+0000 7fe451be7640 1 -- 172.21.15.27:0/1192986974 >> 172.21.15.27:0/1192986974 conn(0x7fe44c0d2fa0 msgr2=0x7fe44c0d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:12.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.359+0000 7fe451be7640 1 -- 172.21.15.27:0/1192986974 shutdown_connections 2023-12-19T18:28:12.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.359+0000 7fe451be7640 1 -- 172.21.15.27:0/1192986974 wait complete. 2023-12-19T18:28:12.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.361+0000 7fe451be7640 1 Processor -- start 2023-12-19T18:28:12.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.361+0000 7fe451be7640 1 -- start start 2023-12-19T18:28:12.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.362+0000 7fe451be7640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44c0b8c30 0x7fe44c1f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:12.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.362+0000 7fe451be7640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe44c0cda20 0x7fe44c15ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:12.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.362+0000 7fe451be7640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44c0cf9b0 0x7fe44c15b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:12.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.362+0000 7fe451be7640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c0b4710 con 0x7fe44c0b8c30 2023-12-19T18:28:12.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.362+0000 7fe451be7640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c151f50 con 0x7fe44c0cda20 2023-12-19T18:28:12.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.362+0000 7fe451be7640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe44c1f6140 con 0x7fe44c0cf9b0 2023-12-19T18:28:12.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.362+0000 7fe44b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44c0b8c30 0x7fe44c1f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:12.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.362+0000 7fe44b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44c0b8c30 0x7fe44c1f5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42230/0 (socket says 172.21.15.27:42230) 2023-12-19T18:28:12.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.362+0000 7fe44b7fe640 1 -- 172.21.15.27:0/1735986356 learned_addr learned my addr 172.21.15.27:0/1735986356 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:12.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.363+0000 7fe44affd640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe44c0cda20 0x7fe44c15ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:12.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.363+0000 7fe44bfff640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44c0cf9b0 0x7fe44c15b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:12.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.363+0000 7fe44b7fe640 1 -- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44c0cf9b0 msgr2=0x7fe44c15b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:12.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.363+0000 7fe44b7fe640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44c0cf9b0 0x7fe44c15b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.363+0000 7fe44b7fe640 1 -- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe44c0cda20 msgr2=0x7fe44c15ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:12.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.363+0000 7fe44b7fe640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe44c0cda20 0x7fe44c15ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.363+0000 7fe44b7fe640 1 -- 172.21.15.27:0/1735986356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4340094d0 con 0x7fe44c0b8c30 2023-12-19T18:28:12.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.363+0000 7fe44affd640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe44c0cda20 0x7fe44c15ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:12.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.363+0000 7fe44b7fe640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44c0b8c30 0x7fe44c1f5c00 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00ce70 tx=0x7fe43c00a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:12.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.363+0000 7fe443fff640 1 -- 172.21.15.27:0/1735986356 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c003b90 con 0x7fe44c0b8c30 2023-12-19T18:28:12.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.363+0000 7fe443fff640 1 -- 172.21.15.27:0/1735986356 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe43c003cf0 con 0x7fe44c0b8c30 2023-12-19T18:28:12.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.363+0000 7fe443fff640 1 -- 172.21.15.27:0/1735986356 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe43c004260 con 0x7fe44c0b8c30 2023-12-19T18:28:12.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.363+0000 7fe451be7640 1 -- 172.21.15.27:0/1735986356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe44c0b3ad0 con 0x7fe44c0b8c30 2023-12-19T18:28:12.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.364+0000 7fe451be7640 1 -- 172.21.15.27:0/1735986356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe44c0b4050 con 0x7fe44c0b8c30 2023-12-19T18:28:12.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.365+0000 7fe451be7640 1 -- 172.21.15.27:0/1735986356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe44c067110 con 0x7fe44c0b8c30 2023-12-19T18:28:12.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.366+0000 7fe443fff640 1 -- 172.21.15.27:0/1735986356 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe43c0043c0 con 0x7fe44c0b8c30 2023-12-19T18:28:12.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.367+0000 7fe443fff640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe42c06a430 0x7fe42c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:12.366 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.367+0000 7fe443fff640 1 -- 172.21.15.27:0/1735986356 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe43c012070 con 0x7fe44c0b8c30 2023-12-19T18:28:12.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.367+0000 7fe44affd640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe42c06a430 0x7fe42c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:12.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.368+0000 7fe44affd640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe42c06a430 0x7fe42c06c8f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe434036000 tx=0x7fe43403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:12.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.371+0000 7fe443fff640 1 -- 172.21.15.27:0/1735986356 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe43c0548a0 con 0x7fe44c0b8c30 2023-12-19T18:28:12.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.605+0000 7fe451be7640 1 -- 172.21.15.27:0/1735986356 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe44c0b1e50 con 0x7fe44c0b8c30 2023-12-19T18:28:12.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.606+0000 7fe443fff640 1 -- 172.21.15.27:0/1735986356 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4713 (secure 0 0 0) 0x7fe43c003e60 con 0x7fe44c0b8c30 2023-12-19T18:28:12.606 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:12.606 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1864159643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1864159643},{"type":"v1","addr":"172.21.15.192:6836","nonce":1864159643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1348219767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1348219767},{"type":"v1","addr":"172.21.15.192:6837","nonce":1348219767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/181137639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":181137639},{"type":"v1","addr":"172.21.15.27:6833","nonce":181137639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:28:08.577742+0000","modified":"2023-12-19T18:28:12.083302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/152568921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":152568921},{"type":"v1","addr":"172.21.15.27:6835","nonce":152568921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:28:12.606 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 85 2023-12-19T18:28:12.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.608+0000 7fe451be7640 1 -- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe42c06a430 msgr2=0x7fe42c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:12.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.608+0000 7fe451be7640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe42c06a430 0x7fe42c06c8f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fe434036000 tx=0x7fe43403d040 comp rx=0 tx=0).stop 2023-12-19T18:28:12.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.608+0000 7fe451be7640 1 -- 172.21.15.27:0/1735986356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44c0b8c30 msgr2=0x7fe44c1f5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:12.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.608+0000 7fe451be7640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44c0b8c30 0x7fe44c1f5c00 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7fe43c00ce70 tx=0x7fe43c00a660 comp rx=0 tx=0).stop 2023-12-19T18:28:12.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.608+0000 7fe451be7640 1 -- 172.21.15.27:0/1735986356 shutdown_connections 2023-12-19T18:28:12.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.608+0000 7fe451be7640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe42c06a430 0x7fe42c06c8f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.608+0000 7fe451be7640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe44c0cf9b0 0x7fe44c15b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.608+0000 7fe451be7640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe44c0cda20 0x7fe44c15ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.608+0000 7fe451be7640 1 --2- 172.21.15.27:0/1735986356 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe44c0b8c30 0x7fe44c1f5c00 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.608+0000 7fe451be7640 1 -- 172.21.15.27:0/1735986356 >> 172.21.15.27:0/1735986356 conn(0x7fe44c0d2fa0 msgr2=0x7fe44c0d1ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:12.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.609+0000 7fe451be7640 1 -- 172.21.15.27:0/1735986356 shutdown_connections 2023-12-19T18:28:12.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.609+0000 7fe451be7640 1 -- 172.21.15.27:0/1735986356 wait complete. 2023-12-19T18:28:12.627 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:28:12.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.858+0000 7f7db4a5e640 1 Processor -- start 2023-12-19T18:28:12.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.858+0000 7f7db4a5e640 1 -- start start 2023-12-19T18:28:12.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.860+0000 7f7db4a5e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f7db00cab50 0x7f7db015c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:12.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.860+0000 7f7db4a5e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7db00cb370 0x7f7db00cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:12.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.860+0000 7f7db4a5e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db00ca310 con 0x7f7db00c3820 2023-12-19T18:28:12.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.860+0000 7f7db4a5e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db00c3cb0 con 0x7f7db00cb370 2023-12-19T18:28:12.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.860+0000 7f7db4a5e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db00ae490 con 0x7f7db00cab50 2023-12-19T18:28:12.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.860+0000 7f7daf59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7db00cb370 0x7f7db00cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:12.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.860+0000 7f7daf59e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7db00cb370 0x7f7db00cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42232/0 (socket says 172.21.15.27:42232) 2023-12-19T18:28:12.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.860+0000 7f7daf59e640 1 -- 172.21.15.27:0/2864857497 learned_addr learned my addr 172.21.15.27:0/2864857497 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:12.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.861+0000 7f7daed9d640 1 --2- 172.21.15.27:0/2864857497 >> v2:172.21.15.192:3300/0 conn(0x7f7db00cab50 0x7f7db015c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:12.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.861+0000 7f7daf59e640 1 -- 172.21.15.27:0/2864857497 >> v2:172.21.15.192:3300/0 conn(0x7f7db00cab50 msgr2=0x7f7db015c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:12.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.861+0000 7f7daf59e640 1 --2- 172.21.15.27:0/2864857497 >> v2:172.21.15.192:3300/0 conn(0x7f7db00cab50 0x7f7db015c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.861+0000 7f7daf59e640 1 -- 172.21.15.27:0/2864857497 >> v1:172.21.15.27:6789/0 conn(0x7f7db00c3820 legacy=0x7f7db00c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:12.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.861+0000 7f7daf59e640 1 -- 172.21.15.27:0/2864857497 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db00ae770 con 0x7f7db00cb370 2023-12-19T18:28:12.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.861+0000 7f7daed9d640 1 --2- 172.21.15.27:0/2864857497 >> v2:172.21.15.192:3300/0 conn(0x7f7db00cab50 0x7f7db015c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:12.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.862+0000 7f7daf59e640 1 --2- 172.21.15.27:0/2864857497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db00cb370 0x7f7db00cc510 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f7db006d540 tx=0x7f7da802b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e91e4218de95a77c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:12.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.862+0000 7f7dae59c640 1 -- 172.21.15.27:0/2864857497 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da80317e0 con 0x7f7db00cb370 2023-12-19T18:28:12.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.862+0000 7f7dae59c640 1 -- 172.21.15.27:0/2864857497 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da8031940 con 0x7f7db00cb370 2023-12-19T18:28:12.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.862+0000 7f7dae59c640 1 -- 172.21.15.27:0/2864857497 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da8031cd0 con 0x7f7db00cb370 2023-12-19T18:28:12.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.863+0000 7f7db4a5e640 1 -- 172.21.15.27:0/2864857497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db00cb370 msgr2=0x7f7db00cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:12.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.863+0000 7f7db4a5e640 1 --2- 172.21.15.27:0/2864857497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db00cb370 0x7f7db00cc510 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f7db006d540 tx=0x7f7da802b880 comp rx=0 tx=0).stop 2023-12-19T18:28:12.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.863+0000 7f7db4a5e640 1 -- 172.21.15.27:0/2864857497 shutdown_connections 2023-12-19T18:28:12.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.863+0000 7f7db4a5e640 1 --2- 172.21.15.27:0/2864857497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db00cb370 0x7f7db00cc510 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.863+0000 7f7db4a5e640 1 --2- 172.21.15.27:0/2864857497 >> v2:172.21.15.192:3300/0 conn(0x7f7db00cab50 0x7f7db015c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.863+0000 7f7db4a5e640 1 -- 172.21.15.27:0/2864857497 >> 172.21.15.27:0/2864857497 conn(0x7f7db0069c70 msgr2=0x7f7db006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:12.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.863+0000 7f7db4a5e640 1 -- 172.21.15.27:0/2864857497 shutdown_connections 2023-12-19T18:28:12.864 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.863+0000 7f7db4a5e640 1 -- 172.21.15.27:0/2864857497 wait complete. 2023-12-19T18:28:12.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.865+0000 7f7db4a5e640 1 Processor -- start 2023-12-19T18:28:12.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.865+0000 7f7db4a5e640 1 -- start start 2023-12-19T18:28:12.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.866+0000 7f7db4a5e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db00c3820 0x7f7db0147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:12.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.866+0000 7f7db4a5e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7db00cab50 0x7f7db015b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:12.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.866+0000 7f7db4a5e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7db00cb370 0x7f7db015b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:12.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.866+0000 7f7db4a5e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0152220 con 0x7f7db00c3820 2023-12-19T18:28:12.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.866+0000 7f7db4a5e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0147960 con 0x7f7db00cb370 2023-12-19T18:28:12.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.866+0000 7f7db4a5e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db0147c40 con 0x7f7db00cab50 2023-12-19T18:28:12.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.866+0000 7f7daed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db00c3820 0x7f7db0147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:12.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.866+0000 7f7daed9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db00c3820 0x7f7db0147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42238/0 (socket says 172.21.15.27:42238) 2023-12-19T18:28:12.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.866+0000 7f7daed9d640 1 -- 172.21.15.27:0/1614656227 learned_addr learned my addr 172.21.15.27:0/1614656227 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:12.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.867+0000 7f7daf59e640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7db00cb370 0x7f7db015b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:12.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.867+0000 7f7da7fff640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7db00cab50 0x7f7db015b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.867+0000 7f7daed9d640 1 -- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7db00cab50 msgr2=0x7f7db015b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.867+0000 7f7daed9d640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7db00cab50 0x7f7db015b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.867+0000 7f7daed9d640 1 -- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7db00cb370 msgr2=0x7f7db015b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.867+0000 7f7daed9d640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7db00cb370 0x7f7db015b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.867+0000 7f7daed9d640 1 -- 172.21.15.27:0/1614656227 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da80094d0 con 0x7f7db00c3820 2023-12-19T18:28:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.867+0000 7f7daed9d640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db00c3820 0x7f7db0147030 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f7da000cf60 tx=0x7f7da000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.868+0000 7f7da77fe640 1 -- 172.21.15.27:0/1614656227 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da000d840 con 0x7f7db00c3820 2023-12-19T18:28:12.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.868+0000 7f7da77fe640 1 -- 172.21.15.27:0/1614656227 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da000d9a0 con 0x7f7db00c3820 2023-12-19T18:28:12.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.868+0000 7f7db4a5e640 1 -- 172.21.15.27:0/1614656227 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db0147f20 con 0x7f7db00c3820 2023-12-19T18:28:12.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.868+0000 7f7da77fe640 1 -- 172.21.15.27:0/1614656227 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da000dd30 con 0x7f7db00c3820 2023-12-19T18:28:12.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.868+0000 7f7db4a5e640 1 -- 172.21.15.27:0/1614656227 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db0147570 con 0x7f7db00c3820 2023-12-19T18:28:12.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.869+0000 7f7db4a5e640 1 -- 172.21.15.27:0/1614656227 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db006a990 con 0x7f7db00c3820 2023-12-19T18:28:12.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.870+0000 7f7da77fe640 1 -- 172.21.15.27:0/1614656227 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7da0015440 con 0x7f7db00c3820 2023-12-19T18:28:12.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.871+0000 7f7da77fe640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d9006a430 0x7f7d9006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:12.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.871+0000 7f7da77fe640 1 -- 172.21.15.27:0/1614656227 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7da00892d0 con 0x7f7db00c3820 2023-12-19T18:28:12.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.871+0000 7f7da7fff640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d9006a430 0x7f7d9006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:12.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.872+0000 7f7da7fff640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d9006a430 0x7f7d9006c8f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7db00b4a40 tx=0x7f7d98009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:12.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:12.875+0000 7f7da77fe640 1 -- 172.21.15.27:0/1614656227 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7da0054970 con 0x7f7db00c3820 2023-12-19T18:28:13.021 INFO:tasks.ceph.mds.c.smithi027.stderr:2023-12-19T18:28:13.021+0000 7f96068f0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:28:13.065 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.065+0000 7f7db4a5e640 1 -- 172.21.15.27:0/1614656227 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7db0148110 con 0x7f7db00c3820 2023-12-19T18:28:13.068 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.068+0000 7f7da77fe640 1 -- 172.21.15.27:0/1614656227 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v202) v1 ==== 74+0+16112 (secure 0 0 0) 0x7f7da0057f90 con 0x7f7db00c3820 2023-12-19T18:28:13.068 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:13.069 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":202,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:28:09.580840+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:28:06.542489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:28:07.623298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"202","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:28:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.071+0000 7f7db4a5e640 1 -- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d9006a430 msgr2=0x7f7d9006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.071+0000 7f7db4a5e640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d9006a430 0x7f7d9006c8f0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7db00b4a40 tx=0x7f7d98009040 comp rx=0 tx=0).stop 2023-12-19T18:28:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.071+0000 7f7db4a5e640 1 -- 172.21.15.27:0/1614656227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db00c3820 msgr2=0x7f7db0147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.071+0000 7f7db4a5e640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db00c3820 0x7f7db0147030 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f7da000cf60 tx=0x7f7da000cf90 comp rx=0 tx=0).stop 2023-12-19T18:28:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.071+0000 7f7db4a5e640 1 -- 172.21.15.27:0/1614656227 shutdown_connections 2023-12-19T18:28:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.071+0000 7f7db4a5e640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7d9006a430 0x7f7d9006c8f0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.071+0000 7f7db4a5e640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7db00cb370 0x7f7db015b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.071+0000 7f7db4a5e640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7db00cab50 0x7f7db015b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.071+0000 7f7db4a5e640 1 --2- 172.21.15.27:0/1614656227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db00c3820 0x7f7db0147030 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.071+0000 7f7db4a5e640 1 -- 172.21.15.27:0/1614656227 >> 172.21.15.27:0/1614656227 conn(0x7f7db0069c70 msgr2=0x7f7db00c44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.072+0000 7f7db4a5e640 1 -- 172.21.15.27:0/1614656227 shutdown_connections 2023-12-19T18:28:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.072+0000 7f7db4a5e640 1 -- 172.21.15.27:0/1614656227 wait complete. 2023-12-19T18:28:13.089 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-19T18:28:13.089 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:28:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.318+0000 7fd7a3fff640 1 Processor -- start 2023-12-19T18:28:13.319 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.319+0000 7fd7a3fff640 1 -- start start 2023-12-19T18:28:13.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.320+0000 7fd7a3fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd7a41553a0 0x7fd7a40c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.320+0000 7fd7a3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7a40afeb0 0x7fd7a415e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.320+0000 7fd7a3fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7a40b0330 con 0x7fd7a415ec60 2023-12-19T18:28:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.320+0000 7fd7a3fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7a40cbcf0 con 0x7fd7a41553a0 2023-12-19T18:28:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.320+0000 7fd7a3fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a415f0d0 con 0x7fd7a40afeb0 2023-12-19T18:28:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.321+0000 7fd7a2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd7a41553a0 0x7fd7a40c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.321+0000 7fd7a2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fd7a41553a0 0x7fd7a40c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42246/0 (socket says 172.21.15.27:42246) 2023-12-19T18:28:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.321+0000 7fd7a2ffd640 1 -- 172.21.15.27:0/580804096 learned_addr learned my addr 172.21.15.27:0/580804096 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.321+0000 7fd7a2ffd640 1 -- 172.21.15.27:0/580804096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7a40afeb0 msgr2=0x7fd7a415e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.321+0000 7fd79bfff640 1 --2- 172.21.15.27:0/580804096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7a40afeb0 0x7fd7a415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.321+0000 7fd7a2ffd640 1 --2- 172.21.15.27:0/580804096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7a40afeb0 0x7fd7a415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.321+0000 7fd7a2ffd640 1 -- 172.21.15.27:0/580804096 >> v1:172.21.15.192:6789/0 conn(0x7fd7a415ec60 legacy=0x7fd7a415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.321+0000 7fd7a2ffd640 1 -- 172.21.15.27:0/580804096 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7a40cb350 con 0x7fd7a41553a0 2023-12-19T18:28:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.322+0000 7fd7a2ffd640 1 --2- 172.21.15.27:0/580804096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a41553a0 0x7fd7a40c50a0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fd7940098a0 tx=0x7fd79402b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1152ef7872afce0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:13.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.323+0000 7fd7a27fc640 1 -- 172.21.15.27:0/580804096 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7940317c0 con 0x7fd7a41553a0 2023-12-19T18:28:13.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.323+0000 7fd7a27fc640 1 -- 172.21.15.27:0/580804096 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd794031920 con 0x7fd7a41553a0 2023-12-19T18:28:13.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.323+0000 7fd7a27fc640 1 -- 172.21.15.27:0/580804096 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd794031cb0 con 0x7fd7a41553a0 2023-12-19T18:28:13.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.323+0000 7fd7a3fff640 1 -- 172.21.15.27:0/580804096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a41553a0 msgr2=0x7fd7a40c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:13.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.323+0000 7fd7a3fff640 1 --2- 172.21.15.27:0/580804096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a41553a0 0x7fd7a40c50a0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fd7940098a0 tx=0x7fd79402b860 comp rx=0 tx=0).stop 2023-12-19T18:28:13.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.324+0000 7fd7a3fff640 1 -- 172.21.15.27:0/580804096 shutdown_connections 2023-12-19T18:28:13.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.324+0000 7fd7a3fff640 1 --2- 172.21.15.27:0/580804096 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7a40afeb0 0x7fd7a415e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.324+0000 7fd7a3fff640 1 --2- 172.21.15.27:0/580804096 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a41553a0 0x7fd7a40c50a0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.324+0000 7fd7a3fff640 1 -- 172.21.15.27:0/580804096 >> 172.21.15.27:0/580804096 conn(0x7fd7a4069c70 msgr2=0x7fd7a406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:13.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.324+0000 7fd7a3fff640 1 -- 172.21.15.27:0/580804096 shutdown_connections 2023-12-19T18:28:13.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.324+0000 7fd7a3fff640 1 -- 172.21.15.27:0/580804096 wait complete. 2023-12-19T18:28:13.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.325+0000 7fd7a3fff640 1 Processor -- start 2023-12-19T18:28:13.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.326+0000 7fd7a3fff640 1 -- start start 2023-12-19T18:28:13.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.326+0000 7fd7a3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a40afeb0 0x7fd7a415ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:13.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.326+0000 7fd7a3fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7a41553a0 0x7fd7a41520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.326+0000 7fd7a3fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7a415ec60 0x7fd7a4152620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.326+0000 7fd7a3fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40cde80 con 0x7fd7a40afeb0 2023-12-19T18:28:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.326+0000 7fd7a3fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40cd200 con 0x7fd7a41553a0 2023-12-19T18:28:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.326+0000 7fd7a3fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a40cd4e0 con 0x7fd7a415ec60 2023-12-19T18:28:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.327+0000 7fd7a2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a40afeb0 0x7fd7a415ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.327+0000 7fd7a2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a40afeb0 0x7fd7a415ca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42254/0 (socket says 172.21.15.27:42254) 2023-12-19T18:28:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.327+0000 7fd7a2ffd640 1 -- 172.21.15.27:0/1147232563 learned_addr learned my addr 172.21.15.27:0/1147232563 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.327+0000 7fd79bfff640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7a41553a0 0x7fd7a41520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.327+0000 7fd7a37fe640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7a415ec60 0x7fd7a4152620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.327+0000 7fd7a2ffd640 1 -- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7a415ec60 msgr2=0x7fd7a4152620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.327+0000 7fd7a2ffd640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7a415ec60 0x7fd7a4152620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.327+0000 7fd7a2ffd640 1 -- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7a41553a0 msgr2=0x7fd7a41520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.327+0000 7fd7a2ffd640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7a41553a0 0x7fd7a41520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.327+0000 7fd7a2ffd640 1 -- 172.21.15.27:0/1147232563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7940094d0 con 0x7fd7a40afeb0 2023-12-19T18:28:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.327+0000 7fd7a37fe640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7a415ec60 0x7fd7a4152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.328+0000 7fd7a2ffd640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a40afeb0 0x7fd7a415ca70 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fd7940098a0 tx=0x7fd7940046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.328+0000 7fd79b7fe640 1 -- 172.21.15.27:0/1147232563 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7940329a0 con 0x7fd7a40afeb0 2023-12-19T18:28:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.328+0000 7fd79b7fe640 1 -- 172.21.15.27:0/1147232563 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd794032b20 con 0x7fd7a40afeb0 2023-12-19T18:28:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.328+0000 7fd7a3fff640 1 -- 172.21.15.27:0/1147232563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a40cd7c0 con 0x7fd7a40afeb0 2023-12-19T18:28:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.328+0000 7fd79b7fe640 1 -- 172.21.15.27:0/1147232563 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7940453f0 con 0x7fd7a40afeb0 2023-12-19T18:28:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.328+0000 7fd7a3fff640 1 -- 172.21.15.27:0/1147232563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a415cfb0 con 0x7fd7a40afeb0 2023-12-19T18:28:13.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.329+0000 7fd7a3fff640 1 -- 172.21.15.27:0/1147232563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a40cb610 con 0x7fd7a40afeb0 2023-12-19T18:28:13.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.330+0000 7fd79b7fe640 1 -- 172.21.15.27:0/1147232563 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd79404d3f0 con 0x7fd7a40afeb0 2023-12-19T18:28:13.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.331+0000 7fd79b7fe640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd78406a430 0x7fd78406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:13.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.331+0000 7fd79b7fe640 1 -- 172.21.15.27:0/1147232563 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7940aa920 con 0x7fd7a40afeb0 2023-12-19T18:28:13.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.332+0000 7fd79bfff640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd78406a430 0x7fd78406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:13.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.333+0000 7fd79bfff640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd78406a430 0x7fd78406c8f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd7a41f5cf0 tx=0x7fd78c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:13.335 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.335+0000 7fd79b7fe640 1 -- 172.21.15.27:0/1147232563 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd7a40cb610 con 0x7fd7a40afeb0 2023-12-19T18:28:13.524 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.524+0000 7fd7a3fff640 1 -- 172.21.15.27:0/1147232563 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7a40b1880 con 0x7fd78406a430 2023-12-19T18:28:13.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.548+0000 7fd79b7fe640 1 -- 172.21.15.27:0/1147232563 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282169 (secure 0 0 0) 0x7fd7a40b1880 con 0x7fd78406a430 2023-12-19T18:28:13.552 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:13.554 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:28:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.554+0000 7fd7a3fff640 1 -- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd78406a430 msgr2=0x7fd78406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.554+0000 7fd7a3fff640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd78406a430 0x7fd78406c8f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fd7a41f5cf0 tx=0x7fd78c008040 comp rx=0 tx=0).stop 2023-12-19T18:28:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.554+0000 7fd7a3fff640 1 -- 172.21.15.27:0/1147232563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a40afeb0 msgr2=0x7fd7a415ca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.554+0000 7fd7a3fff640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a40afeb0 0x7fd7a415ca70 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fd7940098a0 tx=0x7fd7940046c0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.555+0000 7fd7a3fff640 1 -- 172.21.15.27:0/1147232563 shutdown_connections 2023-12-19T18:28:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.555+0000 7fd7a3fff640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd78406a430 0x7fd78406c8f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.555+0000 7fd7a3fff640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd7a415ec60 0x7fd7a4152620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.555+0000 7fd7a3fff640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd7a41553a0 0x7fd7a41520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.555+0000 7fd7a3fff640 1 --2- 172.21.15.27:0/1147232563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a40afeb0 0x7fd7a415ca70 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.555+0000 7fd7a3fff640 1 -- 172.21.15.27:0/1147232563 >> 172.21.15.27:0/1147232563 conn(0x7fd7a4069c70 msgr2=0x7fd7a4158d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.555+0000 7fd7a3fff640 1 -- 172.21.15.27:0/1147232563 shutdown_connections 2023-12-19T18:28:13.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.555+0000 7fd7a3fff640 1 -- 172.21.15.27:0/1147232563 wait complete. 2023-12-19T18:28:13.577 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":598,"stamp":"2023-12-19T18:28:11.999557+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":34,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":9981,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552518,"statfs":{"total":1600000000,"available":1589779121,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.936086"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.597391+0000","last_change":"2023-12-19T18:28:08.597014+0000","last_active":"2023-12-19T18:28:08.597391+0000","last_peered":"2023-12-19T18:28:08.597391+0000","last_clean":"2023-12-19T18:28:08.597391+0000","last_became_active":"2023-12-19T18:28:08.596776+0000","last_became_peered":"2023-12-19T18:28:08.596776+0000","last_unstale":"2023-12-19T18:28:08.597391+0000","last_undegraded":"2023-12-19T18:28:08.597391+0000","last_fullsized":"2023-12-19T18:28:08.597391+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:49:02.216601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.594984+0000","last_change":"2023-12-19T18:28:08.594909+0000","last_active":"2023-12-19T18:28:08.594984+0000","last_peered":"2023-12-19T18:28:08.594984+0000","last_clean":"2023-12-19T18:28:08.594984+0000","last_became_active":"2023-12-19T18:28:08.594690+0000","last_became_peered":"2023-12-19T18:28:08.594690+0000","last_unstale":"2023-12-19T18:28:08.594984+0000","last_undegraded":"2023-12-19T18:28:08.594984+0000","last_fullsized":"2023-12-19T18:28:08.594984+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:49:35.513007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.605331+0000","last_change":"2023-12-19T18:28:08.604103+0000","last_active":"2023-12-19T18:28:08.605331+0000","last_peered":"2023-12-19T18:28:08.605331+0000","last_clean":"2023-12-19T18:28:08.605331+0000","last_became_active":"2023-12-19T18:28:08.603423+0000","last_became_peered":"2023-12-19T18:28:08.603423+0000","last_unstale":"2023-12-19T18:28:08.605331+0000","last_undegraded":"2023-12-19T18:28:08.605331+0000","last_fullsized":"2023-12-19T18:28:08.605331+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:09.966919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.614419+0000","last_change":"2023-12-19T18:28:08.612975+0000","last_active":"2023-12-19T18:28:08.614419+0000","last_peered":"2023-12-19T18:28:08.614419+0000","last_clean":"2023-12-19T18:28:08.614419+0000","last_became_active":"2023-12-19T18:28:08.611480+0000","last_became_peered":"2023-12-19T18:28:08.611480+0000","last_unstale":"2023-12-19T18:28:08.614419+0000","last_undegraded":"2023-12-19T18:28:08.614419+0000","last_fullsized":"2023-12-19T18:28:08.614419+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:02.299966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.595225+0000","last_change":"2023-12-19T18:28:08.591948+0000","last_active":"2023-12-19T18:28:08.595225+0000","last_peered":"2023-12-19T18:28:08.595225+0000","last_clean":"2023-12-19T18:28:08.595225+0000","last_became_active":"2023-12-19T18:28:08.591724+0000","last_became_peered":"2023-12-19T18:28:08.591724+0000","last_unstale":"2023-12-19T18:28:08.595225+0000","last_undegraded":"2023-12-19T18:28:08.595225+0000","last_fullsized":"2023-12-19T18:28:08.595225+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:18:47.517627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.618834+0000","last_change":"2023-12-19T18:28:08.618752+0000","last_active":"2023-12-19T18:28:08.618834+0000","last_peered":"2023-12-19T18:28:08.618834+0000","last_clean":"2023-12-19T18:28:08.618834+0000","last_became_active":"2023-12-19T18:28:08.618519+0000","last_became_peered":"2023-12-19T18:28:08.618519+0000","last_unstale":"2023-12-19T18:28:08.618834+0000","last_undegraded":"2023-12-19T18:28:08.618834+0000","last_fullsized":"2023-12-19T18:28:08.618834+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:16:51.265478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.610794+0000","last_change":"2023-12-19T18:28:08.609823+0000","last_active":"2023-12-19T18:28:08.610794+0000","last_peered":"2023-12-19T18:28:08.610794+0000","last_clean":"2023-12-19T18:28:08.610794+0000","last_became_active":"2023-12-19T18:28:08.608182+0000","last_became_peered":"2023-12-19T18:28:08.608182+0000","last_unstale":"2023-12-19T18:28:08.610794+0000","last_undegraded":"2023-12-19T18:28:08.610794+0000","last_fullsized":"2023-12-19T18:28:08.610794+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:18:16.064659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.594861+0000","last_change":"2023-12-19T18:28:08.594783+0000","last_active":"2023-12-19T18:28:08.594861+0000","last_peered":"2023-12-19T18:28:08.594861+0000","last_clean":"2023-12-19T18:28:08.594861+0000","last_became_active":"2023-12-19T18:28:08.594433+0000","last_became_peered":"2023-12-19T18:28:08.594433+0000","last_unstale":"2023-12-19T18:28:08.594861+0000","last_undegraded":"2023-12-19T18:28:08.594861+0000","last_fullsized":"2023-12-19T18:28:08.594861+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:40:04.871763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.606300+0000","last_change":"2023-12-19T18:28:08.606083+0000","last_active":"2023-12-19T18:28:08.606300+0000","last_peered":"2023-12-19T18:28:08.606300+0000","last_clean":"2023-12-19T18:28:08.606300+0000","last_became_active":"2023-12-19T18:28:08.605529+0000","last_became_peered":"2023-12-19T18:28:08.605529+0000","last_unstale":"2023-12-19T18:28:08.606300+0000","last_undegraded":"2023-12-19T18:28:08.606300+0000","last_fullsized":"2023-12-19T18:28:08.606300+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:26:58.564690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.612704+0000","last_change":"2023-12-19T18:28:08.612552+0000","last_active":"2023-12-19T18:28:08.612704+0000","last_peered":"2023-12-19T18:28:08.612704+0000","last_clean":"2023-12-19T18:28:08.612704+0000","last_became_active":"2023-12-19T18:28:08.612313+0000","last_became_peered":"2023-12-19T18:28:08.612313+0000","last_unstale":"2023-12-19T18:28:08.612704+0000","last_undegraded":"2023-12-19T18:28:08.612704+0000","last_fullsized":"2023-12-19T18:28:08.612704+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:34:00.405702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.606634+0000","last_change":"2023-12-19T18:28:08.606559+0000","last_active":"2023-12-19T18:28:08.606634+0000","last_peered":"2023-12-19T18:28:08.606634+0000","last_clean":"2023-12-19T18:28:08.606634+0000","last_became_active":"2023-12-19T18:28:08.603968+0000","last_became_peered":"2023-12-19T18:28:08.603968+0000","last_unstale":"2023-12-19T18:28:08.606634+0000","last_undegraded":"2023-12-19T18:28:08.606634+0000","last_fullsized":"2023-12-19T18:28:08.606634+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:21.745636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.611593+0000","last_change":"2023-12-19T18:28:08.611523+0000","last_active":"2023-12-19T18:28:08.611593+0000","last_peered":"2023-12-19T18:28:08.611593+0000","last_clean":"2023-12-19T18:28:08.611593+0000","last_became_active":"2023-12-19T18:28:08.611299+0000","last_became_peered":"2023-12-19T18:28:08.611299+0000","last_unstale":"2023-12-19T18:28:08.611593+0000","last_undegraded":"2023-12-19T18:28:08.611593+0000","last_fullsized":"2023-12-19T18:28:08.611593+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:21:08.357423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.602816+0000","last_change":"2023-12-19T18:28:08.602608+0000","last_active":"2023-12-19T18:28:08.602816+0000","last_peered":"2023-12-19T18:28:08.602816+0000","last_clean":"2023-12-19T18:28:08.602816+0000","last_became_active":"2023-12-19T18:28:08.601919+0000","last_became_peered":"2023-12-19T18:28:08.601919+0000","last_unstale":"2023-12-19T18:28:08.602816+0000","last_undegraded":"2023-12-19T18:28:08.602816+0000","last_fullsized":"2023-12-19T18:28:08.602816+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:02:10.193717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.596823+0000","last_change":"2023-12-19T18:28:08.596733+0000","last_active":"2023-12-19T18:28:08.596823+0000","last_peered":"2023-12-19T18:28:08.596823+0000","last_clean":"2023-12-19T18:28:08.596823+0000","last_became_active":"2023-12-19T18:28:08.596386+0000","last_became_peered":"2023-12-19T18:28:08.596386+0000","last_unstale":"2023-12-19T18:28:08.596823+0000","last_undegraded":"2023-12-19T18:28:08.596823+0000","last_fullsized":"2023-12-19T18:28:08.596823+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:16:51.265478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.605657+0000","last_change":"2023-12-19T18:28:08.604577+0000","last_active":"2023-12-19T18:28:08.605657+0000","last_peered":"2023-12-19T18:28:08.605657+0000","last_clean":"2023-12-19T18:28:08.605657+0000","last_became_active":"2023-12-19T18:28:08.604207+0000","last_became_peered":"2023-12-19T18:28:08.604207+0000","last_unstale":"2023-12-19T18:28:08.605657+0000","last_undegraded":"2023-12-19T18:28:08.605657+0000","last_fullsized":"2023-12-19T18:28:08.605657+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:46.937478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.608878+0000","last_change":"2023-12-19T18:28:08.608679+0000","last_active":"2023-12-19T18:28:08.608878+0000","last_peered":"2023-12-19T18:28:08.608878+0000","last_clean":"2023-12-19T18:28:08.608878+0000","last_became_active":"2023-12-19T18:28:08.608086+0000","last_became_peered":"2023-12-19T18:28:08.608086+0000","last_unstale":"2023-12-19T18:28:08.608878+0000","last_undegraded":"2023-12-19T18:28:08.608878+0000","last_fullsized":"2023-12-19T18:28:08.608878+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:15:16.292212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.616103+0000","last_change":"2023-12-19T18:28:08.615747+0000","last_active":"2023-12-19T18:28:08.616103+0000","last_peered":"2023-12-19T18:28:08.616103+0000","last_clean":"2023-12-19T18:28:08.616103+0000","last_became_active":"2023-12-19T18:28:08.614903+0000","last_became_peered":"2023-12-19T18:28:08.614903+0000","last_unstale":"2023-12-19T18:28:08.616103+0000","last_undegraded":"2023-12-19T18:28:08.616103+0000","last_fullsized":"2023-12-19T18:28:08.616103+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:07.478205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.614357+0000","last_change":"2023-12-19T18:28:08.611701+0000","last_active":"2023-12-19T18:28:08.614357+0000","last_peered":"2023-12-19T18:28:08.614357+0000","last_clean":"2023-12-19T18:28:08.614357+0000","last_became_active":"2023-12-19T18:28:08.610115+0000","last_became_peered":"2023-12-19T18:28:08.610115+0000","last_unstale":"2023-12-19T18:28:08.614357+0000","last_undegraded":"2023-12-19T18:28:08.614357+0000","last_fullsized":"2023-12-19T18:28:08.614357+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:40:41.811992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.608091+0000","last_change":"2023-12-19T18:28:08.602277+0000","last_active":"2023-12-19T18:28:08.608091+0000","last_peered":"2023-12-19T18:28:08.608091+0000","last_clean":"2023-12-19T18:28:08.608091+0000","last_became_active":"2023-12-19T18:28:08.601898+0000","last_became_peered":"2023-12-19T18:28:08.601898+0000","last_unstale":"2023-12-19T18:28:08.608091+0000","last_undegraded":"2023-12-19T18:28:08.608091+0000","last_fullsized":"2023-12-19T18:28:08.608091+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:40:04.871763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.592436+0000","last_change":"2023-12-19T18:28:08.591488+0000","last_active":"2023-12-19T18:28:08.592436+0000","last_peered":"2023-12-19T18:28:08.592436+0000","last_clean":"2023-12-19T18:28:08.592436+0000","last_became_active":"2023-12-19T18:28:08.591233+0000","last_became_peered":"2023-12-19T18:28:08.591233+0000","last_unstale":"2023-12-19T18:28:08.592436+0000","last_undegraded":"2023-12-19T18:28:08.592436+0000","last_fullsized":"2023-12-19T18:28:08.592436+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:29.241920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.617024+0000","last_change":"2023-12-19T18:28:08.616849+0000","last_active":"2023-12-19T18:28:08.617024+0000","last_peered":"2023-12-19T18:28:08.617024+0000","last_clean":"2023-12-19T18:28:08.617024+0000","last_became_active":"2023-12-19T18:28:08.615946+0000","last_became_peered":"2023-12-19T18:28:08.615946+0000","last_unstale":"2023-12-19T18:28:08.617024+0000","last_undegraded":"2023-12-19T18:28:08.617024+0000","last_fullsized":"2023-12-19T18:28:08.617024+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:16.483526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.596213+0000","last_change":"2023-12-19T18:28:08.596127+0000","last_active":"2023-12-19T18:28:08.596213+0000","last_peered":"2023-12-19T18:28:08.596213+0000","last_clean":"2023-12-19T18:28:08.596213+0000","last_became_active":"2023-12-19T18:28:08.595904+0000","last_became_peered":"2023-12-19T18:28:08.595904+0000","last_unstale":"2023-12-19T18:28:08.596213+0000","last_undegraded":"2023-12-19T18:28:08.596213+0000","last_fullsized":"2023-12-19T18:28:08.596213+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:37.185535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.615689+0000","last_change":"2023-12-19T18:28:08.612477+0000","last_active":"2023-12-19T18:28:08.615689+0000","last_peered":"2023-12-19T18:28:08.615689+0000","last_clean":"2023-12-19T18:28:08.615689+0000","last_became_active":"2023-12-19T18:28:08.610162+0000","last_became_peered":"2023-12-19T18:28:08.610162+0000","last_unstale":"2023-12-19T18:28:08.615689+0000","last_undegraded":"2023-12-19T18:28:08.615689+0000","last_fullsized":"2023-12-19T18:28:08.615689+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:46.937478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.598723+0000","last_change":"2023-12-19T18:28:08.596231+0000","last_active":"2023-12-19T18:28:08.598723+0000","last_peered":"2023-12-19T18:28:08.598723+0000","last_clean":"2023-12-19T18:28:08.598723+0000","last_became_active":"2023-12-19T18:28:08.595895+0000","last_became_peered":"2023-12-19T18:28:08.595895+0000","last_unstale":"2023-12-19T18:28:08.598723+0000","last_undegraded":"2023-12-19T18:28:08.598723+0000","last_fullsized":"2023-12-19T18:28:08.598723+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:34:12.524415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.600729+0000","last_change":"2023-12-19T18:28:08.600509+0000","last_active":"2023-12-19T18:28:08.600729+0000","last_peered":"2023-12-19T18:28:08.600729+0000","last_clean":"2023-12-19T18:28:08.600729+0000","last_became_active":"2023-12-19T18:28:08.599822+0000","last_became_peered":"2023-12-19T18:28:08.599822+0000","last_unstale":"2023-12-19T18:28:08.600729+0000","last_undegraded":"2023-12-19T18:28:08.600729+0000","last_fullsized":"2023-12-19T18:28:08.600729+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:01:58.890674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.618631+0000","last_change":"2023-12-19T18:28:08.618546+0000","last_active":"2023-12-19T18:28:08.618631+0000","last_peered":"2023-12-19T18:28:08.618631+0000","last_clean":"2023-12-19T18:28:08.618631+0000","last_became_active":"2023-12-19T18:28:08.618322+0000","last_became_peered":"2023-12-19T18:28:08.618322+0000","last_unstale":"2023-12-19T18:28:08.618631+0000","last_undegraded":"2023-12-19T18:28:08.618631+0000","last_fullsized":"2023-12-19T18:28:08.618631+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:25:46.176399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.610595+0000","last_change":"2023-12-19T18:28:08.610520+0000","last_active":"2023-12-19T18:28:08.610595+0000","last_peered":"2023-12-19T18:28:08.610595+0000","last_clean":"2023-12-19T18:28:08.610595+0000","last_became_active":"2023-12-19T18:28:08.609944+0000","last_became_peered":"2023-12-19T18:28:08.609944+0000","last_unstale":"2023-12-19T18:28:08.610595+0000","last_undegraded":"2023-12-19T18:28:08.610595+0000","last_fullsized":"2023-12-19T18:28:08.610595+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:16:20.661937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.608192+0000","last_change":"2023-12-19T18:28:08.605904+0000","last_active":"2023-12-19T18:28:08.608192+0000","last_peered":"2023-12-19T18:28:08.608192+0000","last_clean":"2023-12-19T18:28:08.608192+0000","last_became_active":"2023-12-19T18:28:08.605067+0000","last_became_peered":"2023-12-19T18:28:08.605067+0000","last_unstale":"2023-12-19T18:28:08.608192+0000","last_undegraded":"2023-12-19T18:28:08.608192+0000","last_fullsized":"2023-12-19T18:28:08.608192+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:46.937478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.602559+0000","last_change":"2023-12-19T18:28:08.602267+0000","last_active":"2023-12-19T18:28:08.602559+0000","last_peered":"2023-12-19T18:28:08.602559+0000","last_clean":"2023-12-19T18:28:08.602559+0000","last_became_active":"2023-12-19T18:28:08.601923+0000","last_became_peered":"2023-12-19T18:28:08.601923+0000","last_unstale":"2023-12-19T18:28:08.602559+0000","last_undegraded":"2023-12-19T18:28:08.602559+0000","last_fullsized":"2023-12-19T18:28:08.602559+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:05:15.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.596602+0000","last_change":"2023-12-19T18:28:08.596532+0000","last_active":"2023-12-19T18:28:08.596602+0000","last_peered":"2023-12-19T18:28:08.596602+0000","last_clean":"2023-12-19T18:28:08.596602+0000","last_became_active":"2023-12-19T18:28:08.596171+0000","last_became_peered":"2023-12-19T18:28:08.596171+0000","last_unstale":"2023-12-19T18:28:08.596602+0000","last_undegraded":"2023-12-19T18:28:08.596602+0000","last_fullsized":"2023-12-19T18:28:08.596602+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:07.478205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.606782+0000","last_change":"2023-12-19T18:28:08.606485+0000","last_active":"2023-12-19T18:28:08.606782+0000","last_peered":"2023-12-19T18:28:08.606782+0000","last_clean":"2023-12-19T18:28:08.606782+0000","last_became_active":"2023-12-19T18:28:08.606229+0000","last_became_peered":"2023-12-19T18:28:08.606229+0000","last_unstale":"2023-12-19T18:28:08.606782+0000","last_undegraded":"2023-12-19T18:28:08.606782+0000","last_fullsized":"2023-12-19T18:28:08.606782+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:52:40.164956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.619634+0000","last_change":"2023-12-19T18:28:08.619373+0000","last_active":"2023-12-19T18:28:08.619634+0000","last_peered":"2023-12-19T18:28:08.619634+0000","last_clean":"2023-12-19T18:28:08.619634+0000","last_became_active":"2023-12-19T18:28:08.616356+0000","last_became_peered":"2023-12-19T18:28:08.616356+0000","last_unstale":"2023-12-19T18:28:08.619634+0000","last_undegraded":"2023-12-19T18:28:08.619634+0000","last_fullsized":"2023-12-19T18:28:08.619634+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:46.937478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.614402+0000","last_change":"2023-12-19T18:28:08.611866+0000","last_active":"2023-12-19T18:28:08.614402+0000","last_peered":"2023-12-19T18:28:08.614402+0000","last_clean":"2023-12-19T18:28:08.614402+0000","last_became_active":"2023-12-19T18:28:08.611103+0000","last_became_peered":"2023-12-19T18:28:08.611103+0000","last_unstale":"2023-12-19T18:28:08.614402+0000","last_undegraded":"2023-12-19T18:28:08.614402+0000","last_fullsized":"2023-12-19T18:28:08.614402+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:09.952598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"201'4","reported_seq":14,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.616501+0000","last_change":"2023-12-19T18:28:08.097575+0000","last_active":"2023-12-19T18:28:08.616501+0000","last_peered":"2023-12-19T18:28:08.616501+0000","last_clean":"2023-12-19T18:28:08.616501+0000","last_became_active":"2023-12-19T18:28:08.093961+0000","last_became_peered":"2023-12-19T18:28:08.093961+0000","last_unstale":"2023-12-19T18:28:08.616501+0000","last_undegraded":"2023-12-19T18:28:08.616501+0000","last_fullsized":"2023-12-19T18:28:08.616501+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:18:15.050649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.617335+0000","last_change":"2023-12-19T18:28:08.617266+0000","last_active":"2023-12-19T18:28:08.617335+0000","last_peered":"2023-12-19T18:28:08.617335+0000","last_clean":"2023-12-19T18:28:08.617335+0000","last_became_active":"2023-12-19T18:28:08.617049+0000","last_became_peered":"2023-12-19T18:28:08.617049+0000","last_unstale":"2023-12-19T18:28:08.617335+0000","last_undegraded":"2023-12-19T18:28:08.617335+0000","last_fullsized":"2023-12-19T18:28:08.617335+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:09.952598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"201'1","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.601190+0000","last_change":"2023-12-19T18:28:08.106602+0000","last_active":"2023-12-19T18:28:08.601190+0000","last_peered":"2023-12-19T18:28:08.601190+0000","last_clean":"2023-12-19T18:28:08.601190+0000","last_became_active":"2023-12-19T18:28:08.106367+0000","last_became_peered":"2023-12-19T18:28:08.106367+0000","last_unstale":"2023-12-19T18:28:08.601190+0000","last_undegraded":"2023-12-19T18:28:08.601190+0000","last_fullsized":"2023-12-19T18:28:08.601190+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:59.338756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.617378+0000","last_change":"2023-12-19T18:28:08.617011+0000","last_active":"2023-12-19T18:28:08.617378+0000","last_peered":"2023-12-19T18:28:08.617378+0000","last_clean":"2023-12-19T18:28:08.617378+0000","last_became_active":"2023-12-19T18:28:08.616479+0000","last_became_peered":"2023-12-19T18:28:08.616479+0000","last_unstale":"2023-12-19T18:28:08.617378+0000","last_undegraded":"2023-12-19T18:28:08.617378+0000","last_fullsized":"2023-12-19T18:28:08.617378+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:26:58.564690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"201'2","reported_seq":12,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.618778+0000","last_change":"2023-12-19T18:28:08.107314+0000","last_active":"2023-12-19T18:28:08.618778+0000","last_peered":"2023-12-19T18:28:08.618778+0000","last_clean":"2023-12-19T18:28:08.618778+0000","last_became_active":"2023-12-19T18:28:08.107070+0000","last_became_peered":"2023-12-19T18:28:08.107070+0000","last_unstale":"2023-12-19T18:28:08.618778+0000","last_undegraded":"2023-12-19T18:28:08.618778+0000","last_fullsized":"2023-12-19T18:28:08.618778+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:14:19.016691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.605639+0000","last_change":"2023-12-19T18:28:08.604444+0000","last_active":"2023-12-19T18:28:08.605639+0000","last_peered":"2023-12-19T18:28:08.605639+0000","last_clean":"2023-12-19T18:28:08.605639+0000","last_became_active":"2023-12-19T18:28:08.603881+0000","last_became_peered":"2023-12-19T18:28:08.603881+0000","last_unstale":"2023-12-19T18:28:08.605639+0000","last_undegraded":"2023-12-19T18:28:08.605639+0000","last_fullsized":"2023-12-19T18:28:08.605639+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:49.397031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"201'2","reported_seq":12,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.616111+0000","last_change":"2023-12-19T18:28:08.100518+0000","last_active":"2023-12-19T18:28:08.616111+0000","last_peered":"2023-12-19T18:28:08.616111+0000","last_clean":"2023-12-19T18:28:08.616111+0000","last_became_active":"2023-12-19T18:28:08.100001+0000","last_became_peered":"2023-12-19T18:28:08.100001+0000","last_unstale":"2023-12-19T18:28:08.616111+0000","last_undegraded":"2023-12-19T18:28:08.616111+0000","last_fullsized":"2023-12-19T18:28:08.616111+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:46.547684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.618262+0000","last_change":"2023-12-19T18:28:08.618018+0000","last_active":"2023-12-19T18:28:08.618262+0000","last_peered":"2023-12-19T18:28:08.618262+0000","last_clean":"2023-12-19T18:28:08.618262+0000","last_became_active":"2023-12-19T18:28:08.617777+0000","last_became_peered":"2023-12-19T18:28:08.617777+0000","last_unstale":"2023-12-19T18:28:08.618262+0000","last_undegraded":"2023-12-19T18:28:08.618262+0000","last_fullsized":"2023-12-19T18:28:08.618262+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:52:40.164956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"201'1","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.600751+0000","last_change":"2023-12-19T18:28:08.095843+0000","last_active":"2023-12-19T18:28:08.600751+0000","last_peered":"2023-12-19T18:28:08.600751+0000","last_clean":"2023-12-19T18:28:08.600751+0000","last_became_active":"2023-12-19T18:28:08.095618+0000","last_became_peered":"2023-12-19T18:28:08.095618+0000","last_unstale":"2023-12-19T18:28:08.600751+0000","last_undegraded":"2023-12-19T18:28:08.600751+0000","last_fullsized":"2023-12-19T18:28:08.600751+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:48.383020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.619650+0000","last_change":"2023-12-19T18:28:08.619560+0000","last_active":"2023-12-19T18:28:08.619650+0000","last_peered":"2023-12-19T18:28:08.619650+0000","last_clean":"2023-12-19T18:28:08.619650+0000","last_became_active":"2023-12-19T18:28:08.616888+0000","last_became_peered":"2023-12-19T18:28:08.616888+0000","last_unstale":"2023-12-19T18:28:08.619650+0000","last_undegraded":"2023-12-19T18:28:08.619650+0000","last_fullsized":"2023-12-19T18:28:08.619650+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:49.397031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"201'1","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.610851+0000","last_change":"2023-12-19T18:28:08.099799+0000","last_active":"2023-12-19T18:28:08.610851+0000","last_peered":"2023-12-19T18:28:08.610851+0000","last_clean":"2023-12-19T18:28:08.610851+0000","last_became_active":"2023-12-19T18:28:08.099342+0000","last_became_peered":"2023-12-19T18:28:08.099342+0000","last_unstale":"2023-12-19T18:28:08.610851+0000","last_undegraded":"2023-12-19T18:28:08.610851+0000","last_fullsized":"2023-12-19T18:28:08.610851+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:56:56.287166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.604163+0000","last_change":"2023-12-19T18:28:08.603857+0000","last_active":"2023-12-19T18:28:08.604163+0000","last_peered":"2023-12-19T18:28:08.604163+0000","last_clean":"2023-12-19T18:28:08.604163+0000","last_became_active":"2023-12-19T18:28:08.601114+0000","last_became_peered":"2023-12-19T18:28:08.601114+0000","last_unstale":"2023-12-19T18:28:08.604163+0000","last_undegraded":"2023-12-19T18:28:08.604163+0000","last_fullsized":"2023-12-19T18:28:08.604163+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:10:03.084333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.592744+0000","last_change":"2023-12-19T18:28:08.104571+0000","last_active":"2023-12-19T18:28:08.592744+0000","last_peered":"2023-12-19T18:28:08.592744+0000","last_clean":"2023-12-19T18:28:08.592744+0000","last_became_active":"2023-12-19T18:28:08.103955+0000","last_became_peered":"2023-12-19T18:28:08.103955+0000","last_unstale":"2023-12-19T18:28:08.592744+0000","last_undegraded":"2023-12-19T18:28:08.592744+0000","last_fullsized":"2023-12-19T18:28:08.592744+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:10:02.070323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.614373+0000","last_change":"2023-12-19T18:28:08.611730+0000","last_active":"2023-12-19T18:28:08.614373+0000","last_peered":"2023-12-19T18:28:08.614373+0000","last_clean":"2023-12-19T18:28:08.614373+0000","last_became_active":"2023-12-19T18:28:08.610727+0000","last_became_peered":"2023-12-19T18:28:08.610727+0000","last_unstale":"2023-12-19T18:28:08.614373+0000","last_undegraded":"2023-12-19T18:28:08.614373+0000","last_fullsized":"2023-12-19T18:28:08.614373+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:10.641773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.593877+0000","last_change":"2023-12-19T18:28:08.103408+0000","last_active":"2023-12-19T18:28:08.593877+0000","last_peered":"2023-12-19T18:28:08.593877+0000","last_clean":"2023-12-19T18:28:08.593877+0000","last_became_active":"2023-12-19T18:28:08.102509+0000","last_became_peered":"2023-12-19T18:28:08.102509+0000","last_unstale":"2023-12-19T18:28:08.593877+0000","last_undegraded":"2023-12-19T18:28:08.593877+0000","last_fullsized":"2023-12-19T18:28:08.593877+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:12:42.480638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.619025+0000","last_change":"2023-12-19T18:28:08.618940+0000","last_active":"2023-12-19T18:28:08.619025+0000","last_peered":"2023-12-19T18:28:08.619025+0000","last_clean":"2023-12-19T18:28:08.619025+0000","last_became_active":"2023-12-19T18:28:08.618233+0000","last_became_peered":"2023-12-19T18:28:08.618233+0000","last_unstale":"2023-12-19T18:28:08.619025+0000","last_undegraded":"2023-12-19T18:28:08.619025+0000","last_fullsized":"2023-12-19T18:28:08.619025+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:07:31.276114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.603244+0000","last_change":"2023-12-19T18:28:08.108723+0000","last_active":"2023-12-19T18:28:08.603244+0000","last_peered":"2023-12-19T18:28:08.603244+0000","last_clean":"2023-12-19T18:28:08.603244+0000","last_became_active":"2023-12-19T18:28:08.108057+0000","last_became_peered":"2023-12-19T18:28:08.108057+0000","last_unstale":"2023-12-19T18:28:08.603244+0000","last_undegraded":"2023-12-19T18:28:08.603244+0000","last_fullsized":"2023-12-19T18:28:08.603244+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:59.338756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.613763+0000","last_change":"2023-12-19T18:28:08.613688+0000","last_active":"2023-12-19T18:28:08.613763+0000","last_peered":"2023-12-19T18:28:08.613763+0000","last_clean":"2023-12-19T18:28:08.613763+0000","last_became_active":"2023-12-19T18:28:08.613455+0000","last_became_peered":"2023-12-19T18:28:08.613455+0000","last_unstale":"2023-12-19T18:28:08.613763+0000","last_undegraded":"2023-12-19T18:28:08.613763+0000","last_fullsized":"2023-12-19T18:28:08.613763+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:53.893910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"201'3","reported_seq":13,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.616849+0000","last_change":"2023-12-19T18:28:08.099588+0000","last_active":"2023-12-19T18:28:08.616849+0000","last_peered":"2023-12-19T18:28:08.616849+0000","last_clean":"2023-12-19T18:28:08.616849+0000","last_became_active":"2023-12-19T18:28:08.098664+0000","last_became_peered":"2023-12-19T18:28:08.098664+0000","last_unstale":"2023-12-19T18:28:08.616849+0000","last_undegraded":"2023-12-19T18:28:08.616849+0000","last_fullsized":"2023-12-19T18:28:08.616849+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:21.999153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.605303+0000","last_change":"2023-12-19T18:28:08.603937+0000","last_active":"2023-12-19T18:28:08.605303+0000","last_peered":"2023-12-19T18:28:08.605303+0000","last_clean":"2023-12-19T18:28:08.605303+0000","last_became_active":"2023-12-19T18:28:08.603112+0000","last_became_peered":"2023-12-19T18:28:08.603112+0000","last_unstale":"2023-12-19T18:28:08.605303+0000","last_undegraded":"2023-12-19T18:28:08.605303+0000","last_fullsized":"2023-12-19T18:28:08.605303+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:44:26.745474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"201'6","reported_seq":16,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.611530+0000","last_change":"2023-12-19T18:28:08.106591+0000","last_active":"2023-12-19T18:28:08.611530+0000","last_peered":"2023-12-19T18:28:08.611530+0000","last_clean":"2023-12-19T18:28:08.611530+0000","last_became_active":"2023-12-19T18:28:08.106341+0000","last_became_peered":"2023-12-19T18:28:08.106341+0000","last_unstale":"2023-12-19T18:28:08.611530+0000","last_undegraded":"2023-12-19T18:28:08.611530+0000","last_fullsized":"2023-12-19T18:28:08.611530+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:59:25.726461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.608633+0000","last_change":"2023-12-19T18:28:08.608399+0000","last_active":"2023-12-19T18:28:08.608633+0000","last_peered":"2023-12-19T18:28:08.608633+0000","last_clean":"2023-12-19T18:28:08.608633+0000","last_became_active":"2023-12-19T18:28:08.607278+0000","last_became_peered":"2023-12-19T18:28:08.607278+0000","last_unstale":"2023-12-19T18:28:08.608633+0000","last_undegraded":"2023-12-19T18:28:08.608633+0000","last_fullsized":"2023-12-19T18:28:08.608633+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:56:57.301176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"201'2","reported_seq":12,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.610165+0000","last_change":"2023-12-19T18:28:08.103422+0000","last_active":"2023-12-19T18:28:08.610165+0000","last_peered":"2023-12-19T18:28:08.610165+0000","last_clean":"2023-12-19T18:28:08.610165+0000","last_became_active":"2023-12-19T18:28:08.102867+0000","last_became_peered":"2023-12-19T18:28:08.102867+0000","last_unstale":"2023-12-19T18:28:08.610165+0000","last_undegraded":"2023-12-19T18:28:08.610165+0000","last_fullsized":"2023-12-19T18:28:08.610165+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:52.879900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.615671+0000","last_change":"2023-12-19T18:28:08.612258+0000","last_active":"2023-12-19T18:28:08.615671+0000","last_peered":"2023-12-19T18:28:08.615671+0000","last_clean":"2023-12-19T18:28:08.615671+0000","last_became_active":"2023-12-19T18:28:08.609755+0000","last_became_peered":"2023-12-19T18:28:08.609755+0000","last_unstale":"2023-12-19T18:28:08.615671+0000","last_undegraded":"2023-12-19T18:28:08.615671+0000","last_fullsized":"2023-12-19T18:28:08.615671+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:12:46.711265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.603596+0000","last_change":"2023-12-19T18:28:08.106401+0000","last_active":"2023-12-19T18:28:08.603596+0000","last_peered":"2023-12-19T18:28:08.603596+0000","last_clean":"2023-12-19T18:28:08.603596+0000","last_became_active":"2023-12-19T18:28:08.106146+0000","last_became_peered":"2023-12-19T18:28:08.106146+0000","last_unstale":"2023-12-19T18:28:08.603596+0000","last_undegraded":"2023-12-19T18:28:08.603596+0000","last_fullsized":"2023-12-19T18:28:08.603596+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:52:22.064538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.614625+0000","last_change":"2023-12-19T18:28:08.614542+0000","last_active":"2023-12-19T18:28:08.614625+0000","last_peered":"2023-12-19T18:28:08.614625+0000","last_clean":"2023-12-19T18:28:08.614625+0000","last_became_active":"2023-12-19T18:28:08.614314+0000","last_became_peered":"2023-12-19T18:28:08.614314+0000","last_unstale":"2023-12-19T18:28:08.614625+0000","last_undegraded":"2023-12-19T18:28:08.614625+0000","last_fullsized":"2023-12-19T18:28:08.614625+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:42:59.325633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.603117+0000","last_change":"2023-12-19T18:28:08.098900+0000","last_active":"2023-12-19T18:28:08.603117+0000","last_peered":"2023-12-19T18:28:08.603117+0000","last_clean":"2023-12-19T18:28:08.603117+0000","last_became_active":"2023-12-19T18:28:08.098436+0000","last_became_peered":"2023-12-19T18:28:08.098436+0000","last_unstale":"2023-12-19T18:28:08.603117+0000","last_undegraded":"2023-12-19T18:28:08.603117+0000","last_fullsized":"2023-12-19T18:28:08.603117+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:46.547684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.609801+0000","last_change":"2023-12-19T18:28:08.609733+0000","last_active":"2023-12-19T18:28:08.609801+0000","last_peered":"2023-12-19T18:28:08.609801+0000","last_clean":"2023-12-19T18:28:08.609801+0000","last_became_active":"2023-12-19T18:28:08.609337+0000","last_became_peered":"2023-12-19T18:28:08.609337+0000","last_unstale":"2023-12-19T18:28:08.609801+0000","last_undegraded":"2023-12-19T18:28:08.609801+0000","last_fullsized":"2023-12-19T18:28:08.609801+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:16.483526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"201'8","reported_seq":18,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.611393+0000","last_change":"2023-12-19T18:28:08.107407+0000","last_active":"2023-12-19T18:28:08.611393+0000","last_peered":"2023-12-19T18:28:08.611393+0000","last_clean":"2023-12-19T18:28:08.611393+0000","last_became_active":"2023-12-19T18:28:08.107163+0000","last_became_peered":"2023-12-19T18:28:08.107163+0000","last_unstale":"2023-12-19T18:28:08.611393+0000","last_undegraded":"2023-12-19T18:28:08.611393+0000","last_fullsized":"2023-12-19T18:28:08.611393+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:44:16.886653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.617646+0000","last_change":"2023-12-19T18:28:08.617496+0000","last_active":"2023-12-19T18:28:08.617646+0000","last_peered":"2023-12-19T18:28:08.617646+0000","last_clean":"2023-12-19T18:28:08.617646+0000","last_became_active":"2023-12-19T18:28:08.617228+0000","last_became_peered":"2023-12-19T18:28:08.617228+0000","last_unstale":"2023-12-19T18:28:08.617646+0000","last_undegraded":"2023-12-19T18:28:08.617646+0000","last_fullsized":"2023-12-19T18:28:08.617646+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:12:46.711265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"201'1","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.603849+0000","last_change":"2023-12-19T18:28:08.107605+0000","last_active":"2023-12-19T18:28:08.603849+0000","last_peered":"2023-12-19T18:28:08.603849+0000","last_clean":"2023-12-19T18:28:08.603849+0000","last_became_active":"2023-12-19T18:28:08.107354+0000","last_became_peered":"2023-12-19T18:28:08.107354+0000","last_unstale":"2023-12-19T18:28:08.603849+0000","last_undegraded":"2023-12-19T18:28:08.603849+0000","last_fullsized":"2023-12-19T18:28:08.603849+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:37:54.253272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.607982+0000","last_change":"2023-12-19T18:28:08.606585+0000","last_active":"2023-12-19T18:28:08.607982+0000","last_peered":"2023-12-19T18:28:08.607982+0000","last_clean":"2023-12-19T18:28:08.607982+0000","last_became_active":"2023-12-19T18:28:08.606227+0000","last_became_peered":"2023-12-19T18:28:08.606227+0000","last_unstale":"2023-12-19T18:28:08.607982+0000","last_undegraded":"2023-12-19T18:28:08.607982+0000","last_fullsized":"2023-12-19T18:28:08.607982+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:59:10.061644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"201'2","reported_seq":12,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.613351+0000","last_change":"2023-12-19T18:28:08.104589+0000","last_active":"2023-12-19T18:28:08.613351+0000","last_peered":"2023-12-19T18:28:08.613351+0000","last_clean":"2023-12-19T18:28:08.613351+0000","last_became_active":"2023-12-19T18:28:08.104328+0000","last_became_peered":"2023-12-19T18:28:08.104328+0000","last_unstale":"2023-12-19T18:28:08.613351+0000","last_undegraded":"2023-12-19T18:28:08.613351+0000","last_fullsized":"2023-12-19T18:28:08.613351+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:12.156824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.615060+0000","last_change":"2023-12-19T18:28:08.614095+0000","last_active":"2023-12-19T18:28:08.615060+0000","last_peered":"2023-12-19T18:28:08.615060+0000","last_clean":"2023-12-19T18:28:08.615060+0000","last_became_active":"2023-12-19T18:28:08.613709+0000","last_became_peered":"2023-12-19T18:28:08.613709+0000","last_unstale":"2023-12-19T18:28:08.615060+0000","last_undegraded":"2023-12-19T18:28:08.615060+0000","last_fullsized":"2023-12-19T18:28:08.615060+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:44:26.745474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"201'1","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.602484+0000","last_change":"2023-12-19T18:28:08.097340+0000","last_active":"2023-12-19T18:28:08.602484+0000","last_peered":"2023-12-19T18:28:08.602484+0000","last_clean":"2023-12-19T18:28:08.602484+0000","last_became_active":"2023-12-19T18:28:08.096841+0000","last_became_peered":"2023-12-19T18:28:08.096841+0000","last_unstale":"2023-12-19T18:28:08.602484+0000","last_undegraded":"2023-12-19T18:28:08.602484+0000","last_fullsized":"2023-12-19T18:28:08.602484+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:39:53.529181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.616160+0000","last_change":"2023-12-19T18:28:08.614237+0000","last_active":"2023-12-19T18:28:08.616160+0000","last_peered":"2023-12-19T18:28:08.616160+0000","last_clean":"2023-12-19T18:28:08.616160+0000","last_became_active":"2023-12-19T18:28:08.613804+0000","last_became_peered":"2023-12-19T18:28:08.613804+0000","last_unstale":"2023-12-19T18:28:08.616160+0000","last_undegraded":"2023-12-19T18:28:08.616160+0000","last_fullsized":"2023-12-19T18:28:08.616160+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:18:16.064659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.586502+0000","last_change":"2023-12-19T18:28:08.104648+0000","last_active":"2023-12-19T18:28:08.586502+0000","last_peered":"2023-12-19T18:28:08.586502+0000","last_clean":"2023-12-19T18:28:08.586502+0000","last_became_active":"2023-12-19T18:28:08.104380+0000","last_became_peered":"2023-12-19T18:28:08.104380+0000","last_unstale":"2023-12-19T18:28:08.586502+0000","last_undegraded":"2023-12-19T18:28:08.586502+0000","last_fullsized":"2023-12-19T18:28:08.586502+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:29:04.592957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.606762+0000","last_change":"2023-12-19T18:28:08.606432+0000","last_active":"2023-12-19T18:28:08.606762+0000","last_peered":"2023-12-19T18:28:08.606762+0000","last_clean":"2023-12-19T18:28:08.606762+0000","last_became_active":"2023-12-19T18:28:08.606199+0000","last_became_peered":"2023-12-19T18:28:08.606199+0000","last_unstale":"2023-12-19T18:28:08.606762+0000","last_undegraded":"2023-12-19T18:28:08.606762+0000","last_fullsized":"2023-12-19T18:28:08.606762+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:34:10.949301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"201'2","reported_seq":12,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.617617+0000","last_change":"2023-12-19T18:28:08.102847+0000","last_active":"2023-12-19T18:28:08.617617+0000","last_peered":"2023-12-19T18:28:08.617617+0000","last_clean":"2023-12-19T18:28:08.617617+0000","last_became_active":"2023-12-19T18:28:08.102609+0000","last_became_peered":"2023-12-19T18:28:08.102609+0000","last_unstale":"2023-12-19T18:28:08.617617+0000","last_undegraded":"2023-12-19T18:28:08.617617+0000","last_fullsized":"2023-12-19T18:28:08.617617+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:59:14.820598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.615647+0000","last_change":"2023-12-19T18:28:08.612219+0000","last_active":"2023-12-19T18:28:08.615647+0000","last_peered":"2023-12-19T18:28:08.615647+0000","last_clean":"2023-12-19T18:28:08.615647+0000","last_became_active":"2023-12-19T18:28:08.604214+0000","last_became_peered":"2023-12-19T18:28:08.604214+0000","last_unstale":"2023-12-19T18:28:08.615647+0000","last_undegraded":"2023-12-19T18:28:08.615647+0000","last_fullsized":"2023-12-19T18:28:08.615647+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:40:04.871763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.586321+0000","last_change":"2023-12-19T18:28:08.106813+0000","last_active":"2023-12-19T18:28:08.586321+0000","last_peered":"2023-12-19T18:28:08.586321+0000","last_clean":"2023-12-19T18:28:08.586321+0000","last_became_active":"2023-12-19T18:28:08.106573+0000","last_became_peered":"2023-12-19T18:28:08.106573+0000","last_unstale":"2023-12-19T18:28:08.586321+0000","last_undegraded":"2023-12-19T18:28:08.586321+0000","last_fullsized":"2023-12-19T18:28:08.586321+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:48:52.520397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.601823+0000","last_change":"2023-12-19T18:28:08.599393+0000","last_active":"2023-12-19T18:28:08.601823+0000","last_peered":"2023-12-19T18:28:08.601823+0000","last_clean":"2023-12-19T18:28:08.601823+0000","last_became_active":"2023-12-19T18:28:08.599169+0000","last_became_peered":"2023-12-19T18:28:08.599169+0000","last_unstale":"2023-12-19T18:28:08.601823+0000","last_undegraded":"2023-12-19T18:28:08.601823+0000","last_fullsized":"2023-12-19T18:28:08.601823+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:00.473295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.590069+0000","last_change":"2023-12-19T18:28:08.098465+0000","last_active":"2023-12-19T18:28:08.590069+0000","last_peered":"2023-12-19T18:28:08.590069+0000","last_clean":"2023-12-19T18:28:08.590069+0000","last_became_active":"2023-12-19T18:28:08.097821+0000","last_became_peered":"2023-12-19T18:28:08.097821+0000","last_unstale":"2023-12-19T18:28:08.590069+0000","last_undegraded":"2023-12-19T18:28:08.590069+0000","last_fullsized":"2023-12-19T18:28:08.590069+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:07:30.262104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.608333+0000","last_change":"2023-12-19T18:28:08.607093+0000","last_active":"2023-12-19T18:28:08.608333+0000","last_peered":"2023-12-19T18:28:08.608333+0000","last_clean":"2023-12-19T18:28:08.608333+0000","last_became_active":"2023-12-19T18:28:08.606383+0000","last_became_peered":"2023-12-19T18:28:08.606383+0000","last_unstale":"2023-12-19T18:28:08.608333+0000","last_undegraded":"2023-12-19T18:28:08.608333+0000","last_fullsized":"2023-12-19T18:28:08.608333+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:25:46.176399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.586176+0000","last_change":"2023-12-19T18:28:08.103729+0000","last_active":"2023-12-19T18:28:08.586176+0000","last_peered":"2023-12-19T18:28:08.586176+0000","last_clean":"2023-12-19T18:28:08.586176+0000","last_became_active":"2023-12-19T18:28:08.103485+0000","last_became_peered":"2023-12-19T18:28:08.103485+0000","last_unstale":"2023-12-19T18:28:08.586176+0000","last_undegraded":"2023-12-19T18:28:08.586176+0000","last_fullsized":"2023-12-19T18:28:08.586176+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:33.904383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.617428+0000","last_change":"2023-12-19T18:28:08.616997+0000","last_active":"2023-12-19T18:28:08.617428+0000","last_peered":"2023-12-19T18:28:08.617428+0000","last_clean":"2023-12-19T18:28:08.617428+0000","last_became_active":"2023-12-19T18:28:08.616499+0000","last_became_peered":"2023-12-19T18:28:08.616499+0000","last_unstale":"2023-12-19T18:28:08.617428+0000","last_undegraded":"2023-12-19T18:28:08.617428+0000","last_fullsized":"2023-12-19T18:28:08.617428+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:28:17.475387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"201'2","reported_seq":12,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.617413+0000","last_change":"2023-12-19T18:28:08.108917+0000","last_active":"2023-12-19T18:28:08.617413+0000","last_peered":"2023-12-19T18:28:08.617413+0000","last_clean":"2023-12-19T18:28:08.617413+0000","last_became_active":"2023-12-19T18:28:08.108489+0000","last_became_peered":"2023-12-19T18:28:08.108489+0000","last_unstale":"2023-12-19T18:28:08.617413+0000","last_undegraded":"2023-12-19T18:28:08.617413+0000","last_fullsized":"2023-12-19T18:28:08.617413+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:37.313662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.603803+0000","last_change":"2023-12-19T18:28:08.603491+0000","last_active":"2023-12-19T18:28:08.603803+0000","last_peered":"2023-12-19T18:28:08.603803+0000","last_clean":"2023-12-19T18:28:08.603803+0000","last_became_active":"2023-12-19T18:28:08.603042+0000","last_became_peered":"2023-12-19T18:28:08.603042+0000","last_unstale":"2023-12-19T18:28:08.603803+0000","last_undegraded":"2023-12-19T18:28:08.603803+0000","last_fullsized":"2023-12-19T18:28:08.603803+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:48.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.586630+0000","last_change":"2023-12-19T18:28:08.101563+0000","last_active":"2023-12-19T18:28:08.586630+0000","last_peered":"2023-12-19T18:28:08.586630+0000","last_clean":"2023-12-19T18:28:08.586630+0000","last_became_active":"2023-12-19T18:28:08.101320+0000","last_became_peered":"2023-12-19T18:28:08.101320+0000","last_unstale":"2023-12-19T18:28:08.586630+0000","last_undegraded":"2023-12-19T18:28:08.586630+0000","last_fullsized":"2023-12-19T18:28:08.586630+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:08:10.434690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.614338+0000","last_change":"2023-12-19T18:28:08.610379+0000","last_active":"2023-12-19T18:28:08.614338+0000","last_peered":"2023-12-19T18:28:08.614338+0000","last_clean":"2023-12-19T18:28:08.614338+0000","last_became_active":"2023-12-19T18:28:08.609663+0000","last_became_peered":"2023-12-19T18:28:08.609663+0000","last_unstale":"2023-12-19T18:28:08.614338+0000","last_undegraded":"2023-12-19T18:28:08.614338+0000","last_fullsized":"2023-12-19T18:28:08.614338+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:09.966919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.592530+0000","last_change":"2023-12-19T18:28:08.093810+0000","last_active":"2023-12-19T18:28:08.592530+0000","last_peered":"2023-12-19T18:28:08.592530+0000","last_clean":"2023-12-19T18:28:08.592530+0000","last_became_active":"2023-12-19T18:28:08.093297+0000","last_became_peered":"2023-12-19T18:28:08.093297+0000","last_unstale":"2023-12-19T18:28:08.592530+0000","last_undegraded":"2023-12-19T18:28:08.592530+0000","last_fullsized":"2023-12-19T18:28:08.592530+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:40:03.857753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.605090+0000","last_change":"2023-12-19T18:28:08.604835+0000","last_active":"2023-12-19T18:28:08.605090+0000","last_peered":"2023-12-19T18:28:08.605090+0000","last_clean":"2023-12-19T18:28:08.605090+0000","last_became_active":"2023-12-19T18:28:08.604364+0000","last_became_peered":"2023-12-19T18:28:08.604364+0000","last_unstale":"2023-12-19T18:28:08.605090+0000","last_undegraded":"2023-12-19T18:28:08.605090+0000","last_fullsized":"2023-12-19T18:28:08.605090+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:00.473295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.594105+0000","last_change":"2023-12-19T18:28:08.105010+0000","last_active":"2023-12-19T18:28:08.594105+0000","last_peered":"2023-12-19T18:28:08.594105+0000","last_clean":"2023-12-19T18:28:08.594105+0000","last_became_active":"2023-12-19T18:28:08.103878+0000","last_became_peered":"2023-12-19T18:28:08.103878+0000","last_unstale":"2023-12-19T18:28:08.594105+0000","last_undegraded":"2023-12-19T18:28:08.594105+0000","last_fullsized":"2023-12-19T18:28:08.594105+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:11:05.754382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.619876+0000","last_change":"2023-12-19T18:28:08.619723+0000","last_active":"2023-12-19T18:28:08.619876+0000","last_peered":"2023-12-19T18:28:08.619876+0000","last_clean":"2023-12-19T18:28:08.619876+0000","last_became_active":"2023-12-19T18:28:08.617193+0000","last_became_peered":"2023-12-19T18:28:08.617193+0000","last_unstale":"2023-12-19T18:28:08.619876+0000","last_undegraded":"2023-12-19T18:28:08.619876+0000","last_fullsized":"2023-12-19T18:28:08.619876+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:34:00.405702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.585151+0000","last_change":"2023-12-19T18:28:08.095504+0000","last_active":"2023-12-19T18:28:08.585151+0000","last_peered":"2023-12-19T18:28:08.585151+0000","last_clean":"2023-12-19T18:28:08.585151+0000","last_became_active":"2023-12-19T18:28:08.093962+0000","last_became_peered":"2023-12-19T18:28:08.093962+0000","last_unstale":"2023-12-19T18:28:08.585151+0000","last_undegraded":"2023-12-19T18:28:08.585151+0000","last_fullsized":"2023-12-19T18:28:08.585151+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:20:59.865768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.609812+0000","last_change":"2023-12-19T18:28:08.609589+0000","last_active":"2023-12-19T18:28:08.609812+0000","last_peered":"2023-12-19T18:28:08.609812+0000","last_clean":"2023-12-19T18:28:08.609812+0000","last_became_active":"2023-12-19T18:28:08.609368+0000","last_became_peered":"2023-12-19T18:28:08.609368+0000","last_unstale":"2023-12-19T18:28:08.609812+0000","last_undegraded":"2023-12-19T18:28:08.609812+0000","last_fullsized":"2023-12-19T18:28:08.609812+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:48:13.756314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.603258+0000","last_change":"2023-12-19T18:28:08.106735+0000","last_active":"2023-12-19T18:28:08.603258+0000","last_peered":"2023-12-19T18:28:08.603258+0000","last_clean":"2023-12-19T18:28:08.603258+0000","last_became_active":"2023-12-19T18:28:08.106493+0000","last_became_peered":"2023-12-19T18:28:08.106493+0000","last_unstale":"2023-12-19T18:28:08.603258+0000","last_undegraded":"2023-12-19T18:28:08.603258+0000","last_fullsized":"2023-12-19T18:28:08.603258+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:21.999153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.608168+0000","last_change":"2023-12-19T18:28:08.605546+0000","last_active":"2023-12-19T18:28:08.608168+0000","last_peered":"2023-12-19T18:28:08.608168+0000","last_clean":"2023-12-19T18:28:08.608168+0000","last_became_active":"2023-12-19T18:28:08.604914+0000","last_became_peered":"2023-12-19T18:28:08.604914+0000","last_unstale":"2023-12-19T18:28:08.608168+0000","last_undegraded":"2023-12-19T18:28:08.608168+0000","last_fullsized":"2023-12-19T18:28:08.608168+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:28:17.475387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.593469+0000","last_change":"2023-12-19T18:28:08.098072+0000","last_active":"2023-12-19T18:28:08.593469+0000","last_peered":"2023-12-19T18:28:08.593469+0000","last_clean":"2023-12-19T18:28:08.593469+0000","last_became_active":"2023-12-19T18:28:08.096845+0000","last_became_peered":"2023-12-19T18:28:08.096845+0000","last_unstale":"2023-12-19T18:28:08.593469+0000","last_undegraded":"2023-12-19T18:28:08.593469+0000","last_fullsized":"2023-12-19T18:28:08.593469+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:45.923467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.605348+0000","last_change":"2023-12-19T18:28:08.604237+0000","last_active":"2023-12-19T18:28:08.605348+0000","last_peered":"2023-12-19T18:28:08.605348+0000","last_clean":"2023-12-19T18:28:08.605348+0000","last_became_active":"2023-12-19T18:28:08.603720+0000","last_became_peered":"2023-12-19T18:28:08.603720+0000","last_unstale":"2023-12-19T18:28:08.605348+0000","last_undegraded":"2023-12-19T18:28:08.605348+0000","last_fullsized":"2023-12-19T18:28:08.605348+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:10.641773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.592629+0000","last_change":"2023-12-19T18:28:08.104765+0000","last_active":"2023-12-19T18:28:08.592629+0000","last_peered":"2023-12-19T18:28:08.592629+0000","last_clean":"2023-12-19T18:28:08.592629+0000","last_became_active":"2023-12-19T18:28:08.104333+0000","last_became_peered":"2023-12-19T18:28:08.104333+0000","last_unstale":"2023-12-19T18:28:08.592629+0000","last_undegraded":"2023-12-19T18:28:08.592629+0000","last_fullsized":"2023-12-19T18:28:08.592629+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:28:16.461376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.612435+0000","last_change":"2023-12-19T18:28:08.611196+0000","last_active":"2023-12-19T18:28:08.612435+0000","last_peered":"2023-12-19T18:28:08.612435+0000","last_clean":"2023-12-19T18:28:08.612435+0000","last_became_active":"2023-12-19T18:28:08.608529+0000","last_became_peered":"2023-12-19T18:28:08.608529+0000","last_unstale":"2023-12-19T18:28:08.612435+0000","last_undegraded":"2023-12-19T18:28:08.612435+0000","last_fullsized":"2023-12-19T18:28:08.612435+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:10:03.084333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.584366+0000","last_change":"2023-12-19T18:28:08.098572+0000","last_active":"2023-12-19T18:28:08.584366+0000","last_peered":"2023-12-19T18:28:08.584366+0000","last_clean":"2023-12-19T18:28:08.584366+0000","last_became_active":"2023-12-19T18:28:08.098280+0000","last_became_peered":"2023-12-19T18:28:08.098280+0000","last_unstale":"2023-12-19T18:28:08.584366+0000","last_undegraded":"2023-12-19T18:28:08.584366+0000","last_fullsized":"2023-12-19T18:28:08.584366+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:00:26.251041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":326,"reported_epoch":200,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.087468+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:28:08.087468+0000","last_peered":"2023-12-19T18:28:08.087468+0000","last_clean":"2023-12-19T18:28:08.087468+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:28:08.087468+0000","last_undegraded":"2023-12-19T18:28:08.087468+0000","last_fullsized":"2023-12-19T18:28:08.087468+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575011,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1105,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194206,"statfs":{"total":200000000,"available":198867629,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.169,"15min":1.1719999999999999},"min":{"1min":0.66500000000000004,"5min":0.59299999999999997,"15min":0.45900000000000002},"max":{"1min":1.599,"5min":1.756,"15min":5.6879999999999997},"last":0.995},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.23,"15min":1.2190000000000001},"min":{"1min":0.90700000000000003,"5min":0.71899999999999997,"15min":0.55800000000000005},"max":{"1min":1.5700000000000001,"5min":1.7030000000000001,"15min":1.79},"last":1.123}]},{"osd":1,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.175,"15min":1.2330000000000001},"min":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.54400000000000004},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":10.926},"last":0.81699999999999995},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2709999999999999,"15min":1.27},"min":{"1min":0.85099999999999998,"5min":0.80700000000000005,"15min":0.67900000000000005},"max":{"1min":1.619,"5min":1.736,"15min":10.904999999999999},"last":0.86199999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1020000000000001,"15min":1.1100000000000001},"min":{"1min":0.74099999999999999,"5min":0.66000000000000003,"15min":0.61499999999999999},"max":{"1min":1.365,"5min":1.6200000000000001,"15min":3.1960000000000002},"last":0.94099999999999995},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.167,"15min":1.198},"min":{"1min":0.78700000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.587,"5min":1.6799999999999999,"15min":6.008},"last":0.91500000000000004}]},{"osd":3,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.244,"15min":1.2130000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.61399999999999999},"max":{"1min":1.575,"5min":1.732,"15min":3.0379999999999998},"last":1.1499999999999999},{"interface":"front","average":{"1min":1.319,"5min":1.333,"15min":1.333},"min":{"1min":0.89500000000000002,"5min":0.77500000000000002,"15min":0.69599999999999995},"max":{"1min":1.704,"5min":1.772,"15min":5.8120000000000003},"last":0.69899999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1919999999999999,"15min":1.173},"min":{"1min":0.67700000000000005,"5min":0.55700000000000005,"15min":0.47899999999999998},"max":{"1min":1.454,"5min":1.847,"15min":4.0110000000000001},"last":1.038},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.242,"15min":1.2529999999999999},"min":{"1min":0.65500000000000003,"5min":0.59299999999999997,"15min":0.57599999999999996},"max":{"1min":1.502,"5min":1.8,"15min":3.3180000000000001},"last":1.2090000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.28,"15min":1.331},"min":{"1min":0.91900000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.617,"5min":1.956,"15min":7.6429999999999998},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.323,"5min":1.3460000000000001,"15min":1.3580000000000001},"min":{"1min":0.81799999999999995,"5min":0.627,"15min":0.53500000000000003},"max":{"1min":1.6799999999999999,"5min":1.8280000000000001,"15min":6.0839999999999996},"last":1.337}]},{"osd":6,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.492,"15min":1.4970000000000001},"min":{"1min":0.95599999999999996,"5min":0.628,"15min":0.59899999999999998},"max":{"1min":1.7290000000000001,"5min":1.8959999999999999,"15min":3.1309999999999998},"last":1.673},{"interface":"front","average":{"1min":1.415,"5min":1.4319999999999999,"15min":1.419},"min":{"1min":0.73199999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.641,"5min":1.929,"15min":4.1479999999999997},"last":1.393}]}]},{"osd":6,"up_from":13,"seq":55834575011,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1105,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194206,"statfs":{"total":200000000,"available":198867629,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.099,"15min":1.0680000000000001},"min":{"1min":0.68300000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.413,"5min":1.522,"15min":1.8129999999999999},"last":1.413},{"interface":"front","average":{"1min":1.147,"5min":1.151,"15min":1.1220000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.55600000000000005},"max":{"1min":1.587,"5min":1.6000000000000001,"15min":3.5529999999999999},"last":1.347}]},{"osd":1,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1220000000000001,"15min":1.131},"min":{"1min":0.75700000000000001,"5min":0.68600000000000005,"15min":0.68200000000000005},"max":{"1min":1.573,"5min":1.7909999999999999,"15min":4.5540000000000003},"last":0.96099999999999997},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.177,"15min":1.179},"min":{"1min":0.80100000000000005,"5min":0.64600000000000002,"15min":0.60299999999999998},"max":{"1min":1.5569999999999999,"5min":1.5960000000000001,"15min":1.885},"last":1.1890000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.258,"15min":1.21},"min":{"1min":0.89900000000000002,"5min":0.78900000000000003,"15min":0.626},"max":{"1min":1.5569999999999999,"5min":1.6479999999999999,"15min":1.821},"last":1.27},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2190000000000001,"15min":1.216},"min":{"1min":0.85899999999999999,"5min":0.62,"15min":0.60899999999999999},"max":{"1min":1.514,"5min":1.7669999999999999,"15min":3.7909999999999999},"last":1.22}]},{"osd":3,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3129999999999999,"15min":1.2569999999999999},"min":{"1min":0.90800000000000003,"5min":0.71799999999999997,"15min":0.67800000000000005},"max":{"1min":1.5469999999999999,"5min":1.804,"15min":3.8820000000000001},"last":1.121},{"interface":"front","average":{"1min":1.258,"5min":1.248,"15min":1.2050000000000001},"min":{"1min":0.96099999999999997,"5min":0.71199999999999997,"15min":0.69899999999999995},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":4.6189999999999998},"last":1.149}]},{"osd":4,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.306,"15min":1.2809999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.65600000000000003},"max":{"1min":1.6259999999999999,"5min":1.7529999999999999,"15min":3.0680000000000001},"last":1.3240000000000001},{"interface":"front","average":{"1min":1.417,"5min":1.415,"15min":1.3939999999999999},"min":{"1min":1.0529999999999999,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.7090000000000001,"5min":2.1040000000000001,"15min":3.7000000000000002},"last":1.2949999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2809999999999999,"15min":1.2470000000000001},"min":{"1min":0.91500000000000004,"5min":0.84899999999999998,"15min":0.65500000000000003},"max":{"1min":1.7929999999999999,"5min":1.9510000000000001,"15min":4.6680000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.375,"5min":1.4079999999999999,"15min":1.3959999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.626},"max":{"1min":1.609,"5min":1.794,"15min":4.1159999999999997},"last":1.4810000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.444,"15min":1.403},"min":{"1min":1.0149999999999999,"5min":0.65700000000000003,"15min":0.61899999999999999},"max":{"1min":1.663,"5min":2.02,"15min":3.9670000000000001},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4470000000000001,"15min":1.413},"min":{"1min":1.0309999999999999,"5min":0.999,"15min":0.79300000000000004},"max":{"1min":1.855,"5min":2.2149999999999999,"15min":5.6639999999999997},"last":1.238}]}]},{"osd":5,"up_from":13,"seq":55834575011,"num_pgs":6,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1676,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193636,"statfs":{"total":200000000,"available":198283536,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.26,"15min":1.21},"min":{"1min":0.92900000000000005,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.089,"5min":2.089,"15min":2.089},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.341,"5min":1.29,"15min":1.24},"min":{"1min":1.0209999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.641,"5min":1.913,"15min":1.913},"last":1.0149999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.1699999999999999,"15min":1.1240000000000001},"min":{"1min":0.80600000000000005,"5min":0.71599999999999997,"15min":0.65700000000000003},"max":{"1min":1.851,"5min":1.851,"15min":1.9490000000000001},"last":1.0569999999999999},{"interface":"front","average":{"1min":1.262,"5min":1.1930000000000001,"15min":1.1279999999999999},"min":{"1min":0.71399999999999997,"5min":0.505,"15min":0.505},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":1.224}]},{"osd":2,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2370000000000001,"5min":1.2010000000000001,"15min":1.141},"min":{"1min":0.98999999999999999,"5min":0.72199999999999998,"15min":0.59499999999999997},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.27,"15min":1.206},"min":{"1min":0.74099999999999999,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.7430000000000001,"5min":1.8,"15min":2.036},"last":1.149}]},{"osd":3,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.167,"5min":1.1910000000000001,"15min":1.1499999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.68899999999999995},"max":{"1min":1.589,"5min":1.635,"15min":1.7450000000000001},"last":1.266},{"interface":"front","average":{"1min":1.278,"5min":1.347,"15min":1.2909999999999999},"min":{"1min":0.95899999999999996,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":2.0369999999999999,"5min":2.0369999999999999,"15min":2.0590000000000002},"last":1.2549999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.24,"5min":1.2949999999999999,"15min":1.2470000000000001},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.46700000000000003},"max":{"1min":1.659,"5min":1.944,"15min":2.0329999999999999},"last":1.327},{"interface":"front","average":{"1min":1.103,"5min":1.1559999999999999,"15min":1.153},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.57099999999999995},"max":{"1min":1.47,"5min":1.764,"15min":1.901},"last":1.1020000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.4470000000000001,"15min":1.3959999999999999},"min":{"1min":0.56699999999999995,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":2.1579999999999999,"5min":2.1579999999999999,"15min":2.1589999999999998},"last":1.381},{"interface":"front","average":{"1min":1.232,"5min":1.3069999999999999,"15min":1.282},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.49299999999999999},"max":{"1min":1.5800000000000001,"5min":1.704,"15min":1.976},"last":0.94999999999999996}]},{"osd":7,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.256,"5min":1.29,"15min":1.2629999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.748,"5min":1.863,"15min":2.0270000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.4099999999999999,"5min":1.4550000000000001,"15min":1.4370000000000001},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.623},"max":{"1min":1.776,"5min":1.8620000000000001,"15min":2.117},"last":1.1759999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575011,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1099,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194212,"statfs":{"total":200000000,"available":198873904,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.0269999999999999,"5min":1.0569999999999999,"15min":1.018},"min":{"1min":0.64900000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.494,"5min":1.972,"15min":1.972},"last":1.373},{"interface":"front","average":{"1min":1.1339999999999999,"5min":1.1100000000000001,"15min":1.0620000000000001},"min":{"1min":0.77300000000000002,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.4039999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2330000000000001,"5min":1.1870000000000001,"15min":1.1639999999999999},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":1.7689999999999999,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.129},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.256,"15min":1.224},"min":{"1min":0.95099999999999996,"5min":0.64700000000000002,"15min":0.61699999999999999},"max":{"1min":1.5289999999999999,"5min":1.875,"15min":1.875},"last":1.1479999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.129,"15min":1.1299999999999999},"min":{"1min":0.76100000000000001,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.744,"5min":1.744,"15min":1.744},"last":1.097},{"interface":"front","average":{"1min":1.2210000000000001,"5min":1.1759999999999999,"15min":1.139},"min":{"1min":0.879,"5min":0.747,"15min":0.70499999999999996},"max":{"1min":1.653,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.99399999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.238,"5min":1.2490000000000001,"15min":1.2629999999999999},"min":{"1min":0.78300000000000003,"5min":0.754,"15min":0.754},"max":{"1min":1.6240000000000001,"5min":1.946,"15min":1.946},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.3660000000000001,"5min":1.357,"15min":1.3280000000000001},"min":{"1min":0.94799999999999995,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.8200000000000001,"5min":1.927,"15min":1.927},"last":1.177}]},{"osd":5,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3,"5min":1.266,"15min":1.2410000000000001},"min":{"1min":0.90600000000000003,"5min":0.82299999999999995,"15min":0.44500000000000001},"max":{"1min":1.704,"5min":1.8129999999999999,"15min":1.8740000000000001},"last":1.339},{"interface":"front","average":{"1min":1.417,"5min":1.343,"15min":1.3640000000000001},"min":{"1min":0.81100000000000005,"5min":0.59099999999999997,"15min":0.504},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.82599999999999996}]},{"osd":6,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3440000000000001,"5min":1.288,"15min":1.3009999999999999},"min":{"1min":0.83799999999999997,"5min":0.68100000000000005,"15min":0.432},"max":{"1min":2.0329999999999999,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.466,"5min":1.413,"15min":1.415},"min":{"1min":1.1639999999999999,"5min":0.90800000000000003,"15min":0.64500000000000002},"max":{"1min":1.7250000000000001,"5min":1.8819999999999999,"15min":1.8819999999999999},"last":1.2270000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:27:09 2023","interfaces":[{"interface":"back","average":{"1min":1.448,"5min":1.335,"15min":1.355},"min":{"1min":0.95799999999999996,"5min":0.66000000000000003,"15min":0.56399999999999995},"max":{"1min":1.851,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":1.2829999999999999},{"interface":"front","average":{"1min":1.446,"5min":1.4119999999999999,"15min":1.4139999999999999},"min":{"1min":0.95299999999999996,"5min":0.78600000000000003,"15min":0.57399999999999995},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.2649999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575012,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1099,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194212,"statfs":{"total":200000000,"available":198873904,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90900000000000003,"15min":0.92900000000000005},"min":{"1min":0.72299999999999998,"5min":0.435,"15min":0.35299999999999998},"max":{"1min":1.2450000000000001,"5min":1.7450000000000001,"15min":3.4350000000000001},"last":0.873},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0409999999999999,"15min":0.99399999999999999},"min":{"1min":0.67100000000000004,"5min":0.54600000000000004,"15min":0.39500000000000002},"max":{"1min":1.401,"5min":1.9159999999999999,"15min":4.3490000000000002},"last":1.1479999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.034,"15min":1.032},"min":{"1min":0.79600000000000004,"5min":0.60899999999999999,"15min":0.41299999999999998},"max":{"1min":1.698,"5min":1.9690000000000001,"15min":5.2510000000000003},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.139,"5min":1.0900000000000001,"15min":1.107},"min":{"1min":0.88100000000000001,"5min":0.621,"15min":0.47799999999999998},"max":{"1min":1.6419999999999999,"5min":1.643,"15min":5.4119999999999999},"last":1.2549999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.214,"15min":1.242},"min":{"1min":0.872,"5min":0.628,"15min":0.46000000000000002},"max":{"1min":1.6619999999999999,"5min":2.4649999999999999,"15min":13.833},"last":1.3},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3220000000000001,"15min":1.3520000000000001},"min":{"1min":0.95499999999999996,"5min":0.76200000000000001,"15min":0.53200000000000003},"max":{"1min":1.6459999999999999,"5min":2.379,"15min":16.748999999999999},"last":1.4099999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.306,"15min":1.286},"min":{"1min":0.94799999999999995,"5min":0.77900000000000003,"15min":0.65600000000000003},"max":{"1min":1.744,"5min":2.0150000000000001,"15min":5.1790000000000003},"last":1.3220000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.387,"15min":1.397},"min":{"1min":1.0169999999999999,"5min":0.94199999999999995,"15min":0.73199999999999998},"max":{"1min":1.534,"5min":2.0819999999999999,"15min":4.569},"last":1.206}]},{"osd":5,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.411,"15min":1.429},"min":{"1min":1.1719999999999999,"5min":0.80700000000000005,"15min":0.76000000000000001},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":5.3289999999999997},"last":1.0649999999999999},{"interface":"front","average":{"1min":1.431,"5min":1.389,"15min":1.3400000000000001},"min":{"1min":1.0409999999999999,"5min":0.95199999999999996,"15min":0.69999999999999996},"max":{"1min":1.9610000000000001,"5min":2.1469999999999998,"15min":4.6369999999999996},"last":1.4419999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.46,"15min":1.456},"min":{"1min":1.139,"5min":0.86499999999999999,"15min":0.81599999999999995},"max":{"1min":1.774,"5min":2.2040000000000002,"15min":4.5039999999999996},"last":1.5},{"interface":"front","average":{"1min":1.548,"5min":1.506,"15min":1.5089999999999999},"min":{"1min":1.3120000000000001,"5min":0.876,"15min":0.80500000000000005},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":5.3739999999999997},"last":1.4770000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.4870000000000001,"15min":1.4430000000000001},"min":{"1min":1.224,"5min":0.89700000000000002,"15min":0.73199999999999998},"max":{"1min":2.1120000000000001,"5min":2.2869999999999999,"15min":5.1349999999999998},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5660000000000001,"15min":1.556},"min":{"1min":1.367,"5min":1.1020000000000001,"15min":0.85399999999999998},"max":{"1min":1.8260000000000001,"5min":2.351,"15min":4.4619999999999997},"last":1.3680000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575011,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1105,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194206,"statfs":{"total":200000000,"available":198867629,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.94599999999999995,"15min":0.98099999999999998},"min":{"1min":0.45000000000000001,"5min":0.43099999999999999,"15min":0.34399999999999997},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":9.2550000000000008},"last":1.4199999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0389999999999999,"15min":1.024},"min":{"1min":0.745,"5min":0.38400000000000001,"15min":0.33900000000000002},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":4.7850000000000001},"last":0.88300000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.95199999999999996,"15min":1.018},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.262,"5min":2.0169999999999999,"15min":8.2720000000000002},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.069,"5min":1.1180000000000001,"15min":1.1200000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.46600000000000003},"max":{"1min":1.413,"5min":1.8460000000000001,"15min":5.0620000000000003},"last":1.0640000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2749999999999999,"15min":1.252},"min":{"1min":0.64200000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.23,"5min":2.8159999999999998,"15min":8.3109999999999999},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1180000000000001,"15min":1.1850000000000001},"min":{"1min":0.53100000000000003,"5min":0.49299999999999999,"15min":0.41799999999999998},"max":{"1min":1.778,"5min":2.6240000000000001,"15min":8.0690000000000008},"last":1.292}]},{"osd":4,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.369,"15min":1.403},"min":{"1min":1,"5min":0.79400000000000004,"15min":0.79300000000000004},"max":{"1min":1.8620000000000001,"5min":2.2160000000000002,"15min":5.3929999999999998},"last":1.506},{"interface":"front","average":{"1min":1.46,"5min":1.4610000000000001,"15min":1.4750000000000001},"min":{"1min":1.036,"5min":0.85799999999999998,"15min":0.78300000000000003},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":7.3570000000000002},"last":1.373}]},{"osd":5,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.379,"15min":1.429},"min":{"1min":0.84499999999999997,"5min":0.747,"15min":0.747},"max":{"1min":2.133,"5min":2.1680000000000001,"15min":8.1839999999999993},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3720000000000001,"15min":1.3919999999999999},"min":{"1min":1.008,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9239999999999999,"5min":1.9930000000000001,"15min":5.165},"last":1.581}]},{"osd":6,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.4299999999999999,"15min":1.4279999999999999},"min":{"1min":0.91100000000000003,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.1629999999999998,"5min":2.4630000000000001,"15min":5.1059999999999999},"last":1.4610000000000001},{"interface":"front","average":{"1min":1.462,"5min":1.4930000000000001,"15min":1.4970000000000001},"min":{"1min":0.89600000000000002,"5min":0.83799999999999997,"15min":0.77000000000000002},"max":{"1min":2.3399999999999999,"5min":2.7559999999999998,"15min":8.1370000000000005},"last":1.1299999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.524,"15min":1.514},"min":{"1min":0.96699999999999997,"5min":0.80000000000000004,"15min":0.78200000000000003},"max":{"1min":1.968,"5min":2.3959999999999999,"15min":5.1449999999999996},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.575,"5min":1.593,"15min":1.585},"min":{"1min":0.94699999999999995,"5min":0.93000000000000005,"15min":0.86499999999999999},"max":{"1min":2.419,"5min":2.6320000000000001,"15min":8.2300000000000004},"last":1.405}]}]},{"osd":1,"up_from":13,"seq":55834575012,"num_pgs":7,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1682,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193630,"statfs":{"total":200000000,"available":198277261,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.90900000000000003,"15min":0.86799999999999999},"min":{"1min":0.63900000000000001,"5min":0.41199999999999998,"15min":0.34699999999999998},"max":{"1min":1.3520000000000001,"5min":2.262,"15min":2.262},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.016,"5min":1.01,"15min":0.96499999999999997},"min":{"1min":0.46800000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.349,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.113}]},{"osd":2,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.016,"15min":1.036},"min":{"1min":0.70399999999999996,"5min":0.57499999999999996,"15min":0.41399999999999998},"max":{"1min":1.377,"5min":1.53,"15min":2.5739999999999998},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.014,"15min":0.97499999999999998},"min":{"1min":0.621,"5min":0.39900000000000002,"15min":0.38500000000000001},"max":{"1min":1.379,"5min":1.718,"15min":1.845},"last":0.80600000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.347,"15min":1.254},"min":{"1min":0.66900000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.825,"5min":2.089,"15min":2.137},"last":1.0549999999999999},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.2450000000000001,"15min":1.204},"min":{"1min":0.59299999999999997,"5min":0.50800000000000001,"15min":0.44900000000000001},"max":{"1min":1.8700000000000001,"5min":2.0720000000000001,"15min":2.3959999999999999},"last":1.194}]},{"osd":4,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3149999999999999,"15min":1.3120000000000001},"min":{"1min":1.1990000000000001,"5min":0.80500000000000005,"15min":0.69099999999999995},"max":{"1min":1.7929999999999999,"5min":2.1859999999999999,"15min":2.444},"last":1.2330000000000001},{"interface":"front","average":{"1min":1.387,"5min":1.381,"15min":1.3480000000000001},"min":{"1min":0.94499999999999995,"5min":0.92500000000000004,"15min":0.71099999999999997},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.6440000000000001},"last":1.1000000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4259999999999999,"15min":1.3640000000000001},"min":{"1min":0.79200000000000004,"5min":0.76700000000000002,"15min":0.76000000000000001},"max":{"1min":1.915,"5min":1.998,"15min":2.5459999999999998},"last":1.6970000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.4330000000000001,"15min":1.4350000000000001},"min":{"1min":1.0760000000000001,"5min":0.85899999999999999,"15min":0.70499999999999996},"max":{"1min":1.9339999999999999,"5min":2.141,"15min":2.6179999999999999},"last":1.0700000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4530000000000001,"15min":1.4119999999999999},"min":{"1min":0.83299999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.1179999999999999,"5min":2.274,"15min":2.7000000000000002},"last":1.8879999999999999},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.49,"15min":1.48},"min":{"1min":1.159,"5min":0.88100000000000001,"15min":0.76400000000000001},"max":{"1min":2.137,"5min":2.2509999999999999,"15min":2.895},"last":1.161}]},{"osd":7,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4350000000000001,"15min":1.4419999999999999},"min":{"1min":1.0349999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.835,"5min":2.1629999999999998,"15min":9.4830000000000005},"last":1.351},{"interface":"front","average":{"1min":1.554,"5min":1.554,"15min":1.542},"min":{"1min":0.91500000000000004,"5min":0.80800000000000005,"15min":0.71899999999999997},"max":{"1min":2.0720000000000001,"5min":2.3420000000000001,"15min":9.4109999999999996},"last":1.823}]}]},{"osd":0,"up_from":13,"seq":55834575011,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1105,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194206,"statfs":{"total":200000000,"available":198867629,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":0.86599999999999999,"5min":0.875,"15min":0.91300000000000003},"min":{"1min":0.49099999999999999,"5min":0.311,"15min":0.311},"max":{"1min":1.393,"5min":1.671,"15min":2.3199999999999998},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.1180000000000001,"5min":1.0629999999999999,"15min":1.0669999999999999},"min":{"1min":0.59299999999999997,"5min":0.438,"15min":0.42799999999999999},"max":{"1min":1.6299999999999999,"5min":2.2170000000000001,"15min":2.4180000000000001},"last":1.2330000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":1.012,"5min":1,"15min":0.98899999999999999},"min":{"1min":0.53100000000000003,"5min":0.434,"15min":0.434},"max":{"1min":1.839,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0049999999999999,"15min":0.98199999999999998},"min":{"1min":0.54500000000000004,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.5309999999999999,"5min":2.3340000000000001,"15min":2.7810000000000001},"last":1.419}]},{"osd":3,"last update":"Tue Dec 19 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.0129999999999999,"15min":1.0269999999999999},"min":{"1min":0.58899999999999997,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.6910000000000001,"5min":1.8200000000000001,"15min":2.5190000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.2869999999999999,"15min":1.27},"min":{"1min":0.65100000000000002,"5min":0.53000000000000003,"15min":0.47499999999999998},"max":{"1min":1.804,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.345}]},{"osd":4,"last update":"Tue Dec 19 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":1.286,"5min":1.282,"15min":1.2629999999999999},"min":{"1min":0.90800000000000003,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.8580000000000001,"5min":2.1920000000000002,"15min":2.5529999999999999},"last":1.504},{"interface":"front","average":{"1min":1.25,"5min":1.2430000000000001,"15min":1.2809999999999999},"min":{"1min":0.85299999999999998,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.851,"5min":2.056,"15min":2.9529999999999998},"last":1.2789999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":1.343,"5min":1.3109999999999999,"15min":1.393},"min":{"1min":0.91600000000000004,"5min":0.68500000000000005,"15min":0.64500000000000002},"max":{"1min":1.8480000000000001,"5min":1.97,"15min":16.298999999999999},"last":1.4410000000000001},{"interface":"front","average":{"1min":1.3979999999999999,"5min":1.3360000000000001,"15min":1.3240000000000001},"min":{"1min":1.0069999999999999,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.7030000000000001,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.464}]},{"osd":6,"last update":"Tue Dec 19 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.4490000000000001,"15min":1.4379999999999999},"min":{"1min":0.84599999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.929,"5min":2.4169999999999998,"15min":2.5910000000000002},"last":1.5700000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.4299999999999999,"15min":1.409},"min":{"1min":0.83999999999999997,"5min":0.745,"15min":0.745},"max":{"1min":1.7270000000000001,"5min":2.1709999999999998,"15min":2.46},"last":1.6679999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:27:11 2023","interfaces":[{"interface":"back","average":{"1min":1.409,"5min":1.4339999999999999,"15min":1.4430000000000001},"min":{"1min":0.89400000000000002,"5min":0.72099999999999997,"15min":0.70899999999999996},"max":{"1min":1.796,"5min":2.2549999999999999,"15min":2.4969999999999999},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.397,"5min":1.381,"15min":1.4179999999999999},"min":{"1min":0.93000000000000005,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.794,"5min":1.9370000000000001,"15min":3.1429999999999998},"last":1.3859999999999999}]}]}],"pool_statfs":[]}} 2023-12-19T18:28:13.620 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:28:13.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.863+0000 7f6579320640 1 Processor -- start 2023-12-19T18:28:13.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.863+0000 7f6579320640 1 -- start start 2023-12-19T18:28:13.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.865+0000 7f6579320640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6574159c40 0x7f657406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:13.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.865+0000 7f6579320640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f657406c880 0x7f657406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:13.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.865+0000 7f6579320640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f657415a020 0x7f65740cd800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:13.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.865+0000 7f6579320640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f657415b180 con 0x7f657406c880 2023-12-19T18:28:13.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.865+0000 7f6579320640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65740ca850 con 0x7f6574159c40 2023-12-19T18:28:13.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.865+0000 7f6579320640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f657415a570 con 0x7f657415a020 2023-12-19T18:28:13.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.865+0000 7f65727fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f657406c880 0x7f657406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:13.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.865+0000 7f65727fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f657406c880 0x7f657406ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42260/0 (socket says 172.21.15.27:42260) 2023-12-19T18:28:13.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.865+0000 7f65727fc640 1 -- 172.21.15.27:0/1240410973 learned_addr learned my addr 172.21.15.27:0/1240410973 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:13.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.866+0000 7f65737fe640 1 --2- 172.21.15.27:0/1240410973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f657415a020 0x7f65740cd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:13.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.866+0000 7f65727fc640 1 -- 172.21.15.27:0/1240410973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f657415a020 msgr2=0x7f65740cd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:13.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.866+0000 7f6572ffd640 1 --2- 172.21.15.27:0/1240410973 >> v2:172.21.15.192:3300/0 conn(0x7f6574159c40 0x7f657406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:13.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.866+0000 7f65727fc640 1 --2- 172.21.15.27:0/1240410973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f657415a020 0x7f65740cd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.866+0000 7f65727fc640 1 -- 172.21.15.27:0/1240410973 >> v2:172.21.15.192:3300/0 conn(0x7f6574159c40 msgr2=0x7f657406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:13.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.866+0000 7f65727fc640 1 --2- 172.21.15.27:0/1240410973 >> v2:172.21.15.192:3300/0 conn(0x7f6574159c40 0x7f657406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.866+0000 7f65727fc640 1 -- 172.21.15.27:0/1240410973 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f657415a850 con 0x7f657406c880 2023-12-19T18:28:13.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.866+0000 7f6572ffd640 1 --2- 172.21.15.27:0/1240410973 >> v2:172.21.15.192:3300/0 conn(0x7f6574159c40 0x7f657406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:13.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.866+0000 7f65737fe640 1 --2- 172.21.15.27:0/1240410973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f657415a020 0x7f65740cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:13.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.867+0000 7f65727fc640 1 --2- 172.21.15.27:0/1240410973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f657406c880 0x7f657406ba20 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f657406d6e0 tx=0x7f655c02e0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31917045407ebabc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:13.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.867+0000 7f6571ffb640 1 -- 172.21.15.27:0/1240410973 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c02ee80 con 0x7f657406c880 2023-12-19T18:28:13.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.867+0000 7f6571ffb640 1 -- 172.21.15.27:0/1240410973 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c036440 con 0x7f657406c880 2023-12-19T18:28:13.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.867+0000 7f6571ffb640 1 -- 172.21.15.27:0/1240410973 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c0367b0 con 0x7f657406c880 2023-12-19T18:28:13.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.868+0000 7f6579320640 1 -- 172.21.15.27:0/1240410973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f657406c880 msgr2=0x7f657406ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:13.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.868+0000 7f6579320640 1 --2- 172.21.15.27:0/1240410973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f657406c880 0x7f657406ba20 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7f657406d6e0 tx=0x7f655c02e0d0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.868+0000 7f6579320640 1 -- 172.21.15.27:0/1240410973 shutdown_connections 2023-12-19T18:28:13.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.868+0000 7f6579320640 1 --2- 172.21.15.27:0/1240410973 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f657415a020 0x7f65740cd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.868+0000 7f6579320640 1 --2- 172.21.15.27:0/1240410973 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f657406c880 0x7f657406ba20 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.868+0000 7f6579320640 1 --2- 172.21.15.27:0/1240410973 >> v2:172.21.15.192:3300/0 conn(0x7f6574159c40 0x7f657406b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.868+0000 7f6579320640 1 -- 172.21.15.27:0/1240410973 >> 172.21.15.27:0/1240410973 conn(0x7f657406a330 msgr2=0x7f65740b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:13.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.868+0000 7f6579320640 1 -- 172.21.15.27:0/1240410973 shutdown_connections 2023-12-19T18:28:13.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.868+0000 7f6579320640 1 -- 172.21.15.27:0/1240410973 wait complete. 2023-12-19T18:28:13.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.871+0000 7f6579320640 1 Processor -- start 2023-12-19T18:28:13.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.871+0000 7f6579320640 1 -- start start 2023-12-19T18:28:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.872+0000 7f6579320640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f657406c880 0x7f65740c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.872+0000 7f6579320640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6574159c40 0x7f65740cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.872+0000 7f6579320640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f657415a020 0x7f65741585e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.872+0000 7f6579320640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65740ccb40 con 0x7f6574159c40 2023-12-19T18:28:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.872+0000 7f6579320640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65740cbf30 con 0x7f657415a020 2023-12-19T18:28:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.872+0000 7f6579320640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65740cc210 con 0x7f657406c880 2023-12-19T18:28:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.873+0000 7f65727fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6574159c40 0x7f65740cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.873+0000 7f65727fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6574159c40 0x7f65740cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42270/0 (socket says 172.21.15.27:42270) 2023-12-19T18:28:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.873+0000 7f65727fc640 1 -- 172.21.15.27:0/327092555 learned_addr learned my addr 172.21.15.27:0/327092555 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:13.873 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.873+0000 7f65727fc640 1 -- 172.21.15.27:0/327092555 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f657406c880 msgr2=0x7f65740c79b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:13.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.873+0000 7f65727fc640 1 --2- 172.21.15.27:0/327092555 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f657406c880 0x7f65740c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.873+0000 7f65727fc640 1 -- 172.21.15.27:0/327092555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f657415a020 msgr2=0x7f65741585e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:13.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.873+0000 7f65727fc640 1 --2- 172.21.15.27:0/327092555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f657415a020 0x7f65741585e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:13.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.873+0000 7f65727fc640 1 -- 172.21.15.27:0/327092555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c00b770 con 0x7f6574159c40 2023-12-19T18:28:13.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.873+0000 7f65727fc640 1 --2- 172.21.15.27:0/327092555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6574159c40 0x7f65740cb5c0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f655c02e110 tx=0x7f655c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:13.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.874+0000 7f656b7fe640 1 -- 172.21.15.27:0/327092555 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c034b80 con 0x7f6574159c40 2023-12-19T18:28:13.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.874+0000 7f656b7fe640 1 -- 172.21.15.27:0/327092555 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655c036e90 con 0x7f6574159c40 2023-12-19T18:28:13.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.874+0000 7f656b7fe640 1 -- 172.21.15.27:0/327092555 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655c041460 con 0x7f6574159c40 2023-12-19T18:28:13.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.874+0000 7f6579320640 1 -- 172.21.15.27:0/327092555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65740cc4f0 con 0x7f6574159c40 2023-12-19T18:28:13.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.874+0000 7f6579320640 1 -- 172.21.15.27:0/327092555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65740c7ef0 con 0x7f6574159c40 2023-12-19T18:28:13.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.875+0000 7f6579320640 1 -- 172.21.15.27:0/327092555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65740c8080 con 0x7f6574159c40 2023-12-19T18:28:13.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.876+0000 7f656b7fe640 1 -- 172.21.15.27:0/327092555 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f655c049360 con 0x7f6574159c40 2023-12-19T18:28:13.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.877+0000 7f656b7fe640 1 --2- 172.21.15.27:0/327092555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f655406a430 0x7f655406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:13.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.877+0000 7f656b7fe640 1 -- 172.21.15.27:0/327092555 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f655c0aadf0 con 0x7f6574159c40 2023-12-19T18:28:13.877 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.878+0000 7f6572ffd640 1 --2- 172.21.15.27:0/327092555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f655406a430 0x7f655406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.879+0000 7f6572ffd640 1 --2- 172.21.15.27:0/327092555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f655406a430 0x7f655406c8f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6564006fd0 tx=0x7f6564008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:13.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:13.883+0000 7f656b7fe640 1 -- 172.21.15.27:0/327092555 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f655c0763e0 con 0x7f6574159c40 2023-12-19T18:28:14.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.080+0000 7f6579320640 1 -- 172.21.15.27:0/327092555 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f657415ae10 con 0x7f655406a430 2023-12-19T18:28:14.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.102+0000 7f656b7fe640 1 -- 172.21.15.27:0/327092555 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+282251 (secure 0 0 0) 0x7f656400ae40 con 0x7f655406a430 2023-12-19T18:28:14.109 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:14.111 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:28:14.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.110+0000 7f6579320640 1 -- 172.21.15.27:0/327092555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f655406a430 msgr2=0x7f655406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:14.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.110+0000 7f6579320640 1 --2- 172.21.15.27:0/327092555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f655406a430 0x7f655406c8f0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6564006fd0 tx=0x7f6564008040 comp rx=0 tx=0).stop 2023-12-19T18:28:14.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.110+0000 7f6579320640 1 -- 172.21.15.27:0/327092555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6574159c40 msgr2=0x7f65740cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:14.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.110+0000 7f6579320640 1 --2- 172.21.15.27:0/327092555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6574159c40 0x7f65740cb5c0 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f655c02e110 tx=0x7f655c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.110+0000 7f6579320640 1 -- 172.21.15.27:0/327092555 shutdown_connections 2023-12-19T18:28:14.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.110+0000 7f6579320640 1 --2- 172.21.15.27:0/327092555 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f655406a430 0x7f655406c8f0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.110+0000 7f6579320640 1 --2- 172.21.15.27:0/327092555 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f657415a020 0x7f65741585e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.110+0000 7f6579320640 1 --2- 172.21.15.27:0/327092555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6574159c40 0x7f65740cb5c0 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.110+0000 7f6579320640 1 --2- 172.21.15.27:0/327092555 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f657406c880 0x7f65740c79b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.110+0000 7f6579320640 1 -- 172.21.15.27:0/327092555 >> 172.21.15.27:0/327092555 conn(0x7f657406a330 msgr2=0x7f65740ae5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:14.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.110+0000 7f6579320640 1 -- 172.21.15.27:0/327092555 shutdown_connections 2023-12-19T18:28:14.113 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.110+0000 7f6579320640 1 -- 172.21.15.27:0/327092555 wait complete. 2023-12-19T18:28:14.134 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":599,"stamp":"2023-12-19T18:28:14.000131+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10106,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552393,"statfs":{"total":1600000000,"available":1589650972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001770"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816648+0000","last_change":"2023-12-19T18:28:08.597014+0000","last_active":"2023-12-19T18:28:08.816648+0000","last_peered":"2023-12-19T18:28:08.816648+0000","last_clean":"2023-12-19T18:28:08.816648+0000","last_became_active":"2023-12-19T18:28:08.596776+0000","last_became_peered":"2023-12-19T18:28:08.596776+0000","last_unstale":"2023-12-19T18:28:08.816648+0000","last_undegraded":"2023-12-19T18:28:08.816648+0000","last_fullsized":"2023-12-19T18:28:08.816648+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:49:02.216601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816592+0000","last_change":"2023-12-19T18:28:08.594909+0000","last_active":"2023-12-19T18:28:08.816592+0000","last_peered":"2023-12-19T18:28:08.816592+0000","last_clean":"2023-12-19T18:28:08.816592+0000","last_became_active":"2023-12-19T18:28:08.594690+0000","last_became_peered":"2023-12-19T18:28:08.594690+0000","last_unstale":"2023-12-19T18:28:08.816592+0000","last_undegraded":"2023-12-19T18:28:08.816592+0000","last_fullsized":"2023-12-19T18:28:08.816592+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:49:35.513007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816432+0000","last_change":"2023-12-19T18:28:08.604103+0000","last_active":"2023-12-19T18:28:08.816432+0000","last_peered":"2023-12-19T18:28:08.816432+0000","last_clean":"2023-12-19T18:28:08.816432+0000","last_became_active":"2023-12-19T18:28:08.603423+0000","last_became_peered":"2023-12-19T18:28:08.603423+0000","last_unstale":"2023-12-19T18:28:08.816432+0000","last_undegraded":"2023-12-19T18:28:08.816432+0000","last_fullsized":"2023-12-19T18:28:08.816432+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:09.966919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.824104+0000","last_change":"2023-12-19T18:28:08.612975+0000","last_active":"2023-12-19T18:28:08.824104+0000","last_peered":"2023-12-19T18:28:08.824104+0000","last_clean":"2023-12-19T18:28:08.824104+0000","last_became_active":"2023-12-19T18:28:08.611480+0000","last_became_peered":"2023-12-19T18:28:08.611480+0000","last_unstale":"2023-12-19T18:28:08.824104+0000","last_undegraded":"2023-12-19T18:28:08.824104+0000","last_fullsized":"2023-12-19T18:28:08.824104+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:39:02.299966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807464+0000","last_change":"2023-12-19T18:28:08.591948+0000","last_active":"2023-12-19T18:28:08.807464+0000","last_peered":"2023-12-19T18:28:08.807464+0000","last_clean":"2023-12-19T18:28:08.807464+0000","last_became_active":"2023-12-19T18:28:08.591724+0000","last_became_peered":"2023-12-19T18:28:08.591724+0000","last_unstale":"2023-12-19T18:28:08.807464+0000","last_undegraded":"2023-12-19T18:28:08.807464+0000","last_fullsized":"2023-12-19T18:28:08.807464+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:18:47.517627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.817591+0000","last_change":"2023-12-19T18:28:08.618752+0000","last_active":"2023-12-19T18:28:08.817591+0000","last_peered":"2023-12-19T18:28:08.817591+0000","last_clean":"2023-12-19T18:28:08.817591+0000","last_became_active":"2023-12-19T18:28:08.618519+0000","last_became_peered":"2023-12-19T18:28:08.618519+0000","last_unstale":"2023-12-19T18:28:08.817591+0000","last_undegraded":"2023-12-19T18:28:08.817591+0000","last_fullsized":"2023-12-19T18:28:08.817591+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:16:51.265478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.823554+0000","last_change":"2023-12-19T18:28:08.609823+0000","last_active":"2023-12-19T18:28:08.823554+0000","last_peered":"2023-12-19T18:28:08.823554+0000","last_clean":"2023-12-19T18:28:08.823554+0000","last_became_active":"2023-12-19T18:28:08.608182+0000","last_became_peered":"2023-12-19T18:28:08.608182+0000","last_unstale":"2023-12-19T18:28:08.823554+0000","last_undegraded":"2023-12-19T18:28:08.823554+0000","last_fullsized":"2023-12-19T18:28:08.823554+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:18:16.064659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.808375+0000","last_change":"2023-12-19T18:28:08.594783+0000","last_active":"2023-12-19T18:28:08.808375+0000","last_peered":"2023-12-19T18:28:08.808375+0000","last_clean":"2023-12-19T18:28:08.808375+0000","last_became_active":"2023-12-19T18:28:08.594433+0000","last_became_peered":"2023-12-19T18:28:08.594433+0000","last_unstale":"2023-12-19T18:28:08.808375+0000","last_undegraded":"2023-12-19T18:28:08.808375+0000","last_fullsized":"2023-12-19T18:28:08.808375+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:40:04.871763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.808291+0000","last_change":"2023-12-19T18:28:08.606083+0000","last_active":"2023-12-19T18:28:08.808291+0000","last_peered":"2023-12-19T18:28:08.808291+0000","last_clean":"2023-12-19T18:28:08.808291+0000","last_became_active":"2023-12-19T18:28:08.605529+0000","last_became_peered":"2023-12-19T18:28:08.605529+0000","last_unstale":"2023-12-19T18:28:08.808291+0000","last_undegraded":"2023-12-19T18:28:08.808291+0000","last_fullsized":"2023-12-19T18:28:08.808291+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:26:58.564690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.808420+0000","last_change":"2023-12-19T18:28:08.612552+0000","last_active":"2023-12-19T18:28:08.808420+0000","last_peered":"2023-12-19T18:28:08.808420+0000","last_clean":"2023-12-19T18:28:08.808420+0000","last_became_active":"2023-12-19T18:28:08.612313+0000","last_became_peered":"2023-12-19T18:28:08.612313+0000","last_unstale":"2023-12-19T18:28:08.808420+0000","last_undegraded":"2023-12-19T18:28:08.808420+0000","last_fullsized":"2023-12-19T18:28:08.808420+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:34:00.405702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807440+0000","last_change":"2023-12-19T18:28:08.606559+0000","last_active":"2023-12-19T18:28:08.807440+0000","last_peered":"2023-12-19T18:28:08.807440+0000","last_clean":"2023-12-19T18:28:08.807440+0000","last_became_active":"2023-12-19T18:28:08.603968+0000","last_became_peered":"2023-12-19T18:28:08.603968+0000","last_unstale":"2023-12-19T18:28:08.807440+0000","last_undegraded":"2023-12-19T18:28:08.807440+0000","last_fullsized":"2023-12-19T18:28:08.807440+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:37:21.745636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.822254+0000","last_change":"2023-12-19T18:28:08.611523+0000","last_active":"2023-12-19T18:28:08.822254+0000","last_peered":"2023-12-19T18:28:08.822254+0000","last_clean":"2023-12-19T18:28:08.822254+0000","last_became_active":"2023-12-19T18:28:08.611299+0000","last_became_peered":"2023-12-19T18:28:08.611299+0000","last_unstale":"2023-12-19T18:28:08.822254+0000","last_undegraded":"2023-12-19T18:28:08.822254+0000","last_fullsized":"2023-12-19T18:28:08.822254+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:21:08.357423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.822235+0000","last_change":"2023-12-19T18:28:08.602608+0000","last_active":"2023-12-19T18:28:08.822235+0000","last_peered":"2023-12-19T18:28:08.822235+0000","last_clean":"2023-12-19T18:28:08.822235+0000","last_became_active":"2023-12-19T18:28:08.601919+0000","last_became_peered":"2023-12-19T18:28:08.601919+0000","last_unstale":"2023-12-19T18:28:08.822235+0000","last_undegraded":"2023-12-19T18:28:08.822235+0000","last_fullsized":"2023-12-19T18:28:08.822235+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:02:10.193717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.808322+0000","last_change":"2023-12-19T18:28:08.596733+0000","last_active":"2023-12-19T18:28:08.808322+0000","last_peered":"2023-12-19T18:28:08.808322+0000","last_clean":"2023-12-19T18:28:08.808322+0000","last_became_active":"2023-12-19T18:28:08.596386+0000","last_became_peered":"2023-12-19T18:28:08.596386+0000","last_unstale":"2023-12-19T18:28:08.808322+0000","last_undegraded":"2023-12-19T18:28:08.808322+0000","last_fullsized":"2023-12-19T18:28:08.808322+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:16:51.265478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.808440+0000","last_change":"2023-12-19T18:28:08.604577+0000","last_active":"2023-12-19T18:28:08.808440+0000","last_peered":"2023-12-19T18:28:08.808440+0000","last_clean":"2023-12-19T18:28:08.808440+0000","last_became_active":"2023-12-19T18:28:08.604207+0000","last_became_peered":"2023-12-19T18:28:08.604207+0000","last_unstale":"2023-12-19T18:28:08.808440+0000","last_undegraded":"2023-12-19T18:28:08.808440+0000","last_fullsized":"2023-12-19T18:28:08.808440+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:46.937478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.823618+0000","last_change":"2023-12-19T18:28:08.608679+0000","last_active":"2023-12-19T18:28:08.823618+0000","last_peered":"2023-12-19T18:28:08.823618+0000","last_clean":"2023-12-19T18:28:08.823618+0000","last_became_active":"2023-12-19T18:28:08.608086+0000","last_became_peered":"2023-12-19T18:28:08.608086+0000","last_unstale":"2023-12-19T18:28:08.823618+0000","last_undegraded":"2023-12-19T18:28:08.823618+0000","last_fullsized":"2023-12-19T18:28:08.823618+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:15:16.292212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.817504+0000","last_change":"2023-12-19T18:28:08.615747+0000","last_active":"2023-12-19T18:28:08.817504+0000","last_peered":"2023-12-19T18:28:08.817504+0000","last_clean":"2023-12-19T18:28:08.817504+0000","last_became_active":"2023-12-19T18:28:08.614903+0000","last_became_peered":"2023-12-19T18:28:08.614903+0000","last_unstale":"2023-12-19T18:28:08.817504+0000","last_undegraded":"2023-12-19T18:28:08.817504+0000","last_fullsized":"2023-12-19T18:28:08.817504+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:07.478205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.824225+0000","last_change":"2023-12-19T18:28:08.611701+0000","last_active":"2023-12-19T18:28:08.824225+0000","last_peered":"2023-12-19T18:28:08.824225+0000","last_clean":"2023-12-19T18:28:08.824225+0000","last_became_active":"2023-12-19T18:28:08.610115+0000","last_became_peered":"2023-12-19T18:28:08.610115+0000","last_unstale":"2023-12-19T18:28:08.824225+0000","last_undegraded":"2023-12-19T18:28:08.824225+0000","last_fullsized":"2023-12-19T18:28:08.824225+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:40:41.811992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.823489+0000","last_change":"2023-12-19T18:28:08.602277+0000","last_active":"2023-12-19T18:28:08.823489+0000","last_peered":"2023-12-19T18:28:08.823489+0000","last_clean":"2023-12-19T18:28:08.823489+0000","last_became_active":"2023-12-19T18:28:08.601898+0000","last_became_peered":"2023-12-19T18:28:08.601898+0000","last_unstale":"2023-12-19T18:28:08.823489+0000","last_undegraded":"2023-12-19T18:28:08.823489+0000","last_fullsized":"2023-12-19T18:28:08.823489+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:40:04.871763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.822289+0000","last_change":"2023-12-19T18:28:08.591488+0000","last_active":"2023-12-19T18:28:08.822289+0000","last_peered":"2023-12-19T18:28:08.822289+0000","last_clean":"2023-12-19T18:28:08.822289+0000","last_became_active":"2023-12-19T18:28:08.591233+0000","last_became_peered":"2023-12-19T18:28:08.591233+0000","last_unstale":"2023-12-19T18:28:08.822289+0000","last_undegraded":"2023-12-19T18:28:08.822289+0000","last_fullsized":"2023-12-19T18:28:08.822289+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:09:29.241920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.817631+0000","last_change":"2023-12-19T18:28:08.616849+0000","last_active":"2023-12-19T18:28:08.817631+0000","last_peered":"2023-12-19T18:28:08.817631+0000","last_clean":"2023-12-19T18:28:08.817631+0000","last_became_active":"2023-12-19T18:28:08.615946+0000","last_became_peered":"2023-12-19T18:28:08.615946+0000","last_unstale":"2023-12-19T18:28:08.817631+0000","last_undegraded":"2023-12-19T18:28:08.817631+0000","last_fullsized":"2023-12-19T18:28:08.817631+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:16.483526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807398+0000","last_change":"2023-12-19T18:28:08.596127+0000","last_active":"2023-12-19T18:28:08.807398+0000","last_peered":"2023-12-19T18:28:08.807398+0000","last_clean":"2023-12-19T18:28:08.807398+0000","last_became_active":"2023-12-19T18:28:08.595904+0000","last_became_peered":"2023-12-19T18:28:08.595904+0000","last_unstale":"2023-12-19T18:28:08.807398+0000","last_undegraded":"2023-12-19T18:28:08.807398+0000","last_fullsized":"2023-12-19T18:28:08.807398+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:58:37.185535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.806815+0000","last_change":"2023-12-19T18:28:08.612477+0000","last_active":"2023-12-19T18:28:08.806815+0000","last_peered":"2023-12-19T18:28:08.806815+0000","last_clean":"2023-12-19T18:28:08.806815+0000","last_became_active":"2023-12-19T18:28:08.610162+0000","last_became_peered":"2023-12-19T18:28:08.610162+0000","last_unstale":"2023-12-19T18:28:08.806815+0000","last_undegraded":"2023-12-19T18:28:08.806815+0000","last_fullsized":"2023-12-19T18:28:08.806815+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:46.937478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807343+0000","last_change":"2023-12-19T18:28:08.596231+0000","last_active":"2023-12-19T18:28:08.807343+0000","last_peered":"2023-12-19T18:28:08.807343+0000","last_clean":"2023-12-19T18:28:08.807343+0000","last_became_active":"2023-12-19T18:28:08.595895+0000","last_became_peered":"2023-12-19T18:28:08.595895+0000","last_unstale":"2023-12-19T18:28:08.807343+0000","last_undegraded":"2023-12-19T18:28:08.807343+0000","last_fullsized":"2023-12-19T18:28:08.807343+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:34:12.524415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807473+0000","last_change":"2023-12-19T18:28:08.600509+0000","last_active":"2023-12-19T18:28:08.807473+0000","last_peered":"2023-12-19T18:28:08.807473+0000","last_clean":"2023-12-19T18:28:08.807473+0000","last_became_active":"2023-12-19T18:28:08.599822+0000","last_became_peered":"2023-12-19T18:28:08.599822+0000","last_unstale":"2023-12-19T18:28:08.807473+0000","last_undegraded":"2023-12-19T18:28:08.807473+0000","last_fullsized":"2023-12-19T18:28:08.807473+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:01:58.890674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.806972+0000","last_change":"2023-12-19T18:28:08.618546+0000","last_active":"2023-12-19T18:28:08.806972+0000","last_peered":"2023-12-19T18:28:08.806972+0000","last_clean":"2023-12-19T18:28:08.806972+0000","last_became_active":"2023-12-19T18:28:08.618322+0000","last_became_peered":"2023-12-19T18:28:08.618322+0000","last_unstale":"2023-12-19T18:28:08.806972+0000","last_undegraded":"2023-12-19T18:28:08.806972+0000","last_fullsized":"2023-12-19T18:28:08.806972+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:25:46.176399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807406+0000","last_change":"2023-12-19T18:28:08.610520+0000","last_active":"2023-12-19T18:28:08.807406+0000","last_peered":"2023-12-19T18:28:08.807406+0000","last_clean":"2023-12-19T18:28:08.807406+0000","last_became_active":"2023-12-19T18:28:08.609944+0000","last_became_peered":"2023-12-19T18:28:08.609944+0000","last_unstale":"2023-12-19T18:28:08.807406+0000","last_undegraded":"2023-12-19T18:28:08.807406+0000","last_fullsized":"2023-12-19T18:28:08.807406+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:16:20.661937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.823513+0000","last_change":"2023-12-19T18:28:08.605904+0000","last_active":"2023-12-19T18:28:08.823513+0000","last_peered":"2023-12-19T18:28:08.823513+0000","last_clean":"2023-12-19T18:28:08.823513+0000","last_became_active":"2023-12-19T18:28:08.605067+0000","last_became_peered":"2023-12-19T18:28:08.605067+0000","last_unstale":"2023-12-19T18:28:08.823513+0000","last_undegraded":"2023-12-19T18:28:08.823513+0000","last_fullsized":"2023-12-19T18:28:08.823513+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:46.937478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816296+0000","last_change":"2023-12-19T18:28:08.602267+0000","last_active":"2023-12-19T18:28:08.816296+0000","last_peered":"2023-12-19T18:28:08.816296+0000","last_clean":"2023-12-19T18:28:08.816296+0000","last_became_active":"2023-12-19T18:28:08.601923+0000","last_became_peered":"2023-12-19T18:28:08.601923+0000","last_unstale":"2023-12-19T18:28:08.816296+0000","last_undegraded":"2023-12-19T18:28:08.816296+0000","last_fullsized":"2023-12-19T18:28:08.816296+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:05:15.046255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.808461+0000","last_change":"2023-12-19T18:28:08.596532+0000","last_active":"2023-12-19T18:28:08.808461+0000","last_peered":"2023-12-19T18:28:08.808461+0000","last_clean":"2023-12-19T18:28:08.808461+0000","last_became_active":"2023-12-19T18:28:08.596171+0000","last_became_peered":"2023-12-19T18:28:08.596171+0000","last_unstale":"2023-12-19T18:28:08.808461+0000","last_undegraded":"2023-12-19T18:28:08.808461+0000","last_fullsized":"2023-12-19T18:28:08.808461+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:57:07.478205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816670+0000","last_change":"2023-12-19T18:28:08.606485+0000","last_active":"2023-12-19T18:28:08.816670+0000","last_peered":"2023-12-19T18:28:08.816670+0000","last_clean":"2023-12-19T18:28:08.816670+0000","last_became_active":"2023-12-19T18:28:08.606229+0000","last_became_peered":"2023-12-19T18:28:08.606229+0000","last_unstale":"2023-12-19T18:28:08.816670+0000","last_undegraded":"2023-12-19T18:28:08.816670+0000","last_fullsized":"2023-12-19T18:28:08.816670+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:52:40.164956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.817275+0000","last_change":"2023-12-19T18:28:08.619373+0000","last_active":"2023-12-19T18:28:08.817275+0000","last_peered":"2023-12-19T18:28:08.817275+0000","last_clean":"2023-12-19T18:28:08.817275+0000","last_became_active":"2023-12-19T18:28:08.616356+0000","last_became_peered":"2023-12-19T18:28:08.616356+0000","last_unstale":"2023-12-19T18:28:08.817275+0000","last_undegraded":"2023-12-19T18:28:08.817275+0000","last_fullsized":"2023-12-19T18:28:08.817275+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:46.937478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.824134+0000","last_change":"2023-12-19T18:28:08.611866+0000","last_active":"2023-12-19T18:28:08.824134+0000","last_peered":"2023-12-19T18:28:08.824134+0000","last_clean":"2023-12-19T18:28:08.824134+0000","last_became_active":"2023-12-19T18:28:08.611103+0000","last_became_peered":"2023-12-19T18:28:08.611103+0000","last_unstale":"2023-12-19T18:28:08.824134+0000","last_undegraded":"2023-12-19T18:28:08.824134+0000","last_fullsized":"2023-12-19T18:28:08.824134+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:09.952598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"201'4","reported_seq":15,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816536+0000","last_change":"2023-12-19T18:28:08.097575+0000","last_active":"2023-12-19T18:28:08.816536+0000","last_peered":"2023-12-19T18:28:08.816536+0000","last_clean":"2023-12-19T18:28:08.816536+0000","last_became_active":"2023-12-19T18:28:08.093961+0000","last_became_peered":"2023-12-19T18:28:08.093961+0000","last_unstale":"2023-12-19T18:28:08.816536+0000","last_undegraded":"2023-12-19T18:28:08.816536+0000","last_fullsized":"2023-12-19T18:28:08.816536+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:18:15.050649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816457+0000","last_change":"2023-12-19T18:28:08.617266+0000","last_active":"2023-12-19T18:28:08.816457+0000","last_peered":"2023-12-19T18:28:08.816457+0000","last_clean":"2023-12-19T18:28:08.816457+0000","last_became_active":"2023-12-19T18:28:08.617049+0000","last_became_peered":"2023-12-19T18:28:08.617049+0000","last_unstale":"2023-12-19T18:28:08.816457+0000","last_undegraded":"2023-12-19T18:28:08.816457+0000","last_fullsized":"2023-12-19T18:28:08.816457+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:33:09.952598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"201'1","reported_seq":12,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.808348+0000","last_change":"2023-12-19T18:28:08.106602+0000","last_active":"2023-12-19T18:28:08.808348+0000","last_peered":"2023-12-19T18:28:08.808348+0000","last_clean":"2023-12-19T18:28:08.808348+0000","last_became_active":"2023-12-19T18:28:08.106367+0000","last_became_peered":"2023-12-19T18:28:08.106367+0000","last_unstale":"2023-12-19T18:28:08.808348+0000","last_undegraded":"2023-12-19T18:28:08.808348+0000","last_fullsized":"2023-12-19T18:28:08.808348+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:59.338756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.817671+0000","last_change":"2023-12-19T18:28:08.617011+0000","last_active":"2023-12-19T18:28:08.817671+0000","last_peered":"2023-12-19T18:28:08.817671+0000","last_clean":"2023-12-19T18:28:08.817671+0000","last_became_active":"2023-12-19T18:28:08.616479+0000","last_became_peered":"2023-12-19T18:28:08.616479+0000","last_unstale":"2023-12-19T18:28:08.817671+0000","last_undegraded":"2023-12-19T18:28:08.817671+0000","last_fullsized":"2023-12-19T18:28:08.817671+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:26:58.564690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"201'2","reported_seq":13,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.817754+0000","last_change":"2023-12-19T18:28:08.107314+0000","last_active":"2023-12-19T18:28:08.817754+0000","last_peered":"2023-12-19T18:28:08.817754+0000","last_clean":"2023-12-19T18:28:08.817754+0000","last_became_active":"2023-12-19T18:28:08.107070+0000","last_became_peered":"2023-12-19T18:28:08.107070+0000","last_unstale":"2023-12-19T18:28:08.817754+0000","last_undegraded":"2023-12-19T18:28:08.817754+0000","last_fullsized":"2023-12-19T18:28:08.817754+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:14:19.016691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.808482+0000","last_change":"2023-12-19T18:28:08.604444+0000","last_active":"2023-12-19T18:28:08.808482+0000","last_peered":"2023-12-19T18:28:08.808482+0000","last_clean":"2023-12-19T18:28:08.808482+0000","last_became_active":"2023-12-19T18:28:08.603881+0000","last_became_peered":"2023-12-19T18:28:08.603881+0000","last_unstale":"2023-12-19T18:28:08.808482+0000","last_undegraded":"2023-12-19T18:28:08.808482+0000","last_fullsized":"2023-12-19T18:28:08.808482+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:49.397031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"201'2","reported_seq":13,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.806995+0000","last_change":"2023-12-19T18:28:08.100518+0000","last_active":"2023-12-19T18:28:08.806995+0000","last_peered":"2023-12-19T18:28:08.806995+0000","last_clean":"2023-12-19T18:28:08.806995+0000","last_became_active":"2023-12-19T18:28:08.100001+0000","last_became_peered":"2023-12-19T18:28:08.100001+0000","last_unstale":"2023-12-19T18:28:08.806995+0000","last_undegraded":"2023-12-19T18:28:08.806995+0000","last_fullsized":"2023-12-19T18:28:08.806995+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:46.547684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.824246+0000","last_change":"2023-12-19T18:28:08.618018+0000","last_active":"2023-12-19T18:28:08.824246+0000","last_peered":"2023-12-19T18:28:08.824246+0000","last_clean":"2023-12-19T18:28:08.824246+0000","last_became_active":"2023-12-19T18:28:08.617777+0000","last_became_peered":"2023-12-19T18:28:08.617777+0000","last_unstale":"2023-12-19T18:28:08.824246+0000","last_undegraded":"2023-12-19T18:28:08.824246+0000","last_fullsized":"2023-12-19T18:28:08.824246+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:52:40.164956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"201'1","reported_seq":12,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816621+0000","last_change":"2023-12-19T18:28:08.095843+0000","last_active":"2023-12-19T18:28:08.816621+0000","last_peered":"2023-12-19T18:28:08.816621+0000","last_clean":"2023-12-19T18:28:08.816621+0000","last_became_active":"2023-12-19T18:28:08.095618+0000","last_became_peered":"2023-12-19T18:28:08.095618+0000","last_unstale":"2023-12-19T18:28:08.816621+0000","last_undegraded":"2023-12-19T18:28:08.816621+0000","last_fullsized":"2023-12-19T18:28:08.816621+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:48.383020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.817378+0000","last_change":"2023-12-19T18:28:08.619560+0000","last_active":"2023-12-19T18:28:08.817378+0000","last_peered":"2023-12-19T18:28:08.817378+0000","last_clean":"2023-12-19T18:28:08.817378+0000","last_became_active":"2023-12-19T18:28:08.616888+0000","last_became_peered":"2023-12-19T18:28:08.616888+0000","last_unstale":"2023-12-19T18:28:08.817378+0000","last_undegraded":"2023-12-19T18:28:08.817378+0000","last_fullsized":"2023-12-19T18:28:08.817378+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:00:49.397031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"201'1","reported_seq":12,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.824270+0000","last_change":"2023-12-19T18:28:08.099799+0000","last_active":"2023-12-19T18:28:08.824270+0000","last_peered":"2023-12-19T18:28:08.824270+0000","last_clean":"2023-12-19T18:28:08.824270+0000","last_became_active":"2023-12-19T18:28:08.099342+0000","last_became_peered":"2023-12-19T18:28:08.099342+0000","last_unstale":"2023-12-19T18:28:08.824270+0000","last_undegraded":"2023-12-19T18:28:08.824270+0000","last_fullsized":"2023-12-19T18:28:08.824270+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:56:56.287166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807018+0000","last_change":"2023-12-19T18:28:08.603857+0000","last_active":"2023-12-19T18:28:08.807018+0000","last_peered":"2023-12-19T18:28:08.807018+0000","last_clean":"2023-12-19T18:28:08.807018+0000","last_became_active":"2023-12-19T18:28:08.601114+0000","last_became_peered":"2023-12-19T18:28:08.601114+0000","last_unstale":"2023-12-19T18:28:08.807018+0000","last_undegraded":"2023-12-19T18:28:08.807018+0000","last_fullsized":"2023-12-19T18:28:08.807018+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:10:03.084333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816409+0000","last_change":"2023-12-19T18:28:08.104571+0000","last_active":"2023-12-19T18:28:08.816409+0000","last_peered":"2023-12-19T18:28:08.816409+0000","last_clean":"2023-12-19T18:28:08.816409+0000","last_became_active":"2023-12-19T18:28:08.103955+0000","last_became_peered":"2023-12-19T18:28:08.103955+0000","last_unstale":"2023-12-19T18:28:08.816409+0000","last_undegraded":"2023-12-19T18:28:08.816409+0000","last_fullsized":"2023-12-19T18:28:08.816409+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:10:02.070323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.824157+0000","last_change":"2023-12-19T18:28:08.611730+0000","last_active":"2023-12-19T18:28:08.824157+0000","last_peered":"2023-12-19T18:28:08.824157+0000","last_clean":"2023-12-19T18:28:08.824157+0000","last_became_active":"2023-12-19T18:28:08.610727+0000","last_became_peered":"2023-12-19T18:28:08.610727+0000","last_unstale":"2023-12-19T18:28:08.824157+0000","last_undegraded":"2023-12-19T18:28:08.824157+0000","last_fullsized":"2023-12-19T18:28:08.824157+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:10.641773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807062+0000","last_change":"2023-12-19T18:28:08.103408+0000","last_active":"2023-12-19T18:28:08.807062+0000","last_peered":"2023-12-19T18:28:08.807062+0000","last_clean":"2023-12-19T18:28:08.807062+0000","last_became_active":"2023-12-19T18:28:08.102509+0000","last_became_peered":"2023-12-19T18:28:08.102509+0000","last_unstale":"2023-12-19T18:28:08.807062+0000","last_undegraded":"2023-12-19T18:28:08.807062+0000","last_fullsized":"2023-12-19T18:28:08.807062+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:12:42.480638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.823639+0000","last_change":"2023-12-19T18:28:08.618940+0000","last_active":"2023-12-19T18:28:08.823639+0000","last_peered":"2023-12-19T18:28:08.823639+0000","last_clean":"2023-12-19T18:28:08.823639+0000","last_became_active":"2023-12-19T18:28:08.618233+0000","last_became_peered":"2023-12-19T18:28:08.618233+0000","last_unstale":"2023-12-19T18:28:08.823639+0000","last_undegraded":"2023-12-19T18:28:08.823639+0000","last_fullsized":"2023-12-19T18:28:08.823639+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:07:31.276114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.817464+0000","last_change":"2023-12-19T18:28:08.108723+0000","last_active":"2023-12-19T18:28:08.817464+0000","last_peered":"2023-12-19T18:28:08.817464+0000","last_clean":"2023-12-19T18:28:08.817464+0000","last_became_active":"2023-12-19T18:28:08.108057+0000","last_became_peered":"2023-12-19T18:28:08.108057+0000","last_unstale":"2023-12-19T18:28:08.817464+0000","last_undegraded":"2023-12-19T18:28:08.817464+0000","last_fullsized":"2023-12-19T18:28:08.817464+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:28:59.338756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.823438+0000","last_change":"2023-12-19T18:28:08.613688+0000","last_active":"2023-12-19T18:28:08.823438+0000","last_peered":"2023-12-19T18:28:08.823438+0000","last_clean":"2023-12-19T18:28:08.823438+0000","last_became_active":"2023-12-19T18:28:08.613455+0000","last_became_peered":"2023-12-19T18:28:08.613455+0000","last_unstale":"2023-12-19T18:28:08.823438+0000","last_undegraded":"2023-12-19T18:28:08.823438+0000","last_fullsized":"2023-12-19T18:28:08.823438+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:53.893910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"201'3","reported_seq":14,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.806895+0000","last_change":"2023-12-19T18:28:08.099588+0000","last_active":"2023-12-19T18:28:08.806895+0000","last_peered":"2023-12-19T18:28:08.806895+0000","last_clean":"2023-12-19T18:28:08.806895+0000","last_became_active":"2023-12-19T18:28:08.098664+0000","last_became_peered":"2023-12-19T18:28:08.098664+0000","last_unstale":"2023-12-19T18:28:08.806895+0000","last_undegraded":"2023-12-19T18:28:08.806895+0000","last_fullsized":"2023-12-19T18:28:08.806895+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:21.999153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816484+0000","last_change":"2023-12-19T18:28:08.603937+0000","last_active":"2023-12-19T18:28:08.816484+0000","last_peered":"2023-12-19T18:28:08.816484+0000","last_clean":"2023-12-19T18:28:08.816484+0000","last_became_active":"2023-12-19T18:28:08.603112+0000","last_became_peered":"2023-12-19T18:28:08.603112+0000","last_unstale":"2023-12-19T18:28:08.816484+0000","last_undegraded":"2023-12-19T18:28:08.816484+0000","last_fullsized":"2023-12-19T18:28:08.816484+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:44:26.745474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"201'6","reported_seq":17,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807360+0000","last_change":"2023-12-19T18:28:08.106591+0000","last_active":"2023-12-19T18:28:08.807360+0000","last_peered":"2023-12-19T18:28:08.807360+0000","last_clean":"2023-12-19T18:28:08.807360+0000","last_became_active":"2023-12-19T18:28:08.106341+0000","last_became_peered":"2023-12-19T18:28:08.106341+0000","last_unstale":"2023-12-19T18:28:08.807360+0000","last_undegraded":"2023-12-19T18:28:08.807360+0000","last_fullsized":"2023-12-19T18:28:08.807360+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:59:25.726461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.823660+0000","last_change":"2023-12-19T18:28:08.608399+0000","last_active":"2023-12-19T18:28:08.823660+0000","last_peered":"2023-12-19T18:28:08.823660+0000","last_clean":"2023-12-19T18:28:08.823660+0000","last_became_active":"2023-12-19T18:28:08.607278+0000","last_became_peered":"2023-12-19T18:28:08.607278+0000","last_unstale":"2023-12-19T18:28:08.823660+0000","last_undegraded":"2023-12-19T18:28:08.823660+0000","last_fullsized":"2023-12-19T18:28:08.823660+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:56:57.301176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"201'2","reported_seq":13,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.824408+0000","last_change":"2023-12-19T18:28:08.103422+0000","last_active":"2023-12-19T18:28:08.824408+0000","last_peered":"2023-12-19T18:28:08.824408+0000","last_clean":"2023-12-19T18:28:08.824408+0000","last_became_active":"2023-12-19T18:28:08.102867+0000","last_became_peered":"2023-12-19T18:28:08.102867+0000","last_unstale":"2023-12-19T18:28:08.824408+0000","last_undegraded":"2023-12-19T18:28:08.824408+0000","last_fullsized":"2023-12-19T18:28:08.824408+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:52.879900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807040+0000","last_change":"2023-12-19T18:28:08.612258+0000","last_active":"2023-12-19T18:28:08.807040+0000","last_peered":"2023-12-19T18:28:08.807040+0000","last_clean":"2023-12-19T18:28:08.807040+0000","last_became_active":"2023-12-19T18:28:08.609755+0000","last_became_peered":"2023-12-19T18:28:08.609755+0000","last_unstale":"2023-12-19T18:28:08.807040+0000","last_undegraded":"2023-12-19T18:28:08.807040+0000","last_fullsized":"2023-12-19T18:28:08.807040+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:12:46.711265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.817711+0000","last_change":"2023-12-19T18:28:08.106401+0000","last_active":"2023-12-19T18:28:08.817711+0000","last_peered":"2023-12-19T18:28:08.817711+0000","last_clean":"2023-12-19T18:28:08.817711+0000","last_became_active":"2023-12-19T18:28:08.106146+0000","last_became_peered":"2023-12-19T18:28:08.106146+0000","last_unstale":"2023-12-19T18:28:08.817711+0000","last_undegraded":"2023-12-19T18:28:08.817711+0000","last_fullsized":"2023-12-19T18:28:08.817711+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:52:22.064538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.817223+0000","last_change":"2023-12-19T18:28:08.614542+0000","last_active":"2023-12-19T18:28:08.817223+0000","last_peered":"2023-12-19T18:28:08.817223+0000","last_clean":"2023-12-19T18:28:08.817223+0000","last_became_active":"2023-12-19T18:28:08.614314+0000","last_became_peered":"2023-12-19T18:28:08.614314+0000","last_unstale":"2023-12-19T18:28:08.817223+0000","last_undegraded":"2023-12-19T18:28:08.817223+0000","last_fullsized":"2023-12-19T18:28:08.817223+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:42:59.325633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.823574+0000","last_change":"2023-12-19T18:28:08.098900+0000","last_active":"2023-12-19T18:28:08.823574+0000","last_peered":"2023-12-19T18:28:08.823574+0000","last_clean":"2023-12-19T18:28:08.823574+0000","last_became_active":"2023-12-19T18:28:08.098436+0000","last_became_peered":"2023-12-19T18:28:08.098436+0000","last_unstale":"2023-12-19T18:28:08.823574+0000","last_undegraded":"2023-12-19T18:28:08.823574+0000","last_fullsized":"2023-12-19T18:28:08.823574+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:38:46.547684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.808398+0000","last_change":"2023-12-19T18:28:08.609733+0000","last_active":"2023-12-19T18:28:08.808398+0000","last_peered":"2023-12-19T18:28:08.808398+0000","last_clean":"2023-12-19T18:28:08.808398+0000","last_became_active":"2023-12-19T18:28:08.609337+0000","last_became_peered":"2023-12-19T18:28:08.609337+0000","last_unstale":"2023-12-19T18:28:08.808398+0000","last_undegraded":"2023-12-19T18:28:08.808398+0000","last_fullsized":"2023-12-19T18:28:08.808398+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:09:16.483526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"201'8","reported_seq":19,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807388+0000","last_change":"2023-12-19T18:28:08.107407+0000","last_active":"2023-12-19T18:28:08.807388+0000","last_peered":"2023-12-19T18:28:08.807388+0000","last_clean":"2023-12-19T18:28:08.807388+0000","last_became_active":"2023-12-19T18:28:08.107163+0000","last_became_peered":"2023-12-19T18:28:08.107163+0000","last_unstale":"2023-12-19T18:28:08.807388+0000","last_undegraded":"2023-12-19T18:28:08.807388+0000","last_fullsized":"2023-12-19T18:28:08.807388+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:44:16.886653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.823697+0000","last_change":"2023-12-19T18:28:08.617496+0000","last_active":"2023-12-19T18:28:08.823697+0000","last_peered":"2023-12-19T18:28:08.823697+0000","last_clean":"2023-12-19T18:28:08.823697+0000","last_became_active":"2023-12-19T18:28:08.617228+0000","last_became_peered":"2023-12-19T18:28:08.617228+0000","last_unstale":"2023-12-19T18:28:08.823697+0000","last_undegraded":"2023-12-19T18:28:08.823697+0000","last_fullsized":"2023-12-19T18:28:08.823697+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:12:46.711265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"201'1","reported_seq":12,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807370+0000","last_change":"2023-12-19T18:28:08.107605+0000","last_active":"2023-12-19T18:28:08.807370+0000","last_peered":"2023-12-19T18:28:08.807370+0000","last_clean":"2023-12-19T18:28:08.807370+0000","last_became_active":"2023-12-19T18:28:08.107354+0000","last_became_peered":"2023-12-19T18:28:08.107354+0000","last_unstale":"2023-12-19T18:28:08.807370+0000","last_undegraded":"2023-12-19T18:28:08.807370+0000","last_fullsized":"2023-12-19T18:28:08.807370+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:37:54.253272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.824179+0000","last_change":"2023-12-19T18:28:08.606585+0000","last_active":"2023-12-19T18:28:08.824179+0000","last_peered":"2023-12-19T18:28:08.824179+0000","last_clean":"2023-12-19T18:28:08.824179+0000","last_became_active":"2023-12-19T18:28:08.606227+0000","last_became_peered":"2023-12-19T18:28:08.606227+0000","last_unstale":"2023-12-19T18:28:08.824179+0000","last_undegraded":"2023-12-19T18:28:08.824179+0000","last_fullsized":"2023-12-19T18:28:08.824179+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:59:10.061644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"201'2","reported_seq":13,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.808505+0000","last_change":"2023-12-19T18:28:08.104589+0000","last_active":"2023-12-19T18:28:08.808505+0000","last_peered":"2023-12-19T18:28:08.808505+0000","last_clean":"2023-12-19T18:28:08.808505+0000","last_became_active":"2023-12-19T18:28:08.104328+0000","last_became_peered":"2023-12-19T18:28:08.104328+0000","last_unstale":"2023-12-19T18:28:08.808505+0000","last_undegraded":"2023-12-19T18:28:08.808505+0000","last_fullsized":"2023-12-19T18:28:08.808505+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:50:12.156824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.824341+0000","last_change":"2023-12-19T18:28:08.614095+0000","last_active":"2023-12-19T18:28:08.824341+0000","last_peered":"2023-12-19T18:28:08.824341+0000","last_clean":"2023-12-19T18:28:08.824341+0000","last_became_active":"2023-12-19T18:28:08.613709+0000","last_became_peered":"2023-12-19T18:28:08.613709+0000","last_unstale":"2023-12-19T18:28:08.824341+0000","last_undegraded":"2023-12-19T18:28:08.824341+0000","last_fullsized":"2023-12-19T18:28:08.824341+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:44:26.745474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"201'1","reported_seq":12,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807429+0000","last_change":"2023-12-19T18:28:08.097340+0000","last_active":"2023-12-19T18:28:08.807429+0000","last_peered":"2023-12-19T18:28:08.807429+0000","last_clean":"2023-12-19T18:28:08.807429+0000","last_became_active":"2023-12-19T18:28:08.096841+0000","last_became_peered":"2023-12-19T18:28:08.096841+0000","last_unstale":"2023-12-19T18:28:08.807429+0000","last_undegraded":"2023-12-19T18:28:08.807429+0000","last_fullsized":"2023-12-19T18:28:08.807429+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:39:53.529181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.806949+0000","last_change":"2023-12-19T18:28:08.614237+0000","last_active":"2023-12-19T18:28:08.806949+0000","last_peered":"2023-12-19T18:28:08.806949+0000","last_clean":"2023-12-19T18:28:08.806949+0000","last_became_active":"2023-12-19T18:28:08.613804+0000","last_became_peered":"2023-12-19T18:28:08.613804+0000","last_unstale":"2023-12-19T18:28:08.806949+0000","last_undegraded":"2023-12-19T18:28:08.806949+0000","last_fullsized":"2023-12-19T18:28:08.806949+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:18:16.064659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.822264+0000","last_change":"2023-12-19T18:28:08.104648+0000","last_active":"2023-12-19T18:28:08.822264+0000","last_peered":"2023-12-19T18:28:08.822264+0000","last_clean":"2023-12-19T18:28:08.822264+0000","last_became_active":"2023-12-19T18:28:08.104380+0000","last_became_peered":"2023-12-19T18:28:08.104380+0000","last_unstale":"2023-12-19T18:28:08.822264+0000","last_undegraded":"2023-12-19T18:28:08.822264+0000","last_fullsized":"2023-12-19T18:28:08.822264+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:29:04.592957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816329+0000","last_change":"2023-12-19T18:28:08.606432+0000","last_active":"2023-12-19T18:28:08.816329+0000","last_peered":"2023-12-19T18:28:08.816329+0000","last_clean":"2023-12-19T18:28:08.816329+0000","last_became_active":"2023-12-19T18:28:08.606199+0000","last_became_peered":"2023-12-19T18:28:08.606199+0000","last_unstale":"2023-12-19T18:28:08.816329+0000","last_undegraded":"2023-12-19T18:28:08.816329+0000","last_fullsized":"2023-12-19T18:28:08.816329+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:34:10.949301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"201'2","reported_seq":13,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.806925+0000","last_change":"2023-12-19T18:28:08.102847+0000","last_active":"2023-12-19T18:28:08.806925+0000","last_peered":"2023-12-19T18:28:08.806925+0000","last_clean":"2023-12-19T18:28:08.806925+0000","last_became_active":"2023-12-19T18:28:08.102609+0000","last_became_peered":"2023-12-19T18:28:08.102609+0000","last_unstale":"2023-12-19T18:28:08.806925+0000","last_undegraded":"2023-12-19T18:28:08.806925+0000","last_fullsized":"2023-12-19T18:28:08.806925+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:59:14.820598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.806844+0000","last_change":"2023-12-19T18:28:08.612219+0000","last_active":"2023-12-19T18:28:08.806844+0000","last_peered":"2023-12-19T18:28:08.806844+0000","last_clean":"2023-12-19T18:28:08.806844+0000","last_became_active":"2023-12-19T18:28:08.604214+0000","last_became_peered":"2023-12-19T18:28:08.604214+0000","last_unstale":"2023-12-19T18:28:08.806844+0000","last_undegraded":"2023-12-19T18:28:08.806844+0000","last_fullsized":"2023-12-19T18:28:08.806844+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:40:04.871763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.822222+0000","last_change":"2023-12-19T18:28:08.106813+0000","last_active":"2023-12-19T18:28:08.822222+0000","last_peered":"2023-12-19T18:28:08.822222+0000","last_clean":"2023-12-19T18:28:08.822222+0000","last_became_active":"2023-12-19T18:28:08.106573+0000","last_became_peered":"2023-12-19T18:28:08.106573+0000","last_unstale":"2023-12-19T18:28:08.822222+0000","last_undegraded":"2023-12-19T18:28:08.822222+0000","last_fullsized":"2023-12-19T18:28:08.822222+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:48:52.520397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816696+0000","last_change":"2023-12-19T18:28:08.599393+0000","last_active":"2023-12-19T18:28:08.816696+0000","last_peered":"2023-12-19T18:28:08.816696+0000","last_clean":"2023-12-19T18:28:08.816696+0000","last_became_active":"2023-12-19T18:28:08.599169+0000","last_became_peered":"2023-12-19T18:28:08.599169+0000","last_unstale":"2023-12-19T18:28:08.816696+0000","last_undegraded":"2023-12-19T18:28:08.816696+0000","last_fullsized":"2023-12-19T18:28:08.816696+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:00.473295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.824431+0000","last_change":"2023-12-19T18:28:08.098465+0000","last_active":"2023-12-19T18:28:08.824431+0000","last_peered":"2023-12-19T18:28:08.824431+0000","last_clean":"2023-12-19T18:28:08.824431+0000","last_became_active":"2023-12-19T18:28:08.097821+0000","last_became_peered":"2023-12-19T18:28:08.097821+0000","last_unstale":"2023-12-19T18:28:08.824431+0000","last_undegraded":"2023-12-19T18:28:08.824431+0000","last_fullsized":"2023-12-19T18:28:08.824431+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T19:07:30.262104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.823737+0000","last_change":"2023-12-19T18:28:08.607093+0000","last_active":"2023-12-19T18:28:08.823737+0000","last_peered":"2023-12-19T18:28:08.823737+0000","last_clean":"2023-12-19T18:28:08.823737+0000","last_became_active":"2023-12-19T18:28:08.606383+0000","last_became_peered":"2023-12-19T18:28:08.606383+0000","last_unstale":"2023-12-19T18:28:08.823737+0000","last_undegraded":"2023-12-19T18:28:08.823737+0000","last_fullsized":"2023-12-19T18:28:08.823737+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:25:46.176399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.822280+0000","last_change":"2023-12-19T18:28:08.103729+0000","last_active":"2023-12-19T18:28:08.822280+0000","last_peered":"2023-12-19T18:28:08.822280+0000","last_clean":"2023-12-19T18:28:08.822280+0000","last_became_active":"2023-12-19T18:28:08.103485+0000","last_became_peered":"2023-12-19T18:28:08.103485+0000","last_unstale":"2023-12-19T18:28:08.822280+0000","last_undegraded":"2023-12-19T18:28:08.822280+0000","last_fullsized":"2023-12-19T18:28:08.822280+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T02:48:33.904383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.806872+0000","last_change":"2023-12-19T18:28:08.616997+0000","last_active":"2023-12-19T18:28:08.806872+0000","last_peered":"2023-12-19T18:28:08.806872+0000","last_clean":"2023-12-19T18:28:08.806872+0000","last_became_active":"2023-12-19T18:28:08.616499+0000","last_became_peered":"2023-12-19T18:28:08.616499+0000","last_unstale":"2023-12-19T18:28:08.806872+0000","last_undegraded":"2023-12-19T18:28:08.806872+0000","last_fullsized":"2023-12-19T18:28:08.806872+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:28:17.475387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"201'2","reported_seq":13,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.817547+0000","last_change":"2023-12-19T18:28:08.108917+0000","last_active":"2023-12-19T18:28:08.817547+0000","last_peered":"2023-12-19T18:28:08.817547+0000","last_clean":"2023-12-19T18:28:08.817547+0000","last_became_active":"2023-12-19T18:28:08.108489+0000","last_became_peered":"2023-12-19T18:28:08.108489+0000","last_unstale":"2023-12-19T18:28:08.817547+0000","last_undegraded":"2023-12-19T18:28:08.817547+0000","last_fullsized":"2023-12-19T18:28:08.817547+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:29:37.313662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.824363+0000","last_change":"2023-12-19T18:28:08.603491+0000","last_active":"2023-12-19T18:28:08.824363+0000","last_peered":"2023-12-19T18:28:08.824363+0000","last_clean":"2023-12-19T18:28:08.824363+0000","last_became_active":"2023-12-19T18:28:08.603042+0000","last_became_peered":"2023-12-19T18:28:08.603042+0000","last_unstale":"2023-12-19T18:28:08.824363+0000","last_undegraded":"2023-12-19T18:28:08.824363+0000","last_fullsized":"2023-12-19T18:28:08.824363+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:18:48.940768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"202'1","reported_seq":12,"reported_epoch":202,"state":"active+clean","last_fresh":"2023-12-19T18:28:09.972223+0000","last_change":"2023-12-19T18:28:08.101563+0000","last_active":"2023-12-19T18:28:09.972223+0000","last_peered":"2023-12-19T18:28:09.972223+0000","last_clean":"2023-12-19T18:28:09.972223+0000","last_became_active":"2023-12-19T18:28:08.101320+0000","last_became_peered":"2023-12-19T18:28:08.101320+0000","last_unstale":"2023-12-19T18:28:09.972223+0000","last_undegraded":"2023-12-19T18:28:09.972223+0000","last_fullsized":"2023-12-19T18:28:09.972223+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T00:08:10.434690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.824384+0000","last_change":"2023-12-19T18:28:08.610379+0000","last_active":"2023-12-19T18:28:08.824384+0000","last_peered":"2023-12-19T18:28:08.824384+0000","last_clean":"2023-12-19T18:28:08.824384+0000","last_became_active":"2023-12-19T18:28:08.609663+0000","last_became_peered":"2023-12-19T18:28:08.609663+0000","last_unstale":"2023-12-19T18:28:08.824384+0000","last_undegraded":"2023-12-19T18:28:08.824384+0000","last_fullsized":"2023-12-19T18:28:08.824384+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:09:09.966919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816356+0000","last_change":"2023-12-19T18:28:08.093810+0000","last_active":"2023-12-19T18:28:08.816356+0000","last_peered":"2023-12-19T18:28:08.816356+0000","last_clean":"2023-12-19T18:28:08.816356+0000","last_became_active":"2023-12-19T18:28:08.093297+0000","last_became_peered":"2023-12-19T18:28:08.093297+0000","last_unstale":"2023-12-19T18:28:08.816356+0000","last_undegraded":"2023-12-19T18:28:08.816356+0000","last_fullsized":"2023-12-19T18:28:08.816356+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:40:03.857753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.824204+0000","last_change":"2023-12-19T18:28:08.604835+0000","last_active":"2023-12-19T18:28:08.824204+0000","last_peered":"2023-12-19T18:28:08.824204+0000","last_clean":"2023-12-19T18:28:08.824204+0000","last_became_active":"2023-12-19T18:28:08.604364+0000","last_became_peered":"2023-12-19T18:28:08.604364+0000","last_unstale":"2023-12-19T18:28:08.824204+0000","last_undegraded":"2023-12-19T18:28:08.824204+0000","last_fullsized":"2023-12-19T18:28:08.824204+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:08:00.473295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807114+0000","last_change":"2023-12-19T18:28:08.105010+0000","last_active":"2023-12-19T18:28:08.807114+0000","last_peered":"2023-12-19T18:28:08.807114+0000","last_clean":"2023-12-19T18:28:08.807114+0000","last_became_active":"2023-12-19T18:28:08.103878+0000","last_became_peered":"2023-12-19T18:28:08.103878+0000","last_unstale":"2023-12-19T18:28:08.807114+0000","last_undegraded":"2023-12-19T18:28:08.807114+0000","last_fullsized":"2023-12-19T18:28:08.807114+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:11:05.754382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.817423+0000","last_change":"2023-12-19T18:28:08.619723+0000","last_active":"2023-12-19T18:28:08.817423+0000","last_peered":"2023-12-19T18:28:08.817423+0000","last_clean":"2023-12-19T18:28:08.817423+0000","last_became_active":"2023-12-19T18:28:08.617193+0000","last_became_peered":"2023-12-19T18:28:08.617193+0000","last_unstale":"2023-12-19T18:28:08.817423+0000","last_undegraded":"2023-12-19T18:28:08.817423+0000","last_fullsized":"2023-12-19T18:28:08.817423+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:34:00.405702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807452+0000","last_change":"2023-12-19T18:28:08.095504+0000","last_active":"2023-12-19T18:28:08.807452+0000","last_peered":"2023-12-19T18:28:08.807452+0000","last_clean":"2023-12-19T18:28:08.807452+0000","last_became_active":"2023-12-19T18:28:08.093962+0000","last_became_peered":"2023-12-19T18:28:08.093962+0000","last_unstale":"2023-12-19T18:28:08.807452+0000","last_undegraded":"2023-12-19T18:28:08.807452+0000","last_fullsized":"2023-12-19T18:28:08.807452+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:20:59.865768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816720+0000","last_change":"2023-12-19T18:28:08.609589+0000","last_active":"2023-12-19T18:28:08.816720+0000","last_peered":"2023-12-19T18:28:08.816720+0000","last_clean":"2023-12-19T18:28:08.816720+0000","last_became_active":"2023-12-19T18:28:08.609368+0000","last_became_peered":"2023-12-19T18:28:08.609368+0000","last_unstale":"2023-12-19T18:28:08.816720+0000","last_undegraded":"2023-12-19T18:28:08.816720+0000","last_fullsized":"2023-12-19T18:28:08.816720+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T22:48:13.756314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.823596+0000","last_change":"2023-12-19T18:28:08.106735+0000","last_active":"2023-12-19T18:28:08.823596+0000","last_peered":"2023-12-19T18:28:08.823596+0000","last_clean":"2023-12-19T18:28:08.823596+0000","last_became_active":"2023-12-19T18:28:08.106493+0000","last_became_peered":"2023-12-19T18:28:08.106493+0000","last_unstale":"2023-12-19T18:28:08.823596+0000","last_undegraded":"2023-12-19T18:28:08.823596+0000","last_fullsized":"2023-12-19T18:28:08.823596+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T03:06:21.999153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.823467+0000","last_change":"2023-12-19T18:28:08.605546+0000","last_active":"2023-12-19T18:28:08.823467+0000","last_peered":"2023-12-19T18:28:08.823467+0000","last_clean":"2023-12-19T18:28:08.823467+0000","last_became_active":"2023-12-19T18:28:08.604914+0000","last_became_peered":"2023-12-19T18:28:08.604914+0000","last_unstale":"2023-12-19T18:28:08.823467+0000","last_undegraded":"2023-12-19T18:28:08.823467+0000","last_fullsized":"2023-12-19T18:28:08.823467+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:28:17.475387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816565+0000","last_change":"2023-12-19T18:28:08.098072+0000","last_active":"2023-12-19T18:28:08.816565+0000","last_peered":"2023-12-19T18:28:08.816565+0000","last_clean":"2023-12-19T18:28:08.816565+0000","last_became_active":"2023-12-19T18:28:08.096845+0000","last_became_peered":"2023-12-19T18:28:08.096845+0000","last_unstale":"2023-12-19T18:28:08.816565+0000","last_undegraded":"2023-12-19T18:28:08.816565+0000","last_fullsized":"2023-12-19T18:28:08.816565+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T23:47:45.923467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816511+0000","last_change":"2023-12-19T18:28:08.604237+0000","last_active":"2023-12-19T18:28:08.816511+0000","last_peered":"2023-12-19T18:28:08.816511+0000","last_clean":"2023-12-19T18:28:08.816511+0000","last_became_active":"2023-12-19T18:28:08.603720+0000","last_became_peered":"2023-12-19T18:28:08.603720+0000","last_unstale":"2023-12-19T18:28:08.816511+0000","last_undegraded":"2023-12-19T18:28:08.816511+0000","last_fullsized":"2023-12-19T18:28:08.816511+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T20:30:10.641773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.816384+0000","last_change":"2023-12-19T18:28:08.104765+0000","last_active":"2023-12-19T18:28:08.816384+0000","last_peered":"2023-12-19T18:28:08.816384+0000","last_clean":"2023-12-19T18:28:08.816384+0000","last_became_active":"2023-12-19T18:28:08.104333+0000","last_became_peered":"2023-12-19T18:28:08.104333+0000","last_unstale":"2023-12-19T18:28:08.816384+0000","last_undegraded":"2023-12-19T18:28:08.816384+0000","last_fullsized":"2023-12-19T18:28:08.816384+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:28:16.461376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.823534+0000","last_change":"2023-12-19T18:28:08.611196+0000","last_active":"2023-12-19T18:28:08.823534+0000","last_peered":"2023-12-19T18:28:08.823534+0000","last_clean":"2023-12-19T18:28:08.823534+0000","last_became_active":"2023-12-19T18:28:08.608529+0000","last_became_peered":"2023-12-19T18:28:08.608529+0000","last_unstale":"2023-12-19T18:28:08.823534+0000","last_undegraded":"2023-12-19T18:28:08.823534+0000","last_fullsized":"2023-12-19T18:28:08.823534+0000","mapping_epoch":200,"log_start":"0'0","ondisk_log_start":"0'0","created":200,"last_epoch_clean":201,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:08.058334+0000","last_clean_scrub_stamp":"2023-12-19T18:28:08.058334+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T01:10:03.084333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.807416+0000","last_change":"2023-12-19T18:28:08.098572+0000","last_active":"2023-12-19T18:28:08.807416+0000","last_peered":"2023-12-19T18:28:08.807416+0000","last_clean":"2023-12-19T18:28:08.807416+0000","last_became_active":"2023-12-19T18:28:08.098280+0000","last_became_peered":"2023-12-19T18:28:08.098280+0000","last_unstale":"2023-12-19T18:28:08.807416+0000","last_undegraded":"2023-12-19T18:28:08.807416+0000","last_fullsized":"2023-12-19T18:28:08.807416+0000","mapping_epoch":199,"log_start":"0'0","ondisk_log_start":"0'0","created":199,"last_epoch_clean":200,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:28:07.044323+0000","last_clean_scrub_stamp":"2023-12-19T18:28:07.044323+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:00:26.251041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":327,"reported_epoch":201,"state":"active+clean","last_fresh":"2023-12-19T18:28:08.584971+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:28:08.584971+0000","last_peered":"2023-12-19T18:28:08.584971+0000","last_clean":"2023-12-19T18:28:08.584971+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:28:08.584971+0000","last_undegraded":"2023-12-19T18:28:08.584971+0000","last_fullsized":"2023-12-19T18:28:08.584971+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":0},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575012,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1118,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194193,"statfs":{"total":200000000,"available":198854554,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.169,"15min":1.1719999999999999},"min":{"1min":0.66500000000000004,"5min":0.59299999999999997,"15min":0.45900000000000002},"max":{"1min":1.599,"5min":1.756,"15min":5.6879999999999997},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.2669999999999999,"5min":1.23,"15min":1.2190000000000001},"min":{"1min":0.90700000000000003,"5min":0.71899999999999997,"15min":0.55800000000000005},"max":{"1min":1.5700000000000001,"5min":1.7030000000000001,"15min":1.79},"last":1.1419999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.242,"5min":1.175,"15min":1.2330000000000001},"min":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.54400000000000004},"max":{"1min":1.6499999999999999,"5min":1.6499999999999999,"15min":10.926},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.2889999999999999,"5min":1.2709999999999999,"15min":1.27},"min":{"1min":0.85099999999999998,"5min":0.80700000000000005,"15min":0.67900000000000005},"max":{"1min":1.619,"5min":1.736,"15min":10.904999999999999},"last":0.76600000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":1.1020000000000001,"15min":1.1100000000000001},"min":{"1min":0.74099999999999999,"5min":0.66000000000000003,"15min":0.61499999999999999},"max":{"1min":1.365,"5min":1.6200000000000001,"15min":3.1960000000000002},"last":1.4179999999999999},{"interface":"front","average":{"1min":1.1850000000000001,"5min":1.167,"15min":1.198},"min":{"1min":0.78700000000000003,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.587,"5min":1.6799999999999999,"15min":6.008},"last":1.1020000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.236,"5min":1.244,"15min":1.2130000000000001},"min":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.61399999999999999},"max":{"1min":1.575,"5min":1.732,"15min":3.0379999999999998},"last":1.4690000000000001},{"interface":"front","average":{"1min":1.319,"5min":1.333,"15min":1.333},"min":{"1min":0.89500000000000002,"5min":0.77500000000000002,"15min":0.69599999999999995},"max":{"1min":1.704,"5min":1.772,"15min":5.8120000000000003},"last":1.284}]},{"osd":4,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1919999999999999,"15min":1.173},"min":{"1min":0.67700000000000005,"5min":0.55700000000000005,"15min":0.47899999999999998},"max":{"1min":1.454,"5min":1.847,"15min":4.0110000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.1990000000000001,"5min":1.242,"15min":1.2529999999999999},"min":{"1min":0.65500000000000003,"5min":0.59299999999999997,"15min":0.57599999999999996},"max":{"1min":1.502,"5min":1.8,"15min":3.3180000000000001},"last":1.3460000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.28,"15min":1.331},"min":{"1min":0.91900000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.617,"5min":1.956,"15min":7.6429999999999998},"last":0.995},{"interface":"front","average":{"1min":1.323,"5min":1.3460000000000001,"15min":1.3580000000000001},"min":{"1min":0.81799999999999995,"5min":0.627,"15min":0.53500000000000003},"max":{"1min":1.6799999999999999,"5min":1.8280000000000001,"15min":6.0839999999999996},"last":1.246}]},{"osd":6,"last update":"Tue Dec 19 18:27:14 2023","interfaces":[{"interface":"back","average":{"1min":1.486,"5min":1.492,"15min":1.4970000000000001},"min":{"1min":0.95599999999999996,"5min":0.628,"15min":0.59899999999999998},"max":{"1min":1.7290000000000001,"5min":1.8959999999999999,"15min":3.1309999999999998},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.4319999999999999,"15min":1.419},"min":{"1min":0.73199999999999998,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.641,"5min":1.929,"15min":4.1479999999999997},"last":1.5169999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575012,"num_pgs":23,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1118,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194193,"statfs":{"total":200000000,"available":198854520,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.099,"15min":1.0680000000000001},"min":{"1min":0.68300000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.413,"5min":1.522,"15min":1.8129999999999999},"last":1.4019999999999999},{"interface":"front","average":{"1min":1.147,"5min":1.151,"15min":1.1220000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.55600000000000005},"max":{"1min":1.587,"5min":1.6000000000000001,"15min":3.5529999999999999},"last":1.2370000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1220000000000001,"15min":1.131},"min":{"1min":0.75700000000000001,"5min":0.68600000000000005,"15min":0.68200000000000005},"max":{"1min":1.573,"5min":1.7909999999999999,"15min":4.5540000000000003},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.177,"15min":1.179},"min":{"1min":0.80100000000000005,"5min":0.64600000000000002,"15min":0.60299999999999998},"max":{"1min":1.5569999999999999,"5min":1.5960000000000001,"15min":1.885},"last":1.3160000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.258,"15min":1.21},"min":{"1min":0.89900000000000002,"5min":0.78900000000000003,"15min":0.626},"max":{"1min":1.5569999999999999,"5min":1.6479999999999999,"15min":1.821},"last":1.1459999999999999},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2190000000000001,"15min":1.216},"min":{"1min":0.85899999999999999,"5min":0.62,"15min":0.60899999999999999},"max":{"1min":1.514,"5min":1.7669999999999999,"15min":3.7909999999999999},"last":1.3480000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3129999999999999,"15min":1.2569999999999999},"min":{"1min":0.90800000000000003,"5min":0.71799999999999997,"15min":0.67800000000000005},"max":{"1min":1.5469999999999999,"5min":1.804,"15min":3.8820000000000001},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.258,"5min":1.248,"15min":1.2050000000000001},"min":{"1min":0.96099999999999997,"5min":0.71199999999999997,"15min":0.69899999999999995},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":4.6189999999999998},"last":1.1879999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.306,"15min":1.2809999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.65600000000000003},"max":{"1min":1.6259999999999999,"5min":1.7529999999999999,"15min":3.0680000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.417,"5min":1.415,"15min":1.3939999999999999},"min":{"1min":1.0529999999999999,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.7090000000000001,"5min":2.1040000000000001,"15min":3.7000000000000002},"last":1.4330000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2809999999999999,"15min":1.2470000000000001},"min":{"1min":0.91500000000000004,"5min":0.84899999999999998,"15min":0.65500000000000003},"max":{"1min":1.7929999999999999,"5min":1.9510000000000001,"15min":4.6680000000000001},"last":1.46},{"interface":"front","average":{"1min":1.375,"5min":1.4079999999999999,"15min":1.3959999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.626},"max":{"1min":1.609,"5min":1.794,"15min":4.1159999999999997},"last":1.046}]},{"osd":7,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.444,"15min":1.403},"min":{"1min":1.0149999999999999,"5min":0.65700000000000003,"15min":0.61899999999999999},"max":{"1min":1.663,"5min":2.02,"15min":3.9670000000000001},"last":1.536},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4470000000000001,"15min":1.413},"min":{"1min":1.0309999999999999,"5min":0.999,"15min":0.79300000000000004},"max":{"1min":1.855,"5min":2.2149999999999999,"15min":5.6639999999999997},"last":1.381}]}]},{"osd":5,"up_from":13,"seq":55834575012,"num_pgs":29,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1695,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193616,"statfs":{"total":200000000,"available":198263514,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.282,"15min":1.2150000000000001},"min":{"1min":0.76900000000000002,"5min":0.69999999999999996,"15min":0.61899999999999999},"max":{"1min":1.6830000000000001,"5min":2.089,"15min":2.089},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.298,"5min":1.304,"15min":1.2509999999999999},"min":{"1min":0.69399999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6020000000000001,"5min":1.641,"15min":1.913},"last":1.208}]},{"osd":1,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.167,"15min":1.1399999999999999},"min":{"1min":0.91600000000000004,"5min":0.71999999999999997,"15min":0.65700000000000003},"max":{"1min":1.571,"5min":1.851,"15min":1.9490000000000001},"last":1.0740000000000001},{"interface":"front","average":{"1min":1.202,"5min":1.1890000000000001,"15min":1.141},"min":{"1min":0.89400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.452,"5min":1.899,"15min":1.899},"last":1.107}]},{"osd":2,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2050000000000001,"15min":1.149},"min":{"1min":0.73999999999999999,"5min":0.72199999999999998,"15min":0.59499999999999997},"max":{"1min":1.468,"5min":1.806,"15min":1.806},"last":1.363},{"interface":"front","average":{"1min":1.343,"5min":1.282,"15min":1.2230000000000001},"min":{"1min":0.92500000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.651,"5min":1.8,"15min":2.036},"last":0.98399999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.198,"15min":1.1599999999999999},"min":{"1min":0.93200000000000005,"5min":0.76600000000000001,"15min":0.68899999999999995},"max":{"1min":1.425,"5min":1.635,"15min":1.7450000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.411,"5min":1.3540000000000001,"15min":1.3080000000000001},"min":{"1min":0.99199999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.843,"5min":2.0369999999999999,"15min":2.0590000000000002},"last":0.84999999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.288,"15min":1.2629999999999999},"min":{"1min":1.024,"5min":0.52300000000000002,"15min":0.46700000000000003},"max":{"1min":1.5589999999999999,"5min":1.7250000000000001,"15min":2.0329999999999999},"last":1.3979999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.1379999999999999,"15min":1.159},"min":{"1min":0.86899999999999999,"5min":0.64500000000000002,"15min":0.57099999999999995},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.901},"last":1.038}]},{"osd":6,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.425,"15min":1.411},"min":{"1min":1.1850000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.952,"5min":2.1579999999999999,"15min":2.1589999999999998},"last":1.3129999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.2969999999999999,"15min":1.2889999999999999},"min":{"1min":0.94999999999999996,"5min":0.58299999999999996,"15min":0.49299999999999999},"max":{"1min":1.5720000000000001,"5min":1.704,"15min":1.976},"last":1.258}]},{"osd":7,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.2769999999999999,"15min":1.2729999999999999},"min":{"1min":0.83599999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.88},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4419999999999999,"15min":1.4430000000000001},"min":{"1min":1.1759999999999999,"5min":0.67100000000000004,"15min":0.623},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":2.117},"last":1.429}]}]},{"osd":4,"up_from":13,"seq":55834575012,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1118,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194193,"statfs":{"total":200000000,"available":198854520,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.069,"15min":1.024},"min":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.52700000000000002},"max":{"1min":1.403,"5min":1.972,"15min":1.972},"last":0.99099999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.1180000000000001,"15min":1.073},"min":{"1min":0.88300000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5309999999999999,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.2809999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.204,"15min":1.1679999999999999},"min":{"1min":0.85799999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.4199999999999999,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":0.94899999999999995},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.288,"15min":1.2350000000000001},"min":{"1min":1.056,"5min":0.64700000000000002,"15min":0.61699999999999999},"max":{"1min":1.6459999999999999,"5min":1.7450000000000001,"15min":1.875},"last":1.5149999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.1559999999999999,"15min":1.135},"min":{"1min":0.77200000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.597,"5min":1.744,"15min":1.744},"last":1.9350000000000001},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1839999999999999,"15min":1.1459999999999999},"min":{"1min":0.93000000000000005,"5min":0.747,"15min":0.70499999999999996},"max":{"1min":1.8149999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":1.125}]},{"osd":3,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2509999999999999,"15min":1.268},"min":{"1min":1.0549999999999999,"5min":0.754,"15min":0.754},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.946},"last":1.3859999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.369,"15min":1.335},"min":{"1min":1.123,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.6619999999999999,"5min":1.927,"15min":1.927},"last":1.8819999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2569999999999999,"15min":1.248},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.44500000000000001},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.357,"15min":1.369},"min":{"1min":0.82599999999999996,"5min":0.59099999999999997,"15min":0.504},"max":{"1min":1.7729999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.633}]},{"osd":6,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.2969999999999999,"15min":1.304},"min":{"1min":0.88500000000000001,"5min":0.68100000000000005,"15min":0.432},"max":{"1min":1.7490000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.772},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4219999999999999,"15min":1.421},"min":{"1min":1.1599999999999999,"5min":0.93700000000000006,"15min":0.64500000000000002},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.474}]},{"osd":7,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.3560000000000001,"15min":1.363},"min":{"1min":0.996,"5min":0.77300000000000002,"15min":0.56399999999999995},"max":{"1min":2.702,"5min":2.702,"15min":2.702},"last":1.8180000000000001},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4259999999999999,"15min":1.421},"min":{"1min":1.115,"5min":0.78600000000000003,"15min":0.57399999999999995},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.9930000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575013,"num_pgs":21,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1120,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194192,"statfs":{"total":200000000,"available":198852968,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.90900000000000003,"15min":0.92900000000000005},"min":{"1min":0.72299999999999998,"5min":0.435,"15min":0.35299999999999998},"max":{"1min":1.2450000000000001,"5min":1.7450000000000001,"15min":3.4350000000000001},"last":1.754},{"interface":"front","average":{"1min":1.0620000000000001,"5min":1.0409999999999999,"15min":0.99399999999999999},"min":{"1min":0.67100000000000004,"5min":0.54600000000000004,"15min":0.39500000000000002},"max":{"1min":1.401,"5min":1.9159999999999999,"15min":4.3490000000000002},"last":0.89100000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.131,"5min":1.034,"15min":1.032},"min":{"1min":0.79600000000000004,"5min":0.60899999999999999,"15min":0.41299999999999998},"max":{"1min":1.698,"5min":1.9690000000000001,"15min":5.2510000000000003},"last":0.76800000000000002},{"interface":"front","average":{"1min":1.139,"5min":1.0900000000000001,"15min":1.107},"min":{"1min":0.88100000000000001,"5min":0.621,"15min":0.47799999999999998},"max":{"1min":1.6419999999999999,"5min":1.643,"15min":5.4119999999999999},"last":1.363}]},{"osd":2,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.214,"15min":1.242},"min":{"1min":0.872,"5min":0.628,"15min":0.46000000000000002},"max":{"1min":1.6619999999999999,"5min":2.4649999999999999,"15min":13.833},"last":1.0109999999999999},{"interface":"front","average":{"1min":1.3580000000000001,"5min":1.3220000000000001,"15min":1.3520000000000001},"min":{"1min":0.95499999999999996,"5min":0.76200000000000001,"15min":0.53200000000000003},"max":{"1min":1.6459999999999999,"5min":2.379,"15min":16.748999999999999},"last":1.8460000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.306,"15min":1.286},"min":{"1min":0.94799999999999995,"5min":0.77900000000000003,"15min":0.65600000000000003},"max":{"1min":1.744,"5min":2.0150000000000001,"15min":5.1790000000000003},"last":1.274},{"interface":"front","average":{"1min":1.327,"5min":1.387,"15min":1.397},"min":{"1min":1.0169999999999999,"5min":0.94199999999999995,"15min":0.73199999999999998},"max":{"1min":1.534,"5min":2.0819999999999999,"15min":4.569},"last":1.173}]},{"osd":5,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.4199999999999999,"5min":1.411,"15min":1.429},"min":{"1min":1.1719999999999999,"5min":0.80700000000000005,"15min":0.76000000000000001},"max":{"1min":2.0449999999999999,"5min":2.0449999999999999,"15min":5.3289999999999997},"last":1.4950000000000001},{"interface":"front","average":{"1min":1.431,"5min":1.389,"15min":1.3400000000000001},"min":{"1min":1.0409999999999999,"5min":0.95199999999999996,"15min":0.69999999999999996},"max":{"1min":1.9610000000000001,"5min":2.1469999999999998,"15min":4.6369999999999996},"last":1.5800000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":1.46,"15min":1.456},"min":{"1min":1.139,"5min":0.86499999999999999,"15min":0.81599999999999995},"max":{"1min":1.774,"5min":2.2040000000000002,"15min":4.5039999999999996},"last":1.536},{"interface":"front","average":{"1min":1.548,"5min":1.506,"15min":1.5089999999999999},"min":{"1min":1.3120000000000001,"5min":0.876,"15min":0.80500000000000005},"max":{"1min":2.1339999999999999,"5min":2.1339999999999999,"15min":5.3739999999999997},"last":1.637}]},{"osd":7,"last update":"Tue Dec 19 18:28:01 2023","interfaces":[{"interface":"back","average":{"1min":1.536,"5min":1.4870000000000001,"15min":1.4430000000000001},"min":{"1min":1.224,"5min":0.89700000000000002,"15min":0.73199999999999998},"max":{"1min":2.1120000000000001,"5min":2.2869999999999999,"15min":5.1349999999999998},"last":1.708},{"interface":"front","average":{"1min":1.5669999999999999,"5min":1.5660000000000001,"15min":1.556},"min":{"1min":1.367,"5min":1.1020000000000001,"15min":0.85399999999999998},"max":{"1min":1.8260000000000001,"5min":2.351,"15min":4.4619999999999997},"last":1.2310000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575012,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1119,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194193,"statfs":{"total":200000000,"available":198853972,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.048,"5min":0.94599999999999995,"15min":0.98099999999999998},"min":{"1min":0.45000000000000001,"5min":0.43099999999999999,"15min":0.34399999999999997},"max":{"1min":2.2799999999999998,"5min":2.2799999999999998,"15min":9.2550000000000008},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.0960000000000001,"5min":1.0389999999999999,"15min":1.024},"min":{"1min":0.745,"5min":0.38400000000000001,"15min":0.33900000000000002},"max":{"1min":1.8049999999999999,"5min":1.8049999999999999,"15min":4.7850000000000001},"last":1.4399999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.95199999999999996,"15min":1.018},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.262,"5min":2.0169999999999999,"15min":8.2720000000000002},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.069,"5min":1.1180000000000001,"15min":1.1200000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.46600000000000003},"max":{"1min":1.413,"5min":1.8460000000000001,"15min":5.0620000000000003},"last":1.0469999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.2749999999999999,"15min":1.252},"min":{"1min":0.64200000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":2.23,"5min":2.8159999999999998,"15min":8.3109999999999999},"last":1.123},{"interface":"front","average":{"1min":1.1080000000000001,"5min":1.1180000000000001,"15min":1.1850000000000001},"min":{"1min":0.53100000000000003,"5min":0.49299999999999999,"15min":0.41799999999999998},"max":{"1min":1.778,"5min":2.6240000000000001,"15min":8.0690000000000008},"last":0.93700000000000006}]},{"osd":4,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.417,"5min":1.369,"15min":1.403},"min":{"1min":1,"5min":0.79400000000000004,"15min":0.79300000000000004},"max":{"1min":1.8620000000000001,"5min":2.2160000000000002,"15min":5.3929999999999998},"last":1.639},{"interface":"front","average":{"1min":1.46,"5min":1.4610000000000001,"15min":1.4750000000000001},"min":{"1min":1.036,"5min":0.85799999999999998,"15min":0.78300000000000003},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":7.3570000000000002},"last":1.306}]},{"osd":5,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.361,"5min":1.379,"15min":1.429},"min":{"1min":0.84499999999999997,"5min":0.747,"15min":0.747},"max":{"1min":2.133,"5min":2.1680000000000001,"15min":8.1839999999999993},"last":1.337},{"interface":"front","average":{"1min":1.4299999999999999,"5min":1.3720000000000001,"15min":1.3919999999999999},"min":{"1min":1.008,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.9239999999999999,"5min":1.9930000000000001,"15min":5.165},"last":1.3819999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.47,"5min":1.4299999999999999,"15min":1.4279999999999999},"min":{"1min":0.91100000000000003,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":2.1629999999999998,"5min":2.4630000000000001,"15min":5.1059999999999999},"last":1.21},{"interface":"front","average":{"1min":1.462,"5min":1.4930000000000001,"15min":1.4970000000000001},"min":{"1min":0.89600000000000002,"5min":0.83799999999999997,"15min":0.77000000000000002},"max":{"1min":2.3399999999999999,"5min":2.7559999999999998,"15min":8.1370000000000005},"last":1.498}]},{"osd":7,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.518,"5min":1.524,"15min":1.514},"min":{"1min":0.96699999999999997,"5min":0.80000000000000004,"15min":0.78200000000000003},"max":{"1min":1.968,"5min":2.3959999999999999,"15min":5.1449999999999996},"last":1.605},{"interface":"front","average":{"1min":1.575,"5min":1.593,"15min":1.585},"min":{"1min":0.94699999999999995,"5min":0.93000000000000005,"15min":0.86499999999999999},"max":{"1min":2.419,"5min":2.6320000000000001,"15min":8.2300000000000004},"last":1.8260000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575013,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1696,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193615,"statfs":{"total":200000000,"available":198262440,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.90900000000000003,"15min":0.86799999999999999},"min":{"1min":0.63900000000000001,"5min":0.41199999999999998,"15min":0.34699999999999998},"max":{"1min":1.3520000000000001,"5min":2.262,"15min":2.262},"last":0.86899999999999999},{"interface":"front","average":{"1min":1.016,"5min":1.01,"15min":0.96499999999999997},"min":{"1min":0.46800000000000003,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.349,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.2430000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":1.0800000000000001,"5min":1.016,"15min":1.036},"min":{"1min":0.70399999999999996,"5min":0.57499999999999996,"15min":0.41399999999999998},"max":{"1min":1.377,"5min":1.53,"15min":2.5739999999999998},"last":1.129},{"interface":"front","average":{"1min":0.95299999999999996,"5min":1.014,"15min":0.97499999999999998},"min":{"1min":0.621,"5min":0.39900000000000002,"15min":0.38500000000000001},"max":{"1min":1.379,"5min":1.718,"15min":1.845},"last":1.276}]},{"osd":3,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3759999999999999,"5min":1.347,"15min":1.254},"min":{"1min":0.66900000000000004,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.825,"5min":2.089,"15min":2.137},"last":1.081},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.2450000000000001,"15min":1.204},"min":{"1min":0.59299999999999997,"5min":0.50800000000000001,"15min":0.44900000000000001},"max":{"1min":1.8700000000000001,"5min":2.0720000000000001,"15min":2.3959999999999999},"last":0.97399999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":1.3939999999999999,"5min":1.3149999999999999,"15min":1.3120000000000001},"min":{"1min":1.1990000000000001,"5min":0.80500000000000005,"15min":0.69099999999999995},"max":{"1min":1.7929999999999999,"5min":2.1859999999999999,"15min":2.444},"last":1.3200000000000001},{"interface":"front","average":{"1min":1.387,"5min":1.381,"15min":1.3480000000000001},"min":{"1min":0.94499999999999995,"5min":0.92500000000000004,"15min":0.71099999999999997},"max":{"1min":1.8859999999999999,"5min":1.8859999999999999,"15min":2.6440000000000001},"last":1.5580000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4259999999999999,"15min":1.3640000000000001},"min":{"1min":0.79200000000000004,"5min":0.76700000000000002,"15min":0.76000000000000001},"max":{"1min":1.915,"5min":1.998,"15min":2.5459999999999998},"last":1.9990000000000001},{"interface":"front","average":{"1min":1.504,"5min":1.4330000000000001,"15min":1.4350000000000001},"min":{"1min":1.0760000000000001,"5min":0.85899999999999999,"15min":0.70499999999999996},"max":{"1min":1.9339999999999999,"5min":2.141,"15min":2.6179999999999999},"last":1.2150000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":1.45,"5min":1.4530000000000001,"15min":1.4119999999999999},"min":{"1min":0.83299999999999996,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":2.1179999999999999,"5min":2.274,"15min":2.7000000000000002},"last":1.8879999999999999},{"interface":"front","average":{"1min":1.6000000000000001,"5min":1.49,"15min":1.48},"min":{"1min":1.159,"5min":0.88100000000000001,"15min":0.76400000000000001},"max":{"1min":2.137,"5min":2.2509999999999999,"15min":2.895},"last":1.498}]},{"osd":7,"last update":"Tue Dec 19 18:28:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4870000000000001,"5min":1.4350000000000001,"15min":1.4419999999999999},"min":{"1min":1.0349999999999999,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.835,"5min":2.1629999999999998,"15min":9.4830000000000005},"last":2.044},{"interface":"front","average":{"1min":1.554,"5min":1.554,"15min":1.542},"min":{"1min":0.91500000000000004,"5min":0.80800000000000005,"15min":0.71899999999999997},"max":{"1min":2.0720000000000001,"5min":2.3420000000000001,"15min":9.4109999999999996},"last":2.198}]}]},{"osd":0,"up_from":13,"seq":55834575012,"num_pgs":24,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1118,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194193,"statfs":{"total":200000000,"available":198854484,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92200000000000004,"15min":0.92100000000000004},"min":{"1min":0.60699999999999998,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":1.425,"5min":1.671,"15min":2.3199999999999998},"last":0.69199999999999995},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1100000000000001,"15min":1.0720000000000001},"min":{"1min":0.58899999999999997,"5min":0.48699999999999999,"15min":0.42799999999999999},"max":{"1min":1.3480000000000001,"5min":2.2170000000000001,"15min":2.4180000000000001},"last":1.1519999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0289999999999999,"15min":0.996},"min":{"1min":0.69399999999999995,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":1.4550000000000001,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.0609999999999999,"15min":0.996},"min":{"1min":0.64000000000000001,"5min":0.42699999999999999,"15min":0.41099999999999998},"max":{"1min":1.6870000000000001,"5min":2.3340000000000001,"15min":2.7810000000000001},"last":0.64000000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.032,"15min":1.0249999999999999},"min":{"1min":0.63300000000000001,"5min":0.44,"15min":0.38900000000000001},"max":{"1min":1.5589999999999999,"5min":1.8200000000000001,"15min":2.5190000000000001},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.349,"15min":1.2769999999999999},"min":{"1min":0.81299999999999994,"5min":0.65100000000000002,"15min":0.47499999999999998},"max":{"1min":1.6759999999999999,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.466}]},{"osd":4,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.323,"15min":1.2749999999999999},"min":{"1min":0.94499999999999995,"5min":0.67600000000000005,"15min":0.63200000000000001},"max":{"1min":1.7290000000000001,"5min":2.1920000000000002,"15min":2.5529999999999999},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.27,"5min":1.2789999999999999,"15min":1.28},"min":{"1min":0.81899999999999995,"5min":0.70399999999999996,"15min":0.67600000000000005},"max":{"1min":1.6699999999999999,"5min":2.056,"15min":2.9529999999999998},"last":1.0089999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3460000000000001,"15min":1.3919999999999999},"min":{"1min":0.92800000000000005,"5min":0.70699999999999996,"15min":0.64500000000000002},"max":{"1min":1.542,"5min":1.97,"15min":16.298999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.387,"15min":1.3360000000000001},"min":{"1min":1.115,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.641,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.393}]},{"osd":6,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.4470000000000001},"min":{"1min":0.93300000000000005,"5min":0.79800000000000004,"15min":0.71699999999999997},"max":{"1min":1.744,"5min":2.4169999999999998,"15min":2.5910000000000002},"last":1.282},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4810000000000001,"15min":1.4179999999999999},"min":{"1min":1.097,"5min":0.745,"15min":0.745},"max":{"1min":1.8,"5min":2.1709999999999998,"15min":2.46},"last":1.097}]},{"osd":7,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4910000000000001,"15min":1.454},"min":{"1min":1.052,"5min":0.72099999999999997,"15min":0.70899999999999996},"max":{"1min":1.895,"5min":2.2549999999999999,"15min":2.4969999999999999},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.407,"15min":1.415},"min":{"1min":1.077,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6439999999999999,"5min":1.9370000000000001,"15min":3.1429999999999998},"last":1.1299999999999999}]}]}],"pool_statfs":[]}} 2023-12-19T18:28:14.139 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-19T18:28:14.140 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-19T18:28:14.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.426+0000 7f4de56be640 1 Processor -- start 2023-12-19T18:28:14.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.426+0000 7f4de56be640 1 -- start start 2023-12-19T18:28:14.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.428+0000 7f4de56be640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de0070650 0x7f4de015a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.428+0000 7f4de56be640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4de0070ac0 0x7f4de015a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.428+0000 7f4de56be640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4de00ccba0 con 0x7f4de006c030 2023-12-19T18:28:14.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.428+0000 7f4de56be640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4de006f960 con 0x7f4de0070ac0 2023-12-19T18:28:14.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.428+0000 7f4de56be640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de015b070 con 0x7f4de0070650 2023-12-19T18:28:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.428+0000 7f4ddffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4de006c030 0x7f4de006c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47596/0 (socket says 172.21.15.27:47596) 2023-12-19T18:28:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.428+0000 7f4ddffff640 1 -- 172.21.15.27:0/1427688914 learned_addr learned my addr 172.21.15.27:0/1427688914 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4dd7fff640 1 --2- 172.21.15.27:0/1427688914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de0070650 0x7f4de015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502635601 0 0) 0x7f4de00ccba0 con 0x7f4de006c030 2023-12-19T18:28:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4dcc003520 con 0x7f4de006c030 2023-12-19T18:28:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4de4ebd640 1 --2- 172.21.15.27:0/1427688914 >> v2:172.21.15.192:3300/0 conn(0x7f4de0070ac0 0x7f4de015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3654646260 0 0) 0x7f4dcc003520 con 0x7f4de006c030 2023-12-19T18:28:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de0070650 msgr2=0x7f4de015a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 --2- 172.21.15.27:0/1427688914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de0070650 0x7f4de015a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 >> v2:172.21.15.192:3300/0 conn(0x7f4de0070ac0 msgr2=0x7f4de015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 --2- 172.21.15.27:0/1427688914 >> v2:172.21.15.192:3300/0 conn(0x7f4de0070ac0 0x7f4de015a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de00701c0 con 0x7f4de006c030 2023-12-19T18:28:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f4dd0001670 con 0x7f4de006c030 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4dd7fff640 1 --2- 172.21.15.27:0/1427688914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de0070650 0x7f4de015a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 >> v1:172.21.15.27:6789/0 conn(0x7f4de006c030 legacy=0x7f4de006c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 --2- 172.21.15.27:0/1427688914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de0070650 0x7f4dcc003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 --2- 172.21.15.27:0/1427688914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4dcc003f60 0x7f4dcc0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 --2- 172.21.15.27:0/1427688914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dcc004d00 0x7f4dcc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de006f960 con 0x7f4dcc003f60 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de015b070 con 0x7f4dcc004d00 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.429+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dcc00dcf0 con 0x7f4de0070650 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.430+0000 7f4dd7fff640 1 --2- 172.21.15.27:0/1427688914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4dcc003f60 0x7f4dcc0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.430+0000 7f4ddffff640 1 --2- 172.21.15.27:0/1427688914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de0070650 0x7f4dcc003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.430+0000 7f4dd7fff640 1 -- 172.21.15.27:0/1427688914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de0070650 msgr2=0x7f4dcc003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.430+0000 7f4dd7fff640 1 --2- 172.21.15.27:0/1427688914 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de0070650 0x7f4dcc003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.430+0000 7f4dd7fff640 1 -- 172.21.15.27:0/1427688914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dcc004d00 msgr2=0x7f4dcc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.430+0000 7f4dd7fff640 1 --2- 172.21.15.27:0/1427688914 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dcc004d00 0x7f4dcc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.430+0000 7f4dd7fff640 1 -- 172.21.15.27:0/1427688914 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dcc003520 con 0x7f4dcc003f60 2023-12-19T18:28:14.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.431+0000 7f4dd7fff640 1 --2- 172.21.15.27:0/1427688914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4dcc003f60 0x7f4dcc0047c0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80098a0 tx=0x7f4dcc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886b033ab19835e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:14.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.432+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8004230 con 0x7f4dcc003f60 2023-12-19T18:28:14.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.432+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dc8004390 con 0x7f4dcc003f60 2023-12-19T18:28:14.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.432+0000 7f4ddf7fe640 1 -- 172.21.15.27:0/1427688914 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8003830 con 0x7f4dcc003f60 2023-12-19T18:28:14.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.432+0000 7f4de56be640 1 -- 172.21.15.27:0/1427688914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4dcc003f60 msgr2=0x7f4dcc0047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:14.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.432+0000 7f4de56be640 1 --2- 172.21.15.27:0/1427688914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4dcc003f60 0x7f4dcc0047c0 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f4dc80098a0 tx=0x7f4dcc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.432+0000 7f4de4ebd640 1 -- 172.21.15.27:0/1427688914 reap_dead start 2023-12-19T18:28:14.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.432+0000 7f4de56be640 1 -- 172.21.15.27:0/1427688914 shutdown_connections 2023-12-19T18:28:14.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.432+0000 7f4de56be640 1 -- 172.21.15.27:0/1427688914 >> 172.21.15.27:0/1427688914 conn(0x7f4de00cb3f0 msgr2=0x7f4de00cb800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:14.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.433+0000 7f4de56be640 1 -- 172.21.15.27:0/1427688914 shutdown_connections 2023-12-19T18:28:14.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.433+0000 7f4de56be640 1 -- 172.21.15.27:0/1427688914 wait complete. 2023-12-19T18:28:14.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.434+0000 7f4de56be640 1 Processor -- start 2023-12-19T18:28:14.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.434+0000 7f4de56be640 1 -- start start 2023-12-19T18:28:14.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.435+0000 7f4de56be640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dcc003f60 0x7f4de00b04f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.435+0000 7f4de56be640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de01f19d0 0x7f4de00c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.435+0000 7f4de56be640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de00ae620 0x7f4de00aeab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.435+0000 7f4de56be640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de00adeb0 con 0x7f4de00ae620 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.435+0000 7f4de56be640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de01f2140 con 0x7f4dcc003f60 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.435+0000 7f4de56be640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4de01f2420 con 0x7f4de01f19d0 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.436+0000 7f4de4ebd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de00ae620 0x7f4de00aeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.436+0000 7f4de4ebd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de00ae620 0x7f4de00aeab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42288/0 (socket says 172.21.15.27:42288) 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.436+0000 7f4de4ebd640 1 -- 172.21.15.27:0/2794292552 learned_addr learned my addr 172.21.15.27:0/2794292552 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.436+0000 7f4dd7fff640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de01f19d0 0x7f4de00c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.436+0000 7f4ddffff640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dcc003f60 0x7f4de00b04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.436+0000 7f4de4ebd640 1 -- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de01f19d0 msgr2=0x7f4de00c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.436+0000 7f4de4ebd640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de01f19d0 0x7f4de00c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.436+0000 7f4de4ebd640 1 -- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dcc003f60 msgr2=0x7f4de00b04f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.436+0000 7f4de4ebd640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dcc003f60 0x7f4de00b04f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.436+0000 7f4de4ebd640 1 -- 172.21.15.27:0/2794292552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de01f2700 con 0x7f4de00ae620 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.436+0000 7f4ddffff640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dcc003f60 0x7f4de00b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.437+0000 7f4de4ebd640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de00ae620 0x7f4de00aeab0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f4de0070ea0 tx=0x7f4dd800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.437+0000 7f4ddd7fa640 1 -- 172.21.15.27:0/2794292552 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800ed90 con 0x7f4de00ae620 2023-12-19T18:28:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.437+0000 7f4ddd7fa640 1 -- 172.21.15.27:0/2794292552 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dd8014420 con 0x7f4de00ae620 2023-12-19T18:28:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.437+0000 7f4de56be640 1 -- 172.21.15.27:0/2794292552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de00ae100 con 0x7f4de00ae620 2023-12-19T18:28:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.437+0000 7f4ddd7fa640 1 -- 172.21.15.27:0/2794292552 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd80147d0 con 0x7f4de00ae620 2023-12-19T18:28:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.437+0000 7f4de56be640 1 -- 172.21.15.27:0/2794292552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de00c5660 con 0x7f4de00ae620 2023-12-19T18:28:14.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.439+0000 7f4de56be640 1 -- 172.21.15.27:0/2794292552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de00ae2f0 con 0x7f4de00ae620 2023-12-19T18:28:14.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.440+0000 7f4ddd7fa640 1 -- 172.21.15.27:0/2794292552 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4dd8014930 con 0x7f4de00ae620 2023-12-19T18:28:14.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.441+0000 7f4ddd7fa640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4dc006a460 0x7f4dc006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.441+0000 7f4ddd7fa640 1 -- 172.21.15.27:0/2794292552 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4dd8088e10 con 0x7f4de00ae620 2023-12-19T18:28:14.441 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.441+0000 7f4ddffff640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4dc006a460 0x7f4dc006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.442+0000 7f4ddffff640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4dc006a460 0x7f4dc006c920 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0005990 tx=0x7f4dd0001d90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:14.445 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.446+0000 7f4ddd7fa640 1 -- 172.21.15.27:0/2794292552 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4dd8054490 con 0x7f4de00ae620 2023-12-19T18:28:14.722 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.722+0000 7f4de56be640 1 -- 172.21.15.27:0/2794292552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4de004e840 con 0x7f4de00ae620 2023-12-19T18:28:14.728 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.727+0000 7f4ddd7fa640 1 -- 172.21.15.27:0/2794292552 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7f4dd8057ab0 con 0x7f4de00ae620 2023-12-19T18:28:14.728 INFO:teuthology.orchestra.run.smithi027.stderr:updated caps for client.0 2023-12-19T18:28:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.730+0000 7f4de56be640 1 -- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4dc006a460 msgr2=0x7f4dc006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.730+0000 7f4de56be640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4dc006a460 0x7f4dc006c920 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f4dd0005990 tx=0x7f4dd0001d90 comp rx=0 tx=0).stop 2023-12-19T18:28:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.730+0000 7f4de56be640 1 -- 172.21.15.27:0/2794292552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de00ae620 msgr2=0x7f4de00aeab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.730+0000 7f4de56be640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de00ae620 0x7f4de00aeab0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f4de0070ea0 tx=0x7f4dd800e4b0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.730+0000 7f4de56be640 1 -- 172.21.15.27:0/2794292552 shutdown_connections 2023-12-19T18:28:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.730+0000 7f4de56be640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4dc006a460 0x7f4dc006c920 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.730+0000 7f4de56be640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4de00ae620 0x7f4de00aeab0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.730+0000 7f4de56be640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4de01f19d0 0x7f4de00c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.730+0000 7f4de56be640 1 --2- 172.21.15.27:0/2794292552 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4dcc003f60 0x7f4de00b04f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.730+0000 7f4de56be640 1 -- 172.21.15.27:0/2794292552 >> 172.21.15.27:0/2794292552 conn(0x7f4de00cb3f0 msgr2=0x7f4de015dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.731+0000 7f4de56be640 1 -- 172.21.15.27:0/2794292552 shutdown_connections 2023-12-19T18:28:14.731 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.731+0000 7f4de56be640 1 -- 172.21.15.27:0/2794292552 wait complete. 2023-12-19T18:28:14.751 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:28:14.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.983+0000 7fde9bfff640 1 Processor -- start 2023-12-19T18:28:14.983 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.983+0000 7fde9bfff640 1 -- start start 2023-12-19T18:28:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.985+0000 7fde9bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fde9c15ee30 0x7fde9c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.985+0000 7fde9bfff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9c0cb350 con 0x7fde9c0afeb0 2023-12-19T18:28:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.985+0000 7fde9bfff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde9c0cb630 con 0x7fde9c1553a0 2023-12-19T18:28:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.985+0000 7fde9bfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde9c0cbcf0 con 0x7fde9c15ee30 2023-12-19T18:28:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.985+0000 7fde9b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fde9c0afeb0 0x7fde9c0c39d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47602/0 (socket says 172.21.15.27:47602) 2023-12-19T18:28:14.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.985+0000 7fde9b7fe640 1 -- 172.21.15.27:0/1247698279 learned_addr learned my addr 172.21.15.27:0/1247698279 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.986+0000 7fde9a7fc640 1 --2- 172.21.15.27:0/1247698279 >> v2:172.21.15.192:3300/0 conn(0x7fde9c15ee30 0x7fde9c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.986+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2263565917 0 0) 0x7fde9c0cb350 con 0x7fde9c0afeb0 2023-12-19T18:28:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.986+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde84003520 con 0x7fde9c0afeb0 2023-12-19T18:28:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.986+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3353655341 0 0) 0x7fde84003520 con 0x7fde9c0afeb0 2023-12-19T18:28:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.986+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 >> v2:172.21.15.192:3300/0 conn(0x7fde9c15ee30 msgr2=0x7fde9c0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.986+0000 7fde99ffb640 1 --2- 172.21.15.27:0/1247698279 >> v2:172.21.15.192:3300/0 conn(0x7fde9c15ee30 0x7fde9c0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.986+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 >> v1:172.21.15.192:6789/0 conn(0x7fde9c1553a0 legacy=0x7fde9c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.986+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c0cb910 con 0x7fde9c0afeb0 2023-12-19T18:28:14.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.986+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fde940019f0 con 0x7fde9c0afeb0 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.986+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 >> v1:172.21.15.27:6789/0 conn(0x7fde9c0afeb0 legacy=0x7fde9c0c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.986+0000 7fde99ffb640 1 --2- 172.21.15.27:0/1247698279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde840039c0 0x7fde84003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.986+0000 7fde99ffb640 1 --2- 172.21.15.27:0/1247698279 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde840047c0 0x7fde84004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.987+0000 7fde99ffb640 1 --2- 172.21.15.27:0/1247698279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde840051d0 0x7fde8400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.987+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c0cbcf0 con 0x7fde840051d0 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.987+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde8400c120 con 0x7fde840039c0 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.987+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde8400c2e0 con 0x7fde840047c0 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.987+0000 7fde9b7fe640 1 --2- 172.21.15.27:0/1247698279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde840051d0 0x7fde8400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.987+0000 7fde9affd640 1 --2- 172.21.15.27:0/1247698279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde840039c0 0x7fde84003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.987+0000 7fde9b7fe640 1 -- 172.21.15.27:0/1247698279 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde840047c0 msgr2=0x7fde84004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.987+0000 7fde9b7fe640 1 --2- 172.21.15.27:0/1247698279 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde840047c0 0x7fde84004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.987+0000 7fde9b7fe640 1 -- 172.21.15.27:0/1247698279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde840039c0 msgr2=0x7fde84003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.987+0000 7fde9b7fe640 1 --2- 172.21.15.27:0/1247698279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde840039c0 0x7fde84003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.987+0000 7fde9b7fe640 1 -- 172.21.15.27:0/1247698279 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde84003520 con 0x7fde840051d0 2023-12-19T18:28:14.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.987+0000 7fde9affd640 1 --2- 172.21.15.27:0/1247698279 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde840039c0 0x7fde84003de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.988+0000 7fde9b7fe640 1 -- 172.21.15.27:0/1247698279 reap_dead start 2023-12-19T18:28:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.988+0000 7fde9b7fe640 1 --2- 172.21.15.27:0/1247698279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde840051d0 0x7fde8400b980 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fde94002230 tx=0x7fde840057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c81fad9c0f17681 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.989+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94034630 con 0x7fde840051d0 2023-12-19T18:28:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.989+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9402f070 con 0x7fde840051d0 2023-12-19T18:28:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.989+0000 7fde99ffb640 1 -- 172.21.15.27:0/1247698279 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde940349d0 con 0x7fde840051d0 2023-12-19T18:28:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.989+0000 7fde9bfff640 1 -- 172.21.15.27:0/1247698279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde840051d0 msgr2=0x7fde8400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.989+0000 7fde9bfff640 1 --2- 172.21.15.27:0/1247698279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde840051d0 0x7fde8400b980 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7fde94002230 tx=0x7fde840057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.989+0000 7fde9bfff640 1 -- 172.21.15.27:0/1247698279 shutdown_connections 2023-12-19T18:28:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.989+0000 7fde9bfff640 1 --2- 172.21.15.27:0/1247698279 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde840051d0 0x7fde8400b980 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.989+0000 7fde9bfff640 1 -- 172.21.15.27:0/1247698279 >> 172.21.15.27:0/1247698279 conn(0x7fde9c069c70 msgr2=0x7fde9c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.989+0000 7fde9bfff640 1 -- 172.21.15.27:0/1247698279 shutdown_connections 2023-12-19T18:28:14.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.989+0000 7fde9bfff640 1 -- 172.21.15.27:0/1247698279 wait complete. 2023-12-19T18:28:14.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.991+0000 7fde9bfff640 1 Processor -- start 2023-12-19T18:28:14.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.991+0000 7fde9bfff640 1 -- start start 2023-12-19T18:28:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.992+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde840051d0 0x7fde9c147580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.992+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde9c0c5a10 0x7fde9c0c7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.992+0000 7fde9bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde9c0c5e90 0x7fde9c0c7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.992+0000 7fde9bfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c0c7ff0 con 0x7fde9c0c5e90 2023-12-19T18:28:14.992 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.992+0000 7fde9bfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c15d530 con 0x7fde9c0c5a10 2023-12-19T18:28:14.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.992+0000 7fde9bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde9c15d810 con 0x7fde840051d0 2023-12-19T18:28:14.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.993+0000 7fde9b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde9c0c5e90 0x7fde9c0c7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.993+0000 7fde9b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde9c0c5e90 0x7fde9c0c7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42306/0 (socket says 172.21.15.27:42306) 2023-12-19T18:28:14.993 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.993+0000 7fde9b7fe640 1 -- 172.21.15.27:0/2851765627 learned_addr learned my addr 172.21.15.27:0/2851765627 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.993+0000 7fde9a7fc640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde9c0c5a10 0x7fde9c0c7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.993+0000 7fde9b7fe640 1 -- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde840051d0 msgr2=0x7fde9c147580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.993+0000 7fde9affd640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde840051d0 0x7fde9c147580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.993+0000 7fde9b7fe640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde840051d0 0x7fde9c147580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.993+0000 7fde9b7fe640 1 -- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde9c0c5a10 msgr2=0x7fde9c0c7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.993+0000 7fde9b7fe640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde9c0c5a10 0x7fde9c0c7570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.993+0000 7fde9b7fe640 1 -- 172.21.15.27:0/2851765627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde9c15daf0 con 0x7fde9c0c5e90 2023-12-19T18:28:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.994+0000 7fde9b7fe640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde9c0c5e90 0x7fde9c0c7ab0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fde94000f80 tx=0x7fde94001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.994+0000 7fde837fe640 1 -- 172.21.15.27:0/2851765627 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde94004d60 con 0x7fde9c0c5e90 2023-12-19T18:28:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.994+0000 7fde837fe640 1 -- 172.21.15.27:0/2851765627 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde9403f470 con 0x7fde9c0c5e90 2023-12-19T18:28:14.994 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.994+0000 7fde837fe640 1 -- 172.21.15.27:0/2851765627 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde9403f820 con 0x7fde9c0c5e90 2023-12-19T18:28:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.994+0000 7fde9bfff640 1 -- 172.21.15.27:0/2851765627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde9c15ddd0 con 0x7fde9c0c5e90 2023-12-19T18:28:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.994+0000 7fde9bfff640 1 -- 172.21.15.27:0/2851765627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde9c1518c0 con 0x7fde9c0c5e90 2023-12-19T18:28:14.995 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.995+0000 7fde9bfff640 1 -- 172.21.15.27:0/2851765627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde9c04e840 con 0x7fde9c0c5e90 2023-12-19T18:28:14.996 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.996+0000 7fde837fe640 1 -- 172.21.15.27:0/2851765627 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fde94031030 con 0x7fde9c0c5e90 2023-12-19T18:28:14.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.997+0000 7fde837fe640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde8806a460 0x7fde8806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:14.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.997+0000 7fde837fe640 1 -- 172.21.15.27:0/2851765627 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde940a9e40 con 0x7fde9c0c5e90 2023-12-19T18:28:14.997 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.998+0000 7fde9affd640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde8806a460 0x7fde8806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:14.998 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:14.999+0000 7fde9affd640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde8806a460 0x7fde8806c920 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fde8c001630 tx=0x7fde8c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:15.001 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.002+0000 7fde837fe640 1 -- 172.21.15.27:0/2851765627 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fde94075420 con 0x7fde9c0c5e90 2023-12-19T18:28:15.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.238+0000 7fde9bfff640 1 -- 172.21.15.27:0/2851765627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fde9c0b4820 con 0x7fde9c0c5e90 2023-12-19T18:28:15.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.239+0000 7fde837fe640 1 -- 172.21.15.27:0/2851765627 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4713 (secure 0 0 0) 0x7fde94078a40 con 0x7fde9c0c5e90 2023-12-19T18:28:15.239 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:15.239 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1864159643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1864159643},{"type":"v1","addr":"172.21.15.192:6836","nonce":1864159643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1348219767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1348219767},{"type":"v1","addr":"172.21.15.192:6837","nonce":1348219767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/181137639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":181137639},{"type":"v1","addr":"172.21.15.27:6833","nonce":181137639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:28:08.577742+0000","modified":"2023-12-19T18:28:12.083302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/152568921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":152568921},{"type":"v1","addr":"172.21.15.27:6835","nonce":152568921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:28:15.240 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 85 2023-12-19T18:28:15.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.242+0000 7fde9bfff640 1 -- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde8806a460 msgr2=0x7fde8806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:15.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.242+0000 7fde9bfff640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde8806a460 0x7fde8806c920 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fde8c001630 tx=0x7fde8c009040 comp rx=0 tx=0).stop 2023-12-19T18:28:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.242+0000 7fde9bfff640 1 -- 172.21.15.27:0/2851765627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde9c0c5e90 msgr2=0x7fde9c0c7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.242+0000 7fde9bfff640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde9c0c5e90 0x7fde9c0c7ab0 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7fde94000f80 tx=0x7fde94001cb0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.242+0000 7fde9bfff640 1 -- 172.21.15.27:0/2851765627 shutdown_connections 2023-12-19T18:28:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.242+0000 7fde9bfff640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fde8806a460 0x7fde8806c920 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.242+0000 7fde9bfff640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fde9c0c5e90 0x7fde9c0c7ab0 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.242+0000 7fde9bfff640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fde9c0c5a10 0x7fde9c0c7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.242+0000 7fde9bfff640 1 --2- 172.21.15.27:0/2851765627 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fde840051d0 0x7fde9c147580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.242+0000 7fde9bfff640 1 -- 172.21.15.27:0/2851765627 >> 172.21.15.27:0/2851765627 conn(0x7fde9c069c70 msgr2=0x7fde9c0c4940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.242+0000 7fde9bfff640 1 -- 172.21.15.27:0/2851765627 shutdown_connections 2023-12-19T18:28:15.243 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.242+0000 7fde9bfff640 1 -- 172.21.15.27:0/2851765627 wait complete. 2023-12-19T18:28:15.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 85, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-19T18:28:08.577742+0000', 'modified': '2023-12-19T18:28:12.083302+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8997}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8997': {'gid': 8997, 'name': 'c', 'rank': 0, 'incarnation': 81, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.27:6835/152568921', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.27:6834', 'nonce': 152568921}, {'type': 'v1', 'addr': '172.21.15.27:6835', 'nonce': 152568921}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-19T18:28:15.663 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-19T18:28:15.664 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-19T18:28:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.911+0000 7fc8a129e640 1 Processor -- start 2023-12-19T18:28:15.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.911+0000 7fc8a129e640 1 -- start start 2023-12-19T18:28:15.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.912+0000 7fc8a129e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc89c1540e0 0x7fc89c160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.912+0000 7fc8a129e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c06c6e0 0x7fc89c15dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.913+0000 7fc8a129e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc89c15d600 con 0x7fc89c15e2e0 2023-12-19T18:28:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.913+0000 7fc8a129e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc89c161130 con 0x7fc89c1540e0 2023-12-19T18:28:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.913+0000 7fc8a129e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1613d0 con 0x7fc89c06c6e0 2023-12-19T18:28:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.913+0000 7fc89bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc89c1540e0 0x7fc89c160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.913+0000 7fc89b7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c06c6e0 0x7fc89c15dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.913+0000 7fc89bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc89c1540e0 0x7fc89c160bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:58696/0 (socket says 172.21.15.27:58696) 2023-12-19T18:28:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.913+0000 7fc89bfff640 1 -- 172.21.15.27:0/2264186509 learned_addr learned my addr 172.21.15.27:0/2264186509 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.914+0000 7fc89bfff640 1 -- 172.21.15.27:0/2264186509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c06c6e0 msgr2=0x7fc89c15dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.914+0000 7fc89bfff640 1 --2- 172.21.15.27:0/2264186509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c06c6e0 0x7fc89c15dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.914+0000 7fc89bfff640 1 -- 172.21.15.27:0/2264186509 >> v1:172.21.15.192:6789/0 conn(0x7fc89c15e2e0 legacy=0x7fc89c06b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:15.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.914+0000 7fc89bfff640 1 -- 172.21.15.27:0/2264186509 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c1616b0 con 0x7fc89c1540e0 2023-12-19T18:28:15.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.915+0000 7fc89bfff640 1 --2- 172.21.15.27:0/2264186509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c1540e0 0x7fc89c160bf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009870 tx=0x7fc88c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72c7f2c57f08fba server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.916+0000 7fc89affd640 1 -- 172.21.15.27:0/2264186509 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c030840 con 0x7fc89c1540e0 2023-12-19T18:28:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.916+0000 7fc89affd640 1 -- 172.21.15.27:0/2264186509 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c0309a0 con 0x7fc89c1540e0 2023-12-19T18:28:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.916+0000 7fc89affd640 1 -- 172.21.15.27:0/2264186509 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c030d30 con 0x7fc89c1540e0 2023-12-19T18:28:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.916+0000 7fc8a129e640 1 -- 172.21.15.27:0/2264186509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c1540e0 msgr2=0x7fc89c160bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.916+0000 7fc8a129e640 1 --2- 172.21.15.27:0/2264186509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c1540e0 0x7fc89c160bf0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009870 tx=0x7fc88c02b980 comp rx=0 tx=0).stop 2023-12-19T18:28:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.917+0000 7fc8a129e640 1 -- 172.21.15.27:0/2264186509 shutdown_connections 2023-12-19T18:28:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.917+0000 7fc8a129e640 1 --2- 172.21.15.27:0/2264186509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c06c6e0 0x7fc89c15dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.917+0000 7fc8a129e640 1 --2- 172.21.15.27:0/2264186509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c1540e0 0x7fc89c160bf0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.917+0000 7fc8a129e640 1 -- 172.21.15.27:0/2264186509 >> 172.21.15.27:0/2264186509 conn(0x7fc89c06a7d0 msgr2=0x7fc89c0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.917+0000 7fc8a129e640 1 -- 172.21.15.27:0/2264186509 shutdown_connections 2023-12-19T18:28:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.917+0000 7fc8a129e640 1 -- 172.21.15.27:0/2264186509 wait complete. 2023-12-19T18:28:15.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.918+0000 7fc8a129e640 1 Processor -- start 2023-12-19T18:28:15.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.919+0000 7fc8a129e640 1 -- start start 2023-12-19T18:28:15.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.920+0000 7fc8a129e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c06c6e0 0x7fc89c0b00b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.920+0000 7fc8a129e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c1540e0 0x7fc89c0c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.920+0000 7fc8a129e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c15e2e0 0x7fc89c0c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.920+0000 7fc8a129e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1f6840 con 0x7fc89c15e2e0 2023-12-19T18:28:15.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.920+0000 7fc8a129e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1f5c50 con 0x7fc89c06c6e0 2023-12-19T18:28:15.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.920+0000 7fc8a129e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc89c1f5ea0 con 0x7fc89c1540e0 2023-12-19T18:28:15.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.920+0000 7fc8a0a9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c15e2e0 0x7fc89c0c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.920+0000 7fc8a0a9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c15e2e0 0x7fc89c0c9420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42312/0 (socket says 172.21.15.27:42312) 2023-12-19T18:28:15.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.920+0000 7fc8a0a9d640 1 -- 172.21.15.27:0/2285175995 learned_addr learned my addr 172.21.15.27:0/2285175995 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:15.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.920+0000 7fc89bfff640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c06c6e0 0x7fc89c0b00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.920+0000 7fc89b7fe640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c1540e0 0x7fc89c0c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.921+0000 7fc8a0a9d640 1 -- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c1540e0 msgr2=0x7fc89c0c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:15.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.921+0000 7fc8a0a9d640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c1540e0 0x7fc89c0c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.921+0000 7fc8a0a9d640 1 -- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c06c6e0 msgr2=0x7fc89c0b00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:15.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.921+0000 7fc8a0a9d640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c06c6e0 0x7fc89c0b00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.921+0000 7fc8a0a9d640 1 -- 172.21.15.27:0/2285175995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88c0094d0 con 0x7fc89c15e2e0 2023-12-19T18:28:15.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.921+0000 7fc8a0a9d640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c15e2e0 0x7fc89c0c9420 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fc89400c7a0 tx=0x7fc89400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:15.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.921+0000 7fc898ff9640 1 -- 172.21.15.27:0/2285175995 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894003b70 con 0x7fc89c15e2e0 2023-12-19T18:28:15.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.921+0000 7fc898ff9640 1 -- 172.21.15.27:0/2285175995 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc894003cf0 con 0x7fc89c15e2e0 2023-12-19T18:28:15.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.921+0000 7fc898ff9640 1 -- 172.21.15.27:0/2285175995 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894003ea0 con 0x7fc89c15e2e0 2023-12-19T18:28:15.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.921+0000 7fc8a129e640 1 -- 172.21.15.27:0/2285175995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc89c1f6180 con 0x7fc89c15e2e0 2023-12-19T18:28:15.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.922+0000 7fc8a129e640 1 -- 172.21.15.27:0/2285175995 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c0b05f0 con 0x7fc89c15e2e0 2023-12-19T18:28:15.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.924+0000 7fc898ff9640 1 -- 172.21.15.27:0/2285175995 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc89400aa90 con 0x7fc89c15e2e0 2023-12-19T18:28:15.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.925+0000 7fc898ff9640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc88806a460 0x7fc88806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.925+0000 7fc898ff9640 1 -- 172.21.15.27:0/2285175995 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8940886d0 con 0x7fc89c15e2e0 2023-12-19T18:28:15.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.925+0000 7fc89bfff640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc88806a460 0x7fc88806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.925+0000 7fc8a5078740 1 Processor -- start 2023-12-19T18:28:15.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.926+0000 7fc8a5078740 1 -- start start 2023-12-19T18:28:15.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.926+0000 7fc89bfff640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc88806a460 0x7fc88806c920 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc88c036000 tx=0x7fc88c03b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.926+0000 7fc8a5078740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf81e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc8a5078740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf7ee090 0x55baaf81b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc8a5078740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 0x55baaf81b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc8a5078740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baaf7c9650 con 0x55baaf81e110 2023-12-19T18:28:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc8a5078740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baaf75e000 con 0x55baaf7ee090 2023-12-19T18:28:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc8a5078740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55baaf75b790 con 0x55baaf7f1c90 2023-12-19T18:28:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc89bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf81e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc89bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf81e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42324/0 (socket says 172.21.15.27:42324) 2023-12-19T18:28:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc89bfff640 1 -- 172.21.15.27:0/3870120182 learned_addr learned my addr 172.21.15.27:0/3870120182 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc89b7fe640 1 --2- 172.21.15.27:0/3870120182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf7ee090 0x55baaf81b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc8a0a9d640 1 --2- 172.21.15.27:0/3870120182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 0x55baaf81b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc89bfff640 1 -- 172.21.15.27:0/3870120182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 msgr2=0x55baaf81b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:15.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc89bfff640 1 --2- 172.21.15.27:0/3870120182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 0x55baaf81b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc89bfff640 1 -- 172.21.15.27:0/3870120182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf7ee090 msgr2=0x55baaf81b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc89bfff640 1 --2- 172.21.15.27:0/3870120182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf7ee090 0x55baaf81b380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.927+0000 7fc89bfff640 1 -- 172.21.15.27:0/3870120182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55baaf71e710 con 0x55baaf81e110 2023-12-19T18:28:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.928+0000 7fc89bfff640 1 --2- 172.21.15.27:0/3870120182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf81e4f0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0090b0 tx=0x7fc88c030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99ac5de99fe44cd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.928+0000 7fc881ffb640 1 -- 172.21.15.27:0/3870120182 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c044d80 con 0x55baaf81e110 2023-12-19T18:28:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.928+0000 7fc881ffb640 1 -- 172.21.15.27:0/3870120182 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c04a040 con 0x55baaf81e110 2023-12-19T18:28:15.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.928+0000 7fc881ffb640 1 -- 172.21.15.27:0/3870120182 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c039690 con 0x55baaf81e110 2023-12-19T18:28:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.929+0000 7fc8a5078740 1 -- 172.21.15.27:0/3870120182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 msgr2=0x55baaf81e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.929+0000 7fc8a5078740 1 --2- 172.21.15.27:0/3870120182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf81e4f0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7fc88c0090b0 tx=0x7fc88c030000 comp rx=0 tx=0).stop 2023-12-19T18:28:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.929+0000 7fc8a5078740 1 -- 172.21.15.27:0/3870120182 shutdown_connections 2023-12-19T18:28:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.929+0000 7fc8a5078740 1 --2- 172.21.15.27:0/3870120182 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 0x55baaf81b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.929+0000 7fc8a5078740 1 --2- 172.21.15.27:0/3870120182 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf7ee090 0x55baaf81b380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.929+0000 7fc8a5078740 1 --2- 172.21.15.27:0/3870120182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf81e4f0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.929+0000 7fc8a5078740 1 -- 172.21.15.27:0/3870120182 >> 172.21.15.27:0/3870120182 conn(0x55baaf7f03a0 msgr2=0x55baaf838f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.929+0000 7fc8a5078740 1 -- 172.21.15.27:0/3870120182 shutdown_connections 2023-12-19T18:28:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.929+0000 7fc8a5078740 1 -- 172.21.15.27:0/3870120182 wait complete. 2023-12-19T18:28:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.930+0000 7fc8a5078740 1 -- start start 2023-12-19T18:28:15.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.930+0000 7fc8a5078740 1 Processor -- start 2023-12-19T18:28:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.931+0000 7fc8a5078740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 0x55baaf8189c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.931+0000 7fc8a5078740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf816b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.931+0000 7fc8a5078740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf8132c0 0x55baaf802110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.931+0000 7fc8a5078740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baaf757080 con 0x55baaf81e110 2023-12-19T18:28:15.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.931+0000 7fc8a5078740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baaf6f8850 con 0x55baaf8132c0 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.931+0000 7fc8a5078740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55baaf6f84f0 con 0x55baaf7f1c90 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.931+0000 7fc8a5078740 10 client.0 _ll_register_callbacks cb 0x55baaf7509c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.931+0000 7fc89b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf816b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.931+0000 7fc89b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf816b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42334/0 (socket says 172.21.15.27:42334) 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.931+0000 7fc89b7fe640 1 -- 172.21.15.27:0/2166312908 learned_addr learned my addr 172.21.15.27:0/2166312908 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.932+0000 7fc8a0a9d640 1 --2- 172.21.15.27:0/2166312908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf8132c0 0x55baaf802110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.932+0000 7fc89bfff640 1 --2- 172.21.15.27:0/2166312908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 0x55baaf8189c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.932+0000 7fc89b7fe640 1 -- 172.21.15.27:0/2166312908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 msgr2=0x55baaf8189c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.932+0000 7fc89b7fe640 1 --2- 172.21.15.27:0/2166312908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 0x55baaf8189c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.932+0000 7fc89b7fe640 1 -- 172.21.15.27:0/2166312908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf8132c0 msgr2=0x55baaf802110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.932+0000 7fc89b7fe640 1 --2- 172.21.15.27:0/2166312908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf8132c0 0x55baaf802110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.932+0000 7fc89b7fe640 1 -- 172.21.15.27:0/2166312908 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c1616b0 con 0x55baaf81e110 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.932+0000 7fc89b7fe640 1 --2- 172.21.15.27:0/2166312908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf816b50 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fc890009cb0 tx=0x7fc8900136b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=823073deed4eaae6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.932+0000 7fc87affd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:28:15.934 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.932+0000 7fc87affd640 1 -- 172.21.15.27:0/2166312908 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8900095d0 con 0x55baaf81e110 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.932+0000 7fc87affd640 1 -- 172.21.15.27:0/2166312908 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc890016460 con 0x55baaf81e110 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.933+0000 7fc8a5078740 1 -- 172.21.15.27:0/2166312908 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55baaf7815d0 con 0x55baaf81e110 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.933+0000 7fc87affd640 1 -- 172.21.15.27:0/2166312908 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc890005990 con 0x55baaf81e110 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.933+0000 7fc8a5078740 10 client.9138 fetch_fsmap learned FSMap version 85 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.933+0000 7fc8a5078740 1 -- 172.21.15.27:0/2166312908 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55baaf755700 con 0x55baaf81e110 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.933+0000 7fc87affd640 1 -- 172.21.15.27:0/2166312908 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55baaf7815d0 con 0x55baaf81e110 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.933+0000 7fc87affd640 1 -- 172.21.15.27:0/2166312908 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 85) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc890005af0 con 0x55baaf81e110 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.933+0000 7fc8a5078740 10 client.9138 fetch_fsmap finished waiting for FSMap version 85 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.933+0000 7fc8a5078740 10 client.9138 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.933+0000 7fc8a5078740 20 client.9138 populate_metadata read hostname 'smithi027' 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.933+0000 7fc8a5078740 1 --2- 172.21.15.27:0/2166312908 >> [v2:172.21.15.27:6834/152568921,v1:172.21.15.27:6835/152568921] conn(0x55baaf8041a0 0x55baaf804580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.933+0000 7fc8a5078740 4 client.9138 mds_command: new command op to 8997 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.933+0000 7fc8a5078740 1 -- 172.21.15.27:0/2166312908 --> [v2:172.21.15.27:6834/152568921,v1:172.21.15.27:6835/152568921] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55baaf7498d0 con 0x55baaf8041a0 2023-12-19T18:28:15.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.934+0000 7fc89bfff640 1 --2- 172.21.15.27:0/2166312908 >> [v2:172.21.15.27:6834/152568921,v1:172.21.15.27:6835/152568921] conn(0x55baaf8041a0 0x55baaf804580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.934+0000 7fc89bfff640 1 --2- 172.21.15.27:0/2166312908 >> [v2:172.21.15.27:6834/152568921,v1:172.21.15.27:6835/152568921] conn(0x55baaf8041a0 0x55baaf804580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb1b7aaf25bd18f1 server_cookie=8fda210d33f993db in_seq=0 out_seq=0 2023-12-19T18:28:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.934+0000 7fc87affd640 10 client.9138 ms_handle_connect on v2:172.21.15.27:6834/152568921 2023-12-19T18:28:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.935+0000 7fc87affd640 1 -- 172.21.15.27:0/2166312908 <== mds.0 v2:172.21.15.27:6834/152568921 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55baaf7498d0 con 0x55baaf8041a0 2023-12-19T18:28:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.935+0000 7fc87affd640 10 client.9138 handle_command_reply: tid=0 2023-12-19T18:28:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.935+0000 7fc8a5078740 1 client.9138 shutdown 2023-12-19T18:28:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.935+0000 7fc8a5078740 1 -- 172.21.15.27:0/2166312908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 msgr2=0x55baaf816b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.935+0000 7fc8a5078740 1 --2- 172.21.15.27:0/2166312908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf816b50 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7fc890009cb0 tx=0x7fc8900136b0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.935+0000 7fc8a5078740 1 -- 172.21.15.27:0/2166312908 shutdown_connections 2023-12-19T18:28:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.935+0000 7fc8a5078740 1 --2- 172.21.15.27:0/2166312908 >> [v2:172.21.15.27:6834/152568921,v1:172.21.15.27:6835/152568921] conn(0x55baaf8041a0 0x55baaf804580 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.935+0000 7fc8a5078740 1 --2- 172.21.15.27:0/2166312908 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf8132c0 0x55baaf802110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.935+0000 7fc8a5078740 1 --2- 172.21.15.27:0/2166312908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf816b50 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.935+0000 7fc8a5078740 1 --2- 172.21.15.27:0/2166312908 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 0x55baaf8189c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.935+0000 7fc8a5078740 1 -- 172.21.15.27:0/2166312908 >> 172.21.15.27:0/2166312908 conn(0x55baaf7ee090 msgr2=0x55baaf7f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.935+0000 7fc87affd640 0 client.9138 ms_handle_reset on v2:172.21.15.27:6834/152568921 2023-12-19T18:28:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.936+0000 7fc8a5078740 1 -- 172.21.15.27:0/2166312908 shutdown_connections 2023-12-19T18:28:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.936+0000 7fc8a5078740 1 -- 172.21.15.27:0/2166312908 wait complete. 2023-12-19T18:28:15.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.936+0000 7fc8a5078740 20 client.9138 trim_cache size 0 max 16384 2023-12-19T18:28:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.952+0000 7fc8a5078740 1 Processor -- start 2023-12-19T18:28:15.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.952+0000 7fc8a5078740 1 -- start start 2023-12-19T18:28:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.953+0000 7fc8a5078740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 0x55baaf801ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.953+0000 7fc8a5078740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf8132c0 0x55baaf7ef0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.953+0000 7fc8a5078740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf81aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.953 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.953+0000 7fc8a5078740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baaf4bef20 con 0x55baaf81e110 2023-12-19T18:28:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.953+0000 7fc8a5078740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baaf812670 con 0x55baaf8132c0 2023-12-19T18:28:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.953+0000 7fc8a5078740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55baaf6f78a0 con 0x55baaf7f1c90 2023-12-19T18:28:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.954+0000 7fc8a0a9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf81aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.954+0000 7fc8a0a9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf81aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42350/0 (socket says 172.21.15.27:42350) 2023-12-19T18:28:15.954 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.954+0000 7fc8a0a9d640 1 -- 172.21.15.27:0/2772331470 learned_addr learned my addr 172.21.15.27:0/2772331470 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:15.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.954+0000 7fc89bfff640 1 --2- 172.21.15.27:0/2772331470 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 0x55baaf801ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.954+0000 7fc8a0a9d640 1 -- 172.21.15.27:0/2772331470 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 msgr2=0x55baaf801ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:15.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.954+0000 7fc8a0a9d640 1 --2- 172.21.15.27:0/2772331470 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 0x55baaf801ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.954+0000 7fc89b7fe640 1 --2- 172.21.15.27:0/2772331470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf8132c0 0x55baaf7ef0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.954+0000 7fc8a0a9d640 1 -- 172.21.15.27:0/2772331470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf8132c0 msgr2=0x55baaf7ef0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.954+0000 7fc8a0a9d640 1 --2- 172.21.15.27:0/2772331470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf8132c0 0x55baaf7ef0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.954+0000 7fc8a0a9d640 1 -- 172.21.15.27:0/2772331470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55baaf78b280 con 0x55baaf81e110 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.954+0000 7fc8a0a9d640 1 --2- 172.21.15.27:0/2772331470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf81aec0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fc894053f10 tx=0x7fc894057370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd79ccea195154d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.955+0000 7fc8827fc640 1 -- 172.21.15.27:0/2772331470 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89405b840 con 0x55baaf81e110 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.955+0000 7fc8827fc640 1 -- 172.21.15.27:0/2772331470 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc89405f9f0 con 0x55baaf81e110 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.955+0000 7fc8827fc640 1 -- 172.21.15.27:0/2772331470 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89400ad40 con 0x55baaf81e110 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.955+0000 7fc8a5078740 1 -- 172.21.15.27:0/2772331470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 msgr2=0x55baaf81aec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.955+0000 7fc8a5078740 1 --2- 172.21.15.27:0/2772331470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf81aec0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7fc894053f10 tx=0x7fc894057370 comp rx=0 tx=0).stop 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.955+0000 7fc8a5078740 1 -- 172.21.15.27:0/2772331470 shutdown_connections 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.955+0000 7fc8a5078740 1 --2- 172.21.15.27:0/2772331470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf81e110 0x55baaf81aec0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.955+0000 7fc8a5078740 1 --2- 172.21.15.27:0/2772331470 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf8132c0 0x55baaf7ef0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.955+0000 7fc8a5078740 1 --2- 172.21.15.27:0/2772331470 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf7f1c90 0x55baaf801ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.956 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.955+0000 7fc8a5078740 1 -- 172.21.15.27:0/2772331470 >> 172.21.15.27:0/2772331470 conn(0x55baaf7ee090 msgr2=0x55baaf813720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.955+0000 7fc8a5078740 1 -- 172.21.15.27:0/2772331470 shutdown_connections 2023-12-19T18:28:15.957 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.955+0000 7fc8a5078740 1 -- 172.21.15.27:0/2772331470 wait complete. 2023-12-19T18:28:15.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.956+0000 7fc8a5078740 1 -- start start 2023-12-19T18:28:15.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.956+0000 7fc8a5078740 1 Processor -- start 2023-12-19T18:28:15.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.957+0000 7fc8a5078740 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf7f1c90 0x55baaf7f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.957+0000 7fc8a5078740 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf8132c0 0x55baaf81a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.957+0000 7fc8a5078740 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf81e110 0x55baaf84ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.957+0000 7fc8a5078740 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baaf7f38e0 con 0x55baaf7f1c90 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.957+0000 7fc8a5078740 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x55baaf890180 con 0x55baaf81e110 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.957+0000 7fc8a5078740 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x55baaf819fc0 con 0x55baaf8132c0 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.957+0000 7fc8a5078740 10 client.0 _ll_register_callbacks cb 0x7fc890009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.958+0000 7fc89bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf7f1c90 0x55baaf7f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.958+0000 7fc89bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf7f1c90 0x55baaf7f2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42352/0 (socket says 172.21.15.27:42352) 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.958+0000 7fc89bfff640 1 -- 172.21.15.27:0/1856320045 learned_addr learned my addr 172.21.15.27:0/1856320045 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.958+0000 7fc8a0a9d640 1 --2- 172.21.15.27:0/1856320045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf81e110 0x55baaf84ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.958+0000 7fc89bfff640 1 -- 172.21.15.27:0/1856320045 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf8132c0 msgr2=0x55baaf81a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.958+0000 7fc89bfff640 1 --2- 172.21.15.27:0/1856320045 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf8132c0 0x55baaf81a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.958+0000 7fc89bfff640 1 -- 172.21.15.27:0/1856320045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf81e110 msgr2=0x55baaf84ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.958+0000 7fc89bfff640 1 --2- 172.21.15.27:0/1856320045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf81e110 0x55baaf84ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.958+0000 7fc89bfff640 1 -- 172.21.15.27:0/1856320045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc89c0b05f0 con 0x55baaf7f1c90 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.958+0000 7fc89bfff640 1 --2- 172.21.15.27:0/1856320045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf7f1c90 0x55baaf7f2ed0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009990 tx=0x7fc88c02e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18f4e5cc08622c38 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.958+0000 7fc87affd640 10 client.0 ms_handle_connect on v2:172.21.15.27:3300/0 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.958+0000 7fc87affd640 1 -- 172.21.15.27:0/1856320045 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc88c05d040 con 0x55baaf7f1c90 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.959+0000 7fc87affd640 1 -- 172.21.15.27:0/1856320045 <== mon.0 v2:172.21.15.27:3300/0 2 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc88c05c040 con 0x55baaf7f1c90 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.959+0000 7fc8a5078740 1 -- 172.21.15.27:0/1856320045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc88c04a040 con 0x55baaf7f1c90 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.959+0000 7fc87affd640 1 -- 172.21.15.27:0/1856320045 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc88c044e70 con 0x55baaf7f1c90 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.959+0000 7fc8a5078740 10 client.9144 fetch_fsmap learned FSMap version 85 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.959+0000 7fc8a5078740 1 -- 172.21.15.27:0/1856320045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,fsmap=85,monmap=2+}) v3 -- 0x55baaf6f7a60 con 0x55baaf7f1c90 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.959+0000 7fc87affd640 1 -- 172.21.15.27:0/1856320045 <== mon.0 v2:172.21.15.27:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc88c04a040 con 0x55baaf7f1c90 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.959+0000 7fc87affd640 1 -- 172.21.15.27:0/1856320045 <== mon.0 v2:172.21.15.27:3300/0 5 ==== fsmap(e 85) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc88c04d070 con 0x55baaf7f1c90 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.959+0000 7fc8a5078740 10 client.9144 fetch_fsmap finished waiting for FSMap version 85 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.959+0000 7fc8a5078740 10 client.9144 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.959+0000 7fc8a5078740 20 client.9144 populate_metadata read hostname 'smithi027' 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.959+0000 7fc8a5078740 1 --2- 172.21.15.27:0/1856320045 >> [v2:172.21.15.27:6834/152568921,v1:172.21.15.27:6835/152568921] conn(0x55baaf817cc0 0x55baaf84b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:15.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.959+0000 7fc8a5078740 4 client.9144 mds_command: new command op to 8997 tid=0[{"prefix": "status"}] 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.959+0000 7fc8a5078740 1 -- 172.21.15.27:0/1856320045 --> [v2:172.21.15.27:6834/152568921,v1:172.21.15.27:6835/152568921] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55baaf750100 con 0x55baaf817cc0 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.960+0000 7fc89b7fe640 1 --2- 172.21.15.27:0/1856320045 >> [v2:172.21.15.27:6834/152568921,v1:172.21.15.27:6835/152568921] conn(0x55baaf817cc0 0x55baaf84b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.960+0000 7fc89b7fe640 1 --2- 172.21.15.27:0/1856320045 >> [v2:172.21.15.27:6834/152568921,v1:172.21.15.27:6835/152568921] conn(0x55baaf817cc0 0x55baaf84b290 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1db3a8af8567ebf server_cookie=7d9aa1e26f8b7c0e in_seq=0 out_seq=0 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.960+0000 7fc87affd640 10 client.9144 ms_handle_connect on v2:172.21.15.27:6834/152568921 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.960+0000 7fc87affd640 1 -- 172.21.15.27:0/1856320045 <== mds.0 v2:172.21.15.27:6834/152568921 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55baaf750100 con 0x55baaf817cc0 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.960+0000 7fc87affd640 10 client.9144 handle_command_reply: tid=0 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.960+0000 7fc8a5078740 1 client.9144 shutdown 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.961+0000 7fc8a5078740 1 -- 172.21.15.27:0/1856320045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf7f1c90 msgr2=0x55baaf7f2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.961+0000 7fc8a5078740 1 --2- 172.21.15.27:0/1856320045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf7f1c90 0x55baaf7f2ed0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009990 tx=0x7fc88c02e000 comp rx=0 tx=0).stop 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.961+0000 7fc8a5078740 1 -- 172.21.15.27:0/1856320045 shutdown_connections 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.961+0000 7fc8a5078740 1 --2- 172.21.15.27:0/1856320045 >> [v2:172.21.15.27:6834/152568921,v1:172.21.15.27:6835/152568921] conn(0x55baaf817cc0 0x55baaf84b290 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.961+0000 7fc8a5078740 1 --2- 172.21.15.27:0/1856320045 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x55baaf81e110 0x55baaf84ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.961+0000 7fc8a5078740 1 --2- 172.21.15.27:0/1856320045 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x55baaf8132c0 0x55baaf81a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.961+0000 7fc8a5078740 1 --2- 172.21.15.27:0/1856320045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x55baaf7f1c90 0x55baaf7f2ed0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.961+0000 7fc87affd640 0 client.9144 ms_handle_reset on v2:172.21.15.27:6834/152568921 2023-12-19T18:28:15.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.961+0000 7fc8a5078740 1 -- 172.21.15.27:0/1856320045 >> 172.21.15.27:0/1856320045 conn(0x55baaf7ee090 msgr2=0x55baaf7ed250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:15.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.961+0000 7fc8a5078740 1 -- 172.21.15.27:0/1856320045 shutdown_connections 2023-12-19T18:28:15.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.961+0000 7fc8a5078740 1 -- 172.21.15.27:0/1856320045 wait complete. 2023-12-19T18:28:15.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.961+0000 7fc8a5078740 20 client.9144 trim_cache size 0 max 16384 2023-12-19T18:28:15.968 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:28:15.968 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:28:15.968 INFO:teuthology.orchestra.run.smithi027.stdout: "whoami": 0, 2023-12-19T18:28:15.968 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 8997, 2023-12-19T18:28:15.968 INFO:teuthology.orchestra.run.smithi027.stdout: "want_state": "up:active", 2023-12-19T18:28:15.968 INFO:teuthology.orchestra.run.smithi027.stdout: "state": "up:active", 2023-12-19T18:28:15.968 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs", 2023-12-19T18:28:15.969 INFO:teuthology.orchestra.run.smithi027.stdout: "rank_uptime": 7.3801057639999996, 2023-12-19T18:28:15.969 INFO:teuthology.orchestra.run.smithi027.stdout: "mdsmap_epoch": 85, 2023-12-19T18:28:15.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch": 202, 2023-12-19T18:28:15.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osdmap_epoch_barrier": 202, 2023-12-19T18:28:15.969 INFO:teuthology.orchestra.run.smithi027.stdout: "uptime": 13.174012086999999 2023-12-19T18:28:15.969 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:28:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.962+0000 7fc8a129e640 1 -- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc88806a460 msgr2=0x7fc88806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.962+0000 7fc8a129e640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc88806a460 0x7fc88806c920 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fc88c036000 tx=0x7fc88c03b040 comp rx=0 tx=0).stop 2023-12-19T18:28:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.962+0000 7fc8a129e640 1 -- 172.21.15.27:0/2285175995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c15e2e0 msgr2=0x7fc89c0c9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.962+0000 7fc8a129e640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c15e2e0 0x7fc89c0c9420 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7fc89400c7a0 tx=0x7fc89400cc60 comp rx=0 tx=0).stop 2023-12-19T18:28:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.962+0000 7fc8a129e640 1 -- 172.21.15.27:0/2285175995 shutdown_connections 2023-12-19T18:28:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.962+0000 7fc8a129e640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc88806a460 0x7fc88806c920 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.962+0000 7fc8a129e640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc89c15e2e0 0x7fc89c0c9420 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.962+0000 7fc8a129e640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc89c1540e0 0x7fc89c0c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.962+0000 7fc8a129e640 1 --2- 172.21.15.27:0/2285175995 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc89c06c6e0 0x7fc89c0b00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.962+0000 7fc8a129e640 1 -- 172.21.15.27:0/2285175995 >> 172.21.15.27:0/2285175995 conn(0x7fc89c06a7d0 msgr2=0x7fc89c160740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.962+0000 7fc8a129e640 1 -- 172.21.15.27:0/2285175995 shutdown_connections 2023-12-19T18:28:15.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:15.962+0000 7fc8a129e640 1 -- 172.21.15.27:0/2285175995 wait complete. 2023-12-19T18:28:15.979 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:28:16.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.210+0000 7f339e696640 1 Processor -- start 2023-12-19T18:28:16.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.210+0000 7f339e696640 1 -- start start 2023-12-19T18:28:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.211+0000 7f339e696640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f33980cea40 0x7f33980d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.212+0000 7f339e696640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f339815e4c0 con 0x7f339815ae80 2023-12-19T18:28:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.212+0000 7f339e696640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f339806c940 con 0x7f339806b390 2023-12-19T18:28:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.212+0000 7f339e696640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f339815e7a0 con 0x7f33980cea40 2023-12-19T18:28:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.212+0000 7f3397fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f339815ae80 0x7f339806ac80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47610/0 (socket says 172.21.15.27:47610) 2023-12-19T18:28:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.212+0000 7f3397fff640 1 -- 172.21.15.27:0/2552074373 learned_addr learned my addr 172.21.15.27:0/2552074373 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.212+0000 7f33977fe640 1 --2- 172.21.15.27:0/2552074373 >> v2:172.21.15.27:3300/0 conn(0x7f33980cea40 0x7f33980d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.212+0000 7f3396ffd640 1 -- 172.21.15.27:0/2552074373 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1627555090 0 0) 0x7f339815e4c0 con 0x7f339815ae80 2023-12-19T18:28:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.212+0000 7f3396ffd640 1 -- 172.21.15.27:0/2552074373 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3380003520 con 0x7f339815ae80 2023-12-19T18:28:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.213+0000 7f33977fe640 1 -- 172.21.15.27:0/2552074373 >> v1:172.21.15.192:6789/0 conn(0x7f339806b390 legacy=0x7f33980cc3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.213+0000 7f33977fe640 1 -- 172.21.15.27:0/2552074373 >> v1:172.21.15.27:6789/0 conn(0x7f339815ae80 legacy=0x7f339806ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.213+0000 7f33977fe640 1 -- 172.21.15.27:0/2552074373 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f339806cc20 con 0x7f33980cea40 2023-12-19T18:28:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.214+0000 7f33977fe640 1 --2- 172.21.15.27:0/2552074373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33980cea40 0x7f33980d0830 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f338c009d90 tx=0x7f338c02bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfb60d29d35fc21a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:16.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.214+0000 7f3396ffd640 1 -- 172.21.15.27:0/2552074373 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c02cc70 con 0x7f33980cea40 2023-12-19T18:28:16.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.214+0000 7f3396ffd640 1 -- 172.21.15.27:0/2552074373 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c02cdd0 con 0x7f33980cea40 2023-12-19T18:28:16.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.214+0000 7f3396ffd640 1 -- 172.21.15.27:0/2552074373 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c034640 con 0x7f33980cea40 2023-12-19T18:28:16.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.215+0000 7f339e696640 1 -- 172.21.15.27:0/2552074373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33980cea40 msgr2=0x7f33980d0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:16.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.215+0000 7f339e696640 1 --2- 172.21.15.27:0/2552074373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33980cea40 0x7f33980d0830 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7f338c009d90 tx=0x7f338c02bec0 comp rx=0 tx=0).stop 2023-12-19T18:28:16.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.215+0000 7f339e696640 1 -- 172.21.15.27:0/2552074373 shutdown_connections 2023-12-19T18:28:16.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.215+0000 7f339e696640 1 --2- 172.21.15.27:0/2552074373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33980cea40 0x7f33980d0830 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:16.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.215+0000 7f339e696640 1 -- 172.21.15.27:0/2552074373 >> 172.21.15.27:0/2552074373 conn(0x7f33980cd810 msgr2=0x7f33980cdbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:16.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.215+0000 7f339e696640 1 -- 172.21.15.27:0/2552074373 shutdown_connections 2023-12-19T18:28:16.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.215+0000 7f339e696640 1 -- 172.21.15.27:0/2552074373 wait complete. 2023-12-19T18:28:16.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.217+0000 7f339e696640 1 Processor -- start 2023-12-19T18:28:16.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.217+0000 7f339e696640 1 -- start start 2023-12-19T18:28:16.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.218+0000 7f339e696640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f339806b390 0x7f33980b5190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:16.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.218+0000 7f339e696640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33980cea40 0x7f33980c6770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:16.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.218+0000 7f339e696640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f339815ae80 0x7f339815c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:16.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.218+0000 7f339e696640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33980c6350 con 0x7f33980cea40 2023-12-19T18:28:16.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.218+0000 7f339e696640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33980af0d0 con 0x7f339806b390 2023-12-19T18:28:16.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.218+0000 7f339e696640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33980af3b0 con 0x7f339815ae80 2023-12-19T18:28:16.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.218+0000 7f33977fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33980cea40 0x7f33980c6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:16.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.218+0000 7f33977fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33980cea40 0x7f33980c6770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42374/0 (socket says 172.21.15.27:42374) 2023-12-19T18:28:16.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.218+0000 7f33977fe640 1 -- 172.21.15.27:0/1260820194 learned_addr learned my addr 172.21.15.27:0/1260820194 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:16.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.219+0000 7f3397fff640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f339806b390 0x7f33980b5190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:16.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.219+0000 7f339cc33640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f339815ae80 0x7f339815c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:16.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.219+0000 7f33977fe640 1 -- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f339815ae80 msgr2=0x7f339815c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:16.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.219+0000 7f33977fe640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f339815ae80 0x7f339815c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:16.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.219+0000 7f33977fe640 1 -- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f339806b390 msgr2=0x7f33980b5190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:16.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.219+0000 7f33977fe640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f339806b390 0x7f33980b5190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:16.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.219+0000 7f33977fe640 1 -- 172.21.15.27:0/1260820194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f338c009a70 con 0x7f33980cea40 2023-12-19T18:28:16.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.219+0000 7f33977fe640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33980cea40 0x7f33980c6770 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f338c009d90 tx=0x7f338c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:16.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.220+0000 7f3394ff9640 1 -- 172.21.15.27:0/1260820194 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c03e610 con 0x7f33980cea40 2023-12-19T18:28:16.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.220+0000 7f3394ff9640 1 -- 172.21.15.27:0/1260820194 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f338c03e770 con 0x7f33980cea40 2023-12-19T18:28:16.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.220+0000 7f3394ff9640 1 -- 172.21.15.27:0/1260820194 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f338c03eb20 con 0x7f33980cea40 2023-12-19T18:28:16.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.220+0000 7f339e696640 1 -- 172.21.15.27:0/1260820194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33980af690 con 0x7f33980cea40 2023-12-19T18:28:16.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.220+0000 7f339e696640 1 -- 172.21.15.27:0/1260820194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33980b4550 con 0x7f33980cea40 2023-12-19T18:28:16.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.221+0000 7f339e696640 1 -- 172.21.15.27:0/1260820194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33980b46e0 con 0x7f33980cea40 2023-12-19T18:28:16.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.222+0000 7f3394ff9640 1 -- 172.21.15.27:0/1260820194 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f338c03b050 con 0x7f33980cea40 2023-12-19T18:28:16.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.223+0000 7f3394ff9640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f338406a460 0x7f338406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:16.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.223+0000 7f3394ff9640 1 -- 172.21.15.27:0/1260820194 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f338c0aaab0 con 0x7f33980cea40 2023-12-19T18:28:16.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.224+0000 7f3397fff640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f338406a460 0x7f338406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:16.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.225+0000 7f3397fff640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f338406a460 0x7f338406c920 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f33880011e0 tx=0x7f3388009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:16.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.228+0000 7f3394ff9640 1 -- 172.21.15.27:0/1260820194 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f338c0760a0 con 0x7f33980cea40 2023-12-19T18:28:16.460 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.460+0000 7f339e696640 1 -- 172.21.15.27:0/1260820194 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f33980af820 con 0x7f33980cea40 2023-12-19T18:28:16.461 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.461+0000 7f3394ff9640 1 -- 172.21.15.27:0/1260820194 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) v1 ==== 53+0+73 (secure 0 0 0) 0x7f338c04d060 con 0x7f33980cea40 2023-12-19T18:28:16.461 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:28:16.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.464+0000 7f339e696640 1 -- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f338406a460 msgr2=0x7f338406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:16.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.464+0000 7f339e696640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f338406a460 0x7f338406c920 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f33880011e0 tx=0x7f3388009040 comp rx=0 tx=0).stop 2023-12-19T18:28:16.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.464+0000 7f339e696640 1 -- 172.21.15.27:0/1260820194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33980cea40 msgr2=0x7f33980c6770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:16.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.464+0000 7f339e696640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33980cea40 0x7f33980c6770 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f338c009d90 tx=0x7f338c004a40 comp rx=0 tx=0).stop 2023-12-19T18:28:16.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.464+0000 7f339e696640 1 -- 172.21.15.27:0/1260820194 shutdown_connections 2023-12-19T18:28:16.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.464+0000 7f339e696640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f338406a460 0x7f338406c920 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:16.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.464+0000 7f339e696640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f339815ae80 0x7f339815c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:16.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.464+0000 7f339e696640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33980cea40 0x7f33980c6770 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:16.464 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.464+0000 7f339e696640 1 --2- 172.21.15.27:0/1260820194 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f339806b390 0x7f33980b5190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:16.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.464+0000 7f339e696640 1 -- 172.21.15.27:0/1260820194 >> 172.21.15.27:0/1260820194 conn(0x7f33980cd810 msgr2=0x7f339815d880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:16.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.464+0000 7f339e696640 1 -- 172.21.15.27:0/1260820194 shutdown_connections 2023-12-19T18:28:16.465 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:16.464+0000 7f339e696640 1 -- 172.21.15.27:0/1260820194 wait complete. 2023-12-19T18:28:16.482 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:28:16.482 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:28:16.482 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2023-12-19T18:28:16.482 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-19T18:28:16.482 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-19T18:28:16.482 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:28:16.482 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:28:16.482 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:28:16.482 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:28:16.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:16.483 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2023-12-19T18:28:16.540 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:28:16.540 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:28:16.540 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:28:16.540 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:16.540 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2023-12-19T18:28:16.540 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:16.540 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:28:16.540 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f0 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp7s0f1 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5a brd ff:ff:ff:ff:ff:ff 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f0 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 41544sec preferred_lft 41544sec 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ae1f:6bff:fef8:395a/64 scope link noprefixroute 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether ac:1f:6b:f8:39:5b brd ff:ff:ff:ff:ff:ff 2023-12-19T18:28:16.541 INFO:teuthology.orchestra.run.smithi027.stdout: altname enp1s0f1 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::8053:76ff:fe69:c66c/64 scope link 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 7a:1d:d6:b4:7a:2d brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::781d:d6ff:feb4:7a2d/64 scope link 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether f2:29:63:cc:14:0e brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::f029:63ff:fecc:140e/64 scope link 2023-12-19T18:28:16.542 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:16.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:16.543 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:28:16.648 INFO:teuthology.orchestra.run.smithi027.stdout:1 2023-12-19T18:28:16.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:16.654 DEBUG:teuthology.orchestra.run.smithi027:> route 2023-12-19T18:28:16.712 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2023-12-19T18:28:16.712 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:28:16.713 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-12-19T18:28:16.713 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-12-19T18:28:16.713 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:28:16.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:16.713 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2023-12-19T18:28:16.713 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2023-12-19T18:28:16.713 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-19T18:28:16.713 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-19T18:28:16.713 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-19T18:28:16.713 DEBUG:teuthology.orchestra.run.smithi027:> ') 2023-12-19T18:28:16.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:16.925 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:28:16.979 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:28:16.979 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:28:16.980 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:28:16.980 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:16.980 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2023-12-19T18:28:17.074 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-12-19T18:28:17.074 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:28:17.074 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-19T18:28:17.131 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-19T18:28:17.131 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:28:17.131 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:28:17.189 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:28:17.305 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:28:17.306 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:28:17.416 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:28:17.421 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:28:17.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:17.421 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:28:17.485 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:28:17.486 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-12-19T18:28:17.491 DEBUG:teuthology.orchestra.run.smithi027:> sudo modprobe fuse 2023-12-19T18:28:17.576 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:28:17.576 DEBUG:teuthology.orchestra.run.smithi027:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:28:17.653 INFO:teuthology.orchestra.run.smithi027.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:28:17.656 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:28:17.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:17.657 DEBUG:teuthology.orchestra.run.smithi027:> ls /sys/fs/fuse/connections 2023-12-19T18:28:17.693 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:2023-12-19T18:28:17.693+0000 7f3d67c45480 -1 init, newargv = 0x558f79c614a0 newargc=15 2023-12-19T18:28:17.693 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[122775]: starting ceph client 2023-12-19T18:28:17.701 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[122775]: starting fuse 2023-12-19T18:28:17.702 INFO:teuthology.orchestra.run.smithi027.stdout:41 2023-12-19T18:28:17.702 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [41] 2023-12-19T18:28:17.702 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:28:17.702 DEBUG:teuthology.orchestra.run.smithi027:> import glob 2023-12-19T18:28:17.702 DEBUG:teuthology.orchestra.run.smithi027:> import re 2023-12-19T18:28:17.702 DEBUG:teuthology.orchestra.run.smithi027:> import os 2023-12-19T18:28:17.702 DEBUG:teuthology.orchestra.run.smithi027:> import subprocess 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> def _find_admin_socket(client_name): 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> files = glob.glob(asok_path) 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> # Given a non-glob path, it better be there 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> if "*" not in asok_path: 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> assert(len(files) == 1) 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> return files[0] 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> for f in files: 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> contents = proc_f.read() 2023-12-19T18:28:17.703 DEBUG:teuthology.orchestra.run.smithi027:> if mountpoint in contents: 2023-12-19T18:28:17.704 DEBUG:teuthology.orchestra.run.smithi027:> return f 2023-12-19T18:28:17.704 DEBUG:teuthology.orchestra.run.smithi027:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:28:17.704 DEBUG:teuthology.orchestra.run.smithi027:> 2023-12-19T18:28:17.704 DEBUG:teuthology.orchestra.run.smithi027:> print(_find_admin_socket("client.0")) 2023-12-19T18:28:17.704 DEBUG:teuthology.orchestra.run.smithi027:> ' 2023-12-19T18:28:17.906 INFO:teuthology.orchestra.run.smithi027.stdout:/var/run/ceph/ceph-client.0.122775.asok 2023-12-19T18:28:18.084 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.122775.asok 2023-12-19T18:28:18.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:18.085 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.122775.asok status 2023-12-19T18:28:18.346 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:28:18.346 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": { 2023-12-19T18:28:18.346 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:28:18.346 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:28:18.346 INFO:teuthology.orchestra.run.smithi027.stdout: "entity_id": "0", 2023-12-19T18:28:18.346 INFO:teuthology.orchestra.run.smithi027.stdout: "hostname": "smithi027", 2023-12-19T18:28:18.346 INFO:teuthology.orchestra.run.smithi027.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-12-19T18:28:18.346 INFO:teuthology.orchestra.run.smithi027.stdout: "pid": "122775", 2023-12-19T18:28:18.346 INFO:teuthology.orchestra.run.smithi027.stdout: "root": "/" 2023-12-19T18:28:18.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:18.346 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_count": 0, 2023-12-19T18:28:18.346 INFO:teuthology.orchestra.run.smithi027.stdout: "dentry_pinned_count": 0, 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: "id": 9159, 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: "inst": { 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: "name": { 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "client", 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: "num": 9159 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2707562116 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": { 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:18.347 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:28:18.348 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 2707562116 2023-12-19T18:28:18.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:18.348 INFO:teuthology.orchestra.run.smithi027.stdout: "inst_str": "client.9159 192.168.0.1:0/2707562116", 2023-12-19T18:28:18.348 INFO:teuthology.orchestra.run.smithi027.stdout: "addr_str": "192.168.0.1:0/2707562116", 2023-12-19T18:28:18.348 INFO:teuthology.orchestra.run.smithi027.stdout: "inode_count": 1, 2023-12-19T18:28:18.348 INFO:teuthology.orchestra.run.smithi027.stdout: "mds_epoch": 85, 2023-12-19T18:28:18.348 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch": 202, 2023-12-19T18:28:18.348 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:28:18.348 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklisted": false, 2023-12-19T18:28:18.348 INFO:teuthology.orchestra.run.smithi027.stdout: "fs_name": "cephfs" 2023-12-19T18:28:18.348 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:28:18.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:18.361 DEBUG:teuthology.orchestra.run.smithi027:> stat --file-system '--printf=%T 2023-12-19T18:28:18.361 DEBUG:teuthology.orchestra.run.smithi027:> ' -- /home/ubuntu/cephtest/mnt.0 2023-12-19T18:28:18.420 INFO:teuthology.orchestra.run.smithi027.stdout:fuseblk 2023-12-19T18:28:18.420 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-12-19T18:28:18.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:18.420 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-19T18:28:18.494 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:28:18.504 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:28:18.528 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-19T18:28:18.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.797+0000 7f8081e0e640 1 Processor -- start 2023-12-19T18:28:18.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.797+0000 7f8081e0e640 1 -- start start 2023-12-19T18:28:18.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.799+0000 7f8081e0e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f807c1604d0 0x7f807c0b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:18.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.799+0000 7f8081e0e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f807c0d2350 0x7f807c0b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:18.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.799+0000 7f8081e0e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f807c0d0300 0x7f807c0d0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:18.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.799+0000 7f8081e0e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807c0d2b30 con 0x7f807c1604d0 2023-12-19T18:28:18.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.799+0000 7f8081e0e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f807c0b5e90 con 0x7f807c0d0300 2023-12-19T18:28:18.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.799+0000 7f8081e0e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0d2850 con 0x7f807c0d2350 2023-12-19T18:28:18.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.799+0000 7f8080e0c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f807c1604d0 0x7f807c0b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:18.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.800+0000 7f8080e0c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f807c1604d0 0x7f807c0b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42380/0 (socket says 172.21.15.27:42380) 2023-12-19T18:28:18.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.800+0000 7f8080e0c640 1 -- 172.21.15.27:0/4225545486 learned_addr learned my addr 172.21.15.27:0/4225545486 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:18.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.800+0000 7f807bfff640 1 --2- 172.21.15.27:0/4225545486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f807c0d2350 0x7f807c0b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:18.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.800+0000 7f808160d640 1 --2- 172.21.15.27:0/4225545486 >> v2:172.21.15.192:3300/0 conn(0x7f807c0d0300 0x7f807c0d0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:18.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.800+0000 7f8080e0c640 1 -- 172.21.15.27:0/4225545486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f807c0d2350 msgr2=0x7f807c0b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:18.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.800+0000 7f8080e0c640 1 --2- 172.21.15.27:0/4225545486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f807c0d2350 0x7f807c0b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:18.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.800+0000 7f8080e0c640 1 -- 172.21.15.27:0/4225545486 >> v2:172.21.15.192:3300/0 conn(0x7f807c0d0300 msgr2=0x7f807c0d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:18.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.800+0000 7f8080e0c640 1 --2- 172.21.15.27:0/4225545486 >> v2:172.21.15.192:3300/0 conn(0x7f807c0d0300 0x7f807c0d0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:18.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.800+0000 7f8080e0c640 1 -- 172.21.15.27:0/4225545486 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c0cded0 con 0x7f807c1604d0 2023-12-19T18:28:18.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.802+0000 7f8080e0c640 1 --2- 172.21.15.27:0/4225545486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f807c1604d0 0x7f807c0b47e0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f807c071e80 tx=0x7f807002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7aa3b8fdbaff3977 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:18.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.802+0000 7f807b7fe640 1 -- 172.21.15.27:0/4225545486 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070031870 con 0x7f807c1604d0 2023-12-19T18:28:18.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.803+0000 7f807b7fe640 1 -- 172.21.15.27:0/4225545486 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80700319d0 con 0x7f807c1604d0 2023-12-19T18:28:18.803 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.803+0000 7f807b7fe640 1 -- 172.21.15.27:0/4225545486 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8070031d60 con 0x7f807c1604d0 2023-12-19T18:28:18.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.803+0000 7f8081e0e640 1 -- 172.21.15.27:0/4225545486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f807c1604d0 msgr2=0x7f807c0b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:18.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.803+0000 7f8081e0e640 1 --2- 172.21.15.27:0/4225545486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f807c1604d0 0x7f807c0b47e0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f807c071e80 tx=0x7f807002b910 comp rx=0 tx=0).stop 2023-12-19T18:28:18.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.803+0000 7f8081e0e640 1 -- 172.21.15.27:0/4225545486 shutdown_connections 2023-12-19T18:28:18.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.803+0000 7f8081e0e640 1 --2- 172.21.15.27:0/4225545486 >> v2:172.21.15.192:3300/0 conn(0x7f807c0d0300 0x7f807c0d0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:18.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.803+0000 7f8081e0e640 1 --2- 172.21.15.27:0/4225545486 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f807c0d2350 0x7f807c0b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:18.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.803+0000 7f8081e0e640 1 --2- 172.21.15.27:0/4225545486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f807c1604d0 0x7f807c0b47e0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:18.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.803+0000 7f8081e0e640 1 -- 172.21.15.27:0/4225545486 >> 172.21.15.27:0/4225545486 conn(0x7f807c0cf7e0 msgr2=0x7f807c0cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:18.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.803+0000 7f8081e0e640 1 -- 172.21.15.27:0/4225545486 shutdown_connections 2023-12-19T18:28:18.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.803+0000 7f8081e0e640 1 -- 172.21.15.27:0/4225545486 wait complete. 2023-12-19T18:28:18.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.805+0000 7f8081e0e640 1 Processor -- start 2023-12-19T18:28:18.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.805+0000 7f8081e0e640 1 -- start start 2023-12-19T18:28:18.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.806+0000 7f8081e0e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f807c0d0300 0x7f807c0c9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:18.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.806+0000 7f8081e0e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f807c0d2350 0x7f807c155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:18.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f8081e0e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f807c159030 0x7f807c0b44b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:18.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f8081e0e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c0c9680 con 0x7f807c159030 2023-12-19T18:28:18.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f8081e0e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f807c157f90 con 0x7f807c0d2350 2023-12-19T18:28:18.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f8081e0e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f807c158270 con 0x7f807c0d0300 2023-12-19T18:28:18.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f808160d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f807c159030 0x7f807c0b44b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:18.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f808160d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f807c159030 0x7f807c0b44b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:42388/0 (socket says 172.21.15.27:42388) 2023-12-19T18:28:18.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f808160d640 1 -- 172.21.15.27:0/2955747059 learned_addr learned my addr 172.21.15.27:0/2955747059 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:18.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f8080e0c640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f807c0d0300 0x7f807c0c9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:18.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f807bfff640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f807c0d2350 0x7f807c155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:18.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f808160d640 1 -- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f807c0d0300 msgr2=0x7f807c0c9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:18.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f808160d640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f807c0d0300 0x7f807c0c9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:18.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f808160d640 1 -- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f807c0d2350 msgr2=0x7f807c155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:18.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f808160d640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f807c0d2350 0x7f807c155300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:18.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.807+0000 7f808160d640 1 -- 172.21.15.27:0/2955747059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80700094d0 con 0x7f807c159030 2023-12-19T18:28:18.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.808+0000 7f808160d640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f807c159030 0x7f807c0b44b0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f8074007e60 tx=0x7f807400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.808+0000 7f80797fa640 1 -- 172.21.15.27:0/2955747059 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074014620 con 0x7f807c159030 2023-12-19T18:28:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.808+0000 7f80797fa640 1 -- 172.21.15.27:0/2955747059 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f807400ee90 con 0x7f807c159030 2023-12-19T18:28:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.808+0000 7f8081e0e640 1 -- 172.21.15.27:0/2955747059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f807c158550 con 0x7f807c159030 2023-12-19T18:28:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.808+0000 7f80797fa640 1 -- 172.21.15.27:0/2955747059 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80740149e0 con 0x7f807c159030 2023-12-19T18:28:18.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.808+0000 7f8081e0e640 1 -- 172.21.15.27:0/2955747059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f807c158b10 con 0x7f807c159030 2023-12-19T18:28:18.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.810+0000 7f8081e0e640 1 -- 172.21.15.27:0/2955747059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f807c066f80 con 0x7f807c159030 2023-12-19T18:28:18.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.811+0000 7f80797fa640 1 -- 172.21.15.27:0/2955747059 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8074014b40 con 0x7f807c159030 2023-12-19T18:28:18.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.812+0000 7f80797fa640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f806006a430 0x7f806006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:18.811 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.812+0000 7f80797fa640 1 -- 172.21.15.27:0/2955747059 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8074089160 con 0x7f807c159030 2023-12-19T18:28:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.812+0000 7f8080e0c640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f806006a430 0x7f806006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:18.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.814+0000 7f8080e0c640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f806006a430 0x7f806006c8f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8070036000 tx=0x7f807003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:18.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:18.818+0000 7f80797fa640 1 -- 172.21.15.27:0/2955747059 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f80740547e0 con 0x7f807c159030 2023-12-19T18:28:19.052 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.052+0000 7f8081e0e640 1 -- 172.21.15.27:0/2955747059 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f807c04e840 con 0x7f807c159030 2023-12-19T18:28:19.053 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.053+0000 7f80797fa640 1 -- 172.21.15.27:0/2955747059 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v85) v1 ==== 53+0+73 (secure 0 0 0) 0x7f8074057e00 con 0x7f807c159030 2023-12-19T18:28:19.054 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-19T18:28:19.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.056+0000 7f8081e0e640 1 -- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f806006a430 msgr2=0x7f806006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:19.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.056+0000 7f8081e0e640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f806006a430 0x7f806006c8f0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f8070036000 tx=0x7f807003c040 comp rx=0 tx=0).stop 2023-12-19T18:28:19.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.056+0000 7f8081e0e640 1 -- 172.21.15.27:0/2955747059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f807c159030 msgr2=0x7f807c0b44b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:19.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.056+0000 7f8081e0e640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f807c159030 0x7f807c0b44b0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f8074007e60 tx=0x7f807400e4b0 comp rx=0 tx=0).stop 2023-12-19T18:28:19.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.056+0000 7f8081e0e640 1 -- 172.21.15.27:0/2955747059 shutdown_connections 2023-12-19T18:28:19.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.056+0000 7f8081e0e640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f806006a430 0x7f806006c8f0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:19.056 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.056+0000 7f8081e0e640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f807c159030 0x7f807c0b44b0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:19.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.056+0000 7f8081e0e640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f807c0d2350 0x7f807c155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:19.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.056+0000 7f8081e0e640 1 --2- 172.21.15.27:0/2955747059 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f807c0d0300 0x7f807c0c9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:19.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.056+0000 7f8081e0e640 1 -- 172.21.15.27:0/2955747059 >> 172.21.15.27:0/2955747059 conn(0x7f807c0cf7e0 msgr2=0x7f807c0b85c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:19.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.056+0000 7f8081e0e640 1 -- 172.21.15.27:0/2955747059 shutdown_connections 2023-12-19T18:28:19.057 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:19.056+0000 7f8081e0e640 1 -- 172.21.15.27:0/2955747059 wait complete. 2023-12-19T18:28:19.078 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-19T18:28:19.078 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-19T18:28:19.078 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi192.front.sepia.ceph.com 2023-12-19T18:28:19.078 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-19T18:28:19.079 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-19T18:28:19.079 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-19T18:28:19.079 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-19T18:28:19.079 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-19T18:28:19.079 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:28:19.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:19.079 DEBUG:teuthology.orchestra.run.smithi192:> ip addr 2023-12-19T18:28:19.141 INFO:teuthology.orchestra.run.smithi192.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-19T18:28:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-19T18:28:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: inet 127.0.0.1/8 scope host lo 2023-12-19T18:28:19.141 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 ::1/128 scope host 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4e brd ff:ff:ff:ff:ff:ff 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f0 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:d9:38:4f brd ff:ff:ff:ff:ff:ff 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: altname enp7s0f1 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:48 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 0c:c4:7a:88:80:49 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: inet 172.21.15.192/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft 41504sec preferred_lft 41504sec 2023-12-19T18:28:19.142 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::ec4:7aff:fe88:8049/64 scope link 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::b892:1cff:fedc:4bed/64 scope link 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether a6:e1:fc:e6:56:fa brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::a4e1:fcff:fee6:56fa/64 scope link 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: link/ether 42:4f:6c:73:9a:a2 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: inet6 fe80::404f:6cff:fe73:9aa2/64 scope link 2023-12-19T18:28:19.143 INFO:teuthology.orchestra.run.smithi192.stdout: valid_lft forever preferred_lft forever 2023-12-19T18:28:19.144 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:19.144 DEBUG:teuthology.orchestra.run.smithi192:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-19T18:28:19.237 INFO:teuthology.orchestra.run.smithi192.stdout:1 2023-12-19T18:28:19.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:19.241 DEBUG:teuthology.orchestra.run.smithi192:> route 2023-12-19T18:28:19.294 INFO:teuthology.orchestra.run.smithi192.stdout:Kernel IP routing table 2023-12-19T18:28:19.294 INFO:teuthology.orchestra.run.smithi192.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-19T18:28:19.294 INFO:teuthology.orchestra.run.smithi192.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-19T18:28:19.294 INFO:teuthology.orchestra.run.smithi192.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-19T18:28:19.294 INFO:teuthology.orchestra.run.smithi192.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-19T18:28:19.295 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:19.295 DEBUG:teuthology.orchestra.run.smithi192:> (cd / && exec bash -c ' 2023-12-19T18:28:19.295 DEBUG:teuthology.orchestra.run.smithi192:> set -e 2023-12-19T18:28:19.295 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-19T18:28:19.295 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-19T18:28:19.295 DEBUG:teuthology.orchestra.run.smithi192:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-19T18:28:19.295 DEBUG:teuthology.orchestra.run.smithi192:> ') 2023-12-19T18:28:19.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:19.473 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:28:19.525 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T18:28:19.525 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-19T18:28:19.526 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:28:19.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:19.526 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link set brx.0 up 2023-12-19T18:28:19.617 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-12-19T18:28:19.618 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:28:19.618 DEBUG:teuthology.orchestra.run.smithi192:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-19T18:28:19.672 INFO:teuthology.orchestra.run.smithi192.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-19T18:28:19.673 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-19T18:28:19.673 DEBUG:teuthology.orchestra.run.smithi192:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:28:19.725 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:28:19.818 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:28:19.818 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:28:19.916 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:28:19.921 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:28:19.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:19.922 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:28:19.972 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-19T18:28:19.972 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-12-19T18:28:19.978 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:28:20.055 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:28:20.055 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:28:20.128 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:28:20.131 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:28:20.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:20.131 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:28:20.167 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:2023-12-19T18:28:20.166+0000 7f4e7763f480 -1 init, newargv = 0x55d0441e3740 newargc=15 2023-12-19T18:28:20.167 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[92745]: starting ceph client 2023-12-19T18:28:20.178 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[92745]: starting fuse 2023-12-19T18:28:21.275 DEBUG:teuthology.orchestra.run.smithi192:> sudo modprobe fuse 2023-12-19T18:28:21.372 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-19T18:28:21.372 DEBUG:teuthology.orchestra.run.smithi192:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-19T18:28:21.460 INFO:teuthology.orchestra.run.smithi192.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-19T18:28:21.463 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-19T18:28:21.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:21.464 DEBUG:teuthology.orchestra.run.smithi192:> ls /sys/fs/fuse/connections 2023-12-19T18:28:21.516 INFO:teuthology.orchestra.run.smithi192.stdout:40 2023-12-19T18:28:21.516 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [40] 2023-12-19T18:28:21.517 DEBUG:teuthology.orchestra.run.smithi192:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-19T18:28:21.517 DEBUG:teuthology.orchestra.run.smithi192:> import glob 2023-12-19T18:28:21.517 DEBUG:teuthology.orchestra.run.smithi192:> import re 2023-12-19T18:28:21.517 DEBUG:teuthology.orchestra.run.smithi192:> import os 2023-12-19T18:28:21.517 DEBUG:teuthology.orchestra.run.smithi192:> import subprocess 2023-12-19T18:28:21.517 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:28:21.517 DEBUG:teuthology.orchestra.run.smithi192:> def _find_admin_socket(client_name): 2023-12-19T18:28:21.517 DEBUG:teuthology.orchestra.run.smithi192:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-12-19T18:28:21.517 DEBUG:teuthology.orchestra.run.smithi192:> files = glob.glob(asok_path) 2023-12-19T18:28:21.517 DEBUG:teuthology.orchestra.run.smithi192:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-12-19T18:28:21.517 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:28:21.517 DEBUG:teuthology.orchestra.run.smithi192:> # Given a non-glob path, it better be there 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> if "*" not in asok_path: 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> assert(len(files) == 1) 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> return files[0] 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> for f in files: 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> contents = proc_f.read() 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> if mountpoint in contents: 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> return f 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> print(_find_admin_socket("client.1")) 2023-12-19T18:28:21.518 DEBUG:teuthology.orchestra.run.smithi192:> ' 2023-12-19T18:28:21.692 INFO:teuthology.orchestra.run.smithi192.stdout:/var/run/ceph/ceph-client.1.92745.asok 2023-12-19T18:28:21.874 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.92745.asok 2023-12-19T18:28:21.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:21.874 DEBUG:teuthology.orchestra.run.smithi192:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.92745.asok status 2023-12-19T18:28:22.119 INFO:teuthology.orchestra.run.smithi192.stdout:{ 2023-12-19T18:28:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: "metadata": { 2023-12-19T18:28:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_sha1": "568df92b7e50b2108bce4c77c393ff294d6a5e5e", 2023-12-19T18:28:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: "ceph_version": "ceph version 18.2.1-132-g568df92b (568df92b7e50b2108bce4c77c393ff294d6a5e5e) reef (stable)", 2023-12-19T18:28:22.119 INFO:teuthology.orchestra.run.smithi192.stdout: "entity_id": "1", 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "hostname": "smithi192", 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "pid": "92745", 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "root": "/" 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_count": 0, 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "dentry_pinned_count": 0, 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "id": 6335, 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "inst": { 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "name": { 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "client", 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "num": 6335 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:28:22.120 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 888549178 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: } 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": { 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "type": "v1", 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "addr": "192.168.0.1:0", 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "nonce": 888549178 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: }, 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "inst_str": "client.6335 192.168.0.1:0/888549178", 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "addr_str": "192.168.0.1:0/888549178", 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "inode_count": 1, 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "mds_epoch": 85, 2023-12-19T18:28:22.121 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch": 202, 2023-12-19T18:28:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: "osd_epoch_barrier": 0, 2023-12-19T18:28:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: "blocklisted": false, 2023-12-19T18:28:22.122 INFO:teuthology.orchestra.run.smithi192.stdout: "fs_name": "cephfs" 2023-12-19T18:28:22.122 INFO:teuthology.orchestra.run.smithi192.stdout:} 2023-12-19T18:28:22.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:22.134 DEBUG:teuthology.orchestra.run.smithi192:> stat --file-system '--printf=%T 2023-12-19T18:28:22.134 DEBUG:teuthology.orchestra.run.smithi192:> ' -- /home/ubuntu/cephtest/mnt.1 2023-12-19T18:28:22.187 INFO:teuthology.orchestra.run.smithi192.stdout:fuseblk 2023-12-19T18:28:22.187 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-12-19T18:28:22.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:22.188 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-19T18:28:22.277 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:28:22.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.526+0000 7f8247fff640 1 Processor -- start 2023-12-19T18:28:22.526 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.526+0000 7f8247fff640 1 -- start start 2023-12-19T18:28:22.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.528+0000 7f8247fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f82481553a0 0x7f82480c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:22.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.528+0000 7f8247fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f82480afeb0 0x7f824815e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:22.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.528+0000 7f8247fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f824815ec60 0x7f824815b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:22.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.528+0000 7f8247fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82480cbcf0 con 0x7f82481553a0 2023-12-19T18:28:22.528 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.528+0000 7f8247fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f824815f0d0 con 0x7f82480afeb0 2023-12-19T18:28:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.528+0000 7f8247fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82480b0360 con 0x7f824815ec60 2023-12-19T18:28:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.528+0000 7f8246ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f82481553a0 0x7f82480c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.528+0000 7f8246ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f82481553a0 0x7f82480c50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52554/0 (socket says 172.21.15.27:52554) 2023-12-19T18:28:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.528+0000 7f8246ffd640 1 -- 172.21.15.27:0/3862987161 learned_addr learned my addr 172.21.15.27:0/3862987161 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.529+0000 7f8246ffd640 1 -- 172.21.15.27:0/3862987161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f824815ec60 msgr2=0x7f824815b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.529+0000 7f8246ffd640 1 --2- 172.21.15.27:0/3862987161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f824815ec60 0x7f824815b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.529+0000 7f8246ffd640 1 -- 172.21.15.27:0/3862987161 >> v2:172.21.15.192:3300/0 conn(0x7f82480afeb0 msgr2=0x7f824815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.529+0000 7f8246ffd640 1 --2- 172.21.15.27:0/3862987161 >> v2:172.21.15.192:3300/0 conn(0x7f82480afeb0 0x7f824815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:22.529 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.529+0000 7f8246ffd640 1 -- 172.21.15.27:0/3862987161 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82480cb350 con 0x7f82481553a0 2023-12-19T18:28:22.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.530+0000 7f8246ffd640 1 --2- 172.21.15.27:0/3862987161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481553a0 0x7f82480c50a0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f824806d4a0 tx=0x7f823802b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a590f115874667bc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:22.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.530+0000 7f8245ffb640 1 -- 172.21.15.27:0/3862987161 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82380317c0 con 0x7f82481553a0 2023-12-19T18:28:22.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.530+0000 7f8245ffb640 1 -- 172.21.15.27:0/3862987161 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8238031920 con 0x7f82481553a0 2023-12-19T18:28:22.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.530+0000 7f8245ffb640 1 -- 172.21.15.27:0/3862987161 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8238031cb0 con 0x7f82481553a0 2023-12-19T18:28:22.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.531+0000 7f8247fff640 1 -- 172.21.15.27:0/3862987161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481553a0 msgr2=0x7f82480c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:22.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.531+0000 7f8247fff640 1 --2- 172.21.15.27:0/3862987161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481553a0 0x7f82480c50a0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f824806d4a0 tx=0x7f823802b860 comp rx=0 tx=0).stop 2023-12-19T18:28:22.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.531+0000 7f8247fff640 1 -- 172.21.15.27:0/3862987161 shutdown_connections 2023-12-19T18:28:22.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.531+0000 7f8247fff640 1 --2- 172.21.15.27:0/3862987161 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f824815ec60 0x7f824815b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:22.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.531+0000 7f8247fff640 1 --2- 172.21.15.27:0/3862987161 >> v2:172.21.15.192:3300/0 conn(0x7f82480afeb0 0x7f824815e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:22.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.531+0000 7f8247fff640 1 --2- 172.21.15.27:0/3862987161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481553a0 0x7f82480c50a0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:22.531 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.531+0000 7f8247fff640 1 -- 172.21.15.27:0/3862987161 >> 172.21.15.27:0/3862987161 conn(0x7f8248069c70 msgr2=0x7f824806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:22.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.531+0000 7f8247fff640 1 -- 172.21.15.27:0/3862987161 shutdown_connections 2023-12-19T18:28:22.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.531+0000 7f8247fff640 1 -- 172.21.15.27:0/3862987161 wait complete. 2023-12-19T18:28:22.532 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.533+0000 7f8247fff640 1 Processor -- start 2023-12-19T18:28:22.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.533+0000 7f8247fff640 1 -- start start 2023-12-19T18:28:22.535 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.534+0000 7f8247fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82480afeb0 0x7f8248149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.534+0000 7f8247fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481553a0 0x7f8248148ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.534+0000 7f8247fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f824815ec60 0x7f8248151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.534+0000 7f8247fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824806cde0 con 0x7f82481553a0 2023-12-19T18:28:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.534+0000 7f8247fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82481e4dc0 con 0x7f82480afeb0 2023-12-19T18:28:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.534+0000 7f8247fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8248149910 con 0x7f824815ec60 2023-12-19T18:28:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.534+0000 7f82467fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481553a0 0x7f8248148ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.534+0000 7f82467fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481553a0 0x7f8248148ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52556/0 (socket says 172.21.15.27:52556) 2023-12-19T18:28:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.534+0000 7f82467fc640 1 -- 172.21.15.27:0/2434053413 learned_addr learned my addr 172.21.15.27:0/2434053413 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.535+0000 7f8246ffd640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82480afeb0 0x7f8248149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.535+0000 7f82477fe640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f824815ec60 0x7f8248151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.535+0000 7f82467fc640 1 -- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f824815ec60 msgr2=0x7f8248151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.535+0000 7f82467fc640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f824815ec60 0x7f8248151d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:22.536 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.535+0000 7f82467fc640 1 -- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82480afeb0 msgr2=0x7f8248149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:22.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.535+0000 7f82467fc640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82480afeb0 0x7f8248149070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:22.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.535+0000 7f82467fc640 1 -- 172.21.15.27:0/2434053413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82380094d0 con 0x7f82481553a0 2023-12-19T18:28:22.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.535+0000 7f82467fc640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481553a0 0x7f8248148ae0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f823c00b410 tx=0x7f823c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:22.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.535+0000 7f822f7fe640 1 -- 172.21.15.27:0/2434053413 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c011830 con 0x7f82481553a0 2023-12-19T18:28:22.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.535+0000 7f822f7fe640 1 -- 172.21.15.27:0/2434053413 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f823c0119b0 con 0x7f82481553a0 2023-12-19T18:28:22.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.535+0000 7f8247fff640 1 -- 172.21.15.27:0/2434053413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f824806cb00 con 0x7f82481553a0 2023-12-19T18:28:22.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.536+0000 7f822f7fe640 1 -- 172.21.15.27:0/2434053413 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f823c011d60 con 0x7f82481553a0 2023-12-19T18:28:22.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.536+0000 7f8247fff640 1 -- 172.21.15.27:0/2434053413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82481e49e0 con 0x7f82481553a0 2023-12-19T18:28:22.537 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.537+0000 7f8247fff640 1 -- 172.21.15.27:0/2434053413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82481e4560 con 0x7f82481553a0 2023-12-19T18:28:22.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.538+0000 7f822f7fe640 1 -- 172.21.15.27:0/2434053413 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f823c01a400 con 0x7f82481553a0 2023-12-19T18:28:22.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.539+0000 7f822f7fe640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f823406a460 0x7f823406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:22.538 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.539+0000 7f822f7fe640 1 -- 172.21.15.27:0/2434053413 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f823c088ad0 con 0x7f82481553a0 2023-12-19T18:28:22.539 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.539+0000 7f8246ffd640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f823406a460 0x7f823406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:22.540 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.541+0000 7f8246ffd640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f823406a460 0x7f823406c920 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8238036000 tx=0x7f823803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:22.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.546+0000 7f822f7fe640 1 -- 172.21.15.27:0/2434053413 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f82481e4560 con 0x7f82481553a0 2023-12-19T18:28:22.780 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.780+0000 7f8247fff640 1 -- 172.21.15.27:0/2434053413 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f82481e4f00 con 0x7f82481553a0 2023-12-19T18:28:22.781 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.781+0000 7f822f7fe640 1 -- 172.21.15.27:0/2434053413 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f82481e4f00 con 0x7f82481553a0 2023-12-19T18:28:22.781 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:22.781 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1864159643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1864159643},{"type":"v1","addr":"172.21.15.192:6836","nonce":1864159643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1348219767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1348219767},{"type":"v1","addr":"172.21.15.192:6837","nonce":1348219767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/181137639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":181137639},{"type":"v1","addr":"172.21.15.27:6833","nonce":181137639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:28:08.577742+0000","modified":"2023-12-19T18:28:12.083302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/152568921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":152568921},{"type":"v1","addr":"172.21.15.27:6835","nonce":152568921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:28:22.782 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 85 2023-12-19T18:28:22.783 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.784+0000 7f8247fff640 1 -- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f823406a460 msgr2=0x7f823406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:22.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.784+0000 7f8247fff640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f823406a460 0x7f823406c920 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f8238036000 tx=0x7f823803d040 comp rx=0 tx=0).stop 2023-12-19T18:28:22.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.784+0000 7f8247fff640 1 -- 172.21.15.27:0/2434053413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481553a0 msgr2=0x7f8248148ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:22.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.784+0000 7f8247fff640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481553a0 0x7f8248148ae0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f823c00b410 tx=0x7f823c00b8d0 comp rx=0 tx=0).stop 2023-12-19T18:28:22.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.784+0000 7f8247fff640 1 -- 172.21.15.27:0/2434053413 shutdown_connections 2023-12-19T18:28:22.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.784+0000 7f8247fff640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f823406a460 0x7f823406c920 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:22.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.784+0000 7f8247fff640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f824815ec60 0x7f8248151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:22.784 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.784+0000 7f8247fff640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82481553a0 0x7f8248148ae0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:22.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.784+0000 7f8247fff640 1 --2- 172.21.15.27:0/2434053413 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f82480afeb0 0x7f8248149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:22.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.784+0000 7f8247fff640 1 -- 172.21.15.27:0/2434053413 >> 172.21.15.27:0/2434053413 conn(0x7f8248069c70 msgr2=0x7f82480c4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:22.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.784+0000 7f8247fff640 1 -- 172.21.15.27:0/2434053413 shutdown_connections 2023-12-19T18:28:22.785 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:22.784+0000 7f8247fff640 1 -- 172.21.15.27:0/2434053413 wait complete. 2023-12-19T18:28:22.801 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-12-19T18:28:23.053 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":8997,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":14.455854177000001,"mdsmap_epoch":85,"osdmap_epoch":202,"osdmap_epoch_barrier":202,"uptime":20.249782888999999} 2023-12-19T18:28:23.053 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 8997, "mdsmap_epoch": 85, "osdmap_epoch": 202, "osdmap_epoch_barrier": 202, "rank_uptime": 14.455854177, "state": "up:active", "uptime": 20.249782889, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:28:23.054 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set-quota cephfs_data max_bytes 33554432 2023-12-19T18:28:23.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.285+0000 7f52690dc640 1 Processor -- start 2023-12-19T18:28:23.285 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.285+0000 7f52690dc640 1 -- start start 2023-12-19T18:28:23.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.287+0000 7f52690dc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52640c55b0 0x7f52640cb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:23.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.287+0000 7f52690dc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f52640ce520 0x7f52640cb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:23.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.287+0000 7f52690dc640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52640cb100 con 0x7f5264161030 2023-12-19T18:28:23.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.287+0000 7f52690dc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52640c95f0 con 0x7f52640c55b0 2023-12-19T18:28:23.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.287+0000 7f52690dc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52640cae20 con 0x7f52640ce520 2023-12-19T18:28:23.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.287+0000 7f525bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52640c55b0 0x7f52640cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:23.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.288+0000 7f525bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f52640c55b0 0x7f52640cb350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52566/0 (socket says 172.21.15.27:52566) 2023-12-19T18:28:23.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.288+0000 7f525bfff640 1 -- 172.21.15.27:0/1140887618 learned_addr learned my addr 172.21.15.27:0/1140887618 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:23.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.288+0000 7f52688db640 1 --2- 172.21.15.27:0/1140887618 >> v2:172.21.15.192:3300/0 conn(0x7f52640ce520 0x7f52640cb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:23.288 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.288+0000 7f525bfff640 1 -- 172.21.15.27:0/1140887618 >> v2:172.21.15.192:3300/0 conn(0x7f52640ce520 msgr2=0x7f52640cb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:23.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.288+0000 7f525bfff640 1 --2- 172.21.15.27:0/1140887618 >> v2:172.21.15.192:3300/0 conn(0x7f52640ce520 0x7f52640cb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:23.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.288+0000 7f525bfff640 1 -- 172.21.15.27:0/1140887618 >> v1:172.21.15.192:6789/0 conn(0x7f5264161030 legacy=0x7f52641657c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:23.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.288+0000 7f525bfff640 1 -- 172.21.15.27:0/1140887618 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52640c9030 con 0x7f52640c55b0 2023-12-19T18:28:23.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.289+0000 7f525bfff640 1 --2- 172.21.15.27:0/1140887618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52640c55b0 0x7f52640cb350 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f526406d7d0 tx=0x7f524c02b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b71d0c694bc25c04 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:23.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.290+0000 7f52637fe640 1 -- 172.21.15.27:0/1140887618 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c031870 con 0x7f52640c55b0 2023-12-19T18:28:23.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.290+0000 7f52637fe640 1 -- 172.21.15.27:0/1140887618 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c0319d0 con 0x7f52640c55b0 2023-12-19T18:28:23.289 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.290+0000 7f52637fe640 1 -- 172.21.15.27:0/1140887618 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c031d60 con 0x7f52640c55b0 2023-12-19T18:28:23.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.290+0000 7f52690dc640 1 -- 172.21.15.27:0/1140887618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52640c55b0 msgr2=0x7f52640cb350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:23.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.290+0000 7f52690dc640 1 --2- 172.21.15.27:0/1140887618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52640c55b0 0x7f52640cb350 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f526406d7d0 tx=0x7f524c02b910 comp rx=0 tx=0).stop 2023-12-19T18:28:23.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.290+0000 7f52690dc640 1 -- 172.21.15.27:0/1140887618 shutdown_connections 2023-12-19T18:28:23.290 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.290+0000 7f52690dc640 1 --2- 172.21.15.27:0/1140887618 >> v2:172.21.15.192:3300/0 conn(0x7f52640ce520 0x7f52640cb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:23.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.290+0000 7f52690dc640 1 --2- 172.21.15.27:0/1140887618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52640c55b0 0x7f52640cb350 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:23.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.290+0000 7f52690dc640 1 -- 172.21.15.27:0/1140887618 >> 172.21.15.27:0/1140887618 conn(0x7f52640b05b0 msgr2=0x7f52640b0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:23.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.290+0000 7f52690dc640 1 -- 172.21.15.27:0/1140887618 shutdown_connections 2023-12-19T18:28:23.291 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.290+0000 7f52690dc640 1 -- 172.21.15.27:0/1140887618 wait complete. 2023-12-19T18:28:23.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.292+0000 7f52690dc640 1 Processor -- start 2023-12-19T18:28:23.292 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.292+0000 7f52690dc640 1 -- start start 2023-12-19T18:28:23.293 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.293+0000 7f52690dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52640c55b0 0x7f5264154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:23.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.293+0000 7f52690dc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52640ce520 0x7f5264153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:23.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.293+0000 7f52690dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5264161030 0x7f52641eb6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:23.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.293+0000 7f52690dc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52641ec200 con 0x7f52640ce520 2023-12-19T18:28:23.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.293+0000 7f52690dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5264153dc0 con 0x7f52640c55b0 2023-12-19T18:28:23.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.293+0000 7f52690dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52641ebf20 con 0x7f5264161030 2023-12-19T18:28:23.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.293+0000 7f525bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52640ce520 0x7f5264153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:23.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.293+0000 7f525bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52640ce520 0x7f5264153880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52580/0 (socket says 172.21.15.27:52580) 2023-12-19T18:28:23.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.294+0000 7f525bfff640 1 -- 172.21.15.27:0/782424514 learned_addr learned my addr 172.21.15.27:0/782424514 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:23.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.294+0000 7f525bfff640 1 -- 172.21.15.27:0/782424514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5264161030 msgr2=0x7f52641eb6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:23.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.294+0000 7f525bfff640 1 --2- 172.21.15.27:0/782424514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5264161030 0x7f52641eb6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:23.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.294+0000 7f525bfff640 1 -- 172.21.15.27:0/782424514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52640c55b0 msgr2=0x7f5264154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:23.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.294+0000 7f5263fff640 1 --2- 172.21.15.27:0/782424514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52640c55b0 0x7f5264154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:23.294 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.294+0000 7f525bfff640 1 --2- 172.21.15.27:0/782424514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52640c55b0 0x7f5264154010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:23.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.294+0000 7f525bfff640 1 -- 172.21.15.27:0/782424514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f524c0094d0 con 0x7f52640ce520 2023-12-19T18:28:23.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.294+0000 7f525bfff640 1 --2- 172.21.15.27:0/782424514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52640ce520 0x7f5264153880 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f524c000c00 tx=0x7f524c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:23.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.295+0000 7f52617fa640 1 -- 172.21.15.27:0/782424514 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c03abe0 con 0x7f52640ce520 2023-12-19T18:28:23.295 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.295+0000 7f52617fa640 1 -- 172.21.15.27:0/782424514 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f524c03ad40 con 0x7f52640ce520 2023-12-19T18:28:23.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.295+0000 7f52617fa640 1 -- 172.21.15.27:0/782424514 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f524c044610 con 0x7f52640ce520 2023-12-19T18:28:23.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.295+0000 7f52690dc640 1 -- 172.21.15.27:0/782424514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52641ea2d0 con 0x7f52640ce520 2023-12-19T18:28:23.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.295+0000 7f52690dc640 1 -- 172.21.15.27:0/782424514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52641ea890 con 0x7f52640ce520 2023-12-19T18:28:23.296 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.296+0000 7f52690dc640 1 -- 172.21.15.27:0/782424514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52641eaa80 con 0x7f52640ce520 2023-12-19T18:28:23.297 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.297+0000 7f52617fa640 1 -- 172.21.15.27:0/782424514 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f524c04d450 con 0x7f52640ce520 2023-12-19T18:28:23.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.298+0000 7f52617fa640 1 --2- 172.21.15.27:0/782424514 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f524406a460 0x7f524406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:23.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.298+0000 7f52617fa640 1 -- 172.21.15.27:0/782424514 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f524c02d040 con 0x7f52640ce520 2023-12-19T18:28:23.298 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.299+0000 7f5263fff640 1 --2- 172.21.15.27:0/782424514 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f524406a460 0x7f524406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:23.299 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.300+0000 7f5263fff640 1 --2- 172.21.15.27:0/782424514 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f524406a460 0x7f524406c920 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5254006fd0 tx=0x7f5254008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:23.302 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.302+0000 7f52617fa640 1 -- 172.21.15.27:0/782424514 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f524c0af070 con 0x7f52640ce520 2023-12-19T18:28:23.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.497+0000 7f52690dc640 1 -- 172.21.15.27:0/782424514 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"} v 0) v1 -- 0x7f526404e840 con 0x7f52640ce520 2023-12-19T18:28:23.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.865+0000 7f52617fa640 1 -- 172.21.15.27:0/782424514 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "cephfs_data", "field": "max_bytes", "val": "33554432"}]=0 set-quota max_bytes = 33554432 for pool cephfs_data v203) v1 ==== 181+0+0 (secure 0 0 0) 0x7f524c0350c0 con 0x7f52640ce520 2023-12-19T18:28:23.865 INFO:teuthology.orchestra.run.smithi027.stderr:set-quota max_bytes = 33554432 for pool cephfs_data 2023-12-19T18:28:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.868+0000 7f52690dc640 1 -- 172.21.15.27:0/782424514 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f524406a460 msgr2=0x7f524406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.868+0000 7f52690dc640 1 --2- 172.21.15.27:0/782424514 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f524406a460 0x7f524406c920 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f5254006fd0 tx=0x7f5254008040 comp rx=0 tx=0).stop 2023-12-19T18:28:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.868+0000 7f52690dc640 1 -- 172.21.15.27:0/782424514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52640ce520 msgr2=0x7f5264153880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.868+0000 7f52690dc640 1 --2- 172.21.15.27:0/782424514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52640ce520 0x7f5264153880 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f524c000c00 tx=0x7f524c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:28:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.869+0000 7f52690dc640 1 -- 172.21.15.27:0/782424514 shutdown_connections 2023-12-19T18:28:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.869+0000 7f52690dc640 1 --2- 172.21.15.27:0/782424514 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f524406a460 0x7f524406c920 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.869+0000 7f52690dc640 1 --2- 172.21.15.27:0/782424514 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5264161030 0x7f52641eb6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.869+0000 7f52690dc640 1 --2- 172.21.15.27:0/782424514 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52640ce520 0x7f5264153880 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.869+0000 7f52690dc640 1 --2- 172.21.15.27:0/782424514 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f52640c55b0 0x7f5264154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:23.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.869+0000 7f52690dc640 1 -- 172.21.15.27:0/782424514 >> 172.21.15.27:0/782424514 conn(0x7f52640b05b0 msgr2=0x7f52640af7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.869+0000 7f52690dc640 1 -- 172.21.15.27:0/782424514 shutdown_connections 2023-12-19T18:28:23.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:23.869+0000 7f52690dc640 1 -- 172.21.15.27:0/782424514 wait complete. 2023-12-19T18:28:23.891 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok --format=json config get osd_mon_report_interval 2023-12-19T18:28:24.144 INFO:teuthology.orchestra.run.smithi027.stdout:{"osd_mon_report_interval":"5"} 2023-12-19T18:28:24.144 DEBUG:tasks.cephfs.filesystem:_json_asok output { "osd_mon_report_interval": "5" } 2023-12-19T18:28:24.144 INFO:tasks.cephfs.test_full:Writing 32MB should fill this cluster 2023-12-19T18:28:24.144 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:28:24.144 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:28:24.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:24.202 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_a bs=1M conv=fdatasync count=16 seek=0) 2023-12-19T18:28:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:16+0 records in 2023-12-19T18:28:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:16+0 records out 2023-12-19T18:28:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.35701 s, 47.0 MB/s 2023-12-19T18:28:24.625 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:28:24.625 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:28:24.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:24.690 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=17 seek=0) 2023-12-19T18:28:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:17+0 records in 2023-12-19T18:28:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:17+0 records out 2023-12-19T18:28:25.076 INFO:teuthology.orchestra.run.smithi027.stderr:17825792 bytes (18 MB, 17 MiB) copied, 0.326269 s, 54.6 MB/s 2023-12-19T18:28:25.078 INFO:tasks.cephfs.test_full:Writing file B succeeded (full status will happen soon) 2023-12-19T18:28:25.078 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:28:25.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.315+0000 7faa81101640 1 Processor -- start 2023-12-19T18:28:25.315 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.315+0000 7faa81101640 1 -- start start 2023-12-19T18:28:25.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.317+0000 7faa81101640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa7c070aa0 0x7faa7c070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:25.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.317+0000 7faa81101640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7c071440 con 0x7faa7c15a3a0 2023-12-19T18:28:25.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.317+0000 7faa81101640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7c15b110 con 0x7faa7c154170 2023-12-19T18:28:25.317 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.317+0000 7faa81101640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c06fce0 con 0x7faa7c070aa0 2023-12-19T18:28:25.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.318+0000 7faa80900640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7faa7c15a3a0 0x7faa7c15a830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59696/0 (socket says 172.21.15.27:59696) 2023-12-19T18:28:25.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.318+0000 7faa80900640 1 -- 172.21.15.27:0/851403408 learned_addr learned my addr 172.21.15.27:0/851403408 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:25.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.318+0000 7faa7b7fe640 1 --2- 172.21.15.27:0/851403408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa7c070aa0 0x7faa7c070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:25.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.318+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1378740826 0 0) 0x7faa7c071440 con 0x7faa7c15a3a0 2023-12-19T18:28:25.318 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.318+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa64003520 con 0x7faa7c15a3a0 2023-12-19T18:28:25.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1320317795 0 0) 0x7faa64003520 con 0x7faa7c15a3a0 2023-12-19T18:28:25.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa7c070aa0 msgr2=0x7faa7c070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:25.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7affd640 1 --2- 172.21.15.27:0/851403408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa7c070aa0 0x7faa7c070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:25.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 >> v1:172.21.15.192:6789/0 conn(0x7faa7c154170 legacy=0x7faa7c070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:25.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c070880 con 0x7faa7c15a3a0 2023-12-19T18:28:25.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7faa740019f0 con 0x7faa7c15a3a0 2023-12-19T18:28:25.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 >> v1:172.21.15.27:6789/0 conn(0x7faa7c15a3a0 legacy=0x7faa7c15a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:25.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7affd640 1 --2- 172.21.15.27:0/851403408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa640039a0 0x7faa64003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:25.320 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7affd640 1 --2- 172.21.15.27:0/851403408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa640047c0 0x7faa64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:25.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7affd640 1 --2- 172.21.15.27:0/851403408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa7c070aa0 0x7faa64005230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:25.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c06fce0 con 0x7faa640047c0 2023-12-19T18:28:25.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa6400bb40 con 0x7faa640039a0 2023-12-19T18:28:25.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa6400df10 con 0x7faa7c070aa0 2023-12-19T18:28:25.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7b7fe640 1 --2- 172.21.15.27:0/851403408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa640047c0 0x7faa64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:25.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.319+0000 7faa7bfff640 1 --2- 172.21.15.27:0/851403408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa640039a0 0x7faa64003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:25.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.320+0000 7faa80900640 1 --2- 172.21.15.27:0/851403408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa7c070aa0 0x7faa64005230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:25.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.320+0000 7faa7b7fe640 1 -- 172.21.15.27:0/851403408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa7c070aa0 msgr2=0x7faa64005230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:25.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.320+0000 7faa7b7fe640 1 --2- 172.21.15.27:0/851403408 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa7c070aa0 0x7faa64005230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:25.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.320+0000 7faa7b7fe640 1 -- 172.21.15.27:0/851403408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa640039a0 msgr2=0x7faa64003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:25.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.320+0000 7faa7b7fe640 1 --2- 172.21.15.27:0/851403408 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa640039a0 0x7faa64003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:25.321 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.320+0000 7faa7b7fe640 1 -- 172.21.15.27:0/851403408 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa64003520 con 0x7faa640047c0 2023-12-19T18:28:25.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.321+0000 7faa7b7fe640 1 --2- 172.21.15.27:0/851403408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa640047c0 0x7faa64004c40 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7faa70009ac0 tx=0x7faa640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57bb335a27658840 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:25.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.321+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7002c6b0 con 0x7faa640047c0 2023-12-19T18:28:25.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.321+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7002c810 con 0x7faa640047c0 2023-12-19T18:28:25.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.321+0000 7faa7affd640 1 -- 172.21.15.27:0/851403408 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7002cba0 con 0x7faa640047c0 2023-12-19T18:28:25.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.322+0000 7faa81101640 1 -- 172.21.15.27:0/851403408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa640047c0 msgr2=0x7faa64004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:25.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.322+0000 7faa81101640 1 --2- 172.21.15.27:0/851403408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa640047c0 0x7faa64004c40 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7faa70009ac0 tx=0x7faa640057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:25.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.322+0000 7faa80900640 1 -- 172.21.15.27:0/851403408 reap_dead start 2023-12-19T18:28:25.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.322+0000 7faa81101640 1 -- 172.21.15.27:0/851403408 shutdown_connections 2023-12-19T18:28:25.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.322+0000 7faa81101640 1 -- 172.21.15.27:0/851403408 >> 172.21.15.27:0/851403408 conn(0x7faa7c06a7d0 msgr2=0x7faa7c0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:25.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.322+0000 7faa81101640 1 -- 172.21.15.27:0/851403408 shutdown_connections 2023-12-19T18:28:25.323 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.322+0000 7faa81101640 1 -- 172.21.15.27:0/851403408 wait complete. 2023-12-19T18:28:25.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.324+0000 7faa81101640 1 Processor -- start 2023-12-19T18:28:25.324 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.324+0000 7faa81101640 1 -- start start 2023-12-19T18:28:25.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.325+0000 7faa81101640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa640047c0 0x7faa7c1453b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:25.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.325+0000 7faa81101640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa7c14fe70 0x7faa7c0b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:25.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.325+0000 7faa81101640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa7c154ef0 0x7faa7c0b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:25.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.325+0000 7faa81101640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c1458f0 con 0x7faa7c154ef0 2023-12-19T18:28:25.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.325+0000 7faa81101640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c0b1b40 con 0x7faa7c14fe70 2023-12-19T18:28:25.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.325+0000 7faa81101640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c1503e0 con 0x7faa640047c0 2023-12-19T18:28:25.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.325+0000 7faa80900640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa7c154ef0 0x7faa7c0b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:25.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.325+0000 7faa80900640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa7c154ef0 0x7faa7c0b00f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52594/0 (socket says 172.21.15.27:52594) 2023-12-19T18:28:25.326 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.325+0000 7faa80900640 1 -- 172.21.15.27:0/3461652190 learned_addr learned my addr 172.21.15.27:0/3461652190 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:25.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.326+0000 7faa7bfff640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa640047c0 0x7faa7c1453b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:25.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.326+0000 7faa7b7fe640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa7c14fe70 0x7faa7c0b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:25.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.326+0000 7faa80900640 1 -- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa640047c0 msgr2=0x7faa7c1453b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:25.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.326+0000 7faa80900640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa640047c0 0x7faa7c1453b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:25.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.326+0000 7faa80900640 1 -- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa7c14fe70 msgr2=0x7faa7c0b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:25.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.326+0000 7faa80900640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa7c14fe70 0x7faa7c0b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:25.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.326+0000 7faa80900640 1 -- 172.21.15.27:0/3461652190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c1506c0 con 0x7faa7c154ef0 2023-12-19T18:28:25.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.326+0000 7faa7b7fe640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa7c14fe70 0x7faa7c0b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:25.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.326+0000 7faa80900640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa7c154ef0 0x7faa7c0b00f0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7faa74001370 tx=0x7faa7400b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:25.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.326+0000 7faa78ff9640 1 -- 172.21.15.27:0/3461652190 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa7400deb0 con 0x7faa7c154ef0 2023-12-19T18:28:25.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.327+0000 7faa78ff9640 1 -- 172.21.15.27:0/3461652190 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa7400bb80 con 0x7faa7c154ef0 2023-12-19T18:28:25.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.327+0000 7faa81101640 1 -- 172.21.15.27:0/3461652190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa7c1509a0 con 0x7faa7c154ef0 2023-12-19T18:28:25.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.327+0000 7faa78ff9640 1 -- 172.21.15.27:0/3461652190 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa74007070 con 0x7faa7c154ef0 2023-12-19T18:28:25.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.327+0000 7faa81101640 1 -- 172.21.15.27:0/3461652190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa7c150f60 con 0x7faa7c154ef0 2023-12-19T18:28:25.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.328+0000 7faa81101640 1 -- 172.21.15.27:0/3461652190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa7c04e840 con 0x7faa7c154ef0 2023-12-19T18:28:25.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.329+0000 7faa78ff9640 1 -- 172.21.15.27:0/3461652190 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7faa740071d0 con 0x7faa7c154ef0 2023-12-19T18:28:25.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.330+0000 7faa78ff9640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faa6806a430 0x7faa6806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:25.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.331+0000 7faa78ff9640 1 -- 172.21.15.27:0/3461652190 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faa74088de0 con 0x7faa7c154ef0 2023-12-19T18:28:25.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.331+0000 7faa7bfff640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faa6806a430 0x7faa6806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:25.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.332+0000 7faa7bfff640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faa6806a430 0x7faa6806c8f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7faa6c007fd0 tx=0x7faa6c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:25.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.335+0000 7faa78ff9640 1 -- 172.21.15.27:0/3461652190 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7faa740543c0 con 0x7faa7c154ef0 2023-12-19T18:28:25.530 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.530+0000 7faa81101640 1 -- 172.21.15.27:0/3461652190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7faa7c0ad720 con 0x7faa7c154ef0 2023-12-19T18:28:25.533 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.533+0000 7faa78ff9640 1 -- 172.21.15.27:0/3461652190 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v203) v1 ==== 81+0+33554 (secure 0 0 0) 0x7faa740579e0 con 0x7faa7c154ef0 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 203, 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:28:23.847199+0000", 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:28:25.534 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 23, 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:28:25.535 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:25.536 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:25.537 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:28:25.537 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:28:25.537 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:28:25.537 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:28:25.537 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:28:25.537 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:25.537 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:25.537 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:25.537 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:25.537 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:25.537 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:25.538 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:25.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.540 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.541 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 22, 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:06.542489+0000", 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:25.542 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "201", 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:25.543 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:25.544 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:25.545 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.546 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 23, 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:28:25.547 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:07.623298+0000", 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:28:25.548 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "203", 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:25.549 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:25.550 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:25.551 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:28:25.552 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:25.553 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:25.554 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.555 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:28:25.557 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:25.558 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:28:25.559 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:25.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.561 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.564 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.565 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.566 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.567 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:25.568 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:25.569 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.570 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:25.571 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.572 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:28:25.573 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:25.574 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:25.575 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.576 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.577 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.578 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:25.579 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.581 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:28:25.582 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:25.583 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:25.584 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.585 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:28:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.588 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:25.589 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:25.590 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.591 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.592 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:28:25.593 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:25.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.594 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.594 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:25.594 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:28:25.594 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:25.594 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.594 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.594 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:28:25.594 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:25.595 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.596 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:25.597 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.598 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:25.599 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.600 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:28:25.601 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:25.602 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:28:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.536+0000 7faa81101640 1 -- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faa6806a430 msgr2=0x7faa6806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.536+0000 7faa81101640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faa6806a430 0x7faa6806c8f0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7faa6c007fd0 tx=0x7faa6c009040 comp rx=0 tx=0).stop 2023-12-19T18:28:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.536+0000 7faa81101640 1 -- 172.21.15.27:0/3461652190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa7c154ef0 msgr2=0x7faa7c0b00f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.536+0000 7faa81101640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa7c154ef0 0x7faa7c0b00f0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7faa74001370 tx=0x7faa7400b4a0 comp rx=0 tx=0).stop 2023-12-19T18:28:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.536+0000 7faa81101640 1 -- 172.21.15.27:0/3461652190 shutdown_connections 2023-12-19T18:28:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.536+0000 7faa81101640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faa6806a430 0x7faa6806c8f0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:25.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.536+0000 7faa81101640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa7c154ef0 0x7faa7c0b00f0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:25.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.536+0000 7faa81101640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7faa7c14fe70 0x7faa7c0b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:25.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.536+0000 7faa81101640 1 --2- 172.21.15.27:0/3461652190 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7faa640047c0 0x7faa7c1453b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:25.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.536+0000 7faa81101640 1 -- 172.21.15.27:0/3461652190 >> 172.21.15.27:0/3461652190 conn(0x7faa7c06a7d0 msgr2=0x7faa7c0ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:25.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.536+0000 7faa81101640 1 -- 172.21.15.27:0/3461652190 shutdown_connections 2023-12-19T18:28:25.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:25.536+0000 7faa81101640 1 -- 172.21.15.27:0/3461652190 wait complete. 2023-12-19T18:28:25.605 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-12-19T18:28:30.607 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:28:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.850+0000 7fd6e7f47640 1 Processor -- start 2023-12-19T18:28:30.850 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.850+0000 7fd6e7f47640 1 -- start start 2023-12-19T18:28:30.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.852+0000 7fd6e7f47640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd6e015d260 0x7fd6e006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.852+0000 7fd6e7f47640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e00d1a50 con 0x7fd6e00d1580 2023-12-19T18:28:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.852+0000 7fd6e7f47640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6e00d1d30 con 0x7fd6e00d2af0 2023-12-19T18:28:30.852 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.852+0000 7fd6e7f47640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6e00cdaf0 con 0x7fd6e015d260 2023-12-19T18:28:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.852+0000 7fd6e64e4640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fd6e00d1580 0x7fd6e00b61a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:59710/0 (socket says 172.21.15.27:59710) 2023-12-19T18:28:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.852+0000 7fd6e64e4640 1 -- 172.21.15.27:0/4101785695 learned_addr learned my addr 172.21.15.27:0/4101785695 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e5ce3640 1 --2- 172.21.15.27:0/4101785695 >> v2:172.21.15.192:3300/0 conn(0x7fd6e015d260 0x7fd6e006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4179881677 0 0) 0x7fd6e00d1a50 con 0x7fd6e00d1580 2023-12-19T18:28:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6cc003520 con 0x7fd6e00d1580 2023-12-19T18:28:30.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 337870499 0 0) 0x7fd6cc003520 con 0x7fd6e00d1580 2023-12-19T18:28:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 >> v2:172.21.15.192:3300/0 conn(0x7fd6e015d260 msgr2=0x7fd6e006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 --2- 172.21.15.27:0/4101785695 >> v2:172.21.15.192:3300/0 conn(0x7fd6e015d260 0x7fd6e006a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 >> v1:172.21.15.192:6789/0 conn(0x7fd6e00d2af0 legacy=0x7fd6e00d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e00d2010 con 0x7fd6e00d1580 2023-12-19T18:28:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fd6d8001670 con 0x7fd6e00d1580 2023-12-19T18:28:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e5ce3640 1 --2- 172.21.15.27:0/4101785695 >> v2:172.21.15.192:3300/0 conn(0x7fd6e015d260 0x7fd6e006a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:28:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 >> v1:172.21.15.27:6789/0 conn(0x7fd6e00d1580 legacy=0x7fd6e00b61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 --2- 172.21.15.27:0/4101785695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6cc0039c0 0x7fd6cc003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 --2- 172.21.15.27:0/4101785695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6cc0047c0 0x7fd6cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 --2- 172.21.15.27:0/4101785695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6cc0051d0 0x7fd6cc00b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e00cdaf0 con 0x7fd6cc0051d0 2023-12-19T18:28:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc00c120 con 0x7fd6cc0047c0 2023-12-19T18:28:30.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.853+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc00c2e0 con 0x7fd6cc0039c0 2023-12-19T18:28:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.854+0000 7fd6e64e4640 1 --2- 172.21.15.27:0/4101785695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6cc0051d0 0x7fd6cc00b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.854+0000 7fd6e5ce3640 1 --2- 172.21.15.27:0/4101785695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6cc0039c0 0x7fd6cc003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.854+0000 7fd6e54e2640 1 --2- 172.21.15.27:0/4101785695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6cc0047c0 0x7fd6cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.854+0000 7fd6e64e4640 1 -- 172.21.15.27:0/4101785695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6cc0039c0 msgr2=0x7fd6cc003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.854+0000 7fd6e64e4640 1 --2- 172.21.15.27:0/4101785695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6cc0039c0 0x7fd6cc003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.854+0000 7fd6e64e4640 1 -- 172.21.15.27:0/4101785695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6cc0047c0 msgr2=0x7fd6cc004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.854+0000 7fd6e64e4640 1 --2- 172.21.15.27:0/4101785695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6cc0047c0 0x7fd6cc004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:30.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.854+0000 7fd6e64e4640 1 -- 172.21.15.27:0/4101785695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc003520 con 0x7fd6cc0051d0 2023-12-19T18:28:30.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.855+0000 7fd6e64e4640 1 -- 172.21.15.27:0/4101785695 reap_dead start 2023-12-19T18:28:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.855+0000 7fd6e64e4640 1 --2- 172.21.15.27:0/4101785695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6cc0051d0 0x7fd6cc00b980 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00b5a20 tx=0x7fd6cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a377183bb1d62890 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.856+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8005ad0 con 0x7fd6cc0051d0 2023-12-19T18:28:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.856+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d8005c50 con 0x7fd6cc0051d0 2023-12-19T18:28:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.856+0000 7fd6e4ce1640 1 -- 172.21.15.27:0/4101785695 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8007070 con 0x7fd6cc0051d0 2023-12-19T18:28:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.856+0000 7fd6e7f47640 1 -- 172.21.15.27:0/4101785695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6cc0051d0 msgr2=0x7fd6cc00b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.856+0000 7fd6e7f47640 1 --2- 172.21.15.27:0/4101785695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6cc0051d0 0x7fd6cc00b980 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00b5a20 tx=0x7fd6cc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.856+0000 7fd6e7f47640 1 -- 172.21.15.27:0/4101785695 shutdown_connections 2023-12-19T18:28:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.856+0000 7fd6e7f47640 1 --2- 172.21.15.27:0/4101785695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6cc0051d0 0x7fd6cc00b980 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.856+0000 7fd6e7f47640 1 -- 172.21.15.27:0/4101785695 >> 172.21.15.27:0/4101785695 conn(0x7fd6e00d2f70 msgr2=0x7fd6e00d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.857+0000 7fd6e7f47640 1 -- 172.21.15.27:0/4101785695 shutdown_connections 2023-12-19T18:28:30.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.857+0000 7fd6e7f47640 1 -- 172.21.15.27:0/4101785695 wait complete. 2023-12-19T18:28:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.858+0000 7fd6e7f47640 1 Processor -- start 2023-12-19T18:28:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.859+0000 7fd6e7f47640 1 -- start start 2023-12-19T18:28:30.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.859+0000 7fd6e7f47640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6cc0051d0 0x7fd6e0151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:30.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.859+0000 7fd6e7f47640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e00b7380 0x7fd6e00b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:30.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.859+0000 7fd6e7f47640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6e014e690 0x7fd6e014eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:30.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.859+0000 7fd6e7f47640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e00b7d20 con 0x7fd6cc0051d0 2023-12-19T18:28:30.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.859+0000 7fd6e7f47640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e015f380 con 0x7fd6e014e690 2023-12-19T18:28:30.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.859+0000 7fd6e7f47640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e01522b0 con 0x7fd6e00b7380 2023-12-19T18:28:30.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.860+0000 7fd6e5ce3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6cc0051d0 0x7fd6e0151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:30.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.860+0000 7fd6e5ce3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6cc0051d0 0x7fd6e0151d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:52610/0 (socket says 172.21.15.27:52610) 2023-12-19T18:28:30.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.860+0000 7fd6e5ce3640 1 -- 172.21.15.27:0/1693715379 learned_addr learned my addr 172.21.15.27:0/1693715379 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:30.861 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.860+0000 7fd6e64e4640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6e014e690 0x7fd6e014eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.860+0000 7fd6e5ce3640 1 -- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e00b7380 msgr2=0x7fd6e00b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.860+0000 7fd6e5ce3640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e00b7380 0x7fd6e00b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.860+0000 7fd6e5ce3640 1 -- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6e014e690 msgr2=0x7fd6e014eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.860+0000 7fd6e5ce3640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6e014e690 0x7fd6e014eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.860+0000 7fd6e5ce3640 1 -- 172.21.15.27:0/1693715379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e0153db0 con 0x7fd6cc0051d0 2023-12-19T18:28:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.860+0000 7fd6e64e4640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6e014e690 0x7fd6e014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.861+0000 7fd6e5ce3640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6cc0051d0 0x7fd6e0151d70 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc010890 tx=0x7fd6dc010d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.861+0000 7fd6d27fc640 1 -- 172.21.15.27:0/1693715379 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00cae0 con 0x7fd6cc0051d0 2023-12-19T18:28:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.861+0000 7fd6d27fc640 1 -- 172.21.15.27:0/1693715379 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6dc00cc40 con 0x7fd6cc0051d0 2023-12-19T18:28:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.861+0000 7fd6e7f47640 1 -- 172.21.15.27:0/1693715379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6e014f350 con 0x7fd6cc0051d0 2023-12-19T18:28:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.861+0000 7fd6d27fc640 1 -- 172.21.15.27:0/1693715379 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6dc00ee50 con 0x7fd6cc0051d0 2023-12-19T18:28:30.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.861+0000 7fd6e7f47640 1 -- 172.21.15.27:0/1693715379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6e015f010 con 0x7fd6cc0051d0 2023-12-19T18:28:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.862+0000 7fd6e7f47640 1 -- 172.21.15.27:0/1693715379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6e0066810 con 0x7fd6cc0051d0 2023-12-19T18:28:30.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.863+0000 7fd6d27fc640 1 -- 172.21.15.27:0/1693715379 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd6dc015470 con 0x7fd6cc0051d0 2023-12-19T18:28:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.865+0000 7fd6d27fc640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6c406a460 0x7fd6c406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.866+0000 7fd6d27fc640 1 -- 172.21.15.27:0/1693715379 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6dc088e70 con 0x7fd6cc0051d0 2023-12-19T18:28:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.866+0000 7fd6e54e2640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6c406a460 0x7fd6c406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.867+0000 7fd6e54e2640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6c406a460 0x7fd6c406c920 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00b82d0 tx=0x7fd6d4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:30.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:30.870+0000 7fd6d27fc640 1 -- 172.21.15.27:0/1693715379 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd6dc0544f0 con 0x7fd6cc0051d0 2023-12-19T18:28:31.063 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.063+0000 7fd6e7f47640 1 -- 172.21.15.27:0/1693715379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fd6e0068e90 con 0x7fd6cc0051d0 2023-12-19T18:28:31.066 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.066+0000 7fd6d27fc640 1 -- 172.21.15.27:0/1693715379 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v203) v1 ==== 81+0+33554 (secure 0 0 0) 0x7fd6dc057b10 con 0x7fd6cc0051d0 2023-12-19T18:28:31.067 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:31.067 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:28:31.067 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 203, 2023-12-19T18:28:31.067 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:28:31.067 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:28:31.067 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:28:23.847199+0000", 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:28:31.068 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 23, 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:31.069 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:31.070 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:31.071 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:31.071 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:31.071 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:31.071 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:31.071 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.071 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:28:31.071 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:31.071 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:31.071 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:31.071 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:31.072 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:31.073 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:28:31.074 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 22, 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:06.542489+0000", 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:31.075 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:31.076 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "201", 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:31.077 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.078 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.079 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.080 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.081 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 23, 2023-12-19T18:28:31.081 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:28:31.081 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:07.623298+0000", 2023-12-19T18:28:31.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:31.081 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:31.081 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:31.081 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:31.081 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:31.081 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:31.081 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:31.081 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:31.081 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:31.081 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:31.082 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:31.082 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:31.082 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:28:31.082 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:28:31.082 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:28:31.082 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:28:31.082 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:28:31.082 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:31.082 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:31.082 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:31.082 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "203", 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:31.083 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:31.084 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:31.085 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:31.086 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:31.087 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.088 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:31.089 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.090 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:31.091 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:31.092 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:31.092 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.092 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.092 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:31.092 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:31.092 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:31.092 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:31.092 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:31.092 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.093 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:28:31.094 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:31.095 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.096 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:31.097 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.098 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:28:31.099 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.100 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.102 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:28:31.102 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:28:31.102 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:28:31.102 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:28:31.102 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:31.102 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:31.102 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:31.102 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.102 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.102 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:31.103 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:31.103 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:31.103 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:31.103 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:31.103 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:31.103 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:31.103 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:31.103 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:31.103 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:31.103 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:31.103 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:31.103 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:31.103 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.104 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:28:31.105 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.106 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.107 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.108 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.109 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:31.110 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.111 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.112 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:31.113 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:31.113 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:31.113 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:31.113 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:31.113 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:31.113 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:31.113 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:31.113 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:31.114 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.115 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.116 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.117 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:31.118 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.119 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:28:31.120 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:31.121 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.122 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.123 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:28:31.123 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:28:31.123 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:28:31.123 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:28:31.123 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:31.123 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:31.123 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:31.123 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:31.124 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.125 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.126 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:31.127 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:31.128 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:31.129 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:31.130 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.131 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:31.132 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:31.133 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:31.134 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:31.134 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:31.134 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:31.134 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:31.134 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:31.134 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:28:31.134 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:31.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.134 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:28:31.135 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:31.136 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:28:31.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.069+0000 7fd6e7f47640 1 -- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6c406a460 msgr2=0x7fd6c406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:31.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.069+0000 7fd6e7f47640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6c406a460 0x7fd6c406c920 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fd6e00b82d0 tx=0x7fd6d4009040 comp rx=0 tx=0).stop 2023-12-19T18:28:31.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.069+0000 7fd6e7f47640 1 -- 172.21.15.27:0/1693715379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6cc0051d0 msgr2=0x7fd6e0151d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:31.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.069+0000 7fd6e7f47640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6cc0051d0 0x7fd6e0151d70 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd6dc010890 tx=0x7fd6dc010d50 comp rx=0 tx=0).stop 2023-12-19T18:28:31.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.069+0000 7fd6e7f47640 1 -- 172.21.15.27:0/1693715379 shutdown_connections 2023-12-19T18:28:31.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.069+0000 7fd6e7f47640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd6c406a460 0x7fd6c406c920 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:31.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.069+0000 7fd6e7f47640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd6e014e690 0x7fd6e014eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:31.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.069+0000 7fd6e7f47640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd6e00b7380 0x7fd6e00b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:31.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.069+0000 7fd6e7f47640 1 --2- 172.21.15.27:0/1693715379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6cc0051d0 0x7fd6e0151d70 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:31.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.069+0000 7fd6e7f47640 1 -- 172.21.15.27:0/1693715379 >> 172.21.15.27:0/1693715379 conn(0x7fd6e00d2f70 msgr2=0x7fd6e014ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:31.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.070+0000 7fd6e7f47640 1 -- 172.21.15.27:0/1693715379 shutdown_connections 2023-12-19T18:28:31.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:31.070+0000 7fd6e7f47640 1 -- 172.21.15.27:0/1693715379 wait complete. 2023-12-19T18:28:31.139 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-12-19T18:28:36.140 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:28:36.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.380+0000 7f96c3563640 1 Processor -- start 2023-12-19T18:28:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.380+0000 7f96c3563640 1 -- start start 2023-12-19T18:28:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.381+0000 7f96c3563640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96bc15adf0 0x7f96bc06d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.381+0000 7f96c3563640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f96bc06fc10 0x7f96bc0712e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.381+0000 7f96c3563640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96bc071820 con 0x7f96bc070c00 2023-12-19T18:28:36.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.381+0000 7f96c3563640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96bc0703c0 con 0x7f96bc06fc10 2023-12-19T18:28:36.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.382+0000 7f96c3563640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc06d8c0 con 0x7f96bc15adf0 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.382+0000 7f96c1b00640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f96bc06fc10 0x7f96bc0712e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.382+0000 7f96c1b00640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f96bc06fc10 0x7f96bc0712e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36604/0 (socket says 172.21.15.27:36604) 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.382+0000 7f96c1b00640 1 -- 172.21.15.27:0/1594503183 learned_addr learned my addr 172.21.15.27:0/1594503183 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.382+0000 7f96c12ff640 1 --2- 172.21.15.27:0/1594503183 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96bc15adf0 0x7f96bc06d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.382+0000 7f96c1b00640 1 -- 172.21.15.27:0/1594503183 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96bc15adf0 msgr2=0x7f96bc06d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.382+0000 7f96c1b00640 1 --2- 172.21.15.27:0/1594503183 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96bc15adf0 0x7f96bc06d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.382+0000 7f96c1b00640 1 -- 172.21.15.27:0/1594503183 >> v1:172.21.15.192:6789/0 conn(0x7f96bc070c00 legacy=0x7f96bc06f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.382+0000 7f96c1b00640 1 -- 172.21.15.27:0/1594503183 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96bc06b520 con 0x7f96bc06fc10 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.384+0000 7f96c1b00640 1 --2- 172.21.15.27:0/1594503183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc06fc10 0x7f96bc0712e0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009e80 tx=0x7f96b402bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14f98cf3c343f1b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.384+0000 7f96b3fff640 1 -- 172.21.15.27:0/1594503183 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b402c930 con 0x7f96bc06fc10 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.384+0000 7f96b3fff640 1 -- 172.21.15.27:0/1594503183 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b402ca90 con 0x7f96bc06fc10 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.384+0000 7f96b3fff640 1 -- 172.21.15.27:0/1594503183 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b402ce20 con 0x7f96bc06fc10 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.385+0000 7f96c3563640 1 -- 172.21.15.27:0/1594503183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc06fc10 msgr2=0x7f96bc0712e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.385+0000 7f96c3563640 1 --2- 172.21.15.27:0/1594503183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc06fc10 0x7f96bc0712e0 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009e80 tx=0x7f96b402bd90 comp rx=0 tx=0).stop 2023-12-19T18:28:36.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.385+0000 7f96c3563640 1 -- 172.21.15.27:0/1594503183 shutdown_connections 2023-12-19T18:28:36.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.385+0000 7f96c3563640 1 --2- 172.21.15.27:0/1594503183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc06fc10 0x7f96bc0712e0 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:36.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.385+0000 7f96c3563640 1 --2- 172.21.15.27:0/1594503183 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96bc15adf0 0x7f96bc06d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:36.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.385+0000 7f96c3563640 1 -- 172.21.15.27:0/1594503183 >> 172.21.15.27:0/1594503183 conn(0x7f96bc0ceb50 msgr2=0x7f96bc0cef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:36.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.385+0000 7f96c3563640 1 -- 172.21.15.27:0/1594503183 shutdown_connections 2023-12-19T18:28:36.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.385+0000 7f96c3563640 1 -- 172.21.15.27:0/1594503183 wait complete. 2023-12-19T18:28:36.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.387+0000 7f96c3563640 1 Processor -- start 2023-12-19T18:28:36.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.387+0000 7f96c3563640 1 -- start start 2023-12-19T18:28:36.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.388+0000 7f96c3563640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96bc06fc10 0x7f96bc0c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:36.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.388+0000 7f96c3563640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96bc070c00 0x7f96bc0c6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:36.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.388+0000 7f96c3563640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc15adf0 0x7f96bc0afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:36.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.388+0000 7f96c3563640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0c7a50 con 0x7f96bc15adf0 2023-12-19T18:28:36.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.388+0000 7f96c3563640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0c6b70 con 0x7f96bc06fc10 2023-12-19T18:28:36.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.388+0000 7f96c3563640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc0c6e40 con 0x7f96bc070c00 2023-12-19T18:28:36.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.388+0000 7f96c1b00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc15adf0 0x7f96bc0afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:36.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.388+0000 7f96c1b00640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc15adf0 0x7f96bc0afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36620/0 (socket says 172.21.15.27:36620) 2023-12-19T18:28:36.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.388+0000 7f96c1b00640 1 -- 172.21.15.27:0/1762032644 learned_addr learned my addr 172.21.15.27:0/1762032644 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:36.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.389+0000 7f96c12ff640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96bc06fc10 0x7f96bc0c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:36.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.389+0000 7f96c0afe640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96bc070c00 0x7f96bc0c6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.389+0000 7f96c1b00640 1 -- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96bc070c00 msgr2=0x7f96bc0c6630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.389+0000 7f96c1b00640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96bc070c00 0x7f96bc0c6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.389+0000 7f96c1b00640 1 -- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96bc06fc10 msgr2=0x7f96bc0c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.389+0000 7f96c1b00640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96bc06fc10 0x7f96bc0c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.389+0000 7f96c1b00640 1 -- 172.21.15.27:0/1762032644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b4009b30 con 0x7f96bc15adf0 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.389+0000 7f96c1b00640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc15adf0 0x7f96bc0afed0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009e80 tx=0x7f96b4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.390+0000 7f96b1ffb640 1 -- 172.21.15.27:0/1762032644 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b403e450 con 0x7f96bc15adf0 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.390+0000 7f96b1ffb640 1 -- 172.21.15.27:0/1762032644 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96b403e5b0 con 0x7f96bc15adf0 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.390+0000 7f96b1ffb640 1 -- 172.21.15.27:0/1762032644 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96b403e960 con 0x7f96bc15adf0 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.390+0000 7f96c3563640 1 -- 172.21.15.27:0/1762032644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96bc0c7120 con 0x7f96bc15adf0 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.390+0000 7f96c3563640 1 -- 172.21.15.27:0/1762032644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96bc0c76e0 con 0x7f96bc15adf0 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.391+0000 7f96c3563640 1 -- 172.21.15.27:0/1762032644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96bc1f0310 con 0x7f96bc15adf0 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.392+0000 7f96b1ffb640 1 -- 172.21.15.27:0/1762032644 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f96b403eac0 con 0x7f96bc15adf0 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.393+0000 7f96b1ffb640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f96a006a430 0x7f96a006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:36.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.393+0000 7f96b1ffb640 1 -- 172.21.15.27:0/1762032644 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96b40ab7c0 con 0x7f96bc15adf0 2023-12-19T18:28:36.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.393+0000 7f96b1ffb640 1 -- 172.21.15.27:0/1762032644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=205}) v3 -- 0x7f96a0071f60 con 0x7f96bc15adf0 2023-12-19T18:28:36.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.394+0000 7f96c12ff640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f96a006a430 0x7f96a006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:36.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.395+0000 7f96c12ff640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f96a006a430 0x7f96a006c8f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f96b8006fd0 tx=0x7f96b8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:36.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.398+0000 7f96b1ffb640 1 -- 172.21.15.27:0/1762032644 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f96b4076db0 con 0x7f96bc15adf0 2023-12-19T18:28:36.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.600+0000 7f96c3563640 1 -- 172.21.15.27:0/1762032644 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f96bc068e90 con 0x7f96bc15adf0 2023-12-19T18:28:36.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.604+0000 7f96b1ffb640 1 -- 172.21.15.27:0/1762032644 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v204) v1 ==== 81+0+33573 (secure 0 0 0) 0x7f96b4043070 con 0x7f96bc15adf0 2023-12-19T18:28:36.604 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:36.604 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 204, 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:28:31.836861+0000", 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:28:36.605 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 23, 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:28:36.606 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:28:36.607 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:36.608 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:36.609 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:36.610 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:28:36.611 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 22, 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:06.542489+0000", 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:36.612 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:36.613 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "201", 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:36.614 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:36.615 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:36.616 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:28:36.617 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 23, 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:07.623298+0000", 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1027, 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full,full_quota", 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:36.618 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:36.619 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "204", 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:28:36.620 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:36.621 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:36.622 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:36.623 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:36.624 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:36.625 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:36.626 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.627 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.628 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:36.629 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:36.630 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.631 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.632 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.633 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.634 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.635 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:28:36.636 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:28:36.637 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:28:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.638 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:36.639 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:36.640 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:28:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:28:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:36.641 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:28:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:36.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.643 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.644 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.645 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:36.646 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:36.647 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.648 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.649 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:28:36.650 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:36.651 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.652 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.653 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.654 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:28:36.655 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:36.656 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:28:36.657 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.658 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.659 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:28:36.660 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:36.661 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.662 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:36.663 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:28:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.664 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:28:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:28:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:36.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:28:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:28:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:28:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:28:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:28:36.666 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:28:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:36.667 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:36.668 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:36.669 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:28:36.670 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:36.671 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:36.672 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:28:36.673 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:28:36.674 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:36.675 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:28:36.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.606+0000 7f96c3563640 1 -- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f96a006a430 msgr2=0x7f96a006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:36.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.606+0000 7f96c3563640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f96a006a430 0x7f96a006c8f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f96b8006fd0 tx=0x7f96b8008040 comp rx=0 tx=0).stop 2023-12-19T18:28:36.676 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.606+0000 7f96c3563640 1 -- 172.21.15.27:0/1762032644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc15adf0 msgr2=0x7f96bc0afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:36.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.606+0000 7f96c3563640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc15adf0 0x7f96bc0afed0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f96b4009e80 tx=0x7f96b4004a40 comp rx=0 tx=0).stop 2023-12-19T18:28:36.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.607+0000 7f96c3563640 1 -- 172.21.15.27:0/1762032644 shutdown_connections 2023-12-19T18:28:36.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.607+0000 7f96c3563640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f96a006a430 0x7f96a006c8f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:36.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.607+0000 7f96c3563640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc15adf0 0x7f96bc0afed0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:36.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.607+0000 7f96c3563640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f96bc070c00 0x7f96bc0c6630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:36.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.607+0000 7f96c3563640 1 --2- 172.21.15.27:0/1762032644 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f96bc06fc10 0x7f96bc0c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:36.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.607+0000 7f96c3563640 1 -- 172.21.15.27:0/1762032644 >> 172.21.15.27:0/1762032644 conn(0x7f96bc0ceb50 msgr2=0x7f96bc15d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:36.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.607+0000 7f96c3563640 1 -- 172.21.15.27:0/1762032644 shutdown_connections 2023-12-19T18:28:36.677 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:36.607+0000 7f96c3563640 1 -- 172.21.15.27:0/1762032644 wait complete. 2023-12-19T18:28:36.678 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-12-19T18:28:36.678 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:28:36.678 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:28:36.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:36.741 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file_b bs=1M conv=fdatasync count=50 seek=16) 2023-12-19T18:28:36.960 INFO:teuthology.orchestra.run.smithi027.stderr:dd: error writing 'large_file_b': No space left on device 2023-12-19T18:28:36.960 INFO:teuthology.orchestra.run.smithi027.stderr:dd: closing output file 'large_file_b': No space left on device 2023-12-19T18:28:36.961 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:28:36.962 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:28:37.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.196+0000 7fe820fc6640 1 Processor -- start 2023-12-19T18:28:37.196 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.196+0000 7fe820fc6640 1 -- start start 2023-12-19T18:28:37.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.198+0000 7fe820fc6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81c154170 0x7fe81c070170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.198+0000 7fe820fc6640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe81c15aab0 0x7fe81c070f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.198+0000 7fe820fc6640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe81c071440 con 0x7fe81c070aa0 2023-12-19T18:28:37.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.198+0000 7fe820fc6640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe81c15b110 con 0x7fe81c15aab0 2023-12-19T18:28:37.198 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.198+0000 7fe820fc6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c06fce0 con 0x7fe81c154170 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.199+0000 7fe813fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe81c070aa0 0x7fe81c15a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57698/0 (socket says 172.21.15.27:57698) 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.199+0000 7fe813fff640 1 -- 172.21.15.27:0/3561377668 learned_addr learned my addr 172.21.15.27:0/3561377668 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.199+0000 7fe81b7fe640 1 --2- 172.21.15.27:0/3561377668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81c154170 0x7fe81c070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.199+0000 7fe81bfff640 1 --2- 172.21.15.27:0/3561377668 >> v2:172.21.15.192:3300/0 conn(0x7fe81c15aab0 0x7fe81c070f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.199+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1017804674 0 0) 0x7fe81c071440 con 0x7fe81c070aa0 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.199+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe808003520 con 0x7fe81c070aa0 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1784350186 0 0) 0x7fe808003520 con 0x7fe81c070aa0 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81c154170 msgr2=0x7fe81c070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 --2- 172.21.15.27:0/3561377668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81c154170 0x7fe81c070170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 >> v2:172.21.15.192:3300/0 conn(0x7fe81c15aab0 msgr2=0x7fe81c070f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 --2- 172.21.15.27:0/3561377668 >> v2:172.21.15.192:3300/0 conn(0x7fe81c15aab0 0x7fe81c070f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c06f3b0 con 0x7fe81c070aa0 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe804001670 con 0x7fe81c070aa0 2023-12-19T18:28:37.200 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 >> v1:172.21.15.27:6789/0 conn(0x7fe81c070aa0 legacy=0x7fe81c15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:37.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 --2- 172.21.15.27:0/3561377668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81c154170 0x7fe808003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 --2- 172.21.15.27:0/3561377668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe808003f40 0x7fe8080047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 --2- 172.21.15.27:0/3561377668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe808004d00 0x7fe808005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c15b110 con 0x7fe808003f40 2023-12-19T18:28:37.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c06fce0 con 0x7fe808004d00 2023-12-19T18:28:37.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe80800fe40 con 0x7fe81c154170 2023-12-19T18:28:37.201 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe813fff640 1 --2- 172.21.15.27:0/3561377668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe808003f40 0x7fe8080047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe81b7fe640 1 --2- 172.21.15.27:0/3561377668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81c154170 0x7fe808003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe813fff640 1 -- 172.21.15.27:0/3561377668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81c154170 msgr2=0x7fe808003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe813fff640 1 --2- 172.21.15.27:0/3561377668 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81c154170 0x7fe808003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe813fff640 1 -- 172.21.15.27:0/3561377668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe808004d00 msgr2=0x7fe808005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.200+0000 7fe813fff640 1 --2- 172.21.15.27:0/3561377668 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe808004d00 0x7fe808005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.201+0000 7fe813fff640 1 -- 172.21.15.27:0/3561377668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe808003520 con 0x7fe808003f40 2023-12-19T18:28:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.202+0000 7fe813fff640 1 --2- 172.21.15.27:0/3561377668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe808003f40 0x7fe8080047c0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fe80400c190 tx=0x7fe8080057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=436194a6694735c server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.202+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe804005ad0 con 0x7fe808003f40 2023-12-19T18:28:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.202+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe804005c50 con 0x7fe808003f40 2023-12-19T18:28:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.202+0000 7fe81affd640 1 -- 172.21.15.27:0/3561377668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80402c810 con 0x7fe808003f40 2023-12-19T18:28:37.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.203+0000 7fe820fc6640 1 -- 172.21.15.27:0/3561377668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe808003f40 msgr2=0x7fe8080047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:37.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.203+0000 7fe820fc6640 1 --2- 172.21.15.27:0/3561377668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe808003f40 0x7fe8080047c0 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7fe80400c190 tx=0x7fe8080057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.203 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.203+0000 7fe81bfff640 1 -- 172.21.15.27:0/3561377668 reap_dead start 2023-12-19T18:28:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.203+0000 7fe820fc6640 1 -- 172.21.15.27:0/3561377668 shutdown_connections 2023-12-19T18:28:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.203+0000 7fe820fc6640 1 -- 172.21.15.27:0/3561377668 >> 172.21.15.27:0/3561377668 conn(0x7fe81c06a7d0 msgr2=0x7fe81c0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.203+0000 7fe820fc6640 1 -- 172.21.15.27:0/3561377668 shutdown_connections 2023-12-19T18:28:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.203+0000 7fe820fc6640 1 -- 172.21.15.27:0/3561377668 wait complete. 2023-12-19T18:28:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.205+0000 7fe820fc6640 1 Processor -- start 2023-12-19T18:28:37.204 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.205+0000 7fe820fc6640 1 -- start start 2023-12-19T18:28:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.206+0000 7fe820fc6640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe808003f40 0x7fe81c1eeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.206+0000 7fe820fc6640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe81c14e770 0x7fe81c14ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.206+0000 7fe820fc6640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81c14f270 0x7fe81c14f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.206+0000 7fe820fc6640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c1f2610 con 0x7fe808003f40 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.206+0000 7fe820fc6640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c14fc60 con 0x7fe81c14e770 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.206+0000 7fe820fc6640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe81c1f1990 con 0x7fe81c14f270 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.206+0000 7fe81b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe808003f40 0x7fe81c1eeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.206+0000 7fe81b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe808003f40 0x7fe81c1eeca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36644/0 (socket says 172.21.15.27:36644) 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.206+0000 7fe81b7fe640 1 -- 172.21.15.27:0/424452134 learned_addr learned my addr 172.21.15.27:0/424452134 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.207+0000 7fe81bfff640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81c14f270 0x7fe81c14f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.207+0000 7fe813fff640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe81c14e770 0x7fe81c14ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.207+0000 7fe81b7fe640 1 -- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81c14f270 msgr2=0x7fe81c14f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.207+0000 7fe81b7fe640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81c14f270 0x7fe81c14f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.207+0000 7fe81b7fe640 1 -- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe81c14e770 msgr2=0x7fe81c14ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.207+0000 7fe81b7fe640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe81c14e770 0x7fe81c14ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.207+0000 7fe81b7fe640 1 -- 172.21.15.27:0/424452134 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c1f1c70 con 0x7fe808003f40 2023-12-19T18:28:37.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.207+0000 7fe81b7fe640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe808003f40 0x7fe81c1eeca0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0115c0 tx=0x7fe80c011a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:37.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.207+0000 7fe818ff9640 1 -- 172.21.15.27:0/424452134 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c015870 con 0x7fe808003f40 2023-12-19T18:28:37.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.207+0000 7fe818ff9640 1 -- 172.21.15.27:0/424452134 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe80c0159d0 con 0x7fe808003f40 2023-12-19T18:28:37.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.207+0000 7fe818ff9640 1 -- 172.21.15.27:0/424452134 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe80c015d80 con 0x7fe808003f40 2023-12-19T18:28:37.208 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.207+0000 7fe820fc6640 1 -- 172.21.15.27:0/424452134 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81c1f1f50 con 0x7fe808003f40 2023-12-19T18:28:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.208+0000 7fe820fc6640 1 -- 172.21.15.27:0/424452134 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81c1ef1e0 con 0x7fe808003f40 2023-12-19T18:28:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.209+0000 7fe820fc6640 1 -- 172.21.15.27:0/424452134 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe81c04e840 con 0x7fe808003f40 2023-12-19T18:28:37.209 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.210+0000 7fe818ff9640 1 -- 172.21.15.27:0/424452134 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe80c01d400 con 0x7fe808003f40 2023-12-19T18:28:37.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.211+0000 7fe818ff9640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe7fc06a460 0x7fe7fc06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.211+0000 7fe818ff9640 1 -- 172.21.15.27:0/424452134 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe80c088ec0 con 0x7fe808003f40 2023-12-19T18:28:37.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.211+0000 7fe818ff9640 1 -- 172.21.15.27:0/424452134 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=205}) v3 -- 0x7fe7fc071f90 con 0x7fe808003f40 2023-12-19T18:28:37.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.211+0000 7fe813fff640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe7fc06a460 0x7fe7fc06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.213+0000 7fe813fff640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe7fc06a460 0x7fe7fc06c920 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe804034000 tx=0x7fe804007000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:37.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.216+0000 7fe818ff9640 1 -- 172.21.15.27:0/424452134 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe80c054540 con 0x7fe808003f40 2023-12-19T18:28:37.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.409+0000 7fe820fc6640 1 -- 172.21.15.27:0/424452134 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fe81c0cabf0 con 0x7fe808003f40 2023-12-19T18:28:37.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.412+0000 7fe818ff9640 1 -- 172.21.15.27:0/424452134 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v204) v1 ==== 81+0+33573 (secure 0 0 0) 0x7fe80c022020 con 0x7fe808003f40 2023-12-19T18:28:37.413 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:37.413 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:28:37.413 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 204, 2023-12-19T18:28:37.413 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:28:37.413 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:28:37.413 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:28:31.836861+0000", 2023-12-19T18:28:37.413 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:28:37.413 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 23, 2023-12-19T18:28:37.414 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:37.415 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 22, 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:06.542489+0000", 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:37.421 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:37.422 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "201", 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:37.423 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:37.424 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:37.425 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:28:37.426 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 23, 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:07.623298+0000", 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1027, 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full,full_quota", 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:37.427 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:37.428 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "204", 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:37.429 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:37.430 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:37.431 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:37.432 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:37.433 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:37.434 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.435 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.437 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:37.438 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.440 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:28:37.441 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:37.442 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:37.443 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:37.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.446 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.447 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:37.448 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.449 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:28:37.450 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.451 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.452 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.453 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.454 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.455 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:28:37.456 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:37.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.456 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:28:37.456 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:37.456 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.456 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.456 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:37.456 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.456 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.457 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:28:37.457 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:37.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.457 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.457 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.457 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:28:37.457 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:37.457 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.457 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.457 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:37.457 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.458 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.458 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:28:37.458 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:37.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.458 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.458 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.459 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.460 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.461 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.462 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:28:37.463 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:28:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:28:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:28:37.464 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:37.465 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.466 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.467 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.468 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:28:37.469 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:37.470 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.472 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.473 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:28:37.474 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:37.475 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:37.477 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:37.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:37.479 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:28:37.481 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:28:37.482 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:28:37.483 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:28:37.483 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:28:37.483 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:28:37.483 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:28:37.483 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:28:37.483 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:28:37.483 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:28:37.483 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:28:37.483 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:37.483 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:28:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.415+0000 7fe820fc6640 1 -- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe7fc06a460 msgr2=0x7fe7fc06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.415+0000 7fe820fc6640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe7fc06a460 0x7fe7fc06c920 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe804034000 tx=0x7fe804007000 comp rx=0 tx=0).stop 2023-12-19T18:28:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.415+0000 7fe820fc6640 1 -- 172.21.15.27:0/424452134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe808003f40 msgr2=0x7fe81c1eeca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.415+0000 7fe820fc6640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe808003f40 0x7fe81c1eeca0 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7fe80c0115c0 tx=0x7fe80c011a80 comp rx=0 tx=0).stop 2023-12-19T18:28:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.415+0000 7fe820fc6640 1 -- 172.21.15.27:0/424452134 shutdown_connections 2023-12-19T18:28:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.415+0000 7fe820fc6640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe7fc06a460 0x7fe7fc06c920 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.415+0000 7fe820fc6640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe81c14f270 0x7fe81c14f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.415+0000 7fe820fc6640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe81c14e770 0x7fe81c14ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.415+0000 7fe820fc6640 1 --2- 172.21.15.27:0/424452134 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe808003f40 0x7fe81c1eeca0 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.415+0000 7fe820fc6640 1 -- 172.21.15.27:0/424452134 >> 172.21.15.27:0/424452134 conn(0x7fe81c06a7d0 msgr2=0x7fe81c0b0c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:37.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.415+0000 7fe820fc6640 1 -- 172.21.15.27:0/424452134 shutdown_connections 2023-12-19T18:28:37.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.415+0000 7fe820fc6640 1 -- 172.21.15.27:0/424452134 wait complete. 2023-12-19T18:28:37.486 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:28:37.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.732+0000 7fe3d1a1f640 1 Processor -- start 2023-12-19T18:28:37.732 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.732+0000 7fe3d1a1f640 1 -- start start 2023-12-19T18:28:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.734+0000 7fe3d1a1f640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe3cc1553a0 0x7fe3cc0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.734+0000 7fe3d1a1f640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe3cc15ee30 0x7fe3cc0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.734+0000 7fe3d1a1f640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3cc0b0360 con 0x7fe3cc0afeb0 2023-12-19T18:28:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.734+0000 7fe3d1a1f640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3cc0cbcf0 con 0x7fe3cc15ee30 2023-12-19T18:28:37.734 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.734+0000 7fe3d1a1f640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3cc0cb350 con 0x7fe3cc1553a0 2023-12-19T18:28:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.735+0000 7fe3d121e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe3cc15ee30 0x7fe3cc0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.735+0000 7fe3d121e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe3cc15ee30 0x7fe3cc0c39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36650/0 (socket says 172.21.15.27:36650) 2023-12-19T18:28:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.735+0000 7fe3d121e640 1 -- 172.21.15.27:0/3057098233 learned_addr learned my addr 172.21.15.27:0/3057098233 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.735+0000 7fe3d0a1d640 1 --2- 172.21.15.27:0/3057098233 >> v2:172.21.15.192:3300/0 conn(0x7fe3cc1553a0 0x7fe3cc0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.735+0000 7fe3d121e640 1 -- 172.21.15.27:0/3057098233 >> v2:172.21.15.192:3300/0 conn(0x7fe3cc1553a0 msgr2=0x7fe3cc0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.735+0000 7fe3d121e640 1 --2- 172.21.15.27:0/3057098233 >> v2:172.21.15.192:3300/0 conn(0x7fe3cc1553a0 0x7fe3cc0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.735 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.735+0000 7fe3d121e640 1 -- 172.21.15.27:0/3057098233 >> v1:172.21.15.192:6789/0 conn(0x7fe3cc0afeb0 legacy=0x7fe3cc15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:37.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.735+0000 7fe3d121e640 1 -- 172.21.15.27:0/3057098233 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3cc0cb630 con 0x7fe3cc15ee30 2023-12-19T18:28:37.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.736+0000 7fe3d121e640 1 --2- 172.21.15.27:0/3057098233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc15ee30 0x7fe3cc0c39d0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0cc1c0 tx=0x7fe3c402be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c23d3e417caa313 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:37.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.737+0000 7fe3cb7fe640 1 -- 172.21.15.27:0/3057098233 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c402c9f0 con 0x7fe3cc15ee30 2023-12-19T18:28:37.736 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.737+0000 7fe3cb7fe640 1 -- 172.21.15.27:0/3057098233 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c402cb50 con 0x7fe3cc15ee30 2023-12-19T18:28:37.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.737+0000 7fe3cb7fe640 1 -- 172.21.15.27:0/3057098233 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c40343f0 con 0x7fe3cc15ee30 2023-12-19T18:28:37.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.737+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/3057098233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc15ee30 msgr2=0x7fe3cc0c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:37.737 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.737+0000 7fe3d1a1f640 1 --2- 172.21.15.27:0/3057098233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc15ee30 0x7fe3cc0c39d0 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7fe3cc0cc1c0 tx=0x7fe3c402be50 comp rx=0 tx=0).stop 2023-12-19T18:28:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.738+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/3057098233 shutdown_connections 2023-12-19T18:28:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.738+0000 7fe3d1a1f640 1 --2- 172.21.15.27:0/3057098233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc15ee30 0x7fe3cc0c39d0 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.738+0000 7fe3d1a1f640 1 --2- 172.21.15.27:0/3057098233 >> v2:172.21.15.192:3300/0 conn(0x7fe3cc1553a0 0x7fe3cc0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.738+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/3057098233 >> 172.21.15.27:0/3057098233 conn(0x7fe3cc069c70 msgr2=0x7fe3cc06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.738+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/3057098233 shutdown_connections 2023-12-19T18:28:37.738 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.738+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/3057098233 wait complete. 2023-12-19T18:28:37.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.740+0000 7fe3d1a1f640 1 Processor -- start 2023-12-19T18:28:37.740 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.740+0000 7fe3d1a1f640 1 -- start start 2023-12-19T18:28:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.742+0000 7fe3d1a1f640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3cc0afeb0 0x7fe3cc0c8920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.742+0000 7fe3d1a1f640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc1553a0 0x7fe3cc15d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.742+0000 7fe3d1a1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc15ee30 0x7fe3cc15d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.742+0000 7fe3d1a1f640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc0c63a0 con 0x7fe3cc15ee30 2023-12-19T18:28:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.742+0000 7fe3d1a1f640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc0c5ab0 con 0x7fe3cc1553a0 2023-12-19T18:28:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.742+0000 7fe3d1a1f640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3cc0c65f0 con 0x7fe3cc0afeb0 2023-12-19T18:28:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.742+0000 7fe3d121e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc15ee30 0x7fe3cc15d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.742+0000 7fe3d121e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc15ee30 0x7fe3cc15d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36658/0 (socket says 172.21.15.27:36658) 2023-12-19T18:28:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.742+0000 7fe3d121e640 1 -- 172.21.15.27:0/34731723 learned_addr learned my addr 172.21.15.27:0/34731723 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.742+0000 7fe3d0a1d640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3cc0afeb0 0x7fe3cc0c8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.742+0000 7fe3cbfff640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc1553a0 0x7fe3cc15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.743+0000 7fe3d121e640 1 -- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3cc0afeb0 msgr2=0x7fe3cc0c8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.743+0000 7fe3d121e640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3cc0afeb0 0x7fe3cc0c8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.743+0000 7fe3d121e640 1 -- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc1553a0 msgr2=0x7fe3cc15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.743+0000 7fe3d121e640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc1553a0 0x7fe3cc15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.743+0000 7fe3d121e640 1 -- 172.21.15.27:0/34731723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c4009b30 con 0x7fe3cc15ee30 2023-12-19T18:28:37.744 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.743+0000 7fe3d0a1d640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3cc0afeb0 0x7fe3cc0c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:37.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.743+0000 7fe3cbfff640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc1553a0 0x7fe3cc15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:37.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.743+0000 7fe3d121e640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc15ee30 0x7fe3cc15d860 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4036040 tx=0x7fe3c4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:37.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.743+0000 7fe3c97fa640 1 -- 172.21.15.27:0/34731723 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c4009510 con 0x7fe3cc15ee30 2023-12-19T18:28:37.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.743+0000 7fe3c97fa640 1 -- 172.21.15.27:0/34731723 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3c403e450 con 0x7fe3cc15ee30 2023-12-19T18:28:37.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.743+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/34731723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3cc0c5d50 con 0x7fe3cc15ee30 2023-12-19T18:28:37.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.743+0000 7fe3c97fa640 1 -- 172.21.15.27:0/34731723 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c403e7e0 con 0x7fe3cc15ee30 2023-12-19T18:28:37.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.743+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/34731723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3cc0c68d0 con 0x7fe3cc15ee30 2023-12-19T18:28:37.745 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.745+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/34731723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3cc0c5f90 con 0x7fe3cc15ee30 2023-12-19T18:28:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.746+0000 7fe3c97fa640 1 -- 172.21.15.27:0/34731723 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe3c403e940 con 0x7fe3cc15ee30 2023-12-19T18:28:37.746 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.747+0000 7fe3c97fa640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe3b006a430 0x7fe3b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.747+0000 7fe3c97fa640 1 -- 172.21.15.27:0/34731723 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe3c40ab7d0 con 0x7fe3cc15ee30 2023-12-19T18:28:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.747+0000 7fe3c97fa640 1 -- 172.21.15.27:0/34731723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=205}) v3 -- 0x7fe3b0071f60 con 0x7fe3cc15ee30 2023-12-19T18:28:37.747 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.748+0000 7fe3d0a1d640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe3b006a430 0x7fe3b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:37.749 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.749+0000 7fe3d0a1d640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe3b006a430 0x7fe3b006c8f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0006fd0 tx=0x7fe3c0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:37.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.752+0000 7fe3c97fa640 1 -- 172.21.15.27:0/34731723 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe3c4043060 con 0x7fe3cc15ee30 2023-12-19T18:28:37.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.987+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/34731723 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3cc15a2d0 con 0x7fe3cc15ee30 2023-12-19T18:28:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.988+0000 7fe3c97fa640 1 -- 172.21.15.27:0/34731723 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4713 (secure 0 0 0) 0x7fe3c4043030 con 0x7fe3cc15ee30 2023-12-19T18:28:37.988 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:37.988 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1864159643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1864159643},{"type":"v1","addr":"172.21.15.192:6836","nonce":1864159643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1348219767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1348219767},{"type":"v1","addr":"172.21.15.192:6837","nonce":1348219767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/181137639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":181137639},{"type":"v1","addr":"172.21.15.27:6833","nonce":181137639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:28:08.577742+0000","modified":"2023-12-19T18:28:12.083302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/152568921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":152568921},{"type":"v1","addr":"172.21.15.27:6835","nonce":152568921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:28:37.988 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 85 2023-12-19T18:28:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.990+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe3b006a430 msgr2=0x7fe3b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:37.990 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.990+0000 7fe3d1a1f640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe3b006a430 0x7fe3b006c8f0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0006fd0 tx=0x7fe3c0008040 comp rx=0 tx=0).stop 2023-12-19T18:28:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.990+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/34731723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc15ee30 msgr2=0x7fe3cc15d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.990+0000 7fe3d1a1f640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc15ee30 0x7fe3cc15d860 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fe3c4036040 tx=0x7fe3c4004a40 comp rx=0 tx=0).stop 2023-12-19T18:28:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.991+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/34731723 shutdown_connections 2023-12-19T18:28:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.991+0000 7fe3d1a1f640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe3b006a430 0x7fe3b006c8f0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.991+0000 7fe3d1a1f640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3cc15ee30 0x7fe3cc15d860 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.991+0000 7fe3d1a1f640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe3cc1553a0 0x7fe3cc15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.991+0000 7fe3d1a1f640 1 --2- 172.21.15.27:0/34731723 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe3cc0afeb0 0x7fe3cc0c8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.991+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/34731723 >> 172.21.15.27:0/34731723 conn(0x7fe3cc069c70 msgr2=0x7fe3cc159bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.991+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/34731723 shutdown_connections 2023-12-19T18:28:37.991 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:37.991+0000 7fe3d1a1f640 1 -- 172.21.15.27:0/34731723 wait complete. 2023-12-19T18:28:38.010 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-12-19T18:28:38.265 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":8997,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":29.667867559000001,"mdsmap_epoch":85,"osdmap_epoch":204,"osdmap_epoch_barrier":202,"uptime":35.461788361000004} 2023-12-19T18:28:38.265 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 8997, "mdsmap_epoch": 85, "osdmap_epoch": 204, "osdmap_epoch_barrier": 202, "rank_uptime": 29.667867559, "state": "up:active", "uptime": 35.461788361, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:28:38.265 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-12-19T18:28:38.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:38.266 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_b) 2023-12-19T18:28:38.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:38.329 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec rm -f large_file_a) 2023-12-19T18:28:38.394 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:28:38.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.618+0000 7f800b269640 1 Processor -- start 2023-12-19T18:28:38.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.618+0000 7f800b269640 1 -- start start 2023-12-19T18:28:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.620+0000 7f800b269640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800415d260 0x7f800406a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.620+0000 7f800b269640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f80040d2af0 0x7f800406a930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.620+0000 7f800b269640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80040b71b0 con 0x7f80040d0e70 2023-12-19T18:28:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.620+0000 7f800b269640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80040cdaf0 con 0x7f80040d2af0 2023-12-19T18:28:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.620+0000 7f800b269640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80040d1a50 con 0x7f800415d260 2023-12-19T18:28:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.620+0000 7f8009806640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f80040d0e70 0x7f80040b49f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:57700/0 (socket says 172.21.15.27:57700) 2023-12-19T18:28:38.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.620+0000 7f8009806640 1 -- 172.21.15.27:0/3230805677 learned_addr learned my addr 172.21.15.27:0/3230805677 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:38.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.620+0000 7f8008804640 1 --2- 172.21.15.27:0/3230805677 >> v2:172.21.15.192:3300/0 conn(0x7f80040d2af0 0x7f800406a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:38.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.620+0000 7f8009005640 1 --2- 172.21.15.27:0/3230805677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800415d260 0x7f800406a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:38.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 574924663 0 0) 0x7f7ffc001490 con 0x7f80040d0e70 2023-12-19T18:28:38.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ff0003520 con 0x7f80040d0e70 2023-12-19T18:28:38.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2301411914 0 0) 0x7f7ff0003520 con 0x7f80040d0e70 2023-12-19T18:28:38.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800415d260 msgr2=0x7f800406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:38.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 --2- 172.21.15.27:0/3230805677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800415d260 0x7f800406a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:38.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 >> v2:172.21.15.192:3300/0 conn(0x7f80040d2af0 msgr2=0x7f800406a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:38.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 --2- 172.21.15.27:0/3230805677 >> v2:172.21.15.192:3300/0 conn(0x7f80040d2af0 0x7f800406a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:38.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80040d1d30 con 0x7f80040d0e70 2023-12-19T18:28:38.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f7ffc001b80 con 0x7f80040d0e70 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 >> v1:172.21.15.27:6789/0 conn(0x7f80040d0e70 legacy=0x7f80040b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 --2- 172.21.15.27:0/3230805677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff0003970 0x7f7ff0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 --2- 172.21.15.27:0/3230805677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff00047c0 0x7f7ff0004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 --2- 172.21.15.27:0/3230805677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800415d260 0x7f7ff0005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80040cdaf0 con 0x7f7ff0003970 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80040d1a50 con 0x7f7ff00047c0 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.621+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff000b980 con 0x7f800415d260 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.622+0000 7f8009005640 1 --2- 172.21.15.27:0/3230805677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff0003970 0x7f7ff0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.622+0000 7f8008804640 1 --2- 172.21.15.27:0/3230805677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff00047c0 0x7f7ff0004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.622+0000 7f8009005640 1 -- 172.21.15.27:0/3230805677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800415d260 msgr2=0x7f7ff0005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.622+0000 7f8009005640 1 --2- 172.21.15.27:0/3230805677 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f800415d260 0x7f7ff0005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.622+0000 7f8009005640 1 -- 172.21.15.27:0/3230805677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff00047c0 msgr2=0x7f7ff0004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.622+0000 7f8009005640 1 --2- 172.21.15.27:0/3230805677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff00047c0 0x7f7ff0004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:38.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.622+0000 7f8009005640 1 -- 172.21.15.27:0/3230805677 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff0003520 con 0x7f7ff0003970 2023-12-19T18:28:38.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.622+0000 7f8008804640 1 --2- 172.21.15.27:0/3230805677 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7ff00047c0 0x7f7ff0004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:38.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.623+0000 7f8009005640 1 --2- 172.21.15.27:0/3230805677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff0003970 0x7f7ff0003d90 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f8000009ac0 tx=0x7f7ff00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298ccd800fdc94eb server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:38.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.623+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800002c6d0 con 0x7f7ff0003970 2023-12-19T18:28:38.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.623+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f800002c830 con 0x7f7ff0003970 2023-12-19T18:28:38.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.623+0000 7f7ffbfff640 1 -- 172.21.15.27:0/3230805677 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f800002cbc0 con 0x7f7ff0003970 2023-12-19T18:28:38.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.624+0000 7f800b269640 1 -- 172.21.15.27:0/3230805677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff0003970 msgr2=0x7f7ff0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:38.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.624+0000 7f800b269640 1 --2- 172.21.15.27:0/3230805677 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff0003970 0x7f7ff0003d90 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f8000009ac0 tx=0x7f7ff00057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:38.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.624+0000 7f8009806640 1 -- 172.21.15.27:0/3230805677 reap_dead start 2023-12-19T18:28:38.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.624+0000 7f800b269640 1 -- 172.21.15.27:0/3230805677 shutdown_connections 2023-12-19T18:28:38.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.624+0000 7f800b269640 1 -- 172.21.15.27:0/3230805677 >> 172.21.15.27:0/3230805677 conn(0x7f80040d2f70 msgr2=0x7f80040d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:38.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.624+0000 7f800b269640 1 -- 172.21.15.27:0/3230805677 shutdown_connections 2023-12-19T18:28:38.625 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.624+0000 7f800b269640 1 -- 172.21.15.27:0/3230805677 wait complete. 2023-12-19T18:28:38.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.626+0000 7f800b269640 1 Processor -- start 2023-12-19T18:28:38.626 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.626+0000 7f800b269640 1 -- start start 2023-12-19T18:28:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.627+0000 7f800b269640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff0003970 0x7f80040b5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.627+0000 7f800b269640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80041f3380 0x7f800415fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.627+0000 7f800b269640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80040aa290 0x7f80040aa710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.627+0000 7f800b269640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80040aac50 con 0x7f7ff0003970 2023-12-19T18:28:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.627+0000 7f800b269640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80040b4210 con 0x7f80041f3380 2023-12-19T18:28:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.627+0000 7f800b269640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80041602c0 con 0x7f80040aa290 2023-12-19T18:28:38.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.627+0000 7f8009005640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff0003970 0x7f80040b5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.628+0000 7f8009005640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff0003970 0x7f80040b5570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36676/0 (socket says 172.21.15.27:36676) 2023-12-19T18:28:38.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.628+0000 7f8009005640 1 -- 172.21.15.27:0/922220315 learned_addr learned my addr 172.21.15.27:0/922220315 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.628+0000 7f8008804640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80041f3380 0x7f800415fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.628+0000 7f8009806640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80040aa290 0x7f80040aa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.628+0000 7f8009005640 1 -- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80040aa290 msgr2=0x7f80040aa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.628+0000 7f8009005640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80040aa290 0x7f80040aa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.628+0000 7f8009005640 1 -- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80041f3380 msgr2=0x7f800415fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.628+0000 7f8009005640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80041f3380 0x7f800415fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.628+0000 7f8009005640 1 -- 172.21.15.27:0/922220315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80040b44d0 con 0x7f7ff0003970 2023-12-19T18:28:38.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.629+0000 7f8009005640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff0003970 0x7f80040b5570 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f800002c2d0 tx=0x7f8000004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:38.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.629+0000 7f7ff9ffb640 1 -- 172.21.15.27:0/922220315 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000003810 con 0x7f7ff0003970 2023-12-19T18:28:38.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.629+0000 7f7ff9ffb640 1 -- 172.21.15.27:0/922220315 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8000003970 con 0x7f7ff0003970 2023-12-19T18:28:38.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.629+0000 7f7ff9ffb640 1 -- 172.21.15.27:0/922220315 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8000003ce0 con 0x7f7ff0003970 2023-12-19T18:28:38.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.629+0000 7f800b269640 1 -- 172.21.15.27:0/922220315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80040b47b0 con 0x7f7ff0003970 2023-12-19T18:28:38.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.629+0000 7f800b269640 1 -- 172.21.15.27:0/922220315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80040b4d70 con 0x7f7ff0003970 2023-12-19T18:28:38.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.630+0000 7f800b269640 1 -- 172.21.15.27:0/922220315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8004066810 con 0x7f7ff0003970 2023-12-19T18:28:38.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.631+0000 7f7ff9ffb640 1 -- 172.21.15.27:0/922220315 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8000040450 con 0x7f7ff0003970 2023-12-19T18:28:38.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.633+0000 7f7ff9ffb640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fe806a460 0x7f7fe806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:38.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.633+0000 7f7ff9ffb640 1 -- 172.21.15.27:0/922220315 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f80000aa800 con 0x7f7ff0003970 2023-12-19T18:28:38.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.633+0000 7f7ff9ffb640 1 -- 172.21.15.27:0/922220315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=205}) v3 -- 0x7f7fe8071f90 con 0x7f7ff0003970 2023-12-19T18:28:38.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.633+0000 7f8008804640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fe806a460 0x7f7fe806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:38.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.634+0000 7f8008804640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fe806a460 0x7f7fe806c920 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f80040b3500 tx=0x7f7ff4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:38.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.637+0000 7f7ff9ffb640 1 -- 172.21.15.27:0/922220315 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f8000075d50 con 0x7f7ff0003970 2023-12-19T18:28:38.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.834+0000 7f800b269640 1 -- 172.21.15.27:0/922220315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f8004068e90 con 0x7f7ff0003970 2023-12-19T18:28:38.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.837+0000 7f7ff9ffb640 1 -- 172.21.15.27:0/922220315 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v204) v1 ==== 81+0+33573 (secure 0 0 0) 0x7f8000039090 con 0x7f7ff0003970 2023-12-19T18:28:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:38.838 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:28:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 204, 2023-12-19T18:28:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:28:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:28:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:28:31.836861+0000", 2023-12-19T18:28:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:28:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:28:38.838 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 23, 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:28:38.839 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:38.840 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:28:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:28:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:28:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:28:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:28:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:38.841 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:28:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:38.842 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:38.843 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:38.844 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.845 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 22, 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:06.542489+0000", 2023-12-19T18:28:38.846 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:28:38.847 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:28:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:28:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:28:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "201", 2023-12-19T18:28:38.848 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:38.849 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:38.850 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:28:38.851 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 23, 2023-12-19T18:28:38.852 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:28:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:07.623298+0000", 2023-12-19T18:28:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1027, 2023-12-19T18:28:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full,full_quota", 2023-12-19T18:28:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:38.853 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "204", 2023-12-19T18:28:38.854 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:38.855 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:38.856 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:38.857 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.858 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:38.859 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.860 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:38.861 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:38.862 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:28:38.863 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:28:38.864 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:28:38.864 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:38.864 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:38.864 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:38.864 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.864 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.864 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:38.864 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:38.864 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:38.864 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:38.864 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:38.864 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.865 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:28:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:28:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:38.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:38.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.868 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:38.869 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:38.870 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:38.871 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.872 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:28:38.872 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:38.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.872 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.872 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.872 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:28:38.872 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:38.872 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.872 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.872 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.872 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:38.873 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:28:38.874 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:38.875 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:28:38.876 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:38.877 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.878 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.879 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:38.880 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.881 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.882 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.883 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:28:38.884 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:38.885 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.885 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.885 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:28:38.885 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:28:38.885 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:28:38.885 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:28:38.885 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:38.885 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:38.885 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:38.885 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:38.886 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:38.887 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.888 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.889 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:28:38.890 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:38.891 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.892 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:38.893 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:38.894 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:28:38.895 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:28:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:28:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:38.901 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.902 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:38.903 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.904 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:38.905 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:38.906 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.907 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:38.908 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:28:38.908 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:28:38.908 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:28:38.908 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:28:38.908 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:28:38.908 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:28:38.908 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:28:38.908 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:28:38.908 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:28:38.908 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:28:38.908 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:28:38.908 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:38.908 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:28:38.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.840+0000 7f800b269640 1 -- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fe806a460 msgr2=0x7f7fe806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:38.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.840+0000 7f800b269640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fe806a460 0x7f7fe806c920 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f80040b3500 tx=0x7f7ff4008040 comp rx=0 tx=0).stop 2023-12-19T18:28:38.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.840+0000 7f800b269640 1 -- 172.21.15.27:0/922220315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff0003970 msgr2=0x7f80040b5570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:38.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.840+0000 7f800b269640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff0003970 0x7f80040b5570 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f800002c2d0 tx=0x7f8000004730 comp rx=0 tx=0).stop 2023-12-19T18:28:38.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.840+0000 7f800b269640 1 -- 172.21.15.27:0/922220315 shutdown_connections 2023-12-19T18:28:38.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.840+0000 7f800b269640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f7fe806a460 0x7f7fe806c920 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:38.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.840+0000 7f800b269640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f80040aa290 0x7f80040aa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:38.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.840+0000 7f800b269640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f80041f3380 0x7f800415fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:38.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.840+0000 7f800b269640 1 --2- 172.21.15.27:0/922220315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff0003970 0x7f80040b5570 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:38.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.840+0000 7f800b269640 1 -- 172.21.15.27:0/922220315 >> 172.21.15.27:0/922220315 conn(0x7f80040d2f70 msgr2=0x7f800415b450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:38.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.840+0000 7f800b269640 1 -- 172.21.15.27:0/922220315 shutdown_connections 2023-12-19T18:28:38.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:38.840+0000 7f800b269640 1 -- 172.21.15.27:0/922220315 wait complete. 2023-12-19T18:28:38.911 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-12-19T18:28:43.912 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:28:44.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.156+0000 7fa8c6949640 1 Processor -- start 2023-12-19T18:28:44.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.156+0000 7fa8c6949640 1 -- start start 2023-12-19T18:28:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.158+0000 7fa8c6949640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa8c015d0a0 0x7fa8c00cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.158+0000 7fa8c6949640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8c00cfdb0 0x7fa8c0160f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.158+0000 7fa8c6949640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c00cdb20 con 0x7fa8c0160b40 2023-12-19T18:28:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.158+0000 7fa8c6949640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c01614c0 con 0x7fa8c015d0a0 2023-12-19T18:28:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.158+0000 7fa8c6949640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c015d480 con 0x7fa8c00cfdb0 2023-12-19T18:28:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.158+0000 7fa8c5146640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa8c0160b40 0x7fa8c00cf6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44230/0 (socket says 172.21.15.27:44230) 2023-12-19T18:28:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.158+0000 7fa8c5146640 1 -- 172.21.15.27:0/338732309 learned_addr learned my addr 172.21.15.27:0/338732309 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.159+0000 7fa8c5947640 1 --2- 172.21.15.27:0/338732309 >> v2:172.21.15.192:3300/0 conn(0x7fa8c015d0a0 0x7fa8c00cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.159+0000 7fa8c6148640 1 --2- 172.21.15.27:0/338732309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8c00cfdb0 0x7fa8c0160f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.159+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2344294917 0 0) 0x7fa8c00cdb20 con 0x7fa8c0160b40 2023-12-19T18:28:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.159+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8ac003520 con 0x7fa8c0160b40 2023-12-19T18:28:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.159+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2409918706 0 0) 0x7fa8ac003520 con 0x7fa8c0160b40 2023-12-19T18:28:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.159+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8c00cfdb0 msgr2=0x7fa8c0160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.159+0000 7fa8c4945640 1 --2- 172.21.15.27:0/338732309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8c00cfdb0 0x7fa8c0160f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.159+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 >> v2:172.21.15.192:3300/0 conn(0x7fa8c015d0a0 msgr2=0x7fa8c00cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.159+0000 7fa8c4945640 1 --2- 172.21.15.27:0/338732309 >> v2:172.21.15.192:3300/0 conn(0x7fa8c015d0a0 0x7fa8c00cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.160+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c015d8b0 con 0x7fa8c0160b40 2023-12-19T18:28:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.160+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fa8b0001670 con 0x7fa8c0160b40 2023-12-19T18:28:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.160+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 >> v1:172.21.15.27:6789/0 conn(0x7fa8c0160b40 legacy=0x7fa8c00cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.160+0000 7fa8c4945640 1 --2- 172.21.15.27:0/338732309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8ac0039a0 0x7fa8ac003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.160+0000 7fa8c4945640 1 --2- 172.21.15.27:0/338732309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8c00cfdb0 0x7fa8ac0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.160+0000 7fa8c4945640 1 --2- 172.21.15.27:0/338732309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8ac004d00 0x7fa8ac005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.160+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c015d480 con 0x7fa8ac004d00 2023-12-19T18:28:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.160+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac00b980 con 0x7fa8ac0039a0 2023-12-19T18:28:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.160+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8ac00bac0 con 0x7fa8c00cfdb0 2023-12-19T18:28:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.160+0000 7fa8c6148640 1 --2- 172.21.15.27:0/338732309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8ac004d00 0x7fa8ac005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.160+0000 7fa8c5947640 1 --2- 172.21.15.27:0/338732309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8ac0039a0 0x7fa8ac003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.160+0000 7fa8c5146640 1 --2- 172.21.15.27:0/338732309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8c00cfdb0 0x7fa8ac0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.161+0000 7fa8c6148640 1 -- 172.21.15.27:0/338732309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8c00cfdb0 msgr2=0x7fa8ac0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.161+0000 7fa8c6148640 1 --2- 172.21.15.27:0/338732309 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8c00cfdb0 0x7fa8ac0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.161+0000 7fa8c6148640 1 -- 172.21.15.27:0/338732309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8ac0039a0 msgr2=0x7fa8ac003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.161+0000 7fa8c6148640 1 --2- 172.21.15.27:0/338732309 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8ac0039a0 0x7fa8ac003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.161+0000 7fa8c6148640 1 -- 172.21.15.27:0/338732309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8ac003520 con 0x7fa8ac004d00 2023-12-19T18:28:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.162+0000 7fa8c6148640 1 --2- 172.21.15.27:0/338732309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8ac004d00 0x7fa8ac005180 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800e170 tx=0x7fa8ac0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d07680bce63ef8d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.162+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b802cbf0 con 0x7fa8ac004d00 2023-12-19T18:28:44.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.162+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8b800f040 con 0x7fa8ac004d00 2023-12-19T18:28:44.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.162+0000 7fa8c4945640 1 -- 172.21.15.27:0/338732309 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8b8034440 con 0x7fa8ac004d00 2023-12-19T18:28:44.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.163+0000 7fa8c6949640 1 -- 172.21.15.27:0/338732309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8ac004d00 msgr2=0x7fa8ac005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:44.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.163+0000 7fa8c6949640 1 --2- 172.21.15.27:0/338732309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8ac004d00 0x7fa8ac005180 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7fa8b800e170 tx=0x7fa8ac0057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:44.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.163+0000 7fa8c6148640 1 -- 172.21.15.27:0/338732309 reap_dead start 2023-12-19T18:28:44.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.163+0000 7fa8c6949640 1 -- 172.21.15.27:0/338732309 shutdown_connections 2023-12-19T18:28:44.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.163+0000 7fa8c6949640 1 -- 172.21.15.27:0/338732309 >> 172.21.15.27:0/338732309 conn(0x7fa8c006a1f0 msgr2=0x7fa8c006a600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:44.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.163+0000 7fa8c6949640 1 -- 172.21.15.27:0/338732309 shutdown_connections 2023-12-19T18:28:44.164 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.163+0000 7fa8c6949640 1 -- 172.21.15.27:0/338732309 wait complete. 2023-12-19T18:28:44.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.165+0000 7fa8c6949640 1 Processor -- start 2023-12-19T18:28:44.165 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.165+0000 7fa8c6949640 1 -- start start 2023-12-19T18:28:44.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.166+0000 7fa8c6949640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8ac004d00 0x7fa8c014ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:44.166 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.166+0000 7fa8c6949640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8c014b270 0x7fa8c014b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.166+0000 7fa8c6949640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8c00cb6e0 0x7fa8c00cbb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.166+0000 7fa8c6949640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0148ca0 con 0x7fa8ac004d00 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.166+0000 7fa8c6949640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00c6bc0 con 0x7fa8c014b270 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.166+0000 7fa8c6949640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00cc7d0 con 0x7fa8c00cb6e0 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.166+0000 7fa8c5947640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8ac004d00 0x7fa8c014ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.166+0000 7fa8c5947640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8ac004d00 0x7fa8c014ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39824/0 (socket says 172.21.15.27:39824) 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.166+0000 7fa8c5947640 1 -- 172.21.15.27:0/3558258790 learned_addr learned my addr 172.21.15.27:0/3558258790 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.166+0000 7fa8c6148640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8c00cb6e0 0x7fa8c00cbb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.166+0000 7fa8c5146640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8c014b270 0x7fa8c014b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.167+0000 7fa8c5947640 1 -- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8c00cb6e0 msgr2=0x7fa8c00cbb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.167+0000 7fa8c5947640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8c00cb6e0 0x7fa8c00cbb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.167+0000 7fa8c5947640 1 -- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8c014b270 msgr2=0x7fa8c014b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.167+0000 7fa8c5947640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8c014b270 0x7fa8c014b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:44.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.167+0000 7fa8c5947640 1 -- 172.21.15.27:0/3558258790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c0148ef0 con 0x7fa8ac004d00 2023-12-19T18:28:44.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.167+0000 7fa8c5146640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8c014b270 0x7fa8c014b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:44.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.167+0000 7fa8c5947640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8ac004d00 0x7fa8c014ad30 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc011aa0 tx=0x7fa8bc011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:44.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.167+0000 7fa8b67fc640 1 -- 172.21.15.27:0/3558258790 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc009680 con 0x7fa8ac004d00 2023-12-19T18:28:44.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.167+0000 7fa8b67fc640 1 -- 172.21.15.27:0/3558258790 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8bc00fb80 con 0x7fa8ac004d00 2023-12-19T18:28:44.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.167+0000 7fa8b67fc640 1 -- 172.21.15.27:0/3558258790 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc014440 con 0x7fa8ac004d00 2023-12-19T18:28:44.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.167+0000 7fa8c6949640 1 -- 172.21.15.27:0/3558258790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c01491d0 con 0x7fa8ac004d00 2023-12-19T18:28:44.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.167+0000 7fa8c6949640 1 -- 172.21.15.27:0/3558258790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c00cc0a0 con 0x7fa8ac004d00 2023-12-19T18:28:44.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.169+0000 7fa8c6949640 1 -- 172.21.15.27:0/3558258790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c004e840 con 0x7fa8ac004d00 2023-12-19T18:28:44.169 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.170+0000 7fa8b67fc640 1 -- 172.21.15.27:0/3558258790 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa8bc0145a0 con 0x7fa8ac004d00 2023-12-19T18:28:44.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.171+0000 7fa8b67fc640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa8a406a460 0x7fa8a406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:44.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.171+0000 7fa8b67fc640 1 -- 172.21.15.27:0/3558258790 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8bc089300 con 0x7fa8ac004d00 2023-12-19T18:28:44.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.171+0000 7fa8b67fc640 1 -- 172.21.15.27:0/3558258790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=205}) v3 -- 0x7fa8a4071f90 con 0x7fa8ac004d00 2023-12-19T18:28:44.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.171+0000 7fa8c5146640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa8a406a460 0x7fa8a406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:44.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.172+0000 7fa8c5146640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa8a406a460 0x7fa8a406c920 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00c6b50 tx=0x7fa8b0000f50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:44.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.176+0000 7fa8b67fc640 1 -- 172.21.15.27:0/3558258790 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa8bc054850 con 0x7fa8ac004d00 2023-12-19T18:28:44.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.373+0000 7fa8c6949640 1 -- 172.21.15.27:0/3558258790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7fa8c00d0b00 con 0x7fa8ac004d00 2023-12-19T18:28:44.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.377+0000 7fa8b67fc640 1 -- 172.21.15.27:0/3558258790 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v204) v1 ==== 81+0+33573 (secure 0 0 0) 0x7fa8bc022020 con 0x7fa8ac004d00 2023-12-19T18:28:44.377 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:44.377 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 204, 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:28:31.836861+0000", 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:44.378 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 23, 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "jewel", 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:28:44.379 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:28:44.380 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:28:44.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:44.380 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:44.380 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:44.380 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:44.380 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:44.380 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:44.380 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:44.380 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:44.380 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:44.380 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:44.380 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:44.380 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:28:44.381 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:44.382 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:44.383 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:28:44.384 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 22, 2023-12-19T18:28:44.385 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:06.542489+0000", 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:28:44.386 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "201", 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:44.387 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:44.388 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:44.389 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:28:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 23, 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:28:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:07.623298+0000", 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1027, 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool,full,full_quota", 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:28:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "204", 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:44.395 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.396 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.397 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.398 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.399 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:28:44.400 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:44.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.401 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:28:44.401 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:44.401 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.401 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.401 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:44.401 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.401 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.401 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.401 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:44.402 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:44.403 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.404 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.405 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.406 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.407 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:44.408 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.409 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.410 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.411 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:28:44.412 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:44.412 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.412 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.412 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:44.412 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.412 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.412 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:28:44.412 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:44.412 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.412 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.413 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.413 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:28:44.413 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:44.413 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.413 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.413 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.413 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:28:44.413 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:28:44.413 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:28:44.413 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:28:44.413 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:44.413 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:44.414 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.415 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.416 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.417 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:44.419 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:28:44.420 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:28:44.421 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:44.422 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:44.423 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.424 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:44.425 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:44.426 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:44.427 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.428 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:28:44.429 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:44.430 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:28:44.431 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:44.432 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:44.433 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.434 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:44.435 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.436 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.437 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.438 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:28:44.439 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:28:44.440 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:44.441 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:44.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:44.443 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:28:44.444 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:44.445 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [], 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:28:44.446 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:28:44.447 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:28:44.448 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:28:44.448 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:28:44.448 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:28:44.448 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:44.448 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:28:44.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.380+0000 7fa8c6949640 1 -- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa8a406a460 msgr2=0x7fa8a406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:44.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.380+0000 7fa8c6949640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa8a406a460 0x7fa8a406c920 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00c6b50 tx=0x7fa8b0000f50 comp rx=0 tx=0).stop 2023-12-19T18:28:44.449 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.380+0000 7fa8c6949640 1 -- 172.21.15.27:0/3558258790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8ac004d00 msgr2=0x7fa8c014ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:44.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.380+0000 7fa8c6949640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8ac004d00 0x7fa8c014ad30 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc011aa0 tx=0x7fa8bc011f60 comp rx=0 tx=0).stop 2023-12-19T18:28:44.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.380+0000 7fa8c6949640 1 -- 172.21.15.27:0/3558258790 shutdown_connections 2023-12-19T18:28:44.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.380+0000 7fa8c6949640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa8a406a460 0x7fa8a406c920 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:44.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.380+0000 7fa8c6949640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8c00cb6e0 0x7fa8c00cbb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:44.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.380+0000 7fa8c6949640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8c014b270 0x7fa8c014b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:44.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.380+0000 7fa8c6949640 1 --2- 172.21.15.27:0/3558258790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8ac004d00 0x7fa8c014ad30 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:44.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.380+0000 7fa8c6949640 1 -- 172.21.15.27:0/3558258790 >> 172.21.15.27:0/3558258790 conn(0x7fa8c006a1f0 msgr2=0x7fa8c00d1230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:44.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.380+0000 7fa8c6949640 1 -- 172.21.15.27:0/3558258790 shutdown_connections 2023-12-19T18:28:44.450 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:44.381+0000 7fa8c6949640 1 -- 172.21.15.27:0/3558258790 wait complete. 2023-12-19T18:28:44.451 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=600 retry_count=0)... 2023-12-19T18:28:48.659 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:28:48.669 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:28:49.451 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:28:49.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.696+0000 7ff859620640 1 Processor -- start 2023-12-19T18:28:49.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.697+0000 7ff859620640 1 -- start start 2023-12-19T18:28:49.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.698+0000 7ff859620640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff85415ac50 0x7ff854070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:49.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.698+0000 7ff859620640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8540af360 0x7ff85406ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:49.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.698+0000 7ff859620640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8540adf10 con 0x7ff8540aef20 2023-12-19T18:28:49.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.698+0000 7ff859620640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8540adac0 con 0x7ff85415ac50 2023-12-19T18:28:49.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.698+0000 7ff859620640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8540ae160 con 0x7ff8540af360 2023-12-19T18:28:49.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.698+0000 7ff853fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff85415ac50 0x7ff854070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:49.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.698+0000 7ff853fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff85415ac50 0x7ff854070fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39834/0 (socket says 172.21.15.27:39834) 2023-12-19T18:28:49.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.698+0000 7ff853fff640 1 -- 172.21.15.27:0/420263876 learned_addr learned my addr 172.21.15.27:0/420263876 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:49.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.699+0000 7ff853fff640 1 -- 172.21.15.27:0/420263876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8540af360 msgr2=0x7ff85406ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:49.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.699+0000 7ff853fff640 1 --2- 172.21.15.27:0/420263876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8540af360 0x7ff85406ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:49.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.699+0000 7ff853fff640 1 -- 172.21.15.27:0/420263876 >> v1:172.21.15.192:6789/0 conn(0x7ff8540aef20 legacy=0x7ff85406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:49.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.699+0000 7ff853fff640 1 -- 172.21.15.27:0/420263876 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8540ae440 con 0x7ff85415ac50 2023-12-19T18:28:49.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.700+0000 7ff853fff640 1 --2- 172.21.15.27:0/420263876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff85415ac50 0x7ff854070fb0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7ff844009870 tx=0x7ff84402b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f86d3b47b8cf03 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:49.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.700+0000 7ff8537fe640 1 -- 172.21.15.27:0/420263876 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844031740 con 0x7ff85415ac50 2023-12-19T18:28:49.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.700+0000 7ff8537fe640 1 -- 172.21.15.27:0/420263876 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8440318a0 con 0x7ff85415ac50 2023-12-19T18:28:49.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.700+0000 7ff8537fe640 1 -- 172.21.15.27:0/420263876 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff844031c30 con 0x7ff85415ac50 2023-12-19T18:28:49.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.701+0000 7ff859620640 1 -- 172.21.15.27:0/420263876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff85415ac50 msgr2=0x7ff854070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:49.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.701+0000 7ff859620640 1 --2- 172.21.15.27:0/420263876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff85415ac50 0x7ff854070fb0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7ff844009870 tx=0x7ff84402b7e0 comp rx=0 tx=0).stop 2023-12-19T18:28:49.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.701+0000 7ff859620640 1 -- 172.21.15.27:0/420263876 shutdown_connections 2023-12-19T18:28:49.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.701+0000 7ff859620640 1 --2- 172.21.15.27:0/420263876 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8540af360 0x7ff85406ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:49.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.701+0000 7ff859620640 1 --2- 172.21.15.27:0/420263876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff85415ac50 0x7ff854070fb0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:49.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.701+0000 7ff859620640 1 -- 172.21.15.27:0/420263876 >> 172.21.15.27:0/420263876 conn(0x7ff85406a7d0 msgr2=0x7ff8540ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:49.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.701+0000 7ff859620640 1 -- 172.21.15.27:0/420263876 shutdown_connections 2023-12-19T18:28:49.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.701+0000 7ff859620640 1 -- 172.21.15.27:0/420263876 wait complete. 2023-12-19T18:28:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.703+0000 7ff859620640 1 Processor -- start 2023-12-19T18:28:49.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.703+0000 7ff859620640 1 -- start start 2023-12-19T18:28:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.704+0000 7ff859620640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8540aef20 0x7ff85414ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.704+0000 7ff859620640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8540af360 0x7ff854156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.704+0000 7ff859620640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff85415ac50 0x7ff85414eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:49.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.704+0000 7ff859620640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff85415a4c0 con 0x7ff85415ac50 2023-12-19T18:28:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.704+0000 7ff859620640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff854157300 con 0x7ff8540aef20 2023-12-19T18:28:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.704+0000 7ff859620640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8541598b0 con 0x7ff8540af360 2023-12-19T18:28:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.704+0000 7ff858e1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff85415ac50 0x7ff85414eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.704+0000 7ff858e1f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff85415ac50 0x7ff85414eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39848/0 (socket says 172.21.15.27:39848) 2023-12-19T18:28:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.704+0000 7ff858e1f640 1 -- 172.21.15.27:0/3455396331 learned_addr learned my addr 172.21.15.27:0/3455396331 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.705+0000 7ff853fff640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8540aef20 0x7ff85414ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:49.705 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.705+0000 7ff84bfff640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8540af360 0x7ff854156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.705+0000 7ff858e1f640 1 -- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8540af360 msgr2=0x7ff854156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.705+0000 7ff858e1f640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8540af360 0x7ff854156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.705+0000 7ff858e1f640 1 -- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8540aef20 msgr2=0x7ff85414ab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.705+0000 7ff858e1f640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8540aef20 0x7ff85414ab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.705+0000 7ff858e1f640 1 -- 172.21.15.27:0/3455396331 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8440094d0 con 0x7ff85415ac50 2023-12-19T18:28:49.706 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.705+0000 7ff858e1f640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff85415ac50 0x7ff85414eb60 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00b970 tx=0x7ff84c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:49.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.706+0000 7ff8517fa640 1 -- 172.21.15.27:0/3455396331 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c00cb40 con 0x7ff85415ac50 2023-12-19T18:28:49.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.706+0000 7ff8517fa640 1 -- 172.21.15.27:0/3455396331 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff84c00ccc0 con 0x7ff85415ac50 2023-12-19T18:28:49.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.706+0000 7ff8517fa640 1 -- 172.21.15.27:0/3455396331 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84c014680 con 0x7ff85415ac50 2023-12-19T18:28:49.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.706+0000 7ff859620640 1 -- 172.21.15.27:0/3455396331 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff85414f0a0 con 0x7ff85415ac50 2023-12-19T18:28:49.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.706+0000 7ff859620640 1 -- 172.21.15.27:0/3455396331 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff854159e00 con 0x7ff85415ac50 2023-12-19T18:28:49.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.707+0000 7ff859620640 1 -- 172.21.15.27:0/3455396331 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff85414bd50 con 0x7ff85415ac50 2023-12-19T18:28:49.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.708+0000 7ff8517fa640 1 -- 172.21.15.27:0/3455396331 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff84c01d3b0 con 0x7ff85415ac50 2023-12-19T18:28:49.708 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.709+0000 7ff8517fa640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff83406a430 0x7ff83406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:49.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.709+0000 7ff8517fa640 1 -- 172.21.15.27:0/3455396331 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff84c088750 con 0x7ff85415ac50 2023-12-19T18:28:49.709 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.710+0000 7ff853fff640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff83406a430 0x7ff83406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:49.710 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.711+0000 7ff853fff640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff83406a430 0x7ff83406c8f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff844035000 tx=0x7ff844008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.714+0000 7ff8517fa640 1 -- 172.21.15.27:0/3455396331 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff85414bd50 con 0x7ff85415ac50 2023-12-19T18:28:49.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.910+0000 7ff859620640 1 -- 172.21.15.27:0/3455396331 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7ff85414bd50 con 0x7ff85415ac50 2023-12-19T18:28:49.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.912+0000 7ff8517fa640 1 -- 172.21.15.27:0/3455396331 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v206) v1 ==== 81+0+33746 (secure 0 0 0) 0x7ff85414bd50 con 0x7ff85415ac50 2023-12-19T18:28:49.912 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:49.912 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:28:49.912 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 206, 2023-12-19T18:28:49.912 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:28:49.912 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:28:49.913 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:28:46.843951+0000", 2023-12-19T18:28:49.913 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:28:49.913 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:28:49.913 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:28:49.913 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:28:49.913 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:28:49.913 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:28:49.913 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:28:49.913 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:28:49.913 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:28:49.913 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:49.913 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 23, 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "luminous", 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:28:49.914 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:28:49.915 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:49.916 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:49.917 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:49.918 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:28:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:28:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:28:49.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 22, 2023-12-19T18:28:49.920 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:28:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:06.542489+0000", 2023-12-19T18:28:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:49.921 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.922 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "201", 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:49.923 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:49.924 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.925 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:49.926 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 23, 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:07.623298+0000", 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:49.927 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:49.928 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:49.928 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:49.928 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:49.928 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:28:49.928 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:28:49.928 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:28:49.928 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:28:49.928 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:28:49.928 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:49.928 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:49.928 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:49.928 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:49.928 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:49.929 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:49.929 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:49.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.929 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "206", 2023-12-19T18:28:49.929 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:49.929 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:49.929 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:49.929 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:49.929 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:49.929 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:49.929 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:49.930 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:49.931 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:28:49.932 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:49.933 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.934 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.935 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:28:49.936 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.937 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:49.938 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:49.939 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:49.939 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:49.939 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:49.939 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:49.939 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:49.939 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:49.939 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:49.939 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:49.939 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:49.939 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.940 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.941 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.942 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:49.943 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:49.944 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.945 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.946 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.947 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:28:49.948 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:28:49.949 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:49.949 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:49.949 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:49.949 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.949 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.949 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:49.949 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:49.949 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 205, 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.950 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:28:49.951 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.952 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.953 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.954 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.955 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.956 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.957 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.958 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:49.959 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:49.960 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.962 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:49.963 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:28:49.964 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:49.965 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:28:49.966 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.967 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.968 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:28:49.969 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:49.970 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:28:49.971 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:49.972 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.973 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.974 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:49.975 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:28:49.976 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:49.977 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:49.978 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:49.979 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:49.980 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [ 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "23.26", 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 3 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:49.981 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:28:49.982 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:28:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:28:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:28:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:28:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:28:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:28:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:28:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:28:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:28:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:28:49.983 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:49.983 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:28:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.914+0000 7ff859620640 1 -- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff83406a430 msgr2=0x7ff83406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.914+0000 7ff859620640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff83406a430 0x7ff83406c8f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7ff844035000 tx=0x7ff844008030 comp rx=0 tx=0).stop 2023-12-19T18:28:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.914+0000 7ff859620640 1 -- 172.21.15.27:0/3455396331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff85415ac50 msgr2=0x7ff85414eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.914+0000 7ff859620640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff85415ac50 0x7ff85414eb60 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7ff84c00b970 tx=0x7ff84c00be30 comp rx=0 tx=0).stop 2023-12-19T18:28:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.915+0000 7ff859620640 1 -- 172.21.15.27:0/3455396331 shutdown_connections 2023-12-19T18:28:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.915+0000 7ff859620640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff83406a430 0x7ff83406c8f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.915+0000 7ff859620640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff85415ac50 0x7ff85414eb60 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.915+0000 7ff859620640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff8540af360 0x7ff854156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.915+0000 7ff859620640 1 --2- 172.21.15.27:0/3455396331 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff8540aef20 0x7ff85414ab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.915+0000 7ff859620640 1 -- 172.21.15.27:0/3455396331 >> 172.21.15.27:0/3455396331 conn(0x7ff85406a7d0 msgr2=0x7ff8540cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:49.985 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.915+0000 7ff859620640 1 -- 172.21.15.27:0/3455396331 shutdown_connections 2023-12-19T18:28:49.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:49.915+0000 7ff859620640 1 -- 172.21.15.27:0/3455396331 wait complete. 2023-12-19T18:28:49.986 DEBUG:tasks.ceph_test_case:wait_until_true: success in 10s and 0 retries 2023-12-19T18:28:49.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json-pretty 2023-12-19T18:28:50.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.226+0000 7f40b2311640 1 Processor -- start 2023-12-19T18:28:50.226 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.226+0000 7f40b2311640 1 -- start start 2023-12-19T18:28:50.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.228+0000 7f40b2311640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac06f8e0 0x7f40ac06fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.228+0000 7f40b2311640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f40ac161a40 0x7f40ac161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.228+0000 7f40b2311640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40ac06b310 con 0x7f40ac15b9e0 2023-12-19T18:28:50.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.228+0000 7f40b2311640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40ac0702a0 con 0x7f40ac161a40 2023-12-19T18:28:50.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.228+0000 7f40b2311640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac162790 con 0x7f40ac06f8e0 2023-12-19T18:28:50.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.229+0000 7f40b130f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f40ac15b9e0 0x7f40ac0c9410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:44238/0 (socket says 172.21.15.27:44238) 2023-12-19T18:28:50.229 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.229+0000 7f40b130f640 1 -- 172.21.15.27:0/3890798247 learned_addr learned my addr 172.21.15.27:0/3890798247 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:50.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.229+0000 7f40b0b0e640 1 --2- 172.21.15.27:0/3890798247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac06f8e0 0x7f40ac06fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.229+0000 7f40b1b10640 1 --2- 172.21.15.27:0/3890798247 >> v2:172.21.15.192:3300/0 conn(0x7f40ac161a40 0x7f40ac161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.229+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3031796854 0 0) 0x7f40ac06b310 con 0x7f40ac15b9e0 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.229+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4098003520 con 0x7f40ac15b9e0 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4249212779 0 0) 0x7f4098003520 con 0x7f40ac15b9e0 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac06f8e0 msgr2=0x7f40ac06fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 --2- 172.21.15.27:0/3890798247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac06f8e0 0x7f40ac06fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 >> v2:172.21.15.192:3300/0 conn(0x7f40ac161a40 msgr2=0x7f40ac161ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 --2- 172.21.15.27:0/3890798247 >> v2:172.21.15.192:3300/0 conn(0x7f40ac161a40 0x7f40ac161ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac06b560 con 0x7f40ac15b9e0 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f40a80019f0 con 0x7f40ac15b9e0 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 >> v1:172.21.15.27:6789/0 conn(0x7f40ac15b9e0 legacy=0x7f40ac0c9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 --2- 172.21.15.27:0/3890798247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4098003970 0x7f4098003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 --2- 172.21.15.27:0/3890798247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac06f8e0 0x7f40980047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 --2- 172.21.15.27:0/3890798247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4098004d00 0x7f4098005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac0702a0 con 0x7f4098003970 2023-12-19T18:28:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac162790 con 0x7f4098004d00 2023-12-19T18:28:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409800b980 con 0x7f40ac06f8e0 2023-12-19T18:28:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.230+0000 7f40b130f640 1 --2- 172.21.15.27:0/3890798247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4098003970 0x7f4098003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.231+0000 7f40b1b10640 1 --2- 172.21.15.27:0/3890798247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4098004d00 0x7f4098005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.231+0000 7f40b0b0e640 1 --2- 172.21.15.27:0/3890798247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac06f8e0 0x7f40980047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.231+0000 7f40b130f640 1 -- 172.21.15.27:0/3890798247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac06f8e0 msgr2=0x7f40980047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.231+0000 7f40b130f640 1 --2- 172.21.15.27:0/3890798247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac06f8e0 0x7f40980047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.231+0000 7f40b130f640 1 -- 172.21.15.27:0/3890798247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4098004d00 msgr2=0x7f4098005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.231+0000 7f40b130f640 1 --2- 172.21.15.27:0/3890798247 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4098004d00 0x7f4098005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.231+0000 7f40b130f640 1 -- 172.21.15.27:0/3890798247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4098003520 con 0x7f4098003970 2023-12-19T18:28:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.231+0000 7f40b0b0e640 1 --2- 172.21.15.27:0/3890798247 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac06f8e0 0x7f40980047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.232+0000 7f40b130f640 1 --2- 172.21.15.27:0/3890798247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4098003970 0x7f4098003d90 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f40a800a790 tx=0x7f40980057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=147396c88f928616 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.232+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8034630 con 0x7f4098003970 2023-12-19T18:28:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.232+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a802f070 con 0x7f4098003970 2023-12-19T18:28:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.232+0000 7f40a3fff640 1 -- 172.21.15.27:0/3890798247 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a80349f0 con 0x7f4098003970 2023-12-19T18:28:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.233+0000 7f40b2311640 1 -- 172.21.15.27:0/3890798247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4098003970 msgr2=0x7f4098003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.233+0000 7f40b2311640 1 --2- 172.21.15.27:0/3890798247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4098003970 0x7f4098003d90 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f40a800a790 tx=0x7f40980057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.233+0000 7f40b1b10640 1 -- 172.21.15.27:0/3890798247 reap_dead start 2023-12-19T18:28:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.233+0000 7f40b2311640 1 -- 172.21.15.27:0/3890798247 shutdown_connections 2023-12-19T18:28:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.233+0000 7f40b2311640 1 -- 172.21.15.27:0/3890798247 >> 172.21.15.27:0/3890798247 conn(0x7f40ac06a7d0 msgr2=0x7f40ac0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.233+0000 7f40b2311640 1 -- 172.21.15.27:0/3890798247 shutdown_connections 2023-12-19T18:28:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.233+0000 7f40b2311640 1 -- 172.21.15.27:0/3890798247 wait complete. 2023-12-19T18:28:50.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.235+0000 7f40b2311640 1 Processor -- start 2023-12-19T18:28:50.235 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.235+0000 7f40b2311640 1 -- start start 2023-12-19T18:28:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.236+0000 7f40b2311640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4098003970 0x7f40ac158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.236+0000 7f40b2311640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40ac0c6870 0x7f40ac0a8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.236+0000 7f40b2311640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac0a8e90 0x7f40ac0aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.236+0000 7f40b2311640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac0a9310 con 0x7f4098003970 2023-12-19T18:28:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.236+0000 7f40b2311640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac158d80 con 0x7f40ac0c6870 2023-12-19T18:28:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.236+0000 7f40b2311640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac0c6e10 con 0x7f40ac0a8e90 2023-12-19T18:28:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.236+0000 7f40b130f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4098003970 0x7f40ac158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.236+0000 7f40b130f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4098003970 0x7f40ac158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39874/0 (socket says 172.21.15.27:39874) 2023-12-19T18:28:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.236+0000 7f40b130f640 1 -- 172.21.15.27:0/1093272833 learned_addr learned my addr 172.21.15.27:0/1093272833 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.237+0000 7f40b0b0e640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40ac0c6870 0x7f40ac0a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.237+0000 7f40b1b10640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac0a8e90 0x7f40ac0aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.237+0000 7f40b130f640 1 -- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac0a8e90 msgr2=0x7f40ac0aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:50.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.237+0000 7f40b130f640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac0a8e90 0x7f40ac0aae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.237+0000 7f40b130f640 1 -- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40ac0c6870 msgr2=0x7f40ac0a8950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:50.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.237+0000 7f40b130f640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40ac0c6870 0x7f40ac0a8950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.237+0000 7f40b130f640 1 -- 172.21.15.27:0/1093272833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40ac0c5210 con 0x7f4098003970 2023-12-19T18:28:50.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.237+0000 7f40b1b10640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac0a8e90 0x7f40ac0aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:50.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.237+0000 7f40b130f640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4098003970 0x7f40ac158840 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f40a802cad0 tx=0x7f40a8001cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:50.238 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.237+0000 7f40a1ffb640 1 -- 172.21.15.27:0/1093272833 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8034630 con 0x7f4098003970 2023-12-19T18:28:50.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.237+0000 7f40a1ffb640 1 -- 172.21.15.27:0/1093272833 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a8034790 con 0x7f4098003970 2023-12-19T18:28:50.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.237+0000 7f40a1ffb640 1 -- 172.21.15.27:0/1093272833 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8034b40 con 0x7f4098003970 2023-12-19T18:28:50.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.238+0000 7f40b2311640 1 -- 172.21.15.27:0/1093272833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac0c54f0 con 0x7f4098003970 2023-12-19T18:28:50.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.238+0000 7f40b2311640 1 -- 172.21.15.27:0/1093272833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ac0c5ab0 con 0x7f4098003970 2023-12-19T18:28:50.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.239+0000 7f40b2311640 1 -- 172.21.15.27:0/1093272833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40ac04e840 con 0x7f4098003970 2023-12-19T18:28:50.239 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.240+0000 7f40a1ffb640 1 -- 172.21.15.27:0/1093272833 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f40a8031030 con 0x7f4098003970 2023-12-19T18:28:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.241+0000 7f40a1ffb640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f409006a460 0x7f409006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.241+0000 7f40a1ffb640 1 -- 172.21.15.27:0/1093272833 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f40a80aaec0 con 0x7f4098003970 2023-12-19T18:28:50.241 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.242+0000 7f40b0b0e640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f409006a460 0x7f409006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.242 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.243+0000 7f40b0b0e640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f409006a460 0x7f409006c920 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f40ac06b2c0 tx=0x7f409c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:50.244 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.245+0000 7f40a1ffb640 1 -- 172.21.15.27:0/1093272833 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f40a80765b0 con 0x7f4098003970 2023-12-19T18:28:50.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.440+0000 7f40b2311640 1 -- 172.21.15.27:0/1093272833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json-pretty"} v 0) v1 -- 0x7f40ac04e840 con 0x7f4098003970 2023-12-19T18:28:50.444 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.444+0000 7f40a1ffb640 1 -- 172.21.15.27:0/1093272833 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json-pretty"}]=0 v206) v1 ==== 81+0+33746 (secure 0 0 0) 0x7f40a8079bd0 con 0x7f4098003970 2023-12-19T18:28:50.445 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:50.445 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:28:50.445 INFO:teuthology.orchestra.run.smithi027.stdout: "epoch": 206, 2023-12-19T18:28:50.445 INFO:teuthology.orchestra.run.smithi027.stdout: "fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", 2023-12-19T18:28:50.445 INFO:teuthology.orchestra.run.smithi027.stdout: "created": "2023-12-19T18:14:36.525286+0000", 2023-12-19T18:28:50.445 INFO:teuthology.orchestra.run.smithi027.stdout: "modified": "2023-12-19T18:28:46.843951+0000", 2023-12-19T18:28:50.445 INFO:teuthology.orchestra.run.smithi027.stdout: "last_up_change": "2023-12-19T18:14:43.040687+0000", 2023-12-19T18:28:50.445 INFO:teuthology.orchestra.run.smithi027.stdout: "last_in_change": "2023-12-19T18:14:40.022208+0000", 2023-12-19T18:28:50.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2023-12-19T18:28:50.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_num": 5799936, 2023-12-19T18:28:50.445 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_set": [ 2023-12-19T18:28:50.445 INFO:teuthology.orchestra.run.smithi027.stdout: "pglog_hardlimit", 2023-12-19T18:28:50.445 INFO:teuthology.orchestra.run.smithi027.stdout: "purged_snapdirs", 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_deletes", 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "sortbitwise" 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_version": 4, 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "full_ratio": 0.69999998807907104, 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "backfillfull_ratio": 0.60000002384185791, 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "nearfull_ratio": 0.60000002384185791, 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_snapshot": "", 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_max": 23, 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "max_osd": 8, 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "require_min_compat_client": "luminous", 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "min_compat_client": "luminous", 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "require_osd_release": "reef", 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "allow_crimson": false, 2023-12-19T18:28:50.446 INFO:teuthology.orchestra.run.smithi027.stdout: "pools": [ 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 1, 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": ".mgr", 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:14:43.789712+0000", 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:50.447 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 1, 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 1, 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 1, 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 1, 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 1, 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "123", 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:50.448 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:50.449 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:50.450 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_max": 32, 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 1 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": {} 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.451 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 8, 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 8, 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 0.25, 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 2, 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 2, 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 22, 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_metadata", 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:06.542489+0000", 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:50.452 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 32, 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 32, 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 32, 2023-12-19T18:28:50.453 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 32, 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 32, 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "201", 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:50.454 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 0, 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:50.455 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:50.456 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_bias": 4, 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 16, 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: "recovery_priority": 5 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: "metadata": "cephfs" 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.5, 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.5, 2023-12-19T18:28:50.457 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.5, 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.5, 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: "pool": 23, 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_name": "cephfs_data", 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: "create_time": "2023-12-19T18:28:07.623298+0000", 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: "flags": 1, 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: "flags_names": "hashpspool", 2023-12-19T18:28:50.458 INFO:teuthology.orchestra.run.smithi027.stdout: "type": 1, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "size": 2, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "min_size": 1, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_rule": 0, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_count": 0, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_target": 0, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_barrier": 0, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "object_hash": 2, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_autoscale_mode": "off", 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num": 64, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num": 64, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_placement_num_target": 64, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_target": 64, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_pending": 64, 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "last_pg_merge_meta": { 2023-12-19T18:28:50.459 INFO:teuthology.orchestra.run.smithi027.stdout: "source_pgid": "0.0", 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "ready_epoch": 0, 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_started": 0, 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "last_epoch_clean": 0, 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "source_version": "0'0", 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "target_version": "0'0" 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "last_change": "206", 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend": "0", 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_prenautilus": "0", 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "last_force_op_resend_preluminous": "0", 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "auid": 0, 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_mode": "selfmanaged", 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_seq": 0, 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "snap_epoch": 0, 2023-12-19T18:28:50.460 INFO:teuthology.orchestra.run.smithi027.stdout: "pool_snaps": [], 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps": "[]", 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_bytes": 33554432, 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "quota_max_objects": 0, 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "tiers": [], 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "tier_of": -1, 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "read_tier": -1, 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "write_tier": -1, 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_mode": "none", 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_bytes": 0, 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "target_max_objects": 0, 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_ratio_micro": 400000, 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_target_full_ratio_micro": 800000, 2023-12-19T18:28:50.461 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_flush_age": 0, 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: "cache_min_evict_age": 0, 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profile": "", 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_params": { 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "none" 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_period": 0, 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_count": 0, 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: "use_gmt_hitset": true, 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: "min_read_recency_for_promote": 0, 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: "min_write_recency_for_promote": 0, 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_grade_decay_rate": 0, 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: "hit_set_search_last_n": 0, 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: "grade_table": [], 2023-12-19T18:28:50.462 INFO:teuthology.orchestra.run.smithi027.stdout: "stripe_width": 0, 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: "expected_num_objects": 0, 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: "fast_read": false, 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: "options": { 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_num_min": 64, 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: "target_size_ratio": 0.89999997615814209 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: "application_metadata": { 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: "cephfs": { 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: "data": "cephfs" 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: "read_balance": { 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: "score_acting": 1.3799999952316284, 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: "score_stable": 1.3799999952316284, 2023-12-19T18:28:50.463 INFO:teuthology.orchestra.run.smithi027.stdout: "optimal_score": 1, 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_acting": 1.3799999952316284, 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: "raw_score_stable": 1.3799999952316284, 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity_weighted": 1, 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity": 1, 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: "average_primary_affinity_weighted": 1 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: "osds": [ 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:50.464 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6808", 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.465 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6809", 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6810", 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.466 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6811", 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6814", 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.467 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6815", 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6812", 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6813", 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84136 2023-12-19T18:28:50.468 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6809/84136", 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6811/84136", 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6815/84136", 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6813/84136", 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:50.469 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.470 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6800", 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6801", 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6802", 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:50.471 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6803", 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6806", 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.472 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6807", 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6804", 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.473 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6805", 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84133 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6801/84133", 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6803/84133", 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6807/84133", 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6805/84133", 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.474 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.475 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6820", 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6822", 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6824", 2023-12-19T18:28:50.476 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6826", 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6830", 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:50.477 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6831", 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6828", 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.478 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6829", 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84138 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6822/84138", 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6826/84138", 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6831/84138", 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6829/84138", 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:50.479 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 205, 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:50.480 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6816", 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6817", 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.481 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6818", 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6819", 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.482 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6825", 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6827", 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6821", 2023-12-19T18:28:50.483 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.27:6823", 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84146 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.27:6817/84146", 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.27:6819/84146", 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.27:6827/84146", 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.27:6823/84146", 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:50.484 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:50.485 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6800", 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6801", 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:50.486 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6802", 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6803", 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.487 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6806", 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6807", 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.488 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.489 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6804", 2023-12-19T18:28:50.489 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:50.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.489 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.489 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.489 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6805", 2023-12-19T18:28:50.489 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84205 2023-12-19T18:28:50.489 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.489 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.489 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.489 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6801/84205", 2023-12-19T18:28:50.489 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6803/84205", 2023-12-19T18:28:50.489 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6807/84205", 2023-12-19T18:28:50.490 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6805/84205", 2023-12-19T18:28:50.490 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:50.490 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:50.490 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:50.490 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.490 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.490 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.490 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:50.490 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:50.490 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:50.490 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:50.490 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:50.490 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6808", 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.491 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6809", 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6810", 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.492 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6811", 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6814", 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.493 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6815", 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6812", 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6813", 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84206 2023-12-19T18:28:50.494 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6809/84206", 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6811/84206", 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6815/84206", 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6813/84206", 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:50.495 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6824", 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:50.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6825", 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6826", 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:50.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6827", 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6830", 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.498 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6831", 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6828", 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.499 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6829", 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84255 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6825/84255", 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6827/84255", 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6831/84255", 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6829/84255", 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:50.500 INFO:teuthology.orchestra.run.smithi027.stdout: "uuid": "00000000-0000-0000-0000-000000000000", 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "up": 1, 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "in": 1, 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "weight": 1, 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_affinity": 1, 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_begin": 0, 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "last_clean_end": 0, 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "up_from": 13, 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "up_thru": 200, 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "down_at": 0, 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "lost_at": 0, 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addrs": { 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.501 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6816", 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6817", 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addrs": { 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6818", 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:50.502 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6819", 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addrs": { 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6822", 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.503 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6823", 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addrs": { 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: "addrvec": [ 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v2", 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6820", 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: "type": "v1", 2023-12-19T18:28:50.504 INFO:teuthology.orchestra.run.smithi027.stdout: "addr": "172.21.15.192:6821", 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: "nonce": 84207 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: "public_addr": "172.21.15.192:6817/84207", 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: "cluster_addr": "172.21.15.192:6819/84207", 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_back_addr": "172.21.15.192:6823/84207", 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: "heartbeat_front_addr": "172.21.15.192:6821/84207", 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: "state": [ 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: "exists", 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: "up" 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.505 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: "osd_xinfo": [ 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 0, 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.347271+0000", 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 1, 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:50.506 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.319477+0000", 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 2, 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:50.507 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.378368+0000", 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 3, 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.279853+0000", 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 4, 2023-12-19T18:28:50.508 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.250859+0000", 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 5, 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:50.509 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.292556+0000", 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 6, 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.348967+0000", 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": 7, 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: "down_stamp": "0.000000", 2023-12-19T18:28:50.510 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_probability": 0, 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: "laggy_interval": 0, 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: "features": 4540138322906710015, 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: "old_weight": 0, 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: "last_purged_snaps_scrub": "2023-12-19T18:14:41.339002+0000", 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: "dead_epoch": 0 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap": [], 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_items": [ 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: "pgid": "23.26", 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: "mappings": [ 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: { 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: "from": 5, 2023-12-19T18:28:50.511 INFO:teuthology.orchestra.run.smithi027.stdout: "to": 3 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: ] 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: ], 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_upmap_primaries": [], 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: "pg_temp": [], 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: "primary_temp": [], 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: "blocklist": {}, 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: "range_blocklist": {}, 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: "erasure_code_profiles": { 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: "default": { 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: "crush-failure-domain": "osd", 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: "k": "2", 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: "m": "1", 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: "plugin": "jerasure", 2023-12-19T18:28:50.512 INFO:teuthology.orchestra.run.smithi027.stdout: "technique": "reed_sol_van" 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: "removed_snaps_queue": [], 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: "new_removed_snaps": [], 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: "new_purged_snaps": [], 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: "crush_node_flags": {}, 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: "device_class_flags": {}, 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode": { 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_enabled": false, 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_bucket_count": 0, 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: "degraded_stretch_mode": 0, 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: "recovering_stretch_mode": 0, 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: "stretch_mode_bucket": 0 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout: } 2023-12-19T18:28:50.513 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:28:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.447+0000 7f40b2311640 1 -- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f409006a460 msgr2=0x7f409006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.447+0000 7f40b2311640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f409006a460 0x7f409006c920 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f40ac06b2c0 tx=0x7f409c008040 comp rx=0 tx=0).stop 2023-12-19T18:28:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.447+0000 7f40b2311640 1 -- 172.21.15.27:0/1093272833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4098003970 msgr2=0x7f40ac158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.447+0000 7f40b2311640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4098003970 0x7f40ac158840 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f40a802cad0 tx=0x7f40a8001cb0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.447+0000 7f40b2311640 1 -- 172.21.15.27:0/1093272833 shutdown_connections 2023-12-19T18:28:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.447+0000 7f40b2311640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f409006a460 0x7f409006c920 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.447+0000 7f40b2311640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f40ac0a8e90 0x7f40ac0aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.447+0000 7f40b2311640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f40ac0c6870 0x7f40ac0a8950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.447+0000 7f40b2311640 1 --2- 172.21.15.27:0/1093272833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4098003970 0x7f40ac158840 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.447+0000 7f40b2311640 1 -- 172.21.15.27:0/1093272833 >> 172.21.15.27:0/1093272833 conn(0x7f40ac06a7d0 msgr2=0x7f40ac0caa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.447+0000 7f40b2311640 1 -- 172.21.15.27:0/1093272833 shutdown_connections 2023-12-19T18:28:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.447+0000 7f40b2311640 1 -- 172.21.15.27:0/1093272833 wait complete. 2023-12-19T18:28:50.517 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:28:50.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.760+0000 7fe48d8dc640 1 Processor -- start 2023-12-19T18:28:50.760 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.760+0000 7fe48d8dc640 1 -- start start 2023-12-19T18:28:50.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.762+0000 7fe48d8dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe48815c7f0 0x7fe4880c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.762+0000 7fe48d8dc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe4880cab50 0x7fe4880c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.762+0000 7fe48d8dc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4880cc510 0x7fe4880cc980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.762+0000 7fe48d8dc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4880ccec0 con 0x7fe4880cc510 2023-12-19T18:28:50.762 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.762+0000 7fe48d8dc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4880ca310 con 0x7fe4880cab50 2023-12-19T18:28:50.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.762+0000 7fe48d8dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe48815cbd0 con 0x7fe48815c7f0 2023-12-19T18:28:50.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.762+0000 7fe48d0db640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4880cc510 0x7fe4880cc980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.763+0000 7fe48d0db640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe4880cc510 0x7fe4880cc980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39882/0 (socket says 172.21.15.27:39882) 2023-12-19T18:28:50.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.763+0000 7fe48d0db640 1 -- 172.21.15.27:0/3269085884 learned_addr learned my addr 172.21.15.27:0/3269085884 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:50.763 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.763+0000 7fe48c8da640 1 --2- 172.21.15.27:0/3269085884 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe48815c7f0 0x7fe4880c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.763+0000 7fe487fff640 1 --2- 172.21.15.27:0/3269085884 >> v2:172.21.15.192:3300/0 conn(0x7fe4880cab50 0x7fe4880c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.763+0000 7fe48d0db640 1 -- 172.21.15.27:0/3269085884 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe48815c7f0 msgr2=0x7fe4880c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:50.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.763+0000 7fe48d0db640 1 --2- 172.21.15.27:0/3269085884 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe48815c7f0 0x7fe4880c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.763+0000 7fe48d0db640 1 -- 172.21.15.27:0/3269085884 >> v2:172.21.15.192:3300/0 conn(0x7fe4880cab50 msgr2=0x7fe4880c9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:50.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.763+0000 7fe48d0db640 1 --2- 172.21.15.27:0/3269085884 >> v2:172.21.15.192:3300/0 conn(0x7fe4880cab50 0x7fe4880c9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.763+0000 7fe48d0db640 1 -- 172.21.15.27:0/3269085884 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4880ae490 con 0x7fe4880cc510 2023-12-19T18:28:50.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.763+0000 7fe487fff640 1 --2- 172.21.15.27:0/3269085884 >> v2:172.21.15.192:3300/0 conn(0x7fe4880cab50 0x7fe4880c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:50.764 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.765+0000 7fe48d0db640 1 --2- 172.21.15.27:0/3269085884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4880cc510 0x7fe4880cc980 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fe48806d630 tx=0x7fe48002bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d99746e1522535 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:50.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.765+0000 7fe4877fe640 1 -- 172.21.15.27:0/3269085884 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48002c960 con 0x7fe4880cc510 2023-12-19T18:28:50.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.765+0000 7fe4877fe640 1 -- 172.21.15.27:0/3269085884 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48002cac0 con 0x7fe4880cc510 2023-12-19T18:28:50.765 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.765+0000 7fe4877fe640 1 -- 172.21.15.27:0/3269085884 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48002ce50 con 0x7fe4880cc510 2023-12-19T18:28:50.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.766+0000 7fe48d8dc640 1 -- 172.21.15.27:0/3269085884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4880cc510 msgr2=0x7fe4880cc980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:50.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.766+0000 7fe48d8dc640 1 --2- 172.21.15.27:0/3269085884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4880cc510 0x7fe4880cc980 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fe48806d630 tx=0x7fe48002bdc0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.766+0000 7fe48d8dc640 1 -- 172.21.15.27:0/3269085884 shutdown_connections 2023-12-19T18:28:50.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.766+0000 7fe48d8dc640 1 --2- 172.21.15.27:0/3269085884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4880cc510 0x7fe4880cc980 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.766+0000 7fe48d8dc640 1 --2- 172.21.15.27:0/3269085884 >> v2:172.21.15.192:3300/0 conn(0x7fe4880cab50 0x7fe4880c9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.766 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.766+0000 7fe48d8dc640 1 --2- 172.21.15.27:0/3269085884 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe48815c7f0 0x7fe4880c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.766+0000 7fe48d8dc640 1 -- 172.21.15.27:0/3269085884 >> 172.21.15.27:0/3269085884 conn(0x7fe488069c70 msgr2=0x7fe48806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:50.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.766+0000 7fe48d8dc640 1 -- 172.21.15.27:0/3269085884 shutdown_connections 2023-12-19T18:28:50.767 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.766+0000 7fe48d8dc640 1 -- 172.21.15.27:0/3269085884 wait complete. 2023-12-19T18:28:50.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.768+0000 7fe48d8dc640 1 Processor -- start 2023-12-19T18:28:50.768 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.768+0000 7fe48d8dc640 1 -- start start 2023-12-19T18:28:50.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.769+0000 7fe48d8dc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4880cab50 0x7fe48814d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.769+0000 7fe48d8dc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4880cc510 0x7fe48815a110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.769+0000 7fe48d8dc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe48815c7f0 0x7fe4880708f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.769+0000 7fe48d8dc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe48815a650 con 0x7fe48815c7f0 2023-12-19T18:28:50.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.769+0000 7fe48d8dc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe488151310 con 0x7fe4880cab50 2023-12-19T18:28:50.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.769+0000 7fe48d8dc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4881515d0 con 0x7fe4880cc510 2023-12-19T18:28:50.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.769+0000 7fe48d0db640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe48815c7f0 0x7fe4880708f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.769+0000 7fe48d0db640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe48815c7f0 0x7fe4880708f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39888/0 (socket says 172.21.15.27:39888) 2023-12-19T18:28:50.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.769+0000 7fe48d0db640 1 -- 172.21.15.27:0/706468837 learned_addr learned my addr 172.21.15.27:0/706468837 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:50.770 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.770+0000 7fe48c8da640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4880cab50 0x7fe48814d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.770+0000 7fe487fff640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4880cc510 0x7fe48815a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.770+0000 7fe48d0db640 1 -- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4880cc510 msgr2=0x7fe48815a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:50.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.770+0000 7fe48d0db640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4880cc510 0x7fe48815a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.770+0000 7fe48d0db640 1 -- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4880cab50 msgr2=0x7fe48814d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:50.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.770+0000 7fe48d0db640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4880cab50 0x7fe48814d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:50.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.770+0000 7fe48d0db640 1 -- 172.21.15.27:0/706468837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe480009b30 con 0x7fe48815c7f0 2023-12-19T18:28:50.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.770+0000 7fe487fff640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4880cc510 0x7fe48815a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:50.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.770+0000 7fe48d0db640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe48815c7f0 0x7fe4880708f0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fe480036040 tx=0x7fe480004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:50.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.770+0000 7fe4857fa640 1 -- 172.21.15.27:0/706468837 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480009510 con 0x7fe48815c7f0 2023-12-19T18:28:50.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.771+0000 7fe4857fa640 1 -- 172.21.15.27:0/706468837 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe48003e450 con 0x7fe48815c7f0 2023-12-19T18:28:50.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.771+0000 7fe48d8dc640 1 -- 172.21.15.27:0/706468837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4881518b0 con 0x7fe48815c7f0 2023-12-19T18:28:50.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.771+0000 7fe4857fa640 1 -- 172.21.15.27:0/706468837 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe48003e800 con 0x7fe48815c7f0 2023-12-19T18:28:50.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.771+0000 7fe48d8dc640 1 -- 172.21.15.27:0/706468837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe488151e70 con 0x7fe48815c7f0 2023-12-19T18:28:50.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.772+0000 7fe48d8dc640 1 -- 172.21.15.27:0/706468837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe48806a990 con 0x7fe48815c7f0 2023-12-19T18:28:50.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.773+0000 7fe4857fa640 1 -- 172.21.15.27:0/706468837 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe48003e960 con 0x7fe48815c7f0 2023-12-19T18:28:50.773 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.774+0000 7fe4857fa640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe46c06a460 0x7fe46c06c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:50.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.774+0000 7fe4857fa640 1 -- 172.21.15.27:0/706468837 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe4800ab6b0 con 0x7fe48815c7f0 2023-12-19T18:28:50.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.774+0000 7fe48c8da640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe46c06a460 0x7fe46c06c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:50.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.775+0000 7fe48c8da640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe46c06a460 0x7fe46c06c920 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe47c006fd0 tx=0x7fe47c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:50.777 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:50.778+0000 7fe4857fa640 1 -- 172.21.15.27:0/706468837 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe480076c80 con 0x7fe48815c7f0 2023-12-19T18:28:51.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.008+0000 7fe48d8dc640 1 -- 172.21.15.27:0/706468837 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe488152060 con 0x7fe48815c7f0 2023-12-19T18:28:51.009 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.009+0000 7fe4857fa640 1 -- 172.21.15.27:0/706468837 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4713 (secure 0 0 0) 0x7fe48007a2a0 con 0x7fe48815c7f0 2023-12-19T18:28:51.009 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:51.009 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1864159643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1864159643},{"type":"v1","addr":"172.21.15.192:6836","nonce":1864159643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1348219767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1348219767},{"type":"v1","addr":"172.21.15.192:6837","nonce":1348219767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/181137639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":181137639},{"type":"v1","addr":"172.21.15.27:6833","nonce":181137639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:28:08.577742+0000","modified":"2023-12-19T18:28:12.083302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/152568921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":152568921},{"type":"v1","addr":"172.21.15.27:6835","nonce":152568921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:28:51.009 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 85 2023-12-19T18:28:51.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.011+0000 7fe48d8dc640 1 -- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe46c06a460 msgr2=0x7fe46c06c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.011+0000 7fe48d8dc640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe46c06a460 0x7fe46c06c920 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe47c006fd0 tx=0x7fe47c008040 comp rx=0 tx=0).stop 2023-12-19T18:28:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.011+0000 7fe48d8dc640 1 -- 172.21.15.27:0/706468837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe48815c7f0 msgr2=0x7fe4880708f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.011+0000 7fe48d8dc640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe48815c7f0 0x7fe4880708f0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7fe480036040 tx=0x7fe480004a40 comp rx=0 tx=0).stop 2023-12-19T18:28:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.011+0000 7fe48d8dc640 1 -- 172.21.15.27:0/706468837 shutdown_connections 2023-12-19T18:28:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.011+0000 7fe48d8dc640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe46c06a460 0x7fe46c06c920 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.011+0000 7fe48d8dc640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe48815c7f0 0x7fe4880708f0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.011+0000 7fe48d8dc640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4880cc510 0x7fe48815a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.011+0000 7fe48d8dc640 1 --2- 172.21.15.27:0/706468837 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4880cab50 0x7fe48814d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.011+0000 7fe48d8dc640 1 -- 172.21.15.27:0/706468837 >> 172.21.15.27:0/706468837 conn(0x7fe488069c70 msgr2=0x7fe488153760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.012+0000 7fe48d8dc640 1 -- 172.21.15.27:0/706468837 shutdown_connections 2023-12-19T18:28:51.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:51.012+0000 7fe48d8dc640 1 -- 172.21.15.27:0/706468837 wait complete. 2023-12-19T18:28:51.029 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json status 2023-12-19T18:28:51.270 INFO:teuthology.orchestra.run.smithi027.stdout:{"cluster_fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","whoami":0,"id":8997,"want_state":"up:active","state":"up:active","fs_name":"cephfs","rank_uptime":42.673433011,"mdsmap_epoch":85,"osdmap_epoch":206,"osdmap_epoch_barrier":204,"uptime":48.467352480000002} 2023-12-19T18:28:51.270 DEBUG:tasks.cephfs.filesystem:_json_asok output { "cluster_fsid": "dc17deb5-5eca-4f5c-983d-094ffb50339f", "fs_name": "cephfs", "id": 8997, "mdsmap_epoch": 85, "osdmap_epoch": 206, "osdmap_epoch_barrier": 204, "rank_uptime": 42.673433011, "state": "up:active", "uptime": 48.46735248, "want_state": "up:active", "whoami": 0 } 2023-12-19T18:28:51.271 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2023-12-19T18:28:51.271 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:28:51.271 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:28:51.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:51.327 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=large_file bs=1M conv=fdatasync count=50 seek=0) 2023-12-19T18:28:52.360 INFO:teuthology.orchestra.run.smithi027.stderr:50+0 records in 2023-12-19T18:28:52.360 INFO:teuthology.orchestra.run.smithi027.stderr:50+0 records out 2023-12-19T18:28:52.360 INFO:teuthology.orchestra.run.smithi027.stderr:52428800 bytes (52 MB, 50 MiB) copied, 0.97311 s, 53.9 MB/s 2023-12-19T18:28:52.362 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-19T18:28:52.362 DEBUG:teuthology.orchestra.run.smithi027:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:28:52.513 DEBUG:teuthology.orchestra.run.smithi192:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-19T18:28:52.640 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-19T18:28:52.640 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:28:52.641 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:28:52.705 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:28:52.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.936+0000 7f164bfff640 1 Processor -- start 2023-12-19T18:28:52.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.936+0000 7f164bfff640 1 -- start start 2023-12-19T18:28:52.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.938+0000 7f164bfff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f164c1553a0 0x7f164c0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:52.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.938+0000 7f164bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f164c0afeb0 0x7f164c15e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:52.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.938+0000 7f164bfff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f164c0b0330 con 0x7f164c15ec60 2023-12-19T18:28:52.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.938+0000 7f164bfff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f164c0cbcf0 con 0x7f164c1553a0 2023-12-19T18:28:52.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.938+0000 7f164bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164c15f0d0 con 0x7f164c0afeb0 2023-12-19T18:28:52.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.938+0000 7f164b7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f164c15ec60 0x7f164c15b130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55212/0 (socket says 172.21.15.27:55212) 2023-12-19T18:28:52.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.938+0000 7f164b7fe640 1 -- 172.21.15.27:0/1454462610 learned_addr learned my addr 172.21.15.27:0/1454462610 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:52.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.939+0000 7f164affd640 1 --2- 172.21.15.27:0/1454462610 >> v2:172.21.15.192:3300/0 conn(0x7f164c1553a0 0x7f164c0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:52.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.939+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4225074544 0 0) 0x7f164c0b0330 con 0x7f164c15ec60 2023-12-19T18:28:52.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.939+0000 7f164a7fc640 1 --2- 172.21.15.27:0/1454462610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f164c0afeb0 0x7f164c15e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.939+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1634003520 con 0x7f164c15ec60 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.939+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4127682672 0 0) 0x7f1634003520 con 0x7f164c15ec60 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.939+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f164c0afeb0 msgr2=0x7f164c15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.939+0000 7f1649ffb640 1 --2- 172.21.15.27:0/1454462610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f164c0afeb0 0x7f164c15e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.939+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 >> v2:172.21.15.192:3300/0 conn(0x7f164c1553a0 msgr2=0x7f164c0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.939+0000 7f1649ffb640 1 --2- 172.21.15.27:0/1454462610 >> v2:172.21.15.192:3300/0 conn(0x7f164c1553a0 0x7f164c0c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.939+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c0cb350 con 0x7f164c15ec60 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.939+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f16440019f0 con 0x7f164c15ec60 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.939+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 >> v1:172.21.15.27:6789/0 conn(0x7f164c15ec60 legacy=0x7f164c15b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.940+0000 7f1649ffb640 1 --2- 172.21.15.27:0/1454462610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16340039a0 0x7f1634003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.940+0000 7f1649ffb640 1 --2- 172.21.15.27:0/1454462610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16340047c0 0x7f1634004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.940+0000 7f1649ffb640 1 --2- 172.21.15.27:0/1454462610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f164c0afeb0 0x7f1634005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.940+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c0cbcf0 con 0x7f16340039a0 2023-12-19T18:28:52.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.940+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c15f0d0 con 0x7f16340047c0 2023-12-19T18:28:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.940+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f163400ba00 con 0x7f164c0afeb0 2023-12-19T18:28:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.940+0000 7f164affd640 1 --2- 172.21.15.27:0/1454462610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16340039a0 0x7f1634003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.940+0000 7f164a7fc640 1 --2- 172.21.15.27:0/1454462610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16340047c0 0x7f1634004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.940+0000 7f164affd640 1 -- 172.21.15.27:0/1454462610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f164c0afeb0 msgr2=0x7f1634005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.940+0000 7f164affd640 1 --2- 172.21.15.27:0/1454462610 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f164c0afeb0 0x7f1634005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.940+0000 7f164affd640 1 -- 172.21.15.27:0/1454462610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16340047c0 msgr2=0x7f1634004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.940+0000 7f164affd640 1 --2- 172.21.15.27:0/1454462610 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f16340047c0 0x7f1634004c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.940+0000 7f164affd640 1 -- 172.21.15.27:0/1454462610 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1634003520 con 0x7f16340039a0 2023-12-19T18:28:52.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.941+0000 7f164affd640 1 --2- 172.21.15.27:0/1454462610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16340039a0 0x7f1634003dc0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f163c009ac0 tx=0x7f16340057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d73927c275e6d5a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:52.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.942+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c02c710 con 0x7f16340039a0 2023-12-19T18:28:52.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.942+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c02c870 con 0x7f16340039a0 2023-12-19T18:28:52.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.942+0000 7f1649ffb640 1 -- 172.21.15.27:0/1454462610 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c02cc00 con 0x7f16340039a0 2023-12-19T18:28:52.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.942+0000 7f164bfff640 1 -- 172.21.15.27:0/1454462610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16340039a0 msgr2=0x7f1634003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:52.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.942+0000 7f164bfff640 1 --2- 172.21.15.27:0/1454462610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16340039a0 0x7f1634003dc0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f163c009ac0 tx=0x7f16340057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:52.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.942+0000 7f164b7fe640 1 -- 172.21.15.27:0/1454462610 reap_dead start 2023-12-19T18:28:52.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.942+0000 7f164bfff640 1 -- 172.21.15.27:0/1454462610 shutdown_connections 2023-12-19T18:28:52.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.942+0000 7f164bfff640 1 -- 172.21.15.27:0/1454462610 >> 172.21.15.27:0/1454462610 conn(0x7f164c069c70 msgr2=0x7f164c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:52.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.943+0000 7f164bfff640 1 -- 172.21.15.27:0/1454462610 shutdown_connections 2023-12-19T18:28:52.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.943+0000 7f164bfff640 1 -- 172.21.15.27:0/1454462610 wait complete. 2023-12-19T18:28:52.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.944+0000 7f164bfff640 1 Processor -- start 2023-12-19T18:28:52.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.944+0000 7f164bfff640 1 -- start start 2023-12-19T18:28:52.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.945+0000 7f164bfff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f16340039a0 0x7f164c0c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:52.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.945+0000 7f164bfff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f164c0b3b00 0x7f164c150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:52.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.945+0000 7f164bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f164c06c9e0 0x7f164c06ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:52.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.945+0000 7f164bfff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c15ca30 con 0x7f164c06c9e0 2023-12-19T18:28:52.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.945+0000 7f164bfff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164c0c8c60 con 0x7f164c0b3b00 2023-12-19T18:28:52.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.945+0000 7f164bfff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164c06d3a0 con 0x7f16340039a0 2023-12-19T18:28:52.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.946+0000 7f164b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f164c06c9e0 0x7f164c06ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:52.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.946+0000 7f164b7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f164c06c9e0 0x7f164c06ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32906/0 (socket says 172.21.15.27:32906) 2023-12-19T18:28:52.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.946+0000 7f164b7fe640 1 -- 172.21.15.27:0/3780519769 learned_addr learned my addr 172.21.15.27:0/3780519769 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:52.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.946+0000 7f164affd640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f16340039a0 0x7f164c0c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:52.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.946+0000 7f164a7fc640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f164c0b3b00 0x7f164c150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:52.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.946+0000 7f164b7fe640 1 -- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f16340039a0 msgr2=0x7f164c0c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:52.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.946+0000 7f164b7fe640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f16340039a0 0x7f164c0c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:52.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.946+0000 7f164b7fe640 1 -- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f164c0b3b00 msgr2=0x7f164c150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:52.947 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.946+0000 7f164b7fe640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f164c0b3b00 0x7f164c150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:52.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.946+0000 7f164b7fe640 1 -- 172.21.15.27:0/3780519769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f164c0c5a10 con 0x7f164c06c9e0 2023-12-19T18:28:52.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.947+0000 7f164b7fe640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f164c06c9e0 0x7f164c06ce60 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f164c0af640 tx=0x7f164400ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:52.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.947+0000 7f16337fe640 1 -- 172.21.15.27:0/3780519769 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164400a610 con 0x7f164c06c9e0 2023-12-19T18:28:52.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.947+0000 7f16337fe640 1 -- 172.21.15.27:0/3780519769 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f164400a790 con 0x7f164c06c9e0 2023-12-19T18:28:52.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.947+0000 7f16337fe640 1 -- 172.21.15.27:0/3780519769 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f164400ab40 con 0x7f164c06c9e0 2023-12-19T18:28:52.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.947+0000 7f164bfff640 1 -- 172.21.15.27:0/3780519769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164c0b42c0 con 0x7f164c06c9e0 2023-12-19T18:28:52.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.947+0000 7f164bfff640 1 -- 172.21.15.27:0/3780519769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f164c15c650 con 0x7f164c06c9e0 2023-12-19T18:28:52.948 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.949+0000 7f164bfff640 1 -- 172.21.15.27:0/3780519769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164c15cb70 con 0x7f164c06c9e0 2023-12-19T18:28:52.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.950+0000 7f16337fe640 1 -- 172.21.15.27:0/3780519769 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f164400aca0 con 0x7f164c06c9e0 2023-12-19T18:28:52.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.951+0000 7f16337fe640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f163806a430 0x7f163806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:52.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.951+0000 7f16337fe640 1 -- 172.21.15.27:0/3780519769 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1644088540 con 0x7f164c06c9e0 2023-12-19T18:28:52.951 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.951+0000 7f164affd640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f163806a430 0x7f163806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:52.952 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.953+0000 7f164affd640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f163806a430 0x7f163806c8f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f163c009ac0 tx=0x7f163c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:52.955 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:52.956+0000 7f16337fe640 1 -- 172.21.15.27:0/3780519769 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f164c15cb70 con 0x7f164c06c9e0 2023-12-19T18:28:53.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.154+0000 7f164bfff640 1 -- 172.21.15.27:0/3780519769 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f164c15cb70 con 0x7f164c06c9e0 2023-12-19T18:28:53.155 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.156+0000 7f16337fe640 1 -- 172.21.15.27:0/3780519769 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) v1 ==== 74+0+16169 (secure 0 0 0) 0x7f164c15cb70 con 0x7f164c06c9e0 2023-12-19T18:28:53.156 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:53.156 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":206,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:28:46.843951+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:28:06.542489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:28:07.623298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.26","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:28:53.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.158+0000 7f164bfff640 1 -- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f163806a430 msgr2=0x7f163806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:53.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.158+0000 7f164bfff640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f163806a430 0x7f163806c8f0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f163c009ac0 tx=0x7f163c030000 comp rx=0 tx=0).stop 2023-12-19T18:28:53.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.158+0000 7f164bfff640 1 -- 172.21.15.27:0/3780519769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f164c06c9e0 msgr2=0x7f164c06ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:53.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.158+0000 7f164bfff640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f164c06c9e0 0x7f164c06ce60 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f164c0af640 tx=0x7f164400ce80 comp rx=0 tx=0).stop 2023-12-19T18:28:53.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.159+0000 7f164bfff640 1 -- 172.21.15.27:0/3780519769 shutdown_connections 2023-12-19T18:28:53.159 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.159+0000 7f164bfff640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f163806a430 0x7f163806c8f0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.159+0000 7f164bfff640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f164c06c9e0 0x7f164c06ce60 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.159+0000 7f164bfff640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f164c0b3b00 0x7f164c150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.159+0000 7f164bfff640 1 --2- 172.21.15.27:0/3780519769 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f16340039a0 0x7f164c0c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.159+0000 7f164bfff640 1 -- 172.21.15.27:0/3780519769 >> 172.21.15.27:0/3780519769 conn(0x7f164c069c70 msgr2=0x7f164c0c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.159+0000 7f164bfff640 1 -- 172.21.15.27:0/3780519769 shutdown_connections 2023-12-19T18:28:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.159+0000 7f164bfff640 1 -- 172.21.15.27:0/3780519769 wait complete. 2023-12-19T18:28:53.178 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:28:06.542489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '201', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:28:07.623298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '206', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:28:53.179 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:28:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.399+0000 7f3016c6d640 1 Processor -- start 2023-12-19T18:28:53.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.399+0000 7f3016c6d640 1 -- start start 2023-12-19T18:28:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.401+0000 7f3016c6d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f301015d120 0x7f30100cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.401+0000 7f3016c6d640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30100d3850 con 0x7f30100ce950 2023-12-19T18:28:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.401+0000 7f3016c6d640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30100d0b10 con 0x7f30100d1310 2023-12-19T18:28:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.401+0000 7f3016c6d640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f301015d500 con 0x7f301015d120 2023-12-19T18:28:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.401+0000 7f3015c6b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f301015d120 0x7f30100cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:53.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.401+0000 7f301646c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f30100ce950 0x7f30100b3080 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55218/0 (socket says 172.21.15.27:55218) 2023-12-19T18:28:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.401+0000 7f301646c640 1 -- 172.21.15.27:0/4136925326 learned_addr learned my addr 172.21.15.27:0/4136925326 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.402+0000 7f3014c69640 1 -- 172.21.15.27:0/4136925326 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792280922 0 0) 0x7f30100d3850 con 0x7f30100ce950 2023-12-19T18:28:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.402+0000 7f3014c69640 1 -- 172.21.15.27:0/4136925326 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ff8003520 con 0x7f30100ce950 2023-12-19T18:28:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.402+0000 7f3015c6b640 1 -- 172.21.15.27:0/4136925326 >> v1:172.21.15.192:6789/0 conn(0x7f30100d1310 legacy=0x7f30100b2970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.402+0000 7f3015c6b640 1 -- 172.21.15.27:0/4136925326 >> v1:172.21.15.27:6789/0 conn(0x7f30100ce950 legacy=0x7f30100b3080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:53.402 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.402+0000 7f3015c6b640 1 -- 172.21.15.27:0/4136925326 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30100d1040 con 0x7f301015d120 2023-12-19T18:28:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.403+0000 7f3015c6b640 1 --2- 172.21.15.27:0/4136925326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301015d120 0x7f30100cfd70 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f3000009d90 tx=0x7f300002bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4bfddddce35c2b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.403+0000 7f3014c69640 1 -- 172.21.15.27:0/4136925326 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300002cd00 con 0x7f301015d120 2023-12-19T18:28:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.403+0000 7f3014c69640 1 -- 172.21.15.27:0/4136925326 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300002ce60 con 0x7f301015d120 2023-12-19T18:28:53.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.403+0000 7f3014c69640 1 -- 172.21.15.27:0/4136925326 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3000034660 con 0x7f301015d120 2023-12-19T18:28:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.404+0000 7f3016c6d640 1 -- 172.21.15.27:0/4136925326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301015d120 msgr2=0x7f30100cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.404+0000 7f3016c6d640 1 --2- 172.21.15.27:0/4136925326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301015d120 0x7f30100cfd70 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f3000009d90 tx=0x7f300002bf50 comp rx=0 tx=0).stop 2023-12-19T18:28:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.404+0000 7f3016c6d640 1 -- 172.21.15.27:0/4136925326 shutdown_connections 2023-12-19T18:28:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.404+0000 7f3016c6d640 1 --2- 172.21.15.27:0/4136925326 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f301015d120 0x7f30100cfd70 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.404+0000 7f3016c6d640 1 -- 172.21.15.27:0/4136925326 >> 172.21.15.27:0/4136925326 conn(0x7f30100d2d30 msgr2=0x7f30100d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:53.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.404+0000 7f3016c6d640 1 -- 172.21.15.27:0/4136925326 shutdown_connections 2023-12-19T18:28:53.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.404+0000 7f3016c6d640 1 -- 172.21.15.27:0/4136925326 wait complete. 2023-12-19T18:28:53.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.406+0000 7f3016c6d640 1 Processor -- start 2023-12-19T18:28:53.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.406+0000 7f3016c6d640 1 -- start start 2023-12-19T18:28:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.407+0000 7f3016c6d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30100ce950 0x7f301015e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.407+0000 7f3016c6d640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30100d1310 0x7f301015a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.407+0000 7f3016c6d640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f301015d120 0x7f30100b38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.407+0000 7f3016c6d640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301015e640 con 0x7f30100ce950 2023-12-19T18:28:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.407+0000 7f3016c6d640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f301015f530 con 0x7f30100d1310 2023-12-19T18:28:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.407+0000 7f3016c6d640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301015f810 con 0x7f301015d120 2023-12-19T18:28:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.407+0000 7f3015c6b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30100ce950 0x7f301015e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.407+0000 7f3015c6b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30100ce950 0x7f301015e100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32918/0 (socket says 172.21.15.27:32918) 2023-12-19T18:28:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.407+0000 7f3015c6b640 1 -- 172.21.15.27:0/691380193 learned_addr learned my addr 172.21.15.27:0/691380193 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.408+0000 7f3015c6b640 1 -- 172.21.15.27:0/691380193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f301015d120 msgr2=0x7f30100b38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:53.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.408+0000 7f3015c6b640 1 --2- 172.21.15.27:0/691380193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f301015d120 0x7f30100b38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.408+0000 7f3015c6b640 1 -- 172.21.15.27:0/691380193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30100d1310 msgr2=0x7f301015a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:53.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.408+0000 7f301546a640 1 --2- 172.21.15.27:0/691380193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30100d1310 0x7f301015a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:53.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.408+0000 7f3015c6b640 1 --2- 172.21.15.27:0/691380193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30100d1310 0x7f301015a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.408+0000 7f3015c6b640 1 -- 172.21.15.27:0/691380193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3000009a70 con 0x7f30100ce950 2023-12-19T18:28:53.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.408+0000 7f3015c6b640 1 --2- 172.21.15.27:0/691380193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30100ce950 0x7f301015e100 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f3000009d90 tx=0x7f3000004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.408+0000 7f300e7fc640 1 -- 172.21.15.27:0/691380193 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300003d600 con 0x7f30100ce950 2023-12-19T18:28:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.408+0000 7f300e7fc640 1 -- 172.21.15.27:0/691380193 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f300003d7a0 con 0x7f30100ce950 2023-12-19T18:28:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.408+0000 7f300e7fc640 1 -- 172.21.15.27:0/691380193 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f300003db50 con 0x7f30100ce950 2023-12-19T18:28:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.408+0000 7f3016c6d640 1 -- 172.21.15.27:0/691380193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f301015faf0 con 0x7f30100ce950 2023-12-19T18:28:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.409+0000 7f3016c6d640 1 -- 172.21.15.27:0/691380193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30101600b0 con 0x7f30100ce950 2023-12-19T18:28:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.410+0000 7f3016c6d640 1 -- 172.21.15.27:0/691380193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f301004e840 con 0x7f30100ce950 2023-12-19T18:28:53.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.411+0000 7f300e7fc640 1 -- 172.21.15.27:0/691380193 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f300003b040 con 0x7f30100ce950 2023-12-19T18:28:53.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.412+0000 7f300e7fc640 1 --2- 172.21.15.27:0/691380193 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ffc06a430 0x7f2ffc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:53.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.412+0000 7f300e7fc640 1 -- 172.21.15.27:0/691380193 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f30000aabb0 con 0x7f30100ce950 2023-12-19T18:28:53.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.413+0000 7f301546a640 1 --2- 172.21.15.27:0/691380193 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ffc06a430 0x7f2ffc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:53.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.414+0000 7f301546a640 1 --2- 172.21.15.27:0/691380193 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ffc06a430 0x7f2ffc06c8f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f30101f6d60 tx=0x7f3004009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:53.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.417+0000 7f300e7fc640 1 -- 172.21.15.27:0/691380193 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3000076290 con 0x7f30100ce950 2023-12-19T18:28:53.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.614+0000 7f3016c6d640 1 -- 172.21.15.27:0/691380193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3010069f20 con 0x7f30100ce950 2023-12-19T18:28:53.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.614+0000 7f300e7fc640 1 -- 172.21.15.27:0/691380193 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v206) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3000039070 con 0x7f30100ce950 2023-12-19T18:28:53.614 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:28:53.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.617+0000 7f3016c6d640 1 -- 172.21.15.27:0/691380193 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ffc06a430 msgr2=0x7f2ffc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:53.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.617+0000 7f3016c6d640 1 --2- 172.21.15.27:0/691380193 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ffc06a430 0x7f2ffc06c8f0 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f30101f6d60 tx=0x7f3004009040 comp rx=0 tx=0).stop 2023-12-19T18:28:53.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.617+0000 7f3016c6d640 1 -- 172.21.15.27:0/691380193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30100ce950 msgr2=0x7f301015e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:53.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.617+0000 7f3016c6d640 1 --2- 172.21.15.27:0/691380193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30100ce950 0x7f301015e100 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f3000009d90 tx=0x7f3000004a40 comp rx=0 tx=0).stop 2023-12-19T18:28:53.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.617+0000 7f3016c6d640 1 -- 172.21.15.27:0/691380193 shutdown_connections 2023-12-19T18:28:53.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.617+0000 7f3016c6d640 1 --2- 172.21.15.27:0/691380193 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2ffc06a430 0x7f2ffc06c8f0 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.617+0000 7f3016c6d640 1 --2- 172.21.15.27:0/691380193 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f301015d120 0x7f30100b38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.617+0000 7f3016c6d640 1 --2- 172.21.15.27:0/691380193 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30100d1310 0x7f301015a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.617+0000 7f3016c6d640 1 --2- 172.21.15.27:0/691380193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30100ce950 0x7f301015e100 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.617+0000 7f3016c6d640 1 -- 172.21.15.27:0/691380193 >> 172.21.15.27:0/691380193 conn(0x7f30100d2d30 msgr2=0x7f301006a350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:53.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.617+0000 7f3016c6d640 1 -- 172.21.15.27:0/691380193 shutdown_connections 2023-12-19T18:28:53.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.617+0000 7f3016c6d640 1 -- 172.21.15.27:0/691380193 wait complete. 2023-12-19T18:28:53.635 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:28:53.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.863+0000 7fdc35957640 1 Processor -- start 2023-12-19T18:28:53.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.863+0000 7fdc35957640 1 -- start start 2023-12-19T18:28:53.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.865+0000 7fdc35957640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fdc30151c90 0x7fdc3006b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:53.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.865+0000 7fdc35957640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3015b3e0 con 0x7fdc300cf550 2023-12-19T18:28:53.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.865+0000 7fdc35957640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc3015eed0 con 0x7fdc3015e9d0 2023-12-19T18:28:53.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.865+0000 7fdc35957640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc3015f1b0 con 0x7fdc30151c90 2023-12-19T18:28:53.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.865+0000 7fdc35156640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fdc300cf550 0x7fdc3015b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55232/0 (socket says 172.21.15.27:55232) 2023-12-19T18:28:53.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.865+0000 7fdc35156640 1 -- 172.21.15.27:0/2308977589 learned_addr learned my addr 172.21.15.27:0/2308977589 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:53.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.865+0000 7fdc34955640 1 --2- 172.21.15.27:0/2308977589 >> v2:172.21.15.27:3300/0 conn(0x7fdc30151c90 0x7fdc3006b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:53.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.866+0000 7fdc2f7fe640 1 -- 172.21.15.27:0/2308977589 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2874064829 0 0) 0x7fdc3015b3e0 con 0x7fdc300cf550 2023-12-19T18:28:53.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.866+0000 7fdc2f7fe640 1 -- 172.21.15.27:0/2308977589 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc20003520 con 0x7fdc300cf550 2023-12-19T18:28:53.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.866+0000 7fdc34955640 1 -- 172.21.15.27:0/2308977589 >> v1:172.21.15.192:6789/0 conn(0x7fdc3015e9d0 legacy=0x7fdc300cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:53.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.866+0000 7fdc34955640 1 -- 172.21.15.27:0/2308977589 >> v1:172.21.15.27:6789/0 conn(0x7fdc300cf550 legacy=0x7fdc3015b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:53.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.866+0000 7fdc34955640 1 -- 172.21.15.27:0/2308977589 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc3015f490 con 0x7fdc30151c90 2023-12-19T18:28:53.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.867+0000 7fdc34955640 1 --2- 172.21.15.27:0/2308977589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc30151c90 0x7fdc3006b4d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fdc240097d0 tx=0x7fdc2402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d437c471e1a88920 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:53.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.868+0000 7fdc2f7fe640 1 -- 172.21.15.27:0/2308977589 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2402e070 con 0x7fdc30151c90 2023-12-19T18:28:53.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.868+0000 7fdc2f7fe640 1 -- 172.21.15.27:0/2308977589 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc240315a0 con 0x7fdc30151c90 2023-12-19T18:28:53.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.868+0000 7fdc2f7fe640 1 -- 172.21.15.27:0/2308977589 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc24031930 con 0x7fdc30151c90 2023-12-19T18:28:53.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.868+0000 7fdc35957640 1 -- 172.21.15.27:0/2308977589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc30151c90 msgr2=0x7fdc3006b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:53.868 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.868+0000 7fdc35957640 1 --2- 172.21.15.27:0/2308977589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc30151c90 0x7fdc3006b4d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fdc240097d0 tx=0x7fdc2402b840 comp rx=0 tx=0).stop 2023-12-19T18:28:53.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.868+0000 7fdc35957640 1 -- 172.21.15.27:0/2308977589 shutdown_connections 2023-12-19T18:28:53.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.868+0000 7fdc35957640 1 --2- 172.21.15.27:0/2308977589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc30151c90 0x7fdc3006b4d0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.868+0000 7fdc35957640 1 -- 172.21.15.27:0/2308977589 >> 172.21.15.27:0/2308977589 conn(0x7fdc3006a060 msgr2=0x7fdc3006a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:53.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.868+0000 7fdc35957640 1 -- 172.21.15.27:0/2308977589 shutdown_connections 2023-12-19T18:28:53.869 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.868+0000 7fdc35957640 1 -- 172.21.15.27:0/2308977589 wait complete. 2023-12-19T18:28:53.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.870+0000 7fdc35957640 1 Processor -- start 2023-12-19T18:28:53.870 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.870+0000 7fdc35957640 1 -- start start 2023-12-19T18:28:53.871 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.871+0000 7fdc35957640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc300cf550 0x7fdc300b2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.871+0000 7fdc35957640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc30151c90 0x7fdc3015de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.871+0000 7fdc35957640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc3015e9d0 0x7fdc3015e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.871+0000 7fdc35957640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc300b2f10 con 0x7fdc3015e9d0 2023-12-19T18:28:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.871+0000 7fdc35957640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc300b31f0 con 0x7fdc300cf550 2023-12-19T18:28:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.871+0000 7fdc35957640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc300ae720 con 0x7fdc30151c90 2023-12-19T18:28:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.872+0000 7fdc35156640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc3015e9d0 0x7fdc3015e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.872+0000 7fdc35156640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc3015e9d0 0x7fdc3015e340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32940/0 (socket says 172.21.15.27:32940) 2023-12-19T18:28:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.872+0000 7fdc35156640 1 -- 172.21.15.27:0/1167853319 learned_addr learned my addr 172.21.15.27:0/1167853319 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:53.872 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.872+0000 7fdc34955640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc300cf550 0x7fdc300b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.872+0000 7fdc2ffff640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc30151c90 0x7fdc3015de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.872+0000 7fdc35156640 1 -- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc30151c90 msgr2=0x7fdc3015de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.872+0000 7fdc35156640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc30151c90 0x7fdc3015de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.872+0000 7fdc35156640 1 -- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc300cf550 msgr2=0x7fdc300b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.872+0000 7fdc35156640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc300cf550 0x7fdc300b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.872+0000 7fdc35156640 1 -- 172.21.15.27:0/1167853319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc24009400 con 0x7fdc3015e9d0 2023-12-19T18:28:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.872+0000 7fdc34955640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc300cf550 0x7fdc300b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:28:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.873+0000 7fdc35156640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc3015e9d0 0x7fdc3015e340 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800caa0 tx=0x7fdc2800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.873+0000 7fdc2d7fa640 1 -- 172.21.15.27:0/1167853319 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2800ac00 con 0x7fdc3015e9d0 2023-12-19T18:28:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.873+0000 7fdc2d7fa640 1 -- 172.21.15.27:0/1167853319 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2800ad60 con 0x7fdc3015e9d0 2023-12-19T18:28:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.873+0000 7fdc2d7fa640 1 -- 172.21.15.27:0/1167853319 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc28012620 con 0x7fdc3015e9d0 2023-12-19T18:28:53.874 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.873+0000 7fdc35957640 1 -- 172.21.15.27:0/1167853319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc300aea00 con 0x7fdc3015e9d0 2023-12-19T18:28:53.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.873+0000 7fdc35957640 1 -- 172.21.15.27:0/1167853319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc300b2b70 con 0x7fdc3015e9d0 2023-12-19T18:28:53.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.874+0000 7fdc35957640 1 -- 172.21.15.27:0/1167853319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc300aebf0 con 0x7fdc3015e9d0 2023-12-19T18:28:53.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.875+0000 7fdc2d7fa640 1 -- 172.21.15.27:0/1167853319 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fdc28007070 con 0x7fdc3015e9d0 2023-12-19T18:28:53.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.876+0000 7fdc2d7fa640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc1406a430 0x7fdc1406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:53.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.876+0000 7fdc2d7fa640 1 -- 172.21.15.27:0/1167853319 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fdc280888a0 con 0x7fdc3015e9d0 2023-12-19T18:28:53.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.877+0000 7fdc34955640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc1406a430 0x7fdc1406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:53.878 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.878+0000 7fdc34955640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc1406a430 0x7fdc1406c8f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdc24036000 tx=0x7fdc2403d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:53.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:53.881+0000 7fdc2d7fa640 1 -- 172.21.15.27:0/1167853319 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fdc28053f00 con 0x7fdc3015e9d0 2023-12-19T18:28:54.078 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.078+0000 7fdc35957640 1 -- 172.21.15.27:0/1167853319 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdc3004e840 con 0x7fdc3015e9d0 2023-12-19T18:28:54.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.079+0000 7fdc2d7fa640 1 -- 172.21.15.27:0/1167853319 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v206) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdc280170a0 con 0x7fdc3015e9d0 2023-12-19T18:28:54.079 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:28:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.082+0000 7fdc35957640 1 -- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc1406a430 msgr2=0x7fdc1406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.082+0000 7fdc35957640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc1406a430 0x7fdc1406c8f0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7fdc24036000 tx=0x7fdc2403d040 comp rx=0 tx=0).stop 2023-12-19T18:28:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.082+0000 7fdc35957640 1 -- 172.21.15.27:0/1167853319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc3015e9d0 msgr2=0x7fdc3015e340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.082+0000 7fdc35957640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc3015e9d0 0x7fdc3015e340 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7fdc2800caa0 tx=0x7fdc2800cf60 comp rx=0 tx=0).stop 2023-12-19T18:28:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.082+0000 7fdc35957640 1 -- 172.21.15.27:0/1167853319 shutdown_connections 2023-12-19T18:28:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.082+0000 7fdc35957640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fdc1406a430 0x7fdc1406c8f0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.082+0000 7fdc35957640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc3015e9d0 0x7fdc3015e340 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.082+0000 7fdc35957640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fdc30151c90 0x7fdc3015de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.082+0000 7fdc35957640 1 --2- 172.21.15.27:0/1167853319 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fdc300cf550 0x7fdc300b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.082+0000 7fdc35957640 1 -- 172.21.15.27:0/1167853319 >> 172.21.15.27:0/1167853319 conn(0x7fdc3006a060 msgr2=0x7fdc3015c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.083+0000 7fdc35957640 1 -- 172.21.15.27:0/1167853319 shutdown_connections 2023-12-19T18:28:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.083+0000 7fdc35957640 1 -- 172.21.15.27:0/1167853319 wait complete. 2023-12-19T18:28:54.103 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:28:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.335+0000 7f2f6c95b640 1 Processor -- start 2023-12-19T18:28:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.335+0000 7f2f6c95b640 1 -- start start 2023-12-19T18:28:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.337+0000 7f2f6c95b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f2f68151c90 0x7f2f6806b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.337+0000 7f2f6c95b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f6815e9d0 0x7f2f680cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.337+0000 7f2f6c95b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6815b3e0 con 0x7f2f68152070 2023-12-19T18:28:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.337+0000 7f2f6c95b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6815eed0 con 0x7f2f68151c90 2023-12-19T18:28:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.337+0000 7f2f6c95b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6815f1b0 con 0x7f2f6815e9d0 2023-12-19T18:28:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.338+0000 7f2f6759e640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f2f68152070 0x7f2f6815b8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55246/0 (socket says 172.21.15.27:55246) 2023-12-19T18:28:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.338+0000 7f2f6759e640 1 -- 172.21.15.27:0/19937783 learned_addr learned my addr 172.21.15.27:0/19937783 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.338+0000 7f2f66d9d640 1 --2- 172.21.15.27:0/19937783 >> v2:172.21.15.192:3300/0 conn(0x7f2f68151c90 0x7f2f6806b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.338+0000 7f2f5ffff640 1 --2- 172.21.15.27:0/19937783 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f6815e9d0 0x7f2f680cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.338+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4045780193 0 0) 0x7f2f6815b3e0 con 0x7f2f68152070 2023-12-19T18:28:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.338+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f54003520 con 0x7f2f68152070 2023-12-19T18:28:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1302277849 0 0) 0x7f2f54003520 con 0x7f2f68152070 2023-12-19T18:28:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f6815e9d0 msgr2=0x7f2f680cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 --2- 172.21.15.27:0/19937783 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f6815e9d0 0x7f2f680cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 >> v2:172.21.15.192:3300/0 conn(0x7f2f68151c90 msgr2=0x7f2f6806b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 --2- 172.21.15.27:0/19937783 >> v2:172.21.15.192:3300/0 conn(0x7f2f68151c90 0x7f2f6806b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6815f490 con 0x7f2f68152070 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f2f600019f0 con 0x7f2f68152070 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 >> v1:172.21.15.27:6789/0 conn(0x7f2f68152070 legacy=0x7f2f6815b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 --2- 172.21.15.27:0/19937783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f54003970 0x7f2f54003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 --2- 172.21.15.27:0/19937783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f540047c0 0x7f2f54004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 --2- 172.21.15.27:0/19937783 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f54005180 0x7f2f5400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6815eed0 con 0x7f2f540047c0 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6815f1b0 con 0x7f2f54003970 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f5400bef0 con 0x7f2f54005180 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f5ffff640 1 --2- 172.21.15.27:0/19937783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f540047c0 0x7f2f54004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.339+0000 7f2f66d9d640 1 --2- 172.21.15.27:0/19937783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f54003970 0x7f2f54003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.340+0000 7f2f6759e640 1 --2- 172.21.15.27:0/19937783 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f54005180 0x7f2f5400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.340+0000 7f2f5ffff640 1 -- 172.21.15.27:0/19937783 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f54005180 msgr2=0x7f2f5400b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.340+0000 7f2f5ffff640 1 --2- 172.21.15.27:0/19937783 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f54005180 0x7f2f5400b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.340+0000 7f2f5ffff640 1 -- 172.21.15.27:0/19937783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f54003970 msgr2=0x7f2f54003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.340+0000 7f2f5ffff640 1 --2- 172.21.15.27:0/19937783 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f54003970 0x7f2f54003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.340+0000 7f2f5ffff640 1 -- 172.21.15.27:0/19937783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f54003520 con 0x7f2f540047c0 2023-12-19T18:28:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.340+0000 7f2f6759e640 1 --2- 172.21.15.27:0/19937783 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f54005180 0x7f2f5400b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.341+0000 7f2f5ffff640 1 --2- 172.21.15.27:0/19937783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f540047c0 0x7f2f54004c40 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f2f6815edb0 tx=0x7f2f540057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395e059db20e37d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.341+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5002c540 con 0x7f2f540047c0 2023-12-19T18:28:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.341+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5002c6a0 con 0x7f2f540047c0 2023-12-19T18:28:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.341+0000 7f2f6659c640 1 -- 172.21.15.27:0/19937783 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5002ca10 con 0x7f2f540047c0 2023-12-19T18:28:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.342+0000 7f2f6c95b640 1 -- 172.21.15.27:0/19937783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f540047c0 msgr2=0x7f2f54004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.342+0000 7f2f6c95b640 1 --2- 172.21.15.27:0/19937783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f540047c0 0x7f2f54004c40 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f2f6815edb0 tx=0x7f2f540057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.342+0000 7f2f6759e640 1 -- 172.21.15.27:0/19937783 reap_dead start 2023-12-19T18:28:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.342+0000 7f2f6c95b640 1 -- 172.21.15.27:0/19937783 shutdown_connections 2023-12-19T18:28:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.342+0000 7f2f6c95b640 1 -- 172.21.15.27:0/19937783 >> 172.21.15.27:0/19937783 conn(0x7f2f6806a060 msgr2=0x7f2f6806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.342+0000 7f2f6c95b640 1 -- 172.21.15.27:0/19937783 shutdown_connections 2023-12-19T18:28:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.342+0000 7f2f6c95b640 1 -- 172.21.15.27:0/19937783 wait complete. 2023-12-19T18:28:54.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.344+0000 7f2f6c95b640 1 Processor -- start 2023-12-19T18:28:54.345 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.344+0000 7f2f6c95b640 1 -- start start 2023-12-19T18:28:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.345+0000 7f2f6c95b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f540047c0 0x7f2f6815e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.345+0000 7f2f6c95b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f6815d780 0x7f2f68148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.346 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.345+0000 7f2f6c95b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f680ae5a0 0x7f2f680aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.345+0000 7f2f6c95b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f680aef60 con 0x7f2f540047c0 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.345+0000 7f2f6c95b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6815ce60 con 0x7f2f680ae5a0 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.345+0000 7f2f6c95b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6815c250 con 0x7f2f6815d780 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.345+0000 7f2f66d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f540047c0 0x7f2f6815e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.345+0000 7f2f66d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f540047c0 0x7f2f6815e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32954/0 (socket says 172.21.15.27:32954) 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.345+0000 7f2f66d9d640 1 -- 172.21.15.27:0/1380959184 learned_addr learned my addr 172.21.15.27:0/1380959184 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.346+0000 7f2f66d9d640 1 -- 172.21.15.27:0/1380959184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f6815d780 msgr2=0x7f2f68148670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.346+0000 7f2f66d9d640 1 --2- 172.21.15.27:0/1380959184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f6815d780 0x7f2f68148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.346+0000 7f2f66d9d640 1 -- 172.21.15.27:0/1380959184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f680ae5a0 msgr2=0x7f2f680aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.346+0000 7f2f66d9d640 1 --2- 172.21.15.27:0/1380959184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f680ae5a0 0x7f2f680aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.346+0000 7f2f66d9d640 1 -- 172.21.15.27:0/1380959184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6815c530 con 0x7f2f540047c0 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.346+0000 7f2f66d9d640 1 --2- 172.21.15.27:0/1380959184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f540047c0 0x7f2f6815e110 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f2f580116d0 tx=0x7f2f58011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.346+0000 7f2f5f7fe640 1 -- 172.21.15.27:0/1380959184 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5800f960 con 0x7f2f540047c0 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.347+0000 7f2f5f7fe640 1 -- 172.21.15.27:0/1380959184 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f5800fac0 con 0x7f2f540047c0 2023-12-19T18:28:54.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.347+0000 7f2f5f7fe640 1 -- 172.21.15.27:0/1380959184 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f5800fe30 con 0x7f2f540047c0 2023-12-19T18:28:54.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.347+0000 7f2f6c95b640 1 -- 172.21.15.27:0/1380959184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6815c810 con 0x7f2f540047c0 2023-12-19T18:28:54.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.347+0000 7f2f6c95b640 1 -- 172.21.15.27:0/1380959184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f680af300 con 0x7f2f540047c0 2023-12-19T18:28:54.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.348+0000 7f2f6c95b640 1 -- 172.21.15.27:0/1380959184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6804e840 con 0x7f2f540047c0 2023-12-19T18:28:54.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.349+0000 7f2f5f7fe640 1 -- 172.21.15.27:0/1380959184 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f2f58015460 con 0x7f2f540047c0 2023-12-19T18:28:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.350+0000 7f2f5f7fe640 1 --2- 172.21.15.27:0/1380959184 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2f4806a460 0x7f2f4806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.350+0000 7f2f5f7fe640 1 -- 172.21.15.27:0/1380959184 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f2f58088ef0 con 0x7f2f540047c0 2023-12-19T18:28:54.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.350+0000 7f2f5ffff640 1 --2- 172.21.15.27:0/1380959184 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2f4806a460 0x7f2f4806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:54.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.352+0000 7f2f5ffff640 1 --2- 172.21.15.27:0/1380959184 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2f4806a460 0x7f2f4806c920 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2f50036000 tx=0x7f2f5002e000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:54.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.354+0000 7f2f5f7fe640 1 -- 172.21.15.27:0/1380959184 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f2f58054560 con 0x7f2f540047c0 2023-12-19T18:28:54.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.544+0000 7f2f6c95b640 1 -- 172.21.15.27:0/1380959184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f6806b780 con 0x7f2f540047c0 2023-12-19T18:28:54.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.546+0000 7f2f5f7fe640 1 -- 172.21.15.27:0/1380959184 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v206) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f58057b80 con 0x7f2f540047c0 2023-12-19T18:28:54.546 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:28:54.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.548+0000 7f2f6c95b640 1 -- 172.21.15.27:0/1380959184 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2f4806a460 msgr2=0x7f2f4806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:54.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.548+0000 7f2f6c95b640 1 --2- 172.21.15.27:0/1380959184 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2f4806a460 0x7f2f4806c920 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f2f50036000 tx=0x7f2f5002e000 comp rx=0 tx=0).stop 2023-12-19T18:28:54.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.548+0000 7f2f6c95b640 1 -- 172.21.15.27:0/1380959184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f540047c0 msgr2=0x7f2f6815e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:54.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.548+0000 7f2f6c95b640 1 --2- 172.21.15.27:0/1380959184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f540047c0 0x7f2f6815e110 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7f2f580116d0 tx=0x7f2f58011b90 comp rx=0 tx=0).stop 2023-12-19T18:28:54.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.548+0000 7f2f6c95b640 1 -- 172.21.15.27:0/1380959184 shutdown_connections 2023-12-19T18:28:54.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.548+0000 7f2f6c95b640 1 --2- 172.21.15.27:0/1380959184 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f2f4806a460 0x7f2f4806c920 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.548+0000 7f2f6c95b640 1 --2- 172.21.15.27:0/1380959184 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f2f680ae5a0 0x7f2f680aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.548+0000 7f2f6c95b640 1 --2- 172.21.15.27:0/1380959184 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f2f6815d780 0x7f2f68148670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.548+0000 7f2f6c95b640 1 --2- 172.21.15.27:0/1380959184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f540047c0 0x7f2f6815e110 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.548+0000 7f2f6c95b640 1 -- 172.21.15.27:0/1380959184 >> 172.21.15.27:0/1380959184 conn(0x7f2f6806a060 msgr2=0x7f2f6815ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:54.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.549+0000 7f2f6c95b640 1 -- 172.21.15.27:0/1380959184 shutdown_connections 2023-12-19T18:28:54.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.549+0000 7f2f6c95b640 1 -- 172.21.15.27:0/1380959184 wait complete. 2023-12-19T18:28:54.567 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:28:54.795 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.795+0000 7f1e9ddf0640 1 Processor -- start 2023-12-19T18:28:54.796 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.796+0000 7f1e9ddf0640 1 -- start start 2023-12-19T18:28:54.797 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.797+0000 7f1e9ddf0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1e980d1580 0x7f1e98167e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.797+0000 7f1e9ddf0640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e980b71b0 con 0x7f1e9806ab00 2023-12-19T18:28:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.797+0000 7f1e9ddf0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e980d1a50 con 0x7f1e9815d260 2023-12-19T18:28:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.797+0000 7f1e9ddf0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e980cdaf0 con 0x7f1e980d1580 2023-12-19T18:28:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.798+0000 7f1e97fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1e980d1580 0x7f1e98167e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.798+0000 7f1e97fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f1e980d1580 0x7f1e98167e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32964/0 (socket says 172.21.15.27:32964) 2023-12-19T18:28:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.798+0000 7f1e97fff640 1 -- 172.21.15.27:0/4260818688 learned_addr learned my addr 172.21.15.27:0/4260818688 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:54.798 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.798+0000 7f1e96ffd640 1 --1- 172.21.15.27:0/4260818688 >> v1:172.21.15.27:6789/0 conn(0x7f1e9806ab00 0x7f1e980d0e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55258/0 (socket says 172.21.15.27:55258) 2023-12-19T18:28:54.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.798+0000 7f1e97fff640 1 -- 172.21.15.27:0/4260818688 >> v1:172.21.15.192:6789/0 conn(0x7f1e9815d260 legacy=0x7f1e9806a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:54.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.798+0000 7f1e97fff640 1 -- 172.21.15.27:0/4260818688 >> v1:172.21.15.27:6789/0 conn(0x7f1e9806ab00 legacy=0x7f1e980d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:54.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.798+0000 7f1e97fff640 1 -- 172.21.15.27:0/4260818688 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e980d1d30 con 0x7f1e980d1580 2023-12-19T18:28:54.799 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.800+0000 7f1e97fff640 1 --2- 172.21.15.27:0/4260818688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e980d1580 0x7f1e98167e20 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009e50 tx=0x7f1e9002bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b9154a456540993 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:54.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.800+0000 7f1e967fc640 1 -- 172.21.15.27:0/4260818688 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e9002cb20 con 0x7f1e980d1580 2023-12-19T18:28:54.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.800+0000 7f1e967fc640 1 -- 172.21.15.27:0/4260818688 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e9002cc80 con 0x7f1e980d1580 2023-12-19T18:28:54.800 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.800+0000 7f1e967fc640 1 -- 172.21.15.27:0/4260818688 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e900343f0 con 0x7f1e980d1580 2023-12-19T18:28:54.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.800+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/4260818688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e980d1580 msgr2=0x7f1e98167e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:54.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.800+0000 7f1e9ddf0640 1 --2- 172.21.15.27:0/4260818688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e980d1580 0x7f1e98167e20 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f1e90009e50 tx=0x7f1e9002bf80 comp rx=0 tx=0).stop 2023-12-19T18:28:54.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.801+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/4260818688 shutdown_connections 2023-12-19T18:28:54.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.801+0000 7f1e9ddf0640 1 --2- 172.21.15.27:0/4260818688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e980d1580 0x7f1e98167e20 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.801+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/4260818688 >> 172.21.15.27:0/4260818688 conn(0x7f1e980d2f70 msgr2=0x7f1e980d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:54.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.801+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/4260818688 shutdown_connections 2023-12-19T18:28:54.801 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.801+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/4260818688 wait complete. 2023-12-19T18:28:54.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.802+0000 7f1e9ddf0640 1 Processor -- start 2023-12-19T18:28:54.802 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.803+0000 7f1e9ddf0640 1 -- start start 2023-12-19T18:28:54.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e9ddf0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e9806ab00 0x7f1e9815e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e9ddf0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e980d1580 0x7f1e9815ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e9ddf0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e9815d260 0x7f1e981f1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e9ddf0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980c9fb0 con 0x7f1e980d1580 2023-12-19T18:28:54.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e9ddf0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9815efc0 con 0x7f1e9806ab00 2023-12-19T18:28:54.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e9ddf0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e9815f240 con 0x7f1e9815d260 2023-12-19T18:28:54.804 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e96ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e980d1580 0x7f1e9815ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:54.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e96ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e980d1580 0x7f1e9815ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32976/0 (socket says 172.21.15.27:32976) 2023-12-19T18:28:54.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e96ffd640 1 -- 172.21.15.27:0/2682637041 learned_addr learned my addr 172.21.15.27:0/2682637041 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:54.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e977fe640 1 --2- 172.21.15.27:0/2682637041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e9806ab00 0x7f1e9815e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:54.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e96ffd640 1 -- 172.21.15.27:0/2682637041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e9815d260 msgr2=0x7f1e981f1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:54.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e96ffd640 1 --2- 172.21.15.27:0/2682637041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e9815d260 0x7f1e981f1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e96ffd640 1 -- 172.21.15.27:0/2682637041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e9806ab00 msgr2=0x7f1e9815e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:54.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e96ffd640 1 --2- 172.21.15.27:0/2682637041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e9806ab00 0x7f1e9815e280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:54.805 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.804+0000 7f1e96ffd640 1 -- 172.21.15.27:0/2682637041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e90009b30 con 0x7f1e980d1580 2023-12-19T18:28:54.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.805+0000 7f1e96ffd640 1 --2- 172.21.15.27:0/2682637041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e980d1580 0x7f1e9815ea80 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00c4d0 tx=0x7f1e8c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:54.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.805+0000 7f1e7ffff640 1 -- 172.21.15.27:0/2682637041 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c007480 con 0x7f1e980d1580 2023-12-19T18:28:54.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.805+0000 7f1e7ffff640 1 -- 172.21.15.27:0/2682637041 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8c0075e0 con 0x7f1e980d1580 2023-12-19T18:28:54.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.805+0000 7f1e7ffff640 1 -- 172.21.15.27:0/2682637041 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c0049e0 con 0x7f1e980d1580 2023-12-19T18:28:54.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.805+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/2682637041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e9815f520 con 0x7f1e980d1580 2023-12-19T18:28:54.806 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.805+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/2682637041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e9815fae0 con 0x7f1e980d1580 2023-12-19T18:28:54.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.807+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/2682637041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e98066810 con 0x7f1e980d1580 2023-12-19T18:28:54.807 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.807+0000 7f1e7ffff640 1 -- 172.21.15.27:0/2682637041 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f1e8c004b40 con 0x7f1e980d1580 2023-12-19T18:28:54.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.808+0000 7f1e7ffff640 1 --2- 172.21.15.27:0/2682637041 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1e8406a460 0x7f1e8406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:54.808 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.809+0000 7f1e7ffff640 1 -- 172.21.15.27:0/2682637041 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f1e8c0878e0 con 0x7f1e980d1580 2023-12-19T18:28:54.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.809+0000 7f1e977fe640 1 --2- 172.21.15.27:0/2682637041 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1e8406a460 0x7f1e8406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:54.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.810+0000 7f1e977fe640 1 --2- 172.21.15.27:0/2682637041 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1e8406a460 0x7f1e8406c920 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800a750 tx=0x7f1e88008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:54.812 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:54.813+0000 7f1e7ffff640 1 -- 172.21.15.27:0/2682637041 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f1e8c052eb0 con 0x7f1e980d1580 2023-12-19T18:28:55.024 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.024+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/2682637041 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1e980b71b0 con 0x7f1e980d1580 2023-12-19T18:28:55.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.025+0000 7f1e7ffff640 1 -- 172.21.15.27:0/2682637041 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v206) v1 ==== 80+0+0 (secure 0 0 0) 0x7f1e980b71b0 con 0x7f1e980d1580 2023-12-19T18:28:55.025 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:28:55.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.027+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/2682637041 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1e8406a460 msgr2=0x7f1e8406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:55.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.027+0000 7f1e9ddf0640 1 --2- 172.21.15.27:0/2682637041 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1e8406a460 0x7f1e8406c920 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800a750 tx=0x7f1e88008040 comp rx=0 tx=0).stop 2023-12-19T18:28:55.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.027+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/2682637041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e980d1580 msgr2=0x7f1e9815ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:55.027 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.027+0000 7f1e9ddf0640 1 --2- 172.21.15.27:0/2682637041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e980d1580 0x7f1e9815ea80 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c00c4d0 tx=0x7f1e8c00c990 comp rx=0 tx=0).stop 2023-12-19T18:28:55.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.027+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/2682637041 shutdown_connections 2023-12-19T18:28:55.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.027+0000 7f1e9ddf0640 1 --2- 172.21.15.27:0/2682637041 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f1e8406a460 0x7f1e8406c920 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.027+0000 7f1e9ddf0640 1 --2- 172.21.15.27:0/2682637041 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f1e9815d260 0x7f1e981f1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.027+0000 7f1e9ddf0640 1 --2- 172.21.15.27:0/2682637041 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e980d1580 0x7f1e9815ea80 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.027+0000 7f1e9ddf0640 1 --2- 172.21.15.27:0/2682637041 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f1e9806ab00 0x7f1e9815e280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.027+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/2682637041 >> 172.21.15.27:0/2682637041 conn(0x7f1e980d2f70 msgr2=0x7f1e9815b860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:55.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.027+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/2682637041 shutdown_connections 2023-12-19T18:28:55.028 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.027+0000 7f1e9ddf0640 1 -- 172.21.15.27:0/2682637041 wait complete. 2023-12-19T18:28:55.048 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi027.front.sepia.ceph.com... 2023-12-19T18:28:55.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:55.048 DEBUG:teuthology.orchestra.run.smithi027:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-12-19T18:28:55.144 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi027.stderr:ceph-fuse[122775]: fuse finished with error 0 and tester_r 0 2023-12-19T18:28:55.167 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:28:55.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:55.167 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-19T18:28:55.344 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:28:55.345 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:28:55.401 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:28:55.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.645+0000 7f9b25370640 1 Processor -- start 2023-12-19T18:28:55.646 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.646+0000 7f9b25370640 1 -- start start 2023-12-19T18:28:55.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.647+0000 7f9b25370640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9b20154030 0x7f9b200cd4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.648+0000 7f9b25370640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9b200c5560 0x7f9b200afac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.648+0000 7f9b25370640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b200cc190 con 0x7f9b200af630 2023-12-19T18:28:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.648+0000 7f9b25370640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b200b0000 con 0x7f9b20154030 2023-12-19T18:28:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.648+0000 7f9b25370640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b200cb2a0 con 0x7f9b200c5560 2023-12-19T18:28:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.648+0000 7f9b1ffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9b20154030 0x7f9b200cd4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.648+0000 7f9b1f7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9b200af630 0x7f9b2015d620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55260/0 (socket says 172.21.15.27:55260) 2023-12-19T18:28:55.648 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.648+0000 7f9b1f7fe640 1 -- 172.21.15.27:0/1789938216 learned_addr learned my addr 172.21.15.27:0/1789938216 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.648+0000 7f9b24b6f640 1 --2- 172.21.15.27:0/1789938216 >> v2:172.21.15.192:3300/0 conn(0x7f9b200c5560 0x7f9b200afac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.649+0000 7f9b1ffff640 1 -- 172.21.15.27:0/1789938216 >> v2:172.21.15.192:3300/0 conn(0x7f9b200c5560 msgr2=0x7f9b200afac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.649+0000 7f9b1ffff640 1 --2- 172.21.15.27:0/1789938216 >> v2:172.21.15.192:3300/0 conn(0x7f9b200c5560 0x7f9b200afac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.649+0000 7f9b1ffff640 1 -- 172.21.15.27:0/1789938216 >> v1:172.21.15.27:6789/0 conn(0x7f9b200af630 legacy=0x7f9b2015d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.649+0000 7f9b1ffff640 1 -- 172.21.15.27:0/1789938216 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b200cb580 con 0x7f9b20154030 2023-12-19T18:28:55.649 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.649+0000 7f9b1effd640 1 -- 172.21.15.27:0/1789938216 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1261303455 0 0) 0x7f9b200cc190 con 0x7f9b200af630 2023-12-19T18:28:55.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.650+0000 7f9b1ffff640 1 --2- 172.21.15.27:0/1789938216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b20154030 0x7f9b200cd4f0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f9b100097a0 tx=0x7f9b1002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=688f180bf2accaf7 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:55.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.651+0000 7f9b1effd640 1 -- 172.21.15.27:0/1789938216 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b10031830 con 0x7f9b20154030 2023-12-19T18:28:55.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.651+0000 7f9b1effd640 1 -- 172.21.15.27:0/1789938216 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b10031990 con 0x7f9b20154030 2023-12-19T18:28:55.650 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.651+0000 7f9b1effd640 1 -- 172.21.15.27:0/1789938216 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b10031d20 con 0x7f9b20154030 2023-12-19T18:28:55.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.651+0000 7f9b25370640 1 -- 172.21.15.27:0/1789938216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b20154030 msgr2=0x7f9b200cd4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:55.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.651+0000 7f9b25370640 1 --2- 172.21.15.27:0/1789938216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b20154030 0x7f9b200cd4f0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f9b100097a0 tx=0x7f9b1002b8d0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.651 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.651+0000 7f9b25370640 1 -- 172.21.15.27:0/1789938216 shutdown_connections 2023-12-19T18:28:55.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.651+0000 7f9b25370640 1 --2- 172.21.15.27:0/1789938216 >> v2:172.21.15.192:3300/0 conn(0x7f9b200c5560 0x7f9b200afac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.651+0000 7f9b25370640 1 --2- 172.21.15.27:0/1789938216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b20154030 0x7f9b200cd4f0 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.651+0000 7f9b25370640 1 -- 172.21.15.27:0/1789938216 >> 172.21.15.27:0/1789938216 conn(0x7f9b20069c70 msgr2=0x7f9b2006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:55.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.651+0000 7f9b25370640 1 -- 172.21.15.27:0/1789938216 shutdown_connections 2023-12-19T18:28:55.652 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.651+0000 7f9b25370640 1 -- 172.21.15.27:0/1789938216 wait complete. 2023-12-19T18:28:55.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.653+0000 7f9b25370640 1 Processor -- start 2023-12-19T18:28:55.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.653+0000 7f9b25370640 1 -- start start 2023-12-19T18:28:55.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.654+0000 7f9b25370640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b200af630 0x7f9b200b4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.654+0000 7f9b25370640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b200c5560 0x7f9b2014c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.654+0000 7f9b25370640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b20154030 0x7f9b2006fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.654+0000 7f9b25370640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b200b2120 con 0x7f9b200af630 2023-12-19T18:28:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.654+0000 7f9b25370640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b200b15c0 con 0x7f9b200c5560 2023-12-19T18:28:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.654+0000 7f9b25370640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b200b1880 con 0x7f9b20154030 2023-12-19T18:28:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.654+0000 7f9b1ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b200af630 0x7f9b200b4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.654+0000 7f9b1ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b200af630 0x7f9b200b4020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32990/0 (socket says 172.21.15.27:32990) 2023-12-19T18:28:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.654+0000 7f9b1ffff640 1 -- 172.21.15.27:0/2846001538 learned_addr learned my addr 172.21.15.27:0/2846001538 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.655+0000 7f9b1ffff640 1 -- 172.21.15.27:0/2846001538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b20154030 msgr2=0x7f9b2006fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.655+0000 7f9b1ffff640 1 --2- 172.21.15.27:0/2846001538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b20154030 0x7f9b2006fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.655+0000 7f9b1ffff640 1 -- 172.21.15.27:0/2846001538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b200c5560 msgr2=0x7f9b2014c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.655+0000 7f9b1ffff640 1 --2- 172.21.15.27:0/2846001538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b200c5560 0x7f9b2014c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.655+0000 7f9b1ffff640 1 -- 172.21.15.27:0/2846001538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b10009400 con 0x7f9b200af630 2023-12-19T18:28:55.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.655+0000 7f9b1ffff640 1 --2- 172.21.15.27:0/2846001538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b200af630 0x7f9b200b4020 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f9b100097a0 tx=0x7f9b100046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:55.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.656+0000 7f9b1cff9640 1 -- 172.21.15.27:0/2846001538 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b10004230 con 0x7f9b200af630 2023-12-19T18:28:55.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.656+0000 7f9b1cff9640 1 -- 172.21.15.27:0/2846001538 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b10004390 con 0x7f9b200af630 2023-12-19T18:28:55.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.656+0000 7f9b1cff9640 1 -- 172.21.15.27:0/2846001538 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b1003a870 con 0x7f9b200af630 2023-12-19T18:28:55.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.656+0000 7f9b25370640 1 -- 172.21.15.27:0/2846001538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b200b1b60 con 0x7f9b200af630 2023-12-19T18:28:55.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.656+0000 7f9b25370640 1 -- 172.21.15.27:0/2846001538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b2006f7c0 con 0x7f9b200af630 2023-12-19T18:28:55.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.657+0000 7f9b25370640 1 -- 172.21.15.27:0/2846001538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b2006f600 con 0x7f9b200af630 2023-12-19T18:28:55.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.658+0000 7f9b1cff9640 1 -- 172.21.15.27:0/2846001538 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9b1003a9d0 con 0x7f9b200af630 2023-12-19T18:28:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.659+0000 7f9b1cff9640 1 --2- 172.21.15.27:0/2846001538 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9b0c06a430 0x7f9b0c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.659+0000 7f9b1cff9640 1 -- 172.21.15.27:0/2846001538 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f9b100aab80 con 0x7f9b200af630 2023-12-19T18:28:55.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.660+0000 7f9b1f7fe640 1 --2- 172.21.15.27:0/2846001538 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9b0c06a430 0x7f9b0c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:55.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.661+0000 7f9b1f7fe640 1 --2- 172.21.15.27:0/2846001538 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9b0c06a430 0x7f9b0c06c8f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9b14001560 tx=0x7f9b14007ca0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:55.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.664+0000 7f9b1cff9640 1 -- 172.21.15.27:0/2846001538 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9b10076260 con 0x7f9b200af630 2023-12-19T18:28:55.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.857+0000 7f9b25370640 1 -- 172.21.15.27:0/2846001538 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9b2006f9b0 con 0x7f9b200af630 2023-12-19T18:28:55.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.858+0000 7f9b1cff9640 1 -- 172.21.15.27:0/2846001538 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v206) v1 ==== 74+0+16169 (secure 0 0 0) 0x7f9b10035090 con 0x7f9b200af630 2023-12-19T18:28:55.859 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:55.859 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":206,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:28:46.843951+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:28:06.542489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:28:07.623298+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"206","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.26","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:28:55.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.861+0000 7f9b25370640 1 -- 172.21.15.27:0/2846001538 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9b0c06a430 msgr2=0x7f9b0c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:55.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.861+0000 7f9b25370640 1 --2- 172.21.15.27:0/2846001538 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9b0c06a430 0x7f9b0c06c8f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f9b14001560 tx=0x7f9b14007ca0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.861+0000 7f9b25370640 1 -- 172.21.15.27:0/2846001538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b200af630 msgr2=0x7f9b200b4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:55.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.861+0000 7f9b25370640 1 --2- 172.21.15.27:0/2846001538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b200af630 0x7f9b200b4020 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f9b100097a0 tx=0x7f9b100046c0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.862+0000 7f9b25370640 1 -- 172.21.15.27:0/2846001538 shutdown_connections 2023-12-19T18:28:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.862+0000 7f9b25370640 1 --2- 172.21.15.27:0/2846001538 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9b0c06a430 0x7f9b0c06c8f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.862+0000 7f9b25370640 1 --2- 172.21.15.27:0/2846001538 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9b20154030 0x7f9b2006fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.862+0000 7f9b25370640 1 --2- 172.21.15.27:0/2846001538 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9b200c5560 0x7f9b2014c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.862+0000 7f9b25370640 1 --2- 172.21.15.27:0/2846001538 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b200af630 0x7f9b200b4020 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.862+0000 7f9b25370640 1 -- 172.21.15.27:0/2846001538 >> 172.21.15.27:0/2846001538 conn(0x7f9b20069c70 msgr2=0x7f9b20159e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.862+0000 7f9b25370640 1 -- 172.21.15.27:0/2846001538 shutdown_connections 2023-12-19T18:28:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:55.862+0000 7f9b25370640 1 -- 172.21.15.27:0/2846001538 wait complete. 2023-12-19T18:28:55.884 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:28:06.542489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '201', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:28:07.623298+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '206', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:28:55.884 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:28:56.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.123+0000 7f6e89262640 1 Processor -- start 2023-12-19T18:28:56.123 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.123+0000 7f6e89262640 1 -- start start 2023-12-19T18:28:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.125+0000 7f6e89262640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6e8415a720 0x7f6e840b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.125+0000 7f6e89262640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e840b2b30 con 0x7f6e840b3500 2023-12-19T18:28:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.125+0000 7f6e89262640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e8406b000 con 0x7f6e84159330 2023-12-19T18:28:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.125+0000 7f6e89262640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e8406b2a0 con 0x7f6e8415a720 2023-12-19T18:28:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.125+0000 7f6e837fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f6e8415a720 0x7f6e840b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.125+0000 7f6e88a61640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6e840b3500 0x7f6e8406f320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55268/0 (socket says 172.21.15.27:55268) 2023-12-19T18:28:56.125 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.125+0000 7f6e88a61640 1 -- 172.21.15.27:0/3807656491 learned_addr learned my addr 172.21.15.27:0/3807656491 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:56.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.126+0000 7f6e837fe640 1 -- 172.21.15.27:0/3807656491 >> v1:172.21.15.192:6789/0 conn(0x7f6e84159330 legacy=0x7f6e8415da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:56.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.126+0000 7f6e837fe640 1 -- 172.21.15.27:0/3807656491 >> v1:172.21.15.27:6789/0 conn(0x7f6e840b3500 legacy=0x7f6e8406f320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:56.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.126+0000 7f6e837fe640 1 -- 172.21.15.27:0/3807656491 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e8406b580 con 0x7f6e8415a720 2023-12-19T18:28:56.127 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.127+0000 7f6e837fe640 1 --2- 172.21.15.27:0/3807656491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e8415a720 0x7f6e840b2fc0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0097a0 tx=0x7f6e6c02ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d5177ee29ee235b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:56.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.128+0000 7f6e82ffd640 1 -- 172.21.15.27:0/3807656491 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c02e830 con 0x7f6e8415a720 2023-12-19T18:28:56.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.128+0000 7f6e82ffd640 1 -- 172.21.15.27:0/3807656491 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6c02e990 con 0x7f6e8415a720 2023-12-19T18:28:56.128 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.128+0000 7f6e82ffd640 1 -- 172.21.15.27:0/3807656491 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6c02ed20 con 0x7f6e8415a720 2023-12-19T18:28:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.128+0000 7f6e89262640 1 -- 172.21.15.27:0/3807656491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e8415a720 msgr2=0x7f6e840b2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.128+0000 7f6e89262640 1 --2- 172.21.15.27:0/3807656491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e8415a720 0x7f6e840b2fc0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f6e6c0097a0 tx=0x7f6e6c02ca70 comp rx=0 tx=0).stop 2023-12-19T18:28:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.129+0000 7f6e89262640 1 -- 172.21.15.27:0/3807656491 shutdown_connections 2023-12-19T18:28:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.129+0000 7f6e89262640 1 --2- 172.21.15.27:0/3807656491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e8415a720 0x7f6e840b2fc0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.129+0000 7f6e89262640 1 -- 172.21.15.27:0/3807656491 >> 172.21.15.27:0/3807656491 conn(0x7f6e8406a060 msgr2=0x7f6e8406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.129+0000 7f6e89262640 1 -- 172.21.15.27:0/3807656491 shutdown_connections 2023-12-19T18:28:56.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.129+0000 7f6e89262640 1 -- 172.21.15.27:0/3807656491 wait complete. 2023-12-19T18:28:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.130+0000 7f6e89262640 1 Processor -- start 2023-12-19T18:28:56.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.131+0000 7f6e89262640 1 -- start start 2023-12-19T18:28:56.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.131+0000 7f6e89262640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e840b3500 0x7f6e840adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e89262640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e84159330 0x7f6e840b07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e89262640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e840b0ce0 0x7f6e8414eba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e89262640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840b14d0 con 0x7f6e840b0ce0 2023-12-19T18:28:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e89262640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840b11f0 con 0x7f6e840b3500 2023-12-19T18:28:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e89262640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e840ae8c0 con 0x7f6e84159330 2023-12-19T18:28:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e88a61640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e840b0ce0 0x7f6e8414eba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e88a61640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e840b0ce0 0x7f6e8414eba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33014/0 (socket says 172.21.15.27:33014) 2023-12-19T18:28:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e88a61640 1 -- 172.21.15.27:0/3008253893 learned_addr learned my addr 172.21.15.27:0/3008253893 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e837fe640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e84159330 0x7f6e840b07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:56.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e88a61640 1 -- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e84159330 msgr2=0x7f6e840b07a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:56.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e83fff640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e840b3500 0x7f6e840adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:56.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e88a61640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e84159330 0x7f6e840b07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e88a61640 1 -- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e840b3500 msgr2=0x7f6e840adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:56.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e88a61640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e840b3500 0x7f6e840adcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.132+0000 7f6e88a61640 1 -- 172.21.15.27:0/3008253893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c009400 con 0x7f6e840b0ce0 2023-12-19T18:28:56.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.133+0000 7f6e88a61640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e840b0ce0 0x7f6e8414eba0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00c3c0 tx=0x7f6e7c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:56.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.133+0000 7f6e80ff9640 1 -- 172.21.15.27:0/3008253893 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c005ce0 con 0x7f6e840b0ce0 2023-12-19T18:28:56.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.133+0000 7f6e80ff9640 1 -- 172.21.15.27:0/3008253893 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e7c005e40 con 0x7f6e840b0ce0 2023-12-19T18:28:56.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.133+0000 7f6e80ff9640 1 -- 172.21.15.27:0/3008253893 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e7c0072d0 con 0x7f6e840b0ce0 2023-12-19T18:28:56.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.133+0000 7f6e89262640 1 -- 172.21.15.27:0/3008253893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e840aeba0 con 0x7f6e840b0ce0 2023-12-19T18:28:56.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.133+0000 7f6e89262640 1 -- 172.21.15.27:0/3008253893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e841f6e60 con 0x7f6e840b0ce0 2023-12-19T18:28:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.135+0000 7f6e89262640 1 -- 172.21.15.27:0/3008253893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e840ae720 con 0x7f6e840b0ce0 2023-12-19T18:28:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.135+0000 7f6e80ff9640 1 -- 172.21.15.27:0/3008253893 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6e7c0074f0 con 0x7f6e840b0ce0 2023-12-19T18:28:56.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.136+0000 7f6e80ff9640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6e6406a430 0x7f6e6406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:56.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.137+0000 7f6e80ff9640 1 -- 172.21.15.27:0/3008253893 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f6e7c087db0 con 0x7f6e840b0ce0 2023-12-19T18:28:56.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.137+0000 7f6e83fff640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6e6406a430 0x7f6e6406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:56.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.138+0000 7f6e83fff640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6e6406a430 0x7f6e6406c8f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6e74007fd0 tx=0x7f6e74009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:56.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.141+0000 7f6e80ff9640 1 -- 172.21.15.27:0/3008253893 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6e840ae720 con 0x7f6e840b0ce0 2023-12-19T18:28:56.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.337+0000 7f6e89262640 1 -- 172.21.15.27:0/3008253893 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6e840af0c0 con 0x7f6e840b0ce0 2023-12-19T18:28:56.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.339+0000 7f6e80ff9640 1 -- 172.21.15.27:0/3008253893 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v206) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6e840af0c0 con 0x7f6e840b0ce0 2023-12-19T18:28:56.339 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:28:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.341+0000 7f6e89262640 1 -- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6e6406a430 msgr2=0x7f6e6406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.341+0000 7f6e89262640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6e6406a430 0x7f6e6406c8f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f6e74007fd0 tx=0x7f6e74009040 comp rx=0 tx=0).stop 2023-12-19T18:28:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.341+0000 7f6e89262640 1 -- 172.21.15.27:0/3008253893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e840b0ce0 msgr2=0x7f6e8414eba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.341+0000 7f6e89262640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e840b0ce0 0x7f6e8414eba0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00c3c0 tx=0x7f6e7c00c880 comp rx=0 tx=0).stop 2023-12-19T18:28:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.342+0000 7f6e89262640 1 -- 172.21.15.27:0/3008253893 shutdown_connections 2023-12-19T18:28:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.342+0000 7f6e89262640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6e6406a430 0x7f6e6406c8f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.342+0000 7f6e89262640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e840b0ce0 0x7f6e8414eba0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.342+0000 7f6e89262640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6e84159330 0x7f6e840b07a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.342+0000 7f6e89262640 1 --2- 172.21.15.27:0/3008253893 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6e840b3500 0x7f6e840adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.342+0000 7f6e89262640 1 -- 172.21.15.27:0/3008253893 >> 172.21.15.27:0/3008253893 conn(0x7f6e8406a060 msgr2=0x7f6e84070280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:56.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.342+0000 7f6e89262640 1 -- 172.21.15.27:0/3008253893 shutdown_connections 2023-12-19T18:28:56.343 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.342+0000 7f6e89262640 1 -- 172.21.15.27:0/3008253893 wait complete. 2023-12-19T18:28:56.362 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:28:56.593 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.594+0000 7f7a119ca640 1 Processor -- start 2023-12-19T18:28:56.594 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.594+0000 7f7a119ca640 1 -- start start 2023-12-19T18:28:56.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.595+0000 7f7a119ca640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a0c0d1a20 0x7f7a0c0cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:56.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.595+0000 7f7a119ca640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7a0c0b3980 0x7f7a0c0cd940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:56.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.595+0000 7f7a119ca640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a0c0d3850 con 0x7f7a0c15d120 2023-12-19T18:28:56.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.595+0000 7f7a119ca640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a0c0d0b10 con 0x7f7a0c0b3980 2023-12-19T18:28:56.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.595+0000 7f7a119ca640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c15d500 con 0x7f7a0c0d1a20 2023-12-19T18:28:56.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.595+0000 7f7a111c9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f7a0c0b3980 0x7f7a0c0cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:56.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.595+0000 7f7a109c8640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f7a0c15d120 0x7f7a0c0d1310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55282/0 (socket says 172.21.15.27:55282) 2023-12-19T18:28:56.595 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.595+0000 7f7a109c8640 1 -- 172.21.15.27:0/2973246784 learned_addr learned my addr 172.21.15.27:0/2973246784 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:56.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.596+0000 7f7a0bfff640 1 --2- 172.21.15.27:0/2973246784 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a0c0d1a20 0x7f7a0c0cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:56.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.596+0000 7f7a111c9640 1 -- 172.21.15.27:0/2973246784 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a0c0d1a20 msgr2=0x7f7a0c0cfd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:56.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.596+0000 7f7a111c9640 1 --2- 172.21.15.27:0/2973246784 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a0c0d1a20 0x7f7a0c0cfd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.596 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.596+0000 7f7a111c9640 1 -- 172.21.15.27:0/2973246784 >> v1:172.21.15.27:6789/0 conn(0x7f7a0c15d120 legacy=0x7f7a0c0d1310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:56.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.596+0000 7f7a111c9640 1 -- 172.21.15.27:0/2973246784 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a0c0d1040 con 0x7f7a0c0b3980 2023-12-19T18:28:56.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.596+0000 7f7a0bfff640 1 --2- 172.21.15.27:0/2973246784 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a0c0d1a20 0x7f7a0c0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:56.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.598+0000 7f7a111c9640 1 --2- 172.21.15.27:0/2973246784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a0c0b3980 0x7f7a0c0cd940 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f7a040097a0 tx=0x7f7a0402b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a560e60c6710bc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:56.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.598+0000 7f7a0b7fe640 1 -- 172.21.15.27:0/2973246784 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0402d040 con 0x7f7a0c0b3980 2023-12-19T18:28:56.597 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.598+0000 7f7a0b7fe640 1 -- 172.21.15.27:0/2973246784 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a04031810 con 0x7f7a0c0b3980 2023-12-19T18:28:56.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.598+0000 7f7a0b7fe640 1 -- 172.21.15.27:0/2973246784 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04031ba0 con 0x7f7a0c0b3980 2023-12-19T18:28:56.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.598+0000 7f7a119ca640 1 -- 172.21.15.27:0/2973246784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a0c0b3980 msgr2=0x7f7a0c0cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:56.598 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.598+0000 7f7a119ca640 1 --2- 172.21.15.27:0/2973246784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a0c0b3980 0x7f7a0c0cd940 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f7a040097a0 tx=0x7f7a0402b8b0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.598+0000 7f7a119ca640 1 -- 172.21.15.27:0/2973246784 shutdown_connections 2023-12-19T18:28:56.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.599+0000 7f7a119ca640 1 --2- 172.21.15.27:0/2973246784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a0c0b3980 0x7f7a0c0cd940 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.599+0000 7f7a119ca640 1 --2- 172.21.15.27:0/2973246784 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a0c0d1a20 0x7f7a0c0cfd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.599+0000 7f7a119ca640 1 -- 172.21.15.27:0/2973246784 >> 172.21.15.27:0/2973246784 conn(0x7f7a0c0d2d30 msgr2=0x7f7a0c0d3140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:56.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.599+0000 7f7a119ca640 1 -- 172.21.15.27:0/2973246784 shutdown_connections 2023-12-19T18:28:56.599 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.599+0000 7f7a119ca640 1 -- 172.21.15.27:0/2973246784 wait complete. 2023-12-19T18:28:56.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.600+0000 7f7a119ca640 1 Processor -- start 2023-12-19T18:28:56.600 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.601+0000 7f7a119ca640 1 -- start start 2023-12-19T18:28:56.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.602+0000 7f7a119ca640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a0c0b3980 0x7f7a0c0b6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:56.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.602+0000 7f7a119ca640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a0c0d1a20 0x7f7a0c15d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:56.602 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.602+0000 7f7a119ca640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a0c15d120 0x7f7a0c1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.602+0000 7f7a119ca640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c0b8bd0 con 0x7f7a0c15d120 2023-12-19T18:28:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.602+0000 7f7a119ca640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c15e6b0 con 0x7f7a0c0b3980 2023-12-19T18:28:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.602+0000 7f7a119ca640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a0c1516f0 con 0x7f7a0c0d1a20 2023-12-19T18:28:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.602+0000 7f7a111c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a0c15d120 0x7f7a0c1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.602+0000 7f7a111c9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a0c15d120 0x7f7a0c1511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33026/0 (socket says 172.21.15.27:33026) 2023-12-19T18:28:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.602+0000 7f7a111c9640 1 -- 172.21.15.27:0/4129289791 learned_addr learned my addr 172.21.15.27:0/4129289791 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.602+0000 7f7a109c8640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a0c0b3980 0x7f7a0c0b6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.602+0000 7f7a0bfff640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a0c0d1a20 0x7f7a0c15d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.603+0000 7f7a111c9640 1 -- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a0c0d1a20 msgr2=0x7f7a0c15d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.603+0000 7f7a111c9640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a0c0d1a20 0x7f7a0c15d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.603+0000 7f7a111c9640 1 -- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a0c0b3980 msgr2=0x7f7a0c0b6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:56.603 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.603+0000 7f7a111c9640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a0c0b3980 0x7f7a0c0b6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.603+0000 7f7a111c9640 1 -- 172.21.15.27:0/4129289791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a04009400 con 0x7f7a0c15d120 2023-12-19T18:28:56.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.603+0000 7f7a111c9640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a0c15d120 0x7f7a0c1511b0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f7a0402b8f0 tx=0x7f7a040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:56.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.604+0000 7f7a097fa640 1 -- 172.21.15.27:0/4129289791 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a0403ace0 con 0x7f7a0c15d120 2023-12-19T18:28:56.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.604+0000 7f7a097fa640 1 -- 172.21.15.27:0/4129289791 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a0403ae60 con 0x7f7a0c15d120 2023-12-19T18:28:56.604 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.604+0000 7f7a097fa640 1 -- 172.21.15.27:0/4129289791 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a04044630 con 0x7f7a0c15d120 2023-12-19T18:28:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.604+0000 7f7a119ca640 1 -- 172.21.15.27:0/4129289791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a0c15de10 con 0x7f7a0c15d120 2023-12-19T18:28:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.604+0000 7f7a119ca640 1 -- 172.21.15.27:0/4129289791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a0c15e3d0 con 0x7f7a0c15d120 2023-12-19T18:28:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.605+0000 7f7a119ca640 1 -- 172.21.15.27:0/4129289791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a0c04e840 con 0x7f7a0c15d120 2023-12-19T18:28:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.606+0000 7f7a097fa640 1 -- 172.21.15.27:0/4129289791 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f7a0404d450 con 0x7f7a0c15d120 2023-12-19T18:28:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.607+0000 7f7a097fa640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79ec06a430 0x7f79ec06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:56.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.608+0000 7f7a097fa640 1 -- 172.21.15.27:0/4129289791 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7a040aac50 con 0x7f7a0c15d120 2023-12-19T18:28:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.608+0000 7f7a109c8640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79ec06a430 0x7f79ec06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.609+0000 7f7a109c8640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79ec06a430 0x7f79ec06c8f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f79fc005c00 tx=0x7f79fc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.612+0000 7f7a097fa640 1 -- 172.21.15.27:0/4129289791 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7a04076210 con 0x7f7a0c15d120 2023-12-19T18:28:56.809 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.809+0000 7f7a119ca640 1 -- 172.21.15.27:0/4129289791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7a0c0d0030 con 0x7f7a0c15d120 2023-12-19T18:28:56.810 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.810+0000 7f7a097fa640 1 -- 172.21.15.27:0/4129289791 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v206) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7a04079830 con 0x7f7a0c15d120 2023-12-19T18:28:56.810 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:28:56.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.813+0000 7f7a119ca640 1 -- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79ec06a430 msgr2=0x7f79ec06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:56.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.813+0000 7f7a119ca640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79ec06a430 0x7f79ec06c8f0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f79fc005c00 tx=0x7f79fc009040 comp rx=0 tx=0).stop 2023-12-19T18:28:56.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.813+0000 7f7a119ca640 1 -- 172.21.15.27:0/4129289791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a0c15d120 msgr2=0x7f7a0c1511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:56.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.813+0000 7f7a119ca640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a0c15d120 0x7f7a0c1511b0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f7a0402b8f0 tx=0x7f7a040046c0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.813+0000 7f7a119ca640 1 -- 172.21.15.27:0/4129289791 shutdown_connections 2023-12-19T18:28:56.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.813+0000 7f7a119ca640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f79ec06a430 0x7f79ec06c8f0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.813+0000 7f7a119ca640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a0c15d120 0x7f7a0c1511b0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.813 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.813+0000 7f7a119ca640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f7a0c0d1a20 0x7f7a0c15d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.813+0000 7f7a119ca640 1 --2- 172.21.15.27:0/4129289791 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f7a0c0b3980 0x7f7a0c0b6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:56.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.813+0000 7f7a119ca640 1 -- 172.21.15.27:0/4129289791 >> 172.21.15.27:0/4129289791 conn(0x7f7a0c0d2d30 msgr2=0x7f7a0c06a400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:56.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.813+0000 7f7a119ca640 1 -- 172.21.15.27:0/4129289791 shutdown_connections 2023-12-19T18:28:56.814 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:56.813+0000 7f7a119ca640 1 -- 172.21.15.27:0/4129289791 wait complete. 2023-12-19T18:28:56.831 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:28:57.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.073+0000 7f78452df640 1 Processor -- start 2023-12-19T18:28:57.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.073+0000 7f78452df640 1 -- start start 2023-12-19T18:28:57.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.075+0000 7f78452df640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f78400d2100 0x7f78400cda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:57.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.075+0000 7f78452df640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f78400cdf80 0x7f78400b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:57.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.075+0000 7f78452df640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78400d0e10 con 0x7f784015cf80 2023-12-19T18:28:57.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.075+0000 7f78452df640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78400b8cf0 con 0x7f78400cdf80 2023-12-19T18:28:57.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.075+0000 7f78452df640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78400ce3f0 con 0x7f78400d2100 2023-12-19T18:28:57.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.075+0000 7f7844ade640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f78400cdf80 0x7f78400b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:57.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.075+0000 7f7844ade640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f78400cdf80 0x7f78400b5d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33028/0 (socket says 172.21.15.27:33028) 2023-12-19T18:28:57.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.075+0000 7f7844ade640 1 -- 172.21.15.27:0/2831352367 learned_addr learned my addr 172.21.15.27:0/2831352367 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:57.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.075+0000 7f783ffff640 1 --1- 172.21.15.27:0/2831352367 >> v1:172.21.15.27:6789/0 conn(0x7f784015cf80 0x7f78400b85e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55286/0 (socket says 172.21.15.27:55286) 2023-12-19T18:28:57.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.076+0000 7f783f7fe640 1 --2- 172.21.15.27:0/2831352367 >> v2:172.21.15.192:3300/0 conn(0x7f78400d2100 0x7f78400cda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.076+0000 7f7844ade640 1 -- 172.21.15.27:0/2831352367 >> v2:172.21.15.192:3300/0 conn(0x7f78400d2100 msgr2=0x7f78400cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.076+0000 7f7844ade640 1 --2- 172.21.15.27:0/2831352367 >> v2:172.21.15.192:3300/0 conn(0x7f78400d2100 0x7f78400cda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.076+0000 7f7844ade640 1 -- 172.21.15.27:0/2831352367 >> v1:172.21.15.27:6789/0 conn(0x7f784015cf80 legacy=0x7f78400b85e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.076+0000 7f7844ade640 1 -- 172.21.15.27:0/2831352367 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78400d1c70 con 0x7f78400cdf80 2023-12-19T18:28:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.076+0000 7f783f7fe640 1 --2- 172.21.15.27:0/2831352367 >> v2:172.21.15.192:3300/0 conn(0x7f78400d2100 0x7f78400cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:28:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.076+0000 7f783effd640 1 -- 172.21.15.27:0/2831352367 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1923147785 0 0) 0x7f78400d0e10 con 0x7f784015cf80 2023-12-19T18:28:57.077 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.077+0000 7f7844ade640 1 --2- 172.21.15.27:0/2831352367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78400cdf80 0x7f78400b5d60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f78400cd270 tx=0x7f783802c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35765014ff66035 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.078+0000 7f783effd640 1 -- 172.21.15.27:0/2831352367 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783802cbb0 con 0x7f78400cdf80 2023-12-19T18:28:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.078+0000 7f783effd640 1 -- 172.21.15.27:0/2831352367 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f783802cd10 con 0x7f78400cdf80 2023-12-19T18:28:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.078+0000 7f783effd640 1 -- 172.21.15.27:0/2831352367 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7838034610 con 0x7f78400cdf80 2023-12-19T18:28:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.078+0000 7f78452df640 1 -- 172.21.15.27:0/2831352367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78400cdf80 msgr2=0x7f78400b5d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.078+0000 7f78452df640 1 --2- 172.21.15.27:0/2831352367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78400cdf80 0x7f78400b5d60 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f78400cd270 tx=0x7f783802c010 comp rx=0 tx=0).stop 2023-12-19T18:28:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.078+0000 7f78452df640 1 -- 172.21.15.27:0/2831352367 shutdown_connections 2023-12-19T18:28:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.078+0000 7f78452df640 1 --2- 172.21.15.27:0/2831352367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78400cdf80 0x7f78400b5d60 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.078+0000 7f78452df640 1 --2- 172.21.15.27:0/2831352367 >> v2:172.21.15.192:3300/0 conn(0x7f78400d2100 0x7f78400cda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.078+0000 7f78452df640 1 -- 172.21.15.27:0/2831352367 >> 172.21.15.27:0/2831352367 conn(0x7f7840072e20 msgr2=0x7f7840073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.079+0000 7f78452df640 1 -- 172.21.15.27:0/2831352367 shutdown_connections 2023-12-19T18:28:57.079 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.079+0000 7f78452df640 1 -- 172.21.15.27:0/2831352367 wait complete. 2023-12-19T18:28:57.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.080+0000 7f78452df640 1 Processor -- start 2023-12-19T18:28:57.080 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.081+0000 7f78452df640 1 -- start start 2023-12-19T18:28:57.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f78452df640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78400cdf80 0x7f784015fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:57.082 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f78452df640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78400d2100 0x7f784015a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f78452df640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f784015cf80 0x7f784015b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f78452df640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7840160370 con 0x7f78400d2100 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f78452df640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78400cadc0 con 0x7f784015cf80 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f78452df640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78400cb840 con 0x7f78400cdf80 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f783f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78400d2100 0x7f784015a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f783f7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78400d2100 0x7f784015a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33036/0 (socket says 172.21.15.27:33036) 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f783f7fe640 1 -- 172.21.15.27:0/4206172601 learned_addr learned my addr 172.21.15.27:0/4206172601 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f7844ade640 1 --2- 172.21.15.27:0/4206172601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f784015cf80 0x7f784015b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f783f7fe640 1 -- 172.21.15.27:0/4206172601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78400cdf80 msgr2=0x7f784015fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f783f7fe640 1 --2- 172.21.15.27:0/4206172601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78400cdf80 0x7f784015fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f783f7fe640 1 -- 172.21.15.27:0/4206172601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f784015cf80 msgr2=0x7f784015b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f783f7fe640 1 --2- 172.21.15.27:0/4206172601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f784015cf80 0x7f784015b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.082+0000 7f783f7fe640 1 -- 172.21.15.27:0/4206172601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7838009b30 con 0x7f78400d2100 2023-12-19T18:28:57.083 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.083+0000 7f783f7fe640 1 --2- 172.21.15.27:0/4206172601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78400d2100 0x7f784015a2f0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f783400e730 tx=0x7f783400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.083+0000 7f783cff9640 1 -- 172.21.15.27:0/4206172601 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7834004230 con 0x7f78400d2100 2023-12-19T18:28:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.083+0000 7f783cff9640 1 -- 172.21.15.27:0/4206172601 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7834004390 con 0x7f78400d2100 2023-12-19T18:28:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.083+0000 7f783cff9640 1 -- 172.21.15.27:0/4206172601 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f783400c8f0 con 0x7f78400d2100 2023-12-19T18:28:57.084 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.083+0000 7f78452df640 1 -- 172.21.15.27:0/4206172601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78400ca1b0 con 0x7f78400d2100 2023-12-19T18:28:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.083+0000 7f78452df640 1 -- 172.21.15.27:0/4206172601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78400ca770 con 0x7f78400d2100 2023-12-19T18:28:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.085+0000 7f78452df640 1 -- 172.21.15.27:0/4206172601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f784004e840 con 0x7f78400d2100 2023-12-19T18:28:57.085 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.086+0000 7f783cff9640 1 -- 172.21.15.27:0/4206172601 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f783400ca50 con 0x7f78400d2100 2023-12-19T18:28:57.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.086+0000 7f783cff9640 1 --2- 172.21.15.27:0/4206172601 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f782c06a430 0x7f782c06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:57.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.087+0000 7f783cff9640 1 -- 172.21.15.27:0/4206172601 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f7834088620 con 0x7f78400d2100 2023-12-19T18:28:57.086 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.087+0000 7f783cff9640 1 -- 172.21.15.27:0/4206172601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=208}) v3 -- 0x7f782c071fc0 con 0x7f78400d2100 2023-12-19T18:28:57.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.087+0000 7f783ffff640 1 --2- 172.21.15.27:0/4206172601 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f782c06a430 0x7f782c06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:57.088 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.088+0000 7f783ffff640 1 --2- 172.21.15.27:0/4206172601 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f782c06a430 0x7f782c06c8f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7830007fd0 tx=0x7f7830007960 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:57.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.091+0000 7f783cff9640 1 -- 172.21.15.27:0/4206172601 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f7834053b50 con 0x7f78400d2100 2023-12-19T18:28:57.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.283+0000 7f78452df640 1 -- 172.21.15.27:0/4206172601 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78400caf00 con 0x7f78400d2100 2023-12-19T18:28:57.283 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.284+0000 7f783cff9640 1 -- 172.21.15.27:0/4206172601 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v207) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78400caf00 con 0x7f78400d2100 2023-12-19T18:28:57.284 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:28:57.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.286+0000 7f78452df640 1 -- 172.21.15.27:0/4206172601 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f782c06a430 msgr2=0x7f782c06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:57.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.286+0000 7f78452df640 1 --2- 172.21.15.27:0/4206172601 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f782c06a430 0x7f782c06c8f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f7830007fd0 tx=0x7f7830007960 comp rx=0 tx=0).stop 2023-12-19T18:28:57.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.286+0000 7f78452df640 1 -- 172.21.15.27:0/4206172601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78400d2100 msgr2=0x7f784015a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:57.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.286+0000 7f78452df640 1 --2- 172.21.15.27:0/4206172601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78400d2100 0x7f784015a2f0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f783400e730 tx=0x7f783400ebf0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.287+0000 7f78452df640 1 -- 172.21.15.27:0/4206172601 shutdown_connections 2023-12-19T18:28:57.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.287+0000 7f78452df640 1 --2- 172.21.15.27:0/4206172601 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f782c06a430 0x7f782c06c8f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.287+0000 7f78452df640 1 --2- 172.21.15.27:0/4206172601 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f784015cf80 0x7f784015b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.287+0000 7f78452df640 1 --2- 172.21.15.27:0/4206172601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78400d2100 0x7f784015a2f0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.287+0000 7f78452df640 1 --2- 172.21.15.27:0/4206172601 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f78400cdf80 0x7f784015fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.287+0000 7f78452df640 1 -- 172.21.15.27:0/4206172601 >> 172.21.15.27:0/4206172601 conn(0x7f7840072e20 msgr2=0x7f78400b3810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:57.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.287+0000 7f78452df640 1 -- 172.21.15.27:0/4206172601 shutdown_connections 2023-12-19T18:28:57.287 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.287+0000 7f78452df640 1 -- 172.21.15.27:0/4206172601 wait complete. 2023-12-19T18:28:57.308 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-19T18:28:57.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.543+0000 7f79ac9ef640 1 Processor -- start 2023-12-19T18:28:57.543 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.543+0000 7f79ac9ef640 1 -- start start 2023-12-19T18:28:57.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.545+0000 7f79ac9ef640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f79a815cf90 0x7f79a8167c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:57.544 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.545+0000 7f79ac9ef640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a80b5ca0 con 0x7f79a80b89b0 2023-12-19T18:28:57.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.545+0000 7f79ac9ef640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79a8161800 con 0x7f79a80b49d0 2023-12-19T18:28:57.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.545+0000 7f79ac9ef640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79a8161050 con 0x7f79a815cf90 2023-12-19T18:28:57.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.545+0000 7f79a659c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f79a80b89b0 0x7f79a815c880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55290/0 (socket says 172.21.15.27:55290) 2023-12-19T18:28:57.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.545+0000 7f79a659c640 1 -- 172.21.15.27:0/752431544 learned_addr learned my addr 172.21.15.27:0/752431544 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:57.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.545+0000 7f79a759e640 1 --2- 172.21.15.27:0/752431544 >> v2:172.21.15.27:3300/0 conn(0x7f79a815cf90 0x7f79a8167c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:57.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.545+0000 7f79a759e640 1 -- 172.21.15.27:0/752431544 >> v1:172.21.15.192:6789/0 conn(0x7f79a80b49d0 legacy=0x7f79a8160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:57.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.545+0000 7f79a759e640 1 -- 172.21.15.27:0/752431544 >> v1:172.21.15.27:6789/0 conn(0x7f79a80b89b0 legacy=0x7f79a815c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:57.545 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.545+0000 7f79a759e640 1 -- 172.21.15.27:0/752431544 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a8161240 con 0x7f79a815cf90 2023-12-19T18:28:57.546 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.547+0000 7f79a759e640 1 --2- 172.21.15.27:0/752431544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a815cf90 0x7f79a8167c70 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f79a00097a0 tx=0x7f79a002b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d8f69644e75279 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:57.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.547+0000 7f79a5d9b640 1 -- 172.21.15.27:0/752431544 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0030610 con 0x7f79a815cf90 2023-12-19T18:28:57.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.547+0000 7f79a5d9b640 1 -- 172.21.15.27:0/752431544 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a0030770 con 0x7f79a815cf90 2023-12-19T18:28:57.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.547+0000 7f79a5d9b640 1 -- 172.21.15.27:0/752431544 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0030b00 con 0x7f79a815cf90 2023-12-19T18:28:57.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.548+0000 7f79ac9ef640 1 -- 172.21.15.27:0/752431544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a815cf90 msgr2=0x7f79a8167c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:57.547 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.548+0000 7f79ac9ef640 1 --2- 172.21.15.27:0/752431544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a815cf90 0x7f79a8167c70 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f79a00097a0 tx=0x7f79a002b960 comp rx=0 tx=0).stop 2023-12-19T18:28:57.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.548+0000 7f79ac9ef640 1 -- 172.21.15.27:0/752431544 shutdown_connections 2023-12-19T18:28:57.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.548+0000 7f79ac9ef640 1 --2- 172.21.15.27:0/752431544 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a815cf90 0x7f79a8167c70 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.548+0000 7f79ac9ef640 1 -- 172.21.15.27:0/752431544 >> 172.21.15.27:0/752431544 conn(0x7f79a80cf760 msgr2=0x7f79a80cfb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:57.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.548+0000 7f79ac9ef640 1 -- 172.21.15.27:0/752431544 shutdown_connections 2023-12-19T18:28:57.548 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.548+0000 7f79ac9ef640 1 -- 172.21.15.27:0/752431544 wait complete. 2023-12-19T18:28:57.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.550+0000 7f79ac9ef640 1 Processor -- start 2023-12-19T18:28:57.549 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.550+0000 7f79ac9ef640 1 -- start start 2023-12-19T18:28:57.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.551+0000 7f79ac9ef640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79a80b49d0 0x7f79a815d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:57.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.551+0000 7f79ac9ef640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a80b89b0 0x7f79a80c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:57.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.551+0000 7f79ac9ef640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a815cf90 0x7f79a8153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:57.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.551+0000 7f79ac9ef640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a80c92e0 con 0x7f79a815cf90 2023-12-19T18:28:57.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.551+0000 7f79ac9ef640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a80c95c0 con 0x7f79a80b89b0 2023-12-19T18:28:57.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.551+0000 7f79ac9ef640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79a8157c10 con 0x7f79a80b49d0 2023-12-19T18:28:57.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.551+0000 7f79a759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a815cf90 0x7f79a8153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:57.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.551+0000 7f79a759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a815cf90 0x7f79a8153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33044/0 (socket says 172.21.15.27:33044) 2023-12-19T18:28:57.551 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.551+0000 7f79a759e640 1 -- 172.21.15.27:0/3699288998 learned_addr learned my addr 172.21.15.27:0/3699288998 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:57.552 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.552+0000 7f79a6d9d640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79a80b49d0 0x7f79a815d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.552+0000 7f79a659c640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a80b89b0 0x7f79a80c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.552+0000 7f79a759e640 1 -- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79a80b49d0 msgr2=0x7f79a815d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.552+0000 7f79a759e640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79a80b49d0 0x7f79a815d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.552+0000 7f79a759e640 1 -- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a80b89b0 msgr2=0x7f79a80c7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.552+0000 7f79a759e640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a80b89b0 0x7f79a80c7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.552+0000 7f79a759e640 1 -- 172.21.15.27:0/3699288998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a0009400 con 0x7f79a815cf90 2023-12-19T18:28:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.552+0000 7f79a6d9d640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79a80b49d0 0x7f79a815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:28:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.552+0000 7f79a759e640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a815cf90 0x7f79a8153270 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f79a00098c0 tx=0x7f79a00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.553+0000 7f798f7fe640 1 -- 172.21.15.27:0/3699288998 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a003e070 con 0x7f79a815cf90 2023-12-19T18:28:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.553+0000 7f798f7fe640 1 -- 172.21.15.27:0/3699288998 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a0039990 con 0x7f79a815cf90 2023-12-19T18:28:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.553+0000 7f798f7fe640 1 -- 172.21.15.27:0/3699288998 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0039d40 con 0x7f79a815cf90 2023-12-19T18:28:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.553+0000 7f79ac9ef640 1 -- 172.21.15.27:0/3699288998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a8157ef0 con 0x7f79a815cf90 2023-12-19T18:28:57.553 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.553+0000 7f79ac9ef640 1 -- 172.21.15.27:0/3699288998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a815d940 con 0x7f79a815cf90 2023-12-19T18:28:57.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.554+0000 7f79ac9ef640 1 -- 172.21.15.27:0/3699288998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79a8158410 con 0x7f79a815cf90 2023-12-19T18:28:57.554 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.555+0000 7f798f7fe640 1 -- 172.21.15.27:0/3699288998 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f79a00423f0 con 0x7f79a815cf90 2023-12-19T18:28:57.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.556+0000 7f798f7fe640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f799406a430 0x7f799406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:57.555 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.556+0000 7f798f7fe640 1 -- 172.21.15.27:0/3699288998 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f79a00aa8b0 con 0x7f79a815cf90 2023-12-19T18:28:57.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.556+0000 7f798f7fe640 1 -- 172.21.15.27:0/3699288998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=208}) v3 -- 0x7f7994071fc0 con 0x7f79a815cf90 2023-12-19T18:28:57.556 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.556+0000 7f79a6d9d640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f799406a430 0x7f799406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:57.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.557+0000 7f79a6d9d640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f799406a430 0x7f799406c8f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f79980045e0 tx=0x7f7998009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:57.560 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.560+0000 7f798f7fe640 1 -- 172.21.15.27:0/3699288998 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f79a8158410 con 0x7f79a815cf90 2023-12-19T18:28:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.771+0000 7f79ac9ef640 1 -- 172.21.15.27:0/3699288998 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f79a8158410 con 0x7f79a815cf90 2023-12-19T18:28:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.772+0000 7f798f7fe640 1 -- 172.21.15.27:0/3699288998 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v207) v1 ==== 80+0+0 (secure 0 0 0) 0x7f79a8158410 con 0x7f79a815cf90 2023-12-19T18:28:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:listed 0 entries 2023-12-19T18:28:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.774+0000 7f79ac9ef640 1 -- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f799406a430 msgr2=0x7f799406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:57.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.774+0000 7f79ac9ef640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f799406a430 0x7f799406c8f0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f79980045e0 tx=0x7f7998009040 comp rx=0 tx=0).stop 2023-12-19T18:28:57.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.774+0000 7f79ac9ef640 1 -- 172.21.15.27:0/3699288998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a815cf90 msgr2=0x7f79a8153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:57.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.774+0000 7f79ac9ef640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a815cf90 0x7f79a8153270 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f79a00098c0 tx=0x7f79a00046c0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.775+0000 7f79ac9ef640 1 -- 172.21.15.27:0/3699288998 shutdown_connections 2023-12-19T18:28:57.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.775+0000 7f79ac9ef640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f799406a430 0x7f799406c8f0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.775+0000 7f79ac9ef640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79a815cf90 0x7f79a8153270 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.775+0000 7f79ac9ef640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f79a80b89b0 0x7f79a80c7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.775+0000 7f79ac9ef640 1 --2- 172.21.15.27:0/3699288998 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f79a80b49d0 0x7f79a815d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:57.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.775+0000 7f79ac9ef640 1 -- 172.21.15.27:0/3699288998 >> 172.21.15.27:0/3699288998 conn(0x7f79a80cf760 msgr2=0x7f79a80b4fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:57.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.775+0000 7f79ac9ef640 1 -- 172.21.15.27:0/3699288998 shutdown_connections 2023-12-19T18:28:57.775 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:57.775+0000 7f79ac9ef640 1 -- 172.21.15.27:0/3699288998 wait complete. 2023-12-19T18:28:57.796 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi192.front.sepia.ceph.com... 2023-12-19T18:28:57.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:57.796 DEBUG:teuthology.orchestra.run.smithi192:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-12-19T18:28:57.899 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi192.stderr:ceph-fuse[92745]: fuse finished with error 0 and tester_r 0 2023-12-19T18:28:57.905 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:28:57.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:57.905 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-19T18:28:58.017 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:28:58.017 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:28:58.081 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:28:58.082 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:28:58.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:58.082 DEBUG:teuthology.orchestra.run.smithi027:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-19T18:28:58.139 INFO:teuthology.orchestra.run.smithi027.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-12-19T18:28:58.140 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:28:58.140 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:28:58.140 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:28:58.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:28:58.199 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:28:58.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:28:58.199 DEBUG:teuthology.orchestra.run.smithi192:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-19T18:28:58.250 INFO:teuthology.orchestra.run.smithi192.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-12-19T18:28:58.250 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:28:58.251 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:28:58.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.492+0000 7fb8d7fff640 1 Processor -- start 2023-12-19T18:28:58.492 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.493+0000 7fb8d7fff640 1 -- start start 2023-12-19T18:28:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.495+0000 7fb8d7fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb8d81553a0 0x7fb8d80c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.495+0000 7fb8d7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d815ee30 0x7fb8d80c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.495+0000 7fb8d7fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8d80b0360 con 0x7fb8d80afeb0 2023-12-19T18:28:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.495+0000 7fb8d7fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8d80cbcf0 con 0x7fb8d81553a0 2023-12-19T18:28:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.495+0000 7fb8d7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d80cb350 con 0x7fb8d815ee30 2023-12-19T18:28:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.495+0000 7fb8d6ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb8d81553a0 0x7fb8d80c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.495+0000 7fb8cffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb8d80afeb0 0x7fb8d815e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55298/0 (socket says 172.21.15.27:55298) 2023-12-19T18:28:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.495+0000 7fb8cffff640 1 -- 172.21.15.27:0/1631179463 learned_addr learned my addr 172.21.15.27:0/1631179463 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:58.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.496+0000 7fb8d77fe640 1 --2- 172.21.15.27:0/1631179463 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d815ee30 0x7fb8d80c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:58.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.496+0000 7fb8d6ffd640 1 -- 172.21.15.27:0/1631179463 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d815ee30 msgr2=0x7fb8d80c39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:58.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.496+0000 7fb8d6ffd640 1 --2- 172.21.15.27:0/1631179463 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d815ee30 0x7fb8d80c39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:58.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.496+0000 7fb8d6ffd640 1 -- 172.21.15.27:0/1631179463 >> v1:172.21.15.27:6789/0 conn(0x7fb8d80afeb0 legacy=0x7fb8d815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:58.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.496+0000 7fb8d6ffd640 1 -- 172.21.15.27:0/1631179463 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8d80cb630 con 0x7fb8d81553a0 2023-12-19T18:28:58.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.496+0000 7fb8d77fe640 1 --2- 172.21.15.27:0/1631179463 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d815ee30 0x7fb8d80c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:58.496 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.496+0000 7fb8d67fc640 1 -- 172.21.15.27:0/1631179463 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050557514 0 0) 0x7fb8d80b0360 con 0x7fb8d80afeb0 2023-12-19T18:28:58.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.497+0000 7fb8d6ffd640 1 --2- 172.21.15.27:0/1631179463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8d81553a0 0x7fb8d80c50a0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80097a0 tx=0x7fb8c802b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c408b20f20f6da server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:58.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.497+0000 7fb8d67fc640 1 -- 172.21.15.27:0/1631179463 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8031780 con 0x7fb8d81553a0 2023-12-19T18:28:58.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.497+0000 7fb8d67fc640 1 -- 172.21.15.27:0/1631179463 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c80318e0 con 0x7fb8d81553a0 2023-12-19T18:28:58.497 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.497+0000 7fb8d67fc640 1 -- 172.21.15.27:0/1631179463 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c8031c70 con 0x7fb8d81553a0 2023-12-19T18:28:58.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.498+0000 7fb8d7fff640 1 -- 172.21.15.27:0/1631179463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8d81553a0 msgr2=0x7fb8d80c50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:58.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.498+0000 7fb8d7fff640 1 --2- 172.21.15.27:0/1631179463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8d81553a0 0x7fb8d80c50a0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80097a0 tx=0x7fb8c802b820 comp rx=0 tx=0).stop 2023-12-19T18:28:58.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.498+0000 7fb8d7fff640 1 -- 172.21.15.27:0/1631179463 shutdown_connections 2023-12-19T18:28:58.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.498+0000 7fb8d7fff640 1 --2- 172.21.15.27:0/1631179463 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d815ee30 0x7fb8d80c39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:58.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.498+0000 7fb8d7fff640 1 --2- 172.21.15.27:0/1631179463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8d81553a0 0x7fb8d80c50a0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:58.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.498+0000 7fb8d7fff640 1 -- 172.21.15.27:0/1631179463 >> 172.21.15.27:0/1631179463 conn(0x7fb8d8069c70 msgr2=0x7fb8d806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:58.498 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.498+0000 7fb8d7fff640 1 -- 172.21.15.27:0/1631179463 shutdown_connections 2023-12-19T18:28:58.499 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.498+0000 7fb8d7fff640 1 -- 172.21.15.27:0/1631179463 wait complete. 2023-12-19T18:28:58.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.500+0000 7fb8d7fff640 1 Processor -- start 2023-12-19T18:28:58.500 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.500+0000 7fb8d7fff640 1 -- start start 2023-12-19T18:28:58.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.501+0000 7fb8d7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d80afeb0 0x7fb8d80c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:58.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.501+0000 7fb8d7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8d81553a0 0x7fb8d80b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:58.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.501+0000 7fb8d7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8d815ee30 0x7fb8d80c8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:58.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.501+0000 7fb8d7fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d80c8c80 con 0x7fb8d815ee30 2023-12-19T18:28:58.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.501+0000 7fb8d7fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d80c89a0 con 0x7fb8d81553a0 2023-12-19T18:28:58.501 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.501+0000 7fb8d7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8d81f0430 con 0x7fb8d80afeb0 2023-12-19T18:28:58.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.501+0000 7fb8d77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8d815ee30 0x7fb8d80c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:58.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.502+0000 7fb8d77fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8d815ee30 0x7fb8d80c8100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33062/0 (socket says 172.21.15.27:33062) 2023-12-19T18:28:58.502 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.502+0000 7fb8d77fe640 1 -- 172.21.15.27:0/256756959 learned_addr learned my addr 172.21.15.27:0/256756959 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.502+0000 7fb8d6ffd640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d80afeb0 0x7fb8d80c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.502+0000 7fb8cffff640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8d81553a0 0x7fb8d80b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.502+0000 7fb8d77fe640 1 -- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d80afeb0 msgr2=0x7fb8d80c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.502+0000 7fb8d77fe640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d80afeb0 0x7fb8d80c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.502+0000 7fb8d77fe640 1 -- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8d81553a0 msgr2=0x7fb8d80b3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.502+0000 7fb8d77fe640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8d81553a0 0x7fb8d80b3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.502+0000 7fb8d77fe640 1 -- 172.21.15.27:0/256756959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c8009400 con 0x7fb8d815ee30 2023-12-19T18:28:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.502+0000 7fb8d6ffd640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d80afeb0 0x7fb8d80c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.502+0000 7fb8cffff640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8d81553a0 0x7fb8d80b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.502+0000 7fb8d77fe640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8d815ee30 0x7fb8d80c8100 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000cd10 tx=0x7fb8d000a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.503+0000 7fb8d4ff9640 1 -- 172.21.15.27:0/256756959 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0003b90 con 0x7fb8d815ee30 2023-12-19T18:28:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.503+0000 7fb8d4ff9640 1 -- 172.21.15.27:0/256756959 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8d0003d10 con 0x7fb8d815ee30 2023-12-19T18:28:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.503+0000 7fb8d4ff9640 1 -- 172.21.15.27:0/256756959 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d0004260 con 0x7fb8d815ee30 2023-12-19T18:28:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.503+0000 7fb8d7fff640 1 -- 172.21.15.27:0/256756959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8d81f0710 con 0x7fb8d815ee30 2023-12-19T18:28:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.503+0000 7fb8d7fff640 1 -- 172.21.15.27:0/256756959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8d80b4330 con 0x7fb8d815ee30 2023-12-19T18:28:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.504+0000 7fb8d7fff640 1 -- 172.21.15.27:0/256756959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8d80c3ef0 con 0x7fb8d815ee30 2023-12-19T18:28:58.505 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.505+0000 7fb8d4ff9640 1 -- 172.21.15.27:0/256756959 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb8d000b040 con 0x7fb8d815ee30 2023-12-19T18:28:58.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.506+0000 7fb8d4ff9640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb8b806a430 0x7fb8b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:58.506 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.507+0000 7fb8d4ff9640 1 -- 172.21.15.27:0/256756959 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fb8d0089080 con 0x7fb8d815ee30 2023-12-19T18:28:58.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.507+0000 7fb8d4ff9640 1 -- 172.21.15.27:0/256756959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=208}) v3 -- 0x7fb8b8071fc0 con 0x7fb8d815ee30 2023-12-19T18:28:58.507 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.507+0000 7fb8d6ffd640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb8b806a430 0x7fb8b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:58.508 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.509+0000 7fb8d6ffd640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb8b806a430 0x7fb8b806c8f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8036000 tx=0x7fb8c803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:58.511 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.511+0000 7fb8d4ff9640 1 -- 172.21.15.27:0/256756959 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb8d00bbca0 con 0x7fb8d815ee30 2023-12-19T18:28:58.752 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.752+0000 7fb8d7fff640 1 -- 172.21.15.27:0/256756959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb8d80cb8e0 con 0x7fb8d815ee30 2023-12-19T18:28:58.753 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.753+0000 7fb8d4ff9640 1 -- 172.21.15.27:0/256756959 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4713 (secure 0 0 0) 0x7fb8d001d060 con 0x7fb8d815ee30 2023-12-19T18:28:58.753 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:58.753 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1864159643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1864159643},{"type":"v1","addr":"172.21.15.192:6836","nonce":1864159643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1348219767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1348219767},{"type":"v1","addr":"172.21.15.192:6837","nonce":1348219767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/181137639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":181137639},{"type":"v1","addr":"172.21.15.27:6833","nonce":181137639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:28:08.577742+0000","modified":"2023-12-19T18:28:12.083302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/152568921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":152568921},{"type":"v1","addr":"172.21.15.27:6835","nonce":152568921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:28:58.754 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 85 2023-12-19T18:28:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.756+0000 7fb8d7fff640 1 -- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb8b806a430 msgr2=0x7fb8b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.756+0000 7fb8d7fff640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb8b806a430 0x7fb8b806c8f0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb8c8036000 tx=0x7fb8c803d040 comp rx=0 tx=0).stop 2023-12-19T18:28:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.756+0000 7fb8d7fff640 1 -- 172.21.15.27:0/256756959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8d815ee30 msgr2=0x7fb8d80c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.756+0000 7fb8d7fff640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8d815ee30 0x7fb8d80c8100 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7fb8d000cd10 tx=0x7fb8d000a650 comp rx=0 tx=0).stop 2023-12-19T18:28:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.756+0000 7fb8d7fff640 1 -- 172.21.15.27:0/256756959 shutdown_connections 2023-12-19T18:28:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.756+0000 7fb8d7fff640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb8b806a430 0x7fb8b806c8f0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.756+0000 7fb8d7fff640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8d815ee30 0x7fb8d80c8100 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:58.756 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.756+0000 7fb8d7fff640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb8d81553a0 0x7fb8d80b3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:58.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.756+0000 7fb8d7fff640 1 --2- 172.21.15.27:0/256756959 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb8d80afeb0 0x7fb8d80c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:58.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.756+0000 7fb8d7fff640 1 -- 172.21.15.27:0/256756959 >> 172.21.15.27:0/256756959 conn(0x7fb8d8069c70 msgr2=0x7fb8d8158d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:58.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.756+0000 7fb8d7fff640 1 -- 172.21.15.27:0/256756959 shutdown_connections 2023-12-19T18:28:58.757 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:58.756+0000 7fb8d7fff640 1 -- 172.21.15.27:0/256756959 wait complete. 2023-12-19T18:28:58.774 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:28:59.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.008+0000 7f48c56c5640 1 Processor -- start 2023-12-19T18:28:59.008 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.008+0000 7f48c56c5640 1 -- start start 2023-12-19T18:28:59.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.010+0000 7f48c56c5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48c015c7f0 0x7f48c00c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.010+0000 7f48c56c5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f48c00cab50 0x7f48c00c9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.010+0000 7f48c56c5640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48c00ca310 con 0x7f48c00cc510 2023-12-19T18:28:59.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.010+0000 7f48c56c5640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48c015cbd0 con 0x7f48c00cab50 2023-12-19T18:28:59.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.010+0000 7f48c56c5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c00ae490 con 0x7f48c015c7f0 2023-12-19T18:28:59.010 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.010+0000 7f48bf7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f48c00cab50 0x7f48c00c9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.010+0000 7f48bf7fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f48c00cab50 0x7f48c00c9a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33068/0 (socket says 172.21.15.27:33068) 2023-12-19T18:28:59.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.011+0000 7f48bf7fe640 1 -- 172.21.15.27:0/2638362018 learned_addr learned my addr 172.21.15.27:0/2638362018 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:59.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.011+0000 7f48bffff640 1 --2- 172.21.15.27:0/2638362018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48c015c7f0 0x7f48c00c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.011+0000 7f48bf7fe640 1 -- 172.21.15.27:0/2638362018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48c015c7f0 msgr2=0x7f48c00c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:59.011 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.011+0000 7f48bf7fe640 1 --2- 172.21.15.27:0/2638362018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48c015c7f0 0x7f48c00c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.011+0000 7f48bf7fe640 1 -- 172.21.15.27:0/2638362018 >> v1:172.21.15.192:6789/0 conn(0x7f48c00cc510 legacy=0x7f48c00af250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:59.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.011+0000 7f48bf7fe640 1 -- 172.21.15.27:0/2638362018 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c00ae770 con 0x7f48c00cab50 2023-12-19T18:28:59.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.011+0000 7f48bffff640 1 --2- 172.21.15.27:0/2638362018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48c015c7f0 0x7f48c00c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:59.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.012+0000 7f48bf7fe640 1 --2- 172.21.15.27:0/2638362018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48c00cab50 0x7f48c00c9a30 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009870 tx=0x7f48b402b7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d04d313fb6872102 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:59.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.012+0000 7f48beffd640 1 -- 172.21.15.27:0/2638362018 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b4033620 con 0x7f48c00cab50 2023-12-19T18:28:59.012 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.013+0000 7f48beffd640 1 -- 172.21.15.27:0/2638362018 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b4033780 con 0x7f48c00cab50 2023-12-19T18:28:59.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.013+0000 7f48beffd640 1 -- 172.21.15.27:0/2638362018 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b4033b10 con 0x7f48c00cab50 2023-12-19T18:28:59.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.013+0000 7f48c56c5640 1 -- 172.21.15.27:0/2638362018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48c00cab50 msgr2=0x7f48c00c9a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.013 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.013+0000 7f48c56c5640 1 --2- 172.21.15.27:0/2638362018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48c00cab50 0x7f48c00c9a30 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f48b4009870 tx=0x7f48b402b7d0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.013+0000 7f48c56c5640 1 -- 172.21.15.27:0/2638362018 shutdown_connections 2023-12-19T18:28:59.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.013+0000 7f48c56c5640 1 --2- 172.21.15.27:0/2638362018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48c00cab50 0x7f48c00c9a30 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.013+0000 7f48c56c5640 1 --2- 172.21.15.27:0/2638362018 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48c015c7f0 0x7f48c00c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.013+0000 7f48c56c5640 1 -- 172.21.15.27:0/2638362018 >> 172.21.15.27:0/2638362018 conn(0x7f48c0069c70 msgr2=0x7f48c006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:59.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.013+0000 7f48c56c5640 1 -- 172.21.15.27:0/2638362018 shutdown_connections 2023-12-19T18:28:59.014 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.014+0000 7f48c56c5640 1 -- 172.21.15.27:0/2638362018 wait complete. 2023-12-19T18:28:59.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.015+0000 7f48c56c5640 1 Processor -- start 2023-12-19T18:28:59.015 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.015+0000 7f48c56c5640 1 -- start start 2023-12-19T18:28:59.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.016+0000 7f48c56c5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48c00cab50 0x7f48c01554c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.016+0000 7f48c56c5640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48c00cc510 0x7f48c014e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.016 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.016+0000 7f48c56c5640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48c015c7f0 0x7f48c015a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.016+0000 7f48c56c5640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c006f470 con 0x7f48c00cab50 2023-12-19T18:28:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.016+0000 7f48c56c5640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c015a290 con 0x7f48c015c7f0 2023-12-19T18:28:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.017+0000 7f48c56c5640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48c006cba0 con 0x7f48c00cc510 2023-12-19T18:28:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.017+0000 7f48bffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48c00cab50 0x7f48c01554c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.017+0000 7f48bffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48c00cab50 0x7f48c01554c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33072/0 (socket says 172.21.15.27:33072) 2023-12-19T18:28:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.017+0000 7f48bffff640 1 -- 172.21.15.27:0/2934256186 learned_addr learned my addr 172.21.15.27:0/2934256186 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:59.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.017+0000 7f48bf7fe640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48c00cc510 0x7f48c014e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.017+0000 7f48c4ec4640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48c015c7f0 0x7f48c015a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.018 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.017+0000 7f48bffff640 1 -- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48c00cc510 msgr2=0x7f48c014e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.017+0000 7f48bffff640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48c00cc510 0x7f48c014e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.017+0000 7f48bffff640 1 -- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48c015c7f0 msgr2=0x7f48c015a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.017+0000 7f48bffff640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48c015c7f0 0x7f48c015a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.017+0000 7f48bffff640 1 -- 172.21.15.27:0/2934256186 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b40094d0 con 0x7f48c00cab50 2023-12-19T18:28:59.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.017+0000 7f48c4ec4640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48c015c7f0 0x7f48c015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:59.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.018+0000 7f48bffff640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48c00cab50 0x7f48c01554c0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f48b000c7f0 tx=0x7f48b000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:59.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.018+0000 7f48bcff9640 1 -- 172.21.15.27:0/2934256186 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0007820 con 0x7f48c00cab50 2023-12-19T18:28:59.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.018+0000 7f48bcff9640 1 -- 172.21.15.27:0/2934256186 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48b0007980 con 0x7f48c00cab50 2023-12-19T18:28:59.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.018+0000 7f48c56c5640 1 -- 172.21.15.27:0/2934256186 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c006ce80 con 0x7f48c00cab50 2023-12-19T18:28:59.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.018+0000 7f48bcff9640 1 -- 172.21.15.27:0/2934256186 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48b0007cf0 con 0x7f48c00cab50 2023-12-19T18:28:59.019 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.018+0000 7f48c56c5640 1 -- 172.21.15.27:0/2934256186 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c014e930 con 0x7f48c00cab50 2023-12-19T18:28:59.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.019+0000 7f48c56c5640 1 -- 172.21.15.27:0/2934256186 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c006a990 con 0x7f48c00cab50 2023-12-19T18:28:59.020 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.020+0000 7f48bcff9640 1 -- 172.21.15.27:0/2934256186 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f48b00113f0 con 0x7f48c00cab50 2023-12-19T18:28:59.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.021+0000 7f48bcff9640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f48ac06a430 0x7f48ac06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.021+0000 7f48bcff9640 1 -- 172.21.15.27:0/2934256186 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f48b0088cf0 con 0x7f48c00cab50 2023-12-19T18:28:59.021 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.022+0000 7f48bcff9640 1 -- 172.21.15.27:0/2934256186 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=208}) v3 -- 0x7f48ac071fc0 con 0x7f48c00cab50 2023-12-19T18:28:59.022 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.022+0000 7f48bf7fe640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f48ac06a430 0x7f48ac06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.023 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.023+0000 7f48bf7fe640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f48ac06a430 0x7f48ac06c8f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f48b4035000 tx=0x7f48b4008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:59.025 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.026+0000 7f48bcff9640 1 -- 172.21.15.27:0/2934256186 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f48b001f020 con 0x7f48c00cab50 2023-12-19T18:28:59.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.224+0000 7f48c56c5640 1 -- 172.21.15.27:0/2934256186 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f48c004e840 con 0x7f48c00cab50 2023-12-19T18:28:59.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.228+0000 7f48bcff9640 1 -- 172.21.15.27:0/2934256186 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v207) v1 ==== 74+0+16188 (secure 0 0 0) 0x7f48b008d070 con 0x7f48c00cab50 2023-12-19T18:28:59.228 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:28:59.228 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":207,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:28:56.849208+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:28:06.542489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:28:07.623298+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"207","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.26","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:28:59.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.230+0000 7f48c56c5640 1 -- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f48ac06a430 msgr2=0x7f48ac06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.230 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.230+0000 7f48c56c5640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f48ac06a430 0x7f48ac06c8f0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f48b4035000 tx=0x7f48b4008030 comp rx=0 tx=0).stop 2023-12-19T18:28:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.230+0000 7f48c56c5640 1 -- 172.21.15.27:0/2934256186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48c00cab50 msgr2=0x7f48c01554c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.230+0000 7f48c56c5640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48c00cab50 0x7f48c01554c0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f48b000c7f0 tx=0x7f48b000ccb0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.231+0000 7f48c56c5640 1 -- 172.21.15.27:0/2934256186 shutdown_connections 2023-12-19T18:28:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.231+0000 7f48c56c5640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f48ac06a430 0x7f48ac06c8f0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.231+0000 7f48c56c5640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f48c015c7f0 0x7f48c015a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.231+0000 7f48c56c5640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f48c00cc510 0x7f48c014e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.231+0000 7f48c56c5640 1 --2- 172.21.15.27:0/2934256186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f48c00cab50 0x7f48c01554c0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.231+0000 7f48c56c5640 1 -- 172.21.15.27:0/2934256186 >> 172.21.15.27:0/2934256186 conn(0x7f48c0069c70 msgr2=0x7f48c015cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.231+0000 7f48c56c5640 1 -- 172.21.15.27:0/2934256186 shutdown_connections 2023-12-19T18:28:59.231 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.231+0000 7f48c56c5640 1 -- 172.21.15.27:0/2934256186 wait complete. 2023-12-19T18:28:59.249 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-19T18:14:43.789712+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-19T18:28:06.542489+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '201', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-12-19T18:28:07.623298+0000', 'flags': 1027, 'flags_names': 'hashpspool,full,full_quota', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '207', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 33554432, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-19T18:28:59.250 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-19T18:28:59.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.477+0000 7ff5c9170640 1 Processor -- start 2023-12-19T18:28:59.477 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.477+0000 7ff5c9170640 1 -- start start 2023-12-19T18:28:59.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.479+0000 7ff5c9170640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff5c40cf550 0x7ff5c415b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.479+0000 7ff5c9170640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c415b3e0 con 0x7ff5c4151c90 2023-12-19T18:28:59.479 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.479+0000 7ff5c9170640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5c415eed0 con 0x7ff5c415e9d0 2023-12-19T18:28:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.479+0000 7ff5c9170640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5c415f1b0 con 0x7ff5c40cf550 2023-12-19T18:28:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.479+0000 7ff5c3fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7ff5c4151c90 0x7ff5c406b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55304/0 (socket says 172.21.15.27:55304) 2023-12-19T18:28:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.479+0000 7ff5c3fff640 1 -- 172.21.15.27:0/40861090 learned_addr learned my addr 172.21.15.27:0/40861090 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.480+0000 7ff5c896f640 1 --2- 172.21.15.27:0/40861090 >> v2:172.21.15.27:3300/0 conn(0x7ff5c40cf550 0x7ff5c415b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.480+0000 7ff5c896f640 1 -- 172.21.15.27:0/40861090 >> v1:172.21.15.192:6789/0 conn(0x7ff5c415e9d0 legacy=0x7ff5c40cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.480+0000 7ff5c896f640 1 -- 172.21.15.27:0/40861090 >> v1:172.21.15.27:6789/0 conn(0x7ff5c4151c90 legacy=0x7ff5c406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.480 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.480+0000 7ff5c896f640 1 -- 172.21.15.27:0/40861090 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5c415f490 con 0x7ff5c40cf550 2023-12-19T18:28:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.481+0000 7ff5c896f640 1 --2- 172.21.15.27:0/40861090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c40cf550 0x7ff5c415b8a0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009dc0 tx=0x7ff5bc02bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cabca929251d47f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.482+0000 7ff5c2ffd640 1 -- 172.21.15.27:0/40861090 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc02cb20 con 0x7ff5c40cf550 2023-12-19T18:28:59.481 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.482+0000 7ff5c2ffd640 1 -- 172.21.15.27:0/40861090 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5bc02cc80 con 0x7ff5c40cf550 2023-12-19T18:28:59.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.482+0000 7ff5c2ffd640 1 -- 172.21.15.27:0/40861090 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5bc034440 con 0x7ff5c40cf550 2023-12-19T18:28:59.482 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.482+0000 7ff5c9170640 1 -- 172.21.15.27:0/40861090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c40cf550 msgr2=0x7ff5c415b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.482+0000 7ff5c9170640 1 --2- 172.21.15.27:0/40861090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c40cf550 0x7ff5c415b8a0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7ff5bc009dc0 tx=0x7ff5bc02bd70 comp rx=0 tx=0).stop 2023-12-19T18:28:59.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.482+0000 7ff5c9170640 1 -- 172.21.15.27:0/40861090 shutdown_connections 2023-12-19T18:28:59.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.483+0000 7ff5c9170640 1 --2- 172.21.15.27:0/40861090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c40cf550 0x7ff5c415b8a0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.483+0000 7ff5c9170640 1 -- 172.21.15.27:0/40861090 >> 172.21.15.27:0/40861090 conn(0x7ff5c406a060 msgr2=0x7ff5c406a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:59.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.483+0000 7ff5c9170640 1 -- 172.21.15.27:0/40861090 shutdown_connections 2023-12-19T18:28:59.483 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.483+0000 7ff5c9170640 1 -- 172.21.15.27:0/40861090 wait complete. 2023-12-19T18:28:59.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.484+0000 7ff5c9170640 1 Processor -- start 2023-12-19T18:28:59.484 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.485+0000 7ff5c9170640 1 -- start start 2023-12-19T18:28:59.485 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.485+0000 7ff5c9170640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c40cf550 0x7ff5c4154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.485+0000 7ff5c9170640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c4151c90 0x7ff5c413c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.485+0000 7ff5c9170640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c415e9d0 0x7ff5c413ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.485+0000 7ff5c9170640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c413ec60 con 0x7ff5c40cf550 2023-12-19T18:28:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.485+0000 7ff5c9170640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c413cb40 con 0x7ff5c415e9d0 2023-12-19T18:28:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.485+0000 7ff5c9170640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5c413ce20 con 0x7ff5c4151c90 2023-12-19T18:28:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.486+0000 7ff5c3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c40cf550 0x7ff5c4154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.486+0000 7ff5c3fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c40cf550 0x7ff5c4154990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33104/0 (socket says 172.21.15.27:33104) 2023-12-19T18:28:59.486 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.486+0000 7ff5c3fff640 1 -- 172.21.15.27:0/2736414620 learned_addr learned my addr 172.21.15.27:0/2736414620 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:59.487 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.486+0000 7ff5c3fff640 1 -- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c4151c90 msgr2=0x7ff5c413c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.486+0000 7ff5c37fe640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c4151c90 0x7ff5c413c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.486+0000 7ff5c896f640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c415e9d0 0x7ff5c413ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.486+0000 7ff5c3fff640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c4151c90 0x7ff5c413c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.486+0000 7ff5c3fff640 1 -- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c415e9d0 msgr2=0x7ff5c413ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.486+0000 7ff5c3fff640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c415e9d0 0x7ff5c413ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.486+0000 7ff5c3fff640 1 -- 172.21.15.27:0/2736414620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5bc009a70 con 0x7ff5c40cf550 2023-12-19T18:28:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.486+0000 7ff5c37fe640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c4151c90 0x7ff5c413c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:28:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.487+0000 7ff5c3fff640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c40cf550 0x7ff5c4154990 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400c4a0 tx=0x7ff5b400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.487+0000 7ff5c0ff9640 1 -- 172.21.15.27:0/2736414620 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4005ce0 con 0x7ff5c40cf550 2023-12-19T18:28:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.487+0000 7ff5c0ff9640 1 -- 172.21.15.27:0/2736414620 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5b4005e40 con 0x7ff5c40cf550 2023-12-19T18:28:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.487+0000 7ff5c0ff9640 1 -- 172.21.15.27:0/2736414620 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b4007270 con 0x7ff5c40cf550 2023-12-19T18:28:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.487+0000 7ff5c9170640 1 -- 172.21.15.27:0/2736414620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5c413d100 con 0x7ff5c40cf550 2023-12-19T18:28:59.488 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.487+0000 7ff5c9170640 1 -- 172.21.15.27:0/2736414620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5c4154ed0 con 0x7ff5c40cf550 2023-12-19T18:28:59.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.488+0000 7ff5c9170640 1 -- 172.21.15.27:0/2736414620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5c404e840 con 0x7ff5c40cf550 2023-12-19T18:28:59.489 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.489+0000 7ff5c0ff9640 1 -- 172.21.15.27:0/2736414620 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff5b4007490 con 0x7ff5c40cf550 2023-12-19T18:28:59.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.490+0000 7ff5c0ff9640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5b006a430 0x7ff5b006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.490+0000 7ff5c0ff9640 1 -- 172.21.15.27:0/2736414620 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff5b4087e90 con 0x7ff5c40cf550 2023-12-19T18:28:59.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.490+0000 7ff5c0ff9640 1 -- 172.21.15.27:0/2736414620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=208}) v3 -- 0x7ff5b0071fc0 con 0x7ff5c40cf550 2023-12-19T18:28:59.490 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.491+0000 7ff5c37fe640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5b006a430 0x7ff5b006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.491 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.492+0000 7ff5c37fe640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5b006a430 0x7ff5b006c8f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4141e80 tx=0x7ff5b8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:59.494 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.494+0000 7ff5c0ff9640 1 -- 172.21.15.27:0/2736414620 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff5b4017060 con 0x7ff5c40cf550 2023-12-19T18:28:59.691 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.691+0000 7ff5c9170640 1 -- 172.21.15.27:0/2736414620 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff5c40cb980 con 0x7ff5c40cf550 2023-12-19T18:28:59.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.692+0000 7ff5c0ff9640 1 -- 172.21.15.27:0/2736414620 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v207) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff5b4089030 con 0x7ff5c40cf550 2023-12-19T18:28:59.692 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2023-12-19T18:28:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.695+0000 7ff5c9170640 1 -- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5b006a430 msgr2=0x7ff5b006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.695+0000 7ff5c9170640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5b006a430 0x7ff5b006c8f0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff5c4141e80 tx=0x7ff5b8008040 comp rx=0 tx=0).stop 2023-12-19T18:28:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.695+0000 7ff5c9170640 1 -- 172.21.15.27:0/2736414620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c40cf550 msgr2=0x7ff5c4154990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.695+0000 7ff5c9170640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c40cf550 0x7ff5c4154990 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7ff5b400c4a0 tx=0x7ff5b400c960 comp rx=0 tx=0).stop 2023-12-19T18:28:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.695+0000 7ff5c9170640 1 -- 172.21.15.27:0/2736414620 shutdown_connections 2023-12-19T18:28:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.695+0000 7ff5c9170640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff5b006a430 0x7ff5b006c8f0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.695+0000 7ff5c9170640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff5c415e9d0 0x7ff5c413ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.695+0000 7ff5c9170640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff5c4151c90 0x7ff5c413c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.695+0000 7ff5c9170640 1 --2- 172.21.15.27:0/2736414620 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff5c40cf550 0x7ff5c4154990 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.695+0000 7ff5c9170640 1 -- 172.21.15.27:0/2736414620 >> 172.21.15.27:0/2736414620 conn(0x7ff5c406a060 msgr2=0x7ff5c40ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:59.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.696+0000 7ff5c9170640 1 -- 172.21.15.27:0/2736414620 shutdown_connections 2023-12-19T18:28:59.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.696+0000 7ff5c9170640 1 -- 172.21.15.27:0/2736414620 wait complete. 2023-12-19T18:28:59.717 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-19T18:28:59.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.960+0000 7fe4e2a22640 1 Processor -- start 2023-12-19T18:28:59.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.960+0000 7fe4e2a22640 1 -- start start 2023-12-19T18:28:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.962+0000 7fe4e2a22640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe4dc15e9d0 0x7fe4dc0cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.962+0000 7fe4e2a22640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc152070 0x7fe4dc0cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.962+0000 7fe4e2a22640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4dc15b3e0 con 0x7fe4dc151c90 2023-12-19T18:28:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.962+0000 7fe4e2a22640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4dc15eed0 con 0x7fe4dc15e9d0 2023-12-19T18:28:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.962+0000 7fe4e2a22640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc15f1b0 con 0x7fe4dc152070 2023-12-19T18:28:59.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.963+0000 7fe4e1a20640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fe4dc151c90 0x7fe4dc06b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55314/0 (socket says 172.21.15.27:55314) 2023-12-19T18:28:59.963 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.963+0000 7fe4e1a20640 1 -- 172.21.15.27:0/3534381756 learned_addr learned my addr 172.21.15.27:0/3534381756 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:28:59.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.963+0000 7fe4e121f640 1 --2- 172.21.15.27:0/3534381756 >> v2:172.21.15.192:3300/0 conn(0x7fe4dc15e9d0 0x7fe4dc0cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.963+0000 7fe4e2221640 1 --2- 172.21.15.27:0/3534381756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc152070 0x7fe4dc0cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.963+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020876617 0 0) 0x7fe4dc15b3e0 con 0x7fe4dc151c90 2023-12-19T18:28:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.963+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4c8003520 con 0x7fe4dc151c90 2023-12-19T18:28:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 176706676 0 0) 0x7fe4c8003520 con 0x7fe4dc151c90 2023-12-19T18:28:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc152070 msgr2=0x7fe4dc0cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 --2- 172.21.15.27:0/3534381756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc152070 0x7fe4dc0cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 >> v2:172.21.15.192:3300/0 conn(0x7fe4dc15e9d0 msgr2=0x7fe4dc0cee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 --2- 172.21.15.27:0/3534381756 >> v2:172.21.15.192:3300/0 conn(0x7fe4dc15e9d0 0x7fe4dc0cee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc15f490 con 0x7fe4dc151c90 2023-12-19T18:28:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fe4d8001670 con 0x7fe4dc151c90 2023-12-19T18:28:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 >> v1:172.21.15.27:6789/0 conn(0x7fe4dc151c90 legacy=0x7fe4dc06b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 --2- 172.21.15.27:0/3534381756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc152070 0x7fe4c8003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.965 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 --2- 172.21.15.27:0/3534381756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4c8003f40 0x7fe4c80047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 --2- 172.21.15.27:0/3534381756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4c8004d00 0x7fe4c8005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc15eed0 con 0x7fe4c8003f40 2023-12-19T18:28:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc15f1b0 con 0x7fe4c8004d00 2023-12-19T18:28:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c800fe40 con 0x7fe4dc152070 2023-12-19T18:28:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e121f640 1 --2- 172.21.15.27:0/3534381756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4c8003f40 0x7fe4c80047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.964+0000 7fe4e1a20640 1 --2- 172.21.15.27:0/3534381756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc152070 0x7fe4c8003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.965+0000 7fe4e121f640 1 -- 172.21.15.27:0/3534381756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc152070 msgr2=0x7fe4c8003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.965+0000 7fe4e121f640 1 --2- 172.21.15.27:0/3534381756 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4dc152070 0x7fe4c8003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.965+0000 7fe4e121f640 1 -- 172.21.15.27:0/3534381756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4c8004d00 msgr2=0x7fe4c8005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:28:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.965+0000 7fe4e121f640 1 --2- 172.21.15.27:0/3534381756 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4c8004d00 0x7fe4c8005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.965+0000 7fe4e121f640 1 -- 172.21.15.27:0/3534381756 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c8003520 con 0x7fe4c8003f40 2023-12-19T18:28:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.966+0000 7fe4e121f640 1 --2- 172.21.15.27:0/3534381756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4c8003f40 0x7fe4c80047c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc15ee00 tx=0x7fe4c80057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d44fe03f3edabb20 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.966+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc02c680 con 0x7fe4c8003f40 2023-12-19T18:28:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.966+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc02c7e0 con 0x7fe4c8003f40 2023-12-19T18:28:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.966+0000 7fe4e0a1e640 1 -- 172.21.15.27:0/3534381756 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc02cb70 con 0x7fe4c8003f40 2023-12-19T18:28:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.967+0000 7fe4e2a22640 1 -- 172.21.15.27:0/3534381756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4c8003f40 msgr2=0x7fe4c80047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.967+0000 7fe4e2a22640 1 --2- 172.21.15.27:0/3534381756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4c8003f40 0x7fe4c80047c0 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc15ee00 tx=0x7fe4c80057f0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.967+0000 7fe4e2221640 1 -- 172.21.15.27:0/3534381756 reap_dead start 2023-12-19T18:28:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.967+0000 7fe4e2a22640 1 -- 172.21.15.27:0/3534381756 shutdown_connections 2023-12-19T18:28:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.967+0000 7fe4e2a22640 1 -- 172.21.15.27:0/3534381756 >> 172.21.15.27:0/3534381756 conn(0x7fe4dc06a060 msgr2=0x7fe4dc06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:28:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.967+0000 7fe4e2a22640 1 -- 172.21.15.27:0/3534381756 shutdown_connections 2023-12-19T18:28:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.967+0000 7fe4e2a22640 1 -- 172.21.15.27:0/3534381756 wait complete. 2023-12-19T18:28:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.969+0000 7fe4e2a22640 1 Processor -- start 2023-12-19T18:28:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.969+0000 7fe4e2a22640 1 -- start start 2023-12-19T18:28:59.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.970+0000 7fe4e2a22640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4c8003f40 0x7fe4dc0b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.970+0000 7fe4e2a22640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4dc144a60 0x7fe4dc0b2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.970+0000 7fe4e2a22640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4dc0b3290 0x7fe4dc15b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.970+0000 7fe4e2a22640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc0b3740 con 0x7fe4dc144a60 2023-12-19T18:28:59.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.970+0000 7fe4e2a22640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc0b1280 con 0x7fe4dc0b3290 2023-12-19T18:28:59.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.970+0000 7fe4e2a22640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4dc15d240 con 0x7fe4c8003f40 2023-12-19T18:28:59.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.970+0000 7fe4e121f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4dc144a60 0x7fe4dc0b2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.970+0000 7fe4e121f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4dc144a60 0x7fe4dc0b2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33116/0 (socket says 172.21.15.27:33116) 2023-12-19T18:28:59.970 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.970+0000 7fe4e121f640 1 -- 172.21.15.27:0/458251672 learned_addr learned my addr 172.21.15.27:0/458251672 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:28:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.971+0000 7fe4e2221640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4dc0b3290 0x7fe4dc15b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.971+0000 7fe4e1a20640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4c8003f40 0x7fe4dc0b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.971+0000 7fe4e121f640 1 -- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4c8003f40 msgr2=0x7fe4dc0b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.971+0000 7fe4e121f640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4c8003f40 0x7fe4dc0b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.971+0000 7fe4e121f640 1 -- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4dc0b3290 msgr2=0x7fe4dc15b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:28:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.971+0000 7fe4e121f640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4dc0b3290 0x7fe4dc15b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:28:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.971+0000 7fe4e121f640 1 -- 172.21.15.27:0/458251672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc15d520 con 0x7fe4dc144a60 2023-12-19T18:28:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.971+0000 7fe4e121f640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4dc144a60 0x7fe4dc0b2d50 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc000c00 tx=0x7fe4cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.972+0000 7fe4d27fc640 1 -- 172.21.15.27:0/458251672 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc003810 con 0x7fe4dc144a60 2023-12-19T18:28:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.972+0000 7fe4d27fc640 1 -- 172.21.15.27:0/458251672 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4cc003970 con 0x7fe4dc144a60 2023-12-19T18:28:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.972+0000 7fe4d27fc640 1 -- 172.21.15.27:0/458251672 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4cc003d00 con 0x7fe4dc144a60 2023-12-19T18:28:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.972+0000 7fe4e2a22640 1 -- 172.21.15.27:0/458251672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4dc15d800 con 0x7fe4dc144a60 2023-12-19T18:28:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.972+0000 7fe4e2a22640 1 -- 172.21.15.27:0/458251672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4dc15ddc0 con 0x7fe4dc144a60 2023-12-19T18:28:59.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.973+0000 7fe4e2a22640 1 -- 172.21.15.27:0/458251672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4dc15df50 con 0x7fe4dc144a60 2023-12-19T18:28:59.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.974+0000 7fe4d27fc640 1 -- 172.21.15.27:0/458251672 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe4cc040450 con 0x7fe4dc144a60 2023-12-19T18:28:59.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.975+0000 7fe4d27fc640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe4c006a430 0x7fe4c006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:28:59.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.975+0000 7fe4d27fc640 1 -- 172.21.15.27:0/458251672 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fe4cc0aa800 con 0x7fe4dc144a60 2023-12-19T18:28:59.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.975+0000 7fe4d27fc640 1 -- 172.21.15.27:0/458251672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=208}) v3 -- 0x7fe4c0071fc0 con 0x7fe4dc144a60 2023-12-19T18:28:59.975 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.976+0000 7fe4e1a20640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe4c006a430 0x7fe4c006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:28:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.977+0000 7fe4e1a20640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe4c006a430 0x7fe4c006c8f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8001050 tx=0x7fe4d8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:28:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:28:59.979+0000 7fe4d27fc640 1 -- 172.21.15.27:0/458251672 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe4cc075e60 con 0x7fe4dc144a60 2023-12-19T18:29:00.172 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.172+0000 7fe4e2a22640 1 -- 172.21.15.27:0/458251672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe4dc15f740 con 0x7fe4dc144a60 2023-12-19T18:29:00.173 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.173+0000 7fe4d27fc640 1 -- 172.21.15.27:0/458251672 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v207) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe4cc039020 con 0x7fe4dc144a60 2023-12-19T18:29:00.173 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2023-12-19T18:29:00.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.176+0000 7fe4e2a22640 1 -- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe4c006a430 msgr2=0x7fe4c006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:00.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.176+0000 7fe4e2a22640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe4c006a430 0x7fe4c006c8f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7fe4d8001050 tx=0x7fe4d8009040 comp rx=0 tx=0).stop 2023-12-19T18:29:00.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.176+0000 7fe4e2a22640 1 -- 172.21.15.27:0/458251672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4dc144a60 msgr2=0x7fe4dc0b2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:00.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.176+0000 7fe4e2a22640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4dc144a60 0x7fe4dc0b2d50 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7fe4cc000c00 tx=0x7fe4cc004730 comp rx=0 tx=0).stop 2023-12-19T18:29:00.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.176+0000 7fe4e2a22640 1 -- 172.21.15.27:0/458251672 shutdown_connections 2023-12-19T18:29:00.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.176+0000 7fe4e2a22640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe4c006a430 0x7fe4c006c8f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.176+0000 7fe4e2a22640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe4dc0b3290 0x7fe4dc15b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.176+0000 7fe4e2a22640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4dc144a60 0x7fe4dc0b2d50 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.176+0000 7fe4e2a22640 1 --2- 172.21.15.27:0/458251672 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe4c8003f40 0x7fe4dc0b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.176+0000 7fe4e2a22640 1 -- 172.21.15.27:0/458251672 >> 172.21.15.27:0/458251672 conn(0x7fe4dc06a060 msgr2=0x7fe4dc150810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:00.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.176+0000 7fe4e2a22640 1 -- 172.21.15.27:0/458251672 shutdown_connections 2023-12-19T18:29:00.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.176+0000 7fe4e2a22640 1 -- 172.21.15.27:0/458251672 wait complete. 2023-12-19T18:29:00.197 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-19T18:29:00.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.426+0000 7f30c8a40640 1 Processor -- start 2023-12-19T18:29:00.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.426+0000 7f30c8a40640 1 -- start start 2023-12-19T18:29:00.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.427+0000 7f30c8a40640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f30c415cf80 0x7f30c40d2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:00.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.427+0000 7f30c8a40640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f30c40b8a60 0x7f30c40b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:00.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.427+0000 7f30c8a40640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30c40d0e10 con 0x7f30c40b85e0 2023-12-19T18:29:00.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.427+0000 7f30c8a40640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c40d1c70 con 0x7f30c415cf80 2023-12-19T18:29:00.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.427+0000 7f30c8a40640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30c415d360 con 0x7f30c40b8a60 2023-12-19T18:29:00.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.427+0000 7f30c2d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f30c415cf80 0x7f30c40d2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:00.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.428+0000 7f30c259c640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f30c40b85e0 0x7f30c40cda40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:55328/0 (socket says 172.21.15.27:55328) 2023-12-19T18:29:00.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.428+0000 7f30c2d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f30c415cf80 0x7f30c40d2100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33130/0 (socket says 172.21.15.27:33130) 2023-12-19T18:29:00.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.428+0000 7f30c259c640 1 -- 172.21.15.27:0/3178192599 learned_addr learned my addr 172.21.15.27:0/3178192599 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:00.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.428+0000 7f30c359e640 1 --2- 172.21.15.27:0/3178192599 >> v2:172.21.15.192:3300/0 conn(0x7f30c40b8a60 0x7f30c40b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:00.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.428+0000 7f30c2d9d640 1 -- 172.21.15.27:0/3178192599 >> v2:172.21.15.192:3300/0 conn(0x7f30c40b8a60 msgr2=0x7f30c40b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:00.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.428+0000 7f30c2d9d640 1 --2- 172.21.15.27:0/3178192599 >> v2:172.21.15.192:3300/0 conn(0x7f30c40b8a60 0x7f30c40b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.428+0000 7f30c2d9d640 1 -- 172.21.15.27:0/3178192599 >> v1:172.21.15.27:6789/0 conn(0x7f30c40b85e0 legacy=0x7f30c40cda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:00.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.428+0000 7f30c2d9d640 1 -- 172.21.15.27:0/3178192599 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c40d1060 con 0x7f30c415cf80 2023-12-19T18:29:00.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.429+0000 7f30c2d9d640 1 --2- 172.21.15.27:0/3178192599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c415cf80 0x7f30c40d2100 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f30c40c9cd0 tx=0x7f30b402bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff32d2bac68f525 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:00.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.430+0000 7f30c1d9b640 1 -- 172.21.15.27:0/3178192599 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b402cd10 con 0x7f30c415cf80 2023-12-19T18:29:00.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.430+0000 7f30c1d9b640 1 -- 172.21.15.27:0/3178192599 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b402ce70 con 0x7f30c415cf80 2023-12-19T18:29:00.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.430+0000 7f30c1d9b640 1 -- 172.21.15.27:0/3178192599 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4034640 con 0x7f30c415cf80 2023-12-19T18:29:00.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.430+0000 7f30c8a40640 1 -- 172.21.15.27:0/3178192599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c415cf80 msgr2=0x7f30c40d2100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.430+0000 7f30c8a40640 1 --2- 172.21.15.27:0/3178192599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c415cf80 0x7f30c40d2100 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f30c40c9cd0 tx=0x7f30b402bf60 comp rx=0 tx=0).stop 2023-12-19T18:29:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.430+0000 7f30c8a40640 1 -- 172.21.15.27:0/3178192599 shutdown_connections 2023-12-19T18:29:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.430+0000 7f30c8a40640 1 --2- 172.21.15.27:0/3178192599 >> v2:172.21.15.192:3300/0 conn(0x7f30c40b8a60 0x7f30c40b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.430+0000 7f30c8a40640 1 --2- 172.21.15.27:0/3178192599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c415cf80 0x7f30c40d2100 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.431+0000 7f30c8a40640 1 -- 172.21.15.27:0/3178192599 >> 172.21.15.27:0/3178192599 conn(0x7f30c4072e20 msgr2=0x7f30c4073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.431+0000 7f30c8a40640 1 -- 172.21.15.27:0/3178192599 shutdown_connections 2023-12-19T18:29:00.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.431+0000 7f30c8a40640 1 -- 172.21.15.27:0/3178192599 wait complete. 2023-12-19T18:29:00.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.432+0000 7f30c8a40640 1 Processor -- start 2023-12-19T18:29:00.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.433+0000 7f30c8a40640 1 -- start start 2023-12-19T18:29:00.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.434+0000 7f30c8a40640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30c40b85e0 0x7f30c415a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:00.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.434+0000 7f30c8a40640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c415cf80 0x7f30c40ab410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:00.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.434+0000 7f30c8a40640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30c415fee0 0x7f30c41f24a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:00.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.434+0000 7f30c8a40640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40aa0b0 con 0x7f30c415cf80 2023-12-19T18:29:00.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.434+0000 7f30c8a40640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30c415ab70 con 0x7f30c40b85e0 2023-12-19T18:29:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.434+0000 7f30c8a40640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30c40aa370 con 0x7f30c415fee0 2023-12-19T18:29:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.434+0000 7f30c259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c415cf80 0x7f30c40ab410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.434+0000 7f30c259c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c415cf80 0x7f30c40ab410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33138/0 (socket says 172.21.15.27:33138) 2023-12-19T18:29:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.434+0000 7f30c259c640 1 -- 172.21.15.27:0/1022261883 learned_addr learned my addr 172.21.15.27:0/1022261883 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.434+0000 7f30c259c640 1 -- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30c415fee0 msgr2=0x7f30c41f24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.434+0000 7f30c359e640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30c415fee0 0x7f30c41f24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.434+0000 7f30c2d9d640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30c40b85e0 0x7f30c415a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.434+0000 7f30c259c640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30c415fee0 0x7f30c41f24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.435+0000 7f30c259c640 1 -- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30c40b85e0 msgr2=0x7f30c415a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.435+0000 7f30c259c640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30c40b85e0 0x7f30c415a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.435+0000 7f30c259c640 1 -- 172.21.15.27:0/1022261883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b4009a70 con 0x7f30c415cf80 2023-12-19T18:29:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.435+0000 7f30c259c640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c415cf80 0x7f30c40ab410 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00c400 tx=0x7f30ac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:00.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.435+0000 7f30bb7fe640 1 -- 172.21.15.27:0/1022261883 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac005ce0 con 0x7f30c415cf80 2023-12-19T18:29:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.435+0000 7f30bb7fe640 1 -- 172.21.15.27:0/1022261883 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30ac005e40 con 0x7f30c415cf80 2023-12-19T18:29:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.435+0000 7f30bb7fe640 1 -- 172.21.15.27:0/1022261883 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30ac0072d0 con 0x7f30c415cf80 2023-12-19T18:29:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.435+0000 7f30c8a40640 1 -- 172.21.15.27:0/1022261883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30c40aa650 con 0x7f30c415cf80 2023-12-19T18:29:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.435+0000 7f30c8a40640 1 -- 172.21.15.27:0/1022261883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c40aac10 con 0x7f30c415cf80 2023-12-19T18:29:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.437+0000 7f30c8a40640 1 -- 172.21.15.27:0/1022261883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30c404e840 con 0x7f30c415cf80 2023-12-19T18:29:00.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.437+0000 7f30bb7fe640 1 -- 172.21.15.27:0/1022261883 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f30ac0074f0 con 0x7f30c415cf80 2023-12-19T18:29:00.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.438+0000 7f30bb7fe640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f30a406a480 0x7f30a406c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:00.438 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.439+0000 7f30bb7fe640 1 -- 172.21.15.27:0/1022261883 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f30ac087fd0 con 0x7f30c415cf80 2023-12-19T18:29:00.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.439+0000 7f30bb7fe640 1 -- 172.21.15.27:0/1022261883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=208}) v3 -- 0x7f30a4072010 con 0x7f30c415cf80 2023-12-19T18:29:00.439 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.439+0000 7f30c2d9d640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f30a406a480 0x7f30a406c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:00.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.440+0000 7f30c2d9d640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f30a406a480 0x7f30a406c940 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f30b4036000 tx=0x7f30b4030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:00.442 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.443+0000 7f30bb7fe640 1 -- 172.21.15.27:0/1022261883 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f30ac017060 con 0x7f30c415cf80 2023-12-19T18:29:00.638 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.638+0000 7f30c8a40640 1 -- 172.21.15.27:0/1022261883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f30c40b2b90 con 0x7f30c415cf80 2023-12-19T18:29:00.639 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.639+0000 7f30bb7fe640 1 -- 172.21.15.27:0/1022261883 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v207) v1 ==== 100+0+11 (secure 0 0 0) 0x7f30ac053580 con 0x7f30c415cf80 2023-12-19T18:29:00.639 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 64 2023-12-19T18:29:00.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.642+0000 7f30c8a40640 1 -- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f30a406a480 msgr2=0x7f30a406c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:00.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.642+0000 7f30c8a40640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f30a406a480 0x7f30a406c940 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f30b4036000 tx=0x7f30b4030000 comp rx=0 tx=0).stop 2023-12-19T18:29:00.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.642+0000 7f30c8a40640 1 -- 172.21.15.27:0/1022261883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c415cf80 msgr2=0x7f30c40ab410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:00.642 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.642+0000 7f30c8a40640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c415cf80 0x7f30c40ab410 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f30ac00c400 tx=0x7f30ac00c8c0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.642+0000 7f30c8a40640 1 -- 172.21.15.27:0/1022261883 shutdown_connections 2023-12-19T18:29:00.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.642+0000 7f30c8a40640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f30a406a480 0x7f30a406c940 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.642+0000 7f30c8a40640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f30c415fee0 0x7f30c41f24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.642+0000 7f30c8a40640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30c415cf80 0x7f30c40ab410 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.642+0000 7f30c8a40640 1 --2- 172.21.15.27:0/1022261883 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f30c40b85e0 0x7f30c415a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.642+0000 7f30c8a40640 1 -- 172.21.15.27:0/1022261883 >> 172.21.15.27:0/1022261883 conn(0x7f30c4072e20 msgr2=0x7f30c40d3090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:00.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.642+0000 7f30c8a40640 1 -- 172.21.15.27:0/1022261883 shutdown_connections 2023-12-19T18:29:00.643 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.643+0000 7f30c8a40640 1 -- 172.21.15.27:0/1022261883 wait complete. 2023-12-19T18:29:00.663 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:29:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.898+0000 7f8405cdc640 1 Processor -- start 2023-12-19T18:29:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.899+0000 7f8405cdc640 1 -- start start 2023-12-19T18:29:00.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.900+0000 7f8405cdc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f840015b870 0x7f84000c5260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:00.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.900+0000 7f8405cdc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f84000cdf90 0x7f8400164d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:00.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.900+0000 7f8405cdc640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8400165290 0x7f8400161760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:00.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.900+0000 7f8405cdc640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84000ce750 con 0x7f84000cdf90 2023-12-19T18:29:00.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.900+0000 7f8405cdc640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84001656e0 con 0x7f8400165290 2023-12-19T18:29:00.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.900+0000 7f8405cdc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84000c4a20 con 0x7f840015b870 2023-12-19T18:29:00.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.900+0000 7f83fffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f84000cdf90 0x7f8400164d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:00.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.901+0000 7f83fffff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f84000cdf90 0x7f8400164d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33142/0 (socket says 172.21.15.27:33142) 2023-12-19T18:29:00.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.901+0000 7f83fffff640 1 -- 172.21.15.27:0/2626173349 learned_addr learned my addr 172.21.15.27:0/2626173349 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:00.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.901+0000 7f84054db640 1 --2- 172.21.15.27:0/2626173349 >> v2:172.21.15.192:3300/0 conn(0x7f8400165290 0x7f8400161760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:00.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.901+0000 7f83fffff640 1 -- 172.21.15.27:0/2626173349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f840015b870 msgr2=0x7f84000c5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:00.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.901+0000 7f8404cda640 1 --2- 172.21.15.27:0/2626173349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f840015b870 0x7f84000c5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:00.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.901+0000 7f83fffff640 1 --2- 172.21.15.27:0/2626173349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f840015b870 0x7f84000c5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.901+0000 7f83fffff640 1 -- 172.21.15.27:0/2626173349 >> v2:172.21.15.192:3300/0 conn(0x7f8400165290 msgr2=0x7f8400161760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:00.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.901+0000 7f83fffff640 1 --2- 172.21.15.27:0/2626173349 >> v2:172.21.15.192:3300/0 conn(0x7f8400165290 0x7f8400161760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.901+0000 7f83fffff640 1 -- 172.21.15.27:0/2626173349 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f840015bc50 con 0x7f84000cdf90 2023-12-19T18:29:00.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.901+0000 7f84054db640 1 --2- 172.21.15.27:0/2626173349 >> v2:172.21.15.192:3300/0 conn(0x7f8400165290 0x7f8400161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:00.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.901+0000 7f8404cda640 1 --2- 172.21.15.27:0/2626173349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f840015b870 0x7f84000c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:00.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.902+0000 7f83fffff640 1 --2- 172.21.15.27:0/2626173349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84000cdf90 0x7f8400164d50 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f8400165e70 tx=0x7f83ec02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ed30a15c72c8a68 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:00.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.905+0000 7f83ff7fe640 1 -- 172.21.15.27:0/2626173349 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec031880 con 0x7f84000cdf90 2023-12-19T18:29:00.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.905+0000 7f83ff7fe640 1 -- 172.21.15.27:0/2626173349 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83ec0319e0 con 0x7f84000cdf90 2023-12-19T18:29:00.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.905+0000 7f83ff7fe640 1 -- 172.21.15.27:0/2626173349 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83ec031d70 con 0x7f84000cdf90 2023-12-19T18:29:00.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.905+0000 7f8405cdc640 1 -- 172.21.15.27:0/2626173349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84000cdf90 msgr2=0x7f8400164d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:00.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.905+0000 7f8405cdc640 1 --2- 172.21.15.27:0/2626173349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84000cdf90 0x7f8400164d50 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7f8400165e70 tx=0x7f83ec02b920 comp rx=0 tx=0).stop 2023-12-19T18:29:00.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.906+0000 7f8405cdc640 1 -- 172.21.15.27:0/2626173349 shutdown_connections 2023-12-19T18:29:00.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.906+0000 7f8405cdc640 1 --2- 172.21.15.27:0/2626173349 >> v2:172.21.15.192:3300/0 conn(0x7f8400165290 0x7f8400161760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.906+0000 7f8405cdc640 1 --2- 172.21.15.27:0/2626173349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84000cdf90 0x7f8400164d50 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.906+0000 7f8405cdc640 1 --2- 172.21.15.27:0/2626173349 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f840015b870 0x7f84000c5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.906+0000 7f8405cdc640 1 -- 172.21.15.27:0/2626173349 >> 172.21.15.27:0/2626173349 conn(0x7f8400069c70 msgr2=0x7f840006a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:00.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.906+0000 7f8405cdc640 1 -- 172.21.15.27:0/2626173349 shutdown_connections 2023-12-19T18:29:00.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.906+0000 7f8405cdc640 1 -- 172.21.15.27:0/2626173349 wait complete. 2023-12-19T18:29:00.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.908+0000 7f8405cdc640 1 Processor -- start 2023-12-19T18:29:00.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.908+0000 7f8405cdc640 1 -- start start 2023-12-19T18:29:00.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.909+0000 7f8405cdc640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84000cdf90 0x7f8400155de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:00.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.909+0000 7f8405cdc640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f840015b870 0x7f84001586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:00.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.909+0000 7f8405cdc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8400165290 0x7f84000b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:00.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.909+0000 7f8405cdc640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840006b990 con 0x7f8400165290 2023-12-19T18:29:00.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.909+0000 7f8405cdc640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840006bbe0 con 0x7f84000cdf90 2023-12-19T18:29:00.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.909+0000 7f8405cdc640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8400157ed0 con 0x7f840015b870 2023-12-19T18:29:00.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.909+0000 7f84054db640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8400165290 0x7f84000b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:00.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.909+0000 7f84054db640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8400165290 0x7f84000b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33156/0 (socket says 172.21.15.27:33156) 2023-12-19T18:29:00.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.909+0000 7f84054db640 1 -- 172.21.15.27:0/2099213604 learned_addr learned my addr 172.21.15.27:0/2099213604 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:00.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.909+0000 7f8404cda640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84000cdf90 0x7f8400155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:00.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.909+0000 7f83fffff640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f840015b870 0x7f84001586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:00.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.910+0000 7f84054db640 1 -- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f840015b870 msgr2=0x7f84001586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:00.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.910+0000 7f84054db640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f840015b870 0x7f84001586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.910+0000 7f84054db640 1 -- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84000cdf90 msgr2=0x7f8400155de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:00.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.910+0000 7f84054db640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84000cdf90 0x7f8400155de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.910+0000 7f84054db640 1 -- 172.21.15.27:0/2099213604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0094d0 con 0x7f8400165290 2023-12-19T18:29:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.910+0000 7f8404cda640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84000cdf90 0x7f8400155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:29:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.910+0000 7f84054db640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8400165290 0x7f84000b15c0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f83f800cd50 tx=0x7f83f800a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.911+0000 7f83fd7fa640 1 -- 172.21.15.27:0/2099213604 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f800f680 con 0x7f8400165290 2023-12-19T18:29:00.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.911+0000 7f83fd7fa640 1 -- 172.21.15.27:0/2099213604 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f800f800 con 0x7f8400165290 2023-12-19T18:29:00.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.911+0000 7f83fd7fa640 1 -- 172.21.15.27:0/2099213604 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f800fbb0 con 0x7f8400165290 2023-12-19T18:29:00.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.911+0000 7f8405cdc640 1 -- 172.21.15.27:0/2099213604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84001581b0 con 0x7f8400165290 2023-12-19T18:29:00.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.912+0000 7f8405cdc640 1 -- 172.21.15.27:0/2099213604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840006bec0 con 0x7f8400165290 2023-12-19T18:29:00.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.913+0000 7f8405cdc640 1 -- 172.21.15.27:0/2099213604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84000671b0 con 0x7f8400165290 2023-12-19T18:29:00.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.913+0000 7f83fd7fa640 1 -- 172.21.15.27:0/2099213604 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f83f800b040 con 0x7f8400165290 2023-12-19T18:29:00.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.914+0000 7f83fd7fa640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f83e406a460 0x7f83e406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:00.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.915+0000 7f83fd7fa640 1 -- 172.21.15.27:0/2099213604 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f83f80893a0 con 0x7f8400165290 2023-12-19T18:29:00.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.915+0000 7f83fd7fa640 1 -- 172.21.15.27:0/2099213604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=208}) v3 -- 0x7f83e4071ff0 con 0x7f8400165290 2023-12-19T18:29:00.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.915+0000 7f8404cda640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f83e406a460 0x7f83e406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:00.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.916+0000 7f8404cda640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f83e406a460 0x7f83e406c920 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006b10 tx=0x7f83f4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:00.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:00.919+0000 7f83fd7fa640 1 -- 172.21.15.27:0/2099213604 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f83f801d060 con 0x7f8400165290 2023-12-19T18:29:01.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.149+0000 7f8405cdc640 1 -- 172.21.15.27:0/2099213604 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84001583f0 con 0x7f8400165290 2023-12-19T18:29:01.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.150+0000 7f83fd7fa640 1 -- 172.21.15.27:0/2099213604 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f84001583f0 con 0x7f8400165290 2023-12-19T18:29:01.150 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:29:01.150 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1864159643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1864159643},{"type":"v1","addr":"172.21.15.192:6836","nonce":1864159643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1348219767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1348219767},{"type":"v1","addr":"172.21.15.192:6837","nonce":1348219767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/181137639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":181137639},{"type":"v1","addr":"172.21.15.27:6833","nonce":181137639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:28:08.577742+0000","modified":"2023-12-19T18:28:12.083302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/152568921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":152568921},{"type":"v1","addr":"172.21.15.27:6835","nonce":152568921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:29:01.151 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 85 2023-12-19T18:29:01.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.153+0000 7f8405cdc640 1 -- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f83e406a460 msgr2=0x7f83e406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:01.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.153+0000 7f8405cdc640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f83e406a460 0x7f83e406c920 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f83f4006b10 tx=0x7f83f4008040 comp rx=0 tx=0).stop 2023-12-19T18:29:01.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.153+0000 7f8405cdc640 1 -- 172.21.15.27:0/2099213604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8400165290 msgr2=0x7f84000b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:01.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.153+0000 7f8405cdc640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8400165290 0x7f84000b15c0 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f83f800cd50 tx=0x7f83f800a650 comp rx=0 tx=0).stop 2023-12-19T18:29:01.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.153+0000 7f8405cdc640 1 -- 172.21.15.27:0/2099213604 shutdown_connections 2023-12-19T18:29:01.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.153+0000 7f8405cdc640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f83e406a460 0x7f83e406c920 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.153+0000 7f8405cdc640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8400165290 0x7f84000b15c0 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.153+0000 7f8405cdc640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f840015b870 0x7f84001586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.153+0000 7f8405cdc640 1 --2- 172.21.15.27:0/2099213604 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f84000cdf90 0x7f8400155de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.153+0000 7f8405cdc640 1 -- 172.21.15.27:0/2099213604 >> 172.21.15.27:0/2099213604 conn(0x7f8400069c70 msgr2=0x7f8400162c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:01.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.153+0000 7f8405cdc640 1 -- 172.21.15.27:0/2099213604 shutdown_connections 2023-12-19T18:29:01.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.153+0000 7f8405cdc640 1 -- 172.21.15.27:0/2099213604 wait complete. 2023-12-19T18:29:01.174 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:29:01.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.410+0000 7fa8d4865640 1 Processor -- start 2023-12-19T18:29:01.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.410+0000 7fa8d4865640 1 -- start start 2023-12-19T18:29:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.411+0000 7fa8d4865640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8d01604d0 0x7fa8d00b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.411+0000 7fa8d4865640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa8d00b7310 0x7fa8d00b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.411+0000 7fa8d4865640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8d00b5e90 con 0x7fa8d00d2350 2023-12-19T18:29:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.411+0000 7fa8d4865640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8d00d2b30 con 0x7fa8d00b7310 2023-12-19T18:29:01.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.411+0000 7fa8d4865640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00d2850 con 0x7fa8d01604d0 2023-12-19T18:29:01.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.412+0000 7fa8cf7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8d01604d0 0x7fa8d00b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:01.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.412+0000 7fa8cffff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa8d00b7310 0x7fa8d00b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:01.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.412+0000 7fa8cf7fe640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8d01604d0 0x7fa8d00b47e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3301/0 says I am v2:172.21.15.27:58322/0 (socket says 172.21.15.27:58322) 2023-12-19T18:29:01.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.412+0000 7fa8cf7fe640 1 -- 172.21.15.27:0/2512787360 learned_addr learned my addr 172.21.15.27:0/2512787360 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:01.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.412+0000 7fa8cffff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa8d00b7310 0x7fa8d00b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:45494/0 (socket says 172.21.15.27:45494) 2023-12-19T18:29:01.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.413+0000 7fa8cf7fe640 1 -- 172.21.15.27:0/2512787360 >> v2:172.21.15.192:3300/0 conn(0x7fa8d00b7310 msgr2=0x7fa8d00b4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:01.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.413+0000 7fa8cf7fe640 1 --2- 172.21.15.27:0/2512787360 >> v2:172.21.15.192:3300/0 conn(0x7fa8d00b7310 0x7fa8d00b4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.413 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.413+0000 7fa8cf7fe640 1 -- 172.21.15.27:0/2512787360 >> v1:172.21.15.192:6789/0 conn(0x7fa8d00d2350 legacy=0x7fa8d00d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:01.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.413+0000 7fa8cf7fe640 1 -- 172.21.15.27:0/2512787360 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d00cded0 con 0x7fa8d01604d0 2023-12-19T18:29:01.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.414+0000 7fa8cf7fe640 1 --2- 172.21.15.27:0/2512787360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8d01604d0 0x7fa8d00b47e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009870 tx=0x7fa8c002b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17cd528e729922c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:01.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.415+0000 7fa8ce7fc640 1 -- 172.21.15.27:0/2512787360 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0030840 con 0x7fa8d01604d0 2023-12-19T18:29:01.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.415+0000 7fa8ce7fc640 1 -- 172.21.15.27:0/2512787360 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c00309a0 con 0x7fa8d01604d0 2023-12-19T18:29:01.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.415+0000 7fa8ce7fc640 1 -- 172.21.15.27:0/2512787360 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c0030d30 con 0x7fa8d01604d0 2023-12-19T18:29:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.416+0000 7fa8d4865640 1 -- 172.21.15.27:0/2512787360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8d01604d0 msgr2=0x7fa8d00b47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.416+0000 7fa8d4865640 1 --2- 172.21.15.27:0/2512787360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8d01604d0 0x7fa8d00b47e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009870 tx=0x7fa8c002b980 comp rx=0 tx=0).stop 2023-12-19T18:29:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.416+0000 7fa8d4865640 1 -- 172.21.15.27:0/2512787360 shutdown_connections 2023-12-19T18:29:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.416+0000 7fa8d4865640 1 --2- 172.21.15.27:0/2512787360 >> v2:172.21.15.192:3300/0 conn(0x7fa8d00b7310 0x7fa8d00b4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.416+0000 7fa8d4865640 1 --2- 172.21.15.27:0/2512787360 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8d01604d0 0x7fa8d00b47e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.416+0000 7fa8d4865640 1 -- 172.21.15.27:0/2512787360 >> 172.21.15.27:0/2512787360 conn(0x7fa8d00cf7e0 msgr2=0x7fa8d00cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.416+0000 7fa8d4865640 1 -- 172.21.15.27:0/2512787360 shutdown_connections 2023-12-19T18:29:01.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.416+0000 7fa8d4865640 1 -- 172.21.15.27:0/2512787360 wait complete. 2023-12-19T18:29:01.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.418+0000 7fa8d4865640 1 Processor -- start 2023-12-19T18:29:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.418+0000 7fa8d4865640 1 -- start start 2023-12-19T18:29:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.420+0000 7fa8d4865640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8d00b7310 0x7fa8d0150240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.420+0000 7fa8d4865640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8d00d2350 0x7fa8d015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.420+0000 7fa8d4865640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8d01604d0 0x7fa8d00c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.420+0000 7fa8d4865640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d0150780 con 0x7fa8d01604d0 2023-12-19T18:29:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.420+0000 7fa8d4865640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00cfbc0 con 0x7fa8d00d2350 2023-12-19T18:29:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.420+0000 7fa8d4865640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8d00cfea0 con 0x7fa8d00b7310 2023-12-19T18:29:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.420+0000 7fa8cffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8d01604d0 0x7fa8d00c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:01.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.420+0000 7fa8cffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8d01604d0 0x7fa8d00c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33170/0 (socket says 172.21.15.27:33170) 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.420+0000 7fa8cffff640 1 -- 172.21.15.27:0/3111163509 learned_addr learned my addr 172.21.15.27:0/3111163509 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.420+0000 7fa8ceffd640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8d00d2350 0x7fa8d015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.420+0000 7fa8cf7fe640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8d00b7310 0x7fa8d0150240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.421+0000 7fa8cffff640 1 -- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8d00b7310 msgr2=0x7fa8d0150240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.421+0000 7fa8cffff640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8d00b7310 0x7fa8d0150240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.421+0000 7fa8cffff640 1 -- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8d00d2350 msgr2=0x7fa8d015aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.421+0000 7fa8cffff640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8d00d2350 0x7fa8d015aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.421+0000 7fa8cffff640 1 -- 172.21.15.27:0/3111163509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c00094d0 con 0x7fa8d01604d0 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.421+0000 7fa8cffff640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8d01604d0 0x7fa8d00c77f0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800e7f0 tx=0x7fa8c800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.421+0000 7fa8b7fff640 1 -- 172.21.15.27:0/3111163509 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c80094b0 con 0x7fa8d01604d0 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.422+0000 7fa8b7fff640 1 -- 172.21.15.27:0/3111163509 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8c800c730 con 0x7fa8d01604d0 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.422+0000 7fa8b7fff640 1 -- 172.21.15.27:0/3111163509 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c800cae0 con 0x7fa8d01604d0 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.422+0000 7fa8d4865640 1 -- 172.21.15.27:0/3111163509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8d00d0180 con 0x7fa8d01604d0 2023-12-19T18:29:01.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.422+0000 7fa8d4865640 1 -- 172.21.15.27:0/3111163509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d00d0740 con 0x7fa8d01604d0 2023-12-19T18:29:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.423+0000 7fa8d4865640 1 -- 172.21.15.27:0/3111163509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8d0066f80 con 0x7fa8d01604d0 2023-12-19T18:29:01.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.424+0000 7fa8b7fff640 1 -- 172.21.15.27:0/3111163509 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa8c800cc40 con 0x7fa8d01604d0 2023-12-19T18:29:01.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.425+0000 7fa8b7fff640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa8bc06a430 0x7fa8bc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:01.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.425+0000 7fa8b7fff640 1 -- 172.21.15.27:0/3111163509 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5738+0+0 (secure 0 0 0) 0x7fa8c8088980 con 0x7fa8d01604d0 2023-12-19T18:29:01.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.425+0000 7fa8b7fff640 1 -- 172.21.15.27:0/3111163509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=208}) v3 -- 0x7fa8bc071fc0 con 0x7fa8d01604d0 2023-12-19T18:29:01.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.426+0000 7fa8cf7fe640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa8bc06a430 0x7fa8bc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:01.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.427+0000 7fa8cf7fe640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa8bc06a430 0x7fa8bc06c8f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0036000 tx=0x7fa8c0008030 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:01.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.430+0000 7fa8b7fff640 1 -- 172.21.15.27:0/3111163509 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa8c8021060 con 0x7fa8d01604d0 2023-12-19T18:29:01.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.629+0000 7fa8d4865640 1 -- 172.21.15.27:0/3111163509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8d00d0960 con 0x7fa8d01604d0 2023-12-19T18:29:01.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.632+0000 7fa8b7fff640 1 -- 172.21.15.27:0/3111163509 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v207) v1 ==== 74+0+16188 (secure 0 0 0) 0x7fa8c8021070 con 0x7fa8d01604d0 2023-12-19T18:29:01.632 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:29:01.632 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":207,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:28:56.849208+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:28:06.542489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:28:07.623298+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"207","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.26","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:29:01.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.635+0000 7fa8d4865640 1 -- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa8bc06a430 msgr2=0x7fa8bc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:01.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.635+0000 7fa8d4865640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa8bc06a430 0x7fa8bc06c8f0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0036000 tx=0x7fa8c0008030 comp rx=0 tx=0).stop 2023-12-19T18:29:01.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.635+0000 7fa8d4865640 1 -- 172.21.15.27:0/3111163509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8d01604d0 msgr2=0x7fa8d00c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:01.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.635+0000 7fa8d4865640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8d01604d0 0x7fa8d00c77f0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fa8c800e7f0 tx=0x7fa8c800ecb0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.635+0000 7fa8d4865640 1 -- 172.21.15.27:0/3111163509 shutdown_connections 2023-12-19T18:29:01.635 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.635+0000 7fa8d4865640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa8bc06a430 0x7fa8bc06c8f0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.635+0000 7fa8d4865640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8d01604d0 0x7fa8d00c77f0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.635+0000 7fa8d4865640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa8d00d2350 0x7fa8d015aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.635+0000 7fa8d4865640 1 --2- 172.21.15.27:0/3111163509 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8d00b7310 0x7fa8d0150240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.635+0000 7fa8d4865640 1 -- 172.21.15.27:0/3111163509 >> 172.21.15.27:0/3111163509 conn(0x7fa8d00cf7e0 msgr2=0x7fa8d00b2710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.635+0000 7fa8d4865640 1 -- 172.21.15.27:0/3111163509 shutdown_connections 2023-12-19T18:29:01.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.635+0000 7fa8d4865640 1 -- 172.21.15.27:0/3111163509 wait complete. 2023-12-19T18:29:01.656 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:29:01.657 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:29:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.888+0000 7ff373fff640 1 Processor -- start 2023-12-19T18:29:01.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.889+0000 7ff373fff640 1 -- start start 2023-12-19T18:29:01.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.891+0000 7ff373fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3740b4ef0 0x7ff3740b6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:01.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.891+0000 7ff373fff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff3740d2350 0x7ff3740b6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:01.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.891+0000 7ff373fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3740b5e90 con 0x7ff3741604d0 2023-12-19T18:29:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.891+0000 7ff373fff640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3740d2b30 con 0x7ff3740d2350 2023-12-19T18:29:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.891+0000 7ff373fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740d2850 con 0x7ff3740b4ef0 2023-12-19T18:29:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.891+0000 7ff3737fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff3740d2350 0x7ff3740b6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.891+0000 7ff3737fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7ff3740d2350 0x7ff3740b6840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33180/0 (socket says 172.21.15.27:33180) 2023-12-19T18:29:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.891+0000 7ff3737fe640 1 -- 172.21.15.27:0/1276160757 learned_addr learned my addr 172.21.15.27:0/1276160757 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.891+0000 7ff36bfff640 1 --2- 172.21.15.27:0/1276160757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3740b4ef0 0x7ff3740b6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.892+0000 7ff3737fe640 1 -- 172.21.15.27:0/1276160757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3740b4ef0 msgr2=0x7ff3740b6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.892+0000 7ff3737fe640 1 --2- 172.21.15.27:0/1276160757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3740b4ef0 0x7ff3740b6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.892+0000 7ff3737fe640 1 -- 172.21.15.27:0/1276160757 >> v1:172.21.15.192:6789/0 conn(0x7ff3741604d0 legacy=0x7ff3740b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.892+0000 7ff3737fe640 1 -- 172.21.15.27:0/1276160757 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3740cded0 con 0x7ff3740d2350 2023-12-19T18:29:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.893+0000 7ff3737fe640 1 --2- 172.21.15.27:0/1276160757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3740d2350 0x7ff3740b6840 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7ff36c009e50 tx=0x7ff36c02bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72bf2690fbff16dc server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.893+0000 7ff3727fc640 1 -- 172.21.15.27:0/1276160757 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c02ca70 con 0x7ff3740d2350 2023-12-19T18:29:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.893+0000 7ff3727fc640 1 -- 172.21.15.27:0/1276160757 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c02cbd0 con 0x7ff3740d2350 2023-12-19T18:29:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.893+0000 7ff3727fc640 1 -- 172.21.15.27:0/1276160757 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c0343f0 con 0x7ff3740d2350 2023-12-19T18:29:01.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.894+0000 7ff373fff640 1 -- 172.21.15.27:0/1276160757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3740d2350 msgr2=0x7ff3740b6840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:01.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.894+0000 7ff373fff640 1 --2- 172.21.15.27:0/1276160757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3740d2350 0x7ff3740b6840 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7ff36c009e50 tx=0x7ff36c02bed0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.894+0000 7ff373fff640 1 -- 172.21.15.27:0/1276160757 shutdown_connections 2023-12-19T18:29:01.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.894+0000 7ff373fff640 1 --2- 172.21.15.27:0/1276160757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3740d2350 0x7ff3740b6840 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.894+0000 7ff373fff640 1 --2- 172.21.15.27:0/1276160757 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3740b4ef0 0x7ff3740b6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.894+0000 7ff373fff640 1 -- 172.21.15.27:0/1276160757 >> 172.21.15.27:0/1276160757 conn(0x7ff3740cf7e0 msgr2=0x7ff3740cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:01.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.894+0000 7ff373fff640 1 -- 172.21.15.27:0/1276160757 shutdown_connections 2023-12-19T18:29:01.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.894+0000 7ff373fff640 1 -- 172.21.15.27:0/1276160757 wait complete. 2023-12-19T18:29:01.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.896+0000 7ff373fff640 1 Processor -- start 2023-12-19T18:29:01.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.896+0000 7ff373fff640 1 -- start start 2023-12-19T18:29:01.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.897+0000 7ff373fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3740b4ef0 0x7ff3740c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.897+0000 7ff373fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3740d2350 0x7ff3741542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.897+0000 7ff373fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3741604d0 0x7ff3740b71f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.897+0000 7ff373fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740ca660 con 0x7ff3741604d0 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.897+0000 7ff373fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740ca940 con 0x7ff3740b4ef0 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.897+0000 7ff373fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3740cac20 con 0x7ff3740d2350 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.897+0000 7ff3737fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3741604d0 0x7ff3740b71f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.897+0000 7ff3737fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3741604d0 0x7ff3740b71f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33186/0 (socket says 172.21.15.27:33186) 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.897+0000 7ff3737fe640 1 -- 172.21.15.27:0/4155763368 learned_addr learned my addr 172.21.15.27:0/4155763368 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.898+0000 7ff372ffd640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3740b4ef0 0x7ff3740c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.898+0000 7ff36bfff640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3740d2350 0x7ff3741542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.898+0000 7ff3737fe640 1 -- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3740d2350 msgr2=0x7ff3741542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.898+0000 7ff3737fe640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3740d2350 0x7ff3741542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.898+0000 7ff3737fe640 1 -- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3740b4ef0 msgr2=0x7ff3740c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:01.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.898+0000 7ff3737fe640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3740b4ef0 0x7ff3740c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:01.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.898+0000 7ff3737fe640 1 -- 172.21.15.27:0/4155763368 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff36c009b30 con 0x7ff3741604d0 2023-12-19T18:29:01.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.898+0000 7ff3737fe640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3741604d0 0x7ff3740b71f0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7ff36c009e50 tx=0x7ff36c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:01.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.899+0000 7ff370ff9640 1 -- 172.21.15.27:0/4155763368 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c009510 con 0x7ff3741604d0 2023-12-19T18:29:01.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.899+0000 7ff370ff9640 1 -- 172.21.15.27:0/4155763368 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff36c03e450 con 0x7ff3741604d0 2023-12-19T18:29:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.899+0000 7ff370ff9640 1 -- 172.21.15.27:0/4155763368 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c03e7e0 con 0x7ff3741604d0 2023-12-19T18:29:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.899+0000 7ff373fff640 1 -- 172.21.15.27:0/4155763368 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3740c6d20 con 0x7ff3741604d0 2023-12-19T18:29:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.899+0000 7ff373fff640 1 -- 172.21.15.27:0/4155763368 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3741547e0 con 0x7ff3741604d0 2023-12-19T18:29:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.900+0000 7ff373fff640 1 -- 172.21.15.27:0/4155763368 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3740ce190 con 0x7ff3741604d0 2023-12-19T18:29:01.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.901+0000 7ff370ff9640 1 -- 172.21.15.27:0/4155763368 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7ff36c03e940 con 0x7ff3741604d0 2023-12-19T18:29:01.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.902+0000 7ff370ff9640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff35406a430 0x7ff35406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:01.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.903+0000 7ff370ff9640 1 -- 172.21.15.27:0/4155763368 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5738+0+0 (secure 0 0 0) 0x7ff36c0ab740 con 0x7ff3741604d0 2023-12-19T18:29:01.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.903+0000 7ff370ff9640 1 -- 172.21.15.27:0/4155763368 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=208}) v3 -- 0x7ff354071fc0 con 0x7ff3741604d0 2023-12-19T18:29:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.903+0000 7ff372ffd640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff35406a430 0x7ff35406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:01.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.904+0000 7ff372ffd640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff35406a430 0x7ff35406c8f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff364004ea0 tx=0x7ff364009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:01.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:01.907+0000 7ff370ff9640 1 -- 172.21.15.27:0/4155763368 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7ff3740ce190 con 0x7ff3741604d0 2023-12-19T18:29:02.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.140+0000 7ff373fff640 1 -- 172.21.15.27:0/4155763368 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3740ce190 con 0x7ff3741604d0 2023-12-19T18:29:02.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.141+0000 7ff370ff9640 1 -- 172.21.15.27:0/4155763368 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4713 (secure 0 0 0) 0x7ff3740ce190 con 0x7ff3741604d0 2023-12-19T18:29:02.143 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:29:02.143 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1864159643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1864159643},{"type":"v1","addr":"172.21.15.192:6836","nonce":1864159643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1348219767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1348219767},{"type":"v1","addr":"172.21.15.192:6837","nonce":1348219767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/181137639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":181137639},{"type":"v1","addr":"172.21.15.27:6833","nonce":181137639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:28:08.577742+0000","modified":"2023-12-19T18:28:12.083302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/152568921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":152568921},{"type":"v1","addr":"172.21.15.27:6835","nonce":152568921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:29:02.150 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 85 2023-12-19T18:29:02.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.144+0000 7ff373fff640 1 -- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff35406a430 msgr2=0x7ff35406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:02.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.144+0000 7ff373fff640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff35406a430 0x7ff35406c8f0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff364004ea0 tx=0x7ff364009040 comp rx=0 tx=0).stop 2023-12-19T18:29:02.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.144+0000 7ff373fff640 1 -- 172.21.15.27:0/4155763368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3741604d0 msgr2=0x7ff3740b71f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:02.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.144+0000 7ff373fff640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3741604d0 0x7ff3740b71f0 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7ff36c009e50 tx=0x7ff36c004a40 comp rx=0 tx=0).stop 2023-12-19T18:29:02.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.144+0000 7ff373fff640 1 -- 172.21.15.27:0/4155763368 shutdown_connections 2023-12-19T18:29:02.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.144+0000 7ff373fff640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7ff35406a430 0x7ff35406c8f0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.144+0000 7ff373fff640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3741604d0 0x7ff3740b71f0 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.144+0000 7ff373fff640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7ff3740d2350 0x7ff3741542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.144+0000 7ff373fff640 1 --2- 172.21.15.27:0/4155763368 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7ff3740b4ef0 0x7ff3740c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.144+0000 7ff373fff640 1 -- 172.21.15.27:0/4155763368 >> 172.21.15.27:0/4155763368 conn(0x7ff3740cf7e0 msgr2=0x7ff3740cfbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:02.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.144+0000 7ff373fff640 1 -- 172.21.15.27:0/4155763368 shutdown_connections 2023-12-19T18:29:02.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.144+0000 7ff373fff640 1 -- 172.21.15.27:0/4155763368 wait complete. 2023-12-19T18:29:02.164 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:29:02.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.403+0000 7f64392fa640 1 Processor -- start 2023-12-19T18:29:02.403 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.403+0000 7f64392fa640 1 -- start start 2023-12-19T18:29:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.405+0000 7f64392fa640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64340afac0 0x7f64340c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.405+0000 7f64392fa640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64340cc190 con 0x7f6434154030 2023-12-19T18:29:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.405+0000 7f64392fa640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64340aff30 con 0x7f64340af630 2023-12-19T18:29:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.405+0000 7f64392fa640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64340cdc00 con 0x7f64340afac0 2023-12-19T18:29:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.405+0000 7f6433fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6434154030 0x7f64340cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53326/0 (socket says 172.21.15.27:53326) 2023-12-19T18:29:02.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.405+0000 7f6433fff640 1 -- 172.21.15.27:0/2046267826 learned_addr learned my addr 172.21.15.27:0/2046267826 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:02.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6438af9640 1 --2- 172.21.15.27:0/2046267826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64340afac0 0x7f64340c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013180728 0 0) 0x7f64340cc190 con 0x7f6434154030 2023-12-19T18:29:02.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f641c003520 con 0x7f6434154030 2023-12-19T18:29:02.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3871019114 0 0) 0x7f641c003520 con 0x7f6434154030 2023-12-19T18:29:02.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64340afac0 msgr2=0x7f64340c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:02.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 --2- 172.21.15.27:0/2046267826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64340afac0 0x7f64340c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 >> v1:172.21.15.192:6789/0 conn(0x7f64340af630 legacy=0x7f64340c4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64340cb2a0 con 0x7f6434154030 2023-12-19T18:29:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f6424001670 con 0x7f6434154030 2023-12-19T18:29:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 >> v1:172.21.15.27:6789/0 conn(0x7f6434154030 legacy=0x7f64340cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 --2- 172.21.15.27:0/2046267826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f641c0039a0 0x7f641c003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 --2- 172.21.15.27:0/2046267826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641c0047c0 0x7f641c004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 --2- 172.21.15.27:0/2046267826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64340afac0 0x7f641c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64340cdc00 con 0x7f641c0047c0 2023-12-19T18:29:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641c00ba30 con 0x7f641c0039a0 2023-12-19T18:29:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.406+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f641c00bbf0 con 0x7f64340afac0 2023-12-19T18:29:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.407+0000 7f64337fe640 1 --2- 172.21.15.27:0/2046267826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641c0047c0 0x7f641c004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.407+0000 7f6433fff640 1 --2- 172.21.15.27:0/2046267826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f641c0039a0 0x7f641c003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.407+0000 7f6438af9640 1 --2- 172.21.15.27:0/2046267826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64340afac0 0x7f641c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.407+0000 7f64337fe640 1 -- 172.21.15.27:0/2046267826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64340afac0 msgr2=0x7f641c005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.407+0000 7f64337fe640 1 --2- 172.21.15.27:0/2046267826 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64340afac0 0x7f641c005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.407+0000 7f64337fe640 1 -- 172.21.15.27:0/2046267826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f641c0039a0 msgr2=0x7f641c003dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.407+0000 7f64337fe640 1 --2- 172.21.15.27:0/2046267826 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f641c0039a0 0x7f641c003dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.407+0000 7f64337fe640 1 -- 172.21.15.27:0/2046267826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f641c003520 con 0x7f641c0047c0 2023-12-19T18:29:02.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.408+0000 7f64337fe640 1 --2- 172.21.15.27:0/2046267826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641c0047c0 0x7f641c004c40 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f6428001720 tx=0x7f641c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d37cc9f37d6d644f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:02.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.409+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280052d0 con 0x7f641c0047c0 2023-12-19T18:29:02.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.409+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f642802e050 con 0x7f641c0047c0 2023-12-19T18:29:02.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.409+0000 7f6432ffd640 1 -- 172.21.15.27:0/2046267826 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280035f0 con 0x7f641c0047c0 2023-12-19T18:29:02.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.409+0000 7f64392fa640 1 -- 172.21.15.27:0/2046267826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641c0047c0 msgr2=0x7f641c004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:02.410 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.409+0000 7f64392fa640 1 --2- 172.21.15.27:0/2046267826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f641c0047c0 0x7f641c004c40 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7f6428001720 tx=0x7f641c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.409+0000 7f6438af9640 1 -- 172.21.15.27:0/2046267826 reap_dead start 2023-12-19T18:29:02.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.409+0000 7f64392fa640 1 -- 172.21.15.27:0/2046267826 shutdown_connections 2023-12-19T18:29:02.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.409+0000 7f64392fa640 1 -- 172.21.15.27:0/2046267826 >> 172.21.15.27:0/2046267826 conn(0x7f6434069c70 msgr2=0x7f643406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:02.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.409+0000 7f64392fa640 1 -- 172.21.15.27:0/2046267826 shutdown_connections 2023-12-19T18:29:02.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.410+0000 7f64392fa640 1 -- 172.21.15.27:0/2046267826 wait complete. 2023-12-19T18:29:02.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.411+0000 7f64392fa640 1 Processor -- start 2023-12-19T18:29:02.411 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.412+0000 7f64392fa640 1 -- start start 2023-12-19T18:29:02.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64392fa640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f641c0047c0 0x7f643414b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64392fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f643414b5e0 0x7f643414a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64392fa640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64341e64b0 0x7f64341e6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64392fa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64341e6e70 con 0x7f643414b5e0 2023-12-19T18:29:02.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64392fa640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f643414c000 con 0x7f641c0047c0 2023-12-19T18:29:02.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64392fa640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64340b29f0 con 0x7f64341e64b0 2023-12-19T18:29:02.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f643414b5e0 0x7f643414a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64337fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f643414b5e0 0x7f643414a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39526/0 (socket says 172.21.15.27:39526) 2023-12-19T18:29:02.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64337fe640 1 -- 172.21.15.27:0/3731148245 learned_addr learned my addr 172.21.15.27:0/3731148245 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:02.414 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f6433fff640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f641c0047c0 0x7f643414b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f6438af9640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64341e64b0 0x7f64341e6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64337fe640 1 -- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64341e64b0 msgr2=0x7f64341e6930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64337fe640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64341e64b0 0x7f64341e6930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64337fe640 1 -- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f641c0047c0 msgr2=0x7f643414b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64337fe640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f641c0047c0 0x7f643414b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f64337fe640 1 -- 172.21.15.27:0/3731148245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f643406f3c0 con 0x7f643414b5e0 2023-12-19T18:29:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.413+0000 7f6438af9640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64341e64b0 0x7f64341e6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.414+0000 7f64337fe640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f643414b5e0 0x7f643414a7b0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f6428008fd0 tx=0x7f6428001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.414+0000 7f6430ff9640 1 -- 172.21.15.27:0/3731148245 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64280076c0 con 0x7f643414b5e0 2023-12-19T18:29:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.414+0000 7f6430ff9640 1 -- 172.21.15.27:0/3731148245 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6428007a30 con 0x7f643414b5e0 2023-12-19T18:29:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.414+0000 7f6430ff9640 1 -- 172.21.15.27:0/3731148245 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6428006be0 con 0x7f643414b5e0 2023-12-19T18:29:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.414+0000 7f64392fa640 1 -- 172.21.15.27:0/3731148245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f643406f6a0 con 0x7f643414b5e0 2023-12-19T18:29:02.415 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.414+0000 7f64392fa640 1 -- 172.21.15.27:0/3731148245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f643406fc60 con 0x7f643414b5e0 2023-12-19T18:29:02.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.415+0000 7f64392fa640 1 -- 172.21.15.27:0/3731148245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f643404e840 con 0x7f643414b5e0 2023-12-19T18:29:02.416 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.416+0000 7f6430ff9640 1 -- 172.21.15.27:0/3731148245 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6428006d40 con 0x7f643414b5e0 2023-12-19T18:29:02.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.417+0000 7f6430ff9640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f642006a460 0x7f642006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.417 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.417+0000 7f6430ff9640 1 -- 172.21.15.27:0/3731148245 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f64280aaa40 con 0x7f643414b5e0 2023-12-19T18:29:02.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.418+0000 7f6430ff9640 1 -- 172.21.15.27:0/3731148245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=208}) v3 -- 0x7f6420071ff0 con 0x7f643414b5e0 2023-12-19T18:29:02.418 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.418+0000 7f6433fff640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f642006a460 0x7f642006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.419 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.419+0000 7f6433fff640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f642006a460 0x7f642006c920 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6424001050 tx=0x7f6424009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:02.421 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.422+0000 7f6430ff9640 1 -- 172.21.15.27:0/3731148245 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f642804b060 con 0x7f643414b5e0 2023-12-19T18:29:02.654 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.654+0000 7f64392fa640 1 -- 172.21.15.27:0/3731148245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f64340ca5c0 con 0x7f643414b5e0 2023-12-19T18:29:02.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.655+0000 7f6430ff9640 1 -- 172.21.15.27:0/3731148245 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 85 v85) v1 ==== 94+0+4713 (secure 0 0 0) 0x7f6428076120 con 0x7f643414b5e0 2023-12-19T18:29:02.655 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:29:02.655 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":85,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1864159643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1864159643},{"type":"v1","addr":"172.21.15.192:6836","nonce":1864159643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1348219767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1348219767},{"type":"v1","addr":"172.21.15.192:6837","nonce":1348219767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/181137639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":181137639},{"type":"v1","addr":"172.21.15.27:6833","nonce":181137639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80}],"filesystems":[{"mdsmap":{"epoch":85,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-19T18:28:08.577742+0000","modified":"2023-12-19T18:28:12.083302+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8997},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8997":{"gid":8997,"name":"c","rank":0,"incarnation":81,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6835/152568921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":152568921},{"type":"v1","addr":"172.21.15.27:6835","nonce":152568921}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-19T18:29:02.655 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 85 2023-12-19T18:29:02.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.657+0000 7f64392fa640 1 -- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f642006a460 msgr2=0x7f642006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:02.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.657+0000 7f64392fa640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f642006a460 0x7f642006c920 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f6424001050 tx=0x7f6424009040 comp rx=0 tx=0).stop 2023-12-19T18:29:02.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.657+0000 7f64392fa640 1 -- 172.21.15.27:0/3731148245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f643414b5e0 msgr2=0x7f643414a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:02.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.657+0000 7f64392fa640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f643414b5e0 0x7f643414a7b0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f6428008fd0 tx=0x7f6428001120 comp rx=0 tx=0).stop 2023-12-19T18:29:02.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.658+0000 7f64392fa640 1 -- 172.21.15.27:0/3731148245 shutdown_connections 2023-12-19T18:29:02.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.658+0000 7f64392fa640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f642006a460 0x7f642006c920 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.658+0000 7f64392fa640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f64341e64b0 0x7f64341e6930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.658+0000 7f64392fa640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f643414b5e0 0x7f643414a7b0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.658+0000 7f64392fa640 1 --2- 172.21.15.27:0/3731148245 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f641c0047c0 0x7f643414b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.658+0000 7f64392fa640 1 -- 172.21.15.27:0/3731148245 >> 172.21.15.27:0/3731148245 conn(0x7f6434069c70 msgr2=0x7f643406c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:02.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.658+0000 7f64392fa640 1 -- 172.21.15.27:0/3731148245 shutdown_connections 2023-12-19T18:29:02.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.658+0000 7f64392fa640 1 -- 172.21.15.27:0/3731148245 wait complete. 2023-12-19T18:29:02.679 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:29:02.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.914+0000 7f51863cb640 1 Processor -- start 2023-12-19T18:29:02.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.914+0000 7f51863cb640 1 -- start start 2023-12-19T18:29:02.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.916+0000 7f51863cb640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f51800b4f80 0x7f51800d09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.916+0000 7f51863cb640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f51800d2370 0x7f51800d0f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.916+0000 7f51863cb640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51800d2ba0 con 0x7f5180160780 2023-12-19T18:29:02.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.916+0000 7f51863cb640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51800d0830 con 0x7f51800d2370 2023-12-19T18:29:02.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.916+0000 7f51863cb640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51800d21a0 con 0x7f51800b4f80 2023-12-19T18:29:02.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.916+0000 7f517ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f5180160780 0x7f51800b4870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53342/0 (socket says 172.21.15.27:53342) 2023-12-19T18:29:02.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.916+0000 7f517ffff640 1 -- 172.21.15.27:0/2308642377 learned_addr learned my addr 172.21.15.27:0/2308642377 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.916+0000 7f5184968640 1 --2- 172.21.15.27:0/2308642377 >> v2:172.21.15.27:3300/0 conn(0x7f51800d2370 0x7f51800d0f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.916+0000 7f5177fff640 1 --2- 172.21.15.27:0/2308642377 >> v2:172.21.15.192:3300/0 conn(0x7f51800b4f80 0x7f51800d09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.917+0000 7f517f7fe640 1 -- 172.21.15.27:0/2308642377 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262531889 0 0) 0x7f51800d2ba0 con 0x7f5180160780 2023-12-19T18:29:02.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.917+0000 7f517f7fe640 1 -- 172.21.15.27:0/2308642377 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f516c003520 con 0x7f5180160780 2023-12-19T18:29:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.917+0000 7f5184968640 1 -- 172.21.15.27:0/2308642377 >> v2:172.21.15.192:3300/0 conn(0x7f51800b4f80 msgr2=0x7f51800d09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.917+0000 7f5184968640 1 --2- 172.21.15.27:0/2308642377 >> v2:172.21.15.192:3300/0 conn(0x7f51800b4f80 0x7f51800d09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.917+0000 7f5184968640 1 -- 172.21.15.27:0/2308642377 >> v1:172.21.15.27:6789/0 conn(0x7f5180160780 legacy=0x7f51800b4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.917+0000 7f5184968640 1 -- 172.21.15.27:0/2308642377 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51800b6ef0 con 0x7f51800d2370 2023-12-19T18:29:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.918+0000 7f5184968640 1 --2- 172.21.15.27:0/2308642377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51800d2370 0x7f51800d0f00 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f51800661f0 tx=0x7f517802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a1b6452b054dfe server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:02.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.918+0000 7f517f7fe640 1 -- 172.21.15.27:0/2308642377 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178031830 con 0x7f51800d2370 2023-12-19T18:29:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.918+0000 7f517f7fe640 1 -- 172.21.15.27:0/2308642377 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5178031990 con 0x7f51800d2370 2023-12-19T18:29:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.918+0000 7f517f7fe640 1 -- 172.21.15.27:0/2308642377 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5178031d20 con 0x7f51800d2370 2023-12-19T18:29:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.919+0000 7f51863cb640 1 -- 172.21.15.27:0/2308642377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51800d2370 msgr2=0x7f51800d0f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:02.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.919+0000 7f51863cb640 1 --2- 172.21.15.27:0/2308642377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51800d2370 0x7f51800d0f00 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7f51800661f0 tx=0x7f517802b8d0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.919+0000 7f51863cb640 1 -- 172.21.15.27:0/2308642377 shutdown_connections 2023-12-19T18:29:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.919+0000 7f51863cb640 1 --2- 172.21.15.27:0/2308642377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51800d2370 0x7f51800d0f00 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.919+0000 7f51863cb640 1 --2- 172.21.15.27:0/2308642377 >> v2:172.21.15.192:3300/0 conn(0x7f51800b4f80 0x7f51800d09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.919+0000 7f51863cb640 1 -- 172.21.15.27:0/2308642377 >> 172.21.15.27:0/2308642377 conn(0x7f51800cf870 msgr2=0x7f51800cfc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.919+0000 7f51863cb640 1 -- 172.21.15.27:0/2308642377 shutdown_connections 2023-12-19T18:29:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.919+0000 7f51863cb640 1 -- 172.21.15.27:0/2308642377 wait complete. 2023-12-19T18:29:02.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.921+0000 7f51863cb640 1 Processor -- start 2023-12-19T18:29:02.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.921+0000 7f51863cb640 1 -- start start 2023-12-19T18:29:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.922+0000 7f51863cb640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51800b4f80 0x7f518014fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.922+0000 7f51863cb640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51800d2370 0x7f51800cccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.922+0000 7f51863cb640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5180160780 0x7f51801e6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.922+0000 7f51863cb640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51800c7d90 con 0x7f51800b4f80 2023-12-19T18:29:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.922+0000 7f51863cb640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5180150330 con 0x7f51800d2370 2023-12-19T18:29:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.922+0000 7f51863cb640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51800ca060 con 0x7f5180160780 2023-12-19T18:29:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.923+0000 7f517ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51800b4f80 0x7f518014fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.923+0000 7f517ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51800b4f80 0x7f518014fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39548/0 (socket says 172.21.15.27:39548) 2023-12-19T18:29:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.923+0000 7f517ffff640 1 -- 172.21.15.27:0/1229564178 learned_addr learned my addr 172.21.15.27:0/1229564178 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.923+0000 7f5177fff640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51800d2370 0x7f51800cccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.923+0000 7f5184968640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5180160780 0x7f51801e6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.923+0000 7f517ffff640 1 -- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5180160780 msgr2=0x7f51801e6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:02.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.923+0000 7f517ffff640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5180160780 0x7f51801e6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.923+0000 7f517ffff640 1 -- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51800d2370 msgr2=0x7f51800cccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:02.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.923+0000 7f517ffff640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51800d2370 0x7f51800cccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:02.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.923+0000 7f517ffff640 1 -- 172.21.15.27:0/1229564178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5178009400 con 0x7f51800b4f80 2023-12-19T18:29:02.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.923+0000 7f5177fff640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51800d2370 0x7f51800cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:02.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.924+0000 7f517ffff640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51800b4f80 0x7f518014fdf0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f517000cbc0 tx=0x7f517000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:02.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.924+0000 7f517d7fa640 1 -- 172.21.15.27:0/1229564178 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517000ad40 con 0x7f51800b4f80 2023-12-19T18:29:02.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.924+0000 7f517d7fa640 1 -- 172.21.15.27:0/1229564178 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5170012400 con 0x7f51800b4f80 2023-12-19T18:29:02.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.924+0000 7f517d7fa640 1 -- 172.21.15.27:0/1229564178 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f517000aea0 con 0x7f51800b4f80 2023-12-19T18:29:02.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.924+0000 7f51863cb640 1 -- 172.21.15.27:0/1229564178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51800ca340 con 0x7f51800b4f80 2023-12-19T18:29:02.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.924+0000 7f51863cb640 1 -- 172.21.15.27:0/1229564178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51800c8280 con 0x7f51800b4f80 2023-12-19T18:29:02.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.925+0000 7f51863cb640 1 -- 172.21.15.27:0/1229564178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5180068e90 con 0x7f51800b4f80 2023-12-19T18:29:02.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.926+0000 7f517d7fa640 1 -- 172.21.15.27:0/1229564178 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f5170007070 con 0x7f51800b4f80 2023-12-19T18:29:02.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.928+0000 7f517d7fa640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f516006a460 0x7f516006c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:02.927 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.928+0000 7f517d7fa640 1 -- 172.21.15.27:0/1229564178 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f5170088930 con 0x7f51800b4f80 2023-12-19T18:29:02.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.928+0000 7f517d7fa640 1 -- 172.21.15.27:0/1229564178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=208}) v3 -- 0x7f5160071ff0 con 0x7f51800b4f80 2023-12-19T18:29:02.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.928+0000 7f5177fff640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f516006a460 0x7f516006c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:02.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.929+0000 7f5177fff640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f516006a460 0x7f516006c920 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f518015ca80 tx=0x7f5168008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:02.932+0000 7f517d7fa640 1 -- 172.21.15.27:0/1229564178 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f5170017020 con 0x7f51800b4f80 2023-12-19T18:29:03.126 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.126+0000 7f51863cb640 1 -- 172.21.15.27:0/1229564178 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51800ca4d0 con 0x7f51800b4f80 2023-12-19T18:29:03.129 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.129+0000 7f517d7fa640 1 -- 172.21.15.27:0/1229564178 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v207) v1 ==== 74+0+16188 (secure 0 0 0) 0x7f5170053f00 con 0x7f51800b4f80 2023-12-19T18:29:03.129 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:29:03.129 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":207,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:28:56.849208+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-19T18:28:06.542489+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"201","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-19T18:28:07.623298+0000","flags":1027,"flags_names":"hashpspool,full,full_quota","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"207","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":33554432,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"23.26","mappings":[{"from":5,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:29:03.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.131+0000 7f51863cb640 1 -- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f516006a460 msgr2=0x7f516006c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:03.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.131+0000 7f51863cb640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f516006a460 0x7f516006c920 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f518015ca80 tx=0x7f5168008040 comp rx=0 tx=0).stop 2023-12-19T18:29:03.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.131+0000 7f51863cb640 1 -- 172.21.15.27:0/1229564178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51800b4f80 msgr2=0x7f518014fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:03.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.131+0000 7f51863cb640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51800b4f80 0x7f518014fdf0 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f517000cbc0 tx=0x7f517000a430 comp rx=0 tx=0).stop 2023-12-19T18:29:03.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.132+0000 7f51863cb640 1 -- 172.21.15.27:0/1229564178 shutdown_connections 2023-12-19T18:29:03.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.132+0000 7f51863cb640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f516006a460 0x7f516006c920 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.132+0000 7f51863cb640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f5180160780 0x7f51801e6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.132+0000 7f51863cb640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f51800d2370 0x7f51800cccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.132 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.132+0000 7f51863cb640 1 --2- 172.21.15.27:0/1229564178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51800b4f80 0x7f518014fdf0 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.132+0000 7f51863cb640 1 -- 172.21.15.27:0/1229564178 >> 172.21.15.27:0/1229564178 conn(0x7f51800cf870 msgr2=0x7f51800b4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:03.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.132+0000 7f51863cb640 1 -- 172.21.15.27:0/1229564178 shutdown_connections 2023-12-19T18:29:03.133 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.132+0000 7f51863cb640 1 -- 172.21.15.27:0/1229564178 wait complete. 2023-12-19T18:29:03.153 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-19T18:29:03.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.381+0000 7f3f9359e640 1 Processor -- start 2023-12-19T18:29:03.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.381+0000 7f3f9359e640 1 -- start start 2023-12-19T18:29:03.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.383+0000 7f3f9359e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f3f94070fb0 0x7f3f9406b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:03.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.383+0000 7f3f9359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f9406b850 0x7f3f9406f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:03.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.383+0000 7f3f9359e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f940adf10 con 0x7f3f9415ac50 2023-12-19T18:29:03.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.383+0000 7f3f9359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f940adac0 con 0x7f3f9406b850 2023-12-19T18:29:03.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.383+0000 7f3f9359e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f940af630 con 0x7f3f94070fb0 2023-12-19T18:29:03.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.383+0000 7f3f92d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f9406b850 0x7f3f9406f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:03.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.383+0000 7f3f92d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3f9406b850 0x7f3f9406f650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39550/0 (socket says 172.21.15.27:39550) 2023-12-19T18:29:03.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.383+0000 7f3f92d9d640 1 -- 172.21.15.27:0/956055755 learned_addr learned my addr 172.21.15.27:0/956055755 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:03.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.383+0000 7f3f91d9b640 1 --2- 172.21.15.27:0/956055755 >> v2:172.21.15.192:3300/0 conn(0x7f3f94070fb0 0x7f3f9406b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:03.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.384+0000 7f3f92d9d640 1 -- 172.21.15.27:0/956055755 >> v2:172.21.15.192:3300/0 conn(0x7f3f94070fb0 msgr2=0x7f3f9406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:03.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.384+0000 7f3f92d9d640 1 --2- 172.21.15.27:0/956055755 >> v2:172.21.15.192:3300/0 conn(0x7f3f94070fb0 0x7f3f9406b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.384+0000 7f3f92d9d640 1 -- 172.21.15.27:0/956055755 >> v1:172.21.15.192:6789/0 conn(0x7f3f9415ac50 legacy=0x7f3f940aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:03.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.384+0000 7f3f92d9d640 1 -- 172.21.15.27:0/956055755 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f940713f0 con 0x7f3f9406b850 2023-12-19T18:29:03.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.385+0000 7f3f92d9d640 1 --2- 172.21.15.27:0/956055755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f9406b850 0x7f3f9406f650 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f3f94071730 tx=0x7f3f8c02b7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a11f27f9848a9ee server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:03.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.385+0000 7f3f9159a640 1 -- 172.21.15.27:0/956055755 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c031750 con 0x7f3f9406b850 2023-12-19T18:29:03.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.385+0000 7f3f9159a640 1 -- 172.21.15.27:0/956055755 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8c0318b0 con 0x7f3f9406b850 2023-12-19T18:29:03.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.385+0000 7f3f9159a640 1 -- 172.21.15.27:0/956055755 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c031c40 con 0x7f3f9406b850 2023-12-19T18:29:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.386+0000 7f3f9359e640 1 -- 172.21.15.27:0/956055755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f9406b850 msgr2=0x7f3f9406f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.386+0000 7f3f9359e640 1 --2- 172.21.15.27:0/956055755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f9406b850 0x7f3f9406f650 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f3f94071730 tx=0x7f3f8c02b7f0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.386+0000 7f3f9359e640 1 -- 172.21.15.27:0/956055755 shutdown_connections 2023-12-19T18:29:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.386+0000 7f3f9359e640 1 --2- 172.21.15.27:0/956055755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f9406b850 0x7f3f9406f650 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.386+0000 7f3f9359e640 1 --2- 172.21.15.27:0/956055755 >> v2:172.21.15.192:3300/0 conn(0x7f3f94070fb0 0x7f3f9406b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.386+0000 7f3f9359e640 1 -- 172.21.15.27:0/956055755 >> 172.21.15.27:0/956055755 conn(0x7f3f9406a7d0 msgr2=0x7f3f940ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:03.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.386+0000 7f3f9359e640 1 -- 172.21.15.27:0/956055755 shutdown_connections 2023-12-19T18:29:03.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.386+0000 7f3f9359e640 1 -- 172.21.15.27:0/956055755 wait complete. 2023-12-19T18:29:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.388+0000 7f3f9359e640 1 Processor -- start 2023-12-19T18:29:03.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.388+0000 7f3f9359e640 1 -- start start 2023-12-19T18:29:03.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.389+0000 7f3f9359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f9406b850 0x7f3f940b3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:03.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.389+0000 7f3f9359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f94070fb0 0x7f3f94145880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:03.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.389+0000 7f3f9359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f9415ac50 0x7f3f94153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:03.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.389+0000 7f3f9359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f94153490 con 0x7f3f9415ac50 2023-12-19T18:29:03.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.389+0000 7f3f9359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f940b3a80 con 0x7f3f94070fb0 2023-12-19T18:29:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.389+0000 7f3f9359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f941528c0 con 0x7f3f9406b850 2023-12-19T18:29:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.389+0000 7f3f92d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f9415ac50 0x7f3f94153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.390+0000 7f3f92d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f9415ac50 0x7f3f94153920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39560/0 (socket says 172.21.15.27:39560) 2023-12-19T18:29:03.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.390+0000 7f3f92d9d640 1 -- 172.21.15.27:0/605547673 learned_addr learned my addr 172.21.15.27:0/605547673 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.390+0000 7f3f91d9b640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f94070fb0 0x7f3f94145880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.390+0000 7f3f9259c640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f9406b850 0x7f3f940b3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.390+0000 7f3f92d9d640 1 -- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f9406b850 msgr2=0x7f3f940b3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.390+0000 7f3f92d9d640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f9406b850 0x7f3f940b3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.390+0000 7f3f92d9d640 1 -- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f94070fb0 msgr2=0x7f3f94145880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.390+0000 7f3f92d9d640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f94070fb0 0x7f3f94145880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.390+0000 7f3f92d9d640 1 -- 172.21.15.27:0/605547673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f8c0094d0 con 0x7f3f9415ac50 2023-12-19T18:29:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.390+0000 7f3f9259c640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f9406b850 0x7f3f940b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.391+0000 7f3f92d9d640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f9415ac50 0x7f3f94153920 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c02b830 tx=0x7f3f8c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:03.391 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.391+0000 7f3f8affd640 1 -- 172.21.15.27:0/605547673 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c033c80 con 0x7f3f9415ac50 2023-12-19T18:29:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.391+0000 7f3f8affd640 1 -- 172.21.15.27:0/605547673 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f8c033de0 con 0x7f3f9415ac50 2023-12-19T18:29:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.391+0000 7f3f8affd640 1 -- 172.21.15.27:0/605547673 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c03c640 con 0x7f3f9415ac50 2023-12-19T18:29:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.391+0000 7f3f9359e640 1 -- 172.21.15.27:0/605547673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f94152b60 con 0x7f3f9415ac50 2023-12-19T18:29:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.391+0000 7f3f9359e640 1 -- 172.21.15.27:0/605547673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f94153120 con 0x7f3f9415ac50 2023-12-19T18:29:03.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.392+0000 7f3f9359e640 1 -- 172.21.15.27:0/605547673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f94153640 con 0x7f3f9415ac50 2023-12-19T18:29:03.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.393+0000 7f3f8affd640 1 -- 172.21.15.27:0/605547673 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3f8c03a040 con 0x7f3f9415ac50 2023-12-19T18:29:03.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.394+0000 7f3f8affd640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3f7406a430 0x7f3f7406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:03.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.394+0000 7f3f8affd640 1 -- 172.21.15.27:0/605547673 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5738+0+0 (secure 0 0 0) 0x7f3f8c0aa450 con 0x7f3f9415ac50 2023-12-19T18:29:03.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.395+0000 7f3f8affd640 1 -- 172.21.15.27:0/605547673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=208}) v3 -- 0x7f3f74071fc0 con 0x7f3f9415ac50 2023-12-19T18:29:03.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.395+0000 7f3f9259c640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3f7406a430 0x7f3f7406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:03.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.396+0000 7f3f9259c640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3f7406a430 0x7f3f7406c8f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3f84003eb0 tx=0x7f3f84009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:03.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.399+0000 7f3f8affd640 1 -- 172.21.15.27:0/605547673 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f3f94153640 con 0x7f3f9415ac50 2023-12-19T18:29:03.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.630+0000 7f3f9359e640 1 -- 172.21.15.27:0/605547673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f3f9404e840 con 0x7f3f9415ac50 2023-12-19T18:29:03.631 INFO:tasks.ceph.mon.a.smithi027.stderr:2023-12-19T18:29:03.631+0000 7f8f3865c640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-19T18:29:03.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.634+0000 7f3f8affd640 1 -- 172.21.15.27:0/605547673 <== mon.0 v2:172.21.15.27:3300/0 7 ==== osd_map(208..208 src has 1..208) v4 ==== 374+0+0 (secure 0 0 0) 0x7f3f8c0796d0 con 0x7f3f9415ac50 2023-12-19T18:29:03.634 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.635+0000 7f3f8affd640 1 -- 172.21.15.27:0/605547673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=209}) v3 -- 0x7f3f74072870 con 0x7f3f9415ac50 2023-12-19T18:29:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.652+0000 7f3f8affd640 1 -- 172.21.15.27:0/605547673 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v86) v1 ==== 161+0+0 (secure 0 0 0) 0x7f3f8c04d030 con 0x7f3f9415ac50 2023-12-19T18:29:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-19T18:29:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.657+0000 7f3f9359e640 1 -- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3f7406a430 msgr2=0x7f3f7406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.657+0000 7f3f9359e640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3f7406a430 0x7f3f7406c8f0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f3f84003eb0 tx=0x7f3f84009040 comp rx=0 tx=0).stop 2023-12-19T18:29:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.657+0000 7f3f9359e640 1 -- 172.21.15.27:0/605547673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f9415ac50 msgr2=0x7f3f94153920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:03.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.657+0000 7f3f9359e640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f9415ac50 0x7f3f94153920 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c02b830 tx=0x7f3f8c0046c0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.657+0000 7f3f9359e640 1 -- 172.21.15.27:0/605547673 shutdown_connections 2023-12-19T18:29:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.657+0000 7f3f9359e640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3f7406a430 0x7f3f7406c8f0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.657+0000 7f3f9359e640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f9415ac50 0x7f3f94153920 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.657+0000 7f3f9359e640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3f94070fb0 0x7f3f94145880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.657+0000 7f3f9359e640 1 --2- 172.21.15.27:0/605547673 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3f9406b850 0x7f3f940b3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.657+0000 7f3f9359e640 1 -- 172.21.15.27:0/605547673 >> 172.21.15.27:0/605547673 conn(0x7f3f9406a7d0 msgr2=0x7f3f940cc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.657+0000 7f3f9359e640 1 -- 172.21.15.27:0/605547673 shutdown_connections 2023-12-19T18:29:03.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.657+0000 7f3f9359e640 1 -- 172.21.15.27:0/605547673 wait complete. 2023-12-19T18:29:03.673 INFO:tasks.ceph.mds.c.smithi027.stderr: -8360> 2023-12-19T18:28:13.021+0000 7f96068f0640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-19T18:29:03.686 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-19T18:29:03.734 INFO:tasks.ceph.mds.c.smithi027.stdout:starting mds.c at 2023-12-19T18:29:03.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.897+0000 7f0d74028640 1 Processor -- start 2023-12-19T18:29:03.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.897+0000 7f0d74028640 1 -- start start 2023-12-19T18:29:03.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.898+0000 7f0d74028640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f0d6c0b8920 0x7f0d6c160c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:03.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.898+0000 7f0d74028640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0d6c0b5060 0x7f0d6c0d1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:03.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.898+0000 7f0d74028640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d6c0d0d20 con 0x7f0d6c0b4c80 2023-12-19T18:29:03.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.898+0000 7f0d74028640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6c0b6130 con 0x7f0d6c0b5060 2023-12-19T18:29:03.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.898+0000 7f0d74028640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6c161bf0 con 0x7f0d6c0b8920 2023-12-19T18:29:03.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.899+0000 7f0d725c5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0d6c0b5060 0x7f0d6c0d1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:03.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.899+0000 7f0d725c5640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f0d6c0b5060 0x7f0d6c0d1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39578/0 (socket says 172.21.15.27:39578) 2023-12-19T18:29:03.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.899+0000 7f0d725c5640 1 -- 172.21.15.27:0/1074029508 learned_addr learned my addr 172.21.15.27:0/1074029508 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:03.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.899+0000 7f0d715c3640 1 --2- 172.21.15.27:0/1074029508 >> v2:172.21.15.192:3300/0 conn(0x7f0d6c0b8920 0x7f0d6c160c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:03.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.899+0000 7f0d725c5640 1 -- 172.21.15.27:0/1074029508 >> v2:172.21.15.192:3300/0 conn(0x7f0d6c0b8920 msgr2=0x7f0d6c160c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:03.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.899+0000 7f0d725c5640 1 --2- 172.21.15.27:0/1074029508 >> v2:172.21.15.192:3300/0 conn(0x7f0d6c0b8920 0x7f0d6c160c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.900+0000 7f0d725c5640 1 -- 172.21.15.27:0/1074029508 >> v1:172.21.15.192:6789/0 conn(0x7f0d6c0b4c80 legacy=0x7f0d6c15cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:03.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.900+0000 7f0d725c5640 1 -- 172.21.15.27:0/1074029508 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c161350 con 0x7f0d6c0b5060 2023-12-19T18:29:03.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.901+0000 7f0d725c5640 1 --2- 172.21.15.27:0/1074029508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d6c0b5060 0x7f0d6c0d1a20 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0cf730 tx=0x7f0d6402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e7bb792feb4a098 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:03.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.902+0000 7f0d70dc2640 1 -- 172.21.15.27:0/1074029508 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64031870 con 0x7f0d6c0b5060 2023-12-19T18:29:03.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.902+0000 7f0d70dc2640 1 -- 172.21.15.27:0/1074029508 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d640319d0 con 0x7f0d6c0b5060 2023-12-19T18:29:03.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.902+0000 7f0d70dc2640 1 -- 172.21.15.27:0/1074029508 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d64031d60 con 0x7f0d6c0b5060 2023-12-19T18:29:03.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.902+0000 7f0d74028640 1 -- 172.21.15.27:0/1074029508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d6c0b5060 msgr2=0x7f0d6c0d1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:03.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.902+0000 7f0d74028640 1 --2- 172.21.15.27:0/1074029508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d6c0b5060 0x7f0d6c0d1a20 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0cf730 tx=0x7f0d6402b910 comp rx=0 tx=0).stop 2023-12-19T18:29:03.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.903+0000 7f0d74028640 1 -- 172.21.15.27:0/1074029508 shutdown_connections 2023-12-19T18:29:03.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.903+0000 7f0d74028640 1 --2- 172.21.15.27:0/1074029508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d6c0b5060 0x7f0d6c0d1a20 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.903+0000 7f0d74028640 1 --2- 172.21.15.27:0/1074029508 >> v2:172.21.15.192:3300/0 conn(0x7f0d6c0b8920 0x7f0d6c160c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.903+0000 7f0d74028640 1 -- 172.21.15.27:0/1074029508 >> 172.21.15.27:0/1074029508 conn(0x7f0d6c0d2f90 msgr2=0x7f0d6c0d3370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:03.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.903+0000 7f0d74028640 1 -- 172.21.15.27:0/1074029508 shutdown_connections 2023-12-19T18:29:03.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.903+0000 7f0d74028640 1 -- 172.21.15.27:0/1074029508 wait complete. 2023-12-19T18:29:03.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.905+0000 7f0d74028640 1 Processor -- start 2023-12-19T18:29:03.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.905+0000 7f0d74028640 1 -- start start 2023-12-19T18:29:03.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d74028640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d6c0b4c80 0x7f0d6c0c9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:03.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d74028640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d6c0b5060 0x7f0d6c1f2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:03.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d74028640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d6c0b8920 0x7f0d6c1537e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:03.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d74028640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c1f34a0 con 0x7f0d6c0b4c80 2023-12-19T18:29:03.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d74028640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c0cb1e0 con 0x7f0d6c0b8920 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d74028640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c0ca5d0 con 0x7f0d6c0b5060 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d71dc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d6c0b4c80 0x7f0d6c0c9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d71dc4640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d6c0b4c80 0x7f0d6c0c9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39586/0 (socket says 172.21.15.27:39586) 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d71dc4640 1 -- 172.21.15.27:0/3615734745 learned_addr learned my addr 172.21.15.27:0/3615734745 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d725c5640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d6c0b8920 0x7f0d6c1537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d71dc4640 1 -- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d6c0b5060 msgr2=0x7f0d6c1f2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d715c3640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d6c0b5060 0x7f0d6c1f2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d71dc4640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d6c0b5060 0x7f0d6c1f2f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.906+0000 7f0d71dc4640 1 -- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d6c0b8920 msgr2=0x7f0d6c1537e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.907+0000 7f0d71dc4640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d6c0b8920 0x7f0d6c1537e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.907+0000 7f0d71dc4640 1 -- 172.21.15.27:0/3615734745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d640094d0 con 0x7f0d6c0b4c80 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.907+0000 7f0d725c5640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d6c0b8920 0x7f0d6c1537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.907+0000 7f0d71dc4640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d6c0b4c80 0x7f0d6c0c9270 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000c870 tx=0x7f0d6000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:03.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.907+0000 7f0d6a7fc640 1 -- 172.21.15.27:0/3615734745 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60003b70 con 0x7f0d6c0b4c80 2023-12-19T18:29:03.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.907+0000 7f0d6a7fc640 1 -- 172.21.15.27:0/3615734745 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d60003cf0 con 0x7f0d6c0b4c80 2023-12-19T18:29:03.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.907+0000 7f0d74028640 1 -- 172.21.15.27:0/3615734745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d6c0ca8b0 con 0x7f0d6c0b4c80 2023-12-19T18:29:03.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.907+0000 7f0d6a7fc640 1 -- 172.21.15.27:0/3615734745 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6000aa90 con 0x7f0d6c0b4c80 2023-12-19T18:29:03.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.907+0000 7f0d74028640 1 -- 172.21.15.27:0/3615734745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6c0cae70 con 0x7f0d6c0b4c80 2023-12-19T18:29:03.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.908+0000 7f0d74028640 1 -- 172.21.15.27:0/3615734745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6c066cc0 con 0x7f0d6c0b4c80 2023-12-19T18:29:03.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.910+0000 7f0d6a7fc640 1 -- 172.21.15.27:0/3615734745 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f0d6000abf0 con 0x7f0d6c0b4c80 2023-12-19T18:29:03.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.911+0000 7f0d6a7fc640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d5006a430 0x7f0d5006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:03.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.911+0000 7f0d6a7fc640 1 -- 172.21.15.27:0/3615734745 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f0d60088860 con 0x7f0d6c0b4c80 2023-12-19T18:29:03.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.911+0000 7f0d6a7fc640 1 -- 172.21.15.27:0/3615734745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=209}) v3 -- 0x7f0d500720d0 con 0x7f0d6c0b4c80 2023-12-19T18:29:03.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.911+0000 7f0d715c3640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d5006a430 0x7f0d5006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:03.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.913+0000 7f0d715c3640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d5006a430 0x7f0d5006c8f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0cd590 tx=0x7f0d58008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:03.915 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:03.915+0000 7f0d6a7fc640 1 -- 172.21.15.27:0/3615734745 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f0d60003e60 con 0x7f0d6c0b4c80 2023-12-19T18:29:04.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.153+0000 7f0d74028640 1 -- 172.21.15.27:0/3615734745 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0d6c161600 con 0x7f0d6c0b4c80 2023-12-19T18:29:04.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.657+0000 7f0d6a7fc640 1 -- 172.21.15.27:0/3615734745 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v87) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0d60053de0 con 0x7f0d6c0b4c80 2023-12-19T18:29:04.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.660+0000 7f0d74028640 1 -- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d5006a430 msgr2=0x7f0d5006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:04.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.660+0000 7f0d74028640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d5006a430 0x7f0d5006c8f0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c0cd590 tx=0x7f0d58008040 comp rx=0 tx=0).stop 2023-12-19T18:29:04.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.660+0000 7f0d74028640 1 -- 172.21.15.27:0/3615734745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d6c0b4c80 msgr2=0x7f0d6c0c9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:04.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.660+0000 7f0d74028640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d6c0b4c80 0x7f0d6c0c9270 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f0d6000c870 tx=0x7f0d6000cd30 comp rx=0 tx=0).stop 2023-12-19T18:29:04.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.661+0000 7f0d74028640 1 -- 172.21.15.27:0/3615734745 shutdown_connections 2023-12-19T18:29:04.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.661+0000 7f0d74028640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f0d5006a430 0x7f0d5006c8f0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:04.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.661+0000 7f0d74028640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f0d6c0b8920 0x7f0d6c1537e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:04.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.661+0000 7f0d74028640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f0d6c0b5060 0x7f0d6c1f2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:04.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.661+0000 7f0d74028640 1 --2- 172.21.15.27:0/3615734745 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d6c0b4c80 0x7f0d6c0c9270 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:04.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.661+0000 7f0d74028640 1 -- 172.21.15.27:0/3615734745 >> 172.21.15.27:0/3615734745 conn(0x7f0d6c0d2f90 msgr2=0x7f0d6c0b6850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:04.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.661+0000 7f0d74028640 1 -- 172.21.15.27:0/3615734745 shutdown_connections 2023-12-19T18:29:04.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.661+0000 7f0d74028640 1 -- 172.21.15.27:0/3615734745 wait complete. 2023-12-19T18:29:04.681 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-19T18:29:04.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.917+0000 7f319a7fd640 1 Processor -- start 2023-12-19T18:29:04.917 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.917+0000 7f319a7fd640 1 -- start start 2023-12-19T18:29:04.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.918+0000 7f319a7fd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319406ad80 0x7f319406b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:04.918 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.918+0000 7f319a7fd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f31940cdf90 0x7f31940ce3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.918+0000 7f319a7fd640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f319415da80 con 0x7f3194159920 2023-12-19T18:29:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.918+0000 7f319a7fd640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f319415b3b0 con 0x7f31940cdf90 2023-12-19T18:29:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.918+0000 7f319a7fd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319406b700 con 0x7f319406ad80 2023-12-19T18:29:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.919+0000 7f31997fb640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f3194159920 0x7f31940ca500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53346/0 (socket says 172.21.15.27:53346) 2023-12-19T18:29:04.919 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.919+0000 7f31997fb640 1 -- 172.21.15.27:0/2268280796 learned_addr learned my addr 172.21.15.27:0/2268280796 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:04.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.919+0000 7f3198ffa640 1 --2- 172.21.15.27:0/2268280796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319406ad80 0x7f319406b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:04.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.919+0000 7f3199ffc640 1 --2- 172.21.15.27:0/2268280796 >> v2:172.21.15.192:3300/0 conn(0x7f31940cdf90 0x7f31940ce3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:04.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.919+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267165728 0 0) 0x7f319415da80 con 0x7f3194159920 2023-12-19T18:29:04.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.919+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f317c005730 con 0x7f3194159920 2023-12-19T18:29:04.921 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3983218583 0 0) 0x7f317c005730 con 0x7f3194159920 2023-12-19T18:29:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319406ad80 msgr2=0x7f319406b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 --2- 172.21.15.27:0/2268280796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319406ad80 0x7f319406b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 >> v2:172.21.15.192:3300/0 conn(0x7f31940cdf90 msgr2=0x7f31940ce3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 --2- 172.21.15.27:0/2268280796 >> v2:172.21.15.192:3300/0 conn(0x7f31940cdf90 0x7f31940ce3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31940ce920 con 0x7f3194159920 2023-12-19T18:29:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f31840019f0 con 0x7f3194159920 2023-12-19T18:29:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 >> v1:172.21.15.27:6789/0 conn(0x7f3194159920 legacy=0x7f31940ca500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 --2- 172.21.15.27:0/2268280796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f317c005b80 0x7f317c005fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 --2- 172.21.15.27:0/2268280796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319406ad80 0x7f317c0069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 --2- 172.21.15.27:0/2268280796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f317c006f10 0x7f317c007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319415b3b0 con 0x7f317c006f10 2023-12-19T18:29:04.922 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319406b700 con 0x7f317c005b80 2023-12-19T18:29:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.920+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f317c00db90 con 0x7f319406ad80 2023-12-19T18:29:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.921+0000 7f3199ffc640 1 --2- 172.21.15.27:0/2268280796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f317c006f10 0x7f317c007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.921+0000 7f31997fb640 1 --2- 172.21.15.27:0/2268280796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f317c005b80 0x7f317c005fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.921+0000 7f3198ffa640 1 --2- 172.21.15.27:0/2268280796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319406ad80 0x7f317c0069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.921+0000 7f3199ffc640 1 -- 172.21.15.27:0/2268280796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319406ad80 msgr2=0x7f317c0069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.921+0000 7f3199ffc640 1 --2- 172.21.15.27:0/2268280796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319406ad80 0x7f317c0069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.921+0000 7f3199ffc640 1 -- 172.21.15.27:0/2268280796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f317c005b80 msgr2=0x7f317c005fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.921+0000 7f3199ffc640 1 --2- 172.21.15.27:0/2268280796 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f317c005b80 0x7f317c005fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.921+0000 7f3199ffc640 1 -- 172.21.15.27:0/2268280796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f317c005730 con 0x7f317c006f10 2023-12-19T18:29:04.923 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.921+0000 7f3198ffa640 1 --2- 172.21.15.27:0/2268280796 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f319406ad80 0x7f317c0069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:04.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.922+0000 7f3199ffc640 1 --2- 172.21.15.27:0/2268280796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f317c006f10 0x7f317c007390 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f318c009af0 tx=0x7f317c007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6aa37a51fadb0755 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:04.924 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.923+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c02c710 con 0x7f317c006f10 2023-12-19T18:29:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.923+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c02c870 con 0x7f317c006f10 2023-12-19T18:29:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.923+0000 7f3193fff640 1 -- 172.21.15.27:0/2268280796 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c02cc00 con 0x7f317c006f10 2023-12-19T18:29:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.923+0000 7f319a7fd640 1 -- 172.21.15.27:0/2268280796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f317c006f10 msgr2=0x7f317c007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.923+0000 7f319a7fd640 1 --2- 172.21.15.27:0/2268280796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f317c006f10 0x7f317c007390 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f318c009af0 tx=0x7f317c007a00 comp rx=0 tx=0).stop 2023-12-19T18:29:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.923+0000 7f3199ffc640 1 -- 172.21.15.27:0/2268280796 reap_dead start 2023-12-19T18:29:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.923+0000 7f319a7fd640 1 -- 172.21.15.27:0/2268280796 shutdown_connections 2023-12-19T18:29:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.923+0000 7f319a7fd640 1 -- 172.21.15.27:0/2268280796 >> 172.21.15.27:0/2268280796 conn(0x7f31940cfa60 msgr2=0x7f31940cfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.923+0000 7f319a7fd640 1 -- 172.21.15.27:0/2268280796 shutdown_connections 2023-12-19T18:29:04.925 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.924+0000 7f319a7fd640 1 -- 172.21.15.27:0/2268280796 wait complete. 2023-12-19T18:29:04.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.925+0000 7f319a7fd640 1 Processor -- start 2023-12-19T18:29:04.926 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.926+0000 7f319a7fd640 1 -- start start 2023-12-19T18:29:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.927+0000 7f319a7fd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f317c006f10 0x7f31940c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.927+0000 7f319a7fd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31940b4f50 0x7f3194151f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.927+0000 7f319a7fd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31940b2600 0x7f31940c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.927+0000 7f319a7fd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f319415d420 con 0x7f31940b2600 2023-12-19T18:29:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.927+0000 7f319a7fd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31940c7230 con 0x7f31940b4f50 2023-12-19T18:29:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.927+0000 7f319a7fd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f319415b6d0 con 0x7f317c006f10 2023-12-19T18:29:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.927+0000 7f3199ffc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31940b2600 0x7f31940c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.927+0000 7f3199ffc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31940b2600 0x7f31940c77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39610/0 (socket says 172.21.15.27:39610) 2023-12-19T18:29:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.927+0000 7f3199ffc640 1 -- 172.21.15.27:0/2484111380 learned_addr learned my addr 172.21.15.27:0/2484111380 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.927+0000 7f3198ffa640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31940b4f50 0x7f3194151f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.927+0000 7f31997fb640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f317c006f10 0x7f31940c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.928+0000 7f3199ffc640 1 -- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f317c006f10 msgr2=0x7f31940c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:04.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.928+0000 7f3199ffc640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f317c006f10 0x7f31940c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:04.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.928+0000 7f3199ffc640 1 -- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31940b4f50 msgr2=0x7f3194151f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:04.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.928+0000 7f3199ffc640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31940b4f50 0x7f3194151f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:04.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.928+0000 7f3199ffc640 1 -- 172.21.15.27:0/2484111380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f319415b9b0 con 0x7f31940b2600 2023-12-19T18:29:04.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.928+0000 7f3198ffa640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31940b4f50 0x7f3194151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:04.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.928+0000 7f3199ffc640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31940b2600 0x7f31940c77f0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f318c009af0 tx=0x7f318c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:04.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.928+0000 7f3191ffb640 1 -- 172.21.15.27:0/2484111380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c004450 con 0x7f31940b2600 2023-12-19T18:29:04.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.928+0000 7f3191ffb640 1 -- 172.21.15.27:0/2484111380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318c003610 con 0x7f31940b2600 2023-12-19T18:29:04.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.928+0000 7f319a7fd640 1 -- 172.21.15.27:0/2484111380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f319415bc90 con 0x7f31940b2600 2023-12-19T18:29:04.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.929+0000 7f3191ffb640 1 -- 172.21.15.27:0/2484111380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318c0039c0 con 0x7f31940b2600 2023-12-19T18:29:04.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.929+0000 7f319a7fd640 1 -- 172.21.15.27:0/2484111380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31940b2a80 con 0x7f31940b2600 2023-12-19T18:29:04.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.930+0000 7f319a7fd640 1 -- 172.21.15.27:0/2484111380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f319415c1b0 con 0x7f31940b2600 2023-12-19T18:29:04.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.931+0000 7f3191ffb640 1 -- 172.21.15.27:0/2484111380 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f318c003b20 con 0x7f31940b2600 2023-12-19T18:29:04.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.932+0000 7f3191ffb640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f318006a430 0x7f318006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:04.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.932+0000 7f3191ffb640 1 -- 172.21.15.27:0/2484111380 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5824+0+0 (secure 0 0 0) 0x7f318c0a9940 con 0x7f31940b2600 2023-12-19T18:29:04.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.932+0000 7f3191ffb640 1 -- 172.21.15.27:0/2484111380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=209}) v3 -- 0x7f31800720d0 con 0x7f31940b2600 2023-12-19T18:29:04.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.932+0000 7f31997fb640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f318006a430 0x7f318006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:04.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.934+0000 7f31997fb640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f318006a430 0x7f318006c8f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3184005e10 tx=0x7f3184009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:04.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:04.936+0000 7f3191ffb640 1 -- 172.21.15.27:0/2484111380 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f318c039020 con 0x7f31940b2600 2023-12-19T18:29:05.130 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.130+0000 7f319a7fd640 1 -- 172.21.15.27:0/2484111380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f319404e840 con 0x7f31940b2600 2023-12-19T18:29:05.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.949+0000 7f3191ffb640 1 -- 172.21.15.27:0/2484111380 <== mon.0 v2:172.21.15.27:3300/0 7 ==== osd_map(209..209 src has 1..209) v4 ==== 296+0+0 (secure 0 0 0) 0x7f318c079200 con 0x7f31940b2600 2023-12-19T18:29:05.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.949+0000 7f3191ffb640 1 -- 172.21.15.27:0/2484111380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=210}) v3 -- 0x7f3180070500 con 0x7f31940b2600 2023-12-19T18:29:05.960 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.957+0000 7f3191ffb640 1 -- 172.21.15.27:0/2484111380 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v209) v1 ==== 181+0+0 (secure 0 0 0) 0x7f319415c1b0 con 0x7f31940b2600 2023-12-19T18:29:05.960 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_metadata' removed 2023-12-19T18:29:05.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.964+0000 7f319a7fd640 1 -- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f318006a430 msgr2=0x7f318006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:05.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.964+0000 7f319a7fd640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f318006a430 0x7f318006c8f0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f3184005e10 tx=0x7f3184009040 comp rx=0 tx=0).stop 2023-12-19T18:29:05.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.964+0000 7f319a7fd640 1 -- 172.21.15.27:0/2484111380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31940b2600 msgr2=0x7f31940c77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:05.964 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.964+0000 7f319a7fd640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31940b2600 0x7f31940c77f0 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f318c009af0 tx=0x7f318c004730 comp rx=0 tx=0).stop 2023-12-19T18:29:05.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.966+0000 7f319a7fd640 1 -- 172.21.15.27:0/2484111380 shutdown_connections 2023-12-19T18:29:05.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.966+0000 7f319a7fd640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f318006a430 0x7f318006c8f0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:05.966 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.966+0000 7f319a7fd640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31940b2600 0x7f31940c77f0 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:05.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.966+0000 7f319a7fd640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f31940b4f50 0x7f3194151f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:05.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.966+0000 7f319a7fd640 1 --2- 172.21.15.27:0/2484111380 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f317c006f10 0x7f31940c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:05.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.967+0000 7f319a7fd640 1 -- 172.21.15.27:0/2484111380 >> 172.21.15.27:0/2484111380 conn(0x7f31940cfa60 msgr2=0x7f3194069e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:05.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.967+0000 7f319a7fd640 1 -- 172.21.15.27:0/2484111380 shutdown_connections 2023-12-19T18:29:05.967 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:05.967+0000 7f319a7fd640 1 -- 172.21.15.27:0/2484111380 wait complete. 2023-12-19T18:29:05.989 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-19T18:29:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.210+0000 7fc2dd3f0640 1 Processor -- start 2023-12-19T18:29:06.210 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.211+0000 7fc2dd3f0640 1 -- start start 2023-12-19T18:29:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.212+0000 7fc2dd3f0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2d815b0d0 0x7fc2d80cabf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.212+0000 7fc2dd3f0640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc2d815b4b0 0x7fc2d80cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.213+0000 7fc2dd3f0640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2d8070fb0 con 0x7fc2d815fa70 2023-12-19T18:29:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.213+0000 7fc2dd3f0640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2d806fab0 con 0x7fc2d815b4b0 2023-12-19T18:29:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.213+0000 7fc2dd3f0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d806f4f0 con 0x7fc2d815b0d0 2023-12-19T18:29:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.213+0000 7fc2d67fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc2d815b4b0 0x7fc2d80cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:06.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.213+0000 7fc2d67fc640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc2d815b4b0 0x7fc2d80cb450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39618/0 (socket says 172.21.15.27:39618) 2023-12-19T18:29:06.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.213+0000 7fc2d67fc640 1 -- 172.21.15.27:0/1843212263 learned_addr learned my addr 172.21.15.27:0/1843212263 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:06.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.213+0000 7fc2d6ffd640 1 --2- 172.21.15.27:0/1843212263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2d815b0d0 0x7fc2d80cabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:06.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.213+0000 7fc2d67fc640 1 -- 172.21.15.27:0/1843212263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2d815b0d0 msgr2=0x7fc2d80cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:06.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.213+0000 7fc2d67fc640 1 --2- 172.21.15.27:0/1843212263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2d815b0d0 0x7fc2d80cabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:06.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.213+0000 7fc2d67fc640 1 -- 172.21.15.27:0/1843212263 >> v1:172.21.15.192:6789/0 conn(0x7fc2d815fa70 legacy=0x7fc2d806b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:06.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.213+0000 7fc2d67fc640 1 -- 172.21.15.27:0/1843212263 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d806f7d0 con 0x7fc2d815b4b0 2023-12-19T18:29:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.214+0000 7fc2d67fc640 1 --2- 172.21.15.27:0/1843212263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2d815b4b0 0x7fc2d80cb450 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009e50 tx=0x7fc2c002bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb4710fd803f6f11 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.215+0000 7fc2d5ffb640 1 -- 172.21.15.27:0/1843212263 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c002cb00 con 0x7fc2d815b4b0 2023-12-19T18:29:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.215+0000 7fc2d5ffb640 1 -- 172.21.15.27:0/1843212263 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c002cc60 con 0x7fc2d815b4b0 2023-12-19T18:29:06.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.215+0000 7fc2d5ffb640 1 -- 172.21.15.27:0/1843212263 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c00343f0 con 0x7fc2d815b4b0 2023-12-19T18:29:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.215+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/1843212263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2d815b4b0 msgr2=0x7fc2d80cb450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.215+0000 7fc2dd3f0640 1 --2- 172.21.15.27:0/1843212263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2d815b4b0 0x7fc2d80cb450 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009e50 tx=0x7fc2c002bf60 comp rx=0 tx=0).stop 2023-12-19T18:29:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.216+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/1843212263 shutdown_connections 2023-12-19T18:29:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.216+0000 7fc2dd3f0640 1 --2- 172.21.15.27:0/1843212263 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2d815b4b0 0x7fc2d80cb450 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.216+0000 7fc2dd3f0640 1 --2- 172.21.15.27:0/1843212263 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2d815b0d0 0x7fc2d80cabf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.216+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/1843212263 >> 172.21.15.27:0/1843212263 conn(0x7fc2d80b0620 msgr2=0x7fc2d80b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.216+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/1843212263 shutdown_connections 2023-12-19T18:29:06.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.216+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/1843212263 wait complete. 2023-12-19T18:29:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.217+0000 7fc2dd3f0640 1 Processor -- start 2023-12-19T18:29:06.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.218+0000 7fc2dd3f0640 1 -- start start 2023-12-19T18:29:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2dd3f0640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2d815b0d0 0x7fc2d80c2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2dd3f0640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2d815b4b0 0x7fc2d80af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2dd3f0640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2d815fa70 0x7fc2d80ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2dd3f0640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d80c7100 con 0x7fc2d815b4b0 2023-12-19T18:29:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2dd3f0640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d80c73e0 con 0x7fc2d815b0d0 2023-12-19T18:29:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2dd3f0640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2d80c76c0 con 0x7fc2d815fa70 2023-12-19T18:29:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2d67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2d815b4b0 0x7fc2d80af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2d67fc640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2d815b4b0 0x7fc2d80af4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39624/0 (socket says 172.21.15.27:39624) 2023-12-19T18:29:06.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2d67fc640 1 -- 172.21.15.27:0/696470599 learned_addr learned my addr 172.21.15.27:0/696470599 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:06.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2d67fc640 1 -- 172.21.15.27:0/696470599 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2d815fa70 msgr2=0x7fc2d80ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2d67fc640 1 --2- 172.21.15.27:0/696470599 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2d815fa70 0x7fc2d80ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2d67fc640 1 -- 172.21.15.27:0/696470599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2d815b0d0 msgr2=0x7fc2d80c2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2d6ffd640 1 --2- 172.21.15.27:0/696470599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2d815b0d0 0x7fc2d80c2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2d67fc640 1 --2- 172.21.15.27:0/696470599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2d815b0d0 0x7fc2d80c2770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.219+0000 7fc2d67fc640 1 -- 172.21.15.27:0/696470599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c0009b30 con 0x7fc2d815b4b0 2023-12-19T18:29:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.220+0000 7fc2d67fc640 1 --2- 172.21.15.27:0/696470599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2d815b4b0 0x7fc2d80af4b0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009e50 tx=0x7fc2c0004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.220+0000 7fc2cf7fe640 1 -- 172.21.15.27:0/696470599 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c003e450 con 0x7fc2d815b4b0 2023-12-19T18:29:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.220+0000 7fc2cf7fe640 1 -- 172.21.15.27:0/696470599 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2c003e5f0 con 0x7fc2d815b4b0 2023-12-19T18:29:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.220+0000 7fc2cf7fe640 1 -- 172.21.15.27:0/696470599 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2c003e9a0 con 0x7fc2d815b4b0 2023-12-19T18:29:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.220+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/696470599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2d80c79a0 con 0x7fc2d815b4b0 2023-12-19T18:29:06.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.220+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/696470599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2d80c2cb0 con 0x7fc2d815b4b0 2023-12-19T18:29:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.221+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/696470599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2d8068e90 con 0x7fc2d815b4b0 2023-12-19T18:29:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.222+0000 7fc2cf7fe640 1 -- 172.21.15.27:0/696470599 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc2c003eb00 con 0x7fc2d815b4b0 2023-12-19T18:29:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.223+0000 7fc2cf7fe640 1 --2- 172.21.15.27:0/696470599 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc2b806a430 0x7fc2b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.223+0000 7fc2cf7fe640 1 -- 172.21.15.27:0/696470599 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5386+0+0 (secure 0 0 0) 0x7fc2c00ab770 con 0x7fc2d815b4b0 2023-12-19T18:29:06.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.224+0000 7fc2cf7fe640 1 -- 172.21.15.27:0/696470599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=210}) v3 -- 0x7fc2b8071260 con 0x7fc2d815b4b0 2023-12-19T18:29:06.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.224+0000 7fc2d6ffd640 1 --2- 172.21.15.27:0/696470599 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc2b806a430 0x7fc2b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:06.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.225+0000 7fc2d6ffd640 1 --2- 172.21.15.27:0/696470599 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc2b806a430 0x7fc2b806c8f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8006fd0 tx=0x7fc2c8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:06.227 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.228+0000 7fc2cf7fe640 1 -- 172.21.15.27:0/696470599 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc2c0076f20 con 0x7fc2d815b4b0 2023-12-19T18:29:06.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.424+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/696470599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc2d80c7800 con 0x7fc2d815b4b0 2023-12-19T18:29:06.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.958+0000 7fc2cf7fe640 1 -- 172.21.15.27:0/696470599 <== mon.0 v2:172.21.15.27:3300/0 7 ==== osd_map(210..210 src has 1..210) v4 ==== 313+0+0 (secure 0 0 0) 0x7fc2c007b2e0 con 0x7fc2d815b4b0 2023-12-19T18:29:06.958 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.958+0000 7fc2cf7fe640 1 -- 172.21.15.27:0/696470599 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=211}) v3 -- 0x7fc2b806fff0 con 0x7fc2d815b4b0 2023-12-19T18:29:06.979 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.978+0000 7fc2cf7fe640 1 -- 172.21.15.27:0/696470599 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v210) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc2c003edb0 con 0x7fc2d815b4b0 2023-12-19T18:29:06.979 INFO:teuthology.orchestra.run.smithi027.stderr:pool 'cephfs_data' removed 2023-12-19T18:29:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.986+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/696470599 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc2b806a430 msgr2=0x7fc2b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.986+0000 7fc2dd3f0640 1 --2- 172.21.15.27:0/696470599 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc2b806a430 0x7fc2b806c8f0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fc2c8006fd0 tx=0x7fc2c8008040 comp rx=0 tx=0).stop 2023-12-19T18:29:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.986+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/696470599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2d815b4b0 msgr2=0x7fc2d80af4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:06.986 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.986+0000 7fc2dd3f0640 1 --2- 172.21.15.27:0/696470599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2d815b4b0 0x7fc2d80af4b0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009e50 tx=0x7fc2c0004a40 comp rx=0 tx=0).stop 2023-12-19T18:29:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.986+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/696470599 shutdown_connections 2023-12-19T18:29:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.986+0000 7fc2dd3f0640 1 --2- 172.21.15.27:0/696470599 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc2b806a430 0x7fc2b806c8f0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.986+0000 7fc2dd3f0640 1 --2- 172.21.15.27:0/696470599 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc2d815fa70 0x7fc2d80ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.986+0000 7fc2dd3f0640 1 --2- 172.21.15.27:0/696470599 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2d815b4b0 0x7fc2d80af4b0 secure :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7fc2c0009e50 tx=0x7fc2c0004a40 comp rx=0 tx=0).stop 2023-12-19T18:29:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.986+0000 7fc2dd3f0640 1 --2- 172.21.15.27:0/696470599 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc2d815b0d0 0x7fc2d80c2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.987+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/696470599 >> 172.21.15.27:0/696470599 conn(0x7fc2d80b0620 msgr2=0x7fc2d80c3470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.987+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/696470599 shutdown_connections 2023-12-19T18:29:06.987 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:06.987+0000 7fc2dd3f0640 1 -- 172.21.15.27:0/696470599 wait complete. 2023-12-19T18:29:07.006 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-19T18:29:07.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.210+0000 7fb2dc955640 1 Processor -- start 2023-12-19T18:29:07.211 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.211+0000 7fb2dc955640 1 -- start start 2023-12-19T18:29:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.212+0000 7fb2dc955640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb2d8159de0 0x7fb2d80b1050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:07.212 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.212+0000 7fb2dc955640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d815d810 con 0x7fb2d815df80 2023-12-19T18:29:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.213+0000 7fb2dc955640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d80ca0c0 con 0x7fb2d8159a00 2023-12-19T18:29:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.213+0000 7fb2dc955640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d80ca3a0 con 0x7fb2d8159de0 2023-12-19T18:29:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.213+0000 7fb2cdd9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb2d8159de0 0x7fb2d80b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.213+0000 7fb2d6d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fb2d815df80 0x7fb2d806c0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53352/0 (socket says 172.21.15.27:53352) 2023-12-19T18:29:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.213+0000 7fb2cdd9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb2d8159de0 0x7fb2d80b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39638/0 (socket says 172.21.15.27:39638) 2023-12-19T18:29:07.213 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.213+0000 7fb2d6d9d640 1 -- 172.21.15.27:0/2219215333 learned_addr learned my addr 172.21.15.27:0/2219215333 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.213+0000 7fb2cdd9b640 1 -- 172.21.15.27:0/2219215333 >> v1:172.21.15.192:6789/0 conn(0x7fb2d8159a00 legacy=0x7fb2d80c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.213+0000 7fb2cdd9b640 1 -- 172.21.15.27:0/2219215333 >> v1:172.21.15.27:6789/0 conn(0x7fb2d815df80 legacy=0x7fb2d806c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.213+0000 7fb2d5d9b640 1 -- 172.21.15.27:0/2219215333 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 60172018 0 0) 0x7fb2d815d810 con 0x7fb2d815df80 2023-12-19T18:29:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.213+0000 7fb2cdd9b640 1 -- 172.21.15.27:0/2219215333 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d80ca680 con 0x7fb2d8159de0 2023-12-19T18:29:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.215+0000 7fb2cdd9b640 1 --2- 172.21.15.27:0/2219215333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d8159de0 0x7fb2d80b1050 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00097a0 tx=0x7fb2c002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a9554b739c1979 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:07.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.215+0000 7fb2d5d9b640 1 -- 172.21.15.27:0/2219215333 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0031830 con 0x7fb2d8159de0 2023-12-19T18:29:07.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.215+0000 7fb2d5d9b640 1 -- 172.21.15.27:0/2219215333 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c0031990 con 0x7fb2d8159de0 2023-12-19T18:29:07.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.215+0000 7fb2d5d9b640 1 -- 172.21.15.27:0/2219215333 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c0031d20 con 0x7fb2d8159de0 2023-12-19T18:29:07.215 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.215+0000 7fb2dc955640 1 -- 172.21.15.27:0/2219215333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d8159de0 msgr2=0x7fb2d80b1050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.215+0000 7fb2dc955640 1 --2- 172.21.15.27:0/2219215333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d8159de0 0x7fb2d80b1050 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7fb2c00097a0 tx=0x7fb2c002b8d0 comp rx=0 tx=0).stop 2023-12-19T18:29:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.216+0000 7fb2dc955640 1 -- 172.21.15.27:0/2219215333 shutdown_connections 2023-12-19T18:29:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.216+0000 7fb2dc955640 1 --2- 172.21.15.27:0/2219215333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d8159de0 0x7fb2d80b1050 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.216+0000 7fb2dc955640 1 -- 172.21.15.27:0/2219215333 >> 172.21.15.27:0/2219215333 conn(0x7fb2d80cb5b0 msgr2=0x7fb2d80cb990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.216+0000 7fb2dc955640 1 -- 172.21.15.27:0/2219215333 shutdown_connections 2023-12-19T18:29:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.216+0000 7fb2dc955640 1 -- 172.21.15.27:0/2219215333 wait complete. 2023-12-19T18:29:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.217+0000 7fb2dc955640 1 Processor -- start 2023-12-19T18:29:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.218+0000 7fb2dc955640 1 -- start start 2023-12-19T18:29:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.219+0000 7fb2dc955640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2d8159a00 0x7fb2d80afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.219+0000 7fb2dc955640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2d8159de0 0x7fb2d80acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.219+0000 7fb2dc955640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d815df80 0x7fb2d8152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.219+0000 7fb2dc955640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d8153790 con 0x7fb2d815df80 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.219+0000 7fb2dc955640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d815bf50 con 0x7fb2d8159a00 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.219+0000 7fb2dc955640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d815c230 con 0x7fb2d8159de0 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.219+0000 7fb2d6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d815df80 0x7fb2d8152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.219+0000 7fb2d6d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d815df80 0x7fb2d8152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39650/0 (socket says 172.21.15.27:39650) 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.219+0000 7fb2d6d9d640 1 -- 172.21.15.27:0/2577865819 learned_addr learned my addr 172.21.15.27:0/2577865819 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.219+0000 7fb2d659c640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2d8159a00 0x7fb2d80afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.219+0000 7fb2cdd9b640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2d8159de0 0x7fb2d80acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.220+0000 7fb2d6d9d640 1 -- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2d8159de0 msgr2=0x7fb2d80acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.220+0000 7fb2d6d9d640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2d8159de0 0x7fb2d80acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.220+0000 7fb2d6d9d640 1 -- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2d8159a00 msgr2=0x7fb2d80afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.220+0000 7fb2d6d9d640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2d8159a00 0x7fb2d80afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.220+0000 7fb2d6d9d640 1 -- 172.21.15.27:0/2577865819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c0009400 con 0x7fb2d815df80 2023-12-19T18:29:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.220+0000 7fb2d6d9d640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d815df80 0x7fb2d8152370 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c780 tx=0x7fb2d000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.220+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/2577865819 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000ce90 con 0x7fb2d815df80 2023-12-19T18:29:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.220+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/2577865819 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2d000a640 con 0x7fb2d815df80 2023-12-19T18:29:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.220+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/2577865819 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2d000a9d0 con 0x7fb2d815df80 2023-12-19T18:29:07.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.220+0000 7fb2dc955640 1 -- 172.21.15.27:0/2577865819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d815c510 con 0x7fb2d815df80 2023-12-19T18:29:07.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.220+0000 7fb2dc955640 1 -- 172.21.15.27:0/2577865819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d80ad080 con 0x7fb2d815df80 2023-12-19T18:29:07.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.222+0000 7fb2dc955640 1 -- 172.21.15.27:0/2577865819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d80ad2d0 con 0x7fb2d815df80 2023-12-19T18:29:07.222 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.223+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/2577865819 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb2d000ab30 con 0x7fb2d815df80 2023-12-19T18:29:07.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.224+0000 7fb2cf7fe640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb2b806a430 0x7fb2b806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:07.223 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.224+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/2577865819 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb2d0087b20 con 0x7fb2d815df80 2023-12-19T18:29:07.224 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.224+0000 7fb2d659c640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb2b806a430 0x7fb2b806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:07.225 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.225+0000 7fb2d659c640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb2b806a430 0x7fb2b806c8f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8004ea0 tx=0x7fb2c8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:07.228 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.228+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/2577865819 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb2d0053480 con 0x7fb2d815df80 2023-12-19T18:29:07.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.423+0000 7fb2dc955640 1 -- 172.21.15.27:0/2577865819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb2d8068e90 con 0x7fb2d815df80 2023-12-19T18:29:07.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.431+0000 7fb2cf7fe640 1 -- 172.21.15.27:0/2577865819 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb2d00180a0 con 0x7fb2d815df80 2023-12-19T18:29:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.434+0000 7fb2dc955640 1 -- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb2b806a430 msgr2=0x7fb2b806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.434+0000 7fb2dc955640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb2b806a430 0x7fb2b806c8f0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fb2c8004ea0 tx=0x7fb2c8009040 comp rx=0 tx=0).stop 2023-12-19T18:29:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.434+0000 7fb2dc955640 1 -- 172.21.15.27:0/2577865819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d815df80 msgr2=0x7fb2d8152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.434+0000 7fb2dc955640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d815df80 0x7fb2d8152370 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7fb2d000c780 tx=0x7fb2d000cc40 comp rx=0 tx=0).stop 2023-12-19T18:29:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.434+0000 7fb2dc955640 1 -- 172.21.15.27:0/2577865819 shutdown_connections 2023-12-19T18:29:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.434+0000 7fb2dc955640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fb2b806a430 0x7fb2b806c8f0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.434+0000 7fb2dc955640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2d815df80 0x7fb2d8152370 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.434+0000 7fb2dc955640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb2d8159de0 0x7fb2d80acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.434+0000 7fb2dc955640 1 --2- 172.21.15.27:0/2577865819 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb2d8159a00 0x7fb2d80afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.434+0000 7fb2dc955640 1 -- 172.21.15.27:0/2577865819 >> 172.21.15.27:0/2577865819 conn(0x7fb2d80cb5b0 msgr2=0x7fb2d80c8520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.434+0000 7fb2dc955640 1 -- 172.21.15.27:0/2577865819 shutdown_connections 2023-12-19T18:29:07.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.434+0000 7fb2dc955640 1 -- 172.21.15.27:0/2577865819 wait complete. 2023-12-19T18:29:07.454 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file' 2023-12-19T18:29:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.690+0000 7efd672ec640 1 Processor -- start 2023-12-19T18:29:07.690 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.690+0000 7efd672ec640 1 -- start start 2023-12-19T18:29:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.691+0000 7efd672ec640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7efd601604d0 0x7efd600b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.691+0000 7efd672ec640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efd600d2350 0x7efd600b4d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.691+0000 7efd672ec640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd600b5e90 con 0x7efd600d0300 2023-12-19T18:29:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.691+0000 7efd672ec640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd600d2b30 con 0x7efd600d2350 2023-12-19T18:29:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.691+0000 7efd672ec640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd600d2850 con 0x7efd601604d0 2023-12-19T18:29:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.692+0000 7efd65ae9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efd600d2350 0x7efd600b4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.692+0000 7efd65ae9640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7efd600d2350 0x7efd600b4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39656/0 (socket says 172.21.15.27:39656) 2023-12-19T18:29:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.692+0000 7efd65ae9640 1 -- 172.21.15.27:0/2355213907 learned_addr learned my addr 172.21.15.27:0/2355213907 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.692+0000 7efd662ea640 1 --2- 172.21.15.27:0/2355213907 >> v2:172.21.15.192:3300/0 conn(0x7efd601604d0 0x7efd600b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.692+0000 7efd65ae9640 1 -- 172.21.15.27:0/2355213907 >> v2:172.21.15.192:3300/0 conn(0x7efd601604d0 msgr2=0x7efd600b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.692+0000 7efd65ae9640 1 --2- 172.21.15.27:0/2355213907 >> v2:172.21.15.192:3300/0 conn(0x7efd601604d0 0x7efd600b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.692+0000 7efd65ae9640 1 -- 172.21.15.27:0/2355213907 >> v1:172.21.15.192:6789/0 conn(0x7efd600d0300 legacy=0x7efd600d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.692+0000 7efd65ae9640 1 -- 172.21.15.27:0/2355213907 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd600cded0 con 0x7efd600d2350 2023-12-19T18:29:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.692+0000 7efd662ea640 1 --2- 172.21.15.27:0/2355213907 >> v2:172.21.15.192:3300/0 conn(0x7efd601604d0 0x7efd600b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:07.693 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.693+0000 7efd65ae9640 1 --2- 172.21.15.27:0/2355213907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd600d2350 0x7efd600b4d20 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7efd6006a040 tx=0x7efd5402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c14db6df250e503b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:07.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.694+0000 7efd652e8640 1 -- 172.21.15.27:0/2355213907 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54031870 con 0x7efd600d2350 2023-12-19T18:29:07.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.694+0000 7efd652e8640 1 -- 172.21.15.27:0/2355213907 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd540319d0 con 0x7efd600d2350 2023-12-19T18:29:07.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.694+0000 7efd652e8640 1 -- 172.21.15.27:0/2355213907 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54031d60 con 0x7efd600d2350 2023-12-19T18:29:07.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.694+0000 7efd672ec640 1 -- 172.21.15.27:0/2355213907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd600d2350 msgr2=0x7efd600b4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:07.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.694+0000 7efd672ec640 1 --2- 172.21.15.27:0/2355213907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd600d2350 0x7efd600b4d20 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7efd6006a040 tx=0x7efd5402b910 comp rx=0 tx=0).stop 2023-12-19T18:29:07.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.694+0000 7efd672ec640 1 -- 172.21.15.27:0/2355213907 shutdown_connections 2023-12-19T18:29:07.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.694+0000 7efd672ec640 1 --2- 172.21.15.27:0/2355213907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd600d2350 0x7efd600b4d20 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:07.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.695+0000 7efd672ec640 1 --2- 172.21.15.27:0/2355213907 >> v2:172.21.15.192:3300/0 conn(0x7efd601604d0 0x7efd600b47e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:07.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.695+0000 7efd672ec640 1 -- 172.21.15.27:0/2355213907 >> 172.21.15.27:0/2355213907 conn(0x7efd600cf7e0 msgr2=0x7efd600cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:07.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.695+0000 7efd672ec640 1 -- 172.21.15.27:0/2355213907 shutdown_connections 2023-12-19T18:29:07.695 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.695+0000 7efd672ec640 1 -- 172.21.15.27:0/2355213907 wait complete. 2023-12-19T18:29:07.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.696+0000 7efd672ec640 1 Processor -- start 2023-12-19T18:29:07.696 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.697+0000 7efd672ec640 1 -- start start 2023-12-19T18:29:07.697 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.697+0000 7efd672ec640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd600d0300 0x7efd6015a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.697+0000 7efd672ec640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd600d2350 0x7efd600b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.697+0000 7efd672ec640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd601604d0 0x7efd600b3f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.697+0000 7efd672ec640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd6015ab40 con 0x7efd600d2350 2023-12-19T18:29:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.697+0000 7efd672ec640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd600cc000 con 0x7efd601604d0 2023-12-19T18:29:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.697+0000 7efd672ec640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd600ccb40 con 0x7efd600d0300 2023-12-19T18:29:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.698+0000 7efd65ae9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd600d2350 0x7efd600b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.698+0000 7efd65ae9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd600d2350 0x7efd600b3a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39668/0 (socket says 172.21.15.27:39668) 2023-12-19T18:29:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.698+0000 7efd65ae9640 1 -- 172.21.15.27:0/1653867482 learned_addr learned my addr 172.21.15.27:0/1653867482 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.698+0000 7efd662ea640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd600d0300 0x7efd6015a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.698+0000 7efd66aeb640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd601604d0 0x7efd600b3f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.698+0000 7efd65ae9640 1 -- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd600d0300 msgr2=0x7efd6015a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.698+0000 7efd65ae9640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd600d0300 0x7efd6015a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.698+0000 7efd65ae9640 1 -- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd601604d0 msgr2=0x7efd600b3f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.698+0000 7efd65ae9640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd601604d0 0x7efd600b3f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.698+0000 7efd65ae9640 1 -- 172.21.15.27:0/1653867482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd540094d0 con 0x7efd600d2350 2023-12-19T18:29:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.698+0000 7efd66aeb640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd601604d0 0x7efd600b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.699+0000 7efd65ae9640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd600d2350 0x7efd600b3a20 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7efd54000c00 tx=0x7efd540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.699+0000 7efd52ffd640 1 -- 172.21.15.27:0/1653867482 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd54031840 con 0x7efd600d2350 2023-12-19T18:29:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.699+0000 7efd52ffd640 1 -- 172.21.15.27:0/1653867482 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd540443f0 con 0x7efd600d2350 2023-12-19T18:29:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.699+0000 7efd52ffd640 1 -- 172.21.15.27:0/1653867482 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd540447a0 con 0x7efd600d2350 2023-12-19T18:29:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.699+0000 7efd672ec640 1 -- 172.21.15.27:0/1653867482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd600cc2a0 con 0x7efd600d2350 2023-12-19T18:29:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.699+0000 7efd672ec640 1 -- 172.21.15.27:0/1653867482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd600cc860 con 0x7efd600d2350 2023-12-19T18:29:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.700+0000 7efd672ec640 1 -- 172.21.15.27:0/1653867482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd60066f80 con 0x7efd600d2350 2023-12-19T18:29:07.701 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.701+0000 7efd52ffd640 1 -- 172.21.15.27:0/1653867482 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7efd5404d450 con 0x7efd600d2350 2023-12-19T18:29:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.703+0000 7efd52ffd640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd4406a430 0x7efd4406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:07.702 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.703+0000 7efd52ffd640 1 -- 172.21.15.27:0/1653867482 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7efd5402d040 con 0x7efd600d2350 2023-12-19T18:29:07.703 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.703+0000 7efd662ea640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd4406a430 0x7efd4406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:07.704 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.705+0000 7efd662ea640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd4406a430 0x7efd4406c8f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00b840 tx=0x7efd5c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:07.707 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.708+0000 7efd52ffd640 1 -- 172.21.15.27:0/1653867482 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7efd540766f0 con 0x7efd600d2350 2023-12-19T18:29:07.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:07.973+0000 7efd672ec640 1 -- 172.21.15.27:0/1653867482 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]} v 0) v1 -- 0x7efd6004e840 con 0x7efd600d2350 2023-12-19T18:29:08.969 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:08.969+0000 7efd52ffd640 1 -- 172.21.15.27:0/1653867482 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_full.TestQuotaFull.test_full_same_file"]}]=0 v678) v1 ==== 135+0+0 (secure 0 0 0) 0x7efd540350c0 con 0x7efd600d2350 2023-12-19T18:29:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:08.972+0000 7efd672ec640 1 -- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd4406a430 msgr2=0x7efd4406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:08.972+0000 7efd672ec640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd4406a430 0x7efd4406c8f0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7efd5c00b840 tx=0x7efd5c008040 comp rx=0 tx=0).stop 2023-12-19T18:29:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:08.973+0000 7efd672ec640 1 -- 172.21.15.27:0/1653867482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd600d2350 msgr2=0x7efd600b3a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:08.973+0000 7efd672ec640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd600d2350 0x7efd600b3a20 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7efd54000c00 tx=0x7efd540046c0 comp rx=0 tx=0).stop 2023-12-19T18:29:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:08.973+0000 7efd672ec640 1 -- 172.21.15.27:0/1653867482 shutdown_connections 2023-12-19T18:29:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:08.973+0000 7efd672ec640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7efd4406a430 0x7efd4406c8f0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:08.973+0000 7efd672ec640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7efd601604d0 0x7efd600b3f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:08.973+0000 7efd672ec640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd600d2350 0x7efd600b3a20 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:08.973+0000 7efd672ec640 1 --2- 172.21.15.27:0/1653867482 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7efd600d0300 0x7efd6015a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:08.973 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:08.973+0000 7efd672ec640 1 -- 172.21.15.27:0/1653867482 >> 172.21.15.27:0/1653867482 conn(0x7efd600cf7e0 msgr2=0x7efd600d06e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:08.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:08.973+0000 7efd672ec640 1 -- 172.21.15.27:0/1653867482 shutdown_connections 2023-12-19T18:29:08.974 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:08.973+0000 7efd672ec640 1 -- 172.21.15.27:0/1653867482 wait complete. 2023-12-19T18:29:08.994 INFO:tasks.cephfs_test_runner:test_full_same_file (tasks.cephfs.test_full.TestQuotaFull) ... ok 2023-12-19T18:29:08.995 INFO:tasks.cephfs_test_runner: 2023-12-19T18:29:08.995 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-19T18:29:08.995 INFO:tasks.cephfs_test_runner:Ran 10 tests in 803.334s 2023-12-19T18:29:08.995 INFO:tasks.cephfs_test_runner: 2023-12-19T18:29:08.995 INFO:tasks.cephfs_test_runner:OK 2023-12-19T18:29:08.995 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-12-19T18:29:09.005 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-12-19T18:29:09.145 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-12-19T18:29:09.146 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:29:09.146 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:29:09.213 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:29:09.213 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2023-12-19T18:29:09.213 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:29:09.275 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054283,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249396k,nr_inodes=812349,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498800k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-19T18:29:09.276 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:29:09.276 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:29:09.336 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:29:09.336 DEBUG:teuthology.orchestra.run.smithi192:> set -ex 2023-12-19T18:29:09.336 DEBUG:teuthology.orchestra.run.smithi192:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-19T18:29:09.389 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4054288,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=27,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3249400k,nr_inodes=812350,mode=700,uid=1000,gid=1301 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6498808k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-19T18:29:09.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:29:09.389 DEBUG:teuthology.orchestra.run.smithi192:> ip netns list 2023-12-19T18:29:09.445 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-19T18:29:09.445 INFO:teuthology.orchestra.run.smithi192.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-19T18:29:09.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:29:09.446 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-19T18:29:09.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:29:09.542 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-19T18:29:09.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:29:09.635 DEBUG:teuthology.orchestra.run.smithi192:> sudo ip link delete ceph-brx 2023-12-19T18:29:09.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:29:09.781 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2023-12-19T18:29:09.845 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-19T18:29:09.845 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-19T18:29:09.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:29:09.847 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-19T18:29:09.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:29:09.959 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-19T18:29:10.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-19T18:29:10.053 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2023-12-19T18:29:10.205 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-12-19T18:29:10.219 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-19T18:29:10.220 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:29:10.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.422+0000 7fcf6a48a640 1 Processor -- start 2023-12-19T18:29:10.422 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.422+0000 7fcf6a48a640 1 -- start start 2023-12-19T18:29:10.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.424+0000 7fcf6a48a640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fcf641553a0 0x7fcf640c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.424+0000 7fcf6a48a640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcf640afeb0 0x7fcf6415e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.424+0000 7fcf6a48a640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf6415f090 con 0x7fcf6415ec60 2023-12-19T18:29:10.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.424+0000 7fcf6a48a640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf640cbcf0 con 0x7fcf640afeb0 2023-12-19T18:29:10.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.424+0000 7fcf6a48a640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf640b0360 con 0x7fcf641553a0 2023-12-19T18:29:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.425+0000 7fcf68c87640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcf640afeb0 0x7fcf6415e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.425+0000 7fcf68c87640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fcf640afeb0 0x7fcf6415e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39682/0 (socket says 172.21.15.27:39682) 2023-12-19T18:29:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.425+0000 7fcf68c87640 1 -- 172.21.15.27:0/2260111883 learned_addr learned my addr 172.21.15.27:0/2260111883 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.425+0000 7fcf69c89640 1 --1- 172.21.15.27:0/2260111883 >> v1:172.21.15.27:6789/0 conn(0x7fcf6415ec60 0x7fcf6415b130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53366/0 (socket says 172.21.15.27:53366) 2023-12-19T18:29:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.425+0000 7fcf69488640 1 --2- 172.21.15.27:0/2260111883 >> v2:172.21.15.192:3300/0 conn(0x7fcf641553a0 0x7fcf640c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.425+0000 7fcf68c87640 1 -- 172.21.15.27:0/2260111883 >> v2:172.21.15.192:3300/0 conn(0x7fcf641553a0 msgr2=0x7fcf640c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.425+0000 7fcf68c87640 1 --2- 172.21.15.27:0/2260111883 >> v2:172.21.15.192:3300/0 conn(0x7fcf641553a0 0x7fcf640c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.425+0000 7fcf68c87640 1 -- 172.21.15.27:0/2260111883 >> v1:172.21.15.27:6789/0 conn(0x7fcf6415ec60 legacy=0x7fcf6415b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.425+0000 7fcf5bfff640 1 -- 172.21.15.27:0/2260111883 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 449567838 0 0) 0x7fcf6415f090 con 0x7fcf6415ec60 2023-12-19T18:29:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.425+0000 7fcf68c87640 1 -- 172.21.15.27:0/2260111883 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf640cb350 con 0x7fcf640afeb0 2023-12-19T18:29:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.426+0000 7fcf68c87640 1 --2- 172.21.15.27:0/2260111883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf640afeb0 0x7fcf6415e720 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fcf640cc1c0 tx=0x7fcf5402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4474f762a46836ca server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.427+0000 7fcf5bfff640 1 -- 172.21.15.27:0/2260111883 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54031870 con 0x7fcf640afeb0 2023-12-19T18:29:10.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.427+0000 7fcf5bfff640 1 -- 172.21.15.27:0/2260111883 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf540319d0 con 0x7fcf640afeb0 2023-12-19T18:29:10.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.427+0000 7fcf5bfff640 1 -- 172.21.15.27:0/2260111883 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf54031d60 con 0x7fcf640afeb0 2023-12-19T18:29:10.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.427+0000 7fcf6a48a640 1 -- 172.21.15.27:0/2260111883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf640afeb0 msgr2=0x7fcf6415e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:10.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.427+0000 7fcf6a48a640 1 --2- 172.21.15.27:0/2260111883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf640afeb0 0x7fcf6415e720 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7fcf640cc1c0 tx=0x7fcf5402b910 comp rx=0 tx=0).stop 2023-12-19T18:29:10.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.428+0000 7fcf6a48a640 1 -- 172.21.15.27:0/2260111883 shutdown_connections 2023-12-19T18:29:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.428+0000 7fcf6a48a640 1 --2- 172.21.15.27:0/2260111883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf640afeb0 0x7fcf6415e720 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.428+0000 7fcf6a48a640 1 --2- 172.21.15.27:0/2260111883 >> v2:172.21.15.192:3300/0 conn(0x7fcf641553a0 0x7fcf640c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.428+0000 7fcf6a48a640 1 -- 172.21.15.27:0/2260111883 >> 172.21.15.27:0/2260111883 conn(0x7fcf64069c70 msgr2=0x7fcf6406a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.428+0000 7fcf6a48a640 1 -- 172.21.15.27:0/2260111883 shutdown_connections 2023-12-19T18:29:10.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.428+0000 7fcf6a48a640 1 -- 172.21.15.27:0/2260111883 wait complete. 2023-12-19T18:29:10.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.429+0000 7fcf6a48a640 1 Processor -- start 2023-12-19T18:29:10.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.430+0000 7fcf6a48a640 1 -- start start 2023-12-19T18:29:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf6a48a640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf640afeb0 0x7fcf641557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf6a48a640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf641553a0 0x7fcf6406c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf6a48a640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf6415ec60 0x7fcf6406c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf6a48a640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf64156e30 con 0x7fcf640afeb0 2023-12-19T18:29:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf6a48a640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf640b2340 con 0x7fcf6415ec60 2023-12-19T18:29:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf6a48a640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf641562b0 con 0x7fcf641553a0 2023-12-19T18:29:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf69488640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf640afeb0 0x7fcf641557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf69488640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf640afeb0 0x7fcf641557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39696/0 (socket says 172.21.15.27:39696) 2023-12-19T18:29:10.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf69488640 1 -- 172.21.15.27:0/610362695 learned_addr learned my addr 172.21.15.27:0/610362695 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf69488640 1 -- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf641553a0 msgr2=0x7fcf6406c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf69c89640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf6415ec60 0x7fcf6406c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:10.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf69488640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf641553a0 0x7fcf6406c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf69488640 1 -- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf6415ec60 msgr2=0x7fcf6406c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf69488640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf6415ec60 0x7fcf6406c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf69488640 1 -- 172.21.15.27:0/610362695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf540094d0 con 0x7fcf640afeb0 2023-12-19T18:29:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.431+0000 7fcf69c89640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf6415ec60 0x7fcf6406c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.432+0000 7fcf69488640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf640afeb0 0x7fcf641557f0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000d800 tx=0x7fcf6000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.432+0000 7fcf59ffb640 1 -- 172.21.15.27:0/610362695 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6000b6c0 con 0x7fcf640afeb0 2023-12-19T18:29:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.432+0000 7fcf59ffb640 1 -- 172.21.15.27:0/610362695 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6000b840 con 0x7fcf640afeb0 2023-12-19T18:29:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.432+0000 7fcf59ffb640 1 -- 172.21.15.27:0/610362695 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6000bbf0 con 0x7fcf640afeb0 2023-12-19T18:29:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.432+0000 7fcf6a48a640 1 -- 172.21.15.27:0/610362695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf64156590 con 0x7fcf640afeb0 2023-12-19T18:29:10.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.432+0000 7fcf6a48a640 1 -- 172.21.15.27:0/610362695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf64156b50 con 0x7fcf640afeb0 2023-12-19T18:29:10.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.433+0000 7fcf6a48a640 1 -- 172.21.15.27:0/610362695 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf6404e840 con 0x7fcf640afeb0 2023-12-19T18:29:10.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.434+0000 7fcf59ffb640 1 -- 172.21.15.27:0/610362695 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fcf6000bd50 con 0x7fcf640afeb0 2023-12-19T18:29:10.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.436+0000 7fcf59ffb640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf4806a430 0x7fcf4806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.436+0000 7fcf59ffb640 1 -- 172.21.15.27:0/610362695 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fcf60088c80 con 0x7fcf640afeb0 2023-12-19T18:29:10.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.436+0000 7fcf68c87640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf4806a430 0x7fcf4806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:10.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.438+0000 7fcf68c87640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf4806a430 0x7fcf4806c8f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcf54036000 tx=0x7fcf54030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:10.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.440+0000 7fcf59ffb640 1 -- 172.21.15.27:0/610362695 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fcf600544d0 con 0x7fcf640afeb0 2023-12-19T18:29:10.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.630+0000 7fcf6a48a640 1 -- 172.21.15.27:0/610362695 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcf640cb910 con 0x7fcf4806a430 2023-12-19T18:29:10.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.637+0000 7fcf59ffb640 1 -- 172.21.15.27:0/610362695 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47286 (secure 0 0 0) 0x7fcf54031b40 con 0x7fcf4806a430 2023-12-19T18:29:10.638 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:29:10.638 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:29:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.640+0000 7fcf6a48a640 1 -- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf4806a430 msgr2=0x7fcf4806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.640+0000 7fcf6a48a640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf4806a430 0x7fcf4806c8f0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fcf54036000 tx=0x7fcf54030000 comp rx=0 tx=0).stop 2023-12-19T18:29:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.640+0000 7fcf6a48a640 1 -- 172.21.15.27:0/610362695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf640afeb0 msgr2=0x7fcf641557f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.640+0000 7fcf6a48a640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf640afeb0 0x7fcf641557f0 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7fcf6000d800 tx=0x7fcf6000dcc0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.640+0000 7fcf6a48a640 1 -- 172.21.15.27:0/610362695 shutdown_connections 2023-12-19T18:29:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.640+0000 7fcf6a48a640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fcf4806a430 0x7fcf4806c8f0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.640 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.640+0000 7fcf6a48a640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fcf6415ec60 0x7fcf6406c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.640+0000 7fcf6a48a640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fcf641553a0 0x7fcf6406c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.640+0000 7fcf6a48a640 1 --2- 172.21.15.27:0/610362695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf640afeb0 0x7fcf641557f0 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.640+0000 7fcf6a48a640 1 -- 172.21.15.27:0/610362695 >> 172.21.15.27:0/610362695 conn(0x7fcf64069c70 msgr2=0x7fcf64154650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.640+0000 7fcf6a48a640 1 -- 172.21.15.27:0/610362695 shutdown_connections 2023-12-19T18:29:10.641 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.640+0000 7fcf6a48a640 1 -- 172.21.15.27:0/610362695 wait complete. 2023-12-19T18:29:10.659 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":635,"stamp":"2023-12-19T18:29:10.021060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10484,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552015,"statfs":{"total":1600000000,"available":1589264040,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52485472,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-128,"num_write_kb":-85084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.382243"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":336,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-19T18:29:05.956617+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:29:05.956617+0000","last_peered":"2023-12-19T18:29:05.956617+0000","last_clean":"2023-12-19T18:29:05.956617+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:29:05.956617+0000","last_undegraded":"2023-12-19T18:29:05.956617+0000","last_fullsized":"2023-12-19T18:29:05.956617+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":0.67000000000000004,"5min":0.59299999999999997,"15min":0.45900000000000002},"max":{"1min":1.5369999999999999,"5min":1.756,"15min":5.6879999999999997},"last":1.54},{"interface":"front","average":{"1min":1.274,"5min":1.236,"15min":1.228},"min":{"1min":0.78600000000000003,"5min":0.71899999999999997,"15min":0.55800000000000005},"max":{"1min":1.6599999999999999,"5min":1.7030000000000001,"15min":1.79},"last":1.5009999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.179,"15min":1.23},"min":{"1min":0.75600000000000001,"5min":0.65800000000000003,"15min":0.54400000000000004},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":10.926},"last":1.264},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.26,"15min":1.268},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.67900000000000005},"max":{"1min":1.534,"5min":1.736,"15min":10.904999999999999},"last":1.4350000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1379999999999999,"15min":1.117},"min":{"1min":0.90700000000000003,"5min":0.66000000000000003,"15min":0.61499999999999999},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":3.1960000000000002},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1830000000000001,"15min":1.2},"min":{"1min":0.91500000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5469999999999999,"5min":1.6799999999999999,"15min":6.008},"last":1.371}]},{"osd":3,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.2749999999999999,"15min":1.2290000000000001},"min":{"1min":0.96899999999999997,"5min":0.79000000000000004,"15min":0.61399999999999999},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":3.0379999999999998},"last":1.468},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3420000000000001,"15min":1.3400000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69599999999999995},"max":{"1min":1.6220000000000001,"5min":1.772,"15min":5.8120000000000003},"last":1.6459999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.202,"15min":1.1850000000000001},"min":{"1min":0.81299999999999994,"5min":0.61099999999999999,"15min":0.47899999999999998},"max":{"1min":1.8340000000000001,"5min":1.847,"15min":4.0110000000000001},"last":1.147},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.256,"15min":1.256},"min":{"1min":0.94699999999999995,"5min":0.624,"15min":0.57599999999999996},"max":{"1min":1.579,"5min":1.8,"15min":3.3180000000000001},"last":1.347}]},{"osd":5,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3049999999999999,"15min":1.3380000000000001},"min":{"1min":0.995,"5min":0.68999999999999995,"15min":0.53800000000000003},"max":{"1min":1.7789999999999999,"5min":1.8140000000000001,"15min":7.6429999999999998},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3620000000000001,"15min":1.3660000000000001},"min":{"1min":0.96899999999999997,"5min":0.627,"15min":0.53500000000000003},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":6.0839999999999996},"last":1.3169999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.5129999999999999,"15min":1.506},"min":{"1min":1.175,"5min":0.70899999999999996,"15min":0.59899999999999998},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":3.1309999999999998},"last":1.587},{"interface":"front","average":{"1min":1.522,"5min":1.4690000000000001,"15min":1.4330000000000001},"min":{"1min":1.131,"5min":0.73199999999999998,"15min":0.56100000000000005},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":4.1479999999999997},"last":1.2050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575023,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.099,"15min":1.0680000000000001},"min":{"1min":0.68300000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.413,"5min":1.522,"15min":1.8129999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.151,"15min":1.1220000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.55600000000000005},"max":{"1min":1.587,"5min":1.6000000000000001,"15min":3.5529999999999999},"last":1.2529999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1220000000000001,"15min":1.131},"min":{"1min":0.75700000000000001,"5min":0.68600000000000005,"15min":0.68200000000000005},"max":{"1min":1.573,"5min":1.7909999999999999,"15min":4.5540000000000003},"last":1.419},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.177,"15min":1.179},"min":{"1min":0.80100000000000005,"5min":0.64600000000000002,"15min":0.60299999999999998},"max":{"1min":1.5569999999999999,"5min":1.5960000000000001,"15min":1.885},"last":1.0469999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.258,"15min":1.21},"min":{"1min":0.89900000000000002,"5min":0.78900000000000003,"15min":0.626},"max":{"1min":1.5569999999999999,"5min":1.6479999999999999,"15min":1.821},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2190000000000001,"15min":1.216},"min":{"1min":0.85899999999999999,"5min":0.62,"15min":0.60899999999999999},"max":{"1min":1.514,"5min":1.7669999999999999,"15min":3.7909999999999999},"last":0.98299999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3129999999999999,"15min":1.2569999999999999},"min":{"1min":0.90800000000000003,"5min":0.71799999999999997,"15min":0.67800000000000005},"max":{"1min":1.5469999999999999,"5min":1.804,"15min":3.8820000000000001},"last":1.179},{"interface":"front","average":{"1min":1.258,"5min":1.248,"15min":1.2050000000000001},"min":{"1min":0.96099999999999997,"5min":0.71199999999999997,"15min":0.69899999999999995},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":4.6189999999999998},"last":1.141}]},{"osd":4,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.306,"15min":1.2809999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.65600000000000003},"max":{"1min":1.6259999999999999,"5min":1.7529999999999999,"15min":3.0680000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.415,"15min":1.3939999999999999},"min":{"1min":1.0529999999999999,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.7090000000000001,"5min":2.1040000000000001,"15min":3.7000000000000002},"last":0.86499999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2809999999999999,"15min":1.2470000000000001},"min":{"1min":0.91500000000000004,"5min":0.84899999999999998,"15min":0.65500000000000003},"max":{"1min":1.7929999999999999,"5min":1.9510000000000001,"15min":4.6680000000000001},"last":1.284},{"interface":"front","average":{"1min":1.375,"5min":1.4079999999999999,"15min":1.3959999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.626},"max":{"1min":1.609,"5min":1.794,"15min":4.1159999999999997},"last":1.4650000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.444,"15min":1.403},"min":{"1min":1.0149999999999999,"5min":0.65700000000000003,"15min":0.61899999999999999},"max":{"1min":1.663,"5min":2.02,"15min":3.9670000000000001},"last":1.089},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4470000000000001,"15min":1.413},"min":{"1min":1.0309999999999999,"5min":0.999,"15min":0.79300000000000004},"max":{"1min":1.855,"5min":2.2149999999999999,"15min":5.6639999999999997},"last":1.4930000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575023,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1742,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193569,"statfs":{"total":200000000,"available":198215229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.282,"15min":1.2150000000000001},"min":{"1min":0.76900000000000002,"5min":0.69999999999999996,"15min":0.61899999999999999},"max":{"1min":1.6830000000000001,"5min":2.089,"15min":2.089},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.304,"15min":1.2509999999999999},"min":{"1min":0.69399999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6020000000000001,"5min":1.641,"15min":1.913},"last":1.1299999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.167,"15min":1.1399999999999999},"min":{"1min":0.91600000000000004,"5min":0.71999999999999997,"15min":0.65700000000000003},"max":{"1min":1.571,"5min":1.851,"15min":1.9490000000000001},"last":1.532},{"interface":"front","average":{"1min":1.202,"5min":1.1890000000000001,"15min":1.141},"min":{"1min":0.89400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.452,"5min":1.899,"15min":1.899},"last":1.1739999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2050000000000001,"15min":1.149},"min":{"1min":0.73999999999999999,"5min":0.72199999999999998,"15min":0.59499999999999997},"max":{"1min":1.468,"5min":1.806,"15min":1.806},"last":1.083},{"interface":"front","average":{"1min":1.343,"5min":1.282,"15min":1.2230000000000001},"min":{"1min":0.92500000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.651,"5min":1.8,"15min":2.036},"last":1.4410000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.198,"15min":1.1599999999999999},"min":{"1min":0.93200000000000005,"5min":0.76600000000000001,"15min":0.68899999999999995},"max":{"1min":1.425,"5min":1.635,"15min":1.7450000000000001},"last":1.262},{"interface":"front","average":{"1min":1.411,"5min":1.3540000000000001,"15min":1.3080000000000001},"min":{"1min":0.99199999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.843,"5min":2.0369999999999999,"15min":2.0590000000000002},"last":1.3520000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.288,"15min":1.2629999999999999},"min":{"1min":1.024,"5min":0.52300000000000002,"15min":0.46700000000000003},"max":{"1min":1.5589999999999999,"5min":1.7250000000000001,"15min":2.0329999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.1379999999999999,"15min":1.159},"min":{"1min":0.86899999999999999,"5min":0.64500000000000002,"15min":0.57099999999999995},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.901},"last":1.1399999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.425,"15min":1.411},"min":{"1min":1.1850000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.952,"5min":2.1579999999999999,"15min":2.1589999999999998},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.2969999999999999,"15min":1.2889999999999999},"min":{"1min":0.94999999999999996,"5min":0.58299999999999996,"15min":0.49299999999999999},"max":{"1min":1.5720000000000001,"5min":1.704,"15min":1.976},"last":1.3160000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.2769999999999999,"15min":1.2729999999999999},"min":{"1min":0.83599999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4419999999999999,"15min":1.4430000000000001},"min":{"1min":1.1759999999999999,"5min":0.67100000000000004,"15min":0.623},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":2.117},"last":1.3799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575023,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.069,"15min":1.024},"min":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.52700000000000002},"max":{"1min":1.403,"5min":1.972,"15min":1.972},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.1180000000000001,"15min":1.073},"min":{"1min":0.88300000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5309999999999999,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.0780000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.204,"15min":1.1679999999999999},"min":{"1min":0.85799999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.4199999999999999,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.288,"15min":1.2350000000000001},"min":{"1min":1.056,"5min":0.64700000000000002,"15min":0.61699999999999999},"max":{"1min":1.6459999999999999,"5min":1.7450000000000001,"15min":1.875},"last":1.115}]},{"osd":2,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.1559999999999999,"15min":1.135},"min":{"1min":0.77200000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.597,"5min":1.744,"15min":1.744},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1839999999999999,"15min":1.1459999999999999},"min":{"1min":0.93000000000000005,"5min":0.747,"15min":0.70499999999999996},"max":{"1min":1.8149999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.93500000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2509999999999999,"15min":1.268},"min":{"1min":1.0549999999999999,"5min":0.754,"15min":0.754},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.946},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.369,"15min":1.335},"min":{"1min":1.123,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.6619999999999999,"5min":1.927,"15min":1.927},"last":1.4019999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2569999999999999,"15min":1.248},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.44500000000000001},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.357,"15min":1.369},"min":{"1min":0.82599999999999996,"5min":0.59099999999999997,"15min":0.504},"max":{"1min":1.7729999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.3220000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.2969999999999999,"15min":1.304},"min":{"1min":0.88500000000000001,"5min":0.68100000000000005,"15min":0.432},"max":{"1min":1.7490000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4219999999999999,"15min":1.421},"min":{"1min":1.1599999999999999,"5min":0.93700000000000006,"15min":0.64500000000000002},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.4399999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.3560000000000001,"15min":1.363},"min":{"1min":0.996,"5min":0.77300000000000002,"15min":0.56399999999999995},"max":{"1min":2.702,"5min":2.702,"15min":2.702},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4259999999999999,"15min":1.421},"min":{"1min":1.115,"5min":0.78600000000000003,"15min":0.57399999999999995},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.5069999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575024,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.90900000000000003,"15min":0.92200000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35299999999999998},"max":{"1min":1.754,"5min":1.754,"15min":3.4350000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.038,"15min":0.99299999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.39500000000000002},"max":{"1min":1.591,"5min":1.8109999999999999,"15min":4.3490000000000002},"last":0.61699999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.01,"15min":1.024},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.41299999999999998},"max":{"1min":1.8520000000000001,"5min":1.9690000000000001,"15min":5.2510000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.4299999999999999,"5min":1.643,"15min":5.4119999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1899999999999999,"15min":1.236},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.46000000000000002},"max":{"1min":2.1059999999999999,"5min":2.4649999999999999,"15min":13.833},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.304,"15min":1.3520000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8460000000000001,"5min":2.379,"15min":16.748999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3080000000000001,"15min":1.29},"min":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":0.65600000000000003},"max":{"1min":1.75,"5min":2.0150000000000001,"15min":5.1790000000000003},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.343,"5min":1.367,"15min":1.3919999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.73199999999999998},"max":{"1min":1.929,"5min":2.0819999999999999,"15min":4.569},"last":0.89000000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.4339999999999999,"15min":1.429},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.76000000000000001},"max":{"1min":1.925,"5min":2.0449999999999999,"15min":5.3289999999999997},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.327,"5min":1.3859999999999999,"15min":1.347},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.69999999999999996},"max":{"1min":2.004,"5min":2.1469999999999998,"15min":4.6369999999999996},"last":0.95699999999999996}]},{"osd":6,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.454,"15min":1.4510000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.81599999999999995},"max":{"1min":1.9790000000000001,"5min":2.2040000000000002,"15min":4.5039999999999996},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.516,"5min":1.5189999999999999,"15min":1.5089999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.80500000000000005},"max":{"1min":2.254,"5min":2.254,"15min":5.3739999999999997},"last":1.1950000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.4990000000000001,"15min":1.454},"min":{"1min":0.98899999999999999,"5min":0.89700000000000002,"15min":0.73199999999999998},"max":{"1min":2.1389999999999998,"5min":2.2869999999999999,"15min":5.1349999999999998},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.5369999999999999,"15min":1.5489999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.0550000000000002,"5min":2.351,"15min":4.4619999999999997},"last":0.84199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.92600000000000005,"15min":0.97499999999999998},"min":{"1min":0.48099999999999998,"5min":0.43099999999999999,"15min":0.34399999999999997},"max":{"1min":1.5309999999999999,"5min":2.2799999999999998,"15min":9.2550000000000008},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.04,"15min":1.0189999999999999},"min":{"1min":0.45900000000000002,"5min":0.38400000000000001,"15min":0.33900000000000002},"max":{"1min":1.4399999999999999,"5min":1.8049999999999999,"15min":4.7850000000000001},"last":1.4179999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.93999999999999995,"15min":0.98999999999999999},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.4039999999999999,"5min":2.0169999999999999,"15min":8.2720000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.101,"15min":1.1040000000000001},"min":{"1min":0.61199999999999999,"5min":0.54200000000000004,"15min":0.46600000000000003},"max":{"1min":1.339,"5min":1.8460000000000001,"15min":5.0620000000000003},"last":1.339}]},{"osd":3,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.2669999999999999,"15min":1.226},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5940000000000001,"5min":2.8159999999999998,"15min":8.3109999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1160000000000001,"15min":1.163},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.41799999999999998},"max":{"1min":1.7869999999999999,"5min":2.6240000000000001,"15min":8.0690000000000008},"last":1.089}]},{"osd":4,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3819999999999999,"15min":1.395},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9850000000000001,"5min":2.0059999999999998,"15min":5.3929999999999998},"last":1.446},{"interface":"front","average":{"1min":1.399,"5min":1.458,"15min":1.46},"min":{"1min":1.0209999999999999,"5min":0.85799999999999998,"15min":0.78300000000000003},"max":{"1min":2.3079999999999998,"5min":2.4300000000000002,"15min":7.3570000000000002},"last":1.649}]},{"osd":5,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3720000000000001,"15min":1.4059999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.613,"5min":2.1680000000000001,"15min":8.1839999999999993},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.385,"15min":1.391},"min":{"1min":0.90000000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":5.165},"last":1.377}]},{"osd":6,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.413,"15min":1.4219999999999999},"min":{"1min":0.99399999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.96,"5min":2.4630000000000001,"15min":5.1059999999999999},"last":1.96},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.468,"15min":1.48},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77000000000000002},"max":{"1min":1.7709999999999999,"5min":2.7559999999999998,"15min":8.1370000000000005},"last":1.7709999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.504,"15min":1.5089999999999999},"min":{"1min":0.91800000000000004,"5min":0.80000000000000004,"15min":0.78200000000000003},"max":{"1min":2.0870000000000002,"5min":2.3959999999999999,"15min":5.1449999999999996},"last":1.821},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5629999999999999,"15min":1.5680000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86499999999999999},"max":{"1min":1.8720000000000001,"5min":2.6320000000000001,"15min":8.2300000000000004},"last":1.8720000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575024,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1742,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193569,"statfs":{"total":200000000,"available":198215229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93200000000000005,"15min":0.874},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.34699999999999998},"max":{"1min":1.472,"5min":2.262,"15min":2.262},"last":1.472},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.014,"15min":0.96299999999999997},"min":{"1min":0.60599999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4970000000000001,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.1220000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.02,"15min":1.0309999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.41399999999999998},"max":{"1min":1.621,"5min":1.621,"15min":2.5739999999999998},"last":1.621},{"interface":"front","average":{"1min":0.92900000000000005,"5min":1.0069999999999999,"15min":0.97799999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.38500000000000001},"max":{"1min":1.276,"5min":1.718,"15min":1.845},"last":1.2190000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.3300000000000001,"15min":1.2569999999999999},"min":{"1min":0.624,"5min":0.53000000000000003,"15min":0.48599999999999999},"max":{"1min":1.603,"5min":2.089,"15min":2.137},"last":1.571},{"interface":"front","average":{"1min":1.099,"5min":1.2350000000000001,"15min":1.198},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.44900000000000001},"max":{"1min":1.6659999999999999,"5min":2.0720000000000001,"15min":2.3959999999999999},"last":1.425}]},{"osd":4,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3140000000000001,"15min":1.3120000000000001},"min":{"1min":0.83099999999999996,"5min":0.80500000000000005,"15min":0.69099999999999995},"max":{"1min":1.694,"5min":2.1859999999999999,"15min":2.444},"last":1.694},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.381,"15min":1.3520000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.71099999999999997},"max":{"1min":1.665,"5min":1.8859999999999999,"15min":2.6440000000000001},"last":1.351}]},{"osd":5,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.425,"15min":1.371},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":2.5459999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.431,"15min":1.4330000000000001},"min":{"1min":1.0249999999999999,"5min":0.85899999999999999,"15min":0.70499999999999996},"max":{"1min":1.8240000000000001,"5min":1.9339999999999999,"15min":2.6179999999999999},"last":1.806}]},{"osd":6,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.466,"15min":1.4139999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":2.1619999999999999,"5min":2.274,"15min":2.7000000000000002},"last":1.534},{"interface":"front","average":{"1min":1.379,"5min":1.5,"15min":1.478},"min":{"1min":0.872,"5min":0.872,"15min":0.76400000000000001},"max":{"1min":2.153,"5min":2.153,"15min":2.895},"last":1.748}]},{"osd":7,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4490000000000001,"15min":1.4470000000000001},"min":{"1min":0.92600000000000005,"5min":0.83199999999999996,"15min":0.69099999999999995},"max":{"1min":2.044,"5min":2.044,"15min":9.4830000000000005},"last":1.288},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.5600000000000001,"15min":1.546},"min":{"1min":0.93899999999999995,"5min":0.80800000000000005,"15min":0.71899999999999997},"max":{"1min":2.198,"5min":2.3420000000000001,"15min":9.4109999999999996},"last":1.657}]}]},{"osd":0,"up_from":13,"seq":55834575023,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92200000000000004,"15min":0.92100000000000004},"min":{"1min":0.60699999999999998,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":1.425,"5min":1.671,"15min":2.3199999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1100000000000001,"15min":1.0720000000000001},"min":{"1min":0.58899999999999997,"5min":0.48699999999999999,"15min":0.42799999999999999},"max":{"1min":1.3480000000000001,"5min":2.2170000000000001,"15min":2.4180000000000001},"last":0.54500000000000004}]},{"osd":2,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0289999999999999,"15min":0.996},"min":{"1min":0.69399999999999995,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":1.4550000000000001,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.0609999999999999,"15min":0.996},"min":{"1min":0.64000000000000001,"5min":0.42699999999999999,"15min":0.41099999999999998},"max":{"1min":1.6870000000000001,"5min":2.3340000000000001,"15min":2.7810000000000001},"last":0.68799999999999994}]},{"osd":3,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.032,"15min":1.0249999999999999},"min":{"1min":0.63300000000000001,"5min":0.44,"15min":0.38900000000000001},"max":{"1min":1.5589999999999999,"5min":1.8200000000000001,"15min":2.5190000000000001},"last":0.747},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.349,"15min":1.2769999999999999},"min":{"1min":0.81299999999999994,"5min":0.65100000000000002,"15min":0.47499999999999998},"max":{"1min":1.6759999999999999,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.72599999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.323,"15min":1.2749999999999999},"min":{"1min":0.94499999999999995,"5min":0.67600000000000005,"15min":0.63200000000000001},"max":{"1min":1.7290000000000001,"5min":2.1920000000000002,"15min":2.5529999999999999},"last":1.002},{"interface":"front","average":{"1min":1.27,"5min":1.2789999999999999,"15min":1.28},"min":{"1min":0.81899999999999995,"5min":0.70399999999999996,"15min":0.67600000000000005},"max":{"1min":1.6699999999999999,"5min":2.056,"15min":2.9529999999999998},"last":0.85099999999999998}]},{"osd":5,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3460000000000001,"15min":1.3919999999999999},"min":{"1min":0.92800000000000005,"5min":0.70699999999999996,"15min":0.64500000000000002},"max":{"1min":1.542,"5min":1.97,"15min":16.298999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.387,"15min":1.3360000000000001},"min":{"1min":1.115,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.641,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.89500000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.4470000000000001},"min":{"1min":0.93300000000000005,"5min":0.79800000000000004,"15min":0.71699999999999997},"max":{"1min":1.744,"5min":2.4169999999999998,"15min":2.5910000000000002},"last":1.089},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4810000000000001,"15min":1.4179999999999999},"min":{"1min":1.097,"5min":0.745,"15min":0.745},"max":{"1min":1.8,"5min":2.1709999999999998,"15min":2.46},"last":1.1240000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4910000000000001,"15min":1.454},"min":{"1min":1.052,"5min":0.72099999999999997,"15min":0.70899999999999996},"max":{"1min":1.895,"5min":2.2549999999999999,"15min":2.4969999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.407,"15min":1.415},"min":{"1min":1.077,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6439999999999999,"5min":1.9370000000000001,"15min":3.1429999999999998},"last":1.179}]}]}],"pool_statfs":[]}} 2023-12-19T18:29:10.661 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:29:10.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.894+0000 7f9a37fff640 1 Processor -- start 2023-12-19T18:29:10.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.894+0000 7f9a37fff640 1 -- start start 2023-12-19T18:29:10.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.895+0000 7f9a37fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380af630 0x7f9a380afa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.895+0000 7f9a37fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a380affd0 con 0x7f9a38154030 2023-12-19T18:29:10.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.895+0000 7f9a37fff640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a380cc190 con 0x7f9a380c4550 2023-12-19T18:29:10.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.895+0000 7f9a37fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a380cdc00 con 0x7f9a380af630 2023-12-19T18:29:10.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.896+0000 7f9a36ffd640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f9a38154030 0x7f9a380cd4f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53372/0 (socket says 172.21.15.27:53372) 2023-12-19T18:29:10.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.896+0000 7f9a36ffd640 1 -- 172.21.15.27:0/338505521 learned_addr learned my addr 172.21.15.27:0/338505521 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.896+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020310715 0 0) 0x7f9a380affd0 con 0x7f9a38154030 2023-12-19T18:29:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.896+0000 7f9a367fc640 1 --2- 172.21.15.27:0/338505521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380af630 0x7f9a380afa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.896+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a20005730 con 0x7f9a38154030 2023-12-19T18:29:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3527785051 0 0) 0x7f9a20005730 con 0x7f9a38154030 2023-12-19T18:29:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380af630 msgr2=0x7f9a380afa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a35ffb640 1 --2- 172.21.15.27:0/338505521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380af630 0x7f9a380afa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 >> v1:172.21.15.192:6789/0 conn(0x7f9a380c4550 legacy=0x7f9a380c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a380cb2a0 con 0x7f9a38154030 2023-12-19T18:29:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f9a280019f0 con 0x7f9a38154030 2023-12-19T18:29:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 >> v1:172.21.15.27:6789/0 conn(0x7f9a38154030 legacy=0x7f9a380cd4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a35ffb640 1 --2- 172.21.15.27:0/338505521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380af630 0x7f9a20005bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a35ffb640 1 --2- 172.21.15.27:0/338505521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a20006180 0x7f9a200069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a35ffb640 1 --2- 172.21.15.27:0/338505521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a20006f10 0x7f9a20007390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a380cdc00 con 0x7f9a20006f10 2023-12-19T18:29:10.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2000dcd0 con 0x7f9a20006180 2023-12-19T18:29:10.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a2000de90 con 0x7f9a380af630 2023-12-19T18:29:10.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.897+0000 7f9a377fe640 1 --2- 172.21.15.27:0/338505521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a20006f10 0x7f9a20007390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.898+0000 7f9a36ffd640 1 --2- 172.21.15.27:0/338505521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380af630 0x7f9a20005bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.898+0000 7f9a367fc640 1 --2- 172.21.15.27:0/338505521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a20006180 0x7f9a200069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.898+0000 7f9a377fe640 1 -- 172.21.15.27:0/338505521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380af630 msgr2=0x7f9a20005bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.898+0000 7f9a377fe640 1 --2- 172.21.15.27:0/338505521 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a380af630 0x7f9a20005bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.898+0000 7f9a377fe640 1 -- 172.21.15.27:0/338505521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a20006180 msgr2=0x7f9a200069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.898+0000 7f9a377fe640 1 --2- 172.21.15.27:0/338505521 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a20006180 0x7f9a200069d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.898+0000 7f9a377fe640 1 -- 172.21.15.27:0/338505521 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a20005730 con 0x7f9a20006f10 2023-12-19T18:29:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.899+0000 7f9a377fe640 1 --2- 172.21.15.27:0/338505521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a20006f10 0x7f9a20007390 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f9a30001750 tx=0x7f9a20007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46cb67228c93c6b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:10.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.899+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a300052d0 con 0x7f9a20006f10 2023-12-19T18:29:10.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.899+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3002e050 con 0x7f9a20006f10 2023-12-19T18:29:10.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.899+0000 7f9a35ffb640 1 -- 172.21.15.27:0/338505521 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a30003830 con 0x7f9a20006f10 2023-12-19T18:29:10.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.900+0000 7f9a37fff640 1 -- 172.21.15.27:0/338505521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a20006f10 msgr2=0x7f9a20007390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:10.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.900+0000 7f9a37fff640 1 --2- 172.21.15.27:0/338505521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a20006f10 0x7f9a20007390 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f9a30001750 tx=0x7f9a20007a00 comp rx=0 tx=0).stop 2023-12-19T18:29:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.900+0000 7f9a377fe640 1 -- 172.21.15.27:0/338505521 reap_dead start 2023-12-19T18:29:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.900+0000 7f9a37fff640 1 -- 172.21.15.27:0/338505521 shutdown_connections 2023-12-19T18:29:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.900+0000 7f9a37fff640 1 -- 172.21.15.27:0/338505521 >> 172.21.15.27:0/338505521 conn(0x7f9a38069c70 msgr2=0x7f9a3806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.900+0000 7f9a37fff640 1 -- 172.21.15.27:0/338505521 shutdown_connections 2023-12-19T18:29:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.900+0000 7f9a37fff640 1 -- 172.21.15.27:0/338505521 wait complete. 2023-12-19T18:29:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.902+0000 7f9a37fff640 1 Processor -- start 2023-12-19T18:29:10.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.902+0000 7f9a37fff640 1 -- start start 2023-12-19T18:29:10.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.903+0000 7f9a37fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a20006f10 0x7f9a380b1600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.903+0000 7f9a37fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a3815a0a0 0x7f9a3815a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.903+0000 7f9a37fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a38148c00 0x7f9a38149080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.903+0000 7f9a37fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a380b2ab0 con 0x7f9a20006f10 2023-12-19T18:29:10.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.903+0000 7f9a37fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a3815aa60 con 0x7f9a3815a0a0 2023-12-19T18:29:10.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.903+0000 7f9a37fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a380b1e30 con 0x7f9a38148c00 2023-12-19T18:29:10.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.903+0000 7f9a36ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a20006f10 0x7f9a380b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:10.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.903+0000 7f9a36ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a20006f10 0x7f9a380b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39716/0 (socket says 172.21.15.27:39716) 2023-12-19T18:29:10.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.903+0000 7f9a36ffd640 1 -- 172.21.15.27:0/3086220472 learned_addr learned my addr 172.21.15.27:0/3086220472 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:10.904 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.903+0000 7f9a367fc640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a3815a0a0 0x7f9a3815a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:10.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.904+0000 7f9a377fe640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a38148c00 0x7f9a38149080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:10.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.904+0000 7f9a36ffd640 1 -- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a38148c00 msgr2=0x7f9a38149080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:10.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.904+0000 7f9a36ffd640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a38148c00 0x7f9a38149080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.904+0000 7f9a36ffd640 1 -- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a3815a0a0 msgr2=0x7f9a3815a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:10.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.904+0000 7f9a36ffd640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a3815a0a0 0x7f9a3815a520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:10.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.904+0000 7f9a36ffd640 1 -- 172.21.15.27:0/3086220472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a380b2110 con 0x7f9a20006f10 2023-12-19T18:29:10.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.904+0000 7f9a377fe640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a38148c00 0x7f9a38149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:10.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.904+0000 7f9a36ffd640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a20006f10 0x7f9a380b1600 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f9a280079f0 tx=0x7f9a2800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:10.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.904+0000 7f9a1f7fe640 1 -- 172.21.15.27:0/3086220472 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a2800ee50 con 0x7f9a20006f10 2023-12-19T18:29:10.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.904+0000 7f9a1f7fe640 1 -- 172.21.15.27:0/3086220472 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a28013420 con 0x7f9a20006f10 2023-12-19T18:29:10.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.904+0000 7f9a1f7fe640 1 -- 172.21.15.27:0/3086220472 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a280137d0 con 0x7f9a20006f10 2023-12-19T18:29:10.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.904+0000 7f9a37fff640 1 -- 172.21.15.27:0/3086220472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a380b23f0 con 0x7f9a20006f10 2023-12-19T18:29:10.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.905+0000 7f9a37fff640 1 -- 172.21.15.27:0/3086220472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a380b1b40 con 0x7f9a20006f10 2023-12-19T18:29:10.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.906+0000 7f9a37fff640 1 -- 172.21.15.27:0/3086220472 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3804e840 con 0x7f9a20006f10 2023-12-19T18:29:10.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.907+0000 7f9a1f7fe640 1 -- 172.21.15.27:0/3086220472 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9a28013930 con 0x7f9a20006f10 2023-12-19T18:29:10.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.908+0000 7f9a1f7fe640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a2406a430 0x7f9a2406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:10.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.908+0000 7f9a1f7fe640 1 -- 172.21.15.27:0/3086220472 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9a28088e40 con 0x7f9a20006f10 2023-12-19T18:29:10.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.908+0000 7f9a367fc640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a2406a430 0x7f9a2406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:10.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.909+0000 7f9a367fc640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a2406a430 0x7f9a2406c8f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9a380cbfc0 tx=0x7f9a2c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:10.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:10.912+0000 7f9a1f7fe640 1 -- 172.21.15.27:0/3086220472 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9a28054710 con 0x7f9a20006f10 2023-12-19T18:29:11.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.102+0000 7f9a37fff640 1 -- 172.21.15.27:0/3086220472 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a380cb860 con 0x7f9a2406a430 2023-12-19T18:29:11.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.109+0000 7f9a1f7fe640 1 -- 172.21.15.27:0/3086220472 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47286 (secure 0 0 0) 0x7f9a2c004030 con 0x7f9a2406a430 2023-12-19T18:29:11.109 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:29:11.109 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:29:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.111+0000 7f9a37fff640 1 -- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a2406a430 msgr2=0x7f9a2406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.111+0000 7f9a37fff640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a2406a430 0x7f9a2406c8f0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f9a380cbfc0 tx=0x7f9a2c008040 comp rx=0 tx=0).stop 2023-12-19T18:29:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.111+0000 7f9a37fff640 1 -- 172.21.15.27:0/3086220472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a20006f10 msgr2=0x7f9a380b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:11.111 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.111+0000 7f9a37fff640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a20006f10 0x7f9a380b1600 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f9a280079f0 tx=0x7f9a2800e530 comp rx=0 tx=0).stop 2023-12-19T18:29:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.111+0000 7f9a37fff640 1 -- 172.21.15.27:0/3086220472 shutdown_connections 2023-12-19T18:29:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.111+0000 7f9a37fff640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9a2406a430 0x7f9a2406c8f0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.111+0000 7f9a37fff640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9a38148c00 0x7f9a38149080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.111+0000 7f9a37fff640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9a3815a0a0 0x7f9a3815a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.111+0000 7f9a37fff640 1 --2- 172.21.15.27:0/3086220472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a20006f10 0x7f9a380b1600 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.111+0000 7f9a37fff640 1 -- 172.21.15.27:0/3086220472 >> 172.21.15.27:0/3086220472 conn(0x7f9a38069c70 msgr2=0x7f9a3806cc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.112+0000 7f9a37fff640 1 -- 172.21.15.27:0/3086220472 shutdown_connections 2023-12-19T18:29:11.112 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.112+0000 7f9a37fff640 1 -- 172.21.15.27:0/3086220472 wait complete. 2023-12-19T18:29:11.131 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":635,"stamp":"2023-12-19T18:29:10.021060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10484,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552015,"statfs":{"total":1600000000,"available":1589264040,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52485472,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-128,"num_write_kb":-85084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.382243"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":336,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-19T18:29:05.956617+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:29:05.956617+0000","last_peered":"2023-12-19T18:29:05.956617+0000","last_clean":"2023-12-19T18:29:05.956617+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:29:05.956617+0000","last_undegraded":"2023-12-19T18:29:05.956617+0000","last_fullsized":"2023-12-19T18:29:05.956617+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":0.67000000000000004,"5min":0.59299999999999997,"15min":0.45900000000000002},"max":{"1min":1.5369999999999999,"5min":1.756,"15min":5.6879999999999997},"last":1.54},{"interface":"front","average":{"1min":1.274,"5min":1.236,"15min":1.228},"min":{"1min":0.78600000000000003,"5min":0.71899999999999997,"15min":0.55800000000000005},"max":{"1min":1.6599999999999999,"5min":1.7030000000000001,"15min":1.79},"last":1.5009999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.179,"15min":1.23},"min":{"1min":0.75600000000000001,"5min":0.65800000000000003,"15min":0.54400000000000004},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":10.926},"last":1.264},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.26,"15min":1.268},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.67900000000000005},"max":{"1min":1.534,"5min":1.736,"15min":10.904999999999999},"last":1.4350000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1379999999999999,"15min":1.117},"min":{"1min":0.90700000000000003,"5min":0.66000000000000003,"15min":0.61499999999999999},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":3.1960000000000002},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1830000000000001,"15min":1.2},"min":{"1min":0.91500000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5469999999999999,"5min":1.6799999999999999,"15min":6.008},"last":1.371}]},{"osd":3,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.2749999999999999,"15min":1.2290000000000001},"min":{"1min":0.96899999999999997,"5min":0.79000000000000004,"15min":0.61399999999999999},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":3.0379999999999998},"last":1.468},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3420000000000001,"15min":1.3400000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69599999999999995},"max":{"1min":1.6220000000000001,"5min":1.772,"15min":5.8120000000000003},"last":1.6459999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.202,"15min":1.1850000000000001},"min":{"1min":0.81299999999999994,"5min":0.61099999999999999,"15min":0.47899999999999998},"max":{"1min":1.8340000000000001,"5min":1.847,"15min":4.0110000000000001},"last":1.147},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.256,"15min":1.256},"min":{"1min":0.94699999999999995,"5min":0.624,"15min":0.57599999999999996},"max":{"1min":1.579,"5min":1.8,"15min":3.3180000000000001},"last":1.347}]},{"osd":5,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3049999999999999,"15min":1.3380000000000001},"min":{"1min":0.995,"5min":0.68999999999999995,"15min":0.53800000000000003},"max":{"1min":1.7789999999999999,"5min":1.8140000000000001,"15min":7.6429999999999998},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3620000000000001,"15min":1.3660000000000001},"min":{"1min":0.96899999999999997,"5min":0.627,"15min":0.53500000000000003},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":6.0839999999999996},"last":1.3169999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.5129999999999999,"15min":1.506},"min":{"1min":1.175,"5min":0.70899999999999996,"15min":0.59899999999999998},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":3.1309999999999998},"last":1.587},{"interface":"front","average":{"1min":1.522,"5min":1.4690000000000001,"15min":1.4330000000000001},"min":{"1min":1.131,"5min":0.73199999999999998,"15min":0.56100000000000005},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":4.1479999999999997},"last":1.2050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575023,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.099,"15min":1.0680000000000001},"min":{"1min":0.68300000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.413,"5min":1.522,"15min":1.8129999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.151,"15min":1.1220000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.55600000000000005},"max":{"1min":1.587,"5min":1.6000000000000001,"15min":3.5529999999999999},"last":1.2529999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1220000000000001,"15min":1.131},"min":{"1min":0.75700000000000001,"5min":0.68600000000000005,"15min":0.68200000000000005},"max":{"1min":1.573,"5min":1.7909999999999999,"15min":4.5540000000000003},"last":1.419},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.177,"15min":1.179},"min":{"1min":0.80100000000000005,"5min":0.64600000000000002,"15min":0.60299999999999998},"max":{"1min":1.5569999999999999,"5min":1.5960000000000001,"15min":1.885},"last":1.0469999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.258,"15min":1.21},"min":{"1min":0.89900000000000002,"5min":0.78900000000000003,"15min":0.626},"max":{"1min":1.5569999999999999,"5min":1.6479999999999999,"15min":1.821},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2190000000000001,"15min":1.216},"min":{"1min":0.85899999999999999,"5min":0.62,"15min":0.60899999999999999},"max":{"1min":1.514,"5min":1.7669999999999999,"15min":3.7909999999999999},"last":0.98299999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3129999999999999,"15min":1.2569999999999999},"min":{"1min":0.90800000000000003,"5min":0.71799999999999997,"15min":0.67800000000000005},"max":{"1min":1.5469999999999999,"5min":1.804,"15min":3.8820000000000001},"last":1.179},{"interface":"front","average":{"1min":1.258,"5min":1.248,"15min":1.2050000000000001},"min":{"1min":0.96099999999999997,"5min":0.71199999999999997,"15min":0.69899999999999995},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":4.6189999999999998},"last":1.141}]},{"osd":4,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.306,"15min":1.2809999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.65600000000000003},"max":{"1min":1.6259999999999999,"5min":1.7529999999999999,"15min":3.0680000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.415,"15min":1.3939999999999999},"min":{"1min":1.0529999999999999,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.7090000000000001,"5min":2.1040000000000001,"15min":3.7000000000000002},"last":0.86499999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2809999999999999,"15min":1.2470000000000001},"min":{"1min":0.91500000000000004,"5min":0.84899999999999998,"15min":0.65500000000000003},"max":{"1min":1.7929999999999999,"5min":1.9510000000000001,"15min":4.6680000000000001},"last":1.284},{"interface":"front","average":{"1min":1.375,"5min":1.4079999999999999,"15min":1.3959999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.626},"max":{"1min":1.609,"5min":1.794,"15min":4.1159999999999997},"last":1.4650000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.444,"15min":1.403},"min":{"1min":1.0149999999999999,"5min":0.65700000000000003,"15min":0.61899999999999999},"max":{"1min":1.663,"5min":2.02,"15min":3.9670000000000001},"last":1.089},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4470000000000001,"15min":1.413},"min":{"1min":1.0309999999999999,"5min":0.999,"15min":0.79300000000000004},"max":{"1min":1.855,"5min":2.2149999999999999,"15min":5.6639999999999997},"last":1.4930000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575023,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1742,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193569,"statfs":{"total":200000000,"available":198215229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.282,"15min":1.2150000000000001},"min":{"1min":0.76900000000000002,"5min":0.69999999999999996,"15min":0.61899999999999999},"max":{"1min":1.6830000000000001,"5min":2.089,"15min":2.089},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.304,"15min":1.2509999999999999},"min":{"1min":0.69399999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6020000000000001,"5min":1.641,"15min":1.913},"last":1.1299999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.167,"15min":1.1399999999999999},"min":{"1min":0.91600000000000004,"5min":0.71999999999999997,"15min":0.65700000000000003},"max":{"1min":1.571,"5min":1.851,"15min":1.9490000000000001},"last":1.532},{"interface":"front","average":{"1min":1.202,"5min":1.1890000000000001,"15min":1.141},"min":{"1min":0.89400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.452,"5min":1.899,"15min":1.899},"last":1.1739999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2050000000000001,"15min":1.149},"min":{"1min":0.73999999999999999,"5min":0.72199999999999998,"15min":0.59499999999999997},"max":{"1min":1.468,"5min":1.806,"15min":1.806},"last":1.083},{"interface":"front","average":{"1min":1.343,"5min":1.282,"15min":1.2230000000000001},"min":{"1min":0.92500000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.651,"5min":1.8,"15min":2.036},"last":1.4410000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.198,"15min":1.1599999999999999},"min":{"1min":0.93200000000000005,"5min":0.76600000000000001,"15min":0.68899999999999995},"max":{"1min":1.425,"5min":1.635,"15min":1.7450000000000001},"last":1.262},{"interface":"front","average":{"1min":1.411,"5min":1.3540000000000001,"15min":1.3080000000000001},"min":{"1min":0.99199999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.843,"5min":2.0369999999999999,"15min":2.0590000000000002},"last":1.3520000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.288,"15min":1.2629999999999999},"min":{"1min":1.024,"5min":0.52300000000000002,"15min":0.46700000000000003},"max":{"1min":1.5589999999999999,"5min":1.7250000000000001,"15min":2.0329999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.1379999999999999,"15min":1.159},"min":{"1min":0.86899999999999999,"5min":0.64500000000000002,"15min":0.57099999999999995},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.901},"last":1.1399999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.425,"15min":1.411},"min":{"1min":1.1850000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.952,"5min":2.1579999999999999,"15min":2.1589999999999998},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.2969999999999999,"15min":1.2889999999999999},"min":{"1min":0.94999999999999996,"5min":0.58299999999999996,"15min":0.49299999999999999},"max":{"1min":1.5720000000000001,"5min":1.704,"15min":1.976},"last":1.3160000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.2769999999999999,"15min":1.2729999999999999},"min":{"1min":0.83599999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4419999999999999,"15min":1.4430000000000001},"min":{"1min":1.1759999999999999,"5min":0.67100000000000004,"15min":0.623},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":2.117},"last":1.3799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575023,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.069,"15min":1.024},"min":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.52700000000000002},"max":{"1min":1.403,"5min":1.972,"15min":1.972},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.1180000000000001,"15min":1.073},"min":{"1min":0.88300000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5309999999999999,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.0780000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.204,"15min":1.1679999999999999},"min":{"1min":0.85799999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.4199999999999999,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.288,"15min":1.2350000000000001},"min":{"1min":1.056,"5min":0.64700000000000002,"15min":0.61699999999999999},"max":{"1min":1.6459999999999999,"5min":1.7450000000000001,"15min":1.875},"last":1.115}]},{"osd":2,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.1559999999999999,"15min":1.135},"min":{"1min":0.77200000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.597,"5min":1.744,"15min":1.744},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1839999999999999,"15min":1.1459999999999999},"min":{"1min":0.93000000000000005,"5min":0.747,"15min":0.70499999999999996},"max":{"1min":1.8149999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.93500000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2509999999999999,"15min":1.268},"min":{"1min":1.0549999999999999,"5min":0.754,"15min":0.754},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.946},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.369,"15min":1.335},"min":{"1min":1.123,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.6619999999999999,"5min":1.927,"15min":1.927},"last":1.4019999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2569999999999999,"15min":1.248},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.44500000000000001},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.357,"15min":1.369},"min":{"1min":0.82599999999999996,"5min":0.59099999999999997,"15min":0.504},"max":{"1min":1.7729999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.3220000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.2969999999999999,"15min":1.304},"min":{"1min":0.88500000000000001,"5min":0.68100000000000005,"15min":0.432},"max":{"1min":1.7490000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4219999999999999,"15min":1.421},"min":{"1min":1.1599999999999999,"5min":0.93700000000000006,"15min":0.64500000000000002},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.4399999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.3560000000000001,"15min":1.363},"min":{"1min":0.996,"5min":0.77300000000000002,"15min":0.56399999999999995},"max":{"1min":2.702,"5min":2.702,"15min":2.702},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4259999999999999,"15min":1.421},"min":{"1min":1.115,"5min":0.78600000000000003,"15min":0.57399999999999995},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.5069999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575024,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.90900000000000003,"15min":0.92200000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35299999999999998},"max":{"1min":1.754,"5min":1.754,"15min":3.4350000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.038,"15min":0.99299999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.39500000000000002},"max":{"1min":1.591,"5min":1.8109999999999999,"15min":4.3490000000000002},"last":0.61699999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.01,"15min":1.024},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.41299999999999998},"max":{"1min":1.8520000000000001,"5min":1.9690000000000001,"15min":5.2510000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.4299999999999999,"5min":1.643,"15min":5.4119999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1899999999999999,"15min":1.236},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.46000000000000002},"max":{"1min":2.1059999999999999,"5min":2.4649999999999999,"15min":13.833},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.304,"15min":1.3520000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8460000000000001,"5min":2.379,"15min":16.748999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3080000000000001,"15min":1.29},"min":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":0.65600000000000003},"max":{"1min":1.75,"5min":2.0150000000000001,"15min":5.1790000000000003},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.343,"5min":1.367,"15min":1.3919999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.73199999999999998},"max":{"1min":1.929,"5min":2.0819999999999999,"15min":4.569},"last":0.89000000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.4339999999999999,"15min":1.429},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.76000000000000001},"max":{"1min":1.925,"5min":2.0449999999999999,"15min":5.3289999999999997},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.327,"5min":1.3859999999999999,"15min":1.347},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.69999999999999996},"max":{"1min":2.004,"5min":2.1469999999999998,"15min":4.6369999999999996},"last":0.95699999999999996}]},{"osd":6,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.454,"15min":1.4510000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.81599999999999995},"max":{"1min":1.9790000000000001,"5min":2.2040000000000002,"15min":4.5039999999999996},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.516,"5min":1.5189999999999999,"15min":1.5089999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.80500000000000005},"max":{"1min":2.254,"5min":2.254,"15min":5.3739999999999997},"last":1.1950000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.4990000000000001,"15min":1.454},"min":{"1min":0.98899999999999999,"5min":0.89700000000000002,"15min":0.73199999999999998},"max":{"1min":2.1389999999999998,"5min":2.2869999999999999,"15min":5.1349999999999998},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.5369999999999999,"15min":1.5489999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.0550000000000002,"5min":2.351,"15min":4.4619999999999997},"last":0.84199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.92600000000000005,"15min":0.97499999999999998},"min":{"1min":0.48099999999999998,"5min":0.43099999999999999,"15min":0.34399999999999997},"max":{"1min":1.5309999999999999,"5min":2.2799999999999998,"15min":9.2550000000000008},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.04,"15min":1.0189999999999999},"min":{"1min":0.45900000000000002,"5min":0.38400000000000001,"15min":0.33900000000000002},"max":{"1min":1.4399999999999999,"5min":1.8049999999999999,"15min":4.7850000000000001},"last":1.4179999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.93999999999999995,"15min":0.98999999999999999},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.4039999999999999,"5min":2.0169999999999999,"15min":8.2720000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.101,"15min":1.1040000000000001},"min":{"1min":0.61199999999999999,"5min":0.54200000000000004,"15min":0.46600000000000003},"max":{"1min":1.339,"5min":1.8460000000000001,"15min":5.0620000000000003},"last":1.339}]},{"osd":3,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.2669999999999999,"15min":1.226},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5940000000000001,"5min":2.8159999999999998,"15min":8.3109999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1160000000000001,"15min":1.163},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.41799999999999998},"max":{"1min":1.7869999999999999,"5min":2.6240000000000001,"15min":8.0690000000000008},"last":1.089}]},{"osd":4,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3819999999999999,"15min":1.395},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9850000000000001,"5min":2.0059999999999998,"15min":5.3929999999999998},"last":1.446},{"interface":"front","average":{"1min":1.399,"5min":1.458,"15min":1.46},"min":{"1min":1.0209999999999999,"5min":0.85799999999999998,"15min":0.78300000000000003},"max":{"1min":2.3079999999999998,"5min":2.4300000000000002,"15min":7.3570000000000002},"last":1.649}]},{"osd":5,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3720000000000001,"15min":1.4059999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.613,"5min":2.1680000000000001,"15min":8.1839999999999993},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.385,"15min":1.391},"min":{"1min":0.90000000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":5.165},"last":1.377}]},{"osd":6,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.413,"15min":1.4219999999999999},"min":{"1min":0.99399999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.96,"5min":2.4630000000000001,"15min":5.1059999999999999},"last":1.96},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.468,"15min":1.48},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77000000000000002},"max":{"1min":1.7709999999999999,"5min":2.7559999999999998,"15min":8.1370000000000005},"last":1.7709999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.504,"15min":1.5089999999999999},"min":{"1min":0.91800000000000004,"5min":0.80000000000000004,"15min":0.78200000000000003},"max":{"1min":2.0870000000000002,"5min":2.3959999999999999,"15min":5.1449999999999996},"last":1.821},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5629999999999999,"15min":1.5680000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86499999999999999},"max":{"1min":1.8720000000000001,"5min":2.6320000000000001,"15min":8.2300000000000004},"last":1.8720000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575024,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1742,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193569,"statfs":{"total":200000000,"available":198215229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93200000000000005,"15min":0.874},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.34699999999999998},"max":{"1min":1.472,"5min":2.262,"15min":2.262},"last":1.472},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.014,"15min":0.96299999999999997},"min":{"1min":0.60599999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4970000000000001,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.1220000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.02,"15min":1.0309999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.41399999999999998},"max":{"1min":1.621,"5min":1.621,"15min":2.5739999999999998},"last":1.621},{"interface":"front","average":{"1min":0.92900000000000005,"5min":1.0069999999999999,"15min":0.97799999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.38500000000000001},"max":{"1min":1.276,"5min":1.718,"15min":1.845},"last":1.2190000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.3300000000000001,"15min":1.2569999999999999},"min":{"1min":0.624,"5min":0.53000000000000003,"15min":0.48599999999999999},"max":{"1min":1.603,"5min":2.089,"15min":2.137},"last":1.571},{"interface":"front","average":{"1min":1.099,"5min":1.2350000000000001,"15min":1.198},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.44900000000000001},"max":{"1min":1.6659999999999999,"5min":2.0720000000000001,"15min":2.3959999999999999},"last":1.425}]},{"osd":4,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3140000000000001,"15min":1.3120000000000001},"min":{"1min":0.83099999999999996,"5min":0.80500000000000005,"15min":0.69099999999999995},"max":{"1min":1.694,"5min":2.1859999999999999,"15min":2.444},"last":1.694},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.381,"15min":1.3520000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.71099999999999997},"max":{"1min":1.665,"5min":1.8859999999999999,"15min":2.6440000000000001},"last":1.351}]},{"osd":5,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.425,"15min":1.371},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":2.5459999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.431,"15min":1.4330000000000001},"min":{"1min":1.0249999999999999,"5min":0.85899999999999999,"15min":0.70499999999999996},"max":{"1min":1.8240000000000001,"5min":1.9339999999999999,"15min":2.6179999999999999},"last":1.806}]},{"osd":6,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.466,"15min":1.4139999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":2.1619999999999999,"5min":2.274,"15min":2.7000000000000002},"last":1.534},{"interface":"front","average":{"1min":1.379,"5min":1.5,"15min":1.478},"min":{"1min":0.872,"5min":0.872,"15min":0.76400000000000001},"max":{"1min":2.153,"5min":2.153,"15min":2.895},"last":1.748}]},{"osd":7,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4490000000000001,"15min":1.4470000000000001},"min":{"1min":0.92600000000000005,"5min":0.83199999999999996,"15min":0.69099999999999995},"max":{"1min":2.044,"5min":2.044,"15min":9.4830000000000005},"last":1.288},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.5600000000000001,"15min":1.546},"min":{"1min":0.93899999999999995,"5min":0.80800000000000005,"15min":0.71899999999999997},"max":{"1min":2.198,"5min":2.3420000000000001,"15min":9.4109999999999996},"last":1.657}]}]},{"osd":0,"up_from":13,"seq":55834575023,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92200000000000004,"15min":0.92100000000000004},"min":{"1min":0.60699999999999998,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":1.425,"5min":1.671,"15min":2.3199999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1100000000000001,"15min":1.0720000000000001},"min":{"1min":0.58899999999999997,"5min":0.48699999999999999,"15min":0.42799999999999999},"max":{"1min":1.3480000000000001,"5min":2.2170000000000001,"15min":2.4180000000000001},"last":0.54500000000000004}]},{"osd":2,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0289999999999999,"15min":0.996},"min":{"1min":0.69399999999999995,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":1.4550000000000001,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.0609999999999999,"15min":0.996},"min":{"1min":0.64000000000000001,"5min":0.42699999999999999,"15min":0.41099999999999998},"max":{"1min":1.6870000000000001,"5min":2.3340000000000001,"15min":2.7810000000000001},"last":0.68799999999999994}]},{"osd":3,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.032,"15min":1.0249999999999999},"min":{"1min":0.63300000000000001,"5min":0.44,"15min":0.38900000000000001},"max":{"1min":1.5589999999999999,"5min":1.8200000000000001,"15min":2.5190000000000001},"last":0.747},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.349,"15min":1.2769999999999999},"min":{"1min":0.81299999999999994,"5min":0.65100000000000002,"15min":0.47499999999999998},"max":{"1min":1.6759999999999999,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.72599999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.323,"15min":1.2749999999999999},"min":{"1min":0.94499999999999995,"5min":0.67600000000000005,"15min":0.63200000000000001},"max":{"1min":1.7290000000000001,"5min":2.1920000000000002,"15min":2.5529999999999999},"last":1.002},{"interface":"front","average":{"1min":1.27,"5min":1.2789999999999999,"15min":1.28},"min":{"1min":0.81899999999999995,"5min":0.70399999999999996,"15min":0.67600000000000005},"max":{"1min":1.6699999999999999,"5min":2.056,"15min":2.9529999999999998},"last":0.85099999999999998}]},{"osd":5,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3460000000000001,"15min":1.3919999999999999},"min":{"1min":0.92800000000000005,"5min":0.70699999999999996,"15min":0.64500000000000002},"max":{"1min":1.542,"5min":1.97,"15min":16.298999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.387,"15min":1.3360000000000001},"min":{"1min":1.115,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.641,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.89500000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.4470000000000001},"min":{"1min":0.93300000000000005,"5min":0.79800000000000004,"15min":0.71699999999999997},"max":{"1min":1.744,"5min":2.4169999999999998,"15min":2.5910000000000002},"last":1.089},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4810000000000001,"15min":1.4179999999999999},"min":{"1min":1.097,"5min":0.745,"15min":0.745},"max":{"1min":1.8,"5min":2.1709999999999998,"15min":2.46},"last":1.1240000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4910000000000001,"15min":1.454},"min":{"1min":1.052,"5min":0.72099999999999997,"15min":0.70899999999999996},"max":{"1min":1.895,"5min":2.2549999999999999,"15min":2.4969999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.407,"15min":1.415},"min":{"1min":1.077,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6439999999999999,"5min":1.9370000000000001,"15min":3.1429999999999998},"last":1.179}]}]}],"pool_statfs":[]}} 2023-12-19T18:29:11.132 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-19T18:29:11.133 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:29:11.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.358+0000 7fb0119fa640 1 Processor -- start 2023-12-19T18:29:11.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.358+0000 7fb0119fa640 1 -- start start 2023-12-19T18:29:11.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.360+0000 7fb0119fa640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb00c0c8f90 0x7fb00c160900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.360+0000 7fb0119fa640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fb00c160e40 0x7fb00c0cd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.360+0000 7fb0119fa640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb00c1612b0 con 0x7fb00c0cb300 2023-12-19T18:29:11.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.360+0000 7fb0119fa640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00c0c4480 con 0x7fb00c0c8f90 2023-12-19T18:29:11.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.360+0000 7fb0119fa640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb00c1468c0 con 0x7fb00c160e40 2023-12-19T18:29:11.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.360+0000 7fb00bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb00c0c8f90 0x7fb00c160900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.361+0000 7fb00bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fb00c0c8f90 0x7fb00c160900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39728/0 (socket says 172.21.15.27:39728) 2023-12-19T18:29:11.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.361+0000 7fb00bfff640 1 -- 172.21.15.27:0/2207046977 learned_addr learned my addr 172.21.15.27:0/2207046977 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:11.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.361+0000 7fb0111f9640 1 --2- 172.21.15.27:0/2207046977 >> v2:172.21.15.192:3300/0 conn(0x7fb00c160e40 0x7fb00c0cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.361+0000 7fb00bfff640 1 -- 172.21.15.27:0/2207046977 >> v2:172.21.15.192:3300/0 conn(0x7fb00c160e40 msgr2=0x7fb00c0cd090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:11.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.361+0000 7fb00bfff640 1 --2- 172.21.15.27:0/2207046977 >> v2:172.21.15.192:3300/0 conn(0x7fb00c160e40 0x7fb00c0cd090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.361+0000 7fb00bfff640 1 -- 172.21.15.27:0/2207046977 >> v1:172.21.15.192:6789/0 conn(0x7fb00c0cb300 legacy=0x7fb00c0cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:11.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.361+0000 7fb00bfff640 1 -- 172.21.15.27:0/2207046977 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00c145d60 con 0x7fb00c0c8f90 2023-12-19T18:29:11.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.361+0000 7fb0111f9640 1 --2- 172.21.15.27:0/2207046977 >> v2:172.21.15.192:3300/0 conn(0x7fb00c160e40 0x7fb00c0cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:11.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.362+0000 7fb00bfff640 1 --2- 172.21.15.27:0/2207046977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c0c8f90 0x7fb00c160900 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c326a54d2fe96ec server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:11.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.363+0000 7fb00b7fe640 1 -- 172.21.15.27:0/2207046977 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8031870 con 0x7fb00c0c8f90 2023-12-19T18:29:11.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.363+0000 7fb00b7fe640 1 -- 172.21.15.27:0/2207046977 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff80319d0 con 0x7fb00c0c8f90 2023-12-19T18:29:11.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.363+0000 7fb00b7fe640 1 -- 172.21.15.27:0/2207046977 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff8031d60 con 0x7fb00c0c8f90 2023-12-19T18:29:11.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.363+0000 7fb0119fa640 1 -- 172.21.15.27:0/2207046977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c0c8f90 msgr2=0x7fb00c160900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:11.363 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.363+0000 7fb0119fa640 1 --2- 172.21.15.27:0/2207046977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c0c8f90 0x7fb00c160900 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7faff8009870 tx=0x7faff802b910 comp rx=0 tx=0).stop 2023-12-19T18:29:11.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.364+0000 7fb0119fa640 1 -- 172.21.15.27:0/2207046977 shutdown_connections 2023-12-19T18:29:11.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.364+0000 7fb0119fa640 1 --2- 172.21.15.27:0/2207046977 >> v2:172.21.15.192:3300/0 conn(0x7fb00c160e40 0x7fb00c0cd090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.364+0000 7fb0119fa640 1 --2- 172.21.15.27:0/2207046977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c0c8f90 0x7fb00c160900 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.364+0000 7fb0119fa640 1 -- 172.21.15.27:0/2207046977 >> 172.21.15.27:0/2207046977 conn(0x7fb00c069c70 msgr2=0x7fb00c06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:11.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.364+0000 7fb0119fa640 1 -- 172.21.15.27:0/2207046977 shutdown_connections 2023-12-19T18:29:11.364 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.364+0000 7fb0119fa640 1 -- 172.21.15.27:0/2207046977 wait complete. 2023-12-19T18:29:11.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.365+0000 7fb0119fa640 1 Processor -- start 2023-12-19T18:29:11.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.366+0000 7fb0119fa640 1 -- start start 2023-12-19T18:29:11.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.367+0000 7fb0119fa640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb00c0c8f90 0x7fb00c1f32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.367+0000 7fb0119fa640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb00c0cb300 0x7fb00c15e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.367+0000 7fb0119fa640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c160e40 0x7fb00c0b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.367+0000 7fb0119fa640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0aef60 con 0x7fb00c160e40 2023-12-19T18:29:11.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.367+0000 7fb0119fa640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c15ef10 con 0x7fb00c0cb300 2023-12-19T18:29:11.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.367+0000 7fb0119fa640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb00c0af200 con 0x7fb00c0c8f90 2023-12-19T18:29:11.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.367+0000 7fb0111f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c160e40 0x7fb00c0b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.367+0000 7fb0111f9640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c160e40 0x7fb00c0b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39740/0 (socket says 172.21.15.27:39740) 2023-12-19T18:29:11.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.367+0000 7fb0111f9640 1 -- 172.21.15.27:0/3689460856 learned_addr learned my addr 172.21.15.27:0/3689460856 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:11.368 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.367+0000 7fb00bfff640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb00c0cb300 0x7fb00c15e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.367+0000 7fb0109f8640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb00c0c8f90 0x7fb00c1f32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.367+0000 7fb0111f9640 1 -- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb00c0c8f90 msgr2=0x7fb00c1f32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:11.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.368+0000 7fb0111f9640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb00c0c8f90 0x7fb00c1f32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.368+0000 7fb0111f9640 1 -- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb00c0cb300 msgr2=0x7fb00c15e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:11.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.368+0000 7fb0111f9640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb00c0cb300 0x7fb00c15e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.368+0000 7fb0111f9640 1 -- 172.21.15.27:0/3689460856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff80094d0 con 0x7fb00c160e40 2023-12-19T18:29:11.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.368+0000 7fb00bfff640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb00c0cb300 0x7fb00c15e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:11.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.368+0000 7fb0109f8640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb00c0c8f90 0x7fb00c1f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:11.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.368+0000 7fb0111f9640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c160e40 0x7fb00c0b15c0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fb00400cef0 tx=0x7fb00400a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:11.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.368+0000 7fb0097fa640 1 -- 172.21.15.27:0/3689460856 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00400d840 con 0x7fb00c160e40 2023-12-19T18:29:11.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.368+0000 7fb0097fa640 1 -- 172.21.15.27:0/3689460856 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb00400d9c0 con 0x7fb00c160e40 2023-12-19T18:29:11.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.368+0000 7fb0097fa640 1 -- 172.21.15.27:0/3689460856 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb00400dd70 con 0x7fb00c160e40 2023-12-19T18:29:11.369 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.368+0000 7fb0119fa640 1 -- 172.21.15.27:0/3689460856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb00c0af4e0 con 0x7fb00c160e40 2023-12-19T18:29:11.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.368+0000 7fb0119fa640 1 -- 172.21.15.27:0/3689460856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb00c0afaa0 con 0x7fb00c160e40 2023-12-19T18:29:11.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.370+0000 7fb0119fa640 1 -- 172.21.15.27:0/3689460856 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00c0cc590 con 0x7fb00c160e40 2023-12-19T18:29:11.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.371+0000 7fb0097fa640 1 -- 172.21.15.27:0/3689460856 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fb00401e3b0 con 0x7fb00c160e40 2023-12-19T18:29:11.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.371+0000 7fb0097fa640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faff006a430 0x7faff006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.372+0000 7fb0097fa640 1 -- 172.21.15.27:0/3689460856 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fb004089050 con 0x7fb00c160e40 2023-12-19T18:29:11.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.372+0000 7fb0109f8640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faff006a430 0x7faff006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.373+0000 7fb0109f8640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faff006a430 0x7faff006c8f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0c8f20 tx=0x7fb000008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:11.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.376+0000 7fb0097fa640 1 -- 172.21.15.27:0/3689460856 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fb00c0cc590 con 0x7fb00c160e40 2023-12-19T18:29:11.566 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.566+0000 7fb0119fa640 1 -- 172.21.15.27:0/3689460856 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb00c161510 con 0x7faff006a430 2023-12-19T18:29:11.573 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.573+0000 7fb0097fa640 1 -- 172.21.15.27:0/3689460856 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47286 (secure 0 0 0) 0x7fb00000ad80 con 0x7faff006a430 2023-12-19T18:29:11.573 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:29:11.574 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:29:11.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.575+0000 7fb0119fa640 1 -- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faff006a430 msgr2=0x7faff006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:11.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.575+0000 7fb0119fa640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faff006a430 0x7faff006c8f0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb00c0c8f20 tx=0x7fb000008040 comp rx=0 tx=0).stop 2023-12-19T18:29:11.575 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.575+0000 7fb0119fa640 1 -- 172.21.15.27:0/3689460856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c160e40 msgr2=0x7fb00c0b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.575+0000 7fb0119fa640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c160e40 0x7fb00c0b15c0 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fb00400cef0 tx=0x7fb00400a710 comp rx=0 tx=0).stop 2023-12-19T18:29:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.576+0000 7fb0119fa640 1 -- 172.21.15.27:0/3689460856 shutdown_connections 2023-12-19T18:29:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.576+0000 7fb0119fa640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7faff006a430 0x7faff006c8f0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.576+0000 7fb0119fa640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb00c160e40 0x7fb00c0b15c0 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.576+0000 7fb0119fa640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fb00c0cb300 0x7fb00c15e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.576+0000 7fb0119fa640 1 --2- 172.21.15.27:0/3689460856 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fb00c0c8f90 0x7fb00c1f32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.576+0000 7fb0119fa640 1 -- 172.21.15.27:0/3689460856 >> 172.21.15.27:0/3689460856 conn(0x7fb00c069c70 msgr2=0x7fb00c06c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.576+0000 7fb0119fa640 1 -- 172.21.15.27:0/3689460856 shutdown_connections 2023-12-19T18:29:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.576+0000 7fb0119fa640 1 -- 172.21.15.27:0/3689460856 wait complete. 2023-12-19T18:29:11.595 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":635,"stamp":"2023-12-19T18:29:10.021060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":113,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10484,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552015,"statfs":{"total":1600000000,"available":1589264040,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-52485472,"num_objects":-36,"num_object_clones":0,"num_object_copies":-72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-36,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-128,"num_write_kb":-85084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.382243"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":336,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-19T18:29:05.956617+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:29:05.956617+0000","last_peered":"2023-12-19T18:29:05.956617+0000","last_clean":"2023-12-19T18:29:05.956617+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:29:05.956617+0000","last_undegraded":"2023-12-19T18:29:05.956617+0000","last_fullsized":"2023-12-19T18:29:05.956617+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":0.67000000000000004,"5min":0.59299999999999997,"15min":0.45900000000000002},"max":{"1min":1.5369999999999999,"5min":1.756,"15min":5.6879999999999997},"last":1.54},{"interface":"front","average":{"1min":1.274,"5min":1.236,"15min":1.228},"min":{"1min":0.78600000000000003,"5min":0.71899999999999997,"15min":0.55800000000000005},"max":{"1min":1.6599999999999999,"5min":1.7030000000000001,"15min":1.79},"last":1.5009999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.179,"15min":1.23},"min":{"1min":0.75600000000000001,"5min":0.65800000000000003,"15min":0.54400000000000004},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":10.926},"last":1.264},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.26,"15min":1.268},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.67900000000000005},"max":{"1min":1.534,"5min":1.736,"15min":10.904999999999999},"last":1.4350000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1379999999999999,"15min":1.117},"min":{"1min":0.90700000000000003,"5min":0.66000000000000003,"15min":0.61499999999999999},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":3.1960000000000002},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1830000000000001,"15min":1.2},"min":{"1min":0.91500000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5469999999999999,"5min":1.6799999999999999,"15min":6.008},"last":1.371}]},{"osd":3,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.2749999999999999,"15min":1.2290000000000001},"min":{"1min":0.96899999999999997,"5min":0.79000000000000004,"15min":0.61399999999999999},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":3.0379999999999998},"last":1.468},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3420000000000001,"15min":1.3400000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69599999999999995},"max":{"1min":1.6220000000000001,"5min":1.772,"15min":5.8120000000000003},"last":1.6459999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.202,"15min":1.1850000000000001},"min":{"1min":0.81299999999999994,"5min":0.61099999999999999,"15min":0.47899999999999998},"max":{"1min":1.8340000000000001,"5min":1.847,"15min":4.0110000000000001},"last":1.147},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.256,"15min":1.256},"min":{"1min":0.94699999999999995,"5min":0.624,"15min":0.57599999999999996},"max":{"1min":1.579,"5min":1.8,"15min":3.3180000000000001},"last":1.347}]},{"osd":5,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3049999999999999,"15min":1.3380000000000001},"min":{"1min":0.995,"5min":0.68999999999999995,"15min":0.53800000000000003},"max":{"1min":1.7789999999999999,"5min":1.8140000000000001,"15min":7.6429999999999998},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3620000000000001,"15min":1.3660000000000001},"min":{"1min":0.96899999999999997,"5min":0.627,"15min":0.53500000000000003},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":6.0839999999999996},"last":1.3169999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.5129999999999999,"15min":1.506},"min":{"1min":1.175,"5min":0.70899999999999996,"15min":0.59899999999999998},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":3.1309999999999998},"last":1.587},{"interface":"front","average":{"1min":1.522,"5min":1.4690000000000001,"15min":1.4330000000000001},"min":{"1min":1.131,"5min":0.73199999999999998,"15min":0.56100000000000005},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":4.1479999999999997},"last":1.2050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575023,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.0880000000000001,"5min":1.099,"15min":1.0680000000000001},"min":{"1min":0.68300000000000005,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.413,"5min":1.522,"15min":1.8129999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.147,"5min":1.151,"15min":1.1220000000000001},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.55600000000000005},"max":{"1min":1.587,"5min":1.6000000000000001,"15min":3.5529999999999999},"last":1.2529999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.1220000000000001,"15min":1.131},"min":{"1min":0.75700000000000001,"5min":0.68600000000000005,"15min":0.68200000000000005},"max":{"1min":1.573,"5min":1.7909999999999999,"15min":4.5540000000000003},"last":1.419},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.177,"15min":1.179},"min":{"1min":0.80100000000000005,"5min":0.64600000000000002,"15min":0.60299999999999998},"max":{"1min":1.5569999999999999,"5min":1.5960000000000001,"15min":1.885},"last":1.0469999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.3169999999999999,"5min":1.258,"15min":1.21},"min":{"1min":0.89900000000000002,"5min":0.78900000000000003,"15min":0.626},"max":{"1min":1.5569999999999999,"5min":1.6479999999999999,"15min":1.821},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.2070000000000001,"5min":1.2190000000000001,"15min":1.216},"min":{"1min":0.85899999999999999,"5min":0.62,"15min":0.60899999999999999},"max":{"1min":1.514,"5min":1.7669999999999999,"15min":3.7909999999999999},"last":0.98299999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.28,"5min":1.3129999999999999,"15min":1.2569999999999999},"min":{"1min":0.90800000000000003,"5min":0.71799999999999997,"15min":0.67800000000000005},"max":{"1min":1.5469999999999999,"5min":1.804,"15min":3.8820000000000001},"last":1.179},{"interface":"front","average":{"1min":1.258,"5min":1.248,"15min":1.2050000000000001},"min":{"1min":0.96099999999999997,"5min":0.71199999999999997,"15min":0.69899999999999995},"max":{"1min":1.6459999999999999,"5min":1.6459999999999999,"15min":4.6189999999999998},"last":1.141}]},{"osd":4,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.2649999999999999,"5min":1.306,"15min":1.2809999999999999},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.65600000000000003},"max":{"1min":1.6259999999999999,"5min":1.7529999999999999,"15min":3.0680000000000001},"last":1.5449999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.415,"15min":1.3939999999999999},"min":{"1min":1.0529999999999999,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.7090000000000001,"5min":2.1040000000000001,"15min":3.7000000000000002},"last":0.86499999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.27,"5min":1.2809999999999999,"15min":1.2470000000000001},"min":{"1min":0.91500000000000004,"5min":0.84899999999999998,"15min":0.65500000000000003},"max":{"1min":1.7929999999999999,"5min":1.9510000000000001,"15min":4.6680000000000001},"last":1.284},{"interface":"front","average":{"1min":1.375,"5min":1.4079999999999999,"15min":1.3959999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.626},"max":{"1min":1.609,"5min":1.794,"15min":4.1159999999999997},"last":1.4650000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:28:06 2023","interfaces":[{"interface":"back","average":{"1min":1.369,"5min":1.444,"15min":1.403},"min":{"1min":1.0149999999999999,"5min":0.65700000000000003,"15min":0.61899999999999999},"max":{"1min":1.663,"5min":2.02,"15min":3.9670000000000001},"last":1.089},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4470000000000001,"15min":1.413},"min":{"1min":1.0309999999999999,"5min":0.999,"15min":0.79300000000000004},"max":{"1min":1.855,"5min":2.2149999999999999,"15min":5.6639999999999997},"last":1.4930000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575023,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1742,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193569,"statfs":{"total":200000000,"available":198215229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1950000000000001,"5min":1.282,"15min":1.2150000000000001},"min":{"1min":0.76900000000000002,"5min":0.69999999999999996,"15min":0.61899999999999999},"max":{"1min":1.6830000000000001,"5min":2.089,"15min":2.089},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.298,"5min":1.304,"15min":1.2509999999999999},"min":{"1min":0.69399999999999995,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.6020000000000001,"5min":1.641,"15min":1.913},"last":1.1299999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.167,"15min":1.1399999999999999},"min":{"1min":0.91600000000000004,"5min":0.71999999999999997,"15min":0.65700000000000003},"max":{"1min":1.571,"5min":1.851,"15min":1.9490000000000001},"last":1.532},{"interface":"front","average":{"1min":1.202,"5min":1.1890000000000001,"15min":1.141},"min":{"1min":0.89400000000000002,"5min":0.505,"15min":0.505},"max":{"1min":1.452,"5min":1.899,"15min":1.899},"last":1.1739999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.161,"5min":1.2050000000000001,"15min":1.149},"min":{"1min":0.73999999999999999,"5min":0.72199999999999998,"15min":0.59499999999999997},"max":{"1min":1.468,"5min":1.806,"15min":1.806},"last":1.083},{"interface":"front","average":{"1min":1.343,"5min":1.282,"15min":1.2230000000000001},"min":{"1min":0.92500000000000004,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.651,"5min":1.8,"15min":2.036},"last":1.4410000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.1839999999999999,"5min":1.198,"15min":1.1599999999999999},"min":{"1min":0.93200000000000005,"5min":0.76600000000000001,"15min":0.68899999999999995},"max":{"1min":1.425,"5min":1.635,"15min":1.7450000000000001},"last":1.262},{"interface":"front","average":{"1min":1.411,"5min":1.3540000000000001,"15min":1.3080000000000001},"min":{"1min":0.99199999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":1.843,"5min":2.0369999999999999,"15min":2.0590000000000002},"last":1.3520000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.288,"15min":1.2629999999999999},"min":{"1min":1.024,"5min":0.52300000000000002,"15min":0.46700000000000003},"max":{"1min":1.5589999999999999,"5min":1.7250000000000001,"15min":2.0329999999999999},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.181,"5min":1.1379999999999999,"15min":1.159},"min":{"1min":0.86899999999999999,"5min":0.64500000000000002,"15min":0.57099999999999995},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.901},"last":1.1399999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.492,"5min":1.425,"15min":1.411},"min":{"1min":1.1850000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.952,"5min":2.1579999999999999,"15min":2.1589999999999998},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.2969999999999999,"15min":1.2889999999999999},"min":{"1min":0.94999999999999996,"5min":0.58299999999999996,"15min":0.49299999999999999},"max":{"1min":1.5720000000000001,"5min":1.704,"15min":1.976},"last":1.3160000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:28:09 2023","interfaces":[{"interface":"back","average":{"1min":1.3260000000000001,"5min":1.2769999999999999,"15min":1.2729999999999999},"min":{"1min":0.83599999999999997,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":2.036,"5min":2.036,"15min":2.036},"last":0.99399999999999999},{"interface":"front","average":{"1min":1.4650000000000001,"5min":1.4419999999999999,"15min":1.4430000000000001},"min":{"1min":1.1759999999999999,"5min":0.67100000000000004,"15min":0.623},"max":{"1min":1.8540000000000001,"5min":1.8540000000000001,"15min":2.117},"last":1.3799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575023,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.0860000000000001,"5min":1.069,"15min":1.024},"min":{"1min":0.66300000000000003,"5min":0.64700000000000002,"15min":0.52700000000000002},"max":{"1min":1.403,"5min":1.972,"15min":1.972},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.181,"5min":1.1180000000000001,"15min":1.073},"min":{"1min":0.88300000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.5309999999999999,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.0780000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.204,"15min":1.1679999999999999},"min":{"1min":0.85799999999999998,"5min":0.621,"15min":0.621},"max":{"1min":1.4199999999999999,"5min":1.8400000000000001,"15min":1.8400000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.3029999999999999,"5min":1.288,"15min":1.2350000000000001},"min":{"1min":1.056,"5min":0.64700000000000002,"15min":0.61699999999999999},"max":{"1min":1.6459999999999999,"5min":1.7450000000000001,"15min":1.875},"last":1.115}]},{"osd":2,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.1559999999999999,"15min":1.135},"min":{"1min":0.77200000000000002,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.597,"5min":1.744,"15min":1.744},"last":0.82699999999999996},{"interface":"front","average":{"1min":1.1819999999999999,"5min":1.1839999999999999,"15min":1.1459999999999999},"min":{"1min":0.93000000000000005,"5min":0.747,"15min":0.70499999999999996},"max":{"1min":1.8149999999999999,"5min":2.0270000000000001,"15min":2.0270000000000001},"last":0.93500000000000005}]},{"osd":3,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.2509999999999999,"15min":1.268},"min":{"1min":1.0549999999999999,"5min":0.754,"15min":0.754},"max":{"1min":1.6719999999999999,"5min":1.6719999999999999,"15min":1.946},"last":1.2490000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.369,"15min":1.335},"min":{"1min":1.123,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.6619999999999999,"5min":1.927,"15min":1.927},"last":1.4019999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2569999999999999,"15min":1.248},"min":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.44500000000000001},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":1.3580000000000001},{"interface":"front","average":{"1min":1.381,"5min":1.357,"15min":1.369},"min":{"1min":0.82599999999999996,"5min":0.59099999999999997,"15min":0.504},"max":{"1min":1.7729999999999999,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.3220000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.2969999999999999,"15min":1.304},"min":{"1min":0.88500000000000001,"5min":0.68100000000000005,"15min":0.432},"max":{"1min":1.7490000000000001,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.2669999999999999},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4219999999999999,"15min":1.421},"min":{"1min":1.1599999999999999,"5min":0.93700000000000006,"15min":0.64500000000000002},"max":{"1min":2.149,"5min":2.149,"15min":2.149},"last":1.4399999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:28:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4259999999999999,"5min":1.3560000000000001,"15min":1.363},"min":{"1min":0.996,"5min":0.77300000000000002,"15min":0.56399999999999995},"max":{"1min":2.702,"5min":2.702,"15min":2.702},"last":1.5469999999999999},{"interface":"front","average":{"1min":1.4970000000000001,"5min":1.4259999999999999,"15min":1.421},"min":{"1min":1.115,"5min":0.78600000000000003,"15min":0.57399999999999995},"max":{"1min":2.4199999999999999,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.5069999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575024,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.90900000000000003,"15min":0.92200000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35299999999999998},"max":{"1min":1.754,"5min":1.754,"15min":3.4350000000000001},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.038,"15min":0.99299999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.39500000000000002},"max":{"1min":1.591,"5min":1.8109999999999999,"15min":4.3490000000000002},"last":0.61699999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.01,"15min":1.024},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.41299999999999998},"max":{"1min":1.8520000000000001,"5min":1.9690000000000001,"15min":5.2510000000000003},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.4299999999999999,"5min":1.643,"15min":5.4119999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1899999999999999,"15min":1.236},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.46000000000000002},"max":{"1min":2.1059999999999999,"5min":2.4649999999999999,"15min":13.833},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.304,"15min":1.3520000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8460000000000001,"5min":2.379,"15min":16.748999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3080000000000001,"15min":1.29},"min":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":0.65600000000000003},"max":{"1min":1.75,"5min":2.0150000000000001,"15min":5.1790000000000003},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.343,"5min":1.367,"15min":1.3919999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.73199999999999998},"max":{"1min":1.929,"5min":2.0819999999999999,"15min":4.569},"last":0.89000000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.4339999999999999,"15min":1.429},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.76000000000000001},"max":{"1min":1.925,"5min":2.0449999999999999,"15min":5.3289999999999997},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.327,"5min":1.3859999999999999,"15min":1.347},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.69999999999999996},"max":{"1min":2.004,"5min":2.1469999999999998,"15min":4.6369999999999996},"last":0.95699999999999996}]},{"osd":6,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.454,"15min":1.4510000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.81599999999999995},"max":{"1min":1.9790000000000001,"5min":2.2040000000000002,"15min":4.5039999999999996},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.516,"5min":1.5189999999999999,"15min":1.5089999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.80500000000000005},"max":{"1min":2.254,"5min":2.254,"15min":5.3739999999999997},"last":1.1950000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.4990000000000001,"15min":1.454},"min":{"1min":0.98899999999999999,"5min":0.89700000000000002,"15min":0.73199999999999998},"max":{"1min":2.1389999999999998,"5min":2.2869999999999999,"15min":5.1349999999999998},"last":0.98899999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.5369999999999999,"15min":1.5489999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.0550000000000002,"5min":2.351,"15min":4.4619999999999997},"last":0.84199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575023,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.92600000000000005,"15min":0.97499999999999998},"min":{"1min":0.48099999999999998,"5min":0.43099999999999999,"15min":0.34399999999999997},"max":{"1min":1.5309999999999999,"5min":2.2799999999999998,"15min":9.2550000000000008},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.04,"15min":1.0189999999999999},"min":{"1min":0.45900000000000002,"5min":0.38400000000000001,"15min":0.33900000000000002},"max":{"1min":1.4399999999999999,"5min":1.8049999999999999,"15min":4.7850000000000001},"last":1.4179999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.93999999999999995,"15min":0.98999999999999999},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.4039999999999999,"5min":2.0169999999999999,"15min":8.2720000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.101,"15min":1.1040000000000001},"min":{"1min":0.61199999999999999,"5min":0.54200000000000004,"15min":0.46600000000000003},"max":{"1min":1.339,"5min":1.8460000000000001,"15min":5.0620000000000003},"last":1.339}]},{"osd":3,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.2669999999999999,"15min":1.226},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5940000000000001,"5min":2.8159999999999998,"15min":8.3109999999999999},"last":0.96399999999999997},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1160000000000001,"15min":1.163},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.41799999999999998},"max":{"1min":1.7869999999999999,"5min":2.6240000000000001,"15min":8.0690000000000008},"last":1.089}]},{"osd":4,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3819999999999999,"15min":1.395},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9850000000000001,"5min":2.0059999999999998,"15min":5.3929999999999998},"last":1.446},{"interface":"front","average":{"1min":1.399,"5min":1.458,"15min":1.46},"min":{"1min":1.0209999999999999,"5min":0.85799999999999998,"15min":0.78300000000000003},"max":{"1min":2.3079999999999998,"5min":2.4300000000000002,"15min":7.3570000000000002},"last":1.649}]},{"osd":5,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3720000000000001,"15min":1.4059999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.613,"5min":2.1680000000000001,"15min":8.1839999999999993},"last":1.5940000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.385,"15min":1.391},"min":{"1min":0.90000000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":5.165},"last":1.377}]},{"osd":6,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.413,"15min":1.4219999999999999},"min":{"1min":0.99399999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.96,"5min":2.4630000000000001,"15min":5.1059999999999999},"last":1.96},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.468,"15min":1.48},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77000000000000002},"max":{"1min":1.7709999999999999,"5min":2.7559999999999998,"15min":8.1370000000000005},"last":1.7709999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.504,"15min":1.5089999999999999},"min":{"1min":0.91800000000000004,"5min":0.80000000000000004,"15min":0.78200000000000003},"max":{"1min":2.0870000000000002,"5min":2.3959999999999999,"15min":5.1449999999999996},"last":1.821},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5629999999999999,"15min":1.5680000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86499999999999999},"max":{"1min":1.8720000000000001,"5min":2.6320000000000001,"15min":8.2300000000000004},"last":1.8720000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575024,"num_pgs":26,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1742,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193569,"statfs":{"total":200000000,"available":198215229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93200000000000005,"15min":0.874},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.34699999999999998},"max":{"1min":1.472,"5min":2.262,"15min":2.262},"last":1.472},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.014,"15min":0.96299999999999997},"min":{"1min":0.60599999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4970000000000001,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.1220000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.02,"15min":1.0309999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.41399999999999998},"max":{"1min":1.621,"5min":1.621,"15min":2.5739999999999998},"last":1.621},{"interface":"front","average":{"1min":0.92900000000000005,"5min":1.0069999999999999,"15min":0.97799999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.38500000000000001},"max":{"1min":1.276,"5min":1.718,"15min":1.845},"last":1.2190000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.3300000000000001,"15min":1.2569999999999999},"min":{"1min":0.624,"5min":0.53000000000000003,"15min":0.48599999999999999},"max":{"1min":1.603,"5min":2.089,"15min":2.137},"last":1.571},{"interface":"front","average":{"1min":1.099,"5min":1.2350000000000001,"15min":1.198},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.44900000000000001},"max":{"1min":1.6659999999999999,"5min":2.0720000000000001,"15min":2.3959999999999999},"last":1.425}]},{"osd":4,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3140000000000001,"15min":1.3120000000000001},"min":{"1min":0.83099999999999996,"5min":0.80500000000000005,"15min":0.69099999999999995},"max":{"1min":1.694,"5min":2.1859999999999999,"15min":2.444},"last":1.694},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.381,"15min":1.3520000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.71099999999999997},"max":{"1min":1.665,"5min":1.8859999999999999,"15min":2.6440000000000001},"last":1.351}]},{"osd":5,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.425,"15min":1.371},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":2.5459999999999998},"last":1.2529999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.431,"15min":1.4330000000000001},"min":{"1min":1.0249999999999999,"5min":0.85899999999999999,"15min":0.70499999999999996},"max":{"1min":1.8240000000000001,"5min":1.9339999999999999,"15min":2.6179999999999999},"last":1.806}]},{"osd":6,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.466,"15min":1.4139999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":2.1619999999999999,"5min":2.274,"15min":2.7000000000000002},"last":1.534},{"interface":"front","average":{"1min":1.379,"5min":1.5,"15min":1.478},"min":{"1min":0.872,"5min":0.872,"15min":0.76400000000000001},"max":{"1min":2.153,"5min":2.153,"15min":2.895},"last":1.748}]},{"osd":7,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4490000000000001,"15min":1.4470000000000001},"min":{"1min":0.92600000000000005,"5min":0.83199999999999996,"15min":0.69099999999999995},"max":{"1min":2.044,"5min":2.044,"15min":9.4830000000000005},"last":1.288},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.5600000000000001,"15min":1.546},"min":{"1min":0.93899999999999995,"5min":0.80800000000000005,"15min":0.71899999999999997},"max":{"1min":2.198,"5min":2.3420000000000001,"15min":9.4109999999999996},"last":1.657}]}]},{"osd":0,"up_from":13,"seq":55834575023,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92200000000000004,"15min":0.92100000000000004},"min":{"1min":0.60699999999999998,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":1.425,"5min":1.671,"15min":2.3199999999999998},"last":0.48199999999999998},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1100000000000001,"15min":1.0720000000000001},"min":{"1min":0.58899999999999997,"5min":0.48699999999999999,"15min":0.42799999999999999},"max":{"1min":1.3480000000000001,"5min":2.2170000000000001,"15min":2.4180000000000001},"last":0.54500000000000004}]},{"osd":2,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0289999999999999,"15min":0.996},"min":{"1min":0.69399999999999995,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":1.4550000000000001,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.113,"5min":1.0609999999999999,"15min":0.996},"min":{"1min":0.64000000000000001,"5min":0.42699999999999999,"15min":0.41099999999999998},"max":{"1min":1.6870000000000001,"5min":2.3340000000000001,"15min":2.7810000000000001},"last":0.68799999999999994}]},{"osd":3,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.032,"15min":1.0249999999999999},"min":{"1min":0.63300000000000001,"5min":0.44,"15min":0.38900000000000001},"max":{"1min":1.5589999999999999,"5min":1.8200000000000001,"15min":2.5190000000000001},"last":0.747},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.349,"15min":1.2769999999999999},"min":{"1min":0.81299999999999994,"5min":0.65100000000000002,"15min":0.47499999999999998},"max":{"1min":1.6759999999999999,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.72599999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.323,"15min":1.2749999999999999},"min":{"1min":0.94499999999999995,"5min":0.67600000000000005,"15min":0.63200000000000001},"max":{"1min":1.7290000000000001,"5min":2.1920000000000002,"15min":2.5529999999999999},"last":1.002},{"interface":"front","average":{"1min":1.27,"5min":1.2789999999999999,"15min":1.28},"min":{"1min":0.81899999999999995,"5min":0.70399999999999996,"15min":0.67600000000000005},"max":{"1min":1.6699999999999999,"5min":2.056,"15min":2.9529999999999998},"last":0.85099999999999998}]},{"osd":5,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3460000000000001,"15min":1.3919999999999999},"min":{"1min":0.92800000000000005,"5min":0.70699999999999996,"15min":0.64500000000000002},"max":{"1min":1.542,"5min":1.97,"15min":16.298999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.387,"15min":1.3360000000000001},"min":{"1min":1.115,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.641,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.89500000000000002}]},{"osd":6,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.4470000000000001},"min":{"1min":0.93300000000000005,"5min":0.79800000000000004,"15min":0.71699999999999997},"max":{"1min":1.744,"5min":2.4169999999999998,"15min":2.5910000000000002},"last":1.089},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4810000000000001,"15min":1.4179999999999999},"min":{"1min":1.097,"5min":0.745,"15min":0.745},"max":{"1min":1.8,"5min":2.1709999999999998,"15min":2.46},"last":1.1240000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4910000000000001,"15min":1.454},"min":{"1min":1.052,"5min":0.72099999999999997,"15min":0.70899999999999996},"max":{"1min":1.895,"5min":2.2549999999999999,"15min":2.4969999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.381,"5min":1.407,"15min":1.415},"min":{"1min":1.077,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6439999999999999,"5min":1.9370000000000001,"15min":3.1429999999999998},"last":1.179}]}]}],"pool_statfs":[]}} 2023-12-19T18:29:11.598 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-19T18:29:11.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.828+0000 7fa85759e640 1 Processor -- start 2023-12-19T18:29:11.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.828+0000 7fa85759e640 1 -- start start 2023-12-19T18:29:11.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.830+0000 7fa85759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8581553a0 0x7fa8580c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.830+0000 7fa85759e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fa8580afeb0 0x7fa85815e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.830+0000 7fa85759e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8580b0330 con 0x7fa85815ec60 2023-12-19T18:29:11.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.830+0000 7fa85759e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8580cbcf0 con 0x7fa8580afeb0 2023-12-19T18:29:11.832 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.830+0000 7fa85759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85815f0d0 con 0x7fa8581553a0 2023-12-19T18:29:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.830+0000 7fa856d9d640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fa85815ec60 0x7fa85815b130 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:53386/0 (socket says 172.21.15.27:53386) 2023-12-19T18:29:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.830+0000 7fa856d9d640 1 -- 172.21.15.27:0/1877931949 learned_addr learned my addr 172.21.15.27:0/1877931949 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.831+0000 7fa855d9b640 1 --2- 172.21.15.27:0/1877931949 >> v2:172.21.15.192:3300/0 conn(0x7fa8580afeb0 0x7fa85815e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.831+0000 7fa85659c640 1 --2- 172.21.15.27:0/1877931949 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8581553a0 0x7fa8580c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.831+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3248342921 0 0) 0x7fa8580b0330 con 0x7fa85815ec60 2023-12-19T18:29:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.831+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa840003520 con 0x7fa85815ec60 2023-12-19T18:29:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.831+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2256732916 0 0) 0x7fa840003520 con 0x7fa85815ec60 2023-12-19T18:29:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.831+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8581553a0 msgr2=0x7fa8580c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.831+0000 7fa85559a640 1 --2- 172.21.15.27:0/1877931949 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8581553a0 0x7fa8580c50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.831+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 >> v2:172.21.15.192:3300/0 conn(0x7fa8580afeb0 msgr2=0x7fa85815e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.831+0000 7fa85559a640 1 --2- 172.21.15.27:0/1877931949 >> v2:172.21.15.192:3300/0 conn(0x7fa8580afeb0 0x7fa85815e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.831+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8580cb350 con 0x7fa85815ec60 2023-12-19T18:29:11.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.831+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fa8500019f0 con 0x7fa85815ec60 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.831+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 >> v1:172.21.15.27:6789/0 conn(0x7fa85815ec60 legacy=0x7fa85815b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa85559a640 1 --2- 172.21.15.27:0/1877931949 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8581553a0 0x7fa8400039a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa85559a640 1 --2- 172.21.15.27:0/1877931949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa840003f70 0x7fa8400047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa85559a640 1 --2- 172.21.15.27:0/1877931949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa840004d00 0x7fa840005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580cbcf0 con 0x7fa840003f70 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85815f0d0 con 0x7fa840004d00 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84000ba60 con 0x7fa8581553a0 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa855d9b640 1 --2- 172.21.15.27:0/1877931949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa840003f70 0x7fa8400047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa856d9d640 1 --2- 172.21.15.27:0/1877931949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa840004d00 0x7fa840005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa85659c640 1 --2- 172.21.15.27:0/1877931949 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8581553a0 0x7fa8400039a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa855d9b640 1 -- 172.21.15.27:0/1877931949 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8581553a0 msgr2=0x7fa8400039a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa855d9b640 1 --2- 172.21.15.27:0/1877931949 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa8581553a0 0x7fa8400039a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa855d9b640 1 -- 172.21.15.27:0/1877931949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa840004d00 msgr2=0x7fa840005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:11.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa855d9b640 1 --2- 172.21.15.27:0/1877931949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa840004d00 0x7fa840005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa855d9b640 1 -- 172.21.15.27:0/1877931949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa840003520 con 0x7fa840003f70 2023-12-19T18:29:11.835 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.832+0000 7fa856d9d640 1 --2- 172.21.15.27:0/1877931949 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa840004d00 0x7fa840005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.833+0000 7fa855d9b640 1 --2- 172.21.15.27:0/1877931949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa840003f70 0x7fa8400047c0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009ac0 tx=0x7fa8400057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b343410f3bbe820d server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.834+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c02c710 con 0x7fa840003f70 2023-12-19T18:29:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.834+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa84c02c870 con 0x7fa840003f70 2023-12-19T18:29:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.834+0000 7fa85559a640 1 -- 172.21.15.27:0/1877931949 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa84c02cc00 con 0x7fa840003f70 2023-12-19T18:29:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.834+0000 7fa85759e640 1 -- 172.21.15.27:0/1877931949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa840003f70 msgr2=0x7fa8400047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.834+0000 7fa85759e640 1 --2- 172.21.15.27:0/1877931949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa840003f70 0x7fa8400047c0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7fa84c009ac0 tx=0x7fa8400057f0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.834+0000 7fa856d9d640 1 -- 172.21.15.27:0/1877931949 reap_dead start 2023-12-19T18:29:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.834+0000 7fa85759e640 1 -- 172.21.15.27:0/1877931949 shutdown_connections 2023-12-19T18:29:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.834+0000 7fa85759e640 1 -- 172.21.15.27:0/1877931949 >> 172.21.15.27:0/1877931949 conn(0x7fa858069c70 msgr2=0x7fa85806a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.835+0000 7fa85759e640 1 -- 172.21.15.27:0/1877931949 shutdown_connections 2023-12-19T18:29:11.836 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.835+0000 7fa85759e640 1 -- 172.21.15.27:0/1877931949 wait complete. 2023-12-19T18:29:11.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.836+0000 7fa85759e640 1 Processor -- start 2023-12-19T18:29:11.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.837+0000 7fa85759e640 1 -- start start 2023-12-19T18:29:11.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa85759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa840003f70 0x7fa85814ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa85759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa85815c4a0 0x7fa8580aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.838 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa85759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8581f1e00 0x7fa858151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa85759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa85814f650 con 0x7fa8581f1e00 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa85759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8580ccc90 con 0x7fa85815c4a0 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa85759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa85814f370 con 0x7fa840003f70 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa856d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8581f1e00 0x7fa858151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa856d9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8581f1e00 0x7fa858151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:39756/0 (socket says 172.21.15.27:39756) 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa856d9d640 1 -- 172.21.15.27:0/3974691099 learned_addr learned my addr 172.21.15.27:0/3974691099 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa85659c640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa840003f70 0x7fa85814ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa855d9b640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa85815c4a0 0x7fa8580aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa856d9d640 1 -- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa840003f70 msgr2=0x7fa85814ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa856d9d640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa840003f70 0x7fa85814ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa856d9d640 1 -- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa85815c4a0 msgr2=0x7fa8580aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.838+0000 7fa856d9d640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa85815c4a0 0x7fa8580aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.839+0000 7fa856d9d640 1 -- 172.21.15.27:0/3974691099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8580cc640 con 0x7fa8581f1e00 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.839+0000 7fa85659c640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa840003f70 0x7fa85814ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:11.839 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.839+0000 7fa856d9d640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8581f1e00 0x7fa858151640 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fa8580af640 tx=0x7fa85000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:11.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.839+0000 7fa846ffd640 1 -- 172.21.15.27:0/3974691099 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850004d60 con 0x7fa8581f1e00 2023-12-19T18:29:11.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.839+0000 7fa846ffd640 1 -- 172.21.15.27:0/3974691099 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa85000bb50 con 0x7fa8581f1e00 2023-12-19T18:29:11.840 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.839+0000 7fa85759e640 1 -- 172.21.15.27:0/3974691099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8580cc920 con 0x7fa8581f1e00 2023-12-19T18:29:11.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.839+0000 7fa846ffd640 1 -- 172.21.15.27:0/3974691099 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa85000bec0 con 0x7fa8581f1e00 2023-12-19T18:29:11.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.839+0000 7fa85759e640 1 -- 172.21.15.27:0/3974691099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8581f2460 con 0x7fa8581f1e00 2023-12-19T18:29:11.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.841+0000 7fa85759e640 1 -- 172.21.15.27:0/3974691099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa858150fe0 con 0x7fa8581f1e00 2023-12-19T18:29:11.841 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.842+0000 7fa846ffd640 1 -- 172.21.15.27:0/3974691099 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fa850007110 con 0x7fa8581f1e00 2023-12-19T18:29:11.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.842+0000 7fa846ffd640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa83806a430 0x7fa83806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:11.842 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.843+0000 7fa846ffd640 1 -- 172.21.15.27:0/3974691099 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fa850088950 con 0x7fa8581f1e00 2023-12-19T18:29:11.843 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.843+0000 7fa85659c640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa83806a430 0x7fa83806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:11.844 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.844+0000 7fa85659c640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa83806a430 0x7fa83806c8f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa848009910 tx=0x7fa848008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:11.846 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:11.847+0000 7fa846ffd640 1 -- 172.21.15.27:0/3974691099 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fa8500542d0 con 0x7fa8581f1e00 2023-12-19T18:29:12.043 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.043+0000 7fa85759e640 1 -- 172.21.15.27:0/3974691099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa85815c880 con 0x7fa8581f1e00 2023-12-19T18:29:12.045 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.045+0000 7fa846ffd640 1 -- 172.21.15.27:0/3974691099 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v210) v1 ==== 74+0+12527 (secure 0 0 0) 0x7fa85815c880 con 0x7fa8581f1e00 2023-12-19T18:29:12.046 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:29:12.046 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":210,"fsid":"dc17deb5-5eca-4f5c-983d-094ffb50339f","created":"2023-12-19T18:14:36.525286+0000","modified":"2023-12-19T18:29:06.949820+0000","last_up_change":"2023-12-19T18:14:43.040687+0000","last_in_change":"2023-12-19T18:14:40.022208+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.69999998807907104,"backfillfull_ratio":0.60000002384185791,"nearfull_ratio":0.60000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-19T18:14:43.789712+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":84136},{"type":"v1","addr":"172.21.15.27:6809","nonce":84136}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":84136},{"type":"v1","addr":"172.21.15.27:6811","nonce":84136}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":84136},{"type":"v1","addr":"172.21.15.27:6815","nonce":84136}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":84136},{"type":"v1","addr":"172.21.15.27:6813","nonce":84136}]},"public_addr":"172.21.15.27:6809/84136","cluster_addr":"172.21.15.27:6811/84136","heartbeat_back_addr":"172.21.15.27:6815/84136","heartbeat_front_addr":"172.21.15.27:6813/84136","state":["exists","up"]},{"osd":1,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":84133},{"type":"v1","addr":"172.21.15.27:6801","nonce":84133}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":84133},{"type":"v1","addr":"172.21.15.27:6803","nonce":84133}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":84133},{"type":"v1","addr":"172.21.15.27:6807","nonce":84133}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":84133},{"type":"v1","addr":"172.21.15.27:6805","nonce":84133}]},"public_addr":"172.21.15.27:6801/84133","cluster_addr":"172.21.15.27:6803/84133","heartbeat_back_addr":"172.21.15.27:6807/84133","heartbeat_front_addr":"172.21.15.27:6805/84133","state":["exists","up"]},{"osd":2,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":84138},{"type":"v1","addr":"172.21.15.27:6822","nonce":84138}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":84138},{"type":"v1","addr":"172.21.15.27:6826","nonce":84138}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6830","nonce":84138},{"type":"v1","addr":"172.21.15.27:6831","nonce":84138}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":84138},{"type":"v1","addr":"172.21.15.27:6829","nonce":84138}]},"public_addr":"172.21.15.27:6822/84138","cluster_addr":"172.21.15.27:6826/84138","heartbeat_back_addr":"172.21.15.27:6831/84138","heartbeat_front_addr":"172.21.15.27:6829/84138","state":["exists","up"]},{"osd":3,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":205,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":84146},{"type":"v1","addr":"172.21.15.27:6817","nonce":84146}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":84146},{"type":"v1","addr":"172.21.15.27:6819","nonce":84146}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6825","nonce":84146},{"type":"v1","addr":"172.21.15.27:6827","nonce":84146}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6821","nonce":84146},{"type":"v1","addr":"172.21.15.27:6823","nonce":84146}]},"public_addr":"172.21.15.27:6817/84146","cluster_addr":"172.21.15.27:6819/84146","heartbeat_back_addr":"172.21.15.27:6827/84146","heartbeat_front_addr":"172.21.15.27:6823/84146","state":["exists","up"]},{"osd":4,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6800","nonce":84205},{"type":"v1","addr":"172.21.15.192:6801","nonce":84205}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6802","nonce":84205},{"type":"v1","addr":"172.21.15.192:6803","nonce":84205}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6806","nonce":84205},{"type":"v1","addr":"172.21.15.192:6807","nonce":84205}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6804","nonce":84205},{"type":"v1","addr":"172.21.15.192:6805","nonce":84205}]},"public_addr":"172.21.15.192:6801/84205","cluster_addr":"172.21.15.192:6803/84205","heartbeat_back_addr":"172.21.15.192:6807/84205","heartbeat_front_addr":"172.21.15.192:6805/84205","state":["exists","up"]},{"osd":5,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6808","nonce":84206},{"type":"v1","addr":"172.21.15.192:6809","nonce":84206}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6810","nonce":84206},{"type":"v1","addr":"172.21.15.192:6811","nonce":84206}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6814","nonce":84206},{"type":"v1","addr":"172.21.15.192:6815","nonce":84206}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6812","nonce":84206},{"type":"v1","addr":"172.21.15.192:6813","nonce":84206}]},"public_addr":"172.21.15.192:6809/84206","cluster_addr":"172.21.15.192:6811/84206","heartbeat_back_addr":"172.21.15.192:6815/84206","heartbeat_front_addr":"172.21.15.192:6813/84206","state":["exists","up"]},{"osd":6,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6824","nonce":84255},{"type":"v1","addr":"172.21.15.192:6825","nonce":84255}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6826","nonce":84255},{"type":"v1","addr":"172.21.15.192:6827","nonce":84255}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6830","nonce":84255},{"type":"v1","addr":"172.21.15.192:6831","nonce":84255}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6828","nonce":84255},{"type":"v1","addr":"172.21.15.192:6829","nonce":84255}]},"public_addr":"172.21.15.192:6825/84255","cluster_addr":"172.21.15.192:6827/84255","heartbeat_back_addr":"172.21.15.192:6831/84255","heartbeat_front_addr":"172.21.15.192:6829/84255","state":["exists","up"]},{"osd":7,"uuid":"00000000-0000-0000-0000-000000000000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":200,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6816","nonce":84207},{"type":"v1","addr":"172.21.15.192:6817","nonce":84207}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6818","nonce":84207},{"type":"v1","addr":"172.21.15.192:6819","nonce":84207}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6822","nonce":84207},{"type":"v1","addr":"172.21.15.192:6823","nonce":84207}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6820","nonce":84207},{"type":"v1","addr":"172.21.15.192:6821","nonce":84207}]},"public_addr":"172.21.15.192:6817/84207","cluster_addr":"172.21.15.192:6819/84207","heartbeat_back_addr":"172.21.15.192:6823/84207","heartbeat_front_addr":"172.21.15.192:6821/84207","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.347271+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.319477+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.378368+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.279853+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.250859+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.292556+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.348967+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-19T18:14:41.339002+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6835/152568921":"2023-12-20T18:29:03.631771+0000","172.21.15.27:6834/152568921":"2023-12-20T18:29:03.631771+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-19T18:29:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.048+0000 7fa85759e640 1 -- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa83806a430 msgr2=0x7fa83806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.048+0000 7fa85759e640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa83806a430 0x7fa83806c8f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fa848009910 tx=0x7fa848008040 comp rx=0 tx=0).stop 2023-12-19T18:29:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.048+0000 7fa85759e640 1 -- 172.21.15.27:0/3974691099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8581f1e00 msgr2=0x7fa858151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.048+0000 7fa85759e640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8581f1e00 0x7fa858151640 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7fa8580af640 tx=0x7fa85000b4a0 comp rx=0 tx=0).stop 2023-12-19T18:29:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.048+0000 7fa85759e640 1 -- 172.21.15.27:0/3974691099 shutdown_connections 2023-12-19T18:29:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.048+0000 7fa85759e640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fa83806a430 0x7fa83806c8f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.048+0000 7fa85759e640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8581f1e00 0x7fa858151640 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.048+0000 7fa85759e640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fa85815c4a0 0x7fa8580aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.048+0000 7fa85759e640 1 --2- 172.21.15.27:0/3974691099 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fa840003f70 0x7fa85814ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:12.048 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.048+0000 7fa85759e640 1 -- 172.21.15.27:0/3974691099 >> 172.21.15.27:0/3974691099 conn(0x7fa858069c70 msgr2=0x7fa8580c4860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:12.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.048+0000 7fa85759e640 1 -- 172.21.15.27:0/3974691099 shutdown_connections 2023-12-19T18:29:12.049 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:12.048+0000 7fa85759e640 1 -- 172.21.15.27:0/3974691099 wait complete. 2023-12-19T18:29:13.069 INFO:tasks.ceph:Scrubbing osd.0 2023-12-19T18:29:13.070 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:29:13.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.325+0000 7f032759e640 1 Processor -- start 2023-12-19T18:29:13.325 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.325+0000 7f032759e640 1 -- start start 2023-12-19T18:29:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.327+0000 7f032759e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f032815cfe0 0x7f03280d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.327+0000 7f032759e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f03280d0fd0 0x7f03280d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.327+0000 7f032759e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03280ceb90 con 0x7f03280cf730 2023-12-19T18:29:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.327+0000 7f032759e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03280d1990 con 0x7f032815cfe0 2023-12-19T18:29:13.327 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.327+0000 7f032759e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03280d0c80 con 0x7f03280d0fd0 2023-12-19T18:29:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.327+0000 7f032659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f032815cfe0 0x7f03280d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.328+0000 7f032659c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f032815cfe0 0x7f03280d3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55478/0 (socket says 172.21.15.27:55478) 2023-12-19T18:29:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.328+0000 7f032659c640 1 -- 172.21.15.27:0/2479303682 learned_addr learned my addr 172.21.15.27:0/2479303682 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.328+0000 7f0325d9b640 1 --2- 172.21.15.27:0/2479303682 >> v2:172.21.15.192:3300/0 conn(0x7f03280d0fd0 0x7f03280d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.328+0000 7f032659c640 1 -- 172.21.15.27:0/2479303682 >> v2:172.21.15.192:3300/0 conn(0x7f03280d0fd0 msgr2=0x7f03280d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:13.328 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.328+0000 7f032659c640 1 --2- 172.21.15.27:0/2479303682 >> v2:172.21.15.192:3300/0 conn(0x7f03280d0fd0 0x7f03280d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.328+0000 7f032659c640 1 -- 172.21.15.27:0/2479303682 >> v1:172.21.15.192:6789/0 conn(0x7f03280cf730 legacy=0x7f03280cfba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.328+0000 7f032659c640 1 -- 172.21.15.27:0/2479303682 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03280ce2f0 con 0x7f032815cfe0 2023-12-19T18:29:13.329 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.330+0000 7f032659c640 1 --2- 172.21.15.27:0/2479303682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032815cfe0 0x7f03280d3860 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f032808beb0 tx=0x7f031802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56e4e987c1ad790b server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:13.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.330+0000 7f032559a640 1 -- 172.21.15.27:0/2479303682 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318031870 con 0x7f032815cfe0 2023-12-19T18:29:13.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.330+0000 7f032559a640 1 -- 172.21.15.27:0/2479303682 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03180319d0 con 0x7f032815cfe0 2023-12-19T18:29:13.330 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.331+0000 7f032559a640 1 -- 172.21.15.27:0/2479303682 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0318031d60 con 0x7f032815cfe0 2023-12-19T18:29:13.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.331+0000 7f032759e640 1 -- 172.21.15.27:0/2479303682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032815cfe0 msgr2=0x7f03280d3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:13.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.331+0000 7f032759e640 1 --2- 172.21.15.27:0/2479303682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032815cfe0 0x7f03280d3860 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7f032808beb0 tx=0x7f031802b910 comp rx=0 tx=0).stop 2023-12-19T18:29:13.331 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.331+0000 7f032759e640 1 -- 172.21.15.27:0/2479303682 shutdown_connections 2023-12-19T18:29:13.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.331+0000 7f032759e640 1 --2- 172.21.15.27:0/2479303682 >> v2:172.21.15.192:3300/0 conn(0x7f03280d0fd0 0x7f03280d1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.331+0000 7f032759e640 1 --2- 172.21.15.27:0/2479303682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032815cfe0 0x7f03280d3860 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.331+0000 7f032759e640 1 -- 172.21.15.27:0/2479303682 >> 172.21.15.27:0/2479303682 conn(0x7f03280d2d40 msgr2=0x7f03280d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:13.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.331+0000 7f032759e640 1 -- 172.21.15.27:0/2479303682 shutdown_connections 2023-12-19T18:29:13.332 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.332+0000 7f032759e640 1 -- 172.21.15.27:0/2479303682 wait complete. 2023-12-19T18:29:13.333 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.334+0000 7f032759e640 1 Processor -- start 2023-12-19T18:29:13.334 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.334+0000 7f032759e640 1 -- start start 2023-12-19T18:29:13.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.336+0000 7f032759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03280cf730 0x7f032815a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.336+0000 7f032759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280d0fd0 0x7f032815d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.336+0000 7f032759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f032815cfe0 0x7f032815d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.336+0000 7f032759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03280ca460 con 0x7f03280d0fd0 2023-12-19T18:29:13.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.336+0000 7f032759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032815df30 con 0x7f03280cf730 2023-12-19T18:29:13.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.336+0000 7f032759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f032815e210 con 0x7f032815cfe0 2023-12-19T18:29:13.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.336+0000 7f0325d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280d0fd0 0x7f032815d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:13.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.336+0000 7f0325d9b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280d0fd0 0x7f032815d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55482/0 (socket says 172.21.15.27:55482) 2023-12-19T18:29:13.336 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.336+0000 7f0325d9b640 1 -- 172.21.15.27:0/836827859 learned_addr learned my addr 172.21.15.27:0/836827859 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:13.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.336+0000 7f0325d9b640 1 -- 172.21.15.27:0/836827859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f032815cfe0 msgr2=0x7f032815d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:13.337 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.336+0000 7f0325d9b640 1 --2- 172.21.15.27:0/836827859 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f032815cfe0 0x7f032815d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.337+0000 7f0325d9b640 1 -- 172.21.15.27:0/836827859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03280cf730 msgr2=0x7f032815a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:13.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.337+0000 7f0325d9b640 1 --2- 172.21.15.27:0/836827859 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f03280cf730 0x7f032815a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.337+0000 7f0325d9b640 1 -- 172.21.15.27:0/836827859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03180094d0 con 0x7f03280d0fd0 2023-12-19T18:29:13.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.337+0000 7f0325d9b640 1 --2- 172.21.15.27:0/836827859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280d0fd0 0x7f032815d3c0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f031000d800 tx=0x7f031000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:13.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.337+0000 7f031effd640 1 -- 172.21.15.27:0/836827859 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031000b6c0 con 0x7f03280d0fd0 2023-12-19T18:29:13.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.337+0000 7f031effd640 1 -- 172.21.15.27:0/836827859 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f031000b820 con 0x7f03280d0fd0 2023-12-19T18:29:13.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.337+0000 7f032759e640 1 -- 172.21.15.27:0/836827859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03280c9e10 con 0x7f03280d0fd0 2023-12-19T18:29:13.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.337+0000 7f031effd640 1 -- 172.21.15.27:0/836827859 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f031000bbb0 con 0x7f03280d0fd0 2023-12-19T18:29:13.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.337+0000 7f032759e640 1 -- 172.21.15.27:0/836827859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032815eef0 con 0x7f03280d0fd0 2023-12-19T18:29:13.338 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.338+0000 7f032759e640 1 -- 172.21.15.27:0/836827859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f03280ce8b0 con 0x7f03280d0fd0 2023-12-19T18:29:13.339 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.340+0000 7f031effd640 1 -- 172.21.15.27:0/836827859 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f031000bd10 con 0x7f03280d0fd0 2023-12-19T18:29:13.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.341+0000 7f031effd640 1 --2- 172.21.15.27:0/836827859 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f030806a430 0x7f030806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.340 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.341+0000 7f031effd640 1 -- 172.21.15.27:0/836827859 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f0310088c00 con 0x7f03280d0fd0 2023-12-19T18:29:13.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.341+0000 7f031effd640 1 --2- 172.21.15.27:0/836827859 >> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] conn(0x7f0308070290 0x7f03080726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.341+0000 7f031effd640 1 -- 172.21.15.27:0/836827859 --> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0308072d60 con 0x7f0308070290 2023-12-19T18:29:13.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.341+0000 7f031effd640 1 -- 172.21.15.27:0/836827859 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=210) v2 ==== 24+0+0 (secure 0 0 0) 0x7f031008d2a0 con 0x7f03280d0fd0 2023-12-19T18:29:13.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.341+0000 7f032659c640 1 --2- 172.21.15.27:0/836827859 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f030806a430 0x7f030806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:13.341 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.341+0000 7f0326d9d640 1 --2- 172.21.15.27:0/836827859 >> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] conn(0x7f0308070290 0x7f03080726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:13.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.342+0000 7f0326d9d640 1 --2- 172.21.15.27:0/836827859 >> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] conn(0x7f0308070290 0x7f03080726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:13.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.342+0000 7f032659c640 1 --2- 172.21.15.27:0/836827859 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f030806a430 0x7f030806c8f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0318036000 tx=0x7f0318030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:13.342 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.343+0000 7f031effd640 1 -- 172.21.15.27:0/836827859 <== osd.0 v2:172.21.15.27:6808/84136 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f0308072d60 con 0x7f0308070290 2023-12-19T18:29:13.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.357+0000 7f032759e640 1 -- 172.21.15.27:0/836827859 --> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f03280ce5d0 con 0x7f0308070290 2023-12-19T18:29:13.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.357+0000 7f031effd640 1 -- 172.21.15.27:0/836827859 <== osd.0 v2:172.21.15.27:6808/84136 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f0320006b50 con 0x7f0308070290 2023-12-19T18:29:13.358 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:29:13.358 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-12-19T18:29:13.358 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:29:13.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.358+0000 7f032759e640 1 -- 172.21.15.27:0/836827859 >> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] conn(0x7f0308070290 msgr2=0x7f03080726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:13.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.358+0000 7f032759e640 1 --2- 172.21.15.27:0/836827859 >> [v2:172.21.15.27:6808/84136,v1:172.21.15.27:6809/84136] conn(0x7f0308070290 0x7f03080726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.358+0000 7f032759e640 1 -- 172.21.15.27:0/836827859 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f030806a430 msgr2=0x7f030806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:13.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.358+0000 7f032759e640 1 --2- 172.21.15.27:0/836827859 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f030806a430 0x7f030806c8f0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0318036000 tx=0x7f0318030000 comp rx=0 tx=0).stop 2023-12-19T18:29:13.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.358+0000 7f032759e640 1 -- 172.21.15.27:0/836827859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280d0fd0 msgr2=0x7f032815d3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:13.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.358+0000 7f032759e640 1 --2- 172.21.15.27:0/836827859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03280d0fd0 0x7f032815d3c0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7f031000d800 tx=0x7f031000dcc0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.358+0000 7f0326d9d640 1 -- 172.21.15.27:0/836827859 reap_dead start 2023-12-19T18:29:13.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.358+0000 7f032759e640 1 -- 172.21.15.27:0/836827859 shutdown_connections 2023-12-19T18:29:13.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.358+0000 7f032759e640 1 -- 172.21.15.27:0/836827859 >> 172.21.15.27:0/836827859 conn(0x7f03280d2d40 msgr2=0x7f03280b5960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:13.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.358+0000 7f032759e640 1 -- 172.21.15.27:0/836827859 shutdown_connections 2023-12-19T18:29:13.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.358+0000 7f032759e640 1 -- 172.21.15.27:0/836827859 wait complete. 2023-12-19T18:29:13.376 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-12-19T18:29:13.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.605+0000 7f6f7d4ee640 1 Processor -- start 2023-12-19T18:29:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.606+0000 7f6f7d4ee640 1 -- start start 2023-12-19T18:29:13.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.607+0000 7f6f7d4ee640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6f7815ac50 0x7f6f78070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.607+0000 7f6f7d4ee640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f780adf10 con 0x7f6f780aef20 2023-12-19T18:29:13.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.607+0000 7f6f7d4ee640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f780adac0 con 0x7f6f780af360 2023-12-19T18:29:13.607 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.607+0000 7f6f7d4ee640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f780ae160 con 0x7f6f7815ac50 2023-12-19T18:29:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.608+0000 7f6f6ffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f6f780aef20 0x7f6f7806b310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47564/0 (socket says 172.21.15.27:47564) 2023-12-19T18:29:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.608+0000 7f6f6ffff640 1 -- 172.21.15.27:0/989421741 learned_addr learned my addr 172.21.15.27:0/989421741 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.608+0000 7f6f77fff640 1 --2- 172.21.15.27:0/989421741 >> v2:172.21.15.192:3300/0 conn(0x7f6f7815ac50 0x7f6f78070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.608+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451460465 0 0) 0x7f6f780adf10 con 0x7f6f780aef20 2023-12-19T18:29:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.608+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f64003520 con 0x7f6f780aef20 2023-12-19T18:29:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2612367542 0 0) 0x7f6f64003520 con 0x7f6f780aef20 2023-12-19T18:29:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 >> v2:172.21.15.192:3300/0 conn(0x7f6f7815ac50 msgr2=0x7f6f78070fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f777fe640 1 --2- 172.21.15.27:0/989421741 >> v2:172.21.15.192:3300/0 conn(0x7f6f7815ac50 0x7f6f78070fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 >> v1:172.21.15.192:6789/0 conn(0x7f6f780af360 legacy=0x7f6f7806f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f780ae440 con 0x7f6f780aef20 2023-12-19T18:29:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f6f60001670 con 0x7f6f780aef20 2023-12-19T18:29:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f77fff640 1 --2- 172.21.15.27:0/989421741 >> v2:172.21.15.192:3300/0 conn(0x7f6f7815ac50 0x7f6f78070fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:29:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 >> v1:172.21.15.27:6789/0 conn(0x7f6f780aef20 legacy=0x7f6f7806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f777fe640 1 --2- 172.21.15.27:0/989421741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f640039c0 0x7f6f64003de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f777fe640 1 --2- 172.21.15.27:0/989421741 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f640047c0 0x7f6f64004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f777fe640 1 --2- 172.21.15.27:0/989421741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f640052a0 0x7f6f6400b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f780ae160 con 0x7f6f640052a0 2023-12-19T18:29:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6400c1a0 con 0x7f6f640039c0 2023-12-19T18:29:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6400c360 con 0x7f6f640047c0 2023-12-19T18:29:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.609+0000 7f6f7cced640 1 --2- 172.21.15.27:0/989421741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f640052a0 0x7f6f6400b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:13.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.610+0000 7f6f77fff640 1 --2- 172.21.15.27:0/989421741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f640039c0 0x7f6f64003de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:13.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.610+0000 7f6f6ffff640 1 --2- 172.21.15.27:0/989421741 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f640047c0 0x7f6f64004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:13.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.610+0000 7f6f7cced640 1 -- 172.21.15.27:0/989421741 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f640047c0 msgr2=0x7f6f64004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:13.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.610+0000 7f6f7cced640 1 --2- 172.21.15.27:0/989421741 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f640047c0 0x7f6f64004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.610+0000 7f6f7cced640 1 -- 172.21.15.27:0/989421741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f640039c0 msgr2=0x7f6f64003de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:13.612 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.610+0000 7f6f7cced640 1 --2- 172.21.15.27:0/989421741 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f640039c0 0x7f6f64003de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.610+0000 7f6f7cced640 1 -- 172.21.15.27:0/989421741 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64003520 con 0x7f6f640052a0 2023-12-19T18:29:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.611+0000 7f6f7cced640 1 -- 172.21.15.27:0/989421741 reap_dead start 2023-12-19T18:29:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.611+0000 7f6f7cced640 1 --2- 172.21.15.27:0/989421741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f640052a0 0x7f6f6400b980 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f6f70001e50 tx=0x7f6f640057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa38ca9bc41adf3e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.611+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70003b70 con 0x7f6f640052a0 2023-12-19T18:29:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.611+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f70003d10 con 0x7f6f640052a0 2023-12-19T18:29:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.611+0000 7f6f777fe640 1 -- 172.21.15.27:0/989421741 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f70005310 con 0x7f6f640052a0 2023-12-19T18:29:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.612+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/989421741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f640052a0 msgr2=0x7f6f6400b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.612+0000 7f6f7d4ee640 1 --2- 172.21.15.27:0/989421741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f640052a0 0x7f6f6400b980 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f6f70001e50 tx=0x7f6f640057f0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.612+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/989421741 shutdown_connections 2023-12-19T18:29:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.612+0000 7f6f7d4ee640 1 --2- 172.21.15.27:0/989421741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f640052a0 0x7f6f6400b980 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.612+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/989421741 >> 172.21.15.27:0/989421741 conn(0x7f6f7806a7d0 msgr2=0x7f6f780ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:13.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.612+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/989421741 shutdown_connections 2023-12-19T18:29:13.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.612+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/989421741 wait complete. 2023-12-19T18:29:13.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.614+0000 7f6f7d4ee640 1 Processor -- start 2023-12-19T18:29:13.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.614+0000 7f6f7d4ee640 1 -- start start 2023-12-19T18:29:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.615+0000 7f6f7d4ee640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f640052a0 0x7f6f78153800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.615+0000 7f6f7d4ee640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd450 0x7f6f780cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.615+0000 7f6f7d4ee640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f780b0c20 0x7f6f780b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.615+0000 7f6f7d4ee640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f780cde10 con 0x7f6f780cd450 2023-12-19T18:29:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.615+0000 7f6f7d4ee640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78158730 con 0x7f6f780b0c20 2023-12-19T18:29:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.615+0000 7f6f7d4ee640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f78158430 con 0x7f6f640052a0 2023-12-19T18:29:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.615+0000 7f6f6ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd450 0x7f6f780cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.616+0000 7f6f6ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd450 0x7f6f780cd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55504/0 (socket says 172.21.15.27:55504) 2023-12-19T18:29:13.616 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.616+0000 7f6f6ffff640 1 -- 172.21.15.27:0/3901121395 learned_addr learned my addr 172.21.15.27:0/3901121395 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:13.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.616+0000 7f6f77fff640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f640052a0 0x7f6f78153800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:13.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.616+0000 7f6f7cced640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f780b0c20 0x7f6f780b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:13.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.616+0000 7f6f6ffff640 1 -- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f640052a0 msgr2=0x7f6f78153800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:13.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.616+0000 7f6f6ffff640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f640052a0 0x7f6f78153800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.616+0000 7f6f6ffff640 1 -- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f780b0c20 msgr2=0x7f6f780b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:13.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.616+0000 7f6f6ffff640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f780b0c20 0x7f6f780b10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.616+0000 7f6f6ffff640 1 -- 172.21.15.27:0/3901121395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f780b19f0 con 0x7f6f780cd450 2023-12-19T18:29:13.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.616+0000 7f6f6ffff640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd450 0x7f6f780cd8d0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000e760 tx=0x7f6f6000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:13.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.617+0000 7f6f757fa640 1 -- 172.21.15.27:0/3901121395 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6000c6c0 con 0x7f6f780cd450 2023-12-19T18:29:13.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.617+0000 7f6f757fa640 1 -- 172.21.15.27:0/3901121395 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6000c820 con 0x7f6f780cd450 2023-12-19T18:29:13.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.617+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/3901121395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f780b1cd0 con 0x7f6f780cd450 2023-12-19T18:29:13.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.617+0000 7f6f757fa640 1 -- 172.21.15.27:0/3901121395 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6000cbd0 con 0x7f6f780cd450 2023-12-19T18:29:13.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.617+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/3901121395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f780b15e0 con 0x7f6f780cd450 2023-12-19T18:29:13.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.618+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/3901121395 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f780b17a0 con 0x7f6f780cd450 2023-12-19T18:29:13.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.619+0000 7f6f757fa640 1 -- 172.21.15.27:0/3901121395 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f6f6000cd30 con 0x7f6f780cd450 2023-12-19T18:29:13.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.620+0000 7f6f757fa640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5806a460 0x7f6f5806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:13.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.620+0000 7f6f757fa640 1 -- 172.21.15.27:0/3901121395 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6f600883f0 con 0x7f6f780cd450 2023-12-19T18:29:13.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.620+0000 7f6f77fff640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5806a460 0x7f6f5806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:13.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.622+0000 7f6f77fff640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5806a460 0x7f6f5806c920 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6f780b2970 tx=0x7f6f68008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:13.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.625+0000 7f6f757fa640 1 -- 172.21.15.27:0/3901121395 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6f60053cd0 con 0x7f6f780cd450 2023-12-19T18:29:13.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.820+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/3901121395 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f780aea00 con 0x7f6f5806a460 2023-12-19T18:29:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.821+0000 7f6f757fa640 1 -- 172.21.15.27:0/3901121395 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6f68012a10 con 0x7f6f5806a460 2023-12-19T18:29:13.821 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 0 to deep-scrub 2023-12-19T18:29:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.824+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5806a460 msgr2=0x7f6f5806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.824+0000 7f6f7d4ee640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5806a460 0x7f6f5806c920 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6f780b2970 tx=0x7f6f68008040 comp rx=0 tx=0).stop 2023-12-19T18:29:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.824+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/3901121395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd450 msgr2=0x7f6f780cd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.824+0000 7f6f7d4ee640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd450 0x7f6f780cd8d0 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f6f6000e760 tx=0x7f6f6000ec20 comp rx=0 tx=0).stop 2023-12-19T18:29:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.824+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/3901121395 shutdown_connections 2023-12-19T18:29:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.824+0000 7f6f7d4ee640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f6f5806a460 0x7f6f5806c920 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.824+0000 7f6f7d4ee640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6f780b0c20 0x7f6f780b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.824+0000 7f6f7d4ee640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6f780cd450 0x7f6f780cd8d0 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.824+0000 7f6f7d4ee640 1 --2- 172.21.15.27:0/3901121395 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f6f640052a0 0x7f6f78153800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.824+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/3901121395 >> 172.21.15.27:0/3901121395 conn(0x7f6f7806a7d0 msgr2=0x7f6f780c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.824+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/3901121395 shutdown_connections 2023-12-19T18:29:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:13.824+0000 7f6f7d4ee640 1 -- 172.21.15.27:0/3901121395 wait complete. 2023-12-19T18:29:13.846 INFO:tasks.ceph:Scrubbing osd.1 2023-12-19T18:29:13.846 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:29:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.087+0000 7f416c926640 1 Processor -- start 2023-12-19T18:29:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.087+0000 7f416c926640 1 -- start start 2023-12-19T18:29:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.089+0000 7f416c926640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f416815cec0 0x7f4168161390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.089+0000 7f416c926640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f416806e930 0x7f41680ad160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.089+0000 7f416c926640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41680accd0 con 0x7f41680ad6a0 2023-12-19T18:29:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.089+0000 7f416c926640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416815d2a0 con 0x7f416815cec0 2023-12-19T18:29:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.089+0000 7f416c926640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416806e4d0 con 0x7f416806e930 2023-12-19T18:29:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.089+0000 7f4166d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f416815cec0 0x7f4168161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.089+0000 7f4166d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f416815cec0 0x7f4168161390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55518/0 (socket says 172.21.15.27:55518) 2023-12-19T18:29:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.089+0000 7f4166d9d640 1 -- 172.21.15.27:0/2234871204 learned_addr learned my addr 172.21.15.27:0/2234871204 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.090+0000 7f416659c640 1 --2- 172.21.15.27:0/2234871204 >> v2:172.21.15.192:3300/0 conn(0x7f416806e930 0x7f41680ad160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.090+0000 7f4166d9d640 1 -- 172.21.15.27:0/2234871204 >> v2:172.21.15.192:3300/0 conn(0x7f416806e930 msgr2=0x7f41680ad160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.090+0000 7f4166d9d640 1 --2- 172.21.15.27:0/2234871204 >> v2:172.21.15.192:3300/0 conn(0x7f416806e930 0x7f41680ad160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.090+0000 7f4166d9d640 1 -- 172.21.15.27:0/2234871204 >> v1:172.21.15.192:6789/0 conn(0x7f41680ad6a0 legacy=0x7f4168069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.090+0000 7f4166d9d640 1 -- 172.21.15.27:0/2234871204 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41680af460 con 0x7f416815cec0 2023-12-19T18:29:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.091+0000 7f4166d9d640 1 --2- 172.21.15.27:0/2234871204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416815cec0 0x7f4168161390 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f4168071650 tx=0x7f415802bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13e453a616ce760a server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.091+0000 7f4165d9b640 1 -- 172.21.15.27:0/2234871204 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415802cad0 con 0x7f416815cec0 2023-12-19T18:29:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.091+0000 7f4165d9b640 1 -- 172.21.15.27:0/2234871204 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415802cc30 con 0x7f416815cec0 2023-12-19T18:29:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.091+0000 7f4165d9b640 1 -- 172.21.15.27:0/2234871204 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41580343f0 con 0x7f416815cec0 2023-12-19T18:29:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.092+0000 7f416c926640 1 -- 172.21.15.27:0/2234871204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416815cec0 msgr2=0x7f4168161390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.092+0000 7f416c926640 1 --2- 172.21.15.27:0/2234871204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416815cec0 0x7f4168161390 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f4168071650 tx=0x7f415802bf30 comp rx=0 tx=0).stop 2023-12-19T18:29:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.092+0000 7f416c926640 1 -- 172.21.15.27:0/2234871204 shutdown_connections 2023-12-19T18:29:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.092+0000 7f416c926640 1 --2- 172.21.15.27:0/2234871204 >> v2:172.21.15.192:3300/0 conn(0x7f416806e930 0x7f41680ad160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.092+0000 7f416c926640 1 --2- 172.21.15.27:0/2234871204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416815cec0 0x7f4168161390 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.092+0000 7f416c926640 1 -- 172.21.15.27:0/2234871204 >> 172.21.15.27:0/2234871204 conn(0x7f41680b1710 msgr2=0x7f41680b1b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.092+0000 7f416c926640 1 -- 172.21.15.27:0/2234871204 shutdown_connections 2023-12-19T18:29:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.092+0000 7f416c926640 1 -- 172.21.15.27:0/2234871204 wait complete. 2023-12-19T18:29:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.094+0000 7f416c926640 1 Processor -- start 2023-12-19T18:29:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.094+0000 7f416c926640 1 -- start start 2023-12-19T18:29:14.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.095+0000 7f416c926640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f416806e930 0x7f41681f2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.095+0000 7f416c926640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41680ad6a0 0x7f416815a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.095 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.095+0000 7f416c926640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416815cec0 0x7f41681f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.095+0000 7f416c926640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416815a6a0 con 0x7f416815cec0 2023-12-19T18:29:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.095+0000 7f416c926640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41681f65f0 con 0x7f41680ad6a0 2023-12-19T18:29:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.095+0000 7f416c926640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41681f68d0 con 0x7f416806e930 2023-12-19T18:29:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.095+0000 7f416759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416815cec0 0x7f41681f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.095+0000 7f416759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416815cec0 0x7f41681f5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55532/0 (socket says 172.21.15.27:55532) 2023-12-19T18:29:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.095+0000 7f416759e640 1 -- 172.21.15.27:0/4165801924 learned_addr learned my addr 172.21.15.27:0/4165801924 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.096+0000 7f416659c640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41680ad6a0 0x7f416815a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.096+0000 7f4166d9d640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f416806e930 0x7f41681f2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.096+0000 7f416759e640 1 -- 172.21.15.27:0/4165801924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f416806e930 msgr2=0x7f41681f2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.096+0000 7f416759e640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f416806e930 0x7f41681f2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.096+0000 7f416759e640 1 -- 172.21.15.27:0/4165801924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41680ad6a0 msgr2=0x7f416815a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.096+0000 7f416759e640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f41680ad6a0 0x7f416815a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.096+0000 7f416759e640 1 -- 172.21.15.27:0/4165801924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4158009b30 con 0x7f416815cec0 2023-12-19T18:29:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.096+0000 7f416759e640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416815cec0 0x7f41681f5a80 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f416000c360 tx=0x7f416000c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.096+0000 7f415f7fe640 1 -- 172.21.15.27:0/4165801924 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41600035f0 con 0x7f416815cec0 2023-12-19T18:29:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.096+0000 7f415f7fe640 1 -- 172.21.15.27:0/4165801924 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4160003750 con 0x7f416815cec0 2023-12-19T18:29:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.096+0000 7f415f7fe640 1 -- 172.21.15.27:0/4165801924 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4160003b40 con 0x7f416815cec0 2023-12-19T18:29:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.096+0000 7f416c926640 1 -- 172.21.15.27:0/4165801924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41681f27e0 con 0x7f416815cec0 2023-12-19T18:29:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.097+0000 7f416c926640 1 -- 172.21.15.27:0/4165801924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416815fc70 con 0x7f416815cec0 2023-12-19T18:29:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.098+0000 7f416c926640 1 -- 172.21.15.27:0/4165801924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f41680ac960 con 0x7f416815cec0 2023-12-19T18:29:14.099 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.099+0000 7f415f7fe640 1 -- 172.21.15.27:0/4165801924 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4160003ca0 con 0x7f416815cec0 2023-12-19T18:29:14.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.100+0000 7f415f7fe640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f414806a430 0x7f414806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.100+0000 7f415f7fe640 1 -- 172.21.15.27:0/4165801924 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4160087d20 con 0x7f416815cec0 2023-12-19T18:29:14.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.100+0000 7f415f7fe640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] conn(0x7f4148070290 0x7f41480726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.100+0000 7f415f7fe640 1 -- 172.21.15.27:0/4165801924 --> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4148072d60 con 0x7f4148070290 2023-12-19T18:29:14.100 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.100+0000 7f415f7fe640 1 -- 172.21.15.27:0/4165801924 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=210) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4160088090 con 0x7f416815cec0 2023-12-19T18:29:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.101+0000 7f4166d9d640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f414806a430 0x7f414806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.101+0000 7f416659c640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] conn(0x7f4148070290 0x7f41480726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.101+0000 7f416659c640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] conn(0x7f4148070290 0x7f41480726b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.102+0000 7f4166d9d640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f414806a430 0x7f414806c8f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4158035000 tx=0x7f4158030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:14.102 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.102+0000 7f415f7fe640 1 -- 172.21.15.27:0/4165801924 <== osd.1 v2:172.21.15.27:6800/84133 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f4148072d60 con 0x7f4148070290 2023-12-19T18:29:14.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.115+0000 7f416c926640 1 -- 172.21.15.27:0/4165801924 --> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f41680af180 con 0x7f4148070290 2023-12-19T18:29:14.115 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.116+0000 7f415f7fe640 1 -- 172.21.15.27:0/4165801924 <== osd.1 v2:172.21.15.27:6800/84133 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f415000bb20 con 0x7f4148070290 2023-12-19T18:29:14.115 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:29:14.116 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-12-19T18:29:14.116 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:29:14.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.116+0000 7f416c926640 1 -- 172.21.15.27:0/4165801924 >> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] conn(0x7f4148070290 msgr2=0x7f41480726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.116+0000 7f416c926640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.27:6800/84133,v1:172.21.15.27:6801/84133] conn(0x7f4148070290 0x7f41480726b0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.116+0000 7f416c926640 1 -- 172.21.15.27:0/4165801924 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f414806a430 msgr2=0x7f414806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.116+0000 7f416c926640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f414806a430 0x7f414806c8f0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f4158035000 tx=0x7f4158030000 comp rx=0 tx=0).stop 2023-12-19T18:29:14.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.116+0000 7f416c926640 1 -- 172.21.15.27:0/4165801924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416815cec0 msgr2=0x7f41681f5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.116+0000 7f416c926640 1 --2- 172.21.15.27:0/4165801924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f416815cec0 0x7f41681f5a80 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f416000c360 tx=0x7f416000c820 comp rx=0 tx=0).stop 2023-12-19T18:29:14.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.116+0000 7f416759e640 1 -- 172.21.15.27:0/4165801924 reap_dead start 2023-12-19T18:29:14.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.116+0000 7f416c926640 1 -- 172.21.15.27:0/4165801924 shutdown_connections 2023-12-19T18:29:14.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.116+0000 7f416c926640 1 -- 172.21.15.27:0/4165801924 >> 172.21.15.27:0/4165801924 conn(0x7f41680b1710 msgr2=0x7f416806d850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:14.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.116+0000 7f416c926640 1 -- 172.21.15.27:0/4165801924 shutdown_connections 2023-12-19T18:29:14.117 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.116+0000 7f416c926640 1 -- 172.21.15.27:0/4165801924 wait complete. 2023-12-19T18:29:14.135 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-12-19T18:29:14.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.375+0000 7fe55e93e640 1 Processor -- start 2023-12-19T18:29:14.376 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.376+0000 7fe55e93e640 1 -- start start 2023-12-19T18:29:14.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.377+0000 7fe55e93e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe55815d1f0 0x7fe5580cb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.377+0000 7fe55e93e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe55806c380 0x7fe5580cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.377+0000 7fe55e93e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe55806cc20 con 0x7fe5580cc1f0 2023-12-19T18:29:14.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.377+0000 7fe55e93e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe55806c940 con 0x7fe55815d1f0 2023-12-19T18:29:14.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.377+0000 7fe55e93e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5580ccff0 con 0x7fe55806c380 2023-12-19T18:29:14.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.378+0000 7fe5577fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe55815d1f0 0x7fe5580cb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.378+0000 7fe5577fe640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fe55815d1f0 0x7fe5580cb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55534/0 (socket says 172.21.15.27:55534) 2023-12-19T18:29:14.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.378+0000 7fe5577fe640 1 -- 172.21.15.27:0/546111784 learned_addr learned my addr 172.21.15.27:0/546111784 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:14.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.378+0000 7fe55cedb640 1 --2- 172.21.15.27:0/546111784 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe55806c380 0x7fe5580cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.378+0000 7fe5577fe640 1 -- 172.21.15.27:0/546111784 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe55806c380 msgr2=0x7fe5580cdad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:14.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.378+0000 7fe5577fe640 1 --2- 172.21.15.27:0/546111784 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe55806c380 0x7fe5580cdad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.378+0000 7fe5577fe640 1 -- 172.21.15.27:0/546111784 >> v1:172.21.15.192:6789/0 conn(0x7fe5580cc1f0 legacy=0x7fe5580cc5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:14.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.378+0000 7fe5577fe640 1 -- 172.21.15.27:0/546111784 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5580cd2d0 con 0x7fe55815d1f0 2023-12-19T18:29:14.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.380+0000 7fe5577fe640 1 --2- 172.21.15.27:0/546111784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55815d1f0 0x7fe5580cb5c0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fe540009870 tx=0x7fe54002b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d326a2d9ce94f6f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:14.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.380+0000 7fe556ffd640 1 -- 172.21.15.27:0/546111784 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe540031640 con 0x7fe55815d1f0 2023-12-19T18:29:14.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.380+0000 7fe556ffd640 1 -- 172.21.15.27:0/546111784 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5400317a0 con 0x7fe55815d1f0 2023-12-19T18:29:14.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.380+0000 7fe556ffd640 1 -- 172.21.15.27:0/546111784 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe540031b30 con 0x7fe55815d1f0 2023-12-19T18:29:14.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.381+0000 7fe55e93e640 1 -- 172.21.15.27:0/546111784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55815d1f0 msgr2=0x7fe5580cb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.381+0000 7fe55e93e640 1 --2- 172.21.15.27:0/546111784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55815d1f0 0x7fe5580cb5c0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7fe540009870 tx=0x7fe54002b8f0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.381+0000 7fe55e93e640 1 -- 172.21.15.27:0/546111784 shutdown_connections 2023-12-19T18:29:14.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.381+0000 7fe55e93e640 1 --2- 172.21.15.27:0/546111784 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe55806c380 0x7fe5580cdad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.381+0000 7fe55e93e640 1 --2- 172.21.15.27:0/546111784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55815d1f0 0x7fe5580cb5c0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.381+0000 7fe55e93e640 1 -- 172.21.15.27:0/546111784 >> 172.21.15.27:0/546111784 conn(0x7fe55806a060 msgr2=0x7fe55806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:14.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.381+0000 7fe55e93e640 1 -- 172.21.15.27:0/546111784 shutdown_connections 2023-12-19T18:29:14.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.381+0000 7fe55e93e640 1 -- 172.21.15.27:0/546111784 wait complete. 2023-12-19T18:29:14.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.383+0000 7fe55e93e640 1 Processor -- start 2023-12-19T18:29:14.383 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.383+0000 7fe55e93e640 1 -- start start 2023-12-19T18:29:14.385 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.384+0000 7fe55e93e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55806c380 0x7fe55814a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.384+0000 7fe55e93e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5580cc1f0 0x7fe5580aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.384+0000 7fe55e93e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5580af190 0x7fe558070e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.384+0000 7fe55e93e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5580c4a80 con 0x7fe55806c380 2023-12-19T18:29:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.384+0000 7fe55e93e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5580af7a0 con 0x7fe5580cc1f0 2023-12-19T18:29:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.384+0000 7fe55e93e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5580afa80 con 0x7fe5580af190 2023-12-19T18:29:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.384+0000 7fe557fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55806c380 0x7fe55814a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.384+0000 7fe557fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55806c380 0x7fe55814a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55540/0 (socket says 172.21.15.27:55540) 2023-12-19T18:29:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.384+0000 7fe557fff640 1 -- 172.21.15.27:0/2035323102 learned_addr learned my addr 172.21.15.27:0/2035323102 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe55cedb640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5580af190 0x7fe558070e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe5577fe640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5580cc1f0 0x7fe5580aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.386 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe557fff640 1 -- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5580af190 msgr2=0x7fe558070e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe557fff640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5580af190 0x7fe558070e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe557fff640 1 -- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5580cc1f0 msgr2=0x7fe5580aec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe557fff640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5580cc1f0 0x7fe5580aec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe557fff640 1 -- 172.21.15.27:0/2035323102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5400094d0 con 0x7fe55806c380 2023-12-19T18:29:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe5577fe640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5580cc1f0 0x7fe5580aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe55cedb640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5580af190 0x7fe558070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe557fff640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55806c380 0x7fe55814a680 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fe54800c7c0 tx=0x7fe54800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe554ff9640 1 -- 172.21.15.27:0/2035323102 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54800aa50 con 0x7fe55806c380 2023-12-19T18:29:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe554ff9640 1 -- 172.21.15.27:0/2035323102 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54800abb0 con 0x7fe55806c380 2023-12-19T18:29:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe554ff9640 1 -- 172.21.15.27:0/2035323102 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe548012420 con 0x7fe55806c380 2023-12-19T18:29:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.385+0000 7fe55e93e640 1 -- 172.21.15.27:0/2035323102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5580afd60 con 0x7fe55806c380 2023-12-19T18:29:14.387 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.386+0000 7fe55e93e640 1 -- 172.21.15.27:0/2035323102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5580b0320 con 0x7fe55806c380 2023-12-19T18:29:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.387+0000 7fe55e93e640 1 -- 172.21.15.27:0/2035323102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5580b0510 con 0x7fe55806c380 2023-12-19T18:29:14.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.388+0000 7fe554ff9640 1 -- 172.21.15.27:0/2035323102 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe548012580 con 0x7fe55806c380 2023-12-19T18:29:14.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.389+0000 7fe554ff9640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe53806a430 0x7fe53806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.389+0000 7fe554ff9640 1 -- 172.21.15.27:0/2035323102 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe548088570 con 0x7fe55806c380 2023-12-19T18:29:14.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.390+0000 7fe5577fe640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe53806a430 0x7fe53806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.390 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.391+0000 7fe5577fe640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe53806a430 0x7fe53806c8f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe540036000 tx=0x7fe54003c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:14.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.394+0000 7fe554ff9640 1 -- 172.21.15.27:0/2035323102 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe548053e60 con 0x7fe55806c380 2023-12-19T18:29:14.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.587+0000 7fe55e93e640 1 -- 172.21.15.27:0/2035323102 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5580cbb00 con 0x7fe53806a430 2023-12-19T18:29:14.587 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.588+0000 7fe554ff9640 1 -- 172.21.15.27:0/2035323102 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe540031a10 con 0x7fe53806a430 2023-12-19T18:29:14.588 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 1 to deep-scrub 2023-12-19T18:29:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.590+0000 7fe55e93e640 1 -- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe53806a430 msgr2=0x7fe53806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.590+0000 7fe55e93e640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe53806a430 0x7fe53806c8f0 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fe540036000 tx=0x7fe54003c040 comp rx=0 tx=0).stop 2023-12-19T18:29:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.590+0000 7fe55e93e640 1 -- 172.21.15.27:0/2035323102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55806c380 msgr2=0x7fe55814a680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.590+0000 7fe55e93e640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55806c380 0x7fe55814a680 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7fe54800c7c0 tx=0x7fe54800cc80 comp rx=0 tx=0).stop 2023-12-19T18:29:14.590 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.590+0000 7fe55e93e640 1 -- 172.21.15.27:0/2035323102 shutdown_connections 2023-12-19T18:29:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.590+0000 7fe55e93e640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe53806a430 0x7fe53806c8f0 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.590+0000 7fe55e93e640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe5580af190 0x7fe558070e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.590+0000 7fe55e93e640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe5580cc1f0 0x7fe5580aec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.590+0000 7fe55e93e640 1 --2- 172.21.15.27:0/2035323102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe55806c380 0x7fe55814a680 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.590+0000 7fe55e93e640 1 -- 172.21.15.27:0/2035323102 >> 172.21.15.27:0/2035323102 conn(0x7fe55806a060 msgr2=0x7fe5580cf0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.590+0000 7fe55e93e640 1 -- 172.21.15.27:0/2035323102 shutdown_connections 2023-12-19T18:29:14.591 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.591+0000 7fe55e93e640 1 -- 172.21.15.27:0/2035323102 wait complete. 2023-12-19T18:29:14.610 INFO:tasks.ceph:Scrubbing osd.2 2023-12-19T18:29:14.610 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:29:14.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.851+0000 7f142359e640 1 Processor -- start 2023-12-19T18:29:14.851 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.851+0000 7f142359e640 1 -- start start 2023-12-19T18:29:14.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.853+0000 7f142359e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f14240ccdb0 0x7f142415d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.853+0000 7f142359e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14240b0370 0x7f14240b0780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.853+0000 7f142359e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1424160a60 con 0x7f14240cae00 2023-12-19T18:29:14.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.853+0000 7f142359e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14240b0cc0 con 0x7f14240b0370 2023-12-19T18:29:14.853 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.853+0000 7f142359e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1424161900 con 0x7f14240ccdb0 2023-12-19T18:29:14.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.853+0000 7f1422d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14240b0370 0x7f14240b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.853+0000 7f1422d9d640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f14240b0370 0x7f14240b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55550/0 (socket says 172.21.15.27:55550) 2023-12-19T18:29:14.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.853+0000 7f141bfff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f14240cae00 0x7f14241611f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47568/0 (socket says 172.21.15.27:47568) 2023-12-19T18:29:14.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.853+0000 7f1422d9d640 1 -- 172.21.15.27:0/2803689729 learned_addr learned my addr 172.21.15.27:0/2803689729 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:14.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.854+0000 7f142259c640 1 --2- 172.21.15.27:0/2803689729 >> v2:172.21.15.192:3300/0 conn(0x7f14240ccdb0 0x7f142415d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.854+0000 7f1422d9d640 1 -- 172.21.15.27:0/2803689729 >> v2:172.21.15.192:3300/0 conn(0x7f14240ccdb0 msgr2=0x7f142415d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:14.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.854+0000 7f1422d9d640 1 --2- 172.21.15.27:0/2803689729 >> v2:172.21.15.192:3300/0 conn(0x7f14240ccdb0 0x7f142415d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.854+0000 7f1422d9d640 1 -- 172.21.15.27:0/2803689729 >> v1:172.21.15.27:6789/0 conn(0x7f14240cae00 legacy=0x7f14241611f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.854 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.854+0000 7f1422d9d640 1 -- 172.21.15.27:0/2803689729 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142415dd00 con 0x7f14240b0370 2023-12-19T18:29:14.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.855+0000 7f1422d9d640 1 --2- 172.21.15.27:0/2803689729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14240b0370 0x7f14240b0780 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f142406d550 tx=0x7f141c02bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0cb02a7bd0e2339 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:14.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.855+0000 7f1421d9b640 1 -- 172.21.15.27:0/2803689729 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c02cb10 con 0x7f14240b0370 2023-12-19T18:29:14.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.855+0000 7f1421d9b640 1 -- 172.21.15.27:0/2803689729 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141c02cc70 con 0x7f14240b0370 2023-12-19T18:29:14.855 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.855+0000 7f1421d9b640 1 -- 172.21.15.27:0/2803689729 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141c0343f0 con 0x7f14240b0370 2023-12-19T18:29:14.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.856+0000 7f142359e640 1 -- 172.21.15.27:0/2803689729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14240b0370 msgr2=0x7f14240b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.856+0000 7f142359e640 1 --2- 172.21.15.27:0/2803689729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14240b0370 0x7f14240b0780 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f142406d550 tx=0x7f141c02bf70 comp rx=0 tx=0).stop 2023-12-19T18:29:14.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.856+0000 7f142359e640 1 -- 172.21.15.27:0/2803689729 shutdown_connections 2023-12-19T18:29:14.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.856+0000 7f142359e640 1 --2- 172.21.15.27:0/2803689729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14240b0370 0x7f14240b0780 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.856 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.856+0000 7f142359e640 1 --2- 172.21.15.27:0/2803689729 >> v2:172.21.15.192:3300/0 conn(0x7f14240ccdb0 0x7f142415d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.856+0000 7f142359e640 1 -- 172.21.15.27:0/2803689729 >> 172.21.15.27:0/2803689729 conn(0x7f14240cb370 msgr2=0x7f14240cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:14.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.856+0000 7f142359e640 1 -- 172.21.15.27:0/2803689729 shutdown_connections 2023-12-19T18:29:14.857 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.856+0000 7f142359e640 1 -- 172.21.15.27:0/2803689729 wait complete. 2023-12-19T18:29:14.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.859+0000 7f142359e640 1 Processor -- start 2023-12-19T18:29:14.858 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.859+0000 7f142359e640 1 -- start start 2023-12-19T18:29:14.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.860+0000 7f142359e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14240b0370 0x7f142414c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.860+0000 7f142359e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14240cae00 0x7f14240c8370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.860+0000 7f142359e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14240ccdb0 0x7f1424145fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.860+0000 7f142359e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14240b3530 con 0x7f14240cae00 2023-12-19T18:29:14.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.860+0000 7f142359e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14240c88b0 con 0x7f14240ccdb0 2023-12-19T18:29:14.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.860+0000 7f142359e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14241464f0 con 0x7f14240b0370 2023-12-19T18:29:14.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.860+0000 7f141bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14240cae00 0x7f14240c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.860+0000 7f141bfff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14240cae00 0x7f14240c8370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55558/0 (socket says 172.21.15.27:55558) 2023-12-19T18:29:14.860 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.860+0000 7f141bfff640 1 -- 172.21.15.27:0/3135105306 learned_addr learned my addr 172.21.15.27:0/3135105306 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:14.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.861+0000 7f1422d9d640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14240ccdb0 0x7f1424145fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.861+0000 7f142259c640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14240b0370 0x7f142414c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.861+0000 7f141bfff640 1 -- 172.21.15.27:0/3135105306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14240b0370 msgr2=0x7f142414c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.861+0000 7f141bfff640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14240b0370 0x7f142414c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.861+0000 7f141bfff640 1 -- 172.21.15.27:0/3135105306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14240ccdb0 msgr2=0x7f1424145fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.861+0000 7f141bfff640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14240ccdb0 0x7f1424145fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.861+0000 7f141bfff640 1 -- 172.21.15.27:0/3135105306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c009b30 con 0x7f14240cae00 2023-12-19T18:29:14.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.861+0000 7f142259c640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f14240b0370 0x7f142414c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:14.862 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.861+0000 7f1422d9d640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f14240ccdb0 0x7f1424145fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:14.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.861+0000 7f141bfff640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14240cae00 0x7f14240c8370 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c410 tx=0x7f140c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:14.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.861+0000 7f141affd640 1 -- 172.21.15.27:0/3135105306 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c0074e0 con 0x7f14240cae00 2023-12-19T18:29:14.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.862+0000 7f141affd640 1 -- 172.21.15.27:0/3135105306 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f140c007640 con 0x7f14240cae00 2023-12-19T18:29:14.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.862+0000 7f141affd640 1 -- 172.21.15.27:0/3135105306 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f140c0049e0 con 0x7f14240cae00 2023-12-19T18:29:14.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.862+0000 7f142359e640 1 -- 172.21.15.27:0/3135105306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1424149320 con 0x7f14240cae00 2023-12-19T18:29:14.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.862+0000 7f142359e640 1 -- 172.21.15.27:0/3135105306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14240b2e70 con 0x7f14240cae00 2023-12-19T18:29:14.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.863+0000 7f142359e640 1 -- 172.21.15.27:0/3135105306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1424160cd0 con 0x7f14240cae00 2023-12-19T18:29:14.863 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.864+0000 7f141affd640 1 -- 172.21.15.27:0/3135105306 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f140c004b40 con 0x7f14240cae00 2023-12-19T18:29:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.865+0000 7f141affd640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f140406a430 0x7f140406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.865+0000 7f141affd640 1 -- 172.21.15.27:0/3135105306 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f140c088020 con 0x7f14240cae00 2023-12-19T18:29:14.865 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.865+0000 7f141affd640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] conn(0x7f1404070290 0x7f14040726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:14.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.865+0000 7f141affd640 1 -- 172.21.15.27:0/3135105306 --> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1404072d60 con 0x7f1404070290 2023-12-19T18:29:14.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.865+0000 7f141affd640 1 -- 172.21.15.27:0/3135105306 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=210) v2 ==== 24+0+0 (secure 0 0 0) 0x7f140c08a600 con 0x7f14240cae00 2023-12-19T18:29:14.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.865+0000 7f1422d9d640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] conn(0x7f1404070290 0x7f14040726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.866 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.865+0000 7f142259c640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f140406a430 0x7f140406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:14.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.866+0000 7f1422d9d640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] conn(0x7f1404070290 0x7f14040726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:14.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.867+0000 7f142259c640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f140406a430 0x7f140406c8f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1414006fd0 tx=0x7f1414008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:14.867 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.867+0000 7f141affd640 1 -- 172.21.15.27:0/3135105306 <== osd.2 v2:172.21.15.27:6820/84138 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f1404072d60 con 0x7f1404070290 2023-12-19T18:29:14.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.880+0000 7f142359e640 1 -- 172.21.15.27:0/3135105306 --> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f142415dfe0 con 0x7f1404070290 2023-12-19T18:29:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.881+0000 7f141affd640 1 -- 172.21.15.27:0/3135105306 <== osd.2 v2:172.21.15.27:6820/84138 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f141c02cb00 con 0x7f1404070290 2023-12-19T18:29:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:29:14.881 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-12-19T18:29:14.881 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:29:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.881+0000 7f142359e640 1 -- 172.21.15.27:0/3135105306 >> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] conn(0x7f1404070290 msgr2=0x7f14040726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.881+0000 7f142359e640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.27:6820/84138,v1:172.21.15.27:6822/84138] conn(0x7f1404070290 0x7f14040726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.882+0000 7f142359e640 1 -- 172.21.15.27:0/3135105306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f140406a430 msgr2=0x7f140406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.882+0000 7f142359e640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f140406a430 0x7f140406c8f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1414006fd0 tx=0x7f1414008040 comp rx=0 tx=0).stop 2023-12-19T18:29:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.882+0000 7f142359e640 1 -- 172.21.15.27:0/3135105306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14240cae00 msgr2=0x7f14240c8370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.882+0000 7f142359e640 1 --2- 172.21.15.27:0/3135105306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14240cae00 0x7f14240c8370 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f140c00c410 tx=0x7f140c00c8d0 comp rx=0 tx=0).stop 2023-12-19T18:29:14.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.882+0000 7f1422d9d640 1 -- 172.21.15.27:0/3135105306 reap_dead start 2023-12-19T18:29:14.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.882+0000 7f142359e640 1 -- 172.21.15.27:0/3135105306 shutdown_connections 2023-12-19T18:29:14.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.882+0000 7f142359e640 1 -- 172.21.15.27:0/3135105306 >> 172.21.15.27:0/3135105306 conn(0x7f14240cb370 msgr2=0x7f142406ca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:14.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.882+0000 7f142359e640 1 -- 172.21.15.27:0/3135105306 shutdown_connections 2023-12-19T18:29:14.883 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:14.882+0000 7f142359e640 1 -- 172.21.15.27:0/3135105306 wait complete. 2023-12-19T18:29:14.904 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-12-19T18:29:15.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.139+0000 7fe0f759e640 1 Processor -- start 2023-12-19T18:29:15.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.139+0000 7fe0f759e640 1 -- start start 2023-12-19T18:29:15.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.140+0000 7fe0f759e640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe0f815ac50 0x7fe0f8070fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.140+0000 7fe0f759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0f80aef20 0x7fe0f806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.140+0000 7fe0f759e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0f80adf10 con 0x7fe0f80af360 2023-12-19T18:29:15.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.140+0000 7fe0f759e640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0f80adac0 con 0x7fe0f815ac50 2023-12-19T18:29:15.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.140+0000 7fe0f759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f80ae160 con 0x7fe0f80aef20 2023-12-19T18:29:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.141+0000 7fe0f659c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe0f815ac50 0x7fe0f8070fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.141+0000 7fe0effff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0f80aef20 0x7fe0f806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.141+0000 7fe0f659c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fe0f815ac50 0x7fe0f8070fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:53258/0 (socket says 172.21.15.27:53258) 2023-12-19T18:29:15.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.141+0000 7fe0f659c640 1 -- 172.21.15.27:0/246197122 learned_addr learned my addr 172.21.15.27:0/246197122 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.142+0000 7fe0f659c640 1 -- 172.21.15.27:0/246197122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0f80aef20 msgr2=0x7fe0f806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.142+0000 7fe0f659c640 1 --2- 172.21.15.27:0/246197122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0f80aef20 0x7fe0f806b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.142+0000 7fe0f659c640 1 -- 172.21.15.27:0/246197122 >> v1:172.21.15.192:6789/0 conn(0x7fe0f80af360 legacy=0x7fe0f806b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.142+0000 7fe0f659c640 1 -- 172.21.15.27:0/246197122 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f80ae440 con 0x7fe0f815ac50 2023-12-19T18:29:15.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.142+0000 7fe0effff640 1 --2- 172.21.15.27:0/246197122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0f80aef20 0x7fe0f806b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-19T18:29:15.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.143+0000 7fe0f659c640 1 --2- 172.21.15.27:0/246197122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f815ac50 0x7fe0f8070fb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009870 tx=0x7fe0e802b860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a868efee6121f29 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:15.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.144+0000 7fe0f5d9b640 1 -- 172.21.15.27:0/246197122 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e80317c0 con 0x7fe0f815ac50 2023-12-19T18:29:15.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.144+0000 7fe0f5d9b640 1 -- 172.21.15.27:0/246197122 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e8031920 con 0x7fe0f815ac50 2023-12-19T18:29:15.144 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.144+0000 7fe0f5d9b640 1 -- 172.21.15.27:0/246197122 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8031cb0 con 0x7fe0f815ac50 2023-12-19T18:29:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.145+0000 7fe0f759e640 1 -- 172.21.15.27:0/246197122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f815ac50 msgr2=0x7fe0f8070fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.145+0000 7fe0f759e640 1 --2- 172.21.15.27:0/246197122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f815ac50 0x7fe0f8070fb0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fe0e8009870 tx=0x7fe0e802b860 comp rx=0 tx=0).stop 2023-12-19T18:29:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.145+0000 7fe0f759e640 1 -- 172.21.15.27:0/246197122 shutdown_connections 2023-12-19T18:29:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.145+0000 7fe0f759e640 1 --2- 172.21.15.27:0/246197122 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0f80aef20 0x7fe0f806b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.145+0000 7fe0f759e640 1 --2- 172.21.15.27:0/246197122 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f815ac50 0x7fe0f8070fb0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.145+0000 7fe0f759e640 1 -- 172.21.15.27:0/246197122 >> 172.21.15.27:0/246197122 conn(0x7fe0f806a7d0 msgr2=0x7fe0f80ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.145+0000 7fe0f759e640 1 -- 172.21.15.27:0/246197122 shutdown_connections 2023-12-19T18:29:15.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.145+0000 7fe0f759e640 1 -- 172.21.15.27:0/246197122 wait complete. 2023-12-19T18:29:15.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.147+0000 7fe0f759e640 1 Processor -- start 2023-12-19T18:29:15.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.147+0000 7fe0f759e640 1 -- start start 2023-12-19T18:29:15.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.148+0000 7fe0f759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f80aef20 0x7fe0f813e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.148+0000 7fe0f759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f80af360 0x7fe0f813fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.148+0000 7fe0f759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0f815ac50 0x7fe0f80c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.148+0000 7fe0f759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f813edf0 con 0x7fe0f80aef20 2023-12-19T18:29:15.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.148+0000 7fe0f759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f81e3580 con 0x7fe0f80af360 2023-12-19T18:29:15.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.148+0000 7fe0f759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f8140530 con 0x7fe0f815ac50 2023-12-19T18:29:15.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.148+0000 7fe0f659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f80aef20 0x7fe0f813e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.148+0000 7fe0f659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f80aef20 0x7fe0f813e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55568/0 (socket says 172.21.15.27:55568) 2023-12-19T18:29:15.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.148+0000 7fe0f659c640 1 -- 172.21.15.27:0/470501684 learned_addr learned my addr 172.21.15.27:0/470501684 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.149+0000 7fe0effff640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f80af360 0x7fe0f813fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.149+0000 7fe0f6d9d640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0f815ac50 0x7fe0f80c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.149+0000 7fe0f659c640 1 -- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0f815ac50 msgr2=0x7fe0f80c7bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.149+0000 7fe0f659c640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0f815ac50 0x7fe0f80c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.149+0000 7fe0f659c640 1 -- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f80af360 msgr2=0x7fe0f813fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.149+0000 7fe0f659c640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f80af360 0x7fe0f813fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.149+0000 7fe0f659c640 1 -- 172.21.15.27:0/470501684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e0009400 con 0x7fe0f80aef20 2023-12-19T18:29:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.149+0000 7fe0effff640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f80af360 0x7fe0f813fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.149+0000 7fe0f659c640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f80aef20 0x7fe0f813e8b0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80090b0 tx=0x7fe0e80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.150+0000 7fe0eeffd640 1 -- 172.21.15.27:0/470501684 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8032860 con 0x7fe0f80aef20 2023-12-19T18:29:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.150+0000 7fe0eeffd640 1 -- 172.21.15.27:0/470501684 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0e80329c0 con 0x7fe0f80aef20 2023-12-19T18:29:15.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.150+0000 7fe0eeffd640 1 -- 172.21.15.27:0/470501684 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0e8032d70 con 0x7fe0f80aef20 2023-12-19T18:29:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.150+0000 7fe0f759e640 1 -- 172.21.15.27:0/470501684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0e80094d0 con 0x7fe0f80aef20 2023-12-19T18:29:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.150+0000 7fe0f759e640 1 -- 172.21.15.27:0/470501684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f8140af0 con 0x7fe0f80aef20 2023-12-19T18:29:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.151+0000 7fe0f759e640 1 -- 172.21.15.27:0/470501684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f804e840 con 0x7fe0f80aef20 2023-12-19T18:29:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.152+0000 7fe0eeffd640 1 -- 172.21.15.27:0/470501684 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fe0e804d3f0 con 0x7fe0f80aef20 2023-12-19T18:29:15.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.153+0000 7fe0eeffd640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe0d806a460 0x7fe0d806c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.153+0000 7fe0eeffd640 1 -- 172.21.15.27:0/470501684 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fe0e80aa570 con 0x7fe0f80aef20 2023-12-19T18:29:15.153 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.153+0000 7fe0effff640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe0d806a460 0x7fe0d806c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.154+0000 7fe0effff640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe0d806a460 0x7fe0d806c920 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe0f815de60 tx=0x7fe0e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:15.156 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.157+0000 7fe0eeffd640 1 -- 172.21.15.27:0/470501684 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fe0e8075ee0 con 0x7fe0f80aef20 2023-12-19T18:29:15.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.354+0000 7fe0f759e640 1 -- 172.21.15.27:0/470501684 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0f80aea00 con 0x7fe0d806a460 2023-12-19T18:29:15.355 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.355+0000 7fe0eeffd640 1 -- 172.21.15.27:0/470501684 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe0e00109a0 con 0x7fe0d806a460 2023-12-19T18:29:15.355 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 2 to deep-scrub 2023-12-19T18:29:15.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.357+0000 7fe0f759e640 1 -- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe0d806a460 msgr2=0x7fe0d806c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.357+0000 7fe0f759e640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe0d806a460 0x7fe0d806c920 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fe0f815de60 tx=0x7fe0e0008040 comp rx=0 tx=0).stop 2023-12-19T18:29:15.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.357+0000 7fe0f759e640 1 -- 172.21.15.27:0/470501684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f80aef20 msgr2=0x7fe0f813e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.357+0000 7fe0f759e640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f80aef20 0x7fe0f813e8b0 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7fe0e80090b0 tx=0x7fe0e80046c0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.357+0000 7fe0f759e640 1 -- 172.21.15.27:0/470501684 shutdown_connections 2023-12-19T18:29:15.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.357+0000 7fe0f759e640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fe0d806a460 0x7fe0d806c920 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.357+0000 7fe0f759e640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fe0f815ac50 0x7fe0f80c7bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.357+0000 7fe0f759e640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fe0f80af360 0x7fe0f813fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.357+0000 7fe0f759e640 1 --2- 172.21.15.27:0/470501684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f80aef20 0x7fe0f813e8b0 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.357+0000 7fe0f759e640 1 -- 172.21.15.27:0/470501684 >> 172.21.15.27:0/470501684 conn(0x7fe0f806a7d0 msgr2=0x7fe0f815bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:15.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.357+0000 7fe0f759e640 1 -- 172.21.15.27:0/470501684 shutdown_connections 2023-12-19T18:29:15.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.358+0000 7fe0f759e640 1 -- 172.21.15.27:0/470501684 wait complete. 2023-12-19T18:29:15.377 INFO:tasks.ceph:Scrubbing osd.3 2023-12-19T18:29:15.377 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:29:15.617 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.617+0000 7f3e2669e640 1 Processor -- start 2023-12-19T18:29:15.618 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.618+0000 7f3e2669e640 1 -- start start 2023-12-19T18:29:15.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.619+0000 7f3e2669e640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3e200b0a80 0x7f3e200ca510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.619 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.619+0000 7f3e2669e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e200cb1d0 con 0x7f3e2006b370 2023-12-19T18:29:15.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.619+0000 7f3e2669e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e200708c0 con 0x7f3e2015ab90 2023-12-19T18:29:15.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.619+0000 7f3e2669e640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e2006f500 con 0x7f3e200b0a80 2023-12-19T18:29:15.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.620+0000 7f3e24e9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3e200b0a80 0x7f3e200ca510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.620+0000 7f3e24e9b640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f3e200b0a80 0x7f3e200ca510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55578/0 (socket says 172.21.15.27:55578) 2023-12-19T18:29:15.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.620+0000 7f3e24e9b640 1 -- 172.21.15.27:0/3298001814 learned_addr learned my addr 172.21.15.27:0/3298001814 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:15.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.620+0000 7f3e24e9b640 1 -- 172.21.15.27:0/3298001814 >> v1:172.21.15.192:6789/0 conn(0x7f3e2015ab90 legacy=0x7f3e200b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.620 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.620+0000 7f3e24e9b640 1 -- 172.21.15.27:0/3298001814 >> v1:172.21.15.27:6789/0 conn(0x7f3e2006b370 legacy=0x7f3e200ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.620+0000 7f3e24e9b640 1 -- 172.21.15.27:0/3298001814 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e20070b60 con 0x7f3e200b0a80 2023-12-19T18:29:15.621 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.622+0000 7f3e24e9b640 1 --2- 172.21.15.27:0/3298001814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e200b0a80 0x7f3e200ca510 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009870 tx=0x7f3e1002bab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d88dc70325ec0f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:15.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.622+0000 7f3e17fff640 1 -- 172.21.15.27:0/3298001814 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1002d870 con 0x7f3e200b0a80 2023-12-19T18:29:15.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.622+0000 7f3e17fff640 1 -- 172.21.15.27:0/3298001814 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1002d9d0 con 0x7f3e200b0a80 2023-12-19T18:29:15.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.622+0000 7f3e17fff640 1 -- 172.21.15.27:0/3298001814 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1002dd60 con 0x7f3e200b0a80 2023-12-19T18:29:15.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.622+0000 7f3e2669e640 1 -- 172.21.15.27:0/3298001814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e200b0a80 msgr2=0x7f3e200ca510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.622 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.622+0000 7f3e2669e640 1 --2- 172.21.15.27:0/3298001814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e200b0a80 0x7f3e200ca510 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f3e10009870 tx=0x7f3e1002bab0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.623+0000 7f3e2669e640 1 -- 172.21.15.27:0/3298001814 shutdown_connections 2023-12-19T18:29:15.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.623+0000 7f3e2669e640 1 --2- 172.21.15.27:0/3298001814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e200b0a80 0x7f3e200ca510 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.623+0000 7f3e2669e640 1 -- 172.21.15.27:0/3298001814 >> 172.21.15.27:0/3298001814 conn(0x7f3e200cb370 msgr2=0x7f3e200cb750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:15.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.623+0000 7f3e2669e640 1 -- 172.21.15.27:0/3298001814 shutdown_connections 2023-12-19T18:29:15.623 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.623+0000 7f3e2669e640 1 -- 172.21.15.27:0/3298001814 wait complete. 2023-12-19T18:29:15.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.624+0000 7f3e2669e640 1 Processor -- start 2023-12-19T18:29:15.624 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.625+0000 7f3e2669e640 1 -- start start 2023-12-19T18:29:15.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e2669e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e2006b370 0x7f3e200c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e2669e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e200b0a80 0x7f3e201535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e2669e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e2015ab90 0x7f3e20153b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e2669e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e200c7f40 con 0x7f3e2015ab90 2023-12-19T18:29:15.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e2669e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e200c73c0 con 0x7f3e200b0a80 2023-12-19T18:29:15.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e2669e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e200c76a0 con 0x7f3e2006b370 2023-12-19T18:29:15.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e25e9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e2015ab90 0x7f3e20153b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e25e9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e2015ab90 0x7f3e20153b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55588/0 (socket says 172.21.15.27:55588) 2023-12-19T18:29:15.627 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e25e9d640 1 -- 172.21.15.27:0/1527335102 learned_addr learned my addr 172.21.15.27:0/1527335102 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e2569c640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e2006b370 0x7f3e200c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e24e9b640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e200b0a80 0x7f3e201535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e25e9d640 1 -- 172.21.15.27:0/1527335102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e2006b370 msgr2=0x7f3e200c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e25e9d640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e2006b370 0x7f3e200c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e25e9d640 1 -- 172.21.15.27:0/1527335102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e200b0a80 msgr2=0x7f3e201535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e25e9d640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e200b0a80 0x7f3e201535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e25e9d640 1 -- 172.21.15.27:0/1527335102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e100094d0 con 0x7f3e2015ab90 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e24e9b640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f3e200b0a80 0x7f3e201535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.626+0000 7f3e2569c640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f3e2006b370 0x7f3e200c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.627+0000 7f3e25e9d640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e2015ab90 0x7f3e20153b30 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800c7a0 tx=0x7f3e1800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.627+0000 7f3e15ffb640 1 -- 172.21.15.27:0/1527335102 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18003b70 con 0x7f3e2015ab90 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.627+0000 7f3e15ffb640 1 -- 172.21.15.27:0/1527335102 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e18003cf0 con 0x7f3e2015ab90 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.627+0000 7f3e15ffb640 1 -- 172.21.15.27:0/1527335102 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e18003ea0 con 0x7f3e2015ab90 2023-12-19T18:29:15.628 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.627+0000 7f3e2669e640 1 -- 172.21.15.27:0/1527335102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e200c7980 con 0x7f3e2015ab90 2023-12-19T18:29:15.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.627+0000 7f3e2669e640 1 -- 172.21.15.27:0/1527335102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e200c65b0 con 0x7f3e2015ab90 2023-12-19T18:29:15.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.628+0000 7f3e2669e640 1 -- 172.21.15.27:0/1527335102 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3e20071120 con 0x7f3e2015ab90 2023-12-19T18:29:15.629 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.629+0000 7f3e15ffb640 1 -- 172.21.15.27:0/1527335102 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f3e1800aa90 con 0x7f3e2015ab90 2023-12-19T18:29:15.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.630+0000 7f3e15ffb640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e0406a430 0x7f3e0406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.630+0000 7f3e15ffb640 1 -- 172.21.15.27:0/1527335102 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f3e180883f0 con 0x7f3e2015ab90 2023-12-19T18:29:15.630 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.631+0000 7f3e15ffb640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] conn(0x7f3e04070290 0x7f3e040726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.631+0000 7f3e15ffb640 1 -- 172.21.15.27:0/1527335102 --> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3e04072d60 con 0x7f3e04070290 2023-12-19T18:29:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.631+0000 7f3e15ffb640 1 -- 172.21.15.27:0/1527335102 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=210) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e1808a9d0 con 0x7f3e2015ab90 2023-12-19T18:29:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.631+0000 7f3e24e9b640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] conn(0x7f3e04070290 0x7f3e040726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.631+0000 7f3e2569c640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e0406a430 0x7f3e0406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.631 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.631+0000 7f3e24e9b640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] conn(0x7f3e04070290 0x7f3e040726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:15.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.632+0000 7f3e2569c640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e0406a430 0x7f3e0406c8f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c000f60 tx=0x7f3e1c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:15.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.632+0000 7f3e15ffb640 1 -- 172.21.15.27:0/1527335102 <== osd.3 v2:172.21.15.27:6816/84146 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f3e04072d60 con 0x7f3e04070290 2023-12-19T18:29:15.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.645+0000 7f3e2669e640 1 -- 172.21.15.27:0/1527335102 --> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3e20070e40 con 0x7f3e04070290 2023-12-19T18:29:15.645 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.646+0000 7f3e15ffb640 1 -- 172.21.15.27:0/1527335102 <== osd.3 v2:172.21.15.27:6816/84146 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f3e1002d810 con 0x7f3e04070290 2023-12-19T18:29:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:29:15.646 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-12-19T18:29:15.646 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:29:15.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.646+0000 7f3e2669e640 1 -- 172.21.15.27:0/1527335102 >> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] conn(0x7f3e04070290 msgr2=0x7f3e040726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.646+0000 7f3e2669e640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.27:6816/84146,v1:172.21.15.27:6817/84146] conn(0x7f3e04070290 0x7f3e040726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.646+0000 7f3e2669e640 1 -- 172.21.15.27:0/1527335102 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e0406a430 msgr2=0x7f3e0406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.646+0000 7f3e2669e640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f3e0406a430 0x7f3e0406c8f0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f3e1c000f60 tx=0x7f3e1c009040 comp rx=0 tx=0).stop 2023-12-19T18:29:15.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.646+0000 7f3e2669e640 1 -- 172.21.15.27:0/1527335102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e2015ab90 msgr2=0x7f3e20153b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.646+0000 7f3e2669e640 1 --2- 172.21.15.27:0/1527335102 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e2015ab90 0x7f3e20153b30 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f3e1800c7a0 tx=0x7f3e1800cc60 comp rx=0 tx=0).stop 2023-12-19T18:29:15.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.646+0000 7f3e25e9d640 1 -- 172.21.15.27:0/1527335102 reap_dead start 2023-12-19T18:29:15.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.647+0000 7f3e2669e640 1 -- 172.21.15.27:0/1527335102 shutdown_connections 2023-12-19T18:29:15.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.647+0000 7f3e2669e640 1 -- 172.21.15.27:0/1527335102 >> 172.21.15.27:0/1527335102 conn(0x7f3e200cb370 msgr2=0x7f3e200c8f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:15.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.647+0000 7f3e2669e640 1 -- 172.21.15.27:0/1527335102 shutdown_connections 2023-12-19T18:29:15.647 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.647+0000 7f3e2669e640 1 -- 172.21.15.27:0/1527335102 wait complete. 2023-12-19T18:29:15.668 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-12-19T18:29:15.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.894+0000 7fd8a9e07640 1 Processor -- start 2023-12-19T18:29:15.894 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.894+0000 7fd8a9e07640 1 -- start start 2023-12-19T18:29:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.895+0000 7fd8a9e07640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8a41540e0 0x7fd8a4160bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.895+0000 7fd8a9e07640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd8a406c6e0 0x7fd8a415dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.895+0000 7fd8a9e07640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8a415d600 con 0x7fd8a415e2e0 2023-12-19T18:29:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.895+0000 7fd8a9e07640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8a4161130 con 0x7fd8a406c6e0 2023-12-19T18:29:15.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.895+0000 7fd8a9e07640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a41613d0 con 0x7fd8a41540e0 2023-12-19T18:29:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.896+0000 7fd8a3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd8a406c6e0 0x7fd8a415dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.896+0000 7fd8a8e05640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8a41540e0 0x7fd8a4160bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.896+0000 7fd8a3fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fd8a406c6e0 0x7fd8a415dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:53284/0 (socket says 172.21.15.27:53284) 2023-12-19T18:29:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.896+0000 7fd8a3fff640 1 -- 172.21.15.27:0/1958368934 learned_addr learned my addr 172.21.15.27:0/1958368934 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.897+0000 7fd8a3fff640 1 -- 172.21.15.27:0/1958368934 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8a41540e0 msgr2=0x7fd8a4160bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.897+0000 7fd8a3fff640 1 --2- 172.21.15.27:0/1958368934 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8a41540e0 0x7fd8a4160bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.897+0000 7fd8a3fff640 1 -- 172.21.15.27:0/1958368934 >> v1:172.21.15.192:6789/0 conn(0x7fd8a415e2e0 legacy=0x7fd8a406b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.897 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.897+0000 7fd8a3fff640 1 -- 172.21.15.27:0/1958368934 --> v2:172.21.15.192:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8a41616b0 con 0x7fd8a406c6e0 2023-12-19T18:29:15.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.898+0000 7fd8a3fff640 1 --2- 172.21.15.27:0/1958368934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a406c6e0 0x7fd8a415dda0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009870 tx=0x7fd88c02b980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=daf831774d11e424 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.899+0000 7fd8a37fe640 1 -- 172.21.15.27:0/1958368934 <== mon.1 v2:172.21.15.192:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c030840 con 0x7fd8a406c6e0 2023-12-19T18:29:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.899+0000 7fd8a37fe640 1 -- 172.21.15.27:0/1958368934 <== mon.1 v2:172.21.15.192:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd88c0309a0 con 0x7fd8a406c6e0 2023-12-19T18:29:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.899+0000 7fd8a37fe640 1 -- 172.21.15.27:0/1958368934 <== mon.1 v2:172.21.15.192:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c030d30 con 0x7fd8a406c6e0 2023-12-19T18:29:15.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.899+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1958368934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a406c6e0 msgr2=0x7fd8a415dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.899+0000 7fd8a9e07640 1 --2- 172.21.15.27:0/1958368934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a406c6e0 0x7fd8a415dda0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd88c009870 tx=0x7fd88c02b980 comp rx=0 tx=0).stop 2023-12-19T18:29:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.900+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1958368934 shutdown_connections 2023-12-19T18:29:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.900+0000 7fd8a9e07640 1 --2- 172.21.15.27:0/1958368934 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a406c6e0 0x7fd8a415dda0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.900+0000 7fd8a9e07640 1 --2- 172.21.15.27:0/1958368934 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8a41540e0 0x7fd8a4160bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.900+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1958368934 >> 172.21.15.27:0/1958368934 conn(0x7fd8a406a7d0 msgr2=0x7fd8a40ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.900+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1958368934 shutdown_connections 2023-12-19T18:29:15.900 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.900+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1958368934 wait complete. 2023-12-19T18:29:15.901 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.901+0000 7fd8a9e07640 1 Processor -- start 2023-12-19T18:29:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.902+0000 7fd8a9e07640 1 -- start start 2023-12-19T18:29:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.902+0000 7fd8a9e07640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a406c6e0 0x7fd8a40af690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.902+0000 7fd8a9e07640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a41540e0 0x7fd8a40b0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.902+0000 7fd8a9e07640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8a415e2e0 0x7fd8a40cc430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.902+0000 7fd8a9e07640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a40b4600 con 0x7fd8a406c6e0 2023-12-19T18:29:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.903+0000 7fd8a9e07640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a40afbd0 con 0x7fd8a41540e0 2023-12-19T18:29:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.903+0000 7fd8a9e07640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a40b39f0 con 0x7fd8a415e2e0 2023-12-19T18:29:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.903+0000 7fd8a8e05640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a406c6e0 0x7fd8a40af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.903+0000 7fd8a8e05640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a406c6e0 0x7fd8a40af690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55592/0 (socket says 172.21.15.27:55592) 2023-12-19T18:29:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.903+0000 7fd8a8e05640 1 -- 172.21.15.27:0/1660733409 learned_addr learned my addr 172.21.15.27:0/1660733409 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.903+0000 7fd8a9606640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8a415e2e0 0x7fd8a40cc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.903+0000 7fd8a3fff640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a41540e0 0x7fd8a40b0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.903+0000 7fd8a8e05640 1 -- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8a415e2e0 msgr2=0x7fd8a40cc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.903+0000 7fd8a8e05640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8a415e2e0 0x7fd8a40cc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.903+0000 7fd8a8e05640 1 -- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a41540e0 msgr2=0x7fd8a40b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.903+0000 7fd8a8e05640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a41540e0 0x7fd8a40b0080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.903+0000 7fd8a8e05640 1 -- 172.21.15.27:0/1660733409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd88c0094d0 con 0x7fd8a406c6e0 2023-12-19T18:29:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.903+0000 7fd8a3fff640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a41540e0 0x7fd8a40b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.904+0000 7fd8a8e05640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a406c6e0 0x7fd8a40af690 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fd89400e870 tx=0x7fd89400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.904+0000 7fd8a17fa640 1 -- 172.21.15.27:0/1660733409 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8940094f0 con 0x7fd8a406c6e0 2023-12-19T18:29:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.904+0000 7fd8a17fa640 1 -- 172.21.15.27:0/1660733409 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89400c940 con 0x7fd8a406c6e0 2023-12-19T18:29:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.904+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1660733409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a40b3cd0 con 0x7fd8a406c6e0 2023-12-19T18:29:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.904+0000 7fd8a17fa640 1 -- 172.21.15.27:0/1660733409 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89400ccf0 con 0x7fd8a406c6e0 2023-12-19T18:29:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.904+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1660733409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a40b4290 con 0x7fd8a406c6e0 2023-12-19T18:29:15.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.905+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1660733409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a404e840 con 0x7fd8a406c6e0 2023-12-19T18:29:15.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.906+0000 7fd8a17fa640 1 -- 172.21.15.27:0/1660733409 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fd894013470 con 0x7fd8a406c6e0 2023-12-19T18:29:15.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.908+0000 7fd8a17fa640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd88406a460 0x7fd88406c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:15.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.908+0000 7fd8a17fa640 1 -- 172.21.15.27:0/1660733409 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fd894088760 con 0x7fd8a406c6e0 2023-12-19T18:29:15.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.908+0000 7fd8a3fff640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd88406a460 0x7fd88406c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:15.909 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.910+0000 7fd8a3fff640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd88406a460 0x7fd88406c920 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd88c036000 tx=0x7fd88c03b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:15.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:15.912+0000 7fd8a17fa640 1 -- 172.21.15.27:0/1660733409 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fd894053fb0 con 0x7fd8a406c6e0 2023-12-19T18:29:16.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.105+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1660733409 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8a415d880 con 0x7fd88406a460 2023-12-19T18:29:16.106 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.106+0000 7fd8a17fa640 1 -- 172.21.15.27:0/1660733409 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd88c030c10 con 0x7fd88406a460 2023-12-19T18:29:16.106 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 3 to deep-scrub 2023-12-19T18:29:16.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.108+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd88406a460 msgr2=0x7fd88406c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.108+0000 7fd8a9e07640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd88406a460 0x7fd88406c920 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd88c036000 tx=0x7fd88c03b040 comp rx=0 tx=0).stop 2023-12-19T18:29:16.108 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.108+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1660733409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a406c6e0 msgr2=0x7fd8a40af690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.108+0000 7fd8a9e07640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a406c6e0 0x7fd8a40af690 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fd89400e870 tx=0x7fd89400ed30 comp rx=0 tx=0).stop 2023-12-19T18:29:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.108+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1660733409 shutdown_connections 2023-12-19T18:29:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.108+0000 7fd8a9e07640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fd88406a460 0x7fd88406c920 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.108+0000 7fd8a9e07640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fd8a415e2e0 0x7fd8a40cc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.108+0000 7fd8a9e07640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fd8a41540e0 0x7fd8a40b0080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.108+0000 7fd8a9e07640 1 --2- 172.21.15.27:0/1660733409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8a406c6e0 0x7fd8a40af690 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.108+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1660733409 >> 172.21.15.27:0/1660733409 conn(0x7fd8a406a7d0 msgr2=0x7fd8a41534d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.109+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1660733409 shutdown_connections 2023-12-19T18:29:16.109 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.109+0000 7fd8a9e07640 1 -- 172.21.15.27:0/1660733409 wait complete. 2023-12-19T18:29:16.128 INFO:tasks.ceph:Scrubbing osd.4 2023-12-19T18:29:16.128 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:29:16.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.367+0000 7fadf4518640 1 Processor -- start 2023-12-19T18:29:16.367 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.368+0000 7fadf4518640 1 -- start start 2023-12-19T18:29:16.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.369+0000 7fadf4518640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadec15b4b0 0x7fadec0cb450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:16.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.369+0000 7fadf4518640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fadec15fa70 0x7fadec0cb990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:16.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.369+0000 7fadf4518640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadec070fb0 con 0x7fadec15b0d0 2023-12-19T18:29:16.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.369+0000 7fadf4518640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadec06fab0 con 0x7fadec15fa70 2023-12-19T18:29:16.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.369+0000 7fadf4518640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec06f4f0 con 0x7fadec15b4b0 2023-12-19T18:29:16.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.369+0000 7fadf22b4640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fadec15b0d0 0x7fadec0cabf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47592/0 (socket says 172.21.15.27:47592) 2023-12-19T18:29:16.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.369+0000 7fadf22b4640 1 -- 172.21.15.27:0/59023140 learned_addr learned my addr 172.21.15.27:0/59023140 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:16.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.369+0000 7fadf2ab5640 1 --2- 172.21.15.27:0/59023140 >> v2:172.21.15.27:3300/0 conn(0x7fadec15fa70 0x7fadec0cb990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:16.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.370+0000 7fadf1ab3640 1 --2- 172.21.15.27:0/59023140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadec15b4b0 0x7fadec0cb450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:16.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.370+0000 7fadf12b2640 1 -- 172.21.15.27:0/59023140 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536372100 0 0) 0x7fadec070fb0 con 0x7fadec15b0d0 2023-12-19T18:29:16.370 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.370+0000 7fadf12b2640 1 -- 172.21.15.27:0/59023140 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fadd8003520 con 0x7fadec15b0d0 2023-12-19T18:29:16.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.370+0000 7fadf2ab5640 1 -- 172.21.15.27:0/59023140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadec15b4b0 msgr2=0x7fadec0cb450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:16.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.370+0000 7fadf2ab5640 1 --2- 172.21.15.27:0/59023140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadec15b4b0 0x7fadec0cb450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.370+0000 7fadf2ab5640 1 -- 172.21.15.27:0/59023140 >> v1:172.21.15.27:6789/0 conn(0x7fadec15b0d0 legacy=0x7fadec0cabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.371 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.370+0000 7fadf2ab5640 1 -- 172.21.15.27:0/59023140 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadec06f7d0 con 0x7fadec15fa70 2023-12-19T18:29:16.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.371+0000 7fadf2ab5640 1 --2- 172.21.15.27:0/59023140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadec15fa70 0x7fadec0cb990 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fade4009d90 tx=0x7fade402bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f71ba4da135d049f server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:16.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.371+0000 7fadf12b2640 1 -- 172.21.15.27:0/59023140 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade402cce0 con 0x7fadec15fa70 2023-12-19T18:29:16.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.371+0000 7fadf12b2640 1 -- 172.21.15.27:0/59023140 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade402ce40 con 0x7fadec15fa70 2023-12-19T18:29:16.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.371+0000 7fadf12b2640 1 -- 172.21.15.27:0/59023140 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4034660 con 0x7fadec15fa70 2023-12-19T18:29:16.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.372+0000 7fadf4518640 1 -- 172.21.15.27:0/59023140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadec15fa70 msgr2=0x7fadec0cb990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.372+0000 7fadf4518640 1 --2- 172.21.15.27:0/59023140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadec15fa70 0x7fadec0cb990 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fade4009d90 tx=0x7fade402bf30 comp rx=0 tx=0).stop 2023-12-19T18:29:16.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.372+0000 7fadf4518640 1 -- 172.21.15.27:0/59023140 shutdown_connections 2023-12-19T18:29:16.372 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.372+0000 7fadf4518640 1 --2- 172.21.15.27:0/59023140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadec15fa70 0x7fadec0cb990 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.372+0000 7fadf4518640 1 --2- 172.21.15.27:0/59023140 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadec15b4b0 0x7fadec0cb450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.372+0000 7fadf4518640 1 -- 172.21.15.27:0/59023140 >> 172.21.15.27:0/59023140 conn(0x7fadec0b0620 msgr2=0x7fadec0b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:16.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.372+0000 7fadf4518640 1 -- 172.21.15.27:0/59023140 shutdown_connections 2023-12-19T18:29:16.373 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.372+0000 7fadf4518640 1 -- 172.21.15.27:0/59023140 wait complete. 2023-12-19T18:29:16.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.374+0000 7fadf4518640 1 Processor -- start 2023-12-19T18:29:16.374 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.374+0000 7fadf4518640 1 -- start start 2023-12-19T18:29:16.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.375+0000 7fadf4518640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadec15b0d0 0x7fadec0c7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:16.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.375+0000 7fadf4518640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadec15b4b0 0x7fadec1520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:16.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.375+0000 7fadf4518640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadec15fa70 0x7fadec15c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:16.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.375+0000 7fadf4518640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec15d8c0 con 0x7fadec15fa70 2023-12-19T18:29:16.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf4518640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadec15ccb0 con 0x7fadec15b4b0 2023-12-19T18:29:16.377 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf4518640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadec15cf90 con 0x7fadec15b0d0 2023-12-19T18:29:16.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf2ab5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadec15fa70 0x7fadec15c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:16.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf2ab5640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadec15fa70 0x7fadec15c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55610/0 (socket says 172.21.15.27:55610) 2023-12-19T18:29:16.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf2ab5640 1 -- 172.21.15.27:0/4066890289 learned_addr learned my addr 172.21.15.27:0/4066890289 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:16.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf1ab3640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadec15b4b0 0x7fadec1520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:16.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf22b4640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadec15b0d0 0x7fadec0c7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:16.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf2ab5640 1 -- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadec15b0d0 msgr2=0x7fadec0c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf2ab5640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadec15b0d0 0x7fadec0c7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.378 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf2ab5640 1 -- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadec15b4b0 msgr2=0x7fadec1520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf2ab5640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadec15b4b0 0x7fadec1520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf2ab5640 1 -- 172.21.15.27:0/4066890289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade4009a70 con 0x7fadec15fa70 2023-12-19T18:29:16.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf1ab3640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fadec15b4b0 0x7fadec1520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:16.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.376+0000 7fadf22b4640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fadec15b0d0 0x7fadec0c7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:16.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.377+0000 7fadf2ab5640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadec15fa70 0x7fadec15c460 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fade4009d90 tx=0x7fade4004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:16.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.377+0000 7fade2ffd640 1 -- 172.21.15.27:0/4066890289 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade403d600 con 0x7fadec15fa70 2023-12-19T18:29:16.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.377+0000 7fade2ffd640 1 -- 172.21.15.27:0/4066890289 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fade403d780 con 0x7fadec15fa70 2023-12-19T18:29:16.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.377+0000 7fade2ffd640 1 -- 172.21.15.27:0/4066890289 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade403db30 con 0x7fadec15fa70 2023-12-19T18:29:16.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.377+0000 7fadf4518640 1 -- 172.21.15.27:0/4066890289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadec15d270 con 0x7fadec15fa70 2023-12-19T18:29:16.379 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.377+0000 7fadf4518640 1 -- 172.21.15.27:0/4066890289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadec152620 con 0x7fadec15fa70 2023-12-19T18:29:16.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.378+0000 7fadf4518640 1 -- 172.21.15.27:0/4066890289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fadec0714c0 con 0x7fadec15fa70 2023-12-19T18:29:16.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.379+0000 7fade2ffd640 1 -- 172.21.15.27:0/4066890289 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fade403b040 con 0x7fadec15fa70 2023-12-19T18:29:16.380 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.380+0000 7fade2ffd640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fadd006a430 0x7fadd006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:16.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.380+0000 7fade2ffd640 1 -- 172.21.15.27:0/4066890289 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fade40aab30 con 0x7fadec15fa70 2023-12-19T18:29:16.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.381+0000 7fade2ffd640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] conn(0x7fadd0070290 0x7fadd00726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:16.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.381+0000 7fade2ffd640 1 -- 172.21.15.27:0/4066890289 --> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fadd0072d60 con 0x7fadd0070290 2023-12-19T18:29:16.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.381+0000 7fade2ffd640 1 -- 172.21.15.27:0/4066890289 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=210) v2 ==== 24+0+0 (secure 0 0 0) 0x7fade40af2c0 con 0x7fadec15fa70 2023-12-19T18:29:16.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.381+0000 7fadf22b4640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fadd006a430 0x7fadd006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:16.381 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.381+0000 7fadf1ab3640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] conn(0x7fadd0070290 0x7fadd00726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:16.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.383+0000 7fadf1ab3640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] conn(0x7fadd0070290 0x7fadd00726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:16.382 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.383+0000 7fadf22b4640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fadd006a430 0x7fadd006c8f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fade8007fd0 tx=0x7fade8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:16.384 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.384+0000 7fade2ffd640 1 -- 172.21.15.27:0/4066890289 <== osd.4 v2:172.21.15.192:6800/84205 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7fadd0072d60 con 0x7fadd0070290 2023-12-19T18:29:16.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.397+0000 7fadf4518640 1 -- 172.21.15.27:0/4066890289 --> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fadec0711e0 con 0x7fadd0070290 2023-12-19T18:29:16.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.398+0000 7fade2ffd640 1 -- 172.21.15.27:0/4066890289 <== osd.4 v2:172.21.15.192:6800/84205 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7faddc009550 con 0x7fadd0070290 2023-12-19T18:29:16.398 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:29:16.399 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-12-19T18:29:16.399 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:29:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.399+0000 7fadf4518640 1 -- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] conn(0x7fadd0070290 msgr2=0x7fadd00726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.399+0000 7fadf4518640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:6800/84205,v1:172.21.15.192:6801/84205] conn(0x7fadd0070290 0x7fadd00726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.399+0000 7fadf4518640 1 -- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fadd006a430 msgr2=0x7fadd006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.399+0000 7fadf4518640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fadd006a430 0x7fadd006c8f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fade8007fd0 tx=0x7fade8009040 comp rx=0 tx=0).stop 2023-12-19T18:29:16.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.399+0000 7fadf4518640 1 -- 172.21.15.27:0/4066890289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadec15fa70 msgr2=0x7fadec15c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.399+0000 7fadf4518640 1 --2- 172.21.15.27:0/4066890289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fadec15fa70 0x7fadec15c460 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fade4009d90 tx=0x7fade4004a40 comp rx=0 tx=0).stop 2023-12-19T18:29:16.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.399+0000 7fadf2ab5640 1 -- 172.21.15.27:0/4066890289 reap_dead start 2023-12-19T18:29:16.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.399+0000 7fadf4518640 1 -- 172.21.15.27:0/4066890289 shutdown_connections 2023-12-19T18:29:16.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.399+0000 7fadf4518640 1 -- 172.21.15.27:0/4066890289 >> 172.21.15.27:0/4066890289 conn(0x7fadec0b0620 msgr2=0x7fadec15edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:16.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.399+0000 7fadf4518640 1 -- 172.21.15.27:0/4066890289 shutdown_connections 2023-12-19T18:29:16.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.399+0000 7fadf4518640 1 -- 172.21.15.27:0/4066890289 wait complete. 2023-12-19T18:29:16.416 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-12-19T18:29:16.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.655+0000 7facf3111640 1 Processor -- start 2023-12-19T18:29:16.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.655+0000 7facf3111640 1 -- start start 2023-12-19T18:29:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.657+0000 7facf3111640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7facec070aa0 0x7facec070f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.657+0000 7facf3111640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec15b110 con 0x7facec154170 2023-12-19T18:29:16.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.657+0000 7facf3111640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7facec06fce0 con 0x7facec15a3a0 2023-12-19T18:29:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.657+0000 7facf3111640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7facec15af20 con 0x7facec070aa0 2023-12-19T18:29:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.658+0000 7facf210f640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7facec154170 0x7facec070170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47598/0 (socket says 172.21.15.27:47598) 2023-12-19T18:29:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.658+0000 7facf210f640 1 -- 172.21.15.27:0/182087725 learned_addr learned my addr 172.21.15.27:0/182087725 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.658+0000 7facf190e640 1 --2- 172.21.15.27:0/182087725 >> v2:172.21.15.27:3300/0 conn(0x7facec070aa0 0x7facec070f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.658+0000 7facf190e640 1 -- 172.21.15.27:0/182087725 >> v1:172.21.15.192:6789/0 conn(0x7facec15a3a0 legacy=0x7facec15a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.658+0000 7facf190e640 1 -- 172.21.15.27:0/182087725 >> v1:172.21.15.27:6789/0 conn(0x7facec154170 legacy=0x7facec070170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.658+0000 7facf190e640 1 -- 172.21.15.27:0/182087725 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facec070880 con 0x7facec070aa0 2023-12-19T18:29:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.659+0000 7facf190e640 1 --2- 172.21.15.27:0/182087725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facec070aa0 0x7facec070f30 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097a0 tx=0x7facdc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edddcc5541c3916 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:16.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.660+0000 7facf110d640 1 -- 172.21.15.27:0/182087725 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc02e070 con 0x7facec070aa0 2023-12-19T18:29:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.660+0000 7facf110d640 1 -- 172.21.15.27:0/182087725 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facdc0315a0 con 0x7facec070aa0 2023-12-19T18:29:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.660+0000 7facf110d640 1 -- 172.21.15.27:0/182087725 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facdc031930 con 0x7facec070aa0 2023-12-19T18:29:16.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.661+0000 7facf3111640 1 -- 172.21.15.27:0/182087725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facec070aa0 msgr2=0x7facec070f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.661+0000 7facf3111640 1 --2- 172.21.15.27:0/182087725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facec070aa0 0x7facec070f30 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7facdc0097a0 tx=0x7facdc02b840 comp rx=0 tx=0).stop 2023-12-19T18:29:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.661+0000 7facf3111640 1 -- 172.21.15.27:0/182087725 shutdown_connections 2023-12-19T18:29:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.661+0000 7facf3111640 1 --2- 172.21.15.27:0/182087725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facec070aa0 0x7facec070f30 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.661+0000 7facf3111640 1 -- 172.21.15.27:0/182087725 >> 172.21.15.27:0/182087725 conn(0x7facec06a7d0 msgr2=0x7facec0ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.661+0000 7facf3111640 1 -- 172.21.15.27:0/182087725 shutdown_connections 2023-12-19T18:29:16.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.661+0000 7facf3111640 1 -- 172.21.15.27:0/182087725 wait complete. 2023-12-19T18:29:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.663+0000 7facf3111640 1 Processor -- start 2023-12-19T18:29:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.664+0000 7facf3111640 1 -- start start 2023-12-19T18:29:16.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.665+0000 7facf3111640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facec070aa0 0x7facec145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.665+0000 7facf3111640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facec154170 0x7facec1460a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.665+0000 7facf3111640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facec15a3a0 0x7facec1580a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.665+0000 7facf3111640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec155bf0 con 0x7facec070aa0 2023-12-19T18:29:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.665+0000 7facf3111640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7facec159720 con 0x7facec154170 2023-12-19T18:29:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.665+0000 7facf3111640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7facec159160 con 0x7facec15a3a0 2023-12-19T18:29:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.665+0000 7facf210f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facec070aa0 0x7facec145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.665+0000 7facf210f640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facec070aa0 0x7facec145b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55630/0 (socket says 172.21.15.27:55630) 2023-12-19T18:29:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.665+0000 7facf210f640 1 -- 172.21.15.27:0/630321719 learned_addr learned my addr 172.21.15.27:0/630321719 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.666+0000 7facf190e640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facec154170 0x7facec1460a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.666+0000 7facf210f640 1 -- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facec15a3a0 msgr2=0x7facec1580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.666+0000 7facf210f640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facec15a3a0 0x7facec1580a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.666+0000 7facf210f640 1 -- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facec154170 msgr2=0x7facec1460a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.666+0000 7facf210f640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facec154170 0x7facec1460a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.666+0000 7facf210f640 1 -- 172.21.15.27:0/630321719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facdc009400 con 0x7facec070aa0 2023-12-19T18:29:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.666+0000 7facf190e640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facec154170 0x7facec1460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.666+0000 7facf210f640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facec070aa0 0x7facec145b60 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7face800c4a0 tx=0x7face800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.666+0000 7face2ffd640 1 -- 172.21.15.27:0/630321719 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8005ce0 con 0x7facec070aa0 2023-12-19T18:29:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.666+0000 7face2ffd640 1 -- 172.21.15.27:0/630321719 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face8005e40 con 0x7facec070aa0 2023-12-19T18:29:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.666+0000 7face2ffd640 1 -- 172.21.15.27:0/630321719 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face8007270 con 0x7facec070aa0 2023-12-19T18:29:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.666+0000 7facf3111640 1 -- 172.21.15.27:0/630321719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facec159440 con 0x7facec070aa0 2023-12-19T18:29:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.667+0000 7facf3111640 1 -- 172.21.15.27:0/630321719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facec154f40 con 0x7facec070aa0 2023-12-19T18:29:16.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.668+0000 7facf3111640 1 -- 172.21.15.27:0/630321719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7facec155d30 con 0x7facec070aa0 2023-12-19T18:29:16.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.669+0000 7face2ffd640 1 -- 172.21.15.27:0/630321719 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7face8007490 con 0x7facec070aa0 2023-12-19T18:29:16.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.670+0000 7face2ffd640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7facd006a430 0x7facd006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:16.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.670+0000 7face2ffd640 1 -- 172.21.15.27:0/630321719 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7face8087ac0 con 0x7facec070aa0 2023-12-19T18:29:16.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.671+0000 7facf190e640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7facd006a430 0x7facd006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:16.671 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.672+0000 7facf190e640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7facd006a430 0x7facd006c8f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7facdc036000 tx=0x7facdc03d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:16.674 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.674+0000 7face2ffd640 1 -- 172.21.15.27:0/630321719 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7facec155d30 con 0x7facec070aa0 2023-12-19T18:29:16.875 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.875+0000 7facf3111640 1 -- 172.21.15.27:0/630321719 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7facec06f690 con 0x7facd006a430 2023-12-19T18:29:16.876 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.876+0000 7face2ffd640 1 -- 172.21.15.27:0/630321719 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7facdc031bc0 con 0x7facd006a430 2023-12-19T18:29:16.876 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 4 to deep-scrub 2023-12-19T18:29:16.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.879+0000 7facf3111640 1 -- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7facd006a430 msgr2=0x7facd006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.879+0000 7facf3111640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7facd006a430 0x7facd006c8f0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7facdc036000 tx=0x7facdc03d040 comp rx=0 tx=0).stop 2023-12-19T18:29:16.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.879+0000 7facf3111640 1 -- 172.21.15.27:0/630321719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facec070aa0 msgr2=0x7facec145b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:16.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.879+0000 7facf3111640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facec070aa0 0x7facec145b60 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7face800c4a0 tx=0x7face800c960 comp rx=0 tx=0).stop 2023-12-19T18:29:16.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.879+0000 7facf3111640 1 -- 172.21.15.27:0/630321719 shutdown_connections 2023-12-19T18:29:16.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.879+0000 7facf3111640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7facd006a430 0x7facd006c8f0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.879+0000 7facf3111640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7facec15a3a0 0x7facec1580a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.879 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.879+0000 7facf3111640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7facec154170 0x7facec1460a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.879+0000 7facf3111640 1 --2- 172.21.15.27:0/630321719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7facec070aa0 0x7facec145b60 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:16.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.879+0000 7facf3111640 1 -- 172.21.15.27:0/630321719 >> 172.21.15.27:0/630321719 conn(0x7facec06a7d0 msgr2=0x7facec149a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:16.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.879+0000 7facf3111640 1 -- 172.21.15.27:0/630321719 shutdown_connections 2023-12-19T18:29:16.880 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:16.879+0000 7facf3111640 1 -- 172.21.15.27:0/630321719 wait complete. 2023-12-19T18:29:16.898 INFO:tasks.ceph:Scrubbing osd.5 2023-12-19T18:29:16.898 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:29:17.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.133+0000 7f97a7fff640 1 Processor -- start 2023-12-19T18:29:17.134 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.134+0000 7f97a7fff640 1 -- start start 2023-12-19T18:29:17.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.135+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a815cfe0 0x7f97a80d3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.135+0000 7f97a7fff640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f97a80d0fd0 0x7f97a80d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.135+0000 7f97a7fff640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97a80ceb90 con 0x7f97a80cf730 2023-12-19T18:29:17.135 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.136+0000 7f97a7fff640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97a80d1990 con 0x7f97a80d0fd0 2023-12-19T18:29:17.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.136+0000 7f97a7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80d0c80 con 0x7f97a815cfe0 2023-12-19T18:29:17.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.136+0000 7f97a77fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f97a80cf730 0x7f97a80cfbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47600/0 (socket says 172.21.15.27:47600) 2023-12-19T18:29:17.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.136+0000 7f97a77fe640 1 -- 172.21.15.27:0/1801018273 learned_addr learned my addr 172.21.15.27:0/1801018273 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:17.136 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.136+0000 7f979ffff640 1 --2- 172.21.15.27:0/1801018273 >> v2:172.21.15.192:3300/0 conn(0x7f97a80d0fd0 0x7f97a80d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.136+0000 7f97a6ffd640 1 --2- 172.21.15.27:0/1801018273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a815cfe0 0x7f97a80d3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1373184730 0 0) 0x7f97a80ceb90 con 0x7f97a80cf730 2023-12-19T18:29:17.137 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9794003520 con 0x7f97a80cf730 2023-12-19T18:29:17.138 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1829039117 0 0) 0x7f9794003520 con 0x7f97a80cf730 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a815cfe0 msgr2=0x7f97a80d3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 --2- 172.21.15.27:0/1801018273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a815cfe0 0x7f97a80d3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 >> v2:172.21.15.192:3300/0 conn(0x7f97a80d0fd0 msgr2=0x7f97a80d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 --2- 172.21.15.27:0/1801018273 >> v2:172.21.15.192:3300/0 conn(0x7f97a80d0fd0 0x7f97a80d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a80ce2f0 con 0x7f97a80cf730 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f97a00019f0 con 0x7f97a80cf730 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 >> v1:172.21.15.27:6789/0 conn(0x7f97a80cf730 legacy=0x7f97a80cfbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 --2- 172.21.15.27:0/1801018273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9794003970 0x7f9794003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 --2- 172.21.15.27:0/1801018273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97940047c0 0x7f9794004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 --2- 172.21.15.27:0/1801018273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a815cfe0 0x7f9794005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80d1990 con 0x7f9794003970 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a80d0c80 con 0x7f97940047c0 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.137+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f979400b980 con 0x7f97a815cfe0 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.138+0000 7f97a6ffd640 1 --2- 172.21.15.27:0/1801018273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9794003970 0x7f9794003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.139 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.138+0000 7f979ffff640 1 --2- 172.21.15.27:0/1801018273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97940047c0 0x7f9794004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.138+0000 7f97a6ffd640 1 -- 172.21.15.27:0/1801018273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a815cfe0 msgr2=0x7f9794005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:17.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.138+0000 7f97a77fe640 1 --2- 172.21.15.27:0/1801018273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a815cfe0 0x7f9794005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.138+0000 7f97a6ffd640 1 --2- 172.21.15.27:0/1801018273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a815cfe0 0x7f9794005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.138+0000 7f97a6ffd640 1 -- 172.21.15.27:0/1801018273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97940047c0 msgr2=0x7f9794004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:17.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.138+0000 7f97a6ffd640 1 --2- 172.21.15.27:0/1801018273 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97940047c0 0x7f9794004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.138+0000 7f97a6ffd640 1 -- 172.21.15.27:0/1801018273 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9794003520 con 0x7f9794003970 2023-12-19T18:29:17.140 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.138+0000 7f97a77fe640 1 --2- 172.21.15.27:0/1801018273 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f97a815cfe0 0x7f9794005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:17.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.139+0000 7f97a6ffd640 1 --2- 172.21.15.27:0/1801018273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9794003970 0x7f9794003d90 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f9798009ac0 tx=0x7f97940057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44f99535a7ad2c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:17.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.140+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979802c6b0 con 0x7f9794003970 2023-12-19T18:29:17.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.140+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979802c810 con 0x7f9794003970 2023-12-19T18:29:17.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.140+0000 7f97a67fc640 1 -- 172.21.15.27:0/1801018273 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979802cba0 con 0x7f9794003970 2023-12-19T18:29:17.141 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.140+0000 7f97a7fff640 1 -- 172.21.15.27:0/1801018273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9794003970 msgr2=0x7f9794003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.140+0000 7f97a7fff640 1 --2- 172.21.15.27:0/1801018273 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9794003970 0x7f9794003d90 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7f9798009ac0 tx=0x7f97940057f0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.141+0000 7f97a77fe640 1 -- 172.21.15.27:0/1801018273 reap_dead start 2023-12-19T18:29:17.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.141+0000 7f97a7fff640 1 -- 172.21.15.27:0/1801018273 shutdown_connections 2023-12-19T18:29:17.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.141+0000 7f97a7fff640 1 -- 172.21.15.27:0/1801018273 >> 172.21.15.27:0/1801018273 conn(0x7f97a80d2d40 msgr2=0x7f97a80d3150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:17.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.141+0000 7f97a7fff640 1 -- 172.21.15.27:0/1801018273 shutdown_connections 2023-12-19T18:29:17.142 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.141+0000 7f97a7fff640 1 -- 172.21.15.27:0/1801018273 wait complete. 2023-12-19T18:29:17.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.143+0000 7f97a7fff640 1 Processor -- start 2023-12-19T18:29:17.143 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.143+0000 7f97a7fff640 1 -- start start 2023-12-19T18:29:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.145+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9794003970 0x7f97a815a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.145+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97a81f5db0 0x7f97a81f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.145+0000 7f97a7fff640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a806b2c0 0x7f97a806b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.145+0000 7f97a7fff640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a815e890 con 0x7f97a81f5db0 2023-12-19T18:29:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.145+0000 7f97a7fff640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97a815aa00 con 0x7f97a806b2c0 2023-12-19T18:29:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.145+0000 7f97a7fff640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a806bc80 con 0x7f9794003970 2023-12-19T18:29:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.145+0000 7f979ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97a81f5db0 0x7f97a81f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.145+0000 7f979ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97a81f5db0 0x7f97a81f6230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55658/0 (socket says 172.21.15.27:55658) 2023-12-19T18:29:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.145+0000 7f979ffff640 1 -- 172.21.15.27:0/4017582850 learned_addr learned my addr 172.21.15.27:0/4017582850 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.145+0000 7f97a6ffd640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9794003970 0x7f97a815a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.145+0000 7f97a77fe640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a806b2c0 0x7f97a806b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.146+0000 7f979ffff640 1 -- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9794003970 msgr2=0x7f97a815a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.146+0000 7f979ffff640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9794003970 0x7f97a815a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.146+0000 7f979ffff640 1 -- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a806b2c0 msgr2=0x7f97a806b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.146+0000 7f979ffff640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f97a806b2c0 0x7f97a806b740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.146+0000 7f979ffff640 1 -- 172.21.15.27:0/4017582850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a81f2820 con 0x7f97a81f5db0 2023-12-19T18:29:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.146+0000 7f97a6ffd640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9794003970 0x7f97a815a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-19T18:29:17.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.146+0000 7f979ffff640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97a81f5db0 0x7f97a81f6230 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f97900116d0 tx=0x7f9790011b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:17.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.146+0000 7f979f7fe640 1 -- 172.21.15.27:0/4017582850 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979000f960 con 0x7f97a81f5db0 2023-12-19T18:29:17.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.146+0000 7f979f7fe640 1 -- 172.21.15.27:0/4017582850 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979000fac0 con 0x7f97a81f5db0 2023-12-19T18:29:17.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.146+0000 7f979f7fe640 1 -- 172.21.15.27:0/4017582850 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979000fe30 con 0x7f97a81f5db0 2023-12-19T18:29:17.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.146+0000 7f97a7fff640 1 -- 172.21.15.27:0/4017582850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97a81f6870 con 0x7f97a81f5db0 2023-12-19T18:29:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.147+0000 7f97a7fff640 1 -- 172.21.15.27:0/4017582850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97a815eaa0 con 0x7f97a81f5db0 2023-12-19T18:29:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.148+0000 7f97a7fff640 1 -- 172.21.15.27:0/4017582850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f97a80ce8b0 con 0x7f97a81f5db0 2023-12-19T18:29:17.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.149+0000 7f979f7fe640 1 -- 172.21.15.27:0/4017582850 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9790015460 con 0x7f97a81f5db0 2023-12-19T18:29:17.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.150+0000 7f979f7fe640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f978806a430 0x7f978806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.150+0000 7f979f7fe640 1 -- 172.21.15.27:0/4017582850 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9790088a80 con 0x7f97a81f5db0 2023-12-19T18:29:17.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.151+0000 7f979f7fe640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] conn(0x7f9788070290 0x7f97880726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.151+0000 7f979f7fe640 1 -- 172.21.15.27:0/4017582850 --> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9788072d60 con 0x7f9788070290 2023-12-19T18:29:17.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.151+0000 7f979f7fe640 1 -- 172.21.15.27:0/4017582850 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=210) v2 ==== 24+0+0 (secure 0 0 0) 0x7f979008d2a0 con 0x7f97a81f5db0 2023-12-19T18:29:17.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.151+0000 7f97a6ffd640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f978806a430 0x7f978806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.151 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.151+0000 7f97a77fe640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] conn(0x7f9788070290 0x7f97880726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.152+0000 7f97a6ffd640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f978806a430 0x7f978806c8f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9798036000 tx=0x7f9798030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:17.152 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.153+0000 7f97a77fe640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] conn(0x7f9788070290 0x7f97880726b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:17.154 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.155+0000 7f979f7fe640 1 -- 172.21.15.27:0/4017582850 <== osd.5 v2:172.21.15.192:6808/84206 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f9788072d60 con 0x7f9788070290 2023-12-19T18:29:17.167 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.167+0000 7f97a7fff640 1 -- 172.21.15.27:0/4017582850 --> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f97a80ce5d0 con 0x7f9788070290 2023-12-19T18:29:17.168 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.169+0000 7f979f7fe640 1 -- 172.21.15.27:0/4017582850 <== osd.5 v2:172.21.15.192:6808/84206 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f97a000d040 con 0x7f9788070290 2023-12-19T18:29:17.169 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:29:17.169 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-12-19T18:29:17.169 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:29:17.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.169+0000 7f97a7fff640 1 -- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] conn(0x7f9788070290 msgr2=0x7f97880726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.169+0000 7f97a7fff640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:6808/84206,v1:172.21.15.192:6809/84206] conn(0x7f9788070290 0x7f97880726b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.169+0000 7f97a7fff640 1 -- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f978806a430 msgr2=0x7f978806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.169+0000 7f97a7fff640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f978806a430 0x7f978806c8f0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f9798036000 tx=0x7f9798030000 comp rx=0 tx=0).stop 2023-12-19T18:29:17.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.169+0000 7f97a7fff640 1 -- 172.21.15.27:0/4017582850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97a81f5db0 msgr2=0x7f97a81f6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.169+0000 7f97a7fff640 1 --2- 172.21.15.27:0/4017582850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97a81f5db0 0x7f97a81f6230 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7f97900116d0 tx=0x7f9790011b90 comp rx=0 tx=0).stop 2023-12-19T18:29:17.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.169+0000 7f97a77fe640 1 -- 172.21.15.27:0/4017582850 reap_dead start 2023-12-19T18:29:17.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.170+0000 7f97a7fff640 1 -- 172.21.15.27:0/4017582850 shutdown_connections 2023-12-19T18:29:17.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.170+0000 7f97a7fff640 1 -- 172.21.15.27:0/4017582850 >> 172.21.15.27:0/4017582850 conn(0x7f97a80d2d40 msgr2=0x7f97a80b0b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:17.170 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.170+0000 7f97a7fff640 1 -- 172.21.15.27:0/4017582850 shutdown_connections 2023-12-19T18:29:17.171 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.170+0000 7f97a7fff640 1 -- 172.21.15.27:0/4017582850 wait complete. 2023-12-19T18:29:17.187 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-12-19T18:29:17.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.423+0000 7f683c8c3640 1 Processor -- start 2023-12-19T18:29:17.423 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.423+0000 7f683c8c3640 1 -- start start 2023-12-19T18:29:17.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.424+0000 7f683c8c3640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6838070fb0 0x7f683806b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.424+0000 7f683c8c3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f683806b850 0x7f683806f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.424+0000 7f683c8c3640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68380adf10 con 0x7f683815ac50 2023-12-19T18:29:17.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.424+0000 7f683c8c3640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68380adac0 con 0x7f6838070fb0 2023-12-19T18:29:17.424 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.424+0000 7f683c8c3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68380af630 con 0x7f683806b850 2023-12-19T18:29:17.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.425+0000 7f683659c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6838070fb0 0x7f683806b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.425+0000 7f683759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f683806b850 0x7f683806f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.425+0000 7f683659c640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f6838070fb0 0x7f683806b310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.192:3300/0 says I am v2:172.21.15.27:53330/0 (socket says 172.21.15.27:53330) 2023-12-19T18:29:17.425 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.425+0000 7f683659c640 1 -- 172.21.15.27:0/1518091791 learned_addr learned my addr 172.21.15.27:0/1518091791 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:17.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.426+0000 7f683759e640 1 -- 172.21.15.27:0/1518091791 >> v2:172.21.15.192:3300/0 conn(0x7f6838070fb0 msgr2=0x7f683806b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:17.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.426+0000 7f683759e640 1 --2- 172.21.15.27:0/1518091791 >> v2:172.21.15.192:3300/0 conn(0x7f6838070fb0 0x7f683806b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.426+0000 7f683759e640 1 -- 172.21.15.27:0/1518091791 >> v1:172.21.15.192:6789/0 conn(0x7f683815ac50 legacy=0x7f68380aef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.426 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.426+0000 7f683759e640 1 -- 172.21.15.27:0/1518091791 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68380713f0 con 0x7f683806b850 2023-12-19T18:29:17.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.427+0000 7f683759e640 1 --2- 172.21.15.27:0/1518091791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f683806b850 0x7f683806f650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f68300097a0 tx=0x7f683002b7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c71950e381d9ba6 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:17.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.428+0000 7f6835d9b640 1 -- 172.21.15.27:0/1518091791 <== mon.2 v2:172.21.15.192:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f683002f050 con 0x7f683806b850 2023-12-19T18:29:17.427 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.428+0000 7f6835d9b640 1 -- 172.21.15.27:0/1518091791 <== mon.2 v2:172.21.15.192:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6830031700 con 0x7f683806b850 2023-12-19T18:29:17.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.428+0000 7f6835d9b640 1 -- 172.21.15.27:0/1518091791 <== mon.2 v2:172.21.15.192:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6830031a90 con 0x7f683806b850 2023-12-19T18:29:17.428 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.428+0000 7f683c8c3640 1 -- 172.21.15.27:0/1518091791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f683806b850 msgr2=0x7f683806f650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.428+0000 7f683c8c3640 1 --2- 172.21.15.27:0/1518091791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f683806b850 0x7f683806f650 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f68300097a0 tx=0x7f683002b7a0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.428+0000 7f683c8c3640 1 -- 172.21.15.27:0/1518091791 shutdown_connections 2023-12-19T18:29:17.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.428+0000 7f683c8c3640 1 --2- 172.21.15.27:0/1518091791 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f683806b850 0x7f683806f650 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.428+0000 7f683c8c3640 1 --2- 172.21.15.27:0/1518091791 >> v2:172.21.15.192:3300/0 conn(0x7f6838070fb0 0x7f683806b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.428+0000 7f683c8c3640 1 -- 172.21.15.27:0/1518091791 >> 172.21.15.27:0/1518091791 conn(0x7f683806a7d0 msgr2=0x7f68380ce8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:17.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.428+0000 7f683c8c3640 1 -- 172.21.15.27:0/1518091791 shutdown_connections 2023-12-19T18:29:17.429 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.429+0000 7f683c8c3640 1 -- 172.21.15.27:0/1518091791 wait complete. 2023-12-19T18:29:17.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.430+0000 7f683c8c3640 1 Processor -- start 2023-12-19T18:29:17.430 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.430+0000 7f683c8c3640 1 -- start start 2023-12-19T18:29:17.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.431+0000 7f683c8c3640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f683806b850 0x7f683814c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.431+0000 7f683c8c3640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838070fb0 0x7f683814fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.431 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.431+0000 7f683c8c3640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f683815ac50 0x7f68380c8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.431+0000 7f683c8c3640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683814f2c0 con 0x7f6838070fb0 2023-12-19T18:29:17.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.431+0000 7f683c8c3640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f683814c650 con 0x7f683806b850 2023-12-19T18:29:17.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.431+0000 7f683c8c3640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6838150bf0 con 0x7f683815ac50 2023-12-19T18:29:17.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.432+0000 7f683659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838070fb0 0x7f683814fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.432+0000 7f683659c640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838070fb0 0x7f683814fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55666/0 (socket says 172.21.15.27:55666) 2023-12-19T18:29:17.432 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.432+0000 7f683659c640 1 -- 172.21.15.27:0/701278216 learned_addr learned my addr 172.21.15.27:0/701278216 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.432+0000 7f683759e640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f683815ac50 0x7f68380c8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.432+0000 7f6836d9d640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f683806b850 0x7f683814c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.432+0000 7f683659c640 1 -- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f683815ac50 msgr2=0x7f68380c8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.432+0000 7f683659c640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f683815ac50 0x7f68380c8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.432+0000 7f683659c640 1 -- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f683806b850 msgr2=0x7f683814c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.432+0000 7f683659c640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f683806b850 0x7f683814c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.433 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.432+0000 7f683659c640 1 -- 172.21.15.27:0/701278216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6830009400 con 0x7f6838070fb0 2023-12-19T18:29:17.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.433+0000 7f683659c640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838070fb0 0x7f683814fe80 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f682c009fd0 tx=0x7f682c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:17.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.433+0000 7f681f7fe640 1 -- 172.21.15.27:0/701278216 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c00c940 con 0x7f6838070fb0 2023-12-19T18:29:17.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.433+0000 7f681f7fe640 1 -- 172.21.15.27:0/701278216 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f682c00caa0 con 0x7f6838070fb0 2023-12-19T18:29:17.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.433+0000 7f681f7fe640 1 -- 172.21.15.27:0/701278216 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c00ce30 con 0x7f6838070fb0 2023-12-19T18:29:17.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.433+0000 7f683c8c3640 1 -- 172.21.15.27:0/701278216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f683814ec70 con 0x7f6838070fb0 2023-12-19T18:29:17.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.433+0000 7f683c8c3640 1 -- 172.21.15.27:0/701278216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6838150ed0 con 0x7f6838070fb0 2023-12-19T18:29:17.434 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.434+0000 7f683c8c3640 1 -- 172.21.15.27:0/701278216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f683804e840 con 0x7f6838070fb0 2023-12-19T18:29:17.435 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.435+0000 7f681f7fe640 1 -- 172.21.15.27:0/701278216 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f682c013470 con 0x7f6838070fb0 2023-12-19T18:29:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.436+0000 7f681f7fe640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f682406a430 0x7f682406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.436+0000 7f681f7fe640 1 -- 172.21.15.27:0/701278216 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f682c088800 con 0x7f6838070fb0 2023-12-19T18:29:17.436 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.437+0000 7f6836d9d640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f682406a430 0x7f682406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.437 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.438+0000 7f6836d9d640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f682406a430 0x7f682406c8f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f68280012e0 tx=0x7f6828009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:17.440 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.441+0000 7f681f7fe640 1 -- 172.21.15.27:0/701278216 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f682c054050 con 0x7f6838070fb0 2023-12-19T18:29:17.632 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.632+0000 7f683c8c3640 1 -- 172.21.15.27:0/701278216 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f68380ae440 con 0x7f682406a430 2023-12-19T18:29:17.633 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.633+0000 7f681f7fe640 1 -- 172.21.15.27:0/701278216 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6828001ab0 con 0x7f682406a430 2023-12-19T18:29:17.634 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 5 to deep-scrub 2023-12-19T18:29:17.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.636+0000 7f683c8c3640 1 -- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f682406a430 msgr2=0x7f682406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.636+0000 7f683c8c3640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f682406a430 0x7f682406c8f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f68280012e0 tx=0x7f6828009040 comp rx=0 tx=0).stop 2023-12-19T18:29:17.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.636+0000 7f683c8c3640 1 -- 172.21.15.27:0/701278216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838070fb0 msgr2=0x7f683814fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.636+0000 7f683c8c3640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838070fb0 0x7f683814fe80 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f682c009fd0 tx=0x7f682c00ed40 comp rx=0 tx=0).stop 2023-12-19T18:29:17.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.636+0000 7f683c8c3640 1 -- 172.21.15.27:0/701278216 shutdown_connections 2023-12-19T18:29:17.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.636+0000 7f683c8c3640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f682406a430 0x7f682406c8f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.636+0000 7f683c8c3640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f683815ac50 0x7f68380c8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.636 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.636+0000 7f683c8c3640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6838070fb0 0x7f683814fe80 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.636+0000 7f683c8c3640 1 --2- 172.21.15.27:0/701278216 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f683806b850 0x7f683814c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.636+0000 7f683c8c3640 1 -- 172.21.15.27:0/701278216 >> 172.21.15.27:0/701278216 conn(0x7f683806a7d0 msgr2=0x7f68380cdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:17.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.636+0000 7f683c8c3640 1 -- 172.21.15.27:0/701278216 shutdown_connections 2023-12-19T18:29:17.637 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.636+0000 7f683c8c3640 1 -- 172.21.15.27:0/701278216 wait complete. 2023-12-19T18:29:17.657 INFO:tasks.ceph:Scrubbing osd.6 2023-12-19T18:29:17.657 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:29:17.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.898+0000 7f8d44c5b640 1 Processor -- start 2023-12-19T18:29:17.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.898+0000 7f8d44c5b640 1 -- start start 2023-12-19T18:29:17.905 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.900+0000 7f8d44c5b640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f8d4015d280 0x7f8d400b1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.900+0000 7f8d44c5b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015d660 0x7f8d400aded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.900+0000 7f8d44c5b640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d400ae410 con 0x7f8d400afee0 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.900+0000 7f8d44c5b640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d400b1b40 con 0x7f8d4015d280 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.900+0000 7f8d44c5b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d400b24a0 con 0x7f8d4015d660 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.900+0000 7f8d37fff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f8d400afee0 0x7f8d400ad7c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47610/0 (socket says 172.21.15.27:47610) 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.900+0000 7f8d37fff640 1 -- 172.21.15.27:0/87362447 learned_addr learned my addr 172.21.15.27:0/87362447 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.900+0000 7f8d3ffff640 1 --2- 172.21.15.27:0/87362447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015d660 0x7f8d400aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.900+0000 7f8d3f7fe640 1 --2- 172.21.15.27:0/87362447 >> v2:172.21.15.192:3300/0 conn(0x7f8d4015d280 0x7f8d400b1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540486492 0 0) 0x7f8d400ae410 con 0x7f8d400afee0 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8d2c003520 con 0x7f8d400afee0 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1355127270 0 0) 0x7f8d2c003520 con 0x7f8d400afee0 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015d660 msgr2=0x7f8d400aded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 --2- 172.21.15.27:0/87362447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015d660 0x7f8d400aded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 >> v2:172.21.15.192:3300/0 conn(0x7f8d4015d280 msgr2=0x7f8d400b1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:17.906 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 --2- 172.21.15.27:0/87362447 >> v2:172.21.15.192:3300/0 conn(0x7f8d4015d280 0x7f8d400b1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d400b0ec0 con 0x7f8d400afee0 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f8d28001670 con 0x7f8d400afee0 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 >> v1:172.21.15.27:6789/0 conn(0x7f8d400afee0 legacy=0x7f8d400ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 --2- 172.21.15.27:0/87362447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015d660 0x7f8d2c003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 --2- 172.21.15.27:0/87362447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d2c003ef0 0x7f8d2c0047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 --2- 172.21.15.27:0/87362447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d2c004d00 0x7f8d2c005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d400b1b40 con 0x7f8d2c004d00 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d400b24a0 con 0x7f8d2c003ef0 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.901+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c00b980 con 0x7f8d4015d660 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.902+0000 7f8d3ffff640 1 --2- 172.21.15.27:0/87362447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d2c004d00 0x7f8d2c005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.902+0000 7f8d37fff640 1 --2- 172.21.15.27:0/87362447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d2c003ef0 0x7f8d2c0047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.902+0000 7f8d3f7fe640 1 --2- 172.21.15.27:0/87362447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015d660 0x7f8d2c003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.902+0000 7f8d3ffff640 1 -- 172.21.15.27:0/87362447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015d660 msgr2=0x7f8d2c003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.902+0000 7f8d3ffff640 1 --2- 172.21.15.27:0/87362447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015d660 0x7f8d2c003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.907 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.902+0000 7f8d3ffff640 1 -- 172.21.15.27:0/87362447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d2c003ef0 msgr2=0x7f8d2c0047c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.902+0000 7f8d3ffff640 1 --2- 172.21.15.27:0/87362447 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d2c003ef0 0x7f8d2c0047c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.902+0000 7f8d3ffff640 1 -- 172.21.15.27:0/87362447 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c003520 con 0x7f8d2c004d00 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.902+0000 7f8d3f7fe640 1 --2- 172.21.15.27:0/87362447 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d4015d660 0x7f8d2c003970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.903+0000 7f8d3ffff640 1 --2- 172.21.15.27:0/87362447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d2c004d00 0x7f8d2c005180 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f8d400c4690 tx=0x7f8d2c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a30bc48a2713ba0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.903+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3802cb90 con 0x7f8d2c004d00 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.903+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3800f040 con 0x7f8d2c004d00 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.903+0000 7f8d3effd640 1 -- 172.21.15.27:0/87362447 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38034440 con 0x7f8d2c004d00 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.904+0000 7f8d44c5b640 1 -- 172.21.15.27:0/87362447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d2c004d00 msgr2=0x7f8d2c005180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.904+0000 7f8d44c5b640 1 --2- 172.21.15.27:0/87362447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d2c004d00 0x7f8d2c005180 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7f8d400c4690 tx=0x7f8d2c0057f0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.904+0000 7f8d3ffff640 1 -- 172.21.15.27:0/87362447 reap_dead start 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.904+0000 7f8d44c5b640 1 -- 172.21.15.27:0/87362447 shutdown_connections 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.904+0000 7f8d44c5b640 1 -- 172.21.15.27:0/87362447 >> 172.21.15.27:0/87362447 conn(0x7f8d400d0b90 msgr2=0x7f8d400d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.904+0000 7f8d44c5b640 1 -- 172.21.15.27:0/87362447 shutdown_connections 2023-12-19T18:29:17.908 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.904+0000 7f8d44c5b640 1 -- 172.21.15.27:0/87362447 wait complete. 2023-12-19T18:29:17.910 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.906+0000 7f8d44c5b640 1 Processor -- start 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.906+0000 7f8d44c5b640 1 -- start start 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.907+0000 7f8d44c5b640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d2c004d00 0x7f8d4015ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.907+0000 7f8d44c5b640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d401e6610 0x7f8d400cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.907+0000 7f8d44c5b640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d400b3740 0x7f8d400b3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.907+0000 7f8d44c5b640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d400b4100 con 0x7f8d400b3740 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.907+0000 7f8d44c5b640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4015b320 con 0x7f8d2c004d00 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.907+0000 7f8d44c5b640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d401e6b40 con 0x7f8d401e6610 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.907+0000 7f8d3ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d400b3740 0x7f8d400b3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.908+0000 7f8d3ffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d400b3740 0x7f8d400b3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55690/0 (socket says 172.21.15.27:55690) 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.908+0000 7f8d3ffff640 1 -- 172.21.15.27:0/3010709531 learned_addr learned my addr 172.21.15.27:0/3010709531 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.908+0000 7f8d37fff640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d401e6610 0x7f8d400cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.908+0000 7f8d3f7fe640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d2c004d00 0x7f8d4015ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.908+0000 7f8d3ffff640 1 -- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d401e6610 msgr2=0x7f8d400cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.911 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.908+0000 7f8d3ffff640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d401e6610 0x7f8d400cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.908+0000 7f8d3ffff640 1 -- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d2c004d00 msgr2=0x7f8d4015ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.908+0000 7f8d3ffff640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f8d2c004d00 0x7f8d4015ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.908+0000 7f8d3ffff640 1 -- 172.21.15.27:0/3010709531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d401e6e20 con 0x7f8d400b3740 2023-12-19T18:29:17.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.908+0000 7f8d37fff640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f8d401e6610 0x7f8d400cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:17.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.908+0000 7f8d3ffff640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d400b3740 0x7f8d400b3bc0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f8d38002230 tx=0x7f8d38009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:17.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.909+0000 7f8d3cff9640 1 -- 172.21.15.27:0/3010709531 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d38034d50 con 0x7f8d400b3740 2023-12-19T18:29:17.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.909+0000 7f8d3cff9640 1 -- 172.21.15.27:0/3010709531 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d3802de20 con 0x7f8d400b3740 2023-12-19T18:29:17.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.909+0000 7f8d44c5b640 1 -- 172.21.15.27:0/3010709531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d401e7100 con 0x7f8d400b3740 2023-12-19T18:29:17.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.909+0000 7f8d3cff9640 1 -- 172.21.15.27:0/3010709531 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d3803e680 con 0x7f8d400b3740 2023-12-19T18:29:17.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.909+0000 7f8d44c5b640 1 -- 172.21.15.27:0/3010709531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d400cbaa0 con 0x7f8d400b3740 2023-12-19T18:29:17.912 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.910+0000 7f8d44c5b640 1 -- 172.21.15.27:0/3010709531 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8d400b1480 con 0x7f8d400b3740 2023-12-19T18:29:17.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.911+0000 7f8d3cff9640 1 -- 172.21.15.27:0/3010709531 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f8d3803e7e0 con 0x7f8d400b3740 2023-12-19T18:29:17.913 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.912+0000 7f8d3cff9640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8d2006a430 0x7f8d2006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.912+0000 7f8d3cff9640 1 -- 172.21.15.27:0/3010709531 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f8d380aa8e0 con 0x7f8d400b3740 2023-12-19T18:29:17.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.912+0000 7f8d3cff9640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] conn(0x7f8d20070290 0x7f8d200726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:17.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.912+0000 7f8d3cff9640 1 -- 172.21.15.27:0/3010709531 --> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d20072d60 con 0x7f8d20070290 2023-12-19T18:29:17.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.912+0000 7f8d3cff9640 1 -- 172.21.15.27:0/3010709531 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=210) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d380af2a0 con 0x7f8d400b3740 2023-12-19T18:29:17.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.912+0000 7f8d3f7fe640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8d2006a430 0x7f8d2006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.913+0000 7f8d37fff640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] conn(0x7f8d20070290 0x7f8d200726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:17.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.914+0000 7f8d3f7fe640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8d2006a430 0x7f8d2006c8f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8d4008b1f0 tx=0x7f8d30008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:17.914 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.914+0000 7f8d37fff640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] conn(0x7f8d20070290 0x7f8d200726b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:17.916 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.916+0000 7f8d3cff9640 1 -- 172.21.15.27:0/3010709531 <== osd.6 v2:172.21.15.192:6824/84255 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f8d20072d60 con 0x7f8d20070290 2023-12-19T18:29:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.929+0000 7f8d44c5b640 1 -- 172.21.15.27:0/3010709531 --> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8d400b11a0 con 0x7f8d20070290 2023-12-19T18:29:17.930 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.930+0000 7f8d3cff9640 1 -- 172.21.15.27:0/3010709531 <== osd.6 v2:172.21.15.192:6824/84255 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f8d28001610 con 0x7f8d20070290 2023-12-19T18:29:17.930 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:29:17.931 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-12-19T18:29:17.931 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:29:17.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.931+0000 7f8d44c5b640 1 -- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] conn(0x7f8d20070290 msgr2=0x7f8d200726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.931+0000 7f8d44c5b640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:6824/84255,v1:172.21.15.192:6825/84255] conn(0x7f8d20070290 0x7f8d200726b0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:17.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.931+0000 7f8d44c5b640 1 -- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8d2006a430 msgr2=0x7f8d2006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.931+0000 7f8d44c5b640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f8d2006a430 0x7f8d2006c8f0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f8d4008b1f0 tx=0x7f8d30008040 comp rx=0 tx=0).stop 2023-12-19T18:29:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.931+0000 7f8d44c5b640 1 -- 172.21.15.27:0/3010709531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d400b3740 msgr2=0x7f8d400b3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.931+0000 7f8d44c5b640 1 --2- 172.21.15.27:0/3010709531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d400b3740 0x7f8d400b3bc0 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f8d38002230 tx=0x7f8d38009750 comp rx=0 tx=0).stop 2023-12-19T18:29:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.931+0000 7f8d3ffff640 1 -- 172.21.15.27:0/3010709531 reap_dead start 2023-12-19T18:29:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.931+0000 7f8d44c5b640 1 -- 172.21.15.27:0/3010709531 shutdown_connections 2023-12-19T18:29:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.931+0000 7f8d44c5b640 1 -- 172.21.15.27:0/3010709531 >> 172.21.15.27:0/3010709531 conn(0x7f8d400d0b90 msgr2=0x7f8d400ad850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.932+0000 7f8d44c5b640 1 -- 172.21.15.27:0/3010709531 shutdown_connections 2023-12-19T18:29:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:17.932+0000 7f8d44c5b640 1 -- 172.21.15.27:0/3010709531 wait complete. 2023-12-19T18:29:17.949 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-12-19T18:29:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.174+0000 7fc020ace640 1 Processor -- start 2023-12-19T18:29:18.174 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.174+0000 7fc020ace640 1 -- start start 2023-12-19T18:29:18.175 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.175+0000 7fc020ace640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc01c0cff30 0x7fc01c0b3210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.176+0000 7fc020ace640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc01c0cccc0 0x7fc01c0cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.176+0000 7fc020ace640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc01c15c200 con 0x7fc01c06fcf0 2023-12-19T18:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.176+0000 7fc020ace640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01c0701a0 con 0x7fc01c0cff30 2023-12-19T18:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.176+0000 7fc020ace640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc01c0cb6e0 con 0x7fc01c0cccc0 2023-12-19T18:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.176+0000 7fc01a59c640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7fc01c0cff30 0x7fc01c0b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.176+0000 7fc019d9b640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc01c06fcf0 0x7fc01c0d0310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47624/0 (socket says 172.21.15.27:47624) 2023-12-19T18:29:18.176 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.176+0000 7fc019d9b640 1 -- 172.21.15.27:0/2436772614 learned_addr learned my addr 172.21.15.27:0/2436772614 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.176+0000 7fc01ad9d640 1 --2- 172.21.15.27:0/2436772614 >> v2:172.21.15.192:3300/0 conn(0x7fc01c0cccc0 0x7fc01c0cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.176+0000 7fc01a59c640 1 -- 172.21.15.27:0/2436772614 >> v2:172.21.15.192:3300/0 conn(0x7fc01c0cccc0 msgr2=0x7fc01c0cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.176+0000 7fc01a59c640 1 --2- 172.21.15.27:0/2436772614 >> v2:172.21.15.192:3300/0 conn(0x7fc01c0cccc0 0x7fc01c0cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.176+0000 7fc01a59c640 1 -- 172.21.15.27:0/2436772614 >> v1:172.21.15.27:6789/0 conn(0x7fc01c06fcf0 legacy=0x7fc01c0d0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.177+0000 7fc01a59c640 1 -- 172.21.15.27:0/2436772614 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc01c15b610 con 0x7fc01c0cff30 2023-12-19T18:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.177+0000 7fc01ad9d640 1 --2- 172.21.15.27:0/2436772614 >> v2:172.21.15.192:3300/0 conn(0x7fc01c0cccc0 0x7fc01c0cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:18.177 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.176+0000 7fc01959a640 1 -- 172.21.15.27:0/2436772614 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1743874852 0 0) 0x7fc01c15c200 con 0x7fc01c06fcf0 2023-12-19T18:29:18.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.178+0000 7fc01a59c640 1 --2- 172.21.15.27:0/2436772614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0cff30 0x7fc01c0b3210 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009d90 tx=0x7fc00c02be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6e85b2e46116e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:18.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.178+0000 7fc01959a640 1 -- 172.21.15.27:0/2436772614 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c02cbd0 con 0x7fc01c0cff30 2023-12-19T18:29:18.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.178+0000 7fc01959a640 1 -- 172.21.15.27:0/2436772614 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc00c02cd30 con 0x7fc01c0cff30 2023-12-19T18:29:18.178 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.178+0000 7fc01959a640 1 -- 172.21.15.27:0/2436772614 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc00c02cea0 con 0x7fc01c0cff30 2023-12-19T18:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.179+0000 7fc020ace640 1 -- 172.21.15.27:0/2436772614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0cff30 msgr2=0x7fc01c0b3210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.179+0000 7fc020ace640 1 --2- 172.21.15.27:0/2436772614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0cff30 0x7fc01c0b3210 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fc00c009d90 tx=0x7fc00c02be20 comp rx=0 tx=0).stop 2023-12-19T18:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.179+0000 7fc020ace640 1 -- 172.21.15.27:0/2436772614 shutdown_connections 2023-12-19T18:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.179+0000 7fc020ace640 1 --2- 172.21.15.27:0/2436772614 >> v2:172.21.15.192:3300/0 conn(0x7fc01c0cccc0 0x7fc01c0cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.179+0000 7fc020ace640 1 --2- 172.21.15.27:0/2436772614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0cff30 0x7fc01c0b3210 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.179+0000 7fc020ace640 1 -- 172.21.15.27:0/2436772614 >> 172.21.15.27:0/2436772614 conn(0x7fc01c06f1d0 msgr2=0x7fc01c06f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.179+0000 7fc020ace640 1 -- 172.21.15.27:0/2436772614 shutdown_connections 2023-12-19T18:29:18.179 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.179+0000 7fc020ace640 1 -- 172.21.15.27:0/2436772614 wait complete. 2023-12-19T18:29:18.180 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.181+0000 7fc020ace640 1 Processor -- start 2023-12-19T18:29:18.181 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.181+0000 7fc020ace640 1 -- start start 2023-12-19T18:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.182+0000 7fc020ace640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc01c06fcf0 0x7fc01c0b1d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.182+0000 7fc020ace640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc01c0cccc0 0x7fc01c152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.182+0000 7fc020ace640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0cff30 0x7fc01c0c5d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.182+0000 7fc020ace640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0b33e0 con 0x7fc01c0cff30 2023-12-19T18:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.182+0000 7fc020ace640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c153450 con 0x7fc01c06fcf0 2023-12-19T18:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.182+0000 7fc020ace640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc01c0b2340 con 0x7fc01c0cccc0 2023-12-19T18:29:18.182 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.182+0000 7fc01ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0cff30 0x7fc01c0c5d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.182+0000 7fc01ad9d640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0cff30 0x7fc01c0c5d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55718/0 (socket says 172.21.15.27:55718) 2023-12-19T18:29:18.183 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.182+0000 7fc01ad9d640 1 -- 172.21.15.27:0/3926171523 learned_addr learned my addr 172.21.15.27:0/3926171523 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:18.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.183+0000 7fc019d9b640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc01c0cccc0 0x7fc01c152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.183+0000 7fc01ad9d640 1 -- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc01c0cccc0 msgr2=0x7fc01c152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.183+0000 7fc01ad9d640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc01c0cccc0 0x7fc01c152f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.183+0000 7fc01a59c640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc01c06fcf0 0x7fc01c0b1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.183+0000 7fc01ad9d640 1 -- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc01c06fcf0 msgr2=0x7fc01c0b1d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.183+0000 7fc01ad9d640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc01c06fcf0 0x7fc01c0b1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.183+0000 7fc01ad9d640 1 -- 172.21.15.27:0/3926171523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c009a70 con 0x7fc01c0cff30 2023-12-19T18:29:18.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.183+0000 7fc01ad9d640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0cff30 0x7fc01c0c5d50 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fc01400c7f0 tx=0x7fc01400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:18.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.183+0000 7fc012ffd640 1 -- 172.21.15.27:0/3926171523 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014007820 con 0x7fc01c0cff30 2023-12-19T18:29:18.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.184+0000 7fc012ffd640 1 -- 172.21.15.27:0/3926171523 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc014007980 con 0x7fc01c0cff30 2023-12-19T18:29:18.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.184+0000 7fc012ffd640 1 -- 172.21.15.27:0/3926171523 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc014007d70 con 0x7fc01c0cff30 2023-12-19T18:29:18.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.184+0000 7fc020ace640 1 -- 172.21.15.27:0/3926171523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc01c0b2620 con 0x7fc01c0cff30 2023-12-19T18:29:18.184 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.184+0000 7fc020ace640 1 -- 172.21.15.27:0/3926171523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc01c0b2be0 con 0x7fc01c0cff30 2023-12-19T18:29:18.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.185+0000 7fc020ace640 1 -- 172.21.15.27:0/3926171523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc01c15be70 con 0x7fc01c0cff30 2023-12-19T18:29:18.185 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.186+0000 7fc012ffd640 1 -- 172.21.15.27:0/3926171523 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc0140113f0 con 0x7fc01c0cff30 2023-12-19T18:29:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.187+0000 7fc012ffd640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbffc06a430 0x7fbffc06c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.186 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.187+0000 7fc012ffd640 1 -- 172.21.15.27:0/3926171523 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc014088a10 con 0x7fc01c0cff30 2023-12-19T18:29:18.187 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.187+0000 7fc01a59c640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbffc06a430 0x7fbffc06c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.188 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.188+0000 7fc01a59c640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbffc06a430 0x7fbffc06c8f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc00c036000 tx=0x7fc00c030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:18.191 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.191+0000 7fc012ffd640 1 -- 172.21.15.27:0/3926171523 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc01c15be70 con 0x7fc01c0cff30 2023-12-19T18:29:18.388 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.388+0000 7fc020ace640 1 -- 172.21.15.27:0/3926171523 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fc01c15bbb0 con 0x7fbffc06a430 2023-12-19T18:29:18.389 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.389+0000 7fc012ffd640 1 -- 172.21.15.27:0/3926171523 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc00c044090 con 0x7fbffc06a430 2023-12-19T18:29:18.389 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 6 to deep-scrub 2023-12-19T18:29:18.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.392+0000 7fc020ace640 1 -- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbffc06a430 msgr2=0x7fbffc06c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.392+0000 7fc020ace640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbffc06a430 0x7fbffc06c8f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fc00c036000 tx=0x7fc00c030000 comp rx=0 tx=0).stop 2023-12-19T18:29:18.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.392+0000 7fc020ace640 1 -- 172.21.15.27:0/3926171523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0cff30 msgr2=0x7fc01c0c5d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.392+0000 7fc020ace640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0cff30 0x7fc01c0c5d50 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fc01400c7f0 tx=0x7fc01400ccb0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.392+0000 7fc020ace640 1 -- 172.21.15.27:0/3926171523 shutdown_connections 2023-12-19T18:29:18.392 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.392+0000 7fc020ace640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbffc06a430 0x7fbffc06c8f0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.392+0000 7fc020ace640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc01c0cff30 0x7fc01c0c5d50 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.392+0000 7fc020ace640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc01c0cccc0 0x7fc01c152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.392+0000 7fc020ace640 1 --2- 172.21.15.27:0/3926171523 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc01c06fcf0 0x7fc01c0b1d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.392+0000 7fc020ace640 1 -- 172.21.15.27:0/3926171523 >> 172.21.15.27:0/3926171523 conn(0x7fc01c06f1d0 msgr2=0x7fc01c0b5810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.392+0000 7fc020ace640 1 -- 172.21.15.27:0/3926171523 shutdown_connections 2023-12-19T18:29:18.393 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.392+0000 7fc020ace640 1 -- 172.21.15.27:0/3926171523 wait complete. 2023-12-19T18:29:18.412 INFO:tasks.ceph:Scrubbing osd.7 2023-12-19T18:29:18.412 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-12-19T18:29:18.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.653+0000 7f912a6bd640 1 Processor -- start 2023-12-19T18:29:18.653 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.653+0000 7f912a6bd640 1 -- start start 2023-12-19T18:29:18.655 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.655+0000 7f912a6bd640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9124159fc0 0x7f91240710d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.655+0000 7f912a6bd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f91240add30 0x7f91240ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.655+0000 7f912a6bd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91240ca800 0x7f91240cb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.655+0000 7f912a6bd640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9124070c40 con 0x7f91240add30 2023-12-19T18:29:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.655+0000 7f912a6bd640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9124070030 con 0x7f9124159fc0 2023-12-19T18:29:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.655+0000 7f912a6bd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9124070310 con 0x7f91240ca800 2023-12-19T18:29:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.656+0000 7f911bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f91240add30 0x7f91240ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.656+0000 7f911bfff640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f91240add30 0x7f91240ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55722/0 (socket says 172.21.15.27:55722) 2023-12-19T18:29:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.656+0000 7f911bfff640 1 -- 172.21.15.27:0/3768427656 learned_addr learned my addr 172.21.15.27:0/3768427656 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.656+0000 7f9123fff640 1 --2- 172.21.15.27:0/3768427656 >> v2:172.21.15.192:3300/0 conn(0x7f9124159fc0 0x7f91240710d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.656 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.656+0000 7f9128c5a640 1 --2- 172.21.15.27:0/3768427656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91240ca800 0x7f91240cb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.656+0000 7f911bfff640 1 -- 172.21.15.27:0/3768427656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91240ca800 msgr2=0x7f91240cb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.656+0000 7f911bfff640 1 --2- 172.21.15.27:0/3768427656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91240ca800 0x7f91240cb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.656+0000 7f911bfff640 1 -- 172.21.15.27:0/3768427656 >> v2:172.21.15.192:3300/0 conn(0x7f9124159fc0 msgr2=0x7f91240710d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.656+0000 7f911bfff640 1 --2- 172.21.15.27:0/3768427656 >> v2:172.21.15.192:3300/0 conn(0x7f9124159fc0 0x7f91240710d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.657 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.656+0000 7f911bfff640 1 -- 172.21.15.27:0/3768427656 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91240705f0 con 0x7f91240add30 2023-12-19T18:29:18.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.657+0000 7f911bfff640 1 --2- 172.21.15.27:0/3768427656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91240add30 0x7f91240ca2c0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f912406b130 tx=0x7f910c02b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5af4ff67a9bac6e server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:18.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.658+0000 7f91237fe640 1 -- 172.21.15.27:0/3768427656 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c031880 con 0x7f91240add30 2023-12-19T18:29:18.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.658+0000 7f91237fe640 1 -- 172.21.15.27:0/3768427656 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f910c0319e0 con 0x7f91240add30 2023-12-19T18:29:18.658 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.658+0000 7f91237fe640 1 -- 172.21.15.27:0/3768427656 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f910c031d70 con 0x7f91240add30 2023-12-19T18:29:18.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.659+0000 7f912a6bd640 1 -- 172.21.15.27:0/3768427656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91240add30 msgr2=0x7f91240ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.659+0000 7f912a6bd640 1 --2- 172.21.15.27:0/3768427656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91240add30 0x7f91240ca2c0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7f912406b130 tx=0x7f910c02b920 comp rx=0 tx=0).stop 2023-12-19T18:29:18.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.659+0000 7f912a6bd640 1 -- 172.21.15.27:0/3768427656 shutdown_connections 2023-12-19T18:29:18.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.659+0000 7f912a6bd640 1 --2- 172.21.15.27:0/3768427656 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91240ca800 0x7f91240cb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.659+0000 7f912a6bd640 1 --2- 172.21.15.27:0/3768427656 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91240add30 0x7f91240ca2c0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.659+0000 7f912a6bd640 1 --2- 172.21.15.27:0/3768427656 >> v2:172.21.15.192:3300/0 conn(0x7f9124159fc0 0x7f91240710d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.659+0000 7f912a6bd640 1 -- 172.21.15.27:0/3768427656 >> 172.21.15.27:0/3768427656 conn(0x7f91240b0620 msgr2=0x7f91240b0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:18.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.659+0000 7f912a6bd640 1 -- 172.21.15.27:0/3768427656 shutdown_connections 2023-12-19T18:29:18.659 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.659+0000 7f912a6bd640 1 -- 172.21.15.27:0/3768427656 wait complete. 2023-12-19T18:29:18.660 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.661+0000 7f912a6bd640 1 Processor -- start 2023-12-19T18:29:18.661 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.661+0000 7f912a6bd640 1 -- start start 2023-12-19T18:29:18.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.662+0000 7f912a6bd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91240add30 0x7f912415c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.662+0000 7f912a6bd640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91240ca800 0x7f9124146a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.662+0000 7f912a6bd640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9124159fc0 0x7f9124146fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.662+0000 7f912a6bd640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9124147500 con 0x7f91240add30 2023-12-19T18:29:18.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.662+0000 7f912a6bd640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91240c4160 con 0x7f9124159fc0 2023-12-19T18:29:18.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.662+0000 7f912a6bd640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91240c4440 con 0x7f91240ca800 2023-12-19T18:29:18.662 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.662+0000 7f9123fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91240add30 0x7f912415c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.662+0000 7f9123fff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91240add30 0x7f912415c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55734/0 (socket says 172.21.15.27:55734) 2023-12-19T18:29:18.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.662+0000 7f9123fff640 1 -- 172.21.15.27:0/1437403580 learned_addr learned my addr 172.21.15.27:0/1437403580 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:18.663 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.663+0000 7f911bfff640 1 --2- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91240ca800 0x7f9124146a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.663+0000 7f9123fff640 1 -- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91240ca800 msgr2=0x7f9124146a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.663+0000 7f9123fff640 1 --2- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f91240ca800 0x7f9124146a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.663+0000 7f9123fff640 1 -- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9124159fc0 msgr2=0x7f9124146fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.663+0000 7f9123fff640 1 --2- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9124159fc0 0x7f9124146fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.663+0000 7f9123fff640 1 -- 172.21.15.27:0/1437403580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910c0094d0 con 0x7f91240add30 2023-12-19T18:29:18.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.663+0000 7f9123fff640 1 --2- 172.21.15.27:0/1437403580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91240add30 0x7f912415c930 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f911400d5f0 tx=0x7f911400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:18.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.663+0000 7f91217fa640 1 -- 172.21.15.27:0/1437403580 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91140125f0 con 0x7f91240add30 2023-12-19T18:29:18.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.663+0000 7f91217fa640 1 -- 172.21.15.27:0/1437403580 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9114012750 con 0x7f91240add30 2023-12-19T18:29:18.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.663+0000 7f91217fa640 1 -- 172.21.15.27:0/1437403580 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114012b00 con 0x7f91240add30 2023-12-19T18:29:18.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.663+0000 7f912a6bd640 1 -- 172.21.15.27:0/1437403580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912415a520 con 0x7f91240add30 2023-12-19T18:29:18.664 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.664+0000 7f912a6bd640 1 -- 172.21.15.27:0/1437403580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91240c4a00 con 0x7f91240add30 2023-12-19T18:29:18.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.665+0000 7f912a6bd640 1 -- 172.21.15.27:0/1437403580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f912415a3a0 con 0x7f91240add30 2023-12-19T18:29:18.665 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.666+0000 7f91217fa640 1 -- 172.21.15.27:0/1437403580 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9114012c60 con 0x7f91240add30 2023-12-19T18:29:18.666 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.667+0000 7f91217fa640 1 --2- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f910406a430 0x7f910406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.667+0000 7f91217fa640 1 -- 172.21.15.27:0/1437403580 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9114088b30 con 0x7f91240add30 2023-12-19T18:29:18.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.667+0000 7f91217fa640 1 --2- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] conn(0x7f9104070290 0x7f91040726b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.667+0000 7f91217fa640 1 -- 172.21.15.27:0/1437403580 --> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9104072d60 con 0x7f9104070290 2023-12-19T18:29:18.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.667+0000 7f91217fa640 1 -- 172.21.15.27:0/1437403580 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=210) v2 ==== 24+0+0 (secure 0 0 0) 0x7f911408d2a0 con 0x7f91240add30 2023-12-19T18:29:18.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.667+0000 7f911bfff640 1 --2- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f910406a430 0x7f910406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.667 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.668+0000 7f9128c5a640 1 --2- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] conn(0x7f9104070290 0x7f91040726b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.668+0000 7f911bfff640 1 --2- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f910406a430 0x7f910406c8f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f910c036000 tx=0x7f910c03c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:18.668 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.668+0000 7f9128c5a640 1 --2- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] conn(0x7f9104070290 0x7f91040726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:18.670 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.670+0000 7f91217fa640 1 -- 172.21.15.27:0/1437403580 <== osd.7 v2:172.21.15.192:6816/84207 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23447 (crc 0 0 0) 0x7f9104072d60 con 0x7f9104070290 2023-12-19T18:29:18.683 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.683+0000 7f912a6bd640 1 -- 172.21.15.27:0/1437403580 --> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f91240708d0 con 0x7f9104070290 2023-12-19T18:29:18.684 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.684+0000 7f91217fa640 1 -- 172.21.15.27:0/1437403580 <== osd.7 v2:172.21.15.192:6816/84207 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+757 (crc 0 0 0) 0x7f911c009ad0 con 0x7f9104070290 2023-12-19T18:29:18.684 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2023-12-19T18:29:18.684 INFO:teuthology.orchestra.run.smithi027.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) osd_delete_sleep = '0.000000' osd_delete_sleep_hdd = '0.000000' osd_delete_sleep_hybrid = '0.000000' osd_delete_sleep_ssd = '0.000000' osd_max_backfills = '1' osd_pg_delete_cost = '15728640' (not observed, change may require restart) osd_recovery_max_active = '0' osd_recovery_max_active_hdd = '3' osd_recovery_max_active_ssd = '10' osd_recovery_sleep = '0.000000' osd_recovery_sleep_hdd = '0.000000' osd_recovery_sleep_hybrid = '0.000000' osd_recovery_sleep_ssd = '0.000000' osd_scrub_sleep = '0.000000' osd_snap_trim_sleep = '0.000000' osd_snap_trim_sleep_hdd = '0.000000' osd_snap_trim_sleep_hybrid = '0.000000' osd_snap_trim_sleep_ssd = '0.000000' " 2023-12-19T18:29:18.684 INFO:teuthology.orchestra.run.smithi027.stdout:} 2023-12-19T18:29:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.685+0000 7f912a6bd640 1 -- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] conn(0x7f9104070290 msgr2=0x7f91040726b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.685+0000 7f912a6bd640 1 --2- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:6816/84207,v1:172.21.15.192:6817/84207] conn(0x7f9104070290 0x7f91040726b0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.685+0000 7f912a6bd640 1 -- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f910406a430 msgr2=0x7f910406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.685+0000 7f912a6bd640 1 --2- 172.21.15.27:0/1437403580 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f910406a430 0x7f910406c8f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f910c036000 tx=0x7f910c03c040 comp rx=0 tx=0).stop 2023-12-19T18:29:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.685+0000 7f912a6bd640 1 -- 172.21.15.27:0/1437403580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91240add30 msgr2=0x7f912415c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.685+0000 7f912a6bd640 1 --2- 172.21.15.27:0/1437403580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91240add30 0x7f912415c930 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f911400d5f0 tx=0x7f911400dab0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.685+0000 7f9128c5a640 1 -- 172.21.15.27:0/1437403580 reap_dead start 2023-12-19T18:29:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.685+0000 7f912a6bd640 1 -- 172.21.15.27:0/1437403580 shutdown_connections 2023-12-19T18:29:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.685+0000 7f912a6bd640 1 -- 172.21.15.27:0/1437403580 >> 172.21.15.27:0/1437403580 conn(0x7f91240b0620 msgr2=0x7f91240cd150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.685+0000 7f912a6bd640 1 -- 172.21.15.27:0/1437403580 shutdown_connections 2023-12-19T18:29:18.686 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.686+0000 7f912a6bd640 1 -- 172.21.15.27:0/1437403580 wait complete. 2023-12-19T18:29:18.703 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-12-19T18:29:18.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.931+0000 7fc0e6238640 1 Processor -- start 2023-12-19T18:29:18.931 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.931+0000 7fc0e6238640 1 -- start start 2023-12-19T18:29:18.932 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.932+0000 7fc0e6238640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7fc0e006ab00 0x7fc0e00d0e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.932+0000 7fc0e6238640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e00d2af0 0x7fc0e00d13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.932+0000 7fc0e6238640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0e00d1a50 con 0x7fc0e015d260 2023-12-19T18:29:18.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.932+0000 7fc0e6238640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0e00cdaf0 con 0x7fc0e006ab00 2023-12-19T18:29:18.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.932+0000 7fc0e6238640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e00b71b0 con 0x7fc0e00d2af0 2023-12-19T18:29:18.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.933+0000 7fc0df7fe640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fc0e015d260 0x7fc0e006a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47636/0 (socket says 172.21.15.27:47636) 2023-12-19T18:29:18.933 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.933+0000 7fc0df7fe640 1 -- 172.21.15.27:0/488824966 learned_addr learned my addr 172.21.15.27:0/488824966 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:18.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.933+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3915638578 0 0) 0x7fc0e00d1a50 con 0x7fc0e015d260 2023-12-19T18:29:18.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.933+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0cc003520 con 0x7fc0e015d260 2023-12-19T18:29:18.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.933+0000 7fc0deffd640 1 --2- 172.21.15.27:0/488824966 >> v2:172.21.15.192:3300/0 conn(0x7fc0e006ab00 0x7fc0e00d0e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.933+0000 7fc0dffff640 1 --2- 172.21.15.27:0/488824966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e00d2af0 0x7fc0e00d13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2226957983 0 0) 0x7fc0cc003520 con 0x7fc0e015d260 2023-12-19T18:29:18.935 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e00d2af0 msgr2=0x7fc0e00d13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:18.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 --2- 172.21.15.27:0/488824966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e00d2af0 0x7fc0e00d13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 >> v2:172.21.15.192:3300/0 conn(0x7fc0e006ab00 msgr2=0x7fc0e00d0e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:18.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 --2- 172.21.15.27:0/488824966 >> v2:172.21.15.192:3300/0 conn(0x7fc0e006ab00 0x7fc0e00d0e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e00d1d30 con 0x7fc0e015d260 2023-12-19T18:29:18.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fc0d00019f0 con 0x7fc0e015d260 2023-12-19T18:29:18.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 >> v1:172.21.15.27:6789/0 conn(0x7fc0e015d260 legacy=0x7fc0e006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 --2- 172.21.15.27:0/488824966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0cc003970 0x7fc0cc003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 --2- 172.21.15.27:0/488824966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0cc0047c0 0x7fc0cc004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.936 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 --2- 172.21.15.27:0/488824966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e00d2af0 0x7fc0cc005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e00cdaf0 con 0x7fc0cc0047c0 2023-12-19T18:29:18.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e00b71b0 con 0x7fc0cc003970 2023-12-19T18:29:18.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0cc00ba30 con 0x7fc0e00d2af0 2023-12-19T18:29:18.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.934+0000 7fc0deffd640 1 --2- 172.21.15.27:0/488824966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0cc0047c0 0x7fc0cc004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.935+0000 7fc0dffff640 1 --2- 172.21.15.27:0/488824966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e00d2af0 0x7fc0cc005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.935+0000 7fc0df7fe640 1 --2- 172.21.15.27:0/488824966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0cc003970 0x7fc0cc003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.935+0000 7fc0deffd640 1 -- 172.21.15.27:0/488824966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e00d2af0 msgr2=0x7fc0cc005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:18.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.935+0000 7fc0deffd640 1 --2- 172.21.15.27:0/488824966 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e00d2af0 0x7fc0cc005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.935+0000 7fc0deffd640 1 -- 172.21.15.27:0/488824966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0cc003970 msgr2=0x7fc0cc003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:18.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.935+0000 7fc0deffd640 1 --2- 172.21.15.27:0/488824966 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0cc003970 0x7fc0cc003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.937 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.935+0000 7fc0deffd640 1 -- 172.21.15.27:0/488824966 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0cc003520 con 0x7fc0cc0047c0 2023-12-19T18:29:18.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.936+0000 7fc0deffd640 1 --2- 172.21.15.27:0/488824966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0cc0047c0 0x7fc0cc004c40 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800ba00 tx=0x7fc0cc0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e57e3621ec53de1 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:18.938 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.936+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8032630 con 0x7fc0cc0047c0 2023-12-19T18:29:18.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.936+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0c8032790 con 0x7fc0cc0047c0 2023-12-19T18:29:18.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.936+0000 7fc0de7fc640 1 -- 172.21.15.27:0/488824966 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0c8032b20 con 0x7fc0cc0047c0 2023-12-19T18:29:18.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.937+0000 7fc0e6238640 1 -- 172.21.15.27:0/488824966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0cc0047c0 msgr2=0x7fc0cc004c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.937+0000 7fc0e6238640 1 --2- 172.21.15.27:0/488824966 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0cc0047c0 0x7fc0cc004c40 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7fc0c800ba00 tx=0x7fc0cc0057f0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.937+0000 7fc0dffff640 1 -- 172.21.15.27:0/488824966 reap_dead start 2023-12-19T18:29:18.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.937+0000 7fc0e6238640 1 -- 172.21.15.27:0/488824966 shutdown_connections 2023-12-19T18:29:18.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.937+0000 7fc0e6238640 1 -- 172.21.15.27:0/488824966 >> 172.21.15.27:0/488824966 conn(0x7fc0e00d2f70 msgr2=0x7fc0e00d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:18.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.937+0000 7fc0e6238640 1 -- 172.21.15.27:0/488824966 shutdown_connections 2023-12-19T18:29:18.939 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.937+0000 7fc0e6238640 1 -- 172.21.15.27:0/488824966 wait complete. 2023-12-19T18:29:18.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.939+0000 7fc0e6238640 1 Processor -- start 2023-12-19T18:29:18.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.939+0000 7fc0e6238640 1 -- start start 2023-12-19T18:29:18.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.940+0000 7fc0e6238640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0cc0047c0 0x7fc0e00cbac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.940+0000 7fc0e6238640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e01f5e30 0x7fc0e01f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.940 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.940+0000 7fc0e6238640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e0152b70 0x7fc0e0152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.940+0000 7fc0e6238640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e00cb6a0 con 0x7fc0cc0047c0 2023-12-19T18:29:18.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.940+0000 7fc0e6238640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e0153530 con 0x7fc0e0152b70 2023-12-19T18:29:18.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.940+0000 7fc0e6238640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0e00cc000 con 0x7fc0e01f5e30 2023-12-19T18:29:18.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.940+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0cc0047c0 0x7fc0e00cbac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.941+0000 7fc0df7fe640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0cc0047c0 0x7fc0e00cbac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55760/0 (socket says 172.21.15.27:55760) 2023-12-19T18:29:18.941 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.941+0000 7fc0df7fe640 1 -- 172.21.15.27:0/3574680228 learned_addr learned my addr 172.21.15.27:0/3574680228 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.941+0000 7fc0dffff640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e0152b70 0x7fc0e0152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.941+0000 7fc0df7fe640 1 -- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e01f5e30 msgr2=0x7fc0e01f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.941+0000 7fc0deffd640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e01f5e30 0x7fc0e01f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.941+0000 7fc0df7fe640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e01f5e30 0x7fc0e01f6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.941+0000 7fc0df7fe640 1 -- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e0152b70 msgr2=0x7fc0e0152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.941+0000 7fc0df7fe640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e0152b70 0x7fc0e0152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:18.942 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.941+0000 7fc0df7fe640 1 -- 172.21.15.27:0/3574680228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e00cb000 con 0x7fc0cc0047c0 2023-12-19T18:29:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.941+0000 7fc0dffff640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e0152b70 0x7fc0e0152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.941+0000 7fc0df7fe640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0cc0047c0 0x7fc0e00cbac0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0002170 tx=0x7fc0d000b4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.942+0000 7fc0d7fff640 1 -- 172.21.15.27:0/3574680228 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0004d60 con 0x7fc0cc0047c0 2023-12-19T18:29:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.942+0000 7fc0d7fff640 1 -- 172.21.15.27:0/3574680228 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0d000be80 con 0x7fc0cc0047c0 2023-12-19T18:29:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.942+0000 7fc0d7fff640 1 -- 172.21.15.27:0/3574680228 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0d0007310 con 0x7fc0cc0047c0 2023-12-19T18:29:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.942+0000 7fc0e6238640 1 -- 172.21.15.27:0/3574680228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0e01f68f0 con 0x7fc0cc0047c0 2023-12-19T18:29:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.942+0000 7fc0e6238640 1 -- 172.21.15.27:0/3574680228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0e00cb2c0 con 0x7fc0cc0047c0 2023-12-19T18:29:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.943+0000 7fc0e6238640 1 -- 172.21.15.27:0/3574680228 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0e0066810 con 0x7fc0cc0047c0 2023-12-19T18:29:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.944+0000 7fc0d7fff640 1 -- 172.21.15.27:0/3574680228 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fc0d0007470 con 0x7fc0cc0047c0 2023-12-19T18:29:18.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.945+0000 7fc0d7fff640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0c006a430 0x7fc0c006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:18.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.945+0000 7fc0d7fff640 1 -- 172.21.15.27:0/3574680228 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fc0d0088620 con 0x7fc0cc0047c0 2023-12-19T18:29:18.945 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.946+0000 7fc0deffd640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0c006a430 0x7fc0c006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:18.946 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.947+0000 7fc0deffd640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0c006a430 0x7fc0c006c8f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0057a0 tx=0x7fc0c803d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:18.949 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:18.949+0000 7fc0d7fff640 1 -- 172.21.15.27:0/3574680228 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fc0d0053fa0 con 0x7fc0cc0047c0 2023-12-19T18:29:19.050 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:29:19.059 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:29:19.146 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.146+0000 7fc0e6238640 1 -- 172.21.15.27:0/3574680228 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0e00d22f0 con 0x7fc0c006a430 2023-12-19T18:29:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.147+0000 7fc0d7fff640 1 -- 172.21.15.27:0/3574680228 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc0c8032a80 con 0x7fc0c006a430 2023-12-19T18:29:19.147 INFO:teuthology.orchestra.run.smithi027.stderr:instructed osd(s) 7 to deep-scrub 2023-12-19T18:29:19.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.149+0000 7fc0e6238640 1 -- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0c006a430 msgr2=0x7fc0c006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:19.149 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.149+0000 7fc0e6238640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0c006a430 0x7fc0c006c8f0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fc0cc0057a0 tx=0x7fc0c803d040 comp rx=0 tx=0).stop 2023-12-19T18:29:19.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.149+0000 7fc0e6238640 1 -- 172.21.15.27:0/3574680228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0cc0047c0 msgr2=0x7fc0e00cbac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:19.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.149+0000 7fc0e6238640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0cc0047c0 0x7fc0e00cbac0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7fc0d0002170 tx=0x7fc0d000b4a0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.149+0000 7fc0e6238640 1 -- 172.21.15.27:0/3574680228 shutdown_connections 2023-12-19T18:29:19.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.149+0000 7fc0e6238640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fc0c006a430 0x7fc0c006c8f0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.149+0000 7fc0e6238640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fc0e0152b70 0x7fc0e0152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.149+0000 7fc0e6238640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fc0e01f5e30 0x7fc0e01f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.149+0000 7fc0e6238640 1 --2- 172.21.15.27:0/3574680228 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0cc0047c0 0x7fc0e00cbac0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.149+0000 7fc0e6238640 1 -- 172.21.15.27:0/3574680228 >> 172.21.15.27:0/3574680228 conn(0x7fc0e00d2f70 msgr2=0x7fc0e006a3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:19.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.150+0000 7fc0e6238640 1 -- 172.21.15.27:0/3574680228 shutdown_connections 2023-12-19T18:29:19.150 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.150+0000 7fc0e6238640 1 -- 172.21.15.27:0/3574680228 wait complete. 2023-12-19T18:29:19.169 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:29:19.394 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.394+0000 7f4ee2529640 1 Processor -- start 2023-12-19T18:29:19.395 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.395+0000 7f4ee2529640 1 -- start start 2023-12-19T18:29:19.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.396+0000 7f4ee2529640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f4edc15ee30 0x7f4edc0c50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:19.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.396+0000 7f4ee2529640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f4edc0afeb0 0x7f4edc0c39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:19.396 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.396+0000 7f4ee2529640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4edc0cb350 con 0x7f4edc1553a0 2023-12-19T18:29:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.396+0000 7f4ee2529640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4edc0cbcf0 con 0x7f4edc0afeb0 2023-12-19T18:29:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.396+0000 7f4ee2529640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4edc0cb630 con 0x7f4edc15ee30 2023-12-19T18:29:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.397+0000 7f4ee1527640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f4edc1553a0 0x7f4edc15e720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:47644/0 (socket says 172.21.15.27:47644) 2023-12-19T18:29:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.397+0000 7f4ee1527640 1 -- 172.21.15.27:0/2970660678 learned_addr learned my addr 172.21.15.27:0/2970660678 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.397+0000 7f4ee1d28640 1 --2- 172.21.15.27:0/2970660678 >> v2:172.21.15.27:3300/0 conn(0x7f4edc0afeb0 0x7f4edc0c39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.397+0000 7f4ed3fff640 1 -- 172.21.15.27:0/2970660678 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 197905627 0 0) 0x7f4edc0cb350 con 0x7f4edc1553a0 2023-12-19T18:29:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.397+0000 7f4ed3fff640 1 -- 172.21.15.27:0/2970660678 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec8003520 con 0x7f4edc1553a0 2023-12-19T18:29:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.397+0000 7f4ee1d28640 1 -- 172.21.15.27:0/2970660678 >> v2:172.21.15.192:3300/0 conn(0x7f4edc15ee30 msgr2=0x7f4edc0c50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:19.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.397+0000 7f4ee1d28640 1 --2- 172.21.15.27:0/2970660678 >> v2:172.21.15.192:3300/0 conn(0x7f4edc15ee30 0x7f4edc0c50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.397+0000 7f4ee1d28640 1 -- 172.21.15.27:0/2970660678 >> v1:172.21.15.27:6789/0 conn(0x7f4edc1553a0 legacy=0x7f4edc15e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:19.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.397+0000 7f4ee1d28640 1 -- 172.21.15.27:0/2970660678 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4edc0cb910 con 0x7f4edc0afeb0 2023-12-19T18:29:19.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.398+0000 7f4ee1d28640 1 --2- 172.21.15.27:0/2970660678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4edc0afeb0 0x7f4edc0c39d0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f4edc06d4a0 tx=0x7f4ed402be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1f6c422f7b602fa server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:19.398 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.399+0000 7f4ed3fff640 1 -- 172.21.15.27:0/2970660678 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed402cbf0 con 0x7f4edc0afeb0 2023-12-19T18:29:19.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.399+0000 7f4ed3fff640 1 -- 172.21.15.27:0/2970660678 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed402cd50 con 0x7f4edc0afeb0 2023-12-19T18:29:19.399 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.399+0000 7f4ed3fff640 1 -- 172.21.15.27:0/2970660678 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4034660 con 0x7f4edc0afeb0 2023-12-19T18:29:19.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.399+0000 7f4ee2529640 1 -- 172.21.15.27:0/2970660678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4edc0afeb0 msgr2=0x7f4edc0c39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:19.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.400+0000 7f4ee2529640 1 --2- 172.21.15.27:0/2970660678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4edc0afeb0 0x7f4edc0c39d0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f4edc06d4a0 tx=0x7f4ed402be40 comp rx=0 tx=0).stop 2023-12-19T18:29:19.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.400+0000 7f4ee2529640 1 -- 172.21.15.27:0/2970660678 shutdown_connections 2023-12-19T18:29:19.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.400+0000 7f4ee2529640 1 --2- 172.21.15.27:0/2970660678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4edc0afeb0 0x7f4edc0c39d0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.400+0000 7f4ee2529640 1 --2- 172.21.15.27:0/2970660678 >> v2:172.21.15.192:3300/0 conn(0x7f4edc15ee30 0x7f4edc0c50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.400+0000 7f4ee2529640 1 -- 172.21.15.27:0/2970660678 >> 172.21.15.27:0/2970660678 conn(0x7f4edc069c70 msgr2=0x7f4edc06a050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:19.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.400+0000 7f4ee2529640 1 -- 172.21.15.27:0/2970660678 shutdown_connections 2023-12-19T18:29:19.400 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.400+0000 7f4ee2529640 1 -- 172.21.15.27:0/2970660678 wait complete. 2023-12-19T18:29:19.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.402+0000 7f4ee2529640 1 Processor -- start 2023-12-19T18:29:19.401 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.402+0000 7f4ee2529640 1 -- start start 2023-12-19T18:29:19.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.403+0000 7f4ee2529640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4edc0afeb0 0x7f4edc1f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:19.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.403+0000 7f4ee2529640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4edc1553a0 0x7f4edc0c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:19.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.403+0000 7f4ee2529640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4edc15ee30 0x7f4edc0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:19.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.403+0000 7f4ee2529640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc0c5c20 con 0x7f4edc0afeb0 2023-12-19T18:29:19.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.403+0000 7f4ee2529640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc1f39b0 con 0x7f4edc1553a0 2023-12-19T18:29:19.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.403+0000 7f4ee2529640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4edc06b760 con 0x7f4edc15ee30 2023-12-19T18:29:19.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.403+0000 7f4ee1527640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4edc0afeb0 0x7f4edc1f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:19.404 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.403+0000 7f4ee1527640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4edc0afeb0 0x7f4edc1f3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55778/0 (socket says 172.21.15.27:55778) 2023-12-19T18:29:19.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.403+0000 7f4ee1527640 1 -- 172.21.15.27:0/3147115586 learned_addr learned my addr 172.21.15.27:0/3147115586 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:19.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.403+0000 7f4ee1d28640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4edc15ee30 0x7f4edc0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:19.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.403+0000 7f4ee0d26640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4edc1553a0 0x7f4edc0c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:19.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.403+0000 7f4ee1527640 1 -- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4edc15ee30 msgr2=0x7f4edc0ce8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:19.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.403+0000 7f4ee1527640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4edc15ee30 0x7f4edc0ce8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.404+0000 7f4ee1527640 1 -- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4edc1553a0 msgr2=0x7f4edc0c82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:19.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.404+0000 7f4ee1527640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4edc1553a0 0x7f4edc0c82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.404+0000 7f4ee1527640 1 -- 172.21.15.27:0/3147115586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed4009a70 con 0x7f4edc0afeb0 2023-12-19T18:29:19.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.404+0000 7f4ee1d28640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4edc15ee30 0x7f4edc0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:19.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.404+0000 7f4ee0d26640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4edc1553a0 0x7f4edc0c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:19.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.404+0000 7f4ee1527640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4edc0afeb0 0x7f4edc1f3470 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800c500 tx=0x7f4ed800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:19.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.404+0000 7f4ed1ffb640 1 -- 172.21.15.27:0/3147115586 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8005ce0 con 0x7f4edc0afeb0 2023-12-19T18:29:19.405 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.404+0000 7f4ed1ffb640 1 -- 172.21.15.27:0/3147115586 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed8005e40 con 0x7f4edc0afeb0 2023-12-19T18:29:19.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.404+0000 7f4ed1ffb640 1 -- 172.21.15.27:0/3147115586 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8007330 con 0x7f4edc0afeb0 2023-12-19T18:29:19.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.404+0000 7f4ee2529640 1 -- 172.21.15.27:0/3147115586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4edc06ba40 con 0x7f4edc0afeb0 2023-12-19T18:29:19.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.404+0000 7f4ee2529640 1 -- 172.21.15.27:0/3147115586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4edc06c000 con 0x7f4edc0afeb0 2023-12-19T18:29:19.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.406+0000 7f4ee2529640 1 -- 172.21.15.27:0/3147115586 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4edc0cc660 con 0x7f4edc0afeb0 2023-12-19T18:29:19.406 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.407+0000 7f4ed1ffb640 1 -- 172.21.15.27:0/3147115586 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f4ed8007550 con 0x7f4edc0afeb0 2023-12-19T18:29:19.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.408+0000 7f4ed1ffb640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ec006a430 0x7f4ec006c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:19.407 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.408+0000 7f4ed1ffb640 1 -- 172.21.15.27:0/3147115586 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f4ed8087ce0 con 0x7f4edc0afeb0 2023-12-19T18:29:19.408 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.408+0000 7f4ee0d26640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ec006a430 0x7f4ec006c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:19.409 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.409+0000 7f4ee0d26640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ec006a430 0x7f4ec006c8f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00a870 tx=0x7f4ecc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:19.412 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.412+0000 7f4ed1ffb640 1 -- 172.21.15.27:0/3147115586 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f4edc0cc660 con 0x7f4edc0afeb0 2023-12-19T18:29:19.605 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.604+0000 7f4ee2529640 1 -- 172.21.15.27:0/3147115586 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4edc0c5730 con 0x7f4ec006a430 2023-12-19T18:29:19.611 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.611+0000 7f4ed1ffb640 1 -- 172.21.15.27:0/3147115586 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47412 (secure 0 0 0) 0x7f4ecc00c7d0 con 0x7f4ec006a430 2023-12-19T18:29:19.611 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:29:19.612 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:29:19.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.613+0000 7f4ee2529640 1 -- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ec006a430 msgr2=0x7f4ec006c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:19.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.613+0000 7f4ee2529640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ec006a430 0x7f4ec006c8f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4ecc00a870 tx=0x7f4ecc008040 comp rx=0 tx=0).stop 2023-12-19T18:29:19.613 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.613+0000 7f4ee2529640 1 -- 172.21.15.27:0/3147115586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4edc0afeb0 msgr2=0x7f4edc1f3470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:19.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.613+0000 7f4ee2529640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4edc0afeb0 0x7f4edc1f3470 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800c500 tx=0x7f4ed800c9c0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.613+0000 7f4ee2529640 1 -- 172.21.15.27:0/3147115586 shutdown_connections 2023-12-19T18:29:19.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.613+0000 7f4ee2529640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f4ec006a430 0x7f4ec006c8f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.613+0000 7f4ee2529640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f4edc15ee30 0x7f4edc0ce8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.613+0000 7f4ee2529640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f4edc1553a0 0x7f4edc0c82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.613+0000 7f4ee2529640 1 --2- 172.21.15.27:0/3147115586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4edc0afeb0 0x7f4edc1f3470 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:19.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.613+0000 7f4ee2529640 1 -- 172.21.15.27:0/3147115586 >> 172.21.15.27:0/3147115586 conn(0x7f4edc069c70 msgr2=0x7f4edc15a5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:19.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.613+0000 7f4ee2529640 1 -- 172.21.15.27:0/3147115586 shutdown_connections 2023-12-19T18:29:19.614 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:19.614+0000 7f4ee2529640 1 -- 172.21.15.27:0/3147115586 wait complete. 2023-12-19T18:29:19.632 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":639,"stamp":"2023-12-19T18:29:18.023364+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10484,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552015,"statfs":{"total":1600000000,"available":1589264040,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.053773"},"pg_stats":[{"pgid":"1.0","version":"16'117","reported_seq":336,"reported_epoch":209,"state":"active+clean","last_fresh":"2023-12-19T18:29:05.956617+0000","last_change":"2023-12-19T18:14:45.366461+0000","last_active":"2023-12-19T18:29:05.956617+0000","last_peered":"2023-12-19T18:29:05.956617+0000","last_clean":"2023-12-19T18:29:05.956617+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:29:05.956617+0000","last_undegraded":"2023-12-19T18:29:05.956617+0000","last_fullsized":"2023-12-19T18:29:05.956617+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-19T18:14:44.066064+0000","last_clean_scrub_stamp":"2023-12-19T18:14:44.066064+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-20T21:36:05.275013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.22,"5min":1.1859999999999999,"15min":1.1799999999999999},"min":{"1min":0.67000000000000004,"5min":0.59299999999999997,"15min":0.45900000000000002},"max":{"1min":1.5369999999999999,"5min":1.756,"15min":5.6879999999999997},"last":1.54},{"interface":"front","average":{"1min":1.274,"5min":1.236,"15min":1.228},"min":{"1min":0.78600000000000003,"5min":0.71899999999999997,"15min":0.55800000000000005},"max":{"1min":1.6599999999999999,"5min":1.7030000000000001,"15min":1.79},"last":1.5009999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.133,"5min":1.179,"15min":1.23},"min":{"1min":0.75600000000000001,"5min":0.65800000000000003,"15min":0.54400000000000004},"max":{"1min":1.6970000000000001,"5min":1.6970000000000001,"15min":10.926},"last":1.264},{"interface":"front","average":{"1min":1.1619999999999999,"5min":1.26,"15min":1.268},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.67900000000000005},"max":{"1min":1.534,"5min":1.736,"15min":10.904999999999999},"last":1.4350000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.1379999999999999,"15min":1.117},"min":{"1min":0.90700000000000003,"5min":0.66000000000000003,"15min":0.61499999999999999},"max":{"1min":1.7130000000000001,"5min":1.7130000000000001,"15min":3.1960000000000002},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1830000000000001,"15min":1.2},"min":{"1min":0.91500000000000004,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.5469999999999999,"5min":1.6799999999999999,"15min":6.008},"last":1.371}]},{"osd":3,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.333,"5min":1.2749999999999999,"15min":1.2290000000000001},"min":{"1min":0.96899999999999997,"5min":0.79000000000000004,"15min":0.61399999999999999},"max":{"1min":1.8160000000000001,"5min":1.8160000000000001,"15min":3.0379999999999998},"last":1.468},{"interface":"front","average":{"1min":1.3220000000000001,"5min":1.3420000000000001,"15min":1.3400000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69599999999999995},"max":{"1min":1.6220000000000001,"5min":1.772,"15min":5.8120000000000003},"last":1.6459999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.202,"15min":1.1850000000000001},"min":{"1min":0.81299999999999994,"5min":0.61099999999999999,"15min":0.47899999999999998},"max":{"1min":1.8340000000000001,"5min":1.847,"15min":4.0110000000000001},"last":1.147},{"interface":"front","average":{"1min":1.2529999999999999,"5min":1.256,"15min":1.256},"min":{"1min":0.94699999999999995,"5min":0.624,"15min":0.57599999999999996},"max":{"1min":1.579,"5min":1.8,"15min":3.3180000000000001},"last":1.347}]},{"osd":5,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.3879999999999999,"5min":1.3049999999999999,"15min":1.3380000000000001},"min":{"1min":0.995,"5min":0.68999999999999995,"15min":0.53800000000000003},"max":{"1min":1.7789999999999999,"5min":1.8140000000000001,"15min":7.6429999999999998},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.4059999999999999,"5min":1.3620000000000001,"15min":1.3660000000000001},"min":{"1min":0.96899999999999997,"5min":0.627,"15min":0.53500000000000003},"max":{"1min":1.9359999999999999,"5min":1.9359999999999999,"15min":6.0839999999999996},"last":1.3169999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:28:15 2023","interfaces":[{"interface":"back","average":{"1min":1.5580000000000001,"5min":1.5129999999999999,"15min":1.506},"min":{"1min":1.175,"5min":0.70899999999999996,"15min":0.59899999999999998},"max":{"1min":2.2349999999999999,"5min":2.2349999999999999,"15min":3.1309999999999998},"last":1.587},{"interface":"front","average":{"1min":1.522,"5min":1.4690000000000001,"15min":1.4330000000000001},"min":{"1min":1.131,"5min":0.73199999999999998,"15min":0.56100000000000005},"max":{"1min":2.0289999999999999,"5min":2.0289999999999999,"15min":4.1479999999999997},"last":1.2050000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.1060000000000001,"15min":1.081},"min":{"1min":0.70699999999999996,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.8129999999999999},"last":1.4790000000000001},{"interface":"front","average":{"1min":1.141,"5min":1.1439999999999999,"15min":1.131},"min":{"1min":0.73199999999999998,"5min":0.67900000000000005,"15min":0.55600000000000005},"max":{"1min":1.5700000000000001,"5min":1.587,"15min":3.5529999999999999},"last":1.256}]},{"osd":1,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1439999999999999,"15min":1.135},"min":{"1min":0.83299999999999996,"5min":0.70099999999999996,"15min":0.68200000000000005},"max":{"1min":1.4470000000000001,"5min":1.7909999999999999,"15min":4.5540000000000003},"last":1.093},{"interface":"front","average":{"1min":1.278,"5min":1.2230000000000001,"15min":1.1870000000000001},"min":{"1min":0.84199999999999997,"5min":0.78900000000000003,"15min":0.60299999999999998},"max":{"1min":1.619,"5min":1.619,"15min":1.885},"last":1.5409999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2749999999999999,"15min":1.2210000000000001},"min":{"1min":0.85399999999999998,"5min":0.79200000000000004,"15min":0.626},"max":{"1min":1.645,"5min":1.6479999999999999,"15min":1.821},"last":1.645},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2370000000000001,"15min":1.214},"min":{"1min":0.85699999999999998,"5min":0.62,"15min":0.60899999999999999},"max":{"1min":1.607,"5min":1.7669999999999999,"15min":3.7909999999999999},"last":1.0640000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2989999999999999,"15min":1.2629999999999999},"min":{"1min":0.96299999999999997,"5min":0.71799999999999997,"15min":0.67800000000000005},"max":{"1min":1.776,"5min":1.804,"15min":3.8820000000000001},"last":1.173},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2849999999999999,"15min":1.2170000000000001},"min":{"1min":1.054,"5min":0.71199999999999997,"15min":0.69899999999999995},"max":{"1min":1.738,"5min":1.738,"15min":4.6189999999999998},"last":1.3169999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.302,"15min":1.282},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.65600000000000003},"max":{"1min":1.6639999999999999,"5min":1.7529999999999999,"15min":3.0680000000000001},"last":1.5960000000000001},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.383,"15min":1.395},"min":{"1min":0.86499999999999999,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.728,"5min":2.1040000000000001,"15min":3.7000000000000002},"last":1.421}]},{"osd":5,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2809999999999999,"15min":1.254},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.65500000000000003},"max":{"1min":1.5389999999999999,"5min":1.9510000000000001,"15min":4.6680000000000001},"last":1.228},{"interface":"front","average":{"1min":1.325,"5min":1.403,"15min":1.3979999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.626},"max":{"1min":1.7330000000000001,"5min":1.794,"15min":4.1159999999999997},"last":1.6930000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.4359999999999999,"15min":1.409},"min":{"1min":0.98899999999999999,"5min":0.65700000000000003,"15min":0.61899999999999999},"max":{"1min":1.7110000000000001,"5min":2.02,"15min":3.9670000000000001},"last":1.345},{"interface":"front","average":{"1min":1.357,"5min":1.4379999999999999,"15min":1.4159999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.752,"5min":2.2149999999999999,"15min":5.6639999999999997},"last":1.3859999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575024,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1742,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193569,"statfs":{"total":200000000,"available":198215229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.284,"15min":1.218},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.61899999999999999},"max":{"1min":1.6459999999999999,"5min":2.089,"15min":2.089},"last":1.6459999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2949999999999999,"15min":1.256},"min":{"1min":0.71699999999999997,"5min":0.69399999999999995,"15min":0.61099999999999999},"max":{"1min":1.5329999999999999,"5min":1.641,"15min":1.913},"last":1.476}]},{"osd":1,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.149,"15min":1.145},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.65700000000000003},"max":{"1min":1.532,"5min":1.851,"15min":1.9490000000000001},"last":1.264},{"interface":"front","average":{"1min":1.171,"5min":1.198,"15min":1.1519999999999999},"min":{"1min":0.77000000000000002,"5min":0.71399999999999997,"15min":0.505},"max":{"1min":1.6459999999999999,"5min":1.899,"15min":1.899},"last":0.98899999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.2050000000000001,"15min":1.157},"min":{"1min":0.77600000000000002,"5min":0.73999999999999999,"15min":0.59499999999999997},"max":{"1min":1.7030000000000001,"5min":1.806,"15min":1.806},"last":0.875},{"interface":"front","average":{"1min":1.208,"5min":1.268,"15min":1.2310000000000001},"min":{"1min":0.82999999999999996,"5min":0.74099999999999999,"15min":0.55400000000000005},"max":{"1min":1.857,"5min":1.857,"15min":2.036},"last":1.3660000000000001}]},{"osd":3,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2030000000000001,"15min":1.171},"min":{"1min":0.82299999999999995,"5min":0.76600000000000001,"15min":0.68899999999999995},"max":{"1min":1.5069999999999999,"5min":1.589,"15min":1.7450000000000001},"last":0.92900000000000005},{"interface":"front","average":{"1min":1.373,"5min":1.371,"15min":1.3220000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.58799999999999997},"max":{"1min":1.7030000000000001,"5min":2.0369999999999999,"15min":2.0590000000000002},"last":1.5189999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2589999999999999,"15min":1.2709999999999999},"min":{"1min":0.70199999999999996,"5min":0.52300000000000002,"15min":0.46700000000000003},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":2.0329999999999999},"last":1.585},{"interface":"front","average":{"1min":1.2,"5min":1.141,"15min":1.167},"min":{"1min":0.76000000000000001,"5min":0.64500000000000002,"15min":0.57099999999999995},"max":{"1min":1.6870000000000001,"5min":1.8839999999999999,"15min":1.901},"last":1.0249999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.401,"15min":1.4139999999999999},"min":{"1min":0.64100000000000001,"5min":0.56699999999999995,"15min":0.52400000000000002},"max":{"1min":1.927,"5min":2.1579999999999999,"15min":2.1589999999999998},"last":1.694},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.3160000000000001,"15min":1.3049999999999999},"min":{"1min":0.94699999999999995,"5min":0.58299999999999996,"15min":0.49299999999999999},"max":{"1min":2.9039999999999999,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.3220000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.284,"15min":1.282},"min":{"1min":0.60299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.752,"5min":2.036,"15min":2.036},"last":1.0960000000000001},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.444,"15min":1.448},"min":{"1min":0.89300000000000002,"5min":0.67100000000000004,"15min":0.623},"max":{"1min":1.823,"5min":1.8540000000000001,"15min":2.117},"last":1.4359999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0609999999999999,"15min":1.028},"min":{"1min":0.68700000000000006,"5min":0.64700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4530000000000001,"5min":1.494,"15min":1.972},"last":0.98099999999999998},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.097,"15min":1.0760000000000001},"min":{"1min":0.61399999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.611,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.83799999999999997}]},{"osd":1,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.208,"15min":1.171},"min":{"1min":0.65900000000000003,"5min":0.621,"15min":0.621},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.357},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3029999999999999,"15min":1.2470000000000001},"min":{"1min":0.92400000000000004,"5min":0.64700000000000002,"15min":0.61699999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.875},"last":1.258}]},{"osd":2,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1659999999999999,"15min":1.139},"min":{"1min":0.70299999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.196},{"interface":"front","average":{"1min":1.087,"5min":1.1679999999999999,"15min":1.147},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.70499999999999996},"max":{"1min":1.5029999999999999,"5min":1.8149999999999999,"15min":2.0270000000000001},"last":1.133}]},{"osd":3,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.258,"15min":1.274},"min":{"1min":0.96499999999999997,"5min":0.754,"15min":0.754},"max":{"1min":1.601,"5min":1.6719999999999999,"15min":1.946},"last":1.103},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.357,"15min":1.341},"min":{"1min":0.89000000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.927},"last":1.3160000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2490000000000001,"15min":1.252},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.44500000000000001},"max":{"1min":1.542,"5min":1.968,"15min":1.968},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.3740000000000001,"15min":1.3779999999999999},"min":{"1min":0.77500000000000002,"5min":0.59099999999999997,"15min":0.504},"max":{"1min":1.756,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":1.4259999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3049999999999999,"15min":1.3069999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.432},"max":{"1min":1.851,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.49,"5min":1.4419999999999999,"15min":1.4299999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0339999999999998,"5min":2.149,"15min":2.149},"last":1.637}]},{"osd":7,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.3859999999999999,"15min":1.3740000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.56399999999999995},"max":{"1min":2.0720000000000001,"5min":2.702,"15min":2.702},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.4330000000000001,"15min":1.4279999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.9930000000000001,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.456}]}]},{"osd":3,"up_from":13,"seq":55834575025,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.90900000000000003,"15min":0.92200000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35299999999999998},"max":{"1min":1.754,"5min":1.754,"15min":3.4350000000000001},"last":1.4950000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.038,"15min":0.99299999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.39500000000000002},"max":{"1min":1.591,"5min":1.8109999999999999,"15min":4.3490000000000002},"last":0.96799999999999997}]},{"osd":1,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.01,"15min":1.024},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.41299999999999998},"max":{"1min":1.8520000000000001,"5min":1.9690000000000001,"15min":5.2510000000000003},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.4299999999999999,"5min":1.643,"15min":5.4119999999999999},"last":1.022}]},{"osd":2,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1899999999999999,"15min":1.236},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.46000000000000002},"max":{"1min":2.1059999999999999,"5min":2.4649999999999999,"15min":13.833},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.304,"15min":1.3520000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8460000000000001,"5min":2.379,"15min":16.748999999999999},"last":1.135}]},{"osd":4,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3080000000000001,"15min":1.29},"min":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":0.65600000000000003},"max":{"1min":1.75,"5min":2.0150000000000001,"15min":5.1790000000000003},"last":1.3049999999999999},{"interface":"front","average":{"1min":1.343,"5min":1.367,"15min":1.3919999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.73199999999999998},"max":{"1min":1.929,"5min":2.0819999999999999,"15min":4.569},"last":1.821}]},{"osd":5,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.4339999999999999,"15min":1.429},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.76000000000000001},"max":{"1min":1.925,"5min":2.0449999999999999,"15min":5.3289999999999997},"last":1.774},{"interface":"front","average":{"1min":1.327,"5min":1.3859999999999999,"15min":1.347},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.69999999999999996},"max":{"1min":2.004,"5min":2.1469999999999998,"15min":4.6369999999999996},"last":1.544}]},{"osd":6,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.454,"15min":1.4510000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.81599999999999995},"max":{"1min":1.9790000000000001,"5min":2.2040000000000002,"15min":4.5039999999999996},"last":1.601},{"interface":"front","average":{"1min":1.516,"5min":1.5189999999999999,"15min":1.5089999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.80500000000000005},"max":{"1min":2.254,"5min":2.254,"15min":5.3739999999999997},"last":1.6439999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.4990000000000001,"15min":1.454},"min":{"1min":0.98899999999999999,"5min":0.89700000000000002,"15min":0.73199999999999998},"max":{"1min":2.1389999999999998,"5min":2.2869999999999999,"15min":5.1349999999999998},"last":1.4099999999999999},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.5369999999999999,"15min":1.5489999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.0550000000000002,"5min":2.351,"15min":4.4619999999999997},"last":1.712}]}]},{"osd":2,"up_from":13,"seq":55834575024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.92600000000000005,"15min":0.97499999999999998},"min":{"1min":0.48099999999999998,"5min":0.43099999999999999,"15min":0.34399999999999997},"max":{"1min":1.5309999999999999,"5min":2.2799999999999998,"15min":9.2550000000000008},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.04,"15min":1.0189999999999999},"min":{"1min":0.45900000000000002,"5min":0.38400000000000001,"15min":0.33900000000000002},"max":{"1min":1.4399999999999999,"5min":1.8049999999999999,"15min":4.7850000000000001},"last":0.47099999999999997}]},{"osd":1,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.93999999999999995,"15min":0.98999999999999999},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.4039999999999999,"5min":2.0169999999999999,"15min":8.2720000000000002},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.101,"15min":1.1040000000000001},"min":{"1min":0.61199999999999999,"5min":0.54200000000000004,"15min":0.46600000000000003},"max":{"1min":1.339,"5min":1.8460000000000001,"15min":5.0620000000000003},"last":0.56200000000000006}]},{"osd":3,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.2669999999999999,"15min":1.226},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5940000000000001,"5min":2.8159999999999998,"15min":8.3109999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1160000000000001,"15min":1.163},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.41799999999999998},"max":{"1min":1.7869999999999999,"5min":2.6240000000000001,"15min":8.0690000000000008},"last":0.68899999999999995}]},{"osd":4,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3819999999999999,"15min":1.395},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9850000000000001,"5min":2.0059999999999998,"15min":5.3929999999999998},"last":0.97099999999999997},{"interface":"front","average":{"1min":1.399,"5min":1.458,"15min":1.46},"min":{"1min":1.0209999999999999,"5min":0.85799999999999998,"15min":0.78300000000000003},"max":{"1min":2.3079999999999998,"5min":2.4300000000000002,"15min":7.3570000000000002},"last":0.86799999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3720000000000001,"15min":1.4059999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.613,"5min":2.1680000000000001,"15min":8.1839999999999993},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.415,"5min":1.385,"15min":1.391},"min":{"1min":0.90000000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":5.165},"last":0.83499999999999996}]},{"osd":6,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.413,"15min":1.4219999999999999},"min":{"1min":0.99399999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.96,"5min":2.4630000000000001,"15min":5.1059999999999999},"last":0.98799999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.468,"15min":1.48},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77000000000000002},"max":{"1min":1.7709999999999999,"5min":2.7559999999999998,"15min":8.1370000000000005},"last":0.88600000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.504,"15min":1.5089999999999999},"min":{"1min":0.91800000000000004,"5min":0.80000000000000004,"15min":0.78200000000000003},"max":{"1min":2.0870000000000002,"5min":2.3959999999999999,"15min":5.1449999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5629999999999999,"15min":1.5680000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86499999999999999},"max":{"1min":1.8720000000000001,"5min":2.6320000000000001,"15min":8.2300000000000004},"last":0.77200000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575025,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1742,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193569,"statfs":{"total":200000000,"available":198215229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93200000000000005,"15min":0.874},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.34699999999999998},"max":{"1min":1.472,"5min":2.262,"15min":2.262},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.014,"15min":0.96299999999999997},"min":{"1min":0.60599999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4970000000000001,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":0.93000000000000005}]},{"osd":2,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.02,"15min":1.0309999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.41399999999999998},"max":{"1min":1.621,"5min":1.621,"15min":2.5739999999999998},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":1.0069999999999999,"15min":0.97799999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.38500000000000001},"max":{"1min":1.276,"5min":1.718,"15min":1.845},"last":0.89800000000000002}]},{"osd":3,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.3300000000000001,"15min":1.2569999999999999},"min":{"1min":0.624,"5min":0.53000000000000003,"15min":0.48599999999999999},"max":{"1min":1.603,"5min":2.089,"15min":2.137},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.099,"5min":1.2350000000000001,"15min":1.198},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.44900000000000001},"max":{"1min":1.6659999999999999,"5min":2.0720000000000001,"15min":2.3959999999999999},"last":1.2769999999999999}]},{"osd":4,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3140000000000001,"15min":1.3120000000000001},"min":{"1min":0.83099999999999996,"5min":0.80500000000000005,"15min":0.69099999999999995},"max":{"1min":1.694,"5min":2.1859999999999999,"15min":2.444},"last":1.331},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.381,"15min":1.3520000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.71099999999999997},"max":{"1min":1.665,"5min":1.8859999999999999,"15min":2.6440000000000001},"last":1.3540000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.425,"15min":1.371},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":2.5459999999999998},"last":1.4370000000000001},{"interface":"front","average":{"1min":1.339,"5min":1.431,"15min":1.4330000000000001},"min":{"1min":1.0249999999999999,"5min":0.85899999999999999,"15min":0.70499999999999996},"max":{"1min":1.8240000000000001,"5min":1.9339999999999999,"15min":2.6179999999999999},"last":1.577}]},{"osd":6,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.466,"15min":1.4139999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":2.1619999999999999,"5min":2.274,"15min":2.7000000000000002},"last":1.1930000000000001},{"interface":"front","average":{"1min":1.379,"5min":1.5,"15min":1.478},"min":{"1min":0.872,"5min":0.872,"15min":0.76400000000000001},"max":{"1min":2.153,"5min":2.153,"15min":2.895},"last":1.502}]},{"osd":7,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4490000000000001,"15min":1.4470000000000001},"min":{"1min":0.92600000000000005,"5min":0.83199999999999996,"15min":0.69099999999999995},"max":{"1min":2.044,"5min":2.044,"15min":9.4830000000000005},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.5600000000000001,"15min":1.546},"min":{"1min":0.93899999999999995,"5min":0.80800000000000005,"15min":0.71899999999999997},"max":{"1min":2.198,"5min":2.3420000000000001,"15min":9.4109999999999996},"last":1.369}]}]},{"osd":0,"up_from":13,"seq":55834575024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.92200000000000004,"15min":0.92100000000000004},"min":{"1min":0.60699999999999998,"5min":0.34499999999999997,"15min":0.311},"max":{"1min":1.425,"5min":1.671,"15min":2.3199999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":1.0800000000000001,"5min":1.1100000000000001,"15min":1.0720000000000001},"min":{"1min":0.58899999999999997,"5min":0.48699999999999999,"15min":0.42799999999999999},"max":{"1min":1.3480000000000001,"5min":2.2170000000000001,"15min":2.4180000000000001},"last":0.67100000000000004}]},{"osd":2,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":1.0289999999999999,"15min":0.996},"min":{"1min":0.69399999999999995,"5min":0.46700000000000003,"15min":0.434},"max":{"1min":1.4550000000000001,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":0.52700000000000002},{"interface":"front","average":{"1min":1.113,"5min":1.0609999999999999,"15min":0.996},"min":{"1min":0.64000000000000001,"5min":0.42699999999999999,"15min":0.41099999999999998},"max":{"1min":1.6870000000000001,"5min":2.3340000000000001,"15min":2.7810000000000001},"last":0.73199999999999998}]},{"osd":3,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":1.032,"15min":1.0249999999999999},"min":{"1min":0.63300000000000001,"5min":0.44,"15min":0.38900000000000001},"max":{"1min":1.5589999999999999,"5min":1.8200000000000001,"15min":2.5190000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":1.3100000000000001,"5min":1.349,"15min":1.2769999999999999},"min":{"1min":0.81299999999999994,"5min":0.65100000000000002,"15min":0.47499999999999998},"max":{"1min":1.6759999999999999,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":0.97199999999999998}]},{"osd":4,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3240000000000001,"5min":1.323,"15min":1.2749999999999999},"min":{"1min":0.94499999999999995,"5min":0.67600000000000005,"15min":0.63200000000000001},"max":{"1min":1.7290000000000001,"5min":2.1920000000000002,"15min":2.5529999999999999},"last":1.2210000000000001},{"interface":"front","average":{"1min":1.27,"5min":1.2789999999999999,"15min":1.28},"min":{"1min":0.81899999999999995,"5min":0.70399999999999996,"15min":0.67600000000000005},"max":{"1min":1.6699999999999999,"5min":2.056,"15min":2.9529999999999998},"last":1.004}]},{"osd":5,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3320000000000001,"5min":1.3460000000000001,"15min":1.3919999999999999},"min":{"1min":0.92800000000000005,"5min":0.70699999999999996,"15min":0.64500000000000002},"max":{"1min":1.542,"5min":1.97,"15min":16.298999999999999},"last":1.1180000000000001},{"interface":"front","average":{"1min":1.4259999999999999,"5min":1.387,"15min":1.3360000000000001},"min":{"1min":1.115,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.641,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":0.91300000000000003}]},{"osd":6,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.496,"5min":1.496,"15min":1.4470000000000001},"min":{"1min":0.93300000000000005,"5min":0.79800000000000004,"15min":0.71699999999999997},"max":{"1min":1.744,"5min":2.4169999999999998,"15min":2.5910000000000002},"last":1.137},{"interface":"front","average":{"1min":1.4530000000000001,"5min":1.4810000000000001,"15min":1.4179999999999999},"min":{"1min":1.097,"5min":0.745,"15min":0.745},"max":{"1min":1.8,"5min":2.1709999999999998,"15min":2.46},"last":1.292}]},{"osd":7,"last update":"Tue Dec 19 18:28:12 2023","interfaces":[{"interface":"back","average":{"1min":1.5029999999999999,"5min":1.4910000000000001,"15min":1.454},"min":{"1min":1.052,"5min":0.72099999999999997,"15min":0.70899999999999996},"max":{"1min":1.895,"5min":2.2549999999999999,"15min":2.4969999999999999},"last":1.351},{"interface":"front","average":{"1min":1.381,"5min":1.407,"15min":1.415},"min":{"1min":1.077,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.6439999999999999,"5min":1.9370000000000001,"15min":3.1429999999999998},"last":1.069}]}]}],"pool_statfs":[]}} 2023-12-19T18:29:19.635 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-12-19T18:14:44.066064+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=19, tm_hour=18, tm_min=14, tm_sec=44, tm_wday=1, tm_yday=353, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=19, tm_hour=18, tm_min=29, tm_sec=12, tm_wday=1, tm_yday=353, tm_isdst=0) 2023-12-19T18:29:19.636 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-12-19T18:29:39.637 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-19T18:29:39.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.884+0000 7f698c0ab640 1 Processor -- start 2023-12-19T18:29:39.884 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.884+0000 7f698c0ab640 1 -- start start 2023-12-19T18:29:39.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.885+0000 7f698c0ab640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840cbce0 0x7f69840ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:39.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.885+0000 7f698c0ab640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f69840adc20 0x7f698406b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:39.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.885+0000 7f698c0ab640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69840cb850 con 0x7f698415af70 2023-12-19T18:29:39.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.885+0000 7f698c0ab640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f698415b350 con 0x7f69840adc20 2023-12-19T18:29:39.885 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.885+0000 7f698c0ab640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840cac40 con 0x7f69840cbce0 2023-12-19T18:29:39.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.886+0000 7f6989e47640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7f698415af70 0x7f69840ad510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56930/0 (socket says 172.21.15.27:56930) 2023-12-19T18:29:39.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.886+0000 7f6989e47640 1 -- 172.21.15.27:0/2777320707 learned_addr learned my addr 172.21.15.27:0/2777320707 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:39.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.886+0000 7f698a648640 1 --2- 172.21.15.27:0/2777320707 >> v2:172.21.15.192:3300/0 conn(0x7f69840adc20 0x7f698406b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:39.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.886+0000 7f6989646640 1 --2- 172.21.15.27:0/2777320707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840cbce0 0x7f69840ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:39.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.886+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 679081768 0 0) 0x7f69840cb850 con 0x7f698415af70 2023-12-19T18:29:39.886 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.886+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6970003520 con 0x7f698415af70 2023-12-19T18:29:39.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3692741316 0 0) 0x7f6970003520 con 0x7f698415af70 2023-12-19T18:29:39.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840cbce0 msgr2=0x7f69840ca540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:39.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 --2- 172.21.15.27:0/2777320707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840cbce0 0x7f69840ca540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:39.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 >> v2:172.21.15.192:3300/0 conn(0x7f69840adc20 msgr2=0x7f698406b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:39.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 --2- 172.21.15.27:0/2777320707 >> v2:172.21.15.192:3300/0 conn(0x7f69840adc20 0x7f698406b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:39.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840caf20 con 0x7f698415af70 2023-12-19T18:29:39.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7f6980001670 con 0x7f698415af70 2023-12-19T18:29:39.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 >> v1:172.21.15.27:6789/0 conn(0x7f698415af70 legacy=0x7f69840ad510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:39.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 --2- 172.21.15.27:0/2777320707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840cbce0 0x7f6970003970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:39.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 --2- 172.21.15.27:0/2777320707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6970003f40 0x7f69700047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:39.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 --2- 172.21.15.27:0/2777320707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6970004d00 0x7f6970005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:39.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f698415b350 con 0x7f6970003f40 2023-12-19T18:29:39.888 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840cac40 con 0x7f6970004d00 2023-12-19T18:29:39.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697000b980 con 0x7f69840cbce0 2023-12-19T18:29:39.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.887+0000 7f6989646640 1 --2- 172.21.15.27:0/2777320707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6970003f40 0x7f69700047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:39.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.888+0000 7f6989e47640 1 --2- 172.21.15.27:0/2777320707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840cbce0 0x7f6970003970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:39.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.888+0000 7f6989646640 1 -- 172.21.15.27:0/2777320707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840cbce0 msgr2=0x7f6970003970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:39.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.888+0000 7f6989646640 1 --2- 172.21.15.27:0/2777320707 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f69840cbce0 0x7f6970003970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:39.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.888+0000 7f6989646640 1 -- 172.21.15.27:0/2777320707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6970004d00 msgr2=0x7f6970005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:39.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.888+0000 7f6989646640 1 --2- 172.21.15.27:0/2777320707 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f6970004d00 0x7f6970005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:39.889 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.888+0000 7f6989646640 1 -- 172.21.15.27:0/2777320707 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6970003520 con 0x7f6970003f40 2023-12-19T18:29:39.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.889+0000 7f6989646640 1 --2- 172.21.15.27:0/2777320707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6970003f40 0x7f69700047c0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f6974009ac0 tx=0x7f69700057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8112e26bf98ba65 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:39.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.889+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697402c680 con 0x7f6970003f40 2023-12-19T18:29:39.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.889+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f697402c7e0 con 0x7f6970003f40 2023-12-19T18:29:39.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.889+0000 7f6988e45640 1 -- 172.21.15.27:0/2777320707 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697402cb70 con 0x7f6970003f40 2023-12-19T18:29:39.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.890+0000 7f698c0ab640 1 -- 172.21.15.27:0/2777320707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6970003f40 msgr2=0x7f69700047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:39.890 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.890+0000 7f698c0ab640 1 --2- 172.21.15.27:0/2777320707 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6970003f40 0x7f69700047c0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7f6974009ac0 tx=0x7f69700057f0 comp rx=0 tx=0).stop 2023-12-19T18:29:39.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.890+0000 7f698a648640 1 -- 172.21.15.27:0/2777320707 reap_dead start 2023-12-19T18:29:39.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.890+0000 7f698c0ab640 1 -- 172.21.15.27:0/2777320707 shutdown_connections 2023-12-19T18:29:39.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.890+0000 7f698c0ab640 1 -- 172.21.15.27:0/2777320707 >> 172.21.15.27:0/2777320707 conn(0x7f698406a330 msgr2=0x7f69840b0690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:39.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.890+0000 7f698c0ab640 1 -- 172.21.15.27:0/2777320707 shutdown_connections 2023-12-19T18:29:39.891 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.890+0000 7f698c0ab640 1 -- 172.21.15.27:0/2777320707 wait complete. 2023-12-19T18:29:39.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.892+0000 7f698c0ab640 1 Processor -- start 2023-12-19T18:29:39.892 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.892+0000 7f698c0ab640 1 -- start start 2023-12-19T18:29:39.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.893+0000 7f698c0ab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6970003f40 0x7f6984150210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:39.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.893+0000 7f698c0ab640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f698415ce60 0x7f69840ce010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:39.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.893+0000 7f698c0ab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f698415d2e0 0x7f69840af990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:39.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.893+0000 7f698c0ab640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840c8aa0 con 0x7f6970003f40 2023-12-19T18:29:39.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.893+0000 7f698c0ab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69840c8200 con 0x7f698415d2e0 2023-12-19T18:29:39.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.893+0000 7f698c0ab640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69840c84e0 con 0x7f698415ce60 2023-12-19T18:29:39.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.893+0000 7f6989e47640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6970003f40 0x7f6984150210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:39.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.893+0000 7f6989e47640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6970003f40 0x7f6984150210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59678/0 (socket says 172.21.15.27:59678) 2023-12-19T18:29:39.893 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.893+0000 7f6989e47640 1 -- 172.21.15.27:0/2352824439 learned_addr learned my addr 172.21.15.27:0/2352824439 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:39.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.894+0000 7f6989646640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f698415ce60 0x7f69840ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:39.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.894+0000 7f698a648640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f698415d2e0 0x7f69840af990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:39.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.894+0000 7f6989e47640 1 -- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f698415ce60 msgr2=0x7f69840ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:39.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.894+0000 7f6989e47640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f698415ce60 0x7f69840ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:39.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.894+0000 7f6989e47640 1 -- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f698415d2e0 msgr2=0x7f69840af990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:39.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.894+0000 7f6989e47640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f698415d2e0 0x7f69840af990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:39.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.894+0000 7f6989e47640 1 -- 172.21.15.27:0/2352824439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69840c87c0 con 0x7f6970003f40 2023-12-19T18:29:39.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.894+0000 7f6989e47640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6970003f40 0x7f6984150210 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f6980001050 tx=0x7f698000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:39.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.894+0000 7f697a7fc640 1 -- 172.21.15.27:0/2352824439 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698000c6c0 con 0x7f6970003f40 2023-12-19T18:29:39.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.894+0000 7f697a7fc640 1 -- 172.21.15.27:0/2352824439 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f698000c840 con 0x7f6970003f40 2023-12-19T18:29:39.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.894+0000 7f697a7fc640 1 -- 172.21.15.27:0/2352824439 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f698000cbf0 con 0x7f6970003f40 2023-12-19T18:29:39.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.894+0000 7f698c0ab640 1 -- 172.21.15.27:0/2352824439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69841f2a50 con 0x7f6970003f40 2023-12-19T18:29:39.895 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.895+0000 7f698c0ab640 1 -- 172.21.15.27:0/2352824439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69840c7fc0 con 0x7f6970003f40 2023-12-19T18:29:39.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.896+0000 7f698c0ab640 1 -- 172.21.15.27:0/2352824439 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6984068e90 con 0x7f6970003f40 2023-12-19T18:29:39.896 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.897+0000 7f697a7fc640 1 -- 172.21.15.27:0/2352824439 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f698000cd50 con 0x7f6970003f40 2023-12-19T18:29:39.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.898+0000 7f697a7fc640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f696806a430 0x7f696806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:39.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.898+0000 7f697a7fc640 1 -- 172.21.15.27:0/2352824439 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f6980088310 con 0x7f6970003f40 2023-12-19T18:29:39.898 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.899+0000 7f6989646640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f696806a430 0x7f696806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:39.899 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.900+0000 7f6989646640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f696806a430 0x7f696806c8f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6974036000 tx=0x7f6974030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:39.902 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:39.902+0000 7f697a7fc640 1 -- 172.21.15.27:0/2352824439 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f6980053bf0 con 0x7f6970003f40 2023-12-19T18:29:40.094 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.094+0000 7f698c0ab640 1 -- 172.21.15.27:0/2352824439 --> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f69840cb4e0 con 0x7f696806a430 2023-12-19T18:29:40.101 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.101+0000 7f697a7fc640 1 -- 172.21.15.27:0/2352824439 <== mgr.4111 v2:172.21.15.192:6832/83634 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+47473 (secure 0 0 0) 0x7f6974004040 con 0x7f696806a430 2023-12-19T18:29:40.102 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:29:40.102 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2023-12-19T18:29:40.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.103+0000 7f698c0ab640 1 -- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f696806a430 msgr2=0x7f696806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:40.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.103+0000 7f698c0ab640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f696806a430 0x7f696806c8f0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f6974036000 tx=0x7f6974030000 comp rx=0 tx=0).stop 2023-12-19T18:29:40.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.103+0000 7f698c0ab640 1 -- 172.21.15.27:0/2352824439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6970003f40 msgr2=0x7f6984150210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:40.103 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.103+0000 7f698c0ab640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6970003f40 0x7f6984150210 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f6980001050 tx=0x7f698000eb90 comp rx=0 tx=0).stop 2023-12-19T18:29:40.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.104+0000 7f698c0ab640 1 -- 172.21.15.27:0/2352824439 shutdown_connections 2023-12-19T18:29:40.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.104+0000 7f698c0ab640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f696806a430 0x7f696806c8f0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.104+0000 7f698c0ab640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f698415d2e0 0x7f69840af990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.104+0000 7f698c0ab640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f698415ce60 0x7f69840ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.104 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.104+0000 7f698c0ab640 1 --2- 172.21.15.27:0/2352824439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6970003f40 0x7f6984150210 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.104+0000 7f698c0ab640 1 -- 172.21.15.27:0/2352824439 >> 172.21.15.27:0/2352824439 conn(0x7f698406a330 msgr2=0x7f69840b3c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:40.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.104+0000 7f698c0ab640 1 -- 172.21.15.27:0/2352824439 shutdown_connections 2023-12-19T18:29:40.105 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.104+0000 7f698c0ab640 1 -- 172.21.15.27:0/2352824439 wait complete. 2023-12-19T18:29:40.125 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":650,"stamp":"2023-12-19T18:29:40.029579+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":6,"num_per_pool_omap_osds":0,"kb":1562500,"kb_used":10484,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":1552015,"statfs":{"total":1600000000,"available":1589264040,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003504"},"pg_stats":[{"pgid":"1.0","version":"210'119","reported_seq":350,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-12-19T18:29:14.774658+0000","last_change":"2023-12-19T18:29:14.774753+0000","last_active":"2023-12-19T18:29:14.774658+0000","last_peered":"2023-12-19T18:29:14.774658+0000","last_clean":"2023-12-19T18:29:14.774658+0000","last_became_active":"2023-12-19T18:14:45.365934+0000","last_became_peered":"2023-12-19T18:14:45.365934+0000","last_unstale":"2023-12-19T18:29:14.774658+0000","last_undegraded":"2023-12-19T18:29:14.774658+0000","last_fullsized":"2023-12-19T18:29:14.774658+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"210'119","last_scrub_stamp":"2023-12-19T18:29:14.774618+0000","last_deep_scrub":"210'119","last_deep_scrub_stamp":"2023-12-19T18:29:14.774618+0000","last_clean_scrub_stamp":"2023-12-19T18:29:14.774618+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:53:57.792436+0000","scrub_duration":0.0053338600000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2190000000000001,"5min":1.2210000000000001,"15min":1.1870000000000001},"min":{"1min":0.89200000000000002,"5min":0.66500000000000004,"15min":0.45900000000000002},"max":{"1min":1.54,"5min":1.756,"15min":5.6879999999999997},"last":1.3560000000000001},{"interface":"front","average":{"1min":1.2030000000000001,"5min":1.2430000000000001,"15min":1.232},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.55800000000000005},"max":{"1min":1.5269999999999999,"5min":1.7030000000000001,"15min":1.79},"last":1.381}]},{"osd":1,"last update":"Tue Dec 19 18:29:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0489999999999999,"5min":1.1779999999999999,"15min":1.2230000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.54400000000000004},"max":{"1min":1.4359999999999999,"5min":1.6970000000000001,"15min":10.926},"last":0.754},{"interface":"front","average":{"1min":1.198,"5min":1.2490000000000001,"15min":1.2689999999999999},"min":{"1min":0.72499999999999998,"5min":0.69899999999999995,"15min":0.67900000000000005},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":10.904999999999999},"last":1.2430000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:29:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1479999999999999,"15min":1.119},"min":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.61499999999999999},"max":{"1min":1.3480000000000001,"5min":1.7130000000000001,"15min":3.1960000000000002},"last":1.1559999999999999},{"interface":"front","average":{"1min":1.165,"5min":1.181,"15min":1.2010000000000001},"min":{"1min":0.81100000000000005,"5min":0.78700000000000003,"15min":0.64800000000000002},"max":{"1min":1.571,"5min":1.587,"15min":6.008},"last":1.089}]},{"osd":3,"last update":"Tue Dec 19 18:29:18 2023","interfaces":[{"interface":"back","average":{"1min":1.228,"5min":1.2789999999999999,"15min":1.2370000000000001},"min":{"1min":0.98199999999999998,"5min":0.79000000000000004,"15min":0.61399999999999999},"max":{"1min":1.468,"5min":1.8160000000000001,"15min":3.0379999999999998},"last":1.45},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.3360000000000001,"15min":1.343},"min":{"1min":0.83699999999999997,"5min":0.69899999999999995,"15min":0.69599999999999995},"max":{"1min":1.6459999999999999,"5min":1.772,"15min":5.8120000000000003},"last":1.208}]},{"osd":4,"last update":"Tue Dec 19 18:29:18 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.165,"15min":1.1910000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.47899999999999998},"max":{"1min":1.601,"5min":1.8340000000000001,"15min":4.0110000000000001},"last":1.5900000000000001},{"interface":"front","average":{"1min":1.2470000000000001,"5min":1.246,"15min":1.2589999999999999},"min":{"1min":0.623,"5min":0.623,"15min":0.57599999999999996},"max":{"1min":1.4790000000000001,"5min":1.8,"15min":3.3180000000000001},"last":1.306}]},{"osd":5,"last update":"Tue Dec 19 18:29:18 2023","interfaces":[{"interface":"back","average":{"1min":1.2749999999999999,"5min":1.304,"15min":1.337},"min":{"1min":0.74299999999999999,"5min":0.72799999999999998,"15min":0.53800000000000003},"max":{"1min":1.7290000000000001,"5min":1.804,"15min":7.6429999999999998},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.2829999999999999,"5min":1.339,"15min":1.3660000000000001},"min":{"1min":0.68100000000000005,"5min":0.65400000000000003,"15min":0.53500000000000003},"max":{"1min":1.8169999999999999,"5min":1.9359999999999999,"15min":6.0839999999999996},"last":1.4970000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:29:18 2023","interfaces":[{"interface":"back","average":{"1min":1.46,"5min":1.5009999999999999,"15min":1.51},"min":{"1min":0.83599999999999997,"5min":0.70899999999999996,"15min":0.59899999999999998},"max":{"1min":1.8919999999999999,"5min":2.2349999999999999,"15min":3.1309999999999998},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.46,"15min":1.4390000000000001},"min":{"1min":0.86199999999999999,"5min":0.73199999999999998,"15min":0.56100000000000005},"max":{"1min":1.77,"5min":2.0289999999999999,"15min":4.1479999999999997},"last":1.5620000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.147,"5min":1.1060000000000001,"15min":1.081},"min":{"1min":0.70699999999999996,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.8129999999999999},"last":1.095},{"interface":"front","average":{"1min":1.141,"5min":1.1439999999999999,"15min":1.131},"min":{"1min":0.73199999999999998,"5min":0.67900000000000005,"15min":0.55600000000000005},"max":{"1min":1.5700000000000001,"5min":1.587,"15min":3.5529999999999999},"last":1.4330000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1819999999999999,"5min":1.1439999999999999,"15min":1.135},"min":{"1min":0.83299999999999996,"5min":0.70099999999999996,"15min":0.68200000000000005},"max":{"1min":1.4470000000000001,"5min":1.7909999999999999,"15min":4.5540000000000003},"last":1.056},{"interface":"front","average":{"1min":1.278,"5min":1.2230000000000001,"15min":1.1870000000000001},"min":{"1min":0.84199999999999997,"5min":0.78900000000000003,"15min":0.60299999999999998},"max":{"1min":1.619,"5min":1.619,"15min":1.885},"last":1.383}]},{"osd":2,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2749999999999999,"15min":1.2210000000000001},"min":{"1min":0.85399999999999998,"5min":0.79200000000000004,"15min":0.626},"max":{"1min":1.645,"5min":1.6479999999999999,"15min":1.821},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.2270000000000001,"5min":1.2370000000000001,"15min":1.214},"min":{"1min":0.85699999999999998,"5min":0.62,"15min":0.60899999999999999},"max":{"1min":1.607,"5min":1.7669999999999999,"15min":3.7909999999999999},"last":0.98899999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.254,"5min":1.2989999999999999,"15min":1.2629999999999999},"min":{"1min":0.96299999999999997,"5min":0.71799999999999997,"15min":0.67800000000000005},"max":{"1min":1.776,"5min":1.804,"15min":3.8820000000000001},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.3300000000000001,"5min":1.2849999999999999,"15min":1.2170000000000001},"min":{"1min":1.054,"5min":0.71199999999999997,"15min":0.69899999999999995},"max":{"1min":1.738,"5min":1.738,"15min":4.6189999999999998},"last":1.3080000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.232,"5min":1.302,"15min":1.282},"min":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.65600000000000003},"max":{"1min":1.6639999999999999,"5min":1.7529999999999999,"15min":3.0680000000000001},"last":1.502},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.383,"15min":1.395},"min":{"1min":0.86499999999999999,"5min":0.59099999999999997,"15min":0.57999999999999996},"max":{"1min":1.728,"5min":2.1040000000000001,"15min":3.7000000000000002},"last":1.5509999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2270000000000001,"5min":1.2809999999999999,"15min":1.254},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.65500000000000003},"max":{"1min":1.5389999999999999,"5min":1.9510000000000001,"15min":4.6680000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":1.325,"5min":1.403,"15min":1.3979999999999999},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.626},"max":{"1min":1.7330000000000001,"5min":1.794,"15min":4.1159999999999997},"last":1.2749999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.4359999999999999,"15min":1.409},"min":{"1min":0.98899999999999999,"5min":0.65700000000000003,"15min":0.61899999999999999},"max":{"1min":1.7110000000000001,"5min":2.02,"15min":3.9670000000000001},"last":1.3400000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.4379999999999999,"15min":1.4159999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.752,"5min":2.2149999999999999,"15min":5.6639999999999997},"last":1.1910000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575029,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1742,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193569,"statfs":{"total":200000000,"available":198215229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.1890000000000001,"5min":1.284,"15min":1.218},"min":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.61899999999999999},"max":{"1min":1.6459999999999999,"5min":2.089,"15min":2.089},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2949999999999999,"15min":1.256},"min":{"1min":0.71699999999999997,"5min":0.69399999999999995,"15min":0.61099999999999999},"max":{"1min":1.5329999999999999,"5min":1.641,"15min":1.913},"last":1.5269999999999999}]},{"osd":1,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.149,"15min":1.145},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.65700000000000003},"max":{"1min":1.532,"5min":1.851,"15min":1.9490000000000001},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.171,"5min":1.198,"15min":1.1519999999999999},"min":{"1min":0.77000000000000002,"5min":0.71399999999999997,"15min":0.505},"max":{"1min":1.6459999999999999,"5min":1.899,"15min":1.899},"last":1.198}]},{"osd":2,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.2050000000000001,"15min":1.157},"min":{"1min":0.77600000000000002,"5min":0.73999999999999999,"15min":0.59499999999999997},"max":{"1min":1.7030000000000001,"5min":1.806,"15min":1.806},"last":1.3340000000000001},{"interface":"front","average":{"1min":1.208,"5min":1.268,"15min":1.2310000000000001},"min":{"1min":0.82999999999999996,"5min":0.74099999999999999,"15min":0.55400000000000005},"max":{"1min":1.857,"5min":1.857,"15min":2.036},"last":1.401}]},{"osd":3,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2,"5min":1.2030000000000001,"15min":1.171},"min":{"1min":0.82299999999999995,"5min":0.76600000000000001,"15min":0.68899999999999995},"max":{"1min":1.5069999999999999,"5min":1.589,"15min":1.7450000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.373,"5min":1.371,"15min":1.3220000000000001},"min":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.58799999999999997},"max":{"1min":1.7030000000000001,"5min":2.0369999999999999,"15min":2.0590000000000002},"last":1.4950000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.2250000000000001,"5min":1.2589999999999999,"15min":1.2709999999999999},"min":{"1min":0.70199999999999996,"5min":0.52300000000000002,"15min":0.46700000000000003},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":2.0329999999999999},"last":1.2869999999999999},{"interface":"front","average":{"1min":1.2,"5min":1.141,"15min":1.167},"min":{"1min":0.76000000000000001,"5min":0.64500000000000002,"15min":0.57099999999999995},"max":{"1min":1.6870000000000001,"5min":1.8839999999999999,"15min":1.901},"last":1.371}]},{"osd":6,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.3080000000000001,"5min":1.401,"15min":1.4139999999999999},"min":{"1min":0.64100000000000001,"5min":0.56699999999999995,"15min":0.52400000000000002},"max":{"1min":1.927,"5min":2.1579999999999999,"15min":2.1589999999999998},"last":1.6479999999999999},{"interface":"front","average":{"1min":1.4379999999999999,"5min":1.3160000000000001,"15min":1.3049999999999999},"min":{"1min":0.94699999999999995,"5min":0.58299999999999996,"15min":0.49299999999999999},"max":{"1min":2.9039999999999999,"5min":2.9039999999999999,"15min":2.9039999999999999},"last":1.252}]},{"osd":7,"last update":"Tue Dec 19 18:29:12 2023","interfaces":[{"interface":"back","average":{"1min":1.329,"5min":1.284,"15min":1.282},"min":{"1min":0.60299999999999998,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.752,"5min":2.036,"15min":2.036},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.4690000000000001,"5min":1.444,"15min":1.448},"min":{"1min":0.89300000000000002,"5min":0.67100000000000004,"15min":0.623},"max":{"1min":1.823,"5min":1.8540000000000001,"15min":2.117},"last":1.444}]}]},{"osd":4,"up_from":13,"seq":55834575029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":1.0609999999999999,"15min":1.028},"min":{"1min":0.68700000000000006,"5min":0.64700000000000002,"15min":0.52700000000000002},"max":{"1min":1.4530000000000001,"5min":1.494,"15min":1.972},"last":1.032},{"interface":"front","average":{"1min":1.0629999999999999,"5min":1.097,"15min":1.0760000000000001},"min":{"1min":0.61399999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.611,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.145}]},{"osd":1,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1859999999999999,"5min":1.208,"15min":1.171},"min":{"1min":0.65900000000000003,"5min":0.621,"15min":0.621},"max":{"1min":1.8779999999999999,"5min":1.8779999999999999,"15min":1.8779999999999999},"last":1.298},{"interface":"front","average":{"1min":1.3129999999999999,"5min":1.3029999999999999,"15min":1.2470000000000001},"min":{"1min":0.92400000000000004,"5min":0.64700000000000002,"15min":0.61699999999999999},"max":{"1min":1.6819999999999999,"5min":1.6819999999999999,"15min":1.875},"last":1.2070000000000001}]},{"osd":2,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.1879999999999999,"5min":1.1659999999999999,"15min":1.139},"min":{"1min":0.70299999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.2629999999999999},{"interface":"front","average":{"1min":1.087,"5min":1.1679999999999999,"15min":1.147},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.70499999999999996},"max":{"1min":1.5029999999999999,"5min":1.8149999999999999,"15min":2.0270000000000001},"last":1.238}]},{"osd":3,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.258,"15min":1.274},"min":{"1min":0.96499999999999997,"5min":0.754,"15min":0.754},"max":{"1min":1.601,"5min":1.6719999999999999,"15min":1.946},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.3680000000000001,"5min":1.357,"15min":1.341},"min":{"1min":0.89000000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"max":{"1min":1.8819999999999999,"5min":1.8819999999999999,"15min":1.927},"last":1.397}]},{"osd":5,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.2110000000000001,"5min":1.2490000000000001,"15min":1.252},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.44500000000000001},"max":{"1min":1.542,"5min":1.968,"15min":1.968},"last":1.091},{"interface":"front","average":{"1min":1.4450000000000001,"5min":1.3740000000000001,"15min":1.3779999999999999},"min":{"1min":0.77500000000000002,"5min":0.59099999999999997,"15min":0.504},"max":{"1min":1.756,"5min":1.9359999999999999,"15min":1.9359999999999999},"last":0.75800000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.341,"5min":1.3049999999999999,"15min":1.3069999999999999},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.432},"max":{"1min":1.851,"5min":2.0329999999999999,"15min":2.0329999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.49,"5min":1.4419999999999999,"15min":1.4299999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":2.0339999999999998,"5min":2.149,"15min":2.149},"last":1.3600000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:29:10 2023","interfaces":[{"interface":"back","average":{"1min":1.4910000000000001,"5min":1.3859999999999999,"15min":1.3740000000000001},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.56399999999999995},"max":{"1min":2.0720000000000001,"5min":2.702,"15min":2.702},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.4990000000000001,"5min":1.4330000000000001,"15min":1.4279999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.9930000000000001,"5min":2.4199999999999999,"15min":2.4199999999999999},"last":1.3180000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575030,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.90900000000000003,"15min":0.92200000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.35299999999999998},"max":{"1min":1.754,"5min":1.754,"15min":3.4350000000000001},"last":1.0920000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":1.038,"15min":0.99299999999999999},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.39500000000000002},"max":{"1min":1.591,"5min":1.8109999999999999,"15min":4.3490000000000002},"last":0.84999999999999998}]},{"osd":1,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":0.96499999999999997,"5min":1.01,"15min":1.024},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.41299999999999998},"max":{"1min":1.8520000000000001,"5min":1.9690000000000001,"15min":5.2510000000000003},"last":1.4219999999999999},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0940000000000001,"15min":1.1040000000000001},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.4299999999999999,"5min":1.643,"15min":5.4119999999999999},"last":0.96999999999999997}]},{"osd":2,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.1299999999999999,"5min":1.1899999999999999,"15min":1.236},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.46000000000000002},"max":{"1min":2.1059999999999999,"5min":2.4649999999999999,"15min":13.833},"last":1.028},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.304,"15min":1.3520000000000001},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.8460000000000001,"5min":2.379,"15min":16.748999999999999},"last":1.389}]},{"osd":4,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.224,"5min":1.3080000000000001,"15min":1.29},"min":{"1min":0.81299999999999994,"5min":0.77900000000000003,"15min":0.65600000000000003},"max":{"1min":1.75,"5min":2.0150000000000001,"15min":5.1790000000000003},"last":1.615},{"interface":"front","average":{"1min":1.343,"5min":1.367,"15min":1.3919999999999999},"min":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.73199999999999998},"max":{"1min":1.929,"5min":2.0819999999999999,"15min":4.569},"last":1.2829999999999999}]},{"osd":5,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.452,"5min":1.4339999999999999,"15min":1.429},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.76000000000000001},"max":{"1min":1.925,"5min":2.0449999999999999,"15min":5.3289999999999997},"last":1.462},{"interface":"front","average":{"1min":1.327,"5min":1.3859999999999999,"15min":1.347},"min":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.69999999999999996},"max":{"1min":2.004,"5min":2.1469999999999998,"15min":4.6369999999999996},"last":1.3120000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.4390000000000001,"5min":1.454,"15min":1.4510000000000001},"min":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.81599999999999995},"max":{"1min":1.9790000000000001,"5min":2.2040000000000002,"15min":4.5039999999999996},"last":1.4910000000000001},{"interface":"front","average":{"1min":1.516,"5min":1.5189999999999999,"15min":1.5089999999999999},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.80500000000000005},"max":{"1min":2.254,"5min":2.254,"15min":5.3739999999999997},"last":1.579}]},{"osd":7,"last update":"Tue Dec 19 18:29:03 2023","interfaces":[{"interface":"back","average":{"1min":1.508,"5min":1.4990000000000001,"15min":1.454},"min":{"1min":0.98899999999999999,"5min":0.89700000000000002,"15min":0.73199999999999998},"max":{"1min":2.1389999999999998,"5min":2.2869999999999999,"15min":5.1349999999999998},"last":1.7130000000000001},{"interface":"front","average":{"1min":1.4419999999999999,"5min":1.5369999999999999,"15min":1.5489999999999999},"min":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"max":{"1min":2.0550000000000002,"5min":2.351,"15min":4.4619999999999997},"last":1.546}]}]},{"osd":2,"up_from":13,"seq":55834575029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.91500000000000004,"5min":0.92600000000000005,"15min":0.97499999999999998},"min":{"1min":0.48099999999999998,"5min":0.43099999999999999,"15min":0.34399999999999997},"max":{"1min":1.5309999999999999,"5min":2.2799999999999998,"15min":9.2550000000000008},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.96899999999999997,"5min":1.04,"15min":1.0189999999999999},"min":{"1min":0.45900000000000002,"5min":0.38400000000000001,"15min":0.33900000000000002},"max":{"1min":1.4399999999999999,"5min":1.8049999999999999,"15min":4.7850000000000001},"last":1.3160000000000001}]},{"osd":1,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":0.878,"5min":0.93999999999999995,"15min":0.98999999999999999},"min":{"1min":0.46400000000000002,"5min":0.439,"15min":0.439},"max":{"1min":1.4039999999999999,"5min":2.0169999999999999,"15min":8.2720000000000002},"last":0.877},{"interface":"front","average":{"1min":0.96399999999999997,"5min":1.101,"15min":1.1040000000000001},"min":{"1min":0.61199999999999999,"5min":0.54200000000000004,"15min":0.46600000000000003},"max":{"1min":1.339,"5min":1.8460000000000001,"15min":5.0620000000000003},"last":1.5009999999999999}]},{"osd":3,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.1120000000000001,"5min":1.2669999999999999,"15min":1.226},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.5940000000000001,"5min":2.8159999999999998,"15min":8.3109999999999999},"last":1.0900000000000001},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1160000000000001,"15min":1.163},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.41799999999999998},"max":{"1min":1.7869999999999999,"5min":2.6240000000000001,"15min":8.0690000000000008},"last":1.2430000000000001}]},{"osd":4,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3899999999999999,"5min":1.3819999999999999,"15min":1.395},"min":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"max":{"1min":1.9850000000000001,"5min":2.0059999999999998,"15min":5.3929999999999998},"last":1.409},{"interface":"front","average":{"1min":1.399,"5min":1.458,"15min":1.46},"min":{"1min":1.0209999999999999,"5min":0.85799999999999998,"15min":0.78300000000000003},"max":{"1min":2.3079999999999998,"5min":2.4300000000000002,"15min":7.3570000000000002},"last":1.736}]},{"osd":5,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.325,"5min":1.3720000000000001,"15min":1.4059999999999999},"min":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"max":{"1min":1.613,"5min":2.1680000000000001,"15min":8.1839999999999993},"last":1.4530000000000001},{"interface":"front","average":{"1min":1.415,"5min":1.385,"15min":1.391},"min":{"1min":0.90000000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":2.2519999999999998,"5min":2.2519999999999998,"15min":5.165},"last":1.603}]},{"osd":6,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.3959999999999999,"5min":1.413,"15min":1.4219999999999999},"min":{"1min":0.99399999999999999,"5min":0.70699999999999996,"15min":0.70699999999999996},"max":{"1min":1.96,"5min":2.4630000000000001,"15min":5.1059999999999999},"last":1.6559999999999999},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.468,"15min":1.48},"min":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77000000000000002},"max":{"1min":1.7709999999999999,"5min":2.7559999999999998,"15min":8.1370000000000005},"last":1.5620000000000001}]},{"osd":7,"last update":"Tue Dec 19 18:29:07 2023","interfaces":[{"interface":"back","average":{"1min":1.506,"5min":1.504,"15min":1.5089999999999999},"min":{"1min":0.91800000000000004,"5min":0.80000000000000004,"15min":0.78200000000000003},"max":{"1min":2.0870000000000002,"5min":2.3959999999999999,"15min":5.1449999999999996},"last":1.7929999999999999},{"interface":"front","average":{"1min":1.4950000000000001,"5min":1.5629999999999999,"15min":1.5680000000000001},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86499999999999999},"max":{"1min":1.8720000000000001,"5min":2.6320000000000001,"15min":8.2300000000000004},"last":1.696}]}]},{"osd":1,"up_from":13,"seq":55834575030,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1742,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":193569,"statfs":{"total":200000000,"available":198215229,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.93200000000000005,"15min":0.874},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.34699999999999998},"max":{"1min":1.472,"5min":2.262,"15min":2.262},"last":0.91100000000000003},{"interface":"front","average":{"1min":0.93799999999999994,"5min":1.014,"15min":0.96299999999999997},"min":{"1min":0.60599999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.4970000000000001,"5min":2.5939999999999999,"15min":2.5939999999999999},"last":1.1419999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1,"5min":1.02,"15min":1.0309999999999999},"min":{"1min":0.496,"5min":0.496,"15min":0.41399999999999998},"max":{"1min":1.621,"5min":1.621,"15min":2.5739999999999998},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":1.0069999999999999,"15min":0.97799999999999998},"min":{"1min":0.51700000000000002,"5min":0.47399999999999998,"15min":0.38500000000000001},"max":{"1min":1.276,"5min":1.718,"15min":1.845},"last":0.96599999999999997}]},{"osd":3,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.141,"5min":1.3300000000000001,"15min":1.2569999999999999},"min":{"1min":0.624,"5min":0.53000000000000003,"15min":0.48599999999999999},"max":{"1min":1.603,"5min":2.089,"15min":2.137},"last":1.468},{"interface":"front","average":{"1min":1.099,"5min":1.2350000000000001,"15min":1.198},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.44900000000000001},"max":{"1min":1.6659999999999999,"5min":2.0720000000000001,"15min":2.3959999999999999},"last":1.52}]},{"osd":4,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.3140000000000001,"15min":1.3120000000000001},"min":{"1min":0.83099999999999996,"5min":0.80500000000000005,"15min":0.69099999999999995},"max":{"1min":1.694,"5min":2.1859999999999999,"15min":2.444},"last":1.397},{"interface":"front","average":{"1min":1.3109999999999999,"5min":1.381,"15min":1.3520000000000001},"min":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.71099999999999997},"max":{"1min":1.665,"5min":1.8859999999999999,"15min":2.6440000000000001},"last":1.3700000000000001}]},{"osd":5,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3520000000000001,"5min":1.425,"15min":1.371},"min":{"1min":0.753,"5min":0.753,"15min":0.753},"max":{"1min":1.9990000000000001,"5min":1.9990000000000001,"15min":2.5459999999999998},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.339,"5min":1.431,"15min":1.4330000000000001},"min":{"1min":1.0249999999999999,"5min":0.85899999999999999,"15min":0.70499999999999996},"max":{"1min":1.8240000000000001,"5min":1.9339999999999999,"15min":2.6179999999999999},"last":1.6659999999999999}]},{"osd":6,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.3999999999999999,"5min":1.466,"15min":1.4139999999999999},"min":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72299999999999998},"max":{"1min":2.1619999999999999,"5min":2.274,"15min":2.7000000000000002},"last":1.429},{"interface":"front","average":{"1min":1.379,"5min":1.5,"15min":1.478},"min":{"1min":0.872,"5min":0.872,"15min":0.76400000000000001},"max":{"1min":2.153,"5min":2.153,"15min":2.895},"last":1.7589999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:29:05 2023","interfaces":[{"interface":"back","average":{"1min":1.381,"5min":1.4490000000000001,"15min":1.4470000000000001},"min":{"1min":0.92600000000000005,"5min":0.83199999999999996,"15min":0.69099999999999995},"max":{"1min":2.044,"5min":2.044,"15min":9.4830000000000005},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.4930000000000001,"5min":1.5600000000000001,"15min":1.546},"min":{"1min":0.93899999999999995,"5min":0.80800000000000005,"15min":0.71899999999999997},"max":{"1min":2.198,"5min":2.3420000000000001,"15min":9.4109999999999996},"last":1.72}]}]},{"osd":0,"up_from":13,"seq":55834575029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":195312,"kb_used":1166,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":194146,"statfs":{"total":200000000,"available":198805597,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Dec 19 18:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.92800000000000005,"15min":0.92200000000000004},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.311},"max":{"1min":1.3140000000000001,"5min":1.671,"15min":2.3199999999999998},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.93999999999999995,"5min":1.0980000000000001,"15min":1.069},"min":{"1min":0.54500000000000004,"5min":0.504,"15min":0.42799999999999999},"max":{"1min":1.286,"5min":2.2170000000000001,"15min":2.4180000000000001},"last":0.98499999999999999}]},{"osd":2,"last update":"Tue Dec 19 18:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":1.024,"15min":0.99199999999999999},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.633,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.4159999999999999},{"interface":"front","average":{"1min":0.91900000000000004,"5min":1.05,"15min":0.997},"min":{"1min":0.66800000000000004,"5min":0.42699999999999999,"15min":0.41099999999999998},"max":{"1min":1.4139999999999999,"5min":2.3340000000000001,"15min":2.7810000000000001},"last":1.054}]},{"osd":3,"last update":"Tue Dec 19 18:29:14 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":1.048,"15min":1.024},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.38900000000000001},"max":{"1min":1.6819999999999999,"5min":1.8200000000000001,"15min":2.5190000000000001},"last":1.5229999999999999},{"interface":"front","average":{"1min":1.1439999999999999,"5min":1.3480000000000001,"15min":1.274},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.47499999999999998},"max":{"1min":1.724,"5min":2.3620000000000001,"15min":2.3620000000000001},"last":1.833}]},{"osd":4,"last update":"Tue Dec 19 18:29:14 2023","interfaces":[{"interface":"back","average":{"1min":1.276,"5min":1.3380000000000001,"15min":1.284},"min":{"1min":0.90800000000000003,"5min":0.84699999999999998,"15min":0.63200000000000001},"max":{"1min":1.6479999999999999,"5min":2.1920000000000002,"15min":2.5529999999999999},"last":1.117},{"interface":"front","average":{"1min":1.1020000000000001,"5min":1.252,"15min":1.268},"min":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.67600000000000005},"max":{"1min":1.452,"5min":1.851,"15min":2.9529999999999998},"last":1.302}]},{"osd":5,"last update":"Tue Dec 19 18:29:14 2023","interfaces":[{"interface":"back","average":{"1min":1.282,"5min":1.337,"15min":1.3879999999999999},"min":{"1min":0.88900000000000001,"5min":0.70699999999999996,"15min":0.64500000000000002},"max":{"1min":1.6719999999999999,"5min":1.97,"15min":16.298999999999999},"last":1.6200000000000001},{"interface":"front","average":{"1min":1.264,"5min":1.3939999999999999,"15min":1.3380000000000001},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.7370000000000001,"5min":2.2909999999999999,"15min":2.2909999999999999},"last":1.5820000000000001}]},{"osd":6,"last update":"Tue Dec 19 18:29:14 2023","interfaces":[{"interface":"back","average":{"1min":1.351,"5min":1.494,"15min":1.446},"min":{"1min":0.89100000000000001,"5min":0.79800000000000004,"15min":0.71699999999999997},"max":{"1min":1.8839999999999999,"5min":2.4169999999999998,"15min":2.5910000000000002},"last":1.724},{"interface":"front","average":{"1min":1.3899999999999999,"5min":1.4770000000000001,"15min":1.4239999999999999},"min":{"1min":1.109,"5min":0.745,"15min":0.745},"max":{"1min":1.6739999999999999,"5min":2.1709999999999998,"15min":2.46},"last":1.5029999999999999}]},{"osd":7,"last update":"Tue Dec 19 18:29:14 2023","interfaces":[{"interface":"back","average":{"1min":1.407,"5min":1.4810000000000001,"15min":1.4590000000000001},"min":{"1min":0.92300000000000004,"5min":0.81899999999999995,"15min":0.70899999999999996},"max":{"1min":1.9630000000000001,"5min":2.2549999999999999,"15min":2.4969999999999999},"last":1.3700000000000001},{"interface":"front","average":{"1min":1.3360000000000001,"5min":1.4139999999999999,"15min":1.409},"min":{"1min":0.98599999999999999,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.8500000000000001,"5min":1.913,"15min":3.1429999999999998},"last":1.6839999999999999}]}]}],"pool_statfs":[]}} 2023-12-19T18:29:40.128 DEBUG:teuthology.orchestra.run.smithi027:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-12-19T18:29:40.347 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.348+0000 7f9df92ab640 1 Processor -- start 2023-12-19T18:29:40.348 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.348+0000 7f9df92ab640 1 -- start start 2023-12-19T18:29:40.349 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.349+0000 7f9df92ab640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9df40b8c30 0x7f9df40d1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.349+0000 7f9df92ab640 1 --2- >> v2:172.21.15.192:3300/0 conn(0x7f9df40cf9b0 0x7f9df40cfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.350+0000 7f9df92ab640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9df40ce540 con 0x7f9df40cda20 2023-12-19T18:29:40.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.350+0000 7f9df92ab640 1 -- --> v2:172.21.15.27:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df40cea80 con 0x7f9df40b8c30 2023-12-19T18:29:40.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.350+0000 7f9df92ab640 1 -- --> v2:172.21.15.192:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9df40b5f10 con 0x7f9df40cf9b0 2023-12-19T18:29:40.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.350+0000 7f9df2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9df40b8c30 0x7f9df40d1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:40.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.350+0000 7f9df2ffd640 1 --2- >> v2:172.21.15.27:3300/0 conn(0x7f9df40b8c30 0x7f9df40d1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59680/0 (socket says 172.21.15.27:59680) 2023-12-19T18:29:40.350 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.350+0000 7f9df2ffd640 1 -- 172.21.15.27:0/1825070375 learned_addr learned my addr 172.21.15.27:0/1825070375 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:40.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.350+0000 7f9df2ffd640 1 -- 172.21.15.27:0/1825070375 >> v2:172.21.15.192:3300/0 conn(0x7f9df40cf9b0 msgr2=0x7f9df40cfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:40.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.350+0000 7f9debfff640 1 --2- 172.21.15.27:0/1825070375 >> v2:172.21.15.192:3300/0 conn(0x7f9df40cf9b0 0x7f9df40cfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:40.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.350+0000 7f9df2ffd640 1 --2- 172.21.15.27:0/1825070375 >> v2:172.21.15.192:3300/0 conn(0x7f9df40cf9b0 0x7f9df40cfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.350+0000 7f9df2ffd640 1 -- 172.21.15.27:0/1825070375 >> v1:172.21.15.192:6789/0 conn(0x7f9df40cda20 legacy=0x7f9df40cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:40.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.350+0000 7f9df2ffd640 1 -- 172.21.15.27:0/1825070375 --> v2:172.21.15.27:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df40ce7a0 con 0x7f9df40b8c30 2023-12-19T18:29:40.351 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.350+0000 7f9debfff640 1 --2- 172.21.15.27:0/1825070375 >> v2:172.21.15.192:3300/0 conn(0x7f9df40cf9b0 0x7f9df40cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:40.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.352+0000 7f9df2ffd640 1 --2- 172.21.15.27:0/1825070375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9df40b8c30 0x7f9df40d1fc0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f9df40ca000 tx=0x7f9de402b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31e28450d53068e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:40.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.352+0000 7f9df27fc640 1 -- 172.21.15.27:0/1825070375 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de40317e0 con 0x7f9df40b8c30 2023-12-19T18:29:40.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.352+0000 7f9df27fc640 1 -- 172.21.15.27:0/1825070375 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de4031940 con 0x7f9df40b8c30 2023-12-19T18:29:40.352 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.352+0000 7f9df27fc640 1 -- 172.21.15.27:0/1825070375 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de4031cd0 con 0x7f9df40b8c30 2023-12-19T18:29:40.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.353+0000 7f9df92ab640 1 -- 172.21.15.27:0/1825070375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9df40b8c30 msgr2=0x7f9df40d1fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:40.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.353+0000 7f9df92ab640 1 --2- 172.21.15.27:0/1825070375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9df40b8c30 0x7f9df40d1fc0 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7f9df40ca000 tx=0x7f9de402b880 comp rx=0 tx=0).stop 2023-12-19T18:29:40.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.353+0000 7f9df92ab640 1 -- 172.21.15.27:0/1825070375 shutdown_connections 2023-12-19T18:29:40.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.353+0000 7f9df92ab640 1 --2- 172.21.15.27:0/1825070375 >> v2:172.21.15.192:3300/0 conn(0x7f9df40cf9b0 0x7f9df40cfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.353+0000 7f9df92ab640 1 --2- 172.21.15.27:0/1825070375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9df40b8c30 0x7f9df40d1fc0 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.353+0000 7f9df92ab640 1 -- 172.21.15.27:0/1825070375 >> 172.21.15.27:0/1825070375 conn(0x7f9df40d2fa0 msgr2=0x7f9df40d3380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:40.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.353+0000 7f9df92ab640 1 -- 172.21.15.27:0/1825070375 shutdown_connections 2023-12-19T18:29:40.353 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.353+0000 7f9df92ab640 1 -- 172.21.15.27:0/1825070375 wait complete. 2023-12-19T18:29:40.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.355+0000 7f9df92ab640 1 Processor -- start 2023-12-19T18:29:40.354 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.355+0000 7f9df92ab640 1 -- start start 2023-12-19T18:29:40.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.356+0000 7f9df92ab640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9df40b8c30 0x7f9df40b1270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.356 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.356+0000 7f9df92ab640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9df40cda20 0x7f9df41f2920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.356+0000 7f9df92ab640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9df40cf9b0 0x7f9df40b32d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.356+0000 7f9df92ab640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df40cb3f0 con 0x7f9df40b8c30 2023-12-19T18:29:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.356+0000 7f9df92ab640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df40b17b0 con 0x7f9df40cda20 2023-12-19T18:29:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.356+0000 7f9df92ab640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df415c830 con 0x7f9df40cf9b0 2023-12-19T18:29:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.356+0000 7f9df2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9df40b8c30 0x7f9df40b1270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.356+0000 7f9df2ffd640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9df40b8c30 0x7f9df40b1270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59686/0 (socket says 172.21.15.27:59686) 2023-12-19T18:29:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.356+0000 7f9df2ffd640 1 -- 172.21.15.27:0/2570222647 learned_addr learned my addr 172.21.15.27:0/2570222647 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.357+0000 7f9df2ffd640 1 -- 172.21.15.27:0/2570222647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9df40cf9b0 msgr2=0x7f9df40b32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.357+0000 7f9df2ffd640 1 --2- 172.21.15.27:0/2570222647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9df40cf9b0 0x7f9df40b32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.357+0000 7f9df2ffd640 1 -- 172.21.15.27:0/2570222647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9df40cda20 msgr2=0x7f9df41f2920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.357+0000 7f9df2ffd640 1 --2- 172.21.15.27:0/2570222647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9df40cda20 0x7f9df41f2920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.357 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.357+0000 7f9df2ffd640 1 -- 172.21.15.27:0/2570222647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9de40094d0 con 0x7f9df40b8c30 2023-12-19T18:29:40.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.357+0000 7f9df2ffd640 1 --2- 172.21.15.27:0/2570222647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9df40b8c30 0x7f9df40b1270 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f9de402b8c0 tx=0x7f9de40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:40.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.357+0000 7f9deb7fe640 1 -- 172.21.15.27:0/2570222647 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de40329c0 con 0x7f9df40b8c30 2023-12-19T18:29:40.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.357+0000 7f9deb7fe640 1 -- 172.21.15.27:0/2570222647 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9de4032b40 con 0x7f9df40b8c30 2023-12-19T18:29:40.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.357+0000 7f9deb7fe640 1 -- 172.21.15.27:0/2570222647 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de40453f0 con 0x7f9df40b8c30 2023-12-19T18:29:40.358 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.357+0000 7f9df92ab640 1 -- 172.21.15.27:0/2570222647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df415cb10 con 0x7f9df40b8c30 2023-12-19T18:29:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.357+0000 7f9df92ab640 1 -- 172.21.15.27:0/2570222647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df415a160 con 0x7f9df40b8c30 2023-12-19T18:29:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.359+0000 7f9df92ab640 1 -- 172.21.15.27:0/2570222647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9df415a380 con 0x7f9df40b8c30 2023-12-19T18:29:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.360+0000 7f9deb7fe640 1 -- 172.21.15.27:0/2570222647 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7f9de404d3f0 con 0x7f9df40b8c30 2023-12-19T18:29:40.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.360+0000 7f9deb7fe640 1 --2- 172.21.15.27:0/2570222647 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9dd406a430 0x7f9dd406c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.360 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.361+0000 7f9deb7fe640 1 -- 172.21.15.27:0/2570222647 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7f9de40aa530 con 0x7f9df40b8c30 2023-12-19T18:29:40.361 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.361+0000 7f9debfff640 1 --2- 172.21.15.27:0/2570222647 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9dd406a430 0x7f9dd406c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:40.362 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.362+0000 7f9debfff640 1 --2- 172.21.15.27:0/2570222647 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9dd406a430 0x7f9dd406c8f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9df40b2ce0 tx=0x7f9ddc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.365+0000 7f9deb7fe640 1 -- 172.21.15.27:0/2570222647 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7f9de4075ec0 con 0x7f9df40b8c30 2023-12-19T18:29:40.557 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.557+0000 7f9df92ab640 1 -- 172.21.15.27:0/2570222647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f9df4068e90 con 0x7f9df40b8c30 2023-12-19T18:29:40.564 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.564+0000 7f9deb7fe640 1 -- 172.21.15.27:0/2570222647 <== mon.0 v2:172.21.15.27:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f9de4075ec0 con 0x7f9df40b8c30 2023-12-19T18:29:40.565 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.564+0000 7f9deb7fe640 1 -- 172.21.15.27:0/2570222647 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v22)=0 v22) v1 ==== 123+0+0 (secure 0 0 0) 0x7f9de40794e0 con 0x7f9df40b8c30 2023-12-19T18:29:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.567+0000 7f9df92ab640 1 -- 172.21.15.27:0/2570222647 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9dd406a430 msgr2=0x7f9dd406c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.567+0000 7f9df92ab640 1 --2- 172.21.15.27:0/2570222647 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9dd406a430 0x7f9dd406c8f0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f9df40b2ce0 tx=0x7f9ddc008040 comp rx=0 tx=0).stop 2023-12-19T18:29:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.567+0000 7f9df92ab640 1 -- 172.21.15.27:0/2570222647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9df40b8c30 msgr2=0x7f9df40b1270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.567+0000 7f9df92ab640 1 --2- 172.21.15.27:0/2570222647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9df40b8c30 0x7f9df40b1270 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f9de402b8c0 tx=0x7f9de40046c0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.568+0000 7f9df92ab640 1 -- 172.21.15.27:0/2570222647 shutdown_connections 2023-12-19T18:29:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.568+0000 7f9df92ab640 1 --2- 172.21.15.27:0/2570222647 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7f9dd406a430 0x7f9dd406c8f0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.568 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.568+0000 7f9df92ab640 1 --2- 172.21.15.27:0/2570222647 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7f9df40cf9b0 0x7f9df40b32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.568+0000 7f9df92ab640 1 --2- 172.21.15.27:0/2570222647 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7f9df40cda20 0x7f9df41f2920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.568+0000 7f9df92ab640 1 --2- 172.21.15.27:0/2570222647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9df40b8c30 0x7f9df40b1270 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.568+0000 7f9df92ab640 1 -- 172.21.15.27:0/2570222647 >> 172.21.15.27:0/2570222647 conn(0x7f9df40d2fa0 msgr2=0x7f9df41604e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.568+0000 7f9df92ab640 1 -- 172.21.15.27:0/2570222647 shutdown_connections 2023-12-19T18:29:40.569 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.568+0000 7f9df92ab640 1 -- 172.21.15.27:0/2570222647 wait complete. 2023-12-19T18:29:40.588 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-19T18:29:40.588 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-19T18:29:40.818 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.818+0000 7fbbc759e640 1 Processor -- start 2023-12-19T18:29:40.819 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.819+0000 7fbbc759e640 1 -- start start 2023-12-19T18:29:40.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.820+0000 7fbbc759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc81604d0 0x7fbbc80b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.820 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.820+0000 7fbbc759e640 1 -- --> v1:172.21.15.27:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc80b5e90 con 0x7fbbc80d2350 2023-12-19T18:29:40.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.820+0000 7fbbc759e640 1 -- --> v1:172.21.15.192:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc80d1750 con 0x7fbbc80b7310 2023-12-19T18:29:40.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.820+0000 7fbbc759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80d2b30 con 0x7fbbc81604d0 2023-12-19T18:29:40.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.821+0000 7fbbbffff640 1 --1- >> v1:172.21.15.27:6789/0 conn(0x7fbbc80d2350 0x7fbbc80d0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.27:6789/0 says I am v1:172.21.15.27:56944/0 (socket says 172.21.15.27:56944) 2023-12-19T18:29:40.821 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.821+0000 7fbbbffff640 1 -- 172.21.15.27:0/963216252 learned_addr learned my addr 172.21.15.27:0/963216252 (peer_addr_for_me v1:172.21.15.27:0/0) 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.821+0000 7fbbc659c640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc81604d0 0x7fbbc80b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.821+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 <== mon.0 v1:172.21.15.27:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2979938404 0 0) 0x7fbbc80b5e90 con 0x7fbbc80d2350 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.821+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 --> v1:172.21.15.27:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbb4003520 con 0x7fbbc80d2350 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 <== mon.0 v1:172.21.15.27:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 210204606 0 0) 0x7fbbb4003520 con 0x7fbbc80d2350 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc81604d0 msgr2=0x7fbbc80b47e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc5d9b640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc81604d0 0x7fbbc80b47e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 >> v1:172.21.15.192:6789/0 conn(0x7fbbc80b7310 legacy=0x7fbbc80d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 --> v1:172.21.15.27:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc80d2850 con 0x7fbbc80d2350 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 <== mon.0 v1:172.21.15.27:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 1398146169 0 0) 0x7fbbb0001670 con 0x7fbbc80d2350 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 >> v1:172.21.15.27:6789/0 conn(0x7fbbc80d2350 legacy=0x7fbbc80d0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc5d9b640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbb40039a0 0x7fbbb4003dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc5d9b640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbb40047c0 0x7fbbb4004c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc5d9b640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc81604d0 0x7fbbb4005180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.823 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80d2b30 con 0x7fbbb40039a0 2023-12-19T18:29:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb400ba60 con 0x7fbbb40047c0 2023-12-19T18:29:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb400bc20 con 0x7fbbc81604d0 2023-12-19T18:29:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.822+0000 7fbbc659c640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbb40039a0 0x7fbbb4003dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.823+0000 7fbbbffff640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbb40047c0 0x7fbbb4004c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.823+0000 7fbbc659c640 1 -- 172.21.15.27:0/963216252 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc81604d0 msgr2=0x7fbbb4005180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.823+0000 7fbbc6d9d640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc81604d0 0x7fbbb4005180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.823+0000 7fbbc659c640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc81604d0 0x7fbbb4005180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.823+0000 7fbbc659c640 1 -- 172.21.15.27:0/963216252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbb40047c0 msgr2=0x7fbbb4004c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-19T18:29:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.823+0000 7fbbc659c640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbb40047c0 0x7fbbb4004c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.823+0000 7fbbc659c640 1 -- 172.21.15.27:0/963216252 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb4003520 con 0x7fbbb40039a0 2023-12-19T18:29:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.823+0000 7fbbc6d9d640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbc81604d0 0x7fbbb4005180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:40.824 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.823+0000 7fbbbffff640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbb40047c0 0x7fbbb4004c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-19T18:29:40.825 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.824+0000 7fbbc659c640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbb40039a0 0x7fbbb4003dc0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009a90 tx=0x7fbbb40057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=309a37a3770d74ea server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.824+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb802c660 con 0x7fbbb40039a0 2023-12-19T18:29:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.824+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbbb802c7c0 con 0x7fbbb40039a0 2023-12-19T18:29:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.824+0000 7fbbc5d9b640 1 -- 172.21.15.27:0/963216252 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb802cb70 con 0x7fbbb40039a0 2023-12-19T18:29:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.825+0000 7fbbc759e640 1 -- 172.21.15.27:0/963216252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbb40039a0 msgr2=0x7fbbb4003dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.825+0000 7fbbc759e640 1 --2- 172.21.15.27:0/963216252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbb40039a0 0x7fbbb4003dc0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009a90 tx=0x7fbbb40057f0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.825+0000 7fbbc6d9d640 1 -- 172.21.15.27:0/963216252 reap_dead start 2023-12-19T18:29:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.825+0000 7fbbc759e640 1 -- 172.21.15.27:0/963216252 shutdown_connections 2023-12-19T18:29:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.825+0000 7fbbc759e640 1 -- 172.21.15.27:0/963216252 >> 172.21.15.27:0/963216252 conn(0x7fbbc80cf7e0 msgr2=0x7fbbc80cfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.825+0000 7fbbc759e640 1 -- 172.21.15.27:0/963216252 shutdown_connections 2023-12-19T18:29:40.826 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.825+0000 7fbbc759e640 1 -- 172.21.15.27:0/963216252 wait complete. 2023-12-19T18:29:40.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.827+0000 7fbbc759e640 1 Processor -- start 2023-12-19T18:29:40.827 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.827+0000 7fbbc759e640 1 -- start start 2023-12-19T18:29:40.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.828+0000 7fbbc759e640 1 --2- >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbb40039a0 0x7fbbc81e5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.828+0000 7fbbc759e640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbc80cad00 0x7fbbc80cb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.828+0000 7fbbc759e640 1 --2- >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbc814ac40 0x7fbbc814b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.828+0000 7fbbc759e640 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc814bc60 con 0x7fbbc80cad00 2023-12-19T18:29:40.828 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.828+0000 7fbbc759e640 1 -- --> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc80cb6c0 con 0x7fbbc814ac40 2023-12-19T18:29:40.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.828+0000 7fbbc759e640 1 -- --> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc81f69f0 con 0x7fbbb40039a0 2023-12-19T18:29:40.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.828+0000 7fbbbffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbc80cad00 0x7fbbc80cb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:40.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.828+0000 7fbbbffff640 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbc80cad00 0x7fbbc80cb180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59704/0 (socket says 172.21.15.27:59704) 2023-12-19T18:29:40.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.828+0000 7fbbbffff640 1 -- 172.21.15.27:0/3988903232 learned_addr learned my addr 172.21.15.27:0/3988903232 (peer_addr_for_me v2:172.21.15.27:0/0) 2023-12-19T18:29:40.829 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.829+0000 7fbbc659c640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbb40039a0 0x7fbbc81e5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.829+0000 7fbbc6d9d640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbc814ac40 0x7fbbc814b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.829+0000 7fbbbffff640 1 -- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbb40039a0 msgr2=0x7fbbc81e5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.829+0000 7fbbbffff640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbb40039a0 0x7fbbc81e5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.829+0000 7fbbbffff640 1 -- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbc814ac40 msgr2=0x7fbbc814b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:40.830 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.829+0000 7fbbbffff640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbc814ac40 0x7fbbc814b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:40.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.829+0000 7fbbbffff640 1 -- 172.21.15.27:0/3988903232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc81f6560 con 0x7fbbc80cad00 2023-12-19T18:29:40.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.829+0000 7fbbbffff640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbc80cad00 0x7fbbc80cb180 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000d840 tx=0x7fbbb000dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:40.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.830+0000 7fbbbeffd640 1 -- 172.21.15.27:0/3988903232 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb00074e0 con 0x7fbbc80cad00 2023-12-19T18:29:40.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.830+0000 7fbbbeffd640 1 -- 172.21.15.27:0/3988903232 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fbbb0007640 con 0x7fbbc80cad00 2023-12-19T18:29:40.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.830+0000 7fbbbeffd640 1 -- 172.21.15.27:0/3988903232 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0004a00 con 0x7fbbc80cad00 2023-12-19T18:29:40.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.830+0000 7fbbc759e640 1 -- 172.21.15.27:0/3988903232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc81e6510 con 0x7fbbc80cad00 2023-12-19T18:29:40.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.830+0000 7fbbc759e640 1 -- 172.21.15.27:0/3988903232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc81f5f10 con 0x7fbbc80cad00 2023-12-19T18:29:40.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.831+0000 7fbbc759e640 1 -- 172.21.15.27:0/3988903232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc8066f80 con 0x7fbbc80cad00 2023-12-19T18:29:40.831 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.832+0000 7fbbbeffd640 1 -- 172.21.15.27:0/3988903232 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 88607+0+0 (secure 0 0 0) 0x7fbbb0004b60 con 0x7fbbc80cad00 2023-12-19T18:29:40.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.833+0000 7fbbbeffd640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbba806a430 0x7fbba806c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-19T18:29:40.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.833+0000 7fbbbeffd640 1 -- 172.21.15.27:0/3988903232 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fbbb0088010 con 0x7fbbc80cad00 2023-12-19T18:29:40.833 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.834+0000 7fbbc659c640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbba806a430 0x7fbba806c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-19T18:29:40.834 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.835+0000 7fbbc659c640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbba806a430 0x7fbba806c8f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8036000 tx=0x7fbbb8030000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-19T18:29:40.837 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:40.837+0000 7fbbbeffd640 1 -- 172.21.15.27:0/3988903232 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+179343 (secure 0 0 0) 0x7fbbb0053870 con 0x7fbbc80cad00 2023-12-19T18:29:41.072 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.072+0000 7fbbc759e640 1 -- 172.21.15.27:0/3988903232 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbc81f66f0 con 0x7fbbc80cad00 2023-12-19T18:29:41.073 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.073+0000 7fbbbeffd640 1 -- 172.21.15.27:0/3988903232 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+3529 (secure 0 0 0) 0x7fbbb0019020 con 0x7fbbc80cad00 2023-12-19T18:29:41.073 INFO:teuthology.orchestra.run.smithi027.stdout: 2023-12-19T18:29:41.073 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":87,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6836/1864159643","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6834","nonce":1864159643},{"type":"v1","addr":"172.21.15.192:6836","nonce":1864159643}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.192:6837/1348219767","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.192:6835","nonce":1348219767},{"type":"v1","addr":"172.21.15.192:6837","nonce":1348219767}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":8994,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6833/181137639","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6832","nonce":181137639},{"type":"v1","addr":"172.21.15.27:6833","nonce":181137639}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":80},{"gid":9405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.27:6835/617672069","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6834","nonce":617672069},{"type":"v1","addr":"172.21.15.27:6835","nonce":617672069}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":87}],"filesystems":[]} 2023-12-19T18:29:41.073 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 87 2023-12-19T18:29:41.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.075+0000 7fbbc759e640 1 -- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbba806a430 msgr2=0x7fbba806c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:41.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.075+0000 7fbbc759e640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbba806a430 0x7fbba806c8f0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8036000 tx=0x7fbbb8030000 comp rx=0 tx=0).stop 2023-12-19T18:29:41.075 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.075+0000 7fbbc759e640 1 -- 172.21.15.27:0/3988903232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbc80cad00 msgr2=0x7fbbc80cb180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-19T18:29:41.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.075+0000 7fbbc759e640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbc80cad00 0x7fbbc80cb180 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000d840 tx=0x7fbbb000dd00 comp rx=0 tx=0).stop 2023-12-19T18:29:41.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.076+0000 7fbbc759e640 1 -- 172.21.15.27:0/3988903232 shutdown_connections 2023-12-19T18:29:41.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.076+0000 7fbbc759e640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:6832/83634,v1:172.21.15.192:6833/83634] conn(0x7fbba806a430 0x7fbba806c8f0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:41.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.076+0000 7fbbc759e640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:3300/0,v1:172.21.15.192:6789/0] conn(0x7fbbc814ac40 0x7fbbc814b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:41.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.076+0000 7fbbc759e640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbc80cad00 0x7fbbc80cb180 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:41.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.076+0000 7fbbc759e640 1 --2- 172.21.15.27:0/3988903232 >> [v2:172.21.15.192:3301/0,v1:172.21.15.192:6790/0] conn(0x7fbbb40039a0 0x7fbbc81e5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-19T18:29:41.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.076+0000 7fbbc759e640 1 -- 172.21.15.27:0/3988903232 >> 172.21.15.27:0/3988903232 conn(0x7fbbc80cf7e0 msgr2=0x7fbbc80cbb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-19T18:29:41.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.076+0000 7fbbc759e640 1 -- 172.21.15.27:0/3988903232 shutdown_connections 2023-12-19T18:29:41.076 INFO:teuthology.orchestra.run.smithi027.stderr:2023-12-19T18:29:41.076+0000 7fbbc759e640 1 -- 172.21.15.27:0/3988903232 wait complete. 2023-12-19T18:29:41.094 DEBUG:tasks.cephfs.filesystem:already dead... 2023-12-19T18:29:41.094 INFO:teuthology.misc:Shutting down mds daemons... 2023-12-19T18:29:41.094 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-19T18:29:41.094 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:41.112 INFO:tasks.ceph.mds.a:Stopped 2023-12-19T18:29:41.113 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-19T18:29:41.113 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:41.185 INFO:tasks.ceph.mds.c:Stopped 2023-12-19T18:29:41.186 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-19T18:29:41.186 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:41.214 INFO:tasks.ceph.mds.b:Stopped 2023-12-19T18:29:41.215 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-19T18:29:41.215 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:41.286 INFO:tasks.ceph.mds.d:Stopped 2023-12-19T18:29:41.287 INFO:teuthology.misc:Shutting down osd daemons... 2023-12-19T18:29:41.287 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-12-19T18:29:41.287 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:41.342 INFO:tasks.ceph.osd.0:Stopped 2023-12-19T18:29:41.342 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-12-19T18:29:41.342 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:41.414 INFO:tasks.ceph.osd.1:Stopped 2023-12-19T18:29:41.414 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-12-19T18:29:41.414 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:41.495 INFO:tasks.ceph.osd.2:Stopped 2023-12-19T18:29:41.496 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-12-19T18:29:41.496 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:41.559 INFO:tasks.ceph.osd.3:Stopped 2023-12-19T18:29:41.559 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-12-19T18:29:41.559 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:41.647 INFO:tasks.ceph.osd.4:Stopped 2023-12-19T18:29:41.648 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-12-19T18:29:41.648 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:41.709 INFO:tasks.ceph.osd.5:Stopped 2023-12-19T18:29:41.710 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-12-19T18:29:41.710 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:41.796 INFO:tasks.ceph.osd.6:Stopped 2023-12-19T18:29:41.797 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-12-19T18:29:41.797 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:41.855 INFO:tasks.ceph.osd.7:Stopped 2023-12-19T18:29:41.855 INFO:teuthology.misc:Shutting down mgr daemons... 2023-12-19T18:29:41.855 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-12-19T18:29:41.856 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:41.951 INFO:tasks.ceph.mgr.y:Stopped 2023-12-19T18:29:41.951 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-12-19T18:29:41.952 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:42.022 INFO:tasks.ceph.mgr.x:Stopped 2023-12-19T18:29:42.022 INFO:teuthology.misc:Shutting down mon daemons... 2023-12-19T18:29:42.023 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-12-19T18:29:42.023 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:42.077 INFO:tasks.ceph.mon.a:Stopped 2023-12-19T18:29:42.077 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-19T18:29:42.077 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:42.102 INFO:tasks.ceph.mon.b:Stopped 2023-12-19T18:29:42.102 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-19T18:29:42.102 INFO:teuthology.orchestra.run:waiting for 300 2023-12-19T18:29:42.180 INFO:tasks.ceph.mon.c:Stopped 2023-12-19T18:29:42.180 INFO:tasks.ceph:Checking cluster log for badness... 2023-12-19T18:29:42.180 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-12-19T18:29:42.302 INFO:teuthology.orchestra.run.smithi027.stdout:2023-12-19T18:14:42.069165+0000 osd.5 (osd.5) 3 : cluster [WRN] OSD bench result of 119970.578415 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.5. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-12-19T18:29:42.303 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-12-19T18:29:42.303 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-12-19T18:29:42.405 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-12-19T18:29:42.501 DEBUG:teuthology.orchestra.run.smithi027:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | egrep -v PG_RECOVERY_FULL | egrep -v PG_DEGRADED | head -n 1 2023-12-19T18:29:42.602 INFO:teuthology.orchestra.run.smithi027.stdout:2023-12-19T18:14:42.069165+0000 osd.5 (osd.5) 3 : cluster [WRN] OSD bench result of 119970.578415 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.5. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd's IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]. 2023-12-19T18:29:42.603 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:29:42.604 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-12-19T18:29:42.880 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:29:42.881 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-12-19T18:29:43.051 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:29:43.051 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-12-19T18:29:43.219 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:29:43.219 DEBUG:teuthology.orchestra.run.smithi027:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-12-19T18:29:43.410 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:29:43.410 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-12-19T18:29:43.693 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:29:43.693 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-12-19T18:29:43.859 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:29:43.860 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-12-19T18:29:44.018 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:29:44.019 DEBUG:teuthology.orchestra.run.smithi192:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-12-19T18:29:44.180 INFO:tasks.ceph:Archiving mon data... 2023-12-19T18:29:44.181 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229/data/mon.a.tgz 2023-12-19T18:29:44.182 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2023-12-19T18:29:44.238 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.BwMWTpwJs9 2023-12-19T18:29:44.238 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.BwMWTpwJs9 2023-12-19T18:29:44.461 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.BwMWTpwJs9 2023-12-19T18:29:44.606 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.BwMWTpwJs9 is 500KB 2023-12-19T18:29:44.710 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.BwMWTpwJs9 2023-12-19T18:29:44.767 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229/data/mon.b.tgz 2023-12-19T18:29:44.767 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2023-12-19T18:29:44.826 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.mVBbKr1NbO 2023-12-19T18:29:44.826 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.mVBbKr1NbO 2023-12-19T18:29:45.056 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.mVBbKr1NbO 2023-12-19T18:29:45.209 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.mVBbKr1NbO is 535KB 2023-12-19T18:29:45.320 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.mVBbKr1NbO 2023-12-19T18:29:45.378 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229/data/mon.c.tgz 2023-12-19T18:29:45.378 DEBUG:teuthology.orchestra.run.smithi192:> mktemp 2023-12-19T18:29:45.430 INFO:teuthology.orchestra.run.smithi192.stdout:/tmp/tmp.ByMsOFIlwc 2023-12-19T18:29:45.431 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ByMsOFIlwc 2023-12-19T18:29:45.642 DEBUG:teuthology.orchestra.run.smithi192:> sudo chmod 0666 /tmp/tmp.ByMsOFIlwc 2023-12-19T18:29:45.785 DEBUG:teuthology.orchestra.remote:smithi192:/tmp/tmp.ByMsOFIlwc is 535KB 2023-12-19T18:29:45.897 DEBUG:teuthology.orchestra.run.smithi192:> rm -fr /tmp/tmp.ByMsOFIlwc 2023-12-19T18:29:45.957 INFO:tasks.ceph:Cleaning ceph cluster... 2023-12-19T18:29:45.957 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-19T18:29:45.967 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-19T18:29:46.075 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-12-19T18:29:46.075 DEBUG:teuthology.orchestra.run.smithi027:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-12-19T18:29:46.082 DEBUG:teuthology.orchestra.run.smithi192:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-12-19T18:29:46.190 INFO:teuthology.orchestra.run.smithi192.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-12-19T18:29:46.194 INFO:teuthology.orchestra.run.smithi027.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-12-19T18:29:46.208 INFO:tasks.ceph:Archiving crash dumps... 2023-12-19T18:29:46.210 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229/remote/smithi027/crash 2023-12-19T18:29:46.211 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-19T18:29:46.306 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229/remote/smithi192/crash 2023-12-19T18:29:46.308 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-19T18:29:46.401 INFO:tasks.ceph:Shutting down logrotate 2023-12-19T18:29:46.401 DEBUG:teuthology.orchestra.run.smithi027:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:29:46.410 DEBUG:teuthology.orchestra.run.smithi192:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-19T18:29:46.561 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-19T18:29:46.679 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-19T18:29:46.782 INFO:tasks.ceph:Compressing logs... 2023-12-19T18:29:46.782 DEBUG:teuthology.orchestra.run.smithi027:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-19T18:29:46.792 DEBUG:teuthology.orchestra.run.smithi192:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-19T18:29:54.009 INFO:tasks.ceph:Archiving logs... 2023-12-19T18:29:54.009 DEBUG:teuthology.misc:Transferring archived files from smithi027:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229/remote/smithi027/log 2023-12-19T18:29:54.010 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-19T18:29:55.431 DEBUG:teuthology.misc:Transferring archived files from smithi192:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229/remote/smithi192/log 2023-12-19T18:29:55.432 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-19T18:29:56.115 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-19T18:29:56.125 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-19T18:29:56.125 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-19T18:29:56.135 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-19T18:29:56.300 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-19T18:29:56.311 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-19T18:29:56.312 DEBUG:teuthology.orchestra.run.smithi027:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-19T18:29:56.321 DEBUG:teuthology.orchestra.run.smithi192:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-19T18:29:56.374 INFO:teuthology.orchestra.run.smithi027.stderr:bash: line 1: ntpq: command not found 2023-12-19T18:29:56.379 INFO:teuthology.orchestra.run.smithi192.stderr:bash: line 1: ntpq: command not found 2023-12-19T18:29:56.384 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-19T18:29:56.385 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2023-12-19T18:29:56.385 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 26 -156us[ -153us] +/- 93ms 2023-12-19T18:29:56.385 INFO:teuthology.orchestra.run.smithi027.stdout:^+ hv02.front.sepia.ceph.com 3 7 377 91 +47us[ +54us] +/- 77ms 2023-12-19T18:29:56.385 INFO:teuthology.orchestra.run.smithi027.stdout:^* hv03.front.sepia.ceph.com 3 6 377 24 +145us[ +148us] +/- 76ms 2023-12-19T18:29:56.385 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-12-19T18:29:56.389 INFO:teuthology.orchestra.run.smithi192.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-19T18:29:56.389 INFO:teuthology.orchestra.run.smithi192.stdout:=============================================================================== 2023-12-19T18:29:56.389 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 27 -169us[ -169us] +/- 93ms 2023-12-19T18:29:56.389 INFO:teuthology.orchestra.run.smithi192.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 25 +48us[ +48us] +/- 78ms 2023-12-19T18:29:56.389 INFO:teuthology.orchestra.run.smithi192.stdout:^* hv03.front.sepia.ceph.com 3 6 377 29 +136us[ +139us] +/- 76ms 2023-12-19T18:29:56.390 INFO:teuthology.orchestra.run.smithi192.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-12-19T18:29:56.391 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-19T18:29:56.404 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-19T18:29:56.404 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-19T18:29:56.416 DEBUG:teuthology.orchestra.run.smithi027:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-19T18:29:56.592 DEBUG:teuthology.orchestra.run.smithi192:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-19T18:29:56.865 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-19T18:29:56.983 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:29:56.983 DEBUG:teuthology.orchestra.run.smithi192:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-19T18:29:57.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:29:57.094 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-19T18:29:57.106 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-19T18:29:57.116 INFO:teuthology.task.internal:Duration was 1712.126407 seconds 2023-12-19T18:29:57.116 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-19T18:29:57.128 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-19T18:29:57.128 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-19T18:29:57.138 DEBUG:teuthology.orchestra.run.smithi192:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-19T18:29:57.316 INFO:teuthology.orchestra.run.smithi192.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-19T18:29:57.331 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-19T18:29:57.732 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-19T18:29:57.732 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi027.front.sepia.ceph.com 2023-12-19T18:29:57.732 DEBUG:teuthology.orchestra.run.smithi027:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-19T18:29:57.807 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi192.front.sepia.ceph.com 2023-12-19T18:29:57.807 DEBUG:teuthology.orchestra.run.smithi192:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-19T18:29:57.885 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-19T18:29:57.885 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-19T18:29:57.891 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-19T18:29:58.003 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-19T18:29:58.003 DEBUG:teuthology.orchestra.run.smithi027:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-19T18:29:58.009 DEBUG:teuthology.orchestra.run.smithi192:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-19T18:29:58.236 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-19T18:29:58.247 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-19T18:29:58.247 DEBUG:teuthology.orchestra.run.smithi027:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-19T18:29:58.362 DEBUG:teuthology.orchestra.run.smithi192:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-19T18:29:58.463 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-19T18:29:58.476 DEBUG:teuthology.orchestra.run.smithi027:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-19T18:29:58.486 DEBUG:teuthology.orchestra.run.smithi192:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-19T18:29:58.601 INFO:teuthology.orchestra.run.smithi192.stdout:kernel.core_pattern = core 2023-12-19T18:29:58.602 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = core 2023-12-19T18:29:58.667 DEBUG:teuthology.orchestra.run.smithi027:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-19T18:29:58.720 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:29:58.721 DEBUG:teuthology.orchestra.run.smithi192:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-19T18:29:58.778 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-19T18:29:58.778 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-19T18:29:58.789 INFO:teuthology.task.internal:Transferring archived files... 2023-12-19T18:29:58.790 DEBUG:teuthology.misc:Transferring archived files from smithi027:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229/remote/smithi027 2023-12-19T18:29:58.790 DEBUG:teuthology.orchestra.run.smithi027:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-19T18:29:58.908 DEBUG:teuthology.misc:Transferring archived files from smithi192:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229/remote/smithi192 2023-12-19T18:29:58.908 DEBUG:teuthology.orchestra.run.smithi192:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-19T18:29:59.019 INFO:teuthology.task.internal:Removing archive directory... 2023-12-19T18:29:59.019 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-19T18:29:59.029 DEBUG:teuthology.orchestra.run.smithi192:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-19T18:29:59.086 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-19T18:29:59.098 INFO:teuthology.task.internal:Not uploading archives. 2023-12-19T18:29:59.099 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-19T18:29:59.112 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-19T18:29:59.112 DEBUG:teuthology.orchestra.run.smithi027:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-19T18:29:59.119 DEBUG:teuthology.orchestra.run.smithi192:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-19T18:29:59.173 INFO:teuthology.orchestra.run.smithi027.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 19 18:29 /home/ubuntu/cephtest 2023-12-19T18:29:59.177 INFO:teuthology.orchestra.run.smithi192.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 19 18:29 /home/ubuntu/cephtest 2023-12-19T18:29:59.180 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-19T18:29:59.192 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-19T18:29:59.222 INFO:teuthology.nuke:Checking targets against current locks 2023-12-19T18:29:59.247 DEBUG:teuthology.nuke:shortname: smithi027 2023-12-19T18:29:59.247 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-19T18:29:59.267 DEBUG:teuthology.nuke:shortname: smithi192 2023-12-19T18:29:59.268 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-19T18:29:59.271 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 17:49:49.206968', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKiwWr0Trl+pDhPf5nNOLJ6oFgL3wEGSWpZ+bLW0N63dQX6maGQvLOlhSSCJ/rdZIk1z6+T2Pv8zhBcahDNw244='} 2023-12-19T18:29:59.292 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi192.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-19_13:22:41-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-testing-default-smithi/7497229', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-19 17:49:49.209057', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLG3+8ZD65WGw+gpZxw4zFCbHyN3UXqVOhaEfVSHFoGNnoGBQCbRrunv60cTWg5kiZC1Dx8YHbcYlR5C6vpK1TI='} 2023-12-19T18:29:59.325 INFO:teuthology.orchestra.console.smithi027:Power off 2023-12-19T18:29:59.325 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T18:29:59.352 DEBUG:teuthology.orchestra.console.smithi027:power off output: Chassis Power Control: Down/Off 2023-12-19T18:29:59.352 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:29:59.371 INFO:teuthology.orchestra.console.smithi192:Power off 2023-12-19T18:29:59.371 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-19T18:29:59.383 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2023-12-19T18:29:59.393 DEBUG:teuthology.orchestra.console.smithi192:power off output: Chassis Power Control: Down/Off 2023-12-19T18:29:59.393 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:29:59.415 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2023-12-19T18:30:03.385 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:30:03.417 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:30:03.495 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is on 2023-12-19T18:30:03.526 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is on 2023-12-19T18:30:07.497 DEBUG:teuthology.orchestra.console.smithi027:pexpect command: ipmitool -H smithi027.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:30:07.527 DEBUG:teuthology.orchestra.console.smithi192:pexpect command: ipmitool -H smithi192.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-19T18:30:07.610 DEBUG:teuthology.orchestra.console.smithi027:check power output: Chassis Power is off 2023-12-19T18:30:07.643 DEBUG:teuthology.orchestra.console.smithi192:check power output: Chassis Power is off 2023-12-19T18:30:07.712 INFO:teuthology.orchestra.console.smithi027:Power off completed 2023-12-19T18:30:07.744 INFO:teuthology.orchestra.console.smithi192:Power off completed 2023-12-19T18:30:07.849 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_latest} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/mds-full} duration: 1712.1264069080353 failure_reason: '"2023-12-19T18:14:42.069165+0000 osd.5 (osd.5) 3 : cluster [WRN] OSD bench result of 119970.578415 IOPS exceeded the threshold limit of 500.000000 IOPS for osd.5. IOPS capacity is unchanged at 315.000000 IOPS. The recommendation is to establish the osd''s IOPS capacity using other benchmark tools (e.g. Fio) and then override osd_mclock_max_capacity_iops_[hdd|ssd]." in cluster log' flavor: default mon.a-kernel-sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c mon.b-kernel-sha1: cd68252d7b4d7802f3cae7f5945b38e9b20dad4c owner: scheduled_vshankar@teuthology success: false 2023-12-19T18:30:07.849 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-19T18:30:07.936 INFO:teuthology.run:FAIL